2023-10-10T20:20:50.183 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-10T20:20:50.184 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --name yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} --archive /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420483 --verbose -- /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420483/orig.config.yaml 2023-10-10T20:20:50.232 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-10T20:20:50.317 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420483 branch: wip-yuri3-testing-2023-10-10-0720-quincy description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} email: yweinste@redhat.com first_in_suite: false job_id: '7420483' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri3-testing-2023-10-10-0720-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 46d1f59b549730ba7f93d940a3d32b94c45a4cf7 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 46d1f59b549730ba7f93d940a3d32b94c45a4cf7 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri3-testing-2023-10-10-0720-quincy sha1: 46d1f59b549730ba7f93d940a3d32b94c45a4cf7 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 2048 sha1: 46d1f59b549730ba7f93d940a3d32b94c45a4cf7 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri3-testing-2023-10-10-0720-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 46d1f59b549730ba7f93d940a3d32b94c45a4cf7 targets: smithi089.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJqEHRxhZBHHfulI3XiAdJH7q11m7AkzDAEdNXI7VmObkwOr+dTXboTv+DOxgrtcmZ2PPEDCKxBrxaTcKUU2tNc= smithi111.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG/JCitYYyitn5+62d+6A84Re3tVWjw2tqhRfcK06JGJAtIASTmZNz8vrxZTRHo19Z0g5EBRLNvJ6vhqtJpUAU8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestVolumes - tasks.cephfs.test_volumes.TestSubvolumeGroups - tasks.cephfs.test_volumes.TestSubvolumes teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-10_18:16:52 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-10T20:20:50.318 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa; will attempt to use it 2023-10-10T20:20:50.319 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks 2023-10-10T20:20:50.319 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-10T20:20:50.321 INFO:teuthology.task.internal:Checking packages... 2023-10-10T20:20:50.345 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '46d1f59b549730ba7f93d940a3d32b94c45a4cf7' 2023-10-10T20:20:50.345 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-10T20:20:50.346 INFO:teuthology.packaging:ref: None 2023-10-10T20:20:50.346 INFO:teuthology.packaging:tag: None 2023-10-10T20:20:50.346 INFO:teuthology.packaging:branch: wip-yuri3-testing-2023-10-10-0720-quincy 2023-10-10T20:20:50.346 INFO:teuthology.packaging:sha1: 46d1f59b549730ba7f93d940a3d32b94c45a4cf7 2023-10-10T20:20:50.347 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri3-testing-2023-10-10-0720-quincy 2023-10-10T20:20:50.721 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1374.g46d1f59b 2023-10-10T20:20:50.722 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-10T20:20:50.731 INFO:teuthology.task.internal:no buildpackages task found 2023-10-10T20:20:50.731 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-10T20:20:50.753 INFO:teuthology.task.internal:Saving configuration 2023-10-10T20:20:50.770 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-10T20:20:50.780 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-10T20:20:50.807 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420483', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-10 20:15:08.213134', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJqEHRxhZBHHfulI3XiAdJH7q11m7AkzDAEdNXI7VmObkwOr+dTXboTv+DOxgrtcmZ2PPEDCKxBrxaTcKUU2tNc='} 2023-10-10T20:20:50.830 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420483', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-10 20:15:08.211320', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG/JCitYYyitn5+62d+6A84Re3tVWjw2tqhRfcK06JGJAtIASTmZNz8vrxZTRHo19Z0g5EBRLNvJ6vhqtJpUAU8='} 2023-10-10T20:20:50.833 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-10T20:20:50.842 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-10T20:20:50.843 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-10T20:20:50.843 INFO:teuthology.run_tasks:Running task console_log... 2023-10-10T20:20:50.922 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fea83ff6b80>, signals=[15]) 2023-10-10T20:20:50.923 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-10T20:20:50.934 INFO:teuthology.task.internal:Opening connections... 2023-10-10T20:20:50.934 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:20:50.936 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:20:51.003 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:20:51.004 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:20:51.072 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-10T20:20:51.081 DEBUG:teuthology.orchestra.run.smithi089:> uname -m 2023-10-10T20:20:51.099 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2023-10-10T20:20:51.100 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2023-10-10T20:20:51.156 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="Red Hat Enterprise Linux" 2023-10-10T20:20:51.157 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="8.4 (Ootpa)" 2023-10-10T20:20:51.157 INFO:teuthology.orchestra.run.smithi089.stdout:ID="rhel" 2023-10-10T20:20:51.157 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="fedora" 2023-10-10T20:20:51.158 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="8.4" 2023-10-10T20:20:51.158 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el8" 2023-10-10T20:20:51.158 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-10T20:20:51.159 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2023-10-10T20:20:51.159 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-10T20:20:51.159 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://www.redhat.com/" 2023-10-10T20:20:51.160 INFO:teuthology.orchestra.run.smithi089.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-10T20:20:51.160 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-10T20:20:51.160 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:20:51.160 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-10T20:20:51.161 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-10T20:20:51.161 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-10T20:20:51.161 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-10T20:20:51.162 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2023-10-10T20:20:51.191 DEBUG:teuthology.orchestra.run.smithi111:> uname -m 2023-10-10T20:20:51.208 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64 2023-10-10T20:20:51.209 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release 2023-10-10T20:20:51.264 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="Red Hat Enterprise Linux" 2023-10-10T20:20:51.265 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="8.4 (Ootpa)" 2023-10-10T20:20:51.265 INFO:teuthology.orchestra.run.smithi111.stdout:ID="rhel" 2023-10-10T20:20:51.266 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="fedora" 2023-10-10T20:20:51.266 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="8.4" 2023-10-10T20:20:51.266 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el8" 2023-10-10T20:20:51.266 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-10T20:20:51.266 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31" 2023-10-10T20:20:51.267 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-10T20:20:51.267 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://www.redhat.com/" 2023-10-10T20:20:51.267 INFO:teuthology.orchestra.run.smithi111.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-10T20:20:51.267 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-10T20:20:51.267 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:20:51.268 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-10T20:20:51.269 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-10T20:20:51.269 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-10T20:20:51.269 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-10T20:20:51.270 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2023-10-10T20:20:51.291 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-10T20:20:51.301 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-10T20:20:51.314 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-10T20:20:51.314 DEBUG:teuthology.orchestra.run.smithi089:> test '!' -e /home/ubuntu/cephtest 2023-10-10T20:20:51.317 DEBUG:teuthology.orchestra.run.smithi111:> test '!' -e /home/ubuntu/cephtest 2023-10-10T20:20:51.333 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-10T20:20:51.344 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-10T20:20:51.344 DEBUG:teuthology.orchestra.run.smithi089:> test -z $(ls -A /var/lib/ceph) 2023-10-10T20:20:51.375 DEBUG:teuthology.orchestra.run.smithi111:> test -z $(ls -A /var/lib/ceph) 2023-10-10T20:20:51.389 INFO:teuthology.orchestra.run.smithi089.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-10T20:20:51.390 INFO:teuthology.orchestra.run.smithi111.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-10T20:20:51.391 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-10T20:20:51.448 INFO:teuthology.run_tasks:Running task kernel... 2023-10-10T20:20:51.468 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-10-10T20:20:51.469 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-10-10T20:20:51.469 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-10-10T20:20:51.470 DEBUG:teuthology.orchestra.run.smithi089:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-10T20:20:51.471 DEBUG:teuthology.orchestra.run.smithi111:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-10T20:20:51.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:20:51.488 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2023-10-10T20:20:51.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:20:51.489 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2023-10-10T20:20:51.543 INFO:teuthology.orchestra.run.smithi089.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-10T20:20:51.544 INFO:teuthology.task.kernel:Running kernel on smithi089: 4.18.0-305.25.1.el8_4.x86_64 2023-10-10T20:20:51.544 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum install -y kernel 2023-10-10T20:20:51.546 INFO:teuthology.orchestra.run.smithi111.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-10T20:20:51.546 INFO:teuthology.task.kernel:Running kernel on smithi111: 4.18.0-305.25.1.el8_4.x86_64 2023-10-10T20:20:51.546 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum install -y kernel 2023-10-10T20:20:52.988 INFO:teuthology.orchestra.run.smithi111.stdout:Updating Subscription Management repositories. 2023-10-10T20:20:52.989 INFO:teuthology.orchestra.run.smithi111.stdout:Unable to read consumer identity 2023-10-10T20:20:53.003 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-10-10T20:20:53.004 INFO:teuthology.orchestra.run.smithi089.stdout:Unable to read consumer identity 2023-10-10T20:20:54.388 INFO:teuthology.orchestra.run.smithi111.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-10-10T20:20:54.408 INFO:teuthology.orchestra.run.smithi089.stdout:Copr repo for python3-asyncssh owned by ceph 36 kB/s | 3.5 kB 00:00 2023-10-10T20:20:55.237 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 20 MB/s | 16 MB 00:00 2023-10-10T20:20:55.279 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 19 MB/s | 16 MB 00:00 2023-10-10T20:20:58.564 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2023-10-10T20:20:58.629 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 4.6 MB/s | 24 kB 00:00 2023-10-10T20:20:59.314 INFO:teuthology.orchestra.run.smithi089.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-10T20:20:59.314 INFO:teuthology.orchestra.run.smithi089.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-10T20:20:59.315 INFO:teuthology.orchestra.run.smithi089.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-10T20:20:59.372 INFO:teuthology.orchestra.run.smithi111.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-10T20:20:59.373 INFO:teuthology.orchestra.run.smithi111.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-10T20:20:59.373 INFO:teuthology.orchestra.run.smithi111.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-10T20:20:59.473 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-10-10T20:20:59.474 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2023-10-10T20:20:59.475 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-10-10T20:20:59.537 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2023-10-10T20:20:59.538 INFO:teuthology.orchestra.run.smithi111.stdout:Nothing to do. 2023-10-10T20:20:59.538 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2023-10-10T20:20:59.600 DEBUG:teuthology.orchestra.run.smithi089:> echo no | sudo yum reinstall kernel || true 2023-10-10T20:20:59.627 DEBUG:teuthology.orchestra.run.smithi111:> echo no | sudo yum reinstall kernel || true 2023-10-10T20:20:59.944 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-10-10T20:20:59.945 INFO:teuthology.orchestra.run.smithi089.stdout:Unable to read consumer identity 2023-10-10T20:20:59.969 INFO:teuthology.orchestra.run.smithi111.stdout:Updating Subscription Management repositories. 2023-10-10T20:20:59.969 INFO:teuthology.orchestra.run.smithi111.stdout:Unable to read consumer identity 2023-10-10T20:21:00.239 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:00:02 ago on Tue 10 Oct 2023 08:20:58 PM UTC. 2023-10-10T20:21:00.269 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:00:02 ago on Tue 10 Oct 2023 08:20:58 PM UTC. 2023-10-10T20:21:00.380 INFO:teuthology.orchestra.run.smithi089.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-10T20:21:00.380 INFO:teuthology.orchestra.run.smithi089.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-10T20:21:00.380 INFO:teuthology.orchestra.run.smithi089.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-10T20:21:00.387 INFO:teuthology.orchestra.run.smithi111.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-10T20:21:00.388 INFO:teuthology.orchestra.run.smithi111.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-10T20:21:00.388 INFO:teuthology.orchestra.run.smithi111.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-10T20:21:00.415 INFO:teuthology.orchestra.run.smithi089.stderr:Error: No packages marked for reinstall. 2023-10-10T20:21:00.423 INFO:teuthology.orchestra.run.smithi111.stderr:Error: No packages marked for reinstall. 2023-10-10T20:21:00.465 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum reinstall -y kernel || true 2023-10-10T20:21:00.472 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum reinstall -y kernel || true 2023-10-10T20:21:00.809 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-10-10T20:21:00.810 INFO:teuthology.orchestra.run.smithi089.stdout:Unable to read consumer identity 2023-10-10T20:21:00.812 INFO:teuthology.orchestra.run.smithi111.stdout:Updating Subscription Management repositories. 2023-10-10T20:21:00.813 INFO:teuthology.orchestra.run.smithi111.stdout:Unable to read consumer identity 2023-10-10T20:21:01.097 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:00:03 ago on Tue 10 Oct 2023 08:20:58 PM UTC. 2023-10-10T20:21:01.102 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:00:03 ago on Tue 10 Oct 2023 08:20:58 PM UTC. 2023-10-10T20:21:01.228 INFO:teuthology.orchestra.run.smithi111.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-10T20:21:01.228 INFO:teuthology.orchestra.run.smithi111.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-10T20:21:01.229 INFO:teuthology.orchestra.run.smithi111.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-10T20:21:01.235 INFO:teuthology.orchestra.run.smithi089.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-10T20:21:01.235 INFO:teuthology.orchestra.run.smithi089.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-10T20:21:01.235 INFO:teuthology.orchestra.run.smithi089.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-10T20:21:01.239 INFO:teuthology.orchestra.run.smithi111.stderr:Error: No packages marked for reinstall. 2023-10-10T20:21:01.256 INFO:teuthology.orchestra.run.smithi089.stderr:Error: No packages marked for reinstall. 2023-10-10T20:21:01.288 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q kernel | sort -rV | head -n 1 2023-10-10T20:21:01.306 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q kernel | sort -rV | head -n 1 2023-10-10T20:21:01.413 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-10T20:21:01.413 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:21:01.413 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:21:01.414 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-10T20:21:01.414 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-10T20:21:01.414 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2023-10-10T20:21:01.414 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi111.front.sepia.ceph.com, path=None, version=distro) 2023-10-10T20:21:01.415 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q kernel | sort -rV | head -n 1 2023-10-10T20:21:01.430 INFO:teuthology.orchestra.run.smithi089.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-10T20:21:01.430 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:21:01.431 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:21:01.431 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-10T20:21:01.431 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-10T20:21:01.431 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2023-10-10T20:21:01.432 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi089.front.sepia.ceph.com, path=None, version=distro) 2023-10-10T20:21:01.432 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q kernel | sort -rV | head -n 1 2023-10-10T20:21:01.541 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-10T20:21:01.542 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:21:01.542 DEBUG:teuthology.orchestra.run.smithi111:> sudo rpm -qi grub2-tools 2023-10-10T20:21:01.556 INFO:teuthology.orchestra.run.smithi089.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-10T20:21:01.557 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:21:01.557 DEBUG:teuthology.orchestra.run.smithi089:> sudo rpm -qi grub2-tools 2023-10-10T20:21:01.569 INFO:teuthology.orchestra.run.smithi111.stdout:Name : grub2-tools 2023-10-10T20:21:01.569 INFO:teuthology.orchestra.run.smithi111.stdout:Epoch : 1 2023-10-10T20:21:01.569 INFO:teuthology.orchestra.run.smithi111.stdout:Version : 2.02 2023-10-10T20:21:01.570 INFO:teuthology.orchestra.run.smithi111.stdout:Release : 99.el8_4.1 2023-10-10T20:21:01.570 INFO:teuthology.orchestra.run.smithi111.stdout:Architecture: x86_64 2023-10-10T20:21:01.570 INFO:teuthology.orchestra.run.smithi111.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-10T20:21:01.570 INFO:teuthology.orchestra.run.smithi111.stdout:Group : System Environment/Base 2023-10-10T20:21:01.570 INFO:teuthology.orchestra.run.smithi111.stdout:Size : 9264631 2023-10-10T20:21:01.571 INFO:teuthology.orchestra.run.smithi111.stdout:License : GPLv3+ 2023-10-10T20:21:01.571 INFO:teuthology.orchestra.run.smithi111.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-10T20:21:01.571 INFO:teuthology.orchestra.run.smithi111.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-10T20:21:01.571 INFO:teuthology.orchestra.run.smithi111.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-10T20:21:01.571 INFO:teuthology.orchestra.run.smithi111.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-10T20:21:01.572 INFO:teuthology.orchestra.run.smithi111.stdout:Relocations : (not relocatable) 2023-10-10T20:21:01.572 INFO:teuthology.orchestra.run.smithi111.stdout:Packager : Red Hat, Inc. 2023-10-10T20:21:01.572 INFO:teuthology.orchestra.run.smithi111.stdout:Vendor : Red Hat, Inc. 2023-10-10T20:21:01.572 INFO:teuthology.orchestra.run.smithi111.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-10T20:21:01.572 INFO:teuthology.orchestra.run.smithi111.stdout:Summary : Support tools for GRUB. 2023-10-10T20:21:01.573 INFO:teuthology.orchestra.run.smithi111.stdout:Description : 2023-10-10T20:21:01.573 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:21:01.573 INFO:teuthology.orchestra.run.smithi111.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-10T20:21:01.573 INFO:teuthology.orchestra.run.smithi111.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-10T20:21:01.574 INFO:teuthology.orchestra.run.smithi111.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-10T20:21:01.574 INFO:teuthology.orchestra.run.smithi111.stdout:hardware devices. 2023-10-10T20:21:01.574 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:21:01.574 INFO:teuthology.orchestra.run.smithi111.stdout:This subpackage provides tools for support of all platforms. 2023-10-10T20:21:01.575 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-10T20:21:01.575 INFO:teuthology.task.kernel:Updating grub on smithi111 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:21:01.576 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-10T20:21:01.584 INFO:teuthology.orchestra.run.smithi089.stdout:Name : grub2-tools 2023-10-10T20:21:01.584 INFO:teuthology.orchestra.run.smithi089.stdout:Epoch : 1 2023-10-10T20:21:01.584 INFO:teuthology.orchestra.run.smithi089.stdout:Version : 2.02 2023-10-10T20:21:01.585 INFO:teuthology.orchestra.run.smithi089.stdout:Release : 99.el8_4.1 2023-10-10T20:21:01.585 INFO:teuthology.orchestra.run.smithi089.stdout:Architecture: x86_64 2023-10-10T20:21:01.585 INFO:teuthology.orchestra.run.smithi089.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-10T20:21:01.585 INFO:teuthology.orchestra.run.smithi089.stdout:Group : System Environment/Base 2023-10-10T20:21:01.585 INFO:teuthology.orchestra.run.smithi089.stdout:Size : 9264631 2023-10-10T20:21:01.586 INFO:teuthology.orchestra.run.smithi089.stdout:License : GPLv3+ 2023-10-10T20:21:01.586 INFO:teuthology.orchestra.run.smithi089.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-10T20:21:01.586 INFO:teuthology.orchestra.run.smithi089.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-10T20:21:01.586 INFO:teuthology.orchestra.run.smithi089.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-10T20:21:01.586 INFO:teuthology.orchestra.run.smithi089.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-10T20:21:01.587 INFO:teuthology.orchestra.run.smithi089.stdout:Relocations : (not relocatable) 2023-10-10T20:21:01.587 INFO:teuthology.orchestra.run.smithi089.stdout:Packager : Red Hat, Inc. 2023-10-10T20:21:01.587 INFO:teuthology.orchestra.run.smithi089.stdout:Vendor : Red Hat, Inc. 2023-10-10T20:21:01.587 INFO:teuthology.orchestra.run.smithi089.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-10T20:21:01.587 INFO:teuthology.orchestra.run.smithi089.stdout:Summary : Support tools for GRUB. 2023-10-10T20:21:01.587 INFO:teuthology.orchestra.run.smithi089.stdout:Description : 2023-10-10T20:21:01.588 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:21:01.588 INFO:teuthology.orchestra.run.smithi089.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-10T20:21:01.588 INFO:teuthology.orchestra.run.smithi089.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-10T20:21:01.588 INFO:teuthology.orchestra.run.smithi089.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-10T20:21:01.588 INFO:teuthology.orchestra.run.smithi089.stdout:hardware devices. 2023-10-10T20:21:01.589 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:21:01.589 INFO:teuthology.orchestra.run.smithi089.stdout:This subpackage provides tools for support of all platforms. 2023-10-10T20:21:01.590 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-10T20:21:01.590 INFO:teuthology.task.kernel:Updating grub on smithi089 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:21:01.590 DEBUG:teuthology.orchestra.run.smithi089:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-10T20:21:02.814 INFO:teuthology.orchestra.run.smithi089.stderr:Generating grub configuration file ... 2023-10-10T20:21:02.976 INFO:teuthology.orchestra.run.smithi111.stderr:Generating grub configuration file ... 2023-10-10T20:21:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:done 2023-10-10T20:21:05.024 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2023-10-10T20:21:05.038 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.ja47ew5KDz 2023-10-10T20:21:05.039 DEBUG:teuthology.orchestra.run.smithi089:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ja47ew5KDz 2023-10-10T20:21:05.108 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.ja47ew5KDz 2023-10-10T20:21:05.261 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.ja47ew5KDz is 6KB 2023-10-10T20:21:05.310 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.ja47ew5KDz 2023-10-10T20:21:05.324 DEBUG:teuthology.orchestra.run.smithi089:> sudo /bin/ls /boot/loader/entries || true 2023-10-10T20:21:05.434 INFO:teuthology.orchestra.run.smithi089.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-10T20:21:05.434 INFO:teuthology.orchestra.run.smithi089.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-10T20:21:05.435 INFO:teuthology.orchestra.run.smithi089.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-10T20:21:05.435 INFO:teuthology.orchestra.run.smithi089.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-10T20:21:05.436 DEBUG:teuthology.orchestra.run.smithi089:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-10T20:21:05.453 INFO:teuthology.orchestra.run.smithi111.stderr:done 2023-10-10T20:21:05.455 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2023-10-10T20:21:05.469 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.jEKmjdGNAB 2023-10-10T20:21:05.469 DEBUG:teuthology.orchestra.run.smithi111:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.jEKmjdGNAB 2023-10-10T20:21:05.541 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.jEKmjdGNAB 2023-10-10T20:21:05.552 DEBUG:teuthology.orchestra.run.smithi089:> sudo shutdown -r now 2023-10-10T20:21:05.697 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.jEKmjdGNAB is 6KB 2023-10-10T20:21:05.745 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.jEKmjdGNAB 2023-10-10T20:21:05.759 DEBUG:teuthology.orchestra.run.smithi111:> sudo /bin/ls /boot/loader/entries || true 2023-10-10T20:21:05.829 INFO:teuthology.orchestra.run.smithi111.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-10T20:21:05.829 INFO:teuthology.orchestra.run.smithi111.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-10T20:21:05.829 INFO:teuthology.orchestra.run.smithi111.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-10T20:21:05.829 INFO:teuthology.orchestra.run.smithi111.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-10T20:21:05.830 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-10T20:21:06.018 DEBUG:teuthology.orchestra.run.smithi111:> sudo shutdown -r now 2023-10-10T20:21:35.582 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-10T20:21:35.583 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-10T20:21:35.584 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:21:36.021 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-10T20:21:36.021 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-10-10T20:21:36.022 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:21:53.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.89 2023-10-10T20:21:54.426 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-10-10T20:22:02.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-10T20:22:02.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:22:03.428 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-10-10T20:22:03.428 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:22:06.010 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.89 2023-10-10T20:22:06.490 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-10-10T20:22:18.022 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-10T20:22:18.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:22:18.492 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-10-10T20:22:18.493 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:22:21.566 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-10-10T20:22:36.578 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-10-10T20:22:36.579 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:22:50.050 DEBUG:teuthology.orchestra.run.smithi089:> true 2023-10-10T20:22:50.309 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-10T20:22:50.309 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-10T20:22:50.309 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2023-10-10T20:22:50.323 INFO:teuthology.orchestra.run.smithi089.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-10T20:22:50.324 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:22:50.324 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-10T20:22:50.324 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-10T20:22:51.325 DEBUG:teuthology.parallel:result is None 2023-10-10T20:22:55.098 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-10-10T20:23:13.099 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-10-10T20:23:13.100 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:23:13.798 DEBUG:teuthology.orchestra.run.smithi111:> true 2023-10-10T20:23:14.045 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-10-10T20:23:14.046 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-10T20:23:14.046 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2023-10-10T20:23:14.060 INFO:teuthology.orchestra.run.smithi111.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-10T20:23:14.061 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:23:14.061 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-10T20:23:14.062 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-10T20:23:15.063 DEBUG:teuthology.parallel:result is None 2023-10-10T20:23:15.064 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-10T20:23:15.072 INFO:teuthology.task.internal:Creating test directory... 2023-10-10T20:23:15.072 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-10T20:23:15.076 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-10T20:23:15.093 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-10T20:23:15.100 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-10T20:23:15.111 INFO:teuthology.task.internal:Creating archive directory... 2023-10-10T20:23:15.111 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-10T20:23:15.133 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-10T20:23:15.172 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-10T20:23:15.184 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-10T20:23:15.184 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-10T20:23:15.195 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-10T20:23:15.283 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-10T20:23:15.288 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-10T20:23:15.299 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-10T20:23:15.302 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-10T20:23:15.304 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-10T20:23:15.313 INFO:teuthology.task.internal:Configuring sudo... 2023-10-10T20:23:15.313 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-10T20:23:15.316 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-10T20:23:15.342 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-10T20:23:15.356 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-10T20:23:15.356 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-10T20:23:15.382 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-10T20:23:15.397 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-10T20:23:15.480 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-10T20:23:15.550 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-10T20:23:15.592 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-10T20:23:15.632 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:23:15.632 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-10T20:23:15.697 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-10T20:23:15.740 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-10T20:23:15.798 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-10T20:23:15.881 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-10T20:23:15.920 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:23:15.920 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-10T20:23:15.984 DEBUG:teuthology.orchestra.run.smithi089:> sudo service rsyslog restart 2023-10-10T20:23:15.987 DEBUG:teuthology.orchestra.run.smithi111:> sudo service rsyslog restart 2023-10-10T20:23:16.044 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-10T20:23:16.076 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-10T20:23:16.557 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-10T20:23:16.568 INFO:teuthology.task.internal:Starting timer... 2023-10-10T20:23:16.568 INFO:teuthology.run_tasks:Running task pcp... 2023-10-10T20:23:16.585 INFO:teuthology.run_tasks:Running task selinux... 2023-10-10T20:23:16.598 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-10-10T20:23:16.599 DEBUG:teuthology.orchestra.run.smithi089:> sudo service auditd rotate 2023-10-10T20:23:16.669 INFO:teuthology.orchestra.run.smithi089.stdout:Rotating logs: [ OK ] 2023-10-10T20:23:16.670 DEBUG:teuthology.orchestra.run.smithi111:> sudo service auditd rotate 2023-10-10T20:23:16.737 INFO:teuthology.orchestra.run.smithi111.stdout:Rotating logs: [ OK ] 2023-10-10T20:23:16.738 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-10T20:23:16.739 DEBUG:teuthology.orchestra.run.smithi089:> /usr/sbin/getenforce 2023-10-10T20:23:16.761 INFO:teuthology.orchestra.run.smithi089.stdout:Permissive 2023-10-10T20:23:16.761 DEBUG:teuthology.orchestra.run.smithi111:> /usr/sbin/getenforce 2023-10-10T20:23:16.781 INFO:teuthology.orchestra.run.smithi111.stdout:Permissive 2023-10-10T20:23:16.781 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi089.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi111.front.sepia.ceph.com': 'permissive'} 2023-10-10T20:23:16.782 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-10T20:23:16.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:23:16.827 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-10T20:23:16.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:23:16.852 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-10T20:23:16.852 DEBUG:teuthology.orchestra.run.smithi089:> sudo /usr/sbin/setenforce permissive 2023-10-10T20:23:16.893 DEBUG:teuthology.orchestra.run.smithi111:> sudo /usr/sbin/setenforce permissive 2023-10-10T20:23:16.917 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-10T20:23:16.928 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-10T20:23:16.933 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-10T20:23:17.057 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-10T20:23:17.072 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-10T20:23:17.073 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi089.front.sepia.ceph.com,smithi111.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-10T20:31:34.389 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi089.front.sepia.ceph.com'), Remote(name='ubuntu@smithi111.front.sepia.ceph.com')] 2023-10-10T20:31:34.390 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-10T20:31:34.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:31:34.459 DEBUG:teuthology.orchestra.run.smithi089:> true 2023-10-10T20:31:34.525 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-10T20:31:34.525 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-10-10T20:31:34.526 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:31:34.596 DEBUG:teuthology.orchestra.run.smithi111:> true 2023-10-10T20:31:34.662 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-10-10T20:31:34.662 INFO:teuthology.run_tasks:Running task clock... 2023-10-10T20:31:34.674 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-10T20:31:34.674 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-10T20:31:34.675 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-10T20:31:34.677 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-10T20:31:34.678 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-10T20:31:34.705 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-10T20:31:34.719 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-10T20:31:34.733 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-10T20:31:34.748 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-10T20:31:34.758 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: ntpd: command not found 2023-10-10T20:31:34.768 INFO:teuthology.orchestra.run.smithi089.stdout:506 Cannot talk to daemon 2023-10-10T20:31:34.781 INFO:teuthology.orchestra.run.smithi111.stderr:sudo: ntpd: command not found 2023-10-10T20:31:34.782 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-10T20:31:34.792 INFO:teuthology.orchestra.run.smithi111.stdout:506 Cannot talk to daemon 2023-10-10T20:31:34.795 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-10T20:31:34.805 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-10T20:31:34.819 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-10T20:31:34.870 INFO:teuthology.orchestra.run.smithi089.stderr:bash: ntpq: command not found 2023-10-10T20:31:34.873 INFO:teuthology.orchestra.run.smithi089.stdout:210 Number of sources = 4 2023-10-10T20:31:34.874 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-10T20:31:34.874 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2023-10-10T20:31:34.874 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:31:34.875 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:31:34.875 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:31:34.875 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:31:34.890 INFO:teuthology.orchestra.run.smithi111.stderr:bash: ntpq: command not found 2023-10-10T20:31:34.892 INFO:teuthology.orchestra.run.smithi111.stdout:210 Number of sources = 4 2023-10-10T20:31:34.893 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-10T20:31:34.893 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2023-10-10T20:31:34.893 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:31:34.893 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:31:34.894 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:31:34.894 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:31:34.895 INFO:teuthology.run_tasks:Running task install... 2023-10-10T20:31:34.904 DEBUG:teuthology.task.install:project ceph 2023-10-10T20:31:34.905 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7'}} 2023-10-10T20:31:34.905 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7'} 2023-10-10T20:31:34.906 INFO:teuthology.task.install:Using flavor: default 2023-10-10T20:31:34.912 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-10T20:31:34.913 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-10T20:31:34.913 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7', 'tag': None, 'wait_for_package': False} 2023-10-10T20:31:34.914 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=46d1f59b549730ba7f93d940a3d32b94c45a4cf7 2023-10-10T20:31:34.916 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7', 'tag': None, 'wait_for_package': False} 2023-10-10T20:31:34.916 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=46d1f59b549730ba7f93d940a3d32b94c45a4cf7 2023-10-10T20:31:35.059 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-10-0720-quincy/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/centos/8/flavors/default/ 2023-10-10T20:31:35.059 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1374.g46d1f59b 2023-10-10T20:31:35.136 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-10-0720-quincy/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/centos/8/flavors/default/ 2023-10-10T20:31:35.137 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1374.g46d1f59b 2023-10-10T20:31:35.310 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-10-0720-quincy/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-10-0720-quincy/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-10-0720-quincy/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-10T20:31:35.310 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:31:35.310 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-10T20:31:35.337 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-10T20:31:35.337 DEBUG:teuthology.orchestra.run.smithi111:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-10T20:31:35.343 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-10-0720-quincy/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-10-0720-quincy/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-10-0720-quincy/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-10T20:31:35.344 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:31:35.344 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-10T20:31:35.371 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-10T20:31:35.372 DEBUG:teuthology.orchestra.run.smithi089:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-10T20:31:35.403 DEBUG:teuthology.orchestra.run.smithi111:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-10T20:31:35.437 DEBUG:teuthology.orchestra.run.smithi089:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-10T20:31:35.478 DEBUG:teuthology.orchestra.run.smithi111:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-10T20:31:35.513 DEBUG:teuthology.orchestra.run.smithi089:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-10T20:31:35.542 INFO:teuthology.orchestra.run.smithi111.stdout:check_obsoletes = 1 2023-10-10T20:31:35.543 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum clean all 2023-10-10T20:31:35.577 INFO:teuthology.orchestra.run.smithi089.stdout:check_obsoletes = 1 2023-10-10T20:31:35.578 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum clean all 2023-10-10T20:31:35.931 INFO:teuthology.orchestra.run.smithi111.stdout:Updating Subscription Management repositories. 2023-10-10T20:31:35.967 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-10-10T20:31:36.230 INFO:teuthology.orchestra.run.smithi111.stdout:44 files removed 2023-10-10T20:31:36.272 INFO:teuthology.orchestra.run.smithi089.stdout:44 files removed 2023-10-10T20:31:36.280 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-10T20:31:36.327 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-10T20:31:36.629 INFO:teuthology.orchestra.run.smithi111.stdout:Updating Subscription Management repositories. 2023-10-10T20:31:36.676 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-10-10T20:31:37.783 INFO:teuthology.orchestra.run.smithi111.stdout:ceph packages for x86_64 107 kB/s | 78 kB 00:00 2023-10-10T20:31:37.812 INFO:teuthology.orchestra.run.smithi089.stdout:ceph packages for x86_64 109 kB/s | 78 kB 00:00 2023-10-10T20:31:38.168 INFO:teuthology.orchestra.run.smithi111.stdout:ceph noarch packages 27 kB/s | 9.2 kB 00:00 2023-10-10T20:31:38.172 INFO:teuthology.orchestra.run.smithi089.stdout:ceph noarch packages 29 kB/s | 9.2 kB 00:00 2023-10-10T20:31:38.387 INFO:teuthology.orchestra.run.smithi089.stdout:ceph source packages 7.9 kB/s | 1.4 kB 00:00 2023-10-10T20:31:38.447 INFO:teuthology.orchestra.run.smithi111.stdout:ceph source packages 5.8 kB/s | 1.4 kB 00:00 2023-10-10T20:31:38.497 INFO:teuthology.orchestra.run.smithi089.stdout:Copr repo for python3-asyncssh owned by ceph 43 kB/s | 3.5 kB 00:00 2023-10-10T20:31:38.553 INFO:teuthology.orchestra.run.smithi111.stdout:Copr repo for python3-asyncssh owned by ceph 44 kB/s | 3.5 kB 00:00 2023-10-10T20:31:39.252 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2023-10-10T20:31:40.754 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 7.3 MB/s | 16 MB 00:02 2023-10-10T20:31:42.494 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 4.1 MB/s | 24 kB 00:00 2023-10-10T20:31:42.877 INFO:teuthology.orchestra.run.smithi089.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 126 MB/s | 38 MB 00:00 2023-10-10T20:31:44.040 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2023-10-10T20:31:44.438 INFO:teuthology.orchestra.run.smithi111.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 121 MB/s | 38 MB 00:00 2023-10-10T20:31:48.338 INFO:teuthology.orchestra.run.smithi089.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 120 MB/s | 33 MB 00:00 2023-10-10T20:31:49.951 INFO:teuthology.orchestra.run.smithi111.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 98 MB/s | 33 MB 00:00 2023-10-10T20:31:56.269 INFO:teuthology.orchestra.run.smithi089.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-10T20:31:56.269 INFO:teuthology.orchestra.run.smithi089.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-10T20:31:56.280 INFO:teuthology.orchestra.run.smithi089.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-10T20:31:56.281 INFO:teuthology.orchestra.run.smithi089.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-10T20:31:56.281 INFO:teuthology.orchestra.run.smithi089.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-10T20:31:56.282 INFO:teuthology.orchestra.run.smithi089.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-10T20:31:56.284 INFO:teuthology.orchestra.run.smithi089.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-10T20:31:56.286 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-10T20:31:56.287 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-10T20:31:56.772 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-10-10T20:31:56.783 INFO:teuthology.orchestra.run.smithi089.stdout:====================================================================================================================== 2023-10-10T20:31:56.783 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-10-10T20:31:56.783 INFO:teuthology.orchestra.run.smithi089.stdout:====================================================================================================================== 2023-10-10T20:31:56.783 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-10-10T20:31:56.784 INFO:teuthology.orchestra.run.smithi089.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-10T20:31:56.784 INFO:teuthology.orchestra.run.smithi089.stdout: ceph x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 6.4 k 2023-10-10T20:31:56.784 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 5.2 M 2023-10-10T20:31:56.784 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 876 k 2023-10-10T20:31:56.784 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 161 k 2023-10-10T20:31:56.784 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 1.6 M 2023-10-10T20:31:56.785 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 121 k 2023-10-10T20:31:56.785 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 1.7 M 2023-10-10T20:31:56.785 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 7.4 M 2023-10-10T20:31:56.785 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 47 k 2023-10-10T20:31:56.785 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 17 M 2023-10-10T20:31:56.786 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 52 M 2023-10-10T20:31:56.786 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 267 k 2023-10-10T20:31:56.786 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 80 k 2023-10-10T20:31:56.786 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 255 k 2023-10-10T20:31:56.786 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 18 k 2023-10-10T20:31:56.786 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-10T20:31:56.787 INFO:teuthology.orchestra.run.smithi089.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-10T20:31:56.787 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-10T20:31:56.787 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 25 k 2023-10-10T20:31:56.787 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 797 k 2023-10-10T20:31:56.787 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 130 k 2023-10-10T20:31:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-10T20:31:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-10T20:31:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-10T20:31:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 190 k 2023-10-10T20:31:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 371 k 2023-10-10T20:31:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 358 k 2023-10-10T20:31:56.789 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 107 k 2023-10-10T20:31:56.789 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 90 k 2023-10-10T20:31:56.789 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 3.5 M 2023-10-10T20:31:56.789 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 173 k 2023-10-10T20:31:56.789 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-10T20:31:56.790 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrading: 2023-10-10T20:31:56.790 INFO:teuthology.orchestra.run.smithi089.stdout: librados2 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 3.5 M 2023-10-10T20:31:56.790 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 3.6 M 2023-10-10T20:31:56.790 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2023-10-10T20:31:56.790 INFO:teuthology.orchestra.run.smithi089.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-10T20:31:56.791 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 23 M 2023-10-10T20:31:56.791 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 23 k 2023-10-10T20:31:56.791 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 2.3 M 2023-10-10T20:31:56.791 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 238 k 2023-10-10T20:31:56.791 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 3.9 M 2023-10-10T20:31:56.791 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 17 M 2023-10-10T20:31:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 14 k 2023-10-10T20:31:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 24 k 2023-10-10T20:31:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-10T20:31:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-10T20:31:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-10T20:31:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-10T20:31:56.793 INFO:teuthology.orchestra.run.smithi089.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-10T20:31:56.793 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 174 k 2023-10-10T20:31:56.793 INFO:teuthology.orchestra.run.smithi089.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-10T20:31:56.793 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-10T20:31:56.793 INFO:teuthology.orchestra.run.smithi089.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-10T20:31:56.794 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-10T20:31:56.794 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-10T20:31:56.794 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 507 k 2023-10-10T20:31:56.794 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-10T20:31:56.794 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 5.6 M 2023-10-10T20:31:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-10T20:31:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-10T20:31:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-10T20:31:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-10T20:31:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-10T20:31:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-10T20:31:56.796 INFO:teuthology.orchestra.run.smithi089.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-10T20:31:56.796 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-10T20:31:56.796 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-10T20:31:56.796 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-10T20:31:56.796 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-10T20:31:56.797 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-10T20:31:56.797 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-10T20:31:56.797 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-10T20:31:56.797 INFO:teuthology.orchestra.run.smithi089.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-10T20:31:56.798 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-10T20:31:56.798 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 45 k 2023-10-10T20:31:56.798 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 115 k 2023-10-10T20:31:56.798 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-10T20:31:56.798 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-10T20:31:56.799 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-10T20:31:56.799 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-10T20:31:56.799 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-10T20:31:56.799 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-10T20:31:56.799 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-10T20:31:56.800 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-10T20:31:56.800 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-10T20:31:56.800 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-10T20:31:56.800 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-10T20:31:56.800 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-10T20:31:56.800 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-10T20:31:56.801 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-10T20:31:56.801 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-10T20:31:56.801 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-10T20:31:56.801 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-10T20:31:56.802 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-10T20:31:56.802 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-10T20:31:56.802 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-10T20:31:56.802 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-10T20:31:56.803 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-10T20:31:56.803 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-10T20:31:56.803 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-10T20:31:56.803 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-10T20:31:56.803 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-10T20:31:56.804 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-10T20:31:56.804 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-10T20:31:56.804 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-10T20:31:56.804 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-10T20:31:56.804 INFO:teuthology.orchestra.run.smithi089.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-10T20:31:56.805 INFO:teuthology.orchestra.run.smithi089.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-10T20:31:56.805 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-10T20:31:56.805 INFO:teuthology.orchestra.run.smithi089.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-10T20:31:56.805 INFO:teuthology.orchestra.run.smithi089.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-10T20:31:56.805 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-10T20:31:56.805 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-10T20:31:56.806 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-10T20:31:56.806 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-10T20:31:56.806 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-10T20:31:56.806 INFO:teuthology.orchestra.run.smithi089.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-10T20:31:56.806 INFO:teuthology.orchestra.run.smithi089.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-10T20:31:56.807 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-10T20:31:56.807 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-10T20:31:56.807 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:31:56.807 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-10-10T20:31:56.807 INFO:teuthology.orchestra.run.smithi089.stdout:====================================================================================================================== 2023-10-10T20:31:56.808 INFO:teuthology.orchestra.run.smithi089.stdout:Install 118 Packages 2023-10-10T20:31:56.808 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrade 2 Packages 2023-10-10T20:31:56.808 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:31:56.809 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 202 M 2023-10-10T20:31:56.809 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-10-10T20:31:57.295 INFO:teuthology.orchestra.run.smithi089.stdout:(1/120): ceph-17.2.6-1374.g46d1f59b.el8.x86_64. 40 kB/s | 6.4 kB 00:00 2023-10-10T20:31:57.596 INFO:teuthology.orchestra.run.smithi089.stdout:(2/120): ceph-fuse-17.2.6-1374.g46d1f59b.el8.x8 2.8 MB/s | 876 kB 00:00 2023-10-10T20:31:57.649 INFO:teuthology.orchestra.run.smithi089.stdout:(3/120): ceph-immutable-object-cache-17.2.6-137 3.0 MB/s | 161 kB 00:00 2023-10-10T20:31:57.766 INFO:teuthology.orchestra.run.smithi111.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-10T20:31:57.767 INFO:teuthology.orchestra.run.smithi111.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-10T20:31:57.777 INFO:teuthology.orchestra.run.smithi111.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-10T20:31:57.778 INFO:teuthology.orchestra.run.smithi111.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-10T20:31:57.779 INFO:teuthology.orchestra.run.smithi111.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-10T20:31:57.780 INFO:teuthology.orchestra.run.smithi111.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-10T20:31:57.782 INFO:teuthology.orchestra.run.smithi111.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-10T20:31:57.784 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-10T20:31:57.785 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-10T20:31:57.859 INFO:teuthology.orchestra.run.smithi089.stdout:(4/120): ceph-base-17.2.6-1374.g46d1f59b.el8.x8 7.1 MB/s | 5.2 MB 00:00 2023-10-10T20:31:58.011 INFO:teuthology.orchestra.run.smithi089.stdout:(5/120): ceph-mgr-17.2.6-1374.g46d1f59b.el8.x86 10 MB/s | 1.6 MB 00:00 2023-10-10T20:31:58.045 INFO:teuthology.orchestra.run.smithi089.stdout:(6/120): ceph-mds-17.2.6-1374.g46d1f59b.el8.x86 5.8 MB/s | 2.3 MB 00:00 2023-10-10T20:31:58.282 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2023-10-10T20:31:58.293 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================================================== 2023-10-10T20:31:58.293 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2023-10-10T20:31:58.293 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================================================== 2023-10-10T20:31:58.294 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2023-10-10T20:31:58.294 INFO:teuthology.orchestra.run.smithi111.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-10T20:31:58.294 INFO:teuthology.orchestra.run.smithi111.stdout: ceph x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 6.4 k 2023-10-10T20:31:58.294 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 5.2 M 2023-10-10T20:31:58.294 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 876 k 2023-10-10T20:31:58.295 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 161 k 2023-10-10T20:31:58.295 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 1.6 M 2023-10-10T20:31:58.295 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 121 k 2023-10-10T20:31:58.295 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 1.7 M 2023-10-10T20:31:58.295 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 7.4 M 2023-10-10T20:31:58.296 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 47 k 2023-10-10T20:31:58.296 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 17 M 2023-10-10T20:31:58.296 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 52 M 2023-10-10T20:31:58.296 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 267 k 2023-10-10T20:31:58.296 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 80 k 2023-10-10T20:31:58.297 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 255 k 2023-10-10T20:31:58.297 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 18 k 2023-10-10T20:31:58.297 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-10T20:31:58.297 INFO:teuthology.orchestra.run.smithi111.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-10T20:31:58.297 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-10T20:31:58.298 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 25 k 2023-10-10T20:31:58.298 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 797 k 2023-10-10T20:31:58.298 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 130 k 2023-10-10T20:31:58.298 INFO:teuthology.orchestra.run.smithi111.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-10T20:31:58.298 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-10T20:31:58.298 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-10T20:31:58.299 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 190 k 2023-10-10T20:31:58.299 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 371 k 2023-10-10T20:31:58.299 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 358 k 2023-10-10T20:31:58.299 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 107 k 2023-10-10T20:31:58.299 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 90 k 2023-10-10T20:31:58.300 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 3.5 M 2023-10-10T20:31:58.300 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 173 k 2023-10-10T20:31:58.300 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-10T20:31:58.300 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrading: 2023-10-10T20:31:58.300 INFO:teuthology.orchestra.run.smithi111.stdout: librados2 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 3.5 M 2023-10-10T20:31:58.301 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 3.6 M 2023-10-10T20:31:58.301 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2023-10-10T20:31:58.301 INFO:teuthology.orchestra.run.smithi111.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-10T20:31:58.301 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 23 M 2023-10-10T20:31:58.301 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 23 k 2023-10-10T20:31:58.302 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 2.3 M 2023-10-10T20:31:58.302 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 238 k 2023-10-10T20:31:58.302 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 3.9 M 2023-10-10T20:31:58.302 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 17 M 2023-10-10T20:31:58.302 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 14 k 2023-10-10T20:31:58.303 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 24 k 2023-10-10T20:31:58.303 INFO:teuthology.orchestra.run.smithi111.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-10T20:31:58.303 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-10T20:31:58.303 INFO:teuthology.orchestra.run.smithi111.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-10T20:31:58.304 INFO:teuthology.orchestra.run.smithi111.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-10T20:31:58.304 INFO:teuthology.orchestra.run.smithi111.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-10T20:31:58.304 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 174 k 2023-10-10T20:31:58.304 INFO:teuthology.orchestra.run.smithi111.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-10T20:31:58.304 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-10T20:31:58.305 INFO:teuthology.orchestra.run.smithi111.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-10T20:31:58.305 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-10T20:31:58.305 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-10T20:31:58.305 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 507 k 2023-10-10T20:31:58.305 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-10T20:31:58.305 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 5.6 M 2023-10-10T20:31:58.306 INFO:teuthology.orchestra.run.smithi111.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-10T20:31:58.306 INFO:teuthology.orchestra.run.smithi111.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-10T20:31:58.306 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-10T20:31:58.306 INFO:teuthology.orchestra.run.smithi111.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-10T20:31:58.307 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-10T20:31:58.307 INFO:teuthology.orchestra.run.smithi111.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-10T20:31:58.307 INFO:teuthology.orchestra.run.smithi111.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-10T20:31:58.307 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-10T20:31:58.307 INFO:teuthology.orchestra.run.smithi111.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-10T20:31:58.308 INFO:teuthology.orchestra.run.smithi111.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-10T20:31:58.308 INFO:teuthology.orchestra.run.smithi111.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-10T20:31:58.308 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-10T20:31:58.308 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-10T20:31:58.308 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-10T20:31:58.309 INFO:teuthology.orchestra.run.smithi111.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-10T20:31:58.309 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-10T20:31:58.309 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 45 k 2023-10-10T20:31:58.309 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 115 k 2023-10-10T20:31:58.309 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-10T20:31:58.310 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-10T20:31:58.310 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-10T20:31:58.310 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-10T20:31:58.310 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-10T20:31:58.310 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-10T20:31:58.311 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-10T20:31:58.311 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-10T20:31:58.311 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-10T20:31:58.311 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-10T20:31:58.311 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-10T20:31:58.312 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-10T20:31:58.312 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-10T20:31:58.312 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-10T20:31:58.312 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-10T20:31:58.312 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-10T20:31:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-10T20:31:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-10T20:31:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-10T20:31:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-10T20:31:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-10T20:31:58.314 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-10T20:31:58.314 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-10T20:31:58.314 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-10T20:31:58.314 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-10T20:31:58.315 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-10T20:31:58.315 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-10T20:31:58.315 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-10T20:31:58.315 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-10T20:31:58.315 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-10T20:31:58.316 INFO:teuthology.orchestra.run.smithi111.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-10T20:31:58.316 INFO:teuthology.orchestra.run.smithi111.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-10T20:31:58.316 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-10T20:31:58.316 INFO:teuthology.orchestra.run.smithi111.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-10T20:31:58.316 INFO:teuthology.orchestra.run.smithi111.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-10T20:31:58.317 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-10T20:31:58.317 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-10T20:31:58.317 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-10T20:31:58.317 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-10T20:31:58.317 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-10T20:31:58.317 INFO:teuthology.orchestra.run.smithi111.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-10T20:31:58.318 INFO:teuthology.orchestra.run.smithi111.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-10T20:31:58.318 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-10T20:31:58.318 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-10T20:31:58.318 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:31:58.318 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2023-10-10T20:31:58.319 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================================================== 2023-10-10T20:31:58.319 INFO:teuthology.orchestra.run.smithi111.stdout:Install 118 Packages 2023-10-10T20:31:58.319 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrade 2 Packages 2023-10-10T20:31:58.319 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:31:58.320 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 202 M 2023-10-10T20:31:58.320 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2023-10-10T20:31:58.484 INFO:teuthology.orchestra.run.smithi089.stdout:(7/120): ceph-mon-17.2.6-1374.g46d1f59b.el8.x86 8.3 MB/s | 3.9 MB 00:00 2023-10-10T20:31:58.541 INFO:teuthology.orchestra.run.smithi111.stdout:(1/120): ceph-17.2.6-1374.g46d1f59b.el8.x86_64. 31 kB/s | 6.4 kB 00:00 2023-10-10T20:31:59.385 INFO:teuthology.orchestra.run.smithi111.stdout:(2/120): ceph-fuse-17.2.6-1374.g46d1f59b.el8.x8 1.0 MB/s | 876 kB 00:00 2023-10-10T20:31:59.500 INFO:teuthology.orchestra.run.smithi111.stdout:(3/120): ceph-immutable-object-cache-17.2.6-137 1.4 MB/s | 161 kB 00:00 2023-10-10T20:31:59.655 INFO:teuthology.orchestra.run.smithi089.stdout:(8/120): ceph-common-17.2.6-1374.g46d1f59b.el8. 9.3 MB/s | 23 MB 00:02 2023-10-10T20:31:59.722 INFO:teuthology.orchestra.run.smithi089.stdout:(9/120): ceph-selinux-17.2.6-1374.g46d1f59b.el8 362 kB/s | 24 kB 00:00 2023-10-10T20:32:00.445 INFO:teuthology.orchestra.run.smithi111.stdout:(4/120): ceph-mds-17.2.6-1374.g46d1f59b.el8.x86 2.4 MB/s | 2.3 MB 00:00 2023-10-10T20:32:01.215 INFO:teuthology.orchestra.run.smithi089.stdout:(10/120): ceph-radosgw-17.2.6-1374.g46d1f59b.el 6.1 MB/s | 17 MB 00:02 2023-10-10T20:32:01.221 INFO:teuthology.orchestra.run.smithi111.stdout:(5/120): ceph-mgr-17.2.6-1374.g46d1f59b.el8.x86 2.0 MB/s | 1.6 MB 00:00 2023-10-10T20:32:01.385 INFO:teuthology.orchestra.run.smithi089.stdout:(11/120): ceph-osd-17.2.6-1374.g46d1f59b.el8.x8 5.0 MB/s | 17 MB 00:03 2023-10-10T20:32:01.388 INFO:teuthology.orchestra.run.smithi089.stdout:(12/120): cephfs-mirror-17.2.6-1374.g46d1f59b.e 1.4 MB/s | 255 kB 00:00 2023-10-10T20:32:01.440 INFO:teuthology.orchestra.run.smithi089.stdout:(13/120): libcephfs-devel-17.2.6-1374.g46d1f59b 455 kB/s | 25 kB 00:00 2023-10-10T20:32:01.517 INFO:teuthology.orchestra.run.smithi089.stdout:(14/120): libcephsqlite-17.2.6-1374.g46d1f59b.e 2.2 MB/s | 174 kB 00:00 2023-10-10T20:32:01.532 INFO:teuthology.orchestra.run.smithi089.stdout:(15/120): libcephfs2-17.2.6-1374.g46d1f59b.el8. 5.4 MB/s | 797 kB 00:00 2023-10-10T20:32:01.590 INFO:teuthology.orchestra.run.smithi089.stdout:(16/120): librados-devel-17.2.6-1374.g46d1f59b. 1.7 MB/s | 130 kB 00:00 2023-10-10T20:32:01.682 INFO:teuthology.orchestra.run.smithi089.stdout:(17/120): libradosstriper1-17.2.6-1374.g46d1f59 3.3 MB/s | 507 kB 00:00 2023-10-10T20:32:01.756 INFO:teuthology.orchestra.run.smithi089.stdout:(18/120): python3-ceph-argparse-17.2.6-1374.g46 606 kB/s | 45 kB 00:00 2023-10-10T20:32:01.831 INFO:teuthology.orchestra.run.smithi089.stdout:(19/120): python3-ceph-common-17.2.6-1374.g46d1 1.5 MB/s | 115 kB 00:00 2023-10-10T20:32:01.911 INFO:teuthology.orchestra.run.smithi089.stdout:(20/120): python3-cephfs-17.2.6-1374.g46d1f59b. 2.3 MB/s | 190 kB 00:00 2023-10-10T20:32:01.998 INFO:teuthology.orchestra.run.smithi089.stdout:(21/120): python3-rados-17.2.6-1374.g46d1f59b.e 4.2 MB/s | 371 kB 00:00 2023-10-10T20:32:02.084 INFO:teuthology.orchestra.run.smithi089.stdout:(22/120): python3-rbd-17.2.6-1374.g46d1f59b.el8 4.1 MB/s | 358 kB 00:00 2023-10-10T20:32:02.161 INFO:teuthology.orchestra.run.smithi089.stdout:(23/120): python3-rgw-17.2.6-1374.g46d1f59b.el8 1.4 MB/s | 107 kB 00:00 2023-10-10T20:32:02.237 INFO:teuthology.orchestra.run.smithi089.stdout:(24/120): rbd-fuse-17.2.6-1374.g46d1f59b.el8.x8 1.2 MB/s | 90 kB 00:00 2023-10-10T20:32:02.757 INFO:teuthology.orchestra.run.smithi111.stdout:(6/120): ceph-mon-17.2.6-1374.g46d1f59b.el8.x86 2.6 MB/s | 3.9 MB 00:01 2023-10-10T20:32:02.824 INFO:teuthology.orchestra.run.smithi111.stdout:(7/120): ceph-base-17.2.6-1374.g46d1f59b.el8.x8 1.2 MB/s | 5.2 MB 00:04 2023-10-10T20:32:02.856 INFO:teuthology.orchestra.run.smithi089.stdout:(25/120): librgw2-17.2.6-1374.g46d1f59b.el8.x86 4.4 MB/s | 5.6 MB 00:01 2023-10-10T20:32:02.879 INFO:teuthology.orchestra.run.smithi089.stdout:(26/120): rbd-mirror-17.2.6-1374.g46d1f59b.el8. 5.4 MB/s | 3.5 MB 00:00 2023-10-10T20:32:02.936 INFO:teuthology.orchestra.run.smithi089.stdout:(27/120): rbd-nbd-17.2.6-1374.g46d1f59b.el8.x86 2.1 MB/s | 173 kB 00:00 2023-10-10T20:32:02.947 INFO:teuthology.orchestra.run.smithi089.stdout:(28/120): ceph-grafana-dashboards-17.2.6-1374.g 337 kB/s | 23 kB 00:00 2023-10-10T20:32:03.004 INFO:teuthology.orchestra.run.smithi089.stdout:(29/120): ceph-mgr-cephadm-17.2.6-1374.g46d1f59 1.7 MB/s | 121 kB 00:00 2023-10-10T20:32:03.858 INFO:teuthology.orchestra.run.smithi089.stdout:(30/120): ceph-mgr-dashboard-17.2.6-1374.g46d1f 1.9 MB/s | 1.7 MB 00:00 2023-10-10T20:32:03.953 INFO:teuthology.orchestra.run.smithi111.stdout:(8/120): ceph-common-17.2.6-1374.g46d1f59b.el8. 4.2 MB/s | 23 MB 00:05 2023-10-10T20:32:04.004 INFO:teuthology.orchestra.run.smithi111.stdout:(9/120): ceph-selinux-17.2.6-1374.g46d1f59b.el8 475 kB/s | 24 kB 00:00 2023-10-10T20:32:04.204 INFO:teuthology.orchestra.run.smithi089.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1374.g46 690 kB/s | 238 kB 00:00 2023-10-10T20:32:04.283 INFO:teuthology.orchestra.run.smithi089.stdout:(32/120): ceph-mgr-rook-17.2.6-1374.g46d1f59b.e 601 kB/s | 47 kB 00:00 2023-10-10T20:32:04.366 INFO:teuthology.orchestra.run.smithi089.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1374.g4 174 kB/s | 14 kB 00:00 2023-10-10T20:32:04.589 INFO:teuthology.orchestra.run.smithi089.stdout:(34/120): ceph-volume-17.2.6-1374.g46d1f59b.el8 1.2 MB/s | 267 kB 00:00 2023-10-10T20:32:04.639 INFO:teuthology.orchestra.run.smithi089.stdout:(35/120): cephadm-17.2.6-1374.g46d1f59b.el8.noa 1.6 MB/s | 80 kB 00:00 2023-10-10T20:32:04.665 INFO:teuthology.orchestra.run.smithi089.stdout:(36/120): ceph-mgr-diskprediction-local-17.2.6- 4.5 MB/s | 7.4 MB 00:01 2023-10-10T20:32:04.680 INFO:teuthology.orchestra.run.smithi089.stdout:(37/120): cephfs-top-17.2.6-1374.g46d1f59b.el8. 443 kB/s | 18 kB 00:00 2023-10-10T20:32:05.167 INFO:teuthology.orchestra.run.smithi089.stdout:(38/120): ceph-test-17.2.6-1374.g46d1f59b.el8.x 9.5 MB/s | 52 MB 00:05 2023-10-10T20:32:05.413 INFO:teuthology.orchestra.run.smithi089.stdout:(39/120): liboath-2.6.2-3.el8.x86_64.rpm 80 kB/s | 59 kB 00:00 2023-10-10T20:32:05.591 INFO:teuthology.orchestra.run.smithi089.stdout:(40/120): libunwind-1.3.1-3.el8.x86_64.rpm 177 kB/s | 75 kB 00:00 2023-10-10T20:32:05.596 INFO:teuthology.orchestra.run.smithi089.stdout:(41/120): gperftools-libs-2.7-9.el8.x86_64.rpm 329 kB/s | 306 kB 00:00 2023-10-10T20:32:05.697 INFO:teuthology.orchestra.run.smithi089.stdout:(42/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 411 kB/s | 44 kB 00:00 2023-10-10T20:32:05.712 INFO:teuthology.orchestra.run.smithi089.stdout:(43/120): python3-beautifulsoup4-4.6.3-2.el8.1. 1.6 MB/s | 185 kB 00:00 2023-10-10T20:32:05.744 INFO:teuthology.orchestra.run.smithi089.stdout:(44/120): python3-asyncssh-2.7.0-2.el8.noarch.r 1.4 MB/s | 476 kB 00:00 2023-10-10T20:32:05.795 INFO:teuthology.orchestra.run.smithi089.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 195 kB/s | 16 kB 00:00 2023-10-10T20:32:05.796 INFO:teuthology.orchestra.run.smithi089.stdout:(46/120): python3-cachetools-3.1.1-4.el8.noarch 335 kB/s | 33 kB 00:00 2023-10-10T20:32:05.800 INFO:teuthology.orchestra.run.smithi111.stdout:(10/120): ceph-osd-17.2.6-1374.g46d1f59b.el8.x8 5.5 MB/s | 17 MB 00:03 2023-10-10T20:32:05.844 INFO:teuthology.orchestra.run.smithi111.stdout:(11/120): cephfs-mirror-17.2.6-1374.g46d1f59b.e 5.7 MB/s | 255 kB 00:00 2023-10-10T20:32:05.845 INFO:teuthology.orchestra.run.smithi089.stdout:(47/120): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2023-10-10T20:32:05.865 INFO:teuthology.orchestra.run.smithi111.stdout:(12/120): libcephfs-devel-17.2.6-1374.g46d1f59b 1.2 MB/s | 25 kB 00:00 2023-10-10T20:32:05.929 INFO:teuthology.orchestra.run.smithi089.stdout:(48/120): python3-google-auth-1.1.1-10.el8.noar 611 kB/s | 81 kB 00:00 2023-10-10T20:32:05.930 INFO:teuthology.orchestra.run.smithi089.stdout:(49/120): python3-jaraco-6.2-6.el8.noarch.rpm 131 kB/s | 11 kB 00:00 2023-10-10T20:32:05.952 INFO:teuthology.orchestra.run.smithi111.stdout:(13/120): libcephfs2-17.2.6-1374.g46d1f59b.el8. 9.0 MB/s | 797 kB 00:00 2023-10-10T20:32:05.974 INFO:teuthology.orchestra.run.smithi111.stdout:(14/120): libcephsqlite-17.2.6-1374.g46d1f59b.e 7.8 MB/s | 174 kB 00:00 2023-10-10T20:32:05.974 INFO:teuthology.orchestra.run.smithi089.stdout:(50/120): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2023-10-10T20:32:05.995 INFO:teuthology.orchestra.run.smithi111.stdout:(15/120): librados-devel-17.2.6-1374.g46d1f59b. 5.9 MB/s | 130 kB 00:00 2023-10-10T20:32:06.022 INFO:teuthology.orchestra.run.smithi089.stdout:(51/120): python3-jaraco-functools-2.0-4.el8.no 191 kB/s | 18 kB 00:00 2023-10-10T20:32:06.058 INFO:teuthology.orchestra.run.smithi111.stdout:(16/120): libradosstriper1-17.2.6-1374.g46d1f59 7.9 MB/s | 507 kB 00:00 2023-10-10T20:32:06.059 INFO:teuthology.orchestra.run.smithi089.stdout:(52/120): python3-logutils-0.3.5-11.el8.noarch. 575 kB/s | 49 kB 00:00 2023-10-10T20:32:06.109 INFO:teuthology.orchestra.run.smithi089.stdout:(53/120): python3-more-itertools-7.2.0-3.el8.no 671 kB/s | 59 kB 00:00 2023-10-10T20:32:06.145 INFO:teuthology.orchestra.run.smithi089.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 703 kB/s | 60 kB 00:00 2023-10-10T20:32:06.223 INFO:teuthology.orchestra.run.smithi089.stdout:(55/120): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2023-10-10T20:32:06.229 INFO:teuthology.orchestra.run.smithi089.stdout:(56/120): python3-portend-2.6-1.el8.noarch.rpm 187 kB/s | 16 kB 00:00 2023-10-10T20:32:06.339 INFO:teuthology.orchestra.run.smithi089.stdout:(57/120): python3-kubernetes-11.0.0-6.el8.noarc 2.8 MB/s | 1.2 MB 00:00 2023-10-10T20:32:06.342 INFO:teuthology.orchestra.run.smithi089.stdout:(58/120): python3-repoze-lru-0.7-6.el8.noarch.r 283 kB/s | 34 kB 00:00 2023-10-10T20:32:06.364 INFO:teuthology.orchestra.run.smithi089.stdout:(59/120): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-10-10T20:32:06.458 INFO:teuthology.orchestra.run.smithi089.stdout:(60/120): python3-rsa-4.9-2.el8.noarch.rpm 540 kB/s | 64 kB 00:00 2023-10-10T20:32:06.464 INFO:teuthology.orchestra.run.smithi089.stdout:(61/120): python3-simplegeneric-0.8.1-17.el8.no 153 kB/s | 19 kB 00:00 2023-10-10T20:32:06.488 INFO:teuthology.orchestra.run.smithi089.stdout:(62/120): python3-singledispatch-3.4.0.3-18.el8 193 kB/s | 24 kB 00:00 2023-10-10T20:32:06.539 INFO:teuthology.orchestra.run.smithi089.stdout:(63/120): python3-tempora-1.14.1-5.el8.noarch.r 365 kB/s | 29 kB 00:00 2023-10-10T20:32:06.579 INFO:teuthology.orchestra.run.smithi089.stdout:(64/120): python3-trustme-0.6.0-4.el8.noarch.rp 241 kB/s | 27 kB 00:00 2023-10-10T20:32:06.582 INFO:teuthology.orchestra.run.smithi111.stdout:(17/120): librgw2-17.2.6-1374.g46d1f59b.el8.x86 11 MB/s | 5.6 MB 00:00 2023-10-10T20:32:06.606 INFO:teuthology.orchestra.run.smithi111.stdout:(18/120): python3-ceph-argparse-17.2.6-1374.g46 1.9 MB/s | 45 kB 00:00 2023-10-10T20:32:06.629 INFO:teuthology.orchestra.run.smithi111.stdout:(19/120): python3-ceph-common-17.2.6-1374.g46d1 5.0 MB/s | 115 kB 00:00 2023-10-10T20:32:06.652 INFO:teuthology.orchestra.run.smithi111.stdout:(20/120): python3-cephfs-17.2.6-1374.g46d1f59b. 8.0 MB/s | 190 kB 00:00 2023-10-10T20:32:06.665 INFO:teuthology.orchestra.run.smithi089.stdout:(65/120): python3-waitress-1.4.3-1.el8.noarch.r 1.4 MB/s | 255 kB 00:00 2023-10-10T20:32:06.695 INFO:teuthology.orchestra.run.smithi111.stdout:(21/120): python3-rados-17.2.6-1374.g46d1f59b.e 8.5 MB/s | 371 kB 00:00 2023-10-10T20:32:06.708 INFO:teuthology.orchestra.run.smithi089.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 475 kB/s | 61 kB 00:00 2023-10-10T20:32:06.738 INFO:teuthology.orchestra.run.smithi111.stdout:(22/120): python3-rbd-17.2.6-1374.g46d1f59b.el8 8.2 MB/s | 358 kB 00:00 2023-10-10T20:32:06.760 INFO:teuthology.orchestra.run.smithi111.stdout:(23/120): python3-rgw-17.2.6-1374.g46d1f59b.el8 4.8 MB/s | 107 kB 00:00 2023-10-10T20:32:06.762 INFO:teuthology.orchestra.run.smithi089.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 880 kB/s | 85 kB 00:00 2023-10-10T20:32:06.782 INFO:teuthology.orchestra.run.smithi111.stdout:(24/120): rbd-fuse-17.2.6-1374.g46d1f59b.el8.x8 4.1 MB/s | 90 kB 00:00 2023-10-10T20:32:06.795 INFO:teuthology.orchestra.run.smithi089.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 264 kB/s | 23 kB 00:00 2023-10-10T20:32:06.800 INFO:teuthology.orchestra.run.smithi089.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 14 MB/s | 70 kB 00:00 2023-10-10T20:32:06.831 INFO:teuthology.orchestra.run.smithi089.stdout:(70/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 601 kB/s | 19 kB 00:00 2023-10-10T20:32:06.847 INFO:teuthology.orchestra.run.smithi089.stdout:(71/120): python3-jwt-1.6.1-2.el8.noarch.rpm 2.7 MB/s | 43 kB 00:00 2023-10-10T20:32:06.854 INFO:teuthology.orchestra.run.smithi089.stdout:(72/120): python3-webob-1.8.5-1.el8.1.noarch.rp 796 kB/s | 251 kB 00:00 2023-10-10T20:32:06.862 INFO:teuthology.orchestra.run.smithi089.stdout:(73/120): python3-oauthlib-2.1.0-1.el8.noarch.r 10 MB/s | 155 kB 00:00 2023-10-10T20:32:06.878 INFO:teuthology.orchestra.run.smithi089.stdout:(74/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 2.9 MB/s | 48 kB 00:00 2023-10-10T20:32:06.898 INFO:teuthology.orchestra.run.smithi089.stdout:(75/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.9 MB/s | 220 kB 00:00 2023-10-10T20:32:06.901 INFO:teuthology.orchestra.run.smithi089.stdout:(76/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 10 MB/s | 229 kB 00:00 2023-10-10T20:32:06.914 INFO:teuthology.orchestra.run.smithi089.stdout:(77/120): python3-requests-oauthlib-1.0.0-1.el8 2.7 MB/s | 43 kB 00:00 2023-10-10T20:32:06.933 INFO:teuthology.orchestra.run.smithi089.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 951 kB/s | 18 kB 00:00 2023-10-10T20:32:06.949 INFO:teuthology.orchestra.run.smithi089.stdout:(79/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 48 MB/s | 2.3 MB 00:00 2023-10-10T20:32:06.957 INFO:teuthology.orchestra.run.smithi089.stdout:(80/120): pcre2-devel-10.32-2.el8.x86_64.rpm 25 MB/s | 605 kB 00:00 2023-10-10T20:32:06.971 INFO:teuthology.orchestra.run.smithi089.stdout:(81/120): fuse-2.9.7-12.el8.x86_64.rpm 3.9 MB/s | 83 kB 00:00 2023-10-10T20:32:06.975 INFO:teuthology.orchestra.run.smithi089.stdout:(82/120): elfutils-libelf-devel-0.182-3.el8.x86 3.2 MB/s | 59 kB 00:00 2023-10-10T20:32:06.994 INFO:teuthology.orchestra.run.smithi089.stdout:(83/120): libquadmath-8.4.1-1.el8.x86_64.rpm 7.3 MB/s | 169 kB 00:00 2023-10-10T20:32:07.003 INFO:teuthology.orchestra.run.smithi089.stdout:(84/120): libcom_err-devel-1.45.6-1.el8.x86_64. 1.4 MB/s | 38 kB 00:00 2023-10-10T20:32:07.014 INFO:teuthology.orchestra.run.smithi089.stdout:(85/120): libsepol-devel-2.9-2.el8.x86_64.rpm 4.1 MB/s | 86 kB 00:00 2023-10-10T20:32:07.026 INFO:teuthology.orchestra.run.smithi089.stdout:(86/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 2.5 MB/s | 58 kB 00:00 2023-10-10T20:32:07.037 INFO:teuthology.orchestra.run.smithi089.stdout:(87/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.0 MB/s | 47 kB 00:00 2023-10-10T20:32:07.054 INFO:teuthology.orchestra.run.smithi089.stdout:(88/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 19 MB/s | 559 kB 00:00 2023-10-10T20:32:07.061 INFO:teuthology.orchestra.run.smithi089.stdout:(89/120): thrift-0.13.0-2.el8.x86_64.rpm 5.8 MB/s | 1.7 MB 00:00 2023-10-10T20:32:07.068 INFO:teuthology.orchestra.run.smithi089.stdout:(90/120): libgfortran-8.4.1-1.el8.x86_64.rpm 20 MB/s | 639 kB 00:00 2023-10-10T20:32:07.071 INFO:teuthology.orchestra.run.smithi089.stdout:(91/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2023-10-10T20:32:07.093 INFO:teuthology.orchestra.run.smithi089.stdout:(92/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 3.9 MB/s | 97 kB 00:00 2023-10-10T20:32:07.098 INFO:teuthology.orchestra.run.smithi111.stdout:(25/120): rbd-mirror-17.2.6-1374.g46d1f59b.el8. 11 MB/s | 3.5 MB 00:00 2023-10-10T20:32:07.117 INFO:teuthology.orchestra.run.smithi089.stdout:(93/120): libselinux-devel-2.9-5.el8.x86_64.rpm 3.5 MB/s | 200 kB 00:00 2023-10-10T20:32:07.120 INFO:teuthology.orchestra.run.smithi111.stdout:(26/120): rbd-nbd-17.2.6-1374.g46d1f59b.el8.x86 7.6 MB/s | 173 kB 00:00 2023-10-10T20:32:07.130 INFO:teuthology.orchestra.run.smithi089.stdout:(94/120): python3-prettytable-0.7.2-14.el8.noar 1.2 MB/s | 44 kB 00:00 2023-10-10T20:32:07.135 INFO:teuthology.orchestra.run.smithi089.stdout:(95/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.4 MB/s | 27 kB 00:00 2023-10-10T20:32:07.141 INFO:teuthology.orchestra.run.smithi111.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1374.g 1.1 MB/s | 23 kB 00:00 2023-10-10T20:32:07.148 INFO:teuthology.orchestra.run.smithi089.stdout:(96/120): jq-1.5-12.el8.x86_64.rpm 8.7 MB/s | 161 kB 00:00 2023-10-10T20:32:07.162 INFO:teuthology.orchestra.run.smithi089.stdout:(97/120): flex-2.6.1-9.el8.x86_64.rpm 11 MB/s | 320 kB 00:00 2023-10-10T20:32:07.163 INFO:teuthology.orchestra.run.smithi111.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1374.g46d1f59 5.5 MB/s | 121 kB 00:00 2023-10-10T20:32:07.177 INFO:teuthology.orchestra.run.smithi089.stdout:(98/120): python3-werkzeug-0.12.2-4.el8.noarch. 15 MB/s | 457 kB 00:00 2023-10-10T20:32:07.194 INFO:teuthology.orchestra.run.smithi089.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 2.3 MB/s | 39 kB 00:00 2023-10-10T20:32:07.218 INFO:teuthology.orchestra.run.smithi089.stdout:(100/120): bison-3.0.4-10.el8.x86_64.rpm 28 MB/s | 688 kB 00:00 2023-10-10T20:32:07.315 INFO:teuthology.orchestra.run.smithi111.stdout:(29/120): ceph-mgr-dashboard-17.2.6-1374.g46d1f 11 MB/s | 1.7 MB 00:00 2023-10-10T20:32:07.336 INFO:teuthology.orchestra.run.smithi089.stdout:(101/120): python3-scipy-1.0.0-20.module+el8.1. 55 MB/s | 15 MB 00:00 2023-10-10T20:32:07.340 INFO:teuthology.orchestra.run.smithi089.stdout:(102/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.1 MB/s | 259 kB 00:00 2023-10-10T20:32:07.358 INFO:teuthology.orchestra.run.smithi089.stdout:(103/120): python3-babel-2.5.1-5.el8.noarch.rpm 24 MB/s | 4.8 MB 00:00 2023-10-10T20:32:07.360 INFO:teuthology.orchestra.run.smithi089.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 6.4 MB/s | 126 kB 00:00 2023-10-10T20:32:07.362 INFO:teuthology.orchestra.run.smithi089.stdout:(105/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 8.8 MB/s | 225 kB 00:00 2023-10-10T20:32:07.382 INFO:teuthology.orchestra.run.smithi089.stdout:(106/120): oniguruma-6.8.2-2.el8.x86_64.rpm 8.3 MB/s | 187 kB 00:00 2023-10-10T20:32:07.389 INFO:teuthology.orchestra.run.smithi089.stdout:(107/120): python3-jinja2-2.10.1-2.el8_0.noarch 19 MB/s | 538 kB 00:00 2023-10-10T20:32:07.401 INFO:teuthology.orchestra.run.smithi089.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 16 MB/s | 302 kB 00:00 2023-10-10T20:32:07.412 INFO:teuthology.orchestra.run.smithi089.stdout:(109/120): python3-mako-1.0.6-13.el8.noarch.rpm 6.7 MB/s | 157 kB 00:00 2023-10-10T20:32:07.424 INFO:teuthology.orchestra.run.smithi089.stdout:(110/120): python3-numpy-1.14.3-9.el8.x86_64.rp 57 MB/s | 3.7 MB 00:00 2023-10-10T20:32:07.426 INFO:teuthology.orchestra.run.smithi089.stdout:(111/120): python3-pyasn1-modules-0.3.7-6.el8.n 4.3 MB/s | 110 kB 00:00 2023-10-10T20:32:07.435 INFO:teuthology.orchestra.run.smithi089.stdout:(112/120): librdkafka-0.11.4-1.el8.x86_64.rpm 15 MB/s | 353 kB 00:00 2023-10-10T20:32:07.445 INFO:teuthology.orchestra.run.smithi089.stdout:(113/120): python3-jsonpointer-1.10-11.el8.noar 987 kB/s | 20 kB 00:00 2023-10-10T20:32:07.452 INFO:teuthology.orchestra.run.smithi089.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 26 MB/s | 709 kB 00:00 2023-10-10T20:32:07.459 INFO:teuthology.orchestra.run.smithi089.stdout:(115/120): python3-pytz-2017.2-9.el8.noarch.rpm 2.3 MB/s | 54 kB 00:00 2023-10-10T20:32:07.493 INFO:teuthology.orchestra.run.smithi089.stdout:(116/120): automake-1.16.1-7.el8.noarch.rpm 21 MB/s | 713 kB 00:00 2023-10-10T20:32:07.529 INFO:teuthology.orchestra.run.smithi089.stdout:(117/120): openblas-threads-0.3.12-1.el8.x86_64 56 MB/s | 4.7 MB 00:00 2023-10-10T20:32:07.547 INFO:teuthology.orchestra.run.smithi089.stdout:(118/120): openblas-0.3.12-1.el8.x86_64.rpm 48 MB/s | 4.6 MB 00:00 2023-10-10T20:32:08.190 INFO:teuthology.orchestra.run.smithi089.stdout:(119/120): librados2-17.2.6-1374.g46d1f59b.el8. 5.0 MB/s | 3.5 MB 00:00 2023-10-10T20:32:08.508 INFO:teuthology.orchestra.run.smithi111.stdout:(30/120): ceph-test-17.2.6-1374.g46d1f59b.el8.x 11 MB/s | 52 MB 00:04 2023-10-10T20:32:08.530 INFO:teuthology.orchestra.run.smithi111.stdout:(31/120): ceph-mgr-diskprediction-local-17.2.6- 6.1 MB/s | 7.4 MB 00:01 2023-10-10T20:32:08.553 INFO:teuthology.orchestra.run.smithi111.stdout:(32/120): ceph-mgr-modules-core-17.2.6-1374.g46 5.2 MB/s | 238 kB 00:00 2023-10-10T20:32:08.560 INFO:teuthology.orchestra.run.smithi111.stdout:(33/120): ceph-mgr-rook-17.2.6-1374.g46d1f59b.e 1.6 MB/s | 47 kB 00:00 2023-10-10T20:32:08.598 INFO:teuthology.orchestra.run.smithi111.stdout:(34/120): ceph-prometheus-alerts-17.2.6-1374.g4 327 kB/s | 14 kB 00:00 2023-10-10T20:32:08.623 INFO:teuthology.orchestra.run.smithi111.stdout:(35/120): cephadm-17.2.6-1374.g46d1f59b.el8.noa 3.1 MB/s | 80 kB 00:00 2023-10-10T20:32:08.631 INFO:teuthology.orchestra.run.smithi111.stdout:(36/120): ceph-volume-17.2.6-1374.g46d1f59b.el8 3.7 MB/s | 267 kB 00:00 2023-10-10T20:32:08.645 INFO:teuthology.orchestra.run.smithi111.stdout:(37/120): cephfs-top-17.2.6-1374.g46d1f59b.el8. 855 kB/s | 18 kB 00:00 2023-10-10T20:32:08.755 INFO:teuthology.orchestra.run.smithi089.stdout:(120/120): librbd1-17.2.6-1374.g46d1f59b.el8.x8 2.9 MB/s | 3.6 MB 00:01 2023-10-10T20:32:08.760 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-10-10T20:32:08.761 INFO:teuthology.orchestra.run.smithi089.stdout:Total 17 MB/s | 202 MB 00:11 2023-10-10T20:32:08.793 INFO:teuthology.orchestra.run.smithi111.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 397 kB/s | 59 kB 00:00 2023-10-10T20:32:08.807 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-10-10T20:32:08.877 INFO:teuthology.orchestra.run.smithi111.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 898 kB/s | 75 kB 00:00 2023-10-10T20:32:08.919 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-10-10T20:32:08.919 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-10-10T20:32:09.035 INFO:teuthology.orchestra.run.smithi111.stdout:(40/120): gperftools-libs-2.7-9.el8.x86_64.rpm 758 kB/s | 306 kB 00:00 2023-10-10T20:32:09.086 INFO:teuthology.orchestra.run.smithi111.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 863 kB/s | 44 kB 00:00 2023-10-10T20:32:09.270 INFO:teuthology.orchestra.run.smithi111.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 1.0 MB/s | 185 kB 00:00 2023-10-10T20:32:09.304 INFO:teuthology.orchestra.run.smithi111.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 974 kB/s | 33 kB 00:00 2023-10-10T20:32:09.321 INFO:teuthology.orchestra.run.smithi111.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 953 kB/s | 16 kB 00:00 2023-10-10T20:32:09.327 INFO:teuthology.orchestra.run.smithi111.stdout:(45/120): python3-asyncssh-2.7.0-2.el8.noarch.r 1.0 MB/s | 476 kB 00:00 2023-10-10T20:32:09.472 INFO:teuthology.orchestra.run.smithi111.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 1.1 MB/s | 173 kB 00:00 2023-10-10T20:32:09.540 INFO:teuthology.orchestra.run.smithi111.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2023-10-10T20:32:09.557 INFO:teuthology.orchestra.run.smithi111.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 644 kB/s | 11 kB 00:00 2023-10-10T20:32:09.575 INFO:teuthology.orchestra.run.smithi111.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 1.0 MB/s | 18 kB 00:00 2023-10-10T20:32:09.611 INFO:teuthology.orchestra.run.smithi111.stdout:(50/120): python3-cherrypy-18.4.0-1.el8.noarch. 1.3 MB/s | 384 kB 00:00 2023-10-10T20:32:09.645 INFO:teuthology.orchestra.run.smithi111.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 1.4 MB/s | 49 kB 00:00 2023-10-10T20:32:09.696 INFO:teuthology.orchestra.run.smithi111.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2023-10-10T20:32:09.746 INFO:teuthology.orchestra.run.smithi111.stdout:(53/120): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-10-10T20:32:09.931 INFO:teuthology.orchestra.run.smithi111.stdout:(54/120): python3-pecan-1.3.2-9.el8.noarch.rpm 1.5 MB/s | 283 kB 00:00 2023-10-10T20:32:09.948 INFO:teuthology.orchestra.run.smithi111.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 914 kB/s | 16 kB 00:00 2023-10-10T20:32:09.982 INFO:teuthology.orchestra.run.smithi111.stdout:(56/120): python3-repoze-lru-0.7-6.el8.noarch.r 993 kB/s | 34 kB 00:00 2023-10-10T20:32:10.099 INFO:teuthology.orchestra.run.smithi111.stdout:(57/120): python3-routes-2.4.1-12.el8.noarch.rp 1.6 MB/s | 196 kB 00:00 2023-10-10T20:32:10.150 INFO:teuthology.orchestra.run.smithi111.stdout:(58/120): python3-rsa-4.9-2.el8.noarch.rpm 1.2 MB/s | 64 kB 00:00 2023-10-10T20:32:10.168 INFO:teuthology.orchestra.run.smithi111.stdout:(59/120): python3-simplegeneric-0.8.1-17.el8.no 1.1 MB/s | 19 kB 00:00 2023-10-10T20:32:10.185 INFO:teuthology.orchestra.run.smithi111.stdout:(60/120): python3-singledispatch-3.4.0.3-18.el8 1.3 MB/s | 24 kB 00:00 2023-10-10T20:32:10.203 INFO:teuthology.orchestra.run.smithi111.stdout:(61/120): python3-tempora-1.14.1-5.el8.noarch.r 1.7 MB/s | 29 kB 00:00 2023-10-10T20:32:10.221 INFO:teuthology.orchestra.run.smithi111.stdout:(62/120): python3-trustme-0.6.0-4.el8.noarch.rp 1.5 MB/s | 27 kB 00:00 2023-10-10T20:32:10.362 INFO:teuthology.orchestra.run.smithi111.stdout:(63/120): python3-kubernetes-11.0.0-6.el8.noarc 1.5 MB/s | 1.2 MB 00:00 2023-10-10T20:32:10.447 INFO:teuthology.orchestra.run.smithi111.stdout:(64/120): python3-waitress-1.4.3-1.el8.noarch.r 1.1 MB/s | 255 kB 00:00 2023-10-10T20:32:10.546 INFO:teuthology.orchestra.run.smithi111.stdout:(65/120): python3-websocket-client-0.56.0-5.el8 623 kB/s | 61 kB 00:00 2023-10-10T20:32:10.618 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-10-10T20:32:10.623 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-10-10T20:32:10.627 INFO:teuthology.orchestra.run.smithi111.stdout:(66/120): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2023-10-10T20:32:10.645 INFO:teuthology.orchestra.run.smithi111.stdout:(67/120): python3-zc-lockfile-2.0-2.el8.noarch. 1.2 MB/s | 23 kB 00:00 2023-10-10T20:32:10.651 INFO:teuthology.orchestra.run.smithi111.stdout:(68/120): python3-webob-1.8.5-1.el8.1.noarch.rp 870 kB/s | 251 kB 00:00 2023-10-10T20:32:10.656 INFO:teuthology.orchestra.run.smithi111.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 14 MB/s | 70 kB 00:00 2023-10-10T20:32:10.689 INFO:teuthology.orchestra.run.smithi111.stdout:(70/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 564 kB/s | 19 kB 00:00 2023-10-10T20:32:10.703 INFO:teuthology.orchestra.run.smithi111.stdout:(71/120): python3-jwt-1.6.1-2.el8.noarch.rpm 3.0 MB/s | 43 kB 00:00 2023-10-10T20:32:10.718 INFO:teuthology.orchestra.run.smithi111.stdout:(72/120): python3-oauthlib-2.1.0-1.el8.noarch.r 11 MB/s | 155 kB 00:00 2023-10-10T20:32:10.737 INFO:teuthology.orchestra.run.smithi111.stdout:(73/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 11 MB/s | 220 kB 00:00 2023-10-10T20:32:10.751 INFO:teuthology.orchestra.run.smithi111.stdout:(74/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 3.4 MB/s | 48 kB 00:00 2023-10-10T20:32:10.768 INFO:teuthology.orchestra.run.smithi111.stdout:(75/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 13 MB/s | 229 kB 00:00 2023-10-10T20:32:10.782 INFO:teuthology.orchestra.run.smithi111.stdout:(76/120): python3-requests-oauthlib-1.0.0-1.el8 3.1 MB/s | 43 kB 00:00 2023-10-10T20:32:10.823 INFO:teuthology.orchestra.run.smithi111.stdout:(77/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 58 MB/s | 2.3 MB 00:00 2023-10-10T20:32:10.836 INFO:teuthology.orchestra.run.smithi111.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.3 MB/s | 18 kB 00:00 2023-10-10T20:32:10.856 INFO:teuthology.orchestra.run.smithi111.stdout:(79/120): pcre2-devel-10.32-2.el8.x86_64.rpm 31 MB/s | 605 kB 00:00 2023-10-10T20:32:10.871 INFO:teuthology.orchestra.run.smithi111.stdout:(80/120): fuse-2.9.7-12.el8.x86_64.rpm 5.5 MB/s | 83 kB 00:00 2023-10-10T20:32:10.885 INFO:teuthology.orchestra.run.smithi111.stdout:(81/120): elfutils-libelf-devel-0.182-3.el8.x86 4.1 MB/s | 59 kB 00:00 2023-10-10T20:32:10.901 INFO:teuthology.orchestra.run.smithi111.stdout:(82/120): libquadmath-8.4.1-1.el8.x86_64.rpm 10 MB/s | 169 kB 00:00 2023-10-10T20:32:10.916 INFO:teuthology.orchestra.run.smithi111.stdout:(83/120): libcom_err-devel-1.45.6-1.el8.x86_64. 2.5 MB/s | 38 kB 00:00 2023-10-10T20:32:10.934 INFO:teuthology.orchestra.run.smithi111.stdout:(84/120): libsepol-devel-2.9-2.el8.x86_64.rpm 4.9 MB/s | 86 kB 00:00 2023-10-10T20:32:10.949 INFO:teuthology.orchestra.run.smithi111.stdout:(85/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 3.6 MB/s | 58 kB 00:00 2023-10-10T20:32:10.965 INFO:teuthology.orchestra.run.smithi111.stdout:(86/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.0 MB/s | 47 kB 00:00 2023-10-10T20:32:10.983 INFO:teuthology.orchestra.run.smithi111.stdout:(87/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 30 MB/s | 559 kB 00:00 2023-10-10T20:32:11.004 INFO:teuthology.orchestra.run.smithi111.stdout:(88/120): libgfortran-8.4.1-1.el8.x86_64.rpm 31 MB/s | 639 kB 00:00 2023-10-10T20:32:11.019 INFO:teuthology.orchestra.run.smithi111.stdout:(89/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 4.7 MB/s | 70 kB 00:00 2023-10-10T20:32:11.035 INFO:teuthology.orchestra.run.smithi111.stdout:(90/120): libselinux-devel-2.9-5.el8.x86_64.rpm 12 MB/s | 200 kB 00:00 2023-10-10T20:32:11.051 INFO:teuthology.orchestra.run.smithi111.stdout:(91/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 6.1 MB/s | 97 kB 00:00 2023-10-10T20:32:11.293 INFO:teuthology.orchestra.run.smithi111.stdout:(92/120): python3-scipy-1.0.0-20.module+el8.1.0 60 MB/s | 15 MB 00:00 2023-10-10T20:32:11.308 INFO:teuthology.orchestra.run.smithi111.stdout:(93/120): python3-prettytable-0.7.2-14.el8.noar 2.9 MB/s | 44 kB 00:00 2023-10-10T20:32:11.321 INFO:teuthology.orchestra.run.smithi111.stdout:(94/120): python3-jsonpatch-1.21-2.el8.noarch.r 2.0 MB/s | 27 kB 00:00 2023-10-10T20:32:11.335 INFO:teuthology.orchestra.run.smithi111.stdout:(95/120): jq-1.5-12.el8.x86_64.rpm 11 MB/s | 161 kB 00:00 2023-10-10T20:32:11.351 INFO:teuthology.orchestra.run.smithi111.stdout:(96/120): flex-2.6.1-9.el8.x86_64.rpm 20 MB/s | 320 kB 00:00 2023-10-10T20:32:11.369 INFO:teuthology.orchestra.run.smithi111.stdout:(97/120): python3-werkzeug-0.12.2-4.el8.noarch. 25 MB/s | 457 kB 00:00 2023-10-10T20:32:11.432 INFO:teuthology.orchestra.run.smithi111.stdout:(98/120): python3-babel-2.5.1-5.el8.noarch.rpm 76 MB/s | 4.8 MB 00:00 2023-10-10T20:32:11.448 INFO:teuthology.orchestra.run.smithi111.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 2.5 MB/s | 39 kB 00:00 2023-10-10T20:32:11.467 INFO:teuthology.orchestra.run.smithi111.stdout:(100/120): bison-3.0.4-10.el8.x86_64.rpm 36 MB/s | 688 kB 00:00 2023-10-10T20:32:11.482 INFO:teuthology.orchestra.run.smithi111.stdout:(101/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 17 MB/s | 259 kB 00:00 2023-10-10T20:32:11.497 INFO:teuthology.orchestra.run.smithi111.stdout:(102/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 16 MB/s | 225 kB 00:00 2023-10-10T20:32:11.510 INFO:teuthology.orchestra.run.smithi111.stdout:(103/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 9.7 MB/s | 126 kB 00:00 2023-10-10T20:32:11.539 INFO:teuthology.orchestra.run.smithi111.stdout:(104/120): thrift-0.13.0-2.el8.x86_64.rpm 1.9 MB/s | 1.7 MB 00:00 2023-10-10T20:32:11.571 INFO:teuthology.orchestra.run.smithi111.stdout:(105/120): python3-numpy-1.14.3-9.el8.x86_64.rp 61 MB/s | 3.7 MB 00:00 2023-10-10T20:32:11.573 INFO:teuthology.orchestra.run.smithi111.stdout:(106/120): oniguruma-6.8.2-2.el8.x86_64.rpm 5.4 MB/s | 187 kB 00:00 2023-10-10T20:32:11.590 INFO:teuthology.orchestra.run.smithi111.stdout:(107/120): python3-jinja2-2.10.1-2.el8_0.noarch 28 MB/s | 538 kB 00:00 2023-10-10T20:32:11.591 INFO:teuthology.orchestra.run.smithi111.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 16 MB/s | 302 kB 00:00 2023-10-10T20:32:11.605 INFO:teuthology.orchestra.run.smithi111.stdout:(109/120): python3-pyasn1-modules-0.3.7-6.el8.n 7.8 MB/s | 110 kB 00:00 2023-10-10T20:32:11.607 INFO:teuthology.orchestra.run.smithi111.stdout:(110/120): python3-mako-1.0.6-13.el8.noarch.rpm 9.1 MB/s | 157 kB 00:00 2023-10-10T20:32:11.621 INFO:teuthology.orchestra.run.smithi111.stdout:(111/120): python3-jsonpointer-1.10-11.el8.noar 1.4 MB/s | 20 kB 00:00 2023-10-10T20:32:11.622 INFO:teuthology.orchestra.run.smithi111.stdout:(112/120): librdkafka-0.11.4-1.el8.x86_64.rpm 20 MB/s | 353 kB 00:00 2023-10-10T20:32:11.637 INFO:teuthology.orchestra.run.smithi111.stdout:(113/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.8 MB/s | 54 kB 00:00 2023-10-10T20:32:11.642 INFO:teuthology.orchestra.run.smithi111.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 33 MB/s | 709 kB 00:00 2023-10-10T20:32:11.757 INFO:teuthology.orchestra.run.smithi111.stdout:(115/120): ceph-radosgw-17.2.6-1374.g46d1f59b.e 1.9 MB/s | 17 MB 00:08 2023-10-10T20:32:11.795 INFO:teuthology.orchestra.run.smithi111.stdout:(116/120): automake-1.16.1-7.el8.noarch.rpm 19 MB/s | 713 kB 00:00 2023-10-10T20:32:11.827 INFO:teuthology.orchestra.run.smithi111.stdout:(117/120): openblas-0.3.12-1.el8.x86_64.rpm 25 MB/s | 4.6 MB 00:00 2023-10-10T20:32:11.841 INFO:teuthology.orchestra.run.smithi111.stdout:(118/120): openblas-threads-0.3.12-1.el8.x86_64 23 MB/s | 4.7 MB 00:00 2023-10-10T20:32:12.164 INFO:teuthology.orchestra.run.smithi111.stdout:(119/120): librados2-17.2.6-1374.g46d1f59b.el8. 9.4 MB/s | 3.5 MB 00:00 2023-10-10T20:32:12.765 INFO:teuthology.orchestra.run.smithi111.stdout:(120/120): librbd1-17.2.6-1374.g46d1f59b.el8.x8 3.8 MB/s | 3.6 MB 00:00 2023-10-10T20:32:12.770 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2023-10-10T20:32:12.770 INFO:teuthology.orchestra.run.smithi111.stdout:Total 14 MB/s | 202 MB 00:14 2023-10-10T20:32:12.816 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2023-10-10T20:32:12.929 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2023-10-10T20:32:12.929 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2023-10-10T20:32:14.651 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2023-10-10T20:32:14.655 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2023-10-10T20:32:16.281 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-10-10T20:32:16.524 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-10T20:32:16.751 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-10T20:32:16.887 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-10T20:32:17.150 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-10T20:32:17.540 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-10T20:32:17.834 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librados2-2:17.2.6-1374.g46d1f59b.el8.x86_64 4/122 2023-10-10T20:32:18.221 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:17.2.6-1374.g46d1f59b.el8.x86_64 4/122 2023-10-10T20:32:18.467 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librbd1-2:17.2.6-1374.g46d1f59b.el8.x86_64 5/122 2023-10-10T20:32:18.611 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:17.2.6-1374.g46d1f59b.el8.x86_64 5/122 2023-10-10T20:32:18.817 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs2-2:17.2.6-1374.g46d1f59b.el8.x86_64 6/122 2023-10-10T20:32:19.476 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephfs2-2:17.2.6-1374.g46d1f59b.el8.x86_64 6/122 2023-10-10T20:32:19.572 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2023-10-10T20:32:19.714 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rados-2:17.2.6-1374.g46d1f59b.el8.x86_64 7/122 2023-10-10T20:32:19.827 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-10T20:32:20.045 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-10T20:32:20.058 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-10-10T20:32:20.211 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-10T20:32:20.428 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-10T20:32:20.743 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-10T20:32:20.798 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-10T20:32:20.950 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librgw2-2:17.2.6-1374.g46d1f59b.el8.x86_64 10/122 2023-10-10T20:32:21.037 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librados2-2:17.2.6-1374.g46d1f59b.el8.x86_64 4/122 2023-10-10T20:32:21.066 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librgw2-2:17.2.6-1374.g46d1f59b.el8.x86_64 10/122 2023-10-10T20:32:21.321 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-10T20:32:21.416 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:17.2.6-1374.g46d1f59b.el8.x86_64 4/122 2023-10-10T20:32:21.507 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-10T20:32:21.670 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librbd1-2:17.2.6-1374.g46d1f59b.el8.x86_64 5/122 2023-10-10T20:32:21.767 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephsqlite-2:17.2.6-1374.g46d1f59b.el8.x86_64 13/122 2023-10-10T20:32:21.782 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:17.2.6-1374.g46d1f59b.el8.x86_64 5/122 2023-10-10T20:32:21.894 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1374.g46d1f59b.el8.x86_64 13/122 2023-10-10T20:32:21.978 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs2-2:17.2.6-1374.g46d1f59b.el8.x86_64 6/122 2023-10-10T20:32:22.084 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libradosstriper1-2:17.2.6-1374.g46d1f59b.el8.x86 14/122 2023-10-10T20:32:22.110 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephfs2-2:17.2.6-1374.g46d1f59b.el8.x86_64 6/122 2023-10-10T20:32:22.169 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1374.g46d1f59b.el8.x86 14/122 2023-10-10T20:32:22.325 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rados-2:17.2.6-1374.g46d1f59b.el8.x86_64 7/122 2023-10-10T20:32:22.363 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-10-10T20:32:22.655 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-10-10T20:32:22.658 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-10-10T20:32:22.907 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-10-10T20:32:23.093 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-10-10T20:32:23.225 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-10T20:32:23.316 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-10T20:32:23.398 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-10T20:32:23.503 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librgw2-2:17.2.6-1374.g46d1f59b.el8.x86_64 10/122 2023-10-10T20:32:23.600 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-10T20:32:23.617 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librgw2-2:17.2.6-1374.g46d1f59b.el8.x86_64 10/122 2023-10-10T20:32:23.873 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-10T20:32:24.068 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-10T20:32:24.146 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-10T20:32:24.303 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephsqlite-2:17.2.6-1374.g46d1f59b.el8.x86_64 13/122 2023-10-10T20:32:24.333 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-10T20:32:24.409 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1374.g46d1f59b.el8.x86_64 13/122 2023-10-10T20:32:24.892 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-10T20:32:24.931 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libradosstriper1-2:17.2.6-1374.g46d1f59b.el8.x86 14/122 2023-10-10T20:32:25.099 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-10-10T20:32:25.475 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1374.g46d1f59b.el8.x86 14/122 2023-10-10T20:32:25.686 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-10-10T20:32:25.823 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-10-10T20:32:25.985 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-10-10T20:32:26.111 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-10-10T20:32:26.235 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-10-10T20:32:26.391 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-10-10T20:32:26.427 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-10-10T20:32:26.561 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-10T20:32:26.660 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-10T20:32:26.729 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-10T20:32:26.856 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-common-2:17.2.6-1374.g46d1f59b.el8. 26/122 2023-10-10T20:32:26.911 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-10T20:32:27.027 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-argparse-2:17.2.6-1374.g46d1f59b.el 27/122 2023-10-10T20:32:27.314 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cephfs-2:17.2.6-1374.g46d1f59b.el8.x86_6 28/122 2023-10-10T20:32:27.451 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-10T20:32:27.580 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-10-10T20:32:27.611 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-10T20:32:27.921 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-10-10T20:32:28.121 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-10T20:32:28.182 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-10T20:32:28.346 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-10-10T20:32:29.058 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-10-10T20:32:29.306 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-10-10T20:32:29.561 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-10-10T20:32:29.855 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-10T20:32:30.034 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-common-2:17.2.6-1374.g46d1f59b.el8. 26/122 2023-10-10T20:32:30.212 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-argparse-2:17.2.6-1374.g46d1f59b.el 27/122 2023-10-10T20:32:30.246 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-10T20:32:30.485 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cephfs-2:17.2.6-1374.g46d1f59b.el8.x86_6 28/122 2023-10-10T20:32:30.664 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-10-10T20:32:30.775 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-10-10T20:32:31.077 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-10-10T20:32:31.352 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-10T20:32:31.717 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-10-10T20:32:32.080 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-10-10T20:32:32.360 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-10-10T20:32:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-10T20:32:32.850 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-10T20:32:33.083 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 38/122 2023-10-10T20:32:33.303 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 39/122 2023-10-10T20:32:33.422 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-10T20:32:33.590 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librados-devel-2:17.2.6-1374.g46d1f59b.el8.x86_6 40/122 2023-10-10T20:32:33.782 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-10-10T20:32:33.847 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-10-10T20:32:34.065 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-10-10T20:32:34.287 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-10-10T20:32:34.510 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-10-10T20:32:34.698 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-10T20:32:34.780 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-10T20:32:34.836 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-10-10T20:32:34.981 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-10-10T20:32:35.183 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-10-10T20:32:35.197 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-10-10T20:32:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-10-10T20:32:35.479 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-10-10T20:32:35.602 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-10-10T20:32:35.748 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-10T20:32:35.800 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-10-10T20:32:35.958 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-10-10T20:32:35.963 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-10T20:32:36.143 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-10-10T20:32:36.194 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 38/122 2023-10-10T20:32:36.374 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-10-10T20:32:36.406 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 39/122 2023-10-10T20:32:36.559 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-10-10T20:32:36.669 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librados-devel-2:17.2.6-1374.g46d1f59b.el8.x86_6 40/122 2023-10-10T20:32:36.883 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-10-10T20:32:36.889 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-10-10T20:32:37.100 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-10-10T20:32:37.169 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-10-10T20:32:37.348 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-10-10T20:32:37.497 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-10-10T20:32:37.555 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-10-10T20:32:37.709 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-10T20:32:37.795 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-10T20:32:37.802 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-10-10T20:32:37.951 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-10-10T20:32:37.998 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-10T20:32:38.161 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-10-10T20:32:38.203 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-10T20:32:38.382 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-10-10T20:32:38.486 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-10T20:32:38.548 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-10-10T20:32:38.731 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-10T20:32:38.744 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-10-10T20:32:38.936 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-10-10T20:32:38.963 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-10T20:32:39.079 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-10-10T20:32:39.159 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-10T20:32:39.299 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-10-10T20:32:39.341 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-10T20:32:39.495 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-10-10T20:32:39.523 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-10T20:32:39.734 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-10T20:32:39.818 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-10-10T20:32:40.034 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-10T20:32:40.099 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-10-10T20:32:40.267 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-10T20:32:40.395 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-10-10T20:32:40.420 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-10T20:32:40.667 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-10T20:32:40.724 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-10-10T20:32:40.901 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-10T20:32:40.925 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-10T20:32:41.090 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-10T20:32:41.161 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-10T20:32:41.376 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-10T20:32:41.466 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-10T20:32:41.609 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-10T20:32:41.754 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-10T20:32:41.833 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-10T20:32:41.979 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-10T20:32:42.035 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-10T20:32:42.239 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-10T20:32:42.253 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-10T20:32:42.442 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-10T20:32:42.685 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-10T20:32:42.925 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-10T20:32:43.162 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-10T20:32:43.366 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-10T20:32:43.430 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-10T20:32:43.642 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-10T20:32:43.794 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-10T20:32:43.924 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-10T20:32:44.040 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-10T20:32:44.174 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-10T20:32:44.424 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-10T20:32:44.500 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-10T20:32:44.717 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-10T20:32:44.801 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-10T20:32:45.019 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-10T20:32:45.088 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-10T20:32:45.308 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-10T20:32:45.331 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-10T20:32:45.556 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-10T20:32:45.847 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-10T20:32:46.498 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-10T20:32:46.831 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-10T20:32:47.077 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-10T20:32:47.444 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-10T20:32:47.734 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-10T20:32:48.091 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-10T20:32:48.346 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-10T20:32:48.575 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-10T20:32:48.808 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-10T20:32:50.970 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 87/122 2023-10-10T20:32:51.221 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 87/122 2023-10-10T20:32:51.812 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 87/122 2023-10-10T20:32:52.154 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-base-2:17.2.6-1374.g46d1f59b.el8.x86_64 88/122 2023-10-10T20:32:52.250 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-base-2:17.2.6-1374.g46d1f59b.el8.x86_64 88/122 2023-10-10T20:32:52.431 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-selinux-2:17.2.6-1374.g46d1f59b.el8.x86_64 89/122 2023-10-10T20:32:54.172 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 87/122 2023-10-10T20:32:54.390 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 87/122 2023-10-10T20:32:54.978 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 87/122 2023-10-10T20:32:55.278 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-base-2:17.2.6-1374.g46d1f59b.el8.x86_64 88/122 2023-10-10T20:32:55.349 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-base-2:17.2.6-1374.g46d1f59b.el8.x86_64 88/122 2023-10-10T20:32:55.493 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-selinux-2:17.2.6-1374.g46d1f59b.el8.x86_64 89/122 2023-10-10T20:33:10.265 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1374.g46d1f59b.el8.x86_64 89/122 2023-10-10T20:33:10.265 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /sys 2023-10-10T20:33:10.265 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /proc 2023-10-10T20:33:10.265 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /dev 2023-10-10T20:33:10.266 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /run 2023-10-10T20:33:10.266 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /mnt 2023-10-10T20:33:10.266 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /var/tmp 2023-10-10T20:33:10.266 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /home 2023-10-10T20:33:10.266 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /tmp 2023-10-10T20:33:10.267 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /dev 2023-10-10T20:33:10.267 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:10.726 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-volume-2:17.2.6-1374.g46d1f59b.el8.noarch 90/122 2023-10-10T20:33:12.221 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-volume-2:17.2.6-1374.g46d1f59b.el8.noarch 90/122 2023-10-10T20:33:12.401 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-osd-2:17.2.6-1374.g46d1f59b.el8.x86_64 91/122 2023-10-10T20:33:12.622 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-osd-2:17.2.6-1374.g46d1f59b.el8.x86_64 91/122 2023-10-10T20:33:12.809 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mds-2:17.2.6-1374.g46d1f59b.el8.x86_64 92/122 2023-10-10T20:33:13.100 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1374.g46d1f59b.el8.x86_64 89/122 2023-10-10T20:33:13.100 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /sys 2023-10-10T20:33:13.100 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /proc 2023-10-10T20:33:13.101 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /dev 2023-10-10T20:33:13.101 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /run 2023-10-10T20:33:13.101 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /mnt 2023-10-10T20:33:13.101 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /var/tmp 2023-10-10T20:33:13.101 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /home 2023-10-10T20:33:13.102 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /tmp 2023-10-10T20:33:13.102 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /dev 2023-10-10T20:33:13.102 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:13.200 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mds-2:17.2.6-1374.g46d1f59b.el8.x86_64 92/122 2023-10-10T20:33:13.342 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mon-2:17.2.6-1374.g46d1f59b.el8.x86_64 93/122 2023-10-10T20:33:13.359 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mon-2:17.2.6-1374.g46d1f59b.el8.x86_64 93/122 2023-10-10T20:33:13.505 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-volume-2:17.2.6-1374.g46d1f59b.el8.noarch 90/122 2023-10-10T20:33:14.991 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-volume-2:17.2.6-1374.g46d1f59b.el8.noarch 90/122 2023-10-10T20:33:15.183 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-osd-2:17.2.6-1374.g46d1f59b.el8.x86_64 91/122 2023-10-10T20:33:15.404 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-osd-2:17.2.6-1374.g46d1f59b.el8.x86_64 91/122 2023-10-10T20:33:15.579 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mds-2:17.2.6-1374.g46d1f59b.el8.x86_64 92/122 2023-10-10T20:33:15.969 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mds-2:17.2.6-1374.g46d1f59b.el8.x86_64 92/122 2023-10-10T20:33:16.170 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mon-2:17.2.6-1374.g46d1f59b.el8.x86_64 93/122 2023-10-10T20:33:16.188 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mon-2:17.2.6-1374.g46d1f59b.el8.x86_64 93/122 2023-10-10T20:33:16.301 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 94/122 2023-10-10T20:33:16.453 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 94/122 2023-10-10T20:33:16.692 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1374.g46d1f59b.e 95/122 2023-10-10T20:33:16.907 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1374.g46d1f59b. 96/122 2023-10-10T20:33:17.109 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1374.g46d1f59b.el8.noa 97/122 2023-10-10T20:33:17.325 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1374.g46d1f59b.el8.noa 97/122 2023-10-10T20:33:17.583 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1374.g46d1f59b.el8.n 98/122 2023-10-10T20:33:18.553 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1374.g46d1f59b.el8.n 98/122 2023-10-10T20:33:18.733 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1374.g46d 99/122 2023-10-10T20:33:18.832 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1374.g46d 99/122 2023-10-10T20:33:18.993 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 94/122 2023-10-10T20:33:19.211 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 94/122 2023-10-10T20:33:19.230 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1374.g46d1f59b.el 100/122 2023-10-10T20:33:19.417 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1374.g46d1f59b.e 95/122 2023-10-10T20:33:19.441 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-2:17.2.6-1374.g46d1f59b.el8.x86_64 101/122 2023-10-10T20:33:19.479 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1374.g46d1f59b.el8.x86_64 101/122 2023-10-10T20:33:19.612 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1374.g46d1f59b. 96/122 2023-10-10T20:33:19.682 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-rook-2:17.2.6-1374.g46d1f59b.el8.noarch 102/122 2023-10-10T20:33:19.694 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1374.g46d1f59b.el8.noarch 102/122 2023-10-10T20:33:19.786 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1374.g46d1f59b.el8.noa 97/122 2023-10-10T20:33:20.000 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1374.g46d1f59b.el8.noa 97/122 2023-10-10T20:33:20.232 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1374.g46d1f59b.el8.n 98/122 2023-10-10T20:33:21.200 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1374.g46d1f59b.el8.n 98/122 2023-10-10T20:33:21.305 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-2:17.2.6-1374.g46d1f59b.el8.x86_64 103/122 2023-10-10T20:33:21.402 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1374.g46d 99/122 2023-10-10T20:33:21.499 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1374.g46d 99/122 2023-10-10T20:33:21.558 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-radosgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 104/122 2023-10-10T20:33:21.638 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 104/122 2023-10-10T20:33:21.808 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1374.g46d1f 105/122 2023-10-10T20:33:21.846 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1374.g46d1f59b.el 100/122 2023-10-10T20:33:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1374.g46d1f 105/122 2023-10-10T20:33:22.043 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 106/122 2023-10-10T20:33:22.057 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-2:17.2.6-1374.g46d1f59b.el8.x86_64 101/122 2023-10-10T20:33:22.095 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1374.g46d1f59b.el8.x86_64 101/122 2023-10-10T20:33:22.260 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-rook-2:17.2.6-1374.g46d1f59b.el8.noarch 102/122 2023-10-10T20:33:22.272 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1374.g46d1f59b.el8.noarch 102/122 2023-10-10T20:33:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 106/122 2023-10-10T20:33:22.591 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 107/122 2023-10-10T20:33:23.893 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-2:17.2.6-1374.g46d1f59b.el8.x86_64 103/122 2023-10-10T20:33:24.116 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-radosgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 104/122 2023-10-10T20:33:24.235 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 104/122 2023-10-10T20:33:24.386 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1374.g46d1f 105/122 2023-10-10T20:33:24.428 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1374.g46d1f 105/122 2023-10-10T20:33:24.616 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 106/122 2023-10-10T20:33:24.963 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 106/122 2023-10-10T20:33:25.127 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 107/122 2023-10-10T20:33:27.589 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 107/122 2023-10-10T20:33:28.550 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-test-2:17.2.6-1374.g46d1f59b.el8.x86_64 108/122 2023-10-10T20:33:29.025 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-10-10T20:33:29.235 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 110/122 2023-10-10T20:33:29.497 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-10-10T20:33:29.706 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-10-10T20:33:29.718 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-10-10T20:33:29.911 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs-devel-2:17.2.6-1374.g46d1f59b.el8.x86_ 113/122 2023-10-10T20:33:30.052 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 107/122 2023-10-10T20:33:30.053 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-10T20:33:30.239 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-top-2:17.2.6-1374.g46d1f59b.el8.noarch 115/122 2023-10-10T20:33:30.439 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 116/122 2023-10-10T20:33:30.736 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-nbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 117/122 2023-10-10T20:33:30.956 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-10-10T20:33:31.004 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-test-2:17.2.6-1374.g46d1f59b.el8.x86_64 108/122 2023-10-10T20:33:31.012 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-10-10T20:33:31.264 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-10-10T20:33:31.273 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-10-10T20:33:31.377 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-10-10T20:33:31.408 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-10-10T20:33:31.408 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-10T20:33:31.578 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 110/122 2023-10-10T20:33:31.642 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-10T20:33:31.642 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-10T20:33:31.825 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-10-10T20:33:32.017 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-10-10T20:33:32.028 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-10-10T20:33:32.247 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs-devel-2:17.2.6-1374.g46d1f59b.el8.x86_ 113/122 2023-10-10T20:33:32.389 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-10T20:33:32.543 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-10T20:33:32.544 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-2:17.2.6-1374.g46d1f59b.el8.x86_64 1/122 2023-10-10T20:33:32.544 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-base-2:17.2.6-1374.g46d1f59b.el8.x86_64 2/122 2023-10-10T20:33:32.544 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 3/122 2023-10-10T20:33:32.544 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 4/122 2023-10-10T20:33:32.544 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1374.g46d1f 5/122 2023-10-10T20:33:32.545 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mds-2:17.2.6-1374.g46d1f59b.el8.x86_64 6/122 2023-10-10T20:33:32.545 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-2:17.2.6-1374.g46d1f59b.el8.x86_64 7/122 2023-10-10T20:33:32.545 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mon-2:17.2.6-1374.g46d1f59b.el8.x86_64 8/122 2023-10-10T20:33:32.545 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-osd-2:17.2.6-1374.g46d1f59b.el8.x86_64 9/122 2023-10-10T20:33:32.546 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-radosgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 10/122 2023-10-10T20:33:32.546 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-selinux-2:17.2.6-1374.g46d1f59b.el8.x86_64 11/122 2023-10-10T20:33:32.546 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-test-2:17.2.6-1374.g46d1f59b.el8.x86_64 12/122 2023-10-10T20:33:32.546 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 13/122 2023-10-10T20:33:32.546 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs-devel-2:17.2.6-1374.g46d1f59b.el8.x86_ 14/122 2023-10-10T20:33:32.547 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs2-2:17.2.6-1374.g46d1f59b.el8.x86_64 15/122 2023-10-10T20:33:32.547 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephsqlite-2:17.2.6-1374.g46d1f59b.el8.x86_64 16/122 2023-10-10T20:33:32.547 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados-devel-2:17.2.6-1374.g46d1f59b.el8.x86_6 17/122 2023-10-10T20:33:32.548 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libradosstriper1-2:17.2.6-1374.g46d1f59b.el8.x86 18/122 2023-10-10T20:33:32.548 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librgw2-2:17.2.6-1374.g46d1f59b.el8.x86_64 19/122 2023-10-10T20:33:32.548 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1374.g46d1f59b.el 20/122 2023-10-10T20:33:32.548 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-common-2:17.2.6-1374.g46d1f59b.el8. 21/122 2023-10-10T20:33:32.549 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cephfs-2:17.2.6-1374.g46d1f59b.el8.x86_6 22/122 2023-10-10T20:33:32.549 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rados-2:17.2.6-1374.g46d1f59b.el8.x86_64 23/122 2023-10-10T20:33:32.549 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 24/122 2023-10-10T20:33:32.549 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 25/122 2023-10-10T20:33:32.549 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 26/122 2023-10-10T20:33:32.550 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 27/122 2023-10-10T20:33:32.550 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-nbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 28/122 2023-10-10T20:33:32.550 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1374.g46d1f59b. 29/122 2023-10-10T20:33:32.550 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1374.g46d1f59b.el8.noa 30/122 2023-10-10T20:33:32.551 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1374.g46d1f59b.el8.n 31/122 2023-10-10T20:33:32.561 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1374.g46d 32/122 2023-10-10T20:33:32.562 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1374.g46d1f59b.el 33/122 2023-10-10T20:33:32.562 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1374.g46d1f59b.el8.noarch 34/122 2023-10-10T20:33:32.562 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1374.g46d1f59b.e 35/122 2023-10-10T20:33:32.563 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-volume-2:17.2.6-1374.g46d1f59b.el8.noarch 36/122 2023-10-10T20:33:32.563 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 37/122 2023-10-10T20:33:32.563 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-top-2:17.2.6-1374.g46d1f59b.el8.noarch 38/122 2023-10-10T20:33:32.563 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-10T20:33:32.564 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-10T20:33:32.564 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-10T20:33:32.564 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-10T20:33:32.564 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-10T20:33:32.565 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-10T20:33:32.565 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-10T20:33:32.565 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-10T20:33:32.566 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-10T20:33:32.566 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-10T20:33:32.566 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-10T20:33:32.566 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-10T20:33:32.567 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-10T20:33:32.567 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-10T20:33:32.567 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-10T20:33:32.568 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-10T20:33:32.568 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-10T20:33:32.568 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-10T20:33:32.568 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-10T20:33:32.569 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-10T20:33:32.569 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-10T20:33:32.569 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-10T20:33:32.569 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-10T20:33:32.570 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-10T20:33:32.570 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-10T20:33:32.570 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-10T20:33:32.570 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-10T20:33:32.570 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-10T20:33:32.571 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-10T20:33:32.571 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-10T20:33:32.571 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-10T20:33:32.571 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-10T20:33:32.572 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-10T20:33:32.572 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-10-10T20:33:32.572 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-10-10T20:33:32.572 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-10-10T20:33:32.572 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-10-10T20:33:32.573 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-10-10T20:33:32.573 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-10-10T20:33:32.573 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-10-10T20:33:32.573 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-10-10T20:33:32.573 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-10-10T20:33:32.574 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-10-10T20:33:32.574 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-10-10T20:33:32.574 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-10-10T20:33:32.574 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-10-10T20:33:32.575 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-10-10T20:33:32.575 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-10-10T20:33:32.575 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-10-10T20:33:32.575 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-10-10T20:33:32.575 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-10-10T20:33:32.576 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-10-10T20:33:32.576 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-10-10T20:33:32.576 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-10-10T20:33:32.576 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-10-10T20:33:32.576 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-10-10T20:33:32.577 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-10-10T20:33:32.577 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-10-10T20:33:32.577 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-10-10T20:33:32.577 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-10-10T20:33:32.577 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-10-10T20:33:32.578 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-10-10T20:33:32.578 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-10-10T20:33:32.578 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-10-10T20:33:32.578 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-10-10T20:33:32.578 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-10-10T20:33:32.579 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-10-10T20:33:32.579 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-10-10T20:33:32.579 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-10-10T20:33:32.579 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-10-10T20:33:32.580 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-10-10T20:33:32.580 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-10-10T20:33:32.580 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-10-10T20:33:32.580 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-10-10T20:33:32.580 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-10-10T20:33:32.581 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-10-10T20:33:32.581 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-10-10T20:33:32.581 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-10-10T20:33:32.581 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-10-10T20:33:32.581 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-10-10T20:33:32.582 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:17.2.6-1374.g46d1f59b.el8.x86_64 119/122 2023-10-10T20:33:32.582 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-10T20:33:32.582 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:17.2.6-1374.g46d1f59b.el8.x86_64 121/122 2023-10-10T20:33:32.592 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-top-2:17.2.6-1374.g46d1f59b.el8.noarch 115/122 2023-10-10T20:33:32.800 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 116/122 2023-10-10T20:33:33.053 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-nbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 117/122 2023-10-10T20:33:33.267 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-10-10T20:33:33.330 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-10-10T20:33:33.501 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-10-10T20:33:33.509 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-10-10T20:33:33.661 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-10-10T20:33:33.662 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-10T20:33:33.895 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-10T20:33:33.895 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-10T20:33:34.774 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-10T20:33:34.774 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-2:17.2.6-1374.g46d1f59b.el8.x86_64 1/122 2023-10-10T20:33:34.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-base-2:17.2.6-1374.g46d1f59b.el8.x86_64 2/122 2023-10-10T20:33:34.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 3/122 2023-10-10T20:33:34.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 4/122 2023-10-10T20:33:34.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1374.g46d1f 5/122 2023-10-10T20:33:34.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mds-2:17.2.6-1374.g46d1f59b.el8.x86_64 6/122 2023-10-10T20:33:34.776 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-2:17.2.6-1374.g46d1f59b.el8.x86_64 7/122 2023-10-10T20:33:34.776 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mon-2:17.2.6-1374.g46d1f59b.el8.x86_64 8/122 2023-10-10T20:33:34.776 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-osd-2:17.2.6-1374.g46d1f59b.el8.x86_64 9/122 2023-10-10T20:33:34.776 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-radosgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 10/122 2023-10-10T20:33:34.776 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-selinux-2:17.2.6-1374.g46d1f59b.el8.x86_64 11/122 2023-10-10T20:33:34.777 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-test-2:17.2.6-1374.g46d1f59b.el8.x86_64 12/122 2023-10-10T20:33:34.777 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 13/122 2023-10-10T20:33:34.777 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs-devel-2:17.2.6-1374.g46d1f59b.el8.x86_ 14/122 2023-10-10T20:33:34.777 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs2-2:17.2.6-1374.g46d1f59b.el8.x86_64 15/122 2023-10-10T20:33:34.778 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephsqlite-2:17.2.6-1374.g46d1f59b.el8.x86_64 16/122 2023-10-10T20:33:34.778 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados-devel-2:17.2.6-1374.g46d1f59b.el8.x86_6 17/122 2023-10-10T20:33:34.778 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libradosstriper1-2:17.2.6-1374.g46d1f59b.el8.x86 18/122 2023-10-10T20:33:34.778 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librgw2-2:17.2.6-1374.g46d1f59b.el8.x86_64 19/122 2023-10-10T20:33:34.778 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1374.g46d1f59b.el 20/122 2023-10-10T20:33:34.779 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-common-2:17.2.6-1374.g46d1f59b.el8. 21/122 2023-10-10T20:33:34.779 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cephfs-2:17.2.6-1374.g46d1f59b.el8.x86_6 22/122 2023-10-10T20:33:34.779 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rados-2:17.2.6-1374.g46d1f59b.el8.x86_64 23/122 2023-10-10T20:33:34.779 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 24/122 2023-10-10T20:33:34.779 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 25/122 2023-10-10T20:33:34.780 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 26/122 2023-10-10T20:33:34.780 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 27/122 2023-10-10T20:33:34.780 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-nbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 28/122 2023-10-10T20:33:34.780 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1374.g46d1f59b. 29/122 2023-10-10T20:33:34.780 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1374.g46d1f59b.el8.noa 30/122 2023-10-10T20:33:34.781 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1374.g46d1f59b.el8.n 31/122 2023-10-10T20:33:34.789 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1374.g46d 32/122 2023-10-10T20:33:34.789 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1374.g46d1f59b.el 33/122 2023-10-10T20:33:34.789 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1374.g46d1f59b.el8.noarch 34/122 2023-10-10T20:33:34.789 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1374.g46d1f59b.e 35/122 2023-10-10T20:33:34.790 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-volume-2:17.2.6-1374.g46d1f59b.el8.noarch 36/122 2023-10-10T20:33:34.790 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 37/122 2023-10-10T20:33:34.790 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-top-2:17.2.6-1374.g46d1f59b.el8.noarch 38/122 2023-10-10T20:33:34.790 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-10T20:33:34.791 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-10T20:33:34.791 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-10T20:33:34.791 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-10T20:33:34.791 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-10T20:33:34.791 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-10T20:33:34.792 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-10T20:33:34.792 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-10T20:33:34.792 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-10T20:33:34.792 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-10T20:33:34.792 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-10T20:33:34.793 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-10T20:33:34.793 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-10T20:33:34.793 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-10T20:33:34.793 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-10T20:33:34.793 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-10T20:33:34.794 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-10T20:33:34.794 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-10T20:33:34.794 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-10T20:33:34.794 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-10T20:33:34.794 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-10T20:33:34.794 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-10T20:33:34.795 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-10T20:33:34.795 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-10T20:33:34.795 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-10T20:33:34.795 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-10T20:33:34.795 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-10T20:33:34.796 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-10T20:33:34.796 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-10T20:33:34.796 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-10T20:33:34.796 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-10T20:33:34.796 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-10T20:33:34.797 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-10T20:33:34.797 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-10-10T20:33:34.797 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-10-10T20:33:34.797 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-10-10T20:33:34.797 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-10-10T20:33:34.798 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-10-10T20:33:34.798 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-10-10T20:33:34.798 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-10-10T20:33:34.798 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-10-10T20:33:34.799 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-10-10T20:33:34.799 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-10-10T20:33:34.799 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-10-10T20:33:34.799 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-10-10T20:33:34.799 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-10-10T20:33:34.800 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-10-10T20:33:34.800 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-10-10T20:33:34.800 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-10-10T20:33:34.800 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-10-10T20:33:34.800 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-10-10T20:33:34.800 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-10-10T20:33:34.801 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-10-10T20:33:34.801 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-10-10T20:33:34.801 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-10-10T20:33:34.801 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-10-10T20:33:34.801 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-10-10T20:33:34.802 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-10-10T20:33:34.802 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-10-10T20:33:34.802 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-10-10T20:33:34.802 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-10-10T20:33:34.802 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-10-10T20:33:34.803 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-10-10T20:33:34.803 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-10-10T20:33:34.803 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-10-10T20:33:34.803 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-10-10T20:33:34.803 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-10-10T20:33:34.804 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-10-10T20:33:34.804 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-10-10T20:33:34.804 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-10-10T20:33:34.804 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-10-10T20:33:34.804 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-10-10T20:33:34.805 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-10-10T20:33:34.805 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-10-10T20:33:34.805 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-10-10T20:33:34.805 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-10-10T20:33:34.805 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-10-10T20:33:34.806 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-10-10T20:33:34.806 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-10-10T20:33:34.806 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-10-10T20:33:34.806 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:17.2.6-1374.g46d1f59b.el8.x86_64 119/122 2023-10-10T20:33:34.806 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-10T20:33:34.806 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:17.2.6-1374.g46d1f59b.el8.x86_64 121/122 2023-10-10T20:33:34.850 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-10T20:33:34.850 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:46 ago on Tue 10 Oct 2023 08:31:48 PM UTC. 2023-10-10T20:33:37.065 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-10T20:33:37.065 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:01:48 ago on Tue 10 Oct 2023 08:31:49 PM UTC. 2023-10-10T20:33:37.136 INFO:teuthology.orchestra.run.smithi089.stdout:Installed products updated. 2023-10-10T20:33:37.287 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:37.287 INFO:teuthology.orchestra.run.smithi089.stdout:Upgraded: 2023-10-10T20:33:37.287 INFO:teuthology.orchestra.run.smithi089.stdout: librados2-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.288 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.288 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-10-10T20:33:37.288 INFO:teuthology.orchestra.run.smithi089.stdout: automake-1.16.1-7.el8.noarch 2023-10-10T20:33:37.288 INFO:teuthology.orchestra.run.smithi089.stdout: bison-3.0.4-10.el8.x86_64 2023-10-10T20:33:37.288 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.288 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.289 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.289 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.289 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:37.289 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.289 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.290 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.290 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:37.290 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:37.290 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:37.290 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:37.291 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:37.291 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.291 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.291 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:37.291 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.291 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.292 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.292 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:37.292 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:37.292 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.292 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:37.293 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-10T20:33:37.293 INFO:teuthology.orchestra.run.smithi089.stdout: flex-2.6.1-9.el8.x86_64 2023-10-10T20:33:37.293 INFO:teuthology.orchestra.run.smithi089.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-10T20:33:37.293 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-10T20:33:37.293 INFO:teuthology.orchestra.run.smithi089.stdout: jq-1.5-12.el8.x86_64 2023-10-10T20:33:37.293 INFO:teuthology.orchestra.run.smithi089.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-10T20:33:37.294 INFO:teuthology.orchestra.run.smithi089.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-10T20:33:37.294 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-10T20:33:37.294 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.294 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.294 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.295 INFO:teuthology.orchestra.run.smithi089.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-10T20:33:37.295 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-10T20:33:37.295 INFO:teuthology.orchestra.run.smithi089.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-10T20:33:37.295 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-10T20:33:37.295 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-10T20:33:37.295 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.296 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.296 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-10T20:33:37.296 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.297 INFO:teuthology.orchestra.run.smithi089.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-10T20:33:37.297 INFO:teuthology.orchestra.run.smithi089.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-10T20:33:37.297 INFO:teuthology.orchestra.run.smithi089.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-10T20:33:37.297 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-10T20:33:37.298 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-10T20:33:37.298 INFO:teuthology.orchestra.run.smithi089.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-10T20:33:37.298 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-10T20:33:37.298 INFO:teuthology.orchestra.run.smithi089.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-10T20:33:37.298 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-10T20:33:37.299 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-10T20:33:37.299 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-10T20:33:37.299 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-10T20:33:37.299 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-10T20:33:37.299 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-10T20:33:37.299 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-10T20:33:37.300 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-10T20:33:37.300 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-10T20:33:37.300 INFO:teuthology.orchestra.run.smithi089.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-10T20:33:37.300 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-10T20:33:37.300 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.301 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.301 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.301 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-10T20:33:37.301 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-10T20:33:37.301 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-10T20:33:37.301 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-10T20:33:37.302 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-10T20:33:37.302 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-10T20:33:37.302 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-10T20:33:37.302 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-10T20:33:37.302 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-10T20:33:37.303 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-10T20:33:37.303 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-10T20:33:37.303 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-10T20:33:37.303 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-10T20:33:37.303 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-10T20:33:37.304 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-10T20:33:37.304 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-10T20:33:37.304 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-10T20:33:37.304 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-10T20:33:37.304 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-10T20:33:37.304 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-10T20:33:37.305 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend-2.6-1.el8.noarch 2023-10-10T20:33:37.305 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-10T20:33:37.305 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-10T20:33:37.305 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-10T20:33:37.305 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-10T20:33:37.305 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.306 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.306 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-10T20:33:37.306 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-10T20:33:37.306 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.306 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-10T20:33:37.307 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-10T20:33:37.307 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-10T20:33:37.307 INFO:teuthology.orchestra.run.smithi089.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-10T20:33:37.307 INFO:teuthology.orchestra.run.smithi089.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-10T20:33:37.307 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-10T20:33:37.307 INFO:teuthology.orchestra.run.smithi089.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-10T20:33:37.308 INFO:teuthology.orchestra.run.smithi089.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-10T20:33:37.308 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-10T20:33:37.308 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-10T20:33:37.308 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-10T20:33:37.308 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-10T20:33:37.309 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-10T20:33:37.309 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.309 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.309 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:37.309 INFO:teuthology.orchestra.run.smithi089.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-10T20:33:37.309 INFO:teuthology.orchestra.run.smithi089.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-10T20:33:37.310 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-10T20:33:37.310 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-10T20:33:37.310 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-10T20:33:37.311 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:37.311 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-10-10T20:33:37.595 DEBUG:teuthology.parallel:result is None 2023-10-10T20:33:39.218 INFO:teuthology.orchestra.run.smithi111.stdout:Installed products updated. 2023-10-10T20:33:39.371 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:39.371 INFO:teuthology.orchestra.run.smithi111.stdout:Upgraded: 2023-10-10T20:33:39.371 INFO:teuthology.orchestra.run.smithi111.stdout: librados2-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.372 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.372 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2023-10-10T20:33:39.372 INFO:teuthology.orchestra.run.smithi111.stdout: automake-1.16.1-7.el8.noarch 2023-10-10T20:33:39.372 INFO:teuthology.orchestra.run.smithi111.stdout: bison-3.0.4-10.el8.x86_64 2023-10-10T20:33:39.372 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.373 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.373 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.373 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.373 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:39.373 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.374 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.374 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.374 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:39.374 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:39.374 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:39.375 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:39.375 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:39.375 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.375 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.375 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:39.376 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.376 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.376 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.376 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:39.376 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:39.377 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.377 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:33:39.377 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-10T20:33:39.377 INFO:teuthology.orchestra.run.smithi111.stdout: flex-2.6.1-9.el8.x86_64 2023-10-10T20:33:39.377 INFO:teuthology.orchestra.run.smithi111.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-10T20:33:39.378 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-10T20:33:39.378 INFO:teuthology.orchestra.run.smithi111.stdout: jq-1.5-12.el8.x86_64 2023-10-10T20:33:39.378 INFO:teuthology.orchestra.run.smithi111.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-10T20:33:39.378 INFO:teuthology.orchestra.run.smithi111.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-10T20:33:39.378 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-10T20:33:39.379 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.379 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.379 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.379 INFO:teuthology.orchestra.run.smithi111.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-10T20:33:39.379 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-10T20:33:39.380 INFO:teuthology.orchestra.run.smithi111.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-10T20:33:39.380 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-10T20:33:39.380 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-10T20:33:39.380 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.381 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.381 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-10T20:33:39.381 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.381 INFO:teuthology.orchestra.run.smithi111.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-10T20:33:39.382 INFO:teuthology.orchestra.run.smithi111.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-10T20:33:39.382 INFO:teuthology.orchestra.run.smithi111.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-10T20:33:39.382 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-10T20:33:39.382 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-10T20:33:39.382 INFO:teuthology.orchestra.run.smithi111.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-10T20:33:39.383 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-10T20:33:39.383 INFO:teuthology.orchestra.run.smithi111.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-10T20:33:39.383 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-10T20:33:39.383 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-10T20:33:39.383 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-10T20:33:39.384 INFO:teuthology.orchestra.run.smithi111.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-10T20:33:39.385 INFO:teuthology.orchestra.run.smithi111.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-10T20:33:39.385 INFO:teuthology.orchestra.run.smithi111.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-10T20:33:39.385 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-10T20:33:39.385 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-10T20:33:39.386 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-10T20:33:39.386 INFO:teuthology.orchestra.run.smithi111.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-10T20:33:39.386 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-10T20:33:39.386 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.387 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.387 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.387 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-10T20:33:39.387 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-10T20:33:39.387 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-10T20:33:39.388 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-10T20:33:39.388 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-10T20:33:39.388 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-10T20:33:39.388 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-10T20:33:39.388 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-10T20:33:39.389 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-10T20:33:39.389 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-10T20:33:39.389 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-10T20:33:39.389 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-10T20:33:39.389 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-10T20:33:39.390 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-10T20:33:39.390 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-10T20:33:39.390 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-10T20:33:39.390 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-10T20:33:39.390 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-10T20:33:39.391 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-10T20:33:39.391 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-10T20:33:39.391 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend-2.6-1.el8.noarch 2023-10-10T20:33:39.391 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-10T20:33:39.391 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-10T20:33:39.392 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-10T20:33:39.392 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-10T20:33:39.392 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.392 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.392 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-10T20:33:39.393 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-10T20:33:39.393 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.393 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-10T20:33:39.393 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-10T20:33:39.393 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-10T20:33:39.394 INFO:teuthology.orchestra.run.smithi111.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-10T20:33:39.394 INFO:teuthology.orchestra.run.smithi111.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-10T20:33:39.394 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-10T20:33:39.394 INFO:teuthology.orchestra.run.smithi111.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-10T20:33:39.394 INFO:teuthology.orchestra.run.smithi111.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-10T20:33:39.395 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-10T20:33:39.395 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-10T20:33:39.395 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-10T20:33:39.395 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-10T20:33:39.395 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-10T20:33:39.396 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.396 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.396 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:33:39.396 INFO:teuthology.orchestra.run.smithi111.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-10T20:33:39.396 INFO:teuthology.orchestra.run.smithi111.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-10T20:33:39.396 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-10T20:33:39.397 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-10T20:33:39.397 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-10T20:33:39.397 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:39.398 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2023-10-10T20:33:39.662 DEBUG:teuthology.parallel:result is None 2023-10-10T20:33:39.662 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=46d1f59b549730ba7f93d940a3d32b94c45a4cf7 2023-10-10T20:33:39.806 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-10T20:33:39.830 INFO:teuthology.orchestra.run.smithi089.stdout:17.2.6-1374.g46d1f59b.el8 2023-10-10T20:33:39.831 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1374.g46d1f59b.el8 2023-10-10T20:33:39.831 INFO:teuthology.task.install:The correct ceph version 17.2.6-1374.g46d1f59b is installed. 2023-10-10T20:33:39.832 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=46d1f59b549730ba7f93d940a3d32b94c45a4cf7 2023-10-10T20:33:39.974 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-10T20:33:39.995 INFO:teuthology.orchestra.run.smithi111.stdout:17.2.6-1374.g46d1f59b.el8 2023-10-10T20:33:39.995 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1374.g46d1f59b.el8 2023-10-10T20:33:39.996 INFO:teuthology.task.install:The correct ceph version 17.2.6-1374.g46d1f59b is installed. 2023-10-10T20:33:39.996 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-10T20:33:39.997 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:33:39.997 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-10T20:33:40.039 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:33:40.039 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-10T20:33:40.078 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-10T20:33:40.079 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:33:40.079 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/daemon-helper 2023-10-10T20:33:40.106 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-10T20:33:40.171 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:33:40.171 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/daemon-helper 2023-10-10T20:33:40.196 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-10T20:33:40.260 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-10T20:33:40.260 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:33:40.261 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-10T20:33:40.286 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-10T20:33:40.350 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:33:40.351 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-10T20:33:40.375 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-10T20:33:40.440 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-10T20:33:40.441 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:33:40.441 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/stdin-killer 2023-10-10T20:33:40.466 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-10T20:33:40.531 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:33:40.531 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/stdin-killer 2023-10-10T20:33:40.555 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-10T20:33:40.620 INFO:teuthology.run_tasks:Running task ceph... 2023-10-10T20:33:40.685 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-10T20:33:40.686 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 777 /var/log/ceph 2023-10-10T20:33:40.689 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 777 /var/log/ceph 2023-10-10T20:33:40.713 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-10T20:33:40.713 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-10T20:33:40.753 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-10T20:33:40.777 INFO:tasks.ceph:Creating extra log directories... 2023-10-10T20:33:40.777 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-10T20:33:40.817 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-10T20:33:40.851 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-10T20:33:40.851 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-10T20:33:40.851 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-10T20:33:40.852 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:33:40.852 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-10T20:33:40.893 DEBUG:teuthology.orchestra.run.smithi089:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-10T20:33:41.000 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-10T20:33:41.025 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:33:41.025 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-10T20:33:41.040 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-10T20:33:41.153 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-10T20:33:41.197 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-10T20:33:41.197 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-10T20:33:41.197 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420483', 'branch': 'wip-yuri3-testing-2023-10-10-0720-quincy', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7420483', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri3-testing-2023-10-10-0720-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri3-testing-2023-10-10-0720-quincy', 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 2048, 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri3-testing-2023-10-10-0720-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7', 'targets': {'smithi089.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJqEHRxhZBHHfulI3XiAdJH7q11m7AkzDAEdNXI7VmObkwOr+dTXboTv+DOxgrtcmZ2PPEDCKxBrxaTcKUU2tNc=', 'smithi111.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG/JCitYYyitn5+62d+6A84Re3tVWjw2tqhRfcK06JGJAtIASTmZNz8vrxZTRHo19Z0g5EBRLNvJ6vhqtJpUAU8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestVolumes', 'tasks.cephfs.test_volumes.TestSubvolumeGroups', 'tasks.cephfs.test_volumes.TestSubvolumes']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-10_18:16:52', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-10T20:33:41.198 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-10T20:33:41.201 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-10T20:33:41.245 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m0777 -- /var/run/ceph 2023-10-10T20:33:41.255 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m0777 -- /var/run/ceph 2023-10-10T20:33:41.309 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:33:41.310 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/scratch_devs of=/dev/stdout 2023-10-10T20:33:41.331 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-10T20:33:41.332 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_1 2023-10-10T20:33:41.387 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-10T20:33:41.387 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-10T20:33:41.387 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 222456 Links: 1 2023-10-10T20:33:41.388 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-10T20:33:41.388 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-10-10T20:33:41.388 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-10-10 20:33:32.126384986 +0000 2023-10-10T20:33:41.388 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-10-10 20:30:57.889213358 +0000 2023-10-10T20:33:41.388 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-10-10 20:30:57.889213358 +0000 2023-10-10T20:33:41.388 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-10-10T20:33:41.389 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-10T20:33:41.450 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-10-10T20:33:41.450 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-10-10T20:33:41.450 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000253436 s, 2.0 MB/s 2023-10-10T20:33:41.451 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-10T20:33:41.507 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_2 2023-10-10T20:33:41.563 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-10T20:33:41.563 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-10T20:33:41.563 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 220568 Links: 1 2023-10-10T20:33:41.564 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-10T20:33:41.564 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-10-10T20:33:41.564 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-10-10 20:33:32.126384986 +0000 2023-10-10T20:33:41.564 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-10-10 20:30:58.388202168 +0000 2023-10-10T20:33:41.564 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-10-10 20:30:58.388202168 +0000 2023-10-10T20:33:41.564 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-10-10T20:33:41.565 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-10T20:33:41.625 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-10-10T20:33:41.626 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-10-10T20:33:41.626 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.00022793 s, 2.2 MB/s 2023-10-10T20:33:41.627 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-10T20:33:41.682 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_3 2023-10-10T20:33:41.736 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-10T20:33:41.736 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-10T20:33:41.737 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 220683 Links: 1 2023-10-10T20:33:41.737 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-10T20:33:41.737 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-10-10T20:33:41.737 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-10-10 20:33:32.126384986 +0000 2023-10-10T20:33:41.737 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-10-10 20:30:58.865191471 +0000 2023-10-10T20:33:41.738 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-10-10 20:30:58.865191471 +0000 2023-10-10T20:33:41.738 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-10-10T20:33:41.738 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-10T20:33:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-10-10T20:33:41.800 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-10-10T20:33:41.800 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000222836 s, 2.3 MB/s 2023-10-10T20:33:41.801 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-10T20:33:41.856 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_4 2023-10-10T20:33:41.911 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-10T20:33:41.911 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-10T20:33:41.911 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 222590 Links: 1 2023-10-10T20:33:41.912 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-10T20:33:41.912 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-10-10T20:33:41.912 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-10-10 20:33:32.126384986 +0000 2023-10-10T20:33:41.912 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-10-10 20:30:59.365180258 +0000 2023-10-10T20:33:41.912 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-10-10 20:30:59.365180258 +0000 2023-10-10T20:33:41.913 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-10-10T20:33:41.913 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-10T20:33:41.973 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-10-10T20:33:41.974 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-10-10T20:33:41.974 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000261769 s, 2.0 MB/s 2023-10-10T20:33:41.975 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-10T20:33:42.030 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-10T20:33:42.030 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:33:42.031 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/scratch_devs of=/dev/stdout 2023-10-10T20:33:42.046 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-10T20:33:42.047 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_1 2023-10-10T20:33:42.101 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-10T20:33:42.102 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-10T20:33:42.102 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 213527 Links: 1 2023-10-10T20:33:42.102 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-10T20:33:42.102 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2023-10-10T20:33:42.102 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2023-10-10 20:33:34.337243975 +0000 2023-10-10T20:33:42.103 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2023-10-10 20:31:03.196078743 +0000 2023-10-10T20:33:42.103 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2023-10-10 20:31:03.196078743 +0000 2023-10-10T20:33:42.103 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: - 2023-10-10T20:33:42.103 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-10T20:33:42.165 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2023-10-10T20:33:42.166 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2023-10-10T20:33:42.166 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000322888 s, 1.6 MB/s 2023-10-10T20:33:42.167 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-10T20:33:42.223 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_2 2023-10-10T20:33:42.278 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-10T20:33:42.278 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-10T20:33:42.278 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 220016 Links: 1 2023-10-10T20:33:42.279 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-10T20:33:42.279 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2023-10-10T20:33:42.279 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2023-10-10 20:33:34.337243975 +0000 2023-10-10T20:33:42.279 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2023-10-10 20:31:03.704070079 +0000 2023-10-10T20:33:42.279 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2023-10-10 20:31:03.704070079 +0000 2023-10-10T20:33:42.280 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: - 2023-10-10T20:33:42.280 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-10T20:33:42.341 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2023-10-10T20:33:42.341 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2023-10-10T20:33:42.341 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000288877 s, 1.8 MB/s 2023-10-10T20:33:42.342 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-10T20:33:42.398 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_3 2023-10-10T20:33:42.453 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-10T20:33:42.453 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-10T20:33:42.453 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 216899 Links: 1 2023-10-10T20:33:42.454 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-10T20:33:42.454 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2023-10-10T20:33:42.454 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2023-10-10 20:33:34.337243975 +0000 2023-10-10T20:33:42.454 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2023-10-10 20:31:04.229061126 +0000 2023-10-10T20:33:42.454 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2023-10-10 20:31:04.229061126 +0000 2023-10-10T20:33:42.455 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: - 2023-10-10T20:33:42.455 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-10T20:33:42.516 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2023-10-10T20:33:42.516 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2023-10-10T20:33:42.517 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000224645 s, 2.3 MB/s 2023-10-10T20:33:42.518 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-10T20:33:42.574 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_4 2023-10-10T20:33:42.628 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-10T20:33:42.629 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-10T20:33:42.629 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 216971 Links: 1 2023-10-10T20:33:42.629 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-10T20:33:42.629 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2023-10-10T20:33:42.630 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2023-10-10 20:33:34.337243975 +0000 2023-10-10T20:33:42.630 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2023-10-10 20:31:04.721052734 +0000 2023-10-10T20:33:42.630 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2023-10-10 20:31:04.721052734 +0000 2023-10-10T20:33:42.630 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: - 2023-10-10T20:33:42.631 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-10T20:33:42.692 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2023-10-10T20:33:42.692 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2023-10-10T20:33:42.692 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000185612 s, 2.8 MB/s 2023-10-10T20:33:42.693 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-10T20:33:42.748 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-10T20:33:42.748 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi089.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi111.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-10T20:33:42.749 INFO:tasks.ceph:Generating config... 2023-10-10T20:33:42.750 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-10T20:33:42.750 INFO:tasks.ceph:[client] debug client = 20 2023-10-10T20:33:42.751 INFO:tasks.ceph:[client] debug ms = 1 2023-10-10T20:33:42.751 INFO:tasks.ceph:[client] pid file = 2023-10-10T20:33:42.751 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-10T20:33:42.751 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-10T20:33:42.752 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-10T20:33:42.752 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-10T20:33:42.752 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-10T20:33:42.752 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-10T20:33:42.753 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-10T20:33:42.753 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-10T20:33:42.753 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-10T20:33:42.753 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-10T20:33:42.754 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-10T20:33:42.754 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-10T20:33:42.754 INFO:tasks.ceph:[mgr] debug client = 20 2023-10-10T20:33:42.754 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-10-10T20:33:42.755 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-10T20:33:42.755 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-10T20:33:42.755 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-10T20:33:42.755 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-10T20:33:42.756 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-10T20:33:42.756 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-10T20:33:42.756 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-10T20:33:42.756 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-10T20:33:42.757 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-10T20:33:42.757 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-10T20:33:42.757 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-10T20:33:42.757 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-10T20:33:42.758 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-10T20:33:42.758 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-10T20:33:42.758 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-10T20:33:42.758 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-10T20:33:42.759 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-10T20:33:42.759 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-10T20:33:42.759 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-10T20:33:42.759 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-10T20:33:42.760 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-10T20:33:42.760 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-10T20:33:42.760 INFO:tasks.ceph:Setting up mon.a... 2023-10-10T20:33:42.761 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-10T20:33:42.815 INFO:teuthology.orchestra.run.smithi089.stdout:creating /etc/ceph/ceph.keyring 2023-10-10T20:33:42.818 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-10T20:33:42.857 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-10T20:33:42.921 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.89'), ('mon.b', '172.21.15.111'), ('mon.c', '[v2:172.21.15.111:3301,v1:172.21.15.111:6790]')] 2023-10-10T20:33:42.922 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.89,172.21.15.111,[v2:172.21.15.111:3301,v1:172.21.15.111:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-10T20:33:42.923 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:33:42.923 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-10T20:33:42.975 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.89 --add b 172.21.15.111 --addv c '[v2:172.21.15.111:3301,v1:172.21.15.111:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:33:43.057 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:33:43.057 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool: generated fsid 5dd675b8-9026-4d8a-a1bf-62fdc5e7de67 2023-10-10T20:33:43.058 INFO:teuthology.orchestra.run.smithi089.stdout:setting min_mon_release = octopus 2023-10-10T20:33:43.058 INFO:teuthology.orchestra.run.smithi089.stdout:epoch 0 2023-10-10T20:33:43.058 INFO:teuthology.orchestra.run.smithi089.stdout:fsid 5dd675b8-9026-4d8a-a1bf-62fdc5e7de67 2023-10-10T20:33:43.058 INFO:teuthology.orchestra.run.smithi089.stdout:last_changed 2023-10-10T20:33:43.056359+0000 2023-10-10T20:33:43.058 INFO:teuthology.orchestra.run.smithi089.stdout:created 2023-10-10T20:33:43.056359+0000 2023-10-10T20:33:43.059 INFO:teuthology.orchestra.run.smithi089.stdout:min_mon_release 15 (octopus) 2023-10-10T20:33:43.059 INFO:teuthology.orchestra.run.smithi089.stdout:election_strategy: 1 2023-10-10T20:33:43.059 INFO:teuthology.orchestra.run.smithi089.stdout:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2023-10-10T20:33:43.059 INFO:teuthology.orchestra.run.smithi089.stdout:1: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2023-10-10T20:33:43.059 INFO:teuthology.orchestra.run.smithi089.stdout:2: [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] mon.c 2023-10-10T20:33:43.060 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-10T20:33:43.061 DEBUG:teuthology.orchestra.run.smithi089:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-10T20:33:43.116 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 5dd675b8-9026-4d8a-a1bf-62fdc5e7de67... 2023-10-10T20:33:43.117 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-10T20:33:43.158 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-10T20:33:43.198 INFO:teuthology.orchestra.run.smithi089.stdout:[global] 2023-10-10T20:33:43.199 INFO:teuthology.orchestra.run.smithi089.stdout: chdir = "" 2023-10-10T20:33:43.199 INFO:teuthology.orchestra.run.smithi089.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-10T20:33:43.199 INFO:teuthology.orchestra.run.smithi089.stdout: auth supported = cephx 2023-10-10T20:33:43.199 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.199 INFO:teuthology.orchestra.run.smithi089.stdout: filestore xattr use omap = true 2023-10-10T20:33:43.200 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.200 INFO:teuthology.orchestra.run.smithi089.stdout: mon clock drift allowed = 1.000 2023-10-10T20:33:43.200 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.200 INFO:teuthology.orchestra.run.smithi089.stdout: osd crush chooseleaf type = 0 2023-10-10T20:33:43.200 INFO:teuthology.orchestra.run.smithi089.stdout: auth debug = true 2023-10-10T20:33:43.201 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.201 INFO:teuthology.orchestra.run.smithi089.stdout: ms die on old message = true 2023-10-10T20:33:43.201 INFO:teuthology.orchestra.run.smithi089.stdout: ms die on bug = true 2023-10-10T20:33:43.201 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.201 INFO:teuthology.orchestra.run.smithi089.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-10T20:33:43.201 INFO:teuthology.orchestra.run.smithi089.stdout: mon pg warn max object skew = 0 2023-10-10T20:33:43.202 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.202 INFO:teuthology.orchestra.run.smithi089.stdout: # disable pg_autoscaler by default for new pools 2023-10-10T20:33:43.202 INFO:teuthology.orchestra.run.smithi089.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-10T20:33:43.202 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.202 INFO:teuthology.orchestra.run.smithi089.stdout: osd pool default size = 2 2023-10-10T20:33:43.203 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.203 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd allow primary affinity = true 2023-10-10T20:33:43.203 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd allow pg remap = true 2023-10-10T20:33:43.203 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on legacy crush tunables = false 2023-10-10T20:33:43.203 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on crush straw calc version zero = false 2023-10-10T20:33:43.204 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on no sortbitwise = false 2023-10-10T20:33:43.204 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on osd down out interval zero = false 2023-10-10T20:33:43.204 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on too few osds = false 2023-10-10T20:33:43.204 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-10T20:33:43.204 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-10T20:33:43.205 INFO:teuthology.orchestra.run.smithi089.stdout: mon_allow_pool_size_one = true 2023-10-10T20:33:43.205 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.205 INFO:teuthology.orchestra.run.smithi089.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-10T20:33:43.205 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.205 INFO:teuthology.orchestra.run.smithi089.stdout: osd default data pool replay window = 5 2023-10-10T20:33:43.206 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.206 INFO:teuthology.orchestra.run.smithi089.stdout: mon allow pool delete = true 2023-10-10T20:33:43.206 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.206 INFO:teuthology.orchestra.run.smithi089.stdout: mon cluster log file level = debug 2023-10-10T20:33:43.206 INFO:teuthology.orchestra.run.smithi089.stdout: debug asserts on shutdown = true 2023-10-10T20:33:43.207 INFO:teuthology.orchestra.run.smithi089.stdout: mon health detail to clog = false 2023-10-10T20:33:43.207 INFO:teuthology.orchestra.run.smithi089.stdout: mon host = "172.21.15.89,172.21.15.111,[v2:172.21.15.111:3301,v1:172.21.15.111:6790]" 2023-10-10T20:33:43.207 INFO:teuthology.orchestra.run.smithi089.stdout: ms die on skipped message = False 2023-10-10T20:33:43.207 INFO:teuthology.orchestra.run.smithi089.stdout: fsid = 5dd675b8-9026-4d8a-a1bf-62fdc5e7de67 2023-10-10T20:33:43.207 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.208 INFO:teuthology.orchestra.run.smithi089.stdout:[osd] 2023-10-10T20:33:43.208 INFO:teuthology.orchestra.run.smithi089.stdout: osd journal size = 100 2023-10-10T20:33:43.208 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.208 INFO:teuthology.orchestra.run.smithi089.stdout: osd scrub load threshold = 5.0 2023-10-10T20:33:43.208 INFO:teuthology.orchestra.run.smithi089.stdout: osd scrub max interval = 600 2023-10-10T20:33:43.208 INFO:teuthology.orchestra.run.smithi089.stdout: osd mclock profile = high_recovery_ops 2023-10-10T20:33:43.209 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.209 INFO:teuthology.orchestra.run.smithi089.stdout: osd recover clone overlap = true 2023-10-10T20:33:43.209 INFO:teuthology.orchestra.run.smithi089.stdout: osd recovery max chunk = 1048576 2023-10-10T20:33:43.209 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.209 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug shutdown = true 2023-10-10T20:33:43.210 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug op order = true 2023-10-10T20:33:43.210 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug verify stray on activate = true 2023-10-10T20:33:43.210 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.210 INFO:teuthology.orchestra.run.smithi089.stdout: osd open classes on start = true 2023-10-10T20:33:43.210 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug pg log writeout = true 2023-10-10T20:33:43.211 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.211 INFO:teuthology.orchestra.run.smithi089.stdout: osd deep scrub update digest min age = 30 2023-10-10T20:33:43.211 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.211 INFO:teuthology.orchestra.run.smithi089.stdout: osd map max advance = 10 2023-10-10T20:33:43.211 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.212 INFO:teuthology.orchestra.run.smithi089.stdout: journal zero on create = true 2023-10-10T20:33:43.212 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.212 INFO:teuthology.orchestra.run.smithi089.stdout: filestore ondisk finisher threads = 3 2023-10-10T20:33:43.212 INFO:teuthology.orchestra.run.smithi089.stdout: filestore apply finisher threads = 3 2023-10-10T20:33:43.212 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.212 INFO:teuthology.orchestra.run.smithi089.stdout: bdev debug aio = true 2023-10-10T20:33:43.213 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug misdirected ops = true 2023-10-10T20:33:43.213 INFO:teuthology.orchestra.run.smithi089.stdout: bdev async discard = True 2023-10-10T20:33:43.213 INFO:teuthology.orchestra.run.smithi089.stdout: bdev enable discard = True 2023-10-10T20:33:43.213 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore allocator = bitmap 2023-10-10T20:33:43.213 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore block size = 96636764160 2023-10-10T20:33:43.214 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore fsck on mount = True 2023-10-10T20:33:43.214 INFO:teuthology.orchestra.run.smithi089.stdout: debug bluefs = 1/20 2023-10-10T20:33:43.214 INFO:teuthology.orchestra.run.smithi089.stdout: debug bluestore = 1/20 2023-10-10T20:33:43.214 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-10-10T20:33:43.214 INFO:teuthology.orchestra.run.smithi089.stdout: debug osd = 20 2023-10-10T20:33:43.215 INFO:teuthology.orchestra.run.smithi089.stdout: debug rocksdb = 4/10 2023-10-10T20:33:43.215 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd backfillfull_ratio = 0.85 2023-10-10T20:33:43.215 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd full ratio = 0.9 2023-10-10T20:33:43.215 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd nearfull ratio = 0.8 2023-10-10T20:33:43.215 INFO:teuthology.orchestra.run.smithi089.stdout: osd failsafe full ratio = 0.95 2023-10-10T20:33:43.216 INFO:teuthology.orchestra.run.smithi089.stdout: osd objectstore = bluestore 2023-10-10T20:33:43.216 INFO:teuthology.orchestra.run.smithi089.stdout: osd op complaint time = 180 2023-10-10T20:33:43.216 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.216 INFO:teuthology.orchestra.run.smithi089.stdout:[mgr] 2023-10-10T20:33:43.216 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-10-10T20:33:43.216 INFO:teuthology.orchestra.run.smithi089.stdout: debug mgr = 20 2023-10-10T20:33:43.217 INFO:teuthology.orchestra.run.smithi089.stdout: debug mon = 20 2023-10-10T20:33:43.217 INFO:teuthology.orchestra.run.smithi089.stdout: debug auth = 20 2023-10-10T20:33:43.217 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min pgs per osd = 4 2023-10-10T20:33:43.217 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min bytes per osd = 10 2023-10-10T20:33:43.217 INFO:teuthology.orchestra.run.smithi089.stdout: mgr/telemetry/nag = false 2023-10-10T20:33:43.218 INFO:teuthology.orchestra.run.smithi089.stdout: debug client = 20 2023-10-10T20:33:43.218 INFO:teuthology.orchestra.run.smithi089.stdout: debug finisher = 20 2023-10-10T20:33:43.218 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.218 INFO:teuthology.orchestra.run.smithi089.stdout:[mon] 2023-10-10T20:33:43.218 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-10-10T20:33:43.219 INFO:teuthology.orchestra.run.smithi089.stdout: debug mon = 20 2023-10-10T20:33:43.219 INFO:teuthology.orchestra.run.smithi089.stdout: debug paxos = 20 2023-10-10T20:33:43.219 INFO:teuthology.orchestra.run.smithi089.stdout: debug auth = 20 2023-10-10T20:33:43.219 INFO:teuthology.orchestra.run.smithi089.stdout: mon data avail warn = 5 2023-10-10T20:33:43.219 INFO:teuthology.orchestra.run.smithi089.stdout: mon mgr mkfs grace = 240 2023-10-10T20:33:43.220 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min pgs per osd = 4 2023-10-10T20:33:43.220 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd reporter subtree level = osd 2023-10-10T20:33:43.220 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd prime pg temp = true 2023-10-10T20:33:43.220 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min bytes per osd = 10 2023-10-10T20:33:43.221 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.221 INFO:teuthology.orchestra.run.smithi089.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-10T20:33:43.221 INFO:teuthology.orchestra.run.smithi089.stdout: auth mon ticket ttl = 660 # 11m 2023-10-10T20:33:43.221 INFO:teuthology.orchestra.run.smithi089.stdout: auth service ticket ttl = 240 # 4m 2023-10-10T20:33:43.221 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.221 INFO:teuthology.orchestra.run.smithi089.stdout: # don't complain about insecure global_id in the test suite 2023-10-10T20:33:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-10T20:33:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-10T20:33:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: # 1m isn't quite enough 2023-10-10T20:33:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: mon_down_mkfs_grace = 2m 2023-10-10T20:33:43.223 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.223 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_filestore_osds = false 2023-10-10T20:33:43.223 INFO:teuthology.orchestra.run.smithi089.stdout: mon op complaint time = 120 2023-10-10T20:33:43.223 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:33:43.223 INFO:teuthology.orchestra.run.smithi089.stdout:[client] 2023-10-10T20:33:43.224 INFO:teuthology.orchestra.run.smithi089.stdout: rgw cache enabled = true 2023-10-10T20:33:43.224 INFO:teuthology.orchestra.run.smithi089.stdout: rgw enable ops log = true 2023-10-10T20:33:43.224 INFO:teuthology.orchestra.run.smithi089.stdout: rgw enable usage log = true 2023-10-10T20:33:43.224 INFO:teuthology.orchestra.run.smithi089.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-10T20:33:43.224 INFO:teuthology.orchestra.run.smithi089.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-10T20:33:43.224 INFO:teuthology.orchestra.run.smithi089.stdout: client mount timeout = 600 2023-10-10T20:33:43.225 INFO:teuthology.orchestra.run.smithi089.stdout: debug client = 20 2023-10-10T20:33:43.225 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-10-10T20:33:43.225 INFO:teuthology.orchestra.run.smithi089.stdout: pid file = "" 2023-10-10T20:33:43.225 INFO:teuthology.orchestra.run.smithi089.stdout: rados mon op timeout = 900 2023-10-10T20:33:43.225 INFO:teuthology.orchestra.run.smithi089.stdout: rados osd op timeout = 900 2023-10-10T20:33:43.226 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.a] 2023-10-10T20:33:43.226 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.b] 2023-10-10T20:33:43.226 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.c] 2023-10-10T20:33:43.226 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.a] 2023-10-10T20:33:43.226 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.c] 2023-10-10T20:33:43.227 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.b] 2023-10-10T20:33:43.227 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.d] 2023-10-10T20:33:43.227 INFO:teuthology.orchestra.run.smithi089.stdout:[mds] 2023-10-10T20:33:43.227 INFO:teuthology.orchestra.run.smithi089.stdout: debug mds = 20 2023-10-10T20:33:43.227 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-10-10T20:33:43.227 INFO:teuthology.orchestra.run.smithi089.stdout: mds debug frag = True 2023-10-10T20:33:43.228 INFO:teuthology.orchestra.run.smithi089.stdout: mds debug scatterstat = True 2023-10-10T20:33:43.228 INFO:teuthology.orchestra.run.smithi089.stdout: mds op complaint time = 180 2023-10-10T20:33:43.228 INFO:teuthology.orchestra.run.smithi089.stdout: mds verify scatter = True 2023-10-10T20:33:43.228 INFO:teuthology.orchestra.run.smithi089.stdout: osd op complaint time = 180 2023-10-10T20:33:43.228 INFO:teuthology.orchestra.run.smithi089.stdout: rados mon op timeout = 900 2023-10-10T20:33:43.229 INFO:teuthology.orchestra.run.smithi089.stdout: rados osd op timeout = 900 2023-10-10T20:33:43.230 INFO:teuthology.orchestra.run.smithi111.stdout:[global] 2023-10-10T20:33:43.230 INFO:teuthology.orchestra.run.smithi111.stdout: chdir = "" 2023-10-10T20:33:43.231 INFO:teuthology.orchestra.run.smithi111.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-10T20:33:43.231 INFO:teuthology.orchestra.run.smithi111.stdout: auth supported = cephx 2023-10-10T20:33:43.231 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.231 INFO:teuthology.orchestra.run.smithi111.stdout: filestore xattr use omap = true 2023-10-10T20:33:43.231 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.232 INFO:teuthology.orchestra.run.smithi111.stdout: mon clock drift allowed = 1.000 2023-10-10T20:33:43.232 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.232 INFO:teuthology.orchestra.run.smithi111.stdout: osd crush chooseleaf type = 0 2023-10-10T20:33:43.232 INFO:teuthology.orchestra.run.smithi111.stdout: auth debug = true 2023-10-10T20:33:43.232 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.233 INFO:teuthology.orchestra.run.smithi111.stdout: ms die on old message = true 2023-10-10T20:33:43.233 INFO:teuthology.orchestra.run.smithi111.stdout: ms die on bug = true 2023-10-10T20:33:43.233 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.233 INFO:teuthology.orchestra.run.smithi111.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-10T20:33:43.233 INFO:teuthology.orchestra.run.smithi111.stdout: mon pg warn max object skew = 0 2023-10-10T20:33:43.233 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.234 INFO:teuthology.orchestra.run.smithi111.stdout: # disable pg_autoscaler by default for new pools 2023-10-10T20:33:43.234 INFO:teuthology.orchestra.run.smithi111.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-10T20:33:43.234 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.234 INFO:teuthology.orchestra.run.smithi111.stdout: osd pool default size = 2 2023-10-10T20:33:43.234 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.235 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd allow primary affinity = true 2023-10-10T20:33:43.235 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd allow pg remap = true 2023-10-10T20:33:43.235 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on legacy crush tunables = false 2023-10-10T20:33:43.235 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on crush straw calc version zero = false 2023-10-10T20:33:43.235 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on no sortbitwise = false 2023-10-10T20:33:43.236 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on osd down out interval zero = false 2023-10-10T20:33:43.236 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on too few osds = false 2023-10-10T20:33:43.236 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-10T20:33:43.236 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-10T20:33:43.236 INFO:teuthology.orchestra.run.smithi111.stdout: mon_allow_pool_size_one = true 2023-10-10T20:33:43.237 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.237 INFO:teuthology.orchestra.run.smithi111.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-10T20:33:43.237 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.237 INFO:teuthology.orchestra.run.smithi111.stdout: osd default data pool replay window = 5 2023-10-10T20:33:43.237 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.237 INFO:teuthology.orchestra.run.smithi111.stdout: mon allow pool delete = true 2023-10-10T20:33:43.238 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.238 INFO:teuthology.orchestra.run.smithi111.stdout: mon cluster log file level = debug 2023-10-10T20:33:43.238 INFO:teuthology.orchestra.run.smithi111.stdout: debug asserts on shutdown = true 2023-10-10T20:33:43.238 INFO:teuthology.orchestra.run.smithi111.stdout: mon health detail to clog = false 2023-10-10T20:33:43.238 INFO:teuthology.orchestra.run.smithi111.stdout: mon host = "172.21.15.89,172.21.15.111,[v2:172.21.15.111:3301,v1:172.21.15.111:6790]" 2023-10-10T20:33:43.239 INFO:teuthology.orchestra.run.smithi111.stdout: ms die on skipped message = False 2023-10-10T20:33:43.239 INFO:teuthology.orchestra.run.smithi111.stdout: fsid = 5dd675b8-9026-4d8a-a1bf-62fdc5e7de67 2023-10-10T20:33:43.239 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.239 INFO:teuthology.orchestra.run.smithi111.stdout:[osd] 2023-10-10T20:33:43.240 INFO:teuthology.orchestra.run.smithi111.stdout: osd journal size = 100 2023-10-10T20:33:43.240 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.240 INFO:teuthology.orchestra.run.smithi111.stdout: osd scrub load threshold = 5.0 2023-10-10T20:33:43.240 INFO:teuthology.orchestra.run.smithi111.stdout: osd scrub max interval = 600 2023-10-10T20:33:43.240 INFO:teuthology.orchestra.run.smithi111.stdout: osd mclock profile = high_recovery_ops 2023-10-10T20:33:43.241 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.241 INFO:teuthology.orchestra.run.smithi111.stdout: osd recover clone overlap = true 2023-10-10T20:33:43.241 INFO:teuthology.orchestra.run.smithi111.stdout: osd recovery max chunk = 1048576 2023-10-10T20:33:43.241 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.241 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug shutdown = true 2023-10-10T20:33:43.242 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug op order = true 2023-10-10T20:33:43.242 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug verify stray on activate = true 2023-10-10T20:33:43.242 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.242 INFO:teuthology.orchestra.run.smithi111.stdout: osd open classes on start = true 2023-10-10T20:33:43.242 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug pg log writeout = true 2023-10-10T20:33:43.243 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.243 INFO:teuthology.orchestra.run.smithi111.stdout: osd deep scrub update digest min age = 30 2023-10-10T20:33:43.243 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.243 INFO:teuthology.orchestra.run.smithi111.stdout: osd map max advance = 10 2023-10-10T20:33:43.243 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.243 INFO:teuthology.orchestra.run.smithi111.stdout: journal zero on create = true 2023-10-10T20:33:43.244 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.244 INFO:teuthology.orchestra.run.smithi111.stdout: filestore ondisk finisher threads = 3 2023-10-10T20:33:43.244 INFO:teuthology.orchestra.run.smithi111.stdout: filestore apply finisher threads = 3 2023-10-10T20:33:43.244 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.244 INFO:teuthology.orchestra.run.smithi111.stdout: bdev debug aio = true 2023-10-10T20:33:43.245 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug misdirected ops = true 2023-10-10T20:33:43.245 INFO:teuthology.orchestra.run.smithi111.stdout: bdev async discard = True 2023-10-10T20:33:43.245 INFO:teuthology.orchestra.run.smithi111.stdout: bdev enable discard = True 2023-10-10T20:33:43.245 INFO:teuthology.orchestra.run.smithi111.stdout: bluestore allocator = bitmap 2023-10-10T20:33:43.245 INFO:teuthology.orchestra.run.smithi111.stdout: bluestore block size = 96636764160 2023-10-10T20:33:43.246 INFO:teuthology.orchestra.run.smithi111.stdout: bluestore fsck on mount = True 2023-10-10T20:33:43.246 INFO:teuthology.orchestra.run.smithi111.stdout: debug bluefs = 1/20 2023-10-10T20:33:43.246 INFO:teuthology.orchestra.run.smithi111.stdout: debug bluestore = 1/20 2023-10-10T20:33:43.246 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2023-10-10T20:33:43.246 INFO:teuthology.orchestra.run.smithi111.stdout: debug osd = 20 2023-10-10T20:33:43.247 INFO:teuthology.orchestra.run.smithi111.stdout: debug rocksdb = 4/10 2023-10-10T20:33:43.247 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd backfillfull_ratio = 0.85 2023-10-10T20:33:43.247 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd full ratio = 0.9 2023-10-10T20:33:43.247 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd nearfull ratio = 0.8 2023-10-10T20:33:43.247 INFO:teuthology.orchestra.run.smithi111.stdout: osd failsafe full ratio = 0.95 2023-10-10T20:33:43.248 INFO:teuthology.orchestra.run.smithi111.stdout: osd objectstore = bluestore 2023-10-10T20:33:43.248 INFO:teuthology.orchestra.run.smithi111.stdout: osd op complaint time = 180 2023-10-10T20:33:43.248 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.248 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr] 2023-10-10T20:33:43.248 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2023-10-10T20:33:43.248 INFO:teuthology.orchestra.run.smithi111.stdout: debug mgr = 20 2023-10-10T20:33:43.249 INFO:teuthology.orchestra.run.smithi111.stdout: debug mon = 20 2023-10-10T20:33:43.249 INFO:teuthology.orchestra.run.smithi111.stdout: debug auth = 20 2023-10-10T20:33:43.249 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min pgs per osd = 4 2023-10-10T20:33:43.249 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min bytes per osd = 10 2023-10-10T20:33:43.249 INFO:teuthology.orchestra.run.smithi111.stdout: mgr/telemetry/nag = false 2023-10-10T20:33:43.250 INFO:teuthology.orchestra.run.smithi111.stdout: debug client = 20 2023-10-10T20:33:43.250 INFO:teuthology.orchestra.run.smithi111.stdout: debug finisher = 20 2023-10-10T20:33:43.250 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.250 INFO:teuthology.orchestra.run.smithi111.stdout:[mon] 2023-10-10T20:33:43.250 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2023-10-10T20:33:43.251 INFO:teuthology.orchestra.run.smithi111.stdout: debug mon = 20 2023-10-10T20:33:43.251 INFO:teuthology.orchestra.run.smithi111.stdout: debug paxos = 20 2023-10-10T20:33:43.251 INFO:teuthology.orchestra.run.smithi111.stdout: debug auth = 20 2023-10-10T20:33:43.251 INFO:teuthology.orchestra.run.smithi111.stdout: mon data avail warn = 5 2023-10-10T20:33:43.251 INFO:teuthology.orchestra.run.smithi111.stdout: mon mgr mkfs grace = 240 2023-10-10T20:33:43.251 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min pgs per osd = 4 2023-10-10T20:33:43.252 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd reporter subtree level = osd 2023-10-10T20:33:43.252 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd prime pg temp = true 2023-10-10T20:33:43.252 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min bytes per osd = 10 2023-10-10T20:33:43.252 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.252 INFO:teuthology.orchestra.run.smithi111.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-10T20:33:43.253 INFO:teuthology.orchestra.run.smithi111.stdout: auth mon ticket ttl = 660 # 11m 2023-10-10T20:33:43.253 INFO:teuthology.orchestra.run.smithi111.stdout: auth service ticket ttl = 240 # 4m 2023-10-10T20:33:43.253 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.253 INFO:teuthology.orchestra.run.smithi111.stdout: # don't complain about insecure global_id in the test suite 2023-10-10T20:33:43.253 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-10T20:33:43.254 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-10T20:33:43.254 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.254 INFO:teuthology.orchestra.run.smithi111.stdout: # 1m isn't quite enough 2023-10-10T20:33:43.254 INFO:teuthology.orchestra.run.smithi111.stdout: mon_down_mkfs_grace = 2m 2023-10-10T20:33:43.254 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.255 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_filestore_osds = false 2023-10-10T20:33:43.255 INFO:teuthology.orchestra.run.smithi111.stdout: mon op complaint time = 120 2023-10-10T20:33:43.255 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-10-10T20:33:43.255 INFO:teuthology.orchestra.run.smithi111.stdout:[client] 2023-10-10T20:33:43.255 INFO:teuthology.orchestra.run.smithi111.stdout: rgw cache enabled = true 2023-10-10T20:33:43.255 INFO:teuthology.orchestra.run.smithi111.stdout: rgw enable ops log = true 2023-10-10T20:33:43.256 INFO:teuthology.orchestra.run.smithi111.stdout: rgw enable usage log = true 2023-10-10T20:33:43.256 INFO:teuthology.orchestra.run.smithi111.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-10T20:33:43.256 INFO:teuthology.orchestra.run.smithi111.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-10T20:33:43.256 INFO:teuthology.orchestra.run.smithi111.stdout: client mount timeout = 600 2023-10-10T20:33:43.256 INFO:teuthology.orchestra.run.smithi111.stdout: debug client = 20 2023-10-10T20:33:43.257 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2023-10-10T20:33:43.257 INFO:teuthology.orchestra.run.smithi111.stdout: pid file = "" 2023-10-10T20:33:43.257 INFO:teuthology.orchestra.run.smithi111.stdout: rados mon op timeout = 900 2023-10-10T20:33:43.257 INFO:teuthology.orchestra.run.smithi111.stdout: rados osd op timeout = 900 2023-10-10T20:33:43.257 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.a] 2023-10-10T20:33:43.258 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.b] 2023-10-10T20:33:43.258 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.c] 2023-10-10T20:33:43.258 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.a] 2023-10-10T20:33:43.258 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.c] 2023-10-10T20:33:43.258 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.b] 2023-10-10T20:33:43.258 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.d] 2023-10-10T20:33:43.259 INFO:teuthology.orchestra.run.smithi111.stdout:[mds] 2023-10-10T20:33:43.259 INFO:teuthology.orchestra.run.smithi111.stdout: debug mds = 20 2023-10-10T20:33:43.259 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2023-10-10T20:33:43.259 INFO:teuthology.orchestra.run.smithi111.stdout: mds debug frag = True 2023-10-10T20:33:43.259 INFO:teuthology.orchestra.run.smithi111.stdout: mds debug scatterstat = True 2023-10-10T20:33:43.260 INFO:teuthology.orchestra.run.smithi111.stdout: mds op complaint time = 180 2023-10-10T20:33:43.260 INFO:teuthology.orchestra.run.smithi111.stdout: mds verify scatter = True 2023-10-10T20:33:43.260 INFO:teuthology.orchestra.run.smithi111.stdout: osd op complaint time = 180 2023-10-10T20:33:43.260 INFO:teuthology.orchestra.run.smithi111.stdout: rados mon op timeout = 900 2023-10-10T20:33:43.260 INFO:teuthology.orchestra.run.smithi111.stdout: rados osd op timeout = 900 2023-10-10T20:33:43.261 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-10T20:33:43.261 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-10T20:33:43.321 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-10T20:33:43.322 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:33:43.322 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-10T20:33:43.337 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:33:43.337 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-10T20:33:43.391 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:33:43.391 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:33:43.392 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-10T20:33:43.392 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-10T20:33:43.464 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:33:43.465 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:33:43.519 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:33:43.520 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:33:43.520 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-10T20:33:43.520 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-10T20:33:43.555 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:33:43.556 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:33:43.609 INFO:tasks.ceph:Setting up mon nodes... 2023-10-10T20:33:43.610 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-10T20:33:43.611 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-10T20:33:43.658 INFO:teuthology.orchestra.run.smithi089.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-10T20:33:43.661 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-10T20:33:43.731 INFO:teuthology.orchestra.run.smithi111.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-10T20:33:43.734 INFO:tasks.ceph:Setting up mds nodes... 2023-10-10T20:33:43.734 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-10T20:33:43.782 INFO:teuthology.orchestra.run.smithi089.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-10T20:33:43.784 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-10T20:33:43.808 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-10T20:33:43.895 INFO:teuthology.orchestra.run.smithi089.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-10T20:33:43.898 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-10T20:33:43.922 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-10T20:33:43.970 INFO:teuthology.orchestra.run.smithi111.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-10T20:33:43.972 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-10T20:33:43.996 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-10T20:33:44.083 INFO:teuthology.orchestra.run.smithi111.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-10T20:33:44.085 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-10T20:33:44.109 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-10T20:33:44.110 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-10T20:33:44.148 INFO:teuthology.orchestra.run.smithi089.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-10T20:33:44.160 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-10T20:33:44.239 INFO:teuthology.orchestra.run.smithi089.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-10T20:33:44.251 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-10T20:33:44.289 INFO:teuthology.orchestra.run.smithi111.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-10T20:33:44.301 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-10T20:33:44.379 INFO:teuthology.orchestra.run.smithi111.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-10T20:33:44.391 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-10T20:33:44.391 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi089.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi111.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-10T20:33:44.392 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-10T20:33:44.416 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-10T20:33:44.417 INFO:tasks.ceph:role: osd.0 2023-10-10T20:33:44.417 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:33:44.417 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-10T20:33:44.488 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-10T20:33:44.489 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-10T20:33:44.489 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-10T20:33:44.489 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 2023-10-10T20:33:44.489 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-10T20:33:44.489 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2023-10-10T20:33:44.490 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-10T20:33:44.490 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-10T20:33:44.490 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-10T20:33:44.490 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-10T20:33:44.918 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2023-10-10T20:33:44.920 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2023-10-10T20:33:44.920 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-10T20:33:45.082 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-10T20:33:45.116 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-10T20:33:45.181 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-10T20:33:45.181 INFO:tasks.ceph:role: osd.1 2023-10-10T20:33:45.181 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:33:45.182 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-10T20:33:45.253 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-10T20:33:45.253 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-10T20:33:45.253 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-10T20:33:45.253 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 2023-10-10T20:33:45.254 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-10T20:33:45.254 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2023-10-10T20:33:45.254 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-10T20:33:45.254 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-10T20:33:45.254 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-10T20:33:45.255 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-10T20:33:45.689 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2023-10-10T20:33:45.691 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2023-10-10T20:33:45.691 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-10T20:33:45.801 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-10T20:33:45.831 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-10T20:33:45.898 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-10T20:33:45.898 INFO:tasks.ceph:role: osd.2 2023-10-10T20:33:45.898 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:33:45.899 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-10T20:33:45.970 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-10T20:33:45.971 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-10T20:33:45.971 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-10T20:33:45.971 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 2023-10-10T20:33:45.971 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-10T20:33:45.971 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2023-10-10T20:33:45.972 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-10T20:33:45.972 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-10T20:33:45.972 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-10T20:33:45.972 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-10T20:33:46.425 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2023-10-10T20:33:46.427 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2023-10-10T20:33:46.427 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-10T20:33:46.591 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-10T20:33:46.624 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-10T20:33:46.691 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-10T20:33:46.691 INFO:tasks.ceph:role: osd.3 2023-10-10T20:33:46.691 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:33:46.692 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-10T20:33:46.763 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-10T20:33:46.763 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-10T20:33:46.764 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-10T20:33:46.764 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 2023-10-10T20:33:46.764 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-10T20:33:46.764 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2023-10-10T20:33:46.764 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-10T20:33:46.765 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-10T20:33:46.765 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-10T20:33:46.765 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-10T20:33:47.199 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2023-10-10T20:33:47.202 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2023-10-10T20:33:47.203 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-10T20:33:47.342 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-10T20:33:47.376 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:33:47.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:47.486+0000 7f2373a8f540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-10T20:33:47.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:47.486+0000 7f2373a8f540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-10T20:33:47.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:47.486+0000 7f2373a8f540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-10T20:33:47.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:47.486+0000 7f2373a8f540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-10T20:33:47.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:47.486+0000 7f2373a8f540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-10T20:33:47.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:47.487+0000 7f2373a8f540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-10T20:33:48.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:48.015+0000 7f2373a8f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:33:48.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:48.779+0000 7f2373a8f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:33:49.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:49.540+0000 7f2373a8f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:33:50.339 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-10T20:33:50.368 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:33:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:50.450+0000 7f1af8a99540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-10T20:33:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:50.450+0000 7f1af8a99540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-10T20:33:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:50.450+0000 7f1af8a99540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-10T20:33:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:50.450+0000 7f1af8a99540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-10T20:33:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:50.450+0000 7f1af8a99540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-10T20:33:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:50.451+0000 7f1af8a99540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-10T20:33:50.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:50.978+0000 7f1af8a99540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:33:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:51.742+0000 7f1af8a99540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:33:52.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:52.506+0000 7f1af8a99540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:33:53.311 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-10T20:33:53.340 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:33:53.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:53.423+0000 7f881abf9540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-10T20:33:53.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:53.423+0000 7f881abf9540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-10T20:33:53.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:53.424+0000 7f881abf9540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-10T20:33:53.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:53.424+0000 7f881abf9540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-10T20:33:53.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:53.424+0000 7f881abf9540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-10T20:33:53.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:53.424+0000 7f881abf9540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-10T20:33:53.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:53.966+0000 7f881abf9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:33:54.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:54.729+0000 7f881abf9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:33:55.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:55.500+0000 7f881abf9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:33:56.300 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-10T20:33:56.329 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:33:56.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:56.411+0000 7f5c58da2540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-10T20:33:56.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:56.412+0000 7f5c58da2540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-10T20:33:56.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:56.412+0000 7f5c58da2540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-10T20:33:56.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:56.412+0000 7f5c58da2540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-10T20:33:56.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:56.412+0000 7f5c58da2540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-10T20:33:56.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:56.412+0000 7f5c58da2540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-10T20:33:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:56.935+0000 7f5c58da2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:33:57.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:57.698+0000 7f5c58da2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:33:58.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:33:58.462+0000 7f5c58da2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:33:59.263 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-10T20:33:59.291 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-10T20:33:59.323 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-10T20:33:59.323 INFO:tasks.ceph:role: osd.4 2023-10-10T20:33:59.324 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:33:59.324 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-10T20:33:59.400 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-10T20:33:59.401 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-10T20:33:59.401 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-10T20:33:59.401 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 2023-10-10T20:33:59.401 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-10T20:33:59.402 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2023-10-10T20:33:59.402 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-10T20:33:59.402 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-10T20:33:59.402 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-10T20:33:59.402 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-10T20:33:59.826 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2023-10-10T20:33:59.828 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2023-10-10T20:33:59.828 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-10T20:34:00.007 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-10T20:34:00.041 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-10T20:34:00.108 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-10T20:34:00.108 INFO:tasks.ceph:role: osd.5 2023-10-10T20:34:00.109 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:34:00.109 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-10T20:34:00.181 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-10T20:34:00.182 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-10T20:34:00.182 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-10T20:34:00.182 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 2023-10-10T20:34:00.183 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-10T20:34:00.183 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2023-10-10T20:34:00.183 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-10T20:34:00.183 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-10T20:34:00.183 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-10T20:34:00.184 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-10T20:34:00.600 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2023-10-10T20:34:00.602 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2023-10-10T20:34:00.603 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-10T20:34:00.715 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-10T20:34:00.749 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-10T20:34:00.819 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-10T20:34:00.819 INFO:tasks.ceph:role: osd.6 2023-10-10T20:34:00.819 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:34:00.820 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-10T20:34:00.897 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-10T20:34:00.897 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-10T20:34:00.897 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-10T20:34:00.898 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 2023-10-10T20:34:00.898 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-10T20:34:00.898 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2023-10-10T20:34:00.898 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-10T20:34:00.899 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-10T20:34:00.899 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-10T20:34:00.899 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-10T20:34:01.340 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2023-10-10T20:34:01.343 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2023-10-10T20:34:01.343 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-10T20:34:01.476 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-10T20:34:01.508 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-10T20:34:01.574 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-10T20:34:01.575 INFO:tasks.ceph:role: osd.7 2023-10-10T20:34:01.575 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:34:01.575 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-10T20:34:01.648 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-10T20:34:01.648 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-10T20:34:01.648 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-10T20:34:01.649 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 2023-10-10T20:34:01.649 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-10T20:34:01.649 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2023-10-10T20:34:01.649 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-10T20:34:01.649 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-10T20:34:01.650 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-10T20:34:01.650 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-10T20:34:02.100 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2023-10-10T20:34:02.102 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2023-10-10T20:34:02.102 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-10T20:34:02.276 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-10T20:34:02.309 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:34:02.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:02.398+0000 7f5011368540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-10T20:34:02.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:02.398+0000 7f5011368540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-10T20:34:02.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:02.398+0000 7f5011368540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-10T20:34:02.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:02.398+0000 7f5011368540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-10T20:34:02.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:02.398+0000 7f5011368540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-10T20:34:02.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:02.399+0000 7f5011368540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-10T20:34:02.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:02.922+0000 7f5011368540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:34:03.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:03.690+0000 7f5011368540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:34:04.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:04.453+0000 7f5011368540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:34:05.254 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-10T20:34:05.282 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:34:05.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:05.362+0000 7f07bbe41540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-10T20:34:05.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:05.362+0000 7f07bbe41540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-10T20:34:05.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:05.362+0000 7f07bbe41540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-10T20:34:05.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:05.362+0000 7f07bbe41540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-10T20:34:05.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:05.362+0000 7f07bbe41540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-10T20:34:05.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:05.363+0000 7f07bbe41540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-10T20:34:05.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:05.885+0000 7f07bbe41540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:34:06.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:06.647+0000 7f07bbe41540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:34:07.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:07.409+0000 7f07bbe41540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:34:08.205 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-10T20:34:08.270 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:34:08.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:08.348+0000 7f21d5129540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-10T20:34:08.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:08.348+0000 7f21d5129540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-10T20:34:08.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:08.348+0000 7f21d5129540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-10T20:34:08.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:08.348+0000 7f21d5129540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-10T20:34:08.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:08.348+0000 7f21d5129540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-10T20:34:08.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:08.348+0000 7f21d5129540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-10T20:34:08.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:08.873+0000 7f21d5129540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:34:09.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:09.637+0000 7f21d5129540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:34:10.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:10.399+0000 7f21d5129540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:34:11.197 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-10T20:34:11.199 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:34:11.202 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:34:11.266 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:34:11.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:11.346+0000 7f244f8c1540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-10T20:34:11.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:11.346+0000 7f244f8c1540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-10T20:34:11.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:11.346+0000 7f244f8c1540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-10T20:34:11.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:11.346+0000 7f244f8c1540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-10T20:34:11.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:11.346+0000 7f244f8c1540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-10T20:34:11.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:11.347+0000 7f244f8c1540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-10T20:34:11.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:11.869+0000 7f244f8c1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:34:12.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:12.631+0000 7f244f8c1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:34:13.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:13.399+0000 7f244f8c1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-10T20:34:14.199 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-10T20:34:14.269 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-10T20:34:14.269 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:34:14.270 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-10T20:34:14.296 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:34:14.297 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-10T20:34:14.363 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:34:14.363 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-10T20:34:14.428 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:34:14.428 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-10T20:34:14.492 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:34:14.493 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-10T20:34:14.557 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:34:14.558 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-10T20:34:14.621 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:34:14.621 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-10T20:34:14.685 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:34:14.685 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-10T20:34:14.713 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:34:14.714 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-10T20:34:14.780 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:34:14.781 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-10T20:34:14.847 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:34:14.847 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-10T20:34:14.912 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:34:14.912 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-10T20:34:14.977 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:34:14.977 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-10T20:34:15.041 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:34:15.042 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-10T20:34:15.106 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:34:15.106 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-10T20:34:15.121 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:34:15.122 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-10T20:34:15.176 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:34:15.176 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-10T20:34:15.191 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:34:15.192 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-10T20:34:15.246 INFO:tasks.ceph:Adding keys to all mons... 2023-10-10T20:34:15.247 DEBUG:teuthology.orchestra.run.smithi089:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-10T20:34:15.249 DEBUG:teuthology.orchestra.run.smithi111:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-10T20:34:15.290 INFO:teuthology.orchestra.run.smithi089.stdout:[mgr.y] 2023-10-10T20:34:15.290 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAntSVlTZIyJxAAWAOsitHwsIxJWwp3b1Z98Q== 2023-10-10T20:34:15.290 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.a] 2023-10-10T20:34:15.291 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAntSVlZLSNLhAA8WZTrsht6tAOP+gxaTXKMg== 2023-10-10T20:34:15.291 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.c] 2023-10-10T20:34:15.291 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAntSVlMFxSNRAASuQEJFghLIOkTQWa4ckTvg== 2023-10-10T20:34:15.291 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.0] 2023-10-10T20:34:15.291 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQArtSVltOcKHRAAk8+zUus04oTVI9KvfhjlXg== 2023-10-10T20:34:15.292 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.1] 2023-10-10T20:34:15.292 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAutSVly8P0GhAAe1bvR63+ptJg84lizMNdQw== 2023-10-10T20:34:15.292 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.2] 2023-10-10T20:34:15.292 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAxtSVl0x9fGRAAEKi7+6EW5mzPbebmz0+sgQ== 2023-10-10T20:34:15.292 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.3] 2023-10-10T20:34:15.293 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQA0tSVlfJyaGBAAa+8w54U6v7+npbdufKJruA== 2023-10-10T20:34:15.293 INFO:teuthology.orchestra.run.smithi089.stdout:[mgr.x] 2023-10-10T20:34:15.293 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAntSVlBhaJKxAArraBxT5xWg515LfpJ7MlbA== 2023-10-10T20:34:15.293 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.b] 2023-10-10T20:34:15.293 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAntSVlgyDEORAAMMoSxPFZc0bCIwWy8UsXgA== 2023-10-10T20:34:15.294 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.d] 2023-10-10T20:34:15.294 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAotSVlPkvrBBAA2VLJmv2t8szTEGtCVwalJw== 2023-10-10T20:34:15.294 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.4] 2023-10-10T20:34:15.294 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQA6tSVlEQ/QFxAA/PRB3c90BuAlF6xekGfX9A== 2023-10-10T20:34:15.294 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.5] 2023-10-10T20:34:15.294 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQA9tSVlMOKrFRAAuBnIsEz2VmeSZi9XWzjlzA== 2023-10-10T20:34:15.295 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.6] 2023-10-10T20:34:15.295 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBAtSVlH7PQFBAAHr6c7d4TYJEu/KSFEf+EOA== 2023-10-10T20:34:15.295 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.7] 2023-10-10T20:34:15.295 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBDtSVlAhC1FBAAuE2BwAA/f7VOkZfVdqAG7A== 2023-10-10T20:34:15.295 INFO:teuthology.orchestra.run.smithi089.stdout:[client.0] 2023-10-10T20:34:15.296 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAotSVlRd3HCBAAb0QP2VfTS7aelno1mFpGAA== 2023-10-10T20:34:15.296 INFO:teuthology.orchestra.run.smithi089.stdout:[client.2] 2023-10-10T20:34:15.296 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAotSVlXA4tDhAAoPVYQjECpzD+C67mnsY5IA== 2023-10-10T20:34:15.296 INFO:teuthology.orchestra.run.smithi089.stdout:[client.1] 2023-10-10T20:34:15.296 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAotSVlgBouERAAqPdE0KvJU3M53B6Y6AH5GQ== 2023-10-10T20:34:15.297 INFO:teuthology.orchestra.run.smithi089.stdout:[client.3] 2023-10-10T20:34:15.297 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAotSVlZ5KGFhAAuKmZwbOfmmw1nAj5ry6tHw== 2023-10-10T20:34:15.309 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr.y] 2023-10-10T20:34:15.309 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAntSVlTZIyJxAAWAOsitHwsIxJWwp3b1Z98Q== 2023-10-10T20:34:15.309 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.a] 2023-10-10T20:34:15.310 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAntSVlZLSNLhAA8WZTrsht6tAOP+gxaTXKMg== 2023-10-10T20:34:15.310 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.c] 2023-10-10T20:34:15.310 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAntSVlMFxSNRAASuQEJFghLIOkTQWa4ckTvg== 2023-10-10T20:34:15.310 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.0] 2023-10-10T20:34:15.310 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQArtSVltOcKHRAAk8+zUus04oTVI9KvfhjlXg== 2023-10-10T20:34:15.311 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.1] 2023-10-10T20:34:15.311 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAutSVly8P0GhAAe1bvR63+ptJg84lizMNdQw== 2023-10-10T20:34:15.311 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.2] 2023-10-10T20:34:15.311 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAxtSVl0x9fGRAAEKi7+6EW5mzPbebmz0+sgQ== 2023-10-10T20:34:15.311 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.3] 2023-10-10T20:34:15.312 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQA0tSVlfJyaGBAAa+8w54U6v7+npbdufKJruA== 2023-10-10T20:34:15.312 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr.x] 2023-10-10T20:34:15.312 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAntSVlBhaJKxAArraBxT5xWg515LfpJ7MlbA== 2023-10-10T20:34:15.312 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.b] 2023-10-10T20:34:15.312 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAntSVlgyDEORAAMMoSxPFZc0bCIwWy8UsXgA== 2023-10-10T20:34:15.312 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.d] 2023-10-10T20:34:15.313 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAotSVlPkvrBBAA2VLJmv2t8szTEGtCVwalJw== 2023-10-10T20:34:15.313 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.4] 2023-10-10T20:34:15.313 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQA6tSVlEQ/QFxAA/PRB3c90BuAlF6xekGfX9A== 2023-10-10T20:34:15.313 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.5] 2023-10-10T20:34:15.313 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQA9tSVlMOKrFRAAuBnIsEz2VmeSZi9XWzjlzA== 2023-10-10T20:34:15.314 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.6] 2023-10-10T20:34:15.314 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBAtSVlH7PQFBAAHr6c7d4TYJEu/KSFEf+EOA== 2023-10-10T20:34:15.314 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.7] 2023-10-10T20:34:15.314 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBDtSVlAhC1FBAAuE2BwAA/f7VOkZfVdqAG7A== 2023-10-10T20:34:15.314 INFO:teuthology.orchestra.run.smithi111.stdout:[client.0] 2023-10-10T20:34:15.315 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAotSVlRd3HCBAAb0QP2VfTS7aelno1mFpGAA== 2023-10-10T20:34:15.315 INFO:teuthology.orchestra.run.smithi111.stdout:[client.2] 2023-10-10T20:34:15.315 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAotSVlXA4tDhAAoPVYQjECpzD+C67mnsY5IA== 2023-10-10T20:34:15.315 INFO:teuthology.orchestra.run.smithi111.stdout:[client.1] 2023-10-10T20:34:15.315 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAotSVlgBouERAAqPdE0KvJU3M53B6Y6AH5GQ== 2023-10-10T20:34:15.315 INFO:teuthology.orchestra.run.smithi111.stdout:[client.3] 2023-10-10T20:34:15.316 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAotSVlZ5KGFhAAuKmZwbOfmmw1nAj5ry6tHw== 2023-10-10T20:34:15.316 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-10T20:34:15.340 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-10T20:34:15.404 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-10T20:34:15.406 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-10T20:34:15.456 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-10T20:34:15.458 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-10T20:34:15.508 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:34:15.511 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:34:15.558 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:34:15.560 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:34:15.607 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:34:15.610 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:34:15.657 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:34:15.659 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:34:15.706 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-10T20:34:15.708 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-10T20:34:15.755 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-10T20:34:15.757 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-10T20:34:15.807 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-10T20:34:15.809 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-10T20:34:15.860 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:34:15.863 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:34:15.910 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:34:15.912 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:34:15.960 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:34:15.963 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:34:16.009 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:34:16.012 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:34:16.060 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-10T20:34:16.062 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-10T20:34:16.112 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-10T20:34:16.114 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-10T20:34:16.164 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-10T20:34:16.166 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-10T20:34:16.216 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-10T20:34:16.218 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-10T20:34:16.268 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-10T20:34:16.268 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-10T20:34:16.293 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-10T20:34:16.385 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-10T20:34:16.409 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-10T20:34:16.434 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-10T20:34:16.525 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-10T20:34:16.549 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-10T20:34:16.614 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-10T20:34:16.705 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-10T20:34:16.729 DEBUG:teuthology.orchestra.run.smithi089:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:34:16.732 DEBUG:teuthology.orchestra.run.smithi111:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:34:16.786 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-10T20:34:16.787 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-10T20:34:16.788 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-10T20:34:16.790 INFO:tasks.ceph.mon.a:Started 2023-10-10T20:34:16.791 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-10T20:34:16.791 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-10T20:34:16.828 INFO:tasks.ceph.mon.b:Started 2023-10-10T20:34:16.829 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-10T20:34:16.829 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-10T20:34:16.831 INFO:tasks.ceph.mon.c:Started 2023-10-10T20:34:16.831 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-10T20:34:16.832 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-10T20:34:16.832 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-10T20:34:16.834 INFO:tasks.ceph.mgr.y:Started 2023-10-10T20:34:16.835 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-10T20:34:16.835 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-10T20:34:16.837 INFO:tasks.ceph.mgr.x:Started 2023-10-10T20:34:16.837 INFO:tasks.ceph:Setting crush tunables to default 2023-10-10T20:34:16.837 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd crush tunables default 2023-10-10T20:34:16.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:16.951+0000 7f4fda03d700 1 Processor -- start 2023-10-10T20:34:16.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:16.951+0000 7f4fda03d700 1 -- start start 2023-10-10T20:34:16.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:16.952+0000 7f4fda03d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fd4140350 0x7f4fd4069c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:16.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:16.952+0000 7f4fda03d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4fd4068aa0 0x7f4fd40c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:16.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:16.952+0000 7f4fda03d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd40c7de0 con 0x7f4fd406a1c0 2023-10-10T20:34:16.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:16.952+0000 7f4fda03d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fd4140c10 con 0x7f4fd4068aa0 2023-10-10T20:34:16.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:16.952+0000 7f4fda03d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd4141490 con 0x7f4fd4140350 2023-10-10T20:34:16.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:16.952+0000 7f4fd3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4fd4068aa0 0x7f4fd40c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:16.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:16.952+0000 7f4fd3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4fd4068aa0 0x7f4fd40c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50910/0 (socket says 172.21.15.89:50910) 2023-10-10T20:34:16.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:16.952+0000 7f4fd3fff700 1 -- 172.21.15.89:0/3550031699 learned_addr learned my addr 172.21.15.89:0/3550031699 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:16.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:16.953+0000 7f4fd3fff700 1 -- 172.21.15.89:0/3550031699 >> v2:172.21.15.89:3300/0 conn(0x7f4fd4068aa0 msgr2=0x7f4fd40c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-10T20:34:16.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:16.953+0000 7f4fd3fff700 1 -- 172.21.15.89:0/3550031699 >> v2:172.21.15.89:3300/0 conn(0x7f4fd4068aa0 msgr2=0x7f4fd40c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-10T20:34:16.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:16.953+0000 7f4fd3fff700 1 --2- 172.21.15.89:0/3550031699 >> v2:172.21.15.89:3300/0 conn(0x7f4fd4068aa0 0x7f4fd40c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-10T20:34:16.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:16.953+0000 7f4fd3fff700 1 --2- 172.21.15.89:0/3550031699 >> v2:172.21.15.89:3300/0 conn(0x7f4fd4068aa0 0x7f4fd40c8270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-10T20:34:16.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:16.953+0000 7f4fd37fe700 1 --2- 172.21.15.89:0/3550031699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fd4140350 0x7f4fd4069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:16.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:16.953+0000 7f4fd37fe700 1 -- 172.21.15.89:0/3550031699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fd4140350 msgr2=0x7f4fd4069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-10T20:34:16.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:16.953+0000 7f4fd37fe700 1 -- 172.21.15.89:0/3550031699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fd4140350 msgr2=0x7f4fd4069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-10T20:34:16.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:16.953+0000 7f4fd37fe700 1 --2- 172.21.15.89:0/3550031699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fd4140350 0x7f4fd4069c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-10T20:34:16.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:16.953+0000 7f4fd37fe700 1 --2- 172.21.15.89:0/3550031699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fd4140350 0x7f4fd4069c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-10T20:34:17.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.153+0000 7f4fd27fc700 1 -- 172.21.15.89:0/3550031699 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913546966 0 0) 0x7f4fd40c7de0 con 0x7f4fd406a1c0 2023-10-10T20:34:17.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.153+0000 7f4fd27fc700 1 -- 172.21.15.89:0/3550031699 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fbc003570 con 0x7f4fd406a1c0 2023-10-10T20:34:17.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd27fc700 1 -- 172.21.15.89:0/3550031699 <== mon.1 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 952232363 0 0) 0x7f4fbc003570 con 0x7f4fd406a1c0 2023-10-10T20:34:17.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd27fc700 1 -- 172.21.15.89:0/3550031699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fd4140350 msgr2=0x7f4fd4069c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-10T20:34:17.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd27fc700 1 --2- 172.21.15.89:0/3550031699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fd4140350 0x7f4fd4069c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd27fc700 1 -- 172.21.15.89:0/3550031699 >> v2:172.21.15.89:3300/0 conn(0x7f4fd4068aa0 msgr2=0x7f4fd40c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:17.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd27fc700 1 --2- 172.21.15.89:0/3550031699 >> v2:172.21.15.89:3300/0 conn(0x7f4fd4068aa0 0x7f4fd40c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd27fc700 1 -- 172.21.15.89:0/3550031699 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd4140ed0 con 0x7f4fd406a1c0 2023-10-10T20:34:17.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd27fc700 1 -- 172.21.15.89:0/3550031699 <== mon.1 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f4fc8001470 con 0x7f4fd406a1c0 2023-10-10T20:34:17.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd27fc700 1 -- 172.21.15.89:0/3550031699 >> v1:172.21.15.111:6789/0 conn(0x7f4fd406a1c0 legacy=0x7f4fd4068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:17.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd27fc700 1 --2- 172.21.15.89:0/3550031699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fbc003a60 0x7f4fbc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:17.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd27fc700 1 --2- 172.21.15.89:0/3550031699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fbc004810 0x7f4fbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:17.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd27fc700 1 --2- 172.21.15.89:0/3550031699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fd4140350 0x7f4fbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:17.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd27fc700 1 -- 172.21.15.89:0/3550031699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd4140c10 con 0x7f4fbc003a60 2023-10-10T20:34:17.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd27fc700 1 -- 172.21.15.89:0/3550031699 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd4141490 con 0x7f4fbc004810 2023-10-10T20:34:17.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd27fc700 1 -- 172.21.15.89:0/3550031699 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc00ffa0 con 0x7f4fd4140350 2023-10-10T20:34:17.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd37fe700 1 --2- 172.21.15.89:0/3550031699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fbc003a60 0x7f4fbc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:17.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd2ffd700 1 --2- 172.21.15.89:0/3550031699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fbc004810 0x7f4fbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:17.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd3fff700 1 --2- 172.21.15.89:0/3550031699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fd4140350 0x7f4fbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:17.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd2ffd700 1 -- 172.21.15.89:0/3550031699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fd4140350 msgr2=0x7f4fbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:17.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd2ffd700 1 --2- 172.21.15.89:0/3550031699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fd4140350 0x7f4fbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd2ffd700 1 -- 172.21.15.89:0/3550031699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fbc003a60 msgr2=0x7f4fbc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:17.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd2ffd700 1 --2- 172.21.15.89:0/3550031699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fbc003a60 0x7f4fbc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd2ffd700 1 -- 172.21.15.89:0/3550031699 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc003570 con 0x7f4fbc004810 2023-10-10T20:34:17.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd37fe700 1 --2- 172.21.15.89:0/3550031699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fbc003a60 0x7f4fbc003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:17.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.154+0000 7f4fd2ffd700 1 --2- 172.21.15.89:0/3550031699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fbc004810 0x7f4fbc004c80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800c250 tx=0x7f4fd40c7d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e736873c3cfa8aa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:17.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.155+0000 7f4fd27fc700 1 -- 172.21.15.89:0/3550031699 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc800cb40 con 0x7f4fbc004810 2023-10-10T20:34:17.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.155+0000 7f4fd27fc700 1 -- 172.21.15.89:0/3550031699 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc800ccc0 con 0x7f4fbc004810 2023-10-10T20:34:17.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.155+0000 7f4fd27fc700 1 -- 172.21.15.89:0/3550031699 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8012670 con 0x7f4fbc004810 2023-10-10T20:34:17.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.155+0000 7f4fda03d700 1 -- 172.21.15.89:0/3550031699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fbc004810 msgr2=0x7f4fbc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:17.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.155+0000 7f4fda03d700 1 --2- 172.21.15.89:0/3550031699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fbc004810 0x7f4fbc004c80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800c250 tx=0x7f4fd40c7d50 comp rx=0 tx=0).stop 2023-10-10T20:34:17.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.155+0000 7f4fd3fff700 1 -- 172.21.15.89:0/3550031699 reap_dead start 2023-10-10T20:34:17.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.155+0000 7f4fda03d700 1 -- 172.21.15.89:0/3550031699 shutdown_connections 2023-10-10T20:34:17.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.155+0000 7f4fda03d700 1 -- 172.21.15.89:0/3550031699 >> 172.21.15.89:0/3550031699 conn(0x7f4fd40cd670 msgr2=0x7f4fd40cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:17.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.155+0000 7f4fda03d700 1 -- 172.21.15.89:0/3550031699 shutdown_connections 2023-10-10T20:34:17.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.156+0000 7f4fda03d700 1 -- 172.21.15.89:0/3550031699 wait complete. 2023-10-10T20:34:17.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.157+0000 7f4fda03d700 1 Processor -- start 2023-10-10T20:34:17.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.157+0000 7f4fda03d700 1 -- start start 2023-10-10T20:34:17.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.158+0000 7f4fda03d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fd413c050 0x7f4fd4157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:17.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.158+0000 7f4fda03d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fd413c420 0x7f4fd40c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:17.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.158+0000 7f4fda03d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fd40c9b00 0x7f4fd40b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:17.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.158+0000 7f4fda03d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd40c9fa0 con 0x7f4fd413c050 2023-10-10T20:34:17.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.158+0000 7f4fda03d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd40b1fa0 con 0x7f4fd40c9b00 2023-10-10T20:34:17.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.158+0000 7f4fda03d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd41559e0 con 0x7f4fd413c420 2023-10-10T20:34:17.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.158+0000 7f4fd37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fd413c050 0x7f4fd4157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:17.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.158+0000 7f4fd37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fd413c050 0x7f4fd4157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50952/0 (socket says 172.21.15.89:50952) 2023-10-10T20:34:17.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.158+0000 7f4fd37fe700 1 -- 172.21.15.89:0/2916860399 learned_addr learned my addr 172.21.15.89:0/2916860399 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:17.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.158+0000 7f4fd2ffd700 1 --2- 172.21.15.89:0/2916860399 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fd413c420 0x7f4fd40c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:17.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.158+0000 7f4fd3fff700 1 --2- 172.21.15.89:0/2916860399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fd40c9b00 0x7f4fd40b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:17.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.158+0000 7f4fd37fe700 1 -- 172.21.15.89:0/2916860399 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fd413c420 msgr2=0x7f4fd40c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:17.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.158+0000 7f4fd37fe700 1 --2- 172.21.15.89:0/2916860399 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fd413c420 0x7f4fd40c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.158+0000 7f4fd37fe700 1 -- 172.21.15.89:0/2916860399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fd40c9b00 msgr2=0x7f4fd40b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:17.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.158+0000 7f4fd37fe700 1 --2- 172.21.15.89:0/2916860399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fd40c9b00 0x7f4fd40b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.158+0000 7f4fd37fe700 1 -- 172.21.15.89:0/2916860399 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd40b1950 con 0x7f4fd413c050 2023-10-10T20:34:17.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.158+0000 7f4fd37fe700 1 --2- 172.21.15.89:0/2916860399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fd413c050 0x7f4fd4157ec0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4007bf0 tx=0x7f4fc4016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:17.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.160+0000 7f4fbb7fe700 1 -- 172.21.15.89:0/2916860399 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4019040 con 0x7f4fd413c050 2023-10-10T20:34:17.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.160+0000 7f4fbb7fe700 1 -- 172.21.15.89:0/2916860399 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc40142e0 con 0x7f4fd413c050 2023-10-10T20:34:17.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.160+0000 7f4fda03d700 1 -- 172.21.15.89:0/2916860399 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd40b1c30 con 0x7f4fd413c050 2023-10-10T20:34:17.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.160+0000 7f4fbb7fe700 1 -- 172.21.15.89:0/2916860399 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc40146a0 con 0x7f4fd413c050 2023-10-10T20:34:17.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.160+0000 7f4fda03d700 1 -- 172.21.15.89:0/2916860399 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd4158400 con 0x7f4fd413c050 2023-10-10T20:34:17.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.160+0000 7f4fbb7fe700 1 -- 172.21.15.89:0/2916860399 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4fc4012a60 con 0x7f4fd413c050 2023-10-10T20:34:17.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.160+0000 7f4fbb7fe700 1 -- 172.21.15.89:0/2916860399 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f4fc401e470 con 0x7f4fd413c050 2023-10-10T20:34:17.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.162+0000 7f4fda03d700 1 -- 172.21.15.89:0/2916860399 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd404c3c0 con 0x7f4fd413c050 2023-10-10T20:34:17.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.164+0000 7f4fbb7fe700 1 -- 172.21.15.89:0/2916860399 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f4fc401e680 con 0x7f4fd413c050 2023-10-10T20:34:17.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.253+0000 7f4fda03d700 1 -- 172.21.15.89:0/2916860399 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f4fd41411b0 con 0x7f4fd413c050 2023-10-10T20:34:17.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.260+0000 7f4fbb7fe700 1 -- 172.21.15.89:0/2916860399 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4fc401e820 con 0x7f4fd413c050 2023-10-10T20:34:17.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.261+0000 7f4fda03d700 1 -- 172.21.15.89:0/2916860399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fd413c050 msgr2=0x7f4fd4157ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:17.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.261+0000 7f4fda03d700 1 --2- 172.21.15.89:0/2916860399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fd413c050 0x7f4fd4157ec0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4007bf0 tx=0x7f4fc4016040 comp rx=0 tx=0).stop 2023-10-10T20:34:17.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.261+0000 7f4fda03d700 1 -- 172.21.15.89:0/2916860399 shutdown_connections 2023-10-10T20:34:17.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.261+0000 7f4fda03d700 1 --2- 172.21.15.89:0/2916860399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fd40c9b00 0x7f4fd40b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.261+0000 7f4fda03d700 1 --2- 172.21.15.89:0/2916860399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fd413c050 0x7f4fd4157ec0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.261+0000 7f4fda03d700 1 --2- 172.21.15.89:0/2916860399 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fd413c420 0x7f4fd40c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.261+0000 7f4fda03d700 1 -- 172.21.15.89:0/2916860399 >> 172.21.15.89:0/2916860399 conn(0x7f4fd40cd670 msgr2=0x7f4fd4157420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:17.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.261+0000 7f4fda03d700 1 -- 172.21.15.89:0/2916860399 shutdown_connections 2023-10-10T20:34:17.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:17.261+0000 7f4fda03d700 1 -- 172.21.15.89:0/2916860399 wait complete. 2023-10-10T20:34:17.266 INFO:teuthology.orchestra.run.smithi089.stderr:adjusted tunables profile to default 2023-10-10T20:34:17.275 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-10T20:34:17.275 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:34:17.276 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-10T20:34:17.300 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:34:17.300 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-10T20:34:17.364 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:34:17.365 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-10T20:34:17.429 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-10T20:34:17.430 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-10T20:34:17.494 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:34:17.495 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-10T20:34:17.520 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:34:17.521 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-10T20:34:17.585 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:34:17.585 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-10T20:34:17.650 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-10-10T20:34:17.650 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-10T20:34:17.715 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd new 824a5c96-7951-4900-849d-6d6eee4f103f 0 2023-10-10T20:34:17.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.869+0000 7f9e34c9b700 1 Processor -- start 2023-10-10T20:34:17.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.869+0000 7f9e34c9b700 1 -- start start 2023-10-10T20:34:17.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.870+0000 7f9e34c9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9e3006bcd0 0x7f9e300caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:17.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.870+0000 7f9e34c9b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9e3006c0a0 0x7f9e3015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:17.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.870+0000 7f9e34c9b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9e3015df50 0x7f9e3015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:17.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.870+0000 7f9e34c9b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3015d7e0 con 0x7f9e3006bcd0 2023-10-10T20:34:17.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.870+0000 7f9e34c9b700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3015dac0 con 0x7f9e3015df50 2023-10-10T20:34:17.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.870+0000 7f9e34c9b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3015aa30 con 0x7f9e3006c0a0 2023-10-10T20:34:17.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.870+0000 7f9e2dd9b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9e3006c0a0 0x7f9e3015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:17.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.870+0000 7f9e2ed9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9e3015df50 0x7f9e3015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:17.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.870+0000 7f9e2dd9b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9e3006c0a0 0x7f9e3015a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:32804/0 (socket says 172.21.15.111:32804) 2023-10-10T20:34:17.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.870+0000 7f9e2ed9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9e3015df50 0x7f9e3015e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39144/0 (socket says 172.21.15.111:39144) 2023-10-10T20:34:17.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.870+0000 7f9e2dd9b700 1 -- 172.21.15.111:0/2679676623 learned_addr learned my addr 172.21.15.111:0/2679676623 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-10-10T20:34:17.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.870+0000 7f9e2e59c700 1 --2- 172.21.15.111:0/2679676623 >> v2:172.21.15.89:3300/0 conn(0x7f9e3006bcd0 0x7f9e300caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:17.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.870+0000 7f9e2dd9b700 1 -- 172.21.15.111:0/2679676623 >> v2:172.21.15.111:3300/0 conn(0x7f9e3015df50 msgr2=0x7f9e3015e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:17.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.870+0000 7f9e2dd9b700 1 --2- 172.21.15.111:0/2679676623 >> v2:172.21.15.111:3300/0 conn(0x7f9e3015df50 0x7f9e3015e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.870+0000 7f9e2dd9b700 1 -- 172.21.15.111:0/2679676623 >> v2:172.21.15.89:3300/0 conn(0x7f9e3006bcd0 msgr2=0x7f9e300caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:17.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.870+0000 7f9e2dd9b700 1 --2- 172.21.15.111:0/2679676623 >> v2:172.21.15.89:3300/0 conn(0x7f9e3006bcd0 0x7f9e300caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.870+0000 7f9e2dd9b700 1 -- 172.21.15.111:0/2679676623 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3015ad10 con 0x7f9e3006c0a0 2023-10-10T20:34:17.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.870+0000 7f9e2ed9d700 1 --2- 172.21.15.111:0/2679676623 >> v2:172.21.15.111:3300/0 conn(0x7f9e3015df50 0x7f9e3015e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:17.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.871+0000 7f9e2dd9b700 1 --2- 172.21.15.111:0/2679676623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9e3006c0a0 0x7f9e3015a1e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9e300b4600 tx=0x7f9e20009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=912848dc12f00408 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:17.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.871+0000 7f9e2d59a700 1 -- 172.21.15.111:0/2679676623 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2000b640 con 0x7f9e3006c0a0 2023-10-10T20:34:17.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.871+0000 7f9e2d59a700 1 -- 172.21.15.111:0/2679676623 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2000b7a0 con 0x7f9e3006c0a0 2023-10-10T20:34:17.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.871+0000 7f9e2d59a700 1 -- 172.21.15.111:0/2679676623 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2000bb40 con 0x7f9e3006c0a0 2023-10-10T20:34:17.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.871+0000 7f9e34c9b700 1 -- 172.21.15.111:0/2679676623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9e3006c0a0 msgr2=0x7f9e3015a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:17.879 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.871+0000 7f9e34c9b700 1 --2- 172.21.15.111:0/2679676623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9e3006c0a0 0x7f9e3015a1e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9e300b4600 tx=0x7f9e20009250 comp rx=0 tx=0).stop 2023-10-10T20:34:17.879 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.872+0000 7f9e34c9b700 1 -- 172.21.15.111:0/2679676623 shutdown_connections 2023-10-10T20:34:17.879 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.872+0000 7f9e34c9b700 1 --2- 172.21.15.111:0/2679676623 >> v2:172.21.15.111:3300/0 conn(0x7f9e3015df50 0x7f9e3015e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.879 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.872+0000 7f9e34c9b700 1 --2- 172.21.15.111:0/2679676623 >> v2:172.21.15.89:3300/0 conn(0x7f9e3006bcd0 0x7f9e300caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.879 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.872+0000 7f9e34c9b700 1 --2- 172.21.15.111:0/2679676623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9e3006c0a0 0x7f9e3015a1e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.879 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.872+0000 7f9e34c9b700 1 -- 172.21.15.111:0/2679676623 >> 172.21.15.111:0/2679676623 conn(0x7f9e300ce600 msgr2=0x7f9e300cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:17.880 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.872+0000 7f9e34c9b700 1 -- 172.21.15.111:0/2679676623 shutdown_connections 2023-10-10T20:34:17.880 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.872+0000 7f9e34c9b700 1 -- 172.21.15.111:0/2679676623 wait complete. 2023-10-10T20:34:17.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.873+0000 7f9e34c9b700 1 Processor -- start 2023-10-10T20:34:17.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.873+0000 7f9e34c9b700 1 -- start start 2023-10-10T20:34:17.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.873+0000 7f9e34c9b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9e3006bcd0 0x7f9e3016a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:17.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.873+0000 7f9e34c9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e3006c0a0 0x7f9e300c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:17.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.873+0000 7f9e34c9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e3015df50 0x7f9e300b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:17.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.873+0000 7f9e34c9b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e300c9110 con 0x7f9e3015df50 2023-10-10T20:34:17.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.873+0000 7f9e34c9b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e300c5740 con 0x7f9e3006c0a0 2023-10-10T20:34:17.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.873+0000 7f9e34c9b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e300b4060 con 0x7f9e3006bcd0 2023-10-10T20:34:17.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.874+0000 7f9e2e59c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9e3006bcd0 0x7f9e3016a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:17.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.874+0000 7f9e2dd9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e3006c0a0 0x7f9e300c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:17.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.874+0000 7f9e2e59c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9e3006bcd0 0x7f9e3016a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:32818/0 (socket says 172.21.15.111:32818) 2023-10-10T20:34:17.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.874+0000 7f9e2e59c700 1 -- 172.21.15.111:0/3841322214 learned_addr learned my addr 172.21.15.111:0/3841322214 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-10-10T20:34:17.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.874+0000 7f9e2ed9d700 1 --2- 172.21.15.111:0/3841322214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e3015df50 0x7f9e300b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:17.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.874+0000 7f9e2e59c700 1 -- 172.21.15.111:0/3841322214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e3006c0a0 msgr2=0x7f9e300c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:17.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.874+0000 7f9e2e59c700 1 --2- 172.21.15.111:0/3841322214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e3006c0a0 0x7f9e300c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.874+0000 7f9e2e59c700 1 -- 172.21.15.111:0/3841322214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e3015df50 msgr2=0x7f9e300b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:17.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.874+0000 7f9e2e59c700 1 --2- 172.21.15.111:0/3841322214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e3015df50 0x7f9e300b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.874+0000 7f9e2e59c700 1 -- 172.21.15.111:0/3841322214 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e28009a20 con 0x7f9e3006bcd0 2023-10-10T20:34:17.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.874+0000 7f9e2dd9b700 1 --2- 172.21.15.111:0/3841322214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e3006c0a0 0x7f9e300c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:17.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.874+0000 7f9e2e59c700 1 --2- 172.21.15.111:0/3841322214 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9e3006bcd0 0x7f9e3016a380 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400e930 tx=0x7f9e240091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:17.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.874+0000 7f9e1e7fc700 1 -- 172.21.15.111:0/3841322214 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e24013680 con 0x7f9e3006bcd0 2023-10-10T20:34:17.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.874+0000 7f9e1e7fc700 1 -- 172.21.15.111:0/3841322214 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e240137e0 con 0x7f9e3006bcd0 2023-10-10T20:34:17.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.874+0000 7f9e1e7fc700 1 -- 172.21.15.111:0/3841322214 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e24013b80 con 0x7f9e3006bcd0 2023-10-10T20:34:17.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.874+0000 7f9e34c9b700 1 -- 172.21.15.111:0/3841322214 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e200094f0 con 0x7f9e3006bcd0 2023-10-10T20:34:17.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.874+0000 7f9e34c9b700 1 -- 172.21.15.111:0/3841322214 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3016ac40 con 0x7f9e3006bcd0 2023-10-10T20:34:17.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.874+0000 7f9e1e7fc700 1 -- 172.21.15.111:0/3841322214 <== mon.2 v2:172.21.15.111:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f9e24005c80 con 0x7f9e3006bcd0 2023-10-10T20:34:17.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.874+0000 7f9e1e7fc700 1 -- 172.21.15.111:0/3841322214 <== mon.2 v2:172.21.15.111:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9e2400c460 con 0x7f9e3006bcd0 2023-10-10T20:34:17.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.875+0000 7f9e34c9b700 1 -- 172.21.15.111:0/3841322214 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e3004c3c0 con 0x7f9e3006bcd0 2023-10-10T20:34:17.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.877+0000 7f9e1e7fc700 1 -- 172.21.15.111:0/3841322214 <== mon.2 v2:172.21.15.111:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f9e24021060 con 0x7f9e3006bcd0 2023-10-10T20:34:17.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.960+0000 7f9e34c9b700 1 -- 172.21.15.111:0/3841322214 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "osd new", "uuid": "824a5c96-7951-4900-849d-6d6eee4f103f", "id": 0} v 0) v1 -- 0x7f9e3015aff0 con 0x7f9e3006bcd0 2023-10-10T20:34:17.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.965+0000 7f9e1e7fc700 1 -- 172.21.15.111:0/3841322214 <== mon.2 v2:172.21.15.111:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "824a5c96-7951-4900-849d-6d6eee4f103f", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9e24017070 con 0x7f9e3006bcd0 2023-10-10T20:34:17.968 INFO:teuthology.orchestra.run.smithi111.stdout:0 2023-10-10T20:34:17.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.966+0000 7f9e34c9b700 1 -- 172.21.15.111:0/3841322214 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9e3006bcd0 msgr2=0x7f9e3016a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:17.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.966+0000 7f9e34c9b700 1 --2- 172.21.15.111:0/3841322214 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9e3006bcd0 0x7f9e3016a380 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400e930 tx=0x7f9e240091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.966+0000 7f9e34c9b700 1 -- 172.21.15.111:0/3841322214 shutdown_connections 2023-10-10T20:34:17.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.966+0000 7f9e34c9b700 1 --2- 172.21.15.111:0/3841322214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e3015df50 0x7f9e300b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.966+0000 7f9e34c9b700 1 --2- 172.21.15.111:0/3841322214 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9e3006bcd0 0x7f9e3016a380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.966+0000 7f9e34c9b700 1 --2- 172.21.15.111:0/3841322214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e3006c0a0 0x7f9e300c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:17.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.966+0000 7f9e34c9b700 1 -- 172.21.15.111:0/3841322214 >> 172.21.15.111:0/3841322214 conn(0x7f9e300ce600 msgr2=0x7f9e30156420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:17.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.966+0000 7f9e34c9b700 1 -- 172.21.15.111:0/3841322214 shutdown_connections 2023-10-10T20:34:17.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:17.966+0000 7f9e34c9b700 1 -- 172.21.15.111:0/3841322214 wait complete. 2023-10-10T20:34:17.980 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd new 2292ce8e-5b0e-403d-a78c-1bb6ef0fab05 1 2023-10-10T20:34:18.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.093+0000 7fe76c210700 1 Processor -- start 2023-10-10T20:34:18.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.093+0000 7fe76c210700 1 -- start start 2023-10-10T20:34:18.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.094+0000 7fe76c210700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe764159c60 0x7fe76415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.094+0000 7fe76c210700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe76415db10 0x7fe764069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.094+0000 7fe76c210700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe76415a160 con 0x7fe764159890 2023-10-10T20:34:18.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.094+0000 7fe76c210700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7640d13a0 con 0x7fe76415db10 2023-10-10T20:34:18.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.094+0000 7fe76c210700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76415e140 con 0x7fe764159c60 2023-10-10T20:34:18.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.095+0000 7fe7697ab700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe764159c60 0x7fe76415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.095+0000 7fe76a7ad700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe76415db10 0x7fe764069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.095+0000 7fe7697ab700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe764159c60 0x7fe76415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:32822/0 (socket says 172.21.15.111:32822) 2023-10-10T20:34:18.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.095+0000 7fe76a7ad700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe76415db10 0x7fe764069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39158/0 (socket says 172.21.15.111:39158) 2023-10-10T20:34:18.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.095+0000 7fe7697ab700 1 -- 172.21.15.111:0/1632815357 learned_addr learned my addr 172.21.15.111:0/1632815357 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-10-10T20:34:18.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.095+0000 7fe76a7ad700 1 -- 172.21.15.111:0/1632815357 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe764159c60 msgr2=0x7fe76415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:18.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.095+0000 7fe76a7ad700 1 --2- 172.21.15.111:0/1632815357 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe764159c60 0x7fe76415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.095+0000 7fe76a7ad700 1 -- 172.21.15.111:0/1632815357 >> v1:172.21.15.89:6789/0 conn(0x7fe764159890 legacy=0x7fe76415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.095+0000 7fe76a7ad700 1 -- 172.21.15.111:0/1632815357 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76415a3d0 con 0x7fe76415db10 2023-10-10T20:34:18.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.095+0000 7fe7697ab700 1 --2- 172.21.15.111:0/1632815357 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe764159c60 0x7fe76415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:18.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.095+0000 7fe76a7ad700 1 --2- 172.21.15.111:0/1632815357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe76415db10 0x7fe764069560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009e60 tx=0x7fe75c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=524939e941ce9500 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:18.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.095+0000 7fe768faa700 1 -- 172.21.15.111:0/1632815357 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00e070 con 0x7fe76415db10 2023-10-10T20:34:18.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.095+0000 7fe768faa700 1 -- 172.21.15.111:0/1632815357 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c00bb60 con 0x7fe76415db10 2023-10-10T20:34:18.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.095+0000 7fe768faa700 1 -- 172.21.15.111:0/1632815357 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c012430 con 0x7fe76415db10 2023-10-10T20:34:18.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.096+0000 7fe76c210700 1 -- 172.21.15.111:0/1632815357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe76415db10 msgr2=0x7fe764069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.096+0000 7fe76c210700 1 --2- 172.21.15.111:0/1632815357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe76415db10 0x7fe764069560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009e60 tx=0x7fe75c009780 comp rx=0 tx=0).stop 2023-10-10T20:34:18.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.096+0000 7fe76c210700 1 -- 172.21.15.111:0/1632815357 shutdown_connections 2023-10-10T20:34:18.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.096+0000 7fe76c210700 1 --2- 172.21.15.111:0/1632815357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe76415db10 0x7fe764069560 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.096+0000 7fe76c210700 1 --2- 172.21.15.111:0/1632815357 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe764159c60 0x7fe76415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.096+0000 7fe76c210700 1 -- 172.21.15.111:0/1632815357 >> 172.21.15.111:0/1632815357 conn(0x7fe764068e20 msgr2=0x7fe7640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:18.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.096+0000 7fe76c210700 1 -- 172.21.15.111:0/1632815357 shutdown_connections 2023-10-10T20:34:18.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.096+0000 7fe76c210700 1 -- 172.21.15.111:0/1632815357 wait complete. 2023-10-10T20:34:18.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.098+0000 7fe76c210700 1 Processor -- start 2023-10-10T20:34:18.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.098+0000 7fe76c210700 1 -- start start 2023-10-10T20:34:18.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.098+0000 7fe76c210700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe764159890 0x7fe7640ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.098+0000 7fe76c210700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe764159c60 0x7fe764158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.098+0000 7fe76c210700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe76415db10 0x7fe7640b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.098+0000 7fe76c210700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76416cfe0 con 0x7fe764159890 2023-10-10T20:34:18.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.098+0000 7fe76c210700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76416db60 con 0x7fe76415db10 2023-10-10T20:34:18.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.098+0000 7fe76c210700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76416d230 con 0x7fe764159c60 2023-10-10T20:34:18.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.098+0000 7fe7697ab700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe764159c60 0x7fe764158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.099+0000 7fe7697ab700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe764159c60 0x7fe764158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:32832/0 (socket says 172.21.15.111:32832) 2023-10-10T20:34:18.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.099+0000 7fe7697ab700 1 -- 172.21.15.111:0/2861540628 learned_addr learned my addr 172.21.15.111:0/2861540628 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-10-10T20:34:18.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.099+0000 7fe76a7ad700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe76415db10 0x7fe7640b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.099+0000 7fe769fac700 1 --2- 172.21.15.111:0/2861540628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe764159890 0x7fe7640ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.099+0000 7fe7697ab700 1 -- 172.21.15.111:0/2861540628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe76415db10 msgr2=0x7fe7640b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.099+0000 7fe7697ab700 1 --2- 172.21.15.111:0/2861540628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe76415db10 0x7fe7640b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.099+0000 7fe7697ab700 1 -- 172.21.15.111:0/2861540628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe764159890 msgr2=0x7fe7640ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.099+0000 7fe7697ab700 1 --2- 172.21.15.111:0/2861540628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe764159890 0x7fe7640ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.099+0000 7fe7697ab700 1 -- 172.21.15.111:0/2861540628 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7600094f0 con 0x7fe764159c60 2023-10-10T20:34:18.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.099+0000 7fe76a7ad700 1 --2- 172.21.15.111:0/2861540628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe76415db10 0x7fe7640b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:18.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.099+0000 7fe7697ab700 1 --2- 172.21.15.111:0/2861540628 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe764159c60 0x7fe764158440 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe76000ecb0 tx=0x7fe760009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:18.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.099+0000 7fe759ffb700 1 -- 172.21.15.111:0/2861540628 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76000c920 con 0x7fe764159c60 2023-10-10T20:34:18.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.099+0000 7fe759ffb700 1 -- 172.21.15.111:0/2861540628 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76000ca80 con 0x7fe764159c60 2023-10-10T20:34:18.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.099+0000 7fe76c210700 1 -- 172.21.15.111:0/2861540628 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75c009a20 con 0x7fe764159c60 2023-10-10T20:34:18.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.099+0000 7fe759ffb700 1 -- 172.21.15.111:0/2861540628 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76000ce00 con 0x7fe764159c60 2023-10-10T20:34:18.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.099+0000 7fe76c210700 1 -- 172.21.15.111:0/2861540628 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76416d7f0 con 0x7fe764159c60 2023-10-10T20:34:18.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.100+0000 7fe759ffb700 1 -- 172.21.15.111:0/2861540628 <== mon.2 v2:172.21.15.111:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fe760005c80 con 0x7fe764159c60 2023-10-10T20:34:18.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.100+0000 7fe759ffb700 1 -- 172.21.15.111:0/2861540628 <== mon.2 v2:172.21.15.111:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fe7600063e0 con 0x7fe764159c60 2023-10-10T20:34:18.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.100+0000 7fe76c210700 1 -- 172.21.15.111:0/2861540628 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe76404c3c0 con 0x7fe764159c60 2023-10-10T20:34:18.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.102+0000 7fe759ffb700 1 -- 172.21.15.111:0/2861540628 <== mon.2 v2:172.21.15.111:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fe760021060 con 0x7fe764159c60 2023-10-10T20:34:18.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.185+0000 7fe76c210700 1 -- 172.21.15.111:0/2861540628 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2292ce8e-5b0e-403d-a78c-1bb6ef0fab05", "id": 1} v 0) v1 -- 0x7fe76416dca0 con 0x7fe764159c60 2023-10-10T20:34:18.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.189+0000 7fe759ffb700 1 -- 172.21.15.111:0/2861540628 <== mon.2 v2:172.21.15.111:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2292ce8e-5b0e-403d-a78c-1bb6ef0fab05", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe760017070 con 0x7fe764159c60 2023-10-10T20:34:18.192 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-10-10T20:34:18.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.190+0000 7fe76c210700 1 -- 172.21.15.111:0/2861540628 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe764159c60 msgr2=0x7fe764158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.190+0000 7fe76c210700 1 --2- 172.21.15.111:0/2861540628 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe764159c60 0x7fe764158440 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe76000ecb0 tx=0x7fe760009250 comp rx=0 tx=0).stop 2023-10-10T20:34:18.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.190+0000 7fe76c210700 1 -- 172.21.15.111:0/2861540628 shutdown_connections 2023-10-10T20:34:18.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.190+0000 7fe76c210700 1 --2- 172.21.15.111:0/2861540628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe76415db10 0x7fe7640b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.190+0000 7fe76c210700 1 --2- 172.21.15.111:0/2861540628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe764159890 0x7fe7640ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.190+0000 7fe76c210700 1 --2- 172.21.15.111:0/2861540628 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe764159c60 0x7fe764158440 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.190+0000 7fe76c210700 1 -- 172.21.15.111:0/2861540628 >> 172.21.15.111:0/2861540628 conn(0x7fe764068e20 msgr2=0x7fe7641530f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:18.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.190+0000 7fe76c210700 1 -- 172.21.15.111:0/2861540628 shutdown_connections 2023-10-10T20:34:18.196 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.190+0000 7fe76c210700 1 -- 172.21.15.111:0/2861540628 wait complete. 2023-10-10T20:34:18.206 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd new f8ffe4e4-71f7-4b8c-9010-d4102f63af46 2 2023-10-10T20:34:18.318 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:18.316+0000 7f02c1de9180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-10T20:34:18.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.322+0000 7f5aa4081700 1 Processor -- start 2023-10-10T20:34:18.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.322+0000 7f5aa4081700 1 -- start start 2023-10-10T20:34:18.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.323+0000 7f5aa4081700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5a9c159b10 0x7f5a9c0cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.323+0000 7f5aa4081700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a9c06b420 0x7f5a9c15dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.323+0000 7f5aa4081700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9c0c74b0 con 0x7f5a9c15d730 2023-10-10T20:34:18.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.323+0000 7f5aa4081700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a9c0c8700 con 0x7f5a9c159b10 2023-10-10T20:34:18.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.323+0000 7f5aa4081700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c0c9070 con 0x7f5a9c06b420 2023-10-10T20:34:18.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.323+0000 7f5aa261e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a9c06b420 0x7f5a9c15dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.323+0000 7f5aa1e1d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5a9c159b10 0x7f5a9c0cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.323+0000 7f5aa261e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a9c06b420 0x7f5a9c15dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:32846/0 (socket says 172.21.15.111:32846) 2023-10-10T20:34:18.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.323+0000 7f5aa261e700 1 -- 172.21.15.111:0/1926369421 learned_addr learned my addr 172.21.15.111:0/1926369421 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-10-10T20:34:18.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.323+0000 7f5aa261e700 1 -- 172.21.15.111:0/1926369421 >> v2:172.21.15.111:3300/0 conn(0x7f5a9c159b10 msgr2=0x7f5a9c0cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:18.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.323+0000 7f5aa261e700 1 --2- 172.21.15.111:0/1926369421 >> v2:172.21.15.111:3300/0 conn(0x7f5a9c159b10 0x7f5a9c0cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.323+0000 7f5aa261e700 1 -- 172.21.15.111:0/1926369421 >> v1:172.21.15.89:6789/0 conn(0x7f5a9c15d730 legacy=0x7f5a9c159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.323+0000 7f5aa261e700 1 -- 172.21.15.111:0/1926369421 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c15e0e0 con 0x7f5a9c06b420 2023-10-10T20:34:18.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.323+0000 7f5aa1e1d700 1 --2- 172.21.15.111:0/1926369421 >> v2:172.21.15.111:3300/0 conn(0x7f5a9c159b10 0x7f5a9c0cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:18.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.324+0000 7f5aa261e700 1 --2- 172.21.15.111:0/1926369421 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a9c06b420 0x7f5a9c15dba0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5a94009f20 tx=0x7f5a94009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4f57e176e7f401a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:18.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.324+0000 7f5aa0e1b700 1 -- 172.21.15.111:0/1926369421 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9400e070 con 0x7f5a9c06b420 2023-10-10T20:34:18.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.324+0000 7f5aa0e1b700 1 -- 172.21.15.111:0/1926369421 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9400ba60 con 0x7f5a9c06b420 2023-10-10T20:34:18.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.324+0000 7f5aa0e1b700 1 -- 172.21.15.111:0/1926369421 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a940092e0 con 0x7f5a9c06b420 2023-10-10T20:34:18.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.324+0000 7f5aa4081700 1 -- 172.21.15.111:0/1926369421 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a9c06b420 msgr2=0x7f5a9c15dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.324+0000 7f5aa4081700 1 --2- 172.21.15.111:0/1926369421 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a9c06b420 0x7f5a9c15dba0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5a94009f20 tx=0x7f5a94009840 comp rx=0 tx=0).stop 2023-10-10T20:34:18.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.324+0000 7f5aa4081700 1 -- 172.21.15.111:0/1926369421 shutdown_connections 2023-10-10T20:34:18.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.324+0000 7f5aa4081700 1 --2- 172.21.15.111:0/1926369421 >> v2:172.21.15.111:3300/0 conn(0x7f5a9c159b10 0x7f5a9c0cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.324+0000 7f5aa4081700 1 --2- 172.21.15.111:0/1926369421 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a9c06b420 0x7f5a9c15dba0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.324+0000 7f5aa4081700 1 -- 172.21.15.111:0/1926369421 >> 172.21.15.111:0/1926369421 conn(0x7f5a9c0c9870 msgr2=0x7f5a9c0c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:18.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.325+0000 7f5aa4081700 1 -- 172.21.15.111:0/1926369421 shutdown_connections 2023-10-10T20:34:18.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.325+0000 7f5aa4081700 1 -- 172.21.15.111:0/1926369421 wait complete. 2023-10-10T20:34:18.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.326+0000 7f5aa4081700 1 Processor -- start 2023-10-10T20:34:18.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.326+0000 7f5aa4081700 1 -- start start 2023-10-10T20:34:18.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.326+0000 7f5aa4081700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a9c06b420 0x7f5a9c143430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.326+0000 7f5aa4081700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9c159b10 0x7f5a9c157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.336 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.326+0000 7f5aa4081700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a9c15d730 0x7f5a9c157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.336 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.326+0000 7f5aa4081700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c15d570 con 0x7f5a9c159b10 2023-10-10T20:34:18.336 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.326+0000 7f5aa4081700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c144150 con 0x7f5a9c15d730 2023-10-10T20:34:18.336 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.326+0000 7f5aa4081700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c15cfb0 con 0x7f5a9c06b420 2023-10-10T20:34:18.336 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.326+0000 7f5aa1e1d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a9c06b420 0x7f5a9c143430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.326+0000 7f5aa1e1d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a9c06b420 0x7f5a9c143430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:32854/0 (socket says 172.21.15.111:32854) 2023-10-10T20:34:18.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.326+0000 7f5aa1e1d700 1 -- 172.21.15.111:0/1095547295 learned_addr learned my addr 172.21.15.111:0/1095547295 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-10-10T20:34:18.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.326+0000 7f5aa261e700 1 --2- 172.21.15.111:0/1095547295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a9c15d730 0x7f5a9c157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.326+0000 7f5aa161c700 1 --2- 172.21.15.111:0/1095547295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9c159b10 0x7f5a9c157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.327+0000 7f5aa1e1d700 1 -- 172.21.15.111:0/1095547295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a9c15d730 msgr2=0x7f5a9c157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.327+0000 7f5aa1e1d700 1 --2- 172.21.15.111:0/1095547295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a9c15d730 0x7f5a9c157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.327+0000 7f5aa1e1d700 1 -- 172.21.15.111:0/1095547295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9c159b10 msgr2=0x7f5a9c157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.327+0000 7f5aa1e1d700 1 --2- 172.21.15.111:0/1095547295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9c159b10 0x7f5a9c157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.327+0000 7f5aa1e1d700 1 -- 172.21.15.111:0/1095547295 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c009420 con 0x7f5a9c06b420 2023-10-10T20:34:18.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.327+0000 7f5aa1e1d700 1 --2- 172.21.15.111:0/1095547295 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a9c06b420 0x7f5a9c143430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00ec80 tx=0x7f5a8c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:18.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.327+0000 7f5a91ffb700 1 -- 172.21.15.111:0/1095547295 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c00c920 con 0x7f5a9c06b420 2023-10-10T20:34:18.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.327+0000 7f5a91ffb700 1 -- 172.21.15.111:0/1095547295 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8c00ca80 con 0x7f5a9c06b420 2023-10-10T20:34:18.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.327+0000 7f5aa4081700 1 -- 172.21.15.111:0/1095547295 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a94009ae0 con 0x7f5a9c06b420 2023-10-10T20:34:18.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.327+0000 7f5a91ffb700 1 -- 172.21.15.111:0/1095547295 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c00ce00 con 0x7f5a9c06b420 2023-10-10T20:34:18.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.327+0000 7f5aa4081700 1 -- 172.21.15.111:0/1095547295 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9c143de0 con 0x7f5a9c06b420 2023-10-10T20:34:18.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.327+0000 7f5a91ffb700 1 -- 172.21.15.111:0/1095547295 <== mon.2 v2:172.21.15.111:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f5a8c013900 con 0x7f5a9c06b420 2023-10-10T20:34:18.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.327+0000 7f5a91ffb700 1 -- 172.21.15.111:0/1095547295 <== mon.2 v2:172.21.15.111:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f5a8c01cba0 con 0x7f5a9c06b420 2023-10-10T20:34:18.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.328+0000 7f5aa4081700 1 -- 172.21.15.111:0/1095547295 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a9c144430 con 0x7f5a9c06b420 2023-10-10T20:34:18.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.333+0000 7f5a91ffb700 1 -- 172.21.15.111:0/1095547295 <== mon.2 v2:172.21.15.111:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f5a8c03b6d0 con 0x7f5a9c06b420 2023-10-10T20:34:18.369 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:18.367+0000 7fb98747c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-10T20:34:18.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.414+0000 7f5aa4081700 1 -- 172.21.15.111:0/1095547295 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f8ffe4e4-71f7-4b8c-9010-d4102f63af46", "id": 2} v 0) v1 -- 0x7f5a9c04c3c0 con 0x7f5a9c06b420 2023-10-10T20:34:18.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.418+0000 7f5a91ffb700 1 -- 172.21.15.111:0/1095547295 <== mon.2 v2:172.21.15.111:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f8ffe4e4-71f7-4b8c-9010-d4102f63af46", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5a8c02a030 con 0x7f5a9c06b420 2023-10-10T20:34:18.421 INFO:teuthology.orchestra.run.smithi111.stdout:2 2023-10-10T20:34:18.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.419+0000 7f5aa4081700 1 -- 172.21.15.111:0/1095547295 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a9c06b420 msgr2=0x7f5a9c143430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.419+0000 7f5aa4081700 1 --2- 172.21.15.111:0/1095547295 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a9c06b420 0x7f5a9c143430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00ec80 tx=0x7f5a8c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.419+0000 7f5aa4081700 1 -- 172.21.15.111:0/1095547295 shutdown_connections 2023-10-10T20:34:18.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.419+0000 7f5aa4081700 1 --2- 172.21.15.111:0/1095547295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a9c15d730 0x7f5a9c157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.419+0000 7f5aa4081700 1 --2- 172.21.15.111:0/1095547295 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a9c06b420 0x7f5a9c143430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.419+0000 7f5aa4081700 1 --2- 172.21.15.111:0/1095547295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9c159b10 0x7f5a9c157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.419+0000 7f5aa4081700 1 -- 172.21.15.111:0/1095547295 >> 172.21.15.111:0/1095547295 conn(0x7f5a9c0c9870 msgr2=0x7f5a9c0695b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:18.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.420+0000 7f5aa4081700 1 -- 172.21.15.111:0/1095547295 shutdown_connections 2023-10-10T20:34:18.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.420+0000 7f5aa4081700 1 -- 172.21.15.111:0/1095547295 wait complete. 2023-10-10T20:34:18.434 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd new f065bed3-b82e-489b-a87b-57a5d762dbee 3 2023-10-10T20:34:18.450 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:18.448+0000 7f02c1de9180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-10T20:34:18.508 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:18.505+0000 7fb98747c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-10T20:34:18.525 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:18.523+0000 7f02c1de9180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-10T20:34:18.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.547+0000 7f3d7598c700 1 Processor -- start 2023-10-10T20:34:18.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.547+0000 7f3d7598c700 1 -- start start 2023-10-10T20:34:18.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.548+0000 7f3d7598c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3d700c93a0 0x7f3d7006c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.548+0000 7f3d7598c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d700cb5d0 con 0x7f3d700c7fd0 2023-10-10T20:34:18.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.548+0000 7f3d7598c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d700af490 con 0x7f3d700c9770 2023-10-10T20:34:18.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.548+0000 7f3d7598c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d700af770 con 0x7f3d700c93a0 2023-10-10T20:34:18.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.548+0000 7f3d6ffff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f3d700c9770 0x7f3d700caec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48494/0 (socket says 172.21.15.111:48494) 2023-10-10T20:34:18.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.548+0000 7f3d6ffff700 1 -- 172.21.15.111:0/497652097 learned_addr learned my addr 172.21.15.111:0/497652097 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-10-10T20:34:18.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.548+0000 7f3d7498a700 1 --2- 172.21.15.111:0/497652097 >> v2:172.21.15.89:3300/0 conn(0x7f3d700c93a0 0x7f3d7006c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.548+0000 7f3d6f7fe700 1 -- 172.21.15.111:0/497652097 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054066560 0 0) 0x7f3d700af490 con 0x7f3d700c9770 2023-10-10T20:34:18.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.548+0000 7f3d6f7fe700 1 -- 172.21.15.111:0/497652097 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d60003570 con 0x7f3d700c9770 2023-10-10T20:34:18.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.548+0000 7f3d7498a700 1 -- 172.21.15.111:0/497652097 >> v1:172.21.15.111:6789/0 conn(0x7f3d700c9770 legacy=0x7f3d700caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.548+0000 7f3d7498a700 1 -- 172.21.15.111:0/497652097 >> v1:172.21.15.89:6789/0 conn(0x7f3d700c7fd0 legacy=0x7f3d700c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.548+0000 7f3d7498a700 1 -- 172.21.15.111:0/497652097 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d700afa50 con 0x7f3d700c93a0 2023-10-10T20:34:18.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.549+0000 7f3d7498a700 1 --2- 172.21.15.111:0/497652097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d700c93a0 0x7f3d7006c980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3d640098e0 tx=0x7f3d640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6fe147209eb9b8f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:18.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.549+0000 7f3d6f7fe700 1 -- 172.21.15.111:0/497652097 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400b640 con 0x7f3d700c93a0 2023-10-10T20:34:18.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.549+0000 7f3d6f7fe700 1 -- 172.21.15.111:0/497652097 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6400b7a0 con 0x7f3d700c93a0 2023-10-10T20:34:18.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.549+0000 7f3d6f7fe700 1 -- 172.21.15.111:0/497652097 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400bb40 con 0x7f3d700c93a0 2023-10-10T20:34:18.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.549+0000 7f3d7598c700 1 -- 172.21.15.111:0/497652097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d700c93a0 msgr2=0x7f3d7006c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.549+0000 7f3d7598c700 1 --2- 172.21.15.111:0/497652097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d700c93a0 0x7f3d7006c980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3d640098e0 tx=0x7f3d640091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.549+0000 7f3d7598c700 1 -- 172.21.15.111:0/497652097 shutdown_connections 2023-10-10T20:34:18.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.549+0000 7f3d7598c700 1 --2- 172.21.15.111:0/497652097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d700c93a0 0x7f3d7006c980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.549+0000 7f3d7598c700 1 -- 172.21.15.111:0/497652097 >> 172.21.15.111:0/497652097 conn(0x7f3d700cddc0 msgr2=0x7f3d700c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:18.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.549+0000 7f3d7598c700 1 -- 172.21.15.111:0/497652097 shutdown_connections 2023-10-10T20:34:18.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.549+0000 7f3d7598c700 1 -- 172.21.15.111:0/497652097 wait complete. 2023-10-10T20:34:18.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.550+0000 7f3d7598c700 1 Processor -- start 2023-10-10T20:34:18.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.551+0000 7f3d7598c700 1 -- start start 2023-10-10T20:34:18.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.551+0000 7f3d7598c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d700c7fd0 0x7f3d700c6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.551+0000 7f3d7598c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d700c93a0 0x7f3d700c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.551+0000 7f3d7598c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d700c9770 0x7f3d700c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.551+0000 7f3d7598c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d700c4ac0 con 0x7f3d700c93a0 2023-10-10T20:34:18.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.551+0000 7f3d7598c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d700a5a30 con 0x7f3d700c9770 2023-10-10T20:34:18.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.551+0000 7f3d7598c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d700c4d10 con 0x7f3d700c7fd0 2023-10-10T20:34:18.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d7498a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d700c7fd0 0x7f3d700c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d7518b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d700c9770 0x7f3d700c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d7498a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d700c7fd0 0x7f3d700c6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:32856/0 (socket says 172.21.15.111:32856) 2023-10-10T20:34:18.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d7498a700 1 -- 172.21.15.111:0/1156775802 learned_addr learned my addr 172.21.15.111:0/1156775802 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-10-10T20:34:18.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d6ffff700 1 --2- 172.21.15.111:0/1156775802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d700c93a0 0x7f3d700c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d7518b700 1 -- 172.21.15.111:0/1156775802 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d700c7fd0 msgr2=0x7f3d700c6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d7518b700 1 --2- 172.21.15.111:0/1156775802 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d700c7fd0 0x7f3d700c6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d7518b700 1 -- 172.21.15.111:0/1156775802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d700c93a0 msgr2=0x7f3d700c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d7518b700 1 --2- 172.21.15.111:0/1156775802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d700c93a0 0x7f3d700c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d7518b700 1 -- 172.21.15.111:0/1156775802 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d64009420 con 0x7f3d700c9770 2023-10-10T20:34:18.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d7498a700 1 --2- 172.21.15.111:0/1156775802 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d700c7fd0 0x7f3d700c6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:18.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d6ffff700 1 --2- 172.21.15.111:0/1156775802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d700c93a0 0x7f3d700c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:18.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d7518b700 1 --2- 172.21.15.111:0/1156775802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d700c9770 0x7f3d700c5b50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800c790 tx=0x7f3d680015f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:18.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d6cff9700 1 -- 172.21.15.111:0/1156775802 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d68012630 con 0x7f3d700c9770 2023-10-10T20:34:18.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d6cff9700 1 -- 172.21.15.111:0/1156775802 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d68012790 con 0x7f3d700c9770 2023-10-10T20:34:18.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d6cff9700 1 -- 172.21.15.111:0/1156775802 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d68012b50 con 0x7f3d700c9770 2023-10-10T20:34:18.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d7598c700 1 -- 172.21.15.111:0/1156775802 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d700c4ff0 con 0x7f3d700c9770 2023-10-10T20:34:18.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d7598c700 1 -- 172.21.15.111:0/1156775802 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d700c6960 con 0x7f3d700c9770 2023-10-10T20:34:18.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d6cff9700 1 -- 172.21.15.111:0/1156775802 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f3d68003030 con 0x7f3d700c9770 2023-10-10T20:34:18.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.552+0000 7f3d6cff9700 1 -- 172.21.15.111:0/1156775802 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f3d68011380 con 0x7f3d700c9770 2023-10-10T20:34:18.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.553+0000 7f3d7598c700 1 -- 172.21.15.111:0/1156775802 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7006d560 con 0x7f3d700c9770 2023-10-10T20:34:18.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.555+0000 7f3d6cff9700 1 -- 172.21.15.111:0/1156775802 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f3d6801b400 con 0x7f3d700c9770 2023-10-10T20:34:18.587 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:18.585+0000 7fb98747c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-10T20:34:18.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.638+0000 7f3d7598c700 1 -- 172.21.15.111:0/1156775802 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f065bed3-b82e-489b-a87b-57a5d762dbee", "id": 3} v 0) v1 -- 0x7f3d7006b500 con 0x7f3d700c9770 2023-10-10T20:34:18.647 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.642+0000 7f3d6cff9700 1 -- 172.21.15.111:0/1156775802 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f065bed3-b82e-489b-a87b-57a5d762dbee", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3d68006ba0 con 0x7f3d700c9770 2023-10-10T20:34:18.648 INFO:teuthology.orchestra.run.smithi111.stdout:3 2023-10-10T20:34:18.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.643+0000 7f3d7598c700 1 -- 172.21.15.111:0/1156775802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d700c9770 msgr2=0x7f3d700c5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.643+0000 7f3d7598c700 1 --2- 172.21.15.111:0/1156775802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d700c9770 0x7f3d700c5b50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800c790 tx=0x7f3d680015f0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.643+0000 7f3d7598c700 1 -- 172.21.15.111:0/1156775802 shutdown_connections 2023-10-10T20:34:18.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.643+0000 7f3d7598c700 1 --2- 172.21.15.111:0/1156775802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d700c9770 0x7f3d700c5b50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.643+0000 7f3d7598c700 1 --2- 172.21.15.111:0/1156775802 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d700c7fd0 0x7f3d700c6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.643+0000 7f3d7598c700 1 --2- 172.21.15.111:0/1156775802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d700c93a0 0x7f3d700c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.643+0000 7f3d7598c700 1 -- 172.21.15.111:0/1156775802 >> 172.21.15.111:0/1156775802 conn(0x7f3d700cddc0 msgr2=0x7f3d7006c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:18.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.643+0000 7f3d7598c700 1 -- 172.21.15.111:0/1156775802 shutdown_connections 2023-10-10T20:34:18.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.643+0000 7f3d7598c700 1 -- 172.21.15.111:0/1156775802 wait complete. 2023-10-10T20:34:18.657 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd new a922fa5c-8390-4160-87ff-e11d324d6cc6 4 2023-10-10T20:34:18.709 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:18.706+0000 7f02c1de9180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-10T20:34:18.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.768+0000 7f4270bf0700 1 Processor -- start 2023-10-10T20:34:18.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.768+0000 7f4270bf0700 1 -- start start 2023-10-10T20:34:18.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.769+0000 7f4270bf0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f426c141500 0x7f426c0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.769+0000 7f4270bf0700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f426c06c0a0 0x7f426c0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.769+0000 7f4270bf0700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f426c15c2a0 con 0x7f426c06bcd0 2023-10-10T20:34:18.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.769+0000 7f4270bf0700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f426c0c9880 con 0x7f426c06c0a0 2023-10-10T20:34:18.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.769+0000 7f4270bf0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426c15b690 con 0x7f426c141500 2023-10-10T20:34:18.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.769+0000 7f4269d9b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f426c141500 0x7f426c0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.769+0000 7f426ad9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f426c06c0a0 0x7f426c0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.769+0000 7f426ad9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f426c06c0a0 0x7f426c0c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39192/0 (socket says 172.21.15.111:39192) 2023-10-10T20:34:18.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.769+0000 7f426a59c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f426c06bcd0 0x7f426c140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48510/0 (socket says 172.21.15.111:48510) 2023-10-10T20:34:18.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.769+0000 7f4269d9b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f426c141500 0x7f426c0c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:32866/0 (socket says 172.21.15.111:32866) 2023-10-10T20:34:18.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.769+0000 7f426ad9d700 1 -- 172.21.15.111:0/2305444422 learned_addr learned my addr 172.21.15.111:0/2305444422 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-10-10T20:34:18.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.770+0000 7f4269d9b700 1 -- 172.21.15.111:0/2305444422 >> v2:172.21.15.111:3300/0 conn(0x7f426c06c0a0 msgr2=0x7f426c0c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:18.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.770+0000 7f4269d9b700 1 --2- 172.21.15.111:0/2305444422 >> v2:172.21.15.111:3300/0 conn(0x7f426c06c0a0 0x7f426c0c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.770+0000 7f4269d9b700 1 -- 172.21.15.111:0/2305444422 >> v1:172.21.15.111:6789/0 conn(0x7f426c06bcd0 legacy=0x7f426c140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.770+0000 7f4269d9b700 1 -- 172.21.15.111:0/2305444422 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c15b970 con 0x7f426c141500 2023-10-10T20:34:18.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.770+0000 7f4269d9b700 1 --2- 172.21.15.111:0/2305444422 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f426c141500 0x7f426c0c8e00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f425c0098e0 tx=0x7f425c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f040d0c31d93e9f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:18.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.770+0000 7f426959a700 1 -- 172.21.15.111:0/2305444422 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c00b640 con 0x7f426c141500 2023-10-10T20:34:18.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.770+0000 7f426959a700 1 -- 172.21.15.111:0/2305444422 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c00b7a0 con 0x7f426c141500 2023-10-10T20:34:18.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.770+0000 7f426959a700 1 -- 172.21.15.111:0/2305444422 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c00bb40 con 0x7f426c141500 2023-10-10T20:34:18.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.771+0000 7f4270bf0700 1 -- 172.21.15.111:0/2305444422 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f426c141500 msgr2=0x7f426c0c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.771+0000 7f4270bf0700 1 --2- 172.21.15.111:0/2305444422 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f426c141500 0x7f426c0c8e00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f425c0098e0 tx=0x7f425c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.771+0000 7f4270bf0700 1 -- 172.21.15.111:0/2305444422 shutdown_connections 2023-10-10T20:34:18.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.771+0000 7f4270bf0700 1 --2- 172.21.15.111:0/2305444422 >> v2:172.21.15.111:3300/0 conn(0x7f426c06c0a0 0x7f426c0c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.771+0000 7f4270bf0700 1 --2- 172.21.15.111:0/2305444422 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f426c141500 0x7f426c0c8e00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.771+0000 7f4270bf0700 1 -- 172.21.15.111:0/2305444422 >> 172.21.15.111:0/2305444422 conn(0x7f426c0ce600 msgr2=0x7f426c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:18.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.771+0000 7f4270bf0700 1 -- 172.21.15.111:0/2305444422 shutdown_connections 2023-10-10T20:34:18.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.771+0000 7f4270bf0700 1 -- 172.21.15.111:0/2305444422 wait complete. 2023-10-10T20:34:18.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.772+0000 7f4270bf0700 1 Processor -- start 2023-10-10T20:34:18.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.772+0000 7f4270bf0700 1 -- start start 2023-10-10T20:34:18.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.772+0000 7f4270bf0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f426c06bcd0 0x7f426c0c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.772+0000 7f4270bf0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f426c06c0a0 0x7f426c16c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f4270bf0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f426c141500 0x7f426c1401c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f4270bf0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c16b7b0 con 0x7f426c06c0a0 2023-10-10T20:34:18.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f4270bf0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c16bd70 con 0x7f426c06bcd0 2023-10-10T20:34:18.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f4270bf0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426c0c8290 con 0x7f426c141500 2023-10-10T20:34:18.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f426a59c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f426c06bcd0 0x7f426c0c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f426ad9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f426c141500 0x7f426c1401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f426ad9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f426c141500 0x7f426c1401c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:32868/0 (socket says 172.21.15.111:32868) 2023-10-10T20:34:18.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f426ad9d700 1 -- 172.21.15.111:0/4244272964 learned_addr learned my addr 172.21.15.111:0/4244272964 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-10-10T20:34:18.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f4269d9b700 1 --2- 172.21.15.111:0/4244272964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f426c06c0a0 0x7f426c16c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f426ad9d700 1 -- 172.21.15.111:0/4244272964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f426c06bcd0 msgr2=0x7f426c0c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f426ad9d700 1 --2- 172.21.15.111:0/4244272964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f426c06bcd0 0x7f426c0c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f426ad9d700 1 -- 172.21.15.111:0/4244272964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f426c06c0a0 msgr2=0x7f426c16c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f426ad9d700 1 --2- 172.21.15.111:0/4244272964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f426c06c0a0 0x7f426c16c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f426ad9d700 1 -- 172.21.15.111:0/4244272964 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c009420 con 0x7f426c141500 2023-10-10T20:34:18.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f426ad9d700 1 --2- 172.21.15.111:0/4244272964 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f426c141500 0x7f426c1401c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4264009e50 tx=0x7f426400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:18.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f425a7fc700 1 -- 172.21.15.111:0/4244272964 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426400c940 con 0x7f426c141500 2023-10-10T20:34:18.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f425a7fc700 1 -- 172.21.15.111:0/4244272964 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426400f040 con 0x7f426c141500 2023-10-10T20:34:18.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f4270bf0700 1 -- 172.21.15.111:0/4244272964 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426c0c8570 con 0x7f426c141500 2023-10-10T20:34:18.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f425a7fc700 1 -- 172.21.15.111:0/4244272964 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426400ccf0 con 0x7f426c141500 2023-10-10T20:34:18.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f4270bf0700 1 -- 172.21.15.111:0/4244272964 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426c140700 con 0x7f426c141500 2023-10-10T20:34:18.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f425a7fc700 1 -- 172.21.15.111:0/4244272964 <== mon.2 v2:172.21.15.111:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4264014650 con 0x7f426c141500 2023-10-10T20:34:18.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.773+0000 7f425a7fc700 1 -- 172.21.15.111:0/4244272964 <== mon.2 v2:172.21.15.111:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f426401d910 con 0x7f426c141500 2023-10-10T20:34:18.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.774+0000 7f4270bf0700 1 -- 172.21.15.111:0/4244272964 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f426c16c740 con 0x7f426c141500 2023-10-10T20:34:18.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.778+0000 7f425a7fc700 1 -- 172.21.15.111:0/4244272964 <== mon.2 v2:172.21.15.111:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f426400ce50 con 0x7f426c141500 2023-10-10T20:34:18.793 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:18.778+0000 7fb98747c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-10T20:34:18.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.859+0000 7f4270bf0700 1 -- 172.21.15.111:0/4244272964 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a922fa5c-8390-4160-87ff-e11d324d6cc6", "id": 4} v 0) v1 -- 0x7f426c16bee0 con 0x7f426c141500 2023-10-10T20:34:18.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.862+0000 7f425a7fc700 1 -- 172.21.15.111:0/4244272964 <== mon.2 v2:172.21.15.111:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a922fa5c-8390-4160-87ff-e11d324d6cc6", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4264014ae0 con 0x7f426c141500 2023-10-10T20:34:18.865 INFO:teuthology.orchestra.run.smithi111.stdout:4 2023-10-10T20:34:18.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.863+0000 7f4270bf0700 1 -- 172.21.15.111:0/4244272964 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f426c141500 msgr2=0x7f426c1401c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.863+0000 7f4270bf0700 1 --2- 172.21.15.111:0/4244272964 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f426c141500 0x7f426c1401c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4264009e50 tx=0x7f426400b040 comp rx=0 tx=0).stop 2023-10-10T20:34:18.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.863+0000 7f4270bf0700 1 -- 172.21.15.111:0/4244272964 shutdown_connections 2023-10-10T20:34:18.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.863+0000 7f4270bf0700 1 --2- 172.21.15.111:0/4244272964 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f426c141500 0x7f426c1401c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.863+0000 7f4270bf0700 1 --2- 172.21.15.111:0/4244272964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f426c06bcd0 0x7f426c0c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.863+0000 7f4270bf0700 1 --2- 172.21.15.111:0/4244272964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f426c06c0a0 0x7f426c16c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.863+0000 7f4270bf0700 1 -- 172.21.15.111:0/4244272964 >> 172.21.15.111:0/4244272964 conn(0x7f426c0ce600 msgr2=0x7f426c13c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:18.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.863+0000 7f4270bf0700 1 -- 172.21.15.111:0/4244272964 shutdown_connections 2023-10-10T20:34:18.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.863+0000 7f4270bf0700 1 -- 172.21.15.111:0/4244272964 wait complete. 2023-10-10T20:34:18.877 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd new 94d1f69c-355e-430c-afae-15ba87b76ad7 5 2023-10-10T20:34:18.880 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:18.878+0000 7f02c1de9180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-10T20:34:18.957 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:18.955+0000 7f02c1de9180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-10T20:34:18.959 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:18.957+0000 7fb98747c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-10T20:34:18.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.989+0000 7f92b8a44700 1 Processor -- start 2023-10-10T20:34:18.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.989+0000 7f92b8a44700 1 -- start start 2023-10-10T20:34:18.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.990+0000 7f92b8a44700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92b4159a70 0x7f92b40c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.990+0000 7f92b8a44700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f92b406bfa0 0x7f92b40c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:18.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.990+0000 7f92b8a44700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b40ca920 con 0x7f92b4068390 2023-10-10T20:34:18.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.990+0000 7f92b8a44700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b406af00 con 0x7f92b406bfa0 2023-10-10T20:34:18.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.990+0000 7f92b8a44700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b406b1e0 con 0x7f92b4159a70 2023-10-10T20:34:18.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.990+0000 7f92b259c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f92b406bfa0 0x7f92b40c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.990+0000 7f92b2d9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92b4159a70 0x7f92b40c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:18.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.990+0000 7f92b259c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f92b406bfa0 0x7f92b40c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39218/0 (socket says 172.21.15.111:39218) 2023-10-10T20:34:18.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.990+0000 7f92b2d9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92b4159a70 0x7f92b40c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:32878/0 (socket says 172.21.15.111:32878) 2023-10-10T20:34:18.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.990+0000 7f92b259c700 1 -- 172.21.15.111:0/2246713611 learned_addr learned my addr 172.21.15.111:0/2246713611 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-10-10T20:34:18.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.990+0000 7f92b259c700 1 -- 172.21.15.111:0/2246713611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92b4159a70 msgr2=0x7f92b40c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:18.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.990+0000 7f92b259c700 1 --2- 172.21.15.111:0/2246713611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92b4159a70 0x7f92b40c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:18.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.990+0000 7f92b259c700 1 -- 172.21.15.111:0/2246713611 >> v1:172.21.15.89:6789/0 conn(0x7f92b4068390 legacy=0x7f92b40caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.990+0000 7f92b259c700 1 -- 172.21.15.111:0/2246713611 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b406b4c0 con 0x7f92b406bfa0 2023-10-10T20:34:18.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.990+0000 7f92b259c700 1 --2- 172.21.15.111:0/2246713611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92b406bfa0 0x7f92b40c8830 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f92a4009990 tx=0x7f92a4009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8d3a973fce8c66e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:18.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.990+0000 7f92b1d9b700 1 -- 172.21.15.111:0/2246713611 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4009e80 con 0x7f92b406bfa0 2023-10-10T20:34:18.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.991+0000 7f92b1d9b700 1 -- 172.21.15.111:0/2246713611 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a4005c10 con 0x7f92b406bfa0 2023-10-10T20:34:18.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.991+0000 7f92b1d9b700 1 -- 172.21.15.111:0/2246713611 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a400b650 con 0x7f92b406bfa0 2023-10-10T20:34:18.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.991+0000 7f92b8a44700 1 -- 172.21.15.111:0/2246713611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92b406bfa0 msgr2=0x7f92b40c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:18.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.991+0000 7f92b8a44700 1 --2- 172.21.15.111:0/2246713611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92b406bfa0 0x7f92b40c8830 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f92a4009990 tx=0x7f92a4009220 comp rx=0 tx=0).stop 2023-10-10T20:34:18.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.991+0000 7f92b8a44700 1 -- 172.21.15.111:0/2246713611 shutdown_connections 2023-10-10T20:34:19.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.991+0000 7f92b8a44700 1 --2- 172.21.15.111:0/2246713611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92b4159a70 0x7f92b40c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.991+0000 7f92b8a44700 1 --2- 172.21.15.111:0/2246713611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92b406bfa0 0x7f92b40c8830 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.991+0000 7f92b8a44700 1 -- 172.21.15.111:0/2246713611 >> 172.21.15.111:0/2246713611 conn(0x7f92b40cddc0 msgr2=0x7f92b40c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:19.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.991+0000 7f92b8a44700 1 -- 172.21.15.111:0/2246713611 shutdown_connections 2023-10-10T20:34:19.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.991+0000 7f92b8a44700 1 -- 172.21.15.111:0/2246713611 wait complete. 2023-10-10T20:34:19.004 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.992+0000 7f92b8a44700 1 Processor -- start 2023-10-10T20:34:19.004 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.992+0000 7f92b8a44700 1 -- start start 2023-10-10T20:34:19.004 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.993+0000 7f92b8a44700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92b4068390 0x7f92b40b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:19.004 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.993+0000 7f92b8a44700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92b406bfa0 0x7f92b40c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:19.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.993+0000 7f92b8a44700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92b4159a70 0x7f92b415d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:19.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.993+0000 7f92b8a44700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b40b09c0 con 0x7f92b4159a70 2023-10-10T20:34:19.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.993+0000 7f92b8a44700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b40b2fe0 con 0x7f92b406bfa0 2023-10-10T20:34:19.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.993+0000 7f92b8a44700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b40b0090 con 0x7f92b4068390 2023-10-10T20:34:19.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.993+0000 7f92b2d9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92b4068390 0x7f92b40b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:19.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.993+0000 7f92b259c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92b406bfa0 0x7f92b40c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:19.006 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.993+0000 7f92b2d9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92b4068390 0x7f92b40b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:32892/0 (socket says 172.21.15.111:32892) 2023-10-10T20:34:19.006 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.993+0000 7f92b259c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92b406bfa0 0x7f92b40c6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39232/0 (socket says 172.21.15.111:39232) 2023-10-10T20:34:19.006 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.993+0000 7f92b2d9d700 1 -- 172.21.15.111:0/3501223622 learned_addr learned my addr 172.21.15.111:0/3501223622 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-10-10T20:34:19.006 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.993+0000 7f92b359e700 1 --2- 172.21.15.111:0/3501223622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92b4159a70 0x7f92b415d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:19.006 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.993+0000 7f92b259c700 1 -- 172.21.15.111:0/3501223622 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92b4068390 msgr2=0x7f92b40b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:19.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.993+0000 7f92b259c700 1 --2- 172.21.15.111:0/3501223622 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92b4068390 0x7f92b40b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.993+0000 7f92b259c700 1 -- 172.21.15.111:0/3501223622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92b4159a70 msgr2=0x7f92b415d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:19.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.993+0000 7f92b259c700 1 --2- 172.21.15.111:0/3501223622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92b4159a70 0x7f92b415d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.993+0000 7f92b259c700 1 -- 172.21.15.111:0/3501223622 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a4009480 con 0x7f92b406bfa0 2023-10-10T20:34:19.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.994+0000 7f92b259c700 1 --2- 172.21.15.111:0/3501223622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92b406bfa0 0x7f92b40c6600 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f92a4015000 tx=0x7f92a4016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:19.008 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.994+0000 7f929affd700 1 -- 172.21.15.111:0/3501223622 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4011950 con 0x7f92b406bfa0 2023-10-10T20:34:19.008 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.994+0000 7f929affd700 1 -- 172.21.15.111:0/3501223622 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a4020070 con 0x7f92b406bfa0 2023-10-10T20:34:19.008 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.994+0000 7f92b8a44700 1 -- 172.21.15.111:0/3501223622 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b40b0370 con 0x7f92b406bfa0 2023-10-10T20:34:19.008 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.994+0000 7f929affd700 1 -- 172.21.15.111:0/3501223622 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4011d20 con 0x7f92b406bfa0 2023-10-10T20:34:19.008 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.994+0000 7f92b8a44700 1 -- 172.21.15.111:0/3501223622 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b40b2bb0 con 0x7f92b406bfa0 2023-10-10T20:34:19.009 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.994+0000 7f929affd700 1 -- 172.21.15.111:0/3501223622 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f92a4022030 con 0x7f92b406bfa0 2023-10-10T20:34:19.009 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.994+0000 7f929affd700 1 -- 172.21.15.111:0/3501223622 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f92a402e440 con 0x7f92b406bfa0 2023-10-10T20:34:19.009 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.995+0000 7f92b8a44700 1 -- 172.21.15.111:0/3501223622 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b40b2da0 con 0x7f92b406bfa0 2023-10-10T20:34:19.009 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:18.997+0000 7f929affd700 1 -- 172.21.15.111:0/3501223622 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f92b40b2da0 con 0x7f92b406bfa0 2023-10-10T20:34:19.040 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:19.037+0000 7fb98747c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-10T20:34:19.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.081+0000 7f92b8a44700 1 -- 172.21.15.111:0/3501223622 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd new", "uuid": "94d1f69c-355e-430c-afae-15ba87b76ad7", "id": 5} v 0) v1 -- 0x7f92b40b0b30 con 0x7f92b406bfa0 2023-10-10T20:34:19.088 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.085+0000 7f929affd700 1 -- 172.21.15.111:0/3501223622 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "94d1f69c-355e-430c-afae-15ba87b76ad7", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f92a40258c0 con 0x7f92b406bfa0 2023-10-10T20:34:19.088 INFO:teuthology.orchestra.run.smithi111.stdout:5 2023-10-10T20:34:19.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.086+0000 7f92b8a44700 1 -- 172.21.15.111:0/3501223622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92b406bfa0 msgr2=0x7f92b40c6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:19.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.086+0000 7f92b8a44700 1 --2- 172.21.15.111:0/3501223622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92b406bfa0 0x7f92b40c6600 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f92a4015000 tx=0x7f92a4016000 comp rx=0 tx=0).stop 2023-10-10T20:34:19.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.086+0000 7f92b8a44700 1 -- 172.21.15.111:0/3501223622 shutdown_connections 2023-10-10T20:34:19.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.086+0000 7f92b8a44700 1 --2- 172.21.15.111:0/3501223622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92b4159a70 0x7f92b415d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.086+0000 7f92b8a44700 1 --2- 172.21.15.111:0/3501223622 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92b4068390 0x7f92b40b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.086+0000 7f92b8a44700 1 --2- 172.21.15.111:0/3501223622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92b406bfa0 0x7f92b40c6600 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.086+0000 7f92b8a44700 1 -- 172.21.15.111:0/3501223622 >> 172.21.15.111:0/3501223622 conn(0x7f92b40cddc0 msgr2=0x7f92b4158a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:19.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.087+0000 7f92b8a44700 1 -- 172.21.15.111:0/3501223622 shutdown_connections 2023-10-10T20:34:19.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.087+0000 7f92b8a44700 1 -- 172.21.15.111:0/3501223622 wait complete. 2023-10-10T20:34:19.092 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:19.089+0000 7f02c1de9180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-10T20:34:19.101 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd new d69d657c-9b34-4e9c-a47a-9449d226bfc7 6 2023-10-10T20:34:19.182 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:19.179+0000 7fb98747c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-10T20:34:19.186 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:19.184+0000 7f02c1de9180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-10T20:34:19.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.215+0000 7f38d626c700 1 Processor -- start 2023-10-10T20:34:19.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.215+0000 7f38d626c700 1 -- start start 2023-10-10T20:34:19.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.216+0000 7f38d626c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38d00b4e70 0x7f38d00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:19.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.216+0000 7f38d626c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f38d00b1b40 0x7f38d015d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:19.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.216+0000 7f38d626c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d00cfb00 con 0x7f38d00b16d0 2023-10-10T20:34:19.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.216+0000 7f38d626c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d00b5c40 con 0x7f38d00b1b40 2023-10-10T20:34:19.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.216+0000 7f38d626c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00b5f20 con 0x7f38d00b4e70 2023-10-10T20:34:19.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.216+0000 7f38d4a69700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f38d00b16d0 0x7f38d015d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48518/0 (socket says 172.21.15.111:48518) 2023-10-10T20:34:19.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.216+0000 7f38d4a69700 1 -- 172.21.15.111:0/2857208550 learned_addr learned my addr 172.21.15.111:0/2857208550 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-10-10T20:34:19.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.216+0000 7f38d526a700 1 --2- 172.21.15.111:0/2857208550 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38d00b4e70 0x7f38d00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:19.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.216+0000 7f38d5a6b700 1 --2- 172.21.15.111:0/2857208550 >> v2:172.21.15.89:3300/0 conn(0x7f38d00b1b40 0x7f38d015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:19.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.216+0000 7f38c7fff700 1 -- 172.21.15.111:0/2857208550 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2976629344 0 0) 0x7f38d00cfb00 con 0x7f38d00b16d0 2023-10-10T20:34:19.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.216+0000 7f38c7fff700 1 -- 172.21.15.111:0/2857208550 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38bc003570 con 0x7f38d00b16d0 2023-10-10T20:34:19.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.216+0000 7f38d526a700 1 -- 172.21.15.111:0/2857208550 >> v2:172.21.15.89:3300/0 conn(0x7f38d00b1b40 msgr2=0x7f38d015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:19.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.216+0000 7f38d526a700 1 --2- 172.21.15.111:0/2857208550 >> v2:172.21.15.89:3300/0 conn(0x7f38d00b1b40 0x7f38d015d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.216+0000 7f38d526a700 1 -- 172.21.15.111:0/2857208550 >> v1:172.21.15.111:6789/0 conn(0x7f38d00b16d0 legacy=0x7f38d015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:19.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.216+0000 7f38d526a700 1 -- 172.21.15.111:0/2857208550 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d00b6200 con 0x7f38d00b4e70 2023-10-10T20:34:19.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.217+0000 7f38d526a700 1 --2- 172.21.15.111:0/2857208550 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38d00b4e70 0x7f38d00ccff0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f38c00098e0 tx=0x7f38c00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32cae3d2013b081a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:19.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.217+0000 7f38c7fff700 1 -- 172.21.15.111:0/2857208550 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0011630 con 0x7f38d00b4e70 2023-10-10T20:34:19.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.217+0000 7f38c7fff700 1 -- 172.21.15.111:0/2857208550 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c0011790 con 0x7f38d00b4e70 2023-10-10T20:34:19.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.217+0000 7f38c7fff700 1 -- 172.21.15.111:0/2857208550 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0011b30 con 0x7f38d00b4e70 2023-10-10T20:34:19.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.217+0000 7f38d626c700 1 -- 172.21.15.111:0/2857208550 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38d00b4e70 msgr2=0x7f38d00ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:19.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.217+0000 7f38d626c700 1 --2- 172.21.15.111:0/2857208550 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38d00b4e70 0x7f38d00ccff0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f38c00098e0 tx=0x7f38c00091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.217+0000 7f38d626c700 1 -- 172.21.15.111:0/2857208550 shutdown_connections 2023-10-10T20:34:19.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.217+0000 7f38d626c700 1 --2- 172.21.15.111:0/2857208550 >> v2:172.21.15.89:3300/0 conn(0x7f38d00b1b40 0x7f38d015d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.217+0000 7f38d626c700 1 --2- 172.21.15.111:0/2857208550 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38d00b4e70 0x7f38d00ccff0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.217+0000 7f38d626c700 1 -- 172.21.15.111:0/2857208550 >> 172.21.15.111:0/2857208550 conn(0x7f38d00d10c0 msgr2=0x7f38d00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:19.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.217+0000 7f38d626c700 1 -- 172.21.15.111:0/2857208550 shutdown_connections 2023-10-10T20:34:19.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.217+0000 7f38d626c700 1 -- 172.21.15.111:0/2857208550 wait complete. 2023-10-10T20:34:19.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.219+0000 7f38d626c700 1 Processor -- start 2023-10-10T20:34:19.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.219+0000 7f38d626c700 1 -- start start 2023-10-10T20:34:19.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.219+0000 7f38d626c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38d00b16d0 0x7f38d00c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:19.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.219+0000 7f38d626c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38d00b1b40 0x7f38d0145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:19.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.219+0000 7f38d626c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38d00b4e70 0x7f38d016a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:19.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38d626c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00c6560 con 0x7f38d00b16d0 2023-10-10T20:34:19.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38d626c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d016ad10 con 0x7f38d00b1b40 2023-10-10T20:34:19.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38d626c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00c5950 con 0x7f38d00b4e70 2023-10-10T20:34:19.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38d4a69700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38d00b1b40 0x7f38d0145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:19.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38d5a6b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38d00b4e70 0x7f38d016a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:19.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38d5a6b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38d00b4e70 0x7f38d016a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:32904/0 (socket says 172.21.15.111:32904) 2023-10-10T20:34:19.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38d5a6b700 1 -- 172.21.15.111:0/3628163232 learned_addr learned my addr 172.21.15.111:0/3628163232 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-10-10T20:34:19.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38d526a700 1 --2- 172.21.15.111:0/3628163232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38d00b16d0 0x7f38d00c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:19.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38d4a69700 1 -- 172.21.15.111:0/3628163232 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38d00b4e70 msgr2=0x7f38d016a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:19.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38d4a69700 1 --2- 172.21.15.111:0/3628163232 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38d00b4e70 0x7f38d016a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38d4a69700 1 -- 172.21.15.111:0/3628163232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38d00b16d0 msgr2=0x7f38d00c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:19.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38d4a69700 1 --2- 172.21.15.111:0/3628163232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38d00b16d0 0x7f38d00c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38d4a69700 1 -- 172.21.15.111:0/3628163232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c0009420 con 0x7f38d00b1b40 2023-10-10T20:34:19.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38d5a6b700 1 --2- 172.21.15.111:0/3628163232 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38d00b4e70 0x7f38d016a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:19.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38d4a69700 1 --2- 172.21.15.111:0/3628163232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38d00b1b40 0x7f38d0145c10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00c580 tx=0x7f38cc007d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:19.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38c57fa700 1 -- 172.21.15.111:0/3628163232 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc005b30 con 0x7f38d00b1b40 2023-10-10T20:34:19.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38c57fa700 1 -- 172.21.15.111:0/3628163232 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38cc005cb0 con 0x7f38d00b1b40 2023-10-10T20:34:19.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38d626c700 1 -- 172.21.15.111:0/3628163232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d00c5c30 con 0x7f38d00b1b40 2023-10-10T20:34:19.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38c57fa700 1 -- 172.21.15.111:0/3628163232 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc007320 con 0x7f38d00b1b40 2023-10-10T20:34:19.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38d626c700 1 -- 172.21.15.111:0/3628163232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d00c61f0 con 0x7f38d00b1b40 2023-10-10T20:34:19.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38c57fa700 1 -- 172.21.15.111:0/3628163232 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f38cc01c6c0 con 0x7f38d00b1b40 2023-10-10T20:34:19.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.220+0000 7f38c57fa700 1 -- 172.21.15.111:0/3628163232 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f38cc005e20 con 0x7f38d00b1b40 2023-10-10T20:34:19.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.221+0000 7f38d626c700 1 -- 172.21.15.111:0/3628163232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d00ce180 con 0x7f38d00b1b40 2023-10-10T20:34:19.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.223+0000 7f38c57fa700 1 -- 172.21.15.111:0/3628163232 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f38cc017070 con 0x7f38d00b1b40 2023-10-10T20:34:19.282 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:19.280+0000 7fb98747c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-10T20:34:19.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.307+0000 7f38d626c700 1 -- 172.21.15.111:0/3628163232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d69d657c-9b34-4e9c-a47a-9449d226bfc7", "id": 6} v 0) v1 -- 0x7f38d00c66d0 con 0x7f38d00b1b40 2023-10-10T20:34:19.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.312+0000 7f38c57fa700 1 -- 172.21.15.111:0/3628163232 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d69d657c-9b34-4e9c-a47a-9449d226bfc7", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f38cc007780 con 0x7f38d00b1b40 2023-10-10T20:34:19.316 INFO:teuthology.orchestra.run.smithi111.stdout:6 2023-10-10T20:34:19.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.313+0000 7f38d626c700 1 -- 172.21.15.111:0/3628163232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38d00b1b40 msgr2=0x7f38d0145c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:19.318 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.313+0000 7f38d626c700 1 --2- 172.21.15.111:0/3628163232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38d00b1b40 0x7f38d0145c10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00c580 tx=0x7f38cc007d50 comp rx=0 tx=0).stop 2023-10-10T20:34:19.318 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.314+0000 7f38d626c700 1 -- 172.21.15.111:0/3628163232 shutdown_connections 2023-10-10T20:34:19.318 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.314+0000 7f38d626c700 1 --2- 172.21.15.111:0/3628163232 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38d00b4e70 0x7f38d016a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.318 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.314+0000 7f38d626c700 1 --2- 172.21.15.111:0/3628163232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38d00b16d0 0x7f38d00c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.314+0000 7f38d626c700 1 --2- 172.21.15.111:0/3628163232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38d00b1b40 0x7f38d0145c10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.314+0000 7f38d626c700 1 -- 172.21.15.111:0/3628163232 >> 172.21.15.111:0/3628163232 conn(0x7f38d00d10c0 msgr2=0x7f38d00b53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:19.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.314+0000 7f38d626c700 1 -- 172.21.15.111:0/3628163232 shutdown_connections 2023-10-10T20:34:19.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.314+0000 7f38d626c700 1 -- 172.21.15.111:0/3628163232 wait complete. 2023-10-10T20:34:19.327 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd new 956c4fea-bdfd-455a-a107-7b58155631ec 7 2023-10-10T20:34:19.442 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.440+0000 7f49343f6700 1 Processor -- start 2023-10-10T20:34:19.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.440+0000 7f49343f6700 1 -- start start 2023-10-10T20:34:19.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.441+0000 7f49343f6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f492c140350 0x7f492c069c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:19.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.441+0000 7f49343f6700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f492c068aa0 0x7f492c0c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:19.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.441+0000 7f49343f6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492c0c7de0 con 0x7f492c06a1c0 2023-10-10T20:34:19.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.441+0000 7f49343f6700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492c140c10 con 0x7f492c068aa0 2023-10-10T20:34:19.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.441+0000 7f49343f6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c141490 con 0x7f492c140350 2023-10-10T20:34:19.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.441+0000 7f4932993700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f492c068aa0 0x7f492c0c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:19.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.441+0000 7f4932192700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f492c140350 0x7f492c069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:19.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.441+0000 7f4932993700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f492c068aa0 0x7f492c0c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39256/0 (socket says 172.21.15.111:39256) 2023-10-10T20:34:19.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.441+0000 7f4932993700 1 -- 172.21.15.111:0/687748969 learned_addr learned my addr 172.21.15.111:0/687748969 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-10-10T20:34:19.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.441+0000 7f4932192700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f492c140350 0x7f492c069c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:32918/0 (socket says 172.21.15.111:32918) 2023-10-10T20:34:19.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.441+0000 7f4932993700 1 -- 172.21.15.111:0/687748969 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f492c140350 msgr2=0x7f492c069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:19.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.441+0000 7f4932993700 1 --2- 172.21.15.111:0/687748969 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f492c140350 0x7f492c069c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.441+0000 7f4932993700 1 -- 172.21.15.111:0/687748969 >> v1:172.21.15.89:6789/0 conn(0x7f492c06a1c0 legacy=0x7f492c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:19.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.441+0000 7f4932993700 1 -- 172.21.15.111:0/687748969 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c140ed0 con 0x7f492c068aa0 2023-10-10T20:34:19.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.441+0000 7f4932192700 1 --2- 172.21.15.111:0/687748969 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f492c140350 0x7f492c069c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:19.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.441+0000 7f4932993700 1 --2- 172.21.15.111:0/687748969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f492c068aa0 0x7f492c0c8270 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4924009f20 tx=0x7f4924009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae88822a4e142001 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:19.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.441+0000 7f4931190700 1 -- 172.21.15.111:0/687748969 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492400e070 con 0x7f492c068aa0 2023-10-10T20:34:19.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.441+0000 7f4931190700 1 -- 172.21.15.111:0/687748969 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492400b9d0 con 0x7f492c068aa0 2023-10-10T20:34:19.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.441+0000 7f4931190700 1 -- 172.21.15.111:0/687748969 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49240092e0 con 0x7f492c068aa0 2023-10-10T20:34:19.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.442+0000 7f49343f6700 1 -- 172.21.15.111:0/687748969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f492c068aa0 msgr2=0x7f492c0c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:19.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.442+0000 7f49343f6700 1 --2- 172.21.15.111:0/687748969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f492c068aa0 0x7f492c0c8270 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4924009f20 tx=0x7f4924009840 comp rx=0 tx=0).stop 2023-10-10T20:34:19.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.442+0000 7f49343f6700 1 -- 172.21.15.111:0/687748969 shutdown_connections 2023-10-10T20:34:19.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.442+0000 7f49343f6700 1 --2- 172.21.15.111:0/687748969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f492c068aa0 0x7f492c0c8270 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.442+0000 7f49343f6700 1 --2- 172.21.15.111:0/687748969 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f492c140350 0x7f492c069c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.442+0000 7f49343f6700 1 -- 172.21.15.111:0/687748969 >> 172.21.15.111:0/687748969 conn(0x7f492c0cd670 msgr2=0x7f492c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:19.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.442+0000 7f49343f6700 1 -- 172.21.15.111:0/687748969 shutdown_connections 2023-10-10T20:34:19.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.442+0000 7f49343f6700 1 -- 172.21.15.111:0/687748969 wait complete. 2023-10-10T20:34:19.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.443+0000 7f49343f6700 1 Processor -- start 2023-10-10T20:34:19.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.443+0000 7f49343f6700 1 -- start start 2023-10-10T20:34:19.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f49343f6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f492c068aa0 0x7f492c0973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:19.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f49343f6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f492c06a1c0 0x7f492c0c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:19.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f49343f6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f492c140350 0x7f492c098ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:19.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f49343f6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0c6700 con 0x7f492c06a1c0 2023-10-10T20:34:19.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f49343f6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0c50f0 con 0x7f492c140350 2023-10-10T20:34:19.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f49343f6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0c60b0 con 0x7f492c068aa0 2023-10-10T20:34:19.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f4932192700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f492c068aa0 0x7f492c0973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:19.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f4932993700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f492c140350 0x7f492c098ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:19.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f4932192700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f492c068aa0 0x7f492c0973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.111:32926/0 (socket says 172.21.15.111:32926) 2023-10-10T20:34:19.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f4932192700 1 -- 172.21.15.111:0/3291062306 learned_addr learned my addr 172.21.15.111:0/3291062306 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-10-10T20:34:19.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f4931991700 1 --2- 172.21.15.111:0/3291062306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f492c06a1c0 0x7f492c0c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:19.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f4932993700 1 -- 172.21.15.111:0/3291062306 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f492c068aa0 msgr2=0x7f492c0973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:19.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f4932993700 1 --2- 172.21.15.111:0/3291062306 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f492c068aa0 0x7f492c0973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f4932993700 1 -- 172.21.15.111:0/3291062306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f492c06a1c0 msgr2=0x7f492c0c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:19.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f4932993700 1 --2- 172.21.15.111:0/3291062306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f492c06a1c0 0x7f492c0c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f4932993700 1 -- 172.21.15.111:0/3291062306 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49280093c0 con 0x7f492c140350 2023-10-10T20:34:19.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f4932192700 1 --2- 172.21.15.111:0/3291062306 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f492c068aa0 0x7f492c0973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:19.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f4932993700 1 --2- 172.21.15.111:0/3291062306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f492c140350 0x7f492c098ef0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4924015000 tx=0x7f4924018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:19.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f49227fc700 1 -- 172.21.15.111:0/3291062306 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4924008be0 con 0x7f492c140350 2023-10-10T20:34:19.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f49227fc700 1 -- 172.21.15.111:0/3291062306 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4924008d60 con 0x7f492c140350 2023-10-10T20:34:19.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f49343f6700 1 -- 172.21.15.111:0/3291062306 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4924009ae0 con 0x7f492c140350 2023-10-10T20:34:19.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.444+0000 7f49227fc700 1 -- 172.21.15.111:0/3291062306 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49240266b0 con 0x7f492c140350 2023-10-10T20:34:19.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.445+0000 7f49343f6700 1 -- 172.21.15.111:0/3291062306 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492c0c5d90 con 0x7f492c140350 2023-10-10T20:34:19.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.445+0000 7f49227fc700 1 -- 172.21.15.111:0/3291062306 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4924003040 con 0x7f492c140350 2023-10-10T20:34:19.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.445+0000 7f49227fc700 1 -- 172.21.15.111:0/3291062306 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f4924026db0 con 0x7f492c140350 2023-10-10T20:34:19.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.445+0000 7f49343f6700 1 -- 172.21.15.111:0/3291062306 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492c04c3c0 con 0x7f492c140350 2023-10-10T20:34:19.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.447+0000 7f49227fc700 1 -- 172.21.15.111:0/3291062306 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f492402b020 con 0x7f492c140350 2023-10-10T20:34:19.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.530+0000 7f49343f6700 1 -- 172.21.15.111:0/3291062306 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd new", "uuid": "956c4fea-bdfd-455a-a107-7b58155631ec", "id": 7} v 0) v1 -- 0x7f492c0c6840 con 0x7f492c140350 2023-10-10T20:34:19.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.538+0000 7f49227fc700 1 -- 172.21.15.111:0/3291062306 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "956c4fea-bdfd-455a-a107-7b58155631ec", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4924021070 con 0x7f492c140350 2023-10-10T20:34:19.540 INFO:teuthology.orchestra.run.smithi111.stdout:7 2023-10-10T20:34:19.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.539+0000 7f49343f6700 1 -- 172.21.15.111:0/3291062306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f492c140350 msgr2=0x7f492c098ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:19.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.539+0000 7f49343f6700 1 --2- 172.21.15.111:0/3291062306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f492c140350 0x7f492c098ef0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4924015000 tx=0x7f4924018000 comp rx=0 tx=0).stop 2023-10-10T20:34:19.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.539+0000 7f49343f6700 1 -- 172.21.15.111:0/3291062306 shutdown_connections 2023-10-10T20:34:19.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.539+0000 7f49343f6700 1 --2- 172.21.15.111:0/3291062306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f492c140350 0x7f492c098ef0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.539+0000 7f49343f6700 1 --2- 172.21.15.111:0/3291062306 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f492c068aa0 0x7f492c0973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.539+0000 7f49343f6700 1 --2- 172.21.15.111:0/3291062306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f492c06a1c0 0x7f492c0c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.539+0000 7f49343f6700 1 -- 172.21.15.111:0/3291062306 >> 172.21.15.111:0/3291062306 conn(0x7f492c0cd670 msgr2=0x7f492c157070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:19.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.539+0000 7f49343f6700 1 -- 172.21.15.111:0/3291062306 shutdown_connections 2023-10-10T20:34:19.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-10-10T20:34:19.539+0000 7f49343f6700 1 -- 172.21.15.111:0/3291062306 wait complete. 2023-10-10T20:34:19.552 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-10T20:34:19.553 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-10T20:34:19.555 INFO:tasks.ceph.osd.0:Started 2023-10-10T20:34:19.555 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-10T20:34:19.556 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-10T20:34:19.558 INFO:tasks.ceph.osd.1:Started 2023-10-10T20:34:19.558 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-10T20:34:19.558 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-10T20:34:19.560 INFO:tasks.ceph.osd.2:Started 2023-10-10T20:34:19.561 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-10T20:34:19.561 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-10T20:34:19.563 INFO:tasks.ceph.osd.3:Started 2023-10-10T20:34:19.563 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-10T20:34:19.564 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-10T20:34:19.566 INFO:tasks.ceph.osd.4:Started 2023-10-10T20:34:19.566 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-10T20:34:19.567 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-10T20:34:19.568 INFO:tasks.ceph.osd.5:Started 2023-10-10T20:34:19.569 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-10T20:34:19.569 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-10T20:34:19.571 INFO:tasks.ceph.osd.6:Started 2023-10-10T20:34:19.572 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-10T20:34:19.572 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-10T20:34:19.574 INFO:tasks.ceph.osd.7:Started 2023-10-10T20:34:19.575 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:34:19.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.705+0000 7f38ee1e6700 1 Processor -- start 2023-10-10T20:34:19.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.705+0000 7f38ee1e6700 1 -- start start 2023-10-10T20:34:19.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.706+0000 7f38ee1e6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38e81597f0 0x7f38e80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:19.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.706+0000 7f38ee1e6700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f38e815d230 0x7f38e815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:19.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.706+0000 7f38ee1e6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e815dbd0 con 0x7f38e8159420 2023-10-10T20:34:19.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.706+0000 7f38ee1e6700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e80b11d0 con 0x7f38e815d230 2023-10-10T20:34:19.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.706+0000 7f38ee1e6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e80b14b0 con 0x7f38e81597f0 2023-10-10T20:34:19.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.706+0000 7f38ed1e4700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f38e8159420 0x7f38e80b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35668/0 (socket says 172.21.15.89:35668) 2023-10-10T20:34:19.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.706+0000 7f38ed1e4700 1 -- 172.21.15.89:0/3733546573 learned_addr learned my addr 172.21.15.89:0/3733546573 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:19.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.706+0000 7f38ec9e3700 1 --2- 172.21.15.89:0/3733546573 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38e81597f0 0x7f38e80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:19.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.706+0000 7f38ed9e5700 1 --2- 172.21.15.89:0/3733546573 >> v2:172.21.15.111:3300/0 conn(0x7f38e815d230 0x7f38e815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:19.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.706+0000 7f38dffff700 1 -- 172.21.15.89:0/3733546573 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2600093795 0 0) 0x7f38e4001110 con 0x7f38e8159420 2023-10-10T20:34:19.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.706+0000 7f38dffff700 1 -- 172.21.15.89:0/3733546573 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38d4003570 con 0x7f38e8159420 2023-10-10T20:34:19.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.706+0000 7f38ec9e3700 1 -- 172.21.15.89:0/3733546573 >> v2:172.21.15.111:3300/0 conn(0x7f38e815d230 msgr2=0x7f38e815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:19.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.706+0000 7f38ec9e3700 1 --2- 172.21.15.89:0/3733546573 >> v2:172.21.15.111:3300/0 conn(0x7f38e815d230 0x7f38e815d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.706+0000 7f38ec9e3700 1 -- 172.21.15.89:0/3733546573 >> v1:172.21.15.89:6789/0 conn(0x7f38e8159420 legacy=0x7f38e80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:19.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.706+0000 7f38ec9e3700 1 -- 172.21.15.89:0/3733546573 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e80b1790 con 0x7f38e81597f0 2023-10-10T20:34:19.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.707+0000 7f38ec9e3700 1 --2- 172.21.15.89:0/3733546573 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38e81597f0 0x7f38e80b2270 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f38d80098e0 tx=0x7f38d80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8205758c60e94c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:19.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.707+0000 7f38dffff700 1 -- 172.21.15.89:0/3733546573 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8011630 con 0x7f38e81597f0 2023-10-10T20:34:19.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.707+0000 7f38dffff700 1 -- 172.21.15.89:0/3733546573 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d8011790 con 0x7f38e81597f0 2023-10-10T20:34:19.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.707+0000 7f38dffff700 1 -- 172.21.15.89:0/3733546573 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8011b30 con 0x7f38e81597f0 2023-10-10T20:34:19.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.707+0000 7f38ee1e6700 1 -- 172.21.15.89:0/3733546573 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38e81597f0 msgr2=0x7f38e80b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:19.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.707+0000 7f38ee1e6700 1 --2- 172.21.15.89:0/3733546573 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38e81597f0 0x7f38e80b2270 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f38d80098e0 tx=0x7f38d80091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.707+0000 7f38ee1e6700 1 -- 172.21.15.89:0/3733546573 shutdown_connections 2023-10-10T20:34:19.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.707+0000 7f38ee1e6700 1 --2- 172.21.15.89:0/3733546573 >> v2:172.21.15.111:3300/0 conn(0x7f38e815d230 0x7f38e815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.707+0000 7f38ee1e6700 1 --2- 172.21.15.89:0/3733546573 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38e81597f0 0x7f38e80b2270 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.707+0000 7f38ee1e6700 1 -- 172.21.15.89:0/3733546573 >> 172.21.15.89:0/3733546573 conn(0x7f38e80d0660 msgr2=0x7f38e80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:19.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.707+0000 7f38ee1e6700 1 -- 172.21.15.89:0/3733546573 shutdown_connections 2023-10-10T20:34:19.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.707+0000 7f38ee1e6700 1 -- 172.21.15.89:0/3733546573 wait complete. 2023-10-10T20:34:19.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.708+0000 7f38ee1e6700 1 Processor -- start 2023-10-10T20:34:19.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.709+0000 7f38ee1e6700 1 -- start start 2023-10-10T20:34:19.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.709+0000 7f38ee1e6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e8159420 0x7f38e80af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:19.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.709+0000 7f38ee1e6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38e81597f0 0x7f38e80c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:19.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.709+0000 7f38ee1e6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e815d230 0x7f38e80c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:19.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.709+0000 7f38ee1e6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e816b130 con 0x7f38e8159420 2023-10-10T20:34:19.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.709+0000 7f38ee1e6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e815bbc0 con 0x7f38e815d230 2023-10-10T20:34:19.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.709+0000 7f38ee1e6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e815bea0 con 0x7f38e81597f0 2023-10-10T20:34:19.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.710+0000 7f38ed1e4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e8159420 0x7f38e80af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:19.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.710+0000 7f38ed1e4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e8159420 0x7f38e80af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41542/0 (socket says 172.21.15.89:41542) 2023-10-10T20:34:19.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.710+0000 7f38ec9e3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38e81597f0 0x7f38e80c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:19.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.710+0000 7f38ed1e4700 1 -- 172.21.15.89:0/2002721540 learned_addr learned my addr 172.21.15.89:0/2002721540 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:19.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.710+0000 7f38ed9e5700 1 --2- 172.21.15.89:0/2002721540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e815d230 0x7f38e80c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:19.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.710+0000 7f38ed1e4700 1 -- 172.21.15.89:0/2002721540 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38e81597f0 msgr2=0x7f38e80c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:19.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.710+0000 7f38ed1e4700 1 --2- 172.21.15.89:0/2002721540 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38e81597f0 0x7f38e80c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.710+0000 7f38ed1e4700 1 -- 172.21.15.89:0/2002721540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e815d230 msgr2=0x7f38e80c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:19.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.710+0000 7f38ed1e4700 1 --2- 172.21.15.89:0/2002721540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e815d230 0x7f38e80c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.710+0000 7f38ed1e4700 1 -- 172.21.15.89:0/2002721540 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d8009420 con 0x7f38e8159420 2023-10-10T20:34:19.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.710+0000 7f38ec9e3700 1 --2- 172.21.15.89:0/2002721540 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38e81597f0 0x7f38e80c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:19.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.710+0000 7f38ed1e4700 1 --2- 172.21.15.89:0/2002721540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e8159420 0x7f38e80af240 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f38e400c7c0 tx=0x7f38e4007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:19.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.710+0000 7f38dd7fa700 1 -- 172.21.15.89:0/2002721540 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4005c00 con 0x7f38e8159420 2023-10-10T20:34:19.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.710+0000 7f38dd7fa700 1 -- 172.21.15.89:0/2002721540 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e4005d60 con 0x7f38e8159420 2023-10-10T20:34:19.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.710+0000 7f38ee1e6700 1 -- 172.21.15.89:0/2002721540 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e815c180 con 0x7f38e8159420 2023-10-10T20:34:19.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.710+0000 7f38dd7fa700 1 -- 172.21.15.89:0/2002721540 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4007320 con 0x7f38e8159420 2023-10-10T20:34:19.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.710+0000 7f38ee1e6700 1 -- 172.21.15.89:0/2002721540 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e80af780 con 0x7f38e8159420 2023-10-10T20:34:19.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.710+0000 7f38dd7fa700 1 -- 172.21.15.89:0/2002721540 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f38e401c6c0 con 0x7f38e8159420 2023-10-10T20:34:19.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.710+0000 7f38dd7fa700 1 -- 172.21.15.89:0/2002721540 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f38e4007710 con 0x7f38e8159420 2023-10-10T20:34:19.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.711+0000 7f38ee1e6700 1 -- 172.21.15.89:0/2002721540 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e804c3c0 con 0x7f38e8159420 2023-10-10T20:34:19.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.714+0000 7f38dd7fa700 1 -- 172.21.15.89:0/2002721540 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f38e4017070 con 0x7f38e8159420 2023-10-10T20:34:19.755 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:19.753+0000 7f02c1de9180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-10T20:34:19.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.783+0000 7f38ee1e6700 1 -- 172.21.15.89:0/2002721540 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38e80b1cb0 con 0x7f38e8159420 2023-10-10T20:34:19.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.784+0000 7f38dd7fa700 1 -- 172.21.15.89:0/2002721540 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f38e815dbd0 con 0x7f38e8159420 2023-10-10T20:34:19.787 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:19.787 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":10,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:19.532466+0000","last_up_change":"0.000000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:34:19.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.784+0000 7f38ee1e6700 1 -- 172.21.15.89:0/2002721540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e8159420 msgr2=0x7f38e80af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:19.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.784+0000 7f38ee1e6700 1 --2- 172.21.15.89:0/2002721540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e8159420 0x7f38e80af240 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f38e400c7c0 tx=0x7f38e4007c70 comp rx=0 tx=0).stop 2023-10-10T20:34:19.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.785+0000 7f38ee1e6700 1 -- 172.21.15.89:0/2002721540 shutdown_connections 2023-10-10T20:34:19.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.785+0000 7f38ee1e6700 1 --2- 172.21.15.89:0/2002721540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e815d230 0x7f38e80c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.785+0000 7f38ee1e6700 1 --2- 172.21.15.89:0/2002721540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e8159420 0x7f38e80af240 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.785+0000 7f38ee1e6700 1 --2- 172.21.15.89:0/2002721540 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38e81597f0 0x7f38e80c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:19.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.785+0000 7f38ee1e6700 1 -- 172.21.15.89:0/2002721540 >> 172.21.15.89:0/2002721540 conn(0x7f38e80d0660 msgr2=0x7f38e81552b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:19.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.785+0000 7f38ee1e6700 1 -- 172.21.15.89:0/2002721540 shutdown_connections 2023-10-10T20:34:19.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:19.785+0000 7f38ee1e6700 1 -- 172.21.15.89:0/2002721540 wait complete. 2023-10-10T20:34:19.798 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-10T20:34:19.799 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-10T20:34:19.830 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:19.828+0000 7fb98747c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-10T20:34:19.835 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:19.833+0000 7f02c1de9180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-10T20:34:19.907 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:19.905+0000 7fb98747c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-10T20:34:19.912 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:19.910+0000 7f02c1de9180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-10T20:34:19.986 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:19.984+0000 7fb98747c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-10T20:34:20.157 INFO:tasks.ceph.osd.4.smithi111.stderr:2023-10-10T20:34:20.154+0000 7f6027e6d540 -1 Falling back to public interface 2023-10-10T20:34:20.164 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-10-10T20:34:20.162+0000 7f8c5e395540 -1 Falling back to public interface 2023-10-10T20:34:20.165 INFO:tasks.ceph.osd.6.smithi111.stderr:2023-10-10T20:34:20.163+0000 7fa35eefa540 -1 Falling back to public interface 2023-10-10T20:34:20.170 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-10-10T20:34:20.168+0000 7fefea339540 -1 Falling back to public interface 2023-10-10T20:34:20.183 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-10-10T20:34:20.181+0000 7f22f35fb540 -1 Falling back to public interface 2023-10-10T20:34:20.184 INFO:tasks.ceph.osd.5.smithi111.stderr:2023-10-10T20:34:20.181+0000 7f5a82d5a540 -1 Falling back to public interface 2023-10-10T20:34:20.184 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-10-10T20:34:20.182+0000 7f5b373d0540 -1 Falling back to public interface 2023-10-10T20:34:20.192 INFO:tasks.ceph.osd.7.smithi111.stderr:2023-10-10T20:34:20.190+0000 7f1bb44f0540 -1 Falling back to public interface 2023-10-10T20:34:20.503 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:20.501+0000 7f02c1de9180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-10T20:34:20.565 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:20.563+0000 7fb98747c180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-10T20:34:20.604 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-10T20:34:20.639 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:20.637+0000 7f02c1de9180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-10T20:34:20.702 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:20.700+0000 7fb98747c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-10T20:34:20.711 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.709+0000 7f3a03a3e700 1 Processor -- start 2023-10-10T20:34:20.712 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.710+0000 7f3a03a3e700 1 -- start start 2023-10-10T20:34:20.714 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.710+0000 7f3a03a3e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f39fc0aa870 0x7f39fc0abee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:20.714 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.710+0000 7f3a03a3e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39fc15a8e0 con 0x7f39fc153c90 2023-10-10T20:34:20.715 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.710+0000 7f3a03a3e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39fc15ae10 con 0x7f39fc06ace0 2023-10-10T20:34:20.715 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.710+0000 7f3a03a3e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39fc0ac420 con 0x7f39fc0aa870 2023-10-10T20:34:20.715 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.711+0000 7f3a017da700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f39fc153c90 0x7f39fc06a5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35680/0 (socket says 172.21.15.89:35680) 2023-10-10T20:34:20.715 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.711+0000 7f3a017da700 1 -- 172.21.15.89:0/1998664297 learned_addr learned my addr 172.21.15.89:0/1998664297 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:20.716 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.711+0000 7f3a00fd9700 1 --2- 172.21.15.89:0/1998664297 >> v2:172.21.15.89:3300/0 conn(0x7f39fc0aa870 0x7f39fc0abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:20.716 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.711+0000 7f39f3fff700 1 -- 172.21.15.89:0/1998664297 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583666985 0 0) 0x7f39fc15a8e0 con 0x7f39fc153c90 2023-10-10T20:34:20.716 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.711+0000 7f39f3fff700 1 -- 172.21.15.89:0/1998664297 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39e8003570 con 0x7f39fc153c90 2023-10-10T20:34:20.716 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.711+0000 7f3a00fd9700 1 -- 172.21.15.89:0/1998664297 >> v1:172.21.15.111:6789/0 conn(0x7f39fc06ace0 legacy=0x7f39fc15d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:20.716 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.711+0000 7f39f3fff700 1 -- 172.21.15.89:0/1998664297 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913824960 0 0) 0x7f39fc15ae10 con 0x7f39fc06ace0 2023-10-10T20:34:20.717 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.711+0000 7f3a00fd9700 1 -- 172.21.15.89:0/1998664297 >> v1:172.21.15.89:6789/0 conn(0x7f39fc153c90 legacy=0x7f39fc06a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:20.717 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.711+0000 7f3a00fd9700 1 -- 172.21.15.89:0/1998664297 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc0aa3e0 con 0x7f39fc0aa870 2023-10-10T20:34:20.717 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.711+0000 7f3a00fd9700 1 --2- 172.21.15.89:0/1998664297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc0aa870 0x7f39fc0abee0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0098e0 tx=0x7f39ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cacb6572b3b84b1a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:20.717 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.711+0000 7f39f3fff700 1 -- 172.21.15.89:0/1998664297 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec00b640 con 0x7f39fc0aa870 2023-10-10T20:34:20.718 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.711+0000 7f39f3fff700 1 -- 172.21.15.89:0/1998664297 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ec00b7a0 con 0x7f39fc0aa870 2023-10-10T20:34:20.718 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.711+0000 7f39f3fff700 1 -- 172.21.15.89:0/1998664297 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec00bb40 con 0x7f39fc0aa870 2023-10-10T20:34:20.718 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.712+0000 7f3a03a3e700 1 -- 172.21.15.89:0/1998664297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc0aa870 msgr2=0x7f39fc0abee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:20.718 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.712+0000 7f3a03a3e700 1 --2- 172.21.15.89:0/1998664297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc0aa870 0x7f39fc0abee0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0098e0 tx=0x7f39ec0091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:20.719 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.712+0000 7f3a03a3e700 1 -- 172.21.15.89:0/1998664297 shutdown_connections 2023-10-10T20:34:20.719 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.712+0000 7f3a03a3e700 1 --2- 172.21.15.89:0/1998664297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc0aa870 0x7f39fc0abee0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:20.719 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.712+0000 7f3a03a3e700 1 -- 172.21.15.89:0/1998664297 >> 172.21.15.89:0/1998664297 conn(0x7f39fc067610 msgr2=0x7f39fc0679e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:20.719 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.712+0000 7f3a03a3e700 1 -- 172.21.15.89:0/1998664297 shutdown_connections 2023-10-10T20:34:20.719 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.712+0000 7f3a03a3e700 1 -- 172.21.15.89:0/1998664297 wait complete. 2023-10-10T20:34:20.722 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.713+0000 7f3a03a3e700 1 Processor -- start 2023-10-10T20:34:20.722 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.713+0000 7f3a03a3e700 1 -- start start 2023-10-10T20:34:20.722 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.714+0000 7f3a03a3e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc06ace0 0x7f39fc141840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:20.723 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.714+0000 7f3a03a3e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39fc0aa870 0x7f39fc0afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:20.723 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.714+0000 7f3a03a3e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f39fc153c90 0x7f39fc0cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:20.723 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.714+0000 7f3a03a3e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc14dc90 con 0x7f39fc06ace0 2023-10-10T20:34:20.723 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.714+0000 7f3a03a3e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc0b1840 con 0x7f39fc0aa870 2023-10-10T20:34:20.724 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.714+0000 7f3a03a3e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc0b1230 con 0x7f39fc153c90 2023-10-10T20:34:20.724 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.714+0000 7f3a017da700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc06ace0 0x7f39fc141840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:20.724 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.714+0000 7f3a017da700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc06ace0 0x7f39fc141840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41568/0 (socket says 172.21.15.89:41568) 2023-10-10T20:34:20.724 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.714+0000 7f3a017da700 1 -- 172.21.15.89:0/1632763504 learned_addr learned my addr 172.21.15.89:0/1632763504 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:20.724 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.714+0000 7f3a00fd9700 1 --2- 172.21.15.89:0/1632763504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39fc0aa870 0x7f39fc0afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:20.724 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.714+0000 7f3a01fdb700 1 --2- 172.21.15.89:0/1632763504 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f39fc153c90 0x7f39fc0cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:20.725 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.714+0000 7f3a017da700 1 -- 172.21.15.89:0/1632763504 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f39fc153c90 msgr2=0x7f39fc0cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:20.725 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.714+0000 7f3a017da700 1 --2- 172.21.15.89:0/1632763504 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f39fc153c90 0x7f39fc0cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:20.725 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.714+0000 7f3a017da700 1 -- 172.21.15.89:0/1632763504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39fc0aa870 msgr2=0x7f39fc0afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:20.726 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.714+0000 7f3a017da700 1 --2- 172.21.15.89:0/1632763504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39fc0aa870 0x7f39fc0afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:20.726 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.715+0000 7f3a017da700 1 -- 172.21.15.89:0/1632763504 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ec009420 con 0x7f39fc06ace0 2023-10-10T20:34:20.726 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.715+0000 7f3a017da700 1 --2- 172.21.15.89:0/1632763504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc06ace0 0x7f39fc141840 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f39f800c580 tx=0x7f39f8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:20.726 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.715+0000 7f39f17fa700 1 -- 172.21.15.89:0/1632763504 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f8005b30 con 0x7f39fc06ace0 2023-10-10T20:34:20.726 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.715+0000 7f39f17fa700 1 -- 172.21.15.89:0/1632763504 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f8005c90 con 0x7f39fc06ace0 2023-10-10T20:34:20.727 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.715+0000 7f39f17fa700 1 -- 172.21.15.89:0/1632763504 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f8007320 con 0x7f39fc06ace0 2023-10-10T20:34:20.727 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.715+0000 7f3a03a3e700 1 -- 172.21.15.89:0/1632763504 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39fc0c92d0 con 0x7f39fc06ace0 2023-10-10T20:34:20.727 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.715+0000 7f3a03a3e700 1 -- 172.21.15.89:0/1632763504 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39fc0b0330 con 0x7f39fc06ace0 2023-10-10T20:34:20.727 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.715+0000 7f39f17fa700 1 -- 172.21.15.89:0/1632763504 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f39f801c440 con 0x7f39fc06ace0 2023-10-10T20:34:20.728 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.715+0000 7f39f17fa700 1 -- 172.21.15.89:0/1632763504 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f39f8005e00 con 0x7f39fc06ace0 2023-10-10T20:34:20.728 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.716+0000 7f3a03a3e700 1 -- 172.21.15.89:0/1632763504 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39fc069250 con 0x7f39fc06ace0 2023-10-10T20:34:20.728 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.718+0000 7f39f17fa700 1 -- 172.21.15.89:0/1632763504 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f39f8017070 con 0x7f39fc06ace0 2023-10-10T20:34:20.736 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:20.733+0000 7f02c1de9180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-10T20:34:20.788 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.786+0000 7f3a03a3e700 1 -- 172.21.15.89:0/1632763504 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39fc0a9c20 con 0x7f39fc06ace0 2023-10-10T20:34:20.789 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.787+0000 7f39f17fa700 1 -- 172.21.15.89:0/1632763504 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f39f801c920 con 0x7f39fc06ace0 2023-10-10T20:34:20.789 INFO:teuthology.misc.health.smithi089.stdout: 2023-10-10T20:34:20.790 INFO:teuthology.misc.health.smithi089.stdout:{"epoch":10,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:19.532466+0000","last_up_change":"0.000000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:34:20.791 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.788+0000 7f3a03a3e700 1 -- 172.21.15.89:0/1632763504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc06ace0 msgr2=0x7f39fc141840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:20.792 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.788+0000 7f3a03a3e700 1 --2- 172.21.15.89:0/1632763504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc06ace0 0x7f39fc141840 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f39f800c580 tx=0x7f39f8007df0 comp rx=0 tx=0).stop 2023-10-10T20:34:20.792 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.788+0000 7f3a03a3e700 1 -- 172.21.15.89:0/1632763504 shutdown_connections 2023-10-10T20:34:20.792 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.788+0000 7f3a03a3e700 1 --2- 172.21.15.89:0/1632763504 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f39fc153c90 0x7f39fc0cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:20.792 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.788+0000 7f3a03a3e700 1 --2- 172.21.15.89:0/1632763504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc06ace0 0x7f39fc141840 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:20.793 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.788+0000 7f3a03a3e700 1 --2- 172.21.15.89:0/1632763504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39fc0aa870 0x7f39fc0afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:20.793 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.788+0000 7f3a03a3e700 1 -- 172.21.15.89:0/1632763504 >> 172.21.15.89:0/1632763504 conn(0x7f39fc067610 msgr2=0x7f39fc1518a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:20.793 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.788+0000 7f3a03a3e700 1 -- 172.21.15.89:0/1632763504 shutdown_connections 2023-10-10T20:34:20.794 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:20.788+0000 7f3a03a3e700 1 -- 172.21.15.89:0/1632763504 wait complete. 2023-10-10T20:34:20.795 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:20.793+0000 7fb98747c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-10T20:34:20.802 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-10T20:34:20.986 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:20.984+0000 7f02c1de9180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-10T20:34:21.041 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:21.038+0000 7fb98747c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-10T20:34:21.065 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:21.063+0000 7f02c1de9180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-10T20:34:21.118 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:21.116+0000 7fb98747c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-10T20:34:21.887 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:21.885+0000 7f02c1de9180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-10T20:34:21.941 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:21.939+0000 7fb98747c180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-10T20:34:21.960 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:21.958+0000 7f02c1de9180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-10T20:34:22.014 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:22.012+0000 7fb98747c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-10T20:34:22.034 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:22.032+0000 7f02c1de9180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-10T20:34:22.089 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:22.086+0000 7fb98747c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-10T20:34:22.170 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:22.168+0000 7f02c1de9180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-10T20:34:22.222 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:22.220+0000 7fb98747c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-10T20:34:22.255 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:22.253+0000 7f02c1de9180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-10T20:34:22.307 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:22.304+0000 7fb98747c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-10T20:34:22.407 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:22.405+0000 7f02c1de9180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-10T20:34:22.459 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:22.457+0000 7fb98747c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-10T20:34:22.482 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:22.480+0000 7f02c1de9180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-10T20:34:22.536 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:22.534+0000 7fb98747c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-10T20:34:22.558 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:22.556+0000 7f02c1de9180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-10T20:34:22.612 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:22.610+0000 7fb98747c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-10T20:34:22.633 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:22.631+0000 7f02c1de9180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-10T20:34:22.697 INFO:tasks.ceph.mgr.x.smithi111.stderr:2023-10-10T20:34:22.695+0000 7fb98747c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-10T20:34:23.429 INFO:tasks.ceph.osd.4.smithi111.stderr:2023-10-10T20:34:23.426+0000 7f6027e6d540 -1 osd.4 0 log_to_monitors true 2023-10-10T20:34:23.657 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:23.654+0000 7f027b1e8700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-10T20:34:23.657 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:34:23.654+0000 7f027b1e8700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-10T20:34:23.673 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-10-10T20:34:23.671+0000 7fefea339540 -1 osd.0 0 log_to_monitors true 2023-10-10T20:34:23.695 INFO:tasks.ceph.osd.6.smithi111.stderr:2023-10-10T20:34:23.693+0000 7fa35eefa540 -1 osd.6 0 log_to_monitors true 2023-10-10T20:34:23.723 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-10-10T20:34:23.721+0000 7f5b373d0540 -1 osd.3 0 log_to_monitors true 2023-10-10T20:34:23.727 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-10-10T20:34:23.726+0000 7f22f35fb540 -1 osd.1 0 log_to_monitors true 2023-10-10T20:34:23.731 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-10-10T20:34:23.729+0000 7f8c5e395540 -1 osd.2 0 log_to_monitors true 2023-10-10T20:34:23.733 INFO:tasks.ceph.osd.5.smithi111.stderr:2023-10-10T20:34:23.731+0000 7f5a82d5a540 -1 osd.5 0 log_to_monitors true 2023-10-10T20:34:23.735 INFO:tasks.ceph.osd.7.smithi111.stderr:2023-10-10T20:34:23.733+0000 7f1bb44f0540 -1 osd.7 0 log_to_monitors true 2023-10-10T20:34:26.796 INFO:tasks.ceph.osd.7.smithi111.stderr:2023-10-10T20:34:26.793+0000 7f1bab112700 -1 osd.7 0 waiting for initial osdmap 2023-10-10T20:34:26.802 INFO:tasks.ceph.osd.7.smithi111.stderr:2023-10-10T20:34:26.800+0000 7f1ba3f09700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-10T20:34:26.828 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-10-10T20:34:26.825+0000 7f22ea21d700 -1 osd.1 0 waiting for initial osdmap 2023-10-10T20:34:26.831 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-10-10T20:34:26.829+0000 7f22e3014700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-10T20:34:26.866 INFO:tasks.ceph.osd.6.smithi111.stderr:2023-10-10T20:34:26.864+0000 7fa355b1c700 -1 osd.6 0 waiting for initial osdmap 2023-10-10T20:34:26.871 INFO:tasks.ceph.osd.6.smithi111.stderr:2023-10-10T20:34:26.869+0000 7fa34e913700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-10T20:34:26.876 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-10-10T20:34:26.875+0000 7f8c54fb7700 -1 osd.2 0 waiting for initial osdmap 2023-10-10T20:34:26.881 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-10-10T20:34:26.879+0000 7f8c4ddae700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-10T20:34:26.892 INFO:tasks.ceph.osd.5.smithi111.stderr:2023-10-10T20:34:26.890+0000 7f5a7997c700 -1 osd.5 0 waiting for initial osdmap 2023-10-10T20:34:26.897 INFO:tasks.ceph.osd.5.smithi111.stderr:2023-10-10T20:34:26.895+0000 7f5a72773700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-10T20:34:26.907 INFO:tasks.ceph.osd.4.smithi111.stderr:2023-10-10T20:34:26.905+0000 7f601ea8f700 -1 osd.4 0 waiting for initial osdmap 2023-10-10T20:34:26.911 INFO:tasks.ceph.osd.4.smithi111.stderr:2023-10-10T20:34:26.909+0000 7f6017886700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-10T20:34:26.931 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-10-10T20:34:26.929+0000 7fefe0f5b700 -1 osd.0 0 waiting for initial osdmap 2023-10-10T20:34:26.936 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-10-10T20:34:26.934+0000 7fefd9d52700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-10T20:34:26.944 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-10-10T20:34:26.942+0000 7f5b2dff2700 -1 osd.3 0 waiting for initial osdmap 2023-10-10T20:34:26.947 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-10-10T20:34:26.945+0000 7f5b26de9700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-10T20:34:27.608 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-10T20:34:27.720 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.717+0000 7ff27f330700 1 Processor -- start 2023-10-10T20:34:27.720 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.717+0000 7ff27f330700 1 -- start start 2023-10-10T20:34:27.732 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.718+0000 7ff27f330700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2780b1f30 0x7ff2780cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:27.733 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.718+0000 7ff27f330700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2780cd950 con 0x7ff2780cc550 2023-10-10T20:34:27.733 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.718+0000 7ff27f330700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2780ccd40 con 0x7ff2781578b0 2023-10-10T20:34:27.733 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.718+0000 7ff27f330700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2780cd020 con 0x7ff2780b1f30 2023-10-10T20:34:27.734 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff27d8cd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff2780cc550 0x7ff2780adf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35684/0 (socket says 172.21.15.89:35684) 2023-10-10T20:34:27.734 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff27d8cd700 1 -- 172.21.15.89:0/3380215611 learned_addr learned my addr 172.21.15.89:0/3380215611 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:27.735 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff27c8cb700 1 --2- 172.21.15.89:0/3380215611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2780b1f30 0x7ff2780cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:27.735 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff26ffff700 1 -- 172.21.15.89:0/3380215611 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210999520 0 0) 0x7ff2780cd950 con 0x7ff2780cc550 2023-10-10T20:34:27.735 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff26ffff700 1 -- 172.21.15.89:0/3380215611 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff264003570 con 0x7ff2780cc550 2023-10-10T20:34:27.736 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff26ffff700 1 -- 172.21.15.89:0/3380215611 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4224105788 0 0) 0x7ff264003570 con 0x7ff2780cc550 2023-10-10T20:34:27.736 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff26ffff700 1 -- 172.21.15.89:0/3380215611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2780b1f30 msgr2=0x7ff2780cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:27.736 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff26ffff700 1 --2- 172.21.15.89:0/3380215611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2780b1f30 0x7ff2780cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:27.737 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff26ffff700 1 -- 172.21.15.89:0/3380215611 >> v1:172.21.15.111:6789/0 conn(0x7ff2781578b0 legacy=0x7ff2780b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:27.737 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff26ffff700 1 -- 172.21.15.89:0/3380215611 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2780cd300 con 0x7ff2780cc550 2023-10-10T20:34:27.738 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff26ffff700 1 -- 172.21.15.89:0/3380215611 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7ff2700017f0 con 0x7ff2780cc550 2023-10-10T20:34:27.738 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff26ffff700 1 -- 172.21.15.89:0/3380215611 >> v1:172.21.15.89:6789/0 conn(0x7ff2780cc550 legacy=0x7ff2780adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:27.738 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff26ffff700 1 --2- 172.21.15.89:0/3380215611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2780b1f30 0x7ff264003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:27.739 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff26ffff700 1 --2- 172.21.15.89:0/3380215611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff264003fd0 0x7ff264004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:27.739 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff26ffff700 1 --2- 172.21.15.89:0/3380215611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff264004d50 0x7ff2640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:27.740 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff26ffff700 1 -- 172.21.15.89:0/3380215611 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2780cd020 con 0x7ff264003fd0 2023-10-10T20:34:27.740 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff26ffff700 1 -- 172.21.15.89:0/3380215611 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26400bbf0 con 0x7ff264004d50 2023-10-10T20:34:27.740 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff26ffff700 1 -- 172.21.15.89:0/3380215611 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26400bdb0 con 0x7ff2780b1f30 2023-10-10T20:34:27.741 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff27c8cb700 1 --2- 172.21.15.89:0/3380215611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff264003fd0 0x7ff264004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:27.741 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff27d0cc700 1 --2- 172.21.15.89:0/3380215611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2780b1f30 0x7ff264003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:27.741 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.719+0000 7ff27d8cd700 1 --2- 172.21.15.89:0/3380215611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff264004d50 0x7ff2640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:27.741 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.720+0000 7ff27c8cb700 1 -- 172.21.15.89:0/3380215611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2780b1f30 msgr2=0x7ff264003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:27.742 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.720+0000 7ff27c8cb700 1 --2- 172.21.15.89:0/3380215611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2780b1f30 0x7ff264003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:27.742 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.720+0000 7ff27c8cb700 1 -- 172.21.15.89:0/3380215611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff264004d50 msgr2=0x7ff2640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:27.742 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.720+0000 7ff27c8cb700 1 --2- 172.21.15.89:0/3380215611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff264004d50 0x7ff2640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:27.743 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.720+0000 7ff27c8cb700 1 -- 172.21.15.89:0/3380215611 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff264003570 con 0x7ff264003fd0 2023-10-10T20:34:27.743 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.720+0000 7ff27c8cb700 1 --2- 172.21.15.89:0/3380215611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff264003fd0 0x7ff264004810 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff268009c20 tx=0x7ff2680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83042d19027e2e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:27.743 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.720+0000 7ff26ffff700 1 -- 172.21.15.89:0/3380215611 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2680095b0 con 0x7ff264003fd0 2023-10-10T20:34:27.744 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.720+0000 7ff26ffff700 1 -- 172.21.15.89:0/3380215611 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26800e710 con 0x7ff264003fd0 2023-10-10T20:34:27.744 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.720+0000 7ff26ffff700 1 -- 172.21.15.89:0/3380215611 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26800eab0 con 0x7ff264003fd0 2023-10-10T20:34:27.744 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.720+0000 7ff27f330700 1 -- 172.21.15.89:0/3380215611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff264003fd0 msgr2=0x7ff264004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:27.745 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.720+0000 7ff27f330700 1 --2- 172.21.15.89:0/3380215611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff264003fd0 0x7ff264004810 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff268009c20 tx=0x7ff2680091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:27.745 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.720+0000 7ff27d8cd700 1 -- 172.21.15.89:0/3380215611 reap_dead start 2023-10-10T20:34:27.745 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.720+0000 7ff27f330700 1 -- 172.21.15.89:0/3380215611 shutdown_connections 2023-10-10T20:34:27.745 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.720+0000 7ff27f330700 1 -- 172.21.15.89:0/3380215611 >> 172.21.15.89:0/3380215611 conn(0x7ff278087ff0 msgr2=0x7ff2780883c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:27.746 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.721+0000 7ff27f330700 1 -- 172.21.15.89:0/3380215611 shutdown_connections 2023-10-10T20:34:27.746 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.721+0000 7ff27f330700 1 -- 172.21.15.89:0/3380215611 wait complete. 2023-10-10T20:34:27.746 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.722+0000 7ff27f330700 1 Processor -- start 2023-10-10T20:34:27.747 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.722+0000 7ff27f330700 1 -- start start 2023-10-10T20:34:27.747 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.722+0000 7ff27f330700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff264003fd0 0x7ff278159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:27.747 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.722+0000 7ff27f330700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff27815a880 0x7ff2781567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:27.747 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.722+0000 7ff27f330700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff278157c80 0x7ff278099b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:27.747 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.722+0000 7ff27f330700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff278159190 con 0x7ff264003fd0 2023-10-10T20:34:27.748 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.722+0000 7ff27f330700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff278156cf0 con 0x7ff278157c80 2023-10-10T20:34:27.748 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.722+0000 7ff27f330700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff278158580 con 0x7ff27815a880 2023-10-10T20:34:27.748 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.723+0000 7ff27d0cc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff264003fd0 0x7ff278159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:27.748 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.723+0000 7ff27d0cc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff264003fd0 0x7ff278159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41670/0 (socket says 172.21.15.89:41670) 2023-10-10T20:34:27.748 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.723+0000 7ff27d0cc700 1 -- 172.21.15.89:0/558048525 learned_addr learned my addr 172.21.15.89:0/558048525 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:27.749 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.723+0000 7ff27d8cd700 1 --2- 172.21.15.89:0/558048525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff278157c80 0x7ff278099b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:27.749 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.723+0000 7ff27c8cb700 1 --2- 172.21.15.89:0/558048525 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff27815a880 0x7ff2781567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:27.749 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.723+0000 7ff27d0cc700 1 -- 172.21.15.89:0/558048525 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff27815a880 msgr2=0x7ff2781567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:27.749 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.723+0000 7ff27d0cc700 1 --2- 172.21.15.89:0/558048525 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff27815a880 0x7ff2781567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:27.750 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.723+0000 7ff27d0cc700 1 -- 172.21.15.89:0/558048525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff278157c80 msgr2=0x7ff278099b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:27.750 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.723+0000 7ff27d0cc700 1 --2- 172.21.15.89:0/558048525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff278157c80 0x7ff278099b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:27.750 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.723+0000 7ff27d0cc700 1 -- 172.21.15.89:0/558048525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff278158860 con 0x7ff264003fd0 2023-10-10T20:34:27.750 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.723+0000 7ff27d8cd700 1 --2- 172.21.15.89:0/558048525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff278157c80 0x7ff278099b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:27.750 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.723+0000 7ff27d0cc700 1 --2- 172.21.15.89:0/558048525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff264003fd0 0x7ff278159620 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff27400f900 tx=0x7ff274007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:27.751 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.723+0000 7ff26d7fa700 1 -- 172.21.15.89:0/558048525 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27400fea0 con 0x7ff264003fd0 2023-10-10T20:34:27.751 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.723+0000 7ff26d7fa700 1 -- 172.21.15.89:0/558048525 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27400d850 con 0x7ff264003fd0 2023-10-10T20:34:27.751 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.723+0000 7ff26d7fa700 1 -- 172.21.15.89:0/558048525 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27400dc10 con 0x7ff264003fd0 2023-10-10T20:34:27.751 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.723+0000 7ff27f330700 1 -- 172.21.15.89:0/558048525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff278158b40 con 0x7ff264003fd0 2023-10-10T20:34:27.751 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.723+0000 7ff27f330700 1 -- 172.21.15.89:0/558048525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2781580f0 con 0x7ff264003fd0 2023-10-10T20:34:27.752 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.724+0000 7ff27f330700 1 -- 172.21.15.89:0/558048525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff278069e90 con 0x7ff264003fd0 2023-10-10T20:34:27.752 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.724+0000 7ff26d7fa700 1 -- 172.21.15.89:0/558048525 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7ff27401c400 con 0x7ff264003fd0 2023-10-10T20:34:27.752 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.725+0000 7ff26d7fa700 1 --2- 172.21.15.89:0/558048525 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff25c066ea0 0x7ff25c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:27.752 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.725+0000 7ff26d7fa700 1 -- 172.21.15.89:0/558048525 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7ff274083b00 con 0x7ff264003fd0 2023-10-10T20:34:27.752 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.725+0000 7ff27c8cb700 1 --2- 172.21.15.89:0/558048525 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff25c066ea0 0x7ff25c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:27.753 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.725+0000 7ff27c8cb700 1 --2- 172.21.15.89:0/558048525 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff25c066ea0 0x7ff25c069350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff268009c20 tx=0x7ff26801a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:27.753 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.729+0000 7ff26d7fa700 1 -- 172.21.15.89:0/558048525 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff274051c90 con 0x7ff264003fd0 2023-10-10T20:34:27.929 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.927+0000 7ff27f330700 1 -- 172.21.15.89:0/558048525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2780ca4d0 con 0x7ff264003fd0 2023-10-10T20:34:27.931 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.928+0000 7ff26d7fa700 1 -- 172.21.15.89:0/558048525 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10396 (secure 0 0 0) 0x7ff274083e60 con 0x7ff264003fd0 2023-10-10T20:34:27.931 INFO:teuthology.misc.health.smithi089.stdout: 2023-10-10T20:34:27.931 INFO:teuthology.misc.health.smithi089.stdout:{"epoch":13,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:27.802670+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:34:27.932 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.930+0000 7ff27f330700 1 -- 172.21.15.89:0/558048525 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff25c066ea0 msgr2=0x7ff25c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:27.933 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.930+0000 7ff27f330700 1 --2- 172.21.15.89:0/558048525 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff25c066ea0 0x7ff25c069350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff268009c20 tx=0x7ff26801a000 comp rx=0 tx=0).stop 2023-10-10T20:34:27.933 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.930+0000 7ff27f330700 1 -- 172.21.15.89:0/558048525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff264003fd0 msgr2=0x7ff278159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:27.933 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.930+0000 7ff27f330700 1 --2- 172.21.15.89:0/558048525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff264003fd0 0x7ff278159620 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff27400f900 tx=0x7ff274007d80 comp rx=0 tx=0).stop 2023-10-10T20:34:27.933 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.930+0000 7ff27f330700 1 -- 172.21.15.89:0/558048525 shutdown_connections 2023-10-10T20:34:27.933 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.930+0000 7ff27f330700 1 --2- 172.21.15.89:0/558048525 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff25c066ea0 0x7ff25c069350 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:27.934 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.930+0000 7ff27f330700 1 --2- 172.21.15.89:0/558048525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff278157c80 0x7ff278099b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:27.934 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.930+0000 7ff27f330700 1 --2- 172.21.15.89:0/558048525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff264003fd0 0x7ff278159620 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:27.934 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.930+0000 7ff27f330700 1 --2- 172.21.15.89:0/558048525 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff27815a880 0x7ff2781567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:27.934 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.930+0000 7ff27f330700 1 -- 172.21.15.89:0/558048525 >> 172.21.15.89:0/558048525 conn(0x7ff278087ff0 msgr2=0x7ff2780ca930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:27.935 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.930+0000 7ff27f330700 1 -- 172.21.15.89:0/558048525 shutdown_connections 2023-10-10T20:34:27.935 INFO:teuthology.misc.health.smithi089.stderr:2023-10-10T20:34:27.930+0000 7ff27f330700 1 -- 172.21.15.89:0/558048525 wait complete. 2023-10-10T20:34:27.943 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-10T20:34:27.943 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-10T20:34:27.944 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:34:27.944 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:34:27.946 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:34:27.946 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:34:27.947 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:34:27.949 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:34:27.949 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:34:27.949 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:34:27.952 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:34:27.952 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:34:27.953 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:34:27.955 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:34:27.955 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-10T20:34:27.956 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:34:28.019 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:34:28.021 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-10T20:34:28.025 INFO:tasks.ceph.mds.b.smithi111.stdout:starting mds.b at 2023-10-10T20:34:28.026 INFO:tasks.ceph.mds.d.smithi111.stdout:starting mds.d at 2023-10-10T20:34:28.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.074+0000 7f26b4f0b700 1 Processor -- start 2023-10-10T20:34:28.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.074+0000 7f26b4f0b700 1 -- start start 2023-10-10T20:34:28.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.075+0000 7f26b4f0b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f26b00c91e0 0x7f26b00cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:28.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.075+0000 7f26b4f0b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b00cc210 con 0x7f26b006ae10 2023-10-10T20:34:28.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.075+0000 7f26b4f0b700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b00cc4f0 con 0x7f26b006aa40 2023-10-10T20:34:28.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.075+0000 7f26b4f0b700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b00cc7d0 con 0x7f26b00c91e0 2023-10-10T20:34:28.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.075+0000 7f26aed9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f26b006ae10 0x7f26b006b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35710/0 (socket says 172.21.15.89:35710) 2023-10-10T20:34:28.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.075+0000 7f26aed9d700 1 -- 172.21.15.89:0/4065836414 learned_addr learned my addr 172.21.15.89:0/4065836414 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:28.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.075+0000 7f26a5d9b700 1 --2- 172.21.15.89:0/4065836414 >> v2:172.21.15.111:3300/0 conn(0x7f26b00c91e0 0x7f26b00cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:28.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.076+0000 7f26add9b700 1 -- 172.21.15.89:0/4065836414 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3500734793 0 0) 0x7f26b00cc4f0 con 0x7f26b006aa40 2023-10-10T20:34:28.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.076+0000 7f26add9b700 1 -- 172.21.15.89:0/4065836414 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f269c003570 con 0x7f26b006aa40 2023-10-10T20:34:28.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.076+0000 7f26add9b700 1 -- 172.21.15.89:0/4065836414 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228345817 0 0) 0x7f26b00cc210 con 0x7f26b006ae10 2023-10-10T20:34:28.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.076+0000 7f26add9b700 1 -- 172.21.15.89:0/4065836414 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26b00cc4f0 con 0x7f26b006ae10 2023-10-10T20:34:28.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.076+0000 7f26a5d9b700 1 -- 172.21.15.89:0/4065836414 >> v1:172.21.15.111:6789/0 conn(0x7f26b006aa40 legacy=0x7f26b006bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:28.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.076+0000 7f26a5d9b700 1 -- 172.21.15.89:0/4065836414 >> v1:172.21.15.89:6789/0 conn(0x7f26b006ae10 legacy=0x7f26b006b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:28.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.076+0000 7f26a5d9b700 1 -- 172.21.15.89:0/4065836414 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b00ccab0 con 0x7f26b00c91e0 2023-10-10T20:34:28.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.076+0000 7f26a5d9b700 1 --2- 172.21.15.89:0/4065836414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26b00c91e0 0x7f26b00cbc00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2698009910 tx=0x7f26980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe3f6dba737ffe57 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:28.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.076+0000 7f26add9b700 1 -- 172.21.15.89:0/4065836414 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698011630 con 0x7f26b00c91e0 2023-10-10T20:34:28.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.076+0000 7f26add9b700 1 -- 172.21.15.89:0/4065836414 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2698011790 con 0x7f26b00c91e0 2023-10-10T20:34:28.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.076+0000 7f26add9b700 1 -- 172.21.15.89:0/4065836414 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698011b30 con 0x7f26b00c91e0 2023-10-10T20:34:28.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.076+0000 7f26b4f0b700 1 -- 172.21.15.89:0/4065836414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26b00c91e0 msgr2=0x7f26b00cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:28.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.076+0000 7f26b4f0b700 1 --2- 172.21.15.89:0/4065836414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26b00c91e0 0x7f26b00cbc00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2698009910 tx=0x7f26980091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.076+0000 7f26b4f0b700 1 -- 172.21.15.89:0/4065836414 shutdown_connections 2023-10-10T20:34:28.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.076+0000 7f26b4f0b700 1 --2- 172.21.15.89:0/4065836414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26b00c91e0 0x7f26b00cbc00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.076+0000 7f26b4f0b700 1 -- 172.21.15.89:0/4065836414 >> 172.21.15.89:0/4065836414 conn(0x7f26b00cd670 msgr2=0x7f26b00cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:28.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.076+0000 7f26b4f0b700 1 -- 172.21.15.89:0/4065836414 shutdown_connections 2023-10-10T20:34:28.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.077+0000 7f26b4f0b700 1 -- 172.21.15.89:0/4065836414 wait complete. 2023-10-10T20:34:28.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.078+0000 7f26b4f0b700 1 Processor -- start 2023-10-10T20:34:28.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.078+0000 7f26b4f0b700 1 -- start start 2023-10-10T20:34:28.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.078+0000 7f26b4f0b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26b006aa40 0x7f26b014e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:28.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.078+0000 7f26b4f0b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26b006ae10 0x7f26b00b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:28.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.078+0000 7f26b4f0b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26b00c91e0 0x7f26b00c3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:28.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.078+0000 7f26b4f0b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b00b2650 con 0x7f26b006aa40 2023-10-10T20:34:28.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.078+0000 7f26b4f0b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b00b1c80 con 0x7f26b006ae10 2023-10-10T20:34:28.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.078+0000 7f26b4f0b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b016aa70 con 0x7f26b00c91e0 2023-10-10T20:34:28.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.078+0000 7f26ae59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26b006aa40 0x7f26b014e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:28.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.079+0000 7f26ae59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26b006aa40 0x7f26b014e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41720/0 (socket says 172.21.15.89:41720) 2023-10-10T20:34:28.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.079+0000 7f26ae59c700 1 -- 172.21.15.89:0/603679046 learned_addr learned my addr 172.21.15.89:0/603679046 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:28.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.079+0000 7f26aed9d700 1 --2- 172.21.15.89:0/603679046 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26b00c91e0 0x7f26b00c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:28.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.079+0000 7f26a5d9b700 1 --2- 172.21.15.89:0/603679046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26b006ae10 0x7f26b00b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:28.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.079+0000 7f26ae59c700 1 -- 172.21.15.89:0/603679046 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26b00c91e0 msgr2=0x7f26b00c3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:28.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.079+0000 7f26ae59c700 1 --2- 172.21.15.89:0/603679046 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26b00c91e0 0x7f26b00c3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.079+0000 7f26ae59c700 1 -- 172.21.15.89:0/603679046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26b006ae10 msgr2=0x7f26b00b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:28.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.079+0000 7f26ae59c700 1 --2- 172.21.15.89:0/603679046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26b006ae10 0x7f26b00b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.079+0000 7f26ae59c700 1 -- 172.21.15.89:0/603679046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2698009420 con 0x7f26b006aa40 2023-10-10T20:34:28.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.079+0000 7f26a5d9b700 1 --2- 172.21.15.89:0/603679046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26b006ae10 0x7f26b00b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:28.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.079+0000 7f26aed9d700 1 --2- 172.21.15.89:0/603679046 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26b00c91e0 0x7f26b00c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:28.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.079+0000 7f26ae59c700 1 --2- 172.21.15.89:0/603679046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26b006aa40 0x7f26b014e450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f26a000c7f0 tx=0x7f26a0007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:28.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.079+0000 7f26a6ffd700 1 -- 172.21.15.89:0/603679046 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a0016070 con 0x7f26b006aa40 2023-10-10T20:34:28.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.079+0000 7f26a6ffd700 1 -- 172.21.15.89:0/603679046 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a0011680 con 0x7f26b006aa40 2023-10-10T20:34:28.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.079+0000 7f26a6ffd700 1 -- 172.21.15.89:0/603679046 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a0011a40 con 0x7f26b006aa40 2023-10-10T20:34:28.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.079+0000 7f26b4f0b700 1 -- 172.21.15.89:0/603679046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b00b1630 con 0x7f26b006aa40 2023-10-10T20:34:28.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.079+0000 7f26b4f0b700 1 -- 172.21.15.89:0/603679046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b00b1250 con 0x7f26b006aa40 2023-10-10T20:34:28.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.080+0000 7f26b4f0b700 1 -- 172.21.15.89:0/603679046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b00b14a0 con 0x7f26b006aa40 2023-10-10T20:34:28.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.080+0000 7f26a6ffd700 1 -- 172.21.15.89:0/603679046 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f26a0012440 con 0x7f26b006aa40 2023-10-10T20:34:28.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.081+0000 7f26a6ffd700 1 --2- 172.21.15.89:0/603679046 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2690066ea0 0x7f2690069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:28.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.081+0000 7f26a5d9b700 1 --2- 172.21.15.89:0/603679046 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2690066ea0 0x7f2690069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:28.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.081+0000 7f26a6ffd700 1 -- 172.21.15.89:0/603679046 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f26a0083ce0 con 0x7f26b006aa40 2023-10-10T20:34:28.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.081+0000 7f26a5d9b700 1 --2- 172.21.15.89:0/603679046 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2690066ea0 0x7f2690069350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2698009860 tx=0x7f2698017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:28.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.085+0000 7f26a6ffd700 1 -- 172.21.15.89:0/603679046 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26a00515c0 con 0x7f26b006aa40 2023-10-10T20:34:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.268+0000 7f26b4f0b700 1 -- 172.21.15.89:0/603679046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26b004c3c0 con 0x7f26b006aa40 2023-10-10T20:34:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.269+0000 7f26a6ffd700 1 -- 172.21.15.89:0/603679046 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10396 (secure 0 0 0) 0x7f26a0054be0 con 0x7f26b006aa40 2023-10-10T20:34:28.271 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:28.271 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":13,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:27.802670+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:34:28.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.271+0000 7f26b4f0b700 1 -- 172.21.15.89:0/603679046 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2690066ea0 msgr2=0x7f2690069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:28.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.271+0000 7f26b4f0b700 1 --2- 172.21.15.89:0/603679046 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2690066ea0 0x7f2690069350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2698009860 tx=0x7f2698017000 comp rx=0 tx=0).stop 2023-10-10T20:34:28.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.271+0000 7f26b4f0b700 1 -- 172.21.15.89:0/603679046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26b006aa40 msgr2=0x7f26b014e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:28.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.271+0000 7f26b4f0b700 1 --2- 172.21.15.89:0/603679046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26b006aa40 0x7f26b014e450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f26a000c7f0 tx=0x7f26a0007e20 comp rx=0 tx=0).stop 2023-10-10T20:34:28.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.271+0000 7f26b4f0b700 1 -- 172.21.15.89:0/603679046 shutdown_connections 2023-10-10T20:34:28.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.271+0000 7f26b4f0b700 1 --2- 172.21.15.89:0/603679046 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26b00c91e0 0x7f26b00c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.271+0000 7f26b4f0b700 1 --2- 172.21.15.89:0/603679046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26b006aa40 0x7f26b014e450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.271+0000 7f26b4f0b700 1 --2- 172.21.15.89:0/603679046 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2690066ea0 0x7f2690069350 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.271+0000 7f26b4f0b700 1 --2- 172.21.15.89:0/603679046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26b006ae10 0x7f26b00b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.271+0000 7f26b4f0b700 1 -- 172.21.15.89:0/603679046 >> 172.21.15.89:0/603679046 conn(0x7f26b00cd670 msgr2=0x7f26b0150160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:28.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.271+0000 7f26b4f0b700 1 -- 172.21.15.89:0/603679046 shutdown_connections 2023-10-10T20:34:28.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.271+0000 7f26b4f0b700 1 -- 172.21.15.89:0/603679046 wait complete. 2023-10-10T20:34:28.285 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2023-10-10T20:34:28.285 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:34:28.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.397+0000 7f202dc0c700 1 Processor -- start 2023-10-10T20:34:28.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.397+0000 7f202dc0c700 1 -- start start 2023-10-10T20:34:28.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.397+0000 7f202dc0c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2028159890 0x7f202815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:28.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f202dc0c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202815a160 con 0x7f2028069560 2023-10-10T20:34:28.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f202dc0c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20280d13a0 con 0x7f2028159c60 2023-10-10T20:34:28.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f202dc0c700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f202815e140 con 0x7f2028159890 2023-10-10T20:34:28.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f2027fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2028069560 0x7f20280ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48954/0 (socket says 172.21.15.89:48954) 2023-10-10T20:34:28.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f2027fff700 1 -- 172.21.15.89:0/3244235450 learned_addr learned my addr 172.21.15.89:0/3244235450 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:28.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f20267fc700 1 -- 172.21.15.89:0/3244235450 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1653025974 0 0) 0x7f202815a160 con 0x7f2028069560 2023-10-10T20:34:28.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f20277fe700 1 --2- 172.21.15.89:0/3244235450 >> v2:172.21.15.111:3300/0 conn(0x7f2028159890 0x7f202815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:28.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f20267fc700 1 -- 172.21.15.89:0/3244235450 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2014003570 con 0x7f2028069560 2023-10-10T20:34:28.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f20267fc700 1 -- 172.21.15.89:0/3244235450 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2227037851 0 0) 0x7f2014003570 con 0x7f2028069560 2023-10-10T20:34:28.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f20267fc700 1 -- 172.21.15.89:0/3244235450 >> v2:172.21.15.111:3300/0 conn(0x7f2028159890 msgr2=0x7f202815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:28.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f20267fc700 1 --2- 172.21.15.89:0/3244235450 >> v2:172.21.15.111:3300/0 conn(0x7f2028159890 0x7f202815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f20267fc700 1 -- 172.21.15.89:0/3244235450 >> v1:172.21.15.111:6789/0 conn(0x7f2028159c60 legacy=0x7f202815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:28.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f20267fc700 1 -- 172.21.15.89:0/3244235450 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202815a3d0 con 0x7f2028069560 2023-10-10T20:34:28.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f20267fc700 1 -- 172.21.15.89:0/3244235450 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f20200017f0 con 0x7f2028069560 2023-10-10T20:34:28.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f20267fc700 1 -- 172.21.15.89:0/3244235450 >> v1:172.21.15.89:6789/0 conn(0x7f2028069560 legacy=0x7f20280ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:28.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f20267fc700 1 --2- 172.21.15.89:0/3244235450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2014003a60 0x7f2014003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:28.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f20267fc700 1 --2- 172.21.15.89:0/3244235450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2014004810 0x7f2014004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:28.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f20267fc700 1 --2- 172.21.15.89:0/3244235450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20140051c0 0x7f201400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:28.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f20267fc700 1 -- 172.21.15.89:0/3244235450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202815e140 con 0x7f20140051c0 2023-10-10T20:34:28.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f20267fc700 1 -- 172.21.15.89:0/3244235450 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201400c0d0 con 0x7f2014004810 2023-10-10T20:34:28.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f20267fc700 1 -- 172.21.15.89:0/3244235450 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201400c290 con 0x7f2014003a60 2023-10-10T20:34:28.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f2027fff700 1 --2- 172.21.15.89:0/3244235450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20140051c0 0x7f201400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:28.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f20277fe700 1 --2- 172.21.15.89:0/3244235450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2014003a60 0x7f2014003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:28.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.398+0000 7f2026ffd700 1 --2- 172.21.15.89:0/3244235450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2014004810 0x7f2014004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:28.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.399+0000 7f2027fff700 1 -- 172.21.15.89:0/3244235450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2014003a60 msgr2=0x7f2014003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:28.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.399+0000 7f2027fff700 1 --2- 172.21.15.89:0/3244235450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2014003a60 0x7f2014003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.399+0000 7f2027fff700 1 -- 172.21.15.89:0/3244235450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2014004810 msgr2=0x7f2014004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:28.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.399+0000 7f2027fff700 1 --2- 172.21.15.89:0/3244235450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2014004810 0x7f2014004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.399+0000 7f2027fff700 1 -- 172.21.15.89:0/3244235450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2014003570 con 0x7f20140051c0 2023-10-10T20:34:28.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.399+0000 7f2027fff700 1 -- 172.21.15.89:0/3244235450 reap_dead start 2023-10-10T20:34:28.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.399+0000 7f2027fff700 1 --2- 172.21.15.89:0/3244235450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20140051c0 0x7f201400ba60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f202000a930 tx=0x7f20200021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e873ad226323052 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:28.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.399+0000 7f20267fc700 1 -- 172.21.15.89:0/3244235450 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20200070d0 con 0x7f20140051c0 2023-10-10T20:34:28.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.399+0000 7f20267fc700 1 -- 172.21.15.89:0/3244235450 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2020007230 con 0x7f20140051c0 2023-10-10T20:34:28.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.399+0000 7f20267fc700 1 -- 172.21.15.89:0/3244235450 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20200075d0 con 0x7f20140051c0 2023-10-10T20:34:28.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.399+0000 7f202dc0c700 1 -- 172.21.15.89:0/3244235450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20140051c0 msgr2=0x7f201400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:28.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.399+0000 7f202dc0c700 1 --2- 172.21.15.89:0/3244235450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20140051c0 0x7f201400ba60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f202000a930 tx=0x7f20200021d0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.399+0000 7f202dc0c700 1 -- 172.21.15.89:0/3244235450 shutdown_connections 2023-10-10T20:34:28.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.399+0000 7f202dc0c700 1 --2- 172.21.15.89:0/3244235450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20140051c0 0x7f201400ba60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.399+0000 7f202dc0c700 1 -- 172.21.15.89:0/3244235450 >> 172.21.15.89:0/3244235450 conn(0x7f2028068e20 msgr2=0x7f20280d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:28.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.400+0000 7f202dc0c700 1 -- 172.21.15.89:0/3244235450 shutdown_connections 2023-10-10T20:34:28.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.400+0000 7f202dc0c700 1 -- 172.21.15.89:0/3244235450 wait complete. 2023-10-10T20:34:28.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.401+0000 7f202dc0c700 1 Processor -- start 2023-10-10T20:34:28.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.401+0000 7f202dc0c700 1 -- start start 2023-10-10T20:34:28.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.401+0000 7f202dc0c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20140051c0 0x7f20281493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:28.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.401+0000 7f202dc0c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f20280b1b30 0x7f20280b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:28.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.401+0000 7f202dc0c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20280b4440 0x7f20280b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:28.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.401+0000 7f202dc0c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20280b24e0 con 0x7f20280b4440 2023-10-10T20:34:28.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.401+0000 7f202dc0c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20280cde00 con 0x7f20140051c0 2023-10-10T20:34:28.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.401+0000 7f202dc0c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20280cd1f0 con 0x7f20280b1b30 2023-10-10T20:34:28.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.402+0000 7f2027fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20280b4440 0x7f20280b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:28.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.402+0000 7f2027fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20280b4440 0x7f20280b48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54174/0 (socket says 172.21.15.89:54174) 2023-10-10T20:34:28.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.402+0000 7f2027fff700 1 -- 172.21.15.89:0/444491847 learned_addr learned my addr 172.21.15.89:0/444491847 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:28.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.402+0000 7f2026ffd700 1 --2- 172.21.15.89:0/444491847 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f20280b1b30 0x7f20280b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:28.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.402+0000 7f20277fe700 1 --2- 172.21.15.89:0/444491847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20140051c0 0x7f20281493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:28.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.402+0000 7f2027fff700 1 -- 172.21.15.89:0/444491847 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f20280b1b30 msgr2=0x7f20280b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:28.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.402+0000 7f2027fff700 1 --2- 172.21.15.89:0/444491847 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f20280b1b30 0x7f20280b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.402+0000 7f2027fff700 1 -- 172.21.15.89:0/444491847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20140051c0 msgr2=0x7f20281493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:28.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.402+0000 7f2027fff700 1 --2- 172.21.15.89:0/444491847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20140051c0 0x7f20281493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.402+0000 7f2027fff700 1 -- 172.21.15.89:0/444491847 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20280cd4d0 con 0x7f20280b4440 2023-10-10T20:34:28.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.402+0000 7f2027fff700 1 --2- 172.21.15.89:0/444491847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20280b4440 0x7f20280b48b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f20280d03b0 tx=0x7f2020016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:28.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.402+0000 7f201f7fe700 1 -- 172.21.15.89:0/444491847 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20200075b0 con 0x7f20280b4440 2023-10-10T20:34:28.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.402+0000 7f201f7fe700 1 -- 172.21.15.89:0/444491847 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202001d420 con 0x7f20280b4440 2023-10-10T20:34:28.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.402+0000 7f202dc0c700 1 -- 172.21.15.89:0/444491847 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20280cd7b0 con 0x7f20280b4440 2023-10-10T20:34:28.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.402+0000 7f201f7fe700 1 -- 172.21.15.89:0/444491847 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202001d7e0 con 0x7f20280b4440 2023-10-10T20:34:28.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.402+0000 7f202dc0c700 1 -- 172.21.15.89:0/444491847 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20280b4df0 con 0x7f20280b4440 2023-10-10T20:34:28.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.403+0000 7f202dc0c700 1 -- 172.21.15.89:0/444491847 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20280cdf70 con 0x7f20280b4440 2023-10-10T20:34:28.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.404+0000 7f201f7fe700 1 -- 172.21.15.89:0/444491847 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f202001db50 con 0x7f20280b4440 2023-10-10T20:34:28.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.404+0000 7f201f7fe700 1 --2- 172.21.15.89:0/444491847 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2008066ea0 0x7f2008069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:28.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.404+0000 7f201f7fe700 1 -- 172.21.15.89:0/444491847 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f2020086380 con 0x7f20280b4440 2023-10-10T20:34:28.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.404+0000 7f20277fe700 1 --2- 172.21.15.89:0/444491847 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2008066ea0 0x7f2008069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:28.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.404+0000 7f20277fe700 1 --2- 172.21.15.89:0/444491847 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2008066ea0 0x7f2008069350 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f20280b4380 tx=0x7f20180091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:28.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.408+0000 7f201f7fe700 1 -- 172.21.15.89:0/444491847 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2020053ba0 con 0x7f20280b4440 2023-10-10T20:34:28.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.633+0000 7f202dc0c700 1 -- 172.21.15.89:0/444491847 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f202804c3c0 con 0x7f20280b4440 2023-10-10T20:34:28.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.633+0000 7f201f7fe700 1 -- 172.21.15.89:0/444491847 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1259 (secure 0 0 0) 0x7f20200571c0 con 0x7f20280b4440 2023-10-10T20:34:28.636 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:28.636 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2341577144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2341577144},{"type":"v1","addr":"172.21.15.111:6835","nonce":2341577144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-10T20:34:28.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.635+0000 7f202dc0c700 1 -- 172.21.15.89:0/444491847 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2008066ea0 msgr2=0x7f2008069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:28.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.635+0000 7f202dc0c700 1 --2- 172.21.15.89:0/444491847 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2008066ea0 0x7f2008069350 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f20280b4380 tx=0x7f20180091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.635+0000 7f202dc0c700 1 -- 172.21.15.89:0/444491847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20280b4440 msgr2=0x7f20280b48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:28.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.635+0000 7f202dc0c700 1 --2- 172.21.15.89:0/444491847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20280b4440 0x7f20280b48b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f20280d03b0 tx=0x7f2020016000 comp rx=0 tx=0).stop 2023-10-10T20:34:28.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.635+0000 7f202dc0c700 1 -- 172.21.15.89:0/444491847 shutdown_connections 2023-10-10T20:34:28.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.635+0000 7f202dc0c700 1 --2- 172.21.15.89:0/444491847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20280b4440 0x7f20280b48b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.635+0000 7f202dc0c700 1 --2- 172.21.15.89:0/444491847 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2008066ea0 0x7f2008069350 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.635+0000 7f202dc0c700 1 --2- 172.21.15.89:0/444491847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20140051c0 0x7f20281493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.635+0000 7f202dc0c700 1 --2- 172.21.15.89:0/444491847 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f20280b1b30 0x7f20280b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:28.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.635+0000 7f202dc0c700 1 -- 172.21.15.89:0/444491847 >> 172.21.15.89:0/444491847 conn(0x7f2028068e20 msgr2=0x7f202815d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:28.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.635+0000 7f202dc0c700 1 -- 172.21.15.89:0/444491847 shutdown_connections 2023-10-10T20:34:28.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:28.635+0000 7f202dc0c700 1 -- 172.21.15.89:0/444491847 wait complete. 2023-10-10T20:34:28.640 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 2 2023-10-10T20:34:30.650 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:34:30.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.763+0000 7f8096eb8700 1 Processor -- start 2023-10-10T20:34:30.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.763+0000 7f8096eb8700 1 -- start start 2023-10-10T20:34:30.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.764+0000 7f8096eb8700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8090159890 0x7f809015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:30.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.764+0000 7f8096eb8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8090159c60 0x7f809015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:30.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.764+0000 7f8096eb8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f809015db10 0x7f8090069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:30.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.764+0000 7f8096eb8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f809015a160 con 0x7f809015db10 2023-10-10T20:34:30.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.764+0000 7f8096eb8700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80900d13a0 con 0x7f8090159890 2023-10-10T20:34:30.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.764+0000 7f8096eb8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809015e140 con 0x7f8090159c60 2023-10-10T20:34:30.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.764+0000 7f8095455700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f809015db10 0x7f8090069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:30.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.764+0000 7f8095455700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f809015db10 0x7f8090069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54200/0 (socket says 172.21.15.89:54200) 2023-10-10T20:34:30.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.764+0000 7f8095455700 1 -- 172.21.15.89:0/796935916 learned_addr learned my addr 172.21.15.89:0/796935916 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:30.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.764+0000 7f8087fff700 1 --2- 172.21.15.89:0/796935916 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8090159c60 0x7f809015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:30.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.764+0000 7f8094c54700 1 --2- 172.21.15.89:0/796935916 >> v2:172.21.15.111:3300/0 conn(0x7f8090159890 0x7f809015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:30.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.764+0000 7f8095455700 1 -- 172.21.15.89:0/796935916 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8090159c60 msgr2=0x7f809015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:30.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.764+0000 7f8095455700 1 --2- 172.21.15.89:0/796935916 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8090159c60 0x7f809015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:30.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.765+0000 7f8095455700 1 -- 172.21.15.89:0/796935916 >> v2:172.21.15.111:3300/0 conn(0x7f8090159890 msgr2=0x7f809015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:30.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.765+0000 7f8095455700 1 --2- 172.21.15.89:0/796935916 >> v2:172.21.15.111:3300/0 conn(0x7f8090159890 0x7f809015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:30.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.765+0000 7f8095455700 1 -- 172.21.15.89:0/796935916 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809015a3d0 con 0x7f809015db10 2023-10-10T20:34:30.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.765+0000 7f8095455700 1 --2- 172.21.15.89:0/796935916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f809015db10 0x7f8090069560 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f80900d1090 tx=0x7f8088009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff746ad46a557718 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:30.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.765+0000 7f80877fe700 1 -- 172.21.15.89:0/796935916 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808800b640 con 0x7f809015db10 2023-10-10T20:34:30.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.765+0000 7f80877fe700 1 -- 172.21.15.89:0/796935916 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808800b7a0 con 0x7f809015db10 2023-10-10T20:34:30.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.765+0000 7f80877fe700 1 -- 172.21.15.89:0/796935916 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808800bb60 con 0x7f809015db10 2023-10-10T20:34:30.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.765+0000 7f8096eb8700 1 -- 172.21.15.89:0/796935916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f809015db10 msgr2=0x7f8090069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:30.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.765+0000 7f8096eb8700 1 --2- 172.21.15.89:0/796935916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f809015db10 0x7f8090069560 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f80900d1090 tx=0x7f8088009250 comp rx=0 tx=0).stop 2023-10-10T20:34:30.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.765+0000 7f8096eb8700 1 -- 172.21.15.89:0/796935916 shutdown_connections 2023-10-10T20:34:30.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.765+0000 7f8096eb8700 1 --2- 172.21.15.89:0/796935916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f809015db10 0x7f8090069560 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:30.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.765+0000 7f8096eb8700 1 --2- 172.21.15.89:0/796935916 >> v2:172.21.15.111:3300/0 conn(0x7f8090159890 0x7f809015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:30.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.765+0000 7f8096eb8700 1 --2- 172.21.15.89:0/796935916 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8090159c60 0x7f809015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:30.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.765+0000 7f8096eb8700 1 -- 172.21.15.89:0/796935916 >> 172.21.15.89:0/796935916 conn(0x7f8090068e20 msgr2=0x7f80900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:30.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.765+0000 7f8096eb8700 1 -- 172.21.15.89:0/796935916 shutdown_connections 2023-10-10T20:34:30.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.765+0000 7f8096eb8700 1 -- 172.21.15.89:0/796935916 wait complete. 2023-10-10T20:34:30.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.766+0000 7f8096eb8700 1 Processor -- start 2023-10-10T20:34:30.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.767+0000 7f8096eb8700 1 -- start start 2023-10-10T20:34:30.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.767+0000 7f8096eb8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8090159890 0x7f80900b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:30.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.767+0000 7f8096eb8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8090159c60 0x7f80900cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:30.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.767+0000 7f8096eb8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f809015db10 0x7f80900cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:30.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.767+0000 7f8096eb8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8090153e70 con 0x7f809015db10 2023-10-10T20:34:30.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.767+0000 7f8096eb8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8090153860 con 0x7f8090159890 2023-10-10T20:34:30.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.767+0000 7f8096eb8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80900cfbc0 con 0x7f8090159c60 2023-10-10T20:34:30.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.767+0000 7f8095455700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f809015db10 0x7f80900cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:30.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.767+0000 7f8095455700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f809015db10 0x7f80900cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54212/0 (socket says 172.21.15.89:54212) 2023-10-10T20:34:30.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.767+0000 7f8095455700 1 -- 172.21.15.89:0/4293892325 learned_addr learned my addr 172.21.15.89:0/4293892325 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:30.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.767+0000 7f8087fff700 1 --2- 172.21.15.89:0/4293892325 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8090159c60 0x7f80900cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:30.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.767+0000 7f8094c54700 1 --2- 172.21.15.89:0/4293892325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8090159890 0x7f80900b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:30.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.768+0000 7f8095455700 1 -- 172.21.15.89:0/4293892325 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8090159c60 msgr2=0x7f80900cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:30.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.768+0000 7f8095455700 1 --2- 172.21.15.89:0/4293892325 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8090159c60 0x7f80900cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:30.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.768+0000 7f8095455700 1 -- 172.21.15.89:0/4293892325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8090159890 msgr2=0x7f80900b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:30.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.768+0000 7f8095455700 1 --2- 172.21.15.89:0/4293892325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8090159890 0x7f80900b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:30.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.768+0000 7f8095455700 1 -- 172.21.15.89:0/4293892325 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80880094f0 con 0x7f809015db10 2023-10-10T20:34:30.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.768+0000 7f8095455700 1 --2- 172.21.15.89:0/4293892325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f809015db10 0x7f80900cade0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f80880094c0 tx=0x7f8088017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:30.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.768+0000 7f8084ff9700 1 -- 172.21.15.89:0/4293892325 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808800bb20 con 0x7f809015db10 2023-10-10T20:34:30.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.768+0000 7f8084ff9700 1 -- 172.21.15.89:0/4293892325 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8088011420 con 0x7f809015db10 2023-10-10T20:34:30.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.768+0000 7f8084ff9700 1 -- 172.21.15.89:0/4293892325 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80880117e0 con 0x7f809015db10 2023-10-10T20:34:30.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.768+0000 7f8096eb8700 1 -- 172.21.15.89:0/4293892325 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8090153b00 con 0x7f809015db10 2023-10-10T20:34:30.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.768+0000 7f8096eb8700 1 -- 172.21.15.89:0/4293892325 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8090151010 con 0x7f809015db10 2023-10-10T20:34:30.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.769+0000 7f8096eb8700 1 -- 172.21.15.89:0/4293892325 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f809004c3c0 con 0x7f809015db10 2023-10-10T20:34:30.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.769+0000 7f8084ff9700 1 -- 172.21.15.89:0/4293892325 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f8088011b50 con 0x7f809015db10 2023-10-10T20:34:30.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.770+0000 7f8084ff9700 1 --2- 172.21.15.89:0/4293892325 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f807c066ea0 0x7f807c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:30.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.770+0000 7f8084ff9700 1 -- 172.21.15.89:0/4293892325 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f80880858a0 con 0x7f809015db10 2023-10-10T20:34:30.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.770+0000 7f8094c54700 1 --2- 172.21.15.89:0/4293892325 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f807c066ea0 0x7f807c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:30.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.770+0000 7f8094c54700 1 --2- 172.21.15.89:0/4293892325 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f807c066ea0 0x7f807c069350 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f808c009fd0 tx=0x7f808c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:30.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:30.774+0000 7f8084ff9700 1 -- 172.21.15.89:0/4293892325 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8088052fc0 con 0x7f809015db10 2023-10-10T20:34:31.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.007+0000 7f8096eb8700 1 -- 172.21.15.89:0/4293892325 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8090153fb0 con 0x7f809015db10 2023-10-10T20:34:31.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.007+0000 7f8084ff9700 1 -- 172.21.15.89:0/4293892325 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7f8088015de0 con 0x7f809015db10 2023-10-10T20:34:31.010 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:31.010 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2341577144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2341577144},{"type":"v1","addr":"172.21.15.111:6835","nonce":2341577144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2773123076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2773123076},{"type":"v1","addr":"172.21.15.89:6837","nonce":2773123076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2940044414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2940044414},{"type":"v1","addr":"172.21.15.89:6835","nonce":2940044414}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1112081320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1112081320},{"type":"v1","addr":"172.21.15.111:6834","nonce":1112081320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-10T20:34:31.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.009+0000 7f8096eb8700 1 -- 172.21.15.89:0/4293892325 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f807c066ea0 msgr2=0x7f807c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:31.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.009+0000 7f8096eb8700 1 --2- 172.21.15.89:0/4293892325 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f807c066ea0 0x7f807c069350 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f808c009fd0 tx=0x7f808c00b040 comp rx=0 tx=0).stop 2023-10-10T20:34:31.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.009+0000 7f8096eb8700 1 -- 172.21.15.89:0/4293892325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f809015db10 msgr2=0x7f80900cade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:31.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.009+0000 7f8096eb8700 1 --2- 172.21.15.89:0/4293892325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f809015db10 0x7f80900cade0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f80880094c0 tx=0x7f8088017000 comp rx=0 tx=0).stop 2023-10-10T20:34:31.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.009+0000 7f8096eb8700 1 -- 172.21.15.89:0/4293892325 shutdown_connections 2023-10-10T20:34:31.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.009+0000 7f8096eb8700 1 --2- 172.21.15.89:0/4293892325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f809015db10 0x7f80900cade0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.009+0000 7f8096eb8700 1 --2- 172.21.15.89:0/4293892325 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f807c066ea0 0x7f807c069350 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.009+0000 7f8096eb8700 1 --2- 172.21.15.89:0/4293892325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8090159890 0x7f80900b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.009+0000 7f8096eb8700 1 --2- 172.21.15.89:0/4293892325 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8090159c60 0x7f80900cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.009+0000 7f8096eb8700 1 -- 172.21.15.89:0/4293892325 >> 172.21.15.89:0/4293892325 conn(0x7f8090068e20 msgr2=0x7f8090152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:31.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.009+0000 7f8096eb8700 1 -- 172.21.15.89:0/4293892325 shutdown_connections 2023-10-10T20:34:31.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.009+0000 7f8096eb8700 1 -- 172.21.15.89:0/4293892325 wait complete. 2023-10-10T20:34:31.014 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 3 2023-10-10T20:34:31.023 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:34:31.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.138+0000 7fc889503700 1 Processor -- start 2023-10-10T20:34:31.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.138+0000 7fc889503700 1 -- start start 2023-10-10T20:34:31.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.139+0000 7fc889503700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88406c0f0 0x7fc8841620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.139+0000 7fc889503700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8840b16a0 con 0x7fc8840cc3c0 2023-10-10T20:34:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.139+0000 7fc889503700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8840b1c60 con 0x7fc8841574d0 2023-10-10T20:34:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.139+0000 7fc889503700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8840cd980 con 0x7fc88406c0f0 2023-10-10T20:34:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.139+0000 7fc8837fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc8840cc3c0 0x7fc88406b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48956/0 (socket says 172.21.15.89:48956) 2023-10-10T20:34:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.139+0000 7fc8837fe700 1 -- 172.21.15.89:0/2468279132 learned_addr learned my addr 172.21.15.89:0/2468279132 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.139+0000 7fc882ffd700 1 -- 172.21.15.89:0/2468279132 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4120003999 0 0) 0x7fc8840b16a0 con 0x7fc8840cc3c0 2023-10-10T20:34:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.139+0000 7fc882ffd700 1 -- 172.21.15.89:0/2468279132 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc86c003570 con 0x7fc8840cc3c0 2023-10-10T20:34:31.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.139+0000 7fc888d02700 1 --2- 172.21.15.89:0/2468279132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88406c0f0 0x7fc8841620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc882ffd700 1 -- 172.21.15.89:0/2468279132 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1493625371 0 0) 0x7fc86c003570 con 0x7fc8840cc3c0 2023-10-10T20:34:31.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc882ffd700 1 -- 172.21.15.89:0/2468279132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88406c0f0 msgr2=0x7fc8841620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:31.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc882ffd700 1 --2- 172.21.15.89:0/2468279132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88406c0f0 0x7fc8841620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc882ffd700 1 -- 172.21.15.89:0/2468279132 >> v1:172.21.15.111:6789/0 conn(0x7fc8841574d0 legacy=0x7fc8840cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:31.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc882ffd700 1 -- 172.21.15.89:0/2468279132 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8841578a0 con 0x7fc8840cc3c0 2023-10-10T20:34:31.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc888d02700 1 --2- 172.21.15.89:0/2468279132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88406c0f0 0x7fc8841620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:31.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc882ffd700 1 -- 172.21.15.89:0/2468279132 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fc878001470 con 0x7fc8840cc3c0 2023-10-10T20:34:31.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc882ffd700 1 -- 172.21.15.89:0/2468279132 >> v1:172.21.15.89:6789/0 conn(0x7fc8840cc3c0 legacy=0x7fc88406b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:31.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc882ffd700 1 --2- 172.21.15.89:0/2468279132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88406c0f0 0x7fc86c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc882ffd700 1 --2- 172.21.15.89:0/2468279132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc86c004080 0x7fc86c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc882ffd700 1 --2- 172.21.15.89:0/2468279132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc86c004d50 0x7fc86c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc882ffd700 1 -- 172.21.15.89:0/2468279132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c00bc70 con 0x7fc86c004d50 2023-10-10T20:34:31.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc882ffd700 1 -- 172.21.15.89:0/2468279132 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c00bde0 con 0x7fc86c004080 2023-10-10T20:34:31.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc882ffd700 1 -- 172.21.15.89:0/2468279132 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c00bfa0 con 0x7fc88406c0f0 2023-10-10T20:34:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc8837fe700 1 --2- 172.21.15.89:0/2468279132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc86c004080 0x7fc86c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc888d02700 1 --2- 172.21.15.89:0/2468279132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc86c004d50 0x7fc86c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc883fff700 1 --2- 172.21.15.89:0/2468279132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88406c0f0 0x7fc86c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc888d02700 1 -- 172.21.15.89:0/2468279132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88406c0f0 msgr2=0x7fc86c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc888d02700 1 --2- 172.21.15.89:0/2468279132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88406c0f0 0x7fc86c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc888d02700 1 -- 172.21.15.89:0/2468279132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc86c004080 msgr2=0x7fc86c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc888d02700 1 --2- 172.21.15.89:0/2468279132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc86c004080 0x7fc86c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc888d02700 1 -- 172.21.15.89:0/2468279132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c003570 con 0x7fc86c004d50 2023-10-10T20:34:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc888d02700 1 --2- 172.21.15.89:0/2468279132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc86c004d50 0x7fc86c0051c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc88406c4f0 tx=0x7fc87c0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68a926a0cfe4f41 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc882ffd700 1 -- 172.21.15.89:0/2468279132 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c009ae0 con 0x7fc86c004d50 2023-10-10T20:34:31.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc882ffd700 1 -- 172.21.15.89:0/2468279132 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87c007d70 con 0x7fc86c004d50 2023-10-10T20:34:31.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.140+0000 7fc882ffd700 1 -- 172.21.15.89:0/2468279132 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c014630 con 0x7fc86c004d50 2023-10-10T20:34:31.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.141+0000 7fc889503700 1 -- 172.21.15.89:0/2468279132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc86c004d50 msgr2=0x7fc86c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:31.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.141+0000 7fc889503700 1 --2- 172.21.15.89:0/2468279132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc86c004d50 0x7fc86c0051c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc88406c4f0 tx=0x7fc87c0097d0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.141+0000 7fc888d02700 1 -- 172.21.15.89:0/2468279132 reap_dead start 2023-10-10T20:34:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.141+0000 7fc889503700 1 -- 172.21.15.89:0/2468279132 shutdown_connections 2023-10-10T20:34:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.141+0000 7fc889503700 1 -- 172.21.15.89:0/2468279132 >> 172.21.15.89:0/2468279132 conn(0x7fc8840ce430 msgr2=0x7fc8840ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.141+0000 7fc889503700 1 -- 172.21.15.89:0/2468279132 shutdown_connections 2023-10-10T20:34:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.141+0000 7fc889503700 1 -- 172.21.15.89:0/2468279132 wait complete. 2023-10-10T20:34:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.142+0000 7fc889503700 1 Processor -- start 2023-10-10T20:34:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.142+0000 7fc889503700 1 -- start start 2023-10-10T20:34:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.143+0000 7fc889503700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc884152c50 0x7fc8840afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.143+0000 7fc889503700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc884153020 0x7fc8840adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.143+0000 7fc889503700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc884167660 0x7fc884154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.143+0000 7fc889503700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8840ae230 con 0x7fc884153020 2023-10-10T20:34:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.143+0000 7fc889503700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8840c3300 con 0x7fc884167660 2023-10-10T20:34:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.143+0000 7fc889503700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8840ae440 con 0x7fc884152c50 2023-10-10T20:34:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.143+0000 7fc8837fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc884153020 0x7fc8840adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.143+0000 7fc883fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc884152c50 0x7fc8840afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.143+0000 7fc883fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc884152c50 0x7fc8840afcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:42510/0 (socket says 172.21.15.89:42510) 2023-10-10T20:34:31.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.143+0000 7fc888d02700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc884167660 0x7fc884154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.143+0000 7fc883fff700 1 -- 172.21.15.89:0/3200778010 learned_addr learned my addr 172.21.15.89:0/3200778010 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:31.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.143+0000 7fc8837fe700 1 -- 172.21.15.89:0/3200778010 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc884152c50 msgr2=0x7fc8840afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:31.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.143+0000 7fc8837fe700 1 --2- 172.21.15.89:0/3200778010 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc884152c50 0x7fc8840afcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.143+0000 7fc8837fe700 1 -- 172.21.15.89:0/3200778010 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc884167660 msgr2=0x7fc884154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:31.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.143+0000 7fc8837fe700 1 --2- 172.21.15.89:0/3200778010 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc884167660 0x7fc884154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.143+0000 7fc8837fe700 1 -- 172.21.15.89:0/3200778010 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8840ae720 con 0x7fc884153020 2023-10-10T20:34:31.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.143+0000 7fc8837fe700 1 --2- 172.21.15.89:0/3200778010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc884153020 0x7fc8840adcf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc878007390 tx=0x7fc878001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.144+0000 7fc86bfff700 1 -- 172.21.15.89:0/3200778010 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878018030 con 0x7fc884153020 2023-10-10T20:34:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.144+0000 7fc86bfff700 1 -- 172.21.15.89:0/3200778010 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc878012560 con 0x7fc884153020 2023-10-10T20:34:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.144+0000 7fc889503700 1 -- 172.21.15.89:0/3200778010 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8840aea00 con 0x7fc884153020 2023-10-10T20:34:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.144+0000 7fc86bfff700 1 -- 172.21.15.89:0/3200778010 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878012900 con 0x7fc884153020 2023-10-10T20:34:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.144+0000 7fc889503700 1 -- 172.21.15.89:0/3200778010 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc884167ae0 con 0x7fc884153020 2023-10-10T20:34:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.145+0000 7fc889503700 1 -- 172.21.15.89:0/3200778010 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8840c3470 con 0x7fc884153020 2023-10-10T20:34:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.145+0000 7fc86bfff700 1 -- 172.21.15.89:0/3200778010 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fc87801b070 con 0x7fc884153020 2023-10-10T20:34:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.145+0000 7fc86bfff700 1 --2- 172.21.15.89:0/3200778010 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc870066ea0 0x7fc870069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.145+0000 7fc86bfff700 1 -- 172.21.15.89:0/3200778010 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc878019070 con 0x7fc884153020 2023-10-10T20:34:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.145+0000 7fc883fff700 1 --2- 172.21.15.89:0/3200778010 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc870066ea0 0x7fc870069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.146+0000 7fc883fff700 1 --2- 172.21.15.89:0/3200778010 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc870066ea0 0x7fc870069350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc874007e90 tx=0x7fc874007d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:31.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.150+0000 7fc86bfff700 1 -- 172.21.15.89:0/3200778010 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8780514e0 con 0x7fc884153020 2023-10-10T20:34:31.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.338+0000 7fc889503700 1 -- 172.21.15.89:0/3200778010 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc88404c3c0 con 0x7fc884153020 2023-10-10T20:34:31.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.339+0000 7fc86bfff700 1 -- 172.21.15.89:0/3200778010 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12127 (secure 0 0 0) 0x7fc878054b00 con 0x7fc884153020 2023-10-10T20:34:31.341 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:31.342 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":16,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:30.817980+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:34:31.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.341+0000 7fc889503700 1 -- 172.21.15.89:0/3200778010 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc870066ea0 msgr2=0x7fc870069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:31.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.341+0000 7fc889503700 1 --2- 172.21.15.89:0/3200778010 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc870066ea0 0x7fc870069350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc874007e90 tx=0x7fc874007d80 comp rx=0 tx=0).stop 2023-10-10T20:34:31.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.341+0000 7fc889503700 1 -- 172.21.15.89:0/3200778010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc884153020 msgr2=0x7fc8840adcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:31.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.341+0000 7fc889503700 1 --2- 172.21.15.89:0/3200778010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc884153020 0x7fc8840adcf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc878007390 tx=0x7fc878001e50 comp rx=0 tx=0).stop 2023-10-10T20:34:31.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.341+0000 7fc889503700 1 -- 172.21.15.89:0/3200778010 shutdown_connections 2023-10-10T20:34:31.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.341+0000 7fc889503700 1 --2- 172.21.15.89:0/3200778010 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc870066ea0 0x7fc870069350 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.341+0000 7fc889503700 1 --2- 172.21.15.89:0/3200778010 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc884167660 0x7fc884154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.341+0000 7fc889503700 1 --2- 172.21.15.89:0/3200778010 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc884152c50 0x7fc8840afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.341+0000 7fc889503700 1 --2- 172.21.15.89:0/3200778010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc884153020 0x7fc8840adcf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.341+0000 7fc889503700 1 -- 172.21.15.89:0/3200778010 >> 172.21.15.89:0/3200778010 conn(0x7fc8840ce430 msgr2=0x7fc8840b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:31.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.341+0000 7fc889503700 1 -- 172.21.15.89:0/3200778010 shutdown_connections 2023-10-10T20:34:31.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.341+0000 7fc889503700 1 -- 172.21.15.89:0/3200778010 wait complete. 2023-10-10T20:34:31.355 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:34:31.356 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:34:31.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.471+0000 7f5fa1c2e700 1 Processor -- start 2023-10-10T20:34:31.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.471+0000 7f5fa1c2e700 1 -- start start 2023-10-10T20:34:31.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.472+0000 7f5fa1c2e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5f9c06c0a0 0x7f5f9c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.472+0000 7f5fa1c2e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5f9c15df50 0x7f5f9c15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.472+0000 7f5fa1c2e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f9c15d7e0 con 0x7f5f9c06bcd0 2023-10-10T20:34:31.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.472+0000 7f5fa1c2e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f9c15dac0 con 0x7f5f9c06c0a0 2023-10-10T20:34:31.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.472+0000 7f5fa1c2e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f9c15aa30 con 0x7f5f9c15df50 2023-10-10T20:34:31.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.472+0000 7f5f9affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5f9c06c0a0 0x7f5f9c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.472+0000 7f5f9affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5f9c06c0a0 0x7f5f9c0caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54252/0 (socket says 172.21.15.89:54252) 2023-10-10T20:34:31.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.472+0000 7f5f9affd700 1 -- 172.21.15.89:0/3070107468 learned_addr learned my addr 172.21.15.89:0/3070107468 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:31.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.473+0000 7f5f9bfff700 1 --2- 172.21.15.89:0/3070107468 >> v2:172.21.15.111:3300/0 conn(0x7f5f9c15df50 0x7f5f9c15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.473+0000 7f5f9affd700 1 -- 172.21.15.89:0/3070107468 >> v2:172.21.15.111:3300/0 conn(0x7f5f9c15df50 msgr2=0x7f5f9c15e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:31.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.473+0000 7f5f9affd700 1 --2- 172.21.15.89:0/3070107468 >> v2:172.21.15.111:3300/0 conn(0x7f5f9c15df50 0x7f5f9c15e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.473+0000 7f5f9affd700 1 -- 172.21.15.89:0/3070107468 >> v1:172.21.15.111:6789/0 conn(0x7f5f9c06bcd0 legacy=0x7f5f9c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:31.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.473+0000 7f5f9affd700 1 -- 172.21.15.89:0/3070107468 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c15ad10 con 0x7f5f9c06c0a0 2023-10-10T20:34:31.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.473+0000 7f5f9affd700 1 --2- 172.21.15.89:0/3070107468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f9c06c0a0 0x7f5f9c0caa90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c15d720 tx=0x7f5f90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce49e27bf0444fef server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:31.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.473+0000 7f5f9a7fc700 1 -- 172.21.15.89:0/3070107468 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f90009e70 con 0x7f5f9c06c0a0 2023-10-10T20:34:31.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.473+0000 7f5f9a7fc700 1 -- 172.21.15.89:0/3070107468 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9000b860 con 0x7f5f9c06c0a0 2023-10-10T20:34:31.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.473+0000 7f5f9a7fc700 1 -- 172.21.15.89:0/3070107468 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9000bc20 con 0x7f5f9c06c0a0 2023-10-10T20:34:31.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.473+0000 7f5fa1c2e700 1 -- 172.21.15.89:0/3070107468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f9c06c0a0 msgr2=0x7f5f9c0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:31.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.473+0000 7f5fa1c2e700 1 --2- 172.21.15.89:0/3070107468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f9c06c0a0 0x7f5f9c0caa90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c15d720 tx=0x7f5f90009250 comp rx=0 tx=0).stop 2023-10-10T20:34:31.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.473+0000 7f5fa1c2e700 1 -- 172.21.15.89:0/3070107468 shutdown_connections 2023-10-10T20:34:31.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.473+0000 7f5fa1c2e700 1 --2- 172.21.15.89:0/3070107468 >> v2:172.21.15.111:3300/0 conn(0x7f5f9c15df50 0x7f5f9c15e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.473+0000 7f5fa1c2e700 1 --2- 172.21.15.89:0/3070107468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f9c06c0a0 0x7f5f9c0caa90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.473+0000 7f5fa1c2e700 1 -- 172.21.15.89:0/3070107468 >> 172.21.15.89:0/3070107468 conn(0x7f5f9c0ce600 msgr2=0x7f5f9c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:31.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.474+0000 7f5fa1c2e700 1 -- 172.21.15.89:0/3070107468 shutdown_connections 2023-10-10T20:34:31.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.474+0000 7f5fa1c2e700 1 -- 172.21.15.89:0/3070107468 wait complete. 2023-10-10T20:34:31.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.475+0000 7f5fa1c2e700 1 Processor -- start 2023-10-10T20:34:31.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.475+0000 7f5fa1c2e700 1 -- start start 2023-10-10T20:34:31.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5fa1c2e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f9c06bcd0 0x7f5f9c0c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5fa1c2e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f9c06c0a0 0x7f5f9c0b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5fa1c2e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f9c15df50 0x7f5f9c16aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5fa1c2e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c0c6140 con 0x7f5f9c15df50 2023-10-10T20:34:31.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5fa1c2e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c0c8a40 con 0x7f5f9c06bcd0 2023-10-10T20:34:31.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5fa1c2e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c0c63e0 con 0x7f5f9c06c0a0 2023-10-10T20:34:31.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5f9bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f9c15df50 0x7f5f9c16aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5f9bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f9c15df50 0x7f5f9c16aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54268/0 (socket says 172.21.15.89:54268) 2023-10-10T20:34:31.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5f9bfff700 1 -- 172.21.15.89:0/4194128240 learned_addr learned my addr 172.21.15.89:0/4194128240 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:31.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5f9b7fe700 1 --2- 172.21.15.89:0/4194128240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f9c06bcd0 0x7f5f9c0c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5f9affd700 1 --2- 172.21.15.89:0/4194128240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f9c06c0a0 0x7f5f9c0b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5f9bfff700 1 -- 172.21.15.89:0/4194128240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f9c06c0a0 msgr2=0x7f5f9c0b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:31.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5f9bfff700 1 --2- 172.21.15.89:0/4194128240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f9c06c0a0 0x7f5f9c0b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5f9bfff700 1 -- 172.21.15.89:0/4194128240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f9c06bcd0 msgr2=0x7f5f9c0c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:31.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5f9bfff700 1 --2- 172.21.15.89:0/4194128240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f9c06bcd0 0x7f5f9c0c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5f9bfff700 1 -- 172.21.15.89:0/4194128240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f900094f0 con 0x7f5f9c15df50 2023-10-10T20:34:31.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5f9bfff700 1 --2- 172.21.15.89:0/4194128240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f9c15df50 0x7f5f9c16aa80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400e8d0 tx=0x7f5f940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:31.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5f837fe700 1 -- 172.21.15.89:0/4194128240 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f94013680 con 0x7f5f9c15df50 2023-10-10T20:34:31.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5f837fe700 1 -- 172.21.15.89:0/4194128240 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f940137e0 con 0x7f5f9c15df50 2023-10-10T20:34:31.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5fa1c2e700 1 -- 172.21.15.89:0/4194128240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9c0c66c0 con 0x7f5f9c15df50 2023-10-10T20:34:31.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5f837fe700 1 -- 172.21.15.89:0/4194128240 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f94013ba0 con 0x7f5f9c15df50 2023-10-10T20:34:31.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.476+0000 7f5fa1c2e700 1 -- 172.21.15.89:0/4194128240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9c16a210 con 0x7f5f9c15df50 2023-10-10T20:34:31.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.477+0000 7f5fa1c2e700 1 -- 172.21.15.89:0/4194128240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9c04c3c0 con 0x7f5f9c15df50 2023-10-10T20:34:31.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.478+0000 7f5f837fe700 1 -- 172.21.15.89:0/4194128240 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f5f940123d0 con 0x7f5f9c15df50 2023-10-10T20:34:31.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.478+0000 7f5f837fe700 1 --2- 172.21.15.89:0/4194128240 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f88066ea0 0x7f5f88069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.478+0000 7f5f837fe700 1 -- 172.21.15.89:0/4194128240 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5f940841f0 con 0x7f5f9c15df50 2023-10-10T20:34:31.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.478+0000 7f5f9b7fe700 1 --2- 172.21.15.89:0/4194128240 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f88066ea0 0x7f5f88069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.478+0000 7f5f9b7fe700 1 --2- 172.21.15.89:0/4194128240 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f88066ea0 0x7f5f88069350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c007fd0 tx=0x7f5f8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:31.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.482+0000 7f5f837fe700 1 -- 172.21.15.89:0/4194128240 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f94051890 con 0x7f5f9c15df50 2023-10-10T20:34:31.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.645+0000 7f5f837fe700 1 -- 172.21.15.89:0/4194128240 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5f94051c60 con 0x7f5f9c15df50 2023-10-10T20:34:31.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.670+0000 7f5fa1c2e700 1 -- 172.21.15.89:0/4194128240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f9c15aff0 con 0x7f5f9c15df50 2023-10-10T20:34:31.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.670+0000 7f5f837fe700 1 -- 172.21.15.89:0/4194128240 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f94055320 con 0x7f5f9c15df50 2023-10-10T20:34:31.673 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:34:31.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.672+0000 7f5fa1c2e700 1 -- 172.21.15.89:0/4194128240 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f88066ea0 msgr2=0x7f5f88069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:31.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.672+0000 7f5fa1c2e700 1 --2- 172.21.15.89:0/4194128240 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f88066ea0 0x7f5f88069350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c007fd0 tx=0x7f5f8c009040 comp rx=0 tx=0).stop 2023-10-10T20:34:31.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.672+0000 7f5fa1c2e700 1 -- 172.21.15.89:0/4194128240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f9c15df50 msgr2=0x7f5f9c16aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:31.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.672+0000 7f5fa1c2e700 1 --2- 172.21.15.89:0/4194128240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f9c15df50 0x7f5f9c16aa80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400e8d0 tx=0x7f5f940091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.672+0000 7f5fa1c2e700 1 -- 172.21.15.89:0/4194128240 shutdown_connections 2023-10-10T20:34:31.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.672+0000 7f5fa1c2e700 1 --2- 172.21.15.89:0/4194128240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f9c15df50 0x7f5f9c16aa80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.672+0000 7f5fa1c2e700 1 --2- 172.21.15.89:0/4194128240 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f88066ea0 0x7f5f88069350 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.672+0000 7f5fa1c2e700 1 --2- 172.21.15.89:0/4194128240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f9c06bcd0 0x7f5f9c0c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.672+0000 7f5fa1c2e700 1 --2- 172.21.15.89:0/4194128240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f9c06c0a0 0x7f5f9c0b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.672+0000 7f5fa1c2e700 1 -- 172.21.15.89:0/4194128240 >> 172.21.15.89:0/4194128240 conn(0x7f5f9c0ce600 msgr2=0x7f5f9c0c33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:31.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.672+0000 7f5fa1c2e700 1 -- 172.21.15.89:0/4194128240 shutdown_connections 2023-10-10T20:34:31.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.672+0000 7f5fa1c2e700 1 -- 172.21.15.89:0/4194128240 wait complete. 2023-10-10T20:34:31.686 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:34:31.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.798+0000 7f1ac2228700 1 Processor -- start 2023-10-10T20:34:31.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.798+0000 7f1ac2228700 1 -- start start 2023-10-10T20:34:31.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.799+0000 7f1ac2228700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1abc159420 0x7f1abc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.799+0000 7f1ac2228700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1abc1597f0 0x7f1abc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.799+0000 7f1ac2228700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1abc0b11d0 con 0x7f1abc15d230 2023-10-10T20:34:31.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.799+0000 7f1ac2228700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1abc0b14b0 con 0x7f1abc159420 2023-10-10T20:34:31.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.799+0000 7f1ac2228700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1abc0b1790 con 0x7f1abc1597f0 2023-10-10T20:34:31.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.799+0000 7f1ac1226700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1abc159420 0x7f1abc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.799+0000 7f1ac1a27700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1abc15d230 0x7f1abc15d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48958/0 (socket says 172.21.15.89:48958) 2023-10-10T20:34:31.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.799+0000 7f1ac1a27700 1 -- 172.21.15.89:0/1808403809 learned_addr learned my addr 172.21.15.89:0/1808403809 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:31.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.799+0000 7f1ac0a25700 1 --2- 172.21.15.89:0/1808403809 >> v2:172.21.15.111:3300/0 conn(0x7f1abc1597f0 0x7f1abc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.799+0000 7f1ac1226700 1 -- 172.21.15.89:0/1808403809 >> v2:172.21.15.111:3300/0 conn(0x7f1abc1597f0 msgr2=0x7f1abc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:31.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.799+0000 7f1ac1226700 1 --2- 172.21.15.89:0/1808403809 >> v2:172.21.15.111:3300/0 conn(0x7f1abc1597f0 0x7f1abc0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.799+0000 7f1ac1226700 1 -- 172.21.15.89:0/1808403809 >> v1:172.21.15.89:6789/0 conn(0x7f1abc15d230 legacy=0x7f1abc15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:31.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.799+0000 7f1abbfff700 1 -- 172.21.15.89:0/1808403809 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4082879276 0 0) 0x7f1abc0b11d0 con 0x7f1abc15d230 2023-10-10T20:34:31.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.799+0000 7f1ac1226700 1 -- 172.21.15.89:0/1808403809 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1abc0b1a70 con 0x7f1abc159420 2023-10-10T20:34:31.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.799+0000 7f1ac1226700 1 --2- 172.21.15.89:0/1808403809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1abc159420 0x7f1abc0b5f40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1abc089750 tx=0x7f1ab4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4ebba8aece31f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:31.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.800+0000 7f1abbfff700 1 -- 172.21.15.89:0/1808403809 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab400e070 con 0x7f1abc159420 2023-10-10T20:34:31.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.800+0000 7f1abbfff700 1 -- 172.21.15.89:0/1808403809 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab400bb60 con 0x7f1abc159420 2023-10-10T20:34:31.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.800+0000 7f1abbfff700 1 -- 172.21.15.89:0/1808403809 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4012430 con 0x7f1abc159420 2023-10-10T20:34:31.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.800+0000 7f1ac2228700 1 -- 172.21.15.89:0/1808403809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1abc159420 msgr2=0x7f1abc0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:31.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.800+0000 7f1ac2228700 1 --2- 172.21.15.89:0/1808403809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1abc159420 0x7f1abc0b5f40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1abc089750 tx=0x7f1ab4009780 comp rx=0 tx=0).stop 2023-10-10T20:34:31.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.800+0000 7f1ac2228700 1 -- 172.21.15.89:0/1808403809 shutdown_connections 2023-10-10T20:34:31.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.800+0000 7f1ac2228700 1 --2- 172.21.15.89:0/1808403809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1abc159420 0x7f1abc0b5f40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.800+0000 7f1ac2228700 1 --2- 172.21.15.89:0/1808403809 >> v2:172.21.15.111:3300/0 conn(0x7f1abc1597f0 0x7f1abc0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.800+0000 7f1ac2228700 1 -- 172.21.15.89:0/1808403809 >> 172.21.15.89:0/1808403809 conn(0x7f1abc0d0660 msgr2=0x7f1abc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:31.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.800+0000 7f1ac2228700 1 -- 172.21.15.89:0/1808403809 shutdown_connections 2023-10-10T20:34:31.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.800+0000 7f1ac2228700 1 -- 172.21.15.89:0/1808403809 wait complete. 2023-10-10T20:34:31.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.801+0000 7f1ac2228700 1 Processor -- start 2023-10-10T20:34:31.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.801+0000 7f1ac2228700 1 -- start start 2023-10-10T20:34:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.802+0000 7f1ac2228700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1abc159420 0x7f1abc13f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.802+0000 7f1ac2228700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1abc1597f0 0x7f1abc158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.802+0000 7f1ac2228700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1abc15d230 0x7f1abc15b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.802+0000 7f1ac2228700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc15adb0 con 0x7f1abc1597f0 2023-10-10T20:34:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.802+0000 7f1ac2228700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc13f5b0 con 0x7f1abc15d230 2023-10-10T20:34:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.802+0000 7f1ac2228700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc15a1a0 con 0x7f1abc159420 2023-10-10T20:34:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.802+0000 7f1ac0a25700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1abc1597f0 0x7f1abc158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.802+0000 7f1ac0a25700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1abc1597f0 0x7f1abc158470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54280/0 (socket says 172.21.15.89:54280) 2023-10-10T20:34:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.802+0000 7f1ac0a25700 1 -- 172.21.15.89:0/385893833 learned_addr learned my addr 172.21.15.89:0/385893833 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.802+0000 7f1ac1a27700 1 --2- 172.21.15.89:0/385893833 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1abc15d230 0x7f1abc15b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.802+0000 7f1ac1226700 1 --2- 172.21.15.89:0/385893833 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1abc159420 0x7f1abc13f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.803+0000 7f1ac0a25700 1 -- 172.21.15.89:0/385893833 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1abc159420 msgr2=0x7f1abc13f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.803+0000 7f1ac0a25700 1 --2- 172.21.15.89:0/385893833 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1abc159420 0x7f1abc13f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.803+0000 7f1ac0a25700 1 -- 172.21.15.89:0/385893833 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1abc15d230 msgr2=0x7f1abc15b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.803+0000 7f1ac0a25700 1 --2- 172.21.15.89:0/385893833 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1abc15d230 0x7f1abc15b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:31.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.803+0000 7f1ac0a25700 1 -- 172.21.15.89:0/385893833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab4009a20 con 0x7f1abc1597f0 2023-10-10T20:34:31.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.803+0000 7f1ac1a27700 1 --2- 172.21.15.89:0/385893833 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1abc15d230 0x7f1abc15b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:31.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.803+0000 7f1ac1226700 1 --2- 172.21.15.89:0/385893833 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1abc159420 0x7f1abc13f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:31.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.803+0000 7f1ac0a25700 1 --2- 172.21.15.89:0/385893833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1abc1597f0 0x7f1abc158470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000ec50 tx=0x7f1ab00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:31.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.803+0000 7f1ab97fa700 1 -- 172.21.15.89:0/385893833 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab000c9d0 con 0x7f1abc1597f0 2023-10-10T20:34:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.803+0000 7f1ab97fa700 1 -- 172.21.15.89:0/385893833 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab000cb30 con 0x7f1abc1597f0 2023-10-10T20:34:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.803+0000 7f1ac2228700 1 -- 172.21.15.89:0/385893833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1abc15a480 con 0x7f1abc1597f0 2023-10-10T20:34:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.803+0000 7f1ab97fa700 1 -- 172.21.15.89:0/385893833 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0013470 con 0x7f1abc1597f0 2023-10-10T20:34:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.803+0000 7f1ac2228700 1 -- 172.21.15.89:0/385893833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1abc15aa40 con 0x7f1abc1597f0 2023-10-10T20:34:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.804+0000 7f1ac2228700 1 -- 172.21.15.89:0/385893833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1abc04c3c0 con 0x7f1abc1597f0 2023-10-10T20:34:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.804+0000 7f1ab97fa700 1 -- 172.21.15.89:0/385893833 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1ab0013860 con 0x7f1abc1597f0 2023-10-10T20:34:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.804+0000 7f1ab97fa700 1 --2- 172.21.15.89:0/385893833 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1aa8066f30 0x7f1aa80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.805+0000 7f1ab97fa700 1 -- 172.21.15.89:0/385893833 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1ab00845a0 con 0x7f1abc1597f0 2023-10-10T20:34:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.805+0000 7f1ac1226700 1 --2- 172.21.15.89:0/385893833 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1aa8066f30 0x7f1aa80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.805+0000 7f1ac1226700 1 --2- 172.21.15.89:0/385893833 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1aa8066f30 0x7f1aa80693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40097c0 tx=0x7f1ab4019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.809+0000 7f1ab97fa700 1 -- 172.21.15.89:0/385893833 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ab0051b60 con 0x7f1abc1597f0 2023-10-10T20:34:31.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.997+0000 7f1ac2228700 1 -- 172.21.15.89:0/385893833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1abc15aef0 con 0x7f1abc1597f0 2023-10-10T20:34:31.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.997+0000 7f1ab97fa700 1 -- 172.21.15.89:0/385893833 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1ab0055180 con 0x7f1abc1597f0 2023-10-10T20:34:32.000 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:32.000 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:34:32.000 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:34:32.000 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2023-10-10T20:34:32.001 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2023-10-10T20:34:32.001 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:34:32.001 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:34:32.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.999+0000 7f1ac2228700 1 -- 172.21.15.89:0/385893833 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1aa8066f30 msgr2=0x7f1aa80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:32.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.999+0000 7f1ac2228700 1 --2- 172.21.15.89:0/385893833 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1aa8066f30 0x7f1aa80693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40097c0 tx=0x7f1ab4019000 comp rx=0 tx=0).stop 2023-10-10T20:34:32.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.999+0000 7f1ac2228700 1 -- 172.21.15.89:0/385893833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1abc1597f0 msgr2=0x7f1abc158470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:32.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.999+0000 7f1ac2228700 1 --2- 172.21.15.89:0/385893833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1abc1597f0 0x7f1abc158470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000ec50 tx=0x7f1ab00091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:32.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.999+0000 7f1ac2228700 1 -- 172.21.15.89:0/385893833 shutdown_connections 2023-10-10T20:34:32.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.999+0000 7f1ac2228700 1 --2- 172.21.15.89:0/385893833 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1aa8066f30 0x7f1aa80693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:32.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.999+0000 7f1ac2228700 1 --2- 172.21.15.89:0/385893833 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1abc15d230 0x7f1abc15b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:32.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.999+0000 7f1ac2228700 1 --2- 172.21.15.89:0/385893833 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1abc159420 0x7f1abc13f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:32.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.999+0000 7f1ac2228700 1 --2- 172.21.15.89:0/385893833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1abc1597f0 0x7f1abc158470 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:32.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.999+0000 7f1ac2228700 1 -- 172.21.15.89:0/385893833 >> 172.21.15.89:0/385893833 conn(0x7f1abc0d0660 msgr2=0x7f1abc0ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:32.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.999+0000 7f1ac2228700 1 -- 172.21.15.89:0/385893833 shutdown_connections 2023-10-10T20:34:32.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:31.999+0000 7f1ac2228700 1 -- 172.21.15.89:0/385893833 wait complete. 2023-10-10T20:34:32.012 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:34:32.013 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:34:32.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.129+0000 7fdb4d8ed700 1 Processor -- start 2023-10-10T20:34:32.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.129+0000 7fdb4d8ed700 1 -- start start 2023-10-10T20:34:32.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.130+0000 7fdb4d8ed700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdb48157540 0x7fdb480c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:32.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.130+0000 7fdb4d8ed700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb480c9710 con 0x7fdb48157910 2023-10-10T20:34:32.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.130+0000 7fdb4d8ed700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb4815b560 con 0x7fdb480c92f0 2023-10-10T20:34:32.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.130+0000 7fdb4d8ed700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb480cbbe0 con 0x7fdb48157540 2023-10-10T20:34:32.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.130+0000 7fdb4c8eb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdb48157540 0x7fdb480c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:32.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.130+0000 7fdb3ffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdb48157910 0x7fdb480cadd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48960/0 (socket says 172.21.15.89:48960) 2023-10-10T20:34:32.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.130+0000 7fdb3ffff700 1 -- 172.21.15.89:0/3564830866 learned_addr learned my addr 172.21.15.89:0/3564830866 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:32.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.130+0000 7fdb4c8eb700 1 -- 172.21.15.89:0/3564830866 >> v1:172.21.15.111:6789/0 conn(0x7fdb480c92f0 legacy=0x7fdb480cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:32.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.130+0000 7fdb4c8eb700 1 -- 172.21.15.89:0/3564830866 >> v1:172.21.15.89:6789/0 conn(0x7fdb48157910 legacy=0x7fdb480cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:32.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.130+0000 7fdb4c8eb700 1 -- 172.21.15.89:0/3564830866 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4815b840 con 0x7fdb48157540 2023-10-10T20:34:32.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.130+0000 7fdb3f7fe700 1 -- 172.21.15.89:0/3564830866 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725113210 0 0) 0x7fdb480c9710 con 0x7fdb48157910 2023-10-10T20:34:32.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.131+0000 7fdb4c8eb700 1 --2- 172.21.15.89:0/3564830866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48157540 0x7fdb480c8db0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdb440098e0 tx=0x7fdb440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be6e2a176b6e7ed server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:32.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.131+0000 7fdb3f7fe700 1 -- 172.21.15.89:0/3564830866 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb44009e80 con 0x7fdb48157540 2023-10-10T20:34:32.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.131+0000 7fdb3f7fe700 1 -- 172.21.15.89:0/3564830866 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4400b860 con 0x7fdb48157540 2023-10-10T20:34:32.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.131+0000 7fdb3f7fe700 1 -- 172.21.15.89:0/3564830866 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4400bbe0 con 0x7fdb48157540 2023-10-10T20:34:32.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.131+0000 7fdb4d8ed700 1 -- 172.21.15.89:0/3564830866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48157540 msgr2=0x7fdb480c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:32.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.131+0000 7fdb4d8ed700 1 --2- 172.21.15.89:0/3564830866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48157540 0x7fdb480c8db0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdb440098e0 tx=0x7fdb440091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:32.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.131+0000 7fdb4d8ed700 1 -- 172.21.15.89:0/3564830866 shutdown_connections 2023-10-10T20:34:32.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.131+0000 7fdb4d8ed700 1 --2- 172.21.15.89:0/3564830866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48157540 0x7fdb480c8db0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:32.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.131+0000 7fdb4d8ed700 1 -- 172.21.15.89:0/3564830866 >> 172.21.15.89:0/3564830866 conn(0x7fdb4806baf0 msgr2=0x7fdb4806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:32.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.131+0000 7fdb4d8ed700 1 -- 172.21.15.89:0/3564830866 shutdown_connections 2023-10-10T20:34:32.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.131+0000 7fdb4d8ed700 1 -- 172.21.15.89:0/3564830866 wait complete. 2023-10-10T20:34:32.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.132+0000 7fdb4d8ed700 1 Processor -- start 2023-10-10T20:34:32.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.133+0000 7fdb4d8ed700 1 -- start start 2023-10-10T20:34:32.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.133+0000 7fdb4d8ed700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdb480c92f0 0x7fdb4814a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:32.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.133+0000 7fdb4d8ed700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb48157540 0x7fdb4814d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:32.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.133+0000 7fdb4d8ed700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48157910 0x7fdb4814d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:32.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.133+0000 7fdb4d8ed700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb48152ba0 con 0x7fdb48157910 2023-10-10T20:34:32.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.133+0000 7fdb4d8ed700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb48069a70 con 0x7fdb48157540 2023-10-10T20:34:32.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.133+0000 7fdb4d8ed700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4816aa40 con 0x7fdb480c92f0 2023-10-10T20:34:32.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.133+0000 7fdb4d0ec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48157910 0x7fdb4814d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:32.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.133+0000 7fdb4d0ec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48157910 0x7fdb4814d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54298/0 (socket says 172.21.15.89:54298) 2023-10-10T20:34:32.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.133+0000 7fdb4d0ec700 1 -- 172.21.15.89:0/1590532664 learned_addr learned my addr 172.21.15.89:0/1590532664 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.134+0000 7fdb4c8eb700 1 --2- 172.21.15.89:0/1590532664 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdb480c92f0 0x7fdb4814a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.134+0000 7fdb3ffff700 1 --2- 172.21.15.89:0/1590532664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb48157540 0x7fdb4814d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.134+0000 7fdb4d0ec700 1 -- 172.21.15.89:0/1590532664 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdb480c92f0 msgr2=0x7fdb4814a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.134+0000 7fdb4d0ec700 1 --2- 172.21.15.89:0/1590532664 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdb480c92f0 0x7fdb4814a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.134+0000 7fdb4d0ec700 1 -- 172.21.15.89:0/1590532664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb48157540 msgr2=0x7fdb4814d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.134+0000 7fdb4d0ec700 1 --2- 172.21.15.89:0/1590532664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb48157540 0x7fdb4814d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:32.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.134+0000 7fdb4d0ec700 1 -- 172.21.15.89:0/1590532664 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb44009420 con 0x7fdb48157910 2023-10-10T20:34:32.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.134+0000 7fdb4c8eb700 1 --2- 172.21.15.89:0/1590532664 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdb480c92f0 0x7fdb4814a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:32.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.134+0000 7fdb4d0ec700 1 --2- 172.21.15.89:0/1590532664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48157910 0x7fdb4814d780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000c8e0 tx=0x7fdb40001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:32.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.134+0000 7fdb3cff9700 1 -- 172.21.15.89:0/1590532664 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4000ce80 con 0x7fdb48157910 2023-10-10T20:34:32.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.134+0000 7fdb3cff9700 1 -- 172.21.15.89:0/1590532664 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4000a7e0 con 0x7fdb48157910 2023-10-10T20:34:32.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.134+0000 7fdb4d8ed700 1 -- 172.21.15.89:0/1590532664 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb48069d10 con 0x7fdb48157910 2023-10-10T20:34:32.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.134+0000 7fdb3cff9700 1 -- 172.21.15.89:0/1590532664 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4000ab80 con 0x7fdb48157910 2023-10-10T20:34:32.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.134+0000 7fdb4d8ed700 1 -- 172.21.15.89:0/1590532664 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4816ad20 con 0x7fdb48157910 2023-10-10T20:34:32.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.135+0000 7fdb4d8ed700 1 -- 172.21.15.89:0/1590532664 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb480cb6e0 con 0x7fdb48157910 2023-10-10T20:34:32.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.135+0000 7fdb3cff9700 1 -- 172.21.15.89:0/1590532664 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb40012420 con 0x7fdb48157910 2023-10-10T20:34:32.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.136+0000 7fdb3cff9700 1 --2- 172.21.15.89:0/1590532664 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdb34066f30 0x7fdb340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:32.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.136+0000 7fdb3cff9700 1 -- 172.21.15.89:0/1590532664 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fdb400844e0 con 0x7fdb48157910 2023-10-10T20:34:32.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.136+0000 7fdb4c8eb700 1 --2- 172.21.15.89:0/1590532664 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdb34066f30 0x7fdb340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:32.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.136+0000 7fdb4c8eb700 1 --2- 172.21.15.89:0/1590532664 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdb34066f30 0x7fdb340693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdb440093f0 tx=0x7fdb44017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:32.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.140+0000 7fdb3cff9700 1 -- 172.21.15.89:0/1590532664 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb40051aa0 con 0x7fdb48157910 2023-10-10T20:34:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:32.324+0000 7fdb4d8ed700 1 -- 172.21.15.89:0/1590532664 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdb4804c410 con 0x7fdb48157910 2023-10-10T20:34:33.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.316+0000 7fdb3cff9700 1 -- 172.21.15.89:0/1590532664 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdb40084030 con 0x7fdb48157910 2023-10-10T20:34:33.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.320+0000 7fdb4d8ed700 1 -- 172.21.15.89:0/1590532664 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdb34066f30 msgr2=0x7fdb340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:33.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.320+0000 7fdb4d8ed700 1 --2- 172.21.15.89:0/1590532664 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdb34066f30 0x7fdb340693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdb440093f0 tx=0x7fdb44017000 comp rx=0 tx=0).stop 2023-10-10T20:34:33.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.320+0000 7fdb4d8ed700 1 -- 172.21.15.89:0/1590532664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48157910 msgr2=0x7fdb4814d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:33.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.320+0000 7fdb4d8ed700 1 --2- 172.21.15.89:0/1590532664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48157910 0x7fdb4814d780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000c8e0 tx=0x7fdb40001710 comp rx=0 tx=0).stop 2023-10-10T20:34:33.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.320+0000 7fdb4d8ed700 1 -- 172.21.15.89:0/1590532664 shutdown_connections 2023-10-10T20:34:33.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.320+0000 7fdb4d8ed700 1 --2- 172.21.15.89:0/1590532664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48157910 0x7fdb4814d780 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:33.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.320+0000 7fdb4d8ed700 1 --2- 172.21.15.89:0/1590532664 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdb480c92f0 0x7fdb4814a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:33.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.320+0000 7fdb4d8ed700 1 --2- 172.21.15.89:0/1590532664 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdb34066f30 0x7fdb340693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:33.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.320+0000 7fdb4d8ed700 1 --2- 172.21.15.89:0/1590532664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb48157540 0x7fdb4814d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:33.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.320+0000 7fdb4d8ed700 1 -- 172.21.15.89:0/1590532664 >> 172.21.15.89:0/1590532664 conn(0x7fdb4806baf0 msgr2=0x7fdb4814b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:33.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.321+0000 7fdb4d8ed700 1 -- 172.21.15.89:0/1590532664 shutdown_connections 2023-10-10T20:34:33.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.321+0000 7fdb4d8ed700 1 -- 172.21.15.89:0/1590532664 wait complete. 2023-10-10T20:34:33.325 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2023-10-10T20:34:33.337 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:34:33.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.457+0000 7f2679faa700 1 Processor -- start 2023-10-10T20:34:33.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.457+0000 7f2679faa700 1 -- start start 2023-10-10T20:34:33.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.458+0000 7f2679faa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2674159950 0x7f26740cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:33.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.458+0000 7f2679faa700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26740af320 0x7f2674069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:33.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.458+0000 7f2679faa700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26740ce380 con 0x7f2674069aa0 2023-10-10T20:34:33.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.458+0000 7f2679faa700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2674069f20 con 0x7f2674159950 2023-10-10T20:34:33.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.458+0000 7f2679faa700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26740cec20 con 0x7f26740af320 2023-10-10T20:34:33.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.458+0000 7f26737fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2674159950 0x7f26740cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:33.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.458+0000 7f2673fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2674069aa0 0x7f267415d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48976/0 (socket says 172.21.15.89:48976) 2023-10-10T20:34:33.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.458+0000 7f2673fff700 1 -- 172.21.15.89:0/952696823 learned_addr learned my addr 172.21.15.89:0/952696823 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:33.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.458+0000 7f2672ffd700 1 --2- 172.21.15.89:0/952696823 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26740af320 0x7f2674069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:33.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.458+0000 7f26737fe700 1 -- 172.21.15.89:0/952696823 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26740af320 msgr2=0x7f2674069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:33.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.458+0000 7f26737fe700 1 --2- 172.21.15.89:0/952696823 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26740af320 0x7f2674069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:33.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.458+0000 7f26737fe700 1 -- 172.21.15.89:0/952696823 >> v1:172.21.15.89:6789/0 conn(0x7f2674069aa0 legacy=0x7f267415d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:33.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.458+0000 7f26727fc700 1 -- 172.21.15.89:0/952696823 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1137575171 0 0) 0x7f26740ce380 con 0x7f2674069aa0 2023-10-10T20:34:33.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.458+0000 7f26737fe700 1 -- 172.21.15.89:0/952696823 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26740afa90 con 0x7f2674159950 2023-10-10T20:34:33.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.459+0000 7f2672ffd700 1 --2- 172.21.15.89:0/952696823 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26740af320 0x7f2674069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:33.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.459+0000 7f26737fe700 1 --2- 172.21.15.89:0/952696823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2674159950 0x7f26740cbb90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2664009e60 tx=0x7f2664009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6256610fa0a4aa18 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:33.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.459+0000 7f26727fc700 1 -- 172.21.15.89:0/952696823 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266400e070 con 0x7f2674159950 2023-10-10T20:34:33.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.459+0000 7f26727fc700 1 -- 172.21.15.89:0/952696823 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266400bc30 con 0x7f2674159950 2023-10-10T20:34:33.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.459+0000 7f26727fc700 1 -- 172.21.15.89:0/952696823 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664012430 con 0x7f2674159950 2023-10-10T20:34:33.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.459+0000 7f2679faa700 1 -- 172.21.15.89:0/952696823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2674159950 msgr2=0x7f26740cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:33.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.459+0000 7f2679faa700 1 --2- 172.21.15.89:0/952696823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2674159950 0x7f26740cbb90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2664009e60 tx=0x7f2664009780 comp rx=0 tx=0).stop 2023-10-10T20:34:33.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.459+0000 7f2679faa700 1 -- 172.21.15.89:0/952696823 shutdown_connections 2023-10-10T20:34:33.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.459+0000 7f2679faa700 1 --2- 172.21.15.89:0/952696823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2674159950 0x7f26740cbb90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:33.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.459+0000 7f2679faa700 1 --2- 172.21.15.89:0/952696823 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26740af320 0x7f2674069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:33.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.459+0000 7f2679faa700 1 -- 172.21.15.89:0/952696823 >> 172.21.15.89:0/952696823 conn(0x7f2674068e00 msgr2=0x7f26740d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:33.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.459+0000 7f2679faa700 1 -- 172.21.15.89:0/952696823 shutdown_connections 2023-10-10T20:34:33.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.459+0000 7f2679faa700 1 -- 172.21.15.89:0/952696823 wait complete. 2023-10-10T20:34:33.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.460+0000 7f2679faa700 1 Processor -- start 2023-10-10T20:34:33.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.461+0000 7f2679faa700 1 -- start start 2023-10-10T20:34:33.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.461+0000 7f2679faa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2674069aa0 0x7f267415e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:33.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.461+0000 7f2679faa700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26740af320 0x7f26740c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:33.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.461+0000 7f2679faa700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2674159950 0x7f26740c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:33.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.461+0000 7f2679faa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267415def0 con 0x7f2674069aa0 2023-10-10T20:34:33.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.461+0000 7f2679faa700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26740b64a0 con 0x7f2674159950 2023-10-10T20:34:33.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.461+0000 7f2679faa700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26740b61c0 con 0x7f26740af320 2023-10-10T20:34:33.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.461+0000 7f26737fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2674069aa0 0x7f267415e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:33.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.461+0000 7f26737fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2674069aa0 0x7f267415e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54326/0 (socket says 172.21.15.89:54326) 2023-10-10T20:34:33.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.461+0000 7f26737fe700 1 -- 172.21.15.89:0/828690107 learned_addr learned my addr 172.21.15.89:0/828690107 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:33.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.462+0000 7f2672ffd700 1 --2- 172.21.15.89:0/828690107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26740af320 0x7f26740c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:33.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.462+0000 7f2673fff700 1 --2- 172.21.15.89:0/828690107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2674159950 0x7f26740c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:33.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.462+0000 7f26737fe700 1 -- 172.21.15.89:0/828690107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26740af320 msgr2=0x7f26740c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:33.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.462+0000 7f26737fe700 1 --2- 172.21.15.89:0/828690107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26740af320 0x7f26740c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:33.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.462+0000 7f26737fe700 1 -- 172.21.15.89:0/828690107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2674159950 msgr2=0x7f26740c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:33.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.462+0000 7f26737fe700 1 --2- 172.21.15.89:0/828690107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2674159950 0x7f26740c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:33.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.462+0000 7f26737fe700 1 -- 172.21.15.89:0/828690107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2664009a20 con 0x7f2674069aa0 2023-10-10T20:34:33.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.462+0000 7f2672ffd700 1 --2- 172.21.15.89:0/828690107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26740af320 0x7f26740c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:33.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.462+0000 7f2673fff700 1 --2- 172.21.15.89:0/828690107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2674159950 0x7f26740c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:33.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.462+0000 7f26737fe700 1 --2- 172.21.15.89:0/828690107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2674069aa0 0x7f267415e380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2674070e20 tx=0x7f2664019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:33.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.462+0000 7f265b7fe700 1 -- 172.21.15.89:0/828690107 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664012db0 con 0x7f2674069aa0 2023-10-10T20:34:33.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.462+0000 7f265b7fe700 1 -- 172.21.15.89:0/828690107 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2664027470 con 0x7f2674069aa0 2023-10-10T20:34:33.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.462+0000 7f2679faa700 1 -- 172.21.15.89:0/828690107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267415db80 con 0x7f2674069aa0 2023-10-10T20:34:33.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.462+0000 7f265b7fe700 1 -- 172.21.15.89:0/828690107 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664027830 con 0x7f2674069aa0 2023-10-10T20:34:33.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.462+0000 7f2679faa700 1 -- 172.21.15.89:0/828690107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26740ca430 con 0x7f2674069aa0 2023-10-10T20:34:33.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.463+0000 7f2679faa700 1 -- 172.21.15.89:0/828690107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267415e1d0 con 0x7f2674069aa0 2023-10-10T20:34:33.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.463+0000 7f265b7fe700 1 -- 172.21.15.89:0/828690107 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2664024070 con 0x7f2674069aa0 2023-10-10T20:34:33.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.464+0000 7f265b7fe700 1 --2- 172.21.15.89:0/828690107 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2660066f30 0x7f26600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:33.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.464+0000 7f265b7fe700 1 -- 172.21.15.89:0/828690107 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f26640868b0 con 0x7f2674069aa0 2023-10-10T20:34:33.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.464+0000 7f2672ffd700 1 --2- 172.21.15.89:0/828690107 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2660066f30 0x7f26600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:33.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.464+0000 7f2672ffd700 1 --2- 172.21.15.89:0/828690107 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2660066f30 0x7f26600693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2668006f00 tx=0x7f2668008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:33.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.468+0000 7f265b7fe700 1 -- 172.21.15.89:0/828690107 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2664053d00 con 0x7f2674069aa0 2023-10-10T20:34:33.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:33.654+0000 7f2679faa700 1 -- 172.21.15.89:0/828690107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f26740718b0 con 0x7f2674069aa0 2023-10-10T20:34:34.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.325+0000 7f265b7fe700 1 -- 172.21.15.89:0/828690107 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f266402c0d0 con 0x7f2674069aa0 2023-10-10T20:34:34.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.333+0000 7f2679faa700 1 -- 172.21.15.89:0/828690107 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2660066f30 msgr2=0x7f26600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:34.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.333+0000 7f2679faa700 1 --2- 172.21.15.89:0/828690107 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2660066f30 0x7f26600693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2668006f00 tx=0x7f2668008040 comp rx=0 tx=0).stop 2023-10-10T20:34:34.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.333+0000 7f2679faa700 1 -- 172.21.15.89:0/828690107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2674069aa0 msgr2=0x7f267415e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:34.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.333+0000 7f2679faa700 1 --2- 172.21.15.89:0/828690107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2674069aa0 0x7f267415e380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2674070e20 tx=0x7f2664019000 comp rx=0 tx=0).stop 2023-10-10T20:34:34.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.335+0000 7f2679faa700 1 -- 172.21.15.89:0/828690107 shutdown_connections 2023-10-10T20:34:34.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.335+0000 7f2679faa700 1 --2- 172.21.15.89:0/828690107 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2660066f30 0x7f26600693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:34.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.335+0000 7f2679faa700 1 --2- 172.21.15.89:0/828690107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2674159950 0x7f26740c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:34.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.335+0000 7f2679faa700 1 --2- 172.21.15.89:0/828690107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2674069aa0 0x7f267415e380 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:34.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.335+0000 7f2679faa700 1 --2- 172.21.15.89:0/828690107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26740af320 0x7f26740c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:34.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.335+0000 7f2679faa700 1 -- 172.21.15.89:0/828690107 >> 172.21.15.89:0/828690107 conn(0x7f2674068e00 msgr2=0x7f26740b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:34.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.336+0000 7f2679faa700 1 -- 172.21.15.89:0/828690107 shutdown_connections 2023-10-10T20:34:34.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.337+0000 7f2679faa700 1 -- 172.21.15.89:0/828690107 wait complete. 2023-10-10T20:34:34.342 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2023-10-10T20:34:34.357 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:34:34.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.477+0000 7f0c89b47700 1 Processor -- start 2023-10-10T20:34:34.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.477+0000 7f0c89b47700 1 -- start start 2023-10-10T20:34:34.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.478+0000 7f0c89b47700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c84159890 0x7f0c8415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:34.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.478+0000 7f0c89b47700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0c84159c60 0x7f0c8415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:34.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.478+0000 7f0c89b47700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0c8415db10 0x7f0c84069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:34.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.478+0000 7f0c89b47700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c8415a160 con 0x7f0c8415db10 2023-10-10T20:34:34.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.478+0000 7f0c89b47700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c840d13a0 con 0x7f0c84159c60 2023-10-10T20:34:34.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.478+0000 7f0c89b47700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8415e140 con 0x7f0c84159890 2023-10-10T20:34:34.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.478+0000 7f0c83fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0c8415db10 0x7f0c84069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:34.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.478+0000 7f0c83fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0c8415db10 0x7f0c84069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54330/0 (socket says 172.21.15.89:54330) 2023-10-10T20:34:34.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.478+0000 7f0c83fff700 1 -- 172.21.15.89:0/3545822836 learned_addr learned my addr 172.21.15.89:0/3545822836 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:34.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.478+0000 7f0c82ffd700 1 --2- 172.21.15.89:0/3545822836 >> v2:172.21.15.111:3300/0 conn(0x7f0c84159c60 0x7f0c8415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:34.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.478+0000 7f0c837fe700 1 --2- 172.21.15.89:0/3545822836 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c84159890 0x7f0c8415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:34.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c83fff700 1 -- 172.21.15.89:0/3545822836 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c84159890 msgr2=0x7f0c8415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:34.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c83fff700 1 --2- 172.21.15.89:0/3545822836 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c84159890 0x7f0c8415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:34.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c83fff700 1 -- 172.21.15.89:0/3545822836 >> v2:172.21.15.111:3300/0 conn(0x7f0c84159c60 msgr2=0x7f0c8415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:34.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c83fff700 1 --2- 172.21.15.89:0/3545822836 >> v2:172.21.15.111:3300/0 conn(0x7f0c84159c60 0x7f0c8415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:34.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c83fff700 1 -- 172.21.15.89:0/3545822836 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8415a3d0 con 0x7f0c8415db10 2023-10-10T20:34:34.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c837fe700 1 --2- 172.21.15.89:0/3545822836 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c84159890 0x7f0c8415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:34.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c82ffd700 1 --2- 172.21.15.89:0/3545822836 >> v2:172.21.15.111:3300/0 conn(0x7f0c84159c60 0x7f0c8415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:34.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c83fff700 1 --2- 172.21.15.89:0/3545822836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c8415db10 0x7f0c84069560 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0c840650f0 tx=0x7f0c7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bf3b706cb6889 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:34.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c827fc700 1 -- 172.21.15.89:0/3545822836 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c00b860 con 0x7f0c8415db10 2023-10-10T20:34:34.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c827fc700 1 -- 172.21.15.89:0/3545822836 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c00b9c0 con 0x7f0c8415db10 2023-10-10T20:34:34.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c827fc700 1 -- 172.21.15.89:0/3545822836 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c00bd60 con 0x7f0c8415db10 2023-10-10T20:34:34.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c89b47700 1 -- 172.21.15.89:0/3545822836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c8415db10 msgr2=0x7f0c84069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:34.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c89b47700 1 --2- 172.21.15.89:0/3545822836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c8415db10 0x7f0c84069560 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0c840650f0 tx=0x7f0c7c009250 comp rx=0 tx=0).stop 2023-10-10T20:34:34.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c89b47700 1 -- 172.21.15.89:0/3545822836 shutdown_connections 2023-10-10T20:34:34.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c89b47700 1 --2- 172.21.15.89:0/3545822836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c8415db10 0x7f0c84069560 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:34.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c89b47700 1 --2- 172.21.15.89:0/3545822836 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c84159890 0x7f0c8415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:34.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c89b47700 1 --2- 172.21.15.89:0/3545822836 >> v2:172.21.15.111:3300/0 conn(0x7f0c84159c60 0x7f0c8415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:34.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c89b47700 1 -- 172.21.15.89:0/3545822836 >> 172.21.15.89:0/3545822836 conn(0x7f0c84068e20 msgr2=0x7f0c840d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:34.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c89b47700 1 -- 172.21.15.89:0/3545822836 shutdown_connections 2023-10-10T20:34:34.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.479+0000 7f0c89b47700 1 -- 172.21.15.89:0/3545822836 wait complete. 2023-10-10T20:34:34.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.481+0000 7f0c89b47700 1 Processor -- start 2023-10-10T20:34:34.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.481+0000 7f0c89b47700 1 -- start start 2023-10-10T20:34:34.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.481+0000 7f0c89b47700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c84159890 0x7f0c8416ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:34.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.481+0000 7f0c89b47700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c84159c60 0x7f0c840cf940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:34.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.481+0000 7f0c89b47700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c8415db10 0x7f0c840cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:34.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.481+0000 7f0c89b47700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c84151f30 con 0x7f0c8415db10 2023-10-10T20:34:34.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.481+0000 7f0c89b47700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c840b0b40 con 0x7f0c84159890 2023-10-10T20:34:34.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.481+0000 7f0c89b47700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8416d530 con 0x7f0c84159c60 2023-10-10T20:34:34.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.481+0000 7f0c83fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c8415db10 0x7f0c840cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:34.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.481+0000 7f0c83fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c8415db10 0x7f0c840cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54342/0 (socket says 172.21.15.89:54342) 2023-10-10T20:34:34.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.481+0000 7f0c83fff700 1 -- 172.21.15.89:0/635660132 learned_addr learned my addr 172.21.15.89:0/635660132 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:34.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.482+0000 7f0c837fe700 1 --2- 172.21.15.89:0/635660132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c84159890 0x7f0c8416ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:34.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.482+0000 7f0c82ffd700 1 --2- 172.21.15.89:0/635660132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c84159c60 0x7f0c840cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:34.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.482+0000 7f0c83fff700 1 -- 172.21.15.89:0/635660132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c84159c60 msgr2=0x7f0c840cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:34.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.482+0000 7f0c83fff700 1 --2- 172.21.15.89:0/635660132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c84159c60 0x7f0c840cf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:34.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.482+0000 7f0c83fff700 1 -- 172.21.15.89:0/635660132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c84159890 msgr2=0x7f0c8416ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:34.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.482+0000 7f0c83fff700 1 --2- 172.21.15.89:0/635660132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c84159890 0x7f0c8416ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:34.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.482+0000 7f0c83fff700 1 -- 172.21.15.89:0/635660132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c0094f0 con 0x7f0c8415db10 2023-10-10T20:34:34.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.482+0000 7f0c837fe700 1 --2- 172.21.15.89:0/635660132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c84159890 0x7f0c8416ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:34.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.482+0000 7f0c83fff700 1 --2- 172.21.15.89:0/635660132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c8415db10 0x7f0c840cd0c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0c84070e20 tx=0x7f0c7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:34.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.482+0000 7f0c6b7fe700 1 -- 172.21.15.89:0/635660132 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c01a630 con 0x7f0c8415db10 2023-10-10T20:34:34.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.482+0000 7f0c6b7fe700 1 -- 172.21.15.89:0/635660132 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c01a7b0 con 0x7f0c8415db10 2023-10-10T20:34:34.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.482+0000 7f0c6b7fe700 1 -- 172.21.15.89:0/635660132 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c01ab70 con 0x7f0c8415db10 2023-10-10T20:34:34.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.482+0000 7f0c89b47700 1 -- 172.21.15.89:0/635660132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8416d810 con 0x7f0c8415db10 2023-10-10T20:34:34.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.482+0000 7f0c89b47700 1 -- 172.21.15.89:0/635660132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c840cfe80 con 0x7f0c8415db10 2023-10-10T20:34:34.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.483+0000 7f0c89b47700 1 -- 172.21.15.89:0/635660132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c840b0d00 con 0x7f0c8415db10 2023-10-10T20:34:34.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.483+0000 7f0c6b7fe700 1 -- 172.21.15.89:0/635660132 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c7c02d440 con 0x7f0c8415db10 2023-10-10T20:34:34.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.484+0000 7f0c6b7fe700 1 --2- 172.21.15.89:0/635660132 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0c70066f30 0x7f0c700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:34.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.484+0000 7f0c837fe700 1 --2- 172.21.15.89:0/635660132 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0c70066f30 0x7f0c700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:34.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.484+0000 7f0c6b7fe700 1 -- 172.21.15.89:0/635660132 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f0c7c0861a0 con 0x7f0c8415db10 2023-10-10T20:34:34.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.484+0000 7f0c837fe700 1 --2- 172.21.15.89:0/635660132 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0c70066f30 0x7f0c700693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400d510 tx=0x7f0c740075e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:34.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.488+0000 7f0c6b7fe700 1 -- 172.21.15.89:0/635660132 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c7c053470 con 0x7f0c8415db10 2023-10-10T20:34:34.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:34.716+0000 7f0c89b47700 1 -- 172.21.15.89:0/635660132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0c8404c3c0 con 0x7f0c8415db10 2023-10-10T20:34:35.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.346+0000 7f0c6b7fe700 1 -- 172.21.15.89:0/635660132 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0c7c056a90 con 0x7f0c8415db10 2023-10-10T20:34:35.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.349+0000 7f0c89b47700 1 -- 172.21.15.89:0/635660132 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0c70066f30 msgr2=0x7f0c700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:35.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.349+0000 7f0c89b47700 1 --2- 172.21.15.89:0/635660132 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0c70066f30 0x7f0c700693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400d510 tx=0x7f0c740075e0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.349+0000 7f0c89b47700 1 -- 172.21.15.89:0/635660132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c8415db10 msgr2=0x7f0c840cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:35.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.349+0000 7f0c89b47700 1 --2- 172.21.15.89:0/635660132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c8415db10 0x7f0c840cd0c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0c84070e20 tx=0x7f0c7c017000 comp rx=0 tx=0).stop 2023-10-10T20:34:35.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.349+0000 7f0c89b47700 1 -- 172.21.15.89:0/635660132 shutdown_connections 2023-10-10T20:34:35.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.349+0000 7f0c89b47700 1 --2- 172.21.15.89:0/635660132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c8415db10 0x7f0c840cd0c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.349+0000 7f0c89b47700 1 --2- 172.21.15.89:0/635660132 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0c70066f30 0x7f0c700693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.349+0000 7f0c89b47700 1 --2- 172.21.15.89:0/635660132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c84159890 0x7f0c8416ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.349+0000 7f0c89b47700 1 --2- 172.21.15.89:0/635660132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c84159c60 0x7f0c840cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.349+0000 7f0c89b47700 1 -- 172.21.15.89:0/635660132 >> 172.21.15.89:0/635660132 conn(0x7f0c84068e20 msgr2=0x7f0c84069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:35.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.350+0000 7f0c89b47700 1 -- 172.21.15.89:0/635660132 shutdown_connections 2023-10-10T20:34:35.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.350+0000 7f0c89b47700 1 -- 172.21.15.89:0/635660132 wait complete. 2023-10-10T20:34:35.355 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 2 and data pool 3 2023-10-10T20:34:35.370 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:34:35.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.483+0000 7f0149302700 1 Processor -- start 2023-10-10T20:34:35.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.483+0000 7f0149302700 1 -- start start 2023-10-10T20:34:35.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.484+0000 7f0149302700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f014406bcd0 0x7f0144140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:35.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.484+0000 7f0149302700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f014406c0a0 0x7f0144141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:35.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.484+0000 7f0149302700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f014415c2a0 con 0x7f01440c8e00 2023-10-10T20:34:35.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.484+0000 7f0149302700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f014415b690 con 0x7f014406c0a0 2023-10-10T20:34:35.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.484+0000 7f0149302700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014415b970 con 0x7f014406bcd0 2023-10-10T20:34:35.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.484+0000 7f013a7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f014406c0a0 0x7f0144141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:35.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.484+0000 7f01437fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f01440c8e00 0x7f01440c9260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48986/0 (socket says 172.21.15.89:48986) 2023-10-10T20:34:35.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.484+0000 7f01437fe700 1 -- 172.21.15.89:0/4140244394 learned_addr learned my addr 172.21.15.89:0/4140244394 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:35.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.484+0000 7f0142ffd700 1 --2- 172.21.15.89:0/4140244394 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f014406bcd0 0x7f0144140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:35.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.484+0000 7f013a7fc700 1 -- 172.21.15.89:0/4140244394 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f014406bcd0 msgr2=0x7f0144140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:35.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.484+0000 7f013a7fc700 1 --2- 172.21.15.89:0/4140244394 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f014406bcd0 0x7f0144140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.484+0000 7f013a7fc700 1 -- 172.21.15.89:0/4140244394 >> v1:172.21.15.89:6789/0 conn(0x7f01440c8e00 legacy=0x7f01440c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:35.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.484+0000 7f013a7fc700 1 -- 172.21.15.89:0/4140244394 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014415bc50 con 0x7f014406c0a0 2023-10-10T20:34:35.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.484+0000 7f01427fc700 1 -- 172.21.15.89:0/4140244394 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940060739 0 0) 0x7f014415c2a0 con 0x7f01440c8e00 2023-10-10T20:34:35.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.484+0000 7f013a7fc700 1 --2- 172.21.15.89:0/4140244394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f014406c0a0 0x7f0144141330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f012c0098e0 tx=0x7f012c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd10aaf52e50e12f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:35.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.484+0000 7f01427fc700 1 -- 172.21.15.89:0/4140244394 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c00b640 con 0x7f014406c0a0 2023-10-10T20:34:35.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.484+0000 7f01427fc700 1 -- 172.21.15.89:0/4140244394 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c00b7a0 con 0x7f014406c0a0 2023-10-10T20:34:35.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.484+0000 7f01427fc700 1 -- 172.21.15.89:0/4140244394 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c00bb40 con 0x7f014406c0a0 2023-10-10T20:34:35.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.485+0000 7f0149302700 1 -- 172.21.15.89:0/4140244394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f014406c0a0 msgr2=0x7f0144141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:35.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.485+0000 7f0149302700 1 --2- 172.21.15.89:0/4140244394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f014406c0a0 0x7f0144141330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f012c0098e0 tx=0x7f012c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.485+0000 7f0149302700 1 -- 172.21.15.89:0/4140244394 shutdown_connections 2023-10-10T20:34:35.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.485+0000 7f0149302700 1 --2- 172.21.15.89:0/4140244394 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f014406bcd0 0x7f0144140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.485+0000 7f0149302700 1 --2- 172.21.15.89:0/4140244394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f014406c0a0 0x7f0144141330 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.485+0000 7f0149302700 1 -- 172.21.15.89:0/4140244394 >> 172.21.15.89:0/4140244394 conn(0x7f01440ce600 msgr2=0x7f01440cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:35.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.485+0000 7f0149302700 1 -- 172.21.15.89:0/4140244394 shutdown_connections 2023-10-10T20:34:35.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.485+0000 7f0149302700 1 -- 172.21.15.89:0/4140244394 wait complete. 2023-10-10T20:34:35.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.486+0000 7f0149302700 1 Processor -- start 2023-10-10T20:34:35.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.486+0000 7f0149302700 1 -- start start 2023-10-10T20:34:35.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.487+0000 7f0149302700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f014406bcd0 0x7f014413fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:35.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.487+0000 7f0149302700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f014406c0a0 0x7f01440c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:35.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.487+0000 7f0149302700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01440c8e00 0x7f01440c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:35.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.487+0000 7f0149302700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014413d490 con 0x7f01440c8e00 2023-10-10T20:34:35.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.487+0000 7f0149302700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014413e010 con 0x7f014406bcd0 2023-10-10T20:34:35.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.487+0000 7f0149302700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0144140410 con 0x7f014406c0a0 2023-10-10T20:34:35.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.487+0000 7f01437fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01440c8e00 0x7f01440c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:35.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.487+0000 7f01437fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01440c8e00 0x7f01440c5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54374/0 (socket says 172.21.15.89:54374) 2023-10-10T20:34:35.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.487+0000 7f01437fe700 1 -- 172.21.15.89:0/11409243 learned_addr learned my addr 172.21.15.89:0/11409243 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:35.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.487+0000 7f013a7fc700 1 --2- 172.21.15.89:0/11409243 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f014406c0a0 0x7f01440c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:35.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.487+0000 7f0142ffd700 1 --2- 172.21.15.89:0/11409243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f014406bcd0 0x7f014413fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:35.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.487+0000 7f01437fe700 1 -- 172.21.15.89:0/11409243 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f014406c0a0 msgr2=0x7f01440c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:35.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.487+0000 7f01437fe700 1 --2- 172.21.15.89:0/11409243 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f014406c0a0 0x7f01440c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.487+0000 7f01437fe700 1 -- 172.21.15.89:0/11409243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f014406bcd0 msgr2=0x7f014413fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:35.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.487+0000 7f01437fe700 1 --2- 172.21.15.89:0/11409243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f014406bcd0 0x7f014413fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.487+0000 7f01437fe700 1 -- 172.21.15.89:0/11409243 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012c009420 con 0x7f01440c8e00 2023-10-10T20:34:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.487+0000 7f0142ffd700 1 --2- 172.21.15.89:0/11409243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f014406bcd0 0x7f014413fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.487+0000 7f01437fe700 1 --2- 172.21.15.89:0/11409243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01440c8e00 0x7f01440c5bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f013c00cc40 tx=0x7f013c001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.488+0000 7f013b7fe700 1 -- 172.21.15.89:0/11409243 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c00a920 con 0x7f01440c8e00 2023-10-10T20:34:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.488+0000 7f013b7fe700 1 -- 172.21.15.89:0/11409243 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c00aaa0 con 0x7f01440c8e00 2023-10-10T20:34:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.488+0000 7f0149302700 1 -- 172.21.15.89:0/11409243 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014413d6e0 con 0x7f01440c8e00 2023-10-10T20:34:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.488+0000 7f013b7fe700 1 -- 172.21.15.89:0/11409243 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c00ae60 con 0x7f01440c8e00 2023-10-10T20:34:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.488+0000 7f0149302700 1 -- 172.21.15.89:0/11409243 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014413dca0 con 0x7f01440c8e00 2023-10-10T20:34:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.489+0000 7f0149302700 1 -- 172.21.15.89:0/11409243 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014404c3c0 con 0x7f01440c8e00 2023-10-10T20:34:35.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.489+0000 7f013b7fe700 1 -- 172.21.15.89:0/11409243 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f013c007300 con 0x7f01440c8e00 2023-10-10T20:34:35.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.489+0000 7f013b7fe700 1 --2- 172.21.15.89:0/11409243 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0124066f30 0x7f01240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:35.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.489+0000 7f013b7fe700 1 -- 172.21.15.89:0/11409243 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f013c084920 con 0x7f01440c8e00 2023-10-10T20:34:35.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.489+0000 7f0142ffd700 1 --2- 172.21.15.89:0/11409243 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0124066f30 0x7f01240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:35.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.490+0000 7f0142ffd700 1 --2- 172.21.15.89:0/11409243 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0124066f30 0x7f01240693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0134006f00 tx=0x7f0134008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:35.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.494+0000 7f013b7fe700 1 -- 172.21.15.89:0/11409243 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f013c051c10 con 0x7f01440c8e00 2023-10-10T20:34:35.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.678+0000 7f0149302700 1 -- 172.21.15.89:0/11409243 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f014406c9d0 con 0x7f01440c8e00 2023-10-10T20:34:35.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.678+0000 7f013b7fe700 1 -- 172.21.15.89:0/11409243 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f013c055230 con 0x7f01440c8e00 2023-10-10T20:34:35.682 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:35.682 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":19,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:34.717450+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:34:32.342167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:34:33.671763+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:34:35.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.680+0000 7f0149302700 1 -- 172.21.15.89:0/11409243 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0124066f30 msgr2=0x7f01240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:35.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.680+0000 7f0149302700 1 --2- 172.21.15.89:0/11409243 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0124066f30 0x7f01240693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0134006f00 tx=0x7f0134008040 comp rx=0 tx=0).stop 2023-10-10T20:34:35.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.680+0000 7f0149302700 1 -- 172.21.15.89:0/11409243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01440c8e00 msgr2=0x7f01440c5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:35.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.680+0000 7f0149302700 1 --2- 172.21.15.89:0/11409243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01440c8e00 0x7f01440c5bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f013c00cc40 tx=0x7f013c001970 comp rx=0 tx=0).stop 2023-10-10T20:34:35.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.680+0000 7f0149302700 1 -- 172.21.15.89:0/11409243 shutdown_connections 2023-10-10T20:34:35.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.680+0000 7f0149302700 1 --2- 172.21.15.89:0/11409243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01440c8e00 0x7f01440c5bf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.680+0000 7f0149302700 1 --2- 172.21.15.89:0/11409243 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0124066f30 0x7f01240693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.680+0000 7f0149302700 1 --2- 172.21.15.89:0/11409243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f014406bcd0 0x7f014413fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.680+0000 7f0149302700 1 --2- 172.21.15.89:0/11409243 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f014406c0a0 0x7f01440c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.680+0000 7f0149302700 1 -- 172.21.15.89:0/11409243 >> 172.21.15.89:0/11409243 conn(0x7f01440ce600 msgr2=0x7f01440b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:35.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.680+0000 7f0149302700 1 -- 172.21.15.89:0/11409243 shutdown_connections 2023-10-10T20:34:35.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.680+0000 7f0149302700 1 -- 172.21.15.89:0/11409243 wait complete. 2023-10-10T20:34:35.696 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:34:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.806+0000 7f5ddaf47700 1 Processor -- start 2023-10-10T20:34:35.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.806+0000 7f5ddaf47700 1 -- start start 2023-10-10T20:34:35.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.807+0000 7f5ddaf47700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5dd41597f0 0x7f5dd40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:35.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.807+0000 7f5ddaf47700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5dd415d230 0x7f5dd415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:35.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.807+0000 7f5ddaf47700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd415dbd0 con 0x7f5dd4159420 2023-10-10T20:34:35.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.807+0000 7f5ddaf47700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd40b11d0 con 0x7f5dd415d230 2023-10-10T20:34:35.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.807+0000 7f5ddaf47700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40b14b0 con 0x7f5dd41597f0 2023-10-10T20:34:35.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.807+0000 7f5dda746700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5dd415d230 0x7f5dd415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:35.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.807+0000 7f5dda746700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5dd415d230 0x7f5dd415d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54390/0 (socket says 172.21.15.89:54390) 2023-10-10T20:34:35.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.807+0000 7f5dda746700 1 -- 172.21.15.89:0/2570864709 learned_addr learned my addr 172.21.15.89:0/2570864709 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:35.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.807+0000 7f5dd9744700 1 --2- 172.21.15.89:0/2570864709 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5dd41597f0 0x7f5dd40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:35.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.807+0000 7f5dda746700 1 -- 172.21.15.89:0/2570864709 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5dd41597f0 msgr2=0x7f5dd40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:35.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.807+0000 7f5dda746700 1 --2- 172.21.15.89:0/2570864709 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5dd41597f0 0x7f5dd40b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.807+0000 7f5dda746700 1 -- 172.21.15.89:0/2570864709 >> v1:172.21.15.111:6789/0 conn(0x7f5dd4159420 legacy=0x7f5dd40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:35.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.807+0000 7f5dda746700 1 -- 172.21.15.89:0/2570864709 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd40b1790 con 0x7f5dd415d230 2023-10-10T20:34:35.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.807+0000 7f5dda746700 1 --2- 172.21.15.89:0/2570864709 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dd415d230 0x7f5dd415d690 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc009f20 tx=0x7f5dcc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8689ba43de22b78e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:35.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.808+0000 7f5dd8f43700 1 -- 172.21.15.89:0/2570864709 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc00e070 con 0x7f5dd415d230 2023-10-10T20:34:35.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.808+0000 7f5dd8f43700 1 -- 172.21.15.89:0/2570864709 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dcc00b9d0 con 0x7f5dd415d230 2023-10-10T20:34:35.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.808+0000 7f5dd8f43700 1 -- 172.21.15.89:0/2570864709 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc0092e0 con 0x7f5dd415d230 2023-10-10T20:34:35.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.808+0000 7f5ddaf47700 1 -- 172.21.15.89:0/2570864709 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dd415d230 msgr2=0x7f5dd415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:35.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.808+0000 7f5ddaf47700 1 --2- 172.21.15.89:0/2570864709 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dd415d230 0x7f5dd415d690 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc009f20 tx=0x7f5dcc009840 comp rx=0 tx=0).stop 2023-10-10T20:34:35.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.808+0000 7f5ddaf47700 1 -- 172.21.15.89:0/2570864709 shutdown_connections 2023-10-10T20:34:35.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.808+0000 7f5ddaf47700 1 --2- 172.21.15.89:0/2570864709 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dd415d230 0x7f5dd415d690 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.808+0000 7f5ddaf47700 1 --2- 172.21.15.89:0/2570864709 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5dd41597f0 0x7f5dd40b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.808+0000 7f5ddaf47700 1 -- 172.21.15.89:0/2570864709 >> 172.21.15.89:0/2570864709 conn(0x7f5dd40d0660 msgr2=0x7f5dd40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:35.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.808+0000 7f5ddaf47700 1 -- 172.21.15.89:0/2570864709 shutdown_connections 2023-10-10T20:34:35.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.808+0000 7f5ddaf47700 1 -- 172.21.15.89:0/2570864709 wait complete. 2023-10-10T20:34:35.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.809+0000 7f5ddaf47700 1 Processor -- start 2023-10-10T20:34:35.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.809+0000 7f5ddaf47700 1 -- start start 2023-10-10T20:34:35.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.810+0000 7f5ddaf47700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dd4159420 0x7f5dd416a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:35.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.810+0000 7f5ddaf47700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dd41597f0 0x7f5dd416c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:35.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.810+0000 7f5ddaf47700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5dd415d230 0x7f5dd415aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:35.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.810+0000 7f5ddaf47700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd416ca90 con 0x7f5dd4159420 2023-10-10T20:34:35.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.810+0000 7f5ddaf47700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd415b000 con 0x7f5dd41597f0 2023-10-10T20:34:35.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.810+0000 7f5ddaf47700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40b51b0 con 0x7f5dd415d230 2023-10-10T20:34:35.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.810+0000 7f5dd9f45700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dd4159420 0x7f5dd416a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:35.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.810+0000 7f5dd9f45700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dd4159420 0x7f5dd416a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54392/0 (socket says 172.21.15.89:54392) 2023-10-10T20:34:35.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.810+0000 7f5dd9f45700 1 -- 172.21.15.89:0/1067233611 learned_addr learned my addr 172.21.15.89:0/1067233611 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:35.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.810+0000 7f5dda746700 1 --2- 172.21.15.89:0/1067233611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5dd415d230 0x7f5dd415aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:35.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.810+0000 7f5dd9744700 1 --2- 172.21.15.89:0/1067233611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dd41597f0 0x7f5dd416c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:35.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.810+0000 7f5dd9f45700 1 -- 172.21.15.89:0/1067233611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5dd415d230 msgr2=0x7f5dd415aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:35.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.810+0000 7f5dd9f45700 1 --2- 172.21.15.89:0/1067233611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5dd415d230 0x7f5dd415aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.810+0000 7f5dd9f45700 1 -- 172.21.15.89:0/1067233611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dd41597f0 msgr2=0x7f5dd416c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:35.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.810+0000 7f5dd9f45700 1 --2- 172.21.15.89:0/1067233611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dd41597f0 0x7f5dd416c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:35.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.810+0000 7f5dd9f45700 1 -- 172.21.15.89:0/1067233611 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc009ae0 con 0x7f5dd4159420 2023-10-10T20:34:35.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.810+0000 7f5dd9744700 1 --2- 172.21.15.89:0/1067233611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dd41597f0 0x7f5dd416c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:35.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.811+0000 7f5dd9f45700 1 --2- 172.21.15.89:0/1067233611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dd4159420 0x7f5dd416a1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000c5c0 tx=0x7f5dd000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:35.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.811+0000 7f5dc9ffb700 1 -- 172.21.15.89:0/1067233611 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0011590 con 0x7f5dd4159420 2023-10-10T20:34:35.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.811+0000 7f5dc9ffb700 1 -- 172.21.15.89:0/1067233611 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd0011710 con 0x7f5dd4159420 2023-10-10T20:34:35.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.811+0000 7f5ddaf47700 1 -- 172.21.15.89:0/1067233611 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd415b7f0 con 0x7f5dd4159420 2023-10-10T20:34:35.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.811+0000 7f5dc9ffb700 1 -- 172.21.15.89:0/1067233611 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0011ad0 con 0x7f5dd4159420 2023-10-10T20:34:35.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.811+0000 7f5ddaf47700 1 -- 172.21.15.89:0/1067233611 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd415b510 con 0x7f5dd4159420 2023-10-10T20:34:35.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.812+0000 7f5ddaf47700 1 -- 172.21.15.89:0/1067233611 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd40d1180 con 0x7f5dd4159420 2023-10-10T20:34:35.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.812+0000 7f5dc9ffb700 1 -- 172.21.15.89:0/1067233611 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5dd001b3b0 con 0x7f5dd4159420 2023-10-10T20:34:35.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.812+0000 7f5dc9ffb700 1 --2- 172.21.15.89:0/1067233611 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5db8066f30 0x7f5db80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:35.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.812+0000 7f5dc9ffb700 1 -- 172.21.15.89:0/1067233611 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5dd0083950 con 0x7f5dd4159420 2023-10-10T20:34:35.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.812+0000 7f5dd9744700 1 --2- 172.21.15.89:0/1067233611 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5db8066f30 0x7f5db80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:35.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.813+0000 7f5dd9744700 1 --2- 172.21.15.89:0/1067233611 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5db8066f30 0x7f5db80693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4006f00 tx=0x7f5dc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:35.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:35.816+0000 7f5dc9ffb700 1 -- 172.21.15.89:0/1067233611 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5dd0050bc0 con 0x7f5dd4159420 2023-10-10T20:34:36.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.000+0000 7f5ddaf47700 1 -- 172.21.15.89:0/1067233611 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5dd415dbd0 con 0x7f5dd4159420 2023-10-10T20:34:36.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.001+0000 7f5dc9ffb700 1 -- 172.21.15.89:0/1067233611 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f5dd415dbd0 con 0x7f5dd4159420 2023-10-10T20:34:36.004 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:36.005 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":19,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:34.717450+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:34:32.342167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:34:33.671763+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:34:36.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.003+0000 7f5ddaf47700 1 -- 172.21.15.89:0/1067233611 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5db8066f30 msgr2=0x7f5db80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:36.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.003+0000 7f5ddaf47700 1 --2- 172.21.15.89:0/1067233611 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5db8066f30 0x7f5db80693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4006f00 tx=0x7f5dc4008040 comp rx=0 tx=0).stop 2023-10-10T20:34:36.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.003+0000 7f5ddaf47700 1 -- 172.21.15.89:0/1067233611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dd4159420 msgr2=0x7f5dd416a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:36.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.003+0000 7f5ddaf47700 1 --2- 172.21.15.89:0/1067233611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dd4159420 0x7f5dd416a1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000c5c0 tx=0x7f5dd000c980 comp rx=0 tx=0).stop 2023-10-10T20:34:36.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.003+0000 7f5ddaf47700 1 -- 172.21.15.89:0/1067233611 shutdown_connections 2023-10-10T20:34:36.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.003+0000 7f5ddaf47700 1 --2- 172.21.15.89:0/1067233611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5dd415d230 0x7f5dd415aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:36.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.003+0000 7f5ddaf47700 1 --2- 172.21.15.89:0/1067233611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dd4159420 0x7f5dd416a1a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:36.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.003+0000 7f5ddaf47700 1 --2- 172.21.15.89:0/1067233611 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5db8066f30 0x7f5db80693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:36.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.003+0000 7f5ddaf47700 1 --2- 172.21.15.89:0/1067233611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dd41597f0 0x7f5dd416c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:36.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.003+0000 7f5ddaf47700 1 -- 172.21.15.89:0/1067233611 >> 172.21.15.89:0/1067233611 conn(0x7f5dd40d0660 msgr2=0x7f5dd4157220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:36.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.003+0000 7f5ddaf47700 1 -- 172.21.15.89:0/1067233611 shutdown_connections 2023-10-10T20:34:36.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.003+0000 7f5ddaf47700 1 -- 172.21.15.89:0/1067233611 wait complete. 2023-10-10T20:34:36.019 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:34:36.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.133+0000 7fb1424f6700 1 Processor -- start 2023-10-10T20:34:36.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.133+0000 7fb1424f6700 1 -- start start 2023-10-10T20:34:36.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.134+0000 7fb1424f6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb13c157680 0x7fb13c0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:36.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.134+0000 7fb1424f6700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb13c0b1450 0x7fb13c0c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:36.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.134+0000 7fb1424f6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13c0cd240 con 0x7fb13c157a50 2023-10-10T20:34:36.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.134+0000 7fb1424f6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb13c0cc700 con 0x7fb13c157680 2023-10-10T20:34:36.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.134+0000 7fb1424f6700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb13c0c9a20 con 0x7fb13c0b1450 2023-10-10T20:34:36.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.134+0000 7fb1414f4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb13c157680 0x7fb13c0cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:36.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.134+0000 7fb140cf3700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb13c157a50 0x7fb13c0c8cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48996/0 (socket says 172.21.15.89:48996) 2023-10-10T20:34:36.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.134+0000 7fb140cf3700 1 -- 172.21.15.89:0/3089688841 learned_addr learned my addr 172.21.15.89:0/3089688841 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:36.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.134+0000 7fb1414f4700 1 -- 172.21.15.89:0/3089688841 >> v2:172.21.15.111:3300/0 conn(0x7fb13c0b1450 msgr2=0x7fb13c0c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:36.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.134+0000 7fb141cf5700 1 --2- 172.21.15.89:0/3089688841 >> v2:172.21.15.111:3300/0 conn(0x7fb13c0b1450 0x7fb13c0c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:36.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.134+0000 7fb1414f4700 1 --2- 172.21.15.89:0/3089688841 >> v2:172.21.15.111:3300/0 conn(0x7fb13c0b1450 0x7fb13c0c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:36.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.134+0000 7fb133fff700 1 -- 172.21.15.89:0/3089688841 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082522958 0 0) 0x7fb13c0cd240 con 0x7fb13c157a50 2023-10-10T20:34:36.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.134+0000 7fb1414f4700 1 -- 172.21.15.89:0/3089688841 >> v1:172.21.15.89:6789/0 conn(0x7fb13c157a50 legacy=0x7fb13c0c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:36.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.134+0000 7fb1414f4700 1 -- 172.21.15.89:0/3089688841 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c0cc910 con 0x7fb13c157680 2023-10-10T20:34:36.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.134+0000 7fb141cf5700 1 --2- 172.21.15.89:0/3089688841 >> v2:172.21.15.111:3300/0 conn(0x7fb13c0b1450 0x7fb13c0c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:36.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.134+0000 7fb1414f4700 1 --2- 172.21.15.89:0/3089688841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb13c157680 0x7fb13c0cae80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb13c15b360 tx=0x7fb1380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc48076dc8f9c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:36.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.135+0000 7fb133fff700 1 -- 172.21.15.89:0/3089688841 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13800b6b0 con 0x7fb13c157680 2023-10-10T20:34:36.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.135+0000 7fb133fff700 1 -- 172.21.15.89:0/3089688841 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13800b810 con 0x7fb13c157680 2023-10-10T20:34:36.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.135+0000 7fb133fff700 1 -- 172.21.15.89:0/3089688841 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13800bbb0 con 0x7fb13c157680 2023-10-10T20:34:36.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.135+0000 7fb1424f6700 1 -- 172.21.15.89:0/3089688841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb13c157680 msgr2=0x7fb13c0cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:36.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.135+0000 7fb1424f6700 1 --2- 172.21.15.89:0/3089688841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb13c157680 0x7fb13c0cae80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb13c15b360 tx=0x7fb1380091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:36.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.135+0000 7fb1424f6700 1 -- 172.21.15.89:0/3089688841 shutdown_connections 2023-10-10T20:34:36.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.135+0000 7fb1424f6700 1 --2- 172.21.15.89:0/3089688841 >> v2:172.21.15.111:3300/0 conn(0x7fb13c0b1450 0x7fb13c0c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:36.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.135+0000 7fb1424f6700 1 --2- 172.21.15.89:0/3089688841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb13c157680 0x7fb13c0cae80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:36.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.135+0000 7fb1424f6700 1 -- 172.21.15.89:0/3089688841 >> 172.21.15.89:0/3089688841 conn(0x7fb13c06baa0 msgr2=0x7fb13c06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:36.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.135+0000 7fb1424f6700 1 -- 172.21.15.89:0/3089688841 shutdown_connections 2023-10-10T20:34:36.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.135+0000 7fb1424f6700 1 -- 172.21.15.89:0/3089688841 wait complete. 2023-10-10T20:34:36.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.136+0000 7fb1424f6700 1 Processor -- start 2023-10-10T20:34:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.136+0000 7fb1424f6700 1 -- start start 2023-10-10T20:34:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.137+0000 7fb1424f6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb13c0b1450 0x7fb13c145120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.137+0000 7fb1424f6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb13c157680 0x7fb13c15c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.137+0000 7fb1424f6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb13c157a50 0x7fb13c15c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.137+0000 7fb1424f6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c167c30 con 0x7fb13c0b1450 2023-10-10T20:34:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.137+0000 7fb1424f6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c168a50 con 0x7fb13c157a50 2023-10-10T20:34:36.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.137+0000 7fb1424f6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c145660 con 0x7fb13c157680 2023-10-10T20:34:36.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.137+0000 7fb1414f4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb13c0b1450 0x7fb13c145120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:36.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.137+0000 7fb1414f4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb13c0b1450 0x7fb13c145120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54408/0 (socket says 172.21.15.89:54408) 2023-10-10T20:34:36.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.137+0000 7fb1414f4700 1 -- 172.21.15.89:0/4121853970 learned_addr learned my addr 172.21.15.89:0/4121853970 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:36.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.137+0000 7fb140cf3700 1 --2- 172.21.15.89:0/4121853970 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb13c157680 0x7fb13c15c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:36.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.137+0000 7fb141cf5700 1 --2- 172.21.15.89:0/4121853970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb13c157a50 0x7fb13c15c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:36.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.137+0000 7fb1414f4700 1 -- 172.21.15.89:0/4121853970 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb13c157680 msgr2=0x7fb13c15c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:36.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.137+0000 7fb1414f4700 1 --2- 172.21.15.89:0/4121853970 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb13c157680 0x7fb13c15c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:36.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.137+0000 7fb1414f4700 1 -- 172.21.15.89:0/4121853970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb13c157a50 msgr2=0x7fb13c15c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:36.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.137+0000 7fb1414f4700 1 --2- 172.21.15.89:0/4121853970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb13c157a50 0x7fb13c15c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:36.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.137+0000 7fb1414f4700 1 -- 172.21.15.89:0/4121853970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb138009420 con 0x7fb13c0b1450 2023-10-10T20:34:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.137+0000 7fb140cf3700 1 --2- 172.21.15.89:0/4121853970 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb13c157680 0x7fb13c15c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.138+0000 7fb1414f4700 1 --2- 172.21.15.89:0/4121853970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb13c0b1450 0x7fb13c145120 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb13c06cdc0 tx=0x7fb138017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.138+0000 7fb1317fa700 1 -- 172.21.15.89:0/4121853970 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb138011420 con 0x7fb13c0b1450 2023-10-10T20:34:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.138+0000 7fb1317fa700 1 -- 172.21.15.89:0/4121853970 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb138011580 con 0x7fb13c0b1450 2023-10-10T20:34:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.138+0000 7fb1424f6700 1 -- 172.21.15.89:0/4121853970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13c167e40 con 0x7fb13c0b1450 2023-10-10T20:34:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.138+0000 7fb1317fa700 1 -- 172.21.15.89:0/4121853970 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb138011920 con 0x7fb13c0b1450 2023-10-10T20:34:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.138+0000 7fb1424f6700 1 -- 172.21.15.89:0/4121853970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13c168400 con 0x7fb13c0b1450 2023-10-10T20:34:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.139+0000 7fb1424f6700 1 -- 172.21.15.89:0/4121853970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13c06eba0 con 0x7fb13c0b1450 2023-10-10T20:34:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.139+0000 7fb1317fa700 1 -- 172.21.15.89:0/4121853970 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb138011d10 con 0x7fb13c0b1450 2023-10-10T20:34:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.139+0000 7fb1317fa700 1 --2- 172.21.15.89:0/4121853970 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb120066f30 0x7fb1200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.140+0000 7fb140cf3700 1 --2- 172.21.15.89:0/4121853970 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb120066f30 0x7fb1200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.140+0000 7fb1317fa700 1 -- 172.21.15.89:0/4121853970 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb1380860f0 con 0x7fb13c0b1450 2023-10-10T20:34:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.140+0000 7fb140cf3700 1 --2- 172.21.15.89:0/4121853970 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb120066f30 0x7fb1200693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb12c007fd0 tx=0x7fb12c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.144+0000 7fb1317fa700 1 -- 172.21.15.89:0/4121853970 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1380533e0 con 0x7fb13c0b1450 2023-10-10T20:34:36.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:36.371+0000 7fb1424f6700 1 -- 172.21.15.89:0/4121853970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb13c0ccbf0 con 0x7fb13c0b1450 2023-10-10T20:34:37.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.356+0000 7fb1317fa700 1 -- 172.21.15.89:0/4121853970 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb138056a00 con 0x7fb13c0b1450 2023-10-10T20:34:37.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.358+0000 7fb1424f6700 1 -- 172.21.15.89:0/4121853970 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb120066f30 msgr2=0x7fb1200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:37.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.358+0000 7fb1424f6700 1 --2- 172.21.15.89:0/4121853970 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb120066f30 0x7fb1200693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb12c007fd0 tx=0x7fb12c009040 comp rx=0 tx=0).stop 2023-10-10T20:34:37.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.358+0000 7fb1424f6700 1 -- 172.21.15.89:0/4121853970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb13c0b1450 msgr2=0x7fb13c145120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:37.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.358+0000 7fb1424f6700 1 --2- 172.21.15.89:0/4121853970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb13c0b1450 0x7fb13c145120 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb13c06cdc0 tx=0x7fb138017000 comp rx=0 tx=0).stop 2023-10-10T20:34:37.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.358+0000 7fb1424f6700 1 -- 172.21.15.89:0/4121853970 shutdown_connections 2023-10-10T20:34:37.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.358+0000 7fb1424f6700 1 --2- 172.21.15.89:0/4121853970 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb120066f30 0x7fb1200693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:37.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.358+0000 7fb1424f6700 1 --2- 172.21.15.89:0/4121853970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb13c157a50 0x7fb13c15c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:37.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.358+0000 7fb1424f6700 1 --2- 172.21.15.89:0/4121853970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb13c0b1450 0x7fb13c145120 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:37.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.358+0000 7fb1424f6700 1 --2- 172.21.15.89:0/4121853970 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb13c157680 0x7fb13c15c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:37.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.358+0000 7fb1424f6700 1 -- 172.21.15.89:0/4121853970 >> 172.21.15.89:0/4121853970 conn(0x7fb13c06baa0 msgr2=0x7fb13c0c9180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:37.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.358+0000 7fb1424f6700 1 -- 172.21.15.89:0/4121853970 shutdown_connections 2023-10-10T20:34:37.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.358+0000 7fb1424f6700 1 -- 172.21.15.89:0/4121853970 wait complete. 2023-10-10T20:34:37.373 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:34:37.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.489+0000 7f712df6e700 1 Processor -- start 2023-10-10T20:34:37.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.489+0000 7f712df6e700 1 -- start start 2023-10-10T20:34:37.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.489+0000 7f712df6e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7128155eb0 0x7f712806e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:37.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.489+0000 7f712df6e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f71280c70b0 0x7f7128069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:37.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.489+0000 7f712df6e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712806eb90 con 0x7f7128069cb0 2023-10-10T20:34:37.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.490+0000 7f712df6e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f712806a130 con 0x7f7128155eb0 2023-10-10T20:34:37.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.490+0000 7f712df6e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71280cd350 con 0x7f71280c70b0 2023-10-10T20:34:37.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.490+0000 7f71277fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7128155eb0 0x7f712806e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:37.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.490+0000 7f7127fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7128069cb0 0x7f71280ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48998/0 (socket says 172.21.15.89:48998) 2023-10-10T20:34:37.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.490+0000 7f7127fff700 1 -- 172.21.15.89:0/405294811 learned_addr learned my addr 172.21.15.89:0/405294811 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:37.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.490+0000 7f7126ffd700 1 --2- 172.21.15.89:0/405294811 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f71280c70b0 0x7f7128069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:37.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.490+0000 7f71277fe700 1 -- 172.21.15.89:0/405294811 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f71280c70b0 msgr2=0x7f7128069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:37.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.490+0000 7f71277fe700 1 --2- 172.21.15.89:0/405294811 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f71280c70b0 0x7f7128069770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:37.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.490+0000 7f71277fe700 1 -- 172.21.15.89:0/405294811 >> v1:172.21.15.89:6789/0 conn(0x7f7128069cb0 legacy=0x7f71280ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:37.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.490+0000 7f71267fc700 1 -- 172.21.15.89:0/405294811 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640241945 0 0) 0x7f712806eb90 con 0x7f7128069cb0 2023-10-10T20:34:37.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.490+0000 7f71277fe700 1 -- 172.21.15.89:0/405294811 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712806ee70 con 0x7f7128155eb0 2023-10-10T20:34:37.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.490+0000 7f71277fe700 1 --2- 172.21.15.89:0/405294811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7128155eb0 0x7f712806e240 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7118009910 tx=0x7f71180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f46075f201f2e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:37.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.490+0000 7f71267fc700 1 -- 172.21.15.89:0/405294811 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711800b820 con 0x7f7128155eb0 2023-10-10T20:34:37.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.490+0000 7f71267fc700 1 -- 172.21.15.89:0/405294811 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711800b980 con 0x7f7128155eb0 2023-10-10T20:34:37.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.490+0000 7f71267fc700 1 -- 172.21.15.89:0/405294811 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711800bd20 con 0x7f7128155eb0 2023-10-10T20:34:37.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.491+0000 7f712df6e700 1 -- 172.21.15.89:0/405294811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7128155eb0 msgr2=0x7f712806e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:37.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.491+0000 7f712df6e700 1 --2- 172.21.15.89:0/405294811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7128155eb0 0x7f712806e240 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7118009910 tx=0x7f71180091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:37.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.491+0000 7f712df6e700 1 -- 172.21.15.89:0/405294811 shutdown_connections 2023-10-10T20:34:37.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.491+0000 7f712df6e700 1 --2- 172.21.15.89:0/405294811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7128155eb0 0x7f712806e240 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:37.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.491+0000 7f712df6e700 1 --2- 172.21.15.89:0/405294811 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f71280c70b0 0x7f7128069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:37.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.491+0000 7f712df6e700 1 -- 172.21.15.89:0/405294811 >> 172.21.15.89:0/405294811 conn(0x7f7128068e10 msgr2=0x7f71280cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:37.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.491+0000 7f712df6e700 1 -- 172.21.15.89:0/405294811 shutdown_connections 2023-10-10T20:34:37.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.491+0000 7f712df6e700 1 -- 172.21.15.89:0/405294811 wait complete. 2023-10-10T20:34:37.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.492+0000 7f712df6e700 1 Processor -- start 2023-10-10T20:34:37.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.492+0000 7f712df6e700 1 -- start start 2023-10-10T20:34:37.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f712df6e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7128069cb0 0x7f7128144e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:37.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f712df6e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71280c70b0 0x7f7128158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:37.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f712df6e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7128155eb0 0x7f71280ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:37.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f712df6e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7128145b70 con 0x7f7128069cb0 2023-10-10T20:34:37.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f712df6e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71281594e0 con 0x7f71280c70b0 2023-10-10T20:34:37.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f712df6e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71281597c0 con 0x7f7128155eb0 2023-10-10T20:34:37.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f71277fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7128069cb0 0x7f7128144e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:37.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f71277fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7128069cb0 0x7f7128144e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54426/0 (socket says 172.21.15.89:54426) 2023-10-10T20:34:37.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f71277fe700 1 -- 172.21.15.89:0/3685078005 learned_addr learned my addr 172.21.15.89:0/3685078005 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:37.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f7127fff700 1 --2- 172.21.15.89:0/3685078005 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7128155eb0 0x7f71280ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:37.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f7126ffd700 1 --2- 172.21.15.89:0/3685078005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71280c70b0 0x7f7128158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:37.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f71277fe700 1 -- 172.21.15.89:0/3685078005 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7128155eb0 msgr2=0x7f71280ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:37.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f71277fe700 1 --2- 172.21.15.89:0/3685078005 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7128155eb0 0x7f71280ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:37.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f71277fe700 1 -- 172.21.15.89:0/3685078005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71280c70b0 msgr2=0x7f7128158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:37.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f71277fe700 1 --2- 172.21.15.89:0/3685078005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71280c70b0 0x7f7128158af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:37.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f71277fe700 1 -- 172.21.15.89:0/3685078005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7118009420 con 0x7f7128069cb0 2023-10-10T20:34:37.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f7126ffd700 1 --2- 172.21.15.89:0/3685078005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71280c70b0 0x7f7128158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:37.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f71277fe700 1 --2- 172.21.15.89:0/3685078005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7128069cb0 0x7f7128144e20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f711800b3c0 tx=0x7f7118017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:37.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f710f7fe700 1 -- 172.21.15.89:0/3685078005 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711801f070 con 0x7f7128069cb0 2023-10-10T20:34:37.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f710f7fe700 1 -- 172.21.15.89:0/3685078005 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7118002f70 con 0x7f7128069cb0 2023-10-10T20:34:37.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f712df6e700 1 -- 172.21.15.89:0/3685078005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7128159aa0 con 0x7f7128069cb0 2023-10-10T20:34:37.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f710f7fe700 1 -- 172.21.15.89:0/3685078005 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118025420 con 0x7f7128069cb0 2023-10-10T20:34:37.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.493+0000 7f712df6e700 1 -- 172.21.15.89:0/3685078005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7128145360 con 0x7f7128069cb0 2023-10-10T20:34:37.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.494+0000 7f712df6e700 1 -- 172.21.15.89:0/3685078005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7128145550 con 0x7f7128069cb0 2023-10-10T20:34:37.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.495+0000 7f710f7fe700 1 -- 172.21.15.89:0/3685078005 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7118023070 con 0x7f7128069cb0 2023-10-10T20:34:37.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.495+0000 7f710f7fe700 1 --2- 172.21.15.89:0/3685078005 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7114066fb0 0x7f7114069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:37.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.495+0000 7f710f7fe700 1 -- 172.21.15.89:0/3685078005 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7118085a90 con 0x7f7128069cb0 2023-10-10T20:34:37.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.495+0000 7f7126ffd700 1 --2- 172.21.15.89:0/3685078005 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7114066fb0 0x7f7114069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:37.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.496+0000 7f7126ffd700 1 --2- 172.21.15.89:0/3685078005 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7114066fb0 0x7f7114069460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7128155380 tx=0x7f711c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:37.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.499+0000 7f710f7fe700 1 -- 172.21.15.89:0/3685078005 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7118052c60 con 0x7f7128069cb0 2023-10-10T20:34:37.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:37.723+0000 7f712df6e700 1 -- 172.21.15.89:0/3685078005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f712804c3c0 con 0x7f7128069cb0 2023-10-10T20:34:38.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.357+0000 7f710f7fe700 1 -- 172.21.15.89:0/3685078005 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f711802a0c0 con 0x7f7128069cb0 2023-10-10T20:34:38.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.359+0000 7f712df6e700 1 -- 172.21.15.89:0/3685078005 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7114066fb0 msgr2=0x7f7114069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:38.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.359+0000 7f712df6e700 1 --2- 172.21.15.89:0/3685078005 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7114066fb0 0x7f7114069460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7128155380 tx=0x7f711c008040 comp rx=0 tx=0).stop 2023-10-10T20:34:38.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.359+0000 7f712df6e700 1 -- 172.21.15.89:0/3685078005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7128069cb0 msgr2=0x7f7128144e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:38.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.359+0000 7f712df6e700 1 --2- 172.21.15.89:0/3685078005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7128069cb0 0x7f7128144e20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f711800b3c0 tx=0x7f7118017000 comp rx=0 tx=0).stop 2023-10-10T20:34:38.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.359+0000 7f712df6e700 1 -- 172.21.15.89:0/3685078005 shutdown_connections 2023-10-10T20:34:38.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.359+0000 7f712df6e700 1 --2- 172.21.15.89:0/3685078005 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7128155eb0 0x7f71280ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:38.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.359+0000 7f712df6e700 1 --2- 172.21.15.89:0/3685078005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7128069cb0 0x7f7128144e20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:38.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.359+0000 7f712df6e700 1 --2- 172.21.15.89:0/3685078005 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7114066fb0 0x7f7114069460 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:38.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.359+0000 7f712df6e700 1 --2- 172.21.15.89:0/3685078005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71280c70b0 0x7f7128158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:38.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.359+0000 7f712df6e700 1 -- 172.21.15.89:0/3685078005 >> 172.21.15.89:0/3685078005 conn(0x7f7128068e10 msgr2=0x7f71280c12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:38.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.359+0000 7f712df6e700 1 -- 172.21.15.89:0/3685078005 shutdown_connections 2023-10-10T20:34:38.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.359+0000 7f712df6e700 1 -- 172.21.15.89:0/3685078005 wait complete. 2023-10-10T20:34:38.375 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:34:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.487+0000 7fbbfca0a700 1 Processor -- start 2023-10-10T20:34:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.487+0000 7fbbfca0a700 1 -- start start 2023-10-10T20:34:38.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.488+0000 7fbbfca0a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbbf806aa40 0x7fbbf80cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:38.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.488+0000 7fbbfca0a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbbf80c91e0 0x7fbbf8160390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:38.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.488+0000 7fbbfca0a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf80cc210 con 0x7fbbf806bac0 2023-10-10T20:34:38.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.488+0000 7fbbfca0a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf80cc4f0 con 0x7fbbf806aa40 2023-10-10T20:34:38.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.488+0000 7fbbfca0a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf80cc7d0 con 0x7fbbf80c91e0 2023-10-10T20:34:38.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.488+0000 7fbbf659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbbf806aa40 0x7fbbf80cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:38.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.488+0000 7fbbeffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbbf806bac0 0x7fbbf806ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35620/0 (socket says 172.21.15.89:35620) 2023-10-10T20:34:38.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.488+0000 7fbbeffff700 1 -- 172.21.15.89:0/1640689804 learned_addr learned my addr 172.21.15.89:0/1640689804 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:38.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.489+0000 7fbbf6d9d700 1 --2- 172.21.15.89:0/1640689804 >> v2:172.21.15.111:3300/0 conn(0x7fbbf80c91e0 0x7fbbf8160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.489+0000 7fbbf659c700 1 -- 172.21.15.89:0/1640689804 >> v2:172.21.15.111:3300/0 conn(0x7fbbf80c91e0 msgr2=0x7fbbf8160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.489+0000 7fbbf659c700 1 --2- 172.21.15.89:0/1640689804 >> v2:172.21.15.111:3300/0 conn(0x7fbbf80c91e0 0x7fbbf8160390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.489+0000 7fbbf659c700 1 -- 172.21.15.89:0/1640689804 >> v1:172.21.15.89:6789/0 conn(0x7fbbf806bac0 legacy=0x7fbbf806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.489+0000 7fbbf659c700 1 -- 172.21.15.89:0/1640689804 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf80ccab0 con 0x7fbbf806aa40 2023-10-10T20:34:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.489+0000 7fbbf5d9b700 1 -- 172.21.15.89:0/1640689804 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3052067471 0 0) 0x7fbbf80cc210 con 0x7fbbf806bac0 2023-10-10T20:34:38.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.489+0000 7fbbf659c700 1 --2- 172.21.15.89:0/1640689804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf806aa40 0x7fbbf80cbc00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbbf806e510 tx=0x7fbbe8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72123dde4f1c135 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:38.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.489+0000 7fbbf5d9b700 1 -- 172.21.15.89:0/1640689804 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8005c10 con 0x7fbbf806aa40 2023-10-10T20:34:38.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.489+0000 7fbbf5d9b700 1 -- 172.21.15.89:0/1640689804 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe8005d70 con 0x7fbbf806aa40 2023-10-10T20:34:38.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.489+0000 7fbbf5d9b700 1 -- 172.21.15.89:0/1640689804 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8006260 con 0x7fbbf806aa40 2023-10-10T20:34:38.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.489+0000 7fbbfca0a700 1 -- 172.21.15.89:0/1640689804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf806aa40 msgr2=0x7fbbf80cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:38.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.489+0000 7fbbfca0a700 1 --2- 172.21.15.89:0/1640689804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf806aa40 0x7fbbf80cbc00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbbf806e510 tx=0x7fbbe8009120 comp rx=0 tx=0).stop 2023-10-10T20:34:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.489+0000 7fbbfca0a700 1 -- 172.21.15.89:0/1640689804 shutdown_connections 2023-10-10T20:34:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.489+0000 7fbbfca0a700 1 --2- 172.21.15.89:0/1640689804 >> v2:172.21.15.111:3300/0 conn(0x7fbbf80c91e0 0x7fbbf8160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.489+0000 7fbbfca0a700 1 --2- 172.21.15.89:0/1640689804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf806aa40 0x7fbbf80cbc00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.489+0000 7fbbfca0a700 1 -- 172.21.15.89:0/1640689804 >> 172.21.15.89:0/1640689804 conn(0x7fbbf80cd670 msgr2=0x7fbbf80cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.490+0000 7fbbfca0a700 1 -- 172.21.15.89:0/1640689804 shutdown_connections 2023-10-10T20:34:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.490+0000 7fbbfca0a700 1 -- 172.21.15.89:0/1640689804 wait complete. 2023-10-10T20:34:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.491+0000 7fbbfca0a700 1 Processor -- start 2023-10-10T20:34:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.491+0000 7fbbfca0a700 1 -- start start 2023-10-10T20:34:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.491+0000 7fbbfca0a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf806aa40 0x7fbbf809a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.491+0000 7fbbfca0a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf806bac0 0x7fbbf80c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.491+0000 7fbbfca0a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbf80c91e0 0x7fbbf8068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.491+0000 7fbbfca0a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf80c7ad0 con 0x7fbbf806aa40 2023-10-10T20:34:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.491+0000 7fbbfca0a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf816bcc0 con 0x7fbbf80c91e0 2023-10-10T20:34:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.491+0000 7fbbfca0a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf80c77f0 con 0x7fbbf806bac0 2023-10-10T20:34:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.491+0000 7fbbf659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf806aa40 0x7fbbf809a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.491+0000 7fbbf659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf806aa40 0x7fbbf809a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53182/0 (socket says 172.21.15.89:53182) 2023-10-10T20:34:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.491+0000 7fbbf659c700 1 -- 172.21.15.89:0/992814497 learned_addr learned my addr 172.21.15.89:0/992814497 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.492+0000 7fbbf659c700 1 -- 172.21.15.89:0/992814497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf806bac0 msgr2=0x7fbbf80c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.492+0000 7fbbf659c700 1 --2- 172.21.15.89:0/992814497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf806bac0 0x7fbbf80c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.492+0000 7fbbf659c700 1 -- 172.21.15.89:0/992814497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbf80c91e0 msgr2=0x7fbbf8068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.492+0000 7fbbf659c700 1 --2- 172.21.15.89:0/992814497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbf80c91e0 0x7fbbf8068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.492+0000 7fbbf659c700 1 -- 172.21.15.89:0/992814497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe80093c0 con 0x7fbbf806aa40 2023-10-10T20:34:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.492+0000 7fbbf659c700 1 --2- 172.21.15.89:0/992814497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf806aa40 0x7fbbf809a980 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbbf806c6a0 tx=0x7fbbe8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.492+0000 7fbbee7fc700 1 -- 172.21.15.89:0/992814497 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe800bcf0 con 0x7fbbf806aa40 2023-10-10T20:34:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.492+0000 7fbbee7fc700 1 -- 172.21.15.89:0/992814497 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe800be50 con 0x7fbbf806aa40 2023-10-10T20:34:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.492+0000 7fbbfca0a700 1 -- 172.21.15.89:0/992814497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf816b670 con 0x7fbbf806aa40 2023-10-10T20:34:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.492+0000 7fbbee7fc700 1 -- 172.21.15.89:0/992814497 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe801b6d0 con 0x7fbbf806aa40 2023-10-10T20:34:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.492+0000 7fbbfca0a700 1 -- 172.21.15.89:0/992814497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf809aec0 con 0x7fbbf806aa40 2023-10-10T20:34:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.493+0000 7fbbfca0a700 1 -- 172.21.15.89:0/992814497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf816b4d0 con 0x7fbbf806aa40 2023-10-10T20:34:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.493+0000 7fbbee7fc700 1 -- 172.21.15.89:0/992814497 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbbe801ba80 con 0x7fbbf806aa40 2023-10-10T20:34:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.493+0000 7fbbee7fc700 1 --2- 172.21.15.89:0/992814497 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbbd8066f30 0x7fbbd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.494+0000 7fbbee7fc700 1 -- 172.21.15.89:0/992814497 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbbe8086940 con 0x7fbbf806aa40 2023-10-10T20:34:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.494+0000 7fbbeffff700 1 --2- 172.21.15.89:0/992814497 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbbd8066f30 0x7fbbd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:38.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.494+0000 7fbbeffff700 1 --2- 172.21.15.89:0/992814497 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbbd8066f30 0x7fbbd80693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00011e0 tx=0x7fbbe00079d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:38.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.498+0000 7fbbee7fc700 1 -- 172.21.15.89:0/992814497 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbbf816b4d0 con 0x7fbbf806aa40 2023-10-10T20:34:38.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.723+0000 7fbbfca0a700 1 -- 172.21.15.89:0/992814497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbf816b4d0 con 0x7fbbf806aa40 2023-10-10T20:34:38.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.724+0000 7fbbee7fc700 1 -- 172.21.15.89:0/992814497 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4597 (secure 0 0 0) 0x7fbbf816b4d0 con 0x7fbbf806aa40 2023-10-10T20:34:38.727 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:38.728 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2341577144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2341577144},{"type":"v1","addr":"172.21.15.111:6835","nonce":2341577144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2773123076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2773123076},{"type":"v1","addr":"172.21.15.89:6837","nonce":2773123076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2940044414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2940044414},{"type":"v1","addr":"172.21.15.89:6835","nonce":2940044414}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:34:34.717750+0000","modified":"2023-10-10T20:34:38.356048+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1112081320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1112081320},{"type":"v1","addr":"172.21.15.111:6834","nonce":1112081320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-10T20:34:38.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.726+0000 7fbbfca0a700 1 -- 172.21.15.89:0/992814497 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbbd8066f30 msgr2=0x7fbbd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:38.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.726+0000 7fbbfca0a700 1 --2- 172.21.15.89:0/992814497 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbbd8066f30 0x7fbbd80693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00011e0 tx=0x7fbbe00079d0 comp rx=0 tx=0).stop 2023-10-10T20:34:38.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.726+0000 7fbbfca0a700 1 -- 172.21.15.89:0/992814497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf806aa40 msgr2=0x7fbbf809a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:38.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.726+0000 7fbbfca0a700 1 --2- 172.21.15.89:0/992814497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf806aa40 0x7fbbf809a980 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbbf806c6a0 tx=0x7fbbe8016000 comp rx=0 tx=0).stop 2023-10-10T20:34:38.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.726+0000 7fbbfca0a700 1 -- 172.21.15.89:0/992814497 shutdown_connections 2023-10-10T20:34:38.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.726+0000 7fbbfca0a700 1 --2- 172.21.15.89:0/992814497 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbbd8066f30 0x7fbbd80693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:38.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.726+0000 7fbbfca0a700 1 --2- 172.21.15.89:0/992814497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbf80c91e0 0x7fbbf8068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:38.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.726+0000 7fbbfca0a700 1 --2- 172.21.15.89:0/992814497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf806aa40 0x7fbbf809a980 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:38.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.726+0000 7fbbfca0a700 1 --2- 172.21.15.89:0/992814497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbf806bac0 0x7fbbf80c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:38.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.726+0000 7fbbfca0a700 1 -- 172.21.15.89:0/992814497 >> 172.21.15.89:0/992814497 conn(0x7fbbf80cd670 msgr2=0x7fbbf8153790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:38.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.726+0000 7fbbfca0a700 1 -- 172.21.15.89:0/992814497 shutdown_connections 2023-10-10T20:34:38.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.726+0000 7fbbfca0a700 1 -- 172.21.15.89:0/992814497 wait complete. 2023-10-10T20:34:38.732 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2023-10-10T20:34:38.741 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:34:38.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.852+0000 7f66c5a95700 1 Processor -- start 2023-10-10T20:34:38.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.852+0000 7f66c5a95700 1 -- start start 2023-10-10T20:34:38.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.853+0000 7f66c5a95700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f66c0159890 0x7f66c015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:38.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.853+0000 7f66c5a95700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f66c0159c60 0x7f66c015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:38.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.853+0000 7f66c5a95700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c015a160 con 0x7f66c015db10 2023-10-10T20:34:38.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.853+0000 7f66c5a95700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c015df50 con 0x7f66c0159890 2023-10-10T20:34:38.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.853+0000 7f66c5a95700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c00d13a0 con 0x7f66c0159c60 2023-10-10T20:34:38.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.853+0000 7f66bf7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f66c0159890 0x7f66c015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:38.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.853+0000 7f66bf7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f66c0159890 0x7f66c015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53184/0 (socket says 172.21.15.89:53184) 2023-10-10T20:34:38.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.853+0000 7f66bf7fe700 1 -- 172.21.15.89:0/4270571728 learned_addr learned my addr 172.21.15.89:0/4270571728 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:38.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.853+0000 7f66beffd700 1 --2- 172.21.15.89:0/4270571728 >> v2:172.21.15.111:3300/0 conn(0x7f66c0159c60 0x7f66c015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:38.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.854+0000 7f66bf7fe700 1 -- 172.21.15.89:0/4270571728 >> v2:172.21.15.111:3300/0 conn(0x7f66c0159c60 msgr2=0x7f66c015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:38.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.854+0000 7f66bf7fe700 1 --2- 172.21.15.89:0/4270571728 >> v2:172.21.15.111:3300/0 conn(0x7f66c0159c60 0x7f66c015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:38.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.854+0000 7f66bf7fe700 1 -- 172.21.15.89:0/4270571728 >> v1:172.21.15.111:6789/0 conn(0x7f66c015db10 legacy=0x7f66c0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:38.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.854+0000 7f66bf7fe700 1 -- 172.21.15.89:0/4270571728 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c015e140 con 0x7f66c0159890 2023-10-10T20:34:38.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.854+0000 7f66bf7fe700 1 --2- 172.21.15.89:0/4270571728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c0159890 0x7f66c015abd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66c00b67f0 tx=0x7f66b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb760c23786e77a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:38.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.854+0000 7f66be7fc700 1 -- 172.21.15.89:0/4270571728 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b000b820 con 0x7f66c0159890 2023-10-10T20:34:38.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.854+0000 7f66be7fc700 1 -- 172.21.15.89:0/4270571728 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b000b980 con 0x7f66c0159890 2023-10-10T20:34:38.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.854+0000 7f66be7fc700 1 -- 172.21.15.89:0/4270571728 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b000bd20 con 0x7f66c0159890 2023-10-10T20:34:38.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.854+0000 7f66c5a95700 1 -- 172.21.15.89:0/4270571728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c0159890 msgr2=0x7f66c015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:38.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.854+0000 7f66c5a95700 1 --2- 172.21.15.89:0/4270571728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c0159890 0x7f66c015abd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66c00b67f0 tx=0x7f66b0009250 comp rx=0 tx=0).stop 2023-10-10T20:34:38.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.854+0000 7f66c5a95700 1 -- 172.21.15.89:0/4270571728 shutdown_connections 2023-10-10T20:34:38.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.854+0000 7f66c5a95700 1 --2- 172.21.15.89:0/4270571728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c0159890 0x7f66c015abd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:38.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.854+0000 7f66c5a95700 1 --2- 172.21.15.89:0/4270571728 >> v2:172.21.15.111:3300/0 conn(0x7f66c0159c60 0x7f66c015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:38.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.854+0000 7f66c5a95700 1 -- 172.21.15.89:0/4270571728 >> 172.21.15.89:0/4270571728 conn(0x7f66c0068e20 msgr2=0x7f66c00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:38.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.854+0000 7f66c5a95700 1 -- 172.21.15.89:0/4270571728 shutdown_connections 2023-10-10T20:34:38.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.854+0000 7f66c5a95700 1 -- 172.21.15.89:0/4270571728 wait complete. 2023-10-10T20:34:38.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.856+0000 7f66c5a95700 1 Processor -- start 2023-10-10T20:34:38.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.856+0000 7f66c5a95700 1 -- start start 2023-10-10T20:34:38.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.856+0000 7f66c5a95700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c0159890 0x7f66c00b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:38.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.856+0000 7f66c5a95700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f66c0159c60 0x7f66c00ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:38.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.856+0000 7f66c5a95700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66c015db10 0x7f66c0152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:38.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.856+0000 7f66c5a95700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c00caa70 con 0x7f66c0159890 2023-10-10T20:34:38.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.856+0000 7f66c5a95700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c0152dd0 con 0x7f66c015db10 2023-10-10T20:34:38.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.856+0000 7f66c5a95700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c00c7940 con 0x7f66c0159c60 2023-10-10T20:34:38.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.856+0000 7f66bf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c0159890 0x7f66c00b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:38.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.856+0000 7f66bf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c0159890 0x7f66c00b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53186/0 (socket says 172.21.15.89:53186) 2023-10-10T20:34:38.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.856+0000 7f66bf7fe700 1 -- 172.21.15.89:0/3012392668 learned_addr learned my addr 172.21.15.89:0/3012392668 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:38.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.857+0000 7f66beffd700 1 --2- 172.21.15.89:0/3012392668 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f66c0159c60 0x7f66c00ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:38.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.857+0000 7f66bffff700 1 --2- 172.21.15.89:0/3012392668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66c015db10 0x7f66c0152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:38.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.857+0000 7f66bf7fe700 1 -- 172.21.15.89:0/3012392668 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f66c0159c60 msgr2=0x7f66c00ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:38.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.857+0000 7f66bf7fe700 1 --2- 172.21.15.89:0/3012392668 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f66c0159c60 0x7f66c00ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:38.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.857+0000 7f66bf7fe700 1 -- 172.21.15.89:0/3012392668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66c015db10 msgr2=0x7f66c0152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:38.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.857+0000 7f66bf7fe700 1 --2- 172.21.15.89:0/3012392668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66c015db10 0x7f66c0152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:38.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.857+0000 7f66bf7fe700 1 -- 172.21.15.89:0/3012392668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b00094f0 con 0x7f66c0159890 2023-10-10T20:34:38.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.857+0000 7f66bf7fe700 1 --2- 172.21.15.89:0/3012392668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c0159890 0x7f66c00b4770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009290 tx=0x7f66b0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:38.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.857+0000 7f66a77fe700 1 -- 172.21.15.89:0/3012392668 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b0011e20 con 0x7f66c0159890 2023-10-10T20:34:38.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.857+0000 7f66a77fe700 1 -- 172.21.15.89:0/3012392668 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b0020070 con 0x7f66c0159890 2023-10-10T20:34:38.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.857+0000 7f66c5a95700 1 -- 172.21.15.89:0/3012392668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c00c7ec0 con 0x7f66c0159890 2023-10-10T20:34:38.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.857+0000 7f66a77fe700 1 -- 172.21.15.89:0/3012392668 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b001b6a0 con 0x7f66c0159890 2023-10-10T20:34:38.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.857+0000 7f66c5a95700 1 -- 172.21.15.89:0/3012392668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c00b5070 con 0x7f66c0159890 2023-10-10T20:34:38.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.858+0000 7f66c5a95700 1 -- 172.21.15.89:0/3012392668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c004c3c0 con 0x7f66c0159890 2023-10-10T20:34:38.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.858+0000 7f66a77fe700 1 -- 172.21.15.89:0/3012392668 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f66b0022030 con 0x7f66c0159890 2023-10-10T20:34:38.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.859+0000 7f66a77fe700 1 --2- 172.21.15.89:0/3012392668 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f66ac066f30 0x7f66ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:38.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.859+0000 7f66a77fe700 1 -- 172.21.15.89:0/3012392668 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f66b0086410 con 0x7f66c0159890 2023-10-10T20:34:38.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.859+0000 7f66beffd700 1 --2- 172.21.15.89:0/3012392668 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f66ac066f30 0x7f66ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:38.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.859+0000 7f66beffd700 1 --2- 172.21.15.89:0/3012392668 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f66ac066f30 0x7f66ac0693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f66c0153a50 tx=0x7f66b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:38.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:38.864+0000 7f66a77fe700 1 -- 172.21.15.89:0/3012392668 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f66b0053680 con 0x7f66c0159890 2023-10-10T20:34:39.047 INFO:tasks.ceph.mds.b.smithi111.stderr:2023-10-10T20:34:39.045+0000 7fe167616700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:34:39.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.045+0000 7f66c5a95700 1 -- 172.21.15.89:0/3012392668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66c004c3c0 con 0x7f66c0159890 2023-10-10T20:34:39.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.046+0000 7f66a77fe700 1 -- 172.21.15.89:0/3012392668 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f66b0056ca0 con 0x7f66c0159890 2023-10-10T20:34:39.050 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:39.050 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:36.324920+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:34:32.342167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:34:33.671763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:34:39.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.049+0000 7f66c5a95700 1 -- 172.21.15.89:0/3012392668 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f66ac066f30 msgr2=0x7f66ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:39.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.049+0000 7f66c5a95700 1 --2- 172.21.15.89:0/3012392668 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f66ac066f30 0x7f66ac0693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f66c0153a50 tx=0x7f66b4008040 comp rx=0 tx=0).stop 2023-10-10T20:34:39.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.049+0000 7f66c5a95700 1 -- 172.21.15.89:0/3012392668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c0159890 msgr2=0x7f66c00b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:39.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.049+0000 7f66c5a95700 1 --2- 172.21.15.89:0/3012392668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c0159890 0x7f66c00b4770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009290 tx=0x7f66b0016000 comp rx=0 tx=0).stop 2023-10-10T20:34:39.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.049+0000 7f66c5a95700 1 -- 172.21.15.89:0/3012392668 shutdown_connections 2023-10-10T20:34:39.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.049+0000 7f66c5a95700 1 --2- 172.21.15.89:0/3012392668 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f66ac066f30 0x7f66ac0693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.049+0000 7f66c5a95700 1 --2- 172.21.15.89:0/3012392668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66c015db10 0x7f66c0152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.049+0000 7f66c5a95700 1 --2- 172.21.15.89:0/3012392668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c0159890 0x7f66c00b4770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.049+0000 7f66c5a95700 1 --2- 172.21.15.89:0/3012392668 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f66c0159c60 0x7f66c00ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.049+0000 7f66c5a95700 1 -- 172.21.15.89:0/3012392668 >> 172.21.15.89:0/3012392668 conn(0x7f66c0068e20 msgr2=0x7f66c01530f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:39.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.049+0000 7f66c5a95700 1 -- 172.21.15.89:0/3012392668 shutdown_connections 2023-10-10T20:34:39.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.049+0000 7f66c5a95700 1 -- 172.21.15.89:0/3012392668 wait complete. 2023-10-10T20:34:39.064 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:34:39.064 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:34:39.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.176+0000 7f2734bb3700 1 Processor -- start 2023-10-10T20:34:39.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.176+0000 7f2734bb3700 1 -- start start 2023-10-10T20:34:39.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.177+0000 7f2734bb3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f27300af480 0x7f27300ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:39.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.177+0000 7f2734bb3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2730157e30 con 0x7f2730156a60 2023-10-10T20:34:39.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.177+0000 7f2734bb3700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27301578b0 con 0x7f2730156690 2023-10-10T20:34:39.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.177+0000 7f2734bb3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273006aa00 con 0x7f27300af480 2023-10-10T20:34:39.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.177+0000 7f272dd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f27300af480 0x7f27300ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:39.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.177+0000 7f272ed9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2730156a60 0x7f273006ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35622/0 (socket says 172.21.15.89:35622) 2023-10-10T20:34:39.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.177+0000 7f272ed9d700 1 -- 172.21.15.89:0/842630006 learned_addr learned my addr 172.21.15.89:0/842630006 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:39.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.177+0000 7f272dd9b700 1 -- 172.21.15.89:0/842630006 >> v1:172.21.15.111:6789/0 conn(0x7f2730156690 legacy=0x7f27300c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:39.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.177+0000 7f272dd9b700 1 -- 172.21.15.89:0/842630006 >> v1:172.21.15.89:6789/0 conn(0x7f2730156a60 legacy=0x7f273006ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:39.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.177+0000 7f272dd9b700 1 -- 172.21.15.89:0/842630006 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2730157b50 con 0x7f27300af480 2023-10-10T20:34:39.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.177+0000 7f272d59a700 1 -- 172.21.15.89:0/842630006 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977167808 0 0) 0x7f2730157e30 con 0x7f2730156a60 2023-10-10T20:34:39.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.177+0000 7f272dd9b700 1 --2- 172.21.15.89:0/842630006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27300af480 0x7f27300ca9d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f27180098e0 tx=0x7f27180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d42851a7f5768d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:39.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.177+0000 7f272d59a700 1 -- 172.21.15.89:0/842630006 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2718009e80 con 0x7f27300af480 2023-10-10T20:34:39.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.177+0000 7f272d59a700 1 -- 172.21.15.89:0/842630006 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271800b820 con 0x7f27300af480 2023-10-10T20:34:39.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.177+0000 7f272d59a700 1 -- 172.21.15.89:0/842630006 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271800bba0 con 0x7f27300af480 2023-10-10T20:34:39.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.178+0000 7f2734bb3700 1 -- 172.21.15.89:0/842630006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27300af480 msgr2=0x7f27300ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:39.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.178+0000 7f2734bb3700 1 --2- 172.21.15.89:0/842630006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27300af480 0x7f27300ca9d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f27180098e0 tx=0x7f27180091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.178+0000 7f2734bb3700 1 -- 172.21.15.89:0/842630006 shutdown_connections 2023-10-10T20:34:39.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.178+0000 7f2734bb3700 1 --2- 172.21.15.89:0/842630006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27300af480 0x7f27300ca9d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.178+0000 7f2734bb3700 1 -- 172.21.15.89:0/842630006 >> 172.21.15.89:0/842630006 conn(0x7f27300c9870 msgr2=0x7f27300c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:39.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.178+0000 7f2734bb3700 1 -- 172.21.15.89:0/842630006 shutdown_connections 2023-10-10T20:34:39.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.178+0000 7f2734bb3700 1 -- 172.21.15.89:0/842630006 wait complete. 2023-10-10T20:34:39.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.179+0000 7f2734bb3700 1 Processor -- start 2023-10-10T20:34:39.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.179+0000 7f2734bb3700 1 -- start start 2023-10-10T20:34:39.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f2734bb3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27300af480 0x7f27300c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:39.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f2734bb3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2730156690 0x7f2730153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:39.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f2734bb3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2730156a60 0x7f2730168240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:39.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f2734bb3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27300c7100 con 0x7f27300af480 2023-10-10T20:34:39.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f2734bb3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27300c4890 con 0x7f2730156a60 2023-10-10T20:34:39.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f2734bb3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27300c4c70 con 0x7f2730156690 2023-10-10T20:34:39.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f272e59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27300af480 0x7f27300c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:39.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f272e59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27300af480 0x7f27300c6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53192/0 (socket says 172.21.15.89:53192) 2023-10-10T20:34:39.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f272e59c700 1 -- 172.21.15.89:0/1425117701 learned_addr learned my addr 172.21.15.89:0/1425117701 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:39.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f272dd9b700 1 --2- 172.21.15.89:0/1425117701 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2730156690 0x7f2730153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:39.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f272ed9d700 1 --2- 172.21.15.89:0/1425117701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2730156a60 0x7f2730168240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:39.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f272e59c700 1 -- 172.21.15.89:0/1425117701 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2730156690 msgr2=0x7f2730153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:39.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f272e59c700 1 --2- 172.21.15.89:0/1425117701 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2730156690 0x7f2730153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f272e59c700 1 -- 172.21.15.89:0/1425117701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2730156a60 msgr2=0x7f2730168240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:39.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f272e59c700 1 --2- 172.21.15.89:0/1425117701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2730156a60 0x7f2730168240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f272e59c700 1 -- 172.21.15.89:0/1425117701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2718009420 con 0x7f27300af480 2023-10-10T20:34:39.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f272e59c700 1 --2- 172.21.15.89:0/1425117701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27300af480 0x7f27300c6bc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f272000c950 tx=0x7f2720002760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:39.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f27267fc700 1 -- 172.21.15.89:0/1425117701 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272000a7e0 con 0x7f27300af480 2023-10-10T20:34:39.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f27267fc700 1 -- 172.21.15.89:0/1425117701 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272000a940 con 0x7f27300af480 2023-10-10T20:34:39.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f2734bb3700 1 -- 172.21.15.89:0/1425117701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27300c42d0 con 0x7f27300af480 2023-10-10T20:34:39.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f27267fc700 1 -- 172.21.15.89:0/1425117701 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272000ad00 con 0x7f27300af480 2023-10-10T20:34:39.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.180+0000 7f2734bb3700 1 -- 172.21.15.89:0/1425117701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27300c6900 con 0x7f27300af480 2023-10-10T20:34:39.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.181+0000 7f2734bb3700 1 -- 172.21.15.89:0/1425117701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27300cc570 con 0x7f27300af480 2023-10-10T20:34:39.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.182+0000 7f27267fc700 1 -- 172.21.15.89:0/1425117701 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2720012630 con 0x7f27300af480 2023-10-10T20:34:39.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.182+0000 7f27267fc700 1 --2- 172.21.15.89:0/1425117701 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2710066f30 0x7f27100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:39.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.182+0000 7f27267fc700 1 -- 172.21.15.89:0/1425117701 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f27200849a0 con 0x7f27300af480 2023-10-10T20:34:39.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.182+0000 7f272dd9b700 1 --2- 172.21.15.89:0/1425117701 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2710066f30 0x7f27100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:39.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.183+0000 7f272dd9b700 1 --2- 172.21.15.89:0/1425117701 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2710066f30 0x7f27100693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27180030c0 tx=0x7f2718016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:39.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.187+0000 7f27267fc700 1 -- 172.21.15.89:0/1425117701 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2720051c10 con 0x7f27300af480 2023-10-10T20:34:39.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.367+0000 7f2734bb3700 1 -- 172.21.15.89:0/1425117701 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f273004c3c0 con 0x7f2710066f30 2023-10-10T20:34:39.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.375+0000 7f27267fc700 1 -- 172.21.15.89:0/1425117701 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264649 (secure 0 0 0) 0x7f273004c3c0 con 0x7f2710066f30 2023-10-10T20:34:39.381 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:39.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.379+0000 7f2734bb3700 1 -- 172.21.15.89:0/1425117701 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2710066f30 msgr2=0x7f27100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:39.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.379+0000 7f2734bb3700 1 --2- 172.21.15.89:0/1425117701 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2710066f30 0x7f27100693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27180030c0 tx=0x7f2718016000 comp rx=0 tx=0).stop 2023-10-10T20:34:39.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.379+0000 7f2734bb3700 1 -- 172.21.15.89:0/1425117701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27300af480 msgr2=0x7f27300c6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:39.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.379+0000 7f2734bb3700 1 --2- 172.21.15.89:0/1425117701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27300af480 0x7f27300c6bc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f272000c950 tx=0x7f2720002760 comp rx=0 tx=0).stop 2023-10-10T20:34:39.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.379+0000 7f2734bb3700 1 -- 172.21.15.89:0/1425117701 shutdown_connections 2023-10-10T20:34:39.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.379+0000 7f2734bb3700 1 --2- 172.21.15.89:0/1425117701 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2710066f30 0x7f27100693e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.379+0000 7f2734bb3700 1 --2- 172.21.15.89:0/1425117701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2730156a60 0x7f2730168240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.379+0000 7f2734bb3700 1 --2- 172.21.15.89:0/1425117701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27300af480 0x7f27300c6bc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.379+0000 7f2734bb3700 1 --2- 172.21.15.89:0/1425117701 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2730156690 0x7f2730153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.379+0000 7f2734bb3700 1 -- 172.21.15.89:0/1425117701 >> 172.21.15.89:0/1425117701 conn(0x7f27300c9870 msgr2=0x7f27301545b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:39.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.379+0000 7f2734bb3700 1 -- 172.21.15.89:0/1425117701 shutdown_connections 2023-10-10T20:34:39.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.379+0000 7f2734bb3700 1 -- 172.21.15.89:0/1425117701 wait complete. 2023-10-10T20:34:39.386 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:34:39.393 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-10-10T20:34:38.652691+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381124,"kb_used_data":2820,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593596,"statfs":{"total":773094113280,"available":770655842304,"internally_reserved":0,"allocated":2887680,"data_stored":1749619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.338046"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.347393+0000","last_change":"2023-10-10T20:34:35.344417+0000","last_active":"2023-10-10T20:34:35.347393+0000","last_peered":"2023-10-10T20:34:35.347393+0000","last_clean":"2023-10-10T20:34:35.347393+0000","last_became_active":"2023-10-10T20:34:35.344171+0000","last_became_peered":"2023-10-10T20:34:35.344171+0000","last_unstale":"2023-10-10T20:34:35.347393+0000","last_undegraded":"2023-10-10T20:34:35.347393+0000","last_fullsized":"2023-10-10T20:34:35.347393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:06.136941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350364+0000","last_change":"2023-10-10T20:34:35.344388+0000","last_active":"2023-10-10T20:34:35.350364+0000","last_peered":"2023-10-10T20:34:35.350364+0000","last_clean":"2023-10-10T20:34:35.350364+0000","last_became_active":"2023-10-10T20:34:35.344132+0000","last_became_peered":"2023-10-10T20:34:35.344132+0000","last_unstale":"2023-10-10T20:34:35.350364+0000","last_undegraded":"2023-10-10T20:34:35.350364+0000","last_fullsized":"2023-10-10T20:34:35.350364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:05:02.787176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.341464+0000","last_change":"2023-10-10T20:34:35.339021+0000","last_active":"2023-10-10T20:34:35.341464+0000","last_peered":"2023-10-10T20:34:35.341464+0000","last_clean":"2023-10-10T20:34:35.341464+0000","last_became_active":"2023-10-10T20:34:35.338713+0000","last_became_peered":"2023-10-10T20:34:35.338713+0000","last_unstale":"2023-10-10T20:34:35.341464+0000","last_undegraded":"2023-10-10T20:34:35.341464+0000","last_fullsized":"2023-10-10T20:34:35.341464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:12:13.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349539+0000","last_change":"2023-10-10T20:34:35.345674+0000","last_active":"2023-10-10T20:34:35.349539+0000","last_peered":"2023-10-10T20:34:35.349539+0000","last_clean":"2023-10-10T20:34:35.349539+0000","last_became_active":"2023-10-10T20:34:35.342947+0000","last_became_peered":"2023-10-10T20:34:35.342947+0000","last_unstale":"2023-10-10T20:34:35.349539+0000","last_undegraded":"2023-10-10T20:34:35.349539+0000","last_fullsized":"2023-10-10T20:34:35.349539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:06.136941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.331236+0000","last_change":"2023-10-10T20:34:35.330128+0000","last_active":"2023-10-10T20:34:35.331236+0000","last_peered":"2023-10-10T20:34:35.331236+0000","last_clean":"2023-10-10T20:34:35.331236+0000","last_became_active":"2023-10-10T20:34:35.329809+0000","last_became_peered":"2023-10-10T20:34:35.329809+0000","last_unstale":"2023-10-10T20:34:35.331236+0000","last_undegraded":"2023-10-10T20:34:35.331236+0000","last_fullsized":"2023-10-10T20:34:35.331236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:08.378701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.351515+0000","last_change":"2023-10-10T20:34:35.341852+0000","last_active":"2023-10-10T20:34:35.351515+0000","last_peered":"2023-10-10T20:34:35.351515+0000","last_clean":"2023-10-10T20:34:35.351515+0000","last_became_active":"2023-10-10T20:34:35.341590+0000","last_became_peered":"2023-10-10T20:34:35.341590+0000","last_unstale":"2023-10-10T20:34:35.351515+0000","last_undegraded":"2023-10-10T20:34:35.351515+0000","last_fullsized":"2023-10-10T20:34:35.351515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:38:40.522047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342438+0000","last_change":"2023-10-10T20:34:35.336268+0000","last_active":"2023-10-10T20:34:35.342438+0000","last_peered":"2023-10-10T20:34:35.342438+0000","last_clean":"2023-10-10T20:34:35.342438+0000","last_became_active":"2023-10-10T20:34:35.335108+0000","last_became_peered":"2023-10-10T20:34:35.335108+0000","last_unstale":"2023-10-10T20:34:35.342438+0000","last_undegraded":"2023-10-10T20:34:35.342438+0000","last_fullsized":"2023-10-10T20:34:35.342438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:36.295226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350260+0000","last_change":"2023-10-10T20:34:35.342130+0000","last_active":"2023-10-10T20:34:35.350260+0000","last_peered":"2023-10-10T20:34:35.350260+0000","last_clean":"2023-10-10T20:34:35.350260+0000","last_became_active":"2023-10-10T20:34:35.341794+0000","last_became_peered":"2023-10-10T20:34:35.341794+0000","last_unstale":"2023-10-10T20:34:35.350260+0000","last_undegraded":"2023-10-10T20:34:35.350260+0000","last_fullsized":"2023-10-10T20:34:35.350260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.347376+0000","last_change":"2023-10-10T20:34:35.342521+0000","last_active":"2023-10-10T20:34:35.347376+0000","last_peered":"2023-10-10T20:34:35.347376+0000","last_clean":"2023-10-10T20:34:35.347376+0000","last_became_active":"2023-10-10T20:34:35.342237+0000","last_became_peered":"2023-10-10T20:34:35.342237+0000","last_unstale":"2023-10-10T20:34:35.347376+0000","last_undegraded":"2023-10-10T20:34:35.347376+0000","last_fullsized":"2023-10-10T20:34:35.347376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:35.499006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348040+0000","last_change":"2023-10-10T20:34:35.344521+0000","last_active":"2023-10-10T20:34:35.348040+0000","last_peered":"2023-10-10T20:34:35.348040+0000","last_clean":"2023-10-10T20:34:35.348040+0000","last_became_active":"2023-10-10T20:34:35.344132+0000","last_became_peered":"2023-10-10T20:34:35.344132+0000","last_unstale":"2023-10-10T20:34:35.348040+0000","last_undegraded":"2023-10-10T20:34:35.348040+0000","last_fullsized":"2023-10-10T20:34:35.348040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:05:02.787176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.344573+0000","last_change":"2023-10-10T20:34:35.342152+0000","last_active":"2023-10-10T20:34:35.344573+0000","last_peered":"2023-10-10T20:34:35.344573+0000","last_clean":"2023-10-10T20:34:35.344573+0000","last_became_active":"2023-10-10T20:34:35.341883+0000","last_became_peered":"2023-10-10T20:34:35.341883+0000","last_unstale":"2023-10-10T20:34:35.344573+0000","last_undegraded":"2023-10-10T20:34:35.344573+0000","last_fullsized":"2023-10-10T20:34:35.344573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:10:13.711031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350167+0000","last_change":"2023-10-10T20:34:35.342940+0000","last_active":"2023-10-10T20:34:35.350167+0000","last_peered":"2023-10-10T20:34:35.350167+0000","last_clean":"2023-10-10T20:34:35.350167+0000","last_became_active":"2023-10-10T20:34:35.342645+0000","last_became_peered":"2023-10-10T20:34:35.342645+0000","last_unstale":"2023-10-10T20:34:35.350167+0000","last_undegraded":"2023-10-10T20:34:35.350167+0000","last_fullsized":"2023-10-10T20:34:35.350167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:48:12.831828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350468+0000","last_change":"2023-10-10T20:34:35.343591+0000","last_active":"2023-10-10T20:34:35.350468+0000","last_peered":"2023-10-10T20:34:35.350468+0000","last_clean":"2023-10-10T20:34:35.350468+0000","last_became_active":"2023-10-10T20:34:35.343310+0000","last_became_peered":"2023-10-10T20:34:35.343310+0000","last_unstale":"2023-10-10T20:34:35.350468+0000","last_undegraded":"2023-10-10T20:34:35.350468+0000","last_fullsized":"2023-10-10T20:34:35.350468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:24:50.729855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348660+0000","last_change":"2023-10-10T20:34:35.343316+0000","last_active":"2023-10-10T20:34:35.348660+0000","last_peered":"2023-10-10T20:34:35.348660+0000","last_clean":"2023-10-10T20:34:35.348660+0000","last_became_active":"2023-10-10T20:34:35.343015+0000","last_became_peered":"2023-10-10T20:34:35.343015+0000","last_unstale":"2023-10-10T20:34:35.348660+0000","last_undegraded":"2023-10-10T20:34:35.348660+0000","last_fullsized":"2023-10-10T20:34:35.348660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:05:02.787176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352727+0000","last_change":"2023-10-10T20:34:35.346542+0000","last_active":"2023-10-10T20:34:35.352727+0000","last_peered":"2023-10-10T20:34:35.352727+0000","last_clean":"2023-10-10T20:34:35.352727+0000","last_became_active":"2023-10-10T20:34:35.346323+0000","last_became_peered":"2023-10-10T20:34:35.346323+0000","last_unstale":"2023-10-10T20:34:35.352727+0000","last_undegraded":"2023-10-10T20:34:35.352727+0000","last_fullsized":"2023-10-10T20:34:35.352727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:11:35.427062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349566+0000","last_change":"2023-10-10T20:34:35.346445+0000","last_active":"2023-10-10T20:34:35.349566+0000","last_peered":"2023-10-10T20:34:35.349566+0000","last_clean":"2023-10-10T20:34:35.349566+0000","last_became_active":"2023-10-10T20:34:35.345449+0000","last_became_peered":"2023-10-10T20:34:35.345449+0000","last_unstale":"2023-10-10T20:34:35.349566+0000","last_undegraded":"2023-10-10T20:34:35.349566+0000","last_fullsized":"2023-10-10T20:34:35.349566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350860+0000","last_change":"2023-10-10T20:34:35.348398+0000","last_active":"2023-10-10T20:34:35.350860+0000","last_peered":"2023-10-10T20:34:35.350860+0000","last_clean":"2023-10-10T20:34:35.350860+0000","last_became_active":"2023-10-10T20:34:35.348110+0000","last_became_peered":"2023-10-10T20:34:35.348110+0000","last_unstale":"2023-10-10T20:34:35.350860+0000","last_undegraded":"2023-10-10T20:34:35.350860+0000","last_fullsized":"2023-10-10T20:34:35.350860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:36.295226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350234+0000","last_change":"2023-10-10T20:34:35.343071+0000","last_active":"2023-10-10T20:34:35.350234+0000","last_peered":"2023-10-10T20:34:35.350234+0000","last_clean":"2023-10-10T20:34:35.350234+0000","last_became_active":"2023-10-10T20:34:35.342740+0000","last_became_peered":"2023-10-10T20:34:35.342740+0000","last_unstale":"2023-10-10T20:34:35.350234+0000","last_undegraded":"2023-10-10T20:34:35.350234+0000","last_fullsized":"2023-10-10T20:34:35.350234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:25:34.953681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.331257+0000","last_change":"2023-10-10T20:34:35.330198+0000","last_active":"2023-10-10T20:34:35.331257+0000","last_peered":"2023-10-10T20:34:35.331257+0000","last_clean":"2023-10-10T20:34:35.331257+0000","last_became_active":"2023-10-10T20:34:35.329847+0000","last_became_peered":"2023-10-10T20:34:35.329847+0000","last_unstale":"2023-10-10T20:34:35.331257+0000","last_undegraded":"2023-10-10T20:34:35.331257+0000","last_fullsized":"2023-10-10T20:34:35.331257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:10.218128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362481+0000","last_change":"2023-10-10T20:34:34.340043+0000","last_active":"2023-10-10T20:34:35.362481+0000","last_peered":"2023-10-10T20:34:35.362481+0000","last_clean":"2023-10-10T20:34:35.362481+0000","last_became_active":"2023-10-10T20:34:34.339783+0000","last_became_peered":"2023-10-10T20:34:34.339783+0000","last_unstale":"2023-10-10T20:34:35.362481+0000","last_undegraded":"2023-10-10T20:34:35.362481+0000","last_fullsized":"2023-10-10T20:34:35.362481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:00:46.542353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350303+0000","last_change":"2023-10-10T20:34:35.342670+0000","last_active":"2023-10-10T20:34:35.350303+0000","last_peered":"2023-10-10T20:34:35.350303+0000","last_clean":"2023-10-10T20:34:35.350303+0000","last_became_active":"2023-10-10T20:34:35.342432+0000","last_became_peered":"2023-10-10T20:34:35.342432+0000","last_unstale":"2023-10-10T20:34:35.350303+0000","last_undegraded":"2023-10-10T20:34:35.350303+0000","last_fullsized":"2023-10-10T20:34:35.350303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:38:40.522047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362959+0000","last_change":"2023-10-10T20:34:34.343567+0000","last_active":"2023-10-10T20:34:35.362959+0000","last_peered":"2023-10-10T20:34:35.362959+0000","last_clean":"2023-10-10T20:34:35.362959+0000","last_became_active":"2023-10-10T20:34:34.343344+0000","last_became_peered":"2023-10-10T20:34:34.343344+0000","last_unstale":"2023-10-10T20:34:35.362959+0000","last_undegraded":"2023-10-10T20:34:35.362959+0000","last_fullsized":"2023-10-10T20:34:35.362959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:57:27.930567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348540+0000","last_change":"2023-10-10T20:34:35.341588+0000","last_active":"2023-10-10T20:34:35.348540+0000","last_peered":"2023-10-10T20:34:35.348540+0000","last_clean":"2023-10-10T20:34:35.348540+0000","last_became_active":"2023-10-10T20:34:35.341193+0000","last_became_peered":"2023-10-10T20:34:35.341193+0000","last_unstale":"2023-10-10T20:34:35.348540+0000","last_undegraded":"2023-10-10T20:34:35.348540+0000","last_fullsized":"2023-10-10T20:34:35.348540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:25:55.933179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.357498+0000","last_change":"2023-10-10T20:34:34.347611+0000","last_active":"2023-10-10T20:34:35.357498+0000","last_peered":"2023-10-10T20:34:35.357498+0000","last_clean":"2023-10-10T20:34:35.357498+0000","last_became_active":"2023-10-10T20:34:34.347335+0000","last_became_peered":"2023-10-10T20:34:34.347335+0000","last_unstale":"2023-10-10T20:34:35.357498+0000","last_undegraded":"2023-10-10T20:34:35.357498+0000","last_fullsized":"2023-10-10T20:34:35.357498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:11.616929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350859+0000","last_change":"2023-10-10T20:34:35.348359+0000","last_active":"2023-10-10T20:34:35.350859+0000","last_peered":"2023-10-10T20:34:35.350859+0000","last_clean":"2023-10-10T20:34:35.350859+0000","last_became_active":"2023-10-10T20:34:35.347861+0000","last_became_peered":"2023-10-10T20:34:35.347861+0000","last_unstale":"2023-10-10T20:34:35.350859+0000","last_undegraded":"2023-10-10T20:34:35.350859+0000","last_fullsized":"2023-10-10T20:34:35.350859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:39.183060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.356525+0000","last_change":"2023-10-10T20:34:34.344260+0000","last_active":"2023-10-10T20:34:35.356525+0000","last_peered":"2023-10-10T20:34:35.356525+0000","last_clean":"2023-10-10T20:34:35.356525+0000","last_became_active":"2023-10-10T20:34:34.343945+0000","last_became_peered":"2023-10-10T20:34:34.343945+0000","last_unstale":"2023-10-10T20:34:35.356525+0000","last_undegraded":"2023-10-10T20:34:35.356525+0000","last_fullsized":"2023-10-10T20:34:35.356525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352561+0000","last_change":"2023-10-10T20:34:35.348160+0000","last_active":"2023-10-10T20:34:35.352561+0000","last_peered":"2023-10-10T20:34:35.352561+0000","last_clean":"2023-10-10T20:34:35.352561+0000","last_became_active":"2023-10-10T20:34:35.347847+0000","last_became_peered":"2023-10-10T20:34:35.347847+0000","last_unstale":"2023-10-10T20:34:35.352561+0000","last_undegraded":"2023-10-10T20:34:35.352561+0000","last_fullsized":"2023-10-10T20:34:35.352561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:08.378701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.361981+0000","last_change":"2023-10-10T20:34:34.339973+0000","last_active":"2023-10-10T20:34:35.361981+0000","last_peered":"2023-10-10T20:34:35.361981+0000","last_clean":"2023-10-10T20:34:35.361981+0000","last_became_active":"2023-10-10T20:34:34.339685+0000","last_became_peered":"2023-10-10T20:34:34.339685+0000","last_unstale":"2023-10-10T20:34:35.361981+0000","last_undegraded":"2023-10-10T20:34:35.361981+0000","last_fullsized":"2023-10-10T20:34:35.361981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:34:00.263416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348564+0000","last_change":"2023-10-10T20:34:35.340064+0000","last_active":"2023-10-10T20:34:35.348564+0000","last_peered":"2023-10-10T20:34:35.348564+0000","last_clean":"2023-10-10T20:34:35.348564+0000","last_became_active":"2023-10-10T20:34:35.339811+0000","last_became_peered":"2023-10-10T20:34:35.339811+0000","last_unstale":"2023-10-10T20:34:35.348564+0000","last_undegraded":"2023-10-10T20:34:35.348564+0000","last_fullsized":"2023-10-10T20:34:35.348564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:17.774898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.357351+0000","last_change":"2023-10-10T20:34:34.350170+0000","last_active":"2023-10-10T20:34:35.357351+0000","last_peered":"2023-10-10T20:34:35.357351+0000","last_clean":"2023-10-10T20:34:35.357351+0000","last_became_active":"2023-10-10T20:34:34.349858+0000","last_became_peered":"2023-10-10T20:34:34.349858+0000","last_unstale":"2023-10-10T20:34:35.357351+0000","last_undegraded":"2023-10-10T20:34:35.357351+0000","last_fullsized":"2023-10-10T20:34:35.357351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.351173+0000","last_change":"2023-10-10T20:34:35.342458+0000","last_active":"2023-10-10T20:34:35.351173+0000","last_peered":"2023-10-10T20:34:35.351173+0000","last_clean":"2023-10-10T20:34:35.351173+0000","last_became_active":"2023-10-10T20:34:35.342063+0000","last_became_peered":"2023-10-10T20:34:35.342063+0000","last_unstale":"2023-10-10T20:34:35.351173+0000","last_undegraded":"2023-10-10T20:34:35.351173+0000","last_fullsized":"2023-10-10T20:34:35.351173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:18.709140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.334160+0000","last_change":"2023-10-10T20:34:34.344734+0000","last_active":"2023-10-10T20:34:35.334160+0000","last_peered":"2023-10-10T20:34:35.334160+0000","last_clean":"2023-10-10T20:34:35.334160+0000","last_became_active":"2023-10-10T20:34:34.344418+0000","last_became_peered":"2023-10-10T20:34:34.344418+0000","last_unstale":"2023-10-10T20:34:35.334160+0000","last_undegraded":"2023-10-10T20:34:35.334160+0000","last_fullsized":"2023-10-10T20:34:35.334160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:27:04.988955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350422+0000","last_change":"2023-10-10T20:34:35.343560+0000","last_active":"2023-10-10T20:34:35.350422+0000","last_peered":"2023-10-10T20:34:35.350422+0000","last_clean":"2023-10-10T20:34:35.350422+0000","last_became_active":"2023-10-10T20:34:35.343170+0000","last_became_peered":"2023-10-10T20:34:35.343170+0000","last_unstale":"2023-10-10T20:34:35.350422+0000","last_undegraded":"2023-10-10T20:34:35.350422+0000","last_fullsized":"2023-10-10T20:34:35.350422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:05:02.787176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.334617+0000","last_change":"2023-10-10T20:34:34.339163+0000","last_active":"2023-10-10T20:34:35.334617+0000","last_peered":"2023-10-10T20:34:35.334617+0000","last_clean":"2023-10-10T20:34:35.334617+0000","last_became_active":"2023-10-10T20:34:34.338879+0000","last_became_peered":"2023-10-10T20:34:34.338879+0000","last_unstale":"2023-10-10T20:34:35.334617+0000","last_undegraded":"2023-10-10T20:34:35.334617+0000","last_fullsized":"2023-10-10T20:34:35.334617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:40:03.331998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342534+0000","last_change":"2023-10-10T20:34:35.340814+0000","last_active":"2023-10-10T20:34:35.342534+0000","last_peered":"2023-10-10T20:34:35.342534+0000","last_clean":"2023-10-10T20:34:35.342534+0000","last_became_active":"2023-10-10T20:34:35.340139+0000","last_became_peered":"2023-10-10T20:34:35.340139+0000","last_unstale":"2023-10-10T20:34:35.342534+0000","last_undegraded":"2023-10-10T20:34:35.342534+0000","last_fullsized":"2023-10-10T20:34:35.342534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:47:45.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.336472+0000","last_change":"2023-10-10T20:34:34.348036+0000","last_active":"2023-10-10T20:34:35.336472+0000","last_peered":"2023-10-10T20:34:35.336472+0000","last_clean":"2023-10-10T20:34:35.336472+0000","last_became_active":"2023-10-10T20:34:34.347739+0000","last_became_peered":"2023-10-10T20:34:34.347739+0000","last_unstale":"2023-10-10T20:34:35.336472+0000","last_undegraded":"2023-10-10T20:34:35.336472+0000","last_fullsized":"2023-10-10T20:34:35.336472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:27:04.988955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.354125+0000","last_change":"2023-10-10T20:34:35.352621+0000","last_active":"2023-10-10T20:34:35.354125+0000","last_peered":"2023-10-10T20:34:35.354125+0000","last_clean":"2023-10-10T20:34:35.354125+0000","last_became_active":"2023-10-10T20:34:35.352324+0000","last_became_peered":"2023-10-10T20:34:35.352324+0000","last_unstale":"2023-10-10T20:34:35.354125+0000","last_undegraded":"2023-10-10T20:34:35.354125+0000","last_fullsized":"2023-10-10T20:34:35.354125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:24:50.729855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.334195+0000","last_change":"2023-10-10T20:34:34.344822+0000","last_active":"2023-10-10T20:34:35.334195+0000","last_peered":"2023-10-10T20:34:35.334195+0000","last_clean":"2023-10-10T20:34:35.334195+0000","last_became_active":"2023-10-10T20:34:34.344549+0000","last_became_peered":"2023-10-10T20:34:34.344549+0000","last_unstale":"2023-10-10T20:34:35.334195+0000","last_undegraded":"2023-10-10T20:34:35.334195+0000","last_fullsized":"2023-10-10T20:34:35.334195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.344601+0000","last_change":"2023-10-10T20:34:35.339793+0000","last_active":"2023-10-10T20:34:35.344601+0000","last_peered":"2023-10-10T20:34:35.344601+0000","last_clean":"2023-10-10T20:34:35.344601+0000","last_became_active":"2023-10-10T20:34:35.339226+0000","last_became_peered":"2023-10-10T20:34:35.339226+0000","last_unstale":"2023-10-10T20:34:35.344601+0000","last_undegraded":"2023-10-10T20:34:35.344601+0000","last_fullsized":"2023-10-10T20:34:35.344601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:10:13.711031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.360889+0000","last_change":"2023-10-10T20:34:34.336051+0000","last_active":"2023-10-10T20:34:35.360889+0000","last_peered":"2023-10-10T20:34:35.360889+0000","last_clean":"2023-10-10T20:34:35.360889+0000","last_became_active":"2023-10-10T20:34:34.335689+0000","last_became_peered":"2023-10-10T20:34:34.335689+0000","last_unstale":"2023-10-10T20:34:35.360889+0000","last_undegraded":"2023-10-10T20:34:35.360889+0000","last_fullsized":"2023-10-10T20:34:35.360889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:11.616929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350147+0000","last_change":"2023-10-10T20:34:35.341270+0000","last_active":"2023-10-10T20:34:35.350147+0000","last_peered":"2023-10-10T20:34:35.350147+0000","last_clean":"2023-10-10T20:34:35.350147+0000","last_became_active":"2023-10-10T20:34:35.341024+0000","last_became_peered":"2023-10-10T20:34:35.341024+0000","last_unstale":"2023-10-10T20:34:35.350147+0000","last_undegraded":"2023-10-10T20:34:35.350147+0000","last_fullsized":"2023-10-10T20:34:35.350147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:16.035453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.367035+0000","last_change":"2023-10-10T20:34:34.346412+0000","last_active":"2023-10-10T20:34:35.367035+0000","last_peered":"2023-10-10T20:34:35.367035+0000","last_clean":"2023-10-10T20:34:35.367035+0000","last_became_active":"2023-10-10T20:34:34.346020+0000","last_became_peered":"2023-10-10T20:34:34.346020+0000","last_unstale":"2023-10-10T20:34:35.367035+0000","last_undegraded":"2023-10-10T20:34:35.367035+0000","last_fullsized":"2023-10-10T20:34:35.367035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:15.030808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.345530+0000","last_change":"2023-10-10T20:34:35.340255+0000","last_active":"2023-10-10T20:34:35.345530+0000","last_peered":"2023-10-10T20:34:35.345530+0000","last_clean":"2023-10-10T20:34:35.345530+0000","last_became_active":"2023-10-10T20:34:35.339971+0000","last_became_peered":"2023-10-10T20:34:35.339971+0000","last_unstale":"2023-10-10T20:34:35.345530+0000","last_undegraded":"2023-10-10T20:34:35.345530+0000","last_fullsized":"2023-10-10T20:34:35.345530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:44:40.877955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362334+0000","last_change":"2023-10-10T20:34:34.343026+0000","last_active":"2023-10-10T20:34:35.362334+0000","last_peered":"2023-10-10T20:34:35.362334+0000","last_clean":"2023-10-10T20:34:35.362334+0000","last_became_active":"2023-10-10T20:34:34.342742+0000","last_became_peered":"2023-10-10T20:34:34.342742+0000","last_unstale":"2023-10-10T20:34:35.362334+0000","last_undegraded":"2023-10-10T20:34:35.362334+0000","last_fullsized":"2023-10-10T20:34:35.362334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:57:27.930567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348647+0000","last_change":"2023-10-10T20:34:35.342407+0000","last_active":"2023-10-10T20:34:35.348647+0000","last_peered":"2023-10-10T20:34:35.348647+0000","last_clean":"2023-10-10T20:34:35.348647+0000","last_became_active":"2023-10-10T20:34:35.341971+0000","last_became_peered":"2023-10-10T20:34:35.341971+0000","last_unstale":"2023-10-10T20:34:35.348647+0000","last_undegraded":"2023-10-10T20:34:35.348647+0000","last_fullsized":"2023-10-10T20:34:35.348647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:27:10.934315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.336796+0000","last_change":"2023-10-10T20:34:34.339893+0000","last_active":"2023-10-10T20:34:35.336796+0000","last_peered":"2023-10-10T20:34:35.336796+0000","last_clean":"2023-10-10T20:34:35.336796+0000","last_became_active":"2023-10-10T20:34:34.339585+0000","last_became_peered":"2023-10-10T20:34:34.339585+0000","last_unstale":"2023-10-10T20:34:35.336796+0000","last_undegraded":"2023-10-10T20:34:35.336796+0000","last_fullsized":"2023-10-10T20:34:35.336796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:15.030808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.346782+0000","last_change":"2023-10-10T20:34:35.341978+0000","last_active":"2023-10-10T20:34:35.346782+0000","last_peered":"2023-10-10T20:34:35.346782+0000","last_clean":"2023-10-10T20:34:35.346782+0000","last_became_active":"2023-10-10T20:34:35.341703+0000","last_became_peered":"2023-10-10T20:34:35.341703+0000","last_unstale":"2023-10-10T20:34:35.346782+0000","last_undegraded":"2023-10-10T20:34:35.346782+0000","last_fullsized":"2023-10-10T20:34:35.346782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:38:40.522047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.335441+0000","last_change":"2023-10-10T20:34:34.350137+0000","last_active":"2023-10-10T20:34:35.335441+0000","last_peered":"2023-10-10T20:34:35.335441+0000","last_clean":"2023-10-10T20:34:35.335441+0000","last_became_active":"2023-10-10T20:34:34.349855+0000","last_became_peered":"2023-10-10T20:34:34.349855+0000","last_unstale":"2023-10-10T20:34:35.335441+0000","last_undegraded":"2023-10-10T20:34:35.335441+0000","last_fullsized":"2023-10-10T20:34:35.335441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.347963+0000","last_change":"2023-10-10T20:34:35.344256+0000","last_active":"2023-10-10T20:34:35.347963+0000","last_peered":"2023-10-10T20:34:35.347963+0000","last_clean":"2023-10-10T20:34:35.347963+0000","last_became_active":"2023-10-10T20:34:35.343626+0000","last_became_peered":"2023-10-10T20:34:35.343626+0000","last_unstale":"2023-10-10T20:34:35.347963+0000","last_undegraded":"2023-10-10T20:34:35.347963+0000","last_fullsized":"2023-10-10T20:34:35.347963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:12:13.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.361414+0000","last_change":"2023-10-10T20:34:34.337912+0000","last_active":"2023-10-10T20:34:35.361414+0000","last_peered":"2023-10-10T20:34:35.361414+0000","last_clean":"2023-10-10T20:34:35.361414+0000","last_became_active":"2023-10-10T20:34:34.337588+0000","last_became_peered":"2023-10-10T20:34:34.337588+0000","last_unstale":"2023-10-10T20:34:35.361414+0000","last_undegraded":"2023-10-10T20:34:35.361414+0000","last_fullsized":"2023-10-10T20:34:35.361414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:40:03.331998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342457+0000","last_change":"2023-10-10T20:34:35.337938+0000","last_active":"2023-10-10T20:34:35.342457+0000","last_peered":"2023-10-10T20:34:35.342457+0000","last_clean":"2023-10-10T20:34:35.342457+0000","last_became_active":"2023-10-10T20:34:35.337649+0000","last_became_peered":"2023-10-10T20:34:35.337649+0000","last_unstale":"2023-10-10T20:34:35.342457+0000","last_undegraded":"2023-10-10T20:34:35.342457+0000","last_fullsized":"2023-10-10T20:34:35.342457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362619+0000","last_change":"2023-10-10T20:34:34.344045+0000","last_active":"2023-10-10T20:34:35.362619+0000","last_peered":"2023-10-10T20:34:35.362619+0000","last_clean":"2023-10-10T20:34:35.362619+0000","last_became_active":"2023-10-10T20:34:34.343736+0000","last_became_peered":"2023-10-10T20:34:34.343736+0000","last_unstale":"2023-10-10T20:34:35.362619+0000","last_undegraded":"2023-10-10T20:34:35.362619+0000","last_fullsized":"2023-10-10T20:34:35.362619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:57:27.930567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350691+0000","last_change":"2023-10-10T20:34:35.343035+0000","last_active":"2023-10-10T20:34:35.350691+0000","last_peered":"2023-10-10T20:34:35.350691+0000","last_clean":"2023-10-10T20:34:35.350691+0000","last_became_active":"2023-10-10T20:34:35.342776+0000","last_became_peered":"2023-10-10T20:34:35.342776+0000","last_unstale":"2023-10-10T20:34:35.350691+0000","last_undegraded":"2023-10-10T20:34:35.350691+0000","last_fullsized":"2023-10-10T20:34:35.350691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:52:03.361718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.337288+0000","last_change":"2023-10-10T20:34:34.337961+0000","last_active":"2023-10-10T20:34:35.337288+0000","last_peered":"2023-10-10T20:34:35.337288+0000","last_clean":"2023-10-10T20:34:35.337288+0000","last_became_active":"2023-10-10T20:34:34.337627+0000","last_became_peered":"2023-10-10T20:34:34.337627+0000","last_unstale":"2023-10-10T20:34:35.337288+0000","last_undegraded":"2023-10-10T20:34:35.337288+0000","last_fullsized":"2023-10-10T20:34:35.337288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:34:00.263416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349588+0000","last_change":"2023-10-10T20:34:35.346121+0000","last_active":"2023-10-10T20:34:35.349588+0000","last_peered":"2023-10-10T20:34:35.349588+0000","last_clean":"2023-10-10T20:34:35.349588+0000","last_became_active":"2023-10-10T20:34:35.345847+0000","last_became_peered":"2023-10-10T20:34:35.345847+0000","last_unstale":"2023-10-10T20:34:35.349588+0000","last_undegraded":"2023-10-10T20:34:35.349588+0000","last_fullsized":"2023-10-10T20:34:35.349588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:10.218128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.336695+0000","last_change":"2023-10-10T20:34:34.336680+0000","last_active":"2023-10-10T20:34:35.336695+0000","last_peered":"2023-10-10T20:34:35.336695+0000","last_clean":"2023-10-10T20:34:35.336695+0000","last_became_active":"2023-10-10T20:34:34.335533+0000","last_became_peered":"2023-10-10T20:34:34.335533+0000","last_unstale":"2023-10-10T20:34:35.336695+0000","last_undegraded":"2023-10-10T20:34:35.336695+0000","last_fullsized":"2023-10-10T20:34:35.336695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349524+0000","last_change":"2023-10-10T20:34:35.342266+0000","last_active":"2023-10-10T20:34:35.349524+0000","last_peered":"2023-10-10T20:34:35.349524+0000","last_clean":"2023-10-10T20:34:35.349524+0000","last_became_active":"2023-10-10T20:34:35.341981+0000","last_became_peered":"2023-10-10T20:34:35.341981+0000","last_unstale":"2023-10-10T20:34:35.349524+0000","last_undegraded":"2023-10-10T20:34:35.349524+0000","last_fullsized":"2023-10-10T20:34:35.349524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:11.206264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.345007+0000","last_change":"2023-10-10T20:34:35.341780+0000","last_active":"2023-10-10T20:34:35.345007+0000","last_peered":"2023-10-10T20:34:35.345007+0000","last_clean":"2023-10-10T20:34:35.345007+0000","last_became_active":"2023-10-10T20:34:35.340539+0000","last_became_peered":"2023-10-10T20:34:35.340539+0000","last_unstale":"2023-10-10T20:34:35.345007+0000","last_undegraded":"2023-10-10T20:34:35.345007+0000","last_fullsized":"2023-10-10T20:34:35.345007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:12:13.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.328070+0000","last_change":"2023-10-10T20:34:34.340247+0000","last_active":"2023-10-10T20:34:35.328070+0000","last_peered":"2023-10-10T20:34:35.328070+0000","last_clean":"2023-10-10T20:34:35.328070+0000","last_became_active":"2023-10-10T20:34:34.339946+0000","last_became_peered":"2023-10-10T20:34:34.339946+0000","last_unstale":"2023-10-10T20:34:35.328070+0000","last_undegraded":"2023-10-10T20:34:35.328070+0000","last_fullsized":"2023-10-10T20:34:35.328070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350545+0000","last_change":"2023-10-10T20:34:35.344288+0000","last_active":"2023-10-10T20:34:35.350545+0000","last_peered":"2023-10-10T20:34:35.350545+0000","last_clean":"2023-10-10T20:34:35.350545+0000","last_became_active":"2023-10-10T20:34:35.343736+0000","last_became_peered":"2023-10-10T20:34:35.343736+0000","last_unstale":"2023-10-10T20:34:35.350545+0000","last_undegraded":"2023-10-10T20:34:35.350545+0000","last_fullsized":"2023-10-10T20:34:35.350545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:18.709140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.333336+0000","last_change":"2023-10-10T20:34:34.341870+0000","last_active":"2023-10-10T20:34:35.333336+0000","last_peered":"2023-10-10T20:34:35.333336+0000","last_clean":"2023-10-10T20:34:35.333336+0000","last_became_active":"2023-10-10T20:34:34.341567+0000","last_became_peered":"2023-10-10T20:34:34.341567+0000","last_unstale":"2023-10-10T20:34:35.333336+0000","last_undegraded":"2023-10-10T20:34:35.333336+0000","last_fullsized":"2023-10-10T20:34:35.333336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:11.616929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.327481+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:34:35.327481+0000","last_peered":"2023-10-10T20:34:35.327481+0000","last_clean":"2023-10-10T20:34:35.327481+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:34:35.327481+0000","last_undegraded":"2023-10-10T20:34:35.327481+0000","last_fullsized":"2023-10-10T20:34:35.327481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.344647+0000","last_change":"2023-10-10T20:34:35.340986+0000","last_active":"2023-10-10T20:34:35.344647+0000","last_peered":"2023-10-10T20:34:35.344647+0000","last_clean":"2023-10-10T20:34:35.344647+0000","last_became_active":"2023-10-10T20:34:35.339812+0000","last_became_peered":"2023-10-10T20:34:35.339812+0000","last_unstale":"2023-10-10T20:34:35.344647+0000","last_undegraded":"2023-10-10T20:34:35.344647+0000","last_fullsized":"2023-10-10T20:34:35.344647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:11:35.427062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.327681+0000","last_change":"2023-10-10T20:34:34.345132+0000","last_active":"2023-10-10T20:34:35.327681+0000","last_peered":"2023-10-10T20:34:35.327681+0000","last_clean":"2023-10-10T20:34:35.327681+0000","last_became_active":"2023-10-10T20:34:34.344843+0000","last_became_peered":"2023-10-10T20:34:34.344843+0000","last_unstale":"2023-10-10T20:34:35.327681+0000","last_undegraded":"2023-10-10T20:34:35.327681+0000","last_fullsized":"2023-10-10T20:34:35.327681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:15.030808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.328545+0000","last_change":"2023-10-10T20:34:34.346151+0000","last_active":"2023-10-10T20:34:35.328545+0000","last_peered":"2023-10-10T20:34:35.328545+0000","last_clean":"2023-10-10T20:34:35.328545+0000","last_became_active":"2023-10-10T20:34:34.345765+0000","last_became_peered":"2023-10-10T20:34:34.345765+0000","last_unstale":"2023-10-10T20:34:35.328545+0000","last_undegraded":"2023-10-10T20:34:35.328545+0000","last_fullsized":"2023-10-10T20:34:35.328545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:10.201619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342808+0000","last_change":"2023-10-10T20:34:35.340238+0000","last_active":"2023-10-10T20:34:35.342808+0000","last_peered":"2023-10-10T20:34:35.342808+0000","last_clean":"2023-10-10T20:34:35.342808+0000","last_became_active":"2023-10-10T20:34:35.339944+0000","last_became_peered":"2023-10-10T20:34:35.339944+0000","last_unstale":"2023-10-10T20:34:35.342808+0000","last_undegraded":"2023-10-10T20:34:35.342808+0000","last_fullsized":"2023-10-10T20:34:35.342808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.337406+0000","last_change":"2023-10-10T20:34:34.347897+0000","last_active":"2023-10-10T20:34:35.337406+0000","last_peered":"2023-10-10T20:34:35.337406+0000","last_clean":"2023-10-10T20:34:35.337406+0000","last_became_active":"2023-10-10T20:34:34.347575+0000","last_became_peered":"2023-10-10T20:34:34.347575+0000","last_unstale":"2023-10-10T20:34:35.337406+0000","last_undegraded":"2023-10-10T20:34:35.337406+0000","last_fullsized":"2023-10-10T20:34:35.337406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:15.030808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350573+0000","last_change":"2023-10-10T20:34:35.345207+0000","last_active":"2023-10-10T20:34:35.350573+0000","last_peered":"2023-10-10T20:34:35.350573+0000","last_clean":"2023-10-10T20:34:35.350573+0000","last_became_active":"2023-10-10T20:34:35.344809+0000","last_became_peered":"2023-10-10T20:34:35.344809+0000","last_unstale":"2023-10-10T20:34:35.350573+0000","last_undegraded":"2023-10-10T20:34:35.350573+0000","last_fullsized":"2023-10-10T20:34:35.350573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:36.295226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.358045+0000","last_change":"2023-10-10T20:34:34.348399+0000","last_active":"2023-10-10T20:34:35.358045+0000","last_peered":"2023-10-10T20:34:35.358045+0000","last_clean":"2023-10-10T20:34:35.358045+0000","last_became_active":"2023-10-10T20:34:34.348099+0000","last_became_peered":"2023-10-10T20:34:34.348099+0000","last_unstale":"2023-10-10T20:34:35.358045+0000","last_undegraded":"2023-10-10T20:34:35.358045+0000","last_fullsized":"2023-10-10T20:34:35.358045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:40:03.331998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.344533+0000","last_change":"2023-10-10T20:34:35.340459+0000","last_active":"2023-10-10T20:34:35.344533+0000","last_peered":"2023-10-10T20:34:35.344533+0000","last_clean":"2023-10-10T20:34:35.344533+0000","last_became_active":"2023-10-10T20:34:35.340176+0000","last_became_peered":"2023-10-10T20:34:35.340176+0000","last_unstale":"2023-10-10T20:34:35.344533+0000","last_undegraded":"2023-10-10T20:34:35.344533+0000","last_fullsized":"2023-10-10T20:34:35.344533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:48:12.831828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.327499+0000","last_change":"2023-10-10T20:34:34.346150+0000","last_active":"2023-10-10T20:34:35.327499+0000","last_peered":"2023-10-10T20:34:35.327499+0000","last_clean":"2023-10-10T20:34:35.327499+0000","last_became_active":"2023-10-10T20:34:34.345811+0000","last_became_peered":"2023-10-10T20:34:34.345811+0000","last_unstale":"2023-10-10T20:34:35.327499+0000","last_undegraded":"2023-10-10T20:34:35.327499+0000","last_fullsized":"2023-10-10T20:34:35.327499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:04.665843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350476+0000","last_change":"2023-10-10T20:34:35.345083+0000","last_active":"2023-10-10T20:34:35.350476+0000","last_peered":"2023-10-10T20:34:35.350476+0000","last_clean":"2023-10-10T20:34:35.350476+0000","last_became_active":"2023-10-10T20:34:35.343922+0000","last_became_peered":"2023-10-10T20:34:35.343922+0000","last_unstale":"2023-10-10T20:34:35.350476+0000","last_undegraded":"2023-10-10T20:34:35.350476+0000","last_fullsized":"2023-10-10T20:34:35.350476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:05.670488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.337189+0000","last_change":"2023-10-10T20:34:34.340327+0000","last_active":"2023-10-10T20:34:35.337189+0000","last_peered":"2023-10-10T20:34:35.337189+0000","last_clean":"2023-10-10T20:34:35.337189+0000","last_became_active":"2023-10-10T20:34:34.339737+0000","last_became_peered":"2023-10-10T20:34:34.339737+0000","last_unstale":"2023-10-10T20:34:35.337189+0000","last_undegraded":"2023-10-10T20:34:35.337189+0000","last_fullsized":"2023-10-10T20:34:35.337189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350500+0000","last_change":"2023-10-10T20:34:35.344039+0000","last_active":"2023-10-10T20:34:35.350500+0000","last_peered":"2023-10-10T20:34:35.350500+0000","last_clean":"2023-10-10T20:34:35.350500+0000","last_became_active":"2023-10-10T20:34:35.343789+0000","last_became_peered":"2023-10-10T20:34:35.343789+0000","last_unstale":"2023-10-10T20:34:35.350500+0000","last_undegraded":"2023-10-10T20:34:35.350500+0000","last_fullsized":"2023-10-10T20:34:35.350500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:11:35.427062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.337192+0000","last_change":"2023-10-10T20:34:34.340022+0000","last_active":"2023-10-10T20:34:35.337192+0000","last_peered":"2023-10-10T20:34:35.337192+0000","last_clean":"2023-10-10T20:34:35.337192+0000","last_became_active":"2023-10-10T20:34:34.339652+0000","last_became_peered":"2023-10-10T20:34:34.339652+0000","last_unstale":"2023-10-10T20:34:35.337192+0000","last_undegraded":"2023-10-10T20:34:35.337192+0000","last_fullsized":"2023-10-10T20:34:35.337192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350304+0000","last_change":"2023-10-10T20:34:35.342858+0000","last_active":"2023-10-10T20:34:35.350304+0000","last_peered":"2023-10-10T20:34:35.350304+0000","last_clean":"2023-10-10T20:34:35.350304+0000","last_became_active":"2023-10-10T20:34:35.341911+0000","last_became_peered":"2023-10-10T20:34:35.341911+0000","last_unstale":"2023-10-10T20:34:35.350304+0000","last_undegraded":"2023-10-10T20:34:35.350304+0000","last_fullsized":"2023-10-10T20:34:35.350304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:25:55.933179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.357218+0000","last_change":"2023-10-10T20:34:34.346661+0000","last_active":"2023-10-10T20:34:35.357218+0000","last_peered":"2023-10-10T20:34:35.357218+0000","last_clean":"2023-10-10T20:34:35.357218+0000","last_became_active":"2023-10-10T20:34:34.346384+0000","last_became_peered":"2023-10-10T20:34:34.346384+0000","last_unstale":"2023-10-10T20:34:35.357218+0000","last_undegraded":"2023-10-10T20:34:35.357218+0000","last_fullsized":"2023-10-10T20:34:35.357218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350769+0000","last_change":"2023-10-10T20:34:35.349300+0000","last_active":"2023-10-10T20:34:35.350769+0000","last_peered":"2023-10-10T20:34:35.350769+0000","last_clean":"2023-10-10T20:34:35.350769+0000","last_became_active":"2023-10-10T20:34:35.348997+0000","last_became_peered":"2023-10-10T20:34:35.348997+0000","last_unstale":"2023-10-10T20:34:35.350769+0000","last_undegraded":"2023-10-10T20:34:35.350769+0000","last_fullsized":"2023-10-10T20:34:35.350769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:47:45.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.328113+0000","last_change":"2023-10-10T20:34:34.344048+0000","last_active":"2023-10-10T20:34:35.328113+0000","last_peered":"2023-10-10T20:34:35.328113+0000","last_clean":"2023-10-10T20:34:35.328113+0000","last_became_active":"2023-10-10T20:34:34.343866+0000","last_became_peered":"2023-10-10T20:34:34.343866+0000","last_unstale":"2023-10-10T20:34:35.328113+0000","last_undegraded":"2023-10-10T20:34:35.328113+0000","last_fullsized":"2023-10-10T20:34:35.328113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:27:04.988955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342399+0000","last_change":"2023-10-10T20:34:35.337534+0000","last_active":"2023-10-10T20:34:35.342399+0000","last_peered":"2023-10-10T20:34:35.342399+0000","last_clean":"2023-10-10T20:34:35.342399+0000","last_became_active":"2023-10-10T20:34:35.337277+0000","last_became_peered":"2023-10-10T20:34:35.337277+0000","last_unstale":"2023-10-10T20:34:35.342399+0000","last_undegraded":"2023-10-10T20:34:35.342399+0000","last_fullsized":"2023-10-10T20:34:35.342399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:42:23.450966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.360813+0000","last_change":"2023-10-10T20:34:34.340837+0000","last_active":"2023-10-10T20:34:35.360813+0000","last_peered":"2023-10-10T20:34:35.360813+0000","last_clean":"2023-10-10T20:34:35.360813+0000","last_became_active":"2023-10-10T20:34:34.338971+0000","last_became_peered":"2023-10-10T20:34:34.338971+0000","last_unstale":"2023-10-10T20:34:35.360813+0000","last_undegraded":"2023-10-10T20:34:35.360813+0000","last_fullsized":"2023-10-10T20:34:35.360813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:10.201619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349519+0000","last_change":"2023-10-10T20:34:35.345474+0000","last_active":"2023-10-10T20:34:35.349519+0000","last_peered":"2023-10-10T20:34:35.349519+0000","last_clean":"2023-10-10T20:34:35.349519+0000","last_became_active":"2023-10-10T20:34:35.343255+0000","last_became_peered":"2023-10-10T20:34:35.343255+0000","last_unstale":"2023-10-10T20:34:35.349519+0000","last_undegraded":"2023-10-10T20:34:35.349519+0000","last_fullsized":"2023-10-10T20:34:35.349519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:35.499006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362947+0000","last_change":"2023-10-10T20:34:34.339953+0000","last_active":"2023-10-10T20:34:35.362947+0000","last_peered":"2023-10-10T20:34:35.362947+0000","last_clean":"2023-10-10T20:34:35.362947+0000","last_became_active":"2023-10-10T20:34:34.339676+0000","last_became_peered":"2023-10-10T20:34:34.339676+0000","last_unstale":"2023-10-10T20:34:35.362947+0000","last_undegraded":"2023-10-10T20:34:35.362947+0000","last_fullsized":"2023-10-10T20:34:35.362947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342687+0000","last_change":"2023-10-10T20:34:35.338535+0000","last_active":"2023-10-10T20:34:35.342687+0000","last_peered":"2023-10-10T20:34:35.342687+0000","last_clean":"2023-10-10T20:34:35.342687+0000","last_became_active":"2023-10-10T20:34:35.338266+0000","last_became_peered":"2023-10-10T20:34:35.338266+0000","last_unstale":"2023-10-10T20:34:35.342687+0000","last_undegraded":"2023-10-10T20:34:35.342687+0000","last_fullsized":"2023-10-10T20:34:35.342687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:39.183060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348224+0000","last_change":"2023-10-10T20:34:35.346165+0000","last_active":"2023-10-10T20:34:35.348224+0000","last_peered":"2023-10-10T20:34:35.348224+0000","last_clean":"2023-10-10T20:34:35.348224+0000","last_became_active":"2023-10-10T20:34:35.345862+0000","last_became_peered":"2023-10-10T20:34:35.345862+0000","last_unstale":"2023-10-10T20:34:35.348224+0000","last_undegraded":"2023-10-10T20:34:35.348224+0000","last_fullsized":"2023-10-10T20:34:35.348224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:18.709140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350401+0000","last_change":"2023-10-10T20:34:35.342709+0000","last_active":"2023-10-10T20:34:35.350401+0000","last_peered":"2023-10-10T20:34:35.350401+0000","last_clean":"2023-10-10T20:34:35.350401+0000","last_became_active":"2023-10-10T20:34:35.342433+0000","last_became_peered":"2023-10-10T20:34:35.342433+0000","last_unstale":"2023-10-10T20:34:35.350401+0000","last_undegraded":"2023-10-10T20:34:35.350401+0000","last_fullsized":"2023-10-10T20:34:35.350401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:48:12.831828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342070+0000","last_change":"2023-10-10T20:34:35.340426+0000","last_active":"2023-10-10T20:34:35.342070+0000","last_peered":"2023-10-10T20:34:35.342070+0000","last_clean":"2023-10-10T20:34:35.342070+0000","last_became_active":"2023-10-10T20:34:35.339966+0000","last_became_peered":"2023-10-10T20:34:35.339966+0000","last_unstale":"2023-10-10T20:34:35.342070+0000","last_undegraded":"2023-10-10T20:34:35.342070+0000","last_fullsized":"2023-10-10T20:34:35.342070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:42:23.450966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.351149+0000","last_change":"2023-10-10T20:34:35.341444+0000","last_active":"2023-10-10T20:34:35.351149+0000","last_peered":"2023-10-10T20:34:35.351149+0000","last_clean":"2023-10-10T20:34:35.351149+0000","last_became_active":"2023-10-10T20:34:35.341186+0000","last_became_peered":"2023-10-10T20:34:35.341186+0000","last_unstale":"2023-10-10T20:34:35.351149+0000","last_undegraded":"2023-10-10T20:34:35.351149+0000","last_fullsized":"2023-10-10T20:34:35.351149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352359+0000","last_change":"2023-10-10T20:34:35.347375+0000","last_active":"2023-10-10T20:34:35.352359+0000","last_peered":"2023-10-10T20:34:35.352359+0000","last_clean":"2023-10-10T20:34:35.352359+0000","last_became_active":"2023-10-10T20:34:35.347094+0000","last_became_peered":"2023-10-10T20:34:35.347094+0000","last_unstale":"2023-10-10T20:34:35.352359+0000","last_undegraded":"2023-10-10T20:34:35.352359+0000","last_fullsized":"2023-10-10T20:34:35.352359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:12:13.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342757+0000","last_change":"2023-10-10T20:34:35.340918+0000","last_active":"2023-10-10T20:34:35.342757+0000","last_peered":"2023-10-10T20:34:35.342757+0000","last_clean":"2023-10-10T20:34:35.342757+0000","last_became_active":"2023-10-10T20:34:35.340638+0000","last_became_peered":"2023-10-10T20:34:35.340638+0000","last_unstale":"2023-10-10T20:34:35.342757+0000","last_undegraded":"2023-10-10T20:34:35.342757+0000","last_fullsized":"2023-10-10T20:34:35.342757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:18.709140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348059+0000","last_change":"2023-10-10T20:34:35.345371+0000","last_active":"2023-10-10T20:34:35.348059+0000","last_peered":"2023-10-10T20:34:35.348059+0000","last_clean":"2023-10-10T20:34:35.348059+0000","last_became_active":"2023-10-10T20:34:35.345058+0000","last_became_peered":"2023-10-10T20:34:35.345058+0000","last_unstale":"2023-10-10T20:34:35.348059+0000","last_undegraded":"2023-10-10T20:34:35.348059+0000","last_fullsized":"2023-10-10T20:34:35.348059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352396+0000","last_change":"2023-10-10T20:34:35.349283+0000","last_active":"2023-10-10T20:34:35.352396+0000","last_peered":"2023-10-10T20:34:35.352396+0000","last_clean":"2023-10-10T20:34:35.352396+0000","last_became_active":"2023-10-10T20:34:35.349040+0000","last_became_peered":"2023-10-10T20:34:35.349040+0000","last_unstale":"2023-10-10T20:34:35.352396+0000","last_undegraded":"2023-10-10T20:34:35.352396+0000","last_fullsized":"2023-10-10T20:34:35.352396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:47:45.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349316+0000","last_change":"2023-10-10T20:34:35.343369+0000","last_active":"2023-10-10T20:34:35.349316+0000","last_peered":"2023-10-10T20:34:35.349316+0000","last_clean":"2023-10-10T20:34:35.349316+0000","last_became_active":"2023-10-10T20:34:35.342875+0000","last_became_peered":"2023-10-10T20:34:35.342875+0000","last_unstale":"2023-10-10T20:34:35.349316+0000","last_undegraded":"2023-10-10T20:34:35.349316+0000","last_fullsized":"2023-10-10T20:34:35.349316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:44:40.877955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352369+0000","last_change":"2023-10-10T20:34:35.349284+0000","last_active":"2023-10-10T20:34:35.352369+0000","last_peered":"2023-10-10T20:34:35.352369+0000","last_clean":"2023-10-10T20:34:35.352369+0000","last_became_active":"2023-10-10T20:34:35.349057+0000","last_became_peered":"2023-10-10T20:34:35.349057+0000","last_unstale":"2023-10-10T20:34:35.352369+0000","last_undegraded":"2023-10-10T20:34:35.352369+0000","last_fullsized":"2023-10-10T20:34:35.352369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:52:03.361718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.354153+0000","last_change":"2023-10-10T20:34:35.353504+0000","last_active":"2023-10-10T20:34:35.354153+0000","last_peered":"2023-10-10T20:34:35.354153+0000","last_clean":"2023-10-10T20:34:35.354153+0000","last_became_active":"2023-10-10T20:34:35.350145+0000","last_became_peered":"2023-10-10T20:34:35.350145+0000","last_unstale":"2023-10-10T20:34:35.354153+0000","last_undegraded":"2023-10-10T20:34:35.354153+0000","last_fullsized":"2023-10-10T20:34:35.354153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:52:03.361718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.351192+0000","last_change":"2023-10-10T20:34:35.345734+0000","last_active":"2023-10-10T20:34:35.351192+0000","last_peered":"2023-10-10T20:34:35.351192+0000","last_clean":"2023-10-10T20:34:35.351192+0000","last_became_active":"2023-10-10T20:34:35.345466+0000","last_became_peered":"2023-10-10T20:34:35.345466+0000","last_unstale":"2023-10-10T20:34:35.351192+0000","last_undegraded":"2023-10-10T20:34:35.351192+0000","last_fullsized":"2023-10-10T20:34:35.351192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:47:45.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350754+0000","last_change":"2023-10-10T20:34:35.349383+0000","last_active":"2023-10-10T20:34:35.350754+0000","last_peered":"2023-10-10T20:34:35.350754+0000","last_clean":"2023-10-10T20:34:35.350754+0000","last_became_active":"2023-10-10T20:34:35.348979+0000","last_became_peered":"2023-10-10T20:34:35.348979+0000","last_unstale":"2023-10-10T20:34:35.350754+0000","last_undegraded":"2023-10-10T20:34:35.350754+0000","last_fullsized":"2023-10-10T20:34:35.350754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:25:55.933179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298072,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073768,"statfs":{"total":96636764160,"available":96331538432,"internally_reserved":0,"allocated":802816,"data_stored":661576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":212992,"data_stored":70814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0640000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298076,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073764,"statfs":{"total":96636764160,"available":96331534336,"internally_reserved":0,"allocated":806912,"data_stored":660671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":70654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":70780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":70744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:34:39.401 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:34:39.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.544+0000 7fd5bcc92700 1 Processor -- start 2023-10-10T20:34:39.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.544+0000 7fd5bcc92700 1 -- start start 2023-10-10T20:34:39.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5bcc92700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd5b8159890 0x7fd5b815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:39.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5bcc92700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5b8159c60 0x7fd5b815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:39.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5bcc92700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd5b815db10 0x7fd5b8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:39.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5bcc92700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b815a160 con 0x7fd5b815db10 2023-10-10T20:34:39.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5bcc92700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b80d13a0 con 0x7fd5b8159890 2023-10-10T20:34:39.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5bcc92700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b815e140 con 0x7fd5b8159c60 2023-10-10T20:34:39.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5b6d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd5b815db10 0x7fd5b8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:39.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5b6d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd5b815db10 0x7fd5b8069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53194/0 (socket says 172.21.15.89:53194) 2023-10-10T20:34:39.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5b6d9d700 1 -- 172.21.15.89:0/291687714 learned_addr learned my addr 172.21.15.89:0/291687714 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:39.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5b5d9b700 1 --2- 172.21.15.89:0/291687714 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5b8159c60 0x7fd5b815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:39.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5b659c700 1 --2- 172.21.15.89:0/291687714 >> v2:172.21.15.111:3300/0 conn(0x7fd5b8159890 0x7fd5b815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:39.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5b6d9d700 1 -- 172.21.15.89:0/291687714 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5b8159c60 msgr2=0x7fd5b815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:39.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5b6d9d700 1 --2- 172.21.15.89:0/291687714 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5b8159c60 0x7fd5b815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5b6d9d700 1 -- 172.21.15.89:0/291687714 >> v2:172.21.15.111:3300/0 conn(0x7fd5b8159890 msgr2=0x7fd5b815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:39.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5b6d9d700 1 --2- 172.21.15.89:0/291687714 >> v2:172.21.15.111:3300/0 conn(0x7fd5b8159890 0x7fd5b815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5b6d9d700 1 -- 172.21.15.89:0/291687714 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b815a3d0 con 0x7fd5b815db10 2023-10-10T20:34:39.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5b5d9b700 1 --2- 172.21.15.89:0/291687714 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5b8159c60 0x7fd5b815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:39.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5b6d9d700 1 --2- 172.21.15.89:0/291687714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5b815db10 0x7fd5b8069560 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80d1090 tx=0x7fd5b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baee77ed104fbe71 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:39.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5b559a700 1 -- 172.21.15.89:0/291687714 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b000e070 con 0x7fd5b815db10 2023-10-10T20:34:39.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5b559a700 1 -- 172.21.15.89:0/291687714 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b000ba50 con 0x7fd5b815db10 2023-10-10T20:34:39.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.545+0000 7fd5b559a700 1 -- 172.21.15.89:0/291687714 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b00092e0 con 0x7fd5b815db10 2023-10-10T20:34:39.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.546+0000 7fd5bcc92700 1 -- 172.21.15.89:0/291687714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5b815db10 msgr2=0x7fd5b8069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:39.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.546+0000 7fd5bcc92700 1 --2- 172.21.15.89:0/291687714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5b815db10 0x7fd5b8069560 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80d1090 tx=0x7fd5b0009840 comp rx=0 tx=0).stop 2023-10-10T20:34:39.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.546+0000 7fd5bcc92700 1 -- 172.21.15.89:0/291687714 shutdown_connections 2023-10-10T20:34:39.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.546+0000 7fd5bcc92700 1 --2- 172.21.15.89:0/291687714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5b815db10 0x7fd5b8069560 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.546+0000 7fd5bcc92700 1 --2- 172.21.15.89:0/291687714 >> v2:172.21.15.111:3300/0 conn(0x7fd5b8159890 0x7fd5b815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.546+0000 7fd5bcc92700 1 --2- 172.21.15.89:0/291687714 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5b8159c60 0x7fd5b815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.546+0000 7fd5bcc92700 1 -- 172.21.15.89:0/291687714 >> 172.21.15.89:0/291687714 conn(0x7fd5b8068e20 msgr2=0x7fd5b80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:39.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.546+0000 7fd5bcc92700 1 -- 172.21.15.89:0/291687714 shutdown_connections 2023-10-10T20:34:39.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.546+0000 7fd5bcc92700 1 -- 172.21.15.89:0/291687714 wait complete. 2023-10-10T20:34:39.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.547+0000 7fd5bcc92700 1 Processor -- start 2023-10-10T20:34:39.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.547+0000 7fd5bcc92700 1 -- start start 2023-10-10T20:34:39.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5bcc92700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5b8159890 0x7fd5b8169e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:39.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5bcc92700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5b8159c60 0x7fd5b816cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:39.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5bcc92700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5b815db10 0x7fd5b816d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:39.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5bcc92700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b8160420 con 0x7fd5b8159890 2023-10-10T20:34:39.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5bcc92700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b8169970 con 0x7fd5b815db10 2023-10-10T20:34:39.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5bcc92700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b8169390 con 0x7fd5b8159c60 2023-10-10T20:34:39.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5b659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5b8159890 0x7fd5b8169e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:39.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5b659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5b8159890 0x7fd5b8169e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53206/0 (socket says 172.21.15.89:53206) 2023-10-10T20:34:39.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5b659c700 1 -- 172.21.15.89:0/2779774822 learned_addr learned my addr 172.21.15.89:0/2779774822 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:39.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5b5d9b700 1 --2- 172.21.15.89:0/2779774822 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5b8159c60 0x7fd5b816cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:39.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5b6d9d700 1 --2- 172.21.15.89:0/2779774822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5b815db10 0x7fd5b816d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:39.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5b659c700 1 -- 172.21.15.89:0/2779774822 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5b8159c60 msgr2=0x7fd5b816cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:39.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5b659c700 1 --2- 172.21.15.89:0/2779774822 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5b8159c60 0x7fd5b816cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5b659c700 1 -- 172.21.15.89:0/2779774822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5b815db10 msgr2=0x7fd5b816d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:39.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5b659c700 1 --2- 172.21.15.89:0/2779774822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5b815db10 0x7fd5b816d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5b659c700 1 -- 172.21.15.89:0/2779774822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b0009ae0 con 0x7fd5b8159890 2023-10-10T20:34:39.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5b6d9d700 1 --2- 172.21.15.89:0/2779774822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5b815db10 0x7fd5b816d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:39.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5b659c700 1 --2- 172.21.15.89:0/2779774822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5b8159890 0x7fd5b8169e00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800c800 tx=0x7fd5a8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:39.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5a67fc700 1 -- 172.21.15.89:0/2779774822 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8011600 con 0x7fd5b8159890 2023-10-10T20:34:39.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5a67fc700 1 -- 172.21.15.89:0/2779774822 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a8011760 con 0x7fd5b8159890 2023-10-10T20:34:39.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5bcc92700 1 -- 172.21.15.89:0/2779774822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b80c7a50 con 0x7fd5b8159890 2023-10-10T20:34:39.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.548+0000 7fd5a67fc700 1 -- 172.21.15.89:0/2779774822 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8011b20 con 0x7fd5b8159890 2023-10-10T20:34:39.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.549+0000 7fd5bcc92700 1 -- 172.21.15.89:0/2779774822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b8160140 con 0x7fd5b8159890 2023-10-10T20:34:39.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.549+0000 7fd5bcc92700 1 -- 172.21.15.89:0/2779774822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b804c3c0 con 0x7fd5b8159890 2023-10-10T20:34:39.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.550+0000 7fd5a67fc700 1 -- 172.21.15.89:0/2779774822 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd5a80193f0 con 0x7fd5b8159890 2023-10-10T20:34:39.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.550+0000 7fd5a67fc700 1 --2- 172.21.15.89:0/2779774822 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd598066f30 0x7fd5980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:39.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.550+0000 7fd5a67fc700 1 -- 172.21.15.89:0/2779774822 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd5a8084900 con 0x7fd5b8159890 2023-10-10T20:34:39.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.550+0000 7fd5b5d9b700 1 --2- 172.21.15.89:0/2779774822 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd598066f30 0x7fd5980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:39.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.551+0000 7fd5b5d9b700 1 --2- 172.21.15.89:0/2779774822 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd598066f30 0x7fd5980693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80d00d0 tx=0x7fd5ac006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:39.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.555+0000 7fd5a67fc700 1 -- 172.21.15.89:0/2779774822 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5a8051b70 con 0x7fd5b8159890 2023-10-10T20:34:39.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.732+0000 7fd5bcc92700 1 -- 172.21.15.89:0/2779774822 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd5b8169ab0 con 0x7fd598066f30 2023-10-10T20:34:39.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.739+0000 7fd5a67fc700 1 -- 172.21.15.89:0/2779774822 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264649 (secure 0 0 0) 0x7fd5ac00c040 con 0x7fd598066f30 2023-10-10T20:34:39.745 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:39.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.743+0000 7fd5bcc92700 1 -- 172.21.15.89:0/2779774822 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd598066f30 msgr2=0x7fd5980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:39.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.743+0000 7fd5bcc92700 1 --2- 172.21.15.89:0/2779774822 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd598066f30 0x7fd5980693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80d00d0 tx=0x7fd5ac006f90 comp rx=0 tx=0).stop 2023-10-10T20:34:39.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.743+0000 7fd5bcc92700 1 -- 172.21.15.89:0/2779774822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5b8159890 msgr2=0x7fd5b8169e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:39.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.743+0000 7fd5bcc92700 1 --2- 172.21.15.89:0/2779774822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5b8159890 0x7fd5b8169e00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800c800 tx=0x7fd5a8008040 comp rx=0 tx=0).stop 2023-10-10T20:34:39.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.743+0000 7fd5bcc92700 1 -- 172.21.15.89:0/2779774822 shutdown_connections 2023-10-10T20:34:39.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.743+0000 7fd5bcc92700 1 --2- 172.21.15.89:0/2779774822 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd598066f30 0x7fd5980693e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.743+0000 7fd5bcc92700 1 --2- 172.21.15.89:0/2779774822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5b815db10 0x7fd5b816d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.743+0000 7fd5bcc92700 1 --2- 172.21.15.89:0/2779774822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5b8159890 0x7fd5b8169e00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.743+0000 7fd5bcc92700 1 --2- 172.21.15.89:0/2779774822 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5b8159c60 0x7fd5b816cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.743+0000 7fd5bcc92700 1 -- 172.21.15.89:0/2779774822 >> 172.21.15.89:0/2779774822 conn(0x7fd5b8068e20 msgr2=0x7fd5b815b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:39.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.743+0000 7fd5bcc92700 1 -- 172.21.15.89:0/2779774822 shutdown_connections 2023-10-10T20:34:39.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.743+0000 7fd5bcc92700 1 -- 172.21.15.89:0/2779774822 wait complete. 2023-10-10T20:34:39.750 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:34:39.757 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-10-10T20:34:38.652691+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381124,"kb_used_data":2820,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593596,"statfs":{"total":773094113280,"available":770655842304,"internally_reserved":0,"allocated":2887680,"data_stored":1749619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.338046"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.347393+0000","last_change":"2023-10-10T20:34:35.344417+0000","last_active":"2023-10-10T20:34:35.347393+0000","last_peered":"2023-10-10T20:34:35.347393+0000","last_clean":"2023-10-10T20:34:35.347393+0000","last_became_active":"2023-10-10T20:34:35.344171+0000","last_became_peered":"2023-10-10T20:34:35.344171+0000","last_unstale":"2023-10-10T20:34:35.347393+0000","last_undegraded":"2023-10-10T20:34:35.347393+0000","last_fullsized":"2023-10-10T20:34:35.347393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:06.136941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350364+0000","last_change":"2023-10-10T20:34:35.344388+0000","last_active":"2023-10-10T20:34:35.350364+0000","last_peered":"2023-10-10T20:34:35.350364+0000","last_clean":"2023-10-10T20:34:35.350364+0000","last_became_active":"2023-10-10T20:34:35.344132+0000","last_became_peered":"2023-10-10T20:34:35.344132+0000","last_unstale":"2023-10-10T20:34:35.350364+0000","last_undegraded":"2023-10-10T20:34:35.350364+0000","last_fullsized":"2023-10-10T20:34:35.350364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:05:02.787176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.341464+0000","last_change":"2023-10-10T20:34:35.339021+0000","last_active":"2023-10-10T20:34:35.341464+0000","last_peered":"2023-10-10T20:34:35.341464+0000","last_clean":"2023-10-10T20:34:35.341464+0000","last_became_active":"2023-10-10T20:34:35.338713+0000","last_became_peered":"2023-10-10T20:34:35.338713+0000","last_unstale":"2023-10-10T20:34:35.341464+0000","last_undegraded":"2023-10-10T20:34:35.341464+0000","last_fullsized":"2023-10-10T20:34:35.341464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:12:13.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349539+0000","last_change":"2023-10-10T20:34:35.345674+0000","last_active":"2023-10-10T20:34:35.349539+0000","last_peered":"2023-10-10T20:34:35.349539+0000","last_clean":"2023-10-10T20:34:35.349539+0000","last_became_active":"2023-10-10T20:34:35.342947+0000","last_became_peered":"2023-10-10T20:34:35.342947+0000","last_unstale":"2023-10-10T20:34:35.349539+0000","last_undegraded":"2023-10-10T20:34:35.349539+0000","last_fullsized":"2023-10-10T20:34:35.349539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:06.136941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.331236+0000","last_change":"2023-10-10T20:34:35.330128+0000","last_active":"2023-10-10T20:34:35.331236+0000","last_peered":"2023-10-10T20:34:35.331236+0000","last_clean":"2023-10-10T20:34:35.331236+0000","last_became_active":"2023-10-10T20:34:35.329809+0000","last_became_peered":"2023-10-10T20:34:35.329809+0000","last_unstale":"2023-10-10T20:34:35.331236+0000","last_undegraded":"2023-10-10T20:34:35.331236+0000","last_fullsized":"2023-10-10T20:34:35.331236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:08.378701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.351515+0000","last_change":"2023-10-10T20:34:35.341852+0000","last_active":"2023-10-10T20:34:35.351515+0000","last_peered":"2023-10-10T20:34:35.351515+0000","last_clean":"2023-10-10T20:34:35.351515+0000","last_became_active":"2023-10-10T20:34:35.341590+0000","last_became_peered":"2023-10-10T20:34:35.341590+0000","last_unstale":"2023-10-10T20:34:35.351515+0000","last_undegraded":"2023-10-10T20:34:35.351515+0000","last_fullsized":"2023-10-10T20:34:35.351515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:38:40.522047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342438+0000","last_change":"2023-10-10T20:34:35.336268+0000","last_active":"2023-10-10T20:34:35.342438+0000","last_peered":"2023-10-10T20:34:35.342438+0000","last_clean":"2023-10-10T20:34:35.342438+0000","last_became_active":"2023-10-10T20:34:35.335108+0000","last_became_peered":"2023-10-10T20:34:35.335108+0000","last_unstale":"2023-10-10T20:34:35.342438+0000","last_undegraded":"2023-10-10T20:34:35.342438+0000","last_fullsized":"2023-10-10T20:34:35.342438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:36.295226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350260+0000","last_change":"2023-10-10T20:34:35.342130+0000","last_active":"2023-10-10T20:34:35.350260+0000","last_peered":"2023-10-10T20:34:35.350260+0000","last_clean":"2023-10-10T20:34:35.350260+0000","last_became_active":"2023-10-10T20:34:35.341794+0000","last_became_peered":"2023-10-10T20:34:35.341794+0000","last_unstale":"2023-10-10T20:34:35.350260+0000","last_undegraded":"2023-10-10T20:34:35.350260+0000","last_fullsized":"2023-10-10T20:34:35.350260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.347376+0000","last_change":"2023-10-10T20:34:35.342521+0000","last_active":"2023-10-10T20:34:35.347376+0000","last_peered":"2023-10-10T20:34:35.347376+0000","last_clean":"2023-10-10T20:34:35.347376+0000","last_became_active":"2023-10-10T20:34:35.342237+0000","last_became_peered":"2023-10-10T20:34:35.342237+0000","last_unstale":"2023-10-10T20:34:35.347376+0000","last_undegraded":"2023-10-10T20:34:35.347376+0000","last_fullsized":"2023-10-10T20:34:35.347376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:35.499006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348040+0000","last_change":"2023-10-10T20:34:35.344521+0000","last_active":"2023-10-10T20:34:35.348040+0000","last_peered":"2023-10-10T20:34:35.348040+0000","last_clean":"2023-10-10T20:34:35.348040+0000","last_became_active":"2023-10-10T20:34:35.344132+0000","last_became_peered":"2023-10-10T20:34:35.344132+0000","last_unstale":"2023-10-10T20:34:35.348040+0000","last_undegraded":"2023-10-10T20:34:35.348040+0000","last_fullsized":"2023-10-10T20:34:35.348040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:05:02.787176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.344573+0000","last_change":"2023-10-10T20:34:35.342152+0000","last_active":"2023-10-10T20:34:35.344573+0000","last_peered":"2023-10-10T20:34:35.344573+0000","last_clean":"2023-10-10T20:34:35.344573+0000","last_became_active":"2023-10-10T20:34:35.341883+0000","last_became_peered":"2023-10-10T20:34:35.341883+0000","last_unstale":"2023-10-10T20:34:35.344573+0000","last_undegraded":"2023-10-10T20:34:35.344573+0000","last_fullsized":"2023-10-10T20:34:35.344573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:10:13.711031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350167+0000","last_change":"2023-10-10T20:34:35.342940+0000","last_active":"2023-10-10T20:34:35.350167+0000","last_peered":"2023-10-10T20:34:35.350167+0000","last_clean":"2023-10-10T20:34:35.350167+0000","last_became_active":"2023-10-10T20:34:35.342645+0000","last_became_peered":"2023-10-10T20:34:35.342645+0000","last_unstale":"2023-10-10T20:34:35.350167+0000","last_undegraded":"2023-10-10T20:34:35.350167+0000","last_fullsized":"2023-10-10T20:34:35.350167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:48:12.831828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350468+0000","last_change":"2023-10-10T20:34:35.343591+0000","last_active":"2023-10-10T20:34:35.350468+0000","last_peered":"2023-10-10T20:34:35.350468+0000","last_clean":"2023-10-10T20:34:35.350468+0000","last_became_active":"2023-10-10T20:34:35.343310+0000","last_became_peered":"2023-10-10T20:34:35.343310+0000","last_unstale":"2023-10-10T20:34:35.350468+0000","last_undegraded":"2023-10-10T20:34:35.350468+0000","last_fullsized":"2023-10-10T20:34:35.350468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:24:50.729855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348660+0000","last_change":"2023-10-10T20:34:35.343316+0000","last_active":"2023-10-10T20:34:35.348660+0000","last_peered":"2023-10-10T20:34:35.348660+0000","last_clean":"2023-10-10T20:34:35.348660+0000","last_became_active":"2023-10-10T20:34:35.343015+0000","last_became_peered":"2023-10-10T20:34:35.343015+0000","last_unstale":"2023-10-10T20:34:35.348660+0000","last_undegraded":"2023-10-10T20:34:35.348660+0000","last_fullsized":"2023-10-10T20:34:35.348660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:05:02.787176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352727+0000","last_change":"2023-10-10T20:34:35.346542+0000","last_active":"2023-10-10T20:34:35.352727+0000","last_peered":"2023-10-10T20:34:35.352727+0000","last_clean":"2023-10-10T20:34:35.352727+0000","last_became_active":"2023-10-10T20:34:35.346323+0000","last_became_peered":"2023-10-10T20:34:35.346323+0000","last_unstale":"2023-10-10T20:34:35.352727+0000","last_undegraded":"2023-10-10T20:34:35.352727+0000","last_fullsized":"2023-10-10T20:34:35.352727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:11:35.427062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349566+0000","last_change":"2023-10-10T20:34:35.346445+0000","last_active":"2023-10-10T20:34:35.349566+0000","last_peered":"2023-10-10T20:34:35.349566+0000","last_clean":"2023-10-10T20:34:35.349566+0000","last_became_active":"2023-10-10T20:34:35.345449+0000","last_became_peered":"2023-10-10T20:34:35.345449+0000","last_unstale":"2023-10-10T20:34:35.349566+0000","last_undegraded":"2023-10-10T20:34:35.349566+0000","last_fullsized":"2023-10-10T20:34:35.349566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350860+0000","last_change":"2023-10-10T20:34:35.348398+0000","last_active":"2023-10-10T20:34:35.350860+0000","last_peered":"2023-10-10T20:34:35.350860+0000","last_clean":"2023-10-10T20:34:35.350860+0000","last_became_active":"2023-10-10T20:34:35.348110+0000","last_became_peered":"2023-10-10T20:34:35.348110+0000","last_unstale":"2023-10-10T20:34:35.350860+0000","last_undegraded":"2023-10-10T20:34:35.350860+0000","last_fullsized":"2023-10-10T20:34:35.350860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:36.295226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350234+0000","last_change":"2023-10-10T20:34:35.343071+0000","last_active":"2023-10-10T20:34:35.350234+0000","last_peered":"2023-10-10T20:34:35.350234+0000","last_clean":"2023-10-10T20:34:35.350234+0000","last_became_active":"2023-10-10T20:34:35.342740+0000","last_became_peered":"2023-10-10T20:34:35.342740+0000","last_unstale":"2023-10-10T20:34:35.350234+0000","last_undegraded":"2023-10-10T20:34:35.350234+0000","last_fullsized":"2023-10-10T20:34:35.350234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:25:34.953681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.331257+0000","last_change":"2023-10-10T20:34:35.330198+0000","last_active":"2023-10-10T20:34:35.331257+0000","last_peered":"2023-10-10T20:34:35.331257+0000","last_clean":"2023-10-10T20:34:35.331257+0000","last_became_active":"2023-10-10T20:34:35.329847+0000","last_became_peered":"2023-10-10T20:34:35.329847+0000","last_unstale":"2023-10-10T20:34:35.331257+0000","last_undegraded":"2023-10-10T20:34:35.331257+0000","last_fullsized":"2023-10-10T20:34:35.331257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:10.218128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362481+0000","last_change":"2023-10-10T20:34:34.340043+0000","last_active":"2023-10-10T20:34:35.362481+0000","last_peered":"2023-10-10T20:34:35.362481+0000","last_clean":"2023-10-10T20:34:35.362481+0000","last_became_active":"2023-10-10T20:34:34.339783+0000","last_became_peered":"2023-10-10T20:34:34.339783+0000","last_unstale":"2023-10-10T20:34:35.362481+0000","last_undegraded":"2023-10-10T20:34:35.362481+0000","last_fullsized":"2023-10-10T20:34:35.362481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:00:46.542353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350303+0000","last_change":"2023-10-10T20:34:35.342670+0000","last_active":"2023-10-10T20:34:35.350303+0000","last_peered":"2023-10-10T20:34:35.350303+0000","last_clean":"2023-10-10T20:34:35.350303+0000","last_became_active":"2023-10-10T20:34:35.342432+0000","last_became_peered":"2023-10-10T20:34:35.342432+0000","last_unstale":"2023-10-10T20:34:35.350303+0000","last_undegraded":"2023-10-10T20:34:35.350303+0000","last_fullsized":"2023-10-10T20:34:35.350303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:38:40.522047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362959+0000","last_change":"2023-10-10T20:34:34.343567+0000","last_active":"2023-10-10T20:34:35.362959+0000","last_peered":"2023-10-10T20:34:35.362959+0000","last_clean":"2023-10-10T20:34:35.362959+0000","last_became_active":"2023-10-10T20:34:34.343344+0000","last_became_peered":"2023-10-10T20:34:34.343344+0000","last_unstale":"2023-10-10T20:34:35.362959+0000","last_undegraded":"2023-10-10T20:34:35.362959+0000","last_fullsized":"2023-10-10T20:34:35.362959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:57:27.930567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348540+0000","last_change":"2023-10-10T20:34:35.341588+0000","last_active":"2023-10-10T20:34:35.348540+0000","last_peered":"2023-10-10T20:34:35.348540+0000","last_clean":"2023-10-10T20:34:35.348540+0000","last_became_active":"2023-10-10T20:34:35.341193+0000","last_became_peered":"2023-10-10T20:34:35.341193+0000","last_unstale":"2023-10-10T20:34:35.348540+0000","last_undegraded":"2023-10-10T20:34:35.348540+0000","last_fullsized":"2023-10-10T20:34:35.348540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:25:55.933179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.357498+0000","last_change":"2023-10-10T20:34:34.347611+0000","last_active":"2023-10-10T20:34:35.357498+0000","last_peered":"2023-10-10T20:34:35.357498+0000","last_clean":"2023-10-10T20:34:35.357498+0000","last_became_active":"2023-10-10T20:34:34.347335+0000","last_became_peered":"2023-10-10T20:34:34.347335+0000","last_unstale":"2023-10-10T20:34:35.357498+0000","last_undegraded":"2023-10-10T20:34:35.357498+0000","last_fullsized":"2023-10-10T20:34:35.357498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:11.616929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350859+0000","last_change":"2023-10-10T20:34:35.348359+0000","last_active":"2023-10-10T20:34:35.350859+0000","last_peered":"2023-10-10T20:34:35.350859+0000","last_clean":"2023-10-10T20:34:35.350859+0000","last_became_active":"2023-10-10T20:34:35.347861+0000","last_became_peered":"2023-10-10T20:34:35.347861+0000","last_unstale":"2023-10-10T20:34:35.350859+0000","last_undegraded":"2023-10-10T20:34:35.350859+0000","last_fullsized":"2023-10-10T20:34:35.350859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:39.183060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.356525+0000","last_change":"2023-10-10T20:34:34.344260+0000","last_active":"2023-10-10T20:34:35.356525+0000","last_peered":"2023-10-10T20:34:35.356525+0000","last_clean":"2023-10-10T20:34:35.356525+0000","last_became_active":"2023-10-10T20:34:34.343945+0000","last_became_peered":"2023-10-10T20:34:34.343945+0000","last_unstale":"2023-10-10T20:34:35.356525+0000","last_undegraded":"2023-10-10T20:34:35.356525+0000","last_fullsized":"2023-10-10T20:34:35.356525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352561+0000","last_change":"2023-10-10T20:34:35.348160+0000","last_active":"2023-10-10T20:34:35.352561+0000","last_peered":"2023-10-10T20:34:35.352561+0000","last_clean":"2023-10-10T20:34:35.352561+0000","last_became_active":"2023-10-10T20:34:35.347847+0000","last_became_peered":"2023-10-10T20:34:35.347847+0000","last_unstale":"2023-10-10T20:34:35.352561+0000","last_undegraded":"2023-10-10T20:34:35.352561+0000","last_fullsized":"2023-10-10T20:34:35.352561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:08.378701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.361981+0000","last_change":"2023-10-10T20:34:34.339973+0000","last_active":"2023-10-10T20:34:35.361981+0000","last_peered":"2023-10-10T20:34:35.361981+0000","last_clean":"2023-10-10T20:34:35.361981+0000","last_became_active":"2023-10-10T20:34:34.339685+0000","last_became_peered":"2023-10-10T20:34:34.339685+0000","last_unstale":"2023-10-10T20:34:35.361981+0000","last_undegraded":"2023-10-10T20:34:35.361981+0000","last_fullsized":"2023-10-10T20:34:35.361981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:34:00.263416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348564+0000","last_change":"2023-10-10T20:34:35.340064+0000","last_active":"2023-10-10T20:34:35.348564+0000","last_peered":"2023-10-10T20:34:35.348564+0000","last_clean":"2023-10-10T20:34:35.348564+0000","last_became_active":"2023-10-10T20:34:35.339811+0000","last_became_peered":"2023-10-10T20:34:35.339811+0000","last_unstale":"2023-10-10T20:34:35.348564+0000","last_undegraded":"2023-10-10T20:34:35.348564+0000","last_fullsized":"2023-10-10T20:34:35.348564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:17.774898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.357351+0000","last_change":"2023-10-10T20:34:34.350170+0000","last_active":"2023-10-10T20:34:35.357351+0000","last_peered":"2023-10-10T20:34:35.357351+0000","last_clean":"2023-10-10T20:34:35.357351+0000","last_became_active":"2023-10-10T20:34:34.349858+0000","last_became_peered":"2023-10-10T20:34:34.349858+0000","last_unstale":"2023-10-10T20:34:35.357351+0000","last_undegraded":"2023-10-10T20:34:35.357351+0000","last_fullsized":"2023-10-10T20:34:35.357351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.351173+0000","last_change":"2023-10-10T20:34:35.342458+0000","last_active":"2023-10-10T20:34:35.351173+0000","last_peered":"2023-10-10T20:34:35.351173+0000","last_clean":"2023-10-10T20:34:35.351173+0000","last_became_active":"2023-10-10T20:34:35.342063+0000","last_became_peered":"2023-10-10T20:34:35.342063+0000","last_unstale":"2023-10-10T20:34:35.351173+0000","last_undegraded":"2023-10-10T20:34:35.351173+0000","last_fullsized":"2023-10-10T20:34:35.351173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:18.709140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.334160+0000","last_change":"2023-10-10T20:34:34.344734+0000","last_active":"2023-10-10T20:34:35.334160+0000","last_peered":"2023-10-10T20:34:35.334160+0000","last_clean":"2023-10-10T20:34:35.334160+0000","last_became_active":"2023-10-10T20:34:34.344418+0000","last_became_peered":"2023-10-10T20:34:34.344418+0000","last_unstale":"2023-10-10T20:34:35.334160+0000","last_undegraded":"2023-10-10T20:34:35.334160+0000","last_fullsized":"2023-10-10T20:34:35.334160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:27:04.988955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350422+0000","last_change":"2023-10-10T20:34:35.343560+0000","last_active":"2023-10-10T20:34:35.350422+0000","last_peered":"2023-10-10T20:34:35.350422+0000","last_clean":"2023-10-10T20:34:35.350422+0000","last_became_active":"2023-10-10T20:34:35.343170+0000","last_became_peered":"2023-10-10T20:34:35.343170+0000","last_unstale":"2023-10-10T20:34:35.350422+0000","last_undegraded":"2023-10-10T20:34:35.350422+0000","last_fullsized":"2023-10-10T20:34:35.350422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:05:02.787176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.334617+0000","last_change":"2023-10-10T20:34:34.339163+0000","last_active":"2023-10-10T20:34:35.334617+0000","last_peered":"2023-10-10T20:34:35.334617+0000","last_clean":"2023-10-10T20:34:35.334617+0000","last_became_active":"2023-10-10T20:34:34.338879+0000","last_became_peered":"2023-10-10T20:34:34.338879+0000","last_unstale":"2023-10-10T20:34:35.334617+0000","last_undegraded":"2023-10-10T20:34:35.334617+0000","last_fullsized":"2023-10-10T20:34:35.334617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:40:03.331998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342534+0000","last_change":"2023-10-10T20:34:35.340814+0000","last_active":"2023-10-10T20:34:35.342534+0000","last_peered":"2023-10-10T20:34:35.342534+0000","last_clean":"2023-10-10T20:34:35.342534+0000","last_became_active":"2023-10-10T20:34:35.340139+0000","last_became_peered":"2023-10-10T20:34:35.340139+0000","last_unstale":"2023-10-10T20:34:35.342534+0000","last_undegraded":"2023-10-10T20:34:35.342534+0000","last_fullsized":"2023-10-10T20:34:35.342534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:47:45.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.336472+0000","last_change":"2023-10-10T20:34:34.348036+0000","last_active":"2023-10-10T20:34:35.336472+0000","last_peered":"2023-10-10T20:34:35.336472+0000","last_clean":"2023-10-10T20:34:35.336472+0000","last_became_active":"2023-10-10T20:34:34.347739+0000","last_became_peered":"2023-10-10T20:34:34.347739+0000","last_unstale":"2023-10-10T20:34:35.336472+0000","last_undegraded":"2023-10-10T20:34:35.336472+0000","last_fullsized":"2023-10-10T20:34:35.336472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:27:04.988955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.354125+0000","last_change":"2023-10-10T20:34:35.352621+0000","last_active":"2023-10-10T20:34:35.354125+0000","last_peered":"2023-10-10T20:34:35.354125+0000","last_clean":"2023-10-10T20:34:35.354125+0000","last_became_active":"2023-10-10T20:34:35.352324+0000","last_became_peered":"2023-10-10T20:34:35.352324+0000","last_unstale":"2023-10-10T20:34:35.354125+0000","last_undegraded":"2023-10-10T20:34:35.354125+0000","last_fullsized":"2023-10-10T20:34:35.354125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:24:50.729855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.334195+0000","last_change":"2023-10-10T20:34:34.344822+0000","last_active":"2023-10-10T20:34:35.334195+0000","last_peered":"2023-10-10T20:34:35.334195+0000","last_clean":"2023-10-10T20:34:35.334195+0000","last_became_active":"2023-10-10T20:34:34.344549+0000","last_became_peered":"2023-10-10T20:34:34.344549+0000","last_unstale":"2023-10-10T20:34:35.334195+0000","last_undegraded":"2023-10-10T20:34:35.334195+0000","last_fullsized":"2023-10-10T20:34:35.334195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.344601+0000","last_change":"2023-10-10T20:34:35.339793+0000","last_active":"2023-10-10T20:34:35.344601+0000","last_peered":"2023-10-10T20:34:35.344601+0000","last_clean":"2023-10-10T20:34:35.344601+0000","last_became_active":"2023-10-10T20:34:35.339226+0000","last_became_peered":"2023-10-10T20:34:35.339226+0000","last_unstale":"2023-10-10T20:34:35.344601+0000","last_undegraded":"2023-10-10T20:34:35.344601+0000","last_fullsized":"2023-10-10T20:34:35.344601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:10:13.711031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.360889+0000","last_change":"2023-10-10T20:34:34.336051+0000","last_active":"2023-10-10T20:34:35.360889+0000","last_peered":"2023-10-10T20:34:35.360889+0000","last_clean":"2023-10-10T20:34:35.360889+0000","last_became_active":"2023-10-10T20:34:34.335689+0000","last_became_peered":"2023-10-10T20:34:34.335689+0000","last_unstale":"2023-10-10T20:34:35.360889+0000","last_undegraded":"2023-10-10T20:34:35.360889+0000","last_fullsized":"2023-10-10T20:34:35.360889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:11.616929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350147+0000","last_change":"2023-10-10T20:34:35.341270+0000","last_active":"2023-10-10T20:34:35.350147+0000","last_peered":"2023-10-10T20:34:35.350147+0000","last_clean":"2023-10-10T20:34:35.350147+0000","last_became_active":"2023-10-10T20:34:35.341024+0000","last_became_peered":"2023-10-10T20:34:35.341024+0000","last_unstale":"2023-10-10T20:34:35.350147+0000","last_undegraded":"2023-10-10T20:34:35.350147+0000","last_fullsized":"2023-10-10T20:34:35.350147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:16.035453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.367035+0000","last_change":"2023-10-10T20:34:34.346412+0000","last_active":"2023-10-10T20:34:35.367035+0000","last_peered":"2023-10-10T20:34:35.367035+0000","last_clean":"2023-10-10T20:34:35.367035+0000","last_became_active":"2023-10-10T20:34:34.346020+0000","last_became_peered":"2023-10-10T20:34:34.346020+0000","last_unstale":"2023-10-10T20:34:35.367035+0000","last_undegraded":"2023-10-10T20:34:35.367035+0000","last_fullsized":"2023-10-10T20:34:35.367035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:15.030808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.345530+0000","last_change":"2023-10-10T20:34:35.340255+0000","last_active":"2023-10-10T20:34:35.345530+0000","last_peered":"2023-10-10T20:34:35.345530+0000","last_clean":"2023-10-10T20:34:35.345530+0000","last_became_active":"2023-10-10T20:34:35.339971+0000","last_became_peered":"2023-10-10T20:34:35.339971+0000","last_unstale":"2023-10-10T20:34:35.345530+0000","last_undegraded":"2023-10-10T20:34:35.345530+0000","last_fullsized":"2023-10-10T20:34:35.345530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:44:40.877955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362334+0000","last_change":"2023-10-10T20:34:34.343026+0000","last_active":"2023-10-10T20:34:35.362334+0000","last_peered":"2023-10-10T20:34:35.362334+0000","last_clean":"2023-10-10T20:34:35.362334+0000","last_became_active":"2023-10-10T20:34:34.342742+0000","last_became_peered":"2023-10-10T20:34:34.342742+0000","last_unstale":"2023-10-10T20:34:35.362334+0000","last_undegraded":"2023-10-10T20:34:35.362334+0000","last_fullsized":"2023-10-10T20:34:35.362334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:57:27.930567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348647+0000","last_change":"2023-10-10T20:34:35.342407+0000","last_active":"2023-10-10T20:34:35.348647+0000","last_peered":"2023-10-10T20:34:35.348647+0000","last_clean":"2023-10-10T20:34:35.348647+0000","last_became_active":"2023-10-10T20:34:35.341971+0000","last_became_peered":"2023-10-10T20:34:35.341971+0000","last_unstale":"2023-10-10T20:34:35.348647+0000","last_undegraded":"2023-10-10T20:34:35.348647+0000","last_fullsized":"2023-10-10T20:34:35.348647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:27:10.934315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.336796+0000","last_change":"2023-10-10T20:34:34.339893+0000","last_active":"2023-10-10T20:34:35.336796+0000","last_peered":"2023-10-10T20:34:35.336796+0000","last_clean":"2023-10-10T20:34:35.336796+0000","last_became_active":"2023-10-10T20:34:34.339585+0000","last_became_peered":"2023-10-10T20:34:34.339585+0000","last_unstale":"2023-10-10T20:34:35.336796+0000","last_undegraded":"2023-10-10T20:34:35.336796+0000","last_fullsized":"2023-10-10T20:34:35.336796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:15.030808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.346782+0000","last_change":"2023-10-10T20:34:35.341978+0000","last_active":"2023-10-10T20:34:35.346782+0000","last_peered":"2023-10-10T20:34:35.346782+0000","last_clean":"2023-10-10T20:34:35.346782+0000","last_became_active":"2023-10-10T20:34:35.341703+0000","last_became_peered":"2023-10-10T20:34:35.341703+0000","last_unstale":"2023-10-10T20:34:35.346782+0000","last_undegraded":"2023-10-10T20:34:35.346782+0000","last_fullsized":"2023-10-10T20:34:35.346782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:38:40.522047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.335441+0000","last_change":"2023-10-10T20:34:34.350137+0000","last_active":"2023-10-10T20:34:35.335441+0000","last_peered":"2023-10-10T20:34:35.335441+0000","last_clean":"2023-10-10T20:34:35.335441+0000","last_became_active":"2023-10-10T20:34:34.349855+0000","last_became_peered":"2023-10-10T20:34:34.349855+0000","last_unstale":"2023-10-10T20:34:35.335441+0000","last_undegraded":"2023-10-10T20:34:35.335441+0000","last_fullsized":"2023-10-10T20:34:35.335441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.347963+0000","last_change":"2023-10-10T20:34:35.344256+0000","last_active":"2023-10-10T20:34:35.347963+0000","last_peered":"2023-10-10T20:34:35.347963+0000","last_clean":"2023-10-10T20:34:35.347963+0000","last_became_active":"2023-10-10T20:34:35.343626+0000","last_became_peered":"2023-10-10T20:34:35.343626+0000","last_unstale":"2023-10-10T20:34:35.347963+0000","last_undegraded":"2023-10-10T20:34:35.347963+0000","last_fullsized":"2023-10-10T20:34:35.347963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:12:13.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.361414+0000","last_change":"2023-10-10T20:34:34.337912+0000","last_active":"2023-10-10T20:34:35.361414+0000","last_peered":"2023-10-10T20:34:35.361414+0000","last_clean":"2023-10-10T20:34:35.361414+0000","last_became_active":"2023-10-10T20:34:34.337588+0000","last_became_peered":"2023-10-10T20:34:34.337588+0000","last_unstale":"2023-10-10T20:34:35.361414+0000","last_undegraded":"2023-10-10T20:34:35.361414+0000","last_fullsized":"2023-10-10T20:34:35.361414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:40:03.331998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342457+0000","last_change":"2023-10-10T20:34:35.337938+0000","last_active":"2023-10-10T20:34:35.342457+0000","last_peered":"2023-10-10T20:34:35.342457+0000","last_clean":"2023-10-10T20:34:35.342457+0000","last_became_active":"2023-10-10T20:34:35.337649+0000","last_became_peered":"2023-10-10T20:34:35.337649+0000","last_unstale":"2023-10-10T20:34:35.342457+0000","last_undegraded":"2023-10-10T20:34:35.342457+0000","last_fullsized":"2023-10-10T20:34:35.342457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362619+0000","last_change":"2023-10-10T20:34:34.344045+0000","last_active":"2023-10-10T20:34:35.362619+0000","last_peered":"2023-10-10T20:34:35.362619+0000","last_clean":"2023-10-10T20:34:35.362619+0000","last_became_active":"2023-10-10T20:34:34.343736+0000","last_became_peered":"2023-10-10T20:34:34.343736+0000","last_unstale":"2023-10-10T20:34:35.362619+0000","last_undegraded":"2023-10-10T20:34:35.362619+0000","last_fullsized":"2023-10-10T20:34:35.362619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:57:27.930567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350691+0000","last_change":"2023-10-10T20:34:35.343035+0000","last_active":"2023-10-10T20:34:35.350691+0000","last_peered":"2023-10-10T20:34:35.350691+0000","last_clean":"2023-10-10T20:34:35.350691+0000","last_became_active":"2023-10-10T20:34:35.342776+0000","last_became_peered":"2023-10-10T20:34:35.342776+0000","last_unstale":"2023-10-10T20:34:35.350691+0000","last_undegraded":"2023-10-10T20:34:35.350691+0000","last_fullsized":"2023-10-10T20:34:35.350691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:52:03.361718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.337288+0000","last_change":"2023-10-10T20:34:34.337961+0000","last_active":"2023-10-10T20:34:35.337288+0000","last_peered":"2023-10-10T20:34:35.337288+0000","last_clean":"2023-10-10T20:34:35.337288+0000","last_became_active":"2023-10-10T20:34:34.337627+0000","last_became_peered":"2023-10-10T20:34:34.337627+0000","last_unstale":"2023-10-10T20:34:35.337288+0000","last_undegraded":"2023-10-10T20:34:35.337288+0000","last_fullsized":"2023-10-10T20:34:35.337288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:34:00.263416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349588+0000","last_change":"2023-10-10T20:34:35.346121+0000","last_active":"2023-10-10T20:34:35.349588+0000","last_peered":"2023-10-10T20:34:35.349588+0000","last_clean":"2023-10-10T20:34:35.349588+0000","last_became_active":"2023-10-10T20:34:35.345847+0000","last_became_peered":"2023-10-10T20:34:35.345847+0000","last_unstale":"2023-10-10T20:34:35.349588+0000","last_undegraded":"2023-10-10T20:34:35.349588+0000","last_fullsized":"2023-10-10T20:34:35.349588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:10.218128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.336695+0000","last_change":"2023-10-10T20:34:34.336680+0000","last_active":"2023-10-10T20:34:35.336695+0000","last_peered":"2023-10-10T20:34:35.336695+0000","last_clean":"2023-10-10T20:34:35.336695+0000","last_became_active":"2023-10-10T20:34:34.335533+0000","last_became_peered":"2023-10-10T20:34:34.335533+0000","last_unstale":"2023-10-10T20:34:35.336695+0000","last_undegraded":"2023-10-10T20:34:35.336695+0000","last_fullsized":"2023-10-10T20:34:35.336695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349524+0000","last_change":"2023-10-10T20:34:35.342266+0000","last_active":"2023-10-10T20:34:35.349524+0000","last_peered":"2023-10-10T20:34:35.349524+0000","last_clean":"2023-10-10T20:34:35.349524+0000","last_became_active":"2023-10-10T20:34:35.341981+0000","last_became_peered":"2023-10-10T20:34:35.341981+0000","last_unstale":"2023-10-10T20:34:35.349524+0000","last_undegraded":"2023-10-10T20:34:35.349524+0000","last_fullsized":"2023-10-10T20:34:35.349524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:11.206264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.345007+0000","last_change":"2023-10-10T20:34:35.341780+0000","last_active":"2023-10-10T20:34:35.345007+0000","last_peered":"2023-10-10T20:34:35.345007+0000","last_clean":"2023-10-10T20:34:35.345007+0000","last_became_active":"2023-10-10T20:34:35.340539+0000","last_became_peered":"2023-10-10T20:34:35.340539+0000","last_unstale":"2023-10-10T20:34:35.345007+0000","last_undegraded":"2023-10-10T20:34:35.345007+0000","last_fullsized":"2023-10-10T20:34:35.345007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:12:13.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.328070+0000","last_change":"2023-10-10T20:34:34.340247+0000","last_active":"2023-10-10T20:34:35.328070+0000","last_peered":"2023-10-10T20:34:35.328070+0000","last_clean":"2023-10-10T20:34:35.328070+0000","last_became_active":"2023-10-10T20:34:34.339946+0000","last_became_peered":"2023-10-10T20:34:34.339946+0000","last_unstale":"2023-10-10T20:34:35.328070+0000","last_undegraded":"2023-10-10T20:34:35.328070+0000","last_fullsized":"2023-10-10T20:34:35.328070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350545+0000","last_change":"2023-10-10T20:34:35.344288+0000","last_active":"2023-10-10T20:34:35.350545+0000","last_peered":"2023-10-10T20:34:35.350545+0000","last_clean":"2023-10-10T20:34:35.350545+0000","last_became_active":"2023-10-10T20:34:35.343736+0000","last_became_peered":"2023-10-10T20:34:35.343736+0000","last_unstale":"2023-10-10T20:34:35.350545+0000","last_undegraded":"2023-10-10T20:34:35.350545+0000","last_fullsized":"2023-10-10T20:34:35.350545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:18.709140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.333336+0000","last_change":"2023-10-10T20:34:34.341870+0000","last_active":"2023-10-10T20:34:35.333336+0000","last_peered":"2023-10-10T20:34:35.333336+0000","last_clean":"2023-10-10T20:34:35.333336+0000","last_became_active":"2023-10-10T20:34:34.341567+0000","last_became_peered":"2023-10-10T20:34:34.341567+0000","last_unstale":"2023-10-10T20:34:35.333336+0000","last_undegraded":"2023-10-10T20:34:35.333336+0000","last_fullsized":"2023-10-10T20:34:35.333336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:11.616929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.327481+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:34:35.327481+0000","last_peered":"2023-10-10T20:34:35.327481+0000","last_clean":"2023-10-10T20:34:35.327481+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:34:35.327481+0000","last_undegraded":"2023-10-10T20:34:35.327481+0000","last_fullsized":"2023-10-10T20:34:35.327481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.344647+0000","last_change":"2023-10-10T20:34:35.340986+0000","last_active":"2023-10-10T20:34:35.344647+0000","last_peered":"2023-10-10T20:34:35.344647+0000","last_clean":"2023-10-10T20:34:35.344647+0000","last_became_active":"2023-10-10T20:34:35.339812+0000","last_became_peered":"2023-10-10T20:34:35.339812+0000","last_unstale":"2023-10-10T20:34:35.344647+0000","last_undegraded":"2023-10-10T20:34:35.344647+0000","last_fullsized":"2023-10-10T20:34:35.344647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:11:35.427062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.327681+0000","last_change":"2023-10-10T20:34:34.345132+0000","last_active":"2023-10-10T20:34:35.327681+0000","last_peered":"2023-10-10T20:34:35.327681+0000","last_clean":"2023-10-10T20:34:35.327681+0000","last_became_active":"2023-10-10T20:34:34.344843+0000","last_became_peered":"2023-10-10T20:34:34.344843+0000","last_unstale":"2023-10-10T20:34:35.327681+0000","last_undegraded":"2023-10-10T20:34:35.327681+0000","last_fullsized":"2023-10-10T20:34:35.327681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:15.030808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.328545+0000","last_change":"2023-10-10T20:34:34.346151+0000","last_active":"2023-10-10T20:34:35.328545+0000","last_peered":"2023-10-10T20:34:35.328545+0000","last_clean":"2023-10-10T20:34:35.328545+0000","last_became_active":"2023-10-10T20:34:34.345765+0000","last_became_peered":"2023-10-10T20:34:34.345765+0000","last_unstale":"2023-10-10T20:34:35.328545+0000","last_undegraded":"2023-10-10T20:34:35.328545+0000","last_fullsized":"2023-10-10T20:34:35.328545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:10.201619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342808+0000","last_change":"2023-10-10T20:34:35.340238+0000","last_active":"2023-10-10T20:34:35.342808+0000","last_peered":"2023-10-10T20:34:35.342808+0000","last_clean":"2023-10-10T20:34:35.342808+0000","last_became_active":"2023-10-10T20:34:35.339944+0000","last_became_peered":"2023-10-10T20:34:35.339944+0000","last_unstale":"2023-10-10T20:34:35.342808+0000","last_undegraded":"2023-10-10T20:34:35.342808+0000","last_fullsized":"2023-10-10T20:34:35.342808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.337406+0000","last_change":"2023-10-10T20:34:34.347897+0000","last_active":"2023-10-10T20:34:35.337406+0000","last_peered":"2023-10-10T20:34:35.337406+0000","last_clean":"2023-10-10T20:34:35.337406+0000","last_became_active":"2023-10-10T20:34:34.347575+0000","last_became_peered":"2023-10-10T20:34:34.347575+0000","last_unstale":"2023-10-10T20:34:35.337406+0000","last_undegraded":"2023-10-10T20:34:35.337406+0000","last_fullsized":"2023-10-10T20:34:35.337406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:15.030808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350573+0000","last_change":"2023-10-10T20:34:35.345207+0000","last_active":"2023-10-10T20:34:35.350573+0000","last_peered":"2023-10-10T20:34:35.350573+0000","last_clean":"2023-10-10T20:34:35.350573+0000","last_became_active":"2023-10-10T20:34:35.344809+0000","last_became_peered":"2023-10-10T20:34:35.344809+0000","last_unstale":"2023-10-10T20:34:35.350573+0000","last_undegraded":"2023-10-10T20:34:35.350573+0000","last_fullsized":"2023-10-10T20:34:35.350573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:36.295226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.358045+0000","last_change":"2023-10-10T20:34:34.348399+0000","last_active":"2023-10-10T20:34:35.358045+0000","last_peered":"2023-10-10T20:34:35.358045+0000","last_clean":"2023-10-10T20:34:35.358045+0000","last_became_active":"2023-10-10T20:34:34.348099+0000","last_became_peered":"2023-10-10T20:34:34.348099+0000","last_unstale":"2023-10-10T20:34:35.358045+0000","last_undegraded":"2023-10-10T20:34:35.358045+0000","last_fullsized":"2023-10-10T20:34:35.358045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:40:03.331998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.344533+0000","last_change":"2023-10-10T20:34:35.340459+0000","last_active":"2023-10-10T20:34:35.344533+0000","last_peered":"2023-10-10T20:34:35.344533+0000","last_clean":"2023-10-10T20:34:35.344533+0000","last_became_active":"2023-10-10T20:34:35.340176+0000","last_became_peered":"2023-10-10T20:34:35.340176+0000","last_unstale":"2023-10-10T20:34:35.344533+0000","last_undegraded":"2023-10-10T20:34:35.344533+0000","last_fullsized":"2023-10-10T20:34:35.344533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:48:12.831828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.327499+0000","last_change":"2023-10-10T20:34:34.346150+0000","last_active":"2023-10-10T20:34:35.327499+0000","last_peered":"2023-10-10T20:34:35.327499+0000","last_clean":"2023-10-10T20:34:35.327499+0000","last_became_active":"2023-10-10T20:34:34.345811+0000","last_became_peered":"2023-10-10T20:34:34.345811+0000","last_unstale":"2023-10-10T20:34:35.327499+0000","last_undegraded":"2023-10-10T20:34:35.327499+0000","last_fullsized":"2023-10-10T20:34:35.327499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:04.665843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350476+0000","last_change":"2023-10-10T20:34:35.345083+0000","last_active":"2023-10-10T20:34:35.350476+0000","last_peered":"2023-10-10T20:34:35.350476+0000","last_clean":"2023-10-10T20:34:35.350476+0000","last_became_active":"2023-10-10T20:34:35.343922+0000","last_became_peered":"2023-10-10T20:34:35.343922+0000","last_unstale":"2023-10-10T20:34:35.350476+0000","last_undegraded":"2023-10-10T20:34:35.350476+0000","last_fullsized":"2023-10-10T20:34:35.350476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:05.670488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.337189+0000","last_change":"2023-10-10T20:34:34.340327+0000","last_active":"2023-10-10T20:34:35.337189+0000","last_peered":"2023-10-10T20:34:35.337189+0000","last_clean":"2023-10-10T20:34:35.337189+0000","last_became_active":"2023-10-10T20:34:34.339737+0000","last_became_peered":"2023-10-10T20:34:34.339737+0000","last_unstale":"2023-10-10T20:34:35.337189+0000","last_undegraded":"2023-10-10T20:34:35.337189+0000","last_fullsized":"2023-10-10T20:34:35.337189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350500+0000","last_change":"2023-10-10T20:34:35.344039+0000","last_active":"2023-10-10T20:34:35.350500+0000","last_peered":"2023-10-10T20:34:35.350500+0000","last_clean":"2023-10-10T20:34:35.350500+0000","last_became_active":"2023-10-10T20:34:35.343789+0000","last_became_peered":"2023-10-10T20:34:35.343789+0000","last_unstale":"2023-10-10T20:34:35.350500+0000","last_undegraded":"2023-10-10T20:34:35.350500+0000","last_fullsized":"2023-10-10T20:34:35.350500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:11:35.427062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.337192+0000","last_change":"2023-10-10T20:34:34.340022+0000","last_active":"2023-10-10T20:34:35.337192+0000","last_peered":"2023-10-10T20:34:35.337192+0000","last_clean":"2023-10-10T20:34:35.337192+0000","last_became_active":"2023-10-10T20:34:34.339652+0000","last_became_peered":"2023-10-10T20:34:34.339652+0000","last_unstale":"2023-10-10T20:34:35.337192+0000","last_undegraded":"2023-10-10T20:34:35.337192+0000","last_fullsized":"2023-10-10T20:34:35.337192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350304+0000","last_change":"2023-10-10T20:34:35.342858+0000","last_active":"2023-10-10T20:34:35.350304+0000","last_peered":"2023-10-10T20:34:35.350304+0000","last_clean":"2023-10-10T20:34:35.350304+0000","last_became_active":"2023-10-10T20:34:35.341911+0000","last_became_peered":"2023-10-10T20:34:35.341911+0000","last_unstale":"2023-10-10T20:34:35.350304+0000","last_undegraded":"2023-10-10T20:34:35.350304+0000","last_fullsized":"2023-10-10T20:34:35.350304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:25:55.933179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.357218+0000","last_change":"2023-10-10T20:34:34.346661+0000","last_active":"2023-10-10T20:34:35.357218+0000","last_peered":"2023-10-10T20:34:35.357218+0000","last_clean":"2023-10-10T20:34:35.357218+0000","last_became_active":"2023-10-10T20:34:34.346384+0000","last_became_peered":"2023-10-10T20:34:34.346384+0000","last_unstale":"2023-10-10T20:34:35.357218+0000","last_undegraded":"2023-10-10T20:34:35.357218+0000","last_fullsized":"2023-10-10T20:34:35.357218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350769+0000","last_change":"2023-10-10T20:34:35.349300+0000","last_active":"2023-10-10T20:34:35.350769+0000","last_peered":"2023-10-10T20:34:35.350769+0000","last_clean":"2023-10-10T20:34:35.350769+0000","last_became_active":"2023-10-10T20:34:35.348997+0000","last_became_peered":"2023-10-10T20:34:35.348997+0000","last_unstale":"2023-10-10T20:34:35.350769+0000","last_undegraded":"2023-10-10T20:34:35.350769+0000","last_fullsized":"2023-10-10T20:34:35.350769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:47:45.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.328113+0000","last_change":"2023-10-10T20:34:34.344048+0000","last_active":"2023-10-10T20:34:35.328113+0000","last_peered":"2023-10-10T20:34:35.328113+0000","last_clean":"2023-10-10T20:34:35.328113+0000","last_became_active":"2023-10-10T20:34:34.343866+0000","last_became_peered":"2023-10-10T20:34:34.343866+0000","last_unstale":"2023-10-10T20:34:35.328113+0000","last_undegraded":"2023-10-10T20:34:35.328113+0000","last_fullsized":"2023-10-10T20:34:35.328113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:27:04.988955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342399+0000","last_change":"2023-10-10T20:34:35.337534+0000","last_active":"2023-10-10T20:34:35.342399+0000","last_peered":"2023-10-10T20:34:35.342399+0000","last_clean":"2023-10-10T20:34:35.342399+0000","last_became_active":"2023-10-10T20:34:35.337277+0000","last_became_peered":"2023-10-10T20:34:35.337277+0000","last_unstale":"2023-10-10T20:34:35.342399+0000","last_undegraded":"2023-10-10T20:34:35.342399+0000","last_fullsized":"2023-10-10T20:34:35.342399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:42:23.450966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.360813+0000","last_change":"2023-10-10T20:34:34.340837+0000","last_active":"2023-10-10T20:34:35.360813+0000","last_peered":"2023-10-10T20:34:35.360813+0000","last_clean":"2023-10-10T20:34:35.360813+0000","last_became_active":"2023-10-10T20:34:34.338971+0000","last_became_peered":"2023-10-10T20:34:34.338971+0000","last_unstale":"2023-10-10T20:34:35.360813+0000","last_undegraded":"2023-10-10T20:34:35.360813+0000","last_fullsized":"2023-10-10T20:34:35.360813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:10.201619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349519+0000","last_change":"2023-10-10T20:34:35.345474+0000","last_active":"2023-10-10T20:34:35.349519+0000","last_peered":"2023-10-10T20:34:35.349519+0000","last_clean":"2023-10-10T20:34:35.349519+0000","last_became_active":"2023-10-10T20:34:35.343255+0000","last_became_peered":"2023-10-10T20:34:35.343255+0000","last_unstale":"2023-10-10T20:34:35.349519+0000","last_undegraded":"2023-10-10T20:34:35.349519+0000","last_fullsized":"2023-10-10T20:34:35.349519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:35.499006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362947+0000","last_change":"2023-10-10T20:34:34.339953+0000","last_active":"2023-10-10T20:34:35.362947+0000","last_peered":"2023-10-10T20:34:35.362947+0000","last_clean":"2023-10-10T20:34:35.362947+0000","last_became_active":"2023-10-10T20:34:34.339676+0000","last_became_peered":"2023-10-10T20:34:34.339676+0000","last_unstale":"2023-10-10T20:34:35.362947+0000","last_undegraded":"2023-10-10T20:34:35.362947+0000","last_fullsized":"2023-10-10T20:34:35.362947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342687+0000","last_change":"2023-10-10T20:34:35.338535+0000","last_active":"2023-10-10T20:34:35.342687+0000","last_peered":"2023-10-10T20:34:35.342687+0000","last_clean":"2023-10-10T20:34:35.342687+0000","last_became_active":"2023-10-10T20:34:35.338266+0000","last_became_peered":"2023-10-10T20:34:35.338266+0000","last_unstale":"2023-10-10T20:34:35.342687+0000","last_undegraded":"2023-10-10T20:34:35.342687+0000","last_fullsized":"2023-10-10T20:34:35.342687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:39.183060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348224+0000","last_change":"2023-10-10T20:34:35.346165+0000","last_active":"2023-10-10T20:34:35.348224+0000","last_peered":"2023-10-10T20:34:35.348224+0000","last_clean":"2023-10-10T20:34:35.348224+0000","last_became_active":"2023-10-10T20:34:35.345862+0000","last_became_peered":"2023-10-10T20:34:35.345862+0000","last_unstale":"2023-10-10T20:34:35.348224+0000","last_undegraded":"2023-10-10T20:34:35.348224+0000","last_fullsized":"2023-10-10T20:34:35.348224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:18.709140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350401+0000","last_change":"2023-10-10T20:34:35.342709+0000","last_active":"2023-10-10T20:34:35.350401+0000","last_peered":"2023-10-10T20:34:35.350401+0000","last_clean":"2023-10-10T20:34:35.350401+0000","last_became_active":"2023-10-10T20:34:35.342433+0000","last_became_peered":"2023-10-10T20:34:35.342433+0000","last_unstale":"2023-10-10T20:34:35.350401+0000","last_undegraded":"2023-10-10T20:34:35.350401+0000","last_fullsized":"2023-10-10T20:34:35.350401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:48:12.831828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342070+0000","last_change":"2023-10-10T20:34:35.340426+0000","last_active":"2023-10-10T20:34:35.342070+0000","last_peered":"2023-10-10T20:34:35.342070+0000","last_clean":"2023-10-10T20:34:35.342070+0000","last_became_active":"2023-10-10T20:34:35.339966+0000","last_became_peered":"2023-10-10T20:34:35.339966+0000","last_unstale":"2023-10-10T20:34:35.342070+0000","last_undegraded":"2023-10-10T20:34:35.342070+0000","last_fullsized":"2023-10-10T20:34:35.342070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:42:23.450966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.351149+0000","last_change":"2023-10-10T20:34:35.341444+0000","last_active":"2023-10-10T20:34:35.351149+0000","last_peered":"2023-10-10T20:34:35.351149+0000","last_clean":"2023-10-10T20:34:35.351149+0000","last_became_active":"2023-10-10T20:34:35.341186+0000","last_became_peered":"2023-10-10T20:34:35.341186+0000","last_unstale":"2023-10-10T20:34:35.351149+0000","last_undegraded":"2023-10-10T20:34:35.351149+0000","last_fullsized":"2023-10-10T20:34:35.351149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352359+0000","last_change":"2023-10-10T20:34:35.347375+0000","last_active":"2023-10-10T20:34:35.352359+0000","last_peered":"2023-10-10T20:34:35.352359+0000","last_clean":"2023-10-10T20:34:35.352359+0000","last_became_active":"2023-10-10T20:34:35.347094+0000","last_became_peered":"2023-10-10T20:34:35.347094+0000","last_unstale":"2023-10-10T20:34:35.352359+0000","last_undegraded":"2023-10-10T20:34:35.352359+0000","last_fullsized":"2023-10-10T20:34:35.352359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:12:13.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342757+0000","last_change":"2023-10-10T20:34:35.340918+0000","last_active":"2023-10-10T20:34:35.342757+0000","last_peered":"2023-10-10T20:34:35.342757+0000","last_clean":"2023-10-10T20:34:35.342757+0000","last_became_active":"2023-10-10T20:34:35.340638+0000","last_became_peered":"2023-10-10T20:34:35.340638+0000","last_unstale":"2023-10-10T20:34:35.342757+0000","last_undegraded":"2023-10-10T20:34:35.342757+0000","last_fullsized":"2023-10-10T20:34:35.342757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:18.709140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348059+0000","last_change":"2023-10-10T20:34:35.345371+0000","last_active":"2023-10-10T20:34:35.348059+0000","last_peered":"2023-10-10T20:34:35.348059+0000","last_clean":"2023-10-10T20:34:35.348059+0000","last_became_active":"2023-10-10T20:34:35.345058+0000","last_became_peered":"2023-10-10T20:34:35.345058+0000","last_unstale":"2023-10-10T20:34:35.348059+0000","last_undegraded":"2023-10-10T20:34:35.348059+0000","last_fullsized":"2023-10-10T20:34:35.348059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352396+0000","last_change":"2023-10-10T20:34:35.349283+0000","last_active":"2023-10-10T20:34:35.352396+0000","last_peered":"2023-10-10T20:34:35.352396+0000","last_clean":"2023-10-10T20:34:35.352396+0000","last_became_active":"2023-10-10T20:34:35.349040+0000","last_became_peered":"2023-10-10T20:34:35.349040+0000","last_unstale":"2023-10-10T20:34:35.352396+0000","last_undegraded":"2023-10-10T20:34:35.352396+0000","last_fullsized":"2023-10-10T20:34:35.352396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:47:45.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349316+0000","last_change":"2023-10-10T20:34:35.343369+0000","last_active":"2023-10-10T20:34:35.349316+0000","last_peered":"2023-10-10T20:34:35.349316+0000","last_clean":"2023-10-10T20:34:35.349316+0000","last_became_active":"2023-10-10T20:34:35.342875+0000","last_became_peered":"2023-10-10T20:34:35.342875+0000","last_unstale":"2023-10-10T20:34:35.349316+0000","last_undegraded":"2023-10-10T20:34:35.349316+0000","last_fullsized":"2023-10-10T20:34:35.349316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:44:40.877955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352369+0000","last_change":"2023-10-10T20:34:35.349284+0000","last_active":"2023-10-10T20:34:35.352369+0000","last_peered":"2023-10-10T20:34:35.352369+0000","last_clean":"2023-10-10T20:34:35.352369+0000","last_became_active":"2023-10-10T20:34:35.349057+0000","last_became_peered":"2023-10-10T20:34:35.349057+0000","last_unstale":"2023-10-10T20:34:35.352369+0000","last_undegraded":"2023-10-10T20:34:35.352369+0000","last_fullsized":"2023-10-10T20:34:35.352369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:52:03.361718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.354153+0000","last_change":"2023-10-10T20:34:35.353504+0000","last_active":"2023-10-10T20:34:35.354153+0000","last_peered":"2023-10-10T20:34:35.354153+0000","last_clean":"2023-10-10T20:34:35.354153+0000","last_became_active":"2023-10-10T20:34:35.350145+0000","last_became_peered":"2023-10-10T20:34:35.350145+0000","last_unstale":"2023-10-10T20:34:35.354153+0000","last_undegraded":"2023-10-10T20:34:35.354153+0000","last_fullsized":"2023-10-10T20:34:35.354153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:52:03.361718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.351192+0000","last_change":"2023-10-10T20:34:35.345734+0000","last_active":"2023-10-10T20:34:35.351192+0000","last_peered":"2023-10-10T20:34:35.351192+0000","last_clean":"2023-10-10T20:34:35.351192+0000","last_became_active":"2023-10-10T20:34:35.345466+0000","last_became_peered":"2023-10-10T20:34:35.345466+0000","last_unstale":"2023-10-10T20:34:35.351192+0000","last_undegraded":"2023-10-10T20:34:35.351192+0000","last_fullsized":"2023-10-10T20:34:35.351192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:47:45.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350754+0000","last_change":"2023-10-10T20:34:35.349383+0000","last_active":"2023-10-10T20:34:35.350754+0000","last_peered":"2023-10-10T20:34:35.350754+0000","last_clean":"2023-10-10T20:34:35.350754+0000","last_became_active":"2023-10-10T20:34:35.348979+0000","last_became_peered":"2023-10-10T20:34:35.348979+0000","last_unstale":"2023-10-10T20:34:35.350754+0000","last_undegraded":"2023-10-10T20:34:35.350754+0000","last_fullsized":"2023-10-10T20:34:35.350754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:25:55.933179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298072,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073768,"statfs":{"total":96636764160,"available":96331538432,"internally_reserved":0,"allocated":802816,"data_stored":661576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":212992,"data_stored":70814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0640000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298076,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073764,"statfs":{"total":96636764160,"available":96331534336,"internally_reserved":0,"allocated":806912,"data_stored":660671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":70654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":70780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":70744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:34:39.762 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:34:39.763 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-10T20:34:39.763 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-10T20:34:39.763 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-10T20:34:39.764 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-10T20:34:39.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f5670d9f700 1 Processor -- start 2023-10-10T20:34:39.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f5670d9f700 1 -- start start 2023-10-10T20:34:39.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f5670d9f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f566c0aca80 0x7f566c06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:39.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f5670d9f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f566c069d60 0x7f566c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:39.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f5670d9f700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f566c0c81f0 0x7f566c068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:39.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f5670d9f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566c0c8670 con 0x7f566c0aca80 2023-10-10T20:34:39.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f5670d9f700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566c06bdb0 con 0x7f566c0c81f0 2023-10-10T20:34:39.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f5670d9f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566c0cb370 con 0x7f566c069d60 2023-10-10T20:34:39.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f566a59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f566c0aca80 0x7f566c06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:39.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f566a59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f566c0aca80 0x7f566c06aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53216/0 (socket says 172.21.15.89:53216) 2023-10-10T20:34:39.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f566a59c700 1 -- 172.21.15.89:0/2260007353 learned_addr learned my addr 172.21.15.89:0/2260007353 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:39.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f5669d9b700 1 --2- 172.21.15.89:0/2260007353 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f566c069d60 0x7f566c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:39.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f566ad9d700 1 --2- 172.21.15.89:0/2260007353 >> v2:172.21.15.111:3300/0 conn(0x7f566c0c81f0 0x7f566c068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:39.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f566a59c700 1 -- 172.21.15.89:0/2260007353 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f566c069d60 msgr2=0x7f566c0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:39.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f566a59c700 1 --2- 172.21.15.89:0/2260007353 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f566c069d60 0x7f566c0c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f566a59c700 1 -- 172.21.15.89:0/2260007353 >> v2:172.21.15.111:3300/0 conn(0x7f566c0c81f0 msgr2=0x7f566c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:39.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f566a59c700 1 --2- 172.21.15.89:0/2260007353 >> v2:172.21.15.111:3300/0 conn(0x7f566c0c81f0 0x7f566c068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f566a59c700 1 -- 172.21.15.89:0/2260007353 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566c159f60 con 0x7f566c0aca80 2023-10-10T20:34:39.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f566ad9d700 1 --2- 172.21.15.89:0/2260007353 >> v2:172.21.15.111:3300/0 conn(0x7f566c0c81f0 0x7f566c068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:39.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f566a59c700 1 --2- 172.21.15.89:0/2260007353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f566c0aca80 0x7f566c06aba0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f566c06d520 tx=0x7f565c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b334e77fdfc43003 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:39.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f566959a700 1 -- 172.21.15.89:0/2260007353 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c005c10 con 0x7f566c0aca80 2023-10-10T20:34:39.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f566959a700 1 -- 172.21.15.89:0/2260007353 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565c005d70 con 0x7f566c0aca80 2023-10-10T20:34:39.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.909+0000 7f566959a700 1 -- 172.21.15.89:0/2260007353 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c006260 con 0x7f566c0aca80 2023-10-10T20:34:39.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.910+0000 7f5670d9f700 1 -- 172.21.15.89:0/2260007353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f566c0aca80 msgr2=0x7f566c06aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:39.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.910+0000 7f5670d9f700 1 --2- 172.21.15.89:0/2260007353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f566c0aca80 0x7f566c06aba0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f566c06d520 tx=0x7f565c009220 comp rx=0 tx=0).stop 2023-10-10T20:34:39.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.910+0000 7f5670d9f700 1 -- 172.21.15.89:0/2260007353 shutdown_connections 2023-10-10T20:34:39.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.910+0000 7f5670d9f700 1 --2- 172.21.15.89:0/2260007353 >> v2:172.21.15.111:3300/0 conn(0x7f566c0c81f0 0x7f566c068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.910+0000 7f5670d9f700 1 --2- 172.21.15.89:0/2260007353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f566c0aca80 0x7f566c06aba0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.910+0000 7f5670d9f700 1 --2- 172.21.15.89:0/2260007353 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f566c069d60 0x7f566c0c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.910+0000 7f5670d9f700 1 -- 172.21.15.89:0/2260007353 >> 172.21.15.89:0/2260007353 conn(0x7f566c0cd670 msgr2=0x7f566c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:39.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.910+0000 7f5670d9f700 1 -- 172.21.15.89:0/2260007353 shutdown_connections 2023-10-10T20:34:39.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.910+0000 7f5670d9f700 1 -- 172.21.15.89:0/2260007353 wait complete. 2023-10-10T20:34:39.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.911+0000 7f5670d9f700 1 Processor -- start 2023-10-10T20:34:39.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.911+0000 7f5670d9f700 1 -- start start 2023-10-10T20:34:39.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.912+0000 7f5670d9f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f566c069d60 0x7f566c152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:39.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.912+0000 7f5670d9f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f566c0aca80 0x7f566c15b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:39.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.912+0000 7f5670d9f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f566c0c81f0 0x7f566c15bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:39.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.912+0000 7f5670d9f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566c0c50b0 con 0x7f566c0aca80 2023-10-10T20:34:39.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.912+0000 7f5670d9f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566c15c180 con 0x7f566c0c81f0 2023-10-10T20:34:39.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.912+0000 7f5670d9f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566c0c47c0 con 0x7f566c069d60 2023-10-10T20:34:39.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.912+0000 7f5669d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f566c0aca80 0x7f566c15b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:39.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.912+0000 7f5669d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f566c0aca80 0x7f566c15b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53228/0 (socket says 172.21.15.89:53228) 2023-10-10T20:34:39.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.912+0000 7f5669d9b700 1 -- 172.21.15.89:0/1750307948 learned_addr learned my addr 172.21.15.89:0/1750307948 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:39.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.912+0000 7f566a59c700 1 --2- 172.21.15.89:0/1750307948 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f566c069d60 0x7f566c152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:39.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.912+0000 7f566ad9d700 1 --2- 172.21.15.89:0/1750307948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f566c0c81f0 0x7f566c15bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:39.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.912+0000 7f5669d9b700 1 -- 172.21.15.89:0/1750307948 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f566c069d60 msgr2=0x7f566c152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:39.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.912+0000 7f5669d9b700 1 --2- 172.21.15.89:0/1750307948 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f566c069d60 0x7f566c152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.912+0000 7f5669d9b700 1 -- 172.21.15.89:0/1750307948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f566c0c81f0 msgr2=0x7f566c15bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:39.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.912+0000 7f5669d9b700 1 --2- 172.21.15.89:0/1750307948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f566c0c81f0 0x7f566c15bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:39.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.912+0000 7f5669d9b700 1 -- 172.21.15.89:0/1750307948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565c009480 con 0x7f566c0aca80 2023-10-10T20:34:39.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.912+0000 7f5669d9b700 1 --2- 172.21.15.89:0/1750307948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f566c0aca80 0x7f566c15b700 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f566000ec50 tx=0x7f56600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:39.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.912+0000 7f565a7fc700 1 -- 172.21.15.89:0/1750307948 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566000c850 con 0x7f566c0aca80 2023-10-10T20:34:39.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.913+0000 7f565a7fc700 1 -- 172.21.15.89:0/1750307948 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566000c9b0 con 0x7f566c0aca80 2023-10-10T20:34:39.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.913+0000 7f565a7fc700 1 -- 172.21.15.89:0/1750307948 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566000cd70 con 0x7f566c0aca80 2023-10-10T20:34:39.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.913+0000 7f5670d9f700 1 -- 172.21.15.89:0/1750307948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f566c0c4a60 con 0x7f566c0aca80 2023-10-10T20:34:39.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.913+0000 7f5670d9f700 1 -- 172.21.15.89:0/1750307948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f566c157d10 con 0x7f566c0aca80 2023-10-10T20:34:39.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.913+0000 7f5670d9f700 1 -- 172.21.15.89:0/1750307948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f566c04c3c0 con 0x7f566c0aca80 2023-10-10T20:34:39.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.914+0000 7f565a7fc700 1 -- 172.21.15.89:0/1750307948 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5660013680 con 0x7f566c0aca80 2023-10-10T20:34:39.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.914+0000 7f565a7fc700 1 --2- 172.21.15.89:0/1750307948 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f564c066f30 0x7f564c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:39.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.914+0000 7f565a7fc700 1 -- 172.21.15.89:0/1750307948 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5660084720 con 0x7f566c0aca80 2023-10-10T20:34:39.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.914+0000 7f566a59c700 1 --2- 172.21.15.89:0/1750307948 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f564c066f30 0x7f564c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:39.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.915+0000 7f566a59c700 1 --2- 172.21.15.89:0/1750307948 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f564c066f30 0x7f564c0693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f565c009260 tx=0x7f565c016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:39.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:39.919+0000 7f565a7fc700 1 -- 172.21.15.89:0/1750307948 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5660051990 con 0x7f566c0aca80 2023-10-10T20:34:40.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.157+0000 7f5670d9f700 1 -- 172.21.15.89:0/1750307948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f566c06b510 con 0x7f566c0aca80 2023-10-10T20:34:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.160+0000 7f565a7fc700 1 -- 172.21.15.89:0/1750307948 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163698 (secure 0 0 0) 0x7f5660054fb0 con 0x7f566c0aca80 2023-10-10T20:34:40.164 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:40.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.162+0000 7f5670d9f700 1 -- 172.21.15.89:0/1750307948 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f564c066f30 msgr2=0x7f564c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:40.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.162+0000 7f5670d9f700 1 --2- 172.21.15.89:0/1750307948 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f564c066f30 0x7f564c0693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f565c009260 tx=0x7f565c016000 comp rx=0 tx=0).stop 2023-10-10T20:34:40.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.162+0000 7f5670d9f700 1 -- 172.21.15.89:0/1750307948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f566c0aca80 msgr2=0x7f566c15b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.162+0000 7f5670d9f700 1 --2- 172.21.15.89:0/1750307948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f566c0aca80 0x7f566c15b700 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f566000ec50 tx=0x7f56600091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.162+0000 7f5670d9f700 1 -- 172.21.15.89:0/1750307948 shutdown_connections 2023-10-10T20:34:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.162+0000 7f5670d9f700 1 --2- 172.21.15.89:0/1750307948 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f564c066f30 0x7f564c0693e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.162+0000 7f5670d9f700 1 --2- 172.21.15.89:0/1750307948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f566c0c81f0 0x7f566c15bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.162+0000 7f5670d9f700 1 --2- 172.21.15.89:0/1750307948 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f566c069d60 0x7f566c152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.162+0000 7f5670d9f700 1 --2- 172.21.15.89:0/1750307948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f566c0aca80 0x7f566c15b700 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.162+0000 7f5670d9f700 1 -- 172.21.15.89:0/1750307948 >> 172.21.15.89:0/1750307948 conn(0x7f566c0cd670 msgr2=0x7f566c158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:40.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.162+0000 7f5670d9f700 1 -- 172.21.15.89:0/1750307948 shutdown_connections 2023-10-10T20:34:40.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.162+0000 7f5670d9f700 1 -- 172.21.15.89:0/1750307948 wait complete. 2023-10-10T20:34:40.176 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":5,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":90665},{"type":"v1","addr":"172.21.15.89:6833","nonce":90665}]},"active_addr":"172.21.15.89:6833/90665","active_change":"2023-10-10T20:34:22.644949+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":3258966449}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":727686291}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":2062672636}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":3769670225}]}]} 2023-10-10T20:34:40.180 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-10T20:34:40.180 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-10T20:34:40.181 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:34:40.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.293+0000 7fba18e4d700 1 Processor -- start 2023-10-10T20:34:40.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.293+0000 7fba18e4d700 1 -- start start 2023-10-10T20:34:40.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.294+0000 7fba18e4d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba14159890 0x7fba1415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:40.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.294+0000 7fba18e4d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fba14069560 0x7fba140699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:40.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.294+0000 7fba18e4d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1415a160 con 0x7fba14159c60 2023-10-10T20:34:40.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.294+0000 7fba18e4d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba14069f20 con 0x7fba14069560 2023-10-10T20:34:40.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.294+0000 7fba18e4d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba140d13a0 con 0x7fba14159890 2023-10-10T20:34:40.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.294+0000 7fba1259c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba14159890 0x7fba1415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:40.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.294+0000 7fba12d9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fba14069560 0x7fba140699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:40.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.294+0000 7fba1259c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba14159890 0x7fba1415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:43744/0 (socket says 172.21.15.89:43744) 2023-10-10T20:34:40.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.294+0000 7fba1259c700 1 -- 172.21.15.89:0/3115994184 learned_addr learned my addr 172.21.15.89:0/3115994184 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:40.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.294+0000 7fba1259c700 1 -- 172.21.15.89:0/3115994184 >> v2:172.21.15.111:3300/0 conn(0x7fba14069560 msgr2=0x7fba140699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:40.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.294+0000 7fba1259c700 1 --2- 172.21.15.89:0/3115994184 >> v2:172.21.15.111:3300/0 conn(0x7fba14069560 0x7fba140699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.294+0000 7fba1259c700 1 -- 172.21.15.89:0/3115994184 >> v1:172.21.15.111:6789/0 conn(0x7fba14159c60 legacy=0x7fba1415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:40.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.294+0000 7fba1259c700 1 -- 172.21.15.89:0/3115994184 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1415e140 con 0x7fba14159890 2023-10-10T20:34:40.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.295+0000 7fba1259c700 1 --2- 172.21.15.89:0/3115994184 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba14159890 0x7fba1415abd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0099b0 tx=0x7fba0c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a083e9bde6c9b82 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:40.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.295+0000 7fba1159a700 1 -- 172.21.15.89:0/3115994184 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c009ea0 con 0x7fba14159890 2023-10-10T20:34:40.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.295+0000 7fba1159a700 1 -- 172.21.15.89:0/3115994184 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0c00b860 con 0x7fba14159890 2023-10-10T20:34:40.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.295+0000 7fba1159a700 1 -- 172.21.15.89:0/3115994184 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c00bc00 con 0x7fba14159890 2023-10-10T20:34:40.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.295+0000 7fba18e4d700 1 -- 172.21.15.89:0/3115994184 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba14159890 msgr2=0x7fba1415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:40.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.295+0000 7fba18e4d700 1 --2- 172.21.15.89:0/3115994184 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba14159890 0x7fba1415abd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0099b0 tx=0x7fba0c009250 comp rx=0 tx=0).stop 2023-10-10T20:34:40.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.295+0000 7fba18e4d700 1 -- 172.21.15.89:0/3115994184 shutdown_connections 2023-10-10T20:34:40.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.295+0000 7fba18e4d700 1 --2- 172.21.15.89:0/3115994184 >> v2:172.21.15.111:3300/0 conn(0x7fba14069560 0x7fba140699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.295+0000 7fba18e4d700 1 --2- 172.21.15.89:0/3115994184 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba14159890 0x7fba1415abd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.295+0000 7fba18e4d700 1 -- 172.21.15.89:0/3115994184 >> 172.21.15.89:0/3115994184 conn(0x7fba14068e20 msgr2=0x7fba140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:40.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.295+0000 7fba18e4d700 1 -- 172.21.15.89:0/3115994184 shutdown_connections 2023-10-10T20:34:40.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.295+0000 7fba18e4d700 1 -- 172.21.15.89:0/3115994184 wait complete. 2023-10-10T20:34:40.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.296+0000 7fba18e4d700 1 Processor -- start 2023-10-10T20:34:40.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.296+0000 7fba18e4d700 1 -- start start 2023-10-10T20:34:40.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.297+0000 7fba18e4d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba14069560 0x7fba141504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:40.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.297+0000 7fba18e4d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba14159890 0x7fba14141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:40.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.297+0000 7fba18e4d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba14159c60 0x7fba1415d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:40.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.297+0000 7fba18e4d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba140b3e70 con 0x7fba14069560 2023-10-10T20:34:40.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.297+0000 7fba18e4d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba140b3860 con 0x7fba14159890 2023-10-10T20:34:40.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.297+0000 7fba18e4d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba14150e90 con 0x7fba14159c60 2023-10-10T20:34:40.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.297+0000 7fba1259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba14069560 0x7fba141504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:40.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.297+0000 7fba1259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba14069560 0x7fba141504d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53230/0 (socket says 172.21.15.89:53230) 2023-10-10T20:34:40.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.297+0000 7fba1259c700 1 -- 172.21.15.89:0/3872163048 learned_addr learned my addr 172.21.15.89:0/3872163048 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:40.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.297+0000 7fba12d9d700 1 --2- 172.21.15.89:0/3872163048 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba14159c60 0x7fba1415d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:40.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.297+0000 7fba11d9b700 1 --2- 172.21.15.89:0/3872163048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba14159890 0x7fba14141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:40.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.298+0000 7fba1259c700 1 -- 172.21.15.89:0/3872163048 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba14159c60 msgr2=0x7fba1415d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:40.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.298+0000 7fba1259c700 1 --2- 172.21.15.89:0/3872163048 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba14159c60 0x7fba1415d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.298+0000 7fba1259c700 1 -- 172.21.15.89:0/3872163048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba14159890 msgr2=0x7fba14141fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:40.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.298+0000 7fba1259c700 1 --2- 172.21.15.89:0/3872163048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba14159890 0x7fba14141fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.298+0000 7fba1259c700 1 -- 172.21.15.89:0/3872163048 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0c0094f0 con 0x7fba14069560 2023-10-10T20:34:40.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.298+0000 7fba11d9b700 1 --2- 172.21.15.89:0/3872163048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba14159890 0x7fba14141fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:40.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.298+0000 7fba1259c700 1 --2- 172.21.15.89:0/3872163048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba14069560 0x7fba141504d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fba14070e20 tx=0x7fba0c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:40.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.298+0000 7fba0a7fc700 1 -- 172.21.15.89:0/3872163048 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c00bbe0 con 0x7fba14069560 2023-10-10T20:34:40.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.298+0000 7fba0a7fc700 1 -- 172.21.15.89:0/3872163048 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0c011420 con 0x7fba14069560 2023-10-10T20:34:40.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.298+0000 7fba18e4d700 1 -- 172.21.15.89:0/3872163048 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba140b3b00 con 0x7fba14069560 2023-10-10T20:34:40.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.298+0000 7fba0a7fc700 1 -- 172.21.15.89:0/3872163048 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c0117c0 con 0x7fba14069560 2023-10-10T20:34:40.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.298+0000 7fba18e4d700 1 -- 172.21.15.89:0/3872163048 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba140b6100 con 0x7fba14069560 2023-10-10T20:34:40.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.299+0000 7fba18e4d700 1 -- 172.21.15.89:0/3872163048 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1404c3c0 con 0x7fba14069560 2023-10-10T20:34:40.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.299+0000 7fba0a7fc700 1 -- 172.21.15.89:0/3872163048 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba0c011bf0 con 0x7fba14069560 2023-10-10T20:34:40.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.300+0000 7fba0a7fc700 1 --2- 172.21.15.89:0/3872163048 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb9f4066f30 0x7fb9f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:40.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.300+0000 7fba0a7fc700 1 -- 172.21.15.89:0/3872163048 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fba0c085d80 con 0x7fba14069560 2023-10-10T20:34:40.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.300+0000 7fba11d9b700 1 --2- 172.21.15.89:0/3872163048 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb9f4066f30 0x7fb9f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:40.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.300+0000 7fba11d9b700 1 --2- 172.21.15.89:0/3872163048 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb9f4066f30 0x7fb9f40693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc007fd0 tx=0x7fb9fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:40.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.304+0000 7fba0a7fc700 1 -- 172.21.15.89:0/3872163048 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba0c052ff0 con 0x7fba14069560 2023-10-10T20:34:40.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.487+0000 7fba18e4d700 1 -- 172.21.15.89:0/3872163048 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba1406a5b0 con 0x7fba14069560 2023-10-10T20:34:40.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.488+0000 7fba0a7fc700 1 -- 172.21.15.89:0/3872163048 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fba0c056610 con 0x7fba14069560 2023-10-10T20:34:40.490 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:40.491 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:36.324920+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:34:32.342167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:34:33.671763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:34:40.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.490+0000 7fba18e4d700 1 -- 172.21.15.89:0/3872163048 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb9f4066f30 msgr2=0x7fb9f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:40.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.490+0000 7fba18e4d700 1 --2- 172.21.15.89:0/3872163048 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb9f4066f30 0x7fb9f40693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc007fd0 tx=0x7fb9fc009040 comp rx=0 tx=0).stop 2023-10-10T20:34:40.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.490+0000 7fba18e4d700 1 -- 172.21.15.89:0/3872163048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba14069560 msgr2=0x7fba141504d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:40.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.490+0000 7fba18e4d700 1 --2- 172.21.15.89:0/3872163048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba14069560 0x7fba141504d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fba14070e20 tx=0x7fba0c017000 comp rx=0 tx=0).stop 2023-10-10T20:34:40.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.490+0000 7fba18e4d700 1 -- 172.21.15.89:0/3872163048 shutdown_connections 2023-10-10T20:34:40.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.490+0000 7fba18e4d700 1 --2- 172.21.15.89:0/3872163048 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fba14159c60 0x7fba1415d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.490+0000 7fba18e4d700 1 --2- 172.21.15.89:0/3872163048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba14069560 0x7fba141504d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.490+0000 7fba18e4d700 1 --2- 172.21.15.89:0/3872163048 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb9f4066f30 0x7fb9f40693e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.490+0000 7fba18e4d700 1 --2- 172.21.15.89:0/3872163048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba14159890 0x7fba14141fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.490+0000 7fba18e4d700 1 -- 172.21.15.89:0/3872163048 >> 172.21.15.89:0/3872163048 conn(0x7fba14068e20 msgr2=0x7fba140cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:40.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.490+0000 7fba18e4d700 1 -- 172.21.15.89:0/3872163048 shutdown_connections 2023-10-10T20:34:40.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.490+0000 7fba18e4d700 1 -- 172.21.15.89:0/3872163048 wait complete. 2023-10-10T20:34:40.504 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-10T20:34:40.504 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:34:40.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.617+0000 7f19b4560700 1 Processor -- start 2023-10-10T20:34:40.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.617+0000 7f19b4560700 1 -- start start 2023-10-10T20:34:40.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.618+0000 7f19b4560700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19ac159890 0x7f19ac15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:40.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.618+0000 7f19b4560700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f19ac069560 0x7f19ac0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:40.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.618+0000 7f19b4560700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ac15a160 con 0x7f19ac159c60 2023-10-10T20:34:40.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.618+0000 7f19b4560700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ac069f20 con 0x7f19ac069560 2023-10-10T20:34:40.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.618+0000 7f19b4560700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac0d13a0 con 0x7f19ac159890 2023-10-10T20:34:40.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.618+0000 7f19b2afd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f19ac069560 0x7f19ac0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:40.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.618+0000 7f19b2afd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f19ac069560 0x7f19ac0699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53240/0 (socket says 172.21.15.89:53240) 2023-10-10T20:34:40.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.618+0000 7f19b2afd700 1 -- 172.21.15.89:0/1438315988 learned_addr learned my addr 172.21.15.89:0/1438315988 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:40.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.618+0000 7f19b22fc700 1 --2- 172.21.15.89:0/1438315988 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19ac159890 0x7f19ac15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:40.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.618+0000 7f19b2afd700 1 -- 172.21.15.89:0/1438315988 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19ac159890 msgr2=0x7f19ac15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:40.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.618+0000 7f19b2afd700 1 --2- 172.21.15.89:0/1438315988 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19ac159890 0x7f19ac15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.618+0000 7f19b2afd700 1 -- 172.21.15.89:0/1438315988 >> v1:172.21.15.111:6789/0 conn(0x7f19ac159c60 legacy=0x7f19ac15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:40.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.618+0000 7f19b2afd700 1 -- 172.21.15.89:0/1438315988 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac15e140 con 0x7f19ac069560 2023-10-10T20:34:40.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.618+0000 7f19b2afd700 1 --2- 172.21.15.89:0/1438315988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19ac069560 0x7f19ac0699e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009f20 tx=0x7f19a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804b25f40a2723ac server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:40.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.618+0000 7f19b12fa700 1 -- 172.21.15.89:0/1438315988 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a400e070 con 0x7f19ac069560 2023-10-10T20:34:40.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.618+0000 7f19b12fa700 1 -- 172.21.15.89:0/1438315988 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a400ba40 con 0x7f19ac069560 2023-10-10T20:34:40.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.618+0000 7f19b12fa700 1 -- 172.21.15.89:0/1438315988 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a40092e0 con 0x7f19ac069560 2023-10-10T20:34:40.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.619+0000 7f19b4560700 1 -- 172.21.15.89:0/1438315988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19ac069560 msgr2=0x7f19ac0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:40.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.619+0000 7f19b4560700 1 --2- 172.21.15.89:0/1438315988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19ac069560 0x7f19ac0699e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009f20 tx=0x7f19a4009840 comp rx=0 tx=0).stop 2023-10-10T20:34:40.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.619+0000 7f19b4560700 1 -- 172.21.15.89:0/1438315988 shutdown_connections 2023-10-10T20:34:40.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.619+0000 7f19b4560700 1 --2- 172.21.15.89:0/1438315988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19ac069560 0x7f19ac0699e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.619+0000 7f19b4560700 1 --2- 172.21.15.89:0/1438315988 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19ac159890 0x7f19ac15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.619+0000 7f19b4560700 1 -- 172.21.15.89:0/1438315988 >> 172.21.15.89:0/1438315988 conn(0x7f19ac068e20 msgr2=0x7f19ac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:40.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.619+0000 7f19b4560700 1 -- 172.21.15.89:0/1438315988 shutdown_connections 2023-10-10T20:34:40.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.619+0000 7f19b4560700 1 -- 172.21.15.89:0/1438315988 wait complete. 2023-10-10T20:34:40.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.620+0000 7f19b4560700 1 Processor -- start 2023-10-10T20:34:40.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.620+0000 7f19b4560700 1 -- start start 2023-10-10T20:34:40.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19b4560700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19ac069560 0x7f19ac153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:40.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19b4560700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19ac159890 0x7f19ac16d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:40.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19b4560700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19ac159c60 0x7f19ac16d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:40.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19b4560700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac154030 con 0x7f19ac069560 2023-10-10T20:34:40.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19b4560700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac154310 con 0x7f19ac159890 2023-10-10T20:34:40.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19b4560700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac1545f0 con 0x7f19ac159c60 2023-10-10T20:34:40.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19b22fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19ac069560 0x7f19ac153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:40.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19b22fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19ac069560 0x7f19ac153730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53242/0 (socket says 172.21.15.89:53242) 2023-10-10T20:34:40.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19b22fc700 1 -- 172.21.15.89:0/4241182123 learned_addr learned my addr 172.21.15.89:0/4241182123 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:40.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19b2afd700 1 --2- 172.21.15.89:0/4241182123 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19ac159c60 0x7f19ac16d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:40.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19b1afb700 1 --2- 172.21.15.89:0/4241182123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19ac159890 0x7f19ac16d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:40.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19b22fc700 1 -- 172.21.15.89:0/4241182123 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19ac159c60 msgr2=0x7f19ac16d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:40.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19b22fc700 1 --2- 172.21.15.89:0/4241182123 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19ac159c60 0x7f19ac16d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19b22fc700 1 -- 172.21.15.89:0/4241182123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19ac159890 msgr2=0x7f19ac16d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:40.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19b22fc700 1 --2- 172.21.15.89:0/4241182123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19ac159890 0x7f19ac16d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19b22fc700 1 -- 172.21.15.89:0/4241182123 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a4009ae0 con 0x7f19ac069560 2023-10-10T20:34:40.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19b22fc700 1 --2- 172.21.15.89:0/4241182123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19ac069560 0x7f19ac153730 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f19a800d890 tx=0x7f19a8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:40.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19a27fc700 1 -- 172.21.15.89:0/4241182123 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a80125f0 con 0x7f19ac069560 2023-10-10T20:34:40.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19a27fc700 1 -- 172.21.15.89:0/4241182123 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a8012750 con 0x7f19ac069560 2023-10-10T20:34:40.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19b4560700 1 -- 172.21.15.89:0/4241182123 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ac1548d0 con 0x7f19ac069560 2023-10-10T20:34:40.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19a27fc700 1 -- 172.21.15.89:0/4241182123 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a8012af0 con 0x7f19ac069560 2023-10-10T20:34:40.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.621+0000 7f19b4560700 1 -- 172.21.15.89:0/4241182123 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ac153c70 con 0x7f19ac069560 2023-10-10T20:34:40.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.622+0000 7f19b4560700 1 -- 172.21.15.89:0/4241182123 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ac154f20 con 0x7f19ac069560 2023-10-10T20:34:40.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.623+0000 7f19a27fc700 1 -- 172.21.15.89:0/4241182123 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f19a801e460 con 0x7f19ac069560 2023-10-10T20:34:40.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.623+0000 7f19a27fc700 1 --2- 172.21.15.89:0/4241182123 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1990066f30 0x7f19900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:40.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.623+0000 7f19a27fc700 1 -- 172.21.15.89:0/4241182123 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f19a8084aa0 con 0x7f19ac069560 2023-10-10T20:34:40.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.623+0000 7f19b1afb700 1 --2- 172.21.15.89:0/4241182123 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1990066f30 0x7f19900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:40.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.623+0000 7f19b1afb700 1 --2- 172.21.15.89:0/4241182123 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1990066f30 0x7f19900693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f19ac152880 tx=0x7f199c007ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:40.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.627+0000 7f19a27fc700 1 -- 172.21.15.89:0/4241182123 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19a8051d10 con 0x7f19ac069560 2023-10-10T20:34:40.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.813+0000 7f19b4560700 1 -- 172.21.15.89:0/4241182123 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19ac04c3c0 con 0x7f19ac069560 2023-10-10T20:34:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.814+0000 7f19a27fc700 1 -- 172.21.15.89:0/4241182123 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f19a8055330 con 0x7f19ac069560 2023-10-10T20:34:40.816 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:40.817 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:36.324920+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:34:32.342167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:34:33.671763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:34:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.816+0000 7f19b4560700 1 -- 172.21.15.89:0/4241182123 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1990066f30 msgr2=0x7f19900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.816+0000 7f19b4560700 1 --2- 172.21.15.89:0/4241182123 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1990066f30 0x7f19900693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f19ac152880 tx=0x7f199c007ea0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.816+0000 7f19b4560700 1 -- 172.21.15.89:0/4241182123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19ac069560 msgr2=0x7f19ac153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.816+0000 7f19b4560700 1 --2- 172.21.15.89:0/4241182123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19ac069560 0x7f19ac153730 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f19a800d890 tx=0x7f19a8008040 comp rx=0 tx=0).stop 2023-10-10T20:34:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.816+0000 7f19b4560700 1 -- 172.21.15.89:0/4241182123 shutdown_connections 2023-10-10T20:34:40.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.816+0000 7f19b4560700 1 --2- 172.21.15.89:0/4241182123 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19ac159c60 0x7f19ac16d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.816+0000 7f19b4560700 1 --2- 172.21.15.89:0/4241182123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19ac069560 0x7f19ac153730 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.816+0000 7f19b4560700 1 --2- 172.21.15.89:0/4241182123 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1990066f30 0x7f19900693e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.816+0000 7f19b4560700 1 --2- 172.21.15.89:0/4241182123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19ac159890 0x7f19ac16d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:40.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.816+0000 7f19b4560700 1 -- 172.21.15.89:0/4241182123 >> 172.21.15.89:0/4241182123 conn(0x7f19ac068e20 msgr2=0x7f19ac156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:40.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.816+0000 7f19b4560700 1 -- 172.21.15.89:0/4241182123 shutdown_connections 2023-10-10T20:34:40.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:40.816+0000 7f19b4560700 1 -- 172.21.15.89:0/4241182123 wait complete. 2023-10-10T20:34:40.831 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-10T20:34:40.831 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-10T20:34:40.832 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-10T20:34:40.832 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-10T20:34:40.833 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-10T20:34:40.833 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-10T20:34:40.834 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-10T20:34:40.835 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-10T20:34:41.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.017+0000 7f7290c29700 1 Processor -- start 2023-10-10T20:34:41.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.017+0000 7f7290c29700 1 -- start start 2023-10-10T20:34:41.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.018+0000 7f7290c29700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f728c06b830 0x7f728c0c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.018+0000 7f7290c29700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f728c0cb440 0x7f728c161b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.018+0000 7f7290c29700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f728c0c74b0 con 0x7f728c162080 2023-10-10T20:34:41.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.018+0000 7f7290c29700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728c06bc00 con 0x7f728c0cb440 2023-10-10T20:34:41.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.018+0000 7f7290c29700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728c158a30 con 0x7f728c06b830 2023-10-10T20:34:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.019+0000 7f7289d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f728c0cb440 0x7f728c161b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.019+0000 7f7289d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f728c0cb440 0x7f728c161b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53254/0 (socket says 172.21.15.89:53254) 2023-10-10T20:34:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.019+0000 7f7289d9b700 1 -- 172.21.15.89:0/1310326836 learned_addr learned my addr 172.21.15.89:0/1310326836 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.019+0000 7f728a59c700 1 --2- 172.21.15.89:0/1310326836 >> v2:172.21.15.111:3300/0 conn(0x7f728c06b830 0x7f728c0c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.019+0000 7f728a59c700 1 -- 172.21.15.89:0/1310326836 >> v2:172.21.15.89:3300/0 conn(0x7f728c0cb440 msgr2=0x7f728c161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.019+0000 7f728a59c700 1 --2- 172.21.15.89:0/1310326836 >> v2:172.21.15.89:3300/0 conn(0x7f728c0cb440 0x7f728c161b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.019+0000 7f728a59c700 1 -- 172.21.15.89:0/1310326836 >> v1:172.21.15.89:6789/0 conn(0x7f728c162080 legacy=0x7f728c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.019+0000 7f728a59c700 1 -- 172.21.15.89:0/1310326836 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c162480 con 0x7f728c06b830 2023-10-10T20:34:41.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.019+0000 7f728a59c700 1 --2- 172.21.15.89:0/1310326836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f728c06b830 0x7f728c0c8f50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f728c063e90 tx=0x7f72800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=236bb5b0db1c7e26 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.021+0000 7f728959a700 1 -- 172.21.15.89:0/1310326836 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728000b690 con 0x7f728c06b830 2023-10-10T20:34:41.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.021+0000 7f728959a700 1 -- 172.21.15.89:0/1310326836 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728000b7f0 con 0x7f728c06b830 2023-10-10T20:34:41.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.021+0000 7f728959a700 1 -- 172.21.15.89:0/1310326836 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728000bb90 con 0x7f728c06b830 2023-10-10T20:34:41.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.022+0000 7f7290c29700 1 -- 172.21.15.89:0/1310326836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f728c06b830 msgr2=0x7f728c0c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.022+0000 7f7290c29700 1 --2- 172.21.15.89:0/1310326836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f728c06b830 0x7f728c0c8f50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f728c063e90 tx=0x7f72800091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.022+0000 7f7290c29700 1 -- 172.21.15.89:0/1310326836 shutdown_connections 2023-10-10T20:34:41.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.022+0000 7f7290c29700 1 --2- 172.21.15.89:0/1310326836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f728c06b830 0x7f728c0c8f50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.022+0000 7f7290c29700 1 --2- 172.21.15.89:0/1310326836 >> v2:172.21.15.89:3300/0 conn(0x7f728c0cb440 0x7f728c161b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.022+0000 7f7290c29700 1 -- 172.21.15.89:0/1310326836 >> 172.21.15.89:0/1310326836 conn(0x7f728c0c9870 msgr2=0x7f728c0c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.023+0000 7f7290c29700 1 -- 172.21.15.89:0/1310326836 shutdown_connections 2023-10-10T20:34:41.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.023+0000 7f7290c29700 1 -- 172.21.15.89:0/1310326836 wait complete. 2023-10-10T20:34:41.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.024+0000 7f7290c29700 1 Processor -- start 2023-10-10T20:34:41.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.024+0000 7f7290c29700 1 -- start start 2023-10-10T20:34:41.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.025+0000 7f7290c29700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f728c06b830 0x7f728c14b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.025+0000 7f7290c29700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f728c0cb440 0x7f728c0afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.025+0000 7f7290c29700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f728c162080 0x7f728c15ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.026+0000 7f7290c29700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c0b15b0 con 0x7f728c162080 2023-10-10T20:34:41.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.026+0000 7f7290c29700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c0b1120 con 0x7f728c06b830 2023-10-10T20:34:41.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.026+0000 7f7290c29700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728c0b0510 con 0x7f728c0cb440 2023-10-10T20:34:41.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.026+0000 7f728ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f728c162080 0x7f728c15ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.026+0000 7f728ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f728c162080 0x7f728c15ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53266/0 (socket says 172.21.15.89:53266) 2023-10-10T20:34:41.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.026+0000 7f728ad9d700 1 -- 172.21.15.89:0/1257067829 learned_addr learned my addr 172.21.15.89:0/1257067829 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.026+0000 7f728a59c700 1 --2- 172.21.15.89:0/1257067829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f728c06b830 0x7f728c14b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.026+0000 7f728ad9d700 1 -- 172.21.15.89:0/1257067829 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f728c0cb440 msgr2=0x7f728c0afba0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-10T20:34:41.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.026+0000 7f728ad9d700 1 --2- 172.21.15.89:0/1257067829 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f728c0cb440 0x7f728c0afba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.026+0000 7f728ad9d700 1 -- 172.21.15.89:0/1257067829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f728c06b830 msgr2=0x7f728c14b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.026+0000 7f728ad9d700 1 --2- 172.21.15.89:0/1257067829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f728c06b830 0x7f728c14b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.026+0000 7f728ad9d700 1 -- 172.21.15.89:0/1257067829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7280009420 con 0x7f728c162080 2023-10-10T20:34:41.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.026+0000 7f728ad9d700 1 --2- 172.21.15.89:0/1257067829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f728c162080 0x7f728c15ff20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f728400c8e0 tx=0x7f7284001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.027+0000 7f727a7fc700 1 -- 172.21.15.89:0/1257067829 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7284012630 con 0x7f728c162080 2023-10-10T20:34:41.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.027+0000 7f727a7fc700 1 -- 172.21.15.89:0/1257067829 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7284012790 con 0x7f728c162080 2023-10-10T20:34:41.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.027+0000 7f727a7fc700 1 -- 172.21.15.89:0/1257067829 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7284012b50 con 0x7f728c162080 2023-10-10T20:34:41.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.027+0000 7f7290c29700 1 -- 172.21.15.89:0/1257067829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728c0b07f0 con 0x7f728c162080 2023-10-10T20:34:41.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.027+0000 7f7290c29700 1 -- 172.21.15.89:0/1257067829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728c0b0db0 con 0x7f728c162080 2023-10-10T20:34:41.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.029+0000 7f31c571c700 1 Processor -- start 2023-10-10T20:34:41.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.029+0000 7f31c571c700 1 -- start start 2023-10-10T20:34:41.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.029+0000 7f7290c29700 1 -- 172.21.15.89:0/1257067829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7270000fd0 con 0x7f728c162080 2023-10-10T20:34:41.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.029+0000 7f727a7fc700 1 -- 172.21.15.89:0/1257067829 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f728400a3b0 con 0x7f728c162080 2023-10-10T20:34:41.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.030+0000 7f727a7fc700 1 --2- 172.21.15.89:0/1257067829 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7274067120 0x7f72740695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.030+0000 7f727a7fc700 1 -- 172.21.15.89:0/1257067829 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7284084880 con 0x7f728c162080 2023-10-10T20:34:41.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.030+0000 7f727a7fc700 1 --2- 172.21.15.89:0/1257067829 >> [v2:172.21.15.89:6800/90896,v1:172.21.15.89:6801/90896] conn(0x7f727406f330 0x7f7274071740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.030+0000 7f727a7fc700 1 -- 172.21.15.89:0/1257067829 --> [v2:172.21.15.89:6800/90896,v1:172.21.15.89:6801/90896] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7274071df0 con 0x7f727406f330 2023-10-10T20:34:41.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.030+0000 7f727a7fc700 1 -- 172.21.15.89:0/1257067829 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7284089220 con 0x7f728c162080 2023-10-10T20:34:41.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.030+0000 7f31c571c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31c01579c0 0x7f31c00abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.030+0000 7f31c571c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31c00acfe0 con 0x7f31c0068aa0 2023-10-10T20:34:41.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.030+0000 7f31c571c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31c00ac3d0 con 0x7f31c01575f0 2023-10-10T20:34:41.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.030+0000 7f31c571c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c00ac6b0 con 0x7f31c01579c0 2023-10-10T20:34:41.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.031+0000 7f31bf7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f31c0068aa0 0x7f31c00ad470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35638/0 (socket says 172.21.15.89:35638) 2023-10-10T20:34:41.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.031+0000 7f31bf7fe700 1 -- 172.21.15.89:0/2176073014 learned_addr learned my addr 172.21.15.89:0/2176073014 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:41.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.031+0000 7f728a59c700 1 --2- 172.21.15.89:0/1257067829 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7274067120 0x7f72740695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.031+0000 7f48f01d6700 1 Processor -- start 2023-10-10T20:34:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.031+0000 7f48f01d6700 1 -- start start 2023-10-10T20:34:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.030+0000 7fa3deb7a700 1 Processor -- start 2023-10-10T20:34:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.030+0000 7fa3deb7a700 1 -- start start 2023-10-10T20:34:41.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7fe72f2e0700 1 Processor -- start 2023-10-10T20:34:41.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.031+0000 7f31bdffb700 1 -- 172.21.15.89:0/2176073014 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4220395282 0 0) 0x7f31c00acfe0 con 0x7f31c0068aa0 2023-10-10T20:34:41.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.031+0000 7f31bdffb700 1 -- 172.21.15.89:0/2176073014 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31a8003570 con 0x7f31c0068aa0 2023-10-10T20:34:41.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.031+0000 7f31bdffb700 1 -- 172.21.15.89:0/2176073014 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70595088 0 0) 0x7f31c00ac3d0 con 0x7f31c01575f0 2023-10-10T20:34:41.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.031+0000 7f31bdffb700 1 -- 172.21.15.89:0/2176073014 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31c00acfe0 con 0x7f31c01575f0 2023-10-10T20:34:41.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.031+0000 7f31bdffb700 1 -- 172.21.15.89:0/2176073014 <== mon.1 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2728091854 0 0) 0x7f31c00acfe0 con 0x7f31c01575f0 2023-10-10T20:34:41.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.031+0000 7f31bdffb700 1 -- 172.21.15.89:0/2176073014 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31c01579c0 msgr2=0x7f31c00abdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-10T20:34:41.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.031+0000 7f31bdffb700 1 --2- 172.21.15.89:0/2176073014 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31c01579c0 0x7f31c00abdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.031+0000 7f31bdffb700 1 -- 172.21.15.89:0/2176073014 >> v1:172.21.15.89:6789/0 conn(0x7f31c0068aa0 legacy=0x7f31c00ad470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f31bdffb700 1 -- 172.21.15.89:0/2176073014 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c00ac990 con 0x7f31c01575f0 2023-10-10T20:34:41.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f31bdffb700 1 -- 172.21.15.89:0/2176073014 <== mon.1 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f31b4001470 con 0x7f31c01575f0 2023-10-10T20:34:41.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f31bdffb700 1 -- 172.21.15.89:0/2176073014 >> v1:172.21.15.111:6789/0 conn(0x7f31c01575f0 legacy=0x7f31c00cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f31bdffb700 1 --2- 172.21.15.89:0/2176073014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a8003c20 0x7f31a8003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f31bdffb700 1 --2- 172.21.15.89:0/2176073014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31a8004810 0x7f31a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f31bdffb700 1 --2- 172.21.15.89:0/2176073014 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31a80052e0 0x7f31a800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f31bdffb700 1 -- 172.21.15.89:0/2176073014 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b80015b0 con 0x7f31a8004810 2023-10-10T20:34:41.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f31bdffb700 1 -- 172.21.15.89:0/2176073014 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c00ac6b0 con 0x7f31a8003c20 2023-10-10T20:34:41.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f31bdffb700 1 -- 172.21.15.89:0/2176073014 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a800c210 con 0x7f31a80052e0 2023-10-10T20:34:41.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f31beffd700 1 --2- 172.21.15.89:0/2176073014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a8003c20 0x7f31a8003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f31bf7fe700 1 --2- 172.21.15.89:0/2176073014 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31a80052e0 0x7f31a800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.031+0000 7f728a59c700 1 --2- 172.21.15.89:0/1257067829 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7274067120 0x7f72740695d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f72800093f0 tx=0x7f7280017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.031+0000 7f7289d9b700 1 --2- 172.21.15.89:0/1257067829 >> [v2:172.21.15.89:6800/90896,v1:172.21.15.89:6801/90896] conn(0x7f727406f330 0x7f7274071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f7289d9b700 1 --2- 172.21.15.89:0/1257067829 >> [v2:172.21.15.89:6800/90896,v1:172.21.15.89:6801/90896] conn(0x7f727406f330 0x7f7274071740 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.031+0000 7fa3deb7a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa3d80b0480 0x7fa3d80b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.031+0000 7fa3deb7a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d80aeec0 con 0x7fa3d80ae520 2023-10-10T20:34:41.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.031+0000 7fa3deb7a700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d81567e0 con 0x7fa3d806b980 2023-10-10T20:34:41.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.031+0000 7fa3deb7a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d80ae980 con 0x7fa3d80b0480 2023-10-10T20:34:41.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7fa3dc916700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa3d806b980 0x7fa3d80afd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:49244/0 (socket says 172.21.15.89:49244) 2023-10-10T20:34:41.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7fa3dc916700 1 -- 172.21.15.89:0/204854847 learned_addr learned my addr 172.21.15.89:0/204854847 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:41.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7fa3d7fff700 1 --2- 172.21.15.89:0/204854847 >> v2:172.21.15.111:3300/0 conn(0x7fa3d80b0480 0x7fa3d80b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f48f01d6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e8158440 0x7f48e8068aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f48f01d6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e8069450 con 0x7f48e80b21b0 2023-10-10T20:34:41.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f48f01d6700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e815f260 con 0x7f48e8068fe0 2023-10-10T20:34:41.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f48f01d6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e80aef20 con 0x7f48e8158440 2023-10-10T20:34:41.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f31beffd700 1 -- 172.21.15.89:0/2176073014 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31a80052e0 msgr2=0x7f31a800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:41.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f31beffd700 1 --2- 172.21.15.89:0/2176073014 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31a80052e0 0x7f31a800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f31beffd700 1 -- 172.21.15.89:0/2176073014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31a8004810 msgr2=0x7f31a8004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-10T20:34:41.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f31beffd700 1 --2- 172.21.15.89:0/2176073014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31a8004810 0x7f31a8004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f31beffd700 1 -- 172.21.15.89:0/2176073014 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c00acfe0 con 0x7f31a8003c20 2023-10-10T20:34:41.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f48ee773700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f48e80b21b0 0x7f48e80ab8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35666/0 (socket says 172.21.15.89:35666) 2023-10-10T20:34:41.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7f48ee773700 1 -- 172.21.15.89:0/3495035674 learned_addr learned my addr 172.21.15.89:0/3495035674 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:41.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.032+0000 7fe72f2e0700 1 -- start start 2023-10-10T20:34:41.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.033+0000 7f31beffd700 1 --2- 172.21.15.89:0/2176073014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a8003c20 0x7f31a8003ff0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f31b400c0e0 tx=0x7f31b4002db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57765e4ada7902fe server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.033+0000 7f48edf72700 1 --2- 172.21.15.89:0/3495035674 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e8158440 0x7f48e8068aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.033+0000 7fa3d77fe700 1 -- 172.21.15.89:0/204854847 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3238555280 0 0) 0x7fa3d80aeec0 con 0x7fa3d80ae520 2023-10-10T20:34:41.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.033+0000 7fa3d77fe700 1 -- 172.21.15.89:0/204854847 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3c8004680 con 0x7fa3d80ae520 2023-10-10T20:34:41.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.033+0000 7f48ecf70700 1 -- 172.21.15.89:0/3495035674 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 735030274 0 0) 0x7f48e8069450 con 0x7f48e80b21b0 2023-10-10T20:34:41.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.033+0000 7f48ecf70700 1 -- 172.21.15.89:0/3495035674 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48d4003570 con 0x7f48e80b21b0 2023-10-10T20:34:41.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.033+0000 7f727a7fc700 1 -- 172.21.15.89:0/1257067829 <== osd.2 v2:172.21.15.89:6800/90896 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f7274071df0 con 0x7f727406f330 2023-10-10T20:34:41.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.033+0000 7f48ecf70700 1 -- 172.21.15.89:0/3495035674 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1620138993 0 0) 0x7f48e815f260 con 0x7f48e8068fe0 2023-10-10T20:34:41.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.033+0000 7f48ecf70700 1 -- 172.21.15.89:0/3495035674 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48e8069450 con 0x7f48e8068fe0 2023-10-10T20:34:41.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.033+0000 7fe72f2e0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe7280afc40 0x7fe7280b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.033+0000 7fe72f2e0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7280b2140 0x7fe72815d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7fe72f2e0700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7280b0750 con 0x7fe7280b0010 2023-10-10T20:34:41.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7fe72f2e0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7280aca70 con 0x7fe7280afc40 2023-10-10T20:34:41.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7fe72f2e0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7280acd50 con 0x7fe7280b2140 2023-10-10T20:34:41.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.033+0000 7f31bdffb700 1 -- 172.21.15.89:0/2176073014 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b400ca10 con 0x7f31a8003c20 2023-10-10T20:34:41.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.033+0000 7f31bdffb700 1 -- 172.21.15.89:0/2176073014 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b400cb90 con 0x7f31a8003c20 2023-10-10T20:34:41.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.033+0000 7f31bdffb700 1 -- 172.21.15.89:0/2176073014 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b4012420 con 0x7f31a8003c20 2023-10-10T20:34:41.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7fa3d7fff700 1 -- 172.21.15.89:0/204854847 >> v1:172.21.15.111:6789/0 conn(0x7fa3d806b980 legacy=0x7fa3d80afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7fa3d7fff700 1 -- 172.21.15.89:0/204854847 >> v1:172.21.15.89:6789/0 conn(0x7fa3d80ae520 legacy=0x7fa3d8156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7fa3d7fff700 1 -- 172.21.15.89:0/204854847 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d80cc2a0 con 0x7fa3d80b0480 2023-10-10T20:34:41.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.033+0000 7f48edf72700 1 -- 172.21.15.89:0/3495035674 >> v1:172.21.15.111:6789/0 conn(0x7f48e8068fe0 legacy=0x7f48e80b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.033+0000 7f48edf72700 1 -- 172.21.15.89:0/3495035674 >> v1:172.21.15.89:6789/0 conn(0x7f48e80b21b0 legacy=0x7f48e80ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.033+0000 7f48edf72700 1 -- 172.21.15.89:0/3495035674 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e815e380 con 0x7f48e8158440 2023-10-10T20:34:41.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7fe72d07c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe7280afc40 0x7fe7280b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7fe72d07c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe7280afc40 0x7fe7280b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53278/0 (socket says 172.21.15.89:53278) 2023-10-10T20:34:41.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7fe72d07c700 1 -- 172.21.15.89:0/1100341390 learned_addr learned my addr 172.21.15.89:0/1100341390 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7fe72c87b700 1 --2- 172.21.15.89:0/1100341390 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7280b2140 0x7fe72815d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7fe72d07c700 1 -- 172.21.15.89:0/1100341390 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7280b2140 msgr2=0x7fe72815d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:41.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7fe72d07c700 1 --2- 172.21.15.89:0/1100341390 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7280b2140 0x7fe72815d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7fe72d07c700 1 -- 172.21.15.89:0/1100341390 >> v1:172.21.15.111:6789/0 conn(0x7fe7280b0010 legacy=0x7fe72815dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7fe72d07c700 1 -- 172.21.15.89:0/1100341390 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7280ad030 con 0x7fe7280afc40 2023-10-10T20:34:41.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7f31c571c700 1 -- 172.21.15.89:0/2176073014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a8003c20 msgr2=0x7f31a8003ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7f31c571c700 1 --2- 172.21.15.89:0/2176073014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a8003c20 0x7f31a8003ff0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f31b400c0e0 tx=0x7f31b4002db0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7f31bf7fe700 1 -- 172.21.15.89:0/2176073014 reap_dead start 2023-10-10T20:34:41.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7f31c571c700 1 -- 172.21.15.89:0/2176073014 shutdown_connections 2023-10-10T20:34:41.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7f31c571c700 1 -- 172.21.15.89:0/2176073014 >> 172.21.15.89:0/2176073014 conn(0x7f31c00cd000 msgr2=0x7f31c0068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7fa3d7fff700 1 --2- 172.21.15.89:0/204854847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3d80b0480 0x7fa3d80b1f40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0098e0 tx=0x7fa3cc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79701153f344c317 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7fa3d77fe700 1 -- 172.21.15.89:0/204854847 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00b640 con 0x7fa3d80b0480 2023-10-10T20:34:41.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7fa3d77fe700 1 -- 172.21.15.89:0/204854847 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc00b7a0 con 0x7fa3d80b0480 2023-10-10T20:34:41.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7fa3d77fe700 1 -- 172.21.15.89:0/204854847 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00bb20 con 0x7fa3d80b0480 2023-10-10T20:34:41.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7fe72d07c700 1 --2- 172.21.15.89:0/1100341390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7280afc40 0x7fe7280b1c00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe7180099b0 tx=0x7fe718009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e37a32534ae72a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.034+0000 7f31c571c700 1 -- 172.21.15.89:0/2176073014 shutdown_connections 2023-10-10T20:34:41.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.035+0000 7fc2ac667700 1 Processor -- start 2023-10-10T20:34:41.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.035+0000 7fc2ac667700 1 -- start start 2023-10-10T20:34:41.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.035+0000 7fa3deb7a700 1 -- 172.21.15.89:0/204854847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3d80b0480 msgr2=0x7fa3d80b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.035+0000 7fa3deb7a700 1 --2- 172.21.15.89:0/204854847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3d80b0480 0x7fa3d80b1f40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0098e0 tx=0x7fa3cc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.035+0000 7f48edf72700 1 --2- 172.21.15.89:0/3495035674 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e8158440 0x7f48e8068aa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009910 tx=0x7f48e40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb66e112f0ed299f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.036+0000 7fa3deb7a700 1 -- 172.21.15.89:0/204854847 shutdown_connections 2023-10-10T20:34:41.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.036+0000 7fa3deb7a700 1 --2- 172.21.15.89:0/204854847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3d80b0480 0x7fa3d80b1f40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.036+0000 7fa3deb7a700 1 -- 172.21.15.89:0/204854847 >> 172.21.15.89:0/204854847 conn(0x7fa3d806a320 msgr2=0x7fa3d80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.036+0000 7f0e01ea1700 1 Processor -- start 2023-10-10T20:34:41.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.036+0000 7f0e01ea1700 1 -- start start 2023-10-10T20:34:41.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.036+0000 7f48ecf70700 1 -- 172.21.15.89:0/3495035674 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4011630 con 0x7f48e8158440 2023-10-10T20:34:41.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.036+0000 7f48ecf70700 1 -- 172.21.15.89:0/3495035674 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e4011790 con 0x7f48e8158440 2023-10-10T20:34:41.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.036+0000 7f48ecf70700 1 -- 172.21.15.89:0/3495035674 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4011b50 con 0x7f48e8158440 2023-10-10T20:34:41.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.036+0000 7fe71ffff700 1 -- 172.21.15.89:0/1100341390 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718009ea0 con 0x7fe7280afc40 2023-10-10T20:34:41.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.036+0000 7fe71ffff700 1 -- 172.21.15.89:0/1100341390 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71800b860 con 0x7fe7280afc40 2023-10-10T20:34:41.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.036+0000 7fe71ffff700 1 -- 172.21.15.89:0/1100341390 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71800bc00 con 0x7fe7280afc40 2023-10-10T20:34:41.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.035+0000 7f31c571c700 1 -- 172.21.15.89:0/2176073014 wait complete. 2023-10-10T20:34:41.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.036+0000 7fe72f2e0700 1 -- 172.21.15.89:0/1100341390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7280afc40 msgr2=0x7fe7280b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.036+0000 7fe72f2e0700 1 --2- 172.21.15.89:0/1100341390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7280afc40 0x7fe7280b1c00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe7180099b0 tx=0x7fe718009250 comp rx=0 tx=0).stop 2023-10-10T20:34:41.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.036+0000 7fc2ac667700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc2a41575f0 0x7fc2a40abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.036+0000 7fc2ac667700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc2a4068aa0 0x7fc2a4068f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7fc2ac667700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a4069440 con 0x7fc2a41579c0 2023-10-10T20:34:41.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7fc2ac667700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a40acfe0 con 0x7fc2a4068aa0 2023-10-10T20:34:41.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7fc2ac667700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a40ac3d0 con 0x7fc2a41575f0 2023-10-10T20:34:41.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7f31c571c700 1 Processor -- start 2023-10-10T20:34:41.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7f31c571c700 1 -- start start 2023-10-10T20:34:41.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7f48f01d6700 1 -- 172.21.15.89:0/3495035674 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e8158440 msgr2=0x7f48e8068aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7f48f01d6700 1 --2- 172.21.15.89:0/3495035674 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e8158440 0x7f48e8068aa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009910 tx=0x7f48e40091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7fa3deb7a700 1 -- 172.21.15.89:0/204854847 shutdown_connections 2023-10-10T20:34:41.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7f0e01ea1700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0dfc0ccc40 0x7f0dfc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7f0e01ea1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dfc156150 0x7f0dfc0ac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7f0e01ea1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dfc0ad930 con 0x7f0dfc0cd010 2023-10-10T20:34:41.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7f0e01ea1700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dfc0acd20 con 0x7f0dfc0ccc40 2023-10-10T20:34:41.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7f0e01ea1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc0ad000 con 0x7f0dfc156150 2023-10-10T20:34:41.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7f48f01d6700 1 -- 172.21.15.89:0/3495035674 shutdown_connections 2023-10-10T20:34:41.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7f48f01d6700 1 --2- 172.21.15.89:0/3495035674 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e8158440 0x7f48e8068aa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7f48f01d6700 1 -- 172.21.15.89:0/3495035674 >> 172.21.15.89:0/3495035674 conn(0x7f48e80cd000 msgr2=0x7f48e8068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7fa3deb7a700 1 -- 172.21.15.89:0/204854847 wait complete. 2023-10-10T20:34:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7f0dfaffd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dfc156150 0x7f0dfc0ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7fc2aa403700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc2a41575f0 0x7fc2a40abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7fc2aa403700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc2a41575f0 0x7fc2a40abdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:42822/0 (socket says 172.21.15.89:42822) 2023-10-10T20:34:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7fc2aa403700 1 -- 172.21.15.89:0/2544190363 learned_addr learned my addr 172.21.15.89:0/2544190363 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7f0dfaffd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dfc156150 0x7f0dfc0ac230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:43828/0 (socket says 172.21.15.89:43828) 2023-10-10T20:34:41.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7f0dfaffd700 1 -- 172.21.15.89:0/3930767918 learned_addr learned my addr 172.21.15.89:0/3930767918 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7f0dfb7fe700 1 --2- 172.21.15.89:0/3930767918 >> v2:172.21.15.111:3300/0 conn(0x7f0dfc0ccc40 0x7f0dfc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7f48f01d6700 1 -- 172.21.15.89:0/3495035674 shutdown_connections 2023-10-10T20:34:41.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7f0dfaffd700 1 -- 172.21.15.89:0/3930767918 >> v2:172.21.15.111:3300/0 conn(0x7f0dfc0ccc40 msgr2=0x7f0dfc155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7f0dfaffd700 1 --2- 172.21.15.89:0/3930767918 >> v2:172.21.15.111:3300/0 conn(0x7f0dfc0ccc40 0x7f0dfc155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7f0dfaffd700 1 -- 172.21.15.89:0/3930767918 >> v1:172.21.15.89:6789/0 conn(0x7f0dfc0cd010 legacy=0x7f0dfc1568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7fc2aa403700 1 -- 172.21.15.89:0/2544190363 >> v2:172.21.15.89:3300/0 conn(0x7fc2a4068aa0 msgr2=0x7fc2a4068f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7fc2aa403700 1 --2- 172.21.15.89:0/2544190363 >> v2:172.21.15.89:3300/0 conn(0x7fc2a4068aa0 0x7fc2a4068f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7fc2aa403700 1 -- 172.21.15.89:0/2544190363 >> v1:172.21.15.111:6789/0 conn(0x7fc2a41579c0 legacy=0x7fc2a40cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7fc2aa403700 1 -- 172.21.15.89:0/2544190363 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a40ac6b0 con 0x7fc2a41575f0 2023-10-10T20:34:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7f0dfaffd700 1 -- 172.21.15.89:0/3930767918 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dfc0ad2e0 con 0x7f0dfc156150 2023-10-10T20:34:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7f31c571c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a8003c20 0x7f31c016dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7f31c571c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31c015a8f0 0x7f31c0146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7f31c571c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31c01471c0 0x7f31c01540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7f31c571c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c015a520 con 0x7f31c015a8f0 2023-10-10T20:34:41.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7f31c571c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c015ad60 con 0x7f31a8003c20 2023-10-10T20:34:41.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7f31c571c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c016e3e0 con 0x7f31c01471c0 2023-10-10T20:34:41.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7fc2aa403700 1 --2- 172.21.15.89:0/2544190363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2a41575f0 0x7fc2a40abdf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4063d40 tx=0x7fc29c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93a8eea3a0f508c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7f31bf7fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31c01471c0 0x7f31c01540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7f0dfaffd700 1 --2- 172.21.15.89:0/3930767918 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dfc156150 0x7f0dfc0ac230 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0df00099b0 tx=0x7f0df0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f212bb450f423b63 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7f31bf7fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31c01471c0 0x7f31c01540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:43834/0 (socket says 172.21.15.89:43834) 2023-10-10T20:34:41.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7f31bf7fe700 1 -- 172.21.15.89:0/4035871707 learned_addr learned my addr 172.21.15.89:0/4035871707 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7f0dfa7fc700 1 -- 172.21.15.89:0/3930767918 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df0009ea0 con 0x7f0dfc156150 2023-10-10T20:34:41.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7f0dfa7fc700 1 -- 172.21.15.89:0/3930767918 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df000b820 con 0x7f0dfc156150 2023-10-10T20:34:41.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7f0dfa7fc700 1 -- 172.21.15.89:0/3930767918 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df000bbc0 con 0x7f0dfc156150 2023-10-10T20:34:41.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7fe72f2e0700 1 -- 172.21.15.89:0/1100341390 shutdown_connections 2023-10-10T20:34:41.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7fe72f2e0700 1 --2- 172.21.15.89:0/1100341390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7280afc40 0x7fe7280b1c00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7fe72f2e0700 1 --2- 172.21.15.89:0/1100341390 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7280b2140 0x7fe72815d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7fe72f2e0700 1 -- 172.21.15.89:0/1100341390 >> 172.21.15.89:0/1100341390 conn(0x7fe72806a320 msgr2=0x7fe7280cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7fe72f2e0700 1 -- 172.21.15.89:0/1100341390 shutdown_connections 2023-10-10T20:34:41.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7f31bf7fe700 1 -- 172.21.15.89:0/4035871707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a8003c20 msgr2=0x7f31c016dd90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-10T20:34:41.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7f31bf7fe700 1 --2- 172.21.15.89:0/4035871707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a8003c20 0x7f31c016dd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7f31bf7fe700 1 -- 172.21.15.89:0/4035871707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31c015a8f0 msgr2=0x7f31c0146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7f31bf7fe700 1 --2- 172.21.15.89:0/4035871707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31c015a8f0 0x7f31c0146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7f31bf7fe700 1 -- 172.21.15.89:0/4035871707 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c016e6c0 con 0x7f31c01471c0 2023-10-10T20:34:41.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7f31bf7fe700 1 --2- 172.21.15.89:0/4035871707 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31c01471c0 0x7f31c01540a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f31c006d340 tx=0x7f31b80020a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7f31a6ffd700 1 -- 172.21.15.89:0/4035871707 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b800ac50 con 0x7f31c01471c0 2023-10-10T20:34:41.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7f31a6ffd700 1 -- 172.21.15.89:0/4035871707 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b800adb0 con 0x7f31c01471c0 2023-10-10T20:34:41.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7f31c571c700 1 -- 172.21.15.89:0/4035871707 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31c016e9a0 con 0x7f31c01471c0 2023-10-10T20:34:41.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7f31a6ffd700 1 -- 172.21.15.89:0/4035871707 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b8013640 con 0x7f31c01471c0 2023-10-10T20:34:41.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7f31c571c700 1 -- 172.21.15.89:0/4035871707 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31c0159060 con 0x7f31c01471c0 2023-10-10T20:34:41.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.038+0000 7fc2a9401700 1 -- 172.21.15.89:0/2544190363 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c00e070 con 0x7fc2a41575f0 2023-10-10T20:34:41.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7fc2a9401700 1 -- 172.21.15.89:0/2544190363 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29c00ba50 con 0x7fc2a41575f0 2023-10-10T20:34:41.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7fc2a9401700 1 -- 172.21.15.89:0/2544190363 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0092e0 con 0x7fc2a41575f0 2023-10-10T20:34:41.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7fc2ac667700 1 -- 172.21.15.89:0/2544190363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2a41575f0 msgr2=0x7fc2a40abdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.040+0000 7fc2ac667700 1 --2- 172.21.15.89:0/2544190363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2a41575f0 0x7fc2a40abdf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4063d40 tx=0x7fc29c009840 comp rx=0 tx=0).stop 2023-10-10T20:34:41.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.037+0000 7f48f01d6700 1 -- 172.21.15.89:0/3495035674 wait complete. 2023-10-10T20:34:41.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7f48f01d6700 1 Processor -- start 2023-10-10T20:34:41.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7fa3deb7a700 1 Processor -- start 2023-10-10T20:34:41.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7f0e01ea1700 1 -- 172.21.15.89:0/3930767918 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dfc156150 msgr2=0x7f0dfc0ac230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7f0e01ea1700 1 --2- 172.21.15.89:0/3930767918 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dfc156150 0x7f0dfc0ac230 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0df00099b0 tx=0x7f0df0009250 comp rx=0 tx=0).stop 2023-10-10T20:34:41.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.040+0000 7fc2ac667700 1 -- 172.21.15.89:0/2544190363 shutdown_connections 2023-10-10T20:34:41.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.040+0000 7fc2ac667700 1 --2- 172.21.15.89:0/2544190363 >> v2:172.21.15.89:3300/0 conn(0x7fc2a4068aa0 0x7fc2a4068f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.040+0000 7fc2ac667700 1 --2- 172.21.15.89:0/2544190363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2a41575f0 0x7fc2a40abdf0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.040+0000 7fc2ac667700 1 -- 172.21.15.89:0/2544190363 >> 172.21.15.89:0/2544190363 conn(0x7fc2a40cd000 msgr2=0x7fc2a4068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.040+0000 7f31c571c700 1 -- 172.21.15.89:0/4035871707 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f31c004c3c0 con 0x7f31c01471c0 2023-10-10T20:34:41.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.040+0000 7f0e01ea1700 1 -- 172.21.15.89:0/3930767918 shutdown_connections 2023-10-10T20:34:41.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.040+0000 7f0e01ea1700 1 --2- 172.21.15.89:0/3930767918 >> v2:172.21.15.111:3300/0 conn(0x7f0dfc0ccc40 0x7f0dfc155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7f0e01ea1700 1 --2- 172.21.15.89:0/3930767918 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dfc156150 0x7f0dfc0ac230 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7f0e01ea1700 1 -- 172.21.15.89:0/3930767918 >> 172.21.15.89:0/3930767918 conn(0x7f0dfc06a320 msgr2=0x7f0dfc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7f0e01ea1700 1 -- 172.21.15.89:0/3930767918 shutdown_connections 2023-10-10T20:34:41.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.040+0000 7fa3deb7a700 1 -- start start 2023-10-10T20:34:41.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7fa3deb7a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d806b980 0x7fa3d81590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7fa3deb7a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3d80ae520 0x7fa3d81501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7fa3deb7a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3d8150a90 0x7fa3d816c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7fa3deb7a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d81596d0 con 0x7fa3d806b980 2023-10-10T20:34:41.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7fa3deb7a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d8176d20 con 0x7fa3d80ae520 2023-10-10T20:34:41.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7fa3deb7a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d8176f30 con 0x7fa3d8150a90 2023-10-10T20:34:41.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7fa3dd117700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3d8150a90 0x7fa3d816c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7f48f01d6700 1 -- start start 2023-10-10T20:34:41.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7f48f01d6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e8068fe0 0x7f48e81554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7f48f01d6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e80b21b0 0x7f48e81568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7f48f01d6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48e8158440 0x7f48e8151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7f48f01d6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e815b6c0 con 0x7f48e80b21b0 2023-10-10T20:34:41.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7f48f01d6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e8155ad0 con 0x7f48e8158440 2023-10-10T20:34:41.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7f48f01d6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e80ac960 con 0x7f48e8068fe0 2023-10-10T20:34:41.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7f48ed771700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e80b21b0 0x7f48e81568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7f48ed771700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e80b21b0 0x7f48e81568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53310/0 (socket says 172.21.15.89:53310) 2023-10-10T20:34:41.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7f48ed771700 1 -- 172.21.15.89:0/2629049506 learned_addr learned my addr 172.21.15.89:0/2629049506 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.039+0000 7fe72f2e0700 1 -- 172.21.15.89:0/1100341390 wait complete. 2023-10-10T20:34:41.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7fe72f2e0700 1 Processor -- start 2023-10-10T20:34:41.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7fc2ac667700 1 -- 172.21.15.89:0/2544190363 shutdown_connections 2023-10-10T20:34:41.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7fa3dd117700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3d8150a90 0x7fa3d816c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:43864/0 (socket says 172.21.15.89:43864) 2023-10-10T20:34:41.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7fa3dd117700 1 -- 172.21.15.89:0/770926646 learned_addr learned my addr 172.21.15.89:0/770926646 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7fa3dc916700 1 --2- 172.21.15.89:0/770926646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d806b980 0x7fa3d81590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7f31a6ffd700 1 -- 172.21.15.89:0/4035871707 <== mon.2 v2:172.21.15.111:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f31b80139f0 con 0x7f31c01471c0 2023-10-10T20:34:41.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7fa3d7fff700 1 --2- 172.21.15.89:0/770926646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3d80ae520 0x7fa3d81501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7f48ed771700 1 -- 172.21.15.89:0/2629049506 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e8068fe0 msgr2=0x7f48e81554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7f48edf72700 1 --2- 172.21.15.89:0/2629049506 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e8068fe0 0x7f48e81554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7f48ee773700 1 --2- 172.21.15.89:0/2629049506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48e8158440 0x7f48e8151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7f48ed771700 1 --2- 172.21.15.89:0/2629049506 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e8068fe0 0x7f48e81554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7f48ed771700 1 -- 172.21.15.89:0/2629049506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48e8158440 msgr2=0x7f48e8151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7f48ed771700 1 --2- 172.21.15.89:0/2629049506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48e8158440 0x7f48e8151550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7f48ed771700 1 -- 172.21.15.89:0/2629049506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e4009420 con 0x7f48e80b21b0 2023-10-10T20:34:41.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7fa3dc916700 1 -- 172.21.15.89:0/770926646 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3d8150a90 msgr2=0x7fa3d816c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7fa3dc916700 1 --2- 172.21.15.89:0/770926646 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3d8150a90 0x7fa3d816c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7fa3dc916700 1 -- 172.21.15.89:0/770926646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3d80ae520 msgr2=0x7fa3d81501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7fa3dc916700 1 --2- 172.21.15.89:0/770926646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3d80ae520 0x7fa3d81501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7fa3dc916700 1 -- 172.21.15.89:0/770926646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc009420 con 0x7fa3d806b980 2023-10-10T20:34:41.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7f48ed771700 1 --2- 172.21.15.89:0/2629049506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e80b21b0 0x7f48e81568a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f48d800c7c0 tx=0x7f48d8007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7f0e01ea1700 1 -- 172.21.15.89:0/3930767918 wait complete. 2023-10-10T20:34:41.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7f0e01ea1700 1 Processor -- start 2023-10-10T20:34:41.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.041+0000 7fe72f2e0700 1 -- start start 2023-10-10T20:34:41.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7fe72f2e0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7280afc40 0x7fe728153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7fe72f2e0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7280b0010 0x7fe728177cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7fe72f2e0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7280b2140 0x7fe728179660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7fe72f2e0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7281543c0 con 0x7fe7280b0010 2023-10-10T20:34:41.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7fe72f2e0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7281785c0 con 0x7fe7280b2140 2023-10-10T20:34:41.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7fe72f2e0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7281788a0 con 0x7fe7280afc40 2023-10-10T20:34:41.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7fe72c87b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7280b0010 0x7fe728177cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7fe72c87b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7280b0010 0x7fe728177cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53320/0 (socket says 172.21.15.89:53320) 2023-10-10T20:34:41.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7fe72c87b700 1 -- 172.21.15.89:0/4207838496 learned_addr learned my addr 172.21.15.89:0/4207838496 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7fe72d07c700 1 --2- 172.21.15.89:0/4207838496 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7280afc40 0x7fe728153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7fe72d87d700 1 --2- 172.21.15.89:0/4207838496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7280b2140 0x7fe728179660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7fc2ac667700 1 -- 172.21.15.89:0/2544190363 wait complete. 2023-10-10T20:34:41.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7f48ddffb700 1 -- 172.21.15.89:0/2629049506 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d8012630 con 0x7f48e80b21b0 2023-10-10T20:34:41.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7fa3dc916700 1 --2- 172.21.15.89:0/770926646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d806b980 0x7fa3d81590f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400c7c0 tx=0x7fa3c4007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7fc2ac667700 1 Processor -- start 2023-10-10T20:34:41.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7f31a6ffd700 1 --2- 172.21.15.89:0/4035871707 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31ac066fb0 0x7f31ac069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7f31a6ffd700 1 -- 172.21.15.89:0/4035871707 <== mon.2 v2:172.21.15.111:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f31b8084d20 con 0x7f31c01471c0 2023-10-10T20:34:41.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7f31beffd700 1 --2- 172.21.15.89:0/4035871707 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31ac066fb0 0x7f31ac069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7f31a6ffd700 1 --2- 172.21.15.89:0/4035871707 >> [v2:172.21.15.89:6816/90897,v1:172.21.15.89:6817/90897] conn(0x7f31ac06f1c0 0x7f31ac0715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.042+0000 7f31a6ffd700 1 -- 172.21.15.89:0/4035871707 --> [v2:172.21.15.89:6816/90897,v1:172.21.15.89:6817/90897] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31ac071c80 con 0x7f31ac06f1c0 2023-10-10T20:34:41.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7f31beffd700 1 --2- 172.21.15.89:0/4035871707 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31ac066fb0 0x7f31ac069460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f31c0155860 tx=0x7f31b4017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7f31a6ffd700 1 -- 172.21.15.89:0/4035871707 <== mon.2 v2:172.21.15.111:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31b80872c0 con 0x7f31c01471c0 2023-10-10T20:34:41.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7fe72d07c700 1 -- 172.21.15.89:0/4207838496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7280b2140 msgr2=0x7fe728179660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7fe72d07c700 1 --2- 172.21.15.89:0/4207838496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7280b2140 0x7fe728179660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7fe72d07c700 1 -- 172.21.15.89:0/4207838496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7280b0010 msgr2=0x7fe728177cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7fe72d07c700 1 --2- 172.21.15.89:0/4207838496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7280b0010 0x7fe728177cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7fe72d07c700 1 -- 172.21.15.89:0/4207838496 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7180094f0 con 0x7fe7280afc40 2023-10-10T20:34:41.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7fe72d07c700 1 --2- 172.21.15.89:0/4207838496 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7280afc40 0x7fe728153e80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe72806fca0 tx=0x7fe718017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7fe71d7fa700 1 -- 172.21.15.89:0/4207838496 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718000d10 con 0x7fe7280afc40 2023-10-10T20:34:41.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7fe71d7fa700 1 -- 172.21.15.89:0/4207838496 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71800b9d0 con 0x7fe7280afc40 2023-10-10T20:34:41.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7fe71d7fa700 1 -- 172.21.15.89:0/4207838496 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718026660 con 0x7fe7280afc40 2023-10-10T20:34:41.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f31be7fc700 1 --2- 172.21.15.89:0/4035871707 >> [v2:172.21.15.89:6816/90897,v1:172.21.15.89:6817/90897] conn(0x7f31ac06f1c0 0x7f31ac0715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7f0e01ea1700 1 -- start start 2023-10-10T20:34:41.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f0e01ea1700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0dfc0ccc40 0x7f0dfc161bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f0e01ea1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dfc0cd010 0x7f0dfc15f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f0e01ea1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dfc156150 0x7f0dfc147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f0e01ea1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc162840 con 0x7f0dfc0cd010 2023-10-10T20:34:41.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f0e01ea1700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc147770 con 0x7f0dfc0ccc40 2023-10-10T20:34:41.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f0e01ea1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc162280 con 0x7f0dfc156150 2023-10-10T20:34:41.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f0dfb7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0dfc0ccc40 0x7f0dfc161bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f0dfb7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0dfc0ccc40 0x7f0dfc161bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:42908/0 (socket says 172.21.15.89:42908) 2023-10-10T20:34:41.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f0dfb7fe700 1 -- 172.21.15.89:0/3908871464 learned_addr learned my addr 172.21.15.89:0/3908871464 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7fe72f2e0700 1 -- 172.21.15.89:0/4207838496 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe728178b80 con 0x7fe7280afc40 2023-10-10T20:34:41.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7fe72f2e0700 1 -- 172.21.15.89:0/4207838496 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe728179140 con 0x7fe7280afc40 2023-10-10T20:34:41.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f31be7fc700 1 --2- 172.21.15.89:0/4035871707 >> [v2:172.21.15.89:6816/90897,v1:172.21.15.89:6817/90897] conn(0x7f31ac06f1c0 0x7f31ac0715d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7f48ddffb700 1 -- 172.21.15.89:0/2629049506 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d80127d0 con 0x7f48e80b21b0 2023-10-10T20:34:41.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7f48ddffb700 1 -- 172.21.15.89:0/2629049506 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d8012b90 con 0x7f48e80b21b0 2023-10-10T20:34:41.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.043+0000 7f48f01d6700 1 -- 172.21.15.89:0/2629049506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e81592e0 con 0x7f48e80b21b0 2023-10-10T20:34:41.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f48f01d6700 1 -- 172.21.15.89:0/2629049506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e8158f70 con 0x7f48e80b21b0 2023-10-10T20:34:41.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f0dfaffd700 1 --2- 172.21.15.89:0/3908871464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dfc0cd010 0x7f0dfc15f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f0dfbfff700 1 --2- 172.21.15.89:0/3908871464 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dfc156150 0x7f0dfc147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f0dfb7fe700 1 -- 172.21.15.89:0/3908871464 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dfc156150 msgr2=0x7f0dfc147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f0dfb7fe700 1 --2- 172.21.15.89:0/3908871464 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dfc156150 0x7f0dfc147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f0dfb7fe700 1 -- 172.21.15.89:0/3908871464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dfc0cd010 msgr2=0x7f0dfc15f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f0dfb7fe700 1 --2- 172.21.15.89:0/3908871464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dfc0cd010 0x7f0dfc15f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f0dfb7fe700 1 -- 172.21.15.89:0/3908871464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df00094f0 con 0x7f0dfc0ccc40 2023-10-10T20:34:41.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f48f01d6700 1 -- 172.21.15.89:0/2629049506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f48e8158ad0 con 0x7f48e80b21b0 2023-10-10T20:34:41.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7f0dfb7fe700 1 --2- 172.21.15.89:0/3908871464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0dfc0ccc40 0x7f0dfc161bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00ec80 tx=0x7f0dec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.045+0000 7f0de37fe700 1 -- 172.21.15.89:0/3908871464 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec00c9d0 con 0x7f0dfc0ccc40 2023-10-10T20:34:41.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.045+0000 7f0de37fe700 1 -- 172.21.15.89:0/3908871464 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dec00cb30 con 0x7f0dfc0ccc40 2023-10-10T20:34:41.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.045+0000 7f0de37fe700 1 -- 172.21.15.89:0/3908871464 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec013470 con 0x7f0dfc0ccc40 2023-10-10T20:34:41.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.045+0000 7f0e01ea1700 1 -- 172.21.15.89:0/3908871464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dfc162560 con 0x7f0dfc0ccc40 2023-10-10T20:34:41.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.045+0000 7f0e01ea1700 1 -- 172.21.15.89:0/3908871464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dfc0ca870 con 0x7f0dfc0ccc40 2023-10-10T20:34:41.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.045+0000 7f0e01ea1700 1 -- 172.21.15.89:0/3908871464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0dfc04c3c0 con 0x7f0dfc0ccc40 2023-10-10T20:34:41.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.044+0000 7fc2ac667700 1 -- start start 2023-10-10T20:34:41.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.045+0000 7fc2ac667700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2a4068aa0 0x7fc2a40b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.045+0000 7fc2ac667700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc2a41575f0 0x7fc2a414b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.045+0000 7fc2ac667700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2a41579c0 0x7fc2a414b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.045+0000 7fc2ac667700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a414a080 con 0x7fc2a4068aa0 2023-10-10T20:34:41.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.045+0000 7fc2ac667700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a40b11d0 con 0x7fc2a41579c0 2023-10-10T20:34:41.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.045+0000 7fc2ac667700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a414e970 con 0x7fc2a41575f0 2023-10-10T20:34:41.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.046+0000 7fc2aa403700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2a4068aa0 0x7fc2a40b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.046+0000 7fc2aa403700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2a4068aa0 0x7fc2a40b1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53336/0 (socket says 172.21.15.89:53336) 2023-10-10T20:34:41.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.046+0000 7fc2aa403700 1 -- 172.21.15.89:0/1440695866 learned_addr learned my addr 172.21.15.89:0/1440695866 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.046+0000 7f48edf72700 1 --2- 172.21.15.89:0/2629049506 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e8068fe0 0x7f48e81554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:41.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.046+0000 7fa3d7fff700 1 --2- 172.21.15.89:0/770926646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3d80ae520 0x7fa3d81501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:41.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.046+0000 7fc2aa403700 1 -- 172.21.15.89:0/1440695866 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc2a41575f0 msgr2=0x7fc2a414b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.046+0000 7fc2aa403700 1 --2- 172.21.15.89:0/1440695866 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc2a41575f0 0x7fc2a414b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.046+0000 7fc2aa403700 1 -- 172.21.15.89:0/1440695866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2a41579c0 msgr2=0x7fc2a414b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.046+0000 7fc2aa403700 1 --2- 172.21.15.89:0/1440695866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2a41579c0 0x7fc2a414b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.046+0000 7fc2aa403700 1 -- 172.21.15.89:0/1440695866 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29c009ae0 con 0x7fc2a4068aa0 2023-10-10T20:34:41.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.046+0000 7f48ddffb700 1 -- 172.21.15.89:0/2629049506 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f48d801b400 con 0x7f48e80b21b0 2023-10-10T20:34:41.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.047+0000 7f48ddffb700 1 --2- 172.21.15.89:0/2629049506 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f48cc067080 0x7f48cc069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.047+0000 7f48ddffb700 1 -- 172.21.15.89:0/2629049506 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f48d80848e0 con 0x7f48e80b21b0 2023-10-10T20:34:41.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.047+0000 7f48ddffb700 1 --2- 172.21.15.89:0/2629049506 >> [v2:172.21.15.111:6800/91095,v1:172.21.15.111:6801/91095] conn(0x7f48cc06f210 0x7f48cc071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.047+0000 7f48ddffb700 1 -- 172.21.15.89:0/2629049506 --> [v2:172.21.15.111:6800/91095,v1:172.21.15.111:6801/91095] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f48cc071cd0 con 0x7f48cc06f210 2023-10-10T20:34:41.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.046+0000 7fe71d7fa700 1 -- 172.21.15.89:0/4207838496 <== mon.2 v2:172.21.15.111:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe718023030 con 0x7fe7280afc40 2023-10-10T20:34:41.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.047+0000 7fe72f2e0700 1 -- 172.21.15.89:0/4207838496 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe70c001000 con 0x7fe7280afc40 2023-10-10T20:34:41.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.047+0000 7fe71d7fa700 1 --2- 172.21.15.89:0/4207838496 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe714067160 0x7fe714069610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.047+0000 7fe71d7fa700 1 -- 172.21.15.89:0/4207838496 <== mon.2 v2:172.21.15.111:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe718085c70 con 0x7fe7280afc40 2023-10-10T20:34:41.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.047+0000 7fe72c87b700 1 --2- 172.21.15.89:0/4207838496 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe714067160 0x7fe714069610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.047+0000 7fe71d7fa700 1 --2- 172.21.15.89:0/4207838496 >> [v2:172.21.15.89:6808/90898,v1:172.21.15.89:6809/90898] conn(0x7fe71406f180 0x7fe714071590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.047+0000 7fe71d7fa700 1 -- 172.21.15.89:0/4207838496 --> [v2:172.21.15.89:6808/90898,v1:172.21.15.89:6809/90898] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe714071c40 con 0x7fe71406f180 2023-10-10T20:34:41.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.047+0000 7fe71d7fa700 1 -- 172.21.15.89:0/4207838496 <== mon.2 v2:172.21.15.111:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe718006cf0 con 0x7fe7280afc40 2023-10-10T20:34:41.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.047+0000 7fc2aa403700 1 --2- 172.21.15.89:0/1440695866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2a4068aa0 0x7fc2a40b1660 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009fd0 tx=0x7fc29c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.047+0000 7f0de37fe700 1 -- 172.21.15.89:0/3908871464 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0dec005a00 con 0x7f0dfc0ccc40 2023-10-10T20:34:41.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.047+0000 7fe72c87b700 1 --2- 172.21.15.89:0/4207838496 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe714067160 0x7fe714069610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe728163730 tx=0x7fe72000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.048+0000 7f48edf72700 1 --2- 172.21.15.89:0/2629049506 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f48cc067080 0x7f48cc069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.048+0000 7f0de37fe700 1 --2- 172.21.15.89:0/3908871464 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0de8066f30 0x7f0de80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.048+0000 7f0de37fe700 1 -- 172.21.15.89:0/3908871464 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0dec084b40 con 0x7f0dfc0ccc40 2023-10-10T20:34:41.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.048+0000 7f0dfaffd700 1 --2- 172.21.15.89:0/3908871464 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0de8066f30 0x7f0de80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.048+0000 7f0de37fe700 1 --2- 172.21.15.89:0/3908871464 >> [v2:172.21.15.111:6808/91096,v1:172.21.15.111:6809/91096] conn(0x7f0de806f140 0x7f0de8071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.048+0000 7f0de37fe700 1 -- 172.21.15.89:0/3908871464 --> [v2:172.21.15.111:6808/91096,v1:172.21.15.111:6809/91096] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0de8071c00 con 0x7f0de806f140 2023-10-10T20:34:41.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.048+0000 7f0de37fe700 1 -- 172.21.15.89:0/3908871464 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0dec038ea0 con 0x7f0dfc0ccc40 2023-10-10T20:34:41.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.048+0000 7f48edf72700 1 --2- 172.21.15.89:0/2629049506 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f48cc067080 0x7f48cc069530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f48e4003080 tx=0x7f48e4017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.048+0000 7fc29a7fc700 1 -- 172.21.15.89:0/1440695866 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c008b70 con 0x7fc2a4068aa0 2023-10-10T20:34:41.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.048+0000 7fc29a7fc700 1 -- 172.21.15.89:0/1440695866 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29c008cd0 con 0x7fc2a4068aa0 2023-10-10T20:34:41.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.048+0000 7fc2ac667700 1 -- 172.21.15.89:0/1440695866 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a414ec30 con 0x7fc2a4068aa0 2023-10-10T20:34:41.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.048+0000 7fc29a7fc700 1 -- 172.21.15.89:0/1440695866 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0266b0 con 0x7fc2a4068aa0 2023-10-10T20:34:41.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.048+0000 7fc2ac667700 1 -- 172.21.15.89:0/1440695866 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a40b0e60 con 0x7fc2a4068aa0 2023-10-10T20:34:41.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.048+0000 7f0dfaffd700 1 --2- 172.21.15.89:0/3908871464 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0de8066f30 0x7f0de80693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0df00099b0 tx=0x7f0df0016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.048+0000 7f0dfbfff700 1 --2- 172.21.15.89:0/3908871464 >> [v2:172.21.15.111:6808/91096,v1:172.21.15.111:6809/91096] conn(0x7f0de806f140 0x7f0de8071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.049+0000 7f0dfbfff700 1 --2- 172.21.15.89:0/3908871464 >> [v2:172.21.15.111:6808/91096,v1:172.21.15.111:6809/91096] conn(0x7f0de806f140 0x7f0de8071550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.049+0000 7fc2ac667700 1 -- 172.21.15.89:0/1440695866 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc2a404c3c0 con 0x7fc2a4068aa0 2023-10-10T20:34:41.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.050+0000 7f0de37fe700 1 -- 172.21.15.89:0/3908871464 <== osd.6 v2:172.21.15.111:6808/91096 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f0de8071c00 con 0x7f0de806f140 2023-10-10T20:34:41.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.050+0000 7fe72d87d700 1 --2- 172.21.15.89:0/4207838496 >> [v2:172.21.15.89:6808/90898,v1:172.21.15.89:6809/90898] conn(0x7fe71406f180 0x7fe714071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.050+0000 7fe72d87d700 1 --2- 172.21.15.89:0/4207838496 >> [v2:172.21.15.89:6808/90898,v1:172.21.15.89:6809/90898] conn(0x7fe71406f180 0x7fe714071590 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.050+0000 7f48ee773700 1 --2- 172.21.15.89:0/2629049506 >> [v2:172.21.15.111:6800/91095,v1:172.21.15.111:6801/91095] conn(0x7f48cc06f210 0x7f48cc071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.050+0000 7f48ee773700 1 --2- 172.21.15.89:0/2629049506 >> [v2:172.21.15.111:6800/91095,v1:172.21.15.111:6801/91095] conn(0x7f48cc06f210 0x7f48cc071620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.051+0000 7fe71d7fa700 1 -- 172.21.15.89:0/4207838496 <== osd.0 v2:172.21.15.89:6808/90898 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fe714071c40 con 0x7fe71406f180 2023-10-10T20:34:41.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.051+0000 7f48ddffb700 1 -- 172.21.15.89:0/2629049506 <== osd.4 v2:172.21.15.111:6800/91095 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f48cc071cd0 con 0x7f48cc06f210 2023-10-10T20:34:41.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.045+0000 7f31a6ffd700 1 -- 172.21.15.89:0/4035871707 <== osd.1 v2:172.21.15.89:6816/90897 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f31ac071c80 con 0x7f31ac06f1c0 2023-10-10T20:34:41.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.050+0000 7fc29a7fc700 1 -- 172.21.15.89:0/1440695866 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc29c003040 con 0x7fc2a4068aa0 2023-10-10T20:34:41.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.051+0000 7fc29a7fc700 1 --2- 172.21.15.89:0/1440695866 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc288066fb0 0x7fc288069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.051+0000 7fc29a7fc700 1 -- 172.21.15.89:0/1440695866 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc29c086520 con 0x7fc2a4068aa0 2023-10-10T20:34:41.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.051+0000 7fc29a7fc700 1 --2- 172.21.15.89:0/1440695866 >> [v2:172.21.15.89:6820/90907,v1:172.21.15.89:6822/90907] conn(0x7fc28806f140 0x7fc288071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.051+0000 7fc29a7fc700 1 -- 172.21.15.89:0/1440695866 --> [v2:172.21.15.89:6820/90907,v1:172.21.15.89:6822/90907] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc288071c00 con 0x7fc28806f140 2023-10-10T20:34:41.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.051+0000 7fc2a9c02700 1 --2- 172.21.15.89:0/1440695866 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc288066fb0 0x7fc288069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.051+0000 7fc29a7fc700 1 -- 172.21.15.89:0/1440695866 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc29c0536f0 con 0x7fc2a4068aa0 2023-10-10T20:34:41.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.051+0000 7fc2aac04700 1 --2- 172.21.15.89:0/1440695866 >> [v2:172.21.15.89:6820/90907,v1:172.21.15.89:6822/90907] conn(0x7fc28806f140 0x7fc288071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.052+0000 7fc2a9c02700 1 --2- 172.21.15.89:0/1440695866 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc288066fb0 0x7fc288069460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4068630 tx=0x7fc2a0007ab0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.052+0000 7fc2aac04700 1 --2- 172.21.15.89:0/1440695866 >> [v2:172.21.15.89:6820/90907,v1:172.21.15.89:6822/90907] conn(0x7fc28806f140 0x7fc288071550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.053+0000 7fc29a7fc700 1 -- 172.21.15.89:0/1440695866 <== osd.3 v2:172.21.15.89:6820/90907 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fc288071c00 con 0x7fc28806f140 2023-10-10T20:34:41.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.054+0000 7f48ddffb700 1 -- 172.21.15.89:0/2629049506 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48d8051b30 con 0x7f48e80b21b0 2023-10-10T20:34:41.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.055+0000 7fa3d4ff9700 1 -- 172.21.15.89:0/770926646 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4005b30 con 0x7fa3d806b980 2023-10-10T20:34:41.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.055+0000 7fa3d4ff9700 1 -- 172.21.15.89:0/770926646 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c4005c90 con 0x7fa3d806b980 2023-10-10T20:34:41.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.055+0000 7fa3d4ff9700 1 -- 172.21.15.89:0/770926646 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4004a60 con 0x7fa3d806b980 2023-10-10T20:34:41.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.056+0000 7fa3deb7a700 1 -- 172.21.15.89:0/770926646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d8177210 con 0x7fa3d806b980 2023-10-10T20:34:41.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.056+0000 7fa3deb7a700 1 -- 172.21.15.89:0/770926646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d81777d0 con 0x7fa3d806b980 2023-10-10T20:34:41.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.058+0000 7fa3d4ff9700 1 -- 172.21.15.89:0/770926646 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa3c40070d0 con 0x7fa3d806b980 2023-10-10T20:34:41.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.059+0000 7fa3d4ff9700 1 --2- 172.21.15.89:0/770926646 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa3bc067080 0x7fa3bc069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.059+0000 7fa3d4ff9700 1 -- 172.21.15.89:0/770926646 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa3c4083930 con 0x7fa3d806b980 2023-10-10T20:34:41.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.059+0000 7fa3d7fff700 1 --2- 172.21.15.89:0/770926646 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa3bc067080 0x7fa3bc069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.060+0000 7fa3d7fff700 1 --2- 172.21.15.89:0/770926646 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa3bc067080 0x7fa3bc069530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80cc020 tx=0x7fa3cc017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.063+0000 7fbb99811700 1 Processor -- start 2023-10-10T20:34:41.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.063+0000 7fa3deb7a700 1 --2- 172.21.15.89:0/770926646 >> [v2:172.21.15.111:6816/91121,v1:172.21.15.111:6817/91121] conn(0x7fa3d8154380 0x7fa3d815e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.063+0000 7fa3deb7a700 1 -- 172.21.15.89:0/770926646 --> [v2:172.21.15.111:6816/91121,v1:172.21.15.111:6817/91121] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3d8177070 con 0x7fa3d8154380 2023-10-10T20:34:41.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.064+0000 7fa3dd117700 1 --2- 172.21.15.89:0/770926646 >> [v2:172.21.15.111:6816/91121,v1:172.21.15.111:6817/91121] conn(0x7fa3d8154380 0x7fa3d815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.063+0000 7fbb99811700 1 -- start start 2023-10-10T20:34:41.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.064+0000 7fbb99811700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb940cba20 0x7fbb940ac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.064+0000 7fbb99811700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb94156760 0x7fbb94068ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.064+0000 7fbb99811700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb940b2490 con 0x7fbb94068aa0 2023-10-10T20:34:41.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.064+0000 7fbb99811700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb940aceb0 con 0x7fbb940cba20 2023-10-10T20:34:41.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.064+0000 7fbb99811700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940ac4e0 con 0x7fbb94156760 2023-10-10T20:34:41.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.064+0000 7fbb927fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbb94068aa0 0x7fbb94156050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35674/0 (socket says 172.21.15.89:35674) 2023-10-10T20:34:41.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.064+0000 7fbb92ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb940cba20 0x7fbb940ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.064+0000 7fbb927fc700 1 -- 172.21.15.89:0/3149750371 learned_addr learned my addr 172.21.15.89:0/3149750371 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:41.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.064+0000 7fbb937fe700 1 --2- 172.21.15.89:0/3149750371 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb94156760 0x7fbb94068ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.065+0000 7fbb91ffb700 1 -- 172.21.15.89:0/3149750371 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793047562 0 0) 0x7fbb88001110 con 0x7fbb94068aa0 2023-10-10T20:34:41.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.065+0000 7fbb91ffb700 1 -- 172.21.15.89:0/3149750371 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb7c007980 con 0x7fbb94068aa0 2023-10-10T20:34:41.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.065+0000 7fbb92ffd700 1 -- 172.21.15.89:0/3149750371 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb94156760 msgr2=0x7fbb94068ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:41.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.065+0000 7fbb92ffd700 1 --2- 172.21.15.89:0/3149750371 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb94156760 0x7fbb94068ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.065+0000 7fbb92ffd700 1 -- 172.21.15.89:0/3149750371 >> v1:172.21.15.89:6789/0 conn(0x7fbb94068aa0 legacy=0x7fbb94156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.065+0000 7fbb92ffd700 1 -- 172.21.15.89:0/3149750371 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb94069430 con 0x7fbb940cba20 2023-10-10T20:34:41.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.065+0000 7fbb92ffd700 1 --2- 172.21.15.89:0/3149750371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb940cba20 0x7fbb940ac970 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009ec0 tx=0x7fbb84009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8611da1fe3f4125 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.065+0000 7fbb91ffb700 1 -- 172.21.15.89:0/3149750371 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84014bb0 con 0x7fbb940cba20 2023-10-10T20:34:41.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.065+0000 7fbb91ffb700 1 -- 172.21.15.89:0/3149750371 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb84014d10 con 0x7fbb940cba20 2023-10-10T20:34:41.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.065+0000 7fbb91ffb700 1 -- 172.21.15.89:0/3149750371 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8400ba70 con 0x7fbb940cba20 2023-10-10T20:34:41.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.066+0000 7f7290c29700 1 -- 172.21.15.89:0/1257067829 --> [v2:172.21.15.89:6800/90896,v1:172.21.15.89:6801/90896] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7270002d30 con 0x7f727406f330 2023-10-10T20:34:41.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.066+0000 7fbb99811700 1 -- 172.21.15.89:0/3149750371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb940cba20 msgr2=0x7fbb940ac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.066+0000 7fbb99811700 1 --2- 172.21.15.89:0/3149750371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb940cba20 0x7fbb940ac970 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009ec0 tx=0x7fbb84009780 comp rx=0 tx=0).stop 2023-10-10T20:34:41.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.066+0000 7fbb99811700 1 -- 172.21.15.89:0/3149750371 shutdown_connections 2023-10-10T20:34:41.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.066+0000 7fbb99811700 1 --2- 172.21.15.89:0/3149750371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb940cba20 0x7fbb940ac970 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.066+0000 7fbb99811700 1 --2- 172.21.15.89:0/3149750371 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb94156760 0x7fbb94068ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.066+0000 7fbb99811700 1 -- 172.21.15.89:0/3149750371 >> 172.21.15.89:0/3149750371 conn(0x7fbb940cd000 msgr2=0x7fbb94068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.066+0000 7fbb99811700 1 -- 172.21.15.89:0/3149750371 shutdown_connections 2023-10-10T20:34:41.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.066+0000 7fbb99811700 1 -- 172.21.15.89:0/3149750371 wait complete. 2023-10-10T20:34:41.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.068+0000 7fbb99811700 1 Processor -- start 2023-10-10T20:34:41.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.068+0000 7fbb99811700 1 -- start start 2023-10-10T20:34:41.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.070+0000 7fbb99811700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb94068aa0 0x7fbb940a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.070+0000 7fbb99811700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb940cba20 0x7fbb9413f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.070+0000 7fbb99811700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb94156760 0x7fbb940aacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.070+0000 7fbb99811700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940a70e0 con 0x7fbb940cba20 2023-10-10T20:34:41.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.070+0000 7fbb99811700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940a73c0 con 0x7fbb94068aa0 2023-10-10T20:34:41.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.070+0000 7fbb99811700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940a76a0 con 0x7fbb94156760 2023-10-10T20:34:41.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.070+0000 7fbb927fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb940cba20 0x7fbb9413f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.070+0000 7fbb927fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb940cba20 0x7fbb9413f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53350/0 (socket says 172.21.15.89:53350) 2023-10-10T20:34:41.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.070+0000 7fbb927fc700 1 -- 172.21.15.89:0/2075015569 learned_addr learned my addr 172.21.15.89:0/2075015569 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.070+0000 7fbb937fe700 1 --2- 172.21.15.89:0/2075015569 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb94156760 0x7fbb940aacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.070+0000 7fbb92ffd700 1 --2- 172.21.15.89:0/2075015569 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb94068aa0 0x7fbb940a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.070+0000 7fbb937fe700 1 -- 172.21.15.89:0/2075015569 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb94068aa0 msgr2=0x7fbb940a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.070+0000 7fbb937fe700 1 --2- 172.21.15.89:0/2075015569 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb94068aa0 0x7fbb940a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.070+0000 7fbb937fe700 1 -- 172.21.15.89:0/2075015569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb940cba20 msgr2=0x7fbb9413f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.070+0000 7fbb937fe700 1 --2- 172.21.15.89:0/2075015569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb940cba20 0x7fbb9413f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.070+0000 7fbb937fe700 1 -- 172.21.15.89:0/2075015569 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb84009a20 con 0x7fbb94156760 2023-10-10T20:34:41.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.070+0000 7fbb927fc700 1 --2- 172.21.15.89:0/2075015569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb940cba20 0x7fbb9413f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:41.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.071+0000 7fbb937fe700 1 --2- 172.21.15.89:0/2075015569 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb94156760 0x7fbb940aacf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00ec80 tx=0x7fbb8c00d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.071+0000 7fbb7affd700 1 -- 172.21.15.89:0/2075015569 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c00b260 con 0x7fbb94156760 2023-10-10T20:34:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.071+0000 7fbb7affd700 1 -- 172.21.15.89:0/2075015569 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8c002600 con 0x7fbb94156760 2023-10-10T20:34:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.071+0000 7fbb7affd700 1 -- 172.21.15.89:0/2075015569 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0029c0 con 0x7fbb94156760 2023-10-10T20:34:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.071+0000 7fbb99811700 1 -- 172.21.15.89:0/2075015569 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb940a7980 con 0x7fbb94156760 2023-10-10T20:34:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.071+0000 7fbb99811700 1 -- 172.21.15.89:0/2075015569 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb940a6820 con 0x7fbb94156760 2023-10-10T20:34:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.072+0000 7fbb99811700 1 -- 172.21.15.89:0/2075015569 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb9404c3c0 con 0x7fbb94156760 2023-10-10T20:34:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.072+0000 7f0e01ea1700 1 -- 172.21.15.89:0/3908871464 --> [v2:172.21.15.111:6808/91096,v1:172.21.15.111:6809/91096] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0dfc0ad5c0 con 0x7f0de806f140 2023-10-10T20:34:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.072+0000 7f0de37fe700 1 -- 172.21.15.89:0/3908871464 <== osd.6 v2:172.21.15.111:6808/91096 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0dfc0ad5c0 con 0x7f0de806f140 2023-10-10T20:34:41.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.073+0000 7fbb7affd700 1 -- 172.21.15.89:0/2075015569 <== mon.2 v2:172.21.15.111:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb8c014480 con 0x7fbb94156760 2023-10-10T20:34:41.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.073+0000 7fbb7affd700 1 --2- 172.21.15.89:0/2075015569 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb80066f30 0x7fbb800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.073+0000 7fbb7affd700 1 -- 172.21.15.89:0/2075015569 <== mon.2 v2:172.21.15.111:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbb8c010040 con 0x7fbb94156760 2023-10-10T20:34:41.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.073+0000 7fbb92ffd700 1 --2- 172.21.15.89:0/2075015569 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb80066f30 0x7fbb800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.074+0000 7fbb7affd700 1 --2- 172.21.15.89:0/2075015569 >> [v2:172.21.15.111:6824/91126,v1:172.21.15.111:6825/91126] conn(0x7fbb8006f0c0 0x7fbb800714d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.074+0000 7fbb7affd700 1 -- 172.21.15.89:0/2075015569 --> [v2:172.21.15.111:6824/91126,v1:172.21.15.111:6825/91126] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb80071b80 con 0x7fbb8006f0c0 2023-10-10T20:34:41.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.074+0000 7fbb7affd700 1 -- 172.21.15.89:0/2075015569 <== mon.2 v2:172.21.15.111:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb8c084500 con 0x7fbb94156760 2023-10-10T20:34:41.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.074+0000 7fbb92ffd700 1 --2- 172.21.15.89:0/2075015569 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb80066f30 0x7fbb800693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbb840097c0 tx=0x7fbb8401e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.074+0000 7f31c571c700 1 -- 172.21.15.89:0/4035871707 --> [v2:172.21.15.89:6816/90897,v1:172.21.15.89:6817/90897] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f31c00acc70 con 0x7f31ac06f1c0 2023-10-10T20:34:41.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7fbb927fc700 1 --2- 172.21.15.89:0/2075015569 >> [v2:172.21.15.111:6824/91126,v1:172.21.15.111:6825/91126] conn(0x7fbb8006f0c0 0x7fbb800714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f727a7fc700 1 -- 172.21.15.89:0/1257067829 <== osd.2 v2:172.21.15.89:6800/90896 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7270002d30 con 0x7f727406f330 2023-10-10T20:34:41.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f0e01ea1700 1 -- 172.21.15.89:0/3908871464 >> [v2:172.21.15.111:6808/91096,v1:172.21.15.111:6809/91096] conn(0x7f0de806f140 msgr2=0x7f0de8071550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f0e01ea1700 1 --2- 172.21.15.89:0/3908871464 >> [v2:172.21.15.111:6808/91096,v1:172.21.15.111:6809/91096] conn(0x7f0de806f140 0x7f0de8071550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f0e01ea1700 1 -- 172.21.15.89:0/3908871464 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0de8066f30 msgr2=0x7f0de80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f0e01ea1700 1 --2- 172.21.15.89:0/3908871464 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0de8066f30 0x7f0de80693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0df00099b0 tx=0x7f0df0016000 comp rx=0 tx=0).stop 2023-10-10T20:34:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f0e01ea1700 1 -- 172.21.15.89:0/3908871464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0dfc0ccc40 msgr2=0x7f0dfc161bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f0e01ea1700 1 --2- 172.21.15.89:0/3908871464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0dfc0ccc40 0x7f0dfc161bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00ec80 tx=0x7f0dec0091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f0dfbfff700 1 -- 172.21.15.89:0/3908871464 reap_dead start 2023-10-10T20:34:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f0e01ea1700 1 -- 172.21.15.89:0/3908871464 shutdown_connections 2023-10-10T20:34:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f0e01ea1700 1 -- 172.21.15.89:0/3908871464 >> 172.21.15.89:0/3908871464 conn(0x7f0dfc06a320 msgr2=0x7f0dfc06d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f31a6ffd700 1 -- 172.21.15.89:0/4035871707 <== osd.1 v2:172.21.15.89:6816/90897 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f31c00acc70 con 0x7f31ac06f1c0 2023-10-10T20:34:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f7290c29700 1 -- 172.21.15.89:0/1257067829 >> [v2:172.21.15.89:6800/90896,v1:172.21.15.89:6801/90896] conn(0x7f727406f330 msgr2=0x7f7274071740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f7290c29700 1 --2- 172.21.15.89:0/1257067829 >> [v2:172.21.15.89:6800/90896,v1:172.21.15.89:6801/90896] conn(0x7f727406f330 0x7f7274071740 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7fbb927fc700 1 --2- 172.21.15.89:0/2075015569 >> [v2:172.21.15.111:6824/91126,v1:172.21.15.111:6825/91126] conn(0x7fbb8006f0c0 0x7fbb800714d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f7290c29700 1 -- 172.21.15.89:0/1257067829 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7274067120 msgr2=0x7f72740695d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f7290c29700 1 --2- 172.21.15.89:0/1257067829 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7274067120 0x7f72740695d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f72800093f0 tx=0x7f7280017000 comp rx=0 tx=0).stop 2023-10-10T20:34:41.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f7290c29700 1 -- 172.21.15.89:0/1257067829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f728c162080 msgr2=0x7f728c15ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f7290c29700 1 --2- 172.21.15.89:0/1257067829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f728c162080 0x7f728c15ff20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f728400c8e0 tx=0x7f7284001710 comp rx=0 tx=0).stop 2023-10-10T20:34:41.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f728ad9d700 1 -- 172.21.15.89:0/1257067829 reap_dead start 2023-10-10T20:34:41.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f31c571c700 1 -- 172.21.15.89:0/4035871707 >> [v2:172.21.15.89:6816/90897,v1:172.21.15.89:6817/90897] conn(0x7f31ac06f1c0 msgr2=0x7f31ac0715d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.075+0000 7f31c571c700 1 --2- 172.21.15.89:0/4035871707 >> [v2:172.21.15.89:6816/90897,v1:172.21.15.89:6817/90897] conn(0x7f31ac06f1c0 0x7f31ac0715d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.076+0000 7f31c571c700 1 -- 172.21.15.89:0/4035871707 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31ac066fb0 msgr2=0x7f31ac069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.076+0000 7f31c571c700 1 --2- 172.21.15.89:0/4035871707 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31ac066fb0 0x7f31ac069460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f31c0155860 tx=0x7f31b4017000 comp rx=0 tx=0).stop 2023-10-10T20:34:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.076+0000 7f31c571c700 1 -- 172.21.15.89:0/4035871707 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31c01471c0 msgr2=0x7f31c01540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.076+0000 7f31c571c700 1 --2- 172.21.15.89:0/4035871707 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31c01471c0 0x7f31c01540a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f31c006d340 tx=0x7f31b80020a0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.076+0000 7fe72f2e0700 1 -- 172.21.15.89:0/4207838496 --> [v2:172.21.15.89:6808/90898,v1:172.21.15.89:6809/90898] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe708000f90 con 0x7fe71406f180 2023-10-10T20:34:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.076+0000 7f31bf7fe700 1 -- 172.21.15.89:0/4035871707 reap_dead start 2023-10-10T20:34:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.076+0000 7f31c571c700 1 -- 172.21.15.89:0/4035871707 shutdown_connections 2023-10-10T20:34:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.076+0000 7f31c571c700 1 -- 172.21.15.89:0/4035871707 >> 172.21.15.89:0/4035871707 conn(0x7f31c00cd000 msgr2=0x7f31c0155e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.076+0000 7f31c571c700 1 -- 172.21.15.89:0/4035871707 shutdown_connections 2023-10-10T20:34:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.076+0000 7fbb7affd700 1 -- 172.21.15.89:0/2075015569 <== osd.7 v2:172.21.15.111:6824/91126 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fbb80071b80 con 0x7fbb8006f0c0 2023-10-10T20:34:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.076+0000 7f31c571c700 1 -- 172.21.15.89:0/4035871707 wait complete. 2023-10-10T20:34:41.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.077+0000 7fe71d7fa700 1 -- 172.21.15.89:0/4207838496 <== osd.0 v2:172.21.15.89:6808/90898 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe708000f90 con 0x7fe71406f180 2023-10-10T20:34:41.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.073+0000 7fa3dd117700 1 --2- 172.21.15.89:0/770926646 >> [v2:172.21.15.111:6816/91121,v1:172.21.15.111:6817/91121] conn(0x7fa3d8154380 0x7fa3d815e680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.074+0000 7fa3d4ff9700 1 -- 172.21.15.89:0/770926646 <== osd.5 v2:172.21.15.111:6816/91121 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fa3d8177070 con 0x7fa3d8154380 2023-10-10T20:34:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.079+0000 7fe72f2e0700 1 -- 172.21.15.89:0/4207838496 >> [v2:172.21.15.89:6808/90898,v1:172.21.15.89:6809/90898] conn(0x7fe71406f180 msgr2=0x7fe714071590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.079+0000 7fe72f2e0700 1 --2- 172.21.15.89:0/4207838496 >> [v2:172.21.15.89:6808/90898,v1:172.21.15.89:6809/90898] conn(0x7fe71406f180 0x7fe714071590 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.079+0000 7fe72f2e0700 1 -- 172.21.15.89:0/4207838496 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe714067160 msgr2=0x7fe714069610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.079+0000 7fe72f2e0700 1 --2- 172.21.15.89:0/4207838496 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe714067160 0x7fe714069610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe728163730 tx=0x7fe72000b040 comp rx=0 tx=0).stop 2023-10-10T20:34:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.079+0000 7fe72f2e0700 1 -- 172.21.15.89:0/4207838496 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7280afc40 msgr2=0x7fe728153e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.079+0000 7fe72f2e0700 1 --2- 172.21.15.89:0/4207838496 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7280afc40 0x7fe728153e80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe72806fca0 tx=0x7fe718017000 comp rx=0 tx=0).stop 2023-10-10T20:34:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.079+0000 7fe72d87d700 1 -- 172.21.15.89:0/4207838496 reap_dead start 2023-10-10T20:34:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.079+0000 7fe72f2e0700 1 -- 172.21.15.89:0/4207838496 shutdown_connections 2023-10-10T20:34:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.079+0000 7fe72f2e0700 1 -- 172.21.15.89:0/4207838496 >> 172.21.15.89:0/4207838496 conn(0x7fe72806a320 msgr2=0x7fe72815e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.079+0000 7f7290c29700 1 -- 172.21.15.89:0/1257067829 shutdown_connections 2023-10-10T20:34:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.079+0000 7f7290c29700 1 -- 172.21.15.89:0/1257067829 >> 172.21.15.89:0/1257067829 conn(0x7f728c0c9870 msgr2=0x7f728c0c3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.079+0000 7f7290c29700 1 -- 172.21.15.89:0/1257067829 shutdown_connections 2023-10-10T20:34:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.079+0000 7f7290c29700 1 -- 172.21.15.89:0/1257067829 wait complete. 2023-10-10T20:34:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.080+0000 7fc2ac667700 1 -- 172.21.15.89:0/1440695866 --> [v2:172.21.15.89:6820/90907,v1:172.21.15.89:6822/90907] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc2a40ac990 con 0x7fc28806f140 2023-10-10T20:34:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.080+0000 7fe72f2e0700 1 -- 172.21.15.89:0/4207838496 shutdown_connections 2023-10-10T20:34:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.080+0000 7fe72f2e0700 1 -- 172.21.15.89:0/4207838496 wait complete. 2023-10-10T20:34:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.080+0000 7fc29a7fc700 1 -- 172.21.15.89:0/1440695866 <== osd.3 v2:172.21.15.89:6820/90907 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc2a40ac990 con 0x7fc28806f140 2023-10-10T20:34:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.080+0000 7fc2ac667700 1 -- 172.21.15.89:0/1440695866 >> [v2:172.21.15.89:6820/90907,v1:172.21.15.89:6822/90907] conn(0x7fc28806f140 msgr2=0x7fc288071550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.080+0000 7fc2ac667700 1 --2- 172.21.15.89:0/1440695866 >> [v2:172.21.15.89:6820/90907,v1:172.21.15.89:6822/90907] conn(0x7fc28806f140 0x7fc288071550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.081+0000 7fc2ac667700 1 -- 172.21.15.89:0/1440695866 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc288066fb0 msgr2=0x7fc288069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.081+0000 7fc2ac667700 1 --2- 172.21.15.89:0/1440695866 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc288066fb0 0x7fc288069460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4068630 tx=0x7fc2a0007ab0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.081+0000 7fc2ac667700 1 -- 172.21.15.89:0/1440695866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2a4068aa0 msgr2=0x7fc2a40b1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.081+0000 7fc2ac667700 1 --2- 172.21.15.89:0/1440695866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2a4068aa0 0x7fc2a40b1660 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009fd0 tx=0x7fc29c018000 comp rx=0 tx=0).stop 2023-10-10T20:34:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.081+0000 7fc2aac04700 1 -- 172.21.15.89:0/1440695866 reap_dead start 2023-10-10T20:34:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.081+0000 7fc2ac667700 1 -- 172.21.15.89:0/1440695866 shutdown_connections 2023-10-10T20:34:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.081+0000 7fc2ac667700 1 -- 172.21.15.89:0/1440695866 >> 172.21.15.89:0/1440695866 conn(0x7fc2a40cd000 msgr2=0x7fc2a4155060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.081+0000 7fc2ac667700 1 -- 172.21.15.89:0/1440695866 shutdown_connections 2023-10-10T20:34:41.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.081+0000 7fc2ac667700 1 -- 172.21.15.89:0/1440695866 wait complete. 2023-10-10T20:34:41.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.082+0000 7f48f01d6700 1 -- 172.21.15.89:0/2629049506 --> [v2:172.21.15.111:6800/91095,v1:172.21.15.111:6801/91095] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f48e8064090 con 0x7f48cc06f210 2023-10-10T20:34:41.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.082+0000 7f48ddffb700 1 -- 172.21.15.89:0/2629049506 <== osd.4 v2:172.21.15.111:6800/91095 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f48e8064090 con 0x7f48cc06f210 2023-10-10T20:34:41.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.079+0000 7f0e01ea1700 1 -- 172.21.15.89:0/3908871464 shutdown_connections 2023-10-10T20:34:41.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.082+0000 7f0e01ea1700 1 -- 172.21.15.89:0/3908871464 wait complete. 2023-10-10T20:34:41.213 INFO:teuthology.orchestra.run.smithi089.stdout:55834574855 2023-10-10T20:34:41.214 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-10T20:34:41.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.097+0000 7f48f01d6700 1 -- 172.21.15.89:0/2629049506 >> [v2:172.21.15.111:6800/91095,v1:172.21.15.111:6801/91095] conn(0x7f48cc06f210 msgr2=0x7f48cc071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.097+0000 7f48f01d6700 1 --2- 172.21.15.89:0/2629049506 >> [v2:172.21.15.111:6800/91095,v1:172.21.15.111:6801/91095] conn(0x7f48cc06f210 0x7f48cc071620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.097+0000 7f48f01d6700 1 -- 172.21.15.89:0/2629049506 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f48cc067080 msgr2=0x7f48cc069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.097+0000 7f48f01d6700 1 --2- 172.21.15.89:0/2629049506 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f48cc067080 0x7f48cc069530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f48e4003080 tx=0x7f48e4017000 comp rx=0 tx=0).stop 2023-10-10T20:34:41.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.097+0000 7f48f01d6700 1 -- 172.21.15.89:0/2629049506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e80b21b0 msgr2=0x7f48e81568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.097+0000 7f48f01d6700 1 --2- 172.21.15.89:0/2629049506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e80b21b0 0x7f48e81568a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f48d800c7c0 tx=0x7f48d8007ea0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.097+0000 7f48ee773700 1 -- 172.21.15.89:0/2629049506 reap_dead start 2023-10-10T20:34:41.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.097+0000 7f48f01d6700 1 -- 172.21.15.89:0/2629049506 shutdown_connections 2023-10-10T20:34:41.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.097+0000 7f48f01d6700 1 -- 172.21.15.89:0/2629049506 >> 172.21.15.89:0/2629049506 conn(0x7f48e80cd000 msgr2=0x7f48e80ad3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.099+0000 7fa3deb7a700 1 -- 172.21.15.89:0/770926646 --> [v2:172.21.15.111:6816/91121,v1:172.21.15.111:6817/91121] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa3d804c3c0 con 0x7fa3d8154380 2023-10-10T20:34:41.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.099+0000 7f48f01d6700 1 -- 172.21.15.89:0/2629049506 shutdown_connections 2023-10-10T20:34:41.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.099+0000 7f48f01d6700 1 -- 172.21.15.89:0/2629049506 wait complete. 2023-10-10T20:34:41.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.099+0000 7fa3d4ff9700 1 -- 172.21.15.89:0/770926646 <== osd.5 v2:172.21.15.111:6816/91121 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa3d804c3c0 con 0x7fa3d8154380 2023-10-10T20:34:41.219 INFO:teuthology.orchestra.run.smithi089.stdout:55834574855 2023-10-10T20:34:41.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.102+0000 7fbb99811700 1 -- 172.21.15.89:0/2075015569 --> [v2:172.21.15.111:6824/91126,v1:172.21.15.111:6825/91126] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbb94158340 con 0x7fbb8006f0c0 2023-10-10T20:34:41.220 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-10T20:34:41.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.103+0000 7fa3deb7a700 1 -- 172.21.15.89:0/770926646 >> [v2:172.21.15.111:6816/91121,v1:172.21.15.111:6817/91121] conn(0x7fa3d8154380 msgr2=0x7fa3d815e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.103+0000 7fa3deb7a700 1 --2- 172.21.15.89:0/770926646 >> [v2:172.21.15.111:6816/91121,v1:172.21.15.111:6817/91121] conn(0x7fa3d8154380 0x7fa3d815e680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.103+0000 7fa3deb7a700 1 -- 172.21.15.89:0/770926646 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa3bc067080 msgr2=0x7fa3bc069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.103+0000 7fa3deb7a700 1 --2- 172.21.15.89:0/770926646 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa3bc067080 0x7fa3bc069530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80cc020 tx=0x7fa3cc017000 comp rx=0 tx=0).stop 2023-10-10T20:34:41.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.103+0000 7fa3deb7a700 1 -- 172.21.15.89:0/770926646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d806b980 msgr2=0x7fa3d81590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.103+0000 7fa3deb7a700 1 --2- 172.21.15.89:0/770926646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d806b980 0x7fa3d81590f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400c7c0 tx=0x7fa3c4007eb0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.103+0000 7fa3dd117700 1 -- 172.21.15.89:0/770926646 reap_dead start 2023-10-10T20:34:41.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.103+0000 7fa3deb7a700 1 -- 172.21.15.89:0/770926646 shutdown_connections 2023-10-10T20:34:41.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.103+0000 7fa3deb7a700 1 -- 172.21.15.89:0/770926646 >> 172.21.15.89:0/770926646 conn(0x7fa3d806a320 msgr2=0x7fa3d8157880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.103+0000 7fa3deb7a700 1 -- 172.21.15.89:0/770926646 shutdown_connections 2023-10-10T20:34:41.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.103+0000 7fa3deb7a700 1 -- 172.21.15.89:0/770926646 wait complete. 2023-10-10T20:34:41.225 INFO:teuthology.orchestra.run.smithi089.stdout:55834574853 2023-10-10T20:34:41.226 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-10T20:34:41.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.104+0000 7fbb7affd700 1 -- 172.21.15.89:0/2075015569 <== osd.7 v2:172.21.15.111:6824/91126 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbb94158340 con 0x7fbb8006f0c0 2023-10-10T20:34:41.227 INFO:teuthology.orchestra.run.smithi089.stdout:55834574855 2023-10-10T20:34:41.228 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-10T20:34:41.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.104+0000 7fbb99811700 1 -- 172.21.15.89:0/2075015569 >> [v2:172.21.15.111:6824/91126,v1:172.21.15.111:6825/91126] conn(0x7fbb8006f0c0 msgr2=0x7fbb800714d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.104+0000 7fbb99811700 1 --2- 172.21.15.89:0/2075015569 >> [v2:172.21.15.111:6824/91126,v1:172.21.15.111:6825/91126] conn(0x7fbb8006f0c0 0x7fbb800714d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.104+0000 7fbb99811700 1 -- 172.21.15.89:0/2075015569 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb80066f30 msgr2=0x7fbb800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.104+0000 7fbb99811700 1 --2- 172.21.15.89:0/2075015569 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb80066f30 0x7fbb800693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbb840097c0 tx=0x7fbb8401e000 comp rx=0 tx=0).stop 2023-10-10T20:34:41.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.104+0000 7fbb99811700 1 -- 172.21.15.89:0/2075015569 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb94156760 msgr2=0x7fbb940aacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.104+0000 7fbb99811700 1 --2- 172.21.15.89:0/2075015569 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb94156760 0x7fbb940aacf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00ec80 tx=0x7fbb8c00d040 comp rx=0 tx=0).stop 2023-10-10T20:34:41.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.104+0000 7fbb937fe700 1 -- 172.21.15.89:0/2075015569 reap_dead start 2023-10-10T20:34:41.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.104+0000 7fbb99811700 1 -- 172.21.15.89:0/2075015569 shutdown_connections 2023-10-10T20:34:41.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.104+0000 7fbb99811700 1 -- 172.21.15.89:0/2075015569 >> 172.21.15.89:0/2075015569 conn(0x7fbb940cd000 msgr2=0x7fbb9406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.104+0000 7fbb99811700 1 -- 172.21.15.89:0/2075015569 shutdown_connections 2023-10-10T20:34:41.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.104+0000 7fbb99811700 1 -- 172.21.15.89:0/2075015569 wait complete. 2023-10-10T20:34:41.233 INFO:teuthology.orchestra.run.smithi089.stdout:55834574855 2023-10-10T20:34:41.233 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-10T20:34:41.234 INFO:teuthology.orchestra.run.smithi089.stdout:55834574853 2023-10-10T20:34:41.235 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-10T20:34:41.235 INFO:teuthology.orchestra.run.smithi089.stdout:55834574855 2023-10-10T20:34:41.236 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-10T20:34:41.237 INFO:teuthology.orchestra.run.smithi089.stdout:55834574854 2023-10-10T20:34:41.237 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-10T20:34:41.309 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:34:41.366 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:34:41.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.457+0000 7fd62a980700 1 Processor -- start 2023-10-10T20:34:41.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.458+0000 7fd62a980700 1 -- start start 2023-10-10T20:34:41.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.459+0000 7f10238ee700 1 Processor -- start 2023-10-10T20:34:41.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.459+0000 7fd62a980700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6240b0010 0x7fd62415d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.459+0000 7fd62a980700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd62415dd90 0x7fd6240adb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.459+0000 7fd62a980700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6240b0750 con 0x7fd6240afc40 2023-10-10T20:34:41.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.459+0000 7fd62a980700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6240aca70 con 0x7fd62415dd90 2023-10-10T20:34:41.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.459+0000 7fd62a980700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6240acd50 con 0x7fd6240b0010 2023-10-10T20:34:41.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.458+0000 7f144992a700 1 Processor -- start 2023-10-10T20:34:41.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.458+0000 7f144992a700 1 -- start start 2023-10-10T20:34:41.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.459+0000 7f144992a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f14440b21b0 0x7f1444168d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.459+0000 7f144992a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f144415f260 con 0x7f14440691b0 2023-10-10T20:34:41.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.459+0000 7f144992a700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f144415e380 con 0x7f1444158440 2023-10-10T20:34:41.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.459+0000 7f144992a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14440aef20 con 0x7f14440b21b0 2023-10-10T20:34:41.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.459+0000 7fd6237fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6240b0010 0x7fd62415d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.459+0000 7fd6237fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6240b0010 0x7fd62415d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:43922/0 (socket says 172.21.15.89:43922) 2023-10-10T20:34:41.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.459+0000 7fd6237fe700 1 -- 172.21.15.89:0/3422853940 learned_addr learned my addr 172.21.15.89:0/3422853940 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.459+0000 7f14427fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f14440691b0 0x7f14440b1aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35682/0 (socket says 172.21.15.89:35682) 2023-10-10T20:34:41.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.459+0000 7f14427fc700 1 -- 172.21.15.89:0/2275649746 learned_addr learned my addr 172.21.15.89:0/2275649746 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:41.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.460+0000 7f14437fe700 1 --2- 172.21.15.89:0/2275649746 >> v2:172.21.15.111:3300/0 conn(0x7f14440b21b0 0x7f1444168d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.460+0000 7fd628f1d700 1 --2- 172.21.15.89:0/3422853940 >> v2:172.21.15.111:3300/0 conn(0x7fd62415dd90 0x7fd6240adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.459+0000 7f10238ee700 1 -- start start 2023-10-10T20:34:41.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.460+0000 7f14437fe700 1 -- 172.21.15.89:0/2275649746 >> v1:172.21.15.111:6789/0 conn(0x7f1444158440 legacy=0x7f1444068aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.460+0000 7f14437fe700 1 -- 172.21.15.89:0/2275649746 >> v1:172.21.15.89:6789/0 conn(0x7f14440691b0 legacy=0x7f14440b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:41.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.460+0000 7f14437fe700 1 -- 172.21.15.89:0/2275649746 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144415e650 con 0x7f14440b21b0 2023-10-10T20:34:41.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.460+0000 7fd6237fe700 1 -- 172.21.15.89:0/3422853940 >> v2:172.21.15.111:3300/0 conn(0x7fd62415dd90 msgr2=0x7fd6240adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:41.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.460+0000 7fd6237fe700 1 --2- 172.21.15.89:0/3422853940 >> v2:172.21.15.111:3300/0 conn(0x7fd62415dd90 0x7fd6240adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.460+0000 7fd6237fe700 1 -- 172.21.15.89:0/3422853940 >> v1:172.21.15.111:6789/0 conn(0x7fd6240afc40 legacy=0x7fd6240b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.460+0000 7fd6237fe700 1 -- 172.21.15.89:0/3422853940 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6240ad030 con 0x7fd6240b0010 2023-10-10T20:34:41.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.460+0000 7fd628f1d700 1 --2- 172.21.15.89:0/3422853940 >> v2:172.21.15.111:3300/0 conn(0x7fd62415dd90 0x7fd6240adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:41.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.460+0000 7f10238ee700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f101c06a5e0 0x7f101c06aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.460+0000 7f10238ee700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f101c0cb5a0 0x7f101c0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.460+0000 7f10238ee700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f101c069d30 con 0x7f101c156fb0 2023-10-10T20:34:41.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.460+0000 7f10238ee700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f101c06af70 con 0x7f101c06a5e0 2023-10-10T20:34:41.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.460+0000 7f10238ee700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101c063940 con 0x7f101c0cb5a0 2023-10-10T20:34:41.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.460+0000 7f1020e89700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f101c06a5e0 0x7f101c06aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.460+0000 7f1020e89700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f101c06a5e0 0x7f101c06aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53352/0 (socket says 172.21.15.89:53352) 2023-10-10T20:34:41.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.461+0000 7f1020e89700 1 -- 172.21.15.89:0/1212453190 learned_addr learned my addr 172.21.15.89:0/1212453190 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.460+0000 7f102168a700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f101c156fb0 0x7f101c063230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35686/0 (socket says 172.21.15.89:35686) 2023-10-10T20:34:41.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.460+0000 7f14437fe700 1 --2- 172.21.15.89:0/2275649746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14440b21b0 0x7f1444168d70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f143c0098e0 tx=0x7f143c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b731bc8c67df2f37 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.461+0000 7fd6237fe700 1 --2- 172.21.15.89:0/3422853940 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6240b0010 0x7fd62415d850 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd6180099b0 tx=0x7fd618009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=514777c610f44b34 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.461+0000 7f1021e8b700 1 --2- 172.21.15.89:0/1212453190 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f101c0cb5a0 0x7f101c0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.461+0000 7fd622ffd700 1 -- 172.21.15.89:0/3422853940 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd618009ea0 con 0x7fd6240b0010 2023-10-10T20:34:41.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.461+0000 7fd622ffd700 1 -- 172.21.15.89:0/3422853940 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61800b860 con 0x7fd6240b0010 2023-10-10T20:34:41.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.461+0000 7fd622ffd700 1 -- 172.21.15.89:0/3422853940 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61800bc00 con 0x7fd6240b0010 2023-10-10T20:34:41.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.461+0000 7f1441ffb700 1 -- 172.21.15.89:0/2275649746 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c009e80 con 0x7f14440b21b0 2023-10-10T20:34:41.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.461+0000 7f1441ffb700 1 -- 172.21.15.89:0/2275649746 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143c00b820 con 0x7f14440b21b0 2023-10-10T20:34:41.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.461+0000 7f1441ffb700 1 -- 172.21.15.89:0/2275649746 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c00bba0 con 0x7f14440b21b0 2023-10-10T20:34:41.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.461+0000 7f1020e89700 1 -- 172.21.15.89:0/1212453190 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f101c0cb5a0 msgr2=0x7f101c0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:41.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.461+0000 7f1020e89700 1 --2- 172.21.15.89:0/1212453190 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f101c0cb5a0 0x7f101c0cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.461+0000 7f1020e89700 1 -- 172.21.15.89:0/1212453190 >> v1:172.21.15.89:6789/0 conn(0x7f101c156fb0 legacy=0x7f101c063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.461+0000 7f1020e89700 1 -- 172.21.15.89:0/1212453190 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101c0c8110 con 0x7f101c06a5e0 2023-10-10T20:34:41.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.461+0000 7f1020e89700 1 --2- 172.21.15.89:0/1212453190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f101c06a5e0 0x7f101c06aa30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f100c0099e0 tx=0x7f100c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d61feba03c0de2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.461+0000 7f1013fff700 1 -- 172.21.15.89:0/1212453190 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c00b820 con 0x7f101c06a5e0 2023-10-10T20:34:41.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.461+0000 7f1013fff700 1 -- 172.21.15.89:0/1212453190 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100c00b980 con 0x7f101c06a5e0 2023-10-10T20:34:41.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.461+0000 7f1013fff700 1 -- 172.21.15.89:0/1212453190 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c00bd00 con 0x7f101c06a5e0 2023-10-10T20:34:41.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7fd62a980700 1 -- 172.21.15.89:0/3422853940 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6240b0010 msgr2=0x7fd62415d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7fd62a980700 1 --2- 172.21.15.89:0/3422853940 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6240b0010 0x7fd62415d850 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd6180099b0 tx=0x7fd618009250 comp rx=0 tx=0).stop 2023-10-10T20:34:41.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7f144992a700 1 -- 172.21.15.89:0/2275649746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14440b21b0 msgr2=0x7f1444168d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7f144992a700 1 --2- 172.21.15.89:0/2275649746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14440b21b0 0x7f1444168d70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f143c0098e0 tx=0x7f143c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7fd62a980700 1 -- 172.21.15.89:0/3422853940 shutdown_connections 2023-10-10T20:34:41.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7fd62a980700 1 --2- 172.21.15.89:0/3422853940 >> v2:172.21.15.111:3300/0 conn(0x7fd62415dd90 0x7fd6240adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7fd62a980700 1 --2- 172.21.15.89:0/3422853940 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6240b0010 0x7fd62415d850 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7fd62a980700 1 -- 172.21.15.89:0/3422853940 >> 172.21.15.89:0/3422853940 conn(0x7fd62406a320 msgr2=0x7fd6240cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7fd62a980700 1 -- 172.21.15.89:0/3422853940 shutdown_connections 2023-10-10T20:34:41.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7f144992a700 1 -- 172.21.15.89:0/2275649746 shutdown_connections 2023-10-10T20:34:41.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7f144992a700 1 --2- 172.21.15.89:0/2275649746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14440b21b0 0x7f1444168d70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7f144992a700 1 -- 172.21.15.89:0/2275649746 >> 172.21.15.89:0/2275649746 conn(0x7f14440cd000 msgr2=0x7f1444068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7fd62a980700 1 -- 172.21.15.89:0/3422853940 wait complete. 2023-10-10T20:34:41.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7f144992a700 1 -- 172.21.15.89:0/2275649746 shutdown_connections 2023-10-10T20:34:41.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7f144992a700 1 -- 172.21.15.89:0/2275649746 wait complete. 2023-10-10T20:34:41.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7f9907929700 1 Processor -- start 2023-10-10T20:34:41.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.463+0000 7f9907929700 1 -- start start 2023-10-10T20:34:41.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.463+0000 7f4dd862c700 1 Processor -- start 2023-10-10T20:34:41.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.463+0000 7f5b1de17700 1 Processor -- start 2023-10-10T20:34:41.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.463+0000 7f5b1de17700 1 -- start start 2023-10-10T20:34:41.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.464+0000 7fd62a980700 1 Processor -- start 2023-10-10T20:34:41.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.464+0000 7f4a0ba0b700 1 Processor -- start 2023-10-10T20:34:41.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.464+0000 7f144992a700 1 Processor -- start 2023-10-10T20:34:41.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7f10238ee700 1 -- 172.21.15.89:0/1212453190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f101c06a5e0 msgr2=0x7f101c06aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7f10238ee700 1 --2- 172.21.15.89:0/1212453190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f101c06a5e0 0x7f101c06aa30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f100c0099e0 tx=0x7f100c009250 comp rx=0 tx=0).stop 2023-10-10T20:34:41.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7f10238ee700 1 -- 172.21.15.89:0/1212453190 shutdown_connections 2023-10-10T20:34:41.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7f10238ee700 1 --2- 172.21.15.89:0/1212453190 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f101c0cb5a0 0x7f101c0cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7f10238ee700 1 --2- 172.21.15.89:0/1212453190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f101c06a5e0 0x7f101c06aa30 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f100c0099e0 tx=0x7f100c009250 comp rx=0 tx=0).stop 2023-10-10T20:34:41.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.462+0000 7f10238ee700 1 -- 172.21.15.89:0/1212453190 >> 172.21.15.89:0/1212453190 conn(0x7f101c0c9870 msgr2=0x7f101c0c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.464+0000 7fd62a980700 1 -- start start 2023-10-10T20:34:41.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.464+0000 7f9907929700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f99000b1f40 0x7f99000ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.464+0000 7f9907929700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9900159fb0 0x7f99000aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.464+0000 7f4a0ba0b700 1 -- start start 2023-10-10T20:34:41.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.464+0000 7f9907929700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99000accf0 con 0x7f9900068aa0 2023-10-10T20:34:41.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.464+0000 7f9907929700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99000ab110 con 0x7f99000b1f40 2023-10-10T20:34:41.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.464+0000 7f9907929700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990015a440 con 0x7f9900159fb0 2023-10-10T20:34:41.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.464+0000 7f99056c5700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f99000b1f40 0x7f99000ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.464+0000 7f9904ec4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9900159fb0 0x7f99000aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.464+0000 7f99056c5700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f99000b1f40 0x7f99000ac7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:42962/0 (socket says 172.21.15.89:42962) 2023-10-10T20:34:41.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.464+0000 7f9904ec4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9900159fb0 0x7f99000aabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:43930/0 (socket says 172.21.15.89:43930) 2023-10-10T20:34:41.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.464+0000 7f99056c5700 1 -- 172.21.15.89:0/2295924328 learned_addr learned my addr 172.21.15.89:0/2295924328 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f99056c5700 1 -- 172.21.15.89:0/2295924328 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9900159fb0 msgr2=0x7f99000aabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:41.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f99056c5700 1 --2- 172.21.15.89:0/2295924328 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9900159fb0 0x7f99000aabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f99056c5700 1 -- 172.21.15.89:0/2295924328 >> v1:172.21.15.111:6789/0 conn(0x7f9900068aa0 legacy=0x7f990015d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f99056c5700 1 -- 172.21.15.89:0/2295924328 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990015a720 con 0x7f99000b1f40 2023-10-10T20:34:41.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f5b1de17700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5b180656e0 0x7f5b1806da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f5b1de17700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5b1806a730 0x7f5b1806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f5b1de17700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1806a2a0 con 0x7f5b180c8fd0 2023-10-10T20:34:41.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f5b1de17700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b18156e70 con 0x7f5b180656e0 2023-10-10T20:34:41.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f5b1de17700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b1815c790 con 0x7f5b1806a730 2023-10-10T20:34:41.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.464+0000 7f4dd862c700 1 -- start start 2023-10-10T20:34:41.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f99056c5700 1 --2- 172.21.15.89:0/2295924328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99000b1f40 0x7f99000ac7b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009f20 tx=0x7f98f8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32289920c618dd4a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f5b16ffd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5b1806a730 0x7f5b1806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f5b16ffd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5b1806a730 0x7f5b1806dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:42978/0 (socket says 172.21.15.89:42978) 2023-10-10T20:34:41.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f5b16ffd700 1 -- 172.21.15.89:0/1727606053 learned_addr learned my addr 172.21.15.89:0/1727606053 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f98f7fff700 1 -- 172.21.15.89:0/2295924328 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f800e070 con 0x7f99000b1f40 2023-10-10T20:34:41.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f4dd862c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4dd00c91e0 0x7f4dd00cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f4dd862c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dd006ae10 0x7f4dd006b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f4dd862c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd00cc210 con 0x7f4dd006aa40 2023-10-10T20:34:41.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f4dd862c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd00cc4f0 con 0x7f4dd00c91e0 2023-10-10T20:34:41.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f4dd862c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00cc7d0 con 0x7f4dd006ae10 2023-10-10T20:34:41.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f98f7fff700 1 -- 172.21.15.89:0/2295924328 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f800b9d0 con 0x7f99000b1f40 2023-10-10T20:34:41.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f98f7fff700 1 -- 172.21.15.89:0/2295924328 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f80092e0 con 0x7f99000b1f40 2023-10-10T20:34:41.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7fd62a980700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6240afc40 0x7fd6241616e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7fd62a980700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6240b0010 0x7fd624153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7fd62a980700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd62415dd90 0x7fd624163080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7fd62a980700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd624161fe0 con 0x7fd6240afc40 2023-10-10T20:34:41.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7fd62a980700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6241622c0 con 0x7fd6240b0010 2023-10-10T20:34:41.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7fd62a980700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6241625a0 con 0x7fd62415dd90 2023-10-10T20:34:41.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.464+0000 7f144992a700 1 -- start start 2023-10-10T20:34:41.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f144992a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14440691b0 0x7f1444159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f144992a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14440b21b0 0x7f14441445f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f144992a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1444158440 0x7f1444144b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f144992a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14441591e0 con 0x7f1444158440 2023-10-10T20:34:41.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f144992a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14441588b0 con 0x7f14440b21b0 2023-10-10T20:34:41.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f144992a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1444158b90 con 0x7f14440691b0 2023-10-10T20:34:41.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f14427fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14440b21b0 0x7f14441445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f4dd63c8700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4dd006aa40 0x7f4dd006bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35700/0 (socket says 172.21.15.89:35700) 2023-10-10T20:34:41.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f4dd63c8700 1 -- 172.21.15.89:0/2261739293 learned_addr learned my addr 172.21.15.89:0/2261739293 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:41.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7fab4e5e3700 1 Processor -- start 2023-10-10T20:34:41.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7fab4e5e3700 1 -- start start 2023-10-10T20:34:41.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f14427fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14440b21b0 0x7f14441445f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:42982/0 (socket says 172.21.15.89:42982) 2023-10-10T20:34:41.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f14427fc700 1 -- 172.21.15.89:0/2711326011 learned_addr learned my addr 172.21.15.89:0/2711326011 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f4a0ba0b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4a041575f0 0x7f4a040abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f4a0ba0b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4a04068aa0 0x7f4a04068f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f4a0ba0b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a04069440 con 0x7f4a041579c0 2023-10-10T20:34:41.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f4a0ba0b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a040acfe0 con 0x7f4a04068aa0 2023-10-10T20:34:41.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f4a0ba0b700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a040ac3d0 con 0x7f4a041575f0 2023-10-10T20:34:41.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f10238ee700 1 -- 172.21.15.89:0/1212453190 shutdown_connections 2023-10-10T20:34:41.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7fd6237fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6240b0010 0x7fd624153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7fd6237fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6240b0010 0x7fd624153bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:42984/0 (socket says 172.21.15.89:42984) 2023-10-10T20:34:41.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7fd6237fe700 1 -- 172.21.15.89:0/2761560836 learned_addr learned my addr 172.21.15.89:0/2761560836 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f4dd6bc9700 1 --2- 172.21.15.89:0/2261739293 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dd006ae10 0x7f4dd006b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f4a08fa6700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4a041579c0 0x7f4a040cb770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35722/0 (socket says 172.21.15.89:35722) 2023-10-10T20:34:41.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f4a08fa6700 1 -- 172.21.15.89:0/2591602963 learned_addr learned my addr 172.21.15.89:0/2591602963 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:41.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7fd628f1d700 1 --2- 172.21.15.89:0/2761560836 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd62415dd90 0x7fd624163080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f9907929700 1 -- 172.21.15.89:0/2295924328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99000b1f40 msgr2=0x7f99000ac7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f9907929700 1 --2- 172.21.15.89:0/2295924328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99000b1f40 0x7f99000ac7b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009f20 tx=0x7f98f8009840 comp rx=0 tx=0).stop 2023-10-10T20:34:41.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7fd6237fe700 1 -- 172.21.15.89:0/2761560836 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd62415dd90 msgr2=0x7fd624163080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7fd6237fe700 1 --2- 172.21.15.89:0/2761560836 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd62415dd90 0x7fd624163080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7fd6237fe700 1 -- 172.21.15.89:0/2761560836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6240afc40 msgr2=0x7fd6241616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7fd6237fe700 1 --2- 172.21.15.89:0/2761560836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6240afc40 0x7fd6241616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7fd6237fe700 1 -- 172.21.15.89:0/2761560836 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c009a20 con 0x7fd6240b0010 2023-10-10T20:34:41.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7fd628f1d700 1 --2- 172.21.15.89:0/2761560836 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd62415dd90 0x7fd624163080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:41.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.465+0000 7f5b177fe700 1 --2- 172.21.15.89:0/1727606053 >> v2:172.21.15.89:3300/0 conn(0x7f5b180656e0 0x7f5b1806da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f5b16ffd700 1 -- 172.21.15.89:0/1727606053 >> v2:172.21.15.89:3300/0 conn(0x7f5b180656e0 msgr2=0x7f5b1806da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:41.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f5b16ffd700 1 --2- 172.21.15.89:0/1727606053 >> v2:172.21.15.89:3300/0 conn(0x7f5b180656e0 0x7f5b1806da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f5b16ffd700 1 -- 172.21.15.89:0/1727606053 >> v1:172.21.15.89:6789/0 conn(0x7f5b180c8fd0 legacy=0x7f5b180c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:41.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f5b16ffd700 1 -- 172.21.15.89:0/1727606053 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b18069c50 con 0x7f5b1806a730 2023-10-10T20:34:41.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f4dd6bc9700 1 -- 172.21.15.89:0/2261739293 >> v2:172.21.15.89:3300/0 conn(0x7f4dd00c91e0 msgr2=0x7f4dd00cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:41.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f4dd6bc9700 1 --2- 172.21.15.89:0/2261739293 >> v2:172.21.15.89:3300/0 conn(0x7f4dd00c91e0 0x7f4dd00cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f4dd6bc9700 1 -- 172.21.15.89:0/2261739293 >> v1:172.21.15.89:6789/0 conn(0x7f4dd006aa40 legacy=0x7f4dd006bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7f4dd6bc9700 1 -- 172.21.15.89:0/2261739293 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd00ccab0 con 0x7f4dd006ae10 2023-10-10T20:34:41.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f4a097a7700 1 --2- 172.21.15.89:0/2591602963 >> v2:172.21.15.111:3300/0 conn(0x7f4a041575f0 0x7f4a040abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7f5b16ffd700 1 --2- 172.21.15.89:0/1727606053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b1806a730 0x7f5b1806dfd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5b180ca2a0 tx=0x7f5b0c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f657d8cfa95aeaf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7fd6237fe700 1 --2- 172.21.15.89:0/2761560836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6240b0010 0x7fd624153bd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd6240ace90 tx=0x7fd618017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7fd60bfff700 1 -- 172.21.15.89:0/2761560836 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61800bbe0 con 0x7fd6240b0010 2023-10-10T20:34:41.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7fd60bfff700 1 -- 172.21.15.89:0/2761560836 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd618011420 con 0x7fd6240b0010 2023-10-10T20:34:41.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7fd60bfff700 1 -- 172.21.15.89:0/2761560836 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6180117c0 con 0x7fd6240b0010 2023-10-10T20:34:41.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7f4dd6bc9700 1 --2- 172.21.15.89:0/2261739293 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dd006ae10 0x7f4dd006b2b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009e90 tx=0x7f4dc8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8d86e45af007d08 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7fab4e5e3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fab4815dca0 0x7fab480b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7fab4e5e3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab4815e070 0x7fab48063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7fab4e5e3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab48161b40 con 0x7fab480b0470 2023-10-10T20:34:41.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7fab4e5e3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab480b35f0 con 0x7fab4815dca0 2023-10-10T20:34:41.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7fab4e5e3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab480b19b0 con 0x7fab4815e070 2023-10-10T20:34:41.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7fab477fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab4815e070 0x7fab48063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7fab4cb80700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fab480b0470 0x7fab4806e990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35724/0 (socket says 172.21.15.89:35724) 2023-10-10T20:34:41.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7fab4cb80700 1 -- 172.21.15.89:0/2609800787 learned_addr learned my addr 172.21.15.89:0/2609800787 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:41.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7fab47fff700 1 --2- 172.21.15.89:0/2609800787 >> v2:172.21.15.89:3300/0 conn(0x7fab4815dca0 0x7fab480b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7fab47fff700 1 -- 172.21.15.89:0/2609800787 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab4815e070 msgr2=0x7fab48063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:41.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7fab47fff700 1 --2- 172.21.15.89:0/2609800787 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab4815e070 0x7fab48063230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7fab47fff700 1 -- 172.21.15.89:0/2609800787 >> v1:172.21.15.89:6789/0 conn(0x7fab480b0470 legacy=0x7fab4806e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:41.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7fab47fff700 1 -- 172.21.15.89:0/2609800787 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab48161e20 con 0x7fab4815dca0 2023-10-10T20:34:41.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7fab47fff700 1 --2- 172.21.15.89:0/2609800787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab4815dca0 0x7fab480b2d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0098e0 tx=0x7fab3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2377aa8c7d061ca server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f14427fc700 1 -- 172.21.15.89:0/2711326011 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14440691b0 msgr2=0x7f1444159670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-10T20:34:41.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f14427fc700 1 --2- 172.21.15.89:0/2711326011 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14440691b0 0x7f1444159670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f14427fc700 1 -- 172.21.15.89:0/2711326011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1444158440 msgr2=0x7f1444144b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f14427fc700 1 --2- 172.21.15.89:0/2711326011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1444158440 0x7f1444144b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f14427fc700 1 -- 172.21.15.89:0/2711326011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143c009420 con 0x7f14440b21b0 2023-10-10T20:34:41.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f14427fc700 1 --2- 172.21.15.89:0/2711326011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14440b21b0 0x7f14441445f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f143800c5e0 tx=0x7f1438004a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7f142affd700 1 -- 172.21.15.89:0/2711326011 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1438005b30 con 0x7f14440b21b0 2023-10-10T20:34:41.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7f142affd700 1 -- 172.21.15.89:0/2711326011 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1438005c90 con 0x7f14440b21b0 2023-10-10T20:34:41.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7f142affd700 1 -- 172.21.15.89:0/2711326011 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1438007300 con 0x7f14440b21b0 2023-10-10T20:34:41.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f10238ee700 1 -- 172.21.15.89:0/1212453190 wait complete. 2023-10-10T20:34:41.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7f10238ee700 1 Processor -- start 2023-10-10T20:34:41.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.468+0000 7f5b167fc700 1 -- 172.21.15.89:0/1727606053 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c009e70 con 0x7f5b1806a730 2023-10-10T20:34:41.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.468+0000 7f4a097a7700 1 -- 172.21.15.89:0/2591602963 >> v2:172.21.15.89:3300/0 conn(0x7f4a04068aa0 msgr2=0x7f4a04068f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:41.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.468+0000 7f4a097a7700 1 --2- 172.21.15.89:0/2591602963 >> v2:172.21.15.89:3300/0 conn(0x7f4a04068aa0 0x7f4a04068f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.468+0000 7f4a097a7700 1 -- 172.21.15.89:0/2591602963 >> v1:172.21.15.89:6789/0 conn(0x7f4a041579c0 legacy=0x7f4a040cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.468+0000 7f4a097a7700 1 -- 172.21.15.89:0/2591602963 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a040ac6b0 con 0x7f4a041575f0 2023-10-10T20:34:41.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.468+0000 7f5b167fc700 1 -- 172.21.15.89:0/1727606053 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c00b820 con 0x7f5b1806a730 2023-10-10T20:34:41.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.468+0000 7f5b167fc700 1 -- 172.21.15.89:0/1727606053 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c00bbc0 con 0x7f5b1806a730 2023-10-10T20:34:41.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.468+0000 7fd62a980700 1 -- 172.21.15.89:0/2761560836 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6180094f0 con 0x7fd6240b0010 2023-10-10T20:34:41.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.468+0000 7fd62a980700 1 -- 172.21.15.89:0/2761560836 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd624162b60 con 0x7fd6240b0010 2023-10-10T20:34:41.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f9907929700 1 -- 172.21.15.89:0/2295924328 shutdown_connections 2023-10-10T20:34:41.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.466+0000 7f9907929700 1 --2- 172.21.15.89:0/2295924328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99000b1f40 0x7f99000ac7b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7f9907929700 1 --2- 172.21.15.89:0/2295924328 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9900159fb0 0x7f99000aabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7f9907929700 1 -- 172.21.15.89:0/2295924328 >> 172.21.15.89:0/2295924328 conn(0x7f99000cd000 msgr2=0x7f9900068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7f9907929700 1 -- 172.21.15.89:0/2295924328 shutdown_connections 2023-10-10T20:34:41.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7f9907929700 1 -- 172.21.15.89:0/2295924328 wait complete. 2023-10-10T20:34:41.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.468+0000 7f9907929700 1 Processor -- start 2023-10-10T20:34:41.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7f4dd53c6700 1 -- 172.21.15.89:0/2261739293 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc800bbf0 con 0x7f4dd006ae10 2023-10-10T20:34:41.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7f4dd53c6700 1 -- 172.21.15.89:0/2261739293 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc800bd50 con 0x7f4dd006ae10 2023-10-10T20:34:41.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.467+0000 7f4dd53c6700 1 -- 172.21.15.89:0/2261739293 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc80116b0 con 0x7f4dd006ae10 2023-10-10T20:34:41.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.468+0000 7f4dd862c700 1 -- 172.21.15.89:0/2261739293 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dd006ae10 msgr2=0x7f4dd006b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.468+0000 7f4dd862c700 1 --2- 172.21.15.89:0/2261739293 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dd006ae10 0x7f4dd006b2b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009e90 tx=0x7f4dc8009780 comp rx=0 tx=0).stop 2023-10-10T20:34:41.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.468+0000 7f9907929700 1 -- start start 2023-10-10T20:34:41.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.469+0000 7f4dd862c700 1 -- 172.21.15.89:0/2261739293 shutdown_connections 2023-10-10T20:34:41.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.469+0000 7f4dd862c700 1 --2- 172.21.15.89:0/2261739293 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dd006ae10 0x7f4dd006b2b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.469+0000 7f4dd862c700 1 --2- 172.21.15.89:0/2261739293 >> v2:172.21.15.89:3300/0 conn(0x7f4dd00c91e0 0x7f4dd00cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.469+0000 7f5b1de17700 1 -- 172.21.15.89:0/1727606053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b1806a730 msgr2=0x7f5b1806dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.469+0000 7f4dd862c700 1 -- 172.21.15.89:0/2261739293 >> 172.21.15.89:0/2261739293 conn(0x7f4dd00cd670 msgr2=0x7f4dd00cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.469+0000 7f5b1de17700 1 --2- 172.21.15.89:0/1727606053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b1806a730 0x7f5b1806dfd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5b180ca2a0 tx=0x7f5b0c009250 comp rx=0 tx=0).stop 2023-10-10T20:34:41.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f9907929700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9900068aa0 0x7f99000a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f9907929700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99000b1f40 0x7f99000a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f9907929700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9900159fb0 0x7f990017a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f9907929700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99000a6e50 con 0x7f9900159fb0 2023-10-10T20:34:41.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f9907929700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99000a65b0 con 0x7f99000b1f40 2023-10-10T20:34:41.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f9907929700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99000a6890 con 0x7f9900068aa0 2023-10-10T20:34:41.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.469+0000 7fab46ffd700 1 -- 172.21.15.89:0/2609800787 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c009e80 con 0x7fab4815dca0 2023-10-10T20:34:41.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.469+0000 7fab46ffd700 1 -- 172.21.15.89:0/2609800787 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3c00b820 con 0x7fab4815dca0 2023-10-10T20:34:41.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.469+0000 7fab46ffd700 1 -- 172.21.15.89:0/2609800787 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c00bbc0 con 0x7fab4815dca0 2023-10-10T20:34:41.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7fab4e5e3700 1 -- 172.21.15.89:0/2609800787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab4815dca0 msgr2=0x7fab480b2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7fab4e5e3700 1 --2- 172.21.15.89:0/2609800787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab4815dca0 0x7fab480b2d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0098e0 tx=0x7fab3c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f9905ec6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9900159fb0 0x7f990017a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f9905ec6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9900159fb0 0x7f990017a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53412/0 (socket says 172.21.15.89:53412) 2023-10-10T20:34:41.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f9905ec6700 1 -- 172.21.15.89:0/2753496244 learned_addr learned my addr 172.21.15.89:0/2753496244 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f9904ec4700 1 --2- 172.21.15.89:0/2753496244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99000b1f40 0x7f99000a81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f99056c5700 1 --2- 172.21.15.89:0/2753496244 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9900068aa0 0x7f99000a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7fd60bfff700 1 -- 172.21.15.89:0/2761560836 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd618011bb0 con 0x7fd6240b0010 2023-10-10T20:34:41.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.468+0000 7f144992a700 1 -- 172.21.15.89:0/2711326011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1444158e70 con 0x7f14440b21b0 2023-10-10T20:34:41.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.468+0000 7f144992a700 1 -- 172.21.15.89:0/2711326011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144415dd30 con 0x7f14440b21b0 2023-10-10T20:34:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7fab4e5e3700 1 -- 172.21.15.89:0/2609800787 shutdown_connections 2023-10-10T20:34:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7fab4e5e3700 1 --2- 172.21.15.89:0/2609800787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab4815dca0 0x7fab480b2d80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7fab4e5e3700 1 --2- 172.21.15.89:0/2609800787 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab4815e070 0x7fab48063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7fab4e5e3700 1 -- 172.21.15.89:0/2609800787 >> 172.21.15.89:0/2609800787 conn(0x7fab4806c1d0 msgr2=0x7fab4806c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.468+0000 7f10238ee700 1 -- start start 2023-10-10T20:34:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.469+0000 7f10238ee700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f101c0cb5a0 0x7f101c0b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.469+0000 7f10238ee700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f101c156fb0 0x7f101c0b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.469+0000 7f10238ee700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f101c176c40 0x7f101c1770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f144992a700 1 -- 172.21.15.89:0/2711326011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f144404c3c0 con 0x7f14440b21b0 2023-10-10T20:34:41.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.469+0000 7f10238ee700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c0b1350 con 0x7f101c156fb0 2023-10-10T20:34:41.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.469+0000 7f10238ee700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c157520 con 0x7f101c0cb5a0 2023-10-10T20:34:41.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.469+0000 7f10238ee700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101c157800 con 0x7f101c176c40 2023-10-10T20:34:41.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.469+0000 7f102168a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f101c0cb5a0 0x7f101c0b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.469+0000 7f102168a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f101c0cb5a0 0x7f101c0b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:43000/0 (socket says 172.21.15.89:43000) 2023-10-10T20:34:41.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.469+0000 7f102168a700 1 -- 172.21.15.89:0/1546816618 learned_addr learned my addr 172.21.15.89:0/1546816618 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f9904ec4700 1 -- 172.21.15.89:0/2753496244 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9900068aa0 msgr2=0x7f99000a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f9904ec4700 1 --2- 172.21.15.89:0/2753496244 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9900068aa0 0x7f99000a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f9904ec4700 1 -- 172.21.15.89:0/2753496244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9900159fb0 msgr2=0x7f990017a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f9904ec4700 1 --2- 172.21.15.89:0/2753496244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9900159fb0 0x7f990017a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f9904ec4700 1 -- 172.21.15.89:0/2753496244 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f8009ae0 con 0x7f99000b1f40 2023-10-10T20:34:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7fab4e5e3700 1 -- 172.21.15.89:0/2609800787 shutdown_connections 2023-10-10T20:34:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7fab4e5e3700 1 -- 172.21.15.89:0/2609800787 wait complete. 2023-10-10T20:34:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f5b1de17700 1 -- 172.21.15.89:0/1727606053 shutdown_connections 2023-10-10T20:34:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f5b1de17700 1 --2- 172.21.15.89:0/1727606053 >> v2:172.21.15.89:3300/0 conn(0x7f5b180656e0 0x7f5b1806da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f5b1de17700 1 --2- 172.21.15.89:0/1727606053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b1806a730 0x7f5b1806dfd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f5b1de17700 1 -- 172.21.15.89:0/1727606053 >> 172.21.15.89:0/1727606053 conn(0x7f5b18068390 msgr2=0x7f5b18068790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f5b1de17700 1 -- 172.21.15.89:0/1727606053 shutdown_connections 2023-10-10T20:34:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7f4a097a7700 1 --2- 172.21.15.89:0/2591602963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a041575f0 0x7f4a040abdf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4a04063d40 tx=0x7f49f40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28512be830ef15b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7f1021e8b700 1 --2- 172.21.15.89:0/1546816618 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f101c176c40 0x7f101c1770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.470+0000 7f5b1de17700 1 -- 172.21.15.89:0/1727606053 wait complete. 2023-10-10T20:34:41.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7f9904ec4700 1 --2- 172.21.15.89:0/2753496244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99000b1f40 0x7f99000a81b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00e900 tx=0x7f98fc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7f102168a700 1 -- 172.21.15.89:0/1546816618 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f101c176c40 msgr2=0x7f101c1770e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7f102168a700 1 --2- 172.21.15.89:0/1546816618 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f101c176c40 0x7f101c1770e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7f102168a700 1 -- 172.21.15.89:0/1546816618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f101c156fb0 msgr2=0x7f101c0b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7f102168a700 1 --2- 172.21.15.89:0/1546816618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f101c156fb0 0x7f101c0b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7f102168a700 1 -- 172.21.15.89:0/1546816618 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c0094f0 con 0x7f101c0cb5a0 2023-10-10T20:34:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7f1021e8b700 1 --2- 172.21.15.89:0/1546816618 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f101c176c40 0x7f101c1770e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7fd62a980700 1 -- 172.21.15.89:0/2761560836 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd62404c3c0 con 0x7fd6240b0010 2023-10-10T20:34:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7fd60bfff700 1 --2- 172.21.15.89:0/2761560836 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd610066f30 0x7fd6100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7f4dd862c700 1 -- 172.21.15.89:0/2261739293 shutdown_connections 2023-10-10T20:34:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7f142affd700 1 -- 172.21.15.89:0/2711326011 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1438007670 con 0x7f14440b21b0 2023-10-10T20:34:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7f98f57fa700 1 -- 172.21.15.89:0/2753496244 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc013680 con 0x7f99000b1f40 2023-10-10T20:34:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.472+0000 7f102168a700 1 --2- 172.21.15.89:0/1546816618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f101c0cb5a0 0x7f101c0b05c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f101400cd30 tx=0x7f1014001af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7fd60bfff700 1 -- 172.21.15.89:0/2761560836 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd618085f00 con 0x7fd6240b0010 2023-10-10T20:34:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7fd623fff700 1 --2- 172.21.15.89:0/2761560836 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd610066f30 0x7fd6100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7f142affd700 1 --2- 172.21.15.89:0/2711326011 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1430067080 0x7f1430069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7f142affd700 1 -- 172.21.15.89:0/2711326011 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1438083b90 con 0x7f14440b21b0 2023-10-10T20:34:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.472+0000 7f1442ffd700 1 --2- 172.21.15.89:0/2711326011 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1430067080 0x7f1430069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7f1442ffd700 1 --2- 172.21.15.89:0/2711326011 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1430067080 0x7f1430069530 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f14340012e0 tx=0x7f1434007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7f9907929700 1 -- 172.21.15.89:0/2753496244 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99000a6b70 con 0x7f99000b1f40 2023-10-10T20:34:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7f9907929700 1 -- 172.21.15.89:0/2753496244 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99001599a0 con 0x7f99000b1f40 2023-10-10T20:34:41.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.472+0000 7f5b1de17700 1 Processor -- start 2023-10-10T20:34:41.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.472+0000 7f4dd862c700 1 -- 172.21.15.89:0/2261739293 wait complete. 2023-10-10T20:34:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7f49fbfff700 1 -- 172.21.15.89:0/2591602963 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4011630 con 0x7f4a041575f0 2023-10-10T20:34:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7f49fbfff700 1 -- 172.21.15.89:0/2591602963 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f4011790 con 0x7f4a041575f0 2023-10-10T20:34:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.471+0000 7f49fbfff700 1 -- 172.21.15.89:0/2591602963 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4011b30 con 0x7f4a041575f0 2023-10-10T20:34:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.472+0000 7f4a0ba0b700 1 -- 172.21.15.89:0/2591602963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a041575f0 msgr2=0x7f4a040abdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.472+0000 7f4a0ba0b700 1 --2- 172.21.15.89:0/2591602963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a041575f0 0x7f4a040abdf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4a04063d40 tx=0x7f49f40091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.472+0000 7f4a0ba0b700 1 -- 172.21.15.89:0/2591602963 shutdown_connections 2023-10-10T20:34:41.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.472+0000 7f4a0ba0b700 1 --2- 172.21.15.89:0/2591602963 >> v2:172.21.15.89:3300/0 conn(0x7f4a04068aa0 0x7f4a04068f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.472+0000 7f4a0ba0b700 1 --2- 172.21.15.89:0/2591602963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a041575f0 0x7f4a040abdf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.472+0000 7f4a0ba0b700 1 -- 172.21.15.89:0/2591602963 >> 172.21.15.89:0/2591602963 conn(0x7f4a040cd000 msgr2=0x7f4a04068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.472+0000 7f4a0ba0b700 1 -- 172.21.15.89:0/2591602963 shutdown_connections 2023-10-10T20:34:41.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.472+0000 7f4a0ba0b700 1 -- 172.21.15.89:0/2591602963 wait complete. 2023-10-10T20:34:41.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.472+0000 7fab4e5e3700 1 Processor -- start 2023-10-10T20:34:41.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7f9907929700 1 -- 172.21.15.89:0/2753496244 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99000a8c00 con 0x7f99000b1f40 2023-10-10T20:34:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7fd623fff700 1 --2- 172.21.15.89:0/2761560836 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd610066f30 0x7fd6100693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd614001730 tx=0x7fd614007b40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7f4dd862c700 1 Processor -- start 2023-10-10T20:34:41.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.472+0000 7fab4e5e3700 1 -- start start 2023-10-10T20:34:41.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7fab4e5e3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab480b0470 0x7fab4814f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7fab4e5e3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab4815dca0 0x7fab4815f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7fab4e5e3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab4815e070 0x7fab4815fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7fab4e5e3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab48154d50 con 0x7fab4815e070 2023-10-10T20:34:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7fab4e5e3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4815b910 con 0x7fab480b0470 2023-10-10T20:34:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7fab4e5e3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4815aac0 con 0x7fab4815dca0 2023-10-10T20:34:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.472+0000 7f5b1de17700 1 -- start start 2023-10-10T20:34:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7f5b1de17700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b180656e0 0x7f5b1813b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7f5b1de17700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b1806a730 0x7f5b18143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7f5b1de17700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b180c8fd0 0x7f5b18143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7f5b1de17700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b181467a0 con 0x7f5b180c8fd0 2023-10-10T20:34:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7f5b1de17700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b18146ff0 con 0x7f5b1806a730 2023-10-10T20:34:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7f5b1de17700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b181435a0 con 0x7f5b180656e0 2023-10-10T20:34:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7fab477fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab4815dca0 0x7fab4815f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7fab477fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab4815dca0 0x7fab4815f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:44012/0 (socket says 172.21.15.89:44012) 2023-10-10T20:34:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7fab477fe700 1 -- 172.21.15.89:0/556564023 learned_addr learned my addr 172.21.15.89:0/556564023 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7fab47fff700 1 --2- 172.21.15.89:0/556564023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab480b0470 0x7fab4814f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7f5b16ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b1806a730 0x7f5b18143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7fab4cb80700 1 --2- 172.21.15.89:0/556564023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab4815e070 0x7fab4815fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7f5b16ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b1806a730 0x7f5b18143aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:43046/0 (socket says 172.21.15.89:43046) 2023-10-10T20:34:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7f5b16ffd700 1 -- 172.21.15.89:0/29887383 learned_addr learned my addr 172.21.15.89:0/29887383 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7f5b17fff700 1 --2- 172.21.15.89:0/29887383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b180c8fd0 0x7f5b18143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7fab477fe700 1 -- 172.21.15.89:0/556564023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab480b0470 msgr2=0x7fab4814f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7fab477fe700 1 --2- 172.21.15.89:0/556564023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab480b0470 0x7fab4814f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7fab477fe700 1 -- 172.21.15.89:0/556564023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab4815e070 msgr2=0x7fab4815fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7fab477fe700 1 --2- 172.21.15.89:0/556564023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab4815e070 0x7fab4815fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7f4a0ba0b700 1 Processor -- start 2023-10-10T20:34:41.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7fab477fe700 1 -- 172.21.15.89:0/556564023 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3c009420 con 0x7fab4815dca0 2023-10-10T20:34:41.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7f4a0ba0b700 1 -- start start 2023-10-10T20:34:41.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7f5b16ffd700 1 -- 172.21.15.89:0/29887383 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b180656e0 msgr2=0x7f5b1813b2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-10T20:34:41.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7f5b16ffd700 1 --2- 172.21.15.89:0/29887383 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b180656e0 0x7f5b1813b2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7f5b16ffd700 1 -- 172.21.15.89:0/29887383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b180c8fd0 msgr2=0x7f5b18143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7f5b16ffd700 1 --2- 172.21.15.89:0/29887383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b180c8fd0 0x7f5b18143fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7f5b16ffd700 1 -- 172.21.15.89:0/29887383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c0094f0 con 0x7f5b1806a730 2023-10-10T20:34:41.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7fab477fe700 1 --2- 172.21.15.89:0/556564023 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab4815dca0 0x7fab4815f870 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fab3800d860 tx=0x7fab38008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7f5b16ffd700 1 --2- 172.21.15.89:0/29887383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b1806a730 0x7f5b18143aa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5b1815c8d0 tx=0x7f5b0c016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7fab2ffff700 1 -- 172.21.15.89:0/556564023 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380125f0 con 0x7fab4815dca0 2023-10-10T20:34:41.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.473+0000 7f4dd862c700 1 -- start start 2023-10-10T20:34:41.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7f4dd862c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dd006aa40 0x7f4dd0147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7f4dd862c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dd006ae10 0x7f4dd00c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7f4dd862c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dd00c91e0 0x7f4dd006b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7f4dd862c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00b24b0 con 0x7f4dd006aa40 2023-10-10T20:34:41.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7f4dd862c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00b18a0 con 0x7f4dd00c91e0 2023-10-10T20:34:41.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7f4dd862c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00b1b80 con 0x7f4dd006ae10 2023-10-10T20:34:41.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4dd63c8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dd006aa40 0x7f4dd0147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4dd5bc7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dd006ae10 0x7f4dd00c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4dd63c8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dd006aa40 0x7f4dd0147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53444/0 (socket says 172.21.15.89:53444) 2023-10-10T20:34:41.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4dd63c8700 1 -- 172.21.15.89:0/1400972029 learned_addr learned my addr 172.21.15.89:0/1400972029 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4dd6bc9700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dd00c91e0 0x7f4dd006b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4dd5bc7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dd006ae10 0x7f4dd00c4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:44024/0 (socket says 172.21.15.89:44024) 2023-10-10T20:34:41.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7fab2ffff700 1 -- 172.21.15.89:0/556564023 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab38012750 con 0x7fab4815dca0 2023-10-10T20:34:41.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7fab4e5e3700 1 -- 172.21.15.89:0/556564023 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4815b2c0 con 0x7fab4815dca0 2023-10-10T20:34:41.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7fab2ffff700 1 -- 172.21.15.89:0/556564023 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38012b10 con 0x7fab4815dca0 2023-10-10T20:34:41.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.474+0000 7fab4e5e3700 1 -- 172.21.15.89:0/556564023 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4815ada0 con 0x7fab4815dca0 2023-10-10T20:34:41.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4a0ba0b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a04068aa0 0x7f4a04143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4a0ba0b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4a041575f0 0x7f4a040b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4a0ba0b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a041579c0 0x7f4a0414c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4a0ba0b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a040b14b0 con 0x7f4a041579c0 2023-10-10T20:34:41.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4a0ba0b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a040af2d0 con 0x7f4a04068aa0 2023-10-10T20:34:41.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4a0ba0b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a040af5b0 con 0x7f4a041575f0 2023-10-10T20:34:41.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4a097a7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a04068aa0 0x7f4a04143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4a09fa8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a041579c0 0x7f4a0414c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4a097a7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a04068aa0 0x7f4a04143ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:43064/0 (socket says 172.21.15.89:43064) 2023-10-10T20:34:41.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4a097a7700 1 -- 172.21.15.89:0/1748857423 learned_addr learned my addr 172.21.15.89:0/1748857423 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:41.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4dd5bc7700 1 -- 172.21.15.89:0/1400972029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dd00c91e0 msgr2=0x7f4dd006b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4dd5bc7700 1 --2- 172.21.15.89:0/1400972029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dd00c91e0 0x7f4dd006b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4dd5bc7700 1 -- 172.21.15.89:0/1400972029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dd006aa40 msgr2=0x7f4dd0147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4dd5bc7700 1 --2- 172.21.15.89:0/1400972029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dd006aa40 0x7f4dd0147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4dd5bc7700 1 -- 172.21.15.89:0/1400972029 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc8009a20 con 0x7f4dd006ae10 2023-10-10T20:34:41.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4dd63c8700 1 --2- 172.21.15.89:0/1400972029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dd006aa40 0x7f4dd0147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:41.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.475+0000 7f4dd5bc7700 1 --2- 172.21.15.89:0/1400972029 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dd006ae10 0x7f4dd00c4720 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000b5e0 tx=0x7f4dc00090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.476+0000 7fab4e5e3700 1 -- 172.21.15.89:0/556564023 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4815ba80 con 0x7fab4815dca0 2023-10-10T20:34:41.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.476+0000 7f98f57fa700 1 -- 172.21.15.89:0/2753496244 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98fc005a00 con 0x7f99000b1f40 2023-10-10T20:34:41.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.476+0000 7f98f57fa700 1 -- 172.21.15.89:0/2753496244 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc005b70 con 0x7f99000b1f40 2023-10-10T20:34:41.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.476+0000 7f5aff7fe700 1 -- 172.21.15.89:0/29887383 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c011db0 con 0x7f5b1806a730 2023-10-10T20:34:41.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.476+0000 7f5b1de17700 1 -- 172.21.15.89:0/29887383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b181469a0 con 0x7f5b1806a730 2023-10-10T20:34:41.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.476+0000 7f5b1de17700 1 -- 172.21.15.89:0/29887383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1815a030 con 0x7f5b1806a730 2023-10-10T20:34:41.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.476+0000 7f142affd700 1 -- 172.21.15.89:0/2711326011 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1438050de0 con 0x7f14440b21b0 2023-10-10T20:34:41.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.476+0000 7fab2ffff700 1 -- 172.21.15.89:0/556564023 <== mon.2 v2:172.21.15.111:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fab3801e460 con 0x7fab4815dca0 2023-10-10T20:34:41.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.477+0000 7f4dc67fc700 1 -- 172.21.15.89:0/1400972029 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc000f510 con 0x7f4dd006ae10 2023-10-10T20:34:41.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.476+0000 7f4a097a7700 1 -- 172.21.15.89:0/1748857423 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4a041575f0 msgr2=0x7f4a040b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.476+0000 7f4a097a7700 1 --2- 172.21.15.89:0/1748857423 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4a041575f0 0x7f4a040b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.476+0000 7f4a097a7700 1 -- 172.21.15.89:0/1748857423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a041579c0 msgr2=0x7f4a0414c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.476+0000 7f4a097a7700 1 --2- 172.21.15.89:0/1748857423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a041579c0 0x7f4a0414c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.476+0000 7f4a097a7700 1 -- 172.21.15.89:0/1748857423 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f4009420 con 0x7f4a04068aa0 2023-10-10T20:34:41.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.476+0000 7f4a097a7700 1 --2- 172.21.15.89:0/1748857423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a04068aa0 0x7f4a04143ea0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f49f40030c0 tx=0x7f49f4016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.477+0000 7fab2ffff700 1 --2- 172.21.15.89:0/556564023 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab34066f30 0x7fab340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.477+0000 7fab2ffff700 1 -- 172.21.15.89:0/556564023 <== mon.2 v2:172.21.15.111:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fab38084a30 con 0x7fab4815dca0 2023-10-10T20:34:41.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.477+0000 7fab47fff700 1 --2- 172.21.15.89:0/556564023 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab34066f30 0x7fab340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.477+0000 7f4dc67fc700 1 -- 172.21.15.89:0/1400972029 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc000f670 con 0x7f4dd006ae10 2023-10-10T20:34:41.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.477+0000 7f4dc67fc700 1 -- 172.21.15.89:0/1400972029 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc000fa10 con 0x7f4dd006ae10 2023-10-10T20:34:41.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.477+0000 7f4dd862c700 1 -- 172.21.15.89:0/1400972029 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd00b1e60 con 0x7f4dd006ae10 2023-10-10T20:34:41.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.477+0000 7f4dd862c700 1 -- 172.21.15.89:0/1400972029 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd0147770 con 0x7f4dd006ae10 2023-10-10T20:34:41.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.478+0000 7f5b1de17700 1 -- 172.21.15.89:0/29887383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b18143910 con 0x7f5b1806a730 2023-10-10T20:34:41.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.478+0000 7fab47fff700 1 --2- 172.21.15.89:0/556564023 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab34066f30 0x7fab340693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0098e0 tx=0x7fab3c016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.478+0000 7f4dd862c700 1 -- 172.21.15.89:0/1400972029 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db80052b0 con 0x7f4dd006ae10 2023-10-10T20:34:41.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.480+0000 7f10117fa700 1 -- 172.21.15.89:0/1546816618 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014005df0 con 0x7f101c0cb5a0 2023-10-10T20:34:41.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.481+0000 7f10238ee700 1 -- 172.21.15.89:0/1546816618 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101c157ae0 con 0x7f101c0cb5a0 2023-10-10T20:34:41.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.481+0000 7f10117fa700 1 -- 172.21.15.89:0/1546816618 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101400a960 con 0x7f101c0cb5a0 2023-10-10T20:34:41.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.481+0000 7f10117fa700 1 -- 172.21.15.89:0/1546816618 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101400ad20 con 0x7f101c0cb5a0 2023-10-10T20:34:41.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.481+0000 7f10238ee700 1 -- 172.21.15.89:0/1546816618 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101c0b1040 con 0x7f101c0cb5a0 2023-10-10T20:34:41.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.481+0000 7f5aff7fe700 1 -- 172.21.15.89:0/29887383 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c01b680 con 0x7f5b1806a730 2023-10-10T20:34:41.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.481+0000 7f5aff7fe700 1 -- 172.21.15.89:0/29887383 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c01b7f0 con 0x7f5b1806a730 2023-10-10T20:34:41.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.481+0000 7f49f97fa700 1 -- 172.21.15.89:0/1748857423 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f401a450 con 0x7f4a04068aa0 2023-10-10T20:34:41.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.481+0000 7f49f97fa700 1 -- 172.21.15.89:0/1748857423 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f401a5b0 con 0x7f4a04068aa0 2023-10-10T20:34:41.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.481+0000 7f49f97fa700 1 -- 172.21.15.89:0/1748857423 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f401a950 con 0x7f4a04068aa0 2023-10-10T20:34:41.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.481+0000 7fd60bfff700 1 -- 172.21.15.89:0/2761560836 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6180530d0 con 0x7fd6240b0010 2023-10-10T20:34:41.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.483+0000 7f10238ee700 1 -- 172.21.15.89:0/1546816618 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101c157380 con 0x7f101c0cb5a0 2023-10-10T20:34:41.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.483+0000 7f10117fa700 1 -- 172.21.15.89:0/1546816618 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1014007300 con 0x7f101c0cb5a0 2023-10-10T20:34:41.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.483+0000 7f10117fa700 1 --2- 172.21.15.89:0/1546816618 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1000066f30 0x7f10000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.483+0000 7f98f57fa700 1 -- 172.21.15.89:0/2753496244 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f98fc0139f0 con 0x7f99000b1f40 2023-10-10T20:34:41.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.483+0000 7f98f57fa700 1 --2- 172.21.15.89:0/2753496244 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f98e4066f30 0x7f98e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.483+0000 7f98f57fa700 1 -- 172.21.15.89:0/2753496244 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f98fc084be0 con 0x7f99000b1f40 2023-10-10T20:34:41.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.484+0000 7f99056c5700 1 --2- 172.21.15.89:0/2753496244 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f98e4066f30 0x7f98e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.483+0000 7f4dc67fc700 1 -- 172.21.15.89:0/1400972029 <== mon.2 v2:172.21.15.111:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4dc0024430 con 0x7f4dd006ae10 2023-10-10T20:34:41.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.482+0000 7f4a0ba0b700 1 -- 172.21.15.89:0/1748857423 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a040af890 con 0x7f4a04068aa0 2023-10-10T20:34:41.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.482+0000 7f4a0ba0b700 1 -- 172.21.15.89:0/1748857423 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a041443e0 con 0x7f4a04068aa0 2023-10-10T20:34:41.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.483+0000 7f4a0ba0b700 1 -- 172.21.15.89:0/1748857423 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a0404c3c0 con 0x7f4a04068aa0 2023-10-10T20:34:41.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.482+0000 7fab2ffff700 1 -- 172.21.15.89:0/556564023 <== mon.2 v2:172.21.15.111:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab38055730 con 0x7fab4815dca0 2023-10-10T20:34:41.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.484+0000 7f1020e89700 1 --2- 172.21.15.89:0/1546816618 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1000066f30 0x7f10000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.482+0000 7f5aff7fe700 1 -- 172.21.15.89:0/29887383 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5b0c01b950 con 0x7f5b1806a730 2023-10-10T20:34:41.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.483+0000 7f5aff7fe700 1 --2- 172.21.15.89:0/29887383 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b04066f30 0x7f5b040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.483+0000 7f5b177fe700 1 --2- 172.21.15.89:0/29887383 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b04066f30 0x7f5b040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.484+0000 7f4dc67fc700 1 --2- 172.21.15.89:0/1400972029 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4db4067000 0x7f4db40694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.484+0000 7f4dc67fc700 1 -- 172.21.15.89:0/1400972029 <== mon.2 v2:172.21.15.111:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4dc0084970 con 0x7f4dd006ae10 2023-10-10T20:34:41.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.484+0000 7f99056c5700 1 --2- 172.21.15.89:0/2753496244 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f98e4066f30 0x7f98e40693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009880 tx=0x7f98f8018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.484+0000 7f5b177fe700 1 --2- 172.21.15.89:0/29887383 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b04066f30 0x7f5b040693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009bb0 tx=0x7f5b080091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.484+0000 7f1020e89700 1 --2- 172.21.15.89:0/1546816618 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1000066f30 0x7f10000693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f100c0099e0 tx=0x7f100c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.484+0000 7f4dd63c8700 1 --2- 172.21.15.89:0/1400972029 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4db4067000 0x7f4db40694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.484+0000 7f4dd63c8700 1 --2- 172.21.15.89:0/1400972029 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4db4067000 0x7f4db40694b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc000c00 tx=0x7f4dcc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.486+0000 7f5aff7fe700 1 -- 172.21.15.89:0/29887383 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5b0c057870 con 0x7f5b1806a730 2023-10-10T20:34:41.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.486+0000 7f98f57fa700 1 -- 172.21.15.89:0/2753496244 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98fc051db0 con 0x7f99000b1f40 2023-10-10T20:34:41.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.486+0000 7f10117fa700 1 -- 172.21.15.89:0/1546816618 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1014055d00 con 0x7f101c0cb5a0 2023-10-10T20:34:41.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.487+0000 7f4dc67fc700 1 -- 172.21.15.89:0/1400972029 <== mon.2 v2:172.21.15.111:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4dc0051cf0 con 0x7f4dd006ae10 2023-10-10T20:34:41.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.487+0000 7f49f97fa700 1 -- 172.21.15.89:0/1748857423 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f49f401ad00 con 0x7f4a04068aa0 2023-10-10T20:34:41.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.488+0000 7f49f97fa700 1 --2- 172.21.15.89:0/1748857423 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f49e8066f30 0x7f49e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:41.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.488+0000 7f4a08fa6700 1 --2- 172.21.15.89:0/1748857423 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f49e8066f30 0x7f49e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:41.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.488+0000 7f4a08fa6700 1 --2- 172.21.15.89:0/1748857423 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f49e8066f30 0x7f49e80693e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4a000058d0 tx=0x7f4a00007ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:41.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.490+0000 7f49f97fa700 1 -- 172.21.15.89:0/1748857423 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f49f40579f0 con 0x7f4a04068aa0 2023-10-10T20:34:41.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.490+0000 7f5aff7fe700 1 -- 172.21.15.89:0/29887383 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b0c0534b0 con 0x7f5b1806a730 2023-10-10T20:34:41.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.494+0000 7f10117fa700 1 -- 172.21.15.89:0/1546816618 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f101c157380 con 0x7f101c0cb5a0 2023-10-10T20:34:41.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.499+0000 7f49f97fa700 1 -- 172.21.15.89:0/1748857423 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49f4053d30 con 0x7f4a04068aa0 2023-10-10T20:34:41.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.849+0000 7fd62a980700 1 -- 172.21.15.89:0/2761560836 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd6240ad310 con 0x7fd6240b0010 2023-10-10T20:34:41.855 INFO:teuthology.orchestra.run.smithi089.stdout:55834574854 2023-10-10T20:34:41.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.850+0000 7fd60bfff700 1 -- 172.21.15.89:0/2761560836 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd618015020 con 0x7fd6240b0010 2023-10-10T20:34:41.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.853+0000 7fd62a980700 1 -- 172.21.15.89:0/2761560836 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd610066f30 msgr2=0x7fd6100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.854+0000 7fd62a980700 1 --2- 172.21.15.89:0/2761560836 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd610066f30 0x7fd6100693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd614001730 tx=0x7fd614007b40 comp rx=0 tx=0).stop 2023-10-10T20:34:41.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.854+0000 7fd62a980700 1 -- 172.21.15.89:0/2761560836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6240b0010 msgr2=0x7fd624153bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.854+0000 7fd62a980700 1 --2- 172.21.15.89:0/2761560836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6240b0010 0x7fd624153bd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd6240ace90 tx=0x7fd618017000 comp rx=0 tx=0).stop 2023-10-10T20:34:41.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.854+0000 7fd62a980700 1 -- 172.21.15.89:0/2761560836 shutdown_connections 2023-10-10T20:34:41.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.854+0000 7fd62a980700 1 --2- 172.21.15.89:0/2761560836 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd62415dd90 0x7fd624163080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.854+0000 7fd62a980700 1 --2- 172.21.15.89:0/2761560836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6240afc40 0x7fd6241616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.854+0000 7fd62a980700 1 --2- 172.21.15.89:0/2761560836 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd610066f30 0x7fd6100693e0 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd614001730 tx=0x7fd614007b40 comp rx=0 tx=0).stop 2023-10-10T20:34:41.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.854+0000 7fd62a980700 1 --2- 172.21.15.89:0/2761560836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6240b0010 0x7fd624153bd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.854+0000 7fd62a980700 1 -- 172.21.15.89:0/2761560836 >> 172.21.15.89:0/2761560836 conn(0x7fd62406a320 msgr2=0x7fd62415cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.854+0000 7fd62a980700 1 -- 172.21.15.89:0/2761560836 shutdown_connections 2023-10-10T20:34:41.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.854+0000 7fd62a980700 1 -- 172.21.15.89:0/2761560836 wait complete. 2023-10-10T20:34:41.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.860+0000 7f144992a700 1 -- 172.21.15.89:0/2711326011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f14441589f0 con 0x7f14440b21b0 2023-10-10T20:34:41.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.865+0000 7fab4e5e3700 1 -- 172.21.15.89:0/556564023 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fab4804c3c0 con 0x7fab4815dca0 2023-10-10T20:34:41.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.866+0000 7fab2ffff700 1 -- 172.21.15.89:0/556564023 <== mon.2 v2:172.21.15.111:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fab380552c0 con 0x7fab4815dca0 2023-10-10T20:34:41.868 INFO:teuthology.orchestra.run.smithi089.stdout:55834574853 2023-10-10T20:34:41.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.867+0000 7f142affd700 1 -- 172.21.15.89:0/2711326011 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f14441589f0 con 0x7f14440b21b0 2023-10-10T20:34:41.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.870+0000 7fab4e5e3700 1 -- 172.21.15.89:0/556564023 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab34066f30 msgr2=0x7fab340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.870+0000 7fab4e5e3700 1 --2- 172.21.15.89:0/556564023 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab34066f30 0x7fab340693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0098e0 tx=0x7fab3c016000 comp rx=0 tx=0).stop 2023-10-10T20:34:41.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.870+0000 7fab4e5e3700 1 -- 172.21.15.89:0/556564023 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab4815dca0 msgr2=0x7fab4815f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.870+0000 7fab4e5e3700 1 --2- 172.21.15.89:0/556564023 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab4815dca0 0x7fab4815f870 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fab3800d860 tx=0x7fab38008040 comp rx=0 tx=0).stop 2023-10-10T20:34:41.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.870+0000 7fab4e5e3700 1 -- 172.21.15.89:0/556564023 shutdown_connections 2023-10-10T20:34:41.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.870+0000 7fab4e5e3700 1 --2- 172.21.15.89:0/556564023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab4815e070 0x7fab4815fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.870+0000 7fab4e5e3700 1 --2- 172.21.15.89:0/556564023 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab34066f30 0x7fab340693e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.870+0000 7fab4e5e3700 1 --2- 172.21.15.89:0/556564023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab480b0470 0x7fab4814f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.870+0000 7fab4e5e3700 1 --2- 172.21.15.89:0/556564023 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab4815dca0 0x7fab4815f870 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.870+0000 7fab4e5e3700 1 -- 172.21.15.89:0/556564023 >> 172.21.15.89:0/556564023 conn(0x7fab4806c1d0 msgr2=0x7fab48153cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.875 INFO:teuthology.orchestra.run.smithi089.stdout:55834574852 2023-10-10T20:34:41.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.871+0000 7fab4e5e3700 1 -- 172.21.15.89:0/556564023 shutdown_connections 2023-10-10T20:34:41.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.871+0000 7fab4e5e3700 1 -- 172.21.15.89:0/556564023 wait complete. 2023-10-10T20:34:41.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2023-10-10T20:34:41.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.873+0000 7f144992a700 1 -- 172.21.15.89:0/2711326011 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1430067080 msgr2=0x7f1430069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.873+0000 7f144992a700 1 --2- 172.21.15.89:0/2711326011 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1430067080 0x7f1430069530 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f14340012e0 tx=0x7f1434007ec0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.873+0000 7f144992a700 1 -- 172.21.15.89:0/2711326011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14440b21b0 msgr2=0x7f14441445f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.873+0000 7f144992a700 1 --2- 172.21.15.89:0/2711326011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14440b21b0 0x7f14441445f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f143800c5e0 tx=0x7f1438004a80 comp rx=0 tx=0).stop 2023-10-10T20:34:41.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.874+0000 7f144992a700 1 -- 172.21.15.89:0/2711326011 shutdown_connections 2023-10-10T20:34:41.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.874+0000 7f144992a700 1 --2- 172.21.15.89:0/2711326011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1444158440 0x7f1444144b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.874+0000 7f144992a700 1 --2- 172.21.15.89:0/2711326011 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f14440691b0 0x7f1444159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.874+0000 7f144992a700 1 --2- 172.21.15.89:0/2711326011 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1430067080 0x7f1430069530 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.874+0000 7f144992a700 1 --2- 172.21.15.89:0/2711326011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14440b21b0 0x7f14441445f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.874+0000 7f144992a700 1 -- 172.21.15.89:0/2711326011 >> 172.21.15.89:0/2711326011 conn(0x7f14440cd000 msgr2=0x7f14440ac120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.875+0000 7f144992a700 1 -- 172.21.15.89:0/2711326011 shutdown_connections 2023-10-10T20:34:41.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.875+0000 7f4dd862c700 1 -- 172.21.15.89:0/1400972029 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f4db8005830 con 0x7f4dd006ae10 2023-10-10T20:34:41.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.875+0000 7f144992a700 1 -- 172.21.15.89:0/2711326011 wait complete. 2023-10-10T20:34:41.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.875+0000 7f4dc67fc700 1 -- 172.21.15.89:0/1400972029 <== mon.2 v2:172.21.15.111:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4dc0055310 con 0x7f4dd006ae10 2023-10-10T20:34:41.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.876+0000 7f9907929700 1 -- 172.21.15.89:0/2753496244 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f990004c3c0 con 0x7f99000b1f40 2023-10-10T20:34:41.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.876+0000 7f98f57fa700 1 -- 172.21.15.89:0/2753496244 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f98fc017090 con 0x7f99000b1f40 2023-10-10T20:34:41.883 INFO:teuthology.orchestra.run.smithi089.stdout:55834574854 2023-10-10T20:34:41.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.877+0000 7f5b1de17700 1 -- 172.21.15.89:0/29887383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5b1804c3c0 con 0x7f5b1806a730 2023-10-10T20:34:41.884 INFO:teuthology.orchestra.run.smithi089.stdout:55834574852 2023-10-10T20:34:41.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.877+0000 7f5aff7fe700 1 -- 172.21.15.89:0/29887383 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5b0c02a070 con 0x7f5b1806a730 2023-10-10T20:34:41.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.880+0000 7f9907929700 1 -- 172.21.15.89:0/2753496244 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f98e4066f30 msgr2=0x7f98e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.880+0000 7f9907929700 1 --2- 172.21.15.89:0/2753496244 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f98e4066f30 0x7f98e40693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009880 tx=0x7f98f8018000 comp rx=0 tx=0).stop 2023-10-10T20:34:41.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.880+0000 7f9907929700 1 -- 172.21.15.89:0/2753496244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99000b1f40 msgr2=0x7f99000a81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.880+0000 7f9907929700 1 --2- 172.21.15.89:0/2753496244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99000b1f40 0x7f99000a81b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00e900 tx=0x7f98fc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.880+0000 7f9907929700 1 -- 172.21.15.89:0/2753496244 shutdown_connections 2023-10-10T20:34:41.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.880+0000 7f9907929700 1 --2- 172.21.15.89:0/2753496244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9900159fb0 0x7f990017a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.880+0000 7f9907929700 1 --2- 172.21.15.89:0/2753496244 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9900068aa0 0x7f99000a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.880+0000 7f9907929700 1 --2- 172.21.15.89:0/2753496244 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f98e4066f30 0x7f98e40693e0 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009880 tx=0x7f98f8018000 comp rx=0 tx=0).stop 2023-10-10T20:34:41.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.880+0000 7f9907929700 1 --2- 172.21.15.89:0/2753496244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99000b1f40 0x7f99000a81b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.880+0000 7f9907929700 1 -- 172.21.15.89:0/2753496244 >> 172.21.15.89:0/2753496244 conn(0x7f99000cd000 msgr2=0x7f990006d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.880+0000 7f4dd862c700 1 -- 172.21.15.89:0/1400972029 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4db4067000 msgr2=0x7f4db40694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.880+0000 7f4dd862c700 1 --2- 172.21.15.89:0/1400972029 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4db4067000 0x7f4db40694b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc000c00 tx=0x7f4dcc009040 comp rx=0 tx=0).stop 2023-10-10T20:34:41.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.880+0000 7f4dd862c700 1 -- 172.21.15.89:0/1400972029 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dd006ae10 msgr2=0x7f4dd00c4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.880+0000 7f4dd862c700 1 --2- 172.21.15.89:0/1400972029 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dd006ae10 0x7f4dd00c4720 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000b5e0 tx=0x7f4dc00090f0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.882+0000 7f9907929700 1 -- 172.21.15.89:0/2753496244 shutdown_connections 2023-10-10T20:34:41.889 INFO:teuthology.orchestra.run.smithi089.stdout:55834574854 2023-10-10T20:34:41.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.882+0000 7f9907929700 1 -- 172.21.15.89:0/2753496244 wait complete. 2023-10-10T20:34:41.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.882+0000 7f10238ee700 1 -- 172.21.15.89:0/1546816618 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f101c157380 con 0x7f101c0cb5a0 2023-10-10T20:34:41.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.883+0000 7f4dd862c700 1 -- 172.21.15.89:0/1400972029 shutdown_connections 2023-10-10T20:34:41.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.883+0000 7f4dd862c700 1 --2- 172.21.15.89:0/1400972029 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4db4067000 0x7f4db40694b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.883+0000 7f4dd862c700 1 --2- 172.21.15.89:0/1400972029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dd00c91e0 0x7f4dd006b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.883+0000 7f4dd862c700 1 --2- 172.21.15.89:0/1400972029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dd006aa40 0x7f4dd0147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.883+0000 7f4dd862c700 1 --2- 172.21.15.89:0/1400972029 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dd006ae10 0x7f4dd00c4720 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.883+0000 7f4dd862c700 1 -- 172.21.15.89:0/1400972029 >> 172.21.15.89:0/1400972029 conn(0x7f4dd00cd670 msgr2=0x7f4dd00c2480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.884+0000 7f10117fa700 1 -- 172.21.15.89:0/1546816618 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f101c157380 con 0x7f101c0cb5a0 2023-10-10T20:34:41.892 INFO:teuthology.orchestra.run.smithi089.stdout:55834574854 2023-10-10T20:34:41.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.884+0000 7f5b1de17700 1 -- 172.21.15.89:0/29887383 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b04066f30 msgr2=0x7f5b040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.884+0000 7f5b1de17700 1 --2- 172.21.15.89:0/29887383 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b04066f30 0x7f5b040693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009bb0 tx=0x7f5b080091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.884+0000 7f5b1de17700 1 -- 172.21.15.89:0/29887383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b1806a730 msgr2=0x7f5b18143aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.884+0000 7f5b1de17700 1 --2- 172.21.15.89:0/29887383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b1806a730 0x7f5b18143aa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5b1815c8d0 tx=0x7f5b0c016000 comp rx=0 tx=0).stop 2023-10-10T20:34:41.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.885+0000 7f5b1de17700 1 -- 172.21.15.89:0/29887383 shutdown_connections 2023-10-10T20:34:41.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.885+0000 7f5b1de17700 1 --2- 172.21.15.89:0/29887383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b180c8fd0 0x7f5b18143fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.885+0000 7f5b1de17700 1 --2- 172.21.15.89:0/29887383 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b180656e0 0x7f5b1813b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.885+0000 7f5b1de17700 1 --2- 172.21.15.89:0/29887383 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b04066f30 0x7f5b040693e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.885+0000 7f5b1de17700 1 --2- 172.21.15.89:0/29887383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b1806a730 0x7f5b18143aa0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.885+0000 7f5b1de17700 1 -- 172.21.15.89:0/29887383 >> 172.21.15.89:0/29887383 conn(0x7f5b18068390 msgr2=0x7f5b180b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.887+0000 7f5b1de17700 1 -- 172.21.15.89:0/29887383 shutdown_connections 2023-10-10T20:34:41.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.887+0000 7f5b1de17700 1 -- 172.21.15.89:0/29887383 wait complete. 2023-10-10T20:34:41.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.887+0000 7f10238ee700 1 -- 172.21.15.89:0/1546816618 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1000066f30 msgr2=0x7f10000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.887+0000 7f10238ee700 1 --2- 172.21.15.89:0/1546816618 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1000066f30 0x7f10000693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f100c0099e0 tx=0x7f100c017000 comp rx=0 tx=0).stop 2023-10-10T20:34:41.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.887+0000 7f10238ee700 1 -- 172.21.15.89:0/1546816618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f101c0cb5a0 msgr2=0x7f101c0b05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.887+0000 7f10238ee700 1 --2- 172.21.15.89:0/1546816618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f101c0cb5a0 0x7f101c0b05c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f101400cd30 tx=0x7f1014001af0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.888+0000 7f10238ee700 1 -- 172.21.15.89:0/1546816618 shutdown_connections 2023-10-10T20:34:41.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.888+0000 7f10238ee700 1 --2- 172.21.15.89:0/1546816618 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f101c176c40 0x7f101c1770e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.888+0000 7f10238ee700 1 --2- 172.21.15.89:0/1546816618 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1000066f30 0x7f10000693e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.888+0000 7f10238ee700 1 --2- 172.21.15.89:0/1546816618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f101c0cb5a0 0x7f101c0b05c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.888+0000 7f10238ee700 1 --2- 172.21.15.89:0/1546816618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f101c156fb0 0x7f101c0b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.888+0000 7f10238ee700 1 -- 172.21.15.89:0/1546816618 >> 172.21.15.89:0/1546816618 conn(0x7f101c0c9870 msgr2=0x7f101c0c8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.888+0000 7f10238ee700 1 -- 172.21.15.89:0/1546816618 shutdown_connections 2023-10-10T20:34:41.899 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2023-10-10T20:34:41.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.888+0000 7f10238ee700 1 -- 172.21.15.89:0/1546816618 wait complete. 2023-10-10T20:34:41.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.884+0000 7f4dd862c700 1 -- 172.21.15.89:0/1400972029 shutdown_connections 2023-10-10T20:34:41.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.884+0000 7f4dd862c700 1 -- 172.21.15.89:0/1400972029 wait complete. 2023-10-10T20:34:41.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.893+0000 7f4a0ba0b700 1 -- 172.21.15.89:0/1748857423 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4a040af410 con 0x7f4a04068aa0 2023-10-10T20:34:41.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.893+0000 7f49f97fa700 1 -- 172.21.15.89:0/1748857423 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4a040af410 con 0x7f4a04068aa0 2023-10-10T20:34:41.902 INFO:teuthology.orchestra.run.smithi089.stdout:55834574854 2023-10-10T20:34:41.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.897+0000 7f4a0ba0b700 1 -- 172.21.15.89:0/1748857423 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f49e8066f30 msgr2=0x7f49e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.897+0000 7f4a0ba0b700 1 --2- 172.21.15.89:0/1748857423 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f49e8066f30 0x7f49e80693e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4a000058d0 tx=0x7f4a00007ac0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.897+0000 7f4a0ba0b700 1 -- 172.21.15.89:0/1748857423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a04068aa0 msgr2=0x7f4a04143ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:41.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.897+0000 7f4a0ba0b700 1 --2- 172.21.15.89:0/1748857423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a04068aa0 0x7f4a04143ea0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f49f40030c0 tx=0x7f49f4016000 comp rx=0 tx=0).stop 2023-10-10T20:34:41.904 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.0 2023-10-10T20:34:41.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.899+0000 7f4a0ba0b700 1 -- 172.21.15.89:0/1748857423 shutdown_connections 2023-10-10T20:34:41.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.899+0000 7f4a0ba0b700 1 --2- 172.21.15.89:0/1748857423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a041579c0 0x7f4a0414c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.899+0000 7f4a0ba0b700 1 --2- 172.21.15.89:0/1748857423 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f49e8066f30 0x7f49e80693e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.899+0000 7f4a0ba0b700 1 --2- 172.21.15.89:0/1748857423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a04068aa0 0x7f4a04143ea0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.899+0000 7f4a0ba0b700 1 --2- 172.21.15.89:0/1748857423 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4a041575f0 0x7f4a040b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:41.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.899+0000 7f4a0ba0b700 1 -- 172.21.15.89:0/1748857423 >> 172.21.15.89:0/1748857423 conn(0x7f4a040cd000 msgr2=0x7f4a04159e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:41.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.899+0000 7f4a0ba0b700 1 -- 172.21.15.89:0/1748857423 shutdown_connections 2023-10-10T20:34:41.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:41.899+0000 7f4a0ba0b700 1 -- 172.21.15.89:0/1748857423 wait complete. 2023-10-10T20:34:41.908 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2023-10-10T20:34:41.909 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2023-10-10T20:34:41.913 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.4 2023-10-10T20:34:41.915 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2023-10-10T20:34:41.919 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2023-10-10T20:34:42.878 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-10T20:34:42.901 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-10T20:34:42.905 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-10T20:34:42.908 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-10T20:34:42.910 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-10T20:34:42.914 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-10T20:34:42.916 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-10T20:34:42.921 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-10T20:34:43.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.051+0000 7fe0cc782700 1 Processor -- start 2023-10-10T20:34:43.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.051+0000 7fe0cc782700 1 -- start start 2023-10-10T20:34:43.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.053+0000 7fe0cc782700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c40691b0 0x7fe0c40ac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.053+0000 7fe0cc782700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe0c4156050 0x7fe0c41564d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.053+0000 7fe0cc782700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c40b2490 con 0x7fe0c40cba20 2023-10-10T20:34:43.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.053+0000 7fe0cc782700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c40aceb0 con 0x7fe0c4156050 2023-10-10T20:34:43.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.053+0000 7fe0cc782700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40ac4e0 con 0x7fe0c40691b0 2023-10-10T20:34:43.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.053+0000 7fe0c9d1d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c40691b0 0x7fe0c40ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.053+0000 7fe0c9d1d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c40691b0 0x7fe0c40ac970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:44040/0 (socket says 172.21.15.89:44040) 2023-10-10T20:34:43.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.053+0000 7fe0c9d1d700 1 -- 172.21.15.89:0/2141758145 learned_addr learned my addr 172.21.15.89:0/2141758145 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:43.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.053+0000 7fe0cad1f700 1 --2- 172.21.15.89:0/2141758145 >> v2:172.21.15.111:3300/0 conn(0x7fe0c4156050 0x7fe0c41564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.054+0000 7fe0c9d1d700 1 -- 172.21.15.89:0/2141758145 >> v2:172.21.15.111:3300/0 conn(0x7fe0c4156050 msgr2=0x7fe0c41564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:43.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.054+0000 7fe0c9d1d700 1 --2- 172.21.15.89:0/2141758145 >> v2:172.21.15.111:3300/0 conn(0x7fe0c4156050 0x7fe0c41564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.054+0000 7fe0c9d1d700 1 -- 172.21.15.89:0/2141758145 >> v1:172.21.15.111:6789/0 conn(0x7fe0c40cba20 legacy=0x7fe0c4068aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.054+0000 7fe0c9d1d700 1 -- 172.21.15.89:0/2141758145 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c4156a10 con 0x7fe0c40691b0 2023-10-10T20:34:43.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.054+0000 7fe0c9d1d700 1 --2- 172.21.15.89:0/2141758145 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c40691b0 0x7fe0c40ac970 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00d1a0 tx=0x7fe0bc007690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8920ce896d685d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.054+0000 7fe0c951c700 1 -- 172.21.15.89:0/2141758145 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc010070 con 0x7fe0c40691b0 2023-10-10T20:34:43.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.055+0000 7fe0c951c700 1 -- 172.21.15.89:0/2141758145 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0bc00dc00 con 0x7fe0c40691b0 2023-10-10T20:34:43.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.055+0000 7fe0c951c700 1 -- 172.21.15.89:0/2141758145 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc007b40 con 0x7fe0c40691b0 2023-10-10T20:34:43.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.056+0000 7fe0cc782700 1 -- 172.21.15.89:0/2141758145 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c40691b0 msgr2=0x7fe0c40ac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.056+0000 7fe0cc782700 1 --2- 172.21.15.89:0/2141758145 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c40691b0 0x7fe0c40ac970 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00d1a0 tx=0x7fe0bc007690 comp rx=0 tx=0).stop 2023-10-10T20:34:43.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.056+0000 7fe0cc782700 1 -- 172.21.15.89:0/2141758145 shutdown_connections 2023-10-10T20:34:43.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.056+0000 7fe0cc782700 1 --2- 172.21.15.89:0/2141758145 >> v2:172.21.15.111:3300/0 conn(0x7fe0c4156050 0x7fe0c41564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.056+0000 7fe0cc782700 1 --2- 172.21.15.89:0/2141758145 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c40691b0 0x7fe0c40ac970 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.056+0000 7fe0cc782700 1 -- 172.21.15.89:0/2141758145 >> 172.21.15.89:0/2141758145 conn(0x7fe0c40cd000 msgr2=0x7fe0c4068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:43.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.056+0000 7fe0cc782700 1 -- 172.21.15.89:0/2141758145 shutdown_connections 2023-10-10T20:34:43.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.056+0000 7fe0cc782700 1 -- 172.21.15.89:0/2141758145 wait complete. 2023-10-10T20:34:43.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.058+0000 7fe0cc782700 1 Processor -- start 2023-10-10T20:34:43.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.058+0000 7fe0cc782700 1 -- start start 2023-10-10T20:34:43.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.059+0000 7fe0cc782700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0c40691b0 0x7fe0c4154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.059+0000 7fe0cc782700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c40cba20 0x7fe0c415e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.059+0000 7fe0cc782700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0c4156050 0x7fe0c415ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.059+0000 7fe0cc782700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c415df40 con 0x7fe0c4156050 2023-10-10T20:34:43.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.059+0000 7fe0cc782700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c415d330 con 0x7fe0c40691b0 2023-10-10T20:34:43.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.059+0000 7fe0cc782700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c415d610 con 0x7fe0c40cba20 2023-10-10T20:34:43.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.060+0000 7fe0cad1f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0c4156050 0x7fe0c415ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.060+0000 7fe0cad1f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0c4156050 0x7fe0c415ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53470/0 (socket says 172.21.15.89:53470) 2023-10-10T20:34:43.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.060+0000 7fe0cad1f700 1 -- 172.21.15.89:0/3614064154 learned_addr learned my addr 172.21.15.89:0/3614064154 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:43.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.060+0000 7fe0cad1f700 1 -- 172.21.15.89:0/3614064154 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c40cba20 msgr2=0x7fe0c415e800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-10T20:34:43.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.060+0000 7fe0cad1f700 1 --2- 172.21.15.89:0/3614064154 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c40cba20 0x7fe0c415e800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.060+0000 7fe0cad1f700 1 -- 172.21.15.89:0/3614064154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0c40691b0 msgr2=0x7fe0c4154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.060+0000 7fe0cad1f700 1 --2- 172.21.15.89:0/3614064154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0c40691b0 0x7fe0c4154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.060+0000 7fe0cad1f700 1 -- 172.21.15.89:0/3614064154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0bc00be70 con 0x7fe0c4156050 2023-10-10T20:34:43.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.061+0000 7fe0cad1f700 1 --2- 172.21.15.89:0/3614064154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0c4156050 0x7fe0c415ed40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400e8d0 tx=0x7fe0b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.061+0000 7fe0ba7fc700 1 -- 172.21.15.89:0/3614064154 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b400ee70 con 0x7fe0c4156050 2023-10-10T20:34:43.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.061+0000 7fe0ba7fc700 1 -- 172.21.15.89:0/3614064154 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b400c850 con 0x7fe0c4156050 2023-10-10T20:34:43.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.061+0000 7fe0ba7fc700 1 -- 172.21.15.89:0/3614064154 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b400cc10 con 0x7fe0c4156050 2023-10-10T20:34:43.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.061+0000 7fe0cc782700 1 -- 172.21.15.89:0/3614064154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c415d8f0 con 0x7fe0c4156050 2023-10-10T20:34:43.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.061+0000 7fe0cc782700 1 -- 172.21.15.89:0/3614064154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c4155530 con 0x7fe0c4156050 2023-10-10T20:34:43.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.064+0000 7fe0ba7fc700 1 -- 172.21.15.89:0/3614064154 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe0b4013470 con 0x7fe0c4156050 2023-10-10T20:34:43.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.065+0000 7fe0ba7fc700 1 --2- 172.21.15.89:0/3614064154 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe0b00671a0 0x7fe0b0069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.065+0000 7fe0ba7fc700 1 -- 172.21.15.89:0/3614064154 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe0b4084780 con 0x7fe0c4156050 2023-10-10T20:34:43.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.066+0000 7fe0ca51e700 1 --2- 172.21.15.89:0/3614064154 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe0b00671a0 0x7fe0b0069650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.067+0000 7fe0ca51e700 1 --2- 172.21.15.89:0/3614064154 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe0b00671a0 0x7fe0b0069650 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0007fd0 tx=0x7fe0c0007b20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.067+0000 7fe0cc782700 1 -- 172.21.15.89:0/3614064154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0ac0052b0 con 0x7fe0c4156050 2023-10-10T20:34:43.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.076+0000 7fe0ba7fc700 1 -- 172.21.15.89:0/3614064154 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe0b40519d0 con 0x7fe0c4156050 2023-10-10T20:34:43.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.108+0000 7f8226ce9700 1 Processor -- start 2023-10-10T20:34:43.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.108+0000 7f8226ce9700 1 -- start start 2023-10-10T20:34:43.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.109+0000 7f8226ce9700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8220156f20 0x7f8220063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.109+0000 7f8226ce9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82201572f0 0x7f822006a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.109+0000 7f8226ce9700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8220069d30 con 0x7f822006ab20 2023-10-10T20:34:43.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.109+0000 7f8226ce9700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82200c8110 con 0x7f8220156f20 2023-10-10T20:34:43.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.109+0000 7f8226ce9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82201568e0 con 0x7f82201572f0 2023-10-10T20:34:43.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.110+0000 7f8225286700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f822006ab20 0x7f82200cb5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:49290/0 (socket says 172.21.15.89:49290) 2023-10-10T20:34:43.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.110+0000 7f8225286700 1 -- 172.21.15.89:0/1227320729 learned_addr learned my addr 172.21.15.89:0/1227320729 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:43.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.113+0000 7f821f7fe700 1 -- 172.21.15.89:0/1227320729 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054741049 0 0) 0x7f8220069d30 con 0x7f822006ab20 2023-10-10T20:34:43.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.113+0000 7f821f7fe700 1 -- 172.21.15.89:0/1227320729 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8210003570 con 0x7f822006ab20 2023-10-10T20:34:43.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.113+0000 7f821ffff700 1 --2- 172.21.15.89:0/1227320729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82201572f0 0x7f822006a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.113+0000 7f8224a85700 1 --2- 172.21.15.89:0/1227320729 >> v2:172.21.15.111:3300/0 conn(0x7f8220156f20 0x7f8220063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.113+0000 7f821f7fe700 1 -- 172.21.15.89:0/1227320729 <== mon.1 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 793612021 0 0) 0x7f8210003570 con 0x7f822006ab20 2023-10-10T20:34:43.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.113+0000 7f821f7fe700 1 -- 172.21.15.89:0/1227320729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82201572f0 msgr2=0x7f822006a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:43.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.113+0000 7f821f7fe700 1 --2- 172.21.15.89:0/1227320729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82201572f0 0x7f822006a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.113+0000 7f821f7fe700 1 -- 172.21.15.89:0/1227320729 >> v2:172.21.15.111:3300/0 conn(0x7f8220156f20 msgr2=0x7f8220063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:43.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.113+0000 7f821f7fe700 1 --2- 172.21.15.89:0/1227320729 >> v2:172.21.15.111:3300/0 conn(0x7f8220156f20 0x7f8220063230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.114+0000 7f821f7fe700 1 -- 172.21.15.89:0/1227320729 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82200c7e00 con 0x7f822006ab20 2023-10-10T20:34:43.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.114+0000 7f821f7fe700 1 -- 172.21.15.89:0/1227320729 <== mon.1 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f82180017f0 con 0x7f822006ab20 2023-10-10T20:34:43.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.114+0000 7f821f7fe700 1 -- 172.21.15.89:0/1227320729 >> v1:172.21.15.111:6789/0 conn(0x7f822006ab20 legacy=0x7f82200cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.114+0000 7f821f7fe700 1 --2- 172.21.15.89:0/1227320729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82201572f0 0x7f8210003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.114+0000 7f821f7fe700 1 --2- 172.21.15.89:0/1227320729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8210003fe0 0x7f8210004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.114+0000 7f821f7fe700 1 --2- 172.21.15.89:0/1227320729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8210004d50 0x7f82100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.114+0000 7f821f7fe700 1 -- 172.21.15.89:0/1227320729 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82200c8110 con 0x7f8210004d50 2023-10-10T20:34:43.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.114+0000 7f821f7fe700 1 -- 172.21.15.89:0/1227320729 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82201568e0 con 0x7f8210003fe0 2023-10-10T20:34:43.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.114+0000 7f821f7fe700 1 -- 172.21.15.89:0/1227320729 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821000ba60 con 0x7f82201572f0 2023-10-10T20:34:43.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.114+0000 7f821ffff700 1 --2- 172.21.15.89:0/1227320729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8210003fe0 0x7f8210004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.114+0000 7f821ffff700 1 -- 172.21.15.89:0/1227320729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82201572f0 msgr2=0x7f8210003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:43.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.114+0000 7f821ffff700 1 --2- 172.21.15.89:0/1227320729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82201572f0 0x7f8210003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.114+0000 7f821ffff700 1 -- 172.21.15.89:0/1227320729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8210004d50 msgr2=0x7f82100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:43.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.114+0000 7f821ffff700 1 --2- 172.21.15.89:0/1227320729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8210004d50 0x7f82100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.114+0000 7f821ffff700 1 -- 172.21.15.89:0/1227320729 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82200c7e00 con 0x7f8210003fe0 2023-10-10T20:34:43.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.114+0000 7f821ffff700 1 --2- 172.21.15.89:0/1227320729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8210003fe0 0x7f8210004810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8214009c00 tx=0x7f82140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3386d94ce62def69 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.115+0000 7f821f7fe700 1 -- 172.21.15.89:0/1227320729 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821400e710 con 0x7f8210003fe0 2023-10-10T20:34:43.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.115+0000 7f821f7fe700 1 -- 172.21.15.89:0/1227320729 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821400e870 con 0x7f8210003fe0 2023-10-10T20:34:43.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.115+0000 7f821f7fe700 1 -- 172.21.15.89:0/1227320729 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821400ec30 con 0x7f8210003fe0 2023-10-10T20:34:43.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.115+0000 7f8226ce9700 1 -- 172.21.15.89:0/1227320729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8210003fe0 msgr2=0x7f8210004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.115+0000 7f8226ce9700 1 --2- 172.21.15.89:0/1227320729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8210003fe0 0x7f8210004810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8214009c00 tx=0x7f82140091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.115+0000 7f8225286700 1 -- 172.21.15.89:0/1227320729 reap_dead start 2023-10-10T20:34:43.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.115+0000 7f8226ce9700 1 -- 172.21.15.89:0/1227320729 shutdown_connections 2023-10-10T20:34:43.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.115+0000 7f8226ce9700 1 -- 172.21.15.89:0/1227320729 >> 172.21.15.89:0/1227320729 conn(0x7f82200c9870 msgr2=0x7f82200c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:43.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.116+0000 7f8226ce9700 1 -- 172.21.15.89:0/1227320729 shutdown_connections 2023-10-10T20:34:43.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.116+0000 7f8226ce9700 1 -- 172.21.15.89:0/1227320729 wait complete. 2023-10-10T20:34:43.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.117+0000 7f7582fce700 1 Processor -- start 2023-10-10T20:34:43.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.117+0000 7f5d1b7e8700 1 Processor -- start 2023-10-10T20:34:43.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.118+0000 7f5d1b7e8700 1 -- start start 2023-10-10T20:34:43.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.118+0000 7f8226ce9700 1 Processor -- start 2023-10-10T20:34:43.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.118+0000 7f7582fce700 1 -- start start 2023-10-10T20:34:43.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.118+0000 7fab7a495700 1 Processor -- start 2023-10-10T20:34:43.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.118+0000 7f00995d4700 1 Processor -- start 2023-10-10T20:34:43.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.118+0000 7fab7a495700 1 -- start start 2023-10-10T20:34:43.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f7582fce700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f757c06b160 0x7f757c06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f5d1b7e8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5d140b1f40 0x7f5d140ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f5d1b7e8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5d14159fb0 0x7f5d140aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f5d1b7e8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d140accf0 con 0x7f5d14068aa0 2023-10-10T20:34:43.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f5d1b7e8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d140ab110 con 0x7f5d140b1f40 2023-10-10T20:34:43.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f7582fce700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f757c06a2b0 0x7f757c06a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f7582fce700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757c0ac450 con 0x7f757c0637d0 2023-10-10T20:34:43.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f7582fce700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757c0693d0 con 0x7f757c06b160 2023-10-10T20:34:43.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f7582fce700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757c069d00 con 0x7f757c06a2b0 2023-10-10T20:34:43.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f5d1b7e8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1415a440 con 0x7f5d14159fb0 2023-10-10T20:34:43.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f5d19584700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5d140b1f40 0x7f5d140ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f5d19d85700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5d14068aa0 0x7f5d1415d960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35742/0 (socket says 172.21.15.89:35742) 2023-10-10T20:34:43.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f5d19d85700 1 -- 172.21.15.89:0/1932689549 learned_addr learned my addr 172.21.15.89:0/1932689549 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:43.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f757bfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f757c0637d0 0x7f757c068cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35738/0 (socket says 172.21.15.89:35738) 2023-10-10T20:34:43.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f757bfff700 1 -- 172.21.15.89:0/792571343 learned_addr learned my addr 172.21.15.89:0/792571343 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:43.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f7580d6a700 1 --2- 172.21.15.89:0/792571343 >> v2:172.21.15.89:3300/0 conn(0x7f757c06b160 0x7f757c06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7fab7a495700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fab740c91e0 0x7fab740cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7fab7a495700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fab7406ae10 0x7fab7406b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7fab7a495700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab740cc210 con 0x7fab7406aa40 2023-10-10T20:34:43.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7fab7a495700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab740cc4f0 con 0x7fab7406ae10 2023-10-10T20:34:43.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f5d18d83700 1 --2- 172.21.15.89:0/1932689549 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5d14159fb0 0x7f5d140aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.118+0000 7f8226ce9700 1 -- start start 2023-10-10T20:34:43.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f8226ce9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8210003fe0 0x7f8220141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f8226ce9700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82201401f0 0x7f8220152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f8226ce9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8220152570 0x7f8220150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f8226ce9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82201529d0 con 0x7f8220152570 2023-10-10T20:34:43.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f8226ce9700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8220140a40 con 0x7f82201401f0 2023-10-10T20:34:43.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f8226ce9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8220140d20 con 0x7f8210003fe0 2023-10-10T20:34:43.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7fab7a495700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab740cc7d0 con 0x7fab740c91e0 2023-10-10T20:34:43.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.118+0000 7f00995d4700 1 -- start start 2023-10-10T20:34:43.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f00995d4700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f00940cb5a0 0x7f00940cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f00995d4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0094069d30 con 0x7f00940c85a0 2023-10-10T20:34:43.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f00995d4700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00940c8110 con 0x7f0094156fb0 2023-10-10T20:34:43.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.119+0000 7f00995d4700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00940cbf30 con 0x7f00940cb5a0 2023-10-10T20:34:43.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f00927fc700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f00940cb5a0 0x7f00940cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7fab737fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fab740c91e0 0x7fab740cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7fab737fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fab740c91e0 0x7fab740cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:43112/0 (socket says 172.21.15.89:43112) 2023-10-10T20:34:43.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7fab737fe700 1 -- 172.21.15.89:0/2558358831 learned_addr learned my addr 172.21.15.89:0/2558358831 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:43.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f00927fc700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f00940cb5a0 0x7f00940cb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:43130/0 (socket says 172.21.15.89:43130) 2023-10-10T20:34:43.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f00927fc700 1 -- 172.21.15.89:0/2140810363 learned_addr learned my addr 172.21.15.89:0/2140810363 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:43.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f5d0bfff700 1 -- 172.21.15.89:0/1932689549 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053958149 0 0) 0x7f5d140accf0 con 0x7f5d14068aa0 2023-10-10T20:34:43.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f5d0bfff700 1 -- 172.21.15.89:0/1932689549 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d00003570 con 0x7f5d14068aa0 2023-10-10T20:34:43.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f5d18d83700 1 -- 172.21.15.89:0/1932689549 >> v2:172.21.15.89:3300/0 conn(0x7f5d140b1f40 msgr2=0x7f5d140ac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:43.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f5d18d83700 1 --2- 172.21.15.89:0/1932689549 >> v2:172.21.15.89:3300/0 conn(0x7f5d140b1f40 0x7f5d140ac7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f5d18d83700 1 -- 172.21.15.89:0/1932689549 >> v1:172.21.15.89:6789/0 conn(0x7f5d14068aa0 legacy=0x7f5d1415d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f5d18d83700 1 -- 172.21.15.89:0/1932689549 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1415a720 con 0x7f5d14159fb0 2023-10-10T20:34:43.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7fab737fe700 1 -- 172.21.15.89:0/2558358831 >> v2:172.21.15.89:3300/0 conn(0x7fab7406ae10 msgr2=0x7fab7406b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:43.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7fab737fe700 1 --2- 172.21.15.89:0/2558358831 >> v2:172.21.15.89:3300/0 conn(0x7fab7406ae10 0x7fab7406b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7fab737fe700 1 -- 172.21.15.89:0/2558358831 >> v1:172.21.15.89:6789/0 conn(0x7fab7406aa40 legacy=0x7fab7406bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:43.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f8224a85700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8210003fe0 0x7f8220141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f821ffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82201401f0 0x7f8220152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f821ffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82201401f0 0x7f8220152030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:43128/0 (socket says 172.21.15.89:43128) 2023-10-10T20:34:43.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f821ffff700 1 -- 172.21.15.89:0/2350767569 learned_addr learned my addr 172.21.15.89:0/2350767569 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:43.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7fab737fe700 1 -- 172.21.15.89:0/2558358831 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab740ccab0 con 0x7fab740c91e0 2023-10-10T20:34:43.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f5d18d83700 1 --2- 172.21.15.89:0/1932689549 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5d14159fb0 0x7f5d140aabd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5d040098e0 tx=0x7f5d040091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cc2b18811209a86 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f757b7fe700 1 -- 172.21.15.89:0/792571343 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 990710564 0 0) 0x7f757c0ac450 con 0x7f757c0637d0 2023-10-10T20:34:43.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f757b7fe700 1 -- 172.21.15.89:0/792571343 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f756c003570 con 0x7f757c0637d0 2023-10-10T20:34:43.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7fab737fe700 1 --2- 172.21.15.89:0/2558358831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab740c91e0 0x7fab740cbc00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fab680099b0 tx=0x7fab68009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd94d96ab84c0db5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f8225286700 1 --2- 172.21.15.89:0/2350767569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8220152570 0x7f8220150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f757b7fe700 1 -- 172.21.15.89:0/792571343 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4067310330 0 0) 0x7f756c003570 con 0x7f757c0637d0 2023-10-10T20:34:43.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f757b7fe700 1 -- 172.21.15.89:0/792571343 >> v2:172.21.15.111:3300/0 conn(0x7f757c06a2b0 msgr2=0x7f757c06a6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-10T20:34:43.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f757b7fe700 1 --2- 172.21.15.89:0/792571343 >> v2:172.21.15.111:3300/0 conn(0x7f757c06a2b0 0x7f757c06a6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f757b7fe700 1 -- 172.21.15.89:0/792571343 >> v2:172.21.15.89:3300/0 conn(0x7f757c06b160 msgr2=0x7f757c06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:43.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f757b7fe700 1 --2- 172.21.15.89:0/792571343 >> v2:172.21.15.89:3300/0 conn(0x7f757c06b160 0x7f757c06b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f757b7fe700 1 -- 172.21.15.89:0/792571343 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c068830 con 0x7f757c0637d0 2023-10-10T20:34:43.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f757b7fe700 1 -- 172.21.15.89:0/792571343 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f7568001470 con 0x7f757c0637d0 2023-10-10T20:34:43.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f757b7fe700 1 -- 172.21.15.89:0/792571343 >> v1:172.21.15.89:6789/0 conn(0x7f757c0637d0 legacy=0x7f757c068cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f757b7fe700 1 --2- 172.21.15.89:0/792571343 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f756c003a60 0x7f756c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f757b7fe700 1 --2- 172.21.15.89:0/792571343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f756c004810 0x7f756c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f757b7fe700 1 --2- 172.21.15.89:0/792571343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f756c005210 0x7f756c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f757b7fe700 1 -- 172.21.15.89:0/792571343 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c0693d0 con 0x7f756c005210 2023-10-10T20:34:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f757b7fe700 1 -- 172.21.15.89:0/792571343 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c069d00 con 0x7f756c004810 2023-10-10T20:34:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f757b7fe700 1 -- 172.21.15.89:0/792571343 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756c00c230 con 0x7f756c003a60 2023-10-10T20:34:43.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f0091ffb700 1 -- 172.21.15.89:0/2140810363 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3743311623 0 0) 0x7f00940c8110 con 0x7f0094156fb0 2023-10-10T20:34:43.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f0091ffb700 1 -- 172.21.15.89:0/2140810363 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f007c004680 con 0x7f0094156fb0 2023-10-10T20:34:43.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.118+0000 7f376f9d4700 1 Processor -- start 2023-10-10T20:34:43.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.120+0000 7f376f9d4700 1 -- start start 2023-10-10T20:34:43.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f0091ffb700 1 -- 172.21.15.89:0/2140810363 <== mon.1 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1414060555 0 0) 0x7f007c004680 con 0x7f0094156fb0 2023-10-10T20:34:43.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f0091ffb700 1 -- 172.21.15.89:0/2140810363 >> v2:172.21.15.111:3300/0 conn(0x7f00940cb5a0 msgr2=0x7f00940cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:43.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f0091ffb700 1 --2- 172.21.15.89:0/2140810363 >> v2:172.21.15.111:3300/0 conn(0x7f00940cb5a0 0x7f00940cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f0091ffb700 1 -- 172.21.15.89:0/2140810363 >> v1:172.21.15.89:6789/0 conn(0x7f00940c85a0 legacy=0x7f00940c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f7580d6a700 1 --2- 172.21.15.89:0/792571343 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f756c003a60 0x7f756c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f0091ffb700 1 -- 172.21.15.89:0/2140810363 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00941568e0 con 0x7f0094156fb0 2023-10-10T20:34:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f0091ffb700 1 -- 172.21.15.89:0/2140810363 <== mon.1 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f0088001470 con 0x7f0094156fb0 2023-10-10T20:34:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f0091ffb700 1 -- 172.21.15.89:0/2140810363 >> v1:172.21.15.111:6789/0 conn(0x7f0094156fb0 legacy=0x7f009406a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f0091ffb700 1 --2- 172.21.15.89:0/2140810363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f007c004b70 0x7f007c004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f0091ffb700 1 --2- 172.21.15.89:0/2140810363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f007c005920 0x7f007c005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f5d0bfff700 1 -- 172.21.15.89:0/1932689549 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d04011630 con 0x7f5d14159fb0 2023-10-10T20:34:43.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f5d0bfff700 1 -- 172.21.15.89:0/1932689549 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d04011790 con 0x7f5d14159fb0 2023-10-10T20:34:43.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f5d0bfff700 1 -- 172.21.15.89:0/1932689549 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d04011b30 con 0x7f5d14159fb0 2023-10-10T20:34:43.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f0091ffb700 1 --2- 172.21.15.89:0/2140810363 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f007c0062d0 0x7f007c003490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f0091ffb700 1 -- 172.21.15.89:0/2140810363 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00940cbf30 con 0x7f007c004b70 2023-10-10T20:34:43.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f0091ffb700 1 -- 172.21.15.89:0/2140810363 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c003b00 con 0x7f007c005920 2023-10-10T20:34:43.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f0091ffb700 1 -- 172.21.15.89:0/2140810363 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007c003cc0 con 0x7f007c0062d0 2023-10-10T20:34:43.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f821ffff700 1 -- 172.21.15.89:0/2350767569 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8210003fe0 msgr2=0x7f8220141ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f821ffff700 1 --2- 172.21.15.89:0/2350767569 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8210003fe0 0x7f8220141ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f821ffff700 1 -- 172.21.15.89:0/2350767569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8220152570 msgr2=0x7f8220150b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f821ffff700 1 --2- 172.21.15.89:0/2350767569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8220152570 0x7f8220150b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f821ffff700 1 -- 172.21.15.89:0/2350767569 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8220141000 con 0x7f82201401f0 2023-10-10T20:34:43.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f821ffff700 1 --2- 172.21.15.89:0/2350767569 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82201401f0 0x7f8220152030 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8210004760 tx=0x7f821401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f376f9d4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37680b1c50 0x7f37680ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f376f9d4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37680acdf0 con 0x7f37680691b0 2023-10-10T20:34:43.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f376f9d4700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37680ac4c0 con 0x7f37680875d0 2023-10-10T20:34:43.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f376f9d4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37680ac7a0 con 0x7f37680b1c50 2023-10-10T20:34:43.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f00927fc700 1 --2- 172.21.15.89:0/2140810363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f007c005920 0x7f007c005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.121+0000 7f757bfff700 1 --2- 172.21.15.89:0/792571343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f756c004810 0x7f756c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f00937fe700 1 --2- 172.21.15.89:0/2140810363 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f007c0062d0 0x7f007c003490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f376d770700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37680b1c50 0x7f37680ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f376cf6f700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f37680875d0 0x7f3768068aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:49318/0 (socket says 172.21.15.89:49318) 2023-10-10T20:34:43.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f376cf6f700 1 -- 172.21.15.89:0/1030881477 learned_addr learned my addr 172.21.15.89:0/1030881477 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:43.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f0092ffd700 1 --2- 172.21.15.89:0/2140810363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f007c004b70 0x7f007c004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f7580d6a700 1 -- 172.21.15.89:0/792571343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f756c004810 msgr2=0x7f756c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:43.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f7580d6a700 1 --2- 172.21.15.89:0/792571343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f756c004810 0x7f756c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f7580d6a700 1 -- 172.21.15.89:0/792571343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f756c005210 msgr2=0x7f756c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:43.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f7580d6a700 1 --2- 172.21.15.89:0/792571343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f756c005210 0x7f756c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f7580d6a700 1 -- 172.21.15.89:0/792571343 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c068830 con 0x7f756c003a60 2023-10-10T20:34:43.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f375ffff700 1 -- 172.21.15.89:0/1030881477 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170642341 0 0) 0x7f37680ac4c0 con 0x7f37680875d0 2023-10-10T20:34:43.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f375ffff700 1 -- 172.21.15.89:0/1030881477 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3754003570 con 0x7f37680875d0 2023-10-10T20:34:43.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f376d770700 1 -- 172.21.15.89:0/1030881477 >> v1:172.21.15.111:6789/0 conn(0x7f37680875d0 legacy=0x7f3768068aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f376d770700 1 -- 172.21.15.89:0/1030881477 >> v1:172.21.15.89:6789/0 conn(0x7f37680691b0 legacy=0x7f37680ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:43.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f376d770700 1 -- 172.21.15.89:0/1030881477 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37680aca80 con 0x7f37680b1c50 2023-10-10T20:34:43.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f00937fe700 1 -- 172.21.15.89:0/2140810363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f007c005920 msgr2=0x7f007c005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:43.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f00937fe700 1 --2- 172.21.15.89:0/2140810363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f007c005920 0x7f007c005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f00937fe700 1 -- 172.21.15.89:0/2140810363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f007c004b70 msgr2=0x7f007c004f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:43.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f00937fe700 1 --2- 172.21.15.89:0/2140810363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f007c004b70 0x7f007c004f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f00937fe700 1 -- 172.21.15.89:0/2140810363 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00941568e0 con 0x7f007c0062d0 2023-10-10T20:34:43.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f00937fe700 1 -- 172.21.15.89:0/2140810363 reap_dead start 2023-10-10T20:34:43.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f0092ffd700 1 --2- 172.21.15.89:0/2140810363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f007c004b70 0x7f007c004f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:43.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f758156b700 1 -- 172.21.15.89:0/792571343 reap_dead start 2023-10-10T20:34:43.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f821cff9700 1 -- 172.21.15.89:0/2350767569 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214022070 con 0x7f82201401f0 2023-10-10T20:34:43.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f821cff9700 1 -- 172.21.15.89:0/2350767569 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8214000bc0 con 0x7f82201401f0 2023-10-10T20:34:43.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f821cff9700 1 -- 172.21.15.89:0/2350767569 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821400ec10 con 0x7f82201401f0 2023-10-10T20:34:43.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f7580d6a700 1 --2- 172.21.15.89:0/792571343 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f756c003a60 0x7f756c003e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7570009c00 tx=0x7f75700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3022e547c59fa72 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f376d770700 1 --2- 172.21.15.89:0/1030881477 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37680b1c50 0x7f37680ab8c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3764009910 tx=0x7f37640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5059bfcd274a712 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7fab72ffd700 1 -- 172.21.15.89:0/2558358831 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6800b7e0 con 0x7fab740c91e0 2023-10-10T20:34:43.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7fab72ffd700 1 -- 172.21.15.89:0/2558358831 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6800b940 con 0x7fab740c91e0 2023-10-10T20:34:43.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7fab72ffd700 1 -- 172.21.15.89:0/2558358831 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6800bce0 con 0x7fab740c91e0 2023-10-10T20:34:43.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f5d1b7e8700 1 -- 172.21.15.89:0/1932689549 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5d14159fb0 msgr2=0x7f5d140aabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f5d1b7e8700 1 --2- 172.21.15.89:0/1932689549 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5d14159fb0 0x7f5d140aabd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5d040098e0 tx=0x7f5d040091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f5d1b7e8700 1 -- 172.21.15.89:0/1932689549 shutdown_connections 2023-10-10T20:34:43.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f5d1b7e8700 1 --2- 172.21.15.89:0/1932689549 >> v2:172.21.15.89:3300/0 conn(0x7f5d140b1f40 0x7f5d140ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f5d1b7e8700 1 --2- 172.21.15.89:0/1932689549 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5d14159fb0 0x7f5d140aabd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f5d1b7e8700 1 -- 172.21.15.89:0/1932689549 >> 172.21.15.89:0/1932689549 conn(0x7f5d140cd000 msgr2=0x7f5d14068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:43.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f8226ce9700 1 -- 172.21.15.89:0/2350767569 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82201412e0 con 0x7f82201401f0 2023-10-10T20:34:43.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f8226ce9700 1 -- 172.21.15.89:0/2350767569 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8220140660 con 0x7f82201401f0 2023-10-10T20:34:43.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7f5d1b7e8700 1 -- 172.21.15.89:0/1932689549 shutdown_connections 2023-10-10T20:34:43.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.122+0000 7f375ffff700 1 -- 172.21.15.89:0/1030881477 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376400b640 con 0x7f37680b1c50 2023-10-10T20:34:43.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7fab7a495700 1 -- 172.21.15.89:0/2558358831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab740c91e0 msgr2=0x7fab740cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7fab7a495700 1 --2- 172.21.15.89:0/2558358831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab740c91e0 0x7fab740cbc00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fab680099b0 tx=0x7fab68009250 comp rx=0 tx=0).stop 2023-10-10T20:34:43.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7f375ffff700 1 -- 172.21.15.89:0/1030881477 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376400b7a0 con 0x7f37680b1c50 2023-10-10T20:34:43.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7f375ffff700 1 -- 172.21.15.89:0/1030881477 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376400bb20 con 0x7f37680b1c50 2023-10-10T20:34:43.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7fab7a495700 1 -- 172.21.15.89:0/2558358831 shutdown_connections 2023-10-10T20:34:43.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7fab7a495700 1 --2- 172.21.15.89:0/2558358831 >> v2:172.21.15.89:3300/0 conn(0x7fab7406ae10 0x7fab7406b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7fab7a495700 1 --2- 172.21.15.89:0/2558358831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab740c91e0 0x7fab740cbc00 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fab680099b0 tx=0x7fab68009250 comp rx=0 tx=0).stop 2023-10-10T20:34:43.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7fab7a495700 1 -- 172.21.15.89:0/2558358831 >> 172.21.15.89:0/2558358831 conn(0x7fab740cd670 msgr2=0x7fab740cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:43.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7f5d1b7e8700 1 -- 172.21.15.89:0/1932689549 wait complete. 2023-10-10T20:34:43.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7f00937fe700 1 --2- 172.21.15.89:0/2140810363 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f007c0062d0 0x7f007c003490 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f00840018b0 tx=0x7f0084007d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d555356b44fadee4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7fab7a495700 1 -- 172.21.15.89:0/2558358831 shutdown_connections 2023-10-10T20:34:43.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7f0091ffb700 1 -- 172.21.15.89:0/2140810363 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084012420 con 0x7f007c0062d0 2023-10-10T20:34:43.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7f0091ffb700 1 -- 172.21.15.89:0/2140810363 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00840125a0 con 0x7f007c0062d0 2023-10-10T20:34:43.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7f0091ffb700 1 -- 172.21.15.89:0/2140810363 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084012960 con 0x7f007c0062d0 2023-10-10T20:34:43.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7f376f9d4700 1 -- 172.21.15.89:0/1030881477 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37680b1c50 msgr2=0x7f37680ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7f376f9d4700 1 --2- 172.21.15.89:0/1030881477 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37680b1c50 0x7f37680ab8c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3764009910 tx=0x7f37640091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7f757b7fe700 1 -- 172.21.15.89:0/792571343 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757000e710 con 0x7f756c003a60 2023-10-10T20:34:43.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7f757b7fe700 1 -- 172.21.15.89:0/792571343 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757000e870 con 0x7f756c003a60 2023-10-10T20:34:43.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7f757b7fe700 1 -- 172.21.15.89:0/792571343 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757000ec30 con 0x7f756c003a60 2023-10-10T20:34:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7fab7a495700 1 -- 172.21.15.89:0/2558358831 wait complete. 2023-10-10T20:34:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.123+0000 7f376f9d4700 1 -- 172.21.15.89:0/1030881477 shutdown_connections 2023-10-10T20:34:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.124+0000 7f376f9d4700 1 --2- 172.21.15.89:0/1030881477 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37680b1c50 0x7f37680ab8c0 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3764009910 tx=0x7f37640091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.124+0000 7f376f9d4700 1 -- 172.21.15.89:0/1030881477 >> 172.21.15.89:0/1030881477 conn(0x7f37680cd000 msgr2=0x7f3768068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:43.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.124+0000 7f8226ce9700 1 -- 172.21.15.89:0/2350767569 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8220141800 con 0x7f82201401f0 2023-10-10T20:34:43.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.124+0000 7f7582fce700 1 -- 172.21.15.89:0/792571343 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f756c003a60 msgr2=0x7f756c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.124+0000 7f7582fce700 1 --2- 172.21.15.89:0/792571343 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f756c003a60 0x7f756c003e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7570009c00 tx=0x7f75700091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.124+0000 7f5d1b7e8700 1 Processor -- start 2023-10-10T20:34:43.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.124+0000 7f5d1b7e8700 1 -- start start 2023-10-10T20:34:43.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.124+0000 7fab7a495700 1 Processor -- start 2023-10-10T20:34:43.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.124+0000 7f376f9d4700 1 -- 172.21.15.89:0/1030881477 shutdown_connections 2023-10-10T20:34:43.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.125+0000 7f7582fce700 1 -- 172.21.15.89:0/792571343 shutdown_connections 2023-10-10T20:34:43.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.125+0000 7f7582fce700 1 --2- 172.21.15.89:0/792571343 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f756c003a60 0x7f756c003e70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.125+0000 7f7582fce700 1 -- 172.21.15.89:0/792571343 >> 172.21.15.89:0/792571343 conn(0x7f757c06ded0 msgr2=0x7f757c06e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:43.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.125+0000 7f7582fce700 1 -- 172.21.15.89:0/792571343 shutdown_connections 2023-10-10T20:34:43.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.125+0000 7f00995d4700 1 -- 172.21.15.89:0/2140810363 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f007c0062d0 msgr2=0x7f007c003490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.125+0000 7f00995d4700 1 --2- 172.21.15.89:0/2140810363 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f007c0062d0 0x7f007c003490 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f00840018b0 tx=0x7f0084007d30 comp rx=0 tx=0).stop 2023-10-10T20:34:43.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.125+0000 7f00995d4700 1 -- 172.21.15.89:0/2140810363 shutdown_connections 2023-10-10T20:34:43.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.125+0000 7f00995d4700 1 --2- 172.21.15.89:0/2140810363 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f007c0062d0 0x7f007c003490 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.125+0000 7f00995d4700 1 -- 172.21.15.89:0/2140810363 >> 172.21.15.89:0/2140810363 conn(0x7f00940c9870 msgr2=0x7f00940c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:43.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.125+0000 7f7582fce700 1 -- 172.21.15.89:0/792571343 wait complete. 2023-10-10T20:34:43.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.125+0000 7fab7a495700 1 -- start start 2023-10-10T20:34:43.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.125+0000 7f00995d4700 1 -- 172.21.15.89:0/2140810363 shutdown_connections 2023-10-10T20:34:43.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7f5d1b7e8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d14068aa0 0x7f5d140a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7f5d1b7e8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d140b1f40 0x7f5d140a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7f5d1b7e8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5d14159fb0 0x7f5d14149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7f5d1b7e8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d140a5ca0 con 0x7f5d140b1f40 2023-10-10T20:34:43.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7f5d1b7e8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d140a8300 con 0x7f5d14068aa0 2023-10-10T20:34:43.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7f5d1b7e8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d140a85e0 con 0x7f5d14159fb0 2023-10-10T20:34:43.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7f5d19584700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d14068aa0 0x7f5d140a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.125+0000 7f821cff9700 1 -- 172.21.15.89:0/2350767569 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8214026070 con 0x7f82201401f0 2023-10-10T20:34:43.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7f821cff9700 1 --2- 172.21.15.89:0/2350767569 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8204066fb0 0x7f8204069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7f821cff9700 1 -- 172.21.15.89:0/2350767569 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8214085d40 con 0x7f82201401f0 2023-10-10T20:34:43.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7fab7a495700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab7406aa40 0x7fab74144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7fab7a495700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab7406ae10 0x7fab74145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7fab7a495700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab741490d0 0x7fab74160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7fab7a495700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab741446f0 con 0x7fab7406aa40 2023-10-10T20:34:43.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7fab7a495700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab741449d0 con 0x7fab7406ae10 2023-10-10T20:34:43.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7fab7a495700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab74144cb0 con 0x7fab741490d0 2023-10-10T20:34:43.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7fab73fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab7406aa40 0x7fab74144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7fab73fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab7406aa40 0x7fab74144110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53560/0 (socket says 172.21.15.89:53560) 2023-10-10T20:34:43.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7fab73fff700 1 -- 172.21.15.89:0/2598002212 learned_addr learned my addr 172.21.15.89:0/2598002212 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:43.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7fab737fe700 1 --2- 172.21.15.89:0/2598002212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab7406ae10 0x7fab74145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7fab78a32700 1 --2- 172.21.15.89:0/2598002212 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab741490d0 0x7fab74160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7fab73fff700 1 -- 172.21.15.89:0/2598002212 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab741490d0 msgr2=0x7fab74160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7fab73fff700 1 --2- 172.21.15.89:0/2598002212 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab741490d0 0x7fab74160a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7fab73fff700 1 -- 172.21.15.89:0/2598002212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab7406ae10 msgr2=0x7fab74145790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7fab73fff700 1 --2- 172.21.15.89:0/2598002212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab7406ae10 0x7fab74145790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7fab73fff700 1 -- 172.21.15.89:0/2598002212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab680094f0 con 0x7fab7406aa40 2023-10-10T20:34:43.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.125+0000 7f376f9d4700 1 -- 172.21.15.89:0/1030881477 wait complete. 2023-10-10T20:34:43.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7f376f9d4700 1 Processor -- start 2023-10-10T20:34:43.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7f5d18d83700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d140b1f40 0x7f5d140a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7f5d18d83700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d140b1f40 0x7f5d140a5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53548/0 (socket says 172.21.15.89:53548) 2023-10-10T20:34:43.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.126+0000 7f5d18d83700 1 -- 172.21.15.89:0/1807078565 learned_addr learned my addr 172.21.15.89:0/1807078565 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:43.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7f5d19d85700 1 --2- 172.21.15.89:0/1807078565 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5d14159fb0 0x7f5d14149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7f5d19584700 1 -- 172.21.15.89:0/1807078565 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5d14159fb0 msgr2=0x7f5d14149120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7f5d19584700 1 --2- 172.21.15.89:0/1807078565 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5d14159fb0 0x7f5d14149120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7f5d19584700 1 -- 172.21.15.89:0/1807078565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d140b1f40 msgr2=0x7f5d140a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7f5d19584700 1 --2- 172.21.15.89:0/1807078565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d140b1f40 0x7f5d140a5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7f5d19584700 1 -- 172.21.15.89:0/1807078565 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d04009420 con 0x7f5d14068aa0 2023-10-10T20:34:43.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.125+0000 7f00995d4700 1 -- 172.21.15.89:0/2140810363 wait complete. 2023-10-10T20:34:43.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7f00995d4700 1 Processor -- start 2023-10-10T20:34:43.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7f8224a85700 1 --2- 172.21.15.89:0/2350767569 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8204066fb0 0x7f8204069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7f7582fce700 1 Processor -- start 2023-10-10T20:34:43.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7fab73fff700 1 --2- 172.21.15.89:0/2598002212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab7406aa40 0x7fab74144110 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fab6400c5b0 tx=0x7fab64007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7f5d19584700 1 --2- 172.21.15.89:0/1807078565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d14068aa0 0x7f5d140a7bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000ea50 tx=0x7f5d100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7f8224a85700 1 --2- 172.21.15.89:0/2350767569 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8204066fb0 0x7f8204069460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f820c009dc0 tx=0x7f820c0091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7f00995d4700 1 -- start start 2023-10-10T20:34:43.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f376f9d4700 1 -- start start 2023-10-10T20:34:43.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.127+0000 7f7582fce700 1 -- start start 2023-10-10T20:34:43.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f7582fce700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f756c003a60 0x7f757c15aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f7582fce700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f757c159290 0x7f757c0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f7582fce700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f757c1502f0 0x7f757c150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f7582fce700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c159a70 con 0x7f756c003a60 2023-10-10T20:34:43.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f7582fce700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c159d50 con 0x7f757c1502f0 2023-10-10T20:34:43.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f7582fce700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757c15a030 con 0x7f757c159290 2023-10-10T20:34:43.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7fab5bfff700 1 -- 172.21.15.89:0/2598002212 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab64011590 con 0x7fab7406aa40 2023-10-10T20:34:43.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f757bfff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f757c159290 0x7f757c0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f7580d6a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f756c003a60 0x7f757c15aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f7580d6a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f756c003a60 0x7f757c15aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53576/0 (socket says 172.21.15.89:53576) 2023-10-10T20:34:43.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f7580d6a700 1 -- 172.21.15.89:0/2602833038 learned_addr learned my addr 172.21.15.89:0/2602833038 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:43.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f00995d4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f007c0062d0 0x7f009414c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f00995d4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f009414d490 0x7f00940c4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f00995d4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00940c4eb0 0x7f00940b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f00995d4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00940b1d10 con 0x7f007c0062d0 2023-10-10T20:34:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f00995d4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00940b1100 con 0x7f00940c4eb0 2023-10-10T20:34:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f00995d4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00940b13e0 con 0x7f009414d490 2023-10-10T20:34:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f0092ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f007c0062d0 0x7f009414c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f0092ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f007c0062d0 0x7f009414c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53590/0 (socket says 172.21.15.89:53590) 2023-10-10T20:34:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f0092ffd700 1 -- 172.21.15.89:0/3300742107 learned_addr learned my addr 172.21.15.89:0/3300742107 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f00927fc700 1 --2- 172.21.15.89:0/3300742107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f009414d490 0x7f00940c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7f00937fe700 1 --2- 172.21.15.89:0/3300742107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00940c4eb0 0x7f00940b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f376f9d4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37680691b0 0x7f3768086360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f376f9d4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37680875d0 0x7f3768086e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f376f9d4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37680a54c0 0x7f376815c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f376f9d4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376815a060 con 0x7f37680875d0 2023-10-10T20:34:43.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f376f9d4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37680a5d10 con 0x7f37680a54c0 2023-10-10T20:34:43.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f376f9d4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37680a5ff0 con 0x7f37680691b0 2023-10-10T20:34:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f376d770700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37680691b0 0x7f3768086360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f376df71700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37680a54c0 0x7f376815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.130+0000 7f376df71700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37680a54c0 0x7f376815c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:43180/0 (socket says 172.21.15.89:43180) 2023-10-10T20:34:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.130+0000 7f376df71700 1 -- 172.21.15.89:0/5078704 learned_addr learned my addr 172.21.15.89:0/5078704 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f5d097fa700 1 -- 172.21.15.89:0/1807078565 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1000ee90 con 0x7f5d14068aa0 2023-10-10T20:34:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f5d097fa700 1 -- 172.21.15.89:0/1807078565 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1000c7f0 con 0x7f5d14068aa0 2023-10-10T20:34:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f5d097fa700 1 -- 172.21.15.89:0/1807078565 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1000cb90 con 0x7f5d14068aa0 2023-10-10T20:34:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f5d1b7e8700 1 -- 172.21.15.89:0/1807078565 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d140a88c0 con 0x7f5d14068aa0 2023-10-10T20:34:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f5d1b7e8700 1 -- 172.21.15.89:0/1807078565 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d140a8e80 con 0x7f5d14068aa0 2023-10-10T20:34:43.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f0092ffd700 1 -- 172.21.15.89:0/3300742107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f009414d490 msgr2=0x7f00940c4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f0092ffd700 1 --2- 172.21.15.89:0/3300742107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f009414d490 0x7f00940c4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f0092ffd700 1 -- 172.21.15.89:0/3300742107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00940c4eb0 msgr2=0x7f00940b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f0092ffd700 1 --2- 172.21.15.89:0/3300742107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00940c4eb0 0x7f00940b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f0092ffd700 1 -- 172.21.15.89:0/3300742107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00940b16c0 con 0x7f007c0062d0 2023-10-10T20:34:43.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f0092ffd700 1 --2- 172.21.15.89:0/3300742107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f007c0062d0 0x7f009414c980 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f008800ff00 tx=0x7f0088002e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f821cff9700 1 -- 172.21.15.89:0/2350767569 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8214053090 con 0x7f82201401f0 2023-10-10T20:34:43.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f7580d6a700 1 -- 172.21.15.89:0/2602833038 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f757c159290 msgr2=0x7f757c0a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f7580d6a700 1 --2- 172.21.15.89:0/2602833038 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f757c159290 0x7f757c0a5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f7580d6a700 1 -- 172.21.15.89:0/2602833038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f757c1502f0 msgr2=0x7f757c150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f7580d6a700 1 --2- 172.21.15.89:0/2602833038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f757c1502f0 0x7f757c150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f7580d6a700 1 -- 172.21.15.89:0/2602833038 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c0a7750 con 0x7f756c003a60 2023-10-10T20:34:43.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.129+0000 7f7580d6a700 1 --2- 172.21.15.89:0/2602833038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f756c003a60 0x7f757c15aa70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f756c00c3c0 tx=0x7f757001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.130+0000 7f376d770700 1 -- 172.21.15.89:0/5078704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37680a54c0 msgr2=0x7f376815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.130+0000 7f376d770700 1 --2- 172.21.15.89:0/5078704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37680a54c0 0x7f376815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.130+0000 7f376d770700 1 -- 172.21.15.89:0/5078704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37680875d0 msgr2=0x7f3768086e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.130+0000 7f376d770700 1 --2- 172.21.15.89:0/5078704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37680875d0 0x7f3768086e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.130+0000 7f376d770700 1 -- 172.21.15.89:0/5078704 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3764009420 con 0x7f37680691b0 2023-10-10T20:34:43.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.130+0000 7f376df71700 1 --2- 172.21.15.89:0/5078704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37680a54c0 0x7f376815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:43.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.130+0000 7f376d770700 1 --2- 172.21.15.89:0/5078704 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37680691b0 0x7f3768086360 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3764015000 tx=0x7f3764016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7fab5bfff700 1 -- 172.21.15.89:0/2598002212 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab64011710 con 0x7fab7406aa40 2023-10-10T20:34:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7fab5bfff700 1 -- 172.21.15.89:0/2598002212 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab64011ad0 con 0x7fab7406aa40 2023-10-10T20:34:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7fab7a495700 1 -- 172.21.15.89:0/2598002212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab74144f90 con 0x7fab7406aa40 2023-10-10T20:34:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.128+0000 7fab7a495700 1 -- 172.21.15.89:0/2598002212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab74160f60 con 0x7fab7406aa40 2023-10-10T20:34:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.130+0000 7fab5bfff700 1 -- 172.21.15.89:0/2598002212 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fab6401c460 con 0x7fab7406aa40 2023-10-10T20:34:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.131+0000 7fab5bfff700 1 --2- 172.21.15.89:0/2598002212 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab60067000 0x7fab600694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.131+0000 7fab5bfff700 1 -- 172.21.15.89:0/2598002212 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fab64084240 con 0x7fab7406aa40 2023-10-10T20:34:43.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.130+0000 7f375d7fa700 1 -- 172.21.15.89:0/5078704 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3764000c40 con 0x7f37680691b0 2023-10-10T20:34:43.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.130+0000 7f375d7fa700 1 -- 172.21.15.89:0/5078704 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3764020070 con 0x7f37680691b0 2023-10-10T20:34:43.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.130+0000 7f375d7fa700 1 -- 172.21.15.89:0/5078704 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376401b450 con 0x7f37680691b0 2023-10-10T20:34:43.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.130+0000 7f376f9d4700 1 -- 172.21.15.89:0/5078704 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37680a62d0 con 0x7f37680691b0 2023-10-10T20:34:43.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.131+0000 7f376f9d4700 1 -- 172.21.15.89:0/5078704 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376815c6d0 con 0x7f37680691b0 2023-10-10T20:34:43.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.131+0000 7f7578ff9700 1 -- 172.21.15.89:0/2602833038 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570022070 con 0x7f756c003a60 2023-10-10T20:34:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.131+0000 7f7578ff9700 1 -- 172.21.15.89:0/2602833038 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757000ebf0 con 0x7f756c003a60 2023-10-10T20:34:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.131+0000 7f7578ff9700 1 -- 172.21.15.89:0/2602833038 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757001c6a0 con 0x7f756c003a60 2023-10-10T20:34:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.131+0000 7f7582fce700 1 -- 172.21.15.89:0/2602833038 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757c0a7a30 con 0x7f756c003a60 2023-10-10T20:34:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.131+0000 7f7582fce700 1 -- 172.21.15.89:0/2602833038 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757c15b0c0 con 0x7f756c003a60 2023-10-10T20:34:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.132+0000 7f7582fce700 1 -- 172.21.15.89:0/2602833038 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757c0a7c20 con 0x7f756c003a60 2023-10-10T20:34:43.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.133+0000 7f5d1b7e8700 1 -- 172.21.15.89:0/1807078565 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf40052b0 con 0x7f5d14068aa0 2023-10-10T20:34:43.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.133+0000 7f5d097fa700 1 -- 172.21.15.89:0/1807078565 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5d10013470 con 0x7f5d14068aa0 2023-10-10T20:34:43.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.134+0000 7f5d097fa700 1 --2- 172.21.15.89:0/1807078565 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5cf8067030 0x7f5cf80694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.134+0000 7f5d097fa700 1 -- 172.21.15.89:0/1807078565 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5d100845c0 con 0x7f5d14068aa0 2023-10-10T20:34:43.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.134+0000 7f5d18d83700 1 --2- 172.21.15.89:0/1807078565 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5cf8067030 0x7f5cf80694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.134+0000 7f375d7fa700 1 -- 172.21.15.89:0/5078704 <== mon.2 v2:172.21.15.111:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3764022030 con 0x7f37680691b0 2023-10-10T20:34:43.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.134+0000 7f375d7fa700 1 --2- 172.21.15.89:0/5078704 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f374c066f30 0x7f374c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.134+0000 7f375d7fa700 1 -- 172.21.15.89:0/5078704 <== mon.2 v2:172.21.15.111:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3764085dc0 con 0x7f37680691b0 2023-10-10T20:34:43.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.131+0000 7f007affd700 1 -- 172.21.15.89:0/3300742107 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008800db30 con 0x7f007c0062d0 2023-10-10T20:34:43.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.131+0000 7f007affd700 1 -- 172.21.15.89:0/3300742107 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008800dc90 con 0x7f007c0062d0 2023-10-10T20:34:43.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.131+0000 7f007affd700 1 -- 172.21.15.89:0/3300742107 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00880146a0 con 0x7f007c0062d0 2023-10-10T20:34:43.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.131+0000 7f00995d4700 1 -- 172.21.15.89:0/3300742107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00940b19a0 con 0x7f007c0062d0 2023-10-10T20:34:43.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.131+0000 7f00995d4700 1 -- 172.21.15.89:0/3300742107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009414cec0 con 0x7f007c0062d0 2023-10-10T20:34:43.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.134+0000 7f007affd700 1 -- 172.21.15.89:0/3300742107 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0088014a50 con 0x7f007c0062d0 2023-10-10T20:34:43.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.135+0000 7f5d18d83700 1 --2- 172.21.15.89:0/1807078565 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5cf8067030 0x7f5cf80694e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5d140c90d0 tx=0x7f5d04017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.135+0000 7fab737fe700 1 --2- 172.21.15.89:0/2598002212 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab60067000 0x7fab600694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.135+0000 7f007affd700 1 --2- 172.21.15.89:0/3300742107 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0080066fb0 0x7f0080069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.135+0000 7f007affd700 1 -- 172.21.15.89:0/3300742107 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0088084f30 con 0x7f007c0062d0 2023-10-10T20:34:43.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.135+0000 7fab737fe700 1 --2- 172.21.15.89:0/2598002212 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab60067000 0x7fab600694b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fab680099b0 tx=0x7fab68016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.137+0000 7f00927fc700 1 --2- 172.21.15.89:0/3300742107 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0080066fb0 0x7f0080069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.137+0000 7f00995d4700 1 -- 172.21.15.89:0/3300742107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00740052b0 con 0x7f007c0062d0 2023-10-10T20:34:43.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.137+0000 7f00927fc700 1 --2- 172.21.15.89:0/3300742107 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0080066fb0 0x7f0080069460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f00940c7630 tx=0x7f008c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.138+0000 7fab7a495700 1 -- 172.21.15.89:0/2598002212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab540052b0 con 0x7fab7406aa40 2023-10-10T20:34:43.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.138+0000 7f376f9d4700 1 -- 172.21.15.89:0/5078704 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37480052b0 con 0x7f37680691b0 2023-10-10T20:34:43.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.139+0000 7f7578ff9700 1 -- 172.21.15.89:0/2602833038 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f757001ca50 con 0x7f756c003a60 2023-10-10T20:34:43.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.140+0000 7f7578ff9700 1 --2- 172.21.15.89:0/2602833038 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7560066f30 0x7f75600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.140+0000 7f7578ff9700 1 -- 172.21.15.89:0/2602833038 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7570086660 con 0x7f756c003a60 2023-10-10T20:34:43.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.140+0000 7f757bfff700 1 --2- 172.21.15.89:0/2602833038 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7560066f30 0x7f75600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.140+0000 7f7578ff9700 1 -- 172.21.15.89:0/2602833038 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7570086b90 con 0x7f756c003a60 2023-10-10T20:34:43.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.140+0000 7f757bfff700 1 --2- 172.21.15.89:0/2602833038 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7560066f30 0x7f75600693e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f75680010e0 tx=0x7f7568001e60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.142+0000 7f5d097fa700 1 -- 172.21.15.89:0/1807078565 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d10051890 con 0x7f5d14068aa0 2023-10-10T20:34:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.144+0000 7f376cf6f700 1 --2- 172.21.15.89:0/5078704 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f374c066f30 0x7f374c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.144+0000 7f376cf6f700 1 --2- 172.21.15.89:0/5078704 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f374c066f30 0x7f374c0693e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f37580058d0 tx=0x7f37580079d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.146+0000 7f007affd700 1 -- 172.21.15.89:0/3300742107 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0088055b90 con 0x7f007c0062d0 2023-10-10T20:34:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.147+0000 7f375d7fa700 1 -- 172.21.15.89:0/5078704 <== mon.2 v2:172.21.15.111:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3764052f90 con 0x7f37680691b0 2023-10-10T20:34:43.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.154+0000 7fab5bfff700 1 -- 172.21.15.89:0/2598002212 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab64051490 con 0x7fab7406aa40 2023-10-10T20:34:43.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.160+0000 7f31ad05e700 1 Processor -- start 2023-10-10T20:34:43.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.160+0000 7f31ad05e700 1 -- start start 2023-10-10T20:34:43.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.161+0000 7f31ad05e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31a80cba20 0x7f31a80ac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.161+0000 7f31ad05e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f31a8156760 0x7f31a8068f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.161+0000 7f31ad05e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a8069450 con 0x7f31a8068aa0 2023-10-10T20:34:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.161+0000 7f31ad05e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a80aceb0 con 0x7f31a8156760 2023-10-10T20:34:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.161+0000 7f31ad05e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80b2490 con 0x7f31a80cba20 2023-10-10T20:34:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.166+0000 7f31a6d9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31a80cba20 0x7f31a80ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.166+0000 7f31a6d9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31a80cba20 0x7f31a80ac970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:44150/0 (socket says 172.21.15.89:44150) 2023-10-10T20:34:43.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.166+0000 7f31a6d9d700 1 -- 172.21.15.89:0/1037041246 learned_addr learned my addr 172.21.15.89:0/1037041246 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:43.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.166+0000 7f31a6d9d700 1 -- 172.21.15.89:0/1037041246 >> v2:172.21.15.89:3300/0 conn(0x7f31a8156760 msgr2=0x7f31a8068f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:43.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.166+0000 7f31a6d9d700 1 --2- 172.21.15.89:0/1037041246 >> v2:172.21.15.89:3300/0 conn(0x7f31a8156760 0x7f31a8068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.166+0000 7f31a6d9d700 1 -- 172.21.15.89:0/1037041246 >> v1:172.21.15.89:6789/0 conn(0x7f31a8068aa0 legacy=0x7f31a8156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:43.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.166+0000 7f31a6d9d700 1 -- 172.21.15.89:0/1037041246 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a80ac4e0 con 0x7f31a80cba20 2023-10-10T20:34:43.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.166+0000 7f31a6d9d700 1 --2- 172.21.15.89:0/1037041246 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31a80cba20 0x7f31a80ac970 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f31980099b0 tx=0x7f3198009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=478cf2efcf1017b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.170+0000 7f31a5d9b700 1 -- 172.21.15.89:0/1037041246 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319800b820 con 0x7f31a80cba20 2023-10-10T20:34:43.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.171+0000 7f31a5d9b700 1 -- 172.21.15.89:0/1037041246 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319800b980 con 0x7f31a80cba20 2023-10-10T20:34:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.171+0000 7f31a5d9b700 1 -- 172.21.15.89:0/1037041246 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319800bd20 con 0x7f31a80cba20 2023-10-10T20:34:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.172+0000 7f31ad05e700 1 -- 172.21.15.89:0/1037041246 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31a80cba20 msgr2=0x7f31a80ac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.172+0000 7f31ad05e700 1 --2- 172.21.15.89:0/1037041246 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31a80cba20 0x7f31a80ac970 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f31980099b0 tx=0x7f3198009250 comp rx=0 tx=0).stop 2023-10-10T20:34:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.172+0000 7f31ad05e700 1 -- 172.21.15.89:0/1037041246 shutdown_connections 2023-10-10T20:34:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.172+0000 7f31ad05e700 1 --2- 172.21.15.89:0/1037041246 >> v2:172.21.15.89:3300/0 conn(0x7f31a8156760 0x7f31a8068f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.172+0000 7f31ad05e700 1 --2- 172.21.15.89:0/1037041246 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31a80cba20 0x7f31a80ac970 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.172+0000 7f31ad05e700 1 -- 172.21.15.89:0/1037041246 >> 172.21.15.89:0/1037041246 conn(0x7f31a80cd000 msgr2=0x7f31a8068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.172+0000 7f31ad05e700 1 -- 172.21.15.89:0/1037041246 shutdown_connections 2023-10-10T20:34:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.172+0000 7f31ad05e700 1 -- 172.21.15.89:0/1037041246 wait complete. 2023-10-10T20:34:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.174+0000 7f31ad05e700 1 Processor -- start 2023-10-10T20:34:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.174+0000 7f31ad05e700 1 -- start start 2023-10-10T20:34:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.175+0000 7f31ad05e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a8068aa0 0x7f31a8153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.175+0000 7f31ad05e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31a80cba20 0x7f31a816c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.175+0000 7f31ad05e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31a8156760 0x7f31a816cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.176+0000 7f31ad05e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80a63b0 con 0x7f31a8156760 2023-10-10T20:34:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.176+0000 7f31ad05e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80a6690 con 0x7f31a8068aa0 2023-10-10T20:34:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.176+0000 7f31ad05e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80a6970 con 0x7f31a80cba20 2023-10-10T20:34:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.176+0000 7f31a759e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31a8156760 0x7f31a816cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.176+0000 7f31a759e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31a8156760 0x7f31a816cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53620/0 (socket says 172.21.15.89:53620) 2023-10-10T20:34:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.176+0000 7f31a759e700 1 -- 172.21.15.89:0/4179467752 learned_addr learned my addr 172.21.15.89:0/4179467752 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.177+0000 7f31a659c700 1 --2- 172.21.15.89:0/4179467752 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31a80cba20 0x7f31a816c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.177+0000 7f31a759e700 1 -- 172.21.15.89:0/4179467752 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31a80cba20 msgr2=0x7f31a816c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.177+0000 7f31a759e700 1 --2- 172.21.15.89:0/4179467752 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31a80cba20 0x7f31a816c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.177+0000 7f31a759e700 1 -- 172.21.15.89:0/4179467752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a8068aa0 msgr2=0x7f31a8153160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-10T20:34:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.177+0000 7f31a759e700 1 --2- 172.21.15.89:0/4179467752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a8068aa0 0x7f31a8153160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.177+0000 7f31a759e700 1 -- 172.21.15.89:0/4179467752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31980094f0 con 0x7f31a8156760 2023-10-10T20:34:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.177+0000 7f31a759e700 1 --2- 172.21.15.89:0/4179467752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31a8156760 0x7f31a816cc40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f31a000bc30 tx=0x7f31a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.177+0000 7f3196ffd700 1 -- 172.21.15.89:0/4179467752 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a00092e0 con 0x7f31a8156760 2023-10-10T20:34:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.178+0000 7f3196ffd700 1 -- 172.21.15.89:0/4179467752 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a000c7a0 con 0x7f31a8156760 2023-10-10T20:34:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.178+0000 7f3196ffd700 1 -- 172.21.15.89:0/4179467752 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a000cb80 con 0x7f31a8156760 2023-10-10T20:34:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.178+0000 7f31ad05e700 1 -- 172.21.15.89:0/4179467752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a80a7f40 con 0x7f31a8156760 2023-10-10T20:34:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.178+0000 7f31ad05e700 1 -- 172.21.15.89:0/4179467752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a81536a0 con 0x7f31a8156760 2023-10-10T20:34:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.181+0000 7f3196ffd700 1 -- 172.21.15.89:0/4179467752 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f31a0012070 con 0x7f31a8156760 2023-10-10T20:34:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.182+0000 7f3196ffd700 1 --2- 172.21.15.89:0/4179467752 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31900671f0 0x7f31900696a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.182+0000 7f3196ffd700 1 -- 172.21.15.89:0/4179467752 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f31a00841e0 con 0x7f31a8156760 2023-10-10T20:34:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.183+0000 7f31ad05e700 1 -- 172.21.15.89:0/4179467752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a80a8460 con 0x7f31a8156760 2023-10-10T20:34:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.191+0000 7f31a6d9d700 1 --2- 172.21.15.89:0/4179467752 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31900671f0 0x7f31900696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.192+0000 7f3196ffd700 1 -- 172.21.15.89:0/4179467752 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31a80a8460 con 0x7f31a8156760 2023-10-10T20:34:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.203+0000 7f31a6d9d700 1 --2- 172.21.15.89:0/4179467752 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31900671f0 0x7f31900696a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f31980099b0 tx=0x7f3198016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.462+0000 7fe0cc782700 1 -- 172.21.15.89:0/3614064154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe0ac005560 con 0x7fe0c4156050 2023-10-10T20:34:43.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.462+0000 7fe0ba7fc700 1 -- 172.21.15.89:0/3614064154 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe0b4021090 con 0x7fe0c4156050 2023-10-10T20:34:43.467 INFO:teuthology.orchestra.run.smithi089.stdout:55834574855 2023-10-10T20:34:43.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.465+0000 7fe0cc782700 1 -- 172.21.15.89:0/3614064154 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe0b00671a0 msgr2=0x7fe0b0069650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.465+0000 7fe0cc782700 1 --2- 172.21.15.89:0/3614064154 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe0b00671a0 0x7fe0b0069650 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0007fd0 tx=0x7fe0c0007b20 comp rx=0 tx=0).stop 2023-10-10T20:34:43.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.465+0000 7fe0cc782700 1 -- 172.21.15.89:0/3614064154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0c4156050 msgr2=0x7fe0c415ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.465+0000 7fe0cc782700 1 --2- 172.21.15.89:0/3614064154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0c4156050 0x7fe0c415ed40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400e8d0 tx=0x7fe0b40091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.466+0000 7fe0cc782700 1 -- 172.21.15.89:0/3614064154 shutdown_connections 2023-10-10T20:34:43.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.466+0000 7fe0cc782700 1 --2- 172.21.15.89:0/3614064154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0c4156050 0x7fe0c415ed40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.466+0000 7fe0cc782700 1 --2- 172.21.15.89:0/3614064154 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe0b00671a0 0x7fe0b0069650 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.466+0000 7fe0cc782700 1 --2- 172.21.15.89:0/3614064154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0c40691b0 0x7fe0c4154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.466+0000 7fe0cc782700 1 --2- 172.21.15.89:0/3614064154 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0c40cba20 0x7fe0c415e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.466+0000 7fe0cc782700 1 -- 172.21.15.89:0/3614064154 >> 172.21.15.89:0/3614064154 conn(0x7fe0c40cd000 msgr2=0x7fe0c4156ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:43.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.466+0000 7fe0cc782700 1 -- 172.21.15.89:0/3614064154 shutdown_connections 2023-10-10T20:34:43.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.466+0000 7fe0cc782700 1 -- 172.21.15.89:0/3614064154 wait complete. 2023-10-10T20:34:43.487 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-10-10T20:34:43.488 DEBUG:teuthology.parallel:result is None 2023-10-10T20:34:43.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.512+0000 7f8226ce9700 1 -- 172.21.15.89:0/2350767569 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8220141800 con 0x7f82201401f0 2023-10-10T20:34:43.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.512+0000 7f821cff9700 1 -- 172.21.15.89:0/2350767569 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f821402d020 con 0x7f82201401f0 2023-10-10T20:34:43.515 INFO:teuthology.orchestra.run.smithi089.stdout:55834574853 2023-10-10T20:34:43.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.514+0000 7f7582fce700 1 -- 172.21.15.89:0/2602833038 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f757c04c3c0 con 0x7f756c003a60 2023-10-10T20:34:43.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.514+0000 7f7578ff9700 1 -- 172.21.15.89:0/2602833038 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7570053a00 con 0x7f756c003a60 2023-10-10T20:34:43.516 INFO:teuthology.orchestra.run.smithi089.stdout:55834574855 2023-10-10T20:34:43.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.517+0000 7f7582fce700 1 -- 172.21.15.89:0/2602833038 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7560066f30 msgr2=0x7f75600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.517+0000 7f7582fce700 1 --2- 172.21.15.89:0/2602833038 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7560066f30 0x7f75600693e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f75680010e0 tx=0x7f7568001e60 comp rx=0 tx=0).stop 2023-10-10T20:34:43.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.517+0000 7f7582fce700 1 -- 172.21.15.89:0/2602833038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f756c003a60 msgr2=0x7f757c15aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.517+0000 7f7582fce700 1 --2- 172.21.15.89:0/2602833038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f756c003a60 0x7f757c15aa70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f756c00c3c0 tx=0x7f757001a000 comp rx=0 tx=0).stop 2023-10-10T20:34:43.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.517+0000 7f7582fce700 1 -- 172.21.15.89:0/2602833038 shutdown_connections 2023-10-10T20:34:43.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.517+0000 7f7582fce700 1 --2- 172.21.15.89:0/2602833038 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7560066f30 0x7f75600693e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.517+0000 7f7582fce700 1 --2- 172.21.15.89:0/2602833038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f757c1502f0 0x7f757c150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.517+0000 7f7582fce700 1 --2- 172.21.15.89:0/2602833038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f756c003a60 0x7f757c15aa70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.517+0000 7f7582fce700 1 --2- 172.21.15.89:0/2602833038 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f757c159290 0x7f757c0a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.517+0000 7f7582fce700 1 -- 172.21.15.89:0/2602833038 >> 172.21.15.89:0/2602833038 conn(0x7f757c06ded0 msgr2=0x7f757c0cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:43.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.517+0000 7f7582fce700 1 -- 172.21.15.89:0/2602833038 shutdown_connections 2023-10-10T20:34:43.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.517+0000 7f7582fce700 1 -- 172.21.15.89:0/2602833038 wait complete. 2023-10-10T20:34:43.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.520+0000 7f8226ce9700 1 -- 172.21.15.89:0/2350767569 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8204066fb0 msgr2=0x7f8204069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.520+0000 7f8226ce9700 1 --2- 172.21.15.89:0/2350767569 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8204066fb0 0x7f8204069460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f820c009dc0 tx=0x7f820c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.520+0000 7f8226ce9700 1 -- 172.21.15.89:0/2350767569 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82201401f0 msgr2=0x7f8220152030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.520+0000 7f8226ce9700 1 --2- 172.21.15.89:0/2350767569 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82201401f0 0x7f8220152030 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8210004760 tx=0x7f821401a000 comp rx=0 tx=0).stop 2023-10-10T20:34:43.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.522+0000 7f5d1b7e8700 1 -- 172.21.15.89:0/1807078565 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5cf4005830 con 0x7f5d14068aa0 2023-10-10T20:34:43.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.523+0000 7f8226ce9700 1 -- 172.21.15.89:0/2350767569 shutdown_connections 2023-10-10T20:34:43.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.523+0000 7f8226ce9700 1 --2- 172.21.15.89:0/2350767569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8220152570 0x7f8220150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.523+0000 7f8226ce9700 1 --2- 172.21.15.89:0/2350767569 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8210003fe0 0x7f8220141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.523+0000 7f8226ce9700 1 --2- 172.21.15.89:0/2350767569 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8204066fb0 0x7f8204069460 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.523+0000 7f8226ce9700 1 --2- 172.21.15.89:0/2350767569 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82201401f0 0x7f8220152030 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.523+0000 7f8226ce9700 1 -- 172.21.15.89:0/2350767569 >> 172.21.15.89:0/2350767569 conn(0x7f82200c9870 msgr2=0x7f82200c35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:43.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.523+0000 7f8226ce9700 1 -- 172.21.15.89:0/2350767569 shutdown_connections 2023-10-10T20:34:43.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.523+0000 7f8226ce9700 1 -- 172.21.15.89:0/2350767569 wait complete. 2023-10-10T20:34:43.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.524+0000 7f5d097fa700 1 -- 172.21.15.89:0/1807078565 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5d10054eb0 con 0x7f5d14068aa0 2023-10-10T20:34:43.528 INFO:teuthology.orchestra.run.smithi089.stdout:55834574855 2023-10-10T20:34:43.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.529+0000 7f5d1b7e8700 1 -- 172.21.15.89:0/1807078565 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5cf8067030 msgr2=0x7f5cf80694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.529+0000 7f5d1b7e8700 1 --2- 172.21.15.89:0/1807078565 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5cf8067030 0x7f5cf80694e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5d140c90d0 tx=0x7f5d04017000 comp rx=0 tx=0).stop 2023-10-10T20:34:43.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.529+0000 7f5d1b7e8700 1 -- 172.21.15.89:0/1807078565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d14068aa0 msgr2=0x7f5d140a7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.529+0000 7f5d1b7e8700 1 --2- 172.21.15.89:0/1807078565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d14068aa0 0x7f5d140a7bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000ea50 tx=0x7f5d100091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.529+0000 7f5d1b7e8700 1 -- 172.21.15.89:0/1807078565 shutdown_connections 2023-10-10T20:34:43.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.529+0000 7f5d1b7e8700 1 --2- 172.21.15.89:0/1807078565 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5d14159fb0 0x7f5d14149120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.529+0000 7f5d1b7e8700 1 --2- 172.21.15.89:0/1807078565 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5cf8067030 0x7f5cf80694e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.529+0000 7f5d1b7e8700 1 --2- 172.21.15.89:0/1807078565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d14068aa0 0x7f5d140a7bf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.529+0000 7f5d1b7e8700 1 --2- 172.21.15.89:0/1807078565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d140b1f40 0x7f5d140a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.529+0000 7f5d1b7e8700 1 -- 172.21.15.89:0/1807078565 >> 172.21.15.89:0/1807078565 conn(0x7f5d140cd000 msgr2=0x7f5d14068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.532+0000 7f00995d4700 1 -- 172.21.15.89:0/3300742107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0074005560 con 0x7f007c0062d0 2023-10-10T20:34:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.532+0000 7f007affd700 1 -- 172.21.15.89:0/3300742107 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f00880079b0 con 0x7f007c0062d0 2023-10-10T20:34:43.536 INFO:teuthology.orchestra.run.smithi089.stdout:55834574855 2023-10-10T20:34:43.537 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2023-10-10T20:34:43.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.534+0000 7f5d1b7e8700 1 -- 172.21.15.89:0/1807078565 shutdown_connections 2023-10-10T20:34:43.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.534+0000 7f5d1b7e8700 1 -- 172.21.15.89:0/1807078565 wait complete. 2023-10-10T20:34:43.537 DEBUG:teuthology.parallel:result is None 2023-10-10T20:34:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.536+0000 7f00995d4700 1 -- 172.21.15.89:0/3300742107 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0080066fb0 msgr2=0x7f0080069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.536+0000 7f00995d4700 1 --2- 172.21.15.89:0/3300742107 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0080066fb0 0x7f0080069460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f00940c7630 tx=0x7f008c00b040 comp rx=0 tx=0).stop 2023-10-10T20:34:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.536+0000 7f00995d4700 1 -- 172.21.15.89:0/3300742107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f007c0062d0 msgr2=0x7f009414c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.536+0000 7f00995d4700 1 --2- 172.21.15.89:0/3300742107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f007c0062d0 0x7f009414c980 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f008800ff00 tx=0x7f0088002e60 comp rx=0 tx=0).stop 2023-10-10T20:34:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.537+0000 7f00995d4700 1 -- 172.21.15.89:0/3300742107 shutdown_connections 2023-10-10T20:34:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.537+0000 7f00995d4700 1 --2- 172.21.15.89:0/3300742107 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0080066fb0 0x7f0080069460 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.537+0000 7f00995d4700 1 --2- 172.21.15.89:0/3300742107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00940c4eb0 0x7f00940b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.537+0000 7f00995d4700 1 --2- 172.21.15.89:0/3300742107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f007c0062d0 0x7f009414c980 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.537+0000 7f00995d4700 1 --2- 172.21.15.89:0/3300742107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f009414d490 0x7f00940c4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.537+0000 7f00995d4700 1 -- 172.21.15.89:0/3300742107 >> 172.21.15.89:0/3300742107 conn(0x7f00940c9870 msgr2=0x7f00940c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:43.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.537+0000 7f00995d4700 1 -- 172.21.15.89:0/3300742107 shutdown_connections 2023-10-10T20:34:43.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.537+0000 7f00995d4700 1 -- 172.21.15.89:0/3300742107 wait complete. 2023-10-10T20:34:43.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.538+0000 7fab7a495700 1 -- 172.21.15.89:0/2598002212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fab54005560 con 0x7fab7406aa40 2023-10-10T20:34:43.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2023-10-10T20:34:43.543 DEBUG:teuthology.parallel:result is None 2023-10-10T20:34:43.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.542+0000 7fab5bfff700 1 -- 172.21.15.89:0/2598002212 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fab64054ab0 con 0x7fab7406aa40 2023-10-10T20:34:43.545 INFO:teuthology.orchestra.run.smithi089.stdout:55834574854 2023-10-10T20:34:43.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.544+0000 7f376f9d4700 1 -- 172.21.15.89:0/5078704 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3748005560 con 0x7f37680691b0 2023-10-10T20:34:43.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.544+0000 7fab7a495700 1 -- 172.21.15.89:0/2598002212 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab60067000 msgr2=0x7fab600694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.544+0000 7fab7a495700 1 --2- 172.21.15.89:0/2598002212 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab60067000 0x7fab600694b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fab680099b0 tx=0x7fab68016000 comp rx=0 tx=0).stop 2023-10-10T20:34:43.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.544+0000 7fab7a495700 1 -- 172.21.15.89:0/2598002212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab7406aa40 msgr2=0x7fab74144110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.544+0000 7fab7a495700 1 --2- 172.21.15.89:0/2598002212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab7406aa40 0x7fab74144110 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fab6400c5b0 tx=0x7fab64007f60 comp rx=0 tx=0).stop 2023-10-10T20:34:43.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.544+0000 7fab7a495700 1 -- 172.21.15.89:0/2598002212 shutdown_connections 2023-10-10T20:34:43.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.544+0000 7fab7a495700 1 --2- 172.21.15.89:0/2598002212 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab741490d0 0x7fab74160a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.544+0000 7fab7a495700 1 --2- 172.21.15.89:0/2598002212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab7406aa40 0x7fab74144110 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.544+0000 7fab7a495700 1 --2- 172.21.15.89:0/2598002212 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab60067000 0x7fab600694b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.544+0000 7fab7a495700 1 --2- 172.21.15.89:0/2598002212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab7406ae10 0x7fab74145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.544+0000 7fab7a495700 1 -- 172.21.15.89:0/2598002212 >> 172.21.15.89:0/2598002212 conn(0x7fab740cd670 msgr2=0x7fab7406b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:43.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.544+0000 7fab7a495700 1 -- 172.21.15.89:0/2598002212 shutdown_connections 2023-10-10T20:34:43.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.544+0000 7fab7a495700 1 -- 172.21.15.89:0/2598002212 wait complete. 2023-10-10T20:34:43.552 INFO:teuthology.orchestra.run.smithi089.stdout:55834574853 2023-10-10T20:34:43.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.544+0000 7f375d7fa700 1 -- 172.21.15.89:0/5078704 <== mon.2 v2:172.21.15.111:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f376402b090 con 0x7f37680691b0 2023-10-10T20:34:43.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.547+0000 7f376f9d4700 1 -- 172.21.15.89:0/5078704 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f374c066f30 msgr2=0x7f374c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.547+0000 7f376f9d4700 1 --2- 172.21.15.89:0/5078704 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f374c066f30 0x7f374c0693e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f37580058d0 tx=0x7f37580079d0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.547+0000 7f376f9d4700 1 -- 172.21.15.89:0/5078704 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37680691b0 msgr2=0x7f3768086360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.547+0000 7f376f9d4700 1 --2- 172.21.15.89:0/5078704 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37680691b0 0x7f3768086360 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3764015000 tx=0x7f3764016000 comp rx=0 tx=0).stop 2023-10-10T20:34:43.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.550+0000 7f376f9d4700 1 -- 172.21.15.89:0/5078704 shutdown_connections 2023-10-10T20:34:43.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.550+0000 7f376f9d4700 1 --2- 172.21.15.89:0/5078704 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f374c066f30 0x7f374c0693e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.550+0000 7f376f9d4700 1 --2- 172.21.15.89:0/5078704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37680a54c0 0x7f376815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.550+0000 7f376f9d4700 1 --2- 172.21.15.89:0/5078704 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37680691b0 0x7f3768086360 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.550+0000 7f376f9d4700 1 --2- 172.21.15.89:0/5078704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37680875d0 0x7f3768086e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.550+0000 7f376f9d4700 1 -- 172.21.15.89:0/5078704 >> 172.21.15.89:0/5078704 conn(0x7f37680cd000 msgr2=0x7f3768168d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.550+0000 7f376f9d4700 1 -- 172.21.15.89:0/5078704 shutdown_connections 2023-10-10T20:34:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.550+0000 7f376f9d4700 1 -- 172.21.15.89:0/5078704 wait complete. 2023-10-10T20:34:43.558 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-10-10T20:34:43.558 DEBUG:teuthology.parallel:result is None 2023-10-10T20:34:43.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-10-10T20:34:43.559 DEBUG:teuthology.parallel:result is None 2023-10-10T20:34:43.563 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-10-10T20:34:43.563 DEBUG:teuthology.parallel:result is None 2023-10-10T20:34:43.566 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2023-10-10T20:34:43.567 DEBUG:teuthology.parallel:result is None 2023-10-10T20:34:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.566+0000 7f31ad05e700 1 -- 172.21.15.89:0/4179467752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f31a804c3c0 con 0x7f31a8156760 2023-10-10T20:34:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.566+0000 7f3196ffd700 1 -- 172.21.15.89:0/4179467752 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f31a00514e0 con 0x7f31a8156760 2023-10-10T20:34:43.568 INFO:teuthology.orchestra.run.smithi089.stdout:55834574855 2023-10-10T20:34:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.568+0000 7f31ad05e700 1 -- 172.21.15.89:0/4179467752 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31900671f0 msgr2=0x7f31900696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.568+0000 7f31ad05e700 1 --2- 172.21.15.89:0/4179467752 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31900671f0 0x7f31900696a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f31980099b0 tx=0x7f3198016000 comp rx=0 tx=0).stop 2023-10-10T20:34:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.568+0000 7f31ad05e700 1 -- 172.21.15.89:0/4179467752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31a8156760 msgr2=0x7f31a816cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.568+0000 7f31ad05e700 1 --2- 172.21.15.89:0/4179467752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31a8156760 0x7f31a816cc40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f31a000bc30 tx=0x7f31a0009840 comp rx=0 tx=0).stop 2023-10-10T20:34:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.568+0000 7f31ad05e700 1 -- 172.21.15.89:0/4179467752 shutdown_connections 2023-10-10T20:34:43.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.568+0000 7f31ad05e700 1 --2- 172.21.15.89:0/4179467752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31a8156760 0x7f31a816cc40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.568+0000 7f31ad05e700 1 --2- 172.21.15.89:0/4179467752 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31900671f0 0x7f31900696a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.568+0000 7f31ad05e700 1 --2- 172.21.15.89:0/4179467752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a8068aa0 0x7f31a8153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.568+0000 7f31ad05e700 1 --2- 172.21.15.89:0/4179467752 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31a80cba20 0x7f31a816c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.568+0000 7f31ad05e700 1 -- 172.21.15.89:0/4179467752 >> 172.21.15.89:0/4179467752 conn(0x7f31a80cd000 msgr2=0x7f31a81540e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:43.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.568+0000 7f31ad05e700 1 -- 172.21.15.89:0/4179467752 shutdown_connections 2023-10-10T20:34:43.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.568+0000 7f31ad05e700 1 -- 172.21.15.89:0/4179467752 wait complete. 2023-10-10T20:34:43.582 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2023-10-10T20:34:43.583 DEBUG:teuthology.parallel:result is None 2023-10-10T20:34:43.583 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-10T20:34:43.583 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:34:43.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.735+0000 7feea69b1700 1 Processor -- start 2023-10-10T20:34:43.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.735+0000 7feea69b1700 1 -- start start 2023-10-10T20:34:43.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.736+0000 7feea69b1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feea0159c60 0x7feea015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.736+0000 7feea69b1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea015a160 con 0x7feea015db10 2023-10-10T20:34:43.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.736+0000 7feea69b1700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea00d13a0 con 0x7feea0159890 2023-10-10T20:34:43.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.736+0000 7feea69b1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea015e140 con 0x7feea0159c60 2023-10-10T20:34:43.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.736+0000 7feea4f4e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7feea015db10 0x7feea0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35778/0 (socket says 172.21.15.89:35778) 2023-10-10T20:34:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.736+0000 7feea4f4e700 1 -- 172.21.15.89:0/352937462 learned_addr learned my addr 172.21.15.89:0/352937462 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.736+0000 7fee9f7fe700 1 --2- 172.21.15.89:0/352937462 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feea0159c60 0x7feea015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.737+0000 7fee9effd700 1 -- 172.21.15.89:0/352937462 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876986976 0 0) 0x7feea015a160 con 0x7feea015db10 2023-10-10T20:34:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.737+0000 7fee9effd700 1 -- 172.21.15.89:0/352937462 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee88003570 con 0x7feea015db10 2023-10-10T20:34:43.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.737+0000 7fee9effd700 1 -- 172.21.15.89:0/352937462 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483226237 0 0) 0x7feea00d13a0 con 0x7feea0159890 2023-10-10T20:34:43.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.737+0000 7fee9effd700 1 -- 172.21.15.89:0/352937462 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feea015a160 con 0x7feea0159890 2023-10-10T20:34:43.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.737+0000 7fee9f7fe700 1 -- 172.21.15.89:0/352937462 >> v1:172.21.15.111:6789/0 conn(0x7feea0159890 legacy=0x7feea015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.737+0000 7fee9f7fe700 1 -- 172.21.15.89:0/352937462 >> v1:172.21.15.89:6789/0 conn(0x7feea015db10 legacy=0x7feea0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.737+0000 7fee9f7fe700 1 -- 172.21.15.89:0/352937462 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea015a3d0 con 0x7feea0159c60 2023-10-10T20:34:43.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.737+0000 7fee9f7fe700 1 --2- 172.21.15.89:0/352937462 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feea0159c60 0x7feea015d5d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fee94009910 tx=0x7fee940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aae7a7b871cca154 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.737+0000 7fee9effd700 1 -- 172.21.15.89:0/352937462 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee94011630 con 0x7feea0159c60 2023-10-10T20:34:43.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.737+0000 7fee9effd700 1 -- 172.21.15.89:0/352937462 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee94011790 con 0x7feea0159c60 2023-10-10T20:34:43.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.737+0000 7fee9effd700 1 -- 172.21.15.89:0/352937462 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee94011b10 con 0x7feea0159c60 2023-10-10T20:34:43.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.738+0000 7feea69b1700 1 -- 172.21.15.89:0/352937462 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feea0159c60 msgr2=0x7feea015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.738+0000 7feea69b1700 1 --2- 172.21.15.89:0/352937462 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feea0159c60 0x7feea015d5d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fee94009910 tx=0x7fee940091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.738+0000 7feea69b1700 1 -- 172.21.15.89:0/352937462 shutdown_connections 2023-10-10T20:34:43.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.738+0000 7feea69b1700 1 --2- 172.21.15.89:0/352937462 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feea0159c60 0x7feea015d5d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.738+0000 7feea69b1700 1 -- 172.21.15.89:0/352937462 >> 172.21.15.89:0/352937462 conn(0x7feea0068e20 msgr2=0x7feea00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:43.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.738+0000 7feea69b1700 1 -- 172.21.15.89:0/352937462 shutdown_connections 2023-10-10T20:34:43.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.738+0000 7feea69b1700 1 -- 172.21.15.89:0/352937462 wait complete. 2023-10-10T20:34:43.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.739+0000 7feea69b1700 1 Processor -- start 2023-10-10T20:34:43.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.739+0000 7feea69b1700 1 -- start start 2023-10-10T20:34:43.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7feea69b1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feea0159890 0x7feea00cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7feea69b1700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea0159c60 0x7feea0157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7feea69b1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feea015db10 0x7feea0147d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7feea69b1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea00cea70 con 0x7feea0159890 2023-10-10T20:34:43.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7feea69b1700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea00ce4f0 con 0x7feea0159c60 2023-10-10T20:34:43.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7feea69b1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea00ce790 con 0x7feea015db10 2023-10-10T20:34:43.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7fee9ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feea0159890 0x7feea00cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7fee9ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feea0159890 0x7feea00cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53628/0 (socket says 172.21.15.89:53628) 2023-10-10T20:34:43.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7fee9ffff700 1 -- 172.21.15.89:0/382244900 learned_addr learned my addr 172.21.15.89:0/382244900 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:43.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7feea4f4e700 1 --2- 172.21.15.89:0/382244900 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feea015db10 0x7feea0147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7fee9f7fe700 1 --2- 172.21.15.89:0/382244900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea0159c60 0x7feea0157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7fee9ffff700 1 -- 172.21.15.89:0/382244900 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feea015db10 msgr2=0x7feea0147d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7fee9ffff700 1 --2- 172.21.15.89:0/382244900 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feea015db10 0x7feea0147d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7fee9ffff700 1 -- 172.21.15.89:0/382244900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea0159c60 msgr2=0x7feea0157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7fee9ffff700 1 --2- 172.21.15.89:0/382244900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea0159c60 0x7feea0157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7fee9ffff700 1 -- 172.21.15.89:0/382244900 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee94009420 con 0x7feea0159890 2023-10-10T20:34:43.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7fee9ffff700 1 --2- 172.21.15.89:0/382244900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feea0159890 0x7feea00cdfb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fee9000c790 tx=0x7fee90007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7fee87fff700 1 -- 172.21.15.89:0/382244900 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee90010510 con 0x7feea0159890 2023-10-10T20:34:43.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7fee87fff700 1 -- 172.21.15.89:0/382244900 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee90010670 con 0x7feea0159890 2023-10-10T20:34:43.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7feea69b1700 1 -- 172.21.15.89:0/382244900 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea01551b0 con 0x7feea0159890 2023-10-10T20:34:43.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7fee87fff700 1 -- 172.21.15.89:0/382244900 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee90010a10 con 0x7feea0159890 2023-10-10T20:34:43.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.740+0000 7feea69b1700 1 -- 172.21.15.89:0/382244900 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea0155770 con 0x7feea0159890 2023-10-10T20:34:43.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.741+0000 7feea69b1700 1 -- 172.21.15.89:0/382244900 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea0155010 con 0x7feea0159890 2023-10-10T20:34:43.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.742+0000 7fee87fff700 1 -- 172.21.15.89:0/382244900 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fee9001b3b0 con 0x7feea0159890 2023-10-10T20:34:43.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.742+0000 7fee87fff700 1 --2- 172.21.15.89:0/382244900 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fee8c066f30 0x7fee8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:43.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.742+0000 7fee87fff700 1 -- 172.21.15.89:0/382244900 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fee90083dd0 con 0x7feea0159890 2023-10-10T20:34:43.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.742+0000 7fee9f7fe700 1 --2- 172.21.15.89:0/382244900 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fee8c066f30 0x7fee8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:43.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.742+0000 7fee9f7fe700 1 --2- 172.21.15.89:0/382244900 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fee8c066f30 0x7fee8c0693e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feea015a090 tx=0x7fee94017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:43.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.746+0000 7fee87fff700 1 -- 172.21.15.89:0/382244900 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feea0155010 con 0x7feea0159890 2023-10-10T20:34:43.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.922+0000 7feea69b1700 1 -- 172.21.15.89:0/382244900 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feea0155010 con 0x7fee8c066f30 2023-10-10T20:34:43.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.929+0000 7fee87fff700 1 -- 172.21.15.89:0/382244900 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264504 (secure 0 0 0) 0x7feea0155010 con 0x7fee8c066f30 2023-10-10T20:34:43.934 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:43.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.932+0000 7feea69b1700 1 -- 172.21.15.89:0/382244900 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fee8c066f30 msgr2=0x7fee8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.932+0000 7feea69b1700 1 --2- 172.21.15.89:0/382244900 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fee8c066f30 0x7fee8c0693e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feea015a090 tx=0x7fee94017000 comp rx=0 tx=0).stop 2023-10-10T20:34:43.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.932+0000 7feea69b1700 1 -- 172.21.15.89:0/382244900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feea0159890 msgr2=0x7feea00cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:43.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.932+0000 7feea69b1700 1 --2- 172.21.15.89:0/382244900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feea0159890 0x7feea00cdfb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fee9000c790 tx=0x7fee90007e10 comp rx=0 tx=0).stop 2023-10-10T20:34:43.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.932+0000 7feea69b1700 1 -- 172.21.15.89:0/382244900 shutdown_connections 2023-10-10T20:34:43.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.932+0000 7feea69b1700 1 --2- 172.21.15.89:0/382244900 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feea015db10 0x7feea0147d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.932+0000 7feea69b1700 1 --2- 172.21.15.89:0/382244900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feea0159890 0x7feea00cdfb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.932+0000 7feea69b1700 1 --2- 172.21.15.89:0/382244900 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fee8c066f30 0x7fee8c0693e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.932+0000 7feea69b1700 1 --2- 172.21.15.89:0/382244900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea0159c60 0x7feea0157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:43.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.932+0000 7feea69b1700 1 -- 172.21.15.89:0/382244900 >> 172.21.15.89:0/382244900 conn(0x7feea0068e20 msgr2=0x7feea01530f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.932+0000 7feea69b1700 1 -- 172.21.15.89:0/382244900 shutdown_connections 2023-10-10T20:34:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:43.932+0000 7feea69b1700 1 -- 172.21.15.89:0/382244900 wait complete. 2023-10-10T20:34:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:34:43.946 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-10T20:34:42.653552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381124,"kb_used_data":2820,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593596,"statfs":{"total":773094113280,"available":770655842304,"internally_reserved":0,"allocated":2887680,"data_stored":1749619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001586"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.347393+0000","last_change":"2023-10-10T20:34:35.344417+0000","last_active":"2023-10-10T20:34:35.347393+0000","last_peered":"2023-10-10T20:34:35.347393+0000","last_clean":"2023-10-10T20:34:35.347393+0000","last_became_active":"2023-10-10T20:34:35.344171+0000","last_became_peered":"2023-10-10T20:34:35.344171+0000","last_unstale":"2023-10-10T20:34:35.347393+0000","last_undegraded":"2023-10-10T20:34:35.347393+0000","last_fullsized":"2023-10-10T20:34:35.347393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:06.136941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350364+0000","last_change":"2023-10-10T20:34:35.344388+0000","last_active":"2023-10-10T20:34:35.350364+0000","last_peered":"2023-10-10T20:34:35.350364+0000","last_clean":"2023-10-10T20:34:35.350364+0000","last_became_active":"2023-10-10T20:34:35.344132+0000","last_became_peered":"2023-10-10T20:34:35.344132+0000","last_unstale":"2023-10-10T20:34:35.350364+0000","last_undegraded":"2023-10-10T20:34:35.350364+0000","last_fullsized":"2023-10-10T20:34:35.350364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:05:02.787176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.341464+0000","last_change":"2023-10-10T20:34:35.339021+0000","last_active":"2023-10-10T20:34:35.341464+0000","last_peered":"2023-10-10T20:34:35.341464+0000","last_clean":"2023-10-10T20:34:35.341464+0000","last_became_active":"2023-10-10T20:34:35.338713+0000","last_became_peered":"2023-10-10T20:34:35.338713+0000","last_unstale":"2023-10-10T20:34:35.341464+0000","last_undegraded":"2023-10-10T20:34:35.341464+0000","last_fullsized":"2023-10-10T20:34:35.341464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:12:13.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349539+0000","last_change":"2023-10-10T20:34:35.345674+0000","last_active":"2023-10-10T20:34:35.349539+0000","last_peered":"2023-10-10T20:34:35.349539+0000","last_clean":"2023-10-10T20:34:35.349539+0000","last_became_active":"2023-10-10T20:34:35.342947+0000","last_became_peered":"2023-10-10T20:34:35.342947+0000","last_unstale":"2023-10-10T20:34:35.349539+0000","last_undegraded":"2023-10-10T20:34:35.349539+0000","last_fullsized":"2023-10-10T20:34:35.349539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:06.136941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.331236+0000","last_change":"2023-10-10T20:34:35.330128+0000","last_active":"2023-10-10T20:34:35.331236+0000","last_peered":"2023-10-10T20:34:35.331236+0000","last_clean":"2023-10-10T20:34:35.331236+0000","last_became_active":"2023-10-10T20:34:35.329809+0000","last_became_peered":"2023-10-10T20:34:35.329809+0000","last_unstale":"2023-10-10T20:34:35.331236+0000","last_undegraded":"2023-10-10T20:34:35.331236+0000","last_fullsized":"2023-10-10T20:34:35.331236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:08.378701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.351515+0000","last_change":"2023-10-10T20:34:35.341852+0000","last_active":"2023-10-10T20:34:35.351515+0000","last_peered":"2023-10-10T20:34:35.351515+0000","last_clean":"2023-10-10T20:34:35.351515+0000","last_became_active":"2023-10-10T20:34:35.341590+0000","last_became_peered":"2023-10-10T20:34:35.341590+0000","last_unstale":"2023-10-10T20:34:35.351515+0000","last_undegraded":"2023-10-10T20:34:35.351515+0000","last_fullsized":"2023-10-10T20:34:35.351515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:38:40.522047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342438+0000","last_change":"2023-10-10T20:34:35.336268+0000","last_active":"2023-10-10T20:34:35.342438+0000","last_peered":"2023-10-10T20:34:35.342438+0000","last_clean":"2023-10-10T20:34:35.342438+0000","last_became_active":"2023-10-10T20:34:35.335108+0000","last_became_peered":"2023-10-10T20:34:35.335108+0000","last_unstale":"2023-10-10T20:34:35.342438+0000","last_undegraded":"2023-10-10T20:34:35.342438+0000","last_fullsized":"2023-10-10T20:34:35.342438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:36.295226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350260+0000","last_change":"2023-10-10T20:34:35.342130+0000","last_active":"2023-10-10T20:34:35.350260+0000","last_peered":"2023-10-10T20:34:35.350260+0000","last_clean":"2023-10-10T20:34:35.350260+0000","last_became_active":"2023-10-10T20:34:35.341794+0000","last_became_peered":"2023-10-10T20:34:35.341794+0000","last_unstale":"2023-10-10T20:34:35.350260+0000","last_undegraded":"2023-10-10T20:34:35.350260+0000","last_fullsized":"2023-10-10T20:34:35.350260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.347376+0000","last_change":"2023-10-10T20:34:35.342521+0000","last_active":"2023-10-10T20:34:35.347376+0000","last_peered":"2023-10-10T20:34:35.347376+0000","last_clean":"2023-10-10T20:34:35.347376+0000","last_became_active":"2023-10-10T20:34:35.342237+0000","last_became_peered":"2023-10-10T20:34:35.342237+0000","last_unstale":"2023-10-10T20:34:35.347376+0000","last_undegraded":"2023-10-10T20:34:35.347376+0000","last_fullsized":"2023-10-10T20:34:35.347376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:35.499006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348040+0000","last_change":"2023-10-10T20:34:35.344521+0000","last_active":"2023-10-10T20:34:35.348040+0000","last_peered":"2023-10-10T20:34:35.348040+0000","last_clean":"2023-10-10T20:34:35.348040+0000","last_became_active":"2023-10-10T20:34:35.344132+0000","last_became_peered":"2023-10-10T20:34:35.344132+0000","last_unstale":"2023-10-10T20:34:35.348040+0000","last_undegraded":"2023-10-10T20:34:35.348040+0000","last_fullsized":"2023-10-10T20:34:35.348040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:05:02.787176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.344573+0000","last_change":"2023-10-10T20:34:35.342152+0000","last_active":"2023-10-10T20:34:35.344573+0000","last_peered":"2023-10-10T20:34:35.344573+0000","last_clean":"2023-10-10T20:34:35.344573+0000","last_became_active":"2023-10-10T20:34:35.341883+0000","last_became_peered":"2023-10-10T20:34:35.341883+0000","last_unstale":"2023-10-10T20:34:35.344573+0000","last_undegraded":"2023-10-10T20:34:35.344573+0000","last_fullsized":"2023-10-10T20:34:35.344573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:10:13.711031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350167+0000","last_change":"2023-10-10T20:34:35.342940+0000","last_active":"2023-10-10T20:34:35.350167+0000","last_peered":"2023-10-10T20:34:35.350167+0000","last_clean":"2023-10-10T20:34:35.350167+0000","last_became_active":"2023-10-10T20:34:35.342645+0000","last_became_peered":"2023-10-10T20:34:35.342645+0000","last_unstale":"2023-10-10T20:34:35.350167+0000","last_undegraded":"2023-10-10T20:34:35.350167+0000","last_fullsized":"2023-10-10T20:34:35.350167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:48:12.831828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350468+0000","last_change":"2023-10-10T20:34:35.343591+0000","last_active":"2023-10-10T20:34:35.350468+0000","last_peered":"2023-10-10T20:34:35.350468+0000","last_clean":"2023-10-10T20:34:35.350468+0000","last_became_active":"2023-10-10T20:34:35.343310+0000","last_became_peered":"2023-10-10T20:34:35.343310+0000","last_unstale":"2023-10-10T20:34:35.350468+0000","last_undegraded":"2023-10-10T20:34:35.350468+0000","last_fullsized":"2023-10-10T20:34:35.350468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:24:50.729855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348660+0000","last_change":"2023-10-10T20:34:35.343316+0000","last_active":"2023-10-10T20:34:35.348660+0000","last_peered":"2023-10-10T20:34:35.348660+0000","last_clean":"2023-10-10T20:34:35.348660+0000","last_became_active":"2023-10-10T20:34:35.343015+0000","last_became_peered":"2023-10-10T20:34:35.343015+0000","last_unstale":"2023-10-10T20:34:35.348660+0000","last_undegraded":"2023-10-10T20:34:35.348660+0000","last_fullsized":"2023-10-10T20:34:35.348660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:05:02.787176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352727+0000","last_change":"2023-10-10T20:34:35.346542+0000","last_active":"2023-10-10T20:34:35.352727+0000","last_peered":"2023-10-10T20:34:35.352727+0000","last_clean":"2023-10-10T20:34:35.352727+0000","last_became_active":"2023-10-10T20:34:35.346323+0000","last_became_peered":"2023-10-10T20:34:35.346323+0000","last_unstale":"2023-10-10T20:34:35.352727+0000","last_undegraded":"2023-10-10T20:34:35.352727+0000","last_fullsized":"2023-10-10T20:34:35.352727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:11:35.427062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349566+0000","last_change":"2023-10-10T20:34:35.346445+0000","last_active":"2023-10-10T20:34:35.349566+0000","last_peered":"2023-10-10T20:34:35.349566+0000","last_clean":"2023-10-10T20:34:35.349566+0000","last_became_active":"2023-10-10T20:34:35.345449+0000","last_became_peered":"2023-10-10T20:34:35.345449+0000","last_unstale":"2023-10-10T20:34:35.349566+0000","last_undegraded":"2023-10-10T20:34:35.349566+0000","last_fullsized":"2023-10-10T20:34:35.349566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350860+0000","last_change":"2023-10-10T20:34:35.348398+0000","last_active":"2023-10-10T20:34:35.350860+0000","last_peered":"2023-10-10T20:34:35.350860+0000","last_clean":"2023-10-10T20:34:35.350860+0000","last_became_active":"2023-10-10T20:34:35.348110+0000","last_became_peered":"2023-10-10T20:34:35.348110+0000","last_unstale":"2023-10-10T20:34:35.350860+0000","last_undegraded":"2023-10-10T20:34:35.350860+0000","last_fullsized":"2023-10-10T20:34:35.350860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:36.295226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350234+0000","last_change":"2023-10-10T20:34:35.343071+0000","last_active":"2023-10-10T20:34:35.350234+0000","last_peered":"2023-10-10T20:34:35.350234+0000","last_clean":"2023-10-10T20:34:35.350234+0000","last_became_active":"2023-10-10T20:34:35.342740+0000","last_became_peered":"2023-10-10T20:34:35.342740+0000","last_unstale":"2023-10-10T20:34:35.350234+0000","last_undegraded":"2023-10-10T20:34:35.350234+0000","last_fullsized":"2023-10-10T20:34:35.350234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:25:34.953681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.331257+0000","last_change":"2023-10-10T20:34:35.330198+0000","last_active":"2023-10-10T20:34:35.331257+0000","last_peered":"2023-10-10T20:34:35.331257+0000","last_clean":"2023-10-10T20:34:35.331257+0000","last_became_active":"2023-10-10T20:34:35.329847+0000","last_became_peered":"2023-10-10T20:34:35.329847+0000","last_unstale":"2023-10-10T20:34:35.331257+0000","last_undegraded":"2023-10-10T20:34:35.331257+0000","last_fullsized":"2023-10-10T20:34:35.331257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:10.218128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362481+0000","last_change":"2023-10-10T20:34:34.340043+0000","last_active":"2023-10-10T20:34:35.362481+0000","last_peered":"2023-10-10T20:34:35.362481+0000","last_clean":"2023-10-10T20:34:35.362481+0000","last_became_active":"2023-10-10T20:34:34.339783+0000","last_became_peered":"2023-10-10T20:34:34.339783+0000","last_unstale":"2023-10-10T20:34:35.362481+0000","last_undegraded":"2023-10-10T20:34:35.362481+0000","last_fullsized":"2023-10-10T20:34:35.362481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:00:46.542353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350303+0000","last_change":"2023-10-10T20:34:35.342670+0000","last_active":"2023-10-10T20:34:35.350303+0000","last_peered":"2023-10-10T20:34:35.350303+0000","last_clean":"2023-10-10T20:34:35.350303+0000","last_became_active":"2023-10-10T20:34:35.342432+0000","last_became_peered":"2023-10-10T20:34:35.342432+0000","last_unstale":"2023-10-10T20:34:35.350303+0000","last_undegraded":"2023-10-10T20:34:35.350303+0000","last_fullsized":"2023-10-10T20:34:35.350303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:38:40.522047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362959+0000","last_change":"2023-10-10T20:34:34.343567+0000","last_active":"2023-10-10T20:34:35.362959+0000","last_peered":"2023-10-10T20:34:35.362959+0000","last_clean":"2023-10-10T20:34:35.362959+0000","last_became_active":"2023-10-10T20:34:34.343344+0000","last_became_peered":"2023-10-10T20:34:34.343344+0000","last_unstale":"2023-10-10T20:34:35.362959+0000","last_undegraded":"2023-10-10T20:34:35.362959+0000","last_fullsized":"2023-10-10T20:34:35.362959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:57:27.930567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348540+0000","last_change":"2023-10-10T20:34:35.341588+0000","last_active":"2023-10-10T20:34:35.348540+0000","last_peered":"2023-10-10T20:34:35.348540+0000","last_clean":"2023-10-10T20:34:35.348540+0000","last_became_active":"2023-10-10T20:34:35.341193+0000","last_became_peered":"2023-10-10T20:34:35.341193+0000","last_unstale":"2023-10-10T20:34:35.348540+0000","last_undegraded":"2023-10-10T20:34:35.348540+0000","last_fullsized":"2023-10-10T20:34:35.348540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:25:55.933179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.357498+0000","last_change":"2023-10-10T20:34:34.347611+0000","last_active":"2023-10-10T20:34:35.357498+0000","last_peered":"2023-10-10T20:34:35.357498+0000","last_clean":"2023-10-10T20:34:35.357498+0000","last_became_active":"2023-10-10T20:34:34.347335+0000","last_became_peered":"2023-10-10T20:34:34.347335+0000","last_unstale":"2023-10-10T20:34:35.357498+0000","last_undegraded":"2023-10-10T20:34:35.357498+0000","last_fullsized":"2023-10-10T20:34:35.357498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:11.616929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350859+0000","last_change":"2023-10-10T20:34:35.348359+0000","last_active":"2023-10-10T20:34:35.350859+0000","last_peered":"2023-10-10T20:34:35.350859+0000","last_clean":"2023-10-10T20:34:35.350859+0000","last_became_active":"2023-10-10T20:34:35.347861+0000","last_became_peered":"2023-10-10T20:34:35.347861+0000","last_unstale":"2023-10-10T20:34:35.350859+0000","last_undegraded":"2023-10-10T20:34:35.350859+0000","last_fullsized":"2023-10-10T20:34:35.350859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:39.183060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.356525+0000","last_change":"2023-10-10T20:34:34.344260+0000","last_active":"2023-10-10T20:34:35.356525+0000","last_peered":"2023-10-10T20:34:35.356525+0000","last_clean":"2023-10-10T20:34:35.356525+0000","last_became_active":"2023-10-10T20:34:34.343945+0000","last_became_peered":"2023-10-10T20:34:34.343945+0000","last_unstale":"2023-10-10T20:34:35.356525+0000","last_undegraded":"2023-10-10T20:34:35.356525+0000","last_fullsized":"2023-10-10T20:34:35.356525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352561+0000","last_change":"2023-10-10T20:34:35.348160+0000","last_active":"2023-10-10T20:34:35.352561+0000","last_peered":"2023-10-10T20:34:35.352561+0000","last_clean":"2023-10-10T20:34:35.352561+0000","last_became_active":"2023-10-10T20:34:35.347847+0000","last_became_peered":"2023-10-10T20:34:35.347847+0000","last_unstale":"2023-10-10T20:34:35.352561+0000","last_undegraded":"2023-10-10T20:34:35.352561+0000","last_fullsized":"2023-10-10T20:34:35.352561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:08.378701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.361981+0000","last_change":"2023-10-10T20:34:34.339973+0000","last_active":"2023-10-10T20:34:35.361981+0000","last_peered":"2023-10-10T20:34:35.361981+0000","last_clean":"2023-10-10T20:34:35.361981+0000","last_became_active":"2023-10-10T20:34:34.339685+0000","last_became_peered":"2023-10-10T20:34:34.339685+0000","last_unstale":"2023-10-10T20:34:35.361981+0000","last_undegraded":"2023-10-10T20:34:35.361981+0000","last_fullsized":"2023-10-10T20:34:35.361981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:34:00.263416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348564+0000","last_change":"2023-10-10T20:34:35.340064+0000","last_active":"2023-10-10T20:34:35.348564+0000","last_peered":"2023-10-10T20:34:35.348564+0000","last_clean":"2023-10-10T20:34:35.348564+0000","last_became_active":"2023-10-10T20:34:35.339811+0000","last_became_peered":"2023-10-10T20:34:35.339811+0000","last_unstale":"2023-10-10T20:34:35.348564+0000","last_undegraded":"2023-10-10T20:34:35.348564+0000","last_fullsized":"2023-10-10T20:34:35.348564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:17.774898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.357351+0000","last_change":"2023-10-10T20:34:34.350170+0000","last_active":"2023-10-10T20:34:35.357351+0000","last_peered":"2023-10-10T20:34:35.357351+0000","last_clean":"2023-10-10T20:34:35.357351+0000","last_became_active":"2023-10-10T20:34:34.349858+0000","last_became_peered":"2023-10-10T20:34:34.349858+0000","last_unstale":"2023-10-10T20:34:35.357351+0000","last_undegraded":"2023-10-10T20:34:35.357351+0000","last_fullsized":"2023-10-10T20:34:35.357351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.351173+0000","last_change":"2023-10-10T20:34:35.342458+0000","last_active":"2023-10-10T20:34:35.351173+0000","last_peered":"2023-10-10T20:34:35.351173+0000","last_clean":"2023-10-10T20:34:35.351173+0000","last_became_active":"2023-10-10T20:34:35.342063+0000","last_became_peered":"2023-10-10T20:34:35.342063+0000","last_unstale":"2023-10-10T20:34:35.351173+0000","last_undegraded":"2023-10-10T20:34:35.351173+0000","last_fullsized":"2023-10-10T20:34:35.351173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:18.709140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.334160+0000","last_change":"2023-10-10T20:34:34.344734+0000","last_active":"2023-10-10T20:34:35.334160+0000","last_peered":"2023-10-10T20:34:35.334160+0000","last_clean":"2023-10-10T20:34:35.334160+0000","last_became_active":"2023-10-10T20:34:34.344418+0000","last_became_peered":"2023-10-10T20:34:34.344418+0000","last_unstale":"2023-10-10T20:34:35.334160+0000","last_undegraded":"2023-10-10T20:34:35.334160+0000","last_fullsized":"2023-10-10T20:34:35.334160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:27:04.988955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350422+0000","last_change":"2023-10-10T20:34:35.343560+0000","last_active":"2023-10-10T20:34:35.350422+0000","last_peered":"2023-10-10T20:34:35.350422+0000","last_clean":"2023-10-10T20:34:35.350422+0000","last_became_active":"2023-10-10T20:34:35.343170+0000","last_became_peered":"2023-10-10T20:34:35.343170+0000","last_unstale":"2023-10-10T20:34:35.350422+0000","last_undegraded":"2023-10-10T20:34:35.350422+0000","last_fullsized":"2023-10-10T20:34:35.350422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:05:02.787176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.334617+0000","last_change":"2023-10-10T20:34:34.339163+0000","last_active":"2023-10-10T20:34:35.334617+0000","last_peered":"2023-10-10T20:34:35.334617+0000","last_clean":"2023-10-10T20:34:35.334617+0000","last_became_active":"2023-10-10T20:34:34.338879+0000","last_became_peered":"2023-10-10T20:34:34.338879+0000","last_unstale":"2023-10-10T20:34:35.334617+0000","last_undegraded":"2023-10-10T20:34:35.334617+0000","last_fullsized":"2023-10-10T20:34:35.334617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:40:03.331998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342534+0000","last_change":"2023-10-10T20:34:35.340814+0000","last_active":"2023-10-10T20:34:35.342534+0000","last_peered":"2023-10-10T20:34:35.342534+0000","last_clean":"2023-10-10T20:34:35.342534+0000","last_became_active":"2023-10-10T20:34:35.340139+0000","last_became_peered":"2023-10-10T20:34:35.340139+0000","last_unstale":"2023-10-10T20:34:35.342534+0000","last_undegraded":"2023-10-10T20:34:35.342534+0000","last_fullsized":"2023-10-10T20:34:35.342534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:47:45.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.336472+0000","last_change":"2023-10-10T20:34:34.348036+0000","last_active":"2023-10-10T20:34:35.336472+0000","last_peered":"2023-10-10T20:34:35.336472+0000","last_clean":"2023-10-10T20:34:35.336472+0000","last_became_active":"2023-10-10T20:34:34.347739+0000","last_became_peered":"2023-10-10T20:34:34.347739+0000","last_unstale":"2023-10-10T20:34:35.336472+0000","last_undegraded":"2023-10-10T20:34:35.336472+0000","last_fullsized":"2023-10-10T20:34:35.336472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:27:04.988955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.354125+0000","last_change":"2023-10-10T20:34:35.352621+0000","last_active":"2023-10-10T20:34:35.354125+0000","last_peered":"2023-10-10T20:34:35.354125+0000","last_clean":"2023-10-10T20:34:35.354125+0000","last_became_active":"2023-10-10T20:34:35.352324+0000","last_became_peered":"2023-10-10T20:34:35.352324+0000","last_unstale":"2023-10-10T20:34:35.354125+0000","last_undegraded":"2023-10-10T20:34:35.354125+0000","last_fullsized":"2023-10-10T20:34:35.354125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:24:50.729855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.334195+0000","last_change":"2023-10-10T20:34:34.344822+0000","last_active":"2023-10-10T20:34:35.334195+0000","last_peered":"2023-10-10T20:34:35.334195+0000","last_clean":"2023-10-10T20:34:35.334195+0000","last_became_active":"2023-10-10T20:34:34.344549+0000","last_became_peered":"2023-10-10T20:34:34.344549+0000","last_unstale":"2023-10-10T20:34:35.334195+0000","last_undegraded":"2023-10-10T20:34:35.334195+0000","last_fullsized":"2023-10-10T20:34:35.334195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.344601+0000","last_change":"2023-10-10T20:34:35.339793+0000","last_active":"2023-10-10T20:34:35.344601+0000","last_peered":"2023-10-10T20:34:35.344601+0000","last_clean":"2023-10-10T20:34:35.344601+0000","last_became_active":"2023-10-10T20:34:35.339226+0000","last_became_peered":"2023-10-10T20:34:35.339226+0000","last_unstale":"2023-10-10T20:34:35.344601+0000","last_undegraded":"2023-10-10T20:34:35.344601+0000","last_fullsized":"2023-10-10T20:34:35.344601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:10:13.711031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.360889+0000","last_change":"2023-10-10T20:34:34.336051+0000","last_active":"2023-10-10T20:34:35.360889+0000","last_peered":"2023-10-10T20:34:35.360889+0000","last_clean":"2023-10-10T20:34:35.360889+0000","last_became_active":"2023-10-10T20:34:34.335689+0000","last_became_peered":"2023-10-10T20:34:34.335689+0000","last_unstale":"2023-10-10T20:34:35.360889+0000","last_undegraded":"2023-10-10T20:34:35.360889+0000","last_fullsized":"2023-10-10T20:34:35.360889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:11.616929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350147+0000","last_change":"2023-10-10T20:34:35.341270+0000","last_active":"2023-10-10T20:34:35.350147+0000","last_peered":"2023-10-10T20:34:35.350147+0000","last_clean":"2023-10-10T20:34:35.350147+0000","last_became_active":"2023-10-10T20:34:35.341024+0000","last_became_peered":"2023-10-10T20:34:35.341024+0000","last_unstale":"2023-10-10T20:34:35.350147+0000","last_undegraded":"2023-10-10T20:34:35.350147+0000","last_fullsized":"2023-10-10T20:34:35.350147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:16.035453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.367035+0000","last_change":"2023-10-10T20:34:34.346412+0000","last_active":"2023-10-10T20:34:35.367035+0000","last_peered":"2023-10-10T20:34:35.367035+0000","last_clean":"2023-10-10T20:34:35.367035+0000","last_became_active":"2023-10-10T20:34:34.346020+0000","last_became_peered":"2023-10-10T20:34:34.346020+0000","last_unstale":"2023-10-10T20:34:35.367035+0000","last_undegraded":"2023-10-10T20:34:35.367035+0000","last_fullsized":"2023-10-10T20:34:35.367035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:15.030808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.345530+0000","last_change":"2023-10-10T20:34:35.340255+0000","last_active":"2023-10-10T20:34:35.345530+0000","last_peered":"2023-10-10T20:34:35.345530+0000","last_clean":"2023-10-10T20:34:35.345530+0000","last_became_active":"2023-10-10T20:34:35.339971+0000","last_became_peered":"2023-10-10T20:34:35.339971+0000","last_unstale":"2023-10-10T20:34:35.345530+0000","last_undegraded":"2023-10-10T20:34:35.345530+0000","last_fullsized":"2023-10-10T20:34:35.345530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:44:40.877955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362334+0000","last_change":"2023-10-10T20:34:34.343026+0000","last_active":"2023-10-10T20:34:35.362334+0000","last_peered":"2023-10-10T20:34:35.362334+0000","last_clean":"2023-10-10T20:34:35.362334+0000","last_became_active":"2023-10-10T20:34:34.342742+0000","last_became_peered":"2023-10-10T20:34:34.342742+0000","last_unstale":"2023-10-10T20:34:35.362334+0000","last_undegraded":"2023-10-10T20:34:35.362334+0000","last_fullsized":"2023-10-10T20:34:35.362334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:57:27.930567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348647+0000","last_change":"2023-10-10T20:34:35.342407+0000","last_active":"2023-10-10T20:34:35.348647+0000","last_peered":"2023-10-10T20:34:35.348647+0000","last_clean":"2023-10-10T20:34:35.348647+0000","last_became_active":"2023-10-10T20:34:35.341971+0000","last_became_peered":"2023-10-10T20:34:35.341971+0000","last_unstale":"2023-10-10T20:34:35.348647+0000","last_undegraded":"2023-10-10T20:34:35.348647+0000","last_fullsized":"2023-10-10T20:34:35.348647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:27:10.934315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.336796+0000","last_change":"2023-10-10T20:34:34.339893+0000","last_active":"2023-10-10T20:34:35.336796+0000","last_peered":"2023-10-10T20:34:35.336796+0000","last_clean":"2023-10-10T20:34:35.336796+0000","last_became_active":"2023-10-10T20:34:34.339585+0000","last_became_peered":"2023-10-10T20:34:34.339585+0000","last_unstale":"2023-10-10T20:34:35.336796+0000","last_undegraded":"2023-10-10T20:34:35.336796+0000","last_fullsized":"2023-10-10T20:34:35.336796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:15.030808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.346782+0000","last_change":"2023-10-10T20:34:35.341978+0000","last_active":"2023-10-10T20:34:35.346782+0000","last_peered":"2023-10-10T20:34:35.346782+0000","last_clean":"2023-10-10T20:34:35.346782+0000","last_became_active":"2023-10-10T20:34:35.341703+0000","last_became_peered":"2023-10-10T20:34:35.341703+0000","last_unstale":"2023-10-10T20:34:35.346782+0000","last_undegraded":"2023-10-10T20:34:35.346782+0000","last_fullsized":"2023-10-10T20:34:35.346782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:38:40.522047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.335441+0000","last_change":"2023-10-10T20:34:34.350137+0000","last_active":"2023-10-10T20:34:35.335441+0000","last_peered":"2023-10-10T20:34:35.335441+0000","last_clean":"2023-10-10T20:34:35.335441+0000","last_became_active":"2023-10-10T20:34:34.349855+0000","last_became_peered":"2023-10-10T20:34:34.349855+0000","last_unstale":"2023-10-10T20:34:35.335441+0000","last_undegraded":"2023-10-10T20:34:35.335441+0000","last_fullsized":"2023-10-10T20:34:35.335441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.347963+0000","last_change":"2023-10-10T20:34:35.344256+0000","last_active":"2023-10-10T20:34:35.347963+0000","last_peered":"2023-10-10T20:34:35.347963+0000","last_clean":"2023-10-10T20:34:35.347963+0000","last_became_active":"2023-10-10T20:34:35.343626+0000","last_became_peered":"2023-10-10T20:34:35.343626+0000","last_unstale":"2023-10-10T20:34:35.347963+0000","last_undegraded":"2023-10-10T20:34:35.347963+0000","last_fullsized":"2023-10-10T20:34:35.347963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:12:13.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.361414+0000","last_change":"2023-10-10T20:34:34.337912+0000","last_active":"2023-10-10T20:34:35.361414+0000","last_peered":"2023-10-10T20:34:35.361414+0000","last_clean":"2023-10-10T20:34:35.361414+0000","last_became_active":"2023-10-10T20:34:34.337588+0000","last_became_peered":"2023-10-10T20:34:34.337588+0000","last_unstale":"2023-10-10T20:34:35.361414+0000","last_undegraded":"2023-10-10T20:34:35.361414+0000","last_fullsized":"2023-10-10T20:34:35.361414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:40:03.331998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342457+0000","last_change":"2023-10-10T20:34:35.337938+0000","last_active":"2023-10-10T20:34:35.342457+0000","last_peered":"2023-10-10T20:34:35.342457+0000","last_clean":"2023-10-10T20:34:35.342457+0000","last_became_active":"2023-10-10T20:34:35.337649+0000","last_became_peered":"2023-10-10T20:34:35.337649+0000","last_unstale":"2023-10-10T20:34:35.342457+0000","last_undegraded":"2023-10-10T20:34:35.342457+0000","last_fullsized":"2023-10-10T20:34:35.342457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362619+0000","last_change":"2023-10-10T20:34:34.344045+0000","last_active":"2023-10-10T20:34:35.362619+0000","last_peered":"2023-10-10T20:34:35.362619+0000","last_clean":"2023-10-10T20:34:35.362619+0000","last_became_active":"2023-10-10T20:34:34.343736+0000","last_became_peered":"2023-10-10T20:34:34.343736+0000","last_unstale":"2023-10-10T20:34:35.362619+0000","last_undegraded":"2023-10-10T20:34:35.362619+0000","last_fullsized":"2023-10-10T20:34:35.362619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:57:27.930567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350691+0000","last_change":"2023-10-10T20:34:35.343035+0000","last_active":"2023-10-10T20:34:35.350691+0000","last_peered":"2023-10-10T20:34:35.350691+0000","last_clean":"2023-10-10T20:34:35.350691+0000","last_became_active":"2023-10-10T20:34:35.342776+0000","last_became_peered":"2023-10-10T20:34:35.342776+0000","last_unstale":"2023-10-10T20:34:35.350691+0000","last_undegraded":"2023-10-10T20:34:35.350691+0000","last_fullsized":"2023-10-10T20:34:35.350691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:52:03.361718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.337288+0000","last_change":"2023-10-10T20:34:34.337961+0000","last_active":"2023-10-10T20:34:35.337288+0000","last_peered":"2023-10-10T20:34:35.337288+0000","last_clean":"2023-10-10T20:34:35.337288+0000","last_became_active":"2023-10-10T20:34:34.337627+0000","last_became_peered":"2023-10-10T20:34:34.337627+0000","last_unstale":"2023-10-10T20:34:35.337288+0000","last_undegraded":"2023-10-10T20:34:35.337288+0000","last_fullsized":"2023-10-10T20:34:35.337288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:34:00.263416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349588+0000","last_change":"2023-10-10T20:34:35.346121+0000","last_active":"2023-10-10T20:34:35.349588+0000","last_peered":"2023-10-10T20:34:35.349588+0000","last_clean":"2023-10-10T20:34:35.349588+0000","last_became_active":"2023-10-10T20:34:35.345847+0000","last_became_peered":"2023-10-10T20:34:35.345847+0000","last_unstale":"2023-10-10T20:34:35.349588+0000","last_undegraded":"2023-10-10T20:34:35.349588+0000","last_fullsized":"2023-10-10T20:34:35.349588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:10.218128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.336695+0000","last_change":"2023-10-10T20:34:34.336680+0000","last_active":"2023-10-10T20:34:35.336695+0000","last_peered":"2023-10-10T20:34:35.336695+0000","last_clean":"2023-10-10T20:34:35.336695+0000","last_became_active":"2023-10-10T20:34:34.335533+0000","last_became_peered":"2023-10-10T20:34:34.335533+0000","last_unstale":"2023-10-10T20:34:35.336695+0000","last_undegraded":"2023-10-10T20:34:35.336695+0000","last_fullsized":"2023-10-10T20:34:35.336695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349524+0000","last_change":"2023-10-10T20:34:35.342266+0000","last_active":"2023-10-10T20:34:35.349524+0000","last_peered":"2023-10-10T20:34:35.349524+0000","last_clean":"2023-10-10T20:34:35.349524+0000","last_became_active":"2023-10-10T20:34:35.341981+0000","last_became_peered":"2023-10-10T20:34:35.341981+0000","last_unstale":"2023-10-10T20:34:35.349524+0000","last_undegraded":"2023-10-10T20:34:35.349524+0000","last_fullsized":"2023-10-10T20:34:35.349524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:11.206264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.345007+0000","last_change":"2023-10-10T20:34:35.341780+0000","last_active":"2023-10-10T20:34:35.345007+0000","last_peered":"2023-10-10T20:34:35.345007+0000","last_clean":"2023-10-10T20:34:35.345007+0000","last_became_active":"2023-10-10T20:34:35.340539+0000","last_became_peered":"2023-10-10T20:34:35.340539+0000","last_unstale":"2023-10-10T20:34:35.345007+0000","last_undegraded":"2023-10-10T20:34:35.345007+0000","last_fullsized":"2023-10-10T20:34:35.345007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:12:13.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.328070+0000","last_change":"2023-10-10T20:34:34.340247+0000","last_active":"2023-10-10T20:34:35.328070+0000","last_peered":"2023-10-10T20:34:35.328070+0000","last_clean":"2023-10-10T20:34:35.328070+0000","last_became_active":"2023-10-10T20:34:34.339946+0000","last_became_peered":"2023-10-10T20:34:34.339946+0000","last_unstale":"2023-10-10T20:34:35.328070+0000","last_undegraded":"2023-10-10T20:34:35.328070+0000","last_fullsized":"2023-10-10T20:34:35.328070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350545+0000","last_change":"2023-10-10T20:34:35.344288+0000","last_active":"2023-10-10T20:34:35.350545+0000","last_peered":"2023-10-10T20:34:35.350545+0000","last_clean":"2023-10-10T20:34:35.350545+0000","last_became_active":"2023-10-10T20:34:35.343736+0000","last_became_peered":"2023-10-10T20:34:35.343736+0000","last_unstale":"2023-10-10T20:34:35.350545+0000","last_undegraded":"2023-10-10T20:34:35.350545+0000","last_fullsized":"2023-10-10T20:34:35.350545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:18.709140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.333336+0000","last_change":"2023-10-10T20:34:34.341870+0000","last_active":"2023-10-10T20:34:35.333336+0000","last_peered":"2023-10-10T20:34:35.333336+0000","last_clean":"2023-10-10T20:34:35.333336+0000","last_became_active":"2023-10-10T20:34:34.341567+0000","last_became_peered":"2023-10-10T20:34:34.341567+0000","last_unstale":"2023-10-10T20:34:35.333336+0000","last_undegraded":"2023-10-10T20:34:35.333336+0000","last_fullsized":"2023-10-10T20:34:35.333336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:11.616929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.327481+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:34:35.327481+0000","last_peered":"2023-10-10T20:34:35.327481+0000","last_clean":"2023-10-10T20:34:35.327481+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:34:35.327481+0000","last_undegraded":"2023-10-10T20:34:35.327481+0000","last_fullsized":"2023-10-10T20:34:35.327481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.344647+0000","last_change":"2023-10-10T20:34:35.340986+0000","last_active":"2023-10-10T20:34:35.344647+0000","last_peered":"2023-10-10T20:34:35.344647+0000","last_clean":"2023-10-10T20:34:35.344647+0000","last_became_active":"2023-10-10T20:34:35.339812+0000","last_became_peered":"2023-10-10T20:34:35.339812+0000","last_unstale":"2023-10-10T20:34:35.344647+0000","last_undegraded":"2023-10-10T20:34:35.344647+0000","last_fullsized":"2023-10-10T20:34:35.344647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:11:35.427062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.327681+0000","last_change":"2023-10-10T20:34:34.345132+0000","last_active":"2023-10-10T20:34:35.327681+0000","last_peered":"2023-10-10T20:34:35.327681+0000","last_clean":"2023-10-10T20:34:35.327681+0000","last_became_active":"2023-10-10T20:34:34.344843+0000","last_became_peered":"2023-10-10T20:34:34.344843+0000","last_unstale":"2023-10-10T20:34:35.327681+0000","last_undegraded":"2023-10-10T20:34:35.327681+0000","last_fullsized":"2023-10-10T20:34:35.327681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:15.030808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.328545+0000","last_change":"2023-10-10T20:34:34.346151+0000","last_active":"2023-10-10T20:34:35.328545+0000","last_peered":"2023-10-10T20:34:35.328545+0000","last_clean":"2023-10-10T20:34:35.328545+0000","last_became_active":"2023-10-10T20:34:34.345765+0000","last_became_peered":"2023-10-10T20:34:34.345765+0000","last_unstale":"2023-10-10T20:34:35.328545+0000","last_undegraded":"2023-10-10T20:34:35.328545+0000","last_fullsized":"2023-10-10T20:34:35.328545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:10.201619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342808+0000","last_change":"2023-10-10T20:34:35.340238+0000","last_active":"2023-10-10T20:34:35.342808+0000","last_peered":"2023-10-10T20:34:35.342808+0000","last_clean":"2023-10-10T20:34:35.342808+0000","last_became_active":"2023-10-10T20:34:35.339944+0000","last_became_peered":"2023-10-10T20:34:35.339944+0000","last_unstale":"2023-10-10T20:34:35.342808+0000","last_undegraded":"2023-10-10T20:34:35.342808+0000","last_fullsized":"2023-10-10T20:34:35.342808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-10T20:34:36.526880+0000","last_change":"2023-10-10T20:34:34.347897+0000","last_active":"2023-10-10T20:34:36.526880+0000","last_peered":"2023-10-10T20:34:36.526880+0000","last_clean":"2023-10-10T20:34:36.526880+0000","last_became_active":"2023-10-10T20:34:34.347575+0000","last_became_peered":"2023-10-10T20:34:34.347575+0000","last_unstale":"2023-10-10T20:34:36.526880+0000","last_undegraded":"2023-10-10T20:34:36.526880+0000","last_fullsized":"2023-10-10T20:34:36.526880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:15.030808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350573+0000","last_change":"2023-10-10T20:34:35.345207+0000","last_active":"2023-10-10T20:34:35.350573+0000","last_peered":"2023-10-10T20:34:35.350573+0000","last_clean":"2023-10-10T20:34:35.350573+0000","last_became_active":"2023-10-10T20:34:35.344809+0000","last_became_peered":"2023-10-10T20:34:35.344809+0000","last_unstale":"2023-10-10T20:34:35.350573+0000","last_undegraded":"2023-10-10T20:34:35.350573+0000","last_fullsized":"2023-10-10T20:34:35.350573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:36.295226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.358045+0000","last_change":"2023-10-10T20:34:34.348399+0000","last_active":"2023-10-10T20:34:35.358045+0000","last_peered":"2023-10-10T20:34:35.358045+0000","last_clean":"2023-10-10T20:34:35.358045+0000","last_became_active":"2023-10-10T20:34:34.348099+0000","last_became_peered":"2023-10-10T20:34:34.348099+0000","last_unstale":"2023-10-10T20:34:35.358045+0000","last_undegraded":"2023-10-10T20:34:35.358045+0000","last_fullsized":"2023-10-10T20:34:35.358045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:40:03.331998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.344533+0000","last_change":"2023-10-10T20:34:35.340459+0000","last_active":"2023-10-10T20:34:35.344533+0000","last_peered":"2023-10-10T20:34:35.344533+0000","last_clean":"2023-10-10T20:34:35.344533+0000","last_became_active":"2023-10-10T20:34:35.340176+0000","last_became_peered":"2023-10-10T20:34:35.340176+0000","last_unstale":"2023-10-10T20:34:35.344533+0000","last_undegraded":"2023-10-10T20:34:35.344533+0000","last_fullsized":"2023-10-10T20:34:35.344533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:48:12.831828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.327499+0000","last_change":"2023-10-10T20:34:34.346150+0000","last_active":"2023-10-10T20:34:35.327499+0000","last_peered":"2023-10-10T20:34:35.327499+0000","last_clean":"2023-10-10T20:34:35.327499+0000","last_became_active":"2023-10-10T20:34:34.345811+0000","last_became_peered":"2023-10-10T20:34:34.345811+0000","last_unstale":"2023-10-10T20:34:35.327499+0000","last_undegraded":"2023-10-10T20:34:35.327499+0000","last_fullsized":"2023-10-10T20:34:35.327499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:04.665843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350476+0000","last_change":"2023-10-10T20:34:35.345083+0000","last_active":"2023-10-10T20:34:35.350476+0000","last_peered":"2023-10-10T20:34:35.350476+0000","last_clean":"2023-10-10T20:34:35.350476+0000","last_became_active":"2023-10-10T20:34:35.343922+0000","last_became_peered":"2023-10-10T20:34:35.343922+0000","last_unstale":"2023-10-10T20:34:35.350476+0000","last_undegraded":"2023-10-10T20:34:35.350476+0000","last_fullsized":"2023-10-10T20:34:35.350476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:05.670488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.337189+0000","last_change":"2023-10-10T20:34:34.340327+0000","last_active":"2023-10-10T20:34:35.337189+0000","last_peered":"2023-10-10T20:34:35.337189+0000","last_clean":"2023-10-10T20:34:35.337189+0000","last_became_active":"2023-10-10T20:34:34.339737+0000","last_became_peered":"2023-10-10T20:34:34.339737+0000","last_unstale":"2023-10-10T20:34:35.337189+0000","last_undegraded":"2023-10-10T20:34:35.337189+0000","last_fullsized":"2023-10-10T20:34:35.337189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350500+0000","last_change":"2023-10-10T20:34:35.344039+0000","last_active":"2023-10-10T20:34:35.350500+0000","last_peered":"2023-10-10T20:34:35.350500+0000","last_clean":"2023-10-10T20:34:35.350500+0000","last_became_active":"2023-10-10T20:34:35.343789+0000","last_became_peered":"2023-10-10T20:34:35.343789+0000","last_unstale":"2023-10-10T20:34:35.350500+0000","last_undegraded":"2023-10-10T20:34:35.350500+0000","last_fullsized":"2023-10-10T20:34:35.350500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:11:35.427062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.337192+0000","last_change":"2023-10-10T20:34:34.340022+0000","last_active":"2023-10-10T20:34:35.337192+0000","last_peered":"2023-10-10T20:34:35.337192+0000","last_clean":"2023-10-10T20:34:35.337192+0000","last_became_active":"2023-10-10T20:34:34.339652+0000","last_became_peered":"2023-10-10T20:34:34.339652+0000","last_unstale":"2023-10-10T20:34:35.337192+0000","last_undegraded":"2023-10-10T20:34:35.337192+0000","last_fullsized":"2023-10-10T20:34:35.337192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350304+0000","last_change":"2023-10-10T20:34:35.342858+0000","last_active":"2023-10-10T20:34:35.350304+0000","last_peered":"2023-10-10T20:34:35.350304+0000","last_clean":"2023-10-10T20:34:35.350304+0000","last_became_active":"2023-10-10T20:34:35.341911+0000","last_became_peered":"2023-10-10T20:34:35.341911+0000","last_unstale":"2023-10-10T20:34:35.350304+0000","last_undegraded":"2023-10-10T20:34:35.350304+0000","last_fullsized":"2023-10-10T20:34:35.350304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:25:55.933179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.357218+0000","last_change":"2023-10-10T20:34:34.346661+0000","last_active":"2023-10-10T20:34:35.357218+0000","last_peered":"2023-10-10T20:34:35.357218+0000","last_clean":"2023-10-10T20:34:35.357218+0000","last_became_active":"2023-10-10T20:34:34.346384+0000","last_became_peered":"2023-10-10T20:34:34.346384+0000","last_unstale":"2023-10-10T20:34:35.357218+0000","last_undegraded":"2023-10-10T20:34:35.357218+0000","last_fullsized":"2023-10-10T20:34:35.357218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350769+0000","last_change":"2023-10-10T20:34:35.349300+0000","last_active":"2023-10-10T20:34:35.350769+0000","last_peered":"2023-10-10T20:34:35.350769+0000","last_clean":"2023-10-10T20:34:35.350769+0000","last_became_active":"2023-10-10T20:34:35.348997+0000","last_became_peered":"2023-10-10T20:34:35.348997+0000","last_unstale":"2023-10-10T20:34:35.350769+0000","last_undegraded":"2023-10-10T20:34:35.350769+0000","last_fullsized":"2023-10-10T20:34:35.350769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:47:45.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.328113+0000","last_change":"2023-10-10T20:34:34.344048+0000","last_active":"2023-10-10T20:34:35.328113+0000","last_peered":"2023-10-10T20:34:35.328113+0000","last_clean":"2023-10-10T20:34:35.328113+0000","last_became_active":"2023-10-10T20:34:34.343866+0000","last_became_peered":"2023-10-10T20:34:34.343866+0000","last_unstale":"2023-10-10T20:34:35.328113+0000","last_undegraded":"2023-10-10T20:34:35.328113+0000","last_fullsized":"2023-10-10T20:34:35.328113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:27:04.988955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342399+0000","last_change":"2023-10-10T20:34:35.337534+0000","last_active":"2023-10-10T20:34:35.342399+0000","last_peered":"2023-10-10T20:34:35.342399+0000","last_clean":"2023-10-10T20:34:35.342399+0000","last_became_active":"2023-10-10T20:34:35.337277+0000","last_became_peered":"2023-10-10T20:34:35.337277+0000","last_unstale":"2023-10-10T20:34:35.342399+0000","last_undegraded":"2023-10-10T20:34:35.342399+0000","last_fullsized":"2023-10-10T20:34:35.342399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:42:23.450966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.360813+0000","last_change":"2023-10-10T20:34:34.340837+0000","last_active":"2023-10-10T20:34:35.360813+0000","last_peered":"2023-10-10T20:34:35.360813+0000","last_clean":"2023-10-10T20:34:35.360813+0000","last_became_active":"2023-10-10T20:34:34.338971+0000","last_became_peered":"2023-10-10T20:34:34.338971+0000","last_unstale":"2023-10-10T20:34:35.360813+0000","last_undegraded":"2023-10-10T20:34:35.360813+0000","last_fullsized":"2023-10-10T20:34:35.360813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:10.201619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349519+0000","last_change":"2023-10-10T20:34:35.345474+0000","last_active":"2023-10-10T20:34:35.349519+0000","last_peered":"2023-10-10T20:34:35.349519+0000","last_clean":"2023-10-10T20:34:35.349519+0000","last_became_active":"2023-10-10T20:34:35.343255+0000","last_became_peered":"2023-10-10T20:34:35.343255+0000","last_unstale":"2023-10-10T20:34:35.349519+0000","last_undegraded":"2023-10-10T20:34:35.349519+0000","last_fullsized":"2023-10-10T20:34:35.349519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:35.499006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362947+0000","last_change":"2023-10-10T20:34:34.339953+0000","last_active":"2023-10-10T20:34:35.362947+0000","last_peered":"2023-10-10T20:34:35.362947+0000","last_clean":"2023-10-10T20:34:35.362947+0000","last_became_active":"2023-10-10T20:34:34.339676+0000","last_became_peered":"2023-10-10T20:34:34.339676+0000","last_unstale":"2023-10-10T20:34:35.362947+0000","last_undegraded":"2023-10-10T20:34:35.362947+0000","last_fullsized":"2023-10-10T20:34:35.362947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342687+0000","last_change":"2023-10-10T20:34:35.338535+0000","last_active":"2023-10-10T20:34:35.342687+0000","last_peered":"2023-10-10T20:34:35.342687+0000","last_clean":"2023-10-10T20:34:35.342687+0000","last_became_active":"2023-10-10T20:34:35.338266+0000","last_became_peered":"2023-10-10T20:34:35.338266+0000","last_unstale":"2023-10-10T20:34:35.342687+0000","last_undegraded":"2023-10-10T20:34:35.342687+0000","last_fullsized":"2023-10-10T20:34:35.342687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:39.183060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348224+0000","last_change":"2023-10-10T20:34:35.346165+0000","last_active":"2023-10-10T20:34:35.348224+0000","last_peered":"2023-10-10T20:34:35.348224+0000","last_clean":"2023-10-10T20:34:35.348224+0000","last_became_active":"2023-10-10T20:34:35.345862+0000","last_became_peered":"2023-10-10T20:34:35.345862+0000","last_unstale":"2023-10-10T20:34:35.348224+0000","last_undegraded":"2023-10-10T20:34:35.348224+0000","last_fullsized":"2023-10-10T20:34:35.348224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:18.709140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350401+0000","last_change":"2023-10-10T20:34:35.342709+0000","last_active":"2023-10-10T20:34:35.350401+0000","last_peered":"2023-10-10T20:34:35.350401+0000","last_clean":"2023-10-10T20:34:35.350401+0000","last_became_active":"2023-10-10T20:34:35.342433+0000","last_became_peered":"2023-10-10T20:34:35.342433+0000","last_unstale":"2023-10-10T20:34:35.350401+0000","last_undegraded":"2023-10-10T20:34:35.350401+0000","last_fullsized":"2023-10-10T20:34:35.350401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:48:12.831828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342070+0000","last_change":"2023-10-10T20:34:35.340426+0000","last_active":"2023-10-10T20:34:35.342070+0000","last_peered":"2023-10-10T20:34:35.342070+0000","last_clean":"2023-10-10T20:34:35.342070+0000","last_became_active":"2023-10-10T20:34:35.339966+0000","last_became_peered":"2023-10-10T20:34:35.339966+0000","last_unstale":"2023-10-10T20:34:35.342070+0000","last_undegraded":"2023-10-10T20:34:35.342070+0000","last_fullsized":"2023-10-10T20:34:35.342070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:42:23.450966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.351149+0000","last_change":"2023-10-10T20:34:35.341444+0000","last_active":"2023-10-10T20:34:35.351149+0000","last_peered":"2023-10-10T20:34:35.351149+0000","last_clean":"2023-10-10T20:34:35.351149+0000","last_became_active":"2023-10-10T20:34:35.341186+0000","last_became_peered":"2023-10-10T20:34:35.341186+0000","last_unstale":"2023-10-10T20:34:35.351149+0000","last_undegraded":"2023-10-10T20:34:35.351149+0000","last_fullsized":"2023-10-10T20:34:35.351149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352359+0000","last_change":"2023-10-10T20:34:35.347375+0000","last_active":"2023-10-10T20:34:35.352359+0000","last_peered":"2023-10-10T20:34:35.352359+0000","last_clean":"2023-10-10T20:34:35.352359+0000","last_became_active":"2023-10-10T20:34:35.347094+0000","last_became_peered":"2023-10-10T20:34:35.347094+0000","last_unstale":"2023-10-10T20:34:35.352359+0000","last_undegraded":"2023-10-10T20:34:35.352359+0000","last_fullsized":"2023-10-10T20:34:35.352359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:12:13.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342757+0000","last_change":"2023-10-10T20:34:35.340918+0000","last_active":"2023-10-10T20:34:35.342757+0000","last_peered":"2023-10-10T20:34:35.342757+0000","last_clean":"2023-10-10T20:34:35.342757+0000","last_became_active":"2023-10-10T20:34:35.340638+0000","last_became_peered":"2023-10-10T20:34:35.340638+0000","last_unstale":"2023-10-10T20:34:35.342757+0000","last_undegraded":"2023-10-10T20:34:35.342757+0000","last_fullsized":"2023-10-10T20:34:35.342757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:18.709140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348059+0000","last_change":"2023-10-10T20:34:35.345371+0000","last_active":"2023-10-10T20:34:35.348059+0000","last_peered":"2023-10-10T20:34:35.348059+0000","last_clean":"2023-10-10T20:34:35.348059+0000","last_became_active":"2023-10-10T20:34:35.345058+0000","last_became_peered":"2023-10-10T20:34:35.345058+0000","last_unstale":"2023-10-10T20:34:35.348059+0000","last_undegraded":"2023-10-10T20:34:35.348059+0000","last_fullsized":"2023-10-10T20:34:35.348059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352396+0000","last_change":"2023-10-10T20:34:35.349283+0000","last_active":"2023-10-10T20:34:35.352396+0000","last_peered":"2023-10-10T20:34:35.352396+0000","last_clean":"2023-10-10T20:34:35.352396+0000","last_became_active":"2023-10-10T20:34:35.349040+0000","last_became_peered":"2023-10-10T20:34:35.349040+0000","last_unstale":"2023-10-10T20:34:35.352396+0000","last_undegraded":"2023-10-10T20:34:35.352396+0000","last_fullsized":"2023-10-10T20:34:35.352396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:47:45.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349316+0000","last_change":"2023-10-10T20:34:35.343369+0000","last_active":"2023-10-10T20:34:35.349316+0000","last_peered":"2023-10-10T20:34:35.349316+0000","last_clean":"2023-10-10T20:34:35.349316+0000","last_became_active":"2023-10-10T20:34:35.342875+0000","last_became_peered":"2023-10-10T20:34:35.342875+0000","last_unstale":"2023-10-10T20:34:35.349316+0000","last_undegraded":"2023-10-10T20:34:35.349316+0000","last_fullsized":"2023-10-10T20:34:35.349316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:44:40.877955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352369+0000","last_change":"2023-10-10T20:34:35.349284+0000","last_active":"2023-10-10T20:34:35.352369+0000","last_peered":"2023-10-10T20:34:35.352369+0000","last_clean":"2023-10-10T20:34:35.352369+0000","last_became_active":"2023-10-10T20:34:35.349057+0000","last_became_peered":"2023-10-10T20:34:35.349057+0000","last_unstale":"2023-10-10T20:34:35.352369+0000","last_undegraded":"2023-10-10T20:34:35.352369+0000","last_fullsized":"2023-10-10T20:34:35.352369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:52:03.361718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.354153+0000","last_change":"2023-10-10T20:34:35.353504+0000","last_active":"2023-10-10T20:34:35.354153+0000","last_peered":"2023-10-10T20:34:35.354153+0000","last_clean":"2023-10-10T20:34:35.354153+0000","last_became_active":"2023-10-10T20:34:35.350145+0000","last_became_peered":"2023-10-10T20:34:35.350145+0000","last_unstale":"2023-10-10T20:34:35.354153+0000","last_undegraded":"2023-10-10T20:34:35.354153+0000","last_fullsized":"2023-10-10T20:34:35.354153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:52:03.361718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.351192+0000","last_change":"2023-10-10T20:34:35.345734+0000","last_active":"2023-10-10T20:34:35.351192+0000","last_peered":"2023-10-10T20:34:35.351192+0000","last_clean":"2023-10-10T20:34:35.351192+0000","last_became_active":"2023-10-10T20:34:35.345466+0000","last_became_peered":"2023-10-10T20:34:35.345466+0000","last_unstale":"2023-10-10T20:34:35.351192+0000","last_undegraded":"2023-10-10T20:34:35.351192+0000","last_fullsized":"2023-10-10T20:34:35.351192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:47:45.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350754+0000","last_change":"2023-10-10T20:34:35.349383+0000","last_active":"2023-10-10T20:34:35.350754+0000","last_peered":"2023-10-10T20:34:35.350754+0000","last_clean":"2023-10-10T20:34:35.350754+0000","last_became_active":"2023-10-10T20:34:35.348979+0000","last_became_peered":"2023-10-10T20:34:35.348979+0000","last_unstale":"2023-10-10T20:34:35.350754+0000","last_undegraded":"2023-10-10T20:34:35.350754+0000","last_fullsized":"2023-10-10T20:34:35.350754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:25:55.933179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298072,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073768,"statfs":{"total":96636764160,"available":96331538432,"internally_reserved":0,"allocated":802816,"data_stored":661576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":212992,"data_stored":70814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.0380000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2869999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1159999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.0659999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298076,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073764,"statfs":{"total":96636764160,"available":96331534336,"internally_reserved":0,"allocated":806912,"data_stored":660671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":70654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":70780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":70744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:34:43.952 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:34:44.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.099+0000 7fa3d8fec700 1 Processor -- start 2023-10-10T20:34:44.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.099+0000 7fa3d8fec700 1 -- start start 2023-10-10T20:34:44.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.100+0000 7fa3d8fec700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3d4141500 0x7fa3d40c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:44.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.100+0000 7fa3d8fec700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa3d406c0a0 0x7fa3d40c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:44.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.100+0000 7fa3d8fec700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d415c2a0 con 0x7fa3d406bcd0 2023-10-10T20:34:44.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.100+0000 7fa3d8fec700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d40c9880 con 0x7fa3d406c0a0 2023-10-10T20:34:44.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.100+0000 7fa3d8fec700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d415b690 con 0x7fa3d4141500 2023-10-10T20:34:44.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.100+0000 7fa3d2d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa3d406c0a0 0x7fa3d40c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:44.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.100+0000 7fa3d2d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa3d406c0a0 0x7fa3d40c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53630/0 (socket says 172.21.15.89:53630) 2023-10-10T20:34:44.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.100+0000 7fa3d2d9d700 1 -- 172.21.15.89:0/3660078122 learned_addr learned my addr 172.21.15.89:0/3660078122 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:44.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.101+0000 7fa3d1d9b700 1 --2- 172.21.15.89:0/3660078122 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3d4141500 0x7fa3d40c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:44.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.101+0000 7fa3d2d9d700 1 -- 172.21.15.89:0/3660078122 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3d4141500 msgr2=0x7fa3d40c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:44.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.101+0000 7fa3d2d9d700 1 --2- 172.21.15.89:0/3660078122 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3d4141500 0x7fa3d40c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.101+0000 7fa3d2d9d700 1 -- 172.21.15.89:0/3660078122 >> v1:172.21.15.111:6789/0 conn(0x7fa3d406bcd0 legacy=0x7fa3d4140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:44.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.101+0000 7fa3d2d9d700 1 -- 172.21.15.89:0/3660078122 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d415b970 con 0x7fa3d406c0a0 2023-10-10T20:34:44.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.101+0000 7fa3d2d9d700 1 --2- 172.21.15.89:0/3660078122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d406c0a0 0x7fa3d40c9340 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0099b0 tx=0x7fa3cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94793db675a29e18 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:44.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.101+0000 7fa3d159a700 1 -- 172.21.15.89:0/3660078122 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00b860 con 0x7fa3d406c0a0 2023-10-10T20:34:44.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.101+0000 7fa3d159a700 1 -- 172.21.15.89:0/3660078122 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc00b9c0 con 0x7fa3d406c0a0 2023-10-10T20:34:44.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.101+0000 7fa3d159a700 1 -- 172.21.15.89:0/3660078122 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00bd60 con 0x7fa3d406c0a0 2023-10-10T20:34:44.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.101+0000 7fa3d8fec700 1 -- 172.21.15.89:0/3660078122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d406c0a0 msgr2=0x7fa3d40c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:44.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.101+0000 7fa3d8fec700 1 --2- 172.21.15.89:0/3660078122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d406c0a0 0x7fa3d40c9340 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0099b0 tx=0x7fa3cc009250 comp rx=0 tx=0).stop 2023-10-10T20:34:44.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.101+0000 7fa3d8fec700 1 -- 172.21.15.89:0/3660078122 shutdown_connections 2023-10-10T20:34:44.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.101+0000 7fa3d8fec700 1 --2- 172.21.15.89:0/3660078122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d406c0a0 0x7fa3d40c9340 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.101+0000 7fa3d8fec700 1 --2- 172.21.15.89:0/3660078122 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3d4141500 0x7fa3d40c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.102+0000 7fa3d8fec700 1 -- 172.21.15.89:0/3660078122 >> 172.21.15.89:0/3660078122 conn(0x7fa3d40ce600 msgr2=0x7fa3d40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:44.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.102+0000 7fa3d8fec700 1 -- 172.21.15.89:0/3660078122 shutdown_connections 2023-10-10T20:34:44.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.102+0000 7fa3d8fec700 1 -- 172.21.15.89:0/3660078122 wait complete. 2023-10-10T20:34:44.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.103+0000 7fa3d8fec700 1 Processor -- start 2023-10-10T20:34:44.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.103+0000 7fa3d8fec700 1 -- start start 2023-10-10T20:34:44.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.103+0000 7fa3d8fec700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3d406bcd0 0x7fa3d40b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:44.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.103+0000 7fa3d8fec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d406c0a0 0x7fa3d4157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:44.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.103+0000 7fa3d8fec700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3d4141500 0x7fa3d40c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:44.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.103+0000 7fa3d8fec700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d413f030 con 0x7fa3d406c0a0 2023-10-10T20:34:44.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.103+0000 7fa3d8fec700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d40c6260 con 0x7fa3d406bcd0 2023-10-10T20:34:44.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.103+0000 7fa3d8fec700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d40c6540 con 0x7fa3d4141500 2023-10-10T20:34:44.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.103+0000 7fa3d1d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d406c0a0 0x7fa3d4157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:44.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.103+0000 7fa3d1d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d406c0a0 0x7fa3d4157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53632/0 (socket says 172.21.15.89:53632) 2023-10-10T20:34:44.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.103+0000 7fa3d1d9b700 1 -- 172.21.15.89:0/3410117191 learned_addr learned my addr 172.21.15.89:0/3410117191 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:44.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.103+0000 7fa3d2d9d700 1 --2- 172.21.15.89:0/3410117191 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3d4141500 0x7fa3d40c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:44.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.103+0000 7fa3d259c700 1 --2- 172.21.15.89:0/3410117191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3d406bcd0 0x7fa3d40b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:44.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.104+0000 7fa3d1d9b700 1 -- 172.21.15.89:0/3410117191 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3d4141500 msgr2=0x7fa3d40c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:44.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.104+0000 7fa3d1d9b700 1 --2- 172.21.15.89:0/3410117191 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3d4141500 0x7fa3d40c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.104+0000 7fa3d1d9b700 1 -- 172.21.15.89:0/3410117191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3d406bcd0 msgr2=0x7fa3d40b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:44.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.104+0000 7fa3d1d9b700 1 --2- 172.21.15.89:0/3410117191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3d406bcd0 0x7fa3d40b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.104+0000 7fa3d1d9b700 1 -- 172.21.15.89:0/3410117191 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc0094f0 con 0x7fa3d406c0a0 2023-10-10T20:34:44.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.104+0000 7fa3d259c700 1 --2- 172.21.15.89:0/3410117191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3d406bcd0 0x7fa3d40b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:44.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.104+0000 7fa3d1d9b700 1 --2- 172.21.15.89:0/3410117191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d406c0a0 0x7fa3d4157f30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00d860 tx=0x7fa3bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:44.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.104+0000 7fa3ca7fc700 1 -- 172.21.15.89:0/3410117191 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc0125f0 con 0x7fa3d406c0a0 2023-10-10T20:34:44.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.104+0000 7fa3ca7fc700 1 -- 172.21.15.89:0/3410117191 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3bc012750 con 0x7fa3d406c0a0 2023-10-10T20:34:44.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.104+0000 7fa3ca7fc700 1 -- 172.21.15.89:0/3410117191 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc012b10 con 0x7fa3d406c0a0 2023-10-10T20:34:44.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.104+0000 7fa3d8fec700 1 -- 172.21.15.89:0/3410117191 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d40c6820 con 0x7fa3d406c0a0 2023-10-10T20:34:44.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.104+0000 7fa3d8fec700 1 -- 172.21.15.89:0/3410117191 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d40c6de0 con 0x7fa3d406c0a0 2023-10-10T20:34:44.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.105+0000 7fa3d8fec700 1 -- 172.21.15.89:0/3410117191 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d404c3c0 con 0x7fa3d406c0a0 2023-10-10T20:34:44.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.105+0000 7fa3ca7fc700 1 -- 172.21.15.89:0/3410117191 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa3bc01e460 con 0x7fa3d406c0a0 2023-10-10T20:34:44.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.105+0000 7fa3ca7fc700 1 --2- 172.21.15.89:0/3410117191 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa3b4066f30 0x7fa3b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:44.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.106+0000 7fa3ca7fc700 1 -- 172.21.15.89:0/3410117191 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa3bc084ad0 con 0x7fa3d406c0a0 2023-10-10T20:34:44.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.106+0000 7fa3d259c700 1 --2- 172.21.15.89:0/3410117191 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa3b4066f30 0x7fa3b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:44.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.106+0000 7fa3d259c700 1 --2- 172.21.15.89:0/3410117191 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa3b4066f30 0x7fa3b40693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4007fd0 tx=0x7fa3c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:44.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.110+0000 7fa3ca7fc700 1 -- 172.21.15.89:0/3410117191 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3bc051d40 con 0x7fa3d406c0a0 2023-10-10T20:34:44.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.287+0000 7fa3d8fec700 1 -- 172.21.15.89:0/3410117191 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa3d415bc50 con 0x7fa3b4066f30 2023-10-10T20:34:44.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.294+0000 7fa3ca7fc700 1 -- 172.21.15.89:0/3410117191 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264504 (secure 0 0 0) 0x7fa3d415bc50 con 0x7fa3b4066f30 2023-10-10T20:34:44.300 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:44.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.297+0000 7fa3d8fec700 1 -- 172.21.15.89:0/3410117191 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa3b4066f30 msgr2=0x7fa3b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:44.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.297+0000 7fa3d8fec700 1 --2- 172.21.15.89:0/3410117191 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa3b4066f30 0x7fa3b40693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4007fd0 tx=0x7fa3c4009040 comp rx=0 tx=0).stop 2023-10-10T20:34:44.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.297+0000 7fa3d8fec700 1 -- 172.21.15.89:0/3410117191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d406c0a0 msgr2=0x7fa3d4157f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:44.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.297+0000 7fa3d8fec700 1 --2- 172.21.15.89:0/3410117191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d406c0a0 0x7fa3d4157f30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00d860 tx=0x7fa3bc008040 comp rx=0 tx=0).stop 2023-10-10T20:34:44.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.297+0000 7fa3d8fec700 1 -- 172.21.15.89:0/3410117191 shutdown_connections 2023-10-10T20:34:44.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.297+0000 7fa3d8fec700 1 --2- 172.21.15.89:0/3410117191 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa3d4141500 0x7fa3d40c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.297+0000 7fa3d8fec700 1 --2- 172.21.15.89:0/3410117191 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa3b4066f30 0x7fa3b40693e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.297+0000 7fa3d8fec700 1 --2- 172.21.15.89:0/3410117191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3d406bcd0 0x7fa3d40b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.297+0000 7fa3d8fec700 1 --2- 172.21.15.89:0/3410117191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d406c0a0 0x7fa3d4157f30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.297+0000 7fa3d8fec700 1 -- 172.21.15.89:0/3410117191 >> 172.21.15.89:0/3410117191 conn(0x7fa3d40ce600 msgr2=0x7fa3d406cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:44.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.297+0000 7fa3d8fec700 1 -- 172.21.15.89:0/3410117191 shutdown_connections 2023-10-10T20:34:44.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.297+0000 7fa3d8fec700 1 -- 172.21.15.89:0/3410117191 wait complete. 2023-10-10T20:34:44.306 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:34:44.311 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-10T20:34:42.653552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381124,"kb_used_data":2820,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593596,"statfs":{"total":773094113280,"available":770655842304,"internally_reserved":0,"allocated":2887680,"data_stored":1749619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001586"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.347393+0000","last_change":"2023-10-10T20:34:35.344417+0000","last_active":"2023-10-10T20:34:35.347393+0000","last_peered":"2023-10-10T20:34:35.347393+0000","last_clean":"2023-10-10T20:34:35.347393+0000","last_became_active":"2023-10-10T20:34:35.344171+0000","last_became_peered":"2023-10-10T20:34:35.344171+0000","last_unstale":"2023-10-10T20:34:35.347393+0000","last_undegraded":"2023-10-10T20:34:35.347393+0000","last_fullsized":"2023-10-10T20:34:35.347393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:06.136941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350364+0000","last_change":"2023-10-10T20:34:35.344388+0000","last_active":"2023-10-10T20:34:35.350364+0000","last_peered":"2023-10-10T20:34:35.350364+0000","last_clean":"2023-10-10T20:34:35.350364+0000","last_became_active":"2023-10-10T20:34:35.344132+0000","last_became_peered":"2023-10-10T20:34:35.344132+0000","last_unstale":"2023-10-10T20:34:35.350364+0000","last_undegraded":"2023-10-10T20:34:35.350364+0000","last_fullsized":"2023-10-10T20:34:35.350364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:05:02.787176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.341464+0000","last_change":"2023-10-10T20:34:35.339021+0000","last_active":"2023-10-10T20:34:35.341464+0000","last_peered":"2023-10-10T20:34:35.341464+0000","last_clean":"2023-10-10T20:34:35.341464+0000","last_became_active":"2023-10-10T20:34:35.338713+0000","last_became_peered":"2023-10-10T20:34:35.338713+0000","last_unstale":"2023-10-10T20:34:35.341464+0000","last_undegraded":"2023-10-10T20:34:35.341464+0000","last_fullsized":"2023-10-10T20:34:35.341464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:12:13.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349539+0000","last_change":"2023-10-10T20:34:35.345674+0000","last_active":"2023-10-10T20:34:35.349539+0000","last_peered":"2023-10-10T20:34:35.349539+0000","last_clean":"2023-10-10T20:34:35.349539+0000","last_became_active":"2023-10-10T20:34:35.342947+0000","last_became_peered":"2023-10-10T20:34:35.342947+0000","last_unstale":"2023-10-10T20:34:35.349539+0000","last_undegraded":"2023-10-10T20:34:35.349539+0000","last_fullsized":"2023-10-10T20:34:35.349539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:06.136941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.331236+0000","last_change":"2023-10-10T20:34:35.330128+0000","last_active":"2023-10-10T20:34:35.331236+0000","last_peered":"2023-10-10T20:34:35.331236+0000","last_clean":"2023-10-10T20:34:35.331236+0000","last_became_active":"2023-10-10T20:34:35.329809+0000","last_became_peered":"2023-10-10T20:34:35.329809+0000","last_unstale":"2023-10-10T20:34:35.331236+0000","last_undegraded":"2023-10-10T20:34:35.331236+0000","last_fullsized":"2023-10-10T20:34:35.331236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:08.378701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.351515+0000","last_change":"2023-10-10T20:34:35.341852+0000","last_active":"2023-10-10T20:34:35.351515+0000","last_peered":"2023-10-10T20:34:35.351515+0000","last_clean":"2023-10-10T20:34:35.351515+0000","last_became_active":"2023-10-10T20:34:35.341590+0000","last_became_peered":"2023-10-10T20:34:35.341590+0000","last_unstale":"2023-10-10T20:34:35.351515+0000","last_undegraded":"2023-10-10T20:34:35.351515+0000","last_fullsized":"2023-10-10T20:34:35.351515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:38:40.522047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342438+0000","last_change":"2023-10-10T20:34:35.336268+0000","last_active":"2023-10-10T20:34:35.342438+0000","last_peered":"2023-10-10T20:34:35.342438+0000","last_clean":"2023-10-10T20:34:35.342438+0000","last_became_active":"2023-10-10T20:34:35.335108+0000","last_became_peered":"2023-10-10T20:34:35.335108+0000","last_unstale":"2023-10-10T20:34:35.342438+0000","last_undegraded":"2023-10-10T20:34:35.342438+0000","last_fullsized":"2023-10-10T20:34:35.342438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:36.295226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350260+0000","last_change":"2023-10-10T20:34:35.342130+0000","last_active":"2023-10-10T20:34:35.350260+0000","last_peered":"2023-10-10T20:34:35.350260+0000","last_clean":"2023-10-10T20:34:35.350260+0000","last_became_active":"2023-10-10T20:34:35.341794+0000","last_became_peered":"2023-10-10T20:34:35.341794+0000","last_unstale":"2023-10-10T20:34:35.350260+0000","last_undegraded":"2023-10-10T20:34:35.350260+0000","last_fullsized":"2023-10-10T20:34:35.350260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.347376+0000","last_change":"2023-10-10T20:34:35.342521+0000","last_active":"2023-10-10T20:34:35.347376+0000","last_peered":"2023-10-10T20:34:35.347376+0000","last_clean":"2023-10-10T20:34:35.347376+0000","last_became_active":"2023-10-10T20:34:35.342237+0000","last_became_peered":"2023-10-10T20:34:35.342237+0000","last_unstale":"2023-10-10T20:34:35.347376+0000","last_undegraded":"2023-10-10T20:34:35.347376+0000","last_fullsized":"2023-10-10T20:34:35.347376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:35.499006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348040+0000","last_change":"2023-10-10T20:34:35.344521+0000","last_active":"2023-10-10T20:34:35.348040+0000","last_peered":"2023-10-10T20:34:35.348040+0000","last_clean":"2023-10-10T20:34:35.348040+0000","last_became_active":"2023-10-10T20:34:35.344132+0000","last_became_peered":"2023-10-10T20:34:35.344132+0000","last_unstale":"2023-10-10T20:34:35.348040+0000","last_undegraded":"2023-10-10T20:34:35.348040+0000","last_fullsized":"2023-10-10T20:34:35.348040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:05:02.787176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.344573+0000","last_change":"2023-10-10T20:34:35.342152+0000","last_active":"2023-10-10T20:34:35.344573+0000","last_peered":"2023-10-10T20:34:35.344573+0000","last_clean":"2023-10-10T20:34:35.344573+0000","last_became_active":"2023-10-10T20:34:35.341883+0000","last_became_peered":"2023-10-10T20:34:35.341883+0000","last_unstale":"2023-10-10T20:34:35.344573+0000","last_undegraded":"2023-10-10T20:34:35.344573+0000","last_fullsized":"2023-10-10T20:34:35.344573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:10:13.711031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350167+0000","last_change":"2023-10-10T20:34:35.342940+0000","last_active":"2023-10-10T20:34:35.350167+0000","last_peered":"2023-10-10T20:34:35.350167+0000","last_clean":"2023-10-10T20:34:35.350167+0000","last_became_active":"2023-10-10T20:34:35.342645+0000","last_became_peered":"2023-10-10T20:34:35.342645+0000","last_unstale":"2023-10-10T20:34:35.350167+0000","last_undegraded":"2023-10-10T20:34:35.350167+0000","last_fullsized":"2023-10-10T20:34:35.350167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:48:12.831828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350468+0000","last_change":"2023-10-10T20:34:35.343591+0000","last_active":"2023-10-10T20:34:35.350468+0000","last_peered":"2023-10-10T20:34:35.350468+0000","last_clean":"2023-10-10T20:34:35.350468+0000","last_became_active":"2023-10-10T20:34:35.343310+0000","last_became_peered":"2023-10-10T20:34:35.343310+0000","last_unstale":"2023-10-10T20:34:35.350468+0000","last_undegraded":"2023-10-10T20:34:35.350468+0000","last_fullsized":"2023-10-10T20:34:35.350468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:24:50.729855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348660+0000","last_change":"2023-10-10T20:34:35.343316+0000","last_active":"2023-10-10T20:34:35.348660+0000","last_peered":"2023-10-10T20:34:35.348660+0000","last_clean":"2023-10-10T20:34:35.348660+0000","last_became_active":"2023-10-10T20:34:35.343015+0000","last_became_peered":"2023-10-10T20:34:35.343015+0000","last_unstale":"2023-10-10T20:34:35.348660+0000","last_undegraded":"2023-10-10T20:34:35.348660+0000","last_fullsized":"2023-10-10T20:34:35.348660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:05:02.787176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352727+0000","last_change":"2023-10-10T20:34:35.346542+0000","last_active":"2023-10-10T20:34:35.352727+0000","last_peered":"2023-10-10T20:34:35.352727+0000","last_clean":"2023-10-10T20:34:35.352727+0000","last_became_active":"2023-10-10T20:34:35.346323+0000","last_became_peered":"2023-10-10T20:34:35.346323+0000","last_unstale":"2023-10-10T20:34:35.352727+0000","last_undegraded":"2023-10-10T20:34:35.352727+0000","last_fullsized":"2023-10-10T20:34:35.352727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:11:35.427062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349566+0000","last_change":"2023-10-10T20:34:35.346445+0000","last_active":"2023-10-10T20:34:35.349566+0000","last_peered":"2023-10-10T20:34:35.349566+0000","last_clean":"2023-10-10T20:34:35.349566+0000","last_became_active":"2023-10-10T20:34:35.345449+0000","last_became_peered":"2023-10-10T20:34:35.345449+0000","last_unstale":"2023-10-10T20:34:35.349566+0000","last_undegraded":"2023-10-10T20:34:35.349566+0000","last_fullsized":"2023-10-10T20:34:35.349566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350860+0000","last_change":"2023-10-10T20:34:35.348398+0000","last_active":"2023-10-10T20:34:35.350860+0000","last_peered":"2023-10-10T20:34:35.350860+0000","last_clean":"2023-10-10T20:34:35.350860+0000","last_became_active":"2023-10-10T20:34:35.348110+0000","last_became_peered":"2023-10-10T20:34:35.348110+0000","last_unstale":"2023-10-10T20:34:35.350860+0000","last_undegraded":"2023-10-10T20:34:35.350860+0000","last_fullsized":"2023-10-10T20:34:35.350860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:36.295226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350234+0000","last_change":"2023-10-10T20:34:35.343071+0000","last_active":"2023-10-10T20:34:35.350234+0000","last_peered":"2023-10-10T20:34:35.350234+0000","last_clean":"2023-10-10T20:34:35.350234+0000","last_became_active":"2023-10-10T20:34:35.342740+0000","last_became_peered":"2023-10-10T20:34:35.342740+0000","last_unstale":"2023-10-10T20:34:35.350234+0000","last_undegraded":"2023-10-10T20:34:35.350234+0000","last_fullsized":"2023-10-10T20:34:35.350234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:25:34.953681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.331257+0000","last_change":"2023-10-10T20:34:35.330198+0000","last_active":"2023-10-10T20:34:35.331257+0000","last_peered":"2023-10-10T20:34:35.331257+0000","last_clean":"2023-10-10T20:34:35.331257+0000","last_became_active":"2023-10-10T20:34:35.329847+0000","last_became_peered":"2023-10-10T20:34:35.329847+0000","last_unstale":"2023-10-10T20:34:35.331257+0000","last_undegraded":"2023-10-10T20:34:35.331257+0000","last_fullsized":"2023-10-10T20:34:35.331257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:10.218128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362481+0000","last_change":"2023-10-10T20:34:34.340043+0000","last_active":"2023-10-10T20:34:35.362481+0000","last_peered":"2023-10-10T20:34:35.362481+0000","last_clean":"2023-10-10T20:34:35.362481+0000","last_became_active":"2023-10-10T20:34:34.339783+0000","last_became_peered":"2023-10-10T20:34:34.339783+0000","last_unstale":"2023-10-10T20:34:35.362481+0000","last_undegraded":"2023-10-10T20:34:35.362481+0000","last_fullsized":"2023-10-10T20:34:35.362481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:00:46.542353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350303+0000","last_change":"2023-10-10T20:34:35.342670+0000","last_active":"2023-10-10T20:34:35.350303+0000","last_peered":"2023-10-10T20:34:35.350303+0000","last_clean":"2023-10-10T20:34:35.350303+0000","last_became_active":"2023-10-10T20:34:35.342432+0000","last_became_peered":"2023-10-10T20:34:35.342432+0000","last_unstale":"2023-10-10T20:34:35.350303+0000","last_undegraded":"2023-10-10T20:34:35.350303+0000","last_fullsized":"2023-10-10T20:34:35.350303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:38:40.522047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362959+0000","last_change":"2023-10-10T20:34:34.343567+0000","last_active":"2023-10-10T20:34:35.362959+0000","last_peered":"2023-10-10T20:34:35.362959+0000","last_clean":"2023-10-10T20:34:35.362959+0000","last_became_active":"2023-10-10T20:34:34.343344+0000","last_became_peered":"2023-10-10T20:34:34.343344+0000","last_unstale":"2023-10-10T20:34:35.362959+0000","last_undegraded":"2023-10-10T20:34:35.362959+0000","last_fullsized":"2023-10-10T20:34:35.362959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:57:27.930567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348540+0000","last_change":"2023-10-10T20:34:35.341588+0000","last_active":"2023-10-10T20:34:35.348540+0000","last_peered":"2023-10-10T20:34:35.348540+0000","last_clean":"2023-10-10T20:34:35.348540+0000","last_became_active":"2023-10-10T20:34:35.341193+0000","last_became_peered":"2023-10-10T20:34:35.341193+0000","last_unstale":"2023-10-10T20:34:35.348540+0000","last_undegraded":"2023-10-10T20:34:35.348540+0000","last_fullsized":"2023-10-10T20:34:35.348540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:25:55.933179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.357498+0000","last_change":"2023-10-10T20:34:34.347611+0000","last_active":"2023-10-10T20:34:35.357498+0000","last_peered":"2023-10-10T20:34:35.357498+0000","last_clean":"2023-10-10T20:34:35.357498+0000","last_became_active":"2023-10-10T20:34:34.347335+0000","last_became_peered":"2023-10-10T20:34:34.347335+0000","last_unstale":"2023-10-10T20:34:35.357498+0000","last_undegraded":"2023-10-10T20:34:35.357498+0000","last_fullsized":"2023-10-10T20:34:35.357498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:11.616929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350859+0000","last_change":"2023-10-10T20:34:35.348359+0000","last_active":"2023-10-10T20:34:35.350859+0000","last_peered":"2023-10-10T20:34:35.350859+0000","last_clean":"2023-10-10T20:34:35.350859+0000","last_became_active":"2023-10-10T20:34:35.347861+0000","last_became_peered":"2023-10-10T20:34:35.347861+0000","last_unstale":"2023-10-10T20:34:35.350859+0000","last_undegraded":"2023-10-10T20:34:35.350859+0000","last_fullsized":"2023-10-10T20:34:35.350859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:39.183060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.356525+0000","last_change":"2023-10-10T20:34:34.344260+0000","last_active":"2023-10-10T20:34:35.356525+0000","last_peered":"2023-10-10T20:34:35.356525+0000","last_clean":"2023-10-10T20:34:35.356525+0000","last_became_active":"2023-10-10T20:34:34.343945+0000","last_became_peered":"2023-10-10T20:34:34.343945+0000","last_unstale":"2023-10-10T20:34:35.356525+0000","last_undegraded":"2023-10-10T20:34:35.356525+0000","last_fullsized":"2023-10-10T20:34:35.356525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352561+0000","last_change":"2023-10-10T20:34:35.348160+0000","last_active":"2023-10-10T20:34:35.352561+0000","last_peered":"2023-10-10T20:34:35.352561+0000","last_clean":"2023-10-10T20:34:35.352561+0000","last_became_active":"2023-10-10T20:34:35.347847+0000","last_became_peered":"2023-10-10T20:34:35.347847+0000","last_unstale":"2023-10-10T20:34:35.352561+0000","last_undegraded":"2023-10-10T20:34:35.352561+0000","last_fullsized":"2023-10-10T20:34:35.352561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:08.378701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.361981+0000","last_change":"2023-10-10T20:34:34.339973+0000","last_active":"2023-10-10T20:34:35.361981+0000","last_peered":"2023-10-10T20:34:35.361981+0000","last_clean":"2023-10-10T20:34:35.361981+0000","last_became_active":"2023-10-10T20:34:34.339685+0000","last_became_peered":"2023-10-10T20:34:34.339685+0000","last_unstale":"2023-10-10T20:34:35.361981+0000","last_undegraded":"2023-10-10T20:34:35.361981+0000","last_fullsized":"2023-10-10T20:34:35.361981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:34:00.263416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348564+0000","last_change":"2023-10-10T20:34:35.340064+0000","last_active":"2023-10-10T20:34:35.348564+0000","last_peered":"2023-10-10T20:34:35.348564+0000","last_clean":"2023-10-10T20:34:35.348564+0000","last_became_active":"2023-10-10T20:34:35.339811+0000","last_became_peered":"2023-10-10T20:34:35.339811+0000","last_unstale":"2023-10-10T20:34:35.348564+0000","last_undegraded":"2023-10-10T20:34:35.348564+0000","last_fullsized":"2023-10-10T20:34:35.348564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:17.774898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.357351+0000","last_change":"2023-10-10T20:34:34.350170+0000","last_active":"2023-10-10T20:34:35.357351+0000","last_peered":"2023-10-10T20:34:35.357351+0000","last_clean":"2023-10-10T20:34:35.357351+0000","last_became_active":"2023-10-10T20:34:34.349858+0000","last_became_peered":"2023-10-10T20:34:34.349858+0000","last_unstale":"2023-10-10T20:34:35.357351+0000","last_undegraded":"2023-10-10T20:34:35.357351+0000","last_fullsized":"2023-10-10T20:34:35.357351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.351173+0000","last_change":"2023-10-10T20:34:35.342458+0000","last_active":"2023-10-10T20:34:35.351173+0000","last_peered":"2023-10-10T20:34:35.351173+0000","last_clean":"2023-10-10T20:34:35.351173+0000","last_became_active":"2023-10-10T20:34:35.342063+0000","last_became_peered":"2023-10-10T20:34:35.342063+0000","last_unstale":"2023-10-10T20:34:35.351173+0000","last_undegraded":"2023-10-10T20:34:35.351173+0000","last_fullsized":"2023-10-10T20:34:35.351173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:18.709140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.334160+0000","last_change":"2023-10-10T20:34:34.344734+0000","last_active":"2023-10-10T20:34:35.334160+0000","last_peered":"2023-10-10T20:34:35.334160+0000","last_clean":"2023-10-10T20:34:35.334160+0000","last_became_active":"2023-10-10T20:34:34.344418+0000","last_became_peered":"2023-10-10T20:34:34.344418+0000","last_unstale":"2023-10-10T20:34:35.334160+0000","last_undegraded":"2023-10-10T20:34:35.334160+0000","last_fullsized":"2023-10-10T20:34:35.334160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:27:04.988955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350422+0000","last_change":"2023-10-10T20:34:35.343560+0000","last_active":"2023-10-10T20:34:35.350422+0000","last_peered":"2023-10-10T20:34:35.350422+0000","last_clean":"2023-10-10T20:34:35.350422+0000","last_became_active":"2023-10-10T20:34:35.343170+0000","last_became_peered":"2023-10-10T20:34:35.343170+0000","last_unstale":"2023-10-10T20:34:35.350422+0000","last_undegraded":"2023-10-10T20:34:35.350422+0000","last_fullsized":"2023-10-10T20:34:35.350422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:05:02.787176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.334617+0000","last_change":"2023-10-10T20:34:34.339163+0000","last_active":"2023-10-10T20:34:35.334617+0000","last_peered":"2023-10-10T20:34:35.334617+0000","last_clean":"2023-10-10T20:34:35.334617+0000","last_became_active":"2023-10-10T20:34:34.338879+0000","last_became_peered":"2023-10-10T20:34:34.338879+0000","last_unstale":"2023-10-10T20:34:35.334617+0000","last_undegraded":"2023-10-10T20:34:35.334617+0000","last_fullsized":"2023-10-10T20:34:35.334617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:40:03.331998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342534+0000","last_change":"2023-10-10T20:34:35.340814+0000","last_active":"2023-10-10T20:34:35.342534+0000","last_peered":"2023-10-10T20:34:35.342534+0000","last_clean":"2023-10-10T20:34:35.342534+0000","last_became_active":"2023-10-10T20:34:35.340139+0000","last_became_peered":"2023-10-10T20:34:35.340139+0000","last_unstale":"2023-10-10T20:34:35.342534+0000","last_undegraded":"2023-10-10T20:34:35.342534+0000","last_fullsized":"2023-10-10T20:34:35.342534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:47:45.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.336472+0000","last_change":"2023-10-10T20:34:34.348036+0000","last_active":"2023-10-10T20:34:35.336472+0000","last_peered":"2023-10-10T20:34:35.336472+0000","last_clean":"2023-10-10T20:34:35.336472+0000","last_became_active":"2023-10-10T20:34:34.347739+0000","last_became_peered":"2023-10-10T20:34:34.347739+0000","last_unstale":"2023-10-10T20:34:35.336472+0000","last_undegraded":"2023-10-10T20:34:35.336472+0000","last_fullsized":"2023-10-10T20:34:35.336472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:27:04.988955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.354125+0000","last_change":"2023-10-10T20:34:35.352621+0000","last_active":"2023-10-10T20:34:35.354125+0000","last_peered":"2023-10-10T20:34:35.354125+0000","last_clean":"2023-10-10T20:34:35.354125+0000","last_became_active":"2023-10-10T20:34:35.352324+0000","last_became_peered":"2023-10-10T20:34:35.352324+0000","last_unstale":"2023-10-10T20:34:35.354125+0000","last_undegraded":"2023-10-10T20:34:35.354125+0000","last_fullsized":"2023-10-10T20:34:35.354125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:24:50.729855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.334195+0000","last_change":"2023-10-10T20:34:34.344822+0000","last_active":"2023-10-10T20:34:35.334195+0000","last_peered":"2023-10-10T20:34:35.334195+0000","last_clean":"2023-10-10T20:34:35.334195+0000","last_became_active":"2023-10-10T20:34:34.344549+0000","last_became_peered":"2023-10-10T20:34:34.344549+0000","last_unstale":"2023-10-10T20:34:35.334195+0000","last_undegraded":"2023-10-10T20:34:35.334195+0000","last_fullsized":"2023-10-10T20:34:35.334195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.344601+0000","last_change":"2023-10-10T20:34:35.339793+0000","last_active":"2023-10-10T20:34:35.344601+0000","last_peered":"2023-10-10T20:34:35.344601+0000","last_clean":"2023-10-10T20:34:35.344601+0000","last_became_active":"2023-10-10T20:34:35.339226+0000","last_became_peered":"2023-10-10T20:34:35.339226+0000","last_unstale":"2023-10-10T20:34:35.344601+0000","last_undegraded":"2023-10-10T20:34:35.344601+0000","last_fullsized":"2023-10-10T20:34:35.344601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:10:13.711031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.360889+0000","last_change":"2023-10-10T20:34:34.336051+0000","last_active":"2023-10-10T20:34:35.360889+0000","last_peered":"2023-10-10T20:34:35.360889+0000","last_clean":"2023-10-10T20:34:35.360889+0000","last_became_active":"2023-10-10T20:34:34.335689+0000","last_became_peered":"2023-10-10T20:34:34.335689+0000","last_unstale":"2023-10-10T20:34:35.360889+0000","last_undegraded":"2023-10-10T20:34:35.360889+0000","last_fullsized":"2023-10-10T20:34:35.360889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:11.616929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350147+0000","last_change":"2023-10-10T20:34:35.341270+0000","last_active":"2023-10-10T20:34:35.350147+0000","last_peered":"2023-10-10T20:34:35.350147+0000","last_clean":"2023-10-10T20:34:35.350147+0000","last_became_active":"2023-10-10T20:34:35.341024+0000","last_became_peered":"2023-10-10T20:34:35.341024+0000","last_unstale":"2023-10-10T20:34:35.350147+0000","last_undegraded":"2023-10-10T20:34:35.350147+0000","last_fullsized":"2023-10-10T20:34:35.350147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:16.035453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.367035+0000","last_change":"2023-10-10T20:34:34.346412+0000","last_active":"2023-10-10T20:34:35.367035+0000","last_peered":"2023-10-10T20:34:35.367035+0000","last_clean":"2023-10-10T20:34:35.367035+0000","last_became_active":"2023-10-10T20:34:34.346020+0000","last_became_peered":"2023-10-10T20:34:34.346020+0000","last_unstale":"2023-10-10T20:34:35.367035+0000","last_undegraded":"2023-10-10T20:34:35.367035+0000","last_fullsized":"2023-10-10T20:34:35.367035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:15.030808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.345530+0000","last_change":"2023-10-10T20:34:35.340255+0000","last_active":"2023-10-10T20:34:35.345530+0000","last_peered":"2023-10-10T20:34:35.345530+0000","last_clean":"2023-10-10T20:34:35.345530+0000","last_became_active":"2023-10-10T20:34:35.339971+0000","last_became_peered":"2023-10-10T20:34:35.339971+0000","last_unstale":"2023-10-10T20:34:35.345530+0000","last_undegraded":"2023-10-10T20:34:35.345530+0000","last_fullsized":"2023-10-10T20:34:35.345530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:44:40.877955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362334+0000","last_change":"2023-10-10T20:34:34.343026+0000","last_active":"2023-10-10T20:34:35.362334+0000","last_peered":"2023-10-10T20:34:35.362334+0000","last_clean":"2023-10-10T20:34:35.362334+0000","last_became_active":"2023-10-10T20:34:34.342742+0000","last_became_peered":"2023-10-10T20:34:34.342742+0000","last_unstale":"2023-10-10T20:34:35.362334+0000","last_undegraded":"2023-10-10T20:34:35.362334+0000","last_fullsized":"2023-10-10T20:34:35.362334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:57:27.930567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348647+0000","last_change":"2023-10-10T20:34:35.342407+0000","last_active":"2023-10-10T20:34:35.348647+0000","last_peered":"2023-10-10T20:34:35.348647+0000","last_clean":"2023-10-10T20:34:35.348647+0000","last_became_active":"2023-10-10T20:34:35.341971+0000","last_became_peered":"2023-10-10T20:34:35.341971+0000","last_unstale":"2023-10-10T20:34:35.348647+0000","last_undegraded":"2023-10-10T20:34:35.348647+0000","last_fullsized":"2023-10-10T20:34:35.348647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:27:10.934315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.336796+0000","last_change":"2023-10-10T20:34:34.339893+0000","last_active":"2023-10-10T20:34:35.336796+0000","last_peered":"2023-10-10T20:34:35.336796+0000","last_clean":"2023-10-10T20:34:35.336796+0000","last_became_active":"2023-10-10T20:34:34.339585+0000","last_became_peered":"2023-10-10T20:34:34.339585+0000","last_unstale":"2023-10-10T20:34:35.336796+0000","last_undegraded":"2023-10-10T20:34:35.336796+0000","last_fullsized":"2023-10-10T20:34:35.336796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:15.030808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.346782+0000","last_change":"2023-10-10T20:34:35.341978+0000","last_active":"2023-10-10T20:34:35.346782+0000","last_peered":"2023-10-10T20:34:35.346782+0000","last_clean":"2023-10-10T20:34:35.346782+0000","last_became_active":"2023-10-10T20:34:35.341703+0000","last_became_peered":"2023-10-10T20:34:35.341703+0000","last_unstale":"2023-10-10T20:34:35.346782+0000","last_undegraded":"2023-10-10T20:34:35.346782+0000","last_fullsized":"2023-10-10T20:34:35.346782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:38:40.522047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.335441+0000","last_change":"2023-10-10T20:34:34.350137+0000","last_active":"2023-10-10T20:34:35.335441+0000","last_peered":"2023-10-10T20:34:35.335441+0000","last_clean":"2023-10-10T20:34:35.335441+0000","last_became_active":"2023-10-10T20:34:34.349855+0000","last_became_peered":"2023-10-10T20:34:34.349855+0000","last_unstale":"2023-10-10T20:34:35.335441+0000","last_undegraded":"2023-10-10T20:34:35.335441+0000","last_fullsized":"2023-10-10T20:34:35.335441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.347963+0000","last_change":"2023-10-10T20:34:35.344256+0000","last_active":"2023-10-10T20:34:35.347963+0000","last_peered":"2023-10-10T20:34:35.347963+0000","last_clean":"2023-10-10T20:34:35.347963+0000","last_became_active":"2023-10-10T20:34:35.343626+0000","last_became_peered":"2023-10-10T20:34:35.343626+0000","last_unstale":"2023-10-10T20:34:35.347963+0000","last_undegraded":"2023-10-10T20:34:35.347963+0000","last_fullsized":"2023-10-10T20:34:35.347963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:12:13.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.361414+0000","last_change":"2023-10-10T20:34:34.337912+0000","last_active":"2023-10-10T20:34:35.361414+0000","last_peered":"2023-10-10T20:34:35.361414+0000","last_clean":"2023-10-10T20:34:35.361414+0000","last_became_active":"2023-10-10T20:34:34.337588+0000","last_became_peered":"2023-10-10T20:34:34.337588+0000","last_unstale":"2023-10-10T20:34:35.361414+0000","last_undegraded":"2023-10-10T20:34:35.361414+0000","last_fullsized":"2023-10-10T20:34:35.361414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:40:03.331998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342457+0000","last_change":"2023-10-10T20:34:35.337938+0000","last_active":"2023-10-10T20:34:35.342457+0000","last_peered":"2023-10-10T20:34:35.342457+0000","last_clean":"2023-10-10T20:34:35.342457+0000","last_became_active":"2023-10-10T20:34:35.337649+0000","last_became_peered":"2023-10-10T20:34:35.337649+0000","last_unstale":"2023-10-10T20:34:35.342457+0000","last_undegraded":"2023-10-10T20:34:35.342457+0000","last_fullsized":"2023-10-10T20:34:35.342457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362619+0000","last_change":"2023-10-10T20:34:34.344045+0000","last_active":"2023-10-10T20:34:35.362619+0000","last_peered":"2023-10-10T20:34:35.362619+0000","last_clean":"2023-10-10T20:34:35.362619+0000","last_became_active":"2023-10-10T20:34:34.343736+0000","last_became_peered":"2023-10-10T20:34:34.343736+0000","last_unstale":"2023-10-10T20:34:35.362619+0000","last_undegraded":"2023-10-10T20:34:35.362619+0000","last_fullsized":"2023-10-10T20:34:35.362619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:57:27.930567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350691+0000","last_change":"2023-10-10T20:34:35.343035+0000","last_active":"2023-10-10T20:34:35.350691+0000","last_peered":"2023-10-10T20:34:35.350691+0000","last_clean":"2023-10-10T20:34:35.350691+0000","last_became_active":"2023-10-10T20:34:35.342776+0000","last_became_peered":"2023-10-10T20:34:35.342776+0000","last_unstale":"2023-10-10T20:34:35.350691+0000","last_undegraded":"2023-10-10T20:34:35.350691+0000","last_fullsized":"2023-10-10T20:34:35.350691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:52:03.361718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.337288+0000","last_change":"2023-10-10T20:34:34.337961+0000","last_active":"2023-10-10T20:34:35.337288+0000","last_peered":"2023-10-10T20:34:35.337288+0000","last_clean":"2023-10-10T20:34:35.337288+0000","last_became_active":"2023-10-10T20:34:34.337627+0000","last_became_peered":"2023-10-10T20:34:34.337627+0000","last_unstale":"2023-10-10T20:34:35.337288+0000","last_undegraded":"2023-10-10T20:34:35.337288+0000","last_fullsized":"2023-10-10T20:34:35.337288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:34:00.263416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349588+0000","last_change":"2023-10-10T20:34:35.346121+0000","last_active":"2023-10-10T20:34:35.349588+0000","last_peered":"2023-10-10T20:34:35.349588+0000","last_clean":"2023-10-10T20:34:35.349588+0000","last_became_active":"2023-10-10T20:34:35.345847+0000","last_became_peered":"2023-10-10T20:34:35.345847+0000","last_unstale":"2023-10-10T20:34:35.349588+0000","last_undegraded":"2023-10-10T20:34:35.349588+0000","last_fullsized":"2023-10-10T20:34:35.349588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:10.218128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.336695+0000","last_change":"2023-10-10T20:34:34.336680+0000","last_active":"2023-10-10T20:34:35.336695+0000","last_peered":"2023-10-10T20:34:35.336695+0000","last_clean":"2023-10-10T20:34:35.336695+0000","last_became_active":"2023-10-10T20:34:34.335533+0000","last_became_peered":"2023-10-10T20:34:34.335533+0000","last_unstale":"2023-10-10T20:34:35.336695+0000","last_undegraded":"2023-10-10T20:34:35.336695+0000","last_fullsized":"2023-10-10T20:34:35.336695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349524+0000","last_change":"2023-10-10T20:34:35.342266+0000","last_active":"2023-10-10T20:34:35.349524+0000","last_peered":"2023-10-10T20:34:35.349524+0000","last_clean":"2023-10-10T20:34:35.349524+0000","last_became_active":"2023-10-10T20:34:35.341981+0000","last_became_peered":"2023-10-10T20:34:35.341981+0000","last_unstale":"2023-10-10T20:34:35.349524+0000","last_undegraded":"2023-10-10T20:34:35.349524+0000","last_fullsized":"2023-10-10T20:34:35.349524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:11.206264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.345007+0000","last_change":"2023-10-10T20:34:35.341780+0000","last_active":"2023-10-10T20:34:35.345007+0000","last_peered":"2023-10-10T20:34:35.345007+0000","last_clean":"2023-10-10T20:34:35.345007+0000","last_became_active":"2023-10-10T20:34:35.340539+0000","last_became_peered":"2023-10-10T20:34:35.340539+0000","last_unstale":"2023-10-10T20:34:35.345007+0000","last_undegraded":"2023-10-10T20:34:35.345007+0000","last_fullsized":"2023-10-10T20:34:35.345007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:12:13.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.328070+0000","last_change":"2023-10-10T20:34:34.340247+0000","last_active":"2023-10-10T20:34:35.328070+0000","last_peered":"2023-10-10T20:34:35.328070+0000","last_clean":"2023-10-10T20:34:35.328070+0000","last_became_active":"2023-10-10T20:34:34.339946+0000","last_became_peered":"2023-10-10T20:34:34.339946+0000","last_unstale":"2023-10-10T20:34:35.328070+0000","last_undegraded":"2023-10-10T20:34:35.328070+0000","last_fullsized":"2023-10-10T20:34:35.328070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350545+0000","last_change":"2023-10-10T20:34:35.344288+0000","last_active":"2023-10-10T20:34:35.350545+0000","last_peered":"2023-10-10T20:34:35.350545+0000","last_clean":"2023-10-10T20:34:35.350545+0000","last_became_active":"2023-10-10T20:34:35.343736+0000","last_became_peered":"2023-10-10T20:34:35.343736+0000","last_unstale":"2023-10-10T20:34:35.350545+0000","last_undegraded":"2023-10-10T20:34:35.350545+0000","last_fullsized":"2023-10-10T20:34:35.350545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:18.709140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.333336+0000","last_change":"2023-10-10T20:34:34.341870+0000","last_active":"2023-10-10T20:34:35.333336+0000","last_peered":"2023-10-10T20:34:35.333336+0000","last_clean":"2023-10-10T20:34:35.333336+0000","last_became_active":"2023-10-10T20:34:34.341567+0000","last_became_peered":"2023-10-10T20:34:34.341567+0000","last_unstale":"2023-10-10T20:34:35.333336+0000","last_undegraded":"2023-10-10T20:34:35.333336+0000","last_fullsized":"2023-10-10T20:34:35.333336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:11.616929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.327481+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:34:35.327481+0000","last_peered":"2023-10-10T20:34:35.327481+0000","last_clean":"2023-10-10T20:34:35.327481+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:34:35.327481+0000","last_undegraded":"2023-10-10T20:34:35.327481+0000","last_fullsized":"2023-10-10T20:34:35.327481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.344647+0000","last_change":"2023-10-10T20:34:35.340986+0000","last_active":"2023-10-10T20:34:35.344647+0000","last_peered":"2023-10-10T20:34:35.344647+0000","last_clean":"2023-10-10T20:34:35.344647+0000","last_became_active":"2023-10-10T20:34:35.339812+0000","last_became_peered":"2023-10-10T20:34:35.339812+0000","last_unstale":"2023-10-10T20:34:35.344647+0000","last_undegraded":"2023-10-10T20:34:35.344647+0000","last_fullsized":"2023-10-10T20:34:35.344647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:11:35.427062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.327681+0000","last_change":"2023-10-10T20:34:34.345132+0000","last_active":"2023-10-10T20:34:35.327681+0000","last_peered":"2023-10-10T20:34:35.327681+0000","last_clean":"2023-10-10T20:34:35.327681+0000","last_became_active":"2023-10-10T20:34:34.344843+0000","last_became_peered":"2023-10-10T20:34:34.344843+0000","last_unstale":"2023-10-10T20:34:35.327681+0000","last_undegraded":"2023-10-10T20:34:35.327681+0000","last_fullsized":"2023-10-10T20:34:35.327681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:15.030808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.328545+0000","last_change":"2023-10-10T20:34:34.346151+0000","last_active":"2023-10-10T20:34:35.328545+0000","last_peered":"2023-10-10T20:34:35.328545+0000","last_clean":"2023-10-10T20:34:35.328545+0000","last_became_active":"2023-10-10T20:34:34.345765+0000","last_became_peered":"2023-10-10T20:34:34.345765+0000","last_unstale":"2023-10-10T20:34:35.328545+0000","last_undegraded":"2023-10-10T20:34:35.328545+0000","last_fullsized":"2023-10-10T20:34:35.328545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:10.201619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342808+0000","last_change":"2023-10-10T20:34:35.340238+0000","last_active":"2023-10-10T20:34:35.342808+0000","last_peered":"2023-10-10T20:34:35.342808+0000","last_clean":"2023-10-10T20:34:35.342808+0000","last_became_active":"2023-10-10T20:34:35.339944+0000","last_became_peered":"2023-10-10T20:34:35.339944+0000","last_unstale":"2023-10-10T20:34:35.342808+0000","last_undegraded":"2023-10-10T20:34:35.342808+0000","last_fullsized":"2023-10-10T20:34:35.342808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-10T20:34:36.526880+0000","last_change":"2023-10-10T20:34:34.347897+0000","last_active":"2023-10-10T20:34:36.526880+0000","last_peered":"2023-10-10T20:34:36.526880+0000","last_clean":"2023-10-10T20:34:36.526880+0000","last_became_active":"2023-10-10T20:34:34.347575+0000","last_became_peered":"2023-10-10T20:34:34.347575+0000","last_unstale":"2023-10-10T20:34:36.526880+0000","last_undegraded":"2023-10-10T20:34:36.526880+0000","last_fullsized":"2023-10-10T20:34:36.526880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:15.030808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350573+0000","last_change":"2023-10-10T20:34:35.345207+0000","last_active":"2023-10-10T20:34:35.350573+0000","last_peered":"2023-10-10T20:34:35.350573+0000","last_clean":"2023-10-10T20:34:35.350573+0000","last_became_active":"2023-10-10T20:34:35.344809+0000","last_became_peered":"2023-10-10T20:34:35.344809+0000","last_unstale":"2023-10-10T20:34:35.350573+0000","last_undegraded":"2023-10-10T20:34:35.350573+0000","last_fullsized":"2023-10-10T20:34:35.350573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:36.295226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.358045+0000","last_change":"2023-10-10T20:34:34.348399+0000","last_active":"2023-10-10T20:34:35.358045+0000","last_peered":"2023-10-10T20:34:35.358045+0000","last_clean":"2023-10-10T20:34:35.358045+0000","last_became_active":"2023-10-10T20:34:34.348099+0000","last_became_peered":"2023-10-10T20:34:34.348099+0000","last_unstale":"2023-10-10T20:34:35.358045+0000","last_undegraded":"2023-10-10T20:34:35.358045+0000","last_fullsized":"2023-10-10T20:34:35.358045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:40:03.331998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.344533+0000","last_change":"2023-10-10T20:34:35.340459+0000","last_active":"2023-10-10T20:34:35.344533+0000","last_peered":"2023-10-10T20:34:35.344533+0000","last_clean":"2023-10-10T20:34:35.344533+0000","last_became_active":"2023-10-10T20:34:35.340176+0000","last_became_peered":"2023-10-10T20:34:35.340176+0000","last_unstale":"2023-10-10T20:34:35.344533+0000","last_undegraded":"2023-10-10T20:34:35.344533+0000","last_fullsized":"2023-10-10T20:34:35.344533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:48:12.831828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.327499+0000","last_change":"2023-10-10T20:34:34.346150+0000","last_active":"2023-10-10T20:34:35.327499+0000","last_peered":"2023-10-10T20:34:35.327499+0000","last_clean":"2023-10-10T20:34:35.327499+0000","last_became_active":"2023-10-10T20:34:34.345811+0000","last_became_peered":"2023-10-10T20:34:34.345811+0000","last_unstale":"2023-10-10T20:34:35.327499+0000","last_undegraded":"2023-10-10T20:34:35.327499+0000","last_fullsized":"2023-10-10T20:34:35.327499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:04.665843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350476+0000","last_change":"2023-10-10T20:34:35.345083+0000","last_active":"2023-10-10T20:34:35.350476+0000","last_peered":"2023-10-10T20:34:35.350476+0000","last_clean":"2023-10-10T20:34:35.350476+0000","last_became_active":"2023-10-10T20:34:35.343922+0000","last_became_peered":"2023-10-10T20:34:35.343922+0000","last_unstale":"2023-10-10T20:34:35.350476+0000","last_undegraded":"2023-10-10T20:34:35.350476+0000","last_fullsized":"2023-10-10T20:34:35.350476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:05.670488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.337189+0000","last_change":"2023-10-10T20:34:34.340327+0000","last_active":"2023-10-10T20:34:35.337189+0000","last_peered":"2023-10-10T20:34:35.337189+0000","last_clean":"2023-10-10T20:34:35.337189+0000","last_became_active":"2023-10-10T20:34:34.339737+0000","last_became_peered":"2023-10-10T20:34:34.339737+0000","last_unstale":"2023-10-10T20:34:35.337189+0000","last_undegraded":"2023-10-10T20:34:35.337189+0000","last_fullsized":"2023-10-10T20:34:35.337189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350500+0000","last_change":"2023-10-10T20:34:35.344039+0000","last_active":"2023-10-10T20:34:35.350500+0000","last_peered":"2023-10-10T20:34:35.350500+0000","last_clean":"2023-10-10T20:34:35.350500+0000","last_became_active":"2023-10-10T20:34:35.343789+0000","last_became_peered":"2023-10-10T20:34:35.343789+0000","last_unstale":"2023-10-10T20:34:35.350500+0000","last_undegraded":"2023-10-10T20:34:35.350500+0000","last_fullsized":"2023-10-10T20:34:35.350500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:11:35.427062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.337192+0000","last_change":"2023-10-10T20:34:34.340022+0000","last_active":"2023-10-10T20:34:35.337192+0000","last_peered":"2023-10-10T20:34:35.337192+0000","last_clean":"2023-10-10T20:34:35.337192+0000","last_became_active":"2023-10-10T20:34:34.339652+0000","last_became_peered":"2023-10-10T20:34:34.339652+0000","last_unstale":"2023-10-10T20:34:35.337192+0000","last_undegraded":"2023-10-10T20:34:35.337192+0000","last_fullsized":"2023-10-10T20:34:35.337192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:00:22.187109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350304+0000","last_change":"2023-10-10T20:34:35.342858+0000","last_active":"2023-10-10T20:34:35.350304+0000","last_peered":"2023-10-10T20:34:35.350304+0000","last_clean":"2023-10-10T20:34:35.350304+0000","last_became_active":"2023-10-10T20:34:35.341911+0000","last_became_peered":"2023-10-10T20:34:35.341911+0000","last_unstale":"2023-10-10T20:34:35.350304+0000","last_undegraded":"2023-10-10T20:34:35.350304+0000","last_fullsized":"2023-10-10T20:34:35.350304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:25:55.933179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.357218+0000","last_change":"2023-10-10T20:34:34.346661+0000","last_active":"2023-10-10T20:34:35.357218+0000","last_peered":"2023-10-10T20:34:35.357218+0000","last_clean":"2023-10-10T20:34:35.357218+0000","last_became_active":"2023-10-10T20:34:34.346384+0000","last_became_peered":"2023-10-10T20:34:34.346384+0000","last_unstale":"2023-10-10T20:34:35.357218+0000","last_undegraded":"2023-10-10T20:34:35.357218+0000","last_fullsized":"2023-10-10T20:34:35.357218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350769+0000","last_change":"2023-10-10T20:34:35.349300+0000","last_active":"2023-10-10T20:34:35.350769+0000","last_peered":"2023-10-10T20:34:35.350769+0000","last_clean":"2023-10-10T20:34:35.350769+0000","last_became_active":"2023-10-10T20:34:35.348997+0000","last_became_peered":"2023-10-10T20:34:35.348997+0000","last_unstale":"2023-10-10T20:34:35.350769+0000","last_undegraded":"2023-10-10T20:34:35.350769+0000","last_fullsized":"2023-10-10T20:34:35.350769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:47:45.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.328113+0000","last_change":"2023-10-10T20:34:34.344048+0000","last_active":"2023-10-10T20:34:35.328113+0000","last_peered":"2023-10-10T20:34:35.328113+0000","last_clean":"2023-10-10T20:34:35.328113+0000","last_became_active":"2023-10-10T20:34:34.343866+0000","last_became_peered":"2023-10-10T20:34:34.343866+0000","last_unstale":"2023-10-10T20:34:35.328113+0000","last_undegraded":"2023-10-10T20:34:35.328113+0000","last_fullsized":"2023-10-10T20:34:35.328113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:27:04.988955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342399+0000","last_change":"2023-10-10T20:34:35.337534+0000","last_active":"2023-10-10T20:34:35.342399+0000","last_peered":"2023-10-10T20:34:35.342399+0000","last_clean":"2023-10-10T20:34:35.342399+0000","last_became_active":"2023-10-10T20:34:35.337277+0000","last_became_peered":"2023-10-10T20:34:35.337277+0000","last_unstale":"2023-10-10T20:34:35.342399+0000","last_undegraded":"2023-10-10T20:34:35.342399+0000","last_fullsized":"2023-10-10T20:34:35.342399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:42:23.450966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.360813+0000","last_change":"2023-10-10T20:34:34.340837+0000","last_active":"2023-10-10T20:34:35.360813+0000","last_peered":"2023-10-10T20:34:35.360813+0000","last_clean":"2023-10-10T20:34:35.360813+0000","last_became_active":"2023-10-10T20:34:34.338971+0000","last_became_peered":"2023-10-10T20:34:34.338971+0000","last_unstale":"2023-10-10T20:34:35.360813+0000","last_undegraded":"2023-10-10T20:34:35.360813+0000","last_fullsized":"2023-10-10T20:34:35.360813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:10.201619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349519+0000","last_change":"2023-10-10T20:34:35.345474+0000","last_active":"2023-10-10T20:34:35.349519+0000","last_peered":"2023-10-10T20:34:35.349519+0000","last_clean":"2023-10-10T20:34:35.349519+0000","last_became_active":"2023-10-10T20:34:35.343255+0000","last_became_peered":"2023-10-10T20:34:35.343255+0000","last_unstale":"2023-10-10T20:34:35.349519+0000","last_undegraded":"2023-10-10T20:34:35.349519+0000","last_fullsized":"2023-10-10T20:34:35.349519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:35.499006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.362947+0000","last_change":"2023-10-10T20:34:34.339953+0000","last_active":"2023-10-10T20:34:35.362947+0000","last_peered":"2023-10-10T20:34:35.362947+0000","last_clean":"2023-10-10T20:34:35.362947+0000","last_became_active":"2023-10-10T20:34:34.339676+0000","last_became_peered":"2023-10-10T20:34:34.339676+0000","last_unstale":"2023-10-10T20:34:35.362947+0000","last_undegraded":"2023-10-10T20:34:35.362947+0000","last_fullsized":"2023-10-10T20:34:35.362947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:33.309515+0000","last_clean_scrub_stamp":"2023-10-10T20:34:33.309515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:16.770254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342687+0000","last_change":"2023-10-10T20:34:35.338535+0000","last_active":"2023-10-10T20:34:35.342687+0000","last_peered":"2023-10-10T20:34:35.342687+0000","last_clean":"2023-10-10T20:34:35.342687+0000","last_became_active":"2023-10-10T20:34:35.338266+0000","last_became_peered":"2023-10-10T20:34:35.338266+0000","last_unstale":"2023-10-10T20:34:35.342687+0000","last_undegraded":"2023-10-10T20:34:35.342687+0000","last_fullsized":"2023-10-10T20:34:35.342687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:39.183060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348224+0000","last_change":"2023-10-10T20:34:35.346165+0000","last_active":"2023-10-10T20:34:35.348224+0000","last_peered":"2023-10-10T20:34:35.348224+0000","last_clean":"2023-10-10T20:34:35.348224+0000","last_became_active":"2023-10-10T20:34:35.345862+0000","last_became_peered":"2023-10-10T20:34:35.345862+0000","last_unstale":"2023-10-10T20:34:35.348224+0000","last_undegraded":"2023-10-10T20:34:35.348224+0000","last_fullsized":"2023-10-10T20:34:35.348224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:18.709140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350401+0000","last_change":"2023-10-10T20:34:35.342709+0000","last_active":"2023-10-10T20:34:35.350401+0000","last_peered":"2023-10-10T20:34:35.350401+0000","last_clean":"2023-10-10T20:34:35.350401+0000","last_became_active":"2023-10-10T20:34:35.342433+0000","last_became_peered":"2023-10-10T20:34:35.342433+0000","last_unstale":"2023-10-10T20:34:35.350401+0000","last_undegraded":"2023-10-10T20:34:35.350401+0000","last_fullsized":"2023-10-10T20:34:35.350401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:48:12.831828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342070+0000","last_change":"2023-10-10T20:34:35.340426+0000","last_active":"2023-10-10T20:34:35.342070+0000","last_peered":"2023-10-10T20:34:35.342070+0000","last_clean":"2023-10-10T20:34:35.342070+0000","last_became_active":"2023-10-10T20:34:35.339966+0000","last_became_peered":"2023-10-10T20:34:35.339966+0000","last_unstale":"2023-10-10T20:34:35.342070+0000","last_undegraded":"2023-10-10T20:34:35.342070+0000","last_fullsized":"2023-10-10T20:34:35.342070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:42:23.450966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.351149+0000","last_change":"2023-10-10T20:34:35.341444+0000","last_active":"2023-10-10T20:34:35.351149+0000","last_peered":"2023-10-10T20:34:35.351149+0000","last_clean":"2023-10-10T20:34:35.351149+0000","last_became_active":"2023-10-10T20:34:35.341186+0000","last_became_peered":"2023-10-10T20:34:35.341186+0000","last_unstale":"2023-10-10T20:34:35.351149+0000","last_undegraded":"2023-10-10T20:34:35.351149+0000","last_fullsized":"2023-10-10T20:34:35.351149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352359+0000","last_change":"2023-10-10T20:34:35.347375+0000","last_active":"2023-10-10T20:34:35.352359+0000","last_peered":"2023-10-10T20:34:35.352359+0000","last_clean":"2023-10-10T20:34:35.352359+0000","last_became_active":"2023-10-10T20:34:35.347094+0000","last_became_peered":"2023-10-10T20:34:35.347094+0000","last_unstale":"2023-10-10T20:34:35.352359+0000","last_undegraded":"2023-10-10T20:34:35.352359+0000","last_fullsized":"2023-10-10T20:34:35.352359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:12:13.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.342757+0000","last_change":"2023-10-10T20:34:35.340918+0000","last_active":"2023-10-10T20:34:35.342757+0000","last_peered":"2023-10-10T20:34:35.342757+0000","last_clean":"2023-10-10T20:34:35.342757+0000","last_became_active":"2023-10-10T20:34:35.340638+0000","last_became_peered":"2023-10-10T20:34:35.340638+0000","last_unstale":"2023-10-10T20:34:35.342757+0000","last_undegraded":"2023-10-10T20:34:35.342757+0000","last_fullsized":"2023-10-10T20:34:35.342757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:18.709140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.348059+0000","last_change":"2023-10-10T20:34:35.345371+0000","last_active":"2023-10-10T20:34:35.348059+0000","last_peered":"2023-10-10T20:34:35.348059+0000","last_clean":"2023-10-10T20:34:35.348059+0000","last_became_active":"2023-10-10T20:34:35.345058+0000","last_became_peered":"2023-10-10T20:34:35.345058+0000","last_unstale":"2023-10-10T20:34:35.348059+0000","last_undegraded":"2023-10-10T20:34:35.348059+0000","last_fullsized":"2023-10-10T20:34:35.348059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:26:54.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352396+0000","last_change":"2023-10-10T20:34:35.349283+0000","last_active":"2023-10-10T20:34:35.352396+0000","last_peered":"2023-10-10T20:34:35.352396+0000","last_clean":"2023-10-10T20:34:35.352396+0000","last_became_active":"2023-10-10T20:34:35.349040+0000","last_became_peered":"2023-10-10T20:34:35.349040+0000","last_unstale":"2023-10-10T20:34:35.352396+0000","last_undegraded":"2023-10-10T20:34:35.352396+0000","last_fullsized":"2023-10-10T20:34:35.352396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:47:45.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.349316+0000","last_change":"2023-10-10T20:34:35.343369+0000","last_active":"2023-10-10T20:34:35.349316+0000","last_peered":"2023-10-10T20:34:35.349316+0000","last_clean":"2023-10-10T20:34:35.349316+0000","last_became_active":"2023-10-10T20:34:35.342875+0000","last_became_peered":"2023-10-10T20:34:35.342875+0000","last_unstale":"2023-10-10T20:34:35.349316+0000","last_undegraded":"2023-10-10T20:34:35.349316+0000","last_fullsized":"2023-10-10T20:34:35.349316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:44:40.877955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.352369+0000","last_change":"2023-10-10T20:34:35.349284+0000","last_active":"2023-10-10T20:34:35.352369+0000","last_peered":"2023-10-10T20:34:35.352369+0000","last_clean":"2023-10-10T20:34:35.352369+0000","last_became_active":"2023-10-10T20:34:35.349057+0000","last_became_peered":"2023-10-10T20:34:35.349057+0000","last_unstale":"2023-10-10T20:34:35.352369+0000","last_undegraded":"2023-10-10T20:34:35.352369+0000","last_fullsized":"2023-10-10T20:34:35.352369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:52:03.361718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.354153+0000","last_change":"2023-10-10T20:34:35.353504+0000","last_active":"2023-10-10T20:34:35.354153+0000","last_peered":"2023-10-10T20:34:35.354153+0000","last_clean":"2023-10-10T20:34:35.354153+0000","last_became_active":"2023-10-10T20:34:35.350145+0000","last_became_peered":"2023-10-10T20:34:35.350145+0000","last_unstale":"2023-10-10T20:34:35.354153+0000","last_undegraded":"2023-10-10T20:34:35.354153+0000","last_fullsized":"2023-10-10T20:34:35.354153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:52:03.361718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.351192+0000","last_change":"2023-10-10T20:34:35.345734+0000","last_active":"2023-10-10T20:34:35.351192+0000","last_peered":"2023-10-10T20:34:35.351192+0000","last_clean":"2023-10-10T20:34:35.351192+0000","last_became_active":"2023-10-10T20:34:35.345466+0000","last_became_peered":"2023-10-10T20:34:35.345466+0000","last_unstale":"2023-10-10T20:34:35.351192+0000","last_undegraded":"2023-10-10T20:34:35.351192+0000","last_fullsized":"2023-10-10T20:34:35.351192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:47:45.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:34:35.350754+0000","last_change":"2023-10-10T20:34:35.349383+0000","last_active":"2023-10-10T20:34:35.350754+0000","last_peered":"2023-10-10T20:34:35.350754+0000","last_clean":"2023-10-10T20:34:35.350754+0000","last_became_active":"2023-10-10T20:34:35.348979+0000","last_became_peered":"2023-10-10T20:34:35.348979+0000","last_unstale":"2023-10-10T20:34:35.350754+0000","last_undegraded":"2023-10-10T20:34:35.350754+0000","last_fullsized":"2023-10-10T20:34:35.350754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:34.314160+0000","last_clean_scrub_stamp":"2023-10-10T20:34:34.314160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:25:55.933179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298072,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073768,"statfs":{"total":96636764160,"available":96331538432,"internally_reserved":0,"allocated":802816,"data_stored":661576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":212992,"data_stored":70814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.0380000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2869999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1159999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.0659999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298076,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073764,"statfs":{"total":96636764160,"available":96331534336,"internally_reserved":0,"allocated":806912,"data_stored":660671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":70654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":70780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":70744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:34:44.321 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-10T20:34:44.321 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-10T20:34:44.322 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-10T20:34:44.322 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-10T20:34:44.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.463+0000 7fc70ffff700 1 Processor -- start 2023-10-10T20:34:44.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.463+0000 7fc70ffff700 1 -- start start 2023-10-10T20:34:44.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.464+0000 7fc70ffff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc710159aa0 0x7fc7100cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:44.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.464+0000 7fc70ffff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7100cabc0 0x7fc710069b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:44.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.464+0000 7fc70ffff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7100cd1f0 con 0x7fc71006b2a0 2023-10-10T20:34:44.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.464+0000 7fc70ffff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7100ca130 con 0x7fc710159aa0 2023-10-10T20:34:44.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.464+0000 7fc70ffff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7100c9090 con 0x7fc7100cabc0 2023-10-10T20:34:44.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.464+0000 7fc70effd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc710159aa0 0x7fc7100cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:44.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.464+0000 7fc70f7fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7100cabc0 0x7fc710069b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:44.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.464+0000 7fc70e7fc700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc71006b2a0 0x7fc7100ca4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:49342/0 (socket says 172.21.15.89:49342) 2023-10-10T20:34:44.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.464+0000 7fc70e7fc700 1 -- 172.21.15.89:0/1264503524 learned_addr learned my addr 172.21.15.89:0/1264503524 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:44.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.465+0000 7fc70effd700 1 -- 172.21.15.89:0/1264503524 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7100cabc0 msgr2=0x7fc710069b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:44.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.465+0000 7fc70effd700 1 --2- 172.21.15.89:0/1264503524 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7100cabc0 0x7fc710069b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.465+0000 7fc70effd700 1 -- 172.21.15.89:0/1264503524 >> v1:172.21.15.111:6789/0 conn(0x7fc71006b2a0 legacy=0x7fc7100ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:44.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.465+0000 7fc70effd700 1 -- 172.21.15.89:0/1264503524 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7100c9370 con 0x7fc710159aa0 2023-10-10T20:34:44.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.465+0000 7fc70dffb700 1 -- 172.21.15.89:0/1264503524 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569966816 0 0) 0x7fc7100cd1f0 con 0x7fc71006b2a0 2023-10-10T20:34:44.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.465+0000 7fc70effd700 1 --2- 172.21.15.89:0/1264503524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc710159aa0 0x7fc7100cccb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc708009e60 tx=0x7fc708009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4573125175a44de5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:44.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.465+0000 7fc70dffb700 1 -- 172.21.15.89:0/1264503524 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70800e070 con 0x7fc710159aa0 2023-10-10T20:34:44.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.465+0000 7fc70dffb700 1 -- 172.21.15.89:0/1264503524 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70800bb60 con 0x7fc710159aa0 2023-10-10T20:34:44.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.465+0000 7fc70dffb700 1 -- 172.21.15.89:0/1264503524 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708012430 con 0x7fc710159aa0 2023-10-10T20:34:44.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.465+0000 7fc70ffff700 1 -- 172.21.15.89:0/1264503524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc710159aa0 msgr2=0x7fc7100cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:44.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.465+0000 7fc70ffff700 1 --2- 172.21.15.89:0/1264503524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc710159aa0 0x7fc7100cccb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc708009e60 tx=0x7fc708009780 comp rx=0 tx=0).stop 2023-10-10T20:34:44.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.465+0000 7fc70ffff700 1 -- 172.21.15.89:0/1264503524 shutdown_connections 2023-10-10T20:34:44.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.465+0000 7fc70ffff700 1 --2- 172.21.15.89:0/1264503524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc710159aa0 0x7fc7100cccb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.465+0000 7fc70ffff700 1 --2- 172.21.15.89:0/1264503524 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7100cabc0 0x7fc710069b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.465+0000 7fc70ffff700 1 -- 172.21.15.89:0/1264503524 >> 172.21.15.89:0/1264503524 conn(0x7fc71006baa0 msgr2=0x7fc71006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:44.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.465+0000 7fc70ffff700 1 -- 172.21.15.89:0/1264503524 shutdown_connections 2023-10-10T20:34:44.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.466+0000 7fc70ffff700 1 -- 172.21.15.89:0/1264503524 wait complete. 2023-10-10T20:34:44.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.467+0000 7fc70ffff700 1 Processor -- start 2023-10-10T20:34:44.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.467+0000 7fc70ffff700 1 -- start start 2023-10-10T20:34:44.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.467+0000 7fc70ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc71006b2a0 0x7fc710148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:44.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.467+0000 7fc70ffff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7100cabc0 0x7fc7100c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:44.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.467+0000 7fc70ffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc710159aa0 0x7fc7100c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:44.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.467+0000 7fc70ffff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7100b4440 con 0x7fc71006b2a0 2023-10-10T20:34:44.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.467+0000 7fc70ffff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc710148740 con 0x7fc710159aa0 2023-10-10T20:34:44.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.467+0000 7fc70ffff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7100b3e80 con 0x7fc7100cabc0 2023-10-10T20:34:44.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.468+0000 7fc70effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc71006b2a0 0x7fc710148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:44.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.468+0000 7fc70effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc71006b2a0 0x7fc710148200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53646/0 (socket says 172.21.15.89:53646) 2023-10-10T20:34:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.468+0000 7fc70effd700 1 -- 172.21.15.89:0/338366738 learned_addr learned my addr 172.21.15.89:0/338366738 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.468+0000 7fc70e7fc700 1 --2- 172.21.15.89:0/338366738 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7100cabc0 0x7fc7100c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.468+0000 7fc70f7fe700 1 --2- 172.21.15.89:0/338366738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc710159aa0 0x7fc7100c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.468+0000 7fc70effd700 1 -- 172.21.15.89:0/338366738 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7100cabc0 msgr2=0x7fc7100c36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.468+0000 7fc70effd700 1 --2- 172.21.15.89:0/338366738 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7100cabc0 0x7fc7100c36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.468+0000 7fc70effd700 1 -- 172.21.15.89:0/338366738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc710159aa0 msgr2=0x7fc7100c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:44.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.468+0000 7fc70effd700 1 --2- 172.21.15.89:0/338366738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc710159aa0 0x7fc7100c3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.468+0000 7fc70effd700 1 -- 172.21.15.89:0/338366738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc708009a20 con 0x7fc71006b2a0 2023-10-10T20:34:44.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.468+0000 7fc70effd700 1 --2- 172.21.15.89:0/338366738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc71006b2a0 0x7fc710148200 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc70800b6c0 tx=0x7fc708019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:44.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.468+0000 7fc6f6ffd700 1 -- 172.21.15.89:0/338366738 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708012e10 con 0x7fc71006b2a0 2023-10-10T20:34:44.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.468+0000 7fc6f6ffd700 1 -- 172.21.15.89:0/338366738 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc708027470 con 0x7fc71006b2a0 2023-10-10T20:34:44.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.468+0000 7fc6f6ffd700 1 -- 172.21.15.89:0/338366738 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708027830 con 0x7fc71006b2a0 2023-10-10T20:34:44.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.468+0000 7fc70ffff700 1 -- 172.21.15.89:0/338366738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7100b4160 con 0x7fc71006b2a0 2023-10-10T20:34:44.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.468+0000 7fc70ffff700 1 -- 172.21.15.89:0/338366738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71015a2f0 con 0x7fc71006b2a0 2023-10-10T20:34:44.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.469+0000 7fc70ffff700 1 -- 172.21.15.89:0/338366738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc710159e70 con 0x7fc71006b2a0 2023-10-10T20:34:44.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.469+0000 7fc6f6ffd700 1 -- 172.21.15.89:0/338366738 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc708024070 con 0x7fc71006b2a0 2023-10-10T20:34:44.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.470+0000 7fc6f6ffd700 1 --2- 172.21.15.89:0/338366738 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc6fc066f30 0x7fc6fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:44.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.470+0000 7fc6f6ffd700 1 -- 172.21.15.89:0/338366738 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc708086fb0 con 0x7fc71006b2a0 2023-10-10T20:34:44.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.470+0000 7fc70e7fc700 1 --2- 172.21.15.89:0/338366738 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc6fc066f30 0x7fc6fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:44.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.470+0000 7fc70e7fc700 1 --2- 172.21.15.89:0/338366738 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc6fc066f30 0x7fc6fc0693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc7100b2e50 tx=0x7fc704009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:44.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.474+0000 7fc6f6ffd700 1 -- 172.21.15.89:0/338366738 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7080b85a0 con 0x7fc71006b2a0 2023-10-10T20:34:44.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.699+0000 7fc70ffff700 1 -- 172.21.15.89:0/338366738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc7100c9650 con 0x7fc71006b2a0 2023-10-10T20:34:44.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.700+0000 7fc6f6ffd700 1 -- 172.21.15.89:0/338366738 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc710159e70 con 0x7fc71006b2a0 2023-10-10T20:34:44.703 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:44.703 INFO:teuthology.orchestra.run.smithi089.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-10T20:34:44.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.701+0000 7fc70ffff700 1 -- 172.21.15.89:0/338366738 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc6fc066f30 msgr2=0x7fc6fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:44.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.701+0000 7fc70ffff700 1 --2- 172.21.15.89:0/338366738 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc6fc066f30 0x7fc6fc0693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc7100b2e50 tx=0x7fc704009040 comp rx=0 tx=0).stop 2023-10-10T20:34:44.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.701+0000 7fc70ffff700 1 -- 172.21.15.89:0/338366738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc71006b2a0 msgr2=0x7fc710148200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:44.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.701+0000 7fc70ffff700 1 --2- 172.21.15.89:0/338366738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc71006b2a0 0x7fc710148200 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc70800b6c0 tx=0x7fc708019000 comp rx=0 tx=0).stop 2023-10-10T20:34:44.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.702+0000 7fc70ffff700 1 -- 172.21.15.89:0/338366738 shutdown_connections 2023-10-10T20:34:44.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.702+0000 7fc70ffff700 1 --2- 172.21.15.89:0/338366738 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc6fc066f30 0x7fc6fc0693e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.702+0000 7fc70ffff700 1 --2- 172.21.15.89:0/338366738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc710159aa0 0x7fc7100c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.702+0000 7fc70ffff700 1 --2- 172.21.15.89:0/338366738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc71006b2a0 0x7fc710148200 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.702+0000 7fc70ffff700 1 --2- 172.21.15.89:0/338366738 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7100cabc0 0x7fc7100c36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.702+0000 7fc70ffff700 1 -- 172.21.15.89:0/338366738 >> 172.21.15.89:0/338366738 conn(0x7fc71006baa0 msgr2=0x7fc71006d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:44.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.702+0000 7fc70ffff700 1 -- 172.21.15.89:0/338366738 shutdown_connections 2023-10-10T20:34:44.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.702+0000 7fc70ffff700 1 -- 172.21.15.89:0/338366738 wait complete. 2023-10-10T20:34:44.716 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-10T20:34:44.716 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:34:44.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.828+0000 7ff92bfff700 1 Processor -- start 2023-10-10T20:34:44.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.828+0000 7ff92bfff700 1 -- start start 2023-10-10T20:34:44.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.829+0000 7ff92bfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff92c159420 0x7ff92c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:44.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.829+0000 7ff92bfff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff92c1597f0 0x7ff92c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:44.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.829+0000 7ff92bfff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff92c0b11d0 con 0x7ff92c15d230 2023-10-10T20:34:44.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.829+0000 7ff92bfff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92c0b14b0 con 0x7ff92c159420 2023-10-10T20:34:44.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.829+0000 7ff92bfff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c0b1790 con 0x7ff92c1597f0 2023-10-10T20:34:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.829+0000 7ff9227fc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff92c1597f0 0x7ff92c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.829+0000 7ff92b7fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff92c15d230 0x7ff92c0b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:49350/0 (socket says 172.21.15.89:49350) 2023-10-10T20:34:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.829+0000 7ff92affd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff92c159420 0x7ff92c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.829+0000 7ff92b7fe700 1 -- 172.21.15.89:0/832032111 learned_addr learned my addr 172.21.15.89:0/832032111 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.829+0000 7ff92affd700 1 -- 172.21.15.89:0/832032111 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff92c1597f0 msgr2=0x7ff92c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.829+0000 7ff92affd700 1 --2- 172.21.15.89:0/832032111 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff92c1597f0 0x7ff92c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.829+0000 7ff92affd700 1 -- 172.21.15.89:0/832032111 >> v1:172.21.15.111:6789/0 conn(0x7ff92c15d230 legacy=0x7ff92c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.829+0000 7ff92affd700 1 -- 172.21.15.89:0/832032111 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92c0b1a70 con 0x7ff92c159420 2023-10-10T20:34:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.829+0000 7ff92a7fc700 1 -- 172.21.15.89:0/832032111 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561642023 0 0) 0x7ff92c0b11d0 con 0x7ff92c15d230 2023-10-10T20:34:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.830+0000 7ff92affd700 1 --2- 172.21.15.89:0/832032111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff92c159420 0x7ff92c0b5f40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0098e0 tx=0x7ff91c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b8970622a3efd00 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:44.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.830+0000 7ff92a7fc700 1 -- 172.21.15.89:0/832032111 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c00b640 con 0x7ff92c159420 2023-10-10T20:34:44.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.830+0000 7ff92a7fc700 1 -- 172.21.15.89:0/832032111 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91c00b7a0 con 0x7ff92c159420 2023-10-10T20:34:44.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.830+0000 7ff92a7fc700 1 -- 172.21.15.89:0/832032111 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c00bb40 con 0x7ff92c159420 2023-10-10T20:34:44.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.830+0000 7ff92bfff700 1 -- 172.21.15.89:0/832032111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff92c159420 msgr2=0x7ff92c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:44.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.830+0000 7ff92bfff700 1 --2- 172.21.15.89:0/832032111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff92c159420 0x7ff92c0b5f40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0098e0 tx=0x7ff91c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.830+0000 7ff92bfff700 1 -- 172.21.15.89:0/832032111 shutdown_connections 2023-10-10T20:34:44.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.830+0000 7ff92bfff700 1 --2- 172.21.15.89:0/832032111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff92c159420 0x7ff92c0b5f40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.830+0000 7ff92bfff700 1 --2- 172.21.15.89:0/832032111 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff92c1597f0 0x7ff92c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.830+0000 7ff92bfff700 1 -- 172.21.15.89:0/832032111 >> 172.21.15.89:0/832032111 conn(0x7ff92c0d0660 msgr2=0x7ff92c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:44.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.830+0000 7ff92bfff700 1 -- 172.21.15.89:0/832032111 shutdown_connections 2023-10-10T20:34:44.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.830+0000 7ff92bfff700 1 -- 172.21.15.89:0/832032111 wait complete. 2023-10-10T20:34:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.832+0000 7ff92bfff700 1 Processor -- start 2023-10-10T20:34:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.832+0000 7ff92bfff700 1 -- start start 2023-10-10T20:34:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.832+0000 7ff92bfff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff92c159420 0x7ff92c0c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.832+0000 7ff92bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff92c1597f0 0x7ff92c15c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.832+0000 7ff92bfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff92c15d230 0x7ff92c0c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.832+0000 7ff92bfff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c0ca910 con 0x7ff92c1597f0 2023-10-10T20:34:44.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.832+0000 7ff92bfff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c0c9a30 con 0x7ff92c15d230 2023-10-10T20:34:44.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.832+0000 7ff92bfff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c0c9d00 con 0x7ff92c159420 2023-10-10T20:34:44.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.832+0000 7ff9227fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff92c1597f0 0x7ff92c15c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:44.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.832+0000 7ff9227fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff92c1597f0 0x7ff92c15c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53652/0 (socket says 172.21.15.89:53652) 2023-10-10T20:34:44.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.832+0000 7ff9227fc700 1 -- 172.21.15.89:0/1616444086 learned_addr learned my addr 172.21.15.89:0/1616444086 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:44.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.833+0000 7ff92affd700 1 --2- 172.21.15.89:0/1616444086 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff92c159420 0x7ff92c0c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:44.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.833+0000 7ff92b7fe700 1 --2- 172.21.15.89:0/1616444086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff92c15d230 0x7ff92c0c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:44.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.833+0000 7ff9227fc700 1 -- 172.21.15.89:0/1616444086 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff92c159420 msgr2=0x7ff92c0c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:44.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.833+0000 7ff9227fc700 1 --2- 172.21.15.89:0/1616444086 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff92c159420 0x7ff92c0c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.833+0000 7ff9227fc700 1 -- 172.21.15.89:0/1616444086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff92c15d230 msgr2=0x7ff92c0c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:44.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.833+0000 7ff9227fc700 1 --2- 172.21.15.89:0/1616444086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff92c15d230 0x7ff92c0c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:44.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.833+0000 7ff9227fc700 1 -- 172.21.15.89:0/1616444086 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c009420 con 0x7ff92c1597f0 2023-10-10T20:34:44.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.833+0000 7ff92b7fe700 1 --2- 172.21.15.89:0/1616444086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff92c15d230 0x7ff92c0c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:44.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.833+0000 7ff9227fc700 1 --2- 172.21.15.89:0/1616444086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff92c1597f0 0x7ff92c15c580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff91400ec80 tx=0x7ff9140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:44.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.833+0000 7ff9237fe700 1 -- 172.21.15.89:0/1616444086 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91400c9d0 con 0x7ff92c1597f0 2023-10-10T20:34:44.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.833+0000 7ff9237fe700 1 -- 172.21.15.89:0/1616444086 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91400cb30 con 0x7ff92c1597f0 2023-10-10T20:34:44.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.833+0000 7ff9237fe700 1 -- 172.21.15.89:0/1616444086 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff914013470 con 0x7ff92c1597f0 2023-10-10T20:34:44.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.833+0000 7ff92bfff700 1 -- 172.21.15.89:0/1616444086 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92c0c9fe0 con 0x7ff92c1597f0 2023-10-10T20:34:44.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.833+0000 7ff92bfff700 1 -- 172.21.15.89:0/1616444086 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92c0ca5a0 con 0x7ff92c1597f0 2023-10-10T20:34:44.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.834+0000 7ff92bfff700 1 -- 172.21.15.89:0/1616444086 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92c04c3c0 con 0x7ff92c1597f0 2023-10-10T20:34:44.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.834+0000 7ff9237fe700 1 -- 172.21.15.89:0/1616444086 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff914013820 con 0x7ff92c1597f0 2023-10-10T20:34:44.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.835+0000 7ff9237fe700 1 --2- 172.21.15.89:0/1616444086 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff90c066fb0 0x7ff90c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:44.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.835+0000 7ff9237fe700 1 -- 172.21.15.89:0/1616444086 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff914084990 con 0x7ff92c1597f0 2023-10-10T20:34:44.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.835+0000 7ff92affd700 1 --2- 172.21.15.89:0/1616444086 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff90c066fb0 0x7ff90c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:44.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.835+0000 7ff92affd700 1 --2- 172.21.15.89:0/1616444086 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff90c066fb0 0x7ff90c069460 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0093f0 tx=0x7ff91c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:44.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:44.839+0000 7ff9237fe700 1 -- 172.21.15.89:0/1616444086 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff914051b60 con 0x7ff92c1597f0 2023-10-10T20:34:45.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.023+0000 7ff92bfff700 1 -- 172.21.15.89:0/1616444086 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff92c0caa50 con 0x7ff92c1597f0 2023-10-10T20:34:45.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.024+0000 7ff9237fe700 1 -- 172.21.15.89:0/1616444086 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7ff914055180 con 0x7ff92c1597f0 2023-10-10T20:34:45.027 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:45.027 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:36.324920+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:34:32.342167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:34:33.671763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:34:45.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.026+0000 7ff92bfff700 1 -- 172.21.15.89:0/1616444086 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff90c066fb0 msgr2=0x7ff90c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:45.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.026+0000 7ff92bfff700 1 --2- 172.21.15.89:0/1616444086 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff90c066fb0 0x7ff90c069460 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0093f0 tx=0x7ff91c017000 comp rx=0 tx=0).stop 2023-10-10T20:34:45.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.026+0000 7ff92bfff700 1 -- 172.21.15.89:0/1616444086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff92c1597f0 msgr2=0x7ff92c15c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:45.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.026+0000 7ff92bfff700 1 --2- 172.21.15.89:0/1616444086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff92c1597f0 0x7ff92c15c580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff91400ec80 tx=0x7ff9140091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.026+0000 7ff92bfff700 1 -- 172.21.15.89:0/1616444086 shutdown_connections 2023-10-10T20:34:45.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.026+0000 7ff92bfff700 1 --2- 172.21.15.89:0/1616444086 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff90c066fb0 0x7ff90c069460 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.026+0000 7ff92bfff700 1 --2- 172.21.15.89:0/1616444086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff92c15d230 0x7ff92c0c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.026+0000 7ff92bfff700 1 --2- 172.21.15.89:0/1616444086 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff92c159420 0x7ff92c0c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.026+0000 7ff92bfff700 1 --2- 172.21.15.89:0/1616444086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff92c1597f0 0x7ff92c15c580 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.026+0000 7ff92bfff700 1 -- 172.21.15.89:0/1616444086 >> 172.21.15.89:0/1616444086 conn(0x7ff92c0d0660 msgr2=0x7ff92c0b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:45.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.026+0000 7ff92bfff700 1 -- 172.21.15.89:0/1616444086 shutdown_connections 2023-10-10T20:34:45.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.026+0000 7ff92bfff700 1 -- 172.21.15.89:0/1616444086 wait complete. 2023-10-10T20:34:45.040 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:34:32.342167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:34:33.671763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:34:45.041 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:34:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.156+0000 7faa660c2700 1 Processor -- start 2023-10-10T20:34:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.156+0000 7faa660c2700 1 -- start start 2023-10-10T20:34:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.157+0000 7faa660c2700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faa60159c60 0x7faa6015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.157+0000 7faa660c2700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa6015db10 0x7faa60069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:45.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.157+0000 7faa660c2700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6015a160 con 0x7faa60159890 2023-10-10T20:34:45.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.157+0000 7faa660c2700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa6015df50 con 0x7faa60159c60 2023-10-10T20:34:45.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.157+0000 7faa660c2700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa600d13a0 con 0x7faa6015db10 2023-10-10T20:34:45.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.157+0000 7faa5ffff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa6015db10 0x7faa60069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:45.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.157+0000 7faa5f7fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7faa60159890 0x7faa6015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:49360/0 (socket says 172.21.15.89:49360) 2023-10-10T20:34:45.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.157+0000 7faa5effd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faa60159c60 0x7faa6015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:45.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.157+0000 7faa5f7fe700 1 -- 172.21.15.89:0/1832844478 learned_addr learned my addr 172.21.15.89:0/1832844478 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:45.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.158+0000 7faa5e7fc700 1 -- 172.21.15.89:0/1832844478 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1096142377 0 0) 0x7faa6015a160 con 0x7faa60159890 2023-10-10T20:34:45.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.158+0000 7faa5e7fc700 1 -- 172.21.15.89:0/1832844478 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa48003570 con 0x7faa60159890 2023-10-10T20:34:45.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.158+0000 7faa5effd700 1 -- 172.21.15.89:0/1832844478 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa6015db10 msgr2=0x7faa60069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:45.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.158+0000 7faa5effd700 1 --2- 172.21.15.89:0/1832844478 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa6015db10 0x7faa60069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.158+0000 7faa5effd700 1 -- 172.21.15.89:0/1832844478 >> v1:172.21.15.111:6789/0 conn(0x7faa60159890 legacy=0x7faa6015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:45.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.158+0000 7faa5effd700 1 -- 172.21.15.89:0/1832844478 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6015e140 con 0x7faa60159c60 2023-10-10T20:34:45.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.158+0000 7faa5effd700 1 --2- 172.21.15.89:0/1832844478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa60159c60 0x7faa6015d5d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faa540098e0 tx=0x7faa540091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b00ec596f364246e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:45.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.158+0000 7faa5e7fc700 1 -- 172.21.15.89:0/1832844478 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa54011630 con 0x7faa60159c60 2023-10-10T20:34:45.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.158+0000 7faa5e7fc700 1 -- 172.21.15.89:0/1832844478 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa54011790 con 0x7faa60159c60 2023-10-10T20:34:45.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.158+0000 7faa5e7fc700 1 -- 172.21.15.89:0/1832844478 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa54011b30 con 0x7faa60159c60 2023-10-10T20:34:45.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.158+0000 7faa660c2700 1 -- 172.21.15.89:0/1832844478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa60159c60 msgr2=0x7faa6015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:45.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.158+0000 7faa660c2700 1 --2- 172.21.15.89:0/1832844478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa60159c60 0x7faa6015d5d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faa540098e0 tx=0x7faa540091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.159+0000 7faa660c2700 1 -- 172.21.15.89:0/1832844478 shutdown_connections 2023-10-10T20:34:45.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.159+0000 7faa660c2700 1 --2- 172.21.15.89:0/1832844478 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa6015db10 0x7faa60069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.159+0000 7faa660c2700 1 --2- 172.21.15.89:0/1832844478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa60159c60 0x7faa6015d5d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.159+0000 7faa660c2700 1 -- 172.21.15.89:0/1832844478 >> 172.21.15.89:0/1832844478 conn(0x7faa60068e20 msgr2=0x7faa600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:45.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.159+0000 7faa660c2700 1 -- 172.21.15.89:0/1832844478 shutdown_connections 2023-10-10T20:34:45.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.159+0000 7faa660c2700 1 -- 172.21.15.89:0/1832844478 wait complete. 2023-10-10T20:34:45.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.160+0000 7faa660c2700 1 Processor -- start 2023-10-10T20:34:45.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.160+0000 7faa660c2700 1 -- start start 2023-10-10T20:34:45.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.160+0000 7faa660c2700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa60159890 0x7faa600ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:45.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.160+0000 7faa660c2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa60159c60 0x7faa600cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:45.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.160+0000 7faa660c2700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6015db10 0x7faa600cf720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:45.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.160+0000 7faa660c2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa600b3c00 con 0x7faa60159c60 2023-10-10T20:34:45.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.160+0000 7faa660c2700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa600b64c0 con 0x7faa6015db10 2023-10-10T20:34:45.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.160+0000 7faa660c2700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa600ce810 con 0x7faa60159890 2023-10-10T20:34:45.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.161+0000 7faa5effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa60159c60 0x7faa600cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:45.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.161+0000 7faa5effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa60159c60 0x7faa600cf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53660/0 (socket says 172.21.15.89:53660) 2023-10-10T20:34:45.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.161+0000 7faa5effd700 1 -- 172.21.15.89:0/1763871870 learned_addr learned my addr 172.21.15.89:0/1763871870 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:45.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.161+0000 7faa5f7fe700 1 --2- 172.21.15.89:0/1763871870 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa60159890 0x7faa600ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:45.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.161+0000 7faa5ffff700 1 --2- 172.21.15.89:0/1763871870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6015db10 0x7faa600cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:45.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.161+0000 7faa5effd700 1 -- 172.21.15.89:0/1763871870 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa60159890 msgr2=0x7faa600ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:45.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.161+0000 7faa5effd700 1 --2- 172.21.15.89:0/1763871870 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa60159890 0x7faa600ceca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.161+0000 7faa5effd700 1 -- 172.21.15.89:0/1763871870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6015db10 msgr2=0x7faa600cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:45.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.161+0000 7faa5effd700 1 --2- 172.21.15.89:0/1763871870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6015db10 0x7faa600cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.161+0000 7faa5effd700 1 -- 172.21.15.89:0/1763871870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa54009420 con 0x7faa60159c60 2023-10-10T20:34:45.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.161+0000 7faa5ffff700 1 --2- 172.21.15.89:0/1763871870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6015db10 0x7faa600cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:45.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.161+0000 7faa5effd700 1 --2- 172.21.15.89:0/1763871870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa60159c60 0x7faa600cf1e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faa600899c0 tx=0x7faa54017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:45.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.161+0000 7faa477fe700 1 -- 172.21.15.89:0/1763871870 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa54011b10 con 0x7faa60159c60 2023-10-10T20:34:45.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.161+0000 7faa477fe700 1 -- 172.21.15.89:0/1763871870 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa54000cc0 con 0x7faa60159c60 2023-10-10T20:34:45.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.161+0000 7faa660c2700 1 -- 172.21.15.89:0/1763871870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa600cdc00 con 0x7faa60159c60 2023-10-10T20:34:45.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.161+0000 7faa477fe700 1 -- 172.21.15.89:0/1763871870 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa54025620 con 0x7faa60159c60 2023-10-10T20:34:45.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.161+0000 7faa660c2700 1 -- 172.21.15.89:0/1763871870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa600ce1c0 con 0x7faa60159c60 2023-10-10T20:34:45.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.162+0000 7faa660c2700 1 -- 172.21.15.89:0/1763871870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa6004c3c0 con 0x7faa60159c60 2023-10-10T20:34:45.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.163+0000 7faa477fe700 1 -- 172.21.15.89:0/1763871870 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faa540259d0 con 0x7faa60159c60 2023-10-10T20:34:45.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.163+0000 7faa477fe700 1 --2- 172.21.15.89:0/1763871870 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faa4c066f30 0x7faa4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:45.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.163+0000 7faa477fe700 1 -- 172.21.15.89:0/1763871870 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faa54085ce0 con 0x7faa60159c60 2023-10-10T20:34:45.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.163+0000 7faa5f7fe700 1 --2- 172.21.15.89:0/1763871870 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faa4c066f30 0x7faa4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:45.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.163+0000 7faa5f7fe700 1 --2- 172.21.15.89:0/1763871870 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faa4c066f30 0x7faa4c0693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faa50005c50 tx=0x7faa50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:45.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.167+0000 7faa477fe700 1 -- 172.21.15.89:0/1763871870 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa54052f50 con 0x7faa60159c60 2023-10-10T20:34:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.347+0000 7faa660c2700 1 -- 172.21.15.89:0/1763871870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa600ce950 con 0x7faa60159c60 2023-10-10T20:34:45.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.348+0000 7faa477fe700 1 -- 172.21.15.89:0/1763871870 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa54056570 con 0x7faa60159c60 2023-10-10T20:34:45.350 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:34:45.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.349+0000 7faa660c2700 1 -- 172.21.15.89:0/1763871870 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faa4c066f30 msgr2=0x7faa4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:45.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.349+0000 7faa660c2700 1 --2- 172.21.15.89:0/1763871870 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faa4c066f30 0x7faa4c0693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faa50005c50 tx=0x7faa50009040 comp rx=0 tx=0).stop 2023-10-10T20:34:45.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.349+0000 7faa660c2700 1 -- 172.21.15.89:0/1763871870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa60159c60 msgr2=0x7faa600cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:45.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.349+0000 7faa660c2700 1 --2- 172.21.15.89:0/1763871870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa60159c60 0x7faa600cf1e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faa600899c0 tx=0x7faa54017000 comp rx=0 tx=0).stop 2023-10-10T20:34:45.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.349+0000 7faa660c2700 1 -- 172.21.15.89:0/1763871870 shutdown_connections 2023-10-10T20:34:45.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.350+0000 7faa660c2700 1 --2- 172.21.15.89:0/1763871870 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faa4c066f30 0x7faa4c0693e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.350+0000 7faa660c2700 1 --2- 172.21.15.89:0/1763871870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6015db10 0x7faa600cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.350+0000 7faa660c2700 1 --2- 172.21.15.89:0/1763871870 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa60159890 0x7faa600ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.350+0000 7faa660c2700 1 --2- 172.21.15.89:0/1763871870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa60159c60 0x7faa600cf1e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.350+0000 7faa660c2700 1 -- 172.21.15.89:0/1763871870 >> 172.21.15.89:0/1763871870 conn(0x7faa60068e20 msgr2=0x7faa6014d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.350+0000 7faa660c2700 1 -- 172.21.15.89:0/1763871870 shutdown_connections 2023-10-10T20:34:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.350+0000 7faa660c2700 1 -- 172.21.15.89:0/1763871870 wait complete. 2023-10-10T20:34:45.364 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:34:45.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.480+0000 7f6d1d0bc700 1 Processor -- start 2023-10-10T20:34:45.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.480+0000 7f6d1d0bc700 1 -- start start 2023-10-10T20:34:45.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.481+0000 7f6d1d0bc700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6d18159a70 0x7f6d180c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:45.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.481+0000 7f6d1d0bc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d18068aa0 0x7f6d180c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:45.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.481+0000 7f6d1d0bc700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d180ca920 con 0x7f6d1806bfa0 2023-10-10T20:34:45.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.481+0000 7f6d1d0bc700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1806af00 con 0x7f6d18159a70 2023-10-10T20:34:45.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.481+0000 7f6d1d0bc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1806b1e0 con 0x7f6d18068aa0 2023-10-10T20:34:45.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.482+0000 7f6d1c8bb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d18068aa0 0x7f6d180c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:45.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.482+0000 7f6d17fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6d18159a70 0x7f6d180c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:45.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.482+0000 7f6d1c8bb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d18068aa0 0x7f6d180c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:44288/0 (socket says 172.21.15.89:44288) 2023-10-10T20:34:45.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.482+0000 7f6d1c8bb700 1 -- 172.21.15.89:0/1248878880 learned_addr learned my addr 172.21.15.89:0/1248878880 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:45.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.482+0000 7f6d1c8bb700 1 -- 172.21.15.89:0/1248878880 >> v2:172.21.15.111:3300/0 conn(0x7f6d18159a70 msgr2=0x7f6d180c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:45.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.482+0000 7f6d1c8bb700 1 --2- 172.21.15.89:0/1248878880 >> v2:172.21.15.111:3300/0 conn(0x7f6d18159a70 0x7f6d180c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.482+0000 7f6d1c8bb700 1 -- 172.21.15.89:0/1248878880 >> v1:172.21.15.111:6789/0 conn(0x7f6d1806bfa0 legacy=0x7f6d18068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:45.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.482+0000 7f6d1c8bb700 1 -- 172.21.15.89:0/1248878880 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1806b4c0 con 0x7f6d18068aa0 2023-10-10T20:34:45.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.482+0000 7f6d17fff700 1 --2- 172.21.15.89:0/1248878880 >> v2:172.21.15.111:3300/0 conn(0x7f6d18159a70 0x7f6d180c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:45.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.482+0000 7f6d1c8bb700 1 --2- 172.21.15.89:0/1248878880 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d18068aa0 0x7f6d180c8830 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009f20 tx=0x7f6d10009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=226842164bf502f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:45.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.486+0000 7f6d16ffd700 1 -- 172.21.15.89:0/1248878880 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1000e070 con 0x7f6d18068aa0 2023-10-10T20:34:45.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.486+0000 7f6d16ffd700 1 -- 172.21.15.89:0/1248878880 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1000b9d0 con 0x7f6d18068aa0 2023-10-10T20:34:45.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.486+0000 7f6d16ffd700 1 -- 172.21.15.89:0/1248878880 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d100092e0 con 0x7f6d18068aa0 2023-10-10T20:34:45.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.487+0000 7f6d1d0bc700 1 -- 172.21.15.89:0/1248878880 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d18068aa0 msgr2=0x7f6d180c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:45.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.487+0000 7f6d1d0bc700 1 --2- 172.21.15.89:0/1248878880 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d18068aa0 0x7f6d180c8830 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009f20 tx=0x7f6d10009840 comp rx=0 tx=0).stop 2023-10-10T20:34:45.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.487+0000 7f6d1d0bc700 1 -- 172.21.15.89:0/1248878880 shutdown_connections 2023-10-10T20:34:45.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.487+0000 7f6d1d0bc700 1 --2- 172.21.15.89:0/1248878880 >> v2:172.21.15.111:3300/0 conn(0x7f6d18159a70 0x7f6d180c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.487+0000 7f6d1d0bc700 1 --2- 172.21.15.89:0/1248878880 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d18068aa0 0x7f6d180c8830 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.487+0000 7f6d1d0bc700 1 -- 172.21.15.89:0/1248878880 >> 172.21.15.89:0/1248878880 conn(0x7f6d180cddc0 msgr2=0x7f6d180c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:45.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.487+0000 7f6d1d0bc700 1 -- 172.21.15.89:0/1248878880 shutdown_connections 2023-10-10T20:34:45.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.487+0000 7f6d1d0bc700 1 -- 172.21.15.89:0/1248878880 wait complete. 2023-10-10T20:34:45.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.488+0000 7f6d1d0bc700 1 Processor -- start 2023-10-10T20:34:45.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.488+0000 7f6d1d0bc700 1 -- start start 2023-10-10T20:34:45.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.489+0000 7f6d1d0bc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d18068aa0 0x7f6d18142ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:45.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.489+0000 7f6d1d0bc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d1806bfa0 0x7f6d1815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:45.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.489+0000 7f6d1d0bc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d18159a70 0x7f6d180c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:45.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.489+0000 7f6d1d0bc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d18143530 con 0x7f6d18068aa0 2023-10-10T20:34:45.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.489+0000 7f6d1d0bc700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1815dc30 con 0x7f6d1806bfa0 2023-10-10T20:34:45.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.489+0000 7f6d1d0bc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d180c72a0 con 0x7f6d18159a70 2023-10-10T20:34:45.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.489+0000 7f6d17fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d18068aa0 0x7f6d18142ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:45.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.489+0000 7f6d17fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d18068aa0 0x7f6d18142ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53666/0 (socket says 172.21.15.89:53666) 2023-10-10T20:34:45.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.489+0000 7f6d17fff700 1 -- 172.21.15.89:0/1989638722 learned_addr learned my addr 172.21.15.89:0/1989638722 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:45.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.489+0000 7f6d1c8bb700 1 --2- 172.21.15.89:0/1989638722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d18159a70 0x7f6d180c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:45.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.489+0000 7f6d177fe700 1 --2- 172.21.15.89:0/1989638722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d1806bfa0 0x7f6d1815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:45.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.489+0000 7f6d17fff700 1 -- 172.21.15.89:0/1989638722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d18159a70 msgr2=0x7f6d180c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:45.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.489+0000 7f6d17fff700 1 --2- 172.21.15.89:0/1989638722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d18159a70 0x7f6d180c6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.489+0000 7f6d17fff700 1 -- 172.21.15.89:0/1989638722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d1806bfa0 msgr2=0x7f6d1815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:45.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.489+0000 7f6d17fff700 1 --2- 172.21.15.89:0/1989638722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d1806bfa0 0x7f6d1815d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.489+0000 7f6d17fff700 1 -- 172.21.15.89:0/1989638722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d08009420 con 0x7f6d18068aa0 2023-10-10T20:34:45.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.489+0000 7f6d17fff700 1 --2- 172.21.15.89:0/1989638722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d18068aa0 0x7f6d18142ff0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800ec80 tx=0x7f6d080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:45.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.490+0000 7f6cfffff700 1 -- 172.21.15.89:0/1989638722 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0800c920 con 0x7f6d18068aa0 2023-10-10T20:34:45.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.490+0000 7f6cfffff700 1 -- 172.21.15.89:0/1989638722 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0800ca80 con 0x7f6d18068aa0 2023-10-10T20:34:45.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.490+0000 7f6d1d0bc700 1 -- 172.21.15.89:0/1989638722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d10009ae0 con 0x7f6d18068aa0 2023-10-10T20:34:45.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.490+0000 7f6cfffff700 1 -- 172.21.15.89:0/1989638722 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0800ce00 con 0x7f6d18068aa0 2023-10-10T20:34:45.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.490+0000 7f6d1d0bc700 1 -- 172.21.15.89:0/1989638722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d180c67b0 con 0x7f6d18068aa0 2023-10-10T20:34:45.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.490+0000 7f6d1d0bc700 1 -- 172.21.15.89:0/1989638722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d180b04a0 con 0x7f6d18068aa0 2023-10-10T20:34:45.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.491+0000 7f6cfffff700 1 -- 172.21.15.89:0/1989638722 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d08013680 con 0x7f6d18068aa0 2023-10-10T20:34:45.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.491+0000 7f6cfffff700 1 --2- 172.21.15.89:0/1989638722 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6d04066f30 0x7f6d040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:45.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.491+0000 7f6cfffff700 1 -- 172.21.15.89:0/1989638722 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6d08084be0 con 0x7f6d18068aa0 2023-10-10T20:34:45.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.491+0000 7f6d177fe700 1 --2- 172.21.15.89:0/1989638722 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6d04066f30 0x7f6d040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:45.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.492+0000 7f6d177fe700 1 --2- 172.21.15.89:0/1989638722 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6d04066f30 0x7f6d040693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6d18157b00 tx=0x7f6d0c007f10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:45.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.495+0000 7f6cfffff700 1 -- 172.21.15.89:0/1989638722 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d08051e50 con 0x7f6d18068aa0 2023-10-10T20:34:45.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.679+0000 7f6d1d0bc700 1 -- 172.21.15.89:0/1989638722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6d1806b7a0 con 0x7f6d18068aa0 2023-10-10T20:34:45.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.680+0000 7f6cfffff700 1 -- 172.21.15.89:0/1989638722 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6d08017020 con 0x7f6d18068aa0 2023-10-10T20:34:45.682 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:34:45.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.682+0000 7f6d1d0bc700 1 -- 172.21.15.89:0/1989638722 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6d04066f30 msgr2=0x7f6d040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:45.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.682+0000 7f6d1d0bc700 1 --2- 172.21.15.89:0/1989638722 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6d04066f30 0x7f6d040693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6d18157b00 tx=0x7f6d0c007f10 comp rx=0 tx=0).stop 2023-10-10T20:34:45.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.682+0000 7f6d1d0bc700 1 -- 172.21.15.89:0/1989638722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d18068aa0 msgr2=0x7f6d18142ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:45.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.682+0000 7f6d1d0bc700 1 --2- 172.21.15.89:0/1989638722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d18068aa0 0x7f6d18142ff0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800ec80 tx=0x7f6d080091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.682+0000 7f6d1d0bc700 1 -- 172.21.15.89:0/1989638722 shutdown_connections 2023-10-10T20:34:45.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.682+0000 7f6d1d0bc700 1 --2- 172.21.15.89:0/1989638722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d18159a70 0x7f6d180c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.682+0000 7f6d1d0bc700 1 --2- 172.21.15.89:0/1989638722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d18068aa0 0x7f6d18142ff0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.682+0000 7f6d1d0bc700 1 --2- 172.21.15.89:0/1989638722 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6d04066f30 0x7f6d040693e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.682+0000 7f6d1d0bc700 1 --2- 172.21.15.89:0/1989638722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d1806bfa0 0x7f6d1815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.682+0000 7f6d1d0bc700 1 -- 172.21.15.89:0/1989638722 >> 172.21.15.89:0/1989638722 conn(0x7f6d180cddc0 msgr2=0x7f6d18068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:45.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.682+0000 7f6d1d0bc700 1 -- 172.21.15.89:0/1989638722 shutdown_connections 2023-10-10T20:34:45.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.682+0000 7f6d1d0bc700 1 -- 172.21.15.89:0/1989638722 wait complete. 2023-10-10T20:34:45.696 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:34:45.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.812+0000 7fb04aaf1700 1 Processor -- start 2023-10-10T20:34:45.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.812+0000 7fb04aaf1700 1 -- start start 2023-10-10T20:34:45.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.813+0000 7fb04aaf1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb044159890 0x7fb04415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:45.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.813+0000 7fb04aaf1700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb044069560 0x7fb0440699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:45.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.813+0000 7fb04aaf1700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04415a160 con 0x7fb044159c60 2023-10-10T20:34:45.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.813+0000 7fb04aaf1700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb044069f20 con 0x7fb044069560 2023-10-10T20:34:45.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.813+0000 7fb04aaf1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440d13a0 con 0x7fb044159890 2023-10-10T20:34:45.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.813+0000 7fb04888d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb044159890 0x7fb04415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:45.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.813+0000 7fb04888d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb044159890 0x7fb04415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:44304/0 (socket says 172.21.15.89:44304) 2023-10-10T20:34:45.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.813+0000 7fb04908e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb044069560 0x7fb0440699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:45.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.813+0000 7fb04888d700 1 -- 172.21.15.89:0/1063722145 learned_addr learned my addr 172.21.15.89:0/1063722145 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:45.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.813+0000 7fb043fff700 1 --1- 172.21.15.89:0/1063722145 >> v1:172.21.15.111:6789/0 conn(0x7fb044159c60 0x7fb04415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:49374/0 (socket says 172.21.15.89:49374) 2023-10-10T20:34:45.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.813+0000 7fb04888d700 1 -- 172.21.15.89:0/1063722145 >> v2:172.21.15.111:3300/0 conn(0x7fb044069560 msgr2=0x7fb0440699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:45.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.813+0000 7fb04888d700 1 --2- 172.21.15.89:0/1063722145 >> v2:172.21.15.111:3300/0 conn(0x7fb044069560 0x7fb0440699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.813+0000 7fb04888d700 1 -- 172.21.15.89:0/1063722145 >> v1:172.21.15.111:6789/0 conn(0x7fb044159c60 legacy=0x7fb04415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:45.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.813+0000 7fb04888d700 1 -- 172.21.15.89:0/1063722145 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04415e140 con 0x7fb044159890 2023-10-10T20:34:45.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.814+0000 7fb04888d700 1 --2- 172.21.15.89:0/1063722145 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb044159890 0x7fb04415abd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009f20 tx=0x7fb03c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4262b502b329e4ea server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:45.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.814+0000 7fb0437fe700 1 -- 172.21.15.89:0/1063722145 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00e070 con 0x7fb044159890 2023-10-10T20:34:45.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.814+0000 7fb0437fe700 1 -- 172.21.15.89:0/1063722145 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c00b9d0 con 0x7fb044159890 2023-10-10T20:34:45.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.814+0000 7fb0437fe700 1 -- 172.21.15.89:0/1063722145 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0092e0 con 0x7fb044159890 2023-10-10T20:34:45.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.814+0000 7fb04aaf1700 1 -- 172.21.15.89:0/1063722145 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb044159890 msgr2=0x7fb04415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:45.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.814+0000 7fb04aaf1700 1 --2- 172.21.15.89:0/1063722145 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb044159890 0x7fb04415abd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009f20 tx=0x7fb03c009840 comp rx=0 tx=0).stop 2023-10-10T20:34:45.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.814+0000 7fb04aaf1700 1 -- 172.21.15.89:0/1063722145 shutdown_connections 2023-10-10T20:34:45.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.814+0000 7fb04aaf1700 1 --2- 172.21.15.89:0/1063722145 >> v2:172.21.15.111:3300/0 conn(0x7fb044069560 0x7fb0440699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.814+0000 7fb04aaf1700 1 --2- 172.21.15.89:0/1063722145 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb044159890 0x7fb04415abd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.814+0000 7fb04aaf1700 1 -- 172.21.15.89:0/1063722145 >> 172.21.15.89:0/1063722145 conn(0x7fb044068e20 msgr2=0x7fb0440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:45.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.814+0000 7fb04aaf1700 1 -- 172.21.15.89:0/1063722145 shutdown_connections 2023-10-10T20:34:45.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.814+0000 7fb04aaf1700 1 -- 172.21.15.89:0/1063722145 wait complete. 2023-10-10T20:34:45.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.815+0000 7fb04aaf1700 1 Processor -- start 2023-10-10T20:34:45.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.815+0000 7fb04aaf1700 1 -- start start 2023-10-10T20:34:45.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.816+0000 7fb04aaf1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb044069560 0x7fb04414f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:45.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.816+0000 7fb04aaf1700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb044159890 0x7fb04414c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:45.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.816+0000 7fb04aaf1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb044159c60 0x7fb044150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:45.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.816+0000 7fb04aaf1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0441502a0 con 0x7fb044159c60 2023-10-10T20:34:45.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.816+0000 7fb04aaf1700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04414f690 con 0x7fb044159890 2023-10-10T20:34:45.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.816+0000 7fb04aaf1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04414f970 con 0x7fb044069560 2023-10-10T20:34:45.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.816+0000 7fb04908e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb044159c60 0x7fb044150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:45.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.816+0000 7fb04908e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb044159c60 0x7fb044150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53680/0 (socket says 172.21.15.89:53680) 2023-10-10T20:34:45.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.816+0000 7fb04908e700 1 -- 172.21.15.89:0/2940320585 learned_addr learned my addr 172.21.15.89:0/2940320585 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:45.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.816+0000 7fb04888d700 1 --2- 172.21.15.89:0/2940320585 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb044069560 0x7fb04414f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:45.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.816+0000 7fb043fff700 1 --2- 172.21.15.89:0/2940320585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb044159890 0x7fb04414c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:45.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.817+0000 7fb04908e700 1 -- 172.21.15.89:0/2940320585 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb044069560 msgr2=0x7fb04414f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:45.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.817+0000 7fb04908e700 1 --2- 172.21.15.89:0/2940320585 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb044069560 0x7fb04414f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.817+0000 7fb04908e700 1 -- 172.21.15.89:0/2940320585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb044159890 msgr2=0x7fb04414c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:45.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.817+0000 7fb04908e700 1 --2- 172.21.15.89:0/2940320585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb044159890 0x7fb04414c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:45.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.817+0000 7fb04908e700 1 -- 172.21.15.89:0/2940320585 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c009ae0 con 0x7fb044159c60 2023-10-10T20:34:45.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.817+0000 7fb04908e700 1 --2- 172.21.15.89:0/2940320585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb044159c60 0x7fb044150c50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00e8d0 tx=0x7fb02c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:45.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.817+0000 7fb040ff9700 1 -- 172.21.15.89:0/2940320585 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c013680 con 0x7fb044159c60 2023-10-10T20:34:45.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.817+0000 7fb040ff9700 1 -- 172.21.15.89:0/2940320585 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c0137e0 con 0x7fb044159c60 2023-10-10T20:34:45.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.817+0000 7fb04aaf1700 1 -- 172.21.15.89:0/2940320585 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04414fc50 con 0x7fb044159c60 2023-10-10T20:34:45.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.817+0000 7fb040ff9700 1 -- 172.21.15.89:0/2940320585 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c013ba0 con 0x7fb044159c60 2023-10-10T20:34:45.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.817+0000 7fb04aaf1700 1 -- 172.21.15.89:0/2940320585 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04414cc40 con 0x7fb044159c60 2023-10-10T20:34:45.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.818+0000 7fb04aaf1700 1 -- 172.21.15.89:0/2940320585 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb044150450 con 0x7fb044159c60 2023-10-10T20:34:45.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.818+0000 7fb040ff9700 1 -- 172.21.15.89:0/2940320585 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb02c0123d0 con 0x7fb044159c60 2023-10-10T20:34:45.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.819+0000 7fb040ff9700 1 --2- 172.21.15.89:0/2940320585 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb024066fb0 0x7fb024069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:45.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.819+0000 7fb040ff9700 1 -- 172.21.15.89:0/2940320585 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb02c0846b0 con 0x7fb044159c60 2023-10-10T20:34:45.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.819+0000 7fb04888d700 1 --2- 172.21.15.89:0/2940320585 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb024066fb0 0x7fb024069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:45.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.819+0000 7fb04888d700 1 --2- 172.21.15.89:0/2940320585 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb024066fb0 0x7fb024069460 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009880 tx=0x7fb03c018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:45.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:45.823+0000 7fb040ff9700 1 -- 172.21.15.89:0/2940320585 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb044150450 con 0x7fb044159c60 2023-10-10T20:34:46.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.004+0000 7fb04aaf1700 1 -- 172.21.15.89:0/2940320585 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb04404c3c0 con 0x7fb044159c60 2023-10-10T20:34:46.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.004+0000 7fb040ff9700 1 -- 172.21.15.89:0/2940320585 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb02c051880 con 0x7fb044159c60 2023-10-10T20:34:46.007 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-10T20:34:46.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.006+0000 7fb04aaf1700 1 -- 172.21.15.89:0/2940320585 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb024066fb0 msgr2=0x7fb024069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.006+0000 7fb04aaf1700 1 --2- 172.21.15.89:0/2940320585 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb024066fb0 0x7fb024069460 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009880 tx=0x7fb03c018000 comp rx=0 tx=0).stop 2023-10-10T20:34:46.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.006+0000 7fb04aaf1700 1 -- 172.21.15.89:0/2940320585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb044159c60 msgr2=0x7fb044150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.006+0000 7fb04aaf1700 1 --2- 172.21.15.89:0/2940320585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb044159c60 0x7fb044150c50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00e8d0 tx=0x7fb02c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.006+0000 7fb04aaf1700 1 -- 172.21.15.89:0/2940320585 shutdown_connections 2023-10-10T20:34:46.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.006+0000 7fb04aaf1700 1 --2- 172.21.15.89:0/2940320585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb044159c60 0x7fb044150c50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.006+0000 7fb04aaf1700 1 --2- 172.21.15.89:0/2940320585 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb044069560 0x7fb04414f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.006+0000 7fb04aaf1700 1 --2- 172.21.15.89:0/2940320585 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb024066fb0 0x7fb024069460 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.006+0000 7fb04aaf1700 1 --2- 172.21.15.89:0/2940320585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb044159890 0x7fb04414c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.006+0000 7fb04aaf1700 1 -- 172.21.15.89:0/2940320585 >> 172.21.15.89:0/2940320585 conn(0x7fb044068e20 msgr2=0x7fb04414d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:46.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.006+0000 7fb04aaf1700 1 -- 172.21.15.89:0/2940320585 shutdown_connections 2023-10-10T20:34:46.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.006+0000 7fb04aaf1700 1 -- 172.21.15.89:0/2940320585 wait complete. 2023-10-10T20:34:46.020 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:34:46.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.132+0000 7f402d730700 1 Processor -- start 2023-10-10T20:34:46.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.132+0000 7f402d730700 1 -- start start 2023-10-10T20:34:46.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.133+0000 7f402d730700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4028159420 0x7f40280b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:46.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.133+0000 7f402d730700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40280b2980 con 0x7f40281597f0 2023-10-10T20:34:46.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.133+0000 7f402d730700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40280b11d0 con 0x7f402815d230 2023-10-10T20:34:46.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.133+0000 7f402d730700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40280b14b0 con 0x7f4028159420 2023-10-10T20:34:46.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.133+0000 7f4027fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4028159420 0x7f40280b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:46.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.133+0000 7f40277fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f40281597f0 0x7f40280b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35790/0 (socket says 172.21.15.89:35790) 2023-10-10T20:34:46.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.133+0000 7f40277fe700 1 -- 172.21.15.89:0/3275237868 learned_addr learned my addr 172.21.15.89:0/3275237868 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:46.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.133+0000 7f4027fff700 1 -- 172.21.15.89:0/3275237868 >> v1:172.21.15.111:6789/0 conn(0x7f402815d230 legacy=0x7f402815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.133+0000 7f4027fff700 1 -- 172.21.15.89:0/3275237868 >> v1:172.21.15.89:6789/0 conn(0x7f40281597f0 legacy=0x7f40280b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.133+0000 7f4026ffd700 1 -- 172.21.15.89:0/3275237868 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1829667036 0 0) 0x7f40280b2980 con 0x7f40281597f0 2023-10-10T20:34:46.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.133+0000 7f4027fff700 1 -- 172.21.15.89:0/3275237868 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40280b1790 con 0x7f4028159420 2023-10-10T20:34:46.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.134+0000 7f4027fff700 1 --2- 172.21.15.89:0/3275237868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4028159420 0x7f40280b5f40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f40180098e0 tx=0x7f40180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef5f74870a6551d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:46.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.134+0000 7f4026ffd700 1 -- 172.21.15.89:0/3275237868 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018009e80 con 0x7f4028159420 2023-10-10T20:34:46.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.134+0000 7f4026ffd700 1 -- 172.21.15.89:0/3275237868 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401800b860 con 0x7f4028159420 2023-10-10T20:34:46.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.134+0000 7f4026ffd700 1 -- 172.21.15.89:0/3275237868 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401800bbe0 con 0x7f4028159420 2023-10-10T20:34:46.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.134+0000 7f402d730700 1 -- 172.21.15.89:0/3275237868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4028159420 msgr2=0x7f40280b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.134+0000 7f402d730700 1 --2- 172.21.15.89:0/3275237868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4028159420 0x7f40280b5f40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f40180098e0 tx=0x7f40180091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.134+0000 7f402d730700 1 -- 172.21.15.89:0/3275237868 shutdown_connections 2023-10-10T20:34:46.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.134+0000 7f402d730700 1 --2- 172.21.15.89:0/3275237868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4028159420 0x7f40280b5f40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.134+0000 7f402d730700 1 -- 172.21.15.89:0/3275237868 >> 172.21.15.89:0/3275237868 conn(0x7f40280d0660 msgr2=0x7f40280d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:46.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.134+0000 7f402d730700 1 -- 172.21.15.89:0/3275237868 shutdown_connections 2023-10-10T20:34:46.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.134+0000 7f402d730700 1 -- 172.21.15.89:0/3275237868 wait complete. 2023-10-10T20:34:46.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.135+0000 7f402d730700 1 Processor -- start 2023-10-10T20:34:46.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.135+0000 7f402d730700 1 -- start start 2023-10-10T20:34:46.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.136+0000 7f402d730700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4028159420 0x7f40281552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:46.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.136+0000 7f402d730700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40281597f0 0x7f40280ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:46.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.136+0000 7f402d730700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f402815d230 0x7f40280cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:46.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.136+0000 7f402d730700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4028155810 con 0x7f4028159420 2023-10-10T20:34:46.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.136+0000 7f402d730700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40280cae80 con 0x7f402815d230 2023-10-10T20:34:46.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.136+0000 7f402d730700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40280cb160 con 0x7f40281597f0 2023-10-10T20:34:46.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.136+0000 7f4027fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4028159420 0x7f40281552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:46.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.136+0000 7f4027fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4028159420 0x7f40281552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53700/0 (socket says 172.21.15.89:53700) 2023-10-10T20:34:46.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.136+0000 7f4027fff700 1 -- 172.21.15.89:0/2585259287 learned_addr learned my addr 172.21.15.89:0/2585259287 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:46.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.136+0000 7f40277fe700 1 --2- 172.21.15.89:0/2585259287 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40281597f0 0x7f40280ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:46.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.137+0000 7f4027fff700 1 -- 172.21.15.89:0/2585259287 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40281597f0 msgr2=0x7f40280ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.137+0000 7f402cf2f700 1 --2- 172.21.15.89:0/2585259287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f402815d230 0x7f40280cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:46.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.137+0000 7f4027fff700 1 --2- 172.21.15.89:0/2585259287 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40281597f0 0x7f40280ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.137+0000 7f4027fff700 1 -- 172.21.15.89:0/2585259287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f402815d230 msgr2=0x7f40280cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.137+0000 7f4027fff700 1 --2- 172.21.15.89:0/2585259287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f402815d230 0x7f40280cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.137+0000 7f4027fff700 1 -- 172.21.15.89:0/2585259287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4018009420 con 0x7f4028159420 2023-10-10T20:34:46.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.137+0000 7f40277fe700 1 --2- 172.21.15.89:0/2585259287 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40281597f0 0x7f40280ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:46.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.137+0000 7f402cf2f700 1 --2- 172.21.15.89:0/2585259287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f402815d230 0x7f40280cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:46.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.137+0000 7f4027fff700 1 --2- 172.21.15.89:0/2585259287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4028159420 0x7f40281552d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4028089340 tx=0x7f4018017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:46.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.137+0000 7f400ffff700 1 -- 172.21.15.89:0/2585259287 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018011420 con 0x7f4028159420 2023-10-10T20:34:46.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.137+0000 7f400ffff700 1 -- 172.21.15.89:0/2585259287 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4018011580 con 0x7f4028159420 2023-10-10T20:34:46.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.137+0000 7f402d730700 1 -- 172.21.15.89:0/2585259287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40280cb440 con 0x7f4028159420 2023-10-10T20:34:46.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.137+0000 7f400ffff700 1 -- 172.21.15.89:0/2585259287 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018011940 con 0x7f4028159420 2023-10-10T20:34:46.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.137+0000 7f402d730700 1 -- 172.21.15.89:0/2585259287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40280b4dc0 con 0x7f4028159420 2023-10-10T20:34:46.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.138+0000 7f402d730700 1 -- 172.21.15.89:0/2585259287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40280b4fb0 con 0x7f4028159420 2023-10-10T20:34:46.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.138+0000 7f400ffff700 1 -- 172.21.15.89:0/2585259287 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4018011cf0 con 0x7f4028159420 2023-10-10T20:34:46.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.139+0000 7f400ffff700 1 --2- 172.21.15.89:0/2585259287 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4014066f30 0x7f40140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:46.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.139+0000 7f40277fe700 1 --2- 172.21.15.89:0/2585259287 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4014066f30 0x7f40140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:46.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.139+0000 7f400ffff700 1 -- 172.21.15.89:0/2585259287 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4018085d70 con 0x7f4028159420 2023-10-10T20:34:46.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.139+0000 7f40277fe700 1 --2- 172.21.15.89:0/2585259287 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4014066f30 0x7f40140693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f402806a080 tx=0x7f401c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:46.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.143+0000 7f400ffff700 1 -- 172.21.15.89:0/2585259287 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40280b4fb0 con 0x7f4028159420 2023-10-10T20:34:46.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.364+0000 7f402d730700 1 -- 172.21.15.89:0/2585259287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40280b4fb0 con 0x7f4028159420 2023-10-10T20:34:46.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.365+0000 7f400ffff700 1 -- 172.21.15.89:0/2585259287 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4597 (secure 0 0 0) 0x7f40280b4fb0 con 0x7f4028159420 2023-10-10T20:34:46.368 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:46.368 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2341577144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2341577144},{"type":"v1","addr":"172.21.15.111:6835","nonce":2341577144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2773123076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2773123076},{"type":"v1","addr":"172.21.15.89:6837","nonce":2773123076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2940044414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2940044414},{"type":"v1","addr":"172.21.15.89:6835","nonce":2940044414}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:34:34.717750+0000","modified":"2023-10-10T20:34:38.356048+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1112081320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1112081320},{"type":"v1","addr":"172.21.15.111:6834","nonce":1112081320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-10T20:34:46.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.367+0000 7f402d730700 1 -- 172.21.15.89:0/2585259287 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4014066f30 msgr2=0x7f40140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.367+0000 7f402d730700 1 --2- 172.21.15.89:0/2585259287 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4014066f30 0x7f40140693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f402806a080 tx=0x7f401c009040 comp rx=0 tx=0).stop 2023-10-10T20:34:46.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.367+0000 7f402d730700 1 -- 172.21.15.89:0/2585259287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4028159420 msgr2=0x7f40281552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.367+0000 7f402d730700 1 --2- 172.21.15.89:0/2585259287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4028159420 0x7f40281552d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4028089340 tx=0x7f4018017000 comp rx=0 tx=0).stop 2023-10-10T20:34:46.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.367+0000 7f402d730700 1 -- 172.21.15.89:0/2585259287 shutdown_connections 2023-10-10T20:34:46.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.367+0000 7f402d730700 1 --2- 172.21.15.89:0/2585259287 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4014066f30 0x7f40140693e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.367+0000 7f402d730700 1 --2- 172.21.15.89:0/2585259287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f402815d230 0x7f40280cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.367+0000 7f402d730700 1 --2- 172.21.15.89:0/2585259287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4028159420 0x7f40281552d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.367+0000 7f402d730700 1 --2- 172.21.15.89:0/2585259287 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40281597f0 0x7f40280ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.367+0000 7f402d730700 1 -- 172.21.15.89:0/2585259287 >> 172.21.15.89:0/2585259287 conn(0x7f40280d0660 msgr2=0x7f402815d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:46.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.367+0000 7f402d730700 1 -- 172.21.15.89:0/2585259287 shutdown_connections 2023-10-10T20:34:46.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.367+0000 7f402d730700 1 -- 172.21.15.89:0/2585259287 wait complete. 2023-10-10T20:34:46.372 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2023-10-10T20:34:46.381 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:34:46.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.495+0000 7fd4142f7700 1 Processor -- start 2023-10-10T20:34:46.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.495+0000 7fd4142f7700 1 -- start start 2023-10-10T20:34:46.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.496+0000 7fd4142f7700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd40c159c60 0x7fd40c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:46.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.496+0000 7fd4142f7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd40c15db10 0x7fd40c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:46.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.496+0000 7fd4142f7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40c15a160 con 0x7fd40c159890 2023-10-10T20:34:46.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.496+0000 7fd4142f7700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd40c15df50 con 0x7fd40c159c60 2023-10-10T20:34:46.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.496+0000 7fd4142f7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0d13a0 con 0x7fd40c15db10 2023-10-10T20:34:46.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.496+0000 7fd412093700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd40c159890 0x7fd40c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35792/0 (socket says 172.21.15.89:35792) 2023-10-10T20:34:46.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.496+0000 7fd412093700 1 -- 172.21.15.89:0/767889603 learned_addr learned my addr 172.21.15.89:0/767889603 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:46.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.496+0000 7fd411892700 1 --2- 172.21.15.89:0/767889603 >> v2:172.21.15.111:3300/0 conn(0x7fd40c159c60 0x7fd40c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:46.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.496+0000 7fd412894700 1 --2- 172.21.15.89:0/767889603 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd40c15db10 0x7fd40c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:46.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.496+0000 7fd411091700 1 -- 172.21.15.89:0/767889603 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804339468 0 0) 0x7fd40c15a160 con 0x7fd40c159890 2023-10-10T20:34:46.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411091700 1 -- 172.21.15.89:0/767889603 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3f8003570 con 0x7fd40c159890 2023-10-10T20:34:46.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411091700 1 -- 172.21.15.89:0/767889603 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3194147087 0 0) 0x7fd3f8003570 con 0x7fd40c159890 2023-10-10T20:34:46.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411091700 1 -- 172.21.15.89:0/767889603 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd40c15db10 msgr2=0x7fd40c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:46.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411091700 1 --2- 172.21.15.89:0/767889603 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd40c15db10 0x7fd40c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411091700 1 -- 172.21.15.89:0/767889603 >> v2:172.21.15.111:3300/0 conn(0x7fd40c159c60 msgr2=0x7fd40c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:46.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411091700 1 --2- 172.21.15.89:0/767889603 >> v2:172.21.15.111:3300/0 conn(0x7fd40c159c60 0x7fd40c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411091700 1 -- 172.21.15.89:0/767889603 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c15e140 con 0x7fd40c159890 2023-10-10T20:34:46.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411091700 1 -- 172.21.15.89:0/767889603 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fd4080017f0 con 0x7fd40c159890 2023-10-10T20:34:46.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411091700 1 -- 172.21.15.89:0/767889603 >> v1:172.21.15.89:6789/0 conn(0x7fd40c159890 legacy=0x7fd40c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411091700 1 --2- 172.21.15.89:0/767889603 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd40c15db10 0x7fd3f8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:46.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411091700 1 --2- 172.21.15.89:0/767889603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3f8003fe0 0x7fd3f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:46.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411091700 1 --2- 172.21.15.89:0/767889603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3f8004d50 0x7fd3f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:46.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411091700 1 -- 172.21.15.89:0/767889603 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0d13a0 con 0x7fd3f8003fe0 2023-10-10T20:34:46.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411091700 1 -- 172.21.15.89:0/767889603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f800ba60 con 0x7fd3f8004d50 2023-10-10T20:34:46.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411091700 1 -- 172.21.15.89:0/767889603 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f800bbd0 con 0x7fd40c15db10 2023-10-10T20:34:46.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411892700 1 --2- 172.21.15.89:0/767889603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3f8003fe0 0x7fd3f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:46.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd412093700 1 --2- 172.21.15.89:0/767889603 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd40c15db10 0x7fd3f8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:46.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd412894700 1 --2- 172.21.15.89:0/767889603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3f8004d50 0x7fd3f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:46.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411892700 1 -- 172.21.15.89:0/767889603 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd40c15db10 msgr2=0x7fd3f8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:46.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411892700 1 --2- 172.21.15.89:0/767889603 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd40c15db10 0x7fd3f8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411892700 1 -- 172.21.15.89:0/767889603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3f8004d50 msgr2=0x7fd3f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:46.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411892700 1 --2- 172.21.15.89:0/767889603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3f8004d50 0x7fd3f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411892700 1 -- 172.21.15.89:0/767889603 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f8003570 con 0x7fd3f8003fe0 2023-10-10T20:34:46.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411892700 1 --2- 172.21.15.89:0/767889603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3f8003fe0 0x7fd3f8004810 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009c00 tx=0x7fd3fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca0ca3f6dfc573e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:46.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.497+0000 7fd411091700 1 -- 172.21.15.89:0/767889603 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc00e6b0 con 0x7fd3f8003fe0 2023-10-10T20:34:46.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.498+0000 7fd411091700 1 -- 172.21.15.89:0/767889603 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3fc00e810 con 0x7fd3f8003fe0 2023-10-10T20:34:46.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.498+0000 7fd411091700 1 -- 172.21.15.89:0/767889603 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc00ebd0 con 0x7fd3f8003fe0 2023-10-10T20:34:46.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.498+0000 7fd4142f7700 1 -- 172.21.15.89:0/767889603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3f8003fe0 msgr2=0x7fd3f8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.498+0000 7fd4142f7700 1 --2- 172.21.15.89:0/767889603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3f8003fe0 0x7fd3f8004810 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009c00 tx=0x7fd3fc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.498+0000 7fd412894700 1 -- 172.21.15.89:0/767889603 reap_dead start 2023-10-10T20:34:46.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.498+0000 7fd4142f7700 1 -- 172.21.15.89:0/767889603 shutdown_connections 2023-10-10T20:34:46.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.498+0000 7fd4142f7700 1 -- 172.21.15.89:0/767889603 >> 172.21.15.89:0/767889603 conn(0x7fd40c068e20 msgr2=0x7fd40c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:46.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.498+0000 7fd4142f7700 1 -- 172.21.15.89:0/767889603 shutdown_connections 2023-10-10T20:34:46.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.498+0000 7fd4142f7700 1 -- 172.21.15.89:0/767889603 wait complete. 2023-10-10T20:34:46.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.499+0000 7fd4142f7700 1 Processor -- start 2023-10-10T20:34:46.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.499+0000 7fd4142f7700 1 -- start start 2023-10-10T20:34:46.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd4142f7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd3f8003fe0 0x7fd40c16da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:46.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd4142f7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd40c0cfc00 0x7fd40c140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:46.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd4142f7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd40c141530 0x7fd40c15f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:46.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd4142f7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0cdc40 con 0x7fd40c141530 2023-10-10T20:34:46.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd4142f7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0cd680 con 0x7fd40c0cfc00 2023-10-10T20:34:46.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd4142f7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0cd960 con 0x7fd3f8003fe0 2023-10-10T20:34:46.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd412894700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd40c141530 0x7fd40c15f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:46.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd412894700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd40c141530 0x7fd40c15f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53718/0 (socket says 172.21.15.89:53718) 2023-10-10T20:34:46.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd412894700 1 -- 172.21.15.89:0/1183158957 learned_addr learned my addr 172.21.15.89:0/1183158957 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:46.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd412093700 1 --2- 172.21.15.89:0/1183158957 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd3f8003fe0 0x7fd40c16da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:46.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd411892700 1 --2- 172.21.15.89:0/1183158957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd40c0cfc00 0x7fd40c140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:46.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd412894700 1 -- 172.21.15.89:0/1183158957 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd3f8003fe0 msgr2=0x7fd40c16da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd412894700 1 --2- 172.21.15.89:0/1183158957 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd3f8003fe0 0x7fd40c16da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd412894700 1 -- 172.21.15.89:0/1183158957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd40c0cfc00 msgr2=0x7fd40c140ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd412894700 1 --2- 172.21.15.89:0/1183158957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd40c0cfc00 0x7fd40c140ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd412894700 1 -- 172.21.15.89:0/1183158957 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c16d270 con 0x7fd40c141530 2023-10-10T20:34:46.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd412894700 1 --2- 172.21.15.89:0/1183158957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd40c141530 0x7fd40c15f940 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0d03b0 tx=0x7fd4040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:46.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd4027fc700 1 -- 172.21.15.89:0/1183158957 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404013680 con 0x7fd40c141530 2023-10-10T20:34:46.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd4027fc700 1 -- 172.21.15.89:0/1183158957 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4040137e0 con 0x7fd40c141530 2023-10-10T20:34:46.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.500+0000 7fd4027fc700 1 -- 172.21.15.89:0/1183158957 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404013b60 con 0x7fd40c141530 2023-10-10T20:34:46.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.501+0000 7fd4142f7700 1 -- 172.21.15.89:0/1183158957 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40c16d550 con 0x7fd40c141530 2023-10-10T20:34:46.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.501+0000 7fd4142f7700 1 -- 172.21.15.89:0/1183158957 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c16dfb0 con 0x7fd40c141530 2023-10-10T20:34:46.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.501+0000 7fd4142f7700 1 -- 172.21.15.89:0/1183158957 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40c16d740 con 0x7fd40c141530 2023-10-10T20:34:46.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.502+0000 7fd4027fc700 1 -- 172.21.15.89:0/1183158957 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd404014440 con 0x7fd40c141530 2023-10-10T20:34:46.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.502+0000 7fd4027fc700 1 --2- 172.21.15.89:0/1183158957 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd3f0066f30 0x7fd3f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:46.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.502+0000 7fd4027fc700 1 -- 172.21.15.89:0/1183158957 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd404084870 con 0x7fd40c141530 2023-10-10T20:34:46.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.502+0000 7fd412093700 1 --2- 172.21.15.89:0/1183158957 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd3f0066f30 0x7fd3f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:46.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.502+0000 7fd412093700 1 --2- 172.21.15.89:0/1183158957 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd3f0066f30 0x7fd3f00693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd4080019a0 tx=0x7fd408006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:46.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.506+0000 7fd4027fc700 1 -- 172.21.15.89:0/1183158957 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd404051b60 con 0x7fd40c141530 2023-10-10T20:34:46.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.693+0000 7fd4142f7700 1 -- 172.21.15.89:0/1183158957 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd40c04c3c0 con 0x7fd40c141530 2023-10-10T20:34:46.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.694+0000 7fd4027fc700 1 -- 172.21.15.89:0/1183158957 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fd404022090 con 0x7fd40c141530 2023-10-10T20:34:46.696 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:46.696 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:36.324920+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:34:32.342167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:34:33.671763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:34:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.696+0000 7fd4142f7700 1 -- 172.21.15.89:0/1183158957 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd3f0066f30 msgr2=0x7fd3f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.696+0000 7fd4142f7700 1 --2- 172.21.15.89:0/1183158957 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd3f0066f30 0x7fd3f00693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd4080019a0 tx=0x7fd408006040 comp rx=0 tx=0).stop 2023-10-10T20:34:46.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.696+0000 7fd4142f7700 1 -- 172.21.15.89:0/1183158957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd40c141530 msgr2=0x7fd40c15f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.696+0000 7fd4142f7700 1 --2- 172.21.15.89:0/1183158957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd40c141530 0x7fd40c15f940 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0d03b0 tx=0x7fd4040091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.696+0000 7fd4142f7700 1 -- 172.21.15.89:0/1183158957 shutdown_connections 2023-10-10T20:34:46.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.696+0000 7fd4142f7700 1 --2- 172.21.15.89:0/1183158957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd40c141530 0x7fd40c15f940 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.696+0000 7fd4142f7700 1 --2- 172.21.15.89:0/1183158957 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd3f8003fe0 0x7fd40c16da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.696+0000 7fd4142f7700 1 --2- 172.21.15.89:0/1183158957 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd3f0066f30 0x7fd3f00693e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.696+0000 7fd4142f7700 1 --2- 172.21.15.89:0/1183158957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd40c0cfc00 0x7fd40c140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.696+0000 7fd4142f7700 1 -- 172.21.15.89:0/1183158957 >> 172.21.15.89:0/1183158957 conn(0x7fd40c068e20 msgr2=0x7fd40c15b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.696+0000 7fd4142f7700 1 -- 172.21.15.89:0/1183158957 shutdown_connections 2023-10-10T20:34:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.696+0000 7fd4142f7700 1 -- 172.21.15.89:0/1183158957 wait complete. 2023-10-10T20:34:46.710 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:34:32.342167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:34:33.671763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:34:46.711 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:34:46.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.829+0000 7fab65985700 1 Processor -- start 2023-10-10T20:34:46.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.829+0000 7fab65985700 1 -- start start 2023-10-10T20:34:46.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.829+0000 7fab65985700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab60159890 0x7fab6015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:46.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.829+0000 7fab65985700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6015a160 con 0x7fab60159c60 2023-10-10T20:34:46.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.830+0000 7fab65985700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab600d13a0 con 0x7fab60069560 2023-10-10T20:34:46.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.830+0000 7fab65985700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6015e140 con 0x7fab60159890 2023-10-10T20:34:46.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.830+0000 7fab5e7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fab60159c60 0x7fab6015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35800/0 (socket says 172.21.15.89:35800) 2023-10-10T20:34:46.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.830+0000 7fab5e7fc700 1 -- 172.21.15.89:0/1045822323 learned_addr learned my addr 172.21.15.89:0/1045822323 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:46.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.830+0000 7fab5effd700 1 --2- 172.21.15.89:0/1045822323 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab60159890 0x7fab6015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:46.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.830+0000 7fab5dffb700 1 -- 172.21.15.89:0/1045822323 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1100909384 0 0) 0x7fab6015a160 con 0x7fab60159c60 2023-10-10T20:34:46.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.830+0000 7fab5dffb700 1 -- 172.21.15.89:0/1045822323 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab4c003570 con 0x7fab60159c60 2023-10-10T20:34:46.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.830+0000 7fab5dffb700 1 -- 172.21.15.89:0/1045822323 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1291446370 0 0) 0x7fab600d13a0 con 0x7fab60069560 2023-10-10T20:34:46.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.830+0000 7fab5dffb700 1 -- 172.21.15.89:0/1045822323 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab6015a160 con 0x7fab60069560 2023-10-10T20:34:46.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.830+0000 7fab5effd700 1 -- 172.21.15.89:0/1045822323 >> v1:172.21.15.111:6789/0 conn(0x7fab60069560 legacy=0x7fab600ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.830+0000 7fab5effd700 1 -- 172.21.15.89:0/1045822323 >> v1:172.21.15.89:6789/0 conn(0x7fab60159c60 legacy=0x7fab6015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.830+0000 7fab5dffb700 1 -- 172.21.15.89:0/1045822323 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 202194158 0 0) 0x7fab4c003570 con 0x7fab60159c60 2023-10-10T20:34:46.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.830+0000 7fab5effd700 1 -- 172.21.15.89:0/1045822323 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6015a3d0 con 0x7fab60159890 2023-10-10T20:34:46.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.830+0000 7fab5effd700 1 --2- 172.21.15.89:0/1045822323 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab60159890 0x7fab6015abd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fab50009e90 tx=0x7fab50009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd1da3087501dcd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:46.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.830+0000 7fab5dffb700 1 -- 172.21.15.89:0/1045822323 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5000e070 con 0x7fab60159890 2023-10-10T20:34:46.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.831+0000 7fab5dffb700 1 -- 172.21.15.89:0/1045822323 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5000b870 con 0x7fab60159890 2023-10-10T20:34:46.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.831+0000 7fab5dffb700 1 -- 172.21.15.89:0/1045822323 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48001470 con 0x7fab60159890 2023-10-10T20:34:46.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.831+0000 7fab65985700 1 -- 172.21.15.89:0/1045822323 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab60159890 msgr2=0x7fab6015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.831+0000 7fab65985700 1 --2- 172.21.15.89:0/1045822323 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab60159890 0x7fab6015abd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fab50009e90 tx=0x7fab50009780 comp rx=0 tx=0).stop 2023-10-10T20:34:46.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.831+0000 7fab65985700 1 -- 172.21.15.89:0/1045822323 shutdown_connections 2023-10-10T20:34:46.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.831+0000 7fab65985700 1 --2- 172.21.15.89:0/1045822323 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab60159890 0x7fab6015abd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.831+0000 7fab65985700 1 -- 172.21.15.89:0/1045822323 >> 172.21.15.89:0/1045822323 conn(0x7fab60068e20 msgr2=0x7fab600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:46.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.831+0000 7fab65985700 1 -- 172.21.15.89:0/1045822323 shutdown_connections 2023-10-10T20:34:46.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.831+0000 7fab65985700 1 -- 172.21.15.89:0/1045822323 wait complete. 2023-10-10T20:34:46.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.832+0000 7fab65985700 1 Processor -- start 2023-10-10T20:34:46.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.832+0000 7fab65985700 1 -- start start 2023-10-10T20:34:46.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.833+0000 7fab65985700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab60069560 0x7fab601493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:46.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.833+0000 7fab65985700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab60159890 0x7fab6016a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:46.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.833+0000 7fab65985700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab60159c60 0x7fab60151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:46.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.833+0000 7fab65985700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab601498e0 con 0x7fab60069560 2023-10-10T20:34:46.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.833+0000 7fab65985700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6016b0b0 con 0x7fab60159c60 2023-10-10T20:34:46.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.833+0000 7fab65985700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6016b390 con 0x7fab60159890 2023-10-10T20:34:46.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.833+0000 7fab5effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab60069560 0x7fab601493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:46.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.833+0000 7fab5effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab60069560 0x7fab601493a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53720/0 (socket says 172.21.15.89:53720) 2023-10-10T20:34:46.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.833+0000 7fab5effd700 1 -- 172.21.15.89:0/1087279396 learned_addr learned my addr 172.21.15.89:0/1087279396 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:46.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.833+0000 7fab5e7fc700 1 --2- 172.21.15.89:0/1087279396 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab60159890 0x7fab6016a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:46.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.833+0000 7fab5effd700 1 -- 172.21.15.89:0/1087279396 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab60159890 msgr2=0x7fab6016a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.833+0000 7fab5effd700 1 --2- 172.21.15.89:0/1087279396 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab60159890 0x7fab6016a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.833+0000 7fab5effd700 1 -- 172.21.15.89:0/1087279396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab60159c60 msgr2=0x7fab60151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:46.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.833+0000 7fab5effd700 1 --2- 172.21.15.89:0/1087279396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab60159c60 0x7fab60151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:46.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.833+0000 7fab5effd700 1 -- 172.21.15.89:0/1087279396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab50009a20 con 0x7fab60069560 2023-10-10T20:34:46.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.834+0000 7fab5effd700 1 --2- 172.21.15.89:0/1087279396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab60069560 0x7fab601493a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fab600899c0 tx=0x7fab50017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:46.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.834+0000 7fab56ffd700 1 -- 172.21.15.89:0/1087279396 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50012c90 con 0x7fab60069560 2023-10-10T20:34:46.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.834+0000 7fab56ffd700 1 -- 172.21.15.89:0/1087279396 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab50012e10 con 0x7fab60069560 2023-10-10T20:34:46.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.834+0000 7fab65985700 1 -- 172.21.15.89:0/1087279396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6016b670 con 0x7fab60069560 2023-10-10T20:34:46.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.834+0000 7fab56ffd700 1 -- 172.21.15.89:0/1087279396 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5001e6a0 con 0x7fab60069560 2023-10-10T20:34:46.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.834+0000 7fab65985700 1 -- 172.21.15.89:0/1087279396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6016bc30 con 0x7fab60069560 2023-10-10T20:34:46.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.835+0000 7fab65985700 1 -- 172.21.15.89:0/1087279396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6004c3c0 con 0x7fab60069560 2023-10-10T20:34:46.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.835+0000 7fab56ffd700 1 -- 172.21.15.89:0/1087279396 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fab5001ea90 con 0x7fab60069560 2023-10-10T20:34:46.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.835+0000 7fab56ffd700 1 --2- 172.21.15.89:0/1087279396 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab40066f30 0x7fab400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:46.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.835+0000 7fab56ffd700 1 -- 172.21.15.89:0/1087279396 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fab500869a0 con 0x7fab60069560 2023-10-10T20:34:46.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.835+0000 7fab5e7fc700 1 --2- 172.21.15.89:0/1087279396 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab40066f30 0x7fab400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:46.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.836+0000 7fab5e7fc700 1 --2- 172.21.15.89:0/1087279396 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab40066f30 0x7fab400693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab600d1330 tx=0x7fab48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:46.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:46.840+0000 7fab56ffd700 1 -- 172.21.15.89:0/1087279396 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab50053c10 con 0x7fab60069560 2023-10-10T20:34:47.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.023+0000 7fab65985700 1 -- 172.21.15.89:0/1087279396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab60064e70 con 0x7fab60069560 2023-10-10T20:34:47.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.023+0000 7fab56ffd700 1 -- 172.21.15.89:0/1087279396 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab50057230 con 0x7fab60069560 2023-10-10T20:34:47.026 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:34:47.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.025+0000 7fab65985700 1 -- 172.21.15.89:0/1087279396 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab40066f30 msgr2=0x7fab400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:47.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.025+0000 7fab65985700 1 --2- 172.21.15.89:0/1087279396 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab40066f30 0x7fab400693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab600d1330 tx=0x7fab48009040 comp rx=0 tx=0).stop 2023-10-10T20:34:47.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.025+0000 7fab65985700 1 -- 172.21.15.89:0/1087279396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab60069560 msgr2=0x7fab601493a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:47.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.025+0000 7fab65985700 1 --2- 172.21.15.89:0/1087279396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab60069560 0x7fab601493a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fab600899c0 tx=0x7fab50017000 comp rx=0 tx=0).stop 2023-10-10T20:34:47.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.025+0000 7fab65985700 1 -- 172.21.15.89:0/1087279396 shutdown_connections 2023-10-10T20:34:47.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.025+0000 7fab65985700 1 --2- 172.21.15.89:0/1087279396 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab40066f30 0x7fab400693e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.025+0000 7fab65985700 1 --2- 172.21.15.89:0/1087279396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab60159c60 0x7fab60151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.025+0000 7fab65985700 1 --2- 172.21.15.89:0/1087279396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab60069560 0x7fab601493a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.025+0000 7fab65985700 1 --2- 172.21.15.89:0/1087279396 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab60159890 0x7fab6016a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.025+0000 7fab65985700 1 -- 172.21.15.89:0/1087279396 >> 172.21.15.89:0/1087279396 conn(0x7fab60068e20 msgr2=0x7fab60149a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:47.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.025+0000 7fab65985700 1 -- 172.21.15.89:0/1087279396 shutdown_connections 2023-10-10T20:34:47.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.025+0000 7fab65985700 1 -- 172.21.15.89:0/1087279396 wait complete. 2023-10-10T20:34:47.040 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:34:47.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.155+0000 7f7dc1f79700 1 Processor -- start 2023-10-10T20:34:47.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.155+0000 7f7dc1f79700 1 -- start start 2023-10-10T20:34:47.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.156+0000 7f7dc1f79700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dbc06bcd0 0x7f7dbc0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.156+0000 7f7dc1f79700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7dbc15df50 0x7f7dbc15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.156+0000 7f7dc1f79700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dbc15d7e0 con 0x7f7dbc06c0a0 2023-10-10T20:34:47.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.156+0000 7f7dc1f79700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dbc15dac0 con 0x7f7dbc15df50 2023-10-10T20:34:47.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.156+0000 7f7dc1f79700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc15aa30 con 0x7f7dbc06bcd0 2023-10-10T20:34:47.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.156+0000 7f7dbbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7dbc15df50 0x7f7dbc15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.156+0000 7f7dbbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7dbc15df50 0x7f7dbc15e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53726/0 (socket says 172.21.15.89:53726) 2023-10-10T20:34:47.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.156+0000 7f7dbbfff700 1 -- 172.21.15.89:0/2444756624 learned_addr learned my addr 172.21.15.89:0/2444756624 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:47.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.156+0000 7f7dbb7fe700 1 --2- 172.21.15.89:0/2444756624 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dbc06bcd0 0x7f7dbc0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.157+0000 7f7dbbfff700 1 -- 172.21.15.89:0/2444756624 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dbc06bcd0 msgr2=0x7f7dbc0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:47.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.157+0000 7f7dbbfff700 1 --2- 172.21.15.89:0/2444756624 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dbc06bcd0 0x7f7dbc0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.157+0000 7f7dbbfff700 1 -- 172.21.15.89:0/2444756624 >> v1:172.21.15.111:6789/0 conn(0x7f7dbc06c0a0 legacy=0x7f7dbc15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:47.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.157+0000 7f7dbbfff700 1 -- 172.21.15.89:0/2444756624 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc15ad10 con 0x7f7dbc15df50 2023-10-10T20:34:47.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.157+0000 7f7dbbfff700 1 --2- 172.21.15.89:0/2444756624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dbc15df50 0x7f7dbc15e3b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7db4009f20 tx=0x7f7db4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d737380df7a8cb4b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:47.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.157+0000 7f7dba7fc700 1 -- 172.21.15.89:0/2444756624 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db400e070 con 0x7f7dbc15df50 2023-10-10T20:34:47.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.157+0000 7f7dba7fc700 1 -- 172.21.15.89:0/2444756624 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db400b9d0 con 0x7f7dbc15df50 2023-10-10T20:34:47.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.157+0000 7f7dba7fc700 1 -- 172.21.15.89:0/2444756624 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db40092e0 con 0x7f7dbc15df50 2023-10-10T20:34:47.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.157+0000 7f7dc1f79700 1 -- 172.21.15.89:0/2444756624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dbc15df50 msgr2=0x7f7dbc15e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:47.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.157+0000 7f7dc1f79700 1 --2- 172.21.15.89:0/2444756624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dbc15df50 0x7f7dbc15e3b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7db4009f20 tx=0x7f7db4009840 comp rx=0 tx=0).stop 2023-10-10T20:34:47.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.157+0000 7f7dc1f79700 1 -- 172.21.15.89:0/2444756624 shutdown_connections 2023-10-10T20:34:47.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.157+0000 7f7dc1f79700 1 --2- 172.21.15.89:0/2444756624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dbc15df50 0x7f7dbc15e3b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.157+0000 7f7dc1f79700 1 --2- 172.21.15.89:0/2444756624 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dbc06bcd0 0x7f7dbc0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.157+0000 7f7dc1f79700 1 -- 172.21.15.89:0/2444756624 >> 172.21.15.89:0/2444756624 conn(0x7f7dbc0ce600 msgr2=0x7f7dbc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:47.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.158+0000 7f7dc1f79700 1 -- 172.21.15.89:0/2444756624 shutdown_connections 2023-10-10T20:34:47.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.158+0000 7f7dc1f79700 1 -- 172.21.15.89:0/2444756624 wait complete. 2023-10-10T20:34:47.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.159+0000 7f7dc1f79700 1 Processor -- start 2023-10-10T20:34:47.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.159+0000 7f7dc1f79700 1 -- start start 2023-10-10T20:34:47.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.159+0000 7f7dc1f79700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dbc06bcd0 0x7f7dbc1546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.159+0000 7f7dc1f79700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dbc06c0a0 0x7f7dbc069a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7dc1f79700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dbc15df50 0x7f7dbc069f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7dc1f79700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc0b1de0 con 0x7f7dbc06bcd0 2023-10-10T20:34:47.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7dc1f79700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc15a040 con 0x7f7dbc06c0a0 2023-10-10T20:34:47.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7dc1f79700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc159430 con 0x7f7dbc15df50 2023-10-10T20:34:47.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7dbb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dbc06bcd0 0x7f7dbc1546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7dbb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dbc06bcd0 0x7f7dbc1546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53734/0 (socket says 172.21.15.89:53734) 2023-10-10T20:34:47.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7dbb7fe700 1 -- 172.21.15.89:0/4268096245 learned_addr learned my addr 172.21.15.89:0/4268096245 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:47.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7dbaffd700 1 --2- 172.21.15.89:0/4268096245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dbc06c0a0 0x7f7dbc069a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7dbbfff700 1 --2- 172.21.15.89:0/4268096245 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dbc15df50 0x7f7dbc069f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7dbb7fe700 1 -- 172.21.15.89:0/4268096245 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dbc15df50 msgr2=0x7f7dbc069f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:47.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7dbb7fe700 1 --2- 172.21.15.89:0/4268096245 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dbc15df50 0x7f7dbc069f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7dbb7fe700 1 -- 172.21.15.89:0/4268096245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dbc06c0a0 msgr2=0x7f7dbc069a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:47.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7dbb7fe700 1 --2- 172.21.15.89:0/4268096245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dbc06c0a0 0x7f7dbc069a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7dbb7fe700 1 -- 172.21.15.89:0/4268096245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db4009ae0 con 0x7f7dbc06bcd0 2023-10-10T20:34:47.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7dbb7fe700 1 --2- 172.21.15.89:0/4268096245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dbc06bcd0 0x7f7dbc1546d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00e960 tx=0x7f7dac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:47.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7da37fe700 1 -- 172.21.15.89:0/4268096245 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac013680 con 0x7f7dbc06bcd0 2023-10-10T20:34:47.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7da37fe700 1 -- 172.21.15.89:0/4268096245 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dac0137e0 con 0x7f7dbc06bcd0 2023-10-10T20:34:47.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7dc1f79700 1 -- 172.21.15.89:0/4268096245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dbc159710 con 0x7f7dbc06bcd0 2023-10-10T20:34:47.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7da37fe700 1 -- 172.21.15.89:0/4268096245 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac013b80 con 0x7f7dbc06bcd0 2023-10-10T20:34:47.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.160+0000 7f7dc1f79700 1 -- 172.21.15.89:0/4268096245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dbc159cd0 con 0x7f7dbc06bcd0 2023-10-10T20:34:47.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.161+0000 7f7dc1f79700 1 -- 172.21.15.89:0/4268096245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dbc04c3c0 con 0x7f7dbc06bcd0 2023-10-10T20:34:47.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.162+0000 7f7da37fe700 1 -- 172.21.15.89:0/4268096245 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7dac0123d0 con 0x7f7dbc06bcd0 2023-10-10T20:34:47.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.162+0000 7f7da37fe700 1 --2- 172.21.15.89:0/4268096245 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7da8066f30 0x7f7da80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.162+0000 7f7dbaffd700 1 --2- 172.21.15.89:0/4268096245 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7da8066f30 0x7f7da80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.162+0000 7f7da37fe700 1 -- 172.21.15.89:0/4268096245 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7dac084620 con 0x7f7dbc06bcd0 2023-10-10T20:34:47.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.162+0000 7f7dbaffd700 1 --2- 172.21.15.89:0/4268096245 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7da8066f30 0x7f7da80693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0c8740 tx=0x7f7db0007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:47.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.166+0000 7f7da37fe700 1 -- 172.21.15.89:0/4268096245 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7dac051890 con 0x7f7dbc06bcd0 2023-10-10T20:34:47.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.348+0000 7f7dc1f79700 1 -- 172.21.15.89:0/4268096245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7dbc15aff0 con 0x7f7dbc06bcd0 2023-10-10T20:34:47.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.349+0000 7f7da37fe700 1 -- 172.21.15.89:0/4268096245 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7dac054eb0 con 0x7f7dbc06bcd0 2023-10-10T20:34:47.352 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:34:47.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.351+0000 7f7dc1f79700 1 -- 172.21.15.89:0/4268096245 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7da8066f30 msgr2=0x7f7da80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:47.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.351+0000 7f7dc1f79700 1 --2- 172.21.15.89:0/4268096245 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7da8066f30 0x7f7da80693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0c8740 tx=0x7f7db0007ec0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.351+0000 7f7dc1f79700 1 -- 172.21.15.89:0/4268096245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dbc06bcd0 msgr2=0x7f7dbc1546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:47.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.351+0000 7f7dc1f79700 1 --2- 172.21.15.89:0/4268096245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dbc06bcd0 0x7f7dbc1546d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00e960 tx=0x7f7dac0091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.351+0000 7f7dc1f79700 1 -- 172.21.15.89:0/4268096245 shutdown_connections 2023-10-10T20:34:47.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.351+0000 7f7dc1f79700 1 --2- 172.21.15.89:0/4268096245 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dbc15df50 0x7f7dbc069f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.351+0000 7f7dc1f79700 1 --2- 172.21.15.89:0/4268096245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dbc06bcd0 0x7f7dbc1546d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.351+0000 7f7dc1f79700 1 --2- 172.21.15.89:0/4268096245 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7da8066f30 0x7f7da80693e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.351+0000 7f7dc1f79700 1 --2- 172.21.15.89:0/4268096245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dbc06c0a0 0x7f7dbc069a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.351+0000 7f7dc1f79700 1 -- 172.21.15.89:0/4268096245 >> 172.21.15.89:0/4268096245 conn(0x7f7dbc0ce600 msgr2=0x7f7dbc154f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:47.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.351+0000 7f7dc1f79700 1 -- 172.21.15.89:0/4268096245 shutdown_connections 2023-10-10T20:34:47.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.351+0000 7f7dc1f79700 1 -- 172.21.15.89:0/4268096245 wait complete. 2023-10-10T20:34:47.365 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:34:47.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.480+0000 7f23ef730700 1 Processor -- start 2023-10-10T20:34:47.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.480+0000 7f23ef730700 1 -- start start 2023-10-10T20:34:47.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.481+0000 7f23ef730700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23e80caa90 0x7f23e80c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.481+0000 7f23ef730700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e815d7e0 con 0x7f23e806bcd0 2023-10-10T20:34:47.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.481+0000 7f23ef730700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e815dac0 con 0x7f23e806c0a0 2023-10-10T20:34:47.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.481+0000 7f23ef730700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e80caf30 con 0x7f23e80caa90 2023-10-10T20:34:47.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.481+0000 7f23ed4cc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f23e806bcd0 0x7f23e815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35802/0 (socket says 172.21.15.89:35802) 2023-10-10T20:34:47.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.481+0000 7f23ed4cc700 1 -- 172.21.15.89:0/2254536883 learned_addr learned my addr 172.21.15.89:0/2254536883 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:47.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.482+0000 7f23edccd700 1 --2- 172.21.15.89:0/2254536883 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23e80caa90 0x7f23e80c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.482+0000 7f23dffff700 1 -- 172.21.15.89:0/2254536883 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244250814 0 0) 0x7f23e815d7e0 con 0x7f23e806bcd0 2023-10-10T20:34:47.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.482+0000 7f23dffff700 1 -- 172.21.15.89:0/2254536883 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23d4003570 con 0x7f23e806bcd0 2023-10-10T20:34:47.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.482+0000 7f23dffff700 1 -- 172.21.15.89:0/2254536883 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2880108129 0 0) 0x7f23e815dac0 con 0x7f23e806c0a0 2023-10-10T20:34:47.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.482+0000 7f23dffff700 1 -- 172.21.15.89:0/2254536883 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23e815d7e0 con 0x7f23e806c0a0 2023-10-10T20:34:47.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.482+0000 7f23dffff700 1 -- 172.21.15.89:0/2254536883 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4021753329 0 0) 0x7f23d4003570 con 0x7f23e806bcd0 2023-10-10T20:34:47.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.482+0000 7f23edccd700 1 -- 172.21.15.89:0/2254536883 >> v1:172.21.15.111:6789/0 conn(0x7f23e806c0a0 legacy=0x7f23e815df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:47.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.482+0000 7f23edccd700 1 -- 172.21.15.89:0/2254536883 >> v1:172.21.15.89:6789/0 conn(0x7f23e806bcd0 legacy=0x7f23e815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:47.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.482+0000 7f23edccd700 1 -- 172.21.15.89:0/2254536883 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e815aa30 con 0x7f23e80caa90 2023-10-10T20:34:47.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.482+0000 7f23edccd700 1 --2- 172.21.15.89:0/2254536883 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23e80caa90 0x7f23e80c8c40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009910 tx=0x7f23e00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d53f6985177ae18 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:47.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.482+0000 7f23dffff700 1 -- 172.21.15.89:0/2254536883 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e40017f0 con 0x7f23e80caa90 2023-10-10T20:34:47.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.482+0000 7f23dffff700 1 -- 172.21.15.89:0/2254536883 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e0011420 con 0x7f23e80caa90 2023-10-10T20:34:47.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.482+0000 7f23dffff700 1 -- 172.21.15.89:0/2254536883 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e00117c0 con 0x7f23e80caa90 2023-10-10T20:34:47.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.483+0000 7f23ef730700 1 -- 172.21.15.89:0/2254536883 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23e80caa90 msgr2=0x7f23e80c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:47.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.483+0000 7f23ef730700 1 --2- 172.21.15.89:0/2254536883 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23e80caa90 0x7f23e80c8c40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009910 tx=0x7f23e00091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.483+0000 7f23ef730700 1 -- 172.21.15.89:0/2254536883 shutdown_connections 2023-10-10T20:34:47.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.483+0000 7f23ef730700 1 --2- 172.21.15.89:0/2254536883 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23e80caa90 0x7f23e80c8c40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.483+0000 7f23ef730700 1 -- 172.21.15.89:0/2254536883 >> 172.21.15.89:0/2254536883 conn(0x7f23e80ce600 msgr2=0x7f23e80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:47.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.483+0000 7f23ef730700 1 -- 172.21.15.89:0/2254536883 shutdown_connections 2023-10-10T20:34:47.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.483+0000 7f23ef730700 1 -- 172.21.15.89:0/2254536883 wait complete. 2023-10-10T20:34:47.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.484+0000 7f23ef730700 1 Processor -- start 2023-10-10T20:34:47.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.484+0000 7f23ef730700 1 -- start start 2023-10-10T20:34:47.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.485+0000 7f23ef730700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23e806bcd0 0x7f23e81532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.485+0000 7f23ef730700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23e806c0a0 0x7f23e80b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.485+0000 7f23ef730700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23e80caa90 0x7f23e80b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.485+0000 7f23ef730700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e80b2fc0 con 0x7f23e806bcd0 2023-10-10T20:34:47.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.485+0000 7f23ef730700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e80b2720 con 0x7f23e80caa90 2023-10-10T20:34:47.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.485+0000 7f23ef730700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e80b2a00 con 0x7f23e806c0a0 2023-10-10T20:34:47.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.485+0000 7f23ed4cc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23e806bcd0 0x7f23e81532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.485+0000 7f23ed4cc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23e806bcd0 0x7f23e81532f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53744/0 (socket says 172.21.15.89:53744) 2023-10-10T20:34:47.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.485+0000 7f23ed4cc700 1 -- 172.21.15.89:0/1316375200 learned_addr learned my addr 172.21.15.89:0/1316375200 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:47.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.485+0000 7f23edccd700 1 --2- 172.21.15.89:0/1316375200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23e80caa90 0x7f23e80b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.485+0000 7f23ecccb700 1 --2- 172.21.15.89:0/1316375200 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23e806c0a0 0x7f23e80b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.485+0000 7f23ed4cc700 1 -- 172.21.15.89:0/1316375200 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23e806c0a0 msgr2=0x7f23e80b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:47.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.485+0000 7f23ed4cc700 1 --2- 172.21.15.89:0/1316375200 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23e806c0a0 0x7f23e80b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.485+0000 7f23ed4cc700 1 -- 172.21.15.89:0/1316375200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23e80caa90 msgr2=0x7f23e80b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:47.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.485+0000 7f23ed4cc700 1 --2- 172.21.15.89:0/1316375200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23e80caa90 0x7f23e80b33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.485+0000 7f23ed4cc700 1 -- 172.21.15.89:0/1316375200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e0009420 con 0x7f23e806bcd0 2023-10-10T20:34:47.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.486+0000 7f23ed4cc700 1 --2- 172.21.15.89:0/1316375200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23e806bcd0 0x7f23e81532f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f23e400cbf0 tx=0x7f23e4007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:47.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.486+0000 7f23dd7fa700 1 -- 172.21.15.89:0/1316375200 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4004dc0 con 0x7f23e806bcd0 2023-10-10T20:34:47.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.486+0000 7f23dd7fa700 1 -- 172.21.15.89:0/1316375200 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e400a660 con 0x7f23e806bcd0 2023-10-10T20:34:47.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.486+0000 7f23dd7fa700 1 -- 172.21.15.89:0/1316375200 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e400aa20 con 0x7f23e806bcd0 2023-10-10T20:34:47.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.486+0000 7f23ef730700 1 -- 172.21.15.89:0/1316375200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e80b2ce0 con 0x7f23e806bcd0 2023-10-10T20:34:47.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.486+0000 7f23ef730700 1 -- 172.21.15.89:0/1316375200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e8153830 con 0x7f23e806bcd0 2023-10-10T20:34:47.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.487+0000 7f23ef730700 1 -- 172.21.15.89:0/1316375200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e8157e00 con 0x7f23e806bcd0 2023-10-10T20:34:47.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.487+0000 7f23dd7fa700 1 -- 172.21.15.89:0/1316375200 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f23e4007150 con 0x7f23e806bcd0 2023-10-10T20:34:47.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.487+0000 7f23dd7fa700 1 --2- 172.21.15.89:0/1316375200 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f23cc066f30 0x7f23cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.488+0000 7f23dd7fa700 1 -- 172.21.15.89:0/1316375200 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f23e4083b70 con 0x7f23e806bcd0 2023-10-10T20:34:47.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.488+0000 7f23ecccb700 1 --2- 172.21.15.89:0/1316375200 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f23cc066f30 0x7f23cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.488+0000 7f23ecccb700 1 --2- 172.21.15.89:0/1316375200 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f23cc066f30 0x7f23cc0693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f23d8007fd0 tx=0x7f23d8007d10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:47.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.493+0000 7f23dd7fa700 1 -- 172.21.15.89:0/1316375200 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23e8157e00 con 0x7f23e806bcd0 2023-10-10T20:34:47.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.675+0000 7f23ef730700 1 -- 172.21.15.89:0/1316375200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23e8157e00 con 0x7f23e806bcd0 2023-10-10T20:34:47.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.675+0000 7f23dd7fa700 1 -- 172.21.15.89:0/1316375200 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f23e8157e00 con 0x7f23e806bcd0 2023-10-10T20:34:47.678 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-10T20:34:47.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.677+0000 7f23ef730700 1 -- 172.21.15.89:0/1316375200 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f23cc066f30 msgr2=0x7f23cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:47.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.677+0000 7f23ef730700 1 --2- 172.21.15.89:0/1316375200 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f23cc066f30 0x7f23cc0693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f23d8007fd0 tx=0x7f23d8007d10 comp rx=0 tx=0).stop 2023-10-10T20:34:47.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.677+0000 7f23ef730700 1 -- 172.21.15.89:0/1316375200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23e806bcd0 msgr2=0x7f23e81532f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:47.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.677+0000 7f23ef730700 1 --2- 172.21.15.89:0/1316375200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23e806bcd0 0x7f23e81532f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f23e400cbf0 tx=0x7f23e4007da0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.677+0000 7f23ef730700 1 -- 172.21.15.89:0/1316375200 shutdown_connections 2023-10-10T20:34:47.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.677+0000 7f23ef730700 1 --2- 172.21.15.89:0/1316375200 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f23cc066f30 0x7f23cc0693e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.677+0000 7f23ef730700 1 --2- 172.21.15.89:0/1316375200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23e80caa90 0x7f23e80b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.677+0000 7f23ef730700 1 --2- 172.21.15.89:0/1316375200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23e806bcd0 0x7f23e81532f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.677+0000 7f23ef730700 1 --2- 172.21.15.89:0/1316375200 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23e806c0a0 0x7f23e80b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.677+0000 7f23ef730700 1 -- 172.21.15.89:0/1316375200 >> 172.21.15.89:0/1316375200 conn(0x7f23e80ce600 msgr2=0x7f23e806a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:47.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.677+0000 7f23ef730700 1 -- 172.21.15.89:0/1316375200 shutdown_connections 2023-10-10T20:34:47.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.677+0000 7f23ef730700 1 -- 172.21.15.89:0/1316375200 wait complete. 2023-10-10T20:34:47.691 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:34:47.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.803+0000 7fe033fff700 1 Processor -- start 2023-10-10T20:34:47.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.803+0000 7fe033fff700 1 -- start start 2023-10-10T20:34:47.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.804+0000 7fe033fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe034159420 0x7fe0340b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.804+0000 7fe033fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0341597f0 0x7fe0340b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.804+0000 7fe033fff700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0340b11d0 con 0x7fe03415d230 2023-10-10T20:34:47.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.804+0000 7fe033fff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0340b14b0 con 0x7fe034159420 2023-10-10T20:34:47.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.804+0000 7fe033fff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0340b1790 con 0x7fe0341597f0 2023-10-10T20:34:47.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.804+0000 7fe0337fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe03415d230 0x7fe0340b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35806/0 (socket says 172.21.15.89:35806) 2023-10-10T20:34:47.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.804+0000 7fe0337fe700 1 -- 172.21.15.89:0/2878277813 learned_addr learned my addr 172.21.15.89:0/2878277813 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:47.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.804+0000 7fe0327fc700 1 --2- 172.21.15.89:0/2878277813 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0341597f0 0x7fe0340b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.804+0000 7fe032ffd700 1 --2- 172.21.15.89:0/2878277813 >> v2:172.21.15.111:3300/0 conn(0x7fe034159420 0x7fe0340b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.804+0000 7fe031ffb700 1 -- 172.21.15.89:0/2878277813 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3820750451 0 0) 0x7fe0340b11d0 con 0x7fe03415d230 2023-10-10T20:34:47.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.804+0000 7fe031ffb700 1 -- 172.21.15.89:0/2878277813 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe01c003570 con 0x7fe03415d230 2023-10-10T20:34:47.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe031ffb700 1 -- 172.21.15.89:0/2878277813 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 136305632 0 0) 0x7fe01c003570 con 0x7fe03415d230 2023-10-10T20:34:47.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe031ffb700 1 -- 172.21.15.89:0/2878277813 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0341597f0 msgr2=0x7fe0340b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:47.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe031ffb700 1 --2- 172.21.15.89:0/2878277813 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0341597f0 0x7fe0340b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe031ffb700 1 -- 172.21.15.89:0/2878277813 >> v2:172.21.15.111:3300/0 conn(0x7fe034159420 msgr2=0x7fe0340b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:47.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe031ffb700 1 --2- 172.21.15.89:0/2878277813 >> v2:172.21.15.111:3300/0 conn(0x7fe034159420 0x7fe0340b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe031ffb700 1 -- 172.21.15.89:0/2878277813 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0340b1a70 con 0x7fe03415d230 2023-10-10T20:34:47.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe031ffb700 1 -- 172.21.15.89:0/2878277813 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fe02c001470 con 0x7fe03415d230 2023-10-10T20:34:47.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe031ffb700 1 -- 172.21.15.89:0/2878277813 >> v1:172.21.15.89:6789/0 conn(0x7fe03415d230 legacy=0x7fe0340b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:47.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe031ffb700 1 --2- 172.21.15.89:0/2878277813 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0341597f0 0x7fe01c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe031ffb700 1 --2- 172.21.15.89:0/2878277813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe01c003fe0 0x7fe01c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe031ffb700 1 --2- 172.21.15.89:0/2878277813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe01c004d50 0x7fe01c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe031ffb700 1 -- 172.21.15.89:0/2878277813 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0340b14b0 con 0x7fe01c004d50 2023-10-10T20:34:47.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe031ffb700 1 -- 172.21.15.89:0/2878277813 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0340b1790 con 0x7fe01c003fe0 2023-10-10T20:34:47.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe031ffb700 1 -- 172.21.15.89:0/2878277813 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c00ba90 con 0x7fe0341597f0 2023-10-10T20:34:47.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe0337fe700 1 --2- 172.21.15.89:0/2878277813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe01c004d50 0x7fe01c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe032ffd700 1 --2- 172.21.15.89:0/2878277813 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0341597f0 0x7fe01c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe0327fc700 1 --2- 172.21.15.89:0/2878277813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe01c003fe0 0x7fe01c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe0337fe700 1 -- 172.21.15.89:0/2878277813 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0341597f0 msgr2=0x7fe01c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:47.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe0337fe700 1 --2- 172.21.15.89:0/2878277813 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe0341597f0 0x7fe01c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe0337fe700 1 -- 172.21.15.89:0/2878277813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe01c003fe0 msgr2=0x7fe01c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:47.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe0337fe700 1 --2- 172.21.15.89:0/2878277813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe01c003fe0 0x7fe01c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe0337fe700 1 -- 172.21.15.89:0/2878277813 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c003570 con 0x7fe01c004d50 2023-10-10T20:34:47.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe0337fe700 1 --2- 172.21.15.89:0/2878277813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe01c004d50 0x7fe01c0051c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00c3c0 tx=0x7fe02c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697c9e3042f9fe52 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:47.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe031ffb700 1 -- 172.21.15.89:0/2878277813 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c00cc30 con 0x7fe01c004d50 2023-10-10T20:34:47.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe031ffb700 1 -- 172.21.15.89:0/2878277813 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02c00cdb0 con 0x7fe01c004d50 2023-10-10T20:34:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.805+0000 7fe031ffb700 1 -- 172.21.15.89:0/2878277813 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c005b70 con 0x7fe01c004d50 2023-10-10T20:34:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.806+0000 7fe033fff700 1 -- 172.21.15.89:0/2878277813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe01c004d50 msgr2=0x7fe01c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.806+0000 7fe033fff700 1 --2- 172.21.15.89:0/2878277813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe01c004d50 0x7fe01c0051c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00c3c0 tx=0x7fe02c001e90 comp rx=0 tx=0).stop 2023-10-10T20:34:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.806+0000 7fe0337fe700 1 -- 172.21.15.89:0/2878277813 reap_dead start 2023-10-10T20:34:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.806+0000 7fe033fff700 1 -- 172.21.15.89:0/2878277813 shutdown_connections 2023-10-10T20:34:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.806+0000 7fe033fff700 1 -- 172.21.15.89:0/2878277813 >> 172.21.15.89:0/2878277813 conn(0x7fe0340d0660 msgr2=0x7fe0340d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.806+0000 7fe033fff700 1 -- 172.21.15.89:0/2878277813 shutdown_connections 2023-10-10T20:34:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.806+0000 7fe033fff700 1 -- 172.21.15.89:0/2878277813 wait complete. 2023-10-10T20:34:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.807+0000 7fe033fff700 1 Processor -- start 2023-10-10T20:34:47.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.807+0000 7fe033fff700 1 -- start start 2023-10-10T20:34:47.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.808+0000 7fe033fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe01c004d50 0x7fe0341697e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.808+0000 7fe033fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe034150cf0 0x7fe034151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.808+0000 7fe033fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0341529e0 0x7fe034152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.808+0000 7fe033fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03415c440 con 0x7fe0341529e0 2023-10-10T20:34:47.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.808+0000 7fe033fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe034150b00 con 0x7fe034150cf0 2023-10-10T20:34:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.808+0000 7fe033fff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0341516a0 con 0x7fe01c004d50 2023-10-10T20:34:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.808+0000 7fe0337fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0341529e0 0x7fe034152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.808+0000 7fe0337fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0341529e0 0x7fe034152e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53754/0 (socket says 172.21.15.89:53754) 2023-10-10T20:34:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.808+0000 7fe0337fe700 1 -- 172.21.15.89:0/1088173076 learned_addr learned my addr 172.21.15.89:0/1088173076 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.808+0000 7fe032ffd700 1 --2- 172.21.15.89:0/1088173076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe01c004d50 0x7fe0341697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.808+0000 7fe0327fc700 1 --2- 172.21.15.89:0/1088173076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe034150cf0 0x7fe034151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.808+0000 7fe0337fe700 1 -- 172.21.15.89:0/1088173076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe01c004d50 msgr2=0x7fe0341697e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:47.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.808+0000 7fe0337fe700 1 --2- 172.21.15.89:0/1088173076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe01c004d50 0x7fe0341697e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.808+0000 7fe0337fe700 1 -- 172.21.15.89:0/1088173076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe034150cf0 msgr2=0x7fe034151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:47.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.808+0000 7fe0337fe700 1 --2- 172.21.15.89:0/1088173076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe034150cf0 0x7fe034151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:47.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.808+0000 7fe0337fe700 1 -- 172.21.15.89:0/1088173076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe034151940 con 0x7fe0341529e0 2023-10-10T20:34:47.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.808+0000 7fe0337fe700 1 --2- 172.21.15.89:0/1088173076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0341529e0 0x7fe034152e50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe02c000c00 tx=0x7fe02c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:47.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.809+0000 7fe01affd700 1 -- 172.21.15.89:0/1088173076 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c00cc30 con 0x7fe0341529e0 2023-10-10T20:34:47.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.809+0000 7fe01affd700 1 -- 172.21.15.89:0/1088173076 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02c00c9e0 con 0x7fe0341529e0 2023-10-10T20:34:47.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.809+0000 7fe033fff700 1 -- 172.21.15.89:0/1088173076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe034151c20 con 0x7fe0341529e0 2023-10-10T20:34:47.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.809+0000 7fe01affd700 1 -- 172.21.15.89:0/1088173076 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c001430 con 0x7fe0341529e0 2023-10-10T20:34:47.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.809+0000 7fe033fff700 1 -- 172.21.15.89:0/1088173076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0341521e0 con 0x7fe0341529e0 2023-10-10T20:34:47.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.810+0000 7fe033fff700 1 -- 172.21.15.89:0/1088173076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03404c3c0 con 0x7fe0341529e0 2023-10-10T20:34:47.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.810+0000 7fe01affd700 1 -- 172.21.15.89:0/1088173076 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe02c019030 con 0x7fe0341529e0 2023-10-10T20:34:47.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.810+0000 7fe01affd700 1 --2- 172.21.15.89:0/1088173076 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe020066f30 0x7fe0200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:47.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.810+0000 7fe01affd700 1 -- 172.21.15.89:0/1088173076 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe02c084c40 con 0x7fe0341529e0 2023-10-10T20:34:47.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.810+0000 7fe032ffd700 1 --2- 172.21.15.89:0/1088173076 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe020066f30 0x7fe0200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:47.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.811+0000 7fe032ffd700 1 --2- 172.21.15.89:0/1088173076 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe020066f30 0x7fe0200693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe024009fd0 tx=0x7fe02400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:47.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:47.814+0000 7fe01affd700 1 -- 172.21.15.89:0/1088173076 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe02c051eb0 con 0x7fe0341529e0 2023-10-10T20:34:48.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.040+0000 7fe033fff700 1 -- 172.21.15.89:0/1088173076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0340b1d50 con 0x7fe0341529e0 2023-10-10T20:34:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.040+0000 7fe01affd700 1 -- 172.21.15.89:0/1088173076 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4597 (secure 0 0 0) 0x7fe02c0554d0 con 0x7fe0341529e0 2023-10-10T20:34:48.043 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:48.043 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2341577144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2341577144},{"type":"v1","addr":"172.21.15.111:6835","nonce":2341577144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2773123076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2773123076},{"type":"v1","addr":"172.21.15.89:6837","nonce":2773123076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2940044414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2940044414},{"type":"v1","addr":"172.21.15.89:6835","nonce":2940044414}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:34:34.717750+0000","modified":"2023-10-10T20:34:38.356048+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1112081320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1112081320},{"type":"v1","addr":"172.21.15.111:6834","nonce":1112081320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-10T20:34:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.042+0000 7fe033fff700 1 -- 172.21.15.89:0/1088173076 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe020066f30 msgr2=0x7fe0200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.042+0000 7fe033fff700 1 --2- 172.21.15.89:0/1088173076 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe020066f30 0x7fe0200693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe024009fd0 tx=0x7fe02400b040 comp rx=0 tx=0).stop 2023-10-10T20:34:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.042+0000 7fe033fff700 1 -- 172.21.15.89:0/1088173076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0341529e0 msgr2=0x7fe034152e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.042+0000 7fe033fff700 1 --2- 172.21.15.89:0/1088173076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0341529e0 0x7fe034152e50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe02c000c00 tx=0x7fe02c015000 comp rx=0 tx=0).stop 2023-10-10T20:34:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.042+0000 7fe033fff700 1 -- 172.21.15.89:0/1088173076 shutdown_connections 2023-10-10T20:34:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.042+0000 7fe033fff700 1 --2- 172.21.15.89:0/1088173076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0341529e0 0x7fe034152e50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.042+0000 7fe033fff700 1 --2- 172.21.15.89:0/1088173076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe01c004d50 0x7fe0341697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.042+0000 7fe033fff700 1 --2- 172.21.15.89:0/1088173076 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe020066f30 0x7fe0200693e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.042+0000 7fe033fff700 1 --2- 172.21.15.89:0/1088173076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe034150cf0 0x7fe034151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.042+0000 7fe033fff700 1 -- 172.21.15.89:0/1088173076 >> 172.21.15.89:0/1088173076 conn(0x7fe0340d0660 msgr2=0x7fe0340b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.043+0000 7fe033fff700 1 -- 172.21.15.89:0/1088173076 shutdown_connections 2023-10-10T20:34:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.043+0000 7fe033fff700 1 -- 172.21.15.89:0/1088173076 wait complete. 2023-10-10T20:34:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2023-10-10T20:34:48.057 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:34:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.168+0000 7fbc71f09700 1 Processor -- start 2023-10-10T20:34:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.168+0000 7fbc71f09700 1 -- start start 2023-10-10T20:34:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.169+0000 7fbc71f09700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbc6c1597f0 0x7fbc6c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.169+0000 7fbc71f09700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc6c15d230 0x7fbc6c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.169+0000 7fbc71f09700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc6c15dbb0 con 0x7fbc6c159420 2023-10-10T20:34:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.169+0000 7fbc71f09700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc6c0b11d0 con 0x7fbc6c1597f0 2023-10-10T20:34:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.169+0000 7fbc71f09700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c0b14b0 con 0x7fbc6c15d230 2023-10-10T20:34:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.169+0000 7fbc71708700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc6c15d230 0x7fbc6c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:48.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.169+0000 7fbc6bfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbc6c1597f0 0x7fbc6c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:48.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.169+0000 7fbc71708700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc6c15d230 0x7fbc6c15d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:38978/0 (socket says 172.21.15.89:38978) 2023-10-10T20:34:48.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.169+0000 7fbc6bfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbc6c1597f0 0x7fbc6c0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:49210/0 (socket says 172.21.15.89:49210) 2023-10-10T20:34:48.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.169+0000 7fbc70f07700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fbc6c159420 0x7fbc6c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:60694/0 (socket says 172.21.15.89:60694) 2023-10-10T20:34:48.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.169+0000 7fbc71708700 1 -- 172.21.15.89:0/1148951706 learned_addr learned my addr 172.21.15.89:0/1148951706 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:48.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.169+0000 7fbc71708700 1 -- 172.21.15.89:0/1148951706 >> v2:172.21.15.111:3300/0 conn(0x7fbc6c1597f0 msgr2=0x7fbc6c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:48.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.169+0000 7fbc71708700 1 --2- 172.21.15.89:0/1148951706 >> v2:172.21.15.111:3300/0 conn(0x7fbc6c1597f0 0x7fbc6c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.169+0000 7fbc71708700 1 -- 172.21.15.89:0/1148951706 >> v1:172.21.15.111:6789/0 conn(0x7fbc6c159420 legacy=0x7fbc6c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:48.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.169+0000 7fbc71708700 1 -- 172.21.15.89:0/1148951706 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c0b1790 con 0x7fbc6c15d230 2023-10-10T20:34:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.169+0000 7fbc6b7fe700 1 -- 172.21.15.89:0/1148951706 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4022873713 0 0) 0x7fbc6c15dbb0 con 0x7fbc6c159420 2023-10-10T20:34:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.169+0000 7fbc6bfff700 1 --2- 172.21.15.89:0/1148951706 >> v2:172.21.15.111:3300/0 conn(0x7fbc6c1597f0 0x7fbc6c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.170+0000 7fbc71708700 1 --2- 172.21.15.89:0/1148951706 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc6c15d230 0x7fbc6c15d670 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc540099b0 tx=0x7fbc54009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc5f47f2755f5622 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.170+0000 7fbc6b7fe700 1 -- 172.21.15.89:0/1148951706 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc54009ea0 con 0x7fbc6c15d230 2023-10-10T20:34:48.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.170+0000 7fbc6b7fe700 1 -- 172.21.15.89:0/1148951706 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5400b860 con 0x7fbc6c15d230 2023-10-10T20:34:48.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.170+0000 7fbc6b7fe700 1 -- 172.21.15.89:0/1148951706 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5400bc00 con 0x7fbc6c15d230 2023-10-10T20:34:48.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.170+0000 7fbc71f09700 1 -- 172.21.15.89:0/1148951706 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc6c15d230 msgr2=0x7fbc6c15d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:48.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.170+0000 7fbc71f09700 1 --2- 172.21.15.89:0/1148951706 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc6c15d230 0x7fbc6c15d670 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc540099b0 tx=0x7fbc54009250 comp rx=0 tx=0).stop 2023-10-10T20:34:48.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.170+0000 7fbc71f09700 1 -- 172.21.15.89:0/1148951706 shutdown_connections 2023-10-10T20:34:48.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.170+0000 7fbc71f09700 1 --2- 172.21.15.89:0/1148951706 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc6c15d230 0x7fbc6c15d670 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.170+0000 7fbc71f09700 1 --2- 172.21.15.89:0/1148951706 >> v2:172.21.15.111:3300/0 conn(0x7fbc6c1597f0 0x7fbc6c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.170+0000 7fbc71f09700 1 -- 172.21.15.89:0/1148951706 >> 172.21.15.89:0/1148951706 conn(0x7fbc6c0d0660 msgr2=0x7fbc6c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:48.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.170+0000 7fbc71f09700 1 -- 172.21.15.89:0/1148951706 shutdown_connections 2023-10-10T20:34:48.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.170+0000 7fbc71f09700 1 -- 172.21.15.89:0/1148951706 wait complete. 2023-10-10T20:34:48.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.171+0000 7fbc71f09700 1 Processor -- start 2023-10-10T20:34:48.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.172+0000 7fbc71f09700 1 -- start start 2023-10-10T20:34:48.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.172+0000 7fbc71f09700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc6c159420 0x7fbc6c154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:48.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.172+0000 7fbc71f09700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc6c1597f0 0x7fbc6c169700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:48.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.172+0000 7fbc71f09700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc6c15d230 0x7fbc6c169c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:48.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.172+0000 7fbc71f09700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c166f90 con 0x7fbc6c15d230 2023-10-10T20:34:48.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.172+0000 7fbc71f09700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c14fad0 con 0x7fbc6c159420 2023-10-10T20:34:48.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.172+0000 7fbc71f09700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c159ff0 con 0x7fbc6c1597f0 2023-10-10T20:34:48.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.172+0000 7fbc71708700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc6c15d230 0x7fbc6c169c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:48.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.172+0000 7fbc71708700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc6c15d230 0x7fbc6c169c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44634/0 (socket says 172.21.15.89:44634) 2023-10-10T20:34:48.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.172+0000 7fbc71708700 1 -- 172.21.15.89:0/1571629737 learned_addr learned my addr 172.21.15.89:0/1571629737 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:48.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.172+0000 7fbc6bfff700 1 --2- 172.21.15.89:0/1571629737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc6c1597f0 0x7fbc6c169700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:48.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.173+0000 7fbc70f07700 1 --2- 172.21.15.89:0/1571629737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc6c159420 0x7fbc6c154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:48.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.173+0000 7fbc71708700 1 -- 172.21.15.89:0/1571629737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc6c1597f0 msgr2=0x7fbc6c169700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:48.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.173+0000 7fbc71708700 1 --2- 172.21.15.89:0/1571629737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc6c1597f0 0x7fbc6c169700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.173+0000 7fbc71708700 1 -- 172.21.15.89:0/1571629737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc6c159420 msgr2=0x7fbc6c154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:48.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.173+0000 7fbc71708700 1 --2- 172.21.15.89:0/1571629737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc6c159420 0x7fbc6c154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.173+0000 7fbc71708700 1 -- 172.21.15.89:0/1571629737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5c009a20 con 0x7fbc6c15d230 2023-10-10T20:34:48.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.173+0000 7fbc71708700 1 --2- 172.21.15.89:0/1571629737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc6c15d230 0x7fbc6c169c40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbc54006fd0 tx=0x7fbc54017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:48.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.173+0000 7fbc68ff9700 1 -- 172.21.15.89:0/1571629737 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5400baa0 con 0x7fbc6c15d230 2023-10-10T20:34:48.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.173+0000 7fbc68ff9700 1 -- 172.21.15.89:0/1571629737 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc54025420 con 0x7fbc6c15d230 2023-10-10T20:34:48.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.173+0000 7fbc71f09700 1 -- 172.21.15.89:0/1571629737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc540094f0 con 0x7fbc6c15d230 2023-10-10T20:34:48.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.173+0000 7fbc68ff9700 1 -- 172.21.15.89:0/1571629737 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc540257e0 con 0x7fbc6c15d230 2023-10-10T20:34:48.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.173+0000 7fbc71f09700 1 -- 172.21.15.89:0/1571629737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6c1549f0 con 0x7fbc6c15d230 2023-10-10T20:34:48.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.174+0000 7fbc71f09700 1 -- 172.21.15.89:0/1571629737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6c04c3c0 con 0x7fbc6c15d230 2023-10-10T20:34:48.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.174+0000 7fbc68ff9700 1 -- 172.21.15.89:0/1571629737 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbc54023070 con 0x7fbc6c15d230 2023-10-10T20:34:48.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.175+0000 7fbc68ff9700 1 --2- 172.21.15.89:0/1571629737 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbc4c066f30 0x7fbc4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:48.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.175+0000 7fbc68ff9700 1 -- 172.21.15.89:0/1571629737 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbc54085b70 con 0x7fbc6c15d230 2023-10-10T20:34:48.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.175+0000 7fbc70f07700 1 --2- 172.21.15.89:0/1571629737 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbc4c066f30 0x7fbc4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:48.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.175+0000 7fbc70f07700 1 --2- 172.21.15.89:0/1571629737 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbc4c066f30 0x7fbc4c0693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbc64007fd0 tx=0x7fbc64007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:48.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.179+0000 7fbc68ff9700 1 -- 172.21.15.89:0/1571629737 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc54052de0 con 0x7fbc6c15d230 2023-10-10T20:34:48.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.364+0000 7fbc71f09700 1 -- 172.21.15.89:0/1571629737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc6c15a130 con 0x7fbc6c15d230 2023-10-10T20:34:48.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.365+0000 7fbc68ff9700 1 -- 172.21.15.89:0/1571629737 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fbc54056400 con 0x7fbc6c15d230 2023-10-10T20:34:48.367 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:48.368 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:36.324920+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:34:32.342167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:34:33.671763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:34:48.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.367+0000 7fbc71f09700 1 -- 172.21.15.89:0/1571629737 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbc4c066f30 msgr2=0x7fbc4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:48.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.367+0000 7fbc71f09700 1 --2- 172.21.15.89:0/1571629737 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbc4c066f30 0x7fbc4c0693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbc64007fd0 tx=0x7fbc64007d50 comp rx=0 tx=0).stop 2023-10-10T20:34:48.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.367+0000 7fbc71f09700 1 -- 172.21.15.89:0/1571629737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc6c15d230 msgr2=0x7fbc6c169c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:48.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.367+0000 7fbc71f09700 1 --2- 172.21.15.89:0/1571629737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc6c15d230 0x7fbc6c169c40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbc54006fd0 tx=0x7fbc54017000 comp rx=0 tx=0).stop 2023-10-10T20:34:48.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.367+0000 7fbc71f09700 1 -- 172.21.15.89:0/1571629737 shutdown_connections 2023-10-10T20:34:48.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.367+0000 7fbc71f09700 1 --2- 172.21.15.89:0/1571629737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc6c15d230 0x7fbc6c169c40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.367+0000 7fbc71f09700 1 --2- 172.21.15.89:0/1571629737 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbc4c066f30 0x7fbc4c0693e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.367+0000 7fbc71f09700 1 --2- 172.21.15.89:0/1571629737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc6c159420 0x7fbc6c154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.367+0000 7fbc71f09700 1 --2- 172.21.15.89:0/1571629737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc6c1597f0 0x7fbc6c169700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.367+0000 7fbc71f09700 1 -- 172.21.15.89:0/1571629737 >> 172.21.15.89:0/1571629737 conn(0x7fbc6c0d0660 msgr2=0x7fbc6c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:48.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.367+0000 7fbc71f09700 1 -- 172.21.15.89:0/1571629737 shutdown_connections 2023-10-10T20:34:48.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.367+0000 7fbc71f09700 1 -- 172.21.15.89:0/1571629737 wait complete. 2023-10-10T20:34:48.382 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:34:48.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.494+0000 7fc894247700 1 Processor -- start 2023-10-10T20:34:48.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.494+0000 7fc894247700 1 -- start start 2023-10-10T20:34:48.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.495+0000 7fc894247700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc88c0c91e0 0x7fc88c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:48.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.495+0000 7fc894247700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88c06ae10 0x7fc88c06b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:48.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.495+0000 7fc894247700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc88c0cc210 con 0x7fc88c06aa40 2023-10-10T20:34:48.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.495+0000 7fc894247700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc88c0cc4f0 con 0x7fc88c0c91e0 2023-10-10T20:34:48.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.495+0000 7fc894247700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c0cc7d0 con 0x7fc88c06ae10 2023-10-10T20:34:48.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.495+0000 7fc891fe3700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc88c06aa40 0x7fc88c06bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43844/0 (socket says 172.21.15.89:43844) 2023-10-10T20:34:48.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.495+0000 7fc891fe3700 1 -- 172.21.15.89:0/2928690688 learned_addr learned my addr 172.21.15.89:0/2928690688 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:48.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.495+0000 7fc8917e2700 1 --2- 172.21.15.89:0/2928690688 >> v2:172.21.15.89:3300/0 conn(0x7fc88c0c91e0 0x7fc88c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:48.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.496+0000 7fc8927e4700 1 --2- 172.21.15.89:0/2928690688 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88c06ae10 0x7fc88c06b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:48.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.496+0000 7fc890fe1700 1 -- 172.21.15.89:0/2928690688 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3999574752 0 0) 0x7fc88c0cc210 con 0x7fc88c06aa40 2023-10-10T20:34:48.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.496+0000 7fc890fe1700 1 -- 172.21.15.89:0/2928690688 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc874003570 con 0x7fc88c06aa40 2023-10-10T20:34:48.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.496+0000 7fc8917e2700 1 -- 172.21.15.89:0/2928690688 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88c06ae10 msgr2=0x7fc88c06b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:48.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.496+0000 7fc8917e2700 1 --2- 172.21.15.89:0/2928690688 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88c06ae10 0x7fc88c06b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.496+0000 7fc8917e2700 1 -- 172.21.15.89:0/2928690688 >> v1:172.21.15.89:6789/0 conn(0x7fc88c06aa40 legacy=0x7fc88c06bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:48.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.496+0000 7fc8917e2700 1 -- 172.21.15.89:0/2928690688 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c0ccab0 con 0x7fc88c0c91e0 2023-10-10T20:34:48.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.496+0000 7fc8927e4700 1 --2- 172.21.15.89:0/2928690688 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88c06ae10 0x7fc88c06b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:48.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.496+0000 7fc8917e2700 1 --2- 172.21.15.89:0/2928690688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc88c0c91e0 0x7fc88c0cbc00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc880009910 tx=0x7fc8800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aad5bf76c2d9fa7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:48.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.496+0000 7fc890fe1700 1 -- 172.21.15.89:0/2928690688 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88000b640 con 0x7fc88c0c91e0 2023-10-10T20:34:48.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.496+0000 7fc890fe1700 1 -- 172.21.15.89:0/2928690688 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88000b7a0 con 0x7fc88c0c91e0 2023-10-10T20:34:48.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.496+0000 7fc890fe1700 1 -- 172.21.15.89:0/2928690688 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88000bb60 con 0x7fc88c0c91e0 2023-10-10T20:34:48.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.497+0000 7fc894247700 1 -- 172.21.15.89:0/2928690688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc88c0c91e0 msgr2=0x7fc88c0cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:48.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.497+0000 7fc894247700 1 --2- 172.21.15.89:0/2928690688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc88c0c91e0 0x7fc88c0cbc00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc880009910 tx=0x7fc8800091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.497+0000 7fc894247700 1 -- 172.21.15.89:0/2928690688 shutdown_connections 2023-10-10T20:34:48.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.497+0000 7fc894247700 1 --2- 172.21.15.89:0/2928690688 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88c06ae10 0x7fc88c06b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.497+0000 7fc894247700 1 --2- 172.21.15.89:0/2928690688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc88c0c91e0 0x7fc88c0cbc00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.497+0000 7fc894247700 1 -- 172.21.15.89:0/2928690688 >> 172.21.15.89:0/2928690688 conn(0x7fc88c0cd670 msgr2=0x7fc88c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:48.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.497+0000 7fc894247700 1 -- 172.21.15.89:0/2928690688 shutdown_connections 2023-10-10T20:34:48.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.497+0000 7fc894247700 1 -- 172.21.15.89:0/2928690688 wait complete. 2023-10-10T20:34:48.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.498+0000 7fc894247700 1 Processor -- start 2023-10-10T20:34:48.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.498+0000 7fc894247700 1 -- start start 2023-10-10T20:34:48.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc894247700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88c06aa40 0x7fc88c14bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:48.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc894247700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc88c06ae10 0x7fc88c143380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:48.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc894247700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc88c0c91e0 0x7fc88c0b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:48.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc894247700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c14a790 con 0x7fc88c0c91e0 2023-10-10T20:34:48.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc894247700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c16d9d0 con 0x7fc88c06ae10 2023-10-10T20:34:48.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc894247700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c16dcb0 con 0x7fc88c06aa40 2023-10-10T20:34:48.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc8927e4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc88c0c91e0 0x7fc88c0b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:48.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc8927e4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc88c0c91e0 0x7fc88c0b1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44644/0 (socket says 172.21.15.89:44644) 2023-10-10T20:34:48.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc8927e4700 1 -- 172.21.15.89:0/1822289487 learned_addr learned my addr 172.21.15.89:0/1822289487 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:48.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc891fe3700 1 --2- 172.21.15.89:0/1822289487 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88c06aa40 0x7fc88c14bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:48.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc8917e2700 1 --2- 172.21.15.89:0/1822289487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc88c06ae10 0x7fc88c143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:48.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc8927e4700 1 -- 172.21.15.89:0/1822289487 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88c06aa40 msgr2=0x7fc88c14bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:48.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc8927e4700 1 --2- 172.21.15.89:0/1822289487 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88c06aa40 0x7fc88c14bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc8927e4700 1 -- 172.21.15.89:0/1822289487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc88c06ae10 msgr2=0x7fc88c143380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:48.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc8927e4700 1 --2- 172.21.15.89:0/1822289487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc88c06ae10 0x7fc88c143380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc8927e4700 1 -- 172.21.15.89:0/1822289487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc880009420 con 0x7fc88c0c91e0 2023-10-10T20:34:48.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc891fe3700 1 --2- 172.21.15.89:0/1822289487 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88c06aa40 0x7fc88c14bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:48.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc8927e4700 1 --2- 172.21.15.89:0/1822289487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc88c0c91e0 0x7fc88c0b1a60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc88400e870 tx=0x7fc884008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:48.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc889ffb700 1 -- 172.21.15.89:0/1822289487 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88400c430 con 0x7fc88c0c91e0 2023-10-10T20:34:48.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc889ffb700 1 -- 172.21.15.89:0/1822289487 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88400c590 con 0x7fc88c0c91e0 2023-10-10T20:34:48.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc894247700 1 -- 172.21.15.89:0/1822289487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88c16df90 con 0x7fc88c0c91e0 2023-10-10T20:34:48.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.499+0000 7fc889ffb700 1 -- 172.21.15.89:0/1822289487 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88400c910 con 0x7fc88c0c91e0 2023-10-10T20:34:48.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.500+0000 7fc894247700 1 -- 172.21.15.89:0/1822289487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88c14c4e0 con 0x7fc88c0c91e0 2023-10-10T20:34:48.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.500+0000 7fc894247700 1 -- 172.21.15.89:0/1822289487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88c16e430 con 0x7fc88c0c91e0 2023-10-10T20:34:48.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.501+0000 7fc889ffb700 1 -- 172.21.15.89:0/1822289487 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc88400cd40 con 0x7fc88c0c91e0 2023-10-10T20:34:48.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.501+0000 7fc889ffb700 1 --2- 172.21.15.89:0/1822289487 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc878066f30 0x7fc8780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:48.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.501+0000 7fc889ffb700 1 -- 172.21.15.89:0/1822289487 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc8840842d0 con 0x7fc88c0c91e0 2023-10-10T20:34:48.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.501+0000 7fc891fe3700 1 --2- 172.21.15.89:0/1822289487 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc878066f30 0x7fc8780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:48.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.501+0000 7fc891fe3700 1 --2- 172.21.15.89:0/1822289487 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc878066f30 0x7fc8780693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc87c007fd0 tx=0x7fc87c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:48.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.505+0000 7fc889ffb700 1 -- 172.21.15.89:0/1822289487 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc884051540 con 0x7fc88c0c91e0 2023-10-10T20:34:48.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.730+0000 7fc894247700 1 -- 172.21.15.89:0/1822289487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc88c04c3c0 con 0x7fc88c0c91e0 2023-10-10T20:34:48.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.730+0000 7fc889ffb700 1 -- 172.21.15.89:0/1822289487 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4597 (secure 0 0 0) 0x7fc884054b60 con 0x7fc88c0c91e0 2023-10-10T20:34:48.733 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:48.733 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2341577144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2341577144},{"type":"v1","addr":"172.21.15.111:6835","nonce":2341577144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2773123076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2773123076},{"type":"v1","addr":"172.21.15.89:6837","nonce":2773123076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2940044414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2940044414},{"type":"v1","addr":"172.21.15.89:6835","nonce":2940044414}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:34:34.717750+0000","modified":"2023-10-10T20:34:38.356048+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1112081320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1112081320},{"type":"v1","addr":"172.21.15.111:6834","nonce":1112081320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-10T20:34:48.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.733+0000 7fc894247700 1 -- 172.21.15.89:0/1822289487 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc878066f30 msgr2=0x7fc8780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:48.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.733+0000 7fc894247700 1 --2- 172.21.15.89:0/1822289487 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc878066f30 0x7fc8780693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc87c007fd0 tx=0x7fc87c009040 comp rx=0 tx=0).stop 2023-10-10T20:34:48.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.733+0000 7fc894247700 1 -- 172.21.15.89:0/1822289487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc88c0c91e0 msgr2=0x7fc88c0b1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:48.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.733+0000 7fc894247700 1 --2- 172.21.15.89:0/1822289487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc88c0c91e0 0x7fc88c0b1a60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc88400e870 tx=0x7fc884008040 comp rx=0 tx=0).stop 2023-10-10T20:34:48.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.733+0000 7fc894247700 1 -- 172.21.15.89:0/1822289487 shutdown_connections 2023-10-10T20:34:48.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.733+0000 7fc894247700 1 --2- 172.21.15.89:0/1822289487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc88c0c91e0 0x7fc88c0b1a60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.733+0000 7fc894247700 1 --2- 172.21.15.89:0/1822289487 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc88c06aa40 0x7fc88c14bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.733+0000 7fc894247700 1 --2- 172.21.15.89:0/1822289487 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc878066f30 0x7fc8780693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.733+0000 7fc894247700 1 --2- 172.21.15.89:0/1822289487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc88c06ae10 0x7fc88c143380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:48.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.733+0000 7fc894247700 1 -- 172.21.15.89:0/1822289487 >> 172.21.15.89:0/1822289487 conn(0x7fc88c0cd670 msgr2=0x7fc88c14d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:48.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.733+0000 7fc894247700 1 -- 172.21.15.89:0/1822289487 shutdown_connections 2023-10-10T20:34:48.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:48.733+0000 7fc894247700 1 -- 172.21.15.89:0/1822289487 wait complete. 2023-10-10T20:34:48.741 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2023-10-10T20:34:49.149 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:34:34.717750+0000', 'modified': '2023-10-10T20:34:38.356048+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4270}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4270': {'gid': 4270, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6834/1112081320', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6832', 'nonce': 1112081320}, {'type': 'v1', 'addr': '172.21.15.111:6834', 'nonce': 1112081320}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:34:49.149 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:34:49.150 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-10T20:34:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.264+0000 7f5f394af700 1 Processor -- start 2023-10-10T20:34:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.264+0000 7f5f394af700 1 -- start start 2023-10-10T20:34:49.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f394af700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5f34159890 0x7f5f3415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f394af700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f34069560 0x7f5f340699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f394af700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f34069ee0 con 0x7f5f34159c60 2023-10-10T20:34:49.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f394af700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3415a160 con 0x7f5f34159890 2023-10-10T20:34:49.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f394af700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f340d13a0 con 0x7f5f34069560 2023-10-10T20:34:49.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f327fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5f34159c60 0x7f5f3415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43852/0 (socket says 172.21.15.89:43852) 2023-10-10T20:34:49.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f327fc700 1 -- 172.21.15.89:0/1491859751 learned_addr learned my addr 172.21.15.89:0/1491859751 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:49.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f337fe700 1 --2- 172.21.15.89:0/1491859751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f34069560 0x7f5f340699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f32ffd700 1 --2- 172.21.15.89:0/1491859751 >> v2:172.21.15.111:3300/0 conn(0x7f5f34159890 0x7f5f3415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f31ffb700 1 -- 172.21.15.89:0/1491859751 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 752011654 0 0) 0x7f5f34069ee0 con 0x7f5f34159c60 2023-10-10T20:34:49.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f31ffb700 1 -- 172.21.15.89:0/1491859751 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f20003570 con 0x7f5f34159c60 2023-10-10T20:34:49.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f31ffb700 1 -- 172.21.15.89:0/1491859751 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2734452345 0 0) 0x7f5f20003570 con 0x7f5f34159c60 2023-10-10T20:34:49.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f31ffb700 1 -- 172.21.15.89:0/1491859751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f34069560 msgr2=0x7f5f340699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:49.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f31ffb700 1 --2- 172.21.15.89:0/1491859751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f34069560 0x7f5f340699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f31ffb700 1 -- 172.21.15.89:0/1491859751 >> v2:172.21.15.111:3300/0 conn(0x7f5f34159890 msgr2=0x7f5f3415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:49.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f31ffb700 1 --2- 172.21.15.89:0/1491859751 >> v2:172.21.15.111:3300/0 conn(0x7f5f34159890 0x7f5f3415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f31ffb700 1 -- 172.21.15.89:0/1491859751 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3415e140 con 0x7f5f34159c60 2023-10-10T20:34:49.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f31ffb700 1 -- 172.21.15.89:0/1491859751 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f5f1c001470 con 0x7f5f34159c60 2023-10-10T20:34:49.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f31ffb700 1 -- 172.21.15.89:0/1491859751 >> v1:172.21.15.89:6789/0 conn(0x7f5f34159c60 legacy=0x7f5f3415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:49.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f31ffb700 1 --2- 172.21.15.89:0/1491859751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f34069560 0x7f5f20003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f31ffb700 1 --2- 172.21.15.89:0/1491859751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f20003fe0 0x7f5f20004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f31ffb700 1 --2- 172.21.15.89:0/1491859751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f20004d50 0x7f5f200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f31ffb700 1 -- 172.21.15.89:0/1491859751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3415a160 con 0x7f5f20004d50 2023-10-10T20:34:49.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f31ffb700 1 -- 172.21.15.89:0/1491859751 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f340d13a0 con 0x7f5f20003fe0 2023-10-10T20:34:49.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f31ffb700 1 -- 172.21.15.89:0/1491859751 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2000ba90 con 0x7f5f34069560 2023-10-10T20:34:49.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.265+0000 7f5f337fe700 1 --2- 172.21.15.89:0/1491859751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f20004d50 0x7f5f200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.266+0000 7f5f327fc700 1 --2- 172.21.15.89:0/1491859751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f20003fe0 0x7f5f20004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.266+0000 7f5f32ffd700 1 --2- 172.21.15.89:0/1491859751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f34069560 0x7f5f20003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.266+0000 7f5f337fe700 1 -- 172.21.15.89:0/1491859751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f34069560 msgr2=0x7f5f20003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:49.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.266+0000 7f5f337fe700 1 --2- 172.21.15.89:0/1491859751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f34069560 0x7f5f20003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.266+0000 7f5f337fe700 1 -- 172.21.15.89:0/1491859751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f20003fe0 msgr2=0x7f5f20004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:49.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.266+0000 7f5f337fe700 1 --2- 172.21.15.89:0/1491859751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f20003fe0 0x7f5f20004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.266+0000 7f5f337fe700 1 -- 172.21.15.89:0/1491859751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f20003570 con 0x7f5f20004d50 2023-10-10T20:34:49.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.266+0000 7f5f337fe700 1 --2- 172.21.15.89:0/1491859751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f20004d50 0x7f5f200051c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009c00 tx=0x7f5f240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc1589f5e411515 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:49.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.266+0000 7f5f31ffb700 1 -- 172.21.15.89:0/1491859751 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2400e6b0 con 0x7f5f20004d50 2023-10-10T20:34:49.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.266+0000 7f5f31ffb700 1 -- 172.21.15.89:0/1491859751 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2400e810 con 0x7f5f20004d50 2023-10-10T20:34:49.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.266+0000 7f5f31ffb700 1 -- 172.21.15.89:0/1491859751 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2400ebd0 con 0x7f5f20004d50 2023-10-10T20:34:49.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.266+0000 7f5f394af700 1 -- 172.21.15.89:0/1491859751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f20004d50 msgr2=0x7f5f200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:49.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.266+0000 7f5f394af700 1 --2- 172.21.15.89:0/1491859751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f20004d50 0x7f5f200051c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009c00 tx=0x7f5f240091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.266+0000 7f5f337fe700 1 -- 172.21.15.89:0/1491859751 reap_dead start 2023-10-10T20:34:49.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.267+0000 7f5f394af700 1 -- 172.21.15.89:0/1491859751 shutdown_connections 2023-10-10T20:34:49.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.267+0000 7f5f394af700 1 -- 172.21.15.89:0/1491859751 >> 172.21.15.89:0/1491859751 conn(0x7f5f34068e20 msgr2=0x7f5f340d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:49.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.267+0000 7f5f394af700 1 -- 172.21.15.89:0/1491859751 shutdown_connections 2023-10-10T20:34:49.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.267+0000 7f5f394af700 1 -- 172.21.15.89:0/1491859751 wait complete. 2023-10-10T20:34:49.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.268+0000 7f5f394af700 1 Processor -- start 2023-10-10T20:34:49.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.268+0000 7f5f394af700 1 -- start start 2023-10-10T20:34:49.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f394af700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f340b5100 0x7f5f340b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f394af700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f340b54d0 0x7f5f340cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f394af700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f340ce470 0x7f5f340ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f394af700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f340cd5f0 con 0x7f5f340b5100 2023-10-10T20:34:49.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f394af700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f34156570 con 0x7f5f340b54d0 2023-10-10T20:34:49.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f394af700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f340cd800 con 0x7f5f340ce470 2023-10-10T20:34:49.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f32ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f340b5100 0x7f5f340b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f32ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f340b5100 0x7f5f340b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44662/0 (socket says 172.21.15.89:44662) 2023-10-10T20:34:49.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f32ffd700 1 -- 172.21.15.89:0/672047277 learned_addr learned my addr 172.21.15.89:0/672047277 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:49.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f327fc700 1 --2- 172.21.15.89:0/672047277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f340b54d0 0x7f5f340cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f337fe700 1 --2- 172.21.15.89:0/672047277 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f340ce470 0x7f5f340ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f32ffd700 1 -- 172.21.15.89:0/672047277 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f340ce470 msgr2=0x7f5f340ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:49.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f32ffd700 1 --2- 172.21.15.89:0/672047277 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f340ce470 0x7f5f340ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f32ffd700 1 -- 172.21.15.89:0/672047277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f340b54d0 msgr2=0x7f5f340cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:49.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f32ffd700 1 --2- 172.21.15.89:0/672047277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f340b54d0 0x7f5f340cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f32ffd700 1 -- 172.21.15.89:0/672047277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f340cdae0 con 0x7f5f340b5100 2023-10-10T20:34:49.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f32ffd700 1 --2- 172.21.15.89:0/672047277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f340b5100 0x7f5f340b4050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009fd0 tx=0x7f5f2c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:49.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f2affd700 1 -- 172.21.15.89:0/672047277 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c00e040 con 0x7f5f340b5100 2023-10-10T20:34:49.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f2affd700 1 -- 172.21.15.89:0/672047277 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2c00f040 con 0x7f5f340b5100 2023-10-10T20:34:49.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f394af700 1 -- 172.21.15.89:0/672047277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f340cddc0 con 0x7f5f340b5100 2023-10-10T20:34:49.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f2affd700 1 -- 172.21.15.89:0/672047277 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c00fee0 con 0x7f5f340b5100 2023-10-10T20:34:49.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.269+0000 7f5f394af700 1 -- 172.21.15.89:0/672047277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f340cede0 con 0x7f5f340b5100 2023-10-10T20:34:49.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.271+0000 7f5f2affd700 1 -- 172.21.15.89:0/672047277 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5f2c015640 con 0x7f5f340b5100 2023-10-10T20:34:49.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.271+0000 7f5f2affd700 1 --2- 172.21.15.89:0/672047277 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f14066f30 0x7f5f140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.271+0000 7f5f327fc700 1 --2- 172.21.15.89:0/672047277 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f14066f30 0x7f5f140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.271+0000 7f5f2affd700 1 -- 172.21.15.89:0/672047277 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5f2c0850e0 con 0x7f5f340b5100 2023-10-10T20:34:49.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.271+0000 7f5f327fc700 1 --2- 172.21.15.89:0/672047277 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f14066f30 0x7f5f140693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0079e0 tx=0x7f5f1c001e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:49.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.273+0000 7f5f427a8b80 1 Processor -- start 2023-10-10T20:34:49.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.273+0000 7f5f427a8b80 1 -- start start 2023-10-10T20:34:49.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.274+0000 7f5f427a8b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e951962a50 0x55e951962e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.274+0000 7f5f427a8b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e9519acd00 0x55e95198ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.274+0000 7f5f427a8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e95198bf60 0x55e95195ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.274+0000 7f5f427a8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9518b9a70 con 0x55e95198bf60 2023-10-10T20:34:49.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.274+0000 7f5f427a8b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9518b7270 con 0x55e9519acd00 2023-10-10T20:34:49.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.274+0000 7f5f427a8b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9518b3060 con 0x55e951962a50 2023-10-10T20:34:49.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.274+0000 7f5f337fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e95198bf60 0x55e95195ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.274+0000 7f5f337fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e95198bf60 0x55e95195ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44670/0 (socket says 172.21.15.89:44670) 2023-10-10T20:34:49.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.274+0000 7f5f337fe700 1 -- 172.21.15.89:0/2864336812 learned_addr learned my addr 172.21.15.89:0/2864336812 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:49.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.274+0000 7f5f327fc700 1 --2- 172.21.15.89:0/2864336812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e9519acd00 0x55e95198ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.274+0000 7f5f32ffd700 1 --2- 172.21.15.89:0/2864336812 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e951962a50 0x55e951962e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.274+0000 7f5f337fe700 1 -- 172.21.15.89:0/2864336812 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e951962a50 msgr2=0x55e951962e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:49.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.274+0000 7f5f337fe700 1 --2- 172.21.15.89:0/2864336812 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e951962a50 0x55e951962e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.274+0000 7f5f337fe700 1 -- 172.21.15.89:0/2864336812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e9519acd00 msgr2=0x55e95198ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:49.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.274+0000 7f5f337fe700 1 --2- 172.21.15.89:0/2864336812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e9519acd00 0x55e95198ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.274+0000 7f5f337fe700 1 -- 172.21.15.89:0/2864336812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e95174d270 con 0x55e95198bf60 2023-10-10T20:34:49.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.274+0000 7f5f327fc700 1 --2- 172.21.15.89:0/2864336812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e9519acd00 0x55e95198ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:49.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.274+0000 7f5f337fe700 1 --2- 172.21.15.89:0/2864336812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e95198bf60 0x55e95195ecf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5f24006ad0 tx=0x7f5f2401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb1c40302b3009c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:49.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.275+0000 7f5f13fff700 1 -- 172.21.15.89:0/2864336812 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24022070 con 0x55e95198bf60 2023-10-10T20:34:49.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.275+0000 7f5f13fff700 1 -- 172.21.15.89:0/2864336812 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f24023040 con 0x55e95198bf60 2023-10-10T20:34:49.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.275+0000 7f5f13fff700 1 -- 172.21.15.89:0/2864336812 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2400e6b0 con 0x55e95198bf60 2023-10-10T20:34:49.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.275+0000 7f5f427a8b80 1 -- 172.21.15.89:0/2864336812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e95198bf60 msgr2=0x55e95195ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:49.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.275+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/2864336812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e95198bf60 0x55e95195ecf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5f24006ad0 tx=0x7f5f2401a000 comp rx=0 tx=0).stop 2023-10-10T20:34:49.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.275+0000 7f5f427a8b80 1 -- 172.21.15.89:0/2864336812 shutdown_connections 2023-10-10T20:34:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.275+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/2864336812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e95198bf60 0x55e95195ecf0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.275+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/2864336812 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e951962a50 0x55e951962e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.275+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/2864336812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e9519acd00 0x55e95198ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.275+0000 7f5f427a8b80 1 -- 172.21.15.89:0/2864336812 >> 172.21.15.89:0/2864336812 conn(0x55e951964370 msgr2=0x55e951964740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.276+0000 7f5f427a8b80 1 -- 172.21.15.89:0/2864336812 shutdown_connections 2023-10-10T20:34:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.276+0000 7f5f427a8b80 1 -- 172.21.15.89:0/2864336812 wait complete. 2023-10-10T20:34:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.276+0000 7f5f427a8b80 1 -- start start 2023-10-10T20:34:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.276+0000 7f5f427a8b80 1 Processor -- start 2023-10-10T20:34:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.277+0000 7f5f427a8b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e951962a50 0x55e9519b9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.277+0000 7f5f427a8b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e95198bf60 0x55e9519a6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.277+0000 7f5f427a8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9519acd00 0x55e9519a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.277+0000 7f5f427a8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e95185e890 con 0x55e9519acd00 2023-10-10T20:34:49.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.277+0000 7f5f427a8b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e951879ea0 con 0x55e95198bf60 2023-10-10T20:34:49.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.277+0000 7f5f427a8b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e951735e30 con 0x55e951962a50 2023-10-10T20:34:49.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.277+0000 7f5f427a8b80 10 client.0 _ll_register_callbacks cb 0x55e951926110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:34:49.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.277+0000 7f5f337fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9519acd00 0x55e9519a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.277+0000 7f5f337fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9519acd00 0x55e9519a4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44684/0 (socket says 172.21.15.89:44684) 2023-10-10T20:34:49.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.277+0000 7f5f32ffd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e951962a50 0x55e9519b9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.277+0000 7f5f337fe700 1 -- 172.21.15.89:0/2995086371 learned_addr learned my addr 172.21.15.89:0/2995086371 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:49.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.277+0000 7f5f327fc700 1 --2- 172.21.15.89:0/2995086371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e95198bf60 0x55e9519a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.277+0000 7f5f337fe700 1 -- 172.21.15.89:0/2995086371 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e951962a50 msgr2=0x55e9519b9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:49.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.277+0000 7f5f337fe700 1 --2- 172.21.15.89:0/2995086371 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e951962a50 0x55e9519b9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.277+0000 7f5f337fe700 1 -- 172.21.15.89:0/2995086371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e95198bf60 msgr2=0x55e9519a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:49.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.277+0000 7f5f337fe700 1 --2- 172.21.15.89:0/2995086371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e95198bf60 0x55e9519a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.278+0000 7f5f337fe700 1 -- 172.21.15.89:0/2995086371 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f20003570 con 0x55e9519acd00 2023-10-10T20:34:49.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.278+0000 7f5f327fc700 1 --2- 172.21.15.89:0/2995086371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e95198bf60 0x55e9519a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:49.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.278+0000 7f5f337fe700 1 --2- 172.21.15.89:0/2995086371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9519acd00 0x55e9519a4e60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5f2402a040 tx=0x7f5f24025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14a192dea513877 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:49.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.278+0000 7f5f117fa700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:34:49.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.278+0000 7f5f117fa700 1 -- 172.21.15.89:0/2995086371 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24022050 con 0x55e9519acd00 2023-10-10T20:34:49.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.278+0000 7f5f117fa700 1 -- 172.21.15.89:0/2995086371 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5f24000e40 con 0x55e9519acd00 2023-10-10T20:34:49.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.278+0000 7f5f427a8b80 1 -- 172.21.15.89:0/2995086371 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e951957b70 con 0x55e9519acd00 2023-10-10T20:34:49.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.278+0000 7f5f117fa700 1 -- 172.21.15.89:0/2995086371 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f24006c90 con 0x55e9519acd00 2023-10-10T20:34:49.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.278+0000 7f5f427a8b80 10 client.4629 fetch_fsmap learned FSMap version 8 2023-10-10T20:34:49.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.278+0000 7f5f427a8b80 1 -- 172.21.15.89:0/2995086371 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55e9516effe0 con 0x55e9519acd00 2023-10-10T20:34:49.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.278+0000 7f5f117fa700 1 -- 172.21.15.89:0/2995086371 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e951957b70 con 0x55e9519acd00 2023-10-10T20:34:49.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.278+0000 7f5f117fa700 1 -- 172.21.15.89:0/2995086371 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5f24011070 con 0x55e9519acd00 2023-10-10T20:34:49.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.278+0000 7f5f427a8b80 10 client.4629 fetch_fsmap finished waiting for FSMap version 8 2023-10-10T20:34:49.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.278+0000 7f5f427a8b80 10 client.4629 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-10T20:34:49.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.278+0000 7f5f427a8b80 20 client.4629 populate_metadata read hostname 'smithi089' 2023-10-10T20:34:49.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.278+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/2995086371 >> [v2:172.21.15.111:6832/1112081320,v1:172.21.15.111:6834/1112081320] conn(0x55e9519925d0 0x55e9519a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.278+0000 7f5f427a8b80 4 client.4629 mds_command: new command op to 4270 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:34:49.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.278+0000 7f5f427a8b80 1 -- 172.21.15.89:0/2995086371 --> [v2:172.21.15.111:6832/1112081320,v1:172.21.15.111:6834/1112081320] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e951934aa0 con 0x55e9519925d0 2023-10-10T20:34:49.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.278+0000 7f5f32ffd700 1 --2- 172.21.15.89:0/2995086371 >> [v2:172.21.15.111:6832/1112081320,v1:172.21.15.111:6834/1112081320] conn(0x55e9519925d0 0x55e9519a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.279+0000 7f5f117fa700 10 client.4629 ms_handle_connect on v2:172.21.15.111:6832/1112081320 2023-10-10T20:34:49.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.279+0000 7f5f32ffd700 1 --2- 172.21.15.89:0/2995086371 >> [v2:172.21.15.111:6832/1112081320,v1:172.21.15.111:6834/1112081320] conn(0x55e9519925d0 0x55e9519a0c20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9211b0f346e3031 server_cookie=92ed2d5dd6c8996d in_seq=0 out_seq=0 2023-10-10T20:34:49.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.279+0000 7f5f117fa700 1 -- 172.21.15.89:0/2995086371 <== mds.0 v2:172.21.15.111:6832/1112081320 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55e951934aa0 con 0x55e9519925d0 2023-10-10T20:34:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.279+0000 7f5f117fa700 10 client.4629 handle_command_reply: tid=0 2023-10-10T20:34:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.280+0000 7f5f427a8b80 1 client.4629 shutdown 2023-10-10T20:34:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.280+0000 7f5f427a8b80 1 -- 172.21.15.89:0/2995086371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9519acd00 msgr2=0x55e9519a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.280+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/2995086371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9519acd00 0x55e9519a4e60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5f2402a040 tx=0x7f5f24025000 comp rx=0 tx=0).stop 2023-10-10T20:34:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.280+0000 7f5f427a8b80 1 -- 172.21.15.89:0/2995086371 shutdown_connections 2023-10-10T20:34:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.280+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/2995086371 >> [v2:172.21.15.111:6832/1112081320,v1:172.21.15.111:6834/1112081320] conn(0x55e9519925d0 0x55e9519a0c20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.280+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/2995086371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9519acd00 0x55e9519a4e60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.280+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/2995086371 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e951962a50 0x55e9519b9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.280+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/2995086371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e95198bf60 0x55e9519a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.280+0000 7f5f117fa700 0 client.4629 ms_handle_reset on v2:172.21.15.111:6832/1112081320 2023-10-10T20:34:49.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.280+0000 7f5f427a8b80 1 -- 172.21.15.89:0/2995086371 >> 172.21.15.89:0/2995086371 conn(0x55e951964370 msgr2=0x55e9519a8550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:49.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.280+0000 7f5f427a8b80 1 -- 172.21.15.89:0/2995086371 shutdown_connections 2023-10-10T20:34:49.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.280+0000 7f5f427a8b80 1 -- 172.21.15.89:0/2995086371 wait complete. 2023-10-10T20:34:49.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.280+0000 7f5f427a8b80 20 client.4629 trim_cache size 0 max 16384 2023-10-10T20:34:49.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.297+0000 7f5f427a8b80 1 Processor -- start 2023-10-10T20:34:49.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.297+0000 7f5f427a8b80 1 -- start start 2023-10-10T20:34:49.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.298+0000 7f5f427a8b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e951962a50 0x55e95198a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.298+0000 7f5f427a8b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e95198bf60 0x55e9519a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.298+0000 7f5f427a8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9519acd00 0x55e9519c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.298+0000 7f5f427a8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e951942510 con 0x55e9519acd00 2023-10-10T20:34:49.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.298+0000 7f5f427a8b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9519a09b0 con 0x55e95198bf60 2023-10-10T20:34:49.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.298+0000 7f5f427a8b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9519a5f20 con 0x55e951962a50 2023-10-10T20:34:49.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.298+0000 7f5f337fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9519acd00 0x55e9519c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.298+0000 7f5f337fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9519acd00 0x55e9519c3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44700/0 (socket says 172.21.15.89:44700) 2023-10-10T20:34:49.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.298+0000 7f5f337fe700 1 -- 172.21.15.89:0/679818692 learned_addr learned my addr 172.21.15.89:0/679818692 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:49.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.298+0000 7f5f337fe700 1 -- 172.21.15.89:0/679818692 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e951962a50 msgr2=0x55e95198a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:49.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.298+0000 7f5f337fe700 1 --2- 172.21.15.89:0/679818692 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e951962a50 0x55e95198a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.298+0000 7f5f337fe700 1 -- 172.21.15.89:0/679818692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e95198bf60 msgr2=0x55e9519a41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:49.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.298+0000 7f5f327fc700 1 --2- 172.21.15.89:0/679818692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e95198bf60 0x55e9519a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.298+0000 7f5f337fe700 1 --2- 172.21.15.89:0/679818692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e95198bf60 0x55e9519a41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.298+0000 7f5f337fe700 1 -- 172.21.15.89:0/679818692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9519886b0 con 0x55e9519acd00 2023-10-10T20:34:49.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.298+0000 7f5f337fe700 1 --2- 172.21.15.89:0/679818692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9519acd00 0x55e9519c3b40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400ef70 tx=0x7f5f2402f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745e888a83341725 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:49.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.298+0000 7f5f28ff9700 1 -- 172.21.15.89:0/679818692 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24022040 con 0x55e9519acd00 2023-10-10T20:34:49.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.298+0000 7f5f28ff9700 1 -- 172.21.15.89:0/679818692 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f24013680 con 0x55e9519acd00 2023-10-10T20:34:49.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.298+0000 7f5f28ff9700 1 -- 172.21.15.89:0/679818692 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24013b30 con 0x55e9519acd00 2023-10-10T20:34:49.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.299+0000 7f5f427a8b80 1 -- 172.21.15.89:0/679818692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9519acd00 msgr2=0x55e9519c3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:49.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.299+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/679818692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9519acd00 0x55e9519c3b40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400ef70 tx=0x7f5f2402f000 comp rx=0 tx=0).stop 2023-10-10T20:34:49.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.299+0000 7f5f427a8b80 1 -- 172.21.15.89:0/679818692 shutdown_connections 2023-10-10T20:34:49.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.299+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/679818692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9519acd00 0x55e9519c3b40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.299+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/679818692 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e951962a50 0x55e95198a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.299+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/679818692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e95198bf60 0x55e9519a41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.299+0000 7f5f427a8b80 1 -- 172.21.15.89:0/679818692 >> 172.21.15.89:0/679818692 conn(0x55e951964370 msgr2=0x55e951960c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:49.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.299+0000 7f5f427a8b80 1 -- 172.21.15.89:0/679818692 shutdown_connections 2023-10-10T20:34:49.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.299+0000 7f5f427a8b80 1 -- 172.21.15.89:0/679818692 wait complete. 2023-10-10T20:34:49.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.299+0000 7f5f427a8b80 1 -- start start 2023-10-10T20:34:49.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.299+0000 7f5f427a8b80 1 Processor -- start 2023-10-10T20:34:49.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.300+0000 7f5f427a8b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e951962a50 0x55e9519c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.300+0000 7f5f427a8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e95198bf60 0x55e95195f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.300+0000 7f5f427a8b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e9519acd00 0x55e95195f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.300+0000 7f5f427a8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9519a4890 con 0x55e95198bf60 2023-10-10T20:34:49.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.300+0000 7f5f427a8b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e951a0fc10 con 0x55e9519acd00 2023-10-10T20:34:49.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.300+0000 7f5f427a8b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9519c4260 con 0x55e951962a50 2023-10-10T20:34:49.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.300+0000 7f5f427a8b80 10 client.0 _ll_register_callbacks cb 0x55e951926110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:34:49.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.300+0000 7f5f327fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e95198bf60 0x55e95195f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.300+0000 7f5f327fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e95198bf60 0x55e95195f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44702/0 (socket says 172.21.15.89:44702) 2023-10-10T20:34:49.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.300+0000 7f5f327fc700 1 -- 172.21.15.89:0/53342319 learned_addr learned my addr 172.21.15.89:0/53342319 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:49.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.300+0000 7f5f337fe700 1 --2- 172.21.15.89:0/53342319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e9519acd00 0x55e95195f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.300+0000 7f5f327fc700 1 -- 172.21.15.89:0/53342319 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e951962a50 msgr2=0x55e9519c3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:49.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.300+0000 7f5f32ffd700 1 --2- 172.21.15.89:0/53342319 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e951962a50 0x55e9519c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f327fc700 1 --2- 172.21.15.89:0/53342319 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e951962a50 0x55e9519c3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f327fc700 1 -- 172.21.15.89:0/53342319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e9519acd00 msgr2=0x55e95195f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:49.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f327fc700 1 --2- 172.21.15.89:0/53342319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e9519acd00 0x55e95195f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f327fc700 1 -- 172.21.15.89:0/53342319 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9516effe0 con 0x55e95198bf60 2023-10-10T20:34:49.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f327fc700 1 --2- 172.21.15.89:0/53342319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e95198bf60 0x55e95195f260 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5f340b6600 tx=0x7f5f1c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211381160ab23796 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:49.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f117fa700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:34:49.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f117fa700 1 -- 172.21.15.89:0/53342319 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c01a040 con 0x55e95198bf60 2023-10-10T20:34:49.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f117fa700 1 -- 172.21.15.89:0/53342319 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5f1c014a10 con 0x55e95198bf60 2023-10-10T20:34:49.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f427a8b80 1 -- 172.21.15.89:0/53342319 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f24023040 con 0x55e95198bf60 2023-10-10T20:34:49.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f117fa700 1 -- 172.21.15.89:0/53342319 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f1c007870 con 0x55e95198bf60 2023-10-10T20:34:49.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f427a8b80 10 client.4635 fetch_fsmap learned FSMap version 8 2023-10-10T20:34:49.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f427a8b80 1 -- 172.21.15.89:0/53342319 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55e951881780 con 0x55e95198bf60 2023-10-10T20:34:49.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f117fa700 1 -- 172.21.15.89:0/53342319 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c033070 con 0x55e95198bf60 2023-10-10T20:34:49.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f117fa700 1 -- 172.21.15.89:0/53342319 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5f1c020070 con 0x55e95198bf60 2023-10-10T20:34:49.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f427a8b80 10 client.4635 fetch_fsmap finished waiting for FSMap version 8 2023-10-10T20:34:49.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f427a8b80 10 client.4635 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-10T20:34:49.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f427a8b80 20 client.4635 populate_metadata read hostname 'smithi089' 2023-10-10T20:34:49.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/53342319 >> [v2:172.21.15.111:6832/1112081320,v1:172.21.15.111:6834/1112081320] conn(0x55e951a0eb70 0x55e951a0ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:49.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f427a8b80 4 client.4635 mds_command: new command op to 4270 tid=0[{"prefix": "status"}] 2023-10-10T20:34:49.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f427a8b80 1 -- 172.21.15.89:0/53342319 --> [v2:172.21.15.111:6832/1112081320,v1:172.21.15.111:6834/1112081320] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e951855e00 con 0x55e951a0eb70 2023-10-10T20:34:49.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.301+0000 7f5f32ffd700 1 --2- 172.21.15.89:0/53342319 >> [v2:172.21.15.111:6832/1112081320,v1:172.21.15.111:6834/1112081320] conn(0x55e951a0eb70 0x55e951a0ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:49.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.302+0000 7f5f117fa700 10 client.4635 ms_handle_connect on v2:172.21.15.111:6832/1112081320 2023-10-10T20:34:49.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.302+0000 7f5f32ffd700 1 --2- 172.21.15.89:0/53342319 >> [v2:172.21.15.111:6832/1112081320,v1:172.21.15.111:6834/1112081320] conn(0x55e951a0eb70 0x55e951a0ef40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faf9a132c9a98551 server_cookie=d54031277db8d726 in_seq=0 out_seq=0 2023-10-10T20:34:49.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.302+0000 7f5f117fa700 1 -- 172.21.15.89:0/53342319 <== mds.0 v2:172.21.15.111:6832/1112081320 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55e951855e00 con 0x55e951a0eb70 2023-10-10T20:34:49.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.302+0000 7f5f117fa700 10 client.4635 handle_command_reply: tid=0 2023-10-10T20:34:49.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.302+0000 7f5f427a8b80 1 client.4635 shutdown 2023-10-10T20:34:49.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.302+0000 7f5f427a8b80 1 -- 172.21.15.89:0/53342319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e95198bf60 msgr2=0x55e95195f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:49.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.302+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/53342319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e95198bf60 0x55e95195f260 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5f340b6600 tx=0x7f5f1c017000 comp rx=0 tx=0).stop 2023-10-10T20:34:49.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.302+0000 7f5f427a8b80 1 -- 172.21.15.89:0/53342319 shutdown_connections 2023-10-10T20:34:49.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.302+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/53342319 >> [v2:172.21.15.111:6832/1112081320,v1:172.21.15.111:6834/1112081320] conn(0x55e951a0eb70 0x55e951a0ef40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.302+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/53342319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e9519acd00 0x55e95195f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.302+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/53342319 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e951962a50 0x55e9519c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.302+0000 7f5f427a8b80 1 --2- 172.21.15.89:0/53342319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e95198bf60 0x55e95195f260 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.302+0000 7f5f117fa700 0 client.4635 ms_handle_reset on v2:172.21.15.111:6832/1112081320 2023-10-10T20:34:49.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.302+0000 7f5f427a8b80 1 -- 172.21.15.89:0/53342319 >> 172.21.15.89:0/53342319 conn(0x55e951964370 msgr2=0x55e9519a2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:49.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.302+0000 7f5f427a8b80 1 -- 172.21.15.89:0/53342319 shutdown_connections 2023-10-10T20:34:49.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.302+0000 7f5f427a8b80 1 -- 172.21.15.89:0/53342319 wait complete. 2023-10-10T20:34:49.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.302+0000 7f5f427a8b80 20 client.4635 trim_cache size 0 max 16384 2023-10-10T20:34:49.332 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:34:49.332 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "5dd675b8-9026-4d8a-a1bf-62fdc5e7de67", 2023-10-10T20:34:49.332 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-10-10T20:34:49.332 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 4270, 2023-10-10T20:34:49.333 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-10-10T20:34:49.333 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-10-10T20:34:49.333 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-10-10T20:34:49.333 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 13.957044003, 2023-10-10T20:34:49.333 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 8, 2023-10-10T20:34:49.334 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 20, 2023-10-10T20:34:49.334 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 20, 2023-10-10T20:34:49.334 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 21.277233046999999 2023-10-10T20:34:49.334 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:34:49.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.303+0000 7f5f394af700 1 -- 172.21.15.89:0/672047277 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f14066f30 msgr2=0x7f5f140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:49.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.303+0000 7f5f394af700 1 --2- 172.21.15.89:0/672047277 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f14066f30 0x7f5f140693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0079e0 tx=0x7f5f1c001e90 comp rx=0 tx=0).stop 2023-10-10T20:34:49.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.303+0000 7f5f394af700 1 -- 172.21.15.89:0/672047277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f340b5100 msgr2=0x7f5f340b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:49.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.303+0000 7f5f394af700 1 --2- 172.21.15.89:0/672047277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f340b5100 0x7f5f340b4050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009fd0 tx=0x7f5f2c00b040 comp rx=0 tx=0).stop 2023-10-10T20:34:49.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.303+0000 7f5f394af700 1 -- 172.21.15.89:0/672047277 shutdown_connections 2023-10-10T20:34:49.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.303+0000 7f5f394af700 1 --2- 172.21.15.89:0/672047277 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f340ce470 0x7f5f340ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.303+0000 7f5f394af700 1 --2- 172.21.15.89:0/672047277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f340b5100 0x7f5f340b4050 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.303+0000 7f5f394af700 1 --2- 172.21.15.89:0/672047277 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f14066f30 0x7f5f140693e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.303+0000 7f5f394af700 1 --2- 172.21.15.89:0/672047277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f340b54d0 0x7f5f340cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:49.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.303+0000 7f5f394af700 1 -- 172.21.15.89:0/672047277 >> 172.21.15.89:0/672047277 conn(0x7f5f34068e20 msgr2=0x7f5f34153da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:49.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.303+0000 7f5f394af700 1 -- 172.21.15.89:0/672047277 shutdown_connections 2023-10-10T20:34:49.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:49.303+0000 7f5f394af700 1 -- 172.21.15.89:0/672047277 wait complete. 2023-10-10T20:34:49.339 INFO:teuthology.run_tasks:Running task kclient... 2023-10-10T20:34:49.360 INFO:tasks.kclient:Mounting kernel clients... 2023-10-10T20:34:49.360 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-10T20:34:49.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:49.361 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-10T20:34:49.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:49.378 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link delete ceph-brx 2023-10-10T20:34:49.442 INFO:teuthology.orchestra.run.smithi089.stderr:Cannot find device "ceph-brx" 2023-10-10T20:34:49.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:34:49.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:49.444 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-10T20:34:49.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:49.499 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link delete ceph-brx 2023-10-10T20:34:49.564 INFO:teuthology.orchestra.run.smithi089.stderr:Cannot find device "ceph-brx" 2023-10-10T20:34:49.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:34:49.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:49.566 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-10-10T20:34:49.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:49.585 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link delete ceph-brx 2023-10-10T20:34:49.663 INFO:teuthology.orchestra.run.smithi111.stderr:Cannot find device "ceph-brx" 2023-10-10T20:34:49.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:34:49.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:49.998 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-10-10T20:34:50.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:50.057 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link delete ceph-brx 2023-10-10T20:34:50.127 INFO:teuthology.orchestra.run.smithi111.stderr:Cannot find device "ceph-brx" 2023-10-10T20:34:50.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:34:50.130 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-10-10T20:34:50.131 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:34:50.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.248+0000 7fade2db1700 1 Processor -- start 2023-10-10T20:34:50.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.248+0000 7fade2db1700 1 -- start start 2023-10-10T20:34:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.249+0000 7fade2db1700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faddc159890 0x7faddc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.249+0000 7fade2db1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faddc159c60 0x7faddc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.249+0000 7fade2db1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faddc15a160 con 0x7faddc15db10 2023-10-10T20:34:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.249+0000 7fade2db1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faddc15df50 con 0x7faddc159c60 2023-10-10T20:34:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.249+0000 7fade2db1700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faddc0d13a0 con 0x7faddc159890 2023-10-10T20:34:50.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.249+0000 7faddbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faddc159c60 0x7faddc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:50.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.249+0000 7fade134e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7faddc15db10 0x7faddc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43862/0 (socket says 172.21.15.89:43862) 2023-10-10T20:34:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.249+0000 7fade134e700 1 -- 172.21.15.89:0/2226737282 learned_addr learned my addr 172.21.15.89:0/2226737282 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.249+0000 7fade0b4d700 1 --2- 172.21.15.89:0/2226737282 >> v2:172.21.15.111:3300/0 conn(0x7faddc159890 0x7faddc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.249+0000 7faddbfff700 1 -- 172.21.15.89:0/2226737282 >> v2:172.21.15.111:3300/0 conn(0x7faddc159890 msgr2=0x7faddc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.249+0000 7faddbfff700 1 --2- 172.21.15.89:0/2226737282 >> v2:172.21.15.111:3300/0 conn(0x7faddc159890 0x7faddc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.249+0000 7faddbfff700 1 -- 172.21.15.89:0/2226737282 >> v1:172.21.15.89:6789/0 conn(0x7faddc15db10 legacy=0x7faddc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.249+0000 7faddb7fe700 1 -- 172.21.15.89:0/2226737282 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942197013 0 0) 0x7faddc15a160 con 0x7faddc15db10 2023-10-10T20:34:50.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.249+0000 7faddbfff700 1 -- 172.21.15.89:0/2226737282 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faddc15e140 con 0x7faddc159c60 2023-10-10T20:34:50.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.249+0000 7faddbfff700 1 --2- 172.21.15.89:0/2226737282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faddc159c60 0x7faddc15d5d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faddc0d1090 tx=0x7fadc80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8792f380e317bce6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:50.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.249+0000 7faddb7fe700 1 -- 172.21.15.89:0/2226737282 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc800b6b0 con 0x7faddc159c60 2023-10-10T20:34:50.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.249+0000 7faddb7fe700 1 -- 172.21.15.89:0/2226737282 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc800b810 con 0x7faddc159c60 2023-10-10T20:34:50.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.249+0000 7faddb7fe700 1 -- 172.21.15.89:0/2226737282 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc800bbb0 con 0x7faddc159c60 2023-10-10T20:34:50.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.250+0000 7fade2db1700 1 -- 172.21.15.89:0/2226737282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faddc159c60 msgr2=0x7faddc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:50.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.250+0000 7fade2db1700 1 --2- 172.21.15.89:0/2226737282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faddc159c60 0x7faddc15d5d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faddc0d1090 tx=0x7fadc80091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:50.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.250+0000 7fade2db1700 1 -- 172.21.15.89:0/2226737282 shutdown_connections 2023-10-10T20:34:50.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.250+0000 7fade2db1700 1 --2- 172.21.15.89:0/2226737282 >> v2:172.21.15.111:3300/0 conn(0x7faddc159890 0x7faddc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:50.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.250+0000 7fade2db1700 1 --2- 172.21.15.89:0/2226737282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faddc159c60 0x7faddc15d5d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:50.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.250+0000 7fade2db1700 1 -- 172.21.15.89:0/2226737282 >> 172.21.15.89:0/2226737282 conn(0x7faddc068e20 msgr2=0x7faddc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.250+0000 7fade2db1700 1 -- 172.21.15.89:0/2226737282 shutdown_connections 2023-10-10T20:34:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.250+0000 7fade2db1700 1 -- 172.21.15.89:0/2226737282 wait complete. 2023-10-10T20:34:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.251+0000 7fade2db1700 1 Processor -- start 2023-10-10T20:34:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.251+0000 7fade2db1700 1 -- start start 2023-10-10T20:34:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.252+0000 7fade2db1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faddc159890 0x7faddc0cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:50.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.252+0000 7fade2db1700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faddc159c60 0x7faddc14d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:50.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.252+0000 7fade2db1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faddc15db10 0x7faddc152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:50.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.252+0000 7fade2db1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faddc151ff0 con 0x7faddc159890 2023-10-10T20:34:50.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.252+0000 7fade2db1700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faddc0cdc20 con 0x7faddc159c60 2023-10-10T20:34:50.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.252+0000 7fade2db1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faddc0cdec0 con 0x7faddc15db10 2023-10-10T20:34:50.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.252+0000 7fade0b4d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faddc159890 0x7faddc0cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:50.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.252+0000 7fade0b4d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faddc159890 0x7faddc0cec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44720/0 (socket says 172.21.15.89:44720) 2023-10-10T20:34:50.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.252+0000 7fade0b4d700 1 -- 172.21.15.89:0/2357172594 learned_addr learned my addr 172.21.15.89:0/2357172594 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.252+0000 7faddbfff700 1 --2- 172.21.15.89:0/2357172594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faddc159c60 0x7faddc14d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.252+0000 7fade0b4d700 1 -- 172.21.15.89:0/2357172594 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faddc15db10 msgr2=0x7faddc152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.252+0000 7fade134e700 1 --2- 172.21.15.89:0/2357172594 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faddc15db10 0x7faddc152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.252+0000 7fade0b4d700 1 --2- 172.21.15.89:0/2357172594 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faddc15db10 0x7faddc152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.252+0000 7fade0b4d700 1 -- 172.21.15.89:0/2357172594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faddc159c60 msgr2=0x7faddc14d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.252+0000 7fade0b4d700 1 --2- 172.21.15.89:0/2357172594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faddc159c60 0x7faddc14d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.252+0000 7fade0b4d700 1 -- 172.21.15.89:0/2357172594 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc8009420 con 0x7faddc159890 2023-10-10T20:34:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.252+0000 7faddbfff700 1 --2- 172.21.15.89:0/2357172594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faddc159c60 0x7faddc14d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:50.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.252+0000 7fade0b4d700 1 --2- 172.21.15.89:0/2357172594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faddc159890 0x7faddc0cec40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fadd000d890 tx=0x7fadd0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:50.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.252+0000 7fadd8ff9700 1 -- 172.21.15.89:0/2357172594 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd000b600 con 0x7faddc159890 2023-10-10T20:34:50.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.253+0000 7fadd8ff9700 1 -- 172.21.15.89:0/2357172594 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd000b760 con 0x7faddc159890 2023-10-10T20:34:50.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.253+0000 7fade2db1700 1 -- 172.21.15.89:0/2357172594 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faddc152e50 con 0x7faddc159890 2023-10-10T20:34:50.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.253+0000 7fadd8ff9700 1 -- 172.21.15.89:0/2357172594 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd000bae0 con 0x7faddc159890 2023-10-10T20:34:50.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.253+0000 7fade2db1700 1 -- 172.21.15.89:0/2357172594 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faddc0ce720 con 0x7faddc159890 2023-10-10T20:34:50.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.253+0000 7fade2db1700 1 -- 172.21.15.89:0/2357172594 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faddc0cea90 con 0x7faddc159890 2023-10-10T20:34:50.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.254+0000 7fadd8ff9700 1 -- 172.21.15.89:0/2357172594 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fadd001e460 con 0x7faddc159890 2023-10-10T20:34:50.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.254+0000 7fadd8ff9700 1 --2- 172.21.15.89:0/2357172594 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fadc0066f30 0x7fadc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.254+0000 7faddbfff700 1 --2- 172.21.15.89:0/2357172594 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fadc0066f30 0x7fadc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.254+0000 7fadd8ff9700 1 -- 172.21.15.89:0/2357172594 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fadd0084be0 con 0x7faddc159890 2023-10-10T20:34:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.255+0000 7faddbfff700 1 --2- 172.21.15.89:0/2357172594 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fadc0066f30 0x7fadc00693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faddc0b02f0 tx=0x7fadc8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:50.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.258+0000 7fadd8ff9700 1 -- 172.21.15.89:0/2357172594 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fadd0051e50 con 0x7faddc159890 2023-10-10T20:34:50.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.485+0000 7fade2db1700 1 -- 172.21.15.89:0/2357172594 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7faddc0ce080 con 0x7faddc159890 2023-10-10T20:34:50.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.486+0000 7fadd8ff9700 1 -- 172.21.15.89:0/2357172594 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fadd0055470 con 0x7faddc159890 2023-10-10T20:34:50.488 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:34:50.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.488+0000 7fade2db1700 1 -- 172.21.15.89:0/2357172594 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fadc0066f30 msgr2=0x7fadc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:50.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.488+0000 7fade2db1700 1 --2- 172.21.15.89:0/2357172594 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fadc0066f30 0x7fadc00693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faddc0b02f0 tx=0x7fadc8017000 comp rx=0 tx=0).stop 2023-10-10T20:34:50.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.488+0000 7fade2db1700 1 -- 172.21.15.89:0/2357172594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faddc159890 msgr2=0x7faddc0cec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:50.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.488+0000 7fade2db1700 1 --2- 172.21.15.89:0/2357172594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faddc159890 0x7faddc0cec40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fadd000d890 tx=0x7fadd0008040 comp rx=0 tx=0).stop 2023-10-10T20:34:50.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.488+0000 7fade2db1700 1 -- 172.21.15.89:0/2357172594 shutdown_connections 2023-10-10T20:34:50.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.488+0000 7fade2db1700 1 --2- 172.21.15.89:0/2357172594 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faddc15db10 0x7faddc152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:50.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.488+0000 7fade2db1700 1 --2- 172.21.15.89:0/2357172594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faddc159890 0x7faddc0cec40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:50.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.488+0000 7fade2db1700 1 --2- 172.21.15.89:0/2357172594 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fadc0066f30 0x7fadc00693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:50.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.488+0000 7fade2db1700 1 --2- 172.21.15.89:0/2357172594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faddc159c60 0x7faddc14d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:50.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.488+0000 7fade2db1700 1 -- 172.21.15.89:0/2357172594 >> 172.21.15.89:0/2357172594 conn(0x7faddc068e20 msgr2=0x7faddc069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:50.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.488+0000 7fade2db1700 1 -- 172.21.15.89:0/2357172594 shutdown_connections 2023-10-10T20:34:50.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:50.488+0000 7fade2db1700 1 -- 172.21.15.89:0/2357172594 wait complete. 2023-10-10T20:34:50.502 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-10T20:34:50.502 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:34:50.502 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-10T20:34:50.503 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:34:50.503 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:34:50.503 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-10T20:34:50.503 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-10T20:34:50.504 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:34:50.504 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:34:50.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:50.505 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-10T20:34:50.520 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:34:50.520 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:34:50.520 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:34:50.520 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:50.521 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-10T20:34:50.521 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:50.521 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:34:50.521 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:34:50.521 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-10T20:34:50.521 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:34:50.522 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:34:50.522 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-10T20:34:50.522 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:34:50.522 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:34:50.523 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:34:50.523 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:34:50.523 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:34:50.523 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42475sec preferred_lft 42475sec 2023-10-10T20:34:50.524 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2023-10-10T20:34:50.524 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:50.524 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-10T20:34:50.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:50.525 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:34:50.525 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:34:50.526 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add name ceph-brx type bridge 2023-10-10T20:34:50.526 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr flush dev ceph-brx 2023-10-10T20:34:50.526 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set ceph-brx up 2023-10-10T20:34:50.526 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-10T20:34:50.527 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:34:50.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:50.688 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:34:50.727 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-10T20:34:50.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:50.728 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-10T20:34:50.807 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-10T20:34:50.807 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:34:50.808 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:34:50.808 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:34:50.808 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:34:50.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:50.809 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:34:50.809 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:34:50.809 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:34:50.810 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:34:50.810 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:34:50.810 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:34:51.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:51.132 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-10T20:34:51.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:51.147 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list-id 2023-10-10T20:34:51.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:51.202 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:34:51.202 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:34:51.202 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:34:51.203 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-10T20:34:51.203 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:34:51.291 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-10T20:34:51.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:51.292 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:34:51.292 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:34:51.293 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-10T20:34:51.293 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-10T20:34:51.293 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-10T20:34:51.294 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-10T20:34:51.294 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-10T20:34:51.294 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:34:51.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:51.502 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:34:51.502 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:34:51.502 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-10-10T20:34:51.503 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set dev brx.0 master ceph-brx 2023-10-10T20:34:51.503 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:34:51.594 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:34:51.595 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:34:51.609 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:34:51.610 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:34:51.610 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:34:51.664 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:34:51.664 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:34:51.664 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:34:51.759 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:34:51.759 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:34:51.759 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:34:51.843 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:34:51.844 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:34:51.844 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:34:51.844 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:34:51.844 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:34:51.866 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:34:51.867 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:34:51.867 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:34:51.867 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:34:51.867 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:34:51.868 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:34:51.868 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:34:51.868 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:34:51.869 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:34:51.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:51.870 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:34:51.895 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-10-10T20:34:51.895 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:34:52.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.048+0000 7f30a1570700 1 Processor -- start 2023-10-10T20:34:52.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.048+0000 7f30a1570700 1 -- start start 2023-10-10T20:34:52.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.049+0000 7f30a1570700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f309c0aca80 0x7f309c06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:52.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.049+0000 7f30a1570700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f309c069d60 0x7f309c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:52.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.049+0000 7f30a1570700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f309c06bdb0 con 0x7f309c0c81f0 2023-10-10T20:34:52.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.049+0000 7f30a1570700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f309c0c8670 con 0x7f309c0aca80 2023-10-10T20:34:52.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.049+0000 7f30a1570700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c0cb370 con 0x7f309c069d60 2023-10-10T20:34:52.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.050+0000 7f30927fc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f309c069d60 0x7f309c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:52.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.050+0000 7f309affd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f309c0aca80 0x7f309c06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:52.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.050+0000 7f30927fc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f309c069d60 0x7f309c0c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:39058/0 (socket says 172.21.15.89:39058) 2023-10-10T20:34:52.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.050+0000 7f30927fc700 1 -- 172.21.15.89:0/2828706768 learned_addr learned my addr 172.21.15.89:0/2828706768 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:52.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.050+0000 7f309b7fe700 1 --1- 172.21.15.89:0/2828706768 >> v1:172.21.15.111:6789/0 conn(0x7f309c0c81f0 0x7f309c068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:60710/0 (socket says 172.21.15.89:60710) 2023-10-10T20:34:52.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.050+0000 7f30927fc700 1 -- 172.21.15.89:0/2828706768 >> v2:172.21.15.111:3300/0 conn(0x7f309c0aca80 msgr2=0x7f309c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:52.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.050+0000 7f30927fc700 1 --2- 172.21.15.89:0/2828706768 >> v2:172.21.15.111:3300/0 conn(0x7f309c0aca80 0x7f309c06aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:52.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.050+0000 7f30927fc700 1 -- 172.21.15.89:0/2828706768 >> v1:172.21.15.111:6789/0 conn(0x7f309c0c81f0 legacy=0x7f309c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:52.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.050+0000 7f30927fc700 1 -- 172.21.15.89:0/2828706768 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c159f60 con 0x7f309c069d60 2023-10-10T20:34:52.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.050+0000 7f309affd700 1 --2- 172.21.15.89:0/2828706768 >> v2:172.21.15.111:3300/0 conn(0x7f309c0aca80 0x7f309c06aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:52.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.050+0000 7f30927fc700 1 --2- 172.21.15.89:0/2828706768 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f309c069d60 0x7f309c0c7cb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f30840099b0 tx=0x7f3084009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c513cd5299cdebfd server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:52.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.050+0000 7f309a7fc700 1 -- 172.21.15.89:0/2828706768 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3084009ea0 con 0x7f309c069d60 2023-10-10T20:34:52.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.050+0000 7f309a7fc700 1 -- 172.21.15.89:0/2828706768 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308400b860 con 0x7f309c069d60 2023-10-10T20:34:52.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.050+0000 7f309a7fc700 1 -- 172.21.15.89:0/2828706768 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308400bc00 con 0x7f309c069d60 2023-10-10T20:34:52.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.051+0000 7f30a1570700 1 -- 172.21.15.89:0/2828706768 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f309c069d60 msgr2=0x7f309c0c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:52.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.051+0000 7f30a1570700 1 --2- 172.21.15.89:0/2828706768 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f309c069d60 0x7f309c0c7cb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f30840099b0 tx=0x7f3084009250 comp rx=0 tx=0).stop 2023-10-10T20:34:52.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.051+0000 7f30a1570700 1 -- 172.21.15.89:0/2828706768 shutdown_connections 2023-10-10T20:34:52.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.051+0000 7f30a1570700 1 --2- 172.21.15.89:0/2828706768 >> v2:172.21.15.111:3300/0 conn(0x7f309c0aca80 0x7f309c06aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:52.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.051+0000 7f30a1570700 1 --2- 172.21.15.89:0/2828706768 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f309c069d60 0x7f309c0c7cb0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:52.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.051+0000 7f30a1570700 1 -- 172.21.15.89:0/2828706768 >> 172.21.15.89:0/2828706768 conn(0x7f309c0cd670 msgr2=0x7f309c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:52.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.051+0000 7f30a1570700 1 -- 172.21.15.89:0/2828706768 shutdown_connections 2023-10-10T20:34:52.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.051+0000 7f30a1570700 1 -- 172.21.15.89:0/2828706768 wait complete. 2023-10-10T20:34:52.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.052+0000 7f30a1570700 1 Processor -- start 2023-10-10T20:34:52.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.052+0000 7f30a1570700 1 -- start start 2023-10-10T20:34:52.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.053+0000 7f30a1570700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f309c069d60 0x7f309c16dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:52.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.053+0000 7f30a1570700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f309c0aca80 0x7f309c0c3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:52.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.053+0000 7f30a1570700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f309c0c81f0 0x7f309c16a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.053+0000 7f30a1570700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c0afb70 con 0x7f309c069d60 2023-10-10T20:34:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.053+0000 7f30a1570700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c16e520 con 0x7f309c0aca80 2023-10-10T20:34:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.053+0000 7f30a1570700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c16aab0 con 0x7f309c0c81f0 2023-10-10T20:34:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.053+0000 7f309affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f309c069d60 0x7f309c16dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.053+0000 7f309affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f309c069d60 0x7f309c16dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44736/0 (socket says 172.21.15.89:44736) 2023-10-10T20:34:52.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.053+0000 7f309affd700 1 -- 172.21.15.89:0/1064789565 learned_addr learned my addr 172.21.15.89:0/1064789565 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:52.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.053+0000 7f309b7fe700 1 --2- 172.21.15.89:0/1064789565 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f309c0c81f0 0x7f309c16a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:52.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.053+0000 7f30927fc700 1 --2- 172.21.15.89:0/1064789565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f309c0aca80 0x7f309c0c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:52.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.053+0000 7f309affd700 1 -- 172.21.15.89:0/1064789565 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f309c0c81f0 msgr2=0x7f309c16a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:52.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.053+0000 7f309affd700 1 --2- 172.21.15.89:0/1064789565 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f309c0c81f0 0x7f309c16a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:52.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.053+0000 7f309affd700 1 -- 172.21.15.89:0/1064789565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f309c0aca80 msgr2=0x7f309c0c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:52.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.053+0000 7f309affd700 1 --2- 172.21.15.89:0/1064789565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f309c0aca80 0x7f309c0c3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:52.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.053+0000 7f309affd700 1 -- 172.21.15.89:0/1064789565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308c0093c0 con 0x7f309c069d60 2023-10-10T20:34:52.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.054+0000 7f309affd700 1 --2- 172.21.15.89:0/1064789565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f309c069d60 0x7f309c16dfe0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f308c00ecb0 tx=0x7f308c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:52.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.054+0000 7f30937fe700 1 -- 172.21.15.89:0/1064789565 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c005a00 con 0x7f309c069d60 2023-10-10T20:34:52.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.054+0000 7f30937fe700 1 -- 172.21.15.89:0/1064789565 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308c005b60 con 0x7f309c069d60 2023-10-10T20:34:52.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.054+0000 7f30a1570700 1 -- 172.21.15.89:0/1064789565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30840094f0 con 0x7f309c069d60 2023-10-10T20:34:52.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.054+0000 7f30937fe700 1 -- 172.21.15.89:0/1064789565 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c00c920 con 0x7f309c069d60 2023-10-10T20:34:52.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.054+0000 7f30a1570700 1 -- 172.21.15.89:0/1064789565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309c16dac0 con 0x7f309c069d60 2023-10-10T20:34:52.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.055+0000 7f30a1570700 1 -- 172.21.15.89:0/1064789565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309c04c3c0 con 0x7f309c069d60 2023-10-10T20:34:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.055+0000 7f30937fe700 1 -- 172.21.15.89:0/1064789565 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f308c00ccd0 con 0x7f309c069d60 2023-10-10T20:34:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.055+0000 7f30937fe700 1 --2- 172.21.15.89:0/1064789565 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f307c066f30 0x7f307c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.055+0000 7f30937fe700 1 -- 172.21.15.89:0/1064789565 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f308c084a40 con 0x7f309c069d60 2023-10-10T20:34:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.055+0000 7f30927fc700 1 --2- 172.21.15.89:0/1064789565 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f307c066f30 0x7f307c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.056+0000 7f30927fc700 1 --2- 172.21.15.89:0/1064789565 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f307c066f30 0x7f307c0693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f30840099b0 tx=0x7f3084017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.060+0000 7f30937fe700 1 -- 172.21.15.89:0/1064789565 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f308c086e60 con 0x7f309c069d60 2023-10-10T20:34:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.285+0000 7f30a1570700 1 -- 172.21.15.89:0/1064789565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f309c06b510 con 0x7f309c069d60 2023-10-10T20:34:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.285+0000 7f30937fe700 1 -- 172.21.15.89:0/1064789565 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f308c020020 con 0x7f309c069d60 2023-10-10T20:34:52.288 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:34:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.287+0000 7f30a1570700 1 -- 172.21.15.89:0/1064789565 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f307c066f30 msgr2=0x7f307c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.287+0000 7f30a1570700 1 --2- 172.21.15.89:0/1064789565 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f307c066f30 0x7f307c0693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f30840099b0 tx=0x7f3084017000 comp rx=0 tx=0).stop 2023-10-10T20:34:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.287+0000 7f30a1570700 1 -- 172.21.15.89:0/1064789565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f309c069d60 msgr2=0x7f309c16dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.287+0000 7f30a1570700 1 --2- 172.21.15.89:0/1064789565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f309c069d60 0x7f309c16dfe0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f308c00ecb0 tx=0x7f308c009120 comp rx=0 tx=0).stop 2023-10-10T20:34:52.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.287+0000 7f30a1570700 1 -- 172.21.15.89:0/1064789565 shutdown_connections 2023-10-10T20:34:52.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.287+0000 7f30a1570700 1 --2- 172.21.15.89:0/1064789565 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f309c0c81f0 0x7f309c16a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:52.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.287+0000 7f30a1570700 1 --2- 172.21.15.89:0/1064789565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f309c069d60 0x7f309c16dfe0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:52.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.287+0000 7f30a1570700 1 --2- 172.21.15.89:0/1064789565 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f307c066f30 0x7f307c0693e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:52.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.287+0000 7f30a1570700 1 --2- 172.21.15.89:0/1064789565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f309c0aca80 0x7f309c0c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.287+0000 7f30a1570700 1 -- 172.21.15.89:0/1064789565 >> 172.21.15.89:0/1064789565 conn(0x7f309c0cd670 msgr2=0x7f309c158050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.288+0000 7f30a1570700 1 -- 172.21.15.89:0/1064789565 shutdown_connections 2023-10-10T20:34:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:52.288+0000 7f30a1570700 1 -- 172.21.15.89:0/1064789565 wait complete. 2023-10-10T20:34:52.302 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-10T20:34:52.302 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:34:52.302 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-10T20:34:52.303 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-10T20:34:52.303 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-10T20:34:52.303 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-10T20:34:52.303 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-10T20:34:52.303 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:34:52.304 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-10T20:34:52.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:52.304 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-10T20:34:52.319 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:34:52.319 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:34:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:34:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-10T20:34:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:52.320 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:34:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:34:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-10T20:34:52.321 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:34:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:34:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-10T20:34:52.322 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:34:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:34:52.322 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:34:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:34:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:34:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42473sec preferred_lft 42473sec 2023-10-10T20:34:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2023-10-10T20:34:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:52.323 INFO:teuthology.orchestra.run.smithi089.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:34:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether ee:16:22:10:ca:9a brd ff:ff:ff:ff:ff:ff 2023-10-10T20:34:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:34:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::98a3:d4ff:fec1:83bb/64 scope link 2023-10-10T20:34:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:52.324 INFO:teuthology.orchestra.run.smithi089.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:34:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether ee:16:22:10:ca:9a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:34:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec16:22ff:fe10:ca9a/64 scope link tentative 2023-10-10T20:34:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:52.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:52.326 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:34:52.382 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-10T20:34:52.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:52.384 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-10T20:34:52.442 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-10T20:34:52.442 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:34:52.442 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:34:52.443 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:34:52.443 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:34:52.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:52.443 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:34:52.444 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:34:52.444 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:34:52.444 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:34:52.445 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:34:52.445 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:34:52.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:52.541 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-10T20:34:52.556 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:34:52.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:52.557 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list-id 2023-10-10T20:34:52.610 INFO:teuthology.orchestra.run.smithi089.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-10T20:34:52.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:52.611 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:34:52.611 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:34:52.611 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:34:52.612 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-10T20:34:52.612 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:34:52.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:52.699 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-10T20:34:52.731 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:34:52.731 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:34:52.731 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:34:52.731 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:52.731 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-10T20:34:52.732 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:52.732 INFO:teuthology.orchestra.run.smithi089.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:34:52.732 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether aa:02:b0:6d:48:09 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-10T20:34:52.732 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-10T20:34:52.732 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:52.733 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::a802:b0ff:fe6d:4809/64 scope link tentative 2023-10-10T20:34:52.733 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:52.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:52.744 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-10T20:34:52.819 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:34:52.819 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:34:52.819 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:34:52.819 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:52.820 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-10T20:34:52.820 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:52.820 INFO:teuthology.orchestra.run.smithi089.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:34:52.820 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether aa:02:b0:6d:48:09 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-10T20:34:52.820 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-10T20:34:52.821 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:52.821 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::a802:b0ff:fe6d:4809/64 scope link tentative 2023-10-10T20:34:52.821 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:52.831 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-10T20:34:52.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:52.832 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:34:52.832 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:34:52.832 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-10T20:34:52.833 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-10T20:34:52.833 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-10T20:34:52.833 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-10T20:34:52.834 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-10T20:34:52.834 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:34:53.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:53.045 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:34:53.045 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:34:53.045 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.1 up 2023-10-10T20:34:53.046 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set dev brx.1 master ceph-brx 2023-10-10T20:34:53.046 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:34:53.125 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:34:53.125 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-10T20:34:53.141 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-10T20:34:53.141 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:34:53.141 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-10T20:34:53.195 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:34:53.195 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:34:53.196 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-10T20:34:53.274 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-10T20:34:53.274 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-10T20:34:53.274 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:34:53.282 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-10T20:34:53.282 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-10T20:34:53.283 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:34:53.283 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:34:53.283 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:34:53.305 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-10T20:34:53.305 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-10T20:34:53.305 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:34:53.306 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:34:53.306 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:34:53.306 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:34:53.306 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:34:53.306 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:34:53.308 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:34:53.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:53.308 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-10T20:34:53.372 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-10-10T20:34:53.372 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:34:53.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.523+0000 7f57a72a1700 1 Processor -- start 2023-10-10T20:34:53.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.523+0000 7f57a72a1700 1 -- start start 2023-10-10T20:34:53.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.524+0000 7f57a72a1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f57a0159890 0x7f57a015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:53.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.524+0000 7f57a72a1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57a0069560 0x7f57a00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:53.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.524+0000 7f57a72a1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a0069ee0 con 0x7f57a0159c60 2023-10-10T20:34:53.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.524+0000 7f57a72a1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a015a160 con 0x7f57a0159890 2023-10-10T20:34:53.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.524+0000 7f57a72a1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00d13a0 con 0x7f57a0069560 2023-10-10T20:34:53.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.524+0000 7f57a503d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f57a0159890 0x7f57a015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:53.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.524+0000 7f57a483c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f57a0159c60 0x7f57a015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43876/0 (socket says 172.21.15.89:43876) 2023-10-10T20:34:53.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.524+0000 7f57a483c700 1 -- 172.21.15.89:0/4203497426 learned_addr learned my addr 172.21.15.89:0/4203497426 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:53.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.525+0000 7f57a583e700 1 --2- 172.21.15.89:0/4203497426 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57a0069560 0x7f57a00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:53.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.525+0000 7f57a503d700 1 -- 172.21.15.89:0/4203497426 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57a0069560 msgr2=0x7f57a00699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:53.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.525+0000 7f57a503d700 1 --2- 172.21.15.89:0/4203497426 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57a0069560 0x7f57a00699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:53.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.525+0000 7f57a503d700 1 -- 172.21.15.89:0/4203497426 >> v1:172.21.15.89:6789/0 conn(0x7f57a0159c60 legacy=0x7f57a015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:53.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.525+0000 7f57a503d700 1 -- 172.21.15.89:0/4203497426 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a015e140 con 0x7f57a0159890 2023-10-10T20:34:53.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.525+0000 7f5797fff700 1 -- 172.21.15.89:0/4203497426 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1401103877 0 0) 0x7f57a0069ee0 con 0x7f57a0159c60 2023-10-10T20:34:53.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.525+0000 7f57a503d700 1 --2- 172.21.15.89:0/4203497426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a0159890 0x7f57a015abd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f579c009e60 tx=0x7f579c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96c834980905151 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:53.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.525+0000 7f5797fff700 1 -- 172.21.15.89:0/4203497426 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c00e070 con 0x7f57a0159890 2023-10-10T20:34:53.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.525+0000 7f5797fff700 1 -- 172.21.15.89:0/4203497426 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579c00bc30 con 0x7f57a0159890 2023-10-10T20:34:53.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.525+0000 7f5797fff700 1 -- 172.21.15.89:0/4203497426 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c012430 con 0x7f57a0159890 2023-10-10T20:34:53.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.525+0000 7f57a72a1700 1 -- 172.21.15.89:0/4203497426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a0159890 msgr2=0x7f57a015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:53.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.525+0000 7f57a72a1700 1 --2- 172.21.15.89:0/4203497426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a0159890 0x7f57a015abd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f579c009e60 tx=0x7f579c009780 comp rx=0 tx=0).stop 2023-10-10T20:34:53.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.526+0000 7f57a72a1700 1 -- 172.21.15.89:0/4203497426 shutdown_connections 2023-10-10T20:34:53.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.526+0000 7f57a72a1700 1 --2- 172.21.15.89:0/4203497426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a0159890 0x7f57a015abd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:53.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.526+0000 7f57a72a1700 1 --2- 172.21.15.89:0/4203497426 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57a0069560 0x7f57a00699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:53.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.526+0000 7f57a72a1700 1 -- 172.21.15.89:0/4203497426 >> 172.21.15.89:0/4203497426 conn(0x7f57a0068e20 msgr2=0x7f57a00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:53.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.526+0000 7f57a72a1700 1 -- 172.21.15.89:0/4203497426 shutdown_connections 2023-10-10T20:34:53.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.526+0000 7f57a72a1700 1 -- 172.21.15.89:0/4203497426 wait complete. 2023-10-10T20:34:53.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.527+0000 7f57a72a1700 1 Processor -- start 2023-10-10T20:34:53.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.527+0000 7f57a72a1700 1 -- start start 2023-10-10T20:34:53.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57a72a1700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57a0069560 0x7f57a014dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:53.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57a72a1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57a0159890 0x7f57a0155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:53.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57a72a1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a0159c60 0x7f57a0156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:53.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57a72a1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00ceb40 con 0x7f57a0159c60 2023-10-10T20:34:53.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57a72a1700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00cdea0 con 0x7f57a0069560 2023-10-10T20:34:53.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57a72a1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00ce180 con 0x7f57a0159890 2023-10-10T20:34:53.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57a583e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a0159c60 0x7f57a0156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:53.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57a583e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a0159c60 0x7f57a0156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44762/0 (socket says 172.21.15.89:44762) 2023-10-10T20:34:53.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57a583e700 1 -- 172.21.15.89:0/3499293662 learned_addr learned my addr 172.21.15.89:0/3499293662 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:53.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57a503d700 1 --2- 172.21.15.89:0/3499293662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57a0069560 0x7f57a014dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:53.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57a483c700 1 --2- 172.21.15.89:0/3499293662 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57a0159890 0x7f57a0155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:53.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57a583e700 1 -- 172.21.15.89:0/3499293662 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57a0159890 msgr2=0x7f57a0155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:53.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57a583e700 1 --2- 172.21.15.89:0/3499293662 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57a0159890 0x7f57a0155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:53.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57a583e700 1 -- 172.21.15.89:0/3499293662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57a0069560 msgr2=0x7f57a014dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:53.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57a583e700 1 --2- 172.21.15.89:0/3499293662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57a0069560 0x7f57a014dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:53.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57a583e700 1 -- 172.21.15.89:0/3499293662 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c009a20 con 0x7f57a0159c60 2023-10-10T20:34:53.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57a583e700 1 --2- 172.21.15.89:0/3499293662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a0159c60 0x7f57a0156410 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f579800c880 tx=0x7f5798008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:53.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57957fa700 1 -- 172.21.15.89:0/3499293662 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798011600 con 0x7f57a0159c60 2023-10-10T20:34:53.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57957fa700 1 -- 172.21.15.89:0/3499293662 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5798011760 con 0x7f57a0159c60 2023-10-10T20:34:53.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57a72a1700 1 -- 172.21.15.89:0/3499293662 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a00ce4f0 con 0x7f57a0159c60 2023-10-10T20:34:53.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57957fa700 1 -- 172.21.15.89:0/3499293662 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798011b20 con 0x7f57a0159c60 2023-10-10T20:34:53.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.528+0000 7f57a72a1700 1 -- 172.21.15.89:0/3499293662 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a01517c0 con 0x7f57a0159c60 2023-10-10T20:34:53.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.529+0000 7f57a72a1700 1 -- 172.21.15.89:0/3499293662 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a00cecf0 con 0x7f57a0159c60 2023-10-10T20:34:53.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.530+0000 7f57957fa700 1 -- 172.21.15.89:0/3499293662 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5798018360 con 0x7f57a0159c60 2023-10-10T20:34:53.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.530+0000 7f57957fa700 1 --2- 172.21.15.89:0/3499293662 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5784066f30 0x7f57840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:53.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.530+0000 7f57957fa700 1 -- 172.21.15.89:0/3499293662 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f57980840f0 con 0x7f57a0159c60 2023-10-10T20:34:53.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.530+0000 7f57a503d700 1 --2- 172.21.15.89:0/3499293662 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5784066f30 0x7f57840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:53.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.530+0000 7f57a503d700 1 --2- 172.21.15.89:0/3499293662 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5784066f30 0x7f57840693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f579c009e60 tx=0x7f579c019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:53.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.534+0000 7f57957fa700 1 -- 172.21.15.89:0/3499293662 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f57a00cecf0 con 0x7f57a0159c60 2023-10-10T20:34:53.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.760+0000 7f57a72a1700 1 -- 172.21.15.89:0/3499293662 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f57a004c3c0 con 0x7f57a0159c60 2023-10-10T20:34:53.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.760+0000 7f57957fa700 1 -- 172.21.15.89:0/3499293662 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5798054df0 con 0x7f57a0159c60 2023-10-10T20:34:53.764 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:34:53.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.762+0000 7f57a72a1700 1 -- 172.21.15.89:0/3499293662 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5784066f30 msgr2=0x7f57840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:53.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.762+0000 7f57a72a1700 1 --2- 172.21.15.89:0/3499293662 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5784066f30 0x7f57840693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f579c009e60 tx=0x7f579c019000 comp rx=0 tx=0).stop 2023-10-10T20:34:53.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.762+0000 7f57a72a1700 1 -- 172.21.15.89:0/3499293662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a0159c60 msgr2=0x7f57a0156410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:53.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.762+0000 7f57a72a1700 1 --2- 172.21.15.89:0/3499293662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a0159c60 0x7f57a0156410 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f579800c880 tx=0x7f5798008040 comp rx=0 tx=0).stop 2023-10-10T20:34:53.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.762+0000 7f57a72a1700 1 -- 172.21.15.89:0/3499293662 shutdown_connections 2023-10-10T20:34:53.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.762+0000 7f57a72a1700 1 --2- 172.21.15.89:0/3499293662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a0159c60 0x7f57a0156410 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:53.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.762+0000 7f57a72a1700 1 --2- 172.21.15.89:0/3499293662 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5784066f30 0x7f57840693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:53.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.762+0000 7f57a72a1700 1 --2- 172.21.15.89:0/3499293662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57a0069560 0x7f57a014dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:53.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.762+0000 7f57a72a1700 1 --2- 172.21.15.89:0/3499293662 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57a0159890 0x7f57a0155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:53.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.762+0000 7f57a72a1700 1 -- 172.21.15.89:0/3499293662 >> 172.21.15.89:0/3499293662 conn(0x7f57a0068e20 msgr2=0x7f57a0158a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:53.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.762+0000 7f57a72a1700 1 -- 172.21.15.89:0/3499293662 shutdown_connections 2023-10-10T20:34:53.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:53.762+0000 7f57a72a1700 1 -- 172.21.15.89:0/3499293662 wait complete. 2023-10-10T20:34:53.776 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-10T20:34:53.776 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:34:53.777 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-10-10T20:34:53.777 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:34:53.777 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:34:53.777 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-10T20:34:53.778 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-10T20:34:53.778 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:34:53.778 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:34:53.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:53.779 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-10-10T20:34:53.797 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:34:53.798 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:34:53.798 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:34:53.798 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:53.798 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-10-10T20:34:53.798 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:53.799 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:34:53.799 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:34:53.799 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-10-10T20:34:53.799 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:34:53.799 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:34:53.800 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-10-10T20:34:53.800 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:34:53.800 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:34:53.800 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:34:53.800 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-10-10T20:34:53.801 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:34:53.801 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42489sec preferred_lft 42489sec 2023-10-10T20:34:53.801 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-10-10T20:34:53.801 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:53.801 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-10T20:34:53.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:53.802 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:34:53.802 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:34:53.803 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add name ceph-brx type bridge 2023-10-10T20:34:53.803 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr flush dev ceph-brx 2023-10-10T20:34:53.803 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set ceph-brx up 2023-10-10T20:34:53.804 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-10T20:34:53.804 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:34:53.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:53.965 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:34:54.005 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-10-10T20:34:54.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:54.007 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-10-10T20:34:54.082 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-10-10T20:34:54.082 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:34:54.082 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:34:54.083 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:34:54.083 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:34:54.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:54.084 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:34:54.084 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:34:54.084 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:34:54.085 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:34:54.085 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:34:54.085 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:34:54.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:54.346 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-10-10T20:34:54.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:54.360 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list-id 2023-10-10T20:34:54.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:54.415 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:34:54.415 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:34:54.416 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:34:54.416 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-10T20:34:54.416 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:34:54.504 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-10T20:34:54.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:54.505 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:34:54.506 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:34:54.506 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-10T20:34:54.506 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-10T20:34:54.507 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-10T20:34:54.507 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-10T20:34:54.507 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-10T20:34:54.507 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:34:54.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:54.690 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:34:54.690 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:34:54.691 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-10-10T20:34:54.691 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set dev brx.0 master ceph-brx 2023-10-10T20:34:54.691 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:34:54.783 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:34:54.783 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:34:54.798 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:34:54.798 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:34:54.798 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:34:54.852 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:34:54.853 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:34:54.853 DEBUG:teuthology.orchestra.run.smithi111:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:34:54.944 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:34:54.944 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:34:54.944 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:34:55.020 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:34:55.020 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:34:55.020 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:34:55.021 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:34:55.021 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:34:55.041 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:34:55.042 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:34:55.042 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:34:55.042 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:34:55.042 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:34:55.043 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:34:55.043 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:34:55.043 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:34:55.044 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:34:55.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:55.045 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:34:55.069 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-10-10T20:34:55.070 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:34:55.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.190+0000 7efebf245700 1 Processor -- start 2023-10-10T20:34:55.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.190+0000 7efebf245700 1 -- start start 2023-10-10T20:34:55.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.191+0000 7efebf245700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7efeb8159890 0x7efeb815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:55.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.191+0000 7efebf245700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efeb8159c60 0x7efeb815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:55.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.191+0000 7efebf245700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb815a160 con 0x7efeb815db10 2023-10-10T20:34:55.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.191+0000 7efebf245700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb815df50 con 0x7efeb8159c60 2023-10-10T20:34:55.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.191+0000 7efebf245700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb80d13a0 con 0x7efeb8159890 2023-10-10T20:34:55.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.191+0000 7efeb7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efeb8159c60 0x7efeb815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:55.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.191+0000 7efebd7e2700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7efeb815db10 0x7efeb8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43890/0 (socket says 172.21.15.89:43890) 2023-10-10T20:34:55.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.191+0000 7efebd7e2700 1 -- 172.21.15.89:0/881837814 learned_addr learned my addr 172.21.15.89:0/881837814 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:55.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.191+0000 7efebcfe1700 1 --2- 172.21.15.89:0/881837814 >> v2:172.21.15.111:3300/0 conn(0x7efeb8159890 0x7efeb815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:55.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.191+0000 7efeb7fff700 1 -- 172.21.15.89:0/881837814 >> v2:172.21.15.111:3300/0 conn(0x7efeb8159890 msgr2=0x7efeb815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:55.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.191+0000 7efeb7fff700 1 --2- 172.21.15.89:0/881837814 >> v2:172.21.15.111:3300/0 conn(0x7efeb8159890 0x7efeb815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:55.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.191+0000 7efeb7fff700 1 -- 172.21.15.89:0/881837814 >> v1:172.21.15.89:6789/0 conn(0x7efeb815db10 legacy=0x7efeb8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:55.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.191+0000 7efeb77fe700 1 -- 172.21.15.89:0/881837814 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034760055 0 0) 0x7efeb815a160 con 0x7efeb815db10 2023-10-10T20:34:55.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.191+0000 7efeb7fff700 1 -- 172.21.15.89:0/881837814 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb815e140 con 0x7efeb8159c60 2023-10-10T20:34:55.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.192+0000 7efeb7fff700 1 --2- 172.21.15.89:0/881837814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb8159c60 0x7efeb815d5d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efeb80d1090 tx=0x7efea00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c8215a62eb56f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:55.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.192+0000 7efeb77fe700 1 -- 172.21.15.89:0/881837814 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea000b6b0 con 0x7efeb8159c60 2023-10-10T20:34:55.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.192+0000 7efeb77fe700 1 -- 172.21.15.89:0/881837814 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea000b810 con 0x7efeb8159c60 2023-10-10T20:34:55.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.192+0000 7efeb77fe700 1 -- 172.21.15.89:0/881837814 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea000bbb0 con 0x7efeb8159c60 2023-10-10T20:34:55.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.192+0000 7efebf245700 1 -- 172.21.15.89:0/881837814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb8159c60 msgr2=0x7efeb815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:55.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.192+0000 7efebf245700 1 --2- 172.21.15.89:0/881837814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb8159c60 0x7efeb815d5d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efeb80d1090 tx=0x7efea00091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:55.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.192+0000 7efebf245700 1 -- 172.21.15.89:0/881837814 shutdown_connections 2023-10-10T20:34:55.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.192+0000 7efebf245700 1 --2- 172.21.15.89:0/881837814 >> v2:172.21.15.111:3300/0 conn(0x7efeb8159890 0x7efeb815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:55.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.192+0000 7efebf245700 1 --2- 172.21.15.89:0/881837814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb8159c60 0x7efeb815d5d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:55.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.192+0000 7efebf245700 1 -- 172.21.15.89:0/881837814 >> 172.21.15.89:0/881837814 conn(0x7efeb8068e20 msgr2=0x7efeb80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:55.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.192+0000 7efebf245700 1 -- 172.21.15.89:0/881837814 shutdown_connections 2023-10-10T20:34:55.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.192+0000 7efebf245700 1 -- 172.21.15.89:0/881837814 wait complete. 2023-10-10T20:34:55.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.193+0000 7efebf245700 1 Processor -- start 2023-10-10T20:34:55.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.194+0000 7efebf245700 1 -- start start 2023-10-10T20:34:55.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.194+0000 7efebf245700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeb8159890 0x7efeb80cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:55.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.194+0000 7efebf245700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efeb8159c60 0x7efeb814d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:55.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.194+0000 7efebf245700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb815db10 0x7efeb8152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:55.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.194+0000 7efebf245700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb8151ff0 con 0x7efeb815db10 2023-10-10T20:34:55.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.194+0000 7efebf245700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb80cdc20 con 0x7efeb8159890 2023-10-10T20:34:55.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.194+0000 7efebf245700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb80cdec0 con 0x7efeb8159c60 2023-10-10T20:34:55.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.194+0000 7efebd7e2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb815db10 0x7efeb8152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:55.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.195+0000 7efebd7e2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb815db10 0x7efeb8152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44778/0 (socket says 172.21.15.89:44778) 2023-10-10T20:34:55.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.195+0000 7efebd7e2700 1 -- 172.21.15.89:0/3754584379 learned_addr learned my addr 172.21.15.89:0/3754584379 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:55.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.195+0000 7efebcfe1700 1 --2- 172.21.15.89:0/3754584379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeb8159890 0x7efeb80cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:55.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.195+0000 7efeb7fff700 1 --2- 172.21.15.89:0/3754584379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efeb8159c60 0x7efeb814d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:55.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.195+0000 7efebd7e2700 1 -- 172.21.15.89:0/3754584379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efeb8159c60 msgr2=0x7efeb814d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:55.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.195+0000 7efebd7e2700 1 --2- 172.21.15.89:0/3754584379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efeb8159c60 0x7efeb814d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:55.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.195+0000 7efebd7e2700 1 -- 172.21.15.89:0/3754584379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeb8159890 msgr2=0x7efeb80cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:55.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.195+0000 7efebd7e2700 1 --2- 172.21.15.89:0/3754584379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeb8159890 0x7efeb80cec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:55.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.195+0000 7efebd7e2700 1 -- 172.21.15.89:0/3754584379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea0009420 con 0x7efeb815db10 2023-10-10T20:34:55.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.195+0000 7efebcfe1700 1 --2- 172.21.15.89:0/3754584379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeb8159890 0x7efeb80cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:55.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.195+0000 7efebd7e2700 1 --2- 172.21.15.89:0/3754584379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb815db10 0x7efeb8152300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efeb000c530 tx=0x7efeb0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:55.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.195+0000 7efeb4ff9700 1 -- 172.21.15.89:0/3754584379 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0005b30 con 0x7efeb815db10 2023-10-10T20:34:55.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.195+0000 7efeb4ff9700 1 -- 172.21.15.89:0/3754584379 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb0005c90 con 0x7efeb815db10 2023-10-10T20:34:55.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.195+0000 7efeb4ff9700 1 -- 172.21.15.89:0/3754584379 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0007320 con 0x7efeb815db10 2023-10-10T20:34:55.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.195+0000 7efebf245700 1 -- 172.21.15.89:0/3754584379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb8152e50 con 0x7efeb815db10 2023-10-10T20:34:55.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.195+0000 7efebf245700 1 -- 172.21.15.89:0/3754584379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb80ce720 con 0x7efeb815db10 2023-10-10T20:34:55.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.196+0000 7efebf245700 1 -- 172.21.15.89:0/3754584379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb80ce080 con 0x7efeb815db10 2023-10-10T20:34:55.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.196+0000 7efeb4ff9700 1 -- 172.21.15.89:0/3754584379 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efeb001b3b0 con 0x7efeb815db10 2023-10-10T20:34:55.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.197+0000 7efeb4ff9700 1 --2- 172.21.15.89:0/3754584379 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efe98066f30 0x7efe980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:55.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.197+0000 7efeb4ff9700 1 -- 172.21.15.89:0/3754584379 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efeb0083a50 con 0x7efeb815db10 2023-10-10T20:34:55.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.197+0000 7efebcfe1700 1 --2- 172.21.15.89:0/3754584379 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efe98066f30 0x7efe980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:55.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.197+0000 7efebcfe1700 1 --2- 172.21.15.89:0/3754584379 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efe98066f30 0x7efe980693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efea80075e0 tx=0x7efea80097e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:55.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.201+0000 7efeb4ff9700 1 -- 172.21.15.89:0/3754584379 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efeb0050cc0 con 0x7efeb815db10 2023-10-10T20:34:55.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.428+0000 7efebf245700 1 -- 172.21.15.89:0/3754584379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efeb804c3c0 con 0x7efeb815db10 2023-10-10T20:34:55.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.428+0000 7efeb4ff9700 1 -- 172.21.15.89:0/3754584379 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7efeb0017070 con 0x7efeb815db10 2023-10-10T20:34:55.431 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:34:55.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.430+0000 7efebf245700 1 -- 172.21.15.89:0/3754584379 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efe98066f30 msgr2=0x7efe980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:55.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.430+0000 7efebf245700 1 --2- 172.21.15.89:0/3754584379 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efe98066f30 0x7efe980693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efea80075e0 tx=0x7efea80097e0 comp rx=0 tx=0).stop 2023-10-10T20:34:55.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.430+0000 7efebf245700 1 -- 172.21.15.89:0/3754584379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb815db10 msgr2=0x7efeb8152300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:55.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.430+0000 7efebf245700 1 --2- 172.21.15.89:0/3754584379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb815db10 0x7efeb8152300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efeb000c530 tx=0x7efeb0007f90 comp rx=0 tx=0).stop 2023-10-10T20:34:55.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.430+0000 7efebf245700 1 -- 172.21.15.89:0/3754584379 shutdown_connections 2023-10-10T20:34:55.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.430+0000 7efebf245700 1 --2- 172.21.15.89:0/3754584379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb815db10 0x7efeb8152300 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.430+0000 7efebf245700 1 --2- 172.21.15.89:0/3754584379 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efe98066f30 0x7efe980693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.430+0000 7efebf245700 1 --2- 172.21.15.89:0/3754584379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeb8159890 0x7efeb80cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.430+0000 7efebf245700 1 --2- 172.21.15.89:0/3754584379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efeb8159c60 0x7efeb814d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.430+0000 7efebf245700 1 -- 172.21.15.89:0/3754584379 >> 172.21.15.89:0/3754584379 conn(0x7efeb8068e20 msgr2=0x7efeb8069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.430+0000 7efebf245700 1 -- 172.21.15.89:0/3754584379 shutdown_connections 2023-10-10T20:34:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:55.430+0000 7efebf245700 1 -- 172.21.15.89:0/3754584379 wait complete. 2023-10-10T20:34:55.444 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-10T20:34:55.445 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:34:55.445 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-10-10T20:34:55.445 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-10T20:34:55.445 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-10T20:34:55.446 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-10T20:34:55.446 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-10T20:34:55.446 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:34:55.446 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-10T20:34:55.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:55.447 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-10-10T20:34:55.462 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:34:55.463 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:34:55.463 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:34:55.463 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:55.463 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-10-10T20:34:55.464 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:55.464 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:34:55.464 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:34:55.464 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-10-10T20:34:55.464 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:34:55.464 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:34:55.465 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-10-10T20:34:55.465 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:34:55.465 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:34:55.465 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:34:55.466 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-10-10T20:34:55.466 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:34:55.466 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42487sec preferred_lft 42487sec 2023-10-10T20:34:55.466 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-10-10T20:34:55.466 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:55.466 INFO:teuthology.orchestra.run.smithi111.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:34:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 1e:d0:81:1b:e2:6b brd ff:ff:ff:ff:ff:ff 2023-10-10T20:34:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:34:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::a486:a6ff:fe5f:6021/64 scope link tentative 2023-10-10T20:34:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:55.468 INFO:teuthology.orchestra.run.smithi111.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:34:55.468 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 1e:d0:81:1b:e2:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:34:55.468 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::1cd0:81ff:fe1b:e26b/64 scope link tentative 2023-10-10T20:34:55.468 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:55.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:55.469 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:34:55.526 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-10-10T20:34:55.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:55.528 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-10-10T20:34:55.584 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-10-10T20:34:55.584 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:34:55.585 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:34:55.585 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:34:55.585 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:34:55.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:55.586 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:34:55.586 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:34:55.586 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:34:55.587 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:34:55.587 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:34:55.587 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:34:55.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:55.684 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-10-10T20:34:55.699 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:34:55.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:55.700 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list-id 2023-10-10T20:34:55.753 INFO:teuthology.orchestra.run.smithi111.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-10T20:34:55.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:55.754 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:34:55.755 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:34:55.755 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:34:55.755 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-10T20:34:55.755 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:34:55.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:55.843 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-10T20:34:55.883 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:34:55.883 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:34:55.884 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:34:55.884 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:55.884 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-10-10T20:34:55.884 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:55.884 INFO:teuthology.orchestra.run.smithi111.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:34:55.885 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 66:b8:3a:2c:50:19 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-10T20:34:55.885 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-10T20:34:55.885 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:55.885 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::64b8:3aff:fe2c:5019/64 scope link 2023-10-10T20:34:55.885 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:55.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:55.904 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-10T20:34:55.977 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:34:55.977 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:34:55.978 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:34:55.978 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:55.978 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-10-10T20:34:55.978 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:55.978 INFO:teuthology.orchestra.run.smithi111.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:34:55.979 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 66:b8:3a:2c:50:19 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-10T20:34:55.979 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-10T20:34:55.979 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:55.979 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::64b8:3aff:fe2c:5019/64 scope link 2023-10-10T20:34:55.979 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:34:55.994 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-10T20:34:55.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:55.995 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:34:55.995 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:34:55.995 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-10T20:34:55.996 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-10T20:34:55.996 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-10T20:34:55.996 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-10T20:34:55.996 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-10T20:34:55.997 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:34:56.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:56.221 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:34:56.221 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:34:56.222 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.1 up 2023-10-10T20:34:56.222 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set dev brx.1 master ceph-brx 2023-10-10T20:34:56.222 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:34:56.301 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:34:56.302 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-10T20:34:56.316 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-10T20:34:56.316 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:34:56.317 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-10T20:34:56.370 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:34:56.371 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:34:56.371 DEBUG:teuthology.orchestra.run.smithi111:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-10T20:34:56.450 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-10T20:34:56.450 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-10T20:34:56.450 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:34:56.458 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-10T20:34:56.458 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-10T20:34:56.459 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:34:56.459 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:34:56.459 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:34:56.480 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-10T20:34:56.481 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-10T20:34:56.481 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:34:56.481 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:34:56.481 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:34:56.481 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:34:56.482 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:34:56.482 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:34:56.483 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:34:56.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:34:56.484 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-10T20:34:56.547 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-10T20:34:56.565 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:34:56.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.690+0000 7f9426560700 1 Processor -- start 2023-10-10T20:34:56.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.690+0000 7f9426560700 1 -- start start 2023-10-10T20:34:56.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.691+0000 7f9426560700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9420159c60 0x7f942015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:56.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.691+0000 7f9426560700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f942015db10 0x7f9420069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:56.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.691+0000 7f9426560700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f942015a160 con 0x7f9420159890 2023-10-10T20:34:56.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.691+0000 7f9426560700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942015df50 con 0x7f942015db10 2023-10-10T20:34:56.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.691+0000 7f9426560700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94200d13a0 con 0x7f9420159c60 2023-10-10T20:34:56.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.691+0000 7f941ffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9420159890 0x7f942015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43896/0 (socket says 172.21.15.89:43896) 2023-10-10T20:34:56.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.691+0000 7f9424afd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f942015db10 0x7f9420069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:56.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.691+0000 7f941ffff700 1 -- 172.21.15.89:0/600499580 learned_addr learned my addr 172.21.15.89:0/600499580 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:56.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.691+0000 7f941f7fe700 1 --2- 172.21.15.89:0/600499580 >> v2:172.21.15.111:3300/0 conn(0x7f9420159c60 0x7f942015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:56.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.691+0000 7f9424afd700 1 -- 172.21.15.89:0/600499580 >> v2:172.21.15.111:3300/0 conn(0x7f9420159c60 msgr2=0x7f942015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:56.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.692+0000 7f9424afd700 1 --2- 172.21.15.89:0/600499580 >> v2:172.21.15.111:3300/0 conn(0x7f9420159c60 0x7f942015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:56.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.692+0000 7f9424afd700 1 -- 172.21.15.89:0/600499580 >> v1:172.21.15.89:6789/0 conn(0x7f9420159890 legacy=0x7f942015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:56.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.692+0000 7f9424afd700 1 -- 172.21.15.89:0/600499580 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942015e140 con 0x7f942015db10 2023-10-10T20:34:56.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.692+0000 7f941effd700 1 -- 172.21.15.89:0/600499580 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615784156 0 0) 0x7f942015a160 con 0x7f9420159890 2023-10-10T20:34:56.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.692+0000 7f941f7fe700 1 --2- 172.21.15.89:0/600499580 >> v2:172.21.15.111:3300/0 conn(0x7f9420159c60 0x7f942015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:34:56.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.692+0000 7f9424afd700 1 --2- 172.21.15.89:0/600499580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f942015db10 0x7f9420069560 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f94200b67f0 tx=0x7f9418009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f7830ccd9d9a05 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:56.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.692+0000 7f941effd700 1 -- 172.21.15.89:0/600499580 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941800e070 con 0x7f942015db10 2023-10-10T20:34:56.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.692+0000 7f941effd700 1 -- 172.21.15.89:0/600499580 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941800bc30 con 0x7f942015db10 2023-10-10T20:34:56.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.692+0000 7f941effd700 1 -- 172.21.15.89:0/600499580 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418012430 con 0x7f942015db10 2023-10-10T20:34:56.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.692+0000 7f9426560700 1 -- 172.21.15.89:0/600499580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f942015db10 msgr2=0x7f9420069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:56.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.692+0000 7f9426560700 1 --2- 172.21.15.89:0/600499580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f942015db10 0x7f9420069560 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f94200b67f0 tx=0x7f9418009780 comp rx=0 tx=0).stop 2023-10-10T20:34:56.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.692+0000 7f9426560700 1 -- 172.21.15.89:0/600499580 shutdown_connections 2023-10-10T20:34:56.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.692+0000 7f9426560700 1 --2- 172.21.15.89:0/600499580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f942015db10 0x7f9420069560 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:56.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.692+0000 7f9426560700 1 --2- 172.21.15.89:0/600499580 >> v2:172.21.15.111:3300/0 conn(0x7f9420159c60 0x7f942015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:56.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.692+0000 7f9426560700 1 -- 172.21.15.89:0/600499580 >> 172.21.15.89:0/600499580 conn(0x7f9420068e20 msgr2=0x7f94200d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:56.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.692+0000 7f9426560700 1 -- 172.21.15.89:0/600499580 shutdown_connections 2023-10-10T20:34:56.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.692+0000 7f9426560700 1 -- 172.21.15.89:0/600499580 wait complete. 2023-10-10T20:34:56.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.693+0000 7f9426560700 1 Processor -- start 2023-10-10T20:34:56.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.694+0000 7f9426560700 1 -- start start 2023-10-10T20:34:56.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.694+0000 7f9426560700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9420159890 0x7f94201508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:56.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.694+0000 7f9426560700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9420159c60 0x7f9420154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:56.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.694+0000 7f9426560700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f942015db10 0x7f9420151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:56.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.694+0000 7f9426560700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9420153d20 con 0x7f9420159c60 2023-10-10T20:34:56.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.694+0000 7f9426560700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9420154b40 con 0x7f942015db10 2023-10-10T20:34:56.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.694+0000 7f9426560700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9420153f30 con 0x7f9420159890 2023-10-10T20:34:56.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.695+0000 7f941f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9420159c60 0x7f9420154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:56.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.695+0000 7f941f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9420159c60 0x7f9420154fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44798/0 (socket says 172.21.15.89:44798) 2023-10-10T20:34:56.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.695+0000 7f941f7fe700 1 -- 172.21.15.89:0/2914277403 learned_addr learned my addr 172.21.15.89:0/2914277403 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:56.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.695+0000 7f9424afd700 1 --2- 172.21.15.89:0/2914277403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f942015db10 0x7f9420151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:56.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.695+0000 7f941ffff700 1 --2- 172.21.15.89:0/2914277403 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9420159890 0x7f94201508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:56.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.695+0000 7f941f7fe700 1 -- 172.21.15.89:0/2914277403 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9420159890 msgr2=0x7f94201508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:56.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.695+0000 7f941f7fe700 1 --2- 172.21.15.89:0/2914277403 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9420159890 0x7f94201508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:56.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.695+0000 7f941f7fe700 1 -- 172.21.15.89:0/2914277403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f942015db10 msgr2=0x7f9420151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:56.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.695+0000 7f941f7fe700 1 --2- 172.21.15.89:0/2914277403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f942015db10 0x7f9420151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:56.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.695+0000 7f941f7fe700 1 -- 172.21.15.89:0/2914277403 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9418009a20 con 0x7f9420159c60 2023-10-10T20:34:56.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.695+0000 7f9424afd700 1 --2- 172.21.15.89:0/2914277403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f942015db10 0x7f9420151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:56.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.695+0000 7f941f7fe700 1 --2- 172.21.15.89:0/2914277403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9420159c60 0x7f9420154fd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f941400c830 tx=0x7f9414008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:56.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.695+0000 7f9407fff700 1 -- 172.21.15.89:0/2914277403 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9414011600 con 0x7f9420159c60 2023-10-10T20:34:56.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.695+0000 7f9407fff700 1 -- 172.21.15.89:0/2914277403 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9414011760 con 0x7f9420159c60 2023-10-10T20:34:56.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.695+0000 7f9426560700 1 -- 172.21.15.89:0/2914277403 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9420154210 con 0x7f9420159c60 2023-10-10T20:34:56.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.695+0000 7f9407fff700 1 -- 172.21.15.89:0/2914277403 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9414011b20 con 0x7f9420159c60 2023-10-10T20:34:56.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.695+0000 7f9426560700 1 -- 172.21.15.89:0/2914277403 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94201547d0 con 0x7f9420159c60 2023-10-10T20:34:56.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.696+0000 7f9426560700 1 -- 172.21.15.89:0/2914277403 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942004c3c0 con 0x7f9420159c60 2023-10-10T20:34:56.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.697+0000 7f9407fff700 1 -- 172.21.15.89:0/2914277403 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f941400a3e0 con 0x7f9420159c60 2023-10-10T20:34:56.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.697+0000 7f9407fff700 1 --2- 172.21.15.89:0/2914277403 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f940c066f30 0x7f940c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:56.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.697+0000 7f9407fff700 1 -- 172.21.15.89:0/2914277403 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f94140849f0 con 0x7f9420159c60 2023-10-10T20:34:56.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.697+0000 7f941ffff700 1 --2- 172.21.15.89:0/2914277403 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f940c066f30 0x7f940c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:56.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.697+0000 7f941ffff700 1 --2- 172.21.15.89:0/2914277403 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f940c066f30 0x7f940c0693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f942015c8b0 tx=0x7f9410009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:56.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.701+0000 7f9407fff700 1 -- 172.21.15.89:0/2914277403 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9414051c60 con 0x7f9420159c60 2023-10-10T20:34:56.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.886+0000 7f9426560700 1 -- 172.21.15.89:0/2914277403 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9420154c80 con 0x7f9420159c60 2023-10-10T20:34:56.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.887+0000 7f9407fff700 1 -- 172.21.15.89:0/2914277403 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f9414015090 con 0x7f9420159c60 2023-10-10T20:34:56.889 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:56.889 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:36.324920+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:34:32.342167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:34:33.671763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:34:56.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.889+0000 7f9426560700 1 -- 172.21.15.89:0/2914277403 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f940c066f30 msgr2=0x7f940c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:56.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.889+0000 7f9426560700 1 --2- 172.21.15.89:0/2914277403 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f940c066f30 0x7f940c0693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f942015c8b0 tx=0x7f9410009040 comp rx=0 tx=0).stop 2023-10-10T20:34:56.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.889+0000 7f9426560700 1 -- 172.21.15.89:0/2914277403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9420159c60 msgr2=0x7f9420154fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:56.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.889+0000 7f9426560700 1 --2- 172.21.15.89:0/2914277403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9420159c60 0x7f9420154fd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f941400c830 tx=0x7f9414008040 comp rx=0 tx=0).stop 2023-10-10T20:34:56.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.889+0000 7f9426560700 1 -- 172.21.15.89:0/2914277403 shutdown_connections 2023-10-10T20:34:56.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.889+0000 7f9426560700 1 --2- 172.21.15.89:0/2914277403 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f940c066f30 0x7f940c0693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:56.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.889+0000 7f9426560700 1 --2- 172.21.15.89:0/2914277403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f942015db10 0x7f9420151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:56.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.889+0000 7f9426560700 1 --2- 172.21.15.89:0/2914277403 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9420159890 0x7f94201508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:56.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.889+0000 7f9426560700 1 --2- 172.21.15.89:0/2914277403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9420159c60 0x7f9420154fd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:56.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.889+0000 7f9426560700 1 -- 172.21.15.89:0/2914277403 >> 172.21.15.89:0/2914277403 conn(0x7f9420068e20 msgr2=0x7f942015ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:56.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.889+0000 7f9426560700 1 -- 172.21.15.89:0/2914277403 shutdown_connections 2023-10-10T20:34:56.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:56.889+0000 7f9426560700 1 -- 172.21.15.89:0/2914277403 wait complete. 2023-10-10T20:34:56.904 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:34:32.342167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:34:33.671763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:34:56.904 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:34:57.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.019+0000 7f5e0c74a700 1 Processor -- start 2023-10-10T20:34:57.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.019+0000 7f5e0c74a700 1 -- start start 2023-10-10T20:34:57.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.020+0000 7f5e0c74a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5e04159890 0x7f5e0415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.020+0000 7f5e0c74a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5e04069560 0x7f5e040699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.020+0000 7f5e0c74a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e04069ee0 con 0x7f5e04159c60 2023-10-10T20:34:57.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.020+0000 7f5e0c74a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e0415a160 con 0x7f5e04159890 2023-10-10T20:34:57.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.020+0000 7f5e0c74a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e040d13a0 con 0x7f5e04069560 2023-10-10T20:34:57.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.020+0000 7f5e0a4e6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5e04159890 0x7f5e0415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:57.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.020+0000 7f5e09ce5700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5e04159c60 0x7f5e0415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43910/0 (socket says 172.21.15.89:43910) 2023-10-10T20:34:57.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.020+0000 7f5e0a4e6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5e04159890 0x7f5e0415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44810/0 (socket says 172.21.15.89:44810) 2023-10-10T20:34:57.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.020+0000 7f5e09ce5700 1 -- 172.21.15.89:0/3777883672 learned_addr learned my addr 172.21.15.89:0/3777883672 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:57.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.021+0000 7f5e0ace7700 1 --2- 172.21.15.89:0/3777883672 >> v2:172.21.15.111:3300/0 conn(0x7f5e04069560 0x7f5e040699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:57.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.021+0000 7f5e0a4e6700 1 -- 172.21.15.89:0/3777883672 >> v2:172.21.15.111:3300/0 conn(0x7f5e04069560 msgr2=0x7f5e040699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:57.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.021+0000 7f5e0a4e6700 1 --2- 172.21.15.89:0/3777883672 >> v2:172.21.15.111:3300/0 conn(0x7f5e04069560 0x7f5e040699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.021+0000 7f5e0a4e6700 1 -- 172.21.15.89:0/3777883672 >> v1:172.21.15.89:6789/0 conn(0x7f5e04159c60 legacy=0x7f5e0415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.021+0000 7f5e0a4e6700 1 -- 172.21.15.89:0/3777883672 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0415e140 con 0x7f5e04159890 2023-10-10T20:34:57.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.021+0000 7f5e094e4700 1 -- 172.21.15.89:0/3777883672 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244545987 0 0) 0x7f5e04069ee0 con 0x7f5e04159c60 2023-10-10T20:34:57.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.021+0000 7f5e0a4e6700 1 --2- 172.21.15.89:0/3777883672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e04159890 0x7f5e0415abd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5e0406a890 tx=0x7f5df4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9f16d438ad2007 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:57.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.021+0000 7f5e094e4700 1 -- 172.21.15.89:0/3777883672 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df400e070 con 0x7f5e04159890 2023-10-10T20:34:57.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.021+0000 7f5e094e4700 1 -- 172.21.15.89:0/3777883672 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df400bc30 con 0x7f5e04159890 2023-10-10T20:34:57.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.021+0000 7f5e094e4700 1 -- 172.21.15.89:0/3777883672 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4012430 con 0x7f5e04159890 2023-10-10T20:34:57.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.021+0000 7f5e0c74a700 1 -- 172.21.15.89:0/3777883672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e04159890 msgr2=0x7f5e0415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.021+0000 7f5e0c74a700 1 --2- 172.21.15.89:0/3777883672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e04159890 0x7f5e0415abd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5e0406a890 tx=0x7f5df4009780 comp rx=0 tx=0).stop 2023-10-10T20:34:57.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.022+0000 7f5e0c74a700 1 -- 172.21.15.89:0/3777883672 shutdown_connections 2023-10-10T20:34:57.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.022+0000 7f5e0c74a700 1 --2- 172.21.15.89:0/3777883672 >> v2:172.21.15.111:3300/0 conn(0x7f5e04069560 0x7f5e040699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.022+0000 7f5e0c74a700 1 --2- 172.21.15.89:0/3777883672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e04159890 0x7f5e0415abd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.022+0000 7f5e0c74a700 1 -- 172.21.15.89:0/3777883672 >> 172.21.15.89:0/3777883672 conn(0x7f5e04068e20 msgr2=0x7f5e040d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:57.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.022+0000 7f5e0c74a700 1 -- 172.21.15.89:0/3777883672 shutdown_connections 2023-10-10T20:34:57.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.022+0000 7f5e0c74a700 1 -- 172.21.15.89:0/3777883672 wait complete. 2023-10-10T20:34:57.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.023+0000 7f5e0c74a700 1 Processor -- start 2023-10-10T20:34:57.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.023+0000 7f5e0c74a700 1 -- start start 2023-10-10T20:34:57.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.023+0000 7f5e0c74a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e04069560 0x7f5e0416a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.023+0000 7f5e0c74a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e04159890 0x7f5e040cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.023+0000 7f5e0c74a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e04159c60 0x7f5e040ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.023+0000 7f5e0c74a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040cff20 con 0x7f5e04159c60 2023-10-10T20:34:57.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.023+0000 7f5e0c74a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040ca870 con 0x7f5e04069560 2023-10-10T20:34:57.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.023+0000 7f5e0c74a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040cf960 con 0x7f5e04159890 2023-10-10T20:34:57.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.024+0000 7f5e0ace7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e04159c60 0x7f5e040ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:57.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.024+0000 7f5e0ace7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e04159c60 0x7f5e040ca330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44816/0 (socket says 172.21.15.89:44816) 2023-10-10T20:34:57.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.024+0000 7f5e0ace7700 1 -- 172.21.15.89:0/278424189 learned_addr learned my addr 172.21.15.89:0/278424189 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:57.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.024+0000 7f5e0a4e6700 1 --2- 172.21.15.89:0/278424189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e04069560 0x7f5e0416a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:57.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.024+0000 7f5e09ce5700 1 --2- 172.21.15.89:0/278424189 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e04159890 0x7f5e040cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:57.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.024+0000 7f5e0ace7700 1 -- 172.21.15.89:0/278424189 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e04159890 msgr2=0x7f5e040cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.024+0000 7f5e0ace7700 1 --2- 172.21.15.89:0/278424189 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e04159890 0x7f5e040cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.024+0000 7f5e0ace7700 1 -- 172.21.15.89:0/278424189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e04069560 msgr2=0x7f5e0416a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.024+0000 7f5e0ace7700 1 --2- 172.21.15.89:0/278424189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e04069560 0x7f5e0416a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.024+0000 7f5e0ace7700 1 -- 172.21.15.89:0/278424189 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df4009a20 con 0x7f5e04159c60 2023-10-10T20:34:57.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.024+0000 7f5e0ace7700 1 --2- 172.21.15.89:0/278424189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e04159c60 0x7f5e040ca330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00c880 tx=0x7f5dfc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:57.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.024+0000 7f5e027fc700 1 -- 172.21.15.89:0/278424189 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc011660 con 0x7f5e04159c60 2023-10-10T20:34:57.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.024+0000 7f5e027fc700 1 -- 172.21.15.89:0/278424189 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dfc0117c0 con 0x7f5e04159c60 2023-10-10T20:34:57.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.024+0000 7f5e0c74a700 1 -- 172.21.15.89:0/278424189 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e040cfc40 con 0x7f5e04159c60 2023-10-10T20:34:57.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.024+0000 7f5e027fc700 1 -- 172.21.15.89:0/278424189 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc011b60 con 0x7f5e04159c60 2023-10-10T20:34:57.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.024+0000 7f5e0c74a700 1 -- 172.21.15.89:0/278424189 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e040b39a0 con 0x7f5e04159c60 2023-10-10T20:34:57.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.025+0000 7f5e0c74a700 1 -- 172.21.15.89:0/278424189 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e0404c3c0 con 0x7f5e04159c60 2023-10-10T20:34:57.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.025+0000 7f5e027fc700 1 -- 172.21.15.89:0/278424189 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5dfc0193f0 con 0x7f5e04159c60 2023-10-10T20:34:57.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.026+0000 7f5e027fc700 1 --2- 172.21.15.89:0/278424189 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5df0066f30 0x7f5df00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.026+0000 7f5e027fc700 1 -- 172.21.15.89:0/278424189 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5dfc0847c0 con 0x7f5e04159c60 2023-10-10T20:34:57.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.026+0000 7f5e0a4e6700 1 --2- 172.21.15.89:0/278424189 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5df0066f30 0x7f5df00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:57.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.026+0000 7f5e0a4e6700 1 --2- 172.21.15.89:0/278424189 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5df0066f30 0x7f5df00693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5e0406a890 tx=0x7f5df4019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:57.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.030+0000 7f5e027fc700 1 -- 172.21.15.89:0/278424189 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5dfc0554c0 con 0x7f5e04159c60 2023-10-10T20:34:57.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.212+0000 7f5e0c74a700 1 -- 172.21.15.89:0/278424189 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e0406a5b0 con 0x7f5e04159c60 2023-10-10T20:34:57.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.212+0000 7f5e027fc700 1 -- 172.21.15.89:0/278424189 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5dfc055050 con 0x7f5e04159c60 2023-10-10T20:34:57.215 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:34:57.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.214+0000 7f5e0c74a700 1 -- 172.21.15.89:0/278424189 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5df0066f30 msgr2=0x7f5df00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.214+0000 7f5e0c74a700 1 --2- 172.21.15.89:0/278424189 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5df0066f30 0x7f5df00693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5e0406a890 tx=0x7f5df4019000 comp rx=0 tx=0).stop 2023-10-10T20:34:57.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.214+0000 7f5e0c74a700 1 -- 172.21.15.89:0/278424189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e04159c60 msgr2=0x7f5e040ca330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.214+0000 7f5e0c74a700 1 --2- 172.21.15.89:0/278424189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e04159c60 0x7f5e040ca330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00c880 tx=0x7f5dfc008040 comp rx=0 tx=0).stop 2023-10-10T20:34:57.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.214+0000 7f5e0c74a700 1 -- 172.21.15.89:0/278424189 shutdown_connections 2023-10-10T20:34:57.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.214+0000 7f5e0c74a700 1 --2- 172.21.15.89:0/278424189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e04159c60 0x7f5e040ca330 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.214+0000 7f5e0c74a700 1 --2- 172.21.15.89:0/278424189 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5df0066f30 0x7f5df00693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.214+0000 7f5e0c74a700 1 --2- 172.21.15.89:0/278424189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e04069560 0x7f5e0416a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.214+0000 7f5e0c74a700 1 --2- 172.21.15.89:0/278424189 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e04159890 0x7f5e040cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.214+0000 7f5e0c74a700 1 -- 172.21.15.89:0/278424189 >> 172.21.15.89:0/278424189 conn(0x7f5e04068e20 msgr2=0x7f5e040b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:57.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.214+0000 7f5e0c74a700 1 -- 172.21.15.89:0/278424189 shutdown_connections 2023-10-10T20:34:57.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.214+0000 7f5e0c74a700 1 -- 172.21.15.89:0/278424189 wait complete. 2023-10-10T20:34:57.228 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:34:57.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.343+0000 7fdae3a9a700 1 Processor -- start 2023-10-10T20:34:57.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.343+0000 7fdae3a9a700 1 -- start start 2023-10-10T20:34:57.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.344+0000 7fdae3a9a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdadc159890 0x7fdadc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.344+0000 7fdae3a9a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdadc159c60 0x7fdadc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.344+0000 7fdae3a9a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdadc15a160 con 0x7fdadc15db10 2023-10-10T20:34:57.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.344+0000 7fdae3a9a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdadc0d13a0 con 0x7fdadc159890 2023-10-10T20:34:57.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.344+0000 7fdae3a9a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc15e140 con 0x7fdadc159c60 2023-10-10T20:34:57.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.344+0000 7fdae1836700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdadc159890 0x7fdadc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:57.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.344+0000 7fdae2037700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdadc15db10 0x7fdadc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43926/0 (socket says 172.21.15.89:43926) 2023-10-10T20:34:57.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.344+0000 7fdae2037700 1 -- 172.21.15.89:0/2697438279 learned_addr learned my addr 172.21.15.89:0/2697438279 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:57.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.344+0000 7fdae1035700 1 --2- 172.21.15.89:0/2697438279 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdadc159c60 0x7fdadc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:57.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.344+0000 7fdae1836700 1 -- 172.21.15.89:0/2697438279 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdadc159c60 msgr2=0x7fdadc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:57.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.344+0000 7fdae1836700 1 --2- 172.21.15.89:0/2697438279 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdadc159c60 0x7fdadc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.344+0000 7fdae1836700 1 -- 172.21.15.89:0/2697438279 >> v1:172.21.15.89:6789/0 conn(0x7fdadc15db10 legacy=0x7fdadc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.344+0000 7fdae1836700 1 -- 172.21.15.89:0/2697438279 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc15a3d0 con 0x7fdadc159890 2023-10-10T20:34:57.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.344+0000 7fdae0834700 1 -- 172.21.15.89:0/2697438279 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694006656 0 0) 0x7fdadc15a160 con 0x7fdadc15db10 2023-10-10T20:34:57.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.344+0000 7fdae1836700 1 --2- 172.21.15.89:0/2697438279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdadc159890 0x7fdadc15abd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009e60 tx=0x7fdacc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8445aa1d6232569a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:57.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.344+0000 7fdae0834700 1 -- 172.21.15.89:0/2697438279 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc00e070 con 0x7fdadc159890 2023-10-10T20:34:57.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.344+0000 7fdae0834700 1 -- 172.21.15.89:0/2697438279 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdacc00bb60 con 0x7fdadc159890 2023-10-10T20:34:57.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.344+0000 7fdae0834700 1 -- 172.21.15.89:0/2697438279 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc012430 con 0x7fdadc159890 2023-10-10T20:34:57.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.345+0000 7fdae3a9a700 1 -- 172.21.15.89:0/2697438279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdadc159890 msgr2=0x7fdadc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.345+0000 7fdae3a9a700 1 --2- 172.21.15.89:0/2697438279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdadc159890 0x7fdadc15abd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009e60 tx=0x7fdacc009780 comp rx=0 tx=0).stop 2023-10-10T20:34:57.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.345+0000 7fdae3a9a700 1 -- 172.21.15.89:0/2697438279 shutdown_connections 2023-10-10T20:34:57.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.345+0000 7fdae3a9a700 1 --2- 172.21.15.89:0/2697438279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdadc159890 0x7fdadc15abd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.345+0000 7fdae3a9a700 1 --2- 172.21.15.89:0/2697438279 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdadc159c60 0x7fdadc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.345+0000 7fdae3a9a700 1 -- 172.21.15.89:0/2697438279 >> 172.21.15.89:0/2697438279 conn(0x7fdadc068e20 msgr2=0x7fdadc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:57.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.345+0000 7fdae3a9a700 1 -- 172.21.15.89:0/2697438279 shutdown_connections 2023-10-10T20:34:57.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.345+0000 7fdae3a9a700 1 -- 172.21.15.89:0/2697438279 wait complete. 2023-10-10T20:34:57.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.346+0000 7fdae3a9a700 1 Processor -- start 2023-10-10T20:34:57.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.346+0000 7fdae3a9a700 1 -- start start 2023-10-10T20:34:57.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdae3a9a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdadc159890 0x7fdadc0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdae3a9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdadc159c60 0x7fdadc0cf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdae3a9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdadc15db10 0x7fdadc16d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdae3a9a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc0b46c0 con 0x7fdadc15db10 2023-10-10T20:34:57.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdae3a9a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc0b4100 con 0x7fdadc159c60 2023-10-10T20:34:57.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdae3a9a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc0b43e0 con 0x7fdadc159890 2023-10-10T20:34:57.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdae2037700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdadc15db10 0x7fdadc16d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:57.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdae2037700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdadc15db10 0x7fdadc16d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44838/0 (socket says 172.21.15.89:44838) 2023-10-10T20:34:57.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdae2037700 1 -- 172.21.15.89:0/3469847547 learned_addr learned my addr 172.21.15.89:0/3469847547 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:57.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdae1836700 1 --2- 172.21.15.89:0/3469847547 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdadc159890 0x7fdadc0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:57.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdae1035700 1 --2- 172.21.15.89:0/3469847547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdadc159c60 0x7fdadc0cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:57.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdae2037700 1 -- 172.21.15.89:0/3469847547 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdadc159890 msgr2=0x7fdadc0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdae2037700 1 --2- 172.21.15.89:0/3469847547 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdadc159890 0x7fdadc0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdae2037700 1 -- 172.21.15.89:0/3469847547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdadc159c60 msgr2=0x7fdadc0cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdae2037700 1 --2- 172.21.15.89:0/3469847547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdadc159c60 0x7fdadc0cf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdae2037700 1 -- 172.21.15.89:0/3469847547 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdacc009a20 con 0x7fdadc15db10 2023-10-10T20:34:57.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdae2037700 1 --2- 172.21.15.89:0/3469847547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdadc15db10 0x7fdadc16d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdad400c530 tx=0x7fdad4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:57.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdad9ffb700 1 -- 172.21.15.89:0/3469847547 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad4005b30 con 0x7fdadc15db10 2023-10-10T20:34:57.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdad9ffb700 1 -- 172.21.15.89:0/3469847547 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad4005cb0 con 0x7fdadc15db10 2023-10-10T20:34:57.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdad9ffb700 1 -- 172.21.15.89:0/3469847547 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad4007320 con 0x7fdadc15db10 2023-10-10T20:34:57.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.347+0000 7fdae3a9a700 1 -- 172.21.15.89:0/3469847547 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdadc0c9a90 con 0x7fdadc15db10 2023-10-10T20:34:57.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.348+0000 7fdae3a9a700 1 -- 172.21.15.89:0/3469847547 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdadc0cf860 con 0x7fdadc15db10 2023-10-10T20:34:57.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.349+0000 7fdae3a9a700 1 -- 172.21.15.89:0/3469847547 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdadc04c3c0 con 0x7fdadc15db10 2023-10-10T20:34:57.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.349+0000 7fdad9ffb700 1 -- 172.21.15.89:0/3469847547 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdad401b3b0 con 0x7fdadc15db10 2023-10-10T20:34:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.349+0000 7fdad9ffb700 1 --2- 172.21.15.89:0/3469847547 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdac8066f30 0x7fdac80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.349+0000 7fdad9ffb700 1 -- 172.21.15.89:0/3469847547 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdad4083870 con 0x7fdadc15db10 2023-10-10T20:34:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.349+0000 7fdae1836700 1 --2- 172.21.15.89:0/3469847547 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdac8066f30 0x7fdac80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.350+0000 7fdae1836700 1 --2- 172.21.15.89:0/3469847547 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdac8066f30 0x7fdac80693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0097c0 tx=0x7fdacc019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.354+0000 7fdad9ffb700 1 -- 172.21.15.89:0/3469847547 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdad4050ae0 con 0x7fdadc15db10 2023-10-10T20:34:57.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.535+0000 7fdae3a9a700 1 -- 172.21.15.89:0/3469847547 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdadc06a5b0 con 0x7fdadc15db10 2023-10-10T20:34:57.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.535+0000 7fdad9ffb700 1 -- 172.21.15.89:0/3469847547 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdad40170a0 con 0x7fdadc15db10 2023-10-10T20:34:57.538 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:34:57.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.537+0000 7fdae3a9a700 1 -- 172.21.15.89:0/3469847547 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdac8066f30 msgr2=0x7fdac80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.537+0000 7fdae3a9a700 1 --2- 172.21.15.89:0/3469847547 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdac8066f30 0x7fdac80693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0097c0 tx=0x7fdacc019000 comp rx=0 tx=0).stop 2023-10-10T20:34:57.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.537+0000 7fdae3a9a700 1 -- 172.21.15.89:0/3469847547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdadc15db10 msgr2=0x7fdadc16d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.537+0000 7fdae3a9a700 1 --2- 172.21.15.89:0/3469847547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdadc15db10 0x7fdadc16d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdad400c530 tx=0x7fdad4007f90 comp rx=0 tx=0).stop 2023-10-10T20:34:57.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.537+0000 7fdae3a9a700 1 -- 172.21.15.89:0/3469847547 shutdown_connections 2023-10-10T20:34:57.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.537+0000 7fdae3a9a700 1 --2- 172.21.15.89:0/3469847547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdadc15db10 0x7fdadc16d190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.537+0000 7fdae3a9a700 1 --2- 172.21.15.89:0/3469847547 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdadc159890 0x7fdadc0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.537+0000 7fdae3a9a700 1 --2- 172.21.15.89:0/3469847547 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdac8066f30 0x7fdac80693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.537+0000 7fdae3a9a700 1 --2- 172.21.15.89:0/3469847547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdadc159c60 0x7fdadc0cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.537+0000 7fdae3a9a700 1 -- 172.21.15.89:0/3469847547 >> 172.21.15.89:0/3469847547 conn(0x7fdadc068e20 msgr2=0x7fdadc0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.538+0000 7fdae3a9a700 1 -- 172.21.15.89:0/3469847547 shutdown_connections 2023-10-10T20:34:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.538+0000 7fdae3a9a700 1 -- 172.21.15.89:0/3469847547 wait complete. 2023-10-10T20:34:57.554 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:34:57.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.668+0000 7f428e986700 1 Processor -- start 2023-10-10T20:34:57.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.668+0000 7f428e986700 1 -- start start 2023-10-10T20:34:57.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.669+0000 7f428e986700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42880aca80 0x7f428806aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.669+0000 7f428e986700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f42880c83c0 0x7f42880cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.669+0000 7f428e986700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f428806bdb0 con 0x7f4288069d60 2023-10-10T20:34:57.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.669+0000 7f428e986700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42880cb370 con 0x7f42880c83c0 2023-10-10T20:34:57.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.669+0000 7f428e986700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4288159f60 con 0x7f42880aca80 2023-10-10T20:34:57.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.669+0000 7f428cf23700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f42880c83c0 0x7f42880cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:57.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.669+0000 7f428cf23700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f42880c83c0 0x7f42880cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44844/0 (socket says 172.21.15.89:44844) 2023-10-10T20:34:57.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.669+0000 7f428cf23700 1 -- 172.21.15.89:0/171449167 learned_addr learned my addr 172.21.15.89:0/171449167 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:57.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.670+0000 7f4287fff700 1 --2- 172.21.15.89:0/171449167 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42880aca80 0x7f428806aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:57.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.670+0000 7f428cf23700 1 -- 172.21.15.89:0/171449167 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42880aca80 msgr2=0x7f428806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:57.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.670+0000 7f428cf23700 1 --2- 172.21.15.89:0/171449167 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42880aca80 0x7f428806aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.670+0000 7f428cf23700 1 -- 172.21.15.89:0/171449167 >> v1:172.21.15.111:6789/0 conn(0x7f4288069d60 legacy=0x7f42880c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.670+0000 7f428cf23700 1 -- 172.21.15.89:0/171449167 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428806b510 con 0x7f42880c83c0 2023-10-10T20:34:57.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.670+0000 7f4287fff700 1 --2- 172.21.15.89:0/171449167 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42880aca80 0x7f428806aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:57.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.670+0000 7f428cf23700 1 --2- 172.21.15.89:0/171449167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42880c83c0 0x7f42880cb800 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4280009f20 tx=0x7f4280009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2c4ac35b1262d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:57.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.670+0000 7f4286ffd700 1 -- 172.21.15.89:0/171449167 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428000e070 con 0x7f42880c83c0 2023-10-10T20:34:57.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.670+0000 7f4286ffd700 1 -- 172.21.15.89:0/171449167 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428000ba80 con 0x7f42880c83c0 2023-10-10T20:34:57.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.670+0000 7f4286ffd700 1 -- 172.21.15.89:0/171449167 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42800092e0 con 0x7f42880c83c0 2023-10-10T20:34:57.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.670+0000 7f428e986700 1 -- 172.21.15.89:0/171449167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42880c83c0 msgr2=0x7f42880cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.670+0000 7f428e986700 1 --2- 172.21.15.89:0/171449167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42880c83c0 0x7f42880cb800 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4280009f20 tx=0x7f4280009840 comp rx=0 tx=0).stop 2023-10-10T20:34:57.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.670+0000 7f428e986700 1 -- 172.21.15.89:0/171449167 shutdown_connections 2023-10-10T20:34:57.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.671+0000 7f428e986700 1 --2- 172.21.15.89:0/171449167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42880c83c0 0x7f42880cb800 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.671+0000 7f428e986700 1 --2- 172.21.15.89:0/171449167 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42880aca80 0x7f428806aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.671+0000 7f428e986700 1 -- 172.21.15.89:0/171449167 >> 172.21.15.89:0/171449167 conn(0x7f42880cd670 msgr2=0x7f42880cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:57.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.671+0000 7f428e986700 1 -- 172.21.15.89:0/171449167 shutdown_connections 2023-10-10T20:34:57.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.671+0000 7f428e986700 1 -- 172.21.15.89:0/171449167 wait complete. 2023-10-10T20:34:57.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.672+0000 7f428e986700 1 Processor -- start 2023-10-10T20:34:57.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.672+0000 7f428e986700 1 -- start start 2023-10-10T20:34:57.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.672+0000 7f428e986700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4288069d60 0x7f42881589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.672+0000 7f428e986700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42880aca80 0x7f428816ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f428e986700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42880c83c0 0x7f4288068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f428e986700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428816b390 con 0x7f42880aca80 2023-10-10T20:34:57.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f428e986700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42880b2ca0 con 0x7f4288069d60 2023-10-10T20:34:57.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f428e986700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f428816b040 con 0x7f42880c83c0 2023-10-10T20:34:57.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f42877fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42880aca80 0x7f428816ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:57.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f42877fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42880aca80 0x7f428816ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44846/0 (socket says 172.21.15.89:44846) 2023-10-10T20:34:57.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f42877fe700 1 -- 172.21.15.89:0/533082473 learned_addr learned my addr 172.21.15.89:0/533082473 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:57.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f4287fff700 1 --2- 172.21.15.89:0/533082473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4288069d60 0x7f42881589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:57.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f428cf23700 1 --2- 172.21.15.89:0/533082473 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42880c83c0 0x7f4288068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:57.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f42877fe700 1 -- 172.21.15.89:0/533082473 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42880c83c0 msgr2=0x7f4288068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f42877fe700 1 --2- 172.21.15.89:0/533082473 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42880c83c0 0x7f4288068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f42877fe700 1 -- 172.21.15.89:0/533082473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4288069d60 msgr2=0x7f42881589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f42877fe700 1 --2- 172.21.15.89:0/533082473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4288069d60 0x7f42881589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f42877fe700 1 -- 172.21.15.89:0/533082473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4280009ae0 con 0x7f42880aca80 2023-10-10T20:34:57.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f42877fe700 1 --2- 172.21.15.89:0/533082473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42880aca80 0x7f428816ab00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f427c00c7f0 tx=0x7f427c007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:57.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f426ffff700 1 -- 172.21.15.89:0/533082473 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c011590 con 0x7f42880aca80 2023-10-10T20:34:57.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f426ffff700 1 -- 172.21.15.89:0/533082473 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427c011710 con 0x7f42880aca80 2023-10-10T20:34:57.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f428e986700 1 -- 172.21.15.89:0/533082473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f428809ae80 con 0x7f42880aca80 2023-10-10T20:34:57.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f426ffff700 1 -- 172.21.15.89:0/533082473 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c011ad0 con 0x7f42880aca80 2023-10-10T20:34:57.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.673+0000 7f428e986700 1 -- 172.21.15.89:0/533082473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42880b2f10 con 0x7f42880aca80 2023-10-10T20:34:57.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.674+0000 7f428e986700 1 -- 172.21.15.89:0/533082473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428816b540 con 0x7f42880aca80 2023-10-10T20:34:57.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.675+0000 7f426ffff700 1 -- 172.21.15.89:0/533082473 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f427c012420 con 0x7f42880aca80 2023-10-10T20:34:57.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.675+0000 7f426ffff700 1 --2- 172.21.15.89:0/533082473 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4274066f30 0x7f42740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.675+0000 7f426ffff700 1 -- 172.21.15.89:0/533082473 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f427c0837e0 con 0x7f42880aca80 2023-10-10T20:34:57.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.675+0000 7f4287fff700 1 --2- 172.21.15.89:0/533082473 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4274066f30 0x7f42740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:57.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.675+0000 7f4287fff700 1 --2- 172.21.15.89:0/533082473 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4274066f30 0x7f42740693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4278006ea0 tx=0x7f4278008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:57.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.679+0000 7f426ffff700 1 -- 172.21.15.89:0/533082473 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f427c04b5a0 con 0x7f42880aca80 2023-10-10T20:34:57.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.861+0000 7f428e986700 1 -- 172.21.15.89:0/533082473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f42880ac340 con 0x7f42880aca80 2023-10-10T20:34:57.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.861+0000 7f426ffff700 1 -- 172.21.15.89:0/533082473 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f427c017070 con 0x7f42880aca80 2023-10-10T20:34:57.864 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-10T20:34:57.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.863+0000 7f428e986700 1 -- 172.21.15.89:0/533082473 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4274066f30 msgr2=0x7f42740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.863+0000 7f428e986700 1 --2- 172.21.15.89:0/533082473 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4274066f30 0x7f42740693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4278006ea0 tx=0x7f4278008040 comp rx=0 tx=0).stop 2023-10-10T20:34:57.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.863+0000 7f428e986700 1 -- 172.21.15.89:0/533082473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42880aca80 msgr2=0x7f428816ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.863+0000 7f428e986700 1 --2- 172.21.15.89:0/533082473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42880aca80 0x7f428816ab00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f427c00c7f0 tx=0x7f427c007f50 comp rx=0 tx=0).stop 2023-10-10T20:34:57.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.863+0000 7f428e986700 1 -- 172.21.15.89:0/533082473 shutdown_connections 2023-10-10T20:34:57.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.863+0000 7f428e986700 1 --2- 172.21.15.89:0/533082473 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42880c83c0 0x7f4288068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.863+0000 7f428e986700 1 --2- 172.21.15.89:0/533082473 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4274066f30 0x7f42740693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.863+0000 7f428e986700 1 --2- 172.21.15.89:0/533082473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4288069d60 0x7f42881589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.863+0000 7f428e986700 1 --2- 172.21.15.89:0/533082473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42880aca80 0x7f428816ab00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.863+0000 7f428e986700 1 -- 172.21.15.89:0/533082473 >> 172.21.15.89:0/533082473 conn(0x7f42880cd670 msgr2=0x7f4288159100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:57.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.863+0000 7f428e986700 1 -- 172.21.15.89:0/533082473 shutdown_connections 2023-10-10T20:34:57.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.863+0000 7f428e986700 1 -- 172.21.15.89:0/533082473 wait complete. 2023-10-10T20:34:57.877 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:34:57.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.989+0000 7f9e01a9d700 1 Processor -- start 2023-10-10T20:34:57.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.989+0000 7f9e01a9d700 1 -- start start 2023-10-10T20:34:57.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9e01a9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9dfc159890 0x7f9dfc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9e01a9d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dfc15a160 con 0x7f9dfc159c60 2023-10-10T20:34:57.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9e01a9d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dfc0d13a0 con 0x7f9dfc069560 2023-10-10T20:34:57.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9e01a9d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dfc15e140 con 0x7f9dfc159890 2023-10-10T20:34:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9dfaffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9dfc159c60 0x7f9dfc15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43938/0 (socket says 172.21.15.89:43938) 2023-10-10T20:34:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9dfaffd700 1 -- 172.21.15.89:0/477804631 learned_addr learned my addr 172.21.15.89:0/477804631 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9dfb7fe700 1 --2- 172.21.15.89:0/477804631 >> v2:172.21.15.111:3300/0 conn(0x7f9dfc159890 0x7f9dfc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9dfa7fc700 1 -- 172.21.15.89:0/477804631 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2042046973 0 0) 0x7f9dfc15a160 con 0x7f9dfc159c60 2023-10-10T20:34:57.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9dfa7fc700 1 -- 172.21.15.89:0/477804631 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9de4003570 con 0x7f9dfc159c60 2023-10-10T20:34:57.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9dfa7fc700 1 -- 172.21.15.89:0/477804631 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3090563476 0 0) 0x7f9de4003570 con 0x7f9dfc159c60 2023-10-10T20:34:57.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9dfa7fc700 1 -- 172.21.15.89:0/477804631 >> v2:172.21.15.111:3300/0 conn(0x7f9dfc159890 msgr2=0x7f9dfc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:57.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9dfa7fc700 1 --2- 172.21.15.89:0/477804631 >> v2:172.21.15.111:3300/0 conn(0x7f9dfc159890 0x7f9dfc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:57.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9dfa7fc700 1 -- 172.21.15.89:0/477804631 >> v1:172.21.15.111:6789/0 conn(0x7f9dfc069560 legacy=0x7f9dfc0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9dfa7fc700 1 -- 172.21.15.89:0/477804631 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc15a3d0 con 0x7f9dfc159c60 2023-10-10T20:34:57.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9dfa7fc700 1 -- 172.21.15.89:0/477804631 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f9dec001470 con 0x7f9dfc159c60 2023-10-10T20:34:57.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9dfa7fc700 1 -- 172.21.15.89:0/477804631 >> v1:172.21.15.89:6789/0 conn(0x7f9dfc159c60 legacy=0x7f9dfc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:57.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9dfa7fc700 1 --2- 172.21.15.89:0/477804631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9de4003a60 0x7f9de4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9dfa7fc700 1 --2- 172.21.15.89:0/477804631 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9de4004810 0x7f9de4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9dfa7fc700 1 --2- 172.21.15.89:0/477804631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9de40051c0 0x7f9de400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:57.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9dfa7fc700 1 -- 172.21.15.89:0/477804631 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc15e140 con 0x7f9de40051c0 2023-10-10T20:34:57.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9dfa7fc700 1 -- 172.21.15.89:0/477804631 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de400c0d0 con 0x7f9de4003a60 2023-10-10T20:34:57.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.990+0000 7f9dfa7fc700 1 -- 172.21.15.89:0/477804631 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de400c290 con 0x7f9de4004810 2023-10-10T20:34:57.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.991+0000 7f9dfbfff700 1 --2- 172.21.15.89:0/477804631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9de40051c0 0x7f9de400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.991+0000 7f9dfb7fe700 1 --2- 172.21.15.89:0/477804631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9de4003a60 0x7f9de4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.991+0000 7f9dfaffd700 1 --2- 172.21.15.89:0/477804631 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9de4004810 0x7f9de4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.991+0000 7f9dfbfff700 1 -- 172.21.15.89:0/477804631 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9de4004810 msgr2=0x7f9de4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:58.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.991+0000 7f9dfbfff700 1 --2- 172.21.15.89:0/477804631 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9de4004810 0x7f9de4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.991+0000 7f9dfbfff700 1 -- 172.21.15.89:0/477804631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9de4003a60 msgr2=0x7f9de4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:58.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.991+0000 7f9dfbfff700 1 --2- 172.21.15.89:0/477804631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9de4003a60 0x7f9de4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.991+0000 7f9dfbfff700 1 -- 172.21.15.89:0/477804631 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de4003570 con 0x7f9de40051c0 2023-10-10T20:34:58.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.991+0000 7f9dfbfff700 1 -- 172.21.15.89:0/477804631 reap_dead start 2023-10-10T20:34:58.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.991+0000 7f9dfbfff700 1 --2- 172.21.15.89:0/477804631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9de40051c0 0x7f9de400ba60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9df4001fc0 tx=0x7f9df4001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edcfe7fe21e687e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:58.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.991+0000 7f9dfa7fc700 1 -- 172.21.15.89:0/477804631 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df4001950 con 0x7f9de40051c0 2023-10-10T20:34:58.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.991+0000 7f9dfa7fc700 1 -- 172.21.15.89:0/477804631 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df400c8b0 con 0x7f9de40051c0 2023-10-10T20:34:58.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.991+0000 7f9dfa7fc700 1 -- 172.21.15.89:0/477804631 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df400cc90 con 0x7f9de40051c0 2023-10-10T20:34:58.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.992+0000 7f9e01a9d700 1 -- 172.21.15.89:0/477804631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9de40051c0 msgr2=0x7f9de400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.992+0000 7f9e01a9d700 1 --2- 172.21.15.89:0/477804631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9de40051c0 0x7f9de400ba60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9df4001fc0 tx=0x7f9df4001640 comp rx=0 tx=0).stop 2023-10-10T20:34:58.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.992+0000 7f9e01a9d700 1 -- 172.21.15.89:0/477804631 shutdown_connections 2023-10-10T20:34:58.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.992+0000 7f9e01a9d700 1 --2- 172.21.15.89:0/477804631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9de40051c0 0x7f9de400ba60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.992+0000 7f9e01a9d700 1 -- 172.21.15.89:0/477804631 >> 172.21.15.89:0/477804631 conn(0x7f9dfc068e20 msgr2=0x7f9dfc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:58.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.992+0000 7f9e01a9d700 1 -- 172.21.15.89:0/477804631 shutdown_connections 2023-10-10T20:34:58.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.992+0000 7f9e01a9d700 1 -- 172.21.15.89:0/477804631 wait complete. 2023-10-10T20:34:58.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.993+0000 7f9e01a9d700 1 Processor -- start 2023-10-10T20:34:58.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.993+0000 7f9e01a9d700 1 -- start start 2023-10-10T20:34:58.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9e01a9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9de40051c0 0x7f9dfc158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9e01a9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dfc0cd0c0 0x7f9dfc0ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9e01a9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dfc0cd530 0x7f9dfc0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9e01a9d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0cf820 con 0x7f9dfc0cd530 2023-10-10T20:34:58.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9e01a9d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0c80d0 con 0x7f9dfc0cd0c0 2023-10-10T20:34:58.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9e01a9d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0c83b0 con 0x7f9de40051c0 2023-10-10T20:34:58.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9dfbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dfc0cd530 0x7f9dfc0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9dfbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dfc0cd530 0x7f9dfc0cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44864/0 (socket says 172.21.15.89:44864) 2023-10-10T20:34:58.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9dfbfff700 1 -- 172.21.15.89:0/3381340677 learned_addr learned my addr 172.21.15.89:0/3381340677 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:58.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9dfaffd700 1 --2- 172.21.15.89:0/3381340677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dfc0cd0c0 0x7f9dfc0ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9dfb7fe700 1 --2- 172.21.15.89:0/3381340677 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9de40051c0 0x7f9dfc158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9dfbfff700 1 -- 172.21.15.89:0/3381340677 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9de40051c0 msgr2=0x7f9dfc158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9dfbfff700 1 --2- 172.21.15.89:0/3381340677 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9de40051c0 0x7f9dfc158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9dfbfff700 1 -- 172.21.15.89:0/3381340677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dfc0cd0c0 msgr2=0x7f9dfc0ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9dfbfff700 1 --2- 172.21.15.89:0/3381340677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dfc0cd0c0 0x7f9dfc0ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9dfbfff700 1 -- 172.21.15.89:0/3381340677 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc0c8690 con 0x7f9dfc0cd530 2023-10-10T20:34:58.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9dfbfff700 1 --2- 172.21.15.89:0/3381340677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dfc0cd530 0x7f9dfc0cf2e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc06a300 tx=0x7f9df4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:58.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9de37fe700 1 -- 172.21.15.89:0/3381340677 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df400c6d0 con 0x7f9dfc0cd530 2023-10-10T20:34:58.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9de37fe700 1 -- 172.21.15.89:0/3381340677 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df4012e50 con 0x7f9dfc0cd530 2023-10-10T20:34:58.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9de37fe700 1 -- 172.21.15.89:0/3381340677 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df401e660 con 0x7f9dfc0cd530 2023-10-10T20:34:58.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9e01a9d700 1 -- 172.21.15.89:0/3381340677 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dfc0c8970 con 0x7f9dfc0cd530 2023-10-10T20:34:58.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.994+0000 7f9e01a9d700 1 -- 172.21.15.89:0/3381340677 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc151fe0 con 0x7f9dfc0cd530 2023-10-10T20:34:58.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.995+0000 7f9e01a9d700 1 -- 172.21.15.89:0/3381340677 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dfc04c3c0 con 0x7f9dfc0cd530 2023-10-10T20:34:58.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.996+0000 7f9de37fe700 1 -- 172.21.15.89:0/3381340677 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9df401ea50 con 0x7f9dfc0cd530 2023-10-10T20:34:58.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.996+0000 7f9de37fe700 1 --2- 172.21.15.89:0/3381340677 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9de8066f30 0x7f9de80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.996+0000 7f9dfb7fe700 1 --2- 172.21.15.89:0/3381340677 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9de8066f30 0x7f9de80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.996+0000 7f9de37fe700 1 -- 172.21.15.89:0/3381340677 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9df4085fd0 con 0x7f9dfc0cd530 2023-10-10T20:34:58.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:57.997+0000 7f9dfb7fe700 1 --2- 172.21.15.89:0/3381340677 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9de8066f30 0x7f9de80693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9df0006f00 tx=0x7f9df00091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:58.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.000+0000 7f9de37fe700 1 -- 172.21.15.89:0/3381340677 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9df4053240 con 0x7f9dfc0cd530 2023-10-10T20:34:58.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.184+0000 7f9e01a9d700 1 -- 172.21.15.89:0/3381340677 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9dfc0c7e10 con 0x7f9dfc0cd530 2023-10-10T20:34:58.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.185+0000 7f9de37fe700 1 -- 172.21.15.89:0/3381340677 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f9df4017090 con 0x7f9dfc0cd530 2023-10-10T20:34:58.188 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:58.188 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:36.324920+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:34:32.342167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:34:33.671763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:34:58.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.188+0000 7f9e01a9d700 1 -- 172.21.15.89:0/3381340677 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9de8066f30 msgr2=0x7f9de80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.188+0000 7f9e01a9d700 1 --2- 172.21.15.89:0/3381340677 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9de8066f30 0x7f9de80693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9df0006f00 tx=0x7f9df00091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.188+0000 7f9e01a9d700 1 -- 172.21.15.89:0/3381340677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dfc0cd530 msgr2=0x7f9dfc0cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.188+0000 7f9e01a9d700 1 --2- 172.21.15.89:0/3381340677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dfc0cd530 0x7f9dfc0cf2e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc06a300 tx=0x7f9df4015000 comp rx=0 tx=0).stop 2023-10-10T20:34:58.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.188+0000 7f9e01a9d700 1 -- 172.21.15.89:0/3381340677 shutdown_connections 2023-10-10T20:34:58.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.188+0000 7f9e01a9d700 1 --2- 172.21.15.89:0/3381340677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dfc0cd530 0x7f9dfc0cf2e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.188+0000 7f9e01a9d700 1 --2- 172.21.15.89:0/3381340677 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9de40051c0 0x7f9dfc158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.188+0000 7f9e01a9d700 1 --2- 172.21.15.89:0/3381340677 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9de8066f30 0x7f9de80693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.188+0000 7f9e01a9d700 1 --2- 172.21.15.89:0/3381340677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dfc0cd0c0 0x7f9dfc0ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.188+0000 7f9e01a9d700 1 -- 172.21.15.89:0/3381340677 >> 172.21.15.89:0/3381340677 conn(0x7f9dfc068e20 msgr2=0x7f9dfc15b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:58.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.188+0000 7f9e01a9d700 1 -- 172.21.15.89:0/3381340677 shutdown_connections 2023-10-10T20:34:58.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.188+0000 7f9e01a9d700 1 -- 172.21.15.89:0/3381340677 wait complete. 2023-10-10T20:34:58.202 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:34:32.342167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:34:33.671763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:34:58.203 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:34:58.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.318+0000 7f725ceec700 1 Processor -- start 2023-10-10T20:34:58.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.318+0000 7f725ceec700 1 -- start start 2023-10-10T20:34:58.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.319+0000 7f725ceec700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7258159890 0x7f725815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.319+0000 7f725ceec700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7258069560 0x7f72580699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.319+0000 7f725ceec700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7258069ee0 con 0x7f7258159c60 2023-10-10T20:34:58.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.319+0000 7f725ceec700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725815a160 con 0x7f7258159890 2023-10-10T20:34:58.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.319+0000 7f725ceec700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72580d13a0 con 0x7f7258069560 2023-10-10T20:34:58.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.319+0000 7f725659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7258159890 0x7f725815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.319+0000 7f725659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7258159890 0x7f725815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59146/0 (socket says 172.21.15.89:59146) 2023-10-10T20:34:58.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.319+0000 7f725659c700 1 -- 172.21.15.89:0/3465287500 learned_addr learned my addr 172.21.15.89:0/3465287500 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:58.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.319+0000 7f7256d9d700 1 --2- 172.21.15.89:0/3465287500 >> v2:172.21.15.111:3300/0 conn(0x7f7258069560 0x7f72580699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.319+0000 7f725659c700 1 -- 172.21.15.89:0/3465287500 >> v2:172.21.15.111:3300/0 conn(0x7f7258069560 msgr2=0x7f72580699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:58.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.319+0000 7f725659c700 1 --2- 172.21.15.89:0/3465287500 >> v2:172.21.15.111:3300/0 conn(0x7f7258069560 0x7f72580699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.319+0000 7f725659c700 1 -- 172.21.15.89:0/3465287500 >> v1:172.21.15.111:6789/0 conn(0x7f7258159c60 legacy=0x7f725815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.319+0000 7f725659c700 1 -- 172.21.15.89:0/3465287500 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725815e140 con 0x7f7258159890 2023-10-10T20:34:58.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.319+0000 7f725659c700 1 --2- 172.21.15.89:0/3465287500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258159890 0x7f725815abd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f725806a890 tx=0x7f7248009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60714eacdc04ae88 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:58.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.319+0000 7f7255d9b700 1 -- 172.21.15.89:0/3465287500 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7248009e70 con 0x7f7258159890 2023-10-10T20:34:58.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.319+0000 7f7255d9b700 1 -- 172.21.15.89:0/3465287500 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724800b860 con 0x7f7258159890 2023-10-10T20:34:58.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.319+0000 7f7255d9b700 1 -- 172.21.15.89:0/3465287500 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724800bc00 con 0x7f7258159890 2023-10-10T20:34:58.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.320+0000 7f725ceec700 1 -- 172.21.15.89:0/3465287500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258159890 msgr2=0x7f725815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.320+0000 7f725ceec700 1 --2- 172.21.15.89:0/3465287500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258159890 0x7f725815abd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f725806a890 tx=0x7f7248009250 comp rx=0 tx=0).stop 2023-10-10T20:34:58.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.320+0000 7f725ceec700 1 -- 172.21.15.89:0/3465287500 shutdown_connections 2023-10-10T20:34:58.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.320+0000 7f725ceec700 1 --2- 172.21.15.89:0/3465287500 >> v2:172.21.15.111:3300/0 conn(0x7f7258069560 0x7f72580699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.320+0000 7f725ceec700 1 --2- 172.21.15.89:0/3465287500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258159890 0x7f725815abd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.320+0000 7f725ceec700 1 -- 172.21.15.89:0/3465287500 >> 172.21.15.89:0/3465287500 conn(0x7f7258068e20 msgr2=0x7f72580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:58.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.320+0000 7f725ceec700 1 -- 172.21.15.89:0/3465287500 shutdown_connections 2023-10-10T20:34:58.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.320+0000 7f725ceec700 1 -- 172.21.15.89:0/3465287500 wait complete. 2023-10-10T20:34:58.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.321+0000 7f725ceec700 1 Processor -- start 2023-10-10T20:34:58.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.321+0000 7f725ceec700 1 -- start start 2023-10-10T20:34:58.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f725ceec700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7258069560 0x7f72580b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f725ceec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258159890 0x7f72580c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f725ceec700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7258159c60 0x7f7258152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f725ceec700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72580b36c0 con 0x7f7258159890 2023-10-10T20:34:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f725ceec700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72580c87a0 con 0x7f7258069560 2023-10-10T20:34:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f725ceec700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72580c8a70 con 0x7f7258159c60 2023-10-10T20:34:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f724ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258159890 0x7f72580c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f724ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258159890 0x7f72580c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59156/0 (socket says 172.21.15.89:59156) 2023-10-10T20:34:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f724ffff700 1 -- 172.21.15.89:0/2839354845 learned_addr learned my addr 172.21.15.89:0/2839354845 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f725659c700 1 --2- 172.21.15.89:0/2839354845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7258069560 0x7f72580b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f7256d9d700 1 --2- 172.21.15.89:0/2839354845 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7258159c60 0x7f7258152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f724ffff700 1 -- 172.21.15.89:0/2839354845 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7258159c60 msgr2=0x7f7258152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f724ffff700 1 --2- 172.21.15.89:0/2839354845 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7258159c60 0x7f7258152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f724ffff700 1 -- 172.21.15.89:0/2839354845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7258069560 msgr2=0x7f72580b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f724ffff700 1 --2- 172.21.15.89:0/2839354845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7258069560 0x7f72580b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f724ffff700 1 -- 172.21.15.89:0/2839354845 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72480094f0 con 0x7f7258159890 2023-10-10T20:34:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f724ffff700 1 --2- 172.21.15.89:0/2839354845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258159890 0x7f72580c8260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f724000c4e0 tx=0x7f724000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f724e7fc700 1 -- 172.21.15.89:0/2839354845 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7240011590 con 0x7f7258159890 2023-10-10T20:34:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f724e7fc700 1 -- 172.21.15.89:0/2839354845 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7240011710 con 0x7f7258159890 2023-10-10T20:34:58.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f724e7fc700 1 -- 172.21.15.89:0/2839354845 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7240011ad0 con 0x7f7258159890 2023-10-10T20:34:58.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f725ceec700 1 -- 172.21.15.89:0/2839354845 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72580c8d50 con 0x7f7258159890 2023-10-10T20:34:58.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.322+0000 7f725ceec700 1 -- 172.21.15.89:0/2839354845 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72580c9310 con 0x7f7258159890 2023-10-10T20:34:58.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.323+0000 7f725ceec700 1 -- 172.21.15.89:0/2839354845 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72580b3870 con 0x7f7258159890 2023-10-10T20:34:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.324+0000 7f724e7fc700 1 -- 172.21.15.89:0/2839354845 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f724001b3b0 con 0x7f7258159890 2023-10-10T20:34:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.324+0000 7f724e7fc700 1 --2- 172.21.15.89:0/2839354845 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7238066f30 0x7f72380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.324+0000 7f724e7fc700 1 -- 172.21.15.89:0/2839354845 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f72400838c0 con 0x7f7258159890 2023-10-10T20:34:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.324+0000 7f725659c700 1 --2- 172.21.15.89:0/2839354845 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7238066f30 0x7f72380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.325+0000 7f725659c700 1 --2- 172.21.15.89:0/2839354845 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7238066f30 0x7f72380693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f725806a890 tx=0x7f7248017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:58.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.328+0000 7f724e7fc700 1 -- 172.21.15.89:0/2839354845 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72580b3870 con 0x7f7258159890 2023-10-10T20:34:58.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.511+0000 7f725ceec700 1 -- 172.21.15.89:0/2839354845 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f725804c3c0 con 0x7f7258159890 2023-10-10T20:34:58.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.512+0000 7f724e7fc700 1 -- 172.21.15.89:0/2839354845 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7240017070 con 0x7f7258159890 2023-10-10T20:34:58.514 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:34:58.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.514+0000 7f725ceec700 1 -- 172.21.15.89:0/2839354845 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7238066f30 msgr2=0x7f72380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.514+0000 7f725ceec700 1 --2- 172.21.15.89:0/2839354845 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7238066f30 0x7f72380693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f725806a890 tx=0x7f7248017000 comp rx=0 tx=0).stop 2023-10-10T20:34:58.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.514+0000 7f725ceec700 1 -- 172.21.15.89:0/2839354845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258159890 msgr2=0x7f72580c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.514+0000 7f725ceec700 1 --2- 172.21.15.89:0/2839354845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258159890 0x7f72580c8260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f724000c4e0 tx=0x7f724000c7f0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.514+0000 7f725ceec700 1 -- 172.21.15.89:0/2839354845 shutdown_connections 2023-10-10T20:34:58.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.514+0000 7f725ceec700 1 --2- 172.21.15.89:0/2839354845 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7258159c60 0x7f7258152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.514+0000 7f725ceec700 1 --2- 172.21.15.89:0/2839354845 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7238066f30 0x7f72380693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.514+0000 7f725ceec700 1 --2- 172.21.15.89:0/2839354845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7258069560 0x7f72580b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.514+0000 7f725ceec700 1 --2- 172.21.15.89:0/2839354845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258159890 0x7f72580c8260 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.514+0000 7f725ceec700 1 -- 172.21.15.89:0/2839354845 >> 172.21.15.89:0/2839354845 conn(0x7f7258068e20 msgr2=0x7f7258153390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:58.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.514+0000 7f725ceec700 1 -- 172.21.15.89:0/2839354845 shutdown_connections 2023-10-10T20:34:58.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.514+0000 7f725ceec700 1 -- 172.21.15.89:0/2839354845 wait complete. 2023-10-10T20:34:58.528 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:34:58.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.643+0000 7f12d1970700 1 Processor -- start 2023-10-10T20:34:58.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.643+0000 7f12d1970700 1 -- start start 2023-10-10T20:34:58.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.643+0000 7f12d1970700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f12cc159890 0x7f12cc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.644+0000 7f12d1970700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f12cc159c60 0x7f12cc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.644+0000 7f12d1970700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12cc15a160 con 0x7f12cc15db10 2023-10-10T20:34:58.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.644+0000 7f12d1970700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12cc0d13a0 con 0x7f12cc159c60 2023-10-10T20:34:58.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.644+0000 7f12d1970700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc15e140 con 0x7f12cc159890 2023-10-10T20:34:58.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.644+0000 7f12ca7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f12cc159c60 0x7f12cc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.644+0000 7f12ca7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f12cc159c60 0x7f12cc15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59172/0 (socket says 172.21.15.89:59172) 2023-10-10T20:34:58.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.644+0000 7f12ca7fc700 1 -- 172.21.15.89:0/3335345348 learned_addr learned my addr 172.21.15.89:0/3335345348 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:58.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.644+0000 7f12caffd700 1 --2- 172.21.15.89:0/3335345348 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f12cc159890 0x7f12cc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.644+0000 7f12ca7fc700 1 -- 172.21.15.89:0/3335345348 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f12cc159890 msgr2=0x7f12cc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:58.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.644+0000 7f12ca7fc700 1 --2- 172.21.15.89:0/3335345348 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f12cc159890 0x7f12cc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.644+0000 7f12ca7fc700 1 -- 172.21.15.89:0/3335345348 >> v1:172.21.15.111:6789/0 conn(0x7f12cc15db10 legacy=0x7f12cc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.644+0000 7f12ca7fc700 1 -- 172.21.15.89:0/3335345348 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc15a3d0 con 0x7f12cc159c60 2023-10-10T20:34:58.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.644+0000 7f12c9ffb700 1 -- 172.21.15.89:0/3335345348 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372623495 0 0) 0x7f12cc15a160 con 0x7f12cc15db10 2023-10-10T20:34:58.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.644+0000 7f12ca7fc700 1 --2- 172.21.15.89:0/3335345348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc159c60 0x7f12cc15d5d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009f20 tx=0x7f12c0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b8a3c18d2f08e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:58.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.644+0000 7f12c9ffb700 1 -- 172.21.15.89:0/3335345348 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c000e070 con 0x7f12cc159c60 2023-10-10T20:34:58.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.644+0000 7f12c9ffb700 1 -- 172.21.15.89:0/3335345348 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c000b9d0 con 0x7f12cc159c60 2023-10-10T20:34:58.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.644+0000 7f12c9ffb700 1 -- 172.21.15.89:0/3335345348 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c00092e0 con 0x7f12cc159c60 2023-10-10T20:34:58.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.645+0000 7f12d1970700 1 -- 172.21.15.89:0/3335345348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc159c60 msgr2=0x7f12cc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.645+0000 7f12d1970700 1 --2- 172.21.15.89:0/3335345348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc159c60 0x7f12cc15d5d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009f20 tx=0x7f12c0009840 comp rx=0 tx=0).stop 2023-10-10T20:34:58.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.645+0000 7f12d1970700 1 -- 172.21.15.89:0/3335345348 shutdown_connections 2023-10-10T20:34:58.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.645+0000 7f12d1970700 1 --2- 172.21.15.89:0/3335345348 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f12cc159890 0x7f12cc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.645+0000 7f12d1970700 1 --2- 172.21.15.89:0/3335345348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc159c60 0x7f12cc15d5d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.645+0000 7f12d1970700 1 -- 172.21.15.89:0/3335345348 >> 172.21.15.89:0/3335345348 conn(0x7f12cc068e20 msgr2=0x7f12cc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:58.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.645+0000 7f12d1970700 1 -- 172.21.15.89:0/3335345348 shutdown_connections 2023-10-10T20:34:58.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.645+0000 7f12d1970700 1 -- 172.21.15.89:0/3335345348 wait complete. 2023-10-10T20:34:58.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.646+0000 7f12d1970700 1 Processor -- start 2023-10-10T20:34:58.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.646+0000 7f12d1970700 1 -- start start 2023-10-10T20:34:58.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12d1970700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12cc159890 0x7f12cc0cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12d1970700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f12cc159c60 0x7f12cc1493c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12d1970700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc15db10 0x7f12cc16d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12d1970700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc0b0700 con 0x7f12cc15db10 2023-10-10T20:34:58.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12d1970700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc0cfee0 con 0x7f12cc159890 2023-10-10T20:34:58.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12d1970700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc151880 con 0x7f12cc159c60 2023-10-10T20:34:58.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12cb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc15db10 0x7f12cc16d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12cb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc15db10 0x7f12cc16d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59188/0 (socket says 172.21.15.89:59188) 2023-10-10T20:34:58.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12cb7fe700 1 -- 172.21.15.89:0/2138006118 learned_addr learned my addr 172.21.15.89:0/2138006118 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:58.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12caffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12cc159890 0x7f12cc0cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12ca7fc700 1 --2- 172.21.15.89:0/2138006118 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f12cc159c60 0x7f12cc1493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12cb7fe700 1 -- 172.21.15.89:0/2138006118 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f12cc159c60 msgr2=0x7f12cc1493c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12cb7fe700 1 --2- 172.21.15.89:0/2138006118 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f12cc159c60 0x7f12cc1493c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12cb7fe700 1 -- 172.21.15.89:0/2138006118 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12cc159890 msgr2=0x7f12cc0cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12cb7fe700 1 --2- 172.21.15.89:0/2138006118 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12cc159890 0x7f12cc0cf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12cb7fe700 1 -- 172.21.15.89:0/2138006118 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c0009ae0 con 0x7f12cc15db10 2023-10-10T20:34:58.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12cb7fe700 1 --2- 172.21.15.89:0/2138006118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc15db10 0x7f12cc16d0d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00c4c0 tx=0x7f12bc007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:58.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12b2ffd700 1 -- 172.21.15.89:0/2138006118 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc00ce70 con 0x7f12cc15db10 2023-10-10T20:34:58.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12b2ffd700 1 -- 172.21.15.89:0/2138006118 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc011590 con 0x7f12cc15db10 2023-10-10T20:34:58.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12d1970700 1 -- 172.21.15.89:0/2138006118 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12cc151b60 con 0x7f12cc15db10 2023-10-10T20:34:58.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.647+0000 7f12b2ffd700 1 -- 172.21.15.89:0/2138006118 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc011950 con 0x7f12cc15db10 2023-10-10T20:34:58.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.648+0000 7f12d1970700 1 -- 172.21.15.89:0/2138006118 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc149900 con 0x7f12cc15db10 2023-10-10T20:34:58.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.648+0000 7f12d1970700 1 -- 172.21.15.89:0/2138006118 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12cc0b0870 con 0x7f12cc15db10 2023-10-10T20:34:58.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.649+0000 7f12b2ffd700 1 -- 172.21.15.89:0/2138006118 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f12bc011d00 con 0x7f12cc15db10 2023-10-10T20:34:58.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.649+0000 7f12b2ffd700 1 --2- 172.21.15.89:0/2138006118 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f12b8066f30 0x7f12b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.649+0000 7f12b2ffd700 1 -- 172.21.15.89:0/2138006118 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f12bc083870 con 0x7f12cc15db10 2023-10-10T20:34:58.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.649+0000 7f12caffd700 1 --2- 172.21.15.89:0/2138006118 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f12b8066f30 0x7f12b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.650+0000 7f12caffd700 1 --2- 172.21.15.89:0/2138006118 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f12b8066f30 0x7f12b80693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009db0 tx=0x7f12c40091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:58.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.653+0000 7f12b2ffd700 1 -- 172.21.15.89:0/2138006118 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12cc0b0870 con 0x7f12cc15db10 2023-10-10T20:34:58.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.839+0000 7f12d1970700 1 -- 172.21.15.89:0/2138006118 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f12cc0b0840 con 0x7f12cc15db10 2023-10-10T20:34:58.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.839+0000 7f12b2ffd700 1 -- 172.21.15.89:0/2138006118 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f12bc017020 con 0x7f12cc15db10 2023-10-10T20:34:58.842 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:34:58.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.841+0000 7f12d1970700 1 -- 172.21.15.89:0/2138006118 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f12b8066f30 msgr2=0x7f12b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.841+0000 7f12d1970700 1 --2- 172.21.15.89:0/2138006118 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f12b8066f30 0x7f12b80693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009db0 tx=0x7f12c40091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.841+0000 7f12d1970700 1 -- 172.21.15.89:0/2138006118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc15db10 msgr2=0x7f12cc16d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.841+0000 7f12d1970700 1 --2- 172.21.15.89:0/2138006118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc15db10 0x7f12cc16d0d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00c4c0 tx=0x7f12bc007f40 comp rx=0 tx=0).stop 2023-10-10T20:34:58.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.841+0000 7f12d1970700 1 -- 172.21.15.89:0/2138006118 shutdown_connections 2023-10-10T20:34:58.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.841+0000 7f12d1970700 1 --2- 172.21.15.89:0/2138006118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc15db10 0x7f12cc16d0d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.841+0000 7f12d1970700 1 --2- 172.21.15.89:0/2138006118 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f12b8066f30 0x7f12b80693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.841+0000 7f12d1970700 1 --2- 172.21.15.89:0/2138006118 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12cc159890 0x7f12cc0cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.841+0000 7f12d1970700 1 --2- 172.21.15.89:0/2138006118 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f12cc159c60 0x7f12cc1493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.841+0000 7f12d1970700 1 -- 172.21.15.89:0/2138006118 >> 172.21.15.89:0/2138006118 conn(0x7f12cc068e20 msgr2=0x7f12cc153190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:58.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.841+0000 7f12d1970700 1 -- 172.21.15.89:0/2138006118 shutdown_connections 2023-10-10T20:34:58.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.841+0000 7f12d1970700 1 -- 172.21.15.89:0/2138006118 wait complete. 2023-10-10T20:34:58.856 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:34:58.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.971+0000 7f5748931700 1 Processor -- start 2023-10-10T20:34:58.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.971+0000 7f5748931700 1 -- start start 2023-10-10T20:34:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.972+0000 7f5748931700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f57440cd1e0 0x7f57440c9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.972+0000 7f5748931700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574415c2a0 con 0x7f574406bcd0 2023-10-10T20:34:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.972+0000 7f5748931700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574415b690 con 0x7f5744141500 2023-10-10T20:34:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.972+0000 7f5748931700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574415b970 con 0x7f57440cd1e0 2023-10-10T20:34:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f574259c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f574406bcd0 0x7f5744140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58104/0 (socket says 172.21.15.89:58104) 2023-10-10T20:34:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f574259c700 1 -- 172.21.15.89:0/1032045809 learned_addr learned my addr 172.21.15.89:0/1032045809 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f5742d9d700 1 --2- 172.21.15.89:0/1032045809 >> v2:172.21.15.111:3300/0 conn(0x7f57440cd1e0 0x7f57440c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f574159a700 1 -- 172.21.15.89:0/1032045809 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346629315 0 0) 0x7f574415c2a0 con 0x7f574406bcd0 2023-10-10T20:34:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f574159a700 1 -- 172.21.15.89:0/1032045809 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5730003570 con 0x7f574406bcd0 2023-10-10T20:34:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f574159a700 1 -- 172.21.15.89:0/1032045809 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1253422652 0 0) 0x7f5730003570 con 0x7f574406bcd0 2023-10-10T20:34:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f574159a700 1 -- 172.21.15.89:0/1032045809 >> v2:172.21.15.111:3300/0 conn(0x7f57440cd1e0 msgr2=0x7f57440c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f574159a700 1 --2- 172.21.15.89:0/1032045809 >> v2:172.21.15.111:3300/0 conn(0x7f57440cd1e0 0x7f57440c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f574159a700 1 -- 172.21.15.89:0/1032045809 >> v1:172.21.15.111:6789/0 conn(0x7f5744141500 legacy=0x7f57440c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f574159a700 1 -- 172.21.15.89:0/1032045809 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574415bc50 con 0x7f574406bcd0 2023-10-10T20:34:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f574159a700 1 -- 172.21.15.89:0/1032045809 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f5734001470 con 0x7f574406bcd0 2023-10-10T20:34:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f574159a700 1 -- 172.21.15.89:0/1032045809 >> v1:172.21.15.89:6789/0 conn(0x7f574406bcd0 legacy=0x7f5744140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f574159a700 1 --2- 172.21.15.89:0/1032045809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5730003a60 0x7f5730003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f574159a700 1 --2- 172.21.15.89:0/1032045809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5730004810 0x7f5730004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f574159a700 1 --2- 172.21.15.89:0/1032045809 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57300051c0 0x7f573000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f574159a700 1 -- 172.21.15.89:0/1032045809 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574415b970 con 0x7f5730003a60 2023-10-10T20:34:58.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f574159a700 1 -- 172.21.15.89:0/1032045809 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573000c0d0 con 0x7f5730004810 2023-10-10T20:34:58.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f574159a700 1 -- 172.21.15.89:0/1032045809 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573000c290 con 0x7f57300051c0 2023-10-10T20:34:58.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f574259c700 1 --2- 172.21.15.89:0/1032045809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5730003a60 0x7f5730003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f5741d9b700 1 --2- 172.21.15.89:0/1032045809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5730004810 0x7f5730004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.973+0000 7f5742d9d700 1 --2- 172.21.15.89:0/1032045809 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57300051c0 0x7f573000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.974+0000 7f574259c700 1 -- 172.21.15.89:0/1032045809 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57300051c0 msgr2=0x7f573000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.974+0000 7f574259c700 1 --2- 172.21.15.89:0/1032045809 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57300051c0 0x7f573000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.974+0000 7f574259c700 1 -- 172.21.15.89:0/1032045809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5730004810 msgr2=0x7f5730004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.974+0000 7f574259c700 1 --2- 172.21.15.89:0/1032045809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5730004810 0x7f5730004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.974+0000 7f574259c700 1 -- 172.21.15.89:0/1032045809 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5730003570 con 0x7f5730003a60 2023-10-10T20:34:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.974+0000 7f5742d9d700 1 -- 172.21.15.89:0/1032045809 reap_dead start 2023-10-10T20:34:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.974+0000 7f574259c700 1 --2- 172.21.15.89:0/1032045809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5730003a60 0x7f5730003e70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f573400c390 tx=0x7f574415c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f900c0f360b3c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.974+0000 7f574159a700 1 -- 172.21.15.89:0/1032045809 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573400cbe0 con 0x7f5730003a60 2023-10-10T20:34:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.974+0000 7f574159a700 1 -- 172.21.15.89:0/1032045809 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573400cd40 con 0x7f5730003a60 2023-10-10T20:34:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.974+0000 7f574159a700 1 -- 172.21.15.89:0/1032045809 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734005b70 con 0x7f5730003a60 2023-10-10T20:34:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.974+0000 7f5748931700 1 -- 172.21.15.89:0/1032045809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5730003a60 msgr2=0x7f5730003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.974+0000 7f5748931700 1 --2- 172.21.15.89:0/1032045809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5730003a60 0x7f5730003e70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f573400c390 tx=0x7f574415c210 comp rx=0 tx=0).stop 2023-10-10T20:34:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.974+0000 7f5748931700 1 -- 172.21.15.89:0/1032045809 shutdown_connections 2023-10-10T20:34:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.974+0000 7f5748931700 1 --2- 172.21.15.89:0/1032045809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5730003a60 0x7f5730003e70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.974+0000 7f5748931700 1 -- 172.21.15.89:0/1032045809 >> 172.21.15.89:0/1032045809 conn(0x7f57440ce600 msgr2=0x7f57440cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.974+0000 7f5748931700 1 -- 172.21.15.89:0/1032045809 shutdown_connections 2023-10-10T20:34:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.974+0000 7f5748931700 1 -- 172.21.15.89:0/1032045809 wait complete. 2023-10-10T20:34:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.976+0000 7f5748931700 1 Processor -- start 2023-10-10T20:34:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.976+0000 7f5748931700 1 -- start start 2023-10-10T20:34:58.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.976+0000 7f5748931700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5730003a60 0x7f57440c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.976+0000 7f5748931700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57440c4620 0x7f57440c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.976+0000 7f5748931700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57441687b0 0x7f5744168c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.976+0000 7f5748931700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57440c4fd0 con 0x7f5730003a60 2023-10-10T20:34:58.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.976+0000 7f5748931700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57440c8130 con 0x7f57441687b0 2023-10-10T20:34:58.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.976+0000 7f5748931700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574413cf60 con 0x7f57440c4620 2023-10-10T20:34:58.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.976+0000 7f574259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5730003a60 0x7f57440c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.976+0000 7f574259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5730003a60 0x7f57440c60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59206/0 (socket says 172.21.15.89:59206) 2023-10-10T20:34:58.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.976+0000 7f574259c700 1 -- 172.21.15.89:0/1889573145 learned_addr learned my addr 172.21.15.89:0/1889573145 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:58.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.976+0000 7f5742d9d700 1 --2- 172.21.15.89:0/1889573145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57441687b0 0x7f5744168c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.977+0000 7f5741d9b700 1 --2- 172.21.15.89:0/1889573145 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57440c4620 0x7f57440c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.977+0000 7f574259c700 1 -- 172.21.15.89:0/1889573145 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57440c4620 msgr2=0x7f57440c4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.977+0000 7f574259c700 1 --2- 172.21.15.89:0/1889573145 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57440c4620 0x7f57440c4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.977+0000 7f574259c700 1 -- 172.21.15.89:0/1889573145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57441687b0 msgr2=0x7f5744168c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:58.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.977+0000 7f574259c700 1 --2- 172.21.15.89:0/1889573145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57441687b0 0x7f5744168c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:58.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.977+0000 7f574259c700 1 -- 172.21.15.89:0/1889573145 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574413c9a0 con 0x7f5730003a60 2023-10-10T20:34:58.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.977+0000 7f5742d9d700 1 --2- 172.21.15.89:0/1889573145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57441687b0 0x7f5744168c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:58.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.977+0000 7f574259c700 1 --2- 172.21.15.89:0/1889573145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5730003a60 0x7f57440c60c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f573000c0a0 tx=0x7f5734015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:58.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.977+0000 7f573a7fc700 1 -- 172.21.15.89:0/1889573145 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734022070 con 0x7f5730003a60 2023-10-10T20:34:58.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.977+0000 7f573a7fc700 1 -- 172.21.15.89:0/1889573145 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573400ceb0 con 0x7f5730003a60 2023-10-10T20:34:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.977+0000 7f5748931700 1 -- 172.21.15.89:0/1889573145 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574413cc80 con 0x7f5730003a60 2023-10-10T20:34:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.977+0000 7f573a7fc700 1 -- 172.21.15.89:0/1889573145 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573400cbe0 con 0x7f5730003a60 2023-10-10T20:34:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.977+0000 7f5748931700 1 -- 172.21.15.89:0/1889573145 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5744169140 con 0x7f5730003a60 2023-10-10T20:34:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.978+0000 7f5748931700 1 -- 172.21.15.89:0/1889573145 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57440b3120 con 0x7f5730003a60 2023-10-10T20:34:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.978+0000 7f573a7fc700 1 -- 172.21.15.89:0/1889573145 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f573401e430 con 0x7f5730003a60 2023-10-10T20:34:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.979+0000 7f573a7fc700 1 --2- 172.21.15.89:0/1889573145 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5724066f30 0x7f57240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.979+0000 7f573a7fc700 1 -- 172.21.15.89:0/1889573145 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5734085720 con 0x7f5730003a60 2023-10-10T20:34:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.979+0000 7f5741d9b700 1 --2- 172.21.15.89:0/1889573145 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5724066f30 0x7f57240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.979+0000 7f5741d9b700 1 --2- 172.21.15.89:0/1889573145 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5724066f30 0x7f57240693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f574413f4c0 tx=0x7f572c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:58.983+0000 7f573a7fc700 1 -- 172.21.15.89:0/1889573145 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f57340529a0 con 0x7f5730003a60 2023-10-10T20:34:59.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.168+0000 7f5748931700 1 -- 172.21.15.89:0/1889573145 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57440c8410 con 0x7f5730003a60 2023-10-10T20:34:59.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.168+0000 7f573a7fc700 1 -- 172.21.15.89:0/1889573145 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f57340170a0 con 0x7f5730003a60 2023-10-10T20:34:59.171 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-10T20:34:59.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.170+0000 7f5748931700 1 -- 172.21.15.89:0/1889573145 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5724066f30 msgr2=0x7f57240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:59.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.170+0000 7f5748931700 1 --2- 172.21.15.89:0/1889573145 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5724066f30 0x7f57240693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f574413f4c0 tx=0x7f572c009040 comp rx=0 tx=0).stop 2023-10-10T20:34:59.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.170+0000 7f5748931700 1 -- 172.21.15.89:0/1889573145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5730003a60 msgr2=0x7f57440c60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:59.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.170+0000 7f5748931700 1 --2- 172.21.15.89:0/1889573145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5730003a60 0x7f57440c60c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f573000c0a0 tx=0x7f5734015000 comp rx=0 tx=0).stop 2023-10-10T20:34:59.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.170+0000 7f5748931700 1 -- 172.21.15.89:0/1889573145 shutdown_connections 2023-10-10T20:34:59.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.170+0000 7f5748931700 1 --2- 172.21.15.89:0/1889573145 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5724066f30 0x7f57240693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.170+0000 7f5748931700 1 --2- 172.21.15.89:0/1889573145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57441687b0 0x7f5744168c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.170+0000 7f5748931700 1 --2- 172.21.15.89:0/1889573145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5730003a60 0x7f57440c60c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.170+0000 7f5748931700 1 --2- 172.21.15.89:0/1889573145 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f57440c4620 0x7f57440c4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.170+0000 7f5748931700 1 -- 172.21.15.89:0/1889573145 >> 172.21.15.89:0/1889573145 conn(0x7f57440ce600 msgr2=0x7f57440c3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:59.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.170+0000 7f5748931700 1 -- 172.21.15.89:0/1889573145 shutdown_connections 2023-10-10T20:34:59.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.170+0000 7f5748931700 1 -- 172.21.15.89:0/1889573145 wait complete. 2023-10-10T20:34:59.184 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:34:59.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.296+0000 7f50611ee700 1 Processor -- start 2023-10-10T20:34:59.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.296+0000 7f50611ee700 1 -- start start 2023-10-10T20:34:59.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.297+0000 7f50611ee700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f505c159890 0x7f505c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.297+0000 7f50611ee700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f505c159c60 0x7f505c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.297+0000 7f50611ee700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f505c15db10 0x7f505c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.297+0000 7f50611ee700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c15df50 con 0x7f505c159c60 2023-10-10T20:34:59.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.297+0000 7f50611ee700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c15a160 con 0x7f505c159890 2023-10-10T20:34:59.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.297+0000 7f50611ee700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0d13a0 con 0x7f505c15db10 2023-10-10T20:34:59.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.297+0000 7f505259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f505c159c60 0x7f505c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.297+0000 7f505259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f505c159c60 0x7f505c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59222/0 (socket says 172.21.15.89:59222) 2023-10-10T20:34:59.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.297+0000 7f505259c700 1 -- 172.21.15.89:0/489203420 learned_addr learned my addr 172.21.15.89:0/489203420 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:59.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.298+0000 7f505b59e700 1 --2- 172.21.15.89:0/489203420 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f505c15db10 0x7f505c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.298+0000 7f505ad9d700 1 --2- 172.21.15.89:0/489203420 >> v2:172.21.15.111:3300/0 conn(0x7f505c159890 0x7f505c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.298+0000 7f505259c700 1 -- 172.21.15.89:0/489203420 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f505c15db10 msgr2=0x7f505c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:59.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.298+0000 7f505259c700 1 --2- 172.21.15.89:0/489203420 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f505c15db10 0x7f505c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.298+0000 7f505259c700 1 -- 172.21.15.89:0/489203420 >> v2:172.21.15.111:3300/0 conn(0x7f505c159890 msgr2=0x7f505c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:59.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.298+0000 7f505259c700 1 --2- 172.21.15.89:0/489203420 >> v2:172.21.15.111:3300/0 conn(0x7f505c159890 0x7f505c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.298+0000 7f505259c700 1 -- 172.21.15.89:0/489203420 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c15e140 con 0x7f505c159c60 2023-10-10T20:34:59.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.298+0000 7f505b59e700 1 --2- 172.21.15.89:0/489203420 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f505c15db10 0x7f505c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:59.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.298+0000 7f505259c700 1 --2- 172.21.15.89:0/489203420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505c159c60 0x7f505c15d5d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f505c0d1090 tx=0x7f5044009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32d317e0b67211d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:59.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.298+0000 7f505a59c700 1 -- 172.21.15.89:0/489203420 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504400b860 con 0x7f505c159c60 2023-10-10T20:34:59.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.298+0000 7f505a59c700 1 -- 172.21.15.89:0/489203420 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504400b9c0 con 0x7f505c159c60 2023-10-10T20:34:59.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.298+0000 7f505a59c700 1 -- 172.21.15.89:0/489203420 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504400bd60 con 0x7f505c159c60 2023-10-10T20:34:59.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.298+0000 7f50611ee700 1 -- 172.21.15.89:0/489203420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505c159c60 msgr2=0x7f505c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:59.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.298+0000 7f50611ee700 1 --2- 172.21.15.89:0/489203420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505c159c60 0x7f505c15d5d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f505c0d1090 tx=0x7f5044009250 comp rx=0 tx=0).stop 2023-10-10T20:34:59.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.299+0000 7f50611ee700 1 -- 172.21.15.89:0/489203420 shutdown_connections 2023-10-10T20:34:59.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.299+0000 7f50611ee700 1 --2- 172.21.15.89:0/489203420 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f505c15db10 0x7f505c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.299+0000 7f50611ee700 1 --2- 172.21.15.89:0/489203420 >> v2:172.21.15.111:3300/0 conn(0x7f505c159890 0x7f505c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.299+0000 7f50611ee700 1 --2- 172.21.15.89:0/489203420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505c159c60 0x7f505c15d5d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.299+0000 7f50611ee700 1 -- 172.21.15.89:0/489203420 >> 172.21.15.89:0/489203420 conn(0x7f505c068e20 msgr2=0x7f505c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.299+0000 7f50611ee700 1 -- 172.21.15.89:0/489203420 shutdown_connections 2023-10-10T20:34:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.299+0000 7f50611ee700 1 -- 172.21.15.89:0/489203420 wait complete. 2023-10-10T20:34:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.300+0000 7f50611ee700 1 Processor -- start 2023-10-10T20:34:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.300+0000 7f50611ee700 1 -- start start 2023-10-10T20:34:59.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f50611ee700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f505c159890 0x7f505c16ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f50611ee700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f505c159c60 0x7f505c0ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f50611ee700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505c15db10 0x7f505c169b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f50611ee700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c16aba0 con 0x7f505c15db10 2023-10-10T20:34:59.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f50611ee700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0c8280 con 0x7f505c159890 2023-10-10T20:34:59.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f50611ee700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c16b2a0 con 0x7f505c159c60 2023-10-10T20:34:59.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f505b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505c15db10 0x7f505c169b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f505b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505c15db10 0x7f505c169b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59224/0 (socket says 172.21.15.89:59224) 2023-10-10T20:34:59.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f505b59e700 1 -- 172.21.15.89:0/449978688 learned_addr learned my addr 172.21.15.89:0/449978688 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:59.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f505ad9d700 1 --2- 172.21.15.89:0/449978688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f505c159890 0x7f505c16ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f505259c700 1 --2- 172.21.15.89:0/449978688 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f505c159c60 0x7f505c0ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f505b59e700 1 -- 172.21.15.89:0/449978688 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f505c159c60 msgr2=0x7f505c0ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:59.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f505b59e700 1 --2- 172.21.15.89:0/449978688 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f505c159c60 0x7f505c0ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f505b59e700 1 -- 172.21.15.89:0/449978688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f505c159890 msgr2=0x7f505c16ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:59.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f505b59e700 1 --2- 172.21.15.89:0/449978688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f505c159890 0x7f505c16ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f505b59e700 1 -- 172.21.15.89:0/449978688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50440094f0 con 0x7f505c15db10 2023-10-10T20:34:59.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f505b59e700 1 --2- 172.21.15.89:0/449978688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505c15db10 0x7f505c169b90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f505400d180 tx=0x7f5054009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:59.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f50537fe700 1 -- 172.21.15.89:0/449978688 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505400dcc0 con 0x7f505c15db10 2023-10-10T20:34:59.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f50537fe700 1 -- 172.21.15.89:0/449978688 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505400f040 con 0x7f505c15db10 2023-10-10T20:34:59.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f50611ee700 1 -- 172.21.15.89:0/449978688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505c16a5e0 con 0x7f505c15db10 2023-10-10T20:34:59.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.301+0000 7f50537fe700 1 -- 172.21.15.89:0/449978688 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054015440 con 0x7f505c15db10 2023-10-10T20:34:59.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.302+0000 7f50611ee700 1 -- 172.21.15.89:0/449978688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c0c7f10 con 0x7f505c15db10 2023-10-10T20:34:59.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.302+0000 7f50611ee700 1 -- 172.21.15.89:0/449978688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505c04c3c0 con 0x7f505c15db10 2023-10-10T20:34:59.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.303+0000 7f50537fe700 1 -- 172.21.15.89:0/449978688 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f505401e3b0 con 0x7f505c15db10 2023-10-10T20:34:59.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.303+0000 7f50537fe700 1 --2- 172.21.15.89:0/449978688 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f503c066f30 0x7f503c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.303+0000 7f50537fe700 1 -- 172.21.15.89:0/449978688 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f50540849b0 con 0x7f505c15db10 2023-10-10T20:34:59.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.303+0000 7f505ad9d700 1 --2- 172.21.15.89:0/449978688 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f503c066f30 0x7f503c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.304+0000 7f505ad9d700 1 --2- 172.21.15.89:0/449978688 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f503c066f30 0x7f503c0693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f505c0b2900 tx=0x7f504c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:59.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.307+0000 7f50537fe700 1 -- 172.21.15.89:0/449978688 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5054051c20 con 0x7f505c15db10 2023-10-10T20:34:59.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.493+0000 7f50611ee700 1 -- 172.21.15.89:0/449978688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f505c0c83c0 con 0x7f505c15db10 2023-10-10T20:34:59.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.494+0000 7f50537fe700 1 -- 172.21.15.89:0/449978688 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f505401a090 con 0x7f505c15db10 2023-10-10T20:34:59.496 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:34:59.496 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:36.324920+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:34:32.342167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:34:33.671763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:34:59.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.495+0000 7f50611ee700 1 -- 172.21.15.89:0/449978688 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f503c066f30 msgr2=0x7f503c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:59.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.495+0000 7f50611ee700 1 --2- 172.21.15.89:0/449978688 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f503c066f30 0x7f503c0693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f505c0b2900 tx=0x7f504c008040 comp rx=0 tx=0).stop 2023-10-10T20:34:59.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.495+0000 7f50611ee700 1 -- 172.21.15.89:0/449978688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505c15db10 msgr2=0x7f505c169b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:59.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.495+0000 7f50611ee700 1 --2- 172.21.15.89:0/449978688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505c15db10 0x7f505c169b90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f505400d180 tx=0x7f5054009a50 comp rx=0 tx=0).stop 2023-10-10T20:34:59.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.495+0000 7f50611ee700 1 -- 172.21.15.89:0/449978688 shutdown_connections 2023-10-10T20:34:59.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.495+0000 7f50611ee700 1 --2- 172.21.15.89:0/449978688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505c15db10 0x7f505c169b90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.495+0000 7f50611ee700 1 --2- 172.21.15.89:0/449978688 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f503c066f30 0x7f503c0693e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.495+0000 7f50611ee700 1 --2- 172.21.15.89:0/449978688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f505c159890 0x7f505c16ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.495+0000 7f50611ee700 1 --2- 172.21.15.89:0/449978688 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f505c159c60 0x7f505c0ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.495+0000 7f50611ee700 1 -- 172.21.15.89:0/449978688 >> 172.21.15.89:0/449978688 conn(0x7f505c068e20 msgr2=0x7f505c15b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:59.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.496+0000 7f50611ee700 1 -- 172.21.15.89:0/449978688 shutdown_connections 2023-10-10T20:34:59.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.496+0000 7f50611ee700 1 -- 172.21.15.89:0/449978688 wait complete. 2023-10-10T20:34:59.510 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:34:32.342167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:34:33.671763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:34:59.511 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:34:59.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.625+0000 7f5e53fef700 1 Processor -- start 2023-10-10T20:34:59.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.626+0000 7f5e53fef700 1 -- start start 2023-10-10T20:34:59.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.626+0000 7f5e53fef700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e4c159b10 0x7f5e4c0cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.626+0000 7f5e53fef700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5e4c06b420 0x7f5e4c15dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.626+0000 7f5e53fef700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e4c0c74b0 con 0x7f5e4c15d730 2023-10-10T20:34:59.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.626+0000 7f5e53fef700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4c0c8700 con 0x7f5e4c06b420 2023-10-10T20:34:59.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.626+0000 7f5e53fef700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0c9070 con 0x7f5e4c159b10 2023-10-10T20:34:59.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.627+0000 7f5e5258c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5e4c06b420 0x7f5e4c15dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.627+0000 7f5e51d8b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e4c159b10 0x7f5e4c0cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.627+0000 7f5e5158a700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f5e4c15d730 0x7f5e4c159ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:53856/0 (socket says 172.21.15.89:53856) 2023-10-10T20:34:59.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.627+0000 7f5e5258c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5e4c06b420 0x7f5e4c15dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35114/0 (socket says 172.21.15.89:35114) 2023-10-10T20:34:59.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.627+0000 7f5e5158a700 1 -- 172.21.15.89:0/3944306112 learned_addr learned my addr 172.21.15.89:0/3944306112 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:59.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.627+0000 7f5e50d89700 1 -- 172.21.15.89:0/3944306112 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64119878 0 0) 0x7f5e4c0c74b0 con 0x7f5e4c15d730 2023-10-10T20:34:59.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.627+0000 7f5e50d89700 1 -- 172.21.15.89:0/3944306112 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e38003570 con 0x7f5e4c15d730 2023-10-10T20:34:59.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.627+0000 7f5e51d8b700 1 -- 172.21.15.89:0/3944306112 >> v2:172.21.15.111:3300/0 conn(0x7f5e4c06b420 msgr2=0x7f5e4c15dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:59.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.627+0000 7f5e51d8b700 1 --2- 172.21.15.89:0/3944306112 >> v2:172.21.15.111:3300/0 conn(0x7f5e4c06b420 0x7f5e4c15dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.627+0000 7f5e51d8b700 1 -- 172.21.15.89:0/3944306112 >> v1:172.21.15.111:6789/0 conn(0x7f5e4c15d730 legacy=0x7f5e4c159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:59.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.627+0000 7f5e51d8b700 1 -- 172.21.15.89:0/3944306112 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c15e0e0 con 0x7f5e4c159b10 2023-10-10T20:34:59.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.627+0000 7f5e5258c700 1 --2- 172.21.15.89:0/3944306112 >> v2:172.21.15.111:3300/0 conn(0x7f5e4c06b420 0x7f5e4c15dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:34:59.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.627+0000 7f5e51d8b700 1 --2- 172.21.15.89:0/3944306112 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e4c159b10 0x7f5e4c0cb3e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5e480098e0 tx=0x7f5e480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd4b10a1c138d640 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:59.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.627+0000 7f5e50d89700 1 -- 172.21.15.89:0/3944306112 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e48011630 con 0x7f5e4c159b10 2023-10-10T20:34:59.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.627+0000 7f5e50d89700 1 -- 172.21.15.89:0/3944306112 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e48011790 con 0x7f5e4c159b10 2023-10-10T20:34:59.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.627+0000 7f5e50d89700 1 -- 172.21.15.89:0/3944306112 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e48011b10 con 0x7f5e4c159b10 2023-10-10T20:34:59.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.628+0000 7f5e53fef700 1 -- 172.21.15.89:0/3944306112 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e4c159b10 msgr2=0x7f5e4c0cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:59.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.628+0000 7f5e53fef700 1 --2- 172.21.15.89:0/3944306112 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e4c159b10 0x7f5e4c0cb3e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5e480098e0 tx=0x7f5e480091a0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.628+0000 7f5e53fef700 1 -- 172.21.15.89:0/3944306112 shutdown_connections 2023-10-10T20:34:59.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.628+0000 7f5e53fef700 1 --2- 172.21.15.89:0/3944306112 >> v2:172.21.15.111:3300/0 conn(0x7f5e4c06b420 0x7f5e4c15dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.628+0000 7f5e53fef700 1 --2- 172.21.15.89:0/3944306112 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e4c159b10 0x7f5e4c0cb3e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.628+0000 7f5e53fef700 1 -- 172.21.15.89:0/3944306112 >> 172.21.15.89:0/3944306112 conn(0x7f5e4c0c9870 msgr2=0x7f5e4c0c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:59.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.628+0000 7f5e53fef700 1 -- 172.21.15.89:0/3944306112 shutdown_connections 2023-10-10T20:34:59.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.628+0000 7f5e53fef700 1 -- 172.21.15.89:0/3944306112 wait complete. 2023-10-10T20:34:59.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.629+0000 7f5e53fef700 1 Processor -- start 2023-10-10T20:34:59.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.629+0000 7f5e53fef700 1 -- start start 2023-10-10T20:34:59.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.630+0000 7f5e53fef700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e4c06b420 0x7f5e4c14b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.630+0000 7f5e53fef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e4c159b10 0x7f5e4c0b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.630+0000 7f5e53fef700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e4c15d730 0x7f5e4c153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.630+0000 7f5e53fef700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0b19d0 con 0x7f5e4c159b10 2023-10-10T20:34:59.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.630+0000 7f5e53fef700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c16df40 con 0x7f5e4c15d730 2023-10-10T20:34:59.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.630+0000 7f5e53fef700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c14ba90 con 0x7f5e4c06b420 2023-10-10T20:34:59.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.630+0000 7f5e5158a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e4c159b10 0x7f5e4c0b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.630+0000 7f5e5158a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e4c159b10 0x7f5e4c0b0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59240/0 (socket says 172.21.15.89:59240) 2023-10-10T20:34:59.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.630+0000 7f5e5158a700 1 -- 172.21.15.89:0/1178445058 learned_addr learned my addr 172.21.15.89:0/1178445058 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:59.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.630+0000 7f5e51d8b700 1 --2- 172.21.15.89:0/1178445058 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e4c06b420 0x7f5e4c14b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.630+0000 7f5e5258c700 1 --2- 172.21.15.89:0/1178445058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e4c15d730 0x7f5e4c153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.630+0000 7f5e5158a700 1 -- 172.21.15.89:0/1178445058 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e4c06b420 msgr2=0x7f5e4c14b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:59.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.630+0000 7f5e5158a700 1 --2- 172.21.15.89:0/1178445058 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e4c06b420 0x7f5e4c14b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.630+0000 7f5e5158a700 1 -- 172.21.15.89:0/1178445058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e4c15d730 msgr2=0x7f5e4c153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:59.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.630+0000 7f5e5158a700 1 --2- 172.21.15.89:0/1178445058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e4c15d730 0x7f5e4c153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.630+0000 7f5e5158a700 1 -- 172.21.15.89:0/1178445058 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e44009a20 con 0x7f5e4c159b10 2023-10-10T20:34:59.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.630+0000 7f5e5258c700 1 --2- 172.21.15.89:0/1178445058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e4c15d730 0x7f5e4c153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:34:59.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.630+0000 7f5e5158a700 1 --2- 172.21.15.89:0/1178445058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e4c159b10 0x7f5e4c0b0ef0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00c580 tx=0x7f5e3c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:59.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.630+0000 7f5e41ffb700 1 -- 172.21.15.89:0/1178445058 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c005b30 con 0x7f5e4c159b10 2023-10-10T20:34:59.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.631+0000 7f5e41ffb700 1 -- 172.21.15.89:0/1178445058 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c005c90 con 0x7f5e4c159b10 2023-10-10T20:34:59.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.631+0000 7f5e41ffb700 1 -- 172.21.15.89:0/1178445058 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c007320 con 0x7f5e4c159b10 2023-10-10T20:34:59.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.631+0000 7f5e53fef700 1 -- 172.21.15.89:0/1178445058 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e48009420 con 0x7f5e4c159b10 2023-10-10T20:34:59.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.631+0000 7f5e53fef700 1 -- 172.21.15.89:0/1178445058 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4c16d8f0 con 0x7f5e4c159b10 2023-10-10T20:34:59.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.632+0000 7f5e53fef700 1 -- 172.21.15.89:0/1178445058 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e4c04c3c0 con 0x7f5e4c159b10 2023-10-10T20:34:59.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.632+0000 7f5e41ffb700 1 -- 172.21.15.89:0/1178445058 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5e3c007690 con 0x7f5e4c159b10 2023-10-10T20:34:59.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.632+0000 7f5e41ffb700 1 --2- 172.21.15.89:0/1178445058 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5e30066f30 0x7f5e300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.632+0000 7f5e41ffb700 1 -- 172.21.15.89:0/1178445058 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e3c0839f0 con 0x7f5e4c159b10 2023-10-10T20:34:59.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.633+0000 7f5e51d8b700 1 --2- 172.21.15.89:0/1178445058 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5e30066f30 0x7f5e300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.633+0000 7f5e51d8b700 1 --2- 172.21.15.89:0/1178445058 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5e30066f30 0x7f5e300693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5e480091e0 tx=0x7f5e48017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:59.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.637+0000 7f5e41ffb700 1 -- 172.21.15.89:0/1178445058 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e3c050bc0 con 0x7f5e4c159b10 2023-10-10T20:34:59.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.819+0000 7f5e53fef700 1 -- 172.21.15.89:0/1178445058 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e4c0633e0 con 0x7f5e4c159b10 2023-10-10T20:34:59.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.819+0000 7f5e41ffb700 1 -- 172.21.15.89:0/1178445058 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e3c017020 con 0x7f5e4c159b10 2023-10-10T20:34:59.822 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:34:59.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.821+0000 7f5e53fef700 1 -- 172.21.15.89:0/1178445058 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5e30066f30 msgr2=0x7f5e300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:59.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.821+0000 7f5e53fef700 1 --2- 172.21.15.89:0/1178445058 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5e30066f30 0x7f5e300693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5e480091e0 tx=0x7f5e48017000 comp rx=0 tx=0).stop 2023-10-10T20:34:59.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.821+0000 7f5e53fef700 1 -- 172.21.15.89:0/1178445058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e4c159b10 msgr2=0x7f5e4c0b0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:59.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.821+0000 7f5e53fef700 1 --2- 172.21.15.89:0/1178445058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e4c159b10 0x7f5e4c0b0ef0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00c580 tx=0x7f5e3c007df0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.821+0000 7f5e53fef700 1 -- 172.21.15.89:0/1178445058 shutdown_connections 2023-10-10T20:34:59.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.821+0000 7f5e53fef700 1 --2- 172.21.15.89:0/1178445058 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5e30066f30 0x7f5e300693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.821+0000 7f5e53fef700 1 --2- 172.21.15.89:0/1178445058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e4c15d730 0x7f5e4c153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.821+0000 7f5e53fef700 1 --2- 172.21.15.89:0/1178445058 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e4c06b420 0x7f5e4c14b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.821+0000 7f5e53fef700 1 --2- 172.21.15.89:0/1178445058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e4c159b10 0x7f5e4c0b0ef0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.821+0000 7f5e53fef700 1 -- 172.21.15.89:0/1178445058 >> 172.21.15.89:0/1178445058 conn(0x7f5e4c0c9870 msgr2=0x7f5e4c06e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:59.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.821+0000 7f5e53fef700 1 -- 172.21.15.89:0/1178445058 shutdown_connections 2023-10-10T20:34:59.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.821+0000 7f5e53fef700 1 -- 172.21.15.89:0/1178445058 wait complete. 2023-10-10T20:34:59.835 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:34:59.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.952+0000 7f6dd5a80700 1 Processor -- start 2023-10-10T20:34:59.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.952+0000 7f6dd5a80700 1 -- start start 2023-10-10T20:34:59.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dd5a80700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6dd0159890 0x7f6dd015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dd5a80700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6dd0069560 0x7f6dd00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dd5a80700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dd0069ee0 con 0x7f6dd0159c60 2023-10-10T20:34:59.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dd5a80700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd015a160 con 0x7f6dd0159890 2023-10-10T20:34:59.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dd5a80700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd00d13a0 con 0x7f6dd0069560 2023-10-10T20:34:59.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dceffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6dd0159c60 0x7f6dd015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58120/0 (socket says 172.21.15.89:58120) 2023-10-10T20:34:59.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dceffd700 1 -- 172.21.15.89:0/174185796 learned_addr learned my addr 172.21.15.89:0/174185796 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:34:59.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dcf7fe700 1 --2- 172.21.15.89:0/174185796 >> v2:172.21.15.111:3300/0 conn(0x7f6dd0159890 0x7f6dd015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dcffff700 1 --2- 172.21.15.89:0/174185796 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6dd0069560 0x7f6dd00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dce7fc700 1 -- 172.21.15.89:0/174185796 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789687506 0 0) 0x7f6dd0069ee0 con 0x7f6dd0159c60 2023-10-10T20:34:59.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dce7fc700 1 -- 172.21.15.89:0/174185796 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6db8003570 con 0x7f6dd0159c60 2023-10-10T20:34:59.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dce7fc700 1 -- 172.21.15.89:0/174185796 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4051150623 0 0) 0x7f6db8003570 con 0x7f6dd0159c60 2023-10-10T20:34:59.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dce7fc700 1 -- 172.21.15.89:0/174185796 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6dd0069560 msgr2=0x7f6dd00699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:59.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dce7fc700 1 --2- 172.21.15.89:0/174185796 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6dd0069560 0x7f6dd00699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dce7fc700 1 -- 172.21.15.89:0/174185796 >> v2:172.21.15.111:3300/0 conn(0x7f6dd0159890 msgr2=0x7f6dd015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:59.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dce7fc700 1 --2- 172.21.15.89:0/174185796 >> v2:172.21.15.111:3300/0 conn(0x7f6dd0159890 0x7f6dd015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dce7fc700 1 -- 172.21.15.89:0/174185796 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd015e140 con 0x7f6dd0159c60 2023-10-10T20:34:59.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dce7fc700 1 -- 172.21.15.89:0/174185796 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f6dc4001470 con 0x7f6dd0159c60 2023-10-10T20:34:59.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dce7fc700 1 -- 172.21.15.89:0/174185796 >> v1:172.21.15.89:6789/0 conn(0x7f6dd0159c60 legacy=0x7f6dd015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:59.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dce7fc700 1 --2- 172.21.15.89:0/174185796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6db8003a60 0x7f6db8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dce7fc700 1 --2- 172.21.15.89:0/174185796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6db8004810 0x7f6db8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dce7fc700 1 --2- 172.21.15.89:0/174185796 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6dd0069560 0x7f6db80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dce7fc700 1 -- 172.21.15.89:0/174185796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd015a160 con 0x7f6db8004810 2023-10-10T20:34:59.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dce7fc700 1 -- 172.21.15.89:0/174185796 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd00d13a0 con 0x7f6db8003a60 2023-10-10T20:34:59.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dce7fc700 1 -- 172.21.15.89:0/174185796 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db800ba60 con 0x7f6dd0069560 2023-10-10T20:34:59.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.953+0000 7f6dceffd700 1 --2- 172.21.15.89:0/174185796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6db8004810 0x7f6db8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.954+0000 7f6dcf7fe700 1 --2- 172.21.15.89:0/174185796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6db8003a60 0x7f6db8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.954+0000 7f6dcffff700 1 --2- 172.21.15.89:0/174185796 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6dd0069560 0x7f6db80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.954+0000 7f6dceffd700 1 -- 172.21.15.89:0/174185796 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6dd0069560 msgr2=0x7f6db80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:59.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.954+0000 7f6dceffd700 1 --2- 172.21.15.89:0/174185796 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6dd0069560 0x7f6db80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.954+0000 7f6dceffd700 1 -- 172.21.15.89:0/174185796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6db8003a60 msgr2=0x7f6db8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:34:59.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.954+0000 7f6dceffd700 1 --2- 172.21.15.89:0/174185796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6db8003a60 0x7f6db8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.954+0000 7f6dceffd700 1 -- 172.21.15.89:0/174185796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db8003570 con 0x7f6db8004810 2023-10-10T20:34:59.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.954+0000 7f6dceffd700 1 --2- 172.21.15.89:0/174185796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6db8004810 0x7f6db8004c80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400c3f0 tx=0x7f6dc4001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f851a88d922fe9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:59.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.954+0000 7f6dce7fc700 1 -- 172.21.15.89:0/174185796 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc400cc80 con 0x7f6db8004810 2023-10-10T20:34:59.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.954+0000 7f6dce7fc700 1 -- 172.21.15.89:0/174185796 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc400ce00 con 0x7f6db8004810 2023-10-10T20:34:59.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.954+0000 7f6dce7fc700 1 -- 172.21.15.89:0/174185796 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4005b70 con 0x7f6db8004810 2023-10-10T20:34:59.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.955+0000 7f6dd5a80700 1 -- 172.21.15.89:0/174185796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6db8004810 msgr2=0x7f6db8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:59.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.955+0000 7f6dd5a80700 1 --2- 172.21.15.89:0/174185796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6db8004810 0x7f6db8004c80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400c3f0 tx=0x7f6dc4001e90 comp rx=0 tx=0).stop 2023-10-10T20:34:59.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.955+0000 7f6dcffff700 1 -- 172.21.15.89:0/174185796 reap_dead start 2023-10-10T20:34:59.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.955+0000 7f6dd5a80700 1 -- 172.21.15.89:0/174185796 shutdown_connections 2023-10-10T20:34:59.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.955+0000 7f6dd5a80700 1 -- 172.21.15.89:0/174185796 >> 172.21.15.89:0/174185796 conn(0x7f6dd0068e20 msgr2=0x7f6dd00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:34:59.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.955+0000 7f6dd5a80700 1 -- 172.21.15.89:0/174185796 shutdown_connections 2023-10-10T20:34:59.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.955+0000 7f6dd5a80700 1 -- 172.21.15.89:0/174185796 wait complete. 2023-10-10T20:34:59.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.956+0000 7f6dd5a80700 1 Processor -- start 2023-10-10T20:34:59.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.956+0000 7f6dd5a80700 1 -- start start 2023-10-10T20:34:59.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6dd5a80700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6db8004810 0x7f6dd00cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6dd5a80700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6dd014b660 0x7f6dd00b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6dd5a80700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dd00cf370 0x7f6dd00cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6dd5a80700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd00c78a0 con 0x7f6dd00cf370 2023-10-10T20:34:59.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6dd5a80700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd00cfd20 con 0x7f6db8004810 2023-10-10T20:34:59.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6dd5a80700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd014bad0 con 0x7f6dd014b660 2023-10-10T20:34:59.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6dcffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dd00cf370 0x7f6dd00cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6dcffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dd00cf370 0x7f6dd00cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59270/0 (socket says 172.21.15.89:59270) 2023-10-10T20:34:59.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6dcffff700 1 -- 172.21.15.89:0/870558933 learned_addr learned my addr 172.21.15.89:0/870558933 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:34:59.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6dcf7fe700 1 --2- 172.21.15.89:0/870558933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6db8004810 0x7f6dd00cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6dceffd700 1 --2- 172.21.15.89:0/870558933 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6dd014b660 0x7f6dd00b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6dcffff700 1 -- 172.21.15.89:0/870558933 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6dd014b660 msgr2=0x7f6dd00b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:59.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6dcffff700 1 --2- 172.21.15.89:0/870558933 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6dd014b660 0x7f6dd00b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6dcffff700 1 -- 172.21.15.89:0/870558933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6db8004810 msgr2=0x7f6dd00cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:34:59.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6dcffff700 1 --2- 172.21.15.89:0/870558933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6db8004810 0x7f6dd00cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:34:59.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6dcffff700 1 -- 172.21.15.89:0/870558933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd00ce450 con 0x7f6dd00cf370 2023-10-10T20:34:59.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6dcffff700 1 --2- 172.21.15.89:0/870558933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dd00cf370 0x7f6dd00cf7e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00d03b0 tx=0x7f6dc800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:59.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6db77fe700 1 -- 172.21.15.89:0/870558933 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc800e040 con 0x7f6dd00cf370 2023-10-10T20:34:59.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6db77fe700 1 -- 172.21.15.89:0/870558933 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc800f040 con 0x7f6dd00cf370 2023-10-10T20:34:59.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6dd5a80700 1 -- 172.21.15.89:0/870558933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd00ce730 con 0x7f6dd00cf370 2023-10-10T20:34:59.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6db77fe700 1 -- 172.21.15.89:0/870558933 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc800fd70 con 0x7f6dd00cf370 2023-10-10T20:34:59.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.957+0000 7f6dd5a80700 1 -- 172.21.15.89:0/870558933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd00b0e30 con 0x7f6dd00cf370 2023-10-10T20:34:59.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.958+0000 7f6dd5a80700 1 -- 172.21.15.89:0/870558933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd004c3c0 con 0x7f6dd00cf370 2023-10-10T20:34:59.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.959+0000 7f6db77fe700 1 -- 172.21.15.89:0/870558933 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6dc8015640 con 0x7f6dd00cf370 2023-10-10T20:34:59.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.959+0000 7f6db77fe700 1 --2- 172.21.15.89:0/870558933 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6dbc066f30 0x7f6dbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:34:59.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.959+0000 7f6db77fe700 1 -- 172.21.15.89:0/870558933 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6dc8084bc0 con 0x7f6dd00cf370 2023-10-10T20:34:59.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.959+0000 7f6dcf7fe700 1 --2- 172.21.15.89:0/870558933 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6dbc066f30 0x7f6dbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:34:59.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.959+0000 7f6dcf7fe700 1 --2- 172.21.15.89:0/870558933 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6dbc066f30 0x7f6dbc0693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0006f00 tx=0x7f6dc00091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:34:59.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:34:59.963+0000 7f6db77fe700 1 -- 172.21.15.89:0/870558933 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6dc8051e30 con 0x7f6dd00cf370 2023-10-10T20:35:00.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.144+0000 7f6dd5a80700 1 -- 172.21.15.89:0/870558933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6dd006a5b0 con 0x7f6dd00cf370 2023-10-10T20:35:00.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.144+0000 7f6db77fe700 1 -- 172.21.15.89:0/870558933 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6dc8055450 con 0x7f6dd00cf370 2023-10-10T20:35:00.147 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:35:00.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.146+0000 7f6dd5a80700 1 -- 172.21.15.89:0/870558933 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6dbc066f30 msgr2=0x7f6dbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:00.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.146+0000 7f6dd5a80700 1 --2- 172.21.15.89:0/870558933 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6dbc066f30 0x7f6dbc0693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0006f00 tx=0x7f6dc00091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.146+0000 7f6dd5a80700 1 -- 172.21.15.89:0/870558933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dd00cf370 msgr2=0x7f6dd00cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:00.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.146+0000 7f6dd5a80700 1 --2- 172.21.15.89:0/870558933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dd00cf370 0x7f6dd00cf7e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00d03b0 tx=0x7f6dc800b040 comp rx=0 tx=0).stop 2023-10-10T20:35:00.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.146+0000 7f6dd5a80700 1 -- 172.21.15.89:0/870558933 shutdown_connections 2023-10-10T20:35:00.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.146+0000 7f6dd5a80700 1 --2- 172.21.15.89:0/870558933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dd00cf370 0x7f6dd00cf7e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.146+0000 7f6dd5a80700 1 --2- 172.21.15.89:0/870558933 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6dbc066f30 0x7f6dbc0693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.146+0000 7f6dd5a80700 1 --2- 172.21.15.89:0/870558933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6db8004810 0x7f6dd00cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.146+0000 7f6dd5a80700 1 --2- 172.21.15.89:0/870558933 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6dd014b660 0x7f6dd00b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.146+0000 7f6dd5a80700 1 -- 172.21.15.89:0/870558933 >> 172.21.15.89:0/870558933 conn(0x7f6dd0068e20 msgr2=0x7f6dd014d430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:00.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.146+0000 7f6dd5a80700 1 -- 172.21.15.89:0/870558933 shutdown_connections 2023-10-10T20:35:00.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.146+0000 7f6dd5a80700 1 -- 172.21.15.89:0/870558933 wait complete. 2023-10-10T20:35:00.161 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:35:00.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.276+0000 7f26f2135700 1 Processor -- start 2023-10-10T20:35:00.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.276+0000 7f26f2135700 1 -- start start 2023-10-10T20:35:00.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.277+0000 7f26f2135700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f26ec159890 0x7f26ec15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.277+0000 7f26f2135700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f26ec159c60 0x7f26ec15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.277+0000 7f26f2135700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26ec15db10 0x7f26ec069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.277+0000 7f26f2135700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ec15df50 con 0x7f26ec159c60 2023-10-10T20:35:00.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.277+0000 7f26f2135700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ec15a160 con 0x7f26ec159890 2023-10-10T20:35:00.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.277+0000 7f26f2135700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec0d13a0 con 0x7f26ec15db10 2023-10-10T20:35:00.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.277+0000 7f26eaffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f26ec159c60 0x7f26ec15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:00.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.277+0000 7f26eaffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f26ec159c60 0x7f26ec15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59272/0 (socket says 172.21.15.89:59272) 2023-10-10T20:35:00.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.277+0000 7f26eaffd700 1 -- 172.21.15.89:0/60411849 learned_addr learned my addr 172.21.15.89:0/60411849 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:00.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.277+0000 7f26eb7fe700 1 --2- 172.21.15.89:0/60411849 >> v2:172.21.15.111:3300/0 conn(0x7f26ec159890 0x7f26ec15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:00.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.277+0000 7f26ebfff700 1 --2- 172.21.15.89:0/60411849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26ec15db10 0x7f26ec069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:00.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.277+0000 7f26eaffd700 1 -- 172.21.15.89:0/60411849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26ec15db10 msgr2=0x7f26ec069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:00.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.277+0000 7f26eaffd700 1 --2- 172.21.15.89:0/60411849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26ec15db10 0x7f26ec069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.277+0000 7f26eaffd700 1 -- 172.21.15.89:0/60411849 >> v2:172.21.15.111:3300/0 conn(0x7f26ec159890 msgr2=0x7f26ec15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:00.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.277+0000 7f26eaffd700 1 --2- 172.21.15.89:0/60411849 >> v2:172.21.15.111:3300/0 conn(0x7f26ec159890 0x7f26ec15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.277+0000 7f26eaffd700 1 -- 172.21.15.89:0/60411849 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec15e140 con 0x7f26ec159c60 2023-10-10T20:35:00.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.278+0000 7f26eaffd700 1 --2- 172.21.15.89:0/60411849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ec159c60 0x7f26ec15d5d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0d1090 tx=0x7f26e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9a002a109b44dc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:00.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.278+0000 7f26ea7fc700 1 -- 172.21.15.89:0/60411849 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e000b860 con 0x7f26ec159c60 2023-10-10T20:35:00.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.278+0000 7f26ea7fc700 1 -- 172.21.15.89:0/60411849 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e000b9c0 con 0x7f26ec159c60 2023-10-10T20:35:00.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.278+0000 7f26ea7fc700 1 -- 172.21.15.89:0/60411849 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e000bd60 con 0x7f26ec159c60 2023-10-10T20:35:00.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.278+0000 7f26f2135700 1 -- 172.21.15.89:0/60411849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ec159c60 msgr2=0x7f26ec15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:00.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.278+0000 7f26f2135700 1 --2- 172.21.15.89:0/60411849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ec159c60 0x7f26ec15d5d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0d1090 tx=0x7f26e0009250 comp rx=0 tx=0).stop 2023-10-10T20:35:00.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.278+0000 7f26f2135700 1 -- 172.21.15.89:0/60411849 shutdown_connections 2023-10-10T20:35:00.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.278+0000 7f26f2135700 1 --2- 172.21.15.89:0/60411849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26ec15db10 0x7f26ec069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.278+0000 7f26f2135700 1 --2- 172.21.15.89:0/60411849 >> v2:172.21.15.111:3300/0 conn(0x7f26ec159890 0x7f26ec15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.278+0000 7f26f2135700 1 --2- 172.21.15.89:0/60411849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ec159c60 0x7f26ec15d5d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.278+0000 7f26f2135700 1 -- 172.21.15.89:0/60411849 >> 172.21.15.89:0/60411849 conn(0x7f26ec068e20 msgr2=0x7f26ec0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:00.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.278+0000 7f26f2135700 1 -- 172.21.15.89:0/60411849 shutdown_connections 2023-10-10T20:35:00.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.278+0000 7f26f2135700 1 -- 172.21.15.89:0/60411849 wait complete. 2023-10-10T20:35:00.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.280+0000 7f26f2135700 1 Processor -- start 2023-10-10T20:35:00.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.280+0000 7f26f2135700 1 -- start start 2023-10-10T20:35:00.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.280+0000 7f26f2135700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26ec159890 0x7f26ec152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.280+0000 7f26f2135700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ec159c60 0x7f26ec0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.280+0000 7f26f2135700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26ec15db10 0x7f26ec0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.280+0000 7f26f2135700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec0cdb20 con 0x7f26ec159c60 2023-10-10T20:35:00.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.280+0000 7f26f2135700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec152aa0 con 0x7f26ec15db10 2023-10-10T20:35:00.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.280+0000 7f26f2135700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec0b6470 con 0x7f26ec159890 2023-10-10T20:35:00.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.280+0000 7f26eaffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ec159c60 0x7f26ec0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:00.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.281+0000 7f26eaffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ec159c60 0x7f26ec0cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59284/0 (socket says 172.21.15.89:59284) 2023-10-10T20:35:00.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.281+0000 7f26eaffd700 1 -- 172.21.15.89:0/1226253420 learned_addr learned my addr 172.21.15.89:0/1226253420 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:00.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.281+0000 7f26eaffd700 1 -- 172.21.15.89:0/1226253420 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26ec159890 msgr2=0x7f26ec152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:00.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.281+0000 7f26eaffd700 1 --2- 172.21.15.89:0/1226253420 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26ec159890 0x7f26ec152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.281+0000 7f26eaffd700 1 -- 172.21.15.89:0/1226253420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26ec15db10 msgr2=0x7f26ec0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:00.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.281+0000 7f26eaffd700 1 --2- 172.21.15.89:0/1226253420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26ec15db10 0x7f26ec0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.281+0000 7f26eaffd700 1 -- 172.21.15.89:0/1226253420 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e00094f0 con 0x7f26ec159c60 2023-10-10T20:35:00.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.281+0000 7f26eaffd700 1 --2- 172.21.15.89:0/1226253420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ec159c60 0x7f26ec0cdfb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f26ec070e20 tx=0x7f26e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:00.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.281+0000 7f26d37fe700 1 -- 172.21.15.89:0/1226253420 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e0024440 con 0x7f26ec159c60 2023-10-10T20:35:00.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.281+0000 7f26d37fe700 1 -- 172.21.15.89:0/1226253420 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e00245a0 con 0x7f26ec159c60 2023-10-10T20:35:00.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.281+0000 7f26f2135700 1 -- 172.21.15.89:0/1226253420 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ec0cd230 con 0x7f26ec159c60 2023-10-10T20:35:00.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.281+0000 7f26d37fe700 1 -- 172.21.15.89:0/1226253420 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e0024960 con 0x7f26ec159c60 2023-10-10T20:35:00.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.281+0000 7f26f2135700 1 -- 172.21.15.89:0/1226253420 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ec0cd7b0 con 0x7f26ec159c60 2023-10-10T20:35:00.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.282+0000 7f26f2135700 1 -- 172.21.15.89:0/1226253420 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ec04c3c0 con 0x7f26ec159c60 2023-10-10T20:35:00.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.282+0000 7f26d37fe700 1 -- 172.21.15.89:0/1226253420 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f26e001a420 con 0x7f26ec159c60 2023-10-10T20:35:00.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.283+0000 7f26d37fe700 1 --2- 172.21.15.89:0/1226253420 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f26d8066f30 0x7f26d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.283+0000 7f26d37fe700 1 -- 172.21.15.89:0/1226253420 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f26e0086440 con 0x7f26ec159c60 2023-10-10T20:35:00.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.283+0000 7f26eb7fe700 1 --2- 172.21.15.89:0/1226253420 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f26d8066f30 0x7f26d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:00.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.283+0000 7f26eb7fe700 1 --2- 172.21.15.89:0/1226253420 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f26d8066f30 0x7f26d80693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009a40 tx=0x7f26dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:00.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.287+0000 7f26d37fe700 1 -- 172.21.15.89:0/1226253420 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26e00536b0 con 0x7f26ec159c60 2023-10-10T20:35:00.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.470+0000 7f26f2135700 1 -- 172.21.15.89:0/1226253420 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f26ec0cdc60 con 0x7f26ec159c60 2023-10-10T20:35:00.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.471+0000 7f26d37fe700 1 -- 172.21.15.89:0/1226253420 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f26e0056cd0 con 0x7f26ec159c60 2023-10-10T20:35:00.473 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-10T20:35:00.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.473+0000 7f26f2135700 1 -- 172.21.15.89:0/1226253420 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f26d8066f30 msgr2=0x7f26d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:00.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.473+0000 7f26f2135700 1 --2- 172.21.15.89:0/1226253420 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f26d8066f30 0x7f26d80693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009a40 tx=0x7f26dc008040 comp rx=0 tx=0).stop 2023-10-10T20:35:00.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.473+0000 7f26f2135700 1 -- 172.21.15.89:0/1226253420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ec159c60 msgr2=0x7f26ec0cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:00.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.473+0000 7f26f2135700 1 --2- 172.21.15.89:0/1226253420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ec159c60 0x7f26ec0cdfb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f26ec070e20 tx=0x7f26e0017000 comp rx=0 tx=0).stop 2023-10-10T20:35:00.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.473+0000 7f26f2135700 1 -- 172.21.15.89:0/1226253420 shutdown_connections 2023-10-10T20:35:00.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.473+0000 7f26f2135700 1 --2- 172.21.15.89:0/1226253420 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f26d8066f30 0x7f26d80693e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.473+0000 7f26f2135700 1 --2- 172.21.15.89:0/1226253420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26ec15db10 0x7f26ec0b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.473+0000 7f26f2135700 1 --2- 172.21.15.89:0/1226253420 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26ec159890 0x7f26ec152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.473+0000 7f26f2135700 1 --2- 172.21.15.89:0/1226253420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ec159c60 0x7f26ec0cdfb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.473+0000 7f26f2135700 1 -- 172.21.15.89:0/1226253420 >> 172.21.15.89:0/1226253420 conn(0x7f26ec068e20 msgr2=0x7f26ec1532f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:00.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.473+0000 7f26f2135700 1 -- 172.21.15.89:0/1226253420 shutdown_connections 2023-10-10T20:35:00.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.473+0000 7f26f2135700 1 -- 172.21.15.89:0/1226253420 wait complete. 2023-10-10T20:35:00.487 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:35:00.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.599+0000 7f18e950c700 1 Processor -- start 2023-10-10T20:35:00.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.599+0000 7f18e950c700 1 -- start start 2023-10-10T20:35:00.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.600+0000 7f18e950c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f18e406bcd0 0x7f18e4140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.600+0000 7f18e950c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f18e406c0a0 0x7f18e4141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.600+0000 7f18e950c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f18e40c8e00 0x7f18e40c9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.600+0000 7f18e950c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e415c2a0 con 0x7f18e406bcd0 2023-10-10T20:35:00.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.600+0000 7f18e950c700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e415b690 con 0x7f18e406c0a0 2023-10-10T20:35:00.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.600+0000 7f18e950c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e415b970 con 0x7f18e40c8e00 2023-10-10T20:35:00.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.600+0000 7f18e2ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f18e406bcd0 0x7f18e4140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:00.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.600+0000 7f18e2ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f18e406bcd0 0x7f18e4140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59298/0 (socket says 172.21.15.89:59298) 2023-10-10T20:35:00.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.600+0000 7f18e2ffd700 1 -- 172.21.15.89:0/769374699 learned_addr learned my addr 172.21.15.89:0/769374699 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:00.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.600+0000 7f18da7fc700 1 --2- 172.21.15.89:0/769374699 >> v2:172.21.15.111:3300/0 conn(0x7f18e406c0a0 0x7f18e4141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:00.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.600+0000 7f18e37fe700 1 --2- 172.21.15.89:0/769374699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f18e40c8e00 0x7f18e40c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:00.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.601+0000 7f18e2ffd700 1 -- 172.21.15.89:0/769374699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f18e40c8e00 msgr2=0x7f18e40c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:00.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.601+0000 7f18e2ffd700 1 --2- 172.21.15.89:0/769374699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f18e40c8e00 0x7f18e40c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.601+0000 7f18e2ffd700 1 -- 172.21.15.89:0/769374699 >> v2:172.21.15.111:3300/0 conn(0x7f18e406c0a0 msgr2=0x7f18e4141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:00.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.601+0000 7f18e2ffd700 1 --2- 172.21.15.89:0/769374699 >> v2:172.21.15.111:3300/0 conn(0x7f18e406c0a0 0x7f18e4141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.601+0000 7f18e2ffd700 1 -- 172.21.15.89:0/769374699 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e415bc50 con 0x7f18e406bcd0 2023-10-10T20:35:00.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.601+0000 7f18da7fc700 1 --2- 172.21.15.89:0/769374699 >> v2:172.21.15.111:3300/0 conn(0x7f18e406c0a0 0x7f18e4141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:00.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.601+0000 7f18e2ffd700 1 --2- 172.21.15.89:0/769374699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18e406bcd0 0x7f18e4140df0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f18e406c6e0 tx=0x7f18d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5c07f9c0b1030c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:00.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.601+0000 7f18e27fc700 1 -- 172.21.15.89:0/769374699 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d400b860 con 0x7f18e406bcd0 2023-10-10T20:35:00.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.601+0000 7f18e27fc700 1 -- 172.21.15.89:0/769374699 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d400b9c0 con 0x7f18e406bcd0 2023-10-10T20:35:00.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.601+0000 7f18e27fc700 1 -- 172.21.15.89:0/769374699 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d400bd60 con 0x7f18e406bcd0 2023-10-10T20:35:00.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.601+0000 7f18e950c700 1 -- 172.21.15.89:0/769374699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18e406bcd0 msgr2=0x7f18e4140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:00.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.601+0000 7f18e950c700 1 --2- 172.21.15.89:0/769374699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18e406bcd0 0x7f18e4140df0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f18e406c6e0 tx=0x7f18d4009250 comp rx=0 tx=0).stop 2023-10-10T20:35:00.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.601+0000 7f18e950c700 1 -- 172.21.15.89:0/769374699 shutdown_connections 2023-10-10T20:35:00.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.601+0000 7f18e950c700 1 --2- 172.21.15.89:0/769374699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f18e40c8e00 0x7f18e40c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.601+0000 7f18e950c700 1 --2- 172.21.15.89:0/769374699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18e406bcd0 0x7f18e4140df0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.601+0000 7f18e950c700 1 --2- 172.21.15.89:0/769374699 >> v2:172.21.15.111:3300/0 conn(0x7f18e406c0a0 0x7f18e4141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.601+0000 7f18e950c700 1 -- 172.21.15.89:0/769374699 >> 172.21.15.89:0/769374699 conn(0x7f18e40ce600 msgr2=0x7f18e40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:00.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.601+0000 7f18e950c700 1 -- 172.21.15.89:0/769374699 shutdown_connections 2023-10-10T20:35:00.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.601+0000 7f18e950c700 1 -- 172.21.15.89:0/769374699 wait complete. 2023-10-10T20:35:00.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.603+0000 7f18e950c700 1 Processor -- start 2023-10-10T20:35:00.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.603+0000 7f18e950c700 1 -- start start 2023-10-10T20:35:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.603+0000 7f18e950c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f18e406bcd0 0x7f18e413c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.603+0000 7f18e950c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18e406c0a0 0x7f18e416b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.603+0000 7f18e950c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18e40c8e00 0x7f18e40c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.603+0000 7f18e950c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e40c58d0 con 0x7f18e406c0a0 2023-10-10T20:35:00.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.603+0000 7f18e950c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e40c7a40 con 0x7f18e40c8e00 2023-10-10T20:35:00.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.603+0000 7f18e950c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e40c6e30 con 0x7f18e406bcd0 2023-10-10T20:35:00.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.604+0000 7f18da7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18e406c0a0 0x7f18e416b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:00.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.604+0000 7f18da7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18e406c0a0 0x7f18e416b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59314/0 (socket says 172.21.15.89:59314) 2023-10-10T20:35:00.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.604+0000 7f18da7fc700 1 -- 172.21.15.89:0/3898522633 learned_addr learned my addr 172.21.15.89:0/3898522633 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:00.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.604+0000 7f18e37fe700 1 --2- 172.21.15.89:0/3898522633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18e40c8e00 0x7f18e40c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.604+0000 7f18e2ffd700 1 --2- 172.21.15.89:0/3898522633 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f18e406bcd0 0x7f18e413c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.604+0000 7f18da7fc700 1 -- 172.21.15.89:0/3898522633 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f18e406bcd0 msgr2=0x7f18e413c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.604+0000 7f18da7fc700 1 --2- 172.21.15.89:0/3898522633 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f18e406bcd0 0x7f18e413c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.604+0000 7f18da7fc700 1 -- 172.21.15.89:0/3898522633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18e40c8e00 msgr2=0x7f18e40c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:00.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.604+0000 7f18da7fc700 1 --2- 172.21.15.89:0/3898522633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18e40c8e00 0x7f18e40c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.604+0000 7f18da7fc700 1 -- 172.21.15.89:0/3898522633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d40094f0 con 0x7f18e406c0a0 2023-10-10T20:35:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.604+0000 7f18e37fe700 1 --2- 172.21.15.89:0/3898522633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18e40c8e00 0x7f18e40c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.604+0000 7f18da7fc700 1 --2- 172.21.15.89:0/3898522633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18e406c0a0 0x7f18e416b4f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00e8d0 tx=0x7f18cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.604+0000 7f18db7fe700 1 -- 172.21.15.89:0/3898522633 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc00ee70 con 0x7f18e406c0a0 2023-10-10T20:35:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.604+0000 7f18db7fe700 1 -- 172.21.15.89:0/3898522633 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18cc00c5e0 con 0x7f18e406c0a0 2023-10-10T20:35:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.604+0000 7f18e950c700 1 -- 172.21.15.89:0/3898522633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e40c7110 con 0x7f18e406c0a0 2023-10-10T20:35:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.604+0000 7f18db7fe700 1 -- 172.21.15.89:0/3898522633 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc00c980 con 0x7f18e406c0a0 2023-10-10T20:35:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.604+0000 7f18e950c700 1 -- 172.21.15.89:0/3898522633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e40c76d0 con 0x7f18e406c0a0 2023-10-10T20:35:00.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.605+0000 7f18e950c700 1 -- 172.21.15.89:0/3898522633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e404c3c0 con 0x7f18e406c0a0 2023-10-10T20:35:00.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.605+0000 7f18db7fe700 1 -- 172.21.15.89:0/3898522633 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f18cc00cd30 con 0x7f18e406c0a0 2023-10-10T20:35:00.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.606+0000 7f18db7fe700 1 --2- 172.21.15.89:0/3898522633 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f18c4066f30 0x7f18c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.606+0000 7f18db7fe700 1 -- 172.21.15.89:0/3898522633 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f18cc0843e0 con 0x7f18e406c0a0 2023-10-10T20:35:00.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.606+0000 7f18e2ffd700 1 --2- 172.21.15.89:0/3898522633 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f18c4066f30 0x7f18c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:00.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.606+0000 7f18e2ffd700 1 --2- 172.21.15.89:0/3898522633 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f18c4066f30 0x7f18c40693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f18e406c6e0 tx=0x7f18d4017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:00.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.610+0000 7f18db7fe700 1 -- 172.21.15.89:0/3898522633 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18cc051650 con 0x7f18e406c0a0 2023-10-10T20:35:00.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.795+0000 7f18e950c700 1 -- 172.21.15.89:0/3898522633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18e40cf510 con 0x7f18e406c0a0 2023-10-10T20:35:00.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.796+0000 7f18db7fe700 1 -- 172.21.15.89:0/3898522633 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f18cc054c70 con 0x7f18e406c0a0 2023-10-10T20:35:00.798 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:00.798 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:36.324920+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:34:32.342167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:34:33.671763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:35:00.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.797+0000 7f18e950c700 1 -- 172.21.15.89:0/3898522633 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f18c4066f30 msgr2=0x7f18c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:00.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.798+0000 7f18e950c700 1 --2- 172.21.15.89:0/3898522633 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f18c4066f30 0x7f18c40693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f18e406c6e0 tx=0x7f18d4017000 comp rx=0 tx=0).stop 2023-10-10T20:35:00.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.798+0000 7f18e950c700 1 -- 172.21.15.89:0/3898522633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18e406c0a0 msgr2=0x7f18e416b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:00.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.798+0000 7f18e950c700 1 --2- 172.21.15.89:0/3898522633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18e406c0a0 0x7f18e416b4f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00e8d0 tx=0x7f18cc008040 comp rx=0 tx=0).stop 2023-10-10T20:35:00.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.798+0000 7f18e950c700 1 -- 172.21.15.89:0/3898522633 shutdown_connections 2023-10-10T20:35:00.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.798+0000 7f18e950c700 1 --2- 172.21.15.89:0/3898522633 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f18c4066f30 0x7f18c40693e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.798+0000 7f18e950c700 1 --2- 172.21.15.89:0/3898522633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18e40c8e00 0x7f18e40c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.798+0000 7f18e950c700 1 --2- 172.21.15.89:0/3898522633 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f18e406bcd0 0x7f18e413c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.798+0000 7f18e950c700 1 --2- 172.21.15.89:0/3898522633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18e406c0a0 0x7f18e416b4f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.798+0000 7f18e950c700 1 -- 172.21.15.89:0/3898522633 >> 172.21.15.89:0/3898522633 conn(0x7f18e40ce600 msgr2=0x7f18e4155fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.798+0000 7f18e950c700 1 -- 172.21.15.89:0/3898522633 shutdown_connections 2023-10-10T20:35:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.798+0000 7f18e950c700 1 -- 172.21.15.89:0/3898522633 wait complete. 2023-10-10T20:35:00.812 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:34:32.342167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:34:33.671763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:35:00.813 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:35:00.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.929+0000 7f22e97c2700 1 Processor -- start 2023-10-10T20:35:00.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.929+0000 7f22e97c2700 1 -- start start 2023-10-10T20:35:00.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.930+0000 7f22e97c2700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f22e40b16d0 0x7f22e40b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.930+0000 7f22e97c2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f22e40b33b0 0x7f22e40b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.930+0000 7f22e97c2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e40b2080 con 0x7f22e40b4e70 2023-10-10T20:35:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.930+0000 7f22e97c2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e40cfb00 con 0x7f22e40b33b0 2023-10-10T20:35:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.930+0000 7f22e97c2700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40b5c40 con 0x7f22e40b16d0 2023-10-10T20:35:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.930+0000 7f22e3fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f22e40b4e70 0x7f22e40ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58136/0 (socket says 172.21.15.89:58136) 2023-10-10T20:35:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.930+0000 7f22e3fff700 1 -- 172.21.15.89:0/2079881258 learned_addr learned my addr 172.21.15.89:0/2079881258 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.930+0000 7f22e8fc1700 1 --2- 172.21.15.89:0/2079881258 >> v2:172.21.15.89:3300/0 conn(0x7f22e40b33b0 0x7f22e40b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.930+0000 7f22e2ffd700 1 -- 172.21.15.89:0/2079881258 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3069781655 0 0) 0x7f22e40b2080 con 0x7f22e40b4e70 2023-10-10T20:35:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.930+0000 7f22e2ffd700 1 -- 172.21.15.89:0/2079881258 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22cc003570 con 0x7f22e40b4e70 2023-10-10T20:35:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.930+0000 7f22e37fe700 1 --2- 172.21.15.89:0/2079881258 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f22e40b16d0 0x7f22e40b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.930+0000 7f22e8fc1700 1 -- 172.21.15.89:0/2079881258 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f22e40b16d0 msgr2=0x7f22e40b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.930+0000 7f22e8fc1700 1 --2- 172.21.15.89:0/2079881258 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f22e40b16d0 0x7f22e40b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.930+0000 7f22e8fc1700 1 -- 172.21.15.89:0/2079881258 >> v1:172.21.15.89:6789/0 conn(0x7f22e40b4e70 legacy=0x7f22e40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:00.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.930+0000 7f22e8fc1700 1 -- 172.21.15.89:0/2079881258 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e40b5f20 con 0x7f22e40b33b0 2023-10-10T20:35:00.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.931+0000 7f22e8fc1700 1 --2- 172.21.15.89:0/2079881258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22e40b33b0 0x7f22e40b3830 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009e60 tx=0x7f22dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d5470a8702493a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:00.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.931+0000 7f22e2ffd700 1 -- 172.21.15.89:0/2079881258 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00e070 con 0x7f22e40b33b0 2023-10-10T20:35:00.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.931+0000 7f22e2ffd700 1 -- 172.21.15.89:0/2079881258 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc00bb60 con 0x7f22e40b33b0 2023-10-10T20:35:00.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.931+0000 7f22e2ffd700 1 -- 172.21.15.89:0/2079881258 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc012430 con 0x7f22e40b33b0 2023-10-10T20:35:00.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.931+0000 7f22e97c2700 1 -- 172.21.15.89:0/2079881258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22e40b33b0 msgr2=0x7f22e40b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:00.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.931+0000 7f22e97c2700 1 --2- 172.21.15.89:0/2079881258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22e40b33b0 0x7f22e40b3830 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009e60 tx=0x7f22dc009780 comp rx=0 tx=0).stop 2023-10-10T20:35:00.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.931+0000 7f22e97c2700 1 -- 172.21.15.89:0/2079881258 shutdown_connections 2023-10-10T20:35:00.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.931+0000 7f22e97c2700 1 --2- 172.21.15.89:0/2079881258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22e40b33b0 0x7f22e40b3830 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.931+0000 7f22e97c2700 1 --2- 172.21.15.89:0/2079881258 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f22e40b16d0 0x7f22e40b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.931+0000 7f22e97c2700 1 -- 172.21.15.89:0/2079881258 >> 172.21.15.89:0/2079881258 conn(0x7f22e40d10c0 msgr2=0x7f22e40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.931+0000 7f22e97c2700 1 -- 172.21.15.89:0/2079881258 shutdown_connections 2023-10-10T20:35:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.931+0000 7f22e97c2700 1 -- 172.21.15.89:0/2079881258 wait complete. 2023-10-10T20:35:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.932+0000 7f22e97c2700 1 Processor -- start 2023-10-10T20:35:00.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.933+0000 7f22e97c2700 1 -- start start 2023-10-10T20:35:00.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.933+0000 7f22e97c2700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f22e40b16d0 0x7f22e40c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.933+0000 7f22e97c2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22e40b33b0 0x7f22e40ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.933+0000 7f22e97c2700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e40b4e70 0x7f22e40c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.933+0000 7f22e97c2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40c8220 con 0x7f22e40b33b0 2023-10-10T20:35:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.933+0000 7f22e97c2700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40c8500 con 0x7f22e40b4e70 2023-10-10T20:35:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.933+0000 7f22e97c2700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40c87e0 con 0x7f22e40b16d0 2023-10-10T20:35:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.933+0000 7f22e37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22e40b33b0 0x7f22e40ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.933+0000 7f22e37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22e40b33b0 0x7f22e40ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59336/0 (socket says 172.21.15.89:59336) 2023-10-10T20:35:00.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.933+0000 7f22e37fe700 1 -- 172.21.15.89:0/3890564054 learned_addr learned my addr 172.21.15.89:0/3890564054 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:00.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.934+0000 7f22e37fe700 1 -- 172.21.15.89:0/3890564054 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f22e40b16d0 msgr2=0x7f22e40c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:00.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.934+0000 7f22e37fe700 1 --2- 172.21.15.89:0/3890564054 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f22e40b16d0 0x7f22e40c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.934+0000 7f22e37fe700 1 -- 172.21.15.89:0/3890564054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e40b4e70 msgr2=0x7f22e40c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:00.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.934+0000 7f22e37fe700 1 --2- 172.21.15.89:0/3890564054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e40b4e70 0x7f22e40c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:00.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.934+0000 7f22e37fe700 1 -- 172.21.15.89:0/3890564054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc009a20 con 0x7f22e40b33b0 2023-10-10T20:35:00.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.934+0000 7f22e37fe700 1 --2- 172.21.15.89:0/3890564054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22e40b33b0 0x7f22e40ca140 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f22d800c7d0 tx=0x7f22d8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:00.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.934+0000 7f22cbfff700 1 -- 172.21.15.89:0/3890564054 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8011600 con 0x7f22e40b33b0 2023-10-10T20:35:00.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.934+0000 7f22cbfff700 1 -- 172.21.15.89:0/3890564054 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d8011760 con 0x7f22e40b33b0 2023-10-10T20:35:00.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.934+0000 7f22e97c2700 1 -- 172.21.15.89:0/3890564054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e40c8ac0 con 0x7f22e40b33b0 2023-10-10T20:35:00.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.934+0000 7f22cbfff700 1 -- 172.21.15.89:0/3890564054 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8011b20 con 0x7f22e40b33b0 2023-10-10T20:35:00.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.934+0000 7f22e97c2700 1 -- 172.21.15.89:0/3890564054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e40ca680 con 0x7f22e40b33b0 2023-10-10T20:35:00.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.935+0000 7f22e97c2700 1 -- 172.21.15.89:0/3890564054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e40c8fe0 con 0x7f22e40b33b0 2023-10-10T20:35:00.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.935+0000 7f22cbfff700 1 -- 172.21.15.89:0/3890564054 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f22d80193f0 con 0x7f22e40b33b0 2023-10-10T20:35:00.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.936+0000 7f22cbfff700 1 --2- 172.21.15.89:0/3890564054 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f22d0066f30 0x7f22d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:00.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.936+0000 7f22cbfff700 1 -- 172.21.15.89:0/3890564054 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f22d8084940 con 0x7f22e40b33b0 2023-10-10T20:35:00.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.936+0000 7f22e3fff700 1 --2- 172.21.15.89:0/3890564054 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f22d0066f30 0x7f22d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:00.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.936+0000 7f22e3fff700 1 --2- 172.21.15.89:0/3890564054 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f22d0066f30 0x7f22d00693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f22d40058d0 tx=0x7f22d40079d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:00.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:00.940+0000 7f22cbfff700 1 -- 172.21.15.89:0/3890564054 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22e40c8fe0 con 0x7f22e40b33b0 2023-10-10T20:35:01.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.123+0000 7f22e97c2700 1 -- 172.21.15.89:0/3890564054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f22e40c8fe0 con 0x7f22e40b33b0 2023-10-10T20:35:01.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.124+0000 7f22cbfff700 1 -- 172.21.15.89:0/3890564054 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f22e40c8fe0 con 0x7f22e40b33b0 2023-10-10T20:35:01.126 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:35:01.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.126+0000 7f22e97c2700 1 -- 172.21.15.89:0/3890564054 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f22d0066f30 msgr2=0x7f22d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:01.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.126+0000 7f22e97c2700 1 --2- 172.21.15.89:0/3890564054 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f22d0066f30 0x7f22d00693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f22d40058d0 tx=0x7f22d40079d0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.126+0000 7f22e97c2700 1 -- 172.21.15.89:0/3890564054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22e40b33b0 msgr2=0x7f22e40ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:01.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.126+0000 7f22e97c2700 1 --2- 172.21.15.89:0/3890564054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22e40b33b0 0x7f22e40ca140 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f22d800c7d0 tx=0x7f22d8008040 comp rx=0 tx=0).stop 2023-10-10T20:35:01.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.126+0000 7f22e97c2700 1 -- 172.21.15.89:0/3890564054 shutdown_connections 2023-10-10T20:35:01.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.126+0000 7f22e97c2700 1 --2- 172.21.15.89:0/3890564054 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f22d0066f30 0x7f22d00693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.126+0000 7f22e97c2700 1 --2- 172.21.15.89:0/3890564054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e40b4e70 0x7f22e40c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.126+0000 7f22e97c2700 1 --2- 172.21.15.89:0/3890564054 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f22e40b16d0 0x7f22e40c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.126+0000 7f22e97c2700 1 --2- 172.21.15.89:0/3890564054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22e40b33b0 0x7f22e40ca140 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.126+0000 7f22e97c2700 1 -- 172.21.15.89:0/3890564054 >> 172.21.15.89:0/3890564054 conn(0x7f22e40d10c0 msgr2=0x7f22e415f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:01.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.126+0000 7f22e97c2700 1 -- 172.21.15.89:0/3890564054 shutdown_connections 2023-10-10T20:35:01.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.126+0000 7f22e97c2700 1 -- 172.21.15.89:0/3890564054 wait complete. 2023-10-10T20:35:01.140 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:35:01.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.255+0000 7f060b02e700 1 Processor -- start 2023-10-10T20:35:01.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.256+0000 7f060b02e700 1 -- start start 2023-10-10T20:35:01.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.256+0000 7f060b02e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f060406bcd0 0x7f06040caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.256+0000 7f060b02e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f060406c0a0 0x7f060415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.256+0000 7f060b02e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060415d7e0 con 0x7f060415df50 2023-10-10T20:35:01.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.256+0000 7f060b02e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f060415dac0 con 0x7f060406bcd0 2023-10-10T20:35:01.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.256+0000 7f060b02e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060415aa30 con 0x7f060406c0a0 2023-10-10T20:35:01.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.256+0000 7f06095cb700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f060415df50 0x7f06040c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58152/0 (socket says 172.21.15.89:58152) 2023-10-10T20:35:01.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.256+0000 7f06095cb700 1 -- 172.21.15.89:0/1009812029 learned_addr learned my addr 172.21.15.89:0/1009812029 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:01.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f0608dca700 1 --2- 172.21.15.89:0/1009812029 >> v2:172.21.15.111:3300/0 conn(0x7f060406bcd0 0x7f06040caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:01.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f0603fff700 1 --2- 172.21.15.89:0/1009812029 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f060406c0a0 0x7f060415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:01.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f06037fe700 1 -- 172.21.15.89:0/1009812029 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2138721208 0 0) 0x7f060415d7e0 con 0x7f060415df50 2023-10-10T20:35:01.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f06037fe700 1 -- 172.21.15.89:0/1009812029 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05ec003570 con 0x7f060415df50 2023-10-10T20:35:01.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f06037fe700 1 -- 172.21.15.89:0/1009812029 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3630347304 0 0) 0x7f05ec003570 con 0x7f060415df50 2023-10-10T20:35:01.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f06037fe700 1 -- 172.21.15.89:0/1009812029 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f060406c0a0 msgr2=0x7f060415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:01.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f06037fe700 1 --2- 172.21.15.89:0/1009812029 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f060406c0a0 0x7f060415a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f06037fe700 1 -- 172.21.15.89:0/1009812029 >> v2:172.21.15.111:3300/0 conn(0x7f060406bcd0 msgr2=0x7f06040caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:01.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f06037fe700 1 --2- 172.21.15.89:0/1009812029 >> v2:172.21.15.111:3300/0 conn(0x7f060406bcd0 0x7f06040caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f06037fe700 1 -- 172.21.15.89:0/1009812029 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060415ad10 con 0x7f060415df50 2023-10-10T20:35:01.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f06037fe700 1 -- 172.21.15.89:0/1009812029 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f05fc001470 con 0x7f060415df50 2023-10-10T20:35:01.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f06037fe700 1 -- 172.21.15.89:0/1009812029 >> v1:172.21.15.89:6789/0 conn(0x7f060415df50 legacy=0x7f06040c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:01.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f06037fe700 1 --2- 172.21.15.89:0/1009812029 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f05ec003a60 0x7f05ec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f06037fe700 1 --2- 172.21.15.89:0/1009812029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05ec004810 0x7f05ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f06037fe700 1 --2- 172.21.15.89:0/1009812029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05ec0051c0 0x7f05ec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f06037fe700 1 -- 172.21.15.89:0/1009812029 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060415dac0 con 0x7f05ec0051c0 2023-10-10T20:35:01.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f06037fe700 1 -- 172.21.15.89:0/1009812029 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060415aa30 con 0x7f05ec004810 2023-10-10T20:35:01.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f06037fe700 1 -- 172.21.15.89:0/1009812029 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec00c020 con 0x7f05ec003a60 2023-10-10T20:35:01.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f0608dca700 1 --2- 172.21.15.89:0/1009812029 >> v2:172.21.15.111:3300/0 conn(0x7f060406bcd0 0x7f06040caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:35:01.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f0603fff700 1 --2- 172.21.15.89:0/1009812029 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f060406c0a0 0x7f060415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:35:01.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f06095cb700 1 --2- 172.21.15.89:0/1009812029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05ec0051c0 0x7f05ec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:01.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f06095cb700 1 -- 172.21.15.89:0/1009812029 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f05ec003a60 msgr2=0x7f05ec003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:01.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.257+0000 7f0603fff700 1 --2- 172.21.15.89:0/1009812029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05ec004810 0x7f05ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:01.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.258+0000 7f0608dca700 1 --2- 172.21.15.89:0/1009812029 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f05ec003a60 0x7f05ec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:01.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.258+0000 7f06095cb700 1 --2- 172.21.15.89:0/1009812029 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f05ec003a60 0x7f05ec003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.258+0000 7f06095cb700 1 -- 172.21.15.89:0/1009812029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05ec004810 msgr2=0x7f05ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:01.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.258+0000 7f06095cb700 1 --2- 172.21.15.89:0/1009812029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05ec004810 0x7f05ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.258+0000 7f06095cb700 1 -- 172.21.15.89:0/1009812029 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec003570 con 0x7f05ec0051c0 2023-10-10T20:35:01.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.258+0000 7f0603fff700 1 --2- 172.21.15.89:0/1009812029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05ec004810 0x7f05ec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:01.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.258+0000 7f0608dca700 1 --2- 172.21.15.89:0/1009812029 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f05ec003a60 0x7f05ec003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:01.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.258+0000 7f06095cb700 1 --2- 172.21.15.89:0/1009812029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05ec0051c0 0x7f05ec00ba60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00c3c0 tx=0x7f05fc001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c083cabb96d9e2a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:01.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.258+0000 7f06037fe700 1 -- 172.21.15.89:0/1009812029 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc00cc90 con 0x7f05ec0051c0 2023-10-10T20:35:01.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.258+0000 7f06037fe700 1 -- 172.21.15.89:0/1009812029 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc00ce30 con 0x7f05ec0051c0 2023-10-10T20:35:01.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.258+0000 7f06037fe700 1 -- 172.21.15.89:0/1009812029 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc005b70 con 0x7f05ec0051c0 2023-10-10T20:35:01.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.258+0000 7f060b02e700 1 -- 172.21.15.89:0/1009812029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05ec0051c0 msgr2=0x7f05ec00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:01.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.258+0000 7f060b02e700 1 --2- 172.21.15.89:0/1009812029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05ec0051c0 0x7f05ec00ba60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00c3c0 tx=0x7f05fc001e90 comp rx=0 tx=0).stop 2023-10-10T20:35:01.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.258+0000 7f06095cb700 1 -- 172.21.15.89:0/1009812029 reap_dead start 2023-10-10T20:35:01.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.259+0000 7f060b02e700 1 -- 172.21.15.89:0/1009812029 shutdown_connections 2023-10-10T20:35:01.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.259+0000 7f060b02e700 1 -- 172.21.15.89:0/1009812029 >> 172.21.15.89:0/1009812029 conn(0x7f06040ce600 msgr2=0x7f06040cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:01.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.259+0000 7f060b02e700 1 -- 172.21.15.89:0/1009812029 shutdown_connections 2023-10-10T20:35:01.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.259+0000 7f060b02e700 1 -- 172.21.15.89:0/1009812029 wait complete. 2023-10-10T20:35:01.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.260+0000 7f060b02e700 1 Processor -- start 2023-10-10T20:35:01.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.260+0000 7f060b02e700 1 -- start start 2023-10-10T20:35:01.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.260+0000 7f060b02e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f05ec0051c0 0x7f06041585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.260+0000 7f060b02e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06040a9b00 0x7f0604169d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.260+0000 7f060b02e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06040cc3e0 0x7f06040cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.261+0000 7f060b02e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06040aae50 con 0x7f06040a9b00 2023-10-10T20:35:01.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.261+0000 7f060b02e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06040ccdc0 con 0x7f06040cc3e0 2023-10-10T20:35:01.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.261+0000 7f060b02e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0604158b20 con 0x7f05ec0051c0 2023-10-10T20:35:01.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.261+0000 7f0603fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06040a9b00 0x7f0604169d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:01.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.261+0000 7f0603fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06040a9b00 0x7f0604169d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59348/0 (socket says 172.21.15.89:59348) 2023-10-10T20:35:01.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.261+0000 7f0603fff700 1 -- 172.21.15.89:0/514811933 learned_addr learned my addr 172.21.15.89:0/514811933 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:01.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.261+0000 7f0603fff700 1 -- 172.21.15.89:0/514811933 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f05ec0051c0 msgr2=0x7f06041585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:01.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.261+0000 7f0603fff700 1 --2- 172.21.15.89:0/514811933 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f05ec0051c0 0x7f06041585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.261+0000 7f0603fff700 1 -- 172.21.15.89:0/514811933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06040cc3e0 msgr2=0x7f06040cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:01.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.261+0000 7f0603fff700 1 --2- 172.21.15.89:0/514811933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06040cc3e0 0x7f06040cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.261+0000 7f0603fff700 1 -- 172.21.15.89:0/514811933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060416b210 con 0x7f06040a9b00 2023-10-10T20:35:01.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.261+0000 7f0603fff700 1 --2- 172.21.15.89:0/514811933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06040a9b00 0x7f0604169d10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f05f8006dc0 tx=0x7f05f8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:01.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.261+0000 7f0600ff9700 1 -- 172.21.15.89:0/514811933 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f800e870 con 0x7f06040a9b00 2023-10-10T20:35:01.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.261+0000 7f0600ff9700 1 -- 172.21.15.89:0/514811933 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f800e9d0 con 0x7f06040a9b00 2023-10-10T20:35:01.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.261+0000 7f060b02e700 1 -- 172.21.15.89:0/514811933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060416a590 con 0x7f06040a9b00 2023-10-10T20:35:01.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.261+0000 7f0600ff9700 1 -- 172.21.15.89:0/514811933 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f800ed50 con 0x7f06040a9b00 2023-10-10T20:35:01.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.261+0000 7f060b02e700 1 -- 172.21.15.89:0/514811933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060416ab50 con 0x7f06040a9b00 2023-10-10T20:35:01.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.262+0000 7f060b02e700 1 -- 172.21.15.89:0/514811933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060404c3c0 con 0x7f06040a9b00 2023-10-10T20:35:01.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.263+0000 7f0600ff9700 1 -- 172.21.15.89:0/514811933 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f05f801c610 con 0x7f06040a9b00 2023-10-10T20:35:01.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.263+0000 7f0600ff9700 1 --2- 172.21.15.89:0/514811933 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f05f0066f30 0x7f05f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.263+0000 7f0600ff9700 1 -- 172.21.15.89:0/514811933 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f05f80849e0 con 0x7f06040a9b00 2023-10-10T20:35:01.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.263+0000 7f0608dca700 1 --2- 172.21.15.89:0/514811933 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f05f0066f30 0x7f05f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:01.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.263+0000 7f0608dca700 1 --2- 172.21.15.89:0/514811933 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f05f0066f30 0x7f05f00693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f05f40099f0 tx=0x7f05f400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:01.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.267+0000 7f0600ff9700 1 -- 172.21.15.89:0/514811933 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05f8051c50 con 0x7f06040a9b00 2023-10-10T20:35:01.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.448+0000 7f060b02e700 1 -- 172.21.15.89:0/514811933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f060416b3a0 con 0x7f06040a9b00 2023-10-10T20:35:01.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.449+0000 7f0600ff9700 1 -- 172.21.15.89:0/514811933 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f05f8055270 con 0x7f06040a9b00 2023-10-10T20:35:01.452 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:35:01.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.451+0000 7f060b02e700 1 -- 172.21.15.89:0/514811933 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f05f0066f30 msgr2=0x7f05f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:01.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.451+0000 7f060b02e700 1 --2- 172.21.15.89:0/514811933 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f05f0066f30 0x7f05f00693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f05f40099f0 tx=0x7f05f400b040 comp rx=0 tx=0).stop 2023-10-10T20:35:01.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.451+0000 7f060b02e700 1 -- 172.21.15.89:0/514811933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06040a9b00 msgr2=0x7f0604169d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:01.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.451+0000 7f060b02e700 1 --2- 172.21.15.89:0/514811933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06040a9b00 0x7f0604169d10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f05f8006dc0 tx=0x7f05f8008040 comp rx=0 tx=0).stop 2023-10-10T20:35:01.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.451+0000 7f060b02e700 1 -- 172.21.15.89:0/514811933 shutdown_connections 2023-10-10T20:35:01.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.451+0000 7f060b02e700 1 --2- 172.21.15.89:0/514811933 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f05f0066f30 0x7f05f00693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.451+0000 7f060b02e700 1 --2- 172.21.15.89:0/514811933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06040cc3e0 0x7f06040cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.451+0000 7f060b02e700 1 --2- 172.21.15.89:0/514811933 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f05ec0051c0 0x7f06041585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.451+0000 7f060b02e700 1 --2- 172.21.15.89:0/514811933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06040a9b00 0x7f0604169d10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.451+0000 7f060b02e700 1 -- 172.21.15.89:0/514811933 >> 172.21.15.89:0/514811933 conn(0x7f06040ce600 msgr2=0x7f060406a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:01.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.452+0000 7f060b02e700 1 -- 172.21.15.89:0/514811933 shutdown_connections 2023-10-10T20:35:01.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.452+0000 7f060b02e700 1 -- 172.21.15.89:0/514811933 wait complete. 2023-10-10T20:35:01.466 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:35:01.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.581+0000 7f1e019ef700 1 Processor -- start 2023-10-10T20:35:01.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.581+0000 7f1e019ef700 1 -- start start 2023-10-10T20:35:01.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.582+0000 7f1e019ef700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1dfc1568e0 0x7f1dfc1616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.582+0000 7f1e019ef700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dfc15b0d0 con 0x7f1dfc0ccee0 2023-10-10T20:35:01.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.582+0000 7f1e019ef700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dfc15a4c0 con 0x7f1dfc0c83f0 2023-10-10T20:35:01.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.582+0000 7f1e019ef700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dfc15a7a0 con 0x7f1dfc1568e0 2023-10-10T20:35:01.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.582+0000 7f1dfb7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1dfc1568e0 0x7f1dfc1616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:01.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.582+0000 7f1dfa7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1dfc0ccee0 0x7f1dfc1561d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58164/0 (socket says 172.21.15.89:58164) 2023-10-10T20:35:01.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.582+0000 7f1dfa7fc700 1 -- 172.21.15.89:0/2790962585 learned_addr learned my addr 172.21.15.89:0/2790962585 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:01.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.583+0000 7f1dfb7fe700 1 -- 172.21.15.89:0/2790962585 >> v1:172.21.15.111:6789/0 conn(0x7f1dfc0c83f0 legacy=0x7f1dfc0c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:01.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.583+0000 7f1dfb7fe700 1 -- 172.21.15.89:0/2790962585 >> v1:172.21.15.89:6789/0 conn(0x7f1dfc0ccee0 legacy=0x7f1dfc1561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:01.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.583+0000 7f1df9ffb700 1 -- 172.21.15.89:0/2790962585 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2032603219 0 0) 0x7f1dfc15b0d0 con 0x7f1dfc0ccee0 2023-10-10T20:35:01.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.583+0000 7f1dfb7fe700 1 -- 172.21.15.89:0/2790962585 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc15aa80 con 0x7f1dfc1568e0 2023-10-10T20:35:01.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.583+0000 7f1dfb7fe700 1 --2- 172.21.15.89:0/2790962585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc1568e0 0x7f1dfc1616c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1df40098e0 tx=0x7f1df40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343a04b4492e728c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:01.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.583+0000 7f1df9ffb700 1 -- 172.21.15.89:0/2790962585 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4009e80 con 0x7f1dfc1568e0 2023-10-10T20:35:01.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.583+0000 7f1df9ffb700 1 -- 172.21.15.89:0/2790962585 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df400b860 con 0x7f1dfc1568e0 2023-10-10T20:35:01.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.583+0000 7f1df9ffb700 1 -- 172.21.15.89:0/2790962585 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df400bbe0 con 0x7f1dfc1568e0 2023-10-10T20:35:01.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.583+0000 7f1e019ef700 1 -- 172.21.15.89:0/2790962585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc1568e0 msgr2=0x7f1dfc1616c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:01.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.583+0000 7f1e019ef700 1 --2- 172.21.15.89:0/2790962585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc1568e0 0x7f1dfc1616c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1df40098e0 tx=0x7f1df40091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.583+0000 7f1e019ef700 1 -- 172.21.15.89:0/2790962585 shutdown_connections 2023-10-10T20:35:01.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.583+0000 7f1e019ef700 1 --2- 172.21.15.89:0/2790962585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc1568e0 0x7f1dfc1616c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.583+0000 7f1e019ef700 1 -- 172.21.15.89:0/2790962585 >> 172.21.15.89:0/2790962585 conn(0x7f1dfc068e10 msgr2=0x7f1dfc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:01.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.584+0000 7f1e019ef700 1 -- 172.21.15.89:0/2790962585 shutdown_connections 2023-10-10T20:35:01.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.584+0000 7f1e019ef700 1 -- 172.21.15.89:0/2790962585 wait complete. 2023-10-10T20:35:01.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.585+0000 7f1e019ef700 1 Processor -- start 2023-10-10T20:35:01.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.585+0000 7f1e019ef700 1 -- start start 2023-10-10T20:35:01.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.585+0000 7f1e019ef700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1dfc0c83f0 0x7f1dfc16b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.585+0000 7f1e019ef700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dfc0ccee0 0x7f1dfc168a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.585+0000 7f1e019ef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc1568e0 0x7f1dfc159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.585+0000 7f1e019ef700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc167ec0 con 0x7f1dfc1568e0 2023-10-10T20:35:01.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.585+0000 7f1e019ef700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc16bb90 con 0x7f1dfc0ccee0 2023-10-10T20:35:01.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.585+0000 7f1e019ef700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc16be70 con 0x7f1dfc0c83f0 2023-10-10T20:35:01.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.586+0000 7f1dfb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc1568e0 0x7f1dfc159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:01.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.586+0000 7f1dfb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc1568e0 0x7f1dfc159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59364/0 (socket says 172.21.15.89:59364) 2023-10-10T20:35:01.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.586+0000 7f1dfb7fe700 1 -- 172.21.15.89:0/1343281347 learned_addr learned my addr 172.21.15.89:0/1343281347 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:01.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.586+0000 7f1dfa7fc700 1 --2- 172.21.15.89:0/1343281347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dfc0ccee0 0x7f1dfc168a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:01.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.586+0000 7f1dfb7fe700 1 -- 172.21.15.89:0/1343281347 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1dfc0c83f0 msgr2=0x7f1dfc16b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:01.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.586+0000 7f1dfaffd700 1 --2- 172.21.15.89:0/1343281347 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1dfc0c83f0 0x7f1dfc16b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:01.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.586+0000 7f1dfb7fe700 1 --2- 172.21.15.89:0/1343281347 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1dfc0c83f0 0x7f1dfc16b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.586+0000 7f1dfb7fe700 1 -- 172.21.15.89:0/1343281347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dfc0ccee0 msgr2=0x7f1dfc168a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:01.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.586+0000 7f1dfb7fe700 1 --2- 172.21.15.89:0/1343281347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dfc0ccee0 0x7f1dfc168a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.586+0000 7f1dfb7fe700 1 -- 172.21.15.89:0/1343281347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df4009420 con 0x7f1dfc1568e0 2023-10-10T20:35:01.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.586+0000 7f1dfa7fc700 1 --2- 172.21.15.89:0/1343281347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dfc0ccee0 0x7f1dfc168a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:01.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.586+0000 7f1dfaffd700 1 --2- 172.21.15.89:0/1343281347 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1dfc0c83f0 0x7f1dfc16b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:01.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.586+0000 7f1dfb7fe700 1 --2- 172.21.15.89:0/1343281347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc1568e0 0x7f1dfc159530 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1df40030c0 tx=0x7f1df4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:01.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.586+0000 7f1de2ffd700 1 -- 172.21.15.89:0/1343281347 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df401b450 con 0x7f1dfc1568e0 2023-10-10T20:35:01.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.586+0000 7f1de2ffd700 1 -- 172.21.15.89:0/1343281347 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df4020070 con 0x7f1dfc1568e0 2023-10-10T20:35:01.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.586+0000 7f1e019ef700 1 -- 172.21.15.89:0/1343281347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dfc16c150 con 0x7f1dfc1568e0 2023-10-10T20:35:01.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.586+0000 7f1de2ffd700 1 -- 172.21.15.89:0/1343281347 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df401b800 con 0x7f1dfc1568e0 2023-10-10T20:35:01.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.586+0000 7f1e019ef700 1 -- 172.21.15.89:0/1343281347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc159a70 con 0x7f1dfc1568e0 2023-10-10T20:35:01.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.587+0000 7f1e019ef700 1 -- 172.21.15.89:0/1343281347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dfc159c60 con 0x7f1dfc1568e0 2023-10-10T20:35:01.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.587+0000 7f1de2ffd700 1 -- 172.21.15.89:0/1343281347 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1df4022030 con 0x7f1dfc1568e0 2023-10-10T20:35:01.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.588+0000 7f1de2ffd700 1 --2- 172.21.15.89:0/1343281347 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1de8066f30 0x7f1de80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.588+0000 7f1de2ffd700 1 -- 172.21.15.89:0/1343281347 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1df4086620 con 0x7f1dfc1568e0 2023-10-10T20:35:01.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.588+0000 7f1dfaffd700 1 --2- 172.21.15.89:0/1343281347 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1de8066f30 0x7f1de80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:01.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.588+0000 7f1dfaffd700 1 --2- 172.21.15.89:0/1343281347 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1de8066f30 0x7f1de80693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1dec007fd0 tx=0x7f1dec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:01.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.592+0000 7f1de2ffd700 1 -- 172.21.15.89:0/1343281347 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1dfc159c60 con 0x7f1dfc1568e0 2023-10-10T20:35:01.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.774+0000 7f1e019ef700 1 -- 172.21.15.89:0/1343281347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1dfc159c60 con 0x7f1dfc1568e0 2023-10-10T20:35:01.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.774+0000 7f1de2ffd700 1 -- 172.21.15.89:0/1343281347 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1dfc159c60 con 0x7f1dfc1568e0 2023-10-10T20:35:01.777 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-10T20:35:01.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.776+0000 7f1e019ef700 1 -- 172.21.15.89:0/1343281347 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1de8066f30 msgr2=0x7f1de80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:01.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.776+0000 7f1e019ef700 1 --2- 172.21.15.89:0/1343281347 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1de8066f30 0x7f1de80693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1dec007fd0 tx=0x7f1dec009040 comp rx=0 tx=0).stop 2023-10-10T20:35:01.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.776+0000 7f1e019ef700 1 -- 172.21.15.89:0/1343281347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc1568e0 msgr2=0x7f1dfc159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:01.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.776+0000 7f1e019ef700 1 --2- 172.21.15.89:0/1343281347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc1568e0 0x7f1dfc159530 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1df40030c0 tx=0x7f1df4016000 comp rx=0 tx=0).stop 2023-10-10T20:35:01.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.776+0000 7f1e019ef700 1 -- 172.21.15.89:0/1343281347 shutdown_connections 2023-10-10T20:35:01.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.776+0000 7f1e019ef700 1 --2- 172.21.15.89:0/1343281347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc1568e0 0x7f1dfc159530 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.776+0000 7f1e019ef700 1 --2- 172.21.15.89:0/1343281347 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1dfc0c83f0 0x7f1dfc16b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.776+0000 7f1e019ef700 1 --2- 172.21.15.89:0/1343281347 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1de8066f30 0x7f1de80693e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.776+0000 7f1e019ef700 1 --2- 172.21.15.89:0/1343281347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dfc0ccee0 0x7f1dfc168a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.776+0000 7f1e019ef700 1 -- 172.21.15.89:0/1343281347 >> 172.21.15.89:0/1343281347 conn(0x7f1dfc068e10 msgr2=0x7f1dfc0c1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:01.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.776+0000 7f1e019ef700 1 -- 172.21.15.89:0/1343281347 shutdown_connections 2023-10-10T20:35:01.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.776+0000 7f1e019ef700 1 -- 172.21.15.89:0/1343281347 wait complete. 2023-10-10T20:35:01.856 INFO:tasks.cephfs_test_runner:Starting test: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) That the volume can be created and then cleans up 2023-10-10T20:35:01.857 INFO:tasks.cephfs_test_runner:test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2023-10-10T20:35:01.857 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2023-10-10T20:35:01.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.970+0000 7fa7ae744700 1 Processor -- start 2023-10-10T20:35:01.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.970+0000 7fa7ae744700 1 -- start start 2023-10-10T20:35:01.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.971+0000 7fa7ae744700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa7a8156690 0x7fa7a80ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.971+0000 7fa7ae744700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa7a80af480 0x7fa7a80694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.971+0000 7fa7ae744700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a8157e30 con 0x7fa7a8156a60 2023-10-10T20:35:01.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.971+0000 7fa7ae744700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a806aa00 con 0x7fa7a8156690 2023-10-10T20:35:01.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.971+0000 7fa7ae744700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a80699e0 con 0x7fa7a80af480 2023-10-10T20:35:01.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.971+0000 7fa7a7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa7a8156690 0x7fa7a80ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:01.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.971+0000 7fa7a7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa7a8156690 0x7fa7a80ca9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59370/0 (socket says 172.21.15.89:59370) 2023-10-10T20:35:01.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.971+0000 7fa7a7fff700 1 -- 172.21.15.89:0/3039774971 learned_addr learned my addr 172.21.15.89:0/3039774971 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.972+0000 7fa7a77fe700 1 --2- 172.21.15.89:0/3039774971 >> v2:172.21.15.111:3300/0 conn(0x7fa7a80af480 0x7fa7a80694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.972+0000 7fa7a7fff700 1 -- 172.21.15.89:0/3039774971 >> v2:172.21.15.111:3300/0 conn(0x7fa7a80af480 msgr2=0x7fa7a80694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.972+0000 7fa7a7fff700 1 --2- 172.21.15.89:0/3039774971 >> v2:172.21.15.111:3300/0 conn(0x7fa7a80af480 0x7fa7a80694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.972+0000 7fa7a7fff700 1 -- 172.21.15.89:0/3039774971 >> v1:172.21.15.111:6789/0 conn(0x7fa7a8156a60 legacy=0x7fa7a80c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.972+0000 7fa7a7fff700 1 -- 172.21.15.89:0/3039774971 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a81578b0 con 0x7fa7a8156690 2023-10-10T20:35:01.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.972+0000 7fa7a77fe700 1 --2- 172.21.15.89:0/3039774971 >> v2:172.21.15.111:3300/0 conn(0x7fa7a80af480 0x7fa7a80694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:01.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.972+0000 7fa7a7fff700 1 --2- 172.21.15.89:0/3039774971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7a8156690 0x7fa7a80ca9d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8069e90 tx=0x7fa798009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67b589a3e012279 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:01.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.972+0000 7fa7a6ffd700 1 -- 172.21.15.89:0/3039774971 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79800b820 con 0x7fa7a8156690 2023-10-10T20:35:01.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.972+0000 7fa7a6ffd700 1 -- 172.21.15.89:0/3039774971 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79800b980 con 0x7fa7a8156690 2023-10-10T20:35:01.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.972+0000 7fa7a6ffd700 1 -- 172.21.15.89:0/3039774971 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79800bd40 con 0x7fa7a8156690 2023-10-10T20:35:01.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.973+0000 7fa7ae744700 1 -- 172.21.15.89:0/3039774971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7a8156690 msgr2=0x7fa7a80ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:01.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.973+0000 7fa7ae744700 1 --2- 172.21.15.89:0/3039774971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7a8156690 0x7fa7a80ca9d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8069e90 tx=0x7fa798009250 comp rx=0 tx=0).stop 2023-10-10T20:35:01.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.973+0000 7fa7ae744700 1 -- 172.21.15.89:0/3039774971 shutdown_connections 2023-10-10T20:35:01.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.973+0000 7fa7ae744700 1 --2- 172.21.15.89:0/3039774971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7a8156690 0x7fa7a80ca9d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.973+0000 7fa7ae744700 1 --2- 172.21.15.89:0/3039774971 >> v2:172.21.15.111:3300/0 conn(0x7fa7a80af480 0x7fa7a80694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.973+0000 7fa7ae744700 1 -- 172.21.15.89:0/3039774971 >> 172.21.15.89:0/3039774971 conn(0x7fa7a80c9870 msgr2=0x7fa7a80c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:01.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.973+0000 7fa7ae744700 1 -- 172.21.15.89:0/3039774971 shutdown_connections 2023-10-10T20:35:01.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.973+0000 7fa7ae744700 1 -- 172.21.15.89:0/3039774971 wait complete. 2023-10-10T20:35:01.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.974+0000 7fa7ae744700 1 Processor -- start 2023-10-10T20:35:01.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.974+0000 7fa7ae744700 1 -- start start 2023-10-10T20:35:01.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.974+0000 7fa7ae744700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7a80af480 0x7fa7a8155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa7ae744700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7a8156690 0x7fa7a80c3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa7ae744700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa7a8156a60 0x7fa7a80c3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa7ae744700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a815a020 con 0x7fa7a80af480 2023-10-10T20:35:01.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa7ae744700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a81457f0 con 0x7fa7a8156690 2023-10-10T20:35:01.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa7ae744700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a80b0ce0 con 0x7fa7a8156a60 2023-10-10T20:35:01.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa7a7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7a80af480 0x7fa7a8155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:01.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa7a7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7a80af480 0x7fa7a8155840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59376/0 (socket says 172.21.15.89:59376) 2023-10-10T20:35:01.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa7a7fff700 1 -- 172.21.15.89:0/2878554721 learned_addr learned my addr 172.21.15.89:0/2878554721 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:01.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa7a77fe700 1 --2- 172.21.15.89:0/2878554721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7a8156690 0x7fa7a80c3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:01.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa7acce1700 1 --2- 172.21.15.89:0/2878554721 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa7a8156a60 0x7fa7a80c3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:01.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa7a7fff700 1 -- 172.21.15.89:0/2878554721 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa7a8156a60 msgr2=0x7fa7a80c3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:01.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa7a7fff700 1 --2- 172.21.15.89:0/2878554721 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa7a8156a60 0x7fa7a80c3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa7a7fff700 1 -- 172.21.15.89:0/2878554721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7a8156690 msgr2=0x7fa7a80c3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:01.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa7a7fff700 1 --2- 172.21.15.89:0/2878554721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7a8156690 0x7fa7a80c3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:01.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa7a7fff700 1 -- 172.21.15.89:0/2878554721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7980094f0 con 0x7fa7a80af480 2023-10-10T20:35:01.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa7a7fff700 1 --2- 172.21.15.89:0/2878554721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7a80af480 0x7fa7a8155840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa7a806cc00 tx=0x7fa798016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:01.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa78ffff700 1 -- 172.21.15.89:0/2878554721 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa798011d50 con 0x7fa7a80af480 2023-10-10T20:35:01.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa78ffff700 1 -- 172.21.15.89:0/2878554721 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa798020070 con 0x7fa7a80af480 2023-10-10T20:35:01.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa7ae744700 1 -- 172.21.15.89:0/2878554721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a8159410 con 0x7fa7a80af480 2023-10-10T20:35:01.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa78ffff700 1 -- 172.21.15.89:0/2878554721 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79801b6a0 con 0x7fa7a80af480 2023-10-10T20:35:01.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.975+0000 7fa7ae744700 1 -- 172.21.15.89:0/2878554721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a81599d0 con 0x7fa7a80af480 2023-10-10T20:35:01.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.976+0000 7fa7ae744700 1 -- 172.21.15.89:0/2878554721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a80b0e20 con 0x7fa7a80af480 2023-10-10T20:35:01.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.977+0000 7fa78ffff700 1 -- 172.21.15.89:0/2878554721 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa798022030 con 0x7fa7a80af480 2023-10-10T20:35:01.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.977+0000 7fa78ffff700 1 --2- 172.21.15.89:0/2878554721 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa794066f30 0x7fa7940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:01.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.977+0000 7fa78ffff700 1 -- 172.21.15.89:0/2878554721 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa798086440 con 0x7fa7a80af480 2023-10-10T20:35:01.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.977+0000 7fa7a77fe700 1 --2- 172.21.15.89:0/2878554721 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa794066f30 0x7fa7940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:01.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.977+0000 7fa7a77fe700 1 --2- 172.21.15.89:0/2878554721 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa794066f30 0x7fa7940693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa7a80b2690 tx=0x7fa79c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:01.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:01.981+0000 7fa78ffff700 1 -- 172.21.15.89:0/2878554721 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7980536b0 con 0x7fa7a80af480 2023-10-10T20:35:02.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:02.239+0000 7fa7ae744700 1 -- 172.21.15.89:0/2878554721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7fa7a80b0e20 con 0x7fa7a80af480 2023-10-10T20:35:02.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:02.902+0000 7fa78ffff700 1 -- 172.21.15.89:0/2878554721 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v43) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa7980092d0 con 0x7fa7a80af480 2023-10-10T20:35:02.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:02.904+0000 7fa7ae744700 1 -- 172.21.15.89:0/2878554721 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa794066f30 msgr2=0x7fa7940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:02.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:02.904+0000 7fa7ae744700 1 --2- 172.21.15.89:0/2878554721 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa794066f30 0x7fa7940693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa7a80b2690 tx=0x7fa79c008040 comp rx=0 tx=0).stop 2023-10-10T20:35:02.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:02.904+0000 7fa7ae744700 1 -- 172.21.15.89:0/2878554721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7a80af480 msgr2=0x7fa7a8155840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:02.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:02.904+0000 7fa7ae744700 1 --2- 172.21.15.89:0/2878554721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7a80af480 0x7fa7a8155840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa7a806cc00 tx=0x7fa798016000 comp rx=0 tx=0).stop 2023-10-10T20:35:02.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:02.904+0000 7fa7ae744700 1 -- 172.21.15.89:0/2878554721 shutdown_connections 2023-10-10T20:35:02.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:02.904+0000 7fa7ae744700 1 --2- 172.21.15.89:0/2878554721 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa7a8156a60 0x7fa7a80c3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:02.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:02.904+0000 7fa7ae744700 1 --2- 172.21.15.89:0/2878554721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7a80af480 0x7fa7a8155840 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:02.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:02.904+0000 7fa7ae744700 1 --2- 172.21.15.89:0/2878554721 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa794066f30 0x7fa7940693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:02.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:02.904+0000 7fa7ae744700 1 --2- 172.21.15.89:0/2878554721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7a8156690 0x7fa7a80c3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:02.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:02.904+0000 7fa7ae744700 1 -- 172.21.15.89:0/2878554721 >> 172.21.15.89:0/2878554721 conn(0x7fa7a80c9870 msgr2=0x7fa7a8069260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:02.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:02.904+0000 7fa7ae744700 1 -- 172.21.15.89:0/2878554721 shutdown_connections 2023-10-10T20:35:02.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:02.905+0000 7fa7ae744700 1 -- 172.21.15.89:0/2878554721 wait complete. 2023-10-10T20:35:02.920 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:35:03.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.037+0000 7f2f9207b700 1 Processor -- start 2023-10-10T20:35:03.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.037+0000 7f2f9207b700 1 -- start start 2023-10-10T20:35:03.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.038+0000 7f2f9207b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f8c141500 0x7f2f8c0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:03.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.038+0000 7f2f9207b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2f8c06c0a0 0x7f2f8c0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:03.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.038+0000 7f2f9207b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f8c15c2a0 con 0x7f2f8c06bcd0 2023-10-10T20:35:03.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.038+0000 7f2f9207b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f8c0c9880 con 0x7f2f8c06c0a0 2023-10-10T20:35:03.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.038+0000 7f2f9207b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c15b690 con 0x7f2f8c141500 2023-10-10T20:35:03.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.038+0000 7f2f8bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2f8c06c0a0 0x7f2f8c0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:03.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.038+0000 7f2f8b7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2f8c06bcd0 0x7f2f8c140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58176/0 (socket says 172.21.15.89:58176) 2023-10-10T20:35:03.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.038+0000 7f2f8b7fe700 1 -- 172.21.15.89:0/2531047191 learned_addr learned my addr 172.21.15.89:0/2531047191 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:03.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.039+0000 7f2f8bfff700 1 -- 172.21.15.89:0/2531047191 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f8c141500 msgr2=0x7f2f8c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:03.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.039+0000 7f2f8affd700 1 --2- 172.21.15.89:0/2531047191 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f8c141500 0x7f2f8c0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:03.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.039+0000 7f2f8a7fc700 1 -- 172.21.15.89:0/2531047191 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 177833286 0 0) 0x7f2f8c15c2a0 con 0x7f2f8c06bcd0 2023-10-10T20:35:03.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.039+0000 7f2f8bfff700 1 --2- 172.21.15.89:0/2531047191 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f8c141500 0x7f2f8c0c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:03.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.039+0000 7f2f8bfff700 1 -- 172.21.15.89:0/2531047191 >> v1:172.21.15.89:6789/0 conn(0x7f2f8c06bcd0 legacy=0x7f2f8c140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:03.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.039+0000 7f2f8bfff700 1 -- 172.21.15.89:0/2531047191 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8c15b970 con 0x7f2f8c06c0a0 2023-10-10T20:35:03.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.039+0000 7f2f8affd700 1 --2- 172.21.15.89:0/2531047191 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f8c141500 0x7f2f8c0c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:03.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.039+0000 7f2f8bfff700 1 --2- 172.21.15.89:0/2531047191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f8c06c0a0 0x7f2f8c0c9340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2f840098e0 tx=0x7f2f840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86487f216a053978 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:03.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.039+0000 7f2f8a7fc700 1 -- 172.21.15.89:0/2531047191 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84009e80 con 0x7f2f8c06c0a0 2023-10-10T20:35:03.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.039+0000 7f2f8a7fc700 1 -- 172.21.15.89:0/2531047191 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8400b860 con 0x7f2f8c06c0a0 2023-10-10T20:35:03.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.039+0000 7f2f8a7fc700 1 -- 172.21.15.89:0/2531047191 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8400bbe0 con 0x7f2f8c06c0a0 2023-10-10T20:35:03.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.039+0000 7f2f9207b700 1 -- 172.21.15.89:0/2531047191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f8c06c0a0 msgr2=0x7f2f8c0c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:03.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.039+0000 7f2f9207b700 1 --2- 172.21.15.89:0/2531047191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f8c06c0a0 0x7f2f8c0c9340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2f840098e0 tx=0x7f2f840091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:03.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.040+0000 7f2f9207b700 1 -- 172.21.15.89:0/2531047191 shutdown_connections 2023-10-10T20:35:03.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.040+0000 7f2f9207b700 1 --2- 172.21.15.89:0/2531047191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f8c06c0a0 0x7f2f8c0c9340 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:03.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.040+0000 7f2f9207b700 1 --2- 172.21.15.89:0/2531047191 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f8c141500 0x7f2f8c0c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:03.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.040+0000 7f2f9207b700 1 -- 172.21.15.89:0/2531047191 >> 172.21.15.89:0/2531047191 conn(0x7f2f8c0ce600 msgr2=0x7f2f8c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:03.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.040+0000 7f2f9207b700 1 -- 172.21.15.89:0/2531047191 shutdown_connections 2023-10-10T20:35:03.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.040+0000 7f2f9207b700 1 -- 172.21.15.89:0/2531047191 wait complete. 2023-10-10T20:35:03.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.041+0000 7f2f9207b700 1 Processor -- start 2023-10-10T20:35:03.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.041+0000 7f2f9207b700 1 -- start start 2023-10-10T20:35:03.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f9207b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f8c06bcd0 0x7f2f8c168e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:03.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f9207b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f8c06c0a0 0x7f2f8c13cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:03.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f9207b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f8c141500 0x7f2f8c0b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:03.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f9207b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c0c7040 con 0x7f2f8c06c0a0 2023-10-10T20:35:03.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f9207b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c13e770 con 0x7f2f8c141500 2023-10-10T20:35:03.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f9207b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c13f240 con 0x7f2f8c06bcd0 2023-10-10T20:35:03.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f8affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f8c06c0a0 0x7f2f8c13cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:03.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f8affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f8c06c0a0 0x7f2f8c13cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59400/0 (socket says 172.21.15.89:59400) 2023-10-10T20:35:03.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f8affd700 1 -- 172.21.15.89:0/432817226 learned_addr learned my addr 172.21.15.89:0/432817226 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:03.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f8b7fe700 1 --2- 172.21.15.89:0/432817226 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f8c06bcd0 0x7f2f8c168e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:03.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f8bfff700 1 --2- 172.21.15.89:0/432817226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f8c141500 0x7f2f8c0b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:03.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f8affd700 1 -- 172.21.15.89:0/432817226 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f8c06bcd0 msgr2=0x7f2f8c168e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:03.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f8affd700 1 --2- 172.21.15.89:0/432817226 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f8c06bcd0 0x7f2f8c168e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:03.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f8affd700 1 -- 172.21.15.89:0/432817226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f8c141500 msgr2=0x7f2f8c0b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:03.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f8affd700 1 --2- 172.21.15.89:0/432817226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f8c141500 0x7f2f8c0b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:03.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f8affd700 1 -- 172.21.15.89:0/432817226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f84009420 con 0x7f2f8c06c0a0 2023-10-10T20:35:03.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f8bfff700 1 --2- 172.21.15.89:0/432817226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f8c141500 0x7f2f8c0b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:03.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f8affd700 1 --2- 172.21.15.89:0/432817226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f8c06c0a0 0x7f2f8c13cdc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000c880 tx=0x7f2f80008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:03.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f737fe700 1 -- 172.21.15.89:0/432817226 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f80011600 con 0x7f2f8c06c0a0 2023-10-10T20:35:03.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f737fe700 1 -- 172.21.15.89:0/432817226 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f80011780 con 0x7f2f8c06c0a0 2023-10-10T20:35:03.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f9207b700 1 -- 172.21.15.89:0/432817226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8c0c6d60 con 0x7f2f8c06c0a0 2023-10-10T20:35:03.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f737fe700 1 -- 172.21.15.89:0/432817226 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f80011b40 con 0x7f2f8c06c0a0 2023-10-10T20:35:03.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.042+0000 7f2f9207b700 1 -- 172.21.15.89:0/432817226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8c13eed0 con 0x7f2f8c06c0a0 2023-10-10T20:35:03.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.043+0000 7f2f9207b700 1 -- 172.21.15.89:0/432817226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f8c04c3c0 con 0x7f2f8c06c0a0 2023-10-10T20:35:03.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.044+0000 7f2f737fe700 1 -- 172.21.15.89:0/432817226 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2f80018350 con 0x7f2f8c06c0a0 2023-10-10T20:35:03.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.044+0000 7f2f737fe700 1 --2- 172.21.15.89:0/432817226 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2f78066f30 0x7f2f780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:03.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.044+0000 7f2f737fe700 1 -- 172.21.15.89:0/432817226 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2f800843f0 con 0x7f2f8c06c0a0 2023-10-10T20:35:03.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.044+0000 7f2f8b7fe700 1 --2- 172.21.15.89:0/432817226 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2f78066f30 0x7f2f780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:03.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.044+0000 7f2f8b7fe700 1 --2- 172.21.15.89:0/432817226 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2f78066f30 0x7f2f780693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c002250 tx=0x7f2f7c001a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:03.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.048+0000 7f2f737fe700 1 -- 172.21.15.89:0/432817226 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2f80051660 con 0x7f2f8c06c0a0 2023-10-10T20:35:03.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.231+0000 7f2f9207b700 1 -- 172.21.15.89:0/432817226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2f8c13f380 con 0x7f2f8c06c0a0 2023-10-10T20:35:03.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.235+0000 7f2f737fe700 1 -- 172.21.15.89:0/432817226 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2f80054c80 con 0x7f2f8c06c0a0 2023-10-10T20:35:03.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.237+0000 7f2f9207b700 1 -- 172.21.15.89:0/432817226 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2f78066f30 msgr2=0x7f2f780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:03.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.237+0000 7f2f9207b700 1 --2- 172.21.15.89:0/432817226 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2f78066f30 0x7f2f780693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c002250 tx=0x7f2f7c001a20 comp rx=0 tx=0).stop 2023-10-10T20:35:03.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.237+0000 7f2f9207b700 1 -- 172.21.15.89:0/432817226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f8c06c0a0 msgr2=0x7f2f8c13cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:03.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.237+0000 7f2f9207b700 1 --2- 172.21.15.89:0/432817226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f8c06c0a0 0x7f2f8c13cdc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000c880 tx=0x7f2f80008040 comp rx=0 tx=0).stop 2023-10-10T20:35:03.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.237+0000 7f2f9207b700 1 -- 172.21.15.89:0/432817226 shutdown_connections 2023-10-10T20:35:03.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.237+0000 7f2f9207b700 1 --2- 172.21.15.89:0/432817226 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2f78066f30 0x7f2f780693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:03.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.237+0000 7f2f9207b700 1 --2- 172.21.15.89:0/432817226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f8c141500 0x7f2f8c0b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:03.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.237+0000 7f2f9207b700 1 --2- 172.21.15.89:0/432817226 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f8c06bcd0 0x7f2f8c168e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:03.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.237+0000 7f2f9207b700 1 --2- 172.21.15.89:0/432817226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f8c06c0a0 0x7f2f8c13cdc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:03.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.237+0000 7f2f9207b700 1 -- 172.21.15.89:0/432817226 >> 172.21.15.89:0/432817226 conn(0x7f2f8c0ce600 msgr2=0x7f2f8c0c43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:03.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.237+0000 7f2f9207b700 1 -- 172.21.15.89:0/432817226 shutdown_connections 2023-10-10T20:35:03.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.237+0000 7f2f9207b700 1 -- 172.21.15.89:0/432817226 wait complete. 2023-10-10T20:35:03.251 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:35:03.252 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:35:03.290 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:35:03.341 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:35:03.341 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:35:03.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:35:03.342 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-10T20:35:03.378 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:35:03.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:03.379 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:35:03.434 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:35:03.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:35:03.435 DEBUG:teuthology.orchestra.run.smithi111:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-10T20:35:03.481 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:35:03.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:03.482 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:35:03.498 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-10T20:35:03.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:35:03.499 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-10T20:35:03.531 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:35:03.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:03.532 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-10T20:35:03.587 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-10T20:35:03.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:35:03.588 DEBUG:teuthology.orchestra.run.smithi111:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-10T20:35:03.629 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:35:03.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:03.630 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-10T20:35:03.685 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:35:03.686 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:35:03.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.798+0000 7fda9a8a5700 1 Processor -- start 2023-10-10T20:35:03.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.798+0000 7fda9a8a5700 1 -- start start 2023-10-10T20:35:03.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.799+0000 7fda9a8a5700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fda94159890 0x7fda9415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:03.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.799+0000 7fda9a8a5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda94159c60 0x7fda9415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:03.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.799+0000 7fda9a8a5700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda9415a160 con 0x7fda9415db10 2023-10-10T20:35:03.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.799+0000 7fda9a8a5700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda940d13a0 con 0x7fda94159890 2023-10-10T20:35:03.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.799+0000 7fda9a8a5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9415e140 con 0x7fda94159c60 2023-10-10T20:35:03.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.799+0000 7fda937fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda94159c60 0x7fda9415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:03.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.800+0000 7fda93fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fda94159890 0x7fda9415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:03.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.800+0000 7fda937fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda94159c60 0x7fda9415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:40806/0 (socket says 172.21.15.89:40806) 2023-10-10T20:35:03.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.800+0000 7fda937fe700 1 -- 172.21.15.89:0/122445736 learned_addr learned my addr 172.21.15.89:0/122445736 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:03.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.800+0000 7fda93fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fda94159890 0x7fda9415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35254/0 (socket says 172.21.15.89:35254) 2023-10-10T20:35:03.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.800+0000 7fda937fe700 1 -- 172.21.15.89:0/122445736 >> v2:172.21.15.111:3300/0 conn(0x7fda94159890 msgr2=0x7fda9415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:03.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.800+0000 7fda937fe700 1 --2- 172.21.15.89:0/122445736 >> v2:172.21.15.111:3300/0 conn(0x7fda94159890 0x7fda9415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:03.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.800+0000 7fda937fe700 1 -- 172.21.15.89:0/122445736 >> v1:172.21.15.111:6789/0 conn(0x7fda9415db10 legacy=0x7fda94069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:03.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.800+0000 7fda937fe700 1 -- 172.21.15.89:0/122445736 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9415a3d0 con 0x7fda94159c60 2023-10-10T20:35:03.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.800+0000 7fda93fff700 1 --2- 172.21.15.89:0/122445736 >> v2:172.21.15.111:3300/0 conn(0x7fda94159890 0x7fda9415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:35:03.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.800+0000 7fda937fe700 1 --2- 172.21.15.89:0/122445736 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda94159c60 0x7fda9415d5d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fda880099b0 tx=0x7fda88009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e67b4522b4066206 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:03.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.800+0000 7fda92ffd700 1 -- 172.21.15.89:0/122445736 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda88009ea0 con 0x7fda94159c60 2023-10-10T20:35:03.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.800+0000 7fda92ffd700 1 -- 172.21.15.89:0/122445736 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8800b860 con 0x7fda94159c60 2023-10-10T20:35:03.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.800+0000 7fda92ffd700 1 -- 172.21.15.89:0/122445736 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8800bc00 con 0x7fda94159c60 2023-10-10T20:35:03.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.801+0000 7fda9a8a5700 1 -- 172.21.15.89:0/122445736 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda94159c60 msgr2=0x7fda9415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:03.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.801+0000 7fda9a8a5700 1 --2- 172.21.15.89:0/122445736 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda94159c60 0x7fda9415d5d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fda880099b0 tx=0x7fda88009250 comp rx=0 tx=0).stop 2023-10-10T20:35:03.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.801+0000 7fda9a8a5700 1 -- 172.21.15.89:0/122445736 shutdown_connections 2023-10-10T20:35:03.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.801+0000 7fda9a8a5700 1 --2- 172.21.15.89:0/122445736 >> v2:172.21.15.111:3300/0 conn(0x7fda94159890 0x7fda9415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:03.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.801+0000 7fda9a8a5700 1 --2- 172.21.15.89:0/122445736 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda94159c60 0x7fda9415d5d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:03.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.801+0000 7fda9a8a5700 1 -- 172.21.15.89:0/122445736 >> 172.21.15.89:0/122445736 conn(0x7fda94068e20 msgr2=0x7fda940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:03.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.801+0000 7fda9a8a5700 1 -- 172.21.15.89:0/122445736 shutdown_connections 2023-10-10T20:35:03.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.801+0000 7fda9a8a5700 1 -- 172.21.15.89:0/122445736 wait complete. 2023-10-10T20:35:03.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.802+0000 7fda9a8a5700 1 Processor -- start 2023-10-10T20:35:03.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.802+0000 7fda9a8a5700 1 -- start start 2023-10-10T20:35:03.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.803+0000 7fda9a8a5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda94159890 0x7fda94156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:03.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.803+0000 7fda9a8a5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda94159c60 0x7fda94158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:03.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.803+0000 7fda9a8a5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda9415db10 0x7fda94154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:03.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.803+0000 7fda9a8a5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda94158c80 con 0x7fda9415db10 2023-10-10T20:35:03.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.803+0000 7fda9a8a5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda94140da0 con 0x7fda94159890 2023-10-10T20:35:03.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.803+0000 7fda9a8a5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda941554d0 con 0x7fda94159c60 2023-10-10T20:35:03.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.803+0000 7fda98e42700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda9415db10 0x7fda94154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:03.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.803+0000 7fda98e42700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda9415db10 0x7fda94154f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59402/0 (socket says 172.21.15.89:59402) 2023-10-10T20:35:03.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.803+0000 7fda98e42700 1 -- 172.21.15.89:0/623795340 learned_addr learned my addr 172.21.15.89:0/623795340 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:03.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.803+0000 7fda937fe700 1 --2- 172.21.15.89:0/623795340 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda94159c60 0x7fda94158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:03.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.803+0000 7fda93fff700 1 --2- 172.21.15.89:0/623795340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda94159890 0x7fda94156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:03.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.803+0000 7fda98e42700 1 -- 172.21.15.89:0/623795340 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda94159c60 msgr2=0x7fda94158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:03.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.803+0000 7fda98e42700 1 --2- 172.21.15.89:0/623795340 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda94159c60 0x7fda94158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:03.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.803+0000 7fda98e42700 1 -- 172.21.15.89:0/623795340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda94159890 msgr2=0x7fda94156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:03.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.803+0000 7fda98e42700 1 --2- 172.21.15.89:0/623795340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda94159890 0x7fda94156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:03.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.803+0000 7fda98e42700 1 -- 172.21.15.89:0/623795340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda84009420 con 0x7fda9415db10 2023-10-10T20:35:03.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.804+0000 7fda98e42700 1 --2- 172.21.15.89:0/623795340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda9415db10 0x7fda94154f90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00c840 tx=0x7fda8c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:03.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.804+0000 7fda7bfff700 1 -- 172.21.15.89:0/623795340 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c012630 con 0x7fda9415db10 2023-10-10T20:35:03.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.804+0000 7fda7bfff700 1 -- 172.21.15.89:0/623795340 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8c0127b0 con 0x7fda9415db10 2023-10-10T20:35:03.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.804+0000 7fda9a8a5700 1 -- 172.21.15.89:0/623795340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda880094f0 con 0x7fda9415db10 2023-10-10T20:35:03.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.804+0000 7fda7bfff700 1 -- 172.21.15.89:0/623795340 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c012b70 con 0x7fda9415db10 2023-10-10T20:35:03.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.804+0000 7fda9a8a5700 1 -- 172.21.15.89:0/623795340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda941559e0 con 0x7fda9415db10 2023-10-10T20:35:03.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.805+0000 7fda9a8a5700 1 -- 172.21.15.89:0/623795340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9404c3c0 con 0x7fda9415db10 2023-10-10T20:35:03.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.805+0000 7fda7bfff700 1 -- 172.21.15.89:0/623795340 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fda8c01a3a0 con 0x7fda9415db10 2023-10-10T20:35:03.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.805+0000 7fda7bfff700 1 --2- 172.21.15.89:0/623795340 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fda80066f30 0x7fda800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:03.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.806+0000 7fda7bfff700 1 -- 172.21.15.89:0/623795340 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fda8c084610 con 0x7fda9415db10 2023-10-10T20:35:03.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.806+0000 7fda93fff700 1 --2- 172.21.15.89:0/623795340 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fda80066f30 0x7fda800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:03.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.806+0000 7fda93fff700 1 --2- 172.21.15.89:0/623795340 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fda80066f30 0x7fda800693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fda941476c0 tx=0x7fda840091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:03.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:03.810+0000 7fda7bfff700 1 -- 172.21.15.89:0/623795340 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda8c051880 con 0x7fda9415db10 2023-10-10T20:35:04.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.039+0000 7fda9a8a5700 1 -- 172.21.15.89:0/623795340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda9406a5b0 con 0x7fda9415db10 2023-10-10T20:35:04.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.040+0000 7fda7bfff700 1 -- 172.21.15.89:0/623795340 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4597 (secure 0 0 0) 0x7fda8c054ea0 con 0x7fda9415db10 2023-10-10T20:35:04.043 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:04.043 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2341577144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2341577144},{"type":"v1","addr":"172.21.15.111:6835","nonce":2341577144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2773123076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2773123076},{"type":"v1","addr":"172.21.15.89:6837","nonce":2773123076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2940044414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2940044414},{"type":"v1","addr":"172.21.15.89:6835","nonce":2940044414}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:34:34.717750+0000","modified":"2023-10-10T20:34:38.356048+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1112081320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1112081320},{"type":"v1","addr":"172.21.15.111:6834","nonce":1112081320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-10T20:35:04.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.042+0000 7fda9a8a5700 1 -- 172.21.15.89:0/623795340 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fda80066f30 msgr2=0x7fda800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.042+0000 7fda9a8a5700 1 --2- 172.21.15.89:0/623795340 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fda80066f30 0x7fda800693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fda941476c0 tx=0x7fda840091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.042+0000 7fda9a8a5700 1 -- 172.21.15.89:0/623795340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda9415db10 msgr2=0x7fda94154f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.042+0000 7fda9a8a5700 1 --2- 172.21.15.89:0/623795340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda9415db10 0x7fda94154f90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00c840 tx=0x7fda8c001670 comp rx=0 tx=0).stop 2023-10-10T20:35:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.042+0000 7fda9a8a5700 1 -- 172.21.15.89:0/623795340 shutdown_connections 2023-10-10T20:35:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.042+0000 7fda9a8a5700 1 --2- 172.21.15.89:0/623795340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda9415db10 0x7fda94154f90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.042+0000 7fda9a8a5700 1 --2- 172.21.15.89:0/623795340 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fda80066f30 0x7fda800693e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.042+0000 7fda9a8a5700 1 --2- 172.21.15.89:0/623795340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda94159890 0x7fda94156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.042+0000 7fda9a8a5700 1 --2- 172.21.15.89:0/623795340 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda94159c60 0x7fda94158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.042+0000 7fda9a8a5700 1 -- 172.21.15.89:0/623795340 >> 172.21.15.89:0/623795340 conn(0x7fda94068e20 msgr2=0x7fda940d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:04.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.042+0000 7fda9a8a5700 1 -- 172.21.15.89:0/623795340 shutdown_connections 2023-10-10T20:35:04.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.042+0000 7fda9a8a5700 1 -- 172.21.15.89:0/623795340 wait complete. 2023-10-10T20:35:04.047 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2023-10-10T20:35:04.056 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:35:04.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.169+0000 7f7b8e861700 1 Processor -- start 2023-10-10T20:35:04.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.169+0000 7f7b8e861700 1 -- start start 2023-10-10T20:35:04.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.169+0000 7f7b8e861700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7b8806bcd0 0x7f7b88140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.169+0000 7f7b8e861700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7b880c9510 0x7f7b88141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.169+0000 7f7b8e861700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b8815c2a0 con 0x7f7b8806c0a0 2023-10-10T20:35:04.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.169+0000 7f7b8e861700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b880c9900 con 0x7f7b8806bcd0 2023-10-10T20:35:04.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.169+0000 7f7b8e861700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b8815b690 con 0x7f7b880c9510 2023-10-10T20:35:04.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.170+0000 7f7b87fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7b8806bcd0 0x7f7b88140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.170+0000 7f7b87fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7b8806bcd0 0x7f7b88140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59410/0 (socket says 172.21.15.89:59410) 2023-10-10T20:35:04.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.170+0000 7f7b87fff700 1 -- 172.21.15.89:0/860558266 learned_addr learned my addr 172.21.15.89:0/860558266 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:04.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.170+0000 7f7b8cdfe700 1 --2- 172.21.15.89:0/860558266 >> v2:172.21.15.111:3300/0 conn(0x7f7b880c9510 0x7f7b88141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.170+0000 7f7b87fff700 1 -- 172.21.15.89:0/860558266 >> v2:172.21.15.111:3300/0 conn(0x7f7b880c9510 msgr2=0x7f7b88141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:04.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.170+0000 7f7b87fff700 1 --2- 172.21.15.89:0/860558266 >> v2:172.21.15.111:3300/0 conn(0x7f7b880c9510 0x7f7b88141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.170+0000 7f7b87fff700 1 -- 172.21.15.89:0/860558266 >> v1:172.21.15.111:6789/0 conn(0x7f7b8806c0a0 legacy=0x7f7b880c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:04.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.170+0000 7f7b87fff700 1 -- 172.21.15.89:0/860558266 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8815b970 con 0x7f7b8806bcd0 2023-10-10T20:35:04.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.170+0000 7f7b87fff700 1 --2- 172.21.15.89:0/860558266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b8806bcd0 0x7f7b88140df0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7b880ce190 tx=0x7f7b7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d506b5480597a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:04.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.170+0000 7f7b86ffd700 1 -- 172.21.15.89:0/860558266 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c009e70 con 0x7f7b8806bcd0 2023-10-10T20:35:04.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.170+0000 7f7b86ffd700 1 -- 172.21.15.89:0/860558266 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7c00b860 con 0x7f7b8806bcd0 2023-10-10T20:35:04.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.170+0000 7f7b86ffd700 1 -- 172.21.15.89:0/860558266 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c00bc00 con 0x7f7b8806bcd0 2023-10-10T20:35:04.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.171+0000 7f7b8e861700 1 -- 172.21.15.89:0/860558266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b8806bcd0 msgr2=0x7f7b88140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:04.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.171+0000 7f7b8e861700 1 --2- 172.21.15.89:0/860558266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b8806bcd0 0x7f7b88140df0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7b880ce190 tx=0x7f7b7c009250 comp rx=0 tx=0).stop 2023-10-10T20:35:04.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.171+0000 7f7b8e861700 1 -- 172.21.15.89:0/860558266 shutdown_connections 2023-10-10T20:35:04.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.171+0000 7f7b8e861700 1 --2- 172.21.15.89:0/860558266 >> v2:172.21.15.111:3300/0 conn(0x7f7b880c9510 0x7f7b88141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.171+0000 7f7b8e861700 1 --2- 172.21.15.89:0/860558266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b8806bcd0 0x7f7b88140df0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.171+0000 7f7b8e861700 1 -- 172.21.15.89:0/860558266 >> 172.21.15.89:0/860558266 conn(0x7f7b880ce600 msgr2=0x7f7b880cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.171+0000 7f7b8e861700 1 -- 172.21.15.89:0/860558266 shutdown_connections 2023-10-10T20:35:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.171+0000 7f7b8e861700 1 -- 172.21.15.89:0/860558266 wait complete. 2023-10-10T20:35:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.172+0000 7f7b8e861700 1 Processor -- start 2023-10-10T20:35:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.172+0000 7f7b8e861700 1 -- start start 2023-10-10T20:35:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b8e861700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b8806bcd0 0x7f7b8816adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b8e861700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b8806c0a0 0x7f7b880c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b8e861700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7b880c9510 0x7f7b880b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b8e861700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b880b18e0 con 0x7f7b8806bcd0 2023-10-10T20:35:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b8e861700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b880c72f0 con 0x7f7b8806c0a0 2023-10-10T20:35:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b8e861700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b880c75d0 con 0x7f7b880c9510 2023-10-10T20:35:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b87fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b8806bcd0 0x7f7b8816adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b87fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b8806bcd0 0x7f7b8816adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59416/0 (socket says 172.21.15.89:59416) 2023-10-10T20:35:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b87fff700 1 -- 172.21.15.89:0/3300404414 learned_addr learned my addr 172.21.15.89:0/3300404414 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b877fe700 1 --2- 172.21.15.89:0/3300404414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b8806c0a0 0x7f7b880c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b8cdfe700 1 --2- 172.21.15.89:0/3300404414 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7b880c9510 0x7f7b880b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b87fff700 1 -- 172.21.15.89:0/3300404414 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7b880c9510 msgr2=0x7f7b880b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b87fff700 1 --2- 172.21.15.89:0/3300404414 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7b880c9510 0x7f7b880b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b87fff700 1 -- 172.21.15.89:0/3300404414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b8806c0a0 msgr2=0x7f7b880c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b87fff700 1 --2- 172.21.15.89:0/3300404414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b8806c0a0 0x7f7b880c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b87fff700 1 -- 172.21.15.89:0/3300404414 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c0094f0 con 0x7f7b8806bcd0 2023-10-10T20:35:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b87fff700 1 --2- 172.21.15.89:0/3300404414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b8806bcd0 0x7f7b8816adf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7b88069220 tx=0x7f7b7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b6ffff700 1 -- 172.21.15.89:0/3300404414 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c00b9d0 con 0x7f7b8806bcd0 2023-10-10T20:35:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b6ffff700 1 -- 172.21.15.89:0/3300404414 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7c00bb30 con 0x7f7b8806bcd0 2023-10-10T20:35:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b8e861700 1 -- 172.21.15.89:0/3300404414 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b880c78b0 con 0x7f7b8806bcd0 2023-10-10T20:35:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.173+0000 7f7b6ffff700 1 -- 172.21.15.89:0/3300404414 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c025670 con 0x7f7b8806bcd0 2023-10-10T20:35:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.174+0000 7f7b8e861700 1 -- 172.21.15.89:0/3300404414 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b880c6fa0 con 0x7f7b8806bcd0 2023-10-10T20:35:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.174+0000 7f7b8e861700 1 -- 172.21.15.89:0/3300404414 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b880c7d50 con 0x7f7b8806bcd0 2023-10-10T20:35:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.175+0000 7f7b6ffff700 1 -- 172.21.15.89:0/3300404414 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7b7c023070 con 0x7f7b8806bcd0 2023-10-10T20:35:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.175+0000 7f7b6ffff700 1 --2- 172.21.15.89:0/3300404414 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7b74066f30 0x7f7b740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.175+0000 7f7b6ffff700 1 -- 172.21.15.89:0/3300404414 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7b7c085ae0 con 0x7f7b8806bcd0 2023-10-10T20:35:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.175+0000 7f7b877fe700 1 --2- 172.21.15.89:0/3300404414 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7b74066f30 0x7f7b740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.176+0000 7f7b877fe700 1 --2- 172.21.15.89:0/3300404414 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7b74066f30 0x7f7b740693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7b8806da60 tx=0x7f7b78007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.179+0000 7f7b6ffff700 1 -- 172.21.15.89:0/3300404414 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b7c052cb0 con 0x7f7b8806bcd0 2023-10-10T20:35:04.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.361+0000 7f7b8e861700 1 -- 172.21.15.89:0/3300404414 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b880c7f00 con 0x7f7b8806bcd0 2023-10-10T20:35:04.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.362+0000 7f7b6ffff700 1 -- 172.21.15.89:0/3300404414 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f7b7c0562d0 con 0x7f7b8806bcd0 2023-10-10T20:35:04.364 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:04.365 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:36.324920+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:34:32.342167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:34:33.671763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:35:04.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.364+0000 7f7b8e861700 1 -- 172.21.15.89:0/3300404414 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7b74066f30 msgr2=0x7f7b740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:04.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.364+0000 7f7b8e861700 1 --2- 172.21.15.89:0/3300404414 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7b74066f30 0x7f7b740693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7b8806da60 tx=0x7f7b78007ec0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.364+0000 7f7b8e861700 1 -- 172.21.15.89:0/3300404414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b8806bcd0 msgr2=0x7f7b8816adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:04.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.364+0000 7f7b8e861700 1 --2- 172.21.15.89:0/3300404414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b8806bcd0 0x7f7b8816adf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7b88069220 tx=0x7f7b7c017000 comp rx=0 tx=0).stop 2023-10-10T20:35:04.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.364+0000 7f7b8e861700 1 -- 172.21.15.89:0/3300404414 shutdown_connections 2023-10-10T20:35:04.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.364+0000 7f7b8e861700 1 --2- 172.21.15.89:0/3300404414 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7b880c9510 0x7f7b880b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.364+0000 7f7b8e861700 1 --2- 172.21.15.89:0/3300404414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b8806bcd0 0x7f7b8816adf0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.364+0000 7f7b8e861700 1 --2- 172.21.15.89:0/3300404414 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7b74066f30 0x7f7b740693e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.364+0000 7f7b8e861700 1 --2- 172.21.15.89:0/3300404414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b8806c0a0 0x7f7b880c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.364+0000 7f7b8e861700 1 -- 172.21.15.89:0/3300404414 >> 172.21.15.89:0/3300404414 conn(0x7f7b880ce600 msgr2=0x7f7b880cb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:04.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.364+0000 7f7b8e861700 1 -- 172.21.15.89:0/3300404414 shutdown_connections 2023-10-10T20:35:04.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.364+0000 7f7b8e861700 1 -- 172.21.15.89:0/3300404414 wait complete. 2023-10-10T20:35:04.378 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:34:32.342167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:34:33.671763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:35:04.379 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:35:04.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.494+0000 7f623dd45700 1 Processor -- start 2023-10-10T20:35:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.494+0000 7f623dd45700 1 -- start start 2023-10-10T20:35:04.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.495+0000 7f623dd45700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f62380b16d0 0x7f62380b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.495+0000 7f623dd45700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62380b2080 con 0x7f62380b4e70 2023-10-10T20:35:04.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.495+0000 7f623dd45700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62380cfb00 con 0x7f62380b33b0 2023-10-10T20:35:04.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.495+0000 7f623dd45700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62380b5c40 con 0x7f62380b16d0 2023-10-10T20:35:04.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.495+0000 7f623cd43700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f62380b4e70 0x7f62380ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58182/0 (socket says 172.21.15.89:58182) 2023-10-10T20:35:04.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.495+0000 7f623cd43700 1 -- 172.21.15.89:0/2578388091 learned_addr learned my addr 172.21.15.89:0/2578388091 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:04.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.495+0000 7f622ffff700 1 --2- 172.21.15.89:0/2578388091 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f62380b16d0 0x7f62380b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.495+0000 7f622f7fe700 1 -- 172.21.15.89:0/2578388091 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265112326 0 0) 0x7f62380b2080 con 0x7f62380b4e70 2023-10-10T20:35:04.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.495+0000 7f622f7fe700 1 -- 172.21.15.89:0/2578388091 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6220003570 con 0x7f62380b4e70 2023-10-10T20:35:04.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622f7fe700 1 -- 172.21.15.89:0/2578388091 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3660323563 0 0) 0x7f6220003570 con 0x7f62380b4e70 2023-10-10T20:35:04.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622f7fe700 1 -- 172.21.15.89:0/2578388091 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f62380b16d0 msgr2=0x7f62380b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:04.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622f7fe700 1 --2- 172.21.15.89:0/2578388091 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f62380b16d0 0x7f62380b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622f7fe700 1 -- 172.21.15.89:0/2578388091 >> v1:172.21.15.111:6789/0 conn(0x7f62380b33b0 legacy=0x7f62380b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:04.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622f7fe700 1 -- 172.21.15.89:0/2578388091 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62380b5f20 con 0x7f62380b4e70 2023-10-10T20:35:04.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622f7fe700 1 -- 172.21.15.89:0/2578388091 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f62340017f0 con 0x7f62380b4e70 2023-10-10T20:35:04.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622f7fe700 1 -- 172.21.15.89:0/2578388091 >> v1:172.21.15.89:6789/0 conn(0x7f62380b4e70 legacy=0x7f62380ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:04.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622f7fe700 1 --2- 172.21.15.89:0/2578388091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6220003a90 0x7f6220003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622f7fe700 1 --2- 172.21.15.89:0/2578388091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6220004810 0x7f6220004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622f7fe700 1 --2- 172.21.15.89:0/2578388091 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f62380b16d0 0x7f6220005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622f7fe700 1 -- 172.21.15.89:0/2578388091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62380b5c40 con 0x7f6220004810 2023-10-10T20:35:04.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622f7fe700 1 -- 172.21.15.89:0/2578388091 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622000bb90 con 0x7f6220003a90 2023-10-10T20:35:04.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622f7fe700 1 -- 172.21.15.89:0/2578388091 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622000bd50 con 0x7f62380b16d0 2023-10-10T20:35:04.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622ffff700 1 --2- 172.21.15.89:0/2578388091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6220004810 0x7f6220004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f623cd43700 1 --2- 172.21.15.89:0/2578388091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6220003a90 0x7f6220003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f623d544700 1 --2- 172.21.15.89:0/2578388091 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f62380b16d0 0x7f6220005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622ffff700 1 -- 172.21.15.89:0/2578388091 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f62380b16d0 msgr2=0x7f6220005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:04.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622ffff700 1 --2- 172.21.15.89:0/2578388091 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f62380b16d0 0x7f6220005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622ffff700 1 -- 172.21.15.89:0/2578388091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6220003a90 msgr2=0x7f6220003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:04.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622ffff700 1 --2- 172.21.15.89:0/2578388091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6220003a90 0x7f6220003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622ffff700 1 -- 172.21.15.89:0/2578388091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6220003570 con 0x7f6220004810 2023-10-10T20:35:04.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622ffff700 1 --2- 172.21.15.89:0/2578388091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6220004810 0x7f6220004c80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6228009c00 tx=0x7f62280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb1abb194392036 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:04.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622f7fe700 1 -- 172.21.15.89:0/2578388091 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622800e710 con 0x7f6220004810 2023-10-10T20:35:04.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622f7fe700 1 -- 172.21.15.89:0/2578388091 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622800e870 con 0x7f6220004810 2023-10-10T20:35:04.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.496+0000 7f622f7fe700 1 -- 172.21.15.89:0/2578388091 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622800ec30 con 0x7f6220004810 2023-10-10T20:35:04.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.497+0000 7f623dd45700 1 -- 172.21.15.89:0/2578388091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6220004810 msgr2=0x7f6220004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:04.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.497+0000 7f623dd45700 1 --2- 172.21.15.89:0/2578388091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6220004810 0x7f6220004c80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6228009c00 tx=0x7f62280091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.497+0000 7f623d544700 1 -- 172.21.15.89:0/2578388091 reap_dead start 2023-10-10T20:35:04.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.497+0000 7f623dd45700 1 -- 172.21.15.89:0/2578388091 shutdown_connections 2023-10-10T20:35:04.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.497+0000 7f623dd45700 1 -- 172.21.15.89:0/2578388091 >> 172.21.15.89:0/2578388091 conn(0x7f62380d10c0 msgr2=0x7f62380affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:04.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.497+0000 7f623dd45700 1 -- 172.21.15.89:0/2578388091 shutdown_connections 2023-10-10T20:35:04.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.497+0000 7f623dd45700 1 -- 172.21.15.89:0/2578388091 wait complete. 2023-10-10T20:35:04.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.498+0000 7f623dd45700 1 Processor -- start 2023-10-10T20:35:04.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.498+0000 7f623dd45700 1 -- start start 2023-10-10T20:35:04.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.499+0000 7f623dd45700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6220004810 0x7f623814e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.499+0000 7f623dd45700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f623814dfc0 0x7f62380aa330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.499+0000 7f623dd45700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f623814e430 0x7f62380aa870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.499+0000 7f623dd45700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62380a8c10 con 0x7f6220004810 2023-10-10T20:35:04.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.499+0000 7f623dd45700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623814ee60 con 0x7f623814dfc0 2023-10-10T20:35:04.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.499+0000 7f623dd45700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62380a8000 con 0x7f623814e430 2023-10-10T20:35:04.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.499+0000 7f623cd43700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6220004810 0x7f623814e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.499+0000 7f623cd43700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6220004810 0x7f623814e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59442/0 (socket says 172.21.15.89:59442) 2023-10-10T20:35:04.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.499+0000 7f623cd43700 1 -- 172.21.15.89:0/1428972440 learned_addr learned my addr 172.21.15.89:0/1428972440 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:04.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.499+0000 7f622ffff700 1 --2- 172.21.15.89:0/1428972440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f623814dfc0 0x7f62380aa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.499+0000 7f623d544700 1 --2- 172.21.15.89:0/1428972440 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f623814e430 0x7f62380aa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.499+0000 7f623cd43700 1 -- 172.21.15.89:0/1428972440 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f623814e430 msgr2=0x7f62380aa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:04.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.499+0000 7f623cd43700 1 --2- 172.21.15.89:0/1428972440 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f623814e430 0x7f62380aa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.499+0000 7f623cd43700 1 -- 172.21.15.89:0/1428972440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f623814dfc0 msgr2=0x7f62380aa330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:04.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.499+0000 7f623cd43700 1 --2- 172.21.15.89:0/1428972440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f623814dfc0 0x7f62380aa330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.499+0000 7f623cd43700 1 -- 172.21.15.89:0/1428972440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62380a82e0 con 0x7f6220004810 2023-10-10T20:35:04.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.499+0000 7f623cd43700 1 --2- 172.21.15.89:0/1428972440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6220004810 0x7f623814e920 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6234007ac0 tx=0x7f62340021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:04.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.500+0000 7f622cff9700 1 -- 172.21.15.89:0/1428972440 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623400ad90 con 0x7f6220004810 2023-10-10T20:35:04.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.500+0000 7f622cff9700 1 -- 172.21.15.89:0/1428972440 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62340133f0 con 0x7f6220004810 2023-10-10T20:35:04.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.500+0000 7f622cff9700 1 -- 172.21.15.89:0/1428972440 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234013790 con 0x7f6220004810 2023-10-10T20:35:04.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.500+0000 7f623dd45700 1 -- 172.21.15.89:0/1428972440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62380a85c0 con 0x7f6220004810 2023-10-10T20:35:04.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.500+0000 7f623dd45700 1 -- 172.21.15.89:0/1428972440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62380cbb30 con 0x7f6220004810 2023-10-10T20:35:04.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.500+0000 7f623dd45700 1 -- 172.21.15.89:0/1428972440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6238088990 con 0x7f6220004810 2023-10-10T20:35:04.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.501+0000 7f622cff9700 1 -- 172.21.15.89:0/1428972440 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6234013b40 con 0x7f6220004810 2023-10-10T20:35:04.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.501+0000 7f622cff9700 1 --2- 172.21.15.89:0/1428972440 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6224066f30 0x7f62240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.501+0000 7f622cff9700 1 -- 172.21.15.89:0/1428972440 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6234085390 con 0x7f6220004810 2023-10-10T20:35:04.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.501+0000 7f622ffff700 1 --2- 172.21.15.89:0/1428972440 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6224066f30 0x7f62240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.502+0000 7f622ffff700 1 --2- 172.21.15.89:0/1428972440 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6224066f30 0x7f62240693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f62280091e0 tx=0x7f622801a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:04.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.505+0000 7f622cff9700 1 -- 172.21.15.89:0/1428972440 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6234052600 con 0x7f6220004810 2023-10-10T20:35:04.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.686+0000 7f623dd45700 1 -- 172.21.15.89:0/1428972440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f623804bb80 con 0x7f6220004810 2023-10-10T20:35:04.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.687+0000 7f622cff9700 1 -- 172.21.15.89:0/1428972440 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6234055c20 con 0x7f6220004810 2023-10-10T20:35:04.689 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:35:04.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.689+0000 7f623dd45700 1 -- 172.21.15.89:0/1428972440 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6224066f30 msgr2=0x7f62240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:04.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.689+0000 7f623dd45700 1 --2- 172.21.15.89:0/1428972440 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6224066f30 0x7f62240693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f62280091e0 tx=0x7f622801a000 comp rx=0 tx=0).stop 2023-10-10T20:35:04.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.689+0000 7f623dd45700 1 -- 172.21.15.89:0/1428972440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6220004810 msgr2=0x7f623814e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:04.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.689+0000 7f623dd45700 1 --2- 172.21.15.89:0/1428972440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6220004810 0x7f623814e920 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6234007ac0 tx=0x7f62340021f0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.689+0000 7f623dd45700 1 -- 172.21.15.89:0/1428972440 shutdown_connections 2023-10-10T20:35:04.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.689+0000 7f623dd45700 1 --2- 172.21.15.89:0/1428972440 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f623814e430 0x7f62380aa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.689+0000 7f623dd45700 1 --2- 172.21.15.89:0/1428972440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6220004810 0x7f623814e920 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.689+0000 7f623dd45700 1 --2- 172.21.15.89:0/1428972440 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6224066f30 0x7f62240693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.689+0000 7f623dd45700 1 --2- 172.21.15.89:0/1428972440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f623814dfc0 0x7f62380aa330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.689+0000 7f623dd45700 1 -- 172.21.15.89:0/1428972440 >> 172.21.15.89:0/1428972440 conn(0x7f62380d10c0 msgr2=0x7f62380c5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:04.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.689+0000 7f623dd45700 1 -- 172.21.15.89:0/1428972440 shutdown_connections 2023-10-10T20:35:04.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.689+0000 7f623dd45700 1 -- 172.21.15.89:0/1428972440 wait complete. 2023-10-10T20:35:04.703 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:35:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.817+0000 7fa9b232f700 1 Processor -- start 2023-10-10T20:35:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.817+0000 7fa9b232f700 1 -- start start 2023-10-10T20:35:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.818+0000 7fa9b232f700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa9ac159c60 0x7fa9ac15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.818+0000 7fa9b232f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa9ac15db10 0x7fa9ac069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.818+0000 7fa9b232f700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9ac15a160 con 0x7fa9ac159890 2023-10-10T20:35:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.818+0000 7fa9b232f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9ac15df50 con 0x7fa9ac15db10 2023-10-10T20:35:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.818+0000 7fa9b232f700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9ac0d13a0 con 0x7fa9ac159c60 2023-10-10T20:35:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.818+0000 7fa9b08cc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa9ac15db10 0x7fa9ac069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.818+0000 7fa9b08cc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa9ac15db10 0x7fa9ac069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59448/0 (socket says 172.21.15.89:59448) 2023-10-10T20:35:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.818+0000 7fa9b08cc700 1 -- 172.21.15.89:0/4289457435 learned_addr learned my addr 172.21.15.89:0/4289457435 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.818+0000 7fa9ab7fe700 1 --2- 172.21.15.89:0/4289457435 >> v2:172.21.15.111:3300/0 conn(0x7fa9ac159c60 0x7fa9ac15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.818+0000 7fa9b08cc700 1 -- 172.21.15.89:0/4289457435 >> v2:172.21.15.111:3300/0 conn(0x7fa9ac159c60 msgr2=0x7fa9ac15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.818+0000 7fa9b08cc700 1 --2- 172.21.15.89:0/4289457435 >> v2:172.21.15.111:3300/0 conn(0x7fa9ac159c60 0x7fa9ac15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.818+0000 7fa9b08cc700 1 -- 172.21.15.89:0/4289457435 >> v1:172.21.15.111:6789/0 conn(0x7fa9ac159890 legacy=0x7fa9ac15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.818+0000 7fa9b08cc700 1 -- 172.21.15.89:0/4289457435 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac15e140 con 0x7fa9ac15db10 2023-10-10T20:35:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.819+0000 7fa9ab7fe700 1 --2- 172.21.15.89:0/4289457435 >> v2:172.21.15.111:3300/0 conn(0x7fa9ac159c60 0x7fa9ac15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.819+0000 7fa9b08cc700 1 --2- 172.21.15.89:0/4289457435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac15db10 0x7fa9ac069560 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0b67f0 tx=0x7fa9a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4751f3ed0d841943 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.819+0000 7fa9aaffd700 1 -- 172.21.15.89:0/4289457435 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a400e070 con 0x7fa9ac15db10 2023-10-10T20:35:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.819+0000 7fa9aaffd700 1 -- 172.21.15.89:0/4289457435 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a400ba10 con 0x7fa9ac15db10 2023-10-10T20:35:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.819+0000 7fa9aaffd700 1 -- 172.21.15.89:0/4289457435 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a40092e0 con 0x7fa9ac15db10 2023-10-10T20:35:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.819+0000 7fa9b232f700 1 -- 172.21.15.89:0/4289457435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac15db10 msgr2=0x7fa9ac069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.819+0000 7fa9b232f700 1 --2- 172.21.15.89:0/4289457435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac15db10 0x7fa9ac069560 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0b67f0 tx=0x7fa9a4009840 comp rx=0 tx=0).stop 2023-10-10T20:35:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.819+0000 7fa9b232f700 1 -- 172.21.15.89:0/4289457435 shutdown_connections 2023-10-10T20:35:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.819+0000 7fa9b232f700 1 --2- 172.21.15.89:0/4289457435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac15db10 0x7fa9ac069560 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.819+0000 7fa9b232f700 1 --2- 172.21.15.89:0/4289457435 >> v2:172.21.15.111:3300/0 conn(0x7fa9ac159c60 0x7fa9ac15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.819+0000 7fa9b232f700 1 -- 172.21.15.89:0/4289457435 >> 172.21.15.89:0/4289457435 conn(0x7fa9ac068e20 msgr2=0x7fa9ac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.819+0000 7fa9b232f700 1 -- 172.21.15.89:0/4289457435 shutdown_connections 2023-10-10T20:35:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.819+0000 7fa9b232f700 1 -- 172.21.15.89:0/4289457435 wait complete. 2023-10-10T20:35:04.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.820+0000 7fa9b232f700 1 Processor -- start 2023-10-10T20:35:04.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.821+0000 7fa9b232f700 1 -- start start 2023-10-10T20:35:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.821+0000 7fa9b232f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9ac159890 0x7fa9ac15d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.821+0000 7fa9b232f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac159c60 0x7fa9ac0cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.821+0000 7fa9b232f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9ac15db10 0x7fa9ac151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.821+0000 7fa9b232f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0cfdf0 con 0x7fa9ac159c60 2023-10-10T20:35:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.821+0000 7fa9b232f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0cd0c0 con 0x7fa9ac159890 2023-10-10T20:35:04.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.821+0000 7fa9b232f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac151a80 con 0x7fa9ac15db10 2023-10-10T20:35:04.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.821+0000 7fa9ab7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac159c60 0x7fa9ac0cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.821+0000 7fa9ab7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac159c60 0x7fa9ac0cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59462/0 (socket says 172.21.15.89:59462) 2023-10-10T20:35:04.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.821+0000 7fa9ab7fe700 1 -- 172.21.15.89:0/2464864414 learned_addr learned my addr 172.21.15.89:0/2464864414 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:04.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.821+0000 7fa9b08cc700 1 --2- 172.21.15.89:0/2464864414 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9ac15db10 0x7fa9ac151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.821+0000 7fa9abfff700 1 --2- 172.21.15.89:0/2464864414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9ac159890 0x7fa9ac15d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.822+0000 7fa9ab7fe700 1 -- 172.21.15.89:0/2464864414 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9ac15db10 msgr2=0x7fa9ac151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.822+0000 7fa9ab7fe700 1 --2- 172.21.15.89:0/2464864414 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9ac15db10 0x7fa9ac151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.822+0000 7fa9ab7fe700 1 -- 172.21.15.89:0/2464864414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9ac159890 msgr2=0x7fa9ac15d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.822+0000 7fa9ab7fe700 1 --2- 172.21.15.89:0/2464864414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9ac159890 0x7fa9ac15d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.822+0000 7fa9ab7fe700 1 -- 172.21.15.89:0/2464864414 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a4009ae0 con 0x7fa9ac159c60 2023-10-10T20:35:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.822+0000 7fa9ab7fe700 1 --2- 172.21.15.89:0/2464864414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac159c60 0x7fa9ac0cf8b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000c830 tx=0x7fa9a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.822+0000 7fa993fff700 1 -- 172.21.15.89:0/2464864414 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0011600 con 0x7fa9ac159c60 2023-10-10T20:35:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.822+0000 7fa993fff700 1 -- 172.21.15.89:0/2464864414 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a0011760 con 0x7fa9ac159c60 2023-10-10T20:35:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.822+0000 7fa9b232f700 1 -- 172.21.15.89:0/2464864414 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9ac0cd310 con 0x7fa9ac159c60 2023-10-10T20:35:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.822+0000 7fa993fff700 1 -- 172.21.15.89:0/2464864414 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0011b20 con 0x7fa9ac159c60 2023-10-10T20:35:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.822+0000 7fa9b232f700 1 -- 172.21.15.89:0/2464864414 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ac0b0670 con 0x7fa9ac159c60 2023-10-10T20:35:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.823+0000 7fa9b232f700 1 -- 172.21.15.89:0/2464864414 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9ac0b0890 con 0x7fa9ac159c60 2023-10-10T20:35:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.823+0000 7fa993fff700 1 -- 172.21.15.89:0/2464864414 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa9a0018360 con 0x7fa9ac159c60 2023-10-10T20:35:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.824+0000 7fa993fff700 1 --2- 172.21.15.89:0/2464864414 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa998066f30 0x7fa9980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.824+0000 7fa993fff700 1 -- 172.21.15.89:0/2464864414 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa9a0084430 con 0x7fa9ac159c60 2023-10-10T20:35:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.824+0000 7fa9abfff700 1 --2- 172.21.15.89:0/2464864414 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa998066f30 0x7fa9980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.824+0000 7fa9abfff700 1 --2- 172.21.15.89:0/2464864414 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa998066f30 0x7fa9980693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac063ed0 tx=0x7fa99c007ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:04.828+0000 7fa993fff700 1 -- 172.21.15.89:0/2464864414 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9a00516a0 con 0x7fa9ac159c60 2023-10-10T20:35:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.013+0000 7fa9b232f700 1 -- 172.21.15.89:0/2464864414 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa9ac04c3c0 con 0x7fa9ac159c60 2023-10-10T20:35:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.013+0000 7fa993fff700 1 -- 172.21.15.89:0/2464864414 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa9a0054cc0 con 0x7fa9ac159c60 2023-10-10T20:35:05.016 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:35:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.015+0000 7fa9b232f700 1 -- 172.21.15.89:0/2464864414 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa998066f30 msgr2=0x7fa9980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.015+0000 7fa9b232f700 1 --2- 172.21.15.89:0/2464864414 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa998066f30 0x7fa9980693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac063ed0 tx=0x7fa99c007ea0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.015+0000 7fa9b232f700 1 -- 172.21.15.89:0/2464864414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac159c60 msgr2=0x7fa9ac0cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.015+0000 7fa9b232f700 1 --2- 172.21.15.89:0/2464864414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac159c60 0x7fa9ac0cf8b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000c830 tx=0x7fa9a0008040 comp rx=0 tx=0).stop 2023-10-10T20:35:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.015+0000 7fa9b232f700 1 -- 172.21.15.89:0/2464864414 shutdown_connections 2023-10-10T20:35:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.015+0000 7fa9b232f700 1 --2- 172.21.15.89:0/2464864414 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9ac15db10 0x7fa9ac151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.015+0000 7fa9b232f700 1 --2- 172.21.15.89:0/2464864414 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa998066f30 0x7fa9980693e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.015+0000 7fa9b232f700 1 --2- 172.21.15.89:0/2464864414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9ac159890 0x7fa9ac15d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.015+0000 7fa9b232f700 1 --2- 172.21.15.89:0/2464864414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac159c60 0x7fa9ac0cf8b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.015+0000 7fa9b232f700 1 -- 172.21.15.89:0/2464864414 >> 172.21.15.89:0/2464864414 conn(0x7fa9ac068e20 msgr2=0x7fa9ac156f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.016+0000 7fa9b232f700 1 -- 172.21.15.89:0/2464864414 shutdown_connections 2023-10-10T20:35:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.016+0000 7fa9b232f700 1 -- 172.21.15.89:0/2464864414 wait complete. 2023-10-10T20:35:05.030 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:35:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.153+0000 7f725706f700 1 Processor -- start 2023-10-10T20:35:05.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.153+0000 7f725706f700 1 -- start start 2023-10-10T20:35:05.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.153+0000 7f725706f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f725006bcd0 0x7f72500caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.153+0000 7f725706f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f725015df50 0x7f725015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.153+0000 7f725706f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725015d7e0 con 0x7f725006c0a0 2023-10-10T20:35:05.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.153+0000 7f725706f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725015dac0 con 0x7f725015df50 2023-10-10T20:35:05.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.153+0000 7f725706f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725015aa30 con 0x7f725006bcd0 2023-10-10T20:35:05.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.153+0000 7f725560c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f725015df50 0x7f725015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.154+0000 7f724ffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f725006c0a0 0x7f725015a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58188/0 (socket says 172.21.15.89:58188) 2023-10-10T20:35:05.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.154+0000 7f724ffff700 1 -- 172.21.15.89:0/21831904 learned_addr learned my addr 172.21.15.89:0/21831904 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:05.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.154+0000 7f7254e0b700 1 --2- 172.21.15.89:0/21831904 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f725006bcd0 0x7f72500caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.154+0000 7f725560c700 1 -- 172.21.15.89:0/21831904 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f725006bcd0 msgr2=0x7f72500caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:05.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.154+0000 7f725560c700 1 --2- 172.21.15.89:0/21831904 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f725006bcd0 0x7f72500caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.154+0000 7f725560c700 1 -- 172.21.15.89:0/21831904 >> v1:172.21.15.89:6789/0 conn(0x7f725006c0a0 legacy=0x7f725015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:05.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.154+0000 7f725560c700 1 -- 172.21.15.89:0/21831904 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725015ad10 con 0x7f725015df50 2023-10-10T20:35:05.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.154+0000 7f7254e0b700 1 --2- 172.21.15.89:0/21831904 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f725006bcd0 0x7f72500caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:05.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.154+0000 7f725560c700 1 --2- 172.21.15.89:0/21831904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725015df50 0x7f725015e3b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72480098e0 tx=0x7f72480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4f3766d64390cb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:05.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.154+0000 7f724f7fe700 1 -- 172.21.15.89:0/21831904 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7248009e80 con 0x7f725015df50 2023-10-10T20:35:05.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.154+0000 7f724f7fe700 1 -- 172.21.15.89:0/21831904 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724800b860 con 0x7f725015df50 2023-10-10T20:35:05.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.154+0000 7f724f7fe700 1 -- 172.21.15.89:0/21831904 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724800bbe0 con 0x7f725015df50 2023-10-10T20:35:05.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.155+0000 7f725706f700 1 -- 172.21.15.89:0/21831904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725015df50 msgr2=0x7f725015e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:05.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.155+0000 7f725706f700 1 --2- 172.21.15.89:0/21831904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725015df50 0x7f725015e3b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72480098e0 tx=0x7f72480091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.155+0000 7f725706f700 1 -- 172.21.15.89:0/21831904 shutdown_connections 2023-10-10T20:35:05.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.155+0000 7f725706f700 1 --2- 172.21.15.89:0/21831904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725015df50 0x7f725015e3b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.155+0000 7f725706f700 1 --2- 172.21.15.89:0/21831904 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f725006bcd0 0x7f72500caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.155+0000 7f725706f700 1 -- 172.21.15.89:0/21831904 >> 172.21.15.89:0/21831904 conn(0x7f72500ce600 msgr2=0x7f72500cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:05.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.155+0000 7f725706f700 1 -- 172.21.15.89:0/21831904 shutdown_connections 2023-10-10T20:35:05.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.155+0000 7f725706f700 1 -- 172.21.15.89:0/21831904 wait complete. 2023-10-10T20:35:05.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.156+0000 7f725706f700 1 Processor -- start 2023-10-10T20:35:05.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.156+0000 7f725706f700 1 -- start start 2023-10-10T20:35:05.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.156+0000 7f725706f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725006bcd0 0x7f7250156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f725706f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f725006c0a0 0x7f72500c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f725706f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725015df50 0x7f72500696f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f725706f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7250156e50 con 0x7f725006bcd0 2023-10-10T20:35:05.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f725706f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72500b2520 con 0x7f725015df50 2023-10-10T20:35:05.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f725706f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72500c69f0 con 0x7f725006c0a0 2023-10-10T20:35:05.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f7254e0b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725006bcd0 0x7f7250156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f7254e0b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725006bcd0 0x7f7250156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59482/0 (socket says 172.21.15.89:59482) 2023-10-10T20:35:05.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f7254e0b700 1 -- 172.21.15.89:0/2839873398 learned_addr learned my addr 172.21.15.89:0/2839873398 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:05.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f724ffff700 1 --2- 172.21.15.89:0/2839873398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f725006c0a0 0x7f72500c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f725560c700 1 --2- 172.21.15.89:0/2839873398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725015df50 0x7f72500696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f7254e0b700 1 -- 172.21.15.89:0/2839873398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f725006c0a0 msgr2=0x7f72500c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:05.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f7254e0b700 1 --2- 172.21.15.89:0/2839873398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f725006c0a0 0x7f72500c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f7254e0b700 1 -- 172.21.15.89:0/2839873398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725015df50 msgr2=0x7f72500696f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:05.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f7254e0b700 1 --2- 172.21.15.89:0/2839873398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725015df50 0x7f72500696f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f7254e0b700 1 -- 172.21.15.89:0/2839873398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7248009420 con 0x7f725006bcd0 2023-10-10T20:35:05.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f725560c700 1 --2- 172.21.15.89:0/2839873398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725015df50 0x7f72500696f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:05.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f7254e0b700 1 --2- 172.21.15.89:0/2839873398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725006bcd0 0x7f7250156910 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7240007f90 tx=0x7f7240007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:05.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f724cff9700 1 -- 172.21.15.89:0/2839873398 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7240016420 con 0x7f725006bcd0 2023-10-10T20:35:05.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f724cff9700 1 -- 172.21.15.89:0/2839873398 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7240016580 con 0x7f725006bcd0 2023-10-10T20:35:05.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f725706f700 1 -- 172.21.15.89:0/2839873398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72500b27e0 con 0x7f725006bcd0 2023-10-10T20:35:05.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f724cff9700 1 -- 172.21.15.89:0/2839873398 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7240016940 con 0x7f725006bcd0 2023-10-10T20:35:05.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.157+0000 7f725706f700 1 -- 172.21.15.89:0/2839873398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72500c6430 con 0x7f725006bcd0 2023-10-10T20:35:05.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.158+0000 7f725706f700 1 -- 172.21.15.89:0/2839873398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f725004c3c0 con 0x7f725006bcd0 2023-10-10T20:35:05.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.158+0000 7f724cff9700 1 -- 172.21.15.89:0/2839873398 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7240016cf0 con 0x7f725006bcd0 2023-10-10T20:35:05.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.159+0000 7f724cff9700 1 --2- 172.21.15.89:0/2839873398 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f723c066f30 0x7f723c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.159+0000 7f724cff9700 1 -- 172.21.15.89:0/2839873398 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7240012070 con 0x7f725006bcd0 2023-10-10T20:35:05.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.159+0000 7f724ffff700 1 --2- 172.21.15.89:0/2839873398 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f723c066f30 0x7f723c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.159+0000 7f724ffff700 1 --2- 172.21.15.89:0/2839873398 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f723c066f30 0x7f723c0693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f72500b2310 tx=0x7f7244009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:05.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.163+0000 7f724cff9700 1 -- 172.21.15.89:0/2839873398 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f724004e5a0 con 0x7f725006bcd0 2023-10-10T20:35:05.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.344+0000 7f725706f700 1 -- 172.21.15.89:0/2839873398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f725015aff0 con 0x7f725006bcd0 2023-10-10T20:35:05.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.345+0000 7f724cff9700 1 -- 172.21.15.89:0/2839873398 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7240088030 con 0x7f725006bcd0 2023-10-10T20:35:05.347 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-10T20:35:05.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.346+0000 7f725706f700 1 -- 172.21.15.89:0/2839873398 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f723c066f30 msgr2=0x7f723c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:05.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.346+0000 7f725706f700 1 --2- 172.21.15.89:0/2839873398 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f723c066f30 0x7f723c0693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f72500b2310 tx=0x7f7244009040 comp rx=0 tx=0).stop 2023-10-10T20:35:05.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.346+0000 7f725706f700 1 -- 172.21.15.89:0/2839873398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725006bcd0 msgr2=0x7f7250156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:05.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.346+0000 7f725706f700 1 --2- 172.21.15.89:0/2839873398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725006bcd0 0x7f7250156910 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7240007f90 tx=0x7f7240007870 comp rx=0 tx=0).stop 2023-10-10T20:35:05.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.346+0000 7f725706f700 1 -- 172.21.15.89:0/2839873398 shutdown_connections 2023-10-10T20:35:05.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.347+0000 7f725706f700 1 --2- 172.21.15.89:0/2839873398 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f723c066f30 0x7f723c0693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.347+0000 7f725706f700 1 --2- 172.21.15.89:0/2839873398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725015df50 0x7f72500696f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.347+0000 7f725706f700 1 --2- 172.21.15.89:0/2839873398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725006bcd0 0x7f7250156910 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.347+0000 7f725706f700 1 --2- 172.21.15.89:0/2839873398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f725006c0a0 0x7f72500c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.347+0000 7f725706f700 1 -- 172.21.15.89:0/2839873398 >> 172.21.15.89:0/2839873398 conn(0x7f72500ce600 msgr2=0x7f72500cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:05.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.347+0000 7f725706f700 1 -- 172.21.15.89:0/2839873398 shutdown_connections 2023-10-10T20:35:05.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.347+0000 7f725706f700 1 -- 172.21.15.89:0/2839873398 wait complete. 2023-10-10T20:35:05.361 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:35:05.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.473+0000 7f48399d9700 1 Processor -- start 2023-10-10T20:35:05.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.473+0000 7f48399d9700 1 -- start start 2023-10-10T20:35:05.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.474+0000 7f48399d9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f483415d230 0x7f483415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.474+0000 7f48399d9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48340b2980 con 0x7f48341597f0 2023-10-10T20:35:05.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.474+0000 7f48399d9700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483415dbb0 con 0x7f4834159420 2023-10-10T20:35:05.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.474+0000 7f48399d9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48340b11d0 con 0x7f483415d230 2023-10-10T20:35:05.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.474+0000 7f482bfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f48341597f0 0x7f48340b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58202/0 (socket says 172.21.15.89:58202) 2023-10-10T20:35:05.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.474+0000 7f482bfff700 1 -- 172.21.15.89:0/276932830 learned_addr learned my addr 172.21.15.89:0/276932830 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:05.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.474+0000 7f48391d8700 1 --2- 172.21.15.89:0/276932830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f483415d230 0x7f483415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.474+0000 7f482b7fe700 1 -- 172.21.15.89:0/276932830 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2112330274 0 0) 0x7f48340b2980 con 0x7f48341597f0 2023-10-10T20:35:05.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.474+0000 7f482b7fe700 1 -- 172.21.15.89:0/276932830 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f481c003570 con 0x7f48341597f0 2023-10-10T20:35:05.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.474+0000 7f482b7fe700 1 -- 172.21.15.89:0/276932830 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940700082 0 0) 0x7f483415dbb0 con 0x7f4834159420 2023-10-10T20:35:05.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.474+0000 7f482b7fe700 1 -- 172.21.15.89:0/276932830 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48340b2980 con 0x7f4834159420 2023-10-10T20:35:05.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.474+0000 7f482b7fe700 1 -- 172.21.15.89:0/276932830 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1552771526 0 0) 0x7f481c003570 con 0x7f48341597f0 2023-10-10T20:35:05.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.474+0000 7f482b7fe700 1 -- 172.21.15.89:0/276932830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f483415d230 msgr2=0x7f483415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:05.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.474+0000 7f482b7fe700 1 --2- 172.21.15.89:0/276932830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f483415d230 0x7f483415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f482b7fe700 1 -- 172.21.15.89:0/276932830 >> v1:172.21.15.111:6789/0 conn(0x7f4834159420 legacy=0x7f48340b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:05.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f482b7fe700 1 -- 172.21.15.89:0/276932830 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48340b14b0 con 0x7f48341597f0 2023-10-10T20:35:05.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f482b7fe700 1 -- 172.21.15.89:0/276932830 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f4824001470 con 0x7f48341597f0 2023-10-10T20:35:05.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f482b7fe700 1 -- 172.21.15.89:0/276932830 >> v1:172.21.15.89:6789/0 conn(0x7f48341597f0 legacy=0x7f48340b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:05.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f482b7fe700 1 --2- 172.21.15.89:0/276932830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f481c003c20 0x7f481c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f482b7fe700 1 --2- 172.21.15.89:0/276932830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f481c004810 0x7f481c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f482b7fe700 1 --2- 172.21.15.89:0/276932830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f481c0051c0 0x7f481c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f482b7fe700 1 -- 172.21.15.89:0/276932830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48340b11d0 con 0x7f481c003c20 2023-10-10T20:35:05.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f482b7fe700 1 -- 172.21.15.89:0/276932830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c00c020 con 0x7f481c0051c0 2023-10-10T20:35:05.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f482b7fe700 1 -- 172.21.15.89:0/276932830 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481c00c1e0 con 0x7f481c004810 2023-10-10T20:35:05.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f48391d8700 1 --2- 172.21.15.89:0/276932830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f483415d230 0x7f483415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:35:05.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f48389d7700 1 --2- 172.21.15.89:0/276932830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f481c003c20 0x7f481c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f482bfff700 1 --2- 172.21.15.89:0/276932830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f481c004810 0x7f481c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f48391d8700 1 --2- 172.21.15.89:0/276932830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f481c0051c0 0x7f481c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f48389d7700 1 -- 172.21.15.89:0/276932830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f481c004810 msgr2=0x7f481c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:05.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f48389d7700 1 --2- 172.21.15.89:0/276932830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f481c004810 0x7f481c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f48389d7700 1 -- 172.21.15.89:0/276932830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f481c0051c0 msgr2=0x7f481c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:05.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f48389d7700 1 --2- 172.21.15.89:0/276932830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f481c0051c0 0x7f481c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f48389d7700 1 -- 172.21.15.89:0/276932830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c003570 con 0x7f481c003c20 2023-10-10T20:35:05.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f48389d7700 1 --2- 172.21.15.89:0/276932830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f481c003c20 0x7f481c004010 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4830002150 tx=0x7f4830001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab053c211d83ef2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:05.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f482b7fe700 1 -- 172.21.15.89:0/276932830 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830001b10 con 0x7f481c003c20 2023-10-10T20:35:05.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f482b7fe700 1 -- 172.21.15.89:0/276932830 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483000c980 con 0x7f481c003c20 2023-10-10T20:35:05.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.475+0000 7f482b7fe700 1 -- 172.21.15.89:0/276932830 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483000cd40 con 0x7f481c003c20 2023-10-10T20:35:05.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.476+0000 7f48399d9700 1 -- 172.21.15.89:0/276932830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f481c003c20 msgr2=0x7f481c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:05.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.476+0000 7f48399d9700 1 --2- 172.21.15.89:0/276932830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f481c003c20 0x7f481c004010 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4830002150 tx=0x7f4830001800 comp rx=0 tx=0).stop 2023-10-10T20:35:05.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.476+0000 7f48391d8700 1 -- 172.21.15.89:0/276932830 reap_dead start 2023-10-10T20:35:05.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.476+0000 7f48399d9700 1 -- 172.21.15.89:0/276932830 shutdown_connections 2023-10-10T20:35:05.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.476+0000 7f48399d9700 1 -- 172.21.15.89:0/276932830 >> 172.21.15.89:0/276932830 conn(0x7f48340d0660 msgr2=0x7f48340d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:05.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.476+0000 7f48399d9700 1 -- 172.21.15.89:0/276932830 shutdown_connections 2023-10-10T20:35:05.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.476+0000 7f48399d9700 1 -- 172.21.15.89:0/276932830 wait complete. 2023-10-10T20:35:05.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.477+0000 7f48399d9700 1 Processor -- start 2023-10-10T20:35:05.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.477+0000 7f48399d9700 1 -- start start 2023-10-10T20:35:05.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.478+0000 7f48399d9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f481c003c20 0x7f483414e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.478+0000 7f48399d9700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48341699c0 0x7f483414ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.478+0000 7f48399d9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f483415c250 0x7f483415c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.478+0000 7f48399d9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483415cc10 con 0x7f481c003c20 2023-10-10T20:35:05.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.478+0000 7f48399d9700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483416cfe0 con 0x7f48341699c0 2023-10-10T20:35:05.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.478+0000 7f48399d9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483416a230 con 0x7f483415c250 2023-10-10T20:35:05.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.478+0000 7f48389d7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f481c003c20 0x7f483414e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.478+0000 7f48389d7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f481c003c20 0x7f483414e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59490/0 (socket says 172.21.15.89:59490) 2023-10-10T20:35:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.478+0000 7f48389d7700 1 -- 172.21.15.89:0/3184561578 learned_addr learned my addr 172.21.15.89:0/3184561578 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.478+0000 7f482bfff700 1 --2- 172.21.15.89:0/3184561578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48341699c0 0x7f483414ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.478+0000 7f48391d8700 1 --2- 172.21.15.89:0/3184561578 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f483415c250 0x7f483415c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.478+0000 7f48389d7700 1 -- 172.21.15.89:0/3184561578 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f483415c250 msgr2=0x7f483415c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.478+0000 7f48389d7700 1 --2- 172.21.15.89:0/3184561578 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f483415c250 0x7f483415c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.478+0000 7f48389d7700 1 -- 172.21.15.89:0/3184561578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48341699c0 msgr2=0x7f483414ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.478+0000 7f48389d7700 1 --2- 172.21.15.89:0/3184561578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48341699c0 0x7f483414ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.478+0000 7f48389d7700 1 -- 172.21.15.89:0/3184561578 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483416a510 con 0x7f481c003c20 2023-10-10T20:35:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.479+0000 7f48389d7700 1 --2- 172.21.15.89:0/3184561578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f481c003c20 0x7f483414e4d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f483000c6a0 tx=0x7f4830016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.479+0000 7f4828ff9700 1 -- 172.21.15.89:0/3184561578 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830001b10 con 0x7f481c003c20 2023-10-10T20:35:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.479+0000 7f4828ff9700 1 -- 172.21.15.89:0/3184561578 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4830012e90 con 0x7f481c003c20 2023-10-10T20:35:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.479+0000 7f4828ff9700 1 -- 172.21.15.89:0/3184561578 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483001e650 con 0x7f481c003c20 2023-10-10T20:35:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.479+0000 7f48399d9700 1 -- 172.21.15.89:0/3184561578 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f483416a7f0 con 0x7f481c003c20 2023-10-10T20:35:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.479+0000 7f48399d9700 1 -- 172.21.15.89:0/3184561578 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f483414cf70 con 0x7f481c003c20 2023-10-10T20:35:05.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.480+0000 7f48399d9700 1 -- 172.21.15.89:0/3184561578 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f483416d4e0 con 0x7f481c003c20 2023-10-10T20:35:05.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.480+0000 7f4828ff9700 1 -- 172.21.15.89:0/3184561578 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4830025020 con 0x7f481c003c20 2023-10-10T20:35:05.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.480+0000 7f4828ff9700 1 --2- 172.21.15.89:0/3184561578 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4820066f30 0x7f48200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.481+0000 7f4828ff9700 1 -- 172.21.15.89:0/3184561578 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4830086430 con 0x7f481c003c20 2023-10-10T20:35:05.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.481+0000 7f482bfff700 1 --2- 172.21.15.89:0/3184561578 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4820066f30 0x7f48200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.481+0000 7f482bfff700 1 --2- 172.21.15.89:0/3184561578 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4820066f30 0x7f48200693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4824001200 tx=0x7f4824007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:05.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.485+0000 7f4828ff9700 1 -- 172.21.15.89:0/3184561578 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f483416d4e0 con 0x7f481c003c20 2023-10-10T20:35:05.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.709+0000 7f48399d9700 1 -- 172.21.15.89:0/3184561578 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f483416d4e0 con 0x7f481c003c20 2023-10-10T20:35:05.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.710+0000 7f4828ff9700 1 -- 172.21.15.89:0/3184561578 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4597 (secure 0 0 0) 0x7f483416d4e0 con 0x7f481c003c20 2023-10-10T20:35:05.712 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:05.713 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2341577144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2341577144},{"type":"v1","addr":"172.21.15.111:6835","nonce":2341577144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2773123076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2773123076},{"type":"v1","addr":"172.21.15.89:6837","nonce":2773123076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2940044414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2940044414},{"type":"v1","addr":"172.21.15.89:6835","nonce":2940044414}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:34:34.717750+0000","modified":"2023-10-10T20:34:38.356048+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1112081320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1112081320},{"type":"v1","addr":"172.21.15.111:6834","nonce":1112081320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-10T20:35:05.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.712+0000 7f48399d9700 1 -- 172.21.15.89:0/3184561578 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4820066f30 msgr2=0x7f48200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:05.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.712+0000 7f48399d9700 1 --2- 172.21.15.89:0/3184561578 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4820066f30 0x7f48200693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4824001200 tx=0x7f4824007040 comp rx=0 tx=0).stop 2023-10-10T20:35:05.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.712+0000 7f48399d9700 1 -- 172.21.15.89:0/3184561578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f481c003c20 msgr2=0x7f483414e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:05.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.712+0000 7f48399d9700 1 --2- 172.21.15.89:0/3184561578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f481c003c20 0x7f483414e4d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f483000c6a0 tx=0x7f4830016000 comp rx=0 tx=0).stop 2023-10-10T20:35:05.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.712+0000 7f48399d9700 1 -- 172.21.15.89:0/3184561578 shutdown_connections 2023-10-10T20:35:05.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.712+0000 7f48399d9700 1 --2- 172.21.15.89:0/3184561578 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f483415c250 0x7f483415c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.712+0000 7f48399d9700 1 --2- 172.21.15.89:0/3184561578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f481c003c20 0x7f483414e4d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.712+0000 7f48399d9700 1 --2- 172.21.15.89:0/3184561578 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4820066f30 0x7f48200693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.712+0000 7f48399d9700 1 --2- 172.21.15.89:0/3184561578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48341699c0 0x7f483414ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.712+0000 7f48399d9700 1 -- 172.21.15.89:0/3184561578 >> 172.21.15.89:0/3184561578 conn(0x7f48340d0660 msgr2=0x7f48340cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:05.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.712+0000 7f48399d9700 1 -- 172.21.15.89:0/3184561578 shutdown_connections 2023-10-10T20:35:05.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.712+0000 7f48399d9700 1 -- 172.21.15.89:0/3184561578 wait complete. 2023-10-10T20:35:05.717 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2023-10-10T20:35:05.726 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:35:05.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.838+0000 7f3f4ce3e700 1 Processor -- start 2023-10-10T20:35:05.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.838+0000 7f3f4ce3e700 1 -- start start 2023-10-10T20:35:05.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.839+0000 7f3f4ce3e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3f48159c60 0x7f3f4815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.839+0000 7f3f4ce3e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3f4815db10 0x7f3f48069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.839+0000 7f3f4ce3e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f4815a160 con 0x7f3f48159890 2023-10-10T20:35:05.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.839+0000 7f3f4ce3e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f4815df50 con 0x7f3f48159c60 2023-10-10T20:35:05.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.839+0000 7f3f4ce3e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f480d13a0 con 0x7f3f4815db10 2023-10-10T20:35:05.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.839+0000 7f3f3dd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3f48159c60 0x7f3f4815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.839+0000 7f3f3dd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3f48159c60 0x7f3f4815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59496/0 (socket says 172.21.15.89:59496) 2023-10-10T20:35:05.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.839+0000 7f3f3dd9b700 1 -- 172.21.15.89:0/1904910029 learned_addr learned my addr 172.21.15.89:0/1904910029 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:05.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.839+0000 7f3f46d9d700 1 --2- 172.21.15.89:0/1904910029 >> v2:172.21.15.111:3300/0 conn(0x7f3f4815db10 0x7f3f48069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.839+0000 7f3f3dd9b700 1 -- 172.21.15.89:0/1904910029 >> v2:172.21.15.111:3300/0 conn(0x7f3f4815db10 msgr2=0x7f3f48069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:05.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.839+0000 7f3f3dd9b700 1 --2- 172.21.15.89:0/1904910029 >> v2:172.21.15.111:3300/0 conn(0x7f3f4815db10 0x7f3f48069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.840+0000 7f3f3dd9b700 1 -- 172.21.15.89:0/1904910029 >> v1:172.21.15.111:6789/0 conn(0x7f3f48159890 legacy=0x7f3f4815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:05.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.840+0000 7f3f3dd9b700 1 -- 172.21.15.89:0/1904910029 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4815e140 con 0x7f3f48159c60 2023-10-10T20:35:05.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.840+0000 7f3f3dd9b700 1 --2- 172.21.15.89:0/1904910029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f48159c60 0x7f3f4815d5d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3f4806a890 tx=0x7f3f30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726e82ae20d28deb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:05.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.840+0000 7f3f45d9b700 1 -- 172.21.15.89:0/1904910029 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30009e70 con 0x7f3f48159c60 2023-10-10T20:35:05.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.840+0000 7f3f45d9b700 1 -- 172.21.15.89:0/1904910029 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3000b860 con 0x7f3f48159c60 2023-10-10T20:35:05.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.840+0000 7f3f45d9b700 1 -- 172.21.15.89:0/1904910029 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3000bc00 con 0x7f3f48159c60 2023-10-10T20:35:05.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.840+0000 7f3f4ce3e700 1 -- 172.21.15.89:0/1904910029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f48159c60 msgr2=0x7f3f4815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:05.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.840+0000 7f3f4ce3e700 1 --2- 172.21.15.89:0/1904910029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f48159c60 0x7f3f4815d5d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3f4806a890 tx=0x7f3f30009250 comp rx=0 tx=0).stop 2023-10-10T20:35:05.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.840+0000 7f3f4ce3e700 1 -- 172.21.15.89:0/1904910029 shutdown_connections 2023-10-10T20:35:05.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.840+0000 7f3f4ce3e700 1 --2- 172.21.15.89:0/1904910029 >> v2:172.21.15.111:3300/0 conn(0x7f3f4815db10 0x7f3f48069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.840+0000 7f3f4ce3e700 1 --2- 172.21.15.89:0/1904910029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f48159c60 0x7f3f4815d5d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.840+0000 7f3f4ce3e700 1 -- 172.21.15.89:0/1904910029 >> 172.21.15.89:0/1904910029 conn(0x7f3f48068e20 msgr2=0x7f3f480d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:05.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.840+0000 7f3f4ce3e700 1 -- 172.21.15.89:0/1904910029 shutdown_connections 2023-10-10T20:35:05.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.840+0000 7f3f4ce3e700 1 -- 172.21.15.89:0/1904910029 wait complete. 2023-10-10T20:35:05.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.842+0000 7f3f4ce3e700 1 Processor -- start 2023-10-10T20:35:05.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.842+0000 7f3f4ce3e700 1 -- start start 2023-10-10T20:35:05.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.842+0000 7f3f4ce3e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f48159890 0x7f3f480b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.842+0000 7f3f4ce3e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f48159c60 0x7f3f480c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.842+0000 7f3f4ce3e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f4815db10 0x7f3f480c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.842+0000 7f3f4ce3e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f480cfe10 con 0x7f3f48159890 2023-10-10T20:35:05.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.842+0000 7f3f4ce3e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f480c5f60 con 0x7f3f4815db10 2023-10-10T20:35:05.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.842+0000 7f3f4ce3e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f480b3ad0 con 0x7f3f48159c60 2023-10-10T20:35:05.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.842+0000 7f3f4659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f48159890 0x7f3f480b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.843+0000 7f3f4659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f48159890 0x7f3f480b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59506/0 (socket says 172.21.15.89:59506) 2023-10-10T20:35:05.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.843+0000 7f3f4659c700 1 -- 172.21.15.89:0/3659063994 learned_addr learned my addr 172.21.15.89:0/3659063994 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:05.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.843+0000 7f3f3dd9b700 1 --2- 172.21.15.89:0/3659063994 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f48159c60 0x7f3f480c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.843+0000 7f3f46d9d700 1 --2- 172.21.15.89:0/3659063994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f4815db10 0x7f3f480c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.843+0000 7f3f4659c700 1 -- 172.21.15.89:0/3659063994 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f48159c60 msgr2=0x7f3f480c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:05.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.843+0000 7f3f4659c700 1 --2- 172.21.15.89:0/3659063994 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f48159c60 0x7f3f480c63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.843+0000 7f3f4659c700 1 -- 172.21.15.89:0/3659063994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f4815db10 msgr2=0x7f3f480c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:05.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.843+0000 7f3f4659c700 1 --2- 172.21.15.89:0/3659063994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f4815db10 0x7f3f480c6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:05.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.843+0000 7f3f4659c700 1 -- 172.21.15.89:0/3659063994 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f300094f0 con 0x7f3f48159890 2023-10-10T20:35:05.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.843+0000 7f3f4659c700 1 --2- 172.21.15.89:0/3659063994 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f48159890 0x7f3f480b3590 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800c590 tx=0x7f3f3800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:05.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.843+0000 7f3f3effd700 1 -- 172.21.15.89:0/3659063994 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f38011590 con 0x7f3f48159890 2023-10-10T20:35:05.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.843+0000 7f3f3effd700 1 -- 172.21.15.89:0/3659063994 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f380116f0 con 0x7f3f48159890 2023-10-10T20:35:05.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.843+0000 7f3f4ce3e700 1 -- 172.21.15.89:0/3659063994 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f480c5630 con 0x7f3f48159890 2023-10-10T20:35:05.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.843+0000 7f3f3effd700 1 -- 172.21.15.89:0/3659063994 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f38011ab0 con 0x7f3f48159890 2023-10-10T20:35:05.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.843+0000 7f3f4ce3e700 1 -- 172.21.15.89:0/3659063994 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f480c5bf0 con 0x7f3f48159890 2023-10-10T20:35:05.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.844+0000 7f3f4ce3e700 1 -- 172.21.15.89:0/3659063994 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f480c6240 con 0x7f3f48159890 2023-10-10T20:35:05.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.844+0000 7f3f3effd700 1 -- 172.21.15.89:0/3659063994 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3f3801b3b0 con 0x7f3f48159890 2023-10-10T20:35:05.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.845+0000 7f3f3effd700 1 --2- 172.21.15.89:0/3659063994 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3f28066f30 0x7f3f280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:05.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.845+0000 7f3f3effd700 1 -- 172.21.15.89:0/3659063994 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3f38083ae0 con 0x7f3f48159890 2023-10-10T20:35:05.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.845+0000 7f3f3dd9b700 1 --2- 172.21.15.89:0/3659063994 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3f28066f30 0x7f3f280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:05.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.845+0000 7f3f3dd9b700 1 --2- 172.21.15.89:0/3659063994 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3f28066f30 0x7f3f280693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3f4806a890 tx=0x7f3f30017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:05.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:05.849+0000 7f3f3effd700 1 -- 172.21.15.89:0/3659063994 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f38050d50 con 0x7f3f48159890 2023-10-10T20:35:06.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.034+0000 7f3f4ce3e700 1 -- 172.21.15.89:0/3659063994 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f4804c3c0 con 0x7f3f48159890 2023-10-10T20:35:06.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.034+0000 7f3f3effd700 1 -- 172.21.15.89:0/3659063994 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f3f38017070 con 0x7f3f48159890 2023-10-10T20:35:06.037 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:06.037 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:36.324920+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:34:32.342167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:34:33.671763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:35:06.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.036+0000 7f3f4ce3e700 1 -- 172.21.15.89:0/3659063994 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3f28066f30 msgr2=0x7f3f280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:06.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.036+0000 7f3f4ce3e700 1 --2- 172.21.15.89:0/3659063994 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3f28066f30 0x7f3f280693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3f4806a890 tx=0x7f3f30017000 comp rx=0 tx=0).stop 2023-10-10T20:35:06.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.036+0000 7f3f4ce3e700 1 -- 172.21.15.89:0/3659063994 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f48159890 msgr2=0x7f3f480b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:06.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.036+0000 7f3f4ce3e700 1 --2- 172.21.15.89:0/3659063994 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f48159890 0x7f3f480b3590 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800c590 tx=0x7f3f3800c950 comp rx=0 tx=0).stop 2023-10-10T20:35:06.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.036+0000 7f3f4ce3e700 1 -- 172.21.15.89:0/3659063994 shutdown_connections 2023-10-10T20:35:06.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.036+0000 7f3f4ce3e700 1 --2- 172.21.15.89:0/3659063994 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3f28066f30 0x7f3f280693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.036+0000 7f3f4ce3e700 1 --2- 172.21.15.89:0/3659063994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f4815db10 0x7f3f480c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.036+0000 7f3f4ce3e700 1 --2- 172.21.15.89:0/3659063994 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f48159890 0x7f3f480b3590 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.036+0000 7f3f4ce3e700 1 --2- 172.21.15.89:0/3659063994 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f48159c60 0x7f3f480c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.036+0000 7f3f4ce3e700 1 -- 172.21.15.89:0/3659063994 >> 172.21.15.89:0/3659063994 conn(0x7f3f48068e20 msgr2=0x7f3f4814a300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:06.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.037+0000 7f3f4ce3e700 1 -- 172.21.15.89:0/3659063994 shutdown_connections 2023-10-10T20:35:06.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.037+0000 7f3f4ce3e700 1 -- 172.21.15.89:0/3659063994 wait complete. 2023-10-10T20:35:06.051 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:35:06.051 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:35:06.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.164+0000 7ff79e1f4700 1 Processor -- start 2023-10-10T20:35:06.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.164+0000 7ff79e1f4700 1 -- start start 2023-10-10T20:35:06.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.165+0000 7ff79e1f4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff7980c91e0 0x7ff7980cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:06.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.165+0000 7ff79e1f4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff79806ae10 0x7ff79806b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:06.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.165+0000 7ff79e1f4700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7980cc210 con 0x7ff79806aa40 2023-10-10T20:35:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.165+0000 7ff79e1f4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7980cc4f0 con 0x7ff7980c91e0 2023-10-10T20:35:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.165+0000 7ff79e1f4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980cc7d0 con 0x7ff79806ae10 2023-10-10T20:35:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.165+0000 7ff796ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff7980c91e0 0x7ff7980cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.165+0000 7ff796ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff7980c91e0 0x7ff7980cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59508/0 (socket says 172.21.15.89:59508) 2023-10-10T20:35:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.165+0000 7ff796ffd700 1 -- 172.21.15.89:0/3222900727 learned_addr learned my addr 172.21.15.89:0/3222900727 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.165+0000 7ff797fff700 1 --2- 172.21.15.89:0/3222900727 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff79806ae10 0x7ff79806b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.165+0000 7ff796ffd700 1 -- 172.21.15.89:0/3222900727 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff79806ae10 msgr2=0x7ff79806b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.165+0000 7ff796ffd700 1 --2- 172.21.15.89:0/3222900727 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff79806ae10 0x7ff79806b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.165+0000 7ff796ffd700 1 -- 172.21.15.89:0/3222900727 >> v1:172.21.15.111:6789/0 conn(0x7ff79806aa40 legacy=0x7ff79806bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.165+0000 7ff796ffd700 1 -- 172.21.15.89:0/3222900727 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7980ccab0 con 0x7ff7980c91e0 2023-10-10T20:35:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.165+0000 7ff797fff700 1 --2- 172.21.15.89:0/3222900727 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff79806ae10 0x7ff79806b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.166+0000 7ff796ffd700 1 --2- 172.21.15.89:0/3222900727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7980c91e0 0x7ff7980cbc00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0099e0 tx=0x7ff78c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec227a4ee6918c30 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.166+0000 7ff7967fc700 1 -- 172.21.15.89:0/3222900727 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c00b820 con 0x7ff7980c91e0 2023-10-10T20:35:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.166+0000 7ff7967fc700 1 -- 172.21.15.89:0/3222900727 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78c00b980 con 0x7ff7980c91e0 2023-10-10T20:35:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.166+0000 7ff7967fc700 1 -- 172.21.15.89:0/3222900727 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c00bd20 con 0x7ff7980c91e0 2023-10-10T20:35:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.166+0000 7ff79e1f4700 1 -- 172.21.15.89:0/3222900727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7980c91e0 msgr2=0x7ff7980cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.166+0000 7ff79e1f4700 1 --2- 172.21.15.89:0/3222900727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7980c91e0 0x7ff7980cbc00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0099e0 tx=0x7ff78c009250 comp rx=0 tx=0).stop 2023-10-10T20:35:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.166+0000 7ff79e1f4700 1 -- 172.21.15.89:0/3222900727 shutdown_connections 2023-10-10T20:35:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.166+0000 7ff79e1f4700 1 --2- 172.21.15.89:0/3222900727 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff79806ae10 0x7ff79806b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.166+0000 7ff79e1f4700 1 --2- 172.21.15.89:0/3222900727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7980c91e0 0x7ff7980cbc00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.166+0000 7ff79e1f4700 1 -- 172.21.15.89:0/3222900727 >> 172.21.15.89:0/3222900727 conn(0x7ff7980cd670 msgr2=0x7ff7980cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.166+0000 7ff79e1f4700 1 -- 172.21.15.89:0/3222900727 shutdown_connections 2023-10-10T20:35:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.166+0000 7ff79e1f4700 1 -- 172.21.15.89:0/3222900727 wait complete. 2023-10-10T20:35:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.168+0000 7ff79e1f4700 1 Processor -- start 2023-10-10T20:35:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.168+0000 7ff79e1f4700 1 -- start start 2023-10-10T20:35:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.168+0000 7ff79e1f4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff79806aa40 0x7ff798148a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.168+0000 7ff79e1f4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff79806ae10 0x7ff79814c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.168+0000 7ff79e1f4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7980c91e0 0x7ff798134060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.168+0000 7ff79e1f4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7981345a0 con 0x7ff79806aa40 2023-10-10T20:35:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.168+0000 7ff79e1f4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79814cbc0 con 0x7ff79806ae10 2023-10-10T20:35:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.168+0000 7ff79e1f4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980c7460 con 0x7ff7980c91e0 2023-10-10T20:35:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.168+0000 7ff7977fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff79806aa40 0x7ff798148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.168+0000 7ff7977fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff79806aa40 0x7ff798148a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59524/0 (socket says 172.21.15.89:59524) 2023-10-10T20:35:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.168+0000 7ff7977fe700 1 -- 172.21.15.89:0/4162262623 learned_addr learned my addr 172.21.15.89:0/4162262623 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.169+0000 7ff797fff700 1 --2- 172.21.15.89:0/4162262623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7980c91e0 0x7ff798134060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.169+0000 7ff7977fe700 1 -- 172.21.15.89:0/4162262623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7980c91e0 msgr2=0x7ff798134060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.169+0000 7ff7977fe700 1 --2- 172.21.15.89:0/4162262623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7980c91e0 0x7ff798134060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.169+0000 7ff7977fe700 1 -- 172.21.15.89:0/4162262623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff79806ae10 msgr2=0x7ff79814c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.169+0000 7ff7977fe700 1 --2- 172.21.15.89:0/4162262623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff79806ae10 0x7ff79814c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.169+0000 7ff7977fe700 1 -- 172.21.15.89:0/4162262623 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c0094f0 con 0x7ff79806aa40 2023-10-10T20:35:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.169+0000 7ff7977fe700 1 --2- 172.21.15.89:0/4162262623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff79806aa40 0x7ff798148a50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff78800cde0 tx=0x7ff788001be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.169+0000 7ff77f7fe700 1 -- 172.21.15.89:0/4162262623 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78800ac00 con 0x7ff79806aa40 2023-10-10T20:35:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.169+0000 7ff77f7fe700 1 -- 172.21.15.89:0/4162262623 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78800ad80 con 0x7ff79806aa40 2023-10-10T20:35:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.169+0000 7ff79e1f4700 1 -- 172.21.15.89:0/4162262623 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7980c6b30 con 0x7ff79806aa40 2023-10-10T20:35:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.169+0000 7ff77f7fe700 1 -- 172.21.15.89:0/4162262623 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788012670 con 0x7ff79806aa40 2023-10-10T20:35:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.169+0000 7ff79e1f4700 1 -- 172.21.15.89:0/4162262623 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7980c70f0 con 0x7ff79806aa40 2023-10-10T20:35:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.170+0000 7ff79e1f4700 1 -- 172.21.15.89:0/4162262623 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff79804c3c0 con 0x7ff79806aa40 2023-10-10T20:35:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.170+0000 7ff77f7fe700 1 -- 172.21.15.89:0/4162262623 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff788012a20 con 0x7ff79806aa40 2023-10-10T20:35:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.171+0000 7ff77f7fe700 1 --2- 172.21.15.89:0/4162262623 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff784066f30 0x7ff7840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:06.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.171+0000 7ff77f7fe700 1 -- 172.21.15.89:0/4162262623 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff788084dd0 con 0x7ff79806aa40 2023-10-10T20:35:06.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.171+0000 7ff796ffd700 1 --2- 172.21.15.89:0/4162262623 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff784066f30 0x7ff7840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:06.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.171+0000 7ff796ffd700 1 --2- 172.21.15.89:0/4162262623 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff784066f30 0x7ff7840693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0099e0 tx=0x7ff78c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:06.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.175+0000 7ff77f7fe700 1 -- 172.21.15.89:0/4162262623 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff788052040 con 0x7ff79806aa40 2023-10-10T20:35:06.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.402+0000 7ff79e1f4700 1 -- 172.21.15.89:0/4162262623 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7980c7cb0 con 0x7ff79806aa40 2023-10-10T20:35:06.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.402+0000 7ff77f7fe700 1 -- 172.21.15.89:0/4162262623 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4597 (secure 0 0 0) 0x7ff788055660 con 0x7ff79806aa40 2023-10-10T20:35:06.405 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:06.405 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2341577144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2341577144},{"type":"v1","addr":"172.21.15.111:6835","nonce":2341577144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2773123076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2773123076},{"type":"v1","addr":"172.21.15.89:6837","nonce":2773123076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2940044414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2940044414},{"type":"v1","addr":"172.21.15.89:6835","nonce":2940044414}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:34:34.717750+0000","modified":"2023-10-10T20:34:38.356048+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1112081320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1112081320},{"type":"v1","addr":"172.21.15.111:6834","nonce":1112081320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-10T20:35:06.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.405+0000 7ff79e1f4700 1 -- 172.21.15.89:0/4162262623 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff784066f30 msgr2=0x7ff7840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:06.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.405+0000 7ff79e1f4700 1 --2- 172.21.15.89:0/4162262623 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff784066f30 0x7ff7840693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0099e0 tx=0x7ff78c017000 comp rx=0 tx=0).stop 2023-10-10T20:35:06.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.405+0000 7ff79e1f4700 1 -- 172.21.15.89:0/4162262623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff79806aa40 msgr2=0x7ff798148a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:06.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.405+0000 7ff79e1f4700 1 --2- 172.21.15.89:0/4162262623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff79806aa40 0x7ff798148a50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff78800cde0 tx=0x7ff788001be0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.405+0000 7ff79e1f4700 1 -- 172.21.15.89:0/4162262623 shutdown_connections 2023-10-10T20:35:06.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.405+0000 7ff79e1f4700 1 --2- 172.21.15.89:0/4162262623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7980c91e0 0x7ff798134060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.405+0000 7ff79e1f4700 1 --2- 172.21.15.89:0/4162262623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff79806aa40 0x7ff798148a50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.405+0000 7ff79e1f4700 1 --2- 172.21.15.89:0/4162262623 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff784066f30 0x7ff7840693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.405+0000 7ff79e1f4700 1 --2- 172.21.15.89:0/4162262623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff79806ae10 0x7ff79814c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.405+0000 7ff79e1f4700 1 -- 172.21.15.89:0/4162262623 >> 172.21.15.89:0/4162262623 conn(0x7ff7980cd670 msgr2=0x7ff798152670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:06.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.405+0000 7ff79e1f4700 1 -- 172.21.15.89:0/4162262623 shutdown_connections 2023-10-10T20:35:06.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.405+0000 7ff79e1f4700 1 -- 172.21.15.89:0/4162262623 wait complete. 2023-10-10T20:35:06.410 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2023-10-10T20:35:06.419 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:35:06.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.531+0000 7f1950a98700 1 Processor -- start 2023-10-10T20:35:06.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.531+0000 7f1950a98700 1 -- start start 2023-10-10T20:35:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.532+0000 7f1950a98700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f194c159c10 0x7f194c15b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.532+0000 7f1950a98700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f194c0b5ea0 0x7f194c068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.532+0000 7f1950a98700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c15c480 con 0x7f194c068bb0 2023-10-10T20:35:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.532+0000 7f1950a98700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194c0b2a40 con 0x7f194c159c10 2023-10-10T20:35:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.532+0000 7f1950a98700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c15b5d0 con 0x7f194c0b5ea0 2023-10-10T20:35:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.532+0000 7f194a59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f194c159c10 0x7f194c15b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.532+0000 7f194a59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f194c159c10 0x7f194c15b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59540/0 (socket says 172.21.15.89:59540) 2023-10-10T20:35:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.532+0000 7f194a59c700 1 -- 172.21.15.89:0/2033119345 learned_addr learned my addr 172.21.15.89:0/2033119345 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.532+0000 7f1949d9b700 1 --2- 172.21.15.89:0/2033119345 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f194c0b5ea0 0x7f194c068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.532+0000 7f194a59c700 1 -- 172.21.15.89:0/2033119345 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f194c0b5ea0 msgr2=0x7f194c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.532+0000 7f194a59c700 1 --2- 172.21.15.89:0/2033119345 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f194c0b5ea0 0x7f194c068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.532+0000 7f194a59c700 1 -- 172.21.15.89:0/2033119345 >> v1:172.21.15.111:6789/0 conn(0x7f194c068bb0 legacy=0x7f194c15c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.532+0000 7f194a59c700 1 -- 172.21.15.89:0/2033119345 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c15b870 con 0x7f194c159c10 2023-10-10T20:35:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.532+0000 7f194a59c700 1 --2- 172.21.15.89:0/2033119345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f194c159c10 0x7f194c15b090 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f193c0099b0 tx=0x7f193c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a96498af71681e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.532+0000 7f194959a700 1 -- 172.21.15.89:0/2033119345 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c00b820 con 0x7f194c159c10 2023-10-10T20:35:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.532+0000 7f194959a700 1 -- 172.21.15.89:0/2033119345 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193c00b980 con 0x7f194c159c10 2023-10-10T20:35:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.533+0000 7f194959a700 1 -- 172.21.15.89:0/2033119345 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c00bd20 con 0x7f194c159c10 2023-10-10T20:35:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.533+0000 7f1950a98700 1 -- 172.21.15.89:0/2033119345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f194c159c10 msgr2=0x7f194c15b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.533+0000 7f1950a98700 1 --2- 172.21.15.89:0/2033119345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f194c159c10 0x7f194c15b090 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f193c0099b0 tx=0x7f193c009250 comp rx=0 tx=0).stop 2023-10-10T20:35:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.533+0000 7f1950a98700 1 -- 172.21.15.89:0/2033119345 shutdown_connections 2023-10-10T20:35:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.533+0000 7f1950a98700 1 --2- 172.21.15.89:0/2033119345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f194c159c10 0x7f194c15b090 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.533+0000 7f1950a98700 1 --2- 172.21.15.89:0/2033119345 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f194c0b5ea0 0x7f194c068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.533+0000 7f1950a98700 1 -- 172.21.15.89:0/2033119345 >> 172.21.15.89:0/2033119345 conn(0x7f194c099a20 msgr2=0x7f194c0af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.533+0000 7f1950a98700 1 -- 172.21.15.89:0/2033119345 shutdown_connections 2023-10-10T20:35:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.533+0000 7f1950a98700 1 -- 172.21.15.89:0/2033119345 wait complete. 2023-10-10T20:35:06.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.534+0000 7f1950a98700 1 Processor -- start 2023-10-10T20:35:06.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.534+0000 7f1950a98700 1 -- start start 2023-10-10T20:35:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.535+0000 7f1950a98700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f194c068bb0 0x7f194c15fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.535+0000 7f1950a98700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f194c0b5ea0 0x7f194c144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.535+0000 7f1950a98700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f194c159c10 0x7f194c144e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.535+0000 7f1950a98700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0b0400 con 0x7f194c0b5ea0 2023-10-10T20:35:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.535+0000 7f1950a98700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c145560 con 0x7f194c159c10 2023-10-10T20:35:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.535+0000 7f1950a98700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0af7f0 con 0x7f194c068bb0 2023-10-10T20:35:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.535+0000 7f1949d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f194c0b5ea0 0x7f194c144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.535+0000 7f1949d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f194c0b5ea0 0x7f194c144910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59546/0 (socket says 172.21.15.89:59546) 2023-10-10T20:35:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.535+0000 7f1949d9b700 1 -- 172.21.15.89:0/394335339 learned_addr learned my addr 172.21.15.89:0/394335339 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.535+0000 7f194a59c700 1 --2- 172.21.15.89:0/394335339 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f194c068bb0 0x7f194c15fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.535+0000 7f194ad9d700 1 --2- 172.21.15.89:0/394335339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f194c159c10 0x7f194c144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.535+0000 7f1949d9b700 1 -- 172.21.15.89:0/394335339 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f194c068bb0 msgr2=0x7f194c15fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.535+0000 7f1949d9b700 1 --2- 172.21.15.89:0/394335339 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f194c068bb0 0x7f194c15fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.535+0000 7f1949d9b700 1 -- 172.21.15.89:0/394335339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f194c159c10 msgr2=0x7f194c144e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.535+0000 7f1949d9b700 1 --2- 172.21.15.89:0/394335339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f194c159c10 0x7f194c144e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.535+0000 7f1949d9b700 1 -- 172.21.15.89:0/394335339 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c0094f0 con 0x7f194c0b5ea0 2023-10-10T20:35:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.535+0000 7f194ad9d700 1 --2- 172.21.15.89:0/394335339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f194c159c10 0x7f194c144e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.535+0000 7f1949d9b700 1 --2- 172.21.15.89:0/394335339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f194c0b5ea0 0x7f194c144910 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f194000d890 tx=0x7f1940008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.536+0000 7f193a7fc700 1 -- 172.21.15.89:0/394335339 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19400125f0 con 0x7f194c0b5ea0 2023-10-10T20:35:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.536+0000 7f193a7fc700 1 -- 172.21.15.89:0/394335339 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1940012750 con 0x7f194c0b5ea0 2023-10-10T20:35:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.536+0000 7f1950a98700 1 -- 172.21.15.89:0/394335339 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194c0afad0 con 0x7f194c0b5ea0 2023-10-10T20:35:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.536+0000 7f193a7fc700 1 -- 172.21.15.89:0/394335339 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940012ad0 con 0x7f194c0b5ea0 2023-10-10T20:35:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.536+0000 7f1950a98700 1 -- 172.21.15.89:0/394335339 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194c0b0090 con 0x7f194c0b5ea0 2023-10-10T20:35:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.537+0000 7f1950a98700 1 -- 172.21.15.89:0/394335339 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194c04c3c0 con 0x7f194c0b5ea0 2023-10-10T20:35:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.537+0000 7f193a7fc700 1 -- 172.21.15.89:0/394335339 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f194001e460 con 0x7f194c0b5ea0 2023-10-10T20:35:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.537+0000 7f193a7fc700 1 --2- 172.21.15.89:0/394335339 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f192c066f30 0x7f192c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.537+0000 7f193a7fc700 1 -- 172.21.15.89:0/394335339 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1940084b40 con 0x7f194c0b5ea0 2023-10-10T20:35:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.537+0000 7f194a59c700 1 --2- 172.21.15.89:0/394335339 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f192c066f30 0x7f192c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.538+0000 7f194a59c700 1 --2- 172.21.15.89:0/394335339 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f192c066f30 0x7f192c0693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f193c0099b0 tx=0x7f193c016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.542+0000 7f193a7fc700 1 -- 172.21.15.89:0/394335339 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1940051db0 con 0x7f194c0b5ea0 2023-10-10T20:35:06.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.768+0000 7f1950a98700 1 -- 172.21.15.89:0/394335339 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f194c15bb50 con 0x7f194c0b5ea0 2023-10-10T20:35:06.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.769+0000 7f193a7fc700 1 -- 172.21.15.89:0/394335339 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4597 (secure 0 0 0) 0x7f19400553d0 con 0x7f194c0b5ea0 2023-10-10T20:35:06.771 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:06.771 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2341577144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2341577144},{"type":"v1","addr":"172.21.15.111:6835","nonce":2341577144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2773123076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2773123076},{"type":"v1","addr":"172.21.15.89:6837","nonce":2773123076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2940044414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2940044414},{"type":"v1","addr":"172.21.15.89:6835","nonce":2940044414}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:34:34.717750+0000","modified":"2023-10-10T20:34:38.356048+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1112081320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1112081320},{"type":"v1","addr":"172.21.15.111:6834","nonce":1112081320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-10T20:35:06.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.771+0000 7f1950a98700 1 -- 172.21.15.89:0/394335339 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f192c066f30 msgr2=0x7f192c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:06.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.771+0000 7f1950a98700 1 --2- 172.21.15.89:0/394335339 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f192c066f30 0x7f192c0693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f193c0099b0 tx=0x7f193c016000 comp rx=0 tx=0).stop 2023-10-10T20:35:06.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.771+0000 7f1950a98700 1 -- 172.21.15.89:0/394335339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f194c0b5ea0 msgr2=0x7f194c144910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:06.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.771+0000 7f1950a98700 1 --2- 172.21.15.89:0/394335339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f194c0b5ea0 0x7f194c144910 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f194000d890 tx=0x7f1940008040 comp rx=0 tx=0).stop 2023-10-10T20:35:06.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.771+0000 7f1950a98700 1 -- 172.21.15.89:0/394335339 shutdown_connections 2023-10-10T20:35:06.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.771+0000 7f1950a98700 1 --2- 172.21.15.89:0/394335339 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f192c066f30 0x7f192c0693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.771+0000 7f1950a98700 1 --2- 172.21.15.89:0/394335339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f194c159c10 0x7f194c144e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.771+0000 7f1950a98700 1 --2- 172.21.15.89:0/394335339 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f194c068bb0 0x7f194c15fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.771+0000 7f1950a98700 1 --2- 172.21.15.89:0/394335339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f194c0b5ea0 0x7f194c144910 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.771+0000 7f1950a98700 1 -- 172.21.15.89:0/394335339 >> 172.21.15.89:0/394335339 conn(0x7f194c099a20 msgr2=0x7f194c09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:06.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.771+0000 7f1950a98700 1 -- 172.21.15.89:0/394335339 shutdown_connections 2023-10-10T20:35:06.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.771+0000 7f1950a98700 1 -- 172.21.15.89:0/394335339 wait complete. 2023-10-10T20:35:06.776 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2023-10-10T20:35:06.785 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:35:06.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.897+0000 7fcc2d21f700 1 Processor -- start 2023-10-10T20:35:06.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.897+0000 7fcc2d21f700 1 -- start start 2023-10-10T20:35:06.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.898+0000 7fcc2d21f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc28159c60 0x7fcc2815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:06.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.898+0000 7fcc2d21f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2815a160 con 0x7fcc2815db10 2023-10-10T20:35:06.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.898+0000 7fcc2d21f700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc280d13a0 con 0x7fcc28159890 2023-10-10T20:35:06.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.898+0000 7fcc2d21f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc2815df50 con 0x7fcc28159c60 2023-10-10T20:35:06.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.898+0000 7fcc2659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc28159c60 0x7fcc2815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:06.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.898+0000 7fcc2759e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcc2815db10 0x7fcc28069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58206/0 (socket says 172.21.15.89:58206) 2023-10-10T20:35:06.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.898+0000 7fcc2759e700 1 -- 172.21.15.89:0/2498536296 learned_addr learned my addr 172.21.15.89:0/2498536296 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:06.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.899+0000 7fcc2659c700 1 -- 172.21.15.89:0/2498536296 >> v1:172.21.15.111:6789/0 conn(0x7fcc28159890 legacy=0x7fcc2815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:06.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.899+0000 7fcc2659c700 1 -- 172.21.15.89:0/2498536296 >> v1:172.21.15.89:6789/0 conn(0x7fcc2815db10 legacy=0x7fcc28069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:06.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.899+0000 7fcc2659c700 1 -- 172.21.15.89:0/2498536296 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2815e140 con 0x7fcc28159c60 2023-10-10T20:35:06.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.899+0000 7fcc25d9b700 1 -- 172.21.15.89:0/2498536296 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922283673 0 0) 0x7fcc2815a160 con 0x7fcc2815db10 2023-10-10T20:35:06.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.899+0000 7fcc2659c700 1 --2- 172.21.15.89:0/2498536296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc28159c60 0x7fcc2815d5d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0098e0 tx=0x7fcc1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bd0793b7886afa server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:06.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.899+0000 7fcc25d9b700 1 -- 172.21.15.89:0/2498536296 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c009e80 con 0x7fcc28159c60 2023-10-10T20:35:06.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.899+0000 7fcc25d9b700 1 -- 172.21.15.89:0/2498536296 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c00b860 con 0x7fcc28159c60 2023-10-10T20:35:06.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.899+0000 7fcc25d9b700 1 -- 172.21.15.89:0/2498536296 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c00bbe0 con 0x7fcc28159c60 2023-10-10T20:35:06.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.899+0000 7fcc2d21f700 1 -- 172.21.15.89:0/2498536296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc28159c60 msgr2=0x7fcc2815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:06.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.899+0000 7fcc2d21f700 1 --2- 172.21.15.89:0/2498536296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc28159c60 0x7fcc2815d5d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0098e0 tx=0x7fcc1c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.899+0000 7fcc2d21f700 1 -- 172.21.15.89:0/2498536296 shutdown_connections 2023-10-10T20:35:06.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.899+0000 7fcc2d21f700 1 --2- 172.21.15.89:0/2498536296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc28159c60 0x7fcc2815d5d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.899+0000 7fcc2d21f700 1 -- 172.21.15.89:0/2498536296 >> 172.21.15.89:0/2498536296 conn(0x7fcc28068e20 msgr2=0x7fcc280d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:06.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.900+0000 7fcc2d21f700 1 -- 172.21.15.89:0/2498536296 shutdown_connections 2023-10-10T20:35:06.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.900+0000 7fcc2d21f700 1 -- 172.21.15.89:0/2498536296 wait complete. 2023-10-10T20:35:06.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.901+0000 7fcc2d21f700 1 Processor -- start 2023-10-10T20:35:06.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.901+0000 7fcc2d21f700 1 -- start start 2023-10-10T20:35:06.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.901+0000 7fcc2d21f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc28159890 0x7fcc28155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:06.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.901+0000 7fcc2d21f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcc28159c60 0x7fcc28148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:06.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.901+0000 7fcc2d21f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc2815db10 0x7fcc281492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:06.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.901+0000 7fcc2d21f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc281497f0 con 0x7fcc28159890 2023-10-10T20:35:06.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc2d21f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280cfcf0 con 0x7fcc2815db10 2023-10-10T20:35:06.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc2d21f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2816d280 con 0x7fcc28159c60 2023-10-10T20:35:06.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc26d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc28159890 0x7fcc28155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:06.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc26d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc28159890 0x7fcc28155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59572/0 (socket says 172.21.15.89:59572) 2023-10-10T20:35:06.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc26d9d700 1 -- 172.21.15.89:0/3052504081 learned_addr learned my addr 172.21.15.89:0/3052504081 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:06.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc2759e700 1 --2- 172.21.15.89:0/3052504081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc2815db10 0x7fcc281492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:06.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc2659c700 1 --2- 172.21.15.89:0/3052504081 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcc28159c60 0x7fcc28148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:06.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc26d9d700 1 -- 172.21.15.89:0/3052504081 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcc28159c60 msgr2=0x7fcc28148d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:06.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc26d9d700 1 --2- 172.21.15.89:0/3052504081 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcc28159c60 0x7fcc28148d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc26d9d700 1 -- 172.21.15.89:0/3052504081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc2815db10 msgr2=0x7fcc281492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:06.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc26d9d700 1 --2- 172.21.15.89:0/3052504081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc2815db10 0x7fcc281492b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:06.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc26d9d700 1 -- 172.21.15.89:0/3052504081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c009420 con 0x7fcc28159890 2023-10-10T20:35:06.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc2659c700 1 --2- 172.21.15.89:0/3052504081 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcc28159c60 0x7fcc28148d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:06.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc2759e700 1 --2- 172.21.15.89:0/3052504081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc2815db10 0x7fcc281492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:06.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc26d9d700 1 --2- 172.21.15.89:0/3052504081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc28159890 0x7fcc28155240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800cb10 tx=0x7fcc180017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:06.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc0effd700 1 -- 172.21.15.89:0/3052504081 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1800a760 con 0x7fcc28159890 2023-10-10T20:35:06.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc0effd700 1 -- 172.21.15.89:0/3052504081 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1800a8e0 con 0x7fcc28159890 2023-10-10T20:35:06.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc0effd700 1 -- 172.21.15.89:0/3052504081 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1800aca0 con 0x7fcc28159890 2023-10-10T20:35:06.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc2d21f700 1 -- 172.21.15.89:0/3052504081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc2816d560 con 0x7fcc28159890 2023-10-10T20:35:06.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.902+0000 7fcc2d21f700 1 -- 172.21.15.89:0/3052504081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2816db20 con 0x7fcc28159890 2023-10-10T20:35:06.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.903+0000 7fcc2d21f700 1 -- 172.21.15.89:0/3052504081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2804c3c0 con 0x7fcc28159890 2023-10-10T20:35:06.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.903+0000 7fcc0effd700 1 -- 172.21.15.89:0/3052504081 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc18003030 con 0x7fcc28159890 2023-10-10T20:35:06.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.904+0000 7fcc0effd700 1 --2- 172.21.15.89:0/3052504081 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcc14066f30 0x7fcc140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:06.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.904+0000 7fcc0effd700 1 -- 172.21.15.89:0/3052504081 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcc18084610 con 0x7fcc28159890 2023-10-10T20:35:06.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.904+0000 7fcc2659c700 1 --2- 172.21.15.89:0/3052504081 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcc14066f30 0x7fcc140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:06.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.904+0000 7fcc2659c700 1 --2- 172.21.15.89:0/3052504081 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcc14066f30 0x7fcc140693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0093f0 tx=0x7fcc1c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:06.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:06.908+0000 7fcc0effd700 1 -- 172.21.15.89:0/3052504081 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc18051880 con 0x7fcc28159890 2023-10-10T20:35:07.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.094+0000 7fcc2d21f700 1 -- 172.21.15.89:0/3052504081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc280890e0 con 0x7fcc28159890 2023-10-10T20:35:07.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.094+0000 7fcc0effd700 1 -- 172.21.15.89:0/3052504081 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fcc18054ea0 con 0x7fcc28159890 2023-10-10T20:35:07.097 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:07.097 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:34:36.324920+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:34:32.342167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:34:33.671763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:35:07.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.096+0000 7fcc2d21f700 1 -- 172.21.15.89:0/3052504081 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcc14066f30 msgr2=0x7fcc140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:07.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.096+0000 7fcc2d21f700 1 --2- 172.21.15.89:0/3052504081 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcc14066f30 0x7fcc140693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0093f0 tx=0x7fcc1c017000 comp rx=0 tx=0).stop 2023-10-10T20:35:07.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.096+0000 7fcc2d21f700 1 -- 172.21.15.89:0/3052504081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc28159890 msgr2=0x7fcc28155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:07.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.096+0000 7fcc2d21f700 1 --2- 172.21.15.89:0/3052504081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc28159890 0x7fcc28155240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800cb10 tx=0x7fcc180017b0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.096+0000 7fcc2d21f700 1 -- 172.21.15.89:0/3052504081 shutdown_connections 2023-10-10T20:35:07.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.096+0000 7fcc2d21f700 1 --2- 172.21.15.89:0/3052504081 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcc14066f30 0x7fcc140693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.096+0000 7fcc2d21f700 1 --2- 172.21.15.89:0/3052504081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc2815db10 0x7fcc281492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.096+0000 7fcc2d21f700 1 --2- 172.21.15.89:0/3052504081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc28159890 0x7fcc28155240 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.096+0000 7fcc2d21f700 1 --2- 172.21.15.89:0/3052504081 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcc28159c60 0x7fcc28148d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.096+0000 7fcc2d21f700 1 -- 172.21.15.89:0/3052504081 >> 172.21.15.89:0/3052504081 conn(0x7fcc28068e20 msgr2=0x7fcc2815cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:07.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.096+0000 7fcc2d21f700 1 -- 172.21.15.89:0/3052504081 shutdown_connections 2023-10-10T20:35:07.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.096+0000 7fcc2d21f700 1 -- 172.21.15.89:0/3052504081 wait complete. 2023-10-10T20:35:07.111 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:35:07.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.224+0000 7f8ca0c08700 1 Processor -- start 2023-10-10T20:35:07.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.224+0000 7f8ca0c08700 1 -- start start 2023-10-10T20:35:07.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.225+0000 7f8ca0c08700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8c9c06c0a0 0x7f8c9c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:07.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.225+0000 7f8ca0c08700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c9c15df50 0x7f8c9c15e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:07.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.225+0000 7f8ca0c08700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c9c15d7e0 con 0x7f8c9c06bcd0 2023-10-10T20:35:07.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.225+0000 7f8ca0c08700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9c15dac0 con 0x7f8c9c06c0a0 2023-10-10T20:35:07.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.225+0000 7f8ca0c08700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c15aa30 con 0x7f8c9c15df50 2023-10-10T20:35:07.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.225+0000 7f8c91d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8c9c06c0a0 0x7f8c9c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:07.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.225+0000 7f8c91d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8c9c06c0a0 0x7f8c9c0caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59574/0 (socket says 172.21.15.89:59574) 2023-10-10T20:35:07.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.225+0000 7f8c91d9b700 1 -- 172.21.15.89:0/3275110595 learned_addr learned my addr 172.21.15.89:0/3275110595 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:07.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.225+0000 7f8c9ad9d700 1 --2- 172.21.15.89:0/3275110595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c9c15df50 0x7f8c9c15e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:07.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.226+0000 7f8c91d9b700 1 -- 172.21.15.89:0/3275110595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c9c15df50 msgr2=0x7f8c9c15e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:07.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.226+0000 7f8c91d9b700 1 --2- 172.21.15.89:0/3275110595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c9c15df50 0x7f8c9c15e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.226+0000 7f8c91d9b700 1 -- 172.21.15.89:0/3275110595 >> v1:172.21.15.111:6789/0 conn(0x7f8c9c06bcd0 legacy=0x7f8c9c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:07.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.226+0000 7f8c91d9b700 1 -- 172.21.15.89:0/3275110595 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c15ad10 con 0x7f8c9c06c0a0 2023-10-10T20:35:07.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.226+0000 7f8c91d9b700 1 --2- 172.21.15.89:0/3275110595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c9c06c0a0 0x7f8c9c0caa90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8c840099b0 tx=0x7f8c84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b717cd861d69e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:07.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.226+0000 7f8c99d9b700 1 -- 172.21.15.89:0/3275110595 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84009ea0 con 0x7f8c9c06c0a0 2023-10-10T20:35:07.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.226+0000 7f8c99d9b700 1 -- 172.21.15.89:0/3275110595 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8400b860 con 0x7f8c9c06c0a0 2023-10-10T20:35:07.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.226+0000 7f8c99d9b700 1 -- 172.21.15.89:0/3275110595 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8400bc20 con 0x7f8c9c06c0a0 2023-10-10T20:35:07.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.226+0000 7f8ca0c08700 1 -- 172.21.15.89:0/3275110595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c9c06c0a0 msgr2=0x7f8c9c0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:07.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.226+0000 7f8ca0c08700 1 --2- 172.21.15.89:0/3275110595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c9c06c0a0 0x7f8c9c0caa90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8c840099b0 tx=0x7f8c84009250 comp rx=0 tx=0).stop 2023-10-10T20:35:07.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.226+0000 7f8ca0c08700 1 -- 172.21.15.89:0/3275110595 shutdown_connections 2023-10-10T20:35:07.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.226+0000 7f8ca0c08700 1 --2- 172.21.15.89:0/3275110595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c9c15df50 0x7f8c9c15e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.226+0000 7f8ca0c08700 1 --2- 172.21.15.89:0/3275110595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c9c06c0a0 0x7f8c9c0caa90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.226+0000 7f8ca0c08700 1 -- 172.21.15.89:0/3275110595 >> 172.21.15.89:0/3275110595 conn(0x7f8c9c0ce600 msgr2=0x7f8c9c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:07.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.226+0000 7f8ca0c08700 1 -- 172.21.15.89:0/3275110595 shutdown_connections 2023-10-10T20:35:07.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.227+0000 7f8ca0c08700 1 -- 172.21.15.89:0/3275110595 wait complete. 2023-10-10T20:35:07.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.228+0000 7f8ca0c08700 1 Processor -- start 2023-10-10T20:35:07.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.228+0000 7f8ca0c08700 1 -- start start 2023-10-10T20:35:07.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.228+0000 7f8ca0c08700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c9c06bcd0 0x7f8c9c1440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:07.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.228+0000 7f8ca0c08700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c9c06c0a0 0x7f8c9c069480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:07.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.228+0000 7f8ca0c08700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c9c15df50 0x7f8c9c0699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:07.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8ca0c08700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c16b1a0 con 0x7f8c9c06c0a0 2023-10-10T20:35:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8ca0c08700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c0c7730 con 0x7f8c9c15df50 2023-10-10T20:35:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8ca0c08700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c0c7a10 con 0x7f8c9c06bcd0 2023-10-10T20:35:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8c91d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c9c06c0a0 0x7f8c9c069480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8c91d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c9c06c0a0 0x7f8c9c069480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59584/0 (socket says 172.21.15.89:59584) 2023-10-10T20:35:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8c91d9b700 1 -- 172.21.15.89:0/1676900129 learned_addr learned my addr 172.21.15.89:0/1676900129 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8c9ad9d700 1 --2- 172.21.15.89:0/1676900129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c9c15df50 0x7f8c9c0699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8c9a59c700 1 --2- 172.21.15.89:0/1676900129 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c9c06bcd0 0x7f8c9c1440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8c91d9b700 1 -- 172.21.15.89:0/1676900129 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c9c06bcd0 msgr2=0x7f8c9c1440f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8c91d9b700 1 --2- 172.21.15.89:0/1676900129 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c9c06bcd0 0x7f8c9c1440f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8c91d9b700 1 -- 172.21.15.89:0/1676900129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c9c15df50 msgr2=0x7f8c9c0699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8c91d9b700 1 --2- 172.21.15.89:0/1676900129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c9c15df50 0x7f8c9c0699c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8c91d9b700 1 -- 172.21.15.89:0/1676900129 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c840094f0 con 0x7f8c9c06c0a0 2023-10-10T20:35:07.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8c9a59c700 1 --2- 172.21.15.89:0/1676900129 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c9c06bcd0 0x7f8c9c1440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:07.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8c9ad9d700 1 --2- 172.21.15.89:0/1676900129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c9c15df50 0x7f8c9c0699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:07.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8c91d9b700 1 --2- 172.21.15.89:0/1676900129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c9c06c0a0 0x7f8c9c069480 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8c84006fd0 tx=0x7f8c84017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:07.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8c92ffd700 1 -- 172.21.15.89:0/1676900129 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8401f070 con 0x7f8c9c06c0a0 2023-10-10T20:35:07.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8c92ffd700 1 -- 172.21.15.89:0/1676900129 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c84000c90 con 0x7f8c9c06c0a0 2023-10-10T20:35:07.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8ca0c08700 1 -- 172.21.15.89:0/1676900129 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9c0c7cf0 con 0x7f8c9c06c0a0 2023-10-10T20:35:07.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8c92ffd700 1 -- 172.21.15.89:0/1676900129 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84025420 con 0x7f8c9c06c0a0 2023-10-10T20:35:07.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.229+0000 7f8ca0c08700 1 -- 172.21.15.89:0/1676900129 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9c144630 con 0x7f8c9c06c0a0 2023-10-10T20:35:07.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.230+0000 7f8ca0c08700 1 -- 172.21.15.89:0/1676900129 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9c144820 con 0x7f8c9c06c0a0 2023-10-10T20:35:07.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.231+0000 7f8c92ffd700 1 -- 172.21.15.89:0/1676900129 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c84023070 con 0x7f8c9c06c0a0 2023-10-10T20:35:07.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.231+0000 7f8c92ffd700 1 --2- 172.21.15.89:0/1676900129 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c7c066f30 0x7f8c7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:07.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.231+0000 7f8c92ffd700 1 -- 172.21.15.89:0/1676900129 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8c840858b0 con 0x7f8c9c06c0a0 2023-10-10T20:35:07.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.231+0000 7f8c9a59c700 1 --2- 172.21.15.89:0/1676900129 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c7c066f30 0x7f8c7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:07.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.231+0000 7f8c9a59c700 1 --2- 172.21.15.89:0/1676900129 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c7c066f30 0x7f8c7c0693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0017a0 tx=0x7f8c8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:07.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.235+0000 7f8c92ffd700 1 -- 172.21.15.89:0/1676900129 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c9c144820 con 0x7f8c9c06c0a0 2023-10-10T20:35:07.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.464+0000 7f8ca0c08700 1 -- 172.21.15.89:0/1676900129 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8c9c144820 con 0x7f8c9c06c0a0 2023-10-10T20:35:07.466 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-10-10T20:35:07.464+0000 7fd0184ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:35:07.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.468+0000 7f8c92ffd700 1 -- 172.21.15.89:0/1676900129 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8c9c144820 con 0x7f8c9c06c0a0 2023-10-10T20:35:07.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.470+0000 7f8ca0c08700 1 -- 172.21.15.89:0/1676900129 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c7c066f30 msgr2=0x7f8c7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:07.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.470+0000 7f8ca0c08700 1 --2- 172.21.15.89:0/1676900129 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c7c066f30 0x7f8c7c0693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0017a0 tx=0x7f8c8c009040 comp rx=0 tx=0).stop 2023-10-10T20:35:07.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.470+0000 7f8ca0c08700 1 -- 172.21.15.89:0/1676900129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c9c06c0a0 msgr2=0x7f8c9c069480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:07.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.470+0000 7f8ca0c08700 1 --2- 172.21.15.89:0/1676900129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c9c06c0a0 0x7f8c9c069480 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8c84006fd0 tx=0x7f8c84017000 comp rx=0 tx=0).stop 2023-10-10T20:35:07.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.470+0000 7f8ca0c08700 1 -- 172.21.15.89:0/1676900129 shutdown_connections 2023-10-10T20:35:07.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.470+0000 7f8ca0c08700 1 --2- 172.21.15.89:0/1676900129 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c7c066f30 0x7f8c7c0693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.470+0000 7f8ca0c08700 1 --2- 172.21.15.89:0/1676900129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c9c15df50 0x7f8c9c0699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.470+0000 7f8ca0c08700 1 --2- 172.21.15.89:0/1676900129 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c9c06bcd0 0x7f8c9c1440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.470+0000 7f8ca0c08700 1 --2- 172.21.15.89:0/1676900129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c9c06c0a0 0x7f8c9c069480 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.470+0000 7f8ca0c08700 1 -- 172.21.15.89:0/1676900129 >> 172.21.15.89:0/1676900129 conn(0x7f8c9c0ce600 msgr2=0x7f8c9c0c4570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:07.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.470+0000 7f8ca0c08700 1 -- 172.21.15.89:0/1676900129 shutdown_connections 2023-10-10T20:35:07.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.470+0000 7f8ca0c08700 1 -- 172.21.15.89:0/1676900129 wait complete. 2023-10-10T20:35:07.475 INFO:tasks.ceph.mds.b.smithi111.stderr: -1626> 2023-10-10T20:34:39.045+0000 7fe167616700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:35:07.476 INFO:teuthology.orchestra.run.smithi089.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:35:07.484 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:35:07.497 INFO:tasks.ceph.mds.b.smithi111.stdout:starting mds.b at 2023-10-10T20:35:07.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.598+0000 7f7e85456700 1 Processor -- start 2023-10-10T20:35:07.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.598+0000 7f7e85456700 1 -- start start 2023-10-10T20:35:07.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.599+0000 7f7e85456700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7e80159c60 0x7f7e8015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:07.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.599+0000 7f7e85456700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e8015db10 0x7f7e80069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:07.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.599+0000 7f7e85456700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e8015a160 con 0x7f7e80159890 2023-10-10T20:35:07.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.599+0000 7f7e85456700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e8015df50 con 0x7f7e80159c60 2023-10-10T20:35:07.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.599+0000 7f7e85456700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e800d13a0 con 0x7f7e8015db10 2023-10-10T20:35:07.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.599+0000 7f7e7effd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7e80159890 0x7f7e8015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58210/0 (socket says 172.21.15.89:58210) 2023-10-10T20:35:07.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.599+0000 7f7e7effd700 1 -- 172.21.15.89:0/160625642 learned_addr learned my addr 172.21.15.89:0/160625642 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:07.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.600+0000 7f7e7f7fe700 1 --2- 172.21.15.89:0/160625642 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e8015db10 0x7f7e80069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:07.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.600+0000 7f7e767fc700 1 --2- 172.21.15.89:0/160625642 >> v2:172.21.15.111:3300/0 conn(0x7f7e80159c60 0x7f7e8015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:07.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.600+0000 7f7e7e7fc700 1 -- 172.21.15.89:0/160625642 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341155964 0 0) 0x7f7e8015a160 con 0x7f7e80159890 2023-10-10T20:35:07.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.600+0000 7f7e7e7fc700 1 -- 172.21.15.89:0/160625642 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e6c003570 con 0x7f7e80159890 2023-10-10T20:35:07.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.600+0000 7f7e7e7fc700 1 -- 172.21.15.89:0/160625642 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3023963435 0 0) 0x7f7e6c003570 con 0x7f7e80159890 2023-10-10T20:35:07.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.600+0000 7f7e767fc700 1 -- 172.21.15.89:0/160625642 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e8015db10 msgr2=0x7f7e80069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:07.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.600+0000 7f7e767fc700 1 --2- 172.21.15.89:0/160625642 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e8015db10 0x7f7e80069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.600+0000 7f7e767fc700 1 -- 172.21.15.89:0/160625642 >> v1:172.21.15.89:6789/0 conn(0x7f7e80159890 legacy=0x7f7e8015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:07.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.600+0000 7f7e767fc700 1 -- 172.21.15.89:0/160625642 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8015e140 con 0x7f7e80159c60 2023-10-10T20:35:07.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.600+0000 7f7e767fc700 1 --2- 172.21.15.89:0/160625642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e80159c60 0x7f7e8015d5d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7e680098e0 tx=0x7f7e680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5414688d3f67fe3f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:07.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.600+0000 7f7e7e7fc700 1 -- 172.21.15.89:0/160625642 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70001470 con 0x7f7e80159c60 2023-10-10T20:35:07.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.600+0000 7f7e7e7fc700 1 -- 172.21.15.89:0/160625642 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e68011420 con 0x7f7e80159c60 2023-10-10T20:35:07.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.600+0000 7f7e7e7fc700 1 -- 172.21.15.89:0/160625642 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e680117c0 con 0x7f7e80159c60 2023-10-10T20:35:07.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.601+0000 7f7e85456700 1 -- 172.21.15.89:0/160625642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e80159c60 msgr2=0x7f7e8015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:07.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.601+0000 7f7e85456700 1 --2- 172.21.15.89:0/160625642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e80159c60 0x7f7e8015d5d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7e680098e0 tx=0x7f7e680091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.601+0000 7f7e85456700 1 -- 172.21.15.89:0/160625642 shutdown_connections 2023-10-10T20:35:07.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.601+0000 7f7e85456700 1 --2- 172.21.15.89:0/160625642 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e8015db10 0x7f7e80069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.601+0000 7f7e85456700 1 --2- 172.21.15.89:0/160625642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e80159c60 0x7f7e8015d5d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.601+0000 7f7e85456700 1 -- 172.21.15.89:0/160625642 >> 172.21.15.89:0/160625642 conn(0x7f7e80068e20 msgr2=0x7f7e800d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:07.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.601+0000 7f7e85456700 1 -- 172.21.15.89:0/160625642 shutdown_connections 2023-10-10T20:35:07.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.601+0000 7f7e85456700 1 -- 172.21.15.89:0/160625642 wait complete. 2023-10-10T20:35:07.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.602+0000 7f7e85456700 1 Processor -- start 2023-10-10T20:35:07.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.602+0000 7f7e85456700 1 -- start start 2023-10-10T20:35:07.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e85456700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e80159890 0x7f7e8014d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:07.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e85456700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e80159c60 0x7f7e800c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:07.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e85456700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e8015db10 0x7f7e80157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:07.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e85456700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e80151370 con 0x7f7e8015db10 2023-10-10T20:35:07.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e85456700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8014dbc0 con 0x7f7e80159c60 2023-10-10T20:35:07.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e85456700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e800c9c60 con 0x7f7e80159890 2023-10-10T20:35:07.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e7f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e8015db10 0x7f7e80157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:07.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e7f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e8015db10 0x7f7e80157430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59598/0 (socket says 172.21.15.89:59598) 2023-10-10T20:35:07.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e7f7fe700 1 -- 172.21.15.89:0/854096244 learned_addr learned my addr 172.21.15.89:0/854096244 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:07.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e767fc700 1 --2- 172.21.15.89:0/854096244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e80159c60 0x7f7e800c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:07.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e7effd700 1 --2- 172.21.15.89:0/854096244 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e80159890 0x7f7e8014d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:07.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e7f7fe700 1 -- 172.21.15.89:0/854096244 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e80159890 msgr2=0x7f7e8014d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:07.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e7f7fe700 1 --2- 172.21.15.89:0/854096244 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e80159890 0x7f7e8014d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e7f7fe700 1 -- 172.21.15.89:0/854096244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e80159c60 msgr2=0x7f7e800c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:07.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e7f7fe700 1 --2- 172.21.15.89:0/854096244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e80159c60 0x7f7e800c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:07.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e7f7fe700 1 -- 172.21.15.89:0/854096244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e68009420 con 0x7f7e8015db10 2023-10-10T20:35:07.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e7f7fe700 1 --2- 172.21.15.89:0/854096244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e8015db10 0x7f7e80157430 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009df0 tx=0x7f7e7800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e777fe700 1 -- 172.21.15.89:0/854096244 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800caa0 con 0x7f7e8015db10 2023-10-10T20:35:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e777fe700 1 -- 172.21.15.89:0/854096244 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7800f040 con 0x7f7e8015db10 2023-10-10T20:35:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e85456700 1 -- 172.21.15.89:0/854096244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e800c9f40 con 0x7f7e8015db10 2023-10-10T20:35:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.603+0000 7f7e777fe700 1 -- 172.21.15.89:0/854096244 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800ce70 con 0x7f7e8015db10 2023-10-10T20:35:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.604+0000 7f7e85456700 1 -- 172.21.15.89:0/854096244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e800ca500 con 0x7f7e8015db10 2023-10-10T20:35:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.604+0000 7f7e85456700 1 -- 172.21.15.89:0/854096244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e8004c3c0 con 0x7f7e8015db10 2023-10-10T20:35:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.605+0000 7f7e777fe700 1 -- 172.21.15.89:0/854096244 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7e78014690 con 0x7f7e8015db10 2023-10-10T20:35:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.605+0000 7f7e777fe700 1 --2- 172.21.15.89:0/854096244 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7e60066f30 0x7f7e600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.605+0000 7f7e777fe700 1 -- 172.21.15.89:0/854096244 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f7e78084e10 con 0x7f7e8015db10 2023-10-10T20:35:07.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.605+0000 7f7e7effd700 1 --2- 172.21.15.89:0/854096244 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7e60066f30 0x7f7e600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:07.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.606+0000 7f7e7effd700 1 --2- 172.21.15.89:0/854096244 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7e60066f30 0x7f7e600693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7e700011e0 tx=0x7f7e70001aa0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:07.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.609+0000 7f7e777fe700 1 -- 172.21.15.89:0/854096244 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e78055ac0 con 0x7f7e8015db10 2023-10-10T20:35:07.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:07.836+0000 7f7e85456700 1 -- 172.21.15.89:0/854096244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7e801514b0 con 0x7f7e8015db10 2023-10-10T20:35:08.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.469+0000 7f7e777fe700 1 -- 172.21.15.89:0/854096244 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7e78055650 con 0x7f7e8015db10 2023-10-10T20:35:08.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.471+0000 7f7e85456700 1 -- 172.21.15.89:0/854096244 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7e60066f30 msgr2=0x7f7e600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:08.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.471+0000 7f7e85456700 1 --2- 172.21.15.89:0/854096244 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7e60066f30 0x7f7e600693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7e700011e0 tx=0x7f7e70001aa0 comp rx=0 tx=0).stop 2023-10-10T20:35:08.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.471+0000 7f7e85456700 1 -- 172.21.15.89:0/854096244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e8015db10 msgr2=0x7f7e80157430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:08.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.471+0000 7f7e85456700 1 --2- 172.21.15.89:0/854096244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e8015db10 0x7f7e80157430 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009df0 tx=0x7f7e7800b040 comp rx=0 tx=0).stop 2023-10-10T20:35:08.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.471+0000 7f7e85456700 1 -- 172.21.15.89:0/854096244 shutdown_connections 2023-10-10T20:35:08.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.471+0000 7f7e85456700 1 --2- 172.21.15.89:0/854096244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e8015db10 0x7f7e80157430 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:08.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.471+0000 7f7e85456700 1 --2- 172.21.15.89:0/854096244 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e80159890 0x7f7e8014d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:08.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.471+0000 7f7e85456700 1 --2- 172.21.15.89:0/854096244 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7e60066f30 0x7f7e600693e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:08.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.471+0000 7f7e85456700 1 --2- 172.21.15.89:0/854096244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e80159c60 0x7f7e800c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:08.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.471+0000 7f7e85456700 1 -- 172.21.15.89:0/854096244 >> 172.21.15.89:0/854096244 conn(0x7f7e80068e20 msgr2=0x7f7e8015aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:08.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.471+0000 7f7e85456700 1 -- 172.21.15.89:0/854096244 shutdown_connections 2023-10-10T20:35:08.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.471+0000 7f7e85456700 1 -- 172.21.15.89:0/854096244 wait complete. 2023-10-10T20:35:08.485 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:35:08.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.602+0000 7f0ac4fac700 1 Processor -- start 2023-10-10T20:35:08.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.602+0000 7f0ac4fac700 1 -- start start 2023-10-10T20:35:08.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.603+0000 7f0ac4fac700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ac0159ec0 0x7f0ac00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:08.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.603+0000 7f0ac4fac700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0ac00b53d0 0x7f0ac00cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:08.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.603+0000 7f0ac4fac700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0ac00cce40 0x7f0ac00cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:08.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.603+0000 7f0ac4fac700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac00b5b00 con 0x7f0ac00cce40 2023-10-10T20:35:08.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.603+0000 7f0ac4fac700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac00b5da0 con 0x7f0ac00b53d0 2023-10-10T20:35:08.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.603+0000 7f0ac4fac700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00b6080 con 0x7f0ac0159ec0 2023-10-10T20:35:08.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.603+0000 7f0abed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0ac00cce40 0x7f0ac00cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:08.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.603+0000 7f0abed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0ac00cce40 0x7f0ac00cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42518/0 (socket says 172.21.15.89:42518) 2023-10-10T20:35:08.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.603+0000 7f0abed9d700 1 -- 172.21.15.89:0/4236804160 learned_addr learned my addr 172.21.15.89:0/4236804160 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:08.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.603+0000 7f0abe59c700 1 --2- 172.21.15.89:0/4236804160 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ac0159ec0 0x7f0ac00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:08.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.603+0000 7f0abdd9b700 1 --2- 172.21.15.89:0/4236804160 >> v2:172.21.15.111:3300/0 conn(0x7f0ac00b53d0 0x7f0ac00cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:08.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.603+0000 7f0abed9d700 1 -- 172.21.15.89:0/4236804160 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ac0159ec0 msgr2=0x7f0ac00b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:08.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.603+0000 7f0abed9d700 1 --2- 172.21.15.89:0/4236804160 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ac0159ec0 0x7f0ac00b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:08.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.603+0000 7f0abed9d700 1 -- 172.21.15.89:0/4236804160 >> v2:172.21.15.111:3300/0 conn(0x7f0ac00b53d0 msgr2=0x7f0ac00cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:08.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.603+0000 7f0abed9d700 1 --2- 172.21.15.89:0/4236804160 >> v2:172.21.15.111:3300/0 conn(0x7f0ac00b53d0 0x7f0ac00cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:08.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.603+0000 7f0abed9d700 1 -- 172.21.15.89:0/4236804160 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac00cc100 con 0x7f0ac00cce40 2023-10-10T20:35:08.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.603+0000 7f0abdd9b700 1 --2- 172.21.15.89:0/4236804160 >> v2:172.21.15.111:3300/0 conn(0x7f0ac00b53d0 0x7f0ac00cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:08.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.604+0000 7f0abed9d700 1 --2- 172.21.15.89:0/4236804160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac00cce40 0x7f0ac00cd280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0089880 tx=0x7f0ab8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5048728a47c98ab server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:08.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.604+0000 7f0abd59a700 1 -- 172.21.15.89:0/4236804160 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab800b860 con 0x7f0ac00cce40 2023-10-10T20:35:08.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.604+0000 7f0abd59a700 1 -- 172.21.15.89:0/4236804160 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab800b9c0 con 0x7f0ac00cce40 2023-10-10T20:35:08.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.604+0000 7f0abd59a700 1 -- 172.21.15.89:0/4236804160 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab800bd60 con 0x7f0ac00cce40 2023-10-10T20:35:08.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.604+0000 7f0ac4fac700 1 -- 172.21.15.89:0/4236804160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac00cce40 msgr2=0x7f0ac00cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.604+0000 7f0ac4fac700 1 --2- 172.21.15.89:0/4236804160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac00cce40 0x7f0ac00cd280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0089880 tx=0x7f0ab8009250 comp rx=0 tx=0).stop 2023-10-10T20:35:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.604+0000 7f0ac4fac700 1 -- 172.21.15.89:0/4236804160 shutdown_connections 2023-10-10T20:35:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.604+0000 7f0ac4fac700 1 --2- 172.21.15.89:0/4236804160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac00cce40 0x7f0ac00cd280 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.604+0000 7f0ac4fac700 1 --2- 172.21.15.89:0/4236804160 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ac0159ec0 0x7f0ac00b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.604+0000 7f0ac4fac700 1 --2- 172.21.15.89:0/4236804160 >> v2:172.21.15.111:3300/0 conn(0x7f0ac00b53d0 0x7f0ac00cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.604+0000 7f0ac4fac700 1 -- 172.21.15.89:0/4236804160 >> 172.21.15.89:0/4236804160 conn(0x7f0ac00999f0 msgr2=0x7f0ac00b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.604+0000 7f0ac4fac700 1 -- 172.21.15.89:0/4236804160 shutdown_connections 2023-10-10T20:35:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.604+0000 7f0ac4fac700 1 -- 172.21.15.89:0/4236804160 wait complete. 2023-10-10T20:35:08.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.606+0000 7f0ac4fac700 1 Processor -- start 2023-10-10T20:35:08.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.606+0000 7f0ac4fac700 1 -- start start 2023-10-10T20:35:08.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.606+0000 7f0ac4fac700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ac00b53d0 0x7f0ac0169c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:08.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.606+0000 7f0ac4fac700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac00cce40 0x7f0ac016a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:08.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.606+0000 7f0ac4fac700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ac0159ec0 0x7f0ac01471b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.606+0000 7f0ac4fac700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac0145000 con 0x7f0ac00cce40 2023-10-10T20:35:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.606+0000 7f0ac4fac700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac0144d50 con 0x7f0ac0159ec0 2023-10-10T20:35:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.606+0000 7f0ac4fac700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac016b9d0 con 0x7f0ac00b53d0 2023-10-10T20:35:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.606+0000 7f0abdd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac00cce40 0x7f0ac016a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.606+0000 7f0abdd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac00cce40 0x7f0ac016a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42528/0 (socket says 172.21.15.89:42528) 2023-10-10T20:35:08.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.606+0000 7f0abdd9b700 1 -- 172.21.15.89:0/128800219 learned_addr learned my addr 172.21.15.89:0/128800219 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:08.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.607+0000 7f0abe59c700 1 --2- 172.21.15.89:0/128800219 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ac00b53d0 0x7f0ac0169c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:08.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.607+0000 7f0abed9d700 1 --2- 172.21.15.89:0/128800219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ac0159ec0 0x7f0ac01471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:08.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.607+0000 7f0abdd9b700 1 -- 172.21.15.89:0/128800219 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ac00b53d0 msgr2=0x7f0ac0169c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:08.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.607+0000 7f0abdd9b700 1 --2- 172.21.15.89:0/128800219 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ac00b53d0 0x7f0ac0169c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:08.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.607+0000 7f0abdd9b700 1 -- 172.21.15.89:0/128800219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ac0159ec0 msgr2=0x7f0ac01471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:08.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.607+0000 7f0abdd9b700 1 --2- 172.21.15.89:0/128800219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ac0159ec0 0x7f0ac01471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:08.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.607+0000 7f0abdd9b700 1 -- 172.21.15.89:0/128800219 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab80094f0 con 0x7f0ac00cce40 2023-10-10T20:35:08.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.607+0000 7f0abdd9b700 1 --2- 172.21.15.89:0/128800219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac00cce40 0x7f0ac016a190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400c860 tx=0x7f0ab4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:08.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.607+0000 7f0aae7fc700 1 -- 172.21.15.89:0/128800219 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4011600 con 0x7f0ac00cce40 2023-10-10T20:35:08.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.607+0000 7f0aae7fc700 1 -- 172.21.15.89:0/128800219 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab4011760 con 0x7f0ac00cce40 2023-10-10T20:35:08.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.607+0000 7f0aae7fc700 1 -- 172.21.15.89:0/128800219 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4011b00 con 0x7f0ac00cce40 2023-10-10T20:35:08.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.607+0000 7f0ac4fac700 1 -- 172.21.15.89:0/128800219 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac016ade0 con 0x7f0ac00cce40 2023-10-10T20:35:08.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.607+0000 7f0ac4fac700 1 -- 172.21.15.89:0/128800219 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac016b380 con 0x7f0ac00cce40 2023-10-10T20:35:08.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.608+0000 7f0ac4fac700 1 -- 172.21.15.89:0/128800219 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac0063c90 con 0x7f0ac00cce40 2023-10-10T20:35:08.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.608+0000 7f0aae7fc700 1 -- 172.21.15.89:0/128800219 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0ab400a3e0 con 0x7f0ac00cce40 2023-10-10T20:35:08.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.609+0000 7f0aae7fc700 1 --2- 172.21.15.89:0/128800219 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0aa0066f30 0x7f0aa00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:08.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.609+0000 7f0aae7fc700 1 -- 172.21.15.89:0/128800219 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f0ab4084610 con 0x7f0ac00cce40 2023-10-10T20:35:08.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.609+0000 7f0abe59c700 1 --2- 172.21.15.89:0/128800219 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0aa0066f30 0x7f0aa00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:08.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.609+0000 7f0abe59c700 1 --2- 172.21.15.89:0/128800219 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0aa0066f30 0x7f0aa00693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0007540 tx=0x7f0ab0009750 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:08.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.613+0000 7f0aae7fc700 1 -- 172.21.15.89:0/128800219 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ab4051830 con 0x7f0ac00cce40 2023-10-10T20:35:08.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:08.797+0000 7f0ac4fac700 1 -- 172.21.15.89:0/128800219 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0ac016bb60 con 0x7f0ac00cce40 2023-10-10T20:35:09.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.476+0000 7f0aae7fc700 1 -- 172.21.15.89:0/128800219 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v22) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0ab4054e50 con 0x7f0ac00cce40 2023-10-10T20:35:09.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.480+0000 7f0ac4fac700 1 -- 172.21.15.89:0/128800219 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0aa0066f30 msgr2=0x7f0aa00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:09.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.480+0000 7f0ac4fac700 1 --2- 172.21.15.89:0/128800219 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0aa0066f30 0x7f0aa00693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0007540 tx=0x7f0ab0009750 comp rx=0 tx=0).stop 2023-10-10T20:35:09.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.480+0000 7f0ac4fac700 1 -- 172.21.15.89:0/128800219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac00cce40 msgr2=0x7f0ac016a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:09.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.480+0000 7f0ac4fac700 1 --2- 172.21.15.89:0/128800219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac00cce40 0x7f0ac016a190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400c860 tx=0x7f0ab4008040 comp rx=0 tx=0).stop 2023-10-10T20:35:09.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.480+0000 7f0ac4fac700 1 -- 172.21.15.89:0/128800219 shutdown_connections 2023-10-10T20:35:09.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.480+0000 7f0ac4fac700 1 --2- 172.21.15.89:0/128800219 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0aa0066f30 0x7f0aa00693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:09.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.480+0000 7f0ac4fac700 1 --2- 172.21.15.89:0/128800219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ac0159ec0 0x7f0ac01471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:09.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.480+0000 7f0ac4fac700 1 --2- 172.21.15.89:0/128800219 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ac00b53d0 0x7f0ac0169c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:09.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.480+0000 7f0ac4fac700 1 --2- 172.21.15.89:0/128800219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac00cce40 0x7f0ac016a190 secure :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400c860 tx=0x7f0ab4008040 comp rx=0 tx=0).stop 2023-10-10T20:35:09.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.480+0000 7f0ac4fac700 1 -- 172.21.15.89:0/128800219 >> 172.21.15.89:0/128800219 conn(0x7f0ac00999f0 msgr2=0x7f0ac00c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:09.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.481+0000 7f0ac4fac700 1 -- 172.21.15.89:0/128800219 shutdown_connections 2023-10-10T20:35:09.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.481+0000 7f0ac4fac700 1 -- 172.21.15.89:0/128800219 wait complete. 2023-10-10T20:35:09.485 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:35:09.497 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:35:09.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.616+0000 7f7e3130e700 1 Processor -- start 2023-10-10T20:35:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.616+0000 7f7e3130e700 1 -- start start 2023-10-10T20:35:09.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.617+0000 7f7e3130e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7e2c0b0790 0x7f7e2c0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:09.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.617+0000 7f7e3130e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2c0cfa00 con 0x7f7e2c1591e0 2023-10-10T20:35:09.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.617+0000 7f7e3130e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2c0cb580 con 0x7f7e2c0b03c0 2023-10-10T20:35:09.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.617+0000 7f7e3130e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2c0cf160 con 0x7f7e2c0b0790 2023-10-10T20:35:09.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.617+0000 7f7e30b0d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7e2c1591e0 0x7f7e2c159600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40448/0 (socket says 172.21.15.89:40448) 2023-10-10T20:35:09.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.617+0000 7f7e30b0d700 1 -- 172.21.15.89:0/354522203 learned_addr learned my addr 172.21.15.89:0/354522203 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:09.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.617+0000 7f7e2affd700 1 -- 172.21.15.89:0/354522203 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774352543 0 0) 0x7f7e2c0cfa00 con 0x7f7e2c1591e0 2023-10-10T20:35:09.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.617+0000 7f7e2affd700 1 -- 172.21.15.89:0/354522203 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e14003570 con 0x7f7e2c1591e0 2023-10-10T20:35:09.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.617+0000 7f7e2b7fe700 1 --2- 172.21.15.89:0/354522203 >> v2:172.21.15.111:3300/0 conn(0x7f7e2c0b0790 0x7f7e2c0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:09.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e2affd700 1 -- 172.21.15.89:0/354522203 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2006120161 0 0) 0x7f7e14003570 con 0x7f7e2c1591e0 2023-10-10T20:35:09.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e2affd700 1 -- 172.21.15.89:0/354522203 >> v2:172.21.15.111:3300/0 conn(0x7f7e2c0b0790 msgr2=0x7f7e2c0cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:09.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e2affd700 1 --2- 172.21.15.89:0/354522203 >> v2:172.21.15.111:3300/0 conn(0x7f7e2c0b0790 0x7f7e2c0cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:09.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e2affd700 1 -- 172.21.15.89:0/354522203 >> v1:172.21.15.111:6789/0 conn(0x7f7e2c0b03c0 legacy=0x7f7e2c0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:09.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e2affd700 1 -- 172.21.15.89:0/354522203 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c0cf440 con 0x7f7e2c1591e0 2023-10-10T20:35:09.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e2affd700 1 -- 172.21.15.89:0/354522203 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f7e240017f0 con 0x7f7e2c1591e0 2023-10-10T20:35:09.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e2affd700 1 -- 172.21.15.89:0/354522203 >> v1:172.21.15.89:6789/0 conn(0x7f7e2c1591e0 legacy=0x7f7e2c159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:09.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e2affd700 1 --2- 172.21.15.89:0/354522203 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e14003a90 0x7f7e14003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:09.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e2affd700 1 --2- 172.21.15.89:0/354522203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e14004810 0x7f7e14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:09.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e2affd700 1 --2- 172.21.15.89:0/354522203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e140052e0 0x7f7e1400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:09.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e2affd700 1 -- 172.21.15.89:0/354522203 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c0cf160 con 0x7f7e140052e0 2023-10-10T20:35:09.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e2affd700 1 -- 172.21.15.89:0/354522203 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1400c230 con 0x7f7e14004810 2023-10-10T20:35:09.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e2affd700 1 -- 172.21.15.89:0/354522203 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1400c3f0 con 0x7f7e14003a90 2023-10-10T20:35:09.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e30b0d700 1 --2- 172.21.15.89:0/354522203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e140052e0 0x7f7e1400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:09.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e2b7fe700 1 --2- 172.21.15.89:0/354522203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e14004810 0x7f7e14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:09.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e2bfff700 1 --2- 172.21.15.89:0/354522203 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e14003a90 0x7f7e14003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:09.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e30b0d700 1 -- 172.21.15.89:0/354522203 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e14003a90 msgr2=0x7f7e14003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:09.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e30b0d700 1 --2- 172.21.15.89:0/354522203 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e14003a90 0x7f7e14003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:09.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e30b0d700 1 -- 172.21.15.89:0/354522203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e14004810 msgr2=0x7f7e14004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:09.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e30b0d700 1 --2- 172.21.15.89:0/354522203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e14004810 0x7f7e14004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:09.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e30b0d700 1 -- 172.21.15.89:0/354522203 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e14003570 con 0x7f7e140052e0 2023-10-10T20:35:09.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e2bfff700 1 --2- 172.21.15.89:0/354522203 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e14003a90 0x7f7e14003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:09.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e30b0d700 1 -- 172.21.15.89:0/354522203 reap_dead start 2023-10-10T20:35:09.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e30b0d700 1 --2- 172.21.15.89:0/354522203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e140052e0 0x7f7e1400ba60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400a900 tx=0x7f7e2c0b1a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c308d425e86fe8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:09.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e2affd700 1 -- 172.21.15.89:0/354522203 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e240070d0 con 0x7f7e140052e0 2023-10-10T20:35:09.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e2affd700 1 -- 172.21.15.89:0/354522203 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e24007230 con 0x7f7e140052e0 2023-10-10T20:35:09.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.618+0000 7f7e2affd700 1 -- 172.21.15.89:0/354522203 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e240075f0 con 0x7f7e140052e0 2023-10-10T20:35:09.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.619+0000 7f7e3130e700 1 -- 172.21.15.89:0/354522203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e140052e0 msgr2=0x7f7e1400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:09.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.619+0000 7f7e3130e700 1 --2- 172.21.15.89:0/354522203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e140052e0 0x7f7e1400ba60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400a900 tx=0x7f7e2c0b1a10 comp rx=0 tx=0).stop 2023-10-10T20:35:09.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.619+0000 7f7e3130e700 1 -- 172.21.15.89:0/354522203 shutdown_connections 2023-10-10T20:35:09.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.619+0000 7f7e3130e700 1 --2- 172.21.15.89:0/354522203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e140052e0 0x7f7e1400ba60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:09.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.619+0000 7f7e3130e700 1 -- 172.21.15.89:0/354522203 >> 172.21.15.89:0/354522203 conn(0x7f7e2c063230 msgr2=0x7f7e2c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:09.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.619+0000 7f7e3130e700 1 -- 172.21.15.89:0/354522203 shutdown_connections 2023-10-10T20:35:09.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.619+0000 7f7e3130e700 1 -- 172.21.15.89:0/354522203 wait complete. 2023-10-10T20:35:09.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.620+0000 7f7e3130e700 1 Processor -- start 2023-10-10T20:35:09.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.620+0000 7f7e3130e700 1 -- start start 2023-10-10T20:35:09.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.621+0000 7f7e3130e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e140052e0 0x7f7e2c157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:09.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.621+0000 7f7e3130e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e2c0c5d80 0x7f7e2c1583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:09.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.621+0000 7f7e3130e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e2c16d9e0 0x7f7e2c16de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:09.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.621+0000 7f7e3130e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c1523a0 con 0x7f7e140052e0 2023-10-10T20:35:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.621+0000 7f7e3130e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c156420 con 0x7f7e2c16d9e0 2023-10-10T20:35:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.621+0000 7f7e3130e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c16e710 con 0x7f7e2c0c5d80 2023-10-10T20:35:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.621+0000 7f7e2bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e140052e0 0x7f7e2c157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.621+0000 7f7e2bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e140052e0 0x7f7e2c157e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42554/0 (socket says 172.21.15.89:42554) 2023-10-10T20:35:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.621+0000 7f7e2bfff700 1 -- 172.21.15.89:0/3436486637 learned_addr learned my addr 172.21.15.89:0/3436486637 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.621+0000 7f7e2b7fe700 1 --2- 172.21.15.89:0/3436486637 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e2c0c5d80 0x7f7e2c1583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.621+0000 7f7e30b0d700 1 --2- 172.21.15.89:0/3436486637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e2c16d9e0 0x7f7e2c16de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.621+0000 7f7e2bfff700 1 -- 172.21.15.89:0/3436486637 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e2c0c5d80 msgr2=0x7f7e2c1583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.621+0000 7f7e2bfff700 1 --2- 172.21.15.89:0/3436486637 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e2c0c5d80 0x7f7e2c1583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.621+0000 7f7e2bfff700 1 -- 172.21.15.89:0/3436486637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e2c16d9e0 msgr2=0x7f7e2c16de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.621+0000 7f7e2bfff700 1 --2- 172.21.15.89:0/3436486637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e2c16d9e0 0x7f7e2c16de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.621+0000 7f7e2bfff700 1 -- 172.21.15.89:0/3436486637 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c155af0 con 0x7f7e140052e0 2023-10-10T20:35:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.621+0000 7f7e30b0d700 1 --2- 172.21.15.89:0/3436486637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e2c16d9e0 0x7f7e2c16de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.621+0000 7f7e2bfff700 1 --2- 172.21.15.89:0/3436486637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e140052e0 0x7f7e2c157e90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00e550 tx=0x7f7e1c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.622+0000 7f7e13fff700 1 -- 172.21.15.89:0/3436486637 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c011460 con 0x7f7e140052e0 2023-10-10T20:35:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.622+0000 7f7e13fff700 1 -- 172.21.15.89:0/3436486637 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1c010040 con 0x7f7e140052e0 2023-10-10T20:35:09.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.622+0000 7f7e13fff700 1 -- 172.21.15.89:0/3436486637 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c0117f0 con 0x7f7e140052e0 2023-10-10T20:35:09.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.622+0000 7f7e3130e700 1 -- 172.21.15.89:0/3436486637 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2c155dd0 con 0x7f7e140052e0 2023-10-10T20:35:09.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.622+0000 7f7e3130e700 1 -- 172.21.15.89:0/3436486637 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c16e3a0 con 0x7f7e140052e0 2023-10-10T20:35:09.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.622+0000 7f7e3130e700 1 -- 172.21.15.89:0/3436486637 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2c156700 con 0x7f7e140052e0 2023-10-10T20:35:09.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.623+0000 7f7e13fff700 1 -- 172.21.15.89:0/3436486637 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7e1c011ba0 con 0x7f7e140052e0 2023-10-10T20:35:09.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.623+0000 7f7e13fff700 1 --2- 172.21.15.89:0/3436486637 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7e18066f30 0x7f7e180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:09.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.623+0000 7f7e13fff700 1 -- 172.21.15.89:0/3436486637 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f7e1c083620 con 0x7f7e140052e0 2023-10-10T20:35:09.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.623+0000 7f7e2b7fe700 1 --2- 172.21.15.89:0/3436486637 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7e18066f30 0x7f7e180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:09.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.624+0000 7f7e2b7fe700 1 --2- 172.21.15.89:0/3436486637 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7e18066f30 0x7f7e180693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7e20006f00 tx=0x7f7e200091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:09.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.627+0000 7f7e13fff700 1 -- 172.21.15.89:0/3436486637 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e1c04d5a0 con 0x7f7e140052e0 2023-10-10T20:35:09.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:09.810+0000 7f7e3130e700 1 -- 172.21.15.89:0/3436486637 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7e2c04c3c0 con 0x7f7e140052e0 2023-10-10T20:35:10.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.477+0000 7f7e13fff700 1 -- 172.21.15.89:0/3436486637 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7e1c054530 con 0x7f7e140052e0 2023-10-10T20:35:10.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.480+0000 7f7e3130e700 1 -- 172.21.15.89:0/3436486637 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7e18066f30 msgr2=0x7f7e180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:10.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.480+0000 7f7e3130e700 1 --2- 172.21.15.89:0/3436486637 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7e18066f30 0x7f7e180693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7e20006f00 tx=0x7f7e200091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:10.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.480+0000 7f7e3130e700 1 -- 172.21.15.89:0/3436486637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e140052e0 msgr2=0x7f7e2c157e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:10.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.480+0000 7f7e3130e700 1 --2- 172.21.15.89:0/3436486637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e140052e0 0x7f7e2c157e90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00e550 tx=0x7f7e1c009040 comp rx=0 tx=0).stop 2023-10-10T20:35:10.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.480+0000 7f7e3130e700 1 -- 172.21.15.89:0/3436486637 shutdown_connections 2023-10-10T20:35:10.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.480+0000 7f7e3130e700 1 --2- 172.21.15.89:0/3436486637 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7e18066f30 0x7f7e180693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:10.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.480+0000 7f7e3130e700 1 --2- 172.21.15.89:0/3436486637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e2c16d9e0 0x7f7e2c16de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:10.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.480+0000 7f7e3130e700 1 --2- 172.21.15.89:0/3436486637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e140052e0 0x7f7e2c157e90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:10.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.480+0000 7f7e3130e700 1 --2- 172.21.15.89:0/3436486637 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e2c0c5d80 0x7f7e2c1583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:10.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.480+0000 7f7e3130e700 1 -- 172.21.15.89:0/3436486637 >> 172.21.15.89:0/3436486637 conn(0x7f7e2c063230 msgr2=0x7f7e2c0ce360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:10.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.480+0000 7f7e3130e700 1 -- 172.21.15.89:0/3436486637 shutdown_connections 2023-10-10T20:35:10.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.480+0000 7f7e3130e700 1 -- 172.21.15.89:0/3436486637 wait complete. 2023-10-10T20:35:10.485 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' removed 2023-10-10T20:35:10.496 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:35:10.496 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:35:10.496 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:35:10.497 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:35:10.497 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:35:10.497 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:35:10.498 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:35:10.498 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:35:10.498 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:35:10.498 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:35:10.499 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:35:10.499 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:35:10.499 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:35:10.499 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:35:10.500 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:35:10.500 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:35:10.506 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:35:10.506 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:35:10.508 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:35:10.508 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:35:10.509 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:35:10.509 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:35:10.510 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:35:10.511 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:35:10.550 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:35:10.550 DEBUG:teuthology.parallel:result is None 2023-10-10T20:35:10.551 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:35:10.551 DEBUG:teuthology.parallel:result is None 2023-10-10T20:35:10.552 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:35:10.553 DEBUG:teuthology.parallel:result is None 2023-10-10T20:35:10.554 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:35:10.554 DEBUG:teuthology.parallel:result is None 2023-10-10T20:35:10.554 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:35:10.623 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-10T20:35:10.624 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:35:10.632 INFO:tasks.ceph.mds.b.smithi111.stdout:starting mds.b at 2023-10-10T20:35:10.632 INFO:tasks.ceph.mds.d.smithi111.stdout:starting mds.d at 2023-10-10T20:35:10.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.678+0000 7f9099897700 1 Processor -- start 2023-10-10T20:35:10.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.678+0000 7f9099897700 1 -- start start 2023-10-10T20:35:10.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.679+0000 7f9099897700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9094150550 0x7f9094157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:10.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.679+0000 7f9099897700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f90940c7410 0x7f9094069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:10.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.679+0000 7f9099897700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f909406e2a0 con 0x7f9094150180 2023-10-10T20:35:10.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.679+0000 7f9099897700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f909406b2e0 con 0x7f90940c7410 2023-10-10T20:35:10.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.679+0000 7f9099897700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90940c78e0 con 0x7f9094150550 2023-10-10T20:35:10.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.679+0000 7f9098895700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9094150180 0x7f90940c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40474/0 (socket says 172.21.15.89:40474) 2023-10-10T20:35:10.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.679+0000 7f9098895700 1 -- 172.21.15.89:0/572047262 learned_addr learned my addr 172.21.15.89:0/572047262 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:10.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.679+0000 7f9093fff700 1 --2- 172.21.15.89:0/572047262 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9094150550 0x7f9094157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:10.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.679+0000 7f9099096700 1 --2- 172.21.15.89:0/572047262 >> v2:172.21.15.111:3300/0 conn(0x7f90940c7410 0x7f9094069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:10.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.680+0000 7f90937fe700 1 -- 172.21.15.89:0/572047262 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583819676 0 0) 0x7f909406e2a0 con 0x7f9094150180 2023-10-10T20:35:10.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.680+0000 7f90937fe700 1 -- 172.21.15.89:0/572047262 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9084004680 con 0x7f9094150180 2023-10-10T20:35:10.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.680+0000 7f9093fff700 1 -- 172.21.15.89:0/572047262 >> v2:172.21.15.111:3300/0 conn(0x7f90940c7410 msgr2=0x7f9094069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:10.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.680+0000 7f9093fff700 1 --2- 172.21.15.89:0/572047262 >> v2:172.21.15.111:3300/0 conn(0x7f90940c7410 0x7f9094069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:10.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.680+0000 7f9093fff700 1 -- 172.21.15.89:0/572047262 >> v1:172.21.15.89:6789/0 conn(0x7f9094150180 legacy=0x7f90940c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:10.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.680+0000 7f9093fff700 1 -- 172.21.15.89:0/572047262 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90940c7bc0 con 0x7f9094150550 2023-10-10T20:35:10.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.680+0000 7f9099096700 1 --2- 172.21.15.89:0/572047262 >> v2:172.21.15.111:3300/0 conn(0x7f90940c7410 0x7f9094069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:35:10.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.680+0000 7f9093fff700 1 --2- 172.21.15.89:0/572047262 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9094150550 0x7f9094157770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9080009910 tx=0x7f90800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be73d49d534ade93 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:10.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.680+0000 7f90937fe700 1 -- 172.21.15.89:0/572047262 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080011630 con 0x7f9094150550 2023-10-10T20:35:10.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.680+0000 7f90937fe700 1 -- 172.21.15.89:0/572047262 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9080011790 con 0x7f9094150550 2023-10-10T20:35:10.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.680+0000 7f90937fe700 1 -- 172.21.15.89:0/572047262 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080011b30 con 0x7f9094150550 2023-10-10T20:35:10.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.681+0000 7f9099897700 1 -- 172.21.15.89:0/572047262 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9094150550 msgr2=0x7f9094157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:10.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.681+0000 7f9099897700 1 --2- 172.21.15.89:0/572047262 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9094150550 0x7f9094157770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9080009910 tx=0x7f90800091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:10.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.681+0000 7f9099897700 1 -- 172.21.15.89:0/572047262 shutdown_connections 2023-10-10T20:35:10.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.681+0000 7f9099897700 1 --2- 172.21.15.89:0/572047262 >> v2:172.21.15.111:3300/0 conn(0x7f90940c7410 0x7f9094069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:10.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.681+0000 7f9099897700 1 --2- 172.21.15.89:0/572047262 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9094150550 0x7f9094157770 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:10.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.681+0000 7f9099897700 1 -- 172.21.15.89:0/572047262 >> 172.21.15.89:0/572047262 conn(0x7f90940cc1f0 msgr2=0x7f90940cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:10.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.682+0000 7f9099897700 1 -- 172.21.15.89:0/572047262 shutdown_connections 2023-10-10T20:35:10.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.682+0000 7f9099897700 1 -- 172.21.15.89:0/572047262 wait complete. 2023-10-10T20:35:10.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.683+0000 7f9099897700 1 Processor -- start 2023-10-10T20:35:10.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.684+0000 7f9099897700 1 -- start start 2023-10-10T20:35:10.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.684+0000 7f9099897700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90940c7410 0x7f90940ac490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:10.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.684+0000 7f9099897700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9094150180 0x7f909414a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:10.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.685+0000 7f9099897700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9094150550 0x7f909414ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:10.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.685+0000 7f9099897700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90940afa40 con 0x7f90940c7410 2023-10-10T20:35:10.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.685+0000 7f9099897700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90940afd20 con 0x7f9094150180 2023-10-10T20:35:10.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.685+0000 7f9098895700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90940c7410 0x7f90940ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:10.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.685+0000 7f9099897700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90940b0000 con 0x7f9094150550 2023-10-10T20:35:10.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.685+0000 7f9098895700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90940c7410 0x7f90940ac490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42604/0 (socket says 172.21.15.89:42604) 2023-10-10T20:35:10.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.685+0000 7f9098895700 1 -- 172.21.15.89:0/2583607399 learned_addr learned my addr 172.21.15.89:0/2583607399 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:10.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.685+0000 7f9093fff700 1 --2- 172.21.15.89:0/2583607399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9094150180 0x7f909414a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:10.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.685+0000 7f9099096700 1 --2- 172.21.15.89:0/2583607399 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9094150550 0x7f909414ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:10.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.685+0000 7f9093fff700 1 -- 172.21.15.89:0/2583607399 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9094150550 msgr2=0x7f909414ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:10.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.685+0000 7f9093fff700 1 --2- 172.21.15.89:0/2583607399 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9094150550 0x7f909414ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:10.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.685+0000 7f9093fff700 1 -- 172.21.15.89:0/2583607399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90940c7410 msgr2=0x7f90940ac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:10.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.685+0000 7f9093fff700 1 --2- 172.21.15.89:0/2583607399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90940c7410 0x7f90940ac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:10.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.685+0000 7f9093fff700 1 -- 172.21.15.89:0/2583607399 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c009a20 con 0x7f9094150180 2023-10-10T20:35:10.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.685+0000 7f9093fff700 1 --2- 172.21.15.89:0/2583607399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9094150180 0x7f909414a5e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9080016000 tx=0x7f9080017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:10.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.686+0000 7f9090ff9700 1 -- 172.21.15.89:0/2583607399 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080000c40 con 0x7f9094150180 2023-10-10T20:35:10.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.686+0000 7f9090ff9700 1 -- 172.21.15.89:0/2583607399 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9080011900 con 0x7f9094150180 2023-10-10T20:35:10.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.686+0000 7f9090ff9700 1 -- 172.21.15.89:0/2583607399 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080025420 con 0x7f9094150180 2023-10-10T20:35:10.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.686+0000 7f9099897700 1 -- 172.21.15.89:0/2583607399 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9080009420 con 0x7f9094150180 2023-10-10T20:35:10.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.686+0000 7f9099897700 1 -- 172.21.15.89:0/2583607399 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90940b05c0 con 0x7f9094150180 2023-10-10T20:35:10.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.686+0000 7f9099897700 1 -- 172.21.15.89:0/2583607399 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90940b07b0 con 0x7f9094150180 2023-10-10T20:35:10.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.687+0000 7f9090ff9700 1 -- 172.21.15.89:0/2583607399 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9080023070 con 0x7f9094150180 2023-10-10T20:35:10.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.688+0000 7f9090ff9700 1 --2- 172.21.15.89:0/2583607399 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9078066f30 0x7f90780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:10.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.688+0000 7f9090ff9700 1 -- 172.21.15.89:0/2583607399 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f90800851d0 con 0x7f9094150180 2023-10-10T20:35:10.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.688+0000 7f9098895700 1 --2- 172.21.15.89:0/2583607399 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9078066f30 0x7f90780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:10.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.688+0000 7f9098895700 1 --2- 172.21.15.89:0/2583607399 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9078066f30 0x7f90780693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9088001ad0 tx=0x7f9088009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:10.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.692+0000 7f9090ff9700 1 -- 172.21.15.89:0/2583607399 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9080052740 con 0x7f9094150180 2023-10-10T20:35:10.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:10.890+0000 7f9099897700 1 -- 172.21.15.89:0/2583607399 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f90940afe60 con 0x7f9094150180 2023-10-10T20:35:11.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.490+0000 7f9090ff9700 1 -- 172.21.15.89:0/2583607399 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v25) v1 ==== 97+0+0 (secure 0 0 0) 0x7f90940afe60 con 0x7f9094150180 2023-10-10T20:35:11.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.493+0000 7f9099897700 1 -- 172.21.15.89:0/2583607399 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9078066f30 msgr2=0x7f90780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:11.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.493+0000 7f9099897700 1 --2- 172.21.15.89:0/2583607399 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9078066f30 0x7f90780693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9088001ad0 tx=0x7f9088009040 comp rx=0 tx=0).stop 2023-10-10T20:35:11.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.493+0000 7f9099897700 1 -- 172.21.15.89:0/2583607399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9094150180 msgr2=0x7f909414a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:11.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.493+0000 7f9099897700 1 --2- 172.21.15.89:0/2583607399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9094150180 0x7f909414a5e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9080016000 tx=0x7f9080017000 comp rx=0 tx=0).stop 2023-10-10T20:35:11.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.493+0000 7f9099897700 1 -- 172.21.15.89:0/2583607399 shutdown_connections 2023-10-10T20:35:11.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.493+0000 7f9099897700 1 --2- 172.21.15.89:0/2583607399 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9094150550 0x7f909414ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:11.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.493+0000 7f9099897700 1 --2- 172.21.15.89:0/2583607399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90940c7410 0x7f90940ac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:11.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.493+0000 7f9099897700 1 --2- 172.21.15.89:0/2583607399 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9078066f30 0x7f90780693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:11.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.493+0000 7f9099897700 1 --2- 172.21.15.89:0/2583607399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9094150180 0x7f909414a5e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:11.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.493+0000 7f9099897700 1 -- 172.21.15.89:0/2583607399 >> 172.21.15.89:0/2583607399 conn(0x7f90940cc1f0 msgr2=0x7f90940c8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:11.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.493+0000 7f9099897700 1 -- 172.21.15.89:0/2583607399 shutdown_connections 2023-10-10T20:35:11.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.493+0000 7f9099897700 1 -- 172.21.15.89:0/2583607399 wait complete. 2023-10-10T20:35:11.498 INFO:teuthology.orchestra.run.smithi089.stderr: removed all blocklist entries 2023-10-10T20:35:11.509 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:35:11.610 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:35:11.612 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:35:11.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.623+0000 7f9c04196700 1 Processor -- start 2023-10-10T20:35:11.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.624+0000 7f9c04196700 1 -- start start 2023-10-10T20:35:11.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.624+0000 7f9c04196700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9bfc06aa40 0x7f9bfc0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:11.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.624+0000 7f9c04196700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bfc0c91e0 0x7f9bfc160390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:11.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.624+0000 7f9c04196700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bfc0cc210 con 0x7f9bfc06bac0 2023-10-10T20:35:11.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.624+0000 7f9c04196700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bfc0cc4f0 con 0x7f9bfc06aa40 2023-10-10T20:35:11.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.624+0000 7f9c04196700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0cc7d0 con 0x7f9bfc0c91e0 2023-10-10T20:35:11.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.625+0000 7f9c01f32700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9bfc06aa40 0x7f9bfc0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:11.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.625+0000 7f9c01f32700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9bfc06aa40 0x7f9bfc0cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42608/0 (socket says 172.21.15.89:42608) 2023-10-10T20:35:11.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.625+0000 7f9c01f32700 1 -- 172.21.15.89:0/3798817385 learned_addr learned my addr 172.21.15.89:0/3798817385 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:11.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.625+0000 7f9c02733700 1 --2- 172.21.15.89:0/3798817385 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bfc0c91e0 0x7f9bfc160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:11.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.625+0000 7f9c01f32700 1 -- 172.21.15.89:0/3798817385 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bfc0c91e0 msgr2=0x7f9bfc160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:11.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.625+0000 7f9c01f32700 1 --2- 172.21.15.89:0/3798817385 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bfc0c91e0 0x7f9bfc160390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.625+0000 7f9c01f32700 1 -- 172.21.15.89:0/3798817385 >> v1:172.21.15.111:6789/0 conn(0x7f9bfc06bac0 legacy=0x7f9bfc06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.625+0000 7f9c01f32700 1 -- 172.21.15.89:0/3798817385 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc0ccab0 con 0x7f9bfc06aa40 2023-10-10T20:35:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.625+0000 7f9c01f32700 1 --2- 172.21.15.89:0/3798817385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc06aa40 0x7f9bfc0cbc00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009990 tx=0x7f9bec009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f203e6f1f4bb45 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.625+0000 7f9c00f30700 1 -- 172.21.15.89:0/3798817385 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec009e80 con 0x7f9bfc06aa40 2023-10-10T20:35:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.625+0000 7f9c00f30700 1 -- 172.21.15.89:0/3798817385 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec005c10 con 0x7f9bfc06aa40 2023-10-10T20:35:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.625+0000 7f9c00f30700 1 -- 172.21.15.89:0/3798817385 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00b650 con 0x7f9bfc06aa40 2023-10-10T20:35:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.626+0000 7f9c04196700 1 -- 172.21.15.89:0/3798817385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc06aa40 msgr2=0x7f9bfc0cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.626+0000 7f9c04196700 1 --2- 172.21.15.89:0/3798817385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc06aa40 0x7f9bfc0cbc00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009990 tx=0x7f9bec009220 comp rx=0 tx=0).stop 2023-10-10T20:35:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.626+0000 7f9c04196700 1 -- 172.21.15.89:0/3798817385 shutdown_connections 2023-10-10T20:35:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.626+0000 7f9c04196700 1 --2- 172.21.15.89:0/3798817385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc06aa40 0x7f9bfc0cbc00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.626+0000 7f9c04196700 1 --2- 172.21.15.89:0/3798817385 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bfc0c91e0 0x7f9bfc160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:11.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.626+0000 7f9c04196700 1 -- 172.21.15.89:0/3798817385 >> 172.21.15.89:0/3798817385 conn(0x7f9bfc0cd670 msgr2=0x7f9bfc0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:11.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.626+0000 7f9c04196700 1 -- 172.21.15.89:0/3798817385 shutdown_connections 2023-10-10T20:35:11.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.626+0000 7f9c04196700 1 -- 172.21.15.89:0/3798817385 wait complete. 2023-10-10T20:35:11.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.627+0000 7f9c04196700 1 Processor -- start 2023-10-10T20:35:11.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.627+0000 7f9c04196700 1 -- start start 2023-10-10T20:35:11.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.628+0000 7f9c04196700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bfc06aa40 0x7f9bfc14e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:11.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.628+0000 7f9c04196700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc06bac0 0x7f9bfc0c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:11.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.628+0000 7f9c04196700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bfc0c91e0 0x7f9bfc0c3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:11.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.628+0000 7f9c04196700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0c7200 con 0x7f9bfc06bac0 2023-10-10T20:35:11.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.628+0000 7f9c04196700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0c65f0 con 0x7f9bfc06aa40 2023-10-10T20:35:11.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.628+0000 7f9c04196700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0c68d0 con 0x7f9bfc0c91e0 2023-10-10T20:35:11.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.628+0000 7f9c01731700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc06bac0 0x7f9bfc0c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:11.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.628+0000 7f9c01731700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc06bac0 0x7f9bfc0c5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42620/0 (socket says 172.21.15.89:42620) 2023-10-10T20:35:11.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.628+0000 7f9c01731700 1 -- 172.21.15.89:0/2427939217 learned_addr learned my addr 172.21.15.89:0/2427939217 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:11.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.628+0000 7f9c01f32700 1 --2- 172.21.15.89:0/2427939217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bfc06aa40 0x7f9bfc14e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:11.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.628+0000 7f9c02733700 1 --2- 172.21.15.89:0/2427939217 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bfc0c91e0 0x7f9bfc0c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:11.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.628+0000 7f9c01731700 1 -- 172.21.15.89:0/2427939217 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bfc0c91e0 msgr2=0x7f9bfc0c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:11.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.628+0000 7f9c01731700 1 --2- 172.21.15.89:0/2427939217 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bfc0c91e0 0x7f9bfc0c3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:11.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.628+0000 7f9c01731700 1 -- 172.21.15.89:0/2427939217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bfc06aa40 msgr2=0x7f9bfc14e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:11.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.628+0000 7f9c01731700 1 --2- 172.21.15.89:0/2427939217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bfc06aa40 0x7f9bfc14e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:11.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.628+0000 7f9c01731700 1 -- 172.21.15.89:0/2427939217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec009480 con 0x7f9bfc06bac0 2023-10-10T20:35:11.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.628+0000 7f9c01731700 1 --2- 172.21.15.89:0/2427939217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc06bac0 0x7f9bfc0c5d40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800c590 tx=0x7f9bf800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:11.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.628+0000 7f9bf1ffb700 1 -- 172.21.15.89:0/2427939217 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8011590 con 0x7f9bfc06bac0 2023-10-10T20:35:11.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.628+0000 7f9bf1ffb700 1 -- 172.21.15.89:0/2427939217 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf8011710 con 0x7f9bfc06bac0 2023-10-10T20:35:11.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.629+0000 7f9c04196700 1 -- 172.21.15.89:0/2427939217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bfc0c6bb0 con 0x7f9bfc06bac0 2023-10-10T20:35:11.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.629+0000 7f9bf1ffb700 1 -- 172.21.15.89:0/2427939217 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8011ad0 con 0x7f9bfc06bac0 2023-10-10T20:35:11.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.629+0000 7f9c04196700 1 -- 172.21.15.89:0/2427939217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bfc0c6280 con 0x7f9bfc06bac0 2023-10-10T20:35:11.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.630+0000 7f9c04196700 1 -- 172.21.15.89:0/2427939217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bfc0c7340 con 0x7f9bfc06bac0 2023-10-10T20:35:11.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.630+0000 7f9bf1ffb700 1 -- 172.21.15.89:0/2427939217 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9bf801b3b0 con 0x7f9bfc06bac0 2023-10-10T20:35:11.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.630+0000 7f9bf1ffb700 1 --2- 172.21.15.89:0/2427939217 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9be0066f30 0x7f9be00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:11.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.630+0000 7f9bf1ffb700 1 -- 172.21.15.89:0/2427939217 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9bf8083610 con 0x7f9bfc06bac0 2023-10-10T20:35:11.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.630+0000 7f9c01f32700 1 --2- 172.21.15.89:0/2427939217 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9be0066f30 0x7f9be00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:11.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.631+0000 7f9c01f32700 1 --2- 172.21.15.89:0/2427939217 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9be0066f30 0x7f9be00693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0ce550 tx=0x7f9bec016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:11.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.635+0000 7f9bf1ffb700 1 -- 172.21.15.89:0/2427939217 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bfc0c7340 con 0x7f9bfc06bac0 2023-10-10T20:35:11.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.897+0000 7f9c04196700 1 -- 172.21.15.89:0/2427939217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9bfc0c7340 con 0x7f9bfc06bac0 2023-10-10T20:35:11.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.897+0000 7f9bf1ffb700 1 -- 172.21.15.89:0/2427939217 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f9bfc0c7340 con 0x7f9bfc06bac0 2023-10-10T20:35:11.900 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:11.900 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:35:11.901 INFO:teuthology.orchestra.run.smithi089.stdout: "auth_dump": [ 2023-10-10T20:35:11.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.901 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.a", 2023-10-10T20:35:11.901 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlZLSNLhAA8WZTrsht6tAOP+gxaTXKMg==", 2023-10-10T20:35:11.901 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.902 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:35:11.902 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:35:11.902 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:35:11.902 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:11.902 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.903 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.b", 2023-10-10T20:35:11.903 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgyDEORAAMMoSxPFZc0bCIwWy8UsXgA==", 2023-10-10T20:35:11.903 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.904 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:35:11.904 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:35:11.904 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:35:11.904 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:11.904 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.905 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.c", 2023-10-10T20:35:11.905 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlMFxSNRAASuQEJFghLIOkTQWa4ckTvg==", 2023-10-10T20:35:11.905 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.905 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:35:11.906 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:35:11.906 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:35:11.906 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:11.906 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.907 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.d", 2023-10-10T20:35:11.907 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlPkvrBBAA2VLJmv2t8szTEGtCVwalJw==", 2023-10-10T20:35:11.907 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.907 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:35:11.908 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:35:11.908 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:35:11.908 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:11.908 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.909 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.0", 2023-10-10T20:35:11.909 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQArtSVltOcKHRAAk8+zUus04oTVI9KvfhjlXg==", 2023-10-10T20:35:11.909 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.909 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:35:11.910 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:35:11.910 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:11.910 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.910 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.911 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.1", 2023-10-10T20:35:11.911 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAutSVly8P0GhAAe1bvR63+ptJg84lizMNdQw==", 2023-10-10T20:35:11.911 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.911 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:35:11.911 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:35:11.912 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:11.912 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.912 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.2", 2023-10-10T20:35:11.912 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAxtSVl0x9fGRAAEKi7+6EW5mzPbebmz0+sgQ==", 2023-10-10T20:35:11.913 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.913 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:35:11.913 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:35:11.913 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:11.913 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.914 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.3", 2023-10-10T20:35:11.914 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA0tSVlfJyaGBAAa+8w54U6v7+npbdufKJruA==", 2023-10-10T20:35:11.914 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.915 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:35:11.915 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:35:11.915 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:11.915 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.916 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.4", 2023-10-10T20:35:11.916 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA6tSVlEQ/QFxAA/PRB3c90BuAlF6xekGfX9A==", 2023-10-10T20:35:11.916 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.916 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:35:11.917 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:35:11.917 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:11.917 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.917 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.918 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.5", 2023-10-10T20:35:11.918 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA9tSVlMOKrFRAAuBnIsEz2VmeSZi9XWzjlzA==", 2023-10-10T20:35:11.918 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.918 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:35:11.918 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:35:11.919 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:11.919 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.919 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.919 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.6", 2023-10-10T20:35:11.920 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBAtSVlH7PQFBAAHr6c7d4TYJEu/KSFEf+EOA==", 2023-10-10T20:35:11.920 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.920 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:35:11.920 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:35:11.920 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:11.920 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.921 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.7", 2023-10-10T20:35:11.921 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBDtSVlAhC1FBAAuE2BwAA/f7VOkZfVdqAG7A==", 2023-10-10T20:35:11.921 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.922 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:35:11.922 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:35:11.922 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:11.922 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.923 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.0", 2023-10-10T20:35:11.923 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlRd3HCBAAb0QP2VfTS7aelno1mFpGAA==", 2023-10-10T20:35:11.923 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.923 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:35:11.924 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:35:11.924 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:35:11.924 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:35:11.924 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.925 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.1", 2023-10-10T20:35:11.925 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlgBouERAAqPdE0KvJU3M53B6Y6AH5GQ==", 2023-10-10T20:35:11.925 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.925 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:35:11.925 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:35:11.926 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:35:11.926 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:35:11.926 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.926 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.927 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.2", 2023-10-10T20:35:11.927 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlXA4tDhAAoPVYQjECpzD+C67mnsY5IA==", 2023-10-10T20:35:11.927 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.927 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:35:11.927 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:35:11.928 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:35:11.928 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:35:11.928 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.928 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.928 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.928 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.3", 2023-10-10T20:35:11.929 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlZ5KGFhAAuKmZwbOfmmw1nAj5ry6tHw==", 2023-10-10T20:35:11.929 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.929 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:35:11.929 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:35:11.929 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:35:11.930 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:35:11.930 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.930 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.admin", 2023-10-10T20:35:11.931 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgpL2EhAAoDOki002rfBrauTgfu481Q==", 2023-10-10T20:35:11.931 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.931 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:35:11.931 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:35:11.931 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:35:11.932 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:11.932 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.932 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.932 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.932 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.bootstrap-mds", 2023-10-10T20:35:11.933 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBJtSVlEyD4BxAAPwRadWPE78DDY2nd6sny9Q==", 2023-10-10T20:35:11.933 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.933 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile bootstrap-mds" 2023-10-10T20:35:11.933 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.933 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.934 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.934 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.bootstrap-mgr", 2023-10-10T20:35:11.934 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBJtSVlhCj4BxAACOHi/lixGsifEh4bsgexQg==", 2023-10-10T20:35:11.934 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.934 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-10T20:35:11.935 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.935 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.bootstrap-osd", 2023-10-10T20:35:11.935 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBJtSVlmzP4BxAAJX1qCnO6sXpKDBB0exBaog==", 2023-10-10T20:35:11.935 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.936 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile bootstrap-osd" 2023-10-10T20:35:11.936 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.936 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.936 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.bootstrap-rbd", 2023-10-10T20:35:11.937 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBJtSVl/zr4BxAAyHbxgxhuMSaS7QrO4DiBiw==", 2023-10-10T20:35:11.937 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.937 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-10T20:35:11.937 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.938 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.938 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-10T20:35:11.938 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBJtSVlTkL4BxAAubpk+hLOZb8bmJrs0i791A==", 2023-10-10T20:35:11.938 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.938 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-10T20:35:11.939 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.939 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.bootstrap-rgw", 2023-10-10T20:35:11.939 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBJtSVlL0r4BxAATyVHXi89t2xvy0gXbRN0AQ==", 2023-10-10T20:35:11.940 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.940 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-10T20:35:11.940 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.940 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.940 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.x", 2023-10-10T20:35:11.941 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlBhaJKxAArraBxT5xWg515LfpJ7MlbA==", 2023-10-10T20:35:11.941 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.941 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:35:11.941 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:35:11.941 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:11.942 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:11.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:11.942 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.y", 2023-10-10T20:35:11.942 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlTZIyJxAAWAOsitHwsIxJWwp3b1Z98Q==", 2023-10-10T20:35:11.943 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:11.943 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:35:11.943 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:35:11.943 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:11.943 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.943 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:11.944 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-10T20:35:11.944 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:35:11.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.900+0000 7f9c04196700 1 -- 172.21.15.89:0/2427939217 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9be0066f30 msgr2=0x7f9be00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:11.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.900+0000 7f9c04196700 1 --2- 172.21.15.89:0/2427939217 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9be0066f30 0x7f9be00693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0ce550 tx=0x7f9bec016000 comp rx=0 tx=0).stop 2023-10-10T20:35:11.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.900+0000 7f9c04196700 1 -- 172.21.15.89:0/2427939217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc06bac0 msgr2=0x7f9bfc0c5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:11.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.900+0000 7f9c04196700 1 --2- 172.21.15.89:0/2427939217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc06bac0 0x7f9bfc0c5d40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800c590 tx=0x7f9bf800c8a0 comp rx=0 tx=0).stop 2023-10-10T20:35:11.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.900+0000 7f9c04196700 1 -- 172.21.15.89:0/2427939217 shutdown_connections 2023-10-10T20:35:11.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.900+0000 7f9c04196700 1 --2- 172.21.15.89:0/2427939217 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bfc0c91e0 0x7f9bfc0c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:11.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.900+0000 7f9c04196700 1 --2- 172.21.15.89:0/2427939217 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9be0066f30 0x7f9be00693e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:11.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.900+0000 7f9c04196700 1 --2- 172.21.15.89:0/2427939217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bfc06aa40 0x7f9bfc14e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:11.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.900+0000 7f9c04196700 1 --2- 172.21.15.89:0/2427939217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc06bac0 0x7f9bfc0c5d40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:11.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.900+0000 7f9c04196700 1 -- 172.21.15.89:0/2427939217 >> 172.21.15.89:0/2427939217 conn(0x7f9bfc0cd670 msgr2=0x7f9bfc0b2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:11.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.900+0000 7f9c04196700 1 -- 172.21.15.89:0/2427939217 shutdown_connections 2023-10-10T20:35:11.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:11.900+0000 7f9c04196700 1 -- 172.21.15.89:0/2427939217 wait complete. 2023-10-10T20:35:11.948 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-10T20:35:12.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.061+0000 7f9eef734700 1 Processor -- start 2023-10-10T20:35:12.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.061+0000 7f9eef734700 1 -- start start 2023-10-10T20:35:12.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.062+0000 7f9eef734700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ee8159890 0x7f9ee815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.062+0000 7f9eef734700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9ee8069560 0x7f9ee80699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.062+0000 7f9eef734700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee815a160 con 0x7f9ee8159c60 2023-10-10T20:35:12.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.062+0000 7f9eef734700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee8069f20 con 0x7f9ee8069560 2023-10-10T20:35:12.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.062+0000 7f9eef734700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee80d13a0 con 0x7f9ee8159890 2023-10-10T20:35:12.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.062+0000 7f9eedcd1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9ee8069560 0x7f9ee80699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:12.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.062+0000 7f9eecccf700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9ee8159c60 0x7f9ee815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40480/0 (socket says 172.21.15.89:40480) 2023-10-10T20:35:12.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.062+0000 7f9eecccf700 1 -- 172.21.15.89:0/440039819 learned_addr learned my addr 172.21.15.89:0/440039819 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:12.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.062+0000 7f9eed4d0700 1 --2- 172.21.15.89:0/440039819 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ee8159890 0x7f9ee815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:12.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.062+0000 7f9eedcd1700 1 -- 172.21.15.89:0/440039819 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ee8159890 msgr2=0x7f9ee815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:12.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.062+0000 7f9eedcd1700 1 --2- 172.21.15.89:0/440039819 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ee8159890 0x7f9ee815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.062+0000 7f9eedcd1700 1 -- 172.21.15.89:0/440039819 >> v1:172.21.15.89:6789/0 conn(0x7f9ee8159c60 legacy=0x7f9ee815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:12.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.062+0000 7f9edffff700 1 -- 172.21.15.89:0/440039819 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843498658 0 0) 0x7f9ee815a160 con 0x7f9ee8159c60 2023-10-10T20:35:12.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.062+0000 7f9eedcd1700 1 -- 172.21.15.89:0/440039819 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee815e140 con 0x7f9ee8069560 2023-10-10T20:35:12.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.062+0000 7f9eed4d0700 1 --2- 172.21.15.89:0/440039819 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ee8159890 0x7f9ee815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:12.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.062+0000 7f9eedcd1700 1 --2- 172.21.15.89:0/440039819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ee8069560 0x7f9ee80699e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009e60 tx=0x7f9ee0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb0bb92bb13d72b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:12.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.062+0000 7f9edffff700 1 -- 172.21.15.89:0/440039819 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee000e070 con 0x7f9ee8069560 2023-10-10T20:35:12.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.062+0000 7f9edffff700 1 -- 172.21.15.89:0/440039819 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee000bc30 con 0x7f9ee8069560 2023-10-10T20:35:12.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.062+0000 7f9edffff700 1 -- 172.21.15.89:0/440039819 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0012430 con 0x7f9ee8069560 2023-10-10T20:35:12.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.063+0000 7f9eef734700 1 -- 172.21.15.89:0/440039819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ee8069560 msgr2=0x7f9ee80699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:12.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.063+0000 7f9eef734700 1 --2- 172.21.15.89:0/440039819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ee8069560 0x7f9ee80699e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009e60 tx=0x7f9ee0009780 comp rx=0 tx=0).stop 2023-10-10T20:35:12.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.063+0000 7f9eef734700 1 -- 172.21.15.89:0/440039819 shutdown_connections 2023-10-10T20:35:12.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.063+0000 7f9eef734700 1 --2- 172.21.15.89:0/440039819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ee8069560 0x7f9ee80699e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.063+0000 7f9eef734700 1 --2- 172.21.15.89:0/440039819 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ee8159890 0x7f9ee815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.063+0000 7f9eef734700 1 -- 172.21.15.89:0/440039819 >> 172.21.15.89:0/440039819 conn(0x7f9ee8068e20 msgr2=0x7f9ee80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:12.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.063+0000 7f9eef734700 1 -- 172.21.15.89:0/440039819 shutdown_connections 2023-10-10T20:35:12.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.063+0000 7f9eef734700 1 -- 172.21.15.89:0/440039819 wait complete. 2023-10-10T20:35:12.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.064+0000 7f9eef734700 1 Processor -- start 2023-10-10T20:35:12.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.064+0000 7f9eef734700 1 -- start start 2023-10-10T20:35:12.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.065+0000 7f9eef734700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ee8069560 0x7f9ee8140fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.065+0000 7f9eef734700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ee8159890 0x7f9ee80b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.065+0000 7f9eef734700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ee8159c60 0x7f9ee80b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.065+0000 7f9eef734700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee8155400 con 0x7f9ee8069560 2023-10-10T20:35:12.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.065+0000 7f9eef734700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee815c1d0 con 0x7f9ee8159890 2023-10-10T20:35:12.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.065+0000 7f9eef734700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee80b43e0 con 0x7f9ee8159c60 2023-10-10T20:35:12.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.065+0000 7f9eed4d0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ee8069560 0x7f9ee8140fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:12.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.065+0000 7f9eed4d0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ee8069560 0x7f9ee8140fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42638/0 (socket says 172.21.15.89:42638) 2023-10-10T20:35:12.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.065+0000 7f9eed4d0700 1 -- 172.21.15.89:0/2989959957 learned_addr learned my addr 172.21.15.89:0/2989959957 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:12.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.065+0000 7f9eecccf700 1 --2- 172.21.15.89:0/2989959957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ee8159890 0x7f9ee80b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:12.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.065+0000 7f9eedcd1700 1 --2- 172.21.15.89:0/2989959957 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ee8159c60 0x7f9ee80b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:12.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.065+0000 7f9eed4d0700 1 -- 172.21.15.89:0/2989959957 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ee8159c60 msgr2=0x7f9ee80b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:12.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.065+0000 7f9eed4d0700 1 --2- 172.21.15.89:0/2989959957 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ee8159c60 0x7f9ee80b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.065+0000 7f9eed4d0700 1 -- 172.21.15.89:0/2989959957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ee8159890 msgr2=0x7f9ee80b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:12.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.065+0000 7f9eed4d0700 1 --2- 172.21.15.89:0/2989959957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ee8159890 0x7f9ee80b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.065+0000 7f9eed4d0700 1 -- 172.21.15.89:0/2989959957 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee0009a20 con 0x7f9ee8069560 2023-10-10T20:35:12.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.065+0000 7f9eed4d0700 1 --2- 172.21.15.89:0/2989959957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ee8069560 0x7f9ee8140fe0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9ee400e900 tx=0x7f9ee4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:12.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.066+0000 7f9edd7fa700 1 -- 172.21.15.89:0/2989959957 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee400eea0 con 0x7f9ee8069560 2023-10-10T20:35:12.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.066+0000 7f9edd7fa700 1 -- 172.21.15.89:0/2989959957 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee400c5e0 con 0x7f9ee8069560 2023-10-10T20:35:12.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.066+0000 7f9eef734700 1 -- 172.21.15.89:0/2989959957 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee80b4100 con 0x7f9ee8069560 2023-10-10T20:35:12.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.066+0000 7f9edd7fa700 1 -- 172.21.15.89:0/2989959957 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee400c980 con 0x7f9ee8069560 2023-10-10T20:35:12.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.066+0000 7f9eef734700 1 -- 172.21.15.89:0/2989959957 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee815be60 con 0x7f9ee8069560 2023-10-10T20:35:12.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.067+0000 7f9eef734700 1 -- 172.21.15.89:0/2989959957 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee815b9e0 con 0x7f9ee8069560 2023-10-10T20:35:12.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.067+0000 7f9edd7fa700 1 -- 172.21.15.89:0/2989959957 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ee400cd30 con 0x7f9ee8069560 2023-10-10T20:35:12.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.067+0000 7f9edd7fa700 1 --2- 172.21.15.89:0/2989959957 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ecc066f60 0x7f9ecc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.067+0000 7f9edd7fa700 1 -- 172.21.15.89:0/2989959957 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9ee4083fa0 con 0x7f9ee8069560 2023-10-10T20:35:12.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.067+0000 7f9eecccf700 1 --2- 172.21.15.89:0/2989959957 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ecc066f60 0x7f9ecc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:12.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.068+0000 7f9eecccf700 1 --2- 172.21.15.89:0/2989959957 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ecc066f60 0x7f9ecc069410 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8155080 tx=0x7f9ed8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:12.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.072+0000 7f9edd7fa700 1 -- 172.21.15.89:0/2989959957 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ee815b9e0 con 0x7f9ee8069560 2023-10-10T20:35:12.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.332+0000 7f9eef734700 1 -- 172.21.15.89:0/2989959957 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f9ee815b9e0 con 0x7f9ee8069560 2023-10-10T20:35:12.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.336+0000 7f9edd7fa700 1 -- 172.21.15.89:0/2989959957 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9ee815b9e0 con 0x7f9ee8069560 2023-10-10T20:35:12.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.338+0000 7f9eef734700 1 -- 172.21.15.89:0/2989959957 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ecc066f60 msgr2=0x7f9ecc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:12.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.338+0000 7f9eef734700 1 --2- 172.21.15.89:0/2989959957 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ecc066f60 0x7f9ecc069410 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8155080 tx=0x7f9ed8009040 comp rx=0 tx=0).stop 2023-10-10T20:35:12.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.338+0000 7f9eef734700 1 -- 172.21.15.89:0/2989959957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ee8069560 msgr2=0x7f9ee8140fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:12.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.338+0000 7f9eef734700 1 --2- 172.21.15.89:0/2989959957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ee8069560 0x7f9ee8140fe0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9ee400e900 tx=0x7f9ee4008040 comp rx=0 tx=0).stop 2023-10-10T20:35:12.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.338+0000 7f9eef734700 1 -- 172.21.15.89:0/2989959957 shutdown_connections 2023-10-10T20:35:12.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.338+0000 7f9eef734700 1 --2- 172.21.15.89:0/2989959957 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ee8159c60 0x7f9ee80b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.338+0000 7f9eef734700 1 --2- 172.21.15.89:0/2989959957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ee8069560 0x7f9ee8140fe0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.338+0000 7f9eef734700 1 --2- 172.21.15.89:0/2989959957 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ecc066f60 0x7f9ecc069410 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.338+0000 7f9eef734700 1 --2- 172.21.15.89:0/2989959957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ee8159890 0x7f9ee80b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.338+0000 7f9eef734700 1 -- 172.21.15.89:0/2989959957 >> 172.21.15.89:0/2989959957 conn(0x7f9ee8068e20 msgr2=0x7f9ee8156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:12.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.338+0000 7f9eef734700 1 -- 172.21.15.89:0/2989959957 shutdown_connections 2023-10-10T20:35:12.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.338+0000 7f9eef734700 1 -- 172.21.15.89:0/2989959957 wait complete. 2023-10-10T20:35:12.352 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-10T20:35:12.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.464+0000 7f3703408700 1 Processor -- start 2023-10-10T20:35:12.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.464+0000 7f3703408700 1 -- start start 2023-10-10T20:35:12.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.465+0000 7f3703408700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f36fc159890 0x7f36fc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.465+0000 7f3703408700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f36fc069560 0x7f36fc0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.465+0000 7f3703408700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36fc069ee0 con 0x7f36fc159c60 2023-10-10T20:35:12.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.465+0000 7f3703408700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36fc15a160 con 0x7f36fc069560 2023-10-10T20:35:12.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.465+0000 7f3703408700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36fc0d13a0 con 0x7f36fc159890 2023-10-10T20:35:12.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.465+0000 7f37019a5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f36fc069560 0x7f36fc0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:12.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.465+0000 7f37019a5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f36fc069560 0x7f36fc0699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42654/0 (socket says 172.21.15.89:42654) 2023-10-10T20:35:12.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.465+0000 7f37019a5700 1 -- 172.21.15.89:0/939508166 learned_addr learned my addr 172.21.15.89:0/939508166 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:12.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.465+0000 7f37009a3700 1 --1- 172.21.15.89:0/939508166 >> v1:172.21.15.89:6789/0 conn(0x7f36fc159c60 0x7f36fc15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40486/0 (socket says 172.21.15.89:40486) 2023-10-10T20:35:12.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.465+0000 7f37011a4700 1 --2- 172.21.15.89:0/939508166 >> v2:172.21.15.111:3300/0 conn(0x7f36fc159890 0x7f36fc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:12.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.466+0000 7f37019a5700 1 -- 172.21.15.89:0/939508166 >> v2:172.21.15.111:3300/0 conn(0x7f36fc159890 msgr2=0x7f36fc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:12.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.466+0000 7f37019a5700 1 --2- 172.21.15.89:0/939508166 >> v2:172.21.15.111:3300/0 conn(0x7f36fc159890 0x7f36fc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.466+0000 7f37019a5700 1 -- 172.21.15.89:0/939508166 >> v1:172.21.15.89:6789/0 conn(0x7f36fc159c60 legacy=0x7f36fc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:12.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.466+0000 7f37019a5700 1 -- 172.21.15.89:0/939508166 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc15e140 con 0x7f36fc069560 2023-10-10T20:35:12.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.466+0000 7f36f3fff700 1 -- 172.21.15.89:0/939508166 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3614207826 0 0) 0x7f36fc069ee0 con 0x7f36fc159c60 2023-10-10T20:35:12.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.466+0000 7f37011a4700 1 --2- 172.21.15.89:0/939508166 >> v2:172.21.15.111:3300/0 conn(0x7f36fc159890 0x7f36fc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:12.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.466+0000 7f37019a5700 1 --2- 172.21.15.89:0/939508166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36fc069560 0x7f36fc0699a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0d1090 tx=0x7f36f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5066ad51cc3e44 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:12.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.466+0000 7f36f3fff700 1 -- 172.21.15.89:0/939508166 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f400b860 con 0x7f36fc069560 2023-10-10T20:35:12.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.466+0000 7f36f3fff700 1 -- 172.21.15.89:0/939508166 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f400b9c0 con 0x7f36fc069560 2023-10-10T20:35:12.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.466+0000 7f36f3fff700 1 -- 172.21.15.89:0/939508166 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f400bd60 con 0x7f36fc069560 2023-10-10T20:35:12.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.466+0000 7f3703408700 1 -- 172.21.15.89:0/939508166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36fc069560 msgr2=0x7f36fc0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:12.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.466+0000 7f3703408700 1 --2- 172.21.15.89:0/939508166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36fc069560 0x7f36fc0699a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0d1090 tx=0x7f36f4009250 comp rx=0 tx=0).stop 2023-10-10T20:35:12.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.466+0000 7f3703408700 1 -- 172.21.15.89:0/939508166 shutdown_connections 2023-10-10T20:35:12.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.466+0000 7f3703408700 1 --2- 172.21.15.89:0/939508166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36fc069560 0x7f36fc0699a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.466+0000 7f3703408700 1 --2- 172.21.15.89:0/939508166 >> v2:172.21.15.111:3300/0 conn(0x7f36fc159890 0x7f36fc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.466+0000 7f3703408700 1 -- 172.21.15.89:0/939508166 >> 172.21.15.89:0/939508166 conn(0x7f36fc068e20 msgr2=0x7f36fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:12.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.466+0000 7f3703408700 1 -- 172.21.15.89:0/939508166 shutdown_connections 2023-10-10T20:35:12.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.466+0000 7f3703408700 1 -- 172.21.15.89:0/939508166 wait complete. 2023-10-10T20:35:12.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.468+0000 7f3703408700 1 Processor -- start 2023-10-10T20:35:12.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.468+0000 7f3703408700 1 -- start start 2023-10-10T20:35:12.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.468+0000 7f3703408700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36fc069560 0x7f36fc15dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.468+0000 7f3703408700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36fc159890 0x7f36fc0b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.468+0000 7f3703408700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f36fc159c60 0x7f36fc1542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.468+0000 7f3703408700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc0ce480 con 0x7f36fc069560 2023-10-10T20:35:12.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.468+0000 7f3703408700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc169930 con 0x7f36fc159890 2023-10-10T20:35:12.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.468+0000 7f3703408700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc15b8e0 con 0x7f36fc159c60 2023-10-10T20:35:12.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.469+0000 7f37011a4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36fc069560 0x7f36fc15dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:12.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.469+0000 7f37011a4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36fc069560 0x7f36fc15dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42668/0 (socket says 172.21.15.89:42668) 2023-10-10T20:35:12.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.469+0000 7f37011a4700 1 -- 172.21.15.89:0/2022205532 learned_addr learned my addr 172.21.15.89:0/2022205532 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:12.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.469+0000 7f37011a4700 1 -- 172.21.15.89:0/2022205532 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f36fc159c60 msgr2=0x7f36fc1542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:12.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.469+0000 7f37009a3700 1 --2- 172.21.15.89:0/2022205532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36fc159890 0x7f36fc0b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:12.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.469+0000 7f37011a4700 1 --2- 172.21.15.89:0/2022205532 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f36fc159c60 0x7f36fc1542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.469+0000 7f37011a4700 1 -- 172.21.15.89:0/2022205532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36fc159890 msgr2=0x7f36fc0b5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:12.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.469+0000 7f37011a4700 1 --2- 172.21.15.89:0/2022205532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36fc159890 0x7f36fc0b5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.469+0000 7f37011a4700 1 -- 172.21.15.89:0/2022205532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f40094f0 con 0x7f36fc069560 2023-10-10T20:35:12.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.469+0000 7f37011a4700 1 --2- 172.21.15.89:0/2022205532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36fc069560 0x7f36fc15dac0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f36f800d200 tx=0x7f36f8009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:12.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.469+0000 7f36f17fa700 1 -- 172.21.15.89:0/2022205532 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f800de40 con 0x7f36fc069560 2023-10-10T20:35:12.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.469+0000 7f36f17fa700 1 -- 172.21.15.89:0/2022205532 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f800f040 con 0x7f36fc069560 2023-10-10T20:35:12.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.469+0000 7f3703408700 1 -- 172.21.15.89:0/2022205532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36fc0b5a70 con 0x7f36fc069560 2023-10-10T20:35:12.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.469+0000 7f36f17fa700 1 -- 172.21.15.89:0/2022205532 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8015670 con 0x7f36fc069560 2023-10-10T20:35:12.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.469+0000 7f3703408700 1 -- 172.21.15.89:0/2022205532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36fc15b570 con 0x7f36fc069560 2023-10-10T20:35:12.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.470+0000 7f3703408700 1 -- 172.21.15.89:0/2022205532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36fc15bbc0 con 0x7f36fc069560 2023-10-10T20:35:12.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.470+0000 7f36f17fa700 1 -- 172.21.15.89:0/2022205532 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f36f8015a60 con 0x7f36fc069560 2023-10-10T20:35:12.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.471+0000 7f36f17fa700 1 --2- 172.21.15.89:0/2022205532 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f36e0066f30 0x7f36e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.471+0000 7f36f17fa700 1 -- 172.21.15.89:0/2022205532 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f36f8084630 con 0x7f36fc069560 2023-10-10T20:35:12.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.471+0000 7f37009a3700 1 --2- 172.21.15.89:0/2022205532 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f36e0066f30 0x7f36e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:12.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.471+0000 7f37009a3700 1 --2- 172.21.15.89:0/2022205532 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f36e0066f30 0x7f36e00693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0c69b0 tx=0x7f36ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:12.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.475+0000 7f36f17fa700 1 -- 172.21.15.89:0/2022205532 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36f804e5a0 con 0x7f36fc069560 2023-10-10T20:35:12.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.733+0000 7f3703408700 1 -- 172.21.15.89:0/2022205532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f36fc04c3c0 con 0x7f36fc069560 2023-10-10T20:35:12.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.737+0000 7f36f17fa700 1 -- 172.21.15.89:0/2022205532 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f36f8051ca0 con 0x7f36fc069560 2023-10-10T20:35:12.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.739+0000 7f3703408700 1 -- 172.21.15.89:0/2022205532 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f36e0066f30 msgr2=0x7f36e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:12.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.739+0000 7f3703408700 1 --2- 172.21.15.89:0/2022205532 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f36e0066f30 0x7f36e00693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0c69b0 tx=0x7f36ec009040 comp rx=0 tx=0).stop 2023-10-10T20:35:12.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.739+0000 7f3703408700 1 -- 172.21.15.89:0/2022205532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36fc069560 msgr2=0x7f36fc15dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:12.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.739+0000 7f3703408700 1 --2- 172.21.15.89:0/2022205532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36fc069560 0x7f36fc15dac0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f36f800d200 tx=0x7f36f8009a90 comp rx=0 tx=0).stop 2023-10-10T20:35:12.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.739+0000 7f3703408700 1 -- 172.21.15.89:0/2022205532 shutdown_connections 2023-10-10T20:35:12.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.739+0000 7f3703408700 1 --2- 172.21.15.89:0/2022205532 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f36fc159c60 0x7f36fc1542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.739+0000 7f3703408700 1 --2- 172.21.15.89:0/2022205532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36fc069560 0x7f36fc15dac0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.739+0000 7f3703408700 1 --2- 172.21.15.89:0/2022205532 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f36e0066f30 0x7f36e00693e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.739+0000 7f3703408700 1 --2- 172.21.15.89:0/2022205532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36fc159890 0x7f36fc0b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.739+0000 7f3703408700 1 -- 172.21.15.89:0/2022205532 >> 172.21.15.89:0/2022205532 conn(0x7f36fc068e20 msgr2=0x7f36fc157550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:12.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.739+0000 7f3703408700 1 -- 172.21.15.89:0/2022205532 shutdown_connections 2023-10-10T20:35:12.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.739+0000 7f3703408700 1 -- 172.21.15.89:0/2022205532 wait complete. 2023-10-10T20:35:12.753 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-10T20:35:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.866+0000 7f0752f37700 1 Processor -- start 2023-10-10T20:35:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.866+0000 7f0752f37700 1 -- start start 2023-10-10T20:35:12.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.867+0000 7f0752f37700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f074c1597f0 0x7f074c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.867+0000 7f0752f37700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f074c0b11d0 con 0x7f074c159420 2023-10-10T20:35:12.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.867+0000 7f0752f37700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f074c0b14b0 con 0x7f074c15d230 2023-10-10T20:35:12.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.867+0000 7f0752f37700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f074c0b1790 con 0x7f074c1597f0 2023-10-10T20:35:12.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.867+0000 7f0751f35700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f074c159420 0x7f074c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40498/0 (socket says 172.21.15.89:40498) 2023-10-10T20:35:12.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.867+0000 7f0751f35700 1 -- 172.21.15.89:0/787272133 learned_addr learned my addr 172.21.15.89:0/787272133 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:12.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.867+0000 7f0751734700 1 --2- 172.21.15.89:0/787272133 >> v2:172.21.15.111:3300/0 conn(0x7f074c1597f0 0x7f074c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:12.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.867+0000 7f0750f33700 1 -- 172.21.15.89:0/787272133 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514608673 0 0) 0x7f074c0b11d0 con 0x7f074c159420 2023-10-10T20:35:12.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.867+0000 7f0750f33700 1 -- 172.21.15.89:0/787272133 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0738003570 con 0x7f074c159420 2023-10-10T20:35:12.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.867+0000 7f0750f33700 1 -- 172.21.15.89:0/787272133 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 460117359 0 0) 0x7f0738003570 con 0x7f074c159420 2023-10-10T20:35:12.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.867+0000 7f0750f33700 1 -- 172.21.15.89:0/787272133 >> v2:172.21.15.111:3300/0 conn(0x7f074c1597f0 msgr2=0x7f074c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:12.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.867+0000 7f0750f33700 1 --2- 172.21.15.89:0/787272133 >> v2:172.21.15.111:3300/0 conn(0x7f074c1597f0 0x7f074c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.867+0000 7f0750f33700 1 -- 172.21.15.89:0/787272133 >> v1:172.21.15.111:6789/0 conn(0x7f074c15d230 legacy=0x7f074c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:12.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.867+0000 7f0750f33700 1 -- 172.21.15.89:0/787272133 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074c0b1a70 con 0x7f074c159420 2023-10-10T20:35:12.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.867+0000 7f0750f33700 1 -- 172.21.15.89:0/787272133 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f07440017f0 con 0x7f074c159420 2023-10-10T20:35:12.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0750f33700 1 -- 172.21.15.89:0/787272133 >> v1:172.21.15.89:6789/0 conn(0x7f074c159420 legacy=0x7f074c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:12.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0750f33700 1 --2- 172.21.15.89:0/787272133 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0738003a60 0x7f0738003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0750f33700 1 --2- 172.21.15.89:0/787272133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0738004810 0x7f0738004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0750f33700 1 --2- 172.21.15.89:0/787272133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07380051c0 0x7f073800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0750f33700 1 -- 172.21.15.89:0/787272133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c0b1790 con 0x7f07380051c0 2023-10-10T20:35:12.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0750f33700 1 -- 172.21.15.89:0/787272133 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073800c0d0 con 0x7f0738004810 2023-10-10T20:35:12.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0750f33700 1 -- 172.21.15.89:0/787272133 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073800c290 con 0x7f0738003a60 2023-10-10T20:35:12.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0752736700 1 --2- 172.21.15.89:0/787272133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07380051c0 0x7f073800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:12.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0751734700 1 --2- 172.21.15.89:0/787272133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0738004810 0x7f0738004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:12.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0751f35700 1 --2- 172.21.15.89:0/787272133 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0738003a60 0x7f0738003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:12.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0752736700 1 -- 172.21.15.89:0/787272133 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0738003a60 msgr2=0x7f0738003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:12.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0752736700 1 --2- 172.21.15.89:0/787272133 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0738003a60 0x7f0738003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0752736700 1 -- 172.21.15.89:0/787272133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0738004810 msgr2=0x7f0738004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:12.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0752736700 1 --2- 172.21.15.89:0/787272133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0738004810 0x7f0738004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0752736700 1 -- 172.21.15.89:0/787272133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0738003570 con 0x7f07380051c0 2023-10-10T20:35:12.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0751f35700 1 --2- 172.21.15.89:0/787272133 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0738003a60 0x7f0738003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:12.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0752736700 1 -- 172.21.15.89:0/787272133 reap_dead start 2023-10-10T20:35:12.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0752736700 1 --2- 172.21.15.89:0/787272133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07380051c0 0x7f073800ba60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0748001880 tx=0x7f0748007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e8ed01fea04271 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:12.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0750f33700 1 -- 172.21.15.89:0/787272133 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748012420 con 0x7f07380051c0 2023-10-10T20:35:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0750f33700 1 -- 172.21.15.89:0/787272133 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0748012580 con 0x7f07380051c0 2023-10-10T20:35:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.868+0000 7f0750f33700 1 -- 172.21.15.89:0/787272133 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748012920 con 0x7f07380051c0 2023-10-10T20:35:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.869+0000 7f0752f37700 1 -- 172.21.15.89:0/787272133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07380051c0 msgr2=0x7f073800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.869+0000 7f0752f37700 1 --2- 172.21.15.89:0/787272133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07380051c0 0x7f073800ba60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0748001880 tx=0x7f0748007d30 comp rx=0 tx=0).stop 2023-10-10T20:35:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.869+0000 7f0752f37700 1 -- 172.21.15.89:0/787272133 shutdown_connections 2023-10-10T20:35:12.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.869+0000 7f0752f37700 1 --2- 172.21.15.89:0/787272133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07380051c0 0x7f073800ba60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.869+0000 7f0752f37700 1 -- 172.21.15.89:0/787272133 >> 172.21.15.89:0/787272133 conn(0x7f074c0d0660 msgr2=0x7f074c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:12.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.869+0000 7f0752f37700 1 -- 172.21.15.89:0/787272133 shutdown_connections 2023-10-10T20:35:12.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.869+0000 7f0752f37700 1 -- 172.21.15.89:0/787272133 wait complete. 2023-10-10T20:35:12.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.870+0000 7f0752f37700 1 Processor -- start 2023-10-10T20:35:12.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.870+0000 7f0752f37700 1 -- start start 2023-10-10T20:35:12.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0752f37700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07380051c0 0x7f074c153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0752f37700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f074c1538b0 0x7f074c155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0752f37700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f074c16a220 0x7f074c156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0752f37700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c16cd80 con 0x7f07380051c0 2023-10-10T20:35:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0752f37700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c16d8c0 con 0x7f074c1538b0 2023-10-10T20:35:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0752f37700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074c1540d0 con 0x7f074c16a220 2023-10-10T20:35:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0751f35700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07380051c0 0x7f074c153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:12.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0751f35700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07380051c0 0x7f074c153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42684/0 (socket says 172.21.15.89:42684) 2023-10-10T20:35:12.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0751f35700 1 -- 172.21.15.89:0/3052430671 learned_addr learned my addr 172.21.15.89:0/3052430671 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:12.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0751734700 1 --2- 172.21.15.89:0/3052430671 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f074c1538b0 0x7f074c155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:12.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0752736700 1 --2- 172.21.15.89:0/3052430671 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f074c16a220 0x7f074c156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:12.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0751f35700 1 -- 172.21.15.89:0/3052430671 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f074c16a220 msgr2=0x7f074c156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:12.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0751f35700 1 --2- 172.21.15.89:0/3052430671 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f074c16a220 0x7f074c156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0751f35700 1 -- 172.21.15.89:0/3052430671 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f074c1538b0 msgr2=0x7f074c155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:12.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0751f35700 1 --2- 172.21.15.89:0/3052430671 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f074c1538b0 0x7f074c155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:12.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0751f35700 1 -- 172.21.15.89:0/3052430671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074c153690 con 0x7f07380051c0 2023-10-10T20:35:12.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0752736700 1 --2- 172.21.15.89:0/3052430671 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f074c16a220 0x7f074c156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:12.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0751f35700 1 --2- 172.21.15.89:0/3052430671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07380051c0 0x7f074c153150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0744007b40 tx=0x7f07440010f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:12.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0741ffb700 1 -- 172.21.15.89:0/3052430671 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074400eb60 con 0x7f07380051c0 2023-10-10T20:35:12.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0741ffb700 1 -- 172.21.15.89:0/3052430671 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0744010040 con 0x7f07380051c0 2023-10-10T20:35:12.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0741ffb700 1 -- 172.21.15.89:0/3052430671 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07440143f0 con 0x7f07380051c0 2023-10-10T20:35:12.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.871+0000 7f0752f37700 1 -- 172.21.15.89:0/3052430671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074c16cf90 con 0x7f07380051c0 2023-10-10T20:35:12.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.872+0000 7f0752f37700 1 -- 172.21.15.89:0/3052430671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074c16d550 con 0x7f07380051c0 2023-10-10T20:35:12.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.872+0000 7f0752f37700 1 -- 172.21.15.89:0/3052430671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f074c16dba0 con 0x7f07380051c0 2023-10-10T20:35:12.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.873+0000 7f0741ffb700 1 -- 172.21.15.89:0/3052430671 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0744014760 con 0x7f07380051c0 2023-10-10T20:35:12.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.873+0000 7f0741ffb700 1 --2- 172.21.15.89:0/3052430671 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0730066f30 0x7f07300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:12.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.873+0000 7f0741ffb700 1 -- 172.21.15.89:0/3052430671 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f07440842f0 con 0x7f07380051c0 2023-10-10T20:35:12.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.873+0000 7f0751734700 1 --2- 172.21.15.89:0/3052430671 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0730066f30 0x7f07300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:12.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.873+0000 7f0751734700 1 --2- 172.21.15.89:0/3052430671 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0730066f30 0x7f07300693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f074c0687e0 tx=0x7f073c0091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:12.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:12.877+0000 7f0741ffb700 1 -- 172.21.15.89:0/3052430671 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f07440518b0 con 0x7f07380051c0 2023-10-10T20:35:13.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.133+0000 7f0752f37700 1 -- 172.21.15.89:0/3052430671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f074c04c3c0 con 0x7f07380051c0 2023-10-10T20:35:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.137+0000 7f0741ffb700 1 -- 172.21.15.89:0/3052430671 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0744054ed0 con 0x7f07380051c0 2023-10-10T20:35:13.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.139+0000 7f0752f37700 1 -- 172.21.15.89:0/3052430671 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0730066f30 msgr2=0x7f07300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:13.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.139+0000 7f0752f37700 1 --2- 172.21.15.89:0/3052430671 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0730066f30 0x7f07300693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f074c0687e0 tx=0x7f073c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.139+0000 7f0752f37700 1 -- 172.21.15.89:0/3052430671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07380051c0 msgr2=0x7f074c153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:13.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.139+0000 7f0752f37700 1 --2- 172.21.15.89:0/3052430671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07380051c0 0x7f074c153150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0744007b40 tx=0x7f07440010f0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.139+0000 7f0752f37700 1 -- 172.21.15.89:0/3052430671 shutdown_connections 2023-10-10T20:35:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.139+0000 7f0752f37700 1 --2- 172.21.15.89:0/3052430671 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f074c16a220 0x7f074c156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.139+0000 7f0752f37700 1 --2- 172.21.15.89:0/3052430671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07380051c0 0x7f074c153150 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.139+0000 7f0752f37700 1 --2- 172.21.15.89:0/3052430671 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0730066f30 0x7f07300693e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.139+0000 7f0752f37700 1 --2- 172.21.15.89:0/3052430671 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f074c1538b0 0x7f074c155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.139+0000 7f0752f37700 1 -- 172.21.15.89:0/3052430671 >> 172.21.15.89:0/3052430671 conn(0x7f074c0d0660 msgr2=0x7f074c0d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:13.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.139+0000 7f0752f37700 1 -- 172.21.15.89:0/3052430671 shutdown_connections 2023-10-10T20:35:13.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.139+0000 7f0752f37700 1 -- 172.21.15.89:0/3052430671 wait complete. 2023-10-10T20:35:13.153 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-10T20:35:13.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.266+0000 7f9981531700 1 Processor -- start 2023-10-10T20:35:13.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.266+0000 7f9981531700 1 -- start start 2023-10-10T20:35:13.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.267+0000 7f9981531700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f997c1597f0 0x7f997c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:13.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.267+0000 7f9981531700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f997c15d230 0x7f997c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:13.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.267+0000 7f9981531700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997c15dbb0 con 0x7f997c159420 2023-10-10T20:35:13.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.267+0000 7f9981531700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997c0b11d0 con 0x7f997c1597f0 2023-10-10T20:35:13.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.267+0000 7f9981531700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997c0b14b0 con 0x7f997c15d230 2023-10-10T20:35:13.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.267+0000 7f997b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f997c1597f0 0x7f997c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:13.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.267+0000 7f997b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f997c1597f0 0x7f997c0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42696/0 (socket says 172.21.15.89:42696) 2023-10-10T20:35:13.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.267+0000 7f997b7fe700 1 -- 172.21.15.89:0/291495916 learned_addr learned my addr 172.21.15.89:0/291495916 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:13.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.267+0000 7f9980d30700 1 --2- 172.21.15.89:0/291495916 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f997c15d230 0x7f997c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:13.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.268+0000 7f997b7fe700 1 -- 172.21.15.89:0/291495916 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f997c15d230 msgr2=0x7f997c15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:13.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.268+0000 7f997b7fe700 1 --2- 172.21.15.89:0/291495916 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f997c15d230 0x7f997c15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.268+0000 7f997b7fe700 1 -- 172.21.15.89:0/291495916 >> v1:172.21.15.111:6789/0 conn(0x7f997c159420 legacy=0x7f997c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:13.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.268+0000 7f997b7fe700 1 -- 172.21.15.89:0/291495916 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c0b1790 con 0x7f997c1597f0 2023-10-10T20:35:13.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.268+0000 7f997b7fe700 1 --2- 172.21.15.89:0/291495916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f997c1597f0 0x7f997c0b2270 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f99700099b0 tx=0x7f9970009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98f3bb64ef81fb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:13.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.268+0000 7f997affd700 1 -- 172.21.15.89:0/291495916 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997000b860 con 0x7f997c1597f0 2023-10-10T20:35:13.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.268+0000 7f997affd700 1 -- 172.21.15.89:0/291495916 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997000b9c0 con 0x7f997c1597f0 2023-10-10T20:35:13.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.268+0000 7f997affd700 1 -- 172.21.15.89:0/291495916 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997000bd60 con 0x7f997c1597f0 2023-10-10T20:35:13.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.268+0000 7f9981531700 1 -- 172.21.15.89:0/291495916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f997c1597f0 msgr2=0x7f997c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:13.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.268+0000 7f9981531700 1 --2- 172.21.15.89:0/291495916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f997c1597f0 0x7f997c0b2270 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f99700099b0 tx=0x7f9970009250 comp rx=0 tx=0).stop 2023-10-10T20:35:13.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.268+0000 7f9981531700 1 -- 172.21.15.89:0/291495916 shutdown_connections 2023-10-10T20:35:13.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.268+0000 7f9981531700 1 --2- 172.21.15.89:0/291495916 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f997c15d230 0x7f997c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.268+0000 7f9981531700 1 --2- 172.21.15.89:0/291495916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f997c1597f0 0x7f997c0b2270 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.268+0000 7f9981531700 1 -- 172.21.15.89:0/291495916 >> 172.21.15.89:0/291495916 conn(0x7f997c0d0660 msgr2=0x7f997c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:13.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.268+0000 7f9981531700 1 -- 172.21.15.89:0/291495916 shutdown_connections 2023-10-10T20:35:13.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.268+0000 7f9981531700 1 -- 172.21.15.89:0/291495916 wait complete. 2023-10-10T20:35:13.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.269+0000 7f9981531700 1 Processor -- start 2023-10-10T20:35:13.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.270+0000 7f9981531700 1 -- start start 2023-10-10T20:35:13.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.270+0000 7f9981531700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f997c159420 0x7f997c15a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:13.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.270+0000 7f9981531700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f997c1597f0 0x7f997c152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:13.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.270+0000 7f9981531700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f997c15d230 0x7f997c14e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:13.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.270+0000 7f9981531700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c155810 con 0x7f997c1597f0 2023-10-10T20:35:13.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.270+0000 7f9981531700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c152dc0 con 0x7f997c159420 2023-10-10T20:35:13.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.270+0000 7f9981531700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997c14ead0 con 0x7f997c15d230 2023-10-10T20:35:13.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.270+0000 7f997b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f997c1597f0 0x7f997c152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:13.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.270+0000 7f997b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f997c1597f0 0x7f997c152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42712/0 (socket says 172.21.15.89:42712) 2023-10-10T20:35:13.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.270+0000 7f997b7fe700 1 -- 172.21.15.89:0/1513647737 learned_addr learned my addr 172.21.15.89:0/1513647737 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:13.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.271+0000 7f9980d30700 1 --2- 172.21.15.89:0/1513647737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f997c15d230 0x7f997c14e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:13.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.271+0000 7f997bfff700 1 --2- 172.21.15.89:0/1513647737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f997c159420 0x7f997c15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:13.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.271+0000 7f997b7fe700 1 -- 172.21.15.89:0/1513647737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f997c15d230 msgr2=0x7f997c14e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:13.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.271+0000 7f997b7fe700 1 --2- 172.21.15.89:0/1513647737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f997c15d230 0x7f997c14e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.271+0000 7f997b7fe700 1 -- 172.21.15.89:0/1513647737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f997c159420 msgr2=0x7f997c15a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:13.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.271+0000 7f997b7fe700 1 --2- 172.21.15.89:0/1513647737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f997c159420 0x7f997c15a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.271+0000 7f997b7fe700 1 -- 172.21.15.89:0/1513647737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99700094f0 con 0x7f997c1597f0 2023-10-10T20:35:13.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.271+0000 7f997bfff700 1 --2- 172.21.15.89:0/1513647737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f997c159420 0x7f997c15a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:13.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.271+0000 7f997b7fe700 1 --2- 172.21.15.89:0/1513647737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f997c1597f0 0x7f997c152880 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f997c089340 tx=0x7f9970017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:13.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.271+0000 7f9963fff700 1 -- 172.21.15.89:0/1513647737 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9970024440 con 0x7f997c1597f0 2023-10-10T20:35:13.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.271+0000 7f9963fff700 1 -- 172.21.15.89:0/1513647737 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99700245a0 con 0x7f997c1597f0 2023-10-10T20:35:13.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.271+0000 7f9981531700 1 -- 172.21.15.89:0/1513647737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997c155a60 con 0x7f997c1597f0 2023-10-10T20:35:13.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.271+0000 7f9963fff700 1 -- 172.21.15.89:0/1513647737 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9970024940 con 0x7f997c1597f0 2023-10-10T20:35:13.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.271+0000 7f9981531700 1 -- 172.21.15.89:0/1513647737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997c156020 con 0x7f997c1597f0 2023-10-10T20:35:13.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.272+0000 7f9981531700 1 -- 172.21.15.89:0/1513647737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f997c04c3c0 con 0x7f997c1597f0 2023-10-10T20:35:13.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.272+0000 7f9963fff700 1 -- 172.21.15.89:0/1513647737 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f997001a420 con 0x7f997c1597f0 2023-10-10T20:35:13.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.273+0000 7f9963fff700 1 --2- 172.21.15.89:0/1513647737 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9968066f30 0x7f99680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:13.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.273+0000 7f997bfff700 1 --2- 172.21.15.89:0/1513647737 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9968066f30 0x7f99680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:13.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.273+0000 7f9963fff700 1 -- 172.21.15.89:0/1513647737 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9970086040 con 0x7f997c1597f0 2023-10-10T20:35:13.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.273+0000 7f997bfff700 1 --2- 172.21.15.89:0/1513647737 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9968066f30 0x7f99680693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f996c001840 tx=0x7f996c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:13.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.277+0000 7f9963fff700 1 -- 172.21.15.89:0/1513647737 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9970053680 con 0x7f997c1597f0 2023-10-10T20:35:13.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.534+0000 7f9981531700 1 -- 172.21.15.89:0/1513647737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f997c0b1a70 con 0x7f997c1597f0 2023-10-10T20:35:13.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.537+0000 7f9963fff700 1 -- 172.21.15.89:0/1513647737 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9970056ca0 con 0x7f997c1597f0 2023-10-10T20:35:13.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.539+0000 7f9981531700 1 -- 172.21.15.89:0/1513647737 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9968066f30 msgr2=0x7f99680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:13.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.539+0000 7f9981531700 1 --2- 172.21.15.89:0/1513647737 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9968066f30 0x7f99680693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f996c001840 tx=0x7f996c009040 comp rx=0 tx=0).stop 2023-10-10T20:35:13.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.539+0000 7f9981531700 1 -- 172.21.15.89:0/1513647737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f997c1597f0 msgr2=0x7f997c152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:13.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.539+0000 7f9981531700 1 --2- 172.21.15.89:0/1513647737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f997c1597f0 0x7f997c152880 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f997c089340 tx=0x7f9970017000 comp rx=0 tx=0).stop 2023-10-10T20:35:13.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.539+0000 7f9981531700 1 -- 172.21.15.89:0/1513647737 shutdown_connections 2023-10-10T20:35:13.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.539+0000 7f9981531700 1 --2- 172.21.15.89:0/1513647737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f997c15d230 0x7f997c14e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.539+0000 7f9981531700 1 --2- 172.21.15.89:0/1513647737 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9968066f30 0x7f99680693e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.539+0000 7f9981531700 1 --2- 172.21.15.89:0/1513647737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f997c159420 0x7f997c15a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.539+0000 7f9981531700 1 --2- 172.21.15.89:0/1513647737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f997c1597f0 0x7f997c152880 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.539+0000 7f9981531700 1 -- 172.21.15.89:0/1513647737 >> 172.21.15.89:0/1513647737 conn(0x7f997c0d0660 msgr2=0x7f997c0d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:13.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.539+0000 7f9981531700 1 -- 172.21.15.89:0/1513647737 shutdown_connections 2023-10-10T20:35:13.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.539+0000 7f9981531700 1 -- 172.21.15.89:0/1513647737 wait complete. 2023-10-10T20:35:13.553 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-10T20:35:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.667+0000 7f40dbc35700 1 Processor -- start 2023-10-10T20:35:13.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.667+0000 7f40dbc35700 1 -- start start 2023-10-10T20:35:13.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.668+0000 7f40dbc35700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f40d4159890 0x7f40d415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:13.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.668+0000 7f40dbc35700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40d4069560 0x7f40d40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:13.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.668+0000 7f40dbc35700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d4069ee0 con 0x7f40d4159c60 2023-10-10T20:35:13.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.668+0000 7f40dbc35700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d415a160 con 0x7f40d4159890 2023-10-10T20:35:13.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.668+0000 7f40dbc35700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d40d13a0 con 0x7f40d4069560 2023-10-10T20:35:13.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.668+0000 7f40d91d0700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f40d4159c60 0x7f40d415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40512/0 (socket says 172.21.15.89:40512) 2023-10-10T20:35:13.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.668+0000 7f40d91d0700 1 -- 172.21.15.89:0/2009309835 learned_addr learned my addr 172.21.15.89:0/2009309835 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:13.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.668+0000 7f40d99d1700 1 --2- 172.21.15.89:0/2009309835 >> v2:172.21.15.111:3300/0 conn(0x7f40d4159890 0x7f40d415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:13.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.668+0000 7f40da1d2700 1 --2- 172.21.15.89:0/2009309835 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40d4069560 0x7f40d40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:13.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.668+0000 7f40d89cf700 1 -- 172.21.15.89:0/2009309835 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4091485592 0 0) 0x7f40d4069ee0 con 0x7f40d4159c60 2023-10-10T20:35:13.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d89cf700 1 -- 172.21.15.89:0/2009309835 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40c0003570 con 0x7f40d4159c60 2023-10-10T20:35:13.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d89cf700 1 -- 172.21.15.89:0/2009309835 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3059265071 0 0) 0x7f40c0003570 con 0x7f40d4159c60 2023-10-10T20:35:13.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d89cf700 1 -- 172.21.15.89:0/2009309835 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40d4069560 msgr2=0x7f40d40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:13.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d89cf700 1 --2- 172.21.15.89:0/2009309835 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40d4069560 0x7f40d40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d89cf700 1 -- 172.21.15.89:0/2009309835 >> v2:172.21.15.111:3300/0 conn(0x7f40d4159890 msgr2=0x7f40d415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:13.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d89cf700 1 --2- 172.21.15.89:0/2009309835 >> v2:172.21.15.111:3300/0 conn(0x7f40d4159890 0x7f40d415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d89cf700 1 -- 172.21.15.89:0/2009309835 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d415e140 con 0x7f40d4159c60 2023-10-10T20:35:13.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d89cf700 1 -- 172.21.15.89:0/2009309835 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f40c4001470 con 0x7f40d4159c60 2023-10-10T20:35:13.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d89cf700 1 -- 172.21.15.89:0/2009309835 >> v1:172.21.15.89:6789/0 conn(0x7f40d4159c60 legacy=0x7f40d415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:13.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d89cf700 1 --2- 172.21.15.89:0/2009309835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c0003a60 0x7f40c0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:13.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d89cf700 1 --2- 172.21.15.89:0/2009309835 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40d4069560 0x7f40c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:13.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d89cf700 1 --2- 172.21.15.89:0/2009309835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40c0004d50 0x7f40c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:13.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d89cf700 1 -- 172.21.15.89:0/2009309835 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d415a160 con 0x7f40c0003a60 2023-10-10T20:35:13.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d89cf700 1 -- 172.21.15.89:0/2009309835 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d40d13a0 con 0x7f40c0004d50 2023-10-10T20:35:13.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d89cf700 1 -- 172.21.15.89:0/2009309835 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c000ba60 con 0x7f40d4069560 2023-10-10T20:35:13.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d99d1700 1 --2- 172.21.15.89:0/2009309835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c0003a60 0x7f40c0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:13.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d91d0700 1 --2- 172.21.15.89:0/2009309835 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40d4069560 0x7f40c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:13.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40da1d2700 1 --2- 172.21.15.89:0/2009309835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40c0004d50 0x7f40c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:13.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d99d1700 1 -- 172.21.15.89:0/2009309835 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40d4069560 msgr2=0x7f40c0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:13.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d99d1700 1 --2- 172.21.15.89:0/2009309835 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40d4069560 0x7f40c0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d99d1700 1 -- 172.21.15.89:0/2009309835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40c0004d50 msgr2=0x7f40c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:13.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d99d1700 1 --2- 172.21.15.89:0/2009309835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40c0004d50 0x7f40c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d99d1700 1 -- 172.21.15.89:0/2009309835 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c0003570 con 0x7f40c0003a60 2023-10-10T20:35:13.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d91d0700 1 --2- 172.21.15.89:0/2009309835 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40d4069560 0x7f40c0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:13.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d99d1700 1 --2- 172.21.15.89:0/2009309835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c0003a60 0x7f40c0003e70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f40d000e210 tx=0x7f40d000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afb6e2ed61512e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:13.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d89cf700 1 -- 172.21.15.89:0/2009309835 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d000ea60 con 0x7f40c0003a60 2023-10-10T20:35:13.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d89cf700 1 -- 172.21.15.89:0/2009309835 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d000f040 con 0x7f40c0003a60 2023-10-10T20:35:13.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.669+0000 7f40d89cf700 1 -- 172.21.15.89:0/2009309835 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d000ee10 con 0x7f40c0003a60 2023-10-10T20:35:13.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.670+0000 7f40dbc35700 1 -- 172.21.15.89:0/2009309835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c0003a60 msgr2=0x7f40c0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:13.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.670+0000 7f40dbc35700 1 --2- 172.21.15.89:0/2009309835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c0003a60 0x7f40c0003e70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f40d000e210 tx=0x7f40d000b040 comp rx=0 tx=0).stop 2023-10-10T20:35:13.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.670+0000 7f40da1d2700 1 -- 172.21.15.89:0/2009309835 reap_dead start 2023-10-10T20:35:13.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.670+0000 7f40dbc35700 1 -- 172.21.15.89:0/2009309835 shutdown_connections 2023-10-10T20:35:13.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.670+0000 7f40dbc35700 1 -- 172.21.15.89:0/2009309835 >> 172.21.15.89:0/2009309835 conn(0x7f40d4068e20 msgr2=0x7f40d40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:13.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.670+0000 7f40dbc35700 1 -- 172.21.15.89:0/2009309835 shutdown_connections 2023-10-10T20:35:13.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.670+0000 7f40dbc35700 1 -- 172.21.15.89:0/2009309835 wait complete. 2023-10-10T20:35:13.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.671+0000 7f40dbc35700 1 Processor -- start 2023-10-10T20:35:13.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.671+0000 7f40dbc35700 1 -- start start 2023-10-10T20:35:13.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.672+0000 7f40dbc35700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40c0003a60 0x7f40d40cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:13.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.672+0000 7f40dbc35700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40d4151d10 0x7f40d414d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.672+0000 7f40dbc35700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40d416d400 0x7f40d416d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.672+0000 7f40dbc35700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d416dde0 con 0x7f40d416d400 2023-10-10T20:35:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.672+0000 7f40dbc35700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d41521d0 con 0x7f40c0003a60 2023-10-10T20:35:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.672+0000 7f40dbc35700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d41524b0 con 0x7f40d4151d10 2023-10-10T20:35:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.672+0000 7f40da1d2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40d416d400 0x7f40d416d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.672+0000 7f40da1d2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40d416d400 0x7f40d416d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42724/0 (socket says 172.21.15.89:42724) 2023-10-10T20:35:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.672+0000 7f40da1d2700 1 -- 172.21.15.89:0/1522073776 learned_addr learned my addr 172.21.15.89:0/1522073776 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.672+0000 7f40d91d0700 1 --2- 172.21.15.89:0/1522073776 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40d4151d10 0x7f40d414d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.672+0000 7f40d99d1700 1 --2- 172.21.15.89:0/1522073776 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40c0003a60 0x7f40d40cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.672+0000 7f40da1d2700 1 -- 172.21.15.89:0/1522073776 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40d4151d10 msgr2=0x7f40d414d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.672+0000 7f40da1d2700 1 --2- 172.21.15.89:0/1522073776 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40d4151d10 0x7f40d414d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.672+0000 7f40da1d2700 1 -- 172.21.15.89:0/1522073776 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40c0003a60 msgr2=0x7f40d40cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.672+0000 7f40da1d2700 1 --2- 172.21.15.89:0/1522073776 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40c0003a60 0x7f40d40cfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.672+0000 7f40da1d2700 1 -- 172.21.15.89:0/1522073776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d4152790 con 0x7f40d416d400 2023-10-10T20:35:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.672+0000 7f40d91d0700 1 --2- 172.21.15.89:0/1522073776 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40d4151d10 0x7f40d414d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.672+0000 7f40d99d1700 1 --2- 172.21.15.89:0/1522073776 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40c0003a60 0x7f40d40cfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.672+0000 7f40da1d2700 1 --2- 172.21.15.89:0/1522073776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40d416d400 0x7f40d416d8a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f40d40cadc0 tx=0x7f40cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.673+0000 7f40c9ffb700 1 -- 172.21.15.89:0/1522073776 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc015670 con 0x7f40d416d400 2023-10-10T20:35:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.673+0000 7f40c9ffb700 1 -- 172.21.15.89:0/1522073776 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc0157d0 con 0x7f40d416d400 2023-10-10T20:35:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.673+0000 7f40dbc35700 1 -- 172.21.15.89:0/1522073776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d4152a70 con 0x7f40d416d400 2023-10-10T20:35:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.673+0000 7f40c9ffb700 1 -- 172.21.15.89:0/1522073776 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc015b90 con 0x7f40d416d400 2023-10-10T20:35:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.673+0000 7f40dbc35700 1 -- 172.21.15.89:0/1522073776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d414da50 con 0x7f40d416d400 2023-10-10T20:35:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.674+0000 7f40dbc35700 1 -- 172.21.15.89:0/1522073776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d404c3c0 con 0x7f40d416d400 2023-10-10T20:35:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.674+0000 7f40c9ffb700 1 -- 172.21.15.89:0/1522073776 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f40cc01f410 con 0x7f40d416d400 2023-10-10T20:35:13.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.674+0000 7f40c9ffb700 1 --2- 172.21.15.89:0/1522073776 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f40b8066f30 0x7f40b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:13.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.674+0000 7f40c9ffb700 1 -- 172.21.15.89:0/1522073776 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f40cc083fb0 con 0x7f40d416d400 2023-10-10T20:35:13.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.674+0000 7f40d99d1700 1 --2- 172.21.15.89:0/1522073776 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f40b8066f30 0x7f40b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:13.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.675+0000 7f40d99d1700 1 --2- 172.21.15.89:0/1522073776 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f40b8066f30 0x7f40b80693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f40d000afd0 tx=0x7f40d001a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:13.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.678+0000 7f40c9ffb700 1 -- 172.21.15.89:0/1522073776 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40cc051570 con 0x7f40d416d400 2023-10-10T20:35:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.938+0000 7f40dbc35700 1 -- 172.21.15.89:0/1522073776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f40d4150590 con 0x7f40d416d400 2023-10-10T20:35:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.942+0000 7f40c9ffb700 1 -- 172.21.15.89:0/1522073776 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7f40cc054b90 con 0x7f40d416d400 2023-10-10T20:35:13.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.944+0000 7f40dbc35700 1 -- 172.21.15.89:0/1522073776 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f40b8066f30 msgr2=0x7f40b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:13.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.944+0000 7f40dbc35700 1 --2- 172.21.15.89:0/1522073776 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f40b8066f30 0x7f40b80693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f40d000afd0 tx=0x7f40d001a000 comp rx=0 tx=0).stop 2023-10-10T20:35:13.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.944+0000 7f40dbc35700 1 -- 172.21.15.89:0/1522073776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40d416d400 msgr2=0x7f40d416d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:13.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.944+0000 7f40dbc35700 1 --2- 172.21.15.89:0/1522073776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40d416d400 0x7f40d416d8a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f40d40cadc0 tx=0x7f40cc008040 comp rx=0 tx=0).stop 2023-10-10T20:35:13.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.944+0000 7f40dbc35700 1 -- 172.21.15.89:0/1522073776 shutdown_connections 2023-10-10T20:35:13.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.944+0000 7f40dbc35700 1 --2- 172.21.15.89:0/1522073776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40d416d400 0x7f40d416d8a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.944+0000 7f40dbc35700 1 --2- 172.21.15.89:0/1522073776 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f40b8066f30 0x7f40b80693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.944+0000 7f40dbc35700 1 --2- 172.21.15.89:0/1522073776 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40c0003a60 0x7f40d40cfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.944+0000 7f40dbc35700 1 --2- 172.21.15.89:0/1522073776 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f40d4151d10 0x7f40d414d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:13.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.944+0000 7f40dbc35700 1 -- 172.21.15.89:0/1522073776 >> 172.21.15.89:0/1522073776 conn(0x7f40d4068e20 msgr2=0x7f40d4150950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:13.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.944+0000 7f40dbc35700 1 -- 172.21.15.89:0/1522073776 shutdown_connections 2023-10-10T20:35:13.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:13.944+0000 7f40dbc35700 1 -- 172.21.15.89:0/1522073776 wait complete. 2023-10-10T20:35:13.958 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-10T20:35:14.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.071+0000 7fd86a5f3700 1 Processor -- start 2023-10-10T20:35:14.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.071+0000 7fd86a5f3700 1 -- start start 2023-10-10T20:35:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd86a5f3700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd864159c60 0x7fd86415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd86a5f3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd86415db10 0x7fd864069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd86a5f3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86415a160 con 0x7fd864159890 2023-10-10T20:35:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd86a5f3700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd86415df50 con 0x7fd864159c60 2023-10-10T20:35:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd86a5f3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640d13a0 con 0x7fd86415db10 2023-10-10T20:35:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd863fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd864159890 0x7fd86415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40524/0 (socket says 172.21.15.89:40524) 2023-10-10T20:35:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd863fff700 1 -- 172.21.15.89:0/416605847 learned_addr learned my addr 172.21.15.89:0/416605847 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd868b90700 1 --2- 172.21.15.89:0/416605847 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd86415db10 0x7fd864069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd8637fe700 1 --2- 172.21.15.89:0/416605847 >> v2:172.21.15.111:3300/0 conn(0x7fd864159c60 0x7fd86415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd862ffd700 1 -- 172.21.15.89:0/416605847 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235638049 0 0) 0x7fd86415a160 con 0x7fd864159890 2023-10-10T20:35:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd862ffd700 1 -- 172.21.15.89:0/416605847 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd84c003570 con 0x7fd864159890 2023-10-10T20:35:14.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd862ffd700 1 -- 172.21.15.89:0/416605847 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 868389287 0 0) 0x7fd84c003570 con 0x7fd864159890 2023-10-10T20:35:14.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd862ffd700 1 -- 172.21.15.89:0/416605847 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd86415db10 msgr2=0x7fd864069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:14.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd862ffd700 1 --2- 172.21.15.89:0/416605847 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd86415db10 0x7fd864069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd862ffd700 1 -- 172.21.15.89:0/416605847 >> v2:172.21.15.111:3300/0 conn(0x7fd864159c60 msgr2=0x7fd86415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:14.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd862ffd700 1 --2- 172.21.15.89:0/416605847 >> v2:172.21.15.111:3300/0 conn(0x7fd864159c60 0x7fd86415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd862ffd700 1 -- 172.21.15.89:0/416605847 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86415e140 con 0x7fd864159890 2023-10-10T20:35:14.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd8637fe700 1 --2- 172.21.15.89:0/416605847 >> v2:172.21.15.111:3300/0 conn(0x7fd864159c60 0x7fd86415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:35:14.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd862ffd700 1 -- 172.21.15.89:0/416605847 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fd854001470 con 0x7fd864159890 2023-10-10T20:35:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.072+0000 7fd862ffd700 1 -- 172.21.15.89:0/416605847 >> v1:172.21.15.89:6789/0 conn(0x7fd864159890 legacy=0x7fd86415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.073+0000 7fd862ffd700 1 --2- 172.21.15.89:0/416605847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd84c003a60 0x7fd84c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.073+0000 7fd862ffd700 1 --2- 172.21.15.89:0/416605847 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd86415db10 0x7fd84c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.073+0000 7fd862ffd700 1 --2- 172.21.15.89:0/416605847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd84c004d50 0x7fd84c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.073+0000 7fd862ffd700 1 -- 172.21.15.89:0/416605847 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640d13a0 con 0x7fd84c004d50 2023-10-10T20:35:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.073+0000 7fd862ffd700 1 -- 172.21.15.89:0/416605847 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c00ba60 con 0x7fd84c003a60 2023-10-10T20:35:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.073+0000 7fd862ffd700 1 -- 172.21.15.89:0/416605847 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c00bba0 con 0x7fd86415db10 2023-10-10T20:35:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.073+0000 7fd868b90700 1 --2- 172.21.15.89:0/416605847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd84c004d50 0x7fd84c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.073+0000 7fd863fff700 1 --2- 172.21.15.89:0/416605847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd84c003a60 0x7fd84c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.073+0000 7fd8637fe700 1 --2- 172.21.15.89:0/416605847 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd86415db10 0x7fd84c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.073+0000 7fd868b90700 1 -- 172.21.15.89:0/416605847 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd86415db10 msgr2=0x7fd84c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.073+0000 7fd868b90700 1 --2- 172.21.15.89:0/416605847 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd86415db10 0x7fd84c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.073+0000 7fd868b90700 1 -- 172.21.15.89:0/416605847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd84c003a60 msgr2=0x7fd84c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.073+0000 7fd868b90700 1 --2- 172.21.15.89:0/416605847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd84c003a60 0x7fd84c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.073+0000 7fd868b90700 1 -- 172.21.15.89:0/416605847 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c003570 con 0x7fd84c004d50 2023-10-10T20:35:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.073+0000 7fd868b90700 1 --2- 172.21.15.89:0/416605847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd84c004d50 0x7fd84c0051c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00e210 tx=0x7fd85c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6546c0b27b10bb3a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.073+0000 7fd862ffd700 1 -- 172.21.15.89:0/416605847 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c00ea60 con 0x7fd84c004d50 2023-10-10T20:35:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.073+0000 7fd862ffd700 1 -- 172.21.15.89:0/416605847 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85c00f040 con 0x7fd84c004d50 2023-10-10T20:35:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.073+0000 7fd862ffd700 1 -- 172.21.15.89:0/416605847 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c00ee10 con 0x7fd84c004d50 2023-10-10T20:35:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.074+0000 7fd86a5f3700 1 -- 172.21.15.89:0/416605847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd84c004d50 msgr2=0x7fd84c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.074+0000 7fd86a5f3700 1 --2- 172.21.15.89:0/416605847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd84c004d50 0x7fd84c0051c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00e210 tx=0x7fd85c00b040 comp rx=0 tx=0).stop 2023-10-10T20:35:14.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.074+0000 7fd868b90700 1 -- 172.21.15.89:0/416605847 reap_dead start 2023-10-10T20:35:14.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.074+0000 7fd86a5f3700 1 -- 172.21.15.89:0/416605847 shutdown_connections 2023-10-10T20:35:14.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.074+0000 7fd86a5f3700 1 -- 172.21.15.89:0/416605847 >> 172.21.15.89:0/416605847 conn(0x7fd864068e20 msgr2=0x7fd8640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:14.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.074+0000 7fd86a5f3700 1 -- 172.21.15.89:0/416605847 shutdown_connections 2023-10-10T20:35:14.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.074+0000 7fd86a5f3700 1 -- 172.21.15.89:0/416605847 wait complete. 2023-10-10T20:35:14.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.075+0000 7fd86a5f3700 1 Processor -- start 2023-10-10T20:35:14.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.075+0000 7fd86a5f3700 1 -- start start 2023-10-10T20:35:14.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.076+0000 7fd86a5f3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd86414d740 0x7fd86415c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.076+0000 7fd86a5f3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd86415cd70 0x7fd864155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.076+0000 7fd86a5f3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd864156430 0x7fd86416cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.076+0000 7fd86a5f3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd864157e50 con 0x7fd864156430 2023-10-10T20:35:14.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.076+0000 7fd86a5f3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd864158c70 con 0x7fd86414d740 2023-10-10T20:35:14.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.076+0000 7fd86a5f3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86415d1e0 con 0x7fd86415cd70 2023-10-10T20:35:14.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.076+0000 7fd868b90700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd864156430 0x7fd86416cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.076+0000 7fd868b90700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd864156430 0x7fd86416cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42738/0 (socket says 172.21.15.89:42738) 2023-10-10T20:35:14.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.076+0000 7fd868b90700 1 -- 172.21.15.89:0/2064875252 learned_addr learned my addr 172.21.15.89:0/2064875252 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:14.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.076+0000 7fd863fff700 1 --2- 172.21.15.89:0/2064875252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd86414d740 0x7fd86415c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.076+0000 7fd8637fe700 1 --2- 172.21.15.89:0/2064875252 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd86415cd70 0x7fd864155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.076+0000 7fd868b90700 1 -- 172.21.15.89:0/2064875252 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd86415cd70 msgr2=0x7fd864155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:14.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.076+0000 7fd868b90700 1 --2- 172.21.15.89:0/2064875252 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd86415cd70 0x7fd864155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.076+0000 7fd868b90700 1 -- 172.21.15.89:0/2064875252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd86414d740 msgr2=0x7fd86415c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:14.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.076+0000 7fd868b90700 1 --2- 172.21.15.89:0/2064875252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd86414d740 0x7fd86415c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.076+0000 7fd868b90700 1 -- 172.21.15.89:0/2064875252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd864158060 con 0x7fd864156430 2023-10-10T20:35:14.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.076+0000 7fd868b90700 1 --2- 172.21.15.89:0/2064875252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd864156430 0x7fd86416cfa0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00afd0 tx=0x7fd85c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:14.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.077+0000 7fd84bfff700 1 -- 172.21.15.89:0/2064875252 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c00eee0 con 0x7fd864156430 2023-10-10T20:35:14.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.077+0000 7fd84bfff700 1 -- 172.21.15.89:0/2064875252 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85c022070 con 0x7fd864156430 2023-10-10T20:35:14.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.077+0000 7fd84bfff700 1 -- 172.21.15.89:0/2064875252 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c01e890 con 0x7fd864156430 2023-10-10T20:35:14.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.077+0000 7fd86a5f3700 1 -- 172.21.15.89:0/2064875252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd864158340 con 0x7fd864156430 2023-10-10T20:35:14.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.077+0000 7fd86a5f3700 1 -- 172.21.15.89:0/2064875252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd864158900 con 0x7fd864156430 2023-10-10T20:35:14.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.078+0000 7fd86a5f3700 1 -- 172.21.15.89:0/2064875252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd864158fe0 con 0x7fd864156430 2023-10-10T20:35:14.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.078+0000 7fd84bfff700 1 -- 172.21.15.89:0/2064875252 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd85c003040 con 0x7fd864156430 2023-10-10T20:35:14.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.078+0000 7fd84bfff700 1 --2- 172.21.15.89:0/2064875252 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd850066f30 0x7fd8500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.078+0000 7fd84bfff700 1 -- 172.21.15.89:0/2064875252 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd85c024070 con 0x7fd864156430 2023-10-10T20:35:14.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.078+0000 7fd863fff700 1 --2- 172.21.15.89:0/2064875252 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd850066f30 0x7fd8500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.079+0000 7fd863fff700 1 --2- 172.21.15.89:0/2064875252 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd850066f30 0x7fd8500693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd854001d20 tx=0x7fd854001e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:14.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.082+0000 7fd84bfff700 1 -- 172.21.15.89:0/2064875252 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd85c0534f0 con 0x7fd864156430 2023-10-10T20:35:14.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.341+0000 7fd86a5f3700 1 -- 172.21.15.89:0/2064875252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fd86404c3c0 con 0x7fd864156430 2023-10-10T20:35:14.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.344+0000 7fd84bfff700 1 -- 172.21.15.89:0/2064875252 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd85c056b10 con 0x7fd864156430 2023-10-10T20:35:14.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.346+0000 7fd86a5f3700 1 -- 172.21.15.89:0/2064875252 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd850066f30 msgr2=0x7fd8500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:14.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.346+0000 7fd86a5f3700 1 --2- 172.21.15.89:0/2064875252 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd850066f30 0x7fd8500693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd854001d20 tx=0x7fd854001e90 comp rx=0 tx=0).stop 2023-10-10T20:35:14.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.346+0000 7fd86a5f3700 1 -- 172.21.15.89:0/2064875252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd864156430 msgr2=0x7fd86416cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:14.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.346+0000 7fd86a5f3700 1 --2- 172.21.15.89:0/2064875252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd864156430 0x7fd86416cfa0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00afd0 tx=0x7fd85c01a000 comp rx=0 tx=0).stop 2023-10-10T20:35:14.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.346+0000 7fd86a5f3700 1 -- 172.21.15.89:0/2064875252 shutdown_connections 2023-10-10T20:35:14.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.346+0000 7fd86a5f3700 1 --2- 172.21.15.89:0/2064875252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd864156430 0x7fd86416cfa0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.346+0000 7fd86a5f3700 1 --2- 172.21.15.89:0/2064875252 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd850066f30 0x7fd8500693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.346+0000 7fd86a5f3700 1 --2- 172.21.15.89:0/2064875252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd86414d740 0x7fd86415c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.346+0000 7fd86a5f3700 1 --2- 172.21.15.89:0/2064875252 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd86415cd70 0x7fd864155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.346+0000 7fd86a5f3700 1 -- 172.21.15.89:0/2064875252 >> 172.21.15.89:0/2064875252 conn(0x7fd864068e20 msgr2=0x7fd86414e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:14.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.346+0000 7fd86a5f3700 1 -- 172.21.15.89:0/2064875252 shutdown_connections 2023-10-10T20:35:14.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.346+0000 7fd86a5f3700 1 -- 172.21.15.89:0/2064875252 wait complete. 2023-10-10T20:35:14.361 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:35:14.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.473+0000 7f8ac3cad700 1 Processor -- start 2023-10-10T20:35:14.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.473+0000 7f8ac3cad700 1 -- start start 2023-10-10T20:35:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.474+0000 7f8ac3cad700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8abc159890 0x7f8abc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.474+0000 7f8ac3cad700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8abc069560 0x7f8abc0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.474+0000 7f8ac3cad700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8abc069ee0 con 0x7f8abc159c60 2023-10-10T20:35:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.474+0000 7f8ac3cad700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8abc15a160 con 0x7f8abc069560 2023-10-10T20:35:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.474+0000 7f8ac3cad700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8abc0d13a0 con 0x7f8abc159890 2023-10-10T20:35:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.474+0000 7f8ac224a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8abc069560 0x7f8abc0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.474+0000 7f8ac224a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8abc069560 0x7f8abc0699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42750/0 (socket says 172.21.15.89:42750) 2023-10-10T20:35:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.474+0000 7f8ac224a700 1 -- 172.21.15.89:0/2365242308 learned_addr learned my addr 172.21.15.89:0/2365242308 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.474+0000 7f8ac1248700 1 --1- 172.21.15.89:0/2365242308 >> v1:172.21.15.89:6789/0 conn(0x7f8abc159c60 0x7f8abc15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40528/0 (socket says 172.21.15.89:40528) 2023-10-10T20:35:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.474+0000 7f8ac1a49700 1 --2- 172.21.15.89:0/2365242308 >> v2:172.21.15.111:3300/0 conn(0x7f8abc159890 0x7f8abc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.474+0000 7f8ac224a700 1 -- 172.21.15.89:0/2365242308 >> v2:172.21.15.111:3300/0 conn(0x7f8abc159890 msgr2=0x7f8abc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.474+0000 7f8ac224a700 1 --2- 172.21.15.89:0/2365242308 >> v2:172.21.15.111:3300/0 conn(0x7f8abc159890 0x7f8abc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.474+0000 7f8ac224a700 1 -- 172.21.15.89:0/2365242308 >> v1:172.21.15.89:6789/0 conn(0x7f8abc159c60 legacy=0x7f8abc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.474+0000 7f8ac224a700 1 -- 172.21.15.89:0/2365242308 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc15e140 con 0x7f8abc069560 2023-10-10T20:35:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.474+0000 7f8ac0a47700 1 -- 172.21.15.89:0/2365242308 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2523713375 0 0) 0x7f8abc069ee0 con 0x7f8abc159c60 2023-10-10T20:35:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.474+0000 7f8ac224a700 1 --2- 172.21.15.89:0/2365242308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8abc069560 0x7f8abc0699a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0d1090 tx=0x7f8ab4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa098ece8ad9aecc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.474+0000 7f8ac0a47700 1 -- 172.21.15.89:0/2365242308 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab400e070 con 0x7f8abc069560 2023-10-10T20:35:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.474+0000 7f8ac0a47700 1 -- 172.21.15.89:0/2365242308 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab400ba20 con 0x7f8abc069560 2023-10-10T20:35:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.474+0000 7f8ac0a47700 1 -- 172.21.15.89:0/2365242308 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40092e0 con 0x7f8abc069560 2023-10-10T20:35:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.475+0000 7f8ac3cad700 1 -- 172.21.15.89:0/2365242308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8abc069560 msgr2=0x7f8abc0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.475+0000 7f8ac3cad700 1 --2- 172.21.15.89:0/2365242308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8abc069560 0x7f8abc0699a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0d1090 tx=0x7f8ab4009840 comp rx=0 tx=0).stop 2023-10-10T20:35:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.475+0000 7f8ac3cad700 1 -- 172.21.15.89:0/2365242308 shutdown_connections 2023-10-10T20:35:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.475+0000 7f8ac3cad700 1 --2- 172.21.15.89:0/2365242308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8abc069560 0x7f8abc0699a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.475+0000 7f8ac3cad700 1 --2- 172.21.15.89:0/2365242308 >> v2:172.21.15.111:3300/0 conn(0x7f8abc159890 0x7f8abc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.475+0000 7f8ac3cad700 1 -- 172.21.15.89:0/2365242308 >> 172.21.15.89:0/2365242308 conn(0x7f8abc068e20 msgr2=0x7f8abc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.475+0000 7f8ac3cad700 1 -- 172.21.15.89:0/2365242308 shutdown_connections 2023-10-10T20:35:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.475+0000 7f8ac3cad700 1 -- 172.21.15.89:0/2365242308 wait complete. 2023-10-10T20:35:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.476+0000 7f8ac3cad700 1 Processor -- start 2023-10-10T20:35:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.476+0000 7f8ac3cad700 1 -- start start 2023-10-10T20:35:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ac3cad700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8abc069560 0x7f8abc0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ac3cad700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8abc159890 0x7f8abc16d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ac3cad700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8abc159c60 0x7f8abc0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ac3cad700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc15b600 con 0x7f8abc159c60 2023-10-10T20:35:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ac3cad700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc16df30 con 0x7f8abc069560 2023-10-10T20:35:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ac3cad700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0b2030 con 0x7f8abc159890 2023-10-10T20:35:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ac224a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8abc159c60 0x7f8abc0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ac224a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8abc159c60 0x7f8abc0b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42758/0 (socket says 172.21.15.89:42758) 2023-10-10T20:35:14.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ac224a700 1 -- 172.21.15.89:0/2529754210 learned_addr learned my addr 172.21.15.89:0/2529754210 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:14.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ac1a49700 1 --2- 172.21.15.89:0/2529754210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8abc069560 0x7f8abc0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ac224a700 1 -- 172.21.15.89:0/2529754210 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8abc159890 msgr2=0x7f8abc16d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:14.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ac1248700 1 --2- 172.21.15.89:0/2529754210 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8abc159890 0x7f8abc16d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ac224a700 1 --2- 172.21.15.89:0/2529754210 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8abc159890 0x7f8abc16d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ac224a700 1 -- 172.21.15.89:0/2529754210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8abc069560 msgr2=0x7f8abc0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:14.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ac224a700 1 --2- 172.21.15.89:0/2529754210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8abc069560 0x7f8abc0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ac224a700 1 -- 172.21.15.89:0/2529754210 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab4009ae0 con 0x7f8abc159c60 2023-10-10T20:35:14.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ac1a49700 1 --2- 172.21.15.89:0/2529754210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8abc069560 0x7f8abc0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:14.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ac224a700 1 --2- 172.21.15.89:0/2529754210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8abc159c60 0x7f8abc0b1af0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009880 tx=0x7f8ab4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ab1ffb700 1 -- 172.21.15.89:0/2529754210 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4008be0 con 0x7f8abc159c60 2023-10-10T20:35:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ab1ffb700 1 -- 172.21.15.89:0/2529754210 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab4008d60 con 0x7f8abc159c60 2023-10-10T20:35:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ac3cad700 1 -- 172.21.15.89:0/2529754210 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc16d690 con 0x7f8abc159c60 2023-10-10T20:35:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.477+0000 7f8ab1ffb700 1 -- 172.21.15.89:0/2529754210 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40266b0 con 0x7f8abc159c60 2023-10-10T20:35:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.478+0000 7f8ac3cad700 1 -- 172.21.15.89:0/2529754210 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8abc16dc50 con 0x7f8abc159c60 2023-10-10T20:35:14.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.478+0000 7f8ac3cad700 1 -- 172.21.15.89:0/2529754210 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8abc15b770 con 0x7f8abc159c60 2023-10-10T20:35:14.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.479+0000 7f8ab1ffb700 1 -- 172.21.15.89:0/2529754210 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ab4003040 con 0x7f8abc159c60 2023-10-10T20:35:14.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.479+0000 7f8ab1ffb700 1 --2- 172.21.15.89:0/2529754210 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8aa0066f30 0x7f8aa00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.479+0000 7f8ab1ffb700 1 -- 172.21.15.89:0/2529754210 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8ab40862d0 con 0x7f8abc159c60 2023-10-10T20:35:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.479+0000 7f8ac1a49700 1 --2- 172.21.15.89:0/2529754210 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8aa0066f30 0x7f8aa00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.480+0000 7f8ac1a49700 1 --2- 172.21.15.89:0/2529754210 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8aa0066f30 0x7f8aa00693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8006f00 tx=0x7f8ab8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.483+0000 7f8ab1ffb700 1 -- 172.21.15.89:0/2529754210 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ab40537f0 con 0x7f8abc159c60 2023-10-10T20:35:14.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.667+0000 7f8ac3cad700 1 -- 172.21.15.89:0/2529754210 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8abc04c3c0 con 0x7f8abc159c60 2023-10-10T20:35:14.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.668+0000 7f8ab1ffb700 1 -- 172.21.15.89:0/2529754210 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+12134 (secure 0 0 0) 0x7f8ab4009940 con 0x7f8abc159c60 2023-10-10T20:35:14.670 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:14.670 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":27,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:35:11.495452+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:35:14.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.670+0000 7f8ac3cad700 1 -- 172.21.15.89:0/2529754210 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8aa0066f30 msgr2=0x7f8aa00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.670+0000 7f8ac3cad700 1 --2- 172.21.15.89:0/2529754210 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8aa0066f30 0x7f8aa00693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8006f00 tx=0x7f8ab8008040 comp rx=0 tx=0).stop 2023-10-10T20:35:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.670+0000 7f8ac3cad700 1 -- 172.21.15.89:0/2529754210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8abc159c60 msgr2=0x7f8abc0b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.670+0000 7f8ac3cad700 1 --2- 172.21.15.89:0/2529754210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8abc159c60 0x7f8abc0b1af0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009880 tx=0x7f8ab4018000 comp rx=0 tx=0).stop 2023-10-10T20:35:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.670+0000 7f8ac3cad700 1 -- 172.21.15.89:0/2529754210 shutdown_connections 2023-10-10T20:35:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.670+0000 7f8ac3cad700 1 --2- 172.21.15.89:0/2529754210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8abc159c60 0x7f8abc0b1af0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.670+0000 7f8ac3cad700 1 --2- 172.21.15.89:0/2529754210 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8aa0066f30 0x7f8aa00693e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.670+0000 7f8ac3cad700 1 --2- 172.21.15.89:0/2529754210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8abc069560 0x7f8abc0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.670+0000 7f8ac3cad700 1 --2- 172.21.15.89:0/2529754210 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8abc159890 0x7f8abc16d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.670+0000 7f8ac3cad700 1 -- 172.21.15.89:0/2529754210 >> 172.21.15.89:0/2529754210 conn(0x7f8abc068e20 msgr2=0x7f8abc070530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.670+0000 7f8ac3cad700 1 -- 172.21.15.89:0/2529754210 shutdown_connections 2023-10-10T20:35:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.670+0000 7f8ac3cad700 1 -- 172.21.15.89:0/2529754210 wait complete. 2023-10-10T20:35:14.684 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:35:14.685 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:35:14.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.799+0000 7f841ad7c700 1 Processor -- start 2023-10-10T20:35:14.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.799+0000 7f841ad7c700 1 -- start start 2023-10-10T20:35:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.800+0000 7f841ad7c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8414159890 0x7f841415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.800+0000 7f841ad7c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f841415a160 con 0x7f8414069560 2023-10-10T20:35:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.800+0000 7f841ad7c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84140d13a0 con 0x7f8414159c60 2023-10-10T20:35:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.800+0000 7f841ad7c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841415e140 con 0x7f8414159890 2023-10-10T20:35:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.800+0000 7f8419319700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8414069560 0x7f84140ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40542/0 (socket says 172.21.15.89:40542) 2023-10-10T20:35:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.800+0000 7f8419319700 1 -- 172.21.15.89:0/149462889 learned_addr learned my addr 172.21.15.89:0/149462889 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.800+0000 7f8418b18700 1 --2- 172.21.15.89:0/149462889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8414159890 0x7f841415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.800+0000 7f84137fe700 1 -- 172.21.15.89:0/149462889 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111748909 0 0) 0x7f841415a160 con 0x7f8414069560 2023-10-10T20:35:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.800+0000 7f84137fe700 1 -- 172.21.15.89:0/149462889 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8404003570 con 0x7f8414069560 2023-10-10T20:35:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f84137fe700 1 -- 172.21.15.89:0/149462889 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035402793 0 0) 0x7f84140d13a0 con 0x7f8414159c60 2023-10-10T20:35:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f84137fe700 1 -- 172.21.15.89:0/149462889 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f841415a160 con 0x7f8414159c60 2023-10-10T20:35:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f84137fe700 1 -- 172.21.15.89:0/149462889 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1766494340 0 0) 0x7f8404003570 con 0x7f8414069560 2023-10-10T20:35:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f84137fe700 1 -- 172.21.15.89:0/149462889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8414159890 msgr2=0x7f841415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f84137fe700 1 --2- 172.21.15.89:0/149462889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8414159890 0x7f841415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f84137fe700 1 -- 172.21.15.89:0/149462889 >> v1:172.21.15.111:6789/0 conn(0x7f8414159c60 legacy=0x7f841415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f84137fe700 1 -- 172.21.15.89:0/149462889 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841415a3d0 con 0x7f8414069560 2023-10-10T20:35:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f84137fe700 1 -- 172.21.15.89:0/149462889 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f840c0017f0 con 0x7f8414069560 2023-10-10T20:35:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f84137fe700 1 -- 172.21.15.89:0/149462889 >> v1:172.21.15.89:6789/0 conn(0x7f8414069560 legacy=0x7f84140ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f84137fe700 1 --2- 172.21.15.89:0/149462889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8404003c20 0x7f8404004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f84137fe700 1 --2- 172.21.15.89:0/149462889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8404004810 0x7f8404004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f84137fe700 1 --2- 172.21.15.89:0/149462889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8414159890 0x7f84040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f84137fe700 1 -- 172.21.15.89:0/149462889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841415e140 con 0x7f8404004810 2023-10-10T20:35:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f84137fe700 1 -- 172.21.15.89:0/149462889 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840400bb40 con 0x7f8404003c20 2023-10-10T20:35:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f84137fe700 1 -- 172.21.15.89:0/149462889 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840400bd00 con 0x7f8414159890 2023-10-10T20:35:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f8413fff700 1 --2- 172.21.15.89:0/149462889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8404004810 0x7f8404004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f8418b18700 1 --2- 172.21.15.89:0/149462889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8404003c20 0x7f8404004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f8419319700 1 --2- 172.21.15.89:0/149462889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8414159890 0x7f84040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f8413fff700 1 -- 172.21.15.89:0/149462889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8414159890 msgr2=0x7f84040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f8413fff700 1 --2- 172.21.15.89:0/149462889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8414159890 0x7f84040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f8413fff700 1 -- 172.21.15.89:0/149462889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8404003c20 msgr2=0x7f8404004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f8413fff700 1 --2- 172.21.15.89:0/149462889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8404003c20 0x7f8404004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f8413fff700 1 -- 172.21.15.89:0/149462889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8404003570 con 0x7f8404004810 2023-10-10T20:35:14.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f8413fff700 1 --2- 172.21.15.89:0/149462889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8404004810 0x7f8404004c80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f84000019e0 tx=0x7f8400007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88393e8cab90517 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:14.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f84137fe700 1 -- 172.21.15.89:0/149462889 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840000c5d0 con 0x7f8404004810 2023-10-10T20:35:14.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f84137fe700 1 -- 172.21.15.89:0/149462889 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840000c730 con 0x7f8404004810 2023-10-10T20:35:14.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.801+0000 7f84137fe700 1 -- 172.21.15.89:0/149462889 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840000cad0 con 0x7f8404004810 2023-10-10T20:35:14.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.802+0000 7f841ad7c700 1 -- 172.21.15.89:0/149462889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8404004810 msgr2=0x7f8404004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:14.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.802+0000 7f841ad7c700 1 --2- 172.21.15.89:0/149462889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8404004810 0x7f8404004c80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f84000019e0 tx=0x7f8400007d30 comp rx=0 tx=0).stop 2023-10-10T20:35:14.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.802+0000 7f8419319700 1 -- 172.21.15.89:0/149462889 reap_dead start 2023-10-10T20:35:14.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.802+0000 7f841ad7c700 1 -- 172.21.15.89:0/149462889 shutdown_connections 2023-10-10T20:35:14.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.802+0000 7f841ad7c700 1 -- 172.21.15.89:0/149462889 >> 172.21.15.89:0/149462889 conn(0x7f8414068e20 msgr2=0x7f84140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:14.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.802+0000 7f841ad7c700 1 -- 172.21.15.89:0/149462889 shutdown_connections 2023-10-10T20:35:14.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.802+0000 7f841ad7c700 1 -- 172.21.15.89:0/149462889 wait complete. 2023-10-10T20:35:14.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.803+0000 7f841ad7c700 1 Processor -- start 2023-10-10T20:35:14.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.803+0000 7f841ad7c700 1 -- start start 2023-10-10T20:35:14.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.804+0000 7f841ad7c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8404004810 0x7f84140b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.804+0000 7f841ad7c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84140cda70 0x7f84140b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.804+0000 7f841ad7c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84140b2110 0x7f84141565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.804+0000 7f841ad7c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84140cff40 con 0x7f84140b2110 2023-10-10T20:35:14.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.804+0000 7f841ad7c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8414156180 con 0x7f84140cda70 2023-10-10T20:35:14.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.804+0000 7f841ad7c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84140b2590 con 0x7f8404004810 2023-10-10T20:35:14.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.804+0000 7f8419319700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84140b2110 0x7f84141565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.804+0000 7f8419319700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84140b2110 0x7f84141565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42770/0 (socket says 172.21.15.89:42770) 2023-10-10T20:35:14.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.804+0000 7f8419319700 1 -- 172.21.15.89:0/337636206 learned_addr learned my addr 172.21.15.89:0/337636206 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:14.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.804+0000 7f8413fff700 1 --2- 172.21.15.89:0/337636206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84140cda70 0x7f84140b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.804+0000 7f8418b18700 1 --2- 172.21.15.89:0/337636206 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8404004810 0x7f84140b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.804+0000 7f8419319700 1 -- 172.21.15.89:0/337636206 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8404004810 msgr2=0x7f84140b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:14.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.804+0000 7f8419319700 1 --2- 172.21.15.89:0/337636206 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8404004810 0x7f84140b5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.804+0000 7f8419319700 1 -- 172.21.15.89:0/337636206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84140cda70 msgr2=0x7f84140b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:14.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.804+0000 7f8419319700 1 --2- 172.21.15.89:0/337636206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84140cda70 0x7f84140b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.804+0000 7f8419319700 1 -- 172.21.15.89:0/337636206 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84140cf980 con 0x7f84140b2110 2023-10-10T20:35:14.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.804+0000 7f8419319700 1 --2- 172.21.15.89:0/337636206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84140b2110 0x7f84141565a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f841406a890 tx=0x7f840c002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:14.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.805+0000 7f8410ff9700 1 -- 172.21.15.89:0/337636206 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c00a9e0 con 0x7f84140b2110 2023-10-10T20:35:14.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.805+0000 7f8410ff9700 1 -- 172.21.15.89:0/337636206 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c00ab60 con 0x7f84140b2110 2023-10-10T20:35:14.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.805+0000 7f841ad7c700 1 -- 172.21.15.89:0/337636206 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84140cfc60 con 0x7f84140b2110 2023-10-10T20:35:14.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.805+0000 7f8410ff9700 1 -- 172.21.15.89:0/337636206 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c0133f0 con 0x7f84140b2110 2023-10-10T20:35:14.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.805+0000 7f841ad7c700 1 -- 172.21.15.89:0/337636206 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84140cf630 con 0x7f84140b2110 2023-10-10T20:35:14.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.806+0000 7f841ad7c700 1 -- 172.21.15.89:0/337636206 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84141562c0 con 0x7f84140b2110 2023-10-10T20:35:14.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.806+0000 7f8410ff9700 1 -- 172.21.15.89:0/337636206 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f840c0137a0 con 0x7f84140b2110 2023-10-10T20:35:14.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.806+0000 7f8410ff9700 1 --2- 172.21.15.89:0/337636206 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83f8066f30 0x7f83f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:14.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.806+0000 7f8410ff9700 1 -- 172.21.15.89:0/337636206 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f840c0847a0 con 0x7f84140b2110 2023-10-10T20:35:14.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.807+0000 7f8418b18700 1 --2- 172.21.15.89:0/337636206 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83f8066f30 0x7f83f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:14.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.807+0000 7f8418b18700 1 --2- 172.21.15.89:0/337636206 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83f8066f30 0x7f83f80693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8408006f00 tx=0x7f84080091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:14.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.810+0000 7f8410ff9700 1 -- 172.21.15.89:0/337636206 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84141562c0 con 0x7f84140b2110 2023-10-10T20:35:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.993+0000 7f841ad7c700 1 -- 172.21.15.89:0/337636206 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84141562c0 con 0x7f84140b2110 2023-10-10T20:35:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.993+0000 7f8410ff9700 1 -- 172.21.15.89:0/337636206 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f84141562c0 con 0x7f84140b2110 2023-10-10T20:35:14.996 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:35:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.995+0000 7f841ad7c700 1 -- 172.21.15.89:0/337636206 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83f8066f30 msgr2=0x7f83f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.995+0000 7f841ad7c700 1 --2- 172.21.15.89:0/337636206 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83f8066f30 0x7f83f80693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8408006f00 tx=0x7f84080091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.995+0000 7f841ad7c700 1 -- 172.21.15.89:0/337636206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84140b2110 msgr2=0x7f84141565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.995+0000 7f841ad7c700 1 --2- 172.21.15.89:0/337636206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84140b2110 0x7f84141565a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f841406a890 tx=0x7f840c002170 comp rx=0 tx=0).stop 2023-10-10T20:35:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.995+0000 7f841ad7c700 1 -- 172.21.15.89:0/337636206 shutdown_connections 2023-10-10T20:35:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.995+0000 7f841ad7c700 1 --2- 172.21.15.89:0/337636206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84140b2110 0x7f84141565a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.995+0000 7f841ad7c700 1 --2- 172.21.15.89:0/337636206 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8404004810 0x7f84140b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.995+0000 7f841ad7c700 1 --2- 172.21.15.89:0/337636206 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83f8066f30 0x7f83f80693e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.995+0000 7f841ad7c700 1 --2- 172.21.15.89:0/337636206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84140cda70 0x7f84140b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.995+0000 7f841ad7c700 1 -- 172.21.15.89:0/337636206 >> 172.21.15.89:0/337636206 conn(0x7f8414068e20 msgr2=0x7f8414149a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:15.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.995+0000 7f841ad7c700 1 -- 172.21.15.89:0/337636206 shutdown_connections 2023-10-10T20:35:15.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:14.995+0000 7f841ad7c700 1 -- 172.21.15.89:0/337636206 wait complete. 2023-10-10T20:35:15.009 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:35:15.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.121+0000 7fc852bf5700 1 Processor -- start 2023-10-10T20:35:15.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.121+0000 7fc852bf5700 1 -- start start 2023-10-10T20:35:15.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.122+0000 7fc852bf5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc84c06c0a0 0x7fc84c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:15.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.122+0000 7fc852bf5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc84c15d7e0 con 0x7fc84c15df50 2023-10-10T20:35:15.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.122+0000 7fc852bf5700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc84c15dac0 con 0x7fc84c06bcd0 2023-10-10T20:35:15.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.122+0000 7fc852bf5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c15aa30 con 0x7fc84c06c0a0 2023-10-10T20:35:15.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.122+0000 7fc851192700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc84c15df50 0x7fc84c0c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40546/0 (socket says 172.21.15.89:40546) 2023-10-10T20:35:15.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.122+0000 7fc851192700 1 -- 172.21.15.89:0/1626742713 learned_addr learned my addr 172.21.15.89:0/1626742713 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:15.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.122+0000 7fc843fff700 1 --2- 172.21.15.89:0/1626742713 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc84c06c0a0 0x7fc84c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:15.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.123+0000 7fc8437fe700 1 -- 172.21.15.89:0/1626742713 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531389015 0 0) 0x7fc84c15d7e0 con 0x7fc84c15df50 2023-10-10T20:35:15.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.123+0000 7fc8437fe700 1 -- 172.21.15.89:0/1626742713 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc834003570 con 0x7fc84c15df50 2023-10-10T20:35:15.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.123+0000 7fc8437fe700 1 -- 172.21.15.89:0/1626742713 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4051746588 0 0) 0x7fc84c15dac0 con 0x7fc84c06bcd0 2023-10-10T20:35:15.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.123+0000 7fc8437fe700 1 -- 172.21.15.89:0/1626742713 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc84c15d7e0 con 0x7fc84c06bcd0 2023-10-10T20:35:15.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.123+0000 7fc843fff700 1 -- 172.21.15.89:0/1626742713 >> v1:172.21.15.111:6789/0 conn(0x7fc84c06bcd0 legacy=0x7fc84c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:15.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.123+0000 7fc8437fe700 1 -- 172.21.15.89:0/1626742713 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 55081685 0 0) 0x7fc834003570 con 0x7fc84c15df50 2023-10-10T20:35:15.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.123+0000 7fc843fff700 1 -- 172.21.15.89:0/1626742713 >> v1:172.21.15.89:6789/0 conn(0x7fc84c15df50 legacy=0x7fc84c0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:15.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.123+0000 7fc843fff700 1 -- 172.21.15.89:0/1626742713 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84c15ad10 con 0x7fc84c06c0a0 2023-10-10T20:35:15.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.123+0000 7fc843fff700 1 --2- 172.21.15.89:0/1626742713 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc84c06c0a0 0x7fc84c0caa90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc83c009910 tx=0x7fc83c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bc264847fa139ed server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:15.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.123+0000 7fc8437fe700 1 -- 172.21.15.89:0/1626742713 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc844001470 con 0x7fc84c06c0a0 2023-10-10T20:35:15.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.123+0000 7fc8437fe700 1 -- 172.21.15.89:0/1626742713 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83c011420 con 0x7fc84c06c0a0 2023-10-10T20:35:15.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.123+0000 7fc8437fe700 1 -- 172.21.15.89:0/1626742713 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c0117c0 con 0x7fc84c06c0a0 2023-10-10T20:35:15.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.124+0000 7fc852bf5700 1 -- 172.21.15.89:0/1626742713 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc84c06c0a0 msgr2=0x7fc84c0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:15.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.124+0000 7fc852bf5700 1 --2- 172.21.15.89:0/1626742713 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc84c06c0a0 0x7fc84c0caa90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc83c009910 tx=0x7fc83c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:15.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.124+0000 7fc852bf5700 1 -- 172.21.15.89:0/1626742713 shutdown_connections 2023-10-10T20:35:15.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.124+0000 7fc852bf5700 1 --2- 172.21.15.89:0/1626742713 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc84c06c0a0 0x7fc84c0caa90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:15.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.124+0000 7fc852bf5700 1 -- 172.21.15.89:0/1626742713 >> 172.21.15.89:0/1626742713 conn(0x7fc84c0ce600 msgr2=0x7fc84c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:15.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.124+0000 7fc852bf5700 1 -- 172.21.15.89:0/1626742713 shutdown_connections 2023-10-10T20:35:15.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.124+0000 7fc852bf5700 1 -- 172.21.15.89:0/1626742713 wait complete. 2023-10-10T20:35:15.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.125+0000 7fc852bf5700 1 Processor -- start 2023-10-10T20:35:15.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.125+0000 7fc852bf5700 1 -- start start 2023-10-10T20:35:15.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.126+0000 7fc852bf5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc84c06bcd0 0x7fc84c155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:15.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.126+0000 7fc852bf5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc84c06c0a0 0x7fc84c16ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:15.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.126+0000 7fc852bf5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc84c15df50 0x7fc84c0c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:15.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.126+0000 7fc852bf5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c0c8a10 con 0x7fc84c06c0a0 2023-10-10T20:35:15.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.126+0000 7fc852bf5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c0c7e00 con 0x7fc84c15df50 2023-10-10T20:35:15.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.126+0000 7fc852bf5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c0c80e0 con 0x7fc84c06bcd0 2023-10-10T20:35:15.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.126+0000 7fc843fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc84c06c0a0 0x7fc84c16ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:15.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.126+0000 7fc843fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc84c06c0a0 0x7fc84c16ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42776/0 (socket says 172.21.15.89:42776) 2023-10-10T20:35:15.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.126+0000 7fc843fff700 1 -- 172.21.15.89:0/1112335336 learned_addr learned my addr 172.21.15.89:0/1112335336 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:15.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.126+0000 7fc850991700 1 --2- 172.21.15.89:0/1112335336 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc84c06bcd0 0x7fc84c155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:15.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.126+0000 7fc851192700 1 --2- 172.21.15.89:0/1112335336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc84c15df50 0x7fc84c0c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:15.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.126+0000 7fc843fff700 1 -- 172.21.15.89:0/1112335336 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc84c06bcd0 msgr2=0x7fc84c155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:15.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.126+0000 7fc843fff700 1 --2- 172.21.15.89:0/1112335336 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc84c06bcd0 0x7fc84c155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:15.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.126+0000 7fc843fff700 1 -- 172.21.15.89:0/1112335336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc84c15df50 msgr2=0x7fc84c0c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:15.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.126+0000 7fc843fff700 1 --2- 172.21.15.89:0/1112335336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc84c15df50 0x7fc84c0c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:15.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.126+0000 7fc843fff700 1 -- 172.21.15.89:0/1112335336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83c009420 con 0x7fc84c06c0a0 2023-10-10T20:35:15.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.126+0000 7fc843fff700 1 --2- 172.21.15.89:0/1112335336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc84c06c0a0 0x7fc84c16ab30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc84c069220 tx=0x7fc83c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:15.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.126+0000 7fc840ff9700 1 -- 172.21.15.89:0/1112335336 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c000c40 con 0x7fc84c06c0a0 2023-10-10T20:35:15.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.127+0000 7fc840ff9700 1 -- 172.21.15.89:0/1112335336 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83c01caf0 con 0x7fc84c06c0a0 2023-10-10T20:35:15.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.127+0000 7fc852bf5700 1 -- 172.21.15.89:0/1112335336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84c0c83c0 con 0x7fc84c06c0a0 2023-10-10T20:35:15.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.127+0000 7fc840ff9700 1 -- 172.21.15.89:0/1112335336 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c01a450 con 0x7fc84c06c0a0 2023-10-10T20:35:15.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.127+0000 7fc852bf5700 1 -- 172.21.15.89:0/1112335336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84c16b070 con 0x7fc84c06c0a0 2023-10-10T20:35:15.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.127+0000 7fc852bf5700 1 -- 172.21.15.89:0/1112335336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84c04c3c0 con 0x7fc84c06c0a0 2023-10-10T20:35:15.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.128+0000 7fc840ff9700 1 -- 172.21.15.89:0/1112335336 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc83c01a840 con 0x7fc84c06c0a0 2023-10-10T20:35:15.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.128+0000 7fc840ff9700 1 --2- 172.21.15.89:0/1112335336 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc838066f30 0x7fc8380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:15.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.128+0000 7fc840ff9700 1 -- 172.21.15.89:0/1112335336 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc83c085b40 con 0x7fc84c06c0a0 2023-10-10T20:35:15.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.128+0000 7fc850991700 1 --2- 172.21.15.89:0/1112335336 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc838066f30 0x7fc8380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:15.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.129+0000 7fc850991700 1 --2- 172.21.15.89:0/1112335336 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc838066f30 0x7fc8380693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc8480040a0 tx=0x7fc848007f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:15.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.132+0000 7fc840ff9700 1 -- 172.21.15.89:0/1112335336 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc83c053060 con 0x7fc84c06c0a0 2023-10-10T20:35:15.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.320+0000 7fc852bf5700 1 -- 172.21.15.89:0/1112335336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc84c15aff0 con 0x7fc84c06c0a0 2023-10-10T20:35:15.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.320+0000 7fc840ff9700 1 -- 172.21.15.89:0/1112335336 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v27) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc83c029020 con 0x7fc84c06c0a0 2023-10-10T20:35:15.323 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:15.323 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:35:15.323 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:15.323 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2023-10-10T20:35:15.324 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2023-10-10T20:35:15.324 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:15.324 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:35:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.322+0000 7fc852bf5700 1 -- 172.21.15.89:0/1112335336 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc838066f30 msgr2=0x7fc8380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.322+0000 7fc852bf5700 1 --2- 172.21.15.89:0/1112335336 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc838066f30 0x7fc8380693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc8480040a0 tx=0x7fc848007f90 comp rx=0 tx=0).stop 2023-10-10T20:35:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.322+0000 7fc852bf5700 1 -- 172.21.15.89:0/1112335336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc84c06c0a0 msgr2=0x7fc84c16ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.322+0000 7fc852bf5700 1 --2- 172.21.15.89:0/1112335336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc84c06c0a0 0x7fc84c16ab30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc84c069220 tx=0x7fc83c016000 comp rx=0 tx=0).stop 2023-10-10T20:35:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.322+0000 7fc852bf5700 1 -- 172.21.15.89:0/1112335336 shutdown_connections 2023-10-10T20:35:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.322+0000 7fc852bf5700 1 --2- 172.21.15.89:0/1112335336 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc838066f30 0x7fc8380693e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.322+0000 7fc852bf5700 1 --2- 172.21.15.89:0/1112335336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc84c15df50 0x7fc84c0c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.322+0000 7fc852bf5700 1 --2- 172.21.15.89:0/1112335336 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc84c06bcd0 0x7fc84c155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.322+0000 7fc852bf5700 1 --2- 172.21.15.89:0/1112335336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc84c06c0a0 0x7fc84c16ab30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.322+0000 7fc852bf5700 1 -- 172.21.15.89:0/1112335336 >> 172.21.15.89:0/1112335336 conn(0x7fc84c0ce600 msgr2=0x7fc84c0cc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.322+0000 7fc852bf5700 1 -- 172.21.15.89:0/1112335336 shutdown_connections 2023-10-10T20:35:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.322+0000 7fc852bf5700 1 -- 172.21.15.89:0/1112335336 wait complete. 2023-10-10T20:35:15.336 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:35:15.336 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:35:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.452+0000 7f46cf59e700 1 Processor -- start 2023-10-10T20:35:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.452+0000 7f46cf59e700 1 -- start start 2023-10-10T20:35:15.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.453+0000 7f46cf59e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f46d00b03c0 0x7f46d00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:15.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.453+0000 7f46cf59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46d01591e0 0x7f46d0159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:15.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.453+0000 7f46cf59e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d00cfa00 con 0x7f46d00b0790 2023-10-10T20:35:15.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.453+0000 7f46cf59e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d00cb580 con 0x7f46d00b03c0 2023-10-10T20:35:15.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.453+0000 7f46cf59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d00cf160 con 0x7f46d01591e0 2023-10-10T20:35:15.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.453+0000 7f46ce59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f46d00b03c0 0x7f46d00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:15.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.453+0000 7f46ce59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f46d00b03c0 0x7f46d00cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42778/0 (socket says 172.21.15.89:42778) 2023-10-10T20:35:15.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.453+0000 7f46ce59c700 1 -- 172.21.15.89:0/1600805227 learned_addr learned my addr 172.21.15.89:0/1600805227 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:15.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.453+0000 7f46ced9d700 1 --2- 172.21.15.89:0/1600805227 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46d01591e0 0x7f46d0159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:15.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.453+0000 7f46ce59c700 1 -- 172.21.15.89:0/1600805227 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46d01591e0 msgr2=0x7f46d0159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:15.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.453+0000 7f46ce59c700 1 --2- 172.21.15.89:0/1600805227 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46d01591e0 0x7f46d0159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:15.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.453+0000 7f46ce59c700 1 -- 172.21.15.89:0/1600805227 >> v1:172.21.15.111:6789/0 conn(0x7f46d00b0790 legacy=0x7f46d00b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:15.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.453+0000 7f46ce59c700 1 -- 172.21.15.89:0/1600805227 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d00cf440 con 0x7f46d00b03c0 2023-10-10T20:35:15.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.453+0000 7f46ce59c700 1 --2- 172.21.15.89:0/1600805227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d00b03c0 0x7f46d00cea70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f46c00099e0 tx=0x7f46c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485c08b48310327f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:15.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.453+0000 7f46cd59a700 1 -- 172.21.15.89:0/1600805227 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c000b820 con 0x7f46d00b03c0 2023-10-10T20:35:15.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.454+0000 7f46cd59a700 1 -- 172.21.15.89:0/1600805227 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c000b980 con 0x7f46d00b03c0 2023-10-10T20:35:15.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.454+0000 7f46cd59a700 1 -- 172.21.15.89:0/1600805227 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c000bd20 con 0x7f46d00b03c0 2023-10-10T20:35:15.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.454+0000 7f46cf59e700 1 -- 172.21.15.89:0/1600805227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d00b03c0 msgr2=0x7f46d00cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:15.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.454+0000 7f46cf59e700 1 --2- 172.21.15.89:0/1600805227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d00b03c0 0x7f46d00cea70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f46c00099e0 tx=0x7f46c0009250 comp rx=0 tx=0).stop 2023-10-10T20:35:15.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.454+0000 7f46cf59e700 1 -- 172.21.15.89:0/1600805227 shutdown_connections 2023-10-10T20:35:15.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.454+0000 7f46cf59e700 1 --2- 172.21.15.89:0/1600805227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d00b03c0 0x7f46d00cea70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:15.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.454+0000 7f46cf59e700 1 --2- 172.21.15.89:0/1600805227 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46d01591e0 0x7f46d0159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:15.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.454+0000 7f46cf59e700 1 -- 172.21.15.89:0/1600805227 >> 172.21.15.89:0/1600805227 conn(0x7f46d0063230 msgr2=0x7f46d0063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.454+0000 7f46cf59e700 1 -- 172.21.15.89:0/1600805227 shutdown_connections 2023-10-10T20:35:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.454+0000 7f46cf59e700 1 -- 172.21.15.89:0/1600805227 wait complete. 2023-10-10T20:35:15.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.455+0000 7f46cf59e700 1 Processor -- start 2023-10-10T20:35:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.455+0000 7f46cf59e700 1 -- start start 2023-10-10T20:35:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.456+0000 7f46cf59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d00b03c0 0x7f46d0148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.456+0000 7f46cf59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46d00b0790 0x7f46d00c5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.456+0000 7f46cf59e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46d01591e0 0x7f46d00c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.456+0000 7f46cf59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d015cab0 con 0x7f46d00b03c0 2023-10-10T20:35:15.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.456+0000 7f46cf59e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d0159970 con 0x7f46d01591e0 2023-10-10T20:35:15.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.456+0000 7f46cf59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d016ab00 con 0x7f46d00b0790 2023-10-10T20:35:15.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.456+0000 7f46ce59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d00b03c0 0x7f46d0148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:15.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.456+0000 7f46ce59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d00b03c0 0x7f46d0148920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42786/0 (socket says 172.21.15.89:42786) 2023-10-10T20:35:15.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.456+0000 7f46ce59c700 1 -- 172.21.15.89:0/111695645 learned_addr learned my addr 172.21.15.89:0/111695645 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:15.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.456+0000 7f46ced9d700 1 --2- 172.21.15.89:0/111695645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46d01591e0 0x7f46d00c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:15.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.456+0000 7f46cdd9b700 1 --2- 172.21.15.89:0/111695645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46d00b0790 0x7f46d00c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:15.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.456+0000 7f46ce59c700 1 -- 172.21.15.89:0/111695645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46d00b0790 msgr2=0x7f46d00c5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:15.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.456+0000 7f46ce59c700 1 --2- 172.21.15.89:0/111695645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46d00b0790 0x7f46d00c5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:15.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.456+0000 7f46ce59c700 1 -- 172.21.15.89:0/111695645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46d01591e0 msgr2=0x7f46d00c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:15.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.456+0000 7f46ce59c700 1 --2- 172.21.15.89:0/111695645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46d01591e0 0x7f46d00c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:15.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.456+0000 7f46ce59c700 1 -- 172.21.15.89:0/111695645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c00094f0 con 0x7f46d00b03c0 2023-10-10T20:35:15.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.456+0000 7f46ce59c700 1 --2- 172.21.15.89:0/111695645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d00b03c0 0x7f46d0148920 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f46d00cfe40 tx=0x7f46c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:15.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.457+0000 7f46be7fc700 1 -- 172.21.15.89:0/111695645 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0011d10 con 0x7f46d00b03c0 2023-10-10T20:35:15.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.457+0000 7f46be7fc700 1 -- 172.21.15.89:0/111695645 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c0011e70 con 0x7f46d00b03c0 2023-10-10T20:35:15.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.457+0000 7f46cf59e700 1 -- 172.21.15.89:0/111695645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d016a820 con 0x7f46d00b03c0 2023-10-10T20:35:15.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.457+0000 7f46be7fc700 1 -- 172.21.15.89:0/111695645 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0024640 con 0x7f46d00b03c0 2023-10-10T20:35:15.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.457+0000 7f46cf59e700 1 -- 172.21.15.89:0/111695645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d015c740 con 0x7f46d00b03c0 2023-10-10T20:35:15.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.457+0000 7f46cf59e700 1 -- 172.21.15.89:0/111695645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d015c2c0 con 0x7f46d00b03c0 2023-10-10T20:35:15.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.458+0000 7f46be7fc700 1 -- 172.21.15.89:0/111695645 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f46c001a420 con 0x7f46d00b03c0 2023-10-10T20:35:15.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.458+0000 7f46be7fc700 1 --2- 172.21.15.89:0/111695645 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f46b0066f30 0x7f46b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:15.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.458+0000 7f46cdd9b700 1 --2- 172.21.15.89:0/111695645 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f46b0066f30 0x7f46b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:15.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.458+0000 7f46be7fc700 1 -- 172.21.15.89:0/111695645 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f46c0085440 con 0x7f46d00b03c0 2023-10-10T20:35:15.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.459+0000 7f46cdd9b700 1 --2- 172.21.15.89:0/111695645 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f46b0066f30 0x7f46b00693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f46d00c53b0 tx=0x7f46c4007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:15.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.463+0000 7f46be7fc700 1 -- 172.21.15.89:0/111695645 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46d015c2c0 con 0x7f46d00b03c0 2023-10-10T20:35:15.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:15.649+0000 7f46cf59e700 1 -- 172.21.15.89:0/111695645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f46d015c2c0 con 0x7f46d00b03c0 2023-10-10T20:35:16.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.504+0000 7f46be7fc700 1 -- 172.21.15.89:0/111695645 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v28) v1 ==== 138+0+0 (secure 0 0 0) 0x7f46d015c2c0 con 0x7f46d00b03c0 2023-10-10T20:35:16.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.507+0000 7f46cf59e700 1 -- 172.21.15.89:0/111695645 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f46b0066f30 msgr2=0x7f46b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:16.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.507+0000 7f46cf59e700 1 --2- 172.21.15.89:0/111695645 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f46b0066f30 0x7f46b00693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f46d00c53b0 tx=0x7f46c4007ec0 comp rx=0 tx=0).stop 2023-10-10T20:35:16.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.507+0000 7f46cf59e700 1 -- 172.21.15.89:0/111695645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d00b03c0 msgr2=0x7f46d0148920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:16.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.507+0000 7f46cf59e700 1 --2- 172.21.15.89:0/111695645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d00b03c0 0x7f46d0148920 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f46d00cfe40 tx=0x7f46c0017000 comp rx=0 tx=0).stop 2023-10-10T20:35:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.508+0000 7f46cf59e700 1 -- 172.21.15.89:0/111695645 shutdown_connections 2023-10-10T20:35:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.508+0000 7f46cf59e700 1 --2- 172.21.15.89:0/111695645 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f46b0066f30 0x7f46b00693e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.508+0000 7f46cf59e700 1 --2- 172.21.15.89:0/111695645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46d01591e0 0x7f46d00c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.508+0000 7f46cf59e700 1 --2- 172.21.15.89:0/111695645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d00b03c0 0x7f46d0148920 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.508+0000 7f46cf59e700 1 --2- 172.21.15.89:0/111695645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46d00b0790 0x7f46d00c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.508+0000 7f46cf59e700 1 -- 172.21.15.89:0/111695645 >> 172.21.15.89:0/111695645 conn(0x7f46d0063230 msgr2=0x7f46d00b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.508+0000 7f46cf59e700 1 -- 172.21.15.89:0/111695645 shutdown_connections 2023-10-10T20:35:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.508+0000 7f46cf59e700 1 -- 172.21.15.89:0/111695645 wait complete. 2023-10-10T20:35:16.513 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2023-10-10T20:35:16.525 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:35:16.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.647+0000 7fc605dd3700 1 Processor -- start 2023-10-10T20:35:16.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.647+0000 7fc605dd3700 1 -- start start 2023-10-10T20:35:16.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.648+0000 7fc605dd3700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc600069700 0x7fc6000cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:16.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.648+0000 7fc605dd3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc6001560b0 0x7fc60006a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:16.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.648+0000 7fc605dd3700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6000cc1f0 con 0x7fc600155ce0 2023-10-10T20:35:16.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.648+0000 7fc605dd3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc600068830 con 0x7fc6001560b0 2023-10-10T20:35:16.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.648+0000 7fc605dd3700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6000cbc30 con 0x7fc600069700 2023-10-10T20:35:16.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.648+0000 7fc5fffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc6001560b0 0x7fc60006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:16.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.648+0000 7fc5fffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc6001560b0 0x7fc60006a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42790/0 (socket says 172.21.15.89:42790) 2023-10-10T20:35:16.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.648+0000 7fc5fffff700 1 -- 172.21.15.89:0/4279950860 learned_addr learned my addr 172.21.15.89:0/4279950860 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:16.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.649+0000 7fc5feffd700 1 --2- 172.21.15.89:0/4279950860 >> v2:172.21.15.111:3300/0 conn(0x7fc600069700 0x7fc6000cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:16.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.649+0000 7fc5fffff700 1 -- 172.21.15.89:0/4279950860 >> v2:172.21.15.111:3300/0 conn(0x7fc600069700 msgr2=0x7fc6000cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:16.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.649+0000 7fc5fffff700 1 --2- 172.21.15.89:0/4279950860 >> v2:172.21.15.111:3300/0 conn(0x7fc600069700 0x7fc6000cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:16.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.649+0000 7fc5fffff700 1 -- 172.21.15.89:0/4279950860 >> v1:172.21.15.111:6789/0 conn(0x7fc600155ce0 legacy=0x7fc600068cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:16.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.649+0000 7fc5fffff700 1 -- 172.21.15.89:0/4279950860 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6000cbf10 con 0x7fc6001560b0 2023-10-10T20:35:16.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.649+0000 7fc5fffff700 1 --2- 172.21.15.89:0/4279950860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6001560b0 0x7fc60006a320 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc5f80099b0 tx=0x7fc5f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86ab3a748c91d37 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:16.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.649+0000 7fc5fe7fc700 1 -- 172.21.15.89:0/4279950860 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f800b7e0 con 0x7fc6001560b0 2023-10-10T20:35:16.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.649+0000 7fc5fe7fc700 1 -- 172.21.15.89:0/4279950860 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f800b940 con 0x7fc6001560b0 2023-10-10T20:35:16.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.649+0000 7fc5fe7fc700 1 -- 172.21.15.89:0/4279950860 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f800bd00 con 0x7fc6001560b0 2023-10-10T20:35:16.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.649+0000 7fc605dd3700 1 -- 172.21.15.89:0/4279950860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6001560b0 msgr2=0x7fc60006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:16.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.649+0000 7fc605dd3700 1 --2- 172.21.15.89:0/4279950860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6001560b0 0x7fc60006a320 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc5f80099b0 tx=0x7fc5f8009250 comp rx=0 tx=0).stop 2023-10-10T20:35:16.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.649+0000 7fc605dd3700 1 -- 172.21.15.89:0/4279950860 shutdown_connections 2023-10-10T20:35:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.649+0000 7fc605dd3700 1 --2- 172.21.15.89:0/4279950860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6001560b0 0x7fc60006a320 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.650+0000 7fc605dd3700 1 --2- 172.21.15.89:0/4279950860 >> v2:172.21.15.111:3300/0 conn(0x7fc600069700 0x7fc6000cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.650+0000 7fc605dd3700 1 -- 172.21.15.89:0/4279950860 >> 172.21.15.89:0/4279950860 conn(0x7fc6000c9870 msgr2=0x7fc6000c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.650+0000 7fc605dd3700 1 -- 172.21.15.89:0/4279950860 shutdown_connections 2023-10-10T20:35:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.650+0000 7fc605dd3700 1 -- 172.21.15.89:0/4279950860 wait complete. 2023-10-10T20:35:16.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.651+0000 7fc605dd3700 1 Processor -- start 2023-10-10T20:35:16.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.651+0000 7fc605dd3700 1 -- start start 2023-10-10T20:35:16.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.651+0000 7fc605dd3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc600069700 0x7fc600158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:16.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.651+0000 7fc605dd3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc600155ce0 0x7fc600157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:16.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc605dd3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6001560b0 0x7fc600153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:16.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc605dd3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60014da80 con 0x7fc600069700 2023-10-10T20:35:16.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc605dd3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60014d0b0 con 0x7fc6001560b0 2023-10-10T20:35:16.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc605dd3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60014d350 con 0x7fc600155ce0 2023-10-10T20:35:16.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc5ff7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc600069700 0x7fc600158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:16.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc5ff7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc600069700 0x7fc600158ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42800/0 (socket says 172.21.15.89:42800) 2023-10-10T20:35:16.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc5ff7fe700 1 -- 172.21.15.89:0/606435605 learned_addr learned my addr 172.21.15.89:0/606435605 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:16.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc5fffff700 1 --2- 172.21.15.89:0/606435605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6001560b0 0x7fc600153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:16.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc5feffd700 1 --2- 172.21.15.89:0/606435605 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc600155ce0 0x7fc600157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:16.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc5ff7fe700 1 -- 172.21.15.89:0/606435605 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc600155ce0 msgr2=0x7fc600157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:16.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc5ff7fe700 1 --2- 172.21.15.89:0/606435605 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc600155ce0 0x7fc600157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:16.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc5ff7fe700 1 -- 172.21.15.89:0/606435605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6001560b0 msgr2=0x7fc600153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:16.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc5ff7fe700 1 --2- 172.21.15.89:0/606435605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6001560b0 0x7fc600153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:16.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc5ff7fe700 1 -- 172.21.15.89:0/606435605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f80094f0 con 0x7fc600069700 2023-10-10T20:35:16.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc5fffff700 1 --2- 172.21.15.89:0/606435605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6001560b0 0x7fc600153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:16.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc5ff7fe700 1 --2- 172.21.15.89:0/606435605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc600069700 0x7fc600158ef0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000c950 tx=0x7fc5f0002760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:16.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc5e77fe700 1 -- 172.21.15.89:0/606435605 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f000a7e0 con 0x7fc600069700 2023-10-10T20:35:16.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc605dd3700 1 -- 172.21.15.89:0/606435605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60014d630 con 0x7fc600069700 2023-10-10T20:35:16.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc5e77fe700 1 -- 172.21.15.89:0/606435605 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f000a960 con 0x7fc600069700 2023-10-10T20:35:16.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc5e77fe700 1 -- 172.21.15.89:0/606435605 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f000ad20 con 0x7fc600069700 2023-10-10T20:35:16.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.652+0000 7fc605dd3700 1 -- 172.21.15.89:0/606435605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc600157750 con 0x7fc600069700 2023-10-10T20:35:16.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.653+0000 7fc605dd3700 1 -- 172.21.15.89:0/606435605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc600069210 con 0x7fc600069700 2023-10-10T20:35:16.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.654+0000 7fc5e77fe700 1 -- 172.21.15.89:0/606435605 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc5f0012630 con 0x7fc600069700 2023-10-10T20:35:16.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.654+0000 7fc5e77fe700 1 --2- 172.21.15.89:0/606435605 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc5ec066f30 0x7fc5ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:16.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.654+0000 7fc5e77fe700 1 -- 172.21.15.89:0/606435605 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fc5f0084880 con 0x7fc600069700 2023-10-10T20:35:16.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.654+0000 7fc5feffd700 1 --2- 172.21.15.89:0/606435605 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc5ec066f30 0x7fc5ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:16.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.654+0000 7fc5feffd700 1 --2- 172.21.15.89:0/606435605 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc5ec066f30 0x7fc5ec0693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc600166f40 tx=0x7fc5f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:16.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.658+0000 7fc5e77fe700 1 -- 172.21.15.89:0/606435605 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc5f0051d80 con 0x7fc600069700 2023-10-10T20:35:16.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:16.842+0000 7fc605dd3700 1 -- 172.21.15.89:0/606435605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc60014def0 con 0x7fc600069700 2023-10-10T20:35:17.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.515+0000 7fc5e77fe700 1 -- 172.21.15.89:0/606435605 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v29) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc5f0016070 con 0x7fc600069700 2023-10-10T20:35:17.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.523+0000 7fc605dd3700 1 -- 172.21.15.89:0/606435605 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc5ec066f30 msgr2=0x7fc5ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:17.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.523+0000 7fc605dd3700 1 --2- 172.21.15.89:0/606435605 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc5ec066f30 0x7fc5ec0693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc600166f40 tx=0x7fc5f4008040 comp rx=0 tx=0).stop 2023-10-10T20:35:17.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.523+0000 7fc605dd3700 1 -- 172.21.15.89:0/606435605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc600069700 msgr2=0x7fc600158ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:17.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.523+0000 7fc605dd3700 1 --2- 172.21.15.89:0/606435605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc600069700 0x7fc600158ef0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000c950 tx=0x7fc5f0002760 comp rx=0 tx=0).stop 2023-10-10T20:35:17.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.524+0000 7fc605dd3700 1 -- 172.21.15.89:0/606435605 shutdown_connections 2023-10-10T20:35:17.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.524+0000 7fc605dd3700 1 --2- 172.21.15.89:0/606435605 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc5ec066f30 0x7fc5ec0693e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:17.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.524+0000 7fc605dd3700 1 --2- 172.21.15.89:0/606435605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6001560b0 0x7fc600153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:17.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.524+0000 7fc605dd3700 1 --2- 172.21.15.89:0/606435605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc600069700 0x7fc600158ef0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:17.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.524+0000 7fc605dd3700 1 --2- 172.21.15.89:0/606435605 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc600155ce0 0x7fc600157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:17.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.524+0000 7fc605dd3700 1 -- 172.21.15.89:0/606435605 >> 172.21.15.89:0/606435605 conn(0x7fc6000c9870 msgr2=0x7fc600150c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:17.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.525+0000 7fc605dd3700 1 -- 172.21.15.89:0/606435605 shutdown_connections 2023-10-10T20:35:17.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.525+0000 7fc605dd3700 1 -- 172.21.15.89:0/606435605 wait complete. 2023-10-10T20:35:17.529 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2023-10-10T20:35:17.544 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:35:17.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.661+0000 7f5dfa327700 1 Processor -- start 2023-10-10T20:35:17.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.661+0000 7f5dfa327700 1 -- start start 2023-10-10T20:35:17.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.662+0000 7f5dfa327700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5df4159a60 0x7f5df40c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:17.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.662+0000 7f5dfa327700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df40c9cb0 con 0x7f5df40cb230 2023-10-10T20:35:17.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.662+0000 7f5dfa327700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df40c9f30 con 0x7f5df40ccc00 2023-10-10T20:35:17.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.662+0000 7f5dfa327700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df406dfc0 con 0x7f5df4159a60 2023-10-10T20:35:17.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.662+0000 7f5df9b26700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5df40cb230 0x7f5df406d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40556/0 (socket says 172.21.15.89:40556) 2023-10-10T20:35:17.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.662+0000 7f5df9b26700 1 -- 172.21.15.89:0/4100516369 learned_addr learned my addr 172.21.15.89:0/4100516369 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:17.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5df9325700 1 --2- 172.21.15.89:0/4100516369 >> v2:172.21.15.111:3300/0 conn(0x7f5df4159a60 0x7f5df40c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:17.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5debfff700 1 -- 172.21.15.89:0/4100516369 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457358677 0 0) 0x7f5df40c9cb0 con 0x7f5df40cb230 2023-10-10T20:35:17.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5debfff700 1 -- 172.21.15.89:0/4100516369 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5de0003570 con 0x7f5df40cb230 2023-10-10T20:35:17.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5debfff700 1 -- 172.21.15.89:0/4100516369 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2420926100 0 0) 0x7f5de0003570 con 0x7f5df40cb230 2023-10-10T20:35:17.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5debfff700 1 -- 172.21.15.89:0/4100516369 >> v2:172.21.15.111:3300/0 conn(0x7f5df4159a60 msgr2=0x7f5df40c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:17.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5debfff700 1 --2- 172.21.15.89:0/4100516369 >> v2:172.21.15.111:3300/0 conn(0x7f5df4159a60 0x7f5df40c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:17.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5debfff700 1 -- 172.21.15.89:0/4100516369 >> v1:172.21.15.111:6789/0 conn(0x7f5df40ccc00 legacy=0x7f5df4069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:17.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5debfff700 1 -- 172.21.15.89:0/4100516369 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df40ca470 con 0x7f5df40cb230 2023-10-10T20:35:17.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5debfff700 1 -- 172.21.15.89:0/4100516369 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f5dec001470 con 0x7f5df40cb230 2023-10-10T20:35:17.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5debfff700 1 -- 172.21.15.89:0/4100516369 >> v1:172.21.15.89:6789/0 conn(0x7f5df40cb230 legacy=0x7f5df406d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:17.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5debfff700 1 --2- 172.21.15.89:0/4100516369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5de0003a60 0x7f5de0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:17.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5debfff700 1 --2- 172.21.15.89:0/4100516369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5de0004810 0x7f5de0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:17.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5debfff700 1 --2- 172.21.15.89:0/4100516369 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5de00051c0 0x7f5de000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:17.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5debfff700 1 -- 172.21.15.89:0/4100516369 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df406dfc0 con 0x7f5de0003a60 2023-10-10T20:35:17.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5debfff700 1 -- 172.21.15.89:0/4100516369 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de000c0d0 con 0x7f5de0004810 2023-10-10T20:35:17.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5debfff700 1 -- 172.21.15.89:0/4100516369 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de000c290 con 0x7f5de00051c0 2023-10-10T20:35:17.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5df9325700 1 --2- 172.21.15.89:0/4100516369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5de0003a60 0x7f5de0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:17.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5df8b24700 1 --2- 172.21.15.89:0/4100516369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5de0004810 0x7f5de0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:17.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5df9b26700 1 --2- 172.21.15.89:0/4100516369 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5de00051c0 0x7f5de000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:17.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5df9325700 1 -- 172.21.15.89:0/4100516369 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5de00051c0 msgr2=0x7f5de000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:17.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5df9325700 1 --2- 172.21.15.89:0/4100516369 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5de00051c0 0x7f5de000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:17.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5df9325700 1 -- 172.21.15.89:0/4100516369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5de0004810 msgr2=0x7f5de0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:17.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5df9325700 1 --2- 172.21.15.89:0/4100516369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5de0004810 0x7f5de0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:17.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5df9b26700 1 -- 172.21.15.89:0/4100516369 reap_dead start 2023-10-10T20:35:17.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.663+0000 7f5df9325700 1 -- 172.21.15.89:0/4100516369 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de0003570 con 0x7f5de0003a60 2023-10-10T20:35:17.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.664+0000 7f5df9325700 1 --2- 172.21.15.89:0/4100516369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5de0003a60 0x7f5de0003e70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5df000e210 tx=0x7f5df000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41a6c18f4f849b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:17.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.664+0000 7f5debfff700 1 -- 172.21.15.89:0/4100516369 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df000eac0 con 0x7f5de0003a60 2023-10-10T20:35:17.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.664+0000 7f5debfff700 1 -- 172.21.15.89:0/4100516369 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df000f040 con 0x7f5de0003a60 2023-10-10T20:35:17.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.664+0000 7f5debfff700 1 -- 172.21.15.89:0/4100516369 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df000ee50 con 0x7f5de0003a60 2023-10-10T20:35:17.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.664+0000 7f5dfa327700 1 -- 172.21.15.89:0/4100516369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5de0003a60 msgr2=0x7f5de0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:17.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.664+0000 7f5dfa327700 1 --2- 172.21.15.89:0/4100516369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5de0003a60 0x7f5de0003e70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5df000e210 tx=0x7f5df000b040 comp rx=0 tx=0).stop 2023-10-10T20:35:17.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.664+0000 7f5dfa327700 1 -- 172.21.15.89:0/4100516369 shutdown_connections 2023-10-10T20:35:17.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.664+0000 7f5dfa327700 1 --2- 172.21.15.89:0/4100516369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5de0003a60 0x7f5de0003e70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:17.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.664+0000 7f5dfa327700 1 -- 172.21.15.89:0/4100516369 >> 172.21.15.89:0/4100516369 conn(0x7f5df406baa0 msgr2=0x7f5df406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:17.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.664+0000 7f5dfa327700 1 -- 172.21.15.89:0/4100516369 shutdown_connections 2023-10-10T20:35:17.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.664+0000 7f5dfa327700 1 -- 172.21.15.89:0/4100516369 wait complete. 2023-10-10T20:35:17.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.665+0000 7f5dfa327700 1 Processor -- start 2023-10-10T20:35:17.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.665+0000 7f5dfa327700 1 -- start start 2023-10-10T20:35:17.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.666+0000 7f5dfa327700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5de0003a60 0x7f5df40b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:17.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.666+0000 7f5dfa327700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5df4151f10 0x7f5df4144f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:17.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.666+0000 7f5dfa327700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df415c290 0x7f5df415fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:17.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.666+0000 7f5dfa327700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df4156200 con 0x7f5df4151f10 2023-10-10T20:35:17.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.666+0000 7f5dfa327700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df41564e0 con 0x7f5df415c290 2023-10-10T20:35:17.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.666+0000 7f5dfa327700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df41567c0 con 0x7f5de0003a60 2023-10-10T20:35:17.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.666+0000 7f5df8b24700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5df4151f10 0x7f5df4144f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:17.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.666+0000 7f5df8b24700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5df4151f10 0x7f5df4144f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42806/0 (socket says 172.21.15.89:42806) 2023-10-10T20:35:17.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.666+0000 7f5df8b24700 1 -- 172.21.15.89:0/1782782792 learned_addr learned my addr 172.21.15.89:0/1782782792 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:17.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.667+0000 7f5df9b26700 1 --2- 172.21.15.89:0/1782782792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df415c290 0x7f5df415fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:17.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.667+0000 7f5df9325700 1 --2- 172.21.15.89:0/1782782792 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5de0003a60 0x7f5df40b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:17.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.667+0000 7f5df8b24700 1 -- 172.21.15.89:0/1782782792 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5de0003a60 msgr2=0x7f5df40b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:17.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.667+0000 7f5df8b24700 1 --2- 172.21.15.89:0/1782782792 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5de0003a60 0x7f5df40b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:17.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.667+0000 7f5df8b24700 1 -- 172.21.15.89:0/1782782792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df415c290 msgr2=0x7f5df415fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:17.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.667+0000 7f5df8b24700 1 --2- 172.21.15.89:0/1782782792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df415c290 0x7f5df415fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:17.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.667+0000 7f5df8b24700 1 -- 172.21.15.89:0/1782782792 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df4156aa0 con 0x7f5df4151f10 2023-10-10T20:35:17.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.667+0000 7f5df8b24700 1 --2- 172.21.15.89:0/1782782792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5df4151f10 0x7f5df4144f60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5de4002920 tx=0x7f5de4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:17.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.667+0000 7f5de97fa700 1 -- 172.21.15.89:0/1782782792 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de4011590 con 0x7f5df4151f10 2023-10-10T20:35:17.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.667+0000 7f5de97fa700 1 -- 172.21.15.89:0/1782782792 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de40116f0 con 0x7f5df4151f10 2023-10-10T20:35:17.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.667+0000 7f5dfa327700 1 -- 172.21.15.89:0/1782782792 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df4156d80 con 0x7f5df4151f10 2023-10-10T20:35:17.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.667+0000 7f5de97fa700 1 -- 172.21.15.89:0/1782782792 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de4011a90 con 0x7f5df4151f10 2023-10-10T20:35:17.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.667+0000 7f5dfa327700 1 -- 172.21.15.89:0/1782782792 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df41454a0 con 0x7f5df4151f10 2023-10-10T20:35:17.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.668+0000 7f5dfa327700 1 -- 172.21.15.89:0/1782782792 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df4157220 con 0x7f5df4151f10 2023-10-10T20:35:17.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.668+0000 7f5de97fa700 1 -- 172.21.15.89:0/1782782792 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5de401d460 con 0x7f5df4151f10 2023-10-10T20:35:17.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.669+0000 7f5de97fa700 1 --2- 172.21.15.89:0/1782782792 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5dd8066f30 0x7f5dd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:17.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.669+0000 7f5df9325700 1 --2- 172.21.15.89:0/1782782792 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5dd8066f30 0x7f5dd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:17.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.669+0000 7f5de97fa700 1 -- 172.21.15.89:0/1782782792 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f5de4084620 con 0x7f5df4151f10 2023-10-10T20:35:17.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.669+0000 7f5df9325700 1 --2- 172.21.15.89:0/1782782792 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5dd8066f30 0x7f5dd80693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5df000afd0 tx=0x7f5df001a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:17.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.673+0000 7f5de97fa700 1 -- 172.21.15.89:0/1782782792 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5de4051850 con 0x7f5df4151f10 2023-10-10T20:35:17.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:17.898+0000 7f5dfa327700 1 -- 172.21.15.89:0/1782782792 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5df41573d0 con 0x7f5df4151f10 2023-10-10T20:35:18.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.517+0000 7f5de97fa700 1 -- 172.21.15.89:0/1782782792 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5de4054e70 con 0x7f5df4151f10 2023-10-10T20:35:18.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.520+0000 7f5dfa327700 1 -- 172.21.15.89:0/1782782792 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5dd8066f30 msgr2=0x7f5dd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:18.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.520+0000 7f5dfa327700 1 --2- 172.21.15.89:0/1782782792 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5dd8066f30 0x7f5dd80693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5df000afd0 tx=0x7f5df001a000 comp rx=0 tx=0).stop 2023-10-10T20:35:18.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.520+0000 7f5dfa327700 1 -- 172.21.15.89:0/1782782792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5df4151f10 msgr2=0x7f5df4144f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:18.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.520+0000 7f5dfa327700 1 --2- 172.21.15.89:0/1782782792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5df4151f10 0x7f5df4144f60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5de4002920 tx=0x7f5de4007d30 comp rx=0 tx=0).stop 2023-10-10T20:35:18.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.520+0000 7f5dfa327700 1 -- 172.21.15.89:0/1782782792 shutdown_connections 2023-10-10T20:35:18.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.520+0000 7f5dfa327700 1 --2- 172.21.15.89:0/1782782792 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5dd8066f30 0x7f5dd80693e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:18.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.520+0000 7f5dfa327700 1 --2- 172.21.15.89:0/1782782792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df415c290 0x7f5df415fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:18.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.520+0000 7f5dfa327700 1 --2- 172.21.15.89:0/1782782792 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5de0003a60 0x7f5df40b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:18.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.520+0000 7f5dfa327700 1 --2- 172.21.15.89:0/1782782792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5df4151f10 0x7f5df4144f60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:18.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.520+0000 7f5dfa327700 1 -- 172.21.15.89:0/1782782792 >> 172.21.15.89:0/1782782792 conn(0x7f5df406baa0 msgr2=0x7f5df4155140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:18.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.522+0000 7f5dfa327700 1 -- 172.21.15.89:0/1782782792 shutdown_connections 2023-10-10T20:35:18.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.523+0000 7f5dfa327700 1 -- 172.21.15.89:0/1782782792 wait complete. 2023-10-10T20:35:18.527 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 4 and data pool 5 2023-10-10T20:35:18.549 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:35:18.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.662+0000 7fb407e17700 1 Processor -- start 2023-10-10T20:35:18.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.663+0000 7fb407e17700 1 -- start start 2023-10-10T20:35:18.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.663+0000 7fb407e17700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb400159890 0x7fb40015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:18.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.663+0000 7fb407e17700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb400159c60 0x7fb40015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:18.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.663+0000 7fb407e17700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb40015a160 con 0x7fb40015db10 2023-10-10T20:35:18.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.663+0000 7fb407e17700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40015df50 con 0x7fb400159c60 2023-10-10T20:35:18.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.663+0000 7fb407e17700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4000d13a0 con 0x7fb400159890 2023-10-10T20:35:18.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.664+0000 7fb4053b2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb400159c60 0x7fb40015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:18.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.664+0000 7fb4063b4700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb40015db10 0x7fb400069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47338/0 (socket says 172.21.15.89:47338) 2023-10-10T20:35:18.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.664+0000 7fb4053b2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb400159c60 0x7fb40015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46024/0 (socket says 172.21.15.89:46024) 2023-10-10T20:35:18.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.664+0000 7fb4063b4700 1 -- 172.21.15.89:0/398856526 learned_addr learned my addr 172.21.15.89:0/398856526 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:18.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.664+0000 7fb405bb3700 1 --2- 172.21.15.89:0/398856526 >> v2:172.21.15.111:3300/0 conn(0x7fb400159890 0x7fb40015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:18.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.664+0000 7fb4053b2700 1 -- 172.21.15.89:0/398856526 >> v2:172.21.15.111:3300/0 conn(0x7fb400159890 msgr2=0x7fb40015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:18.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.664+0000 7fb4053b2700 1 --2- 172.21.15.89:0/398856526 >> v2:172.21.15.111:3300/0 conn(0x7fb400159890 0x7fb40015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:18.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.664+0000 7fb4053b2700 1 -- 172.21.15.89:0/398856526 >> v1:172.21.15.89:6789/0 conn(0x7fb40015db10 legacy=0x7fb400069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:18.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.664+0000 7fb4053b2700 1 -- 172.21.15.89:0/398856526 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40015e140 con 0x7fb400159c60 2023-10-10T20:35:18.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.664+0000 7fb404bb1700 1 -- 172.21.15.89:0/398856526 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379144265 0 0) 0x7fb40015a160 con 0x7fb40015db10 2023-10-10T20:35:18.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.664+0000 7fb4053b2700 1 --2- 172.21.15.89:0/398856526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb400159c60 0x7fb40015d5d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb4000d1090 tx=0x7fb3f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c8bd902f0ad804 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:18.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.664+0000 7fb404bb1700 1 -- 172.21.15.89:0/398856526 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f000b6b0 con 0x7fb400159c60 2023-10-10T20:35:18.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.664+0000 7fb404bb1700 1 -- 172.21.15.89:0/398856526 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f000b810 con 0x7fb400159c60 2023-10-10T20:35:18.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.664+0000 7fb404bb1700 1 -- 172.21.15.89:0/398856526 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f000bbb0 con 0x7fb400159c60 2023-10-10T20:35:18.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.665+0000 7fb407e17700 1 -- 172.21.15.89:0/398856526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb400159c60 msgr2=0x7fb40015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:18.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.665+0000 7fb407e17700 1 --2- 172.21.15.89:0/398856526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb400159c60 0x7fb40015d5d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb4000d1090 tx=0x7fb3f00091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:18.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.665+0000 7fb407e17700 1 -- 172.21.15.89:0/398856526 shutdown_connections 2023-10-10T20:35:18.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.665+0000 7fb407e17700 1 --2- 172.21.15.89:0/398856526 >> v2:172.21.15.111:3300/0 conn(0x7fb400159890 0x7fb40015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:18.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.665+0000 7fb407e17700 1 --2- 172.21.15.89:0/398856526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb400159c60 0x7fb40015d5d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:18.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.665+0000 7fb407e17700 1 -- 172.21.15.89:0/398856526 >> 172.21.15.89:0/398856526 conn(0x7fb400068e20 msgr2=0x7fb4000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:18.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.665+0000 7fb407e17700 1 -- 172.21.15.89:0/398856526 shutdown_connections 2023-10-10T20:35:18.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.665+0000 7fb407e17700 1 -- 172.21.15.89:0/398856526 wait complete. 2023-10-10T20:35:18.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.666+0000 7fb407e17700 1 Processor -- start 2023-10-10T20:35:18.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.666+0000 7fb407e17700 1 -- start start 2023-10-10T20:35:18.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb407e17700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb400159890 0x7fb4000b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:18.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb407e17700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb400159c60 0x7fb400140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:18.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb407e17700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb40015db10 0x7fb40016dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:18.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb407e17700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4000b38c0 con 0x7fb400159c60 2023-10-10T20:35:18.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb407e17700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40016e3e0 con 0x7fb400159890 2023-10-10T20:35:18.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb407e17700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4000b5990 con 0x7fb40015db10 2023-10-10T20:35:18.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb4053b2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb400159c60 0x7fb400140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:18.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb4053b2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb400159c60 0x7fb400140ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46030/0 (socket says 172.21.15.89:46030) 2023-10-10T20:35:18.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb4053b2700 1 -- 172.21.15.89:0/2800726099 learned_addr learned my addr 172.21.15.89:0/2800726099 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:18.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb405bb3700 1 --2- 172.21.15.89:0/2800726099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb400159890 0x7fb4000b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:18.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb4063b4700 1 --2- 172.21.15.89:0/2800726099 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb40015db10 0x7fb40016dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:18.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb4053b2700 1 -- 172.21.15.89:0/2800726099 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb40015db10 msgr2=0x7fb40016dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:18.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb4053b2700 1 --2- 172.21.15.89:0/2800726099 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb40015db10 0x7fb40016dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:18.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb4053b2700 1 -- 172.21.15.89:0/2800726099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb400159890 msgr2=0x7fb4000b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb4053b2700 1 --2- 172.21.15.89:0/2800726099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb400159890 0x7fb4000b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb4053b2700 1 -- 172.21.15.89:0/2800726099 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f0009420 con 0x7fb400159c60 2023-10-10T20:35:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb4053b2700 1 --2- 172.21.15.89:0/2800726099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb400159c60 0x7fb400140ef0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb400070e20 tx=0x7fb3f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb3f5ffb700 1 -- 172.21.15.89:0/2800726099 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f000b980 con 0x7fb400159c60 2023-10-10T20:35:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb3f5ffb700 1 -- 172.21.15.89:0/2800726099 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f000bb00 con 0x7fb400159c60 2023-10-10T20:35:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb407e17700 1 -- 172.21.15.89:0/2800726099 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4000b5c70 con 0x7fb400159c60 2023-10-10T20:35:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.667+0000 7fb3f5ffb700 1 -- 172.21.15.89:0/2800726099 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0025670 con 0x7fb400159c60 2023-10-10T20:35:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.668+0000 7fb407e17700 1 -- 172.21.15.89:0/2800726099 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4000b6230 con 0x7fb400159c60 2023-10-10T20:35:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.669+0000 7fb407e17700 1 -- 172.21.15.89:0/2800726099 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb40004c3c0 con 0x7fb400159c60 2023-10-10T20:35:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.669+0000 7fb3f5ffb700 1 -- 172.21.15.89:0/2800726099 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb3f0023070 con 0x7fb400159c60 2023-10-10T20:35:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.669+0000 7fb3f5ffb700 1 --2- 172.21.15.89:0/2800726099 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb3e4066f30 0x7fb3e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.669+0000 7fb3f5ffb700 1 -- 172.21.15.89:0/2800726099 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb3f0085ab0 con 0x7fb400159c60 2023-10-10T20:35:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.669+0000 7fb405bb3700 1 --2- 172.21.15.89:0/2800726099 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb3e4066f30 0x7fb3e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.670+0000 7fb405bb3700 1 --2- 172.21.15.89:0/2800726099 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb3e4066f30 0x7fb3e40693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc006fd0 tx=0x7fb3fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.673+0000 7fb3f5ffb700 1 -- 172.21.15.89:0/2800726099 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3f0052d20 con 0x7fb400159c60 2023-10-10T20:35:18.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.857+0000 7fb407e17700 1 -- 172.21.15.89:0/2800726099 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4000cfee0 con 0x7fb400159c60 2023-10-10T20:35:18.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.858+0000 7fb3f5ffb700 1 -- 172.21.15.89:0/2800726099 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+15363 (secure 0 0 0) 0x7fb3f0056340 con 0x7fb400159c60 2023-10-10T20:35:18.860 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:18.861 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":30,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:35:17.900222+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:35:15.667057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-10T20:35:16.860131+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:35:18.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.860+0000 7fb407e17700 1 -- 172.21.15.89:0/2800726099 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb3e4066f30 msgr2=0x7fb3e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:18.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.860+0000 7fb407e17700 1 --2- 172.21.15.89:0/2800726099 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb3e4066f30 0x7fb3e40693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc006fd0 tx=0x7fb3fc008040 comp rx=0 tx=0).stop 2023-10-10T20:35:18.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.860+0000 7fb407e17700 1 -- 172.21.15.89:0/2800726099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb400159c60 msgr2=0x7fb400140ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:18.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.860+0000 7fb407e17700 1 --2- 172.21.15.89:0/2800726099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb400159c60 0x7fb400140ef0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb400070e20 tx=0x7fb3f0017000 comp rx=0 tx=0).stop 2023-10-10T20:35:18.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.860+0000 7fb407e17700 1 -- 172.21.15.89:0/2800726099 shutdown_connections 2023-10-10T20:35:18.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.860+0000 7fb407e17700 1 --2- 172.21.15.89:0/2800726099 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb40015db10 0x7fb40016dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:18.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.860+0000 7fb407e17700 1 --2- 172.21.15.89:0/2800726099 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb3e4066f30 0x7fb3e40693e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:18.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.860+0000 7fb407e17700 1 --2- 172.21.15.89:0/2800726099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb400159890 0x7fb4000b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:18.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.860+0000 7fb407e17700 1 --2- 172.21.15.89:0/2800726099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb400159c60 0x7fb400140ef0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:18.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.860+0000 7fb407e17700 1 -- 172.21.15.89:0/2800726099 >> 172.21.15.89:0/2800726099 conn(0x7fb400068e20 msgr2=0x7fb4001542a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:18.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.860+0000 7fb407e17700 1 -- 172.21.15.89:0/2800726099 shutdown_connections 2023-10-10T20:35:18.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.860+0000 7fb407e17700 1 -- 172.21.15.89:0/2800726099 wait complete. 2023-10-10T20:35:18.874 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:35:18.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.986+0000 7f6412a8a700 1 Processor -- start 2023-10-10T20:35:18.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.986+0000 7f6412a8a700 1 -- start start 2023-10-10T20:35:18.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.987+0000 7f6412a8a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f640c159c60 0x7f640c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:18.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.987+0000 7f6412a8a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f640c15db10 0x7f640c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:18.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.987+0000 7f6412a8a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f640c15a160 con 0x7f640c159890 2023-10-10T20:35:18.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.987+0000 7f6412a8a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f640c0d13a0 con 0x7f640c15db10 2023-10-10T20:35:18.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.987+0000 7f6412a8a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f640c15e140 con 0x7f640c159c60 2023-10-10T20:35:18.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.987+0000 7f6410826700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f640c159890 0x7f640c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47340/0 (socket says 172.21.15.89:47340) 2023-10-10T20:35:18.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.987+0000 7f6410826700 1 -- 172.21.15.89:0/1225249178 learned_addr learned my addr 172.21.15.89:0/1225249178 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:18.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f6411027700 1 --2- 172.21.15.89:0/1225249178 >> v2:172.21.15.111:3300/0 conn(0x7f640c15db10 0x7f640c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:18.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f6403fff700 1 --2- 172.21.15.89:0/1225249178 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f640c159c60 0x7f640c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f64037fe700 1 -- 172.21.15.89:0/1225249178 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993898179 0 0) 0x7f640c15a160 con 0x7f640c159890 2023-10-10T20:35:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f64037fe700 1 -- 172.21.15.89:0/1225249178 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63f4003570 con 0x7f640c159890 2023-10-10T20:35:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f64037fe700 1 -- 172.21.15.89:0/1225249178 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 700104981 0 0) 0x7f63f4003570 con 0x7f640c159890 2023-10-10T20:35:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f64037fe700 1 -- 172.21.15.89:0/1225249178 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f640c159c60 msgr2=0x7f640c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f64037fe700 1 --2- 172.21.15.89:0/1225249178 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f640c159c60 0x7f640c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:18.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f64037fe700 1 -- 172.21.15.89:0/1225249178 >> v2:172.21.15.111:3300/0 conn(0x7f640c15db10 msgr2=0x7f640c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:18.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f64037fe700 1 --2- 172.21.15.89:0/1225249178 >> v2:172.21.15.111:3300/0 conn(0x7f640c15db10 0x7f640c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:18.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f64037fe700 1 -- 172.21.15.89:0/1225249178 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640c15a3d0 con 0x7f640c159890 2023-10-10T20:35:18.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f64037fe700 1 -- 172.21.15.89:0/1225249178 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f64080017f0 con 0x7f640c159890 2023-10-10T20:35:18.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f64037fe700 1 -- 172.21.15.89:0/1225249178 >> v1:172.21.15.89:6789/0 conn(0x7f640c159890 legacy=0x7f640c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:18.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f64037fe700 1 --2- 172.21.15.89:0/1225249178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63f4003a60 0x7f63f4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:18.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f64037fe700 1 --2- 172.21.15.89:0/1225249178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63f4004810 0x7f63f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:18.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f64037fe700 1 --2- 172.21.15.89:0/1225249178 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f640c159c60 0x7f63f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:18.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f64037fe700 1 -- 172.21.15.89:0/1225249178 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c0d13a0 con 0x7f63f4003a60 2023-10-10T20:35:18.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f64037fe700 1 -- 172.21.15.89:0/1225249178 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c15e140 con 0x7f63f4004810 2023-10-10T20:35:18.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f64037fe700 1 -- 172.21.15.89:0/1225249178 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f400ba60 con 0x7f640c159c60 2023-10-10T20:35:18.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f6410826700 1 --2- 172.21.15.89:0/1225249178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63f4003a60 0x7f63f4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:18.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f6411027700 1 --2- 172.21.15.89:0/1225249178 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f640c159c60 0x7f63f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:18.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f6403fff700 1 --2- 172.21.15.89:0/1225249178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63f4004810 0x7f63f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:18.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f6410826700 1 -- 172.21.15.89:0/1225249178 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f640c159c60 msgr2=0x7f63f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:18.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f6410826700 1 --2- 172.21.15.89:0/1225249178 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f640c159c60 0x7f63f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:18.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f6410826700 1 -- 172.21.15.89:0/1225249178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63f4004810 msgr2=0x7f63f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:18.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f6410826700 1 --2- 172.21.15.89:0/1225249178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63f4004810 0x7f63f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:18.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f6410826700 1 -- 172.21.15.89:0/1225249178 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f4003570 con 0x7f63f4003a60 2023-10-10T20:35:18.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f6403fff700 1 --2- 172.21.15.89:0/1225249178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63f4004810 0x7f63f4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:19.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.988+0000 7f6410826700 1 --2- 172.21.15.89:0/1225249178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63f4003a60 0x7f63f4003e70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f640800a9a0 tx=0x7f6408002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980b95a860f0e3b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:19.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.989+0000 7f64037fe700 1 -- 172.21.15.89:0/1225249178 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6408012440 con 0x7f63f4003a60 2023-10-10T20:35:19.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.989+0000 7f64037fe700 1 -- 172.21.15.89:0/1225249178 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64080125c0 con 0x7f63f4003a60 2023-10-10T20:35:19.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.989+0000 7f64037fe700 1 -- 172.21.15.89:0/1225249178 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6408012980 con 0x7f63f4003a60 2023-10-10T20:35:19.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.989+0000 7f6412a8a700 1 -- 172.21.15.89:0/1225249178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63f4003a60 msgr2=0x7f63f4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:19.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.989+0000 7f6412a8a700 1 --2- 172.21.15.89:0/1225249178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63f4003a60 0x7f63f4003e70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f640800a9a0 tx=0x7f6408002210 comp rx=0 tx=0).stop 2023-10-10T20:35:19.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.989+0000 7f6411027700 1 -- 172.21.15.89:0/1225249178 reap_dead start 2023-10-10T20:35:19.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.989+0000 7f6412a8a700 1 -- 172.21.15.89:0/1225249178 shutdown_connections 2023-10-10T20:35:19.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.989+0000 7f6412a8a700 1 -- 172.21.15.89:0/1225249178 >> 172.21.15.89:0/1225249178 conn(0x7f640c068e20 msgr2=0x7f640c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:19.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.989+0000 7f6412a8a700 1 -- 172.21.15.89:0/1225249178 shutdown_connections 2023-10-10T20:35:19.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.989+0000 7f6412a8a700 1 -- 172.21.15.89:0/1225249178 wait complete. 2023-10-10T20:35:19.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.990+0000 7f6412a8a700 1 Processor -- start 2023-10-10T20:35:19.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.990+0000 7f6412a8a700 1 -- start start 2023-10-10T20:35:19.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.991+0000 7f6412a8a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63f4003a60 0x7f640c143df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:19.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.991+0000 7f6412a8a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f640c0b16d0 0x7f640c0c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:19.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.991+0000 7f6412a8a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f640c0b1b40 0x7f640c0c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:19.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.991+0000 7f6412a8a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c0b4930 con 0x7f640c0b1b40 2023-10-10T20:35:19.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.991+0000 7f6412a8a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c0ca0b0 con 0x7f640c0b16d0 2023-10-10T20:35:19.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.991+0000 7f6412a8a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f640c0b3d20 con 0x7f63f4003a60 2023-10-10T20:35:19.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.991+0000 7f6411027700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f640c0b1b40 0x7f640c0c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:19.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.991+0000 7f6411027700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f640c0b1b40 0x7f640c0c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46054/0 (socket says 172.21.15.89:46054) 2023-10-10T20:35:19.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.991+0000 7f6411027700 1 -- 172.21.15.89:0/716003503 learned_addr learned my addr 172.21.15.89:0/716003503 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:19.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.991+0000 7f6410826700 1 --2- 172.21.15.89:0/716003503 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63f4003a60 0x7f640c143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:19.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.991+0000 7f6403fff700 1 --2- 172.21.15.89:0/716003503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f640c0b16d0 0x7f640c0c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:19.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.991+0000 7f6411027700 1 -- 172.21.15.89:0/716003503 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63f4003a60 msgr2=0x7f640c143df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:19.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.991+0000 7f6411027700 1 --2- 172.21.15.89:0/716003503 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63f4003a60 0x7f640c143df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:19.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.991+0000 7f6411027700 1 -- 172.21.15.89:0/716003503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f640c0b16d0 msgr2=0x7f640c0c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:19.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.991+0000 7f6411027700 1 --2- 172.21.15.89:0/716003503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f640c0b16d0 0x7f640c0c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:19.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.992+0000 7f6411027700 1 -- 172.21.15.89:0/716003503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640c0b4000 con 0x7f640c0b1b40 2023-10-10T20:35:19.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.992+0000 7f6411027700 1 --2- 172.21.15.89:0/716003503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f640c0b1b40 0x7f640c0c9b70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f640c0d03b0 tx=0x7f64040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:19.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.992+0000 7f6400ff9700 1 -- 172.21.15.89:0/716003503 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6404013750 con 0x7f640c0b1b40 2023-10-10T20:35:19.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.992+0000 7f6400ff9700 1 -- 172.21.15.89:0/716003503 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64040138b0 con 0x7f640c0b1b40 2023-10-10T20:35:19.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.992+0000 7f6412a8a700 1 -- 172.21.15.89:0/716003503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f640c0b42e0 con 0x7f640c0b1b40 2023-10-10T20:35:19.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.992+0000 7f6400ff9700 1 -- 172.21.15.89:0/716003503 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6404013c50 con 0x7f640c0b1b40 2023-10-10T20:35:19.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.992+0000 7f6412a8a700 1 -- 172.21.15.89:0/716003503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f640c144330 con 0x7f640c0b1b40 2023-10-10T20:35:19.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.993+0000 7f6412a8a700 1 -- 172.21.15.89:0/716003503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f640c04c3c0 con 0x7f640c0b1b40 2023-10-10T20:35:19.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.993+0000 7f6400ff9700 1 -- 172.21.15.89:0/716003503 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6404014650 con 0x7f640c0b1b40 2023-10-10T20:35:19.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.994+0000 7f6400ff9700 1 --2- 172.21.15.89:0/716003503 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f63f8066f30 0x7f63f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:19.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.994+0000 7f6400ff9700 1 -- 172.21.15.89:0/716003503 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6404084910 con 0x7f640c0b1b40 2023-10-10T20:35:19.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.994+0000 7f6410826700 1 --2- 172.21.15.89:0/716003503 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f63f8066f30 0x7f63f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:19.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.994+0000 7f6410826700 1 --2- 172.21.15.89:0/716003503 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f63f8066f30 0x7f63f80693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f640800a8c0 tx=0x7f6408015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:19.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:18.998+0000 7f6400ff9700 1 -- 172.21.15.89:0/716003503 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6404051c00 con 0x7f640c0b1b40 2023-10-10T20:35:19.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.181+0000 7f6412a8a700 1 -- 172.21.15.89:0/716003503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f640c06a5b0 con 0x7f640c0b1b40 2023-10-10T20:35:19.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.182+0000 7f6400ff9700 1 -- 172.21.15.89:0/716003503 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f6404022090 con 0x7f640c0b1b40 2023-10-10T20:35:19.184 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:19.184 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":30,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:35:17.900222+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:35:15.667057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-10T20:35:16.860131+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:35:19.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.184+0000 7f6412a8a700 1 -- 172.21.15.89:0/716003503 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f63f8066f30 msgr2=0x7f63f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:19.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.184+0000 7f6412a8a700 1 --2- 172.21.15.89:0/716003503 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f63f8066f30 0x7f63f80693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f640800a8c0 tx=0x7f6408015000 comp rx=0 tx=0).stop 2023-10-10T20:35:19.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.184+0000 7f6412a8a700 1 -- 172.21.15.89:0/716003503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f640c0b1b40 msgr2=0x7f640c0c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:19.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.184+0000 7f6412a8a700 1 --2- 172.21.15.89:0/716003503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f640c0b1b40 0x7f640c0c9b70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f640c0d03b0 tx=0x7f64040091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:19.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.184+0000 7f6412a8a700 1 -- 172.21.15.89:0/716003503 shutdown_connections 2023-10-10T20:35:19.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.184+0000 7f6412a8a700 1 --2- 172.21.15.89:0/716003503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f640c0b1b40 0x7f640c0c9b70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:19.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.184+0000 7f6412a8a700 1 --2- 172.21.15.89:0/716003503 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63f4003a60 0x7f640c143df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:19.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.184+0000 7f6412a8a700 1 --2- 172.21.15.89:0/716003503 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f63f8066f30 0x7f63f80693e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:19.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.184+0000 7f6412a8a700 1 --2- 172.21.15.89:0/716003503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f640c0b16d0 0x7f640c0c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:19.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.184+0000 7f6412a8a700 1 -- 172.21.15.89:0/716003503 >> 172.21.15.89:0/716003503 conn(0x7f640c068e20 msgr2=0x7f640c1587b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:19.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.184+0000 7f6412a8a700 1 -- 172.21.15.89:0/716003503 shutdown_connections 2023-10-10T20:35:19.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.184+0000 7f6412a8a700 1 -- 172.21.15.89:0/716003503 wait complete. 2023-10-10T20:35:19.198 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:35:19.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.313+0000 7fbaf44d2700 1 Processor -- start 2023-10-10T20:35:19.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.313+0000 7fbaf44d2700 1 -- start start 2023-10-10T20:35:19.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.314+0000 7fbaf44d2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbaec069560 0x7fbaec0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:19.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.314+0000 7fbaf44d2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaec069ee0 con 0x7fbaec159890 2023-10-10T20:35:19.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.314+0000 7fbaf44d2700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaec15a160 con 0x7fbaec159c60 2023-10-10T20:35:19.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.314+0000 7fbaf44d2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaec0d13a0 con 0x7fbaec069560 2023-10-10T20:35:19.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.314+0000 7fbaf2a6f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbaec069560 0x7fbaec0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:19.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.314+0000 7fbaf226e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbaec159890 0x7fbaec15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47354/0 (socket says 172.21.15.89:47354) 2023-10-10T20:35:19.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.314+0000 7fbaf226e700 1 -- 172.21.15.89:0/2154341854 learned_addr learned my addr 172.21.15.89:0/2154341854 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:19.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.315+0000 7fbaf2a6f700 1 -- 172.21.15.89:0/2154341854 >> v1:172.21.15.111:6789/0 conn(0x7fbaec159c60 legacy=0x7fbaec15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:19.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.315+0000 7fbaf2a6f700 1 -- 172.21.15.89:0/2154341854 >> v1:172.21.15.89:6789/0 conn(0x7fbaec159890 legacy=0x7fbaec15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:19.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.315+0000 7fbaf2a6f700 1 -- 172.21.15.89:0/2154341854 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaec15e140 con 0x7fbaec069560 2023-10-10T20:35:19.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.315+0000 7fbaf126c700 1 -- 172.21.15.89:0/2154341854 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4282238179 0 0) 0x7fbaec069ee0 con 0x7fbaec159890 2023-10-10T20:35:19.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.315+0000 7fbaf2a6f700 1 --2- 172.21.15.89:0/2154341854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbaec069560 0x7fbaec0699a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009e60 tx=0x7fbae4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a286fce2124934 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:19.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.315+0000 7fbaf126c700 1 -- 172.21.15.89:0/2154341854 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae400e070 con 0x7fbaec069560 2023-10-10T20:35:19.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.315+0000 7fbaf126c700 1 -- 172.21.15.89:0/2154341854 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae400bc30 con 0x7fbaec069560 2023-10-10T20:35:19.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.315+0000 7fbaf126c700 1 -- 172.21.15.89:0/2154341854 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae4012430 con 0x7fbaec069560 2023-10-10T20:35:19.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.315+0000 7fbaf44d2700 1 -- 172.21.15.89:0/2154341854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbaec069560 msgr2=0x7fbaec0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:19.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.315+0000 7fbaf44d2700 1 --2- 172.21.15.89:0/2154341854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbaec069560 0x7fbaec0699a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009e60 tx=0x7fbae4009780 comp rx=0 tx=0).stop 2023-10-10T20:35:19.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.315+0000 7fbaf44d2700 1 -- 172.21.15.89:0/2154341854 shutdown_connections 2023-10-10T20:35:19.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.315+0000 7fbaf44d2700 1 --2- 172.21.15.89:0/2154341854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbaec069560 0x7fbaec0699a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:19.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.316+0000 7fbaf44d2700 1 -- 172.21.15.89:0/2154341854 >> 172.21.15.89:0/2154341854 conn(0x7fbaec068e20 msgr2=0x7fbaec0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:19.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.316+0000 7fbaf44d2700 1 -- 172.21.15.89:0/2154341854 shutdown_connections 2023-10-10T20:35:19.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.316+0000 7fbaf44d2700 1 -- 172.21.15.89:0/2154341854 wait complete. 2023-10-10T20:35:19.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.317+0000 7fbaf44d2700 1 Processor -- start 2023-10-10T20:35:19.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.317+0000 7fbaf44d2700 1 -- start start 2023-10-10T20:35:19.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.317+0000 7fbaf44d2700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbaec069560 0x7fbaec149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:19.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.317+0000 7fbaf44d2700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbaec159890 0x7fbaec0cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:19.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.317+0000 7fbaf44d2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbaec159c60 0x7fbaec157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:19.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.317+0000 7fbaf44d2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec0cfd80 con 0x7fbaec159c60 2023-10-10T20:35:19.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.317+0000 7fbaf44d2700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec0b3d10 con 0x7fbaec069560 2023-10-10T20:35:19.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.317+0000 7fbaf44d2700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec1543b0 con 0x7fbaec159890 2023-10-10T20:35:19.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.317+0000 7fbaf2a6f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbaec159c60 0x7fbaec157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:19.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.317+0000 7fbaf2a6f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbaec159c60 0x7fbaec157560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46080/0 (socket says 172.21.15.89:46080) 2023-10-10T20:35:19.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.317+0000 7fbaf2a6f700 1 -- 172.21.15.89:0/3003661960 learned_addr learned my addr 172.21.15.89:0/3003661960 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:19.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.318+0000 7fbaf226e700 1 --2- 172.21.15.89:0/3003661960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbaec069560 0x7fbaec149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:19.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.318+0000 7fbaf1a6d700 1 --2- 172.21.15.89:0/3003661960 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbaec159890 0x7fbaec0cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:19.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.318+0000 7fbaf2a6f700 1 -- 172.21.15.89:0/3003661960 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbaec159890 msgr2=0x7fbaec0cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:19.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.318+0000 7fbaf2a6f700 1 --2- 172.21.15.89:0/3003661960 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbaec159890 0x7fbaec0cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:19.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.318+0000 7fbaf2a6f700 1 -- 172.21.15.89:0/3003661960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbaec069560 msgr2=0x7fbaec149210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:19.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.318+0000 7fbaf2a6f700 1 --2- 172.21.15.89:0/3003661960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbaec069560 0x7fbaec149210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:19.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.318+0000 7fbaf2a6f700 1 -- 172.21.15.89:0/3003661960 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae4009a20 con 0x7fbaec159c60 2023-10-10T20:35:19.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.318+0000 7fbaf2a6f700 1 --2- 172.21.15.89:0/3003661960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbaec159c60 0x7fbaec157560 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbaec06a070 tx=0x7fbae4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:19.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.318+0000 7fbae27fc700 1 -- 172.21.15.89:0/3003661960 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae401b630 con 0x7fbaec159c60 2023-10-10T20:35:19.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.318+0000 7fbae27fc700 1 -- 172.21.15.89:0/3003661960 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae401b790 con 0x7fbaec159c60 2023-10-10T20:35:19.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.318+0000 7fbaf44d2700 1 -- 172.21.15.89:0/3003661960 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaec0cfaa0 con 0x7fbaec159c60 2023-10-10T20:35:19.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.318+0000 7fbae27fc700 1 -- 172.21.15.89:0/3003661960 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae401bb30 con 0x7fbaec159c60 2023-10-10T20:35:19.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.318+0000 7fbaf44d2700 1 -- 172.21.15.89:0/3003661960 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaec0b39a0 con 0x7fbaec159c60 2023-10-10T20:35:19.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.319+0000 7fbaf44d2700 1 -- 172.21.15.89:0/3003661960 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaec04c3c0 con 0x7fbaec159c60 2023-10-10T20:35:19.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.319+0000 7fbae27fc700 1 -- 172.21.15.89:0/3003661960 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbae401c400 con 0x7fbaec159c60 2023-10-10T20:35:19.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.320+0000 7fbae27fc700 1 --2- 172.21.15.89:0/3003661960 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbad0066f30 0x7fbad00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:19.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.320+0000 7fbae27fc700 1 -- 172.21.15.89:0/3003661960 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbae40864c0 con 0x7fbaec159c60 2023-10-10T20:35:19.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.320+0000 7fbaf226e700 1 --2- 172.21.15.89:0/3003661960 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbad0066f30 0x7fbad00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:19.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.320+0000 7fbaf226e700 1 --2- 172.21.15.89:0/3003661960 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbad0066f30 0x7fbad00693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbae80011e0 tx=0x7fbae8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:19.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.324+0000 7fbae27fc700 1 -- 172.21.15.89:0/3003661960 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbae4053730 con 0x7fbaec159c60 2023-10-10T20:35:19.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:19.550+0000 7fbaf44d2700 1 -- 172.21.15.89:0/3003661960 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbaec0b3e50 con 0x7fbaec159c60 2023-10-10T20:35:20.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.534+0000 7fbae27fc700 1 -- 172.21.15.89:0/3003661960 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbae4009880 con 0x7fbaec159c60 2023-10-10T20:35:20.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.536+0000 7fbaf44d2700 1 -- 172.21.15.89:0/3003661960 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbad0066f30 msgr2=0x7fbad00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:20.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.536+0000 7fbaf44d2700 1 --2- 172.21.15.89:0/3003661960 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbad0066f30 0x7fbad00693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbae80011e0 tx=0x7fbae8009040 comp rx=0 tx=0).stop 2023-10-10T20:35:20.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.537+0000 7fbaf44d2700 1 -- 172.21.15.89:0/3003661960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbaec159c60 msgr2=0x7fbaec157560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:20.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.537+0000 7fbaf44d2700 1 --2- 172.21.15.89:0/3003661960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbaec159c60 0x7fbaec157560 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbaec06a070 tx=0x7fbae4019000 comp rx=0 tx=0).stop 2023-10-10T20:35:20.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.537+0000 7fbaf44d2700 1 -- 172.21.15.89:0/3003661960 shutdown_connections 2023-10-10T20:35:20.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.537+0000 7fbaf44d2700 1 --2- 172.21.15.89:0/3003661960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbaec159c60 0x7fbaec157560 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:20.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.537+0000 7fbaf44d2700 1 --2- 172.21.15.89:0/3003661960 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbad0066f30 0x7fbad00693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:20.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.537+0000 7fbaf44d2700 1 --2- 172.21.15.89:0/3003661960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbaec069560 0x7fbaec149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:20.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.537+0000 7fbaf44d2700 1 --2- 172.21.15.89:0/3003661960 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbaec159890 0x7fbaec0cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:20.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.537+0000 7fbaf44d2700 1 -- 172.21.15.89:0/3003661960 >> 172.21.15.89:0/3003661960 conn(0x7fbaec068e20 msgr2=0x7fbaec155120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:20.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.537+0000 7fbaf44d2700 1 -- 172.21.15.89:0/3003661960 shutdown_connections 2023-10-10T20:35:20.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.537+0000 7fbaf44d2700 1 -- 172.21.15.89:0/3003661960 wait complete. 2023-10-10T20:35:20.555 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:35:20.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.678+0000 7fe8078fa700 1 Processor -- start 2023-10-10T20:35:20.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.678+0000 7fe8078fa700 1 -- start start 2023-10-10T20:35:20.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.678+0000 7fe8078fa700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe800159890 0x7fe80015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:20.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe8078fa700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe80015a160 con 0x7fe800069560 2023-10-10T20:35:20.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe8078fa700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8000d13a0 con 0x7fe800159c60 2023-10-10T20:35:20.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe8078fa700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe80015e140 con 0x7fe800159890 2023-10-10T20:35:20.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe805e97700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe800069560 0x7fe8000ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47366/0 (socket says 172.21.15.89:47366) 2023-10-10T20:35:20.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe805e97700 1 -- 172.21.15.89:0/2045322567 learned_addr learned my addr 172.21.15.89:0/2045322567 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:20.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe805696700 1 --2- 172.21.15.89:0/2045322567 >> v2:172.21.15.111:3300/0 conn(0x7fe800159890 0x7fe80015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:20.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe7f7fff700 1 -- 172.21.15.89:0/2045322567 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3906682545 0 0) 0x7fe80015a160 con 0x7fe800069560 2023-10-10T20:35:20.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe7f7fff700 1 -- 172.21.15.89:0/2045322567 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7ec003570 con 0x7fe800069560 2023-10-10T20:35:20.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe7f7fff700 1 -- 172.21.15.89:0/2045322567 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1590328635 0 0) 0x7fe7ec003570 con 0x7fe800069560 2023-10-10T20:35:20.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe7f7fff700 1 -- 172.21.15.89:0/2045322567 >> v2:172.21.15.111:3300/0 conn(0x7fe800159890 msgr2=0x7fe80015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:20.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe7f7fff700 1 --2- 172.21.15.89:0/2045322567 >> v2:172.21.15.111:3300/0 conn(0x7fe800159890 0x7fe80015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:20.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe7f7fff700 1 -- 172.21.15.89:0/2045322567 >> v1:172.21.15.111:6789/0 conn(0x7fe800159c60 legacy=0x7fe80015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe7f7fff700 1 -- 172.21.15.89:0/2045322567 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80015a3d0 con 0x7fe800069560 2023-10-10T20:35:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe7f7fff700 1 -- 172.21.15.89:0/2045322567 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fe7f80017f0 con 0x7fe800069560 2023-10-10T20:35:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe7f7fff700 1 -- 172.21.15.89:0/2045322567 >> v1:172.21.15.89:6789/0 conn(0x7fe800069560 legacy=0x7fe8000ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:20.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe7f7fff700 1 --2- 172.21.15.89:0/2045322567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec003a60 0x7fe7ec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:20.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe7f7fff700 1 --2- 172.21.15.89:0/2045322567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7ec004810 0x7fe7ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:20.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe7f7fff700 1 --2- 172.21.15.89:0/2045322567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7ec0051c0 0x7fe7ec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:20.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe7f7fff700 1 -- 172.21.15.89:0/2045322567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80015e140 con 0x7fe7ec003a60 2023-10-10T20:35:20.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe7f7fff700 1 -- 172.21.15.89:0/2045322567 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec00c0d0 con 0x7fe7ec004810 2023-10-10T20:35:20.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe7f7fff700 1 -- 172.21.15.89:0/2045322567 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec00c290 con 0x7fe7ec0051c0 2023-10-10T20:35:20.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.679+0000 7fe805696700 1 --2- 172.21.15.89:0/2045322567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec003a60 0x7fe7ec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:20.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.680+0000 7fe804e95700 1 --2- 172.21.15.89:0/2045322567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7ec004810 0x7fe7ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:20.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.680+0000 7fe805e97700 1 --2- 172.21.15.89:0/2045322567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7ec0051c0 0x7fe7ec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:20.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.680+0000 7fe805696700 1 -- 172.21.15.89:0/2045322567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7ec0051c0 msgr2=0x7fe7ec00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:20.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.680+0000 7fe805696700 1 --2- 172.21.15.89:0/2045322567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7ec0051c0 0x7fe7ec00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:20.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.680+0000 7fe805696700 1 -- 172.21.15.89:0/2045322567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7ec004810 msgr2=0x7fe7ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:20.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.680+0000 7fe805696700 1 --2- 172.21.15.89:0/2045322567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7ec004810 0x7fe7ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:20.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.680+0000 7fe805696700 1 -- 172.21.15.89:0/2045322567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ec003570 con 0x7fe7ec003a60 2023-10-10T20:35:20.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.680+0000 7fe804e95700 1 --2- 172.21.15.89:0/2045322567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7ec004810 0x7fe7ec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:20.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.680+0000 7fe805e97700 1 --2- 172.21.15.89:0/2045322567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7ec0051c0 0x7fe7ec00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:20.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.680+0000 7fe805e97700 1 -- 172.21.15.89:0/2045322567 reap_dead start 2023-10-10T20:35:20.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.680+0000 7fe805696700 1 --2- 172.21.15.89:0/2045322567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec003a60 0x7fe7ec003e70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc009c00 tx=0x7fe7fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7c7d7dfa368ffa server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:20.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.680+0000 7fe7f7fff700 1 -- 172.21.15.89:0/2045322567 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc00e710 con 0x7fe7ec003a60 2023-10-10T20:35:20.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.680+0000 7fe7f7fff700 1 -- 172.21.15.89:0/2045322567 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7fc00e870 con 0x7fe7ec003a60 2023-10-10T20:35:20.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.680+0000 7fe7f7fff700 1 -- 172.21.15.89:0/2045322567 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc00ec30 con 0x7fe7ec003a60 2023-10-10T20:35:20.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.681+0000 7fe8078fa700 1 -- 172.21.15.89:0/2045322567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec003a60 msgr2=0x7fe7ec003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:20.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.681+0000 7fe8078fa700 1 --2- 172.21.15.89:0/2045322567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec003a60 0x7fe7ec003e70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc009c00 tx=0x7fe7fc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:20.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.681+0000 7fe8078fa700 1 -- 172.21.15.89:0/2045322567 shutdown_connections 2023-10-10T20:35:20.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.681+0000 7fe8078fa700 1 --2- 172.21.15.89:0/2045322567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec003a60 0x7fe7ec003e70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:20.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.681+0000 7fe8078fa700 1 -- 172.21.15.89:0/2045322567 >> 172.21.15.89:0/2045322567 conn(0x7fe800068e20 msgr2=0x7fe8000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:20.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.681+0000 7fe8078fa700 1 -- 172.21.15.89:0/2045322567 shutdown_connections 2023-10-10T20:35:20.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.681+0000 7fe8078fa700 1 -- 172.21.15.89:0/2045322567 wait complete. 2023-10-10T20:35:20.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.682+0000 7fe8078fa700 1 Processor -- start 2023-10-10T20:35:20.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.682+0000 7fe8078fa700 1 -- start start 2023-10-10T20:35:20.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe8078fa700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7ec003a60 0x7fe80016da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:20.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe8078fa700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8000c8760 0x7fe8000ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:20.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe8078fa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe80014d320 0x7fe8000cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:20.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe8078fa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe800153e10 con 0x7fe80014d320 2023-10-10T20:35:20.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe8078fa700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8000c8170 con 0x7fe8000c8760 2023-10-10T20:35:20.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe8078fa700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8000c8450 con 0x7fe7ec003a60 2023-10-10T20:35:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe805e97700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe80014d320 0x7fe8000cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe805e97700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe80014d320 0x7fe8000cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46098/0 (socket says 172.21.15.89:46098) 2023-10-10T20:35:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe805e97700 1 -- 172.21.15.89:0/1400874290 learned_addr learned my addr 172.21.15.89:0/1400874290 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe805696700 1 --2- 172.21.15.89:0/1400874290 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7ec003a60 0x7fe80016da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:20.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe804e95700 1 --2- 172.21.15.89:0/1400874290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8000c8760 0x7fe8000ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:20.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe805e97700 1 -- 172.21.15.89:0/1400874290 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7ec003a60 msgr2=0x7fe80016da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:20.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe805e97700 1 --2- 172.21.15.89:0/1400874290 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7ec003a60 0x7fe80016da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:20.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe805e97700 1 -- 172.21.15.89:0/1400874290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8000c8760 msgr2=0x7fe8000ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:20.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe805e97700 1 --2- 172.21.15.89:0/1400874290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8000c8760 0x7fe8000ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:20.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe805e97700 1 -- 172.21.15.89:0/1400874290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8001537c0 con 0x7fe80014d320 2023-10-10T20:35:20.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe805e97700 1 --2- 172.21.15.89:0/1400874290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe80014d320 0x7fe8000cd0c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8001350 tx=0x7fe7f80017f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:20.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe7f57fa700 1 -- 172.21.15.89:0/1400874290 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f800eab0 con 0x7fe80014d320 2023-10-10T20:35:20.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe7f57fa700 1 -- 172.21.15.89:0/1400874290 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f800ec10 con 0x7fe80014d320 2023-10-10T20:35:20.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe8078fa700 1 -- 172.21.15.89:0/1400874290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe800153aa0 con 0x7fe80014d320 2023-10-10T20:35:20.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.683+0000 7fe7f57fa700 1 -- 172.21.15.89:0/1400874290 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8013420 con 0x7fe80014d320 2023-10-10T20:35:20.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.684+0000 7fe8078fa700 1 -- 172.21.15.89:0/1400874290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8000cd600 con 0x7fe80014d320 2023-10-10T20:35:20.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.684+0000 7fe8078fa700 1 -- 172.21.15.89:0/1400874290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe80004c3c0 con 0x7fe80014d320 2023-10-10T20:35:20.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.685+0000 7fe7f57fa700 1 -- 172.21.15.89:0/1400874290 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe7f80137d0 con 0x7fe80014d320 2023-10-10T20:35:20.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.685+0000 7fe7f57fa700 1 --2- 172.21.15.89:0/1400874290 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe7e4066f30 0x7fe7e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:20.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.685+0000 7fe7f57fa700 1 -- 172.21.15.89:0/1400874290 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe7f80848c0 con 0x7fe80014d320 2023-10-10T20:35:20.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.685+0000 7fe805696700 1 --2- 172.21.15.89:0/1400874290 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe7e4066f30 0x7fe7e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:20.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.686+0000 7fe805696700 1 --2- 172.21.15.89:0/1400874290 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe7e4066f30 0x7fe7e40693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0091e0 tx=0x7fe7fc01a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:20.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.689+0000 7fe7f57fa700 1 -- 172.21.15.89:0/1400874290 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7f8051b30 con 0x7fe80014d320 2023-10-10T20:35:20.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:20.915+0000 7fe8078fa700 1 -- 172.21.15.89:0/1400874290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe80006a5b0 con 0x7fe80014d320 2023-10-10T20:35:21.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.537+0000 7fe7f57fa700 1 -- 172.21.15.89:0/1400874290 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe7f8055150 con 0x7fe80014d320 2023-10-10T20:35:21.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.539+0000 7fe8078fa700 1 -- 172.21.15.89:0/1400874290 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe7e4066f30 msgr2=0x7fe7e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:21.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.539+0000 7fe8078fa700 1 --2- 172.21.15.89:0/1400874290 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe7e4066f30 0x7fe7e40693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0091e0 tx=0x7fe7fc01a000 comp rx=0 tx=0).stop 2023-10-10T20:35:21.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.539+0000 7fe8078fa700 1 -- 172.21.15.89:0/1400874290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe80014d320 msgr2=0x7fe8000cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:21.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.539+0000 7fe8078fa700 1 --2- 172.21.15.89:0/1400874290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe80014d320 0x7fe8000cd0c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8001350 tx=0x7fe7f80017f0 comp rx=0 tx=0).stop 2023-10-10T20:35:21.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.539+0000 7fe8078fa700 1 -- 172.21.15.89:0/1400874290 shutdown_connections 2023-10-10T20:35:21.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.539+0000 7fe8078fa700 1 --2- 172.21.15.89:0/1400874290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe80014d320 0x7fe8000cd0c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:21.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.539+0000 7fe8078fa700 1 --2- 172.21.15.89:0/1400874290 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7ec003a60 0x7fe80016da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:21.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.539+0000 7fe8078fa700 1 --2- 172.21.15.89:0/1400874290 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe7e4066f30 0x7fe7e40693e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:21.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.539+0000 7fe8078fa700 1 --2- 172.21.15.89:0/1400874290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8000c8760 0x7fe8000ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:21.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.539+0000 7fe8078fa700 1 -- 172.21.15.89:0/1400874290 >> 172.21.15.89:0/1400874290 conn(0x7fe800068e20 msgr2=0x7fe80015d300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:21.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.540+0000 7fe8078fa700 1 -- 172.21.15.89:0/1400874290 shutdown_connections 2023-10-10T20:35:21.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.540+0000 7fe8078fa700 1 -- 172.21.15.89:0/1400874290 wait complete. 2023-10-10T20:35:21.554 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:35:21.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.666+0000 7f9642a38700 1 Processor -- start 2023-10-10T20:35:21.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.666+0000 7f9642a38700 1 -- start start 2023-10-10T20:35:21.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.667+0000 7f9642a38700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f963c159c60 0x7f963c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:21.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.667+0000 7f9642a38700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f963c15db10 0x7f963c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:21.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.667+0000 7f9642a38700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f963c15a160 con 0x7f963c159890 2023-10-10T20:35:21.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.667+0000 7f9642a38700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f963c15df50 con 0x7f963c15db10 2023-10-10T20:35:21.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.667+0000 7f9642a38700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f963c0d13a0 con 0x7f963c159c60 2023-10-10T20:35:21.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.667+0000 7f9640fd5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f963c15db10 0x7f963c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:21.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.667+0000 7f9640fd5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f963c15db10 0x7f963c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46108/0 (socket says 172.21.15.89:46108) 2023-10-10T20:35:21.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.667+0000 7f9640fd5700 1 -- 172.21.15.89:0/2788808796 learned_addr learned my addr 172.21.15.89:0/2788808796 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:21.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.667+0000 7f963b7fe700 1 --2- 172.21.15.89:0/2788808796 >> v2:172.21.15.111:3300/0 conn(0x7f963c159c60 0x7f963c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:21.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.667+0000 7f9640fd5700 1 -- 172.21.15.89:0/2788808796 >> v2:172.21.15.111:3300/0 conn(0x7f963c159c60 msgr2=0x7f963c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:21.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.667+0000 7f9640fd5700 1 --2- 172.21.15.89:0/2788808796 >> v2:172.21.15.111:3300/0 conn(0x7f963c159c60 0x7f963c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:21.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.667+0000 7f9640fd5700 1 -- 172.21.15.89:0/2788808796 >> v1:172.21.15.111:6789/0 conn(0x7f963c159890 legacy=0x7f963c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:21.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.667+0000 7f9640fd5700 1 -- 172.21.15.89:0/2788808796 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963c15e140 con 0x7f963c15db10 2023-10-10T20:35:21.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.667+0000 7f963b7fe700 1 --2- 172.21.15.89:0/2788808796 >> v2:172.21.15.111:3300/0 conn(0x7f963c159c60 0x7f963c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:21.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.667+0000 7f9640fd5700 1 --2- 172.21.15.89:0/2788808796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f963c15db10 0x7f963c069560 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f963c0b67f0 tx=0x7f9634009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5552b857fe3faf8e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:21.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.667+0000 7f963affd700 1 -- 172.21.15.89:0/2788808796 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963400e070 con 0x7f963c15db10 2023-10-10T20:35:21.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.668+0000 7f963affd700 1 -- 172.21.15.89:0/2788808796 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963400ba50 con 0x7f963c15db10 2023-10-10T20:35:21.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.668+0000 7f963affd700 1 -- 172.21.15.89:0/2788808796 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96340092e0 con 0x7f963c15db10 2023-10-10T20:35:21.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.668+0000 7f9642a38700 1 -- 172.21.15.89:0/2788808796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f963c15db10 msgr2=0x7f963c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:21.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.668+0000 7f9642a38700 1 --2- 172.21.15.89:0/2788808796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f963c15db10 0x7f963c069560 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f963c0b67f0 tx=0x7f9634009840 comp rx=0 tx=0).stop 2023-10-10T20:35:21.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.668+0000 7f9642a38700 1 -- 172.21.15.89:0/2788808796 shutdown_connections 2023-10-10T20:35:21.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.668+0000 7f9642a38700 1 --2- 172.21.15.89:0/2788808796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f963c15db10 0x7f963c069560 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:21.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.668+0000 7f9642a38700 1 --2- 172.21.15.89:0/2788808796 >> v2:172.21.15.111:3300/0 conn(0x7f963c159c60 0x7f963c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:21.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.668+0000 7f9642a38700 1 -- 172.21.15.89:0/2788808796 >> 172.21.15.89:0/2788808796 conn(0x7f963c068e20 msgr2=0x7f963c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:21.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.668+0000 7f9642a38700 1 -- 172.21.15.89:0/2788808796 shutdown_connections 2023-10-10T20:35:21.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.668+0000 7f9642a38700 1 -- 172.21.15.89:0/2788808796 wait complete. 2023-10-10T20:35:21.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.669+0000 7f9642a38700 1 Processor -- start 2023-10-10T20:35:21.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.669+0000 7f9642a38700 1 -- start start 2023-10-10T20:35:21.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.670+0000 7f9642a38700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f963c159890 0x7f963c158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:21.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.670+0000 7f9642a38700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f963c159c60 0x7f963c0cf480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:21.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.670+0000 7f9642a38700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f963c15db10 0x7f963c0c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:21.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.670+0000 7f9642a38700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c0cecf0 con 0x7f963c159890 2023-10-10T20:35:21.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.670+0000 7f9642a38700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c0c8390 con 0x7f963c15db10 2023-10-10T20:35:21.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.670+0000 7f9642a38700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963c0c8670 con 0x7f963c159c60 2023-10-10T20:35:21.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.670+0000 7f963bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f963c159890 0x7f963c158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:21.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.670+0000 7f963bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f963c159890 0x7f963c158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46124/0 (socket says 172.21.15.89:46124) 2023-10-10T20:35:21.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.670+0000 7f963bfff700 1 -- 172.21.15.89:0/981263616 learned_addr learned my addr 172.21.15.89:0/981263616 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:21.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.670+0000 7f963b7fe700 1 --2- 172.21.15.89:0/981263616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f963c159c60 0x7f963c0cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:21.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.670+0000 7f9640fd5700 1 --2- 172.21.15.89:0/981263616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f963c15db10 0x7f963c0c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:21.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.671+0000 7f963bfff700 1 -- 172.21.15.89:0/981263616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f963c159c60 msgr2=0x7f963c0cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:21.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.671+0000 7f963bfff700 1 --2- 172.21.15.89:0/981263616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f963c159c60 0x7f963c0cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:21.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.671+0000 7f963bfff700 1 -- 172.21.15.89:0/981263616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f963c15db10 msgr2=0x7f963c0c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:21.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.671+0000 7f963bfff700 1 --2- 172.21.15.89:0/981263616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f963c15db10 0x7f963c0c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:21.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.671+0000 7f963bfff700 1 -- 172.21.15.89:0/981263616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9634009ae0 con 0x7f963c159890 2023-10-10T20:35:21.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.671+0000 7f963bfff700 1 --2- 172.21.15.89:0/981263616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f963c159890 0x7f963c158540 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f962c00c590 tx=0x7f962c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:21.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.671+0000 7f9623fff700 1 -- 172.21.15.89:0/981263616 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c011590 con 0x7f963c159890 2023-10-10T20:35:21.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.671+0000 7f9623fff700 1 -- 172.21.15.89:0/981263616 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962c011710 con 0x7f963c159890 2023-10-10T20:35:21.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.671+0000 7f9642a38700 1 -- 172.21.15.89:0/981263616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f963c0cef60 con 0x7f963c159890 2023-10-10T20:35:21.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.671+0000 7f9623fff700 1 -- 172.21.15.89:0/981263616 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c011ad0 con 0x7f963c159890 2023-10-10T20:35:21.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.671+0000 7f9642a38700 1 -- 172.21.15.89:0/981263616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963c0c8950 con 0x7f963c159890 2023-10-10T20:35:21.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.672+0000 7f9642a38700 1 -- 172.21.15.89:0/981263616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f963c0cf150 con 0x7f963c159890 2023-10-10T20:35:21.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.672+0000 7f9623fff700 1 -- 172.21.15.89:0/981263616 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f962c01b3b0 con 0x7f963c159890 2023-10-10T20:35:21.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.673+0000 7f9623fff700 1 --2- 172.21.15.89:0/981263616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9628066f30 0x7f96280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:21.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.673+0000 7f9623fff700 1 -- 172.21.15.89:0/981263616 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f962c0839c0 con 0x7f963c159890 2023-10-10T20:35:21.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.673+0000 7f963b7fe700 1 --2- 172.21.15.89:0/981263616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9628066f30 0x7f96280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:21.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.673+0000 7f963b7fe700 1 --2- 172.21.15.89:0/981263616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9628066f30 0x7f96280693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f963c152ed0 tx=0x7f9630008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:21.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.677+0000 7f9623fff700 1 -- 172.21.15.89:0/981263616 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f962c050c30 con 0x7f963c159890 2023-10-10T20:35:21.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.903+0000 7f9642a38700 1 -- 172.21.15.89:0/981263616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f963c04c3c0 con 0x7f963c159890 2023-10-10T20:35:21.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.903+0000 7f9623fff700 1 -- 172.21.15.89:0/981263616 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f962c017070 con 0x7f963c159890 2023-10-10T20:35:21.907 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:21.907 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1977066906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1977066906},{"type":"v1","addr":"172.21.15.111:6833","nonce":1977066906}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2904313060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2904313060},{"type":"v1","addr":"172.21.15.111:6835","nonce":2904313060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/2135915759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2135915759},{"type":"v1","addr":"172.21.15.89:6836","nonce":2135915759}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:35:17.900533+0000","modified":"2023-10-10T20:35:21.533574+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4899":{"gid":4899,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1330347277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1330347277},{"type":"v1","addr":"172.21.15.89:6837","nonce":1330347277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-10T20:35:21.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.905+0000 7f9642a38700 1 -- 172.21.15.89:0/981263616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9628066f30 msgr2=0x7f96280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:21.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.905+0000 7f9642a38700 1 --2- 172.21.15.89:0/981263616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9628066f30 0x7f96280693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f963c152ed0 tx=0x7f9630008040 comp rx=0 tx=0).stop 2023-10-10T20:35:21.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.905+0000 7f9642a38700 1 -- 172.21.15.89:0/981263616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f963c159890 msgr2=0x7f963c158540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:21.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.905+0000 7f9642a38700 1 --2- 172.21.15.89:0/981263616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f963c159890 0x7f963c158540 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f962c00c590 tx=0x7f962c00c8a0 comp rx=0 tx=0).stop 2023-10-10T20:35:21.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.905+0000 7f9642a38700 1 -- 172.21.15.89:0/981263616 shutdown_connections 2023-10-10T20:35:21.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.905+0000 7f9642a38700 1 --2- 172.21.15.89:0/981263616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9628066f30 0x7f96280693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:21.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.905+0000 7f9642a38700 1 --2- 172.21.15.89:0/981263616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f963c15db10 0x7f963c0c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:21.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.905+0000 7f9642a38700 1 --2- 172.21.15.89:0/981263616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f963c159890 0x7f963c158540 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:21.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.905+0000 7f9642a38700 1 --2- 172.21.15.89:0/981263616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f963c159c60 0x7f963c0cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:21.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.905+0000 7f9642a38700 1 -- 172.21.15.89:0/981263616 >> 172.21.15.89:0/981263616 conn(0x7f963c068e20 msgr2=0x7f963c0c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:21.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.905+0000 7f9642a38700 1 -- 172.21.15.89:0/981263616 shutdown_connections 2023-10-10T20:35:21.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:21.906+0000 7f9642a38700 1 -- 172.21.15.89:0/981263616 wait complete. 2023-10-10T20:35:21.911 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 17 2023-10-10T20:35:21.921 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:35:22.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.031+0000 7feaf811b700 1 Processor -- start 2023-10-10T20:35:22.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.032+0000 7feaf811b700 1 -- start start 2023-10-10T20:35:22.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.032+0000 7feaf811b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feaf006bcd0 0x7feaf00caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:22.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.032+0000 7feaf811b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7feaf015df50 0x7feaf015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:22.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.032+0000 7feaf811b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf015d7e0 con 0x7feaf006c0a0 2023-10-10T20:35:22.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.032+0000 7feaf811b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf015dac0 con 0x7feaf015df50 2023-10-10T20:35:22.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.032+0000 7feaf811b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf015aa30 con 0x7feaf006bcd0 2023-10-10T20:35:22.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.032+0000 7feaf66b8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7feaf015df50 0x7feaf015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:22.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.033+0000 7feaf56b6700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7feaf006c0a0 0x7feaf015a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47372/0 (socket says 172.21.15.89:47372) 2023-10-10T20:35:22.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.033+0000 7feaf56b6700 1 -- 172.21.15.89:0/3701937640 learned_addr learned my addr 172.21.15.89:0/3701937640 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:22.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.033+0000 7feaf5eb7700 1 --2- 172.21.15.89:0/3701937640 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feaf006bcd0 0x7feaf00caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:22.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.033+0000 7feaf66b8700 1 -- 172.21.15.89:0/3701937640 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feaf006bcd0 msgr2=0x7feaf00caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:22.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.033+0000 7feaf66b8700 1 --2- 172.21.15.89:0/3701937640 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feaf006bcd0 0x7feaf00caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.033+0000 7feaf66b8700 1 -- 172.21.15.89:0/3701937640 >> v1:172.21.15.89:6789/0 conn(0x7feaf006c0a0 legacy=0x7feaf015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:22.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.033+0000 7feaf4eb5700 1 -- 172.21.15.89:0/3701937640 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877300534 0 0) 0x7feaf015d7e0 con 0x7feaf006c0a0 2023-10-10T20:35:22.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.033+0000 7feaf66b8700 1 -- 172.21.15.89:0/3701937640 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf015ad10 con 0x7feaf015df50 2023-10-10T20:35:22.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.033+0000 7feaf66b8700 1 --2- 172.21.15.89:0/3701937640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaf015df50 0x7feaf015e3b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feae8009e60 tx=0x7feae8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73cab4863325531 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:22.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.033+0000 7feaf4eb5700 1 -- 172.21.15.89:0/3701937640 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae800e070 con 0x7feaf015df50 2023-10-10T20:35:22.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.033+0000 7feaf4eb5700 1 -- 172.21.15.89:0/3701937640 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae800bc30 con 0x7feaf015df50 2023-10-10T20:35:22.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.033+0000 7feaf4eb5700 1 -- 172.21.15.89:0/3701937640 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8012430 con 0x7feaf015df50 2023-10-10T20:35:22.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.034+0000 7feaf811b700 1 -- 172.21.15.89:0/3701937640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaf015df50 msgr2=0x7feaf015e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:22.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.034+0000 7feaf811b700 1 --2- 172.21.15.89:0/3701937640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaf015df50 0x7feaf015e3b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feae8009e60 tx=0x7feae8009780 comp rx=0 tx=0).stop 2023-10-10T20:35:22.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.034+0000 7feaf811b700 1 -- 172.21.15.89:0/3701937640 shutdown_connections 2023-10-10T20:35:22.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.034+0000 7feaf811b700 1 --2- 172.21.15.89:0/3701937640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaf015df50 0x7feaf015e3b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.034+0000 7feaf811b700 1 --2- 172.21.15.89:0/3701937640 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feaf006bcd0 0x7feaf00caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.034+0000 7feaf811b700 1 -- 172.21.15.89:0/3701937640 >> 172.21.15.89:0/3701937640 conn(0x7feaf00ce600 msgr2=0x7feaf00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:22.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.034+0000 7feaf811b700 1 -- 172.21.15.89:0/3701937640 shutdown_connections 2023-10-10T20:35:22.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.034+0000 7feaf811b700 1 -- 172.21.15.89:0/3701937640 wait complete. 2023-10-10T20:35:22.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.035+0000 7feaf811b700 1 Processor -- start 2023-10-10T20:35:22.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.035+0000 7feaf811b700 1 -- start start 2023-10-10T20:35:22.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.035+0000 7feaf811b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feaf006bcd0 0x7feaf00c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:22.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.035+0000 7feaf811b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feaf006c0a0 0x7feaf01598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:22.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.035+0000 7feaf811b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaf015df50 0x7feaf01491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:22.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.035+0000 7feaf811b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf00c7360 con 0x7feaf015df50 2023-10-10T20:35:22.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.035+0000 7feaf811b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf0159e30 con 0x7feaf006bcd0 2023-10-10T20:35:22.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.035+0000 7feaf811b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf00c6750 con 0x7feaf006c0a0 2023-10-10T20:35:22.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.036+0000 7feaf66b8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaf015df50 0x7feaf01491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:22.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.036+0000 7feaf66b8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaf015df50 0x7feaf01491e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46140/0 (socket says 172.21.15.89:46140) 2023-10-10T20:35:22.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.036+0000 7feaf66b8700 1 -- 172.21.15.89:0/1898463790 learned_addr learned my addr 172.21.15.89:0/1898463790 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:22.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.036+0000 7feaf56b6700 1 --2- 172.21.15.89:0/1898463790 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feaf006c0a0 0x7feaf01598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:22.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.036+0000 7feaf5eb7700 1 --2- 172.21.15.89:0/1898463790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feaf006bcd0 0x7feaf00c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:22.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.036+0000 7feaf66b8700 1 -- 172.21.15.89:0/1898463790 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feaf006c0a0 msgr2=0x7feaf01598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:22.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.036+0000 7feaf66b8700 1 --2- 172.21.15.89:0/1898463790 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feaf006c0a0 0x7feaf01598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.036+0000 7feaf66b8700 1 -- 172.21.15.89:0/1898463790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feaf006bcd0 msgr2=0x7feaf00c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:22.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.036+0000 7feaf66b8700 1 --2- 172.21.15.89:0/1898463790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feaf006bcd0 0x7feaf00c6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.036+0000 7feaf66b8700 1 -- 172.21.15.89:0/1898463790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae8009a20 con 0x7feaf015df50 2023-10-10T20:35:22.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.036+0000 7feaf66b8700 1 --2- 172.21.15.89:0/1898463790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaf015df50 0x7feaf01491e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feae800b6c0 tx=0x7feae8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:22.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.036+0000 7feae5ffb700 1 -- 172.21.15.89:0/1898463790 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8007040 con 0x7feaf015df50 2023-10-10T20:35:22.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.036+0000 7feae5ffb700 1 -- 172.21.15.89:0/1898463790 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae80263f0 con 0x7feaf015df50 2023-10-10T20:35:22.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.036+0000 7feaf811b700 1 -- 172.21.15.89:0/1898463790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf00c6a30 con 0x7feaf015df50 2023-10-10T20:35:22.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.036+0000 7feae5ffb700 1 -- 172.21.15.89:0/1898463790 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae80267b0 con 0x7feaf015df50 2023-10-10T20:35:22.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.037+0000 7feaf811b700 1 -- 172.21.15.89:0/1898463790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf00c6ff0 con 0x7feaf015df50 2023-10-10T20:35:22.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.037+0000 7feaf811b700 1 -- 172.21.15.89:0/1898463790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf00c74d0 con 0x7feaf015df50 2023-10-10T20:35:22.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.038+0000 7feae5ffb700 1 -- 172.21.15.89:0/1898463790 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feae8026ba0 con 0x7feaf015df50 2023-10-10T20:35:22.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.038+0000 7feae5ffb700 1 --2- 172.21.15.89:0/1898463790 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fead4066f30 0x7fead40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:22.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.038+0000 7feaf5eb7700 1 --2- 172.21.15.89:0/1898463790 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fead4066f30 0x7fead40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:22.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.038+0000 7feae5ffb700 1 -- 172.21.15.89:0/1898463790 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feae8086460 con 0x7feaf015df50 2023-10-10T20:35:22.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.039+0000 7feaf5eb7700 1 --2- 172.21.15.89:0/1898463790 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fead4066f30 0x7fead40693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feaec006fd0 tx=0x7feaec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:22.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.043+0000 7feae5ffb700 1 -- 172.21.15.89:0/1898463790 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feae80536d0 con 0x7feaf015df50 2023-10-10T20:35:22.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.226+0000 7feaf811b700 1 -- 172.21.15.89:0/1898463790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feaf004c3c0 con 0x7feaf015df50 2023-10-10T20:35:22.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.227+0000 7feae5ffb700 1 -- 172.21.15.89:0/1898463790 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+15350 (secure 0 0 0) 0x7feae8056cf0 con 0x7feaf015df50 2023-10-10T20:35:22.230 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:22.231 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":31,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:35:19.521788+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:35:15.667057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-10T20:35:16.860131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:35:22.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.229+0000 7feaf811b700 1 -- 172.21.15.89:0/1898463790 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fead4066f30 msgr2=0x7fead40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:22.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.229+0000 7feaf811b700 1 --2- 172.21.15.89:0/1898463790 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fead4066f30 0x7fead40693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feaec006fd0 tx=0x7feaec008040 comp rx=0 tx=0).stop 2023-10-10T20:35:22.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.229+0000 7feaf811b700 1 -- 172.21.15.89:0/1898463790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaf015df50 msgr2=0x7feaf01491e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:22.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.229+0000 7feaf811b700 1 --2- 172.21.15.89:0/1898463790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaf015df50 0x7feaf01491e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feae800b6c0 tx=0x7feae8019000 comp rx=0 tx=0).stop 2023-10-10T20:35:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.229+0000 7feaf811b700 1 -- 172.21.15.89:0/1898463790 shutdown_connections 2023-10-10T20:35:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.229+0000 7feaf811b700 1 --2- 172.21.15.89:0/1898463790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaf015df50 0x7feaf01491e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.229+0000 7feaf811b700 1 --2- 172.21.15.89:0/1898463790 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fead4066f30 0x7fead40693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.229+0000 7feaf811b700 1 --2- 172.21.15.89:0/1898463790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feaf006bcd0 0x7feaf00c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.229+0000 7feaf811b700 1 --2- 172.21.15.89:0/1898463790 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feaf006c0a0 0x7feaf01598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.229+0000 7feaf811b700 1 -- 172.21.15.89:0/1898463790 >> 172.21.15.89:0/1898463790 conn(0x7feaf00ce600 msgr2=0x7feaf006cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.229+0000 7feaf811b700 1 -- 172.21.15.89:0/1898463790 shutdown_connections 2023-10-10T20:35:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.229+0000 7feaf811b700 1 -- 172.21.15.89:0/1898463790 wait complete. 2023-10-10T20:35:22.244 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:35:22.245 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:35:22.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.356+0000 7f9f6f286700 1 Processor -- start 2023-10-10T20:35:22.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.356+0000 7f9f6f286700 1 -- start start 2023-10-10T20:35:22.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.357+0000 7f9f6f286700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9f68159420 0x7f9f680b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:22.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.357+0000 7f9f6f286700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9f681597f0 0x7f9f680b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:22.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.357+0000 7f9f6f286700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f6815d230 0x7f9f6815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:22.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.357+0000 7f9f6f286700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f6815dbb0 con 0x7f9f68159420 2023-10-10T20:35:22.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.357+0000 7f9f6f286700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f680b11d0 con 0x7f9f681597f0 2023-10-10T20:35:22.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.357+0000 7f9f6f286700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f680b14b0 con 0x7f9f6815d230 2023-10-10T20:35:22.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.357+0000 7f9f6e284700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9f68159420 0x7f9f680b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:22.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.357+0000 7f9f6e284700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9f68159420 0x7f9f680b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46150/0 (socket says 172.21.15.89:46150) 2023-10-10T20:35:22.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.357+0000 7f9f6e284700 1 -- 172.21.15.89:0/1495515793 learned_addr learned my addr 172.21.15.89:0/1495515793 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:22.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.357+0000 7f9f6da83700 1 --2- 172.21.15.89:0/1495515793 >> v2:172.21.15.111:3300/0 conn(0x7f9f681597f0 0x7f9f680b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:22.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.357+0000 7f9f6ea85700 1 --2- 172.21.15.89:0/1495515793 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f6815d230 0x7f9f6815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:22.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.357+0000 7f9f6e284700 1 -- 172.21.15.89:0/1495515793 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f6815d230 msgr2=0x7f9f6815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:22.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.357+0000 7f9f6e284700 1 --2- 172.21.15.89:0/1495515793 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f6815d230 0x7f9f6815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.357+0000 7f9f6e284700 1 -- 172.21.15.89:0/1495515793 >> v2:172.21.15.111:3300/0 conn(0x7f9f681597f0 msgr2=0x7f9f680b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:22.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.357+0000 7f9f6e284700 1 --2- 172.21.15.89:0/1495515793 >> v2:172.21.15.111:3300/0 conn(0x7f9f681597f0 0x7f9f680b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.357+0000 7f9f6e284700 1 -- 172.21.15.89:0/1495515793 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f680b1790 con 0x7f9f68159420 2023-10-10T20:35:22.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.358+0000 7f9f6e284700 1 --2- 172.21.15.89:0/1495515793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f68159420 0x7f9f680b5f40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9f68089750 tx=0x7f9f64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bd424db5a57bbd server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:22.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.358+0000 7f9f6d282700 1 -- 172.21.15.89:0/1495515793 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6400b640 con 0x7f9f68159420 2023-10-10T20:35:22.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.358+0000 7f9f6d282700 1 -- 172.21.15.89:0/1495515793 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6400b7a0 con 0x7f9f68159420 2023-10-10T20:35:22.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.358+0000 7f9f6d282700 1 -- 172.21.15.89:0/1495515793 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6400bb60 con 0x7f9f68159420 2023-10-10T20:35:22.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.358+0000 7f9f6f286700 1 -- 172.21.15.89:0/1495515793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f68159420 msgr2=0x7f9f680b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:22.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.358+0000 7f9f6f286700 1 --2- 172.21.15.89:0/1495515793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f68159420 0x7f9f680b5f40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9f68089750 tx=0x7f9f64009250 comp rx=0 tx=0).stop 2023-10-10T20:35:22.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.358+0000 7f9f6f286700 1 -- 172.21.15.89:0/1495515793 shutdown_connections 2023-10-10T20:35:22.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.358+0000 7f9f6f286700 1 --2- 172.21.15.89:0/1495515793 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f6815d230 0x7f9f6815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.358+0000 7f9f6f286700 1 --2- 172.21.15.89:0/1495515793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f68159420 0x7f9f680b5f40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.358+0000 7f9f6f286700 1 --2- 172.21.15.89:0/1495515793 >> v2:172.21.15.111:3300/0 conn(0x7f9f681597f0 0x7f9f680b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.358+0000 7f9f6f286700 1 -- 172.21.15.89:0/1495515793 >> 172.21.15.89:0/1495515793 conn(0x7f9f680d0660 msgr2=0x7f9f680d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:22.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.358+0000 7f9f6f286700 1 -- 172.21.15.89:0/1495515793 shutdown_connections 2023-10-10T20:35:22.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.358+0000 7f9f6f286700 1 -- 172.21.15.89:0/1495515793 wait complete. 2023-10-10T20:35:22.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.359+0000 7f9f6f286700 1 Processor -- start 2023-10-10T20:35:22.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.359+0000 7f9f6f286700 1 -- start start 2023-10-10T20:35:22.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.360+0000 7f9f6f286700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f68159420 0x7f9f68141120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:22.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.360+0000 7f9f6f286700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f681597f0 0x7f9f6815eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:22.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.360+0000 7f9f6f286700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f6815d230 0x7f9f681418c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:22.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.360+0000 7f9f6f286700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f68141660 con 0x7f9f681597f0 2023-10-10T20:35:22.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.360+0000 7f9f6f286700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6815f030 con 0x7f9f6815d230 2023-10-10T20:35:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.360+0000 7f9f6f286700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6815c090 con 0x7f9f68159420 2023-10-10T20:35:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.360+0000 7f9f6da83700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f681597f0 0x7f9f6815eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.360+0000 7f9f6da83700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f681597f0 0x7f9f6815eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46156/0 (socket says 172.21.15.89:46156) 2023-10-10T20:35:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.360+0000 7f9f6da83700 1 -- 172.21.15.89:0/2900626894 learned_addr learned my addr 172.21.15.89:0/2900626894 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.360+0000 7f9f6ea85700 1 --2- 172.21.15.89:0/2900626894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f6815d230 0x7f9f681418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:22.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.360+0000 7f9f6da83700 1 -- 172.21.15.89:0/2900626894 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f68159420 msgr2=0x7f9f68141120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:22.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.360+0000 7f9f6da83700 1 --2- 172.21.15.89:0/2900626894 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f68159420 0x7f9f68141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.360+0000 7f9f6da83700 1 -- 172.21.15.89:0/2900626894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f6815d230 msgr2=0x7f9f681418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:22.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.360+0000 7f9f6da83700 1 --2- 172.21.15.89:0/2900626894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f6815d230 0x7f9f681418c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.360+0000 7f9f6da83700 1 -- 172.21.15.89:0/2900626894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f640094f0 con 0x7f9f681597f0 2023-10-10T20:35:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.361+0000 7f9f6da83700 1 --2- 172.21.15.89:0/2900626894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f681597f0 0x7f9f6815eaf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800ec50 tx=0x7f9f580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.361+0000 7f9f5e7fc700 1 -- 172.21.15.89:0/2900626894 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5800c970 con 0x7f9f681597f0 2023-10-10T20:35:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.361+0000 7f9f5e7fc700 1 -- 172.21.15.89:0/2900626894 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5800cad0 con 0x7f9f681597f0 2023-10-10T20:35:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.361+0000 7f9f6f286700 1 -- 172.21.15.89:0/2900626894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f6815b480 con 0x7f9f681597f0 2023-10-10T20:35:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.361+0000 7f9f5e7fc700 1 -- 172.21.15.89:0/2900626894 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5800ce90 con 0x7f9f681597f0 2023-10-10T20:35:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.361+0000 7f9f6f286700 1 -- 172.21.15.89:0/2900626894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f6815ba40 con 0x7f9f681597f0 2023-10-10T20:35:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.362+0000 7f9f6f286700 1 -- 172.21.15.89:0/2900626894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f6804c3c0 con 0x7f9f681597f0 2023-10-10T20:35:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.362+0000 7f9f5e7fc700 1 -- 172.21.15.89:0/2900626894 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f58013680 con 0x7f9f681597f0 2023-10-10T20:35:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.362+0000 7f9f5e7fc700 1 --2- 172.21.15.89:0/2900626894 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9f4c066f30 0x7f9f4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.363+0000 7f9f6e284700 1 --2- 172.21.15.89:0/2900626894 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9f4c066f30 0x7f9f4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.363+0000 7f9f5e7fc700 1 -- 172.21.15.89:0/2900626894 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9f58084860 con 0x7f9f681597f0 2023-10-10T20:35:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.363+0000 7f9f6e284700 1 --2- 172.21.15.89:0/2900626894 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9f4c066f30 0x7f9f4c0693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9f68089750 tx=0x7f9f64017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.367+0000 7f9f5e7fc700 1 -- 172.21.15.89:0/2900626894 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f58051ad0 con 0x7f9f681597f0 2023-10-10T20:35:22.491 INFO:tasks.ceph.mds.c.smithi089.stderr:2023-10-10T20:35:22.488+0000 7fb4ee9f8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:35:22.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.548+0000 7f9f6f286700 1 -- 172.21.15.89:0/2900626894 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9f6815c200 con 0x7f9f4c066f30 2023-10-10T20:35:22.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.555+0000 7f9f5e7fc700 1 -- 172.21.15.89:0/2900626894 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+261753 (secure 0 0 0) 0x7f9f6815c200 con 0x7f9f4c066f30 2023-10-10T20:35:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:22.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.558+0000 7f9f6f286700 1 -- 172.21.15.89:0/2900626894 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9f4c066f30 msgr2=0x7f9f4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:22.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.558+0000 7f9f6f286700 1 --2- 172.21.15.89:0/2900626894 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9f4c066f30 0x7f9f4c0693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9f68089750 tx=0x7f9f64017000 comp rx=0 tx=0).stop 2023-10-10T20:35:22.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.558+0000 7f9f6f286700 1 -- 172.21.15.89:0/2900626894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f681597f0 msgr2=0x7f9f6815eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:22.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.558+0000 7f9f6f286700 1 --2- 172.21.15.89:0/2900626894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f681597f0 0x7f9f6815eaf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800ec50 tx=0x7f9f580091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.558+0000 7f9f6f286700 1 -- 172.21.15.89:0/2900626894 shutdown_connections 2023-10-10T20:35:22.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.558+0000 7f9f6f286700 1 --2- 172.21.15.89:0/2900626894 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9f4c066f30 0x7f9f4c0693e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.558+0000 7f9f6f286700 1 --2- 172.21.15.89:0/2900626894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f6815d230 0x7f9f681418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.558+0000 7f9f6f286700 1 --2- 172.21.15.89:0/2900626894 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f68159420 0x7f9f68141120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.558+0000 7f9f6f286700 1 --2- 172.21.15.89:0/2900626894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f681597f0 0x7f9f6815eaf0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.558+0000 7f9f6f286700 1 -- 172.21.15.89:0/2900626894 >> 172.21.15.89:0/2900626894 conn(0x7f9f680d0660 msgr2=0x7f9f680cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:22.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.558+0000 7f9f6f286700 1 -- 172.21.15.89:0/2900626894 shutdown_connections 2023-10-10T20:35:22.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.558+0000 7f9f6f286700 1 -- 172.21.15.89:0/2900626894 wait complete. 2023-10-10T20:35:22.570 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:35:22.573 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2023-10-10T20:35:20.660468+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381940,"kb_used_data":3636,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592780,"statfs":{"total":773094113280,"available":770655006720,"internally_reserved":0,"allocated":3723264,"data_stored":2120059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.156636"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.521194+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.521268+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:00.556986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.526421+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:21.510213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.525593+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:21.510213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516210+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:35:54.808400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520741+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:11:00.846076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516606+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:41:58.095674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.517832+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:41:10.435299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519158+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:51.442280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.524219+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:41:58.095674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515771+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.529654+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:52:43.971144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515774+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:54:56.839824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518086+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.527879+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:44:44.192355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.517276+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:21:29.405593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515869+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:11.470026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520829+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:44:44.192355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519614+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:18.869383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515344+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:40.662831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519688+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:50.722704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520816+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.517954+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.524527+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:21:29.405593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.524601+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519260+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:41:10.435299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519727+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.527422+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:11:00.846076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519203+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:57.291254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516633+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:09.537841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519899+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515679+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519787+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.533417+0000","last_change":"2023-10-10T20:35:17.533417+0000","last_active":"2023-10-10T20:35:17.533417+0000","last_peered":"2023-10-10T20:35:17.533417+0000","last_clean":"2023-10-10T20:35:17.533417+0000","last_became_active":"2023-10-10T20:35:17.533123+0000","last_became_peered":"2023-10-10T20:35:17.533123+0000","last_unstale":"2023-10-10T20:35:17.533417+0000","last_undegraded":"2023-10-10T20:35:17.533417+0000","last_fullsized":"2023-10-10T20:35:17.533417+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:50.432471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519924+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.524198+0000","last_change":"2023-10-10T20:35:17.524198+0000","last_active":"2023-10-10T20:35:17.524198+0000","last_peered":"2023-10-10T20:35:17.524198+0000","last_clean":"2023-10-10T20:35:17.524198+0000","last_became_active":"2023-10-10T20:35:17.523790+0000","last_became_peered":"2023-10-10T20:35:17.523790+0000","last_unstale":"2023-10-10T20:35:17.524198+0000","last_undegraded":"2023-10-10T20:35:17.524198+0000","last_fullsized":"2023-10-10T20:35:17.524198+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:50.432471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520226+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530147+0000","last_change":"2023-10-10T20:35:17.530147+0000","last_active":"2023-10-10T20:35:17.530147+0000","last_peered":"2023-10-10T20:35:17.530147+0000","last_clean":"2023-10-10T20:35:17.530147+0000","last_became_active":"2023-10-10T20:35:17.529880+0000","last_became_peered":"2023-10-10T20:35:17.529880+0000","last_unstale":"2023-10-10T20:35:17.530147+0000","last_undegraded":"2023-10-10T20:35:17.530147+0000","last_fullsized":"2023-10-10T20:35:17.530147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:33.487727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520267+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:11.470026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.531915+0000","last_change":"2023-10-10T20:35:17.531915+0000","last_active":"2023-10-10T20:35:17.531915+0000","last_peered":"2023-10-10T20:35:17.531915+0000","last_clean":"2023-10-10T20:35:17.531915+0000","last_became_active":"2023-10-10T20:35:17.531635+0000","last_became_peered":"2023-10-10T20:35:17.531635+0000","last_unstale":"2023-10-10T20:35:17.531915+0000","last_undegraded":"2023-10-10T20:35:17.531915+0000","last_fullsized":"2023-10-10T20:35:17.531915+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:43:03.119821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515638+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:50.722704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.526248+0000","last_change":"2023-10-10T20:35:17.526248+0000","last_active":"2023-10-10T20:35:17.526248+0000","last_peered":"2023-10-10T20:35:17.526248+0000","last_clean":"2023-10-10T20:35:17.526248+0000","last_became_active":"2023-10-10T20:35:17.525970+0000","last_became_peered":"2023-10-10T20:35:17.525970+0000","last_unstale":"2023-10-10T20:35:17.526248+0000","last_undegraded":"2023-10-10T20:35:17.526248+0000","last_fullsized":"2023-10-10T20:35:17.526248+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:06:00.872875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.525937+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:43.729088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.527453+0000","last_change":"2023-10-10T20:35:17.527453+0000","last_active":"2023-10-10T20:35:17.527453+0000","last_peered":"2023-10-10T20:35:17.527453+0000","last_clean":"2023-10-10T20:35:17.527453+0000","last_became_active":"2023-10-10T20:35:17.527213+0000","last_became_peered":"2023-10-10T20:35:17.527213+0000","last_unstale":"2023-10-10T20:35:17.527453+0000","last_undegraded":"2023-10-10T20:35:17.527453+0000","last_fullsized":"2023-10-10T20:35:17.527453+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:10:09.057062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518404+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:26.305061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530182+0000","last_change":"2023-10-10T20:35:17.530182+0000","last_active":"2023-10-10T20:35:17.530182+0000","last_peered":"2023-10-10T20:35:17.530182+0000","last_clean":"2023-10-10T20:35:17.530182+0000","last_became_active":"2023-10-10T20:35:17.529928+0000","last_became_peered":"2023-10-10T20:35:17.529928+0000","last_unstale":"2023-10-10T20:35:17.530182+0000","last_undegraded":"2023-10-10T20:35:17.530182+0000","last_fullsized":"2023-10-10T20:35:17.530182+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:49.540840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518367+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530696+0000","last_change":"2023-10-10T20:35:17.530696+0000","last_active":"2023-10-10T20:35:17.530696+0000","last_peered":"2023-10-10T20:35:17.530696+0000","last_clean":"2023-10-10T20:35:17.530696+0000","last_became_active":"2023-10-10T20:35:17.530438+0000","last_became_peered":"2023-10-10T20:35:17.530438+0000","last_unstale":"2023-10-10T20:35:17.530696+0000","last_undegraded":"2023-10-10T20:35:17.530696+0000","last_fullsized":"2023-10-10T20:35:17.530696+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:52.866212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.526938+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:18.869383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.524785+0000","last_change":"2023-10-10T20:35:17.524785+0000","last_active":"2023-10-10T20:35:17.524785+0000","last_peered":"2023-10-10T20:35:17.524785+0000","last_clean":"2023-10-10T20:35:17.524785+0000","last_became_active":"2023-10-10T20:35:17.524484+0000","last_became_peered":"2023-10-10T20:35:17.524484+0000","last_unstale":"2023-10-10T20:35:17.524785+0000","last_undegraded":"2023-10-10T20:35:17.524785+0000","last_fullsized":"2023-10-10T20:35:17.524785+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:49.712895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518580+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:21:29.405593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.525379+0000","last_change":"2023-10-10T20:35:17.525379+0000","last_active":"2023-10-10T20:35:17.525379+0000","last_peered":"2023-10-10T20:35:17.525379+0000","last_clean":"2023-10-10T20:35:17.525379+0000","last_became_active":"2023-10-10T20:35:17.525088+0000","last_became_peered":"2023-10-10T20:35:17.525088+0000","last_unstale":"2023-10-10T20:35:17.525379+0000","last_undegraded":"2023-10-10T20:35:17.525379+0000","last_fullsized":"2023-10-10T20:35:17.525379+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:56.281445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519263+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:21.510213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.531414+0000","last_change":"2023-10-10T20:35:17.531414+0000","last_active":"2023-10-10T20:35:17.531414+0000","last_peered":"2023-10-10T20:35:17.531414+0000","last_clean":"2023-10-10T20:35:17.531414+0000","last_became_active":"2023-10-10T20:35:17.531101+0000","last_became_peered":"2023-10-10T20:35:17.531101+0000","last_unstale":"2023-10-10T20:35:17.531414+0000","last_undegraded":"2023-10-10T20:35:17.531414+0000","last_fullsized":"2023-10-10T20:35:17.531414+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:49.540840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.525961+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:41:10.435299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.531613+0000","last_change":"2023-10-10T20:35:17.531613+0000","last_active":"2023-10-10T20:35:17.531613+0000","last_peered":"2023-10-10T20:35:17.531613+0000","last_clean":"2023-10-10T20:35:17.531613+0000","last_became_active":"2023-10-10T20:35:17.531349+0000","last_became_peered":"2023-10-10T20:35:17.531349+0000","last_unstale":"2023-10-10T20:35:17.531613+0000","last_undegraded":"2023-10-10T20:35:17.531613+0000","last_fullsized":"2023-10-10T20:35:17.531613+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:10.460218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.521203+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530785+0000","last_change":"2023-10-10T20:35:17.530785+0000","last_active":"2023-10-10T20:35:17.530785+0000","last_peered":"2023-10-10T20:35:17.530785+0000","last_clean":"2023-10-10T20:35:17.530785+0000","last_became_active":"2023-10-10T20:35:17.530499+0000","last_became_peered":"2023-10-10T20:35:17.530499+0000","last_unstale":"2023-10-10T20:35:17.530785+0000","last_undegraded":"2023-10-10T20:35:17.530785+0000","last_fullsized":"2023-10-10T20:35:17.530785+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:54:55.830016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518624+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:43.729088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529571+0000","last_change":"2023-10-10T20:35:17.529571+0000","last_active":"2023-10-10T20:35:17.529571+0000","last_peered":"2023-10-10T20:35:17.529571+0000","last_clean":"2023-10-10T20:35:17.529571+0000","last_became_active":"2023-10-10T20:35:17.529157+0000","last_became_peered":"2023-10-10T20:35:17.529157+0000","last_unstale":"2023-10-10T20:35:17.529571+0000","last_undegraded":"2023-10-10T20:35:17.529571+0000","last_fullsized":"2023-10-10T20:35:17.529571+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:49.712895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516789+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.526164+0000","last_change":"2023-10-10T20:35:17.526164+0000","last_active":"2023-10-10T20:35:17.526164+0000","last_peered":"2023-10-10T20:35:17.526164+0000","last_clean":"2023-10-10T20:35:17.526164+0000","last_became_active":"2023-10-10T20:35:17.525936+0000","last_became_peered":"2023-10-10T20:35:17.525936+0000","last_unstale":"2023-10-10T20:35:17.526164+0000","last_undegraded":"2023-10-10T20:35:17.526164+0000","last_fullsized":"2023-10-10T20:35:17.526164+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:44.534944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520485+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:11.470026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530627+0000","last_change":"2023-10-10T20:35:17.530627+0000","last_active":"2023-10-10T20:35:17.530627+0000","last_peered":"2023-10-10T20:35:17.530627+0000","last_clean":"2023-10-10T20:35:17.530627+0000","last_became_active":"2023-10-10T20:35:17.530401+0000","last_became_peered":"2023-10-10T20:35:17.530401+0000","last_unstale":"2023-10-10T20:35:17.530627+0000","last_undegraded":"2023-10-10T20:35:17.530627+0000","last_fullsized":"2023-10-10T20:35:17.530627+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:08.528032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520288+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.532471+0000","last_change":"2023-10-10T20:35:17.532471+0000","last_active":"2023-10-10T20:35:17.532471+0000","last_peered":"2023-10-10T20:35:17.532471+0000","last_clean":"2023-10-10T20:35:17.532471+0000","last_became_active":"2023-10-10T20:35:17.532240+0000","last_became_peered":"2023-10-10T20:35:17.532240+0000","last_unstale":"2023-10-10T20:35:17.532471+0000","last_undegraded":"2023-10-10T20:35:17.532471+0000","last_fullsized":"2023-10-10T20:35:17.532471+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:56.281445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.524262+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.531072+0000","last_change":"2023-10-10T20:35:17.531072+0000","last_active":"2023-10-10T20:35:17.531072+0000","last_peered":"2023-10-10T20:35:17.531072+0000","last_clean":"2023-10-10T20:35:17.531072+0000","last_became_active":"2023-10-10T20:35:17.530742+0000","last_became_peered":"2023-10-10T20:35:17.530742+0000","last_unstale":"2023-10-10T20:35:17.531072+0000","last_undegraded":"2023-10-10T20:35:17.531072+0000","last_fullsized":"2023-10-10T20:35:17.531072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:35.655280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.523050+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:09.537841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529197+0000","last_change":"2023-10-10T20:35:17.529197+0000","last_active":"2023-10-10T20:35:17.529197+0000","last_peered":"2023-10-10T20:35:17.529197+0000","last_clean":"2023-10-10T20:35:17.529197+0000","last_became_active":"2023-10-10T20:35:17.528968+0000","last_became_peered":"2023-10-10T20:35:17.528968+0000","last_unstale":"2023-10-10T20:35:17.529197+0000","last_undegraded":"2023-10-10T20:35:17.529197+0000","last_fullsized":"2023-10-10T20:35:17.529197+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:28:58.181797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516255+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:03.137417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.527681+0000","last_change":"2023-10-10T20:35:17.527681+0000","last_active":"2023-10-10T20:35:17.527681+0000","last_peered":"2023-10-10T20:35:17.527681+0000","last_clean":"2023-10-10T20:35:17.527681+0000","last_became_active":"2023-10-10T20:35:17.527396+0000","last_became_peered":"2023-10-10T20:35:17.527396+0000","last_unstale":"2023-10-10T20:35:17.527681+0000","last_undegraded":"2023-10-10T20:35:17.527681+0000","last_fullsized":"2023-10-10T20:35:17.527681+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:36:04.680277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515248+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:09.608602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529192+0000","last_change":"2023-10-10T20:35:17.529192+0000","last_active":"2023-10-10T20:35:17.529192+0000","last_peered":"2023-10-10T20:35:17.529192+0000","last_clean":"2023-10-10T20:35:17.529192+0000","last_became_active":"2023-10-10T20:35:17.528805+0000","last_became_peered":"2023-10-10T20:35:17.528805+0000","last_unstale":"2023-10-10T20:35:17.529192+0000","last_undegraded":"2023-10-10T20:35:17.529192+0000","last_fullsized":"2023-10-10T20:35:17.529192+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:25.295253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515240+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:32.081417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.532567+0000","last_change":"2023-10-10T20:35:17.532567+0000","last_active":"2023-10-10T20:35:17.532567+0000","last_peered":"2023-10-10T20:35:17.532567+0000","last_clean":"2023-10-10T20:35:17.532567+0000","last_became_active":"2023-10-10T20:35:17.532268+0000","last_became_peered":"2023-10-10T20:35:17.532268+0000","last_unstale":"2023-10-10T20:35:17.532567+0000","last_undegraded":"2023-10-10T20:35:17.532567+0000","last_fullsized":"2023-10-10T20:35:17.532567+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:56.281445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515208+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:26:37.371735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529012+0000","last_change":"2023-10-10T20:35:17.529012+0000","last_active":"2023-10-10T20:35:17.529012+0000","last_peered":"2023-10-10T20:35:17.529012+0000","last_clean":"2023-10-10T20:35:17.529012+0000","last_became_active":"2023-10-10T20:35:17.528721+0000","last_became_peered":"2023-10-10T20:35:17.528721+0000","last_unstale":"2023-10-10T20:35:17.529012+0000","last_undegraded":"2023-10-10T20:35:17.529012+0000","last_fullsized":"2023-10-10T20:35:17.529012+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:00:46.029806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516041+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.528749+0000","last_change":"2023-10-10T20:35:17.528749+0000","last_active":"2023-10-10T20:35:17.528749+0000","last_peered":"2023-10-10T20:35:17.528749+0000","last_clean":"2023-10-10T20:35:17.528749+0000","last_became_active":"2023-10-10T20:35:17.528444+0000","last_became_peered":"2023-10-10T20:35:17.528444+0000","last_unstale":"2023-10-10T20:35:17.528749+0000","last_undegraded":"2023-10-10T20:35:17.528749+0000","last_fullsized":"2023-10-10T20:35:17.528749+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:10.460218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515445+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:53:25.938086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.533438+0000","last_change":"2023-10-10T20:35:17.533438+0000","last_active":"2023-10-10T20:35:17.533438+0000","last_peered":"2023-10-10T20:35:17.533438+0000","last_clean":"2023-10-10T20:35:17.533438+0000","last_became_active":"2023-10-10T20:35:17.533184+0000","last_became_peered":"2023-10-10T20:35:17.533184+0000","last_unstale":"2023-10-10T20:35:17.533438+0000","last_undegraded":"2023-10-10T20:35:17.533438+0000","last_fullsized":"2023-10-10T20:35:17.533438+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:25.295253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.526872+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:44:06.887938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529064+0000","last_change":"2023-10-10T20:35:17.529064+0000","last_active":"2023-10-10T20:35:17.529064+0000","last_peered":"2023-10-10T20:35:17.529064+0000","last_clean":"2023-10-10T20:35:17.529064+0000","last_became_active":"2023-10-10T20:35:17.528790+0000","last_became_peered":"2023-10-10T20:35:17.528790+0000","last_unstale":"2023-10-10T20:35:17.529064+0000","last_undegraded":"2023-10-10T20:35:17.529064+0000","last_fullsized":"2023-10-10T20:35:17.529064+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:44:01.198878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518367+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529291+0000","last_change":"2023-10-10T20:35:17.529291+0000","last_active":"2023-10-10T20:35:17.529291+0000","last_peered":"2023-10-10T20:35:17.529291+0000","last_clean":"2023-10-10T20:35:17.529291+0000","last_became_active":"2023-10-10T20:35:17.529040+0000","last_became_peered":"2023-10-10T20:35:17.529040+0000","last_unstale":"2023-10-10T20:35:17.529291+0000","last_undegraded":"2023-10-10T20:35:17.529291+0000","last_fullsized":"2023-10-10T20:35:17.529291+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:50.432471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520235+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530765+0000","last_change":"2023-10-10T20:35:17.530765+0000","last_active":"2023-10-10T20:35:17.530765+0000","last_peered":"2023-10-10T20:35:17.530765+0000","last_clean":"2023-10-10T20:35:17.530765+0000","last_became_active":"2023-10-10T20:35:17.530515+0000","last_became_peered":"2023-10-10T20:35:17.530515+0000","last_unstale":"2023-10-10T20:35:17.530765+0000","last_undegraded":"2023-10-10T20:35:17.530765+0000","last_fullsized":"2023-10-10T20:35:17.530765+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:06:39.021756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.517941+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529415+0000","last_change":"2023-10-10T20:35:17.529415+0000","last_active":"2023-10-10T20:35:17.529415+0000","last_peered":"2023-10-10T20:35:17.529415+0000","last_clean":"2023-10-10T20:35:17.529415+0000","last_became_active":"2023-10-10T20:35:17.528871+0000","last_became_peered":"2023-10-10T20:35:17.528871+0000","last_unstale":"2023-10-10T20:35:17.529415+0000","last_undegraded":"2023-10-10T20:35:17.529415+0000","last_fullsized":"2023-10-10T20:35:17.529415+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:06:39.021756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520480+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:44:06.887938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.533254+0000","last_change":"2023-10-10T20:35:17.533254+0000","last_active":"2023-10-10T20:35:17.533254+0000","last_peered":"2023-10-10T20:35:17.533254+0000","last_clean":"2023-10-10T20:35:17.533254+0000","last_became_active":"2023-10-10T20:35:17.532949+0000","last_became_peered":"2023-10-10T20:35:17.532949+0000","last_unstale":"2023-10-10T20:35:17.533254+0000","last_undegraded":"2023-10-10T20:35:17.533254+0000","last_fullsized":"2023-10-10T20:35:17.533254+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:30:04.234585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.521191+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:06:40.031564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.526551+0000","last_change":"2023-10-10T20:35:17.526551+0000","last_active":"2023-10-10T20:35:17.526551+0000","last_peered":"2023-10-10T20:35:17.526551+0000","last_clean":"2023-10-10T20:35:17.526551+0000","last_became_active":"2023-10-10T20:35:17.526307+0000","last_became_peered":"2023-10-10T20:35:17.526307+0000","last_unstale":"2023-10-10T20:35:17.526551+0000","last_undegraded":"2023-10-10T20:35:17.526551+0000","last_fullsized":"2023-10-10T20:35:17.526551+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:28.452443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516047+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529867+0000","last_change":"2023-10-10T20:35:17.529867+0000","last_active":"2023-10-10T20:35:17.529867+0000","last_peered":"2023-10-10T20:35:17.529867+0000","last_clean":"2023-10-10T20:35:17.529867+0000","last_became_active":"2023-10-10T20:35:17.529596+0000","last_became_peered":"2023-10-10T20:35:17.529596+0000","last_unstale":"2023-10-10T20:35:17.529867+0000","last_undegraded":"2023-10-10T20:35:17.529867+0000","last_fullsized":"2023-10-10T20:35:17.529867+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:06:39.021756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":172,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-10T20:35:16.509270+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:35:16.509270+0000","last_peered":"2023-10-10T20:35:16.509270+0000","last_clean":"2023-10-10T20:35:16.509270+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:35:16.509270+0000","last_undegraded":"2023-10-10T20:35:16.509270+0000","last_fullsized":"2023-10-10T20:35:16.509270+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298172,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073668,"statfs":{"total":96636764160,"available":96331436032,"internally_reserved":0,"allocated":905216,"data_stored":704776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":114408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":120663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298180,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073660,"statfs":{"total":96636764160,"available":96331427840,"internally_reserved":0,"allocated":913408,"data_stored":710070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":120663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.253}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":120663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":114408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":114408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:35:22.578 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:35:22.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.726+0000 7f4c49174700 1 Processor -- start 2023-10-10T20:35:22.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.726+0000 7f4c49174700 1 -- start start 2023-10-10T20:35:22.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.726+0000 7f4c49174700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4c44159890 0x7f4c4415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:22.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.726+0000 7f4c49174700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c44069560 0x7f4c440699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:22.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.726+0000 7f4c49174700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c44069ee0 con 0x7f4c44159c60 2023-10-10T20:35:22.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.727+0000 7f4c49174700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c4415a160 con 0x7f4c44159890 2023-10-10T20:35:22.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.727+0000 7f4c49174700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440d13a0 con 0x7f4c44069560 2023-10-10T20:35:22.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.727+0000 7f4c4359e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c44069560 0x7f4c440699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:22.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.727+0000 7f4c42d9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4c44159890 0x7f4c4415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:22.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.727+0000 7f4c42d9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4c44159890 0x7f4c4415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:33714/0 (socket says 172.21.15.89:33714) 2023-10-10T20:35:22.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.727+0000 7f4c4359e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c44069560 0x7f4c440699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:38624/0 (socket says 172.21.15.89:38624) 2023-10-10T20:35:22.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.727+0000 7f4c42d9d700 1 -- 172.21.15.89:0/3073533843 learned_addr learned my addr 172.21.15.89:0/3073533843 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:22.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.727+0000 7f4c4259c700 1 --1- 172.21.15.89:0/3073533843 >> v1:172.21.15.111:6789/0 conn(0x7f4c44159c60 0x7f4c4415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:48934/0 (socket says 172.21.15.89:48934) 2023-10-10T20:35:22.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.727+0000 7f4c41d9b700 1 -- 172.21.15.89:0/3073533843 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4189765198 0 0) 0x7f4c44069ee0 con 0x7f4c44159c60 2023-10-10T20:35:22.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.727+0000 7f4c4359e700 1 -- 172.21.15.89:0/3073533843 >> v2:172.21.15.111:3300/0 conn(0x7f4c44159890 msgr2=0x7f4c4415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:22.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.727+0000 7f4c4359e700 1 --2- 172.21.15.89:0/3073533843 >> v2:172.21.15.111:3300/0 conn(0x7f4c44159890 0x7f4c4415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.727+0000 7f4c4359e700 1 -- 172.21.15.89:0/3073533843 >> v1:172.21.15.111:6789/0 conn(0x7f4c44159c60 legacy=0x7f4c4415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:22.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.727+0000 7f4c4359e700 1 -- 172.21.15.89:0/3073533843 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4415e140 con 0x7f4c44069560 2023-10-10T20:35:22.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.727+0000 7f4c4359e700 1 --2- 172.21.15.89:0/3073533843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c44069560 0x7f4c440699a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009e60 tx=0x7f4c3c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1598c4ad6acdfa75 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:22.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.728+0000 7f4c41d9b700 1 -- 172.21.15.89:0/3073533843 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00e070 con 0x7f4c44069560 2023-10-10T20:35:22.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.728+0000 7f4c41d9b700 1 -- 172.21.15.89:0/3073533843 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c00bb60 con 0x7f4c44069560 2023-10-10T20:35:22.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.728+0000 7f4c41d9b700 1 -- 172.21.15.89:0/3073533843 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c012430 con 0x7f4c44069560 2023-10-10T20:35:22.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.728+0000 7f4c49174700 1 -- 172.21.15.89:0/3073533843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c44069560 msgr2=0x7f4c440699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:22.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.728+0000 7f4c49174700 1 --2- 172.21.15.89:0/3073533843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c44069560 0x7f4c440699a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009e60 tx=0x7f4c3c009780 comp rx=0 tx=0).stop 2023-10-10T20:35:22.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.728+0000 7f4c49174700 1 -- 172.21.15.89:0/3073533843 shutdown_connections 2023-10-10T20:35:22.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.728+0000 7f4c49174700 1 --2- 172.21.15.89:0/3073533843 >> v2:172.21.15.111:3300/0 conn(0x7f4c44159890 0x7f4c4415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.728+0000 7f4c49174700 1 --2- 172.21.15.89:0/3073533843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c44069560 0x7f4c440699a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.728+0000 7f4c49174700 1 -- 172.21.15.89:0/3073533843 >> 172.21.15.89:0/3073533843 conn(0x7f4c44068e20 msgr2=0x7f4c440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:22.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.728+0000 7f4c49174700 1 -- 172.21.15.89:0/3073533843 shutdown_connections 2023-10-10T20:35:22.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.728+0000 7f4c49174700 1 -- 172.21.15.89:0/3073533843 wait complete. 2023-10-10T20:35:22.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.729+0000 7f4c49174700 1 Processor -- start 2023-10-10T20:35:22.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.729+0000 7f4c49174700 1 -- start start 2023-10-10T20:35:22.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.730+0000 7f4c49174700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c44069560 0x7f4c441441c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:22.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.730+0000 7f4c49174700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c44159890 0x7f4c4415c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:22.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.730+0000 7f4c49174700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c44159c60 0x7f4c4415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:22.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.730+0000 7f4c49174700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440b4280 con 0x7f4c44159890 2023-10-10T20:35:22.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.730+0000 7f4c49174700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440b6460 con 0x7f4c44159c60 2023-10-10T20:35:22.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.730+0000 7f4c49174700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440cd1d0 con 0x7f4c44069560 2023-10-10T20:35:22.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.730+0000 7f4c4259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c44159890 0x7f4c4415c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:22.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.730+0000 7f4c4259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c44159890 0x7f4c4415c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46166/0 (socket says 172.21.15.89:46166) 2023-10-10T20:35:22.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.730+0000 7f4c4259c700 1 -- 172.21.15.89:0/3639839587 learned_addr learned my addr 172.21.15.89:0/3639839587 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:22.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.730+0000 7f4c42d9d700 1 --2- 172.21.15.89:0/3639839587 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c44069560 0x7f4c441441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:22.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.730+0000 7f4c4359e700 1 --2- 172.21.15.89:0/3639839587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c44159c60 0x7f4c4415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:22.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.731+0000 7f4c4259c700 1 -- 172.21.15.89:0/3639839587 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c44069560 msgr2=0x7f4c441441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:22.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.731+0000 7f4c4259c700 1 --2- 172.21.15.89:0/3639839587 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c44069560 0x7f4c441441c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.731+0000 7f4c4259c700 1 -- 172.21.15.89:0/3639839587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c44159c60 msgr2=0x7f4c4415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:22.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.731+0000 7f4c4259c700 1 --2- 172.21.15.89:0/3639839587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c44159c60 0x7f4c4415cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.731+0000 7f4c4259c700 1 -- 172.21.15.89:0/3639839587 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c009a20 con 0x7f4c44159890 2023-10-10T20:35:22.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.731+0000 7f4c42d9d700 1 --2- 172.21.15.89:0/3639839587 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c44069560 0x7f4c441441c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:22.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.731+0000 7f4c4359e700 1 --2- 172.21.15.89:0/3639839587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c44159c60 0x7f4c4415cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:22.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.731+0000 7f4c4259c700 1 --2- 172.21.15.89:0/3639839587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c44159890 0x7f4c4415c9b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800c7e0 tx=0x7f4c38007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:22.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.731+0000 7f4c2affd700 1 -- 172.21.15.89:0/3639839587 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38007300 con 0x7f4c44159890 2023-10-10T20:35:22.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.731+0000 7f4c2affd700 1 -- 172.21.15.89:0/3639839587 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c38007460 con 0x7f4c44159890 2023-10-10T20:35:22.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.731+0000 7f4c49174700 1 -- 172.21.15.89:0/3639839587 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c440cd4b0 con 0x7f4c44159890 2023-10-10T20:35:22.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.731+0000 7f4c2affd700 1 -- 172.21.15.89:0/3639839587 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38004830 con 0x7f4c44159890 2023-10-10T20:35:22.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.731+0000 7f4c49174700 1 -- 172.21.15.89:0/3639839587 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c440cda70 con 0x7f4c44159890 2023-10-10T20:35:22.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.732+0000 7f4c49174700 1 -- 172.21.15.89:0/3639839587 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c440cdc60 con 0x7f4c44159890 2023-10-10T20:35:22.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.732+0000 7f4c2affd700 1 -- 172.21.15.89:0/3639839587 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c38004be0 con 0x7f4c44159890 2023-10-10T20:35:22.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.733+0000 7f4c2affd700 1 --2- 172.21.15.89:0/3639839587 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c30066f30 0x7f4c300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:22.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.733+0000 7f4c42d9d700 1 --2- 172.21.15.89:0/3639839587 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c30066f30 0x7f4c300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:22.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.733+0000 7f4c2affd700 1 -- 172.21.15.89:0/3639839587 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4c38083af0 con 0x7f4c44159890 2023-10-10T20:35:22.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.733+0000 7f4c42d9d700 1 --2- 172.21.15.89:0/3639839587 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c30066f30 0x7f4c300693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4c34006d40 tx=0x7f4c34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:22.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.737+0000 7f4c2affd700 1 -- 172.21.15.89:0/3639839587 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c38050d60 con 0x7f4c44159890 2023-10-10T20:35:22.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.915+0000 7f4c49174700 1 -- 172.21.15.89:0/3639839587 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4c4404c3c0 con 0x7f4c30066f30 2023-10-10T20:35:22.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.922+0000 7f4c2affd700 1 -- 172.21.15.89:0/3639839587 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+261753 (secure 0 0 0) 0x7f4c4404c3c0 con 0x7f4c30066f30 2023-10-10T20:35:22.928 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:22.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.925+0000 7f4c49174700 1 -- 172.21.15.89:0/3639839587 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c30066f30 msgr2=0x7f4c300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:22.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.925+0000 7f4c49174700 1 --2- 172.21.15.89:0/3639839587 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c30066f30 0x7f4c300693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4c34006d40 tx=0x7f4c34008040 comp rx=0 tx=0).stop 2023-10-10T20:35:22.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.925+0000 7f4c49174700 1 -- 172.21.15.89:0/3639839587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c44159890 msgr2=0x7f4c4415c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:22.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.925+0000 7f4c49174700 1 --2- 172.21.15.89:0/3639839587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c44159890 0x7f4c4415c9b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800c7e0 tx=0x7f4c38007e10 comp rx=0 tx=0).stop 2023-10-10T20:35:22.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.925+0000 7f4c49174700 1 -- 172.21.15.89:0/3639839587 shutdown_connections 2023-10-10T20:35:22.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.925+0000 7f4c49174700 1 --2- 172.21.15.89:0/3639839587 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c30066f30 0x7f4c300693e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.925+0000 7f4c49174700 1 --2- 172.21.15.89:0/3639839587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c44159c60 0x7f4c4415cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.925+0000 7f4c49174700 1 --2- 172.21.15.89:0/3639839587 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c44069560 0x7f4c441441c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.925+0000 7f4c49174700 1 --2- 172.21.15.89:0/3639839587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c44159890 0x7f4c4415c9b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:22.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.925+0000 7f4c49174700 1 -- 172.21.15.89:0/3639839587 >> 172.21.15.89:0/3639839587 conn(0x7f4c44068e20 msgr2=0x7f4c440d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:22.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.925+0000 7f4c49174700 1 -- 172.21.15.89:0/3639839587 shutdown_connections 2023-10-10T20:35:22.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:22.925+0000 7f4c49174700 1 -- 172.21.15.89:0/3639839587 wait complete. 2023-10-10T20:35:22.933 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:35:22.940 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2023-10-10T20:35:22.660825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381940,"kb_used_data":3636,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592780,"statfs":{"total":773094113280,"available":770655006720,"internally_reserved":0,"allocated":3723264,"data_stored":2120059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000922"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.521194+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.521268+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:00.556986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.526421+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:21.510213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.525593+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:21.510213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516210+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:35:54.808400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520741+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:11:00.846076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516606+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:41:58.095674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.517832+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:41:10.435299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519158+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:51.442280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.524219+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:41:58.095674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515771+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.529654+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:52:43.971144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515774+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:54:56.839824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518086+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.527879+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:44:44.192355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.517276+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:21:29.405593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515869+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:11.470026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520829+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:44:44.192355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519614+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:18.869383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515344+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:40.662831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519688+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:50.722704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520816+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.517954+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.524527+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:21:29.405593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.524601+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519260+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:41:10.435299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519727+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.527422+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:11:00.846076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519203+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:57.291254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516633+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:09.537841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519899+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515679+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519787+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.533417+0000","last_change":"2023-10-10T20:35:17.533417+0000","last_active":"2023-10-10T20:35:17.533417+0000","last_peered":"2023-10-10T20:35:17.533417+0000","last_clean":"2023-10-10T20:35:17.533417+0000","last_became_active":"2023-10-10T20:35:17.533123+0000","last_became_peered":"2023-10-10T20:35:17.533123+0000","last_unstale":"2023-10-10T20:35:17.533417+0000","last_undegraded":"2023-10-10T20:35:17.533417+0000","last_fullsized":"2023-10-10T20:35:17.533417+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:50.432471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519924+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.524198+0000","last_change":"2023-10-10T20:35:17.524198+0000","last_active":"2023-10-10T20:35:17.524198+0000","last_peered":"2023-10-10T20:35:17.524198+0000","last_clean":"2023-10-10T20:35:17.524198+0000","last_became_active":"2023-10-10T20:35:17.523790+0000","last_became_peered":"2023-10-10T20:35:17.523790+0000","last_unstale":"2023-10-10T20:35:17.524198+0000","last_undegraded":"2023-10-10T20:35:17.524198+0000","last_fullsized":"2023-10-10T20:35:17.524198+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:50.432471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520226+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530147+0000","last_change":"2023-10-10T20:35:17.530147+0000","last_active":"2023-10-10T20:35:17.530147+0000","last_peered":"2023-10-10T20:35:17.530147+0000","last_clean":"2023-10-10T20:35:17.530147+0000","last_became_active":"2023-10-10T20:35:17.529880+0000","last_became_peered":"2023-10-10T20:35:17.529880+0000","last_unstale":"2023-10-10T20:35:17.530147+0000","last_undegraded":"2023-10-10T20:35:17.530147+0000","last_fullsized":"2023-10-10T20:35:17.530147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:33.487727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520267+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:11.470026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.531915+0000","last_change":"2023-10-10T20:35:17.531915+0000","last_active":"2023-10-10T20:35:17.531915+0000","last_peered":"2023-10-10T20:35:17.531915+0000","last_clean":"2023-10-10T20:35:17.531915+0000","last_became_active":"2023-10-10T20:35:17.531635+0000","last_became_peered":"2023-10-10T20:35:17.531635+0000","last_unstale":"2023-10-10T20:35:17.531915+0000","last_undegraded":"2023-10-10T20:35:17.531915+0000","last_fullsized":"2023-10-10T20:35:17.531915+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:43:03.119821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515638+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:50.722704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.526248+0000","last_change":"2023-10-10T20:35:17.526248+0000","last_active":"2023-10-10T20:35:17.526248+0000","last_peered":"2023-10-10T20:35:17.526248+0000","last_clean":"2023-10-10T20:35:17.526248+0000","last_became_active":"2023-10-10T20:35:17.525970+0000","last_became_peered":"2023-10-10T20:35:17.525970+0000","last_unstale":"2023-10-10T20:35:17.526248+0000","last_undegraded":"2023-10-10T20:35:17.526248+0000","last_fullsized":"2023-10-10T20:35:17.526248+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:06:00.872875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.525937+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:43.729088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.527453+0000","last_change":"2023-10-10T20:35:17.527453+0000","last_active":"2023-10-10T20:35:17.527453+0000","last_peered":"2023-10-10T20:35:17.527453+0000","last_clean":"2023-10-10T20:35:17.527453+0000","last_became_active":"2023-10-10T20:35:17.527213+0000","last_became_peered":"2023-10-10T20:35:17.527213+0000","last_unstale":"2023-10-10T20:35:17.527453+0000","last_undegraded":"2023-10-10T20:35:17.527453+0000","last_fullsized":"2023-10-10T20:35:17.527453+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:10:09.057062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518404+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:26.305061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530182+0000","last_change":"2023-10-10T20:35:17.530182+0000","last_active":"2023-10-10T20:35:17.530182+0000","last_peered":"2023-10-10T20:35:17.530182+0000","last_clean":"2023-10-10T20:35:17.530182+0000","last_became_active":"2023-10-10T20:35:17.529928+0000","last_became_peered":"2023-10-10T20:35:17.529928+0000","last_unstale":"2023-10-10T20:35:17.530182+0000","last_undegraded":"2023-10-10T20:35:17.530182+0000","last_fullsized":"2023-10-10T20:35:17.530182+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:49.540840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518367+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530696+0000","last_change":"2023-10-10T20:35:17.530696+0000","last_active":"2023-10-10T20:35:17.530696+0000","last_peered":"2023-10-10T20:35:17.530696+0000","last_clean":"2023-10-10T20:35:17.530696+0000","last_became_active":"2023-10-10T20:35:17.530438+0000","last_became_peered":"2023-10-10T20:35:17.530438+0000","last_unstale":"2023-10-10T20:35:17.530696+0000","last_undegraded":"2023-10-10T20:35:17.530696+0000","last_fullsized":"2023-10-10T20:35:17.530696+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:52.866212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.526938+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:18.869383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.524785+0000","last_change":"2023-10-10T20:35:17.524785+0000","last_active":"2023-10-10T20:35:17.524785+0000","last_peered":"2023-10-10T20:35:17.524785+0000","last_clean":"2023-10-10T20:35:17.524785+0000","last_became_active":"2023-10-10T20:35:17.524484+0000","last_became_peered":"2023-10-10T20:35:17.524484+0000","last_unstale":"2023-10-10T20:35:17.524785+0000","last_undegraded":"2023-10-10T20:35:17.524785+0000","last_fullsized":"2023-10-10T20:35:17.524785+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:49.712895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518580+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:21:29.405593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.525379+0000","last_change":"2023-10-10T20:35:17.525379+0000","last_active":"2023-10-10T20:35:17.525379+0000","last_peered":"2023-10-10T20:35:17.525379+0000","last_clean":"2023-10-10T20:35:17.525379+0000","last_became_active":"2023-10-10T20:35:17.525088+0000","last_became_peered":"2023-10-10T20:35:17.525088+0000","last_unstale":"2023-10-10T20:35:17.525379+0000","last_undegraded":"2023-10-10T20:35:17.525379+0000","last_fullsized":"2023-10-10T20:35:17.525379+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:56.281445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519263+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:21.510213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.531414+0000","last_change":"2023-10-10T20:35:17.531414+0000","last_active":"2023-10-10T20:35:17.531414+0000","last_peered":"2023-10-10T20:35:17.531414+0000","last_clean":"2023-10-10T20:35:17.531414+0000","last_became_active":"2023-10-10T20:35:17.531101+0000","last_became_peered":"2023-10-10T20:35:17.531101+0000","last_unstale":"2023-10-10T20:35:17.531414+0000","last_undegraded":"2023-10-10T20:35:17.531414+0000","last_fullsized":"2023-10-10T20:35:17.531414+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:49.540840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.525961+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:41:10.435299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.531613+0000","last_change":"2023-10-10T20:35:17.531613+0000","last_active":"2023-10-10T20:35:17.531613+0000","last_peered":"2023-10-10T20:35:17.531613+0000","last_clean":"2023-10-10T20:35:17.531613+0000","last_became_active":"2023-10-10T20:35:17.531349+0000","last_became_peered":"2023-10-10T20:35:17.531349+0000","last_unstale":"2023-10-10T20:35:17.531613+0000","last_undegraded":"2023-10-10T20:35:17.531613+0000","last_fullsized":"2023-10-10T20:35:17.531613+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:10.460218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.521203+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530785+0000","last_change":"2023-10-10T20:35:17.530785+0000","last_active":"2023-10-10T20:35:17.530785+0000","last_peered":"2023-10-10T20:35:17.530785+0000","last_clean":"2023-10-10T20:35:17.530785+0000","last_became_active":"2023-10-10T20:35:17.530499+0000","last_became_peered":"2023-10-10T20:35:17.530499+0000","last_unstale":"2023-10-10T20:35:17.530785+0000","last_undegraded":"2023-10-10T20:35:17.530785+0000","last_fullsized":"2023-10-10T20:35:17.530785+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:54:55.830016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518624+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:43.729088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529571+0000","last_change":"2023-10-10T20:35:17.529571+0000","last_active":"2023-10-10T20:35:17.529571+0000","last_peered":"2023-10-10T20:35:17.529571+0000","last_clean":"2023-10-10T20:35:17.529571+0000","last_became_active":"2023-10-10T20:35:17.529157+0000","last_became_peered":"2023-10-10T20:35:17.529157+0000","last_unstale":"2023-10-10T20:35:17.529571+0000","last_undegraded":"2023-10-10T20:35:17.529571+0000","last_fullsized":"2023-10-10T20:35:17.529571+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:49.712895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516789+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.526164+0000","last_change":"2023-10-10T20:35:17.526164+0000","last_active":"2023-10-10T20:35:17.526164+0000","last_peered":"2023-10-10T20:35:17.526164+0000","last_clean":"2023-10-10T20:35:17.526164+0000","last_became_active":"2023-10-10T20:35:17.525936+0000","last_became_peered":"2023-10-10T20:35:17.525936+0000","last_unstale":"2023-10-10T20:35:17.526164+0000","last_undegraded":"2023-10-10T20:35:17.526164+0000","last_fullsized":"2023-10-10T20:35:17.526164+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:44.534944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520485+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:11.470026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530627+0000","last_change":"2023-10-10T20:35:17.530627+0000","last_active":"2023-10-10T20:35:17.530627+0000","last_peered":"2023-10-10T20:35:17.530627+0000","last_clean":"2023-10-10T20:35:17.530627+0000","last_became_active":"2023-10-10T20:35:17.530401+0000","last_became_peered":"2023-10-10T20:35:17.530401+0000","last_unstale":"2023-10-10T20:35:17.530627+0000","last_undegraded":"2023-10-10T20:35:17.530627+0000","last_fullsized":"2023-10-10T20:35:17.530627+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:08.528032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520288+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.532471+0000","last_change":"2023-10-10T20:35:17.532471+0000","last_active":"2023-10-10T20:35:17.532471+0000","last_peered":"2023-10-10T20:35:17.532471+0000","last_clean":"2023-10-10T20:35:17.532471+0000","last_became_active":"2023-10-10T20:35:17.532240+0000","last_became_peered":"2023-10-10T20:35:17.532240+0000","last_unstale":"2023-10-10T20:35:17.532471+0000","last_undegraded":"2023-10-10T20:35:17.532471+0000","last_fullsized":"2023-10-10T20:35:17.532471+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:56.281445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.524262+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.531072+0000","last_change":"2023-10-10T20:35:17.531072+0000","last_active":"2023-10-10T20:35:17.531072+0000","last_peered":"2023-10-10T20:35:17.531072+0000","last_clean":"2023-10-10T20:35:17.531072+0000","last_became_active":"2023-10-10T20:35:17.530742+0000","last_became_peered":"2023-10-10T20:35:17.530742+0000","last_unstale":"2023-10-10T20:35:17.531072+0000","last_undegraded":"2023-10-10T20:35:17.531072+0000","last_fullsized":"2023-10-10T20:35:17.531072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:35.655280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.523050+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:09.537841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529197+0000","last_change":"2023-10-10T20:35:17.529197+0000","last_active":"2023-10-10T20:35:17.529197+0000","last_peered":"2023-10-10T20:35:17.529197+0000","last_clean":"2023-10-10T20:35:17.529197+0000","last_became_active":"2023-10-10T20:35:17.528968+0000","last_became_peered":"2023-10-10T20:35:17.528968+0000","last_unstale":"2023-10-10T20:35:17.529197+0000","last_undegraded":"2023-10-10T20:35:17.529197+0000","last_fullsized":"2023-10-10T20:35:17.529197+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:28:58.181797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516255+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:03.137417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.527681+0000","last_change":"2023-10-10T20:35:17.527681+0000","last_active":"2023-10-10T20:35:17.527681+0000","last_peered":"2023-10-10T20:35:17.527681+0000","last_clean":"2023-10-10T20:35:17.527681+0000","last_became_active":"2023-10-10T20:35:17.527396+0000","last_became_peered":"2023-10-10T20:35:17.527396+0000","last_unstale":"2023-10-10T20:35:17.527681+0000","last_undegraded":"2023-10-10T20:35:17.527681+0000","last_fullsized":"2023-10-10T20:35:17.527681+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:36:04.680277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515248+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:09.608602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529192+0000","last_change":"2023-10-10T20:35:17.529192+0000","last_active":"2023-10-10T20:35:17.529192+0000","last_peered":"2023-10-10T20:35:17.529192+0000","last_clean":"2023-10-10T20:35:17.529192+0000","last_became_active":"2023-10-10T20:35:17.528805+0000","last_became_peered":"2023-10-10T20:35:17.528805+0000","last_unstale":"2023-10-10T20:35:17.529192+0000","last_undegraded":"2023-10-10T20:35:17.529192+0000","last_fullsized":"2023-10-10T20:35:17.529192+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:25.295253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515240+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:32.081417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.532567+0000","last_change":"2023-10-10T20:35:17.532567+0000","last_active":"2023-10-10T20:35:17.532567+0000","last_peered":"2023-10-10T20:35:17.532567+0000","last_clean":"2023-10-10T20:35:17.532567+0000","last_became_active":"2023-10-10T20:35:17.532268+0000","last_became_peered":"2023-10-10T20:35:17.532268+0000","last_unstale":"2023-10-10T20:35:17.532567+0000","last_undegraded":"2023-10-10T20:35:17.532567+0000","last_fullsized":"2023-10-10T20:35:17.532567+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:56.281445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515208+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:26:37.371735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529012+0000","last_change":"2023-10-10T20:35:17.529012+0000","last_active":"2023-10-10T20:35:17.529012+0000","last_peered":"2023-10-10T20:35:17.529012+0000","last_clean":"2023-10-10T20:35:17.529012+0000","last_became_active":"2023-10-10T20:35:17.528721+0000","last_became_peered":"2023-10-10T20:35:17.528721+0000","last_unstale":"2023-10-10T20:35:17.529012+0000","last_undegraded":"2023-10-10T20:35:17.529012+0000","last_fullsized":"2023-10-10T20:35:17.529012+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:00:46.029806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516041+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.528749+0000","last_change":"2023-10-10T20:35:17.528749+0000","last_active":"2023-10-10T20:35:17.528749+0000","last_peered":"2023-10-10T20:35:17.528749+0000","last_clean":"2023-10-10T20:35:17.528749+0000","last_became_active":"2023-10-10T20:35:17.528444+0000","last_became_peered":"2023-10-10T20:35:17.528444+0000","last_unstale":"2023-10-10T20:35:17.528749+0000","last_undegraded":"2023-10-10T20:35:17.528749+0000","last_fullsized":"2023-10-10T20:35:17.528749+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:10.460218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515445+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:53:25.938086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.533438+0000","last_change":"2023-10-10T20:35:17.533438+0000","last_active":"2023-10-10T20:35:17.533438+0000","last_peered":"2023-10-10T20:35:17.533438+0000","last_clean":"2023-10-10T20:35:17.533438+0000","last_became_active":"2023-10-10T20:35:17.533184+0000","last_became_peered":"2023-10-10T20:35:17.533184+0000","last_unstale":"2023-10-10T20:35:17.533438+0000","last_undegraded":"2023-10-10T20:35:17.533438+0000","last_fullsized":"2023-10-10T20:35:17.533438+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:25.295253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.526872+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:44:06.887938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529064+0000","last_change":"2023-10-10T20:35:17.529064+0000","last_active":"2023-10-10T20:35:17.529064+0000","last_peered":"2023-10-10T20:35:17.529064+0000","last_clean":"2023-10-10T20:35:17.529064+0000","last_became_active":"2023-10-10T20:35:17.528790+0000","last_became_peered":"2023-10-10T20:35:17.528790+0000","last_unstale":"2023-10-10T20:35:17.529064+0000","last_undegraded":"2023-10-10T20:35:17.529064+0000","last_fullsized":"2023-10-10T20:35:17.529064+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:44:01.198878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518367+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529291+0000","last_change":"2023-10-10T20:35:17.529291+0000","last_active":"2023-10-10T20:35:17.529291+0000","last_peered":"2023-10-10T20:35:17.529291+0000","last_clean":"2023-10-10T20:35:17.529291+0000","last_became_active":"2023-10-10T20:35:17.529040+0000","last_became_peered":"2023-10-10T20:35:17.529040+0000","last_unstale":"2023-10-10T20:35:17.529291+0000","last_undegraded":"2023-10-10T20:35:17.529291+0000","last_fullsized":"2023-10-10T20:35:17.529291+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:50.432471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520235+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530765+0000","last_change":"2023-10-10T20:35:17.530765+0000","last_active":"2023-10-10T20:35:17.530765+0000","last_peered":"2023-10-10T20:35:17.530765+0000","last_clean":"2023-10-10T20:35:17.530765+0000","last_became_active":"2023-10-10T20:35:17.530515+0000","last_became_peered":"2023-10-10T20:35:17.530515+0000","last_unstale":"2023-10-10T20:35:17.530765+0000","last_undegraded":"2023-10-10T20:35:17.530765+0000","last_fullsized":"2023-10-10T20:35:17.530765+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:06:39.021756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.517941+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529415+0000","last_change":"2023-10-10T20:35:17.529415+0000","last_active":"2023-10-10T20:35:17.529415+0000","last_peered":"2023-10-10T20:35:17.529415+0000","last_clean":"2023-10-10T20:35:17.529415+0000","last_became_active":"2023-10-10T20:35:17.528871+0000","last_became_peered":"2023-10-10T20:35:17.528871+0000","last_unstale":"2023-10-10T20:35:17.529415+0000","last_undegraded":"2023-10-10T20:35:17.529415+0000","last_fullsized":"2023-10-10T20:35:17.529415+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:06:39.021756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520480+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:44:06.887938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.533254+0000","last_change":"2023-10-10T20:35:17.533254+0000","last_active":"2023-10-10T20:35:17.533254+0000","last_peered":"2023-10-10T20:35:17.533254+0000","last_clean":"2023-10-10T20:35:17.533254+0000","last_became_active":"2023-10-10T20:35:17.532949+0000","last_became_peered":"2023-10-10T20:35:17.532949+0000","last_unstale":"2023-10-10T20:35:17.533254+0000","last_undegraded":"2023-10-10T20:35:17.533254+0000","last_fullsized":"2023-10-10T20:35:17.533254+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:30:04.234585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.521191+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:06:40.031564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.526551+0000","last_change":"2023-10-10T20:35:17.526551+0000","last_active":"2023-10-10T20:35:17.526551+0000","last_peered":"2023-10-10T20:35:17.526551+0000","last_clean":"2023-10-10T20:35:17.526551+0000","last_became_active":"2023-10-10T20:35:17.526307+0000","last_became_peered":"2023-10-10T20:35:17.526307+0000","last_unstale":"2023-10-10T20:35:17.526551+0000","last_undegraded":"2023-10-10T20:35:17.526551+0000","last_fullsized":"2023-10-10T20:35:17.526551+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:28.452443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516047+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529867+0000","last_change":"2023-10-10T20:35:17.529867+0000","last_active":"2023-10-10T20:35:17.529867+0000","last_peered":"2023-10-10T20:35:17.529867+0000","last_clean":"2023-10-10T20:35:17.529867+0000","last_became_active":"2023-10-10T20:35:17.529596+0000","last_became_peered":"2023-10-10T20:35:17.529596+0000","last_unstale":"2023-10-10T20:35:17.529867+0000","last_undegraded":"2023-10-10T20:35:17.529867+0000","last_fullsized":"2023-10-10T20:35:17.529867+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:06:39.021756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":172,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-10T20:35:16.509270+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:35:16.509270+0000","last_peered":"2023-10-10T20:35:16.509270+0000","last_clean":"2023-10-10T20:35:16.509270+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:35:16.509270+0000","last_undegraded":"2023-10-10T20:35:16.509270+0000","last_fullsized":"2023-10-10T20:35:16.509270+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298172,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073668,"statfs":{"total":96636764160,"available":96331436032,"internally_reserved":0,"allocated":905216,"data_stored":704776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":114408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":120663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298180,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073660,"statfs":{"total":96636764160,"available":96331427840,"internally_reserved":0,"allocated":913408,"data_stored":710070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":120663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.253}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":120663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":114408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":114408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:35:22.945 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:35:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.090+0000 7f9aae45d700 1 Processor -- start 2023-10-10T20:35:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.090+0000 7f9aae45d700 1 -- start start 2023-10-10T20:35:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.091+0000 7f9aae45d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9aa8159c60 0x7f9aa815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.091+0000 7f9aae45d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aa815a160 con 0x7f9aa815db10 2023-10-10T20:35:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.091+0000 7f9aae45d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aa80d13a0 con 0x7f9aa8159890 2023-10-10T20:35:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.091+0000 7f9aae45d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa815e140 con 0x7f9aa8159c60 2023-10-10T20:35:23.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.091+0000 7f9aac9fa700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9aa815db10 0x7f9aa8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47380/0 (socket says 172.21.15.89:47380) 2023-10-10T20:35:23.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.091+0000 7f9aac9fa700 1 -- 172.21.15.89:0/1441714476 learned_addr learned my addr 172.21.15.89:0/1441714476 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:23.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9aa77fe700 1 -- 172.21.15.89:0/1441714476 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403712125 0 0) 0x7f9aa815a160 con 0x7f9aa815db10 2023-10-10T20:35:23.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9a9ffff700 1 --2- 172.21.15.89:0/1441714476 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9aa8159c60 0x7f9aa815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:23.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9aa77fe700 1 -- 172.21.15.89:0/1441714476 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a94003570 con 0x7f9aa815db10 2023-10-10T20:35:23.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9aa77fe700 1 -- 172.21.15.89:0/1441714476 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3664001469 0 0) 0x7f9a94003570 con 0x7f9aa815db10 2023-10-10T20:35:23.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9aa77fe700 1 -- 172.21.15.89:0/1441714476 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9aa8159c60 msgr2=0x7f9aa815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:23.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9aa77fe700 1 --2- 172.21.15.89:0/1441714476 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9aa8159c60 0x7f9aa815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9aa77fe700 1 -- 172.21.15.89:0/1441714476 >> v1:172.21.15.111:6789/0 conn(0x7f9aa8159890 legacy=0x7f9aa815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:23.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9aa77fe700 1 -- 172.21.15.89:0/1441714476 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa815a3d0 con 0x7f9aa815db10 2023-10-10T20:35:23.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9aa77fe700 1 -- 172.21.15.89:0/1441714476 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f9aa00017f0 con 0x7f9aa815db10 2023-10-10T20:35:23.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9aa77fe700 1 -- 172.21.15.89:0/1441714476 >> v1:172.21.15.89:6789/0 conn(0x7f9aa815db10 legacy=0x7f9aa8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:23.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9aa77fe700 1 --2- 172.21.15.89:0/1441714476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a94003a90 0x7f9a94003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:23.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9aa77fe700 1 --2- 172.21.15.89:0/1441714476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a94004810 0x7f9a94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:23.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9aa77fe700 1 --2- 172.21.15.89:0/1441714476 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9aa8159c60 0x7f9a940051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:23.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9aa77fe700 1 -- 172.21.15.89:0/1441714476 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa815e140 con 0x7f9a94004810 2023-10-10T20:35:23.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9aa77fe700 1 -- 172.21.15.89:0/1441714476 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9400bb60 con 0x7f9a94003a90 2023-10-10T20:35:23.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9aa77fe700 1 -- 172.21.15.89:0/1441714476 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9400bd20 con 0x7f9aa8159c60 2023-10-10T20:35:23.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9a9ffff700 1 --2- 172.21.15.89:0/1441714476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a94004810 0x7f9a94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:23.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9aa7fff700 1 --2- 172.21.15.89:0/1441714476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a94003a90 0x7f9a94003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:23.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9aac9fa700 1 --2- 172.21.15.89:0/1441714476 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9aa8159c60 0x7f9a940051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:23.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9a9ffff700 1 -- 172.21.15.89:0/1441714476 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9aa8159c60 msgr2=0x7f9a940051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:23.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9a9ffff700 1 --2- 172.21.15.89:0/1441714476 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9aa8159c60 0x7f9a940051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9a9ffff700 1 -- 172.21.15.89:0/1441714476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a94003a90 msgr2=0x7f9a94003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:23.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9a9ffff700 1 --2- 172.21.15.89:0/1441714476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a94003a90 0x7f9a94003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9a9ffff700 1 -- 172.21.15.89:0/1441714476 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a94003570 con 0x7f9a94004810 2023-10-10T20:35:23.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.092+0000 7f9aa7fff700 1 --2- 172.21.15.89:0/1441714476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a94003a90 0x7f9a94003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:23.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.093+0000 7f9a9ffff700 1 --2- 172.21.15.89:0/1441714476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a94004810 0x7f9a94004c80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9aa815a090 tx=0x7f9a900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5325ef6875520b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:23.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.093+0000 7f9aa77fe700 1 -- 172.21.15.89:0/1441714476 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9000e710 con 0x7f9a94004810 2023-10-10T20:35:23.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.093+0000 7f9aa77fe700 1 -- 172.21.15.89:0/1441714476 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9000e870 con 0x7f9a94004810 2023-10-10T20:35:23.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.093+0000 7f9aa77fe700 1 -- 172.21.15.89:0/1441714476 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9000ec30 con 0x7f9a94004810 2023-10-10T20:35:23.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.093+0000 7f9aae45d700 1 -- 172.21.15.89:0/1441714476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a94004810 msgr2=0x7f9a94004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:23.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.093+0000 7f9aae45d700 1 --2- 172.21.15.89:0/1441714476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a94004810 0x7f9a94004c80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9aa815a090 tx=0x7f9a900091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.093+0000 7f9aac9fa700 1 -- 172.21.15.89:0/1441714476 reap_dead start 2023-10-10T20:35:23.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.093+0000 7f9aae45d700 1 -- 172.21.15.89:0/1441714476 shutdown_connections 2023-10-10T20:35:23.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.093+0000 7f9aae45d700 1 -- 172.21.15.89:0/1441714476 >> 172.21.15.89:0/1441714476 conn(0x7f9aa8068e20 msgr2=0x7f9aa80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:23.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.093+0000 7f9aae45d700 1 -- 172.21.15.89:0/1441714476 shutdown_connections 2023-10-10T20:35:23.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.093+0000 7f9aae45d700 1 -- 172.21.15.89:0/1441714476 wait complete. 2023-10-10T20:35:23.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.094+0000 7f9aae45d700 1 Processor -- start 2023-10-10T20:35:23.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.094+0000 7f9aae45d700 1 -- start start 2023-10-10T20:35:23.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.095+0000 7f9aae45d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a94004810 0x7f9aa80b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:23.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.095+0000 7f9aae45d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9aa80b5c00 0x7f9aa80b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:23.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.095+0000 7f9aae45d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9aa80b5290 0x7f9aa8153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:23.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.095+0000 7f9aae45d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa80ca490 con 0x7f9a94004810 2023-10-10T20:35:23.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.095+0000 7f9aae45d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa80b4560 con 0x7f9aa80b5290 2023-10-10T20:35:23.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.095+0000 7f9aae45d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa80b4e00 con 0x7f9aa80b5c00 2023-10-10T20:35:23.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.095+0000 7f9aa7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a94004810 0x7f9aa80b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:23.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.095+0000 7f9aa7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a94004810 0x7f9aa80b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46182/0 (socket says 172.21.15.89:46182) 2023-10-10T20:35:23.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.095+0000 7f9aa7fff700 1 -- 172.21.15.89:0/4046734368 learned_addr learned my addr 172.21.15.89:0/4046734368 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:23.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.095+0000 7f9aac9fa700 1 --2- 172.21.15.89:0/4046734368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9aa80b5290 0x7f9aa8153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:23.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.095+0000 7f9a9ffff700 1 --2- 172.21.15.89:0/4046734368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9aa80b5c00 0x7f9aa80b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:23.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.095+0000 7f9aa7fff700 1 -- 172.21.15.89:0/4046734368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9aa80b5c00 msgr2=0x7f9aa80b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:23.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.095+0000 7f9aa7fff700 1 --2- 172.21.15.89:0/4046734368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9aa80b5c00 0x7f9aa80b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.095+0000 7f9aa7fff700 1 -- 172.21.15.89:0/4046734368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9aa80b5290 msgr2=0x7f9aa8153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:23.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.095+0000 7f9aa7fff700 1 --2- 172.21.15.89:0/4046734368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9aa80b5290 0x7f9aa8153550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.095+0000 7f9aa7fff700 1 -- 172.21.15.89:0/4046734368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa80b6100 con 0x7f9a94004810 2023-10-10T20:35:23.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.095+0000 7f9a9ffff700 1 --2- 172.21.15.89:0/4046734368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9aa80b5c00 0x7f9aa80b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:23.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.096+0000 7f9aa7fff700 1 --2- 172.21.15.89:0/4046734368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a94004810 0x7f9aa80b2370 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800e840 tx=0x7f9a98009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:23.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.096+0000 7f9aa4ff9700 1 -- 172.21.15.89:0/4046734368 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a98017070 con 0x7f9a94004810 2023-10-10T20:35:23.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.096+0000 7f9aa4ff9700 1 -- 172.21.15.89:0/4046734368 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a98006820 con 0x7f9a94004810 2023-10-10T20:35:23.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.096+0000 7f9aa4ff9700 1 -- 172.21.15.89:0/4046734368 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a98006bc0 con 0x7f9a94004810 2023-10-10T20:35:23.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.096+0000 7f9aae45d700 1 -- 172.21.15.89:0/4046734368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa80b47b0 con 0x7f9a94004810 2023-10-10T20:35:23.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.096+0000 7f9aae45d700 1 -- 172.21.15.89:0/4046734368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa8154310 con 0x7f9a94004810 2023-10-10T20:35:23.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.097+0000 7f9aae45d700 1 -- 172.21.15.89:0/4046734368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa81544d0 con 0x7f9a94004810 2023-10-10T20:35:23.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.097+0000 7f9aa4ff9700 1 -- 172.21.15.89:0/4046734368 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9a98007020 con 0x7f9a94004810 2023-10-10T20:35:23.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.097+0000 7f9aa4ff9700 1 --2- 172.21.15.89:0/4046734368 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9a88066f30 0x7f9a880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:23.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.097+0000 7f9aa4ff9700 1 -- 172.21.15.89:0/4046734368 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9a98084150 con 0x7f9a94004810 2023-10-10T20:35:23.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.097+0000 7f9a9ffff700 1 --2- 172.21.15.89:0/4046734368 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9a88066f30 0x7f9a880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:23.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.098+0000 7f9a9ffff700 1 --2- 172.21.15.89:0/4046734368 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9a88066f30 0x7f9a880693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80b6030 tx=0x7f9a9001a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:23.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.102+0000 7f9aa4ff9700 1 -- 172.21.15.89:0/4046734368 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a980513c0 con 0x7f9a94004810 2023-10-10T20:35:23.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.279+0000 7f9aae45d700 1 -- 172.21.15.89:0/4046734368 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9aa804c3c0 con 0x7f9a88066f30 2023-10-10T20:35:23.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.286+0000 7f9aa4ff9700 1 -- 172.21.15.89:0/4046734368 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+261753 (secure 0 0 0) 0x7f9aa804c3c0 con 0x7f9a88066f30 2023-10-10T20:35:23.292 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:23.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.289+0000 7f9aae45d700 1 -- 172.21.15.89:0/4046734368 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9a88066f30 msgr2=0x7f9a880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:23.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.289+0000 7f9aae45d700 1 --2- 172.21.15.89:0/4046734368 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9a88066f30 0x7f9a880693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80b6030 tx=0x7f9a9001a000 comp rx=0 tx=0).stop 2023-10-10T20:35:23.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.289+0000 7f9aae45d700 1 -- 172.21.15.89:0/4046734368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a94004810 msgr2=0x7f9aa80b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:23.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.289+0000 7f9aae45d700 1 --2- 172.21.15.89:0/4046734368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a94004810 0x7f9aa80b2370 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800e840 tx=0x7f9a98009040 comp rx=0 tx=0).stop 2023-10-10T20:35:23.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.289+0000 7f9aae45d700 1 -- 172.21.15.89:0/4046734368 shutdown_connections 2023-10-10T20:35:23.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.289+0000 7f9aae45d700 1 --2- 172.21.15.89:0/4046734368 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9a88066f30 0x7f9a880693e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.289+0000 7f9aae45d700 1 --2- 172.21.15.89:0/4046734368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9aa80b5290 0x7f9aa8153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.289+0000 7f9aae45d700 1 --2- 172.21.15.89:0/4046734368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a94004810 0x7f9aa80b2370 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.289+0000 7f9aae45d700 1 --2- 172.21.15.89:0/4046734368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9aa80b5c00 0x7f9aa80b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.289+0000 7f9aae45d700 1 -- 172.21.15.89:0/4046734368 >> 172.21.15.89:0/4046734368 conn(0x7f9aa8068e20 msgr2=0x7f9aa80cf0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:23.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.289+0000 7f9aae45d700 1 -- 172.21.15.89:0/4046734368 shutdown_connections 2023-10-10T20:35:23.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.289+0000 7f9aae45d700 1 -- 172.21.15.89:0/4046734368 wait complete. 2023-10-10T20:35:23.297 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:35:23.304 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2023-10-10T20:35:22.660825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381940,"kb_used_data":3636,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592780,"statfs":{"total":773094113280,"available":770655006720,"internally_reserved":0,"allocated":3723264,"data_stored":2120059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000922"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.521194+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.521268+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:00.556986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.526421+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:21.510213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.525593+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:21.510213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516210+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:35:54.808400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520741+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:11:00.846076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516606+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:41:58.095674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.517832+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:41:10.435299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519158+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:51.442280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.524219+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:41:58.095674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515771+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.529654+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:52:43.971144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515774+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:54:56.839824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518086+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.527879+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:44:44.192355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.517276+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:21:29.405593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515869+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:11.470026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520829+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:44:44.192355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519614+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:18.869383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515344+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:40.662831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519688+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:50.722704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520816+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.517954+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.524527+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:21:29.405593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.524601+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519260+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:41:10.435299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519727+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.527422+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:11:00.846076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519203+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:57.291254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516633+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:09.537841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519899+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515679+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519787+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.533417+0000","last_change":"2023-10-10T20:35:17.533417+0000","last_active":"2023-10-10T20:35:17.533417+0000","last_peered":"2023-10-10T20:35:17.533417+0000","last_clean":"2023-10-10T20:35:17.533417+0000","last_became_active":"2023-10-10T20:35:17.533123+0000","last_became_peered":"2023-10-10T20:35:17.533123+0000","last_unstale":"2023-10-10T20:35:17.533417+0000","last_undegraded":"2023-10-10T20:35:17.533417+0000","last_fullsized":"2023-10-10T20:35:17.533417+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:50.432471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519924+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.524198+0000","last_change":"2023-10-10T20:35:17.524198+0000","last_active":"2023-10-10T20:35:17.524198+0000","last_peered":"2023-10-10T20:35:17.524198+0000","last_clean":"2023-10-10T20:35:17.524198+0000","last_became_active":"2023-10-10T20:35:17.523790+0000","last_became_peered":"2023-10-10T20:35:17.523790+0000","last_unstale":"2023-10-10T20:35:17.524198+0000","last_undegraded":"2023-10-10T20:35:17.524198+0000","last_fullsized":"2023-10-10T20:35:17.524198+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:50.432471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520226+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530147+0000","last_change":"2023-10-10T20:35:17.530147+0000","last_active":"2023-10-10T20:35:17.530147+0000","last_peered":"2023-10-10T20:35:17.530147+0000","last_clean":"2023-10-10T20:35:17.530147+0000","last_became_active":"2023-10-10T20:35:17.529880+0000","last_became_peered":"2023-10-10T20:35:17.529880+0000","last_unstale":"2023-10-10T20:35:17.530147+0000","last_undegraded":"2023-10-10T20:35:17.530147+0000","last_fullsized":"2023-10-10T20:35:17.530147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:33.487727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520267+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:11.470026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.531915+0000","last_change":"2023-10-10T20:35:17.531915+0000","last_active":"2023-10-10T20:35:17.531915+0000","last_peered":"2023-10-10T20:35:17.531915+0000","last_clean":"2023-10-10T20:35:17.531915+0000","last_became_active":"2023-10-10T20:35:17.531635+0000","last_became_peered":"2023-10-10T20:35:17.531635+0000","last_unstale":"2023-10-10T20:35:17.531915+0000","last_undegraded":"2023-10-10T20:35:17.531915+0000","last_fullsized":"2023-10-10T20:35:17.531915+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:43:03.119821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515638+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:50.722704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.526248+0000","last_change":"2023-10-10T20:35:17.526248+0000","last_active":"2023-10-10T20:35:17.526248+0000","last_peered":"2023-10-10T20:35:17.526248+0000","last_clean":"2023-10-10T20:35:17.526248+0000","last_became_active":"2023-10-10T20:35:17.525970+0000","last_became_peered":"2023-10-10T20:35:17.525970+0000","last_unstale":"2023-10-10T20:35:17.526248+0000","last_undegraded":"2023-10-10T20:35:17.526248+0000","last_fullsized":"2023-10-10T20:35:17.526248+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:06:00.872875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.525937+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:43.729088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.527453+0000","last_change":"2023-10-10T20:35:17.527453+0000","last_active":"2023-10-10T20:35:17.527453+0000","last_peered":"2023-10-10T20:35:17.527453+0000","last_clean":"2023-10-10T20:35:17.527453+0000","last_became_active":"2023-10-10T20:35:17.527213+0000","last_became_peered":"2023-10-10T20:35:17.527213+0000","last_unstale":"2023-10-10T20:35:17.527453+0000","last_undegraded":"2023-10-10T20:35:17.527453+0000","last_fullsized":"2023-10-10T20:35:17.527453+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:10:09.057062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518404+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:26.305061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530182+0000","last_change":"2023-10-10T20:35:17.530182+0000","last_active":"2023-10-10T20:35:17.530182+0000","last_peered":"2023-10-10T20:35:17.530182+0000","last_clean":"2023-10-10T20:35:17.530182+0000","last_became_active":"2023-10-10T20:35:17.529928+0000","last_became_peered":"2023-10-10T20:35:17.529928+0000","last_unstale":"2023-10-10T20:35:17.530182+0000","last_undegraded":"2023-10-10T20:35:17.530182+0000","last_fullsized":"2023-10-10T20:35:17.530182+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:49.540840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518367+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530696+0000","last_change":"2023-10-10T20:35:17.530696+0000","last_active":"2023-10-10T20:35:17.530696+0000","last_peered":"2023-10-10T20:35:17.530696+0000","last_clean":"2023-10-10T20:35:17.530696+0000","last_became_active":"2023-10-10T20:35:17.530438+0000","last_became_peered":"2023-10-10T20:35:17.530438+0000","last_unstale":"2023-10-10T20:35:17.530696+0000","last_undegraded":"2023-10-10T20:35:17.530696+0000","last_fullsized":"2023-10-10T20:35:17.530696+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:52.866212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.526938+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:18.869383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.524785+0000","last_change":"2023-10-10T20:35:17.524785+0000","last_active":"2023-10-10T20:35:17.524785+0000","last_peered":"2023-10-10T20:35:17.524785+0000","last_clean":"2023-10-10T20:35:17.524785+0000","last_became_active":"2023-10-10T20:35:17.524484+0000","last_became_peered":"2023-10-10T20:35:17.524484+0000","last_unstale":"2023-10-10T20:35:17.524785+0000","last_undegraded":"2023-10-10T20:35:17.524785+0000","last_fullsized":"2023-10-10T20:35:17.524785+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:49.712895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518580+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:21:29.405593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.525379+0000","last_change":"2023-10-10T20:35:17.525379+0000","last_active":"2023-10-10T20:35:17.525379+0000","last_peered":"2023-10-10T20:35:17.525379+0000","last_clean":"2023-10-10T20:35:17.525379+0000","last_became_active":"2023-10-10T20:35:17.525088+0000","last_became_peered":"2023-10-10T20:35:17.525088+0000","last_unstale":"2023-10-10T20:35:17.525379+0000","last_undegraded":"2023-10-10T20:35:17.525379+0000","last_fullsized":"2023-10-10T20:35:17.525379+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:56.281445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519263+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:21.510213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.531414+0000","last_change":"2023-10-10T20:35:17.531414+0000","last_active":"2023-10-10T20:35:17.531414+0000","last_peered":"2023-10-10T20:35:17.531414+0000","last_clean":"2023-10-10T20:35:17.531414+0000","last_became_active":"2023-10-10T20:35:17.531101+0000","last_became_peered":"2023-10-10T20:35:17.531101+0000","last_unstale":"2023-10-10T20:35:17.531414+0000","last_undegraded":"2023-10-10T20:35:17.531414+0000","last_fullsized":"2023-10-10T20:35:17.531414+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:49.540840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.525961+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:41:10.435299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.531613+0000","last_change":"2023-10-10T20:35:17.531613+0000","last_active":"2023-10-10T20:35:17.531613+0000","last_peered":"2023-10-10T20:35:17.531613+0000","last_clean":"2023-10-10T20:35:17.531613+0000","last_became_active":"2023-10-10T20:35:17.531349+0000","last_became_peered":"2023-10-10T20:35:17.531349+0000","last_unstale":"2023-10-10T20:35:17.531613+0000","last_undegraded":"2023-10-10T20:35:17.531613+0000","last_fullsized":"2023-10-10T20:35:17.531613+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:10.460218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.521203+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530785+0000","last_change":"2023-10-10T20:35:17.530785+0000","last_active":"2023-10-10T20:35:17.530785+0000","last_peered":"2023-10-10T20:35:17.530785+0000","last_clean":"2023-10-10T20:35:17.530785+0000","last_became_active":"2023-10-10T20:35:17.530499+0000","last_became_peered":"2023-10-10T20:35:17.530499+0000","last_unstale":"2023-10-10T20:35:17.530785+0000","last_undegraded":"2023-10-10T20:35:17.530785+0000","last_fullsized":"2023-10-10T20:35:17.530785+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:54:55.830016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518624+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:43.729088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529571+0000","last_change":"2023-10-10T20:35:17.529571+0000","last_active":"2023-10-10T20:35:17.529571+0000","last_peered":"2023-10-10T20:35:17.529571+0000","last_clean":"2023-10-10T20:35:17.529571+0000","last_became_active":"2023-10-10T20:35:17.529157+0000","last_became_peered":"2023-10-10T20:35:17.529157+0000","last_unstale":"2023-10-10T20:35:17.529571+0000","last_undegraded":"2023-10-10T20:35:17.529571+0000","last_fullsized":"2023-10-10T20:35:17.529571+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:49.712895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516789+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.526164+0000","last_change":"2023-10-10T20:35:17.526164+0000","last_active":"2023-10-10T20:35:17.526164+0000","last_peered":"2023-10-10T20:35:17.526164+0000","last_clean":"2023-10-10T20:35:17.526164+0000","last_became_active":"2023-10-10T20:35:17.525936+0000","last_became_peered":"2023-10-10T20:35:17.525936+0000","last_unstale":"2023-10-10T20:35:17.526164+0000","last_undegraded":"2023-10-10T20:35:17.526164+0000","last_fullsized":"2023-10-10T20:35:17.526164+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:44.534944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520485+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:11.470026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530627+0000","last_change":"2023-10-10T20:35:17.530627+0000","last_active":"2023-10-10T20:35:17.530627+0000","last_peered":"2023-10-10T20:35:17.530627+0000","last_clean":"2023-10-10T20:35:17.530627+0000","last_became_active":"2023-10-10T20:35:17.530401+0000","last_became_peered":"2023-10-10T20:35:17.530401+0000","last_unstale":"2023-10-10T20:35:17.530627+0000","last_undegraded":"2023-10-10T20:35:17.530627+0000","last_fullsized":"2023-10-10T20:35:17.530627+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:08.528032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520288+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.532471+0000","last_change":"2023-10-10T20:35:17.532471+0000","last_active":"2023-10-10T20:35:17.532471+0000","last_peered":"2023-10-10T20:35:17.532471+0000","last_clean":"2023-10-10T20:35:17.532471+0000","last_became_active":"2023-10-10T20:35:17.532240+0000","last_became_peered":"2023-10-10T20:35:17.532240+0000","last_unstale":"2023-10-10T20:35:17.532471+0000","last_undegraded":"2023-10-10T20:35:17.532471+0000","last_fullsized":"2023-10-10T20:35:17.532471+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:56.281445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.524262+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.531072+0000","last_change":"2023-10-10T20:35:17.531072+0000","last_active":"2023-10-10T20:35:17.531072+0000","last_peered":"2023-10-10T20:35:17.531072+0000","last_clean":"2023-10-10T20:35:17.531072+0000","last_became_active":"2023-10-10T20:35:17.530742+0000","last_became_peered":"2023-10-10T20:35:17.530742+0000","last_unstale":"2023-10-10T20:35:17.531072+0000","last_undegraded":"2023-10-10T20:35:17.531072+0000","last_fullsized":"2023-10-10T20:35:17.531072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:35.655280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.523050+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:09.537841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529197+0000","last_change":"2023-10-10T20:35:17.529197+0000","last_active":"2023-10-10T20:35:17.529197+0000","last_peered":"2023-10-10T20:35:17.529197+0000","last_clean":"2023-10-10T20:35:17.529197+0000","last_became_active":"2023-10-10T20:35:17.528968+0000","last_became_peered":"2023-10-10T20:35:17.528968+0000","last_unstale":"2023-10-10T20:35:17.529197+0000","last_undegraded":"2023-10-10T20:35:17.529197+0000","last_fullsized":"2023-10-10T20:35:17.529197+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:28:58.181797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516255+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:03.137417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.527681+0000","last_change":"2023-10-10T20:35:17.527681+0000","last_active":"2023-10-10T20:35:17.527681+0000","last_peered":"2023-10-10T20:35:17.527681+0000","last_clean":"2023-10-10T20:35:17.527681+0000","last_became_active":"2023-10-10T20:35:17.527396+0000","last_became_peered":"2023-10-10T20:35:17.527396+0000","last_unstale":"2023-10-10T20:35:17.527681+0000","last_undegraded":"2023-10-10T20:35:17.527681+0000","last_fullsized":"2023-10-10T20:35:17.527681+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:36:04.680277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515248+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:09.608602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529192+0000","last_change":"2023-10-10T20:35:17.529192+0000","last_active":"2023-10-10T20:35:17.529192+0000","last_peered":"2023-10-10T20:35:17.529192+0000","last_clean":"2023-10-10T20:35:17.529192+0000","last_became_active":"2023-10-10T20:35:17.528805+0000","last_became_peered":"2023-10-10T20:35:17.528805+0000","last_unstale":"2023-10-10T20:35:17.529192+0000","last_undegraded":"2023-10-10T20:35:17.529192+0000","last_fullsized":"2023-10-10T20:35:17.529192+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:25.295253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515240+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:32.081417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.532567+0000","last_change":"2023-10-10T20:35:17.532567+0000","last_active":"2023-10-10T20:35:17.532567+0000","last_peered":"2023-10-10T20:35:17.532567+0000","last_clean":"2023-10-10T20:35:17.532567+0000","last_became_active":"2023-10-10T20:35:17.532268+0000","last_became_peered":"2023-10-10T20:35:17.532268+0000","last_unstale":"2023-10-10T20:35:17.532567+0000","last_undegraded":"2023-10-10T20:35:17.532567+0000","last_fullsized":"2023-10-10T20:35:17.532567+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:56.281445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515208+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:26:37.371735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529012+0000","last_change":"2023-10-10T20:35:17.529012+0000","last_active":"2023-10-10T20:35:17.529012+0000","last_peered":"2023-10-10T20:35:17.529012+0000","last_clean":"2023-10-10T20:35:17.529012+0000","last_became_active":"2023-10-10T20:35:17.528721+0000","last_became_peered":"2023-10-10T20:35:17.528721+0000","last_unstale":"2023-10-10T20:35:17.529012+0000","last_undegraded":"2023-10-10T20:35:17.529012+0000","last_fullsized":"2023-10-10T20:35:17.529012+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:00:46.029806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516041+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.528749+0000","last_change":"2023-10-10T20:35:17.528749+0000","last_active":"2023-10-10T20:35:17.528749+0000","last_peered":"2023-10-10T20:35:17.528749+0000","last_clean":"2023-10-10T20:35:17.528749+0000","last_became_active":"2023-10-10T20:35:17.528444+0000","last_became_peered":"2023-10-10T20:35:17.528444+0000","last_unstale":"2023-10-10T20:35:17.528749+0000","last_undegraded":"2023-10-10T20:35:17.528749+0000","last_fullsized":"2023-10-10T20:35:17.528749+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:10.460218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515445+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:53:25.938086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.533438+0000","last_change":"2023-10-10T20:35:17.533438+0000","last_active":"2023-10-10T20:35:17.533438+0000","last_peered":"2023-10-10T20:35:17.533438+0000","last_clean":"2023-10-10T20:35:17.533438+0000","last_became_active":"2023-10-10T20:35:17.533184+0000","last_became_peered":"2023-10-10T20:35:17.533184+0000","last_unstale":"2023-10-10T20:35:17.533438+0000","last_undegraded":"2023-10-10T20:35:17.533438+0000","last_fullsized":"2023-10-10T20:35:17.533438+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:25.295253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.526872+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:44:06.887938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529064+0000","last_change":"2023-10-10T20:35:17.529064+0000","last_active":"2023-10-10T20:35:17.529064+0000","last_peered":"2023-10-10T20:35:17.529064+0000","last_clean":"2023-10-10T20:35:17.529064+0000","last_became_active":"2023-10-10T20:35:17.528790+0000","last_became_peered":"2023-10-10T20:35:17.528790+0000","last_unstale":"2023-10-10T20:35:17.529064+0000","last_undegraded":"2023-10-10T20:35:17.529064+0000","last_fullsized":"2023-10-10T20:35:17.529064+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:44:01.198878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518367+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529291+0000","last_change":"2023-10-10T20:35:17.529291+0000","last_active":"2023-10-10T20:35:17.529291+0000","last_peered":"2023-10-10T20:35:17.529291+0000","last_clean":"2023-10-10T20:35:17.529291+0000","last_became_active":"2023-10-10T20:35:17.529040+0000","last_became_peered":"2023-10-10T20:35:17.529040+0000","last_unstale":"2023-10-10T20:35:17.529291+0000","last_undegraded":"2023-10-10T20:35:17.529291+0000","last_fullsized":"2023-10-10T20:35:17.529291+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:50.432471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520235+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530765+0000","last_change":"2023-10-10T20:35:17.530765+0000","last_active":"2023-10-10T20:35:17.530765+0000","last_peered":"2023-10-10T20:35:17.530765+0000","last_clean":"2023-10-10T20:35:17.530765+0000","last_became_active":"2023-10-10T20:35:17.530515+0000","last_became_peered":"2023-10-10T20:35:17.530515+0000","last_unstale":"2023-10-10T20:35:17.530765+0000","last_undegraded":"2023-10-10T20:35:17.530765+0000","last_fullsized":"2023-10-10T20:35:17.530765+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:06:39.021756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.517941+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529415+0000","last_change":"2023-10-10T20:35:17.529415+0000","last_active":"2023-10-10T20:35:17.529415+0000","last_peered":"2023-10-10T20:35:17.529415+0000","last_clean":"2023-10-10T20:35:17.529415+0000","last_became_active":"2023-10-10T20:35:17.528871+0000","last_became_peered":"2023-10-10T20:35:17.528871+0000","last_unstale":"2023-10-10T20:35:17.529415+0000","last_undegraded":"2023-10-10T20:35:17.529415+0000","last_fullsized":"2023-10-10T20:35:17.529415+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:06:39.021756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520480+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:44:06.887938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.533254+0000","last_change":"2023-10-10T20:35:17.533254+0000","last_active":"2023-10-10T20:35:17.533254+0000","last_peered":"2023-10-10T20:35:17.533254+0000","last_clean":"2023-10-10T20:35:17.533254+0000","last_became_active":"2023-10-10T20:35:17.532949+0000","last_became_peered":"2023-10-10T20:35:17.532949+0000","last_unstale":"2023-10-10T20:35:17.533254+0000","last_undegraded":"2023-10-10T20:35:17.533254+0000","last_fullsized":"2023-10-10T20:35:17.533254+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:30:04.234585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.521191+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:06:40.031564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.526551+0000","last_change":"2023-10-10T20:35:17.526551+0000","last_active":"2023-10-10T20:35:17.526551+0000","last_peered":"2023-10-10T20:35:17.526551+0000","last_clean":"2023-10-10T20:35:17.526551+0000","last_became_active":"2023-10-10T20:35:17.526307+0000","last_became_peered":"2023-10-10T20:35:17.526307+0000","last_unstale":"2023-10-10T20:35:17.526551+0000","last_undegraded":"2023-10-10T20:35:17.526551+0000","last_fullsized":"2023-10-10T20:35:17.526551+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:28.452443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516047+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529867+0000","last_change":"2023-10-10T20:35:17.529867+0000","last_active":"2023-10-10T20:35:17.529867+0000","last_peered":"2023-10-10T20:35:17.529867+0000","last_clean":"2023-10-10T20:35:17.529867+0000","last_became_active":"2023-10-10T20:35:17.529596+0000","last_became_peered":"2023-10-10T20:35:17.529596+0000","last_unstale":"2023-10-10T20:35:17.529867+0000","last_undegraded":"2023-10-10T20:35:17.529867+0000","last_fullsized":"2023-10-10T20:35:17.529867+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:06:39.021756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":172,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-10T20:35:16.509270+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:35:16.509270+0000","last_peered":"2023-10-10T20:35:16.509270+0000","last_clean":"2023-10-10T20:35:16.509270+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:35:16.509270+0000","last_undegraded":"2023-10-10T20:35:16.509270+0000","last_fullsized":"2023-10-10T20:35:16.509270+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298172,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073668,"statfs":{"total":96636764160,"available":96331436032,"internally_reserved":0,"allocated":905216,"data_stored":704776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":114408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":120663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298180,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073660,"statfs":{"total":96636764160,"available":96331427840,"internally_reserved":0,"allocated":913408,"data_stored":710070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":120663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.253}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":120663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":114408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":114408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:35:23.310 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2023-10-10T20:35:23.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.521194+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:35:23.310 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2023-10-10T20:35:23.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.521268+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:53:00.556986+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:35:23.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2023-10-10T20:35:23.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.526421+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:51:21.510213+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:35:23.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2023-10-10T20:35:23.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.525593+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:51:21.510213+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:35:23.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2023-10-10T20:35:23.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.516210+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:35:54.808400+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:35:23.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2023-10-10T20:35:23.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.520741+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:11:00.846076+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:35:23.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2023-10-10T20:35:23.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.516606+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T20:41:58.095674+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:35:23.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2023-10-10T20:35:23.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.517832+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:41:10.435299+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:35:23.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2023-10-10T20:35:23.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.519158+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:17:51.442280+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:35:23.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2023-10-10T20:35:23.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.524219+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T20:41:58.095674+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:35:23.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2023-10-10T20:35:23.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.515771+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:35:23.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2023-10-10T20:35:23.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.529654+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:52:43.971144+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:35:23.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2023-10-10T20:35:23.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.515774+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:54:56.839824+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:35:23.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2023-10-10T20:35:23.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.518086+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:35:23.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2023-10-10T20:35:23.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.527879+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:44:44.192355+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:35:23.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2023-10-10T20:35:23.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.517276+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:21:29.405593+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:35:23.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2023-10-10T20:35:23.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.515869+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:34:11.470026+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:35:23.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2023-10-10T20:35:23.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.520829+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:44:44.192355+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:35:23.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2023-10-10T20:35:23.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.519614+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:59:18.869383+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:35:23.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2023-10-10T20:35:23.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.515344+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:38:40.662831+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:35:23.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2023-10-10T20:35:23.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.519688+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:24:50.722704+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:35:23.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2023-10-10T20:35:23.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.520816+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:35:23.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2023-10-10T20:35:23.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.517954+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:35:23.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2023-10-10T20:35:23.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.524527+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:21:29.405593+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:35:23.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2023-10-10T20:35:23.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.524601+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:35:23.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2023-10-10T20:35:23.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.519260+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:41:10.435299+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:35:23.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2023-10-10T20:35:23.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.519727+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:35:23.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2023-10-10T20:35:23.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.527422+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:11:00.846076+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:35:23.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2023-10-10T20:35:23.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.519203+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:18:57.291254+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:35:23.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2023-10-10T20:35:23.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.516633+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:30:09.537841+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:35:23.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2023-10-10T20:35:23.336 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.519899+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:35:23.336 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2023-10-10T20:35:23.337 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.515679+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:35:23.337 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2023-10-10T20:35:23.337 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.519787+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:35:23.338 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2023-10-10T20:35:23.338 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.519924+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:35:23.338 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2023-10-10T20:35:23.339 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.520226+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:35:23.339 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2023-10-10T20:35:23.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.520267+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:34:11.470026+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:35:23.340 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2023-10-10T20:35:23.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.515638+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:24:50.722704+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:35:23.341 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2023-10-10T20:35:23.341 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.525937+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T20:40:43.729088+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:35:23.342 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2023-10-10T20:35:23.342 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.518404+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:07:26.305061+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:35:23.342 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2023-10-10T20:35:23.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.518367+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:35:23.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2023-10-10T20:35:23.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.526938+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:59:18.869383+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:35:23.344 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2023-10-10T20:35:23.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.518580+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:21:29.405593+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:35:23.345 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2023-10-10T20:35:23.345 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.519263+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:51:21.510213+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:35:23.345 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2023-10-10T20:35:23.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.525961+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:41:10.435299+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:35:23.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2023-10-10T20:35:23.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.521203+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:35:23.347 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2023-10-10T20:35:23.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.518624+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T20:40:43.729088+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:35:23.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2023-10-10T20:35:23.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.516789+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:35:23.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2023-10-10T20:35:23.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.520485+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:34:11.470026+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:35:23.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2023-10-10T20:35:23.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.520288+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:35:23.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2023-10-10T20:35:23.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.524262+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:35:23.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2023-10-10T20:35:23.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.523050+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:30:09.537841+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:35:23.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2023-10-10T20:35:23.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.516255+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:34:03.137417+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:35:23.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2023-10-10T20:35:23.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.515248+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:05:09.608602+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:35:23.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2023-10-10T20:35:23.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.515240+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:53:32.081417+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:35:23.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2023-10-10T20:35:23.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.515208+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:26:37.371735+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:35:23.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2023-10-10T20:35:23.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.516041+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:35:23.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2023-10-10T20:35:23.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.515445+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:53:25.938086+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:35:23.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2023-10-10T20:35:23.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.526872+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:44:06.887938+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:35:23.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2023-10-10T20:35:23.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.518367+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:35:23.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2023-10-10T20:35:23.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.520235+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:35:23.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2023-10-10T20:35:23.359 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.517941+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:35:23.359 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2023-10-10T20:35:23.359 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.520480+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:44:06.887938+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:35:23.360 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2023-10-10T20:35:23.360 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.521191+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:06:40.031564+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:35:23.361 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-10-10T20:35:23.361 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:35:17.503982+0000', 'last_change': '2023-10-10T20:35:17.516047+0000', 'last_active': '2023-10-10T20:35:17.503982+0000', 'last_peered': '2023-10-10T20:35:17.503982+0000', 'last_clean': '2023-10-10T20:35:17.503982+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:35:17.503982+0000', 'last_undegraded': '2023-10-10T20:35:17.503982+0000', 'last_fullsized': '2023-10-10T20:35:17.503982+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'last_clean_scrub_stamp': '2023-10-10T20:35:17.503982+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:35:23.362 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-10T20:35:23.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.473+0000 7f4411d83700 1 Processor -- start 2023-10-10T20:35:23.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.473+0000 7f4411d83700 1 -- start start 2023-10-10T20:35:23.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.474+0000 7f4411d83700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f440c0b55a0 0x7f440c164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:23.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.474+0000 7f4411d83700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440c0cfa60 con 0x7f440c0cd910 2023-10-10T20:35:23.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.474+0000 7f4411d83700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440c0b5b00 con 0x7f440c159ec0 2023-10-10T20:35:23.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.474+0000 7f4411d83700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440c0b5da0 con 0x7f440c0b55a0 2023-10-10T20:35:23.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.474+0000 7f440affd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f440c0cd910 0x7f440c0ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47382/0 (socket says 172.21.15.89:47382) 2023-10-10T20:35:23.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.474+0000 7f440affd700 1 -- 172.21.15.89:0/3548041171 learned_addr learned my addr 172.21.15.89:0/3548041171 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:23.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.474+0000 7f440bfff700 1 --2- 172.21.15.89:0/3548041171 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f440c0b55a0 0x7f440c164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:23.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.474+0000 7f440a7fc700 1 -- 172.21.15.89:0/3548041171 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1440791728 0 0) 0x7f440c0cfa60 con 0x7f440c0cd910 2023-10-10T20:35:23.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.474+0000 7f440a7fc700 1 -- 172.21.15.89:0/3548041171 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43f4003570 con 0x7f440c0cd910 2023-10-10T20:35:23.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.474+0000 7f440a7fc700 1 -- 172.21.15.89:0/3548041171 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054444889 0 0) 0x7f440c0b5b00 con 0x7f440c159ec0 2023-10-10T20:35:23.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.474+0000 7f440a7fc700 1 -- 172.21.15.89:0/3548041171 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f440c0cfa60 con 0x7f440c159ec0 2023-10-10T20:35:23.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.474+0000 7f440a7fc700 1 -- 172.21.15.89:0/3548041171 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1121449444 0 0) 0x7f43f4003570 con 0x7f440c0cd910 2023-10-10T20:35:23.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.474+0000 7f440bfff700 1 -- 172.21.15.89:0/3548041171 >> v1:172.21.15.111:6789/0 conn(0x7f440c159ec0 legacy=0x7f440c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:23.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.474+0000 7f440bfff700 1 -- 172.21.15.89:0/3548041171 >> v1:172.21.15.89:6789/0 conn(0x7f440c0cd910 legacy=0x7f440c0ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:23.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.474+0000 7f440bfff700 1 -- 172.21.15.89:0/3548041171 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c0b6080 con 0x7f440c0b55a0 2023-10-10T20:35:23.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.475+0000 7f440bfff700 1 --2- 172.21.15.89:0/3548041171 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f440c0b55a0 0x7f440c164400 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4404009940 tx=0x7f44040091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92fb1a93fb31cd17 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:23.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.475+0000 7f440a7fc700 1 -- 172.21.15.89:0/3548041171 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400001470 con 0x7f440c0b55a0 2023-10-10T20:35:23.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.475+0000 7f440a7fc700 1 -- 172.21.15.89:0/3548041171 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4404011420 con 0x7f440c0b55a0 2023-10-10T20:35:23.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.475+0000 7f440a7fc700 1 -- 172.21.15.89:0/3548041171 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44040117c0 con 0x7f440c0b55a0 2023-10-10T20:35:23.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.475+0000 7f4411d83700 1 -- 172.21.15.89:0/3548041171 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f440c0b55a0 msgr2=0x7f440c164400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:23.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.475+0000 7f4411d83700 1 --2- 172.21.15.89:0/3548041171 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f440c0b55a0 0x7f440c164400 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4404009940 tx=0x7f44040091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.475+0000 7f4411d83700 1 -- 172.21.15.89:0/3548041171 shutdown_connections 2023-10-10T20:35:23.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.475+0000 7f4411d83700 1 --2- 172.21.15.89:0/3548041171 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f440c0b55a0 0x7f440c164400 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.475+0000 7f4411d83700 1 -- 172.21.15.89:0/3548041171 >> 172.21.15.89:0/3548041171 conn(0x7f440c0999f0 msgr2=0x7f440c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:23.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.475+0000 7f4411d83700 1 -- 172.21.15.89:0/3548041171 shutdown_connections 2023-10-10T20:35:23.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.475+0000 7f4411d83700 1 -- 172.21.15.89:0/3548041171 wait complete. 2023-10-10T20:35:23.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.477+0000 7f4411d83700 1 Processor -- start 2023-10-10T20:35:23.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.477+0000 7f4411d83700 1 -- start start 2023-10-10T20:35:23.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.477+0000 7f4411d83700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f440c0b55a0 0x7f440c14ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:23.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.477+0000 7f4411d83700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f440c159ec0 0x7f440c14e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:23.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.477+0000 7f4411d83700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f440c154b30 0x7f440c154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:23.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.477+0000 7f4411d83700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c0694c0 con 0x7f440c159ec0 2023-10-10T20:35:23.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.477+0000 7f4411d83700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c06a040 con 0x7f440c0b55a0 2023-10-10T20:35:23.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.477+0000 7f4411d83700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440c069710 con 0x7f440c154b30 2023-10-10T20:35:23.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.477+0000 7f440affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f440c159ec0 0x7f440c14e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:23.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.477+0000 7f440affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f440c159ec0 0x7f440c14e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46192/0 (socket says 172.21.15.89:46192) 2023-10-10T20:35:23.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.477+0000 7f440affd700 1 -- 172.21.15.89:0/1901512444 learned_addr learned my addr 172.21.15.89:0/1901512444 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:23.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.478+0000 7f440affd700 1 -- 172.21.15.89:0/1901512444 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f440c154b30 msgr2=0x7f440c154fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:23.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.478+0000 7f440affd700 1 --2- 172.21.15.89:0/1901512444 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f440c154b30 0x7f440c154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.478+0000 7f440affd700 1 -- 172.21.15.89:0/1901512444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f440c0b55a0 msgr2=0x7f440c14ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:23.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.478+0000 7f440affd700 1 --2- 172.21.15.89:0/1901512444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f440c0b55a0 0x7f440c14ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.478+0000 7f440affd700 1 -- 172.21.15.89:0/1901512444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4404009420 con 0x7f440c159ec0 2023-10-10T20:35:23.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.478+0000 7f440affd700 1 --2- 172.21.15.89:0/1901512444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f440c159ec0 0x7f440c14e690 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f44000058d0 tx=0x7f4400001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:23.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.478+0000 7f43f37fe700 1 -- 172.21.15.89:0/1901512444 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400005b30 con 0x7f440c159ec0 2023-10-10T20:35:23.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.478+0000 7f43f37fe700 1 -- 172.21.15.89:0/1901512444 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4400005cb0 con 0x7f440c159ec0 2023-10-10T20:35:23.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.478+0000 7f43f37fe700 1 -- 172.21.15.89:0/1901512444 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400007340 con 0x7f440c159ec0 2023-10-10T20:35:23.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.478+0000 7f4411d83700 1 -- 172.21.15.89:0/1901512444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440c0699f0 con 0x7f440c159ec0 2023-10-10T20:35:23.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.478+0000 7f4411d83700 1 -- 172.21.15.89:0/1901512444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440c1554f0 con 0x7f440c159ec0 2023-10-10T20:35:23.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.479+0000 7f4411d83700 1 -- 172.21.15.89:0/1901512444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f440c063c90 con 0x7f440c159ec0 2023-10-10T20:35:23.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.479+0000 7f43f37fe700 1 -- 172.21.15.89:0/1901512444 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f440000a530 con 0x7f440c159ec0 2023-10-10T20:35:23.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.480+0000 7f43f37fe700 1 --2- 172.21.15.89:0/1901512444 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f43f8066f30 0x7f43f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:23.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.480+0000 7f43f37fe700 1 -- 172.21.15.89:0/1901512444 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4400084430 con 0x7f440c159ec0 2023-10-10T20:35:23.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.480+0000 7f440b7fe700 1 --2- 172.21.15.89:0/1901512444 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f43f8066f30 0x7f43f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:23.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.480+0000 7f440b7fe700 1 --2- 172.21.15.89:0/1901512444 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f43f8066f30 0x7f43f80693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f43fc003220 tx=0x7f43fc0027f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:23.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.484+0000 7f43f37fe700 1 -- 172.21.15.89:0/1901512444 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44000516a0 con 0x7f440c159ec0 2023-10-10T20:35:23.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.743+0000 7f4411d83700 1 -- 172.21.15.89:0/1901512444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f440c04c3c0 con 0x7f440c159ec0 2023-10-10T20:35:23.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.743+0000 7f43f37fe700 1 -- 172.21.15.89:0/1901512444 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1035 (secure 0 0 0) 0x7f4400054cc0 con 0x7f440c159ec0 2023-10-10T20:35:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:23.747 INFO:teuthology.orchestra.run.smithi089.stdout:{"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":66,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":31,"num_osds":8,"num_up_osds":8,"osd_up_since":1696970067,"num_in_osds":8,"osd_in_since":1696970059,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2439106560,"bytes_avail":770655006720,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":17,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":4899}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-10-10T20:34:17.134341+0000","services":{}},"progress_events":{}} 2023-10-10T20:35:23.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.745+0000 7f4411d83700 1 -- 172.21.15.89:0/1901512444 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f43f8066f30 msgr2=0x7f43f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:23.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.745+0000 7f4411d83700 1 --2- 172.21.15.89:0/1901512444 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f43f8066f30 0x7f43f80693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f43fc003220 tx=0x7f43fc0027f0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.745+0000 7f4411d83700 1 -- 172.21.15.89:0/1901512444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f440c159ec0 msgr2=0x7f440c14e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:23.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.745+0000 7f4411d83700 1 --2- 172.21.15.89:0/1901512444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f440c159ec0 0x7f440c14e690 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f44000058d0 tx=0x7f4400001ad0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.746+0000 7f4411d83700 1 -- 172.21.15.89:0/1901512444 shutdown_connections 2023-10-10T20:35:23.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.746+0000 7f4411d83700 1 --2- 172.21.15.89:0/1901512444 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f440c154b30 0x7f440c154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.746+0000 7f4411d83700 1 --2- 172.21.15.89:0/1901512444 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f43f8066f30 0x7f43f80693e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.746+0000 7f4411d83700 1 --2- 172.21.15.89:0/1901512444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f440c0b55a0 0x7f440c14ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.746+0000 7f4411d83700 1 --2- 172.21.15.89:0/1901512444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f440c159ec0 0x7f440c14e690 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.746+0000 7f4411d83700 1 -- 172.21.15.89:0/1901512444 >> 172.21.15.89:0/1901512444 conn(0x7f440c0999f0 msgr2=0x7f440c0ce320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:23.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.746+0000 7f4411d83700 1 -- 172.21.15.89:0/1901512444 shutdown_connections 2023-10-10T20:35:23.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.746+0000 7f4411d83700 1 -- 172.21.15.89:0/1901512444 wait complete. 2023-10-10T20:35:23.760 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-10T20:35:23.761 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:35:23.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.872+0000 7f6914f03700 1 Processor -- start 2023-10-10T20:35:23.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.872+0000 7f6914f03700 1 -- start start 2023-10-10T20:35:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.873+0000 7f6914f03700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6910069560 0x7f69100699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.873+0000 7f6914f03700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6910069ee0 con 0x7f6910159890 2023-10-10T20:35:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.873+0000 7f6914f03700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691015a160 con 0x7f6910159c60 2023-10-10T20:35:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.873+0000 7f6914f03700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69100d13a0 con 0x7f6910069560 2023-10-10T20:35:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.873+0000 7f690ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6910069560 0x7f69100699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.873+0000 7f690ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6910069560 0x7f69100699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46202/0 (socket says 172.21.15.89:46202) 2023-10-10T20:35:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.873+0000 7f690e59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6910159890 0x7f691015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47398/0 (socket says 172.21.15.89:47398) 2023-10-10T20:35:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.873+0000 7f690ed9d700 1 -- 172.21.15.89:0/1292794275 learned_addr learned my addr 172.21.15.89:0/1292794275 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.874+0000 7f690ed9d700 1 -- 172.21.15.89:0/1292794275 >> v1:172.21.15.111:6789/0 conn(0x7f6910159c60 legacy=0x7f691015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.874+0000 7f690ed9d700 1 -- 172.21.15.89:0/1292794275 >> v1:172.21.15.89:6789/0 conn(0x7f6910159890 legacy=0x7f691015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.874+0000 7f690ed9d700 1 -- 172.21.15.89:0/1292794275 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691015e140 con 0x7f6910069560 2023-10-10T20:35:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.874+0000 7f690dd9b700 1 -- 172.21.15.89:0/1292794275 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 988275143 0 0) 0x7f6910069ee0 con 0x7f6910159890 2023-10-10T20:35:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.874+0000 7f690ed9d700 1 --2- 172.21.15.89:0/1292794275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6910069560 0x7f69100699a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6908009f20 tx=0x7f6908009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6da051faba9d21 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.874+0000 7f690dd9b700 1 -- 172.21.15.89:0/1292794275 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690800e070 con 0x7f6910069560 2023-10-10T20:35:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.874+0000 7f690dd9b700 1 -- 172.21.15.89:0/1292794275 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690800ba80 con 0x7f6910069560 2023-10-10T20:35:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.874+0000 7f690dd9b700 1 -- 172.21.15.89:0/1292794275 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69080092e0 con 0x7f6910069560 2023-10-10T20:35:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.874+0000 7f6914f03700 1 -- 172.21.15.89:0/1292794275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6910069560 msgr2=0x7f69100699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.874+0000 7f6914f03700 1 --2- 172.21.15.89:0/1292794275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6910069560 0x7f69100699a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6908009f20 tx=0x7f6908009840 comp rx=0 tx=0).stop 2023-10-10T20:35:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.874+0000 7f6914f03700 1 -- 172.21.15.89:0/1292794275 shutdown_connections 2023-10-10T20:35:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.874+0000 7f6914f03700 1 --2- 172.21.15.89:0/1292794275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6910069560 0x7f69100699a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.874+0000 7f6914f03700 1 -- 172.21.15.89:0/1292794275 >> 172.21.15.89:0/1292794275 conn(0x7f6910068e20 msgr2=0x7f69100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.874+0000 7f6914f03700 1 -- 172.21.15.89:0/1292794275 shutdown_connections 2023-10-10T20:35:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.874+0000 7f6914f03700 1 -- 172.21.15.89:0/1292794275 wait complete. 2023-10-10T20:35:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.876+0000 7f6914f03700 1 Processor -- start 2023-10-10T20:35:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.876+0000 7f6914f03700 1 -- start start 2023-10-10T20:35:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.876+0000 7f6914f03700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6910069560 0x7f6910149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.876+0000 7f6914f03700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6910159890 0x7f69100c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.876+0000 7f6914f03700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6910159c60 0x7f69100cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.876+0000 7f6914f03700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69100cde50 con 0x7f6910159890 2023-10-10T20:35:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.876+0000 7f6914f03700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6910149850 con 0x7f6910159c60 2023-10-10T20:35:23.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.876+0000 7f6914f03700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69100cd240 con 0x7f6910069560 2023-10-10T20:35:23.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.876+0000 7f6905d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6910159890 0x7f69100c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:23.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.876+0000 7f6905d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6910159890 0x7f69100c9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46206/0 (socket says 172.21.15.89:46206) 2023-10-10T20:35:23.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.876+0000 7f6905d9b700 1 -- 172.21.15.89:0/4064803207 learned_addr learned my addr 172.21.15.89:0/4064803207 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:23.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.877+0000 7f690ed9d700 1 --2- 172.21.15.89:0/4064803207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6910159c60 0x7f69100cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:23.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.877+0000 7f690e59c700 1 --2- 172.21.15.89:0/4064803207 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6910069560 0x7f6910149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:23.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.877+0000 7f6905d9b700 1 -- 172.21.15.89:0/4064803207 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6910069560 msgr2=0x7f6910149310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:23.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.877+0000 7f6905d9b700 1 --2- 172.21.15.89:0/4064803207 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6910069560 0x7f6910149310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.877+0000 7f6905d9b700 1 -- 172.21.15.89:0/4064803207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6910159c60 msgr2=0x7f69100cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:23.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.877+0000 7f6905d9b700 1 --2- 172.21.15.89:0/4064803207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6910159c60 0x7f69100cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:23.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.877+0000 7f6905d9b700 1 -- 172.21.15.89:0/4064803207 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6908009ae0 con 0x7f6910159890 2023-10-10T20:35:23.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.877+0000 7f690e59c700 1 --2- 172.21.15.89:0/4064803207 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6910069560 0x7f6910149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:23.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.877+0000 7f6905d9b700 1 --2- 172.21.15.89:0/4064803207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6910159890 0x7f69100c9790 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f68f800c4e0 tx=0x7f68f800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:23.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.877+0000 7f6906ffd700 1 -- 172.21.15.89:0/4064803207 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8011590 con 0x7f6910159890 2023-10-10T20:35:23.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.877+0000 7f6906ffd700 1 -- 172.21.15.89:0/4064803207 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f8011710 con 0x7f6910159890 2023-10-10T20:35:23.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.877+0000 7f6914f03700 1 -- 172.21.15.89:0/4064803207 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69100cd520 con 0x7f6910159890 2023-10-10T20:35:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.877+0000 7f6906ffd700 1 -- 172.21.15.89:0/4064803207 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8011ad0 con 0x7f6910159890 2023-10-10T20:35:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.877+0000 7f6914f03700 1 -- 172.21.15.89:0/4064803207 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69100cdae0 con 0x7f6910159890 2023-10-10T20:35:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.878+0000 7f6914f03700 1 -- 172.21.15.89:0/4064803207 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69100b2e40 con 0x7f6910159890 2023-10-10T20:35:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.878+0000 7f6906ffd700 1 -- 172.21.15.89:0/4064803207 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f68f801b3b0 con 0x7f6910159890 2023-10-10T20:35:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.879+0000 7f6906ffd700 1 --2- 172.21.15.89:0/4064803207 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f68f0066f30 0x7f68f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.879+0000 7f690e59c700 1 --2- 172.21.15.89:0/4064803207 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f68f0066f30 0x7f68f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.879+0000 7f6906ffd700 1 -- 172.21.15.89:0/4064803207 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f68f80839f0 con 0x7f6910159890 2023-10-10T20:35:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.879+0000 7f690e59c700 1 --2- 172.21.15.89:0/4064803207 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f68f0066f30 0x7f68f00693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6900000c00 tx=0x7f6900009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:23.883+0000 7f6906ffd700 1 -- 172.21.15.89:0/4064803207 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68f8050c60 con 0x7f6910159890 2023-10-10T20:35:24.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:24.063+0000 7f6914f03700 1 -- 172.21.15.89:0/4064803207 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f691004c3c0 con 0x7f68f0066f30 2023-10-10T20:35:24.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:24.070+0000 7f6906ffd700 1 -- 172.21.15.89:0/4064803207 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+261753 (secure 0 0 0) 0x7f691004c3c0 con 0x7f68f0066f30 2023-10-10T20:35:24.076 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:24.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:24.073+0000 7f6914f03700 1 -- 172.21.15.89:0/4064803207 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f68f0066f30 msgr2=0x7f68f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:24.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:24.073+0000 7f6914f03700 1 --2- 172.21.15.89:0/4064803207 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f68f0066f30 0x7f68f00693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6900000c00 tx=0x7f6900009040 comp rx=0 tx=0).stop 2023-10-10T20:35:24.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:24.073+0000 7f6914f03700 1 -- 172.21.15.89:0/4064803207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6910159890 msgr2=0x7f69100c9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:24.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:24.073+0000 7f6914f03700 1 --2- 172.21.15.89:0/4064803207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6910159890 0x7f69100c9790 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f68f800c4e0 tx=0x7f68f800c8a0 comp rx=0 tx=0).stop 2023-10-10T20:35:24.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:24.073+0000 7f6914f03700 1 -- 172.21.15.89:0/4064803207 shutdown_connections 2023-10-10T20:35:24.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:24.073+0000 7f6914f03700 1 --2- 172.21.15.89:0/4064803207 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f68f0066f30 0x7f68f00693e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:24.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:24.073+0000 7f6914f03700 1 --2- 172.21.15.89:0/4064803207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6910159c60 0x7f69100cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:24.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:24.073+0000 7f6914f03700 1 --2- 172.21.15.89:0/4064803207 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6910069560 0x7f6910149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:24.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:24.073+0000 7f6914f03700 1 --2- 172.21.15.89:0/4064803207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6910159890 0x7f69100c9790 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:24.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:24.073+0000 7f6914f03700 1 -- 172.21.15.89:0/4064803207 >> 172.21.15.89:0/4064803207 conn(0x7f6910068e20 msgr2=0x7f6910155340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:24.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:24.073+0000 7f6914f03700 1 -- 172.21.15.89:0/4064803207 shutdown_connections 2023-10-10T20:35:24.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:24.073+0000 7f6914f03700 1 -- 172.21.15.89:0/4064803207 wait complete. 2023-10-10T20:35:24.081 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:35:24.088 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2023-10-10T20:35:22.660825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381940,"kb_used_data":3636,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592780,"statfs":{"total":773094113280,"available":770655006720,"internally_reserved":0,"allocated":3723264,"data_stored":2120059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000922"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.521194+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.521268+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:00.556986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.526421+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:21.510213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.525593+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:21.510213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516210+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:35:54.808400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520741+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:11:00.846076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516606+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:41:58.095674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.517832+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:41:10.435299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519158+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:51.442280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.524219+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:41:58.095674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515771+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.529654+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:52:43.971144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515774+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:54:56.839824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518086+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.527879+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:44:44.192355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.517276+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:21:29.405593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515869+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:11.470026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520829+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:44:44.192355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519614+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:18.869383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515344+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:40.662831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519688+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:50.722704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520816+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.517954+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.524527+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:21:29.405593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.524601+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519260+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:41:10.435299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519727+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.527422+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:11:00.846076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519203+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:57.291254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516633+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:09.537841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519899+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515679+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519787+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.533417+0000","last_change":"2023-10-10T20:35:17.533417+0000","last_active":"2023-10-10T20:35:17.533417+0000","last_peered":"2023-10-10T20:35:17.533417+0000","last_clean":"2023-10-10T20:35:17.533417+0000","last_became_active":"2023-10-10T20:35:17.533123+0000","last_became_peered":"2023-10-10T20:35:17.533123+0000","last_unstale":"2023-10-10T20:35:17.533417+0000","last_undegraded":"2023-10-10T20:35:17.533417+0000","last_fullsized":"2023-10-10T20:35:17.533417+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:50.432471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519924+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.524198+0000","last_change":"2023-10-10T20:35:17.524198+0000","last_active":"2023-10-10T20:35:17.524198+0000","last_peered":"2023-10-10T20:35:17.524198+0000","last_clean":"2023-10-10T20:35:17.524198+0000","last_became_active":"2023-10-10T20:35:17.523790+0000","last_became_peered":"2023-10-10T20:35:17.523790+0000","last_unstale":"2023-10-10T20:35:17.524198+0000","last_undegraded":"2023-10-10T20:35:17.524198+0000","last_fullsized":"2023-10-10T20:35:17.524198+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:50.432471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520226+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530147+0000","last_change":"2023-10-10T20:35:17.530147+0000","last_active":"2023-10-10T20:35:17.530147+0000","last_peered":"2023-10-10T20:35:17.530147+0000","last_clean":"2023-10-10T20:35:17.530147+0000","last_became_active":"2023-10-10T20:35:17.529880+0000","last_became_peered":"2023-10-10T20:35:17.529880+0000","last_unstale":"2023-10-10T20:35:17.530147+0000","last_undegraded":"2023-10-10T20:35:17.530147+0000","last_fullsized":"2023-10-10T20:35:17.530147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:33.487727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520267+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:11.470026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.531915+0000","last_change":"2023-10-10T20:35:17.531915+0000","last_active":"2023-10-10T20:35:17.531915+0000","last_peered":"2023-10-10T20:35:17.531915+0000","last_clean":"2023-10-10T20:35:17.531915+0000","last_became_active":"2023-10-10T20:35:17.531635+0000","last_became_peered":"2023-10-10T20:35:17.531635+0000","last_unstale":"2023-10-10T20:35:17.531915+0000","last_undegraded":"2023-10-10T20:35:17.531915+0000","last_fullsized":"2023-10-10T20:35:17.531915+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:43:03.119821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515638+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:50.722704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.526248+0000","last_change":"2023-10-10T20:35:17.526248+0000","last_active":"2023-10-10T20:35:17.526248+0000","last_peered":"2023-10-10T20:35:17.526248+0000","last_clean":"2023-10-10T20:35:17.526248+0000","last_became_active":"2023-10-10T20:35:17.525970+0000","last_became_peered":"2023-10-10T20:35:17.525970+0000","last_unstale":"2023-10-10T20:35:17.526248+0000","last_undegraded":"2023-10-10T20:35:17.526248+0000","last_fullsized":"2023-10-10T20:35:17.526248+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:06:00.872875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.525937+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:43.729088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.527453+0000","last_change":"2023-10-10T20:35:17.527453+0000","last_active":"2023-10-10T20:35:17.527453+0000","last_peered":"2023-10-10T20:35:17.527453+0000","last_clean":"2023-10-10T20:35:17.527453+0000","last_became_active":"2023-10-10T20:35:17.527213+0000","last_became_peered":"2023-10-10T20:35:17.527213+0000","last_unstale":"2023-10-10T20:35:17.527453+0000","last_undegraded":"2023-10-10T20:35:17.527453+0000","last_fullsized":"2023-10-10T20:35:17.527453+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:10:09.057062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518404+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:26.305061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530182+0000","last_change":"2023-10-10T20:35:17.530182+0000","last_active":"2023-10-10T20:35:17.530182+0000","last_peered":"2023-10-10T20:35:17.530182+0000","last_clean":"2023-10-10T20:35:17.530182+0000","last_became_active":"2023-10-10T20:35:17.529928+0000","last_became_peered":"2023-10-10T20:35:17.529928+0000","last_unstale":"2023-10-10T20:35:17.530182+0000","last_undegraded":"2023-10-10T20:35:17.530182+0000","last_fullsized":"2023-10-10T20:35:17.530182+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:49.540840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518367+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530696+0000","last_change":"2023-10-10T20:35:17.530696+0000","last_active":"2023-10-10T20:35:17.530696+0000","last_peered":"2023-10-10T20:35:17.530696+0000","last_clean":"2023-10-10T20:35:17.530696+0000","last_became_active":"2023-10-10T20:35:17.530438+0000","last_became_peered":"2023-10-10T20:35:17.530438+0000","last_unstale":"2023-10-10T20:35:17.530696+0000","last_undegraded":"2023-10-10T20:35:17.530696+0000","last_fullsized":"2023-10-10T20:35:17.530696+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:52.866212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.526938+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:18.869383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.524785+0000","last_change":"2023-10-10T20:35:17.524785+0000","last_active":"2023-10-10T20:35:17.524785+0000","last_peered":"2023-10-10T20:35:17.524785+0000","last_clean":"2023-10-10T20:35:17.524785+0000","last_became_active":"2023-10-10T20:35:17.524484+0000","last_became_peered":"2023-10-10T20:35:17.524484+0000","last_unstale":"2023-10-10T20:35:17.524785+0000","last_undegraded":"2023-10-10T20:35:17.524785+0000","last_fullsized":"2023-10-10T20:35:17.524785+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:49.712895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518580+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:21:29.405593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.525379+0000","last_change":"2023-10-10T20:35:17.525379+0000","last_active":"2023-10-10T20:35:17.525379+0000","last_peered":"2023-10-10T20:35:17.525379+0000","last_clean":"2023-10-10T20:35:17.525379+0000","last_became_active":"2023-10-10T20:35:17.525088+0000","last_became_peered":"2023-10-10T20:35:17.525088+0000","last_unstale":"2023-10-10T20:35:17.525379+0000","last_undegraded":"2023-10-10T20:35:17.525379+0000","last_fullsized":"2023-10-10T20:35:17.525379+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:56.281445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.519263+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:21.510213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.531414+0000","last_change":"2023-10-10T20:35:17.531414+0000","last_active":"2023-10-10T20:35:17.531414+0000","last_peered":"2023-10-10T20:35:17.531414+0000","last_clean":"2023-10-10T20:35:17.531414+0000","last_became_active":"2023-10-10T20:35:17.531101+0000","last_became_peered":"2023-10-10T20:35:17.531101+0000","last_unstale":"2023-10-10T20:35:17.531414+0000","last_undegraded":"2023-10-10T20:35:17.531414+0000","last_fullsized":"2023-10-10T20:35:17.531414+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:49.540840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.525961+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:41:10.435299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.531613+0000","last_change":"2023-10-10T20:35:17.531613+0000","last_active":"2023-10-10T20:35:17.531613+0000","last_peered":"2023-10-10T20:35:17.531613+0000","last_clean":"2023-10-10T20:35:17.531613+0000","last_became_active":"2023-10-10T20:35:17.531349+0000","last_became_peered":"2023-10-10T20:35:17.531349+0000","last_unstale":"2023-10-10T20:35:17.531613+0000","last_undegraded":"2023-10-10T20:35:17.531613+0000","last_fullsized":"2023-10-10T20:35:17.531613+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:10.460218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.521203+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530785+0000","last_change":"2023-10-10T20:35:17.530785+0000","last_active":"2023-10-10T20:35:17.530785+0000","last_peered":"2023-10-10T20:35:17.530785+0000","last_clean":"2023-10-10T20:35:17.530785+0000","last_became_active":"2023-10-10T20:35:17.530499+0000","last_became_peered":"2023-10-10T20:35:17.530499+0000","last_unstale":"2023-10-10T20:35:17.530785+0000","last_undegraded":"2023-10-10T20:35:17.530785+0000","last_fullsized":"2023-10-10T20:35:17.530785+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:54:55.830016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518624+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:43.729088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529571+0000","last_change":"2023-10-10T20:35:17.529571+0000","last_active":"2023-10-10T20:35:17.529571+0000","last_peered":"2023-10-10T20:35:17.529571+0000","last_clean":"2023-10-10T20:35:17.529571+0000","last_became_active":"2023-10-10T20:35:17.529157+0000","last_became_peered":"2023-10-10T20:35:17.529157+0000","last_unstale":"2023-10-10T20:35:17.529571+0000","last_undegraded":"2023-10-10T20:35:17.529571+0000","last_fullsized":"2023-10-10T20:35:17.529571+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:49.712895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516789+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.526164+0000","last_change":"2023-10-10T20:35:17.526164+0000","last_active":"2023-10-10T20:35:17.526164+0000","last_peered":"2023-10-10T20:35:17.526164+0000","last_clean":"2023-10-10T20:35:17.526164+0000","last_became_active":"2023-10-10T20:35:17.525936+0000","last_became_peered":"2023-10-10T20:35:17.525936+0000","last_unstale":"2023-10-10T20:35:17.526164+0000","last_undegraded":"2023-10-10T20:35:17.526164+0000","last_fullsized":"2023-10-10T20:35:17.526164+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:44.534944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520485+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:11.470026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530627+0000","last_change":"2023-10-10T20:35:17.530627+0000","last_active":"2023-10-10T20:35:17.530627+0000","last_peered":"2023-10-10T20:35:17.530627+0000","last_clean":"2023-10-10T20:35:17.530627+0000","last_became_active":"2023-10-10T20:35:17.530401+0000","last_became_peered":"2023-10-10T20:35:17.530401+0000","last_unstale":"2023-10-10T20:35:17.530627+0000","last_undegraded":"2023-10-10T20:35:17.530627+0000","last_fullsized":"2023-10-10T20:35:17.530627+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:08.528032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520288+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.532471+0000","last_change":"2023-10-10T20:35:17.532471+0000","last_active":"2023-10-10T20:35:17.532471+0000","last_peered":"2023-10-10T20:35:17.532471+0000","last_clean":"2023-10-10T20:35:17.532471+0000","last_became_active":"2023-10-10T20:35:17.532240+0000","last_became_peered":"2023-10-10T20:35:17.532240+0000","last_unstale":"2023-10-10T20:35:17.532471+0000","last_undegraded":"2023-10-10T20:35:17.532471+0000","last_fullsized":"2023-10-10T20:35:17.532471+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:56.281445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.524262+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.531072+0000","last_change":"2023-10-10T20:35:17.531072+0000","last_active":"2023-10-10T20:35:17.531072+0000","last_peered":"2023-10-10T20:35:17.531072+0000","last_clean":"2023-10-10T20:35:17.531072+0000","last_became_active":"2023-10-10T20:35:17.530742+0000","last_became_peered":"2023-10-10T20:35:17.530742+0000","last_unstale":"2023-10-10T20:35:17.531072+0000","last_undegraded":"2023-10-10T20:35:17.531072+0000","last_fullsized":"2023-10-10T20:35:17.531072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:35.655280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.523050+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:09.537841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529197+0000","last_change":"2023-10-10T20:35:17.529197+0000","last_active":"2023-10-10T20:35:17.529197+0000","last_peered":"2023-10-10T20:35:17.529197+0000","last_clean":"2023-10-10T20:35:17.529197+0000","last_became_active":"2023-10-10T20:35:17.528968+0000","last_became_peered":"2023-10-10T20:35:17.528968+0000","last_unstale":"2023-10-10T20:35:17.529197+0000","last_undegraded":"2023-10-10T20:35:17.529197+0000","last_fullsized":"2023-10-10T20:35:17.529197+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:28:58.181797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516255+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:03.137417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.527681+0000","last_change":"2023-10-10T20:35:17.527681+0000","last_active":"2023-10-10T20:35:17.527681+0000","last_peered":"2023-10-10T20:35:17.527681+0000","last_clean":"2023-10-10T20:35:17.527681+0000","last_became_active":"2023-10-10T20:35:17.527396+0000","last_became_peered":"2023-10-10T20:35:17.527396+0000","last_unstale":"2023-10-10T20:35:17.527681+0000","last_undegraded":"2023-10-10T20:35:17.527681+0000","last_fullsized":"2023-10-10T20:35:17.527681+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:36:04.680277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515248+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:09.608602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529192+0000","last_change":"2023-10-10T20:35:17.529192+0000","last_active":"2023-10-10T20:35:17.529192+0000","last_peered":"2023-10-10T20:35:17.529192+0000","last_clean":"2023-10-10T20:35:17.529192+0000","last_became_active":"2023-10-10T20:35:17.528805+0000","last_became_peered":"2023-10-10T20:35:17.528805+0000","last_unstale":"2023-10-10T20:35:17.529192+0000","last_undegraded":"2023-10-10T20:35:17.529192+0000","last_fullsized":"2023-10-10T20:35:17.529192+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:25.295253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515240+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:32.081417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.532567+0000","last_change":"2023-10-10T20:35:17.532567+0000","last_active":"2023-10-10T20:35:17.532567+0000","last_peered":"2023-10-10T20:35:17.532567+0000","last_clean":"2023-10-10T20:35:17.532567+0000","last_became_active":"2023-10-10T20:35:17.532268+0000","last_became_peered":"2023-10-10T20:35:17.532268+0000","last_unstale":"2023-10-10T20:35:17.532567+0000","last_undegraded":"2023-10-10T20:35:17.532567+0000","last_fullsized":"2023-10-10T20:35:17.532567+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:56.281445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515208+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:26:37.371735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529012+0000","last_change":"2023-10-10T20:35:17.529012+0000","last_active":"2023-10-10T20:35:17.529012+0000","last_peered":"2023-10-10T20:35:17.529012+0000","last_clean":"2023-10-10T20:35:17.529012+0000","last_became_active":"2023-10-10T20:35:17.528721+0000","last_became_peered":"2023-10-10T20:35:17.528721+0000","last_unstale":"2023-10-10T20:35:17.529012+0000","last_undegraded":"2023-10-10T20:35:17.529012+0000","last_fullsized":"2023-10-10T20:35:17.529012+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:00:46.029806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516041+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:29.462252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.528749+0000","last_change":"2023-10-10T20:35:17.528749+0000","last_active":"2023-10-10T20:35:17.528749+0000","last_peered":"2023-10-10T20:35:17.528749+0000","last_clean":"2023-10-10T20:35:17.528749+0000","last_became_active":"2023-10-10T20:35:17.528444+0000","last_became_peered":"2023-10-10T20:35:17.528444+0000","last_unstale":"2023-10-10T20:35:17.528749+0000","last_undegraded":"2023-10-10T20:35:17.528749+0000","last_fullsized":"2023-10-10T20:35:17.528749+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:10.460218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.515445+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:53:25.938086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.533438+0000","last_change":"2023-10-10T20:35:17.533438+0000","last_active":"2023-10-10T20:35:17.533438+0000","last_peered":"2023-10-10T20:35:17.533438+0000","last_clean":"2023-10-10T20:35:17.533438+0000","last_became_active":"2023-10-10T20:35:17.533184+0000","last_became_peered":"2023-10-10T20:35:17.533184+0000","last_unstale":"2023-10-10T20:35:17.533438+0000","last_undegraded":"2023-10-10T20:35:17.533438+0000","last_fullsized":"2023-10-10T20:35:17.533438+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:25.295253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.526872+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:44:06.887938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529064+0000","last_change":"2023-10-10T20:35:17.529064+0000","last_active":"2023-10-10T20:35:17.529064+0000","last_peered":"2023-10-10T20:35:17.529064+0000","last_clean":"2023-10-10T20:35:17.529064+0000","last_became_active":"2023-10-10T20:35:17.528790+0000","last_became_peered":"2023-10-10T20:35:17.528790+0000","last_unstale":"2023-10-10T20:35:17.529064+0000","last_undegraded":"2023-10-10T20:35:17.529064+0000","last_fullsized":"2023-10-10T20:35:17.529064+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:44:01.198878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.518367+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529291+0000","last_change":"2023-10-10T20:35:17.529291+0000","last_active":"2023-10-10T20:35:17.529291+0000","last_peered":"2023-10-10T20:35:17.529291+0000","last_clean":"2023-10-10T20:35:17.529291+0000","last_became_active":"2023-10-10T20:35:17.529040+0000","last_became_peered":"2023-10-10T20:35:17.529040+0000","last_unstale":"2023-10-10T20:35:17.529291+0000","last_undegraded":"2023-10-10T20:35:17.529291+0000","last_fullsized":"2023-10-10T20:35:17.529291+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:50.432471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520235+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:50.550648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.530765+0000","last_change":"2023-10-10T20:35:17.530765+0000","last_active":"2023-10-10T20:35:17.530765+0000","last_peered":"2023-10-10T20:35:17.530765+0000","last_clean":"2023-10-10T20:35:17.530765+0000","last_became_active":"2023-10-10T20:35:17.530515+0000","last_became_peered":"2023-10-10T20:35:17.530515+0000","last_unstale":"2023-10-10T20:35:17.530765+0000","last_undegraded":"2023-10-10T20:35:17.530765+0000","last_fullsized":"2023-10-10T20:35:17.530765+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:06:39.021756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.517941+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:45.544752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529415+0000","last_change":"2023-10-10T20:35:17.529415+0000","last_active":"2023-10-10T20:35:17.529415+0000","last_peered":"2023-10-10T20:35:17.529415+0000","last_clean":"2023-10-10T20:35:17.529415+0000","last_became_active":"2023-10-10T20:35:17.528871+0000","last_became_peered":"2023-10-10T20:35:17.528871+0000","last_unstale":"2023-10-10T20:35:17.529415+0000","last_undegraded":"2023-10-10T20:35:17.529415+0000","last_fullsized":"2023-10-10T20:35:17.529415+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:06:39.021756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.520480+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:44:06.887938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.533254+0000","last_change":"2023-10-10T20:35:17.533254+0000","last_active":"2023-10-10T20:35:17.533254+0000","last_peered":"2023-10-10T20:35:17.533254+0000","last_clean":"2023-10-10T20:35:17.533254+0000","last_became_active":"2023-10-10T20:35:17.532949+0000","last_became_peered":"2023-10-10T20:35:17.532949+0000","last_unstale":"2023-10-10T20:35:17.533254+0000","last_undegraded":"2023-10-10T20:35:17.533254+0000","last_fullsized":"2023-10-10T20:35:17.533254+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:30:04.234585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.521191+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:06:40.031564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.526551+0000","last_change":"2023-10-10T20:35:17.526551+0000","last_active":"2023-10-10T20:35:17.526551+0000","last_peered":"2023-10-10T20:35:17.526551+0000","last_clean":"2023-10-10T20:35:17.526551+0000","last_became_active":"2023-10-10T20:35:17.526307+0000","last_became_peered":"2023-10-10T20:35:17.526307+0000","last_unstale":"2023-10-10T20:35:17.526551+0000","last_undegraded":"2023-10-10T20:35:17.526551+0000","last_fullsized":"2023-10-10T20:35:17.526551+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:28.452443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:35:17.503982+0000","last_change":"2023-10-10T20:35:17.516047+0000","last_active":"2023-10-10T20:35:17.503982+0000","last_peered":"2023-10-10T20:35:17.503982+0000","last_clean":"2023-10-10T20:35:17.503982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:35:17.503982+0000","last_undegraded":"2023-10-10T20:35:17.503982+0000","last_fullsized":"2023-10-10T20:35:17.503982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-10T20:35:17.529867+0000","last_change":"2023-10-10T20:35:17.529867+0000","last_active":"2023-10-10T20:35:17.529867+0000","last_peered":"2023-10-10T20:35:17.529867+0000","last_clean":"2023-10-10T20:35:17.529867+0000","last_became_active":"2023-10-10T20:35:17.529596+0000","last_became_peered":"2023-10-10T20:35:17.529596+0000","last_unstale":"2023-10-10T20:35:17.529867+0000","last_undegraded":"2023-10-10T20:35:17.529867+0000","last_fullsized":"2023-10-10T20:35:17.529867+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:06:39.021756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":172,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-10T20:35:16.509270+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:35:16.509270+0000","last_peered":"2023-10-10T20:35:16.509270+0000","last_clean":"2023-10-10T20:35:16.509270+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:35:16.509270+0000","last_undegraded":"2023-10-10T20:35:16.509270+0000","last_fullsized":"2023-10-10T20:35:16.509270+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298172,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073668,"statfs":{"total":96636764160,"available":96331436032,"internally_reserved":0,"allocated":905216,"data_stored":704776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":114408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":120663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298180,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073660,"statfs":{"total":96636764160,"available":96331427840,"internally_reserved":0,"allocated":913408,"data_stored":710070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":120663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.253}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":120663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":114408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":114408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:35:27.096 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:35:27.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.222+0000 7f9f39e39700 1 Processor -- start 2023-10-10T20:35:27.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.222+0000 7f9f39e39700 1 -- start start 2023-10-10T20:35:27.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.223+0000 7f9f39e39700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f34159420 0x7f9f340b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:27.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.223+0000 7f9f39e39700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9f341597f0 0x7f9f340b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:27.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.223+0000 7f9f39e39700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f340b11d0 con 0x7f9f3415d230 2023-10-10T20:35:27.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.223+0000 7f9f39e39700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f340b14b0 con 0x7f9f341597f0 2023-10-10T20:35:27.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.223+0000 7f9f39e39700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f340b1790 con 0x7f9f34159420 2023-10-10T20:35:27.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.223+0000 7f9f39638700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9f3415d230 0x7f9f340b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47404/0 (socket says 172.21.15.89:47404) 2023-10-10T20:35:27.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.223+0000 7f9f39638700 1 -- 172.21.15.89:0/198807921 learned_addr learned my addr 172.21.15.89:0/198807921 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:27.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.223+0000 7f9f38e37700 1 --2- 172.21.15.89:0/198807921 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f34159420 0x7f9f340b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:27.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.223+0000 7f9f2b7fe700 1 -- 172.21.15.89:0/198807921 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403765533 0 0) 0x7f9f340b11d0 con 0x7f9f3415d230 2023-10-10T20:35:27.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.223+0000 7f9f2bfff700 1 --2- 172.21.15.89:0/198807921 >> v2:172.21.15.111:3300/0 conn(0x7f9f341597f0 0x7f9f340b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:27.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.223+0000 7f9f2b7fe700 1 -- 172.21.15.89:0/198807921 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f1c003570 con 0x7f9f3415d230 2023-10-10T20:35:27.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f2b7fe700 1 -- 172.21.15.89:0/198807921 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2941523670 0 0) 0x7f9f1c003570 con 0x7f9f3415d230 2023-10-10T20:35:27.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f2b7fe700 1 -- 172.21.15.89:0/198807921 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f34159420 msgr2=0x7f9f340b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:27.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f2b7fe700 1 --2- 172.21.15.89:0/198807921 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f34159420 0x7f9f340b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f2b7fe700 1 -- 172.21.15.89:0/198807921 >> v2:172.21.15.111:3300/0 conn(0x7f9f341597f0 msgr2=0x7f9f340b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:27.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f2b7fe700 1 --2- 172.21.15.89:0/198807921 >> v2:172.21.15.111:3300/0 conn(0x7f9f341597f0 0x7f9f340b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f2b7fe700 1 -- 172.21.15.89:0/198807921 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f340b1a70 con 0x7f9f3415d230 2023-10-10T20:35:27.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f2b7fe700 1 -- 172.21.15.89:0/198807921 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f9f2c0017f0 con 0x7f9f3415d230 2023-10-10T20:35:27.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f2b7fe700 1 -- 172.21.15.89:0/198807921 >> v1:172.21.15.89:6789/0 conn(0x7f9f3415d230 legacy=0x7f9f340b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:27.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f2b7fe700 1 --2- 172.21.15.89:0/198807921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f1c003a60 0x7f9f1c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:27.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f2b7fe700 1 --2- 172.21.15.89:0/198807921 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f34159420 0x7f9f1c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:27.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f2b7fe700 1 --2- 172.21.15.89:0/198807921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f1c004d50 0x7f9f1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:27.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f2b7fe700 1 -- 172.21.15.89:0/198807921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f340b14b0 con 0x7f9f1c004d50 2023-10-10T20:35:27.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f2b7fe700 1 -- 172.21.15.89:0/198807921 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f340b1790 con 0x7f9f1c003a60 2023-10-10T20:35:27.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f2b7fe700 1 -- 172.21.15.89:0/198807921 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c00ba60 con 0x7f9f34159420 2023-10-10T20:35:27.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f39638700 1 --2- 172.21.15.89:0/198807921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f1c004d50 0x7f9f1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:27.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f38e37700 1 --2- 172.21.15.89:0/198807921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f1c003a60 0x7f9f1c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:27.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f2bfff700 1 --2- 172.21.15.89:0/198807921 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f34159420 0x7f9f1c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:27.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f39638700 1 -- 172.21.15.89:0/198807921 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f34159420 msgr2=0x7f9f1c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:27.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f39638700 1 --2- 172.21.15.89:0/198807921 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f34159420 0x7f9f1c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f39638700 1 -- 172.21.15.89:0/198807921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f1c003a60 msgr2=0x7f9f1c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:27.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f39638700 1 --2- 172.21.15.89:0/198807921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f1c003a60 0x7f9f1c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f39638700 1 -- 172.21.15.89:0/198807921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c003570 con 0x7f9f1c004d50 2023-10-10T20:35:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f39638700 1 --2- 172.21.15.89:0/198807921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f1c004d50 0x7f9f1c0051c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00a970 tx=0x7f9f2c002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b0c0b29fb61868 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f2b7fe700 1 -- 172.21.15.89:0/198807921 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c012440 con 0x7f9f1c004d50 2023-10-10T20:35:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f2b7fe700 1 -- 172.21.15.89:0/198807921 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c0125c0 con 0x7f9f1c004d50 2023-10-10T20:35:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.224+0000 7f9f2b7fe700 1 -- 172.21.15.89:0/198807921 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c012980 con 0x7f9f1c004d50 2023-10-10T20:35:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.225+0000 7f9f39e39700 1 -- 172.21.15.89:0/198807921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f1c004d50 msgr2=0x7f9f1c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.225+0000 7f9f39e39700 1 --2- 172.21.15.89:0/198807921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f1c004d50 0x7f9f1c0051c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00a970 tx=0x7f9f2c002210 comp rx=0 tx=0).stop 2023-10-10T20:35:27.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.225+0000 7f9f39638700 1 -- 172.21.15.89:0/198807921 reap_dead start 2023-10-10T20:35:27.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.225+0000 7f9f39e39700 1 -- 172.21.15.89:0/198807921 shutdown_connections 2023-10-10T20:35:27.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.225+0000 7f9f39e39700 1 -- 172.21.15.89:0/198807921 >> 172.21.15.89:0/198807921 conn(0x7f9f340d0660 msgr2=0x7f9f340d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:27.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.225+0000 7f9f39e39700 1 -- 172.21.15.89:0/198807921 shutdown_connections 2023-10-10T20:35:27.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.225+0000 7f9f39e39700 1 -- 172.21.15.89:0/198807921 wait complete. 2023-10-10T20:35:27.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.226+0000 7f9f39e39700 1 Processor -- start 2023-10-10T20:35:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.226+0000 7f9f39e39700 1 -- start start 2023-10-10T20:35:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.227+0000 7f9f39e39700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f34140440 0x7f9f34151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.227+0000 7f9f39e39700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f341521d0 0x7f9f34153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.227+0000 7f9f39e39700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f341540a0 0x7f9f3415edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.227+0000 7f9f39e39700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3415ac90 con 0x7f9f34140440 2023-10-10T20:35:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.227+0000 7f9f39e39700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f34152640 con 0x7f9f341540a0 2023-10-10T20:35:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.227+0000 7f9f39e39700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3415e940 con 0x7f9f341521d0 2023-10-10T20:35:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.227+0000 7f9f38e37700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f34140440 0x7f9f34151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.227+0000 7f9f38e37700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f34140440 0x7f9f34151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46220/0 (socket says 172.21.15.89:46220) 2023-10-10T20:35:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.227+0000 7f9f38e37700 1 -- 172.21.15.89:0/814455778 learned_addr learned my addr 172.21.15.89:0/814455778 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.227+0000 7f9f39638700 1 --2- 172.21.15.89:0/814455778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f341540a0 0x7f9f3415edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:27.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.227+0000 7f9f2bfff700 1 --2- 172.21.15.89:0/814455778 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f341521d0 0x7f9f34153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:27.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.227+0000 7f9f38e37700 1 -- 172.21.15.89:0/814455778 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f341521d0 msgr2=0x7f9f34153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:27.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.227+0000 7f9f38e37700 1 --2- 172.21.15.89:0/814455778 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f341521d0 0x7f9f34153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.228+0000 7f9f38e37700 1 -- 172.21.15.89:0/814455778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f341540a0 msgr2=0x7f9f3415edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:27.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.228+0000 7f9f38e37700 1 --2- 172.21.15.89:0/814455778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f341540a0 0x7f9f3415edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.228+0000 7f9f38e37700 1 -- 172.21.15.89:0/814455778 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f34154950 con 0x7f9f34140440 2023-10-10T20:35:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.228+0000 7f9f2bfff700 1 --2- 172.21.15.89:0/814455778 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f341521d0 0x7f9f34153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.228+0000 7f9f38e37700 1 --2- 172.21.15.89:0/814455778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f34140440 0x7f9f34151c90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9f30011b10 tx=0x7f9f300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.228+0000 7f9f28ff9700 1 -- 172.21.15.89:0/814455778 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3000f9f0 con 0x7f9f34140440 2023-10-10T20:35:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.228+0000 7f9f28ff9700 1 -- 172.21.15.89:0/814455778 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3000fb50 con 0x7f9f34140440 2023-10-10T20:35:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.228+0000 7f9f39e39700 1 -- 172.21.15.89:0/814455778 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f34154c30 con 0x7f9f34140440 2023-10-10T20:35:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.228+0000 7f9f28ff9700 1 -- 172.21.15.89:0/814455778 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3000fed0 con 0x7f9f34140440 2023-10-10T20:35:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.228+0000 7f9f39e39700 1 -- 172.21.15.89:0/814455778 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f3415e5d0 con 0x7f9f34140440 2023-10-10T20:35:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.229+0000 7f9f39e39700 1 -- 172.21.15.89:0/814455778 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f3415ec20 con 0x7f9f34140440 2023-10-10T20:35:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.229+0000 7f9f28ff9700 1 -- 172.21.15.89:0/814455778 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f30014650 con 0x7f9f34140440 2023-10-10T20:35:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.230+0000 7f9f28ff9700 1 --2- 172.21.15.89:0/814455778 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9f20066f30 0x7f9f200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.230+0000 7f9f28ff9700 1 -- 172.21.15.89:0/814455778 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9f30018030 con 0x7f9f34140440 2023-10-10T20:35:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.230+0000 7f9f2bfff700 1 --2- 172.21.15.89:0/814455778 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9f20066f30 0x7f9f200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.230+0000 7f9f2bfff700 1 --2- 172.21.15.89:0/814455778 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9f20066f30 0x7f9f200693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9f34158160 tx=0x7f9f24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.234+0000 7f9f28ff9700 1 -- 172.21.15.89:0/814455778 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f30052260 con 0x7f9f34140440 2023-10-10T20:35:27.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.413+0000 7f9f39e39700 1 -- 172.21.15.89:0/814455778 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9f3404c3c0 con 0x7f9f20066f30 2023-10-10T20:35:27.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.420+0000 7f9f28ff9700 1 -- 172.21.15.89:0/814455778 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264613 (secure 0 0 0) 0x7f9f3404c3c0 con 0x7f9f20066f30 2023-10-10T20:35:27.426 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:27.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.423+0000 7f9f39e39700 1 -- 172.21.15.89:0/814455778 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9f20066f30 msgr2=0x7f9f200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:27.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.423+0000 7f9f39e39700 1 --2- 172.21.15.89:0/814455778 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9f20066f30 0x7f9f200693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9f34158160 tx=0x7f9f24008040 comp rx=0 tx=0).stop 2023-10-10T20:35:27.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.423+0000 7f9f39e39700 1 -- 172.21.15.89:0/814455778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f34140440 msgr2=0x7f9f34151c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:27.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.423+0000 7f9f39e39700 1 --2- 172.21.15.89:0/814455778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f34140440 0x7f9f34151c90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9f30011b10 tx=0x7f9f300091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.423+0000 7f9f39e39700 1 -- 172.21.15.89:0/814455778 shutdown_connections 2023-10-10T20:35:27.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.423+0000 7f9f39e39700 1 --2- 172.21.15.89:0/814455778 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9f20066f30 0x7f9f200693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.423+0000 7f9f39e39700 1 --2- 172.21.15.89:0/814455778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f341540a0 0x7f9f3415edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.423+0000 7f9f39e39700 1 --2- 172.21.15.89:0/814455778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f34140440 0x7f9f34151c90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.423+0000 7f9f39e39700 1 --2- 172.21.15.89:0/814455778 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f341521d0 0x7f9f34153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.423+0000 7f9f39e39700 1 -- 172.21.15.89:0/814455778 >> 172.21.15.89:0/814455778 conn(0x7f9f340d0660 msgr2=0x7f9f340b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:27.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.423+0000 7f9f39e39700 1 -- 172.21.15.89:0/814455778 shutdown_connections 2023-10-10T20:35:27.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.423+0000 7f9f39e39700 1 -- 172.21.15.89:0/814455778 wait complete. 2023-10-10T20:35:27.431 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:35:27.437 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2023-10-10T20:35:26.661656+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382244,"kb_used_data":3940,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592476,"statfs":{"total":773094113280,"available":770654695424,"internally_reserved":0,"allocated":4034560,"data_stored":2254099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.147000"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.533273+0000","last_change":"2023-10-10T20:35:18.529769+0000","last_active":"2023-10-10T20:35:18.533273+0000","last_peered":"2023-10-10T20:35:18.533273+0000","last_clean":"2023-10-10T20:35:18.533273+0000","last_became_active":"2023-10-10T20:35:18.529492+0000","last_became_peered":"2023-10-10T20:35:18.529492+0000","last_unstale":"2023-10-10T20:35:18.533273+0000","last_undegraded":"2023-10-10T20:35:18.533273+0000","last_fullsized":"2023-10-10T20:35:18.533273+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:43.729088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.541325+0000","last_change":"2023-10-10T20:35:18.535203+0000","last_active":"2023-10-10T20:35:18.541325+0000","last_peered":"2023-10-10T20:35:18.541325+0000","last_clean":"2023-10-10T20:35:18.541325+0000","last_became_active":"2023-10-10T20:35:18.534936+0000","last_became_peered":"2023-10-10T20:35:18.534936+0000","last_unstale":"2023-10-10T20:35:18.541325+0000","last_undegraded":"2023-10-10T20:35:18.541325+0000","last_fullsized":"2023-10-10T20:35:18.541325+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:52:43.971144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.546427+0000","last_change":"2023-10-10T20:35:18.543100+0000","last_active":"2023-10-10T20:35:18.546427+0000","last_peered":"2023-10-10T20:35:18.546427+0000","last_clean":"2023-10-10T20:35:18.546427+0000","last_became_active":"2023-10-10T20:35:18.542786+0000","last_became_peered":"2023-10-10T20:35:18.542786+0000","last_unstale":"2023-10-10T20:35:18.546427+0000","last_undegraded":"2023-10-10T20:35:18.546427+0000","last_fullsized":"2023-10-10T20:35:18.546427+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:02:09.782878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.542162+0000","last_change":"2023-10-10T20:35:18.534801+0000","last_active":"2023-10-10T20:35:18.542162+0000","last_peered":"2023-10-10T20:35:18.542162+0000","last_clean":"2023-10-10T20:35:18.542162+0000","last_became_active":"2023-10-10T20:35:18.534637+0000","last_became_peered":"2023-10-10T20:35:18.534637+0000","last_unstale":"2023-10-10T20:35:18.542162+0000","last_undegraded":"2023-10-10T20:35:18.542162+0000","last_fullsized":"2023-10-10T20:35:18.542162+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:18.869383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.540113+0000","last_change":"2023-10-10T20:35:18.529770+0000","last_active":"2023-10-10T20:35:18.540113+0000","last_peered":"2023-10-10T20:35:18.540113+0000","last_clean":"2023-10-10T20:35:18.540113+0000","last_became_active":"2023-10-10T20:35:18.529535+0000","last_became_peered":"2023-10-10T20:35:18.529535+0000","last_unstale":"2023-10-10T20:35:18.540113+0000","last_undegraded":"2023-10-10T20:35:18.540113+0000","last_fullsized":"2023-10-10T20:35:18.540113+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:35:00.918120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.541044+0000","last_change":"2023-10-10T20:35:18.527408+0000","last_active":"2023-10-10T20:35:18.541044+0000","last_peered":"2023-10-10T20:35:18.541044+0000","last_clean":"2023-10-10T20:35:18.541044+0000","last_became_active":"2023-10-10T20:35:18.527135+0000","last_became_peered":"2023-10-10T20:35:18.527135+0000","last_unstale":"2023-10-10T20:35:18.541044+0000","last_undegraded":"2023-10-10T20:35:18.541044+0000","last_fullsized":"2023-10-10T20:35:18.541044+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:36:43.656762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.536738+0000","last_change":"2023-10-10T20:35:18.533197+0000","last_active":"2023-10-10T20:35:18.536738+0000","last_peered":"2023-10-10T20:35:18.536738+0000","last_clean":"2023-10-10T20:35:18.536738+0000","last_became_active":"2023-10-10T20:35:18.532270+0000","last_became_peered":"2023-10-10T20:35:18.532270+0000","last_unstale":"2023-10-10T20:35:18.536738+0000","last_undegraded":"2023-10-10T20:35:18.536738+0000","last_fullsized":"2023-10-10T20:35:18.536738+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:11:49.032158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.535447+0000","last_change":"2023-10-10T20:35:18.528784+0000","last_active":"2023-10-10T20:35:18.535447+0000","last_peered":"2023-10-10T20:35:18.535447+0000","last_clean":"2023-10-10T20:35:18.535447+0000","last_became_active":"2023-10-10T20:35:18.528153+0000","last_became_peered":"2023-10-10T20:35:18.528153+0000","last_unstale":"2023-10-10T20:35:18.535447+0000","last_undegraded":"2023-10-10T20:35:18.535447+0000","last_fullsized":"2023-10-10T20:35:18.535447+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:11:00.846076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.548948+0000","last_change":"2023-10-10T20:35:18.542966+0000","last_active":"2023-10-10T20:35:18.548948+0000","last_peered":"2023-10-10T20:35:18.548948+0000","last_clean":"2023-10-10T20:35:18.548948+0000","last_became_active":"2023-10-10T20:35:18.542757+0000","last_became_peered":"2023-10-10T20:35:18.542757+0000","last_unstale":"2023-10-10T20:35:18.548948+0000","last_undegraded":"2023-10-10T20:35:18.548948+0000","last_fullsized":"2023-10-10T20:35:18.548948+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:43.729088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.546594+0000","last_change":"2023-10-10T20:35:18.545937+0000","last_active":"2023-10-10T20:35:18.546594+0000","last_peered":"2023-10-10T20:35:18.546594+0000","last_clean":"2023-10-10T20:35:18.546594+0000","last_became_active":"2023-10-10T20:35:18.545656+0000","last_became_peered":"2023-10-10T20:35:18.545656+0000","last_unstale":"2023-10-10T20:35:18.546594+0000","last_undegraded":"2023-10-10T20:35:18.546594+0000","last_fullsized":"2023-10-10T20:35:18.546594+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:11:00.846076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.540734+0000","last_change":"2023-10-10T20:35:18.540074+0000","last_active":"2023-10-10T20:35:18.540734+0000","last_peered":"2023-10-10T20:35:18.540734+0000","last_clean":"2023-10-10T20:35:18.540734+0000","last_became_active":"2023-10-10T20:35:18.539787+0000","last_became_peered":"2023-10-10T20:35:18.539787+0000","last_unstale":"2023-10-10T20:35:18.540734+0000","last_undegraded":"2023-10-10T20:35:18.540734+0000","last_fullsized":"2023-10-10T20:35:18.540734+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:31:25.404176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.536464+0000","last_change":"2023-10-10T20:35:18.532352+0000","last_active":"2023-10-10T20:35:18.536464+0000","last_peered":"2023-10-10T20:35:18.536464+0000","last_clean":"2023-10-10T20:35:18.536464+0000","last_became_active":"2023-10-10T20:35:18.532084+0000","last_became_peered":"2023-10-10T20:35:18.532084+0000","last_unstale":"2023-10-10T20:35:18.536464+0000","last_undegraded":"2023-10-10T20:35:18.536464+0000","last_fullsized":"2023-10-10T20:35:18.536464+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:34:18.583957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.541797+0000","last_change":"2023-10-10T20:35:18.540613+0000","last_active":"2023-10-10T20:35:18.541797+0000","last_peered":"2023-10-10T20:35:18.541797+0000","last_clean":"2023-10-10T20:35:18.541797+0000","last_became_active":"2023-10-10T20:35:18.540224+0000","last_became_peered":"2023-10-10T20:35:18.540224+0000","last_unstale":"2023-10-10T20:35:18.541797+0000","last_undegraded":"2023-10-10T20:35:18.541797+0000","last_fullsized":"2023-10-10T20:35:18.541797+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:36.665089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.541062+0000","last_change":"2023-10-10T20:35:18.528417+0000","last_active":"2023-10-10T20:35:18.541062+0000","last_peered":"2023-10-10T20:35:18.541062+0000","last_clean":"2023-10-10T20:35:18.541062+0000","last_became_active":"2023-10-10T20:35:18.528132+0000","last_became_peered":"2023-10-10T20:35:18.528132+0000","last_unstale":"2023-10-10T20:35:18.541062+0000","last_undegraded":"2023-10-10T20:35:18.541062+0000","last_fullsized":"2023-10-10T20:35:18.541062+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:11:49.032158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.536499+0000","last_change":"2023-10-10T20:35:18.531067+0000","last_active":"2023-10-10T20:35:18.536499+0000","last_peered":"2023-10-10T20:35:18.536499+0000","last_clean":"2023-10-10T20:35:18.536499+0000","last_became_active":"2023-10-10T20:35:18.530818+0000","last_became_peered":"2023-10-10T20:35:18.530818+0000","last_unstale":"2023-10-10T20:35:18.536499+0000","last_undegraded":"2023-10-10T20:35:18.536499+0000","last_fullsized":"2023-10-10T20:35:18.536499+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:36.755244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.532328+0000","last_change":"2023-10-10T20:35:18.527874+0000","last_active":"2023-10-10T20:35:18.532328+0000","last_peered":"2023-10-10T20:35:18.532328+0000","last_clean":"2023-10-10T20:35:18.532328+0000","last_became_active":"2023-10-10T20:35:18.526634+0000","last_became_peered":"2023-10-10T20:35:18.526634+0000","last_unstale":"2023-10-10T20:35:18.532328+0000","last_undegraded":"2023-10-10T20:35:18.532328+0000","last_fullsized":"2023-10-10T20:35:18.532328+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:44:44.192355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.536728+0000","last_change":"2023-10-10T20:35:18.532507+0000","last_active":"2023-10-10T20:35:18.536728+0000","last_peered":"2023-10-10T20:35:18.536728+0000","last_clean":"2023-10-10T20:35:18.536728+0000","last_became_active":"2023-10-10T20:35:18.532189+0000","last_became_peered":"2023-10-10T20:35:18.532189+0000","last_unstale":"2023-10-10T20:35:18.536728+0000","last_undegraded":"2023-10-10T20:35:18.536728+0000","last_fullsized":"2023-10-10T20:35:18.536728+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:52:43.971144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.545610+0000","last_change":"2023-10-10T20:35:18.544694+0000","last_active":"2023-10-10T20:35:18.545610+0000","last_peered":"2023-10-10T20:35:18.545610+0000","last_clean":"2023-10-10T20:35:18.545610+0000","last_became_active":"2023-10-10T20:35:18.544442+0000","last_became_peered":"2023-10-10T20:35:18.544442+0000","last_unstale":"2023-10-10T20:35:18.545610+0000","last_undegraded":"2023-10-10T20:35:18.545610+0000","last_fullsized":"2023-10-10T20:35:18.545610+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:18:13.506756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.542058+0000","last_change":"2023-10-10T20:35:18.539588+0000","last_active":"2023-10-10T20:35:18.542058+0000","last_peered":"2023-10-10T20:35:18.542058+0000","last_clean":"2023-10-10T20:35:18.542058+0000","last_became_active":"2023-10-10T20:35:18.539116+0000","last_became_peered":"2023-10-10T20:35:18.539116+0000","last_unstale":"2023-10-10T20:35:18.542058+0000","last_undegraded":"2023-10-10T20:35:18.542058+0000","last_fullsized":"2023-10-10T20:35:18.542058+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:02:09.782878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.536574+0000","last_change":"2023-10-10T20:35:18.526699+0000","last_active":"2023-10-10T20:35:18.536574+0000","last_peered":"2023-10-10T20:35:18.536574+0000","last_clean":"2023-10-10T20:35:18.536574+0000","last_became_active":"2023-10-10T20:35:18.526510+0000","last_became_peered":"2023-10-10T20:35:18.526510+0000","last_unstale":"2023-10-10T20:35:18.536574+0000","last_undegraded":"2023-10-10T20:35:18.536574+0000","last_fullsized":"2023-10-10T20:35:18.536574+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:00:47.039614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.534099+0000","last_change":"2023-10-10T20:35:18.532082+0000","last_active":"2023-10-10T20:35:18.534099+0000","last_peered":"2023-10-10T20:35:18.534099+0000","last_clean":"2023-10-10T20:35:18.534099+0000","last_became_active":"2023-10-10T20:35:18.531826+0000","last_became_peered":"2023-10-10T20:35:18.531826+0000","last_unstale":"2023-10-10T20:35:18.534099+0000","last_undegraded":"2023-10-10T20:35:18.534099+0000","last_fullsized":"2023-10-10T20:35:18.534099+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:11.470026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.540383+0000","last_change":"2023-10-10T20:35:18.537799+0000","last_active":"2023-10-10T20:35:18.540383+0000","last_peered":"2023-10-10T20:35:18.540383+0000","last_clean":"2023-10-10T20:35:18.540383+0000","last_became_active":"2023-10-10T20:35:18.537512+0000","last_became_peered":"2023-10-10T20:35:18.537512+0000","last_unstale":"2023-10-10T20:35:18.540383+0000","last_undegraded":"2023-10-10T20:35:18.540383+0000","last_fullsized":"2023-10-10T20:35:18.540383+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:41:58.095674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.536847+0000","last_change":"2023-10-10T20:35:18.532311+0000","last_active":"2023-10-10T20:35:18.536847+0000","last_peered":"2023-10-10T20:35:18.536847+0000","last_clean":"2023-10-10T20:35:18.536847+0000","last_became_active":"2023-10-10T20:35:18.532052+0000","last_became_peered":"2023-10-10T20:35:18.532052+0000","last_unstale":"2023-10-10T20:35:18.536847+0000","last_undegraded":"2023-10-10T20:35:18.536847+0000","last_fullsized":"2023-10-10T20:35:18.536847+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:11.470026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.531192+0000","last_change":"2023-10-10T20:35:18.528155+0000","last_active":"2023-10-10T20:35:18.531192+0000","last_peered":"2023-10-10T20:35:18.531192+0000","last_clean":"2023-10-10T20:35:18.531192+0000","last_became_active":"2023-10-10T20:35:18.527837+0000","last_became_peered":"2023-10-10T20:35:18.527837+0000","last_unstale":"2023-10-10T20:35:18.531192+0000","last_undegraded":"2023-10-10T20:35:18.531192+0000","last_fullsized":"2023-10-10T20:35:18.531192+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:00.556986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.542728+0000","last_change":"2023-10-10T20:35:18.540188+0000","last_active":"2023-10-10T20:35:18.542728+0000","last_peered":"2023-10-10T20:35:18.542728+0000","last_clean":"2023-10-10T20:35:18.542728+0000","last_became_active":"2023-10-10T20:35:18.539925+0000","last_became_peered":"2023-10-10T20:35:18.539925+0000","last_unstale":"2023-10-10T20:35:18.542728+0000","last_undegraded":"2023-10-10T20:35:18.542728+0000","last_fullsized":"2023-10-10T20:35:18.542728+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:44:44.192355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.541225+0000","last_change":"2023-10-10T20:35:18.529666+0000","last_active":"2023-10-10T20:35:18.541225+0000","last_peered":"2023-10-10T20:35:18.541225+0000","last_clean":"2023-10-10T20:35:18.541225+0000","last_became_active":"2023-10-10T20:35:18.529447+0000","last_became_peered":"2023-10-10T20:35:18.529447+0000","last_unstale":"2023-10-10T20:35:18.541225+0000","last_undegraded":"2023-10-10T20:35:18.541225+0000","last_fullsized":"2023-10-10T20:35:18.541225+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:31:25.404176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.543590+0000","last_change":"2023-10-10T20:35:18.540912+0000","last_active":"2023-10-10T20:35:18.543590+0000","last_peered":"2023-10-10T20:35:18.543590+0000","last_clean":"2023-10-10T20:35:18.543590+0000","last_became_active":"2023-10-10T20:35:18.539949+0000","last_became_peered":"2023-10-10T20:35:18.539949+0000","last_unstale":"2023-10-10T20:35:18.543590+0000","last_undegraded":"2023-10-10T20:35:18.543590+0000","last_fullsized":"2023-10-10T20:35:18.543590+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:21:29.405593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.527090+0000","last_change":"2023-10-10T20:35:18.522767+0000","last_active":"2023-10-10T20:35:18.527090+0000","last_peered":"2023-10-10T20:35:18.527090+0000","last_clean":"2023-10-10T20:35:18.527090+0000","last_became_active":"2023-10-10T20:35:18.522502+0000","last_became_peered":"2023-10-10T20:35:18.522502+0000","last_unstale":"2023-10-10T20:35:18.527090+0000","last_undegraded":"2023-10-10T20:35:18.527090+0000","last_fullsized":"2023-10-10T20:35:18.527090+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:36:43.656762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.540646+0000","last_change":"2023-10-10T20:35:18.539161+0000","last_active":"2023-10-10T20:35:18.540646+0000","last_peered":"2023-10-10T20:35:18.540646+0000","last_clean":"2023-10-10T20:35:18.540646+0000","last_became_active":"2023-10-10T20:35:18.538905+0000","last_became_peered":"2023-10-10T20:35:18.538905+0000","last_unstale":"2023-10-10T20:35:18.540646+0000","last_undegraded":"2023-10-10T20:35:18.540646+0000","last_fullsized":"2023-10-10T20:35:18.540646+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:09.537841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.536689+0000","last_change":"2023-10-10T20:35:18.532450+0000","last_active":"2023-10-10T20:35:18.536689+0000","last_peered":"2023-10-10T20:35:18.536689+0000","last_clean":"2023-10-10T20:35:18.536689+0000","last_became_active":"2023-10-10T20:35:18.532093+0000","last_became_peered":"2023-10-10T20:35:18.532093+0000","last_unstale":"2023-10-10T20:35:18.536689+0000","last_undegraded":"2023-10-10T20:35:18.536689+0000","last_fullsized":"2023-10-10T20:35:18.536689+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:18.869383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.541175+0000","last_change":"2023-10-10T20:35:18.539480+0000","last_active":"2023-10-10T20:35:18.541175+0000","last_peered":"2023-10-10T20:35:18.541175+0000","last_clean":"2023-10-10T20:35:18.541175+0000","last_became_active":"2023-10-10T20:35:18.539162+0000","last_became_peered":"2023-10-10T20:35:18.539162+0000","last_unstale":"2023-10-10T20:35:18.541175+0000","last_undegraded":"2023-10-10T20:35:18.541175+0000","last_fullsized":"2023-10-10T20:35:18.541175+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:09.537841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.535403+0000","last_change":"2023-10-10T20:35:18.528419+0000","last_active":"2023-10-10T20:35:18.535403+0000","last_peered":"2023-10-10T20:35:18.535403+0000","last_clean":"2023-10-10T20:35:18.535403+0000","last_became_active":"2023-10-10T20:35:18.526973+0000","last_became_peered":"2023-10-10T20:35:18.526973+0000","last_unstale":"2023-10-10T20:35:18.535403+0000","last_undegraded":"2023-10-10T20:35:18.535403+0000","last_fullsized":"2023-10-10T20:35:18.535403+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:44:06.887938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.536848+0000","last_change":"2023-10-10T20:35:18.532939+0000","last_active":"2023-10-10T20:35:18.536848+0000","last_peered":"2023-10-10T20:35:18.536848+0000","last_clean":"2023-10-10T20:35:18.536848+0000","last_became_active":"2023-10-10T20:35:18.532630+0000","last_became_peered":"2023-10-10T20:35:18.532630+0000","last_unstale":"2023-10-10T20:35:18.536848+0000","last_undegraded":"2023-10-10T20:35:18.536848+0000","last_fullsized":"2023-10-10T20:35:18.536848+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'4","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.539271+0000","last_change":"2023-10-10T20:35:17.533417+0000","last_active":"2023-10-10T20:35:18.539271+0000","last_peered":"2023-10-10T20:35:18.539271+0000","last_clean":"2023-10-10T20:35:18.539271+0000","last_became_active":"2023-10-10T20:35:17.533123+0000","last_became_peered":"2023-10-10T20:35:17.533123+0000","last_unstale":"2023-10-10T20:35:18.539271+0000","last_undegraded":"2023-10-10T20:35:18.539271+0000","last_fullsized":"2023-10-10T20:35:18.539271+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:50.432471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.537651+0000","last_change":"2023-10-10T20:35:18.534009+0000","last_active":"2023-10-10T20:35:18.537651+0000","last_peered":"2023-10-10T20:35:18.537651+0000","last_clean":"2023-10-10T20:35:18.537651+0000","last_became_active":"2023-10-10T20:35:18.531443+0000","last_became_peered":"2023-10-10T20:35:18.531443+0000","last_unstale":"2023-10-10T20:35:18.537651+0000","last_undegraded":"2023-10-10T20:35:18.537651+0000","last_fullsized":"2023-10-10T20:35:18.537651+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.541789+0000","last_change":"2023-10-10T20:35:17.524198+0000","last_active":"2023-10-10T20:35:18.541789+0000","last_peered":"2023-10-10T20:35:18.541789+0000","last_clean":"2023-10-10T20:35:18.541789+0000","last_became_active":"2023-10-10T20:35:17.523790+0000","last_became_peered":"2023-10-10T20:35:17.523790+0000","last_unstale":"2023-10-10T20:35:18.541789+0000","last_undegraded":"2023-10-10T20:35:18.541789+0000","last_fullsized":"2023-10-10T20:35:18.541789+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:50.432471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.540363+0000","last_change":"2023-10-10T20:35:18.537751+0000","last_active":"2023-10-10T20:35:18.540363+0000","last_peered":"2023-10-10T20:35:18.540363+0000","last_clean":"2023-10-10T20:35:18.540363+0000","last_became_active":"2023-10-10T20:35:18.537475+0000","last_became_peered":"2023-10-10T20:35:18.537475+0000","last_unstale":"2023-10-10T20:35:18.540363+0000","last_undegraded":"2023-10-10T20:35:18.540363+0000","last_fullsized":"2023-10-10T20:35:18.540363+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:34.497536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.545022+0000","last_change":"2023-10-10T20:35:17.530147+0000","last_active":"2023-10-10T20:35:18.545022+0000","last_peered":"2023-10-10T20:35:18.545022+0000","last_clean":"2023-10-10T20:35:18.545022+0000","last_became_active":"2023-10-10T20:35:17.529880+0000","last_became_peered":"2023-10-10T20:35:17.529880+0000","last_unstale":"2023-10-10T20:35:18.545022+0000","last_undegraded":"2023-10-10T20:35:18.545022+0000","last_fullsized":"2023-10-10T20:35:18.545022+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:33.487727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.543578+0000","last_change":"2023-10-10T20:35:18.540350+0000","last_active":"2023-10-10T20:35:18.543578+0000","last_peered":"2023-10-10T20:35:18.543578+0000","last_clean":"2023-10-10T20:35:18.543578+0000","last_became_active":"2023-10-10T20:35:18.539657+0000","last_became_peered":"2023-10-10T20:35:18.539657+0000","last_unstale":"2023-10-10T20:35:18.543578+0000","last_undegraded":"2023-10-10T20:35:18.543578+0000","last_fullsized":"2023-10-10T20:35:18.543578+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:41:10.435299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.543051+0000","last_change":"2023-10-10T20:35:17.531915+0000","last_active":"2023-10-10T20:35:18.543051+0000","last_peered":"2023-10-10T20:35:18.543051+0000","last_clean":"2023-10-10T20:35:18.543051+0000","last_became_active":"2023-10-10T20:35:17.531635+0000","last_became_peered":"2023-10-10T20:35:17.531635+0000","last_unstale":"2023-10-10T20:35:18.543051+0000","last_undegraded":"2023-10-10T20:35:18.543051+0000","last_fullsized":"2023-10-10T20:35:18.543051+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:43:03.119821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.529523+0000","last_change":"2023-10-10T20:35:18.527237+0000","last_active":"2023-10-10T20:35:18.529523+0000","last_peered":"2023-10-10T20:35:18.529523+0000","last_clean":"2023-10-10T20:35:18.529523+0000","last_became_active":"2023-10-10T20:35:18.526919+0000","last_became_peered":"2023-10-10T20:35:18.526919+0000","last_unstale":"2023-10-10T20:35:18.529523+0000","last_undegraded":"2023-10-10T20:35:18.529523+0000","last_fullsized":"2023-10-10T20:35:18.529523+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:36.755244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.544922+0000","last_change":"2023-10-10T20:35:17.526248+0000","last_active":"2023-10-10T20:35:18.544922+0000","last_peered":"2023-10-10T20:35:18.544922+0000","last_clean":"2023-10-10T20:35:18.544922+0000","last_became_active":"2023-10-10T20:35:17.525970+0000","last_became_peered":"2023-10-10T20:35:17.525970+0000","last_unstale":"2023-10-10T20:35:18.544922+0000","last_undegraded":"2023-10-10T20:35:18.544922+0000","last_fullsized":"2023-10-10T20:35:18.544922+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:06:00.872875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.536436+0000","last_change":"2023-10-10T20:35:18.531906+0000","last_active":"2023-10-10T20:35:18.536436+0000","last_peered":"2023-10-10T20:35:18.536436+0000","last_clean":"2023-10-10T20:35:18.536436+0000","last_became_active":"2023-10-10T20:35:18.531656+0000","last_became_peered":"2023-10-10T20:35:18.531656+0000","last_unstale":"2023-10-10T20:35:18.536436+0000","last_undegraded":"2023-10-10T20:35:18.536436+0000","last_fullsized":"2023-10-10T20:35:18.536436+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:26:58.211461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.544893+0000","last_change":"2023-10-10T20:35:17.527453+0000","last_active":"2023-10-10T20:35:18.544893+0000","last_peered":"2023-10-10T20:35:18.544893+0000","last_clean":"2023-10-10T20:35:18.544893+0000","last_became_active":"2023-10-10T20:35:17.527213+0000","last_became_peered":"2023-10-10T20:35:17.527213+0000","last_unstale":"2023-10-10T20:35:18.544893+0000","last_undegraded":"2023-10-10T20:35:18.544893+0000","last_fullsized":"2023-10-10T20:35:18.544893+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:10:09.057062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.535856+0000","last_change":"2023-10-10T20:35:18.533067+0000","last_active":"2023-10-10T20:35:18.535856+0000","last_peered":"2023-10-10T20:35:18.535856+0000","last_clean":"2023-10-10T20:35:18.535856+0000","last_became_active":"2023-10-10T20:35:18.532802+0000","last_became_peered":"2023-10-10T20:35:18.532802+0000","last_unstale":"2023-10-10T20:35:18.535856+0000","last_undegraded":"2023-10-10T20:35:18.535856+0000","last_fullsized":"2023-10-10T20:35:18.535856+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:55.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.523967+0000","last_change":"2023-10-10T20:35:17.530182+0000","last_active":"2023-10-10T20:35:18.523967+0000","last_peered":"2023-10-10T20:35:18.523967+0000","last_clean":"2023-10-10T20:35:18.523967+0000","last_became_active":"2023-10-10T20:35:17.529928+0000","last_became_peered":"2023-10-10T20:35:17.529928+0000","last_unstale":"2023-10-10T20:35:18.523967+0000","last_undegraded":"2023-10-10T20:35:18.523967+0000","last_fullsized":"2023-10-10T20:35:18.523967+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:49.540840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.542684+0000","last_change":"2023-10-10T20:35:18.540122+0000","last_active":"2023-10-10T20:35:18.542684+0000","last_peered":"2023-10-10T20:35:18.542684+0000","last_clean":"2023-10-10T20:35:18.542684+0000","last_became_active":"2023-10-10T20:35:18.538426+0000","last_became_peered":"2023-10-10T20:35:18.538426+0000","last_unstale":"2023-10-10T20:35:18.542684+0000","last_undegraded":"2023-10-10T20:35:18.542684+0000","last_fullsized":"2023-10-10T20:35:18.542684+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:43.729088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.522460+0000","last_change":"2023-10-10T20:35:17.530696+0000","last_active":"2023-10-10T20:35:18.522460+0000","last_peered":"2023-10-10T20:35:18.522460+0000","last_clean":"2023-10-10T20:35:18.522460+0000","last_became_active":"2023-10-10T20:35:17.530438+0000","last_became_peered":"2023-10-10T20:35:17.530438+0000","last_unstale":"2023-10-10T20:35:18.522460+0000","last_undegraded":"2023-10-10T20:35:18.522460+0000","last_fullsized":"2023-10-10T20:35:18.522460+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:52.866212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.527824+0000","last_change":"2023-10-10T20:35:18.525885+0000","last_active":"2023-10-10T20:35:18.527824+0000","last_peered":"2023-10-10T20:35:18.527824+0000","last_clean":"2023-10-10T20:35:18.527824+0000","last_became_active":"2023-10-10T20:35:18.525586+0000","last_became_peered":"2023-10-10T20:35:18.525586+0000","last_unstale":"2023-10-10T20:35:18.527824+0000","last_undegraded":"2023-10-10T20:35:18.527824+0000","last_fullsized":"2023-10-10T20:35:18.527824+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:11:49.032158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.524301+0000","last_change":"2023-10-10T20:35:17.524785+0000","last_active":"2023-10-10T20:35:18.524301+0000","last_peered":"2023-10-10T20:35:18.524301+0000","last_clean":"2023-10-10T20:35:18.524301+0000","last_became_active":"2023-10-10T20:35:17.524484+0000","last_became_peered":"2023-10-10T20:35:17.524484+0000","last_unstale":"2023-10-10T20:35:18.524301+0000","last_undegraded":"2023-10-10T20:35:18.524301+0000","last_fullsized":"2023-10-10T20:35:18.524301+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:49.712895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.545613+0000","last_change":"2023-10-10T20:35:18.544831+0000","last_active":"2023-10-10T20:35:18.545613+0000","last_peered":"2023-10-10T20:35:18.545613+0000","last_clean":"2023-10-10T20:35:18.545613+0000","last_became_active":"2023-10-10T20:35:18.544563+0000","last_became_peered":"2023-10-10T20:35:18.544563+0000","last_unstale":"2023-10-10T20:35:18.545613+0000","last_undegraded":"2023-10-10T20:35:18.545613+0000","last_fullsized":"2023-10-10T20:35:18.545613+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:26:58.211461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.521926+0000","last_change":"2023-10-10T20:35:17.525379+0000","last_active":"2023-10-10T20:35:18.521926+0000","last_peered":"2023-10-10T20:35:18.521926+0000","last_clean":"2023-10-10T20:35:18.521926+0000","last_became_active":"2023-10-10T20:35:17.525088+0000","last_became_peered":"2023-10-10T20:35:17.525088+0000","last_unstale":"2023-10-10T20:35:18.521926+0000","last_undegraded":"2023-10-10T20:35:18.521926+0000","last_fullsized":"2023-10-10T20:35:18.521926+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:56.281445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.541304+0000","last_change":"2023-10-10T20:35:18.529121+0000","last_active":"2023-10-10T20:35:18.541304+0000","last_peered":"2023-10-10T20:35:18.541304+0000","last_clean":"2023-10-10T20:35:18.541304+0000","last_became_active":"2023-10-10T20:35:18.528870+0000","last_became_peered":"2023-10-10T20:35:18.528870+0000","last_unstale":"2023-10-10T20:35:18.541304+0000","last_undegraded":"2023-10-10T20:35:18.541304+0000","last_fullsized":"2023-10-10T20:35:18.541304+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:41:55.916350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"30'3","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.539463+0000","last_change":"2023-10-10T20:35:17.531414+0000","last_active":"2023-10-10T20:35:18.539463+0000","last_peered":"2023-10-10T20:35:18.539463+0000","last_clean":"2023-10-10T20:35:18.539463+0000","last_became_active":"2023-10-10T20:35:17.531101+0000","last_became_peered":"2023-10-10T20:35:17.531101+0000","last_unstale":"2023-10-10T20:35:18.539463+0000","last_undegraded":"2023-10-10T20:35:18.539463+0000","last_fullsized":"2023-10-10T20:35:18.539463+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:49.540840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.546413+0000","last_change":"2023-10-10T20:35:18.542901+0000","last_active":"2023-10-10T20:35:18.546413+0000","last_peered":"2023-10-10T20:35:18.546413+0000","last_clean":"2023-10-10T20:35:18.546413+0000","last_became_active":"2023-10-10T20:35:18.542690+0000","last_became_peered":"2023-10-10T20:35:18.542690+0000","last_unstale":"2023-10-10T20:35:18.546413+0000","last_undegraded":"2023-10-10T20:35:18.546413+0000","last_fullsized":"2023-10-10T20:35:18.546413+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:41:55.916350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'6","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.548137+0000","last_change":"2023-10-10T20:35:17.531613+0000","last_active":"2023-10-10T20:35:18.548137+0000","last_peered":"2023-10-10T20:35:18.548137+0000","last_clean":"2023-10-10T20:35:18.548137+0000","last_became_active":"2023-10-10T20:35:17.531349+0000","last_became_peered":"2023-10-10T20:35:17.531349+0000","last_unstale":"2023-10-10T20:35:18.548137+0000","last_undegraded":"2023-10-10T20:35:18.548137+0000","last_fullsized":"2023-10-10T20:35:18.548137+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:10.460218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.542636+0000","last_change":"2023-10-10T20:35:18.538446+0000","last_active":"2023-10-10T20:35:18.542636+0000","last_peered":"2023-10-10T20:35:18.542636+0000","last_clean":"2023-10-10T20:35:18.542636+0000","last_became_active":"2023-10-10T20:35:18.538218+0000","last_became_peered":"2023-10-10T20:35:18.538218+0000","last_unstale":"2023-10-10T20:35:18.542636+0000","last_undegraded":"2023-10-10T20:35:18.542636+0000","last_fullsized":"2023-10-10T20:35:18.542636+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:00.556986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.540979+0000","last_change":"2023-10-10T20:35:17.530785+0000","last_active":"2023-10-10T20:35:18.540979+0000","last_peered":"2023-10-10T20:35:18.540979+0000","last_clean":"2023-10-10T20:35:18.540979+0000","last_became_active":"2023-10-10T20:35:17.530499+0000","last_became_peered":"2023-10-10T20:35:17.530499+0000","last_unstale":"2023-10-10T20:35:18.540979+0000","last_undegraded":"2023-10-10T20:35:18.540979+0000","last_fullsized":"2023-10-10T20:35:18.540979+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:54:55.830016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.541194+0000","last_change":"2023-10-10T20:35:18.532159+0000","last_active":"2023-10-10T20:35:18.541194+0000","last_peered":"2023-10-10T20:35:18.541194+0000","last_clean":"2023-10-10T20:35:18.541194+0000","last_became_active":"2023-10-10T20:35:18.531888+0000","last_became_peered":"2023-10-10T20:35:18.531888+0000","last_unstale":"2023-10-10T20:35:18.541194+0000","last_undegraded":"2023-10-10T20:35:18.541194+0000","last_fullsized":"2023-10-10T20:35:18.541194+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:36.755244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.523988+0000","last_change":"2023-10-10T20:35:17.529571+0000","last_active":"2023-10-10T20:35:18.523988+0000","last_peered":"2023-10-10T20:35:18.523988+0000","last_clean":"2023-10-10T20:35:18.523988+0000","last_became_active":"2023-10-10T20:35:17.529157+0000","last_became_peered":"2023-10-10T20:35:17.529157+0000","last_unstale":"2023-10-10T20:35:18.523988+0000","last_undegraded":"2023-10-10T20:35:18.523988+0000","last_fullsized":"2023-10-10T20:35:18.523988+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:49.712895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.535429+0000","last_change":"2023-10-10T20:35:18.528607+0000","last_active":"2023-10-10T20:35:18.535429+0000","last_peered":"2023-10-10T20:35:18.535429+0000","last_clean":"2023-10-10T20:35:18.535429+0000","last_became_active":"2023-10-10T20:35:18.527611+0000","last_became_peered":"2023-10-10T20:35:18.527611+0000","last_unstale":"2023-10-10T20:35:18.535429+0000","last_undegraded":"2023-10-10T20:35:18.535429+0000","last_fullsized":"2023-10-10T20:35:18.535429+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:21.510213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.524054+0000","last_change":"2023-10-10T20:35:17.526164+0000","last_active":"2023-10-10T20:35:18.524054+0000","last_peered":"2023-10-10T20:35:18.524054+0000","last_clean":"2023-10-10T20:35:18.524054+0000","last_became_active":"2023-10-10T20:35:17.525936+0000","last_became_peered":"2023-10-10T20:35:17.525936+0000","last_unstale":"2023-10-10T20:35:18.524054+0000","last_undegraded":"2023-10-10T20:35:18.524054+0000","last_fullsized":"2023-10-10T20:35:18.524054+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:34:44.534944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.533174+0000","last_change":"2023-10-10T20:35:18.530038+0000","last_active":"2023-10-10T20:35:18.533174+0000","last_peered":"2023-10-10T20:35:18.533174+0000","last_clean":"2023-10-10T20:35:18.533174+0000","last_became_active":"2023-10-10T20:35:18.529557+0000","last_became_peered":"2023-10-10T20:35:18.529557+0000","last_unstale":"2023-10-10T20:35:18.533174+0000","last_undegraded":"2023-10-10T20:35:18.533174+0000","last_fullsized":"2023-10-10T20:35:18.533174+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:41:58.095674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"30'8","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.545409+0000","last_change":"2023-10-10T20:35:17.530627+0000","last_active":"2023-10-10T20:35:18.545409+0000","last_peered":"2023-10-10T20:35:18.545409+0000","last_clean":"2023-10-10T20:35:18.545409+0000","last_became_active":"2023-10-10T20:35:17.530401+0000","last_became_peered":"2023-10-10T20:35:17.530401+0000","last_unstale":"2023-10-10T20:35:18.545409+0000","last_undegraded":"2023-10-10T20:35:18.545409+0000","last_fullsized":"2023-10-10T20:35:18.545409+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:08.528032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.534166+0000","last_change":"2023-10-10T20:35:18.530402+0000","last_active":"2023-10-10T20:35:18.534166+0000","last_peered":"2023-10-10T20:35:18.534166+0000","last_clean":"2023-10-10T20:35:18.534166+0000","last_became_active":"2023-10-10T20:35:18.529167+0000","last_became_peered":"2023-10-10T20:35:18.529167+0000","last_unstale":"2023-10-10T20:35:18.534166+0000","last_undegraded":"2023-10-10T20:35:18.534166+0000","last_fullsized":"2023-10-10T20:35:18.534166+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:41:10.435299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.545901+0000","last_change":"2023-10-10T20:35:17.532471+0000","last_active":"2023-10-10T20:35:18.545901+0000","last_peered":"2023-10-10T20:35:18.545901+0000","last_clean":"2023-10-10T20:35:18.545901+0000","last_became_active":"2023-10-10T20:35:17.532240+0000","last_became_peered":"2023-10-10T20:35:17.532240+0000","last_unstale":"2023-10-10T20:35:18.545901+0000","last_undegraded":"2023-10-10T20:35:18.545901+0000","last_fullsized":"2023-10-10T20:35:18.545901+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:56.281445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.528333+0000","last_change":"2023-10-10T20:35:18.526985+0000","last_active":"2023-10-10T20:35:18.528333+0000","last_peered":"2023-10-10T20:35:18.528333+0000","last_clean":"2023-10-10T20:35:18.528333+0000","last_became_active":"2023-10-10T20:35:18.526715+0000","last_became_peered":"2023-10-10T20:35:18.526715+0000","last_unstale":"2023-10-10T20:35:18.528333+0000","last_undegraded":"2023-10-10T20:35:18.528333+0000","last_fullsized":"2023-10-10T20:35:18.528333+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:31:25.404176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.540175+0000","last_change":"2023-10-10T20:35:17.531072+0000","last_active":"2023-10-10T20:35:18.540175+0000","last_peered":"2023-10-10T20:35:18.540175+0000","last_clean":"2023-10-10T20:35:18.540175+0000","last_became_active":"2023-10-10T20:35:17.530742+0000","last_became_peered":"2023-10-10T20:35:17.530742+0000","last_unstale":"2023-10-10T20:35:18.540175+0000","last_undegraded":"2023-10-10T20:35:18.540175+0000","last_fullsized":"2023-10-10T20:35:18.540175+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:35.655280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.542183+0000","last_change":"2023-10-10T20:35:18.534890+0000","last_active":"2023-10-10T20:35:18.542183+0000","last_peered":"2023-10-10T20:35:18.542183+0000","last_clean":"2023-10-10T20:35:18.542183+0000","last_became_active":"2023-10-10T20:35:18.534632+0000","last_became_peered":"2023-10-10T20:35:18.534632+0000","last_unstale":"2023-10-10T20:35:18.542183+0000","last_undegraded":"2023-10-10T20:35:18.542183+0000","last_fullsized":"2023-10-10T20:35:18.542183+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:44:06.887938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.541889+0000","last_change":"2023-10-10T20:35:17.529197+0000","last_active":"2023-10-10T20:35:18.541889+0000","last_peered":"2023-10-10T20:35:18.541889+0000","last_clean":"2023-10-10T20:35:18.541889+0000","last_became_active":"2023-10-10T20:35:17.528968+0000","last_became_peered":"2023-10-10T20:35:17.528968+0000","last_unstale":"2023-10-10T20:35:18.541889+0000","last_undegraded":"2023-10-10T20:35:18.541889+0000","last_fullsized":"2023-10-10T20:35:18.541889+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:28:58.181797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.541703+0000","last_change":"2023-10-10T20:35:18.531293+0000","last_active":"2023-10-10T20:35:18.541703+0000","last_peered":"2023-10-10T20:35:18.541703+0000","last_clean":"2023-10-10T20:35:18.541703+0000","last_became_active":"2023-10-10T20:35:18.531052+0000","last_became_peered":"2023-10-10T20:35:18.531052+0000","last_unstale":"2023-10-10T20:35:18.541703+0000","last_undegraded":"2023-10-10T20:35:18.541703+0000","last_fullsized":"2023-10-10T20:35:18.541703+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:10:10.066870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.525526+0000","last_change":"2023-10-10T20:35:17.527681+0000","last_active":"2023-10-10T20:35:18.525526+0000","last_peered":"2023-10-10T20:35:18.525526+0000","last_clean":"2023-10-10T20:35:18.525526+0000","last_became_active":"2023-10-10T20:35:17.527396+0000","last_became_peered":"2023-10-10T20:35:17.527396+0000","last_unstale":"2023-10-10T20:35:18.525526+0000","last_undegraded":"2023-10-10T20:35:18.525526+0000","last_fullsized":"2023-10-10T20:35:18.525526+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:36:04.680277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.536426+0000","last_change":"2023-10-10T20:35:18.526379+0000","last_active":"2023-10-10T20:35:18.536426+0000","last_peered":"2023-10-10T20:35:18.536426+0000","last_clean":"2023-10-10T20:35:18.536426+0000","last_became_active":"2023-10-10T20:35:18.526095+0000","last_became_peered":"2023-10-10T20:35:18.526095+0000","last_unstale":"2023-10-10T20:35:18.536426+0000","last_undegraded":"2023-10-10T20:35:18.536426+0000","last_fullsized":"2023-10-10T20:35:18.536426+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:53.876020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.545409+0000","last_change":"2023-10-10T20:35:17.529192+0000","last_active":"2023-10-10T20:35:18.545409+0000","last_peered":"2023-10-10T20:35:18.545409+0000","last_clean":"2023-10-10T20:35:18.545409+0000","last_became_active":"2023-10-10T20:35:17.528805+0000","last_became_peered":"2023-10-10T20:35:17.528805+0000","last_unstale":"2023-10-10T20:35:18.545409+0000","last_undegraded":"2023-10-10T20:35:18.545409+0000","last_fullsized":"2023-10-10T20:35:18.545409+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:25.295253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.528818+0000","last_change":"2023-10-10T20:35:18.525787+0000","last_active":"2023-10-10T20:35:18.528818+0000","last_peered":"2023-10-10T20:35:18.528818+0000","last_clean":"2023-10-10T20:35:18.528818+0000","last_became_active":"2023-10-10T20:35:18.525501+0000","last_became_peered":"2023-10-10T20:35:18.525501+0000","last_unstale":"2023-10-10T20:35:18.528818+0000","last_undegraded":"2023-10-10T20:35:18.528818+0000","last_fullsized":"2023-10-10T20:35:18.528818+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:28:59.191605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.521012+0000","last_change":"2023-10-10T20:35:17.532567+0000","last_active":"2023-10-10T20:35:18.521012+0000","last_peered":"2023-10-10T20:35:18.521012+0000","last_clean":"2023-10-10T20:35:18.521012+0000","last_became_active":"2023-10-10T20:35:17.532268+0000","last_became_peered":"2023-10-10T20:35:17.532268+0000","last_unstale":"2023-10-10T20:35:18.521012+0000","last_undegraded":"2023-10-10T20:35:18.521012+0000","last_fullsized":"2023-10-10T20:35:18.521012+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:56.281445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.525912+0000","last_change":"2023-10-10T20:35:18.522497+0000","last_active":"2023-10-10T20:35:18.525912+0000","last_peered":"2023-10-10T20:35:18.525912+0000","last_clean":"2023-10-10T20:35:18.525912+0000","last_became_active":"2023-10-10T20:35:18.522345+0000","last_became_peered":"2023-10-10T20:35:18.522345+0000","last_unstale":"2023-10-10T20:35:18.525912+0000","last_undegraded":"2023-10-10T20:35:18.525912+0000","last_fullsized":"2023-10-10T20:35:18.525912+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:36:05.690085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.522967+0000","last_change":"2023-10-10T20:35:17.529012+0000","last_active":"2023-10-10T20:35:18.522967+0000","last_peered":"2023-10-10T20:35:18.522967+0000","last_clean":"2023-10-10T20:35:18.522967+0000","last_became_active":"2023-10-10T20:35:17.528721+0000","last_became_peered":"2023-10-10T20:35:17.528721+0000","last_unstale":"2023-10-10T20:35:18.522967+0000","last_undegraded":"2023-10-10T20:35:18.522967+0000","last_fullsized":"2023-10-10T20:35:18.522967+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:00:46.029806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.536393+0000","last_change":"2023-10-10T20:35:18.530580+0000","last_active":"2023-10-10T20:35:18.536393+0000","last_peered":"2023-10-10T20:35:18.536393+0000","last_clean":"2023-10-10T20:35:18.536393+0000","last_became_active":"2023-10-10T20:35:18.530310+0000","last_became_peered":"2023-10-10T20:35:18.530310+0000","last_unstale":"2023-10-10T20:35:18.536393+0000","last_undegraded":"2023-10-10T20:35:18.536393+0000","last_fullsized":"2023-10-10T20:35:18.536393+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:00.556986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.518961+0000","last_change":"2023-10-10T20:35:17.528749+0000","last_active":"2023-10-10T20:35:18.518961+0000","last_peered":"2023-10-10T20:35:18.518961+0000","last_clean":"2023-10-10T20:35:18.518961+0000","last_became_active":"2023-10-10T20:35:17.528444+0000","last_became_peered":"2023-10-10T20:35:17.528444+0000","last_unstale":"2023-10-10T20:35:18.518961+0000","last_undegraded":"2023-10-10T20:35:18.518961+0000","last_fullsized":"2023-10-10T20:35:18.518961+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:10.460218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.537018+0000","last_change":"2023-10-10T20:35:18.528309+0000","last_active":"2023-10-10T20:35:18.537018+0000","last_peered":"2023-10-10T20:35:18.537018+0000","last_clean":"2023-10-10T20:35:18.537018+0000","last_became_active":"2023-10-10T20:35:18.528044+0000","last_became_peered":"2023-10-10T20:35:18.528044+0000","last_unstale":"2023-10-10T20:35:18.537018+0000","last_undegraded":"2023-10-10T20:35:18.537018+0000","last_fullsized":"2023-10-10T20:35:18.537018+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:06:01.882683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.546616+0000","last_change":"2023-10-10T20:35:17.533438+0000","last_active":"2023-10-10T20:35:18.546616+0000","last_peered":"2023-10-10T20:35:18.546616+0000","last_clean":"2023-10-10T20:35:18.546616+0000","last_became_active":"2023-10-10T20:35:17.533184+0000","last_became_peered":"2023-10-10T20:35:17.533184+0000","last_unstale":"2023-10-10T20:35:18.546616+0000","last_undegraded":"2023-10-10T20:35:18.546616+0000","last_fullsized":"2023-10-10T20:35:18.546616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:25.295253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.541267+0000","last_change":"2023-10-10T20:35:18.540223+0000","last_active":"2023-10-10T20:35:18.541267+0000","last_peered":"2023-10-10T20:35:18.541267+0000","last_clean":"2023-10-10T20:35:18.541267+0000","last_became_active":"2023-10-10T20:35:18.539959+0000","last_became_peered":"2023-10-10T20:35:18.539959+0000","last_unstale":"2023-10-10T20:35:18.541267+0000","last_undegraded":"2023-10-10T20:35:18.541267+0000","last_fullsized":"2023-10-10T20:35:18.541267+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:18:13.506756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-10T20:35:19.645991+0000","last_change":"2023-10-10T20:35:17.529064+0000","last_active":"2023-10-10T20:35:19.645991+0000","last_peered":"2023-10-10T20:35:19.645991+0000","last_clean":"2023-10-10T20:35:19.645991+0000","last_became_active":"2023-10-10T20:35:17.528790+0000","last_became_peered":"2023-10-10T20:35:17.528790+0000","last_unstale":"2023-10-10T20:35:19.645991+0000","last_undegraded":"2023-10-10T20:35:19.645991+0000","last_fullsized":"2023-10-10T20:35:19.645991+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:44:01.198878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.544200+0000","last_change":"2023-10-10T20:35:18.543330+0000","last_active":"2023-10-10T20:35:18.544200+0000","last_peered":"2023-10-10T20:35:18.544200+0000","last_clean":"2023-10-10T20:35:18.544200+0000","last_became_active":"2023-10-10T20:35:18.543072+0000","last_became_peered":"2023-10-10T20:35:18.543072+0000","last_unstale":"2023-10-10T20:35:18.544200+0000","last_undegraded":"2023-10-10T20:35:18.544200+0000","last_fullsized":"2023-10-10T20:35:18.544200+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:41:58.095674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.524829+0000","last_change":"2023-10-10T20:35:17.529291+0000","last_active":"2023-10-10T20:35:18.524829+0000","last_peered":"2023-10-10T20:35:18.524829+0000","last_clean":"2023-10-10T20:35:18.524829+0000","last_became_active":"2023-10-10T20:35:17.529040+0000","last_became_peered":"2023-10-10T20:35:17.529040+0000","last_unstale":"2023-10-10T20:35:18.524829+0000","last_undegraded":"2023-10-10T20:35:18.524829+0000","last_fullsized":"2023-10-10T20:35:18.524829+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:50.432471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.536736+0000","last_change":"2023-10-10T20:35:18.532147+0000","last_active":"2023-10-10T20:35:18.536736+0000","last_peered":"2023-10-10T20:35:18.536736+0000","last_clean":"2023-10-10T20:35:18.536736+0000","last_became_active":"2023-10-10T20:35:18.531839+0000","last_became_peered":"2023-10-10T20:35:18.531839+0000","last_unstale":"2023-10-10T20:35:18.536736+0000","last_undegraded":"2023-10-10T20:35:18.536736+0000","last_fullsized":"2023-10-10T20:35:18.536736+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:21:29.405593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.522699+0000","last_change":"2023-10-10T20:35:17.530765+0000","last_active":"2023-10-10T20:35:18.522699+0000","last_peered":"2023-10-10T20:35:18.522699+0000","last_clean":"2023-10-10T20:35:18.522699+0000","last_became_active":"2023-10-10T20:35:17.530515+0000","last_became_peered":"2023-10-10T20:35:17.530515+0000","last_unstale":"2023-10-10T20:35:18.522699+0000","last_undegraded":"2023-10-10T20:35:18.522699+0000","last_fullsized":"2023-10-10T20:35:18.522699+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:06:39.021756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.536832+0000","last_change":"2023-10-10T20:35:18.532180+0000","last_active":"2023-10-10T20:35:18.536832+0000","last_peered":"2023-10-10T20:35:18.536832+0000","last_clean":"2023-10-10T20:35:18.536832+0000","last_became_active":"2023-10-10T20:35:18.531651+0000","last_became_peered":"2023-10-10T20:35:18.531651+0000","last_unstale":"2023-10-10T20:35:18.536832+0000","last_undegraded":"2023-10-10T20:35:18.536832+0000","last_fullsized":"2023-10-10T20:35:18.536832+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:09.537841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.523964+0000","last_change":"2023-10-10T20:35:17.529415+0000","last_active":"2023-10-10T20:35:18.523964+0000","last_peered":"2023-10-10T20:35:18.523964+0000","last_clean":"2023-10-10T20:35:18.523964+0000","last_became_active":"2023-10-10T20:35:17.528871+0000","last_became_peered":"2023-10-10T20:35:17.528871+0000","last_unstale":"2023-10-10T20:35:18.523964+0000","last_undegraded":"2023-10-10T20:35:18.523964+0000","last_fullsized":"2023-10-10T20:35:18.523964+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:06:39.021756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.542229+0000","last_change":"2023-10-10T20:35:18.539312+0000","last_active":"2023-10-10T20:35:18.542229+0000","last_peered":"2023-10-10T20:35:18.542229+0000","last_clean":"2023-10-10T20:35:18.542229+0000","last_became_active":"2023-10-10T20:35:18.539047+0000","last_became_peered":"2023-10-10T20:35:18.539047+0000","last_unstale":"2023-10-10T20:35:18.542229+0000","last_undegraded":"2023-10-10T20:35:18.542229+0000","last_fullsized":"2023-10-10T20:35:18.542229+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:34:18.583957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.523072+0000","last_change":"2023-10-10T20:35:17.533254+0000","last_active":"2023-10-10T20:35:18.523072+0000","last_peered":"2023-10-10T20:35:18.523072+0000","last_clean":"2023-10-10T20:35:18.523072+0000","last_became_active":"2023-10-10T20:35:17.532949+0000","last_became_peered":"2023-10-10T20:35:17.532949+0000","last_unstale":"2023-10-10T20:35:18.523072+0000","last_undegraded":"2023-10-10T20:35:18.523072+0000","last_fullsized":"2023-10-10T20:35:18.523072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:30:04.234585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.540602+0000","last_change":"2023-10-10T20:35:18.538385+0000","last_active":"2023-10-10T20:35:18.540602+0000","last_peered":"2023-10-10T20:35:18.540602+0000","last_clean":"2023-10-10T20:35:18.540602+0000","last_became_active":"2023-10-10T20:35:18.538013+0000","last_became_peered":"2023-10-10T20:35:18.538013+0000","last_unstale":"2023-10-10T20:35:18.540602+0000","last_undegraded":"2023-10-10T20:35:18.540602+0000","last_fullsized":"2023-10-10T20:35:18.540602+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:21:29.405593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.523986+0000","last_change":"2023-10-10T20:35:17.526551+0000","last_active":"2023-10-10T20:35:18.523986+0000","last_peered":"2023-10-10T20:35:18.523986+0000","last_clean":"2023-10-10T20:35:18.523986+0000","last_became_active":"2023-10-10T20:35:17.526307+0000","last_became_peered":"2023-10-10T20:35:17.526307+0000","last_unstale":"2023-10-10T20:35:18.523986+0000","last_undegraded":"2023-10-10T20:35:18.523986+0000","last_fullsized":"2023-10-10T20:35:18.523986+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:03:28.452443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.531691+0000","last_change":"2023-10-10T20:35:18.527596+0000","last_active":"2023-10-10T20:35:18.531691+0000","last_peered":"2023-10-10T20:35:18.531691+0000","last_clean":"2023-10-10T20:35:18.531691+0000","last_became_active":"2023-10-10T20:35:18.527317+0000","last_became_peered":"2023-10-10T20:35:18.527317+0000","last_unstale":"2023-10-10T20:35:18.531691+0000","last_undegraded":"2023-10-10T20:35:18.531691+0000","last_fullsized":"2023-10-10T20:35:18.531691+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:17.503982+0000","last_clean_scrub_stamp":"2023-10-10T20:35:17.503982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:43.729088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.520886+0000","last_change":"2023-10-10T20:35:17.529867+0000","last_active":"2023-10-10T20:35:18.520886+0000","last_peered":"2023-10-10T20:35:18.520886+0000","last_clean":"2023-10-10T20:35:18.520886+0000","last_became_active":"2023-10-10T20:35:17.529596+0000","last_became_peered":"2023-10-10T20:35:17.529596+0000","last_unstale":"2023-10-10T20:35:18.520886+0000","last_undegraded":"2023-10-10T20:35:18.520886+0000","last_fullsized":"2023-10-10T20:35:18.520886+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:16.494174+0000","last_clean_scrub_stamp":"2023-10-10T20:35:16.494174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:06:39.021756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":174,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:35:18.517532+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:35:18.517532+0000","last_peered":"2023-10-10T20:35:18.517532+0000","last_clean":"2023-10-10T20:35:18.517532+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:35:18.517532+0000","last_undegraded":"2023-10-10T20:35:18.517532+0000","last_fullsized":"2023-10-10T20:35:18.517532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298212,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073628,"statfs":{"total":96636764160,"available":96331395072,"internally_reserved":0,"allocated":946176,"data_stored":724172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297636,"kb_used_data":348,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074204,"statfs":{"total":96636764160,"available":96331984896,"internally_reserved":0,"allocated":356352,"data_stored":134302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":135272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298204,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073636,"statfs":{"total":96636764160,"available":96331403264,"internally_reserved":0,"allocated":937984,"data_stored":723121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297632,"kb_used_data":344,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074208,"statfs":{"total":96636764160,"available":96331988992,"internally_reserved":0,"allocated":352256,"data_stored":133784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":135272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]}]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297636,"kb_used_data":348,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074204,"statfs":{"total":96636764160,"available":96331984896,"internally_reserved":0,"allocated":356352,"data_stored":133818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297636,"kb_used_data":348,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074204,"statfs":{"total":96636764160,"available":96331984896,"internally_reserved":0,"allocated":356352,"data_stored":134358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:35:27.442 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:35:27.443 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:35:27.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.596+0000 7f656736b700 1 Processor -- start 2023-10-10T20:35:27.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.596+0000 7f656736b700 1 -- start start 2023-10-10T20:35:27.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.597+0000 7f656736b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6560159950 0x7f65600cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:27.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.597+0000 7f656736b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f65600af320 0x7f6560069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:27.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.597+0000 7f656736b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65600ce380 con 0x7f6560069aa0 2023-10-10T20:35:27.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.597+0000 7f656736b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6560069f20 con 0x7f65600af320 2023-10-10T20:35:27.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.597+0000 7f656736b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65600cec20 con 0x7f6560159950 2023-10-10T20:35:27.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.597+0000 7f6564906700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f65600af320 0x7f6560069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:27.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.597+0000 7f6565908700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6560069aa0 0x7f656015d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47418/0 (socket says 172.21.15.89:47418) 2023-10-10T20:35:27.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.598+0000 7f6565908700 1 -- 172.21.15.89:0/3993568425 learned_addr learned my addr 172.21.15.89:0/3993568425 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:27.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.598+0000 7f6565107700 1 --2- 172.21.15.89:0/3993568425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6560159950 0x7f65600cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:27.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.598+0000 7f6564906700 1 -- 172.21.15.89:0/3993568425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6560159950 msgr2=0x7f65600cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:27.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.598+0000 7f6564906700 1 --2- 172.21.15.89:0/3993568425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6560159950 0x7f65600cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.598+0000 7f6564906700 1 -- 172.21.15.89:0/3993568425 >> v1:172.21.15.89:6789/0 conn(0x7f6560069aa0 legacy=0x7f656015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:27.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.598+0000 7f6564906700 1 -- 172.21.15.89:0/3993568425 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65600afa90 con 0x7f65600af320 2023-10-10T20:35:27.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.598+0000 7f6557fff700 1 -- 172.21.15.89:0/3993568425 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286741882 0 0) 0x7f65600ce380 con 0x7f6560069aa0 2023-10-10T20:35:27.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.598+0000 7f6564906700 1 --2- 172.21.15.89:0/3993568425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65600af320 0x7f6560069560 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f65500098e0 tx=0x7f65500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bf88bc145875a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:27.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.598+0000 7f6557fff700 1 -- 172.21.15.89:0/3993568425 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550009e80 con 0x7f65600af320 2023-10-10T20:35:27.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.598+0000 7f6557fff700 1 -- 172.21.15.89:0/3993568425 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655000b860 con 0x7f65600af320 2023-10-10T20:35:27.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.598+0000 7f6557fff700 1 -- 172.21.15.89:0/3993568425 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655000bbe0 con 0x7f65600af320 2023-10-10T20:35:27.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.598+0000 7f656736b700 1 -- 172.21.15.89:0/3993568425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65600af320 msgr2=0x7f6560069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:27.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.598+0000 7f656736b700 1 --2- 172.21.15.89:0/3993568425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65600af320 0x7f6560069560 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f65500098e0 tx=0x7f65500091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.599+0000 7f656736b700 1 -- 172.21.15.89:0/3993568425 shutdown_connections 2023-10-10T20:35:27.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.599+0000 7f656736b700 1 --2- 172.21.15.89:0/3993568425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6560159950 0x7f65600cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.599+0000 7f656736b700 1 --2- 172.21.15.89:0/3993568425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65600af320 0x7f6560069560 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.599+0000 7f656736b700 1 -- 172.21.15.89:0/3993568425 >> 172.21.15.89:0/3993568425 conn(0x7f6560068e00 msgr2=0x7f65600d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:27.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.599+0000 7f656736b700 1 -- 172.21.15.89:0/3993568425 shutdown_connections 2023-10-10T20:35:27.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.599+0000 7f656736b700 1 -- 172.21.15.89:0/3993568425 wait complete. 2023-10-10T20:35:27.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.600+0000 7f656736b700 1 Processor -- start 2023-10-10T20:35:27.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.600+0000 7f656736b700 1 -- start start 2023-10-10T20:35:27.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f656736b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6560069aa0 0x7f656014ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:27.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f656736b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65600af320 0x7f65600b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:27.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f656736b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6560159950 0x7f656016b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:27.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f656736b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6560155d00 con 0x7f6560159950 2023-10-10T20:35:27.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f656736b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656016abb0 con 0x7f65600af320 2023-10-10T20:35:27.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f656736b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656016ae90 con 0x7f6560069aa0 2023-10-10T20:35:27.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f6565908700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6560159950 0x7f656016b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:27.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f6565908700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6560159950 0x7f656016b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46244/0 (socket says 172.21.15.89:46244) 2023-10-10T20:35:27.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f6565908700 1 -- 172.21.15.89:0/1300120958 learned_addr learned my addr 172.21.15.89:0/1300120958 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:27.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f6564906700 1 --2- 172.21.15.89:0/1300120958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65600af320 0x7f65600b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:27.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f6565107700 1 --2- 172.21.15.89:0/1300120958 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6560069aa0 0x7f656014ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:27.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f6565908700 1 -- 172.21.15.89:0/1300120958 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6560069aa0 msgr2=0x7f656014ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:27.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f6565908700 1 --2- 172.21.15.89:0/1300120958 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6560069aa0 0x7f656014ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f6565908700 1 -- 172.21.15.89:0/1300120958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65600af320 msgr2=0x7f65600b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:27.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f6565908700 1 --2- 172.21.15.89:0/1300120958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65600af320 0x7f65600b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f6565908700 1 -- 172.21.15.89:0/1300120958 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6550009420 con 0x7f6560159950 2023-10-10T20:35:27.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f6565908700 1 --2- 172.21.15.89:0/1300120958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6560159950 0x7f656016b530 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f655800c7e0 tx=0x7f6558007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:27.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f65557fa700 1 -- 172.21.15.89:0/1300120958 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6558005c00 con 0x7f6560159950 2023-10-10T20:35:27.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f65557fa700 1 -- 172.21.15.89:0/1300120958 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6558005d80 con 0x7f6560159950 2023-10-10T20:35:27.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f656736b700 1 -- 172.21.15.89:0/1300120958 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656016b170 con 0x7f6560159950 2023-10-10T20:35:27.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f65557fa700 1 -- 172.21.15.89:0/1300120958 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6558007320 con 0x7f6560159950 2023-10-10T20:35:27.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.601+0000 7f656736b700 1 -- 172.21.15.89:0/1300120958 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f656016ba70 con 0x7f6560159950 2023-10-10T20:35:27.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.602+0000 7f656736b700 1 -- 172.21.15.89:0/1300120958 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65600718b0 con 0x7f6560159950 2023-10-10T20:35:27.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.603+0000 7f65557fa700 1 -- 172.21.15.89:0/1300120958 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f655801b3b0 con 0x7f6560159950 2023-10-10T20:35:27.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.603+0000 7f65557fa700 1 --2- 172.21.15.89:0/1300120958 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6544066f30 0x7f65440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:27.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.603+0000 7f65557fa700 1 -- 172.21.15.89:0/1300120958 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6558083970 con 0x7f6560159950 2023-10-10T20:35:27.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.603+0000 7f6565107700 1 --2- 172.21.15.89:0/1300120958 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6544066f30 0x7f65440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:27.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.604+0000 7f6565107700 1 --2- 172.21.15.89:0/1300120958 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6544066f30 0x7f65440693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f655c0075a0 tx=0x7f655c0097e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:27.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.607+0000 7f65557fa700 1 -- 172.21.15.89:0/1300120958 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6558050be0 con 0x7f6560159950 2023-10-10T20:35:27.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.864+0000 7f656736b700 1 -- 172.21.15.89:0/1300120958 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f65600ce5d0 con 0x7f6560159950 2023-10-10T20:35:27.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.868+0000 7f65557fa700 1 -- 172.21.15.89:0/1300120958 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f6558017070 con 0x7f6560159950 2023-10-10T20:35:27.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.870+0000 7f656736b700 1 -- 172.21.15.89:0/1300120958 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6544066f30 msgr2=0x7f65440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:27.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.870+0000 7f656736b700 1 --2- 172.21.15.89:0/1300120958 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6544066f30 0x7f65440693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f655c0075a0 tx=0x7f655c0097e0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.870+0000 7f656736b700 1 -- 172.21.15.89:0/1300120958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6560159950 msgr2=0x7f656016b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:27.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.870+0000 7f656736b700 1 --2- 172.21.15.89:0/1300120958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6560159950 0x7f656016b530 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f655800c7e0 tx=0x7f6558007d90 comp rx=0 tx=0).stop 2023-10-10T20:35:27.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.870+0000 7f656736b700 1 -- 172.21.15.89:0/1300120958 shutdown_connections 2023-10-10T20:35:27.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.870+0000 7f656736b700 1 --2- 172.21.15.89:0/1300120958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6560159950 0x7f656016b530 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.870+0000 7f656736b700 1 --2- 172.21.15.89:0/1300120958 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6560069aa0 0x7f656014ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.870+0000 7f656736b700 1 --2- 172.21.15.89:0/1300120958 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6544066f30 0x7f65440693e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.870+0000 7f656736b700 1 --2- 172.21.15.89:0/1300120958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65600af320 0x7f65600b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:27.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.870+0000 7f656736b700 1 -- 172.21.15.89:0/1300120958 >> 172.21.15.89:0/1300120958 conn(0x7f6560068e00 msgr2=0x7f65600cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:27.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.870+0000 7f656736b700 1 -- 172.21.15.89:0/1300120958 shutdown_connections 2023-10-10T20:35:27.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.870+0000 7f656736b700 1 -- 172.21.15.89:0/1300120958 wait complete. 2023-10-10T20:35:27.875 INFO:teuthology.orchestra.run.smithi089.stderr:updated caps for client.0 2023-10-10T20:35:27.884 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:35:27.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.996+0000 7f39a9289700 1 Processor -- start 2023-10-10T20:35:27.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.996+0000 7f39a9289700 1 -- start start 2023-10-10T20:35:28.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.997+0000 7f39a9289700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f39a4159890 0x7f39a415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.997+0000 7f39a9289700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f39a4159c60 0x7f39a415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.997+0000 7f39a9289700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a415a160 con 0x7f39a415db10 2023-10-10T20:35:28.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.997+0000 7f39a9289700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39a415df50 con 0x7f39a4159c60 2023-10-10T20:35:28.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.997+0000 7f39a9289700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39a40d13a0 con 0x7f39a4159890 2023-10-10T20:35:28.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.997+0000 7f399a7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f39a4159c60 0x7f39a415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.997+0000 7f399a7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f39a4159c60 0x7f39a415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46258/0 (socket says 172.21.15.89:46258) 2023-10-10T20:35:28.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.997+0000 7f399a7fc700 1 -- 172.21.15.89:0/2364688302 learned_addr learned my addr 172.21.15.89:0/2364688302 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:28.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.998+0000 7f39a2ffd700 1 --2- 172.21.15.89:0/2364688302 >> v2:172.21.15.111:3300/0 conn(0x7f39a4159890 0x7f39a415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.998+0000 7f399a7fc700 1 -- 172.21.15.89:0/2364688302 >> v2:172.21.15.111:3300/0 conn(0x7f39a4159890 msgr2=0x7f39a415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:28.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.998+0000 7f399a7fc700 1 --2- 172.21.15.89:0/2364688302 >> v2:172.21.15.111:3300/0 conn(0x7f39a4159890 0x7f39a415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.998+0000 7f399a7fc700 1 -- 172.21.15.89:0/2364688302 >> v1:172.21.15.111:6789/0 conn(0x7f39a415db10 legacy=0x7f39a4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:28.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.998+0000 7f399a7fc700 1 -- 172.21.15.89:0/2364688302 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a415e140 con 0x7f39a4159c60 2023-10-10T20:35:28.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.998+0000 7f399a7fc700 1 --2- 172.21.15.89:0/2364688302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a4159c60 0x7f39a415d5d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f39a40d1090 tx=0x7f398c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348a4f6a769f2d7a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:28.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.998+0000 7f39a27fc700 1 -- 172.21.15.89:0/2364688302 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c00b820 con 0x7f39a4159c60 2023-10-10T20:35:28.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.998+0000 7f39a27fc700 1 -- 172.21.15.89:0/2364688302 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c00b980 con 0x7f39a4159c60 2023-10-10T20:35:28.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.998+0000 7f39a27fc700 1 -- 172.21.15.89:0/2364688302 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c00bd20 con 0x7f39a4159c60 2023-10-10T20:35:28.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.998+0000 7f39a9289700 1 -- 172.21.15.89:0/2364688302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a4159c60 msgr2=0x7f39a415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.998+0000 7f39a9289700 1 --2- 172.21.15.89:0/2364688302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a4159c60 0x7f39a415d5d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f39a40d1090 tx=0x7f398c009250 comp rx=0 tx=0).stop 2023-10-10T20:35:28.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.998+0000 7f39a9289700 1 -- 172.21.15.89:0/2364688302 shutdown_connections 2023-10-10T20:35:28.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.998+0000 7f39a9289700 1 --2- 172.21.15.89:0/2364688302 >> v2:172.21.15.111:3300/0 conn(0x7f39a4159890 0x7f39a415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.998+0000 7f39a9289700 1 --2- 172.21.15.89:0/2364688302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a4159c60 0x7f39a415d5d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.998+0000 7f39a9289700 1 -- 172.21.15.89:0/2364688302 >> 172.21.15.89:0/2364688302 conn(0x7f39a4068e20 msgr2=0x7f39a40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:28.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.998+0000 7f39a9289700 1 -- 172.21.15.89:0/2364688302 shutdown_connections 2023-10-10T20:35:28.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:27.999+0000 7f39a9289700 1 -- 172.21.15.89:0/2364688302 wait complete. 2023-10-10T20:35:28.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.000+0000 7f39a9289700 1 Processor -- start 2023-10-10T20:35:28.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.000+0000 7f39a9289700 1 -- start start 2023-10-10T20:35:28.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.000+0000 7f39a9289700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a4159890 0x7f39a414d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.000+0000 7f39a9289700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f39a4159c60 0x7f39a4152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.000+0000 7f39a9289700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39a415db10 0x7f39a41529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.000+0000 7f39a9289700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a40b05c0 con 0x7f39a4159890 2023-10-10T20:35:28.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.000+0000 7f39a9289700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a40b1140 con 0x7f39a415db10 2023-10-10T20:35:28.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.000+0000 7f39a9289700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a40b0810 con 0x7f39a4159c60 2023-10-10T20:35:28.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.001+0000 7f39a2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a4159890 0x7f39a414d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.001+0000 7f39a2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a4159890 0x7f39a414d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46272/0 (socket says 172.21.15.89:46272) 2023-10-10T20:35:28.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.001+0000 7f39a2ffd700 1 -- 172.21.15.89:0/2172990379 learned_addr learned my addr 172.21.15.89:0/2172990379 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:28.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.001+0000 7f399a7fc700 1 --2- 172.21.15.89:0/2172990379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f39a4159c60 0x7f39a4152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.001+0000 7f39a37fe700 1 --2- 172.21.15.89:0/2172990379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39a415db10 0x7f39a41529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.001+0000 7f39a2ffd700 1 -- 172.21.15.89:0/2172990379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f39a4159c60 msgr2=0x7f39a4152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.001+0000 7f39a2ffd700 1 --2- 172.21.15.89:0/2172990379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f39a4159c60 0x7f39a4152460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.001+0000 7f39a2ffd700 1 -- 172.21.15.89:0/2172990379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39a415db10 msgr2=0x7f39a41529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.001+0000 7f39a2ffd700 1 --2- 172.21.15.89:0/2172990379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39a415db10 0x7f39a41529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.001+0000 7f39a2ffd700 1 -- 172.21.15.89:0/2172990379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c0094f0 con 0x7f39a4159890 2023-10-10T20:35:28.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.001+0000 7f399a7fc700 1 --2- 172.21.15.89:0/2172990379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f39a4159c60 0x7f39a4152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:28.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.001+0000 7f39a2ffd700 1 --2- 172.21.15.89:0/2172990379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a4159890 0x7f39a414d2c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f399400d8c0 tx=0x7f3994008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:28.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.001+0000 7f399b7fe700 1 -- 172.21.15.89:0/2172990379 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399400de60 con 0x7f39a4159890 2023-10-10T20:35:28.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.001+0000 7f399b7fe700 1 -- 172.21.15.89:0/2172990379 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399400b7e0 con 0x7f39a4159890 2023-10-10T20:35:28.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.001+0000 7f39a9289700 1 -- 172.21.15.89:0/2172990379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a40b0af0 con 0x7f39a4159890 2023-10-10T20:35:28.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.001+0000 7f399b7fe700 1 -- 172.21.15.89:0/2172990379 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399400bba0 con 0x7f39a4159890 2023-10-10T20:35:28.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.001+0000 7f39a9289700 1 -- 172.21.15.89:0/2172990379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a414d800 con 0x7f39a4159890 2023-10-10T20:35:28.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.002+0000 7f39a9289700 1 -- 172.21.15.89:0/2172990379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a404c3c0 con 0x7f39a4159890 2023-10-10T20:35:28.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.003+0000 7f399b7fe700 1 -- 172.21.15.89:0/2172990379 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f399401e460 con 0x7f39a4159890 2023-10-10T20:35:28.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.003+0000 7f399b7fe700 1 --2- 172.21.15.89:0/2172990379 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3984066f30 0x7f39840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.003+0000 7f399a7fc700 1 --2- 172.21.15.89:0/2172990379 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3984066f30 0x7f39840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.003+0000 7f399b7fe700 1 -- 172.21.15.89:0/2172990379 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3994084c40 con 0x7f39a4159890 2023-10-10T20:35:28.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.003+0000 7f399a7fc700 1 --2- 172.21.15.89:0/2172990379 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3984066f30 0x7f39840693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f39a40d1090 tx=0x7f398c016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:28.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.007+0000 7f399b7fe700 1 -- 172.21.15.89:0/2172990379 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3994051eb0 con 0x7f39a4159890 2023-10-10T20:35:28.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.232+0000 7f39a9289700 1 -- 172.21.15.89:0/2172990379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39a406a5b0 con 0x7f39a4159890 2023-10-10T20:35:28.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.232+0000 7f399b7fe700 1 -- 172.21.15.89:0/2172990379 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f39940554d0 con 0x7f39a4159890 2023-10-10T20:35:28.235 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:28.235 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1977066906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1977066906},{"type":"v1","addr":"172.21.15.111:6833","nonce":1977066906}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2904313060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2904313060},{"type":"v1","addr":"172.21.15.111:6835","nonce":2904313060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/2135915759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2135915759},{"type":"v1","addr":"172.21.15.89:6836","nonce":2135915759}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:35:17.900533+0000","modified":"2023-10-10T20:35:21.533574+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4899":{"gid":4899,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1330347277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1330347277},{"type":"v1","addr":"172.21.15.89:6837","nonce":1330347277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-10T20:35:28.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.234+0000 7f39a9289700 1 -- 172.21.15.89:0/2172990379 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3984066f30 msgr2=0x7f39840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.234+0000 7f39a9289700 1 --2- 172.21.15.89:0/2172990379 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3984066f30 0x7f39840693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f39a40d1090 tx=0x7f398c016000 comp rx=0 tx=0).stop 2023-10-10T20:35:28.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.234+0000 7f39a9289700 1 -- 172.21.15.89:0/2172990379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a4159890 msgr2=0x7f39a414d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.234+0000 7f39a9289700 1 --2- 172.21.15.89:0/2172990379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a4159890 0x7f39a414d2c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f399400d8c0 tx=0x7f3994008040 comp rx=0 tx=0).stop 2023-10-10T20:35:28.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.234+0000 7f39a9289700 1 -- 172.21.15.89:0/2172990379 shutdown_connections 2023-10-10T20:35:28.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.234+0000 7f39a9289700 1 --2- 172.21.15.89:0/2172990379 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3984066f30 0x7f39840693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.234+0000 7f39a9289700 1 --2- 172.21.15.89:0/2172990379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39a415db10 0x7f39a41529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.234+0000 7f39a9289700 1 --2- 172.21.15.89:0/2172990379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a4159890 0x7f39a414d2c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.234+0000 7f39a9289700 1 --2- 172.21.15.89:0/2172990379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f39a4159c60 0x7f39a4152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.234+0000 7f39a9289700 1 -- 172.21.15.89:0/2172990379 >> 172.21.15.89:0/2172990379 conn(0x7f39a4068e20 msgr2=0x7f39a415cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:28.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.235+0000 7f39a9289700 1 -- 172.21.15.89:0/2172990379 shutdown_connections 2023-10-10T20:35:28.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.235+0000 7f39a9289700 1 -- 172.21.15.89:0/2172990379 wait complete. 2023-10-10T20:35:28.240 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 17 2023-10-10T20:35:28.651 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:35:17.900533+0000', 'modified': '2023-10-10T20:35:21.533574+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4899}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4899': {'gid': 4899, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6837/1330347277', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6835', 'nonce': 1330347277}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 1330347277}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:35:28.652 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:35:28.652 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-10T20:35:28.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.767+0000 7f3f86ec4700 1 Processor -- start 2023-10-10T20:35:28.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.767+0000 7f3f86ec4700 1 -- start start 2023-10-10T20:35:28.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.768+0000 7f3f86ec4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3f8006bcd0 0x7f3f80140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.768+0000 7f3f86ec4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f8015c2a0 con 0x7f3f8006c0a0 2023-10-10T20:35:28.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.768+0000 7f3f86ec4700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f8015b690 con 0x7f3f800c9510 2023-10-10T20:35:28.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.768+0000 7f3f86ec4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f800c9900 con 0x7f3f8006bcd0 2023-10-10T20:35:28.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.768+0000 7f3f84c60700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3f8006bcd0 0x7f3f80140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.768+0000 7f3f77fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3f8006c0a0 0x7f3f800c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41760/0 (socket says 172.21.15.89:41760) 2023-10-10T20:35:28.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.768+0000 7f3f77fff700 1 -- 172.21.15.89:0/2570810610 learned_addr learned my addr 172.21.15.89:0/2570810610 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:28.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.769+0000 7f3f84c60700 1 -- 172.21.15.89:0/2570810610 >> v1:172.21.15.111:6789/0 conn(0x7f3f800c9510 legacy=0x7f3f800cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.769+0000 7f3f84c60700 1 -- 172.21.15.89:0/2570810610 >> v1:172.21.15.89:6789/0 conn(0x7f3f8006c0a0 legacy=0x7f3f800c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.769+0000 7f3f84c60700 1 -- 172.21.15.89:0/2570810610 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8015b970 con 0x7f3f8006bcd0 2023-10-10T20:35:28.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.769+0000 7f3f777fe700 1 -- 172.21.15.89:0/2570810610 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2848490454 0 0) 0x7f3f8015c2a0 con 0x7f3f8006c0a0 2023-10-10T20:35:28.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.769+0000 7f3f84c60700 1 --2- 172.21.15.89:0/2570810610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f8006bcd0 0x7f3f80140df0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0098e0 tx=0x7f3f7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f8f239f1eb65a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:28.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.769+0000 7f3f777fe700 1 -- 172.21.15.89:0/2570810610 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c009e80 con 0x7f3f8006bcd0 2023-10-10T20:35:28.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.769+0000 7f3f777fe700 1 -- 172.21.15.89:0/2570810610 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7c00b860 con 0x7f3f8006bcd0 2023-10-10T20:35:28.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.769+0000 7f3f777fe700 1 -- 172.21.15.89:0/2570810610 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c00bbe0 con 0x7f3f8006bcd0 2023-10-10T20:35:28.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.769+0000 7f3f86ec4700 1 -- 172.21.15.89:0/2570810610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f8006bcd0 msgr2=0x7f3f80140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.769+0000 7f3f86ec4700 1 --2- 172.21.15.89:0/2570810610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f8006bcd0 0x7f3f80140df0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0098e0 tx=0x7f3f7c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.770+0000 7f3f86ec4700 1 -- 172.21.15.89:0/2570810610 shutdown_connections 2023-10-10T20:35:28.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.770+0000 7f3f86ec4700 1 --2- 172.21.15.89:0/2570810610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f8006bcd0 0x7f3f80140df0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.770+0000 7f3f86ec4700 1 -- 172.21.15.89:0/2570810610 >> 172.21.15.89:0/2570810610 conn(0x7f3f800ce600 msgr2=0x7f3f800cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:28.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.770+0000 7f3f86ec4700 1 -- 172.21.15.89:0/2570810610 shutdown_connections 2023-10-10T20:35:28.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.770+0000 7f3f86ec4700 1 -- 172.21.15.89:0/2570810610 wait complete. 2023-10-10T20:35:28.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.771+0000 7f3f86ec4700 1 Processor -- start 2023-10-10T20:35:28.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.771+0000 7f3f86ec4700 1 -- start start 2023-10-10T20:35:28.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.771+0000 7f3f86ec4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f8006bcd0 0x7f3f800c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.771+0000 7f3f86ec4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f8006c0a0 0x7f3f8013f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.771+0000 7f3f86ec4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f800c9510 0x7f3f8006cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.771+0000 7f3f86ec4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f800c7180 con 0x7f3f8006bcd0 2023-10-10T20:35:28.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.771+0000 7f3f86ec4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f800c7b60 con 0x7f3f800c9510 2023-10-10T20:35:28.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.771+0000 7f3f86ec4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f800c5bb0 con 0x7f3f8006c0a0 2023-10-10T20:35:28.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.772+0000 7f3f84c60700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f8006bcd0 0x7f3f800c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.772+0000 7f3f84c60700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f8006bcd0 0x7f3f800c7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59470/0 (socket says 172.21.15.89:59470) 2023-10-10T20:35:28.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.772+0000 7f3f84c60700 1 -- 172.21.15.89:0/2061157751 learned_addr learned my addr 172.21.15.89:0/2061157751 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:28.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.772+0000 7f3f84c60700 1 -- 172.21.15.89:0/2061157751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f8006c0a0 msgr2=0x7f3f8013f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.772+0000 7f3f84c60700 1 --2- 172.21.15.89:0/2061157751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f8006c0a0 0x7f3f8013f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.772+0000 7f3f84c60700 1 -- 172.21.15.89:0/2061157751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f800c9510 msgr2=0x7f3f8006cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.772+0000 7f3f84c60700 1 --2- 172.21.15.89:0/2061157751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f800c9510 0x7f3f8006cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.772+0000 7f3f84c60700 1 -- 172.21.15.89:0/2061157751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7c009420 con 0x7f3f8006bcd0 2023-10-10T20:35:28.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.772+0000 7f3f84c60700 1 --2- 172.21.15.89:0/2061157751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f8006bcd0 0x7f3f800c7620 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3f80069220 tx=0x7f3f7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:28.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.772+0000 7f3f74ff9700 1 -- 172.21.15.89:0/2061157751 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c01f070 con 0x7f3f8006bcd0 2023-10-10T20:35:28.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.772+0000 7f3f74ff9700 1 -- 172.21.15.89:0/2061157751 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7c011420 con 0x7f3f8006bcd0 2023-10-10T20:35:28.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.772+0000 7f3f86ec4700 1 -- 172.21.15.89:0/2061157751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f800c5e90 con 0x7f3f8006bcd0 2023-10-10T20:35:28.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.772+0000 7f3f74ff9700 1 -- 172.21.15.89:0/2061157751 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c00bbe0 con 0x7f3f8006bcd0 2023-10-10T20:35:28.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.772+0000 7f3f86ec4700 1 -- 172.21.15.89:0/2061157751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f800c6450 con 0x7f3f8006bcd0 2023-10-10T20:35:28.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.774+0000 7f3f74ff9700 1 -- 172.21.15.89:0/2061157751 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3f7c011a50 con 0x7f3f8006bcd0 2023-10-10T20:35:28.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.774+0000 7f3f74ff9700 1 --2- 172.21.15.89:0/2061157751 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3f6c066f30 0x7f3f6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.774+0000 7f3f74ff9700 1 -- 172.21.15.89:0/2061157751 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3f7c085b60 con 0x7f3f8006bcd0 2023-10-10T20:35:28.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.774+0000 7f3f77fff700 1 --2- 172.21.15.89:0/2061157751 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3f6c066f30 0x7f3f6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.774+0000 7f3f77fff700 1 --2- 172.21.15.89:0/2061157751 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3f6c066f30 0x7f3f6c0693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3f700011e0 tx=0x7f3f700079d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:28.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.776+0000 7f3f901bdb80 1 Processor -- start 2023-10-10T20:35:28.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.776+0000 7f3f901bdb80 1 -- start start 2023-10-10T20:35:28.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f901bdb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f67a50 0x561008f67e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f901bdb80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561008fb1d00 0x561008f90a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f901bdb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008f90f60 0x561008f63cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f901bdb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561008ebea70 con 0x561008f67a50 2023-10-10T20:35:28.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f901bdb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561008ebc270 con 0x561008f90f60 2023-10-10T20:35:28.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f901bdb80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x561008eb8060 con 0x561008fb1d00 2023-10-10T20:35:28.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f84c60700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f67a50 0x561008f67e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f84c60700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f67a50 0x561008f67e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59480/0 (socket says 172.21.15.89:59480) 2023-10-10T20:35:28.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f84c60700 1 -- 172.21.15.89:0/3525329226 learned_addr learned my addr 172.21.15.89:0/3525329226 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:28.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f77fff700 1 --2- 172.21.15.89:0/3525329226 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561008fb1d00 0x561008f90a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f85461700 1 --2- 172.21.15.89:0/3525329226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008f90f60 0x561008f63cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f84c60700 1 -- 172.21.15.89:0/3525329226 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561008fb1d00 msgr2=0x561008f90a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:28.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f84c60700 1 --2- 172.21.15.89:0/3525329226 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561008fb1d00 0x561008f90a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f84c60700 1 -- 172.21.15.89:0/3525329226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008f90f60 msgr2=0x561008f63cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:28.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f84c60700 1 --2- 172.21.15.89:0/3525329226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008f90f60 0x561008f63cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f84c60700 1 -- 172.21.15.89:0/3525329226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561008d52270 con 0x561008f67a50 2023-10-10T20:35:28.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f85461700 1 --2- 172.21.15.89:0/3525329226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008f90f60 0x561008f63cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:28.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f84c60700 1 --2- 172.21.15.89:0/3525329226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f67a50 0x561008f67e20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c056650 tx=0x7f3f7c083fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5936676120a228cb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:28.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f65ffb700 1 -- 172.21.15.89:0/3525329226 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c056a90 con 0x561008f67a50 2023-10-10T20:35:28.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f65ffb700 1 -- 172.21.15.89:0/3525329226 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7c059ce0 con 0x561008f67a50 2023-10-10T20:35:28.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.777+0000 7f3f65ffb700 1 -- 172.21.15.89:0/3525329226 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c05da30 con 0x561008f67a50 2023-10-10T20:35:28.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.778+0000 7f3f901bdb80 1 -- 172.21.15.89:0/3525329226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f67a50 msgr2=0x561008f67e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.778+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/3525329226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f67a50 0x561008f67e20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c056650 tx=0x7f3f7c083fe0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.778+0000 7f3f901bdb80 1 -- 172.21.15.89:0/3525329226 shutdown_connections 2023-10-10T20:35:28.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.778+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/3525329226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008f90f60 0x561008f63cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.778+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/3525329226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f67a50 0x561008f67e20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.778+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/3525329226 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561008fb1d00 0x561008f90a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.778+0000 7f3f901bdb80 1 -- 172.21.15.89:0/3525329226 >> 172.21.15.89:0/3525329226 conn(0x561008f69370 msgr2=0x561008f69740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:28.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.778+0000 7f3f901bdb80 1 -- 172.21.15.89:0/3525329226 shutdown_connections 2023-10-10T20:35:28.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.778+0000 7f3f901bdb80 1 -- 172.21.15.89:0/3525329226 wait complete. 2023-10-10T20:35:28.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.778+0000 7f3f901bdb80 1 -- start start 2023-10-10T20:35:28.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.779+0000 7f3f901bdb80 1 Processor -- start 2023-10-10T20:35:28.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.779+0000 7f3f901bdb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008f67a50 0x561008fac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.779+0000 7f3f901bdb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f90f60 0x561008faa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.779+0000 7f3f901bdb80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561008fb1d00 0x561008f64780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.779+0000 7f3f901bdb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561008e63890 con 0x561008f90f60 2023-10-10T20:35:28.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.779+0000 7f3f901bdb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561008e7eea0 con 0x561008f67a50 2023-10-10T20:35:28.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.779+0000 7f3f901bdb80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x561008d3ae30 con 0x561008fb1d00 2023-10-10T20:35:28.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.779+0000 7f3f901bdb80 10 client.0 _ll_register_callbacks cb 0x561008f2b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:35:28.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.780+0000 7f3f77fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f90f60 0x561008faa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.780+0000 7f3f77fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f90f60 0x561008faa150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59484/0 (socket says 172.21.15.89:59484) 2023-10-10T20:35:28.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.780+0000 7f3f77fff700 1 -- 172.21.15.89:0/737958198 learned_addr learned my addr 172.21.15.89:0/737958198 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:28.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.780+0000 7f3f84c60700 1 --2- 172.21.15.89:0/737958198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008f67a50 0x561008fac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.780+0000 7f3f85461700 1 --2- 172.21.15.89:0/737958198 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561008fb1d00 0x561008f64780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.780+0000 7f3f77fff700 1 -- 172.21.15.89:0/737958198 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561008fb1d00 msgr2=0x561008f64780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:28.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.780+0000 7f3f77fff700 1 --2- 172.21.15.89:0/737958198 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561008fb1d00 0x561008f64780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.780+0000 7f3f77fff700 1 -- 172.21.15.89:0/737958198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008f67a50 msgr2=0x561008fac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:28.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.780+0000 7f3f77fff700 1 --2- 172.21.15.89:0/737958198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008f67a50 0x561008fac1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.780+0000 7f3f77fff700 1 -- 172.21.15.89:0/737958198 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f800c6450 con 0x561008f90f60 2023-10-10T20:35:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.780+0000 7f3f77fff700 1 --2- 172.21.15.89:0/737958198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f90f60 0x561008faa150 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3f8006dbe0 tx=0x7f3f70011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cfa822b111f226 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.780+0000 7f3f5effd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:35:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.780+0000 7f3f5effd700 1 -- 172.21.15.89:0/737958198 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f70016bc0 con 0x561008f90f60 2023-10-10T20:35:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.780+0000 7f3f5effd700 1 -- 172.21.15.89:0/737958198 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3f70016d20 con 0x561008f90f60 2023-10-10T20:35:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.780+0000 7f3f901bdb80 1 -- 172.21.15.89:0/737958198 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561008f5cb70 con 0x561008f90f60 2023-10-10T20:35:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.780+0000 7f3f5effd700 1 -- 172.21.15.89:0/737958198 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f70023ce0 con 0x561008f90f60 2023-10-10T20:35:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.780+0000 7f3f901bdb80 10 client.5115 fetch_fsmap learned FSMap version 17 2023-10-10T20:35:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.780+0000 7f3f901bdb80 1 -- 172.21.15.89:0/737958198 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x561008cf4fe0 con 0x561008f90f60 2023-10-10T20:35:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.780+0000 7f3f5effd700 1 -- 172.21.15.89:0/737958198 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561008f5cb70 con 0x561008f90f60 2023-10-10T20:35:28.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.781+0000 7f3f5effd700 1 -- 172.21.15.89:0/737958198 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3f7001f070 con 0x561008f90f60 2023-10-10T20:35:28.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.781+0000 7f3f901bdb80 10 client.5115 fetch_fsmap finished waiting for FSMap version 17 2023-10-10T20:35:28.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.781+0000 7f3f901bdb80 10 client.5115 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-10T20:35:28.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.781+0000 7f3f901bdb80 20 client.5115 populate_metadata read hostname 'smithi089' 2023-10-10T20:35:28.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.781+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/737958198 >> [v2:172.21.15.89:6835/1330347277,v1:172.21.15.89:6837/1330347277] conn(0x561008fa7dd0 0x561008fa81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.781+0000 7f3f901bdb80 4 client.5115 mds_command: new command op to 4899 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:35:28.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.781+0000 7f3f901bdb80 1 -- 172.21.15.89:0/737958198 --> [v2:172.21.15.89:6835/1330347277,v1:172.21.15.89:6837/1330347277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561008f39aa0 con 0x561008fa7dd0 2023-10-10T20:35:28.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.781+0000 7f3f84c60700 1 --2- 172.21.15.89:0/737958198 >> [v2:172.21.15.89:6835/1330347277,v1:172.21.15.89:6837/1330347277] conn(0x561008fa7dd0 0x561008fa81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.781+0000 7f3f5effd700 10 client.5115 ms_handle_connect on v2:172.21.15.89:6835/1330347277 2023-10-10T20:35:28.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.781+0000 7f3f84c60700 1 --2- 172.21.15.89:0/737958198 >> [v2:172.21.15.89:6835/1330347277,v1:172.21.15.89:6837/1330347277] conn(0x561008fa7dd0 0x561008fa81a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68692e5f8123f973 server_cookie=25eb2082479a95c5 in_seq=0 out_seq=0 2023-10-10T20:35:28.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.781+0000 7f3f5effd700 1 -- 172.21.15.89:0/737958198 <== mds.0 v2:172.21.15.89:6835/1330347277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x561008f39aa0 con 0x561008fa7dd0 2023-10-10T20:35:28.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.782+0000 7f3f5effd700 10 client.5115 handle_command_reply: tid=0 2023-10-10T20:35:28.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.782+0000 7f3f901bdb80 1 client.5115 shutdown 2023-10-10T20:35:28.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.782+0000 7f3f901bdb80 1 -- 172.21.15.89:0/737958198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f90f60 msgr2=0x561008faa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.782+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/737958198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f90f60 0x561008faa150 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3f8006dbe0 tx=0x7f3f70011000 comp rx=0 tx=0).stop 2023-10-10T20:35:28.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.782+0000 7f3f901bdb80 1 -- 172.21.15.89:0/737958198 shutdown_connections 2023-10-10T20:35:28.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.782+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/737958198 >> [v2:172.21.15.89:6835/1330347277,v1:172.21.15.89:6837/1330347277] conn(0x561008fa7dd0 0x561008fa81a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.782+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/737958198 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561008fb1d00 0x561008f64780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.782+0000 7f3f5effd700 0 client.5115 ms_handle_reset on v2:172.21.15.89:6835/1330347277 2023-10-10T20:35:28.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.782+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/737958198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008f67a50 0x561008fac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.782+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/737958198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f90f60 0x561008faa150 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.782+0000 7f3f901bdb80 1 -- 172.21.15.89:0/737958198 >> 172.21.15.89:0/737958198 conn(0x561008f69370 msgr2=0x561008f68350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:28.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.782+0000 7f3f901bdb80 1 -- 172.21.15.89:0/737958198 shutdown_connections 2023-10-10T20:35:28.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.782+0000 7f3f901bdb80 1 -- 172.21.15.89:0/737958198 wait complete. 2023-10-10T20:35:28.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.782+0000 7f3f901bdb80 20 client.5115 trim_cache size 0 max 16384 2023-10-10T20:35:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.798+0000 7f3f901bdb80 1 Processor -- start 2023-10-10T20:35:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.799+0000 7f3f901bdb80 1 -- start start 2023-10-10T20:35:28.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.799+0000 7f3f901bdb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008f67a50 0x561008f96940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.799+0000 7f3f901bdb80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561008f90f60 0x561008f604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.799+0000 7f3f901bdb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008fb1d00 0x561008f68240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.799+0000 7f3f901bdb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561008f97060 con 0x561008fb1d00 2023-10-10T20:35:28.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.799+0000 7f3f901bdb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561008fab6c0 con 0x561008f67a50 2023-10-10T20:35:28.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.799+0000 7f3f901bdb80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x561008fc1150 con 0x561008f90f60 2023-10-10T20:35:28.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.799+0000 7f3f85461700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008fb1d00 0x561008f68240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.799+0000 7f3f85461700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008fb1d00 0x561008f68240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59494/0 (socket says 172.21.15.89:59494) 2023-10-10T20:35:28.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.799+0000 7f3f85461700 1 -- 172.21.15.89:0/977202739 learned_addr learned my addr 172.21.15.89:0/977202739 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:28.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.800+0000 7f3f85461700 1 -- 172.21.15.89:0/977202739 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561008f90f60 msgr2=0x561008f604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:28.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.800+0000 7f3f85461700 1 --2- 172.21.15.89:0/977202739 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561008f90f60 0x561008f604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.800+0000 7f3f85461700 1 -- 172.21.15.89:0/977202739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008f67a50 msgr2=0x561008f96940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:28.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.800+0000 7f3f84c60700 1 --2- 172.21.15.89:0/977202739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008f67a50 0x561008f96940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.800+0000 7f3f85461700 1 --2- 172.21.15.89:0/977202739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008f67a50 0x561008f96940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.800+0000 7f3f85461700 1 -- 172.21.15.89:0/977202739 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561008e86780 con 0x561008fb1d00 2023-10-10T20:35:28.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.800+0000 7f3f85461700 1 --2- 172.21.15.89:0/977202739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008fb1d00 0x561008f68240 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3f78002a40 tx=0x7f3f78009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b19d33c8c916fa0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:28.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.800+0000 7f3f667fc700 1 -- 172.21.15.89:0/977202739 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78001b20 con 0x561008fb1d00 2023-10-10T20:35:28.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.800+0000 7f3f667fc700 1 -- 172.21.15.89:0/977202739 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7800d070 con 0x561008fb1d00 2023-10-10T20:35:28.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.800+0000 7f3f667fc700 1 -- 172.21.15.89:0/977202739 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78002030 con 0x561008fb1d00 2023-10-10T20:35:28.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.800+0000 7f3f901bdb80 1 -- 172.21.15.89:0/977202739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008fb1d00 msgr2=0x561008f68240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.800+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/977202739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008fb1d00 0x561008f68240 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3f78002a40 tx=0x7f3f78009040 comp rx=0 tx=0).stop 2023-10-10T20:35:28.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.800+0000 7f3f901bdb80 1 -- 172.21.15.89:0/977202739 shutdown_connections 2023-10-10T20:35:28.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.800+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/977202739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008fb1d00 0x561008f68240 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.800+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/977202739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008f67a50 0x561008f96940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.800+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/977202739 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561008f90f60 0x561008f604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.800+0000 7f3f901bdb80 1 -- 172.21.15.89:0/977202739 >> 172.21.15.89:0/977202739 conn(0x561008f69370 msgr2=0x561008f69740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:28.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.800+0000 7f3f901bdb80 1 -- 172.21.15.89:0/977202739 shutdown_connections 2023-10-10T20:35:28.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.800+0000 7f3f901bdb80 1 -- 172.21.15.89:0/977202739 wait complete. 2023-10-10T20:35:28.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.801+0000 7f3f901bdb80 1 -- start start 2023-10-10T20:35:28.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.801+0000 7f3f901bdb80 1 Processor -- start 2023-10-10T20:35:28.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.802+0000 7f3f901bdb80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561008f67a50 0x561009013950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.802+0000 7f3f901bdb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f90f60 0x561008f68240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.802+0000 7f3f901bdb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008fb1d00 0x561008f68780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.802+0000 7f3f901bdb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561009013e90 con 0x561008f90f60 2023-10-10T20:35:28.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.802+0000 7f3f901bdb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561008f913f0 con 0x561008fb1d00 2023-10-10T20:35:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.802+0000 7f3f901bdb80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x561008f95e10 con 0x561008f67a50 2023-10-10T20:35:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.802+0000 7f3f901bdb80 10 client.0 _ll_register_callbacks cb 0x561008f2b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:35:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.802+0000 7f3f77fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f90f60 0x561008f68240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.802+0000 7f3f77fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f90f60 0x561008f68240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59498/0 (socket says 172.21.15.89:59498) 2023-10-10T20:35:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.802+0000 7f3f77fff700 1 -- 172.21.15.89:0/301303604 learned_addr learned my addr 172.21.15.89:0/301303604 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.802+0000 7f3f84c60700 1 --2- 172.21.15.89:0/301303604 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561008f67a50 0x561009013950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.802+0000 7f3f85461700 1 --2- 172.21.15.89:0/301303604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008fb1d00 0x561008f68780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.802+0000 7f3f77fff700 1 -- 172.21.15.89:0/301303604 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561008f67a50 msgr2=0x561009013950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.802+0000 7f3f77fff700 1 --2- 172.21.15.89:0/301303604 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561008f67a50 0x561009013950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.802+0000 7f3f77fff700 1 -- 172.21.15.89:0/301303604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008fb1d00 msgr2=0x561008f68780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.802+0000 7f3f77fff700 1 --2- 172.21.15.89:0/301303604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008fb1d00 0x561008f68780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.802+0000 7f3f77fff700 1 -- 172.21.15.89:0/301303604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8015b690 con 0x561008f90f60 2023-10-10T20:35:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.802+0000 7f3f77fff700 1 --2- 172.21.15.89:0/301303604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f90f60 0x561008f68240 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3f70007d80 tx=0x7f3f70015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b293bf6b39ac1bf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.802+0000 7f3f5effd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:35:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.803+0000 7f3f5effd700 1 -- 172.21.15.89:0/301303604 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f70007700 con 0x561008f90f60 2023-10-10T20:35:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.803+0000 7f3f5effd700 1 -- 172.21.15.89:0/301303604 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3f7002c680 con 0x561008f90f60 2023-10-10T20:35:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.803+0000 7f3f901bdb80 1 -- 172.21.15.89:0/301303604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f7c059ce0 con 0x561008f90f60 2023-10-10T20:35:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.803+0000 7f3f5effd700 1 -- 172.21.15.89:0/301303604 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f70016c20 con 0x561008f90f60 2023-10-10T20:35:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.803+0000 7f3f901bdb80 10 client.5121 fetch_fsmap learned FSMap version 17 2023-10-10T20:35:28.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.803+0000 7f3f901bdb80 1 -- 172.21.15.89:0/301303604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x561008e6d470 con 0x561008f90f60 2023-10-10T20:35:28.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.803+0000 7f3f5effd700 1 -- 172.21.15.89:0/301303604 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7c059ce0 con 0x561008f90f60 2023-10-10T20:35:28.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.803+0000 7f3f5effd700 1 -- 172.21.15.89:0/301303604 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3f70016450 con 0x561008f90f60 2023-10-10T20:35:28.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.803+0000 7f3f901bdb80 10 client.5121 fetch_fsmap finished waiting for FSMap version 17 2023-10-10T20:35:28.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.803+0000 7f3f901bdb80 10 client.5121 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-10T20:35:28.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.803+0000 7f3f901bdb80 20 client.5121 populate_metadata read hostname 'smithi089' 2023-10-10T20:35:28.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.803+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/301303604 >> [v2:172.21.15.89:6835/1330347277,v1:172.21.15.89:6837/1330347277] conn(0x5610090179b0 0x561009017d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.803+0000 7f3f901bdb80 4 client.5121 mds_command: new command op to 4899 tid=0[{"prefix": "status"}] 2023-10-10T20:35:28.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.803+0000 7f3f901bdb80 1 -- 172.21.15.89:0/301303604 --> [v2:172.21.15.89:6835/1330347277,v1:172.21.15.89:6837/1330347277] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561008e5ae00 con 0x5610090179b0 2023-10-10T20:35:28.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.803+0000 7f3f84c60700 1 --2- 172.21.15.89:0/301303604 >> [v2:172.21.15.89:6835/1330347277,v1:172.21.15.89:6837/1330347277] conn(0x5610090179b0 0x561009017d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.803+0000 7f3f5effd700 10 client.5121 ms_handle_connect on v2:172.21.15.89:6835/1330347277 2023-10-10T20:35:28.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.803+0000 7f3f84c60700 1 --2- 172.21.15.89:0/301303604 >> [v2:172.21.15.89:6835/1330347277,v1:172.21.15.89:6837/1330347277] conn(0x5610090179b0 0x561009017d80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48d44a17af48c298 server_cookie=987254c5926ff981 in_seq=0 out_seq=0 2023-10-10T20:35:28.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f5effd700 1 -- 172.21.15.89:0/301303604 <== mds.0 v2:172.21.15.89:6835/1330347277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x561008e5ae00 con 0x5610090179b0 2023-10-10T20:35:28.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f5effd700 10 client.5121 handle_command_reply: tid=0 2023-10-10T20:35:28.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f901bdb80 1 client.5121 shutdown 2023-10-10T20:35:28.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f901bdb80 1 -- 172.21.15.89:0/301303604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f90f60 msgr2=0x561008f68240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/301303604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f90f60 0x561008f68240 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3f70007d80 tx=0x7f3f70015000 comp rx=0 tx=0).stop 2023-10-10T20:35:28.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f901bdb80 1 -- 172.21.15.89:0/301303604 shutdown_connections 2023-10-10T20:35:28.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/301303604 >> [v2:172.21.15.89:6835/1330347277,v1:172.21.15.89:6837/1330347277] conn(0x5610090179b0 0x561009017d80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/301303604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561008fb1d00 0x561008f68780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/301303604 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561008f67a50 0x561009013950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f5effd700 0 client.5121 ms_handle_reset on v2:172.21.15.89:6835/1330347277 2023-10-10T20:35:28.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f901bdb80 1 --2- 172.21.15.89:0/301303604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561008f90f60 0x561008f68240 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f901bdb80 1 -- 172.21.15.89:0/301303604 >> 172.21.15.89:0/301303604 conn(0x561008f69370 msgr2=0x561008f604b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:28.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f901bdb80 1 -- 172.21.15.89:0/301303604 shutdown_connections 2023-10-10T20:35:28.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f901bdb80 1 -- 172.21.15.89:0/301303604 wait complete. 2023-10-10T20:35:28.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f901bdb80 20 client.5121 trim_cache size 0 max 16384 2023-10-10T20:35:28.830 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:35:28.830 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "5dd675b8-9026-4d8a-a1bf-62fdc5e7de67", 2023-10-10T20:35:28.830 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-10-10T20:35:28.831 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 4899, 2023-10-10T20:35:28.831 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-10-10T20:35:28.831 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-10-10T20:35:28.831 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-10-10T20:35:28.831 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 10.290590651, 2023-10-10T20:35:28.832 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 17, 2023-10-10T20:35:28.832 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 31, 2023-10-10T20:35:28.832 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 31, 2023-10-10T20:35:28.832 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 18.180356907 2023-10-10T20:35:28.832 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:35:28.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f86ec4700 1 -- 172.21.15.89:0/2061157751 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3f6c066f30 msgr2=0x7f3f6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f86ec4700 1 --2- 172.21.15.89:0/2061157751 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3f6c066f30 0x7f3f6c0693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3f700011e0 tx=0x7f3f700079d0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f86ec4700 1 -- 172.21.15.89:0/2061157751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f8006bcd0 msgr2=0x7f3f800c7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f86ec4700 1 --2- 172.21.15.89:0/2061157751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f8006bcd0 0x7f3f800c7620 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3f80069220 tx=0x7f3f7c017000 comp rx=0 tx=0).stop 2023-10-10T20:35:28.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f86ec4700 1 -- 172.21.15.89:0/2061157751 shutdown_connections 2023-10-10T20:35:28.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f86ec4700 1 --2- 172.21.15.89:0/2061157751 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3f6c066f30 0x7f3f6c0693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f86ec4700 1 --2- 172.21.15.89:0/2061157751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f800c9510 0x7f3f8006cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f86ec4700 1 --2- 172.21.15.89:0/2061157751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f8006bcd0 0x7f3f800c7620 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f86ec4700 1 --2- 172.21.15.89:0/2061157751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f8006c0a0 0x7f3f8013f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.804+0000 7f3f86ec4700 1 -- 172.21.15.89:0/2061157751 >> 172.21.15.89:0/2061157751 conn(0x7f3f800ce600 msgr2=0x7f3f8013fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:28.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.805+0000 7f3f86ec4700 1 -- 172.21.15.89:0/2061157751 shutdown_connections 2023-10-10T20:35:28.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.805+0000 7f3f86ec4700 1 -- 172.21.15.89:0/2061157751 wait complete. 2023-10-10T20:35:28.837 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:35:28.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.952+0000 7ff3d4bf3700 1 Processor -- start 2023-10-10T20:35:28.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.952+0000 7ff3d4bf3700 1 -- start start 2023-10-10T20:35:28.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.953+0000 7ff3d4bf3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff3d006bcd0 0x7ff3d0140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.953+0000 7ff3d4bf3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d015c2a0 con 0x7ff3d00c9510 2023-10-10T20:35:28.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.953+0000 7ff3d4bf3700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d015b690 con 0x7ff3d006c0a0 2023-10-10T20:35:28.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.953+0000 7ff3d4bf3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d00c9900 con 0x7ff3d006bcd0 2023-10-10T20:35:28.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.953+0000 7ff3ce59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff3d006bcd0 0x7ff3d0140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.953+0000 7ff3ced9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff3d00c9510 0x7ff3d00cc1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41772/0 (socket says 172.21.15.89:41772) 2023-10-10T20:35:28.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.953+0000 7ff3ced9d700 1 -- 172.21.15.89:0/2607426090 learned_addr learned my addr 172.21.15.89:0/2607426090 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:28.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.954+0000 7ff3ce59c700 1 -- 172.21.15.89:0/2607426090 >> v1:172.21.15.111:6789/0 conn(0x7ff3d006c0a0 legacy=0x7ff3d00c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:28.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.954+0000 7ff3ce59c700 1 -- 172.21.15.89:0/2607426090 >> v1:172.21.15.89:6789/0 conn(0x7ff3d00c9510 legacy=0x7ff3d00cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.954+0000 7ff3ce59c700 1 -- 172.21.15.89:0/2607426090 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d015b970 con 0x7ff3d006bcd0 2023-10-10T20:35:28.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.954+0000 7ff3cdd9b700 1 -- 172.21.15.89:0/2607426090 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477531067 0 0) 0x7ff3d015c2a0 con 0x7ff3d00c9510 2023-10-10T20:35:28.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.954+0000 7ff3ce59c700 1 --2- 172.21.15.89:0/2607426090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3d006bcd0 0x7ff3d0140df0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00098e0 tx=0x7ff3c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e421444477908571 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:28.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.954+0000 7ff3cdd9b700 1 -- 172.21.15.89:0/2607426090 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0009e80 con 0x7ff3d006bcd0 2023-10-10T20:35:28.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.954+0000 7ff3cdd9b700 1 -- 172.21.15.89:0/2607426090 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c000b860 con 0x7ff3d006bcd0 2023-10-10T20:35:28.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.954+0000 7ff3cdd9b700 1 -- 172.21.15.89:0/2607426090 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c000bbe0 con 0x7ff3d006bcd0 2023-10-10T20:35:28.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.954+0000 7ff3d4bf3700 1 -- 172.21.15.89:0/2607426090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3d006bcd0 msgr2=0x7ff3d0140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.954+0000 7ff3d4bf3700 1 --2- 172.21.15.89:0/2607426090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3d006bcd0 0x7ff3d0140df0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00098e0 tx=0x7ff3c00091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.954+0000 7ff3d4bf3700 1 -- 172.21.15.89:0/2607426090 shutdown_connections 2023-10-10T20:35:28.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.954+0000 7ff3d4bf3700 1 --2- 172.21.15.89:0/2607426090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3d006bcd0 0x7ff3d0140df0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.954+0000 7ff3d4bf3700 1 -- 172.21.15.89:0/2607426090 >> 172.21.15.89:0/2607426090 conn(0x7ff3d00ce600 msgr2=0x7ff3d00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:28.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.954+0000 7ff3d4bf3700 1 -- 172.21.15.89:0/2607426090 shutdown_connections 2023-10-10T20:35:28.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.954+0000 7ff3d4bf3700 1 -- 172.21.15.89:0/2607426090 wait complete. 2023-10-10T20:35:28.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.956+0000 7ff3d4bf3700 1 Processor -- start 2023-10-10T20:35:28.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.956+0000 7ff3d4bf3700 1 -- start start 2023-10-10T20:35:28.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.956+0000 7ff3d4bf3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3d006bcd0 0x7ff3d013f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.956+0000 7ff3d4bf3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3d006c0a0 0x7ff3d016b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.956+0000 7ff3d4bf3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff3d00c9510 0x7ff3d016bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.956+0000 7ff3d4bf3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d0168010 con 0x7ff3d006c0a0 2023-10-10T20:35:28.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.956+0000 7ff3d4bf3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d00a5600 con 0x7ff3d006bcd0 2023-10-10T20:35:28.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.956+0000 7ff3d4bf3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d00b2e60 con 0x7ff3d00c9510 2023-10-10T20:35:28.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.956+0000 7ff3c5d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3d006c0a0 0x7ff3d016b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.956+0000 7ff3c5d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3d006c0a0 0x7ff3d016b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59520/0 (socket says 172.21.15.89:59520) 2023-10-10T20:35:28.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.956+0000 7ff3c5d9b700 1 -- 172.21.15.89:0/3269602195 learned_addr learned my addr 172.21.15.89:0/3269602195 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:28.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.957+0000 7ff3ce59c700 1 --2- 172.21.15.89:0/3269602195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3d006bcd0 0x7ff3d013f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.957+0000 7ff3ced9d700 1 --2- 172.21.15.89:0/3269602195 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff3d00c9510 0x7ff3d016bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.957+0000 7ff3c5d9b700 1 -- 172.21.15.89:0/3269602195 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff3d00c9510 msgr2=0x7ff3d016bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.957+0000 7ff3c5d9b700 1 --2- 172.21.15.89:0/3269602195 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff3d00c9510 0x7ff3d016bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.957+0000 7ff3c5d9b700 1 -- 172.21.15.89:0/3269602195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3d006bcd0 msgr2=0x7ff3d013f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:28.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.957+0000 7ff3c5d9b700 1 --2- 172.21.15.89:0/3269602195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3d006bcd0 0x7ff3d013f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:28.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.957+0000 7ff3c5d9b700 1 -- 172.21.15.89:0/3269602195 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c0009420 con 0x7ff3d006c0a0 2023-10-10T20:35:28.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.957+0000 7ff3c5d9b700 1 --2- 172.21.15.89:0/3269602195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3d006c0a0 0x7ff3d016b610 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800c600 tx=0x7ff3b8007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:28.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.957+0000 7ff3c6ffd700 1 -- 172.21.15.89:0/3269602195 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8011590 con 0x7ff3d006c0a0 2023-10-10T20:35:28.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.957+0000 7ff3c6ffd700 1 -- 172.21.15.89:0/3269602195 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b8011710 con 0x7ff3d006c0a0 2023-10-10T20:35:28.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.957+0000 7ff3d4bf3700 1 -- 172.21.15.89:0/3269602195 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d016c180 con 0x7ff3d006c0a0 2023-10-10T20:35:28.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.957+0000 7ff3c6ffd700 1 -- 172.21.15.89:0/3269602195 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8011ad0 con 0x7ff3d006c0a0 2023-10-10T20:35:28.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.957+0000 7ff3d4bf3700 1 -- 172.21.15.89:0/3269602195 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d00c56b0 con 0x7ff3d006c0a0 2023-10-10T20:35:28.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.958+0000 7ff3d4bf3700 1 -- 172.21.15.89:0/3269602195 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d004c3c0 con 0x7ff3d006c0a0 2023-10-10T20:35:28.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.958+0000 7ff3c6ffd700 1 -- 172.21.15.89:0/3269602195 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff3b801c460 con 0x7ff3d006c0a0 2023-10-10T20:35:28.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.959+0000 7ff3c6ffd700 1 --2- 172.21.15.89:0/3269602195 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff3b0066f30 0x7ff3b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:28.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.959+0000 7ff3c6ffd700 1 -- 172.21.15.89:0/3269602195 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff3b8084310 con 0x7ff3d006c0a0 2023-10-10T20:35:28.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.959+0000 7ff3ce59c700 1 --2- 172.21.15.89:0/3269602195 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff3b0066f30 0x7ff3b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:28.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.959+0000 7ff3ce59c700 1 --2- 172.21.15.89:0/3269602195 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff3b0066f30 0x7ff3b00693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00091e0 tx=0x7ff3c0016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:28.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:28.963+0000 7ff3c6ffd700 1 -- 172.21.15.89:0/3269602195 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3b8051580 con 0x7ff3d006c0a0 2023-10-10T20:35:29.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.191+0000 7ff3d4bf3700 1 -- 172.21.15.89:0/3269602195 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff3d00c54d0 con 0x7ff3d006c0a0 2023-10-10T20:35:29.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.191+0000 7ff3c6ffd700 1 -- 172.21.15.89:0/3269602195 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff3b8054ba0 con 0x7ff3d006c0a0 2023-10-10T20:35:29.193 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:35:29.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.193+0000 7ff3d4bf3700 1 -- 172.21.15.89:0/3269602195 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff3b0066f30 msgr2=0x7ff3b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.193+0000 7ff3d4bf3700 1 --2- 172.21.15.89:0/3269602195 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff3b0066f30 0x7ff3b00693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00091e0 tx=0x7ff3c0016000 comp rx=0 tx=0).stop 2023-10-10T20:35:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.193+0000 7ff3d4bf3700 1 -- 172.21.15.89:0/3269602195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3d006c0a0 msgr2=0x7ff3d016b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.193+0000 7ff3d4bf3700 1 --2- 172.21.15.89:0/3269602195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3d006c0a0 0x7ff3d016b610 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800c600 tx=0x7ff3b8007f60 comp rx=0 tx=0).stop 2023-10-10T20:35:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.193+0000 7ff3d4bf3700 1 -- 172.21.15.89:0/3269602195 shutdown_connections 2023-10-10T20:35:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.193+0000 7ff3d4bf3700 1 --2- 172.21.15.89:0/3269602195 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff3d00c9510 0x7ff3d016bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:29.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.193+0000 7ff3d4bf3700 1 --2- 172.21.15.89:0/3269602195 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff3b0066f30 0x7ff3b00693e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:29.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.193+0000 7ff3d4bf3700 1 --2- 172.21.15.89:0/3269602195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3d006bcd0 0x7ff3d013f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:29.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.193+0000 7ff3d4bf3700 1 --2- 172.21.15.89:0/3269602195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3d006c0a0 0x7ff3d016b610 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:29.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.193+0000 7ff3d4bf3700 1 -- 172.21.15.89:0/3269602195 >> 172.21.15.89:0/3269602195 conn(0x7ff3d00ce600 msgr2=0x7ff3d00c4050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:29.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.193+0000 7ff3d4bf3700 1 -- 172.21.15.89:0/3269602195 shutdown_connections 2023-10-10T20:35:29.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.193+0000 7ff3d4bf3700 1 -- 172.21.15.89:0/3269602195 wait complete. 2023-10-10T20:35:29.207 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:35:29.207 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:35:29.207 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-10T20:35:29.208 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:35:29.208 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:35:29.208 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:35:29.209 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:35:29.209 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:35:29.209 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:35:29.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:29.210 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-10T20:35:29.225 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:35:29.225 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:35:29.225 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:35:29.226 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:35:29.226 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-10T20:35:29.226 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:35:29.226 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:35:29.226 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:35:29.227 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-10T20:35:29.227 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:35:29.227 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:35:29.227 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-10T20:35:29.227 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:35:29.227 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:35:29.228 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:35:29.228 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:35:29.228 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:35:29.228 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42436sec preferred_lft 42436sec 2023-10-10T20:35:29.228 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2023-10-10T20:35:29.229 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:35:29.229 INFO:teuthology.orchestra.run.smithi089.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:35:29.229 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:35:29.229 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:35:29.229 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:35:29.230 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::98a3:d4ff:fec1:83bb/64 scope link 2023-10-10T20:35:29.230 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:35:29.230 INFO:teuthology.orchestra.run.smithi089.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:35:29.230 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether ee:16:22:10:ca:9a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:35:29.230 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec16:22ff:fe10:ca9a/64 scope link 2023-10-10T20:35:29.230 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:35:29.231 INFO:teuthology.orchestra.run.smithi089.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:35:29.231 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:35:29.231 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::6864:2bff:fe5c:71d4/64 scope link 2023-10-10T20:35:29.231 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:35:29.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:29.232 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:35:29.287 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-10T20:35:29.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:29.289 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-10T20:35:29.345 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-10T20:35:29.346 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:35:29.346 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:35:29.346 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:35:29.346 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:35:29.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:29.347 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:35:29.348 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:35:29.348 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:35:29.348 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:35:29.349 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:35:29.349 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:35:29.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:29.446 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-10T20:35:29.460 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:35:29.461 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:35:29.461 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:35:29.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:29.462 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-10-10T20:35:29.525 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:35:29.526 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:35:29.579 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:35:29.580 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:35:29.580 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:35:29.634 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:35:29.634 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:35:29.635 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:35:29.713 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:35:29.714 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:35:29.714 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:35:29.722 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:35:29.722 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:35:29.722 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:35:29.722 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:35:29.723 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:35:29.743 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:35:29.743 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:35:29.744 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:35:29.744 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:35:29.744 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:35:29.744 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:35:29.744 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:35:29.744 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:35:29.745 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:35:29.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:29.746 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:35:29.810 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:35:29.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.963+0000 7ff19edbe700 1 Processor -- start 2023-10-10T20:35:29.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.963+0000 7ff19edbe700 1 -- start start 2023-10-10T20:35:29.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.964+0000 7ff19edbe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff198155c10 0x7ff198155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:29.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.964+0000 7ff19edbe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff1980ac6a0 0x7ff1980b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:29.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.964+0000 7ff19edbe700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1980acac0 con 0x7ff1980ac230 2023-10-10T20:35:29.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.964+0000 7ff19edbe700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1980ad930 con 0x7ff1980ac6a0 2023-10-10T20:35:29.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.964+0000 7ff19edbe700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1980acd20 con 0x7ff198155c10 2023-10-10T20:35:29.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.964+0000 7ff19d35b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff1980ac6a0 0x7ff1980b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:29.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.964+0000 7ff19d35b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff1980ac6a0 0x7ff1980b0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59532/0 (socket says 172.21.15.89:59532) 2023-10-10T20:35:29.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.964+0000 7ff18ffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff1980ac230 0x7ff1981568a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41778/0 (socket says 172.21.15.89:41778) 2023-10-10T20:35:29.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.964+0000 7ff19d35b700 1 -- 172.21.15.89:0/3755782559 learned_addr learned my addr 172.21.15.89:0/3755782559 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:29.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.965+0000 7ff19cb5a700 1 --2- 172.21.15.89:0/3755782559 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff198155c10 0x7ff198155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:29.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.965+0000 7ff19d35b700 1 -- 172.21.15.89:0/3755782559 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff198155c10 msgr2=0x7ff198155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:29.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.965+0000 7ff19d35b700 1 --2- 172.21.15.89:0/3755782559 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff198155c10 0x7ff198155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:29.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.965+0000 7ff19d35b700 1 -- 172.21.15.89:0/3755782559 >> v1:172.21.15.89:6789/0 conn(0x7ff1980ac230 legacy=0x7ff1981568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:29.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.965+0000 7ff18f7fe700 1 -- 172.21.15.89:0/3755782559 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1587920444 0 0) 0x7ff1980acac0 con 0x7ff1980ac230 2023-10-10T20:35:29.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.965+0000 7ff19d35b700 1 -- 172.21.15.89:0/3755782559 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1980ad000 con 0x7ff1980ac6a0 2023-10-10T20:35:29.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.965+0000 7ff19cb5a700 1 --2- 172.21.15.89:0/3755782559 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff198155c10 0x7ff198155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:29.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.965+0000 7ff19d35b700 1 --2- 172.21.15.89:0/3755782559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1980ac6a0 0x7ff1980b0300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff19000bc50 tx=0x7ff19000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad1ef34ee1803f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:29.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.965+0000 7ff18f7fe700 1 -- 172.21.15.89:0/3755782559 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19000be60 con 0x7ff1980ac6a0 2023-10-10T20:35:29.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.965+0000 7ff18f7fe700 1 -- 172.21.15.89:0/3755782559 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19000de00 con 0x7ff1980ac6a0 2023-10-10T20:35:29.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.965+0000 7ff18f7fe700 1 -- 172.21.15.89:0/3755782559 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1900196b0 con 0x7ff1980ac6a0 2023-10-10T20:35:29.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.966+0000 7ff19edbe700 1 -- 172.21.15.89:0/3755782559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1980ac6a0 msgr2=0x7ff1980b0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:29.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.966+0000 7ff19edbe700 1 --2- 172.21.15.89:0/3755782559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1980ac6a0 0x7ff1980b0300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff19000bc50 tx=0x7ff19000b4f0 comp rx=0 tx=0).stop 2023-10-10T20:35:29.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.966+0000 7ff19edbe700 1 -- 172.21.15.89:0/3755782559 shutdown_connections 2023-10-10T20:35:29.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.966+0000 7ff19edbe700 1 --2- 172.21.15.89:0/3755782559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1980ac6a0 0x7ff1980b0300 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:29.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.966+0000 7ff19edbe700 1 --2- 172.21.15.89:0/3755782559 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff198155c10 0x7ff198155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:29.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.966+0000 7ff19edbe700 1 -- 172.21.15.89:0/3755782559 >> 172.21.15.89:0/3755782559 conn(0x7ff19806a320 msgr2=0x7ff1980cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:29.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.966+0000 7ff19edbe700 1 -- 172.21.15.89:0/3755782559 shutdown_connections 2023-10-10T20:35:29.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.966+0000 7ff19edbe700 1 -- 172.21.15.89:0/3755782559 wait complete. 2023-10-10T20:35:29.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.967+0000 7ff19edbe700 1 Processor -- start 2023-10-10T20:35:29.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.967+0000 7ff19edbe700 1 -- start start 2023-10-10T20:35:29.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff19edbe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1980ac230 0x7ff1980cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:29.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff19edbe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1980ac6a0 0x7ff198156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:29.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff19edbe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff198155c10 0x7ff198158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:29.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff19edbe700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff198156e70 con 0x7ff1980ac230 2023-10-10T20:35:29.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff19edbe700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff198157cb0 con 0x7ff198155c10 2023-10-10T20:35:29.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff19edbe700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1981570a0 con 0x7ff1980ac6a0 2023-10-10T20:35:29.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff19cb5a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1980ac230 0x7ff1980cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:29.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff19cb5a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1980ac230 0x7ff1980cca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59548/0 (socket says 172.21.15.89:59548) 2023-10-10T20:35:29.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff19cb5a700 1 -- 172.21.15.89:0/3361370483 learned_addr learned my addr 172.21.15.89:0/3361370483 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff18ffff700 1 --2- 172.21.15.89:0/3361370483 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1980ac6a0 0x7ff198156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff19d35b700 1 --2- 172.21.15.89:0/3361370483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff198155c10 0x7ff198158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff19cb5a700 1 -- 172.21.15.89:0/3361370483 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1980ac6a0 msgr2=0x7ff198156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff19cb5a700 1 --2- 172.21.15.89:0/3361370483 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1980ac6a0 0x7ff198156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff19cb5a700 1 -- 172.21.15.89:0/3361370483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff198155c10 msgr2=0x7ff198158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff19cb5a700 1 --2- 172.21.15.89:0/3361370483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff198155c10 0x7ff198158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:29.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff19cb5a700 1 -- 172.21.15.89:0/3361370483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19000b790 con 0x7ff1980ac230 2023-10-10T20:35:29.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff19cb5a700 1 --2- 172.21.15.89:0/3361370483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1980ac230 0x7ff1980cca00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff19400cfb0 tx=0x7ff194007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:29.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff18cff9700 1 -- 172.21.15.89:0/3361370483 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194016420 con 0x7ff1980ac230 2023-10-10T20:35:29.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff18cff9700 1 -- 172.21.15.89:0/3361370483 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff194016580 con 0x7ff1980ac230 2023-10-10T20:35:29.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff19edbe700 1 -- 172.21.15.89:0/3361370483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff198157380 con 0x7ff1980ac230 2023-10-10T20:35:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff18cff9700 1 -- 172.21.15.89:0/3361370483 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194016940 con 0x7ff1980ac230 2023-10-10T20:35:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.968+0000 7ff19edbe700 1 -- 172.21.15.89:0/3361370483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff198157940 con 0x7ff1980ac230 2023-10-10T20:35:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.969+0000 7ff19edbe700 1 -- 172.21.15.89:0/3361370483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19804c3c0 con 0x7ff1980ac230 2023-10-10T20:35:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.970+0000 7ff18cff9700 1 -- 172.21.15.89:0/3361370483 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff194016d30 con 0x7ff1980ac230 2023-10-10T20:35:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.970+0000 7ff18cff9700 1 --2- 172.21.15.89:0/3361370483 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff184066f30 0x7ff1840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.970+0000 7ff18cff9700 1 -- 172.21.15.89:0/3361370483 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff194012070 con 0x7ff1980ac230 2023-10-10T20:35:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.970+0000 7ff18ffff700 1 --2- 172.21.15.89:0/3361370483 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff184066f30 0x7ff1840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.971+0000 7ff18ffff700 1 --2- 172.21.15.89:0/3361370483 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff184066f30 0x7ff1840693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff1880011e0 tx=0x7ff188007bf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:29.974+0000 7ff18cff9700 1 -- 172.21.15.89:0/3361370483 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff19404e5a0 con 0x7ff1980ac230 2023-10-10T20:35:30.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.199+0000 7ff19edbe700 1 -- 172.21.15.89:0/3361370483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff1980678d0 con 0x7ff1980ac230 2023-10-10T20:35:30.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.200+0000 7ff18cff9700 1 -- 172.21.15.89:0/3361370483 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff1940521b0 con 0x7ff1980ac230 2023-10-10T20:35:30.202 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:35:30.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.202+0000 7ff19edbe700 1 -- 172.21.15.89:0/3361370483 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff184066f30 msgr2=0x7ff1840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:30.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.202+0000 7ff19edbe700 1 --2- 172.21.15.89:0/3361370483 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff184066f30 0x7ff1840693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff1880011e0 tx=0x7ff188007bf0 comp rx=0 tx=0).stop 2023-10-10T20:35:30.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.202+0000 7ff19edbe700 1 -- 172.21.15.89:0/3361370483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1980ac230 msgr2=0x7ff1980cca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:30.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.202+0000 7ff19edbe700 1 --2- 172.21.15.89:0/3361370483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1980ac230 0x7ff1980cca00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff19400cfb0 tx=0x7ff194007870 comp rx=0 tx=0).stop 2023-10-10T20:35:30.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.202+0000 7ff19edbe700 1 -- 172.21.15.89:0/3361370483 shutdown_connections 2023-10-10T20:35:30.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.202+0000 7ff19edbe700 1 --2- 172.21.15.89:0/3361370483 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff184066f30 0x7ff1840693e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:30.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.202+0000 7ff19edbe700 1 --2- 172.21.15.89:0/3361370483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff198155c10 0x7ff198158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:30.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.202+0000 7ff19edbe700 1 --2- 172.21.15.89:0/3361370483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1980ac230 0x7ff1980cca00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:30.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.202+0000 7ff19edbe700 1 --2- 172.21.15.89:0/3361370483 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1980ac6a0 0x7ff198156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:30.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.202+0000 7ff19edbe700 1 -- 172.21.15.89:0/3361370483 >> 172.21.15.89:0/3361370483 conn(0x7ff19806a320 msgr2=0x7ff1980ae120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:30.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.202+0000 7ff19edbe700 1 -- 172.21.15.89:0/3361370483 shutdown_connections 2023-10-10T20:35:30.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.202+0000 7ff19edbe700 1 -- 172.21.15.89:0/3361370483 wait complete. 2023-10-10T20:35:30.216 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:35:30.216 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:35:30.216 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-10-10T20:35:30.217 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:35:30.217 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:35:30.218 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:35:30.218 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:35:30.218 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:35:30.219 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:35:30.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:30.219 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-10-10T20:35:30.239 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:35:30.239 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:35:30.239 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:35:30.239 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:35:30.239 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-10-10T20:35:30.240 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:35:30.240 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:35:30.240 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:35:30.240 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-10-10T20:35:30.240 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:35:30.241 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:35:30.241 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-10-10T20:35:30.241 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:35:30.241 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:35:30.241 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:35:30.241 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-10-10T20:35:30.242 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:35:30.242 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42452sec preferred_lft 42452sec 2023-10-10T20:35:30.242 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-10-10T20:35:30.242 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:35:30.243 INFO:teuthology.orchestra.run.smithi111.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:35:30.243 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:35:30.243 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:35:30.243 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:35:30.243 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::a486:a6ff:fe5f:6021/64 scope link 2023-10-10T20:35:30.243 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:35:30.244 INFO:teuthology.orchestra.run.smithi111.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:35:30.244 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 1e:d0:81:1b:e2:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:35:30.244 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::1cd0:81ff:fe1b:e26b/64 scope link 2023-10-10T20:35:30.244 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:35:30.244 INFO:teuthology.orchestra.run.smithi111.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:35:30.245 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:35:30.245 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::88d:cdff:fe75:455e/64 scope link 2023-10-10T20:35:30.245 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:35:30.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:30.246 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:35:30.306 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-10-10T20:35:30.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:30.308 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-10-10T20:35:30.366 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-10-10T20:35:30.367 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:35:30.367 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:35:30.367 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:35:30.369 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:35:30.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:30.370 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:35:30.371 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:35:30.371 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:35:30.372 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:35:30.372 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:35:30.373 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:35:30.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:30.475 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-10-10T20:35:30.490 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:35:30.490 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:35:30.491 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:35:30.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:30.492 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-10-10T20:35:30.555 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:35:30.556 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:35:30.610 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:35:30.611 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:35:30.611 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:35:30.665 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:35:30.666 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:35:30.666 DEBUG:teuthology.orchestra.run.smithi111:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:35:30.745 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:35:30.745 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:35:30.745 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:35:30.753 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:35:30.753 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:35:30.753 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:35:30.754 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:35:30.754 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:35:30.778 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:35:30.778 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:35:30.778 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:35:30.778 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:35:30.779 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:35:30.779 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:35:30.779 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:35:30.779 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:35:30.780 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:35:30.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:30.781 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:35:30.845 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-10T20:35:30.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.965+0000 7fa6612d7700 1 Processor -- start 2023-10-10T20:35:30.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.965+0000 7fa6612d7700 1 -- start start 2023-10-10T20:35:30.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.965+0000 7fa6612d7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa65c159ec0 0x7fa65c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:30.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.965+0000 7fa6612d7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65c0b5b00 con 0x7fa65c0cf350 2023-10-10T20:35:30.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.965+0000 7fa6612d7700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65c0b5850 con 0x7fa65c0b53d0 2023-10-10T20:35:30.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.966+0000 7fa6612d7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa65c0b5da0 con 0x7fa65c159ec0 2023-10-10T20:35:30.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.966+0000 7fa65affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa65c159ec0 0x7fa65c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:30.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.966+0000 7fa65b7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa65c0cf350 0x7fa65c0cd010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41790/0 (socket says 172.21.15.89:41790) 2023-10-10T20:35:30.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.966+0000 7fa65b7fe700 1 -- 172.21.15.89:0/3079876398 learned_addr learned my addr 172.21.15.89:0/3079876398 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:30.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.966+0000 7fa65affd700 1 -- 172.21.15.89:0/3079876398 >> v1:172.21.15.111:6789/0 conn(0x7fa65c0b53d0 legacy=0x7fa65c0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:30.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.966+0000 7fa65a7fc700 1 -- 172.21.15.89:0/3079876398 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777032848 0 0) 0x7fa65c0b5b00 con 0x7fa65c0cf350 2023-10-10T20:35:30.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.966+0000 7fa65affd700 1 -- 172.21.15.89:0/3079876398 >> v1:172.21.15.89:6789/0 conn(0x7fa65c0cf350 legacy=0x7fa65c0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:30.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.966+0000 7fa65affd700 1 -- 172.21.15.89:0/3079876398 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c0b6080 con 0x7fa65c159ec0 2023-10-10T20:35:30.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.966+0000 7fa65affd700 1 --2- 172.21.15.89:0/3079876398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa65c159ec0 0x7fa65c0b4e90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa64c0098d0 tx=0x7fa64c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4f0e30cf1dec46 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:30.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.966+0000 7fa65a7fc700 1 -- 172.21.15.89:0/3079876398 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c009e70 con 0x7fa65c159ec0 2023-10-10T20:35:30.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.966+0000 7fa65a7fc700 1 -- 172.21.15.89:0/3079876398 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64c005c10 con 0x7fa65c159ec0 2023-10-10T20:35:30.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.966+0000 7fa65a7fc700 1 -- 172.21.15.89:0/3079876398 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c00b650 con 0x7fa65c159ec0 2023-10-10T20:35:30.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.967+0000 7fa6612d7700 1 -- 172.21.15.89:0/3079876398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa65c159ec0 msgr2=0x7fa65c0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:30.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.967+0000 7fa6612d7700 1 --2- 172.21.15.89:0/3079876398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa65c159ec0 0x7fa65c0b4e90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa64c0098d0 tx=0x7fa64c009120 comp rx=0 tx=0).stop 2023-10-10T20:35:30.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.967+0000 7fa6612d7700 1 -- 172.21.15.89:0/3079876398 shutdown_connections 2023-10-10T20:35:30.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.967+0000 7fa6612d7700 1 --2- 172.21.15.89:0/3079876398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa65c159ec0 0x7fa65c0b4e90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:30.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.967+0000 7fa6612d7700 1 -- 172.21.15.89:0/3079876398 >> 172.21.15.89:0/3079876398 conn(0x7fa65c0999f0 msgr2=0x7fa65c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:30.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.967+0000 7fa6612d7700 1 -- 172.21.15.89:0/3079876398 shutdown_connections 2023-10-10T20:35:30.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.967+0000 7fa6612d7700 1 -- 172.21.15.89:0/3079876398 wait complete. 2023-10-10T20:35:30.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.968+0000 7fa6612d7700 1 Processor -- start 2023-10-10T20:35:30.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.968+0000 7fa6612d7700 1 -- start start 2023-10-10T20:35:30.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.969+0000 7fa6612d7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa65c0b53d0 0x7fa65c13eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:30.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.969+0000 7fa6612d7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa65c0cf350 0x7fa65c169f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:30.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.969+0000 7fa6612d7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa65c159ec0 0x7fa65c16a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:30.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.969+0000 7fa6612d7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c1593a0 con 0x7fa65c159ec0 2023-10-10T20:35:30.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.969+0000 7fa6612d7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c155d60 con 0x7fa65c0cf350 2023-10-10T20:35:30.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.969+0000 7fa6612d7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c13f020 con 0x7fa65c0b53d0 2023-10-10T20:35:30.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.969+0000 7fa65b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa65c159ec0 0x7fa65c16a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:30.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.969+0000 7fa65b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa65c159ec0 0x7fa65c16a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59566/0 (socket says 172.21.15.89:59566) 2023-10-10T20:35:30.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.969+0000 7fa65b7fe700 1 -- 172.21.15.89:0/3645543679 learned_addr learned my addr 172.21.15.89:0/3645543679 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:30.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.969+0000 7fa6527fc700 1 --2- 172.21.15.89:0/3645543679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa65c0cf350 0x7fa65c169f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:30.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.969+0000 7fa65affd700 1 --2- 172.21.15.89:0/3645543679 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa65c0b53d0 0x7fa65c13eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:30.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.969+0000 7fa65b7fe700 1 -- 172.21.15.89:0/3645543679 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa65c0b53d0 msgr2=0x7fa65c13eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:30.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.969+0000 7fa65b7fe700 1 --2- 172.21.15.89:0/3645543679 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa65c0b53d0 0x7fa65c13eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:30.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.969+0000 7fa65b7fe700 1 -- 172.21.15.89:0/3645543679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa65c0cf350 msgr2=0x7fa65c169f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:30.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.969+0000 7fa65b7fe700 1 --2- 172.21.15.89:0/3645543679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa65c0cf350 0x7fa65c169f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:30.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.969+0000 7fa65b7fe700 1 -- 172.21.15.89:0/3645543679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c0093c0 con 0x7fa65c159ec0 2023-10-10T20:35:30.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.969+0000 7fa6527fc700 1 --2- 172.21.15.89:0/3645543679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa65c0cf350 0x7fa65c169f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:30.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.969+0000 7fa65affd700 1 --2- 172.21.15.89:0/3645543679 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa65c0b53d0 0x7fa65c13eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:30.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.969+0000 7fa65b7fe700 1 --2- 172.21.15.89:0/3645543679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa65c159ec0 0x7fa65c16a440 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa65400cc90 tx=0x7fa654001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:30.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.970+0000 7fa6537fe700 1 -- 172.21.15.89:0/3645543679 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65400cea0 con 0x7fa65c159ec0 2023-10-10T20:35:30.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.970+0000 7fa6537fe700 1 -- 172.21.15.89:0/3645543679 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65400ab30 con 0x7fa65c159ec0 2023-10-10T20:35:30.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.970+0000 7fa6612d7700 1 -- 172.21.15.89:0/3645543679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa65c158b00 con 0x7fa65c159ec0 2023-10-10T20:35:30.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.970+0000 7fa6537fe700 1 -- 172.21.15.89:0/3645543679 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6540070f0 con 0x7fa65c159ec0 2023-10-10T20:35:30.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.970+0000 7fa6612d7700 1 -- 172.21.15.89:0/3645543679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65c1590c0 con 0x7fa65c159ec0 2023-10-10T20:35:30.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.971+0000 7fa6612d7700 1 -- 172.21.15.89:0/3645543679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa65c063c90 con 0x7fa65c159ec0 2023-10-10T20:35:30.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.971+0000 7fa6537fe700 1 -- 172.21.15.89:0/3645543679 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa6540074a0 con 0x7fa65c159ec0 2023-10-10T20:35:30.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.971+0000 7fa6537fe700 1 --2- 172.21.15.89:0/3645543679 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa63c066f30 0x7fa63c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:30.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.971+0000 7fa6537fe700 1 -- 172.21.15.89:0/3645543679 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa654084950 con 0x7fa65c159ec0 2023-10-10T20:35:30.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.972+0000 7fa65affd700 1 --2- 172.21.15.89:0/3645543679 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa63c066f30 0x7fa63c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:30.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.972+0000 7fa65affd700 1 --2- 172.21.15.89:0/3645543679 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa63c066f30 0x7fa63c0693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009160 tx=0x7fa64c016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:30.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:30.976+0000 7fa6537fe700 1 -- 172.21.15.89:0/3645543679 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa654051c40 con 0x7fa65c159ec0 2023-10-10T20:35:31.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:31.202+0000 7fa6612d7700 1 -- 172.21.15.89:0/3645543679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa65c0718b0 con 0x7fa65c159ec0 2023-10-10T20:35:31.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:31.905+0000 7fa6537fe700 1 -- 172.21.15.89:0/3645543679 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 136+0+0 (secure 0 0 0) 0x7fa654055260 con 0x7fa65c159ec0 2023-10-10T20:35:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:31.907+0000 7fa6612d7700 1 -- 172.21.15.89:0/3645543679 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa63c066f30 msgr2=0x7fa63c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:31.907+0000 7fa6612d7700 1 --2- 172.21.15.89:0/3645543679 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa63c066f30 0x7fa63c0693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009160 tx=0x7fa64c016000 comp rx=0 tx=0).stop 2023-10-10T20:35:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:31.907+0000 7fa6612d7700 1 -- 172.21.15.89:0/3645543679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa65c159ec0 msgr2=0x7fa65c16a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:31.907+0000 7fa6612d7700 1 --2- 172.21.15.89:0/3645543679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa65c159ec0 0x7fa65c16a440 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa65400cc90 tx=0x7fa654001a20 comp rx=0 tx=0).stop 2023-10-10T20:35:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:31.907+0000 7fa6612d7700 1 -- 172.21.15.89:0/3645543679 shutdown_connections 2023-10-10T20:35:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:31.907+0000 7fa6612d7700 1 --2- 172.21.15.89:0/3645543679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa65c159ec0 0x7fa65c16a440 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:31.907+0000 7fa6612d7700 1 --2- 172.21.15.89:0/3645543679 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa65c0b53d0 0x7fa65c13eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:31.907+0000 7fa6612d7700 1 --2- 172.21.15.89:0/3645543679 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa63c066f30 0x7fa63c0693e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:31.907+0000 7fa6612d7700 1 --2- 172.21.15.89:0/3645543679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa65c0cf350 0x7fa65c169f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:31.907+0000 7fa6612d7700 1 -- 172.21.15.89:0/3645543679 >> 172.21.15.89:0/3645543679 conn(0x7fa65c0999f0 msgr2=0x7fa65c0c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:31.907+0000 7fa6612d7700 1 -- 172.21.15.89:0/3645543679 shutdown_connections 2023-10-10T20:35:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:31.907+0000 7fa6612d7700 1 -- 172.21.15.89:0/3645543679 wait complete. 2023-10-10T20:35:31.921 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-10T20:35:32.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.035+0000 7f6196916700 1 Processor -- start 2023-10-10T20:35:32.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.035+0000 7f6196916700 1 -- start start 2023-10-10T20:35:32.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.036+0000 7f6196916700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6190159420 0x7f61900b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:32.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.036+0000 7f6196916700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f619015d230 0x7f619015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:32.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.036+0000 7f6196916700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61900b2980 con 0x7f61901597f0 2023-10-10T20:35:32.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.036+0000 7f6196916700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619015dbb0 con 0x7f6190159420 2023-10-10T20:35:32.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.036+0000 7f6196916700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61900b11d0 con 0x7f619015d230 2023-10-10T20:35:32.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.036+0000 7f6195113700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f61901597f0 0x7f61900b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41794/0 (socket says 172.21.15.89:41794) 2023-10-10T20:35:32.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.036+0000 7f6195113700 1 -- 172.21.15.89:0/152520156 learned_addr learned my addr 172.21.15.89:0/152520156 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:32.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.036+0000 7f6195914700 1 --2- 172.21.15.89:0/152520156 >> v2:172.21.15.111:3300/0 conn(0x7f6190159420 0x7f61900b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:32.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.036+0000 7f6196115700 1 --2- 172.21.15.89:0/152520156 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f619015d230 0x7f619015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:32.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.036+0000 7f6194912700 1 -- 172.21.15.89:0/152520156 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4239139974 0 0) 0x7f61900b2980 con 0x7f61901597f0 2023-10-10T20:35:32.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.036+0000 7f6194912700 1 -- 172.21.15.89:0/152520156 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6178003570 con 0x7f61901597f0 2023-10-10T20:35:32.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.036+0000 7f6195914700 1 -- 172.21.15.89:0/152520156 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f619015d230 msgr2=0x7f619015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:32.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.036+0000 7f6195914700 1 --2- 172.21.15.89:0/152520156 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f619015d230 0x7f619015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.036+0000 7f6195914700 1 -- 172.21.15.89:0/152520156 >> v1:172.21.15.89:6789/0 conn(0x7f61901597f0 legacy=0x7f61900b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:32.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.036+0000 7f6195914700 1 -- 172.21.15.89:0/152520156 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61900b14b0 con 0x7f6190159420 2023-10-10T20:35:32.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.036+0000 7f6196115700 1 --2- 172.21.15.89:0/152520156 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f619015d230 0x7f619015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:35:32.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.037+0000 7f6195914700 1 --2- 172.21.15.89:0/152520156 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6190159420 0x7f61900b5f40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6188009e60 tx=0x7f6188009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c89da8d3aafa65e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:32.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.037+0000 7f6194912700 1 -- 172.21.15.89:0/152520156 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618800e070 con 0x7f6190159420 2023-10-10T20:35:32.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.037+0000 7f6194912700 1 -- 172.21.15.89:0/152520156 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618800bab0 con 0x7f6190159420 2023-10-10T20:35:32.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.037+0000 7f6194912700 1 -- 172.21.15.89:0/152520156 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618800be70 con 0x7f6190159420 2023-10-10T20:35:32.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.037+0000 7f6196916700 1 -- 172.21.15.89:0/152520156 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6190159420 msgr2=0x7f61900b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:32.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.037+0000 7f6196916700 1 --2- 172.21.15.89:0/152520156 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6190159420 0x7f61900b5f40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6188009e60 tx=0x7f6188009780 comp rx=0 tx=0).stop 2023-10-10T20:35:32.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.037+0000 7f6196916700 1 -- 172.21.15.89:0/152520156 shutdown_connections 2023-10-10T20:35:32.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.037+0000 7f6196916700 1 --2- 172.21.15.89:0/152520156 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6190159420 0x7f61900b5f40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.037+0000 7f6196916700 1 --2- 172.21.15.89:0/152520156 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f619015d230 0x7f619015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.037+0000 7f6196916700 1 -- 172.21.15.89:0/152520156 >> 172.21.15.89:0/152520156 conn(0x7f61900d0660 msgr2=0x7f61900d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:32.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.037+0000 7f6196916700 1 -- 172.21.15.89:0/152520156 shutdown_connections 2023-10-10T20:35:32.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.037+0000 7f6196916700 1 -- 172.21.15.89:0/152520156 wait complete. 2023-10-10T20:35:32.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.038+0000 7f6196916700 1 Processor -- start 2023-10-10T20:35:32.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.039+0000 7f6196916700 1 -- start start 2023-10-10T20:35:32.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.039+0000 7f6196916700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6190159420 0x7f61900cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:32.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.039+0000 7f6196916700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61901597f0 0x7f61900b6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:32.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.039+0000 7f6196916700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f619015d230 0x7f619014b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:32.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.039+0000 7f6196916700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61900b65b0 con 0x7f619015d230 2023-10-10T20:35:32.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.039+0000 7f6196916700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619014b810 con 0x7f61901597f0 2023-10-10T20:35:32.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.039+0000 7f6196916700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619016b6e0 con 0x7f6190159420 2023-10-10T20:35:32.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.039+0000 7f6196115700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f619015d230 0x7f619014b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:32.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.039+0000 7f6196115700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f619015d230 0x7f619014b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59576/0 (socket says 172.21.15.89:59576) 2023-10-10T20:35:32.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.039+0000 7f6196115700 1 -- 172.21.15.89:0/3849896032 learned_addr learned my addr 172.21.15.89:0/3849896032 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:32.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.040+0000 7f6196115700 1 -- 172.21.15.89:0/3849896032 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6190159420 msgr2=0x7f61900cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:32.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.040+0000 7f6196115700 1 --2- 172.21.15.89:0/3849896032 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6190159420 0x7f61900cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.040+0000 7f6196115700 1 -- 172.21.15.89:0/3849896032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61901597f0 msgr2=0x7f61900b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:32.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.040+0000 7f6195113700 1 --2- 172.21.15.89:0/3849896032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61901597f0 0x7f61900b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:32.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.040+0000 7f6196115700 1 --2- 172.21.15.89:0/3849896032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61901597f0 0x7f61900b6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.040+0000 7f6196115700 1 -- 172.21.15.89:0/3849896032 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6180009420 con 0x7f619015d230 2023-10-10T20:35:32.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.040+0000 7f6196115700 1 --2- 172.21.15.89:0/3849896032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f619015d230 0x7f619014b2d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f618000ea20 tx=0x7f61800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:32.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.040+0000 7f618dffb700 1 -- 172.21.15.89:0/3849896032 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618000c850 con 0x7f619015d230 2023-10-10T20:35:32.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.040+0000 7f618dffb700 1 -- 172.21.15.89:0/3849896032 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618000c9b0 con 0x7f619015d230 2023-10-10T20:35:32.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.040+0000 7f6196916700 1 -- 172.21.15.89:0/3849896032 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6188009a20 con 0x7f619015d230 2023-10-10T20:35:32.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.040+0000 7f618dffb700 1 -- 172.21.15.89:0/3849896032 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618000cd70 con 0x7f619015d230 2023-10-10T20:35:32.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.040+0000 7f6196916700 1 -- 172.21.15.89:0/3849896032 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619016adb0 con 0x7f619015d230 2023-10-10T20:35:32.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.041+0000 7f6196916700 1 -- 172.21.15.89:0/3849896032 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619004c3c0 con 0x7f619015d230 2023-10-10T20:35:32.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.041+0000 7f618dffb700 1 -- 172.21.15.89:0/3849896032 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6180013680 con 0x7f619015d230 2023-10-10T20:35:32.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.042+0000 7f618dffb700 1 --2- 172.21.15.89:0/3849896032 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f617c066fb0 0x7f617c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:32.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.042+0000 7f618dffb700 1 -- 172.21.15.89:0/3849896032 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6180084650 con 0x7f619015d230 2023-10-10T20:35:32.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.042+0000 7f6195914700 1 --2- 172.21.15.89:0/3849896032 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f617c066fb0 0x7f617c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:32.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.042+0000 7f6195914700 1 --2- 172.21.15.89:0/3849896032 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f617c066fb0 0x7f617c069460 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f61880097c0 tx=0x7f6188019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:32.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.046+0000 7f618dffb700 1 -- 172.21.15.89:0/3849896032 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6180051820 con 0x7f619015d230 2023-10-10T20:35:32.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.254+0000 7f6196916700 1 -- 172.21.15.89:0/3849896032 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f619016b850 con 0x7f617c066fb0 2023-10-10T20:35:32.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.255+0000 7f618dffb700 1 -- 172.21.15.89:0/3849896032 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f619016b850 con 0x7f617c066fb0 2023-10-10T20:35:32.258 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:35:32.258 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:32.258 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs" 2023-10-10T20:35:32.258 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:32.259 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:35:32.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.257+0000 7f6196916700 1 -- 172.21.15.89:0/3849896032 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f617c066fb0 msgr2=0x7f617c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:32.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.257+0000 7f6196916700 1 --2- 172.21.15.89:0/3849896032 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f617c066fb0 0x7f617c069460 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f61880097c0 tx=0x7f6188019000 comp rx=0 tx=0).stop 2023-10-10T20:35:32.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.257+0000 7f6196916700 1 -- 172.21.15.89:0/3849896032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f619015d230 msgr2=0x7f619014b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:32.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.257+0000 7f6196916700 1 --2- 172.21.15.89:0/3849896032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f619015d230 0x7f619014b2d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f618000ea20 tx=0x7f61800091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.257+0000 7f6196916700 1 -- 172.21.15.89:0/3849896032 shutdown_connections 2023-10-10T20:35:32.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.257+0000 7f6196916700 1 --2- 172.21.15.89:0/3849896032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f619015d230 0x7f619014b2d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.257+0000 7f6196916700 1 --2- 172.21.15.89:0/3849896032 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6190159420 0x7f61900cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.257+0000 7f6196916700 1 --2- 172.21.15.89:0/3849896032 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f617c066fb0 0x7f617c069460 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.257+0000 7f6196916700 1 --2- 172.21.15.89:0/3849896032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61901597f0 0x7f61900b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.258+0000 7f6196916700 1 -- 172.21.15.89:0/3849896032 >> 172.21.15.89:0/3849896032 conn(0x7f61900d0660 msgr2=0x7f61901571a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:32.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.258+0000 7f6196916700 1 -- 172.21.15.89:0/3849896032 shutdown_connections 2023-10-10T20:35:32.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.258+0000 7f6196916700 1 -- 172.21.15.89:0/3849896032 wait complete. 2023-10-10T20:35:32.271 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:35:32.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.386+0000 7f91d7be3700 1 Processor -- start 2023-10-10T20:35:32.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.386+0000 7f91d7be3700 1 -- start start 2023-10-10T20:35:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.387+0000 7f91d7be3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91d006aa40 0x7f91d00cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.387+0000 7f91d7be3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f91d00c91e0 0x7f91d0160390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.387+0000 7f91d7be3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d00cc210 con 0x7f91d006bac0 2023-10-10T20:35:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.387+0000 7f91d7be3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d00cc4f0 con 0x7f91d00c91e0 2023-10-10T20:35:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.387+0000 7f91d7be3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d00cc7d0 con 0x7f91d006aa40 2023-10-10T20:35:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.387+0000 7f91d517e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f91d006bac0 0x7f91d006ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41810/0 (socket says 172.21.15.89:41810) 2023-10-10T20:35:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.387+0000 7f91d6180700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f91d00c91e0 0x7f91d0160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.387+0000 7f91d517e700 1 -- 172.21.15.89:0/2261107156 learned_addr learned my addr 172.21.15.89:0/2261107156 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.388+0000 7f91d597f700 1 --2- 172.21.15.89:0/2261107156 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91d006aa40 0x7f91d00cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.388+0000 7f91d6180700 1 -- 172.21.15.89:0/2261107156 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91d006aa40 msgr2=0x7f91d00cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.388+0000 7f91d6180700 1 --2- 172.21.15.89:0/2261107156 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91d006aa40 0x7f91d00cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.388+0000 7f91d6180700 1 -- 172.21.15.89:0/2261107156 >> v1:172.21.15.89:6789/0 conn(0x7f91d006bac0 legacy=0x7f91d006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.388+0000 7f91d6180700 1 -- 172.21.15.89:0/2261107156 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d00ccab0 con 0x7f91d00c91e0 2023-10-10T20:35:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.388+0000 7f91d497d700 1 -- 172.21.15.89:0/2261107156 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3087968633 0 0) 0x7f91d00cc210 con 0x7f91d006bac0 2023-10-10T20:35:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.388+0000 7f91d6180700 1 --2- 172.21.15.89:0/2261107156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91d00c91e0 0x7f91d0160390 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f91c80098e0 tx=0x7f91c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0121903880bbd7d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.388+0000 7f91d497d700 1 -- 172.21.15.89:0/2261107156 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c800b640 con 0x7f91d00c91e0 2023-10-10T20:35:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.388+0000 7f91d497d700 1 -- 172.21.15.89:0/2261107156 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c800b7a0 con 0x7f91d00c91e0 2023-10-10T20:35:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.388+0000 7f91d497d700 1 -- 172.21.15.89:0/2261107156 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c800bb40 con 0x7f91d00c91e0 2023-10-10T20:35:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.389+0000 7f91d7be3700 1 -- 172.21.15.89:0/2261107156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91d00c91e0 msgr2=0x7f91d0160390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.389+0000 7f91d7be3700 1 --2- 172.21.15.89:0/2261107156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91d00c91e0 0x7f91d0160390 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f91c80098e0 tx=0x7f91c80091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.389+0000 7f91d7be3700 1 -- 172.21.15.89:0/2261107156 shutdown_connections 2023-10-10T20:35:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.389+0000 7f91d7be3700 1 --2- 172.21.15.89:0/2261107156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91d00c91e0 0x7f91d0160390 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.389+0000 7f91d7be3700 1 --2- 172.21.15.89:0/2261107156 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91d006aa40 0x7f91d00cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.389+0000 7f91d7be3700 1 -- 172.21.15.89:0/2261107156 >> 172.21.15.89:0/2261107156 conn(0x7f91d00cd670 msgr2=0x7f91d00cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:32.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.389+0000 7f91d7be3700 1 -- 172.21.15.89:0/2261107156 shutdown_connections 2023-10-10T20:35:32.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.389+0000 7f91d7be3700 1 -- 172.21.15.89:0/2261107156 wait complete. 2023-10-10T20:35:32.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.390+0000 7f91d7be3700 1 Processor -- start 2023-10-10T20:35:32.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.390+0000 7f91d7be3700 1 -- start start 2023-10-10T20:35:32.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.390+0000 7f91d7be3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91d006aa40 0x7f91d014dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:32.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.390+0000 7f91d7be3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91d006bac0 0x7f91d00ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.390+0000 7f91d7be3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91d00c91e0 0x7f91d01555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.390+0000 7f91d7be3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d014e1b0 con 0x7f91d00c91e0 2023-10-10T20:35:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.390+0000 7f91d7be3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d0068770 con 0x7f91d006bac0 2023-10-10T20:35:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.390+0000 7f91d7be3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d0068a50 con 0x7f91d006aa40 2023-10-10T20:35:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.391+0000 7f91d6180700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91d00c91e0 0x7f91d01555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.391+0000 7f91d6180700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91d00c91e0 0x7f91d01555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59596/0 (socket says 172.21.15.89:59596) 2023-10-10T20:35:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.391+0000 7f91d6180700 1 -- 172.21.15.89:0/79865143 learned_addr learned my addr 172.21.15.89:0/79865143 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.391+0000 7f91d517e700 1 --2- 172.21.15.89:0/79865143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91d006bac0 0x7f91d00ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.391+0000 7f91d597f700 1 --2- 172.21.15.89:0/79865143 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91d006aa40 0x7f91d014dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.391+0000 7f91d6180700 1 -- 172.21.15.89:0/79865143 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91d006aa40 msgr2=0x7f91d014dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.391+0000 7f91d6180700 1 --2- 172.21.15.89:0/79865143 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91d006aa40 0x7f91d014dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.391+0000 7f91d6180700 1 -- 172.21.15.89:0/79865143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91d006bac0 msgr2=0x7f91d00ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.391+0000 7f91d6180700 1 --2- 172.21.15.89:0/79865143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91d006bac0 0x7f91d00ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.391+0000 7f91d6180700 1 -- 172.21.15.89:0/79865143 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c8009420 con 0x7f91d00c91e0 2023-10-10T20:35:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.391+0000 7f91d6180700 1 --2- 172.21.15.89:0/79865143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91d00c91e0 0x7f91d01555c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f91d00ce660 tx=0x7f91c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.391+0000 7f91c5ffb700 1 -- 172.21.15.89:0/79865143 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c800bb20 con 0x7f91d00c91e0 2023-10-10T20:35:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.391+0000 7f91c5ffb700 1 -- 172.21.15.89:0/79865143 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c8011420 con 0x7f91d00c91e0 2023-10-10T20:35:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.391+0000 7f91d7be3700 1 -- 172.21.15.89:0/79865143 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d0068d30 con 0x7f91d00c91e0 2023-10-10T20:35:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.391+0000 7f91c5ffb700 1 -- 172.21.15.89:0/79865143 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c80117a0 con 0x7f91d00c91e0 2023-10-10T20:35:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.391+0000 7f91d7be3700 1 -- 172.21.15.89:0/79865143 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d01519a0 con 0x7f91d00c91e0 2023-10-10T20:35:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.392+0000 7f91d7be3700 1 -- 172.21.15.89:0/79865143 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d0151c10 con 0x7f91d00c91e0 2023-10-10T20:35:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.393+0000 7f91c5ffb700 1 -- 172.21.15.89:0/79865143 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f91c8011b10 con 0x7f91d00c91e0 2023-10-10T20:35:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.393+0000 7f91c5ffb700 1 --2- 172.21.15.89:0/79865143 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f91b4066f30 0x7f91b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.393+0000 7f91c5ffb700 1 -- 172.21.15.89:0/79865143 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f91c8086020 con 0x7f91d00c91e0 2023-10-10T20:35:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.393+0000 7f91d597f700 1 --2- 172.21.15.89:0/79865143 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f91b4066f30 0x7f91b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.393+0000 7f91d597f700 1 --2- 172.21.15.89:0/79865143 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f91b4066f30 0x7f91b40693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00f5b0 tx=0x7f91cc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.397+0000 7f91c5ffb700 1 -- 172.21.15.89:0/79865143 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91c8053290 con 0x7f91d00c91e0 2023-10-10T20:35:32.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.581+0000 7f91d7be3700 1 -- 172.21.15.89:0/79865143 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f91d004c3c0 con 0x7f91d00c91e0 2023-10-10T20:35:32.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.581+0000 7f91c5ffb700 1 -- 172.21.15.89:0/79865143 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f91c80568b0 con 0x7f91d00c91e0 2023-10-10T20:35:32.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.583+0000 7f91d7be3700 1 -- 172.21.15.89:0/79865143 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f91b4066f30 msgr2=0x7f91b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:32.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.583+0000 7f91d7be3700 1 --2- 172.21.15.89:0/79865143 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f91b4066f30 0x7f91b40693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00f5b0 tx=0x7f91cc00b040 comp rx=0 tx=0).stop 2023-10-10T20:35:32.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.583+0000 7f91d7be3700 1 -- 172.21.15.89:0/79865143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91d00c91e0 msgr2=0x7f91d01555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:32.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.583+0000 7f91d7be3700 1 --2- 172.21.15.89:0/79865143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91d00c91e0 0x7f91d01555c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f91d00ce660 tx=0x7f91c8017000 comp rx=0 tx=0).stop 2023-10-10T20:35:32.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.583+0000 7f91d7be3700 1 -- 172.21.15.89:0/79865143 shutdown_connections 2023-10-10T20:35:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.583+0000 7f91d7be3700 1 --2- 172.21.15.89:0/79865143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91d00c91e0 0x7f91d01555c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.583+0000 7f91d7be3700 1 --2- 172.21.15.89:0/79865143 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91d006aa40 0x7f91d014dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.583+0000 7f91d7be3700 1 --2- 172.21.15.89:0/79865143 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f91b4066f30 0x7f91b40693e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.583+0000 7f91d7be3700 1 --2- 172.21.15.89:0/79865143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91d006bac0 0x7f91d00ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.583+0000 7f91d7be3700 1 -- 172.21.15.89:0/79865143 >> 172.21.15.89:0/79865143 conn(0x7f91d00cd670 msgr2=0x7f91d0153890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.583+0000 7f91d7be3700 1 -- 172.21.15.89:0/79865143 shutdown_connections 2023-10-10T20:35:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.583+0000 7f91d7be3700 1 -- 172.21.15.89:0/79865143 wait complete. 2023-10-10T20:35:32.598 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000000680185 2023-10-10T20:35:32.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.712+0000 7fddedd5b700 1 Processor -- start 2023-10-10T20:35:32.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.712+0000 7fddedd5b700 1 -- start start 2023-10-10T20:35:32.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.713+0000 7fddedd5b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdde806c0a0 0x7fdde80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:32.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.713+0000 7fddedd5b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdde815df50 0x7fdde815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:32.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.713+0000 7fddedd5b700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde815d7e0 con 0x7fdde806bcd0 2023-10-10T20:35:32.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.713+0000 7fddedd5b700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde815dac0 con 0x7fdde806c0a0 2023-10-10T20:35:32.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.713+0000 7fddedd5b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde815aa30 con 0x7fdde815df50 2023-10-10T20:35:32.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.713+0000 7fdde7fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdde815df50 0x7fdde815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:32.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.714+0000 7fdde6ffd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdde806c0a0 0x7fdde80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:32.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.714+0000 7fdde7fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdde815df50 0x7fdde815e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:58526/0 (socket says 172.21.15.89:58526) 2023-10-10T20:35:32.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.714+0000 7fdde7fff700 1 -- 172.21.15.89:0/3978036192 learned_addr learned my addr 172.21.15.89:0/3978036192 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:32.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.714+0000 7fdde77fe700 1 --1- 172.21.15.89:0/3978036192 >> v1:172.21.15.111:6789/0 conn(0x7fdde806bcd0 0x7fdde815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:57594/0 (socket says 172.21.15.89:57594) 2023-10-10T20:35:32.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.714+0000 7fdde6ffd700 1 -- 172.21.15.89:0/3978036192 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdde815df50 msgr2=0x7fdde815e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:32.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.714+0000 7fdde6ffd700 1 --2- 172.21.15.89:0/3978036192 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdde815df50 0x7fdde815e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.714+0000 7fdde6ffd700 1 -- 172.21.15.89:0/3978036192 >> v1:172.21.15.111:6789/0 conn(0x7fdde806bcd0 legacy=0x7fdde815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:32.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.714+0000 7fdde6ffd700 1 -- 172.21.15.89:0/3978036192 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde815ad10 con 0x7fdde806c0a0 2023-10-10T20:35:32.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.714+0000 7fdde67fc700 1 -- 172.21.15.89:0/3978036192 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321853429 0 0) 0x7fdde815d7e0 con 0x7fdde806bcd0 2023-10-10T20:35:32.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.714+0000 7fdde7fff700 1 --2- 172.21.15.89:0/3978036192 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdde815df50 0x7fdde815e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:35:32.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.714+0000 7fdde6ffd700 1 --2- 172.21.15.89:0/3978036192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdde806c0a0 0x7fdde80caa90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0098e0 tx=0x7fdddc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ab2c53272979e33 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:32.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.714+0000 7fdde67fc700 1 -- 172.21.15.89:0/3978036192 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc00b640 con 0x7fdde806c0a0 2023-10-10T20:35:32.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.714+0000 7fdde67fc700 1 -- 172.21.15.89:0/3978036192 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdddc00b7a0 con 0x7fdde806c0a0 2023-10-10T20:35:32.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.714+0000 7fdde67fc700 1 -- 172.21.15.89:0/3978036192 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc00bb40 con 0x7fdde806c0a0 2023-10-10T20:35:32.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.715+0000 7fddedd5b700 1 -- 172.21.15.89:0/3978036192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdde806c0a0 msgr2=0x7fdde80caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:32.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.715+0000 7fddedd5b700 1 --2- 172.21.15.89:0/3978036192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdde806c0a0 0x7fdde80caa90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0098e0 tx=0x7fdddc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.715+0000 7fddedd5b700 1 -- 172.21.15.89:0/3978036192 shutdown_connections 2023-10-10T20:35:32.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.715+0000 7fddedd5b700 1 --2- 172.21.15.89:0/3978036192 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdde815df50 0x7fdde815e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.715+0000 7fddedd5b700 1 --2- 172.21.15.89:0/3978036192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdde806c0a0 0x7fdde80caa90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.715+0000 7fddedd5b700 1 -- 172.21.15.89:0/3978036192 >> 172.21.15.89:0/3978036192 conn(0x7fdde80ce600 msgr2=0x7fdde80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:32.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.715+0000 7fddedd5b700 1 -- 172.21.15.89:0/3978036192 shutdown_connections 2023-10-10T20:35:32.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.715+0000 7fddedd5b700 1 -- 172.21.15.89:0/3978036192 wait complete. 2023-10-10T20:35:32.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.716+0000 7fddedd5b700 1 Processor -- start 2023-10-10T20:35:32.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.716+0000 7fddedd5b700 1 -- start start 2023-10-10T20:35:32.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.717+0000 7fddedd5b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdde806bcd0 0x7fdde80b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:32.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.717+0000 7fddedd5b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdde806c0a0 0x7fdde80c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:32.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.717+0000 7fddedd5b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdde815df50 0x7fdde80c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:32.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.717+0000 7fddedd5b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde80b4370 con 0x7fdde806bcd0 2023-10-10T20:35:32.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.717+0000 7fddedd5b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde816e010 con 0x7fdde806c0a0 2023-10-10T20:35:32.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.717+0000 7fddedd5b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde816e2f0 con 0x7fdde815df50 2023-10-10T20:35:32.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.717+0000 7fdde77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdde806bcd0 0x7fdde80b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:32.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.717+0000 7fdde77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdde806bcd0 0x7fdde80b38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59604/0 (socket says 172.21.15.89:59604) 2023-10-10T20:35:32.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.717+0000 7fdde77fe700 1 -- 172.21.15.89:0/1009269869 learned_addr learned my addr 172.21.15.89:0/1009269869 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:32.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.717+0000 7fdde6ffd700 1 --2- 172.21.15.89:0/1009269869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdde806c0a0 0x7fdde80c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:32.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.717+0000 7fdde7fff700 1 --2- 172.21.15.89:0/1009269869 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdde815df50 0x7fdde80c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:32.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.717+0000 7fdde77fe700 1 -- 172.21.15.89:0/1009269869 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdde815df50 msgr2=0x7fdde80c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:32.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.717+0000 7fdde77fe700 1 --2- 172.21.15.89:0/1009269869 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdde815df50 0x7fdde80c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.717+0000 7fdde77fe700 1 -- 172.21.15.89:0/1009269869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdde806c0a0 msgr2=0x7fdde80c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:32.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.717+0000 7fdde77fe700 1 --2- 172.21.15.89:0/1009269869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdde806c0a0 0x7fdde80c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:32.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.717+0000 7fdde77fe700 1 -- 172.21.15.89:0/1009269869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde0009ae0 con 0x7fdde806bcd0 2023-10-10T20:35:32.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.717+0000 7fdde77fe700 1 --2- 172.21.15.89:0/1009269869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdde806bcd0 0x7fdde80b38f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fddd800c5b0 tx=0x7fddd8001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:32.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.717+0000 7fddcf7fe700 1 -- 172.21.15.89:0/1009269869 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8005a70 con 0x7fdde806bcd0 2023-10-10T20:35:32.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.718+0000 7fddcf7fe700 1 -- 172.21.15.89:0/1009269869 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd8005bd0 con 0x7fdde806bcd0 2023-10-10T20:35:32.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.718+0000 7fddedd5b700 1 -- 172.21.15.89:0/1009269869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdddc009420 con 0x7fdde806bcd0 2023-10-10T20:35:32.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.718+0000 7fddcf7fe700 1 -- 172.21.15.89:0/1009269869 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd80070d0 con 0x7fdde806bcd0 2023-10-10T20:35:32.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.718+0000 7fddedd5b700 1 -- 172.21.15.89:0/1009269869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde816e8b0 con 0x7fdde806bcd0 2023-10-10T20:35:32.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.719+0000 7fddedd5b700 1 -- 172.21.15.89:0/1009269869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde80b3e30 con 0x7fdde806bcd0 2023-10-10T20:35:32.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.719+0000 7fddcf7fe700 1 -- 172.21.15.89:0/1009269869 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fddd8007480 con 0x7fdde806bcd0 2023-10-10T20:35:32.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.719+0000 7fddcf7fe700 1 --2- 172.21.15.89:0/1009269869 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fddd4066f30 0x7fddd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:32.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.720+0000 7fdde6ffd700 1 --2- 172.21.15.89:0/1009269869 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fddd4066f30 0x7fddd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:32.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.720+0000 7fddcf7fe700 1 -- 172.21.15.89:0/1009269869 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fddd8083a50 con 0x7fdde806bcd0 2023-10-10T20:35:32.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.720+0000 7fdde6ffd700 1 --2- 172.21.15.89:0/1009269869 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fddd4066f30 0x7fddd40693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0093f0 tx=0x7fdddc017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:32.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.724+0000 7fddcf7fe700 1 -- 172.21.15.89:0/1009269869 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fddd8050cc0 con 0x7fdde806bcd0 2023-10-10T20:35:32.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:32.936+0000 7fddedd5b700 1 -- 172.21.15.89:0/1009269869 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000000680185", "target": ["mon-mgr", ""]}) v1 -- 0x7fdde804c410 con 0x7fddd4066f30 2023-10-10T20:35:35.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:35.941+0000 7fddcf7fe700 1 -- 172.21.15.89:0/1009269869 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7fdde804c410 con 0x7fddd4066f30 2023-10-10T20:35:35.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:35.946+0000 7fddedd5b700 1 -- 172.21.15.89:0/1009269869 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fddd4066f30 msgr2=0x7fddd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:35.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:35.946+0000 7fddedd5b700 1 --2- 172.21.15.89:0/1009269869 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fddd4066f30 0x7fddd40693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0093f0 tx=0x7fdddc017000 comp rx=0 tx=0).stop 2023-10-10T20:35:35.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:35.946+0000 7fddedd5b700 1 -- 172.21.15.89:0/1009269869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdde806bcd0 msgr2=0x7fdde80b38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:35.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:35.946+0000 7fddedd5b700 1 --2- 172.21.15.89:0/1009269869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdde806bcd0 0x7fdde80b38f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fddd800c5b0 tx=0x7fddd8001040 comp rx=0 tx=0).stop 2023-10-10T20:35:35.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:35.947+0000 7fddedd5b700 1 -- 172.21.15.89:0/1009269869 shutdown_connections 2023-10-10T20:35:35.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:35.947+0000 7fddedd5b700 1 --2- 172.21.15.89:0/1009269869 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdde815df50 0x7fdde80c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:35.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:35.947+0000 7fddedd5b700 1 --2- 172.21.15.89:0/1009269869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdde806bcd0 0x7fdde80b38f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:35.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:35.947+0000 7fddedd5b700 1 --2- 172.21.15.89:0/1009269869 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fddd4066f30 0x7fddd40693e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:35.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:35.947+0000 7fddedd5b700 1 --2- 172.21.15.89:0/1009269869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdde806c0a0 0x7fdde80c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:35.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:35.947+0000 7fddedd5b700 1 -- 172.21.15.89:0/1009269869 >> 172.21.15.89:0/1009269869 conn(0x7fdde80ce600 msgr2=0x7fdde806a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:35.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:35.947+0000 7fddedd5b700 1 -- 172.21.15.89:0/1009269869 shutdown_connections 2023-10-10T20:35:35.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:35.947+0000 7fddedd5b700 1 -- 172.21.15.89:0/1009269869 wait complete. 2023-10-10T20:35:35.954 INFO:teuthology.orchestra.run.smithi089.stderr:Volume created successfully (no MDS daemons created) 2023-10-10T20:35:35.967 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-10T20:35:36.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.096+0000 7efde2c7b700 1 Processor -- start 2023-10-10T20:35:36.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.096+0000 7efde2c7b700 1 -- start start 2023-10-10T20:35:36.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.097+0000 7efde2c7b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efddc159840 0x7efddc0cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.097+0000 7efde2c7b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7efddc069bb0 0x7efddc0b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.097+0000 7efde2c7b700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efddc0cfb30 con 0x7efddc0cba70 2023-10-10T20:35:36.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.097+0000 7efde2c7b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efddc0cbf10 con 0x7efddc159840 2023-10-10T20:35:36.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.097+0000 7efde2c7b700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efddc0cf4e0 con 0x7efddc069bb0 2023-10-10T20:35:36.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.097+0000 7efde0a17700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efddc159840 0x7efddc0cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.097+0000 7efde0a17700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efddc159840 0x7efddc0cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59612/0 (socket says 172.21.15.89:59612) 2023-10-10T20:35:36.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.097+0000 7efde0a17700 1 -- 172.21.15.89:0/475720868 learned_addr learned my addr 172.21.15.89:0/475720868 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:36.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.097+0000 7efde1218700 1 --2- 172.21.15.89:0/475720868 >> v2:172.21.15.111:3300/0 conn(0x7efddc069bb0 0x7efddc0b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.098+0000 7efde0a17700 1 -- 172.21.15.89:0/475720868 >> v2:172.21.15.111:3300/0 conn(0x7efddc069bb0 msgr2=0x7efddc0b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:36.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.098+0000 7efde0a17700 1 --2- 172.21.15.89:0/475720868 >> v2:172.21.15.111:3300/0 conn(0x7efddc069bb0 0x7efddc0b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.098+0000 7efde0a17700 1 -- 172.21.15.89:0/475720868 >> v1:172.21.15.111:6789/0 conn(0x7efddc0cba70 legacy=0x7efddc0694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:36.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.098+0000 7efde0a17700 1 -- 172.21.15.89:0/475720868 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc0cf7c0 con 0x7efddc159840 2023-10-10T20:35:36.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.098+0000 7efde0a17700 1 --2- 172.21.15.89:0/475720868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efddc159840 0x7efddc0cb530 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009f20 tx=0x7efdd8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb9727aa7332b72 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:36.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.098+0000 7efdd37fe700 1 -- 172.21.15.89:0/475720868 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd800e070 con 0x7efddc159840 2023-10-10T20:35:36.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.098+0000 7efdd37fe700 1 -- 172.21.15.89:0/475720868 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd800ba00 con 0x7efddc159840 2023-10-10T20:35:36.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.098+0000 7efdd37fe700 1 -- 172.21.15.89:0/475720868 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd80092e0 con 0x7efddc159840 2023-10-10T20:35:36.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.098+0000 7efde2c7b700 1 -- 172.21.15.89:0/475720868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efddc159840 msgr2=0x7efddc0cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:36.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.098+0000 7efde2c7b700 1 --2- 172.21.15.89:0/475720868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efddc159840 0x7efddc0cb530 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009f20 tx=0x7efdd8009840 comp rx=0 tx=0).stop 2023-10-10T20:35:36.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.098+0000 7efde2c7b700 1 -- 172.21.15.89:0/475720868 shutdown_connections 2023-10-10T20:35:36.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.098+0000 7efde2c7b700 1 --2- 172.21.15.89:0/475720868 >> v2:172.21.15.111:3300/0 conn(0x7efddc069bb0 0x7efddc0b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.098+0000 7efde2c7b700 1 --2- 172.21.15.89:0/475720868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efddc159840 0x7efddc0cb530 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.098+0000 7efde2c7b700 1 -- 172.21.15.89:0/475720868 >> 172.21.15.89:0/475720868 conn(0x7efddc068e10 msgr2=0x7efddc0d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:36.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.098+0000 7efde2c7b700 1 -- 172.21.15.89:0/475720868 shutdown_connections 2023-10-10T20:35:36.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.098+0000 7efde2c7b700 1 -- 172.21.15.89:0/475720868 wait complete. 2023-10-10T20:35:36.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.099+0000 7efde2c7b700 1 Processor -- start 2023-10-10T20:35:36.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.100+0000 7efde2c7b700 1 -- start start 2023-10-10T20:35:36.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.100+0000 7efde2c7b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efddc069bb0 0x7efddc159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.100+0000 7efde2c7b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efddc0cba70 0x7efddc169ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.100+0000 7efde2c7b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efddc159840 0x7efddc152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.100+0000 7efde2c7b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc15b290 con 0x7efddc0cba70 2023-10-10T20:35:36.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.100+0000 7efde2c7b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc15a1f0 con 0x7efddc069bb0 2023-10-10T20:35:36.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.100+0000 7efde2c7b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc15a4d0 con 0x7efddc159840 2023-10-10T20:35:36.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.100+0000 7efdd3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efddc0cba70 0x7efddc169ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.100+0000 7efdd3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efddc0cba70 0x7efddc169ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59620/0 (socket says 172.21.15.89:59620) 2023-10-10T20:35:36.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.101+0000 7efdd3fff700 1 -- 172.21.15.89:0/2501685087 learned_addr learned my addr 172.21.15.89:0/2501685087 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:36.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.101+0000 7efde1218700 1 --2- 172.21.15.89:0/2501685087 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efddc159840 0x7efddc152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.101+0000 7efde0a17700 1 --2- 172.21.15.89:0/2501685087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efddc069bb0 0x7efddc159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.101+0000 7efdd3fff700 1 -- 172.21.15.89:0/2501685087 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efddc159840 msgr2=0x7efddc152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:36.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.101+0000 7efdd3fff700 1 --2- 172.21.15.89:0/2501685087 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efddc159840 0x7efddc152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.101+0000 7efdd3fff700 1 -- 172.21.15.89:0/2501685087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efddc069bb0 msgr2=0x7efddc159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:36.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.101+0000 7efdd3fff700 1 --2- 172.21.15.89:0/2501685087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efddc069bb0 0x7efddc159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.101+0000 7efdd3fff700 1 -- 172.21.15.89:0/2501685087 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd8009ae0 con 0x7efddc0cba70 2023-10-10T20:35:36.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.101+0000 7efdd3fff700 1 --2- 172.21.15.89:0/2501685087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efddc0cba70 0x7efddc169ea0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00c5d0 tx=0x7efdcc007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:36.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.101+0000 7efdd0ff9700 1 -- 172.21.15.89:0/2501685087 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc011590 con 0x7efddc0cba70 2023-10-10T20:35:36.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.101+0000 7efdd0ff9700 1 -- 172.21.15.89:0/2501685087 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdcc011710 con 0x7efddc0cba70 2023-10-10T20:35:36.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.101+0000 7efde2c7b700 1 -- 172.21.15.89:0/2501685087 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efddc15a7b0 con 0x7efddc0cba70 2023-10-10T20:35:36.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.101+0000 7efdd0ff9700 1 -- 172.21.15.89:0/2501685087 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc011ad0 con 0x7efddc0cba70 2023-10-10T20:35:36.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.101+0000 7efde2c7b700 1 -- 172.21.15.89:0/2501685087 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efddc15ad70 con 0x7efddc0cba70 2023-10-10T20:35:36.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.102+0000 7efde2c7b700 1 -- 172.21.15.89:0/2501685087 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efddc064550 con 0x7efddc0cba70 2023-10-10T20:35:36.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.102+0000 7efdd0ff9700 1 -- 172.21.15.89:0/2501685087 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efdcc01c460 con 0x7efddc0cba70 2023-10-10T20:35:36.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.103+0000 7efdd0ff9700 1 --2- 172.21.15.89:0/2501685087 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efdc8066f30 0x7efdc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.103+0000 7efdd0ff9700 1 -- 172.21.15.89:0/2501685087 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6603+0+0 (secure 0 0 0) 0x7efdcc084530 con 0x7efddc0cba70 2023-10-10T20:35:36.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.103+0000 7efde0a17700 1 --2- 172.21.15.89:0/2501685087 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efdc8066f30 0x7efdc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.103+0000 7efde0a17700 1 --2- 172.21.15.89:0/2501685087 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efdc8066f30 0x7efdc80693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efddc0cf700 tx=0x7efdd8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:36.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.107+0000 7efdd0ff9700 1 -- 172.21.15.89:0/2501685087 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efdcc051420 con 0x7efddc0cba70 2023-10-10T20:35:36.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.319+0000 7efde2c7b700 1 -- 172.21.15.89:0/2501685087 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7efddc15af90 con 0x7efdc8066f30 2023-10-10T20:35:36.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.320+0000 7efdd0ff9700 1 -- 172.21.15.89:0/2501685087 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+96 (secure 0 0 0) 0x7efddc15af90 con 0x7efdc8066f30 2023-10-10T20:35:36.323 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:35:36.323 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:36.323 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs" 2023-10-10T20:35:36.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:36.324 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:36.324 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "volume_0000000000680185" 2023-10-10T20:35:36.324 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:36.325 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:35:36.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.323+0000 7efde2c7b700 1 -- 172.21.15.89:0/2501685087 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efdc8066f30 msgr2=0x7efdc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:36.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.323+0000 7efde2c7b700 1 --2- 172.21.15.89:0/2501685087 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efdc8066f30 0x7efdc80693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efddc0cf700 tx=0x7efdd8017000 comp rx=0 tx=0).stop 2023-10-10T20:35:36.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.323+0000 7efde2c7b700 1 -- 172.21.15.89:0/2501685087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efddc0cba70 msgr2=0x7efddc169ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:36.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.323+0000 7efde2c7b700 1 --2- 172.21.15.89:0/2501685087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efddc0cba70 0x7efddc169ea0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00c5d0 tx=0x7efdcc007f60 comp rx=0 tx=0).stop 2023-10-10T20:35:36.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.323+0000 7efde2c7b700 1 -- 172.21.15.89:0/2501685087 shutdown_connections 2023-10-10T20:35:36.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.323+0000 7efde2c7b700 1 --2- 172.21.15.89:0/2501685087 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efddc159840 0x7efddc152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.323+0000 7efde2c7b700 1 --2- 172.21.15.89:0/2501685087 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efdc8066f30 0x7efdc80693e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.323+0000 7efde2c7b700 1 --2- 172.21.15.89:0/2501685087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efddc069bb0 0x7efddc159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.323+0000 7efde2c7b700 1 --2- 172.21.15.89:0/2501685087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efddc0cba70 0x7efddc169ea0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.323+0000 7efde2c7b700 1 -- 172.21.15.89:0/2501685087 >> 172.21.15.89:0/2501685087 conn(0x7efddc068e10 msgr2=0x7efddc154cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:36.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.323+0000 7efde2c7b700 1 -- 172.21.15.89:0/2501685087 shutdown_connections 2023-10-10T20:35:36.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.323+0000 7efde2c7b700 1 -- 172.21.15.89:0/2501685087 wait complete. 2023-10-10T20:35:36.337 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool ls detail --format=json 2023-10-10T20:35:36.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.452+0000 7f0dfd923700 1 Processor -- start 2023-10-10T20:35:36.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.452+0000 7f0dfd923700 1 -- start start 2023-10-10T20:35:36.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.453+0000 7f0dfd923700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0df8159890 0x7f0df815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.453+0000 7f0dfd923700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0df8159c60 0x7f0df815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.453+0000 7f0dfd923700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0df815a160 con 0x7f0df815db10 2023-10-10T20:35:36.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.453+0000 7f0dfd923700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df80d13a0 con 0x7f0df8159890 2023-10-10T20:35:36.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.453+0000 7f0dfd923700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df815e140 con 0x7f0df8159c60 2023-10-10T20:35:36.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.453+0000 7f0df6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0df8159890 0x7f0df815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.453+0000 7f0df77fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0df815db10 0x7f0df8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41824/0 (socket says 172.21.15.89:41824) 2023-10-10T20:35:36.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.453+0000 7f0df77fe700 1 -- 172.21.15.89:0/237528679 learned_addr learned my addr 172.21.15.89:0/237528679 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:36.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.453+0000 7f0dee7fc700 1 --2- 172.21.15.89:0/237528679 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0df8159c60 0x7f0df815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.454+0000 7f0df6ffd700 1 -- 172.21.15.89:0/237528679 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0df8159c60 msgr2=0x7f0df815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:36.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.454+0000 7f0df6ffd700 1 --2- 172.21.15.89:0/237528679 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0df8159c60 0x7f0df815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.454+0000 7f0df6ffd700 1 -- 172.21.15.89:0/237528679 >> v1:172.21.15.89:6789/0 conn(0x7f0df815db10 legacy=0x7f0df8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:36.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.454+0000 7f0df6ffd700 1 -- 172.21.15.89:0/237528679 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df815a3d0 con 0x7f0df8159890 2023-10-10T20:35:36.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.454+0000 7f0df67fc700 1 -- 172.21.15.89:0/237528679 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570689057 0 0) 0x7f0df815a160 con 0x7f0df815db10 2023-10-10T20:35:36.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.454+0000 7f0df6ffd700 1 --2- 172.21.15.89:0/237528679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df8159890 0x7f0df815abd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0de80098e0 tx=0x7f0de80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686c3dbc12e2bfb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:36.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.454+0000 7f0df67fc700 1 -- 172.21.15.89:0/237528679 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de800b640 con 0x7f0df8159890 2023-10-10T20:35:36.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.454+0000 7f0df67fc700 1 -- 172.21.15.89:0/237528679 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de800b7a0 con 0x7f0df8159890 2023-10-10T20:35:36.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.454+0000 7f0df67fc700 1 -- 172.21.15.89:0/237528679 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de800bb40 con 0x7f0df8159890 2023-10-10T20:35:36.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.454+0000 7f0dfd923700 1 -- 172.21.15.89:0/237528679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df8159890 msgr2=0x7f0df815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:36.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.454+0000 7f0dfd923700 1 --2- 172.21.15.89:0/237528679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df8159890 0x7f0df815abd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0de80098e0 tx=0x7f0de80091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.455+0000 7f0dfd923700 1 -- 172.21.15.89:0/237528679 shutdown_connections 2023-10-10T20:35:36.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.455+0000 7f0dfd923700 1 --2- 172.21.15.89:0/237528679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df8159890 0x7f0df815abd0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.455+0000 7f0dfd923700 1 --2- 172.21.15.89:0/237528679 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0df8159c60 0x7f0df815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.455+0000 7f0dfd923700 1 -- 172.21.15.89:0/237528679 >> 172.21.15.89:0/237528679 conn(0x7f0df8068e20 msgr2=0x7f0df80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:36.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.455+0000 7f0dfd923700 1 -- 172.21.15.89:0/237528679 shutdown_connections 2023-10-10T20:35:36.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.455+0000 7f0dfd923700 1 -- 172.21.15.89:0/237528679 wait complete. 2023-10-10T20:35:36.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.456+0000 7f0dfd923700 1 Processor -- start 2023-10-10T20:35:36.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.456+0000 7f0dfd923700 1 -- start start 2023-10-10T20:35:36.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.456+0000 7f0dfd923700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0df8159890 0x7f0df80cf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.456+0000 7f0dfd923700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0df8159c60 0x7f0df80c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.456+0000 7f0dfd923700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df815db10 0x7f0df80cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.456+0000 7f0dfd923700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df80ce140 con 0x7f0df815db10 2023-10-10T20:35:36.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.457+0000 7f0dfd923700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df80b36c0 con 0x7f0df8159c60 2023-10-10T20:35:36.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.457+0000 7f0dfd923700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df80cd600 con 0x7f0df8159890 2023-10-10T20:35:36.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.457+0000 7f0df77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df815db10 0x7f0df80cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.457+0000 7f0df77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df815db10 0x7f0df80cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59632/0 (socket says 172.21.15.89:59632) 2023-10-10T20:35:36.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.457+0000 7f0df77fe700 1 -- 172.21.15.89:0/2728915497 learned_addr learned my addr 172.21.15.89:0/2728915497 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:36.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.457+0000 7f0df6ffd700 1 --2- 172.21.15.89:0/2728915497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0df8159890 0x7f0df80cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.457+0000 7f0dee7fc700 1 --2- 172.21.15.89:0/2728915497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0df8159c60 0x7f0df80c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.457+0000 7f0df77fe700 1 -- 172.21.15.89:0/2728915497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0df8159890 msgr2=0x7f0df80cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:36.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.457+0000 7f0df77fe700 1 --2- 172.21.15.89:0/2728915497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0df8159890 0x7f0df80cf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.457+0000 7f0df77fe700 1 -- 172.21.15.89:0/2728915497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0df8159c60 msgr2=0x7f0df80c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:36.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.457+0000 7f0df77fe700 1 --2- 172.21.15.89:0/2728915497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0df8159c60 0x7f0df80c7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.457+0000 7f0df77fe700 1 -- 172.21.15.89:0/2728915497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de8009420 con 0x7f0df815db10 2023-10-10T20:35:36.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.457+0000 7f0df77fe700 1 --2- 172.21.15.89:0/2728915497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df815db10 0x7f0df80cd0c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0df000cc20 tx=0x7f0df00019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:36.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.457+0000 7f0def7fe700 1 -- 172.21.15.89:0/2728915497 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df0004bb0 con 0x7f0df815db10 2023-10-10T20:35:36.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.457+0000 7f0def7fe700 1 -- 172.21.15.89:0/2728915497 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df0004d10 con 0x7f0df815db10 2023-10-10T20:35:36.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.457+0000 7f0dfd923700 1 -- 172.21.15.89:0/2728915497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df80cd8a0 con 0x7f0df815db10 2023-10-10T20:35:36.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.457+0000 7f0def7fe700 1 -- 172.21.15.89:0/2728915497 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df000abd0 con 0x7f0df815db10 2023-10-10T20:35:36.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.457+0000 7f0dfd923700 1 -- 172.21.15.89:0/2728915497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df80cde60 con 0x7f0df815db10 2023-10-10T20:35:36.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.458+0000 7f0dfd923700 1 -- 172.21.15.89:0/2728915497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df804c3c0 con 0x7f0df815db10 2023-10-10T20:35:36.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.459+0000 7f0def7fe700 1 -- 172.21.15.89:0/2728915497 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0df00070f0 con 0x7f0df815db10 2023-10-10T20:35:36.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.459+0000 7f0def7fe700 1 --2- 172.21.15.89:0/2728915497 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0dd8066f30 0x7f0dd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.459+0000 7f0df6ffd700 1 --2- 172.21.15.89:0/2728915497 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0dd8066f30 0x7f0dd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.459+0000 7f0def7fe700 1 -- 172.21.15.89:0/2728915497 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f0df00846f0 con 0x7f0df815db10 2023-10-10T20:35:36.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.460+0000 7f0df6ffd700 1 --2- 172.21.15.89:0/2728915497 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0dd8066f30 0x7f0dd80693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0de8003080 tx=0x7f0de8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:36.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.463+0000 7f0def7fe700 1 -- 172.21.15.89:0/2728915497 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0df0051660 con 0x7f0df815db10 2023-10-10T20:35:36.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.646+0000 7f0dfd923700 1 -- 172.21.15.89:0/2728915497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool ls", "detail": "detail", "format": "json"} v 0) v1 -- 0x7f0df80b3800 con 0x7f0df815db10 2023-10-10T20:35:36.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.646+0000 7f0def7fe700 1 -- 172.21.15.89:0/2728915497 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls", "detail": "detail", "format": "json"}]=0 v34) v1 ==== 97+0+8050 (secure 0 0 0) 0x7f0df00180a0 con 0x7f0df815db10 2023-10-10T20:35:36.649 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:36.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.648+0000 7f0dfd923700 1 -- 172.21.15.89:0/2728915497 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0dd8066f30 msgr2=0x7f0dd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:36.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.648+0000 7f0dfd923700 1 --2- 172.21.15.89:0/2728915497 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0dd8066f30 0x7f0dd80693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0de8003080 tx=0x7f0de8017000 comp rx=0 tx=0).stop 2023-10-10T20:35:36.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.648+0000 7f0dfd923700 1 -- 172.21.15.89:0/2728915497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df815db10 msgr2=0x7f0df80cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:36.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.648+0000 7f0dfd923700 1 --2- 172.21.15.89:0/2728915497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df815db10 0x7f0df80cd0c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0df000cc20 tx=0x7f0df00019b0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.648+0000 7f0dfd923700 1 -- 172.21.15.89:0/2728915497 shutdown_connections 2023-10-10T20:35:36.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.648+0000 7f0dfd923700 1 --2- 172.21.15.89:0/2728915497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df815db10 0x7f0df80cd0c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.648+0000 7f0dfd923700 1 --2- 172.21.15.89:0/2728915497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0df8159890 0x7f0df80cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.648+0000 7f0dfd923700 1 --2- 172.21.15.89:0/2728915497 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0dd8066f30 0x7f0dd80693e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.648+0000 7f0dfd923700 1 --2- 172.21.15.89:0/2728915497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0df8159c60 0x7f0df80c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.648+0000 7f0dfd923700 1 -- 172.21.15.89:0/2728915497 >> 172.21.15.89:0/2728915497 conn(0x7f0df8068e20 msgr2=0x7f0df814d680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:36.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.648+0000 7f0dfd923700 1 -- 172.21.15.89:0/2728915497 shutdown_connections 2023-10-10T20:35:36.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.649+0000 7f0dfd923700 1 -- 172.21.15.89:0/2728915497 wait complete. 2023-10-10T20:35:36.662 INFO:teuthology.orchestra.run.smithi089.stdout:[{"pool_id":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool_id":4,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:35:15.667057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool_id":5,"pool_name":"cephfs_data","create_time":"2023-10-10T20:35:16.860131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool_id":6,"pool_name":"cephfs.volume_0000000000680185.meta","create_time":"2023-10-10T20:35:32.954945+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000680185"}}},{"pool_id":7,"pool_name":"cephfs.volume_0000000000680185.data","create_time":"2023-10-10T20:35:33.949829+0000","flags":163841,"flags_names":"hashpspool,creating,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000680185"}}}] 2023-10-10T20:35:36.663 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs get volume_0000000000680185 --format=json 2023-10-10T20:35:36.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.778+0000 7f145ce0d700 1 Processor -- start 2023-10-10T20:35:36.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.778+0000 7f145ce0d700 1 -- start start 2023-10-10T20:35:36.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f145ce0d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f145806c0a0 0x7f14580caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f145ce0d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f145815df50 0x7f145815e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f145ce0d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f145815d7e0 con 0x7f145806bcd0 2023-10-10T20:35:36.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f145ce0d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f145815dac0 con 0x7f145815df50 2023-10-10T20:35:36.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f145ce0d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145815aa30 con 0x7f145806c0a0 2023-10-10T20:35:36.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f145659c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f145806bcd0 0x7f145815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41826/0 (socket says 172.21.15.89:41826) 2023-10-10T20:35:36.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f145659c700 1 -- 172.21.15.89:0/714849984 learned_addr learned my addr 172.21.15.89:0/714849984 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:36.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f145559a700 1 -- 172.21.15.89:0/714849984 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596973116 0 0) 0x7f145815d7e0 con 0x7f145806bcd0 2023-10-10T20:35:36.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f1456d9d700 1 --2- 172.21.15.89:0/714849984 >> v2:172.21.15.111:3300/0 conn(0x7f145815df50 0x7f145815e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f1455d9b700 1 --2- 172.21.15.89:0/714849984 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f145806c0a0 0x7f14580caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f145559a700 1 -- 172.21.15.89:0/714849984 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1440003570 con 0x7f145806bcd0 2023-10-10T20:35:36.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f145559a700 1 -- 172.21.15.89:0/714849984 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3840213644 0 0) 0x7f1440003570 con 0x7f145806bcd0 2023-10-10T20:35:36.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f145559a700 1 -- 172.21.15.89:0/714849984 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f145806c0a0 msgr2=0x7f14580caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:36.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f145559a700 1 --2- 172.21.15.89:0/714849984 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f145806c0a0 0x7f14580caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f145559a700 1 -- 172.21.15.89:0/714849984 >> v2:172.21.15.111:3300/0 conn(0x7f145815df50 msgr2=0x7f145815e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:36.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f145559a700 1 --2- 172.21.15.89:0/714849984 >> v2:172.21.15.111:3300/0 conn(0x7f145815df50 0x7f145815e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f145559a700 1 -- 172.21.15.89:0/714849984 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145815ad10 con 0x7f145806bcd0 2023-10-10T20:35:36.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f145559a700 1 -- 172.21.15.89:0/714849984 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f14500017f0 con 0x7f145806bcd0 2023-10-10T20:35:36.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f145559a700 1 -- 172.21.15.89:0/714849984 >> v1:172.21.15.89:6789/0 conn(0x7f145806bcd0 legacy=0x7f145815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:36.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f145559a700 1 --2- 172.21.15.89:0/714849984 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f145806c0a0 0x7f1440003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.779+0000 7f145559a700 1 --2- 172.21.15.89:0/714849984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1440003fe0 0x7f1440004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.780+0000 7f145559a700 1 --2- 172.21.15.89:0/714849984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1440004d50 0x7f14400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.780+0000 7f145559a700 1 -- 172.21.15.89:0/714849984 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145815dac0 con 0x7f1440003fe0 2023-10-10T20:35:36.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.780+0000 7f145559a700 1 -- 172.21.15.89:0/714849984 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145815aa30 con 0x7f1440004d50 2023-10-10T20:35:36.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.780+0000 7f145559a700 1 -- 172.21.15.89:0/714849984 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144000bb40 con 0x7f145806c0a0 2023-10-10T20:35:36.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.780+0000 7f1455d9b700 1 --2- 172.21.15.89:0/714849984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1440003fe0 0x7f1440004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.780+0000 7f145659c700 1 --2- 172.21.15.89:0/714849984 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f145806c0a0 0x7f1440003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.780+0000 7f1456d9d700 1 --2- 172.21.15.89:0/714849984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1440004d50 0x7f14400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.780+0000 7f1455d9b700 1 -- 172.21.15.89:0/714849984 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f145806c0a0 msgr2=0x7f1440003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:36.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.780+0000 7f1455d9b700 1 --2- 172.21.15.89:0/714849984 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f145806c0a0 0x7f1440003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.780+0000 7f1455d9b700 1 -- 172.21.15.89:0/714849984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1440004d50 msgr2=0x7f14400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:36.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.780+0000 7f1455d9b700 1 --2- 172.21.15.89:0/714849984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1440004d50 0x7f14400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.780+0000 7f1455d9b700 1 -- 172.21.15.89:0/714849984 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1440003570 con 0x7f1440003fe0 2023-10-10T20:35:36.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.780+0000 7f1455d9b700 1 --2- 172.21.15.89:0/714849984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1440003fe0 0x7f1440004810 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f144c006f00 tx=0x7f144c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9621c92a2fb74b54 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:36.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.780+0000 7f145559a700 1 -- 172.21.15.89:0/714849984 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c00e710 con 0x7f1440003fe0 2023-10-10T20:35:36.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.780+0000 7f145559a700 1 -- 172.21.15.89:0/714849984 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144c00e870 con 0x7f1440003fe0 2023-10-10T20:35:36.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.780+0000 7f145559a700 1 -- 172.21.15.89:0/714849984 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c00ec30 con 0x7f1440003fe0 2023-10-10T20:35:36.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.781+0000 7f145ce0d700 1 -- 172.21.15.89:0/714849984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1440003fe0 msgr2=0x7f1440004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:36.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.781+0000 7f145ce0d700 1 --2- 172.21.15.89:0/714849984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1440003fe0 0x7f1440004810 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f144c006f00 tx=0x7f144c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.781+0000 7f1456d9d700 1 -- 172.21.15.89:0/714849984 reap_dead start 2023-10-10T20:35:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.781+0000 7f145ce0d700 1 -- 172.21.15.89:0/714849984 shutdown_connections 2023-10-10T20:35:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.781+0000 7f145ce0d700 1 -- 172.21.15.89:0/714849984 >> 172.21.15.89:0/714849984 conn(0x7f14580ce600 msgr2=0x7f14580cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.781+0000 7f145ce0d700 1 -- 172.21.15.89:0/714849984 shutdown_connections 2023-10-10T20:35:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.781+0000 7f145ce0d700 1 -- 172.21.15.89:0/714849984 wait complete. 2023-10-10T20:35:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.782+0000 7f145ce0d700 1 Processor -- start 2023-10-10T20:35:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.782+0000 7f145ce0d700 1 -- start start 2023-10-10T20:35:36.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f145ce0d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1440003fe0 0x7f14581546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f145ce0d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14580c4440 0x7f14580c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f145ce0d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14580c7780 0x7f14580c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f145ce0d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14580c8130 con 0x7f14580c4440 2023-10-10T20:35:36.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f145ce0d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1458154c00 con 0x7f14580c7780 2023-10-10T20:35:36.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f145ce0d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14580c33a0 con 0x7f1440003fe0 2023-10-10T20:35:36.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f1455d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14580c4440 0x7f14580c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f1455d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14580c4440 0x7f14580c48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59664/0 (socket says 172.21.15.89:59664) 2023-10-10T20:35:36.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f1455d9b700 1 -- 172.21.15.89:0/3706189616 learned_addr learned my addr 172.21.15.89:0/3706189616 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:36.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f1456d9d700 1 --2- 172.21.15.89:0/3706189616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14580c7780 0x7f14580c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f145659c700 1 --2- 172.21.15.89:0/3706189616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1440003fe0 0x7f14581546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f1455d9b700 1 -- 172.21.15.89:0/3706189616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1440003fe0 msgr2=0x7f14581546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:36.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f1455d9b700 1 --2- 172.21.15.89:0/3706189616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1440003fe0 0x7f14581546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f1455d9b700 1 -- 172.21.15.89:0/3706189616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14580c7780 msgr2=0x7f14580c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:36.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f1455d9b700 1 --2- 172.21.15.89:0/3706189616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14580c7780 0x7f14580c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:36.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f1455d9b700 1 -- 172.21.15.89:0/3706189616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14580c3680 con 0x7f14580c4440 2023-10-10T20:35:36.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f1455d9b700 1 --2- 172.21.15.89:0/3706189616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14580c4440 0x7f14580c48b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f144000ba90 tx=0x7f144c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:36.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f14467fc700 1 -- 172.21.15.89:0/3706189616 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c023070 con 0x7f14580c4440 2023-10-10T20:35:36.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f14467fc700 1 -- 172.21.15.89:0/3706189616 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144c00e9e0 con 0x7f14580c4440 2023-10-10T20:35:36.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f14467fc700 1 -- 172.21.15.89:0/3706189616 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c01e030 con 0x7f14580c4440 2023-10-10T20:35:36.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f145ce0d700 1 -- 172.21.15.89:0/3706189616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14580c3960 con 0x7f14580c4440 2023-10-10T20:35:36.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.783+0000 7f145ce0d700 1 -- 172.21.15.89:0/3706189616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14580c3f20 con 0x7f14580c4440 2023-10-10T20:35:36.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.784+0000 7f145ce0d700 1 -- 172.21.15.89:0/3706189616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14580c4290 con 0x7f14580c4440 2023-10-10T20:35:36.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.785+0000 7f14467fc700 1 -- 172.21.15.89:0/3706189616 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f144c01e1b0 con 0x7f14580c4440 2023-10-10T20:35:36.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.785+0000 7f14467fc700 1 --2- 172.21.15.89:0/3706189616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1438066f30 0x7f14380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:36.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.785+0000 7f14467fc700 1 -- 172.21.15.89:0/3706189616 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f144c085d20 con 0x7f14580c4440 2023-10-10T20:35:36.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.785+0000 7f145659c700 1 --2- 172.21.15.89:0/3706189616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1438066f30 0x7f14380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:36.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.785+0000 7f145659c700 1 --2- 172.21.15.89:0/3706189616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1438066f30 0x7f14380693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1450001350 tx=0x7f1450006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:36.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:36.789+0000 7f14467fc700 1 -- 172.21.15.89:0/3706189616 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f144c052d50 con 0x7f14580c4440 2023-10-10T20:35:37.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.030+0000 7f145ce0d700 1 -- 172.21.15.89:0/3706189616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs get", "fs_name": "volume_0000000000680185", "format": "json"} v 0) v1 -- 0x7f145804c3c0 con 0x7f14580c4440 2023-10-10T20:35:37.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.031+0000 7f14467fc700 1 -- 172.21.15.89:0/3706189616 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs get", "fs_name": "volume_0000000000680185", "format": "json"}]=0 v20) v1 ==== 110+0+1844 (secure 0 0 0) 0x7f144c017020 con 0x7f14580c4440 2023-10-10T20:35:37.034 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:37.034 INFO:teuthology.orchestra.run.smithi089.stdout:{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:35:34.941436+0000","modified":"2023-10-10T20:35:36.889055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4896},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4896":{"gid":4896,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.89:6836/2135915759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2135915759},{"type":"v1","addr":"172.21.15.89:6836","nonce":2135915759}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000680185","balancer":"","standby_count_wanted":1},"id":3} 2023-10-10T20:35:37.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.033+0000 7f145ce0d700 1 -- 172.21.15.89:0/3706189616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1438066f30 msgr2=0x7f14380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:37.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.033+0000 7f145ce0d700 1 --2- 172.21.15.89:0/3706189616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1438066f30 0x7f14380693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1450001350 tx=0x7f1450006040 comp rx=0 tx=0).stop 2023-10-10T20:35:37.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.033+0000 7f145ce0d700 1 -- 172.21.15.89:0/3706189616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14580c4440 msgr2=0x7f14580c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:37.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.033+0000 7f145ce0d700 1 --2- 172.21.15.89:0/3706189616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14580c4440 0x7f14580c48b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f144000ba90 tx=0x7f144c01a000 comp rx=0 tx=0).stop 2023-10-10T20:35:37.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.033+0000 7f145ce0d700 1 -- 172.21.15.89:0/3706189616 shutdown_connections 2023-10-10T20:35:37.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.033+0000 7f145ce0d700 1 --2- 172.21.15.89:0/3706189616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1438066f30 0x7f14380693e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:37.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.033+0000 7f145ce0d700 1 --2- 172.21.15.89:0/3706189616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14580c7780 0x7f14580c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:37.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.033+0000 7f145ce0d700 1 --2- 172.21.15.89:0/3706189616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1440003fe0 0x7f14581546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:37.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.033+0000 7f145ce0d700 1 --2- 172.21.15.89:0/3706189616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14580c4440 0x7f14580c48b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:37.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.033+0000 7f145ce0d700 1 -- 172.21.15.89:0/3706189616 >> 172.21.15.89:0/3706189616 conn(0x7f14580ce600 msgr2=0x7f145815a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:37.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.033+0000 7f145ce0d700 1 -- 172.21.15.89:0/3706189616 shutdown_connections 2023-10-10T20:35:37.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.033+0000 7f145ce0d700 1 -- 172.21.15.89:0/3706189616 wait complete. 2023-10-10T20:35:37.047 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000680185 --yes-i-really-mean-it 2023-10-10T20:35:37.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.164+0000 7f63053eb700 1 Processor -- start 2023-10-10T20:35:37.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.164+0000 7f63053eb700 1 -- start start 2023-10-10T20:35:37.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.165+0000 7f63053eb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63000ae7e0 0x7f63000c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:37.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.165+0000 7f63053eb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f630006ebd0 con 0x7f630006f060 2023-10-10T20:35:37.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.165+0000 7f63053eb700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f630015aa70 con 0x7f63000cb1a0 2023-10-10T20:35:37.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.165+0000 7f63053eb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630015b2e0 con 0x7f63000ae7e0 2023-10-10T20:35:37.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.165+0000 7f62ff7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f630006f060 0x7f63000ae0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41832/0 (socket says 172.21.15.89:41832) 2023-10-10T20:35:37.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.165+0000 7f62ff7fe700 1 -- 172.21.15.89:0/3078050694 learned_addr learned my addr 172.21.15.89:0/3078050694 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:37.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.165+0000 7f62feffd700 1 -- 172.21.15.89:0/3078050694 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487336791 0 0) 0x7f630006ebd0 con 0x7f630006f060 2023-10-10T20:35:37.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.165+0000 7f62feffd700 1 -- 172.21.15.89:0/3078050694 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62e8003570 con 0x7f630006f060 2023-10-10T20:35:37.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.165+0000 7f6304bea700 1 --2- 172.21.15.89:0/3078050694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63000ae7e0 0x7f63000c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:37.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.165+0000 7f62feffd700 1 -- 172.21.15.89:0/3078050694 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3024918044 0 0) 0x7f62e8003570 con 0x7f630006f060 2023-10-10T20:35:37.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62feffd700 1 -- 172.21.15.89:0/3078050694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63000ae7e0 msgr2=0x7f63000c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:37.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62feffd700 1 --2- 172.21.15.89:0/3078050694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63000ae7e0 0x7f63000c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:37.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62feffd700 1 -- 172.21.15.89:0/3078050694 >> v1:172.21.15.111:6789/0 conn(0x7f63000cb1a0 legacy=0x7f630015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:37.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62feffd700 1 -- 172.21.15.89:0/3078050694 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630015ad20 con 0x7f630006f060 2023-10-10T20:35:37.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62feffd700 1 -- 172.21.15.89:0/3078050694 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f62f4001470 con 0x7f630006f060 2023-10-10T20:35:37.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62feffd700 1 -- 172.21.15.89:0/3078050694 >> v1:172.21.15.89:6789/0 conn(0x7f630006f060 legacy=0x7f63000ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:37.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62feffd700 1 --2- 172.21.15.89:0/3078050694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62e8003a90 0x7f62e8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:37.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62feffd700 1 --2- 172.21.15.89:0/3078050694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63000ae7e0 0x7f62e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:37.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62feffd700 1 --2- 172.21.15.89:0/3078050694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62e8004da0 0x7f62e80051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:37.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62feffd700 1 -- 172.21.15.89:0/3078050694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630015b2e0 con 0x7f62e8003a90 2023-10-10T20:35:37.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62feffd700 1 -- 172.21.15.89:0/3078050694 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e800bc20 con 0x7f62e8004da0 2023-10-10T20:35:37.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62feffd700 1 -- 172.21.15.89:0/3078050694 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e800dff0 con 0x7f63000ae7e0 2023-10-10T20:35:37.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62fffff700 1 --2- 172.21.15.89:0/3078050694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62e8003a90 0x7f62e8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:37.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62ff7fe700 1 --2- 172.21.15.89:0/3078050694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63000ae7e0 0x7f62e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:37.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f6304bea700 1 --2- 172.21.15.89:0/3078050694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62e8004da0 0x7f62e80051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:37.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62fffff700 1 -- 172.21.15.89:0/3078050694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63000ae7e0 msgr2=0x7f62e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:37.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62fffff700 1 --2- 172.21.15.89:0/3078050694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63000ae7e0 0x7f62e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:37.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62fffff700 1 -- 172.21.15.89:0/3078050694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62e8004da0 msgr2=0x7f62e80051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:37.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62fffff700 1 --2- 172.21.15.89:0/3078050694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62e8004da0 0x7f62e80051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:37.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62fffff700 1 -- 172.21.15.89:0/3078050694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e8003570 con 0x7f62e8003a90 2023-10-10T20:35:37.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62fffff700 1 --2- 172.21.15.89:0/3078050694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62e8003a90 0x7f62e8003ea0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f62f00018b0 tx=0x7f62f0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11630c9ab648d4b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:37.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62feffd700 1 -- 172.21.15.89:0/3078050694 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f0012420 con 0x7f62e8003a90 2023-10-10T20:35:37.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62feffd700 1 -- 172.21.15.89:0/3078050694 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f0012580 con 0x7f62e8003a90 2023-10-10T20:35:37.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.166+0000 7f62feffd700 1 -- 172.21.15.89:0/3078050694 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f0012920 con 0x7f62e8003a90 2023-10-10T20:35:37.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.167+0000 7f63053eb700 1 -- 172.21.15.89:0/3078050694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62e8003a90 msgr2=0x7f62e8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:37.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.167+0000 7f63053eb700 1 --2- 172.21.15.89:0/3078050694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62e8003a90 0x7f62e8003ea0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f62f00018b0 tx=0x7f62f0007d30 comp rx=0 tx=0).stop 2023-10-10T20:35:37.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.167+0000 7f6304bea700 1 -- 172.21.15.89:0/3078050694 reap_dead start 2023-10-10T20:35:37.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.167+0000 7f63053eb700 1 -- 172.21.15.89:0/3078050694 shutdown_connections 2023-10-10T20:35:37.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.167+0000 7f63053eb700 1 -- 172.21.15.89:0/3078050694 >> 172.21.15.89:0/3078050694 conn(0x7f63000cc1e0 msgr2=0x7f63000cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:37.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.167+0000 7f63053eb700 1 -- 172.21.15.89:0/3078050694 shutdown_connections 2023-10-10T20:35:37.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.167+0000 7f63053eb700 1 -- 172.21.15.89:0/3078050694 wait complete. 2023-10-10T20:35:37.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.168+0000 7f63053eb700 1 Processor -- start 2023-10-10T20:35:37.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.168+0000 7f63053eb700 1 -- start start 2023-10-10T20:35:37.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.169+0000 7f63053eb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f62e8003a90 0x7f63000c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:37.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.169+0000 7f63053eb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63000c6ff0 0x7f63000aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:37.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.169+0000 7f63053eb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63000c7460 0x7f63000af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:37.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.169+0000 7f63053eb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63000b0020 con 0x7f63000c6ff0 2023-10-10T20:35:37.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.169+0000 7f63053eb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63000c5410 con 0x7f63000c7460 2023-10-10T20:35:37.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.169+0000 7f63053eb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63000c5de0 con 0x7f62e8003a90 2023-10-10T20:35:37.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.169+0000 7f62ff7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63000c6ff0 0x7f63000aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:37.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.169+0000 7f62ff7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63000c6ff0 0x7f63000aeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59690/0 (socket says 172.21.15.89:59690) 2023-10-10T20:35:37.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.169+0000 7f62ff7fe700 1 -- 172.21.15.89:0/130689603 learned_addr learned my addr 172.21.15.89:0/130689603 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:37.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.169+0000 7f6304bea700 1 --2- 172.21.15.89:0/130689603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63000c7460 0x7f63000af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:37.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.169+0000 7f62fffff700 1 --2- 172.21.15.89:0/130689603 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f62e8003a90 0x7f63000c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:37.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.169+0000 7f62ff7fe700 1 -- 172.21.15.89:0/130689603 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f62e8003a90 msgr2=0x7f63000c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:37.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.169+0000 7f62ff7fe700 1 --2- 172.21.15.89:0/130689603 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f62e8003a90 0x7f63000c58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:37.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.169+0000 7f62ff7fe700 1 -- 172.21.15.89:0/130689603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63000c7460 msgr2=0x7f63000af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:37.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.169+0000 7f62ff7fe700 1 --2- 172.21.15.89:0/130689603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63000c7460 0x7f63000af530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:37.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.169+0000 7f62ff7fe700 1 -- 172.21.15.89:0/130689603 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63000afa70 con 0x7f63000c6ff0 2023-10-10T20:35:37.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.170+0000 7f62ff7fe700 1 --2- 172.21.15.89:0/130689603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63000c6ff0 0x7f63000aeff0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f62f40073c0 tx=0x7f62f4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:37.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.170+0000 7f62e7fff700 1 -- 172.21.15.89:0/130689603 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f400f8d0 con 0x7f63000c6ff0 2023-10-10T20:35:37.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.170+0000 7f62e7fff700 1 -- 172.21.15.89:0/130689603 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f400fa30 con 0x7f63000c6ff0 2023-10-10T20:35:37.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.170+0000 7f63053eb700 1 -- 172.21.15.89:0/130689603 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63000afd40 con 0x7f63000c6ff0 2023-10-10T20:35:37.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.170+0000 7f62e7fff700 1 -- 172.21.15.89:0/130689603 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f400fdf0 con 0x7f63000c6ff0 2023-10-10T20:35:37.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.170+0000 7f63053eb700 1 -- 172.21.15.89:0/130689603 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63000c50a0 con 0x7f63000c6ff0 2023-10-10T20:35:37.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.171+0000 7f63053eb700 1 -- 172.21.15.89:0/130689603 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630006c450 con 0x7f63000c6ff0 2023-10-10T20:35:37.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.171+0000 7f62e7fff700 1 -- 172.21.15.89:0/130689603 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f62f40135c0 con 0x7f63000c6ff0 2023-10-10T20:35:37.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.171+0000 7f62e7fff700 1 --2- 172.21.15.89:0/130689603 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f62ec066f30 0x7f62ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:37.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.171+0000 7f62fffff700 1 --2- 172.21.15.89:0/130689603 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f62ec066f30 0x7f62ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:37.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.172+0000 7f62e7fff700 1 -- 172.21.15.89:0/130689603 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f62f4084680 con 0x7f63000c6ff0 2023-10-10T20:35:37.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.172+0000 7f62fffff700 1 --2- 172.21.15.89:0/130689603 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f62ec066f30 0x7f62ec0693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f62f00016c0 tx=0x7f62f0015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:37.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.176+0000 7f62e7fff700 1 -- 172.21.15.89:0/130689603 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62f4051570 con 0x7f63000c6ff0 2023-10-10T20:35:37.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:37.384+0000 7f63053eb700 1 -- 172.21.15.89:0/130689603 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000000680185", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f63000c5550 con 0x7f62ec066f30 2023-10-10T20:35:37.388 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-10-10T20:35:37.387+0000 7fd0184ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:35:37.413 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:35:40.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.404+0000 7f62e7fff700 1 -- 172.21.15.89:0/130689603 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+110 (secure 0 0 0) 0x7f63000c5550 con 0x7f62ec066f30 2023-10-10T20:35:40.407 INFO:teuthology.orchestra.run.smithi089.stdout:metadata pool: cephfs.volume_0000000000680185.meta data pool: ['cephfs.volume_0000000000680185.data'] removed 2023-10-10T20:35:40.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.407+0000 7f63053eb700 1 -- 172.21.15.89:0/130689603 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f62ec066f30 msgr2=0x7f62ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:40.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.407+0000 7f63053eb700 1 --2- 172.21.15.89:0/130689603 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f62ec066f30 0x7f62ec0693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f62f00016c0 tx=0x7f62f0015000 comp rx=0 tx=0).stop 2023-10-10T20:35:40.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.407+0000 7f63053eb700 1 -- 172.21.15.89:0/130689603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63000c6ff0 msgr2=0x7f63000aeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:40.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.407+0000 7f63053eb700 1 --2- 172.21.15.89:0/130689603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63000c6ff0 0x7f63000aeff0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f62f40073c0 tx=0x7f62f4001e50 comp rx=0 tx=0).stop 2023-10-10T20:35:40.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.407+0000 7f63053eb700 1 -- 172.21.15.89:0/130689603 shutdown_connections 2023-10-10T20:35:40.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.407+0000 7f63053eb700 1 --2- 172.21.15.89:0/130689603 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f62ec066f30 0x7f62ec0693e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:40.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.407+0000 7f63053eb700 1 --2- 172.21.15.89:0/130689603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63000c7460 0x7f63000af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:40.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.407+0000 7f63053eb700 1 --2- 172.21.15.89:0/130689603 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f62e8003a90 0x7f63000c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:40.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.407+0000 7f63053eb700 1 --2- 172.21.15.89:0/130689603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63000c6ff0 0x7f63000aeff0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:40.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.407+0000 7f63053eb700 1 -- 172.21.15.89:0/130689603 >> 172.21.15.89:0/130689603 conn(0x7f63000cc1e0 msgr2=0x7f6300152750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:40.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.407+0000 7f63053eb700 1 -- 172.21.15.89:0/130689603 shutdown_connections 2023-10-10T20:35:40.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.407+0000 7f63053eb700 1 -- 172.21.15.89:0/130689603 wait complete. 2023-10-10T20:35:40.424 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:35:40.425 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:35:40.472 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:35:40.520 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:35:40.520 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:35:40.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:35:40.521 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:35:40.563 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:35:40.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:40.564 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:35:40.619 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:35:40.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:35:40.620 DEBUG:teuthology.orchestra.run.smithi111:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:35:40.659 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:35:40.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:35:40.659 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:35:40.715 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:35:40.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.830+0000 7f9aaf805700 1 Processor -- start 2023-10-10T20:35:40.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.830+0000 7f9aaf805700 1 -- start start 2023-10-10T20:35:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.830+0000 7f9aaf805700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9aa80aca80 0x7f9aa806aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.830+0000 7f9aaf805700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9aa8069d60 0x7f9aa80c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.830+0000 7f9aaf805700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aa806bdb0 con 0x7f9aa80c81f0 2023-10-10T20:35:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.830+0000 7f9aaf805700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aa80cb370 con 0x7f9aa80aca80 2023-10-10T20:35:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.830+0000 7f9aaf805700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aa8159f60 con 0x7f9aa8069d60 2023-10-10T20:35:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.831+0000 7f9aad5a1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9aa80aca80 0x7f9aa806aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:40.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.831+0000 7f9aadda2700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9aa80c81f0 0x7f9aa8068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45432/0 (socket says 172.21.15.89:45432) 2023-10-10T20:35:40.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.831+0000 7f9aadda2700 1 -- 172.21.15.89:0/2172095636 learned_addr learned my addr 172.21.15.89:0/2172095636 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:40.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.831+0000 7f9aacda0700 1 --2- 172.21.15.89:0/2172095636 >> v2:172.21.15.111:3300/0 conn(0x7f9aa8069d60 0x7f9aa80c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:40.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.831+0000 7f9aad5a1700 1 -- 172.21.15.89:0/2172095636 >> v2:172.21.15.111:3300/0 conn(0x7f9aa8069d60 msgr2=0x7f9aa80c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:40.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.831+0000 7f9aad5a1700 1 --2- 172.21.15.89:0/2172095636 >> v2:172.21.15.111:3300/0 conn(0x7f9aa8069d60 0x7f9aa80c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:40.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.831+0000 7f9aad5a1700 1 -- 172.21.15.89:0/2172095636 >> v1:172.21.15.89:6789/0 conn(0x7f9aa80c81f0 legacy=0x7f9aa8068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.831+0000 7f9aad5a1700 1 -- 172.21.15.89:0/2172095636 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa80c8610 con 0x7f9aa80aca80 2023-10-10T20:35:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.831+0000 7f9a9ffff700 1 -- 172.21.15.89:0/2172095636 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2304495734 0 0) 0x7f9aa806bdb0 con 0x7f9aa80c81f0 2023-10-10T20:35:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.831+0000 7f9aad5a1700 1 --2- 172.21.15.89:0/2172095636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9aa80aca80 0x7f9aa806aba0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9aa806e540 tx=0x7f9aa4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f69ff9952943ae server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.831+0000 7f9a9ffff700 1 -- 172.21.15.89:0/2172095636 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa400e070 con 0x7f9aa80aca80 2023-10-10T20:35:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.831+0000 7f9a9ffff700 1 -- 172.21.15.89:0/2172095636 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa400bc30 con 0x7f9aa80aca80 2023-10-10T20:35:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.831+0000 7f9a9ffff700 1 -- 172.21.15.89:0/2172095636 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4012430 con 0x7f9aa80aca80 2023-10-10T20:35:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.832+0000 7f9aaf805700 1 -- 172.21.15.89:0/2172095636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9aa80aca80 msgr2=0x7f9aa806aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.832+0000 7f9aaf805700 1 --2- 172.21.15.89:0/2172095636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9aa80aca80 0x7f9aa806aba0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9aa806e540 tx=0x7f9aa4009780 comp rx=0 tx=0).stop 2023-10-10T20:35:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.832+0000 7f9aaf805700 1 -- 172.21.15.89:0/2172095636 shutdown_connections 2023-10-10T20:35:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.832+0000 7f9aaf805700 1 --2- 172.21.15.89:0/2172095636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9aa80aca80 0x7f9aa806aba0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.832+0000 7f9aaf805700 1 --2- 172.21.15.89:0/2172095636 >> v2:172.21.15.111:3300/0 conn(0x7f9aa8069d60 0x7f9aa80c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.832+0000 7f9aaf805700 1 -- 172.21.15.89:0/2172095636 >> 172.21.15.89:0/2172095636 conn(0x7f9aa80cd670 msgr2=0x7f9aa80cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.832+0000 7f9aaf805700 1 -- 172.21.15.89:0/2172095636 shutdown_connections 2023-10-10T20:35:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.832+0000 7f9aaf805700 1 -- 172.21.15.89:0/2172095636 wait complete. 2023-10-10T20:35:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.833+0000 7f9aaf805700 1 Processor -- start 2023-10-10T20:35:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.833+0000 7f9aaf805700 1 -- start start 2023-10-10T20:35:40.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.834+0000 7f9aaf805700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9aa8069d60 0x7f9aa8152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:40.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.834+0000 7f9aaf805700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9aa80aca80 0x7f9aa816b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.834+0000 7f9aaf805700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9aa80c81f0 0x7f9aa80aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.834+0000 7f9aaf805700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa8152b50 con 0x7f9aa80c81f0 2023-10-10T20:35:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.834+0000 7f9aaf805700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa815adb0 con 0x7f9aa8069d60 2023-10-10T20:35:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.834+0000 7f9aaf805700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa815a1a0 con 0x7f9aa80aca80 2023-10-10T20:35:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.834+0000 7f9aadda2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9aa80c81f0 0x7f9aa80aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.834+0000 7f9aadda2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9aa80c81f0 0x7f9aa80aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58080/0 (socket says 172.21.15.89:58080) 2023-10-10T20:35:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.834+0000 7f9aadda2700 1 -- 172.21.15.89:0/3364134217 learned_addr learned my addr 172.21.15.89:0/3364134217 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.834+0000 7f9aad5a1700 1 --2- 172.21.15.89:0/3364134217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9aa8069d60 0x7f9aa8152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.834+0000 7f9aacda0700 1 --2- 172.21.15.89:0/3364134217 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9aa80aca80 0x7f9aa816b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.834+0000 7f9aadda2700 1 -- 172.21.15.89:0/3364134217 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9aa80aca80 msgr2=0x7f9aa816b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.834+0000 7f9aadda2700 1 --2- 172.21.15.89:0/3364134217 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9aa80aca80 0x7f9aa816b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.834+0000 7f9aadda2700 1 -- 172.21.15.89:0/3364134217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9aa8069d60 msgr2=0x7f9aa8152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.834+0000 7f9aadda2700 1 --2- 172.21.15.89:0/3364134217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9aa8069d60 0x7f9aa8152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.834+0000 7f9aadda2700 1 -- 172.21.15.89:0/3364134217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa4009a20 con 0x7f9aa80c81f0 2023-10-10T20:35:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.834+0000 7f9aadda2700 1 --2- 172.21.15.89:0/3364134217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9aa80c81f0 0x7f9aa80aed60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000c530 tx=0x7f9aa0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:40.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.834+0000 7f9a9d7fa700 1 -- 172.21.15.89:0/3364134217 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0005b30 con 0x7f9aa80c81f0 2023-10-10T20:35:40.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.834+0000 7f9a9d7fa700 1 -- 172.21.15.89:0/3364134217 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa0005c90 con 0x7f9aa80c81f0 2023-10-10T20:35:40.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.835+0000 7f9a9d7fa700 1 -- 172.21.15.89:0/3364134217 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0007320 con 0x7f9aa80c81f0 2023-10-10T20:35:40.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.835+0000 7f9aaf805700 1 -- 172.21.15.89:0/3364134217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa815a480 con 0x7f9aa80c81f0 2023-10-10T20:35:40.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.835+0000 7f9aaf805700 1 -- 172.21.15.89:0/3364134217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa815aa40 con 0x7f9aa80c81f0 2023-10-10T20:35:40.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.835+0000 7f9aaf805700 1 -- 172.21.15.89:0/3364134217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa8063730 con 0x7f9aa80c81f0 2023-10-10T20:35:40.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.836+0000 7f9a9d7fa700 1 -- 172.21.15.89:0/3364134217 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9aa001b3b0 con 0x7f9aa80c81f0 2023-10-10T20:35:40.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.836+0000 7f9a9d7fa700 1 --2- 172.21.15.89:0/3364134217 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9a8c066f30 0x7f9a8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:40.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.836+0000 7f9a9d7fa700 1 -- 172.21.15.89:0/3364134217 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f9aa0083900 con 0x7f9aa80c81f0 2023-10-10T20:35:40.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.836+0000 7f9aad5a1700 1 --2- 172.21.15.89:0/3364134217 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9a8c066f30 0x7f9a8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:40.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.837+0000 7f9aad5a1700 1 --2- 172.21.15.89:0/3364134217 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9a8c066f30 0x7f9a8c0693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9aa806e540 tx=0x7f9aa4019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:40.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:40.840+0000 7f9a9d7fa700 1 -- 172.21.15.89:0/3364134217 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9aa0050b20 con 0x7f9aa80c81f0 2023-10-10T20:35:41.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.067+0000 7f9aaf805700 1 -- 172.21.15.89:0/3364134217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9aa815af20 con 0x7f9aa80c81f0 2023-10-10T20:35:41.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.067+0000 7f9a9d7fa700 1 -- 172.21.15.89:0/3364134217 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f9aa0017090 con 0x7f9aa80c81f0 2023-10-10T20:35:41.070 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:41.070 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1977066906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1977066906},{"type":"v1","addr":"172.21.15.111:6833","nonce":1977066906}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2904313060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2904313060},{"type":"v1","addr":"172.21.15.111:6835","nonce":2904313060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/4152616589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":4152616589},{"type":"v1","addr":"172.21.15.89:6836","nonce":4152616589}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:35:17.900533+0000","modified":"2023-10-10T20:35:21.533574+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4899":{"gid":4899,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1330347277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1330347277},{"type":"v1","addr":"172.21.15.89:6837","nonce":1330347277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-10T20:35:41.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.069+0000 7f9aaf805700 1 -- 172.21.15.89:0/3364134217 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9a8c066f30 msgr2=0x7f9a8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.069+0000 7f9aaf805700 1 --2- 172.21.15.89:0/3364134217 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9a8c066f30 0x7f9a8c0693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9aa806e540 tx=0x7f9aa4019000 comp rx=0 tx=0).stop 2023-10-10T20:35:41.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.069+0000 7f9aaf805700 1 -- 172.21.15.89:0/3364134217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9aa80c81f0 msgr2=0x7f9aa80aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.069+0000 7f9aaf805700 1 --2- 172.21.15.89:0/3364134217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9aa80c81f0 0x7f9aa80aed60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000c530 tx=0x7f9aa0007f90 comp rx=0 tx=0).stop 2023-10-10T20:35:41.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.069+0000 7f9aaf805700 1 -- 172.21.15.89:0/3364134217 shutdown_connections 2023-10-10T20:35:41.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.069+0000 7f9aaf805700 1 --2- 172.21.15.89:0/3364134217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9aa80c81f0 0x7f9aa80aed60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.069+0000 7f9aaf805700 1 --2- 172.21.15.89:0/3364134217 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9a8c066f30 0x7f9a8c0693e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.069+0000 7f9aaf805700 1 --2- 172.21.15.89:0/3364134217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9aa8069d60 0x7f9aa8152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.069+0000 7f9aaf805700 1 --2- 172.21.15.89:0/3364134217 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9aa80aca80 0x7f9aa816b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.069+0000 7f9aaf805700 1 -- 172.21.15.89:0/3364134217 >> 172.21.15.89:0/3364134217 conn(0x7f9aa80cd670 msgr2=0x7f9aa80ab6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:41.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.069+0000 7f9aaf805700 1 -- 172.21.15.89:0/3364134217 shutdown_connections 2023-10-10T20:35:41.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.069+0000 7f9aaf805700 1 -- 172.21.15.89:0/3364134217 wait complete. 2023-10-10T20:35:41.075 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 22 2023-10-10T20:35:41.084 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:35:41.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.200+0000 7ffae4f10700 1 Processor -- start 2023-10-10T20:35:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.200+0000 7ffae4f10700 1 -- start start 2023-10-10T20:35:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.201+0000 7ffae4f10700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffae0159890 0x7ffae015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.201+0000 7ffae4f10700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae015a160 con 0x7ffae0069560 2023-10-10T20:35:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.201+0000 7ffae4f10700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae00d13a0 con 0x7ffae0159c60 2023-10-10T20:35:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.201+0000 7ffae4f10700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae015e140 con 0x7ffae0159890 2023-10-10T20:35:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.201+0000 7ffaded9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ffae0069560 0x7ffae00ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45442/0 (socket says 172.21.15.89:45442) 2023-10-10T20:35:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.201+0000 7ffaded9d700 1 -- 172.21.15.89:0/2759378718 learned_addr learned my addr 172.21.15.89:0/2759378718 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.201+0000 7ffade59c700 1 --2- 172.21.15.89:0/2759378718 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffae0159890 0x7ffae015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.201+0000 7ffaddd9b700 1 -- 172.21.15.89:0/2759378718 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790724092 0 0) 0x7ffae015a160 con 0x7ffae0069560 2023-10-10T20:35:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.201+0000 7ffaddd9b700 1 -- 172.21.15.89:0/2759378718 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffacc003570 con 0x7ffae0069560 2023-10-10T20:35:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.202+0000 7ffaddd9b700 1 -- 172.21.15.89:0/2759378718 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3113672119 0 0) 0x7ffae00d13a0 con 0x7ffae0159c60 2023-10-10T20:35:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.202+0000 7ffaddd9b700 1 -- 172.21.15.89:0/2759378718 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffae015a160 con 0x7ffae0159c60 2023-10-10T20:35:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.202+0000 7ffade59c700 1 -- 172.21.15.89:0/2759378718 >> v1:172.21.15.111:6789/0 conn(0x7ffae0159c60 legacy=0x7ffae015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.202+0000 7ffade59c700 1 -- 172.21.15.89:0/2759378718 >> v1:172.21.15.89:6789/0 conn(0x7ffae0069560 legacy=0x7ffae00ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.202+0000 7ffaddd9b700 1 -- 172.21.15.89:0/2759378718 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1182774684 0 0) 0x7ffacc003570 con 0x7ffae0069560 2023-10-10T20:35:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.202+0000 7ffade59c700 1 -- 172.21.15.89:0/2759378718 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae015a3d0 con 0x7ffae0159890 2023-10-10T20:35:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.202+0000 7ffade59c700 1 --2- 172.21.15.89:0/2759378718 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffae0159890 0x7ffae015abd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009910 tx=0x7ffad00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b36548a2af7ad1a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.202+0000 7ffaddd9b700 1 -- 172.21.15.89:0/2759378718 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad80017f0 con 0x7ffae0159890 2023-10-10T20:35:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.202+0000 7ffaddd9b700 1 -- 172.21.15.89:0/2759378718 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad0011420 con 0x7ffae0159890 2023-10-10T20:35:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.202+0000 7ffaddd9b700 1 -- 172.21.15.89:0/2759378718 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad00117c0 con 0x7ffae0159890 2023-10-10T20:35:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.202+0000 7ffae4f10700 1 -- 172.21.15.89:0/2759378718 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffae0159890 msgr2=0x7ffae015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.202+0000 7ffae4f10700 1 --2- 172.21.15.89:0/2759378718 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffae0159890 0x7ffae015abd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009910 tx=0x7ffad00091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.202+0000 7ffae4f10700 1 -- 172.21.15.89:0/2759378718 shutdown_connections 2023-10-10T20:35:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.202+0000 7ffae4f10700 1 --2- 172.21.15.89:0/2759378718 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffae0159890 0x7ffae015abd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.202+0000 7ffae4f10700 1 -- 172.21.15.89:0/2759378718 >> 172.21.15.89:0/2759378718 conn(0x7ffae0068e20 msgr2=0x7ffae00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.203+0000 7ffae4f10700 1 -- 172.21.15.89:0/2759378718 shutdown_connections 2023-10-10T20:35:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.203+0000 7ffae4f10700 1 -- 172.21.15.89:0/2759378718 wait complete. 2023-10-10T20:35:41.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.204+0000 7ffae4f10700 1 Processor -- start 2023-10-10T20:35:41.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.204+0000 7ffae4f10700 1 -- start start 2023-10-10T20:35:41.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.204+0000 7ffae4f10700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffae0069560 0x7ffae00cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:41.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.204+0000 7ffae4f10700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffae0159890 0x7ffae00b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:41.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.204+0000 7ffae4f10700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffae0159c60 0x7ffae00cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:41.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.204+0000 7ffae4f10700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae00c81d0 con 0x7ffae0159c60 2023-10-10T20:35:41.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.204+0000 7ffae4f10700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae00ce300 con 0x7ffae0159890 2023-10-10T20:35:41.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.204+0000 7ffae4f10700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae00c78a0 con 0x7ffae0069560 2023-10-10T20:35:41.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.205+0000 7ffaded9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffae0159c60 0x7ffae00cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:41.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.205+0000 7ffaded9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffae0159c60 0x7ffae00cf240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58092/0 (socket says 172.21.15.89:58092) 2023-10-10T20:35:41.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.205+0000 7ffaded9d700 1 -- 172.21.15.89:0/1538574354 learned_addr learned my addr 172.21.15.89:0/1538574354 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:41.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.205+0000 7ffade59c700 1 --2- 172.21.15.89:0/1538574354 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffae0069560 0x7ffae00cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:41.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.205+0000 7ffad5d9b700 1 --2- 172.21.15.89:0/1538574354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffae0159890 0x7ffae00b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:41.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.205+0000 7ffaded9d700 1 -- 172.21.15.89:0/1538574354 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffae0069560 msgr2=0x7ffae00cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.205+0000 7ffaded9d700 1 --2- 172.21.15.89:0/1538574354 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffae0069560 0x7ffae00cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.205+0000 7ffaded9d700 1 -- 172.21.15.89:0/1538574354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffae0159890 msgr2=0x7ffae00b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.205+0000 7ffaded9d700 1 --2- 172.21.15.89:0/1538574354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffae0159890 0x7ffae00b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.205+0000 7ffaded9d700 1 -- 172.21.15.89:0/1538574354 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad0009420 con 0x7ffae0159c60 2023-10-10T20:35:41.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.205+0000 7ffaded9d700 1 --2- 172.21.15.89:0/1538574354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffae0159c60 0x7ffae00cf240 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ffad800ccc0 tx=0x7ffad8007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:41.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.205+0000 7ffad6ffd700 1 -- 172.21.15.89:0/1538574354 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad8004dc0 con 0x7ffae0159c60 2023-10-10T20:35:41.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.205+0000 7ffad6ffd700 1 -- 172.21.15.89:0/1538574354 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad800a630 con 0x7ffae0159c60 2023-10-10T20:35:41.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.205+0000 7ffae4f10700 1 -- 172.21.15.89:0/1538574354 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae00c7b80 con 0x7ffae0159c60 2023-10-10T20:35:41.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.205+0000 7ffad6ffd700 1 -- 172.21.15.89:0/1538574354 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad800a9f0 con 0x7ffae0159c60 2023-10-10T20:35:41.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.205+0000 7ffae4f10700 1 -- 172.21.15.89:0/1538574354 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae00ce090 con 0x7ffae0159c60 2023-10-10T20:35:41.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.206+0000 7ffae4f10700 1 -- 172.21.15.89:0/1538574354 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae00c8380 con 0x7ffae0159c60 2023-10-10T20:35:41.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.207+0000 7ffad6ffd700 1 -- 172.21.15.89:0/1538574354 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffad8007150 con 0x7ffae0159c60 2023-10-10T20:35:41.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.207+0000 7ffad6ffd700 1 --2- 172.21.15.89:0/1538574354 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ffac0066f30 0x7ffac00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:41.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.207+0000 7ffad6ffd700 1 -- 172.21.15.89:0/1538574354 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5791+0+0 (secure 0 0 0) 0x7ffad8083ef0 con 0x7ffae0159c60 2023-10-10T20:35:41.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.207+0000 7ffade59c700 1 --2- 172.21.15.89:0/1538574354 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ffac0066f30 0x7ffac00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:41.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.207+0000 7ffade59c700 1 --2- 172.21.15.89:0/1538574354 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ffac0066f30 0x7ffac00693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009860 tx=0x7ffad0016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:41.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.211+0000 7ffad6ffd700 1 -- 172.21.15.89:0/1538574354 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffae00c8380 con 0x7ffae0159c60 2023-10-10T20:35:41.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.396+0000 7ffae4f10700 1 -- 172.21.15.89:0/1538574354 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffae00b4e40 con 0x7ffae0159c60 2023-10-10T20:35:41.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.397+0000 7ffad6ffd700 1 -- 172.21.15.89:0/1538574354 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+15479 (secure 0 0 0) 0x7ffad8018090 con 0x7ffae0159c60 2023-10-10T20:35:41.399 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:41.399 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":38,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:35:40.398559+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:35:15.667057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-10T20:35:16.860131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6836/2135915759":"2023-10-11T20:35:37.387694+0000","172.21.15.89:6834/2135915759":"2023-10-11T20:35:37.387694+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:35:41.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.399+0000 7ffae4f10700 1 -- 172.21.15.89:0/1538574354 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ffac0066f30 msgr2=0x7ffac00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.399+0000 7ffae4f10700 1 --2- 172.21.15.89:0/1538574354 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ffac0066f30 0x7ffac00693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009860 tx=0x7ffad0016000 comp rx=0 tx=0).stop 2023-10-10T20:35:41.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.399+0000 7ffae4f10700 1 -- 172.21.15.89:0/1538574354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffae0159c60 msgr2=0x7ffae00cf240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.399+0000 7ffae4f10700 1 --2- 172.21.15.89:0/1538574354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffae0159c60 0x7ffae00cf240 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ffad800ccc0 tx=0x7ffad8007d20 comp rx=0 tx=0).stop 2023-10-10T20:35:41.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.399+0000 7ffae4f10700 1 -- 172.21.15.89:0/1538574354 shutdown_connections 2023-10-10T20:35:41.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.399+0000 7ffae4f10700 1 --2- 172.21.15.89:0/1538574354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffae0159c60 0x7ffae00cf240 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.399+0000 7ffae4f10700 1 --2- 172.21.15.89:0/1538574354 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffae0069560 0x7ffae00cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.399+0000 7ffae4f10700 1 --2- 172.21.15.89:0/1538574354 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ffac0066f30 0x7ffac00693e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.399+0000 7ffae4f10700 1 --2- 172.21.15.89:0/1538574354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffae0159890 0x7ffae00b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.399+0000 7ffae4f10700 1 -- 172.21.15.89:0/1538574354 >> 172.21.15.89:0/1538574354 conn(0x7ffae0068e20 msgr2=0x7ffae0152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:41.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.399+0000 7ffae4f10700 1 -- 172.21.15.89:0/1538574354 shutdown_connections 2023-10-10T20:35:41.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.399+0000 7ffae4f10700 1 -- 172.21.15.89:0/1538574354 wait complete. 2023-10-10T20:35:41.414 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:35:15.667057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:35:16.860131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:35:41.414 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:35:41.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.532+0000 7f21f359e700 1 Processor -- start 2023-10-10T20:35:41.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.532+0000 7f21f359e700 1 -- start start 2023-10-10T20:35:41.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.533+0000 7f21f359e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f21f40c97a0 0x7f21f4068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:41.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.533+0000 7f21f359e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f40c9310 con 0x7f21f40688d0 2023-10-10T20:35:41.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.533+0000 7f21f359e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f4157d90 con 0x7f21f41579c0 2023-10-10T20:35:41.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.533+0000 7f21f359e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f4068ca0 con 0x7f21f40c97a0 2023-10-10T20:35:41.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.533+0000 7f21f2d9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f21f40688d0 0x7f21f40b00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45452/0 (socket says 172.21.15.89:45452) 2023-10-10T20:35:41.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.533+0000 7f21f2d9d700 1 -- 172.21.15.89:0/38651386 learned_addr learned my addr 172.21.15.89:0/38651386 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:41.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.533+0000 7f21f1d9b700 1 --2- 172.21.15.89:0/38651386 >> v2:172.21.15.111:3300/0 conn(0x7f21f40c97a0 0x7f21f4068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:41.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.533+0000 7f21f159a700 1 -- 172.21.15.89:0/38651386 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1650378039 0 0) 0x7f21f40c9310 con 0x7f21f40688d0 2023-10-10T20:35:41.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.533+0000 7f21f159a700 1 -- 172.21.15.89:0/38651386 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21dc003570 con 0x7f21f40688d0 2023-10-10T20:35:41.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.533+0000 7f21f1d9b700 1 -- 172.21.15.89:0/38651386 >> v1:172.21.15.111:6789/0 conn(0x7f21f41579c0 legacy=0x7f21f406b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.533+0000 7f21f1d9b700 1 -- 172.21.15.89:0/38651386 >> v1:172.21.15.89:6789/0 conn(0x7f21f40688d0 legacy=0x7f21f40b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.533+0000 7f21f1d9b700 1 -- 172.21.15.89:0/38651386 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f40c8700 con 0x7f21f40c97a0 2023-10-10T20:35:41.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.533+0000 7f21f1d9b700 1 --2- 172.21.15.89:0/38651386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40c97a0 0x7f21f4068390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f21e80098d0 tx=0x7f21e8009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d286cdd9565135cf server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:41.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.533+0000 7f21f159a700 1 -- 172.21.15.89:0/38651386 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e8005c10 con 0x7f21f40c97a0 2023-10-10T20:35:41.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.534+0000 7f21f159a700 1 -- 172.21.15.89:0/38651386 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e8005d70 con 0x7f21f40c97a0 2023-10-10T20:35:41.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.534+0000 7f21f159a700 1 -- 172.21.15.89:0/38651386 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e8006280 con 0x7f21f40c97a0 2023-10-10T20:35:41.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.534+0000 7f21f359e700 1 -- 172.21.15.89:0/38651386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40c97a0 msgr2=0x7f21f4068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.534+0000 7f21f359e700 1 --2- 172.21.15.89:0/38651386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40c97a0 0x7f21f4068390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f21e80098d0 tx=0x7f21e8009120 comp rx=0 tx=0).stop 2023-10-10T20:35:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.534+0000 7f21f359e700 1 -- 172.21.15.89:0/38651386 shutdown_connections 2023-10-10T20:35:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.534+0000 7f21f359e700 1 --2- 172.21.15.89:0/38651386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40c97a0 0x7f21f4068390 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.534+0000 7f21f359e700 1 -- 172.21.15.89:0/38651386 >> 172.21.15.89:0/38651386 conn(0x7f21f40cddc0 msgr2=0x7f21f40c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.534+0000 7f21f359e700 1 -- 172.21.15.89:0/38651386 shutdown_connections 2023-10-10T20:35:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.534+0000 7f21f359e700 1 -- 172.21.15.89:0/38651386 wait complete. 2023-10-10T20:35:41.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.535+0000 7f21f359e700 1 Processor -- start 2023-10-10T20:35:41.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.535+0000 7f21f359e700 1 -- start start 2023-10-10T20:35:41.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.536+0000 7f21f359e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40688d0 0x7f21f4159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:41.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.536+0000 7f21f359e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21f40c97a0 0x7f21f4152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:41.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.536+0000 7f21f359e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f21f41579c0 0x7f21f40b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:41.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.536+0000 7f21f359e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f4168ae0 con 0x7f21f40c97a0 2023-10-10T20:35:41.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.536+0000 7f21f359e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f4152b80 con 0x7f21f40688d0 2023-10-10T20:35:41.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.536+0000 7f21f359e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f4168800 con 0x7f21f41579c0 2023-10-10T20:35:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.536+0000 7f21f1d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21f40c97a0 0x7f21f4152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.536+0000 7f21f1d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21f40c97a0 0x7f21f4152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58100/0 (socket says 172.21.15.89:58100) 2023-10-10T20:35:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.536+0000 7f21f1d9b700 1 -- 172.21.15.89:0/3088473758 learned_addr learned my addr 172.21.15.89:0/3088473758 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.536+0000 7f21f259c700 1 --2- 172.21.15.89:0/3088473758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40688d0 0x7f21f4159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.536+0000 7f21f1d9b700 1 -- 172.21.15.89:0/3088473758 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f21f41579c0 msgr2=0x7f21f40b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.536+0000 7f21f1d9b700 1 --2- 172.21.15.89:0/3088473758 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f21f41579c0 0x7f21f40b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.536+0000 7f21f1d9b700 1 -- 172.21.15.89:0/3088473758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40688d0 msgr2=0x7f21f4159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.536+0000 7f21f1d9b700 1 --2- 172.21.15.89:0/3088473758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40688d0 0x7f21f4159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.536+0000 7f21f1d9b700 1 -- 172.21.15.89:0/3088473758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e80093c0 con 0x7f21f40c97a0 2023-10-10T20:35:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.537+0000 7f21f1d9b700 1 --2- 172.21.15.89:0/3088473758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21f40c97a0 0x7f21f4152640 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f21f4157f20 tx=0x7f21e8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.537+0000 7f21e27fc700 1 -- 172.21.15.89:0/3088473758 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e801d880 con 0x7f21f40c97a0 2023-10-10T20:35:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.537+0000 7f21e27fc700 1 -- 172.21.15.89:0/3088473758 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e801da00 con 0x7f21f40c97a0 2023-10-10T20:35:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.537+0000 7f21f359e700 1 -- 172.21.15.89:0/3088473758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f4159050 con 0x7f21f40c97a0 2023-10-10T20:35:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.537+0000 7f21e27fc700 1 -- 172.21.15.89:0/3088473758 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e801ddc0 con 0x7f21f40c97a0 2023-10-10T20:35:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.537+0000 7f21f359e700 1 -- 172.21.15.89:0/3088473758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f4159610 con 0x7f21f40c97a0 2023-10-10T20:35:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.538+0000 7f21f359e700 1 -- 172.21.15.89:0/3088473758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f4159830 con 0x7f21f40c97a0 2023-10-10T20:35:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.538+0000 7f21e27fc700 1 -- 172.21.15.89:0/3088473758 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f21e801b690 con 0x7f21f40c97a0 2023-10-10T20:35:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.538+0000 7f21e27fc700 1 --2- 172.21.15.89:0/3088473758 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f21d4066f30 0x7f21d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:41.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.539+0000 7f21e27fc700 1 -- 172.21.15.89:0/3088473758 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f21e80862a0 con 0x7f21f40c97a0 2023-10-10T20:35:41.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.539+0000 7f21f259c700 1 --2- 172.21.15.89:0/3088473758 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f21d4066f30 0x7f21d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:41.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.539+0000 7f21f259c700 1 --2- 172.21.15.89:0/3088473758 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f21d4066f30 0x7f21d40693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f21f41552c0 tx=0x7f21e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.543+0000 7f21e27fc700 1 -- 172.21.15.89:0/3088473758 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f21e80534c0 con 0x7f21f40c97a0 2023-10-10T20:35:41.677 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:35:41.680 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:35:41.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.732+0000 7f21f359e700 1 -- 172.21.15.89:0/3088473758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21f40c89e0 con 0x7f21f40c97a0 2023-10-10T20:35:41.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.732+0000 7f21e27fc700 1 -- 172.21.15.89:0/3088473758 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7f21e8056ae0 con 0x7f21f40c97a0 2023-10-10T20:35:41.735 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:35:41.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.734+0000 7f21f359e700 1 -- 172.21.15.89:0/3088473758 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f21d4066f30 msgr2=0x7f21d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.734+0000 7f21f359e700 1 --2- 172.21.15.89:0/3088473758 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f21d4066f30 0x7f21d40693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f21f41552c0 tx=0x7f21e4009040 comp rx=0 tx=0).stop 2023-10-10T20:35:41.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.734+0000 7f21f359e700 1 -- 172.21.15.89:0/3088473758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21f40c97a0 msgr2=0x7f21f4152640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.734+0000 7f21f359e700 1 --2- 172.21.15.89:0/3088473758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21f40c97a0 0x7f21f4152640 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f21f4157f20 tx=0x7f21e8016000 comp rx=0 tx=0).stop 2023-10-10T20:35:41.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.734+0000 7f21f359e700 1 -- 172.21.15.89:0/3088473758 shutdown_connections 2023-10-10T20:35:41.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.734+0000 7f21f359e700 1 --2- 172.21.15.89:0/3088473758 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f21f41579c0 0x7f21f40b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.734+0000 7f21f359e700 1 --2- 172.21.15.89:0/3088473758 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f21d4066f30 0x7f21d40693e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.734+0000 7f21f359e700 1 --2- 172.21.15.89:0/3088473758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40688d0 0x7f21f4159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.734+0000 7f21f359e700 1 --2- 172.21.15.89:0/3088473758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21f40c97a0 0x7f21f4152640 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.734+0000 7f21f359e700 1 -- 172.21.15.89:0/3088473758 >> 172.21.15.89:0/3088473758 conn(0x7f21f40cddc0 msgr2=0x7f21f406a980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:41.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.734+0000 7f21f359e700 1 -- 172.21.15.89:0/3088473758 shutdown_connections 2023-10-10T20:35:41.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.734+0000 7f21f359e700 1 -- 172.21.15.89:0/3088473758 wait complete. 2023-10-10T20:35:41.748 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:35:41.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.905+0000 7fa515eb3700 1 Processor -- start 2023-10-10T20:35:41.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.905+0000 7fa515eb3700 1 -- start start 2023-10-10T20:35:41.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.905+0000 7fa515eb3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa510140350 0x7fa510069c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:41.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.905+0000 7fa515eb3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5100c7de0 con 0x7fa510068aa0 2023-10-10T20:35:41.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.905+0000 7fa515eb3700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa510140c10 con 0x7fa51006a1c0 2023-10-10T20:35:41.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.905+0000 7fa515eb3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa510141490 con 0x7fa510140350 2023-10-10T20:35:41.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.905+0000 7fa50ffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa510068aa0 0x7fa5100c8270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45454/0 (socket says 172.21.15.89:45454) 2023-10-10T20:35:41.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50ffff700 1 -- 172.21.15.89:0/4026692586 learned_addr learned my addr 172.21.15.89:0/4026692586 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:41.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50e7fc700 1 -- 172.21.15.89:0/4026692586 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 54288632 0 0) 0x7fa5100c7de0 con 0x7fa510068aa0 2023-10-10T20:35:41.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50e7fc700 1 -- 172.21.15.89:0/4026692586 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4f8003570 con 0x7fa510068aa0 2023-10-10T20:35:41.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50f7fe700 1 --2- 172.21.15.89:0/4026692586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa510140350 0x7fa510069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:41.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50e7fc700 1 -- 172.21.15.89:0/4026692586 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3423798266 0 0) 0x7fa4f8003570 con 0x7fa510068aa0 2023-10-10T20:35:41.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50e7fc700 1 -- 172.21.15.89:0/4026692586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa510140350 msgr2=0x7fa510069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:41.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50e7fc700 1 --2- 172.21.15.89:0/4026692586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa510140350 0x7fa510069c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50e7fc700 1 -- 172.21.15.89:0/4026692586 >> v1:172.21.15.111:6789/0 conn(0x7fa51006a1c0 legacy=0x7fa510068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50e7fc700 1 -- 172.21.15.89:0/4026692586 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa510140ed0 con 0x7fa510068aa0 2023-10-10T20:35:41.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50e7fc700 1 -- 172.21.15.89:0/4026692586 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fa508001470 con 0x7fa510068aa0 2023-10-10T20:35:41.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50e7fc700 1 -- 172.21.15.89:0/4026692586 >> v1:172.21.15.89:6789/0 conn(0x7fa510068aa0 legacy=0x7fa5100c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50e7fc700 1 --2- 172.21.15.89:0/4026692586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4f8003a90 0x7fa4f8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:41.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50e7fc700 1 --2- 172.21.15.89:0/4026692586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4f8004810 0x7fa4f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:41.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50e7fc700 1 --2- 172.21.15.89:0/4026692586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa4f8005230 0x7fa4f800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:41.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50e7fc700 1 -- 172.21.15.89:0/4026692586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa510141490 con 0x7fa4f8004810 2023-10-10T20:35:41.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50e7fc700 1 -- 172.21.15.89:0/4026692586 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f800c0d0 con 0x7fa4f8003a90 2023-10-10T20:35:41.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50e7fc700 1 -- 172.21.15.89:0/4026692586 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f800c290 con 0x7fa4f8005230 2023-10-10T20:35:41.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50f7fe700 1 --2- 172.21.15.89:0/4026692586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa510140350 0x7fa510069c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:41.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50effd700 1 --2- 172.21.15.89:0/4026692586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4f8004810 0x7fa4f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:41.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50ffff700 1 --2- 172.21.15.89:0/4026692586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa4f8005230 0x7fa4f800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:41.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50f7fe700 1 --2- 172.21.15.89:0/4026692586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4f8003a90 0x7fa4f8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:41.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50effd700 1 -- 172.21.15.89:0/4026692586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa4f8005230 msgr2=0x7fa4f800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:41.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50effd700 1 --2- 172.21.15.89:0/4026692586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa4f8005230 0x7fa4f800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50effd700 1 -- 172.21.15.89:0/4026692586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4f8003a90 msgr2=0x7fa4f8003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:41.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50effd700 1 --2- 172.21.15.89:0/4026692586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4f8003a90 0x7fa4f8003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.906+0000 7fa50effd700 1 -- 172.21.15.89:0/4026692586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f8003570 con 0x7fa4f8004810 2023-10-10T20:35:41.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.907+0000 7fa50effd700 1 --2- 172.21.15.89:0/4026692586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4f8004810 0x7fa4f8004c80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa5040018e0 tx=0x7fa504007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca215d9a808e7dfc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:41.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.907+0000 7fa50e7fc700 1 -- 172.21.15.89:0/4026692586 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504012420 con 0x7fa4f8004810 2023-10-10T20:35:41.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.907+0000 7fa50e7fc700 1 -- 172.21.15.89:0/4026692586 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa504012580 con 0x7fa4f8004810 2023-10-10T20:35:41.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.907+0000 7fa50e7fc700 1 -- 172.21.15.89:0/4026692586 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504012920 con 0x7fa4f8004810 2023-10-10T20:35:41.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.907+0000 7fa515eb3700 1 -- 172.21.15.89:0/4026692586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4f8004810 msgr2=0x7fa4f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.907+0000 7fa515eb3700 1 --2- 172.21.15.89:0/4026692586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4f8004810 0x7fa4f8004c80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa5040018e0 tx=0x7fa504007d30 comp rx=0 tx=0).stop 2023-10-10T20:35:41.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.907+0000 7fa50ffff700 1 -- 172.21.15.89:0/4026692586 reap_dead start 2023-10-10T20:35:41.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.907+0000 7fa515eb3700 1 -- 172.21.15.89:0/4026692586 shutdown_connections 2023-10-10T20:35:41.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.907+0000 7fa515eb3700 1 -- 172.21.15.89:0/4026692586 >> 172.21.15.89:0/4026692586 conn(0x7fa5100cd670 msgr2=0x7fa5100cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:41.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.907+0000 7fa515eb3700 1 -- 172.21.15.89:0/4026692586 shutdown_connections 2023-10-10T20:35:41.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.907+0000 7fa515eb3700 1 -- 172.21.15.89:0/4026692586 wait complete. 2023-10-10T20:35:41.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.909+0000 7fa515eb3700 1 Processor -- start 2023-10-10T20:35:41.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.909+0000 7fa515eb3700 1 -- start start 2023-10-10T20:35:41.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.909+0000 7fa515eb3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa4f8004810 0x7fa5100c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:41.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.909+0000 7fa515eb3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa510154da0 0x7fa51016b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:41.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.909+0000 7fa515eb3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5100c6560 0x7fa51016bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:41.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.909+0000 7fa515eb3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5100c70f0 con 0x7fa5100c6560 2023-10-10T20:35:41.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.909+0000 7fa515eb3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5101697b0 con 0x7fa510154da0 2023-10-10T20:35:41.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.909+0000 7fa515eb3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5100c6b30 con 0x7fa4f8004810 2023-10-10T20:35:41.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.909+0000 7fa50ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5100c6560 0x7fa51016bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:41.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.909+0000 7fa50ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5100c6560 0x7fa51016bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58112/0 (socket says 172.21.15.89:58112) 2023-10-10T20:35:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.909+0000 7fa50ffff700 1 -- 172.21.15.89:0/1723270277 learned_addr learned my addr 172.21.15.89:0/1723270277 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.909+0000 7fa50effd700 1 --2- 172.21.15.89:0/1723270277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa510154da0 0x7fa51016b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.909+0000 7fa50f7fe700 1 --2- 172.21.15.89:0/1723270277 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa4f8004810 0x7fa5100c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.910+0000 7fa50ffff700 1 -- 172.21.15.89:0/1723270277 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa4f8004810 msgr2=0x7fa5100c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.910+0000 7fa50ffff700 1 --2- 172.21.15.89:0/1723270277 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa4f8004810 0x7fa5100c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.910+0000 7fa50ffff700 1 -- 172.21.15.89:0/1723270277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa510154da0 msgr2=0x7fa51016b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:41.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.910+0000 7fa50ffff700 1 --2- 172.21.15.89:0/1723270277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa510154da0 0x7fa51016b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:41.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.910+0000 7fa50ffff700 1 -- 172.21.15.89:0/1723270277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5100c6e10 con 0x7fa5100c6560 2023-10-10T20:35:41.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.910+0000 7fa50ffff700 1 --2- 172.21.15.89:0/1723270277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5100c6560 0x7fa51016bc10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa508000c00 tx=0x7fa5100c7d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:41.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.910+0000 7fa4f77fe700 1 -- 172.21.15.89:0/1723270277 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508018030 con 0x7fa5100c6560 2023-10-10T20:35:41.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.910+0000 7fa4f77fe700 1 -- 172.21.15.89:0/1723270277 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50800f840 con 0x7fa5100c6560 2023-10-10T20:35:41.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.910+0000 7fa4f77fe700 1 -- 172.21.15.89:0/1723270277 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50800fbe0 con 0x7fa5100c6560 2023-10-10T20:35:41.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.910+0000 7fa515eb3700 1 -- 172.21.15.89:0/1723270277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa510169440 con 0x7fa5100c6560 2023-10-10T20:35:41.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.910+0000 7fa515eb3700 1 -- 172.21.15.89:0/1723270277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51013bcd0 con 0x7fa5100c6560 2023-10-10T20:35:41.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.911+0000 7fa515eb3700 1 -- 172.21.15.89:0/1723270277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5101692a0 con 0x7fa5100c6560 2023-10-10T20:35:41.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.911+0000 7fa4f77fe700 1 -- 172.21.15.89:0/1723270277 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa50801b070 con 0x7fa5100c6560 2023-10-10T20:35:41.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.912+0000 7fa4f77fe700 1 --2- 172.21.15.89:0/1723270277 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa4fc066f30 0x7fa4fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:41.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.912+0000 7fa50f7fe700 1 --2- 172.21.15.89:0/1723270277 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa4fc066f30 0x7fa4fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:41.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.912+0000 7fa4f77fe700 1 -- 172.21.15.89:0/1723270277 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fa508019070 con 0x7fa5100c6560 2023-10-10T20:35:41.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.912+0000 7fa50f7fe700 1 --2- 172.21.15.89:0/1723270277 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa4fc066f30 0x7fa4fc0693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa51006d4b0 tx=0x7fa500009800 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:41.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:41.916+0000 7fa4f77fe700 1 -- 172.21.15.89:0/1723270277 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5101692a0 con 0x7fa5100c6560 2023-10-10T20:35:42.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.098+0000 7fa515eb3700 1 -- 172.21.15.89:0/1723270277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa5101692a0 con 0x7fa5100c6560 2023-10-10T20:35:42.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.098+0000 7fa4f77fe700 1 -- 172.21.15.89:0/1723270277 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa5101692a0 con 0x7fa5100c6560 2023-10-10T20:35:42.101 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:35:42.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.100+0000 7fa515eb3700 1 -- 172.21.15.89:0/1723270277 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa4fc066f30 msgr2=0x7fa4fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:42.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.100+0000 7fa515eb3700 1 --2- 172.21.15.89:0/1723270277 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa4fc066f30 0x7fa4fc0693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa51006d4b0 tx=0x7fa500009800 comp rx=0 tx=0).stop 2023-10-10T20:35:42.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.100+0000 7fa515eb3700 1 -- 172.21.15.89:0/1723270277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5100c6560 msgr2=0x7fa51016bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:42.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.100+0000 7fa515eb3700 1 --2- 172.21.15.89:0/1723270277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5100c6560 0x7fa51016bc10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa508000c00 tx=0x7fa5100c7d50 comp rx=0 tx=0).stop 2023-10-10T20:35:42.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.100+0000 7fa515eb3700 1 -- 172.21.15.89:0/1723270277 shutdown_connections 2023-10-10T20:35:42.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.100+0000 7fa515eb3700 1 --2- 172.21.15.89:0/1723270277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5100c6560 0x7fa51016bc10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.100+0000 7fa515eb3700 1 --2- 172.21.15.89:0/1723270277 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa4f8004810 0x7fa5100c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.100+0000 7fa515eb3700 1 --2- 172.21.15.89:0/1723270277 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa4fc066f30 0x7fa4fc0693e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.100+0000 7fa515eb3700 1 --2- 172.21.15.89:0/1723270277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa510154da0 0x7fa51016b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.100+0000 7fa515eb3700 1 -- 172.21.15.89:0/1723270277 >> 172.21.15.89:0/1723270277 conn(0x7fa5100cd670 msgr2=0x7fa5100ca6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:42.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.100+0000 7fa515eb3700 1 -- 172.21.15.89:0/1723270277 shutdown_connections 2023-10-10T20:35:42.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.100+0000 7fa515eb3700 1 -- 172.21.15.89:0/1723270277 wait complete. 2023-10-10T20:35:42.114 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:35:42.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.229+0000 7fb44a24a700 1 Processor -- start 2023-10-10T20:35:42.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.229+0000 7fb44a24a700 1 -- start start 2023-10-10T20:35:42.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.230+0000 7fb44a24a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb4440b16d0 0x7fb4440b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:42.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.230+0000 7fb44a24a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4440b33b0 0x7fb4440b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:42.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.230+0000 7fb44a24a700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4440b2080 con 0x7fb4440b4e70 2023-10-10T20:35:42.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.230+0000 7fb44a24a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4440cfb00 con 0x7fb4440b16d0 2023-10-10T20:35:42.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.230+0000 7fb44a24a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440b5c40 con 0x7fb4440b33b0 2023-10-10T20:35:42.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.230+0000 7fb448a47700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb4440b16d0 0x7fb4440b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:42.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.230+0000 7fb448a47700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb4440b16d0 0x7fb4440b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58114/0 (socket says 172.21.15.89:58114) 2023-10-10T20:35:42.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.230+0000 7fb448a47700 1 -- 172.21.15.89:0/760076574 learned_addr learned my addr 172.21.15.89:0/760076574 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:42.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.230+0000 7fb449248700 1 --1- 172.21.15.89:0/760076574 >> v1:172.21.15.111:6789/0 conn(0x7fb4440b4e70 0x7fb4440ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:48312/0 (socket says 172.21.15.89:48312) 2023-10-10T20:35:42.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.230+0000 7fb449a49700 1 --2- 172.21.15.89:0/760076574 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4440b33b0 0x7fb4440b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:42.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.230+0000 7fb448a47700 1 -- 172.21.15.89:0/760076574 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4440b33b0 msgr2=0x7fb4440b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:42.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.230+0000 7fb448a47700 1 --2- 172.21.15.89:0/760076574 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4440b33b0 0x7fb4440b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.230+0000 7fb448a47700 1 -- 172.21.15.89:0/760076574 >> v1:172.21.15.111:6789/0 conn(0x7fb4440b4e70 legacy=0x7fb4440ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:42.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.230+0000 7fb448a47700 1 -- 172.21.15.89:0/760076574 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4440b5f20 con 0x7fb4440b16d0 2023-10-10T20:35:42.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.231+0000 7fb448a47700 1 --2- 172.21.15.89:0/760076574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4440b16d0 0x7fb4440b1b40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb4340099b0 tx=0x7fb434009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc72afcf69217318 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:42.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.231+0000 7fb43bfff700 1 -- 172.21.15.89:0/760076574 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb434009ea0 con 0x7fb4440b16d0 2023-10-10T20:35:42.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.231+0000 7fb43bfff700 1 -- 172.21.15.89:0/760076574 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43400b860 con 0x7fb4440b16d0 2023-10-10T20:35:42.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.231+0000 7fb43bfff700 1 -- 172.21.15.89:0/760076574 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43400bc20 con 0x7fb4440b16d0 2023-10-10T20:35:42.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.231+0000 7fb44a24a700 1 -- 172.21.15.89:0/760076574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4440b16d0 msgr2=0x7fb4440b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:42.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.231+0000 7fb44a24a700 1 --2- 172.21.15.89:0/760076574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4440b16d0 0x7fb4440b1b40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb4340099b0 tx=0x7fb434009250 comp rx=0 tx=0).stop 2023-10-10T20:35:42.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.231+0000 7fb44a24a700 1 -- 172.21.15.89:0/760076574 shutdown_connections 2023-10-10T20:35:42.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.231+0000 7fb44a24a700 1 --2- 172.21.15.89:0/760076574 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4440b33b0 0x7fb4440b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.231+0000 7fb44a24a700 1 --2- 172.21.15.89:0/760076574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4440b16d0 0x7fb4440b1b40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.231+0000 7fb44a24a700 1 -- 172.21.15.89:0/760076574 >> 172.21.15.89:0/760076574 conn(0x7fb4440d10c0 msgr2=0x7fb4440affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:42.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.231+0000 7fb44a24a700 1 -- 172.21.15.89:0/760076574 shutdown_connections 2023-10-10T20:35:42.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.231+0000 7fb44a24a700 1 -- 172.21.15.89:0/760076574 wait complete. 2023-10-10T20:35:42.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.232+0000 7fb44a24a700 1 Processor -- start 2023-10-10T20:35:42.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.232+0000 7fb44a24a700 1 -- start start 2023-10-10T20:35:42.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.233+0000 7fb44a24a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4440b16d0 0x7fb4440a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:42.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.233+0000 7fb44a24a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4440b33b0 0x7fb4440c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:42.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.233+0000 7fb44a24a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4440b4e70 0x7fb444169ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:42.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.233+0000 7fb44a24a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44416ae60 con 0x7fb4440b16d0 2023-10-10T20:35:42.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.233+0000 7fb44a24a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440a7440 con 0x7fb4440b4e70 2023-10-10T20:35:42.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.233+0000 7fb44a24a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440cac20 con 0x7fb4440b33b0 2023-10-10T20:35:42.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.233+0000 7fb449248700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4440b16d0 0x7fb4440a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:42.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.233+0000 7fb449248700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4440b16d0 0x7fb4440a6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58118/0 (socket says 172.21.15.89:58118) 2023-10-10T20:35:42.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.233+0000 7fb449248700 1 -- 172.21.15.89:0/73258819 learned_addr learned my addr 172.21.15.89:0/73258819 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:42.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.234+0000 7fb449248700 1 -- 172.21.15.89:0/73258819 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4440b33b0 msgr2=0x7fb4440c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:42.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.234+0000 7fb449248700 1 --2- 172.21.15.89:0/73258819 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4440b33b0 0x7fb4440c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.234+0000 7fb449a49700 1 --2- 172.21.15.89:0/73258819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4440b4e70 0x7fb444169ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:42.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.234+0000 7fb449248700 1 -- 172.21.15.89:0/73258819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4440b4e70 msgr2=0x7fb444169ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:42.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.234+0000 7fb449248700 1 --2- 172.21.15.89:0/73258819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4440b4e70 0x7fb444169ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.234+0000 7fb449248700 1 -- 172.21.15.89:0/73258819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4340094f0 con 0x7fb4440b16d0 2023-10-10T20:35:42.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.234+0000 7fb449248700 1 --2- 172.21.15.89:0/73258819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4440b16d0 0x7fb4440a6f00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb44000c7c0 tx=0x7fb440007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:42.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.234+0000 7fb4397fa700 1 -- 172.21.15.89:0/73258819 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440005a70 con 0x7fb4440b16d0 2023-10-10T20:35:42.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.234+0000 7fb4397fa700 1 -- 172.21.15.89:0/73258819 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb440005bf0 con 0x7fb4440b16d0 2023-10-10T20:35:42.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.234+0000 7fb44a24a700 1 -- 172.21.15.89:0/73258819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4440caf00 con 0x7fb4440b16d0 2023-10-10T20:35:42.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.234+0000 7fb4397fa700 1 -- 172.21.15.89:0/73258819 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4400070f0 con 0x7fb4440b16d0 2023-10-10T20:35:42.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.234+0000 7fb44a24a700 1 -- 172.21.15.89:0/73258819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4440cb290 con 0x7fb4440b16d0 2023-10-10T20:35:42.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.235+0000 7fb44a24a700 1 -- 172.21.15.89:0/73258819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb444088990 con 0x7fb4440b16d0 2023-10-10T20:35:42.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.235+0000 7fb4397fa700 1 -- 172.21.15.89:0/73258819 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb440007460 con 0x7fb4440b16d0 2023-10-10T20:35:42.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.236+0000 7fb4397fa700 1 --2- 172.21.15.89:0/73258819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb428066f30 0x7fb4280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:42.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.236+0000 7fb4397fa700 1 -- 172.21.15.89:0/73258819 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fb440083fb0 con 0x7fb4440b16d0 2023-10-10T20:35:42.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.236+0000 7fb448a47700 1 --2- 172.21.15.89:0/73258819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb428066f30 0x7fb4280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:42.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.236+0000 7fb448a47700 1 --2- 172.21.15.89:0/73258819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb428066f30 0x7fb4280693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb4340099b0 tx=0x7fb434017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:42.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.240+0000 7fb4397fa700 1 -- 172.21.15.89:0/73258819 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb440051250 con 0x7fb4440b16d0 2023-10-10T20:35:42.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.424+0000 7fb44a24a700 1 -- 172.21.15.89:0/73258819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb444069b20 con 0x7fb4440b16d0 2023-10-10T20:35:42.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.424+0000 7fb4397fa700 1 -- 172.21.15.89:0/73258819 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb440054870 con 0x7fb4440b16d0 2023-10-10T20:35:42.427 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-10T20:35:42.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.427+0000 7fb44a24a700 1 -- 172.21.15.89:0/73258819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb428066f30 msgr2=0x7fb4280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:42.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.427+0000 7fb44a24a700 1 --2- 172.21.15.89:0/73258819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb428066f30 0x7fb4280693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb4340099b0 tx=0x7fb434017000 comp rx=0 tx=0).stop 2023-10-10T20:35:42.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.427+0000 7fb44a24a700 1 -- 172.21.15.89:0/73258819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4440b16d0 msgr2=0x7fb4440a6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:42.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.427+0000 7fb44a24a700 1 --2- 172.21.15.89:0/73258819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4440b16d0 0x7fb4440a6f00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb44000c7c0 tx=0x7fb440007f70 comp rx=0 tx=0).stop 2023-10-10T20:35:42.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.427+0000 7fb44a24a700 1 -- 172.21.15.89:0/73258819 shutdown_connections 2023-10-10T20:35:42.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.427+0000 7fb44a24a700 1 --2- 172.21.15.89:0/73258819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb428066f30 0x7fb4280693e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.427+0000 7fb44a24a700 1 --2- 172.21.15.89:0/73258819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4440b4e70 0x7fb444169ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.427+0000 7fb44a24a700 1 --2- 172.21.15.89:0/73258819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4440b16d0 0x7fb4440a6f00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.427+0000 7fb44a24a700 1 --2- 172.21.15.89:0/73258819 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4440b33b0 0x7fb4440c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.427+0000 7fb44a24a700 1 -- 172.21.15.89:0/73258819 >> 172.21.15.89:0/73258819 conn(0x7fb4440d10c0 msgr2=0x7fb4441590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:42.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.427+0000 7fb44a24a700 1 -- 172.21.15.89:0/73258819 shutdown_connections 2023-10-10T20:35:42.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.427+0000 7fb44a24a700 1 -- 172.21.15.89:0/73258819 wait complete. 2023-10-10T20:35:42.441 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:35:42.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.555+0000 7f7ba694e700 1 Processor -- start 2023-10-10T20:35:42.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.556+0000 7f7ba694e700 1 -- start start 2023-10-10T20:35:42.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.556+0000 7f7ba694e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7ba0159890 0x7f7ba015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:42.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.556+0000 7f7ba694e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7ba0069560 0x7f7ba00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:42.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.556+0000 7f7ba694e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba0069ee0 con 0x7f7ba0159c60 2023-10-10T20:35:42.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.556+0000 7f7ba694e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba015a160 con 0x7f7ba0159890 2023-10-10T20:35:42.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.556+0000 7f7ba694e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba00d13a0 con 0x7f7ba0069560 2023-10-10T20:35:42.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.556+0000 7f7b9ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7ba0159890 0x7f7ba015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:42.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.556+0000 7f7b9f7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7ba0159c60 0x7f7ba015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45470/0 (socket says 172.21.15.89:45470) 2023-10-10T20:35:42.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.556+0000 7f7b9ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7ba0159890 0x7f7ba015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58126/0 (socket says 172.21.15.89:58126) 2023-10-10T20:35:42.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.556+0000 7f7b9f7fe700 1 -- 172.21.15.89:0/730861352 learned_addr learned my addr 172.21.15.89:0/730861352 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:42.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.557+0000 7f7ba4eeb700 1 --2- 172.21.15.89:0/730861352 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7ba0069560 0x7f7ba00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:42.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.557+0000 7f7b9ffff700 1 -- 172.21.15.89:0/730861352 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7ba0069560 msgr2=0x7f7ba00699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:42.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.557+0000 7f7b9ffff700 1 --2- 172.21.15.89:0/730861352 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7ba0069560 0x7f7ba00699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.557+0000 7f7b9ffff700 1 -- 172.21.15.89:0/730861352 >> v1:172.21.15.89:6789/0 conn(0x7f7ba0159c60 legacy=0x7f7ba015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:42.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.557+0000 7f7b9effd700 1 -- 172.21.15.89:0/730861352 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2281121224 0 0) 0x7f7ba0069ee0 con 0x7f7ba0159c60 2023-10-10T20:35:42.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.557+0000 7f7b9ffff700 1 -- 172.21.15.89:0/730861352 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba015e140 con 0x7f7ba0159890 2023-10-10T20:35:42.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.557+0000 7f7ba4eeb700 1 --2- 172.21.15.89:0/730861352 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7ba0069560 0x7f7ba00699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:42.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.557+0000 7f7b9ffff700 1 --2- 172.21.15.89:0/730861352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ba0159890 0x7f7ba015abd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7b900098e0 tx=0x7f7b900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c718b0a33341bc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:42.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.557+0000 7f7b9effd700 1 -- 172.21.15.89:0/730861352 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b90009e80 con 0x7f7ba0159890 2023-10-10T20:35:42.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.557+0000 7f7b9effd700 1 -- 172.21.15.89:0/730861352 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9000b860 con 0x7f7ba0159890 2023-10-10T20:35:42.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.557+0000 7f7b9effd700 1 -- 172.21.15.89:0/730861352 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9000bbe0 con 0x7f7ba0159890 2023-10-10T20:35:42.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.558+0000 7f7ba694e700 1 -- 172.21.15.89:0/730861352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ba0159890 msgr2=0x7f7ba015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:42.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.558+0000 7f7ba694e700 1 --2- 172.21.15.89:0/730861352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ba0159890 0x7f7ba015abd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7b900098e0 tx=0x7f7b900091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.558+0000 7f7ba694e700 1 -- 172.21.15.89:0/730861352 shutdown_connections 2023-10-10T20:35:42.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.558+0000 7f7ba694e700 1 --2- 172.21.15.89:0/730861352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ba0159890 0x7f7ba015abd0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.558+0000 7f7ba694e700 1 --2- 172.21.15.89:0/730861352 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7ba0069560 0x7f7ba00699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.558+0000 7f7ba694e700 1 -- 172.21.15.89:0/730861352 >> 172.21.15.89:0/730861352 conn(0x7f7ba0068e20 msgr2=0x7f7ba00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:42.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.558+0000 7f7ba694e700 1 -- 172.21.15.89:0/730861352 shutdown_connections 2023-10-10T20:35:42.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.558+0000 7f7ba694e700 1 -- 172.21.15.89:0/730861352 wait complete. 2023-10-10T20:35:42.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.559+0000 7f7ba694e700 1 Processor -- start 2023-10-10T20:35:42.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.559+0000 7f7ba694e700 1 -- start start 2023-10-10T20:35:42.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7ba694e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ba0069560 0x7f7ba00c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:42.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7ba694e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7ba0159890 0x7f7ba014a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:42.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7ba694e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ba0159c60 0x7f7ba00ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:42.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7ba694e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba00c8ce0 con 0x7f7ba0069560 2023-10-10T20:35:42.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7ba694e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba00b6460 con 0x7f7ba0159c60 2023-10-10T20:35:42.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7ba694e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba00b5ee0 con 0x7f7ba0159890 2023-10-10T20:35:42.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7b9ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ba0069560 0x7f7ba00c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:42.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7b9ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ba0069560 0x7f7ba00c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58140/0 (socket says 172.21.15.89:58140) 2023-10-10T20:35:42.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7b9ffff700 1 -- 172.21.15.89:0/1673712435 learned_addr learned my addr 172.21.15.89:0/1673712435 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:42.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7b9ffff700 1 -- 172.21.15.89:0/1673712435 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7ba0159890 msgr2=0x7f7ba014a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:42.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7b9f7fe700 1 --2- 172.21.15.89:0/1673712435 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7ba0159890 0x7f7ba014a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:42.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7ba4eeb700 1 --2- 172.21.15.89:0/1673712435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ba0159c60 0x7f7ba00ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:42.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7b9ffff700 1 --2- 172.21.15.89:0/1673712435 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7ba0159890 0x7f7ba014a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7b9ffff700 1 -- 172.21.15.89:0/1673712435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ba0159c60 msgr2=0x7f7ba00ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:42.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7b9ffff700 1 --2- 172.21.15.89:0/1673712435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ba0159c60 0x7f7ba00ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7b9ffff700 1 -- 172.21.15.89:0/1673712435 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b90009420 con 0x7f7ba0069560 2023-10-10T20:35:42.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7b9ffff700 1 --2- 172.21.15.89:0/1673712435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ba0069560 0x7f7ba00c78a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7ba00899c0 tx=0x7f7b90017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:42.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7b87fff700 1 -- 172.21.15.89:0/1673712435 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9000bbe0 con 0x7f7ba0069560 2023-10-10T20:35:42.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7b87fff700 1 -- 172.21.15.89:0/1673712435 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b90011420 con 0x7f7ba0069560 2023-10-10T20:35:42.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7ba694e700 1 -- 172.21.15.89:0/1673712435 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba00b6180 con 0x7f7ba0069560 2023-10-10T20:35:42.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7b87fff700 1 -- 172.21.15.89:0/1673712435 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b900117c0 con 0x7f7ba0069560 2023-10-10T20:35:42.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.560+0000 7f7ba694e700 1 -- 172.21.15.89:0/1673712435 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba00b5400 con 0x7f7ba0069560 2023-10-10T20:35:42.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.561+0000 7f7ba694e700 1 -- 172.21.15.89:0/1673712435 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba004c3c0 con 0x7f7ba0069560 2023-10-10T20:35:42.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.562+0000 7f7b87fff700 1 -- 172.21.15.89:0/1673712435 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7b90011b70 con 0x7f7ba0069560 2023-10-10T20:35:42.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.562+0000 7f7b87fff700 1 --2- 172.21.15.89:0/1673712435 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7b8c066f30 0x7f7b8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:42.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.562+0000 7f7b87fff700 1 -- 172.21.15.89:0/1673712435 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f7b90085d30 con 0x7f7ba0069560 2023-10-10T20:35:42.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.562+0000 7f7b9f7fe700 1 --2- 172.21.15.89:0/1673712435 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7b8c066f30 0x7f7b8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:42.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.562+0000 7f7b9f7fe700 1 --2- 172.21.15.89:0/1673712435 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7b8c066f30 0x7f7b8c0693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0064fb0 tx=0x7f7b94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:42.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.566+0000 7f7b87fff700 1 -- 172.21.15.89:0/1673712435 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b90052f50 con 0x7f7ba0069560 2023-10-10T20:35:42.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.792+0000 7f7ba694e700 1 -- 172.21.15.89:0/1673712435 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ba006a5b0 con 0x7f7ba0069560 2023-10-10T20:35:42.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.792+0000 7f7b87fff700 1 -- 172.21.15.89:0/1673712435 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f7b90056570 con 0x7f7ba0069560 2023-10-10T20:35:42.795 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:42.795 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1977066906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1977066906},{"type":"v1","addr":"172.21.15.111:6833","nonce":1977066906}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2904313060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2904313060},{"type":"v1","addr":"172.21.15.111:6835","nonce":2904313060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/4152616589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":4152616589},{"type":"v1","addr":"172.21.15.89:6836","nonce":4152616589}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:35:17.900533+0000","modified":"2023-10-10T20:35:21.533574+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4899":{"gid":4899,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1330347277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1330347277},{"type":"v1","addr":"172.21.15.89:6837","nonce":1330347277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-10T20:35:42.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.794+0000 7f7ba694e700 1 -- 172.21.15.89:0/1673712435 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7b8c066f30 msgr2=0x7f7b8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:42.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.794+0000 7f7ba694e700 1 --2- 172.21.15.89:0/1673712435 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7b8c066f30 0x7f7b8c0693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0064fb0 tx=0x7f7b94009040 comp rx=0 tx=0).stop 2023-10-10T20:35:42.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.794+0000 7f7ba694e700 1 -- 172.21.15.89:0/1673712435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ba0069560 msgr2=0x7f7ba00c78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:42.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.794+0000 7f7ba694e700 1 --2- 172.21.15.89:0/1673712435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ba0069560 0x7f7ba00c78a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7ba00899c0 tx=0x7f7b90017000 comp rx=0 tx=0).stop 2023-10-10T20:35:42.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.794+0000 7f7ba694e700 1 -- 172.21.15.89:0/1673712435 shutdown_connections 2023-10-10T20:35:42.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.794+0000 7f7ba694e700 1 --2- 172.21.15.89:0/1673712435 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7b8c066f30 0x7f7b8c0693e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.794+0000 7f7ba694e700 1 --2- 172.21.15.89:0/1673712435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ba0159c60 0x7f7ba00ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.794+0000 7f7ba694e700 1 --2- 172.21.15.89:0/1673712435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ba0069560 0x7f7ba00c78a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.794+0000 7f7ba694e700 1 --2- 172.21.15.89:0/1673712435 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7ba0159890 0x7f7ba014a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.794+0000 7f7ba694e700 1 -- 172.21.15.89:0/1673712435 >> 172.21.15.89:0/1673712435 conn(0x7f7ba0068e20 msgr2=0x7f7ba00b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:42.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.794+0000 7f7ba694e700 1 -- 172.21.15.89:0/1673712435 shutdown_connections 2023-10-10T20:35:42.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.794+0000 7f7ba694e700 1 -- 172.21.15.89:0/1673712435 wait complete. 2023-10-10T20:35:42.799 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 22 2023-10-10T20:35:42.808 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:35:42.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.921+0000 7effa6bfb700 1 Processor -- start 2023-10-10T20:35:42.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.921+0000 7effa6bfb700 1 -- start start 2023-10-10T20:35:42.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.922+0000 7effa6bfb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7effa0159420 0x7effa00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:42.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.922+0000 7effa6bfb700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7effa015d230 0x7effa015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:42.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.922+0000 7effa6bfb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa015dbd0 con 0x7effa01597f0 2023-10-10T20:35:42.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.922+0000 7effa6bfb700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa00b2980 con 0x7effa015d230 2023-10-10T20:35:42.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.922+0000 7effa6bfb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa00b11d0 con 0x7effa0159420 2023-10-10T20:35:42.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.922+0000 7effa53f8700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7effa01597f0 0x7effa00b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45478/0 (socket says 172.21.15.89:45478) 2023-10-10T20:35:42.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.922+0000 7effa53f8700 1 -- 172.21.15.89:0/712704232 learned_addr learned my addr 172.21.15.89:0/712704232 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:42.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.922+0000 7effa5bf9700 1 --2- 172.21.15.89:0/712704232 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7effa0159420 0x7effa00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:42.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.922+0000 7effa63fa700 1 --2- 172.21.15.89:0/712704232 >> v2:172.21.15.111:3300/0 conn(0x7effa015d230 0x7effa015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:42.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.922+0000 7effa4bf7700 1 -- 172.21.15.89:0/712704232 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3150741182 0 0) 0x7eff90001110 con 0x7effa01597f0 2023-10-10T20:35:42.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.923+0000 7effa4bf7700 1 -- 172.21.15.89:0/712704232 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff8c003570 con 0x7effa01597f0 2023-10-10T20:35:42.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.923+0000 7effa5bf9700 1 -- 172.21.15.89:0/712704232 >> v2:172.21.15.111:3300/0 conn(0x7effa015d230 msgr2=0x7effa015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:42.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.923+0000 7effa5bf9700 1 --2- 172.21.15.89:0/712704232 >> v2:172.21.15.111:3300/0 conn(0x7effa015d230 0x7effa015d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.923+0000 7effa5bf9700 1 -- 172.21.15.89:0/712704232 >> v1:172.21.15.89:6789/0 conn(0x7effa01597f0 legacy=0x7effa00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:42.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.923+0000 7effa4bf7700 1 -- 172.21.15.89:0/712704232 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1236853687 0 0) 0x7eff8c003570 con 0x7effa01597f0 2023-10-10T20:35:42.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.923+0000 7effa5bf9700 1 -- 172.21.15.89:0/712704232 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa00b14b0 con 0x7effa0159420 2023-10-10T20:35:42.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.923+0000 7effa63fa700 1 --2- 172.21.15.89:0/712704232 >> v2:172.21.15.111:3300/0 conn(0x7effa015d230 0x7effa015d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:35:42.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.923+0000 7effa5bf9700 1 --2- 172.21.15.89:0/712704232 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7effa0159420 0x7effa00b5f40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009e60 tx=0x7eff9c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=468ff1a20e430dce server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:42.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.923+0000 7effa4bf7700 1 -- 172.21.15.89:0/712704232 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c00e070 con 0x7effa0159420 2023-10-10T20:35:42.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.923+0000 7effa4bf7700 1 -- 172.21.15.89:0/712704232 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c00b7a0 con 0x7effa0159420 2023-10-10T20:35:42.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.923+0000 7effa4bf7700 1 -- 172.21.15.89:0/712704232 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90001600 con 0x7effa0159420 2023-10-10T20:35:42.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.924+0000 7effa6bfb700 1 -- 172.21.15.89:0/712704232 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7effa0159420 msgr2=0x7effa00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:42.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.924+0000 7effa6bfb700 1 --2- 172.21.15.89:0/712704232 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7effa0159420 0x7effa00b5f40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009e60 tx=0x7eff9c009780 comp rx=0 tx=0).stop 2023-10-10T20:35:42.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.924+0000 7effa6bfb700 1 -- 172.21.15.89:0/712704232 shutdown_connections 2023-10-10T20:35:42.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.924+0000 7effa6bfb700 1 --2- 172.21.15.89:0/712704232 >> v2:172.21.15.111:3300/0 conn(0x7effa015d230 0x7effa015d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.924+0000 7effa6bfb700 1 --2- 172.21.15.89:0/712704232 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7effa0159420 0x7effa00b5f40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.924+0000 7effa6bfb700 1 -- 172.21.15.89:0/712704232 >> 172.21.15.89:0/712704232 conn(0x7effa00d0660 msgr2=0x7effa00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:42.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.924+0000 7effa6bfb700 1 -- 172.21.15.89:0/712704232 shutdown_connections 2023-10-10T20:35:42.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.924+0000 7effa6bfb700 1 -- 172.21.15.89:0/712704232 wait complete. 2023-10-10T20:35:42.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.925+0000 7effa6bfb700 1 Processor -- start 2023-10-10T20:35:42.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.925+0000 7effa6bfb700 1 -- start start 2023-10-10T20:35:42.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7effa6bfb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effa0159420 0x7effa00cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:42.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7effa6bfb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7effa01597f0 0x7effa00c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:42.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7effa6bfb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effa015d230 0x7effa00c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:42.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7effa6bfb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa00cac20 con 0x7effa015d230 2023-10-10T20:35:42.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7effa6bfb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa00c96f0 con 0x7effa0159420 2023-10-10T20:35:42.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7effa6bfb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa00cae50 con 0x7effa01597f0 2023-10-10T20:35:42.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7effa63fa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effa015d230 0x7effa00c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:42.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7effa63fa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effa015d230 0x7effa00c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58156/0 (socket says 172.21.15.89:58156) 2023-10-10T20:35:42.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7effa63fa700 1 -- 172.21.15.89:0/921654760 learned_addr learned my addr 172.21.15.89:0/921654760 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:42.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7effa53f8700 1 --2- 172.21.15.89:0/921654760 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7effa01597f0 0x7effa00c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:42.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7effa5bf9700 1 --2- 172.21.15.89:0/921654760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effa0159420 0x7effa00cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:42.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7effa63fa700 1 -- 172.21.15.89:0/921654760 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7effa01597f0 msgr2=0x7effa00c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:42.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7effa63fa700 1 --2- 172.21.15.89:0/921654760 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7effa01597f0 0x7effa00c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7effa63fa700 1 -- 172.21.15.89:0/921654760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effa0159420 msgr2=0x7effa00cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:42.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7effa63fa700 1 --2- 172.21.15.89:0/921654760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effa0159420 0x7effa00cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:42.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7effa63fa700 1 -- 172.21.15.89:0/921654760 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff98009420 con 0x7effa015d230 2023-10-10T20:35:42.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7effa63fa700 1 --2- 172.21.15.89:0/921654760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effa015d230 0x7effa00c9100 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7eff9800ea20 tx=0x7eff980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:42.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7eff95ffb700 1 -- 172.21.15.89:0/921654760 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9800ee60 con 0x7effa015d230 2023-10-10T20:35:42.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7eff95ffb700 1 -- 172.21.15.89:0/921654760 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9800c850 con 0x7effa015d230 2023-10-10T20:35:42.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7eff95ffb700 1 -- 172.21.15.89:0/921654760 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9800cc10 con 0x7effa015d230 2023-10-10T20:35:42.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.926+0000 7effa6bfb700 1 -- 172.21.15.89:0/921654760 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff9c009a20 con 0x7effa015d230 2023-10-10T20:35:42.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.927+0000 7effa6bfb700 1 -- 172.21.15.89:0/921654760 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa00cb490 con 0x7effa015d230 2023-10-10T20:35:42.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.928+0000 7effa6bfb700 1 -- 172.21.15.89:0/921654760 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa004c3c0 con 0x7effa015d230 2023-10-10T20:35:42.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.928+0000 7eff95ffb700 1 -- 172.21.15.89:0/921654760 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7eff98013470 con 0x7effa015d230 2023-10-10T20:35:42.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.928+0000 7eff95ffb700 1 --2- 172.21.15.89:0/921654760 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7eff84066f30 0x7eff840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:42.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.928+0000 7eff95ffb700 1 -- 172.21.15.89:0/921654760 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5791+0+0 (secure 0 0 0) 0x7eff980844f0 con 0x7effa015d230 2023-10-10T20:35:42.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.928+0000 7effa5bf9700 1 --2- 172.21.15.89:0/921654760 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7eff84066f30 0x7eff840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:42.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.928+0000 7effa5bf9700 1 --2- 172.21.15.89:0/921654760 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7eff84066f30 0x7eff840693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0099f0 tx=0x7eff9c018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:42.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:42.932+0000 7eff95ffb700 1 -- 172.21.15.89:0/921654760 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff98051710 con 0x7effa015d230 2023-10-10T20:35:43.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.116+0000 7effa6bfb700 1 -- 172.21.15.89:0/921654760 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7effa00b1790 con 0x7effa015d230 2023-10-10T20:35:43.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.117+0000 7eff95ffb700 1 -- 172.21.15.89:0/921654760 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+15479 (secure 0 0 0) 0x7eff98054d30 con 0x7effa015d230 2023-10-10T20:35:43.120 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:43.120 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":38,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:35:40.398559+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:35:15.667057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-10T20:35:16.860131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6836/2135915759":"2023-10-11T20:35:37.387694+0000","172.21.15.89:6834/2135915759":"2023-10-11T20:35:37.387694+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:35:43.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.119+0000 7effa6bfb700 1 -- 172.21.15.89:0/921654760 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7eff84066f30 msgr2=0x7eff840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:43.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.119+0000 7effa6bfb700 1 --2- 172.21.15.89:0/921654760 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7eff84066f30 0x7eff840693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0099f0 tx=0x7eff9c018000 comp rx=0 tx=0).stop 2023-10-10T20:35:43.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.119+0000 7effa6bfb700 1 -- 172.21.15.89:0/921654760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effa015d230 msgr2=0x7effa00c9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:43.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.119+0000 7effa6bfb700 1 --2- 172.21.15.89:0/921654760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effa015d230 0x7effa00c9100 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7eff9800ea20 tx=0x7eff980091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.119+0000 7effa6bfb700 1 -- 172.21.15.89:0/921654760 shutdown_connections 2023-10-10T20:35:43.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.119+0000 7effa6bfb700 1 --2- 172.21.15.89:0/921654760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effa015d230 0x7effa00c9100 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.119+0000 7effa6bfb700 1 --2- 172.21.15.89:0/921654760 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7eff84066f30 0x7eff840693e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.119+0000 7effa6bfb700 1 --2- 172.21.15.89:0/921654760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effa0159420 0x7effa00cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.119+0000 7effa6bfb700 1 --2- 172.21.15.89:0/921654760 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7effa01597f0 0x7effa00c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.119+0000 7effa6bfb700 1 -- 172.21.15.89:0/921654760 >> 172.21.15.89:0/921654760 conn(0x7effa00d0660 msgr2=0x7effa00cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:43.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.119+0000 7effa6bfb700 1 -- 172.21.15.89:0/921654760 shutdown_connections 2023-10-10T20:35:43.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.119+0000 7effa6bfb700 1 -- 172.21.15.89:0/921654760 wait complete. 2023-10-10T20:35:43.133 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:35:43.134 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:35:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.246+0000 7fcbabfff700 1 Processor -- start 2023-10-10T20:35:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.246+0000 7fcbabfff700 1 -- start start 2023-10-10T20:35:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.247+0000 7fcbabfff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcbac159420 0x7fcbac0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.247+0000 7fcbabfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcbac1597f0 0x7fcbac0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.247+0000 7fcbabfff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbac0b11d0 con 0x7fcbac15d230 2023-10-10T20:35:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.247+0000 7fcbabfff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbac0b14b0 con 0x7fcbac1597f0 2023-10-10T20:35:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.247+0000 7fcbabfff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0b1790 con 0x7fcbac159420 2023-10-10T20:35:43.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.247+0000 7fcbaa7fc700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcbac1597f0 0x7fcbac0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:43.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.247+0000 7fcbab7fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fcbac15d230 0x7fcbac0b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:48324/0 (socket says 172.21.15.89:48324) 2023-10-10T20:35:43.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.247+0000 7fcbaa7fc700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcbac1597f0 0x7fcbac0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:38728/0 (socket says 172.21.15.89:38728) 2023-10-10T20:35:43.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.247+0000 7fcbab7fe700 1 -- 172.21.15.89:0/3956888318 learned_addr learned my addr 172.21.15.89:0/3956888318 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:43.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.247+0000 7fcbaaffd700 1 --2- 172.21.15.89:0/3956888318 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcbac159420 0x7fcbac0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:43.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.248+0000 7fcbaa7fc700 1 -- 172.21.15.89:0/3956888318 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcbac159420 msgr2=0x7fcbac0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:43.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.248+0000 7fcbaa7fc700 1 --2- 172.21.15.89:0/3956888318 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcbac159420 0x7fcbac0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.248+0000 7fcbaa7fc700 1 -- 172.21.15.89:0/3956888318 >> v1:172.21.15.111:6789/0 conn(0x7fcbac15d230 legacy=0x7fcbac0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:43.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.248+0000 7fcbaa7fc700 1 -- 172.21.15.89:0/3956888318 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac0b1a70 con 0x7fcbac1597f0 2023-10-10T20:35:43.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.248+0000 7fcbaa7fc700 1 --2- 172.21.15.89:0/3956888318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbac1597f0 0x7fcbac0b2270 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcb940098e0 tx=0x7fcb940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=219e6433c50ff72b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:43.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.248+0000 7fcba9ffb700 1 -- 172.21.15.89:0/3956888318 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400b640 con 0x7fcbac1597f0 2023-10-10T20:35:43.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.248+0000 7fcba9ffb700 1 -- 172.21.15.89:0/3956888318 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9400b7a0 con 0x7fcbac1597f0 2023-10-10T20:35:43.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.248+0000 7fcba9ffb700 1 -- 172.21.15.89:0/3956888318 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400bb40 con 0x7fcbac1597f0 2023-10-10T20:35:43.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.248+0000 7fcbabfff700 1 -- 172.21.15.89:0/3956888318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbac1597f0 msgr2=0x7fcbac0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:43.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.248+0000 7fcbabfff700 1 --2- 172.21.15.89:0/3956888318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbac1597f0 0x7fcbac0b2270 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcb940098e0 tx=0x7fcb940091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.248+0000 7fcbabfff700 1 -- 172.21.15.89:0/3956888318 shutdown_connections 2023-10-10T20:35:43.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.248+0000 7fcbabfff700 1 --2- 172.21.15.89:0/3956888318 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcbac159420 0x7fcbac0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.248+0000 7fcbabfff700 1 --2- 172.21.15.89:0/3956888318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbac1597f0 0x7fcbac0b2270 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.248+0000 7fcbabfff700 1 -- 172.21.15.89:0/3956888318 >> 172.21.15.89:0/3956888318 conn(0x7fcbac0d0660 msgr2=0x7fcbac0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:43.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.248+0000 7fcbabfff700 1 -- 172.21.15.89:0/3956888318 shutdown_connections 2023-10-10T20:35:43.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.248+0000 7fcbabfff700 1 -- 172.21.15.89:0/3956888318 wait complete. 2023-10-10T20:35:43.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.250+0000 7fcbabfff700 1 Processor -- start 2023-10-10T20:35:43.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.250+0000 7fcbabfff700 1 -- start start 2023-10-10T20:35:43.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.250+0000 7fcbabfff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcbac159420 0x7fcbac15e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:43.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.250+0000 7fcbabfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbac1597f0 0x7fcbac16a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:43.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.250+0000 7fcbabfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbac15d230 0x7fcbac1695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:43.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.250+0000 7fcbabfff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac16aca0 con 0x7fcbac15d230 2023-10-10T20:35:43.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.250+0000 7fcbabfff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac15e800 con 0x7fcbac1597f0 2023-10-10T20:35:43.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.250+0000 7fcbabfff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac15eae0 con 0x7fcbac159420 2023-10-10T20:35:43.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.251+0000 7fcbab7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbac15d230 0x7fcbac1695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:43.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.251+0000 7fcbab7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbac15d230 0x7fcbac1695c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58166/0 (socket says 172.21.15.89:58166) 2023-10-10T20:35:43.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.251+0000 7fcbab7fe700 1 -- 172.21.15.89:0/4161471945 learned_addr learned my addr 172.21.15.89:0/4161471945 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:43.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.251+0000 7fcbaaffd700 1 --2- 172.21.15.89:0/4161471945 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcbac159420 0x7fcbac15e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:43.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.251+0000 7fcbaa7fc700 1 --2- 172.21.15.89:0/4161471945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbac1597f0 0x7fcbac16a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:43.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.251+0000 7fcbab7fe700 1 -- 172.21.15.89:0/4161471945 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcbac159420 msgr2=0x7fcbac15e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:43.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.251+0000 7fcbab7fe700 1 --2- 172.21.15.89:0/4161471945 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcbac159420 0x7fcbac15e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.251+0000 7fcbab7fe700 1 -- 172.21.15.89:0/4161471945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbac1597f0 msgr2=0x7fcbac16a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:43.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.251+0000 7fcbab7fe700 1 --2- 172.21.15.89:0/4161471945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbac1597f0 0x7fcbac16a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.251+0000 7fcbab7fe700 1 -- 172.21.15.89:0/4161471945 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb94009420 con 0x7fcbac15d230 2023-10-10T20:35:43.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.251+0000 7fcbab7fe700 1 --2- 172.21.15.89:0/4161471945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbac15d230 0x7fcbac1695c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00c560 tx=0x7fcb9c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:43.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.251+0000 7fcba2ffd700 1 -- 172.21.15.89:0/4161471945 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c005b30 con 0x7fcbac15d230 2023-10-10T20:35:43.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.251+0000 7fcba2ffd700 1 -- 172.21.15.89:0/4161471945 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9c005c90 con 0x7fcbac15d230 2023-10-10T20:35:43.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.251+0000 7fcbabfff700 1 -- 172.21.15.89:0/4161471945 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbac15edc0 con 0x7fcbac15d230 2023-10-10T20:35:43.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.251+0000 7fcba2ffd700 1 -- 172.21.15.89:0/4161471945 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c0072e0 con 0x7fcbac15d230 2023-10-10T20:35:43.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.251+0000 7fcbabfff700 1 -- 172.21.15.89:0/4161471945 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbac15f380 con 0x7fcbac15d230 2023-10-10T20:35:43.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.252+0000 7fcbabfff700 1 -- 172.21.15.89:0/4161471945 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbac04c3c0 con 0x7fcbac15d230 2023-10-10T20:35:43.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.253+0000 7fcba2ffd700 1 -- 172.21.15.89:0/4161471945 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcb9c007690 con 0x7fcbac15d230 2023-10-10T20:35:43.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.253+0000 7fcba2ffd700 1 --2- 172.21.15.89:0/4161471945 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb8c066f30 0x7fcb8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:43.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.253+0000 7fcbaaffd700 1 --2- 172.21.15.89:0/4161471945 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb8c066f30 0x7fcb8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:43.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.253+0000 7fcba2ffd700 1 -- 172.21.15.89:0/4161471945 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fcb9c083800 con 0x7fcbac15d230 2023-10-10T20:35:43.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.253+0000 7fcbaaffd700 1 --2- 172.21.15.89:0/4161471945 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb8c066f30 0x7fcb8c0693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcba400f570 tx=0x7fcba400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:43.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.257+0000 7fcba2ffd700 1 -- 172.21.15.89:0/4161471945 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb9c050a40 con 0x7fcbac15d230 2023-10-10T20:35:43.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.484+0000 7fcbabfff700 1 -- 172.21.15.89:0/4161471945 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbac0b1d50 con 0x7fcbac15d230 2023-10-10T20:35:43.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.484+0000 7fcba2ffd700 1 -- 172.21.15.89:0/4161471945 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fcb9c017020 con 0x7fcbac15d230 2023-10-10T20:35:43.487 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:43.487 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1977066906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1977066906},{"type":"v1","addr":"172.21.15.111:6833","nonce":1977066906}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2904313060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2904313060},{"type":"v1","addr":"172.21.15.111:6835","nonce":2904313060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/4152616589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":4152616589},{"type":"v1","addr":"172.21.15.89:6836","nonce":4152616589}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:35:17.900533+0000","modified":"2023-10-10T20:35:21.533574+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4899":{"gid":4899,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1330347277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1330347277},{"type":"v1","addr":"172.21.15.89:6837","nonce":1330347277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-10T20:35:43.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.486+0000 7fcbabfff700 1 -- 172.21.15.89:0/4161471945 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb8c066f30 msgr2=0x7fcb8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.486+0000 7fcbabfff700 1 --2- 172.21.15.89:0/4161471945 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb8c066f30 0x7fcb8c0693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcba400f570 tx=0x7fcba400b040 comp rx=0 tx=0).stop 2023-10-10T20:35:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.486+0000 7fcbabfff700 1 -- 172.21.15.89:0/4161471945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbac15d230 msgr2=0x7fcbac1695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.486+0000 7fcbabfff700 1 --2- 172.21.15.89:0/4161471945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbac15d230 0x7fcbac1695c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00c560 tx=0x7fcb9c007f60 comp rx=0 tx=0).stop 2023-10-10T20:35:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.486+0000 7fcbabfff700 1 -- 172.21.15.89:0/4161471945 shutdown_connections 2023-10-10T20:35:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.486+0000 7fcbabfff700 1 --2- 172.21.15.89:0/4161471945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbac15d230 0x7fcbac1695c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.486+0000 7fcbabfff700 1 --2- 172.21.15.89:0/4161471945 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcbac159420 0x7fcbac15e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.486+0000 7fcbabfff700 1 --2- 172.21.15.89:0/4161471945 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb8c066f30 0x7fcb8c0693e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.486+0000 7fcbabfff700 1 --2- 172.21.15.89:0/4161471945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbac1597f0 0x7fcbac16a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.486+0000 7fcbabfff700 1 -- 172.21.15.89:0/4161471945 >> 172.21.15.89:0/4161471945 conn(0x7fcbac0d0660 msgr2=0x7fcbac0b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:43.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.486+0000 7fcbabfff700 1 -- 172.21.15.89:0/4161471945 shutdown_connections 2023-10-10T20:35:43.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.486+0000 7fcbabfff700 1 -- 172.21.15.89:0/4161471945 wait complete. 2023-10-10T20:35:43.491 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 22 2023-10-10T20:35:43.500 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:35:43.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.612+0000 7f3db2f78700 1 Processor -- start 2023-10-10T20:35:43.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.612+0000 7f3db2f78700 1 -- start start 2023-10-10T20:35:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3db2f78700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3dac159c10 0x7f3dac15b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3db2f78700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3dac0b5ea0 0x7f3dac068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3db2f78700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3dac068bb0 0x7f3dac15c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3db2f78700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dac0b2a40 con 0x7f3dac0b5ea0 2023-10-10T20:35:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3db2f78700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dac15c480 con 0x7f3dac159c10 2023-10-10T20:35:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3db2f78700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac15b5d0 con 0x7f3dac068bb0 2023-10-10T20:35:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3dabfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3dac0b5ea0 0x7f3dac068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3dabfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3dac0b5ea0 0x7f3dac068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58174/0 (socket says 172.21.15.89:58174) 2023-10-10T20:35:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3dabfff700 1 -- 172.21.15.89:0/2341395163 learned_addr learned my addr 172.21.15.89:0/2341395163 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3db0d14700 1 --2- 172.21.15.89:0/2341395163 >> v2:172.21.15.111:3300/0 conn(0x7f3dac159c10 0x7f3dac15b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3db1515700 1 --2- 172.21.15.89:0/2341395163 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3dac068bb0 0x7f3dac15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3dabfff700 1 -- 172.21.15.89:0/2341395163 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3dac068bb0 msgr2=0x7f3dac15c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3dabfff700 1 --2- 172.21.15.89:0/2341395163 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3dac068bb0 0x7f3dac15c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3dabfff700 1 -- 172.21.15.89:0/2341395163 >> v2:172.21.15.111:3300/0 conn(0x7f3dac159c10 msgr2=0x7f3dac15b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3dabfff700 1 --2- 172.21.15.89:0/2341395163 >> v2:172.21.15.111:3300/0 conn(0x7f3dac159c10 0x7f3dac15b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3dabfff700 1 -- 172.21.15.89:0/2341395163 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dac15b870 con 0x7f3dac0b5ea0 2023-10-10T20:35:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3dabfff700 1 --2- 172.21.15.89:0/2341395163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dac0b5ea0 0x7f3dac068670 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3dac099f10 tx=0x7f3d9c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78972a343674528 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3dab7fe700 1 -- 172.21.15.89:0/2341395163 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c00e070 con 0x7f3dac0b5ea0 2023-10-10T20:35:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3dab7fe700 1 -- 172.21.15.89:0/2341395163 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9c00ba50 con 0x7f3dac0b5ea0 2023-10-10T20:35:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.613+0000 7f3dab7fe700 1 -- 172.21.15.89:0/2341395163 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c0092e0 con 0x7f3dac0b5ea0 2023-10-10T20:35:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.614+0000 7f3db2f78700 1 -- 172.21.15.89:0/2341395163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dac0b5ea0 msgr2=0x7f3dac068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.614+0000 7f3db2f78700 1 --2- 172.21.15.89:0/2341395163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dac0b5ea0 0x7f3dac068670 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3dac099f10 tx=0x7f3d9c009840 comp rx=0 tx=0).stop 2023-10-10T20:35:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.614+0000 7f3db2f78700 1 -- 172.21.15.89:0/2341395163 shutdown_connections 2023-10-10T20:35:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.614+0000 7f3db2f78700 1 --2- 172.21.15.89:0/2341395163 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3dac068bb0 0x7f3dac15c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.614+0000 7f3db2f78700 1 --2- 172.21.15.89:0/2341395163 >> v2:172.21.15.111:3300/0 conn(0x7f3dac159c10 0x7f3dac15b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.614+0000 7f3db2f78700 1 --2- 172.21.15.89:0/2341395163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dac0b5ea0 0x7f3dac068670 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.614+0000 7f3db2f78700 1 -- 172.21.15.89:0/2341395163 >> 172.21.15.89:0/2341395163 conn(0x7f3dac099a20 msgr2=0x7f3dac0af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:43.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.614+0000 7f3db2f78700 1 -- 172.21.15.89:0/2341395163 shutdown_connections 2023-10-10T20:35:43.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.614+0000 7f3db2f78700 1 -- 172.21.15.89:0/2341395163 wait complete. 2023-10-10T20:35:43.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.615+0000 7f3db2f78700 1 Processor -- start 2023-10-10T20:35:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.615+0000 7f3db2f78700 1 -- start start 2023-10-10T20:35:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.616+0000 7f3db2f78700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3dac068bb0 0x7f3dac169710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.616+0000 7f3db2f78700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3dac0b5ea0 0x7f3dac0cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.616+0000 7f3db2f78700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dac159c10 0x7f3dac0cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.616+0000 7f3db2f78700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac0b1750 con 0x7f3dac159c10 2023-10-10T20:35:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.616+0000 7f3db2f78700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac0c6130 con 0x7f3dac0b5ea0 2023-10-10T20:35:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.616+0000 7f3db2f78700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac0c6410 con 0x7f3dac068bb0 2023-10-10T20:35:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.616+0000 7f3db1515700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dac159c10 0x7f3dac0cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.616+0000 7f3db1515700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dac159c10 0x7f3dac0cffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58180/0 (socket says 172.21.15.89:58180) 2023-10-10T20:35:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.616+0000 7f3db1515700 1 -- 172.21.15.89:0/2700871673 learned_addr learned my addr 172.21.15.89:0/2700871673 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.616+0000 7f3dabfff700 1 --2- 172.21.15.89:0/2700871673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3dac0b5ea0 0x7f3dac0cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.616+0000 7f3db0d14700 1 --2- 172.21.15.89:0/2700871673 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3dac068bb0 0x7f3dac169710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.616+0000 7f3db1515700 1 -- 172.21.15.89:0/2700871673 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3dac068bb0 msgr2=0x7f3dac169710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.616+0000 7f3db1515700 1 --2- 172.21.15.89:0/2700871673 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3dac068bb0 0x7f3dac169710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.616+0000 7f3db1515700 1 -- 172.21.15.89:0/2700871673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3dac0b5ea0 msgr2=0x7f3dac0cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.616+0000 7f3db1515700 1 --2- 172.21.15.89:0/2700871673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3dac0b5ea0 0x7f3dac0cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.616+0000 7f3db1515700 1 -- 172.21.15.89:0/2700871673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c009ae0 con 0x7f3dac159c10 2023-10-10T20:35:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.617+0000 7f3db1515700 1 --2- 172.21.15.89:0/2700871673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dac159c10 0x7f3dac0cffe0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400d890 tx=0x7f3d94008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.617+0000 7f3da8ff9700 1 -- 172.21.15.89:0/2700871673 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d940125f0 con 0x7f3dac159c10 2023-10-10T20:35:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.617+0000 7f3da8ff9700 1 -- 172.21.15.89:0/2700871673 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d94012770 con 0x7f3dac159c10 2023-10-10T20:35:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.617+0000 7f3db2f78700 1 -- 172.21.15.89:0/2700871673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dac0c66f0 con 0x7f3dac159c10 2023-10-10T20:35:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.617+0000 7f3da8ff9700 1 -- 172.21.15.89:0/2700871673 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d94012b30 con 0x7f3dac159c10 2023-10-10T20:35:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.617+0000 7f3db2f78700 1 -- 172.21.15.89:0/2700871673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dac169c50 con 0x7f3dac159c10 2023-10-10T20:35:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.618+0000 7f3db2f78700 1 -- 172.21.15.89:0/2700871673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dac169f30 con 0x7f3dac159c10 2023-10-10T20:35:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.618+0000 7f3da8ff9700 1 -- 172.21.15.89:0/2700871673 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d9401e460 con 0x7f3dac159c10 2023-10-10T20:35:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.618+0000 7f3da8ff9700 1 --2- 172.21.15.89:0/2700871673 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3d8c066f30 0x7f3d8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.618+0000 7f3da8ff9700 1 -- 172.21.15.89:0/2700871673 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f3d94084b70 con 0x7f3dac159c10 2023-10-10T20:35:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.618+0000 7f3db0d14700 1 --2- 172.21.15.89:0/2700871673 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3d8c066f30 0x7f3d8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.619+0000 7f3db0d14700 1 --2- 172.21.15.89:0/2700871673 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3d8c066f30 0x7f3d8c0693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3da4006fd0 tx=0x7f3da4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:43.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.622+0000 7f3da8ff9700 1 -- 172.21.15.89:0/2700871673 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3dac169f30 con 0x7f3dac159c10 2023-10-10T20:35:43.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.848+0000 7f3db2f78700 1 -- 172.21.15.89:0/2700871673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3dac169f30 con 0x7f3dac159c10 2023-10-10T20:35:43.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.848+0000 7f3da8ff9700 1 -- 172.21.15.89:0/2700871673 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f3dac169f30 con 0x7f3dac159c10 2023-10-10T20:35:43.851 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:43.851 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1977066906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1977066906},{"type":"v1","addr":"172.21.15.111:6833","nonce":1977066906}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2904313060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2904313060},{"type":"v1","addr":"172.21.15.111:6835","nonce":2904313060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/4152616589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":4152616589},{"type":"v1","addr":"172.21.15.89:6836","nonce":4152616589}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:35:17.900533+0000","modified":"2023-10-10T20:35:21.533574+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4899":{"gid":4899,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1330347277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1330347277},{"type":"v1","addr":"172.21.15.89:6837","nonce":1330347277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-10T20:35:43.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.850+0000 7f3db2f78700 1 -- 172.21.15.89:0/2700871673 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3d8c066f30 msgr2=0x7f3d8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:43.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.850+0000 7f3db2f78700 1 --2- 172.21.15.89:0/2700871673 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3d8c066f30 0x7f3d8c0693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3da4006fd0 tx=0x7f3da4008040 comp rx=0 tx=0).stop 2023-10-10T20:35:43.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.850+0000 7f3db2f78700 1 -- 172.21.15.89:0/2700871673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dac159c10 msgr2=0x7f3dac0cffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:43.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.850+0000 7f3db2f78700 1 --2- 172.21.15.89:0/2700871673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dac159c10 0x7f3dac0cffe0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400d890 tx=0x7f3d94008040 comp rx=0 tx=0).stop 2023-10-10T20:35:43.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.850+0000 7f3db2f78700 1 -- 172.21.15.89:0/2700871673 shutdown_connections 2023-10-10T20:35:43.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.850+0000 7f3db2f78700 1 --2- 172.21.15.89:0/2700871673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dac159c10 0x7f3dac0cffe0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.850+0000 7f3db2f78700 1 --2- 172.21.15.89:0/2700871673 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3dac068bb0 0x7f3dac169710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.850+0000 7f3db2f78700 1 --2- 172.21.15.89:0/2700871673 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3d8c066f30 0x7f3d8c0693e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.850+0000 7f3db2f78700 1 --2- 172.21.15.89:0/2700871673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3dac0b5ea0 0x7f3dac0cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.850+0000 7f3db2f78700 1 -- 172.21.15.89:0/2700871673 >> 172.21.15.89:0/2700871673 conn(0x7f3dac099a20 msgr2=0x7f3dac09a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:43.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.851+0000 7f3db2f78700 1 -- 172.21.15.89:0/2700871673 shutdown_connections 2023-10-10T20:35:43.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.851+0000 7f3db2f78700 1 -- 172.21.15.89:0/2700871673 wait complete. 2023-10-10T20:35:43.855 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 22 2023-10-10T20:35:43.864 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:35:43.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.977+0000 7f9adca38700 1 Processor -- start 2023-10-10T20:35:43.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.977+0000 7f9adca38700 1 -- start start 2023-10-10T20:35:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.978+0000 7f9adca38700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9ad81597f0 0x7f9ad80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.978+0000 7f9adca38700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9ad815d230 0x7f9ad815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.978+0000 7f9adca38700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad815db70 con 0x7f9ad8159420 2023-10-10T20:35:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.978+0000 7f9adca38700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad80b11d0 con 0x7f9ad815d230 2023-10-10T20:35:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.978+0000 7f9adca38700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad80b14b0 con 0x7f9ad81597f0 2023-10-10T20:35:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.978+0000 7f9ad759e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9ad815d230 0x7f9ad815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.978+0000 7f9ad759e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9ad815d230 0x7f9ad815d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58184/0 (socket says 172.21.15.89:58184) 2023-10-10T20:35:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.978+0000 7f9ad759e700 1 -- 172.21.15.89:0/3285228045 learned_addr learned my addr 172.21.15.89:0/3285228045 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.978+0000 7f9ace59c700 1 --2- 172.21.15.89:0/3285228045 >> v2:172.21.15.111:3300/0 conn(0x7f9ad81597f0 0x7f9ad80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.978+0000 7f9ad759e700 1 -- 172.21.15.89:0/3285228045 >> v2:172.21.15.111:3300/0 conn(0x7f9ad81597f0 msgr2=0x7f9ad80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.978+0000 7f9ad759e700 1 --2- 172.21.15.89:0/3285228045 >> v2:172.21.15.111:3300/0 conn(0x7f9ad81597f0 0x7f9ad80b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.978+0000 7f9ad759e700 1 -- 172.21.15.89:0/3285228045 >> v1:172.21.15.111:6789/0 conn(0x7f9ad8159420 legacy=0x7f9ad80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.978+0000 7f9ad759e700 1 -- 172.21.15.89:0/3285228045 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad80b1790 con 0x7f9ad815d230 2023-10-10T20:35:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.978+0000 7f9ad759e700 1 --2- 172.21.15.89:0/3285228045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad815d230 0x7f9ad815d630 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8089750 tx=0x7f9ad0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666aa9192f5e5d86 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.978+0000 7f9ad659c700 1 -- 172.21.15.89:0/3285228045 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad000e070 con 0x7f9ad815d230 2023-10-10T20:35:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.979+0000 7f9ad659c700 1 -- 172.21.15.89:0/3285228045 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad000ba20 con 0x7f9ad815d230 2023-10-10T20:35:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.979+0000 7f9ad659c700 1 -- 172.21.15.89:0/3285228045 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad00092e0 con 0x7f9ad815d230 2023-10-10T20:35:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.979+0000 7f9adca38700 1 -- 172.21.15.89:0/3285228045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad815d230 msgr2=0x7f9ad815d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.979+0000 7f9adca38700 1 --2- 172.21.15.89:0/3285228045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad815d230 0x7f9ad815d630 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8089750 tx=0x7f9ad0009840 comp rx=0 tx=0).stop 2023-10-10T20:35:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.979+0000 7f9adca38700 1 -- 172.21.15.89:0/3285228045 shutdown_connections 2023-10-10T20:35:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.979+0000 7f9adca38700 1 --2- 172.21.15.89:0/3285228045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad815d230 0x7f9ad815d630 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.979+0000 7f9adca38700 1 --2- 172.21.15.89:0/3285228045 >> v2:172.21.15.111:3300/0 conn(0x7f9ad81597f0 0x7f9ad80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.979+0000 7f9adca38700 1 -- 172.21.15.89:0/3285228045 >> 172.21.15.89:0/3285228045 conn(0x7f9ad80d0660 msgr2=0x7f9ad80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.979+0000 7f9adca38700 1 -- 172.21.15.89:0/3285228045 shutdown_connections 2023-10-10T20:35:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.979+0000 7f9adca38700 1 -- 172.21.15.89:0/3285228045 wait complete. 2023-10-10T20:35:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.980+0000 7f9adca38700 1 Processor -- start 2023-10-10T20:35:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.980+0000 7f9adca38700 1 -- start start 2023-10-10T20:35:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.981+0000 7f9adca38700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ad8159420 0x7f9ad815c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.981+0000 7f9adca38700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ad81597f0 0x7f9ad815a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.981+0000 7f9adca38700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad815d230 0x7f9ad8153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.981+0000 7f9adca38700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad815b650 con 0x7f9ad815d230 2023-10-10T20:35:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.981+0000 7f9adca38700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad815b090 con 0x7f9ad8159420 2023-10-10T20:35:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.981+0000 7f9adca38700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad815b370 con 0x7f9ad81597f0 2023-10-10T20:35:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.981+0000 7f9ad759e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad815d230 0x7f9ad8153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.981+0000 7f9ad759e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad815d230 0x7f9ad8153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58190/0 (socket says 172.21.15.89:58190) 2023-10-10T20:35:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.981+0000 7f9ad759e700 1 -- 172.21.15.89:0/342670001 learned_addr learned my addr 172.21.15.89:0/342670001 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.981+0000 7f9ad6d9d700 1 --2- 172.21.15.89:0/342670001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ad8159420 0x7f9ad815c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.981+0000 7f9ad759e700 1 -- 172.21.15.89:0/342670001 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ad81597f0 msgr2=0x7f9ad815a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.981+0000 7f9ace59c700 1 --2- 172.21.15.89:0/342670001 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ad81597f0 0x7f9ad815a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.981+0000 7f9ad759e700 1 --2- 172.21.15.89:0/342670001 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ad81597f0 0x7f9ad815a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.981+0000 7f9ad759e700 1 -- 172.21.15.89:0/342670001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ad8159420 msgr2=0x7f9ad815c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.981+0000 7f9ad759e700 1 --2- 172.21.15.89:0/342670001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ad8159420 0x7f9ad815c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.981+0000 7f9ad759e700 1 -- 172.21.15.89:0/342670001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad0009ae0 con 0x7f9ad815d230 2023-10-10T20:35:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.981+0000 7f9ad6d9d700 1 --2- 172.21.15.89:0/342670001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ad8159420 0x7f9ad815c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.982+0000 7f9ad759e700 1 --2- 172.21.15.89:0/342670001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad815d230 0x7f9ad8153e40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00072f0 tx=0x7f9ad0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.982+0000 7f9acf7fe700 1 -- 172.21.15.89:0/342670001 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0008b90 con 0x7f9ad815d230 2023-10-10T20:35:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.982+0000 7f9acf7fe700 1 -- 172.21.15.89:0/342670001 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad0008cf0 con 0x7f9ad815d230 2023-10-10T20:35:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.982+0000 7f9adca38700 1 -- 172.21.15.89:0/342670001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad80c55a0 con 0x7f9ad815d230 2023-10-10T20:35:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.982+0000 7f9acf7fe700 1 -- 172.21.15.89:0/342670001 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0025690 con 0x7f9ad815d230 2023-10-10T20:35:43.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.982+0000 7f9adca38700 1 -- 172.21.15.89:0/342670001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad80c7430 con 0x7f9ad815d230 2023-10-10T20:35:43.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.983+0000 7f9adca38700 1 -- 172.21.15.89:0/342670001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad80c7620 con 0x7f9ad815d230 2023-10-10T20:35:43.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.983+0000 7f9acf7fe700 1 -- 172.21.15.89:0/342670001 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ad0003040 con 0x7f9ad815d230 2023-10-10T20:35:43.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.984+0000 7f9acf7fe700 1 --2- 172.21.15.89:0/342670001 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ab8066f30 0x7f9ab80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:43.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.984+0000 7f9acf7fe700 1 -- 172.21.15.89:0/342670001 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f9ad00862f0 con 0x7f9ad815d230 2023-10-10T20:35:43.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.984+0000 7f9ad6d9d700 1 --2- 172.21.15.89:0/342670001 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ab8066f30 0x7f9ab80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:43.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.984+0000 7f9ad6d9d700 1 --2- 172.21.15.89:0/342670001 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ab8066f30 0x7f9ab80693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8001840 tx=0x7f9ac8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:43.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:43.988+0000 7f9acf7fe700 1 -- 172.21.15.89:0/342670001 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ad80c7620 con 0x7f9ad815d230 2023-10-10T20:35:44.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.172+0000 7f9adca38700 1 -- 172.21.15.89:0/342670001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ad80c7620 con 0x7f9ad815d230 2023-10-10T20:35:44.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.173+0000 7f9acf7fe700 1 -- 172.21.15.89:0/342670001 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+15479 (secure 0 0 0) 0x7f9ad80c7620 con 0x7f9ad815d230 2023-10-10T20:35:44.175 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:44.175 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":38,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:35:40.398559+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:35:15.667057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-10T20:35:16.860131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6836/2135915759":"2023-10-11T20:35:37.387694+0000","172.21.15.89:6834/2135915759":"2023-10-11T20:35:37.387694+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:35:44.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.175+0000 7f9adca38700 1 -- 172.21.15.89:0/342670001 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ab8066f30 msgr2=0x7f9ab80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:44.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.175+0000 7f9adca38700 1 --2- 172.21.15.89:0/342670001 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ab8066f30 0x7f9ab80693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8001840 tx=0x7f9ac8009040 comp rx=0 tx=0).stop 2023-10-10T20:35:44.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.175+0000 7f9adca38700 1 -- 172.21.15.89:0/342670001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad815d230 msgr2=0x7f9ad8153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:44.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.175+0000 7f9adca38700 1 --2- 172.21.15.89:0/342670001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad815d230 0x7f9ad8153e40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00072f0 tx=0x7f9ad0018000 comp rx=0 tx=0).stop 2023-10-10T20:35:44.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.175+0000 7f9adca38700 1 -- 172.21.15.89:0/342670001 shutdown_connections 2023-10-10T20:35:44.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.175+0000 7f9adca38700 1 --2- 172.21.15.89:0/342670001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad815d230 0x7f9ad8153e40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:44.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.175+0000 7f9adca38700 1 --2- 172.21.15.89:0/342670001 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ab8066f30 0x7f9ab80693e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:44.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.175+0000 7f9adca38700 1 --2- 172.21.15.89:0/342670001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ad8159420 0x7f9ad815c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:44.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.175+0000 7f9adca38700 1 --2- 172.21.15.89:0/342670001 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ad81597f0 0x7f9ad815a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:44.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.175+0000 7f9adca38700 1 -- 172.21.15.89:0/342670001 >> 172.21.15.89:0/342670001 conn(0x7f9ad80d0660 msgr2=0x7f9ad80b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:44.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.175+0000 7f9adca38700 1 -- 172.21.15.89:0/342670001 shutdown_connections 2023-10-10T20:35:44.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.175+0000 7f9adca38700 1 -- 172.21.15.89:0/342670001 wait complete. 2023-10-10T20:35:44.189 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:35:44.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.302+0000 7f7bc6a15700 1 Processor -- start 2023-10-10T20:35:44.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.302+0000 7f7bc6a15700 1 -- start start 2023-10-10T20:35:44.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc6a15700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bc015d230 0x7f7bc015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:44.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc6a15700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc00b2980 con 0x7f7bc0159420 2023-10-10T20:35:44.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc6a15700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc015dbb0 con 0x7f7bc01597f0 2023-10-10T20:35:44.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc6a15700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc00b11d0 con 0x7f7bc015d230 2023-10-10T20:35:44.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc5a13700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7bc0159420 0x7f7bc00b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45488/0 (socket says 172.21.15.89:45488) 2023-10-10T20:35:44.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc5a13700 1 -- 172.21.15.89:0/4059789388 learned_addr learned my addr 172.21.15.89:0/4059789388 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:44.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc6214700 1 --2- 172.21.15.89:0/4059789388 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bc015d230 0x7f7bc015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:44.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc4a11700 1 -- 172.21.15.89:0/4059789388 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1147873690 0 0) 0x7f7bc00b2980 con 0x7f7bc0159420 2023-10-10T20:35:44.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc4a11700 1 -- 172.21.15.89:0/4059789388 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ba8003570 con 0x7f7bc0159420 2023-10-10T20:35:44.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc4a11700 1 -- 172.21.15.89:0/4059789388 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487168415 0 0) 0x7f7bc015dbb0 con 0x7f7bc01597f0 2023-10-10T20:35:44.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc4a11700 1 -- 172.21.15.89:0/4059789388 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bc00b2980 con 0x7f7bc01597f0 2023-10-10T20:35:44.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc4a11700 1 -- 172.21.15.89:0/4059789388 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 407280251 0 0) 0x7f7ba8003570 con 0x7f7bc0159420 2023-10-10T20:35:44.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc4a11700 1 -- 172.21.15.89:0/4059789388 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bc015d230 msgr2=0x7f7bc015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:44.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc4a11700 1 --2- 172.21.15.89:0/4059789388 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bc015d230 0x7f7bc015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:44.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc4a11700 1 -- 172.21.15.89:0/4059789388 >> v1:172.21.15.111:6789/0 conn(0x7f7bc01597f0 legacy=0x7f7bc00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:44.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc4a11700 1 -- 172.21.15.89:0/4059789388 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc00b14b0 con 0x7f7bc0159420 2023-10-10T20:35:44.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc4a11700 1 -- 172.21.15.89:0/4059789388 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f7bb8001470 con 0x7f7bc0159420 2023-10-10T20:35:44.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc4a11700 1 -- 172.21.15.89:0/4059789388 >> v1:172.21.15.89:6789/0 conn(0x7f7bc0159420 legacy=0x7f7bc00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:44.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc4a11700 1 --2- 172.21.15.89:0/4059789388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ba8003c20 0x7f7ba8004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:44.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.303+0000 7f7bc4a11700 1 --2- 172.21.15.89:0/4059789388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ba8004810 0x7f7ba8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:44.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.304+0000 7f7bc4a11700 1 --2- 172.21.15.89:0/4059789388 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bc015d230 0x7f7ba80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:44.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.304+0000 7f7bc4a11700 1 -- 172.21.15.89:0/4059789388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc00b11d0 con 0x7f7ba8004810 2023-10-10T20:35:44.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.304+0000 7f7bc4a11700 1 -- 172.21.15.89:0/4059789388 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba800bb40 con 0x7f7ba8003c20 2023-10-10T20:35:44.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.304+0000 7f7bc4a11700 1 -- 172.21.15.89:0/4059789388 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba800bd00 con 0x7f7bc015d230 2023-10-10T20:35:44.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.304+0000 7f7bc5212700 1 --2- 172.21.15.89:0/4059789388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ba8004810 0x7f7ba8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:44.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.304+0000 7f7bc5a13700 1 --2- 172.21.15.89:0/4059789388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ba8003c20 0x7f7ba8004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:44.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.304+0000 7f7bc6214700 1 --2- 172.21.15.89:0/4059789388 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bc015d230 0x7f7ba80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:44.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.304+0000 7f7bc5212700 1 -- 172.21.15.89:0/4059789388 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bc015d230 msgr2=0x7f7ba80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:44.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.304+0000 7f7bc5212700 1 --2- 172.21.15.89:0/4059789388 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bc015d230 0x7f7ba80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:44.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.304+0000 7f7bc5212700 1 -- 172.21.15.89:0/4059789388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ba8003c20 msgr2=0x7f7ba8004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:44.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.304+0000 7f7bc5212700 1 --2- 172.21.15.89:0/4059789388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ba8003c20 0x7f7ba8004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:44.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.304+0000 7f7bc5212700 1 -- 172.21.15.89:0/4059789388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba8003570 con 0x7f7ba8004810 2023-10-10T20:35:44.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.304+0000 7f7bc5212700 1 --2- 172.21.15.89:0/4059789388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ba8004810 0x7f7ba8004c80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4001a20 tx=0x7f7bb4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49713a0e2b7143fa server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:44.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.304+0000 7f7bc4a11700 1 -- 172.21.15.89:0/4059789388 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb400c5d0 con 0x7f7ba8004810 2023-10-10T20:35:44.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.304+0000 7f7bc4a11700 1 -- 172.21.15.89:0/4059789388 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb400c730 con 0x7f7ba8004810 2023-10-10T20:35:44.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.304+0000 7f7bc4a11700 1 -- 172.21.15.89:0/4059789388 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb400cad0 con 0x7f7ba8004810 2023-10-10T20:35:44.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.305+0000 7f7bc6a15700 1 -- 172.21.15.89:0/4059789388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ba8004810 msgr2=0x7f7ba8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:44.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.305+0000 7f7bc6a15700 1 --2- 172.21.15.89:0/4059789388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ba8004810 0x7f7ba8004c80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4001a20 tx=0x7f7bb4007d30 comp rx=0 tx=0).stop 2023-10-10T20:35:44.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.305+0000 7f7bc6214700 1 -- 172.21.15.89:0/4059789388 reap_dead start 2023-10-10T20:35:44.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.305+0000 7f7bc6a15700 1 -- 172.21.15.89:0/4059789388 shutdown_connections 2023-10-10T20:35:44.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.305+0000 7f7bc6a15700 1 -- 172.21.15.89:0/4059789388 >> 172.21.15.89:0/4059789388 conn(0x7f7bc00d0660 msgr2=0x7f7bc00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:44.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.305+0000 7f7bc6a15700 1 -- 172.21.15.89:0/4059789388 shutdown_connections 2023-10-10T20:35:44.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.305+0000 7f7bc6a15700 1 -- 172.21.15.89:0/4059789388 wait complete. 2023-10-10T20:35:44.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.306+0000 7f7bc6a15700 1 Processor -- start 2023-10-10T20:35:44.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.306+0000 7f7bc6a15700 1 -- start start 2023-10-10T20:35:44.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bc6a15700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ba8004810 0x7f7bc00c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:44.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bc6a15700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bc016cfa0 0x7f7bc01550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:44.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bc6a15700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bc01555f0 0x7f7bc01699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:44.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bc6a15700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc016d760 con 0x7f7bc01555f0 2023-10-10T20:35:44.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bc6a15700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc016da40 con 0x7f7ba8004810 2023-10-10T20:35:44.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bc6a15700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc016dd20 con 0x7f7bc016cfa0 2023-10-10T20:35:44.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bc6214700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bc01555f0 0x7f7bc01699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:44.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bc6214700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bc01555f0 0x7f7bc01699d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58204/0 (socket says 172.21.15.89:58204) 2023-10-10T20:35:44.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bc6214700 1 -- 172.21.15.89:0/630815979 learned_addr learned my addr 172.21.15.89:0/630815979 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:44.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bc5a13700 1 --2- 172.21.15.89:0/630815979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ba8004810 0x7f7bc00c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:44.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bc5212700 1 --2- 172.21.15.89:0/630815979 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bc016cfa0 0x7f7bc01550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:44.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bc6214700 1 -- 172.21.15.89:0/630815979 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bc016cfa0 msgr2=0x7f7bc01550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:44.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bc6214700 1 --2- 172.21.15.89:0/630815979 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bc016cfa0 0x7f7bc01550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:44.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bc6214700 1 -- 172.21.15.89:0/630815979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ba8004810 msgr2=0x7f7bc00c6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:44.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bc6214700 1 --2- 172.21.15.89:0/630815979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ba8004810 0x7f7bc00c6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:44.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bc6214700 1 -- 172.21.15.89:0/630815979 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc015bab0 con 0x7f7bc01555f0 2023-10-10T20:35:44.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bc5212700 1 --2- 172.21.15.89:0/630815979 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bc016cfa0 0x7f7bc01550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:44.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bc6214700 1 --2- 172.21.15.89:0/630815979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bc01555f0 0x7f7bc01699d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7bc015e340 tx=0x7f7bb000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:44.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bbdffb700 1 -- 172.21.15.89:0/630815979 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb000e040 con 0x7f7bc01555f0 2023-10-10T20:35:44.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bbdffb700 1 -- 172.21.15.89:0/630815979 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb000f040 con 0x7f7bc01555f0 2023-10-10T20:35:44.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bc6a15700 1 -- 172.21.15.89:0/630815979 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc015bd90 con 0x7f7bc01555f0 2023-10-10T20:35:44.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.307+0000 7f7bbdffb700 1 -- 172.21.15.89:0/630815979 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb000fe20 con 0x7f7bc01555f0 2023-10-10T20:35:44.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.308+0000 7f7bc6a15700 1 -- 172.21.15.89:0/630815979 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc0169f10 con 0x7f7bc01555f0 2023-10-10T20:35:44.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.308+0000 7f7bc6a15700 1 -- 172.21.15.89:0/630815979 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc016a1f0 con 0x7f7bc01555f0 2023-10-10T20:35:44.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.309+0000 7f7bbdffb700 1 -- 172.21.15.89:0/630815979 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7bb0015640 con 0x7f7bc01555f0 2023-10-10T20:35:44.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.309+0000 7f7bbdffb700 1 --2- 172.21.15.89:0/630815979 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7bac066f30 0x7f7bac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:44.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.309+0000 7f7bbdffb700 1 -- 172.21.15.89:0/630815979 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f7bb0084de0 con 0x7f7bc01555f0 2023-10-10T20:35:44.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.309+0000 7f7bc5a13700 1 --2- 172.21.15.89:0/630815979 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7bac066f30 0x7f7bac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:44.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.310+0000 7f7bc5a13700 1 --2- 172.21.15.89:0/630815979 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7bac066f30 0x7f7bac0693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8007ef0 tx=0x7f7bb8001df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:44.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.313+0000 7f7bbdffb700 1 -- 172.21.15.89:0/630815979 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7bc016a1f0 con 0x7f7bc01555f0 2023-10-10T20:35:44.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.539+0000 7f7bc6a15700 1 -- 172.21.15.89:0/630815979 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7bc016a1f0 con 0x7f7bc01555f0 2023-10-10T20:35:44.541 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-10-10T20:35:44.539+0000 7fd0184ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:35:44.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.543+0000 7f7bbdffb700 1 -- 172.21.15.89:0/630815979 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7bc016a1f0 con 0x7f7bc01555f0 2023-10-10T20:35:44.546 INFO:tasks.ceph.mds.c.smithi089.stderr: -1205> 2023-10-10T20:35:22.488+0000 7fb4ee9f8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:35:44.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.545+0000 7f7bc6a15700 1 -- 172.21.15.89:0/630815979 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7bac066f30 msgr2=0x7f7bac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:44.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.545+0000 7f7bc6a15700 1 --2- 172.21.15.89:0/630815979 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7bac066f30 0x7f7bac0693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8007ef0 tx=0x7f7bb8001df0 comp rx=0 tx=0).stop 2023-10-10T20:35:44.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.545+0000 7f7bc6a15700 1 -- 172.21.15.89:0/630815979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bc01555f0 msgr2=0x7f7bc01699d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:44.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.545+0000 7f7bc6a15700 1 --2- 172.21.15.89:0/630815979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bc01555f0 0x7f7bc01699d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7bc015e340 tx=0x7f7bb000b040 comp rx=0 tx=0).stop 2023-10-10T20:35:44.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.545+0000 7f7bc6a15700 1 -- 172.21.15.89:0/630815979 shutdown_connections 2023-10-10T20:35:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.545+0000 7f7bc6a15700 1 --2- 172.21.15.89:0/630815979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bc01555f0 0x7f7bc01699d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.545+0000 7f7bc6a15700 1 --2- 172.21.15.89:0/630815979 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7bac066f30 0x7f7bac0693e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.545+0000 7f7bc6a15700 1 --2- 172.21.15.89:0/630815979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ba8004810 0x7f7bc00c6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.545+0000 7f7bc6a15700 1 --2- 172.21.15.89:0/630815979 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bc016cfa0 0x7f7bc01550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.545+0000 7f7bc6a15700 1 -- 172.21.15.89:0/630815979 >> 172.21.15.89:0/630815979 conn(0x7f7bc00d0660 msgr2=0x7f7bc0155ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:44.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.545+0000 7f7bc6a15700 1 -- 172.21.15.89:0/630815979 shutdown_connections 2023-10-10T20:35:44.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.545+0000 7f7bc6a15700 1 -- 172.21.15.89:0/630815979 wait complete. 2023-10-10T20:35:44.550 INFO:teuthology.orchestra.run.smithi089.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:35:44.561 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:35:44.574 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-10T20:35:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.676+0000 7f92196f0700 1 Processor -- start 2023-10-10T20:35:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.676+0000 7f92196f0700 1 -- start start 2023-10-10T20:35:44.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.677+0000 7f92196f0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9214159890 0x7f921415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:44.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.677+0000 7f92196f0700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9214069560 0x7f92140699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:44.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.677+0000 7f92196f0700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9214069ee0 con 0x7f9214159c60 2023-10-10T20:35:44.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.677+0000 7f92196f0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f921415a160 con 0x7f9214159890 2023-10-10T20:35:44.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.677+0000 7f92196f0700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92140d13a0 con 0x7f9214069560 2023-10-10T20:35:44.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.677+0000 7f9212ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9214159890 0x7f921415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:44.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.677+0000 7f9212ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9214159890 0x7f921415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58230/0 (socket says 172.21.15.89:58230) 2023-10-10T20:35:44.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.677+0000 7f9212ffd700 1 -- 172.21.15.89:0/4005127221 learned_addr learned my addr 172.21.15.89:0/4005127221 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:44.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.677+0000 7f9212ffd700 1 -- 172.21.15.89:0/4005127221 >> v2:172.21.15.111:3300/0 conn(0x7f9214069560 msgr2=0x7f92140699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:44.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.677+0000 7f9212ffd700 1 --2- 172.21.15.89:0/4005127221 >> v2:172.21.15.111:3300/0 conn(0x7f9214069560 0x7f92140699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:44.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.677+0000 7f9212ffd700 1 -- 172.21.15.89:0/4005127221 >> v1:172.21.15.111:6789/0 conn(0x7f9214159c60 legacy=0x7f921415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:44.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.677+0000 7f9212ffd700 1 -- 172.21.15.89:0/4005127221 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921415e140 con 0x7f9214159890 2023-10-10T20:35:44.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.677+0000 7f9212ffd700 1 --2- 172.21.15.89:0/4005127221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9214159890 0x7f921415abd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f921406a890 tx=0x7f9204009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a76c0496ace95c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:44.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.677+0000 7f92127fc700 1 -- 172.21.15.89:0/4005127221 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920400b820 con 0x7f9214159890 2023-10-10T20:35:44.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.677+0000 7f92127fc700 1 -- 172.21.15.89:0/4005127221 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920400b980 con 0x7f9214159890 2023-10-10T20:35:44.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.677+0000 7f92127fc700 1 -- 172.21.15.89:0/4005127221 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920400bd20 con 0x7f9214159890 2023-10-10T20:35:44.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.678+0000 7f92196f0700 1 -- 172.21.15.89:0/4005127221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9214159890 msgr2=0x7f921415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:44.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.678+0000 7f92196f0700 1 --2- 172.21.15.89:0/4005127221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9214159890 0x7f921415abd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f921406a890 tx=0x7f9204009250 comp rx=0 tx=0).stop 2023-10-10T20:35:44.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.678+0000 7f92196f0700 1 -- 172.21.15.89:0/4005127221 shutdown_connections 2023-10-10T20:35:44.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.678+0000 7f92196f0700 1 --2- 172.21.15.89:0/4005127221 >> v2:172.21.15.111:3300/0 conn(0x7f9214069560 0x7f92140699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:44.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.678+0000 7f92196f0700 1 --2- 172.21.15.89:0/4005127221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9214159890 0x7f921415abd0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:44.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.678+0000 7f92196f0700 1 -- 172.21.15.89:0/4005127221 >> 172.21.15.89:0/4005127221 conn(0x7f9214068e20 msgr2=0x7f92140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:44.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.678+0000 7f92196f0700 1 -- 172.21.15.89:0/4005127221 shutdown_connections 2023-10-10T20:35:44.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.678+0000 7f92196f0700 1 -- 172.21.15.89:0/4005127221 wait complete. 2023-10-10T20:35:44.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.679+0000 7f92196f0700 1 Processor -- start 2023-10-10T20:35:44.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.679+0000 7f92196f0700 1 -- start start 2023-10-10T20:35:44.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.680+0000 7f92196f0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9214069560 0x7f9214155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:44.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.680+0000 7f92196f0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9214159890 0x7f92140ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:44.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.680+0000 7f92196f0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9214159c60 0x7f921415f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.680+0000 7f92196f0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9214155d70 con 0x7f9214159890 2023-10-10T20:35:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.680+0000 7f92196f0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92140b5040 con 0x7f9214159c60 2023-10-10T20:35:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.680+0000 7f92196f0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92140b4430 con 0x7f9214069560 2023-10-10T20:35:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.680+0000 7f920a7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9214159890 0x7f92140ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.680+0000 7f920a7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9214159890 0x7f92140ce9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58244/0 (socket says 172.21.15.89:58244) 2023-10-10T20:35:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.680+0000 7f920a7fc700 1 -- 172.21.15.89:0/3204416579 learned_addr learned my addr 172.21.15.89:0/3204416579 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:44.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.680+0000 7f9212ffd700 1 --2- 172.21.15.89:0/3204416579 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9214069560 0x7f9214155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:44.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.680+0000 7f92137fe700 1 --2- 172.21.15.89:0/3204416579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9214159c60 0x7f921415f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:44.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.680+0000 7f920a7fc700 1 -- 172.21.15.89:0/3204416579 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9214069560 msgr2=0x7f9214155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:44.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.680+0000 7f920a7fc700 1 --2- 172.21.15.89:0/3204416579 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9214069560 0x7f9214155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:44.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.680+0000 7f920a7fc700 1 -- 172.21.15.89:0/3204416579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9214159c60 msgr2=0x7f921415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:44.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.680+0000 7f920a7fc700 1 --2- 172.21.15.89:0/3204416579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9214159c60 0x7f921415f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:44.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.680+0000 7f920a7fc700 1 -- 172.21.15.89:0/3204416579 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92040094f0 con 0x7f9214159890 2023-10-10T20:35:44.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.680+0000 7f9212ffd700 1 --2- 172.21.15.89:0/3204416579 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9214069560 0x7f9214155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:44.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.680+0000 7f92137fe700 1 --2- 172.21.15.89:0/3204416579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9214159c60 0x7f921415f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:44.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.680+0000 7f920a7fc700 1 --2- 172.21.15.89:0/3204416579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9214159890 0x7f92140ce9f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00b5e0 tx=0x7f91fc0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:44.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.681+0000 7f920b7fe700 1 -- 172.21.15.89:0/3204416579 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc00f510 con 0x7f9214159890 2023-10-10T20:35:44.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.681+0000 7f920b7fe700 1 -- 172.21.15.89:0/3204416579 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91fc00f690 con 0x7f9214159890 2023-10-10T20:35:44.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.681+0000 7f920b7fe700 1 -- 172.21.15.89:0/3204416579 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc00fa50 con 0x7f9214159890 2023-10-10T20:35:44.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.681+0000 7f92196f0700 1 -- 172.21.15.89:0/3204416579 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92140b4710 con 0x7f9214159890 2023-10-10T20:35:44.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.681+0000 7f92196f0700 1 -- 172.21.15.89:0/3204416579 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92140b4cd0 con 0x7f9214159890 2023-10-10T20:35:44.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.681+0000 7f92196f0700 1 -- 172.21.15.89:0/3204416579 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92140b51d0 con 0x7f9214159890 2023-10-10T20:35:44.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.682+0000 7f920b7fe700 1 -- 172.21.15.89:0/3204416579 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f91fc01a400 con 0x7f9214159890 2023-10-10T20:35:44.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.682+0000 7f920b7fe700 1 --2- 172.21.15.89:0/3204416579 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f91f4066f30 0x7f91f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:44.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.682+0000 7f920b7fe700 1 -- 172.21.15.89:0/3204416579 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f91fc084130 con 0x7f9214159890 2023-10-10T20:35:44.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.682+0000 7f9212ffd700 1 --2- 172.21.15.89:0/3204416579 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f91f4066f30 0x7f91f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:44.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.683+0000 7f9212ffd700 1 --2- 172.21.15.89:0/3204416579 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f91f4066f30 0x7f91f40693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f921406a890 tx=0x7f9204016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.686+0000 7f920b7fe700 1 -- 172.21.15.89:0/3204416579 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91fc054e00 con 0x7f9214159890 2023-10-10T20:35:44.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:44.909+0000 7f92196f0700 1 -- 172.21.15.89:0/3204416579 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f921404c3c0 con 0x7f9214159890 2023-10-10T20:35:45.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.548+0000 7f920b7fe700 1 -- 172.21.15.89:0/3204416579 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) v1 ==== 104+0+0 (secure 0 0 0) 0x7f91fc054990 con 0x7f9214159890 2023-10-10T20:35:45.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.551+0000 7f92196f0700 1 -- 172.21.15.89:0/3204416579 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f91f4066f30 msgr2=0x7f91f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:45.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.551+0000 7f92196f0700 1 --2- 172.21.15.89:0/3204416579 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f91f4066f30 0x7f91f40693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f921406a890 tx=0x7f9204016000 comp rx=0 tx=0).stop 2023-10-10T20:35:45.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.551+0000 7f92196f0700 1 -- 172.21.15.89:0/3204416579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9214159890 msgr2=0x7f92140ce9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:45.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.551+0000 7f92196f0700 1 --2- 172.21.15.89:0/3204416579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9214159890 0x7f92140ce9f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00b5e0 tx=0x7f91fc0090f0 comp rx=0 tx=0).stop 2023-10-10T20:35:45.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.551+0000 7f92196f0700 1 -- 172.21.15.89:0/3204416579 shutdown_connections 2023-10-10T20:35:45.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.551+0000 7f92196f0700 1 --2- 172.21.15.89:0/3204416579 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f91f4066f30 0x7f91f40693e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:45.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.551+0000 7f92196f0700 1 --2- 172.21.15.89:0/3204416579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9214159c60 0x7f921415f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:45.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.551+0000 7f92196f0700 1 --2- 172.21.15.89:0/3204416579 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9214069560 0x7f9214155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:45.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.551+0000 7f92196f0700 1 --2- 172.21.15.89:0/3204416579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9214159890 0x7f92140ce9f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:45.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.551+0000 7f92196f0700 1 -- 172.21.15.89:0/3204416579 >> 172.21.15.89:0/3204416579 conn(0x7f9214068e20 msgr2=0x7f9214158590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:45.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.551+0000 7f92196f0700 1 -- 172.21.15.89:0/3204416579 shutdown_connections 2023-10-10T20:35:45.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.551+0000 7f92196f0700 1 -- 172.21.15.89:0/3204416579 wait complete. 2023-10-10T20:35:45.566 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:35:45.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.723+0000 7f5ff31dc700 1 Processor -- start 2023-10-10T20:35:45.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.723+0000 7f5ff31dc700 1 -- start start 2023-10-10T20:35:45.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.724+0000 7f5ff31dc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5fec155360 0x7f5fec0b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:45.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.724+0000 7f5ff31dc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5fec0b40f0 0x7f5fec0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:45.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.724+0000 7f5ff31dc700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fec0b4560 con 0x7f5fec0ceb60 2023-10-10T20:35:45.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.724+0000 7f5ff31dc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fec15c320 con 0x7f5fec0b40f0 2023-10-10T20:35:45.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.724+0000 7f5ff31dc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec0cef90 con 0x7f5fec155360 2023-10-10T20:35:45.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.724+0000 7f5fe3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5fec0b40f0 0x7f5fec0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:45.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.724+0000 7f5fe3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5fec0b40f0 0x7f5fec0ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58246/0 (socket says 172.21.15.89:58246) 2023-10-10T20:35:45.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.724+0000 7f5fe3fff700 1 -- 172.21.15.89:0/4265724553 learned_addr learned my addr 172.21.15.89:0/4265724553 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:45.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.725+0000 7f5fe3fff700 1 -- 172.21.15.89:0/4265724553 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5fec155360 msgr2=0x7f5fec0b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:45.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.725+0000 7f5fe3fff700 1 --2- 172.21.15.89:0/4265724553 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5fec155360 0x7f5fec0b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:45.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.725+0000 7f5fe3fff700 1 -- 172.21.15.89:0/4265724553 >> v1:172.21.15.111:6789/0 conn(0x7f5fec0ceb60 legacy=0x7f5fec06d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:45.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.725+0000 7f5fe3fff700 1 -- 172.21.15.89:0/4265724553 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec15b710 con 0x7f5fec0b40f0 2023-10-10T20:35:45.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.725+0000 7f5fe3fff700 1 --2- 172.21.15.89:0/4265724553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fec0b40f0 0x7f5fec0ce620 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0099b0 tx=0x7f5fdc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3835d3d8caf804af server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:45.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.725+0000 7f5fe37fe700 1 -- 172.21.15.89:0/4265724553 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00b820 con 0x7f5fec0b40f0 2023-10-10T20:35:45.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.725+0000 7f5fe37fe700 1 -- 172.21.15.89:0/4265724553 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fdc00b980 con 0x7f5fec0b40f0 2023-10-10T20:35:45.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.725+0000 7f5fe37fe700 1 -- 172.21.15.89:0/4265724553 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00bd20 con 0x7f5fec0b40f0 2023-10-10T20:35:45.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.725+0000 7f5ff31dc700 1 -- 172.21.15.89:0/4265724553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fec0b40f0 msgr2=0x7f5fec0ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:45.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.725+0000 7f5ff31dc700 1 --2- 172.21.15.89:0/4265724553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fec0b40f0 0x7f5fec0ce620 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0099b0 tx=0x7f5fdc009250 comp rx=0 tx=0).stop 2023-10-10T20:35:45.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.725+0000 7f5ff31dc700 1 -- 172.21.15.89:0/4265724553 shutdown_connections 2023-10-10T20:35:45.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.725+0000 7f5ff31dc700 1 --2- 172.21.15.89:0/4265724553 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5fec155360 0x7f5fec0b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:45.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.725+0000 7f5ff31dc700 1 --2- 172.21.15.89:0/4265724553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fec0b40f0 0x7f5fec0ce620 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:45.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.726+0000 7f5ff31dc700 1 -- 172.21.15.89:0/4265724553 >> 172.21.15.89:0/4265724553 conn(0x7f5fec06baf0 msgr2=0x7f5fec06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:45.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.726+0000 7f5ff31dc700 1 -- 172.21.15.89:0/4265724553 shutdown_connections 2023-10-10T20:35:45.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.726+0000 7f5ff31dc700 1 -- 172.21.15.89:0/4265724553 wait complete. 2023-10-10T20:35:45.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.727+0000 7f5ff31dc700 1 Processor -- start 2023-10-10T20:35:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.727+0000 7f5ff31dc700 1 -- start start 2023-10-10T20:35:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.727+0000 7f5ff31dc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fec0b40f0 0x7f5fec168c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.727+0000 7f5ff31dc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5fec0ceb60 0x7f5fec0c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.727+0000 7f5ff31dc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fec155360 0x7f5fec0c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.727+0000 7f5ff31dc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec0c5280 con 0x7f5fec155360 2023-10-10T20:35:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.727+0000 7f5ff31dc700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec0c54a0 con 0x7f5fec0b40f0 2023-10-10T20:35:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.727+0000 7f5ff31dc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec0c5780 con 0x7f5fec0ceb60 2023-10-10T20:35:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.728+0000 7f5ff1779700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fec155360 0x7f5fec0c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.728+0000 7f5ff1779700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fec155360 0x7f5fec0c96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58250/0 (socket says 172.21.15.89:58250) 2023-10-10T20:35:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.728+0000 7f5ff1779700 1 -- 172.21.15.89:0/2668050451 learned_addr learned my addr 172.21.15.89:0/2668050451 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.728+0000 7f5ff1779700 1 -- 172.21.15.89:0/2668050451 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5fec0ceb60 msgr2=0x7f5fec0c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.728+0000 7f5ff0f78700 1 --2- 172.21.15.89:0/2668050451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fec0b40f0 0x7f5fec168c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.728+0000 7f5ff1779700 1 --2- 172.21.15.89:0/2668050451 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5fec0ceb60 0x7f5fec0c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.728+0000 7f5ff1779700 1 -- 172.21.15.89:0/2668050451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fec0b40f0 msgr2=0x7f5fec168c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.728+0000 7f5ff1779700 1 --2- 172.21.15.89:0/2668050451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fec0b40f0 0x7f5fec168c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.728+0000 7f5ff1779700 1 -- 172.21.15.89:0/2668050451 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc0094f0 con 0x7f5fec155360 2023-10-10T20:35:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.728+0000 7f5ff1779700 1 --2- 172.21.15.89:0/2668050451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fec155360 0x7f5fec0c96c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400bbb0 tx=0x7f5fe4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.728+0000 7f5fe0ff9700 1 -- 172.21.15.89:0/2668050451 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40092e0 con 0x7f5fec155360 2023-10-10T20:35:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.728+0000 7f5fe0ff9700 1 -- 172.21.15.89:0/2668050451 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe400c7a0 con 0x7f5fec155360 2023-10-10T20:35:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.728+0000 7f5ff31dc700 1 -- 172.21.15.89:0/2668050451 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fec0c5a60 con 0x7f5fec155360 2023-10-10T20:35:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.728+0000 7f5fe0ff9700 1 -- 172.21.15.89:0/2668050451 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe400cb40 con 0x7f5fec155360 2023-10-10T20:35:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.728+0000 7f5ff31dc700 1 -- 172.21.15.89:0/2668050451 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fec0c9c00 con 0x7f5fec155360 2023-10-10T20:35:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.729+0000 7f5ff31dc700 1 -- 172.21.15.89:0/2668050451 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fec0718b0 con 0x7f5fec155360 2023-10-10T20:35:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.729+0000 7f5fe0ff9700 1 -- 172.21.15.89:0/2668050451 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5fe4012070 con 0x7f5fec155360 2023-10-10T20:35:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.730+0000 7f5fe0ff9700 1 --2- 172.21.15.89:0/2668050451 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5fd8066f30 0x7f5fd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.730+0000 7f5fe0ff9700 1 -- 172.21.15.89:0/2668050451 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f5fe40842d0 con 0x7f5fec155360 2023-10-10T20:35:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.730+0000 7f5ff0f78700 1 --2- 172.21.15.89:0/2668050451 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5fd8066f30 0x7f5fd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.730+0000 7f5ff0f78700 1 --2- 172.21.15.89:0/2668050451 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5fd8066f30 0x7f5fd80693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8006fd0 tx=0x7f5fe8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.734+0000 7f5fe0ff9700 1 -- 172.21.15.89:0/2668050451 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5fe4051490 con 0x7f5fec155360 2023-10-10T20:35:45.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:45.916+0000 7f5ff31dc700 1 -- 172.21.15.89:0/2668050451 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5fec0c9e20 con 0x7f5fec155360 2023-10-10T20:35:46.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.550+0000 7f5fe0ff9700 1 -- 172.21.15.89:0/2668050451 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v40) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5fe4054ab0 con 0x7f5fec155360 2023-10-10T20:35:46.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.554+0000 7f5ff31dc700 1 -- 172.21.15.89:0/2668050451 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5fd8066f30 msgr2=0x7f5fd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:46.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.554+0000 7f5ff31dc700 1 --2- 172.21.15.89:0/2668050451 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5fd8066f30 0x7f5fd80693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8006fd0 tx=0x7f5fe8008040 comp rx=0 tx=0).stop 2023-10-10T20:35:46.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.554+0000 7f5ff31dc700 1 -- 172.21.15.89:0/2668050451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fec155360 msgr2=0x7f5fec0c96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:46.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.554+0000 7f5ff31dc700 1 --2- 172.21.15.89:0/2668050451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fec155360 0x7f5fec0c96c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400bbb0 tx=0x7f5fe4009840 comp rx=0 tx=0).stop 2023-10-10T20:35:46.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.554+0000 7f5ff31dc700 1 -- 172.21.15.89:0/2668050451 shutdown_connections 2023-10-10T20:35:46.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.554+0000 7f5ff31dc700 1 --2- 172.21.15.89:0/2668050451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fec155360 0x7f5fec0c96c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:46.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.554+0000 7f5ff31dc700 1 --2- 172.21.15.89:0/2668050451 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5fd8066f30 0x7f5fd80693e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:46.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.554+0000 7f5ff31dc700 1 --2- 172.21.15.89:0/2668050451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fec0b40f0 0x7f5fec168c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:46.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.554+0000 7f5ff31dc700 1 --2- 172.21.15.89:0/2668050451 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5fec0ceb60 0x7f5fec0c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:46.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.554+0000 7f5ff31dc700 1 -- 172.21.15.89:0/2668050451 >> 172.21.15.89:0/2668050451 conn(0x7f5fec06baf0 msgr2=0x7f5fec06d130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:46.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.554+0000 7f5ff31dc700 1 -- 172.21.15.89:0/2668050451 shutdown_connections 2023-10-10T20:35:46.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.554+0000 7f5ff31dc700 1 -- 172.21.15.89:0/2668050451 wait complete. 2023-10-10T20:35:46.559 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:35:46.573 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:35:46.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.693+0000 7f5a9c065700 1 Processor -- start 2023-10-10T20:35:46.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.694+0000 7f5a9c065700 1 -- start start 2023-10-10T20:35:46.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.694+0000 7f5a9c065700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5a940cd910 0x7f5a940ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.694+0000 7f5a9c065700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a940b5b00 con 0x7f5a94159ec0 2023-10-10T20:35:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.694+0000 7f5a9c065700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a940b5da0 con 0x7f5a940b55a0 2023-10-10T20:35:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.694+0000 7f5a9c065700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a940b6080 con 0x7f5a940cd910 2023-10-10T20:35:46.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.695+0000 7f5a99e01700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5a94159ec0 0x7f5a940b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45498/0 (socket says 172.21.15.89:45498) 2023-10-10T20:35:46.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.695+0000 7f5a99e01700 1 -- 172.21.15.89:0/1721588601 learned_addr learned my addr 172.21.15.89:0/1721588601 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.695+0000 7f5a99600700 1 --2- 172.21.15.89:0/1721588601 >> v2:172.21.15.111:3300/0 conn(0x7f5a940cd910 0x7f5a940ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.695+0000 7f5a98dff700 1 -- 172.21.15.89:0/1721588601 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256003781 0 0) 0x7f5a940b5b00 con 0x7f5a94159ec0 2023-10-10T20:35:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.695+0000 7f5a98dff700 1 -- 172.21.15.89:0/1721588601 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a80003570 con 0x7f5a94159ec0 2023-10-10T20:35:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.695+0000 7f5a99600700 1 -- 172.21.15.89:0/1721588601 >> v1:172.21.15.111:6789/0 conn(0x7f5a940b55a0 legacy=0x7f5a940ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.695+0000 7f5a99600700 1 -- 172.21.15.89:0/1721588601 >> v1:172.21.15.89:6789/0 conn(0x7f5a94159ec0 legacy=0x7f5a940b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:46.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.695+0000 7f5a99600700 1 -- 172.21.15.89:0/1721588601 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a940cc100 con 0x7f5a940cd910 2023-10-10T20:35:46.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.695+0000 7f5a99600700 1 --2- 172.21.15.89:0/1721588601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a940cd910 0x7f5a940ce340 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009910 tx=0x7f5a840091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9774c4b4886ba093 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:46.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.695+0000 7f5a98dff700 1 -- 172.21.15.89:0/1721588601 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8400b640 con 0x7f5a940cd910 2023-10-10T20:35:46.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.696+0000 7f5a98dff700 1 -- 172.21.15.89:0/1721588601 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8400b7a0 con 0x7f5a940cd910 2023-10-10T20:35:46.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.696+0000 7f5a98dff700 1 -- 172.21.15.89:0/1721588601 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8400bb60 con 0x7f5a940cd910 2023-10-10T20:35:46.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.696+0000 7f5a9c065700 1 -- 172.21.15.89:0/1721588601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a940cd910 msgr2=0x7f5a940ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:46.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.696+0000 7f5a9c065700 1 --2- 172.21.15.89:0/1721588601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a940cd910 0x7f5a940ce340 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009910 tx=0x7f5a840091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:46.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.696+0000 7f5a9c065700 1 -- 172.21.15.89:0/1721588601 shutdown_connections 2023-10-10T20:35:46.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.696+0000 7f5a9c065700 1 --2- 172.21.15.89:0/1721588601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a940cd910 0x7f5a940ce340 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:46.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.696+0000 7f5a9c065700 1 -- 172.21.15.89:0/1721588601 >> 172.21.15.89:0/1721588601 conn(0x7f5a940999f0 msgr2=0x7f5a940b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:46.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.696+0000 7f5a9c065700 1 -- 172.21.15.89:0/1721588601 shutdown_connections 2023-10-10T20:35:46.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.696+0000 7f5a9c065700 1 -- 172.21.15.89:0/1721588601 wait complete. 2023-10-10T20:35:46.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.697+0000 7f5a9c065700 1 Processor -- start 2023-10-10T20:35:46.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.697+0000 7f5a9c065700 1 -- start start 2023-10-10T20:35:46.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.698+0000 7f5a9c065700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a940b55a0 0x7f5a941570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:46.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.698+0000 7f5a9c065700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a94159ec0 0x7f5a9406abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:46.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.698+0000 7f5a9c065700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a9406a180 0x7f5a940c5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:46.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.698+0000 7f5a9c065700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9415c3b0 con 0x7f5a940b55a0 2023-10-10T20:35:46.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.698+0000 7f5a9c065700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9415cf30 con 0x7f5a94159ec0 2023-10-10T20:35:46.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.698+0000 7f5a9c065700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a94157600 con 0x7f5a9406a180 2023-10-10T20:35:46.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.698+0000 7f5a99e01700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a940b55a0 0x7f5a941570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:46.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.698+0000 7f5a99e01700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a940b55a0 0x7f5a941570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58254/0 (socket says 172.21.15.89:58254) 2023-10-10T20:35:46.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.698+0000 7f5a99e01700 1 -- 172.21.15.89:0/2173832361 learned_addr learned my addr 172.21.15.89:0/2173832361 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:46.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.698+0000 7f5a99600700 1 --2- 172.21.15.89:0/2173832361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a94159ec0 0x7f5a9406abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:46.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.698+0000 7f5a99e01700 1 -- 172.21.15.89:0/2173832361 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a9406a180 msgr2=0x7f5a940c5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:46.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.698+0000 7f5a99e01700 1 --2- 172.21.15.89:0/2173832361 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a9406a180 0x7f5a940c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:46.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.698+0000 7f5a99e01700 1 -- 172.21.15.89:0/2173832361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a94159ec0 msgr2=0x7f5a9406abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:46.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.698+0000 7f5a99e01700 1 --2- 172.21.15.89:0/2173832361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a94159ec0 0x7f5a9406abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:46.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.698+0000 7f5a99e01700 1 -- 172.21.15.89:0/2173832361 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a84009420 con 0x7f5a940b55a0 2023-10-10T20:35:46.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.699+0000 7f5a99600700 1 --2- 172.21.15.89:0/2173832361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a94159ec0 0x7f5a9406abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:46.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.699+0000 7f5a99e01700 1 --2- 172.21.15.89:0/2173832361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a940b55a0 0x7f5a941570c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5a90004ed0 tx=0x7f5a90005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:46.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.699+0000 7f5a89ffb700 1 -- 172.21.15.89:0/2173832361 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9000ab70 con 0x7f5a940b55a0 2023-10-10T20:35:46.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.699+0000 7f5a9c065700 1 -- 172.21.15.89:0/2173832361 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9415c600 con 0x7f5a940b55a0 2023-10-10T20:35:46.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.699+0000 7f5a89ffb700 1 -- 172.21.15.89:0/2173832361 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9000acd0 con 0x7f5a940b55a0 2023-10-10T20:35:46.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.699+0000 7f5a89ffb700 1 -- 172.21.15.89:0/2173832361 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a90007300 con 0x7f5a940b55a0 2023-10-10T20:35:46.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.699+0000 7f5a9c065700 1 -- 172.21.15.89:0/2173832361 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9415cbc0 con 0x7f5a940b55a0 2023-10-10T20:35:46.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.700+0000 7f5a9c065700 1 -- 172.21.15.89:0/2173832361 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a94063c90 con 0x7f5a940b55a0 2023-10-10T20:35:46.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.700+0000 7f5a89ffb700 1 -- 172.21.15.89:0/2173832361 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5a90007730 con 0x7f5a940b55a0 2023-10-10T20:35:46.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.700+0000 7f5a89ffb700 1 --2- 172.21.15.89:0/2173832361 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a78066f30 0x7f5a780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:46.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.701+0000 7f5a89ffb700 1 -- 172.21.15.89:0/2173832361 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5439+0+0 (secure 0 0 0) 0x7f5a90084a10 con 0x7f5a940b55a0 2023-10-10T20:35:46.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.701+0000 7f5a99600700 1 --2- 172.21.15.89:0/2173832361 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a78066f30 0x7f5a780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:46.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.701+0000 7f5a99600700 1 --2- 172.21.15.89:0/2173832361 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a78066f30 0x7f5a780693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5a940c75d0 tx=0x7f5a84017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:46.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.705+0000 7f5a89ffb700 1 -- 172.21.15.89:0/2173832361 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a90051d90 con 0x7f5a940b55a0 2023-10-10T20:35:46.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:46.892+0000 7f5a9c065700 1 -- 172.21.15.89:0/2173832361 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5a940718b0 con 0x7f5a940b55a0 2023-10-10T20:35:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:47.895+0000 7f5a89ffb700 1 -- 172.21.15.89:0/2173832361 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v41) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5a900553b0 con 0x7f5a940b55a0 2023-10-10T20:35:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:47.898+0000 7f5a9c065700 1 -- 172.21.15.89:0/2173832361 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a78066f30 msgr2=0x7f5a780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:47.898+0000 7f5a9c065700 1 --2- 172.21.15.89:0/2173832361 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a78066f30 0x7f5a780693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5a940c75d0 tx=0x7f5a84017000 comp rx=0 tx=0).stop 2023-10-10T20:35:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:47.899+0000 7f5a9c065700 1 -- 172.21.15.89:0/2173832361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a940b55a0 msgr2=0x7f5a941570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:47.899+0000 7f5a9c065700 1 --2- 172.21.15.89:0/2173832361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a940b55a0 0x7f5a941570c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5a90004ed0 tx=0x7f5a90005550 comp rx=0 tx=0).stop 2023-10-10T20:35:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:47.899+0000 7f5a9c065700 1 -- 172.21.15.89:0/2173832361 shutdown_connections 2023-10-10T20:35:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:47.899+0000 7f5a9c065700 1 --2- 172.21.15.89:0/2173832361 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a9406a180 0x7f5a940c5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:47.899+0000 7f5a9c065700 1 --2- 172.21.15.89:0/2173832361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a940b55a0 0x7f5a941570c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:47.899+0000 7f5a9c065700 1 --2- 172.21.15.89:0/2173832361 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a78066f30 0x7f5a780693e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:47.899+0000 7f5a9c065700 1 --2- 172.21.15.89:0/2173832361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a94159ec0 0x7f5a9406abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:47.899+0000 7f5a9c065700 1 -- 172.21.15.89:0/2173832361 >> 172.21.15.89:0/2173832361 conn(0x7f5a940999f0 msgr2=0x7f5a940ce020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:47.899+0000 7f5a9c065700 1 -- 172.21.15.89:0/2173832361 shutdown_connections 2023-10-10T20:35:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:47.899+0000 7f5a9c065700 1 -- 172.21.15.89:0/2173832361 wait complete. 2023-10-10T20:35:47.904 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' removed 2023-10-10T20:35:47.915 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:35:48.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.036+0000 7fd312e77700 1 Processor -- start 2023-10-10T20:35:48.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.036+0000 7fd312e77700 1 -- start start 2023-10-10T20:35:48.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.036+0000 7fd312e77700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd30c0aca80 0x7fd30c06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:48.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.036+0000 7fd312e77700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30c06bdb0 con 0x7fd30c0c83c0 2023-10-10T20:35:48.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.036+0000 7fd312e77700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30c0cb370 con 0x7fd30c069d60 2023-10-10T20:35:48.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.036+0000 7fd312e77700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c159f60 con 0x7fd30c0aca80 2023-10-10T20:35:48.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.036+0000 7fd311414700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd30c0c83c0 0x7fd30c0cb800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45500/0 (socket says 172.21.15.89:45500) 2023-10-10T20:35:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd311414700 1 -- 172.21.15.89:0/1104774379 learned_addr learned my addr 172.21.15.89:0/1104774379 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd3037fe700 1 -- 172.21.15.89:0/1104774379 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887168605 0 0) 0x7fd30c06bdb0 con 0x7fd30c0c83c0 2023-10-10T20:35:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd310c13700 1 --2- 172.21.15.89:0/1104774379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd30c0aca80 0x7fd30c06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd3037fe700 1 -- 172.21.15.89:0/1104774379 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2f4003570 con 0x7fd30c0c83c0 2023-10-10T20:35:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd3037fe700 1 -- 172.21.15.89:0/1104774379 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 251222713 0 0) 0x7fd2f4003570 con 0x7fd30c0c83c0 2023-10-10T20:35:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd3037fe700 1 -- 172.21.15.89:0/1104774379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd30c0aca80 msgr2=0x7fd30c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd3037fe700 1 --2- 172.21.15.89:0/1104774379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd30c0aca80 0x7fd30c06aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd3037fe700 1 -- 172.21.15.89:0/1104774379 >> v1:172.21.15.111:6789/0 conn(0x7fd30c069d60 legacy=0x7fd30c0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd3037fe700 1 -- 172.21.15.89:0/1104774379 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c06b510 con 0x7fd30c0c83c0 2023-10-10T20:35:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd3037fe700 1 -- 172.21.15.89:0/1104774379 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fd304001470 con 0x7fd30c0c83c0 2023-10-10T20:35:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd3037fe700 1 -- 172.21.15.89:0/1104774379 >> v1:172.21.15.89:6789/0 conn(0x7fd30c0c83c0 legacy=0x7fd30c0cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd3037fe700 1 --2- 172.21.15.89:0/1104774379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2f4003a90 0x7fd2f4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd3037fe700 1 --2- 172.21.15.89:0/1104774379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd30c0aca80 0x7fd2f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd3037fe700 1 --2- 172.21.15.89:0/1104774379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2f4004d50 0x7fd2f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd3037fe700 1 -- 172.21.15.89:0/1104774379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c159f60 con 0x7fd2f4004d50 2023-10-10T20:35:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd3037fe700 1 -- 172.21.15.89:0/1104774379 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f400bb70 con 0x7fd2f4003a90 2023-10-10T20:35:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd3037fe700 1 -- 172.21.15.89:0/1104774379 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f400bd30 con 0x7fd30c0aca80 2023-10-10T20:35:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd311414700 1 --2- 172.21.15.89:0/1104774379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2f4004d50 0x7fd2f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd310c13700 1 --2- 172.21.15.89:0/1104774379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2f4003a90 0x7fd2f4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.037+0000 7fd303fff700 1 --2- 172.21.15.89:0/1104774379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd30c0aca80 0x7fd2f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.038+0000 7fd311414700 1 -- 172.21.15.89:0/1104774379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd30c0aca80 msgr2=0x7fd2f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.038+0000 7fd311414700 1 --2- 172.21.15.89:0/1104774379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd30c0aca80 0x7fd2f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.038+0000 7fd311414700 1 -- 172.21.15.89:0/1104774379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2f4003a90 msgr2=0x7fd2f4003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.038+0000 7fd311414700 1 --2- 172.21.15.89:0/1104774379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2f4003a90 0x7fd2f4003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.038+0000 7fd311414700 1 -- 172.21.15.89:0/1104774379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f4003570 con 0x7fd2f4004d50 2023-10-10T20:35:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.038+0000 7fd311414700 1 --2- 172.21.15.89:0/1104774379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2f4004d50 0x7fd2f40051c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0c87c0 tx=0x7fd304001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e51344a314ec91 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.038+0000 7fd3037fe700 1 -- 172.21.15.89:0/1104774379 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30400cc00 con 0x7fd2f4004d50 2023-10-10T20:35:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.038+0000 7fd3037fe700 1 -- 172.21.15.89:0/1104774379 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30400cd60 con 0x7fd2f4004d50 2023-10-10T20:35:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.038+0000 7fd3037fe700 1 -- 172.21.15.89:0/1104774379 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304005b50 con 0x7fd2f4004d50 2023-10-10T20:35:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.038+0000 7fd312e77700 1 -- 172.21.15.89:0/1104774379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2f4004d50 msgr2=0x7fd2f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.038+0000 7fd312e77700 1 --2- 172.21.15.89:0/1104774379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2f4004d50 0x7fd2f40051c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0c87c0 tx=0x7fd304001e50 comp rx=0 tx=0).stop 2023-10-10T20:35:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.038+0000 7fd311414700 1 -- 172.21.15.89:0/1104774379 reap_dead start 2023-10-10T20:35:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.038+0000 7fd312e77700 1 -- 172.21.15.89:0/1104774379 shutdown_connections 2023-10-10T20:35:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.038+0000 7fd312e77700 1 -- 172.21.15.89:0/1104774379 >> 172.21.15.89:0/1104774379 conn(0x7fd30c0cd670 msgr2=0x7fd30c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.038+0000 7fd312e77700 1 -- 172.21.15.89:0/1104774379 shutdown_connections 2023-10-10T20:35:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.038+0000 7fd312e77700 1 -- 172.21.15.89:0/1104774379 wait complete. 2023-10-10T20:35:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.039+0000 7fd312e77700 1 Processor -- start 2023-10-10T20:35:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.040+0000 7fd312e77700 1 -- start start 2023-10-10T20:35:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.040+0000 7fd312e77700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2f4004d50 0x7fd30c0c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.040+0000 7fd312e77700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd30c0ae880 0x7fd30c16d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.040+0000 7fd312e77700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd30c16ddd0 0x7fd30c0c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.040+0000 7fd312e77700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c16e240 con 0x7fd30c16ddd0 2023-10-10T20:35:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.040+0000 7fd312e77700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0b0d10 con 0x7fd2f4004d50 2023-10-10T20:35:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.040+0000 7fd312e77700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c16a120 con 0x7fd30c0ae880 2023-10-10T20:35:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.041+0000 7fd311414700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd30c16ddd0 0x7fd30c0c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.041+0000 7fd311414700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd30c16ddd0 0x7fd30c0c6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58278/0 (socket says 172.21.15.89:58278) 2023-10-10T20:35:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.041+0000 7fd311414700 1 -- 172.21.15.89:0/2331957029 learned_addr learned my addr 172.21.15.89:0/2331957029 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.041+0000 7fd310c13700 1 --2- 172.21.15.89:0/2331957029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2f4004d50 0x7fd30c0c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.041+0000 7fd303fff700 1 --2- 172.21.15.89:0/2331957029 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd30c0ae880 0x7fd30c16d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.041+0000 7fd311414700 1 -- 172.21.15.89:0/2331957029 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd30c0ae880 msgr2=0x7fd30c16d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.041+0000 7fd311414700 1 --2- 172.21.15.89:0/2331957029 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd30c0ae880 0x7fd30c16d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.041+0000 7fd311414700 1 -- 172.21.15.89:0/2331957029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2f4004d50 msgr2=0x7fd30c0c4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.041+0000 7fd311414700 1 --2- 172.21.15.89:0/2331957029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2f4004d50 0x7fd30c0c4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.041+0000 7fd311414700 1 -- 172.21.15.89:0/2331957029 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c0aee40 con 0x7fd30c16ddd0 2023-10-10T20:35:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.041+0000 7fd311414700 1 --2- 172.21.15.89:0/2331957029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd30c16ddd0 0x7fd30c0c6300 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd304009020 tx=0x7fd304015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.041+0000 7fd300ff9700 1 -- 172.21.15.89:0/2331957029 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30400cc00 con 0x7fd30c16ddd0 2023-10-10T20:35:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.041+0000 7fd300ff9700 1 -- 172.21.15.89:0/2331957029 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30400ced0 con 0x7fd30c16ddd0 2023-10-10T20:35:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.041+0000 7fd312e77700 1 -- 172.21.15.89:0/2331957029 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30c0af120 con 0x7fd30c16ddd0 2023-10-10T20:35:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.041+0000 7fd300ff9700 1 -- 172.21.15.89:0/2331957029 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304007530 con 0x7fd30c16ddd0 2023-10-10T20:35:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.041+0000 7fd312e77700 1 -- 172.21.15.89:0/2331957029 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30c0af6e0 con 0x7fd30c16ddd0 2023-10-10T20:35:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.042+0000 7fd312e77700 1 -- 172.21.15.89:0/2331957029 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30c0b0e80 con 0x7fd30c16ddd0 2023-10-10T20:35:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.042+0000 7fd300ff9700 1 -- 172.21.15.89:0/2331957029 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd304028460 con 0x7fd30c16ddd0 2023-10-10T20:35:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.043+0000 7fd300ff9700 1 --2- 172.21.15.89:0/2331957029 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd2f8066f30 0x7fd2f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.043+0000 7fd300ff9700 1 -- 172.21.15.89:0/2331957029 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fd3040856e0 con 0x7fd30c16ddd0 2023-10-10T20:35:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.043+0000 7fd310c13700 1 --2- 172.21.15.89:0/2331957029 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd2f8066f30 0x7fd2f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.043+0000 7fd310c13700 1 --2- 172.21.15.89:0/2331957029 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd2f8066f30 0x7fd2f80693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd3080094d0 tx=0x7fd30800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.047+0000 7fd300ff9700 1 -- 172.21.15.89:0/2331957029 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd304052bf0 con 0x7fd30c16ddd0 2023-10-10T20:35:48.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.231+0000 7fd312e77700 1 -- 172.21.15.89:0/2331957029 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd30c04c3c0 con 0x7fd30c16ddd0 2023-10-10T20:35:48.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.234+0000 7fd300ff9700 1 -- 172.21.15.89:0/2331957029 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd304056210 con 0x7fd30c16ddd0 2023-10-10T20:35:48.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.237+0000 7fd312e77700 1 -- 172.21.15.89:0/2331957029 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd2f8066f30 msgr2=0x7fd2f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:48.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.237+0000 7fd312e77700 1 --2- 172.21.15.89:0/2331957029 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd2f8066f30 0x7fd2f80693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd3080094d0 tx=0x7fd30800b040 comp rx=0 tx=0).stop 2023-10-10T20:35:48.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.237+0000 7fd312e77700 1 -- 172.21.15.89:0/2331957029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd30c16ddd0 msgr2=0x7fd30c0c6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:48.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.237+0000 7fd312e77700 1 --2- 172.21.15.89:0/2331957029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd30c16ddd0 0x7fd30c0c6300 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd304009020 tx=0x7fd304015000 comp rx=0 tx=0).stop 2023-10-10T20:35:48.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.237+0000 7fd312e77700 1 -- 172.21.15.89:0/2331957029 shutdown_connections 2023-10-10T20:35:48.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.237+0000 7fd312e77700 1 --2- 172.21.15.89:0/2331957029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd30c16ddd0 0x7fd30c0c6300 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:48.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.237+0000 7fd312e77700 1 --2- 172.21.15.89:0/2331957029 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd2f8066f30 0x7fd2f80693e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:48.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.237+0000 7fd312e77700 1 --2- 172.21.15.89:0/2331957029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2f4004d50 0x7fd30c0c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:48.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.237+0000 7fd312e77700 1 --2- 172.21.15.89:0/2331957029 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd30c0ae880 0x7fd30c16d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:48.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.237+0000 7fd312e77700 1 -- 172.21.15.89:0/2331957029 >> 172.21.15.89:0/2331957029 conn(0x7fd30c0cd670 msgr2=0x7fd30c0ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:48.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.237+0000 7fd312e77700 1 -- 172.21.15.89:0/2331957029 shutdown_connections 2023-10-10T20:35:48.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.237+0000 7fd312e77700 1 -- 172.21.15.89:0/2331957029 wait complete. 2023-10-10T20:35:48.251 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2023-10-10T20:35:48.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.363+0000 7f17b7974700 1 Processor -- start 2023-10-10T20:35:48.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.363+0000 7f17b7974700 1 -- start start 2023-10-10T20:35:48.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.364+0000 7f17b7974700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f17b00ccf90 0x7f17b00ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:48.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.364+0000 7f17b7974700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b00cc300 con 0x7f17b00cbf30 2023-10-10T20:35:48.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.364+0000 7f17b7974700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b00ad370 con 0x7f17b00b1420 2023-10-10T20:35:48.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.364+0000 7f17b7974700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b0156bf0 con 0x7f17b00ccf90 2023-10-10T20:35:48.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.364+0000 7f17b5710700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f17b00cbf30 0x7f17b00b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52694/0 (socket says 172.21.15.89:52694) 2023-10-10T20:35:48.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.364+0000 7f17b5710700 1 -- 172.21.15.89:0/4075131367 learned_addr learned my addr 172.21.15.89:0/4075131367 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:48.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17b5f11700 1 --2- 172.21.15.89:0/4075131367 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f17b00ccf90 0x7f17b00ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:48.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17a7fff700 1 -- 172.21.15.89:0/4075131367 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4007706341 0 0) 0x7f17b00cc300 con 0x7f17b00cbf30 2023-10-10T20:35:48.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17a7fff700 1 -- 172.21.15.89:0/4075131367 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f179c003570 con 0x7f17b00cbf30 2023-10-10T20:35:48.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17a7fff700 1 -- 172.21.15.89:0/4075131367 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 145121552 0 0) 0x7f17b00ad370 con 0x7f17b00b1420 2023-10-10T20:35:48.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17a7fff700 1 -- 172.21.15.89:0/4075131367 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17b00cc300 con 0x7f17b00b1420 2023-10-10T20:35:48.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17a7fff700 1 -- 172.21.15.89:0/4075131367 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3988589254 0 0) 0x7f179c003570 con 0x7f17b00cbf30 2023-10-10T20:35:48.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17a7fff700 1 -- 172.21.15.89:0/4075131367 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f17b00ccf90 msgr2=0x7f17b00ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:48.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17a7fff700 1 --2- 172.21.15.89:0/4075131367 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f17b00ccf90 0x7f17b00ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:48.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17a7fff700 1 -- 172.21.15.89:0/4075131367 >> v1:172.21.15.111:6789/0 conn(0x7f17b00b1420 legacy=0x7f17b00c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:48.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17a7fff700 1 -- 172.21.15.89:0/4075131367 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b00ad650 con 0x7f17b00cbf30 2023-10-10T20:35:48.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17a7fff700 1 -- 172.21.15.89:0/4075131367 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f17ac001470 con 0x7f17b00cbf30 2023-10-10T20:35:48.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17a7fff700 1 -- 172.21.15.89:0/4075131367 >> v1:172.21.15.89:6789/0 conn(0x7f17b00cbf30 legacy=0x7f17b00b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:48.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17a7fff700 1 --2- 172.21.15.89:0/4075131367 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f17b00ccf90 0x7f179c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:48.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17a7fff700 1 --2- 172.21.15.89:0/4075131367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f179c0041d0 0x7f179c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:48.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17a7fff700 1 --2- 172.21.15.89:0/4075131367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f179c004d50 0x7f179c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:48.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17a7fff700 1 -- 172.21.15.89:0/4075131367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b0156bf0 con 0x7f179c0041d0 2023-10-10T20:35:48.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17a7fff700 1 -- 172.21.15.89:0/4075131367 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c00bbf0 con 0x7f179c004d50 2023-10-10T20:35:48.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17a7fff700 1 -- 172.21.15.89:0/4075131367 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f179c00bdb0 con 0x7f17b00ccf90 2023-10-10T20:35:48.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17b4f0f700 1 --2- 172.21.15.89:0/4075131367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f179c0041d0 0x7f179c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:48.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17b5710700 1 --2- 172.21.15.89:0/4075131367 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f17b00ccf90 0x7f179c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:48.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17b5f11700 1 --2- 172.21.15.89:0/4075131367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f179c004d50 0x7f179c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:48.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17b4f0f700 1 -- 172.21.15.89:0/4075131367 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f17b00ccf90 msgr2=0x7f179c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:48.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17b4f0f700 1 --2- 172.21.15.89:0/4075131367 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f17b00ccf90 0x7f179c003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:48.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17b4f0f700 1 -- 172.21.15.89:0/4075131367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f179c004d50 msgr2=0x7f179c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:48.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17b4f0f700 1 --2- 172.21.15.89:0/4075131367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f179c004d50 0x7f179c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:48.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.365+0000 7f17b4f0f700 1 -- 172.21.15.89:0/4075131367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179c003570 con 0x7f179c0041d0 2023-10-10T20:35:48.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.366+0000 7f17b4f0f700 1 --2- 172.21.15.89:0/4075131367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f179c0041d0 0x7f179c004810 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f17b00b1820 tx=0x7f17a0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf7952b66e71179 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:48.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.366+0000 7f17a7fff700 1 -- 172.21.15.89:0/4075131367 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a000c5d0 con 0x7f179c0041d0 2023-10-10T20:35:48.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.366+0000 7f17a7fff700 1 -- 172.21.15.89:0/4075131367 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a000c730 con 0x7f179c0041d0 2023-10-10T20:35:48.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.366+0000 7f17a7fff700 1 -- 172.21.15.89:0/4075131367 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a000cad0 con 0x7f179c0041d0 2023-10-10T20:35:48.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.366+0000 7f17b7974700 1 -- 172.21.15.89:0/4075131367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f179c0041d0 msgr2=0x7f179c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:48.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.366+0000 7f17b7974700 1 --2- 172.21.15.89:0/4075131367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f179c0041d0 0x7f179c004810 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f17b00b1820 tx=0x7f17a0007d30 comp rx=0 tx=0).stop 2023-10-10T20:35:48.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.366+0000 7f17b5f11700 1 -- 172.21.15.89:0/4075131367 reap_dead start 2023-10-10T20:35:48.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.366+0000 7f17b7974700 1 -- 172.21.15.89:0/4075131367 shutdown_connections 2023-10-10T20:35:48.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.366+0000 7f17b7974700 1 -- 172.21.15.89:0/4075131367 >> 172.21.15.89:0/4075131367 conn(0x7f17b006a320 msgr2=0x7f17b00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:48.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.366+0000 7f17b7974700 1 -- 172.21.15.89:0/4075131367 shutdown_connections 2023-10-10T20:35:48.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.366+0000 7f17b7974700 1 -- 172.21.15.89:0/4075131367 wait complete. 2023-10-10T20:35:48.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.367+0000 7f17b7974700 1 Processor -- start 2023-10-10T20:35:48.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.367+0000 7f17b7974700 1 -- start start 2023-10-10T20:35:48.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.368+0000 7f17b7974700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f179c0041d0 0x7f17b0145b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:48.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.368+0000 7f17b7974700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b01683c0 0x7f17b00aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:48.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.368+0000 7f17b7974700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f17b006e8a0 0x7f17b006ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:48.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.368+0000 7f17b7974700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b0168900 con 0x7f17b01683c0 2023-10-10T20:35:48.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.368+0000 7f17b7974700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b006f250 con 0x7f179c0041d0 2023-10-10T20:35:48.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.368+0000 7f17b7974700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b00c1ca0 con 0x7f17b006e8a0 2023-10-10T20:35:48.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.368+0000 7f17b4f0f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b01683c0 0x7f17b00aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:48.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.368+0000 7f17b4f0f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b01683c0 0x7f17b00aa9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60302/0 (socket says 172.21.15.89:60302) 2023-10-10T20:35:48.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.368+0000 7f17b4f0f700 1 -- 172.21.15.89:0/3777200487 learned_addr learned my addr 172.21.15.89:0/3777200487 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:48.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.368+0000 7f17b5710700 1 --2- 172.21.15.89:0/3777200487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f179c0041d0 0x7f17b0145b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:48.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.368+0000 7f17b5f11700 1 --2- 172.21.15.89:0/3777200487 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f17b006e8a0 0x7f17b006ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:48.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.369+0000 7f17b4f0f700 1 -- 172.21.15.89:0/3777200487 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f17b006e8a0 msgr2=0x7f17b006ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:48.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.369+0000 7f17b4f0f700 1 --2- 172.21.15.89:0/3777200487 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f17b006e8a0 0x7f17b006ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:48.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.369+0000 7f17b4f0f700 1 -- 172.21.15.89:0/3777200487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f179c0041d0 msgr2=0x7f17b0145b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:48.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.369+0000 7f17b4f0f700 1 --2- 172.21.15.89:0/3777200487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f179c0041d0 0x7f17b0145b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:48.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.369+0000 7f17b4f0f700 1 -- 172.21.15.89:0/3777200487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b00ab210 con 0x7f17b01683c0 2023-10-10T20:35:48.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.369+0000 7f17b5f11700 1 --2- 172.21.15.89:0/3777200487 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f17b006e8a0 0x7f17b006ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:48.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.369+0000 7f17b5710700 1 --2- 172.21.15.89:0/3777200487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f179c0041d0 0x7f17b0145b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:48.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.369+0000 7f17b4f0f700 1 --2- 172.21.15.89:0/3777200487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b01683c0 0x7f17b00aa9b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f179c0057f0 tx=0x7f17a0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:48.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.369+0000 7f17a57fa700 1 -- 172.21.15.89:0/3777200487 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a000c3f0 con 0x7f17b01683c0 2023-10-10T20:35:48.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.369+0000 7f17a57fa700 1 -- 172.21.15.89:0/3777200487 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a000cab0 con 0x7f17b01683c0 2023-10-10T20:35:48.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.369+0000 7f17a57fa700 1 -- 172.21.15.89:0/3777200487 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0006a50 con 0x7f17b01683c0 2023-10-10T20:35:48.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.369+0000 7f17b7974700 1 -- 172.21.15.89:0/3777200487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b00c1370 con 0x7f17b01683c0 2023-10-10T20:35:48.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.369+0000 7f17b7974700 1 -- 172.21.15.89:0/3777200487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b00c1930 con 0x7f17b01683c0 2023-10-10T20:35:48.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.370+0000 7f17b7974700 1 -- 172.21.15.89:0/3777200487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b00c1e10 con 0x7f17b01683c0 2023-10-10T20:35:48.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.370+0000 7f17a57fa700 1 -- 172.21.15.89:0/3777200487 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f17a0019070 con 0x7f17b01683c0 2023-10-10T20:35:48.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.371+0000 7f17a57fa700 1 --2- 172.21.15.89:0/3777200487 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1794066f30 0x7f17940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:48.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.371+0000 7f17b5710700 1 --2- 172.21.15.89:0/3777200487 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1794066f30 0x7f17940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:48.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.371+0000 7f17a57fa700 1 -- 172.21.15.89:0/3777200487 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f17a0085870 con 0x7f17b01683c0 2023-10-10T20:35:48.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.371+0000 7f17b5710700 1 --2- 172.21.15.89:0/3777200487 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1794066f30 0x7f17940693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f17ac001bb0 tx=0x7f17ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:48.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.375+0000 7f17a57fa700 1 -- 172.21.15.89:0/3777200487 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17a0052d80 con 0x7f17b01683c0 2023-10-10T20:35:48.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.630+0000 7f17b7974700 1 -- 172.21.15.89:0/3777200487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7f17b004c3c0 con 0x7f17b01683c0 2023-10-10T20:35:48.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.898+0000 7f17a57fa700 1 -- 172.21.15.89:0/3777200487 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v86) v1 ==== 135+0+0 (secure 0 0 0) 0x7f17a0022070 con 0x7f17b01683c0 2023-10-10T20:35:48.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.900+0000 7f17b7974700 1 -- 172.21.15.89:0/3777200487 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1794066f30 msgr2=0x7f17940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:48.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.900+0000 7f17b7974700 1 --2- 172.21.15.89:0/3777200487 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1794066f30 0x7f17940693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f17ac001bb0 tx=0x7f17ac009040 comp rx=0 tx=0).stop 2023-10-10T20:35:48.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.900+0000 7f17b7974700 1 -- 172.21.15.89:0/3777200487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b01683c0 msgr2=0x7f17b00aa9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:48.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.900+0000 7f17b7974700 1 --2- 172.21.15.89:0/3777200487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b01683c0 0x7f17b00aa9b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f179c0057f0 tx=0x7f17a0015000 comp rx=0 tx=0).stop 2023-10-10T20:35:48.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.900+0000 7f17b7974700 1 -- 172.21.15.89:0/3777200487 shutdown_connections 2023-10-10T20:35:48.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.900+0000 7f17b7974700 1 --2- 172.21.15.89:0/3777200487 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f17b006e8a0 0x7f17b006ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:48.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.900+0000 7f17b7974700 1 --2- 172.21.15.89:0/3777200487 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1794066f30 0x7f17940693e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:48.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.900+0000 7f17b7974700 1 --2- 172.21.15.89:0/3777200487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f179c0041d0 0x7f17b0145b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:48.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.900+0000 7f17b7974700 1 --2- 172.21.15.89:0/3777200487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b01683c0 0x7f17b00aa9b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:48.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.900+0000 7f17b7974700 1 -- 172.21.15.89:0/3777200487 >> 172.21.15.89:0/3777200487 conn(0x7f17b006a320 msgr2=0x7f17b00b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:48.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.900+0000 7f17b7974700 1 -- 172.21.15.89:0/3777200487 shutdown_connections 2023-10-10T20:35:48.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:48.900+0000 7f17b7974700 1 -- 172.21.15.89:0/3777200487 wait complete. 2023-10-10T20:35:48.915 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up ... ok 2023-10-10T20:35:48.915 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info' command 2023-10-10T20:35:48.915 INFO:tasks.cephfs_test_runner:test_volume_info (tasks.cephfs.test_volumes.TestVolumes) 2023-10-10T20:35:48.916 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info' 2023-10-10T20:35:49.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.027+0000 7f62a9a10700 1 Processor -- start 2023-10-10T20:35:49.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.028+0000 7f62a9a10700 1 -- start start 2023-10-10T20:35:49.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.028+0000 7f62a9a10700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f62a4159b10 0x7f62a40cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:49.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.028+0000 7f62a9a10700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f62a415d730 0x7f62a415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:49.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.028+0000 7f62a9a10700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a40c74b0 con 0x7f62a4159ee0 2023-10-10T20:35:49.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.028+0000 7f62a9a10700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a40c8700 con 0x7f62a4159b10 2023-10-10T20:35:49.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.028+0000 7f62a9a10700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a40c9070 con 0x7f62a415d730 2023-10-10T20:35:49.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.028+0000 7f62a2ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f62a4159b10 0x7f62a40cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:49.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.029+0000 7f62a2ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f62a4159b10 0x7f62a40cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60310/0 (socket says 172.21.15.89:60310) 2023-10-10T20:35:49.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.029+0000 7f62a2ffd700 1 -- 172.21.15.89:0/3493791352 learned_addr learned my addr 172.21.15.89:0/3493791352 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.029+0000 7f629a7fc700 1 --2- 172.21.15.89:0/3493791352 >> v2:172.21.15.111:3300/0 conn(0x7f62a415d730 0x7f62a415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.029+0000 7f62a2ffd700 1 -- 172.21.15.89:0/3493791352 >> v2:172.21.15.111:3300/0 conn(0x7f62a415d730 msgr2=0x7f62a415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.029+0000 7f62a2ffd700 1 --2- 172.21.15.89:0/3493791352 >> v2:172.21.15.111:3300/0 conn(0x7f62a415d730 0x7f62a415dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.029+0000 7f62a2ffd700 1 -- 172.21.15.89:0/3493791352 >> v1:172.21.15.111:6789/0 conn(0x7f62a4159ee0 legacy=0x7f62a415a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.029+0000 7f62a2ffd700 1 -- 172.21.15.89:0/3493791352 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a415e0e0 con 0x7f62a4159b10 2023-10-10T20:35:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.029+0000 7f62a2ffd700 1 --2- 172.21.15.89:0/3493791352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a4159b10 0x7f62a40cb3e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f62a406cce0 tx=0x7f6294009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c7c323e29b3612 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.029+0000 7f62a27fc700 1 -- 172.21.15.89:0/3493791352 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629400b8b0 con 0x7f62a4159b10 2023-10-10T20:35:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.029+0000 7f62a27fc700 1 -- 172.21.15.89:0/3493791352 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629400ba10 con 0x7f62a4159b10 2023-10-10T20:35:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.029+0000 7f62a27fc700 1 -- 172.21.15.89:0/3493791352 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629400bdb0 con 0x7f62a4159b10 2023-10-10T20:35:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.030+0000 7f62a9a10700 1 -- 172.21.15.89:0/3493791352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a4159b10 msgr2=0x7f62a40cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.030+0000 7f62a9a10700 1 --2- 172.21.15.89:0/3493791352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a4159b10 0x7f62a40cb3e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f62a406cce0 tx=0x7f6294009250 comp rx=0 tx=0).stop 2023-10-10T20:35:49.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.030+0000 7f62a9a10700 1 -- 172.21.15.89:0/3493791352 shutdown_connections 2023-10-10T20:35:49.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.030+0000 7f62a9a10700 1 --2- 172.21.15.89:0/3493791352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a4159b10 0x7f62a40cb3e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:49.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.030+0000 7f62a9a10700 1 --2- 172.21.15.89:0/3493791352 >> v2:172.21.15.111:3300/0 conn(0x7f62a415d730 0x7f62a415dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:49.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.030+0000 7f62a9a10700 1 -- 172.21.15.89:0/3493791352 >> 172.21.15.89:0/3493791352 conn(0x7f62a40c9870 msgr2=0x7f62a40c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:49.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.030+0000 7f62a9a10700 1 -- 172.21.15.89:0/3493791352 shutdown_connections 2023-10-10T20:35:49.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.030+0000 7f62a9a10700 1 -- 172.21.15.89:0/3493791352 wait complete. 2023-10-10T20:35:49.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.031+0000 7f62a9a10700 1 Processor -- start 2023-10-10T20:35:49.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.031+0000 7f62a9a10700 1 -- start start 2023-10-10T20:35:49.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f62a9a10700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a4159b10 0x7f62a415b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:49.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f62a9a10700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f62a4159ee0 0x7f62a41498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:49.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f62a9a10700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a415d730 0x7f62a40b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:49.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f62a9a10700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a40c5b50 con 0x7f62a415d730 2023-10-10T20:35:49.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f62a9a10700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a4149e10 con 0x7f62a4159b10 2023-10-10T20:35:49.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f62a9a10700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a40b1230 con 0x7f62a4159ee0 2023-10-10T20:35:49.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f62a37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a415d730 0x7f62a40b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:49.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f62a37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a415d730 0x7f62a40b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60322/0 (socket says 172.21.15.89:60322) 2023-10-10T20:35:49.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f62a37fe700 1 -- 172.21.15.89:0/4028896044 learned_addr learned my addr 172.21.15.89:0/4028896044 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:49.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f62a2ffd700 1 --2- 172.21.15.89:0/4028896044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a4159b10 0x7f62a415b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:49.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f629a7fc700 1 --2- 172.21.15.89:0/4028896044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f62a4159ee0 0x7f62a41498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:49.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f62a37fe700 1 -- 172.21.15.89:0/4028896044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f62a4159ee0 msgr2=0x7f62a41498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:49.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f62a37fe700 1 --2- 172.21.15.89:0/4028896044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f62a4159ee0 0x7f62a41498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:49.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f62a37fe700 1 -- 172.21.15.89:0/4028896044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a4159b10 msgr2=0x7f62a415b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:49.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f62a37fe700 1 --2- 172.21.15.89:0/4028896044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a4159b10 0x7f62a415b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:49.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f62a37fe700 1 -- 172.21.15.89:0/4028896044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62940094f0 con 0x7f62a415d730 2023-10-10T20:35:49.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f629a7fc700 1 --2- 172.21.15.89:0/4028896044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f62a4159ee0 0x7f62a41498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:49.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f62a37fe700 1 --2- 172.21.15.89:0/4028896044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a415d730 0x7f62a40b2070 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c930 tx=0x7f629c0026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:49.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f629b7fe700 1 -- 172.21.15.89:0/4028896044 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c00ce70 con 0x7f62a415d730 2023-10-10T20:35:49.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f629b7fe700 1 -- 172.21.15.89:0/4028896044 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c00a7e0 con 0x7f62a415d730 2023-10-10T20:35:49.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f62a9a10700 1 -- 172.21.15.89:0/4028896044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a40b1510 con 0x7f62a415d730 2023-10-10T20:35:49.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.032+0000 7f629b7fe700 1 -- 172.21.15.89:0/4028896044 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c00aba0 con 0x7f62a415d730 2023-10-10T20:35:49.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.033+0000 7f62a9a10700 1 -- 172.21.15.89:0/4028896044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a40c57e0 con 0x7f62a415d730 2023-10-10T20:35:49.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.033+0000 7f62a9a10700 1 -- 172.21.15.89:0/4028896044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a404c3c0 con 0x7f62a415d730 2023-10-10T20:35:49.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.034+0000 7f629b7fe700 1 -- 172.21.15.89:0/4028896044 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f629c012420 con 0x7f62a415d730 2023-10-10T20:35:49.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.034+0000 7f629b7fe700 1 --2- 172.21.15.89:0/4028896044 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6284066f30 0x7f62840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:49.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.034+0000 7f629b7fe700 1 -- 172.21.15.89:0/4028896044 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f629c084660 con 0x7f62a415d730 2023-10-10T20:35:49.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.034+0000 7f62a2ffd700 1 --2- 172.21.15.89:0/4028896044 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6284066f30 0x7f62840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:49.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.035+0000 7f62a2ffd700 1 --2- 172.21.15.89:0/4028896044 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6284066f30 0x7f62840693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f62a406cce0 tx=0x7f6294016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:49.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.038+0000 7f629b7fe700 1 -- 172.21.15.89:0/4028896044 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f629c0556b0 con 0x7f62a415d730 2023-10-10T20:35:49.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.300+0000 7f62a9a10700 1 -- 172.21.15.89:0/4028896044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]} v 0) v1 -- 0x7f62a40c5c90 con 0x7f62a415d730 2023-10-10T20:35:49.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.897+0000 7f629b7fe700 1 -- 172.21.15.89:0/4028896044 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]}]=0 v87) v1 ==== 136+0+0 (secure 0 0 0) 0x7f629c055240 con 0x7f62a415d730 2023-10-10T20:35:49.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.900+0000 7f62a9a10700 1 -- 172.21.15.89:0/4028896044 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6284066f30 msgr2=0x7f62840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:49.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.900+0000 7f62a9a10700 1 --2- 172.21.15.89:0/4028896044 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6284066f30 0x7f62840693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f62a406cce0 tx=0x7f6294016000 comp rx=0 tx=0).stop 2023-10-10T20:35:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.900+0000 7f62a9a10700 1 -- 172.21.15.89:0/4028896044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a415d730 msgr2=0x7f62a40b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.900+0000 7f62a9a10700 1 --2- 172.21.15.89:0/4028896044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a415d730 0x7f62a40b2070 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c930 tx=0x7f629c0026e0 comp rx=0 tx=0).stop 2023-10-10T20:35:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.900+0000 7f62a9a10700 1 -- 172.21.15.89:0/4028896044 shutdown_connections 2023-10-10T20:35:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.900+0000 7f62a9a10700 1 --2- 172.21.15.89:0/4028896044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a415d730 0x7f62a40b2070 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.900+0000 7f62a9a10700 1 --2- 172.21.15.89:0/4028896044 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6284066f30 0x7f62840693e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:49.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.900+0000 7f62a9a10700 1 --2- 172.21.15.89:0/4028896044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a4159b10 0x7f62a415b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:49.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.900+0000 7f62a9a10700 1 --2- 172.21.15.89:0/4028896044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f62a4159ee0 0x7f62a41498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:49.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.900+0000 7f62a9a10700 1 -- 172.21.15.89:0/4028896044 >> 172.21.15.89:0/4028896044 conn(0x7f62a40c9870 msgr2=0x7f62a4068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:49.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.900+0000 7f62a9a10700 1 -- 172.21.15.89:0/4028896044 shutdown_connections 2023-10-10T20:35:49.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:49.900+0000 7f62a9a10700 1 -- 172.21.15.89:0/4028896044 wait complete. 2023-10-10T20:35:49.916 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:35:50.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.033+0000 7f99aafa5700 1 Processor -- start 2023-10-10T20:35:50.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.033+0000 7f99aafa5700 1 -- start start 2023-10-10T20:35:50.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.034+0000 7f99aafa5700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f99a4159890 0x7f99a415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:50.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.034+0000 7f99aafa5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99a4159c60 0x7f99a415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:50.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.034+0000 7f99aafa5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f99a415db10 0x7f99a4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:50.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.034+0000 7f99aafa5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a415a160 con 0x7f99a415db10 2023-10-10T20:35:50.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.034+0000 7f99aafa5700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a40d13a0 con 0x7f99a4159890 2023-10-10T20:35:50.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.034+0000 7f99aafa5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a415e140 con 0x7f99a4159c60 2023-10-10T20:35:50.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.034+0000 7f99a9542700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f99a415db10 0x7f99a4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:50.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.034+0000 7f99a9542700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f99a415db10 0x7f99a4069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60334/0 (socket says 172.21.15.89:60334) 2023-10-10T20:35:50.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.034+0000 7f99a9542700 1 -- 172.21.15.89:0/2813048193 learned_addr learned my addr 172.21.15.89:0/2813048193 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:50.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.034+0000 7f99a8d41700 1 --2- 172.21.15.89:0/2813048193 >> v2:172.21.15.111:3300/0 conn(0x7f99a4159890 0x7f99a415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:50.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.034+0000 7f999bfff700 1 --2- 172.21.15.89:0/2813048193 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99a4159c60 0x7f99a415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:50.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.034+0000 7f99a9542700 1 -- 172.21.15.89:0/2813048193 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99a4159c60 msgr2=0x7f99a415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:50.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.034+0000 7f99a9542700 1 --2- 172.21.15.89:0/2813048193 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99a4159c60 0x7f99a415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.034+0000 7f99a9542700 1 -- 172.21.15.89:0/2813048193 >> v2:172.21.15.111:3300/0 conn(0x7f99a4159890 msgr2=0x7f99a415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:50.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.034+0000 7f99a9542700 1 --2- 172.21.15.89:0/2813048193 >> v2:172.21.15.111:3300/0 conn(0x7f99a4159890 0x7f99a415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.034+0000 7f99a9542700 1 -- 172.21.15.89:0/2813048193 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a415a3d0 con 0x7f99a415db10 2023-10-10T20:35:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.035+0000 7f99a9542700 1 --2- 172.21.15.89:0/2813048193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a415db10 0x7f99a4069560 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f99a40d1090 tx=0x7f999c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c09ac7fae16ab3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.035+0000 7f999b7fe700 1 -- 172.21.15.89:0/2813048193 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c00b860 con 0x7f99a415db10 2023-10-10T20:35:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.035+0000 7f999b7fe700 1 -- 172.21.15.89:0/2813048193 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999c00b9c0 con 0x7f99a415db10 2023-10-10T20:35:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.035+0000 7f999b7fe700 1 -- 172.21.15.89:0/2813048193 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c00bd60 con 0x7f99a415db10 2023-10-10T20:35:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.035+0000 7f99aafa5700 1 -- 172.21.15.89:0/2813048193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a415db10 msgr2=0x7f99a4069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:50.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.035+0000 7f99aafa5700 1 --2- 172.21.15.89:0/2813048193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a415db10 0x7f99a4069560 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f99a40d1090 tx=0x7f999c009250 comp rx=0 tx=0).stop 2023-10-10T20:35:50.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.035+0000 7f99aafa5700 1 -- 172.21.15.89:0/2813048193 shutdown_connections 2023-10-10T20:35:50.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.035+0000 7f99aafa5700 1 --2- 172.21.15.89:0/2813048193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a415db10 0x7f99a4069560 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.035+0000 7f99aafa5700 1 --2- 172.21.15.89:0/2813048193 >> v2:172.21.15.111:3300/0 conn(0x7f99a4159890 0x7f99a415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.035+0000 7f99aafa5700 1 --2- 172.21.15.89:0/2813048193 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99a4159c60 0x7f99a415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.035+0000 7f99aafa5700 1 -- 172.21.15.89:0/2813048193 >> 172.21.15.89:0/2813048193 conn(0x7f99a4068e20 msgr2=0x7f99a40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:50.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.035+0000 7f99aafa5700 1 -- 172.21.15.89:0/2813048193 shutdown_connections 2023-10-10T20:35:50.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.035+0000 7f99aafa5700 1 -- 172.21.15.89:0/2813048193 wait complete. 2023-10-10T20:35:50.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.036+0000 7f99aafa5700 1 Processor -- start 2023-10-10T20:35:50.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.037+0000 7f99aafa5700 1 -- start start 2023-10-10T20:35:50.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.037+0000 7f99aafa5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a4159890 0x7f99a414c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:50.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.037+0000 7f99aafa5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99a4159c60 0x7f99a4152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:50.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.037+0000 7f99aafa5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99a415db10 0x7f99a415e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:50.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.037+0000 7f99aafa5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a40b1340 con 0x7f99a4159890 2023-10-10T20:35:50.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.037+0000 7f99aafa5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a40b1620 con 0x7f99a415db10 2023-10-10T20:35:50.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.037+0000 7f99aafa5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a40b1900 con 0x7f99a4159c60 2023-10-10T20:35:50.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.037+0000 7f99a8d41700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a4159890 0x7f99a414c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:50.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.037+0000 7f99a8d41700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a4159890 0x7f99a414c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60340/0 (socket says 172.21.15.89:60340) 2023-10-10T20:35:50.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.037+0000 7f99a8d41700 1 -- 172.21.15.89:0/428422402 learned_addr learned my addr 172.21.15.89:0/428422402 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:50.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.038+0000 7f99a9542700 1 --2- 172.21.15.89:0/428422402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99a415db10 0x7f99a415e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:50.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.038+0000 7f999bfff700 1 --2- 172.21.15.89:0/428422402 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99a4159c60 0x7f99a4152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:50.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.038+0000 7f99a8d41700 1 -- 172.21.15.89:0/428422402 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99a4159c60 msgr2=0x7f99a4152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:50.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.038+0000 7f99a8d41700 1 --2- 172.21.15.89:0/428422402 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99a4159c60 0x7f99a4152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.038+0000 7f99a8d41700 1 -- 172.21.15.89:0/428422402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99a415db10 msgr2=0x7f99a415e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:50.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.038+0000 7f99a8d41700 1 --2- 172.21.15.89:0/428422402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99a415db10 0x7f99a415e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.038+0000 7f99a8d41700 1 -- 172.21.15.89:0/428422402 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999c0094f0 con 0x7f99a4159890 2023-10-10T20:35:50.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.038+0000 7f99a9542700 1 --2- 172.21.15.89:0/428422402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99a415db10 0x7f99a415e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:50.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.038+0000 7f99a8d41700 1 --2- 172.21.15.89:0/428422402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a4159890 0x7f99a414c1a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f99a000cfc0 tx=0x7f99a00078b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:50.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.038+0000 7f9998ff9700 1 -- 172.21.15.89:0/428422402 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a0016430 con 0x7f99a4159890 2023-10-10T20:35:50.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.038+0000 7f9998ff9700 1 -- 172.21.15.89:0/428422402 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a0016590 con 0x7f99a4159890 2023-10-10T20:35:50.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.038+0000 7f99aafa5700 1 -- 172.21.15.89:0/428422402 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a40b1be0 con 0x7f99a4159890 2023-10-10T20:35:50.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.038+0000 7f9998ff9700 1 -- 172.21.15.89:0/428422402 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a0016930 con 0x7f99a4159890 2023-10-10T20:35:50.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.038+0000 7f99aafa5700 1 -- 172.21.15.89:0/428422402 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a40b0fd0 con 0x7f99a4159890 2023-10-10T20:35:50.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.039+0000 7f99aafa5700 1 -- 172.21.15.89:0/428422402 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a404c3c0 con 0x7f99a4159890 2023-10-10T20:35:50.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.039+0000 7f9998ff9700 1 -- 172.21.15.89:0/428422402 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f99a0003020 con 0x7f99a4159890 2023-10-10T20:35:50.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.040+0000 7f9998ff9700 1 --2- 172.21.15.89:0/428422402 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9990066f30 0x7f99900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:50.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.040+0000 7f9998ff9700 1 -- 172.21.15.89:0/428422402 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f99a0012070 con 0x7f99a4159890 2023-10-10T20:35:50.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.040+0000 7f999bfff700 1 --2- 172.21.15.89:0/428422402 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9990066f30 0x7f99900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:50.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.040+0000 7f999bfff700 1 --2- 172.21.15.89:0/428422402 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9990066f30 0x7f99900693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f99a40b0330 tx=0x7f9994008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:50.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.044+0000 7f9998ff9700 1 -- 172.21.15.89:0/428422402 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99a00526f0 con 0x7f99a4159890 2023-10-10T20:35:50.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.227+0000 7f99aafa5700 1 -- 172.21.15.89:0/428422402 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f99a4153420 con 0x7f99a4159890 2023-10-10T20:35:50.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.231+0000 7f9998ff9700 1 -- 172.21.15.89:0/428422402 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f99a0055d10 con 0x7f99a4159890 2023-10-10T20:35:50.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.233+0000 7f99aafa5700 1 -- 172.21.15.89:0/428422402 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9990066f30 msgr2=0x7f99900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:50.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.233+0000 7f99aafa5700 1 --2- 172.21.15.89:0/428422402 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9990066f30 0x7f99900693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f99a40b0330 tx=0x7f9994008040 comp rx=0 tx=0).stop 2023-10-10T20:35:50.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.233+0000 7f99aafa5700 1 -- 172.21.15.89:0/428422402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a4159890 msgr2=0x7f99a414c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:50.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.233+0000 7f99aafa5700 1 --2- 172.21.15.89:0/428422402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a4159890 0x7f99a414c1a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f99a000cfc0 tx=0x7f99a00078b0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.233+0000 7f99aafa5700 1 -- 172.21.15.89:0/428422402 shutdown_connections 2023-10-10T20:35:50.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.233+0000 7f99aafa5700 1 --2- 172.21.15.89:0/428422402 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9990066f30 0x7f99900693e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.233+0000 7f99aafa5700 1 --2- 172.21.15.89:0/428422402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99a415db10 0x7f99a415e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.233+0000 7f99aafa5700 1 --2- 172.21.15.89:0/428422402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a4159890 0x7f99a414c1a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.233+0000 7f99aafa5700 1 --2- 172.21.15.89:0/428422402 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99a4159c60 0x7f99a4152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.233+0000 7f99aafa5700 1 -- 172.21.15.89:0/428422402 >> 172.21.15.89:0/428422402 conn(0x7f99a4068e20 msgr2=0x7f99a4154eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:50.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.233+0000 7f99aafa5700 1 -- 172.21.15.89:0/428422402 shutdown_connections 2023-10-10T20:35:50.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.233+0000 7f99aafa5700 1 -- 172.21.15.89:0/428422402 wait complete. 2023-10-10T20:35:50.247 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:35:50.248 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:35:50.287 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:35:50.339 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:35:50.339 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:35:50.340 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:35:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.452+0000 7fe5fe048700 1 Processor -- start 2023-10-10T20:35:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.452+0000 7fe5fe048700 1 -- start start 2023-10-10T20:35:50.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.453+0000 7fe5fe048700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe5f8159420 0x7fe5f80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.453+0000 7fe5fe048700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f80b2980 con 0x7fe5f815d230 2023-10-10T20:35:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.453+0000 7fe5fe048700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f80b11d0 con 0x7fe5f81597f0 2023-10-10T20:35:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.453+0000 7fe5fe048700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f80b14b0 con 0x7fe5f8159420 2023-10-10T20:35:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.453+0000 7fe5fd046700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe5f8159420 0x7fe5f80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:50.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.453+0000 7fe5fd847700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe5f815d230 0x7fe5f815d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52698/0 (socket says 172.21.15.89:52698) 2023-10-10T20:35:50.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.453+0000 7fe5fd847700 1 -- 172.21.15.89:0/2326644320 learned_addr learned my addr 172.21.15.89:0/2326644320 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:50.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.453+0000 7fe5fd046700 1 -- 172.21.15.89:0/2326644320 >> v1:172.21.15.111:6789/0 conn(0x7fe5f81597f0 legacy=0x7fe5f80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:50.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.453+0000 7fe5fd046700 1 -- 172.21.15.89:0/2326644320 >> v1:172.21.15.89:6789/0 conn(0x7fe5f815d230 legacy=0x7fe5f815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:50.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.453+0000 7fe5effff700 1 -- 172.21.15.89:0/2326644320 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049358754 0 0) 0x7fe5f80b2980 con 0x7fe5f815d230 2023-10-10T20:35:50.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.453+0000 7fe5fd046700 1 -- 172.21.15.89:0/2326644320 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f80b1790 con 0x7fe5f8159420 2023-10-10T20:35:50.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.454+0000 7fe5fd046700 1 --2- 172.21.15.89:0/2326644320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5f8159420 0x7fe5f80b5f40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40098e0 tx=0x7fe5f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905f3715f4b88dfe server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:50.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.454+0000 7fe5effff700 1 -- 172.21.15.89:0/2326644320 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4009e80 con 0x7fe5f8159420 2023-10-10T20:35:50.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.454+0000 7fe5effff700 1 -- 172.21.15.89:0/2326644320 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f400b860 con 0x7fe5f8159420 2023-10-10T20:35:50.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.454+0000 7fe5effff700 1 -- 172.21.15.89:0/2326644320 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f400bbe0 con 0x7fe5f8159420 2023-10-10T20:35:50.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.454+0000 7fe5fe048700 1 -- 172.21.15.89:0/2326644320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5f8159420 msgr2=0x7fe5f80b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:50.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.454+0000 7fe5fe048700 1 --2- 172.21.15.89:0/2326644320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5f8159420 0x7fe5f80b5f40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40098e0 tx=0x7fe5f40091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.454+0000 7fe5fe048700 1 -- 172.21.15.89:0/2326644320 shutdown_connections 2023-10-10T20:35:50.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.454+0000 7fe5fe048700 1 --2- 172.21.15.89:0/2326644320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5f8159420 0x7fe5f80b5f40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.454+0000 7fe5fe048700 1 -- 172.21.15.89:0/2326644320 >> 172.21.15.89:0/2326644320 conn(0x7fe5f80d0660 msgr2=0x7fe5f80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:50.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.454+0000 7fe5fe048700 1 -- 172.21.15.89:0/2326644320 shutdown_connections 2023-10-10T20:35:50.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.454+0000 7fe5fe048700 1 -- 172.21.15.89:0/2326644320 wait complete. 2023-10-10T20:35:50.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.455+0000 7fe5fe048700 1 Processor -- start 2023-10-10T20:35:50.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.455+0000 7fe5fe048700 1 -- start start 2023-10-10T20:35:50.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.456+0000 7fe5fe048700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe5f8159420 0x7fe5f80c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:50.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.456+0000 7fe5fe048700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5f81597f0 0x7fe5f8149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:50.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.456+0000 7fe5fe048700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5f815d230 0x7fe5f814a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:50.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.456+0000 7fe5fe048700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f815b0f0 con 0x7fe5f815d230 2023-10-10T20:35:50.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.456+0000 7fe5fe048700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f815b3d0 con 0x7fe5f81597f0 2023-10-10T20:35:50.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.456+0000 7fe5fe048700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f815b6b0 con 0x7fe5f8159420 2023-10-10T20:35:50.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.456+0000 7fe5fd847700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5f815d230 0x7fe5f814a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:50.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.456+0000 7fe5fd847700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5f815d230 0x7fe5f814a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60366/0 (socket says 172.21.15.89:60366) 2023-10-10T20:35:50.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.456+0000 7fe5fd847700 1 -- 172.21.15.89:0/3297747501 learned_addr learned my addr 172.21.15.89:0/3297747501 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:50.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.457+0000 7fe5fc845700 1 --2- 172.21.15.89:0/3297747501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5f81597f0 0x7fe5f8149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:50.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.457+0000 7fe5fd046700 1 --2- 172.21.15.89:0/3297747501 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe5f8159420 0x7fe5f80c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:50.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.457+0000 7fe5fd847700 1 -- 172.21.15.89:0/3297747501 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe5f8159420 msgr2=0x7fe5f80c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:50.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.457+0000 7fe5fd847700 1 --2- 172.21.15.89:0/3297747501 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe5f8159420 0x7fe5f80c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.457+0000 7fe5fd847700 1 -- 172.21.15.89:0/3297747501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5f81597f0 msgr2=0x7fe5f8149e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:50.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.457+0000 7fe5fd847700 1 --2- 172.21.15.89:0/3297747501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5f81597f0 0x7fe5f8149e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.457+0000 7fe5fd847700 1 -- 172.21.15.89:0/3297747501 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f4009420 con 0x7fe5f815d230 2023-10-10T20:35:50.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.457+0000 7fe5fd046700 1 --2- 172.21.15.89:0/3297747501 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe5f8159420 0x7fe5f80c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:50.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.457+0000 7fe5fd847700 1 --2- 172.21.15.89:0/3297747501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5f815d230 0x7fe5f814a3a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000cbc0 tx=0x7fe5f0001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:50.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.457+0000 7fe5ed7fa700 1 -- 172.21.15.89:0/3297747501 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f000a940 con 0x7fe5f815d230 2023-10-10T20:35:50.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.457+0000 7fe5ed7fa700 1 -- 172.21.15.89:0/3297747501 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f000aaa0 con 0x7fe5f815d230 2023-10-10T20:35:50.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.457+0000 7fe5fe048700 1 -- 172.21.15.89:0/3297747501 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f815b990 con 0x7fe5f815d230 2023-10-10T20:35:50.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.457+0000 7fe5ed7fa700 1 -- 172.21.15.89:0/3297747501 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f000ae60 con 0x7fe5f815d230 2023-10-10T20:35:50.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.457+0000 7fe5fe048700 1 -- 172.21.15.89:0/3297747501 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f815a380 con 0x7fe5f815d230 2023-10-10T20:35:50.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.458+0000 7fe5fe048700 1 -- 172.21.15.89:0/3297747501 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f8069f80 con 0x7fe5f815d230 2023-10-10T20:35:50.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.458+0000 7fe5ed7fa700 1 -- 172.21.15.89:0/3297747501 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5f0007300 con 0x7fe5f815d230 2023-10-10T20:35:50.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.459+0000 7fe5ed7fa700 1 --2- 172.21.15.89:0/3297747501 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe5dc066f30 0x7fe5dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:50.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.459+0000 7fe5ed7fa700 1 -- 172.21.15.89:0/3297747501 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fe5f00846d0 con 0x7fe5f815d230 2023-10-10T20:35:50.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.459+0000 7fe5fd046700 1 --2- 172.21.15.89:0/3297747501 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe5dc066f30 0x7fe5dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:50.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.459+0000 7fe5fd046700 1 --2- 172.21.15.89:0/3297747501 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe5dc066f30 0x7fe5dc0693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe5f815be70 tx=0x7fe5f4017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:50.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.463+0000 7fe5ed7fa700 1 -- 172.21.15.89:0/3297747501 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5f0051be0 con 0x7fe5f815d230 2023-10-10T20:35:50.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.688+0000 7fe5fe048700 1 -- 172.21.15.89:0/3297747501 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5f804c3c0 con 0x7fe5f815d230 2023-10-10T20:35:50.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.689+0000 7fe5ed7fa700 1 -- 172.21.15.89:0/3297747501 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fe5f0055200 con 0x7fe5f815d230 2023-10-10T20:35:50.691 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:50.692 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1977066906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1977066906},{"type":"v1","addr":"172.21.15.111:6833","nonce":1977066906}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2904313060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2904313060},{"type":"v1","addr":"172.21.15.111:6835","nonce":2904313060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/4152616589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":4152616589},{"type":"v1","addr":"172.21.15.89:6836","nonce":4152616589}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5277,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2887201141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2887201141},{"type":"v1","addr":"172.21.15.89:6837","nonce":2887201141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[]} 2023-10-10T20:35:50.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.691+0000 7fe5fe048700 1 -- 172.21.15.89:0/3297747501 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe5dc066f30 msgr2=0x7fe5dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:50.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.691+0000 7fe5fe048700 1 --2- 172.21.15.89:0/3297747501 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe5dc066f30 0x7fe5dc0693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe5f815be70 tx=0x7fe5f4017000 comp rx=0 tx=0).stop 2023-10-10T20:35:50.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.691+0000 7fe5fe048700 1 -- 172.21.15.89:0/3297747501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5f815d230 msgr2=0x7fe5f814a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:50.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.691+0000 7fe5fe048700 1 --2- 172.21.15.89:0/3297747501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5f815d230 0x7fe5f814a3a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000cbc0 tx=0x7fe5f0001980 comp rx=0 tx=0).stop 2023-10-10T20:35:50.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.691+0000 7fe5fe048700 1 -- 172.21.15.89:0/3297747501 shutdown_connections 2023-10-10T20:35:50.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.691+0000 7fe5fe048700 1 --2- 172.21.15.89:0/3297747501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5f815d230 0x7fe5f814a3a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.691+0000 7fe5fe048700 1 --2- 172.21.15.89:0/3297747501 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe5f8159420 0x7fe5f80c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.691+0000 7fe5fe048700 1 --2- 172.21.15.89:0/3297747501 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe5dc066f30 0x7fe5dc0693e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.691+0000 7fe5fe048700 1 --2- 172.21.15.89:0/3297747501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5f81597f0 0x7fe5f8149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.691+0000 7fe5fe048700 1 -- 172.21.15.89:0/3297747501 >> 172.21.15.89:0/3297747501 conn(0x7fe5f80d0660 msgr2=0x7fe5f8068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:50.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.691+0000 7fe5fe048700 1 -- 172.21.15.89:0/3297747501 shutdown_connections 2023-10-10T20:35:50.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.691+0000 7fe5fe048700 1 -- 172.21.15.89:0/3297747501 wait complete. 2023-10-10T20:35:50.696 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 24 2023-10-10T20:35:50.705 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:35:50.706 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:35:50.706 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:35:50.706 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:35:50.707 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:35:50.707 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:35:50.707 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:35:50.708 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:35:50.708 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:35:50.708 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:35:50.708 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:35:50.709 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:35:50.709 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:35:50.709 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:35:50.710 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:35:50.710 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:35:50.715 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:35:50.716 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:35:50.717 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:35:50.717 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:35:50.720 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:35:50.720 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:35:50.721 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:35:50.721 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:35:50.758 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:35:50.759 DEBUG:teuthology.parallel:result is None 2023-10-10T20:35:50.760 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:35:50.760 DEBUG:teuthology.parallel:result is None 2023-10-10T20:35:50.762 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:35:50.763 DEBUG:teuthology.parallel:result is None 2023-10-10T20:35:50.763 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:35:50.764 DEBUG:teuthology.parallel:result is None 2023-10-10T20:35:50.764 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:35:50.830 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-10T20:35:50.831 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:35:50.844 INFO:tasks.ceph.mds.b.smithi111.stdout:starting mds.b at 2023-10-10T20:35:50.844 INFO:tasks.ceph.mds.d.smithi111.stdout:starting mds.d at 2023-10-10T20:35:50.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.883+0000 7f67b459c700 1 Processor -- start 2023-10-10T20:35:50.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.883+0000 7f67b459c700 1 -- start start 2023-10-10T20:35:50.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.884+0000 7f67b459c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67ac159890 0x7f67ac15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:50.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.884+0000 7f67b459c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f67ac069560 0x7f67ac0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:50.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.884+0000 7f67b459c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ac15a160 con 0x7f67ac159c60 2023-10-10T20:35:50.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.884+0000 7f67b459c700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67ac069f20 con 0x7f67ac069560 2023-10-10T20:35:50.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.884+0000 7f67b459c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac0d13a0 con 0x7f67ac159890 2023-10-10T20:35:50.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.884+0000 7f67b2b39700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f67ac069560 0x7f67ac0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:50.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.884+0000 7f67b1b37700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f67ac159c60 0x7f67ac15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:35538/0 (socket says 172.21.15.89:35538) 2023-10-10T20:35:50.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.884+0000 7f67b1b37700 1 -- 172.21.15.89:0/3128452352 learned_addr learned my addr 172.21.15.89:0/3128452352 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:50.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.884+0000 7f67b2338700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67ac159890 0x7f67ac15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:50.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.884+0000 7f67b2b39700 1 -- 172.21.15.89:0/3128452352 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67ac159890 msgr2=0x7f67ac15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:50.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.884+0000 7f67b2b39700 1 --2- 172.21.15.89:0/3128452352 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67ac159890 0x7f67ac15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.884+0000 7f67b2b39700 1 -- 172.21.15.89:0/3128452352 >> v1:172.21.15.111:6789/0 conn(0x7f67ac159c60 legacy=0x7f67ac15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:50.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.884+0000 7f67b1336700 1 -- 172.21.15.89:0/3128452352 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2207274885 0 0) 0x7f67ac15a160 con 0x7f67ac159c60 2023-10-10T20:35:50.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.884+0000 7f67b2b39700 1 -- 172.21.15.89:0/3128452352 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac15e140 con 0x7f67ac069560 2023-10-10T20:35:50.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.885+0000 7f67b2b39700 1 --2- 172.21.15.89:0/3128452352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67ac069560 0x7f67ac0699e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009e60 tx=0x7f67a4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3c083db19228796 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:50.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.885+0000 7f67b1336700 1 -- 172.21.15.89:0/3128452352 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a400e070 con 0x7f67ac069560 2023-10-10T20:35:50.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.885+0000 7f67b1336700 1 -- 172.21.15.89:0/3128452352 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a400bb60 con 0x7f67ac069560 2023-10-10T20:35:50.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.885+0000 7f67b1336700 1 -- 172.21.15.89:0/3128452352 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a4012430 con 0x7f67ac069560 2023-10-10T20:35:50.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.885+0000 7f67b459c700 1 -- 172.21.15.89:0/3128452352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67ac069560 msgr2=0x7f67ac0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:50.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.885+0000 7f67b459c700 1 --2- 172.21.15.89:0/3128452352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67ac069560 0x7f67ac0699e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009e60 tx=0x7f67a4009780 comp rx=0 tx=0).stop 2023-10-10T20:35:50.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.885+0000 7f67b459c700 1 -- 172.21.15.89:0/3128452352 shutdown_connections 2023-10-10T20:35:50.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.885+0000 7f67b459c700 1 --2- 172.21.15.89:0/3128452352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67ac069560 0x7f67ac0699e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.885+0000 7f67b459c700 1 --2- 172.21.15.89:0/3128452352 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67ac159890 0x7f67ac15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.885+0000 7f67b459c700 1 -- 172.21.15.89:0/3128452352 >> 172.21.15.89:0/3128452352 conn(0x7f67ac068e20 msgr2=0x7f67ac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:50.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.885+0000 7f67b459c700 1 -- 172.21.15.89:0/3128452352 shutdown_connections 2023-10-10T20:35:50.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.885+0000 7f67b459c700 1 -- 172.21.15.89:0/3128452352 wait complete. 2023-10-10T20:35:50.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.886+0000 7f67b459c700 1 Processor -- start 2023-10-10T20:35:50.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.886+0000 7f67b459c700 1 -- start start 2023-10-10T20:35:50.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.887+0000 7f67b459c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67ac069560 0x7f67ac1492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:50.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.887+0000 7f67b459c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67ac159890 0x7f67ac15cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:50.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.887+0000 7f67b459c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67ac159c60 0x7f67ac0d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:50.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.887+0000 7f67b459c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac0b4740 con 0x7f67ac159890 2023-10-10T20:35:50.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.887+0000 7f67b459c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac1497e0 con 0x7f67ac159c60 2023-10-10T20:35:50.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.887+0000 7f67b459c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac0b3b30 con 0x7f67ac069560 2023-10-10T20:35:50.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.887+0000 7f67b1b37700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67ac159890 0x7f67ac15cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:50.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.887+0000 7f67b1b37700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67ac159890 0x7f67ac15cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60414/0 (socket says 172.21.15.89:60414) 2023-10-10T20:35:50.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.887+0000 7f67b1b37700 1 -- 172.21.15.89:0/4018237346 learned_addr learned my addr 172.21.15.89:0/4018237346 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:50.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.887+0000 7f67b2338700 1 --2- 172.21.15.89:0/4018237346 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67ac069560 0x7f67ac1492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:50.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.887+0000 7f67b2b39700 1 --2- 172.21.15.89:0/4018237346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67ac159c60 0x7f67ac0d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:50.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.887+0000 7f67b1b37700 1 -- 172.21.15.89:0/4018237346 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67ac069560 msgr2=0x7f67ac1492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:50.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.887+0000 7f67b1b37700 1 --2- 172.21.15.89:0/4018237346 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67ac069560 0x7f67ac1492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.888+0000 7f67b1b37700 1 -- 172.21.15.89:0/4018237346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67ac159c60 msgr2=0x7f67ac0d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:50.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.888+0000 7f67b1b37700 1 --2- 172.21.15.89:0/4018237346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67ac159c60 0x7f67ac0d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:50.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.888+0000 7f67b1b37700 1 -- 172.21.15.89:0/4018237346 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a4009a20 con 0x7f67ac159890 2023-10-10T20:35:50.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.888+0000 7f67b1b37700 1 --2- 172.21.15.89:0/4018237346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67ac159890 0x7f67ac15cf10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f679c00c810 tx=0x7f679c007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:50.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.888+0000 7f67a27fc700 1 -- 172.21.15.89:0/4018237346 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c005c00 con 0x7f67ac159890 2023-10-10T20:35:50.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.888+0000 7f67a27fc700 1 -- 172.21.15.89:0/4018237346 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679c005d60 con 0x7f67ac159890 2023-10-10T20:35:50.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.888+0000 7f67b459c700 1 -- 172.21.15.89:0/4018237346 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ac0b3e10 con 0x7f67ac159890 2023-10-10T20:35:50.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.888+0000 7f67a27fc700 1 -- 172.21.15.89:0/4018237346 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c007300 con 0x7f67ac159890 2023-10-10T20:35:50.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.888+0000 7f67b459c700 1 -- 172.21.15.89:0/4018237346 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ac0b43d0 con 0x7f67ac159890 2023-10-10T20:35:50.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.889+0000 7f67b459c700 1 -- 172.21.15.89:0/4018237346 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67ac0b48b0 con 0x7f67ac159890 2023-10-10T20:35:50.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.889+0000 7f67a27fc700 1 -- 172.21.15.89:0/4018237346 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f679c01b3b0 con 0x7f67ac159890 2023-10-10T20:35:50.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.889+0000 7f67a27fc700 1 --2- 172.21.15.89:0/4018237346 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6790066f30 0x7f67900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:50.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.890+0000 7f67a27fc700 1 -- 172.21.15.89:0/4018237346 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f679c0837d0 con 0x7f67ac159890 2023-10-10T20:35:50.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.890+0000 7f67b2338700 1 --2- 172.21.15.89:0/4018237346 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6790066f30 0x7f67900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:50.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.890+0000 7f67b2338700 1 --2- 172.21.15.89:0/4018237346 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6790066f30 0x7f67900693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009db0 tx=0x7f67a80091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:50.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:50.894+0000 7f67a27fc700 1 -- 172.21.15.89:0/4018237346 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f679c050c40 con 0x7f67ac159890 2023-10-10T20:35:51.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:51.092+0000 7f67b459c700 1 -- 172.21.15.89:0/4018237346 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f67ac04c3c0 con 0x7f67ac159890 2023-10-10T20:35:51.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:51.909+0000 7f67a27fc700 1 -- 172.21.15.89:0/4018237346 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v46) v1 ==== 97+0+0 (secure 0 0 0) 0x7f679c017080 con 0x7f67ac159890 2023-10-10T20:35:51.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:51.912+0000 7f67b459c700 1 -- 172.21.15.89:0/4018237346 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6790066f30 msgr2=0x7f67900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:51.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:51.912+0000 7f67b459c700 1 --2- 172.21.15.89:0/4018237346 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6790066f30 0x7f67900693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009db0 tx=0x7f67a80091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:51.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:51.912+0000 7f67b459c700 1 -- 172.21.15.89:0/4018237346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67ac159890 msgr2=0x7f67ac15cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:51.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:51.912+0000 7f67b459c700 1 --2- 172.21.15.89:0/4018237346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67ac159890 0x7f67ac15cf10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f679c00c810 tx=0x7f679c007db0 comp rx=0 tx=0).stop 2023-10-10T20:35:51.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:51.912+0000 7f67b459c700 1 -- 172.21.15.89:0/4018237346 shutdown_connections 2023-10-10T20:35:51.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:51.912+0000 7f67b459c700 1 --2- 172.21.15.89:0/4018237346 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6790066f30 0x7f67900693e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:51.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:51.912+0000 7f67b459c700 1 --2- 172.21.15.89:0/4018237346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67ac159c60 0x7f67ac0d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:51.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:51.912+0000 7f67b459c700 1 --2- 172.21.15.89:0/4018237346 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67ac069560 0x7f67ac1492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:51.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:51.912+0000 7f67b459c700 1 --2- 172.21.15.89:0/4018237346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67ac159890 0x7f67ac15cf10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:51.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:51.912+0000 7f67b459c700 1 -- 172.21.15.89:0/4018237346 >> 172.21.15.89:0/4018237346 conn(0x7f67ac068e20 msgr2=0x7f67ac1576d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:51.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:51.912+0000 7f67b459c700 1 -- 172.21.15.89:0/4018237346 shutdown_connections 2023-10-10T20:35:51.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:51.912+0000 7f67b459c700 1 -- 172.21.15.89:0/4018237346 wait complete. 2023-10-10T20:35:51.917 INFO:teuthology.orchestra.run.smithi089.stderr: removed all blocklist entries 2023-10-10T20:35:51.928 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:35:52.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.045+0000 7f56fe55b700 1 Processor -- start 2023-10-10T20:35:52.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.045+0000 7f56fe55b700 1 -- start start 2023-10-10T20:35:52.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56fe55b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f56f8159c60 0x7f56f815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56fe55b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f815a160 con 0x7f56f8159890 2023-10-10T20:35:52.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56fe55b700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f80d13a0 con 0x7f56f815db10 2023-10-10T20:35:52.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56fe55b700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f815df50 con 0x7f56f8159c60 2023-10-10T20:35:52.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56f7fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f56f8159890 0x7f56f815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52700/0 (socket says 172.21.15.89:52700) 2023-10-10T20:35:52.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56f7fff700 1 -- 172.21.15.89:0/2521323854 learned_addr learned my addr 172.21.15.89:0/2521323854 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:52.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56f6ffd700 1 -- 172.21.15.89:0/2521323854 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329642842 0 0) 0x7f56f815a160 con 0x7f56f8159890 2023-10-10T20:35:52.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56f6ffd700 1 -- 172.21.15.89:0/2521323854 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56e0003570 con 0x7f56f8159890 2023-10-10T20:35:52.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56f77fe700 1 --2- 172.21.15.89:0/2521323854 >> v2:172.21.15.111:3300/0 conn(0x7f56f8159c60 0x7f56f815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56f6ffd700 1 -- 172.21.15.89:0/2521323854 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2692215715 0 0) 0x7f56e0003570 con 0x7f56f8159890 2023-10-10T20:35:52.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56f6ffd700 1 -- 172.21.15.89:0/2521323854 >> v2:172.21.15.111:3300/0 conn(0x7f56f8159c60 msgr2=0x7f56f815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:52.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56f6ffd700 1 --2- 172.21.15.89:0/2521323854 >> v2:172.21.15.111:3300/0 conn(0x7f56f8159c60 0x7f56f815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56f6ffd700 1 -- 172.21.15.89:0/2521323854 >> v1:172.21.15.111:6789/0 conn(0x7f56f815db10 legacy=0x7f56f8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56f6ffd700 1 -- 172.21.15.89:0/2521323854 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f815e140 con 0x7f56f8159890 2023-10-10T20:35:52.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56f6ffd700 1 -- 172.21.15.89:0/2521323854 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f56e8001470 con 0x7f56f8159890 2023-10-10T20:35:52.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56f6ffd700 1 -- 172.21.15.89:0/2521323854 >> v1:172.21.15.89:6789/0 conn(0x7f56f8159890 legacy=0x7f56f815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56f6ffd700 1 --2- 172.21.15.89:0/2521323854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e0003a60 0x7f56e0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56f6ffd700 1 --2- 172.21.15.89:0/2521323854 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f56e0004810 0x7f56e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56f6ffd700 1 --2- 172.21.15.89:0/2521323854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56e00051c0 0x7f56e000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56f6ffd700 1 -- 172.21.15.89:0/2521323854 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e000c0a0 con 0x7f56e0003a60 2023-10-10T20:35:52.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56f6ffd700 1 -- 172.21.15.89:0/2521323854 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e000c210 con 0x7f56e00051c0 2023-10-10T20:35:52.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56f6ffd700 1 -- 172.21.15.89:0/2521323854 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e000c3d0 con 0x7f56e0004810 2023-10-10T20:35:52.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.046+0000 7f56f7fff700 1 --2- 172.21.15.89:0/2521323854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e0003a60 0x7f56e0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.047+0000 7f56fcaf8700 1 --2- 172.21.15.89:0/2521323854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56e00051c0 0x7f56e000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.047+0000 7f56f77fe700 1 --2- 172.21.15.89:0/2521323854 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f56e0004810 0x7f56e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.047+0000 7f56f7fff700 1 -- 172.21.15.89:0/2521323854 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f56e0004810 msgr2=0x7f56e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:52.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.047+0000 7f56f7fff700 1 --2- 172.21.15.89:0/2521323854 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f56e0004810 0x7f56e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.047+0000 7f56f7fff700 1 -- 172.21.15.89:0/2521323854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56e00051c0 msgr2=0x7f56e000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:52.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.047+0000 7f56f7fff700 1 --2- 172.21.15.89:0/2521323854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56e00051c0 0x7f56e000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.047+0000 7f56f7fff700 1 -- 172.21.15.89:0/2521323854 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e0003570 con 0x7f56e0003a60 2023-10-10T20:35:52.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.047+0000 7f56fcaf8700 1 -- 172.21.15.89:0/2521323854 reap_dead start 2023-10-10T20:35:52.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.047+0000 7f56f7fff700 1 --2- 172.21.15.89:0/2521323854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e0003a60 0x7f56e0003e70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f56e800c3c0 tx=0x7f56e8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8d293f72eb31d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:52.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.047+0000 7f56f6ffd700 1 -- 172.21.15.89:0/2521323854 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e800cc90 con 0x7f56e0003a60 2023-10-10T20:35:52.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.047+0000 7f56f6ffd700 1 -- 172.21.15.89:0/2521323854 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e800cdf0 con 0x7f56e0003a60 2023-10-10T20:35:52.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.047+0000 7f56f6ffd700 1 -- 172.21.15.89:0/2521323854 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8005b50 con 0x7f56e0003a60 2023-10-10T20:35:52.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.048+0000 7f56fe55b700 1 -- 172.21.15.89:0/2521323854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e0003a60 msgr2=0x7f56e0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.048+0000 7f56fe55b700 1 --2- 172.21.15.89:0/2521323854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e0003a60 0x7f56e0003e70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f56e800c3c0 tx=0x7f56e8001e50 comp rx=0 tx=0).stop 2023-10-10T20:35:52.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.048+0000 7f56fe55b700 1 -- 172.21.15.89:0/2521323854 shutdown_connections 2023-10-10T20:35:52.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.048+0000 7f56fe55b700 1 --2- 172.21.15.89:0/2521323854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e0003a60 0x7f56e0003e70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.048+0000 7f56fe55b700 1 -- 172.21.15.89:0/2521323854 >> 172.21.15.89:0/2521323854 conn(0x7f56f8068e20 msgr2=0x7f56f80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:52.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.048+0000 7f56fe55b700 1 -- 172.21.15.89:0/2521323854 shutdown_connections 2023-10-10T20:35:52.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.048+0000 7f56fe55b700 1 -- 172.21.15.89:0/2521323854 wait complete. 2023-10-10T20:35:52.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.049+0000 7f56fe55b700 1 Processor -- start 2023-10-10T20:35:52.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.049+0000 7f56fe55b700 1 -- start start 2023-10-10T20:35:52.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56fe55b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f56e0003a60 0x7f56f80b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56fe55b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56f80b6130 0x7f56f8169a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56fe55b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56f8169fa0 0x7f56f816a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56fe55b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f816a980 con 0x7f56f80b6130 2023-10-10T20:35:52.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56fe55b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80ce9c0 con 0x7f56f8169fa0 2023-10-10T20:35:52.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56fe55b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80cddb0 con 0x7f56e0003a60 2023-10-10T20:35:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56f77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56f80b6130 0x7f56f8169a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56f77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56f80b6130 0x7f56f8169a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60440/0 (socket says 172.21.15.89:60440) 2023-10-10T20:35:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56f77fe700 1 -- 172.21.15.89:0/3802024582 learned_addr learned my addr 172.21.15.89:0/3802024582 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56f7fff700 1 --2- 172.21.15.89:0/3802024582 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f56e0003a60 0x7f56f80b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56fcaf8700 1 --2- 172.21.15.89:0/3802024582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56f8169fa0 0x7f56f816a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56f77fe700 1 -- 172.21.15.89:0/3802024582 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f56e0003a60 msgr2=0x7f56f80b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56f77fe700 1 --2- 172.21.15.89:0/3802024582 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f56e0003a60 0x7f56f80b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56f77fe700 1 -- 172.21.15.89:0/3802024582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56f8169fa0 msgr2=0x7f56f816a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56f77fe700 1 --2- 172.21.15.89:0/3802024582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56f8169fa0 0x7f56f816a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56f77fe700 1 -- 172.21.15.89:0/3802024582 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f80ce090 con 0x7f56f80b6130 2023-10-10T20:35:52.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56f7fff700 1 --2- 172.21.15.89:0/3802024582 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f56e0003a60 0x7f56f80b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:52.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56f77fe700 1 --2- 172.21.15.89:0/3802024582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56f80b6130 0x7f56f8169a60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f56ec011d40 tx=0x7f56ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:52.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56dffff700 1 -- 172.21.15.89:0/3802024582 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec00fa60 con 0x7f56f80b6130 2023-10-10T20:35:52.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56dffff700 1 -- 172.21.15.89:0/3802024582 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ec00fbc0 con 0x7f56f80b6130 2023-10-10T20:35:52.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56fe55b700 1 -- 172.21.15.89:0/3802024582 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f80ce370 con 0x7f56f80b6130 2023-10-10T20:35:52.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56dffff700 1 -- 172.21.15.89:0/3802024582 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec014440 con 0x7f56f80b6130 2023-10-10T20:35:52.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.050+0000 7f56fe55b700 1 -- 172.21.15.89:0/3802024582 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f80cda70 con 0x7f56f80b6130 2023-10-10T20:35:52.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.051+0000 7f56fe55b700 1 -- 172.21.15.89:0/3802024582 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f80c8d70 con 0x7f56f80b6130 2023-10-10T20:35:52.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.052+0000 7f56dffff700 1 -- 172.21.15.89:0/3802024582 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f56ec014870 con 0x7f56f80b6130 2023-10-10T20:35:52.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.052+0000 7f56dffff700 1 --2- 172.21.15.89:0/3802024582 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f56e4066f30 0x7f56e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.052+0000 7f56dffff700 1 -- 172.21.15.89:0/3802024582 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f56ec018030 con 0x7f56f80b6130 2023-10-10T20:35:52.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.052+0000 7f56f7fff700 1 --2- 172.21.15.89:0/3802024582 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f56e4066f30 0x7f56e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.052+0000 7f56f7fff700 1 --2- 172.21.15.89:0/3802024582 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f56e4066f30 0x7f56e40693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f56f80c7950 tx=0x7f56e8015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.056+0000 7f56dffff700 1 -- 172.21.15.89:0/3802024582 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f56ec052150 con 0x7f56f80b6130 2023-10-10T20:35:52.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.315+0000 7f56fe55b700 1 -- 172.21.15.89:0/3802024582 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f56f804c3c0 con 0x7f56f80b6130 2023-10-10T20:35:52.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.315+0000 7f56dffff700 1 -- 172.21.15.89:0/3802024582 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f56ec022090 con 0x7f56f80b6130 2023-10-10T20:35:52.318 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:52.318 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:35:52.318 INFO:teuthology.orchestra.run.smithi089.stdout: "auth_dump": [ 2023-10-10T20:35:52.318 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:52.319 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.a", 2023-10-10T20:35:52.319 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlZLSNLhAA8WZTrsht6tAOP+gxaTXKMg==", 2023-10-10T20:35:52.319 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:52.319 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:35:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:35:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:35:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.b", 2023-10-10T20:35:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgyDEORAAMMoSxPFZc0bCIwWy8UsXgA==", 2023-10-10T20:35:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:35:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:35:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:35:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.c", 2023-10-10T20:35:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlMFxSNRAASuQEJFghLIOkTQWa4ckTvg==", 2023-10-10T20:35:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:35:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:35:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:35:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.d", 2023-10-10T20:35:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlPkvrBBAA2VLJmv2t8szTEGtCVwalJw==", 2023-10-10T20:35:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:35:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:35:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:35:52.326 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:52.326 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.326 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:52.326 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:52.326 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.0", 2023-10-10T20:35:52.326 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQArtSVltOcKHRAAk8+zUus04oTVI9KvfhjlXg==", 2023-10-10T20:35:52.327 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:52.327 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:35:52.327 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:35:52.327 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:52.327 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.328 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:52.328 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:52.328 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.1", 2023-10-10T20:35:52.328 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAutSVly8P0GhAAe1bvR63+ptJg84lizMNdQw==", 2023-10-10T20:35:52.328 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:52.328 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:35:52.329 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:35:52.329 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:52.329 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.329 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:52.329 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:52.330 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.2", 2023-10-10T20:35:52.330 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAxtSVl0x9fGRAAEKi7+6EW5mzPbebmz0+sgQ==", 2023-10-10T20:35:52.330 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:52.330 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:35:52.330 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:35:52.331 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:52.331 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.331 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:52.331 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:52.331 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.3", 2023-10-10T20:35:52.331 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA0tSVlfJyaGBAAa+8w54U6v7+npbdufKJruA==", 2023-10-10T20:35:52.332 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:52.332 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:35:52.332 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:35:52.332 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:52.332 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.332 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:52.333 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:52.333 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.4", 2023-10-10T20:35:52.333 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA6tSVlEQ/QFxAA/PRB3c90BuAlF6xekGfX9A==", 2023-10-10T20:35:52.333 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:52.333 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:35:52.334 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:35:52.334 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:52.334 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.334 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:52.334 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:52.334 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.5", 2023-10-10T20:35:52.335 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA9tSVlMOKrFRAAuBnIsEz2VmeSZi9XWzjlzA==", 2023-10-10T20:35:52.335 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:52.335 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:35:52.335 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:35:52.335 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:52.335 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:52.336 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:52.336 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.6", 2023-10-10T20:35:52.336 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBAtSVlH7PQFBAAHr6c7d4TYJEu/KSFEf+EOA==", 2023-10-10T20:35:52.336 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:52.336 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:35:52.337 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:35:52.337 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:52.337 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:52.337 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:52.338 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.7", 2023-10-10T20:35:52.338 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBDtSVlAhC1FBAAuE2BwAA/f7VOkZfVdqAG7A==", 2023-10-10T20:35:52.338 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:52.338 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:35:52.338 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:35:52.339 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:52.339 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.339 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:52.339 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:52.339 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.0", 2023-10-10T20:35:52.340 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlRd3HCBAAb0QP2VfTS7aelno1mFpGAA==", 2023-10-10T20:35:52.340 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:52.340 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:35:52.340 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow r", 2023-10-10T20:35:52.340 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:35:52.341 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.341 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:52.341 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:52.341 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.1", 2023-10-10T20:35:52.341 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlgBouERAAqPdE0KvJU3M53B6Y6AH5GQ==", 2023-10-10T20:35:52.341 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:52.342 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:35:52.342 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:35:52.342 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:35:52.342 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:35:52.342 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.343 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:52.343 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:52.343 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.2", 2023-10-10T20:35:52.343 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlXA4tDhAAoPVYQjECpzD+C67mnsY5IA==", 2023-10-10T20:35:52.343 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:52.344 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:35:52.344 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:35:52.344 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:35:52.344 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:35:52.344 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.344 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:52.345 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:52.345 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.3", 2023-10-10T20:35:52.345 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlZ5KGFhAAuKmZwbOfmmw1nAj5ry6tHw==", 2023-10-10T20:35:52.345 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:52.345 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:35:52.345 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:35:52.346 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:35:52.346 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:35:52.346 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.346 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:52.346 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:52.347 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.admin", 2023-10-10T20:35:52.347 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgpL2EhAAoDOki002rfBrauTgfu481Q==", 2023-10-10T20:35:52.347 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:52.347 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:35:52.347 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:35:52.348 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:35:52.348 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:52.348 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.348 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:52.348 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:52.348 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.x", 2023-10-10T20:35:52.349 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlBhaJKxAArraBxT5xWg515LfpJ7MlbA==", 2023-10-10T20:35:52.349 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:52.349 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:35:52.349 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:35:52.349 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:52.350 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.350 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:35:52.350 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:52.350 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.y", 2023-10-10T20:35:52.350 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlTZIyJxAAWAOsitHwsIxJWwp3b1Z98Q==", 2023-10-10T20:35:52.351 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:35:52.351 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:35:52.351 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:35:52.351 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:35:52.351 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.351 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:52.352 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-10T20:35:52.352 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:35:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.317+0000 7f56fe55b700 1 -- 172.21.15.89:0/3802024582 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f56e4066f30 msgr2=0x7f56e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.317+0000 7f56fe55b700 1 --2- 172.21.15.89:0/3802024582 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f56e4066f30 0x7f56e40693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f56f80c7950 tx=0x7f56e8015000 comp rx=0 tx=0).stop 2023-10-10T20:35:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.317+0000 7f56fe55b700 1 -- 172.21.15.89:0/3802024582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56f80b6130 msgr2=0x7f56f8169a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.317+0000 7f56fe55b700 1 --2- 172.21.15.89:0/3802024582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56f80b6130 0x7f56f8169a60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f56ec011d40 tx=0x7f56ec0091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.317+0000 7f56fe55b700 1 -- 172.21.15.89:0/3802024582 shutdown_connections 2023-10-10T20:35:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.317+0000 7f56fe55b700 1 --2- 172.21.15.89:0/3802024582 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f56e4066f30 0x7f56e40693e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.317+0000 7f56fe55b700 1 --2- 172.21.15.89:0/3802024582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56f8169fa0 0x7f56f816a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.317+0000 7f56fe55b700 1 --2- 172.21.15.89:0/3802024582 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f56e0003a60 0x7f56f80b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.317+0000 7f56fe55b700 1 --2- 172.21.15.89:0/3802024582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56f80b6130 0x7f56f8169a60 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.317+0000 7f56fe55b700 1 -- 172.21.15.89:0/3802024582 >> 172.21.15.89:0/3802024582 conn(0x7f56f8068e20 msgr2=0x7f56f8152c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.317+0000 7f56fe55b700 1 -- 172.21.15.89:0/3802024582 shutdown_connections 2023-10-10T20:35:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.317+0000 7f56fe55b700 1 -- 172.21.15.89:0/3802024582 wait complete. 2023-10-10T20:35:52.356 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:35:52.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.468+0000 7efd070af700 1 Processor -- start 2023-10-10T20:35:52.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.468+0000 7efd070af700 1 -- start start 2023-10-10T20:35:52.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.469+0000 7efd070af700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd0015d230 0x7efd0015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.469+0000 7efd070af700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd000b2980 con 0x7efd001597f0 2023-10-10T20:35:52.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.469+0000 7efd070af700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0015dbb0 con 0x7efd00159420 2023-10-10T20:35:52.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.469+0000 7efd070af700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd000b11d0 con 0x7efd0015d230 2023-10-10T20:35:52.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.469+0000 7efd058ac700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7efd001597f0 0x7efd000b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52704/0 (socket says 172.21.15.89:52704) 2023-10-10T20:35:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.469+0000 7efd058ac700 1 -- 172.21.15.89:0/1462121537 learned_addr learned my addr 172.21.15.89:0/1462121537 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.469+0000 7efd068ae700 1 --2- 172.21.15.89:0/1462121537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd0015d230 0x7efd0015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.469+0000 7efd050ab700 1 -- 172.21.15.89:0/1462121537 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964308467 0 0) 0x7efd000b2980 con 0x7efd001597f0 2023-10-10T20:35:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.469+0000 7efd050ab700 1 -- 172.21.15.89:0/1462121537 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efce8003570 con 0x7efd001597f0 2023-10-10T20:35:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd050ab700 1 -- 172.21.15.89:0/1462121537 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1876750474 0 0) 0x7efce8003570 con 0x7efd001597f0 2023-10-10T20:35:52.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd050ab700 1 -- 172.21.15.89:0/1462121537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd0015d230 msgr2=0x7efd0015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:52.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd050ab700 1 --2- 172.21.15.89:0/1462121537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd0015d230 0x7efd0015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd050ab700 1 -- 172.21.15.89:0/1462121537 >> v1:172.21.15.111:6789/0 conn(0x7efd00159420 legacy=0x7efd000b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd050ab700 1 -- 172.21.15.89:0/1462121537 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd000b14b0 con 0x7efd001597f0 2023-10-10T20:35:52.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd050ab700 1 -- 172.21.15.89:0/1462121537 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7efcf4001470 con 0x7efd001597f0 2023-10-10T20:35:52.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd050ab700 1 -- 172.21.15.89:0/1462121537 >> v1:172.21.15.89:6789/0 conn(0x7efd001597f0 legacy=0x7efd000b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd050ab700 1 --2- 172.21.15.89:0/1462121537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efce8003a90 0x7efce8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd050ab700 1 --2- 172.21.15.89:0/1462121537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd0015d230 0x7efce8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd050ab700 1 --2- 172.21.15.89:0/1462121537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce8004d50 0x7efce80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd050ab700 1 -- 172.21.15.89:0/1462121537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd000b11d0 con 0x7efce8003a90 2023-10-10T20:35:52.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd050ab700 1 -- 172.21.15.89:0/1462121537 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce800bb10 con 0x7efce8004d50 2023-10-10T20:35:52.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd050ab700 1 -- 172.21.15.89:0/1462121537 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce800bcd0 con 0x7efd0015d230 2023-10-10T20:35:52.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd060ad700 1 --2- 172.21.15.89:0/1462121537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efce8003a90 0x7efce8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd058ac700 1 --2- 172.21.15.89:0/1462121537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd0015d230 0x7efce8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd068ae700 1 --2- 172.21.15.89:0/1462121537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce8004d50 0x7efce80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd060ad700 1 -- 172.21.15.89:0/1462121537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd0015d230 msgr2=0x7efce8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:52.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd060ad700 1 --2- 172.21.15.89:0/1462121537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd0015d230 0x7efce8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd060ad700 1 -- 172.21.15.89:0/1462121537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce8004d50 msgr2=0x7efce80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:52.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd060ad700 1 --2- 172.21.15.89:0/1462121537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce8004d50 0x7efce80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd060ad700 1 -- 172.21.15.89:0/1462121537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce8003570 con 0x7efce8003a90 2023-10-10T20:35:52.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd060ad700 1 --2- 172.21.15.89:0/1462121537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efce8003a90 0x7efce8003e60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efcf0002060 tx=0x7efcf00016e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7add9107e544ca96 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:52.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd050ab700 1 -- 172.21.15.89:0/1462121537 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf00019f0 con 0x7efce8003a90 2023-10-10T20:35:52.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd050ab700 1 -- 172.21.15.89:0/1462121537 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf000c8b0 con 0x7efce8003a90 2023-10-10T20:35:52.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.470+0000 7efd050ab700 1 -- 172.21.15.89:0/1462121537 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf000cc30 con 0x7efce8003a90 2023-10-10T20:35:52.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.471+0000 7efd070af700 1 -- 172.21.15.89:0/1462121537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efce8003a90 msgr2=0x7efce8003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.471+0000 7efd070af700 1 --2- 172.21.15.89:0/1462121537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efce8003a90 0x7efce8003e60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efcf0002060 tx=0x7efcf00016e0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.471+0000 7efd068ae700 1 -- 172.21.15.89:0/1462121537 reap_dead start 2023-10-10T20:35:52.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.471+0000 7efd070af700 1 -- 172.21.15.89:0/1462121537 shutdown_connections 2023-10-10T20:35:52.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.471+0000 7efd070af700 1 -- 172.21.15.89:0/1462121537 >> 172.21.15.89:0/1462121537 conn(0x7efd000d0660 msgr2=0x7efd000d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:52.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.471+0000 7efd070af700 1 -- 172.21.15.89:0/1462121537 shutdown_connections 2023-10-10T20:35:52.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.471+0000 7efd070af700 1 -- 172.21.15.89:0/1462121537 wait complete. 2023-10-10T20:35:52.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.472+0000 7efd070af700 1 Processor -- start 2023-10-10T20:35:52.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.472+0000 7efd070af700 1 -- start start 2023-10-10T20:35:52.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.473+0000 7efd070af700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce8003a90 0x7efd0014e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.473+0000 7efd070af700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd00152150 0x7efd000d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.473+0000 7efd070af700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd001525c0 0x7efd000d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.473+0000 7efd070af700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0014f790 con 0x7efd001525c0 2023-10-10T20:35:52.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.473+0000 7efd070af700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0014f460 con 0x7efce8003a90 2023-10-10T20:35:52.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.473+0000 7efd070af700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd00152b10 con 0x7efd00152150 2023-10-10T20:35:52.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.473+0000 7efd068ae700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd001525c0 0x7efd000d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.473+0000 7efd060ad700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce8003a90 0x7efd0014e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.473+0000 7efd068ae700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd001525c0 0x7efd000d0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60444/0 (socket says 172.21.15.89:60444) 2023-10-10T20:35:52.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.473+0000 7efd058ac700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd00152150 0x7efd000d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.473+0000 7efd068ae700 1 -- 172.21.15.89:0/3166910653 learned_addr learned my addr 172.21.15.89:0/3166910653 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:52.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.473+0000 7efd068ae700 1 -- 172.21.15.89:0/3166910653 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd00152150 msgr2=0x7efd000d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.473+0000 7efd068ae700 1 --2- 172.21.15.89:0/3166910653 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd00152150 0x7efd000d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.473+0000 7efd068ae700 1 -- 172.21.15.89:0/3166910653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce8003a90 msgr2=0x7efd0014e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.473+0000 7efd068ae700 1 --2- 172.21.15.89:0/3166910653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce8003a90 0x7efd0014e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.473+0000 7efd068ae700 1 -- 172.21.15.89:0/3166910653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd00152df0 con 0x7efd001525c0 2023-10-10T20:35:52.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.473+0000 7efd068ae700 1 --2- 172.21.15.89:0/3166910653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd001525c0 0x7efd000d0fa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efd000af930 tx=0x7efcf8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:52.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.473+0000 7efcfe7fc700 1 -- 172.21.15.89:0/3166910653 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf8011ed0 con 0x7efd001525c0 2023-10-10T20:35:52.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.474+0000 7efcfe7fc700 1 -- 172.21.15.89:0/3166910653 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf8015670 con 0x7efd001525c0 2023-10-10T20:35:52.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.474+0000 7efcfe7fc700 1 -- 172.21.15.89:0/3166910653 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf8015a70 con 0x7efd001525c0 2023-10-10T20:35:52.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.474+0000 7efd070af700 1 -- 172.21.15.89:0/3166910653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd001530d0 con 0x7efd001525c0 2023-10-10T20:35:52.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.474+0000 7efd070af700 1 -- 172.21.15.89:0/3166910653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd0015c760 con 0x7efd001525c0 2023-10-10T20:35:52.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.474+0000 7efd070af700 1 -- 172.21.15.89:0/3166910653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd0004c3c0 con 0x7efd001525c0 2023-10-10T20:35:52.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.475+0000 7efcfe7fc700 1 -- 172.21.15.89:0/3166910653 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efcf801e400 con 0x7efd001525c0 2023-10-10T20:35:52.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.475+0000 7efcfe7fc700 1 --2- 172.21.15.89:0/3166910653 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efcec066f30 0x7efcec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.475+0000 7efcfe7fc700 1 -- 172.21.15.89:0/3166910653 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4853+0+0 (secure 0 0 0) 0x7efcf8084480 con 0x7efd001525c0 2023-10-10T20:35:52.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.475+0000 7efd060ad700 1 --2- 172.21.15.89:0/3166910653 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efcec066f30 0x7efcec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.476+0000 7efd060ad700 1 --2- 172.21.15.89:0/3166910653 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efcec066f30 0x7efcec0693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efcf0001370 tx=0x7efcf0015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:52.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.479+0000 7efcfe7fc700 1 -- 172.21.15.89:0/3166910653 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efcf8051a40 con 0x7efd001525c0 2023-10-10T20:35:52.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.665+0000 7efd070af700 1 -- 172.21.15.89:0/3166910653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd000b1790 con 0x7efd001525c0 2023-10-10T20:35:52.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.666+0000 7efcfe7fc700 1 -- 172.21.15.89:0/3166910653 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+12134 (secure 0 0 0) 0x7efcf8055060 con 0x7efd001525c0 2023-10-10T20:35:52.668 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:52.668 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":46,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:35:51.907878+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:35:52.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.668+0000 7efd070af700 1 -- 172.21.15.89:0/3166910653 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efcec066f30 msgr2=0x7efcec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.668+0000 7efd070af700 1 --2- 172.21.15.89:0/3166910653 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efcec066f30 0x7efcec0693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efcf0001370 tx=0x7efcf0015000 comp rx=0 tx=0).stop 2023-10-10T20:35:52.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.668+0000 7efd070af700 1 -- 172.21.15.89:0/3166910653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd001525c0 msgr2=0x7efd000d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.668+0000 7efd070af700 1 --2- 172.21.15.89:0/3166910653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd001525c0 0x7efd000d0fa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efd000af930 tx=0x7efcf8008040 comp rx=0 tx=0).stop 2023-10-10T20:35:52.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.668+0000 7efd070af700 1 -- 172.21.15.89:0/3166910653 shutdown_connections 2023-10-10T20:35:52.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.668+0000 7efd070af700 1 --2- 172.21.15.89:0/3166910653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd001525c0 0x7efd000d0fa0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.668+0000 7efd070af700 1 --2- 172.21.15.89:0/3166910653 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efcec066f30 0x7efcec0693e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.668+0000 7efd070af700 1 --2- 172.21.15.89:0/3166910653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce8003a90 0x7efd0014e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.668+0000 7efd070af700 1 --2- 172.21.15.89:0/3166910653 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd00152150 0x7efd000d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.668+0000 7efd070af700 1 -- 172.21.15.89:0/3166910653 >> 172.21.15.89:0/3166910653 conn(0x7efd000d0660 msgr2=0x7efd0015d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:52.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.668+0000 7efd070af700 1 -- 172.21.15.89:0/3166910653 shutdown_connections 2023-10-10T20:35:52.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.668+0000 7efd070af700 1 -- 172.21.15.89:0/3166910653 wait complete. 2023-10-10T20:35:52.682 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:35:52.682 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:35:52.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.797+0000 7f8c34b62700 1 Processor -- start 2023-10-10T20:35:52.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.798+0000 7f8c34b62700 1 -- start start 2023-10-10T20:35:52.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.798+0000 7f8c34b62700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8c30159c60 0x7f8c3015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.798+0000 7f8c34b62700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c3015a160 con 0x7f8c3015db10 2023-10-10T20:35:52.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.798+0000 7f8c34b62700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c300d13a0 con 0x7f8c30159890 2023-10-10T20:35:52.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.798+0000 7f8c34b62700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c3015df50 con 0x7f8c30159c60 2023-10-10T20:35:52.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.798+0000 7f8c25d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8c30159c60 0x7f8c3015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.799+0000 7f8c25d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8c30159c60 0x7f8c3015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60450/0 (socket says 172.21.15.89:60450) 2023-10-10T20:35:52.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.798+0000 7f8c2ed9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8c3015db10 0x7f8c30069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52716/0 (socket says 172.21.15.89:52716) 2023-10-10T20:35:52.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.799+0000 7f8c25d9b700 1 -- 172.21.15.89:0/2750101602 learned_addr learned my addr 172.21.15.89:0/2750101602 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:52.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.799+0000 7f8c25d9b700 1 -- 172.21.15.89:0/2750101602 >> v1:172.21.15.111:6789/0 conn(0x7f8c30159890 legacy=0x7f8c3015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.799+0000 7f8c25d9b700 1 -- 172.21.15.89:0/2750101602 >> v1:172.21.15.89:6789/0 conn(0x7f8c3015db10 legacy=0x7f8c30069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.799+0000 7f8c25d9b700 1 -- 172.21.15.89:0/2750101602 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3015e140 con 0x7f8c30159c60 2023-10-10T20:35:52.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.799+0000 7f8c25d9b700 1 --2- 172.21.15.89:0/2750101602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c30159c60 0x7f8c3015d5d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8c180099b0 tx=0x7f8c18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870f96da84b5c0be server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:52.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.799+0000 7f8c2dd9b700 1 -- 172.21.15.89:0/2750101602 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1800b760 con 0x7f8c30159c60 2023-10-10T20:35:52.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.799+0000 7f8c2dd9b700 1 -- 172.21.15.89:0/2750101602 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1800b8c0 con 0x7f8c30159c60 2023-10-10T20:35:52.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.799+0000 7f8c2dd9b700 1 -- 172.21.15.89:0/2750101602 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1800bc60 con 0x7f8c30159c60 2023-10-10T20:35:52.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.800+0000 7f8c34b62700 1 -- 172.21.15.89:0/2750101602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c30159c60 msgr2=0x7f8c3015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.800+0000 7f8c34b62700 1 --2- 172.21.15.89:0/2750101602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c30159c60 0x7f8c3015d5d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8c180099b0 tx=0x7f8c18009250 comp rx=0 tx=0).stop 2023-10-10T20:35:52.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.800+0000 7f8c34b62700 1 -- 172.21.15.89:0/2750101602 shutdown_connections 2023-10-10T20:35:52.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.800+0000 7f8c34b62700 1 --2- 172.21.15.89:0/2750101602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c30159c60 0x7f8c3015d5d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.800+0000 7f8c34b62700 1 -- 172.21.15.89:0/2750101602 >> 172.21.15.89:0/2750101602 conn(0x7f8c30068e20 msgr2=0x7f8c300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:52.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.800+0000 7f8c34b62700 1 -- 172.21.15.89:0/2750101602 shutdown_connections 2023-10-10T20:35:52.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.800+0000 7f8c34b62700 1 -- 172.21.15.89:0/2750101602 wait complete. 2023-10-10T20:35:52.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.801+0000 7f8c34b62700 1 Processor -- start 2023-10-10T20:35:52.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.801+0000 7f8c34b62700 1 -- start start 2023-10-10T20:35:52.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c34b62700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c30159890 0x7f8c300ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c34b62700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c30159c60 0x7f8c30152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c34b62700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3015db10 0x7f8c300c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c34b62700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c30152d40 con 0x7f8c3015db10 2023-10-10T20:35:52.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c34b62700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c300b5660 con 0x7f8c30159c60 2023-10-10T20:35:52.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c34b62700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c300b64a0 con 0x7f8c30159890 2023-10-10T20:35:52.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c2ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3015db10 0x7f8c300c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c2ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3015db10 0x7f8c300c6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60462/0 (socket says 172.21.15.89:60462) 2023-10-10T20:35:52.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c2ed9d700 1 -- 172.21.15.89:0/1533748293 learned_addr learned my addr 172.21.15.89:0/1533748293 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:52.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c2e59c700 1 --2- 172.21.15.89:0/1533748293 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c30159890 0x7f8c300ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c25d9b700 1 --2- 172.21.15.89:0/1533748293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c30159c60 0x7f8c30152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c2ed9d700 1 -- 172.21.15.89:0/1533748293 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c30159890 msgr2=0x7f8c300ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c2ed9d700 1 --2- 172.21.15.89:0/1533748293 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c30159890 0x7f8c300ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c2ed9d700 1 -- 172.21.15.89:0/1533748293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c30159c60 msgr2=0x7f8c30152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c2ed9d700 1 --2- 172.21.15.89:0/1533748293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c30159c60 0x7f8c30152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c2ed9d700 1 -- 172.21.15.89:0/1533748293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c180094f0 con 0x7f8c3015db10 2023-10-10T20:35:52.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c2ed9d700 1 --2- 172.21.15.89:0/1533748293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3015db10 0x7f8c300c6560 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800cdb0 tx=0x7f8c28001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:52.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c26ffd700 1 -- 172.21.15.89:0/1533748293 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28005df0 con 0x7f8c3015db10 2023-10-10T20:35:52.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c26ffd700 1 -- 172.21.15.89:0/1533748293 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2800a870 con 0x7f8c3015db10 2023-10-10T20:35:52.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c26ffd700 1 -- 172.21.15.89:0/1533748293 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2800ac10 con 0x7f8c3015db10 2023-10-10T20:35:52.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c34b62700 1 -- 172.21.15.89:0/1533748293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c300b5890 con 0x7f8c3015db10 2023-10-10T20:35:52.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.802+0000 7f8c34b62700 1 -- 172.21.15.89:0/1533748293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c300b5e50 con 0x7f8c3015db10 2023-10-10T20:35:52.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.803+0000 7f8c34b62700 1 -- 172.21.15.89:0/1533748293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c300cd100 con 0x7f8c3015db10 2023-10-10T20:35:52.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.804+0000 7f8c26ffd700 1 -- 172.21.15.89:0/1533748293 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c280070f0 con 0x7f8c3015db10 2023-10-10T20:35:52.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.804+0000 7f8c26ffd700 1 --2- 172.21.15.89:0/1533748293 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c10066f30 0x7f8c100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:52.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.804+0000 7f8c2e59c700 1 --2- 172.21.15.89:0/1533748293 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c10066f30 0x7f8c100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:52.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.804+0000 7f8c26ffd700 1 -- 172.21.15.89:0/1533748293 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8c280842a0 con 0x7f8c3015db10 2023-10-10T20:35:52.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.805+0000 7f8c2e59c700 1 --2- 172.21.15.89:0/1533748293 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c10066f30 0x7f8c100693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8c20003a80 tx=0x7f8c20007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:52.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.808+0000 7f8c26ffd700 1 -- 172.21.15.89:0/1533748293 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c300cd100 con 0x7f8c3015db10 2023-10-10T20:35:52.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.991+0000 7f8c34b62700 1 -- 172.21.15.89:0/1533748293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c300cd100 con 0x7f8c3015db10 2023-10-10T20:35:52.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.992+0000 7f8c26ffd700 1 -- 172.21.15.89:0/1533748293 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c300cd100 con 0x7f8c3015db10 2023-10-10T20:35:52.995 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:35:52.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.994+0000 7f8c34b62700 1 -- 172.21.15.89:0/1533748293 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c10066f30 msgr2=0x7f8c100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.994+0000 7f8c34b62700 1 --2- 172.21.15.89:0/1533748293 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c10066f30 0x7f8c100693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8c20003a80 tx=0x7f8c20007ec0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.994+0000 7f8c34b62700 1 -- 172.21.15.89:0/1533748293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3015db10 msgr2=0x7f8c300c6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:52.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.994+0000 7f8c34b62700 1 --2- 172.21.15.89:0/1533748293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3015db10 0x7f8c300c6560 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800cdb0 tx=0x7f8c28001b40 comp rx=0 tx=0).stop 2023-10-10T20:35:52.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.994+0000 7f8c34b62700 1 -- 172.21.15.89:0/1533748293 shutdown_connections 2023-10-10T20:35:52.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.994+0000 7f8c34b62700 1 --2- 172.21.15.89:0/1533748293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3015db10 0x7f8c300c6560 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.994+0000 7f8c34b62700 1 --2- 172.21.15.89:0/1533748293 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c30159890 0x7f8c300ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.994+0000 7f8c34b62700 1 --2- 172.21.15.89:0/1533748293 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c10066f30 0x7f8c100693e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.994+0000 7f8c34b62700 1 --2- 172.21.15.89:0/1533748293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c30159c60 0x7f8c30152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:52.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.994+0000 7f8c34b62700 1 -- 172.21.15.89:0/1533748293 >> 172.21.15.89:0/1533748293 conn(0x7f8c30068e20 msgr2=0x7f8c30155210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:52.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.994+0000 7f8c34b62700 1 -- 172.21.15.89:0/1533748293 shutdown_connections 2023-10-10T20:35:52.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:52.994+0000 7f8c34b62700 1 -- 172.21.15.89:0/1533748293 wait complete. 2023-10-10T20:35:53.008 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:35:53.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.120+0000 7f5058c67700 1 Processor -- start 2023-10-10T20:35:53.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.120+0000 7f5058c67700 1 -- start start 2023-10-10T20:35:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.121+0000 7f5058c67700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5054159420 0x7f50540b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.121+0000 7f5058c67700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50540b2980 con 0x7f505415d230 2023-10-10T20:35:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.121+0000 7f5058c67700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50540b11d0 con 0x7f50541597f0 2023-10-10T20:35:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.121+0000 7f5058c67700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50540b14b0 con 0x7f5054159420 2023-10-10T20:35:53.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.121+0000 7f5053fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f505415d230 0x7f505415d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52726/0 (socket says 172.21.15.89:52726) 2023-10-10T20:35:53.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.121+0000 7f5053fff700 1 -- 172.21.15.89:0/2794928539 learned_addr learned my addr 172.21.15.89:0/2794928539 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:53.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.121+0000 7f50537fe700 1 --2- 172.21.15.89:0/2794928539 >> v2:172.21.15.111:3300/0 conn(0x7f5054159420 0x7f50540b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:53.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.121+0000 7f50527fc700 1 -- 172.21.15.89:0/2794928539 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1178118151 0 0) 0x7f50540b2980 con 0x7f505415d230 2023-10-10T20:35:53.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.121+0000 7f50527fc700 1 -- 172.21.15.89:0/2794928539 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f503c003570 con 0x7f505415d230 2023-10-10T20:35:53.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.122+0000 7f50537fe700 1 -- 172.21.15.89:0/2794928539 >> v1:172.21.15.111:6789/0 conn(0x7f50541597f0 legacy=0x7f50540b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:53.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.122+0000 7f50537fe700 1 -- 172.21.15.89:0/2794928539 >> v1:172.21.15.89:6789/0 conn(0x7f505415d230 legacy=0x7f505415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:53.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.122+0000 7f50537fe700 1 -- 172.21.15.89:0/2794928539 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50540b1790 con 0x7f5054159420 2023-10-10T20:35:53.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.122+0000 7f50537fe700 1 --2- 172.21.15.89:0/2794928539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5054159420 0x7f50540b5f40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5044009e60 tx=0x7f5044009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf337b619d07a1a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:53.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.122+0000 7f50527fc700 1 -- 172.21.15.89:0/2794928539 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504400e070 con 0x7f5054159420 2023-10-10T20:35:53.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.122+0000 7f50527fc700 1 -- 172.21.15.89:0/2794928539 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504400bb60 con 0x7f5054159420 2023-10-10T20:35:53.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.122+0000 7f50527fc700 1 -- 172.21.15.89:0/2794928539 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044012430 con 0x7f5054159420 2023-10-10T20:35:53.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.122+0000 7f5058c67700 1 -- 172.21.15.89:0/2794928539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5054159420 msgr2=0x7f50540b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:53.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.122+0000 7f5058c67700 1 --2- 172.21.15.89:0/2794928539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5054159420 0x7f50540b5f40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5044009e60 tx=0x7f5044009780 comp rx=0 tx=0).stop 2023-10-10T20:35:53.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.122+0000 7f5058c67700 1 -- 172.21.15.89:0/2794928539 shutdown_connections 2023-10-10T20:35:53.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.122+0000 7f5058c67700 1 --2- 172.21.15.89:0/2794928539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5054159420 0x7f50540b5f40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:53.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.122+0000 7f5058c67700 1 -- 172.21.15.89:0/2794928539 >> 172.21.15.89:0/2794928539 conn(0x7f50540d0660 msgr2=0x7f50540d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:53.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.122+0000 7f5058c67700 1 -- 172.21.15.89:0/2794928539 shutdown_connections 2023-10-10T20:35:53.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.123+0000 7f5058c67700 1 -- 172.21.15.89:0/2794928539 wait complete. 2023-10-10T20:35:53.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.124+0000 7f5058c67700 1 Processor -- start 2023-10-10T20:35:53.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.124+0000 7f5058c67700 1 -- start start 2023-10-10T20:35:53.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.124+0000 7f5058c67700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5054159420 0x7f50540c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:53.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.124+0000 7f5058c67700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50541597f0 0x7f50540b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:53.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.124+0000 7f5058c67700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505415d230 0x7f505414a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:53.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.124+0000 7f5058c67700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50540c8840 con 0x7f505415d230 2023-10-10T20:35:53.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.124+0000 7f5058c67700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50540b5a70 con 0x7f50541597f0 2023-10-10T20:35:53.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.124+0000 7f5058c67700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50540c5d90 con 0x7f5054159420 2023-10-10T20:35:53.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.125+0000 7f5053fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505415d230 0x7f505414a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:53.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.125+0000 7f5053fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505415d230 0x7f505414a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60474/0 (socket says 172.21.15.89:60474) 2023-10-10T20:35:53.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.125+0000 7f5053fff700 1 -- 172.21.15.89:0/343141878 learned_addr learned my addr 172.21.15.89:0/343141878 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:53.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.125+0000 7f5052ffd700 1 --2- 172.21.15.89:0/343141878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50541597f0 0x7f50540b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:53.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.125+0000 7f50537fe700 1 --2- 172.21.15.89:0/343141878 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5054159420 0x7f50540c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:53.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.125+0000 7f5053fff700 1 -- 172.21.15.89:0/343141878 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5054159420 msgr2=0x7f50540c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:53.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.125+0000 7f5053fff700 1 --2- 172.21.15.89:0/343141878 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5054159420 0x7f50540c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:53.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.125+0000 7f5053fff700 1 -- 172.21.15.89:0/343141878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50541597f0 msgr2=0x7f50540b52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:53.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.125+0000 7f5053fff700 1 --2- 172.21.15.89:0/343141878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50541597f0 0x7f50540b52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:53.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.125+0000 7f5053fff700 1 -- 172.21.15.89:0/343141878 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5044009a20 con 0x7f505415d230 2023-10-10T20:35:53.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.125+0000 7f5053fff700 1 --2- 172.21.15.89:0/343141878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505415d230 0x7f505414a190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f504c00c560 tx=0x7f504c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:53.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.125+0000 7f503b7fe700 1 -- 172.21.15.89:0/343141878 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c005b30 con 0x7f505415d230 2023-10-10T20:35:53.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.125+0000 7f503b7fe700 1 -- 172.21.15.89:0/343141878 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504c005c90 con 0x7f505415d230 2023-10-10T20:35:53.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.125+0000 7f503b7fe700 1 -- 172.21.15.89:0/343141878 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c007300 con 0x7f505415d230 2023-10-10T20:35:53.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.125+0000 7f5058c67700 1 -- 172.21.15.89:0/343141878 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50540b5810 con 0x7f505415d230 2023-10-10T20:35:53.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.125+0000 7f5058c67700 1 -- 172.21.15.89:0/343141878 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50540c54f0 con 0x7f505415d230 2023-10-10T20:35:53.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.126+0000 7f5058c67700 1 -- 172.21.15.89:0/343141878 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505404c3c0 con 0x7f505415d230 2023-10-10T20:35:53.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.127+0000 7f503b7fe700 1 -- 172.21.15.89:0/343141878 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f504c01b3b0 con 0x7f505415d230 2023-10-10T20:35:53.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.127+0000 7f503b7fe700 1 --2- 172.21.15.89:0/343141878 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5040066f30 0x7f50400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:53.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.127+0000 7f503b7fe700 1 -- 172.21.15.89:0/343141878 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f504c083520 con 0x7f505415d230 2023-10-10T20:35:53.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.127+0000 7f50537fe700 1 --2- 172.21.15.89:0/343141878 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5040066f30 0x7f50400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:53.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.127+0000 7f50537fe700 1 --2- 172.21.15.89:0/343141878 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5040066f30 0x7f50400693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f50440097c0 tx=0x7f5044019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:53.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.131+0000 7f503b7fe700 1 -- 172.21.15.89:0/343141878 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f504c050ae0 con 0x7f505415d230 2023-10-10T20:35:53.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.316+0000 7f5058c67700 1 -- 172.21.15.89:0/343141878 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f50540b1aa0 con 0x7f505415d230 2023-10-10T20:35:53.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.316+0000 7f503b7fe700 1 -- 172.21.15.89:0/343141878 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v46) v1 ==== 84+0+65 (secure 0 0 0) 0x7f504c017090 con 0x7f505415d230 2023-10-10T20:35:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:53.319 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:35:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:35:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2023-10-10T20:35:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2023-10-10T20:35:53.320 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:35:53.320 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:35:53.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.318+0000 7f5058c67700 1 -- 172.21.15.89:0/343141878 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5040066f30 msgr2=0x7f50400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:53.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.318+0000 7f5058c67700 1 --2- 172.21.15.89:0/343141878 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5040066f30 0x7f50400693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f50440097c0 tx=0x7f5044019000 comp rx=0 tx=0).stop 2023-10-10T20:35:53.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.318+0000 7f5058c67700 1 -- 172.21.15.89:0/343141878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505415d230 msgr2=0x7f505414a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:53.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.318+0000 7f5058c67700 1 --2- 172.21.15.89:0/343141878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505415d230 0x7f505414a190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f504c00c560 tx=0x7f504c007d90 comp rx=0 tx=0).stop 2023-10-10T20:35:53.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.318+0000 7f5058c67700 1 -- 172.21.15.89:0/343141878 shutdown_connections 2023-10-10T20:35:53.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.318+0000 7f5058c67700 1 --2- 172.21.15.89:0/343141878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505415d230 0x7f505414a190 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:53.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.318+0000 7f5058c67700 1 --2- 172.21.15.89:0/343141878 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5054159420 0x7f50540c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:53.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.318+0000 7f5058c67700 1 --2- 172.21.15.89:0/343141878 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5040066f30 0x7f50400693e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:53.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.318+0000 7f5058c67700 1 --2- 172.21.15.89:0/343141878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50541597f0 0x7f50540b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:53.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.318+0000 7f5058c67700 1 -- 172.21.15.89:0/343141878 >> 172.21.15.89:0/343141878 conn(0x7f50540d0660 msgr2=0x7f50541586c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:53.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.318+0000 7f5058c67700 1 -- 172.21.15.89:0/343141878 shutdown_connections 2023-10-10T20:35:53.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.318+0000 7f5058c67700 1 -- 172.21.15.89:0/343141878 wait complete. 2023-10-10T20:35:53.332 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:35:53.332 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:35:53.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.448+0000 7f8d4a3a1700 1 Processor -- start 2023-10-10T20:35:53.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.448+0000 7f8d4a3a1700 1 -- start start 2023-10-10T20:35:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.449+0000 7f8d4a3a1700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8d44159ec0 0x7f8d440b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.449+0000 7f8d4a3a1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8d440b53d0 0x7f8d440cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.449+0000 7f8d4a3a1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d440cce40 0x7f8d440cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.449+0000 7f8d4a3a1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d440b5b00 con 0x7f8d440b53d0 2023-10-10T20:35:53.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.449+0000 7f8d4a3a1700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d440b5850 con 0x7f8d44159ec0 2023-10-10T20:35:53.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.449+0000 7f8d4a3a1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d440b5da0 con 0x7f8d440cce40 2023-10-10T20:35:53.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.449+0000 7f8d437fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8d440b53d0 0x7f8d440cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:53.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.449+0000 7f8d437fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8d440b53d0 0x7f8d440cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60490/0 (socket says 172.21.15.89:60490) 2023-10-10T20:35:53.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.449+0000 7f8d437fe700 1 -- 172.21.15.89:0/1218485599 learned_addr learned my addr 172.21.15.89:0/1218485599 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:53.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.449+0000 7f8d43fff700 1 --2- 172.21.15.89:0/1218485599 >> v2:172.21.15.111:3300/0 conn(0x7f8d44159ec0 0x7f8d440b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:53.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.449+0000 7f8d4893e700 1 --2- 172.21.15.89:0/1218485599 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d440cce40 0x7f8d440cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:53.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.450+0000 7f8d437fe700 1 -- 172.21.15.89:0/1218485599 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d440cce40 msgr2=0x7f8d440cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:53.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.450+0000 7f8d437fe700 1 --2- 172.21.15.89:0/1218485599 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d440cce40 0x7f8d440cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:53.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.450+0000 7f8d437fe700 1 -- 172.21.15.89:0/1218485599 >> v2:172.21.15.111:3300/0 conn(0x7f8d44159ec0 msgr2=0x7f8d440b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.450+0000 7f8d437fe700 1 --2- 172.21.15.89:0/1218485599 >> v2:172.21.15.111:3300/0 conn(0x7f8d44159ec0 0x7f8d440b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.450+0000 7f8d437fe700 1 -- 172.21.15.89:0/1218485599 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d440b6080 con 0x7f8d440b53d0 2023-10-10T20:35:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.450+0000 7f8d43fff700 1 --2- 172.21.15.89:0/1218485599 >> v2:172.21.15.111:3300/0 conn(0x7f8d44159ec0 0x7f8d440b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.450+0000 7f8d437fe700 1 --2- 172.21.15.89:0/1218485599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d440b53d0 0x7f8d440cc900 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8d44089120 tx=0x7f8d38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7917e68a3be4d1f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.450+0000 7f8d42ffd700 1 -- 172.21.15.89:0/1218485599 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3800b860 con 0x7f8d440b53d0 2023-10-10T20:35:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.450+0000 7f8d42ffd700 1 -- 172.21.15.89:0/1218485599 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3800b9c0 con 0x7f8d440b53d0 2023-10-10T20:35:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.450+0000 7f8d42ffd700 1 -- 172.21.15.89:0/1218485599 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3800bd60 con 0x7f8d440b53d0 2023-10-10T20:35:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.450+0000 7f8d4a3a1700 1 -- 172.21.15.89:0/1218485599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d440b53d0 msgr2=0x7f8d440cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.450+0000 7f8d4a3a1700 1 --2- 172.21.15.89:0/1218485599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d440b53d0 0x7f8d440cc900 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8d44089120 tx=0x7f8d38009250 comp rx=0 tx=0).stop 2023-10-10T20:35:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.450+0000 7f8d4a3a1700 1 -- 172.21.15.89:0/1218485599 shutdown_connections 2023-10-10T20:35:53.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.450+0000 7f8d4a3a1700 1 --2- 172.21.15.89:0/1218485599 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d440cce40 0x7f8d440cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:53.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.450+0000 7f8d4a3a1700 1 --2- 172.21.15.89:0/1218485599 >> v2:172.21.15.111:3300/0 conn(0x7f8d44159ec0 0x7f8d440b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:53.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.450+0000 7f8d4a3a1700 1 --2- 172.21.15.89:0/1218485599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d440b53d0 0x7f8d440cc900 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:53.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.451+0000 7f8d4a3a1700 1 -- 172.21.15.89:0/1218485599 >> 172.21.15.89:0/1218485599 conn(0x7f8d440999f0 msgr2=0x7f8d440b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:53.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.451+0000 7f8d4a3a1700 1 -- 172.21.15.89:0/1218485599 shutdown_connections 2023-10-10T20:35:53.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.451+0000 7f8d4a3a1700 1 -- 172.21.15.89:0/1218485599 wait complete. 2023-10-10T20:35:53.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.452+0000 7f8d4a3a1700 1 Processor -- start 2023-10-10T20:35:53.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.452+0000 7f8d4a3a1700 1 -- start start 2023-10-10T20:35:53.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.452+0000 7f8d4a3a1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d440b53d0 0x7f8d44068a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:53.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.452+0000 7f8d4a3a1700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d440cce40 0x7f8d4416e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:53.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.452+0000 7f8d4a3a1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d44159ec0 0x7f8d44157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:53.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.452+0000 7f8d4a3a1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4416ef40 con 0x7f8d440b53d0 2023-10-10T20:35:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.452+0000 7f8d4a3a1700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4416ece0 con 0x7f8d440cce40 2023-10-10T20:35:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.452+0000 7f8d4a3a1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d44069200 con 0x7f8d44159ec0 2023-10-10T20:35:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.453+0000 7f8d43fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d440b53d0 0x7f8d44068a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.453+0000 7f8d43fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d440b53d0 0x7f8d44068a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60502/0 (socket says 172.21.15.89:60502) 2023-10-10T20:35:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.453+0000 7f8d43fff700 1 -- 172.21.15.89:0/142485194 learned_addr learned my addr 172.21.15.89:0/142485194 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:53.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.453+0000 7f8d437fe700 1 --2- 172.21.15.89:0/142485194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d440cce40 0x7f8d4416e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:53.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.453+0000 7f8d4893e700 1 --2- 172.21.15.89:0/142485194 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d44159ec0 0x7f8d44157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:53.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.453+0000 7f8d43fff700 1 -- 172.21.15.89:0/142485194 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d44159ec0 msgr2=0x7f8d44157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:53.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.453+0000 7f8d43fff700 1 --2- 172.21.15.89:0/142485194 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d44159ec0 0x7f8d44157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:53.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.453+0000 7f8d43fff700 1 -- 172.21.15.89:0/142485194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d440cce40 msgr2=0x7f8d4416e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:53.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.453+0000 7f8d43fff700 1 --2- 172.21.15.89:0/142485194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d440cce40 0x7f8d4416e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:53.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.453+0000 7f8d43fff700 1 -- 172.21.15.89:0/142485194 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d380094f0 con 0x7f8d440b53d0 2023-10-10T20:35:53.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.453+0000 7f8d437fe700 1 --2- 172.21.15.89:0/142485194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d440cce40 0x7f8d4416e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:53.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.453+0000 7f8d43fff700 1 --2- 172.21.15.89:0/142485194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d440b53d0 0x7f8d44068a70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400d890 tx=0x7f8d34008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:53.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.453+0000 7f8d2bfff700 1 -- 172.21.15.89:0/142485194 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34005a00 con 0x7f8d440b53d0 2023-10-10T20:35:53.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.453+0000 7f8d2bfff700 1 -- 172.21.15.89:0/142485194 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d34005b60 con 0x7f8d440b53d0 2023-10-10T20:35:53.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.453+0000 7f8d4a3a1700 1 -- 172.21.15.89:0/142485194 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d44156070 con 0x7f8d440b53d0 2023-10-10T20:35:53.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.453+0000 7f8d2bfff700 1 -- 172.21.15.89:0/142485194 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d340125f0 con 0x7f8d440b53d0 2023-10-10T20:35:53.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.453+0000 7f8d4a3a1700 1 -- 172.21.15.89:0/142485194 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d441580f0 con 0x7f8d440b53d0 2023-10-10T20:35:53.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.454+0000 7f8d4a3a1700 1 -- 172.21.15.89:0/142485194 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d44069530 con 0x7f8d440b53d0 2023-10-10T20:35:53.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.455+0000 7f8d2bfff700 1 -- 172.21.15.89:0/142485194 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8d340129a0 con 0x7f8d440b53d0 2023-10-10T20:35:53.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.455+0000 7f8d2bfff700 1 --2- 172.21.15.89:0/142485194 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8d30066f30 0x7f8d300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:53.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.455+0000 7f8d2bfff700 1 -- 172.21.15.89:0/142485194 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8d34083f50 con 0x7f8d440b53d0 2023-10-10T20:35:53.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.455+0000 7f8d437fe700 1 --2- 172.21.15.89:0/142485194 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8d30066f30 0x7f8d300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:53.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.455+0000 7f8d437fe700 1 --2- 172.21.15.89:0/142485194 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8d30066f30 0x7f8d300693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8d44089120 tx=0x7f8d38017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:53.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.459+0000 7f8d2bfff700 1 -- 172.21.15.89:0/142485194 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d3404d5a0 con 0x7f8d440b53d0 2023-10-10T20:35:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.640+0000 7f8d4a3a1700 1 -- 172.21.15.89:0/142485194 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8d44063c90 con 0x7f8d440b53d0 2023-10-10T20:35:53.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.921+0000 7f8d2bfff700 1 -- 172.21.15.89:0/142485194 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8d34051510 con 0x7f8d440b53d0 2023-10-10T20:35:53.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.924+0000 7f8d4a3a1700 1 -- 172.21.15.89:0/142485194 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8d30066f30 msgr2=0x7f8d300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:53.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.924+0000 7f8d4a3a1700 1 --2- 172.21.15.89:0/142485194 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8d30066f30 0x7f8d300693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8d44089120 tx=0x7f8d38017000 comp rx=0 tx=0).stop 2023-10-10T20:35:53.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.924+0000 7f8d4a3a1700 1 -- 172.21.15.89:0/142485194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d440b53d0 msgr2=0x7f8d44068a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:53.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.924+0000 7f8d4a3a1700 1 --2- 172.21.15.89:0/142485194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d440b53d0 0x7f8d44068a70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400d890 tx=0x7f8d34008040 comp rx=0 tx=0).stop 2023-10-10T20:35:53.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.924+0000 7f8d4a3a1700 1 -- 172.21.15.89:0/142485194 shutdown_connections 2023-10-10T20:35:53.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.924+0000 7f8d4a3a1700 1 --2- 172.21.15.89:0/142485194 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d44159ec0 0x7f8d44157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:53.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.924+0000 7f8d4a3a1700 1 --2- 172.21.15.89:0/142485194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d440b53d0 0x7f8d44068a70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:53.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.924+0000 7f8d4a3a1700 1 --2- 172.21.15.89:0/142485194 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8d30066f30 0x7f8d300693e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:53.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.924+0000 7f8d4a3a1700 1 --2- 172.21.15.89:0/142485194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d440cce40 0x7f8d4416e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:53.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.924+0000 7f8d4a3a1700 1 -- 172.21.15.89:0/142485194 >> 172.21.15.89:0/142485194 conn(0x7f8d440999f0 msgr2=0x7f8d440ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:53.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.924+0000 7f8d4a3a1700 1 -- 172.21.15.89:0/142485194 shutdown_connections 2023-10-10T20:35:53.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:53.924+0000 7f8d4a3a1700 1 -- 172.21.15.89:0/142485194 wait complete. 2023-10-10T20:35:53.929 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2023-10-10T20:35:53.940 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:35:54.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.059+0000 7f6d02537700 1 Processor -- start 2023-10-10T20:35:54.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.059+0000 7f6d02537700 1 -- start start 2023-10-10T20:35:54.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.060+0000 7f6d02537700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6cfc159950 0x7f6cfc0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:54.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.060+0000 7f6d02537700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6cfc0af320 0x7f6cfc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:54.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.060+0000 7f6d02537700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6cfc069aa0 0x7f6cfc15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:54.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.060+0000 7f6d02537700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cfc069f20 con 0x7f6cfc159950 2023-10-10T20:35:54.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.060+0000 7f6d02537700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cfc0ce380 con 0x7f6cfc069aa0 2023-10-10T20:35:54.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.060+0000 7f6d02537700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc0cec20 con 0x7f6cfc0af320 2023-10-10T20:35:54.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.060+0000 7f6cfbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6cfc159950 0x7f6cfc0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:54.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.060+0000 7f6cfbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6cfc159950 0x7f6cfc0cbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60512/0 (socket says 172.21.15.89:60512) 2023-10-10T20:35:54.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.060+0000 7f6cfbfff700 1 -- 172.21.15.89:0/610275149 learned_addr learned my addr 172.21.15.89:0/610275149 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:54.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.060+0000 7f6cfb7fe700 1 --2- 172.21.15.89:0/610275149 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6cfc0af320 0x7f6cfc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:54.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.060+0000 7f6d00ad4700 1 --2- 172.21.15.89:0/610275149 >> v2:172.21.15.111:3300/0 conn(0x7f6cfc069aa0 0x7f6cfc15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:54.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.060+0000 7f6cfbfff700 1 -- 172.21.15.89:0/610275149 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6cfc0af320 msgr2=0x7f6cfc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:54.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.060+0000 7f6cfbfff700 1 --2- 172.21.15.89:0/610275149 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6cfc0af320 0x7f6cfc069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:54.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.060+0000 7f6cfbfff700 1 -- 172.21.15.89:0/610275149 >> v2:172.21.15.111:3300/0 conn(0x7f6cfc069aa0 msgr2=0x7f6cfc15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:54.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.060+0000 7f6cfbfff700 1 --2- 172.21.15.89:0/610275149 >> v2:172.21.15.111:3300/0 conn(0x7f6cfc069aa0 0x7f6cfc15d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:54.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.060+0000 7f6cfbfff700 1 -- 172.21.15.89:0/610275149 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cfc0afa90 con 0x7f6cfc159950 2023-10-10T20:35:54.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.061+0000 7f6cfbfff700 1 --2- 172.21.15.89:0/610275149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cfc159950 0x7f6cfc0cbb90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0d0760 tx=0x7f6cec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2037be25d66acbaf server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:54.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.061+0000 7f6cfaffd700 1 -- 172.21.15.89:0/610275149 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec00b860 con 0x7f6cfc159950 2023-10-10T20:35:54.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.061+0000 7f6cfaffd700 1 -- 172.21.15.89:0/610275149 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cec00b9c0 con 0x7f6cfc159950 2023-10-10T20:35:54.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.061+0000 7f6cfaffd700 1 -- 172.21.15.89:0/610275149 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec00bd60 con 0x7f6cfc159950 2023-10-10T20:35:54.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.061+0000 7f6d02537700 1 -- 172.21.15.89:0/610275149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cfc159950 msgr2=0x7f6cfc0cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:54.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.061+0000 7f6d02537700 1 --2- 172.21.15.89:0/610275149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cfc159950 0x7f6cfc0cbb90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0d0760 tx=0x7f6cec009250 comp rx=0 tx=0).stop 2023-10-10T20:35:54.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.061+0000 7f6d02537700 1 -- 172.21.15.89:0/610275149 shutdown_connections 2023-10-10T20:35:54.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.061+0000 7f6d02537700 1 --2- 172.21.15.89:0/610275149 >> v2:172.21.15.111:3300/0 conn(0x7f6cfc069aa0 0x7f6cfc15d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:54.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.061+0000 7f6d02537700 1 --2- 172.21.15.89:0/610275149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cfc159950 0x7f6cfc0cbb90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:54.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.061+0000 7f6d02537700 1 --2- 172.21.15.89:0/610275149 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6cfc0af320 0x7f6cfc069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:54.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.061+0000 7f6d02537700 1 -- 172.21.15.89:0/610275149 >> 172.21.15.89:0/610275149 conn(0x7f6cfc068e00 msgr2=0x7f6cfc0d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:54.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.061+0000 7f6d02537700 1 -- 172.21.15.89:0/610275149 shutdown_connections 2023-10-10T20:35:54.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.061+0000 7f6d02537700 1 -- 172.21.15.89:0/610275149 wait complete. 2023-10-10T20:35:54.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.062+0000 7f6d02537700 1 Processor -- start 2023-10-10T20:35:54.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.063+0000 7f6d02537700 1 -- start start 2023-10-10T20:35:54.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.063+0000 7f6d02537700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cfc069aa0 0x7f6cfc151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:54.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.063+0000 7f6d02537700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cfc0af320 0x7f6cfc0ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:54.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.063+0000 7f6d02537700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6cfc159950 0x7f6cfc0b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:54.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.063+0000 7f6d02537700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc151760 con 0x7f6cfc0af320 2023-10-10T20:35:54.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.063+0000 7f6d02537700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc1508f0 con 0x7f6cfc069aa0 2023-10-10T20:35:54.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.063+0000 7f6d02537700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc150b90 con 0x7f6cfc159950 2023-10-10T20:35:54.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.063+0000 7f6cfb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cfc0af320 0x7f6cfc0ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:54.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.063+0000 7f6cfb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cfc0af320 0x7f6cfc0ca390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60516/0 (socket says 172.21.15.89:60516) 2023-10-10T20:35:54.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.063+0000 7f6cfb7fe700 1 -- 172.21.15.89:0/3886387708 learned_addr learned my addr 172.21.15.89:0/3886387708 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:54.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.064+0000 7f6d00ad4700 1 --2- 172.21.15.89:0/3886387708 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6cfc159950 0x7f6cfc0b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:54.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.064+0000 7f6cfbfff700 1 --2- 172.21.15.89:0/3886387708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cfc069aa0 0x7f6cfc151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:54.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.064+0000 7f6cfb7fe700 1 -- 172.21.15.89:0/3886387708 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6cfc159950 msgr2=0x7f6cfc0b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:54.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.064+0000 7f6cfb7fe700 1 --2- 172.21.15.89:0/3886387708 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6cfc159950 0x7f6cfc0b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:54.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.064+0000 7f6cfb7fe700 1 -- 172.21.15.89:0/3886387708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cfc069aa0 msgr2=0x7f6cfc151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:54.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.064+0000 7f6cfb7fe700 1 --2- 172.21.15.89:0/3886387708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cfc069aa0 0x7f6cfc151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:54.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.064+0000 7f6cfb7fe700 1 -- 172.21.15.89:0/3886387708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec0094f0 con 0x7f6cfc0af320 2023-10-10T20:35:54.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.064+0000 7f6cfb7fe700 1 --2- 172.21.15.89:0/3886387708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cfc0af320 0x7f6cfc0ca390 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000d890 tx=0x7f6cf0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:54.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.064+0000 7f6ce3fff700 1 -- 172.21.15.89:0/3886387708 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf00125f0 con 0x7f6cfc0af320 2023-10-10T20:35:54.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.064+0000 7f6ce3fff700 1 -- 172.21.15.89:0/3886387708 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf0012750 con 0x7f6cfc0af320 2023-10-10T20:35:54.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.064+0000 7f6d02537700 1 -- 172.21.15.89:0/3886387708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cfc150e30 con 0x7f6cfc0af320 2023-10-10T20:35:54.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.064+0000 7f6ce3fff700 1 -- 172.21.15.89:0/3886387708 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0012af0 con 0x7f6cfc0af320 2023-10-10T20:35:54.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.064+0000 7f6d02537700 1 -- 172.21.15.89:0/3886387708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cfc1513f0 con 0x7f6cfc0af320 2023-10-10T20:35:54.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.065+0000 7f6d02537700 1 -- 172.21.15.89:0/3886387708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cfc0718b0 con 0x7f6cfc0af320 2023-10-10T20:35:54.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.066+0000 7f6ce3fff700 1 -- 172.21.15.89:0/3886387708 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6cf001e460 con 0x7f6cfc0af320 2023-10-10T20:35:54.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.066+0000 7f6ce3fff700 1 --2- 172.21.15.89:0/3886387708 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ce8066f30 0x7f6ce80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:54.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.066+0000 7f6cfbfff700 1 --2- 172.21.15.89:0/3886387708 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ce8066f30 0x7f6ce80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:54.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.066+0000 7f6ce3fff700 1 -- 172.21.15.89:0/3886387708 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f6cf00848c0 con 0x7f6cfc0af320 2023-10-10T20:35:54.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.066+0000 7f6cfbfff700 1 --2- 172.21.15.89:0/3886387708 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ce8066f30 0x7f6ce80693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0d0760 tx=0x7f6cec017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:54.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.071+0000 7f6ce3fff700 1 -- 172.21.15.89:0/3886387708 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6cf00557a0 con 0x7f6cfc0af320 2023-10-10T20:35:54.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.254+0000 7f6d02537700 1 -- 172.21.15.89:0/3886387708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6cfc0ce5d0 con 0x7f6cfc0af320 2023-10-10T20:35:54.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.930+0000 7f6ce3fff700 1 -- 172.21.15.89:0/3886387708 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v48) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6cf0055330 con 0x7f6cfc0af320 2023-10-10T20:35:54.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.937+0000 7f6d02537700 1 -- 172.21.15.89:0/3886387708 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ce8066f30 msgr2=0x7f6ce80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:54.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.937+0000 7f6d02537700 1 --2- 172.21.15.89:0/3886387708 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ce8066f30 0x7f6ce80693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0d0760 tx=0x7f6cec017000 comp rx=0 tx=0).stop 2023-10-10T20:35:54.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.937+0000 7f6d02537700 1 -- 172.21.15.89:0/3886387708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cfc0af320 msgr2=0x7f6cfc0ca390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:54.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.937+0000 7f6d02537700 1 --2- 172.21.15.89:0/3886387708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cfc0af320 0x7f6cfc0ca390 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000d890 tx=0x7f6cf0008040 comp rx=0 tx=0).stop 2023-10-10T20:35:54.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.938+0000 7f6d02537700 1 -- 172.21.15.89:0/3886387708 shutdown_connections 2023-10-10T20:35:54.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.938+0000 7f6d02537700 1 --2- 172.21.15.89:0/3886387708 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6cfc159950 0x7f6cfc0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:54.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.938+0000 7f6d02537700 1 --2- 172.21.15.89:0/3886387708 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ce8066f30 0x7f6ce80693e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:54.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.938+0000 7f6d02537700 1 --2- 172.21.15.89:0/3886387708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cfc069aa0 0x7f6cfc151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:54.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.938+0000 7f6d02537700 1 --2- 172.21.15.89:0/3886387708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cfc0af320 0x7f6cfc0ca390 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:54.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.938+0000 7f6d02537700 1 -- 172.21.15.89:0/3886387708 >> 172.21.15.89:0/3886387708 conn(0x7f6cfc068e00 msgr2=0x7f6cfc0b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:54.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.938+0000 7f6d02537700 1 -- 172.21.15.89:0/3886387708 shutdown_connections 2023-10-10T20:35:54.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:54.938+0000 7f6d02537700 1 -- 172.21.15.89:0/3886387708 wait complete. 2023-10-10T20:35:54.942 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2023-10-10T20:35:54.957 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:35:55.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.072+0000 7f8335e87700 1 Processor -- start 2023-10-10T20:35:55.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.072+0000 7f8335e87700 1 -- start start 2023-10-10T20:35:55.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f8335e87700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8330159a60 0x7f83300c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:55.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f8335e87700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f83300ccc00 0x7f8330069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:55.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f8335e87700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f833006a020 0x7f83300cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:55.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f8335e87700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833006dfc0 con 0x7f8330159a60 2023-10-10T20:35:55.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f8335e87700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83300c9cb0 con 0x7f83300ccc00 2023-10-10T20:35:55.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f8335e87700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83300c9f30 con 0x7f833006a020 2023-10-10T20:35:55.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f8334e85700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8330159a60 0x7f83300c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:55.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f8334e85700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8330159a60 0x7f83300c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60530/0 (socket says 172.21.15.89:60530) 2023-10-10T20:35:55.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f8334e85700 1 -- 172.21.15.89:0/1771262205 learned_addr learned my addr 172.21.15.89:0/1771262205 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:55.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f8335686700 1 --2- 172.21.15.89:0/1771262205 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f833006a020 0x7f83300cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:55.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f832ffff700 1 --2- 172.21.15.89:0/1771262205 >> v2:172.21.15.111:3300/0 conn(0x7f83300ccc00 0x7f8330069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f8334e85700 1 -- 172.21.15.89:0/1771262205 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f833006a020 msgr2=0x7f83300cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f8334e85700 1 --2- 172.21.15.89:0/1771262205 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f833006a020 0x7f83300cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f8334e85700 1 -- 172.21.15.89:0/1771262205 >> v2:172.21.15.111:3300/0 conn(0x7f83300ccc00 msgr2=0x7f8330069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f8334e85700 1 --2- 172.21.15.89:0/1771262205 >> v2:172.21.15.111:3300/0 conn(0x7f83300ccc00 0x7f8330069ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f8334e85700 1 -- 172.21.15.89:0/1771262205 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83300ca470 con 0x7f8330159a60 2023-10-10T20:35:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f8335686700 1 --2- 172.21.15.89:0/1771262205 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f833006a020 0x7f83300cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f832ffff700 1 --2- 172.21.15.89:0/1771262205 >> v2:172.21.15.111:3300/0 conn(0x7f83300ccc00 0x7f8330069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f8334e85700 1 --2- 172.21.15.89:0/1771262205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8330159a60 0x7f83300c9770 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f83300cf6a0 tx=0x7f8324009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8866fef824acc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f832f7fe700 1 -- 172.21.15.89:0/1771262205 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832400b8f0 con 0x7f8330159a60 2023-10-10T20:35:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f832f7fe700 1 -- 172.21.15.89:0/1771262205 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832400ba50 con 0x7f8330159a60 2023-10-10T20:35:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.073+0000 7f832f7fe700 1 -- 172.21.15.89:0/1771262205 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832400bdf0 con 0x7f8330159a60 2023-10-10T20:35:55.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.074+0000 7f8335e87700 1 -- 172.21.15.89:0/1771262205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8330159a60 msgr2=0x7f83300c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:55.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.074+0000 7f8335e87700 1 --2- 172.21.15.89:0/1771262205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8330159a60 0x7f83300c9770 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f83300cf6a0 tx=0x7f8324009250 comp rx=0 tx=0).stop 2023-10-10T20:35:55.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.074+0000 7f8335e87700 1 -- 172.21.15.89:0/1771262205 shutdown_connections 2023-10-10T20:35:55.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.074+0000 7f8335e87700 1 --2- 172.21.15.89:0/1771262205 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f833006a020 0x7f83300cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:55.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.074+0000 7f8335e87700 1 --2- 172.21.15.89:0/1771262205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8330159a60 0x7f83300c9770 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:55.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.074+0000 7f8335e87700 1 --2- 172.21.15.89:0/1771262205 >> v2:172.21.15.111:3300/0 conn(0x7f83300ccc00 0x7f8330069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:55.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.074+0000 7f8335e87700 1 -- 172.21.15.89:0/1771262205 >> 172.21.15.89:0/1771262205 conn(0x7f833006baa0 msgr2=0x7f833006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:55.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.074+0000 7f8335e87700 1 -- 172.21.15.89:0/1771262205 shutdown_connections 2023-10-10T20:35:55.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.074+0000 7f8335e87700 1 -- 172.21.15.89:0/1771262205 wait complete. 2023-10-10T20:35:55.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.075+0000 7f8335e87700 1 Processor -- start 2023-10-10T20:35:55.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.075+0000 7f8335e87700 1 -- start start 2023-10-10T20:35:55.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.076+0000 7f8335e87700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f833006a020 0x7f8330156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:55.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.076+0000 7f8335e87700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83300ccc00 0x7f833015c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:55.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.076+0000 7f8335e87700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8330159a60 0x7f833015c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:55.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.076+0000 7f8335e87700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83300b30f0 con 0x7f8330159a60 2023-10-10T20:35:55.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.076+0000 7f8335e87700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833015ce70 con 0x7f833006a020 2023-10-10T20:35:55.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.076+0000 7f8335e87700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833015d150 con 0x7f83300ccc00 2023-10-10T20:35:55.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.076+0000 7f8335686700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8330159a60 0x7f833015c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:55.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.076+0000 7f8335686700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8330159a60 0x7f833015c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60546/0 (socket says 172.21.15.89:60546) 2023-10-10T20:35:55.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.076+0000 7f8335686700 1 -- 172.21.15.89:0/3938686939 learned_addr learned my addr 172.21.15.89:0/3938686939 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:55.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.076+0000 7f8334e85700 1 --2- 172.21.15.89:0/3938686939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f833006a020 0x7f8330156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:55.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.076+0000 7f832ffff700 1 --2- 172.21.15.89:0/3938686939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83300ccc00 0x7f833015c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:55.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.076+0000 7f8335686700 1 -- 172.21.15.89:0/3938686939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83300ccc00 msgr2=0x7f833015c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:55.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.076+0000 7f8335686700 1 --2- 172.21.15.89:0/3938686939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83300ccc00 0x7f833015c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:55.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.076+0000 7f8335686700 1 -- 172.21.15.89:0/3938686939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f833006a020 msgr2=0x7f8330156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:55.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.076+0000 7f8335686700 1 --2- 172.21.15.89:0/3938686939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f833006a020 0x7f8330156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:55.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.076+0000 7f8335686700 1 -- 172.21.15.89:0/3938686939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83240094f0 con 0x7f8330159a60 2023-10-10T20:35:55.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.077+0000 7f8335686700 1 --2- 172.21.15.89:0/3938686939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8330159a60 0x7f833015c840 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f832800cf60 tx=0x7f8328007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:55.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.077+0000 7f832cff9700 1 -- 172.21.15.89:0/3938686939 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832800ae40 con 0x7f8330159a60 2023-10-10T20:35:55.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.077+0000 7f832cff9700 1 -- 172.21.15.89:0/3938686939 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8328015440 con 0x7f8330159a60 2023-10-10T20:35:55.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.077+0000 7f8335e87700 1 -- 172.21.15.89:0/3938686939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833015d430 con 0x7f8330159a60 2023-10-10T20:35:55.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.077+0000 7f832cff9700 1 -- 172.21.15.89:0/3938686939 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328015800 con 0x7f8330159a60 2023-10-10T20:35:55.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.077+0000 7f8335e87700 1 -- 172.21.15.89:0/3938686939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833015d9f0 con 0x7f8330159a60 2023-10-10T20:35:55.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.078+0000 7f8335e87700 1 -- 172.21.15.89:0/3938686939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f833015dc10 con 0x7f8330159a60 2023-10-10T20:35:55.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.078+0000 7f832cff9700 1 -- 172.21.15.89:0/3938686939 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f832800d440 con 0x7f8330159a60 2023-10-10T20:35:55.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.078+0000 7f832cff9700 1 --2- 172.21.15.89:0/3938686939 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8314066f30 0x7f83140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:55.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.079+0000 7f832cff9700 1 -- 172.21.15.89:0/3938686939 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f83280850f0 con 0x7f8330159a60 2023-10-10T20:35:55.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.079+0000 7f8334e85700 1 --2- 172.21.15.89:0/3938686939 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8314066f30 0x7f83140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:55.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.079+0000 7f8334e85700 1 --2- 172.21.15.89:0/3938686939 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8314066f30 0x7f83140693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f83300cf6a0 tx=0x7f8324017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:55.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.083+0000 7f832cff9700 1 -- 172.21.15.89:0/3938686939 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83280523c0 con 0x7f8330159a60 2023-10-10T20:35:55.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.312+0000 7f8335e87700 1 -- 172.21.15.89:0/3938686939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f83300ca750 con 0x7f8330159a60 2023-10-10T20:35:55.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.933+0000 7f832cff9700 1 -- 172.21.15.89:0/3938686939 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v26) v1 ==== 172+0+0 (secure 0 0 0) 0x7f83280559e0 con 0x7f8330159a60 2023-10-10T20:35:55.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.937+0000 7f8335e87700 1 -- 172.21.15.89:0/3938686939 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8314066f30 msgr2=0x7f83140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:55.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.937+0000 7f8335e87700 1 --2- 172.21.15.89:0/3938686939 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8314066f30 0x7f83140693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f83300cf6a0 tx=0x7f8324017000 comp rx=0 tx=0).stop 2023-10-10T20:35:55.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.937+0000 7f8335e87700 1 -- 172.21.15.89:0/3938686939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8330159a60 msgr2=0x7f833015c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:55.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.937+0000 7f8335e87700 1 --2- 172.21.15.89:0/3938686939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8330159a60 0x7f833015c840 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f832800cf60 tx=0x7f8328007870 comp rx=0 tx=0).stop 2023-10-10T20:35:55.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.938+0000 7f8335e87700 1 -- 172.21.15.89:0/3938686939 shutdown_connections 2023-10-10T20:35:55.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.938+0000 7f8335e87700 1 --2- 172.21.15.89:0/3938686939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8330159a60 0x7f833015c840 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:55.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.938+0000 7f8335e87700 1 --2- 172.21.15.89:0/3938686939 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8314066f30 0x7f83140693e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:55.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.938+0000 7f8335e87700 1 --2- 172.21.15.89:0/3938686939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f833006a020 0x7f8330156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:55.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.938+0000 7f8335e87700 1 --2- 172.21.15.89:0/3938686939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83300ccc00 0x7f833015c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:55.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.938+0000 7f8335e87700 1 -- 172.21.15.89:0/3938686939 >> 172.21.15.89:0/3938686939 conn(0x7f833006baa0 msgr2=0x7f83300cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:55.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.939+0000 7f8335e87700 1 -- 172.21.15.89:0/3938686939 shutdown_connections 2023-10-10T20:35:55.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:55.939+0000 7f8335e87700 1 -- 172.21.15.89:0/3938686939 wait complete. 2023-10-10T20:35:55.954 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 8 and data pool 9 2023-10-10T20:35:55.965 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:35:56.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.077+0000 7fc804d54700 1 Processor -- start 2023-10-10T20:35:56.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.077+0000 7fc804d54700 1 -- start start 2023-10-10T20:35:56.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc804d54700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc800159890 0x7fc80015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc804d54700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc800159c60 0x7fc80015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc804d54700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc80015db10 0x7fc800069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc804d54700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc80015df50 con 0x7fc800159890 2023-10-10T20:35:56.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc804d54700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc80015a160 con 0x7fc800159c60 2023-10-10T20:35:56.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc804d54700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8000d13a0 con 0x7fc80015db10 2023-10-10T20:35:56.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc7fe59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc800159890 0x7fc80015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc7fe59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc800159890 0x7fc80015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60554/0 (socket says 172.21.15.89:60554) 2023-10-10T20:35:56.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc7fe59c700 1 -- 172.21.15.89:0/3341677475 learned_addr learned my addr 172.21.15.89:0/3341677475 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:56.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc7fdd9b700 1 --2- 172.21.15.89:0/3341677475 >> v2:172.21.15.111:3300/0 conn(0x7fc800159c60 0x7fc80015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc7fe59c700 1 -- 172.21.15.89:0/3341677475 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc80015db10 msgr2=0x7fc800069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:56.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc7fed9d700 1 --2- 172.21.15.89:0/3341677475 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc80015db10 0x7fc800069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc7fe59c700 1 --2- 172.21.15.89:0/3341677475 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc80015db10 0x7fc800069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc7fe59c700 1 -- 172.21.15.89:0/3341677475 >> v2:172.21.15.111:3300/0 conn(0x7fc800159c60 msgr2=0x7fc80015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:56.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc7fe59c700 1 --2- 172.21.15.89:0/3341677475 >> v2:172.21.15.111:3300/0 conn(0x7fc800159c60 0x7fc80015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc7fe59c700 1 -- 172.21.15.89:0/3341677475 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80015e140 con 0x7fc800159890 2023-10-10T20:35:56.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc7fdd9b700 1 --2- 172.21.15.89:0/3341677475 >> v2:172.21.15.111:3300/0 conn(0x7fc800159c60 0x7fc80015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:56.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc7fe59c700 1 --2- 172.21.15.89:0/3341677475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc800159890 0x7fc80015abd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc8000b67f0 tx=0x7fc7f8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f498fe22ffe43f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:56.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc7fd59a700 1 -- 172.21.15.89:0/3341677475 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f800e070 con 0x7fc800159890 2023-10-10T20:35:56.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc7fd59a700 1 -- 172.21.15.89:0/3341677475 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f800ba50 con 0x7fc800159890 2023-10-10T20:35:56.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.078+0000 7fc7fd59a700 1 -- 172.21.15.89:0/3341677475 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f80092e0 con 0x7fc800159890 2023-10-10T20:35:56.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.079+0000 7fc804d54700 1 -- 172.21.15.89:0/3341677475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc800159890 msgr2=0x7fc80015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:56.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.079+0000 7fc804d54700 1 --2- 172.21.15.89:0/3341677475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc800159890 0x7fc80015abd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc8000b67f0 tx=0x7fc7f8009840 comp rx=0 tx=0).stop 2023-10-10T20:35:56.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.079+0000 7fc804d54700 1 -- 172.21.15.89:0/3341677475 shutdown_connections 2023-10-10T20:35:56.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.079+0000 7fc804d54700 1 --2- 172.21.15.89:0/3341677475 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc80015db10 0x7fc800069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.079+0000 7fc804d54700 1 --2- 172.21.15.89:0/3341677475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc800159890 0x7fc80015abd0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.079+0000 7fc804d54700 1 --2- 172.21.15.89:0/3341677475 >> v2:172.21.15.111:3300/0 conn(0x7fc800159c60 0x7fc80015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.079+0000 7fc804d54700 1 -- 172.21.15.89:0/3341677475 >> 172.21.15.89:0/3341677475 conn(0x7fc800068e20 msgr2=0x7fc8000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:56.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.079+0000 7fc804d54700 1 -- 172.21.15.89:0/3341677475 shutdown_connections 2023-10-10T20:35:56.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.079+0000 7fc804d54700 1 -- 172.21.15.89:0/3341677475 wait complete. 2023-10-10T20:35:56.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.080+0000 7fc804d54700 1 Processor -- start 2023-10-10T20:35:56.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.080+0000 7fc804d54700 1 -- start start 2023-10-10T20:35:56.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc804d54700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc800159890 0x7fc8000b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc804d54700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc800159c60 0x7fc80015cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc804d54700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc80015db10 0x7fc80015d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc804d54700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8001557a0 con 0x7fc800159890 2023-10-10T20:35:56.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc804d54700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc800155f10 con 0x7fc800159c60 2023-10-10T20:35:56.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc804d54700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8001561f0 con 0x7fc80015db10 2023-10-10T20:35:56.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc7fe59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc800159890 0x7fc8000b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc7fe59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc800159890 0x7fc8000b5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60566/0 (socket says 172.21.15.89:60566) 2023-10-10T20:35:56.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc7fe59c700 1 -- 172.21.15.89:0/4085467652 learned_addr learned my addr 172.21.15.89:0/4085467652 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:56.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc7fdd9b700 1 --2- 172.21.15.89:0/4085467652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc800159c60 0x7fc80015cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc7fed9d700 1 --2- 172.21.15.89:0/4085467652 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc80015db10 0x7fc80015d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc7fe59c700 1 -- 172.21.15.89:0/4085467652 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc80015db10 msgr2=0x7fc80015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:56.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc7fe59c700 1 --2- 172.21.15.89:0/4085467652 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc80015db10 0x7fc80015d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc7fe59c700 1 -- 172.21.15.89:0/4085467652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc800159c60 msgr2=0x7fc80015cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:56.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc7fe59c700 1 --2- 172.21.15.89:0/4085467652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc800159c60 0x7fc80015cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc7fe59c700 1 -- 172.21.15.89:0/4085467652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f8009ae0 con 0x7fc800159890 2023-10-10T20:35:56.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc7fed9d700 1 --2- 172.21.15.89:0/4085467652 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc80015db10 0x7fc80015d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:56.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc7fe59c700 1 --2- 172.21.15.89:0/4085467652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc800159890 0x7fc8000b5c00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009880 tx=0x7fc7f8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:56.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc7f67fc700 1 -- 172.21.15.89:0/4085467652 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8008a80 con 0x7fc800159890 2023-10-10T20:35:56.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc7f67fc700 1 -- 172.21.15.89:0/4085467652 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f8008be0 con 0x7fc800159890 2023-10-10T20:35:56.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.081+0000 7fc804d54700 1 -- 172.21.15.89:0/4085467652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8001564d0 con 0x7fc800159890 2023-10-10T20:35:56.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.082+0000 7fc7f67fc700 1 -- 172.21.15.89:0/4085467652 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f801c410 con 0x7fc800159890 2023-10-10T20:35:56.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.082+0000 7fc804d54700 1 -- 172.21.15.89:0/4085467652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc800156a90 con 0x7fc800159890 2023-10-10T20:35:56.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.082+0000 7fc804d54700 1 -- 172.21.15.89:0/4085467652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc800156c80 con 0x7fc800159890 2023-10-10T20:35:56.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.083+0000 7fc7f67fc700 1 -- 172.21.15.89:0/4085467652 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc7f8003040 con 0x7fc800159890 2023-10-10T20:35:56.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.083+0000 7fc7f67fc700 1 --2- 172.21.15.89:0/4085467652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc7e0066f30 0x7fc7e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.083+0000 7fc7f67fc700 1 -- 172.21.15.89:0/4085467652 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc7f8085d40 con 0x7fc800159890 2023-10-10T20:35:56.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.083+0000 7fc7fdd9b700 1 --2- 172.21.15.89:0/4085467652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc7e0066f30 0x7fc7e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.084+0000 7fc7fdd9b700 1 --2- 172.21.15.89:0/4085467652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc7e0066f30 0x7fc7e00693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc8000ce2c0 tx=0x7fc7e8006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:56.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.088+0000 7fc7f67fc700 1 -- 172.21.15.89:0/4085467652 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7f8052fb0 con 0x7fc800159890 2023-10-10T20:35:56.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.272+0000 7fc804d54700 1 -- 172.21.15.89:0/4085467652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc80004c3c0 con 0x7fc800159890 2023-10-10T20:35:56.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.273+0000 7fc7f67fc700 1 -- 172.21.15.89:0/4085467652 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+15363 (secure 0 0 0) 0x7fc7f80565d0 con 0x7fc800159890 2023-10-10T20:35:56.275 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:56.275 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":49,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:35:55.314177+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:35:53.658728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-10T20:35:54.272054+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:35:56.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.275+0000 7fc804d54700 1 -- 172.21.15.89:0/4085467652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc7e0066f30 msgr2=0x7fc7e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:56.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.275+0000 7fc804d54700 1 --2- 172.21.15.89:0/4085467652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc7e0066f30 0x7fc7e00693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc8000ce2c0 tx=0x7fc7e8006f90 comp rx=0 tx=0).stop 2023-10-10T20:35:56.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.275+0000 7fc804d54700 1 -- 172.21.15.89:0/4085467652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc800159890 msgr2=0x7fc8000b5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:56.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.275+0000 7fc804d54700 1 --2- 172.21.15.89:0/4085467652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc800159890 0x7fc8000b5c00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009880 tx=0x7fc7f8018000 comp rx=0 tx=0).stop 2023-10-10T20:35:56.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.275+0000 7fc804d54700 1 -- 172.21.15.89:0/4085467652 shutdown_connections 2023-10-10T20:35:56.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.275+0000 7fc804d54700 1 --2- 172.21.15.89:0/4085467652 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc80015db10 0x7fc80015d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.275+0000 7fc804d54700 1 --2- 172.21.15.89:0/4085467652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc800159890 0x7fc8000b5c00 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.275+0000 7fc804d54700 1 --2- 172.21.15.89:0/4085467652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc7e0066f30 0x7fc7e00693e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.275+0000 7fc804d54700 1 --2- 172.21.15.89:0/4085467652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc800159c60 0x7fc80015cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.275+0000 7fc804d54700 1 -- 172.21.15.89:0/4085467652 >> 172.21.15.89:0/4085467652 conn(0x7fc800068e20 msgr2=0x7fc80015f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:56.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.275+0000 7fc804d54700 1 -- 172.21.15.89:0/4085467652 shutdown_connections 2023-10-10T20:35:56.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.275+0000 7fc804d54700 1 -- 172.21.15.89:0/4085467652 wait complete. 2023-10-10T20:35:56.289 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:35:56.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.401+0000 7fb82ceb8700 1 Processor -- start 2023-10-10T20:35:56.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.401+0000 7fb82ceb8700 1 -- start start 2023-10-10T20:35:56.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.402+0000 7fb82ceb8700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb82806c0f0 0x7fb8281642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.402+0000 7fb82ceb8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8280b40f0 con 0x7fb828142ea0 2023-10-10T20:35:56.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.402+0000 7fb82ceb8700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb828069640 con 0x7fb8280ad870 2023-10-10T20:35:56.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.402+0000 7fb82ceb8700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb828069920 con 0x7fb82806c0f0 2023-10-10T20:35:56.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.402+0000 7fb8277fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb828142ea0 0x7fb8280b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52740/0 (socket says 172.21.15.89:52740) 2023-10-10T20:35:56.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.402+0000 7fb8277fe700 1 -- 172.21.15.89:0/3012184876 learned_addr learned my addr 172.21.15.89:0/3012184876 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:56.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.402+0000 7fb827fff700 1 --2- 172.21.15.89:0/3012184876 >> v2:172.21.15.111:3300/0 conn(0x7fb82806c0f0 0x7fb8281642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.402+0000 7fb8267fc700 1 -- 172.21.15.89:0/3012184876 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 688908633 0 0) 0x7fb8280b40f0 con 0x7fb828142ea0 2023-10-10T20:35:56.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.402+0000 7fb8267fc700 1 -- 172.21.15.89:0/3012184876 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb810003570 con 0x7fb828142ea0 2023-10-10T20:35:56.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.402+0000 7fb8267fc700 1 -- 172.21.15.89:0/3012184876 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2258674081 0 0) 0x7fb810003570 con 0x7fb828142ea0 2023-10-10T20:35:56.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.402+0000 7fb8267fc700 1 -- 172.21.15.89:0/3012184876 >> v2:172.21.15.111:3300/0 conn(0x7fb82806c0f0 msgr2=0x7fb8281642e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:56.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.402+0000 7fb8267fc700 1 --2- 172.21.15.89:0/3012184876 >> v2:172.21.15.111:3300/0 conn(0x7fb82806c0f0 0x7fb8281642e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.402+0000 7fb8267fc700 1 -- 172.21.15.89:0/3012184876 >> v1:172.21.15.111:6789/0 conn(0x7fb8280ad870 legacy=0x7fb82806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:56.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.402+0000 7fb8267fc700 1 -- 172.21.15.89:0/3012184876 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb828069c00 con 0x7fb828142ea0 2023-10-10T20:35:56.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb8267fc700 1 -- 172.21.15.89:0/3012184876 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fb8180017f0 con 0x7fb828142ea0 2023-10-10T20:35:56.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb8267fc700 1 -- 172.21.15.89:0/3012184876 >> v1:172.21.15.89:6789/0 conn(0x7fb828142ea0 legacy=0x7fb8280b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:56.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb8267fc700 1 --2- 172.21.15.89:0/3012184876 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb810003a60 0x7fb810003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb8267fc700 1 --2- 172.21.15.89:0/3012184876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb810004810 0x7fb810004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb8267fc700 1 --2- 172.21.15.89:0/3012184876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8100051c0 0x7fb81000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb8267fc700 1 -- 172.21.15.89:0/3012184876 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb828069920 con 0x7fb8100051c0 2023-10-10T20:35:56.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb8267fc700 1 -- 172.21.15.89:0/3012184876 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81000c0d0 con 0x7fb810004810 2023-10-10T20:35:56.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb8267fc700 1 -- 172.21.15.89:0/3012184876 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81000c290 con 0x7fb810003a60 2023-10-10T20:35:56.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb827fff700 1 --2- 172.21.15.89:0/3012184876 >> v2:172.21.15.111:3300/0 conn(0x7fb82806c0f0 0x7fb8281642e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:35:56.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb827fff700 1 --2- 172.21.15.89:0/3012184876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8100051c0 0x7fb81000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb8277fe700 1 --2- 172.21.15.89:0/3012184876 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb810003a60 0x7fb810003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb826ffd700 1 --2- 172.21.15.89:0/3012184876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb810004810 0x7fb810004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb827fff700 1 -- 172.21.15.89:0/3012184876 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb810003a60 msgr2=0x7fb810003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:56.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb827fff700 1 --2- 172.21.15.89:0/3012184876 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb810003a60 0x7fb810003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb827fff700 1 -- 172.21.15.89:0/3012184876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb810004810 msgr2=0x7fb810004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:56.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb827fff700 1 --2- 172.21.15.89:0/3012184876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb810004810 0x7fb810004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb827fff700 1 -- 172.21.15.89:0/3012184876 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb810003570 con 0x7fb8100051c0 2023-10-10T20:35:56.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb827fff700 1 -- 172.21.15.89:0/3012184876 reap_dead start 2023-10-10T20:35:56.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb827fff700 1 --2- 172.21.15.89:0/3012184876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8100051c0 0x7fb81000ba60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb82806c4f0 tx=0x7fb8200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad6fde186139b1e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:56.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb8267fc700 1 -- 172.21.15.89:0/3012184876 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820009740 con 0x7fb8100051c0 2023-10-10T20:35:56.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb8267fc700 1 -- 172.21.15.89:0/3012184876 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82000e920 con 0x7fb8100051c0 2023-10-10T20:35:56.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.403+0000 7fb8267fc700 1 -- 172.21.15.89:0/3012184876 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82000ece0 con 0x7fb8100051c0 2023-10-10T20:35:56.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.404+0000 7fb82ceb8700 1 -- 172.21.15.89:0/3012184876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8100051c0 msgr2=0x7fb81000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:56.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.404+0000 7fb82ceb8700 1 --2- 172.21.15.89:0/3012184876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8100051c0 0x7fb81000ba60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb82806c4f0 tx=0x7fb8200091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.404+0000 7fb82ceb8700 1 -- 172.21.15.89:0/3012184876 shutdown_connections 2023-10-10T20:35:56.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.404+0000 7fb82ceb8700 1 --2- 172.21.15.89:0/3012184876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8100051c0 0x7fb81000ba60 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.404+0000 7fb82ceb8700 1 -- 172.21.15.89:0/3012184876 >> 172.21.15.89:0/3012184876 conn(0x7fb8280ce430 msgr2=0x7fb8280ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:56.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.404+0000 7fb82ceb8700 1 -- 172.21.15.89:0/3012184876 shutdown_connections 2023-10-10T20:35:56.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.404+0000 7fb82ceb8700 1 -- 172.21.15.89:0/3012184876 wait complete. 2023-10-10T20:35:56.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.405+0000 7fb82ceb8700 1 Processor -- start 2023-10-10T20:35:56.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.405+0000 7fb82ceb8700 1 -- start start 2023-10-10T20:35:56.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.406+0000 7fb82ceb8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8100051c0 0x7fb8280c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.406+0000 7fb82ceb8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb82816b090 0x7fb82816a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.406+0000 7fb82ceb8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8280c75e0 0x7fb828140b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.406+0000 7fb82ceb8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8280c9b90 con 0x7fb8280c75e0 2023-10-10T20:35:56.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.406+0000 7fb82ceb8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82816ae70 con 0x7fb82816b090 2023-10-10T20:35:56.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.406+0000 7fb82ceb8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8280b0080 con 0x7fb8100051c0 2023-10-10T20:35:56.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.406+0000 7fb827fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8280c75e0 0x7fb828140b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.406+0000 7fb827fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8280c75e0 0x7fb828140b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60586/0 (socket says 172.21.15.89:60586) 2023-10-10T20:35:56.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.406+0000 7fb827fff700 1 -- 172.21.15.89:0/1543106419 learned_addr learned my addr 172.21.15.89:0/1543106419 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:56.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.406+0000 7fb826ffd700 1 --2- 172.21.15.89:0/1543106419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb82816b090 0x7fb82816a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.406+0000 7fb827fff700 1 -- 172.21.15.89:0/1543106419 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8100051c0 msgr2=0x7fb8280c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:56.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.406+0000 7fb827fff700 1 --2- 172.21.15.89:0/1543106419 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8100051c0 0x7fb8280c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.406+0000 7fb827fff700 1 -- 172.21.15.89:0/1543106419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb82816b090 msgr2=0x7fb82816a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:56.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.406+0000 7fb827fff700 1 --2- 172.21.15.89:0/1543106419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb82816b090 0x7fb82816a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.406+0000 7fb827fff700 1 -- 172.21.15.89:0/1543106419 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82816c1a0 con 0x7fb8280c75e0 2023-10-10T20:35:56.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.406+0000 7fb827fff700 1 --2- 172.21.15.89:0/1543106419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8280c75e0 0x7fb828140b10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb8280b2340 tx=0x7fb820019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:56.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.406+0000 7fb80f7fe700 1 -- 172.21.15.89:0/1543106419 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82000ee90 con 0x7fb8280c75e0 2023-10-10T20:35:56.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.407+0000 7fb80f7fe700 1 -- 172.21.15.89:0/1543106419 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb820021040 con 0x7fb8280c75e0 2023-10-10T20:35:56.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.407+0000 7fb80f7fe700 1 -- 172.21.15.89:0/1543106419 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82001b850 con 0x7fb8280c75e0 2023-10-10T20:35:56.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.407+0000 7fb82ceb8700 1 -- 172.21.15.89:0/1543106419 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82816b590 con 0x7fb8280c75e0 2023-10-10T20:35:56.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.407+0000 7fb82ceb8700 1 -- 172.21.15.89:0/1543106419 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82816bb50 con 0x7fb8280c75e0 2023-10-10T20:35:56.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.408+0000 7fb82ceb8700 1 -- 172.21.15.89:0/1543106419 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82804c3c0 con 0x7fb8280c75e0 2023-10-10T20:35:56.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.408+0000 7fb80f7fe700 1 -- 172.21.15.89:0/1543106419 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb82001bc40 con 0x7fb8280c75e0 2023-10-10T20:35:56.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.408+0000 7fb80f7fe700 1 --2- 172.21.15.89:0/1543106419 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb814066f30 0x7fb8140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.408+0000 7fb80f7fe700 1 -- 172.21.15.89:0/1543106419 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb820085ba0 con 0x7fb8280c75e0 2023-10-10T20:35:56.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.408+0000 7fb8277fe700 1 --2- 172.21.15.89:0/1543106419 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb814066f30 0x7fb8140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.409+0000 7fb8277fe700 1 --2- 172.21.15.89:0/1543106419 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb814066f30 0x7fb8140693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb81800a0c0 tx=0x7fb8180021d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:56.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.413+0000 7fb80f7fe700 1 -- 172.21.15.89:0/1543106419 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb820052e10 con 0x7fb8280c75e0 2023-10-10T20:35:56.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.597+0000 7fb82ceb8700 1 -- 172.21.15.89:0/1543106419 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb82816c330 con 0x7fb8280c75e0 2023-10-10T20:35:56.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.598+0000 7fb80f7fe700 1 -- 172.21.15.89:0/1543106419 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+15363 (secure 0 0 0) 0x7fb820056430 con 0x7fb8280c75e0 2023-10-10T20:35:56.600 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:56.600 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":49,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:35:55.314177+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:35:53.658728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-10T20:35:54.272054+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:35:56.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.600+0000 7fb82ceb8700 1 -- 172.21.15.89:0/1543106419 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb814066f30 msgr2=0x7fb8140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:56.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.600+0000 7fb82ceb8700 1 --2- 172.21.15.89:0/1543106419 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb814066f30 0x7fb8140693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb81800a0c0 tx=0x7fb8180021d0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.600+0000 7fb82ceb8700 1 -- 172.21.15.89:0/1543106419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8280c75e0 msgr2=0x7fb828140b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:56.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.600+0000 7fb82ceb8700 1 --2- 172.21.15.89:0/1543106419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8280c75e0 0x7fb828140b10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb8280b2340 tx=0x7fb820019000 comp rx=0 tx=0).stop 2023-10-10T20:35:56.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.600+0000 7fb82ceb8700 1 -- 172.21.15.89:0/1543106419 shutdown_connections 2023-10-10T20:35:56.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.600+0000 7fb82ceb8700 1 --2- 172.21.15.89:0/1543106419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8280c75e0 0x7fb828140b10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.600+0000 7fb82ceb8700 1 --2- 172.21.15.89:0/1543106419 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8100051c0 0x7fb8280c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.600+0000 7fb82ceb8700 1 --2- 172.21.15.89:0/1543106419 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb814066f30 0x7fb8140693e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.600+0000 7fb82ceb8700 1 --2- 172.21.15.89:0/1543106419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb82816b090 0x7fb82816a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.600+0000 7fb82ceb8700 1 -- 172.21.15.89:0/1543106419 >> 172.21.15.89:0/1543106419 conn(0x7fb8280ce430 msgr2=0x7fb82806b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:56.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.600+0000 7fb82ceb8700 1 -- 172.21.15.89:0/1543106419 shutdown_connections 2023-10-10T20:35:56.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.600+0000 7fb82ceb8700 1 -- 172.21.15.89:0/1543106419 wait complete. 2023-10-10T20:35:56.614 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:35:56.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.730+0000 7f3d228a7700 1 Processor -- start 2023-10-10T20:35:56.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.730+0000 7f3d228a7700 1 -- start start 2023-10-10T20:35:56.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.731+0000 7f3d228a7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d1c159890 0x7f3d1c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.731+0000 7f3d228a7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3d1c069560 0x7f3d1c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.731+0000 7f3d228a7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d1c15a160 con 0x7f3d1c159c60 2023-10-10T20:35:56.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.731+0000 7f3d228a7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1c069f20 con 0x7f3d1c069560 2023-10-10T20:35:56.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.731+0000 7f3d228a7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c0d13a0 con 0x7f3d1c159890 2023-10-10T20:35:56.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.731+0000 7f3d20e44700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3d1c069560 0x7f3d1c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.731+0000 7f3d1b7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3d1c159c60 0x7f3d1c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52748/0 (socket says 172.21.15.89:52748) 2023-10-10T20:35:56.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.731+0000 7f3d1b7fe700 1 -- 172.21.15.89:0/244161045 learned_addr learned my addr 172.21.15.89:0/244161045 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:56.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.731+0000 7f3d1bfff700 1 --2- 172.21.15.89:0/244161045 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d1c159890 0x7f3d1c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.731+0000 7f3d20e44700 1 -- 172.21.15.89:0/244161045 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d1c159890 msgr2=0x7f3d1c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:56.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.731+0000 7f3d20e44700 1 --2- 172.21.15.89:0/244161045 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d1c159890 0x7f3d1c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.732+0000 7f3d20e44700 1 -- 172.21.15.89:0/244161045 >> v1:172.21.15.89:6789/0 conn(0x7f3d1c159c60 legacy=0x7f3d1c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:56.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.731+0000 7f3d1affd700 1 -- 172.21.15.89:0/244161045 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832486975 0 0) 0x7f3d1c15a160 con 0x7f3d1c159c60 2023-10-10T20:35:56.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.732+0000 7f3d20e44700 1 -- 172.21.15.89:0/244161045 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1c15e140 con 0x7f3d1c069560 2023-10-10T20:35:56.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.732+0000 7f3d20e44700 1 --2- 172.21.15.89:0/244161045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d1c069560 0x7f3d1c0699e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3d14009e60 tx=0x7f3d14009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0986edd0cceebcf server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:56.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.732+0000 7f3d1affd700 1 -- 172.21.15.89:0/244161045 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1400e070 con 0x7f3d1c069560 2023-10-10T20:35:56.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.732+0000 7f3d1affd700 1 -- 172.21.15.89:0/244161045 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1400bc30 con 0x7f3d1c069560 2023-10-10T20:35:56.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.732+0000 7f3d1affd700 1 -- 172.21.15.89:0/244161045 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14012430 con 0x7f3d1c069560 2023-10-10T20:35:56.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.732+0000 7f3d228a7700 1 -- 172.21.15.89:0/244161045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d1c069560 msgr2=0x7f3d1c0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:56.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.732+0000 7f3d228a7700 1 --2- 172.21.15.89:0/244161045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d1c069560 0x7f3d1c0699e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3d14009e60 tx=0x7f3d14009780 comp rx=0 tx=0).stop 2023-10-10T20:35:56.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.732+0000 7f3d228a7700 1 -- 172.21.15.89:0/244161045 shutdown_connections 2023-10-10T20:35:56.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.732+0000 7f3d228a7700 1 --2- 172.21.15.89:0/244161045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d1c069560 0x7f3d1c0699e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.732+0000 7f3d228a7700 1 --2- 172.21.15.89:0/244161045 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d1c159890 0x7f3d1c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.732+0000 7f3d228a7700 1 -- 172.21.15.89:0/244161045 >> 172.21.15.89:0/244161045 conn(0x7f3d1c068e20 msgr2=0x7f3d1c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:56.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.732+0000 7f3d228a7700 1 -- 172.21.15.89:0/244161045 shutdown_connections 2023-10-10T20:35:56.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.732+0000 7f3d228a7700 1 -- 172.21.15.89:0/244161045 wait complete. 2023-10-10T20:35:56.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.734+0000 7f3d228a7700 1 Processor -- start 2023-10-10T20:35:56.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.734+0000 7f3d228a7700 1 -- start start 2023-10-10T20:35:56.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.734+0000 7f3d228a7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d1c069560 0x7f3d1c143df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.734+0000 7f3d228a7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d1c159890 0x7f3d1c0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.734+0000 7f3d228a7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d1c159c60 0x7f3d1c16d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.734+0000 7f3d228a7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c0b3e80 con 0x7f3d1c069560 2023-10-10T20:35:56.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.734+0000 7f3d228a7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c0cda30 con 0x7f3d1c159890 2023-10-10T20:35:56.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.734+0000 7f3d228a7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c16e0c0 con 0x7f3d1c159c60 2023-10-10T20:35:56.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.735+0000 7f3d1bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d1c069560 0x7f3d1c143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.735+0000 7f3d1bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d1c069560 0x7f3d1c143df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60600/0 (socket says 172.21.15.89:60600) 2023-10-10T20:35:56.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.735+0000 7f3d1bfff700 1 -- 172.21.15.89:0/2736437720 learned_addr learned my addr 172.21.15.89:0/2736437720 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:56.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.735+0000 7f3d1b7fe700 1 --2- 172.21.15.89:0/2736437720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d1c159890 0x7f3d1c0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.735+0000 7f3d20e44700 1 --2- 172.21.15.89:0/2736437720 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d1c159c60 0x7f3d1c16d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.735+0000 7f3d1bfff700 1 -- 172.21.15.89:0/2736437720 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d1c159c60 msgr2=0x7f3d1c16d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:56.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.735+0000 7f3d1bfff700 1 --2- 172.21.15.89:0/2736437720 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d1c159c60 0x7f3d1c16d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.735+0000 7f3d1bfff700 1 -- 172.21.15.89:0/2736437720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d1c159890 msgr2=0x7f3d1c0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.735+0000 7f3d1bfff700 1 --2- 172.21.15.89:0/2736437720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d1c159890 0x7f3d1c0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.735+0000 7f3d1bfff700 1 -- 172.21.15.89:0/2736437720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d14009a20 con 0x7f3d1c069560 2023-10-10T20:35:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.735+0000 7f3d20e44700 1 --2- 172.21.15.89:0/2736437720 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d1c159c60 0x7f3d1c16d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:35:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.735+0000 7f3d1bfff700 1 --2- 172.21.15.89:0/2736437720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d1c069560 0x7f3d1c143df0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00d8c0 tx=0x7f3d0c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.735+0000 7f3d03fff700 1 -- 172.21.15.89:0/2736437720 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c00de60 con 0x7f3d1c069560 2023-10-10T20:35:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.735+0000 7f3d03fff700 1 -- 172.21.15.89:0/2736437720 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0c00b780 con 0x7f3d1c069560 2023-10-10T20:35:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.735+0000 7f3d228a7700 1 -- 172.21.15.89:0/2736437720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1c0c8020 con 0x7f3d1c069560 2023-10-10T20:35:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.735+0000 7f3d03fff700 1 -- 172.21.15.89:0/2736437720 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c00bb20 con 0x7f3d1c069560 2023-10-10T20:35:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.735+0000 7f3d228a7700 1 -- 172.21.15.89:0/2736437720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1c0c79d0 con 0x7f3d1c069560 2023-10-10T20:35:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.736+0000 7f3d228a7700 1 -- 172.21.15.89:0/2736437720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1c04c3c0 con 0x7f3d1c069560 2023-10-10T20:35:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.736+0000 7f3d03fff700 1 -- 172.21.15.89:0/2736437720 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d0c01e460 con 0x7f3d1c069560 2023-10-10T20:35:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.737+0000 7f3d03fff700 1 --2- 172.21.15.89:0/2736437720 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3d08066f30 0x7f3d080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.737+0000 7f3d03fff700 1 -- 172.21.15.89:0/2736437720 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3d0c084bd0 con 0x7f3d1c069560 2023-10-10T20:35:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.737+0000 7f3d1b7fe700 1 --2- 172.21.15.89:0/2736437720 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3d08066f30 0x7f3d080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.737+0000 7f3d1b7fe700 1 --2- 172.21.15.89:0/2736437720 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3d08066f30 0x7f3d080693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c158050 tx=0x7f3d10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.741+0000 7f3d03fff700 1 -- 172.21.15.89:0/2736437720 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d0c051e40 con 0x7f3d1c069560 2023-10-10T20:35:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:56.971+0000 7f3d228a7700 1 -- 172.21.15.89:0/2736437720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3d1c0d1730 con 0x7f3d1c069560 2023-10-10T20:35:57.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:57.961+0000 7f3d03fff700 1 -- 172.21.15.89:0/2736437720 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v28) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3d0c055460 con 0x7f3d1c069560 2023-10-10T20:35:57.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:57.963+0000 7f3d228a7700 1 -- 172.21.15.89:0/2736437720 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3d08066f30 msgr2=0x7f3d080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:57.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:57.963+0000 7f3d228a7700 1 --2- 172.21.15.89:0/2736437720 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3d08066f30 0x7f3d080693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c158050 tx=0x7f3d10009040 comp rx=0 tx=0).stop 2023-10-10T20:35:57.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:57.963+0000 7f3d228a7700 1 -- 172.21.15.89:0/2736437720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d1c069560 msgr2=0x7f3d1c143df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:57.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:57.963+0000 7f3d228a7700 1 --2- 172.21.15.89:0/2736437720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d1c069560 0x7f3d1c143df0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00d8c0 tx=0x7f3d0c008040 comp rx=0 tx=0).stop 2023-10-10T20:35:57.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:57.964+0000 7f3d228a7700 1 -- 172.21.15.89:0/2736437720 shutdown_connections 2023-10-10T20:35:57.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:57.964+0000 7f3d228a7700 1 --2- 172.21.15.89:0/2736437720 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d1c159c60 0x7f3d1c16d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:57.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:57.964+0000 7f3d228a7700 1 --2- 172.21.15.89:0/2736437720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d1c069560 0x7f3d1c143df0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:57.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:57.964+0000 7f3d228a7700 1 --2- 172.21.15.89:0/2736437720 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3d08066f30 0x7f3d080693e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:57.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:57.964+0000 7f3d228a7700 1 --2- 172.21.15.89:0/2736437720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d1c159890 0x7f3d1c0b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:57.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:57.964+0000 7f3d228a7700 1 -- 172.21.15.89:0/2736437720 >> 172.21.15.89:0/2736437720 conn(0x7f3d1c068e20 msgr2=0x7f3d1c0b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:57.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:57.964+0000 7f3d228a7700 1 -- 172.21.15.89:0/2736437720 shutdown_connections 2023-10-10T20:35:57.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:57.964+0000 7f3d228a7700 1 -- 172.21.15.89:0/2736437720 wait complete. 2023-10-10T20:35:57.979 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:35:58.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.097+0000 7fc2e2086700 1 Processor -- start 2023-10-10T20:35:58.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.097+0000 7fc2e2086700 1 -- start start 2023-10-10T20:35:58.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.098+0000 7fc2e2086700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc2dc159c60 0x7fc2dc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:58.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.098+0000 7fc2e2086700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc2dc15db10 0x7fc2dc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:58.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.098+0000 7fc2e2086700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2dc15a160 con 0x7fc2dc159890 2023-10-10T20:35:58.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.098+0000 7fc2e2086700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2dc0d13a0 con 0x7fc2dc15db10 2023-10-10T20:35:58.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.098+0000 7fc2e2086700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc15e140 con 0x7fc2dc159c60 2023-10-10T20:35:58.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.098+0000 7fc2dbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc2dc15db10 0x7fc2dc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:58.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.098+0000 7fc2dbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc2dc15db10 0x7fc2dc069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60606/0 (socket says 172.21.15.89:60606) 2023-10-10T20:35:58.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.098+0000 7fc2dbfff700 1 -- 172.21.15.89:0/2274937684 learned_addr learned my addr 172.21.15.89:0/2274937684 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:58.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.098+0000 7fc2daffd700 1 --2- 172.21.15.89:0/2274937684 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc2dc159c60 0x7fc2dc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:58.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.098+0000 7fc2dbfff700 1 -- 172.21.15.89:0/2274937684 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc2dc159c60 msgr2=0x7fc2dc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:58.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.098+0000 7fc2dbfff700 1 --2- 172.21.15.89:0/2274937684 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc2dc159c60 0x7fc2dc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:58.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.099+0000 7fc2dbfff700 1 -- 172.21.15.89:0/2274937684 >> v1:172.21.15.111:6789/0 conn(0x7fc2dc159890 legacy=0x7fc2dc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:58.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.099+0000 7fc2dbfff700 1 -- 172.21.15.89:0/2274937684 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc15a3d0 con 0x7fc2dc15db10 2023-10-10T20:35:58.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.099+0000 7fc2dbfff700 1 --2- 172.21.15.89:0/2274937684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2dc15db10 0x7fc2dc069560 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40099b0 tx=0x7fc2d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27987f7b7b51b81d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:58.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.099+0000 7fc2da7fc700 1 -- 172.21.15.89:0/2274937684 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d400b860 con 0x7fc2dc15db10 2023-10-10T20:35:58.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.099+0000 7fc2da7fc700 1 -- 172.21.15.89:0/2274937684 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d400b9c0 con 0x7fc2dc15db10 2023-10-10T20:35:58.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.099+0000 7fc2da7fc700 1 -- 172.21.15.89:0/2274937684 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d400bd60 con 0x7fc2dc15db10 2023-10-10T20:35:58.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.099+0000 7fc2e2086700 1 -- 172.21.15.89:0/2274937684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2dc15db10 msgr2=0x7fc2dc069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:58.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.099+0000 7fc2e2086700 1 --2- 172.21.15.89:0/2274937684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2dc15db10 0x7fc2dc069560 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40099b0 tx=0x7fc2d4009250 comp rx=0 tx=0).stop 2023-10-10T20:35:58.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.099+0000 7fc2e2086700 1 -- 172.21.15.89:0/2274937684 shutdown_connections 2023-10-10T20:35:58.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.099+0000 7fc2e2086700 1 --2- 172.21.15.89:0/2274937684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2dc15db10 0x7fc2dc069560 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:58.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.099+0000 7fc2e2086700 1 --2- 172.21.15.89:0/2274937684 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc2dc159c60 0x7fc2dc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:58.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.099+0000 7fc2e2086700 1 -- 172.21.15.89:0/2274937684 >> 172.21.15.89:0/2274937684 conn(0x7fc2dc068e20 msgr2=0x7fc2dc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:58.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.099+0000 7fc2e2086700 1 -- 172.21.15.89:0/2274937684 shutdown_connections 2023-10-10T20:35:58.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.099+0000 7fc2e2086700 1 -- 172.21.15.89:0/2274937684 wait complete. 2023-10-10T20:35:58.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.101+0000 7fc2e2086700 1 Processor -- start 2023-10-10T20:35:58.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.101+0000 7fc2e2086700 1 -- start start 2023-10-10T20:35:58.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.101+0000 7fc2e2086700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2dc159890 0x7fc2dc14cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:58.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.101+0000 7fc2e2086700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc2dc159c60 0x7fc2dc153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:58.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.101+0000 7fc2e2086700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2dc15db10 0x7fc2dc154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:58.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.101+0000 7fc2e2086700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc0b1b00 con 0x7fc2dc159890 2023-10-10T20:35:58.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.101+0000 7fc2e2086700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc0b1de0 con 0x7fc2dc15db10 2023-10-10T20:35:58.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.101+0000 7fc2e2086700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc0b20c0 con 0x7fc2dc159c60 2023-10-10T20:35:58.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.101+0000 7fc2db7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2dc159890 0x7fc2dc14cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:58.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.101+0000 7fc2db7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2dc159890 0x7fc2dc14cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60612/0 (socket says 172.21.15.89:60612) 2023-10-10T20:35:58.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.101+0000 7fc2db7fe700 1 -- 172.21.15.89:0/2917062016 learned_addr learned my addr 172.21.15.89:0/2917062016 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:58.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.102+0000 7fc2dbfff700 1 --2- 172.21.15.89:0/2917062016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2dc15db10 0x7fc2dc154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:58.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.102+0000 7fc2daffd700 1 --2- 172.21.15.89:0/2917062016 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc2dc159c60 0x7fc2dc153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:58.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.102+0000 7fc2db7fe700 1 -- 172.21.15.89:0/2917062016 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc2dc159c60 msgr2=0x7fc2dc153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:58.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.102+0000 7fc2db7fe700 1 --2- 172.21.15.89:0/2917062016 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc2dc159c60 0x7fc2dc153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:58.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.102+0000 7fc2db7fe700 1 -- 172.21.15.89:0/2917062016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2dc15db10 msgr2=0x7fc2dc154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:58.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.102+0000 7fc2db7fe700 1 --2- 172.21.15.89:0/2917062016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2dc15db10 0x7fc2dc154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:58.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.102+0000 7fc2db7fe700 1 -- 172.21.15.89:0/2917062016 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d40094f0 con 0x7fc2dc159890 2023-10-10T20:35:58.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.102+0000 7fc2db7fe700 1 --2- 172.21.15.89:0/2917062016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2dc159890 0x7fc2dc14cc50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00cb10 tx=0x7fc2cc0017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.102+0000 7fc2c37fe700 1 -- 172.21.15.89:0/2917062016 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc00a760 con 0x7fc2dc159890 2023-10-10T20:35:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.102+0000 7fc2c37fe700 1 -- 172.21.15.89:0/2917062016 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2cc00a8e0 con 0x7fc2dc159890 2023-10-10T20:35:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.102+0000 7fc2e2086700 1 -- 172.21.15.89:0/2917062016 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2dc0b23a0 con 0x7fc2dc159890 2023-10-10T20:35:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.102+0000 7fc2c37fe700 1 -- 172.21.15.89:0/2917062016 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc00aca0 con 0x7fc2dc159890 2023-10-10T20:35:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.102+0000 7fc2e2086700 1 -- 172.21.15.89:0/2917062016 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2dc0b1050 con 0x7fc2dc159890 2023-10-10T20:35:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.103+0000 7fc2e2086700 1 -- 172.21.15.89:0/2917062016 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2dc04c3c0 con 0x7fc2dc159890 2023-10-10T20:35:58.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.103+0000 7fc2c37fe700 1 -- 172.21.15.89:0/2917062016 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc2cc003030 con 0x7fc2dc159890 2023-10-10T20:35:58.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.104+0000 7fc2c37fe700 1 --2- 172.21.15.89:0/2917062016 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc2c8066f30 0x7fc2c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:58.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.104+0000 7fc2daffd700 1 --2- 172.21.15.89:0/2917062016 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc2c8066f30 0x7fc2c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:58.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.104+0000 7fc2c37fe700 1 -- 172.21.15.89:0/2917062016 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc2cc0845d0 con 0x7fc2dc159890 2023-10-10T20:35:58.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.104+0000 7fc2daffd700 1 --2- 172.21.15.89:0/2917062016 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc2c8066f30 0x7fc2c80693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc063ef0 tx=0x7fc2d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:58.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.108+0000 7fc2c37fe700 1 -- 172.21.15.89:0/2917062016 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2cc051840 con 0x7fc2dc159890 2023-10-10T20:35:58.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.335+0000 7fc2e2086700 1 -- 172.21.15.89:0/2917062016 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc2dc156e90 con 0x7fc2dc159890 2023-10-10T20:35:58.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.966+0000 7fc2c37fe700 1 -- 172.21.15.89:0/2917062016 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v29) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc2cc054e60 con 0x7fc2dc159890 2023-10-10T20:35:58.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.968+0000 7fc2e2086700 1 -- 172.21.15.89:0/2917062016 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc2c8066f30 msgr2=0x7fc2c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:58.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.968+0000 7fc2e2086700 1 --2- 172.21.15.89:0/2917062016 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc2c8066f30 0x7fc2c80693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc063ef0 tx=0x7fc2d0008040 comp rx=0 tx=0).stop 2023-10-10T20:35:58.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.968+0000 7fc2e2086700 1 -- 172.21.15.89:0/2917062016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2dc159890 msgr2=0x7fc2dc14cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:58.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.968+0000 7fc2e2086700 1 --2- 172.21.15.89:0/2917062016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2dc159890 0x7fc2dc14cc50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00cb10 tx=0x7fc2cc0017b0 comp rx=0 tx=0).stop 2023-10-10T20:35:58.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.968+0000 7fc2e2086700 1 -- 172.21.15.89:0/2917062016 shutdown_connections 2023-10-10T20:35:58.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.968+0000 7fc2e2086700 1 --2- 172.21.15.89:0/2917062016 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc2c8066f30 0x7fc2c80693e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:58.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.968+0000 7fc2e2086700 1 --2- 172.21.15.89:0/2917062016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2dc15db10 0x7fc2dc154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:58.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.968+0000 7fc2e2086700 1 --2- 172.21.15.89:0/2917062016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2dc159890 0x7fc2dc14cc50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:58.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.968+0000 7fc2e2086700 1 --2- 172.21.15.89:0/2917062016 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc2dc159c60 0x7fc2dc153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:58.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.968+0000 7fc2e2086700 1 -- 172.21.15.89:0/2917062016 >> 172.21.15.89:0/2917062016 conn(0x7fc2dc068e20 msgr2=0x7fc2dc069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:58.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.968+0000 7fc2e2086700 1 -- 172.21.15.89:0/2917062016 shutdown_connections 2023-10-10T20:35:58.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:58.968+0000 7fc2e2086700 1 -- 172.21.15.89:0/2917062016 wait complete. 2023-10-10T20:35:58.983 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:35:59.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.094+0000 7f02082da700 1 Processor -- start 2023-10-10T20:35:59.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.094+0000 7f02082da700 1 -- start start 2023-10-10T20:35:59.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.095+0000 7f02082da700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0200159c60 0x7f020015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:59.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.095+0000 7f02082da700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f020015db10 0x7f0200069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:59.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.095+0000 7f02082da700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020015a160 con 0x7f0200159890 2023-10-10T20:35:59.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.095+0000 7f02082da700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020015df50 con 0x7f0200159c60 2023-10-10T20:35:59.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.095+0000 7f02082da700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02000d13a0 con 0x7f020015db10 2023-10-10T20:35:59.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.096+0000 7f0205875700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0200159c60 0x7f020015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:59.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.096+0000 7f0206076700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0200159890 0x7f020015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58762/0 (socket says 172.21.15.89:58762) 2023-10-10T20:35:59.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.096+0000 7f0206076700 1 -- 172.21.15.89:0/2403927586 learned_addr learned my addr 172.21.15.89:0/2403927586 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:59.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.096+0000 7f0205875700 1 -- 172.21.15.89:0/2403927586 >> v2:172.21.15.111:3300/0 conn(0x7f020015db10 msgr2=0x7f0200069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:59.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.096+0000 7f0205875700 1 --2- 172.21.15.89:0/2403927586 >> v2:172.21.15.111:3300/0 conn(0x7f020015db10 0x7f0200069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.096+0000 7f0205875700 1 -- 172.21.15.89:0/2403927586 >> v1:172.21.15.89:6789/0 conn(0x7f0200159890 legacy=0x7f020015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:59.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.096+0000 7f0205875700 1 -- 172.21.15.89:0/2403927586 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020015e140 con 0x7f0200159c60 2023-10-10T20:35:59.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.096+0000 7f0205074700 1 -- 172.21.15.89:0/2403927586 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972418770 0 0) 0x7f020015a160 con 0x7f0200159890 2023-10-10T20:35:59.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.096+0000 7f0205875700 1 --2- 172.21.15.89:0/2403927586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0200159c60 0x7f020015d5d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f020006a890 tx=0x7f01f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9020e6c8ff9df07c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:59.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.096+0000 7f0205074700 1 -- 172.21.15.89:0/2403927586 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f000b6b0 con 0x7f0200159c60 2023-10-10T20:35:59.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.096+0000 7f0205074700 1 -- 172.21.15.89:0/2403927586 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f000b810 con 0x7f0200159c60 2023-10-10T20:35:59.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.096+0000 7f0205074700 1 -- 172.21.15.89:0/2403927586 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f000bbb0 con 0x7f0200159c60 2023-10-10T20:35:59.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.097+0000 7f02082da700 1 -- 172.21.15.89:0/2403927586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0200159c60 msgr2=0x7f020015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:59.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.097+0000 7f02082da700 1 --2- 172.21.15.89:0/2403927586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0200159c60 0x7f020015d5d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f020006a890 tx=0x7f01f00091a0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.097+0000 7f02082da700 1 -- 172.21.15.89:0/2403927586 shutdown_connections 2023-10-10T20:35:59.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.097+0000 7f02082da700 1 --2- 172.21.15.89:0/2403927586 >> v2:172.21.15.111:3300/0 conn(0x7f020015db10 0x7f0200069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.097+0000 7f02082da700 1 --2- 172.21.15.89:0/2403927586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0200159c60 0x7f020015d5d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.097+0000 7f02082da700 1 -- 172.21.15.89:0/2403927586 >> 172.21.15.89:0/2403927586 conn(0x7f0200068e20 msgr2=0x7f02000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:59.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.097+0000 7f02082da700 1 -- 172.21.15.89:0/2403927586 shutdown_connections 2023-10-10T20:35:59.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.097+0000 7f02082da700 1 -- 172.21.15.89:0/2403927586 wait complete. 2023-10-10T20:35:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.098+0000 7f02082da700 1 Processor -- start 2023-10-10T20:35:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.098+0000 7f02082da700 1 -- start start 2023-10-10T20:35:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.098+0000 7f02082da700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0200159890 0x7f02000cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:59.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.098+0000 7f02082da700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0200159c60 0x7f02000b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:59.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.098+0000 7f02082da700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f020015db10 0x7f020016d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:59.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.098+0000 7f02082da700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02000b20c0 con 0x7f0200159c60 2023-10-10T20:35:59.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.098+0000 7f02082da700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02000b3590 con 0x7f0200159890 2023-10-10T20:35:59.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.098+0000 7f02082da700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0200150aa0 con 0x7f020015db10 2023-10-10T20:35:59.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.099+0000 7f0205875700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0200159c60 0x7f02000b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:59.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.099+0000 7f0205875700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0200159c60 0x7f02000b1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40954/0 (socket says 172.21.15.89:40954) 2023-10-10T20:35:59.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.099+0000 7f0205875700 1 -- 172.21.15.89:0/4050674892 learned_addr learned my addr 172.21.15.89:0/4050674892 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:59.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.099+0000 7f0206076700 1 --2- 172.21.15.89:0/4050674892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0200159890 0x7f02000cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:59.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.099+0000 7f0206877700 1 --2- 172.21.15.89:0/4050674892 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f020015db10 0x7f020016d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.099+0000 7f0205875700 1 -- 172.21.15.89:0/4050674892 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f020015db10 msgr2=0x7f020016d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.099+0000 7f0205875700 1 --2- 172.21.15.89:0/4050674892 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f020015db10 0x7f020016d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.099+0000 7f0205875700 1 -- 172.21.15.89:0/4050674892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0200159890 msgr2=0x7f02000cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.099+0000 7f0205875700 1 --2- 172.21.15.89:0/4050674892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0200159890 0x7f02000cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.099+0000 7f0205875700 1 -- 172.21.15.89:0/4050674892 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f0009420 con 0x7f0200159c60 2023-10-10T20:35:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.099+0000 7f0205875700 1 --2- 172.21.15.89:0/4050674892 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0200159c60 0x7f02000b1410 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f01f000b0e0 tx=0x7f01f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.099+0000 7f01f67fc700 1 -- 172.21.15.89:0/4050674892 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f000bb90 con 0x7f0200159c60 2023-10-10T20:35:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.099+0000 7f01f67fc700 1 -- 172.21.15.89:0/4050674892 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f0011420 con 0x7f0200159c60 2023-10-10T20:35:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.099+0000 7f02082da700 1 -- 172.21.15.89:0/4050674892 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02000b1b00 con 0x7f0200159c60 2023-10-10T20:35:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.099+0000 7f01f67fc700 1 -- 172.21.15.89:0/4050674892 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f00117c0 con 0x7f0200159c60 2023-10-10T20:35:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.099+0000 7f02082da700 1 -- 172.21.15.89:0/4050674892 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0200150730 con 0x7f0200159c60 2023-10-10T20:35:59.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.100+0000 7f02082da700 1 -- 172.21.15.89:0/4050674892 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0200150590 con 0x7f0200159c60 2023-10-10T20:35:59.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.100+0000 7f01f67fc700 1 -- 172.21.15.89:0/4050674892 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01f0011b30 con 0x7f0200159c60 2023-10-10T20:35:59.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.101+0000 7f01f67fc700 1 --2- 172.21.15.89:0/4050674892 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f01e4066f30 0x7f01e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:59.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.101+0000 7f01f67fc700 1 -- 172.21.15.89:0/4050674892 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f01f0085f50 con 0x7f0200159c60 2023-10-10T20:35:59.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.101+0000 7f0206076700 1 --2- 172.21.15.89:0/4050674892 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f01e4066f30 0x7f01e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:59.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.101+0000 7f0206076700 1 --2- 172.21.15.89:0/4050674892 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f01e4066f30 0x7f01e40693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f01fc002230 tx=0x7f01fc001a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:59.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.105+0000 7f01f67fc700 1 -- 172.21.15.89:0/4050674892 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0200150590 con 0x7f0200159c60 2023-10-10T20:35:59.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.329+0000 7f02082da700 1 -- 172.21.15.89:0/4050674892 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0200150590 con 0x7f0200159c60 2023-10-10T20:35:59.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.330+0000 7f01f67fc700 1 -- 172.21.15.89:0/4050674892 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f0200150590 con 0x7f0200159c60 2023-10-10T20:35:59.333 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:59.333 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4922,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1464385125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1464385125},{"type":"v1","addr":"172.21.15.111:6834","nonce":1464385125}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":4925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/689169706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":689169706},{"type":"v1","addr":"172.21.15.111:6835","nonce":689169706}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/418769006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":418769006},{"type":"v1","addr":"172.21.15.89:6836","nonce":418769006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:35:55.314480+0000","modified":"2023-10-10T20:35:58.961430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5340":{"gid":5340,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/3245027879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":3245027879},{"type":"v1","addr":"172.21.15.89:6837","nonce":3245027879}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-10T20:35:59.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.332+0000 7f02082da700 1 -- 172.21.15.89:0/4050674892 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f01e4066f30 msgr2=0x7f01e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:59.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.332+0000 7f02082da700 1 --2- 172.21.15.89:0/4050674892 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f01e4066f30 0x7f01e40693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f01fc002230 tx=0x7f01fc001a20 comp rx=0 tx=0).stop 2023-10-10T20:35:59.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.332+0000 7f02082da700 1 -- 172.21.15.89:0/4050674892 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0200159c60 msgr2=0x7f02000b1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:59.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.332+0000 7f02082da700 1 --2- 172.21.15.89:0/4050674892 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0200159c60 0x7f02000b1410 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f01f000b0e0 tx=0x7f01f0017000 comp rx=0 tx=0).stop 2023-10-10T20:35:59.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.332+0000 7f02082da700 1 -- 172.21.15.89:0/4050674892 shutdown_connections 2023-10-10T20:35:59.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.332+0000 7f02082da700 1 --2- 172.21.15.89:0/4050674892 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f020015db10 0x7f020016d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.332+0000 7f02082da700 1 --2- 172.21.15.89:0/4050674892 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f01e4066f30 0x7f01e40693e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.332+0000 7f02082da700 1 --2- 172.21.15.89:0/4050674892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0200159890 0x7f02000cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.332+0000 7f02082da700 1 --2- 172.21.15.89:0/4050674892 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0200159c60 0x7f02000b1410 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.332+0000 7f02082da700 1 -- 172.21.15.89:0/4050674892 >> 172.21.15.89:0/4050674892 conn(0x7f0200068e20 msgr2=0x7f0200153070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:59.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.332+0000 7f02082da700 1 -- 172.21.15.89:0/4050674892 shutdown_connections 2023-10-10T20:35:59.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.332+0000 7f02082da700 1 -- 172.21.15.89:0/4050674892 wait complete. 2023-10-10T20:35:59.337 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 29 2023-10-10T20:35:59.346 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:35:59.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.458+0000 7f3c8a293700 1 Processor -- start 2023-10-10T20:35:59.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.458+0000 7f3c8a293700 1 -- start start 2023-10-10T20:35:59.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.459+0000 7f3c8a293700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3c84159890 0x7f3c8415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:59.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.459+0000 7f3c8a293700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c84069560 0x7f3c840699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:59.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.459+0000 7f3c8a293700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c84069ee0 con 0x7f3c84159c60 2023-10-10T20:35:59.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.459+0000 7f3c8a293700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c8415a160 con 0x7f3c84159890 2023-10-10T20:35:59.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.459+0000 7f3c8a293700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c840d13a0 con 0x7f3c84069560 2023-10-10T20:35:59.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.459+0000 7f3c83fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3c84159890 0x7f3c8415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:59.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.459+0000 7f3c83fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3c84159890 0x7f3c8415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40964/0 (socket says 172.21.15.89:40964) 2023-10-10T20:35:59.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.459+0000 7f3c83fff700 1 -- 172.21.15.89:0/1725102391 learned_addr learned my addr 172.21.15.89:0/1725102391 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:59.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.459+0000 7f3c88830700 1 --2- 172.21.15.89:0/1725102391 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c84069560 0x7f3c840699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:59.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.460+0000 7f3c83fff700 1 -- 172.21.15.89:0/1725102391 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c84069560 msgr2=0x7f3c840699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:59.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.460+0000 7f3c83fff700 1 --2- 172.21.15.89:0/1725102391 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c84069560 0x7f3c840699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.460+0000 7f3c83fff700 1 -- 172.21.15.89:0/1725102391 >> v1:172.21.15.111:6789/0 conn(0x7f3c84159c60 legacy=0x7f3c8415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:59.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.460+0000 7f3c83fff700 1 -- 172.21.15.89:0/1725102391 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8415e140 con 0x7f3c84159890 2023-10-10T20:35:59.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.460+0000 7f3c88830700 1 --2- 172.21.15.89:0/1725102391 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c84069560 0x7f3c840699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:35:59.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.460+0000 7f3c83fff700 1 --2- 172.21.15.89:0/1725102391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c84159890 0x7f3c8415abd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3c740099b0 tx=0x7f3c74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a73abb9564fece8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:59.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.460+0000 7f3c82ffd700 1 -- 172.21.15.89:0/1725102391 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7400b860 con 0x7f3c84159890 2023-10-10T20:35:59.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.460+0000 7f3c82ffd700 1 -- 172.21.15.89:0/1725102391 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7400b9c0 con 0x7f3c84159890 2023-10-10T20:35:59.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.460+0000 7f3c82ffd700 1 -- 172.21.15.89:0/1725102391 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7400bd60 con 0x7f3c84159890 2023-10-10T20:35:59.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.460+0000 7f3c8a293700 1 -- 172.21.15.89:0/1725102391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c84159890 msgr2=0x7f3c8415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:59.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.460+0000 7f3c8a293700 1 --2- 172.21.15.89:0/1725102391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c84159890 0x7f3c8415abd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3c740099b0 tx=0x7f3c74009250 comp rx=0 tx=0).stop 2023-10-10T20:35:59.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.460+0000 7f3c8a293700 1 -- 172.21.15.89:0/1725102391 shutdown_connections 2023-10-10T20:35:59.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.460+0000 7f3c8a293700 1 --2- 172.21.15.89:0/1725102391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c84159890 0x7f3c8415abd0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.460+0000 7f3c8a293700 1 --2- 172.21.15.89:0/1725102391 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c84069560 0x7f3c840699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.460+0000 7f3c8a293700 1 -- 172.21.15.89:0/1725102391 >> 172.21.15.89:0/1725102391 conn(0x7f3c84068e20 msgr2=0x7f3c840d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:59.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.460+0000 7f3c8a293700 1 -- 172.21.15.89:0/1725102391 shutdown_connections 2023-10-10T20:35:59.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.460+0000 7f3c8a293700 1 -- 172.21.15.89:0/1725102391 wait complete. 2023-10-10T20:35:59.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.461+0000 7f3c8a293700 1 Processor -- start 2023-10-10T20:35:59.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.462+0000 7f3c8a293700 1 -- start start 2023-10-10T20:35:59.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.462+0000 7f3c8a293700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c84069560 0x7f3c841524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:59.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.462+0000 7f3c8a293700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c84159890 0x7f3c840cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:59.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.462+0000 7f3c8a293700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c84159c60 0x7f3c840b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:59.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.462+0000 7f3c8a293700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c840b3a20 con 0x7f3c84159890 2023-10-10T20:35:59.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.462+0000 7f3c8a293700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c840b3d00 con 0x7f3c84159c60 2023-10-10T20:35:59.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.462+0000 7f3c8a293700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c840b3fe0 con 0x7f3c84069560 2023-10-10T20:35:59.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.463+0000 7f3c837fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c84159890 0x7f3c840cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:59.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.463+0000 7f3c837fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c84159890 0x7f3c840cf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40978/0 (socket says 172.21.15.89:40978) 2023-10-10T20:35:59.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.463+0000 7f3c837fe700 1 -- 172.21.15.89:0/2234807421 learned_addr learned my addr 172.21.15.89:0/2234807421 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:59.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.463+0000 7f3c88830700 1 --2- 172.21.15.89:0/2234807421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c84159c60 0x7f3c840b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:59.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.463+0000 7f3c83fff700 1 --2- 172.21.15.89:0/2234807421 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c84069560 0x7f3c841524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:59.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.463+0000 7f3c837fe700 1 -- 172.21.15.89:0/2234807421 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c84069560 msgr2=0x7f3c841524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:59.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.463+0000 7f3c837fe700 1 --2- 172.21.15.89:0/2234807421 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c84069560 0x7f3c841524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.463+0000 7f3c837fe700 1 -- 172.21.15.89:0/2234807421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c84159c60 msgr2=0x7f3c840b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:59.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.463+0000 7f3c837fe700 1 --2- 172.21.15.89:0/2234807421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c84159c60 0x7f3c840b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.463+0000 7f3c837fe700 1 -- 172.21.15.89:0/2234807421 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c740094f0 con 0x7f3c84159890 2023-10-10T20:35:59.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.463+0000 7f3c837fe700 1 --2- 172.21.15.89:0/2234807421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c84159890 0x7f3c840cf7f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3c7800c590 tx=0x7f3c7800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:59.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.463+0000 7f3c6bfff700 1 -- 172.21.15.89:0/2234807421 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78011590 con 0x7f3c84159890 2023-10-10T20:35:59.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.463+0000 7f3c6bfff700 1 -- 172.21.15.89:0/2234807421 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c78011710 con 0x7f3c84159890 2023-10-10T20:35:59.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.463+0000 7f3c8a293700 1 -- 172.21.15.89:0/2234807421 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c840b42c0 con 0x7f3c84159890 2023-10-10T20:35:59.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.463+0000 7f3c6bfff700 1 -- 172.21.15.89:0/2234807421 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78011ad0 con 0x7f3c84159890 2023-10-10T20:35:59.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.463+0000 7f3c8a293700 1 -- 172.21.15.89:0/2234807421 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c841529f0 con 0x7f3c84159890 2023-10-10T20:35:59.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.464+0000 7f3c8a293700 1 -- 172.21.15.89:0/2234807421 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c84169b90 con 0x7f3c84159890 2023-10-10T20:35:59.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.465+0000 7f3c6bfff700 1 -- 172.21.15.89:0/2234807421 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c7801b3b0 con 0x7f3c84159890 2023-10-10T20:35:59.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.465+0000 7f3c6bfff700 1 --2- 172.21.15.89:0/2234807421 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3c70066f30 0x7f3c700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:59.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.465+0000 7f3c6bfff700 1 -- 172.21.15.89:0/2234807421 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3c780839c0 con 0x7f3c84159890 2023-10-10T20:35:59.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.465+0000 7f3c83fff700 1 --2- 172.21.15.89:0/2234807421 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3c70066f30 0x7f3c700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:59.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.465+0000 7f3c83fff700 1 --2- 172.21.15.89:0/2234807421 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3c70066f30 0x7f3c700693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3c740099b0 tx=0x7f3c74017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:59.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.469+0000 7f3c6bfff700 1 -- 172.21.15.89:0/2234807421 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c84169b90 con 0x7f3c84159890 2023-10-10T20:35:59.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.654+0000 7f3c8a293700 1 -- 172.21.15.89:0/2234807421 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c84169b90 con 0x7f3c84159890 2023-10-10T20:35:59.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.654+0000 7f3c6bfff700 1 -- 172.21.15.89:0/2234807421 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f3c84169b90 con 0x7f3c84159890 2023-10-10T20:35:59.658 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:59.659 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":50,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:35:56.929187+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:35:53.658728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-10T20:35:54.272054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:35:59.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.656+0000 7f3c8a293700 1 -- 172.21.15.89:0/2234807421 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3c70066f30 msgr2=0x7f3c700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:59.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.656+0000 7f3c8a293700 1 --2- 172.21.15.89:0/2234807421 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3c70066f30 0x7f3c700693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3c740099b0 tx=0x7f3c74017000 comp rx=0 tx=0).stop 2023-10-10T20:35:59.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.656+0000 7f3c8a293700 1 -- 172.21.15.89:0/2234807421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c84159890 msgr2=0x7f3c840cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:59.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.656+0000 7f3c8a293700 1 --2- 172.21.15.89:0/2234807421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c84159890 0x7f3c840cf7f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3c7800c590 tx=0x7f3c7800c8a0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.656+0000 7f3c8a293700 1 -- 172.21.15.89:0/2234807421 shutdown_connections 2023-10-10T20:35:59.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.656+0000 7f3c8a293700 1 --2- 172.21.15.89:0/2234807421 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3c70066f30 0x7f3c700693e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.656+0000 7f3c8a293700 1 --2- 172.21.15.89:0/2234807421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c84159c60 0x7f3c840b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.656+0000 7f3c8a293700 1 --2- 172.21.15.89:0/2234807421 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c84069560 0x7f3c841524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.656+0000 7f3c8a293700 1 --2- 172.21.15.89:0/2234807421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c84159890 0x7f3c840cf7f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.656+0000 7f3c8a293700 1 -- 172.21.15.89:0/2234807421 >> 172.21.15.89:0/2234807421 conn(0x7f3c84068e20 msgr2=0x7f3c840b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:59.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.657+0000 7f3c8a293700 1 -- 172.21.15.89:0/2234807421 shutdown_connections 2023-10-10T20:35:59.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.657+0000 7f3c8a293700 1 -- 172.21.15.89:0/2234807421 wait complete. 2023-10-10T20:35:59.671 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:35:59.671 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:35:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.783+0000 7f5ef3fff700 1 Processor -- start 2023-10-10T20:35:59.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.783+0000 7f5ef3fff700 1 -- start start 2023-10-10T20:35:59.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.784+0000 7f5ef3fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5ef4159420 0x7f5ef40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:59.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.784+0000 7f5ef3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5ef415d230 0x7f5ef415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:59.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.784+0000 7f5ef3fff700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef40b2980 con 0x7f5ef41597f0 2023-10-10T20:35:59.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.784+0000 7f5ef3fff700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef415db70 con 0x7f5ef415d230 2023-10-10T20:35:59.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.784+0000 7f5ef3fff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef40b11d0 con 0x7f5ef4159420 2023-10-10T20:35:59.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.784+0000 7f5ef37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5ef415d230 0x7f5ef415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:59.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.784+0000 7f5eea7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5ef41597f0 0x7f5ef40b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58776/0 (socket says 172.21.15.89:58776) 2023-10-10T20:35:59.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.784+0000 7f5eea7fc700 1 -- 172.21.15.89:0/1599147414 learned_addr learned my addr 172.21.15.89:0/1599147414 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:35:59.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.784+0000 7f5ef37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5ef415d230 0x7f5ef415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40982/0 (socket says 172.21.15.89:40982) 2023-10-10T20:35:59.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.784+0000 7f5ef2ffd700 1 --2- 172.21.15.89:0/1599147414 >> v2:172.21.15.111:3300/0 conn(0x7f5ef4159420 0x7f5ef40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:59.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.784+0000 7f5ef37fe700 1 -- 172.21.15.89:0/1599147414 >> v2:172.21.15.111:3300/0 conn(0x7f5ef4159420 msgr2=0x7f5ef40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:35:59.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.784+0000 7f5ef37fe700 1 --2- 172.21.15.89:0/1599147414 >> v2:172.21.15.111:3300/0 conn(0x7f5ef4159420 0x7f5ef40b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.784+0000 7f5ef37fe700 1 -- 172.21.15.89:0/1599147414 >> v1:172.21.15.89:6789/0 conn(0x7f5ef41597f0 legacy=0x7f5ef40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:59.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.785+0000 7f5ef37fe700 1 -- 172.21.15.89:0/1599147414 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef40b14b0 con 0x7f5ef415d230 2023-10-10T20:35:59.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.784+0000 7f5ef27fc700 1 -- 172.21.15.89:0/1599147414 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 142558987 0 0) 0x7f5ef40b2980 con 0x7f5ef41597f0 2023-10-10T20:35:59.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.785+0000 7f5ef37fe700 1 --2- 172.21.15.89:0/1599147414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ef415d230 0x7f5ef415d630 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40b2f20 tx=0x7f5eec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3cc1ed54cf0502 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:59.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.785+0000 7f5ef27fc700 1 -- 172.21.15.89:0/1599147414 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec00e070 con 0x7f5ef415d230 2023-10-10T20:35:59.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.785+0000 7f5ef27fc700 1 -- 172.21.15.89:0/1599147414 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eec00bb60 con 0x7f5ef415d230 2023-10-10T20:35:59.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.785+0000 7f5ef27fc700 1 -- 172.21.15.89:0/1599147414 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec012430 con 0x7f5ef415d230 2023-10-10T20:35:59.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.785+0000 7f5ef3fff700 1 -- 172.21.15.89:0/1599147414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ef415d230 msgr2=0x7f5ef415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:59.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.785+0000 7f5ef3fff700 1 --2- 172.21.15.89:0/1599147414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ef415d230 0x7f5ef415d630 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40b2f20 tx=0x7f5eec009780 comp rx=0 tx=0).stop 2023-10-10T20:35:59.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.785+0000 7f5ef3fff700 1 -- 172.21.15.89:0/1599147414 shutdown_connections 2023-10-10T20:35:59.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.785+0000 7f5ef3fff700 1 --2- 172.21.15.89:0/1599147414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ef415d230 0x7f5ef415d630 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.785+0000 7f5ef3fff700 1 --2- 172.21.15.89:0/1599147414 >> v2:172.21.15.111:3300/0 conn(0x7f5ef4159420 0x7f5ef40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.785+0000 7f5ef3fff700 1 -- 172.21.15.89:0/1599147414 >> 172.21.15.89:0/1599147414 conn(0x7f5ef40d0660 msgr2=0x7f5ef40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:59.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.785+0000 7f5ef3fff700 1 -- 172.21.15.89:0/1599147414 shutdown_connections 2023-10-10T20:35:59.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.785+0000 7f5ef3fff700 1 -- 172.21.15.89:0/1599147414 wait complete. 2023-10-10T20:35:59.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.787+0000 7f5ef3fff700 1 Processor -- start 2023-10-10T20:35:59.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.787+0000 7f5ef3fff700 1 -- start start 2023-10-10T20:35:59.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.787+0000 7f5ef3fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ef4159420 0x7f5ef4159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:59.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.787+0000 7f5ef3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ef41597f0 0x7f5ef40c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:59.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.787+0000 7f5ef3fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5ef415d230 0x7f5ef4154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:59.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.787+0000 7f5ef3fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef415cc00 con 0x7f5ef41597f0 2023-10-10T20:35:59.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.787+0000 7f5ef3fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef415aed0 con 0x7f5ef4159420 2023-10-10T20:35:59.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.787+0000 7f5ef3fff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef415a2c0 con 0x7f5ef415d230 2023-10-10T20:35:59.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.787+0000 7f5eea7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ef41597f0 0x7f5ef40c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:59.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.787+0000 7f5eea7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ef41597f0 0x7f5ef40c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40994/0 (socket says 172.21.15.89:40994) 2023-10-10T20:35:59.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.787+0000 7f5eea7fc700 1 -- 172.21.15.89:0/3037292820 learned_addr learned my addr 172.21.15.89:0/3037292820 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:35:59.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.788+0000 7f5eea7fc700 1 -- 172.21.15.89:0/3037292820 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5ef415d230 msgr2=0x7f5ef4154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:59.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.788+0000 7f5eea7fc700 1 --2- 172.21.15.89:0/3037292820 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5ef415d230 0x7f5ef4154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.788+0000 7f5eea7fc700 1 -- 172.21.15.89:0/3037292820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ef4159420 msgr2=0x7f5ef4159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:59.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.788+0000 7f5ef2ffd700 1 --2- 172.21.15.89:0/3037292820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ef4159420 0x7f5ef4159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:59.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.788+0000 7f5eea7fc700 1 --2- 172.21.15.89:0/3037292820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ef4159420 0x7f5ef4159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.788+0000 7f5eea7fc700 1 -- 172.21.15.89:0/3037292820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec009a20 con 0x7f5ef41597f0 2023-10-10T20:35:59.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.788+0000 7f5eea7fc700 1 --2- 172.21.15.89:0/3037292820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ef41597f0 0x7f5ef40c5db0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00c840 tx=0x7f5edc007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:59.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.788+0000 7f5eeb7fe700 1 -- 172.21.15.89:0/3037292820 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc005c00 con 0x7f5ef41597f0 2023-10-10T20:35:59.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.788+0000 7f5eeb7fe700 1 -- 172.21.15.89:0/3037292820 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5edc005d60 con 0x7f5ef41597f0 2023-10-10T20:35:59.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.788+0000 7f5ef3fff700 1 -- 172.21.15.89:0/3037292820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef415a5a0 con 0x7f5ef41597f0 2023-10-10T20:35:59.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.788+0000 7f5eeb7fe700 1 -- 172.21.15.89:0/3037292820 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc0072e0 con 0x7f5ef41597f0 2023-10-10T20:35:59.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.788+0000 7f5ef3fff700 1 -- 172.21.15.89:0/3037292820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef415ab60 con 0x7f5ef41597f0 2023-10-10T20:35:59.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.789+0000 7f5ef3fff700 1 -- 172.21.15.89:0/3037292820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef4069b80 con 0x7f5ef41597f0 2023-10-10T20:35:59.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.789+0000 7f5eeb7fe700 1 -- 172.21.15.89:0/3037292820 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5edc01b3b0 con 0x7f5ef41597f0 2023-10-10T20:35:59.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.790+0000 7f5eeb7fe700 1 --2- 172.21.15.89:0/3037292820 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5ed4066f30 0x7f5ed40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:35:59.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.790+0000 7f5eeb7fe700 1 -- 172.21.15.89:0/3037292820 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5edc0839b0 con 0x7f5ef41597f0 2023-10-10T20:35:59.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.790+0000 7f5ef2ffd700 1 --2- 172.21.15.89:0/3037292820 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5ed4066f30 0x7f5ed40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:35:59.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.790+0000 7f5ef2ffd700 1 --2- 172.21.15.89:0/3037292820 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5ed4066f30 0x7f5ed40693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40093f0 tx=0x7f5ee4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:35:59.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.794+0000 7f5eeb7fe700 1 -- 172.21.15.89:0/3037292820 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5edc04b5a0 con 0x7f5ef41597f0 2023-10-10T20:35:59.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.973+0000 7f5ef3fff700 1 -- 172.21.15.89:0/3037292820 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ef404c3c0 con 0x7f5ed4066f30 2023-10-10T20:35:59.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.981+0000 7f5eeb7fe700 1 -- 172.21.15.89:0/3037292820 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289448 (secure 0 0 0) 0x7f5ef404c3c0 con 0x7f5ed4066f30 2023-10-10T20:35:59.987 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:35:59.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.985+0000 7f5ef3fff700 1 -- 172.21.15.89:0/3037292820 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5ed4066f30 msgr2=0x7f5ed40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:59.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.985+0000 7f5ef3fff700 1 --2- 172.21.15.89:0/3037292820 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5ed4066f30 0x7f5ed40693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40093f0 tx=0x7f5ee4008040 comp rx=0 tx=0).stop 2023-10-10T20:35:59.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.985+0000 7f5ef3fff700 1 -- 172.21.15.89:0/3037292820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ef41597f0 msgr2=0x7f5ef40c5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:35:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.985+0000 7f5ef3fff700 1 --2- 172.21.15.89:0/3037292820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ef41597f0 0x7f5ef40c5db0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00c840 tx=0x7f5edc007db0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.985+0000 7f5ef3fff700 1 -- 172.21.15.89:0/3037292820 shutdown_connections 2023-10-10T20:35:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.985+0000 7f5ef3fff700 1 --2- 172.21.15.89:0/3037292820 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5ef415d230 0x7f5ef4154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.985+0000 7f5ef3fff700 1 --2- 172.21.15.89:0/3037292820 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5ed4066f30 0x7f5ed40693e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.985+0000 7f5ef3fff700 1 --2- 172.21.15.89:0/3037292820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ef4159420 0x7f5ef4159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.985+0000 7f5ef3fff700 1 --2- 172.21.15.89:0/3037292820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ef41597f0 0x7f5ef40c5db0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:35:59.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.985+0000 7f5ef3fff700 1 -- 172.21.15.89:0/3037292820 >> 172.21.15.89:0/3037292820 conn(0x7f5ef40d0660 msgr2=0x7f5ef4157220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:35:59.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.985+0000 7f5ef3fff700 1 -- 172.21.15.89:0/3037292820 shutdown_connections 2023-10-10T20:35:59.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:35:59.985+0000 7f5ef3fff700 1 -- 172.21.15.89:0/3037292820 wait complete. 2023-10-10T20:35:59.993 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:35:59.999 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2023-10-10T20:35:58.667533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2384196,"kb_used_data":5892,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752590524,"statfs":{"total":773094113280,"available":770652696576,"internally_reserved":0,"allocated":6033408,"data_stored":3200715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.746916"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.949172+0000","last_change":"2023-10-10T20:35:55.947951+0000","last_active":"2023-10-10T20:35:55.949172+0000","last_peered":"2023-10-10T20:35:55.949172+0000","last_clean":"2023-10-10T20:35:55.949172+0000","last_became_active":"2023-10-10T20:35:55.947721+0000","last_became_peered":"2023-10-10T20:35:55.947721+0000","last_unstale":"2023-10-10T20:35:55.949172+0000","last_undegraded":"2023-10-10T20:35:55.949172+0000","last_fullsized":"2023-10-10T20:35:55.949172+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:26:38.266002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.954592+0000","last_change":"2023-10-10T20:35:55.953795+0000","last_active":"2023-10-10T20:35:55.954592+0000","last_peered":"2023-10-10T20:35:55.954592+0000","last_clean":"2023-10-10T20:35:55.954592+0000","last_became_active":"2023-10-10T20:35:55.953541+0000","last_became_peered":"2023-10-10T20:35:55.953541+0000","last_unstale":"2023-10-10T20:35:55.954592+0000","last_undegraded":"2023-10-10T20:35:55.954592+0000","last_fullsized":"2023-10-10T20:35:55.954592+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:26:57.532237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.949272+0000","last_change":"2023-10-10T20:35:55.945095+0000","last_active":"2023-10-10T20:35:55.949272+0000","last_peered":"2023-10-10T20:35:55.949272+0000","last_clean":"2023-10-10T20:35:55.949272+0000","last_became_active":"2023-10-10T20:35:55.944559+0000","last_became_peered":"2023-10-10T20:35:55.944559+0000","last_unstale":"2023-10-10T20:35:55.949272+0000","last_undegraded":"2023-10-10T20:35:55.949272+0000","last_fullsized":"2023-10-10T20:35:55.949272+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:20:10.900594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.954258+0000","last_change":"2023-10-10T20:35:55.952175+0000","last_active":"2023-10-10T20:35:55.954258+0000","last_peered":"2023-10-10T20:35:55.954258+0000","last_clean":"2023-10-10T20:35:55.954258+0000","last_became_active":"2023-10-10T20:35:55.951606+0000","last_became_peered":"2023-10-10T20:35:55.951606+0000","last_unstale":"2023-10-10T20:35:55.954258+0000","last_undegraded":"2023-10-10T20:35:55.954258+0000","last_fullsized":"2023-10-10T20:35:55.954258+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:57:31.137405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.949226+0000","last_change":"2023-10-10T20:35:55.946843+0000","last_active":"2023-10-10T20:35:55.949226+0000","last_peered":"2023-10-10T20:35:55.949226+0000","last_clean":"2023-10-10T20:35:55.949226+0000","last_became_active":"2023-10-10T20:35:55.946597+0000","last_became_peered":"2023-10-10T20:35:55.946597+0000","last_unstale":"2023-10-10T20:35:55.949226+0000","last_undegraded":"2023-10-10T20:35:55.949226+0000","last_fullsized":"2023-10-10T20:35:55.949226+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:27:54.804150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960301+0000","last_change":"2023-10-10T20:35:55.947148+0000","last_active":"2023-10-10T20:35:55.960301+0000","last_peered":"2023-10-10T20:35:55.960301+0000","last_clean":"2023-10-10T20:35:55.960301+0000","last_became_active":"2023-10-10T20:35:55.946908+0000","last_became_peered":"2023-10-10T20:35:55.946908+0000","last_unstale":"2023-10-10T20:35:55.960301+0000","last_undegraded":"2023-10-10T20:35:55.960301+0000","last_fullsized":"2023-10-10T20:35:55.960301+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:23:30.836699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.950910+0000","last_change":"2023-10-10T20:35:55.948700+0000","last_active":"2023-10-10T20:35:55.950910+0000","last_peered":"2023-10-10T20:35:55.950910+0000","last_clean":"2023-10-10T20:35:55.950910+0000","last_became_active":"2023-10-10T20:35:55.948399+0000","last_became_peered":"2023-10-10T20:35:55.948399+0000","last_unstale":"2023-10-10T20:35:55.950910+0000","last_undegraded":"2023-10-10T20:35:55.950910+0000","last_fullsized":"2023-10-10T20:35:55.950910+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:27:11.743663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.959888+0000","last_change":"2023-10-10T20:35:55.954053+0000","last_active":"2023-10-10T20:35:55.959888+0000","last_peered":"2023-10-10T20:35:55.959888+0000","last_clean":"2023-10-10T20:35:55.959888+0000","last_became_active":"2023-10-10T20:35:55.953848+0000","last_became_peered":"2023-10-10T20:35:55.953848+0000","last_unstale":"2023-10-10T20:35:55.959888+0000","last_undegraded":"2023-10-10T20:35:55.959888+0000","last_fullsized":"2023-10-10T20:35:55.959888+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:52:47.569198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960249+0000","last_change":"2023-10-10T20:35:55.958056+0000","last_active":"2023-10-10T20:35:55.960249+0000","last_peered":"2023-10-10T20:35:55.960249+0000","last_clean":"2023-10-10T20:35:55.960249+0000","last_became_active":"2023-10-10T20:35:55.957775+0000","last_became_peered":"2023-10-10T20:35:55.957775+0000","last_unstale":"2023-10-10T20:35:55.960249+0000","last_undegraded":"2023-10-10T20:35:55.960249+0000","last_fullsized":"2023-10-10T20:35:55.960249+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:24:19.034235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.958011+0000","last_change":"2023-10-10T20:35:55.953389+0000","last_active":"2023-10-10T20:35:55.958011+0000","last_peered":"2023-10-10T20:35:55.958011+0000","last_clean":"2023-10-10T20:35:55.958011+0000","last_became_active":"2023-10-10T20:35:55.953055+0000","last_became_peered":"2023-10-10T20:35:55.953055+0000","last_unstale":"2023-10-10T20:35:55.958011+0000","last_undegraded":"2023-10-10T20:35:55.958011+0000","last_fullsized":"2023-10-10T20:35:55.958011+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:32:26.746310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.958346+0000","last_change":"2023-10-10T20:35:55.956182+0000","last_active":"2023-10-10T20:35:55.958346+0000","last_peered":"2023-10-10T20:35:55.958346+0000","last_clean":"2023-10-10T20:35:55.958346+0000","last_became_active":"2023-10-10T20:35:55.955947+0000","last_became_peered":"2023-10-10T20:35:55.955947+0000","last_unstale":"2023-10-10T20:35:55.958346+0000","last_undegraded":"2023-10-10T20:35:55.958346+0000","last_fullsized":"2023-10-10T20:35:55.958346+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:17:24.929284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.954505+0000","last_change":"2023-10-10T20:35:55.952367+0000","last_active":"2023-10-10T20:35:55.954505+0000","last_peered":"2023-10-10T20:35:55.954505+0000","last_clean":"2023-10-10T20:35:55.954505+0000","last_became_active":"2023-10-10T20:35:55.951929+0000","last_became_peered":"2023-10-10T20:35:55.951929+0000","last_unstale":"2023-10-10T20:35:55.954505+0000","last_undegraded":"2023-10-10T20:35:55.954505+0000","last_fullsized":"2023-10-10T20:35:55.954505+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:20:10.900594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961843+0000","last_change":"2023-10-10T20:35:55.960804+0000","last_active":"2023-10-10T20:35:55.961843+0000","last_peered":"2023-10-10T20:35:55.961843+0000","last_clean":"2023-10-10T20:35:55.961843+0000","last_became_active":"2023-10-10T20:35:55.960543+0000","last_became_peered":"2023-10-10T20:35:55.960543+0000","last_unstale":"2023-10-10T20:35:55.961843+0000","last_undegraded":"2023-10-10T20:35:55.961843+0000","last_fullsized":"2023-10-10T20:35:55.961843+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:29:16.538930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.946660+0000","last_change":"2023-10-10T20:35:55.944587+0000","last_active":"2023-10-10T20:35:55.946660+0000","last_peered":"2023-10-10T20:35:55.946660+0000","last_clean":"2023-10-10T20:35:55.946660+0000","last_became_active":"2023-10-10T20:35:55.944317+0000","last_became_peered":"2023-10-10T20:35:55.944317+0000","last_unstale":"2023-10-10T20:35:55.946660+0000","last_undegraded":"2023-10-10T20:35:55.946660+0000","last_fullsized":"2023-10-10T20:35:55.946660+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:32:26.746310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.954547+0000","last_change":"2023-10-10T20:35:55.953565+0000","last_active":"2023-10-10T20:35:55.954547+0000","last_peered":"2023-10-10T20:35:55.954547+0000","last_clean":"2023-10-10T20:35:55.954547+0000","last_became_active":"2023-10-10T20:35:55.953239+0000","last_became_peered":"2023-10-10T20:35:55.953239+0000","last_unstale":"2023-10-10T20:35:55.954547+0000","last_undegraded":"2023-10-10T20:35:55.954547+0000","last_fullsized":"2023-10-10T20:35:55.954547+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:56.046059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961800+0000","last_change":"2023-10-10T20:35:55.956399+0000","last_active":"2023-10-10T20:35:55.961800+0000","last_peered":"2023-10-10T20:35:55.961800+0000","last_clean":"2023-10-10T20:35:55.961800+0000","last_became_active":"2023-10-10T20:35:55.956158+0000","last_became_peered":"2023-10-10T20:35:55.956158+0000","last_unstale":"2023-10-10T20:35:55.961800+0000","last_undegraded":"2023-10-10T20:35:55.961800+0000","last_fullsized":"2023-10-10T20:35:55.961800+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:40:17.028747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.964308+0000","last_change":"2023-10-10T20:35:55.961637+0000","last_active":"2023-10-10T20:35:55.964308+0000","last_peered":"2023-10-10T20:35:55.964308+0000","last_clean":"2023-10-10T20:35:55.964308+0000","last_became_active":"2023-10-10T20:35:55.961347+0000","last_became_peered":"2023-10-10T20:35:55.961347+0000","last_unstale":"2023-10-10T20:35:55.964308+0000","last_undegraded":"2023-10-10T20:35:55.964308+0000","last_fullsized":"2023-10-10T20:35:55.964308+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:16.560154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.949316+0000","last_change":"2023-10-10T20:35:55.945438+0000","last_active":"2023-10-10T20:35:55.949316+0000","last_peered":"2023-10-10T20:35:55.949316+0000","last_clean":"2023-10-10T20:35:55.949316+0000","last_became_active":"2023-10-10T20:35:55.945124+0000","last_became_peered":"2023-10-10T20:35:55.945124+0000","last_unstale":"2023-10-10T20:35:55.949316+0000","last_undegraded":"2023-10-10T20:35:55.949316+0000","last_fullsized":"2023-10-10T20:35:55.949316+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:57:31.137405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.949200+0000","last_change":"2023-10-10T20:35:55.947979+0000","last_active":"2023-10-10T20:35:55.949200+0000","last_peered":"2023-10-10T20:35:55.949200+0000","last_clean":"2023-10-10T20:35:55.949200+0000","last_became_active":"2023-10-10T20:35:55.947681+0000","last_became_peered":"2023-10-10T20:35:55.947681+0000","last_unstale":"2023-10-10T20:35:55.949200+0000","last_undegraded":"2023-10-10T20:35:55.949200+0000","last_fullsized":"2023-10-10T20:35:55.949200+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:04:06.958649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.964398+0000","last_change":"2023-10-10T20:35:55.962840+0000","last_active":"2023-10-10T20:35:55.964398+0000","last_peered":"2023-10-10T20:35:55.964398+0000","last_clean":"2023-10-10T20:35:55.964398+0000","last_became_active":"2023-10-10T20:35:55.962576+0000","last_became_peered":"2023-10-10T20:35:55.962576+0000","last_unstale":"2023-10-10T20:35:55.964398+0000","last_undegraded":"2023-10-10T20:35:55.964398+0000","last_fullsized":"2023-10-10T20:35:55.964398+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:22:35.423687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.956361+0000","last_change":"2023-10-10T20:35:55.952472+0000","last_active":"2023-10-10T20:35:55.956361+0000","last_peered":"2023-10-10T20:35:55.956361+0000","last_clean":"2023-10-10T20:35:55.956361+0000","last_became_active":"2023-10-10T20:35:55.952179+0000","last_became_peered":"2023-10-10T20:35:55.952179+0000","last_unstale":"2023-10-10T20:35:55.956361+0000","last_undegraded":"2023-10-10T20:35:55.956361+0000","last_fullsized":"2023-10-10T20:35:55.956361+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:43:59.553286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.952178+0000","last_change":"2023-10-10T20:35:55.950427+0000","last_active":"2023-10-10T20:35:55.952178+0000","last_peered":"2023-10-10T20:35:55.952178+0000","last_clean":"2023-10-10T20:35:55.952178+0000","last_became_active":"2023-10-10T20:35:55.949974+0000","last_became_peered":"2023-10-10T20:35:55.949974+0000","last_unstale":"2023-10-10T20:35:55.952178+0000","last_undegraded":"2023-10-10T20:35:55.952178+0000","last_fullsized":"2023-10-10T20:35:55.952178+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:25.472048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.956294+0000","last_change":"2023-10-10T20:35:55.951213+0000","last_active":"2023-10-10T20:35:55.956294+0000","last_peered":"2023-10-10T20:35:55.956294+0000","last_clean":"2023-10-10T20:35:55.956294+0000","last_became_active":"2023-10-10T20:35:55.950366+0000","last_became_peered":"2023-10-10T20:35:55.950366+0000","last_unstale":"2023-10-10T20:35:55.956294+0000","last_undegraded":"2023-10-10T20:35:55.956294+0000","last_fullsized":"2023-10-10T20:35:55.956294+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:02.640005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.962808+0000","last_change":"2023-10-10T20:35:55.961384+0000","last_active":"2023-10-10T20:35:55.962808+0000","last_peered":"2023-10-10T20:35:55.962808+0000","last_clean":"2023-10-10T20:35:55.962808+0000","last_became_active":"2023-10-10T20:35:55.961136+0000","last_became_peered":"2023-10-10T20:35:55.961136+0000","last_unstale":"2023-10-10T20:35:55.962808+0000","last_undegraded":"2023-10-10T20:35:55.962808+0000","last_fullsized":"2023-10-10T20:35:55.962808+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:49.506716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.959667+0000","last_change":"2023-10-10T20:35:55.955629+0000","last_active":"2023-10-10T20:35:55.959667+0000","last_peered":"2023-10-10T20:35:55.959667+0000","last_clean":"2023-10-10T20:35:55.959667+0000","last_became_active":"2023-10-10T20:35:55.955366+0000","last_became_peered":"2023-10-10T20:35:55.955366+0000","last_unstale":"2023-10-10T20:35:55.959667+0000","last_undegraded":"2023-10-10T20:35:55.959667+0000","last_fullsized":"2023-10-10T20:35:55.959667+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:52:23.620245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.950923+0000","last_change":"2023-10-10T20:35:55.949557+0000","last_active":"2023-10-10T20:35:55.950923+0000","last_peered":"2023-10-10T20:35:55.950923+0000","last_clean":"2023-10-10T20:35:55.950923+0000","last_became_active":"2023-10-10T20:35:55.947280+0000","last_became_peered":"2023-10-10T20:35:55.947280+0000","last_unstale":"2023-10-10T20:35:55.950923+0000","last_undegraded":"2023-10-10T20:35:55.950923+0000","last_fullsized":"2023-10-10T20:35:55.950923+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:10:25.924046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960066+0000","last_change":"2023-10-10T20:35:55.945650+0000","last_active":"2023-10-10T20:35:55.960066+0000","last_peered":"2023-10-10T20:35:55.960066+0000","last_clean":"2023-10-10T20:35:55.960066+0000","last_became_active":"2023-10-10T20:35:55.945405+0000","last_became_peered":"2023-10-10T20:35:55.945405+0000","last_unstale":"2023-10-10T20:35:55.960066+0000","last_undegraded":"2023-10-10T20:35:55.960066+0000","last_fullsized":"2023-10-10T20:35:55.960066+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:00:30.852312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.959671+0000","last_change":"2023-10-10T20:35:55.952327+0000","last_active":"2023-10-10T20:35:55.959671+0000","last_peered":"2023-10-10T20:35:55.959671+0000","last_clean":"2023-10-10T20:35:55.959671+0000","last_became_active":"2023-10-10T20:35:55.952081+0000","last_became_peered":"2023-10-10T20:35:55.952081+0000","last_unstale":"2023-10-10T20:35:55.959671+0000","last_undegraded":"2023-10-10T20:35:55.959671+0000","last_fullsized":"2023-10-10T20:35:55.959671+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:26:28.857713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.963571+0000","last_change":"2023-10-10T20:35:55.961220+0000","last_active":"2023-10-10T20:35:55.963571+0000","last_peered":"2023-10-10T20:35:55.963571+0000","last_clean":"2023-10-10T20:35:55.963571+0000","last_became_active":"2023-10-10T20:35:55.960921+0000","last_became_peered":"2023-10-10T20:35:55.960921+0000","last_unstale":"2023-10-10T20:35:55.963571+0000","last_undegraded":"2023-10-10T20:35:55.963571+0000","last_fullsized":"2023-10-10T20:35:55.963571+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:34:43.703414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.946304+0000","last_change":"2023-10-10T20:35:55.944128+0000","last_active":"2023-10-10T20:35:55.946304+0000","last_peered":"2023-10-10T20:35:55.946304+0000","last_clean":"2023-10-10T20:35:55.946304+0000","last_became_active":"2023-10-10T20:35:55.943872+0000","last_became_peered":"2023-10-10T20:35:55.943872+0000","last_unstale":"2023-10-10T20:35:55.946304+0000","last_undegraded":"2023-10-10T20:35:55.946304+0000","last_fullsized":"2023-10-10T20:35:55.946304+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:53:55.367489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.958226+0000","last_change":"2023-10-10T20:35:55.951763+0000","last_active":"2023-10-10T20:35:55.958226+0000","last_peered":"2023-10-10T20:35:55.958226+0000","last_clean":"2023-10-10T20:35:55.958226+0000","last_became_active":"2023-10-10T20:35:55.951521+0000","last_became_peered":"2023-10-10T20:35:55.951521+0000","last_unstale":"2023-10-10T20:35:55.958226+0000","last_undegraded":"2023-10-10T20:35:55.958226+0000","last_fullsized":"2023-10-10T20:35:55.958226+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:45:13.936944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.964309+0000","last_change":"2023-10-10T20:35:55.961725+0000","last_active":"2023-10-10T20:35:55.964309+0000","last_peered":"2023-10-10T20:35:55.964309+0000","last_clean":"2023-10-10T20:35:55.964309+0000","last_became_active":"2023-10-10T20:35:55.961471+0000","last_became_peered":"2023-10-10T20:35:55.961471+0000","last_unstale":"2023-10-10T20:35:55.964309+0000","last_undegraded":"2023-10-10T20:35:55.964309+0000","last_fullsized":"2023-10-10T20:35:55.964309+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:33:38.647459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960152+0000","last_change":"2023-10-10T20:35:55.944952+0000","last_active":"2023-10-10T20:35:55.960152+0000","last_peered":"2023-10-10T20:35:55.960152+0000","last_clean":"2023-10-10T20:35:55.960152+0000","last_became_active":"2023-10-10T20:35:55.944700+0000","last_became_peered":"2023-10-10T20:35:55.944700+0000","last_unstale":"2023-10-10T20:35:55.960152+0000","last_undegraded":"2023-10-10T20:35:55.960152+0000","last_fullsized":"2023-10-10T20:35:55.960152+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:34:13.167717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"49'4","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961533+0000","last_change":"2023-10-10T20:35:54.943470+0000","last_active":"2023-10-10T20:35:55.961533+0000","last_peered":"2023-10-10T20:35:55.961533+0000","last_clean":"2023-10-10T20:35:55.961533+0000","last_became_active":"2023-10-10T20:35:54.943215+0000","last_became_peered":"2023-10-10T20:35:54.943215+0000","last_unstale":"2023-10-10T20:35:55.961533+0000","last_undegraded":"2023-10-10T20:35:55.961533+0000","last_fullsized":"2023-10-10T20:35:55.961533+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:30:09.063259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.958146+0000","last_change":"2023-10-10T20:35:55.951492+0000","last_active":"2023-10-10T20:35:55.958146+0000","last_peered":"2023-10-10T20:35:55.958146+0000","last_clean":"2023-10-10T20:35:55.958146+0000","last_became_active":"2023-10-10T20:35:55.951252+0000","last_became_peered":"2023-10-10T20:35:55.951252+0000","last_unstale":"2023-10-10T20:35:55.958146+0000","last_undegraded":"2023-10-10T20:35:55.958146+0000","last_fullsized":"2023-10-10T20:35:55.958146+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:27:11.743663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"49'1","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.947489+0000","last_change":"2023-10-10T20:35:54.946572+0000","last_active":"2023-10-10T20:35:55.947489+0000","last_peered":"2023-10-10T20:35:55.947489+0000","last_clean":"2023-10-10T20:35:55.947489+0000","last_became_active":"2023-10-10T20:35:54.946318+0000","last_became_peered":"2023-10-10T20:35:54.946318+0000","last_unstale":"2023-10-10T20:35:55.947489+0000","last_undegraded":"2023-10-10T20:35:55.947489+0000","last_fullsized":"2023-10-10T20:35:55.947489+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:00:29.845458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961746+0000","last_change":"2023-10-10T20:35:55.957973+0000","last_active":"2023-10-10T20:35:55.961746+0000","last_peered":"2023-10-10T20:35:55.961746+0000","last_clean":"2023-10-10T20:35:55.961746+0000","last_became_active":"2023-10-10T20:35:55.957725+0000","last_became_peered":"2023-10-10T20:35:55.957725+0000","last_unstale":"2023-10-10T20:35:55.961746+0000","last_undegraded":"2023-10-10T20:35:55.961746+0000","last_fullsized":"2023-10-10T20:35:55.961746+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:59:36.779168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"49'2","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960013+0000","last_change":"2023-10-10T20:35:54.940850+0000","last_active":"2023-10-10T20:35:55.960013+0000","last_peered":"2023-10-10T20:35:55.960013+0000","last_clean":"2023-10-10T20:35:55.960013+0000","last_became_active":"2023-10-10T20:35:54.940614+0000","last_became_peered":"2023-10-10T20:35:54.940614+0000","last_unstale":"2023-10-10T20:35:55.960013+0000","last_undegraded":"2023-10-10T20:35:55.960013+0000","last_fullsized":"2023-10-10T20:35:55.960013+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:32:18.143428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961566+0000","last_change":"2023-10-10T20:35:55.957441+0000","last_active":"2023-10-10T20:35:55.961566+0000","last_peered":"2023-10-10T20:35:55.961566+0000","last_clean":"2023-10-10T20:35:55.961566+0000","last_became_active":"2023-10-10T20:35:55.956959+0000","last_became_peered":"2023-10-10T20:35:55.956959+0000","last_unstale":"2023-10-10T20:35:55.961566+0000","last_undegraded":"2023-10-10T20:35:55.961566+0000","last_fullsized":"2023-10-10T20:35:55.961566+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:51.054414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"49'2","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961103+0000","last_change":"2023-10-10T20:35:54.946218+0000","last_active":"2023-10-10T20:35:55.961103+0000","last_peered":"2023-10-10T20:35:55.961103+0000","last_clean":"2023-10-10T20:35:55.961103+0000","last_became_active":"2023-10-10T20:35:54.945986+0000","last_became_peered":"2023-10-10T20:35:54.945986+0000","last_unstale":"2023-10-10T20:35:55.961103+0000","last_undegraded":"2023-10-10T20:35:55.961103+0000","last_fullsized":"2023-10-10T20:35:55.961103+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:19:13.265287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.950152+0000","last_change":"2023-10-10T20:35:55.949173+0000","last_active":"2023-10-10T20:35:55.950152+0000","last_peered":"2023-10-10T20:35:55.950152+0000","last_clean":"2023-10-10T20:35:55.950152+0000","last_became_active":"2023-10-10T20:35:55.948869+0000","last_became_peered":"2023-10-10T20:35:55.948869+0000","last_unstale":"2023-10-10T20:35:55.950152+0000","last_undegraded":"2023-10-10T20:35:55.950152+0000","last_fullsized":"2023-10-10T20:35:55.950152+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:37:40.585130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"49'1","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.945361+0000","last_change":"2023-10-10T20:35:54.940564+0000","last_active":"2023-10-10T20:35:55.945361+0000","last_peered":"2023-10-10T20:35:55.945361+0000","last_clean":"2023-10-10T20:35:55.945361+0000","last_became_active":"2023-10-10T20:35:54.940322+0000","last_became_peered":"2023-10-10T20:35:54.940322+0000","last_unstale":"2023-10-10T20:35:55.945361+0000","last_undegraded":"2023-10-10T20:35:55.945361+0000","last_fullsized":"2023-10-10T20:35:55.945361+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:29:15.532075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.964426+0000","last_change":"2023-10-10T20:35:55.963301+0000","last_active":"2023-10-10T20:35:55.964426+0000","last_peered":"2023-10-10T20:35:55.964426+0000","last_clean":"2023-10-10T20:35:55.964426+0000","last_became_active":"2023-10-10T20:35:55.962761+0000","last_became_peered":"2023-10-10T20:35:55.962761+0000","last_unstale":"2023-10-10T20:35:55.964426+0000","last_undegraded":"2023-10-10T20:35:55.964426+0000","last_fullsized":"2023-10-10T20:35:55.964426+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:35:25.594991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"49'1","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.956412+0000","last_change":"2023-10-10T20:35:54.941702+0000","last_active":"2023-10-10T20:35:55.956412+0000","last_peered":"2023-10-10T20:35:55.956412+0000","last_clean":"2023-10-10T20:35:55.956412+0000","last_became_active":"2023-10-10T20:35:54.941461+0000","last_became_peered":"2023-10-10T20:35:54.941461+0000","last_unstale":"2023-10-10T20:35:55.956412+0000","last_undegraded":"2023-10-10T20:35:55.956412+0000","last_fullsized":"2023-10-10T20:35:55.956412+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:32:25.739455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.952775+0000","last_change":"2023-10-10T20:35:55.951488+0000","last_active":"2023-10-10T20:35:55.952775+0000","last_peered":"2023-10-10T20:35:55.952775+0000","last_clean":"2023-10-10T20:35:55.952775+0000","last_became_active":"2023-10-10T20:35:55.950945+0000","last_became_peered":"2023-10-10T20:35:55.950945+0000","last_unstale":"2023-10-10T20:35:55.952775+0000","last_undegraded":"2023-10-10T20:35:55.952775+0000","last_fullsized":"2023-10-10T20:35:55.952775+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:18:34.718096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.935825+0000","last_change":"2023-10-10T20:35:54.943425+0000","last_active":"2023-10-10T20:35:55.935825+0000","last_peered":"2023-10-10T20:35:55.935825+0000","last_clean":"2023-10-10T20:35:55.935825+0000","last_became_active":"2023-10-10T20:35:54.943067+0000","last_became_peered":"2023-10-10T20:35:54.943067+0000","last_unstale":"2023-10-10T20:35:55.935825+0000","last_undegraded":"2023-10-10T20:35:55.935825+0000","last_fullsized":"2023-10-10T20:35:55.935825+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:48:03.966676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.964339+0000","last_change":"2023-10-10T20:35:55.961977+0000","last_active":"2023-10-10T20:35:55.964339+0000","last_peered":"2023-10-10T20:35:55.964339+0000","last_clean":"2023-10-10T20:35:55.964339+0000","last_became_active":"2023-10-10T20:35:55.961321+0000","last_became_peered":"2023-10-10T20:35:55.961321+0000","last_unstale":"2023-10-10T20:35:55.964339+0000","last_undegraded":"2023-10-10T20:35:55.964339+0000","last_fullsized":"2023-10-10T20:35:55.964339+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:05.710248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.936391+0000","last_change":"2023-10-10T20:35:54.947632+0000","last_active":"2023-10-10T20:35:55.936391+0000","last_peered":"2023-10-10T20:35:55.936391+0000","last_clean":"2023-10-10T20:35:55.936391+0000","last_became_active":"2023-10-10T20:35:54.947355+0000","last_became_peered":"2023-10-10T20:35:54.947355+0000","last_unstale":"2023-10-10T20:35:55.936391+0000","last_undegraded":"2023-10-10T20:35:55.936391+0000","last_fullsized":"2023-10-10T20:35:55.936391+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:00:25.701036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.954539+0000","last_change":"2023-10-10T20:35:55.953501+0000","last_active":"2023-10-10T20:35:55.954539+0000","last_peered":"2023-10-10T20:35:55.954539+0000","last_clean":"2023-10-10T20:35:55.954539+0000","last_became_active":"2023-10-10T20:35:55.953280+0000","last_became_peered":"2023-10-10T20:35:55.953280+0000","last_unstale":"2023-10-10T20:35:55.954539+0000","last_undegraded":"2023-10-10T20:35:55.954539+0000","last_fullsized":"2023-10-10T20:35:55.954539+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:56:52.778641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.933687+0000","last_change":"2023-10-10T20:35:54.943028+0000","last_active":"2023-10-10T20:35:55.933687+0000","last_peered":"2023-10-10T20:35:55.933687+0000","last_clean":"2023-10-10T20:35:55.933687+0000","last_became_active":"2023-10-10T20:35:54.942704+0000","last_became_peered":"2023-10-10T20:35:54.942704+0000","last_unstale":"2023-10-10T20:35:55.933687+0000","last_undegraded":"2023-10-10T20:35:55.933687+0000","last_fullsized":"2023-10-10T20:35:55.933687+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:55.039204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.959208+0000","last_change":"2023-10-10T20:35:55.957452+0000","last_active":"2023-10-10T20:35:55.959208+0000","last_peered":"2023-10-10T20:35:55.959208+0000","last_clean":"2023-10-10T20:35:55.959208+0000","last_became_active":"2023-10-10T20:35:55.957155+0000","last_became_peered":"2023-10-10T20:35:55.957155+0000","last_unstale":"2023-10-10T20:35:55.959208+0000","last_undegraded":"2023-10-10T20:35:55.959208+0000","last_fullsized":"2023-10-10T20:35:55.959208+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:55.043141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.937839+0000","last_change":"2023-10-10T20:35:54.943788+0000","last_active":"2023-10-10T20:35:55.937839+0000","last_peered":"2023-10-10T20:35:55.937839+0000","last_clean":"2023-10-10T20:35:55.937839+0000","last_became_active":"2023-10-10T20:35:54.943631+0000","last_became_peered":"2023-10-10T20:35:54.943631+0000","last_unstale":"2023-10-10T20:35:55.937839+0000","last_undegraded":"2023-10-10T20:35:55.937839+0000","last_fullsized":"2023-10-10T20:35:55.937839+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:40:16.021893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.957978+0000","last_change":"2023-10-10T20:35:55.954647+0000","last_active":"2023-10-10T20:35:55.957978+0000","last_peered":"2023-10-10T20:35:55.957978+0000","last_clean":"2023-10-10T20:35:55.957978+0000","last_became_active":"2023-10-10T20:35:55.952784+0000","last_became_peered":"2023-10-10T20:35:55.952784+0000","last_unstale":"2023-10-10T20:35:55.957978+0000","last_undegraded":"2023-10-10T20:35:55.957978+0000","last_fullsized":"2023-10-10T20:35:55.957978+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:56:52.778641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"49'3","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.958634+0000","last_change":"2023-10-10T20:35:54.943076+0000","last_active":"2023-10-10T20:35:55.958634+0000","last_peered":"2023-10-10T20:35:55.958634+0000","last_clean":"2023-10-10T20:35:55.958634+0000","last_became_active":"2023-10-10T20:35:54.942812+0000","last_became_peered":"2023-10-10T20:35:54.942812+0000","last_unstale":"2023-10-10T20:35:55.958634+0000","last_undegraded":"2023-10-10T20:35:55.958634+0000","last_fullsized":"2023-10-10T20:35:55.958634+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:56:51.771786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960155+0000","last_change":"2023-10-10T20:35:55.957956+0000","last_active":"2023-10-10T20:35:55.960155+0000","last_peered":"2023-10-10T20:35:55.960155+0000","last_clean":"2023-10-10T20:35:55.960155+0000","last_became_active":"2023-10-10T20:35:55.957535+0000","last_became_peered":"2023-10-10T20:35:55.957535+0000","last_unstale":"2023-10-10T20:35:55.960155+0000","last_undegraded":"2023-10-10T20:35:55.960155+0000","last_fullsized":"2023-10-10T20:35:55.960155+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:14:22.420266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"49'6","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.967270+0000","last_change":"2023-10-10T20:35:54.944052+0000","last_active":"2023-10-10T20:35:55.967270+0000","last_peered":"2023-10-10T20:35:55.967270+0000","last_clean":"2023-10-10T20:35:55.967270+0000","last_became_active":"2023-10-10T20:35:54.943539+0000","last_became_peered":"2023-10-10T20:35:54.943539+0000","last_unstale":"2023-10-10T20:35:55.967270+0000","last_undegraded":"2023-10-10T20:35:55.967270+0000","last_fullsized":"2023-10-10T20:35:55.967270+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:12:41.189930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961458+0000","last_change":"2023-10-10T20:35:55.960062+0000","last_active":"2023-10-10T20:35:55.961458+0000","last_peered":"2023-10-10T20:35:55.961458+0000","last_clean":"2023-10-10T20:35:55.961458+0000","last_became_active":"2023-10-10T20:35:55.959807+0000","last_became_peered":"2023-10-10T20:35:55.959807+0000","last_unstale":"2023-10-10T20:35:55.961458+0000","last_undegraded":"2023-10-10T20:35:55.961458+0000","last_fullsized":"2023-10-10T20:35:55.961458+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:14:22.420266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"49'2","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.950092+0000","last_change":"2023-10-10T20:35:54.945458+0000","last_active":"2023-10-10T20:35:55.950092+0000","last_peered":"2023-10-10T20:35:55.950092+0000","last_clean":"2023-10-10T20:35:55.950092+0000","last_became_active":"2023-10-10T20:35:54.943874+0000","last_became_peered":"2023-10-10T20:35:54.943874+0000","last_unstale":"2023-10-10T20:35:55.950092+0000","last_undegraded":"2023-10-10T20:35:55.950092+0000","last_fullsized":"2023-10-10T20:35:55.950092+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:54.036286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.950504+0000","last_change":"2023-10-10T20:35:55.946244+0000","last_active":"2023-10-10T20:35:55.950504+0000","last_peered":"2023-10-10T20:35:55.950504+0000","last_clean":"2023-10-10T20:35:55.950504+0000","last_became_active":"2023-10-10T20:35:55.945881+0000","last_became_peered":"2023-10-10T20:35:55.945881+0000","last_unstale":"2023-10-10T20:35:55.950504+0000","last_undegraded":"2023-10-10T20:35:55.950504+0000","last_fullsized":"2023-10-10T20:35:55.950504+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:17.215044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.940109+0000","last_change":"2023-10-10T20:35:54.946176+0000","last_active":"2023-10-10T20:35:55.940109+0000","last_peered":"2023-10-10T20:35:55.940109+0000","last_clean":"2023-10-10T20:35:55.940109+0000","last_became_active":"2023-10-10T20:35:54.945884+0000","last_became_peered":"2023-10-10T20:35:54.945884+0000","last_unstale":"2023-10-10T20:35:55.940109+0000","last_undegraded":"2023-10-10T20:35:55.940109+0000","last_fullsized":"2023-10-10T20:35:55.940109+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:31:11.544868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.945789+0000","last_change":"2023-10-10T20:35:55.943829+0000","last_active":"2023-10-10T20:35:55.945789+0000","last_peered":"2023-10-10T20:35:55.945789+0000","last_clean":"2023-10-10T20:35:55.945789+0000","last_became_active":"2023-10-10T20:35:55.943469+0000","last_became_peered":"2023-10-10T20:35:55.943469+0000","last_unstale":"2023-10-10T20:35:55.945789+0000","last_undegraded":"2023-10-10T20:35:55.945789+0000","last_fullsized":"2023-10-10T20:35:55.945789+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:17:24.929284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.935875+0000","last_change":"2023-10-10T20:35:54.947183+0000","last_active":"2023-10-10T20:35:55.935875+0000","last_peered":"2023-10-10T20:35:55.935875+0000","last_clean":"2023-10-10T20:35:55.935875+0000","last_became_active":"2023-10-10T20:35:54.946917+0000","last_became_peered":"2023-10-10T20:35:54.946917+0000","last_unstale":"2023-10-10T20:35:55.935875+0000","last_undegraded":"2023-10-10T20:35:55.935875+0000","last_fullsized":"2023-10-10T20:35:55.935875+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:18:39.785303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.956270+0000","last_change":"2023-10-10T20:35:55.951094+0000","last_active":"2023-10-10T20:35:55.956270+0000","last_peered":"2023-10-10T20:35:55.956270+0000","last_clean":"2023-10-10T20:35:55.956270+0000","last_became_active":"2023-10-10T20:35:55.950340+0000","last_became_peered":"2023-10-10T20:35:55.950340+0000","last_unstale":"2023-10-10T20:35:55.956270+0000","last_undegraded":"2023-10-10T20:35:55.956270+0000","last_fullsized":"2023-10-10T20:35:55.956270+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:28:25.423217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"49'8","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.958388+0000","last_change":"2023-10-10T20:35:54.944176+0000","last_active":"2023-10-10T20:35:55.958388+0000","last_peered":"2023-10-10T20:35:55.958388+0000","last_clean":"2023-10-10T20:35:55.958388+0000","last_became_active":"2023-10-10T20:35:54.942915+0000","last_became_peered":"2023-10-10T20:35:54.942915+0000","last_unstale":"2023-10-10T20:35:55.958388+0000","last_undegraded":"2023-10-10T20:35:55.958388+0000","last_fullsized":"2023-10-10T20:35:55.958388+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:27:10.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.946158+0000","last_change":"2023-10-10T20:35:55.944174+0000","last_active":"2023-10-10T20:35:55.946158+0000","last_peered":"2023-10-10T20:35:55.946158+0000","last_clean":"2023-10-10T20:35:55.946158+0000","last_became_active":"2023-10-10T20:35:55.943888+0000","last_became_peered":"2023-10-10T20:35:55.943888+0000","last_unstale":"2023-10-10T20:35:55.946158+0000","last_undegraded":"2023-10-10T20:35:55.946158+0000","last_fullsized":"2023-10-10T20:35:55.946158+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:21:32.364960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"49'1","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.947596+0000","last_change":"2023-10-10T20:35:54.945585+0000","last_active":"2023-10-10T20:35:55.947596+0000","last_peered":"2023-10-10T20:35:55.947596+0000","last_clean":"2023-10-10T20:35:55.947596+0000","last_became_active":"2023-10-10T20:35:54.945337+0000","last_became_peered":"2023-10-10T20:35:54.945337+0000","last_unstale":"2023-10-10T20:35:55.947596+0000","last_undegraded":"2023-10-10T20:35:55.947596+0000","last_fullsized":"2023-10-10T20:35:55.947596+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:50.047559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961481+0000","last_change":"2023-10-10T20:35:55.960033+0000","last_active":"2023-10-10T20:35:55.961481+0000","last_peered":"2023-10-10T20:35:55.961481+0000","last_clean":"2023-10-10T20:35:55.961481+0000","last_became_active":"2023-10-10T20:35:55.959771+0000","last_became_peered":"2023-10-10T20:35:55.959771+0000","last_unstale":"2023-10-10T20:35:55.961481+0000","last_undegraded":"2023-10-10T20:35:55.961481+0000","last_fullsized":"2023-10-10T20:35:55.961481+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:28:25.423217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"49'2","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960103+0000","last_change":"2023-10-10T20:35:54.941794+0000","last_active":"2023-10-10T20:35:55.960103+0000","last_peered":"2023-10-10T20:35:55.960103+0000","last_clean":"2023-10-10T20:35:55.960103+0000","last_became_active":"2023-10-10T20:35:54.941553+0000","last_became_peered":"2023-10-10T20:35:54.941553+0000","last_unstale":"2023-10-10T20:35:55.960103+0000","last_undegraded":"2023-10-10T20:35:55.960103+0000","last_fullsized":"2023-10-10T20:35:55.960103+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:41:42.393651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.954562+0000","last_change":"2023-10-10T20:35:55.953629+0000","last_active":"2023-10-10T20:35:55.954562+0000","last_peered":"2023-10-10T20:35:55.954562+0000","last_clean":"2023-10-10T20:35:55.954562+0000","last_became_active":"2023-10-10T20:35:55.953250+0000","last_became_peered":"2023-10-10T20:35:55.953250+0000","last_unstale":"2023-10-10T20:35:55.954562+0000","last_undegraded":"2023-10-10T20:35:55.954562+0000","last_fullsized":"2023-10-10T20:35:55.954562+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:55:41.128475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"49'1","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.956667+0000","last_change":"2023-10-10T20:35:54.946161+0000","last_active":"2023-10-10T20:35:55.956667+0000","last_peered":"2023-10-10T20:35:55.956667+0000","last_clean":"2023-10-10T20:35:55.956667+0000","last_became_active":"2023-10-10T20:35:54.945889+0000","last_became_peered":"2023-10-10T20:35:54.945889+0000","last_unstale":"2023-10-10T20:35:55.956667+0000","last_undegraded":"2023-10-10T20:35:55.956667+0000","last_fullsized":"2023-10-10T20:35:55.956667+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:53:54.456688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960136+0000","last_change":"2023-10-10T20:35:55.957782+0000","last_active":"2023-10-10T20:35:55.960136+0000","last_peered":"2023-10-10T20:35:55.960136+0000","last_clean":"2023-10-10T20:35:55.960136+0000","last_became_active":"2023-10-10T20:35:55.957215+0000","last_became_peered":"2023-10-10T20:35:55.957215+0000","last_unstale":"2023-10-10T20:35:55.960136+0000","last_undegraded":"2023-10-10T20:35:55.960136+0000","last_fullsized":"2023-10-10T20:35:55.960136+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:33:38.647459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.937890+0000","last_change":"2023-10-10T20:35:54.946769+0000","last_active":"2023-10-10T20:35:55.937890+0000","last_peered":"2023-10-10T20:35:55.937890+0000","last_clean":"2023-10-10T20:35:55.937890+0000","last_became_active":"2023-10-10T20:35:54.946534+0000","last_became_peered":"2023-10-10T20:35:54.946534+0000","last_unstale":"2023-10-10T20:35:55.937890+0000","last_undegraded":"2023-10-10T20:35:55.937890+0000","last_fullsized":"2023-10-10T20:35:55.937890+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:25:13.693624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960107+0000","last_change":"2023-10-10T20:35:55.957781+0000","last_active":"2023-10-10T20:35:55.960107+0000","last_peered":"2023-10-10T20:35:55.960107+0000","last_clean":"2023-10-10T20:35:55.960107+0000","last_became_active":"2023-10-10T20:35:55.957533+0000","last_became_peered":"2023-10-10T20:35:55.957533+0000","last_unstale":"2023-10-10T20:35:55.960107+0000","last_undegraded":"2023-10-10T20:35:55.960107+0000","last_fullsized":"2023-10-10T20:35:55.960107+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:02.640005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"49'2","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960608+0000","last_change":"2023-10-10T20:35:54.945381+0000","last_active":"2023-10-10T20:35:55.960608+0000","last_peered":"2023-10-10T20:35:55.960608+0000","last_clean":"2023-10-10T20:35:55.960608+0000","last_became_active":"2023-10-10T20:35:54.943618+0000","last_became_peered":"2023-10-10T20:35:54.943618+0000","last_unstale":"2023-10-10T20:35:55.960608+0000","last_undegraded":"2023-10-10T20:35:55.960608+0000","last_fullsized":"2023-10-10T20:35:55.960608+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:23:29.829844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960072+0000","last_change":"2023-10-10T20:35:55.946118+0000","last_active":"2023-10-10T20:35:55.960072+0000","last_peered":"2023-10-10T20:35:55.960072+0000","last_clean":"2023-10-10T20:35:55.960072+0000","last_became_active":"2023-10-10T20:35:55.945858+0000","last_became_peered":"2023-10-10T20:35:55.945858+0000","last_unstale":"2023-10-10T20:35:55.960072+0000","last_undegraded":"2023-10-10T20:35:55.960072+0000","last_fullsized":"2023-10-10T20:35:55.960072+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:48:04.973531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.945437+0000","last_change":"2023-10-10T20:35:54.932698+0000","last_active":"2023-10-10T20:35:55.945437+0000","last_peered":"2023-10-10T20:35:55.945437+0000","last_clean":"2023-10-10T20:35:55.945437+0000","last_became_active":"2023-10-10T20:35:54.932065+0000","last_became_peered":"2023-10-10T20:35:54.932065+0000","last_unstale":"2023-10-10T20:35:55.945437+0000","last_undegraded":"2023-10-10T20:35:55.945437+0000","last_fullsized":"2023-10-10T20:35:55.945437+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:00:25.701036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960214+0000","last_change":"2023-10-10T20:35:55.946535+0000","last_active":"2023-10-10T20:35:55.960214+0000","last_peered":"2023-10-10T20:35:55.960214+0000","last_clean":"2023-10-10T20:35:55.960214+0000","last_became_active":"2023-10-10T20:35:55.946239+0000","last_became_peered":"2023-10-10T20:35:55.946239+0000","last_unstale":"2023-10-10T20:35:55.960214+0000","last_undegraded":"2023-10-10T20:35:55.960214+0000","last_fullsized":"2023-10-10T20:35:55.960214+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:18:40.792158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.935956+0000","last_change":"2023-10-10T20:35:54.946856+0000","last_active":"2023-10-10T20:35:55.935956+0000","last_peered":"2023-10-10T20:35:55.935956+0000","last_clean":"2023-10-10T20:35:55.935956+0000","last_became_active":"2023-10-10T20:35:54.946598+0000","last_became_peered":"2023-10-10T20:35:54.946598+0000","last_unstale":"2023-10-10T20:35:55.935956+0000","last_undegraded":"2023-10-10T20:35:55.935956+0000","last_fullsized":"2023-10-10T20:35:55.935956+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:17:23.922429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.950156+0000","last_change":"2023-10-10T20:35:55.947319+0000","last_active":"2023-10-10T20:35:55.950156+0000","last_peered":"2023-10-10T20:35:55.950156+0000","last_clean":"2023-10-10T20:35:55.950156+0000","last_became_active":"2023-10-10T20:35:55.947063+0000","last_became_peered":"2023-10-10T20:35:55.947063+0000","last_unstale":"2023-10-10T20:35:55.950156+0000","last_undegraded":"2023-10-10T20:35:55.950156+0000","last_fullsized":"2023-10-10T20:35:55.950156+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:33:38.647459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"49'2","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961970+0000","last_change":"2023-10-10T20:35:54.945967+0000","last_active":"2023-10-10T20:35:55.961970+0000","last_peered":"2023-10-10T20:35:55.961970+0000","last_clean":"2023-10-10T20:35:55.961970+0000","last_became_active":"2023-10-10T20:35:54.945703+0000","last_became_peered":"2023-10-10T20:35:54.945703+0000","last_unstale":"2023-10-10T20:35:55.961970+0000","last_undegraded":"2023-10-10T20:35:55.961970+0000","last_fullsized":"2023-10-10T20:35:55.961970+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:12:41.189930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.950190+0000","last_change":"2023-10-10T20:35:55.947138+0000","last_active":"2023-10-10T20:35:55.950190+0000","last_peered":"2023-10-10T20:35:55.950190+0000","last_clean":"2023-10-10T20:35:55.950190+0000","last_became_active":"2023-10-10T20:35:55.946859+0000","last_became_peered":"2023-10-10T20:35:55.946859+0000","last_unstale":"2023-10-10T20:35:55.950190+0000","last_undegraded":"2023-10-10T20:35:55.950190+0000","last_fullsized":"2023-10-10T20:35:55.950190+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:21:32.364960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.936901+0000","last_change":"2023-10-10T20:35:54.941680+0000","last_active":"2023-10-10T20:35:55.936901+0000","last_peered":"2023-10-10T20:35:55.936901+0000","last_clean":"2023-10-10T20:35:55.936901+0000","last_became_active":"2023-10-10T20:35:54.941440+0000","last_became_peered":"2023-10-10T20:35:54.941440+0000","last_unstale":"2023-10-10T20:35:55.936901+0000","last_undegraded":"2023-10-10T20:35:55.936901+0000","last_fullsized":"2023-10-10T20:35:55.936901+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:21.101690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.957958+0000","last_change":"2023-10-10T20:35:55.950744+0000","last_active":"2023-10-10T20:35:55.957958+0000","last_peered":"2023-10-10T20:35:55.957958+0000","last_clean":"2023-10-10T20:35:55.957958+0000","last_became_active":"2023-10-10T20:35:55.950509+0000","last_became_peered":"2023-10-10T20:35:55.950509+0000","last_unstale":"2023-10-10T20:35:55.957958+0000","last_undegraded":"2023-10-10T20:35:55.957958+0000","last_fullsized":"2023-10-10T20:35:55.957958+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:56.046059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.939675+0000","last_change":"2023-10-10T20:35:54.940210+0000","last_active":"2023-10-10T20:35:55.939675+0000","last_peered":"2023-10-10T20:35:55.939675+0000","last_clean":"2023-10-10T20:35:55.939675+0000","last_became_active":"2023-10-10T20:35:54.939930+0000","last_became_peered":"2023-10-10T20:35:54.939930+0000","last_unstale":"2023-10-10T20:35:55.939675+0000","last_undegraded":"2023-10-10T20:35:55.939675+0000","last_fullsized":"2023-10-10T20:35:55.939675+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:25.718980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961586+0000","last_change":"2023-10-10T20:35:55.957568+0000","last_active":"2023-10-10T20:35:55.961586+0000","last_peered":"2023-10-10T20:35:55.961586+0000","last_clean":"2023-10-10T20:35:55.961586+0000","last_became_active":"2023-10-10T20:35:55.957222+0000","last_became_peered":"2023-10-10T20:35:55.957222+0000","last_unstale":"2023-10-10T20:35:55.961586+0000","last_undegraded":"2023-10-10T20:35:55.961586+0000","last_fullsized":"2023-10-10T20:35:55.961586+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:55.043141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.936606+0000","last_change":"2023-10-10T20:35:54.942350+0000","last_active":"2023-10-10T20:35:55.936606+0000","last_peered":"2023-10-10T20:35:55.936606+0000","last_clean":"2023-10-10T20:35:55.936606+0000","last_became_active":"2023-10-10T20:35:54.942082+0000","last_became_peered":"2023-10-10T20:35:54.942082+0000","last_unstale":"2023-10-10T20:35:55.936606+0000","last_undegraded":"2023-10-10T20:35:55.936606+0000","last_fullsized":"2023-10-10T20:35:55.936606+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:30:14.385873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960745+0000","last_change":"2023-10-10T20:35:55.959128+0000","last_active":"2023-10-10T20:35:55.960745+0000","last_peered":"2023-10-10T20:35:55.960745+0000","last_clean":"2023-10-10T20:35:55.960745+0000","last_became_active":"2023-10-10T20:35:55.958877+0000","last_became_peered":"2023-10-10T20:35:55.958877+0000","last_unstale":"2023-10-10T20:35:55.960745+0000","last_undegraded":"2023-10-10T20:35:55.960745+0000","last_fullsized":"2023-10-10T20:35:55.960745+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:38:30.688214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.945639+0000","last_change":"2023-10-10T20:35:54.940768+0000","last_active":"2023-10-10T20:35:55.945639+0000","last_peered":"2023-10-10T20:35:55.945639+0000","last_clean":"2023-10-10T20:35:55.945639+0000","last_became_active":"2023-10-10T20:35:54.940554+0000","last_became_peered":"2023-10-10T20:35:54.940554+0000","last_unstale":"2023-10-10T20:35:55.945639+0000","last_undegraded":"2023-10-10T20:35:55.945639+0000","last_fullsized":"2023-10-10T20:35:55.945639+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:48.499861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.953982+0000","last_change":"2023-10-10T20:35:55.950625+0000","last_active":"2023-10-10T20:35:55.953982+0000","last_peered":"2023-10-10T20:35:55.953982+0000","last_clean":"2023-10-10T20:35:55.953982+0000","last_became_active":"2023-10-10T20:35:55.950302+0000","last_became_peered":"2023-10-10T20:35:55.950302+0000","last_unstale":"2023-10-10T20:35:55.953982+0000","last_undegraded":"2023-10-10T20:35:55.953982+0000","last_fullsized":"2023-10-10T20:35:55.953982+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:10:25.924046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.939832+0000","last_change":"2023-10-10T20:35:54.943152+0000","last_active":"2023-10-10T20:35:55.939832+0000","last_peered":"2023-10-10T20:35:55.939832+0000","last_clean":"2023-10-10T20:35:55.939832+0000","last_became_active":"2023-10-10T20:35:54.942861+0000","last_became_peered":"2023-10-10T20:35:54.942861+0000","last_unstale":"2023-10-10T20:35:55.939832+0000","last_undegraded":"2023-10-10T20:35:55.939832+0000","last_fullsized":"2023-10-10T20:35:55.939832+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:48.499861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.957888+0000","last_change":"2023-10-10T20:35:55.950195+0000","last_active":"2023-10-10T20:35:55.957888+0000","last_peered":"2023-10-10T20:35:55.957888+0000","last_clean":"2023-10-10T20:35:55.957888+0000","last_became_active":"2023-10-10T20:35:55.949913+0000","last_became_peered":"2023-10-10T20:35:55.949913+0000","last_unstale":"2023-10-10T20:35:55.957888+0000","last_undegraded":"2023-10-10T20:35:55.957888+0000","last_fullsized":"2023-10-10T20:35:55.957888+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:24:19.034235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.945550+0000","last_change":"2023-10-10T20:35:54.938848+0000","last_active":"2023-10-10T20:35:55.945550+0000","last_peered":"2023-10-10T20:35:55.945550+0000","last_clean":"2023-10-10T20:35:55.945550+0000","last_became_active":"2023-10-10T20:35:54.938610+0000","last_became_peered":"2023-10-10T20:35:54.938610+0000","last_unstale":"2023-10-10T20:35:55.945550+0000","last_undegraded":"2023-10-10T20:35:55.945550+0000","last_fullsized":"2023-10-10T20:35:55.945550+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:29:15.532075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.964363+0000","last_change":"2023-10-10T20:35:55.962459+0000","last_active":"2023-10-10T20:35:55.964363+0000","last_peered":"2023-10-10T20:35:55.964363+0000","last_clean":"2023-10-10T20:35:55.964363+0000","last_became_active":"2023-10-10T20:35:55.962220+0000","last_became_peered":"2023-10-10T20:35:55.962220+0000","last_unstale":"2023-10-10T20:35:55.964363+0000","last_undegraded":"2023-10-10T20:35:55.964363+0000","last_fullsized":"2023-10-10T20:35:55.964363+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:50:01.601004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.938642+0000","last_change":"2023-10-10T20:35:54.946647+0000","last_active":"2023-10-10T20:35:55.938642+0000","last_peered":"2023-10-10T20:35:55.938642+0000","last_clean":"2023-10-10T20:35:55.938642+0000","last_became_active":"2023-10-10T20:35:54.946417+0000","last_became_peered":"2023-10-10T20:35:54.946417+0000","last_unstale":"2023-10-10T20:35:55.938642+0000","last_undegraded":"2023-10-10T20:35:55.938642+0000","last_fullsized":"2023-10-10T20:35:55.938642+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:33:37.640604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.964331+0000","last_change":"2023-10-10T20:35:55.961861+0000","last_active":"2023-10-10T20:35:55.964331+0000","last_peered":"2023-10-10T20:35:55.964331+0000","last_clean":"2023-10-10T20:35:55.964331+0000","last_became_active":"2023-10-10T20:35:55.961005+0000","last_became_peered":"2023-10-10T20:35:55.961005+0000","last_unstale":"2023-10-10T20:35:55.964331+0000","last_undegraded":"2023-10-10T20:35:55.964331+0000","last_fullsized":"2023-10-10T20:35:55.964331+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:24:17.176188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.939556+0000","last_change":"2023-10-10T20:35:54.941602+0000","last_active":"2023-10-10T20:35:55.939556+0000","last_peered":"2023-10-10T20:35:55.939556+0000","last_clean":"2023-10-10T20:35:55.939556+0000","last_became_active":"2023-10-10T20:35:54.941344+0000","last_became_peered":"2023-10-10T20:35:54.941344+0000","last_unstale":"2023-10-10T20:35:55.939556+0000","last_undegraded":"2023-10-10T20:35:55.939556+0000","last_fullsized":"2023-10-10T20:35:55.939556+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:56:57.311897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.932286+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:35:55.932286+0000","last_peered":"2023-10-10T20:35:55.932286+0000","last_clean":"2023-10-10T20:35:55.932286+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:35:55.932286+0000","last_undegraded":"2023-10-10T20:35:55.932286+0000","last_fullsized":"2023-10-10T20:35:55.932286+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298460,"kb_used_data":1172,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073380,"statfs":{"total":96636764160,"available":96331141120,"internally_reserved":0,"allocated":1200128,"data_stored":842569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":6.8630000000000004,"5min":6.8630000000000004,"15min":6.8630000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":4.1959999999999997,"5min":4.1959999999999997,"15min":4.1959999999999997},"last":0.59499999999999997}]},{"osd":1,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":10.1,"5min":10.1,"15min":10.1},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":6.806,"5min":6.806,"15min":6.806},"last":0.72199999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":6.9340000000000002,"5min":6.9340000000000002,"15min":6.9340000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":6.9870000000000001,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.60899999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":6.6719999999999997,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":4.1609999999999996,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.64900000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.48799999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.46999999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.436}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297872,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073968,"statfs":{"total":96636764160,"available":96331743232,"internally_reserved":0,"allocated":598016,"data_stored":252041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":7.7590000000000003,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":7.8090000000000002,"5min":7.8090000000000002,"15min":7.8090000000000002},"last":0.55500000000000005}]},{"osd":1,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":7.867,"5min":7.867,"15min":7.867},"last":0.378},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":9.7089999999999996,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.498}]},{"osd":2,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":4.7999999999999998,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":6.7629999999999999,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":0.54000000000000004}]},{"osd":3,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":4.8620000000000001,"5min":4.8620000000000001,"15min":4.8620000000000001},"last":0.433},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":7.9009999999999998,"5min":7.9009999999999998,"15min":7.9009999999999998},"last":0.40500000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.58199999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.624}]},{"osd":7,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.46200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297876,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073964,"statfs":{"total":96636764160,"available":96331739136,"internally_reserved":0,"allocated":602112,"data_stored":252131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":5.0380000000000003,"5min":5.0380000000000003,"15min":5.0380000000000003},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.26700000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.25},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":5.0919999999999996,"5min":5.0919999999999996,"15min":5.0919999999999996},"last":0.28599999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.1159999999999997,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.309}]},{"osd":4,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":4.9850000000000003,"5min":4.9850000000000003,"15min":4.9850000000000003},"last":0.51700000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.42599999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":5.0659999999999998,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.50700000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0499999999999998,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.47599999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298464,"kb_used_data":1176,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073376,"statfs":{"total":96636764160,"available":96331137024,"internally_reserved":0,"allocated":1204224,"data_stored":843538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":7.9249999999999998,"5min":7.9249999999999998,"15min":7.9249999999999998},"last":0.245},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":6.6299999999999999,"5min":6.6299999999999999,"15min":6.6299999999999999},"last":0.39900000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":8.2509999999999994,"5min":8.2509999999999994,"15min":8.2509999999999994},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":6.7370000000000001,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.36899999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":7.7960000000000003,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":7.8929999999999998,"5min":7.8929999999999998,"15min":7.8929999999999998},"last":0.35099999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":6.6929999999999996,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":7.6920000000000002,"5min":7.6920000000000002,"15min":7.6920000000000002},"last":0.56699999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":7.7690000000000001,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.8659999999999997,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":0.61699999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":7.8360000000000003,"5min":7.8360000000000003,"15min":7.8360000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":6.6660000000000004,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.65600000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":6.7160000000000002,"5min":6.7160000000000002,"15min":6.7160000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":7.7370000000000001,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.54600000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297880,"kb_used_data":592,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073960,"statfs":{"total":96636764160,"available":96331735040,"internally_reserved":0,"allocated":606208,"data_stored":252097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.38300000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":11.967000000000001,"5min":11.967000000000001,"15min":11.967000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.1100000000000003,"5min":4.1100000000000003,"15min":4.1100000000000003},"last":0.41899999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":6.3040000000000003,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":11.935,"5min":11.935,"15min":11.935},"last":0.438}]},{"osd":4,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.71899999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.748},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57099999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.68799999999999994}]},{"osd":7,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":6.2649999999999997,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.76400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297880,"kb_used_data":592,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073960,"statfs":{"total":96636764160,"available":96331735040,"internally_reserved":0,"allocated":606208,"data_stored":252097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.317,"15min":0.317},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.436,"5min":0.436,"15min":0.436},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.318}]},{"osd":1,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.38900000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.34499999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.58499999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.57599999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.71599999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.56699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297884,"kb_used_data":596,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073956,"statfs":{"total":96636764160,"available":96331730944,"internally_reserved":0,"allocated":610304,"data_stored":253577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":7.6719999999999997,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.52400000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":7.7160000000000002,"5min":7.7160000000000002,"15min":7.7160000000000002},"last":0.625},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":7.7839999999999998,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.72499999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.68000000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.75}]},{"osd":6,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.83299999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.85699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297880,"kb_used_data":592,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073960,"statfs":{"total":96636764160,"available":96331735040,"internally_reserved":0,"allocated":606208,"data_stored":252665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.441},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.56299999999999994}]},{"osd":1,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":5.2679999999999998,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":5.3109999999999999,"5min":5.3109999999999999,"15min":5.3109999999999999},"last":0.52200000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.2960000000000003,"5min":5.2960000000000003,"15min":5.2960000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.504}]},{"osd":3,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":5.3330000000000002,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.2850000000000001,"5min":5.2850000000000001,"15min":5.2850000000000001},"last":0.57599999999999996}]},{"osd":4,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.68300000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.69399999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.71999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:36:00.005 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:36:00.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.151+0000 7fd40f442700 1 Processor -- start 2023-10-10T20:36:00.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.151+0000 7fd40f442700 1 -- start start 2023-10-10T20:36:00.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.152+0000 7fd40f442700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd408159890 0x7fd40815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.152+0000 7fd40f442700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd408069560 0x7fd4080699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.152+0000 7fd40f442700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd408069ee0 con 0x7fd408159c60 2023-10-10T20:36:00.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.152+0000 7fd40f442700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd40815a160 con 0x7fd408069560 2023-10-10T20:36:00.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.152+0000 7fd40f442700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4080d13a0 con 0x7fd408159890 2023-10-10T20:36:00.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.152+0000 7fd40d9df700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd408069560 0x7fd4080699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:00.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.152+0000 7fd40d9df700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd408069560 0x7fd4080699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41008/0 (socket says 172.21.15.89:41008) 2023-10-10T20:36:00.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.152+0000 7fd40d9df700 1 -- 172.21.15.89:0/3694504927 learned_addr learned my addr 172.21.15.89:0/3694504927 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:00.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.152+0000 7fd40d1de700 1 --2- 172.21.15.89:0/3694504927 >> v2:172.21.15.111:3300/0 conn(0x7fd408159890 0x7fd40815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:00.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.153+0000 7fd40d9df700 1 -- 172.21.15.89:0/3694504927 >> v2:172.21.15.111:3300/0 conn(0x7fd408159890 msgr2=0x7fd40815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:00.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.153+0000 7fd40d9df700 1 --2- 172.21.15.89:0/3694504927 >> v2:172.21.15.111:3300/0 conn(0x7fd408159890 0x7fd40815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.153+0000 7fd40d9df700 1 -- 172.21.15.89:0/3694504927 >> v1:172.21.15.111:6789/0 conn(0x7fd408159c60 legacy=0x7fd40815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:00.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.153+0000 7fd40d9df700 1 -- 172.21.15.89:0/3694504927 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40815e140 con 0x7fd408069560 2023-10-10T20:36:00.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.153+0000 7fd40d9df700 1 --2- 172.21.15.89:0/3694504927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd408069560 0x7fd4080699a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd4080d1090 tx=0x7fd400009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ccac79776c29441 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:00.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.153+0000 7fd3fffff700 1 -- 172.21.15.89:0/3694504927 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd400009e70 con 0x7fd408069560 2023-10-10T20:36:00.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.153+0000 7fd3fffff700 1 -- 172.21.15.89:0/3694504927 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40000b860 con 0x7fd408069560 2023-10-10T20:36:00.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.153+0000 7fd3fffff700 1 -- 172.21.15.89:0/3694504927 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40000bc00 con 0x7fd408069560 2023-10-10T20:36:00.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.153+0000 7fd40f442700 1 -- 172.21.15.89:0/3694504927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd408069560 msgr2=0x7fd4080699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:00.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.153+0000 7fd40f442700 1 --2- 172.21.15.89:0/3694504927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd408069560 0x7fd4080699a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd4080d1090 tx=0x7fd400009250 comp rx=0 tx=0).stop 2023-10-10T20:36:00.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.153+0000 7fd40f442700 1 -- 172.21.15.89:0/3694504927 shutdown_connections 2023-10-10T20:36:00.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.153+0000 7fd40f442700 1 --2- 172.21.15.89:0/3694504927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd408069560 0x7fd4080699a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.153+0000 7fd40f442700 1 --2- 172.21.15.89:0/3694504927 >> v2:172.21.15.111:3300/0 conn(0x7fd408159890 0x7fd40815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.153+0000 7fd40f442700 1 -- 172.21.15.89:0/3694504927 >> 172.21.15.89:0/3694504927 conn(0x7fd408068e20 msgr2=0x7fd4080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:00.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.154+0000 7fd40f442700 1 -- 172.21.15.89:0/3694504927 shutdown_connections 2023-10-10T20:36:00.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.154+0000 7fd40f442700 1 -- 172.21.15.89:0/3694504927 wait complete. 2023-10-10T20:36:00.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.155+0000 7fd40f442700 1 Processor -- start 2023-10-10T20:36:00.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.155+0000 7fd40f442700 1 -- start start 2023-10-10T20:36:00.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.155+0000 7fd40f442700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd408069560 0x7fd4080b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd40f442700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd408159890 0x7fd4080b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd40f442700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd408159c60 0x7fd4080c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd40f442700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4080b6270 con 0x7fd408069560 2023-10-10T20:36:00.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd40f442700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40816d270 con 0x7fd408159c60 2023-10-10T20:36:00.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd40f442700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40816d550 con 0x7fd408159890 2023-10-10T20:36:00.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd40d1de700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd408069560 0x7fd4080b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:00.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd40d1de700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd408069560 0x7fd4080b5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41024/0 (socket says 172.21.15.89:41024) 2023-10-10T20:36:00.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd40d1de700 1 -- 172.21.15.89:0/918554903 learned_addr learned my addr 172.21.15.89:0/918554903 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:00.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd40d9df700 1 --2- 172.21.15.89:0/918554903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd408159c60 0x7fd4080c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:00.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd40c9dd700 1 --2- 172.21.15.89:0/918554903 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd408159890 0x7fd4080b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:00.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd40d1de700 1 -- 172.21.15.89:0/918554903 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd408159890 msgr2=0x7fd4080b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:00.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd40d1de700 1 --2- 172.21.15.89:0/918554903 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd408159890 0x7fd4080b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd40d1de700 1 -- 172.21.15.89:0/918554903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd408159c60 msgr2=0x7fd4080c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:00.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd40d1de700 1 --2- 172.21.15.89:0/918554903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd408159c60 0x7fd4080c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd40d1de700 1 -- 172.21.15.89:0/918554903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4000094f0 con 0x7fd408069560 2023-10-10T20:36:00.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd40d9df700 1 --2- 172.21.15.89:0/918554903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd408159c60 0x7fd4080c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:00.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd40d1de700 1 --2- 172.21.15.89:0/918554903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd408069560 0x7fd4080b5bf0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd404009ef0 tx=0x7fd40400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:00.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd3fd7fa700 1 -- 172.21.15.89:0/918554903 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40400cc50 con 0x7fd408069560 2023-10-10T20:36:00.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd3fd7fa700 1 -- 172.21.15.89:0/918554903 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40400f040 con 0x7fd408069560 2023-10-10T20:36:00.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd40f442700 1 -- 172.21.15.89:0/918554903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40816d830 con 0x7fd408069560 2023-10-10T20:36:00.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd3fd7fa700 1 -- 172.21.15.89:0/918554903 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404014480 con 0x7fd408069560 2023-10-10T20:36:00.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.156+0000 7fd40f442700 1 -- 172.21.15.89:0/918554903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4080b46c0 con 0x7fd408069560 2023-10-10T20:36:00.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.157+0000 7fd40f442700 1 -- 172.21.15.89:0/918554903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40816de80 con 0x7fd408069560 2023-10-10T20:36:00.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.157+0000 7fd3fd7fa700 1 -- 172.21.15.89:0/918554903 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd4040147f0 con 0x7fd408069560 2023-10-10T20:36:00.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.158+0000 7fd3fd7fa700 1 --2- 172.21.15.89:0/918554903 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd3ec066f30 0x7fd3ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.158+0000 7fd3fd7fa700 1 -- 172.21.15.89:0/918554903 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd404084fe0 con 0x7fd408069560 2023-10-10T20:36:00.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.158+0000 7fd40c9dd700 1 --2- 172.21.15.89:0/918554903 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd3ec066f30 0x7fd3ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:00.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.158+0000 7fd40c9dd700 1 --2- 172.21.15.89:0/918554903 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd3ec066f30 0x7fd3ec0693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd408153660 tx=0x7fd3f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:00.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.162+0000 7fd3fd7fa700 1 -- 172.21.15.89:0/918554903 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd404052250 con 0x7fd408069560 2023-10-10T20:36:00.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.342+0000 7fd40f442700 1 -- 172.21.15.89:0/918554903 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd40804c3c0 con 0x7fd3ec066f30 2023-10-10T20:36:00.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.350+0000 7fd3fd7fa700 1 -- 172.21.15.89:0/918554903 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289448 (secure 0 0 0) 0x7fd40804c3c0 con 0x7fd3ec066f30 2023-10-10T20:36:00.355 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:00.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.353+0000 7fd40f442700 1 -- 172.21.15.89:0/918554903 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd3ec066f30 msgr2=0x7fd3ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:00.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.353+0000 7fd40f442700 1 --2- 172.21.15.89:0/918554903 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd3ec066f30 0x7fd3ec0693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd408153660 tx=0x7fd3f8009040 comp rx=0 tx=0).stop 2023-10-10T20:36:00.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.353+0000 7fd40f442700 1 -- 172.21.15.89:0/918554903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd408069560 msgr2=0x7fd4080b5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.353+0000 7fd40f442700 1 --2- 172.21.15.89:0/918554903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd408069560 0x7fd4080b5bf0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd404009ef0 tx=0x7fd40400b040 comp rx=0 tx=0).stop 2023-10-10T20:36:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.353+0000 7fd40f442700 1 -- 172.21.15.89:0/918554903 shutdown_connections 2023-10-10T20:36:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.353+0000 7fd40f442700 1 --2- 172.21.15.89:0/918554903 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd3ec066f30 0x7fd3ec0693e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.353+0000 7fd40f442700 1 --2- 172.21.15.89:0/918554903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd408159c60 0x7fd4080c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.353+0000 7fd40f442700 1 --2- 172.21.15.89:0/918554903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd408069560 0x7fd4080b5bf0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.353+0000 7fd40f442700 1 --2- 172.21.15.89:0/918554903 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd408159890 0x7fd4080b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.353+0000 7fd40f442700 1 -- 172.21.15.89:0/918554903 >> 172.21.15.89:0/918554903 conn(0x7fd408068e20 msgr2=0x7fd40814d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.353+0000 7fd40f442700 1 -- 172.21.15.89:0/918554903 shutdown_connections 2023-10-10T20:36:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.353+0000 7fd40f442700 1 -- 172.21.15.89:0/918554903 wait complete. 2023-10-10T20:36:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:36:00.367 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2023-10-10T20:35:58.667533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2384196,"kb_used_data":5892,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752590524,"statfs":{"total":773094113280,"available":770652696576,"internally_reserved":0,"allocated":6033408,"data_stored":3200715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.746916"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.949172+0000","last_change":"2023-10-10T20:35:55.947951+0000","last_active":"2023-10-10T20:35:55.949172+0000","last_peered":"2023-10-10T20:35:55.949172+0000","last_clean":"2023-10-10T20:35:55.949172+0000","last_became_active":"2023-10-10T20:35:55.947721+0000","last_became_peered":"2023-10-10T20:35:55.947721+0000","last_unstale":"2023-10-10T20:35:55.949172+0000","last_undegraded":"2023-10-10T20:35:55.949172+0000","last_fullsized":"2023-10-10T20:35:55.949172+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:26:38.266002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.954592+0000","last_change":"2023-10-10T20:35:55.953795+0000","last_active":"2023-10-10T20:35:55.954592+0000","last_peered":"2023-10-10T20:35:55.954592+0000","last_clean":"2023-10-10T20:35:55.954592+0000","last_became_active":"2023-10-10T20:35:55.953541+0000","last_became_peered":"2023-10-10T20:35:55.953541+0000","last_unstale":"2023-10-10T20:35:55.954592+0000","last_undegraded":"2023-10-10T20:35:55.954592+0000","last_fullsized":"2023-10-10T20:35:55.954592+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:26:57.532237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.949272+0000","last_change":"2023-10-10T20:35:55.945095+0000","last_active":"2023-10-10T20:35:55.949272+0000","last_peered":"2023-10-10T20:35:55.949272+0000","last_clean":"2023-10-10T20:35:55.949272+0000","last_became_active":"2023-10-10T20:35:55.944559+0000","last_became_peered":"2023-10-10T20:35:55.944559+0000","last_unstale":"2023-10-10T20:35:55.949272+0000","last_undegraded":"2023-10-10T20:35:55.949272+0000","last_fullsized":"2023-10-10T20:35:55.949272+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:20:10.900594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.954258+0000","last_change":"2023-10-10T20:35:55.952175+0000","last_active":"2023-10-10T20:35:55.954258+0000","last_peered":"2023-10-10T20:35:55.954258+0000","last_clean":"2023-10-10T20:35:55.954258+0000","last_became_active":"2023-10-10T20:35:55.951606+0000","last_became_peered":"2023-10-10T20:35:55.951606+0000","last_unstale":"2023-10-10T20:35:55.954258+0000","last_undegraded":"2023-10-10T20:35:55.954258+0000","last_fullsized":"2023-10-10T20:35:55.954258+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:57:31.137405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.949226+0000","last_change":"2023-10-10T20:35:55.946843+0000","last_active":"2023-10-10T20:35:55.949226+0000","last_peered":"2023-10-10T20:35:55.949226+0000","last_clean":"2023-10-10T20:35:55.949226+0000","last_became_active":"2023-10-10T20:35:55.946597+0000","last_became_peered":"2023-10-10T20:35:55.946597+0000","last_unstale":"2023-10-10T20:35:55.949226+0000","last_undegraded":"2023-10-10T20:35:55.949226+0000","last_fullsized":"2023-10-10T20:35:55.949226+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:27:54.804150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960301+0000","last_change":"2023-10-10T20:35:55.947148+0000","last_active":"2023-10-10T20:35:55.960301+0000","last_peered":"2023-10-10T20:35:55.960301+0000","last_clean":"2023-10-10T20:35:55.960301+0000","last_became_active":"2023-10-10T20:35:55.946908+0000","last_became_peered":"2023-10-10T20:35:55.946908+0000","last_unstale":"2023-10-10T20:35:55.960301+0000","last_undegraded":"2023-10-10T20:35:55.960301+0000","last_fullsized":"2023-10-10T20:35:55.960301+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:23:30.836699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.950910+0000","last_change":"2023-10-10T20:35:55.948700+0000","last_active":"2023-10-10T20:35:55.950910+0000","last_peered":"2023-10-10T20:35:55.950910+0000","last_clean":"2023-10-10T20:35:55.950910+0000","last_became_active":"2023-10-10T20:35:55.948399+0000","last_became_peered":"2023-10-10T20:35:55.948399+0000","last_unstale":"2023-10-10T20:35:55.950910+0000","last_undegraded":"2023-10-10T20:35:55.950910+0000","last_fullsized":"2023-10-10T20:35:55.950910+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:27:11.743663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.959888+0000","last_change":"2023-10-10T20:35:55.954053+0000","last_active":"2023-10-10T20:35:55.959888+0000","last_peered":"2023-10-10T20:35:55.959888+0000","last_clean":"2023-10-10T20:35:55.959888+0000","last_became_active":"2023-10-10T20:35:55.953848+0000","last_became_peered":"2023-10-10T20:35:55.953848+0000","last_unstale":"2023-10-10T20:35:55.959888+0000","last_undegraded":"2023-10-10T20:35:55.959888+0000","last_fullsized":"2023-10-10T20:35:55.959888+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:52:47.569198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960249+0000","last_change":"2023-10-10T20:35:55.958056+0000","last_active":"2023-10-10T20:35:55.960249+0000","last_peered":"2023-10-10T20:35:55.960249+0000","last_clean":"2023-10-10T20:35:55.960249+0000","last_became_active":"2023-10-10T20:35:55.957775+0000","last_became_peered":"2023-10-10T20:35:55.957775+0000","last_unstale":"2023-10-10T20:35:55.960249+0000","last_undegraded":"2023-10-10T20:35:55.960249+0000","last_fullsized":"2023-10-10T20:35:55.960249+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:24:19.034235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.958011+0000","last_change":"2023-10-10T20:35:55.953389+0000","last_active":"2023-10-10T20:35:55.958011+0000","last_peered":"2023-10-10T20:35:55.958011+0000","last_clean":"2023-10-10T20:35:55.958011+0000","last_became_active":"2023-10-10T20:35:55.953055+0000","last_became_peered":"2023-10-10T20:35:55.953055+0000","last_unstale":"2023-10-10T20:35:55.958011+0000","last_undegraded":"2023-10-10T20:35:55.958011+0000","last_fullsized":"2023-10-10T20:35:55.958011+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:32:26.746310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.958346+0000","last_change":"2023-10-10T20:35:55.956182+0000","last_active":"2023-10-10T20:35:55.958346+0000","last_peered":"2023-10-10T20:35:55.958346+0000","last_clean":"2023-10-10T20:35:55.958346+0000","last_became_active":"2023-10-10T20:35:55.955947+0000","last_became_peered":"2023-10-10T20:35:55.955947+0000","last_unstale":"2023-10-10T20:35:55.958346+0000","last_undegraded":"2023-10-10T20:35:55.958346+0000","last_fullsized":"2023-10-10T20:35:55.958346+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:17:24.929284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.954505+0000","last_change":"2023-10-10T20:35:55.952367+0000","last_active":"2023-10-10T20:35:55.954505+0000","last_peered":"2023-10-10T20:35:55.954505+0000","last_clean":"2023-10-10T20:35:55.954505+0000","last_became_active":"2023-10-10T20:35:55.951929+0000","last_became_peered":"2023-10-10T20:35:55.951929+0000","last_unstale":"2023-10-10T20:35:55.954505+0000","last_undegraded":"2023-10-10T20:35:55.954505+0000","last_fullsized":"2023-10-10T20:35:55.954505+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:20:10.900594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961843+0000","last_change":"2023-10-10T20:35:55.960804+0000","last_active":"2023-10-10T20:35:55.961843+0000","last_peered":"2023-10-10T20:35:55.961843+0000","last_clean":"2023-10-10T20:35:55.961843+0000","last_became_active":"2023-10-10T20:35:55.960543+0000","last_became_peered":"2023-10-10T20:35:55.960543+0000","last_unstale":"2023-10-10T20:35:55.961843+0000","last_undegraded":"2023-10-10T20:35:55.961843+0000","last_fullsized":"2023-10-10T20:35:55.961843+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:29:16.538930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.946660+0000","last_change":"2023-10-10T20:35:55.944587+0000","last_active":"2023-10-10T20:35:55.946660+0000","last_peered":"2023-10-10T20:35:55.946660+0000","last_clean":"2023-10-10T20:35:55.946660+0000","last_became_active":"2023-10-10T20:35:55.944317+0000","last_became_peered":"2023-10-10T20:35:55.944317+0000","last_unstale":"2023-10-10T20:35:55.946660+0000","last_undegraded":"2023-10-10T20:35:55.946660+0000","last_fullsized":"2023-10-10T20:35:55.946660+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:32:26.746310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.954547+0000","last_change":"2023-10-10T20:35:55.953565+0000","last_active":"2023-10-10T20:35:55.954547+0000","last_peered":"2023-10-10T20:35:55.954547+0000","last_clean":"2023-10-10T20:35:55.954547+0000","last_became_active":"2023-10-10T20:35:55.953239+0000","last_became_peered":"2023-10-10T20:35:55.953239+0000","last_unstale":"2023-10-10T20:35:55.954547+0000","last_undegraded":"2023-10-10T20:35:55.954547+0000","last_fullsized":"2023-10-10T20:35:55.954547+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:56.046059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961800+0000","last_change":"2023-10-10T20:35:55.956399+0000","last_active":"2023-10-10T20:35:55.961800+0000","last_peered":"2023-10-10T20:35:55.961800+0000","last_clean":"2023-10-10T20:35:55.961800+0000","last_became_active":"2023-10-10T20:35:55.956158+0000","last_became_peered":"2023-10-10T20:35:55.956158+0000","last_unstale":"2023-10-10T20:35:55.961800+0000","last_undegraded":"2023-10-10T20:35:55.961800+0000","last_fullsized":"2023-10-10T20:35:55.961800+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:40:17.028747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.964308+0000","last_change":"2023-10-10T20:35:55.961637+0000","last_active":"2023-10-10T20:35:55.964308+0000","last_peered":"2023-10-10T20:35:55.964308+0000","last_clean":"2023-10-10T20:35:55.964308+0000","last_became_active":"2023-10-10T20:35:55.961347+0000","last_became_peered":"2023-10-10T20:35:55.961347+0000","last_unstale":"2023-10-10T20:35:55.964308+0000","last_undegraded":"2023-10-10T20:35:55.964308+0000","last_fullsized":"2023-10-10T20:35:55.964308+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:16.560154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.949316+0000","last_change":"2023-10-10T20:35:55.945438+0000","last_active":"2023-10-10T20:35:55.949316+0000","last_peered":"2023-10-10T20:35:55.949316+0000","last_clean":"2023-10-10T20:35:55.949316+0000","last_became_active":"2023-10-10T20:35:55.945124+0000","last_became_peered":"2023-10-10T20:35:55.945124+0000","last_unstale":"2023-10-10T20:35:55.949316+0000","last_undegraded":"2023-10-10T20:35:55.949316+0000","last_fullsized":"2023-10-10T20:35:55.949316+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:57:31.137405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.949200+0000","last_change":"2023-10-10T20:35:55.947979+0000","last_active":"2023-10-10T20:35:55.949200+0000","last_peered":"2023-10-10T20:35:55.949200+0000","last_clean":"2023-10-10T20:35:55.949200+0000","last_became_active":"2023-10-10T20:35:55.947681+0000","last_became_peered":"2023-10-10T20:35:55.947681+0000","last_unstale":"2023-10-10T20:35:55.949200+0000","last_undegraded":"2023-10-10T20:35:55.949200+0000","last_fullsized":"2023-10-10T20:35:55.949200+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:04:06.958649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.964398+0000","last_change":"2023-10-10T20:35:55.962840+0000","last_active":"2023-10-10T20:35:55.964398+0000","last_peered":"2023-10-10T20:35:55.964398+0000","last_clean":"2023-10-10T20:35:55.964398+0000","last_became_active":"2023-10-10T20:35:55.962576+0000","last_became_peered":"2023-10-10T20:35:55.962576+0000","last_unstale":"2023-10-10T20:35:55.964398+0000","last_undegraded":"2023-10-10T20:35:55.964398+0000","last_fullsized":"2023-10-10T20:35:55.964398+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:22:35.423687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.956361+0000","last_change":"2023-10-10T20:35:55.952472+0000","last_active":"2023-10-10T20:35:55.956361+0000","last_peered":"2023-10-10T20:35:55.956361+0000","last_clean":"2023-10-10T20:35:55.956361+0000","last_became_active":"2023-10-10T20:35:55.952179+0000","last_became_peered":"2023-10-10T20:35:55.952179+0000","last_unstale":"2023-10-10T20:35:55.956361+0000","last_undegraded":"2023-10-10T20:35:55.956361+0000","last_fullsized":"2023-10-10T20:35:55.956361+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:43:59.553286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.952178+0000","last_change":"2023-10-10T20:35:55.950427+0000","last_active":"2023-10-10T20:35:55.952178+0000","last_peered":"2023-10-10T20:35:55.952178+0000","last_clean":"2023-10-10T20:35:55.952178+0000","last_became_active":"2023-10-10T20:35:55.949974+0000","last_became_peered":"2023-10-10T20:35:55.949974+0000","last_unstale":"2023-10-10T20:35:55.952178+0000","last_undegraded":"2023-10-10T20:35:55.952178+0000","last_fullsized":"2023-10-10T20:35:55.952178+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:25.472048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.956294+0000","last_change":"2023-10-10T20:35:55.951213+0000","last_active":"2023-10-10T20:35:55.956294+0000","last_peered":"2023-10-10T20:35:55.956294+0000","last_clean":"2023-10-10T20:35:55.956294+0000","last_became_active":"2023-10-10T20:35:55.950366+0000","last_became_peered":"2023-10-10T20:35:55.950366+0000","last_unstale":"2023-10-10T20:35:55.956294+0000","last_undegraded":"2023-10-10T20:35:55.956294+0000","last_fullsized":"2023-10-10T20:35:55.956294+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:02.640005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.962808+0000","last_change":"2023-10-10T20:35:55.961384+0000","last_active":"2023-10-10T20:35:55.962808+0000","last_peered":"2023-10-10T20:35:55.962808+0000","last_clean":"2023-10-10T20:35:55.962808+0000","last_became_active":"2023-10-10T20:35:55.961136+0000","last_became_peered":"2023-10-10T20:35:55.961136+0000","last_unstale":"2023-10-10T20:35:55.962808+0000","last_undegraded":"2023-10-10T20:35:55.962808+0000","last_fullsized":"2023-10-10T20:35:55.962808+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:49.506716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.959667+0000","last_change":"2023-10-10T20:35:55.955629+0000","last_active":"2023-10-10T20:35:55.959667+0000","last_peered":"2023-10-10T20:35:55.959667+0000","last_clean":"2023-10-10T20:35:55.959667+0000","last_became_active":"2023-10-10T20:35:55.955366+0000","last_became_peered":"2023-10-10T20:35:55.955366+0000","last_unstale":"2023-10-10T20:35:55.959667+0000","last_undegraded":"2023-10-10T20:35:55.959667+0000","last_fullsized":"2023-10-10T20:35:55.959667+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:52:23.620245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.950923+0000","last_change":"2023-10-10T20:35:55.949557+0000","last_active":"2023-10-10T20:35:55.950923+0000","last_peered":"2023-10-10T20:35:55.950923+0000","last_clean":"2023-10-10T20:35:55.950923+0000","last_became_active":"2023-10-10T20:35:55.947280+0000","last_became_peered":"2023-10-10T20:35:55.947280+0000","last_unstale":"2023-10-10T20:35:55.950923+0000","last_undegraded":"2023-10-10T20:35:55.950923+0000","last_fullsized":"2023-10-10T20:35:55.950923+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:10:25.924046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960066+0000","last_change":"2023-10-10T20:35:55.945650+0000","last_active":"2023-10-10T20:35:55.960066+0000","last_peered":"2023-10-10T20:35:55.960066+0000","last_clean":"2023-10-10T20:35:55.960066+0000","last_became_active":"2023-10-10T20:35:55.945405+0000","last_became_peered":"2023-10-10T20:35:55.945405+0000","last_unstale":"2023-10-10T20:35:55.960066+0000","last_undegraded":"2023-10-10T20:35:55.960066+0000","last_fullsized":"2023-10-10T20:35:55.960066+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:00:30.852312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.959671+0000","last_change":"2023-10-10T20:35:55.952327+0000","last_active":"2023-10-10T20:35:55.959671+0000","last_peered":"2023-10-10T20:35:55.959671+0000","last_clean":"2023-10-10T20:35:55.959671+0000","last_became_active":"2023-10-10T20:35:55.952081+0000","last_became_peered":"2023-10-10T20:35:55.952081+0000","last_unstale":"2023-10-10T20:35:55.959671+0000","last_undegraded":"2023-10-10T20:35:55.959671+0000","last_fullsized":"2023-10-10T20:35:55.959671+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:26:28.857713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.963571+0000","last_change":"2023-10-10T20:35:55.961220+0000","last_active":"2023-10-10T20:35:55.963571+0000","last_peered":"2023-10-10T20:35:55.963571+0000","last_clean":"2023-10-10T20:35:55.963571+0000","last_became_active":"2023-10-10T20:35:55.960921+0000","last_became_peered":"2023-10-10T20:35:55.960921+0000","last_unstale":"2023-10-10T20:35:55.963571+0000","last_undegraded":"2023-10-10T20:35:55.963571+0000","last_fullsized":"2023-10-10T20:35:55.963571+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:34:43.703414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.946304+0000","last_change":"2023-10-10T20:35:55.944128+0000","last_active":"2023-10-10T20:35:55.946304+0000","last_peered":"2023-10-10T20:35:55.946304+0000","last_clean":"2023-10-10T20:35:55.946304+0000","last_became_active":"2023-10-10T20:35:55.943872+0000","last_became_peered":"2023-10-10T20:35:55.943872+0000","last_unstale":"2023-10-10T20:35:55.946304+0000","last_undegraded":"2023-10-10T20:35:55.946304+0000","last_fullsized":"2023-10-10T20:35:55.946304+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:53:55.367489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.958226+0000","last_change":"2023-10-10T20:35:55.951763+0000","last_active":"2023-10-10T20:35:55.958226+0000","last_peered":"2023-10-10T20:35:55.958226+0000","last_clean":"2023-10-10T20:35:55.958226+0000","last_became_active":"2023-10-10T20:35:55.951521+0000","last_became_peered":"2023-10-10T20:35:55.951521+0000","last_unstale":"2023-10-10T20:35:55.958226+0000","last_undegraded":"2023-10-10T20:35:55.958226+0000","last_fullsized":"2023-10-10T20:35:55.958226+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:45:13.936944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.964309+0000","last_change":"2023-10-10T20:35:55.961725+0000","last_active":"2023-10-10T20:35:55.964309+0000","last_peered":"2023-10-10T20:35:55.964309+0000","last_clean":"2023-10-10T20:35:55.964309+0000","last_became_active":"2023-10-10T20:35:55.961471+0000","last_became_peered":"2023-10-10T20:35:55.961471+0000","last_unstale":"2023-10-10T20:35:55.964309+0000","last_undegraded":"2023-10-10T20:35:55.964309+0000","last_fullsized":"2023-10-10T20:35:55.964309+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:33:38.647459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960152+0000","last_change":"2023-10-10T20:35:55.944952+0000","last_active":"2023-10-10T20:35:55.960152+0000","last_peered":"2023-10-10T20:35:55.960152+0000","last_clean":"2023-10-10T20:35:55.960152+0000","last_became_active":"2023-10-10T20:35:55.944700+0000","last_became_peered":"2023-10-10T20:35:55.944700+0000","last_unstale":"2023-10-10T20:35:55.960152+0000","last_undegraded":"2023-10-10T20:35:55.960152+0000","last_fullsized":"2023-10-10T20:35:55.960152+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:34:13.167717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"49'4","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961533+0000","last_change":"2023-10-10T20:35:54.943470+0000","last_active":"2023-10-10T20:35:55.961533+0000","last_peered":"2023-10-10T20:35:55.961533+0000","last_clean":"2023-10-10T20:35:55.961533+0000","last_became_active":"2023-10-10T20:35:54.943215+0000","last_became_peered":"2023-10-10T20:35:54.943215+0000","last_unstale":"2023-10-10T20:35:55.961533+0000","last_undegraded":"2023-10-10T20:35:55.961533+0000","last_fullsized":"2023-10-10T20:35:55.961533+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:30:09.063259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.958146+0000","last_change":"2023-10-10T20:35:55.951492+0000","last_active":"2023-10-10T20:35:55.958146+0000","last_peered":"2023-10-10T20:35:55.958146+0000","last_clean":"2023-10-10T20:35:55.958146+0000","last_became_active":"2023-10-10T20:35:55.951252+0000","last_became_peered":"2023-10-10T20:35:55.951252+0000","last_unstale":"2023-10-10T20:35:55.958146+0000","last_undegraded":"2023-10-10T20:35:55.958146+0000","last_fullsized":"2023-10-10T20:35:55.958146+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:27:11.743663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"49'1","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.947489+0000","last_change":"2023-10-10T20:35:54.946572+0000","last_active":"2023-10-10T20:35:55.947489+0000","last_peered":"2023-10-10T20:35:55.947489+0000","last_clean":"2023-10-10T20:35:55.947489+0000","last_became_active":"2023-10-10T20:35:54.946318+0000","last_became_peered":"2023-10-10T20:35:54.946318+0000","last_unstale":"2023-10-10T20:35:55.947489+0000","last_undegraded":"2023-10-10T20:35:55.947489+0000","last_fullsized":"2023-10-10T20:35:55.947489+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:00:29.845458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961746+0000","last_change":"2023-10-10T20:35:55.957973+0000","last_active":"2023-10-10T20:35:55.961746+0000","last_peered":"2023-10-10T20:35:55.961746+0000","last_clean":"2023-10-10T20:35:55.961746+0000","last_became_active":"2023-10-10T20:35:55.957725+0000","last_became_peered":"2023-10-10T20:35:55.957725+0000","last_unstale":"2023-10-10T20:35:55.961746+0000","last_undegraded":"2023-10-10T20:35:55.961746+0000","last_fullsized":"2023-10-10T20:35:55.961746+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:59:36.779168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"49'2","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960013+0000","last_change":"2023-10-10T20:35:54.940850+0000","last_active":"2023-10-10T20:35:55.960013+0000","last_peered":"2023-10-10T20:35:55.960013+0000","last_clean":"2023-10-10T20:35:55.960013+0000","last_became_active":"2023-10-10T20:35:54.940614+0000","last_became_peered":"2023-10-10T20:35:54.940614+0000","last_unstale":"2023-10-10T20:35:55.960013+0000","last_undegraded":"2023-10-10T20:35:55.960013+0000","last_fullsized":"2023-10-10T20:35:55.960013+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:32:18.143428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961566+0000","last_change":"2023-10-10T20:35:55.957441+0000","last_active":"2023-10-10T20:35:55.961566+0000","last_peered":"2023-10-10T20:35:55.961566+0000","last_clean":"2023-10-10T20:35:55.961566+0000","last_became_active":"2023-10-10T20:35:55.956959+0000","last_became_peered":"2023-10-10T20:35:55.956959+0000","last_unstale":"2023-10-10T20:35:55.961566+0000","last_undegraded":"2023-10-10T20:35:55.961566+0000","last_fullsized":"2023-10-10T20:35:55.961566+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:51.054414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"49'2","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961103+0000","last_change":"2023-10-10T20:35:54.946218+0000","last_active":"2023-10-10T20:35:55.961103+0000","last_peered":"2023-10-10T20:35:55.961103+0000","last_clean":"2023-10-10T20:35:55.961103+0000","last_became_active":"2023-10-10T20:35:54.945986+0000","last_became_peered":"2023-10-10T20:35:54.945986+0000","last_unstale":"2023-10-10T20:35:55.961103+0000","last_undegraded":"2023-10-10T20:35:55.961103+0000","last_fullsized":"2023-10-10T20:35:55.961103+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:19:13.265287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.950152+0000","last_change":"2023-10-10T20:35:55.949173+0000","last_active":"2023-10-10T20:35:55.950152+0000","last_peered":"2023-10-10T20:35:55.950152+0000","last_clean":"2023-10-10T20:35:55.950152+0000","last_became_active":"2023-10-10T20:35:55.948869+0000","last_became_peered":"2023-10-10T20:35:55.948869+0000","last_unstale":"2023-10-10T20:35:55.950152+0000","last_undegraded":"2023-10-10T20:35:55.950152+0000","last_fullsized":"2023-10-10T20:35:55.950152+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:37:40.585130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"49'1","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.945361+0000","last_change":"2023-10-10T20:35:54.940564+0000","last_active":"2023-10-10T20:35:55.945361+0000","last_peered":"2023-10-10T20:35:55.945361+0000","last_clean":"2023-10-10T20:35:55.945361+0000","last_became_active":"2023-10-10T20:35:54.940322+0000","last_became_peered":"2023-10-10T20:35:54.940322+0000","last_unstale":"2023-10-10T20:35:55.945361+0000","last_undegraded":"2023-10-10T20:35:55.945361+0000","last_fullsized":"2023-10-10T20:35:55.945361+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:29:15.532075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.964426+0000","last_change":"2023-10-10T20:35:55.963301+0000","last_active":"2023-10-10T20:35:55.964426+0000","last_peered":"2023-10-10T20:35:55.964426+0000","last_clean":"2023-10-10T20:35:55.964426+0000","last_became_active":"2023-10-10T20:35:55.962761+0000","last_became_peered":"2023-10-10T20:35:55.962761+0000","last_unstale":"2023-10-10T20:35:55.964426+0000","last_undegraded":"2023-10-10T20:35:55.964426+0000","last_fullsized":"2023-10-10T20:35:55.964426+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:35:25.594991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"49'1","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.956412+0000","last_change":"2023-10-10T20:35:54.941702+0000","last_active":"2023-10-10T20:35:55.956412+0000","last_peered":"2023-10-10T20:35:55.956412+0000","last_clean":"2023-10-10T20:35:55.956412+0000","last_became_active":"2023-10-10T20:35:54.941461+0000","last_became_peered":"2023-10-10T20:35:54.941461+0000","last_unstale":"2023-10-10T20:35:55.956412+0000","last_undegraded":"2023-10-10T20:35:55.956412+0000","last_fullsized":"2023-10-10T20:35:55.956412+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:32:25.739455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.952775+0000","last_change":"2023-10-10T20:35:55.951488+0000","last_active":"2023-10-10T20:35:55.952775+0000","last_peered":"2023-10-10T20:35:55.952775+0000","last_clean":"2023-10-10T20:35:55.952775+0000","last_became_active":"2023-10-10T20:35:55.950945+0000","last_became_peered":"2023-10-10T20:35:55.950945+0000","last_unstale":"2023-10-10T20:35:55.952775+0000","last_undegraded":"2023-10-10T20:35:55.952775+0000","last_fullsized":"2023-10-10T20:35:55.952775+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:18:34.718096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.935825+0000","last_change":"2023-10-10T20:35:54.943425+0000","last_active":"2023-10-10T20:35:55.935825+0000","last_peered":"2023-10-10T20:35:55.935825+0000","last_clean":"2023-10-10T20:35:55.935825+0000","last_became_active":"2023-10-10T20:35:54.943067+0000","last_became_peered":"2023-10-10T20:35:54.943067+0000","last_unstale":"2023-10-10T20:35:55.935825+0000","last_undegraded":"2023-10-10T20:35:55.935825+0000","last_fullsized":"2023-10-10T20:35:55.935825+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:48:03.966676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.964339+0000","last_change":"2023-10-10T20:35:55.961977+0000","last_active":"2023-10-10T20:35:55.964339+0000","last_peered":"2023-10-10T20:35:55.964339+0000","last_clean":"2023-10-10T20:35:55.964339+0000","last_became_active":"2023-10-10T20:35:55.961321+0000","last_became_peered":"2023-10-10T20:35:55.961321+0000","last_unstale":"2023-10-10T20:35:55.964339+0000","last_undegraded":"2023-10-10T20:35:55.964339+0000","last_fullsized":"2023-10-10T20:35:55.964339+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:05.710248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.936391+0000","last_change":"2023-10-10T20:35:54.947632+0000","last_active":"2023-10-10T20:35:55.936391+0000","last_peered":"2023-10-10T20:35:55.936391+0000","last_clean":"2023-10-10T20:35:55.936391+0000","last_became_active":"2023-10-10T20:35:54.947355+0000","last_became_peered":"2023-10-10T20:35:54.947355+0000","last_unstale":"2023-10-10T20:35:55.936391+0000","last_undegraded":"2023-10-10T20:35:55.936391+0000","last_fullsized":"2023-10-10T20:35:55.936391+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:00:25.701036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.954539+0000","last_change":"2023-10-10T20:35:55.953501+0000","last_active":"2023-10-10T20:35:55.954539+0000","last_peered":"2023-10-10T20:35:55.954539+0000","last_clean":"2023-10-10T20:35:55.954539+0000","last_became_active":"2023-10-10T20:35:55.953280+0000","last_became_peered":"2023-10-10T20:35:55.953280+0000","last_unstale":"2023-10-10T20:35:55.954539+0000","last_undegraded":"2023-10-10T20:35:55.954539+0000","last_fullsized":"2023-10-10T20:35:55.954539+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:56:52.778641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.933687+0000","last_change":"2023-10-10T20:35:54.943028+0000","last_active":"2023-10-10T20:35:55.933687+0000","last_peered":"2023-10-10T20:35:55.933687+0000","last_clean":"2023-10-10T20:35:55.933687+0000","last_became_active":"2023-10-10T20:35:54.942704+0000","last_became_peered":"2023-10-10T20:35:54.942704+0000","last_unstale":"2023-10-10T20:35:55.933687+0000","last_undegraded":"2023-10-10T20:35:55.933687+0000","last_fullsized":"2023-10-10T20:35:55.933687+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:55.039204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.959208+0000","last_change":"2023-10-10T20:35:55.957452+0000","last_active":"2023-10-10T20:35:55.959208+0000","last_peered":"2023-10-10T20:35:55.959208+0000","last_clean":"2023-10-10T20:35:55.959208+0000","last_became_active":"2023-10-10T20:35:55.957155+0000","last_became_peered":"2023-10-10T20:35:55.957155+0000","last_unstale":"2023-10-10T20:35:55.959208+0000","last_undegraded":"2023-10-10T20:35:55.959208+0000","last_fullsized":"2023-10-10T20:35:55.959208+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:55.043141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.937839+0000","last_change":"2023-10-10T20:35:54.943788+0000","last_active":"2023-10-10T20:35:55.937839+0000","last_peered":"2023-10-10T20:35:55.937839+0000","last_clean":"2023-10-10T20:35:55.937839+0000","last_became_active":"2023-10-10T20:35:54.943631+0000","last_became_peered":"2023-10-10T20:35:54.943631+0000","last_unstale":"2023-10-10T20:35:55.937839+0000","last_undegraded":"2023-10-10T20:35:55.937839+0000","last_fullsized":"2023-10-10T20:35:55.937839+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:40:16.021893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.957978+0000","last_change":"2023-10-10T20:35:55.954647+0000","last_active":"2023-10-10T20:35:55.957978+0000","last_peered":"2023-10-10T20:35:55.957978+0000","last_clean":"2023-10-10T20:35:55.957978+0000","last_became_active":"2023-10-10T20:35:55.952784+0000","last_became_peered":"2023-10-10T20:35:55.952784+0000","last_unstale":"2023-10-10T20:35:55.957978+0000","last_undegraded":"2023-10-10T20:35:55.957978+0000","last_fullsized":"2023-10-10T20:35:55.957978+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:56:52.778641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"49'3","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.958634+0000","last_change":"2023-10-10T20:35:54.943076+0000","last_active":"2023-10-10T20:35:55.958634+0000","last_peered":"2023-10-10T20:35:55.958634+0000","last_clean":"2023-10-10T20:35:55.958634+0000","last_became_active":"2023-10-10T20:35:54.942812+0000","last_became_peered":"2023-10-10T20:35:54.942812+0000","last_unstale":"2023-10-10T20:35:55.958634+0000","last_undegraded":"2023-10-10T20:35:55.958634+0000","last_fullsized":"2023-10-10T20:35:55.958634+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:56:51.771786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960155+0000","last_change":"2023-10-10T20:35:55.957956+0000","last_active":"2023-10-10T20:35:55.960155+0000","last_peered":"2023-10-10T20:35:55.960155+0000","last_clean":"2023-10-10T20:35:55.960155+0000","last_became_active":"2023-10-10T20:35:55.957535+0000","last_became_peered":"2023-10-10T20:35:55.957535+0000","last_unstale":"2023-10-10T20:35:55.960155+0000","last_undegraded":"2023-10-10T20:35:55.960155+0000","last_fullsized":"2023-10-10T20:35:55.960155+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:14:22.420266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"49'6","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.967270+0000","last_change":"2023-10-10T20:35:54.944052+0000","last_active":"2023-10-10T20:35:55.967270+0000","last_peered":"2023-10-10T20:35:55.967270+0000","last_clean":"2023-10-10T20:35:55.967270+0000","last_became_active":"2023-10-10T20:35:54.943539+0000","last_became_peered":"2023-10-10T20:35:54.943539+0000","last_unstale":"2023-10-10T20:35:55.967270+0000","last_undegraded":"2023-10-10T20:35:55.967270+0000","last_fullsized":"2023-10-10T20:35:55.967270+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:12:41.189930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961458+0000","last_change":"2023-10-10T20:35:55.960062+0000","last_active":"2023-10-10T20:35:55.961458+0000","last_peered":"2023-10-10T20:35:55.961458+0000","last_clean":"2023-10-10T20:35:55.961458+0000","last_became_active":"2023-10-10T20:35:55.959807+0000","last_became_peered":"2023-10-10T20:35:55.959807+0000","last_unstale":"2023-10-10T20:35:55.961458+0000","last_undegraded":"2023-10-10T20:35:55.961458+0000","last_fullsized":"2023-10-10T20:35:55.961458+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:14:22.420266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"49'2","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.950092+0000","last_change":"2023-10-10T20:35:54.945458+0000","last_active":"2023-10-10T20:35:55.950092+0000","last_peered":"2023-10-10T20:35:55.950092+0000","last_clean":"2023-10-10T20:35:55.950092+0000","last_became_active":"2023-10-10T20:35:54.943874+0000","last_became_peered":"2023-10-10T20:35:54.943874+0000","last_unstale":"2023-10-10T20:35:55.950092+0000","last_undegraded":"2023-10-10T20:35:55.950092+0000","last_fullsized":"2023-10-10T20:35:55.950092+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:54.036286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.950504+0000","last_change":"2023-10-10T20:35:55.946244+0000","last_active":"2023-10-10T20:35:55.950504+0000","last_peered":"2023-10-10T20:35:55.950504+0000","last_clean":"2023-10-10T20:35:55.950504+0000","last_became_active":"2023-10-10T20:35:55.945881+0000","last_became_peered":"2023-10-10T20:35:55.945881+0000","last_unstale":"2023-10-10T20:35:55.950504+0000","last_undegraded":"2023-10-10T20:35:55.950504+0000","last_fullsized":"2023-10-10T20:35:55.950504+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:17.215044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.940109+0000","last_change":"2023-10-10T20:35:54.946176+0000","last_active":"2023-10-10T20:35:55.940109+0000","last_peered":"2023-10-10T20:35:55.940109+0000","last_clean":"2023-10-10T20:35:55.940109+0000","last_became_active":"2023-10-10T20:35:54.945884+0000","last_became_peered":"2023-10-10T20:35:54.945884+0000","last_unstale":"2023-10-10T20:35:55.940109+0000","last_undegraded":"2023-10-10T20:35:55.940109+0000","last_fullsized":"2023-10-10T20:35:55.940109+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:31:11.544868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.945789+0000","last_change":"2023-10-10T20:35:55.943829+0000","last_active":"2023-10-10T20:35:55.945789+0000","last_peered":"2023-10-10T20:35:55.945789+0000","last_clean":"2023-10-10T20:35:55.945789+0000","last_became_active":"2023-10-10T20:35:55.943469+0000","last_became_peered":"2023-10-10T20:35:55.943469+0000","last_unstale":"2023-10-10T20:35:55.945789+0000","last_undegraded":"2023-10-10T20:35:55.945789+0000","last_fullsized":"2023-10-10T20:35:55.945789+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:17:24.929284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.935875+0000","last_change":"2023-10-10T20:35:54.947183+0000","last_active":"2023-10-10T20:35:55.935875+0000","last_peered":"2023-10-10T20:35:55.935875+0000","last_clean":"2023-10-10T20:35:55.935875+0000","last_became_active":"2023-10-10T20:35:54.946917+0000","last_became_peered":"2023-10-10T20:35:54.946917+0000","last_unstale":"2023-10-10T20:35:55.935875+0000","last_undegraded":"2023-10-10T20:35:55.935875+0000","last_fullsized":"2023-10-10T20:35:55.935875+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:18:39.785303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.956270+0000","last_change":"2023-10-10T20:35:55.951094+0000","last_active":"2023-10-10T20:35:55.956270+0000","last_peered":"2023-10-10T20:35:55.956270+0000","last_clean":"2023-10-10T20:35:55.956270+0000","last_became_active":"2023-10-10T20:35:55.950340+0000","last_became_peered":"2023-10-10T20:35:55.950340+0000","last_unstale":"2023-10-10T20:35:55.956270+0000","last_undegraded":"2023-10-10T20:35:55.956270+0000","last_fullsized":"2023-10-10T20:35:55.956270+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:28:25.423217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"49'8","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.958388+0000","last_change":"2023-10-10T20:35:54.944176+0000","last_active":"2023-10-10T20:35:55.958388+0000","last_peered":"2023-10-10T20:35:55.958388+0000","last_clean":"2023-10-10T20:35:55.958388+0000","last_became_active":"2023-10-10T20:35:54.942915+0000","last_became_peered":"2023-10-10T20:35:54.942915+0000","last_unstale":"2023-10-10T20:35:55.958388+0000","last_undegraded":"2023-10-10T20:35:55.958388+0000","last_fullsized":"2023-10-10T20:35:55.958388+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:27:10.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.946158+0000","last_change":"2023-10-10T20:35:55.944174+0000","last_active":"2023-10-10T20:35:55.946158+0000","last_peered":"2023-10-10T20:35:55.946158+0000","last_clean":"2023-10-10T20:35:55.946158+0000","last_became_active":"2023-10-10T20:35:55.943888+0000","last_became_peered":"2023-10-10T20:35:55.943888+0000","last_unstale":"2023-10-10T20:35:55.946158+0000","last_undegraded":"2023-10-10T20:35:55.946158+0000","last_fullsized":"2023-10-10T20:35:55.946158+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:21:32.364960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"49'1","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.947596+0000","last_change":"2023-10-10T20:35:54.945585+0000","last_active":"2023-10-10T20:35:55.947596+0000","last_peered":"2023-10-10T20:35:55.947596+0000","last_clean":"2023-10-10T20:35:55.947596+0000","last_became_active":"2023-10-10T20:35:54.945337+0000","last_became_peered":"2023-10-10T20:35:54.945337+0000","last_unstale":"2023-10-10T20:35:55.947596+0000","last_undegraded":"2023-10-10T20:35:55.947596+0000","last_fullsized":"2023-10-10T20:35:55.947596+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:50.047559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961481+0000","last_change":"2023-10-10T20:35:55.960033+0000","last_active":"2023-10-10T20:35:55.961481+0000","last_peered":"2023-10-10T20:35:55.961481+0000","last_clean":"2023-10-10T20:35:55.961481+0000","last_became_active":"2023-10-10T20:35:55.959771+0000","last_became_peered":"2023-10-10T20:35:55.959771+0000","last_unstale":"2023-10-10T20:35:55.961481+0000","last_undegraded":"2023-10-10T20:35:55.961481+0000","last_fullsized":"2023-10-10T20:35:55.961481+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:28:25.423217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"49'2","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960103+0000","last_change":"2023-10-10T20:35:54.941794+0000","last_active":"2023-10-10T20:35:55.960103+0000","last_peered":"2023-10-10T20:35:55.960103+0000","last_clean":"2023-10-10T20:35:55.960103+0000","last_became_active":"2023-10-10T20:35:54.941553+0000","last_became_peered":"2023-10-10T20:35:54.941553+0000","last_unstale":"2023-10-10T20:35:55.960103+0000","last_undegraded":"2023-10-10T20:35:55.960103+0000","last_fullsized":"2023-10-10T20:35:55.960103+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:41:42.393651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.954562+0000","last_change":"2023-10-10T20:35:55.953629+0000","last_active":"2023-10-10T20:35:55.954562+0000","last_peered":"2023-10-10T20:35:55.954562+0000","last_clean":"2023-10-10T20:35:55.954562+0000","last_became_active":"2023-10-10T20:35:55.953250+0000","last_became_peered":"2023-10-10T20:35:55.953250+0000","last_unstale":"2023-10-10T20:35:55.954562+0000","last_undegraded":"2023-10-10T20:35:55.954562+0000","last_fullsized":"2023-10-10T20:35:55.954562+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:55:41.128475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"49'1","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.956667+0000","last_change":"2023-10-10T20:35:54.946161+0000","last_active":"2023-10-10T20:35:55.956667+0000","last_peered":"2023-10-10T20:35:55.956667+0000","last_clean":"2023-10-10T20:35:55.956667+0000","last_became_active":"2023-10-10T20:35:54.945889+0000","last_became_peered":"2023-10-10T20:35:54.945889+0000","last_unstale":"2023-10-10T20:35:55.956667+0000","last_undegraded":"2023-10-10T20:35:55.956667+0000","last_fullsized":"2023-10-10T20:35:55.956667+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:53:54.456688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960136+0000","last_change":"2023-10-10T20:35:55.957782+0000","last_active":"2023-10-10T20:35:55.960136+0000","last_peered":"2023-10-10T20:35:55.960136+0000","last_clean":"2023-10-10T20:35:55.960136+0000","last_became_active":"2023-10-10T20:35:55.957215+0000","last_became_peered":"2023-10-10T20:35:55.957215+0000","last_unstale":"2023-10-10T20:35:55.960136+0000","last_undegraded":"2023-10-10T20:35:55.960136+0000","last_fullsized":"2023-10-10T20:35:55.960136+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:33:38.647459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.937890+0000","last_change":"2023-10-10T20:35:54.946769+0000","last_active":"2023-10-10T20:35:55.937890+0000","last_peered":"2023-10-10T20:35:55.937890+0000","last_clean":"2023-10-10T20:35:55.937890+0000","last_became_active":"2023-10-10T20:35:54.946534+0000","last_became_peered":"2023-10-10T20:35:54.946534+0000","last_unstale":"2023-10-10T20:35:55.937890+0000","last_undegraded":"2023-10-10T20:35:55.937890+0000","last_fullsized":"2023-10-10T20:35:55.937890+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:25:13.693624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960107+0000","last_change":"2023-10-10T20:35:55.957781+0000","last_active":"2023-10-10T20:35:55.960107+0000","last_peered":"2023-10-10T20:35:55.960107+0000","last_clean":"2023-10-10T20:35:55.960107+0000","last_became_active":"2023-10-10T20:35:55.957533+0000","last_became_peered":"2023-10-10T20:35:55.957533+0000","last_unstale":"2023-10-10T20:35:55.960107+0000","last_undegraded":"2023-10-10T20:35:55.960107+0000","last_fullsized":"2023-10-10T20:35:55.960107+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:02.640005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"49'2","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960608+0000","last_change":"2023-10-10T20:35:54.945381+0000","last_active":"2023-10-10T20:35:55.960608+0000","last_peered":"2023-10-10T20:35:55.960608+0000","last_clean":"2023-10-10T20:35:55.960608+0000","last_became_active":"2023-10-10T20:35:54.943618+0000","last_became_peered":"2023-10-10T20:35:54.943618+0000","last_unstale":"2023-10-10T20:35:55.960608+0000","last_undegraded":"2023-10-10T20:35:55.960608+0000","last_fullsized":"2023-10-10T20:35:55.960608+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:23:29.829844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960072+0000","last_change":"2023-10-10T20:35:55.946118+0000","last_active":"2023-10-10T20:35:55.960072+0000","last_peered":"2023-10-10T20:35:55.960072+0000","last_clean":"2023-10-10T20:35:55.960072+0000","last_became_active":"2023-10-10T20:35:55.945858+0000","last_became_peered":"2023-10-10T20:35:55.945858+0000","last_unstale":"2023-10-10T20:35:55.960072+0000","last_undegraded":"2023-10-10T20:35:55.960072+0000","last_fullsized":"2023-10-10T20:35:55.960072+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:48:04.973531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.945437+0000","last_change":"2023-10-10T20:35:54.932698+0000","last_active":"2023-10-10T20:35:55.945437+0000","last_peered":"2023-10-10T20:35:55.945437+0000","last_clean":"2023-10-10T20:35:55.945437+0000","last_became_active":"2023-10-10T20:35:54.932065+0000","last_became_peered":"2023-10-10T20:35:54.932065+0000","last_unstale":"2023-10-10T20:35:55.945437+0000","last_undegraded":"2023-10-10T20:35:55.945437+0000","last_fullsized":"2023-10-10T20:35:55.945437+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:00:25.701036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960214+0000","last_change":"2023-10-10T20:35:55.946535+0000","last_active":"2023-10-10T20:35:55.960214+0000","last_peered":"2023-10-10T20:35:55.960214+0000","last_clean":"2023-10-10T20:35:55.960214+0000","last_became_active":"2023-10-10T20:35:55.946239+0000","last_became_peered":"2023-10-10T20:35:55.946239+0000","last_unstale":"2023-10-10T20:35:55.960214+0000","last_undegraded":"2023-10-10T20:35:55.960214+0000","last_fullsized":"2023-10-10T20:35:55.960214+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:18:40.792158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.935956+0000","last_change":"2023-10-10T20:35:54.946856+0000","last_active":"2023-10-10T20:35:55.935956+0000","last_peered":"2023-10-10T20:35:55.935956+0000","last_clean":"2023-10-10T20:35:55.935956+0000","last_became_active":"2023-10-10T20:35:54.946598+0000","last_became_peered":"2023-10-10T20:35:54.946598+0000","last_unstale":"2023-10-10T20:35:55.935956+0000","last_undegraded":"2023-10-10T20:35:55.935956+0000","last_fullsized":"2023-10-10T20:35:55.935956+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:17:23.922429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.950156+0000","last_change":"2023-10-10T20:35:55.947319+0000","last_active":"2023-10-10T20:35:55.950156+0000","last_peered":"2023-10-10T20:35:55.950156+0000","last_clean":"2023-10-10T20:35:55.950156+0000","last_became_active":"2023-10-10T20:35:55.947063+0000","last_became_peered":"2023-10-10T20:35:55.947063+0000","last_unstale":"2023-10-10T20:35:55.950156+0000","last_undegraded":"2023-10-10T20:35:55.950156+0000","last_fullsized":"2023-10-10T20:35:55.950156+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:33:38.647459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"49'2","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961970+0000","last_change":"2023-10-10T20:35:54.945967+0000","last_active":"2023-10-10T20:35:55.961970+0000","last_peered":"2023-10-10T20:35:55.961970+0000","last_clean":"2023-10-10T20:35:55.961970+0000","last_became_active":"2023-10-10T20:35:54.945703+0000","last_became_peered":"2023-10-10T20:35:54.945703+0000","last_unstale":"2023-10-10T20:35:55.961970+0000","last_undegraded":"2023-10-10T20:35:55.961970+0000","last_fullsized":"2023-10-10T20:35:55.961970+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:12:41.189930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.950190+0000","last_change":"2023-10-10T20:35:55.947138+0000","last_active":"2023-10-10T20:35:55.950190+0000","last_peered":"2023-10-10T20:35:55.950190+0000","last_clean":"2023-10-10T20:35:55.950190+0000","last_became_active":"2023-10-10T20:35:55.946859+0000","last_became_peered":"2023-10-10T20:35:55.946859+0000","last_unstale":"2023-10-10T20:35:55.950190+0000","last_undegraded":"2023-10-10T20:35:55.950190+0000","last_fullsized":"2023-10-10T20:35:55.950190+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:21:32.364960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.936901+0000","last_change":"2023-10-10T20:35:54.941680+0000","last_active":"2023-10-10T20:35:55.936901+0000","last_peered":"2023-10-10T20:35:55.936901+0000","last_clean":"2023-10-10T20:35:55.936901+0000","last_became_active":"2023-10-10T20:35:54.941440+0000","last_became_peered":"2023-10-10T20:35:54.941440+0000","last_unstale":"2023-10-10T20:35:55.936901+0000","last_undegraded":"2023-10-10T20:35:55.936901+0000","last_fullsized":"2023-10-10T20:35:55.936901+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:21.101690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.957958+0000","last_change":"2023-10-10T20:35:55.950744+0000","last_active":"2023-10-10T20:35:55.957958+0000","last_peered":"2023-10-10T20:35:55.957958+0000","last_clean":"2023-10-10T20:35:55.957958+0000","last_became_active":"2023-10-10T20:35:55.950509+0000","last_became_peered":"2023-10-10T20:35:55.950509+0000","last_unstale":"2023-10-10T20:35:55.957958+0000","last_undegraded":"2023-10-10T20:35:55.957958+0000","last_fullsized":"2023-10-10T20:35:55.957958+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:56.046059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.939675+0000","last_change":"2023-10-10T20:35:54.940210+0000","last_active":"2023-10-10T20:35:55.939675+0000","last_peered":"2023-10-10T20:35:55.939675+0000","last_clean":"2023-10-10T20:35:55.939675+0000","last_became_active":"2023-10-10T20:35:54.939930+0000","last_became_peered":"2023-10-10T20:35:54.939930+0000","last_unstale":"2023-10-10T20:35:55.939675+0000","last_undegraded":"2023-10-10T20:35:55.939675+0000","last_fullsized":"2023-10-10T20:35:55.939675+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:25.718980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.961586+0000","last_change":"2023-10-10T20:35:55.957568+0000","last_active":"2023-10-10T20:35:55.961586+0000","last_peered":"2023-10-10T20:35:55.961586+0000","last_clean":"2023-10-10T20:35:55.961586+0000","last_became_active":"2023-10-10T20:35:55.957222+0000","last_became_peered":"2023-10-10T20:35:55.957222+0000","last_unstale":"2023-10-10T20:35:55.961586+0000","last_undegraded":"2023-10-10T20:35:55.961586+0000","last_fullsized":"2023-10-10T20:35:55.961586+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:55.043141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.936606+0000","last_change":"2023-10-10T20:35:54.942350+0000","last_active":"2023-10-10T20:35:55.936606+0000","last_peered":"2023-10-10T20:35:55.936606+0000","last_clean":"2023-10-10T20:35:55.936606+0000","last_became_active":"2023-10-10T20:35:54.942082+0000","last_became_peered":"2023-10-10T20:35:54.942082+0000","last_unstale":"2023-10-10T20:35:55.936606+0000","last_undegraded":"2023-10-10T20:35:55.936606+0000","last_fullsized":"2023-10-10T20:35:55.936606+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:30:14.385873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.960745+0000","last_change":"2023-10-10T20:35:55.959128+0000","last_active":"2023-10-10T20:35:55.960745+0000","last_peered":"2023-10-10T20:35:55.960745+0000","last_clean":"2023-10-10T20:35:55.960745+0000","last_became_active":"2023-10-10T20:35:55.958877+0000","last_became_peered":"2023-10-10T20:35:55.958877+0000","last_unstale":"2023-10-10T20:35:55.960745+0000","last_undegraded":"2023-10-10T20:35:55.960745+0000","last_fullsized":"2023-10-10T20:35:55.960745+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:38:30.688214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.945639+0000","last_change":"2023-10-10T20:35:54.940768+0000","last_active":"2023-10-10T20:35:55.945639+0000","last_peered":"2023-10-10T20:35:55.945639+0000","last_clean":"2023-10-10T20:35:55.945639+0000","last_became_active":"2023-10-10T20:35:54.940554+0000","last_became_peered":"2023-10-10T20:35:54.940554+0000","last_unstale":"2023-10-10T20:35:55.945639+0000","last_undegraded":"2023-10-10T20:35:55.945639+0000","last_fullsized":"2023-10-10T20:35:55.945639+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:48.499861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.953982+0000","last_change":"2023-10-10T20:35:55.950625+0000","last_active":"2023-10-10T20:35:55.953982+0000","last_peered":"2023-10-10T20:35:55.953982+0000","last_clean":"2023-10-10T20:35:55.953982+0000","last_became_active":"2023-10-10T20:35:55.950302+0000","last_became_peered":"2023-10-10T20:35:55.950302+0000","last_unstale":"2023-10-10T20:35:55.953982+0000","last_undegraded":"2023-10-10T20:35:55.953982+0000","last_fullsized":"2023-10-10T20:35:55.953982+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:10:25.924046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.939832+0000","last_change":"2023-10-10T20:35:54.943152+0000","last_active":"2023-10-10T20:35:55.939832+0000","last_peered":"2023-10-10T20:35:55.939832+0000","last_clean":"2023-10-10T20:35:55.939832+0000","last_became_active":"2023-10-10T20:35:54.942861+0000","last_became_peered":"2023-10-10T20:35:54.942861+0000","last_unstale":"2023-10-10T20:35:55.939832+0000","last_undegraded":"2023-10-10T20:35:55.939832+0000","last_fullsized":"2023-10-10T20:35:55.939832+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:48.499861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.957888+0000","last_change":"2023-10-10T20:35:55.950195+0000","last_active":"2023-10-10T20:35:55.957888+0000","last_peered":"2023-10-10T20:35:55.957888+0000","last_clean":"2023-10-10T20:35:55.957888+0000","last_became_active":"2023-10-10T20:35:55.949913+0000","last_became_peered":"2023-10-10T20:35:55.949913+0000","last_unstale":"2023-10-10T20:35:55.957888+0000","last_undegraded":"2023-10-10T20:35:55.957888+0000","last_fullsized":"2023-10-10T20:35:55.957888+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:24:19.034235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.945550+0000","last_change":"2023-10-10T20:35:54.938848+0000","last_active":"2023-10-10T20:35:55.945550+0000","last_peered":"2023-10-10T20:35:55.945550+0000","last_clean":"2023-10-10T20:35:55.945550+0000","last_became_active":"2023-10-10T20:35:54.938610+0000","last_became_peered":"2023-10-10T20:35:54.938610+0000","last_unstale":"2023-10-10T20:35:55.945550+0000","last_undegraded":"2023-10-10T20:35:55.945550+0000","last_fullsized":"2023-10-10T20:35:55.945550+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:29:15.532075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.964363+0000","last_change":"2023-10-10T20:35:55.962459+0000","last_active":"2023-10-10T20:35:55.964363+0000","last_peered":"2023-10-10T20:35:55.964363+0000","last_clean":"2023-10-10T20:35:55.964363+0000","last_became_active":"2023-10-10T20:35:55.962220+0000","last_became_peered":"2023-10-10T20:35:55.962220+0000","last_unstale":"2023-10-10T20:35:55.964363+0000","last_undegraded":"2023-10-10T20:35:55.964363+0000","last_fullsized":"2023-10-10T20:35:55.964363+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:50:01.601004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.938642+0000","last_change":"2023-10-10T20:35:54.946647+0000","last_active":"2023-10-10T20:35:55.938642+0000","last_peered":"2023-10-10T20:35:55.938642+0000","last_clean":"2023-10-10T20:35:55.938642+0000","last_became_active":"2023-10-10T20:35:54.946417+0000","last_became_peered":"2023-10-10T20:35:54.946417+0000","last_unstale":"2023-10-10T20:35:55.938642+0000","last_undegraded":"2023-10-10T20:35:55.938642+0000","last_fullsized":"2023-10-10T20:35:55.938642+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:33:37.640604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.964331+0000","last_change":"2023-10-10T20:35:55.961861+0000","last_active":"2023-10-10T20:35:55.964331+0000","last_peered":"2023-10-10T20:35:55.964331+0000","last_clean":"2023-10-10T20:35:55.964331+0000","last_became_active":"2023-10-10T20:35:55.961005+0000","last_became_peered":"2023-10-10T20:35:55.961005+0000","last_unstale":"2023-10-10T20:35:55.964331+0000","last_undegraded":"2023-10-10T20:35:55.964331+0000","last_fullsized":"2023-10-10T20:35:55.964331+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:54.920198+0000","last_clean_scrub_stamp":"2023-10-10T20:35:54.920198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:24:17.176188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.939556+0000","last_change":"2023-10-10T20:35:54.941602+0000","last_active":"2023-10-10T20:35:55.939556+0000","last_peered":"2023-10-10T20:35:55.939556+0000","last_clean":"2023-10-10T20:35:55.939556+0000","last_became_active":"2023-10-10T20:35:54.941344+0000","last_became_peered":"2023-10-10T20:35:54.941344+0000","last_unstale":"2023-10-10T20:35:55.939556+0000","last_undegraded":"2023-10-10T20:35:55.939556+0000","last_fullsized":"2023-10-10T20:35:55.939556+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:35:53.913343+0000","last_clean_scrub_stamp":"2023-10-10T20:35:53.913343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:56:57.311897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-10T20:35:55.932286+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:35:55.932286+0000","last_peered":"2023-10-10T20:35:55.932286+0000","last_clean":"2023-10-10T20:35:55.932286+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:35:55.932286+0000","last_undegraded":"2023-10-10T20:35:55.932286+0000","last_fullsized":"2023-10-10T20:35:55.932286+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298460,"kb_used_data":1172,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073380,"statfs":{"total":96636764160,"available":96331141120,"internally_reserved":0,"allocated":1200128,"data_stored":842569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":6.8630000000000004,"5min":6.8630000000000004,"15min":6.8630000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":4.1959999999999997,"5min":4.1959999999999997,"15min":4.1959999999999997},"last":0.59499999999999997}]},{"osd":1,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":10.1,"5min":10.1,"15min":10.1},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":6.806,"5min":6.806,"15min":6.806},"last":0.72199999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":6.9340000000000002,"5min":6.9340000000000002,"15min":6.9340000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":6.9870000000000001,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.60899999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":6.6719999999999997,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":4.1609999999999996,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.64900000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.48799999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.46999999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.436}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297872,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073968,"statfs":{"total":96636764160,"available":96331743232,"internally_reserved":0,"allocated":598016,"data_stored":252041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":7.7590000000000003,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":7.8090000000000002,"5min":7.8090000000000002,"15min":7.8090000000000002},"last":0.55500000000000005}]},{"osd":1,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":7.867,"5min":7.867,"15min":7.867},"last":0.378},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":9.7089999999999996,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.498}]},{"osd":2,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":4.7999999999999998,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":6.7629999999999999,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":0.54000000000000004}]},{"osd":3,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":4.8620000000000001,"5min":4.8620000000000001,"15min":4.8620000000000001},"last":0.433},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":7.9009999999999998,"5min":7.9009999999999998,"15min":7.9009999999999998},"last":0.40500000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.58199999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.624}]},{"osd":7,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.46200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297876,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073964,"statfs":{"total":96636764160,"available":96331739136,"internally_reserved":0,"allocated":602112,"data_stored":252131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":5.0380000000000003,"5min":5.0380000000000003,"15min":5.0380000000000003},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.26700000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.25},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":5.0919999999999996,"5min":5.0919999999999996,"15min":5.0919999999999996},"last":0.28599999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.1159999999999997,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.309}]},{"osd":4,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":4.9850000000000003,"5min":4.9850000000000003,"15min":4.9850000000000003},"last":0.51700000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.42599999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":5.0659999999999998,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.50700000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0499999999999998,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.47599999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298464,"kb_used_data":1176,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073376,"statfs":{"total":96636764160,"available":96331137024,"internally_reserved":0,"allocated":1204224,"data_stored":843538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":7.9249999999999998,"5min":7.9249999999999998,"15min":7.9249999999999998},"last":0.245},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":6.6299999999999999,"5min":6.6299999999999999,"15min":6.6299999999999999},"last":0.39900000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":8.2509999999999994,"5min":8.2509999999999994,"15min":8.2509999999999994},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":6.7370000000000001,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.36899999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":7.7960000000000003,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":7.8929999999999998,"5min":7.8929999999999998,"15min":7.8929999999999998},"last":0.35099999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":6.6929999999999996,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":7.6920000000000002,"5min":7.6920000000000002,"15min":7.6920000000000002},"last":0.56699999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":7.7690000000000001,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.8659999999999997,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":0.61699999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":7.8360000000000003,"5min":7.8360000000000003,"15min":7.8360000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":6.6660000000000004,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.65600000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":6.7160000000000002,"5min":6.7160000000000002,"15min":6.7160000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":7.7370000000000001,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.54600000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297880,"kb_used_data":592,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073960,"statfs":{"total":96636764160,"available":96331735040,"internally_reserved":0,"allocated":606208,"data_stored":252097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.38300000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":11.967000000000001,"5min":11.967000000000001,"15min":11.967000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.1100000000000003,"5min":4.1100000000000003,"15min":4.1100000000000003},"last":0.41899999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":6.3040000000000003,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":11.935,"5min":11.935,"15min":11.935},"last":0.438}]},{"osd":4,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.71899999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.748},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57099999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.68799999999999994}]},{"osd":7,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":6.2649999999999997,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.76400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297880,"kb_used_data":592,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073960,"statfs":{"total":96636764160,"available":96331735040,"internally_reserved":0,"allocated":606208,"data_stored":252097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.317,"15min":0.317},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.436,"5min":0.436,"15min":0.436},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.318}]},{"osd":1,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.38900000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.34499999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.58499999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.57599999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.71599999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.56699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297884,"kb_used_data":596,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073956,"statfs":{"total":96636764160,"available":96331730944,"internally_reserved":0,"allocated":610304,"data_stored":253577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":7.6719999999999997,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.52400000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":7.7160000000000002,"5min":7.7160000000000002,"15min":7.7160000000000002},"last":0.625},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":7.7839999999999998,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.72499999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.68000000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.75}]},{"osd":6,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.83299999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.85699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297880,"kb_used_data":592,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073960,"statfs":{"total":96636764160,"available":96331735040,"internally_reserved":0,"allocated":606208,"data_stored":252665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.441},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.56299999999999994}]},{"osd":1,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":5.2679999999999998,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":5.3109999999999999,"5min":5.3109999999999999,"15min":5.3109999999999999},"last":0.52200000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.2960000000000003,"5min":5.2960000000000003,"15min":5.2960000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.504}]},{"osd":3,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":5.3330000000000002,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.2850000000000001,"5min":5.2850000000000001,"15min":5.2850000000000001},"last":0.57599999999999996}]},{"osd":4,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.68300000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.69399999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.71999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:36:00.373 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:36:00.374 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:36:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.526+0000 7f3bb6c2e700 1 Processor -- start 2023-10-10T20:36:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.526+0000 7f3bb6c2e700 1 -- start start 2023-10-10T20:36:00.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.527+0000 7f3bb6c2e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3bb0159950 0x7f3bb00cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.527+0000 7f3bb6c2e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3bb00af320 0x7f3bb0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.527+0000 7f3bb6c2e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb00ce380 con 0x7f3bb0069aa0 2023-10-10T20:36:00.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.527+0000 7f3bb6c2e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb00cec20 con 0x7f3bb00af320 2023-10-10T20:36:00.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.527+0000 7f3bb6c2e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb0069ee0 con 0x7f3bb0159950 2023-10-10T20:36:00.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.527+0000 7f3ba7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3bb00af320 0x7f3bb0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:00.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.527+0000 7f3ba7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3bb00af320 0x7f3bb0069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41030/0 (socket says 172.21.15.89:41030) 2023-10-10T20:36:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.527+0000 7f3ba7fff700 1 -- 172.21.15.89:0/3364497164 learned_addr learned my addr 172.21.15.89:0/3364497164 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.527+0000 7f3ba7fff700 1 -- 172.21.15.89:0/3364497164 >> v2:172.21.15.111:3300/0 conn(0x7f3bb0159950 msgr2=0x7f3bb00cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.527+0000 7f3ba7fff700 1 --2- 172.21.15.89:0/3364497164 >> v2:172.21.15.111:3300/0 conn(0x7f3bb0159950 0x7f3bb00cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.527+0000 7f3ba7fff700 1 -- 172.21.15.89:0/3364497164 >> v1:172.21.15.111:6789/0 conn(0x7f3bb0069aa0 legacy=0x7f3bb015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.527+0000 7f3ba7fff700 1 -- 172.21.15.89:0/3364497164 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb00afa90 con 0x7f3bb00af320 2023-10-10T20:36:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.528+0000 7f3ba7fff700 1 --2- 172.21.15.89:0/3364497164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bb00af320 0x7f3bb0069560 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00896e0 tx=0x7f3ba0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fe625331db3b01 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.528+0000 7f3ba77fe700 1 -- 172.21.15.89:0/3364497164 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000b820 con 0x7f3bb00af320 2023-10-10T20:36:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.528+0000 7f3ba77fe700 1 -- 172.21.15.89:0/3364497164 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba000b980 con 0x7f3bb00af320 2023-10-10T20:36:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.528+0000 7f3ba77fe700 1 -- 172.21.15.89:0/3364497164 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000bd20 con 0x7f3bb00af320 2023-10-10T20:36:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.528+0000 7f3bb6c2e700 1 -- 172.21.15.89:0/3364497164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bb00af320 msgr2=0x7f3bb0069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.528+0000 7f3bb6c2e700 1 --2- 172.21.15.89:0/3364497164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bb00af320 0x7f3bb0069560 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00896e0 tx=0x7f3ba0009250 comp rx=0 tx=0).stop 2023-10-10T20:36:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.528+0000 7f3bb6c2e700 1 -- 172.21.15.89:0/3364497164 shutdown_connections 2023-10-10T20:36:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.528+0000 7f3bb6c2e700 1 --2- 172.21.15.89:0/3364497164 >> v2:172.21.15.111:3300/0 conn(0x7f3bb0159950 0x7f3bb00cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.528+0000 7f3bb6c2e700 1 --2- 172.21.15.89:0/3364497164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bb00af320 0x7f3bb0069560 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.528+0000 7f3bb6c2e700 1 -- 172.21.15.89:0/3364497164 >> 172.21.15.89:0/3364497164 conn(0x7f3bb0068e00 msgr2=0x7f3bb00d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:00.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.528+0000 7f3bb6c2e700 1 -- 172.21.15.89:0/3364497164 shutdown_connections 2023-10-10T20:36:00.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.528+0000 7f3bb6c2e700 1 -- 172.21.15.89:0/3364497164 wait complete. 2023-10-10T20:36:00.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.529+0000 7f3bb6c2e700 1 Processor -- start 2023-10-10T20:36:00.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.530+0000 7f3bb6c2e700 1 -- start start 2023-10-10T20:36:00.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.530+0000 7f3bb6c2e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bb0069aa0 0x7f3bb00c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.530+0000 7f3bb6c2e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bb00af320 0x7f3bb014d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.530+0000 7f3bb6c2e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bb0159950 0x7f3bb00c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.530+0000 7f3bb6c2e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb00c9e80 con 0x7f3bb00af320 2023-10-10T20:36:00.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.530+0000 7f3bb6c2e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb00c9270 con 0x7f3bb0159950 2023-10-10T20:36:00.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.530+0000 7f3bb6c2e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb00c9550 con 0x7f3bb0069aa0 2023-10-10T20:36:00.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.530+0000 7f3ba7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bb00af320 0x7f3bb014d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:00.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.530+0000 7f3ba7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bb00af320 0x7f3bb014d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41042/0 (socket says 172.21.15.89:41042) 2023-10-10T20:36:00.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.530+0000 7f3ba7fff700 1 -- 172.21.15.89:0/3449018214 learned_addr learned my addr 172.21.15.89:0/3449018214 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:00.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.531+0000 7f3bb51cb700 1 --2- 172.21.15.89:0/3449018214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bb0159950 0x7f3bb00c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:00.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.531+0000 7f3ba7fff700 1 -- 172.21.15.89:0/3449018214 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bb0069aa0 msgr2=0x7f3bb00c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:00.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.531+0000 7f3ba7fff700 1 --2- 172.21.15.89:0/3449018214 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bb0069aa0 0x7f3bb00c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.531+0000 7f3ba7fff700 1 -- 172.21.15.89:0/3449018214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bb0159950 msgr2=0x7f3bb00c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:00.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.531+0000 7f3ba7fff700 1 --2- 172.21.15.89:0/3449018214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bb0159950 0x7f3bb00c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.531+0000 7f3ba7fff700 1 -- 172.21.15.89:0/3449018214 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba00094f0 con 0x7f3bb00af320 2023-10-10T20:36:00.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.531+0000 7f3bb51cb700 1 --2- 172.21.15.89:0/3449018214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bb0159950 0x7f3bb00c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:00.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.531+0000 7f3ba7fff700 1 --2- 172.21.15.89:0/3449018214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bb00af320 0x7f3bb014d640 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009290 tx=0x7f3ba0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:00.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.531+0000 7f3ba4ff9700 1 -- 172.21.15.89:0/3449018214 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba001a460 con 0x7f3bb00af320 2023-10-10T20:36:00.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.531+0000 7f3ba4ff9700 1 -- 172.21.15.89:0/3449018214 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba001a5c0 con 0x7f3bb00af320 2023-10-10T20:36:00.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.531+0000 7f3ba4ff9700 1 -- 172.21.15.89:0/3449018214 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba001a980 con 0x7f3bb00af320 2023-10-10T20:36:00.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.531+0000 7f3bb6c2e700 1 -- 172.21.15.89:0/3449018214 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb00c9830 con 0x7f3bb00af320 2023-10-10T20:36:00.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.531+0000 7f3bb6c2e700 1 -- 172.21.15.89:0/3449018214 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb014db80 con 0x7f3bb00af320 2023-10-10T20:36:00.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.532+0000 7f3bb6c2e700 1 -- 172.21.15.89:0/3449018214 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb00718b0 con 0x7f3bb00af320 2023-10-10T20:36:00.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.532+0000 7f3ba4ff9700 1 -- 172.21.15.89:0/3449018214 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ba001b440 con 0x7f3bb00af320 2023-10-10T20:36:00.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.533+0000 7f3ba4ff9700 1 --2- 172.21.15.89:0/3449018214 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3b9c066f30 0x7f3b9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.533+0000 7f3bb49ca700 1 --2- 172.21.15.89:0/3449018214 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3b9c066f30 0x7f3b9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:00.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.533+0000 7f3ba4ff9700 1 -- 172.21.15.89:0/3449018214 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3ba0085f40 con 0x7f3bb00af320 2023-10-10T20:36:00.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.533+0000 7f3bb49ca700 1 --2- 172.21.15.89:0/3449018214 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3b9c066f30 0x7f3b9c0693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00acf0 tx=0x7f3bac009370 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:00.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.537+0000 7f3ba4ff9700 1 -- 172.21.15.89:0/3449018214 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ba00531b0 con 0x7f3bb00af320 2023-10-10T20:36:00.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.794+0000 7f3bb6c2e700 1 -- 172.21.15.89:0/3449018214 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3bb00ce5d0 con 0x7f3bb00af320 2023-10-10T20:36:00.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.797+0000 7f3ba4ff9700 1 -- 172.21.15.89:0/3449018214 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7f3ba00200e0 con 0x7f3bb00af320 2023-10-10T20:36:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.799+0000 7f3bb6c2e700 1 -- 172.21.15.89:0/3449018214 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3b9c066f30 msgr2=0x7f3b9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.799+0000 7f3bb6c2e700 1 --2- 172.21.15.89:0/3449018214 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3b9c066f30 0x7f3b9c0693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00acf0 tx=0x7f3bac009370 comp rx=0 tx=0).stop 2023-10-10T20:36:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.799+0000 7f3bb6c2e700 1 -- 172.21.15.89:0/3449018214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bb00af320 msgr2=0x7f3bb014d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.799+0000 7f3bb6c2e700 1 --2- 172.21.15.89:0/3449018214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bb00af320 0x7f3bb014d640 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009290 tx=0x7f3ba0016000 comp rx=0 tx=0).stop 2023-10-10T20:36:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.799+0000 7f3bb6c2e700 1 -- 172.21.15.89:0/3449018214 shutdown_connections 2023-10-10T20:36:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.799+0000 7f3bb6c2e700 1 --2- 172.21.15.89:0/3449018214 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3b9c066f30 0x7f3b9c0693e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.799+0000 7f3bb6c2e700 1 --2- 172.21.15.89:0/3449018214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bb0159950 0x7f3bb00c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.799+0000 7f3bb6c2e700 1 --2- 172.21.15.89:0/3449018214 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bb0069aa0 0x7f3bb00c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.799+0000 7f3bb6c2e700 1 --2- 172.21.15.89:0/3449018214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bb00af320 0x7f3bb014d640 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.799+0000 7f3bb6c2e700 1 -- 172.21.15.89:0/3449018214 >> 172.21.15.89:0/3449018214 conn(0x7f3bb0068e00 msgr2=0x7f3bb01538f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.799+0000 7f3bb6c2e700 1 -- 172.21.15.89:0/3449018214 shutdown_connections 2023-10-10T20:36:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.799+0000 7f3bb6c2e700 1 -- 172.21.15.89:0/3449018214 wait complete. 2023-10-10T20:36:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:updated caps for client.0 2023-10-10T20:36:00.813 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:36:00.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.925+0000 7fbdcc5d7700 1 Processor -- start 2023-10-10T20:36:00.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.925+0000 7fbdcc5d7700 1 -- start start 2023-10-10T20:36:00.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.926+0000 7fbdcc5d7700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbdc4159890 0x7fbdc415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.926+0000 7fbdcc5d7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc415a160 con 0x7fbdc4159c60 2023-10-10T20:36:00.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.926+0000 7fbdcc5d7700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc40d13a0 con 0x7fbdc4069560 2023-10-10T20:36:00.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.926+0000 7fbdcc5d7700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc415e140 con 0x7fbdc4159890 2023-10-10T20:36:00.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.926+0000 7fbdc9b72700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbdc4159c60 0x7fbdc415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58790/0 (socket says 172.21.15.89:58790) 2023-10-10T20:36:00.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.926+0000 7fbdc9b72700 1 -- 172.21.15.89:0/2644862753 learned_addr learned my addr 172.21.15.89:0/2644862753 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:00.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9371700 1 -- 172.21.15.89:0/2644862753 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3868181157 0 0) 0x7fbdc415a160 con 0x7fbdc4159c60 2023-10-10T20:36:00.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdca373700 1 --2- 172.21.15.89:0/2644862753 >> v2:172.21.15.111:3300/0 conn(0x7fbdc4159890 0x7fbdc415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9371700 1 -- 172.21.15.89:0/2644862753 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdb0003570 con 0x7fbdc4159c60 2023-10-10T20:36:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9371700 1 -- 172.21.15.89:0/2644862753 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1114874909 0 0) 0x7fbdb0003570 con 0x7fbdc4159c60 2023-10-10T20:36:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9371700 1 -- 172.21.15.89:0/2644862753 >> v2:172.21.15.111:3300/0 conn(0x7fbdc4159890 msgr2=0x7fbdc415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9371700 1 --2- 172.21.15.89:0/2644862753 >> v2:172.21.15.111:3300/0 conn(0x7fbdc4159890 0x7fbdc415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9371700 1 -- 172.21.15.89:0/2644862753 >> v1:172.21.15.111:6789/0 conn(0x7fbdc4069560 legacy=0x7fbdc40ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9371700 1 -- 172.21.15.89:0/2644862753 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc415a3d0 con 0x7fbdc4159c60 2023-10-10T20:36:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9371700 1 -- 172.21.15.89:0/2644862753 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fbdb4001470 con 0x7fbdc4159c60 2023-10-10T20:36:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9371700 1 -- 172.21.15.89:0/2644862753 >> v1:172.21.15.89:6789/0 conn(0x7fbdc4159c60 legacy=0x7fbdc415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9371700 1 --2- 172.21.15.89:0/2644862753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb0003a60 0x7fbdb0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9371700 1 --2- 172.21.15.89:0/2644862753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb0004810 0x7fbdb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9371700 1 --2- 172.21.15.89:0/2644862753 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdb00051c0 0x7fbdb000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9371700 1 -- 172.21.15.89:0/2644862753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc415e140 con 0x7fbdb0004810 2023-10-10T20:36:00.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9371700 1 -- 172.21.15.89:0/2644862753 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb000c0d0 con 0x7fbdb0003a60 2023-10-10T20:36:00.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9371700 1 -- 172.21.15.89:0/2644862753 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb000c290 con 0x7fbdb00051c0 2023-10-10T20:36:00.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9b72700 1 --2- 172.21.15.89:0/2644862753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb0004810 0x7fbdb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:00.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdca373700 1 --2- 172.21.15.89:0/2644862753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb0003a60 0x7fbdb0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:00.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdcab74700 1 --2- 172.21.15.89:0/2644862753 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdb00051c0 0x7fbdb000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:00.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9b72700 1 -- 172.21.15.89:0/2644862753 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdb00051c0 msgr2=0x7fbdb000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:00.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9b72700 1 --2- 172.21.15.89:0/2644862753 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdb00051c0 0x7fbdb000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9b72700 1 -- 172.21.15.89:0/2644862753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb0003a60 msgr2=0x7fbdb0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9b72700 1 --2- 172.21.15.89:0/2644862753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb0003a60 0x7fbdb0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9b72700 1 -- 172.21.15.89:0/2644862753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb0003570 con 0x7fbdb0004810 2023-10-10T20:36:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdcab74700 1 -- 172.21.15.89:0/2644862753 reap_dead start 2023-10-10T20:36:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.927+0000 7fbdc9b72700 1 --2- 172.21.15.89:0/2644862753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb0004810 0x7fbdb0004c80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400c3c0 tx=0x7fbdb4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e972f4a351bffb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:00.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.928+0000 7fbdc9371700 1 -- 172.21.15.89:0/2644862753 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb400cc90 con 0x7fbdb0004810 2023-10-10T20:36:00.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.928+0000 7fbdc9371700 1 -- 172.21.15.89:0/2644862753 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb400cdf0 con 0x7fbdb0004810 2023-10-10T20:36:00.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.928+0000 7fbdc9371700 1 -- 172.21.15.89:0/2644862753 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4005b50 con 0x7fbdb0004810 2023-10-10T20:36:00.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.928+0000 7fbdcc5d7700 1 -- 172.21.15.89:0/2644862753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb0004810 msgr2=0x7fbdb0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:00.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.928+0000 7fbdcc5d7700 1 --2- 172.21.15.89:0/2644862753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb0004810 0x7fbdb0004c80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400c3c0 tx=0x7fbdb4001e50 comp rx=0 tx=0).stop 2023-10-10T20:36:00.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.928+0000 7fbdcc5d7700 1 -- 172.21.15.89:0/2644862753 shutdown_connections 2023-10-10T20:36:00.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.928+0000 7fbdcc5d7700 1 --2- 172.21.15.89:0/2644862753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb0004810 0x7fbdb0004c80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.928+0000 7fbdcc5d7700 1 -- 172.21.15.89:0/2644862753 >> 172.21.15.89:0/2644862753 conn(0x7fbdc4068e20 msgr2=0x7fbdc40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:00.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.928+0000 7fbdcc5d7700 1 -- 172.21.15.89:0/2644862753 shutdown_connections 2023-10-10T20:36:00.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.928+0000 7fbdcc5d7700 1 -- 172.21.15.89:0/2644862753 wait complete. 2023-10-10T20:36:00.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.929+0000 7fbdcc5d7700 1 Processor -- start 2023-10-10T20:36:00.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.929+0000 7fbdcc5d7700 1 -- start start 2023-10-10T20:36:00.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.930+0000 7fbdcc5d7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb0004810 0x7fbdc4156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.930+0000 7fbdcc5d7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdc41587b0 0x7fbdc40cf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.930+0000 7fbdcc5d7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdc41494d0 0x7fbdc4152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.930+0000 7fbdcc5d7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc4156b70 con 0x7fbdb0004810 2023-10-10T20:36:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.930+0000 7fbdcc5d7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40cd800 con 0x7fbdc41587b0 2023-10-10T20:36:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.930+0000 7fbdcc5d7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40cd1f0 con 0x7fbdc41494d0 2023-10-10T20:36:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.930+0000 7fbdca373700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb0004810 0x7fbdc4156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.930+0000 7fbdca373700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb0004810 0x7fbdc4156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41058/0 (socket says 172.21.15.89:41058) 2023-10-10T20:36:00.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.930+0000 7fbdca373700 1 -- 172.21.15.89:0/661699312 learned_addr learned my addr 172.21.15.89:0/661699312 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:00.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.930+0000 7fbdca373700 1 -- 172.21.15.89:0/661699312 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdc41494d0 msgr2=0x7fbdc4152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:00.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.930+0000 7fbdca373700 1 --2- 172.21.15.89:0/661699312 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdc41494d0 0x7fbdc4152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.930+0000 7fbdca373700 1 -- 172.21.15.89:0/661699312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdc41587b0 msgr2=0x7fbdc40cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:00.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.930+0000 7fbdca373700 1 --2- 172.21.15.89:0/661699312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdc41587b0 0x7fbdc40cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:00.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.930+0000 7fbdca373700 1 -- 172.21.15.89:0/661699312 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc40cde20 con 0x7fbdb0004810 2023-10-10T20:36:00.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.931+0000 7fbdca373700 1 --2- 172.21.15.89:0/661699312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb0004810 0x7fbdc4156630 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0011f80 tx=0x7fbdc000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:00.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.931+0000 7fbdba7fc700 1 -- 172.21.15.89:0/661699312 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc000e040 con 0x7fbdb0004810 2023-10-10T20:36:00.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.931+0000 7fbdba7fc700 1 -- 172.21.15.89:0/661699312 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc000f040 con 0x7fbdb0004810 2023-10-10T20:36:00.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.931+0000 7fbdcc5d7700 1 -- 172.21.15.89:0/661699312 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc40cd490 con 0x7fbdb0004810 2023-10-10T20:36:00.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.931+0000 7fbdba7fc700 1 -- 172.21.15.89:0/661699312 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0015430 con 0x7fbdb0004810 2023-10-10T20:36:00.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.931+0000 7fbdcc5d7700 1 -- 172.21.15.89:0/661699312 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc40ce100 con 0x7fbdb0004810 2023-10-10T20:36:00.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.931+0000 7fbdcc5d7700 1 -- 172.21.15.89:0/661699312 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc404c3c0 con 0x7fbdb0004810 2023-10-10T20:36:00.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.932+0000 7fbdba7fc700 1 -- 172.21.15.89:0/661699312 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbdc0015860 con 0x7fbdb0004810 2023-10-10T20:36:00.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.932+0000 7fbdba7fc700 1 --2- 172.21.15.89:0/661699312 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbda8066f30 0x7fbda80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:00.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.932+0000 7fbdba7fc700 1 -- 172.21.15.89:0/661699312 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbdc0085260 con 0x7fbdb0004810 2023-10-10T20:36:00.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.932+0000 7fbdc9b72700 1 --2- 172.21.15.89:0/661699312 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbda8066f30 0x7fbda80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:00.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.933+0000 7fbdc9b72700 1 --2- 172.21.15.89:0/661699312 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbda8066f30 0x7fbda80693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbdc415b5d0 tx=0x7fbdb4015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:00.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:00.936+0000 7fbdba7fc700 1 -- 172.21.15.89:0/661699312 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbdc00524d0 con 0x7fbdb0004810 2023-10-10T20:36:01.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.162+0000 7fbdcc5d7700 1 -- 172.21.15.89:0/661699312 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbdc406a5b0 con 0x7fbdb0004810 2023-10-10T20:36:01.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.162+0000 7fbdba7fc700 1 -- 172.21.15.89:0/661699312 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4597 (secure 0 0 0) 0x7fbdc0055af0 con 0x7fbdb0004810 2023-10-10T20:36:01.165 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:01.165 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4922,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1464385125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1464385125},{"type":"v1","addr":"172.21.15.111:6834","nonce":1464385125}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":4925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/689169706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":689169706},{"type":"v1","addr":"172.21.15.111:6835","nonce":689169706}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/418769006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":418769006},{"type":"v1","addr":"172.21.15.89:6836","nonce":418769006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:35:55.314480+0000","modified":"2023-10-10T20:35:58.961430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5340":{"gid":5340,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/3245027879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":3245027879},{"type":"v1","addr":"172.21.15.89:6837","nonce":3245027879}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-10T20:36:01.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.164+0000 7fbdcc5d7700 1 -- 172.21.15.89:0/661699312 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbda8066f30 msgr2=0x7fbda80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:01.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.164+0000 7fbdcc5d7700 1 --2- 172.21.15.89:0/661699312 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbda8066f30 0x7fbda80693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbdc415b5d0 tx=0x7fbdb4015000 comp rx=0 tx=0).stop 2023-10-10T20:36:01.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.164+0000 7fbdcc5d7700 1 -- 172.21.15.89:0/661699312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb0004810 msgr2=0x7fbdc4156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:01.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.164+0000 7fbdcc5d7700 1 --2- 172.21.15.89:0/661699312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb0004810 0x7fbdc4156630 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0011f80 tx=0x7fbdc000b040 comp rx=0 tx=0).stop 2023-10-10T20:36:01.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.164+0000 7fbdcc5d7700 1 -- 172.21.15.89:0/661699312 shutdown_connections 2023-10-10T20:36:01.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.164+0000 7fbdcc5d7700 1 --2- 172.21.15.89:0/661699312 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdc41494d0 0x7fbdc4152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.164+0000 7fbdcc5d7700 1 --2- 172.21.15.89:0/661699312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb0004810 0x7fbdc4156630 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.164+0000 7fbdcc5d7700 1 --2- 172.21.15.89:0/661699312 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbda8066f30 0x7fbda80693e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.164+0000 7fbdcc5d7700 1 --2- 172.21.15.89:0/661699312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdc41587b0 0x7fbdc40cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.164+0000 7fbdcc5d7700 1 -- 172.21.15.89:0/661699312 >> 172.21.15.89:0/661699312 conn(0x7fbdc4068e20 msgr2=0x7fbdc40c5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:01.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.164+0000 7fbdcc5d7700 1 -- 172.21.15.89:0/661699312 shutdown_connections 2023-10-10T20:36:01.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.164+0000 7fbdcc5d7700 1 -- 172.21.15.89:0/661699312 wait complete. 2023-10-10T20:36:01.170 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 29 2023-10-10T20:36:01.240 INFO:tasks.ceph.mds.a.smithi089.stderr:2023-10-10T20:36:01.238+0000 7fbaf9a29700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:36:01.582 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:35:55.314480+0000', 'modified': '2023-10-10T20:35:58.961430+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5340}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5340': {'gid': 5340, 'name': 'a', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6837/3245027879', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6835', 'nonce': 3245027879}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 3245027879}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:36:01.583 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:36:01.583 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-10T20:36:01.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.699+0000 7f6e6d795700 1 Processor -- start 2023-10-10T20:36:01.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.699+0000 7f6e6d795700 1 -- start start 2023-10-10T20:36:01.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.700+0000 7f6e6d795700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6e68159c60 0x7f6e6815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.700+0000 7f6e6d795700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6815a160 con 0x7f6e68159890 2023-10-10T20:36:01.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.700+0000 7f6e6d795700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e680d13a0 con 0x7f6e6815db10 2023-10-10T20:36:01.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.700+0000 7f6e6d795700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6815df50 con 0x7f6e68159c60 2023-10-10T20:36:01.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.700+0000 7f6e66ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6e68159890 0x7f6e6815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58802/0 (socket says 172.21.15.89:58802) 2023-10-10T20:36:01.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.700+0000 7f6e66ffd700 1 -- 172.21.15.89:0/4125962322 learned_addr learned my addr 172.21.15.89:0/4125962322 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:01.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.700+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/4125962322 >> v2:172.21.15.111:3300/0 conn(0x7f6e68159c60 0x7f6e6815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.700+0000 7f6e667fc700 1 -- 172.21.15.89:0/4125962322 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585709566 0 0) 0x7f6e6815a160 con 0x7f6e68159890 2023-10-10T20:36:01.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.700+0000 7f6e667fc700 1 -- 172.21.15.89:0/4125962322 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e54003570 con 0x7f6e68159890 2023-10-10T20:36:01.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.700+0000 7f6e667fc700 1 -- 172.21.15.89:0/4125962322 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3670675737 0 0) 0x7f6e54003570 con 0x7f6e68159890 2023-10-10T20:36:01.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.700+0000 7f6e667fc700 1 -- 172.21.15.89:0/4125962322 >> v2:172.21.15.111:3300/0 conn(0x7f6e68159c60 msgr2=0x7f6e6815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:01.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.700+0000 7f6e667fc700 1 --2- 172.21.15.89:0/4125962322 >> v2:172.21.15.111:3300/0 conn(0x7f6e68159c60 0x7f6e6815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.700+0000 7f6e667fc700 1 -- 172.21.15.89:0/4125962322 >> v1:172.21.15.111:6789/0 conn(0x7f6e6815db10 legacy=0x7f6e68069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:01.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.700+0000 7f6e667fc700 1 -- 172.21.15.89:0/4125962322 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6815e140 con 0x7f6e68159890 2023-10-10T20:36:01.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.700+0000 7f6e667fc700 1 -- 172.21.15.89:0/4125962322 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f6e58001470 con 0x7f6e68159890 2023-10-10T20:36:01.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.700+0000 7f6e667fc700 1 -- 172.21.15.89:0/4125962322 >> v1:172.21.15.89:6789/0 conn(0x7f6e68159890 legacy=0x7f6e6815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:01.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.701+0000 7f6e667fc700 1 --2- 172.21.15.89:0/4125962322 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e54003a60 0x7f6e54003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.701+0000 7f6e667fc700 1 --2- 172.21.15.89:0/4125962322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e54004810 0x7f6e54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.701+0000 7f6e667fc700 1 --2- 172.21.15.89:0/4125962322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e540051c0 0x7f6e5400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.701+0000 7f6e667fc700 1 -- 172.21.15.89:0/4125962322 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5400c0a0 con 0x7f6e54004810 2023-10-10T20:36:01.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.701+0000 7f6e667fc700 1 -- 172.21.15.89:0/4125962322 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5400c210 con 0x7f6e540051c0 2023-10-10T20:36:01.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.701+0000 7f6e667fc700 1 -- 172.21.15.89:0/4125962322 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5400c3d0 con 0x7f6e54003a60 2023-10-10T20:36:01.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.701+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/4125962322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e54004810 0x7f6e54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.701+0000 7f6e66ffd700 1 --2- 172.21.15.89:0/4125962322 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e54003a60 0x7f6e54003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.701+0000 7f6e677fe700 1 --2- 172.21.15.89:0/4125962322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e540051c0 0x7f6e5400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.701+0000 7f6e5e7fc700 1 -- 172.21.15.89:0/4125962322 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e54003a60 msgr2=0x7f6e54003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:01.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.701+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/4125962322 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e54003a60 0x7f6e54003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.701+0000 7f6e5e7fc700 1 -- 172.21.15.89:0/4125962322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e540051c0 msgr2=0x7f6e5400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:01.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.701+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/4125962322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e540051c0 0x7f6e5400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.701+0000 7f6e5e7fc700 1 -- 172.21.15.89:0/4125962322 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e54003570 con 0x7f6e54004810 2023-10-10T20:36:01.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.701+0000 7f6e677fe700 1 -- 172.21.15.89:0/4125962322 reap_dead start 2023-10-10T20:36:01.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.701+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/4125962322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e54004810 0x7f6e54004c80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009c00 tx=0x7f6e500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8307643ba86fbda5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:01.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.701+0000 7f6e667fc700 1 -- 172.21.15.89:0/4125962322 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5000e710 con 0x7f6e54004810 2023-10-10T20:36:01.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.701+0000 7f6e667fc700 1 -- 172.21.15.89:0/4125962322 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5000e870 con 0x7f6e54004810 2023-10-10T20:36:01.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.701+0000 7f6e667fc700 1 -- 172.21.15.89:0/4125962322 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5000ec30 con 0x7f6e54004810 2023-10-10T20:36:01.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.702+0000 7f6e6d795700 1 -- 172.21.15.89:0/4125962322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e54004810 msgr2=0x7f6e54004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:01.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.702+0000 7f6e6d795700 1 --2- 172.21.15.89:0/4125962322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e54004810 0x7f6e54004c80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009c00 tx=0x7f6e500091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.702+0000 7f6e6d795700 1 -- 172.21.15.89:0/4125962322 shutdown_connections 2023-10-10T20:36:01.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.702+0000 7f6e6d795700 1 --2- 172.21.15.89:0/4125962322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e54004810 0x7f6e54004c80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.702+0000 7f6e6d795700 1 -- 172.21.15.89:0/4125962322 >> 172.21.15.89:0/4125962322 conn(0x7f6e68068e20 msgr2=0x7f6e680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:01.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.702+0000 7f6e6d795700 1 -- 172.21.15.89:0/4125962322 shutdown_connections 2023-10-10T20:36:01.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.702+0000 7f6e6d795700 1 -- 172.21.15.89:0/4125962322 wait complete. 2023-10-10T20:36:01.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.703+0000 7f6e6d795700 1 Processor -- start 2023-10-10T20:36:01.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.703+0000 7f6e6d795700 1 -- start start 2023-10-10T20:36:01.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e6d795700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e54004810 0x7f6e680b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e6d795700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e6814d510 0x7f6e6815d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e6d795700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e6815d680 0x7f6e680cf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e6d795700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6815b800 con 0x7f6e54004810 2023-10-10T20:36:01.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e6d795700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e680cd0c0 con 0x7f6e6815d680 2023-10-10T20:36:01.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e6d795700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e680cd3a0 con 0x7f6e6814d510 2023-10-10T20:36:01.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e66ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e54004810 0x7f6e680b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e66ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e54004810 0x7f6e680b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41070/0 (socket says 172.21.15.89:41070) 2023-10-10T20:36:01.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e66ffd700 1 -- 172.21.15.89:0/1362653232 learned_addr learned my addr 172.21.15.89:0/1362653232 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:01.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/1362653232 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e6814d510 0x7f6e6815d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e677fe700 1 --2- 172.21.15.89:0/1362653232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e6815d680 0x7f6e680cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e66ffd700 1 -- 172.21.15.89:0/1362653232 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e6814d510 msgr2=0x7f6e6815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:01.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e66ffd700 1 --2- 172.21.15.89:0/1362653232 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e6814d510 0x7f6e6815d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e66ffd700 1 -- 172.21.15.89:0/1362653232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e6815d680 msgr2=0x7f6e680cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:01.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e66ffd700 1 --2- 172.21.15.89:0/1362653232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e6815d680 0x7f6e680cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e66ffd700 1 -- 172.21.15.89:0/1362653232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e680cd680 con 0x7f6e54004810 2023-10-10T20:36:01.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e677fe700 1 --2- 172.21.15.89:0/1362653232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e6815d680 0x7f6e680cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:01.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e66ffd700 1 --2- 172.21.15.89:0/1362653232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e54004810 0x7f6e680b3590 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6e58007370 tx=0x7f6e58001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:01.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e5f7fe700 1 -- 172.21.15.89:0/1362653232 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e58018030 con 0x7f6e54004810 2023-10-10T20:36:01.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e5f7fe700 1 -- 172.21.15.89:0/1362653232 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5800f8d0 con 0x7f6e54004810 2023-10-10T20:36:01.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e6d795700 1 -- 172.21.15.89:0/1362653232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e680cd960 con 0x7f6e54004810 2023-10-10T20:36:01.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e5f7fe700 1 -- 172.21.15.89:0/1362653232 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5800fc90 con 0x7f6e54004810 2023-10-10T20:36:01.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.704+0000 7f6e6d795700 1 -- 172.21.15.89:0/1362653232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e680b3ad0 con 0x7f6e54004810 2023-10-10T20:36:01.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.706+0000 7f6e5f7fe700 1 -- 172.21.15.89:0/1362653232 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6e5801b070 con 0x7f6e54004810 2023-10-10T20:36:01.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.706+0000 7f6e5f7fe700 1 --2- 172.21.15.89:0/1362653232 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6e48066f30 0x7f6e480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.706+0000 7f6e5f7fe700 1 -- 172.21.15.89:0/1362653232 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6e58019070 con 0x7f6e54004810 2023-10-10T20:36:01.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.707+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/1362653232 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6e48066f30 0x7f6e480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.707+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/1362653232 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6e48066f30 0x7f6e480693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6e500091e0 tx=0x7f6e5001a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:01.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.708+0000 7f6e76a8eb80 1 Processor -- start 2023-10-10T20:36:01.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.708+0000 7f6e76a8eb80 1 -- start start 2023-10-10T20:36:01.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.709+0000 7f6e76a8eb80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a03dba50 0x55a0a03dbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.709+0000 7f6e76a8eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0425d00 0x55a0a0404a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.709+0000 7f6e76a8eb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0404f60 0x55a0a03d7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.709+0000 7f6e76a8eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0a0332a70 con 0x55a0a0425d00 2023-10-10T20:36:01.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.709+0000 7f6e76a8eb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0a0330270 con 0x55a0a0404f60 2023-10-10T20:36:01.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.709+0000 7f6e76a8eb80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0a032c060 con 0x55a0a03dba50 2023-10-10T20:36:01.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.709+0000 7f6e5e7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0425d00 0x55a0a0404a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.709+0000 7f6e5e7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0425d00 0x55a0a0404a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41074/0 (socket says 172.21.15.89:41074) 2023-10-10T20:36:01.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.709+0000 7f6e5e7fc700 1 -- 172.21.15.89:0/1278079904 learned_addr learned my addr 172.21.15.89:0/1278079904 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:01.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.709+0000 7f6e66ffd700 1 --2- 172.21.15.89:0/1278079904 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a03dba50 0x55a0a03dbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.709+0000 7f6e677fe700 1 --2- 172.21.15.89:0/1278079904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0404f60 0x55a0a03d7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.709+0000 7f6e5e7fc700 1 -- 172.21.15.89:0/1278079904 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a03dba50 msgr2=0x55a0a03dbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:01.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.709+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/1278079904 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a03dba50 0x55a0a03dbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.709+0000 7f6e5e7fc700 1 -- 172.21.15.89:0/1278079904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0404f60 msgr2=0x55a0a03d7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:01.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.709+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/1278079904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0404f60 0x55a0a03d7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.709+0000 7f6e5e7fc700 1 -- 172.21.15.89:0/1278079904 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0a01c6270 con 0x55a0a0425d00 2023-10-10T20:36:01.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.709+0000 7f6e66ffd700 1 --2- 172.21.15.89:0/1278079904 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a03dba50 0x55a0a03dbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:01.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.710+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/1278079904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0425d00 0x55a0a0404a20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e4b0 tx=0x7f6e5001f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e9973ad3ebaee server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:01.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.710+0000 7f6e47fff700 1 -- 172.21.15.89:0/1278079904 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50030070 con 0x55a0a0425d00 2023-10-10T20:36:01.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.710+0000 7f6e47fff700 1 -- 172.21.15.89:0/1278079904 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e50023040 con 0x55a0a0425d00 2023-10-10T20:36:01.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.710+0000 7f6e47fff700 1 -- 172.21.15.89:0/1278079904 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50013470 con 0x55a0a0425d00 2023-10-10T20:36:01.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.710+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/1278079904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0425d00 msgr2=0x55a0a0404a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:01.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.710+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/1278079904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0425d00 0x55a0a0404a20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e4b0 tx=0x7f6e5001f050 comp rx=0 tx=0).stop 2023-10-10T20:36:01.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.710+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/1278079904 shutdown_connections 2023-10-10T20:36:01.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.710+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/1278079904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0404f60 0x55a0a03d7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.710+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/1278079904 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a03dba50 0x55a0a03dbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.710+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/1278079904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0425d00 0x55a0a0404a20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.710+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/1278079904 >> 172.21.15.89:0/1278079904 conn(0x55a0a03dd370 msgr2=0x55a0a03dd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:01.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.710+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/1278079904 shutdown_connections 2023-10-10T20:36:01.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.710+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/1278079904 wait complete. 2023-10-10T20:36:01.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.711+0000 7f6e76a8eb80 1 -- start start 2023-10-10T20:36:01.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.711+0000 7f6e76a8eb80 1 Processor -- start 2023-10-10T20:36:01.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.712+0000 7f6e76a8eb80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a03dba50 0x55a0a0400e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.712+0000 7f6e76a8eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0404f60 0x55a0a0420100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.712+0000 7f6e76a8eb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0425d00 0x55a0a041be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.712+0000 7f6e76a8eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0a02d7890 con 0x55a0a0404f60 2023-10-10T20:36:01.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.712+0000 7f6e76a8eb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0a02f2ea0 con 0x55a0a0425d00 2023-10-10T20:36:01.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.712+0000 7f6e76a8eb80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0a01aee30 con 0x55a0a03dba50 2023-10-10T20:36:01.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.712+0000 7f6e76a8eb80 10 client.0 _ll_register_callbacks cb 0x55a0a039f110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:36:01.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.712+0000 7f6e5e7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0404f60 0x55a0a0420100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.712+0000 7f6e5e7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0404f60 0x55a0a0420100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41076/0 (socket says 172.21.15.89:41076) 2023-10-10T20:36:01.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.712+0000 7f6e5e7fc700 1 -- 172.21.15.89:0/2451796031 learned_addr learned my addr 172.21.15.89:0/2451796031 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:01.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.712+0000 7f6e677fe700 1 --2- 172.21.15.89:0/2451796031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0425d00 0x55a0a041be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.712+0000 7f6e66ffd700 1 --2- 172.21.15.89:0/2451796031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a03dba50 0x55a0a0400e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.712+0000 7f6e5e7fc700 1 -- 172.21.15.89:0/2451796031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a03dba50 msgr2=0x55a0a0400e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.712+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/2451796031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a03dba50 0x55a0a0400e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.712+0000 7f6e5e7fc700 1 -- 172.21.15.89:0/2451796031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0425d00 msgr2=0x55a0a041be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.712+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/2451796031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0425d00 0x55a0a041be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.712+0000 7f6e5e7fc700 1 -- 172.21.15.89:0/2451796031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e54003570 con 0x55a0a0404f60 2023-10-10T20:36:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.712+0000 7f6e66ffd700 1 --2- 172.21.15.89:0/2451796031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a03dba50 0x55a0a0400e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.712+0000 7f6e677fe700 1 --2- 172.21.15.89:0/2451796031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0425d00 0x55a0a041be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:01.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.713+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/2451796031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0404f60 0x55a0a0420100 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e5b0 tx=0x7f6e50030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d860984ff9f2590 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:01.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.713+0000 7f6e457fa700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:36:01.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.713+0000 7f6e457fa700 1 -- 172.21.15.89:0/2451796031 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50039070 con 0x55a0a0404f60 2023-10-10T20:36:01.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.713+0000 7f6e457fa700 1 -- 172.21.15.89:0/2451796031 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6e5003eac0 con 0x55a0a0404f60 2023-10-10T20:36:01.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.713+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/2451796031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a0a03d0b70 con 0x55a0a0404f60 2023-10-10T20:36:01.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.713+0000 7f6e457fa700 1 -- 172.21.15.89:0/2451796031 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e50013470 con 0x55a0a0404f60 2023-10-10T20:36:01.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.713+0000 7f6e76a8eb80 10 client.5481 fetch_fsmap learned FSMap version 29 2023-10-10T20:36:01.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.713+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/2451796031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55a0a0168fe0 con 0x55a0a0404f60 2023-10-10T20:36:01.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.713+0000 7f6e457fa700 1 -- 172.21.15.89:0/2451796031 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a0a03d0b70 con 0x55a0a0404f60 2023-10-10T20:36:01.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.713+0000 7f6e457fa700 1 -- 172.21.15.89:0/2451796031 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6e50044020 con 0x55a0a0404f60 2023-10-10T20:36:01.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.713+0000 7f6e76a8eb80 10 client.5481 fetch_fsmap finished waiting for FSMap version 29 2023-10-10T20:36:01.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.713+0000 7f6e76a8eb80 10 client.5481 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-10T20:36:01.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.713+0000 7f6e76a8eb80 20 client.5481 populate_metadata read hostname 'smithi089' 2023-10-10T20:36:01.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.713+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/2451796031 >> [v2:172.21.15.89:6835/3245027879,v1:172.21.15.89:6837/3245027879] conn(0x55a0a0435340 0x55a0a0435710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.713+0000 7f6e76a8eb80 4 client.5481 mds_command: new command op to 5340 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:36:01.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.713+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/2451796031 --> [v2:172.21.15.89:6835/3245027879,v1:172.21.15.89:6837/3245027879] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a0a03adaa0 con 0x55a0a0435340 2023-10-10T20:36:01.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.713+0000 7f6e66ffd700 1 --2- 172.21.15.89:0/2451796031 >> [v2:172.21.15.89:6835/3245027879,v1:172.21.15.89:6837/3245027879] conn(0x55a0a0435340 0x55a0a0435710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.714+0000 7f6e457fa700 10 client.5481 ms_handle_connect on v2:172.21.15.89:6835/3245027879 2023-10-10T20:36:01.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.714+0000 7f6e66ffd700 1 --2- 172.21.15.89:0/2451796031 >> [v2:172.21.15.89:6835/3245027879,v1:172.21.15.89:6837/3245027879] conn(0x55a0a0435340 0x55a0a0435710 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e5312d446a79647 server_cookie=dd8758114b33dd48 in_seq=0 out_seq=0 2023-10-10T20:36:01.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.714+0000 7f6e457fa700 1 -- 172.21.15.89:0/2451796031 <== mds.0 v2:172.21.15.89:6835/3245027879 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a0a03adaa0 con 0x55a0a0435340 2023-10-10T20:36:01.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.714+0000 7f6e457fa700 10 client.5481 handle_command_reply: tid=0 2023-10-10T20:36:01.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.714+0000 7f6e76a8eb80 1 client.5481 shutdown 2023-10-10T20:36:01.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.714+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/2451796031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0404f60 msgr2=0x55a0a0420100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:01.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.714+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/2451796031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0404f60 0x55a0a0420100 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e5b0 tx=0x7f6e50030000 comp rx=0 tx=0).stop 2023-10-10T20:36:01.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.714+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/2451796031 shutdown_connections 2023-10-10T20:36:01.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.714+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/2451796031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0425d00 0x55a0a041be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.714+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/2451796031 >> [v2:172.21.15.89:6835/3245027879,v1:172.21.15.89:6837/3245027879] conn(0x55a0a0435340 0x55a0a0435710 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.714+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/2451796031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a03dba50 0x55a0a0400e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.714+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/2451796031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0404f60 0x55a0a0420100 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.714+0000 7f6e457fa700 0 client.5481 ms_handle_reset on v2:172.21.15.89:6835/3245027879 2023-10-10T20:36:01.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.714+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/2451796031 >> 172.21.15.89:0/2451796031 conn(0x55a0a03dd370 msgr2=0x55a0a0421550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:01.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.714+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/2451796031 shutdown_connections 2023-10-10T20:36:01.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.714+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/2451796031 wait complete. 2023-10-10T20:36:01.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.715+0000 7f6e76a8eb80 20 client.5481 trim_cache size 0 max 16384 2023-10-10T20:36:01.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.731+0000 7f6e76a8eb80 1 Processor -- start 2023-10-10T20:36:01.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.731+0000 7f6e76a8eb80 1 -- start start 2023-10-10T20:36:01.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.732+0000 7f6e76a8eb80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a03dba50 0x55a0a043bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.732+0000 7f6e76a8eb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0404f60 0x55a0a04244d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.732+0000 7f6e76a8eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0425d00 0x55a0a0424a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.732+0000 7f6e76a8eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0a03d65a0 con 0x55a0a0425d00 2023-10-10T20:36:01.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.732+0000 7f6e76a8eb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0a03d74d0 con 0x55a0a0404f60 2023-10-10T20:36:01.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.732+0000 7f6e76a8eb80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0a0401370 con 0x55a0a03dba50 2023-10-10T20:36:01.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.732+0000 7f6e677fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0425d00 0x55a0a0424a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.732+0000 7f6e677fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0425d00 0x55a0a0424a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41088/0 (socket says 172.21.15.89:41088) 2023-10-10T20:36:01.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.732+0000 7f6e677fe700 1 -- 172.21.15.89:0/3015508260 learned_addr learned my addr 172.21.15.89:0/3015508260 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:01.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.732+0000 7f6e66ffd700 1 --2- 172.21.15.89:0/3015508260 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a03dba50 0x55a0a043bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.732+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/3015508260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0404f60 0x55a0a04244d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.732+0000 7f6e677fe700 1 -- 172.21.15.89:0/3015508260 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a03dba50 msgr2=0x55a0a043bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:01.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.732+0000 7f6e677fe700 1 --2- 172.21.15.89:0/3015508260 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a03dba50 0x55a0a043bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.732+0000 7f6e677fe700 1 -- 172.21.15.89:0/3015508260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0404f60 msgr2=0x55a0a04244d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:01.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.732+0000 7f6e677fe700 1 --2- 172.21.15.89:0/3015508260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0404f60 0x55a0a04244d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.732+0000 7f6e677fe700 1 -- 172.21.15.89:0/3015508260 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0a02fa780 con 0x55a0a0425d00 2023-10-10T20:36:01.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.732+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/3015508260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0404f60 0x55a0a04244d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:01.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.732+0000 7f6e677fe700 1 --2- 172.21.15.89:0/3015508260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0425d00 0x55a0a0424a10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6e600012e0 tx=0x7f6e60009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fc4ae693435e79 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:01.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.733+0000 7f6e5cff9700 1 -- 172.21.15.89:0/3015508260 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e60006ba0 con 0x55a0a0425d00 2023-10-10T20:36:01.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.733+0000 7f6e5cff9700 1 -- 172.21.15.89:0/3015508260 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e60006d00 con 0x55a0a0425d00 2023-10-10T20:36:01.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.733+0000 7f6e5cff9700 1 -- 172.21.15.89:0/3015508260 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e60010850 con 0x55a0a0425d00 2023-10-10T20:36:01.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.733+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/3015508260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0425d00 msgr2=0x55a0a0424a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:01.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.733+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/3015508260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0425d00 0x55a0a0424a10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6e600012e0 tx=0x7f6e60009040 comp rx=0 tx=0).stop 2023-10-10T20:36:01.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.733+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/3015508260 shutdown_connections 2023-10-10T20:36:01.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.733+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/3015508260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a0425d00 0x55a0a0424a10 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.733+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/3015508260 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a03dba50 0x55a0a043bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.733+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/3015508260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0404f60 0x55a0a04244d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.733+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/3015508260 >> 172.21.15.89:0/3015508260 conn(0x55a0a03dd370 msgr2=0x55a0a0421550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:01.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.733+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/3015508260 shutdown_connections 2023-10-10T20:36:01.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.733+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/3015508260 wait complete. 2023-10-10T20:36:01.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.734+0000 7f6e76a8eb80 1 -- start start 2023-10-10T20:36:01.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.734+0000 7f6e76a8eb80 1 Processor -- start 2023-10-10T20:36:01.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e76a8eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a03dba50 0x55a0a04415e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e76a8eb80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a0404f60 0x55a0a04874d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e76a8eb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0425d00 0x55a0a04347d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e76a8eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0a041e010 con 0x55a0a03dba50 2023-10-10T20:36:01.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e76a8eb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0a0444000 con 0x55a0a0425d00 2023-10-10T20:36:01.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e76a8eb80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0a0441ee0 con 0x55a0a0404f60 2023-10-10T20:36:01.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e76a8eb80 10 client.0 _ll_register_callbacks cb 0x55a0a039f110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:36:01.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e66ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a03dba50 0x55a0a04415e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e66ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a03dba50 0x55a0a04415e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41092/0 (socket says 172.21.15.89:41092) 2023-10-10T20:36:01.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e66ffd700 1 -- 172.21.15.89:0/1719295859 learned_addr learned my addr 172.21.15.89:0/1719295859 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:01.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/1719295859 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a0404f60 0x55a0a04874d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e677fe700 1 --2- 172.21.15.89:0/1719295859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0425d00 0x55a0a04347d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e66ffd700 1 -- 172.21.15.89:0/1719295859 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a0404f60 msgr2=0x55a0a04874d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:01.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e66ffd700 1 --2- 172.21.15.89:0/1719295859 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a0404f60 0x55a0a04874d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e66ffd700 1 -- 172.21.15.89:0/1719295859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0425d00 msgr2=0x55a0a04347d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:01.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e66ffd700 1 --2- 172.21.15.89:0/1719295859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0425d00 0x55a0a04347d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e66ffd700 1 -- 172.21.15.89:0/1719295859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6000f040 con 0x55a0a03dba50 2023-10-10T20:36:01.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/1719295859 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a0404f60 0x55a0a04874d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:01.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e677fe700 1 --2- 172.21.15.89:0/1719295859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0425d00 0x55a0a04347d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:01.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e66ffd700 1 --2- 172.21.15.89:0/1719295859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a03dba50 0x55a0a04415e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6e58051a70 tx=0x7f6e58006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9f99cd7dd85384 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:01.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e457fa700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:36:01.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e457fa700 1 -- 172.21.15.89:0/1719295859 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e580518b0 con 0x55a0a03dba50 2023-10-10T20:36:01.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e457fa700 1 -- 172.21.15.89:0/1719295859 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6e58082c40 con 0x55a0a03dba50 2023-10-10T20:36:01.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/1719295859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e50023040 con 0x55a0a03dba50 2023-10-10T20:36:01.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e457fa700 1 -- 172.21.15.89:0/1719295859 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e580557d0 con 0x55a0a03dba50 2023-10-10T20:36:01.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e76a8eb80 10 client.5487 fetch_fsmap learned FSMap version 29 2023-10-10T20:36:01.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.735+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/1719295859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55a0a02e1470 con 0x55a0a03dba50 2023-10-10T20:36:01.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.736+0000 7f6e457fa700 1 -- 172.21.15.89:0/1719295859 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e50023040 con 0x55a0a03dba50 2023-10-10T20:36:01.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.736+0000 7f6e457fa700 1 -- 172.21.15.89:0/1719295859 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6e58054a60 con 0x55a0a03dba50 2023-10-10T20:36:01.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.736+0000 7f6e76a8eb80 10 client.5487 fetch_fsmap finished waiting for FSMap version 29 2023-10-10T20:36:01.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.736+0000 7f6e76a8eb80 10 client.5487 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-10T20:36:01.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.736+0000 7f6e76a8eb80 20 client.5487 populate_metadata read hostname 'smithi089' 2023-10-10T20:36:01.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.736+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/1719295859 >> [v2:172.21.15.89:6835/3245027879,v1:172.21.15.89:6837/3245027879] conn(0x55a0a0424540 0x55a0a0424910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.736+0000 7f6e76a8eb80 4 client.5487 mds_command: new command op to 5340 tid=0[{"prefix": "status"}] 2023-10-10T20:36:01.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.736+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/1719295859 --> [v2:172.21.15.89:6835/3245027879,v1:172.21.15.89:6837/3245027879] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a0a02cee00 con 0x55a0a0424540 2023-10-10T20:36:01.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.736+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/1719295859 >> [v2:172.21.15.89:6835/3245027879,v1:172.21.15.89:6837/3245027879] conn(0x55a0a0424540 0x55a0a0424910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.736+0000 7f6e457fa700 10 client.5487 ms_handle_connect on v2:172.21.15.89:6835/3245027879 2023-10-10T20:36:01.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.736+0000 7f6e5e7fc700 1 --2- 172.21.15.89:0/1719295859 >> [v2:172.21.15.89:6835/3245027879,v1:172.21.15.89:6837/3245027879] conn(0x55a0a0424540 0x55a0a0424910 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f198ce4b07c5b29 server_cookie=8661adc5d42aa0f9 in_seq=0 out_seq=0 2023-10-10T20:36:01.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.736+0000 7f6e457fa700 1 -- 172.21.15.89:0/1719295859 <== mds.0 v2:172.21.15.89:6835/3245027879 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55a0a02cee00 con 0x55a0a0424540 2023-10-10T20:36:01.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.736+0000 7f6e457fa700 10 client.5487 handle_command_reply: tid=0 2023-10-10T20:36:01.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.736+0000 7f6e76a8eb80 1 client.5487 shutdown 2023-10-10T20:36:01.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.736+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/1719295859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a03dba50 msgr2=0x55a0a04415e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:01.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.736+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/1719295859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a03dba50 0x55a0a04415e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6e58051a70 tx=0x7f6e58006040 comp rx=0 tx=0).stop 2023-10-10T20:36:01.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/1719295859 shutdown_connections 2023-10-10T20:36:01.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/1719295859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a0425d00 0x55a0a04347d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/1719295859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0a03dba50 0x55a0a04415e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/1719295859 >> [v2:172.21.15.89:6835/3245027879,v1:172.21.15.89:6837/3245027879] conn(0x55a0a0424540 0x55a0a0424910 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e76a8eb80 1 --2- 172.21.15.89:0/1719295859 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a0a0404f60 0x55a0a04874d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e457fa700 0 client.5487 ms_handle_reset on v2:172.21.15.89:6835/3245027879 2023-10-10T20:36:01.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/1719295859 >> 172.21.15.89:0/1719295859 conn(0x55a0a03dd370 msgr2=0x55a0a0421550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:01.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/1719295859 shutdown_connections 2023-10-10T20:36:01.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e76a8eb80 1 -- 172.21.15.89:0/1719295859 wait complete. 2023-10-10T20:36:01.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e76a8eb80 20 client.5487 trim_cache size 0 max 16384 2023-10-10T20:36:01.765 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:36:01.765 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "5dd675b8-9026-4d8a-a1bf-62fdc5e7de67", 2023-10-10T20:36:01.765 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-10-10T20:36:01.765 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 5340, 2023-10-10T20:36:01.765 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-10-10T20:36:01.766 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-10-10T20:36:01.766 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-10-10T20:36:01.766 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 5.8080631150000004, 2023-10-10T20:36:01.766 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 29, 2023-10-10T20:36:01.766 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 50, 2023-10-10T20:36:01.767 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 50, 2023-10-10T20:36:01.767 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 10.906427214000001 2023-10-10T20:36:01.767 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:36:01.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e6d795700 1 -- 172.21.15.89:0/1362653232 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6e48066f30 msgr2=0x7f6e480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:01.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e6d795700 1 --2- 172.21.15.89:0/1362653232 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6e48066f30 0x7f6e480693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6e500091e0 tx=0x7f6e5001a000 comp rx=0 tx=0).stop 2023-10-10T20:36:01.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e6d795700 1 -- 172.21.15.89:0/1362653232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e54004810 msgr2=0x7f6e680b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:01.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e6d795700 1 --2- 172.21.15.89:0/1362653232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e54004810 0x7f6e680b3590 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6e58007370 tx=0x7f6e58001e50 comp rx=0 tx=0).stop 2023-10-10T20:36:01.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e6d795700 1 -- 172.21.15.89:0/1362653232 shutdown_connections 2023-10-10T20:36:01.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e6d795700 1 --2- 172.21.15.89:0/1362653232 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6e48066f30 0x7f6e480693e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e6d795700 1 --2- 172.21.15.89:0/1362653232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e6815d680 0x7f6e680cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e6d795700 1 --2- 172.21.15.89:0/1362653232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e54004810 0x7f6e680b3590 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e6d795700 1 --2- 172.21.15.89:0/1362653232 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e6814d510 0x7f6e6815d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e6d795700 1 -- 172.21.15.89:0/1362653232 >> 172.21.15.89:0/1362653232 conn(0x7f6e68068e20 msgr2=0x7f6e680c6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:01.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e6d795700 1 -- 172.21.15.89:0/1362653232 shutdown_connections 2023-10-10T20:36:01.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.737+0000 7f6e6d795700 1 -- 172.21.15.89:0/1362653232 wait complete. 2023-10-10T20:36:01.771 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:36:01.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.883+0000 7f7113107700 1 Processor -- start 2023-10-10T20:36:01.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.883+0000 7f7113107700 1 -- start start 2023-10-10T20:36:01.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.884+0000 7f7113107700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f710c159420 0x7f710c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.884+0000 7f7113107700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f710c1597f0 0x7f710c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.884+0000 7f7113107700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f710c0b11d0 con 0x7f710c15d230 2023-10-10T20:36:01.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.884+0000 7f7113107700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f710c0b14b0 con 0x7f710c159420 2023-10-10T20:36:01.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.884+0000 7f7113107700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f710c0b1790 con 0x7f710c1597f0 2023-10-10T20:36:01.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.884+0000 7f7112105700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f710c159420 0x7f710c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.884+0000 7f7112105700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f710c159420 0x7f710c0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41096/0 (socket says 172.21.15.89:41096) 2023-10-10T20:36:01.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.884+0000 7f7112105700 1 -- 172.21.15.89:0/434623192 learned_addr learned my addr 172.21.15.89:0/434623192 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:01.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.884+0000 7f7112906700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f710c15d230 0x7f710c15d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58818/0 (socket says 172.21.15.89:58818) 2023-10-10T20:36:01.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.884+0000 7f7111904700 1 --2- 172.21.15.89:0/434623192 >> v2:172.21.15.111:3300/0 conn(0x7f710c1597f0 0x7f710c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.884+0000 7f7112105700 1 -- 172.21.15.89:0/434623192 >> v2:172.21.15.111:3300/0 conn(0x7f710c1597f0 msgr2=0x7f710c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:01.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.884+0000 7f7112105700 1 --2- 172.21.15.89:0/434623192 >> v2:172.21.15.111:3300/0 conn(0x7f710c1597f0 0x7f710c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.884+0000 7f7112105700 1 -- 172.21.15.89:0/434623192 >> v1:172.21.15.89:6789/0 conn(0x7f710c15d230 legacy=0x7f710c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:01.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.884+0000 7f7112105700 1 -- 172.21.15.89:0/434623192 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710c0b1a70 con 0x7f710c159420 2023-10-10T20:36:01.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.884+0000 7f7111103700 1 -- 172.21.15.89:0/434623192 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3425868198 0 0) 0x7f710c0b11d0 con 0x7f710c15d230 2023-10-10T20:36:01.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.885+0000 7f7112105700 1 --2- 172.21.15.89:0/434623192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f710c159420 0x7f710c0b5f40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f710c089750 tx=0x7f7108009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91082b4d1aae048 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:01.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.885+0000 7f7111103700 1 -- 172.21.15.89:0/434623192 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710800e070 con 0x7f710c159420 2023-10-10T20:36:01.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.885+0000 7f7111103700 1 -- 172.21.15.89:0/434623192 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710800ba20 con 0x7f710c159420 2023-10-10T20:36:01.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.885+0000 7f7111103700 1 -- 172.21.15.89:0/434623192 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71080092e0 con 0x7f710c159420 2023-10-10T20:36:01.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.885+0000 7f7113107700 1 -- 172.21.15.89:0/434623192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f710c159420 msgr2=0x7f710c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:01.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.885+0000 7f7113107700 1 --2- 172.21.15.89:0/434623192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f710c159420 0x7f710c0b5f40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f710c089750 tx=0x7f7108009840 comp rx=0 tx=0).stop 2023-10-10T20:36:01.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.885+0000 7f7113107700 1 -- 172.21.15.89:0/434623192 shutdown_connections 2023-10-10T20:36:01.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.885+0000 7f7113107700 1 --2- 172.21.15.89:0/434623192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f710c159420 0x7f710c0b5f40 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.885+0000 7f7113107700 1 --2- 172.21.15.89:0/434623192 >> v2:172.21.15.111:3300/0 conn(0x7f710c1597f0 0x7f710c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.885+0000 7f7113107700 1 -- 172.21.15.89:0/434623192 >> 172.21.15.89:0/434623192 conn(0x7f710c0d0660 msgr2=0x7f710c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:01.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.885+0000 7f7113107700 1 -- 172.21.15.89:0/434623192 shutdown_connections 2023-10-10T20:36:01.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.885+0000 7f7113107700 1 -- 172.21.15.89:0/434623192 wait complete. 2023-10-10T20:36:01.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.886+0000 7f7113107700 1 Processor -- start 2023-10-10T20:36:01.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.887+0000 7f7113107700 1 -- start start 2023-10-10T20:36:01.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.887+0000 7f7113107700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f710c159420 0x7f710c13f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.887+0000 7f7113107700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f710c1597f0 0x7f710c158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.887+0000 7f7113107700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f710c15d230 0x7f710c15b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.887+0000 7f7113107700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710c15adb0 con 0x7f710c1597f0 2023-10-10T20:36:01.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.887+0000 7f7113107700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710c13f5b0 con 0x7f710c15d230 2023-10-10T20:36:01.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.887+0000 7f7113107700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710c15a1a0 con 0x7f710c159420 2023-10-10T20:36:01.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.887+0000 7f7111904700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f710c1597f0 0x7f710c158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.887+0000 7f7111904700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f710c1597f0 0x7f710c158470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41108/0 (socket says 172.21.15.89:41108) 2023-10-10T20:36:01.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.887+0000 7f7111904700 1 -- 172.21.15.89:0/3249965562 learned_addr learned my addr 172.21.15.89:0/3249965562 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:01.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.888+0000 7f7112906700 1 --2- 172.21.15.89:0/3249965562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f710c15d230 0x7f710c15b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.888+0000 7f7112105700 1 --2- 172.21.15.89:0/3249965562 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f710c159420 0x7f710c13f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.888+0000 7f7111904700 1 -- 172.21.15.89:0/3249965562 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f710c159420 msgr2=0x7f710c13f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:01.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.888+0000 7f7111904700 1 --2- 172.21.15.89:0/3249965562 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f710c159420 0x7f710c13f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.888+0000 7f7111904700 1 -- 172.21.15.89:0/3249965562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f710c15d230 msgr2=0x7f710c15b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:01.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.888+0000 7f7111904700 1 --2- 172.21.15.89:0/3249965562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f710c15d230 0x7f710c15b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:01.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.888+0000 7f7111904700 1 -- 172.21.15.89:0/3249965562 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7108009ae0 con 0x7f710c1597f0 2023-10-10T20:36:01.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.888+0000 7f7112105700 1 --2- 172.21.15.89:0/3249965562 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f710c159420 0x7f710c13f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:01.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.888+0000 7f7112906700 1 --2- 172.21.15.89:0/3249965562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f710c15d230 0x7f710c15b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:01.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.888+0000 7f7111904700 1 --2- 172.21.15.89:0/3249965562 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f710c1597f0 0x7f710c158470 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00ec50 tx=0x7f70fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:01.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.888+0000 7f71027fc700 1 -- 172.21.15.89:0/3249965562 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc00c9d0 con 0x7f710c1597f0 2023-10-10T20:36:01.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.888+0000 7f71027fc700 1 -- 172.21.15.89:0/3249965562 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70fc00cb30 con 0x7f710c1597f0 2023-10-10T20:36:01.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.888+0000 7f71027fc700 1 -- 172.21.15.89:0/3249965562 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc013470 con 0x7f710c1597f0 2023-10-10T20:36:01.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.888+0000 7f7113107700 1 -- 172.21.15.89:0/3249965562 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710c15a480 con 0x7f710c1597f0 2023-10-10T20:36:01.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.888+0000 7f7113107700 1 -- 172.21.15.89:0/3249965562 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710c15aa40 con 0x7f710c1597f0 2023-10-10T20:36:01.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.889+0000 7f7113107700 1 -- 172.21.15.89:0/3249965562 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710c04c3c0 con 0x7f710c1597f0 2023-10-10T20:36:01.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.889+0000 7f71027fc700 1 -- 172.21.15.89:0/3249965562 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f70fc013860 con 0x7f710c1597f0 2023-10-10T20:36:01.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.890+0000 7f71027fc700 1 --2- 172.21.15.89:0/3249965562 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f70f0066f30 0x7f70f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:01.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.890+0000 7f7112105700 1 --2- 172.21.15.89:0/3249965562 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f70f0066f30 0x7f70f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:01.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.890+0000 7f71027fc700 1 -- 172.21.15.89:0/3249965562 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f70fc084990 con 0x7f710c1597f0 2023-10-10T20:36:01.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.890+0000 7f7112105700 1 --2- 172.21.15.89:0/3249965562 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f70f0066f30 0x7f70f00693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7108009ab0 tx=0x7f7108018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:01.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:01.894+0000 7f71027fc700 1 -- 172.21.15.89:0/3249965562 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70fc051b60 con 0x7f710c1597f0 2023-10-10T20:36:02.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.123+0000 7f7113107700 1 -- 172.21.15.89:0/3249965562 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f710c15aef0 con 0x7f710c1597f0 2023-10-10T20:36:02.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.124+0000 7f71027fc700 1 -- 172.21.15.89:0/3249965562 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) v1 ==== 53+0+73 (secure 0 0 0) 0x7f70fc055180 con 0x7f710c1597f0 2023-10-10T20:36:02.126 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:36:02.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.126+0000 7f7113107700 1 -- 172.21.15.89:0/3249965562 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f70f0066f30 msgr2=0x7f70f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:02.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.126+0000 7f7113107700 1 --2- 172.21.15.89:0/3249965562 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f70f0066f30 0x7f70f00693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7108009ab0 tx=0x7f7108018000 comp rx=0 tx=0).stop 2023-10-10T20:36:02.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.126+0000 7f7113107700 1 -- 172.21.15.89:0/3249965562 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f710c1597f0 msgr2=0x7f710c158470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:02.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.126+0000 7f7113107700 1 --2- 172.21.15.89:0/3249965562 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f710c1597f0 0x7f710c158470 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00ec50 tx=0x7f70fc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:02.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.126+0000 7f7113107700 1 -- 172.21.15.89:0/3249965562 shutdown_connections 2023-10-10T20:36:02.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.126+0000 7f7113107700 1 --2- 172.21.15.89:0/3249965562 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f70f0066f30 0x7f70f00693e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:02.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.126+0000 7f7113107700 1 --2- 172.21.15.89:0/3249965562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f710c15d230 0x7f710c15b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:02.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.126+0000 7f7113107700 1 --2- 172.21.15.89:0/3249965562 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f710c159420 0x7f710c13f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:02.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.126+0000 7f7113107700 1 --2- 172.21.15.89:0/3249965562 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f710c1597f0 0x7f710c158470 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.126+0000 7f7113107700 1 -- 172.21.15.89:0/3249965562 >> 172.21.15.89:0/3249965562 conn(0x7f710c0d0660 msgr2=0x7f710c0ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.126+0000 7f7113107700 1 -- 172.21.15.89:0/3249965562 shutdown_connections 2023-10-10T20:36:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.126+0000 7f7113107700 1 -- 172.21.15.89:0/3249965562 wait complete. 2023-10-10T20:36:02.143 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:36:02.144 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:36:02.144 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-10T20:36:02.144 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:36:02.145 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:36:02.145 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:36:02.145 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:36:02.146 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:36:02.146 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:36:02.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:02.147 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-10T20:36:02.165 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:36:02.166 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:36:02.166 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:36:02.166 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:02.166 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-10T20:36:02.166 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:02.167 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:36:02.167 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:02.167 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-10T20:36:02.167 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:36:02.167 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:02.167 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-10T20:36:02.168 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:36:02.168 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:02.168 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:36:02.168 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:02.168 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:36:02.169 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42403sec preferred_lft 42403sec 2023-10-10T20:36:02.169 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2023-10-10T20:36:02.169 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:02.169 INFO:teuthology.orchestra.run.smithi089.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:36:02.169 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:02.170 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:36:02.170 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:02.170 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::98a3:d4ff:fec1:83bb/64 scope link 2023-10-10T20:36:02.170 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:02.170 INFO:teuthology.orchestra.run.smithi089.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:36:02.170 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether ee:16:22:10:ca:9a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:36:02.171 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec16:22ff:fe10:ca9a/64 scope link 2023-10-10T20:36:02.171 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:02.171 INFO:teuthology.orchestra.run.smithi089.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:36:02.171 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:36:02.171 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::6864:2bff:fe5c:71d4/64 scope link 2023-10-10T20:36:02.172 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:02.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:02.172 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:36:02.236 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-10T20:36:02.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:02.238 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-10T20:36:02.299 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-10T20:36:02.299 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:36:02.299 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:36:02.299 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:36:02.300 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:36:02.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:02.300 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:36:02.301 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:36:02.301 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:36:02.301 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:36:02.302 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:36:02.302 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:36:02.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:02.401 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-10T20:36:02.417 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:36:02.418 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:36:02.418 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:36:02.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:02.419 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-10-10T20:36:02.484 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:36:02.485 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:36:02.539 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:36:02.540 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:36:02.540 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:36:02.594 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:36:02.594 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:36:02.594 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:36:02.673 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:36:02.673 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:36:02.673 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:36:02.681 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:36:02.681 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:36:02.682 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:36:02.682 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:36:02.682 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:36:02.703 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:36:02.704 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:36:02.704 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:36:02.704 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:36:02.704 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:36:02.705 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:36:02.705 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:36:02.705 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:36:02.706 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:36:02.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:02.707 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:36:02.770 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:36:02.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.923+0000 7fbb24bcc700 1 Processor -- start 2023-10-10T20:36:02.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.923+0000 7fbb24bcc700 1 -- start start 2023-10-10T20:36:02.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.924+0000 7fbb24bcc700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbb200b1850 0x7fbb200b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:02.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.924+0000 7fbb24bcc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb201579f0 0x7fbb200abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:02.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.924+0000 7fbb24bcc700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb200cb300 con 0x7fbb200ae680 2023-10-10T20:36:02.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.924+0000 7fbb24bcc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb2006fc30 con 0x7fbb201579f0 2023-10-10T20:36:02.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.924+0000 7fbb24bcc700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb200cc260 con 0x7fbb200b1850 2023-10-10T20:36:02.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.924+0000 7fbb1ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb201579f0 0x7fbb200abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:02.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.924+0000 7fbb1ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb201579f0 0x7fbb200abf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41118/0 (socket says 172.21.15.89:41118) 2023-10-10T20:36:02.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.924+0000 7fbb1ed9d700 1 -- 172.21.15.89:0/3395575302 learned_addr learned my addr 172.21.15.89:0/3395575302 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:02.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.924+0000 7fbb1e59c700 1 --2- 172.21.15.89:0/3395575302 >> v2:172.21.15.111:3300/0 conn(0x7fbb200b1850 0x7fbb200b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:02.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.924+0000 7fbb1ed9d700 1 -- 172.21.15.89:0/3395575302 >> v2:172.21.15.111:3300/0 conn(0x7fbb200b1850 msgr2=0x7fbb200b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:02.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.924+0000 7fbb1ed9d700 1 --2- 172.21.15.89:0/3395575302 >> v2:172.21.15.111:3300/0 conn(0x7fbb200b1850 0x7fbb200b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:02.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.925+0000 7fbb1ed9d700 1 -- 172.21.15.89:0/3395575302 >> v1:172.21.15.111:6789/0 conn(0x7fbb200ae680 legacy=0x7fbb201572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:02.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.925+0000 7fbb1ed9d700 1 -- 172.21.15.89:0/3395575302 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb200cb5e0 con 0x7fbb201579f0 2023-10-10T20:36:02.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.925+0000 7fbb1e59c700 1 --2- 172.21.15.89:0/3395575302 >> v2:172.21.15.111:3300/0 conn(0x7fbb200b1850 0x7fbb200b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:02.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.925+0000 7fbb1ed9d700 1 --2- 172.21.15.89:0/3395575302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb201579f0 0x7fbb200abf60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbb200641e0 tx=0x7fbb18009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5f322bf631b37b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:02.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.925+0000 7fbb1dd9b700 1 -- 172.21.15.89:0/3395575302 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1800e070 con 0x7fbb201579f0 2023-10-10T20:36:02.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.925+0000 7fbb1dd9b700 1 -- 172.21.15.89:0/3395575302 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1800ba50 con 0x7fbb201579f0 2023-10-10T20:36:02.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.925+0000 7fbb1dd9b700 1 -- 172.21.15.89:0/3395575302 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb180092e0 con 0x7fbb201579f0 2023-10-10T20:36:02.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.925+0000 7fbb24bcc700 1 -- 172.21.15.89:0/3395575302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb201579f0 msgr2=0x7fbb200abf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:02.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.925+0000 7fbb24bcc700 1 --2- 172.21.15.89:0/3395575302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb201579f0 0x7fbb200abf60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbb200641e0 tx=0x7fbb18009840 comp rx=0 tx=0).stop 2023-10-10T20:36:02.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.925+0000 7fbb24bcc700 1 -- 172.21.15.89:0/3395575302 shutdown_connections 2023-10-10T20:36:02.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.925+0000 7fbb24bcc700 1 --2- 172.21.15.89:0/3395575302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb201579f0 0x7fbb200abf60 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:02.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.925+0000 7fbb24bcc700 1 --2- 172.21.15.89:0/3395575302 >> v2:172.21.15.111:3300/0 conn(0x7fbb200b1850 0x7fbb200b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:02.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.925+0000 7fbb24bcc700 1 -- 172.21.15.89:0/3395575302 >> 172.21.15.89:0/3395575302 conn(0x7fbb2006a320 msgr2=0x7fbb200cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:02.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.925+0000 7fbb24bcc700 1 -- 172.21.15.89:0/3395575302 shutdown_connections 2023-10-10T20:36:02.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.925+0000 7fbb24bcc700 1 -- 172.21.15.89:0/3395575302 wait complete. 2023-10-10T20:36:02.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.927+0000 7fbb24bcc700 1 Processor -- start 2023-10-10T20:36:02.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.927+0000 7fbb24bcc700 1 -- start start 2023-10-10T20:36:02.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.927+0000 7fbb24bcc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb200ae680 0x7fbb2016a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:02.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.927+0000 7fbb24bcc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb200b1850 0x7fbb200ab030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:02.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.927+0000 7fbb24bcc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb201579f0 0x7fbb20154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:02.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.927+0000 7fbb24bcc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb200b06e0 con 0x7fbb201579f0 2023-10-10T20:36:02.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.927+0000 7fbb24bcc700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb200af3f0 con 0x7fbb200b1850 2023-10-10T20:36:02.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.927+0000 7fbb24bcc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb200b0250 con 0x7fbb200ae680 2023-10-10T20:36:02.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.927+0000 7fbb1ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb201579f0 0x7fbb20154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:02.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.927+0000 7fbb1ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb201579f0 0x7fbb20154b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41124/0 (socket says 172.21.15.89:41124) 2023-10-10T20:36:02.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.927+0000 7fbb1ed9d700 1 -- 172.21.15.89:0/3604176249 learned_addr learned my addr 172.21.15.89:0/3604176249 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:02.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.928+0000 7fbb1e59c700 1 --2- 172.21.15.89:0/3604176249 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb200ae680 0x7fbb2016a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:02.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.928+0000 7fbb15d9b700 1 --2- 172.21.15.89:0/3604176249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb200b1850 0x7fbb200ab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:02.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.928+0000 7fbb1ed9d700 1 -- 172.21.15.89:0/3604176249 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb200ae680 msgr2=0x7fbb2016a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:02.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.928+0000 7fbb1ed9d700 1 --2- 172.21.15.89:0/3604176249 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb200ae680 0x7fbb2016a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:02.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.928+0000 7fbb1ed9d700 1 -- 172.21.15.89:0/3604176249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb200b1850 msgr2=0x7fbb200ab030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:02.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.928+0000 7fbb1ed9d700 1 --2- 172.21.15.89:0/3604176249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb200b1850 0x7fbb200ab030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:02.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.928+0000 7fbb1ed9d700 1 -- 172.21.15.89:0/3604176249 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb18009ae0 con 0x7fbb201579f0 2023-10-10T20:36:02.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.928+0000 7fbb1ed9d700 1 --2- 172.21.15.89:0/3604176249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb201579f0 0x7fbb20154b30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009fd0 tx=0x7fbb18018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:02.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.928+0000 7fbb16ffd700 1 -- 172.21.15.89:0/3604176249 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18008e40 con 0x7fbb201579f0 2023-10-10T20:36:02.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.928+0000 7fbb16ffd700 1 -- 172.21.15.89:0/3604176249 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1800be60 con 0x7fbb201579f0 2023-10-10T20:36:02.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.928+0000 7fbb24bcc700 1 -- 172.21.15.89:0/3604176249 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2016aa10 con 0x7fbb201579f0 2023-10-10T20:36:02.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.928+0000 7fbb16ffd700 1 -- 172.21.15.89:0/3604176249 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1801c640 con 0x7fbb201579f0 2023-10-10T20:36:02.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.928+0000 7fbb24bcc700 1 -- 172.21.15.89:0/3604176249 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb200af640 con 0x7fbb201579f0 2023-10-10T20:36:02.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.929+0000 7fbb24bcc700 1 -- 172.21.15.89:0/3604176249 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb2004c3c0 con 0x7fbb201579f0 2023-10-10T20:36:02.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.929+0000 7fbb16ffd700 1 -- 172.21.15.89:0/3604176249 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb18003040 con 0x7fbb201579f0 2023-10-10T20:36:02.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.930+0000 7fbb16ffd700 1 --2- 172.21.15.89:0/3604176249 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb00066f30 0x7fbb000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:02.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.930+0000 7fbb16ffd700 1 -- 172.21.15.89:0/3604176249 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbb18085e20 con 0x7fbb201579f0 2023-10-10T20:36:02.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.930+0000 7fbb1e59c700 1 --2- 172.21.15.89:0/3604176249 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb00066f30 0x7fbb000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:02.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.930+0000 7fbb1e59c700 1 --2- 172.21.15.89:0/3604176249 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb00066f30 0x7fbb000693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbb10006f00 tx=0x7fbb10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:02.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:02.934+0000 7fbb16ffd700 1 -- 172.21.15.89:0/3604176249 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb18053090 con 0x7fbb201579f0 2023-10-10T20:36:03.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.167+0000 7fbb24bcc700 1 -- 172.21.15.89:0/3604176249 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbb200cb8c0 con 0x7fbb201579f0 2023-10-10T20:36:03.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.167+0000 7fbb16ffd700 1 -- 172.21.15.89:0/3604176249 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbb180566b0 con 0x7fbb201579f0 2023-10-10T20:36:03.170 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:36:03.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.169+0000 7fbb24bcc700 1 -- 172.21.15.89:0/3604176249 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb00066f30 msgr2=0x7fbb000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:03.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.169+0000 7fbb24bcc700 1 --2- 172.21.15.89:0/3604176249 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb00066f30 0x7fbb000693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbb10006f00 tx=0x7fbb10008040 comp rx=0 tx=0).stop 2023-10-10T20:36:03.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.169+0000 7fbb24bcc700 1 -- 172.21.15.89:0/3604176249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb201579f0 msgr2=0x7fbb20154b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:03.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.169+0000 7fbb24bcc700 1 --2- 172.21.15.89:0/3604176249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb201579f0 0x7fbb20154b30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009fd0 tx=0x7fbb18018000 comp rx=0 tx=0).stop 2023-10-10T20:36:03.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.169+0000 7fbb24bcc700 1 -- 172.21.15.89:0/3604176249 shutdown_connections 2023-10-10T20:36:03.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.169+0000 7fbb24bcc700 1 --2- 172.21.15.89:0/3604176249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb201579f0 0x7fbb20154b30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:03.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.169+0000 7fbb24bcc700 1 --2- 172.21.15.89:0/3604176249 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb200ae680 0x7fbb2016a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:03.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.169+0000 7fbb24bcc700 1 --2- 172.21.15.89:0/3604176249 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb00066f30 0x7fbb000693e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:03.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.169+0000 7fbb24bcc700 1 --2- 172.21.15.89:0/3604176249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb200b1850 0x7fbb200ab030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:03.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.169+0000 7fbb24bcc700 1 -- 172.21.15.89:0/3604176249 >> 172.21.15.89:0/3604176249 conn(0x7fbb2006a320 msgr2=0x7fbb20152f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:03.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.170+0000 7fbb24bcc700 1 -- 172.21.15.89:0/3604176249 shutdown_connections 2023-10-10T20:36:03.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.170+0000 7fbb24bcc700 1 -- 172.21.15.89:0/3604176249 wait complete. 2023-10-10T20:36:03.183 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:36:03.184 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:36:03.184 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-10-10T20:36:03.184 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:36:03.185 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:36:03.185 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:36:03.185 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:36:03.185 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:36:03.186 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:36:03.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:03.187 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-10-10T20:36:03.206 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:36:03.206 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:36:03.206 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:36:03.207 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:03.207 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-10-10T20:36:03.207 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:03.207 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:36:03.207 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:03.208 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-10-10T20:36:03.208 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:36:03.208 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:03.208 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-10-10T20:36:03.208 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:36:03.208 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:03.209 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:36:03.209 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:03.209 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:36:03.209 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42419sec preferred_lft 42419sec 2023-10-10T20:36:03.209 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-10-10T20:36:03.210 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:03.210 INFO:teuthology.orchestra.run.smithi111.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:36:03.210 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:03.210 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:36:03.210 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:03.211 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::a486:a6ff:fe5f:6021/64 scope link 2023-10-10T20:36:03.211 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:03.211 INFO:teuthology.orchestra.run.smithi111.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:36:03.211 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 1e:d0:81:1b:e2:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:36:03.211 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::1cd0:81ff:fe1b:e26b/64 scope link 2023-10-10T20:36:03.212 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:03.212 INFO:teuthology.orchestra.run.smithi111.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:36:03.212 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:36:03.212 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::88d:cdff:fe75:455e/64 scope link 2023-10-10T20:36:03.212 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:03.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:03.213 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:36:03.273 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-10-10T20:36:03.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:03.275 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-10-10T20:36:03.333 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-10-10T20:36:03.333 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:36:03.334 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:36:03.334 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:36:03.334 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:36:03.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:03.335 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:36:03.335 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:36:03.336 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:36:03.336 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:36:03.336 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:36:03.336 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:36:03.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:03.439 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-10-10T20:36:03.454 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:36:03.454 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:36:03.455 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:36:03.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:03.456 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-10-10T20:36:03.519 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:36:03.519 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:36:03.574 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:36:03.574 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:36:03.575 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:36:03.628 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:36:03.629 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:36:03.629 DEBUG:teuthology.orchestra.run.smithi111:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:36:03.708 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:36:03.708 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:36:03.708 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:36:03.716 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:36:03.716 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:36:03.717 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:36:03.717 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:36:03.717 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:36:03.739 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:36:03.739 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:36:03.739 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:36:03.739 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:36:03.740 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:36:03.740 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:36:03.740 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:36:03.740 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:36:03.741 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:36:03.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:03.742 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:36:03.806 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-10T20:36:03.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.927+0000 7f28dc109700 1 Processor -- start 2023-10-10T20:36:03.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.928+0000 7f28dc109700 1 -- start start 2023-10-10T20:36:03.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.928+0000 7f28dc109700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28d40cd910 0x7f28d40ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:03.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.928+0000 7f28dc109700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f28d40b55a0 0x7f28d40ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:03.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.928+0000 7f28dc109700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d40b5b00 con 0x7f28d4159ec0 2023-10-10T20:36:03.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.928+0000 7f28dc109700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d40b5da0 con 0x7f28d40b55a0 2023-10-10T20:36:03.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.928+0000 7f28dc109700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d40b6080 con 0x7f28d40cd910 2023-10-10T20:36:03.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.929+0000 7f28da6a6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f28d40b55a0 0x7f28d40ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:03.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.929+0000 7f28d9ea5700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f28d4159ec0 0x7f28d40b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58822/0 (socket says 172.21.15.89:58822) 2023-10-10T20:36:03.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.929+0000 7f28d9ea5700 1 -- 172.21.15.89:0/1145302076 learned_addr learned my addr 172.21.15.89:0/1145302076 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:03.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.929+0000 7f28da6a6700 1 -- 172.21.15.89:0/1145302076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28d40cd910 msgr2=0x7f28d40ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:03.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.929+0000 7f28d96a4700 1 --2- 172.21.15.89:0/1145302076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28d40cd910 0x7f28d40ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:03.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.929+0000 7f28da6a6700 1 --2- 172.21.15.89:0/1145302076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28d40cd910 0x7f28d40ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:03.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.929+0000 7f28d8ea3700 1 -- 172.21.15.89:0/1145302076 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3039132310 0 0) 0x7f28d40b5b00 con 0x7f28d4159ec0 2023-10-10T20:36:03.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.929+0000 7f28da6a6700 1 -- 172.21.15.89:0/1145302076 >> v1:172.21.15.89:6789/0 conn(0x7f28d4159ec0 legacy=0x7f28d40b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:03.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.929+0000 7f28da6a6700 1 -- 172.21.15.89:0/1145302076 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d40cc100 con 0x7f28d40b55a0 2023-10-10T20:36:03.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.929+0000 7f28da6a6700 1 --2- 172.21.15.89:0/1145302076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d40b55a0 0x7f28d40ce880 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0098e0 tx=0x7f28cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ab4fe6d56c2002 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:03.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.929+0000 7f28d8ea3700 1 -- 172.21.15.89:0/1145302076 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc009e80 con 0x7f28d40b55a0 2023-10-10T20:36:03.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.929+0000 7f28d8ea3700 1 -- 172.21.15.89:0/1145302076 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28cc00b860 con 0x7f28d40b55a0 2023-10-10T20:36:03.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.929+0000 7f28d8ea3700 1 -- 172.21.15.89:0/1145302076 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc00bbe0 con 0x7f28d40b55a0 2023-10-10T20:36:03.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.930+0000 7f28dc109700 1 -- 172.21.15.89:0/1145302076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d40b55a0 msgr2=0x7f28d40ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:03.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.930+0000 7f28dc109700 1 --2- 172.21.15.89:0/1145302076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d40b55a0 0x7f28d40ce880 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0098e0 tx=0x7f28cc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:03.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.930+0000 7f28dc109700 1 -- 172.21.15.89:0/1145302076 shutdown_connections 2023-10-10T20:36:03.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.930+0000 7f28dc109700 1 --2- 172.21.15.89:0/1145302076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d40b55a0 0x7f28d40ce880 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:03.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.930+0000 7f28dc109700 1 --2- 172.21.15.89:0/1145302076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28d40cd910 0x7f28d40ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:03.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.930+0000 7f28dc109700 1 -- 172.21.15.89:0/1145302076 >> 172.21.15.89:0/1145302076 conn(0x7f28d40999f0 msgr2=0x7f28d40b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:03.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.930+0000 7f28dc109700 1 -- 172.21.15.89:0/1145302076 shutdown_connections 2023-10-10T20:36:03.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.930+0000 7f28dc109700 1 -- 172.21.15.89:0/1145302076 wait complete. 2023-10-10T20:36:03.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.931+0000 7f28dc109700 1 Processor -- start 2023-10-10T20:36:03.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.931+0000 7f28dc109700 1 -- start start 2023-10-10T20:36:03.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28dc109700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28d40b55a0 0x7f28d4150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:03.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28dc109700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d4159ec0 0x7f28d4157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:03.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28dc109700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28d4158120 0x7f28d40c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:03.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28dc109700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d4153c10 con 0x7f28d4159ec0 2023-10-10T20:36:03.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28dc109700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d41586f0 con 0x7f28d4158120 2023-10-10T20:36:03.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28dc109700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d41589d0 con 0x7f28d40b55a0 2023-10-10T20:36:03.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28d96a4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d4159ec0 0x7f28d4157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:03.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28d96a4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d4159ec0 0x7f28d4157be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41148/0 (socket says 172.21.15.89:41148) 2023-10-10T20:36:03.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28d96a4700 1 -- 172.21.15.89:0/1744917660 learned_addr learned my addr 172.21.15.89:0/1744917660 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:03.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28da6a6700 1 --2- 172.21.15.89:0/1744917660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28d4158120 0x7f28d40c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:03.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28d9ea5700 1 --2- 172.21.15.89:0/1744917660 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28d40b55a0 0x7f28d4150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:03.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28d96a4700 1 -- 172.21.15.89:0/1744917660 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28d40b55a0 msgr2=0x7f28d4150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:03.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28d96a4700 1 --2- 172.21.15.89:0/1744917660 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28d40b55a0 0x7f28d4150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:03.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28d96a4700 1 -- 172.21.15.89:0/1744917660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28d4158120 msgr2=0x7f28d40c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:03.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28d96a4700 1 --2- 172.21.15.89:0/1744917660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28d4158120 0x7f28d40c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:03.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28d96a4700 1 -- 172.21.15.89:0/1744917660 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc009420 con 0x7f28d4159ec0 2023-10-10T20:36:03.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28d96a4700 1 --2- 172.21.15.89:0/1744917660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d4159ec0 0x7f28d4157be0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f28c400c7d0 tx=0x7f28c4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:03.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28c9ffb700 1 -- 172.21.15.89:0/1744917660 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4011600 con 0x7f28d4159ec0 2023-10-10T20:36:03.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28c9ffb700 1 -- 172.21.15.89:0/1744917660 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c4011760 con 0x7f28d4159ec0 2023-10-10T20:36:03.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28dc109700 1 -- 172.21.15.89:0/1744917660 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d4158cb0 con 0x7f28d4159ec0 2023-10-10T20:36:03.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.932+0000 7f28c9ffb700 1 -- 172.21.15.89:0/1744917660 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4011b20 con 0x7f28d4159ec0 2023-10-10T20:36:03.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.933+0000 7f28dc109700 1 -- 172.21.15.89:0/1744917660 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d4151160 con 0x7f28d4159ec0 2023-10-10T20:36:03.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.933+0000 7f28dc109700 1 -- 172.21.15.89:0/1744917660 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d4153dc0 con 0x7f28d4159ec0 2023-10-10T20:36:03.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.934+0000 7f28c9ffb700 1 -- 172.21.15.89:0/1744917660 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f28c40193f0 con 0x7f28d4159ec0 2023-10-10T20:36:03.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.934+0000 7f28c9ffb700 1 --2- 172.21.15.89:0/1744917660 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f28b8066f30 0x7f28b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:03.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.934+0000 7f28c9ffb700 1 -- 172.21.15.89:0/1744917660 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f28c40847f0 con 0x7f28d4159ec0 2023-10-10T20:36:03.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.934+0000 7f28d9ea5700 1 --2- 172.21.15.89:0/1744917660 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f28b8066f30 0x7f28b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:03.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.935+0000 7f28d9ea5700 1 --2- 172.21.15.89:0/1744917660 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f28b8066f30 0x7f28b80693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f28d4153aa0 tx=0x7f28d0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:03.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:03.938+0000 7f28c9ffb700 1 -- 172.21.15.89:0/1744917660 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28d4153dc0 con 0x7f28d4159ec0 2023-10-10T20:36:04.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:04.168+0000 7f28dc109700 1 -- 172.21.15.89:0/1744917660 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f28d4063c90 con 0x7f28d4159ec0 2023-10-10T20:36:04.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:04.987+0000 7f28c9ffb700 1 -- 172.21.15.89:0/1744917660 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v30) v1 ==== 136+0+0 (secure 0 0 0) 0x7f28c4051a60 con 0x7f28d4159ec0 2023-10-10T20:36:04.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:04.989+0000 7f28dc109700 1 -- 172.21.15.89:0/1744917660 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f28b8066f30 msgr2=0x7f28b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:04.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:04.989+0000 7f28dc109700 1 --2- 172.21.15.89:0/1744917660 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f28b8066f30 0x7f28b80693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f28d4153aa0 tx=0x7f28d0009040 comp rx=0 tx=0).stop 2023-10-10T20:36:04.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:04.989+0000 7f28dc109700 1 -- 172.21.15.89:0/1744917660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d4159ec0 msgr2=0x7f28d4157be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:04.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:04.989+0000 7f28dc109700 1 --2- 172.21.15.89:0/1744917660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d4159ec0 0x7f28d4157be0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f28c400c7d0 tx=0x7f28c4008040 comp rx=0 tx=0).stop 2023-10-10T20:36:04.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:04.989+0000 7f28dc109700 1 -- 172.21.15.89:0/1744917660 shutdown_connections 2023-10-10T20:36:04.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:04.989+0000 7f28dc109700 1 --2- 172.21.15.89:0/1744917660 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f28b8066f30 0x7f28b80693e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:04.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:04.989+0000 7f28dc109700 1 --2- 172.21.15.89:0/1744917660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28d4158120 0x7f28d40c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:04.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:04.989+0000 7f28dc109700 1 --2- 172.21.15.89:0/1744917660 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28d40b55a0 0x7f28d4150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:04.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:04.989+0000 7f28dc109700 1 --2- 172.21.15.89:0/1744917660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d4159ec0 0x7f28d4157be0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:04.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:04.989+0000 7f28dc109700 1 -- 172.21.15.89:0/1744917660 >> 172.21.15.89:0/1744917660 conn(0x7f28d40999f0 msgr2=0x7f28d40cf310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:04.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:04.989+0000 7f28dc109700 1 -- 172.21.15.89:0/1744917660 shutdown_connections 2023-10-10T20:36:04.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:04.989+0000 7f28dc109700 1 -- 172.21.15.89:0/1744917660 wait complete. 2023-10-10T20:36:05.004 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-10T20:36:05.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.119+0000 7fd96b59e700 1 Processor -- start 2023-10-10T20:36:05.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.119+0000 7fd96b59e700 1 -- start start 2023-10-10T20:36:05.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.120+0000 7fd96b59e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd96c1597f0 0x7fd96c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:05.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.120+0000 7fd96b59e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd96c0b11d0 con 0x7fd96c159420 2023-10-10T20:36:05.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.120+0000 7fd96b59e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd96c0b14b0 con 0x7fd96c15d230 2023-10-10T20:36:05.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.120+0000 7fd96b59e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd96c0b1790 con 0x7fd96c1597f0 2023-10-10T20:36:05.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.120+0000 7fd96a59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd96c159420 0x7fd96c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58824/0 (socket says 172.21.15.89:58824) 2023-10-10T20:36:05.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.120+0000 7fd96a59c700 1 -- 172.21.15.89:0/2330275440 learned_addr learned my addr 172.21.15.89:0/2330275440 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:05.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.120+0000 7fd961d9b700 1 --2- 172.21.15.89:0/2330275440 >> v2:172.21.15.111:3300/0 conn(0x7fd96c1597f0 0x7fd96c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:05.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.120+0000 7fd969d9b700 1 -- 172.21.15.89:0/2330275440 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2630542858 0 0) 0x7fd96c0b11d0 con 0x7fd96c159420 2023-10-10T20:36:05.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.120+0000 7fd969d9b700 1 -- 172.21.15.89:0/2330275440 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd958003570 con 0x7fd96c159420 2023-10-10T20:36:05.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.120+0000 7fd969d9b700 1 -- 172.21.15.89:0/2330275440 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2681040408 0 0) 0x7fd96c0b14b0 con 0x7fd96c15d230 2023-10-10T20:36:05.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.120+0000 7fd969d9b700 1 -- 172.21.15.89:0/2330275440 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd96c0b11d0 con 0x7fd96c15d230 2023-10-10T20:36:05.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.120+0000 7fd961d9b700 1 -- 172.21.15.89:0/2330275440 >> v1:172.21.15.111:6789/0 conn(0x7fd96c15d230 legacy=0x7fd96c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:05.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.120+0000 7fd961d9b700 1 -- 172.21.15.89:0/2330275440 >> v1:172.21.15.89:6789/0 conn(0x7fd96c159420 legacy=0x7fd96c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:05.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.120+0000 7fd961d9b700 1 -- 172.21.15.89:0/2330275440 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96c0b1a70 con 0x7fd96c1597f0 2023-10-10T20:36:05.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.121+0000 7fd961d9b700 1 --2- 172.21.15.89:0/2330275440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd96c1597f0 0x7fd96c0b2270 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd9540098e0 tx=0x7fd9540091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccbffb6fed2c60e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:05.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.121+0000 7fd969d9b700 1 -- 172.21.15.89:0/2330275440 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd954011630 con 0x7fd96c1597f0 2023-10-10T20:36:05.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.121+0000 7fd969d9b700 1 -- 172.21.15.89:0/2330275440 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd954011790 con 0x7fd96c1597f0 2023-10-10T20:36:05.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.121+0000 7fd969d9b700 1 -- 172.21.15.89:0/2330275440 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd954011b10 con 0x7fd96c1597f0 2023-10-10T20:36:05.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.121+0000 7fd96b59e700 1 -- 172.21.15.89:0/2330275440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd96c1597f0 msgr2=0x7fd96c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:05.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.121+0000 7fd96b59e700 1 --2- 172.21.15.89:0/2330275440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd96c1597f0 0x7fd96c0b2270 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd9540098e0 tx=0x7fd9540091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.121+0000 7fd96b59e700 1 -- 172.21.15.89:0/2330275440 shutdown_connections 2023-10-10T20:36:05.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.121+0000 7fd96b59e700 1 --2- 172.21.15.89:0/2330275440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd96c1597f0 0x7fd96c0b2270 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.121+0000 7fd96b59e700 1 -- 172.21.15.89:0/2330275440 >> 172.21.15.89:0/2330275440 conn(0x7fd96c0d0660 msgr2=0x7fd96c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:05.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.121+0000 7fd96b59e700 1 -- 172.21.15.89:0/2330275440 shutdown_connections 2023-10-10T20:36:05.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.121+0000 7fd96b59e700 1 -- 172.21.15.89:0/2330275440 wait complete. 2023-10-10T20:36:05.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.123+0000 7fd96b59e700 1 Processor -- start 2023-10-10T20:36:05.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.123+0000 7fd96b59e700 1 -- start start 2023-10-10T20:36:05.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.123+0000 7fd96b59e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd96c159420 0x7fd96c153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:05.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.123+0000 7fd96b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd96c1597f0 0x7fd96c16d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:05.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.123+0000 7fd96b59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd96c15d230 0x7fd96c155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:05.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.123+0000 7fd96b59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c169ab0 con 0x7fd96c1597f0 2023-10-10T20:36:05.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.123+0000 7fd96b59e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c0cc410 con 0x7fd96c159420 2023-10-10T20:36:05.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.123+0000 7fd96b59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c0c7d20 con 0x7fd96c15d230 2023-10-10T20:36:05.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.123+0000 7fd961d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd96c1597f0 0x7fd96c16d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:05.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.124+0000 7fd961d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd96c1597f0 0x7fd96c16d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41158/0 (socket says 172.21.15.89:41158) 2023-10-10T20:36:05.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.124+0000 7fd961d9b700 1 -- 172.21.15.89:0/810797180 learned_addr learned my addr 172.21.15.89:0/810797180 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:05.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.124+0000 7fd96a59c700 1 --2- 172.21.15.89:0/810797180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd96c159420 0x7fd96c153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:05.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.124+0000 7fd96ad9d700 1 --2- 172.21.15.89:0/810797180 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd96c15d230 0x7fd96c155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:05.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.124+0000 7fd961d9b700 1 -- 172.21.15.89:0/810797180 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd96c15d230 msgr2=0x7fd96c155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:05.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.124+0000 7fd961d9b700 1 --2- 172.21.15.89:0/810797180 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd96c15d230 0x7fd96c155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.124+0000 7fd961d9b700 1 -- 172.21.15.89:0/810797180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd96c159420 msgr2=0x7fd96c153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:05.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.124+0000 7fd961d9b700 1 --2- 172.21.15.89:0/810797180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd96c159420 0x7fd96c153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.124+0000 7fd961d9b700 1 -- 172.21.15.89:0/810797180 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd954009420 con 0x7fd96c1597f0 2023-10-10T20:36:05.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.124+0000 7fd961d9b700 1 --2- 172.21.15.89:0/810797180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd96c1597f0 0x7fd96c16d0f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd954009ed0 tx=0x7fd954017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:05.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.124+0000 7fd962ffd700 1 -- 172.21.15.89:0/810797180 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd954011900 con 0x7fd96c1597f0 2023-10-10T20:36:05.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.124+0000 7fd962ffd700 1 -- 172.21.15.89:0/810797180 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd954011a80 con 0x7fd96c1597f0 2023-10-10T20:36:05.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.124+0000 7fd962ffd700 1 -- 172.21.15.89:0/810797180 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95401a680 con 0x7fd96c1597f0 2023-10-10T20:36:05.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.124+0000 7fd96b59e700 1 -- 172.21.15.89:0/810797180 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd96c0c7110 con 0x7fd96c1597f0 2023-10-10T20:36:05.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.124+0000 7fd96b59e700 1 -- 172.21.15.89:0/810797180 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96c0c76d0 con 0x7fd96c1597f0 2023-10-10T20:36:05.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.125+0000 7fd96b59e700 1 -- 172.21.15.89:0/810797180 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd96c169bf0 con 0x7fd96c1597f0 2023-10-10T20:36:05.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.125+0000 7fd962ffd700 1 -- 172.21.15.89:0/810797180 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd95401a9f0 con 0x7fd96c1597f0 2023-10-10T20:36:05.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.126+0000 7fd962ffd700 1 --2- 172.21.15.89:0/810797180 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd94c066f30 0x7fd94c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:05.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.126+0000 7fd96a59c700 1 --2- 172.21.15.89:0/810797180 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd94c066f30 0x7fd94c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:05.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.126+0000 7fd962ffd700 1 -- 172.21.15.89:0/810797180 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd954086100 con 0x7fd96c1597f0 2023-10-10T20:36:05.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.126+0000 7fd96a59c700 1 --2- 172.21.15.89:0/810797180 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd94c066f30 0x7fd94c0693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd95c007fd0 tx=0x7fd95c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:05.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.130+0000 7fd962ffd700 1 -- 172.21.15.89:0/810797180 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd954053370 con 0x7fd96c1597f0 2023-10-10T20:36:05.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.340+0000 7fd96b59e700 1 -- 172.21.15.89:0/810797180 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd96c04c430 con 0x7fd94c066f30 2023-10-10T20:36:05.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.341+0000 7fd962ffd700 1 -- 172.21.15.89:0/810797180 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fd96c04c430 con 0x7fd94c066f30 2023-10-10T20:36:05.344 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:36:05.344 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:05.344 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs" 2023-10-10T20:36:05.344 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:05.344 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:36:05.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.343+0000 7fd96b59e700 1 -- 172.21.15.89:0/810797180 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd94c066f30 msgr2=0x7fd94c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:05.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.343+0000 7fd96b59e700 1 --2- 172.21.15.89:0/810797180 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd94c066f30 0x7fd94c0693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd95c007fd0 tx=0x7fd95c009040 comp rx=0 tx=0).stop 2023-10-10T20:36:05.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.343+0000 7fd96b59e700 1 -- 172.21.15.89:0/810797180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd96c1597f0 msgr2=0x7fd96c16d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:05.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.343+0000 7fd96b59e700 1 --2- 172.21.15.89:0/810797180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd96c1597f0 0x7fd96c16d0f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd954009ed0 tx=0x7fd954017000 comp rx=0 tx=0).stop 2023-10-10T20:36:05.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.343+0000 7fd96b59e700 1 -- 172.21.15.89:0/810797180 shutdown_connections 2023-10-10T20:36:05.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.343+0000 7fd96b59e700 1 --2- 172.21.15.89:0/810797180 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd96c15d230 0x7fd96c155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.343+0000 7fd96b59e700 1 --2- 172.21.15.89:0/810797180 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd94c066f30 0x7fd94c0693e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.343+0000 7fd96b59e700 1 --2- 172.21.15.89:0/810797180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd96c159420 0x7fd96c153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.343+0000 7fd96b59e700 1 --2- 172.21.15.89:0/810797180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd96c1597f0 0x7fd96c16d0f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.343+0000 7fd96b59e700 1 -- 172.21.15.89:0/810797180 >> 172.21.15.89:0/810797180 conn(0x7fd96c0d0660 msgr2=0x7fd96c0b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:05.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.343+0000 7fd96b59e700 1 -- 172.21.15.89:0/810797180 shutdown_connections 2023-10-10T20:36:05.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.343+0000 7fd96b59e700 1 -- 172.21.15.89:0/810797180 wait complete. 2023-10-10T20:36:05.357 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:36:05.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.472+0000 7fa19e262700 1 Processor -- start 2023-10-10T20:36:05.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.472+0000 7fa19e262700 1 -- start start 2023-10-10T20:36:05.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.473+0000 7fa19e262700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa1980b1b40 0x7fa1980b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:05.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.473+0000 7fa19e262700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1980cfb00 con 0x7fa1980b16d0 2023-10-10T20:36:05.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.473+0000 7fa19e262700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1980b5c40 con 0x7fa1980b4e70 2023-10-10T20:36:05.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.473+0000 7fa19e262700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1980b5f20 con 0x7fa1980b1b40 2023-10-10T20:36:05.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.473+0000 7fa19ca5f700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa1980b16d0 0x7fa1980b33b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58836/0 (socket says 172.21.15.89:58836) 2023-10-10T20:36:05.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.473+0000 7fa19ca5f700 1 -- 172.21.15.89:0/3214864372 learned_addr learned my addr 172.21.15.89:0/3214864372 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:05.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.473+0000 7fa19da61700 1 --2- 172.21.15.89:0/3214864372 >> v2:172.21.15.111:3300/0 conn(0x7fa1980b1b40 0x7fa1980b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:05.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.473+0000 7fa18ffff700 1 -- 172.21.15.89:0/3214864372 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342601402 0 0) 0x7fa1980cfb00 con 0x7fa1980b16d0 2023-10-10T20:36:05.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.473+0000 7fa18ffff700 1 -- 172.21.15.89:0/3214864372 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa184003570 con 0x7fa1980b16d0 2023-10-10T20:36:05.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.473+0000 7fa18ffff700 1 -- 172.21.15.89:0/3214864372 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 451931697 0 0) 0x7fa184003570 con 0x7fa1980b16d0 2023-10-10T20:36:05.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.473+0000 7fa18ffff700 1 -- 172.21.15.89:0/3214864372 >> v2:172.21.15.111:3300/0 conn(0x7fa1980b1b40 msgr2=0x7fa1980b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:05.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.473+0000 7fa18ffff700 1 --2- 172.21.15.89:0/3214864372 >> v2:172.21.15.111:3300/0 conn(0x7fa1980b1b40 0x7fa1980b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.473+0000 7fa18ffff700 1 -- 172.21.15.89:0/3214864372 >> v1:172.21.15.111:6789/0 conn(0x7fa1980b4e70 legacy=0x7fa1980ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:05.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.473+0000 7fa18ffff700 1 -- 172.21.15.89:0/3214864372 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1980b6200 con 0x7fa1980b16d0 2023-10-10T20:36:05.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.473+0000 7fa18ffff700 1 -- 172.21.15.89:0/3214864372 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fa188001470 con 0x7fa1980b16d0 2023-10-10T20:36:05.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa18ffff700 1 -- 172.21.15.89:0/3214864372 >> v1:172.21.15.89:6789/0 conn(0x7fa1980b16d0 legacy=0x7fa1980b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:05.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa18ffff700 1 --2- 172.21.15.89:0/3214864372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa184003a60 0x7fa184003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:05.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa18ffff700 1 --2- 172.21.15.89:0/3214864372 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa184004810 0x7fa184004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:05.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa18ffff700 1 --2- 172.21.15.89:0/3214864372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1840052e0 0x7fa18400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:05.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa18ffff700 1 -- 172.21.15.89:0/3214864372 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980b5f20 con 0x7fa1840052e0 2023-10-10T20:36:05.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa18ffff700 1 -- 172.21.15.89:0/3214864372 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18400c2a0 con 0x7fa184003a60 2023-10-10T20:36:05.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa18ffff700 1 -- 172.21.15.89:0/3214864372 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18400c460 con 0x7fa184004810 2023-10-10T20:36:05.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa19da61700 1 --2- 172.21.15.89:0/3214864372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1840052e0 0x7fa18400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:05.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa19d260700 1 --2- 172.21.15.89:0/3214864372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa184003a60 0x7fa184003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:05.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa19ca5f700 1 --2- 172.21.15.89:0/3214864372 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa184004810 0x7fa184004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:05.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa19da61700 1 -- 172.21.15.89:0/3214864372 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa184004810 msgr2=0x7fa184004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:05.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa19da61700 1 --2- 172.21.15.89:0/3214864372 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa184004810 0x7fa184004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa19da61700 1 -- 172.21.15.89:0/3214864372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa184003a60 msgr2=0x7fa184003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:05.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa19da61700 1 --2- 172.21.15.89:0/3214864372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa184003a60 0x7fa184003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa19da61700 1 -- 172.21.15.89:0/3214864372 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa184003570 con 0x7fa1840052e0 2023-10-10T20:36:05.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa19da61700 1 -- 172.21.15.89:0/3214864372 reap_dead start 2023-10-10T20:36:05.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa19da61700 1 --2- 172.21.15.89:0/3214864372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1840052e0 0x7fa18400ba60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa19000e210 tx=0x7fa19000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601a05f4989380ff server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:05.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa18ffff700 1 -- 172.21.15.89:0/3214864372 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19000eac0 con 0x7fa1840052e0 2023-10-10T20:36:05.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa18ffff700 1 -- 172.21.15.89:0/3214864372 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19000f040 con 0x7fa1840052e0 2023-10-10T20:36:05.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.474+0000 7fa18ffff700 1 -- 172.21.15.89:0/3214864372 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19000ee70 con 0x7fa1840052e0 2023-10-10T20:36:05.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.475+0000 7fa19e262700 1 -- 172.21.15.89:0/3214864372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1840052e0 msgr2=0x7fa18400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:05.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.475+0000 7fa19e262700 1 --2- 172.21.15.89:0/3214864372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1840052e0 0x7fa18400ba60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa19000e210 tx=0x7fa19000b040 comp rx=0 tx=0).stop 2023-10-10T20:36:05.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.475+0000 7fa19e262700 1 -- 172.21.15.89:0/3214864372 shutdown_connections 2023-10-10T20:36:05.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.475+0000 7fa19e262700 1 --2- 172.21.15.89:0/3214864372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1840052e0 0x7fa18400ba60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.475+0000 7fa19e262700 1 -- 172.21.15.89:0/3214864372 >> 172.21.15.89:0/3214864372 conn(0x7fa1980d10c0 msgr2=0x7fa1980affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:05.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.475+0000 7fa19e262700 1 -- 172.21.15.89:0/3214864372 shutdown_connections 2023-10-10T20:36:05.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.475+0000 7fa19e262700 1 -- 172.21.15.89:0/3214864372 wait complete. 2023-10-10T20:36:05.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.476+0000 7fa19e262700 1 Processor -- start 2023-10-10T20:36:05.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.476+0000 7fa19e262700 1 -- start start 2023-10-10T20:36:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa19e262700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa1840052e0 0x7fa1980cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa19e262700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1980c6a20 0x7fa1980c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:05.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa19e262700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1980c9910 0x7fa19816d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:05.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa19e262700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19816af30 con 0x7fa1980c6a20 2023-10-10T20:36:05.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa19e262700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa198158830 con 0x7fa1980c9910 2023-10-10T20:36:05.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa19e262700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980c9d80 con 0x7fa1840052e0 2023-10-10T20:36:05.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa19ca5f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1980c6a20 0x7fa1980c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:05.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa19ca5f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1980c6a20 0x7fa1980c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41172/0 (socket says 172.21.15.89:41172) 2023-10-10T20:36:05.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa19ca5f700 1 -- 172.21.15.89:0/2281357646 learned_addr learned my addr 172.21.15.89:0/2281357646 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:05.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa19d260700 1 --2- 172.21.15.89:0/2281357646 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa1840052e0 0x7fa1980cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:05.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa19da61700 1 --2- 172.21.15.89:0/2281357646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1980c9910 0x7fa19816d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa19ca5f700 1 -- 172.21.15.89:0/2281357646 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa1840052e0 msgr2=0x7fa1980cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa19ca5f700 1 --2- 172.21.15.89:0/2281357646 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa1840052e0 0x7fa1980cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa19ca5f700 1 -- 172.21.15.89:0/2281357646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1980c9910 msgr2=0x7fa19816d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa19ca5f700 1 --2- 172.21.15.89:0/2281357646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1980c9910 0x7fa19816d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa19ca5f700 1 -- 172.21.15.89:0/2281357646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19816a2b0 con 0x7fa1980c6a20 2023-10-10T20:36:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa19ca5f700 1 --2- 172.21.15.89:0/2281357646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1980c6a20 0x7fa1980c93d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa18800eb00 tx=0x7fa188001010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa18d7fa700 1 -- 172.21.15.89:0/2281357646 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18800eec0 con 0x7fa1980c6a20 2023-10-10T20:36:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa18d7fa700 1 -- 172.21.15.89:0/2281357646 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa188010040 con 0x7fa1980c6a20 2023-10-10T20:36:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.477+0000 7fa19e262700 1 -- 172.21.15.89:0/2281357646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa19816a590 con 0x7fa1980c6a20 2023-10-10T20:36:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.478+0000 7fa18d7fa700 1 -- 172.21.15.89:0/2281357646 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18800a970 con 0x7fa1980c6a20 2023-10-10T20:36:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.478+0000 7fa19e262700 1 -- 172.21.15.89:0/2281357646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19816ab50 con 0x7fa1980c6a20 2023-10-10T20:36:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.478+0000 7fa19e262700 1 -- 172.21.15.89:0/2281357646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa198088990 con 0x7fa1980c6a20 2023-10-10T20:36:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.479+0000 7fa18d7fa700 1 -- 172.21.15.89:0/2281357646 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa18800ad20 con 0x7fa1980c6a20 2023-10-10T20:36:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.479+0000 7fa18d7fa700 1 --2- 172.21.15.89:0/2281357646 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa17c066f30 0x7fa17c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:05.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.479+0000 7fa19d260700 1 --2- 172.21.15.89:0/2281357646 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa17c066f30 0x7fa17c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:05.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.479+0000 7fa18d7fa700 1 -- 172.21.15.89:0/2281357646 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa188083840 con 0x7fa1980c6a20 2023-10-10T20:36:05.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.480+0000 7fa19d260700 1 --2- 172.21.15.89:0/2281357646 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa17c066f30 0x7fa17c0693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa194007e40 tx=0x7fa194007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:05.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.483+0000 7fa18d7fa700 1 -- 172.21.15.89:0/2281357646 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa188050b70 con 0x7fa1980c6a20 2023-10-10T20:36:05.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.666+0000 7fa19e262700 1 -- 172.21.15.89:0/2281357646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa1980c91b0 con 0x7fa1980c6a20 2023-10-10T20:36:05.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.667+0000 7fa18d7fa700 1 -- 172.21.15.89:0/2281357646 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa188019070 con 0x7fa1980c6a20 2023-10-10T20:36:05.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.669+0000 7fa19e262700 1 -- 172.21.15.89:0/2281357646 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa17c066f30 msgr2=0x7fa17c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:05.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.669+0000 7fa19e262700 1 --2- 172.21.15.89:0/2281357646 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa17c066f30 0x7fa17c0693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa194007e40 tx=0x7fa194007d30 comp rx=0 tx=0).stop 2023-10-10T20:36:05.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.669+0000 7fa19e262700 1 -- 172.21.15.89:0/2281357646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1980c6a20 msgr2=0x7fa1980c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:05.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.669+0000 7fa19e262700 1 --2- 172.21.15.89:0/2281357646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1980c6a20 0x7fa1980c93d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa18800eb00 tx=0x7fa188001010 comp rx=0 tx=0).stop 2023-10-10T20:36:05.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.669+0000 7fa19e262700 1 -- 172.21.15.89:0/2281357646 shutdown_connections 2023-10-10T20:36:05.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.669+0000 7fa19e262700 1 --2- 172.21.15.89:0/2281357646 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa17c066f30 0x7fa17c0693e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.669+0000 7fa19e262700 1 --2- 172.21.15.89:0/2281357646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1980c9910 0x7fa19816d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.669+0000 7fa19e262700 1 --2- 172.21.15.89:0/2281357646 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa1840052e0 0x7fa1980cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.669+0000 7fa19e262700 1 --2- 172.21.15.89:0/2281357646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1980c6a20 0x7fa1980c93d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.669+0000 7fa19e262700 1 -- 172.21.15.89:0/2281357646 >> 172.21.15.89:0/2281357646 conn(0x7fa1980d10c0 msgr2=0x7fa1980c5390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:05.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.669+0000 7fa19e262700 1 -- 172.21.15.89:0/2281357646 shutdown_connections 2023-10-10T20:36:05.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.669+0000 7fa19e262700 1 -- 172.21.15.89:0/2281357646 wait complete. 2023-10-10T20:36:05.683 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000303131 2023-10-10T20:36:05.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.797+0000 7fabfea9d700 1 Processor -- start 2023-10-10T20:36:05.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.797+0000 7fabfea9d700 1 -- start start 2023-10-10T20:36:05.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.798+0000 7fabfea9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabf8159890 0x7fabf815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:05.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.798+0000 7fabfea9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fabf8159c60 0x7fabf815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:05.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.798+0000 7fabfea9d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf815a160 con 0x7fabf815db10 2023-10-10T20:36:05.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.798+0000 7fabfea9d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf80d13a0 con 0x7fabf8159c60 2023-10-10T20:36:05.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.798+0000 7fabfea9d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf815e140 con 0x7fabf8159890 2023-10-10T20:36:05.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.798+0000 7fabeffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fabf8159c60 0x7fabf815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:05.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.798+0000 7fabfd03a700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fabf815db10 0x7fabf8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58850/0 (socket says 172.21.15.89:58850) 2023-10-10T20:36:05.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.798+0000 7fabeffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fabf8159c60 0x7fabf815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41188/0 (socket says 172.21.15.89:41188) 2023-10-10T20:36:05.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.798+0000 7fabfd03a700 1 -- 172.21.15.89:0/1390230317 learned_addr learned my addr 172.21.15.89:0/1390230317 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:05.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.798+0000 7fabfc839700 1 --2- 172.21.15.89:0/1390230317 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabf8159890 0x7fabf815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:05.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.799+0000 7fabeffff700 1 -- 172.21.15.89:0/1390230317 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabf8159890 msgr2=0x7fabf815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:05.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.799+0000 7fabeffff700 1 --2- 172.21.15.89:0/1390230317 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabf8159890 0x7fabf815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.799+0000 7fabeffff700 1 -- 172.21.15.89:0/1390230317 >> v1:172.21.15.89:6789/0 conn(0x7fabf815db10 legacy=0x7fabf8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:05.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.799+0000 7fabeffff700 1 -- 172.21.15.89:0/1390230317 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf815a3d0 con 0x7fabf8159c60 2023-10-10T20:36:05.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.799+0000 7fabef7fe700 1 -- 172.21.15.89:0/1390230317 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216146120 0 0) 0x7fabf815a160 con 0x7fabf815db10 2023-10-10T20:36:05.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.799+0000 7fabeffff700 1 --2- 172.21.15.89:0/1390230317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabf8159c60 0x7fabf815d5d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fabe80098e0 tx=0x7fabe80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8651a23499d2738 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:05.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.799+0000 7fabef7fe700 1 -- 172.21.15.89:0/1390230317 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe800b640 con 0x7fabf8159c60 2023-10-10T20:36:05.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.799+0000 7fabef7fe700 1 -- 172.21.15.89:0/1390230317 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe800b7a0 con 0x7fabf8159c60 2023-10-10T20:36:05.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.799+0000 7fabef7fe700 1 -- 172.21.15.89:0/1390230317 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe800bb40 con 0x7fabf8159c60 2023-10-10T20:36:05.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.799+0000 7fabfea9d700 1 -- 172.21.15.89:0/1390230317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabf8159c60 msgr2=0x7fabf815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:05.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.799+0000 7fabfea9d700 1 --2- 172.21.15.89:0/1390230317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabf8159c60 0x7fabf815d5d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fabe80098e0 tx=0x7fabe80091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.799+0000 7fabfea9d700 1 -- 172.21.15.89:0/1390230317 shutdown_connections 2023-10-10T20:36:05.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.799+0000 7fabfea9d700 1 --2- 172.21.15.89:0/1390230317 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabf8159890 0x7fabf815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.799+0000 7fabfea9d700 1 --2- 172.21.15.89:0/1390230317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabf8159c60 0x7fabf815d5d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.799+0000 7fabfea9d700 1 -- 172.21.15.89:0/1390230317 >> 172.21.15.89:0/1390230317 conn(0x7fabf8068e20 msgr2=0x7fabf80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:05.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.799+0000 7fabfea9d700 1 -- 172.21.15.89:0/1390230317 shutdown_connections 2023-10-10T20:36:05.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.799+0000 7fabfea9d700 1 -- 172.21.15.89:0/1390230317 wait complete. 2023-10-10T20:36:05.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.801+0000 7fabfea9d700 1 Processor -- start 2023-10-10T20:36:05.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.801+0000 7fabfea9d700 1 -- start start 2023-10-10T20:36:05.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.801+0000 7fabfea9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabf8159890 0x7fabf80cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:05.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.801+0000 7fabfea9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabf8159c60 0x7fabf80cf630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:05.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.801+0000 7fabfea9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabf815db10 0x7fabf8149330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:05.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.801+0000 7fabfea9d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf8151de0 con 0x7fabf815db10 2023-10-10T20:36:05.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.801+0000 7fabfea9d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf80cfb70 con 0x7fabf8159890 2023-10-10T20:36:05.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.801+0000 7fabfea9d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf81514b0 con 0x7fabf8159c60 2023-10-10T20:36:05.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.802+0000 7fabfd03a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabf815db10 0x7fabf8149330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:05.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.802+0000 7fabfd03a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabf815db10 0x7fabf8149330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41190/0 (socket says 172.21.15.89:41190) 2023-10-10T20:36:05.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.802+0000 7fabfd03a700 1 -- 172.21.15.89:0/2487067079 learned_addr learned my addr 172.21.15.89:0/2487067079 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:05.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.802+0000 7fabfc839700 1 --2- 172.21.15.89:0/2487067079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabf8159890 0x7fabf80cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:05.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.802+0000 7fabeffff700 1 --2- 172.21.15.89:0/2487067079 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabf8159c60 0x7fabf80cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:05.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.802+0000 7fabfd03a700 1 -- 172.21.15.89:0/2487067079 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabf8159c60 msgr2=0x7fabf80cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:05.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.802+0000 7fabfd03a700 1 --2- 172.21.15.89:0/2487067079 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabf8159c60 0x7fabf80cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.802+0000 7fabfd03a700 1 -- 172.21.15.89:0/2487067079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabf8159890 msgr2=0x7fabf80cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:05.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.802+0000 7fabfd03a700 1 --2- 172.21.15.89:0/2487067079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabf8159890 0x7fabf80cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:05.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.802+0000 7fabfd03a700 1 -- 172.21.15.89:0/2487067079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe8009420 con 0x7fabf815db10 2023-10-10T20:36:05.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.802+0000 7fabfc839700 1 --2- 172.21.15.89:0/2487067079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabf8159890 0x7fabf80cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:05.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.802+0000 7fabfd03a700 1 --2- 172.21.15.89:0/2487067079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabf815db10 0x7fabf8149330 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fabf000c530 tx=0x7fabf0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:05.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.802+0000 7fabecff9700 1 -- 172.21.15.89:0/2487067079 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf0005b30 con 0x7fabf815db10 2023-10-10T20:36:05.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.802+0000 7fabecff9700 1 -- 172.21.15.89:0/2487067079 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf0005c90 con 0x7fabf815db10 2023-10-10T20:36:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.802+0000 7fabfea9d700 1 -- 172.21.15.89:0/2487067079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf8151790 con 0x7fabf815db10 2023-10-10T20:36:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.802+0000 7fabecff9700 1 -- 172.21.15.89:0/2487067079 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf0007320 con 0x7fabf815db10 2023-10-10T20:36:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.802+0000 7fabfea9d700 1 -- 172.21.15.89:0/2487067079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf80cfe30 con 0x7fabf815db10 2023-10-10T20:36:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.803+0000 7fabfea9d700 1 -- 172.21.15.89:0/2487067079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf804c3c0 con 0x7fabf815db10 2023-10-10T20:36:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.804+0000 7fabecff9700 1 -- 172.21.15.89:0/2487067079 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fabf001b3b0 con 0x7fabf815db10 2023-10-10T20:36:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.804+0000 7fabecff9700 1 --2- 172.21.15.89:0/2487067079 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fabe4066f30 0x7fabe40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:05.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.804+0000 7fabecff9700 1 -- 172.21.15.89:0/2487067079 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fabf0083a50 con 0x7fabf815db10 2023-10-10T20:36:05.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.804+0000 7fabfc839700 1 --2- 172.21.15.89:0/2487067079 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fabe4066f30 0x7fabe40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:05.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.804+0000 7fabfc839700 1 --2- 172.21.15.89:0/2487067079 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fabe4066f30 0x7fabe40693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fabf40076c0 tx=0x7fabf400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:05.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:05.808+0000 7fabecff9700 1 -- 172.21.15.89:0/2487067079 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fabf0050cc0 con 0x7fabf815db10 2023-10-10T20:36:06.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.021+0000 7fabfea9d700 1 -- 172.21.15.89:0/2487067079 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group_0000000000303131", "target": ["mon-mgr", ""]}) v1 -- 0x7fabf8151f20 con 0x7fabe4066f30 2023-10-10T20:36:06.027 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:36:06.025+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:36:06.027 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:36:06.025+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:36:06.027 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:36:06.025+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:36:06.027 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:36:06.025+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:36:06.027 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:36:06.025+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:36:06.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.038+0000 7fabecff9700 1 -- 172.21.15.89:0/2487067079 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fabf8151f20 con 0x7fabe4066f30 2023-10-10T20:36:06.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.040+0000 7fabfea9d700 1 -- 172.21.15.89:0/2487067079 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fabe4066f30 msgr2=0x7fabe40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:06.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.040+0000 7fabfea9d700 1 --2- 172.21.15.89:0/2487067079 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fabe4066f30 0x7fabe40693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fabf40076c0 tx=0x7fabf400b040 comp rx=0 tx=0).stop 2023-10-10T20:36:06.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.040+0000 7fabfea9d700 1 -- 172.21.15.89:0/2487067079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabf815db10 msgr2=0x7fabf8149330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:06.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.040+0000 7fabfea9d700 1 --2- 172.21.15.89:0/2487067079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabf815db10 0x7fabf8149330 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fabf000c530 tx=0x7fabf0007f90 comp rx=0 tx=0).stop 2023-10-10T20:36:06.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.040+0000 7fabfea9d700 1 -- 172.21.15.89:0/2487067079 shutdown_connections 2023-10-10T20:36:06.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.040+0000 7fabfea9d700 1 --2- 172.21.15.89:0/2487067079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabf815db10 0x7fabf8149330 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:06.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.040+0000 7fabfea9d700 1 --2- 172.21.15.89:0/2487067079 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fabe4066f30 0x7fabe40693e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:06.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.040+0000 7fabfea9d700 1 --2- 172.21.15.89:0/2487067079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabf8159890 0x7fabf80cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:06.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.040+0000 7fabfea9d700 1 --2- 172.21.15.89:0/2487067079 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabf8159c60 0x7fabf80cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:06.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.040+0000 7fabfea9d700 1 -- 172.21.15.89:0/2487067079 >> 172.21.15.89:0/2487067079 conn(0x7fabf8068e20 msgr2=0x7fabf8069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:06.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.041+0000 7fabfea9d700 1 -- 172.21.15.89:0/2487067079 shutdown_connections 2023-10-10T20:36:06.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.041+0000 7fabfea9d700 1 -- 172.21.15.89:0/2487067079 wait complete. 2023-10-10T20:36:06.054 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs 2023-10-10T20:36:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.169+0000 7f5bb6c35700 1 Processor -- start 2023-10-10T20:36:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.169+0000 7f5bb6c35700 1 -- start start 2023-10-10T20:36:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5bb6c35700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb0069560 0x7f5bb00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5bb6c35700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bb0069ee0 con 0x7f5bb0159890 2023-10-10T20:36:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5bb6c35700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bb015a160 con 0x7f5bb0159c60 2023-10-10T20:36:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5bb6c35700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb00d13a0 con 0x7f5bb0069560 2023-10-10T20:36:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5bb49d1700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5bb0159890 0x7f5bb015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58860/0 (socket says 172.21.15.89:58860) 2023-10-10T20:36:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5bb49d1700 1 -- 172.21.15.89:0/1821753482 learned_addr learned my addr 172.21.15.89:0/1821753482 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5baf7fe700 1 -- 172.21.15.89:0/1821753482 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2371039570 0 0) 0x7f5bb0069ee0 con 0x7f5bb0159890 2023-10-10T20:36:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5bb51d2700 1 --2- 172.21.15.89:0/1821753482 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb0069560 0x7f5bb00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5baf7fe700 1 -- 172.21.15.89:0/1821753482 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ba0003570 con 0x7f5bb0159890 2023-10-10T20:36:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5baf7fe700 1 -- 172.21.15.89:0/1821753482 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 328539209 0 0) 0x7f5ba0003570 con 0x7f5bb0159890 2023-10-10T20:36:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5baf7fe700 1 -- 172.21.15.89:0/1821753482 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb0069560 msgr2=0x7f5bb00699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5baf7fe700 1 --2- 172.21.15.89:0/1821753482 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb0069560 0x7f5bb00699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5baf7fe700 1 -- 172.21.15.89:0/1821753482 >> v1:172.21.15.111:6789/0 conn(0x7f5bb0159c60 legacy=0x7f5bb015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5baf7fe700 1 -- 172.21.15.89:0/1821753482 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb015e140 con 0x7f5bb0159890 2023-10-10T20:36:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5baf7fe700 1 -- 172.21.15.89:0/1821753482 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f5ba4001470 con 0x7f5bb0159890 2023-10-10T20:36:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5baf7fe700 1 -- 172.21.15.89:0/1821753482 >> v1:172.21.15.89:6789/0 conn(0x7f5bb0159890 legacy=0x7f5bb015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5baf7fe700 1 --2- 172.21.15.89:0/1821753482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ba0003a90 0x7f5ba0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5baf7fe700 1 --2- 172.21.15.89:0/1821753482 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb0069560 0x7f5ba0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5baf7fe700 1 --2- 172.21.15.89:0/1821753482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ba0004d50 0x7f5ba00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5baf7fe700 1 -- 172.21.15.89:0/1821753482 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb00d13a0 con 0x7f5ba0003a90 2023-10-10T20:36:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.170+0000 7f5baf7fe700 1 -- 172.21.15.89:0/1821753482 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba000bba0 con 0x7f5ba0004d50 2023-10-10T20:36:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.171+0000 7f5baf7fe700 1 -- 172.21.15.89:0/1821753482 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba000bd60 con 0x7f5bb0069560 2023-10-10T20:36:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.171+0000 7f5bb49d1700 1 --2- 172.21.15.89:0/1821753482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ba0003a90 0x7f5ba0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.171+0000 7f5baffff700 1 --2- 172.21.15.89:0/1821753482 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb0069560 0x7f5ba0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.171+0000 7f5bb51d2700 1 --2- 172.21.15.89:0/1821753482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ba0004d50 0x7f5ba00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.171+0000 7f5bb49d1700 1 -- 172.21.15.89:0/1821753482 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb0069560 msgr2=0x7f5ba0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.171+0000 7f5bb49d1700 1 --2- 172.21.15.89:0/1821753482 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb0069560 0x7f5ba0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.171+0000 7f5bb49d1700 1 -- 172.21.15.89:0/1821753482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ba0004d50 msgr2=0x7f5ba00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.171+0000 7f5bb49d1700 1 --2- 172.21.15.89:0/1821753482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ba0004d50 0x7f5ba00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.171+0000 7f5bb49d1700 1 -- 172.21.15.89:0/1821753482 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba0003570 con 0x7f5ba0003a90 2023-10-10T20:36:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.171+0000 7f5bb51d2700 1 --2- 172.21.15.89:0/1821753482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ba0004d50 0x7f5ba00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.171+0000 7f5bb49d1700 1 --2- 172.21.15.89:0/1821753482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ba0003a90 0x7f5ba0003ea0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400c3c0 tx=0x7f5ba4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4045890b293b86b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.171+0000 7f5baf7fe700 1 -- 172.21.15.89:0/1821753482 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba400cc90 con 0x7f5ba0003a90 2023-10-10T20:36:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.171+0000 7f5baf7fe700 1 -- 172.21.15.89:0/1821753482 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba400cdf0 con 0x7f5ba0003a90 2023-10-10T20:36:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.171+0000 7f5baf7fe700 1 -- 172.21.15.89:0/1821753482 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4005b50 con 0x7f5ba0003a90 2023-10-10T20:36:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.172+0000 7f5bb6c35700 1 -- 172.21.15.89:0/1821753482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ba0003a90 msgr2=0x7f5ba0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.172+0000 7f5bb6c35700 1 --2- 172.21.15.89:0/1821753482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ba0003a90 0x7f5ba0003ea0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400c3c0 tx=0x7f5ba4001e50 comp rx=0 tx=0).stop 2023-10-10T20:36:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.172+0000 7f5bb51d2700 1 -- 172.21.15.89:0/1821753482 reap_dead start 2023-10-10T20:36:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.172+0000 7f5bb6c35700 1 -- 172.21.15.89:0/1821753482 shutdown_connections 2023-10-10T20:36:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.172+0000 7f5bb6c35700 1 -- 172.21.15.89:0/1821753482 >> 172.21.15.89:0/1821753482 conn(0x7f5bb0068e20 msgr2=0x7f5bb00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.172+0000 7f5bb6c35700 1 -- 172.21.15.89:0/1821753482 shutdown_connections 2023-10-10T20:36:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.172+0000 7f5bb6c35700 1 -- 172.21.15.89:0/1821753482 wait complete. 2023-10-10T20:36:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.173+0000 7f5bb6c35700 1 Processor -- start 2023-10-10T20:36:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.173+0000 7f5bb6c35700 1 -- start start 2023-10-10T20:36:06.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.174+0000 7f5bb6c35700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb0147150 0x7f5bb0169c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:06.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.174+0000 7f5bb6c35700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bb0147520 0x7f5bb00ce510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:06.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.174+0000 7f5bb6c35700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bb00ca870 0x7f5bb00cea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:06.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.174+0000 7f5bb6c35700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb016a580 con 0x7f5bb00ca870 2023-10-10T20:36:06.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.174+0000 7f5bb6c35700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb016a860 con 0x7f5bb0147520 2023-10-10T20:36:06.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.174+0000 7f5bb6c35700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb016ab40 con 0x7f5bb0147150 2023-10-10T20:36:06.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.174+0000 7f5bb51d2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bb00ca870 0x7f5bb00cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:06.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.174+0000 7f5bb51d2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bb00ca870 0x7f5bb00cea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41214/0 (socket says 172.21.15.89:41214) 2023-10-10T20:36:06.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.174+0000 7f5bb51d2700 1 -- 172.21.15.89:0/4136237699 learned_addr learned my addr 172.21.15.89:0/4136237699 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:06.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.174+0000 7f5baffff700 1 --2- 172.21.15.89:0/4136237699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bb0147520 0x7f5bb00ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:06.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.174+0000 7f5bb49d1700 1 --2- 172.21.15.89:0/4136237699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb0147150 0x7f5bb0169c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:06.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.174+0000 7f5bb51d2700 1 -- 172.21.15.89:0/4136237699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb0147150 msgr2=0x7f5bb0169c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:06.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.174+0000 7f5bb51d2700 1 --2- 172.21.15.89:0/4136237699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb0147150 0x7f5bb0169c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:06.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.174+0000 7f5bb51d2700 1 -- 172.21.15.89:0/4136237699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bb0147520 msgr2=0x7f5bb00ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:06.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.174+0000 7f5bb51d2700 1 --2- 172.21.15.89:0/4136237699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bb0147520 0x7f5bb00ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:06.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.174+0000 7f5bb51d2700 1 -- 172.21.15.89:0/4136237699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb00b47d0 con 0x7f5bb00ca870 2023-10-10T20:36:06.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.174+0000 7f5bb51d2700 1 --2- 172.21.15.89:0/4136237699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bb00ca870 0x7f5bb00cea50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00cadc0 tx=0x7f5ba800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:06.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.175+0000 7f5bacff9700 1 -- 172.21.15.89:0/4136237699 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba801b070 con 0x7f5bb00ca870 2023-10-10T20:36:06.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.175+0000 7f5bacff9700 1 -- 172.21.15.89:0/4136237699 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba800f040 con 0x7f5bb00ca870 2023-10-10T20:36:06.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.175+0000 7f5bb6c35700 1 -- 172.21.15.89:0/4136237699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb00b4ab0 con 0x7f5bb00ca870 2023-10-10T20:36:06.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.175+0000 7f5bacff9700 1 -- 172.21.15.89:0/4136237699 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba800e050 con 0x7f5bb00ca870 2023-10-10T20:36:06.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.175+0000 7f5bb6c35700 1 -- 172.21.15.89:0/4136237699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb00b3590 con 0x7f5bb00ca870 2023-10-10T20:36:06.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.175+0000 7f5bb6c35700 1 -- 172.21.15.89:0/4136237699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb00b4ca0 con 0x7f5bb00ca870 2023-10-10T20:36:06.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.176+0000 7f5bacff9700 1 -- 172.21.15.89:0/4136237699 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ba8003020 con 0x7f5bb00ca870 2023-10-10T20:36:06.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.176+0000 7f5bacff9700 1 --2- 172.21.15.89:0/4136237699 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b94066f30 0x7f5b940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.176+0000 7f5bacff9700 1 -- 172.21.15.89:0/4136237699 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5ba80847c0 con 0x7f5bb00ca870 2023-10-10T20:36:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.176+0000 7f5bb49d1700 1 --2- 172.21.15.89:0/4136237699 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b94066f30 0x7f5b940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.177+0000 7f5bb49d1700 1 --2- 172.21.15.89:0/4136237699 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b94066f30 0x7f5b940693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4007cd0 tx=0x7f5ba4015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.180+0000 7f5bacff9700 1 -- 172.21.15.89:0/4136237699 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ba8051a30 con 0x7f5bb00ca870 2023-10-10T20:36:06.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.389+0000 7f5bb6c35700 1 -- 172.21.15.89:0/4136237699 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bb004c3c0 con 0x7f5b94066f30 2023-10-10T20:36:06.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.391+0000 7f5bacff9700 1 -- 172.21.15.89:0/4136237699 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+530 (secure 0 0 0) 0x7f5bb004c3c0 con 0x7f5b94066f30 2023-10-10T20:36:06.394 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:36:06.394 INFO:teuthology.orchestra.run.smithi089.stdout: "mon_addrs": [ 2023-10-10T20:36:06.394 INFO:teuthology.orchestra.run.smithi089.stdout: "172.21.15.89:6789", 2023-10-10T20:36:06.394 INFO:teuthology.orchestra.run.smithi089.stdout: "172.21.15.111:6789", 2023-10-10T20:36:06.395 INFO:teuthology.orchestra.run.smithi089.stdout: "172.21.15.111:6790" 2023-10-10T20:36:06.395 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-10T20:36:06.395 INFO:teuthology.orchestra.run.smithi089.stdout: "pending_subvolume_deletions": 0, 2023-10-10T20:36:06.395 INFO:teuthology.orchestra.run.smithi089.stdout: "pools": { 2023-10-10T20:36:06.395 INFO:teuthology.orchestra.run.smithi089.stdout: "data": [ 2023-10-10T20:36:06.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:06.396 INFO:teuthology.orchestra.run.smithi089.stdout: "avail": 365997195264, 2023-10-10T20:36:06.396 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs_data", 2023-10-10T20:36:06.396 INFO:teuthology.orchestra.run.smithi089.stdout: "used": 0 2023-10-10T20:36:06.396 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:06.397 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-10T20:36:06.397 INFO:teuthology.orchestra.run.smithi089.stdout: "metadata": [ 2023-10-10T20:36:06.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:06.397 INFO:teuthology.orchestra.run.smithi089.stdout: "avail": 365997195264, 2023-10-10T20:36:06.397 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs_metadata", 2023-10-10T20:36:06.397 INFO:teuthology.orchestra.run.smithi089.stdout: "used": 65536 2023-10-10T20:36:06.398 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:06.398 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-10T20:36:06.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:36:06.398 INFO:teuthology.orchestra.run.smithi089.stdout: "used_size": 0 2023-10-10T20:36:06.398 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:36:06.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.393+0000 7f5bb6c35700 1 -- 172.21.15.89:0/4136237699 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b94066f30 msgr2=0x7f5b940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:06.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.393+0000 7f5bb6c35700 1 --2- 172.21.15.89:0/4136237699 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b94066f30 0x7f5b940693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4007cd0 tx=0x7f5ba4015000 comp rx=0 tx=0).stop 2023-10-10T20:36:06.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.393+0000 7f5bb6c35700 1 -- 172.21.15.89:0/4136237699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bb00ca870 msgr2=0x7f5bb00cea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:06.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.393+0000 7f5bb6c35700 1 --2- 172.21.15.89:0/4136237699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bb00ca870 0x7f5bb00cea50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00cadc0 tx=0x7f5ba800b040 comp rx=0 tx=0).stop 2023-10-10T20:36:06.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.393+0000 7f5bb6c35700 1 -- 172.21.15.89:0/4136237699 shutdown_connections 2023-10-10T20:36:06.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.393+0000 7f5bb6c35700 1 --2- 172.21.15.89:0/4136237699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bb00ca870 0x7f5bb00cea50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:06.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.393+0000 7f5bb6c35700 1 --2- 172.21.15.89:0/4136237699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bb0147150 0x7f5bb0169c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:06.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.393+0000 7f5bb6c35700 1 --2- 172.21.15.89:0/4136237699 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b94066f30 0x7f5b940693e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:06.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.393+0000 7f5bb6c35700 1 --2- 172.21.15.89:0/4136237699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bb0147520 0x7f5bb00ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:06.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.393+0000 7f5bb6c35700 1 -- 172.21.15.89:0/4136237699 >> 172.21.15.89:0/4136237699 conn(0x7f5bb0068e20 msgr2=0x7f5bb0153070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:06.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.393+0000 7f5bb6c35700 1 -- 172.21.15.89:0/4136237699 shutdown_connections 2023-10-10T20:36:06.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.393+0000 7f5bb6c35700 1 -- 172.21.15.89:0/4136237699 wait complete. 2023-10-10T20:36:06.407 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:36:06.408 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:36:06.446 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:36:06.499 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:36:06.499 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:36:06.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:06.500 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:36:06.537 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:36:06.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:06.538 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:36:06.593 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:36:06.593 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:06.594 DEBUG:teuthology.orchestra.run.smithi111:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:36:06.644 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:36:06.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:06.645 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:36:06.662 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:36:06.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.774+0000 7fad06239700 1 Processor -- start 2023-10-10T20:36:06.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.774+0000 7fad06239700 1 -- start start 2023-10-10T20:36:06.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.775+0000 7fad06239700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fad001597f0 0x7fad000b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:06.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.775+0000 7fad06239700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad000b11d0 con 0x7fad0015d230 2023-10-10T20:36:06.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.775+0000 7fad06239700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad000b14b0 con 0x7fad00159420 2023-10-10T20:36:06.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.775+0000 7fad06239700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad000b1790 con 0x7fad001597f0 2023-10-10T20:36:06.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.775+0000 7fad04a36700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fad001597f0 0x7fad000b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:06.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.775+0000 7fad05a38700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fad0015d230 0x7fad0015d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58876/0 (socket says 172.21.15.89:58876) 2023-10-10T20:36:06.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.775+0000 7fad05a38700 1 -- 172.21.15.89:0/2718467490 learned_addr learned my addr 172.21.15.89:0/2718467490 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:06.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.775+0000 7facfffff700 1 -- 172.21.15.89:0/2718467490 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 150252488 0 0) 0x7fad000b11d0 con 0x7fad0015d230 2023-10-10T20:36:06.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.775+0000 7facfffff700 1 -- 172.21.15.89:0/2718467490 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7face8003570 con 0x7fad0015d230 2023-10-10T20:36:06.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.775+0000 7fad04a36700 1 -- 172.21.15.89:0/2718467490 >> v1:172.21.15.111:6789/0 conn(0x7fad00159420 legacy=0x7fad000b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:06.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.775+0000 7fad04a36700 1 -- 172.21.15.89:0/2718467490 >> v1:172.21.15.89:6789/0 conn(0x7fad0015d230 legacy=0x7fad0015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:06.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.775+0000 7fad04a36700 1 -- 172.21.15.89:0/2718467490 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad000b1a70 con 0x7fad001597f0 2023-10-10T20:36:06.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.775+0000 7fad04a36700 1 --2- 172.21.15.89:0/2718467490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad001597f0 0x7fad000b2270 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7facf40098e0 tx=0x7facf40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69de4b935b224b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:06.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.775+0000 7facfffff700 1 -- 172.21.15.89:0/2718467490 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf400b640 con 0x7fad001597f0 2023-10-10T20:36:06.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.776+0000 7facfffff700 1 -- 172.21.15.89:0/2718467490 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf400b7a0 con 0x7fad001597f0 2023-10-10T20:36:06.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.776+0000 7facfffff700 1 -- 172.21.15.89:0/2718467490 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf400bb40 con 0x7fad001597f0 2023-10-10T20:36:06.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.776+0000 7fad06239700 1 -- 172.21.15.89:0/2718467490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad001597f0 msgr2=0x7fad000b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:06.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.776+0000 7fad06239700 1 --2- 172.21.15.89:0/2718467490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad001597f0 0x7fad000b2270 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7facf40098e0 tx=0x7facf40091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:06.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.776+0000 7fad06239700 1 -- 172.21.15.89:0/2718467490 shutdown_connections 2023-10-10T20:36:06.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.776+0000 7fad06239700 1 --2- 172.21.15.89:0/2718467490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad001597f0 0x7fad000b2270 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:06.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.776+0000 7fad06239700 1 -- 172.21.15.89:0/2718467490 >> 172.21.15.89:0/2718467490 conn(0x7fad000d0660 msgr2=0x7fad000d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:06.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.776+0000 7fad06239700 1 -- 172.21.15.89:0/2718467490 shutdown_connections 2023-10-10T20:36:06.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.776+0000 7fad06239700 1 -- 172.21.15.89:0/2718467490 wait complete. 2023-10-10T20:36:06.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.777+0000 7fad06239700 1 Processor -- start 2023-10-10T20:36:06.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.777+0000 7fad06239700 1 -- start start 2023-10-10T20:36:06.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.778+0000 7fad06239700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad00159420 0x7fad00154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:06.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.778+0000 7fad06239700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad001597f0 0x7fad000c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:06.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.778+0000 7fad06239700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad0015d230 0x7fad000c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:06.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.778+0000 7fad06239700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0015bf80 con 0x7fad001597f0 2023-10-10T20:36:06.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.778+0000 7fad06239700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad001695d0 con 0x7fad0015d230 2023-10-10T20:36:06.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.778+0000 7fad06239700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad001698b0 con 0x7fad00159420 2023-10-10T20:36:06.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.778+0000 7fad04a36700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad001597f0 0x7fad000c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:06.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.778+0000 7fad04a36700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad001597f0 0x7fad000c5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41230/0 (socket says 172.21.15.89:41230) 2023-10-10T20:36:06.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.778+0000 7fad04a36700 1 -- 172.21.15.89:0/3826073153 learned_addr learned my addr 172.21.15.89:0/3826073153 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:06.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.778+0000 7fad04a36700 1 -- 172.21.15.89:0/3826073153 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad00159420 msgr2=0x7fad00154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:06.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.778+0000 7fad04a36700 1 --2- 172.21.15.89:0/3826073153 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad00159420 0x7fad00154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:06.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.778+0000 7fad04a36700 1 -- 172.21.15.89:0/3826073153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad0015d230 msgr2=0x7fad000c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:06.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.778+0000 7fad04a36700 1 --2- 172.21.15.89:0/3826073153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad0015d230 0x7fad000c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:06.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.778+0000 7fad04a36700 1 -- 172.21.15.89:0/3826073153 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf4009420 con 0x7fad001597f0 2023-10-10T20:36:06.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.779+0000 7fad04a36700 1 --2- 172.21.15.89:0/3826073153 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad001597f0 0x7fad000c5dc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fad00089340 tx=0x7facf4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:06.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.779+0000 7facfd7fa700 1 -- 172.21.15.89:0/3826073153 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf400bb20 con 0x7fad001597f0 2023-10-10T20:36:06.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.779+0000 7facfd7fa700 1 -- 172.21.15.89:0/3826073153 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf4011420 con 0x7fad001597f0 2023-10-10T20:36:06.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.779+0000 7facfd7fa700 1 -- 172.21.15.89:0/3826073153 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf40117a0 con 0x7fad001597f0 2023-10-10T20:36:06.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.779+0000 7fad06239700 1 -- 172.21.15.89:0/3826073153 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad000c7220 con 0x7fad001597f0 2023-10-10T20:36:06.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.779+0000 7fad06239700 1 -- 172.21.15.89:0/3826073153 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad000c6980 con 0x7fad001597f0 2023-10-10T20:36:06.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.780+0000 7fad06239700 1 -- 172.21.15.89:0/3826073153 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0004c3c0 con 0x7fad001597f0 2023-10-10T20:36:06.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.780+0000 7facfd7fa700 1 -- 172.21.15.89:0/3826073153 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7facf4011b50 con 0x7fad001597f0 2023-10-10T20:36:06.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.780+0000 7facfd7fa700 1 --2- 172.21.15.89:0/3826073153 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7facec066f30 0x7facec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:06.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.780+0000 7facfd7fa700 1 -- 172.21.15.89:0/3826073153 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7facf4085cf0 con 0x7fad001597f0 2023-10-10T20:36:06.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.780+0000 7fad05237700 1 --2- 172.21.15.89:0/3826073153 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7facec066f30 0x7facec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:06.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.781+0000 7fad05237700 1 --2- 172.21.15.89:0/3826073153 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7facec066f30 0x7facec0693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7facf8001d90 tx=0x7facf8007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:06.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:06.785+0000 7facfd7fa700 1 -- 172.21.15.89:0/3826073153 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7facf408a070 con 0x7fad001597f0 2023-10-10T20:36:07.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.011+0000 7fad06239700 1 -- 172.21.15.89:0/3826073153 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad000b1d50 con 0x7fad001597f0 2023-10-10T20:36:07.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.011+0000 7facfd7fa700 1 -- 172.21.15.89:0/3826073153 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4597 (secure 0 0 0) 0x7facf4052f60 con 0x7fad001597f0 2023-10-10T20:36:07.015 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:07.015 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4922,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1464385125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1464385125},{"type":"v1","addr":"172.21.15.111:6834","nonce":1464385125}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":4925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/689169706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":689169706},{"type":"v1","addr":"172.21.15.111:6835","nonce":689169706}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/418769006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":418769006},{"type":"v1","addr":"172.21.15.89:6836","nonce":418769006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:35:55.314480+0000","modified":"2023-10-10T20:35:58.961430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5340":{"gid":5340,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/3245027879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":3245027879},{"type":"v1","addr":"172.21.15.89:6837","nonce":3245027879}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-10T20:36:07.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.014+0000 7fad06239700 1 -- 172.21.15.89:0/3826073153 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7facec066f30 msgr2=0x7facec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.014+0000 7fad06239700 1 --2- 172.21.15.89:0/3826073153 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7facec066f30 0x7facec0693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7facf8001d90 tx=0x7facf8007ec0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.014+0000 7fad06239700 1 -- 172.21.15.89:0/3826073153 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad001597f0 msgr2=0x7fad000c5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.014+0000 7fad06239700 1 --2- 172.21.15.89:0/3826073153 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad001597f0 0x7fad000c5dc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fad00089340 tx=0x7facf4017000 comp rx=0 tx=0).stop 2023-10-10T20:36:07.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.014+0000 7fad06239700 1 -- 172.21.15.89:0/3826073153 shutdown_connections 2023-10-10T20:36:07.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.014+0000 7fad06239700 1 --2- 172.21.15.89:0/3826073153 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7facec066f30 0x7facec0693e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.014+0000 7fad06239700 1 --2- 172.21.15.89:0/3826073153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad0015d230 0x7fad000c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.014+0000 7fad06239700 1 --2- 172.21.15.89:0/3826073153 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad00159420 0x7fad00154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.014+0000 7fad06239700 1 --2- 172.21.15.89:0/3826073153 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad001597f0 0x7fad000c5dc0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.014+0000 7fad06239700 1 -- 172.21.15.89:0/3826073153 >> 172.21.15.89:0/3826073153 conn(0x7fad000d0660 msgr2=0x7fad00155240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:07.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.014+0000 7fad06239700 1 -- 172.21.15.89:0/3826073153 shutdown_connections 2023-10-10T20:36:07.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.014+0000 7fad06239700 1 -- 172.21.15.89:0/3826073153 wait complete. 2023-10-10T20:36:07.020 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 30 2023-10-10T20:36:07.029 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:36:07.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.140+0000 7fdd3b436700 1 Processor -- start 2023-10-10T20:36:07.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.140+0000 7fdd3b436700 1 -- start start 2023-10-10T20:36:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.140+0000 7fdd3b436700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdd34159c60 0x7fdd3415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.141+0000 7fdd3b436700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdd3415db10 0x7fdd34069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.141+0000 7fdd3b436700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd3415a160 con 0x7fdd34159890 2023-10-10T20:36:07.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.141+0000 7fdd3b436700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd3415df50 con 0x7fdd34159c60 2023-10-10T20:36:07.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.141+0000 7fdd3b436700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd340d13a0 con 0x7fdd3415db10 2023-10-10T20:36:07.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.141+0000 7fdd389d1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdd34159c60 0x7fdd3415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:07.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.141+0000 7fdd389d1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdd34159c60 0x7fdd3415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41232/0 (socket says 172.21.15.89:41232) 2023-10-10T20:36:07.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.141+0000 7fdd389d1700 1 -- 172.21.15.89:0/901712364 learned_addr learned my addr 172.21.15.89:0/901712364 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:07.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.141+0000 7fdd399d3700 1 --2- 172.21.15.89:0/901712364 >> v2:172.21.15.111:3300/0 conn(0x7fdd3415db10 0x7fdd34069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:07.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.141+0000 7fdd389d1700 1 -- 172.21.15.89:0/901712364 >> v2:172.21.15.111:3300/0 conn(0x7fdd3415db10 msgr2=0x7fdd34069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.141+0000 7fdd389d1700 1 --2- 172.21.15.89:0/901712364 >> v2:172.21.15.111:3300/0 conn(0x7fdd3415db10 0x7fdd34069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.141+0000 7fdd389d1700 1 -- 172.21.15.89:0/901712364 >> v1:172.21.15.111:6789/0 conn(0x7fdd34159890 legacy=0x7fdd3415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.141+0000 7fdd389d1700 1 -- 172.21.15.89:0/901712364 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3415e140 con 0x7fdd34159c60 2023-10-10T20:36:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.141+0000 7fdd389d1700 1 --2- 172.21.15.89:0/901712364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd34159c60 0x7fdd3415d5d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdd3406a890 tx=0x7fdd24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cc3af2113386f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.141+0000 7fdd2bfff700 1 -- 172.21.15.89:0/901712364 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2400b820 con 0x7fdd34159c60 2023-10-10T20:36:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.141+0000 7fdd2bfff700 1 -- 172.21.15.89:0/901712364 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2400b980 con 0x7fdd34159c60 2023-10-10T20:36:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.141+0000 7fdd2bfff700 1 -- 172.21.15.89:0/901712364 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2400bd20 con 0x7fdd34159c60 2023-10-10T20:36:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.142+0000 7fdd3b436700 1 -- 172.21.15.89:0/901712364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd34159c60 msgr2=0x7fdd3415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.142+0000 7fdd3b436700 1 --2- 172.21.15.89:0/901712364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd34159c60 0x7fdd3415d5d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdd3406a890 tx=0x7fdd24009250 comp rx=0 tx=0).stop 2023-10-10T20:36:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.142+0000 7fdd3b436700 1 -- 172.21.15.89:0/901712364 shutdown_connections 2023-10-10T20:36:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.142+0000 7fdd3b436700 1 --2- 172.21.15.89:0/901712364 >> v2:172.21.15.111:3300/0 conn(0x7fdd3415db10 0x7fdd34069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.142+0000 7fdd3b436700 1 --2- 172.21.15.89:0/901712364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd34159c60 0x7fdd3415d5d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.142+0000 7fdd3b436700 1 -- 172.21.15.89:0/901712364 >> 172.21.15.89:0/901712364 conn(0x7fdd34068e20 msgr2=0x7fdd340d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:07.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.142+0000 7fdd3b436700 1 -- 172.21.15.89:0/901712364 shutdown_connections 2023-10-10T20:36:07.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.142+0000 7fdd3b436700 1 -- 172.21.15.89:0/901712364 wait complete. 2023-10-10T20:36:07.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.143+0000 7fdd3b436700 1 Processor -- start 2023-10-10T20:36:07.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.143+0000 7fdd3b436700 1 -- start start 2023-10-10T20:36:07.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.144+0000 7fdd3b436700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd34159890 0x7fdd340cf050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.144+0000 7fdd3b436700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd34159c60 0x7fdd3416e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.144+0000 7fdd3b436700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd3415db10 0x7fdd340cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.144+0000 7fdd3b436700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd340cfef0 con 0x7fdd3415db10 2023-10-10T20:36:07.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.144+0000 7fdd3b436700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd340cf930 con 0x7fdd34159890 2023-10-10T20:36:07.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.144+0000 7fdd3b436700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd340cfc10 con 0x7fdd34159c60 2023-10-10T20:36:07.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.144+0000 7fdd399d3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd3415db10 0x7fdd340cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:07.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.144+0000 7fdd399d3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd3415db10 0x7fdd340cb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41234/0 (socket says 172.21.15.89:41234) 2023-10-10T20:36:07.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.144+0000 7fdd399d3700 1 -- 172.21.15.89:0/130242982 learned_addr learned my addr 172.21.15.89:0/130242982 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:07.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.144+0000 7fdd391d2700 1 --2- 172.21.15.89:0/130242982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd34159890 0x7fdd340cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:07.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.144+0000 7fdd389d1700 1 --2- 172.21.15.89:0/130242982 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd34159c60 0x7fdd3416e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:07.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.144+0000 7fdd399d3700 1 -- 172.21.15.89:0/130242982 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd34159c60 msgr2=0x7fdd3416e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.144+0000 7fdd399d3700 1 --2- 172.21.15.89:0/130242982 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd34159c60 0x7fdd3416e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.144+0000 7fdd399d3700 1 -- 172.21.15.89:0/130242982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd34159890 msgr2=0x7fdd340cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.144+0000 7fdd399d3700 1 --2- 172.21.15.89:0/130242982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd34159890 0x7fdd340cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.144+0000 7fdd399d3700 1 -- 172.21.15.89:0/130242982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd240094f0 con 0x7fdd3415db10 2023-10-10T20:36:07.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.145+0000 7fdd399d3700 1 --2- 172.21.15.89:0/130242982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd3415db10 0x7fdd340cb0f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00c880 tx=0x7fdd2c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:07.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.145+0000 7fdd297fa700 1 -- 172.21.15.89:0/130242982 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c011600 con 0x7fdd3415db10 2023-10-10T20:36:07.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.145+0000 7fdd297fa700 1 -- 172.21.15.89:0/130242982 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2c011760 con 0x7fdd3415db10 2023-10-10T20:36:07.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.145+0000 7fdd3b436700 1 -- 172.21.15.89:0/130242982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd341718c0 con 0x7fdd3415db10 2023-10-10T20:36:07.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.145+0000 7fdd297fa700 1 -- 172.21.15.89:0/130242982 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c011b20 con 0x7fdd3415db10 2023-10-10T20:36:07.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.145+0000 7fdd3b436700 1 -- 172.21.15.89:0/130242982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd340cf590 con 0x7fdd3415db10 2023-10-10T20:36:07.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.146+0000 7fdd3b436700 1 -- 172.21.15.89:0/130242982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd34159120 con 0x7fdd3415db10 2023-10-10T20:36:07.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.146+0000 7fdd297fa700 1 -- 172.21.15.89:0/130242982 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd2c018360 con 0x7fdd3415db10 2023-10-10T20:36:07.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.146+0000 7fdd297fa700 1 --2- 172.21.15.89:0/130242982 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdd18066f30 0x7fdd180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.146+0000 7fdd297fa700 1 -- 172.21.15.89:0/130242982 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdd2c084120 con 0x7fdd3415db10 2023-10-10T20:36:07.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.146+0000 7fdd391d2700 1 --2- 172.21.15.89:0/130242982 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdd18066f30 0x7fdd180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:07.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.147+0000 7fdd391d2700 1 --2- 172.21.15.89:0/130242982 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdd18066f30 0x7fdd180693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdd300075c0 tx=0x7fdd30009780 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:07.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.151+0000 7fdd297fa700 1 -- 172.21.15.89:0/130242982 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd2c051390 con 0x7fdd3415db10 2023-10-10T20:36:07.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.335+0000 7fdd3b436700 1 -- 172.21.15.89:0/130242982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd34171de0 con 0x7fdd3415db10 2023-10-10T20:36:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.336+0000 7fdd297fa700 1 -- 172.21.15.89:0/130242982 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fdd2c0549b0 con 0x7fdd3415db10 2023-10-10T20:36:07.340 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:07.340 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":50,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:35:56.929187+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:35:53.658728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-10T20:35:54.272054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:36:07.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.338+0000 7fdd3b436700 1 -- 172.21.15.89:0/130242982 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdd18066f30 msgr2=0x7fdd180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.338+0000 7fdd3b436700 1 --2- 172.21.15.89:0/130242982 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdd18066f30 0x7fdd180693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdd300075c0 tx=0x7fdd30009780 comp rx=0 tx=0).stop 2023-10-10T20:36:07.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.338+0000 7fdd3b436700 1 -- 172.21.15.89:0/130242982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd3415db10 msgr2=0x7fdd340cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.338+0000 7fdd3b436700 1 --2- 172.21.15.89:0/130242982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd3415db10 0x7fdd340cb0f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00c880 tx=0x7fdd2c008040 comp rx=0 tx=0).stop 2023-10-10T20:36:07.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.338+0000 7fdd3b436700 1 -- 172.21.15.89:0/130242982 shutdown_connections 2023-10-10T20:36:07.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.338+0000 7fdd3b436700 1 --2- 172.21.15.89:0/130242982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd3415db10 0x7fdd340cb0f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.338+0000 7fdd3b436700 1 --2- 172.21.15.89:0/130242982 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdd18066f30 0x7fdd180693e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.338+0000 7fdd3b436700 1 --2- 172.21.15.89:0/130242982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd34159890 0x7fdd340cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.338+0000 7fdd3b436700 1 --2- 172.21.15.89:0/130242982 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd34159c60 0x7fdd3416e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.338+0000 7fdd3b436700 1 -- 172.21.15.89:0/130242982 >> 172.21.15.89:0/130242982 conn(0x7fdd34068e20 msgr2=0x7fdd340af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:07.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.338+0000 7fdd3b436700 1 -- 172.21.15.89:0/130242982 shutdown_connections 2023-10-10T20:36:07.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.338+0000 7fdd3b436700 1 -- 172.21.15.89:0/130242982 wait complete. 2023-10-10T20:36:07.354 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:35:53.658728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:35:54.272054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:36:07.356 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:36:07.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.470+0000 7f9ae9f1b700 1 Processor -- start 2023-10-10T20:36:07.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.470+0000 7f9ae9f1b700 1 -- start start 2023-10-10T20:36:07.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.471+0000 7f9ae9f1b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ae4159890 0x7f9ae415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.471+0000 7f9ae9f1b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9ae4069560 0x7f9ae40699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.471+0000 7f9ae9f1b700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae415a160 con 0x7f9ae4159c60 2023-10-10T20:36:07.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.471+0000 7f9ae9f1b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae4069f20 con 0x7f9ae4069560 2023-10-10T20:36:07.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.471+0000 7f9ae9f1b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40d13a0 con 0x7f9ae4159890 2023-10-10T20:36:07.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.471+0000 7f9ae3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9ae4069560 0x7f9ae40699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:07.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.471+0000 7f9ae3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9ae4069560 0x7f9ae40699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41250/0 (socket says 172.21.15.89:41250) 2023-10-10T20:36:07.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.471+0000 7f9ae3fff700 1 -- 172.21.15.89:0/860904712 learned_addr learned my addr 172.21.15.89:0/860904712 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:07.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.471+0000 7f9ae37fe700 1 --2- 172.21.15.89:0/860904712 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ae4159890 0x7f9ae415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:07.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.471+0000 7f9ae3fff700 1 -- 172.21.15.89:0/860904712 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ae4159890 msgr2=0x7f9ae415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:07.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.471+0000 7f9ae3fff700 1 --2- 172.21.15.89:0/860904712 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ae4159890 0x7f9ae415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.471+0000 7f9ae3fff700 1 -- 172.21.15.89:0/860904712 >> v1:172.21.15.111:6789/0 conn(0x7f9ae4159c60 legacy=0x7f9ae415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.471+0000 7f9ae3fff700 1 -- 172.21.15.89:0/860904712 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae415e140 con 0x7f9ae4069560 2023-10-10T20:36:07.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.471+0000 7f9ae3fff700 1 --2- 172.21.15.89:0/860904712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae4069560 0x7f9ae40699e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0099b0 tx=0x7f9adc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff342aeb77449cb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:07.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.471+0000 7f9ae27fc700 1 -- 172.21.15.89:0/860904712 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc00b860 con 0x7f9ae4069560 2023-10-10T20:36:07.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.472+0000 7f9ae27fc700 1 -- 172.21.15.89:0/860904712 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9adc00b9c0 con 0x7f9ae4069560 2023-10-10T20:36:07.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.472+0000 7f9ae27fc700 1 -- 172.21.15.89:0/860904712 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc00bd60 con 0x7f9ae4069560 2023-10-10T20:36:07.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.472+0000 7f9ae9f1b700 1 -- 172.21.15.89:0/860904712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae4069560 msgr2=0x7f9ae40699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.472+0000 7f9ae9f1b700 1 --2- 172.21.15.89:0/860904712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae4069560 0x7f9ae40699e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0099b0 tx=0x7f9adc009250 comp rx=0 tx=0).stop 2023-10-10T20:36:07.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.472+0000 7f9ae9f1b700 1 -- 172.21.15.89:0/860904712 shutdown_connections 2023-10-10T20:36:07.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.472+0000 7f9ae9f1b700 1 --2- 172.21.15.89:0/860904712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae4069560 0x7f9ae40699e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.472+0000 7f9ae9f1b700 1 --2- 172.21.15.89:0/860904712 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ae4159890 0x7f9ae415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.472+0000 7f9ae9f1b700 1 -- 172.21.15.89:0/860904712 >> 172.21.15.89:0/860904712 conn(0x7f9ae4068e20 msgr2=0x7f9ae40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:07.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.472+0000 7f9ae9f1b700 1 -- 172.21.15.89:0/860904712 shutdown_connections 2023-10-10T20:36:07.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.472+0000 7f9ae9f1b700 1 -- 172.21.15.89:0/860904712 wait complete. 2023-10-10T20:36:07.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.473+0000 7f9ae9f1b700 1 Processor -- start 2023-10-10T20:36:07.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.473+0000 7f9ae9f1b700 1 -- start start 2023-10-10T20:36:07.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.474+0000 7f9ae9f1b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae4069560 0x7f9ae4146370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.474+0000 7f9ae9f1b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ae4159890 0x7f9ae40cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.474+0000 7f9ae9f1b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ae4159c60 0x7f9ae40b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.474+0000 7f9ae9f1b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae4155a00 con 0x7f9ae4069560 2023-10-10T20:36:07.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.474+0000 7f9ae9f1b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40cd470 con 0x7f9ae4159c60 2023-10-10T20:36:07.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.474+0000 7f9ae9f1b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40cd750 con 0x7f9ae4159890 2023-10-10T20:36:07.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.474+0000 7f9ae37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae4069560 0x7f9ae4146370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:07.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.474+0000 7f9ae37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae4069560 0x7f9ae4146370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41262/0 (socket says 172.21.15.89:41262) 2023-10-10T20:36:07.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.474+0000 7f9ae37fe700 1 -- 172.21.15.89:0/1672399380 learned_addr learned my addr 172.21.15.89:0/1672399380 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:07.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.474+0000 7f9ae37fe700 1 -- 172.21.15.89:0/1672399380 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ae4159890 msgr2=0x7f9ae40cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.474+0000 7f9ae37fe700 1 --2- 172.21.15.89:0/1672399380 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ae4159890 0x7f9ae40cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.474+0000 7f9ae37fe700 1 -- 172.21.15.89:0/1672399380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ae4159c60 msgr2=0x7f9ae40b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.474+0000 7f9ae37fe700 1 --2- 172.21.15.89:0/1672399380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ae4159c60 0x7f9ae40b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.474+0000 7f9ae37fe700 1 -- 172.21.15.89:0/1672399380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9adc0094f0 con 0x7f9ae4069560 2023-10-10T20:36:07.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.475+0000 7f9ae37fe700 1 --2- 172.21.15.89:0/1672399380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae4069560 0x7f9ae4146370 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400e190 tx=0x7f9ad4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:07.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.475+0000 7f9acb7fe700 1 -- 172.21.15.89:0/1672399380 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad400ee30 con 0x7f9ae4069560 2023-10-10T20:36:07.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.475+0000 7f9acb7fe700 1 -- 172.21.15.89:0/1672399380 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad4014420 con 0x7f9ae4069560 2023-10-10T20:36:07.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.475+0000 7f9ae9f1b700 1 -- 172.21.15.89:0/1672399380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae40cda30 con 0x7f9ae4069560 2023-10-10T20:36:07.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.475+0000 7f9acb7fe700 1 -- 172.21.15.89:0/1672399380 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40147e0 con 0x7f9ae4069560 2023-10-10T20:36:07.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.475+0000 7f9ae9f1b700 1 -- 172.21.15.89:0/1672399380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae40cdff0 con 0x7f9ae4069560 2023-10-10T20:36:07.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.476+0000 7f9ae9f1b700 1 -- 172.21.15.89:0/1672399380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae4064d80 con 0x7f9ae4069560 2023-10-10T20:36:07.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.476+0000 7f9acb7fe700 1 -- 172.21.15.89:0/1672399380 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ad4014b90 con 0x7f9ae4069560 2023-10-10T20:36:07.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.476+0000 7f9acb7fe700 1 --2- 172.21.15.89:0/1672399380 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ad0066f30 0x7f9ad00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.476+0000 7f9acb7fe700 1 -- 172.21.15.89:0/1672399380 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9ad4084d20 con 0x7f9ae4069560 2023-10-10T20:36:07.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.476+0000 7f9ae2ffd700 1 --2- 172.21.15.89:0/1672399380 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ad0066f30 0x7f9ad00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:07.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.477+0000 7f9ae2ffd700 1 --2- 172.21.15.89:0/1672399380 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ad0066f30 0x7f9ad00693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40b3c40 tx=0x7f9ad8007b40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:07.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.481+0000 7f9acb7fe700 1 -- 172.21.15.89:0/1672399380 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ad4051f90 con 0x7f9ae4069560 2023-10-10T20:36:07.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.663+0000 7f9ae9f1b700 1 -- 172.21.15.89:0/1672399380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ae404c3c0 con 0x7f9ae4069560 2023-10-10T20:36:07.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.663+0000 7f9acb7fe700 1 -- 172.21.15.89:0/1672399380 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ad4009a60 con 0x7f9ae4069560 2023-10-10T20:36:07.667 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:36:07.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.665+0000 7f9ae9f1b700 1 -- 172.21.15.89:0/1672399380 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ad0066f30 msgr2=0x7f9ad00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.665+0000 7f9ae9f1b700 1 --2- 172.21.15.89:0/1672399380 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ad0066f30 0x7f9ad00693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40b3c40 tx=0x7f9ad8007b40 comp rx=0 tx=0).stop 2023-10-10T20:36:07.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.665+0000 7f9ae9f1b700 1 -- 172.21.15.89:0/1672399380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae4069560 msgr2=0x7f9ae4146370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.665+0000 7f9ae9f1b700 1 --2- 172.21.15.89:0/1672399380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae4069560 0x7f9ae4146370 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400e190 tx=0x7f9ad4009920 comp rx=0 tx=0).stop 2023-10-10T20:36:07.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.665+0000 7f9ae9f1b700 1 -- 172.21.15.89:0/1672399380 shutdown_connections 2023-10-10T20:36:07.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.665+0000 7f9ae9f1b700 1 --2- 172.21.15.89:0/1672399380 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ad0066f30 0x7f9ad00693e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.665+0000 7f9ae9f1b700 1 --2- 172.21.15.89:0/1672399380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ae4159c60 0x7f9ae40b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.665+0000 7f9ae9f1b700 1 --2- 172.21.15.89:0/1672399380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae4069560 0x7f9ae4146370 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.665+0000 7f9ae9f1b700 1 --2- 172.21.15.89:0/1672399380 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ae4159890 0x7f9ae40cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.665+0000 7f9ae9f1b700 1 -- 172.21.15.89:0/1672399380 >> 172.21.15.89:0/1672399380 conn(0x7f9ae4068e20 msgr2=0x7f9ae40c6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:07.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.665+0000 7f9ae9f1b700 1 -- 172.21.15.89:0/1672399380 shutdown_connections 2023-10-10T20:36:07.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.665+0000 7f9ae9f1b700 1 -- 172.21.15.89:0/1672399380 wait complete. 2023-10-10T20:36:07.682 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:36:07.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.796+0000 7fad62318700 1 Processor -- start 2023-10-10T20:36:07.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.796+0000 7fad62318700 1 -- start start 2023-10-10T20:36:07.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.797+0000 7fad62318700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fad5c0c83c0 0x7fad5c164690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.797+0000 7fad62318700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5c06bdb0 con 0x7fad5c0aca80 2023-10-10T20:36:07.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.797+0000 7fad62318700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5c0cb370 con 0x7fad5c069d60 2023-10-10T20:36:07.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.797+0000 7fad62318700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5c159f60 con 0x7fad5c0c83c0 2023-10-10T20:36:07.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.797+0000 7fad5bfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fad5c0aca80 0x7fad5c06aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58886/0 (socket says 172.21.15.89:58886) 2023-10-10T20:36:07.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.797+0000 7fad5bfff700 1 -- 172.21.15.89:0/654272105 learned_addr learned my addr 172.21.15.89:0/654272105 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:07.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.797+0000 7fad608b5700 1 --2- 172.21.15.89:0/654272105 >> v2:172.21.15.111:3300/0 conn(0x7fad5c0c83c0 0x7fad5c164690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:07.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.797+0000 7fad5affd700 1 -- 172.21.15.89:0/654272105 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 711790944 0 0) 0x7fad5c06bdb0 con 0x7fad5c0aca80 2023-10-10T20:36:07.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad5affd700 1 -- 172.21.15.89:0/654272105 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad44003570 con 0x7fad5c0aca80 2023-10-10T20:36:07.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad5affd700 1 -- 172.21.15.89:0/654272105 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3400224092 0 0) 0x7fad44003570 con 0x7fad5c0aca80 2023-10-10T20:36:07.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad5affd700 1 -- 172.21.15.89:0/654272105 >> v2:172.21.15.111:3300/0 conn(0x7fad5c0c83c0 msgr2=0x7fad5c164690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:07.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad5affd700 1 --2- 172.21.15.89:0/654272105 >> v2:172.21.15.111:3300/0 conn(0x7fad5c0c83c0 0x7fad5c164690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad5affd700 1 -- 172.21.15.89:0/654272105 >> v1:172.21.15.111:6789/0 conn(0x7fad5c069d60 legacy=0x7fad5c0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad5affd700 1 -- 172.21.15.89:0/654272105 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c06b2b0 con 0x7fad5c0aca80 2023-10-10T20:36:07.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad5affd700 1 -- 172.21.15.89:0/654272105 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fad4c0015f0 con 0x7fad5c0aca80 2023-10-10T20:36:07.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad5affd700 1 -- 172.21.15.89:0/654272105 >> v1:172.21.15.89:6789/0 conn(0x7fad5c0aca80 legacy=0x7fad5c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad5affd700 1 --2- 172.21.15.89:0/654272105 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad44003a60 0x7fad44003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad5affd700 1 --2- 172.21.15.89:0/654272105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad44004810 0x7fad44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad5affd700 1 --2- 172.21.15.89:0/654272105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad440051c0 0x7fad4400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad5affd700 1 -- 172.21.15.89:0/654272105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c159f60 con 0x7fad440051c0 2023-10-10T20:36:07.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad5affd700 1 -- 172.21.15.89:0/654272105 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4400c050 con 0x7fad44003a60 2023-10-10T20:36:07.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad5affd700 1 -- 172.21.15.89:0/654272105 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4400c210 con 0x7fad44004810 2023-10-10T20:36:07.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad608b5700 1 --2- 172.21.15.89:0/654272105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad440051c0 0x7fad4400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:07.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad5bfff700 1 --2- 172.21.15.89:0/654272105 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad44003a60 0x7fad44003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:07.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad5b7fe700 1 --2- 172.21.15.89:0/654272105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad44004810 0x7fad44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:07.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad608b5700 1 -- 172.21.15.89:0/654272105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad44004810 msgr2=0x7fad44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:07.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad608b5700 1 --2- 172.21.15.89:0/654272105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad44004810 0x7fad44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad608b5700 1 -- 172.21.15.89:0/654272105 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad44003a60 msgr2=0x7fad44003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:07.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad608b5700 1 --2- 172.21.15.89:0/654272105 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad44003a60 0x7fad44003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad608b5700 1 -- 172.21.15.89:0/654272105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad44003570 con 0x7fad440051c0 2023-10-10T20:36:07.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad5b7fe700 1 --2- 172.21.15.89:0/654272105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad44004810 0x7fad44004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:07.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad608b5700 1 -- 172.21.15.89:0/654272105 reap_dead start 2023-10-10T20:36:07.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad608b5700 1 --2- 172.21.15.89:0/654272105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad440051c0 0x7fad4400ba60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0c87c0 tx=0x7fad5400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1977434f63eb7f41 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:07.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.798+0000 7fad5affd700 1 -- 172.21.15.89:0/654272105 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5400ea90 con 0x7fad440051c0 2023-10-10T20:36:07.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.799+0000 7fad5affd700 1 -- 172.21.15.89:0/654272105 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5400f040 con 0x7fad440051c0 2023-10-10T20:36:07.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.799+0000 7fad5affd700 1 -- 172.21.15.89:0/654272105 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5400ee40 con 0x7fad440051c0 2023-10-10T20:36:07.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.799+0000 7fad62318700 1 -- 172.21.15.89:0/654272105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad440051c0 msgr2=0x7fad4400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.799+0000 7fad62318700 1 --2- 172.21.15.89:0/654272105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad440051c0 0x7fad4400ba60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0c87c0 tx=0x7fad5400b040 comp rx=0 tx=0).stop 2023-10-10T20:36:07.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.799+0000 7fad62318700 1 -- 172.21.15.89:0/654272105 shutdown_connections 2023-10-10T20:36:07.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.799+0000 7fad62318700 1 --2- 172.21.15.89:0/654272105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad440051c0 0x7fad4400ba60 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.799+0000 7fad62318700 1 -- 172.21.15.89:0/654272105 >> 172.21.15.89:0/654272105 conn(0x7fad5c0cd670 msgr2=0x7fad5c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:07.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.799+0000 7fad62318700 1 -- 172.21.15.89:0/654272105 shutdown_connections 2023-10-10T20:36:07.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.799+0000 7fad62318700 1 -- 172.21.15.89:0/654272105 wait complete. 2023-10-10T20:36:07.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.800+0000 7fad62318700 1 Processor -- start 2023-10-10T20:36:07.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.800+0000 7fad62318700 1 -- start start 2023-10-10T20:36:07.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.801+0000 7fad62318700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad440051c0 0x7fad5c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.801+0000 7fad62318700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad5c0af770 0x7fad5c16c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.801+0000 7fad62318700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5c16cae0 0x7fad5c15dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.801+0000 7fad62318700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c0b3360 con 0x7fad5c16cae0 2023-10-10T20:36:07.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.801+0000 7fad62318700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c0b2750 con 0x7fad5c0af770 2023-10-10T20:36:07.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.801+0000 7fad62318700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c0b2a30 con 0x7fad440051c0 2023-10-10T20:36:07.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.801+0000 7fad608b5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5c16cae0 0x7fad5c15dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:07.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.801+0000 7fad608b5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5c16cae0 0x7fad5c15dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41288/0 (socket says 172.21.15.89:41288) 2023-10-10T20:36:07.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.801+0000 7fad608b5700 1 -- 172.21.15.89:0/2950427202 learned_addr learned my addr 172.21.15.89:0/2950427202 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:07.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.801+0000 7fad5b7fe700 1 --2- 172.21.15.89:0/2950427202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad5c0af770 0x7fad5c16c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:07.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.801+0000 7fad5bfff700 1 --2- 172.21.15.89:0/2950427202 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad440051c0 0x7fad5c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:07.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.801+0000 7fad608b5700 1 -- 172.21.15.89:0/2950427202 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad440051c0 msgr2=0x7fad5c0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.801+0000 7fad608b5700 1 --2- 172.21.15.89:0/2950427202 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad440051c0 0x7fad5c0b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.801+0000 7fad608b5700 1 -- 172.21.15.89:0/2950427202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad5c0af770 msgr2=0x7fad5c16c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.801+0000 7fad608b5700 1 --2- 172.21.15.89:0/2950427202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad5c0af770 0x7fad5c16c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:07.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.801+0000 7fad608b5700 1 -- 172.21.15.89:0/2950427202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c0b2d10 con 0x7fad5c16cae0 2023-10-10T20:36:07.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.802+0000 7fad608b5700 1 --2- 172.21.15.89:0/2950427202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5c16cae0 0x7fad5c15dd50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fad5c15d5b0 tx=0x7fad54019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:07.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.802+0000 7fad43fff700 1 -- 172.21.15.89:0/2950427202 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5401d650 con 0x7fad5c16cae0 2023-10-10T20:36:07.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.802+0000 7fad43fff700 1 -- 172.21.15.89:0/2950427202 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5401d7b0 con 0x7fad5c16cae0 2023-10-10T20:36:07.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.802+0000 7fad62318700 1 -- 172.21.15.89:0/2950427202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5c0b2ff0 con 0x7fad5c16cae0 2023-10-10T20:36:07.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.802+0000 7fad43fff700 1 -- 172.21.15.89:0/2950427202 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5401db70 con 0x7fad5c16cae0 2023-10-10T20:36:07.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.802+0000 7fad62318700 1 -- 172.21.15.89:0/2950427202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5c15e290 con 0x7fad5c16cae0 2023-10-10T20:36:07.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.803+0000 7fad62318700 1 -- 172.21.15.89:0/2950427202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5c15a150 con 0x7fad5c16cae0 2023-10-10T20:36:07.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.803+0000 7fad43fff700 1 -- 172.21.15.89:0/2950427202 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fad54003020 con 0x7fad5c16cae0 2023-10-10T20:36:07.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.803+0000 7fad43fff700 1 --2- 172.21.15.89:0/2950427202 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fad48066f30 0x7fad480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:07.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.803+0000 7fad43fff700 1 -- 172.21.15.89:0/2950427202 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fad54086730 con 0x7fad5c16cae0 2023-10-10T20:36:07.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.804+0000 7fad5bfff700 1 --2- 172.21.15.89:0/2950427202 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fad48066f30 0x7fad480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:07.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.804+0000 7fad5bfff700 1 --2- 172.21.15.89:0/2950427202 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fad48066f30 0x7fad480693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fad4c000ef0 tx=0x7fad4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:07.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.808+0000 7fad43fff700 1 -- 172.21.15.89:0/2950427202 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad540539a0 con 0x7fad5c16cae0 2023-10-10T20:36:07.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.993+0000 7fad62318700 1 -- 172.21.15.89:0/2950427202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fad5c04c3c0 con 0x7fad5c16cae0 2023-10-10T20:36:07.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.993+0000 7fad43fff700 1 -- 172.21.15.89:0/2950427202 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7fad54056fc0 con 0x7fad5c16cae0 2023-10-10T20:36:07.996 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:36:07.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.995+0000 7fad62318700 1 -- 172.21.15.89:0/2950427202 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fad48066f30 msgr2=0x7fad480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.995+0000 7fad62318700 1 --2- 172.21.15.89:0/2950427202 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fad48066f30 0x7fad480693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fad4c000ef0 tx=0x7fad4c009040 comp rx=0 tx=0).stop 2023-10-10T20:36:07.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.995+0000 7fad62318700 1 -- 172.21.15.89:0/2950427202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5c16cae0 msgr2=0x7fad5c15dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:07.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.995+0000 7fad62318700 1 --2- 172.21.15.89:0/2950427202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5c16cae0 0x7fad5c15dd50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fad5c15d5b0 tx=0x7fad54019000 comp rx=0 tx=0).stop 2023-10-10T20:36:08.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.996+0000 7fad62318700 1 -- 172.21.15.89:0/2950427202 shutdown_connections 2023-10-10T20:36:08.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.996+0000 7fad62318700 1 --2- 172.21.15.89:0/2950427202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5c16cae0 0x7fad5c15dd50 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.996+0000 7fad62318700 1 --2- 172.21.15.89:0/2950427202 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad440051c0 0x7fad5c0b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.996+0000 7fad62318700 1 --2- 172.21.15.89:0/2950427202 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fad48066f30 0x7fad480693e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.996+0000 7fad62318700 1 --2- 172.21.15.89:0/2950427202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad5c0af770 0x7fad5c16c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.996+0000 7fad62318700 1 -- 172.21.15.89:0/2950427202 >> 172.21.15.89:0/2950427202 conn(0x7fad5c0cd670 msgr2=0x7fad5c1586b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:08.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.996+0000 7fad62318700 1 -- 172.21.15.89:0/2950427202 shutdown_connections 2023-10-10T20:36:08.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:07.996+0000 7fad62318700 1 -- 172.21.15.89:0/2950427202 wait complete. 2023-10-10T20:36:08.013 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:36:08.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.128+0000 7f236555e700 1 Processor -- start 2023-10-10T20:36:08.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.128+0000 7f236555e700 1 -- start start 2023-10-10T20:36:08.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f236555e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2360069560 0x7f23600699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f236555e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2360069ee0 con 0x7f2360159890 2023-10-10T20:36:08.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f236555e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236015a160 con 0x7f2360159c60 2023-10-10T20:36:08.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f236555e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23600d13a0 con 0x7f2360069560 2023-10-10T20:36:08.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235effd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2360159890 0x7f236015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58902/0 (socket says 172.21.15.89:58902) 2023-10-10T20:36:08.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235effd700 1 -- 172.21.15.89:0/626117785 learned_addr learned my addr 172.21.15.89:0/626117785 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:08.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235f7fe700 1 --2- 172.21.15.89:0/626117785 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2360069560 0x7f23600699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235e7fc700 1 -- 172.21.15.89:0/626117785 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725922932 0 0) 0x7f2360069ee0 con 0x7f2360159890 2023-10-10T20:36:08.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235e7fc700 1 -- 172.21.15.89:0/626117785 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f234c003570 con 0x7f2360159890 2023-10-10T20:36:08.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235e7fc700 1 -- 172.21.15.89:0/626117785 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4239978443 0 0) 0x7f236015a160 con 0x7f2360159c60 2023-10-10T20:36:08.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235e7fc700 1 -- 172.21.15.89:0/626117785 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2360069ee0 con 0x7f2360159c60 2023-10-10T20:36:08.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235e7fc700 1 -- 172.21.15.89:0/626117785 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1743766301 0 0) 0x7f234c003570 con 0x7f2360159890 2023-10-10T20:36:08.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235e7fc700 1 -- 172.21.15.89:0/626117785 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2360069560 msgr2=0x7f23600699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:08.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235e7fc700 1 --2- 172.21.15.89:0/626117785 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2360069560 0x7f23600699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235e7fc700 1 -- 172.21.15.89:0/626117785 >> v1:172.21.15.111:6789/0 conn(0x7f2360159c60 legacy=0x7f236015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:08.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235e7fc700 1 -- 172.21.15.89:0/626117785 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236015e140 con 0x7f2360159890 2023-10-10T20:36:08.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235e7fc700 1 -- 172.21.15.89:0/626117785 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f2350001470 con 0x7f2360159890 2023-10-10T20:36:08.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235e7fc700 1 -- 172.21.15.89:0/626117785 >> v1:172.21.15.89:6789/0 conn(0x7f2360159890 legacy=0x7f236015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:08.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235e7fc700 1 --2- 172.21.15.89:0/626117785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f234c003c20 0x7f234c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235e7fc700 1 --2- 172.21.15.89:0/626117785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f234c004810 0x7f234c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235e7fc700 1 --2- 172.21.15.89:0/626117785 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2360069560 0x7f234c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235e7fc700 1 -- 172.21.15.89:0/626117785 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23600d13a0 con 0x7f234c003c20 2023-10-10T20:36:08.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235e7fc700 1 -- 172.21.15.89:0/626117785 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c00bb70 con 0x7f234c004810 2023-10-10T20:36:08.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235e7fc700 1 -- 172.21.15.89:0/626117785 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f234c00bd30 con 0x7f2360069560 2023-10-10T20:36:08.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.129+0000 7f235effd700 1 --2- 172.21.15.89:0/626117785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f234c003c20 0x7f234c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.130+0000 7f2357fff700 1 --2- 172.21.15.89:0/626117785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f234c004810 0x7f234c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.130+0000 7f235f7fe700 1 --2- 172.21.15.89:0/626117785 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2360069560 0x7f234c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.130+0000 7f235effd700 1 -- 172.21.15.89:0/626117785 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2360069560 msgr2=0x7f234c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:08.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.130+0000 7f235effd700 1 --2- 172.21.15.89:0/626117785 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2360069560 0x7f234c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.130+0000 7f235effd700 1 -- 172.21.15.89:0/626117785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f234c004810 msgr2=0x7f234c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:08.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.130+0000 7f235effd700 1 --2- 172.21.15.89:0/626117785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f234c004810 0x7f234c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.130+0000 7f235effd700 1 -- 172.21.15.89:0/626117785 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234c003570 con 0x7f234c003c20 2023-10-10T20:36:08.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.130+0000 7f235effd700 1 --2- 172.21.15.89:0/626117785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f234c003c20 0x7f234c004010 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f235000c2d0 tx=0x7f2350001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844896ee6239be07 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:08.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.130+0000 7f235e7fc700 1 -- 172.21.15.89:0/626117785 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235000cba0 con 0x7f234c003c20 2023-10-10T20:36:08.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.130+0000 7f235e7fc700 1 -- 172.21.15.89:0/626117785 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235000cd00 con 0x7f234c003c20 2023-10-10T20:36:08.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.130+0000 7f235e7fc700 1 -- 172.21.15.89:0/626117785 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350005b70 con 0x7f234c003c20 2023-10-10T20:36:08.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.131+0000 7f236555e700 1 -- 172.21.15.89:0/626117785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f234c003c20 msgr2=0x7f234c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:08.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.131+0000 7f236555e700 1 --2- 172.21.15.89:0/626117785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f234c003c20 0x7f234c004010 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f235000c2d0 tx=0x7f2350001e10 comp rx=0 tx=0).stop 2023-10-10T20:36:08.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.131+0000 7f235f7fe700 1 -- 172.21.15.89:0/626117785 reap_dead start 2023-10-10T20:36:08.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.131+0000 7f236555e700 1 -- 172.21.15.89:0/626117785 shutdown_connections 2023-10-10T20:36:08.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.131+0000 7f236555e700 1 -- 172.21.15.89:0/626117785 >> 172.21.15.89:0/626117785 conn(0x7f2360068e20 msgr2=0x7f23600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:08.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.131+0000 7f236555e700 1 -- 172.21.15.89:0/626117785 shutdown_connections 2023-10-10T20:36:08.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.131+0000 7f236555e700 1 -- 172.21.15.89:0/626117785 wait complete. 2023-10-10T20:36:08.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.132+0000 7f236555e700 1 Processor -- start 2023-10-10T20:36:08.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.132+0000 7f236555e700 1 -- start start 2023-10-10T20:36:08.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.132+0000 7f236555e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f234c003c20 0x7f236014b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.132+0000 7f236555e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23600b5eb0 0x7f23601521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.132+0000 7f236555e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2360152720 0x7f23600cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.132+0000 7f236555e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2360152b90 con 0x7f23600b5eb0 2023-10-10T20:36:08.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.132+0000 7f236555e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23600b4940 con 0x7f2360152720 2023-10-10T20:36:08.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.132+0000 7f236555e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236014bf30 con 0x7f234c003c20 2023-10-10T20:36:08.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.132+0000 7f2357fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23600b5eb0 0x7f23601521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.133+0000 7f2357fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23600b5eb0 0x7f23601521e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41304/0 (socket says 172.21.15.89:41304) 2023-10-10T20:36:08.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.133+0000 7f2357fff700 1 -- 172.21.15.89:0/2862350278 learned_addr learned my addr 172.21.15.89:0/2862350278 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:08.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.133+0000 7f235f7fe700 1 --2- 172.21.15.89:0/2862350278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2360152720 0x7f23600cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.133+0000 7f235effd700 1 --2- 172.21.15.89:0/2862350278 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f234c003c20 0x7f236014b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.133+0000 7f2357fff700 1 -- 172.21.15.89:0/2862350278 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f234c003c20 msgr2=0x7f236014b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:08.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.133+0000 7f2357fff700 1 --2- 172.21.15.89:0/2862350278 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f234c003c20 0x7f236014b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.133+0000 7f2357fff700 1 -- 172.21.15.89:0/2862350278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2360152720 msgr2=0x7f23600cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:08.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.133+0000 7f2357fff700 1 --2- 172.21.15.89:0/2862350278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2360152720 0x7f23600cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.133+0000 7f2357fff700 1 -- 172.21.15.89:0/2862350278 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23600b4380 con 0x7f23600b5eb0 2023-10-10T20:36:08.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.133+0000 7f2357fff700 1 --2- 172.21.15.89:0/2862350278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23600b5eb0 0x7f23601521e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2348002ee0 tx=0x7f2348007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:08.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.133+0000 7f2356ffd700 1 -- 172.21.15.89:0/2862350278 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348002260 con 0x7f23600b5eb0 2023-10-10T20:36:08.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.133+0000 7f2356ffd700 1 -- 172.21.15.89:0/2862350278 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23480023c0 con 0x7f23600b5eb0 2023-10-10T20:36:08.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.133+0000 7f236555e700 1 -- 172.21.15.89:0/2862350278 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23600b4660 con 0x7f23600b5eb0 2023-10-10T20:36:08.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.133+0000 7f2356ffd700 1 -- 172.21.15.89:0/2862350278 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348001660 con 0x7f23600b5eb0 2023-10-10T20:36:08.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.133+0000 7f236555e700 1 -- 172.21.15.89:0/2862350278 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23600cd440 con 0x7f23600b5eb0 2023-10-10T20:36:08.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.134+0000 7f236555e700 1 -- 172.21.15.89:0/2862350278 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236004c3c0 con 0x7f23600b5eb0 2023-10-10T20:36:08.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.134+0000 7f2356ffd700 1 -- 172.21.15.89:0/2862350278 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f23480019d0 con 0x7f23600b5eb0 2023-10-10T20:36:08.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.135+0000 7f2356ffd700 1 --2- 172.21.15.89:0/2862350278 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2340066fc0 0x7f2340069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.135+0000 7f2356ffd700 1 -- 172.21.15.89:0/2862350278 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f23480846d0 con 0x7f23600b5eb0 2023-10-10T20:36:08.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.135+0000 7f235effd700 1 --2- 172.21.15.89:0/2862350278 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2340066fc0 0x7f2340069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.135+0000 7f235effd700 1 --2- 172.21.15.89:0/2862350278 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2340066fc0 0x7f2340069470 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2350007c90 tx=0x7f2350015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:08.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.139+0000 7f2356ffd700 1 -- 172.21.15.89:0/2862350278 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2348051950 con 0x7f23600b5eb0 2023-10-10T20:36:08.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.321+0000 7f236555e700 1 -- 172.21.15.89:0/2862350278 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23600b1c50 con 0x7f23600b5eb0 2023-10-10T20:36:08.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.321+0000 7f2356ffd700 1 -- 172.21.15.89:0/2862350278 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2348054f70 con 0x7f23600b5eb0 2023-10-10T20:36:08.325 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-10T20:36:08.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.323+0000 7f236555e700 1 -- 172.21.15.89:0/2862350278 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2340066fc0 msgr2=0x7f2340069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:08.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.323+0000 7f236555e700 1 --2- 172.21.15.89:0/2862350278 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2340066fc0 0x7f2340069470 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2350007c90 tx=0x7f2350015000 comp rx=0 tx=0).stop 2023-10-10T20:36:08.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.323+0000 7f236555e700 1 -- 172.21.15.89:0/2862350278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23600b5eb0 msgr2=0x7f23601521e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:08.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.323+0000 7f236555e700 1 --2- 172.21.15.89:0/2862350278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23600b5eb0 0x7f23601521e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2348002ee0 tx=0x7f2348007d10 comp rx=0 tx=0).stop 2023-10-10T20:36:08.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.323+0000 7f236555e700 1 -- 172.21.15.89:0/2862350278 shutdown_connections 2023-10-10T20:36:08.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.323+0000 7f236555e700 1 --2- 172.21.15.89:0/2862350278 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2340066fc0 0x7f2340069470 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.323+0000 7f236555e700 1 --2- 172.21.15.89:0/2862350278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2360152720 0x7f23600cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.323+0000 7f236555e700 1 --2- 172.21.15.89:0/2862350278 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f234c003c20 0x7f236014b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.323+0000 7f236555e700 1 --2- 172.21.15.89:0/2862350278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23600b5eb0 0x7f23601521e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.323+0000 7f236555e700 1 -- 172.21.15.89:0/2862350278 >> 172.21.15.89:0/2862350278 conn(0x7f2360068e20 msgr2=0x7f236014d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:08.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.323+0000 7f236555e700 1 -- 172.21.15.89:0/2862350278 shutdown_connections 2023-10-10T20:36:08.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.323+0000 7f236555e700 1 -- 172.21.15.89:0/2862350278 wait complete. 2023-10-10T20:36:08.339 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:36:08.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.450+0000 7fb8cc1b8700 1 Processor -- start 2023-10-10T20:36:08.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.450+0000 7fb8cc1b8700 1 -- start start 2023-10-10T20:36:08.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.451+0000 7fb8cc1b8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb8c4141500 0x7fb8c40c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.451+0000 7fb8cc1b8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8c406c0a0 0x7fb8c40c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.451+0000 7fb8cc1b8700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c415c2a0 con 0x7fb8c406bcd0 2023-10-10T20:36:08.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.451+0000 7fb8cc1b8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c40c9880 con 0x7fb8c4141500 2023-10-10T20:36:08.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.451+0000 7fb8cc1b8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c415b690 con 0x7fb8c406c0a0 2023-10-10T20:36:08.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.451+0000 7fb8c9753700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb8c4141500 0x7fb8c40c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.451+0000 7fb8c9753700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb8c4141500 0x7fb8c40c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54970/0 (socket says 172.21.15.89:54970) 2023-10-10T20:36:08.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.451+0000 7fb8c9753700 1 -- 172.21.15.89:0/555729321 learned_addr learned my addr 172.21.15.89:0/555729321 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:08.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.451+0000 7fb8ca755700 1 --2- 172.21.15.89:0/555729321 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8c406c0a0 0x7fb8c40c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.452+0000 7fb8c9753700 1 -- 172.21.15.89:0/555729321 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8c406c0a0 msgr2=0x7fb8c40c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:08.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.452+0000 7fb8c9753700 1 --2- 172.21.15.89:0/555729321 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8c406c0a0 0x7fb8c40c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.452+0000 7fb8c9753700 1 -- 172.21.15.89:0/555729321 >> v1:172.21.15.111:6789/0 conn(0x7fb8c406bcd0 legacy=0x7fb8c4140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:08.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.452+0000 7fb8c9753700 1 -- 172.21.15.89:0/555729321 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c415b970 con 0x7fb8c4141500 2023-10-10T20:36:08.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.452+0000 7fb8c9753700 1 --2- 172.21.15.89:0/555729321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8c4141500 0x7fb8c40c8e00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb8b40099b0 tx=0x7fb8b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6b0c10942048d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:08.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.452+0000 7fb8c8f52700 1 -- 172.21.15.89:0/555729321 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b400b860 con 0x7fb8c4141500 2023-10-10T20:36:08.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.452+0000 7fb8c8f52700 1 -- 172.21.15.89:0/555729321 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b400b9c0 con 0x7fb8c4141500 2023-10-10T20:36:08.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.452+0000 7fb8c8f52700 1 -- 172.21.15.89:0/555729321 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b400bd60 con 0x7fb8c4141500 2023-10-10T20:36:08.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.452+0000 7fb8cc1b8700 1 -- 172.21.15.89:0/555729321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8c4141500 msgr2=0x7fb8c40c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:08.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.452+0000 7fb8cc1b8700 1 --2- 172.21.15.89:0/555729321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8c4141500 0x7fb8c40c8e00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb8b40099b0 tx=0x7fb8b4009250 comp rx=0 tx=0).stop 2023-10-10T20:36:08.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.452+0000 7fb8cc1b8700 1 -- 172.21.15.89:0/555729321 shutdown_connections 2023-10-10T20:36:08.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.452+0000 7fb8cc1b8700 1 --2- 172.21.15.89:0/555729321 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8c406c0a0 0x7fb8c40c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.452+0000 7fb8cc1b8700 1 --2- 172.21.15.89:0/555729321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8c4141500 0x7fb8c40c8e00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.452+0000 7fb8cc1b8700 1 -- 172.21.15.89:0/555729321 >> 172.21.15.89:0/555729321 conn(0x7fb8c40ce600 msgr2=0x7fb8c40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:08.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.453+0000 7fb8cc1b8700 1 -- 172.21.15.89:0/555729321 shutdown_connections 2023-10-10T20:36:08.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.453+0000 7fb8cc1b8700 1 -- 172.21.15.89:0/555729321 wait complete. 2023-10-10T20:36:08.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.454+0000 7fb8cc1b8700 1 Processor -- start 2023-10-10T20:36:08.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.454+0000 7fb8cc1b8700 1 -- start start 2023-10-10T20:36:08.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.454+0000 7fb8cc1b8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8c406bcd0 0x7fb8c4139fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.454+0000 7fb8cc1b8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8c406c0a0 0x7fb8c413a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.454+0000 7fb8cc1b8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8c4141500 0x7fb8c40b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.454+0000 7fb8cc1b8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c4069b90 con 0x7fb8c406c0a0 2023-10-10T20:36:08.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.454+0000 7fb8cc1b8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c40b3dd0 con 0x7fb8c4141500 2023-10-10T20:36:08.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.454+0000 7fb8cc1b8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c40b40b0 con 0x7fb8c406bcd0 2023-10-10T20:36:08.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.454+0000 7fb8c9753700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8c406c0a0 0x7fb8c413a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.454+0000 7fb8c9753700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8c406c0a0 0x7fb8c413a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54984/0 (socket says 172.21.15.89:54984) 2023-10-10T20:36:08.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.454+0000 7fb8c9753700 1 -- 172.21.15.89:0/3966368255 learned_addr learned my addr 172.21.15.89:0/3966368255 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:08.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.455+0000 7fb8ca755700 1 --2- 172.21.15.89:0/3966368255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8c4141500 0x7fb8c40b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.455+0000 7fb8c9f54700 1 --2- 172.21.15.89:0/3966368255 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8c406bcd0 0x7fb8c4139fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.455+0000 7fb8c9753700 1 -- 172.21.15.89:0/3966368255 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8c406bcd0 msgr2=0x7fb8c4139fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.455+0000 7fb8c9753700 1 --2- 172.21.15.89:0/3966368255 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8c406bcd0 0x7fb8c4139fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.455+0000 7fb8c9753700 1 -- 172.21.15.89:0/3966368255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8c4141500 msgr2=0x7fb8c40b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.455+0000 7fb8c9753700 1 --2- 172.21.15.89:0/3966368255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8c4141500 0x7fb8c40b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.455+0000 7fb8c9753700 1 -- 172.21.15.89:0/3966368255 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b40094f0 con 0x7fb8c406c0a0 2023-10-10T20:36:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.455+0000 7fb8c9753700 1 --2- 172.21.15.89:0/3966368255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8c406c0a0 0x7fb8c413a510 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4069220 tx=0x7fb8b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.455+0000 7fb8b9ffb700 1 -- 172.21.15.89:0/3966368255 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b401a450 con 0x7fb8c406c0a0 2023-10-10T20:36:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.455+0000 7fb8b9ffb700 1 -- 172.21.15.89:0/3966368255 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b401a5d0 con 0x7fb8c406c0a0 2023-10-10T20:36:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.455+0000 7fb8cc1b8700 1 -- 172.21.15.89:0/3966368255 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c4069540 con 0x7fb8c406c0a0 2023-10-10T20:36:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.455+0000 7fb8b9ffb700 1 -- 172.21.15.89:0/3966368255 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b401a990 con 0x7fb8c406c0a0 2023-10-10T20:36:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.455+0000 7fb8cc1b8700 1 -- 172.21.15.89:0/3966368255 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c40b3550 con 0x7fb8c406c0a0 2023-10-10T20:36:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.456+0000 7fb8cc1b8700 1 -- 172.21.15.89:0/3966368255 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c4069d00 con 0x7fb8c406c0a0 2023-10-10T20:36:08.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.456+0000 7fb8b9ffb700 1 -- 172.21.15.89:0/3966368255 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fb8b401ad40 con 0x7fb8c406c0a0 2023-10-10T20:36:08.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.457+0000 7fb8b9ffb700 1 --2- 172.21.15.89:0/3966368255 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb8a8066fc0 0x7fb8a8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.457+0000 7fb8b9ffb700 1 -- 172.21.15.89:0/3966368255 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb8b40860c0 con 0x7fb8c406c0a0 2023-10-10T20:36:08.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.457+0000 7fb8c9f54700 1 --2- 172.21.15.89:0/3966368255 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb8a8066fc0 0x7fb8a8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.457+0000 7fb8c9f54700 1 --2- 172.21.15.89:0/3966368255 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb8a8066fc0 0x7fb8a8069470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0003860 tx=0x7fb8c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:08.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.461+0000 7fb8b9ffb700 1 -- 172.21.15.89:0/3966368255 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8b4053340 con 0x7fb8c406c0a0 2023-10-10T20:36:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.684+0000 7fb8cc1b8700 1 -- 172.21.15.89:0/3966368255 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8c404c3c0 con 0x7fb8c406c0a0 2023-10-10T20:36:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.684+0000 7fb8b9ffb700 1 -- 172.21.15.89:0/3966368255 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4597 (secure 0 0 0) 0x7fb8b4056960 con 0x7fb8c406c0a0 2023-10-10T20:36:08.688 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:08.688 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4922,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1464385125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1464385125},{"type":"v1","addr":"172.21.15.111:6834","nonce":1464385125}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":4925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/689169706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":689169706},{"type":"v1","addr":"172.21.15.111:6835","nonce":689169706}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/418769006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":418769006},{"type":"v1","addr":"172.21.15.89:6836","nonce":418769006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:35:55.314480+0000","modified":"2023-10-10T20:35:58.961430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5340":{"gid":5340,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/3245027879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":3245027879},{"type":"v1","addr":"172.21.15.89:6837","nonce":3245027879}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-10T20:36:08.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.687+0000 7fb8cc1b8700 1 -- 172.21.15.89:0/3966368255 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb8a8066fc0 msgr2=0x7fb8a8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.687+0000 7fb8cc1b8700 1 --2- 172.21.15.89:0/3966368255 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb8a8066fc0 0x7fb8a8069470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0003860 tx=0x7fb8c0009040 comp rx=0 tx=0).stop 2023-10-10T20:36:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.687+0000 7fb8cc1b8700 1 -- 172.21.15.89:0/3966368255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8c406c0a0 msgr2=0x7fb8c413a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.687+0000 7fb8cc1b8700 1 --2- 172.21.15.89:0/3966368255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8c406c0a0 0x7fb8c413a510 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4069220 tx=0x7fb8b4017000 comp rx=0 tx=0).stop 2023-10-10T20:36:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.687+0000 7fb8cc1b8700 1 -- 172.21.15.89:0/3966368255 shutdown_connections 2023-10-10T20:36:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.687+0000 7fb8cc1b8700 1 --2- 172.21.15.89:0/3966368255 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb8a8066fc0 0x7fb8a8069470 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.687+0000 7fb8cc1b8700 1 --2- 172.21.15.89:0/3966368255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8c4141500 0x7fb8c40b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.687+0000 7fb8cc1b8700 1 --2- 172.21.15.89:0/3966368255 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8c406bcd0 0x7fb8c4139fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.687+0000 7fb8cc1b8700 1 --2- 172.21.15.89:0/3966368255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8c406c0a0 0x7fb8c413a510 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.687+0000 7fb8cc1b8700 1 -- 172.21.15.89:0/3966368255 >> 172.21.15.89:0/3966368255 conn(0x7fb8c40ce600 msgr2=0x7fb8c41592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.687+0000 7fb8cc1b8700 1 -- 172.21.15.89:0/3966368255 shutdown_connections 2023-10-10T20:36:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.687+0000 7fb8cc1b8700 1 -- 172.21.15.89:0/3966368255 wait complete. 2023-10-10T20:36:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 30 2023-10-10T20:36:08.702 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:36:08.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.814+0000 7fd507aa5700 1 Processor -- start 2023-10-10T20:36:08.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.814+0000 7fd507aa5700 1 -- start start 2023-10-10T20:36:08.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.815+0000 7fd507aa5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd500159890 0x7fd50015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.815+0000 7fd507aa5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd500069560 0x7fd5000699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.815+0000 7fd507aa5700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd50015a160 con 0x7fd500159c60 2023-10-10T20:36:08.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.815+0000 7fd507aa5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd500069f20 con 0x7fd500069560 2023-10-10T20:36:08.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.815+0000 7fd507aa5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000d13a0 con 0x7fd500159890 2023-10-10T20:36:08.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.815+0000 7fd506042700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd500069560 0x7fd5000699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.815+0000 7fd506042700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd500069560 0x7fd5000699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54998/0 (socket says 172.21.15.89:54998) 2023-10-10T20:36:08.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.815+0000 7fd506042700 1 -- 172.21.15.89:0/1811225105 learned_addr learned my addr 172.21.15.89:0/1811225105 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:08.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.815+0000 7fd505841700 1 --2- 172.21.15.89:0/1811225105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd500159890 0x7fd50015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.815+0000 7fd506042700 1 -- 172.21.15.89:0/1811225105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd500159890 msgr2=0x7fd50015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:08.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.815+0000 7fd506042700 1 --2- 172.21.15.89:0/1811225105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd500159890 0x7fd50015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.815+0000 7fd506042700 1 -- 172.21.15.89:0/1811225105 >> v1:172.21.15.111:6789/0 conn(0x7fd500159c60 legacy=0x7fd50015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:08.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.815+0000 7fd506042700 1 -- 172.21.15.89:0/1811225105 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50015e140 con 0x7fd500069560 2023-10-10T20:36:08.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.815+0000 7fd506042700 1 --2- 172.21.15.89:0/1811225105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd500069560 0x7fd5000699e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009f20 tx=0x7fd4f8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80e74ad99e6533b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:08.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.816+0000 7fd50483f700 1 -- 172.21.15.89:0/1811225105 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f800e070 con 0x7fd500069560 2023-10-10T20:36:08.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.816+0000 7fd50483f700 1 -- 172.21.15.89:0/1811225105 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f800b9d0 con 0x7fd500069560 2023-10-10T20:36:08.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.816+0000 7fd50483f700 1 -- 172.21.15.89:0/1811225105 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f80092e0 con 0x7fd500069560 2023-10-10T20:36:08.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.816+0000 7fd507aa5700 1 -- 172.21.15.89:0/1811225105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd500069560 msgr2=0x7fd5000699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:08.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.816+0000 7fd507aa5700 1 --2- 172.21.15.89:0/1811225105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd500069560 0x7fd5000699e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009f20 tx=0x7fd4f8009840 comp rx=0 tx=0).stop 2023-10-10T20:36:08.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.816+0000 7fd507aa5700 1 -- 172.21.15.89:0/1811225105 shutdown_connections 2023-10-10T20:36:08.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.816+0000 7fd507aa5700 1 --2- 172.21.15.89:0/1811225105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd500069560 0x7fd5000699e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.816+0000 7fd507aa5700 1 --2- 172.21.15.89:0/1811225105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd500159890 0x7fd50015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.816+0000 7fd507aa5700 1 -- 172.21.15.89:0/1811225105 >> 172.21.15.89:0/1811225105 conn(0x7fd500068e20 msgr2=0x7fd5000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:08.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.816+0000 7fd507aa5700 1 -- 172.21.15.89:0/1811225105 shutdown_connections 2023-10-10T20:36:08.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.816+0000 7fd507aa5700 1 -- 172.21.15.89:0/1811225105 wait complete. 2023-10-10T20:36:08.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.817+0000 7fd507aa5700 1 Processor -- start 2023-10-10T20:36:08.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.817+0000 7fd507aa5700 1 -- start start 2023-10-10T20:36:08.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.818+0000 7fd507aa5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd500069560 0x7fd50014d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.818+0000 7fd507aa5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd500159890 0x7fd500154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.818+0000 7fd507aa5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd500159c60 0x7fd500155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.818+0000 7fd507aa5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd500151ae0 con 0x7fd500159c60 2023-10-10T20:36:08.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.818+0000 7fd507aa5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000b0940 con 0x7fd500159890 2023-10-10T20:36:08.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.818+0000 7fd507aa5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000ce9b0 con 0x7fd500069560 2023-10-10T20:36:08.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.818+0000 7fd506042700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd500159c60 0x7fd500155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.818+0000 7fd506042700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd500159c60 0x7fd500155210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55006/0 (socket says 172.21.15.89:55006) 2023-10-10T20:36:08.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.818+0000 7fd506042700 1 -- 172.21.15.89:0/1116792951 learned_addr learned my addr 172.21.15.89:0/1116792951 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:08.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.818+0000 7fd505040700 1 --2- 172.21.15.89:0/1116792951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd500159890 0x7fd500154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.818+0000 7fd505841700 1 --2- 172.21.15.89:0/1116792951 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd500069560 0x7fd50014d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.818+0000 7fd506042700 1 -- 172.21.15.89:0/1116792951 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd500069560 msgr2=0x7fd50014d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:08.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.818+0000 7fd506042700 1 --2- 172.21.15.89:0/1116792951 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd500069560 0x7fd50014d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.818+0000 7fd506042700 1 -- 172.21.15.89:0/1116792951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd500159890 msgr2=0x7fd500154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:08.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.818+0000 7fd506042700 1 --2- 172.21.15.89:0/1116792951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd500159890 0x7fd500154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:08.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.818+0000 7fd506042700 1 -- 172.21.15.89:0/1116792951 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f8009ae0 con 0x7fd500159c60 2023-10-10T20:36:08.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.818+0000 7fd505040700 1 --2- 172.21.15.89:0/1116792951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd500159890 0x7fd500154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:08.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.818+0000 7fd506042700 1 --2- 172.21.15.89:0/1116792951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd500159c60 0x7fd500155210 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800b610 tx=0x7fd4f8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:08.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.819+0000 7fd4f5ffb700 1 -- 172.21.15.89:0/1116792951 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8008c30 con 0x7fd500159c60 2023-10-10T20:36:08.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.819+0000 7fd4f5ffb700 1 -- 172.21.15.89:0/1116792951 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f8008d90 con 0x7fd500159c60 2023-10-10T20:36:08.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.819+0000 7fd507aa5700 1 -- 172.21.15.89:0/1116792951 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5000cec90 con 0x7fd500159c60 2023-10-10T20:36:08.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.819+0000 7fd4f5ffb700 1 -- 172.21.15.89:0/1116792951 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f801c640 con 0x7fd500159c60 2023-10-10T20:36:08.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.819+0000 7fd507aa5700 1 -- 172.21.15.89:0/1116792951 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5000b05d0 con 0x7fd500159c60 2023-10-10T20:36:08.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.820+0000 7fd507aa5700 1 -- 172.21.15.89:0/1116792951 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5000b0ad0 con 0x7fd500159c60 2023-10-10T20:36:08.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.821+0000 7fd4f5ffb700 1 -- 172.21.15.89:0/1116792951 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fd4f8003040 con 0x7fd500159c60 2023-10-10T20:36:08.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.821+0000 7fd4f5ffb700 1 --2- 172.21.15.89:0/1116792951 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd4e4066fc0 0x7fd4e4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:08.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.821+0000 7fd505841700 1 --2- 172.21.15.89:0/1116792951 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd4e4066fc0 0x7fd4e4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:08.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.821+0000 7fd4f5ffb700 1 -- 172.21.15.89:0/1116792951 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd4f8086680 con 0x7fd500159c60 2023-10-10T20:36:08.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.822+0000 7fd505841700 1 --2- 172.21.15.89:0/1116792951 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd4e4066fc0 0x7fd4e4069470 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009d40 tx=0x7fd4fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:08.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:08.825+0000 7fd4f5ffb700 1 -- 172.21.15.89:0/1116792951 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4f8053870 con 0x7fd500159c60 2023-10-10T20:36:09.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.012+0000 7fd507aa5700 1 -- 172.21.15.89:0/1116792951 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd50004c3c0 con 0x7fd500159c60 2023-10-10T20:36:09.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.012+0000 7fd4f5ffb700 1 -- 172.21.15.89:0/1116792951 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fd4f8056e90 con 0x7fd500159c60 2023-10-10T20:36:09.016 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:09.016 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":50,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:35:56.929187+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:35:53.658728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-10T20:35:54.272054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:36:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.015+0000 7fd507aa5700 1 -- 172.21.15.89:0/1116792951 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd4e4066fc0 msgr2=0x7fd4e4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.015+0000 7fd507aa5700 1 --2- 172.21.15.89:0/1116792951 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd4e4066fc0 0x7fd4e4069470 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009d40 tx=0x7fd4fc008040 comp rx=0 tx=0).stop 2023-10-10T20:36:09.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.015+0000 7fd507aa5700 1 -- 172.21.15.89:0/1116792951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd500159c60 msgr2=0x7fd500155210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:09.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.015+0000 7fd507aa5700 1 --2- 172.21.15.89:0/1116792951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd500159c60 0x7fd500155210 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800b610 tx=0x7fd4f8018000 comp rx=0 tx=0).stop 2023-10-10T20:36:09.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.015+0000 7fd507aa5700 1 -- 172.21.15.89:0/1116792951 shutdown_connections 2023-10-10T20:36:09.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.015+0000 7fd507aa5700 1 --2- 172.21.15.89:0/1116792951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd500159c60 0x7fd500155210 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.015+0000 7fd507aa5700 1 --2- 172.21.15.89:0/1116792951 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd500069560 0x7fd50014d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.015+0000 7fd507aa5700 1 --2- 172.21.15.89:0/1116792951 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd4e4066fc0 0x7fd4e4069470 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.015+0000 7fd507aa5700 1 --2- 172.21.15.89:0/1116792951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd500159890 0x7fd500154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.015+0000 7fd507aa5700 1 -- 172.21.15.89:0/1116792951 >> 172.21.15.89:0/1116792951 conn(0x7fd500068e20 msgr2=0x7fd500158120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:09.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.015+0000 7fd507aa5700 1 -- 172.21.15.89:0/1116792951 shutdown_connections 2023-10-10T20:36:09.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.015+0000 7fd507aa5700 1 -- 172.21.15.89:0/1116792951 wait complete. 2023-10-10T20:36:09.030 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:36:09.031 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:36:09.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.142+0000 7f04fbe46700 1 Processor -- start 2023-10-10T20:36:09.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.142+0000 7f04fbe46700 1 -- start start 2023-10-10T20:36:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.143+0000 7f04fbe46700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f04f4159890 0x7f04f415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.143+0000 7f04fbe46700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f04f4159c60 0x7f04f415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.143+0000 7f04fbe46700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04f415db10 0x7f04f4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.143+0000 7f04fbe46700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04f415df50 con 0x7f04f4159c60 2023-10-10T20:36:09.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.143+0000 7f04fbe46700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04f415a160 con 0x7f04f4159890 2023-10-10T20:36:09.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.143+0000 7f04fbe46700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f40d13a0 con 0x7f04f415db10 2023-10-10T20:36:09.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.143+0000 7f04f93e1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f04f4159c60 0x7f04f415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:09.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.143+0000 7f04f93e1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f04f4159c60 0x7f04f415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55008/0 (socket says 172.21.15.89:55008) 2023-10-10T20:36:09.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.143+0000 7f04f93e1700 1 -- 172.21.15.89:0/2796399946 learned_addr learned my addr 172.21.15.89:0/2796399946 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:09.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.143+0000 7f04f9be2700 1 --2- 172.21.15.89:0/2796399946 >> v2:172.21.15.111:3300/0 conn(0x7f04f4159890 0x7f04f415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:09.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.143+0000 7f04fa3e3700 1 --2- 172.21.15.89:0/2796399946 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04f415db10 0x7f04f4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:09.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.143+0000 7f04f93e1700 1 -- 172.21.15.89:0/2796399946 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04f415db10 msgr2=0x7f04f4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:09.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.143+0000 7f04f93e1700 1 --2- 172.21.15.89:0/2796399946 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04f415db10 0x7f04f4069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.143+0000 7f04f93e1700 1 -- 172.21.15.89:0/2796399946 >> v2:172.21.15.111:3300/0 conn(0x7f04f4159890 msgr2=0x7f04f415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:09.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.143+0000 7f04f93e1700 1 --2- 172.21.15.89:0/2796399946 >> v2:172.21.15.111:3300/0 conn(0x7f04f4159890 0x7f04f415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.143+0000 7f04f93e1700 1 -- 172.21.15.89:0/2796399946 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f415e140 con 0x7f04f4159c60 2023-10-10T20:36:09.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.143+0000 7f04fa3e3700 1 --2- 172.21.15.89:0/2796399946 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04f415db10 0x7f04f4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:09.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.144+0000 7f04f93e1700 1 --2- 172.21.15.89:0/2796399946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f4159c60 0x7f04f415d5d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f04f40d1090 tx=0x7f04e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aac61af2bd5a38 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:09.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.144+0000 7f04f8be0700 1 -- 172.21.15.89:0/2796399946 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e400b860 con 0x7f04f4159c60 2023-10-10T20:36:09.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.144+0000 7f04f8be0700 1 -- 172.21.15.89:0/2796399946 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e400b9c0 con 0x7f04f4159c60 2023-10-10T20:36:09.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.144+0000 7f04f8be0700 1 -- 172.21.15.89:0/2796399946 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e400bd60 con 0x7f04f4159c60 2023-10-10T20:36:09.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.144+0000 7f04fbe46700 1 -- 172.21.15.89:0/2796399946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f4159c60 msgr2=0x7f04f415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:09.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.144+0000 7f04fbe46700 1 --2- 172.21.15.89:0/2796399946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f4159c60 0x7f04f415d5d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f04f40d1090 tx=0x7f04e4009250 comp rx=0 tx=0).stop 2023-10-10T20:36:09.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.144+0000 7f04fbe46700 1 -- 172.21.15.89:0/2796399946 shutdown_connections 2023-10-10T20:36:09.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.144+0000 7f04fbe46700 1 --2- 172.21.15.89:0/2796399946 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04f415db10 0x7f04f4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.144+0000 7f04fbe46700 1 --2- 172.21.15.89:0/2796399946 >> v2:172.21.15.111:3300/0 conn(0x7f04f4159890 0x7f04f415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.144+0000 7f04fbe46700 1 --2- 172.21.15.89:0/2796399946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f4159c60 0x7f04f415d5d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.144+0000 7f04fbe46700 1 -- 172.21.15.89:0/2796399946 >> 172.21.15.89:0/2796399946 conn(0x7f04f4068e20 msgr2=0x7f04f40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:09.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.144+0000 7f04fbe46700 1 -- 172.21.15.89:0/2796399946 shutdown_connections 2023-10-10T20:36:09.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.144+0000 7f04fbe46700 1 -- 172.21.15.89:0/2796399946 wait complete. 2023-10-10T20:36:09.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.145+0000 7f04fbe46700 1 Processor -- start 2023-10-10T20:36:09.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.146+0000 7f04fbe46700 1 -- start start 2023-10-10T20:36:09.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.146+0000 7f04fbe46700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f4159890 0x7f04f40b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:09.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.146+0000 7f04fbe46700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04f4159c60 0x7f04f41473d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:09.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.146+0000 7f04fbe46700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04f415db10 0x7f04f40b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:09.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.146+0000 7f04fbe46700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f4144b90 con 0x7f04f4159890 2023-10-10T20:36:09.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.146+0000 7f04fbe46700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f40b4e60 con 0x7f04f415db10 2023-10-10T20:36:09.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.146+0000 7f04fbe46700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f40b48a0 con 0x7f04f4159c60 2023-10-10T20:36:09.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.146+0000 7f04f9be2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f4159890 0x7f04f40b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:09.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.146+0000 7f04f9be2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f4159890 0x7f04f40b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55010/0 (socket says 172.21.15.89:55010) 2023-10-10T20:36:09.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.146+0000 7f04f9be2700 1 -- 172.21.15.89:0/3446773561 learned_addr learned my addr 172.21.15.89:0/3446773561 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:09.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.146+0000 7f04fa3e3700 1 --2- 172.21.15.89:0/3446773561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04f415db10 0x7f04f40b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:09.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.146+0000 7f04f93e1700 1 --2- 172.21.15.89:0/3446773561 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04f4159c60 0x7f04f41473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:09.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.147+0000 7f04f9be2700 1 -- 172.21.15.89:0/3446773561 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04f4159c60 msgr2=0x7f04f41473d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:09.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.147+0000 7f04f9be2700 1 --2- 172.21.15.89:0/3446773561 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04f4159c60 0x7f04f41473d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.147+0000 7f04f9be2700 1 -- 172.21.15.89:0/3446773561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04f415db10 msgr2=0x7f04f40b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:09.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.147+0000 7f04f9be2700 1 --2- 172.21.15.89:0/3446773561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04f415db10 0x7f04f40b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.147+0000 7f04f9be2700 1 -- 172.21.15.89:0/3446773561 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e40094f0 con 0x7f04f4159890 2023-10-10T20:36:09.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.147+0000 7f04fa3e3700 1 --2- 172.21.15.89:0/3446773561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04f415db10 0x7f04f40b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:09.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.147+0000 7f04f9be2700 1 --2- 172.21.15.89:0/3446773561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f4159890 0x7f04f40b3f50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f04f000d860 tx=0x7f04f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:09.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.147+0000 7f04e9ffb700 1 -- 172.21.15.89:0/3446773561 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f0009610 con 0x7f04f4159890 2023-10-10T20:36:09.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.147+0000 7f04e9ffb700 1 -- 172.21.15.89:0/3446773561 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f0009770 con 0x7f04f4159890 2023-10-10T20:36:09.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.147+0000 7f04fbe46700 1 -- 172.21.15.89:0/3446773561 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f40b4b80 con 0x7f04f4159890 2023-10-10T20:36:09.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.147+0000 7f04e9ffb700 1 -- 172.21.15.89:0/3446773561 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f0009b30 con 0x7f04f4159890 2023-10-10T20:36:09.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.147+0000 7f04fbe46700 1 -- 172.21.15.89:0/3446773561 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f40c7c20 con 0x7f04f4159890 2023-10-10T20:36:09.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.148+0000 7f04fbe46700 1 -- 172.21.15.89:0/3446773561 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f404c3c0 con 0x7f04f4159890 2023-10-10T20:36:09.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.148+0000 7f04e9ffb700 1 -- 172.21.15.89:0/3446773561 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f04f001e460 con 0x7f04f4159890 2023-10-10T20:36:09.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.149+0000 7f04e9ffb700 1 --2- 172.21.15.89:0/3446773561 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f04d8066fc0 0x7f04d8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:09.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.149+0000 7f04e9ffb700 1 -- 172.21.15.89:0/3446773561 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f04f0084bc0 con 0x7f04f4159890 2023-10-10T20:36:09.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.149+0000 7f04f93e1700 1 --2- 172.21.15.89:0/3446773561 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f04d8066fc0 0x7f04d8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:09.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.149+0000 7f04f93e1700 1 --2- 172.21.15.89:0/3446773561 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f04d8066fc0 0x7f04d8069470 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04f40d1090 tx=0x7f04e4017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:09.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.153+0000 7f04e9ffb700 1 -- 172.21.15.89:0/3446773561 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04f0051e40 con 0x7f04f4159890 2023-10-10T20:36:09.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.380+0000 7f04fbe46700 1 -- 172.21.15.89:0/3446773561 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04f406a5b0 con 0x7f04f4159890 2023-10-10T20:36:09.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.381+0000 7f04e9ffb700 1 -- 172.21.15.89:0/3446773561 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f04f0055460 con 0x7f04f4159890 2023-10-10T20:36:09.385 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:09.385 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4922,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1464385125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1464385125},{"type":"v1","addr":"172.21.15.111:6834","nonce":1464385125}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":4925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/689169706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":689169706},{"type":"v1","addr":"172.21.15.111:6835","nonce":689169706}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/418769006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":418769006},{"type":"v1","addr":"172.21.15.89:6836","nonce":418769006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:35:55.314480+0000","modified":"2023-10-10T20:35:58.961430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5340":{"gid":5340,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/3245027879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":3245027879},{"type":"v1","addr":"172.21.15.89:6837","nonce":3245027879}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-10T20:36:09.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.383+0000 7f04fbe46700 1 -- 172.21.15.89:0/3446773561 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f04d8066fc0 msgr2=0x7f04d8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:09.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.383+0000 7f04fbe46700 1 --2- 172.21.15.89:0/3446773561 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f04d8066fc0 0x7f04d8069470 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04f40d1090 tx=0x7f04e4017000 comp rx=0 tx=0).stop 2023-10-10T20:36:09.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.383+0000 7f04fbe46700 1 -- 172.21.15.89:0/3446773561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f4159890 msgr2=0x7f04f40b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:09.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.383+0000 7f04fbe46700 1 --2- 172.21.15.89:0/3446773561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f4159890 0x7f04f40b3f50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f04f000d860 tx=0x7f04f0008040 comp rx=0 tx=0).stop 2023-10-10T20:36:09.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.383+0000 7f04fbe46700 1 -- 172.21.15.89:0/3446773561 shutdown_connections 2023-10-10T20:36:09.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.383+0000 7f04fbe46700 1 --2- 172.21.15.89:0/3446773561 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f04d8066fc0 0x7f04d8069470 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.383+0000 7f04fbe46700 1 --2- 172.21.15.89:0/3446773561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04f415db10 0x7f04f40b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.383+0000 7f04fbe46700 1 --2- 172.21.15.89:0/3446773561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f4159890 0x7f04f40b3f50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.383+0000 7f04fbe46700 1 --2- 172.21.15.89:0/3446773561 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04f4159c60 0x7f04f41473d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.383+0000 7f04fbe46700 1 -- 172.21.15.89:0/3446773561 >> 172.21.15.89:0/3446773561 conn(0x7f04f4068e20 msgr2=0x7f04f415d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:09.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.383+0000 7f04fbe46700 1 -- 172.21.15.89:0/3446773561 shutdown_connections 2023-10-10T20:36:09.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.383+0000 7f04fbe46700 1 -- 172.21.15.89:0/3446773561 wait complete. 2023-10-10T20:36:09.389 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 30 2023-10-10T20:36:09.398 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:36:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.510+0000 7f2a5a789700 1 Processor -- start 2023-10-10T20:36:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.510+0000 7f2a5a789700 1 -- start start 2023-10-10T20:36:09.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.511+0000 7f2a5a789700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2a540b1c20 0x7f2a541572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:09.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.511+0000 7f2a5a789700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a54157c60 con 0x7f2a54157820 2023-10-10T20:36:09.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.511+0000 7f2a5a789700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a540cb300 con 0x7f2a540b1850 2023-10-10T20:36:09.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.511+0000 7f2a5a789700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5406fc30 con 0x7f2a540b1c20 2023-10-10T20:36:09.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.511+0000 7f2a537fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2a540b1c20 0x7f2a541572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:09.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.511+0000 7f2a58d26700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2a54157820 0x7f2a540abf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42824/0 (socket says 172.21.15.89:42824) 2023-10-10T20:36:09.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.511+0000 7f2a58d26700 1 -- 172.21.15.89:0/902396962 learned_addr learned my addr 172.21.15.89:0/902396962 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:09.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.511+0000 7f2a52ffd700 1 -- 172.21.15.89:0/902396962 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214005000 0 0) 0x7f2a54157c60 con 0x7f2a54157820 2023-10-10T20:36:09.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.511+0000 7f2a52ffd700 1 -- 172.21.15.89:0/902396962 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a3c003570 con 0x7f2a54157820 2023-10-10T20:36:09.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.511+0000 7f2a537fe700 1 -- 172.21.15.89:0/902396962 >> v1:172.21.15.111:6789/0 conn(0x7f2a540b1850 legacy=0x7f2a540ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:09.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.511+0000 7f2a537fe700 1 -- 172.21.15.89:0/902396962 >> v1:172.21.15.89:6789/0 conn(0x7f2a54157820 legacy=0x7f2a540abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:09.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.511+0000 7f2a537fe700 1 -- 172.21.15.89:0/902396962 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a540cb5e0 con 0x7f2a540b1c20 2023-10-10T20:36:09.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.511+0000 7f2a537fe700 1 --2- 172.21.15.89:0/902396962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540b1c20 0x7f2a541572e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2a440098e0 tx=0x7f2a440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edac8e7e80cfb1ff server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:09.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.511+0000 7f2a52ffd700 1 -- 172.21.15.89:0/902396962 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4400b640 con 0x7f2a540b1c20 2023-10-10T20:36:09.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.511+0000 7f2a52ffd700 1 -- 172.21.15.89:0/902396962 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4400b7a0 con 0x7f2a540b1c20 2023-10-10T20:36:09.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.511+0000 7f2a52ffd700 1 -- 172.21.15.89:0/902396962 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4400bb40 con 0x7f2a540b1c20 2023-10-10T20:36:09.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.512+0000 7f2a5a789700 1 -- 172.21.15.89:0/902396962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540b1c20 msgr2=0x7f2a541572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:09.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.512+0000 7f2a5a789700 1 --2- 172.21.15.89:0/902396962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540b1c20 0x7f2a541572e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2a440098e0 tx=0x7f2a440091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.512+0000 7f2a5a789700 1 -- 172.21.15.89:0/902396962 shutdown_connections 2023-10-10T20:36:09.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.512+0000 7f2a5a789700 1 --2- 172.21.15.89:0/902396962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540b1c20 0x7f2a541572e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.512+0000 7f2a5a789700 1 -- 172.21.15.89:0/902396962 >> 172.21.15.89:0/902396962 conn(0x7f2a5406a320 msgr2=0x7f2a540cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:09.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.512+0000 7f2a5a789700 1 -- 172.21.15.89:0/902396962 shutdown_connections 2023-10-10T20:36:09.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.512+0000 7f2a5a789700 1 -- 172.21.15.89:0/902396962 wait complete. 2023-10-10T20:36:09.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.513+0000 7f2a5a789700 1 Processor -- start 2023-10-10T20:36:09.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.513+0000 7f2a5a789700 1 -- start start 2023-10-10T20:36:09.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.514+0000 7f2a5a789700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540b1850 0x7f2a54150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:09.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.514+0000 7f2a5a789700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2a540b1c20 0x7f2a541582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:09.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.514+0000 7f2a5a789700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a54157820 0x7f2a54158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:09.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.514+0000 7f2a5a789700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540cc260 con 0x7f2a540b1850 2023-10-10T20:36:09.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.514+0000 7f2a5a789700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a54159a50 con 0x7f2a54157820 2023-10-10T20:36:09.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.514+0000 7f2a5a789700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a54158e40 con 0x7f2a540b1c20 2023-10-10T20:36:09.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.514+0000 7f2a53fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540b1850 0x7f2a54150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:09.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.514+0000 7f2a53fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540b1850 0x7f2a54150ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55026/0 (socket says 172.21.15.89:55026) 2023-10-10T20:36:09.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.514+0000 7f2a53fff700 1 -- 172.21.15.89:0/900614398 learned_addr learned my addr 172.21.15.89:0/900614398 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:09.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.514+0000 7f2a58d26700 1 --2- 172.21.15.89:0/900614398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a54157820 0x7f2a54158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:09.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.514+0000 7f2a537fe700 1 --2- 172.21.15.89:0/900614398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2a540b1c20 0x7f2a541582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:09.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.514+0000 7f2a53fff700 1 -- 172.21.15.89:0/900614398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2a540b1c20 msgr2=0x7f2a541582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:09.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.514+0000 7f2a53fff700 1 --2- 172.21.15.89:0/900614398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2a540b1c20 0x7f2a541582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.514+0000 7f2a53fff700 1 -- 172.21.15.89:0/900614398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a54157820 msgr2=0x7f2a54158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:09.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.514+0000 7f2a53fff700 1 --2- 172.21.15.89:0/900614398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a54157820 0x7f2a54158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.514+0000 7f2a53fff700 1 -- 172.21.15.89:0/900614398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a44009420 con 0x7f2a540b1850 2023-10-10T20:36:09.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.514+0000 7f2a53fff700 1 --2- 172.21.15.89:0/900614398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540b1850 0x7f2a54150ed0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800c590 tx=0x7f2a4800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:09.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.514+0000 7f2a3bfff700 1 -- 172.21.15.89:0/900614398 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a48011590 con 0x7f2a540b1850 2023-10-10T20:36:09.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.514+0000 7f2a3bfff700 1 -- 172.21.15.89:0/900614398 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a48011710 con 0x7f2a540b1850 2023-10-10T20:36:09.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.515+0000 7f2a5a789700 1 -- 172.21.15.89:0/900614398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a54159120 con 0x7f2a540b1850 2023-10-10T20:36:09.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.515+0000 7f2a3bfff700 1 -- 172.21.15.89:0/900614398 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a48011ad0 con 0x7f2a540b1850 2023-10-10T20:36:09.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.515+0000 7f2a5a789700 1 -- 172.21.15.89:0/900614398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a541596e0 con 0x7f2a540b1850 2023-10-10T20:36:09.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.515+0000 7f2a5a789700 1 -- 172.21.15.89:0/900614398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5404c3c0 con 0x7f2a540b1850 2023-10-10T20:36:09.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.516+0000 7f2a3bfff700 1 -- 172.21.15.89:0/900614398 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f2a4801b3b0 con 0x7f2a540b1850 2023-10-10T20:36:09.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.516+0000 7f2a3bfff700 1 --2- 172.21.15.89:0/900614398 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2a40066fc0 0x7f2a40069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:09.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.516+0000 7f2a537fe700 1 --2- 172.21.15.89:0/900614398 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2a40066fc0 0x7f2a40069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:09.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.516+0000 7f2a3bfff700 1 -- 172.21.15.89:0/900614398 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2a480839e0 con 0x7f2a540b1850 2023-10-10T20:36:09.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.517+0000 7f2a537fe700 1 --2- 172.21.15.89:0/900614398 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2a40066fc0 0x7f2a40069470 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2a440093f0 tx=0x7f2a44017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:09.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.521+0000 7f2a3bfff700 1 -- 172.21.15.89:0/900614398 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a48050c60 con 0x7f2a540b1850 2023-10-10T20:36:09.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.743+0000 7f2a5a789700 1 -- 172.21.15.89:0/900614398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a54159d30 con 0x7f2a540b1850 2023-10-10T20:36:09.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.743+0000 7f2a3bfff700 1 -- 172.21.15.89:0/900614398 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f2a48017070 con 0x7f2a540b1850 2023-10-10T20:36:09.747 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:09.748 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4922,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1464385125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1464385125},{"type":"v1","addr":"172.21.15.111:6834","nonce":1464385125}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":4925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/689169706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":689169706},{"type":"v1","addr":"172.21.15.111:6835","nonce":689169706}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/418769006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":418769006},{"type":"v1","addr":"172.21.15.89:6836","nonce":418769006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:35:55.314480+0000","modified":"2023-10-10T20:35:58.961430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5340":{"gid":5340,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/3245027879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":3245027879},{"type":"v1","addr":"172.21.15.89:6837","nonce":3245027879}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-10T20:36:09.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.746+0000 7f2a5a789700 1 -- 172.21.15.89:0/900614398 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2a40066fc0 msgr2=0x7f2a40069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:09.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.746+0000 7f2a5a789700 1 --2- 172.21.15.89:0/900614398 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2a40066fc0 0x7f2a40069470 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2a440093f0 tx=0x7f2a44017000 comp rx=0 tx=0).stop 2023-10-10T20:36:09.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.746+0000 7f2a5a789700 1 -- 172.21.15.89:0/900614398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540b1850 msgr2=0x7f2a54150ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:09.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.746+0000 7f2a5a789700 1 --2- 172.21.15.89:0/900614398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540b1850 0x7f2a54150ed0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800c590 tx=0x7f2a4800c8a0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.746+0000 7f2a5a789700 1 -- 172.21.15.89:0/900614398 shutdown_connections 2023-10-10T20:36:09.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.746+0000 7f2a5a789700 1 --2- 172.21.15.89:0/900614398 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2a40066fc0 0x7f2a40069470 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.746+0000 7f2a5a789700 1 --2- 172.21.15.89:0/900614398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a54157820 0x7f2a54158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.746+0000 7f2a5a789700 1 --2- 172.21.15.89:0/900614398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540b1850 0x7f2a54150ed0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.746+0000 7f2a5a789700 1 --2- 172.21.15.89:0/900614398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2a540b1c20 0x7f2a541582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.746+0000 7f2a5a789700 1 -- 172.21.15.89:0/900614398 >> 172.21.15.89:0/900614398 conn(0x7f2a5406a320 msgr2=0x7f2a54156710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:09.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.746+0000 7f2a5a789700 1 -- 172.21.15.89:0/900614398 shutdown_connections 2023-10-10T20:36:09.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.746+0000 7f2a5a789700 1 -- 172.21.15.89:0/900614398 wait complete. 2023-10-10T20:36:09.752 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 30 2023-10-10T20:36:09.761 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:36:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.872+0000 7f7d28a0a700 1 Processor -- start 2023-10-10T20:36:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.872+0000 7f7d28a0a700 1 -- start start 2023-10-10T20:36:09.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.873+0000 7f7d28a0a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7d24159890 0x7f7d2415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:09.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.873+0000 7f7d28a0a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7d24159c60 0x7f7d2415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:09.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.873+0000 7f7d28a0a700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2415a160 con 0x7f7d2415db10 2023-10-10T20:36:09.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.873+0000 7f7d28a0a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d2415df50 con 0x7f7d24159890 2023-10-10T20:36:09.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.873+0000 7f7d28a0a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d240d13a0 con 0x7f7d24159c60 2023-10-10T20:36:09.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.873+0000 7f7d2259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7d24159890 0x7f7d2415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:09.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.873+0000 7f7d2259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7d24159890 0x7f7d2415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55038/0 (socket says 172.21.15.89:55038) 2023-10-10T20:36:09.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.873+0000 7f7d2259c700 1 -- 172.21.15.89:0/3059141227 learned_addr learned my addr 172.21.15.89:0/3059141227 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:09.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.873+0000 7f7d19d9b700 1 --2- 172.21.15.89:0/3059141227 >> v2:172.21.15.111:3300/0 conn(0x7f7d24159c60 0x7f7d2415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:09.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.873+0000 7f7d2259c700 1 -- 172.21.15.89:0/3059141227 >> v2:172.21.15.111:3300/0 conn(0x7f7d24159c60 msgr2=0x7f7d2415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:09.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.873+0000 7f7d2259c700 1 --2- 172.21.15.89:0/3059141227 >> v2:172.21.15.111:3300/0 conn(0x7f7d24159c60 0x7f7d2415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.873+0000 7f7d2259c700 1 -- 172.21.15.89:0/3059141227 >> v1:172.21.15.111:6789/0 conn(0x7f7d2415db10 legacy=0x7f7d24069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:09.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.873+0000 7f7d2259c700 1 -- 172.21.15.89:0/3059141227 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2415e140 con 0x7f7d24159890 2023-10-10T20:36:09.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.874+0000 7f7d2259c700 1 --2- 172.21.15.89:0/3059141227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d24159890 0x7f7d2415abd0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7d240b67f0 tx=0x7f7d14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d2338435211fd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:09.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.874+0000 7f7d21d9b700 1 -- 172.21.15.89:0/3059141227 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1400b820 con 0x7f7d24159890 2023-10-10T20:36:09.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.874+0000 7f7d21d9b700 1 -- 172.21.15.89:0/3059141227 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1400b980 con 0x7f7d24159890 2023-10-10T20:36:09.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.874+0000 7f7d21d9b700 1 -- 172.21.15.89:0/3059141227 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1400bd20 con 0x7f7d24159890 2023-10-10T20:36:09.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.874+0000 7f7d28a0a700 1 -- 172.21.15.89:0/3059141227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d24159890 msgr2=0x7f7d2415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:09.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.874+0000 7f7d28a0a700 1 --2- 172.21.15.89:0/3059141227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d24159890 0x7f7d2415abd0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7d240b67f0 tx=0x7f7d14009250 comp rx=0 tx=0).stop 2023-10-10T20:36:09.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.874+0000 7f7d28a0a700 1 -- 172.21.15.89:0/3059141227 shutdown_connections 2023-10-10T20:36:09.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.874+0000 7f7d28a0a700 1 --2- 172.21.15.89:0/3059141227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d24159890 0x7f7d2415abd0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.874+0000 7f7d28a0a700 1 --2- 172.21.15.89:0/3059141227 >> v2:172.21.15.111:3300/0 conn(0x7f7d24159c60 0x7f7d2415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.874+0000 7f7d28a0a700 1 -- 172.21.15.89:0/3059141227 >> 172.21.15.89:0/3059141227 conn(0x7f7d24068e20 msgr2=0x7f7d240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:09.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.874+0000 7f7d28a0a700 1 -- 172.21.15.89:0/3059141227 shutdown_connections 2023-10-10T20:36:09.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.874+0000 7f7d28a0a700 1 -- 172.21.15.89:0/3059141227 wait complete. 2023-10-10T20:36:09.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.875+0000 7f7d28a0a700 1 Processor -- start 2023-10-10T20:36:09.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.875+0000 7f7d28a0a700 1 -- start start 2023-10-10T20:36:09.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.876+0000 7f7d28a0a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d24159890 0x7f7d24157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:09.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.876+0000 7f7d28a0a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7d24159c60 0x7f7d2416a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:09.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.876+0000 7f7d28a0a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d2415db10 0x7f7d240ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:09.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.876+0000 7f7d28a0a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2416b0d0 con 0x7f7d24159890 2023-10-10T20:36:09.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.876+0000 7f7d28a0a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2416a600 con 0x7f7d2415db10 2023-10-10T20:36:09.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.876+0000 7f7d28a0a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2416a830 con 0x7f7d24159c60 2023-10-10T20:36:09.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.876+0000 7f7d2259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d24159890 0x7f7d24157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:09.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.876+0000 7f7d2259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d24159890 0x7f7d24157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55054/0 (socket says 172.21.15.89:55054) 2023-10-10T20:36:09.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.876+0000 7f7d2259c700 1 -- 172.21.15.89:0/1073056830 learned_addr learned my addr 172.21.15.89:0/1073056830 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:09.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.876+0000 7f7d2259c700 1 -- 172.21.15.89:0/1073056830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7d24159c60 msgr2=0x7f7d2416a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:09.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.876+0000 7f7d2259c700 1 --2- 172.21.15.89:0/1073056830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7d24159c60 0x7f7d2416a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.876+0000 7f7d2259c700 1 -- 172.21.15.89:0/1073056830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d2415db10 msgr2=0x7f7d240ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:09.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.876+0000 7f7d2259c700 1 --2- 172.21.15.89:0/1073056830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d2415db10 0x7f7d240ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:09.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.876+0000 7f7d2259c700 1 -- 172.21.15.89:0/1073056830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d140094f0 con 0x7f7d24159890 2023-10-10T20:36:09.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.877+0000 7f7d2259c700 1 --2- 172.21.15.89:0/1073056830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d24159890 0x7f7d24157f80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7d24070e20 tx=0x7f7d14016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:09.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.877+0000 7f7d1affd700 1 -- 172.21.15.89:0/1073056830 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14011e20 con 0x7f7d24159890 2023-10-10T20:36:09.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.877+0000 7f7d1affd700 1 -- 172.21.15.89:0/1073056830 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d14020070 con 0x7f7d24159890 2023-10-10T20:36:09.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.877+0000 7f7d28a0a700 1 -- 172.21.15.89:0/1073056830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2416ab10 con 0x7f7d24159890 2023-10-10T20:36:09.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.877+0000 7f7d1affd700 1 -- 172.21.15.89:0/1073056830 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1401b6a0 con 0x7f7d24159890 2023-10-10T20:36:09.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.877+0000 7f7d28a0a700 1 -- 172.21.15.89:0/1073056830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2416b430 con 0x7f7d24159890 2023-10-10T20:36:09.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.878+0000 7f7d28a0a700 1 -- 172.21.15.89:0/1073056830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2416b290 con 0x7f7d24159890 2023-10-10T20:36:09.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.878+0000 7f7d1affd700 1 -- 172.21.15.89:0/1073056830 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f7d14022030 con 0x7f7d24159890 2023-10-10T20:36:09.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.878+0000 7f7d1affd700 1 --2- 172.21.15.89:0/1073056830 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7d04066fc0 0x7f7d04069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:09.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.878+0000 7f7d1affd700 1 -- 172.21.15.89:0/1073056830 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7d140864f0 con 0x7f7d24159890 2023-10-10T20:36:09.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.878+0000 7f7d19d9b700 1 --2- 172.21.15.89:0/1073056830 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7d04066fc0 0x7f7d04069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:09.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.879+0000 7f7d19d9b700 1 --2- 172.21.15.89:0/1073056830 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7d04066fc0 0x7f7d04069470 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7d2415d870 tx=0x7f7d0c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:09.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:09.883+0000 7f7d1affd700 1 -- 172.21.15.89:0/1073056830 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d2416b290 con 0x7f7d24159890 2023-10-10T20:36:10.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.067+0000 7f7d28a0a700 1 -- 172.21.15.89:0/1073056830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d2416b290 con 0x7f7d24159890 2023-10-10T20:36:10.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.068+0000 7f7d1affd700 1 -- 172.21.15.89:0/1073056830 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f7d2416b290 con 0x7f7d24159890 2023-10-10T20:36:10.072 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:10.072 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":50,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:35:56.929187+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:35:53.658728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-10T20:35:54.272054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:36:10.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.070+0000 7f7d28a0a700 1 -- 172.21.15.89:0/1073056830 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7d04066fc0 msgr2=0x7f7d04069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:10.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.070+0000 7f7d28a0a700 1 --2- 172.21.15.89:0/1073056830 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7d04066fc0 0x7f7d04069470 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7d2415d870 tx=0x7f7d0c008040 comp rx=0 tx=0).stop 2023-10-10T20:36:10.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.070+0000 7f7d28a0a700 1 -- 172.21.15.89:0/1073056830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d24159890 msgr2=0x7f7d24157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:10.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.070+0000 7f7d28a0a700 1 --2- 172.21.15.89:0/1073056830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d24159890 0x7f7d24157f80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7d24070e20 tx=0x7f7d14016000 comp rx=0 tx=0).stop 2023-10-10T20:36:10.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.070+0000 7f7d28a0a700 1 -- 172.21.15.89:0/1073056830 shutdown_connections 2023-10-10T20:36:10.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.070+0000 7f7d28a0a700 1 --2- 172.21.15.89:0/1073056830 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7d04066fc0 0x7f7d04069470 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:10.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.070+0000 7f7d28a0a700 1 --2- 172.21.15.89:0/1073056830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d2415db10 0x7f7d240ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:10.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.070+0000 7f7d28a0a700 1 --2- 172.21.15.89:0/1073056830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d24159890 0x7f7d24157f80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:10.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.070+0000 7f7d28a0a700 1 --2- 172.21.15.89:0/1073056830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7d24159c60 0x7f7d2416a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:10.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.070+0000 7f7d28a0a700 1 -- 172.21.15.89:0/1073056830 >> 172.21.15.89:0/1073056830 conn(0x7f7d24068e20 msgr2=0x7f7d240b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:10.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.070+0000 7f7d28a0a700 1 -- 172.21.15.89:0/1073056830 shutdown_connections 2023-10-10T20:36:10.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.070+0000 7f7d28a0a700 1 -- 172.21.15.89:0/1073056830 wait complete. 2023-10-10T20:36:10.086 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:36:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.198+0000 7fb89b2cb700 1 Processor -- start 2023-10-10T20:36:10.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.198+0000 7fb89b2cb700 1 -- start start 2023-10-10T20:36:10.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.199+0000 7fb89b2cb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb894159890 0x7fb89415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:10.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.199+0000 7fb89b2cb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb894069560 0x7fb8940699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:10.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.199+0000 7fb89b2cb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb89415a160 con 0x7fb894159c60 2023-10-10T20:36:10.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.199+0000 7fb89b2cb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb894069f20 con 0x7fb894069560 2023-10-10T20:36:10.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.199+0000 7fb89b2cb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8940d13a0 con 0x7fb894159890 2023-10-10T20:36:10.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.199+0000 7fb899868700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb894069560 0x7fb8940699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:10.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.199+0000 7fb898866700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb894159c60 0x7fb89415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42836/0 (socket says 172.21.15.89:42836) 2023-10-10T20:36:10.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.199+0000 7fb898866700 1 -- 172.21.15.89:0/2514242772 learned_addr learned my addr 172.21.15.89:0/2514242772 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:10.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.199+0000 7fb899067700 1 --2- 172.21.15.89:0/2514242772 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb894159890 0x7fb89415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:10.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.199+0000 7fb899868700 1 -- 172.21.15.89:0/2514242772 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb894159890 msgr2=0x7fb89415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:10.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.200+0000 7fb899868700 1 --2- 172.21.15.89:0/2514242772 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb894159890 0x7fb89415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:10.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.200+0000 7fb899868700 1 -- 172.21.15.89:0/2514242772 >> v1:172.21.15.89:6789/0 conn(0x7fb894159c60 legacy=0x7fb89415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:10.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.200+0000 7fb88bfff700 1 -- 172.21.15.89:0/2514242772 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106396034 0 0) 0x7fb89415a160 con 0x7fb894159c60 2023-10-10T20:36:10.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.200+0000 7fb899868700 1 -- 172.21.15.89:0/2514242772 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89415e140 con 0x7fb894069560 2023-10-10T20:36:10.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.200+0000 7fb899067700 1 --2- 172.21.15.89:0/2514242772 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb894159890 0x7fb89415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:10.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.200+0000 7fb899868700 1 --2- 172.21.15.89:0/2514242772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb894069560 0x7fb8940699e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb88c009e60 tx=0x7fb88c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2541c0926a8d09b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:10.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.200+0000 7fb88bfff700 1 -- 172.21.15.89:0/2514242772 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c00e070 con 0x7fb894069560 2023-10-10T20:36:10.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.200+0000 7fb88bfff700 1 -- 172.21.15.89:0/2514242772 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88c00bc30 con 0x7fb894069560 2023-10-10T20:36:10.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.200+0000 7fb88bfff700 1 -- 172.21.15.89:0/2514242772 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c012430 con 0x7fb894069560 2023-10-10T20:36:10.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.200+0000 7fb89b2cb700 1 -- 172.21.15.89:0/2514242772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb894069560 msgr2=0x7fb8940699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:10.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.200+0000 7fb89b2cb700 1 --2- 172.21.15.89:0/2514242772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb894069560 0x7fb8940699e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb88c009e60 tx=0x7fb88c009780 comp rx=0 tx=0).stop 2023-10-10T20:36:10.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.200+0000 7fb89b2cb700 1 -- 172.21.15.89:0/2514242772 shutdown_connections 2023-10-10T20:36:10.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.200+0000 7fb89b2cb700 1 --2- 172.21.15.89:0/2514242772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb894069560 0x7fb8940699e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:10.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.200+0000 7fb89b2cb700 1 --2- 172.21.15.89:0/2514242772 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb894159890 0x7fb89415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:10.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.200+0000 7fb89b2cb700 1 -- 172.21.15.89:0/2514242772 >> 172.21.15.89:0/2514242772 conn(0x7fb894068e20 msgr2=0x7fb8940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:10.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.200+0000 7fb89b2cb700 1 -- 172.21.15.89:0/2514242772 shutdown_connections 2023-10-10T20:36:10.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.201+0000 7fb89b2cb700 1 -- 172.21.15.89:0/2514242772 wait complete. 2023-10-10T20:36:10.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.202+0000 7fb89b2cb700 1 Processor -- start 2023-10-10T20:36:10.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.202+0000 7fb89b2cb700 1 -- start start 2023-10-10T20:36:10.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.202+0000 7fb89b2cb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb894069560 0x7fb8940b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:10.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.202+0000 7fb89b2cb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb894159890 0x7fb8940b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:10.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.202+0000 7fb89b2cb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb894159c60 0x7fb894154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:10.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.202+0000 7fb89b2cb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8940b5950 con 0x7fb894159c60 2023-10-10T20:36:10.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.202+0000 7fb89b2cb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8940b5c30 con 0x7fb894159890 2023-10-10T20:36:10.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.202+0000 7fb89b2cb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8940b5f10 con 0x7fb894069560 2023-10-10T20:36:10.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.203+0000 7fb899868700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb894159c60 0x7fb894154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:10.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.203+0000 7fb899868700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb894159c60 0x7fb894154fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55076/0 (socket says 172.21.15.89:55076) 2023-10-10T20:36:10.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.203+0000 7fb899868700 1 -- 172.21.15.89:0/2796845825 learned_addr learned my addr 172.21.15.89:0/2796845825 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:10.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.203+0000 7fb898866700 1 --2- 172.21.15.89:0/2796845825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb894159890 0x7fb8940b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:10.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.203+0000 7fb899067700 1 --2- 172.21.15.89:0/2796845825 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb894069560 0x7fb8940b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:10.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.203+0000 7fb899868700 1 -- 172.21.15.89:0/2796845825 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb894069560 msgr2=0x7fb8940b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:10.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.203+0000 7fb899868700 1 --2- 172.21.15.89:0/2796845825 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb894069560 0x7fb8940b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:10.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.203+0000 7fb899868700 1 -- 172.21.15.89:0/2796845825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb894159890 msgr2=0x7fb8940b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:10.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.203+0000 7fb899868700 1 --2- 172.21.15.89:0/2796845825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb894159890 0x7fb8940b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:10.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.203+0000 7fb899868700 1 -- 172.21.15.89:0/2796845825 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c009a20 con 0x7fb894159c60 2023-10-10T20:36:10.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.203+0000 7fb899868700 1 --2- 172.21.15.89:0/2796845825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb894159c60 0x7fb894154fe0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb8940899c0 tx=0x7fb88c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:10.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.203+0000 7fb8897fa700 1 -- 172.21.15.89:0/2796845825 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c01d630 con 0x7fb894159c60 2023-10-10T20:36:10.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.203+0000 7fb8897fa700 1 -- 172.21.15.89:0/2796845825 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88c01d7b0 con 0x7fb894159c60 2023-10-10T20:36:10.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.203+0000 7fb89b2cb700 1 -- 172.21.15.89:0/2796845825 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8940b61f0 con 0x7fb894159c60 2023-10-10T20:36:10.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.203+0000 7fb8897fa700 1 -- 172.21.15.89:0/2796845825 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c01db70 con 0x7fb894159c60 2023-10-10T20:36:10.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.203+0000 7fb89b2cb700 1 -- 172.21.15.89:0/2796845825 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8940b1bc0 con 0x7fb894159c60 2023-10-10T20:36:10.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.204+0000 7fb89b2cb700 1 -- 172.21.15.89:0/2796845825 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8940b1de0 con 0x7fb894159c60 2023-10-10T20:36:10.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.205+0000 7fb8897fa700 1 -- 172.21.15.89:0/2796845825 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fb88c024070 con 0x7fb894159c60 2023-10-10T20:36:10.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.205+0000 7fb8897fa700 1 --2- 172.21.15.89:0/2796845825 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb878066fc0 0x7fb878069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:10.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.205+0000 7fb8897fa700 1 -- 172.21.15.89:0/2796845825 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb88c086f60 con 0x7fb894159c60 2023-10-10T20:36:10.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.205+0000 7fb899067700 1 --2- 172.21.15.89:0/2796845825 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb878066fc0 0x7fb878069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:10.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.205+0000 7fb899067700 1 --2- 172.21.15.89:0/2796845825 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb878066fc0 0x7fb878069470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb89000ac00 tx=0x7fb890006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:10.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.209+0000 7fb8897fa700 1 -- 172.21.15.89:0/2796845825 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb88c0b85a0 con 0x7fb894159c60 2023-10-10T20:36:10.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.435+0000 7fb89b2cb700 1 -- 172.21.15.89:0/2796845825 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb89415b520 con 0x7fb894159c60 2023-10-10T20:36:10.439 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-10-10T20:36:10.436+0000 7fd0184ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:36:10.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.440+0000 7fb8897fa700 1 -- 172.21.15.89:0/2796845825 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb88c0541e0 con 0x7fb894159c60 2023-10-10T20:36:10.444 INFO:tasks.ceph.mds.a.smithi089.stderr: -1764> 2023-10-10T20:36:01.238+0000 7fbaf9a29700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:36:10.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.442+0000 7fb89b2cb700 1 -- 172.21.15.89:0/2796845825 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb878066fc0 msgr2=0x7fb878069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:10.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.442+0000 7fb89b2cb700 1 --2- 172.21.15.89:0/2796845825 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb878066fc0 0x7fb878069470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb89000ac00 tx=0x7fb890006f90 comp rx=0 tx=0).stop 2023-10-10T20:36:10.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.442+0000 7fb89b2cb700 1 -- 172.21.15.89:0/2796845825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb894159c60 msgr2=0x7fb894154fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:10.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.442+0000 7fb89b2cb700 1 --2- 172.21.15.89:0/2796845825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb894159c60 0x7fb894154fe0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb8940899c0 tx=0x7fb88c019000 comp rx=0 tx=0).stop 2023-10-10T20:36:10.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.442+0000 7fb89b2cb700 1 -- 172.21.15.89:0/2796845825 shutdown_connections 2023-10-10T20:36:10.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.442+0000 7fb89b2cb700 1 --2- 172.21.15.89:0/2796845825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb894159c60 0x7fb894154fe0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:10.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.442+0000 7fb89b2cb700 1 --2- 172.21.15.89:0/2796845825 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb894069560 0x7fb8940b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:10.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.442+0000 7fb89b2cb700 1 --2- 172.21.15.89:0/2796845825 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb878066fc0 0x7fb878069470 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:10.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.442+0000 7fb89b2cb700 1 --2- 172.21.15.89:0/2796845825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb894159890 0x7fb8940b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:10.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.442+0000 7fb89b2cb700 1 -- 172.21.15.89:0/2796845825 >> 172.21.15.89:0/2796845825 conn(0x7fb894068e20 msgr2=0x7fb894158810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:10.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.442+0000 7fb89b2cb700 1 -- 172.21.15.89:0/2796845825 shutdown_connections 2023-10-10T20:36:10.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.442+0000 7fb89b2cb700 1 -- 172.21.15.89:0/2796845825 wait complete. 2023-10-10T20:36:10.449 INFO:teuthology.orchestra.run.smithi089.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:36:10.458 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:36:10.466 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:36:10.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.571+0000 7ff59e6f6700 1 Processor -- start 2023-10-10T20:36:10.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.571+0000 7ff59e6f6700 1 -- start start 2023-10-10T20:36:10.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.572+0000 7ff59e6f6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff5980b4e70 0x7ff5980ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:10.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.572+0000 7ff59e6f6700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff5980b16d0 0x7ff5980b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:10.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.572+0000 7ff59e6f6700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5980b2080 con 0x7ff59815d240 2023-10-10T20:36:10.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.572+0000 7ff59e6f6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5980cfb00 con 0x7ff5980b4e70 2023-10-10T20:36:10.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.572+0000 7ff59e6f6700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5980b5c40 con 0x7ff5980b16d0 2023-10-10T20:36:10.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.572+0000 7ff59d6f4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff5980b4e70 0x7ff5980ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:10.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.572+0000 7ff59d6f4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff5980b4e70 0x7ff5980ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55092/0 (socket says 172.21.15.89:55092) 2023-10-10T20:36:10.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.572+0000 7ff59d6f4700 1 -- 172.21.15.89:0/3892433255 learned_addr learned my addr 172.21.15.89:0/3892433255 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.572+0000 7ff59cef3700 1 --2- 172.21.15.89:0/3892433255 >> v2:172.21.15.111:3300/0 conn(0x7ff5980b16d0 0x7ff5980b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.572+0000 7ff59d6f4700 1 -- 172.21.15.89:0/3892433255 >> v2:172.21.15.111:3300/0 conn(0x7ff5980b16d0 msgr2=0x7ff5980b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.572+0000 7ff59d6f4700 1 --2- 172.21.15.89:0/3892433255 >> v2:172.21.15.111:3300/0 conn(0x7ff5980b16d0 0x7ff5980b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.572+0000 7ff59d6f4700 1 -- 172.21.15.89:0/3892433255 >> v1:172.21.15.111:6789/0 conn(0x7ff59815d240 legacy=0x7ff59815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.572+0000 7ff59d6f4700 1 -- 172.21.15.89:0/3892433255 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5980b5f20 con 0x7ff5980b4e70 2023-10-10T20:36:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.572+0000 7ff59cef3700 1 --2- 172.21.15.89:0/3892433255 >> v2:172.21.15.111:3300/0 conn(0x7ff5980b16d0 0x7ff5980b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:10.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.572+0000 7ff59d6f4700 1 --2- 172.21.15.89:0/3892433255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5980b4e70 0x7ff5980ccff0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff598089830 tx=0x7ff594009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf58f0b272e45046 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:10.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.573+0000 7ff58ffff700 1 -- 172.21.15.89:0/3892433255 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59400b820 con 0x7ff5980b4e70 2023-10-10T20:36:10.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.573+0000 7ff58ffff700 1 -- 172.21.15.89:0/3892433255 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59400b980 con 0x7ff5980b4e70 2023-10-10T20:36:10.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.573+0000 7ff58ffff700 1 -- 172.21.15.89:0/3892433255 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59400bd20 con 0x7ff5980b4e70 2023-10-10T20:36:10.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.573+0000 7ff59e6f6700 1 -- 172.21.15.89:0/3892433255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5980b4e70 msgr2=0x7ff5980ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:10.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.573+0000 7ff59e6f6700 1 --2- 172.21.15.89:0/3892433255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5980b4e70 0x7ff5980ccff0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff598089830 tx=0x7ff594009250 comp rx=0 tx=0).stop 2023-10-10T20:36:10.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.573+0000 7ff59e6f6700 1 -- 172.21.15.89:0/3892433255 shutdown_connections 2023-10-10T20:36:10.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.573+0000 7ff59e6f6700 1 --2- 172.21.15.89:0/3892433255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5980b4e70 0x7ff5980ccff0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:10.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.573+0000 7ff59e6f6700 1 --2- 172.21.15.89:0/3892433255 >> v2:172.21.15.111:3300/0 conn(0x7ff5980b16d0 0x7ff5980b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.573+0000 7ff59e6f6700 1 -- 172.21.15.89:0/3892433255 >> 172.21.15.89:0/3892433255 conn(0x7ff5980d10c0 msgr2=0x7ff5980affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.573+0000 7ff59e6f6700 1 -- 172.21.15.89:0/3892433255 shutdown_connections 2023-10-10T20:36:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.573+0000 7ff59e6f6700 1 -- 172.21.15.89:0/3892433255 wait complete. 2023-10-10T20:36:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.574+0000 7ff59e6f6700 1 Processor -- start 2023-10-10T20:36:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.574+0000 7ff59e6f6700 1 -- start start 2023-10-10T20:36:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.575+0000 7ff59e6f6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5980b16d0 0x7ff598147a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.575+0000 7ff59e6f6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5980b4e70 0x7ff5980a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.575+0000 7ff59e6f6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff59815d240 0x7ff5980a7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.575+0000 7ff59e6f6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff598158850 con 0x7ff5980b16d0 2023-10-10T20:36:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.575+0000 7ff59e6f6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5980ca780 con 0x7ff5980b4e70 2023-10-10T20:36:10.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.575+0000 7ff59e6f6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5980c8b40 con 0x7ff59815d240 2023-10-10T20:36:10.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.575+0000 7ff59d6f4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5980b16d0 0x7ff598147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:10.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.575+0000 7ff59d6f4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5980b16d0 0x7ff598147a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55094/0 (socket says 172.21.15.89:55094) 2023-10-10T20:36:10.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.575+0000 7ff59d6f4700 1 -- 172.21.15.89:0/184896326 learned_addr learned my addr 172.21.15.89:0/184896326 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:10.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.576+0000 7ff59cef3700 1 --2- 172.21.15.89:0/184896326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5980b4e70 0x7ff5980a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:10.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.576+0000 7ff59def5700 1 --2- 172.21.15.89:0/184896326 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff59815d240 0x7ff5980a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:10.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.576+0000 7ff59d6f4700 1 -- 172.21.15.89:0/184896326 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff59815d240 msgr2=0x7ff5980a7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:10.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.576+0000 7ff59d6f4700 1 --2- 172.21.15.89:0/184896326 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff59815d240 0x7ff5980a7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:10.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.576+0000 7ff59d6f4700 1 -- 172.21.15.89:0/184896326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5980b4e70 msgr2=0x7ff5980a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:10.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.576+0000 7ff59d6f4700 1 --2- 172.21.15.89:0/184896326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5980b4e70 0x7ff5980a6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:10.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.576+0000 7ff59d6f4700 1 -- 172.21.15.89:0/184896326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5940094f0 con 0x7ff5980b16d0 2023-10-10T20:36:10.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.576+0000 7ff59cef3700 1 --2- 172.21.15.89:0/184896326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5980b4e70 0x7ff5980a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:10.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.576+0000 7ff59d6f4700 1 --2- 172.21.15.89:0/184896326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5980b16d0 0x7ff598147a00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff598070dd0 tx=0x7ff594016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:10.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.576+0000 7ff58d7fa700 1 -- 172.21.15.89:0/184896326 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff594011e20 con 0x7ff5980b16d0 2023-10-10T20:36:10.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.576+0000 7ff58d7fa700 1 -- 172.21.15.89:0/184896326 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff594020070 con 0x7ff5980b16d0 2023-10-10T20:36:10.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.576+0000 7ff59e6f6700 1 -- 172.21.15.89:0/184896326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5980c8e20 con 0x7ff5980b16d0 2023-10-10T20:36:10.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.576+0000 7ff58d7fa700 1 -- 172.21.15.89:0/184896326 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59401b6a0 con 0x7ff5980b16d0 2023-10-10T20:36:10.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.576+0000 7ff59e6f6700 1 -- 172.21.15.89:0/184896326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5980c9100 con 0x7ff5980b16d0 2023-10-10T20:36:10.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.577+0000 7ff59e6f6700 1 -- 172.21.15.89:0/184896326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff598088990 con 0x7ff5980b16d0 2023-10-10T20:36:10.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.577+0000 7ff58d7fa700 1 -- 172.21.15.89:0/184896326 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7ff594022030 con 0x7ff5980b16d0 2023-10-10T20:36:10.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.578+0000 7ff58d7fa700 1 --2- 172.21.15.89:0/184896326 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff57c066fc0 0x7ff57c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:10.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.578+0000 7ff58d7fa700 1 -- 172.21.15.89:0/184896326 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5791+0+0 (secure 0 0 0) 0x7ff594086540 con 0x7ff5980b16d0 2023-10-10T20:36:10.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.578+0000 7ff59cef3700 1 --2- 172.21.15.89:0/184896326 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff57c066fc0 0x7ff57c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:10.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.578+0000 7ff59cef3700 1 --2- 172.21.15.89:0/184896326 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff57c066fc0 0x7ff57c069470 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff59816d460 tx=0x7ff588006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:10.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.582+0000 7ff58d7fa700 1 -- 172.21.15.89:0/184896326 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff594053770 con 0x7ff5980b16d0 2023-10-10T20:36:10.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:10.811+0000 7ff59e6f6700 1 -- 172.21.15.89:0/184896326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff598088990 con 0x7ff5980b16d0 2023-10-10T20:36:11.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.443+0000 7ff58d7fa700 1 -- 172.21.15.89:0/184896326 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff594056d90 con 0x7ff5980b16d0 2023-10-10T20:36:11.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.445+0000 7ff59e6f6700 1 -- 172.21.15.89:0/184896326 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff57c066fc0 msgr2=0x7ff57c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:11.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.445+0000 7ff59e6f6700 1 --2- 172.21.15.89:0/184896326 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff57c066fc0 0x7ff57c069470 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff59816d460 tx=0x7ff588006f90 comp rx=0 tx=0).stop 2023-10-10T20:36:11.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.445+0000 7ff59e6f6700 1 -- 172.21.15.89:0/184896326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5980b16d0 msgr2=0x7ff598147a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:11.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.445+0000 7ff59e6f6700 1 --2- 172.21.15.89:0/184896326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5980b16d0 0x7ff598147a00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff598070dd0 tx=0x7ff594016000 comp rx=0 tx=0).stop 2023-10-10T20:36:11.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.445+0000 7ff59e6f6700 1 -- 172.21.15.89:0/184896326 shutdown_connections 2023-10-10T20:36:11.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.445+0000 7ff59e6f6700 1 --2- 172.21.15.89:0/184896326 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff59815d240 0x7ff5980a7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:11.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.445+0000 7ff59e6f6700 1 --2- 172.21.15.89:0/184896326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5980b16d0 0x7ff598147a00 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:11.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.445+0000 7ff59e6f6700 1 --2- 172.21.15.89:0/184896326 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff57c066fc0 0x7ff57c069470 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:11.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.445+0000 7ff59e6f6700 1 --2- 172.21.15.89:0/184896326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5980b4e70 0x7ff5980a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:11.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.445+0000 7ff59e6f6700 1 -- 172.21.15.89:0/184896326 >> 172.21.15.89:0/184896326 conn(0x7ff5980d10c0 msgr2=0x7ff5980b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:11.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.446+0000 7ff59e6f6700 1 -- 172.21.15.89:0/184896326 shutdown_connections 2023-10-10T20:36:11.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.446+0000 7ff59e6f6700 1 -- 172.21.15.89:0/184896326 wait complete. 2023-10-10T20:36:11.461 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:36:11.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.617+0000 7fa5443af700 1 Processor -- start 2023-10-10T20:36:11.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.617+0000 7fa5443af700 1 -- start start 2023-10-10T20:36:11.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.618+0000 7fa5443af700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa53c06ab40 0x7fa53c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:11.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.618+0000 7fa5443af700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa53c068390 0x7fa53c0c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:11.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.618+0000 7fa5443af700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa53c068900 con 0x7fa53c0af8c0 2023-10-10T20:36:11.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.618+0000 7fa5443af700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa53c068be0 con 0x7fa53c06ab40 2023-10-10T20:36:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.618+0000 7fa5443af700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c068ec0 con 0x7fa53c068390 2023-10-10T20:36:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.618+0000 7fa54294c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa53c0af8c0 0x7fa53c159620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42862/0 (socket says 172.21.15.89:42862) 2023-10-10T20:36:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.618+0000 7fa54294c700 1 -- 172.21.15.89:0/2209469468 learned_addr learned my addr 172.21.15.89:0/2209469468 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa54194a700 1 --2- 172.21.15.89:0/2209469468 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa53c068390 0x7fa53c0c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa54214b700 1 --2- 172.21.15.89:0/2209469468 >> v2:172.21.15.111:3300/0 conn(0x7fa53c06ab40 0x7fa53c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa541149700 1 -- 172.21.15.89:0/2209469468 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345448936 0 0) 0x7fa53c068900 con 0x7fa53c0af8c0 2023-10-10T20:36:11.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa541149700 1 -- 172.21.15.89:0/2209469468 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa528003570 con 0x7fa53c0af8c0 2023-10-10T20:36:11.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa541149700 1 -- 172.21.15.89:0/2209469468 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1880170459 0 0) 0x7fa528003570 con 0x7fa53c0af8c0 2023-10-10T20:36:11.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa541149700 1 -- 172.21.15.89:0/2209469468 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa53c068390 msgr2=0x7fa53c0c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:11.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa541149700 1 --2- 172.21.15.89:0/2209469468 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa53c068390 0x7fa53c0c10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:11.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa541149700 1 -- 172.21.15.89:0/2209469468 >> v2:172.21.15.111:3300/0 conn(0x7fa53c06ab40 msgr2=0x7fa53c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:11.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa541149700 1 --2- 172.21.15.89:0/2209469468 >> v2:172.21.15.111:3300/0 conn(0x7fa53c06ab40 0x7fa53c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:11.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa541149700 1 -- 172.21.15.89:0/2209469468 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53c0691a0 con 0x7fa53c0af8c0 2023-10-10T20:36:11.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa541149700 1 -- 172.21.15.89:0/2209469468 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fa5340017f0 con 0x7fa53c0af8c0 2023-10-10T20:36:11.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa541149700 1 -- 172.21.15.89:0/2209469468 >> v1:172.21.15.89:6789/0 conn(0x7fa53c0af8c0 legacy=0x7fa53c159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:11.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa541149700 1 --2- 172.21.15.89:0/2209469468 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa53c068390 0x7fa528003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:11.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa541149700 1 --2- 172.21.15.89:0/2209469468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa528003fe0 0x7fa528004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:11.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa541149700 1 --2- 172.21.15.89:0/2209469468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa528004d50 0x7fa5280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:11.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa541149700 1 -- 172.21.15.89:0/2209469468 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c068be0 con 0x7fa528004d50 2023-10-10T20:36:11.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa541149700 1 -- 172.21.15.89:0/2209469468 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c068ec0 con 0x7fa528003fe0 2023-10-10T20:36:11.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa541149700 1 -- 172.21.15.89:0/2209469468 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52800ba90 con 0x7fa53c068390 2023-10-10T20:36:11.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa54294c700 1 --2- 172.21.15.89:0/2209469468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa528004d50 0x7fa5280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:11.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa54214b700 1 --2- 172.21.15.89:0/2209469468 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa53c068390 0x7fa528003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:11.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa54194a700 1 --2- 172.21.15.89:0/2209469468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa528003fe0 0x7fa528004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:11.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa54294c700 1 -- 172.21.15.89:0/2209469468 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa53c068390 msgr2=0x7fa528003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:11.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa54294c700 1 --2- 172.21.15.89:0/2209469468 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa53c068390 0x7fa528003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:11.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa54294c700 1 -- 172.21.15.89:0/2209469468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa528003fe0 msgr2=0x7fa528004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:11.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa54294c700 1 --2- 172.21.15.89:0/2209469468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa528003fe0 0x7fa528004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:11.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.619+0000 7fa54294c700 1 -- 172.21.15.89:0/2209469468 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa528003570 con 0x7fa528004d50 2023-10-10T20:36:11.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.620+0000 7fa54294c700 1 --2- 172.21.15.89:0/2209469468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa528004d50 0x7fa5280051c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa53400a970 tx=0x7fa534002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f164444c191e0e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:11.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.620+0000 7fa541149700 1 -- 172.21.15.89:0/2209469468 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa534012440 con 0x7fa528004d50 2023-10-10T20:36:11.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.620+0000 7fa541149700 1 -- 172.21.15.89:0/2209469468 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5340125c0 con 0x7fa528004d50 2023-10-10T20:36:11.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.620+0000 7fa541149700 1 -- 172.21.15.89:0/2209469468 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa534012980 con 0x7fa528004d50 2023-10-10T20:36:11.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.620+0000 7fa5443af700 1 -- 172.21.15.89:0/2209469468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa528004d50 msgr2=0x7fa5280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:11.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.620+0000 7fa5443af700 1 --2- 172.21.15.89:0/2209469468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa528004d50 0x7fa5280051c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa53400a970 tx=0x7fa534002210 comp rx=0 tx=0).stop 2023-10-10T20:36:11.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.620+0000 7fa54294c700 1 -- 172.21.15.89:0/2209469468 reap_dead start 2023-10-10T20:36:11.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.620+0000 7fa5443af700 1 -- 172.21.15.89:0/2209469468 shutdown_connections 2023-10-10T20:36:11.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.620+0000 7fa5443af700 1 -- 172.21.15.89:0/2209469468 >> 172.21.15.89:0/2209469468 conn(0x7fa53c0cd640 msgr2=0x7fa53c0cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:11.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.620+0000 7fa5443af700 1 -- 172.21.15.89:0/2209469468 shutdown_connections 2023-10-10T20:36:11.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.620+0000 7fa5443af700 1 -- 172.21.15.89:0/2209469468 wait complete. 2023-10-10T20:36:11.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.621+0000 7fa5443af700 1 Processor -- start 2023-10-10T20:36:11.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.621+0000 7fa5443af700 1 -- start start 2023-10-10T20:36:11.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.622+0000 7fa5443af700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa528004d50 0x7fa53c0ab730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:11.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.622+0000 7fa5443af700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa53c14e9d0 0x7fa53c0c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:11.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.622+0000 7fa5443af700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa53c14ee40 0x7fa53c0c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:11.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.622+0000 7fa5443af700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c0abc70 con 0x7fa53c14ee40 2023-10-10T20:36:11.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.622+0000 7fa5443af700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c0ad180 con 0x7fa528004d50 2023-10-10T20:36:11.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.622+0000 7fa5443af700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c15c6f0 con 0x7fa53c14e9d0 2023-10-10T20:36:11.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.622+0000 7fa54294c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa53c14ee40 0x7fa53c0c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:11.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.622+0000 7fa54294c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa53c14ee40 0x7fa53c0c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55116/0 (socket says 172.21.15.89:55116) 2023-10-10T20:36:11.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.622+0000 7fa54294c700 1 -- 172.21.15.89:0/3627612439 learned_addr learned my addr 172.21.15.89:0/3627612439 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:11.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.622+0000 7fa54294c700 1 -- 172.21.15.89:0/3627612439 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa53c14e9d0 msgr2=0x7fa53c0c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:11.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.622+0000 7fa54294c700 1 --2- 172.21.15.89:0/3627612439 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa53c14e9d0 0x7fa53c0c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:11.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.622+0000 7fa54294c700 1 -- 172.21.15.89:0/3627612439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa528004d50 msgr2=0x7fa53c0ab730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:11.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.622+0000 7fa54294c700 1 --2- 172.21.15.89:0/3627612439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa528004d50 0x7fa53c0ab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:11.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.622+0000 7fa54294c700 1 -- 172.21.15.89:0/3627612439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53c15c9d0 con 0x7fa53c14ee40 2023-10-10T20:36:11.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.623+0000 7fa54294c700 1 --2- 172.21.15.89:0/3627612439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa53c14ee40 0x7fa53c0c7d20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa53400a000 tx=0x7fa534015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:11.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.623+0000 7fa5327fc700 1 -- 172.21.15.89:0/3627612439 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa534023070 con 0x7fa53c14ee40 2023-10-10T20:36:11.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.623+0000 7fa5327fc700 1 -- 172.21.15.89:0/3627612439 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa534012440 con 0x7fa53c14ee40 2023-10-10T20:36:11.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.623+0000 7fa5327fc700 1 -- 172.21.15.89:0/3627612439 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5340017b0 con 0x7fa53c14ee40 2023-10-10T20:36:11.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.623+0000 7fa5443af700 1 -- 172.21.15.89:0/3627612439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53c15ccb0 con 0x7fa53c14ee40 2023-10-10T20:36:11.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.623+0000 7fa5443af700 1 -- 172.21.15.89:0/3627612439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53c0ab0a0 con 0x7fa53c14ee40 2023-10-10T20:36:11.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.624+0000 7fa5443af700 1 -- 172.21.15.89:0/3627612439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa53c0718b0 con 0x7fa53c14ee40 2023-10-10T20:36:11.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.624+0000 7fa5327fc700 1 -- 172.21.15.89:0/3627612439 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fa534019070 con 0x7fa53c14ee40 2023-10-10T20:36:11.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.625+0000 7fa5327fc700 1 --2- 172.21.15.89:0/3627612439 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa520066fc0 0x7fa520069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:11.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.625+0000 7fa5327fc700 1 -- 172.21.15.89:0/3627612439 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fa534086090 con 0x7fa53c14ee40 2023-10-10T20:36:11.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.625+0000 7fa54214b700 1 --2- 172.21.15.89:0/3627612439 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa520066fc0 0x7fa520069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:11.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.625+0000 7fa54214b700 1 --2- 172.21.15.89:0/3627612439 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa520066fc0 0x7fa520069470 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa538009170 tx=0x7fa5380091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:11.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.630+0000 7fa5327fc700 1 -- 172.21.15.89:0/3627612439 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5340532c0 con 0x7fa53c14ee40 2023-10-10T20:36:11.763 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:36:11.766 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:36:11.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:11.817+0000 7fa5443af700 1 -- 172.21.15.89:0/3627612439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa53c0ad340 con 0x7fa53c14ee40 2023-10-10T20:36:12.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.447+0000 7fa5327fc700 1 -- 172.21.15.89:0/3627612439 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v52) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa534012940 con 0x7fa53c14ee40 2023-10-10T20:36:12.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.452+0000 7fa5443af700 1 -- 172.21.15.89:0/3627612439 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa520066fc0 msgr2=0x7fa520069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:12.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.452+0000 7fa5443af700 1 --2- 172.21.15.89:0/3627612439 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa520066fc0 0x7fa520069470 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa538009170 tx=0x7fa5380091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:12.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.452+0000 7fa5443af700 1 -- 172.21.15.89:0/3627612439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa53c14ee40 msgr2=0x7fa53c0c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:12.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.452+0000 7fa5443af700 1 --2- 172.21.15.89:0/3627612439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa53c14ee40 0x7fa53c0c7d20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa53400a000 tx=0x7fa534015000 comp rx=0 tx=0).stop 2023-10-10T20:36:12.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.452+0000 7fa5443af700 1 -- 172.21.15.89:0/3627612439 shutdown_connections 2023-10-10T20:36:12.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.452+0000 7fa5443af700 1 --2- 172.21.15.89:0/3627612439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa53c14ee40 0x7fa53c0c7d20 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:12.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.452+0000 7fa5443af700 1 --2- 172.21.15.89:0/3627612439 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa520066fc0 0x7fa520069470 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:12.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.452+0000 7fa5443af700 1 --2- 172.21.15.89:0/3627612439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa528004d50 0x7fa53c0ab730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:12.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.452+0000 7fa5443af700 1 --2- 172.21.15.89:0/3627612439 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa53c14e9d0 0x7fa53c0c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:12.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.452+0000 7fa5443af700 1 -- 172.21.15.89:0/3627612439 >> 172.21.15.89:0/3627612439 conn(0x7fa53c0cd640 msgr2=0x7fa53c0b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:12.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.453+0000 7fa5443af700 1 -- 172.21.15.89:0/3627612439 shutdown_connections 2023-10-10T20:36:12.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.453+0000 7fa5443af700 1 -- 172.21.15.89:0/3627612439 wait complete. 2023-10-10T20:36:12.458 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:36:12.471 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:36:12.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.588+0000 7f319d8bc700 1 Processor -- start 2023-10-10T20:36:12.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.588+0000 7f319d8bc700 1 -- start start 2023-10-10T20:36:12.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.589+0000 7f319d8bc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31980b0790 0x7f31980cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:12.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.589+0000 7f319d8bc700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f31981591e0 0x7f3198159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:12.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.589+0000 7f319d8bc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31980cfa00 con 0x7f31980b03c0 2023-10-10T20:36:12.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.589+0000 7f319d8bc700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31980cb580 con 0x7f31981591e0 2023-10-10T20:36:12.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.589+0000 7f319d8bc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31980cf160 con 0x7f31980b0790 2023-10-10T20:36:12.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.589+0000 7f319c8ba700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f31980b03c0 0x7f31980b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42870/0 (socket says 172.21.15.89:42870) 2023-10-10T20:36:12.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.589+0000 7f319c8ba700 1 -- 172.21.15.89:0/293303175 learned_addr learned my addr 172.21.15.89:0/293303175 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:12.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.589+0000 7f319d0bb700 1 --2- 172.21.15.89:0/293303175 >> v2:172.21.15.111:3300/0 conn(0x7f31981591e0 0x7f3198159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:12.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.589+0000 7f31977fe700 1 -- 172.21.15.89:0/293303175 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2991256333 0 0) 0x7f31980cfa00 con 0x7f31980b03c0 2023-10-10T20:36:12.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.589+0000 7f3197fff700 1 --2- 172.21.15.89:0/293303175 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31980b0790 0x7f31980cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:12.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.589+0000 7f31977fe700 1 -- 172.21.15.89:0/293303175 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3188003570 con 0x7f31980b03c0 2023-10-10T20:36:12.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.589+0000 7f31977fe700 1 -- 172.21.15.89:0/293303175 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 364036244 0 0) 0x7f3188003570 con 0x7f31980b03c0 2023-10-10T20:36:12.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.589+0000 7f31977fe700 1 -- 172.21.15.89:0/293303175 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31980b0790 msgr2=0x7f31980cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:12.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.589+0000 7f31977fe700 1 --2- 172.21.15.89:0/293303175 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31980b0790 0x7f31980cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:12.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.589+0000 7f31977fe700 1 -- 172.21.15.89:0/293303175 >> v2:172.21.15.111:3300/0 conn(0x7f31981591e0 msgr2=0x7f3198159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:12.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.589+0000 7f31977fe700 1 --2- 172.21.15.89:0/293303175 >> v2:172.21.15.111:3300/0 conn(0x7f31981591e0 0x7f3198159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:12.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.589+0000 7f31977fe700 1 -- 172.21.15.89:0/293303175 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31980cf440 con 0x7f31980b03c0 2023-10-10T20:36:12.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.589+0000 7f31977fe700 1 -- 172.21.15.89:0/293303175 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f318c001470 con 0x7f31980b03c0 2023-10-10T20:36:12.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.590+0000 7f31977fe700 1 -- 172.21.15.89:0/293303175 >> v1:172.21.15.89:6789/0 conn(0x7f31980b03c0 legacy=0x7f31980b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:12.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.590+0000 7f31977fe700 1 --2- 172.21.15.89:0/293303175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3188003a60 0x7f3188003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:12.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.590+0000 7f31977fe700 1 --2- 172.21.15.89:0/293303175 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31980b0790 0x7f3188004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:12.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.590+0000 7f31977fe700 1 --2- 172.21.15.89:0/293303175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3188004d50 0x7f31880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:12.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.590+0000 7f31977fe700 1 -- 172.21.15.89:0/293303175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31980cb580 con 0x7f3188004d50 2023-10-10T20:36:12.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.590+0000 7f31977fe700 1 -- 172.21.15.89:0/293303175 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31980cf160 con 0x7f3188003a60 2023-10-10T20:36:12.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.590+0000 7f31977fe700 1 -- 172.21.15.89:0/293303175 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318800ba60 con 0x7f31980b0790 2023-10-10T20:36:12.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.590+0000 7f319d0bb700 1 --2- 172.21.15.89:0/293303175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3188004d50 0x7f31880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:12.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.590+0000 7f319c8ba700 1 --2- 172.21.15.89:0/293303175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3188003a60 0x7f3188003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:12.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.590+0000 7f3197fff700 1 --2- 172.21.15.89:0/293303175 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31980b0790 0x7f3188004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:12.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.590+0000 7f319d0bb700 1 -- 172.21.15.89:0/293303175 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31980b0790 msgr2=0x7f3188004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:12.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.590+0000 7f319d0bb700 1 --2- 172.21.15.89:0/293303175 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31980b0790 0x7f3188004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:12.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.590+0000 7f319d0bb700 1 -- 172.21.15.89:0/293303175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3188003a60 msgr2=0x7f3188003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:12.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.590+0000 7f319d0bb700 1 --2- 172.21.15.89:0/293303175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3188003a60 0x7f3188003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:12.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.590+0000 7f319d0bb700 1 -- 172.21.15.89:0/293303175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3188003570 con 0x7f3188004d50 2023-10-10T20:36:12.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.590+0000 7f319d0bb700 1 --2- 172.21.15.89:0/293303175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3188004d50 0x7f31880051c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f319000e210 tx=0x7f319000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9df97b1c5450d5e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:12.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.590+0000 7f31977fe700 1 -- 172.21.15.89:0/293303175 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319000ea60 con 0x7f3188004d50 2023-10-10T20:36:12.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.590+0000 7f31977fe700 1 -- 172.21.15.89:0/293303175 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319000f040 con 0x7f3188004d50 2023-10-10T20:36:12.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.590+0000 7f31977fe700 1 -- 172.21.15.89:0/293303175 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319000ee10 con 0x7f3188004d50 2023-10-10T20:36:12.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.591+0000 7f319d8bc700 1 -- 172.21.15.89:0/293303175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3188004d50 msgr2=0x7f31880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:12.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.591+0000 7f319d8bc700 1 --2- 172.21.15.89:0/293303175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3188004d50 0x7f31880051c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f319000e210 tx=0x7f319000b040 comp rx=0 tx=0).stop 2023-10-10T20:36:12.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.591+0000 7f319d0bb700 1 -- 172.21.15.89:0/293303175 reap_dead start 2023-10-10T20:36:12.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.591+0000 7f319d8bc700 1 -- 172.21.15.89:0/293303175 shutdown_connections 2023-10-10T20:36:12.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.591+0000 7f319d8bc700 1 -- 172.21.15.89:0/293303175 >> 172.21.15.89:0/293303175 conn(0x7f3198063230 msgr2=0x7f3198063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:12.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.591+0000 7f319d8bc700 1 -- 172.21.15.89:0/293303175 shutdown_connections 2023-10-10T20:36:12.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.591+0000 7f319d8bc700 1 -- 172.21.15.89:0/293303175 wait complete. 2023-10-10T20:36:12.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.592+0000 7f319d8bc700 1 Processor -- start 2023-10-10T20:36:12.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.592+0000 7f319d8bc700 1 -- start start 2023-10-10T20:36:12.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.593+0000 7f319d8bc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f319815ccf0 0x7f31980c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:12.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.593+0000 7f319d8bc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31980c3ff0 0x7f31980af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:12.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.593+0000 7f319d8bc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f319815d0c0 0x7f31981586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:12.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.593+0000 7f319d8bc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31980c5100 con 0x7f319815d0c0 2023-10-10T20:36:12.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.593+0000 7f319d8bc700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31980c5ad0 con 0x7f31980c3ff0 2023-10-10T20:36:12.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.593+0000 7f319d8bc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31980c44f0 con 0x7f319815ccf0 2023-10-10T20:36:12.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.593+0000 7f319d0bb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f319815d0c0 0x7f31981586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:12.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.593+0000 7f319d0bb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f319815d0c0 0x7f31981586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55128/0 (socket says 172.21.15.89:55128) 2023-10-10T20:36:12.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.593+0000 7f319d0bb700 1 -- 172.21.15.89:0/3774125128 learned_addr learned my addr 172.21.15.89:0/3774125128 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:12.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.593+0000 7f319c8ba700 1 --2- 172.21.15.89:0/3774125128 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f319815ccf0 0x7f31980c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:12.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.593+0000 7f3197fff700 1 --2- 172.21.15.89:0/3774125128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31980c3ff0 0x7f31980af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:12.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.593+0000 7f319d0bb700 1 -- 172.21.15.89:0/3774125128 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f319815ccf0 msgr2=0x7f31980c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:12.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.593+0000 7f319d0bb700 1 --2- 172.21.15.89:0/3774125128 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f319815ccf0 0x7f31980c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:12.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.593+0000 7f319d0bb700 1 -- 172.21.15.89:0/3774125128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31980c3ff0 msgr2=0x7f31980af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:12.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.593+0000 7f319d0bb700 1 --2- 172.21.15.89:0/3774125128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31980c3ff0 0x7f31980af850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:12.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.593+0000 7f319d0bb700 1 -- 172.21.15.89:0/3774125128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31980c47d0 con 0x7f319815d0c0 2023-10-10T20:36:12.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.593+0000 7f319d0bb700 1 --2- 172.21.15.89:0/3774125128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f319815d0c0 0x7f31981586a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f319000afd0 tx=0x7f319001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:12.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.594+0000 7f3194ff9700 1 -- 172.21.15.89:0/3774125128 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319000eee0 con 0x7f319815d0c0 2023-10-10T20:36:12.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.594+0000 7f3194ff9700 1 -- 172.21.15.89:0/3774125128 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3190022070 con 0x7f319815d0c0 2023-10-10T20:36:12.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.594+0000 7f319d8bc700 1 -- 172.21.15.89:0/3774125128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31980c4ab0 con 0x7f319815d0c0 2023-10-10T20:36:12.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.594+0000 7f3194ff9700 1 -- 172.21.15.89:0/3774125128 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319001e870 con 0x7f319815d0c0 2023-10-10T20:36:12.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.594+0000 7f319d8bc700 1 -- 172.21.15.89:0/3774125128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31980afd90 con 0x7f319815d0c0 2023-10-10T20:36:12.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.594+0000 7f319d8bc700 1 -- 172.21.15.89:0/3774125128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f319806a1e0 con 0x7f319815d0c0 2023-10-10T20:36:12.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.595+0000 7f3194ff9700 1 -- 172.21.15.89:0/3774125128 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f3190003040 con 0x7f319815d0c0 2023-10-10T20:36:12.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.595+0000 7f3194ff9700 1 --2- 172.21.15.89:0/3774125128 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f317c066fc0 0x7f317c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:12.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.595+0000 7f3194ff9700 1 -- 172.21.15.89:0/3774125128 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f3190024070 con 0x7f319815d0c0 2023-10-10T20:36:12.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.595+0000 7f319c8ba700 1 --2- 172.21.15.89:0/3774125128 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f317c066fc0 0x7f317c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:12.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.596+0000 7f319c8ba700 1 --2- 172.21.15.89:0/3774125128 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f317c066fc0 0x7f317c069470 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f318c004890 tx=0x7f31980cb6c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:12.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.600+0000 7f3194ff9700 1 -- 172.21.15.89:0/3774125128 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3190053b50 con 0x7f319815d0c0 2023-10-10T20:36:12.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:12.783+0000 7f319d8bc700 1 -- 172.21.15.89:0/3774125128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f31980ca3d0 con 0x7f319815d0c0 2023-10-10T20:36:13.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.455+0000 7f3194ff9700 1 -- 172.21.15.89:0/3774125128 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v53) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3190057170 con 0x7f319815d0c0 2023-10-10T20:36:13.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.459+0000 7f319d8bc700 1 -- 172.21.15.89:0/3774125128 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f317c066fc0 msgr2=0x7f317c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:13.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.459+0000 7f319d8bc700 1 --2- 172.21.15.89:0/3774125128 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f317c066fc0 0x7f317c069470 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f318c004890 tx=0x7f31980cb6c0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.459+0000 7f319d8bc700 1 -- 172.21.15.89:0/3774125128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f319815d0c0 msgr2=0x7f31981586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:13.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.459+0000 7f319d8bc700 1 --2- 172.21.15.89:0/3774125128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f319815d0c0 0x7f31981586a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f319000afd0 tx=0x7f319001a000 comp rx=0 tx=0).stop 2023-10-10T20:36:13.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.459+0000 7f319d8bc700 1 -- 172.21.15.89:0/3774125128 shutdown_connections 2023-10-10T20:36:13.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.459+0000 7f319d8bc700 1 --2- 172.21.15.89:0/3774125128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f319815d0c0 0x7f31981586a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.459+0000 7f319d8bc700 1 --2- 172.21.15.89:0/3774125128 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f319815ccf0 0x7f31980c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.459+0000 7f319d8bc700 1 --2- 172.21.15.89:0/3774125128 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f317c066fc0 0x7f317c069470 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.459+0000 7f319d8bc700 1 --2- 172.21.15.89:0/3774125128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31980c3ff0 0x7f31980af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.459+0000 7f319d8bc700 1 -- 172.21.15.89:0/3774125128 >> 172.21.15.89:0/3774125128 conn(0x7f3198063230 msgr2=0x7f31980c9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:13.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.459+0000 7f319d8bc700 1 -- 172.21.15.89:0/3774125128 shutdown_connections 2023-10-10T20:36:13.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.459+0000 7f319d8bc700 1 -- 172.21.15.89:0/3774125128 wait complete. 2023-10-10T20:36:13.464 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' removed 2023-10-10T20:36:13.475 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:36:13.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.590+0000 7fcdca05f700 1 Processor -- start 2023-10-10T20:36:13.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.590+0000 7fcdca05f700 1 -- start start 2023-10-10T20:36:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.590+0000 7fcdca05f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcdc4159890 0x7fcdc415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.590+0000 7fcdca05f700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcdc4069560 0x7fcdc40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.590+0000 7fcdca05f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc4069ee0 con 0x7fcdc4159c60 2023-10-10T20:36:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.590+0000 7fcdca05f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc415a160 con 0x7fcdc4159890 2023-10-10T20:36:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.590+0000 7fcdca05f700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc40d13a0 con 0x7fcdc4069560 2023-10-10T20:36:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.591+0000 7fcdc37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcdc4159890 0x7fcdc415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.591+0000 7fcdc2ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcdc4159c60 0x7fcdc415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42886/0 (socket says 172.21.15.89:42886) 2023-10-10T20:36:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.591+0000 7fcdc2ffd700 1 -- 172.21.15.89:0/19828431 learned_addr learned my addr 172.21.15.89:0/19828431 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.591+0000 7fcdc37fe700 1 -- 172.21.15.89:0/19828431 >> v2:172.21.15.111:3300/0 conn(0x7fcdc4069560 msgr2=0x7fcdc40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.591+0000 7fcdc37fe700 1 --2- 172.21.15.89:0/19828431 >> v2:172.21.15.111:3300/0 conn(0x7fcdc4069560 0x7fcdc40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.591+0000 7fcdc37fe700 1 -- 172.21.15.89:0/19828431 >> v1:172.21.15.89:6789/0 conn(0x7fcdc4159c60 legacy=0x7fcdc415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.591+0000 7fcdc37fe700 1 -- 172.21.15.89:0/19828431 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc415e140 con 0x7fcdc4159890 2023-10-10T20:36:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.591+0000 7fcdc27fc700 1 -- 172.21.15.89:0/19828431 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2181360606 0 0) 0x7fcdc4069ee0 con 0x7fcdc4159c60 2023-10-10T20:36:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.591+0000 7fcdc37fe700 1 --2- 172.21.15.89:0/19828431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc4159890 0x7fcdc415abd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fcdc406a890 tx=0x7fcdb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f50e27d1149298f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.591+0000 7fcdc27fc700 1 -- 172.21.15.89:0/19828431 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb400b6b0 con 0x7fcdc4159890 2023-10-10T20:36:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.591+0000 7fcdc27fc700 1 -- 172.21.15.89:0/19828431 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb400b810 con 0x7fcdc4159890 2023-10-10T20:36:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.591+0000 7fcdc27fc700 1 -- 172.21.15.89:0/19828431 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb400bbb0 con 0x7fcdc4159890 2023-10-10T20:36:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.592+0000 7fcdca05f700 1 -- 172.21.15.89:0/19828431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc4159890 msgr2=0x7fcdc415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:13.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.592+0000 7fcdca05f700 1 --2- 172.21.15.89:0/19828431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc4159890 0x7fcdc415abd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fcdc406a890 tx=0x7fcdb40091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.592+0000 7fcdca05f700 1 -- 172.21.15.89:0/19828431 shutdown_connections 2023-10-10T20:36:13.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.592+0000 7fcdca05f700 1 --2- 172.21.15.89:0/19828431 >> v2:172.21.15.111:3300/0 conn(0x7fcdc4069560 0x7fcdc40699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.592+0000 7fcdca05f700 1 --2- 172.21.15.89:0/19828431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc4159890 0x7fcdc415abd0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.592+0000 7fcdca05f700 1 -- 172.21.15.89:0/19828431 >> 172.21.15.89:0/19828431 conn(0x7fcdc4068e20 msgr2=0x7fcdc40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:13.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.592+0000 7fcdca05f700 1 -- 172.21.15.89:0/19828431 shutdown_connections 2023-10-10T20:36:13.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.592+0000 7fcdca05f700 1 -- 172.21.15.89:0/19828431 wait complete. 2023-10-10T20:36:13.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.593+0000 7fcdca05f700 1 Processor -- start 2023-10-10T20:36:13.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.593+0000 7fcdca05f700 1 -- start start 2023-10-10T20:36:13.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdca05f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdc4069560 0x7fcdc41524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:13.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdca05f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc4159890 0x7fcdc40b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:13.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdca05f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcdc4159c60 0x7fcdc415c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:13.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdca05f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40b4200 con 0x7fcdc4159890 2023-10-10T20:36:13.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdca05f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40b3630 con 0x7fcdc4069560 2023-10-10T20:36:13.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdca05f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40b4690 con 0x7fcdc4159c60 2023-10-10T20:36:13.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdc2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc4159890 0x7fcdc40b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:13.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdc2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc4159890 0x7fcdc40b52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55140/0 (socket says 172.21.15.89:55140) 2023-10-10T20:36:13.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdc2ffd700 1 -- 172.21.15.89:0/989442473 learned_addr learned my addr 172.21.15.89:0/989442473 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdc37fe700 1 --2- 172.21.15.89:0/989442473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdc4069560 0x7fcdc41524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdc3fff700 1 --2- 172.21.15.89:0/989442473 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcdc4159c60 0x7fcdc415c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdc2ffd700 1 -- 172.21.15.89:0/989442473 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcdc4159c60 msgr2=0x7fcdc415c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdc2ffd700 1 --2- 172.21.15.89:0/989442473 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcdc4159c60 0x7fcdc415c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdc2ffd700 1 -- 172.21.15.89:0/989442473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdc4069560 msgr2=0x7fcdc41524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdc2ffd700 1 --2- 172.21.15.89:0/989442473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdc4069560 0x7fcdc41524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdc2ffd700 1 -- 172.21.15.89:0/989442473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb4009420 con 0x7fcdc4159890 2023-10-10T20:36:13.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdc2ffd700 1 --2- 172.21.15.89:0/989442473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc4159890 0x7fcdc40b52a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800c5b0 tx=0x7fcdb8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:13.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdab7fe700 1 -- 172.21.15.89:0/989442473 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8005b30 con 0x7fcdc4159890 2023-10-10T20:36:13.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdab7fe700 1 -- 172.21.15.89:0/989442473 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb8005c90 con 0x7fcdc4159890 2023-10-10T20:36:13.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdab7fe700 1 -- 172.21.15.89:0/989442473 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8007300 con 0x7fcdc4159890 2023-10-10T20:36:13.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.594+0000 7fcdca05f700 1 -- 172.21.15.89:0/989442473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc40b38d0 con 0x7fcdc4159890 2023-10-10T20:36:13.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.595+0000 7fcdca05f700 1 -- 172.21.15.89:0/989442473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc40b3e90 con 0x7fcdc4159890 2023-10-10T20:36:13.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.595+0000 7fcdca05f700 1 -- 172.21.15.89:0/989442473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc40b43b0 con 0x7fcdc4159890 2023-10-10T20:36:13.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.596+0000 7fcdab7fe700 1 -- 172.21.15.89:0/989442473 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fcdb801b3b0 con 0x7fcdc4159890 2023-10-10T20:36:13.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.596+0000 7fcdab7fe700 1 --2- 172.21.15.89:0/989442473 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcdb0066fc0 0x7fcdb0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:13.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.596+0000 7fcdab7fe700 1 -- 172.21.15.89:0/989442473 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fcdb8083790 con 0x7fcdc4159890 2023-10-10T20:36:13.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.596+0000 7fcdc37fe700 1 --2- 172.21.15.89:0/989442473 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcdb0066fc0 0x7fcdb0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:13.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.597+0000 7fcdc37fe700 1 --2- 172.21.15.89:0/989442473 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcdb0066fc0 0x7fcdb0069470 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40093f0 tx=0x7fcdb4017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:13.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.600+0000 7fcdab7fe700 1 -- 172.21.15.89:0/989442473 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcdc40b43b0 con 0x7fcdc4159890 2023-10-10T20:36:13.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.784+0000 7fcdca05f700 1 -- 172.21.15.89:0/989442473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fcdc404c3c0 con 0x7fcdc4159890 2023-10-10T20:36:13.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.789+0000 7fcdab7fe700 1 -- 172.21.15.89:0/989442473 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcdb8050d10 con 0x7fcdc4159890 2023-10-10T20:36:13.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.791+0000 7fcdca05f700 1 -- 172.21.15.89:0/989442473 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcdb0066fc0 msgr2=0x7fcdb0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:13.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.791+0000 7fcdca05f700 1 --2- 172.21.15.89:0/989442473 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcdb0066fc0 0x7fcdb0069470 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40093f0 tx=0x7fcdb4017000 comp rx=0 tx=0).stop 2023-10-10T20:36:13.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.791+0000 7fcdca05f700 1 -- 172.21.15.89:0/989442473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc4159890 msgr2=0x7fcdc40b52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:13.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.791+0000 7fcdca05f700 1 --2- 172.21.15.89:0/989442473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc4159890 0x7fcdc40b52a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800c5b0 tx=0x7fcdb8001070 comp rx=0 tx=0).stop 2023-10-10T20:36:13.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.791+0000 7fcdca05f700 1 -- 172.21.15.89:0/989442473 shutdown_connections 2023-10-10T20:36:13.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.791+0000 7fcdca05f700 1 --2- 172.21.15.89:0/989442473 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcdc4159c60 0x7fcdc415c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.791+0000 7fcdca05f700 1 --2- 172.21.15.89:0/989442473 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcdb0066fc0 0x7fcdb0069470 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.791+0000 7fcdca05f700 1 --2- 172.21.15.89:0/989442473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdc4069560 0x7fcdc41524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.791+0000 7fcdca05f700 1 --2- 172.21.15.89:0/989442473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc4159890 0x7fcdc40b52a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.791+0000 7fcdca05f700 1 -- 172.21.15.89:0/989442473 >> 172.21.15.89:0/989442473 conn(0x7fcdc4068e20 msgr2=0x7fcdc4152d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:13.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.791+0000 7fcdca05f700 1 -- 172.21.15.89:0/989442473 shutdown_connections 2023-10-10T20:36:13.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.791+0000 7fcdca05f700 1 -- 172.21.15.89:0/989442473 wait complete. 2023-10-10T20:36:13.805 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info' 2023-10-10T20:36:13.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.917+0000 7fcb72647700 1 Processor -- start 2023-10-10T20:36:13.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.918+0000 7fcb72647700 1 -- start start 2023-10-10T20:36:13.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.918+0000 7fcb72647700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb6c159610 0x7fcb6c06e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:13.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.918+0000 7fcb72647700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb6c0c8010 0x7fcb6c0c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:13.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.918+0000 7fcb72647700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6c06c470 con 0x7fcb6c1599e0 2023-10-10T20:36:13.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.918+0000 7fcb72647700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb6c06b860 con 0x7fcb6c159610 2023-10-10T20:36:13.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.918+0000 7fcb72647700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c06bb40 con 0x7fcb6c0c8010 2023-10-10T20:36:13.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.918+0000 7fcb6bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb6c159610 0x7fcb6c06e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:13.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.918+0000 7fcb6b7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcb6c1599e0 0x7fcb6c06af80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42890/0 (socket says 172.21.15.89:42890) 2023-10-10T20:36:13.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.918+0000 7fcb6b7fe700 1 -- 172.21.15.89:0/1580736443 learned_addr learned my addr 172.21.15.89:0/1580736443 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:13.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.919+0000 7fcb70be4700 1 --2- 172.21.15.89:0/1580736443 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb6c0c8010 0x7fcb6c0c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:13.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.919+0000 7fcb6affd700 1 -- 172.21.15.89:0/1580736443 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1351208823 0 0) 0x7fcb6c06c470 con 0x7fcb6c1599e0 2023-10-10T20:36:13.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.919+0000 7fcb6affd700 1 -- 172.21.15.89:0/1580736443 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb54003570 con 0x7fcb6c1599e0 2023-10-10T20:36:13.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.919+0000 7fcb6bfff700 1 -- 172.21.15.89:0/1580736443 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb6c0c8010 msgr2=0x7fcb6c0c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:13.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.919+0000 7fcb6bfff700 1 --2- 172.21.15.89:0/1580736443 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb6c0c8010 0x7fcb6c0c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.919+0000 7fcb6bfff700 1 -- 172.21.15.89:0/1580736443 >> v1:172.21.15.89:6789/0 conn(0x7fcb6c1599e0 legacy=0x7fcb6c06af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:13.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.919+0000 7fcb6bfff700 1 -- 172.21.15.89:0/1580736443 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6c06be20 con 0x7fcb6c159610 2023-10-10T20:36:13.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.919+0000 7fcb6bfff700 1 --2- 172.21.15.89:0/1580736443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c159610 0x7fcb6c06e590 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0098e0 tx=0x7fcb5c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efa5eea32f2e7b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:13.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.919+0000 7fcb6affd700 1 -- 172.21.15.89:0/1580736443 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c011630 con 0x7fcb6c159610 2023-10-10T20:36:13.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.919+0000 7fcb6affd700 1 -- 172.21.15.89:0/1580736443 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5c011790 con 0x7fcb6c159610 2023-10-10T20:36:13.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.919+0000 7fcb6affd700 1 -- 172.21.15.89:0/1580736443 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c011b30 con 0x7fcb6c159610 2023-10-10T20:36:13.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.920+0000 7fcb72647700 1 -- 172.21.15.89:0/1580736443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c159610 msgr2=0x7fcb6c06e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:13.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.920+0000 7fcb72647700 1 --2- 172.21.15.89:0/1580736443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c159610 0x7fcb6c06e590 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0098e0 tx=0x7fcb5c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.920+0000 7fcb72647700 1 -- 172.21.15.89:0/1580736443 shutdown_connections 2023-10-10T20:36:13.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.920+0000 7fcb72647700 1 --2- 172.21.15.89:0/1580736443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c159610 0x7fcb6c06e590 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.920+0000 7fcb72647700 1 --2- 172.21.15.89:0/1580736443 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb6c0c8010 0x7fcb6c0c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.920+0000 7fcb72647700 1 -- 172.21.15.89:0/1580736443 >> 172.21.15.89:0/1580736443 conn(0x7fcb6c0cc780 msgr2=0x7fcb6c0ccb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:13.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.920+0000 7fcb72647700 1 -- 172.21.15.89:0/1580736443 shutdown_connections 2023-10-10T20:36:13.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.920+0000 7fcb72647700 1 -- 172.21.15.89:0/1580736443 wait complete. 2023-10-10T20:36:13.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.921+0000 7fcb72647700 1 Processor -- start 2023-10-10T20:36:13.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.921+0000 7fcb72647700 1 -- start start 2023-10-10T20:36:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb72647700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb6c0c8010 0x7fcb6c152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb72647700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb6c159610 0x7fcb6c0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb72647700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c1599e0 0x7fcb6c0b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb72647700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c1530d0 con 0x7fcb6c1599e0 2023-10-10T20:36:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb72647700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c153c50 con 0x7fcb6c0c8010 2023-10-10T20:36:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb72647700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c1540e0 con 0x7fcb6c159610 2023-10-10T20:36:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb70be4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c1599e0 0x7fcb6c0b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb70be4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c1599e0 0x7fcb6c0b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55152/0 (socket says 172.21.15.89:55152) 2023-10-10T20:36:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb70be4700 1 -- 172.21.15.89:0/1044056149 learned_addr learned my addr 172.21.15.89:0/1044056149 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb6bfff700 1 --2- 172.21.15.89:0/1044056149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb6c0c8010 0x7fcb6c152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb6b7fe700 1 --2- 172.21.15.89:0/1044056149 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb6c159610 0x7fcb6c0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb70be4700 1 -- 172.21.15.89:0/1044056149 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb6c159610 msgr2=0x7fcb6c0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb70be4700 1 --2- 172.21.15.89:0/1044056149 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb6c159610 0x7fcb6c0b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb70be4700 1 -- 172.21.15.89:0/1044056149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb6c0c8010 msgr2=0x7fcb6c152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb70be4700 1 --2- 172.21.15.89:0/1044056149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb6c0c8010 0x7fcb6c152390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb70be4700 1 -- 172.21.15.89:0/1044056149 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c009420 con 0x7fcb6c1599e0 2023-10-10T20:36:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb6b7fe700 1 --2- 172.21.15.89:0/1044056149 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb6c159610 0x7fcb6c0b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb70be4700 1 --2- 172.21.15.89:0/1044056149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c1599e0 0x7fcb6c0b2030 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009df0 tx=0x7fcb6400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:13.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb53fff700 1 -- 172.21.15.89:0/1044056149 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6400cb50 con 0x7fcb6c1599e0 2023-10-10T20:36:13.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb53fff700 1 -- 172.21.15.89:0/1044056149 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6400f040 con 0x7fcb6c1599e0 2023-10-10T20:36:13.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb53fff700 1 -- 172.21.15.89:0/1044056149 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64014480 con 0x7fcb6c1599e0 2023-10-10T20:36:13.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.922+0000 7fcb72647700 1 -- 172.21.15.89:0/1044056149 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6c153320 con 0x7fcb6c1599e0 2023-10-10T20:36:13.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.923+0000 7fcb72647700 1 -- 172.21.15.89:0/1044056149 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6c1538e0 con 0x7fcb6c1599e0 2023-10-10T20:36:13.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.924+0000 7fcb72647700 1 -- 172.21.15.89:0/1044056149 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6c153dc0 con 0x7fcb6c1599e0 2023-10-10T20:36:13.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.924+0000 7fcb53fff700 1 -- 172.21.15.89:0/1044056149 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fcb64014830 con 0x7fcb6c1599e0 2023-10-10T20:36:13.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.924+0000 7fcb53fff700 1 --2- 172.21.15.89:0/1044056149 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb58066fc0 0x7fcb58069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:13.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.924+0000 7fcb53fff700 1 -- 172.21.15.89:0/1044056149 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fcb64085060 con 0x7fcb6c1599e0 2023-10-10T20:36:13.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.924+0000 7fcb6bfff700 1 --2- 172.21.15.89:0/1044056149 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb58066fc0 0x7fcb58069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:13.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.925+0000 7fcb6bfff700 1 --2- 172.21.15.89:0/1044056149 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb58066fc0 0x7fcb58069470 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0091e0 tx=0x7fcb5c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:13.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:13.929+0000 7fcb53fff700 1 -- 172.21.15.89:0/1044056149 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb640524d0 con 0x7fcb6c1599e0 2023-10-10T20:36:14.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.190+0000 7fcb72647700 1 -- 172.21.15.89:0/1044056149 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]} v 0) v1 -- 0x7fcb6c04c3c0 con 0x7fcb6c1599e0 2023-10-10T20:36:14.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.453+0000 7fcb53fff700 1 -- 172.21.15.89:0/1044056149 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]}]=0 v111) v1 ==== 133+0+0 (secure 0 0 0) 0x7fcb64055af0 con 0x7fcb6c1599e0 2023-10-10T20:36:14.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.455+0000 7fcb72647700 1 -- 172.21.15.89:0/1044056149 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb58066fc0 msgr2=0x7fcb58069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:14.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.455+0000 7fcb72647700 1 --2- 172.21.15.89:0/1044056149 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb58066fc0 0x7fcb58069470 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0091e0 tx=0x7fcb5c017000 comp rx=0 tx=0).stop 2023-10-10T20:36:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.455+0000 7fcb72647700 1 -- 172.21.15.89:0/1044056149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c1599e0 msgr2=0x7fcb6c0b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.455+0000 7fcb72647700 1 --2- 172.21.15.89:0/1044056149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c1599e0 0x7fcb6c0b2030 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009df0 tx=0x7fcb6400b040 comp rx=0 tx=0).stop 2023-10-10T20:36:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.455+0000 7fcb72647700 1 -- 172.21.15.89:0/1044056149 shutdown_connections 2023-10-10T20:36:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.455+0000 7fcb72647700 1 --2- 172.21.15.89:0/1044056149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c1599e0 0x7fcb6c0b2030 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.455+0000 7fcb72647700 1 --2- 172.21.15.89:0/1044056149 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb58066fc0 0x7fcb58069470 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.455+0000 7fcb72647700 1 --2- 172.21.15.89:0/1044056149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb6c0c8010 0x7fcb6c152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.455+0000 7fcb72647700 1 --2- 172.21.15.89:0/1044056149 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb6c159610 0x7fcb6c0b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.455+0000 7fcb72647700 1 -- 172.21.15.89:0/1044056149 >> 172.21.15.89:0/1044056149 conn(0x7fcb6c0cc780 msgr2=0x7fcb6c158890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.455+0000 7fcb72647700 1 -- 172.21.15.89:0/1044056149 shutdown_connections 2023-10-10T20:36:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.455+0000 7fcb72647700 1 -- 172.21.15.89:0/1044056149 wait complete. 2023-10-10T20:36:14.469 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info' command ... ok 2023-10-10T20:36:14.470 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) Tests the pending_subvolume_deletions in 'fs volume info' command 2023-10-10T20:36:14.470 INFO:tasks.cephfs_test_runner:test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) 2023-10-10T20:36:14.471 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions' 2023-10-10T20:36:14.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.584+0000 7f16efa16700 1 Processor -- start 2023-10-10T20:36:14.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.584+0000 7f16efa16700 1 -- start start 2023-10-10T20:36:14.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.585+0000 7f16efa16700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f16e8159c60 0x7f16e815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:14.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.585+0000 7f16efa16700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f16e815db10 0x7f16e8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:14.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.585+0000 7f16efa16700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16e815a160 con 0x7f16e8159890 2023-10-10T20:36:14.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.585+0000 7f16efa16700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16e815df50 con 0x7f16e8159c60 2023-10-10T20:36:14.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.585+0000 7f16efa16700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e80d13a0 con 0x7f16e815db10 2023-10-10T20:36:14.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.585+0000 7f16ecfb1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f16e8159c60 0x7f16e815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:14.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.585+0000 7f16ecfb1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f16e8159c60 0x7f16e815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55160/0 (socket says 172.21.15.89:55160) 2023-10-10T20:36:14.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.585+0000 7f16ecfb1700 1 -- 172.21.15.89:0/192064233 learned_addr learned my addr 172.21.15.89:0/192064233 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:14.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.586+0000 7f16ecfb1700 1 -- 172.21.15.89:0/192064233 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f16e815db10 msgr2=0x7f16e8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:14.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.586+0000 7f16edfb3700 1 --2- 172.21.15.89:0/192064233 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f16e815db10 0x7f16e8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:14.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.586+0000 7f16ecfb1700 1 --2- 172.21.15.89:0/192064233 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f16e815db10 0x7f16e8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:14.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.586+0000 7f16ecfb1700 1 -- 172.21.15.89:0/192064233 >> v1:172.21.15.111:6789/0 conn(0x7f16e8159890 legacy=0x7f16e815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:14.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.586+0000 7f16ecfb1700 1 -- 172.21.15.89:0/192064233 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e815e140 con 0x7f16e8159c60 2023-10-10T20:36:14.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.586+0000 7f16edfb3700 1 --2- 172.21.15.89:0/192064233 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f16e815db10 0x7f16e8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:14.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.586+0000 7f16ecfb1700 1 --2- 172.21.15.89:0/192064233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16e8159c60 0x7f16e815d5d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f16d80099b0 tx=0x7f16d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3644d7da6a5befa0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:14.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.586+0000 7f16dffff700 1 -- 172.21.15.89:0/192064233 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d800b820 con 0x7f16e8159c60 2023-10-10T20:36:14.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.586+0000 7f16dffff700 1 -- 172.21.15.89:0/192064233 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d800b980 con 0x7f16e8159c60 2023-10-10T20:36:14.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.586+0000 7f16dffff700 1 -- 172.21.15.89:0/192064233 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d800bd20 con 0x7f16e8159c60 2023-10-10T20:36:14.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.586+0000 7f16efa16700 1 -- 172.21.15.89:0/192064233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16e8159c60 msgr2=0x7f16e815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:14.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.586+0000 7f16efa16700 1 --2- 172.21.15.89:0/192064233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16e8159c60 0x7f16e815d5d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f16d80099b0 tx=0x7f16d8009250 comp rx=0 tx=0).stop 2023-10-10T20:36:14.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.586+0000 7f16efa16700 1 -- 172.21.15.89:0/192064233 shutdown_connections 2023-10-10T20:36:14.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.586+0000 7f16efa16700 1 --2- 172.21.15.89:0/192064233 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f16e815db10 0x7f16e8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:14.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.586+0000 7f16efa16700 1 --2- 172.21.15.89:0/192064233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16e8159c60 0x7f16e815d5d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:14.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.586+0000 7f16efa16700 1 -- 172.21.15.89:0/192064233 >> 172.21.15.89:0/192064233 conn(0x7f16e8068e20 msgr2=0x7f16e80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:14.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.586+0000 7f16efa16700 1 -- 172.21.15.89:0/192064233 shutdown_connections 2023-10-10T20:36:14.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.586+0000 7f16efa16700 1 -- 172.21.15.89:0/192064233 wait complete. 2023-10-10T20:36:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.588+0000 7f16efa16700 1 Processor -- start 2023-10-10T20:36:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.588+0000 7f16efa16700 1 -- start start 2023-10-10T20:36:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.588+0000 7f16efa16700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16e8159890 0x7f16e80b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.588+0000 7f16efa16700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f16e8159c60 0x7f16e80cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.588+0000 7f16efa16700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16e815db10 0x7f16e80cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.588+0000 7f16efa16700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e8152440 con 0x7f16e8159890 2023-10-10T20:36:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.588+0000 7f16efa16700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e8151fb0 con 0x7f16e815db10 2023-10-10T20:36:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.588+0000 7f16efa16700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e81513a0 con 0x7f16e8159c60 2023-10-10T20:36:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.589+0000 7f16ed7b2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16e8159890 0x7f16e80b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:14.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.589+0000 7f16ed7b2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16e8159890 0x7f16e80b5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55172/0 (socket says 172.21.15.89:55172) 2023-10-10T20:36:14.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.589+0000 7f16ed7b2700 1 -- 172.21.15.89:0/337715434 learned_addr learned my addr 172.21.15.89:0/337715434 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:14.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.589+0000 7f16edfb3700 1 --2- 172.21.15.89:0/337715434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16e815db10 0x7f16e80cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:14.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.589+0000 7f16ed7b2700 1 -- 172.21.15.89:0/337715434 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f16e8159c60 msgr2=0x7f16e80cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:14.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.589+0000 7f16ed7b2700 1 --2- 172.21.15.89:0/337715434 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f16e8159c60 0x7f16e80cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:14.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.589+0000 7f16ed7b2700 1 -- 172.21.15.89:0/337715434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16e815db10 msgr2=0x7f16e80cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:14.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.589+0000 7f16ed7b2700 1 --2- 172.21.15.89:0/337715434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16e815db10 0x7f16e80cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:14.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.589+0000 7f16ed7b2700 1 -- 172.21.15.89:0/337715434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d80094f0 con 0x7f16e8159890 2023-10-10T20:36:14.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.589+0000 7f16edfb3700 1 --2- 172.21.15.89:0/337715434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16e815db10 0x7f16e80cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:14.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.589+0000 7f16ed7b2700 1 --2- 172.21.15.89:0/337715434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16e8159890 0x7f16e80b5d70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f16e400c5d0 tx=0x7f16e4007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:14.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.589+0000 7f16dd7fa700 1 -- 172.21.15.89:0/337715434 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e4011590 con 0x7f16e8159890 2023-10-10T20:36:14.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.589+0000 7f16dd7fa700 1 -- 172.21.15.89:0/337715434 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e4011710 con 0x7f16e8159890 2023-10-10T20:36:14.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.589+0000 7f16efa16700 1 -- 172.21.15.89:0/337715434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e8151680 con 0x7f16e8159890 2023-10-10T20:36:14.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.589+0000 7f16dd7fa700 1 -- 172.21.15.89:0/337715434 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e4011ad0 con 0x7f16e8159890 2023-10-10T20:36:14.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.589+0000 7f16efa16700 1 -- 172.21.15.89:0/337715434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e8151c40 con 0x7f16e8159890 2023-10-10T20:36:14.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.590+0000 7f16efa16700 1 -- 172.21.15.89:0/337715434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16e804c3c0 con 0x7f16e8159890 2023-10-10T20:36:14.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.590+0000 7f16dd7fa700 1 -- 172.21.15.89:0/337715434 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f16e401c460 con 0x7f16e8159890 2023-10-10T20:36:14.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.591+0000 7f16dd7fa700 1 --2- 172.21.15.89:0/337715434 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f16cc066fc0 0x7f16cc069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:14.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.591+0000 7f16dd7fa700 1 -- 172.21.15.89:0/337715434 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f16e4083fb0 con 0x7f16e8159890 2023-10-10T20:36:14.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.591+0000 7f16ecfb1700 1 --2- 172.21.15.89:0/337715434 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f16cc066fc0 0x7f16cc069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:14.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.591+0000 7f16ecfb1700 1 --2- 172.21.15.89:0/337715434 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f16cc066fc0 0x7f16cc069470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f16d80099b0 tx=0x7f16d8016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:14.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.595+0000 7f16dd7fa700 1 -- 172.21.15.89:0/337715434 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16e4051530 con 0x7f16e8159890 2023-10-10T20:36:14.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:14.852+0000 7f16efa16700 1 -- 172.21.15.89:0/337715434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]} v 0) v1 -- 0x7f16e806a5b0 con 0x7f16e8159890 2023-10-10T20:36:15.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.459+0000 7f16dd7fa700 1 -- 172.21.15.89:0/337715434 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]}]=0 v112) v1 ==== 161+0+0 (secure 0 0 0) 0x7f16e4054b50 con 0x7f16e8159890 2023-10-10T20:36:15.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.461+0000 7f16efa16700 1 -- 172.21.15.89:0/337715434 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f16cc066fc0 msgr2=0x7f16cc069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:15.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.461+0000 7f16efa16700 1 --2- 172.21.15.89:0/337715434 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f16cc066fc0 0x7f16cc069470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f16d80099b0 tx=0x7f16d8016000 comp rx=0 tx=0).stop 2023-10-10T20:36:15.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.461+0000 7f16efa16700 1 -- 172.21.15.89:0/337715434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16e8159890 msgr2=0x7f16e80b5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:15.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.461+0000 7f16efa16700 1 --2- 172.21.15.89:0/337715434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16e8159890 0x7f16e80b5d70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f16e400c5d0 tx=0x7f16e4007f60 comp rx=0 tx=0).stop 2023-10-10T20:36:15.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.461+0000 7f16efa16700 1 -- 172.21.15.89:0/337715434 shutdown_connections 2023-10-10T20:36:15.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.461+0000 7f16efa16700 1 --2- 172.21.15.89:0/337715434 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f16cc066fc0 0x7f16cc069470 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:15.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.461+0000 7f16efa16700 1 --2- 172.21.15.89:0/337715434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16e815db10 0x7f16e80cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.461+0000 7f16efa16700 1 --2- 172.21.15.89:0/337715434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16e8159890 0x7f16e80b5d70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.461+0000 7f16efa16700 1 --2- 172.21.15.89:0/337715434 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f16e8159c60 0x7f16e80cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.461+0000 7f16efa16700 1 -- 172.21.15.89:0/337715434 >> 172.21.15.89:0/337715434 conn(0x7f16e8068e20 msgr2=0x7f16e80cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.461+0000 7f16efa16700 1 -- 172.21.15.89:0/337715434 shutdown_connections 2023-10-10T20:36:15.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.461+0000 7f16efa16700 1 -- 172.21.15.89:0/337715434 wait complete. 2023-10-10T20:36:15.476 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:36:15.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.590+0000 7f9b03c0d700 1 Processor -- start 2023-10-10T20:36:15.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.590+0000 7f9b03c0d700 1 -- start start 2023-10-10T20:36:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.591+0000 7f9b03c0d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9afc159890 0x7f9afc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.591+0000 7f9b03c0d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9afc069560 0x7f9afc0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.591+0000 7f9b03c0d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9afc069ee0 con 0x7f9afc159c60 2023-10-10T20:36:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.591+0000 7f9b03c0d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9afc15a160 con 0x7f9afc159890 2023-10-10T20:36:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.591+0000 7f9b03c0d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9afc0d13a0 con 0x7f9afc069560 2023-10-10T20:36:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.591+0000 7f9b019a9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9afc159890 0x7f9afc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.591+0000 7f9b019a9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9afc159890 0x7f9afc15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55174/0 (socket says 172.21.15.89:55174) 2023-10-10T20:36:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.591+0000 7f9b019a9700 1 -- 172.21.15.89:0/1056118410 learned_addr learned my addr 172.21.15.89:0/1056118410 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.591+0000 7f9b021aa700 1 --2- 172.21.15.89:0/1056118410 >> v2:172.21.15.111:3300/0 conn(0x7f9afc069560 0x7f9afc0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.592+0000 7f9b019a9700 1 -- 172.21.15.89:0/1056118410 >> v2:172.21.15.111:3300/0 conn(0x7f9afc069560 msgr2=0x7f9afc0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.592+0000 7f9b019a9700 1 --2- 172.21.15.89:0/1056118410 >> v2:172.21.15.111:3300/0 conn(0x7f9afc069560 0x7f9afc0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.592+0000 7f9b019a9700 1 -- 172.21.15.89:0/1056118410 >> v1:172.21.15.111:6789/0 conn(0x7f9afc159c60 legacy=0x7f9afc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:15.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.592+0000 7f9b019a9700 1 -- 172.21.15.89:0/1056118410 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc15e140 con 0x7f9afc159890 2023-10-10T20:36:15.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.592+0000 7f9b021aa700 1 --2- 172.21.15.89:0/1056118410 >> v2:172.21.15.111:3300/0 conn(0x7f9afc069560 0x7f9afc0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:15.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.592+0000 7f9b019a9700 1 --2- 172.21.15.89:0/1056118410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9afc159890 0x7f9afc15abd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9afc06a890 tx=0x7f9af8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359f712d1701784f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:15.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.592+0000 7f9b009a7700 1 -- 172.21.15.89:0/1056118410 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af800e070 con 0x7f9afc159890 2023-10-10T20:36:15.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.592+0000 7f9b009a7700 1 -- 172.21.15.89:0/1056118410 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af800ba10 con 0x7f9afc159890 2023-10-10T20:36:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.592+0000 7f9b009a7700 1 -- 172.21.15.89:0/1056118410 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80092e0 con 0x7f9afc159890 2023-10-10T20:36:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.592+0000 7f9b03c0d700 1 -- 172.21.15.89:0/1056118410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9afc159890 msgr2=0x7f9afc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.592+0000 7f9b03c0d700 1 --2- 172.21.15.89:0/1056118410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9afc159890 0x7f9afc15abd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9afc06a890 tx=0x7f9af8009840 comp rx=0 tx=0).stop 2023-10-10T20:36:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.592+0000 7f9b03c0d700 1 -- 172.21.15.89:0/1056118410 shutdown_connections 2023-10-10T20:36:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.592+0000 7f9b03c0d700 1 --2- 172.21.15.89:0/1056118410 >> v2:172.21.15.111:3300/0 conn(0x7f9afc069560 0x7f9afc0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:15.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.592+0000 7f9b03c0d700 1 --2- 172.21.15.89:0/1056118410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9afc159890 0x7f9afc15abd0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:15.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.592+0000 7f9b03c0d700 1 -- 172.21.15.89:0/1056118410 >> 172.21.15.89:0/1056118410 conn(0x7f9afc068e20 msgr2=0x7f9afc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:15.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.592+0000 7f9b03c0d700 1 -- 172.21.15.89:0/1056118410 shutdown_connections 2023-10-10T20:36:15.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.592+0000 7f9b03c0d700 1 -- 172.21.15.89:0/1056118410 wait complete. 2023-10-10T20:36:15.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.593+0000 7f9b03c0d700 1 Processor -- start 2023-10-10T20:36:15.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.593+0000 7f9b03c0d700 1 -- start start 2023-10-10T20:36:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.594+0000 7f9b03c0d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9afc069560 0x7f9afc157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.594+0000 7f9b03c0d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9afc159890 0x7f9afc1450c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.594+0000 7f9b03c0d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9afc159c60 0x7f9afc0b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.594+0000 7f9b03c0d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc156a80 con 0x7f9afc159890 2023-10-10T20:36:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.594+0000 7f9b03c0d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc156d60 con 0x7f9afc069560 2023-10-10T20:36:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.594+0000 7f9b03c0d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc157040 con 0x7f9afc159c60 2023-10-10T20:36:15.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.594+0000 7f9b011a8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9afc159890 0x7f9afc1450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:15.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.594+0000 7f9b011a8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9afc159890 0x7f9afc1450c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55178/0 (socket says 172.21.15.89:55178) 2023-10-10T20:36:15.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.594+0000 7f9b011a8700 1 -- 172.21.15.89:0/4049470597 learned_addr learned my addr 172.21.15.89:0/4049470597 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:15.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.595+0000 7f9b011a8700 1 -- 172.21.15.89:0/4049470597 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9afc159c60 msgr2=0x7f9afc0b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:15.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.595+0000 7f9b019a9700 1 --2- 172.21.15.89:0/4049470597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9afc069560 0x7f9afc157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:15.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.595+0000 7f9b011a8700 1 --2- 172.21.15.89:0/4049470597 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9afc159c60 0x7f9afc0b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:15.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.595+0000 7f9b011a8700 1 -- 172.21.15.89:0/4049470597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9afc069560 msgr2=0x7f9afc157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:15.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.595+0000 7f9b011a8700 1 --2- 172.21.15.89:0/4049470597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9afc069560 0x7f9afc157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:15.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.595+0000 7f9b011a8700 1 -- 172.21.15.89:0/4049470597 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af8009ae0 con 0x7f9afc159890 2023-10-10T20:36:15.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.595+0000 7f9b019a9700 1 --2- 172.21.15.89:0/4049470597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9afc069560 0x7f9afc157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:15.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.595+0000 7f9b011a8700 1 --2- 172.21.15.89:0/4049470597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9afc159890 0x7f9afc1450c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00b5e0 tx=0x7f9aec009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:15.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.595+0000 7f9af1ffb700 1 -- 172.21.15.89:0/4049470597 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec00f510 con 0x7f9afc159890 2023-10-10T20:36:15.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.595+0000 7f9af1ffb700 1 -- 172.21.15.89:0/4049470597 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec00f670 con 0x7f9afc159890 2023-10-10T20:36:15.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.595+0000 7f9b03c0d700 1 -- 172.21.15.89:0/4049470597 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc157320 con 0x7f9afc159890 2023-10-10T20:36:15.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.595+0000 7f9af1ffb700 1 -- 172.21.15.89:0/4049470597 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec00fa30 con 0x7f9afc159890 2023-10-10T20:36:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.595+0000 7f9b03c0d700 1 -- 172.21.15.89:0/4049470597 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc145600 con 0x7f9afc159890 2023-10-10T20:36:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.596+0000 7f9b03c0d700 1 -- 172.21.15.89:0/4049470597 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc157840 con 0x7f9afc159890 2023-10-10T20:36:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.596+0000 7f9af1ffb700 1 -- 172.21.15.89:0/4049470597 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f9aec01a400 con 0x7f9afc159890 2023-10-10T20:36:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.597+0000 7f9af1ffb700 1 --2- 172.21.15.89:0/4049470597 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ae0066fc0 0x7f9ae0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.597+0000 7f9af1ffb700 1 -- 172.21.15.89:0/4049470597 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f9aec083e80 con 0x7f9afc159890 2023-10-10T20:36:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.597+0000 7f9b019a9700 1 --2- 172.21.15.89:0/4049470597 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ae0066fc0 0x7f9ae0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.597+0000 7f9b019a9700 1 --2- 172.21.15.89:0/4049470597 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ae0066fc0 0x7f9ae0069470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009ab0 tx=0x7f9af8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.601+0000 7f9af1ffb700 1 -- 172.21.15.89:0/4049470597 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9afc157840 con 0x7f9afc159890 2023-10-10T20:36:15.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.784+0000 7f9b03c0d700 1 -- 172.21.15.89:0/4049470597 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9afc157840 con 0x7f9afc159890 2023-10-10T20:36:15.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.789+0000 7f9af1ffb700 1 -- 172.21.15.89:0/4049470597 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9afc157840 con 0x7f9afc159890 2023-10-10T20:36:15.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.791+0000 7f9b03c0d700 1 -- 172.21.15.89:0/4049470597 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ae0066fc0 msgr2=0x7f9ae0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:15.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.791+0000 7f9b03c0d700 1 --2- 172.21.15.89:0/4049470597 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ae0066fc0 0x7f9ae0069470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009ab0 tx=0x7f9af8017000 comp rx=0 tx=0).stop 2023-10-10T20:36:15.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.791+0000 7f9b03c0d700 1 -- 172.21.15.89:0/4049470597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9afc159890 msgr2=0x7f9afc1450c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:15.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.791+0000 7f9b03c0d700 1 --2- 172.21.15.89:0/4049470597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9afc159890 0x7f9afc1450c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00b5e0 tx=0x7f9aec009120 comp rx=0 tx=0).stop 2023-10-10T20:36:15.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.791+0000 7f9b03c0d700 1 -- 172.21.15.89:0/4049470597 shutdown_connections 2023-10-10T20:36:15.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.791+0000 7f9b03c0d700 1 --2- 172.21.15.89:0/4049470597 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9afc159c60 0x7f9afc0b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:15.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.791+0000 7f9b03c0d700 1 --2- 172.21.15.89:0/4049470597 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ae0066fc0 0x7f9ae0069470 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:15.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.791+0000 7f9b03c0d700 1 --2- 172.21.15.89:0/4049470597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9afc069560 0x7f9afc157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:15.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.791+0000 7f9b03c0d700 1 --2- 172.21.15.89:0/4049470597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9afc159890 0x7f9afc1450c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:15.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.791+0000 7f9b03c0d700 1 -- 172.21.15.89:0/4049470597 >> 172.21.15.89:0/4049470597 conn(0x7f9afc068e20 msgr2=0x7f9afc0d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:15.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.791+0000 7f9b03c0d700 1 -- 172.21.15.89:0/4049470597 shutdown_connections 2023-10-10T20:36:15.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:15.791+0000 7f9b03c0d700 1 -- 172.21.15.89:0/4049470597 wait complete. 2023-10-10T20:36:15.806 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:36:15.806 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:36:15.851 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:36:15.891 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:36:15.891 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:36:15.891 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:36:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.003+0000 7f06b54b3700 1 Processor -- start 2023-10-10T20:36:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.003+0000 7f06b54b3700 1 -- start start 2023-10-10T20:36:16.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.004+0000 7f06b54b3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f06b006bcd0 0x7f06b00caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:16.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.004+0000 7f06b54b3700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f06b006c0a0 0x7f06b015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:16.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.004+0000 7f06b54b3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b015d7e0 con 0x7f06b015df50 2023-10-10T20:36:16.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.004+0000 7f06b54b3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06b015dac0 con 0x7f06b006bcd0 2023-10-10T20:36:16.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.004+0000 7f06b54b3700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06b015aa30 con 0x7f06b006c0a0 2023-10-10T20:36:16.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.004+0000 7f06aeffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f06b006bcd0 0x7f06b00caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:16.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.004+0000 7f06af7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f06b015df50 0x7f06b00c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42894/0 (socket says 172.21.15.89:42894) 2023-10-10T20:36:16.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.004+0000 7f06af7fe700 1 -- 172.21.15.89:0/635732630 learned_addr learned my addr 172.21.15.89:0/635732630 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:16.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.005+0000 7f06ae7fc700 1 -- 172.21.15.89:0/635732630 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2767387901 0 0) 0x7f06b015d7e0 con 0x7f06b015df50 2023-10-10T20:36:16.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.005+0000 7f06ae7fc700 1 -- 172.21.15.89:0/635732630 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f069c003570 con 0x7f06b015df50 2023-10-10T20:36:16.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.005+0000 7f06a67fc700 1 --2- 172.21.15.89:0/635732630 >> v2:172.21.15.111:3300/0 conn(0x7f06b006c0a0 0x7f06b015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.005+0000 7f06aeffd700 1 -- 172.21.15.89:0/635732630 >> v2:172.21.15.111:3300/0 conn(0x7f06b006c0a0 msgr2=0x7f06b015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.005+0000 7f06aeffd700 1 --2- 172.21.15.89:0/635732630 >> v2:172.21.15.111:3300/0 conn(0x7f06b006c0a0 0x7f06b015a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.005+0000 7f06aeffd700 1 -- 172.21.15.89:0/635732630 >> v1:172.21.15.89:6789/0 conn(0x7f06b015df50 legacy=0x7f06b00c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.005+0000 7f06aeffd700 1 -- 172.21.15.89:0/635732630 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b015ad10 con 0x7f06b006bcd0 2023-10-10T20:36:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.005+0000 7f06a67fc700 1 --2- 172.21.15.89:0/635732630 >> v2:172.21.15.111:3300/0 conn(0x7f06b006c0a0 0x7f06b015a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.005+0000 7f06aeffd700 1 --2- 172.21.15.89:0/635732630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06b006bcd0 0x7f06b00caa90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f06b006b4e0 tx=0x7f06a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11f5f2e80c9b5f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.005+0000 7f06ae7fc700 1 -- 172.21.15.89:0/635732630 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0011630 con 0x7f06b006bcd0 2023-10-10T20:36:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.005+0000 7f06ae7fc700 1 -- 172.21.15.89:0/635732630 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a0011790 con 0x7f06b006bcd0 2023-10-10T20:36:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.005+0000 7f06ae7fc700 1 -- 172.21.15.89:0/635732630 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0011b30 con 0x7f06b006bcd0 2023-10-10T20:36:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.006+0000 7f06b54b3700 1 -- 172.21.15.89:0/635732630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06b006bcd0 msgr2=0x7f06b00caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:16.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.006+0000 7f06b54b3700 1 --2- 172.21.15.89:0/635732630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06b006bcd0 0x7f06b00caa90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f06b006b4e0 tx=0x7f06a00091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:16.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.006+0000 7f06b54b3700 1 -- 172.21.15.89:0/635732630 shutdown_connections 2023-10-10T20:36:16.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.006+0000 7f06b54b3700 1 --2- 172.21.15.89:0/635732630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06b006bcd0 0x7f06b00caa90 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:16.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.006+0000 7f06b54b3700 1 --2- 172.21.15.89:0/635732630 >> v2:172.21.15.111:3300/0 conn(0x7f06b006c0a0 0x7f06b015a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:16.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.006+0000 7f06b54b3700 1 -- 172.21.15.89:0/635732630 >> 172.21.15.89:0/635732630 conn(0x7f06b00ce600 msgr2=0x7f06b00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:16.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.006+0000 7f06b54b3700 1 -- 172.21.15.89:0/635732630 shutdown_connections 2023-10-10T20:36:16.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.006+0000 7f06b54b3700 1 -- 172.21.15.89:0/635732630 wait complete. 2023-10-10T20:36:16.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.007+0000 7f06b54b3700 1 Processor -- start 2023-10-10T20:36:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.007+0000 7f06b54b3700 1 -- start start 2023-10-10T20:36:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06b54b3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06b006bcd0 0x7f06b016d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06b54b3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06b006c0a0 0x7f06b0147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06b54b3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06b015df50 0x7f06b0153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06b54b3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b00b2a70 con 0x7f06b015df50 2023-10-10T20:36:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06b54b3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b00b2740 con 0x7f06b006bcd0 2023-10-10T20:36:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06b54b3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b016df30 con 0x7f06b006c0a0 2023-10-10T20:36:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06af7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06b015df50 0x7f06b0153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06af7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06b015df50 0x7f06b0153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55200/0 (socket says 172.21.15.89:55200) 2023-10-10T20:36:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06af7fe700 1 -- 172.21.15.89:0/1976969039 learned_addr learned my addr 172.21.15.89:0/1976969039 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06aeffd700 1 --2- 172.21.15.89:0/1976969039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06b006bcd0 0x7f06b016d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06af7fe700 1 -- 172.21.15.89:0/1976969039 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06b006c0a0 msgr2=0x7f06b0147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06af7fe700 1 --2- 172.21.15.89:0/1976969039 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06b006c0a0 0x7f06b0147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06af7fe700 1 -- 172.21.15.89:0/1976969039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06b006bcd0 msgr2=0x7f06b016d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06af7fe700 1 --2- 172.21.15.89:0/1976969039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06b006bcd0 0x7f06b016d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06af7fe700 1 -- 172.21.15.89:0/1976969039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a0009420 con 0x7f06b015df50 2023-10-10T20:36:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06af7fe700 1 --2- 172.21.15.89:0/1976969039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06b015df50 0x7f06b0153a30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f06a8001460 tx=0x7f06a8001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06a77fe700 1 -- 172.21.15.89:0/1976969039 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a800ce80 con 0x7f06b015df50 2023-10-10T20:36:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06a77fe700 1 -- 172.21.15.89:0/1976969039 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a800ab70 con 0x7f06b015df50 2023-10-10T20:36:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06b54b3700 1 -- 172.21.15.89:0/1976969039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b016e210 con 0x7f06b015df50 2023-10-10T20:36:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06a77fe700 1 -- 172.21.15.89:0/1976969039 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a80070f0 con 0x7f06b015df50 2023-10-10T20:36:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.008+0000 7f06b54b3700 1 -- 172.21.15.89:0/1976969039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b016db70 con 0x7f06b015df50 2023-10-10T20:36:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.009+0000 7f06b54b3700 1 -- 172.21.15.89:0/1976969039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b004c3c0 con 0x7f06b015df50 2023-10-10T20:36:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.009+0000 7f06a77fe700 1 -- 172.21.15.89:0/1976969039 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f06a8007520 con 0x7f06b015df50 2023-10-10T20:36:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.010+0000 7f06a77fe700 1 --2- 172.21.15.89:0/1976969039 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0690066fc0 0x7f0690069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.010+0000 7f06a77fe700 1 -- 172.21.15.89:0/1976969039 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f06a80846e0 con 0x7f06b015df50 2023-10-10T20:36:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.010+0000 7f06aeffd700 1 --2- 172.21.15.89:0/1976969039 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0690066fc0 0x7f0690069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.010+0000 7f06aeffd700 1 --2- 172.21.15.89:0/1976969039 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0690066fc0 0x7f0690069470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f06a0003080 tx=0x7f06a0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.014+0000 7f06a77fe700 1 -- 172.21.15.89:0/1976969039 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06a8051ce0 con 0x7f06b015df50 2023-10-10T20:36:16.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.240+0000 7f06b54b3700 1 -- 172.21.15.89:0/1976969039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06b00c9d10 con 0x7f06b015df50 2023-10-10T20:36:16.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.240+0000 7f06a77fe700 1 -- 172.21.15.89:0/1976969039 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f06a8055300 con 0x7f06b015df50 2023-10-10T20:36:16.243 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:16.243 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":32,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4922,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1464385125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1464385125},{"type":"v1","addr":"172.21.15.111:6834","nonce":1464385125}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":4925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/689169706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":689169706},{"type":"v1","addr":"172.21.15.111:6835","nonce":689169706}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/418769006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":418769006},{"type":"v1","addr":"172.21.15.89:6836","nonce":418769006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/619682886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":619682886},{"type":"v1","addr":"172.21.15.89:6837","nonce":619682886}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[]} 2023-10-10T20:36:16.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.243+0000 7f06b54b3700 1 -- 172.21.15.89:0/1976969039 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0690066fc0 msgr2=0x7f0690069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:16.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.243+0000 7f06b54b3700 1 --2- 172.21.15.89:0/1976969039 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0690066fc0 0x7f0690069470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f06a0003080 tx=0x7f06a0017000 comp rx=0 tx=0).stop 2023-10-10T20:36:16.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.243+0000 7f06b54b3700 1 -- 172.21.15.89:0/1976969039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06b015df50 msgr2=0x7f06b0153a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:16.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.243+0000 7f06b54b3700 1 --2- 172.21.15.89:0/1976969039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06b015df50 0x7f06b0153a30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f06a8001460 tx=0x7f06a8001ae0 comp rx=0 tx=0).stop 2023-10-10T20:36:16.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.243+0000 7f06b54b3700 1 -- 172.21.15.89:0/1976969039 shutdown_connections 2023-10-10T20:36:16.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.243+0000 7f06b54b3700 1 --2- 172.21.15.89:0/1976969039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06b015df50 0x7f06b0153a30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:16.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.243+0000 7f06b54b3700 1 --2- 172.21.15.89:0/1976969039 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0690066fc0 0x7f0690069470 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:16.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.243+0000 7f06b54b3700 1 --2- 172.21.15.89:0/1976969039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06b006bcd0 0x7f06b016d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:16.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.243+0000 7f06b54b3700 1 --2- 172.21.15.89:0/1976969039 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06b006c0a0 0x7f06b0147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:16.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.243+0000 7f06b54b3700 1 -- 172.21.15.89:0/1976969039 >> 172.21.15.89:0/1976969039 conn(0x7f06b00ce600 msgr2=0x7f06b006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:16.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.243+0000 7f06b54b3700 1 -- 172.21.15.89:0/1976969039 shutdown_connections 2023-10-10T20:36:16.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.243+0000 7f06b54b3700 1 -- 172.21.15.89:0/1976969039 wait complete. 2023-10-10T20:36:16.248 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 32 2023-10-10T20:36:16.257 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:36:16.257 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:36:16.258 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:36:16.258 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:36:16.258 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:36:16.258 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:36:16.259 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:36:16.259 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:36:16.259 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:36:16.260 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:36:16.260 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:36:16.260 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:36:16.260 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:36:16.261 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:36:16.261 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:36:16.261 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:36:16.267 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:36:16.267 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:36:16.268 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:36:16.269 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:36:16.271 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:36:16.272 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:36:16.273 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:36:16.273 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:36:16.310 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:36:16.310 DEBUG:teuthology.parallel:result is None 2023-10-10T20:36:16.311 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:36:16.312 DEBUG:teuthology.parallel:result is None 2023-10-10T20:36:16.315 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:36:16.315 DEBUG:teuthology.parallel:result is None 2023-10-10T20:36:16.316 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:36:16.316 DEBUG:teuthology.parallel:result is None 2023-10-10T20:36:16.317 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:36:16.383 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-10T20:36:16.383 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:36:16.396 INFO:tasks.ceph.mds.d.smithi111.stdout:starting mds.d at 2023-10-10T20:36:16.396 INFO:tasks.ceph.mds.b.smithi111.stdout:starting mds.b at 2023-10-10T20:36:16.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.437+0000 7f683f59e700 1 Processor -- start 2023-10-10T20:36:16.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.437+0000 7f683f59e700 1 -- start start 2023-10-10T20:36:16.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.438+0000 7f683f59e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f68400c8830 0x7f6840164790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:16.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.438+0000 7f683f59e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684015aa30 con 0x7f6840153df0 2023-10-10T20:36:16.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.438+0000 7f683f59e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684015a100 con 0x7f684006e8c0 2023-10-10T20:36:16.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.438+0000 7f683f59e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684015a3e0 con 0x7f68400c8830 2023-10-10T20:36:16.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.438+0000 7f683ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f68400c8830 0x7f6840164790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:16.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.438+0000 7f683e59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6840153df0 0x7f6840063230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42898/0 (socket says 172.21.15.89:42898) 2023-10-10T20:36:16.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.438+0000 7f683e59c700 1 -- 172.21.15.89:0/1112585804 learned_addr learned my addr 172.21.15.89:0/1112585804 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:16.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.438+0000 7f683ed9d700 1 -- 172.21.15.89:0/1112585804 >> v1:172.21.15.111:6789/0 conn(0x7f684006e8c0 legacy=0x7f68400c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:16.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.438+0000 7f683ed9d700 1 -- 172.21.15.89:0/1112585804 >> v1:172.21.15.89:6789/0 conn(0x7f6840153df0 legacy=0x7f6840063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:16.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.439+0000 7f683ed9d700 1 -- 172.21.15.89:0/1112585804 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684015a6c0 con 0x7f68400c8830 2023-10-10T20:36:16.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.439+0000 7f683dd9b700 1 -- 172.21.15.89:0/1112585804 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612271080 0 0) 0x7f684015aa30 con 0x7f6840153df0 2023-10-10T20:36:16.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.439+0000 7f683ed9d700 1 --2- 172.21.15.89:0/1112585804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68400c8830 0x7f6840164790 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6838009e60 tx=0x7f6838009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f69f68acde3e8a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:16.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.439+0000 7f683dd9b700 1 -- 172.21.15.89:0/1112585804 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683800e070 con 0x7f68400c8830 2023-10-10T20:36:16.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.439+0000 7f683dd9b700 1 -- 172.21.15.89:0/1112585804 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683800bbf0 con 0x7f68400c8830 2023-10-10T20:36:16.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.439+0000 7f683dd9b700 1 -- 172.21.15.89:0/1112585804 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838012430 con 0x7f68400c8830 2023-10-10T20:36:16.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.439+0000 7f683f59e700 1 -- 172.21.15.89:0/1112585804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68400c8830 msgr2=0x7f6840164790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:16.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.439+0000 7f683f59e700 1 --2- 172.21.15.89:0/1112585804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68400c8830 0x7f6840164790 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6838009e60 tx=0x7f6838009780 comp rx=0 tx=0).stop 2023-10-10T20:36:16.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.439+0000 7f683f59e700 1 -- 172.21.15.89:0/1112585804 shutdown_connections 2023-10-10T20:36:16.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.439+0000 7f683f59e700 1 --2- 172.21.15.89:0/1112585804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68400c8830 0x7f6840164790 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:16.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.439+0000 7f683f59e700 1 -- 172.21.15.89:0/1112585804 >> 172.21.15.89:0/1112585804 conn(0x7f684006ddb0 msgr2=0x7f684006e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:16.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.439+0000 7f683f59e700 1 -- 172.21.15.89:0/1112585804 shutdown_connections 2023-10-10T20:36:16.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.439+0000 7f683f59e700 1 -- 172.21.15.89:0/1112585804 wait complete. 2023-10-10T20:36:16.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.441+0000 7f683f59e700 1 Processor -- start 2023-10-10T20:36:16.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.441+0000 7f683f59e700 1 -- start start 2023-10-10T20:36:16.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.441+0000 7f683f59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f684006e8c0 0x7f68401390d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:16.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.441+0000 7f683f59e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68400c8830 0x7f6840149d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:16.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.441+0000 7f683f59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6840153df0 0x7f68400cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:16.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.441+0000 7f683f59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6840139610 con 0x7f6840153df0 2023-10-10T20:36:16.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.441+0000 7f683f59e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684014b210 con 0x7f68400c8830 2023-10-10T20:36:16.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.441+0000 7f683f59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684014a600 con 0x7f684006e8c0 2023-10-10T20:36:16.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.441+0000 7f683ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6840153df0 0x7f68400cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:16.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.441+0000 7f683ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6840153df0 0x7f68400cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55254/0 (socket says 172.21.15.89:55254) 2023-10-10T20:36:16.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.441+0000 7f6835d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68400c8830 0x7f6840149d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:16.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.441+0000 7f683ed9d700 1 -- 172.21.15.89:0/2936486665 learned_addr learned my addr 172.21.15.89:0/2936486665 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:16.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.442+0000 7f683e59c700 1 --2- 172.21.15.89:0/2936486665 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f684006e8c0 0x7f68401390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:16.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.442+0000 7f683ed9d700 1 -- 172.21.15.89:0/2936486665 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f684006e8c0 msgr2=0x7f68401390d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:16.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.442+0000 7f683ed9d700 1 --2- 172.21.15.89:0/2936486665 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f684006e8c0 0x7f68401390d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:16.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.442+0000 7f683ed9d700 1 -- 172.21.15.89:0/2936486665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68400c8830 msgr2=0x7f6840149d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:16.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.442+0000 7f683ed9d700 1 --2- 172.21.15.89:0/2936486665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68400c8830 0x7f6840149d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:16.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.442+0000 7f683ed9d700 1 -- 172.21.15.89:0/2936486665 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6838009a20 con 0x7f6840153df0 2023-10-10T20:36:16.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.442+0000 7f6835d9b700 1 --2- 172.21.15.89:0/2936486665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68400c8830 0x7f6840149d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:36:16.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.442+0000 7f683ed9d700 1 --2- 172.21.15.89:0/2936486665 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6840153df0 0x7f68400cb800 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6838003000 tx=0x7f6838018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:16.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.442+0000 7f6836ffd700 1 -- 172.21.15.89:0/2936486665 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683800bde0 con 0x7f6840153df0 2023-10-10T20:36:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.442+0000 7f6836ffd700 1 -- 172.21.15.89:0/2936486665 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683801e640 con 0x7f6840153df0 2023-10-10T20:36:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.442+0000 7f683f59e700 1 -- 172.21.15.89:0/2936486665 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684014a8e0 con 0x7f6840153df0 2023-10-10T20:36:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.442+0000 7f6836ffd700 1 -- 172.21.15.89:0/2936486665 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683801e9e0 con 0x7f6840153df0 2023-10-10T20:36:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.442+0000 7f683f59e700 1 -- 172.21.15.89:0/2936486665 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684014aea0 con 0x7f6840153df0 2023-10-10T20:36:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.443+0000 7f683f59e700 1 -- 172.21.15.89:0/2936486665 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684014b380 con 0x7f6840153df0 2023-10-10T20:36:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.443+0000 7f6836ffd700 1 -- 172.21.15.89:0/2936486665 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f6838024070 con 0x7f6840153df0 2023-10-10T20:36:16.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.444+0000 7f6836ffd700 1 --2- 172.21.15.89:0/2936486665 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6820066fc0 0x7f6820069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:16.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.444+0000 7f6836ffd700 1 -- 172.21.15.89:0/2936486665 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6838086b80 con 0x7f6840153df0 2023-10-10T20:36:16.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.444+0000 7f683e59c700 1 --2- 172.21.15.89:0/2936486665 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6820066fc0 0x7f6820069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:16.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.444+0000 7f683e59c700 1 --2- 172.21.15.89:0/2936486665 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6820066fc0 0x7f6820069470 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6830007160 tx=0x7f6830007f50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:16.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.448+0000 7f6836ffd700 1 -- 172.21.15.89:0/2936486665 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6838054100 con 0x7f6840153df0 2023-10-10T20:36:16.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:16.646+0000 7f683f59e700 1 -- 172.21.15.89:0/2936486665 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f684004c3c0 con 0x7f6840153df0 2023-10-10T20:36:17.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.476+0000 7f6836ffd700 1 -- 172.21.15.89:0/2936486665 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v58) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6838057720 con 0x7f6840153df0 2023-10-10T20:36:17.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.478+0000 7f683f59e700 1 -- 172.21.15.89:0/2936486665 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6820066fc0 msgr2=0x7f6820069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:17.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.478+0000 7f683f59e700 1 --2- 172.21.15.89:0/2936486665 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6820066fc0 0x7f6820069470 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6830007160 tx=0x7f6830007f50 comp rx=0 tx=0).stop 2023-10-10T20:36:17.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.478+0000 7f683f59e700 1 -- 172.21.15.89:0/2936486665 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6840153df0 msgr2=0x7f68400cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:17.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.478+0000 7f683f59e700 1 --2- 172.21.15.89:0/2936486665 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6840153df0 0x7f68400cb800 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6838003000 tx=0x7f6838018000 comp rx=0 tx=0).stop 2023-10-10T20:36:17.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.479+0000 7f683f59e700 1 -- 172.21.15.89:0/2936486665 shutdown_connections 2023-10-10T20:36:17.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.479+0000 7f683f59e700 1 --2- 172.21.15.89:0/2936486665 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6840153df0 0x7f68400cb800 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:17.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.479+0000 7f683f59e700 1 --2- 172.21.15.89:0/2936486665 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f684006e8c0 0x7f68401390d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:17.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.479+0000 7f683f59e700 1 --2- 172.21.15.89:0/2936486665 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6820066fc0 0x7f6820069470 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:17.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.479+0000 7f683f59e700 1 --2- 172.21.15.89:0/2936486665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68400c8830 0x7f6840149d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:17.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.479+0000 7f683f59e700 1 -- 172.21.15.89:0/2936486665 >> 172.21.15.89:0/2936486665 conn(0x7f684006ddb0 msgr2=0x7f684006e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:17.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.479+0000 7f683f59e700 1 -- 172.21.15.89:0/2936486665 shutdown_connections 2023-10-10T20:36:17.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.479+0000 7f683f59e700 1 -- 172.21.15.89:0/2936486665 wait complete. 2023-10-10T20:36:17.484 INFO:teuthology.orchestra.run.smithi089.stderr: removed all blocklist entries 2023-10-10T20:36:17.493 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:36:17.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.606+0000 7f9adb326700 1 Processor -- start 2023-10-10T20:36:17.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.606+0000 7f9adb326700 1 -- start start 2023-10-10T20:36:17.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.607+0000 7f9adb326700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9ad4159420 0x7f9ad40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.607+0000 7f9adb326700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad40b2980 con 0x7f9ad41597f0 2023-10-10T20:36:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.607+0000 7f9adb326700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad40b11d0 con 0x7f9ad415d230 2023-10-10T20:36:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.607+0000 7f9adb326700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad40b14b0 con 0x7f9ad4159420 2023-10-10T20:36:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.607+0000 7f9ada324700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9ad4159420 0x7f9ad40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.607+0000 7f9ad9b23700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9ad41597f0 0x7f9ad40b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42914/0 (socket says 172.21.15.89:42914) 2023-10-10T20:36:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.607+0000 7f9ad9b23700 1 -- 172.21.15.89:0/2587489641 learned_addr learned my addr 172.21.15.89:0/2587489641 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.607+0000 7f9ada324700 1 -- 172.21.15.89:0/2587489641 >> v1:172.21.15.111:6789/0 conn(0x7f9ad415d230 legacy=0x7f9ad415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:17.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.607+0000 7f9ada324700 1 -- 172.21.15.89:0/2587489641 >> v1:172.21.15.89:6789/0 conn(0x7f9ad41597f0 legacy=0x7f9ad40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:17.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.607+0000 7f9ada324700 1 -- 172.21.15.89:0/2587489641 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad40b1790 con 0x7f9ad4159420 2023-10-10T20:36:17.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.607+0000 7f9ada324700 1 --2- 172.21.15.89:0/2587489641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad4159420 0x7f9ad40b5f40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00098e0 tx=0x7f9ad00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f64b9c19e2d6087 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:17.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.607+0000 7f9ad9322700 1 -- 172.21.15.89:0/2587489641 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0009e80 con 0x7f9ad4159420 2023-10-10T20:36:17.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.607+0000 7f9ad9322700 1 -- 172.21.15.89:0/2587489641 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad000b760 con 0x7f9ad4159420 2023-10-10T20:36:17.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.607+0000 7f9ad9322700 1 -- 172.21.15.89:0/2587489641 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad000bb00 con 0x7f9ad4159420 2023-10-10T20:36:17.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.608+0000 7f9adb326700 1 -- 172.21.15.89:0/2587489641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad4159420 msgr2=0x7f9ad40b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.608+0000 7f9adb326700 1 --2- 172.21.15.89:0/2587489641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad4159420 0x7f9ad40b5f40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00098e0 tx=0x7f9ad00091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.608+0000 7f9adb326700 1 -- 172.21.15.89:0/2587489641 shutdown_connections 2023-10-10T20:36:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.608+0000 7f9adb326700 1 --2- 172.21.15.89:0/2587489641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad4159420 0x7f9ad40b5f40 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.608+0000 7f9adb326700 1 -- 172.21.15.89:0/2587489641 >> 172.21.15.89:0/2587489641 conn(0x7f9ad40d0660 msgr2=0x7f9ad40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.608+0000 7f9adb326700 1 -- 172.21.15.89:0/2587489641 shutdown_connections 2023-10-10T20:36:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.608+0000 7f9adb326700 1 -- 172.21.15.89:0/2587489641 wait complete. 2023-10-10T20:36:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.609+0000 7f9adb326700 1 Processor -- start 2023-10-10T20:36:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.609+0000 7f9adb326700 1 -- start start 2023-10-10T20:36:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.610+0000 7f9adb326700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ad4159420 0x7f9ad40c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.610+0000 7f9adb326700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad41597f0 0x7f9ad4155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.610+0000 7f9adb326700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ad415d230 0x7f9ad40ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.610+0000 7f9adb326700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad4155ad0 con 0x7f9ad41597f0 2023-10-10T20:36:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.610+0000 7f9adb326700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad40caa80 con 0x7f9ad4159420 2023-10-10T20:36:17.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.610+0000 7f9adb326700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad40cad60 con 0x7f9ad415d230 2023-10-10T20:36:17.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.610+0000 7f9ad9b23700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad41597f0 0x7f9ad4155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:17.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.610+0000 7f9ad9b23700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad41597f0 0x7f9ad4155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55284/0 (socket says 172.21.15.89:55284) 2023-10-10T20:36:17.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.610+0000 7f9ad9b23700 1 -- 172.21.15.89:0/1663077282 learned_addr learned my addr 172.21.15.89:0/1663077282 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:17.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.610+0000 7f9ada324700 1 --2- 172.21.15.89:0/1663077282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ad4159420 0x7f9ad40c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:17.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.610+0000 7f9adab25700 1 --2- 172.21.15.89:0/1663077282 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ad415d230 0x7f9ad40ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:17.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.610+0000 7f9ad9b23700 1 -- 172.21.15.89:0/1663077282 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ad415d230 msgr2=0x7f9ad40ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:17.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.610+0000 7f9ad9b23700 1 --2- 172.21.15.89:0/1663077282 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ad415d230 0x7f9ad40ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:17.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.610+0000 7f9ad9b23700 1 -- 172.21.15.89:0/1663077282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ad4159420 msgr2=0x7f9ad40c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:17.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.610+0000 7f9ad9b23700 1 --2- 172.21.15.89:0/1663077282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ad4159420 0x7f9ad40c6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:17.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.610+0000 7f9ad9b23700 1 -- 172.21.15.89:0/1663077282 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad0009420 con 0x7f9ad41597f0 2023-10-10T20:36:17.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.611+0000 7f9ad9b23700 1 --2- 172.21.15.89:0/1663077282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad41597f0 0x7f9ad4155590 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400c7e0 tx=0x7f9ac4007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:17.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.611+0000 7f9aca7fc700 1 -- 172.21.15.89:0/1663077282 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4005b30 con 0x7f9ad41597f0 2023-10-10T20:36:17.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.611+0000 7f9aca7fc700 1 -- 172.21.15.89:0/1663077282 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac4005c90 con 0x7f9ad41597f0 2023-10-10T20:36:17.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.611+0000 7f9aca7fc700 1 -- 172.21.15.89:0/1663077282 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4007320 con 0x7f9ad41597f0 2023-10-10T20:36:17.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.611+0000 7f9adb326700 1 -- 172.21.15.89:0/1663077282 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad40cb040 con 0x7f9ad41597f0 2023-10-10T20:36:17.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.611+0000 7f9adb326700 1 -- 172.21.15.89:0/1663077282 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad40cb600 con 0x7f9ad41597f0 2023-10-10T20:36:17.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.612+0000 7f9adb326700 1 -- 172.21.15.89:0/1663077282 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad40cb820 con 0x7f9ad41597f0 2023-10-10T20:36:17.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.612+0000 7f9aca7fc700 1 -- 172.21.15.89:0/1663077282 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f9ac401b400 con 0x7f9ad41597f0 2023-10-10T20:36:17.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.612+0000 7f9aca7fc700 1 --2- 172.21.15.89:0/1663077282 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ab8066fc0 0x7f9ab8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:17.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.612+0000 7f9aca7fc700 1 -- 172.21.15.89:0/1663077282 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9ac4083da0 con 0x7f9ad41597f0 2023-10-10T20:36:17.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.612+0000 7f9ada324700 1 --2- 172.21.15.89:0/1663077282 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ab8066fc0 0x7f9ab8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:17.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.613+0000 7f9ada324700 1 --2- 172.21.15.89:0/1663077282 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ab8066fc0 0x7f9ab8069470 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00091e0 tx=0x7f9ad0016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:17.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.617+0000 7f9aca7fc700 1 -- 172.21.15.89:0/1663077282 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ac4051370 con 0x7f9ad41597f0 2023-10-10T20:36:17.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.874+0000 7f9adb326700 1 -- 172.21.15.89:0/1663077282 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9ad404c3c0 con 0x7f9ad41597f0 2023-10-10T20:36:17.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.875+0000 7f9aca7fc700 1 -- 172.21.15.89:0/1663077282 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9ac4054990 con 0x7f9ad41597f0 2023-10-10T20:36:17.877 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:17.878 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:36:17.878 INFO:teuthology.orchestra.run.smithi089.stdout: "auth_dump": [ 2023-10-10T20:36:17.878 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:17.878 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.a", 2023-10-10T20:36:17.879 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlZLSNLhAA8WZTrsht6tAOP+gxaTXKMg==", 2023-10-10T20:36:17.879 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:36:17.879 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:36:17.879 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:36:17.879 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:36:17.880 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:36:17.880 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.880 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:36:17.880 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:17.880 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.b", 2023-10-10T20:36:17.880 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgyDEORAAMMoSxPFZc0bCIwWy8UsXgA==", 2023-10-10T20:36:17.881 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:36:17.881 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:36:17.881 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:36:17.881 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:36:17.881 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:36:17.882 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.882 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:36:17.882 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:17.882 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.c", 2023-10-10T20:36:17.882 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlMFxSNRAASuQEJFghLIOkTQWa4ckTvg==", 2023-10-10T20:36:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:36:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:36:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:36:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:36:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:36:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.884 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:36:17.884 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:17.884 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.d", 2023-10-10T20:36:17.884 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlPkvrBBAA2VLJmv2t8szTEGtCVwalJw==", 2023-10-10T20:36:17.884 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:36:17.885 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:36:17.885 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:36:17.885 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:36:17.885 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:36:17.885 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.886 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:36:17.886 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:17.886 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.0", 2023-10-10T20:36:17.886 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQArtSVltOcKHRAAk8+zUus04oTVI9KvfhjlXg==", 2023-10-10T20:36:17.886 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:36:17.887 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:36:17.887 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:36:17.887 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:36:17.887 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:36:17.888 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:17.888 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.1", 2023-10-10T20:36:17.888 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAutSVly8P0GhAAe1bvR63+ptJg84lizMNdQw==", 2023-10-10T20:36:17.888 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:36:17.888 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:36:17.889 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:36:17.889 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:36:17.889 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:36:17.889 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:17.889 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.2", 2023-10-10T20:36:17.890 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAxtSVl0x9fGRAAEKi7+6EW5mzPbebmz0+sgQ==", 2023-10-10T20:36:17.890 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:36:17.890 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:36:17.890 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:36:17.890 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:36:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:36:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.3", 2023-10-10T20:36:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA0tSVlfJyaGBAAa+8w54U6v7+npbdufKJruA==", 2023-10-10T20:36:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:36:17.892 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:36:17.892 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:36:17.892 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:36:17.892 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:36:17.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:17.893 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.4", 2023-10-10T20:36:17.893 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA6tSVlEQ/QFxAA/PRB3c90BuAlF6xekGfX9A==", 2023-10-10T20:36:17.893 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:36:17.894 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:36:17.894 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:36:17.894 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:36:17.894 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:36:17.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:17.895 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.5", 2023-10-10T20:36:17.895 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA9tSVlMOKrFRAAuBnIsEz2VmeSZi9XWzjlzA==", 2023-10-10T20:36:17.895 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:36:17.895 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:36:17.895 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:36:17.896 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:36:17.896 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:36:17.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:17.896 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.6", 2023-10-10T20:36:17.897 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBAtSVlH7PQFBAAHr6c7d4TYJEu/KSFEf+EOA==", 2023-10-10T20:36:17.897 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:36:17.897 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:36:17.897 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:36:17.897 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:36:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:36:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.7", 2023-10-10T20:36:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBDtSVlAhC1FBAAuE2BwAA/f7VOkZfVdqAG7A==", 2023-10-10T20:36:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:36:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:36:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:36:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:36:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:36:17.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:17.900 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.0", 2023-10-10T20:36:17.900 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlRd3HCBAAb0QP2VfTS7aelno1mFpGAA==", 2023-10-10T20:36:17.900 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:36:17.900 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:36:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow r", 2023-10-10T20:36:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:36:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:36:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.1", 2023-10-10T20:36:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlgBouERAAqPdE0KvJU3M53B6Y6AH5GQ==", 2023-10-10T20:36:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:36:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:36:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:36:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:36:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:36:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:36:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.2", 2023-10-10T20:36:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlXA4tDhAAoPVYQjECpzD+C67mnsY5IA==", 2023-10-10T20:36:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:36:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:36:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:36:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:36:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:36:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:36:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.3", 2023-10-10T20:36:17.906 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlZ5KGFhAAuKmZwbOfmmw1nAj5ry6tHw==", 2023-10-10T20:36:17.906 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:36:17.906 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:36:17.906 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:36:17.906 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:36:17.906 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:36:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:36:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.admin", 2023-10-10T20:36:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgpL2EhAAoDOki002rfBrauTgfu481Q==", 2023-10-10T20:36:17.908 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:36:17.908 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:36:17.908 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:36:17.908 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:36:17.908 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:36:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:36:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.x", 2023-10-10T20:36:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlBhaJKxAArraBxT5xWg515LfpJ7MlbA==", 2023-10-10T20:36:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:36:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:36:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:36:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:36:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:36:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.y", 2023-10-10T20:36:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlTZIyJxAAWAOsitHwsIxJWwp3b1Z98Q==", 2023-10-10T20:36:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:36:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:36:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:36:17.912 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:36:17.912 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.912 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:17.912 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-10T20:36:17.912 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:36:17.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.877+0000 7f9adb326700 1 -- 172.21.15.89:0/1663077282 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ab8066fc0 msgr2=0x7f9ab8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:17.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.877+0000 7f9adb326700 1 --2- 172.21.15.89:0/1663077282 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ab8066fc0 0x7f9ab8069470 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00091e0 tx=0x7f9ad0016000 comp rx=0 tx=0).stop 2023-10-10T20:36:17.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.877+0000 7f9adb326700 1 -- 172.21.15.89:0/1663077282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad41597f0 msgr2=0x7f9ad4155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:17.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.877+0000 7f9adb326700 1 --2- 172.21.15.89:0/1663077282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad41597f0 0x7f9ad4155590 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400c7e0 tx=0x7f9ac4007f00 comp rx=0 tx=0).stop 2023-10-10T20:36:17.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.877+0000 7f9adb326700 1 -- 172.21.15.89:0/1663077282 shutdown_connections 2023-10-10T20:36:17.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.877+0000 7f9adb326700 1 --2- 172.21.15.89:0/1663077282 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ad415d230 0x7f9ad40ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:17.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.877+0000 7f9adb326700 1 --2- 172.21.15.89:0/1663077282 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ab8066fc0 0x7f9ab8069470 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:17.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.877+0000 7f9adb326700 1 --2- 172.21.15.89:0/1663077282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ad4159420 0x7f9ad40c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:17.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.877+0000 7f9adb326700 1 --2- 172.21.15.89:0/1663077282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ad41597f0 0x7f9ad4155590 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:17.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.877+0000 7f9adb326700 1 -- 172.21.15.89:0/1663077282 >> 172.21.15.89:0/1663077282 conn(0x7f9ad40d0660 msgr2=0x7f9ad41562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:17.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.877+0000 7f9adb326700 1 -- 172.21.15.89:0/1663077282 shutdown_connections 2023-10-10T20:36:17.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:17.877+0000 7f9adb326700 1 -- 172.21.15.89:0/1663077282 wait complete. 2023-10-10T20:36:17.917 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:36:18.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.029+0000 7f9d6759e700 1 Processor -- start 2023-10-10T20:36:18.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.029+0000 7f9d6759e700 1 -- start start 2023-10-10T20:36:18.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.030+0000 7f9d6759e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9d68159420 0x7f9d680b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.030+0000 7f9d6759e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9d681597f0 0x7f9d680b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.030+0000 7f9d6759e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9d6815d230 0x7f9d6815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.030+0000 7f9d6759e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d6815dbd0 con 0x7f9d6815d230 2023-10-10T20:36:18.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.030+0000 7f9d6759e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d680b11d0 con 0x7f9d681597f0 2023-10-10T20:36:18.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.030+0000 7f9d6759e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d680b14b0 con 0x7f9d68159420 2023-10-10T20:36:18.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.030+0000 7f9d66d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9d6815d230 0x7f9d6815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.030+0000 7f9d66d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9d6815d230 0x7f9d6815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55292/0 (socket says 172.21.15.89:55292) 2023-10-10T20:36:18.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.030+0000 7f9d66d9d700 1 -- 172.21.15.89:0/2163410141 learned_addr learned my addr 172.21.15.89:0/2163410141 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:18.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.030+0000 7f9d5dd9b700 1 --2- 172.21.15.89:0/2163410141 >> v2:172.21.15.111:3300/0 conn(0x7f9d681597f0 0x7f9d680b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.031+0000 7f9d6659c700 1 --2- 172.21.15.89:0/2163410141 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9d68159420 0x7f9d680b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.031+0000 7f9d66d9d700 1 -- 172.21.15.89:0/2163410141 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9d68159420 msgr2=0x7f9d680b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:18.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.031+0000 7f9d66d9d700 1 --2- 172.21.15.89:0/2163410141 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9d68159420 0x7f9d680b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.031+0000 7f9d66d9d700 1 -- 172.21.15.89:0/2163410141 >> v2:172.21.15.111:3300/0 conn(0x7f9d681597f0 msgr2=0x7f9d680b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:18.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.031+0000 7f9d66d9d700 1 --2- 172.21.15.89:0/2163410141 >> v2:172.21.15.111:3300/0 conn(0x7f9d681597f0 0x7f9d680b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.031+0000 7f9d66d9d700 1 -- 172.21.15.89:0/2163410141 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d680b1790 con 0x7f9d6815d230 2023-10-10T20:36:18.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.031+0000 7f9d6659c700 1 --2- 172.21.15.89:0/2163410141 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9d68159420 0x7f9d680b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:18.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.031+0000 7f9d66d9d700 1 --2- 172.21.15.89:0/2163410141 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d6815d230 0x7f9d6815d690 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9d680d1770 tx=0x7f9d60009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c15528c8769272a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:18.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.031+0000 7f9d65d9b700 1 -- 172.21.15.89:0/2163410141 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6000e070 con 0x7f9d6815d230 2023-10-10T20:36:18.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.031+0000 7f9d65d9b700 1 -- 172.21.15.89:0/2163410141 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6000ba20 con 0x7f9d6815d230 2023-10-10T20:36:18.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.031+0000 7f9d65d9b700 1 -- 172.21.15.89:0/2163410141 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d600092e0 con 0x7f9d6815d230 2023-10-10T20:36:18.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.031+0000 7f9d6759e700 1 -- 172.21.15.89:0/2163410141 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d6815d230 msgr2=0x7f9d6815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:18.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.031+0000 7f9d6759e700 1 --2- 172.21.15.89:0/2163410141 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d6815d230 0x7f9d6815d690 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9d680d1770 tx=0x7f9d60009840 comp rx=0 tx=0).stop 2023-10-10T20:36:18.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.032+0000 7f9d6759e700 1 -- 172.21.15.89:0/2163410141 shutdown_connections 2023-10-10T20:36:18.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.032+0000 7f9d6759e700 1 --2- 172.21.15.89:0/2163410141 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d6815d230 0x7f9d6815d690 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.032+0000 7f9d6759e700 1 --2- 172.21.15.89:0/2163410141 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9d68159420 0x7f9d680b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.032+0000 7f9d6759e700 1 --2- 172.21.15.89:0/2163410141 >> v2:172.21.15.111:3300/0 conn(0x7f9d681597f0 0x7f9d680b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.032+0000 7f9d6759e700 1 -- 172.21.15.89:0/2163410141 >> 172.21.15.89:0/2163410141 conn(0x7f9d680d0660 msgr2=0x7f9d680d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:18.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.032+0000 7f9d6759e700 1 -- 172.21.15.89:0/2163410141 shutdown_connections 2023-10-10T20:36:18.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.032+0000 7f9d6759e700 1 -- 172.21.15.89:0/2163410141 wait complete. 2023-10-10T20:36:18.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.033+0000 7f9d6759e700 1 Processor -- start 2023-10-10T20:36:18.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.033+0000 7f9d6759e700 1 -- start start 2023-10-10T20:36:18.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.033+0000 7f9d6759e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d68159420 0x7f9d680c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.033+0000 7f9d6759e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9d681597f0 0x7f9d680c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d6759e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d6815d230 0x7f9d68169690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d6759e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d680c6760 con 0x7f9d68159420 2023-10-10T20:36:18.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d6759e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d680c7040 con 0x7f9d6815d230 2023-10-10T20:36:18.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d6759e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d680c7320 con 0x7f9d681597f0 2023-10-10T20:36:18.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d6659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d68159420 0x7f9d680c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d6659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d68159420 0x7f9d680c5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55304/0 (socket says 172.21.15.89:55304) 2023-10-10T20:36:18.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d6659c700 1 -- 172.21.15.89:0/3179716390 learned_addr learned my addr 172.21.15.89:0/3179716390 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:18.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d5dd9b700 1 --2- 172.21.15.89:0/3179716390 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9d681597f0 0x7f9d680c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d66d9d700 1 --2- 172.21.15.89:0/3179716390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d6815d230 0x7f9d68169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d6659c700 1 -- 172.21.15.89:0/3179716390 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9d681597f0 msgr2=0x7f9d680c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:18.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d6659c700 1 --2- 172.21.15.89:0/3179716390 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9d681597f0 0x7f9d680c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d6659c700 1 -- 172.21.15.89:0/3179716390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d6815d230 msgr2=0x7f9d68169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:18.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d6659c700 1 --2- 172.21.15.89:0/3179716390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d6815d230 0x7f9d68169690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d6659c700 1 -- 172.21.15.89:0/3179716390 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d60009ae0 con 0x7f9d68159420 2023-10-10T20:36:18.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d6659c700 1 --2- 172.21.15.89:0/3179716390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d68159420 0x7f9d680c5ce0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800e8d0 tx=0x7f9d58008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:18.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d5effd700 1 -- 172.21.15.89:0/3179716390 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d58013410 con 0x7f9d68159420 2023-10-10T20:36:18.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d5effd700 1 -- 172.21.15.89:0/3179716390 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d58013570 con 0x7f9d68159420 2023-10-10T20:36:18.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d6759e700 1 -- 172.21.15.89:0/3179716390 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d680c7600 con 0x7f9d68159420 2023-10-10T20:36:18.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d5effd700 1 -- 172.21.15.89:0/3179716390 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d58013910 con 0x7f9d68159420 2023-10-10T20:36:18.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.034+0000 7f9d6759e700 1 -- 172.21.15.89:0/3179716390 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d680c7bc0 con 0x7f9d68159420 2023-10-10T20:36:18.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.035+0000 7f9d6759e700 1 -- 172.21.15.89:0/3179716390 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d680c7f30 con 0x7f9d68159420 2023-10-10T20:36:18.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.036+0000 7f9d5effd700 1 -- 172.21.15.89:0/3179716390 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f9d58013d00 con 0x7f9d68159420 2023-10-10T20:36:18.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.036+0000 7f9d5effd700 1 --2- 172.21.15.89:0/3179716390 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9d48066fc0 0x7f9d48069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.036+0000 7f9d5effd700 1 -- 172.21.15.89:0/3179716390 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9d58084060 con 0x7f9d68159420 2023-10-10T20:36:18.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.036+0000 7f9d5dd9b700 1 --2- 172.21.15.89:0/3179716390 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9d48066fc0 0x7f9d48069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.037+0000 7f9d5dd9b700 1 --2- 172.21.15.89:0/3179716390 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9d48066fc0 0x7f9d48069470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9d6816cbf0 tx=0x7f9d500091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:18.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.040+0000 7f9d5effd700 1 -- 172.21.15.89:0/3179716390 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d58051520 con 0x7f9d68159420 2023-10-10T20:36:18.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.224+0000 7f9d6759e700 1 -- 172.21.15.89:0/3179716390 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d6804c3c0 con 0x7f9d68159420 2023-10-10T20:36:18.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.225+0000 7f9d5effd700 1 -- 172.21.15.89:0/3179716390 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+12134 (secure 0 0 0) 0x7f9d5805a070 con 0x7f9d68159420 2023-10-10T20:36:18.228 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:18.228 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":58,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:36:17.472222+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:36:18.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.227+0000 7f9d6759e700 1 -- 172.21.15.89:0/3179716390 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9d48066fc0 msgr2=0x7f9d48069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:18.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.227+0000 7f9d6759e700 1 --2- 172.21.15.89:0/3179716390 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9d48066fc0 0x7f9d48069470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9d6816cbf0 tx=0x7f9d500091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.227+0000 7f9d6759e700 1 -- 172.21.15.89:0/3179716390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d68159420 msgr2=0x7f9d680c5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:18.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.227+0000 7f9d6759e700 1 --2- 172.21.15.89:0/3179716390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d68159420 0x7f9d680c5ce0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800e8d0 tx=0x7f9d58008040 comp rx=0 tx=0).stop 2023-10-10T20:36:18.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.227+0000 7f9d6759e700 1 -- 172.21.15.89:0/3179716390 shutdown_connections 2023-10-10T20:36:18.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.227+0000 7f9d6759e700 1 --2- 172.21.15.89:0/3179716390 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9d48066fc0 0x7f9d48069470 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.227+0000 7f9d6759e700 1 --2- 172.21.15.89:0/3179716390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d6815d230 0x7f9d68169690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.227+0000 7f9d6759e700 1 --2- 172.21.15.89:0/3179716390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d68159420 0x7f9d680c5ce0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.227+0000 7f9d6759e700 1 --2- 172.21.15.89:0/3179716390 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9d681597f0 0x7f9d680c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.227+0000 7f9d6759e700 1 -- 172.21.15.89:0/3179716390 >> 172.21.15.89:0/3179716390 conn(0x7f9d680d0660 msgr2=0x7f9d681552b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:18.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.227+0000 7f9d6759e700 1 -- 172.21.15.89:0/3179716390 shutdown_connections 2023-10-10T20:36:18.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.227+0000 7f9d6759e700 1 -- 172.21.15.89:0/3179716390 wait complete. 2023-10-10T20:36:18.242 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:36:18.242 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:36:18.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.357+0000 7f0b8c1eb700 1 Processor -- start 2023-10-10T20:36:18.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.357+0000 7f0b8c1eb700 1 -- start start 2023-10-10T20:36:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.358+0000 7f0b8c1eb700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0b84159c10 0x7f0b8415b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.358+0000 7f0b8c1eb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0b8415c910 0x7f0b8415cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.358+0000 7f0b8c1eb700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b8415c480 con 0x7f0b840b5ea0 2023-10-10T20:36:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.358+0000 7f0b8c1eb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b840b2a40 con 0x7f0b8415c910 2023-10-10T20:36:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.358+0000 7f0b8c1eb700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b8415d290 con 0x7f0b84159c10 2023-10-10T20:36:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.358+0000 7f0b8a788700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0b8415c910 0x7f0b8415cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.358+0000 7f0b8a788700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0b8415c910 0x7f0b8415cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46686/0 (socket says 172.21.15.89:46686) 2023-10-10T20:36:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.358+0000 7f0b8a788700 1 -- 172.21.15.89:0/3089189818 learned_addr learned my addr 172.21.15.89:0/3089189818 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.358+0000 7f0b89f87700 1 --2- 172.21.15.89:0/3089189818 >> v2:172.21.15.111:3300/0 conn(0x7f0b84159c10 0x7f0b8415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.358+0000 7f0b8a788700 1 -- 172.21.15.89:0/3089189818 >> v2:172.21.15.111:3300/0 conn(0x7f0b84159c10 msgr2=0x7f0b8415b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.358+0000 7f0b8a788700 1 --2- 172.21.15.89:0/3089189818 >> v2:172.21.15.111:3300/0 conn(0x7f0b84159c10 0x7f0b8415b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.358+0000 7f0b8a788700 1 -- 172.21.15.89:0/3089189818 >> v1:172.21.15.111:6789/0 conn(0x7f0b840b5ea0 legacy=0x7f0b84068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:18.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.358+0000 7f0b8a788700 1 -- 172.21.15.89:0/3089189818 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8415b5d0 con 0x7f0b8415c910 2023-10-10T20:36:18.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.358+0000 7f0b8a788700 1 --2- 172.21.15.89:0/3089189818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8415c910 0x7f0b8415cd50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0b84099f10 tx=0x7f0b7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0be2a44a69cabe8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:18.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.358+0000 7f0b88f85700 1 -- 172.21.15.89:0/3089189818 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c009e70 con 0x7f0b8415c910 2023-10-10T20:36:18.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.358+0000 7f0b88f85700 1 -- 172.21.15.89:0/3089189818 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7c00b860 con 0x7f0b8415c910 2023-10-10T20:36:18.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.358+0000 7f0b88f85700 1 -- 172.21.15.89:0/3089189818 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c00bc00 con 0x7f0b8415c910 2023-10-10T20:36:18.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.359+0000 7f0b8c1eb700 1 -- 172.21.15.89:0/3089189818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8415c910 msgr2=0x7f0b8415cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:18.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.359+0000 7f0b8c1eb700 1 --2- 172.21.15.89:0/3089189818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8415c910 0x7f0b8415cd50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0b84099f10 tx=0x7f0b7c009250 comp rx=0 tx=0).stop 2023-10-10T20:36:18.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.359+0000 7f0b8c1eb700 1 -- 172.21.15.89:0/3089189818 shutdown_connections 2023-10-10T20:36:18.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.359+0000 7f0b8c1eb700 1 --2- 172.21.15.89:0/3089189818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8415c910 0x7f0b8415cd50 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.359+0000 7f0b8c1eb700 1 --2- 172.21.15.89:0/3089189818 >> v2:172.21.15.111:3300/0 conn(0x7f0b84159c10 0x7f0b8415b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.359+0000 7f0b8c1eb700 1 -- 172.21.15.89:0/3089189818 >> 172.21.15.89:0/3089189818 conn(0x7f0b84099a20 msgr2=0x7f0b840af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:18.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.359+0000 7f0b8c1eb700 1 -- 172.21.15.89:0/3089189818 shutdown_connections 2023-10-10T20:36:18.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.359+0000 7f0b8c1eb700 1 -- 172.21.15.89:0/3089189818 wait complete. 2023-10-10T20:36:18.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.360+0000 7f0b8c1eb700 1 Processor -- start 2023-10-10T20:36:18.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.360+0000 7f0b8c1eb700 1 -- start start 2023-10-10T20:36:18.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.361+0000 7f0b8c1eb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b840b5ea0 0x7f0b840c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.361+0000 7f0b8c1eb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0b84159c10 0x7f0b84145730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.361+0000 7f0b8c1eb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8415c910 0x7f0b84144f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.361+0000 7f0b8c1eb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b840b01f0 con 0x7f0b8415c910 2023-10-10T20:36:18.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.361+0000 7f0b8c1eb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b84145480 con 0x7f0b840b5ea0 2023-10-10T20:36:18.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.361+0000 7f0b8c1eb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b840aeae0 con 0x7f0b84159c10 2023-10-10T20:36:18.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.361+0000 7f0b8a788700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8415c910 0x7f0b84144f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.361+0000 7f0b8a788700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8415c910 0x7f0b84144f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46702/0 (socket says 172.21.15.89:46702) 2023-10-10T20:36:18.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.361+0000 7f0b8a788700 1 -- 172.21.15.89:0/2663514116 learned_addr learned my addr 172.21.15.89:0/2663514116 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:18.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.361+0000 7f0b89f87700 1 --2- 172.21.15.89:0/2663514116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b840b5ea0 0x7f0b840c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.361+0000 7f0b89786700 1 --2- 172.21.15.89:0/2663514116 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0b84159c10 0x7f0b84145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.361+0000 7f0b8a788700 1 -- 172.21.15.89:0/2663514116 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0b84159c10 msgr2=0x7f0b84145730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:18.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.361+0000 7f0b8a788700 1 --2- 172.21.15.89:0/2663514116 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0b84159c10 0x7f0b84145730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.361+0000 7f0b8a788700 1 -- 172.21.15.89:0/2663514116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b840b5ea0 msgr2=0x7f0b840c5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:18.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.361+0000 7f0b8a788700 1 --2- 172.21.15.89:0/2663514116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b840b5ea0 0x7f0b840c5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.361+0000 7f0b8a788700 1 -- 172.21.15.89:0/2663514116 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7c0094f0 con 0x7f0b8415c910 2023-10-10T20:36:18.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.361+0000 7f0b8a788700 1 --2- 172.21.15.89:0/2663514116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8415c910 0x7f0b84144f40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0b84063db0 tx=0x7f0b7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:18.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.362+0000 7f0b79ffb700 1 -- 172.21.15.89:0/2663514116 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c00bbe0 con 0x7f0b8415c910 2023-10-10T20:36:18.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.362+0000 7f0b79ffb700 1 -- 172.21.15.89:0/2663514116 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7c011420 con 0x7f0b8415c910 2023-10-10T20:36:18.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.362+0000 7f0b8c1eb700 1 -- 172.21.15.89:0/2663514116 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b840aeda0 con 0x7f0b8415c910 2023-10-10T20:36:18.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.362+0000 7f0b79ffb700 1 -- 172.21.15.89:0/2663514116 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c0117e0 con 0x7f0b8415c910 2023-10-10T20:36:18.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.362+0000 7f0b8c1eb700 1 -- 172.21.15.89:0/2663514116 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b840afe80 con 0x7f0b8415c910 2023-10-10T20:36:18.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.363+0000 7f0b8c1eb700 1 -- 172.21.15.89:0/2663514116 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b840af2c0 con 0x7f0b8415c910 2023-10-10T20:36:18.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.363+0000 7f0b79ffb700 1 -- 172.21.15.89:0/2663514116 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f0b7c011b90 con 0x7f0b8415c910 2023-10-10T20:36:18.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.363+0000 7f0b79ffb700 1 --2- 172.21.15.89:0/2663514116 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0b68066fc0 0x7f0b68069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.364+0000 7f0b79ffb700 1 -- 172.21.15.89:0/2663514116 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f0b7c085af0 con 0x7f0b8415c910 2023-10-10T20:36:18.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.364+0000 7f0b89f87700 1 --2- 172.21.15.89:0/2663514116 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0b68066fc0 0x7f0b68069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.364+0000 7f0b89f87700 1 --2- 172.21.15.89:0/2663514116 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0b68066fc0 0x7f0b68069470 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009fd0 tx=0x7f0b8000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:18.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.368+0000 7f0b79ffb700 1 -- 172.21.15.89:0/2663514116 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b7c0530c0 con 0x7f0b8415c910 2023-10-10T20:36:18.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.551+0000 7f0b8c1eb700 1 -- 172.21.15.89:0/2663514116 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b840b03a0 con 0x7f0b8415c910 2023-10-10T20:36:18.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.551+0000 7f0b79ffb700 1 -- 172.21.15.89:0/2663514116 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b7c015020 con 0x7f0b8415c910 2023-10-10T20:36:18.554 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:36:18.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.553+0000 7f0b8c1eb700 1 -- 172.21.15.89:0/2663514116 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0b68066fc0 msgr2=0x7f0b68069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:18.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.553+0000 7f0b8c1eb700 1 --2- 172.21.15.89:0/2663514116 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0b68066fc0 0x7f0b68069470 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009fd0 tx=0x7f0b8000b040 comp rx=0 tx=0).stop 2023-10-10T20:36:18.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.553+0000 7f0b8c1eb700 1 -- 172.21.15.89:0/2663514116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8415c910 msgr2=0x7f0b84144f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:18.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.553+0000 7f0b8c1eb700 1 --2- 172.21.15.89:0/2663514116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8415c910 0x7f0b84144f40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0b84063db0 tx=0x7f0b7c017000 comp rx=0 tx=0).stop 2023-10-10T20:36:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.553+0000 7f0b8c1eb700 1 -- 172.21.15.89:0/2663514116 shutdown_connections 2023-10-10T20:36:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.553+0000 7f0b8c1eb700 1 --2- 172.21.15.89:0/2663514116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8415c910 0x7f0b84144f40 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.553+0000 7f0b8c1eb700 1 --2- 172.21.15.89:0/2663514116 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0b68066fc0 0x7f0b68069470 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.553+0000 7f0b8c1eb700 1 --2- 172.21.15.89:0/2663514116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b840b5ea0 0x7f0b840c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.553+0000 7f0b8c1eb700 1 --2- 172.21.15.89:0/2663514116 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0b84159c10 0x7f0b84145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.553+0000 7f0b8c1eb700 1 -- 172.21.15.89:0/2663514116 >> 172.21.15.89:0/2663514116 conn(0x7f0b84099a20 msgr2=0x7f0b8415a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.553+0000 7f0b8c1eb700 1 -- 172.21.15.89:0/2663514116 shutdown_connections 2023-10-10T20:36:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.553+0000 7f0b8c1eb700 1 -- 172.21.15.89:0/2663514116 wait complete. 2023-10-10T20:36:18.567 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:36:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.680+0000 7fcd9b02a700 1 Processor -- start 2023-10-10T20:36:18.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.680+0000 7fcd9b02a700 1 -- start start 2023-10-10T20:36:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.681+0000 7fcd9b02a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcd941579f0 0x7fcd94162530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.681+0000 7fcd9b02a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd940cb300 con 0x7fcd940b1850 2023-10-10T20:36:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.681+0000 7fcd9b02a700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd940cb5e0 con 0x7fcd940b1c20 2023-10-10T20:36:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.681+0000 7fcd9b02a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9406fc30 con 0x7fcd941579f0 2023-10-10T20:36:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.681+0000 7fcd98dc6700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcd940b1850 0x7fcd940ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56590/0 (socket says 172.21.15.89:56590) 2023-10-10T20:36:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.681+0000 7fcd98dc6700 1 -- 172.21.15.89:0/2291835673 learned_addr learned my addr 172.21.15.89:0/2291835673 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.681+0000 7fcd995c7700 1 --2- 172.21.15.89:0/2291835673 >> v2:172.21.15.111:3300/0 conn(0x7fcd941579f0 0x7fcd94162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.681+0000 7fcd8b7fe700 1 -- 172.21.15.89:0/2291835673 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 971931027 0 0) 0x7fcd940cb300 con 0x7fcd940b1850 2023-10-10T20:36:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8b7fe700 1 -- 172.21.15.89:0/2291835673 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd7c003570 con 0x7fcd940b1850 2023-10-10T20:36:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8b7fe700 1 -- 172.21.15.89:0/2291835673 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2428237003 0 0) 0x7fcd7c003570 con 0x7fcd940b1850 2023-10-10T20:36:18.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8b7fe700 1 -- 172.21.15.89:0/2291835673 >> v2:172.21.15.111:3300/0 conn(0x7fcd941579f0 msgr2=0x7fcd94162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:18.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8b7fe700 1 --2- 172.21.15.89:0/2291835673 >> v2:172.21.15.111:3300/0 conn(0x7fcd941579f0 0x7fcd94162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8b7fe700 1 -- 172.21.15.89:0/2291835673 >> v1:172.21.15.111:6789/0 conn(0x7fcd940b1c20 legacy=0x7fcd941572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:18.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8b7fe700 1 -- 172.21.15.89:0/2291835673 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd940cb8c0 con 0x7fcd940b1850 2023-10-10T20:36:18.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8b7fe700 1 -- 172.21.15.89:0/2291835673 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fcd90001470 con 0x7fcd940b1850 2023-10-10T20:36:18.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8b7fe700 1 -- 172.21.15.89:0/2291835673 >> v1:172.21.15.89:6789/0 conn(0x7fcd940b1850 legacy=0x7fcd940ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:18.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8b7fe700 1 --2- 172.21.15.89:0/2291835673 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd7c003a60 0x7fcd7c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8b7fe700 1 --2- 172.21.15.89:0/2291835673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd7c004810 0x7fcd7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8b7fe700 1 --2- 172.21.15.89:0/2291835673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd7c0051c0 0x7fcd7c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8b7fe700 1 -- 172.21.15.89:0/2291835673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9406fc30 con 0x7fcd7c004810 2023-10-10T20:36:18.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8b7fe700 1 -- 172.21.15.89:0/2291835673 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c00c0d0 con 0x7fcd7c0051c0 2023-10-10T20:36:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8b7fe700 1 -- 172.21.15.89:0/2291835673 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c00c290 con 0x7fcd7c003a60 2023-10-10T20:36:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8bfff700 1 --2- 172.21.15.89:0/2291835673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd7c004810 0x7fcd7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd995c7700 1 --2- 172.21.15.89:0/2291835673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd7c0051c0 0x7fcd7c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd98dc6700 1 --2- 172.21.15.89:0/2291835673 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd7c003a60 0x7fcd7c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8bfff700 1 -- 172.21.15.89:0/2291835673 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd7c003a60 msgr2=0x7fcd7c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8bfff700 1 --2- 172.21.15.89:0/2291835673 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd7c003a60 0x7fcd7c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8bfff700 1 -- 172.21.15.89:0/2291835673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd7c0051c0 msgr2=0x7fcd7c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8bfff700 1 --2- 172.21.15.89:0/2291835673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd7c0051c0 0x7fcd7c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8bfff700 1 -- 172.21.15.89:0/2291835673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c003570 con 0x7fcd7c004810 2023-10-10T20:36:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd995c7700 1 -- 172.21.15.89:0/2291835673 reap_dead start 2023-10-10T20:36:18.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.682+0000 7fcd8bfff700 1 --2- 172.21.15.89:0/2291835673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd7c004810 0x7fcd7c004c80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fcd840018e0 tx=0x7fcd84007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ae9bb9e82a9147 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:18.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.683+0000 7fcd8b7fe700 1 -- 172.21.15.89:0/2291835673 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84012420 con 0x7fcd7c004810 2023-10-10T20:36:18.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.683+0000 7fcd8b7fe700 1 -- 172.21.15.89:0/2291835673 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd840125a0 con 0x7fcd7c004810 2023-10-10T20:36:18.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.683+0000 7fcd8b7fe700 1 -- 172.21.15.89:0/2291835673 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84012960 con 0x7fcd7c004810 2023-10-10T20:36:18.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.683+0000 7fcd9b02a700 1 -- 172.21.15.89:0/2291835673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd7c004810 msgr2=0x7fcd7c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:18.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.683+0000 7fcd9b02a700 1 --2- 172.21.15.89:0/2291835673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd7c004810 0x7fcd7c004c80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fcd840018e0 tx=0x7fcd84007d30 comp rx=0 tx=0).stop 2023-10-10T20:36:18.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.683+0000 7fcd9b02a700 1 -- 172.21.15.89:0/2291835673 shutdown_connections 2023-10-10T20:36:18.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.683+0000 7fcd9b02a700 1 --2- 172.21.15.89:0/2291835673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd7c004810 0x7fcd7c004c80 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.683+0000 7fcd9b02a700 1 -- 172.21.15.89:0/2291835673 >> 172.21.15.89:0/2291835673 conn(0x7fcd9406a320 msgr2=0x7fcd940cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:18.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.683+0000 7fcd9b02a700 1 -- 172.21.15.89:0/2291835673 shutdown_connections 2023-10-10T20:36:18.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.683+0000 7fcd9b02a700 1 -- 172.21.15.89:0/2291835673 wait complete. 2023-10-10T20:36:18.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.684+0000 7fcd9b02a700 1 Processor -- start 2023-10-10T20:36:18.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.684+0000 7fcd9b02a700 1 -- start start 2023-10-10T20:36:18.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.685+0000 7fcd9b02a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd7c004810 0x7fcd94146190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.685+0000 7fcd9b02a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd9415ae70 0x7fcd9406f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.685+0000 7fcd9b02a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd9415c2c0 0x7fcd9415c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.685+0000 7fcd9b02a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940cc260 con 0x7fcd9415ae70 2023-10-10T20:36:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.685+0000 7fcd9b02a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd941466d0 con 0x7fcd9415c2c0 2023-10-10T20:36:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.685+0000 7fcd9b02a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9415cca0 con 0x7fcd7c004810 2023-10-10T20:36:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.685+0000 7fcd8bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd9415ae70 0x7fcd9406f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.685+0000 7fcd8bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd9415ae70 0x7fcd9406f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46726/0 (socket says 172.21.15.89:46726) 2023-10-10T20:36:18.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.685+0000 7fcd8bfff700 1 -- 172.21.15.89:0/1200835482 learned_addr learned my addr 172.21.15.89:0/1200835482 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:18.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.685+0000 7fcd8bfff700 1 -- 172.21.15.89:0/1200835482 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd7c004810 msgr2=0x7fcd94146190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:18.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.685+0000 7fcd8bfff700 1 --2- 172.21.15.89:0/1200835482 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd7c004810 0x7fcd94146190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.685+0000 7fcd8bfff700 1 -- 172.21.15.89:0/1200835482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd9415c2c0 msgr2=0x7fcd9415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.685+0000 7fcd995c7700 1 --2- 172.21.15.89:0/1200835482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd9415c2c0 0x7fcd9415c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.686+0000 7fcd8bfff700 1 --2- 172.21.15.89:0/1200835482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd9415c2c0 0x7fcd9415c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.686+0000 7fcd8bfff700 1 -- 172.21.15.89:0/1200835482 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9415b430 con 0x7fcd9415ae70 2023-10-10T20:36:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.686+0000 7fcd8bfff700 1 --2- 172.21.15.89:0/1200835482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd9415ae70 0x7fcd9406f420 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400c0e0 tx=0x7fcd84015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.686+0000 7fcd88ff9700 1 -- 172.21.15.89:0/1200835482 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84001230 con 0x7fcd9415ae70 2023-10-10T20:36:18.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.686+0000 7fcd88ff9700 1 -- 172.21.15.89:0/1200835482 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd84012710 con 0x7fcd9415ae70 2023-10-10T20:36:18.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.686+0000 7fcd9b02a700 1 -- 172.21.15.89:0/1200835482 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9415b710 con 0x7fcd9415ae70 2023-10-10T20:36:18.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.686+0000 7fcd88ff9700 1 -- 172.21.15.89:0/1200835482 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84006820 con 0x7fcd9415ae70 2023-10-10T20:36:18.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.686+0000 7fcd9b02a700 1 -- 172.21.15.89:0/1200835482 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9415bcd0 con 0x7fcd9415ae70 2023-10-10T20:36:18.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.687+0000 7fcd9b02a700 1 -- 172.21.15.89:0/1200835482 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9404c3c0 con 0x7fcd9415ae70 2023-10-10T20:36:18.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.687+0000 7fcd88ff9700 1 -- 172.21.15.89:0/1200835482 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fcd84019030 con 0x7fcd9415ae70 2023-10-10T20:36:18.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.687+0000 7fcd88ff9700 1 --2- 172.21.15.89:0/1200835482 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcd80067040 0x7fcd800694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:18.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.688+0000 7fcd88ff9700 1 -- 172.21.15.89:0/1200835482 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fcd84085820 con 0x7fcd9415ae70 2023-10-10T20:36:18.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.688+0000 7fcd98dc6700 1 --2- 172.21.15.89:0/1200835482 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcd80067040 0x7fcd800694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:18.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.688+0000 7fcd98dc6700 1 --2- 172.21.15.89:0/1200835482 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcd80067040 0x7fcd800694f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcd90001050 tx=0x7fcd90001e50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:18.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.692+0000 7fcd88ff9700 1 -- 172.21.15.89:0/1200835482 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd84052cf0 con 0x7fcd9415ae70 2023-10-10T20:36:18.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.878+0000 7fcd9b02a700 1 -- 172.21.15.89:0/1200835482 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcd940cbba0 con 0x7fcd9415ae70 2023-10-10T20:36:18.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.879+0000 7fcd88ff9700 1 -- 172.21.15.89:0/1200835482 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v58) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcd8402b060 con 0x7fcd9415ae70 2023-10-10T20:36:18.881 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:18.882 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:36:18.882 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:18.882 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2023-10-10T20:36:18.882 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2023-10-10T20:36:18.882 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:18.883 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:36:18.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.881+0000 7fcd9b02a700 1 -- 172.21.15.89:0/1200835482 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcd80067040 msgr2=0x7fcd800694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:18.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.881+0000 7fcd9b02a700 1 --2- 172.21.15.89:0/1200835482 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcd80067040 0x7fcd800694f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcd90001050 tx=0x7fcd90001e50 comp rx=0 tx=0).stop 2023-10-10T20:36:18.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.881+0000 7fcd9b02a700 1 -- 172.21.15.89:0/1200835482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd9415ae70 msgr2=0x7fcd9406f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:18.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.881+0000 7fcd9b02a700 1 --2- 172.21.15.89:0/1200835482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd9415ae70 0x7fcd9406f420 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400c0e0 tx=0x7fcd84015000 comp rx=0 tx=0).stop 2023-10-10T20:36:18.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.881+0000 7fcd9b02a700 1 -- 172.21.15.89:0/1200835482 shutdown_connections 2023-10-10T20:36:18.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.881+0000 7fcd9b02a700 1 --2- 172.21.15.89:0/1200835482 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcd80067040 0x7fcd800694f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.881+0000 7fcd9b02a700 1 --2- 172.21.15.89:0/1200835482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd9415c2c0 0x7fcd9415c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.881+0000 7fcd9b02a700 1 --2- 172.21.15.89:0/1200835482 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd7c004810 0x7fcd94146190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.881+0000 7fcd9b02a700 1 --2- 172.21.15.89:0/1200835482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd9415ae70 0x7fcd9406f420 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:18.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.881+0000 7fcd9b02a700 1 -- 172.21.15.89:0/1200835482 >> 172.21.15.89:0/1200835482 conn(0x7fcd9406a320 msgr2=0x7fcd941570b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:18.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.881+0000 7fcd9b02a700 1 -- 172.21.15.89:0/1200835482 shutdown_connections 2023-10-10T20:36:18.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:18.881+0000 7fcd9b02a700 1 -- 172.21.15.89:0/1200835482 wait complete. 2023-10-10T20:36:18.895 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:36:18.895 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:36:19.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.012+0000 7f4c4431b700 1 Processor -- start 2023-10-10T20:36:19.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.012+0000 7f4c4431b700 1 -- start start 2023-10-10T20:36:19.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.012+0000 7f4c4431b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c3c0b42c0 0x7f4c3c0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:19.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.013+0000 7f4c4431b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3c15c320 con 0x7f4c3c155360 2023-10-10T20:36:19.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.013+0000 7f4c4431b700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3c0cefc0 con 0x7f4c3c0ceb60 2023-10-10T20:36:19.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.013+0000 7f4c4431b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c15b710 con 0x7f4c3c0b42c0 2023-10-10T20:36:19.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.013+0000 7f4c420b7700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4c3c155360 0x7f4c3c0b3bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56592/0 (socket says 172.21.15.89:56592) 2023-10-10T20:36:19.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.013+0000 7f4c420b7700 1 -- 172.21.15.89:0/241888998 learned_addr learned my addr 172.21.15.89:0/241888998 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:19.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.013+0000 7f4c418b6700 1 --2- 172.21.15.89:0/241888998 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c3c0b42c0 0x7f4c3c0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:19.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.013+0000 7f4c410b5700 1 -- 172.21.15.89:0/241888998 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1777906240 0 0) 0x7f4c3c15c320 con 0x7f4c3c155360 2023-10-10T20:36:19.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.013+0000 7f4c410b5700 1 -- 172.21.15.89:0/241888998 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c28003570 con 0x7f4c3c155360 2023-10-10T20:36:19.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.013+0000 7f4c410b5700 1 -- 172.21.15.89:0/241888998 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296880587 0 0) 0x7f4c3c0cefc0 con 0x7f4c3c0ceb60 2023-10-10T20:36:19.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.013+0000 7f4c410b5700 1 -- 172.21.15.89:0/241888998 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c3c15c320 con 0x7f4c3c0ceb60 2023-10-10T20:36:19.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.013+0000 7f4c410b5700 1 -- 172.21.15.89:0/241888998 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1866022811 0 0) 0x7f4c28003570 con 0x7f4c3c155360 2023-10-10T20:36:19.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.013+0000 7f4c418b6700 1 -- 172.21.15.89:0/241888998 >> v1:172.21.15.111:6789/0 conn(0x7f4c3c0ceb60 legacy=0x7f4c3c0cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:19.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.013+0000 7f4c418b6700 1 -- 172.21.15.89:0/241888998 >> v1:172.21.15.89:6789/0 conn(0x7f4c3c155360 legacy=0x7f4c3c0b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:19.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.013+0000 7f4c418b6700 1 -- 172.21.15.89:0/241888998 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c15b9f0 con 0x7f4c3c0b42c0 2023-10-10T20:36:19.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.013+0000 7f4c418b6700 1 --2- 172.21.15.89:0/241888998 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c3c0b42c0 0x7f4c3c0ce620 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009910 tx=0x7f4c2c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6e021b9ce0f5aaa server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:19.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.014+0000 7f4c410b5700 1 -- 172.21.15.89:0/241888998 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c380017f0 con 0x7f4c3c0b42c0 2023-10-10T20:36:19.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.014+0000 7f4c410b5700 1 -- 172.21.15.89:0/241888998 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c011420 con 0x7f4c3c0b42c0 2023-10-10T20:36:19.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.014+0000 7f4c410b5700 1 -- 172.21.15.89:0/241888998 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0117a0 con 0x7f4c3c0b42c0 2023-10-10T20:36:19.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.014+0000 7f4c4431b700 1 -- 172.21.15.89:0/241888998 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c3c0b42c0 msgr2=0x7f4c3c0ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:19.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.014+0000 7f4c4431b700 1 --2- 172.21.15.89:0/241888998 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c3c0b42c0 0x7f4c3c0ce620 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009910 tx=0x7f4c2c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:19.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.014+0000 7f4c4431b700 1 -- 172.21.15.89:0/241888998 shutdown_connections 2023-10-10T20:36:19.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.014+0000 7f4c4431b700 1 --2- 172.21.15.89:0/241888998 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c3c0b42c0 0x7f4c3c0ce620 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:19.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.014+0000 7f4c4431b700 1 -- 172.21.15.89:0/241888998 >> 172.21.15.89:0/241888998 conn(0x7f4c3c06baf0 msgr2=0x7f4c3c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:19.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.014+0000 7f4c4431b700 1 -- 172.21.15.89:0/241888998 shutdown_connections 2023-10-10T20:36:19.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.014+0000 7f4c4431b700 1 -- 172.21.15.89:0/241888998 wait complete. 2023-10-10T20:36:19.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.015+0000 7f4c4431b700 1 Processor -- start 2023-10-10T20:36:19.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.015+0000 7f4c4431b700 1 -- start start 2023-10-10T20:36:19.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.016+0000 7f4c4431b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c3c0b42c0 0x7f4c3c14ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:19.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.016+0000 7f4c4431b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c3c0ceb60 0x7f4c3c14e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:19.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.016+0000 7f4c4431b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c3c155360 0x7f4c3c14d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:19.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.016+0000 7f4c4431b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c14c130 con 0x7f4c3c155360 2023-10-10T20:36:19.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.016+0000 7f4c4431b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c14cf40 con 0x7f4c3c0b42c0 2023-10-10T20:36:19.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.016+0000 7f4c4431b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c14b890 con 0x7f4c3c0ceb60 2023-10-10T20:36:19.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.016+0000 7f4c428b8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c3c155360 0x7f4c3c14d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:19.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.016+0000 7f4c428b8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c3c155360 0x7f4c3c14d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46740/0 (socket says 172.21.15.89:46740) 2023-10-10T20:36:19.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.016+0000 7f4c428b8700 1 -- 172.21.15.89:0/1627820747 learned_addr learned my addr 172.21.15.89:0/1627820747 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:19.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.016+0000 7f4c420b7700 1 --2- 172.21.15.89:0/1627820747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c3c0b42c0 0x7f4c3c14ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:19.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.016+0000 7f4c418b6700 1 --2- 172.21.15.89:0/1627820747 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c3c0ceb60 0x7f4c3c14e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:19.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.016+0000 7f4c428b8700 1 -- 172.21.15.89:0/1627820747 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c3c0ceb60 msgr2=0x7f4c3c14e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:19.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.016+0000 7f4c428b8700 1 --2- 172.21.15.89:0/1627820747 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c3c0ceb60 0x7f4c3c14e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:19.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.016+0000 7f4c428b8700 1 -- 172.21.15.89:0/1627820747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c3c0b42c0 msgr2=0x7f4c3c14ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:19.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.016+0000 7f4c428b8700 1 --2- 172.21.15.89:0/1627820747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c3c0b42c0 0x7f4c3c14ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:19.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.016+0000 7f4c428b8700 1 -- 172.21.15.89:0/1627820747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c009420 con 0x7f4c3c155360 2023-10-10T20:36:19.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.017+0000 7f4c428b8700 1 --2- 172.21.15.89:0/1627820747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c3c155360 0x7f4c3c14d3d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400c590 tx=0x7f4c34007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:19.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.017+0000 7f4c327fc700 1 -- 172.21.15.89:0/1627820747 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34016070 con 0x7f4c3c155360 2023-10-10T20:36:19.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.017+0000 7f4c327fc700 1 -- 172.21.15.89:0/1627820747 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c34011590 con 0x7f4c3c155360 2023-10-10T20:36:19.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.017+0000 7f4c4431b700 1 -- 172.21.15.89:0/1627820747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3c14bb70 con 0x7f4c3c155360 2023-10-10T20:36:19.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.017+0000 7f4c327fc700 1 -- 172.21.15.89:0/1627820747 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34011970 con 0x7f4c3c155360 2023-10-10T20:36:19.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.017+0000 7f4c4431b700 1 -- 172.21.15.89:0/1627820747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3c14ee00 con 0x7f4c3c155360 2023-10-10T20:36:19.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.018+0000 7f4c4431b700 1 -- 172.21.15.89:0/1627820747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3c14cd10 con 0x7f4c3c155360 2023-10-10T20:36:19.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.018+0000 7f4c327fc700 1 -- 172.21.15.89:0/1627820747 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f4c34012420 con 0x7f4c3c155360 2023-10-10T20:36:19.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.018+0000 7f4c327fc700 1 --2- 172.21.15.89:0/1627820747 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c20066fc0 0x7f4c20069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:19.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.019+0000 7f4c327fc700 1 -- 172.21.15.89:0/1627820747 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4c34083f60 con 0x7f4c3c155360 2023-10-10T20:36:19.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.019+0000 7f4c420b7700 1 --2- 172.21.15.89:0/1627820747 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c20066fc0 0x7f4c20069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:19.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.019+0000 7f4c420b7700 1 --2- 172.21.15.89:0/1627820747 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c20066fc0 0x7f4c20069470 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4c38005fd0 tx=0x7f4c38000f40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:19.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.023+0000 7f4c327fc700 1 -- 172.21.15.89:0/1627820747 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c3c14cd10 con 0x7f4c3c155360 2023-10-10T20:36:19.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.203+0000 7f4c4431b700 1 -- 172.21.15.89:0/1627820747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4c3c14cd10 con 0x7f4c3c155360 2023-10-10T20:36:19.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.482+0000 7f4c327fc700 1 -- 172.21.15.89:0/1627820747 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4c3c14cd10 con 0x7f4c3c155360 2023-10-10T20:36:19.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.485+0000 7f4c4431b700 1 -- 172.21.15.89:0/1627820747 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c20066fc0 msgr2=0x7f4c20069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:19.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.485+0000 7f4c4431b700 1 --2- 172.21.15.89:0/1627820747 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c20066fc0 0x7f4c20069470 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4c38005fd0 tx=0x7f4c38000f40 comp rx=0 tx=0).stop 2023-10-10T20:36:19.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.485+0000 7f4c4431b700 1 -- 172.21.15.89:0/1627820747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c3c155360 msgr2=0x7f4c3c14d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:19.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.485+0000 7f4c4431b700 1 --2- 172.21.15.89:0/1627820747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c3c155360 0x7f4c3c14d3d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400c590 tx=0x7f4c34007d80 comp rx=0 tx=0).stop 2023-10-10T20:36:19.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.485+0000 7f4c4431b700 1 -- 172.21.15.89:0/1627820747 shutdown_connections 2023-10-10T20:36:19.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.485+0000 7f4c4431b700 1 --2- 172.21.15.89:0/1627820747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c3c155360 0x7f4c3c14d3d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:19.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.485+0000 7f4c4431b700 1 --2- 172.21.15.89:0/1627820747 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c20066fc0 0x7f4c20069470 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:19.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.485+0000 7f4c4431b700 1 --2- 172.21.15.89:0/1627820747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c3c0b42c0 0x7f4c3c14ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:19.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.485+0000 7f4c4431b700 1 --2- 172.21.15.89:0/1627820747 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c3c0ceb60 0x7f4c3c14e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:19.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.485+0000 7f4c4431b700 1 -- 172.21.15.89:0/1627820747 >> 172.21.15.89:0/1627820747 conn(0x7f4c3c06baf0 msgr2=0x7f4c3c14c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:19.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.486+0000 7f4c4431b700 1 -- 172.21.15.89:0/1627820747 shutdown_connections 2023-10-10T20:36:19.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.486+0000 7f4c4431b700 1 -- 172.21.15.89:0/1627820747 wait complete. 2023-10-10T20:36:19.490 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2023-10-10T20:36:19.501 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:36:19.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.621+0000 7f7b0372e700 1 Processor -- start 2023-10-10T20:36:19.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.621+0000 7f7b0372e700 1 -- start start 2023-10-10T20:36:19.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.622+0000 7f7b0372e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7afc069c70 0x7f7afc0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:19.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.622+0000 7f7b0372e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7afc0af320 0x7f7afc0cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:19.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.622+0000 7f7b0372e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7afc0ce380 con 0x7f7afc159950 2023-10-10T20:36:19.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.622+0000 7f7b0372e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7afc0cec20 con 0x7f7afc069c70 2023-10-10T20:36:19.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.622+0000 7f7b0372e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7afc0afa90 con 0x7f7afc0af320 2023-10-10T20:36:19.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.622+0000 7f7b00cc9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7afc069c70 0x7f7afc0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:19.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.622+0000 7f7b014ca700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7afc159950 0x7f7afc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56596/0 (socket says 172.21.15.89:56596) 2023-10-10T20:36:19.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.622+0000 7f7b00cc9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7afc069c70 0x7f7afc0cbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46754/0 (socket says 172.21.15.89:46754) 2023-10-10T20:36:19.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.622+0000 7f7b014ca700 1 -- 172.21.15.89:0/2257395053 learned_addr learned my addr 172.21.15.89:0/2257395053 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:19.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.623+0000 7f7b01ccb700 1 --2- 172.21.15.89:0/2257395053 >> v2:172.21.15.111:3300/0 conn(0x7f7afc0af320 0x7f7afc0cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:19.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.623+0000 7f7b00cc9700 1 -- 172.21.15.89:0/2257395053 >> v2:172.21.15.111:3300/0 conn(0x7f7afc0af320 msgr2=0x7f7afc0cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:19.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.623+0000 7f7b00cc9700 1 --2- 172.21.15.89:0/2257395053 >> v2:172.21.15.111:3300/0 conn(0x7f7afc0af320 0x7f7afc0cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:19.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.623+0000 7f7b00cc9700 1 -- 172.21.15.89:0/2257395053 >> v1:172.21.15.89:6789/0 conn(0x7f7afc159950 legacy=0x7f7afc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:19.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.623+0000 7f7b00cc9700 1 -- 172.21.15.89:0/2257395053 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc0ce5d0 con 0x7f7afc069c70 2023-10-10T20:36:19.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.623+0000 7f7af3fff700 1 -- 172.21.15.89:0/2257395053 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890717824 0 0) 0x7f7afc0ce380 con 0x7f7afc159950 2023-10-10T20:36:19.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.623+0000 7f7b00cc9700 1 --2- 172.21.15.89:0/2257395053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7afc069c70 0x7f7afc0cbb90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0cb5c0 tx=0x7f7aec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55467545b49873d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:19.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.623+0000 7f7af3fff700 1 -- 172.21.15.89:0/2257395053 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec00b6b0 con 0x7f7afc069c70 2023-10-10T20:36:19.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.623+0000 7f7af3fff700 1 -- 172.21.15.89:0/2257395053 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec00b810 con 0x7f7afc069c70 2023-10-10T20:36:19.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.623+0000 7f7af3fff700 1 -- 172.21.15.89:0/2257395053 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec00bbb0 con 0x7f7afc069c70 2023-10-10T20:36:19.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.623+0000 7f7b0372e700 1 -- 172.21.15.89:0/2257395053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7afc069c70 msgr2=0x7f7afc0cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:19.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.623+0000 7f7b0372e700 1 --2- 172.21.15.89:0/2257395053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7afc069c70 0x7f7afc0cbb90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0cb5c0 tx=0x7f7aec0091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:19.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.623+0000 7f7b0372e700 1 -- 172.21.15.89:0/2257395053 shutdown_connections 2023-10-10T20:36:19.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.623+0000 7f7b0372e700 1 --2- 172.21.15.89:0/2257395053 >> v2:172.21.15.111:3300/0 conn(0x7f7afc0af320 0x7f7afc0cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:19.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.623+0000 7f7b0372e700 1 --2- 172.21.15.89:0/2257395053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7afc069c70 0x7f7afc0cbb90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:19.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.623+0000 7f7b0372e700 1 -- 172.21.15.89:0/2257395053 >> 172.21.15.89:0/2257395053 conn(0x7f7afc068e00 msgr2=0x7f7afc0d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:19.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.624+0000 7f7b0372e700 1 -- 172.21.15.89:0/2257395053 shutdown_connections 2023-10-10T20:36:19.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.624+0000 7f7b0372e700 1 -- 172.21.15.89:0/2257395053 wait complete. 2023-10-10T20:36:19.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.625+0000 7f7b0372e700 1 Processor -- start 2023-10-10T20:36:19.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.625+0000 7f7b0372e700 1 -- start start 2023-10-10T20:36:19.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b0372e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7afc069c70 0x7f7afc0b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:19.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b0372e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7afc0af320 0x7f7afc16d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:19.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b0372e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7afc159950 0x7f7afc16a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:19.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b0372e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc152c40 con 0x7f7afc069c70 2023-10-10T20:36:19.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b0372e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc159e50 con 0x7f7afc159950 2023-10-10T20:36:19.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b0372e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc16a670 con 0x7f7afc0af320 2023-10-10T20:36:19.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b014ca700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7afc069c70 0x7f7afc0b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:19.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b014ca700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7afc069c70 0x7f7afc0b5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46756/0 (socket says 172.21.15.89:46756) 2023-10-10T20:36:19.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b014ca700 1 -- 172.21.15.89:0/3886746065 learned_addr learned my addr 172.21.15.89:0/3886746065 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:19.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b01ccb700 1 --2- 172.21.15.89:0/3886746065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7afc159950 0x7f7afc16a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:19.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b00cc9700 1 --2- 172.21.15.89:0/3886746065 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7afc0af320 0x7f7afc16d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:19.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b014ca700 1 -- 172.21.15.89:0/3886746065 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7afc0af320 msgr2=0x7f7afc16d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:19.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b014ca700 1 --2- 172.21.15.89:0/3886746065 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7afc0af320 0x7f7afc16d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:19.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b014ca700 1 -- 172.21.15.89:0/3886746065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7afc159950 msgr2=0x7f7afc16a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:19.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b014ca700 1 --2- 172.21.15.89:0/3886746065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7afc159950 0x7f7afc16a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:19.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b014ca700 1 -- 172.21.15.89:0/3886746065 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec009420 con 0x7f7afc069c70 2023-10-10T20:36:19.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b01ccb700 1 --2- 172.21.15.89:0/3886746065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7afc159950 0x7f7afc16a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:19.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b014ca700 1 --2- 172.21.15.89:0/3886746065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7afc069c70 0x7f7afc0b5580 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7af800cc70 tx=0x7f7af8001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:19.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7af17fa700 1 -- 172.21.15.89:0/3886746065 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af800ce80 con 0x7f7afc069c70 2023-10-10T20:36:19.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7af17fa700 1 -- 172.21.15.89:0/3886746065 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af800ab70 con 0x7f7afc069c70 2023-10-10T20:36:19.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b0372e700 1 -- 172.21.15.89:0/3886746065 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7afc0b5b60 con 0x7f7afc069c70 2023-10-10T20:36:19.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7af17fa700 1 -- 172.21.15.89:0/3886746065 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af80070f0 con 0x7f7afc069c70 2023-10-10T20:36:19.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.626+0000 7f7b0372e700 1 -- 172.21.15.89:0/3886746065 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7afc0b6120 con 0x7f7afc069c70 2023-10-10T20:36:19.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.627+0000 7f7b0372e700 1 -- 172.21.15.89:0/3886746065 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7afc0718b0 con 0x7f7afc069c70 2023-10-10T20:36:19.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.628+0000 7f7af17fa700 1 -- 172.21.15.89:0/3886746065 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f7af80074e0 con 0x7f7afc069c70 2023-10-10T20:36:19.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.628+0000 7f7af17fa700 1 --2- 172.21.15.89:0/3886746065 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7ae0066fc0 0x7f7ae0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:19.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.628+0000 7f7af17fa700 1 -- 172.21.15.89:0/3886746065 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f7af8084810 con 0x7f7afc069c70 2023-10-10T20:36:19.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.628+0000 7f7b00cc9700 1 --2- 172.21.15.89:0/3886746065 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7ae0066fc0 0x7f7ae0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:19.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.629+0000 7f7b00cc9700 1 --2- 172.21.15.89:0/3886746065 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7ae0066fc0 0x7f7ae0069470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0093f0 tx=0x7f7aec017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:19.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.633+0000 7f7af17fa700 1 -- 172.21.15.89:0/3886746065 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7af8051cf0 con 0x7f7afc069c70 2023-10-10T20:36:19.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:19.816+0000 7f7b0372e700 1 -- 172.21.15.89:0/3886746065 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7afc0ce8b0 con 0x7f7afc069c70 2023-10-10T20:36:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.487+0000 7f7af17fa700 1 -- 172.21.15.89:0/3886746065 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v60) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7af8055310 con 0x7f7afc069c70 2023-10-10T20:36:20.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.492+0000 7f7b0372e700 1 -- 172.21.15.89:0/3886746065 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7ae0066fc0 msgr2=0x7f7ae0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:20.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.492+0000 7f7b0372e700 1 --2- 172.21.15.89:0/3886746065 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7ae0066fc0 0x7f7ae0069470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0093f0 tx=0x7f7aec017000 comp rx=0 tx=0).stop 2023-10-10T20:36:20.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.492+0000 7f7b0372e700 1 -- 172.21.15.89:0/3886746065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7afc069c70 msgr2=0x7f7afc0b5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:20.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.492+0000 7f7b0372e700 1 --2- 172.21.15.89:0/3886746065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7afc069c70 0x7f7afc0b5580 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7af800cc70 tx=0x7f7af8001a30 comp rx=0 tx=0).stop 2023-10-10T20:36:20.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.493+0000 7f7b0372e700 1 -- 172.21.15.89:0/3886746065 shutdown_connections 2023-10-10T20:36:20.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.493+0000 7f7b0372e700 1 --2- 172.21.15.89:0/3886746065 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7ae0066fc0 0x7f7ae0069470 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.493+0000 7f7b0372e700 1 --2- 172.21.15.89:0/3886746065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7afc159950 0x7f7afc16a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.493+0000 7f7b0372e700 1 --2- 172.21.15.89:0/3886746065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7afc069c70 0x7f7afc0b5580 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.493+0000 7f7b0372e700 1 --2- 172.21.15.89:0/3886746065 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7afc0af320 0x7f7afc16d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.493+0000 7f7b0372e700 1 -- 172.21.15.89:0/3886746065 >> 172.21.15.89:0/3886746065 conn(0x7f7afc068e00 msgr2=0x7f7afc069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.494+0000 7f7b0372e700 1 -- 172.21.15.89:0/3886746065 shutdown_connections 2023-10-10T20:36:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.495+0000 7f7b0372e700 1 -- 172.21.15.89:0/3886746065 wait complete. 2023-10-10T20:36:20.499 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2023-10-10T20:36:20.515 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:36:20.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.630+0000 7f137359e700 1 Processor -- start 2023-10-10T20:36:20.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.630+0000 7f137359e700 1 -- start start 2023-10-10T20:36:20.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.631+0000 7f137359e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f13740cc9d0 0x7f137415d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:20.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.631+0000 7f137359e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13740cd0f0 0x7f137415d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:20.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.631+0000 7f137359e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1374150b50 con 0x7f13740cc600 2023-10-10T20:36:20.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.631+0000 7f137359e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13740cede0 con 0x7f13740cc9d0 2023-10-10T20:36:20.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.631+0000 7f137359e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13740b2260 con 0x7f13740cd0f0 2023-10-10T20:36:20.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.631+0000 7f1369d9b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f13740cc9d0 0x7f137415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:20.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.631+0000 7f1372d9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13740cd0f0 0x7f137415d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:20.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.631+0000 7f137259c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f13740cc600 0x7f13740b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:53136/0 (socket says 172.21.15.89:53136) 2023-10-10T20:36:20.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.631+0000 7f137259c700 1 -- 172.21.15.89:0/1066698728 learned_addr learned my addr 172.21.15.89:0/1066698728 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:20.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.632+0000 7f1372d9d700 1 -- 172.21.15.89:0/1066698728 >> v2:172.21.15.111:3300/0 conn(0x7f13740cc9d0 msgr2=0x7f137415d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:20.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.632+0000 7f1372d9d700 1 --2- 172.21.15.89:0/1066698728 >> v2:172.21.15.111:3300/0 conn(0x7f13740cc9d0 0x7f137415d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:20.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.632+0000 7f1372d9d700 1 -- 172.21.15.89:0/1066698728 >> v1:172.21.15.111:6789/0 conn(0x7f13740cc600 legacy=0x7f13740b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:20.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.632+0000 7f1372d9d700 1 -- 172.21.15.89:0/1066698728 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13740b2540 con 0x7f13740cd0f0 2023-10-10T20:36:20.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.632+0000 7f1369d9b700 1 --2- 172.21.15.89:0/1066698728 >> v2:172.21.15.111:3300/0 conn(0x7f13740cc9d0 0x7f137415d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:36:20.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.632+0000 7f1372d9d700 1 --2- 172.21.15.89:0/1066698728 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13740cd0f0 0x7f137415d7c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f136c009e60 tx=0x7f136c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4632f8846010b1f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:20.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.632+0000 7f1371d9b700 1 -- 172.21.15.89:0/1066698728 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c00e070 con 0x7f13740cd0f0 2023-10-10T20:36:20.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.632+0000 7f1371d9b700 1 -- 172.21.15.89:0/1066698728 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c00bb60 con 0x7f13740cd0f0 2023-10-10T20:36:20.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.632+0000 7f1371d9b700 1 -- 172.21.15.89:0/1066698728 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c012430 con 0x7f13740cd0f0 2023-10-10T20:36:20.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.632+0000 7f137359e700 1 -- 172.21.15.89:0/1066698728 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13740cd0f0 msgr2=0x7f137415d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:20.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.632+0000 7f137359e700 1 --2- 172.21.15.89:0/1066698728 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13740cd0f0 0x7f137415d7c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f136c009e60 tx=0x7f136c009780 comp rx=0 tx=0).stop 2023-10-10T20:36:20.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.632+0000 7f137359e700 1 -- 172.21.15.89:0/1066698728 shutdown_connections 2023-10-10T20:36:20.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.632+0000 7f137359e700 1 --2- 172.21.15.89:0/1066698728 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13740cd0f0 0x7f137415d7c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:20.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.633+0000 7f137359e700 1 --2- 172.21.15.89:0/1066698728 >> v2:172.21.15.111:3300/0 conn(0x7f13740cc9d0 0x7f137415d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:20.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.633+0000 7f137359e700 1 -- 172.21.15.89:0/1066698728 >> 172.21.15.89:0/1066698728 conn(0x7f13740d0630 msgr2=0x7f13740d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:20.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.633+0000 7f137359e700 1 -- 172.21.15.89:0/1066698728 shutdown_connections 2023-10-10T20:36:20.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.633+0000 7f137359e700 1 -- 172.21.15.89:0/1066698728 wait complete. 2023-10-10T20:36:20.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.634+0000 7f137359e700 1 Processor -- start 2023-10-10T20:36:20.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.634+0000 7f137359e700 1 -- start start 2023-10-10T20:36:20.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f137359e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13740cc600 0x7f1374153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:20.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f137359e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13740cc9d0 0x7f137414e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:20.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f137359e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13740cd0f0 0x7f1374151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:20.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f137359e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1374156220 con 0x7f13740cd0f0 2023-10-10T20:36:20.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f137359e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1374156500 con 0x7f13740cc9d0 2023-10-10T20:36:20.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f137359e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13741567e0 con 0x7f13740cc600 2023-10-10T20:36:20.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f1372d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13740cd0f0 0x7f1374151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:20.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f1372d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13740cd0f0 0x7f1374151b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46758/0 (socket says 172.21.15.89:46758) 2023-10-10T20:36:20.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f1372d9d700 1 -- 172.21.15.89:0/2132392913 learned_addr learned my addr 172.21.15.89:0/2132392913 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:20.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f1369d9b700 1 --2- 172.21.15.89:0/2132392913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13740cc9d0 0x7f137414e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:20.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f137259c700 1 --2- 172.21.15.89:0/2132392913 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13740cc600 0x7f1374153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:20.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f1372d9d700 1 -- 172.21.15.89:0/2132392913 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13740cc600 msgr2=0x7f1374153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:20.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f1372d9d700 1 --2- 172.21.15.89:0/2132392913 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13740cc600 0x7f1374153b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:20.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f1372d9d700 1 -- 172.21.15.89:0/2132392913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13740cc9d0 msgr2=0x7f137414e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:20.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f1372d9d700 1 --2- 172.21.15.89:0/2132392913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13740cc9d0 0x7f137414e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:20.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f1372d9d700 1 -- 172.21.15.89:0/2132392913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135c009420 con 0x7f13740cd0f0 2023-10-10T20:36:20.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f1372d9d700 1 --2- 172.21.15.89:0/2132392913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13740cd0f0 0x7f1374151b70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1374071390 tx=0x7f136c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:20.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f136affd700 1 -- 172.21.15.89:0/2132392913 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c012ea0 con 0x7f13740cd0f0 2023-10-10T20:36:20.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f136affd700 1 -- 172.21.15.89:0/2132392913 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c027470 con 0x7f13740cd0f0 2023-10-10T20:36:20.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f137359e700 1 -- 172.21.15.89:0/2132392913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136c009a20 con 0x7f13740cd0f0 2023-10-10T20:36:20.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f136affd700 1 -- 172.21.15.89:0/2132392913 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c027850 con 0x7f13740cd0f0 2023-10-10T20:36:20.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.635+0000 7f137359e700 1 -- 172.21.15.89:0/2132392913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1374156da0 con 0x7f13740cd0f0 2023-10-10T20:36:20.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.636+0000 7f137359e700 1 -- 172.21.15.89:0/2132392913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13740889d0 con 0x7f13740cd0f0 2023-10-10T20:36:20.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.637+0000 7f136affd700 1 -- 172.21.15.89:0/2132392913 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f136c024070 con 0x7f13740cd0f0 2023-10-10T20:36:20.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.637+0000 7f136affd700 1 --2- 172.21.15.89:0/2132392913 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1354066fc0 0x7f1354069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:20.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.637+0000 7f136affd700 1 -- 172.21.15.89:0/2132392913 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f136c086fb0 con 0x7f13740cd0f0 2023-10-10T20:36:20.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.637+0000 7f137259c700 1 --2- 172.21.15.89:0/2132392913 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1354066fc0 0x7f1354069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:20.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.638+0000 7f137259c700 1 --2- 172.21.15.89:0/2132392913 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1354066fc0 0x7f1354069470 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1364007130 tx=0x7f1364007eb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:20.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.642+0000 7f136affd700 1 -- 172.21.15.89:0/2132392913 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f136c054290 con 0x7f13740cd0f0 2023-10-10T20:36:20.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:20.865+0000 7f137359e700 1 -- 172.21.15.89:0/2132392913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f13740b2820 con 0x7f13740cd0f0 2023-10-10T20:36:21.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.516+0000 7f136affd700 1 -- 172.21.15.89:0/2132392913 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v34) v1 ==== 174+0+0 (secure 0 0 0) 0x7f136c0578b0 con 0x7f13740cd0f0 2023-10-10T20:36:21.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.520+0000 7f137359e700 1 -- 172.21.15.89:0/2132392913 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1354066fc0 msgr2=0x7f1354069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:21.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.520+0000 7f137359e700 1 --2- 172.21.15.89:0/2132392913 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1354066fc0 0x7f1354069470 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1364007130 tx=0x7f1364007eb0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.520+0000 7f137359e700 1 -- 172.21.15.89:0/2132392913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13740cd0f0 msgr2=0x7f1374151b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:21.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.520+0000 7f137359e700 1 --2- 172.21.15.89:0/2132392913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13740cd0f0 0x7f1374151b70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1374071390 tx=0x7f136c019000 comp rx=0 tx=0).stop 2023-10-10T20:36:21.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.520+0000 7f137359e700 1 -- 172.21.15.89:0/2132392913 shutdown_connections 2023-10-10T20:36:21.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.520+0000 7f137359e700 1 --2- 172.21.15.89:0/2132392913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13740cd0f0 0x7f1374151b70 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.520+0000 7f137359e700 1 --2- 172.21.15.89:0/2132392913 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f13740cc600 0x7f1374153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.520+0000 7f137359e700 1 --2- 172.21.15.89:0/2132392913 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1354066fc0 0x7f1354069470 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.520+0000 7f137359e700 1 --2- 172.21.15.89:0/2132392913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13740cc9d0 0x7f137414e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.520+0000 7f137359e700 1 -- 172.21.15.89:0/2132392913 >> 172.21.15.89:0/2132392913 conn(0x7f13740d0630 msgr2=0x7f1374069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:21.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.521+0000 7f137359e700 1 -- 172.21.15.89:0/2132392913 shutdown_connections 2023-10-10T20:36:21.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.521+0000 7f137359e700 1 -- 172.21.15.89:0/2132392913 wait complete. 2023-10-10T20:36:21.526 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 10 and data pool 11 2023-10-10T20:36:21.541 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:36:21.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.655+0000 7fb1858f5700 1 Processor -- start 2023-10-10T20:36:21.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.655+0000 7fb1858f5700 1 -- start start 2023-10-10T20:36:21.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.656+0000 7fb1858f5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb18006bcd0 0x7fb180140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:21.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.656+0000 7fb1858f5700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb18006c0a0 0x7fb180141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.656+0000 7fb1858f5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb1800c8e00 0x7fb1800c9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.656+0000 7fb1858f5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18015c2a0 con 0x7fb18006bcd0 2023-10-10T20:36:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.656+0000 7fb1858f5700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18015b690 con 0x7fb18006c0a0 2023-10-10T20:36:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.656+0000 7fb1858f5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18015b970 con 0x7fb1800c8e00 2023-10-10T20:36:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.656+0000 7fb17effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb18006bcd0 0x7fb180140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:21.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.656+0000 7fb17effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb18006bcd0 0x7fb180140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46760/0 (socket says 172.21.15.89:46760) 2023-10-10T20:36:21.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.656+0000 7fb17effd700 1 -- 172.21.15.89:0/221479076 learned_addr learned my addr 172.21.15.89:0/221479076 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:21.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.657+0000 7fb1767fc700 1 --2- 172.21.15.89:0/221479076 >> v2:172.21.15.111:3300/0 conn(0x7fb18006c0a0 0x7fb180141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:21.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.657+0000 7fb17effd700 1 -- 172.21.15.89:0/221479076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb1800c8e00 msgr2=0x7fb1800c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:21.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.657+0000 7fb17f7fe700 1 --2- 172.21.15.89:0/221479076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb1800c8e00 0x7fb1800c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:21.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.657+0000 7fb17effd700 1 --2- 172.21.15.89:0/221479076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb1800c8e00 0x7fb1800c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.657+0000 7fb17effd700 1 -- 172.21.15.89:0/221479076 >> v2:172.21.15.111:3300/0 conn(0x7fb18006c0a0 msgr2=0x7fb180141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.657+0000 7fb17effd700 1 --2- 172.21.15.89:0/221479076 >> v2:172.21.15.111:3300/0 conn(0x7fb18006c0a0 0x7fb180141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.657+0000 7fb17effd700 1 -- 172.21.15.89:0/221479076 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18015bc50 con 0x7fb18006bcd0 2023-10-10T20:36:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.657+0000 7fb1767fc700 1 --2- 172.21.15.89:0/221479076 >> v2:172.21.15.111:3300/0 conn(0x7fb18006c0a0 0x7fb180141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.657+0000 7fb17f7fe700 1 --2- 172.21.15.89:0/221479076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb1800c8e00 0x7fb1800c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.657+0000 7fb17effd700 1 --2- 172.21.15.89:0/221479076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb18006bcd0 0x7fb180140df0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb18006c6e0 tx=0x7fb170009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77839d6ee5d7cb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.657+0000 7fb17e7fc700 1 -- 172.21.15.89:0/221479076 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17000b860 con 0x7fb18006bcd0 2023-10-10T20:36:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.657+0000 7fb17e7fc700 1 -- 172.21.15.89:0/221479076 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17000b9c0 con 0x7fb18006bcd0 2023-10-10T20:36:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.657+0000 7fb17e7fc700 1 -- 172.21.15.89:0/221479076 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17000bd60 con 0x7fb18006bcd0 2023-10-10T20:36:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.658+0000 7fb1858f5700 1 -- 172.21.15.89:0/221479076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb18006bcd0 msgr2=0x7fb180140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.658+0000 7fb1858f5700 1 --2- 172.21.15.89:0/221479076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb18006bcd0 0x7fb180140df0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb18006c6e0 tx=0x7fb170009250 comp rx=0 tx=0).stop 2023-10-10T20:36:21.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.658+0000 7fb1858f5700 1 -- 172.21.15.89:0/221479076 shutdown_connections 2023-10-10T20:36:21.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.658+0000 7fb1858f5700 1 --2- 172.21.15.89:0/221479076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb1800c8e00 0x7fb1800c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.658+0000 7fb1858f5700 1 --2- 172.21.15.89:0/221479076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb18006bcd0 0x7fb180140df0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.658+0000 7fb1858f5700 1 --2- 172.21.15.89:0/221479076 >> v2:172.21.15.111:3300/0 conn(0x7fb18006c0a0 0x7fb180141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.658+0000 7fb1858f5700 1 -- 172.21.15.89:0/221479076 >> 172.21.15.89:0/221479076 conn(0x7fb1800ce600 msgr2=0x7fb1800cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.658+0000 7fb1858f5700 1 -- 172.21.15.89:0/221479076 shutdown_connections 2023-10-10T20:36:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.658+0000 7fb1858f5700 1 -- 172.21.15.89:0/221479076 wait complete. 2023-10-10T20:36:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.659+0000 7fb1858f5700 1 Processor -- start 2023-10-10T20:36:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.659+0000 7fb1858f5700 1 -- start start 2023-10-10T20:36:21.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb1858f5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18006bcd0 0x7fb180157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:21.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb1858f5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb18006c0a0 0x7fb180158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:21.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb1858f5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1800c8e00 0x7fb180168d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb1858f5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1801582f0 con 0x7fb1800c8e00 2023-10-10T20:36:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb1858f5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1801692c0 con 0x7fb18006bcd0 2023-10-10T20:36:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb1858f5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1801576e0 con 0x7fb18006c0a0 2023-10-10T20:36:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb17f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1800c8e00 0x7fb180168d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb17f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1800c8e00 0x7fb180168d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46766/0 (socket says 172.21.15.89:46766) 2023-10-10T20:36:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb17f7fe700 1 -- 172.21.15.89:0/1720743616 learned_addr learned my addr 172.21.15.89:0/1720743616 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:21.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb17effd700 1 --2- 172.21.15.89:0/1720743616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18006bcd0 0x7fb180157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:21.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb1767fc700 1 --2- 172.21.15.89:0/1720743616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb18006c0a0 0x7fb180158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:21.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb17f7fe700 1 -- 172.21.15.89:0/1720743616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb18006c0a0 msgr2=0x7fb180158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:21.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb17f7fe700 1 --2- 172.21.15.89:0/1720743616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb18006c0a0 0x7fb180158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb17f7fe700 1 -- 172.21.15.89:0/1720743616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18006bcd0 msgr2=0x7fb180157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:21.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb17f7fe700 1 --2- 172.21.15.89:0/1720743616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18006bcd0 0x7fb180157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb17f7fe700 1 -- 172.21.15.89:0/1720743616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1700094f0 con 0x7fb1800c8e00 2023-10-10T20:36:21.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb1767fc700 1 --2- 172.21.15.89:0/1720743616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb18006c0a0 0x7fb180158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:21.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb17f7fe700 1 --2- 172.21.15.89:0/1720743616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1800c8e00 0x7fb180168d80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb17800cef0 tx=0x7fb178007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:21.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb1777fe700 1 -- 172.21.15.89:0/1720743616 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17800ae00 con 0x7fb1800c8e00 2023-10-10T20:36:21.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb1777fe700 1 -- 172.21.15.89:0/1720743616 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb178016420 con 0x7fb1800c8e00 2023-10-10T20:36:21.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb1777fe700 1 -- 172.21.15.89:0/1720743616 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1780167e0 con 0x7fb1800c8e00 2023-10-10T20:36:21.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb1858f5700 1 -- 172.21.15.89:0/1720743616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1801579c0 con 0x7fb1800c8e00 2023-10-10T20:36:21.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.660+0000 7fb1858f5700 1 -- 172.21.15.89:0/1720743616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb180157f80 con 0x7fb1800c8e00 2023-10-10T20:36:21.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.661+0000 7fb1858f5700 1 -- 172.21.15.89:0/1720743616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1801584a0 con 0x7fb1800c8e00 2023-10-10T20:36:21.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.662+0000 7fb1777fe700 1 -- 172.21.15.89:0/1720743616 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fb178016bd0 con 0x7fb1800c8e00 2023-10-10T20:36:21.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.662+0000 7fb1777fe700 1 --2- 172.21.15.89:0/1720743616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb160066fc0 0x7fb160069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:21.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.662+0000 7fb1777fe700 1 -- 172.21.15.89:0/1720743616 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb178012070 con 0x7fb1800c8e00 2023-10-10T20:36:21.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.662+0000 7fb17effd700 1 --2- 172.21.15.89:0/1720743616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb160066fc0 0x7fb160069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:21.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.662+0000 7fb17effd700 1 --2- 172.21.15.89:0/1720743616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb160066fc0 0x7fb160069470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb18006c6e0 tx=0x7fb170017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:21.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.666+0000 7fb1777fe700 1 -- 172.21.15.89:0/1720743616 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb17804e5a0 con 0x7fb1800c8e00 2023-10-10T20:36:21.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.850+0000 7fb1858f5700 1 -- 172.21.15.89:0/1720743616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb18004c3c0 con 0x7fb1800c8e00 2023-10-10T20:36:21.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.850+0000 7fb1777fe700 1 -- 172.21.15.89:0/1720743616 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+15366 (secure 0 0 0) 0x7fb1801584a0 con 0x7fb1800c8e00 2023-10-10T20:36:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:21.853 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":61,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:36:20.867024+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:36:19.221740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-10T20:36:19.834213+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:36:21.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.852+0000 7fb1858f5700 1 -- 172.21.15.89:0/1720743616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb160066fc0 msgr2=0x7fb160069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:21.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.852+0000 7fb1858f5700 1 --2- 172.21.15.89:0/1720743616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb160066fc0 0x7fb160069470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb18006c6e0 tx=0x7fb170017000 comp rx=0 tx=0).stop 2023-10-10T20:36:21.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.852+0000 7fb1858f5700 1 -- 172.21.15.89:0/1720743616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1800c8e00 msgr2=0x7fb180168d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:21.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.852+0000 7fb1858f5700 1 --2- 172.21.15.89:0/1720743616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1800c8e00 0x7fb180168d80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb17800cef0 tx=0x7fb178007860 comp rx=0 tx=0).stop 2023-10-10T20:36:21.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.852+0000 7fb1858f5700 1 -- 172.21.15.89:0/1720743616 shutdown_connections 2023-10-10T20:36:21.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.852+0000 7fb1858f5700 1 --2- 172.21.15.89:0/1720743616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1800c8e00 0x7fb180168d80 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.852+0000 7fb1858f5700 1 --2- 172.21.15.89:0/1720743616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb160066fc0 0x7fb160069470 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.853+0000 7fb1858f5700 1 --2- 172.21.15.89:0/1720743616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18006bcd0 0x7fb180157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.853+0000 7fb1858f5700 1 --2- 172.21.15.89:0/1720743616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb18006c0a0 0x7fb180158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.853+0000 7fb1858f5700 1 -- 172.21.15.89:0/1720743616 >> 172.21.15.89:0/1720743616 conn(0x7fb1800ce600 msgr2=0x7fb1800c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:21.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.853+0000 7fb1858f5700 1 -- 172.21.15.89:0/1720743616 shutdown_connections 2023-10-10T20:36:21.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.853+0000 7fb1858f5700 1 -- 172.21.15.89:0/1720743616 wait complete. 2023-10-10T20:36:21.867 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:36:21.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.980+0000 7f36a8597700 1 Processor -- start 2023-10-10T20:36:21.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.980+0000 7f36a8597700 1 -- start start 2023-10-10T20:36:21.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.981+0000 7f36a8597700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f36a0159890 0x7f36a015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:21.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.981+0000 7f36a8597700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a015a160 con 0x7f36a0069560 2023-10-10T20:36:21.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.981+0000 7f36a8597700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a00d13a0 con 0x7f36a0159c60 2023-10-10T20:36:21.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.981+0000 7f36a8597700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a015e140 con 0x7f36a0159890 2023-10-10T20:36:21.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.981+0000 7f36a6333700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f36a0159890 0x7f36a015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:21.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.981+0000 7f36a6b34700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f36a0069560 0x7f36a00ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56612/0 (socket says 172.21.15.89:56612) 2023-10-10T20:36:21.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.981+0000 7f36a6333700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f36a0159890 0x7f36a015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46780/0 (socket says 172.21.15.89:46780) 2023-10-10T20:36:21.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.981+0000 7f36a6b34700 1 -- 172.21.15.89:0/4226032702 learned_addr learned my addr 172.21.15.89:0/4226032702 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:21.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.981+0000 7f36a6333700 1 -- 172.21.15.89:0/4226032702 >> v1:172.21.15.111:6789/0 conn(0x7f36a0159c60 legacy=0x7f36a015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:21.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.981+0000 7f36a6333700 1 -- 172.21.15.89:0/4226032702 >> v1:172.21.15.89:6789/0 conn(0x7f36a0069560 legacy=0x7f36a00ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:21.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.981+0000 7f36a5331700 1 -- 172.21.15.89:0/4226032702 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2937894071 0 0) 0x7f36a015a160 con 0x7f36a0069560 2023-10-10T20:36:21.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.981+0000 7f36a6333700 1 -- 172.21.15.89:0/4226032702 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a015a3d0 con 0x7f36a0159890 2023-10-10T20:36:21.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.982+0000 7f36a6333700 1 --2- 172.21.15.89:0/4226032702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36a0159890 0x7f36a015abd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f369c0098e0 tx=0x7f369c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9502eb02911c9a9d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:21.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.982+0000 7f36a5331700 1 -- 172.21.15.89:0/4226032702 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c009e80 con 0x7f36a0159890 2023-10-10T20:36:21.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.982+0000 7f36a5331700 1 -- 172.21.15.89:0/4226032702 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369c00b820 con 0x7f36a0159890 2023-10-10T20:36:21.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.982+0000 7f36a5331700 1 -- 172.21.15.89:0/4226032702 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c00bba0 con 0x7f36a0159890 2023-10-10T20:36:21.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.982+0000 7f36a8597700 1 -- 172.21.15.89:0/4226032702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36a0159890 msgr2=0x7f36a015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:21.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.982+0000 7f36a8597700 1 --2- 172.21.15.89:0/4226032702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36a0159890 0x7f36a015abd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f369c0098e0 tx=0x7f369c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.982+0000 7f36a8597700 1 -- 172.21.15.89:0/4226032702 shutdown_connections 2023-10-10T20:36:21.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.982+0000 7f36a8597700 1 --2- 172.21.15.89:0/4226032702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36a0159890 0x7f36a015abd0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.982+0000 7f36a8597700 1 -- 172.21.15.89:0/4226032702 >> 172.21.15.89:0/4226032702 conn(0x7f36a0068e20 msgr2=0x7f36a00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:21.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.982+0000 7f36a8597700 1 -- 172.21.15.89:0/4226032702 shutdown_connections 2023-10-10T20:36:21.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.982+0000 7f36a8597700 1 -- 172.21.15.89:0/4226032702 wait complete. 2023-10-10T20:36:21.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.983+0000 7f36a8597700 1 Processor -- start 2023-10-10T20:36:21.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.983+0000 7f36a8597700 1 -- start start 2023-10-10T20:36:21.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.984+0000 7f36a8597700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36a0069560 0x7f36a015ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:21.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.984+0000 7f36a8597700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36a0159890 0x7f36a00ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:21.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.984+0000 7f36a8597700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f36a0159c60 0x7f36a016dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:21.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.984+0000 7f36a8597700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a00ca750 con 0x7f36a0159890 2023-10-10T20:36:21.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.984+0000 7f36a8597700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a016e2b0 con 0x7f36a0069560 2023-10-10T20:36:21.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.984+0000 7f36a8597700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a015c030 con 0x7f36a0159c60 2023-10-10T20:36:21.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.984+0000 7f36a5b32700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36a0159890 0x7f36a00ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:21.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.984+0000 7f36a5b32700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36a0159890 0x7f36a00ca210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46788/0 (socket says 172.21.15.89:46788) 2023-10-10T20:36:21.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.984+0000 7f36a5b32700 1 -- 172.21.15.89:0/2292174675 learned_addr learned my addr 172.21.15.89:0/2292174675 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:21.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.984+0000 7f36a6333700 1 --2- 172.21.15.89:0/2292174675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36a0069560 0x7f36a015ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:21.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.984+0000 7f36a6b34700 1 --2- 172.21.15.89:0/2292174675 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f36a0159c60 0x7f36a016dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:21.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.984+0000 7f36a5b32700 1 -- 172.21.15.89:0/2292174675 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f36a0159c60 msgr2=0x7f36a016dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:21.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.984+0000 7f36a5b32700 1 --2- 172.21.15.89:0/2292174675 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f36a0159c60 0x7f36a016dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.984+0000 7f36a5b32700 1 -- 172.21.15.89:0/2292174675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36a0069560 msgr2=0x7f36a015ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:21.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.984+0000 7f36a5b32700 1 --2- 172.21.15.89:0/2292174675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36a0069560 0x7f36a015ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:21.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.984+0000 7f36a5b32700 1 -- 172.21.15.89:0/2292174675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c009420 con 0x7f36a0159890 2023-10-10T20:36:21.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.984+0000 7f36a6333700 1 --2- 172.21.15.89:0/2292174675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36a0069560 0x7f36a015ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:21.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.985+0000 7f36a5b32700 1 --2- 172.21.15.89:0/2292174675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36a0159890 0x7f36a00ca210 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f369000c5d0 tx=0x7f3690007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:21.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.985+0000 7f36967fc700 1 -- 172.21.15.89:0/2292174675 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3690011590 con 0x7f36a0159890 2023-10-10T20:36:21.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.985+0000 7f36967fc700 1 -- 172.21.15.89:0/2292174675 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3690011710 con 0x7f36a0159890 2023-10-10T20:36:21.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.985+0000 7f36967fc700 1 -- 172.21.15.89:0/2292174675 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3690011ad0 con 0x7f36a0159890 2023-10-10T20:36:21.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.985+0000 7f36a8597700 1 -- 172.21.15.89:0/2292174675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a015c310 con 0x7f36a0159890 2023-10-10T20:36:21.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.985+0000 7f36a8597700 1 -- 172.21.15.89:0/2292174675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a015c8d0 con 0x7f36a0159890 2023-10-10T20:36:21.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.986+0000 7f36a8597700 1 -- 172.21.15.89:0/2292174675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a004c3c0 con 0x7f36a0159890 2023-10-10T20:36:21.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.986+0000 7f36967fc700 1 -- 172.21.15.89:0/2292174675 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f369001c460 con 0x7f36a0159890 2023-10-10T20:36:21.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.986+0000 7f36967fc700 1 --2- 172.21.15.89:0/2292174675 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3684066fc0 0x7f3684069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:21.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.987+0000 7f36967fc700 1 -- 172.21.15.89:0/2292174675 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3690084350 con 0x7f36a0159890 2023-10-10T20:36:21.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.987+0000 7f36a6333700 1 --2- 172.21.15.89:0/2292174675 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3684066fc0 0x7f3684069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:21.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.987+0000 7f36a6333700 1 --2- 172.21.15.89:0/2292174675 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3684066fc0 0x7f3684069470 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f369c0030c0 tx=0x7f369c016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:21.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:21.991+0000 7f36967fc700 1 -- 172.21.15.89:0/2292174675 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36900515d0 con 0x7f36a0159890 2023-10-10T20:36:22.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.174+0000 7f36a8597700 1 -- 172.21.15.89:0/2292174675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36a0069ad0 con 0x7f36a0159890 2023-10-10T20:36:22.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.175+0000 7f36967fc700 1 -- 172.21.15.89:0/2292174675 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+15366 (secure 0 0 0) 0x7f3690054bf0 con 0x7f36a0159890 2023-10-10T20:36:22.178 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:22.178 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":61,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:36:20.867024+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:36:19.221740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-10T20:36:19.834213+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:36:22.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.177+0000 7f36a8597700 1 -- 172.21.15.89:0/2292174675 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3684066fc0 msgr2=0x7f3684069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.177+0000 7f36a8597700 1 --2- 172.21.15.89:0/2292174675 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3684066fc0 0x7f3684069470 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f369c0030c0 tx=0x7f369c016000 comp rx=0 tx=0).stop 2023-10-10T20:36:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.177+0000 7f36a8597700 1 -- 172.21.15.89:0/2292174675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36a0159890 msgr2=0x7f36a00ca210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.177+0000 7f36a8597700 1 --2- 172.21.15.89:0/2292174675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36a0159890 0x7f36a00ca210 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f369000c5d0 tx=0x7f3690007f60 comp rx=0 tx=0).stop 2023-10-10T20:36:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.177+0000 7f36a8597700 1 -- 172.21.15.89:0/2292174675 shutdown_connections 2023-10-10T20:36:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.177+0000 7f36a8597700 1 --2- 172.21.15.89:0/2292174675 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f36a0159c60 0x7f36a016dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:22.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.177+0000 7f36a8597700 1 --2- 172.21.15.89:0/2292174675 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3684066fc0 0x7f3684069470 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:22.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.177+0000 7f36a8597700 1 --2- 172.21.15.89:0/2292174675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36a0069560 0x7f36a015ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:22.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.177+0000 7f36a8597700 1 --2- 172.21.15.89:0/2292174675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36a0159890 0x7f36a00ca210 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:22.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.177+0000 7f36a8597700 1 -- 172.21.15.89:0/2292174675 >> 172.21.15.89:0/2292174675 conn(0x7f36a0068e20 msgr2=0x7f36a00b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:22.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.177+0000 7f36a8597700 1 -- 172.21.15.89:0/2292174675 shutdown_connections 2023-10-10T20:36:22.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.177+0000 7f36a8597700 1 -- 172.21.15.89:0/2292174675 wait complete. 2023-10-10T20:36:22.191 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:36:22.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.306+0000 7ff8215a1700 1 Processor -- start 2023-10-10T20:36:22.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.306+0000 7ff8215a1700 1 -- start start 2023-10-10T20:36:22.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.307+0000 7ff8215a1700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff81c069560 0x7ff81c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:22.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.307+0000 7ff8215a1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c069ee0 con 0x7ff81c159c60 2023-10-10T20:36:22.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.307+0000 7ff8215a1700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c15a160 con 0x7ff81c159890 2023-10-10T20:36:22.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.307+0000 7ff8215a1700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c0d13a0 con 0x7ff81c069560 2023-10-10T20:36:22.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.307+0000 7ff81a7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff81c159c60 0x7ff81c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56616/0 (socket says 172.21.15.89:56616) 2023-10-10T20:36:22.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.307+0000 7ff81a7fc700 1 -- 172.21.15.89:0/953356778 learned_addr learned my addr 172.21.15.89:0/953356778 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:22.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.307+0000 7ff81b7fe700 1 --2- 172.21.15.89:0/953356778 >> v2:172.21.15.111:3300/0 conn(0x7ff81c069560 0x7ff81c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:22.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.307+0000 7ff819ffb700 1 -- 172.21.15.89:0/953356778 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487532340 0 0) 0x7ff81c069ee0 con 0x7ff81c159c60 2023-10-10T20:36:22.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.307+0000 7ff819ffb700 1 -- 172.21.15.89:0/953356778 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff808003570 con 0x7ff81c159c60 2023-10-10T20:36:22.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.307+0000 7ff819ffb700 1 -- 172.21.15.89:0/953356778 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2233089599 0 0) 0x7ff808003570 con 0x7ff81c159c60 2023-10-10T20:36:22.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.307+0000 7ff819ffb700 1 -- 172.21.15.89:0/953356778 >> v2:172.21.15.111:3300/0 conn(0x7ff81c069560 msgr2=0x7ff81c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:22.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.307+0000 7ff819ffb700 1 --2- 172.21.15.89:0/953356778 >> v2:172.21.15.111:3300/0 conn(0x7ff81c069560 0x7ff81c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:22.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.307+0000 7ff819ffb700 1 -- 172.21.15.89:0/953356778 >> v1:172.21.15.111:6789/0 conn(0x7ff81c159890 legacy=0x7ff81c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:22.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.307+0000 7ff819ffb700 1 -- 172.21.15.89:0/953356778 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c15e140 con 0x7ff81c159c60 2023-10-10T20:36:22.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.307+0000 7ff819ffb700 1 -- 172.21.15.89:0/953356778 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7ff804001470 con 0x7ff81c159c60 2023-10-10T20:36:22.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.307+0000 7ff819ffb700 1 -- 172.21.15.89:0/953356778 >> v1:172.21.15.89:6789/0 conn(0x7ff81c159c60 legacy=0x7ff81c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:22.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.307+0000 7ff819ffb700 1 --2- 172.21.15.89:0/953356778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff808003a60 0x7ff808003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:22.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.308+0000 7ff819ffb700 1 --2- 172.21.15.89:0/953356778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff808004810 0x7ff808004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:22.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.308+0000 7ff819ffb700 1 --2- 172.21.15.89:0/953356778 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff8080051c0 0x7ff80800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:22.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.308+0000 7ff819ffb700 1 -- 172.21.15.89:0/953356778 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0d13a0 con 0x7ff808003a60 2023-10-10T20:36:22.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.308+0000 7ff819ffb700 1 -- 172.21.15.89:0/953356778 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80800c0d0 con 0x7ff808004810 2023-10-10T20:36:22.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.308+0000 7ff819ffb700 1 -- 172.21.15.89:0/953356778 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80800c290 con 0x7ff8080051c0 2023-10-10T20:36:22.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.308+0000 7ff81affd700 1 --2- 172.21.15.89:0/953356778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff808003a60 0x7ff808003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:22.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.308+0000 7ff81a7fc700 1 --2- 172.21.15.89:0/953356778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff808004810 0x7ff808004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:22.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.308+0000 7ff81b7fe700 1 --2- 172.21.15.89:0/953356778 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff8080051c0 0x7ff80800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:22.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.308+0000 7ff81affd700 1 -- 172.21.15.89:0/953356778 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff8080051c0 msgr2=0x7ff80800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:22.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.308+0000 7ff81affd700 1 --2- 172.21.15.89:0/953356778 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff8080051c0 0x7ff80800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:22.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.308+0000 7ff81affd700 1 -- 172.21.15.89:0/953356778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff808004810 msgr2=0x7ff808004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:22.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.308+0000 7ff81affd700 1 --2- 172.21.15.89:0/953356778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff808004810 0x7ff808004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:22.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.308+0000 7ff81affd700 1 -- 172.21.15.89:0/953356778 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff808003570 con 0x7ff808003a60 2023-10-10T20:36:22.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.308+0000 7ff81b7fe700 1 --2- 172.21.15.89:0/953356778 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff8080051c0 0x7ff80800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:22.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.308+0000 7ff81b7fe700 1 -- 172.21.15.89:0/953356778 reap_dead start 2023-10-10T20:36:22.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.308+0000 7ff81affd700 1 --2- 172.21.15.89:0/953356778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff808003a60 0x7ff808003e70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff8140018b0 tx=0x7ff814007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f882c3503fc277d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:22.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.308+0000 7ff819ffb700 1 -- 172.21.15.89:0/953356778 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814012420 con 0x7ff808003a60 2023-10-10T20:36:22.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.308+0000 7ff819ffb700 1 -- 172.21.15.89:0/953356778 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8140125a0 con 0x7ff808003a60 2023-10-10T20:36:22.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.308+0000 7ff819ffb700 1 -- 172.21.15.89:0/953356778 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814012960 con 0x7ff808003a60 2023-10-10T20:36:22.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.309+0000 7ff8215a1700 1 -- 172.21.15.89:0/953356778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff808003a60 msgr2=0x7ff808003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:22.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.309+0000 7ff8215a1700 1 --2- 172.21.15.89:0/953356778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff808003a60 0x7ff808003e70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff8140018b0 tx=0x7ff814007d30 comp rx=0 tx=0).stop 2023-10-10T20:36:22.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.309+0000 7ff8215a1700 1 -- 172.21.15.89:0/953356778 shutdown_connections 2023-10-10T20:36:22.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.309+0000 7ff8215a1700 1 --2- 172.21.15.89:0/953356778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff808003a60 0x7ff808003e70 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:22.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.309+0000 7ff8215a1700 1 -- 172.21.15.89:0/953356778 >> 172.21.15.89:0/953356778 conn(0x7ff81c068e20 msgr2=0x7ff81c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:22.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.309+0000 7ff8215a1700 1 -- 172.21.15.89:0/953356778 shutdown_connections 2023-10-10T20:36:22.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.309+0000 7ff8215a1700 1 -- 172.21.15.89:0/953356778 wait complete. 2023-10-10T20:36:22.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.310+0000 7ff8215a1700 1 Processor -- start 2023-10-10T20:36:22.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.310+0000 7ff8215a1700 1 -- start start 2023-10-10T20:36:22.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff8215a1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff808003a60 0x7ff81c149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:22.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff8215a1700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff81c148160 0x7ff81c1537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:22.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff8215a1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff81c1485d0 0x7ff81c153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:22.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff8215a1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c14a6f0 con 0x7ff81c1485d0 2023-10-10T20:36:22.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff8215a1700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c149ae0 con 0x7ff81c148160 2023-10-10T20:36:22.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff8215a1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c149dc0 con 0x7ff808003a60 2023-10-10T20:36:22.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff81b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff81c1485d0 0x7ff81c153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:22.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff81b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff81c1485d0 0x7ff81c153d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46794/0 (socket says 172.21.15.89:46794) 2023-10-10T20:36:22.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff81b7fe700 1 -- 172.21.15.89:0/2055684362 learned_addr learned my addr 172.21.15.89:0/2055684362 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:22.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff81affd700 1 --2- 172.21.15.89:0/2055684362 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff808003a60 0x7ff81c149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:22.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff81a7fc700 1 --2- 172.21.15.89:0/2055684362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff81c148160 0x7ff81c1537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:22.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff81b7fe700 1 -- 172.21.15.89:0/2055684362 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff808003a60 msgr2=0x7ff81c149170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:22.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff81b7fe700 1 --2- 172.21.15.89:0/2055684362 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff808003a60 0x7ff81c149170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:22.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff81b7fe700 1 -- 172.21.15.89:0/2055684362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff81c148160 msgr2=0x7ff81c1537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:22.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff81b7fe700 1 --2- 172.21.15.89:0/2055684362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff81c148160 0x7ff81c1537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:22.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff81b7fe700 1 -- 172.21.15.89:0/2055684362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c14a0a0 con 0x7ff81c1485d0 2023-10-10T20:36:22.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff81b7fe700 1 --2- 172.21.15.89:0/2055684362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff81c1485d0 0x7ff81c153d00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0d03b0 tx=0x7ff80c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:22.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff812ffd700 1 -- 172.21.15.89:0/2055684362 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c01b070 con 0x7ff81c1485d0 2023-10-10T20:36:22.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff812ffd700 1 -- 172.21.15.89:0/2055684362 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80c00f040 con 0x7ff81c1485d0 2023-10-10T20:36:22.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff8215a1700 1 -- 172.21.15.89:0/2055684362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81c14a380 con 0x7ff81c1485d0 2023-10-10T20:36:22.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff812ffd700 1 -- 172.21.15.89:0/2055684362 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c00e050 con 0x7ff81c1485d0 2023-10-10T20:36:22.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.311+0000 7ff8215a1700 1 -- 172.21.15.89:0/2055684362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c149870 con 0x7ff81c1485d0 2023-10-10T20:36:22.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.312+0000 7ff8215a1700 1 -- 172.21.15.89:0/2055684362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81c14aa80 con 0x7ff81c1485d0 2023-10-10T20:36:22.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.313+0000 7ff812ffd700 1 -- 172.21.15.89:0/2055684362 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7ff80c003020 con 0x7ff81c1485d0 2023-10-10T20:36:22.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.313+0000 7ff812ffd700 1 --2- 172.21.15.89:0/2055684362 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff7fc066fc0 0x7ff7fc069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:22.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.313+0000 7ff812ffd700 1 -- 172.21.15.89:0/2055684362 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff80c084980 con 0x7ff81c1485d0 2023-10-10T20:36:22.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.313+0000 7ff81affd700 1 --2- 172.21.15.89:0/2055684362 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff7fc066fc0 0x7ff7fc069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:22.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.313+0000 7ff81affd700 1 --2- 172.21.15.89:0/2055684362 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff7fc066fc0 0x7ff7fc069470 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff814001010 tx=0x7ff814015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:22.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.317+0000 7ff812ffd700 1 -- 172.21.15.89:0/2055684362 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff81c14aa80 con 0x7ff81c1485d0 2023-10-10T20:36:22.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:22.543+0000 7ff8215a1700 1 -- 172.21.15.89:0/2055684362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff81c15bbe0 con 0x7ff81c1485d0 2023-10-10T20:36:23.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.518+0000 7ff812ffd700 1 -- 172.21.15.89:0/2055684362 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v36) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff80c051af0 con 0x7ff81c1485d0 2023-10-10T20:36:23.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.521+0000 7ff8215a1700 1 -- 172.21.15.89:0/2055684362 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff7fc066fc0 msgr2=0x7ff7fc069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.521+0000 7ff8215a1700 1 --2- 172.21.15.89:0/2055684362 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff7fc066fc0 0x7ff7fc069470 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff814001010 tx=0x7ff814015000 comp rx=0 tx=0).stop 2023-10-10T20:36:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.521+0000 7ff8215a1700 1 -- 172.21.15.89:0/2055684362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff81c1485d0 msgr2=0x7ff81c153d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.521+0000 7ff8215a1700 1 --2- 172.21.15.89:0/2055684362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff81c1485d0 0x7ff81c153d00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0d03b0 tx=0x7ff80c00b040 comp rx=0 tx=0).stop 2023-10-10T20:36:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.521+0000 7ff8215a1700 1 -- 172.21.15.89:0/2055684362 shutdown_connections 2023-10-10T20:36:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.521+0000 7ff8215a1700 1 --2- 172.21.15.89:0/2055684362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff81c1485d0 0x7ff81c153d00 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.521+0000 7ff8215a1700 1 --2- 172.21.15.89:0/2055684362 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff808003a60 0x7ff81c149170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.521+0000 7ff8215a1700 1 --2- 172.21.15.89:0/2055684362 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff7fc066fc0 0x7ff7fc069470 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.521+0000 7ff8215a1700 1 --2- 172.21.15.89:0/2055684362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff81c148160 0x7ff81c1537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.521+0000 7ff8215a1700 1 -- 172.21.15.89:0/2055684362 >> 172.21.15.89:0/2055684362 conn(0x7ff81c068e20 msgr2=0x7ff81c15b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.521+0000 7ff8215a1700 1 -- 172.21.15.89:0/2055684362 shutdown_connections 2023-10-10T20:36:23.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.521+0000 7ff8215a1700 1 -- 172.21.15.89:0/2055684362 wait complete. 2023-10-10T20:36:23.538 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:36:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.657+0000 7f14a34ed700 1 Processor -- start 2023-10-10T20:36:23.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.657+0000 7f14a34ed700 1 -- start start 2023-10-10T20:36:23.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.658+0000 7f14a34ed700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f149c159c60 0x7f149c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:23.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.658+0000 7f14a34ed700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f149c15db10 0x7f149c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:23.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.658+0000 7f14a34ed700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f149c15a160 con 0x7f149c159890 2023-10-10T20:36:23.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.658+0000 7f14a34ed700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f149c0d13a0 con 0x7f149c15db10 2023-10-10T20:36:23.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.658+0000 7f14a34ed700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149c15e140 con 0x7f149c159c60 2023-10-10T20:36:23.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.658+0000 7f14a1289700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f149c159890 0x7f149c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:53170/0 (socket says 172.21.15.89:53170) 2023-10-10T20:36:23.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.658+0000 7f14a1289700 1 -- 172.21.15.89:0/2625875287 learned_addr learned my addr 172.21.15.89:0/2625875287 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:23.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.658+0000 7f14a0a88700 1 --2- 172.21.15.89:0/2625875287 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f149c159c60 0x7f149c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:23.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.658+0000 7f14a1a8a700 1 --2- 172.21.15.89:0/2625875287 >> v2:172.21.15.111:3300/0 conn(0x7f149c15db10 0x7f149c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:23.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.659+0000 7f1493fff700 1 -- 172.21.15.89:0/2625875287 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680933260 0 0) 0x7f149c15a160 con 0x7f149c159890 2023-10-10T20:36:23.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.659+0000 7f14a0a88700 1 -- 172.21.15.89:0/2625875287 >> v2:172.21.15.111:3300/0 conn(0x7f149c15db10 msgr2=0x7f149c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:23.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.659+0000 7f14a0a88700 1 --2- 172.21.15.89:0/2625875287 >> v2:172.21.15.111:3300/0 conn(0x7f149c15db10 0x7f149c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:23.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.659+0000 7f14a0a88700 1 -- 172.21.15.89:0/2625875287 >> v1:172.21.15.111:6789/0 conn(0x7f149c159890 legacy=0x7f149c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:23.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.659+0000 7f14a0a88700 1 -- 172.21.15.89:0/2625875287 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c15a3d0 con 0x7f149c159c60 2023-10-10T20:36:23.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.659+0000 7f14a0a88700 1 --2- 172.21.15.89:0/2625875287 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f149c159c60 0x7f149c15d5d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f148c0098e0 tx=0x7f148c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba1c7e1c60c7d5d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:23.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.659+0000 7f1493fff700 1 -- 172.21.15.89:0/2625875287 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c00b640 con 0x7f149c159c60 2023-10-10T20:36:23.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.659+0000 7f1493fff700 1 -- 172.21.15.89:0/2625875287 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c00b7a0 con 0x7f149c159c60 2023-10-10T20:36:23.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.659+0000 7f1493fff700 1 -- 172.21.15.89:0/2625875287 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c00bb40 con 0x7f149c159c60 2023-10-10T20:36:23.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.660+0000 7f14a34ed700 1 -- 172.21.15.89:0/2625875287 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f149c159c60 msgr2=0x7f149c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:23.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.660+0000 7f14a34ed700 1 --2- 172.21.15.89:0/2625875287 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f149c159c60 0x7f149c15d5d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f148c0098e0 tx=0x7f148c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:23.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.660+0000 7f14a34ed700 1 -- 172.21.15.89:0/2625875287 shutdown_connections 2023-10-10T20:36:23.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.660+0000 7f14a34ed700 1 --2- 172.21.15.89:0/2625875287 >> v2:172.21.15.111:3300/0 conn(0x7f149c15db10 0x7f149c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:23.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.660+0000 7f14a34ed700 1 --2- 172.21.15.89:0/2625875287 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f149c159c60 0x7f149c15d5d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:23.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.660+0000 7f14a34ed700 1 -- 172.21.15.89:0/2625875287 >> 172.21.15.89:0/2625875287 conn(0x7f149c068e20 msgr2=0x7f149c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:23.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.660+0000 7f14a34ed700 1 -- 172.21.15.89:0/2625875287 shutdown_connections 2023-10-10T20:36:23.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.660+0000 7f14a34ed700 1 -- 172.21.15.89:0/2625875287 wait complete. 2023-10-10T20:36:23.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.661+0000 7f14a34ed700 1 Processor -- start 2023-10-10T20:36:23.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.661+0000 7f14a34ed700 1 -- start start 2023-10-10T20:36:23.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.662+0000 7f14a34ed700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f149c159890 0x7f149c1523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:23.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.662+0000 7f14a34ed700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f149c159c60 0x7f149c15c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:23.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.662+0000 7f14a34ed700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f149c15db10 0x7f149c15c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:23.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.662+0000 7f14a34ed700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c0cdd70 con 0x7f149c159890 2023-10-10T20:36:23.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.662+0000 7f14a34ed700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c152900 con 0x7f149c159c60 2023-10-10T20:36:23.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.662+0000 7f14a34ed700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149c0cd160 con 0x7f149c15db10 2023-10-10T20:36:23.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.662+0000 7f14a1289700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f149c159890 0x7f149c1523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:23.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.662+0000 7f14a1289700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f149c159890 0x7f149c1523c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46808/0 (socket says 172.21.15.89:46808) 2023-10-10T20:36:23.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.662+0000 7f14a1289700 1 -- 172.21.15.89:0/4269998703 learned_addr learned my addr 172.21.15.89:0/4269998703 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:23.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.662+0000 7f14a1a8a700 1 --2- 172.21.15.89:0/4269998703 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f149c15db10 0x7f149c15c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:23.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.662+0000 7f14a0a88700 1 --2- 172.21.15.89:0/4269998703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f149c159c60 0x7f149c15c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:23.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.662+0000 7f14a1289700 1 -- 172.21.15.89:0/4269998703 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f149c15db10 msgr2=0x7f149c15c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:23.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.662+0000 7f14a1289700 1 --2- 172.21.15.89:0/4269998703 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f149c15db10 0x7f149c15c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:23.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.662+0000 7f14a1289700 1 -- 172.21.15.89:0/4269998703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f149c159c60 msgr2=0x7f149c15c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:23.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.662+0000 7f14a1289700 1 --2- 172.21.15.89:0/4269998703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f149c159c60 0x7f149c15c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:23.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.662+0000 7f14a1289700 1 -- 172.21.15.89:0/4269998703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c009420 con 0x7f149c159890 2023-10-10T20:36:23.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.662+0000 7f14a1289700 1 --2- 172.21.15.89:0/4269998703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f149c159890 0x7f149c1523c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f149800cd80 tx=0x7f1498001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:23.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.663+0000 7f14917fa700 1 -- 172.21.15.89:0/4269998703 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149800ab10 con 0x7f149c159890 2023-10-10T20:36:23.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.663+0000 7f14917fa700 1 -- 172.21.15.89:0/4269998703 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149800ac70 con 0x7f149c159890 2023-10-10T20:36:23.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.663+0000 7f14917fa700 1 -- 172.21.15.89:0/4269998703 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14980070f0 con 0x7f149c159890 2023-10-10T20:36:23.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.663+0000 7f14a34ed700 1 -- 172.21.15.89:0/4269998703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149c0cd440 con 0x7f149c159890 2023-10-10T20:36:23.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.663+0000 7f14a34ed700 1 -- 172.21.15.89:0/4269998703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149c0cda00 con 0x7f149c159890 2023-10-10T20:36:23.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.663+0000 7f14a34ed700 1 -- 172.21.15.89:0/4269998703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f149c04c3c0 con 0x7f149c159890 2023-10-10T20:36:23.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.664+0000 7f14917fa700 1 -- 172.21.15.89:0/4269998703 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f14980074e0 con 0x7f149c159890 2023-10-10T20:36:23.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.664+0000 7f14917fa700 1 --2- 172.21.15.89:0/4269998703 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1480066fc0 0x7f1480069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:23.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.665+0000 7f14a0a88700 1 --2- 172.21.15.89:0/4269998703 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1480066fc0 0x7f1480069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:23.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.665+0000 7f14917fa700 1 -- 172.21.15.89:0/4269998703 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1498084ae0 con 0x7f149c159890 2023-10-10T20:36:23.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.665+0000 7f14a0a88700 1 --2- 172.21.15.89:0/4269998703 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1480066fc0 0x7f1480069470 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f149c15a090 tx=0x7f148c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:23.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.669+0000 7f14917fa700 1 -- 172.21.15.89:0/4269998703 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1498051de0 con 0x7f149c159890 2023-10-10T20:36:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:23.891+0000 7f14a34ed700 1 -- 172.21.15.89:0/4269998703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f149c06a5b0 con 0x7f149c159890 2023-10-10T20:36:24.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.520+0000 7f14917fa700 1 -- 172.21.15.89:0/4269998703 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v37) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1498055400 con 0x7f149c159890 2023-10-10T20:36:24.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.523+0000 7f14a34ed700 1 -- 172.21.15.89:0/4269998703 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1480066fc0 msgr2=0x7f1480069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:24.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.523+0000 7f14a34ed700 1 --2- 172.21.15.89:0/4269998703 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1480066fc0 0x7f1480069470 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f149c15a090 tx=0x7f148c017000 comp rx=0 tx=0).stop 2023-10-10T20:36:24.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.523+0000 7f14a34ed700 1 -- 172.21.15.89:0/4269998703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f149c159890 msgr2=0x7f149c1523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:24.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.523+0000 7f14a34ed700 1 --2- 172.21.15.89:0/4269998703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f149c159890 0x7f149c1523c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f149800cd80 tx=0x7f1498001ae0 comp rx=0 tx=0).stop 2023-10-10T20:36:24.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.523+0000 7f14a34ed700 1 -- 172.21.15.89:0/4269998703 shutdown_connections 2023-10-10T20:36:24.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.523+0000 7f14a34ed700 1 --2- 172.21.15.89:0/4269998703 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f149c15db10 0x7f149c15c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:24.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.523+0000 7f14a34ed700 1 --2- 172.21.15.89:0/4269998703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f149c159890 0x7f149c1523c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:24.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.523+0000 7f14a34ed700 1 --2- 172.21.15.89:0/4269998703 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1480066fc0 0x7f1480069470 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:24.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.523+0000 7f14a34ed700 1 --2- 172.21.15.89:0/4269998703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f149c159c60 0x7f149c15c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:24.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.523+0000 7f14a34ed700 1 -- 172.21.15.89:0/4269998703 >> 172.21.15.89:0/4269998703 conn(0x7f149c068e20 msgr2=0x7f149c153170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:24.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.523+0000 7f14a34ed700 1 -- 172.21.15.89:0/4269998703 shutdown_connections 2023-10-10T20:36:24.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.523+0000 7f14a34ed700 1 -- 172.21.15.89:0/4269998703 wait complete. 2023-10-10T20:36:24.538 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:36:24.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.651+0000 7f46fb968700 1 Processor -- start 2023-10-10T20:36:24.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.651+0000 7f46fb968700 1 -- start start 2023-10-10T20:36:24.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.652+0000 7f46fb968700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f46f4068920 0x7f46f406f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:24.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.652+0000 7f46fb968700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f46f406f640 0x7f46f4159510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:24.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.652+0000 7f46fb968700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f406ec70 con 0x7f46f4068550 2023-10-10T20:36:24.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.652+0000 7f46fb968700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f4068d90 con 0x7f46f4068920 2023-10-10T20:36:24.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.652+0000 7f46fb968700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f40b3430 con 0x7f46f406f640 2023-10-10T20:36:24.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.652+0000 7f46f9704700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f46f4068550 0x7f46f415e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56620/0 (socket says 172.21.15.89:56620) 2023-10-10T20:36:24.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.652+0000 7f46f8f03700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f46f4068920 0x7f46f406f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:24.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.652+0000 7f46f9704700 1 -- 172.21.15.89:0/1609495059 learned_addr learned my addr 172.21.15.89:0/1609495059 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:24.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.653+0000 7f46f9f05700 1 --2- 172.21.15.89:0/1609495059 >> v2:172.21.15.111:3300/0 conn(0x7f46f406f640 0x7f46f4159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:24.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.653+0000 7f46f8f03700 1 -- 172.21.15.89:0/1609495059 >> v2:172.21.15.111:3300/0 conn(0x7f46f406f640 msgr2=0x7f46f4159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:24.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.653+0000 7f46f8f03700 1 --2- 172.21.15.89:0/1609495059 >> v2:172.21.15.111:3300/0 conn(0x7f46f406f640 0x7f46f4159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:24.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.653+0000 7f46f8f03700 1 -- 172.21.15.89:0/1609495059 >> v1:172.21.15.89:6789/0 conn(0x7f46f4068550 legacy=0x7f46f415e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:24.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.653+0000 7f46f8f03700 1 -- 172.21.15.89:0/1609495059 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f40b2e70 con 0x7f46f4068920 2023-10-10T20:36:24.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.653+0000 7f46f8f03700 1 --2- 172.21.15.89:0/1609495059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f4068920 0x7f46f406f100 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f46f40aceb0 tx=0x7f46e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610c3365cbd577a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:24.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.653+0000 7f46ebfff700 1 -- 172.21.15.89:0/1609495059 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e4009e80 con 0x7f46f4068920 2023-10-10T20:36:24.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.653+0000 7f46ebfff700 1 -- 172.21.15.89:0/1609495059 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e400b820 con 0x7f46f4068920 2023-10-10T20:36:24.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.653+0000 7f46ebfff700 1 -- 172.21.15.89:0/1609495059 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e400bba0 con 0x7f46f4068920 2023-10-10T20:36:24.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.653+0000 7f46fb968700 1 -- 172.21.15.89:0/1609495059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f4068920 msgr2=0x7f46f406f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:24.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.653+0000 7f46fb968700 1 --2- 172.21.15.89:0/1609495059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f4068920 0x7f46f406f100 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f46f40aceb0 tx=0x7f46e40091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:24.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.654+0000 7f46fb968700 1 -- 172.21.15.89:0/1609495059 shutdown_connections 2023-10-10T20:36:24.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.654+0000 7f46fb968700 1 --2- 172.21.15.89:0/1609495059 >> v2:172.21.15.111:3300/0 conn(0x7f46f406f640 0x7f46f4159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:24.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.654+0000 7f46fb968700 1 --2- 172.21.15.89:0/1609495059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f4068920 0x7f46f406f100 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:24.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.654+0000 7f46fb968700 1 -- 172.21.15.89:0/1609495059 >> 172.21.15.89:0/1609495059 conn(0x7f46f40cd480 msgr2=0x7f46f40cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:24.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.654+0000 7f46fb968700 1 -- 172.21.15.89:0/1609495059 shutdown_connections 2023-10-10T20:36:24.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.654+0000 7f46fb968700 1 -- 172.21.15.89:0/1609495059 wait complete. 2023-10-10T20:36:24.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.655+0000 7f46fb968700 1 Processor -- start 2023-10-10T20:36:24.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.655+0000 7f46fb968700 1 -- start start 2023-10-10T20:36:24.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46fb968700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46f4068550 0x7f46f414a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:24.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46fb968700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f4068920 0x7f46f414ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:24.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46fb968700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f406f640 0x7f46f4169f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:24.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46fb968700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f414db30 con 0x7f46f406f640 2023-10-10T20:36:24.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46fb968700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f414d6a0 con 0x7f46f4068920 2023-10-10T20:36:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46fb968700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f414ca90 con 0x7f46f4068550 2023-10-10T20:36:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46f9f05700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f406f640 0x7f46f4169f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46f9f05700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f406f640 0x7f46f4169f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46814/0 (socket says 172.21.15.89:46814) 2023-10-10T20:36:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46f9f05700 1 -- 172.21.15.89:0/104227052 learned_addr learned my addr 172.21.15.89:0/104227052 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46f8f03700 1 --2- 172.21.15.89:0/104227052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f4068920 0x7f46f414ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:24.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46f9704700 1 --2- 172.21.15.89:0/104227052 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46f4068550 0x7f46f414a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:24.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46f9f05700 1 -- 172.21.15.89:0/104227052 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46f4068550 msgr2=0x7f46f414a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:24.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46f9f05700 1 --2- 172.21.15.89:0/104227052 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46f4068550 0x7f46f414a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:24.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46f9f05700 1 -- 172.21.15.89:0/104227052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f4068920 msgr2=0x7f46f414ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:24.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46f9f05700 1 --2- 172.21.15.89:0/104227052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f4068920 0x7f46f414ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:24.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46f9f05700 1 -- 172.21.15.89:0/104227052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e4009420 con 0x7f46f406f640 2023-10-10T20:36:24.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46f8f03700 1 --2- 172.21.15.89:0/104227052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f4068920 0x7f46f414ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:24.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46f9f05700 1 --2- 172.21.15.89:0/104227052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f406f640 0x7f46f4169f30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009fa0 tx=0x7f46ec009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:24.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46e97fa700 1 -- 172.21.15.89:0/104227052 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec00ed20 con 0x7f46f406f640 2023-10-10T20:36:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46e97fa700 1 -- 172.21.15.89:0/104227052 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec00ee80 con 0x7f46f406f640 2023-10-10T20:36:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46fb968700 1 -- 172.21.15.89:0/104227052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f414cd70 con 0x7f46f406f640 2023-10-10T20:36:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46e97fa700 1 -- 172.21.15.89:0/104227052 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec014670 con 0x7f46f406f640 2023-10-10T20:36:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.656+0000 7f46fb968700 1 -- 172.21.15.89:0/104227052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f414d330 con 0x7f46f406f640 2023-10-10T20:36:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.657+0000 7f46fb968700 1 -- 172.21.15.89:0/104227052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f404c3c0 con 0x7f46f406f640 2023-10-10T20:36:24.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.658+0000 7f46e97fa700 1 -- 172.21.15.89:0/104227052 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f46ec0149e0 con 0x7f46f406f640 2023-10-10T20:36:24.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.658+0000 7f46e97fa700 1 --2- 172.21.15.89:0/104227052 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f46d8066fc0 0x7f46d8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:24.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.658+0000 7f46e97fa700 1 -- 172.21.15.89:0/104227052 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f46ec084dd0 con 0x7f46f406f640 2023-10-10T20:36:24.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.658+0000 7f46f9704700 1 --2- 172.21.15.89:0/104227052 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f46d8066fc0 0x7f46d8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:24.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.658+0000 7f46f9704700 1 --2- 172.21.15.89:0/104227052 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f46d8066fc0 0x7f46d8069470 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f46f0001780 tx=0x7f46f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:24.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.662+0000 7f46e97fa700 1 -- 172.21.15.89:0/104227052 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46ec052050 con 0x7f46f406f640 2023-10-10T20:36:24.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.887+0000 7f46fb968700 1 -- 172.21.15.89:0/104227052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46f40b3150 con 0x7f46f406f640 2023-10-10T20:36:24.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.887+0000 7f46e97fa700 1 -- 172.21.15.89:0/104227052 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f46ec055670 con 0x7f46f406f640 2023-10-10T20:36:24.890 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:24.890 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/3362909084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":3362909084},{"type":"v1","addr":"172.21.15.111:6835","nonce":3362909084}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5102,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1472423656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1472423656},{"type":"v1","addr":"172.21.15.111:6834","nonce":1472423656}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/3183325104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3183325104},{"type":"v1","addr":"172.21.15.89:6836","nonce":3183325104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:36:20.867329+0000","modified":"2023-10-10T20:36:24.518115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/862652054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":862652054},{"type":"v1","addr":"172.21.15.89:6837","nonce":862652054}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-10T20:36:24.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.889+0000 7f46fb968700 1 -- 172.21.15.89:0/104227052 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f46d8066fc0 msgr2=0x7f46d8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:24.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.889+0000 7f46fb968700 1 --2- 172.21.15.89:0/104227052 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f46d8066fc0 0x7f46d8069470 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f46f0001780 tx=0x7f46f0009040 comp rx=0 tx=0).stop 2023-10-10T20:36:24.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.889+0000 7f46fb968700 1 -- 172.21.15.89:0/104227052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f406f640 msgr2=0x7f46f4169f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:24.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.889+0000 7f46fb968700 1 --2- 172.21.15.89:0/104227052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f406f640 0x7f46f4169f30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009fa0 tx=0x7f46ec009900 comp rx=0 tx=0).stop 2023-10-10T20:36:24.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.889+0000 7f46fb968700 1 -- 172.21.15.89:0/104227052 shutdown_connections 2023-10-10T20:36:24.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.889+0000 7f46fb968700 1 --2- 172.21.15.89:0/104227052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f406f640 0x7f46f4169f30 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:24.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.890+0000 7f46fb968700 1 --2- 172.21.15.89:0/104227052 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46f4068550 0x7f46f414a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:24.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.890+0000 7f46fb968700 1 --2- 172.21.15.89:0/104227052 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f46d8066fc0 0x7f46d8069470 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:24.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.890+0000 7f46fb968700 1 --2- 172.21.15.89:0/104227052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f4068920 0x7f46f414ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:24.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.890+0000 7f46fb968700 1 -- 172.21.15.89:0/104227052 >> 172.21.15.89:0/104227052 conn(0x7f46f40cd480 msgr2=0x7f46f40b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:24.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.890+0000 7f46fb968700 1 -- 172.21.15.89:0/104227052 shutdown_connections 2023-10-10T20:36:24.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:24.890+0000 7f46fb968700 1 -- 172.21.15.89:0/104227052 wait complete. 2023-10-10T20:36:24.895 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 37 2023-10-10T20:36:24.904 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:36:25.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.015+0000 7f2d0759e700 1 Processor -- start 2023-10-10T20:36:25.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.015+0000 7f2d0759e700 1 -- start start 2023-10-10T20:36:25.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.016+0000 7f2d0759e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d08159420 0x7f2d080b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:25.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.016+0000 7f2d0759e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2d0815d230 0x7f2d0815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:25.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.016+0000 7f2d0759e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0815dbd0 con 0x7f2d081597f0 2023-10-10T20:36:25.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.016+0000 7f2d0759e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d080b2980 con 0x7f2d0815d230 2023-10-10T20:36:25.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.016+0000 7f2d0759e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080b11d0 con 0x7f2d08159420 2023-10-10T20:36:25.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.016+0000 7f2d06d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2d0815d230 0x7f2d0815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:25.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.016+0000 7f2d06d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2d0815d230 0x7f2d0815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46822/0 (socket says 172.21.15.89:46822) 2023-10-10T20:36:25.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.016+0000 7f2d06d9d700 1 -- 172.21.15.89:0/2613228705 learned_addr learned my addr 172.21.15.89:0/2613228705 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:25.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.016+0000 7f2d0659c700 1 --2- 172.21.15.89:0/2613228705 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d08159420 0x7f2d080b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:25.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.016+0000 7f2d06d9d700 1 -- 172.21.15.89:0/2613228705 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d08159420 msgr2=0x7f2d080b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:25.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.016+0000 7f2d06d9d700 1 --2- 172.21.15.89:0/2613228705 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d08159420 0x7f2d080b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.016+0000 7f2d06d9d700 1 -- 172.21.15.89:0/2613228705 >> v1:172.21.15.111:6789/0 conn(0x7f2d081597f0 legacy=0x7f2d080b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:25.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.016+0000 7f2d06d9d700 1 -- 172.21.15.89:0/2613228705 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d080b14b0 con 0x7f2d0815d230 2023-10-10T20:36:25.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.016+0000 7f2d06d9d700 1 --2- 172.21.15.89:0/2613228705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d0815d230 0x7f2d0815d690 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009f20 tx=0x7f2d00009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4419305c930961b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.016+0000 7f2d05d9b700 1 -- 172.21.15.89:0/2613228705 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0000e070 con 0x7f2d0815d230 2023-10-10T20:36:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.016+0000 7f2d05d9b700 1 -- 172.21.15.89:0/2613228705 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0000ba40 con 0x7f2d0815d230 2023-10-10T20:36:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.016+0000 7f2d05d9b700 1 -- 172.21.15.89:0/2613228705 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d000092e0 con 0x7f2d0815d230 2023-10-10T20:36:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.017+0000 7f2d0759e700 1 -- 172.21.15.89:0/2613228705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d0815d230 msgr2=0x7f2d0815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.017+0000 7f2d0759e700 1 --2- 172.21.15.89:0/2613228705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d0815d230 0x7f2d0815d690 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009f20 tx=0x7f2d00009840 comp rx=0 tx=0).stop 2023-10-10T20:36:25.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.017+0000 7f2d0759e700 1 -- 172.21.15.89:0/2613228705 shutdown_connections 2023-10-10T20:36:25.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.017+0000 7f2d0759e700 1 --2- 172.21.15.89:0/2613228705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d0815d230 0x7f2d0815d690 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.017+0000 7f2d0759e700 1 --2- 172.21.15.89:0/2613228705 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d08159420 0x7f2d080b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.017+0000 7f2d0759e700 1 -- 172.21.15.89:0/2613228705 >> 172.21.15.89:0/2613228705 conn(0x7f2d080d0660 msgr2=0x7f2d080d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:25.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.017+0000 7f2d0759e700 1 -- 172.21.15.89:0/2613228705 shutdown_connections 2023-10-10T20:36:25.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.017+0000 7f2d0759e700 1 -- 172.21.15.89:0/2613228705 wait complete. 2023-10-10T20:36:25.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.018+0000 7f2d0759e700 1 Processor -- start 2023-10-10T20:36:25.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.018+0000 7f2d0759e700 1 -- start start 2023-10-10T20:36:25.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.019+0000 7f2d0759e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d08159420 0x7f2d0814af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:25.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.019+0000 7f2d0759e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d081597f0 0x7f2d080b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:25.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.019+0000 7f2d0759e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d0815d230 0x7f2d0815c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:25.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.019+0000 7f2d0759e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0815a420 con 0x7f2d08159420 2023-10-10T20:36:25.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.019+0000 7f2d0759e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0815a6c0 con 0x7f2d081597f0 2023-10-10T20:36:25.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.019+0000 7f2d0759e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0815a9a0 con 0x7f2d0815d230 2023-10-10T20:36:25.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.019+0000 7f2d0659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d08159420 0x7f2d0814af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:25.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.019+0000 7f2d0659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d08159420 0x7f2d0814af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46824/0 (socket says 172.21.15.89:46824) 2023-10-10T20:36:25.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.019+0000 7f2d0659c700 1 -- 172.21.15.89:0/2216976075 learned_addr learned my addr 172.21.15.89:0/2216976075 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:25.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.019+0000 7f2cfdd9b700 1 --2- 172.21.15.89:0/2216976075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d081597f0 0x7f2d080b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:25.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.019+0000 7f2d06d9d700 1 --2- 172.21.15.89:0/2216976075 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d0815d230 0x7f2d0815c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:25.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.019+0000 7f2d0659c700 1 -- 172.21.15.89:0/2216976075 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d0815d230 msgr2=0x7f2d0815c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:25.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.019+0000 7f2d0659c700 1 --2- 172.21.15.89:0/2216976075 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d0815d230 0x7f2d0815c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.019+0000 7f2d0659c700 1 -- 172.21.15.89:0/2216976075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d081597f0 msgr2=0x7f2d080b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:25.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.019+0000 7f2d0659c700 1 --2- 172.21.15.89:0/2216976075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d081597f0 0x7f2d080b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.019+0000 7f2d0659c700 1 -- 172.21.15.89:0/2216976075 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d00009ae0 con 0x7f2d08159420 2023-10-10T20:36:25.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.019+0000 7f2d06d9d700 1 --2- 172.21.15.89:0/2216976075 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d0815d230 0x7f2d0815c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:25.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.020+0000 7f2d0659c700 1 --2- 172.21.15.89:0/2216976075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d08159420 0x7f2d0814af90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800c7d0 tx=0x7f2cf8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:25.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.020+0000 7f2cfeffd700 1 -- 172.21.15.89:0/2216976075 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8011600 con 0x7f2d08159420 2023-10-10T20:36:25.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.020+0000 7f2cfeffd700 1 -- 172.21.15.89:0/2216976075 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf8011760 con 0x7f2d08159420 2023-10-10T20:36:25.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.020+0000 7f2d0759e700 1 -- 172.21.15.89:0/2216976075 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0815ac80 con 0x7f2d08159420 2023-10-10T20:36:25.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.020+0000 7f2cfeffd700 1 -- 172.21.15.89:0/2216976075 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8011b20 con 0x7f2d08159420 2023-10-10T20:36:25.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.020+0000 7f2d0759e700 1 -- 172.21.15.89:0/2216976075 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0814b4d0 con 0x7f2d08159420 2023-10-10T20:36:25.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.021+0000 7f2d0759e700 1 -- 172.21.15.89:0/2216976075 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0804c3c0 con 0x7f2d08159420 2023-10-10T20:36:25.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.021+0000 7f2cfeffd700 1 -- 172.21.15.89:0/2216976075 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f2cf8018350 con 0x7f2d08159420 2023-10-10T20:36:25.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.021+0000 7f2cfeffd700 1 --2- 172.21.15.89:0/2216976075 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2ce8066fc0 0x7f2ce8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:25.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.022+0000 7f2cfeffd700 1 -- 172.21.15.89:0/2216976075 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2cf8084570 con 0x7f2d08159420 2023-10-10T20:36:25.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.022+0000 7f2cfdd9b700 1 --2- 172.21.15.89:0/2216976075 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2ce8066fc0 0x7f2ce8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:25.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.022+0000 7f2cfdd9b700 1 --2- 172.21.15.89:0/2216976075 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2ce8066fc0 0x7f2ce8069470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2d0816ce50 tx=0x7f2cf0007ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:25.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.026+0000 7f2cfeffd700 1 -- 172.21.15.89:0/2216976075 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2cf80516e0 con 0x7f2d08159420 2023-10-10T20:36:25.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.207+0000 7f2d0759e700 1 -- 172.21.15.89:0/2216976075 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d080af710 con 0x7f2d08159420 2023-10-10T20:36:25.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.208+0000 7f2cfeffd700 1 -- 172.21.15.89:0/2216976075 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+15353 (secure 0 0 0) 0x7f2cf8054d00 con 0x7f2d08159420 2023-10-10T20:36:25.210 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:25.211 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":62,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:36:22.497950+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:36:19.221740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-10T20:36:19.834213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:36:25.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.210+0000 7f2d0759e700 1 -- 172.21.15.89:0/2216976075 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2ce8066fc0 msgr2=0x7f2ce8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:25.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.210+0000 7f2d0759e700 1 --2- 172.21.15.89:0/2216976075 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2ce8066fc0 0x7f2ce8069470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2d0816ce50 tx=0x7f2cf0007ea0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.210+0000 7f2d0759e700 1 -- 172.21.15.89:0/2216976075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d08159420 msgr2=0x7f2d0814af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:25.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.210+0000 7f2d0759e700 1 --2- 172.21.15.89:0/2216976075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d08159420 0x7f2d0814af90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800c7d0 tx=0x7f2cf8008040 comp rx=0 tx=0).stop 2023-10-10T20:36:25.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.210+0000 7f2d0759e700 1 -- 172.21.15.89:0/2216976075 shutdown_connections 2023-10-10T20:36:25.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.210+0000 7f2d0759e700 1 --2- 172.21.15.89:0/2216976075 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d0815d230 0x7f2d0815c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.210+0000 7f2d0759e700 1 --2- 172.21.15.89:0/2216976075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d08159420 0x7f2d0814af90 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.210+0000 7f2d0759e700 1 --2- 172.21.15.89:0/2216976075 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2ce8066fc0 0x7f2ce8069470 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.210+0000 7f2d0759e700 1 --2- 172.21.15.89:0/2216976075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d081597f0 0x7f2d080b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.210+0000 7f2d0759e700 1 -- 172.21.15.89:0/2216976075 >> 172.21.15.89:0/2216976075 conn(0x7f2d080d0660 msgr2=0x7f2d08069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:25.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.210+0000 7f2d0759e700 1 -- 172.21.15.89:0/2216976075 shutdown_connections 2023-10-10T20:36:25.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.210+0000 7f2d0759e700 1 -- 172.21.15.89:0/2216976075 wait complete. 2023-10-10T20:36:25.225 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:36:25.225 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:36:25.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.337+0000 7f3626ce7700 1 Processor -- start 2023-10-10T20:36:25.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.337+0000 7f3626ce7700 1 -- start start 2023-10-10T20:36:25.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.338+0000 7f3626ce7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3620069560 0x7f36200699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:25.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.338+0000 7f3626ce7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3620069ee0 con 0x7f3620159c60 2023-10-10T20:36:25.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.338+0000 7f3626ce7700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362015a160 con 0x7f3620159890 2023-10-10T20:36:25.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.338+0000 7f3626ce7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36200d13a0 con 0x7f3620069560 2023-10-10T20:36:25.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.338+0000 7f3625284700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3620069560 0x7f36200699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:25.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.338+0000 7f3617fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3620159c60 0x7f362015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56634/0 (socket says 172.21.15.89:56634) 2023-10-10T20:36:25.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.338+0000 7f3617fff700 1 -- 172.21.15.89:0/4054538937 learned_addr learned my addr 172.21.15.89:0/4054538937 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:25.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.339+0000 7f3625284700 1 -- 172.21.15.89:0/4054538937 >> v1:172.21.15.111:6789/0 conn(0x7f3620159890 legacy=0x7f362015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:25.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.339+0000 7f3625284700 1 -- 172.21.15.89:0/4054538937 >> v1:172.21.15.89:6789/0 conn(0x7f3620159c60 legacy=0x7f362015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:25.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.339+0000 7f36177fe700 1 -- 172.21.15.89:0/4054538937 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059236930 0 0) 0x7f3620069ee0 con 0x7f3620159c60 2023-10-10T20:36:25.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.339+0000 7f3625284700 1 -- 172.21.15.89:0/4054538937 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362015e140 con 0x7f3620069560 2023-10-10T20:36:25.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.339+0000 7f3625284700 1 --2- 172.21.15.89:0/4054538937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3620069560 0x7f36200699a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f36180098e0 tx=0x7f36180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86996f7c6c694519 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:25.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.339+0000 7f36177fe700 1 -- 172.21.15.89:0/4054538937 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618009e80 con 0x7f3620069560 2023-10-10T20:36:25.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.339+0000 7f36177fe700 1 -- 172.21.15.89:0/4054538937 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361800b820 con 0x7f3620069560 2023-10-10T20:36:25.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.339+0000 7f36177fe700 1 -- 172.21.15.89:0/4054538937 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361800bba0 con 0x7f3620069560 2023-10-10T20:36:25.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.339+0000 7f3626ce7700 1 -- 172.21.15.89:0/4054538937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3620069560 msgr2=0x7f36200699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:25.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.339+0000 7f3626ce7700 1 --2- 172.21.15.89:0/4054538937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3620069560 0x7f36200699a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f36180098e0 tx=0x7f36180091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.339+0000 7f3626ce7700 1 -- 172.21.15.89:0/4054538937 shutdown_connections 2023-10-10T20:36:25.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.339+0000 7f3626ce7700 1 --2- 172.21.15.89:0/4054538937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3620069560 0x7f36200699a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.339+0000 7f3626ce7700 1 -- 172.21.15.89:0/4054538937 >> 172.21.15.89:0/4054538937 conn(0x7f3620068e20 msgr2=0x7f36200d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:25.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.339+0000 7f3626ce7700 1 -- 172.21.15.89:0/4054538937 shutdown_connections 2023-10-10T20:36:25.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.339+0000 7f3626ce7700 1 -- 172.21.15.89:0/4054538937 wait complete. 2023-10-10T20:36:25.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.340+0000 7f3626ce7700 1 Processor -- start 2023-10-10T20:36:25.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.341+0000 7f3626ce7700 1 -- start start 2023-10-10T20:36:25.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.341+0000 7f3626ce7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3620069560 0x7f362014d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:25.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.341+0000 7f3626ce7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3620159890 0x7f3620153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:25.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.341+0000 7f3626ce7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3620159c60 0x7f3620153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:25.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.341+0000 7f3626ce7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362015c4e0 con 0x7f3620159c60 2023-10-10T20:36:25.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.341+0000 7f3626ce7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36200ce170 con 0x7f3620159890 2023-10-10T20:36:25.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.341+0000 7f3626ce7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36200ce450 con 0x7f3620069560 2023-10-10T20:36:25.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.341+0000 7f3625284700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3620159c60 0x7f3620153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:25.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.341+0000 7f3625284700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3620159c60 0x7f3620153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46838/0 (socket says 172.21.15.89:46838) 2023-10-10T20:36:25.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.341+0000 7f3625284700 1 -- 172.21.15.89:0/1191757373 learned_addr learned my addr 172.21.15.89:0/1191757373 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:25.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.342+0000 7f3617fff700 1 --2- 172.21.15.89:0/1191757373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3620159890 0x7f3620153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:25.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.342+0000 7f3624a83700 1 --2- 172.21.15.89:0/1191757373 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3620069560 0x7f362014d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:25.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.342+0000 7f3625284700 1 -- 172.21.15.89:0/1191757373 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3620069560 msgr2=0x7f362014d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:25.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.342+0000 7f3625284700 1 --2- 172.21.15.89:0/1191757373 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3620069560 0x7f362014d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.342+0000 7f3625284700 1 -- 172.21.15.89:0/1191757373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3620159890 msgr2=0x7f3620153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:25.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.342+0000 7f3625284700 1 --2- 172.21.15.89:0/1191757373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3620159890 0x7f3620153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.342+0000 7f3625284700 1 -- 172.21.15.89:0/1191757373 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3618009420 con 0x7f3620159c60 2023-10-10T20:36:25.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.342+0000 7f3625284700 1 --2- 172.21.15.89:0/1191757373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3620159c60 0x7f3620153d70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f36180093f0 tx=0x7f3618016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:25.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.342+0000 7f3614ff9700 1 -- 172.21.15.89:0/1191757373 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361801b680 con 0x7f3620159c60 2023-10-10T20:36:25.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.342+0000 7f3614ff9700 1 -- 172.21.15.89:0/1191757373 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361801b800 con 0x7f3620159c60 2023-10-10T20:36:25.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.342+0000 7f3626ce7700 1 -- 172.21.15.89:0/1191757373 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36200ce730 con 0x7f3620159c60 2023-10-10T20:36:25.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.342+0000 7f3614ff9700 1 -- 172.21.15.89:0/1191757373 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361801bbc0 con 0x7f3620159c60 2023-10-10T20:36:25.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.342+0000 7f3626ce7700 1 -- 172.21.15.89:0/1191757373 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36200cd5c0 con 0x7f3620159c60 2023-10-10T20:36:25.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.343+0000 7f3626ce7700 1 -- 172.21.15.89:0/1191757373 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362015c650 con 0x7f3620159c60 2023-10-10T20:36:25.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.343+0000 7f3614ff9700 1 -- 172.21.15.89:0/1191757373 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f361801c450 con 0x7f3620159c60 2023-10-10T20:36:25.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.344+0000 7f3614ff9700 1 --2- 172.21.15.89:0/1191757373 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f360c066fc0 0x7f360c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:25.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.344+0000 7f3614ff9700 1 -- 172.21.15.89:0/1191757373 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f36180864c0 con 0x7f3620159c60 2023-10-10T20:36:25.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.344+0000 7f3624a83700 1 --2- 172.21.15.89:0/1191757373 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f360c066fc0 0x7f360c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:25.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.344+0000 7f3624a83700 1 --2- 172.21.15.89:0/1191757373 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f360c066fc0 0x7f360c069470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f361c007fd0 tx=0x7f361c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:25.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.348+0000 7f3614ff9700 1 -- 172.21.15.89:0/1191757373 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3618053740 con 0x7f3620159c60 2023-10-10T20:36:25.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.528+0000 7f3626ce7700 1 -- 172.21.15.89:0/1191757373 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f36200ce5c0 con 0x7f360c066fc0 2023-10-10T20:36:25.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.537+0000 7f3614ff9700 1 -- 172.21.15.89:0/1191757373 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289503 (secure 0 0 0) 0x7f36200ce5c0 con 0x7f360c066fc0 2023-10-10T20:36:25.542 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.540+0000 7f3626ce7700 1 -- 172.21.15.89:0/1191757373 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f360c066fc0 msgr2=0x7f360c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.540+0000 7f3626ce7700 1 --2- 172.21.15.89:0/1191757373 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f360c066fc0 0x7f360c069470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f361c007fd0 tx=0x7f361c009040 comp rx=0 tx=0).stop 2023-10-10T20:36:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.540+0000 7f3626ce7700 1 -- 172.21.15.89:0/1191757373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3620159c60 msgr2=0x7f3620153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.540+0000 7f3626ce7700 1 --2- 172.21.15.89:0/1191757373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3620159c60 0x7f3620153d70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f36180093f0 tx=0x7f3618016000 comp rx=0 tx=0).stop 2023-10-10T20:36:25.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.540+0000 7f3626ce7700 1 -- 172.21.15.89:0/1191757373 shutdown_connections 2023-10-10T20:36:25.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.540+0000 7f3626ce7700 1 --2- 172.21.15.89:0/1191757373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3620159c60 0x7f3620153d70 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.540+0000 7f3626ce7700 1 --2- 172.21.15.89:0/1191757373 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3620069560 0x7f362014d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.540+0000 7f3626ce7700 1 --2- 172.21.15.89:0/1191757373 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f360c066fc0 0x7f360c069470 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.540+0000 7f3626ce7700 1 --2- 172.21.15.89:0/1191757373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3620159890 0x7f3620153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.540+0000 7f3626ce7700 1 -- 172.21.15.89:0/1191757373 >> 172.21.15.89:0/1191757373 conn(0x7f3620068e20 msgr2=0x7f3620157cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:25.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.540+0000 7f3626ce7700 1 -- 172.21.15.89:0/1191757373 shutdown_connections 2023-10-10T20:36:25.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.540+0000 7f3626ce7700 1 -- 172.21.15.89:0/1191757373 wait complete. 2023-10-10T20:36:25.547 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:36:25.554 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2023-10-10T20:36:24.672494+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385340,"kb_used_data":7036,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752589380,"statfs":{"total":773094113280,"available":770651525120,"internally_reserved":0,"allocated":7204864,"data_stored":3709955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.190750"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.521217+0000","last_change":"2023-10-10T20:36:21.511544+0000","last_active":"2023-10-10T20:36:21.521217+0000","last_peered":"2023-10-10T20:36:21.521217+0000","last_clean":"2023-10-10T20:36:21.521217+0000","last_became_active":"2023-10-10T20:36:21.511287+0000","last_became_peered":"2023-10-10T20:36:21.511287+0000","last_unstale":"2023-10-10T20:36:21.521217+0000","last_undegraded":"2023-10-10T20:36:21.521217+0000","last_fullsized":"2023-10-10T20:36:21.521217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:56:38.109485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.519727+0000","last_change":"2023-10-10T20:36:21.519727+0000","last_active":"2023-10-10T20:36:21.519727+0000","last_peered":"2023-10-10T20:36:21.519727+0000","last_clean":"2023-10-10T20:36:21.519727+0000","last_became_active":"2023-10-10T20:36:21.519455+0000","last_became_peered":"2023-10-10T20:36:21.519455+0000","last_unstale":"2023-10-10T20:36:21.519727+0000","last_undegraded":"2023-10-10T20:36:21.519727+0000","last_fullsized":"2023-10-10T20:36:21.519727+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:01:01.948430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.515655+0000","last_change":"2023-10-10T20:36:21.515655+0000","last_active":"2023-10-10T20:36:21.515655+0000","last_peered":"2023-10-10T20:36:21.515655+0000","last_clean":"2023-10-10T20:36:21.515655+0000","last_became_active":"2023-10-10T20:36:21.515419+0000","last_became_peered":"2023-10-10T20:36:21.515419+0000","last_unstale":"2023-10-10T20:36:21.515655+0000","last_undegraded":"2023-10-10T20:36:21.515655+0000","last_fullsized":"2023-10-10T20:36:21.515655+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:21:52.585716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.518305+0000","last_change":"2023-10-10T20:36:21.518305+0000","last_active":"2023-10-10T20:36:21.518305+0000","last_peered":"2023-10-10T20:36:21.518305+0000","last_clean":"2023-10-10T20:36:21.518305+0000","last_became_active":"2023-10-10T20:36:21.518057+0000","last_became_peered":"2023-10-10T20:36:21.518057+0000","last_unstale":"2023-10-10T20:36:21.518305+0000","last_undegraded":"2023-10-10T20:36:21.518305+0000","last_fullsized":"2023-10-10T20:36:21.518305+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:57.315341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.504118+0000","last_change":"2023-10-10T20:36:21.504118+0000","last_active":"2023-10-10T20:36:21.504118+0000","last_peered":"2023-10-10T20:36:21.504118+0000","last_clean":"2023-10-10T20:36:21.504118+0000","last_became_active":"2023-10-10T20:36:21.503804+0000","last_became_peered":"2023-10-10T20:36:21.503804+0000","last_unstale":"2023-10-10T20:36:21.504118+0000","last_undegraded":"2023-10-10T20:36:21.504118+0000","last_fullsized":"2023-10-10T20:36:21.504118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:34:04.208699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.515472+0000","last_change":"2023-10-10T20:36:21.515472+0000","last_active":"2023-10-10T20:36:21.515472+0000","last_peered":"2023-10-10T20:36:21.515472+0000","last_clean":"2023-10-10T20:36:21.515472+0000","last_became_active":"2023-10-10T20:36:21.515186+0000","last_became_peered":"2023-10-10T20:36:21.515186+0000","last_unstale":"2023-10-10T20:36:21.515472+0000","last_undegraded":"2023-10-10T20:36:21.515472+0000","last_fullsized":"2023-10-10T20:36:21.515472+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:07:19.870599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.526249+0000","last_change":"2023-10-10T20:36:21.520963+0000","last_active":"2023-10-10T20:36:21.526249+0000","last_peered":"2023-10-10T20:36:21.526249+0000","last_clean":"2023-10-10T20:36:21.526249+0000","last_became_active":"2023-10-10T20:36:21.520699+0000","last_became_peered":"2023-10-10T20:36:21.520699+0000","last_unstale":"2023-10-10T20:36:21.526249+0000","last_undegraded":"2023-10-10T20:36:21.526249+0000","last_fullsized":"2023-10-10T20:36:21.526249+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:57:03.984128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.516964+0000","last_change":"2023-10-10T20:36:21.516964+0000","last_active":"2023-10-10T20:36:21.516964+0000","last_peered":"2023-10-10T20:36:21.516964+0000","last_clean":"2023-10-10T20:36:21.516964+0000","last_became_active":"2023-10-10T20:36:21.516644+0000","last_became_peered":"2023-10-10T20:36:21.516644+0000","last_unstale":"2023-10-10T20:36:21.516964+0000","last_undegraded":"2023-10-10T20:36:21.516964+0000","last_fullsized":"2023-10-10T20:36:21.516964+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:14.041794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.515833+0000","last_change":"2023-10-10T20:36:21.515833+0000","last_active":"2023-10-10T20:36:21.515833+0000","last_peered":"2023-10-10T20:36:21.515833+0000","last_clean":"2023-10-10T20:36:21.515833+0000","last_became_active":"2023-10-10T20:36:21.515566+0000","last_became_peered":"2023-10-10T20:36:21.515566+0000","last_unstale":"2023-10-10T20:36:21.515833+0000","last_undegraded":"2023-10-10T20:36:21.515833+0000","last_fullsized":"2023-10-10T20:36:21.515833+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:28.201245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.532818+0000","last_change":"2023-10-10T20:36:21.525476+0000","last_active":"2023-10-10T20:36:21.532818+0000","last_peered":"2023-10-10T20:36:21.532818+0000","last_clean":"2023-10-10T20:36:21.532818+0000","last_became_active":"2023-10-10T20:36:21.525200+0000","last_became_peered":"2023-10-10T20:36:21.525200+0000","last_unstale":"2023-10-10T20:36:21.532818+0000","last_undegraded":"2023-10-10T20:36:21.532818+0000","last_fullsized":"2023-10-10T20:36:21.532818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:45:39.050374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.521912+0000","last_change":"2023-10-10T20:36:21.513019+0000","last_active":"2023-10-10T20:36:21.521912+0000","last_peered":"2023-10-10T20:36:21.521912+0000","last_clean":"2023-10-10T20:36:21.521912+0000","last_became_active":"2023-10-10T20:36:21.512756+0000","last_became_peered":"2023-10-10T20:36:21.512756+0000","last_unstale":"2023-10-10T20:36:21.521912+0000","last_undegraded":"2023-10-10T20:36:21.521912+0000","last_fullsized":"2023-10-10T20:36:21.521912+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:36:10.486811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.510556+0000","last_change":"2023-10-10T20:36:21.510556+0000","last_active":"2023-10-10T20:36:21.510556+0000","last_peered":"2023-10-10T20:36:21.510556+0000","last_clean":"2023-10-10T20:36:21.510556+0000","last_became_active":"2023-10-10T20:36:21.510245+0000","last_became_peered":"2023-10-10T20:36:21.510245+0000","last_unstale":"2023-10-10T20:36:21.510556+0000","last_undegraded":"2023-10-10T20:36:21.510556+0000","last_fullsized":"2023-10-10T20:36:21.510556+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:27:20.416644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.515701+0000","last_change":"2023-10-10T20:36:21.515701+0000","last_active":"2023-10-10T20:36:21.515701+0000","last_peered":"2023-10-10T20:36:21.515701+0000","last_clean":"2023-10-10T20:36:21.515701+0000","last_became_active":"2023-10-10T20:36:21.515442+0000","last_became_peered":"2023-10-10T20:36:21.515442+0000","last_unstale":"2023-10-10T20:36:21.515701+0000","last_undegraded":"2023-10-10T20:36:21.515701+0000","last_fullsized":"2023-10-10T20:36:21.515701+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:07.707339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.516371+0000","last_change":"2023-10-10T20:36:21.516371+0000","last_active":"2023-10-10T20:36:21.516371+0000","last_peered":"2023-10-10T20:36:21.516371+0000","last_clean":"2023-10-10T20:36:21.516371+0000","last_became_active":"2023-10-10T20:36:21.516074+0000","last_became_peered":"2023-10-10T20:36:21.516074+0000","last_unstale":"2023-10-10T20:36:21.516371+0000","last_undegraded":"2023-10-10T20:36:21.516371+0000","last_fullsized":"2023-10-10T20:36:21.516371+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:20.928729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.507633+0000","last_change":"2023-10-10T20:36:21.507633+0000","last_active":"2023-10-10T20:36:21.507633+0000","last_peered":"2023-10-10T20:36:21.507633+0000","last_clean":"2023-10-10T20:36:21.507633+0000","last_became_active":"2023-10-10T20:36:21.507359+0000","last_became_peered":"2023-10-10T20:36:21.507359+0000","last_unstale":"2023-10-10T20:36:21.507633+0000","last_undegraded":"2023-10-10T20:36:21.507633+0000","last_fullsized":"2023-10-10T20:36:21.507633+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:31.961936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.531588+0000","last_change":"2023-10-10T20:36:21.522891+0000","last_active":"2023-10-10T20:36:21.531588+0000","last_peered":"2023-10-10T20:36:21.531588+0000","last_clean":"2023-10-10T20:36:21.531588+0000","last_became_active":"2023-10-10T20:36:21.522646+0000","last_became_peered":"2023-10-10T20:36:21.522646+0000","last_unstale":"2023-10-10T20:36:21.531588+0000","last_undegraded":"2023-10-10T20:36:21.531588+0000","last_fullsized":"2023-10-10T20:36:21.531588+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:39:14.489065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.516141+0000","last_change":"2023-10-10T20:36:21.516141+0000","last_active":"2023-10-10T20:36:21.516141+0000","last_peered":"2023-10-10T20:36:21.516141+0000","last_clean":"2023-10-10T20:36:21.516141+0000","last_became_active":"2023-10-10T20:36:21.515786+0000","last_became_peered":"2023-10-10T20:36:21.515786+0000","last_unstale":"2023-10-10T20:36:21.516141+0000","last_undegraded":"2023-10-10T20:36:21.516141+0000","last_fullsized":"2023-10-10T20:36:21.516141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:14:47.981506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.517645+0000","last_change":"2023-10-10T20:36:21.517645+0000","last_active":"2023-10-10T20:36:21.517645+0000","last_peered":"2023-10-10T20:36:21.517645+0000","last_clean":"2023-10-10T20:36:21.517645+0000","last_became_active":"2023-10-10T20:36:21.517409+0000","last_became_peered":"2023-10-10T20:36:21.517409+0000","last_unstale":"2023-10-10T20:36:21.517645+0000","last_undegraded":"2023-10-10T20:36:21.517645+0000","last_fullsized":"2023-10-10T20:36:21.517645+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:14.041794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.517258+0000","last_change":"2023-10-10T20:36:21.517258+0000","last_active":"2023-10-10T20:36:21.517258+0000","last_peered":"2023-10-10T20:36:21.517258+0000","last_clean":"2023-10-10T20:36:21.517258+0000","last_became_active":"2023-10-10T20:36:21.516848+0000","last_became_peered":"2023-10-10T20:36:21.516848+0000","last_unstale":"2023-10-10T20:36:21.517258+0000","last_undegraded":"2023-10-10T20:36:21.517258+0000","last_fullsized":"2023-10-10T20:36:21.517258+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:37.824587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.515143+0000","last_change":"2023-10-10T20:36:21.515143+0000","last_active":"2023-10-10T20:36:21.515143+0000","last_peered":"2023-10-10T20:36:21.515143+0000","last_clean":"2023-10-10T20:36:21.515143+0000","last_became_active":"2023-10-10T20:36:21.514886+0000","last_became_peered":"2023-10-10T20:36:21.514886+0000","last_unstale":"2023-10-10T20:36:21.515143+0000","last_undegraded":"2023-10-10T20:36:21.515143+0000","last_fullsized":"2023-10-10T20:36:21.515143+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:28:02.757484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.524465+0000","last_change":"2023-10-10T20:36:21.521017+0000","last_active":"2023-10-10T20:36:21.524465+0000","last_peered":"2023-10-10T20:36:21.524465+0000","last_clean":"2023-10-10T20:36:21.524465+0000","last_became_active":"2023-10-10T20:36:21.520761+0000","last_became_peered":"2023-10-10T20:36:21.520761+0000","last_unstale":"2023-10-10T20:36:21.524465+0000","last_undegraded":"2023-10-10T20:36:21.524465+0000","last_fullsized":"2023-10-10T20:36:21.524465+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:07.707339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.524481+0000","last_change":"2023-10-10T20:36:21.522024+0000","last_active":"2023-10-10T20:36:21.524481+0000","last_peered":"2023-10-10T20:36:21.524481+0000","last_clean":"2023-10-10T20:36:21.524481+0000","last_became_active":"2023-10-10T20:36:21.521753+0000","last_became_peered":"2023-10-10T20:36:21.521753+0000","last_unstale":"2023-10-10T20:36:21.524481+0000","last_undegraded":"2023-10-10T20:36:21.524481+0000","last_fullsized":"2023-10-10T20:36:21.524481+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:01:01.948430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.528031+0000","last_change":"2023-10-10T20:36:21.519328+0000","last_active":"2023-10-10T20:36:21.528031+0000","last_peered":"2023-10-10T20:36:21.528031+0000","last_clean":"2023-10-10T20:36:21.528031+0000","last_became_active":"2023-10-10T20:36:21.519068+0000","last_became_peered":"2023-10-10T20:36:21.519068+0000","last_unstale":"2023-10-10T20:36:21.528031+0000","last_undegraded":"2023-10-10T20:36:21.528031+0000","last_fullsized":"2023-10-10T20:36:21.528031+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:23:47.669757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.528039+0000","last_change":"2023-10-10T20:36:21.521383+0000","last_active":"2023-10-10T20:36:21.528039+0000","last_peered":"2023-10-10T20:36:21.528039+0000","last_clean":"2023-10-10T20:36:21.528039+0000","last_became_active":"2023-10-10T20:36:21.517956+0000","last_became_peered":"2023-10-10T20:36:21.517956+0000","last_unstale":"2023-10-10T20:36:21.528039+0000","last_undegraded":"2023-10-10T20:36:21.528039+0000","last_fullsized":"2023-10-10T20:36:21.528039+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:00:10.479829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.532734+0000","last_change":"2023-10-10T20:36:21.525888+0000","last_active":"2023-10-10T20:36:21.532734+0000","last_peered":"2023-10-10T20:36:21.532734+0000","last_clean":"2023-10-10T20:36:21.532734+0000","last_became_active":"2023-10-10T20:36:21.525043+0000","last_became_peered":"2023-10-10T20:36:21.525043+0000","last_unstale":"2023-10-10T20:36:21.532734+0000","last_undegraded":"2023-10-10T20:36:21.532734+0000","last_fullsized":"2023-10-10T20:36:21.532734+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:52:24.203489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.522142+0000","last_change":"2023-10-10T20:36:21.517680+0000","last_active":"2023-10-10T20:36:21.522142+0000","last_peered":"2023-10-10T20:36:21.522142+0000","last_clean":"2023-10-10T20:36:21.522142+0000","last_became_active":"2023-10-10T20:36:21.517432+0000","last_became_peered":"2023-10-10T20:36:21.517432+0000","last_unstale":"2023-10-10T20:36:21.522142+0000","last_undegraded":"2023-10-10T20:36:21.522142+0000","last_fullsized":"2023-10-10T20:36:21.522142+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:44:10.575443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.519544+0000","last_change":"2023-10-10T20:36:21.519544+0000","last_active":"2023-10-10T20:36:21.519544+0000","last_peered":"2023-10-10T20:36:21.519544+0000","last_clean":"2023-10-10T20:36:21.519544+0000","last_became_active":"2023-10-10T20:36:21.519243+0000","last_became_peered":"2023-10-10T20:36:21.519243+0000","last_unstale":"2023-10-10T20:36:21.519544+0000","last_undegraded":"2023-10-10T20:36:21.519544+0000","last_fullsized":"2023-10-10T20:36:21.519544+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:09:19.504447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.532687+0000","last_change":"2023-10-10T20:36:21.525603+0000","last_active":"2023-10-10T20:36:21.532687+0000","last_peered":"2023-10-10T20:36:21.532687+0000","last_clean":"2023-10-10T20:36:21.532687+0000","last_became_active":"2023-10-10T20:36:21.525318+0000","last_became_peered":"2023-10-10T20:36:21.525318+0000","last_unstale":"2023-10-10T20:36:21.532687+0000","last_undegraded":"2023-10-10T20:36:21.532687+0000","last_fullsized":"2023-10-10T20:36:21.532687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:09.868421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.518446+0000","last_change":"2023-10-10T20:36:21.518446+0000","last_active":"2023-10-10T20:36:21.518446+0000","last_peered":"2023-10-10T20:36:21.518446+0000","last_clean":"2023-10-10T20:36:21.518446+0000","last_became_active":"2023-10-10T20:36:21.518099+0000","last_became_peered":"2023-10-10T20:36:21.518099+0000","last_unstale":"2023-10-10T20:36:21.518446+0000","last_undegraded":"2023-10-10T20:36:21.518446+0000","last_fullsized":"2023-10-10T20:36:21.518446+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:45:20.306180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.520878+0000","last_change":"2023-10-10T20:36:21.515422+0000","last_active":"2023-10-10T20:36:21.520878+0000","last_peered":"2023-10-10T20:36:21.520878+0000","last_clean":"2023-10-10T20:36:21.520878+0000","last_became_active":"2023-10-10T20:36:21.515179+0000","last_became_peered":"2023-10-10T20:36:21.515179+0000","last_unstale":"2023-10-10T20:36:21.520878+0000","last_undegraded":"2023-10-10T20:36:21.520878+0000","last_fullsized":"2023-10-10T20:36:21.520878+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:36:10.486811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.526272+0000","last_change":"2023-10-10T20:36:21.523186+0000","last_active":"2023-10-10T20:36:21.526272+0000","last_peered":"2023-10-10T20:36:21.526272+0000","last_clean":"2023-10-10T20:36:21.526272+0000","last_became_active":"2023-10-10T20:36:21.522938+0000","last_became_peered":"2023-10-10T20:36:21.522938+0000","last_unstale":"2023-10-10T20:36:21.526272+0000","last_undegraded":"2023-10-10T20:36:21.526272+0000","last_fullsized":"2023-10-10T20:36:21.526272+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:56:38.109485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.521756+0000","last_change":"2023-10-10T20:36:21.521756+0000","last_active":"2023-10-10T20:36:21.521756+0000","last_peered":"2023-10-10T20:36:21.521756+0000","last_clean":"2023-10-10T20:36:21.521756+0000","last_became_active":"2023-10-10T20:36:21.521484+0000","last_became_peered":"2023-10-10T20:36:21.521484+0000","last_unstale":"2023-10-10T20:36:21.521756+0000","last_undegraded":"2023-10-10T20:36:21.521756+0000","last_fullsized":"2023-10-10T20:36:21.521756+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:39:14.489065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.523993+0000","last_change":"2023-10-10T20:36:21.515995+0000","last_active":"2023-10-10T20:36:21.523993+0000","last_peered":"2023-10-10T20:36:21.523993+0000","last_clean":"2023-10-10T20:36:21.523993+0000","last_became_active":"2023-10-10T20:36:21.515463+0000","last_became_peered":"2023-10-10T20:36:21.515463+0000","last_unstale":"2023-10-10T20:36:21.523993+0000","last_undegraded":"2023-10-10T20:36:21.523993+0000","last_fullsized":"2023-10-10T20:36:21.523993+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:48:14.169260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"61'4","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.531211+0000","last_change":"2023-10-10T20:36:20.498938+0000","last_active":"2023-10-10T20:36:21.531211+0000","last_peered":"2023-10-10T20:36:21.531211+0000","last_clean":"2023-10-10T20:36:21.531211+0000","last_became_active":"2023-10-10T20:36:20.498679+0000","last_became_peered":"2023-10-10T20:36:20.498679+0000","last_unstale":"2023-10-10T20:36:21.531211+0000","last_undegraded":"2023-10-10T20:36:21.531211+0000","last_fullsized":"2023-10-10T20:36:21.531211+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:43:19.431501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.518818+0000","last_change":"2023-10-10T20:36:21.518818+0000","last_active":"2023-10-10T20:36:21.518818+0000","last_peered":"2023-10-10T20:36:21.518818+0000","last_clean":"2023-10-10T20:36:21.518818+0000","last_became_active":"2023-10-10T20:36:21.518607+0000","last_became_peered":"2023-10-10T20:36:21.518607+0000","last_unstale":"2023-10-10T20:36:21.518818+0000","last_undegraded":"2023-10-10T20:36:21.518818+0000","last_fullsized":"2023-10-10T20:36:21.518818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:10:51.485286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.502947+0000","last_change":"2023-10-10T20:36:20.506790+0000","last_active":"2023-10-10T20:36:21.502947+0000","last_peered":"2023-10-10T20:36:21.502947+0000","last_clean":"2023-10-10T20:36:21.502947+0000","last_became_active":"2023-10-10T20:36:20.506501+0000","last_became_peered":"2023-10-10T20:36:20.506501+0000","last_unstale":"2023-10-10T20:36:21.502947+0000","last_undegraded":"2023-10-10T20:36:21.502947+0000","last_fullsized":"2023-10-10T20:36:21.502947+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:56.311673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.525588+0000","last_change":"2023-10-10T20:36:21.525588+0000","last_active":"2023-10-10T20:36:21.525588+0000","last_peered":"2023-10-10T20:36:21.525588+0000","last_clean":"2023-10-10T20:36:21.525588+0000","last_became_active":"2023-10-10T20:36:21.525338+0000","last_became_peered":"2023-10-10T20:36:21.525338+0000","last_unstale":"2023-10-10T20:36:21.525588+0000","last_undegraded":"2023-10-10T20:36:21.525588+0000","last_fullsized":"2023-10-10T20:36:21.525588+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:52:24.203489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"61'2","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.532332+0000","last_change":"2023-10-10T20:36:20.500244+0000","last_active":"2023-10-10T20:36:21.532332+0000","last_peered":"2023-10-10T20:36:21.532332+0000","last_clean":"2023-10-10T20:36:21.532332+0000","last_became_active":"2023-10-10T20:36:20.499989+0000","last_became_peered":"2023-10-10T20:36:20.499989+0000","last_unstale":"2023-10-10T20:36:21.532332+0000","last_undegraded":"2023-10-10T20:36:21.532332+0000","last_fullsized":"2023-10-10T20:36:21.532332+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:27:22.089809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.517730+0000","last_change":"2023-10-10T20:36:21.510979+0000","last_active":"2023-10-10T20:36:21.517730+0000","last_peered":"2023-10-10T20:36:21.517730+0000","last_clean":"2023-10-10T20:36:21.517730+0000","last_became_active":"2023-10-10T20:36:21.510713+0000","last_became_peered":"2023-10-10T20:36:21.510713+0000","last_unstale":"2023-10-10T20:36:21.517730+0000","last_undegraded":"2023-10-10T20:36:21.517730+0000","last_fullsized":"2023-10-10T20:36:21.517730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:33:07.729616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.530864+0000","last_change":"2023-10-10T20:36:20.500071+0000","last_active":"2023-10-10T20:36:21.530864+0000","last_peered":"2023-10-10T20:36:21.530864+0000","last_clean":"2023-10-10T20:36:21.530864+0000","last_became_active":"2023-10-10T20:36:20.499578+0000","last_became_peered":"2023-10-10T20:36:20.499578+0000","last_unstale":"2023-10-10T20:36:21.530864+0000","last_undegraded":"2023-10-10T20:36:21.530864+0000","last_fullsized":"2023-10-10T20:36:21.530864+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:41.117726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.520858+0000","last_change":"2023-10-10T20:36:21.514919+0000","last_active":"2023-10-10T20:36:21.520858+0000","last_peered":"2023-10-10T20:36:21.520858+0000","last_clean":"2023-10-10T20:36:21.520858+0000","last_became_active":"2023-10-10T20:36:21.514650+0000","last_became_peered":"2023-10-10T20:36:21.514650+0000","last_unstale":"2023-10-10T20:36:21.520858+0000","last_undegraded":"2023-10-10T20:36:21.520858+0000","last_fullsized":"2023-10-10T20:36:21.520858+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:45:07.874792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.508985+0000","last_change":"2023-10-10T20:36:20.498535+0000","last_active":"2023-10-10T20:36:21.508985+0000","last_peered":"2023-10-10T20:36:21.508985+0000","last_clean":"2023-10-10T20:36:21.508985+0000","last_became_active":"2023-10-10T20:36:20.498275+0000","last_became_peered":"2023-10-10T20:36:20.498275+0000","last_unstale":"2023-10-10T20:36:21.508985+0000","last_undegraded":"2023-10-10T20:36:21.508985+0000","last_fullsized":"2023-10-10T20:36:21.508985+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:23:46.666090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.532798+0000","last_change":"2023-10-10T20:36:21.527199+0000","last_active":"2023-10-10T20:36:21.532798+0000","last_peered":"2023-10-10T20:36:21.532798+0000","last_clean":"2023-10-10T20:36:21.532798+0000","last_became_active":"2023-10-10T20:36:21.526474+0000","last_became_peered":"2023-10-10T20:36:21.526474+0000","last_unstale":"2023-10-10T20:36:21.532798+0000","last_undegraded":"2023-10-10T20:36:21.532798+0000","last_fullsized":"2023-10-10T20:36:21.532798+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:11:36.397020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.511709+0000","last_change":"2023-10-10T20:36:20.501670+0000","last_active":"2023-10-10T20:36:21.511709+0000","last_peered":"2023-10-10T20:36:21.511709+0000","last_clean":"2023-10-10T20:36:21.511709+0000","last_became_active":"2023-10-10T20:36:20.501390+0000","last_became_peered":"2023-10-10T20:36:20.501390+0000","last_unstale":"2023-10-10T20:36:21.511709+0000","last_undegraded":"2023-10-10T20:36:21.511709+0000","last_fullsized":"2023-10-10T20:36:21.511709+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:50.029620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.515235+0000","last_change":"2023-10-10T20:36:21.515235+0000","last_active":"2023-10-10T20:36:21.515235+0000","last_peered":"2023-10-10T20:36:21.515235+0000","last_clean":"2023-10-10T20:36:21.515235+0000","last_became_active":"2023-10-10T20:36:21.514989+0000","last_became_peered":"2023-10-10T20:36:21.514989+0000","last_unstale":"2023-10-10T20:36:21.515235+0000","last_undegraded":"2023-10-10T20:36:21.515235+0000","last_fullsized":"2023-10-10T20:36:21.515235+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:44:25.114526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.499925+0000","last_change":"2023-10-10T20:36:20.497222+0000","last_active":"2023-10-10T20:36:20.499925+0000","last_peered":"2023-10-10T20:36:20.499925+0000","last_clean":"2023-10-10T20:36:20.499925+0000","last_became_active":"2023-10-10T20:36:20.496978+0000","last_became_peered":"2023-10-10T20:36:20.496978+0000","last_unstale":"2023-10-10T20:36:20.499925+0000","last_undegraded":"2023-10-10T20:36:20.499925+0000","last_fullsized":"2023-10-10T20:36:20.499925+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:53:12.126771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.509669+0000","last_change":"2023-10-10T20:36:21.509669+0000","last_active":"2023-10-10T20:36:21.509669+0000","last_peered":"2023-10-10T20:36:21.509669+0000","last_clean":"2023-10-10T20:36:21.509669+0000","last_became_active":"2023-10-10T20:36:21.509433+0000","last_became_peered":"2023-10-10T20:36:21.509433+0000","last_unstale":"2023-10-10T20:36:21.509669+0000","last_undegraded":"2023-10-10T20:36:21.509669+0000","last_fullsized":"2023-10-10T20:36:21.509669+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:24:21.171886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.506166+0000","last_change":"2023-10-10T20:36:20.500783+0000","last_active":"2023-10-10T20:36:20.506166+0000","last_peered":"2023-10-10T20:36:20.506166+0000","last_clean":"2023-10-10T20:36:20.506166+0000","last_became_active":"2023-10-10T20:36:20.500503+0000","last_became_peered":"2023-10-10T20:36:20.500503+0000","last_unstale":"2023-10-10T20:36:20.506166+0000","last_undegraded":"2023-10-10T20:36:20.506166+0000","last_fullsized":"2023-10-10T20:36:20.506166+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:19:05.349730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.522122+0000","last_change":"2023-10-10T20:36:21.516803+0000","last_active":"2023-10-10T20:36:21.522122+0000","last_peered":"2023-10-10T20:36:21.522122+0000","last_clean":"2023-10-10T20:36:21.522122+0000","last_became_active":"2023-10-10T20:36:21.516512+0000","last_became_peered":"2023-10-10T20:36:21.516512+0000","last_unstale":"2023-10-10T20:36:21.522122+0000","last_undegraded":"2023-10-10T20:36:21.522122+0000","last_fullsized":"2023-10-10T20:36:21.522122+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:33:13.394154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.512956+0000","last_change":"2023-10-10T20:36:20.498783+0000","last_active":"2023-10-10T20:36:21.512956+0000","last_peered":"2023-10-10T20:36:21.512956+0000","last_clean":"2023-10-10T20:36:21.512956+0000","last_became_active":"2023-10-10T20:36:20.497548+0000","last_became_peered":"2023-10-10T20:36:20.497548+0000","last_unstale":"2023-10-10T20:36:21.512956+0000","last_undegraded":"2023-10-10T20:36:21.512956+0000","last_fullsized":"2023-10-10T20:36:21.512956+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:53:12.126771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.524056+0000","last_change":"2023-10-10T20:36:21.516709+0000","last_active":"2023-10-10T20:36:21.524056+0000","last_peered":"2023-10-10T20:36:21.524056+0000","last_clean":"2023-10-10T20:36:21.524056+0000","last_became_active":"2023-10-10T20:36:21.516423+0000","last_became_peered":"2023-10-10T20:36:21.516423+0000","last_unstale":"2023-10-10T20:36:21.524056+0000","last_undegraded":"2023-10-10T20:36:21.524056+0000","last_fullsized":"2023-10-10T20:36:21.524056+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:11:23.892822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.506162+0000","last_change":"2023-10-10T20:36:20.504774+0000","last_active":"2023-10-10T20:36:20.506162+0000","last_peered":"2023-10-10T20:36:20.506162+0000","last_clean":"2023-10-10T20:36:20.506162+0000","last_became_active":"2023-10-10T20:36:20.504518+0000","last_became_peered":"2023-10-10T20:36:20.504518+0000","last_unstale":"2023-10-10T20:36:20.506162+0000","last_undegraded":"2023-10-10T20:36:20.506162+0000","last_fullsized":"2023-10-10T20:36:20.506162+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:17:20.214488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.520222+0000","last_change":"2023-10-10T20:36:21.520222+0000","last_active":"2023-10-10T20:36:21.520222+0000","last_peered":"2023-10-10T20:36:21.520222+0000","last_clean":"2023-10-10T20:36:21.520222+0000","last_became_active":"2023-10-10T20:36:21.519816+0000","last_became_peered":"2023-10-10T20:36:21.519816+0000","last_unstale":"2023-10-10T20:36:21.520222+0000","last_undegraded":"2023-10-10T20:36:21.520222+0000","last_fullsized":"2023-10-10T20:36:21.520222+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:57.315341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"61'3","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.530707+0000","last_change":"2023-10-10T20:36:20.494259+0000","last_active":"2023-10-10T20:36:21.530707+0000","last_peered":"2023-10-10T20:36:21.530707+0000","last_clean":"2023-10-10T20:36:21.530707+0000","last_became_active":"2023-10-10T20:36:20.493992+0000","last_became_peered":"2023-10-10T20:36:20.493992+0000","last_unstale":"2023-10-10T20:36:21.530707+0000","last_undegraded":"2023-10-10T20:36:21.530707+0000","last_fullsized":"2023-10-10T20:36:21.530707+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:09:09.540955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.511431+0000","last_change":"2023-10-10T20:36:21.511431+0000","last_active":"2023-10-10T20:36:21.511431+0000","last_peered":"2023-10-10T20:36:21.511431+0000","last_clean":"2023-10-10T20:36:21.511431+0000","last_became_active":"2023-10-10T20:36:21.511162+0000","last_became_peered":"2023-10-10T20:36:21.511162+0000","last_unstale":"2023-10-10T20:36:21.511431+0000","last_undegraded":"2023-10-10T20:36:21.511431+0000","last_fullsized":"2023-10-10T20:36:21.511431+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:21:57.926200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"61'6","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.530767+0000","last_change":"2023-10-10T20:36:20.498323+0000","last_active":"2023-10-10T20:36:21.530767+0000","last_peered":"2023-10-10T20:36:21.530767+0000","last_clean":"2023-10-10T20:36:21.530767+0000","last_became_active":"2023-10-10T20:36:20.497587+0000","last_became_peered":"2023-10-10T20:36:20.497587+0000","last_unstale":"2023-10-10T20:36:21.530767+0000","last_undegraded":"2023-10-10T20:36:21.530767+0000","last_fullsized":"2023-10-10T20:36:21.530767+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:50.029620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.521183+0000","last_change":"2023-10-10T20:36:21.511754+0000","last_active":"2023-10-10T20:36:21.521183+0000","last_peered":"2023-10-10T20:36:21.521183+0000","last_clean":"2023-10-10T20:36:21.521183+0000","last_became_active":"2023-10-10T20:36:21.511357+0000","last_became_peered":"2023-10-10T20:36:21.511357+0000","last_unstale":"2023-10-10T20:36:21.521183+0000","last_undegraded":"2023-10-10T20:36:21.521183+0000","last_fullsized":"2023-10-10T20:36:21.521183+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:33:13.394154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.529310+0000","last_change":"2023-10-10T20:36:20.501259+0000","last_active":"2023-10-10T20:36:21.529310+0000","last_peered":"2023-10-10T20:36:21.529310+0000","last_clean":"2023-10-10T20:36:21.529310+0000","last_became_active":"2023-10-10T20:36:20.497900+0000","last_became_peered":"2023-10-10T20:36:20.497900+0000","last_unstale":"2023-10-10T20:36:21.529310+0000","last_undegraded":"2023-10-10T20:36:21.529310+0000","last_fullsized":"2023-10-10T20:36:21.529310+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:35:50.152564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.521193+0000","last_change":"2023-10-10T20:36:21.516087+0000","last_active":"2023-10-10T20:36:21.521193+0000","last_peered":"2023-10-10T20:36:21.521193+0000","last_clean":"2023-10-10T20:36:21.521193+0000","last_became_active":"2023-10-10T20:36:21.515800+0000","last_became_peered":"2023-10-10T20:36:21.515800+0000","last_unstale":"2023-10-10T20:36:21.521193+0000","last_undegraded":"2023-10-10T20:36:21.521193+0000","last_fullsized":"2023-10-10T20:36:21.521193+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:24:56.826111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.508889+0000","last_change":"2023-10-10T20:36:20.508382+0000","last_active":"2023-10-10T20:36:20.508889+0000","last_peered":"2023-10-10T20:36:20.508889+0000","last_clean":"2023-10-10T20:36:20.508889+0000","last_became_active":"2023-10-10T20:36:20.508154+0000","last_became_peered":"2023-10-10T20:36:20.508154+0000","last_unstale":"2023-10-10T20:36:20.508889+0000","last_undegraded":"2023-10-10T20:36:20.508889+0000","last_fullsized":"2023-10-10T20:36:20.508889+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:24:41.733760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.517285+0000","last_change":"2023-10-10T20:36:21.517285+0000","last_active":"2023-10-10T20:36:21.517285+0000","last_peered":"2023-10-10T20:36:21.517285+0000","last_clean":"2023-10-10T20:36:21.517285+0000","last_became_active":"2023-10-10T20:36:21.517040+0000","last_became_peered":"2023-10-10T20:36:21.517040+0000","last_unstale":"2023-10-10T20:36:21.517285+0000","last_undegraded":"2023-10-10T20:36:21.517285+0000","last_fullsized":"2023-10-10T20:36:21.517285+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:13:56.076698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.511861+0000","last_change":"2023-10-10T20:36:20.503387+0000","last_active":"2023-10-10T20:36:21.511861+0000","last_peered":"2023-10-10T20:36:21.511861+0000","last_clean":"2023-10-10T20:36:21.511861+0000","last_became_active":"2023-10-10T20:36:20.503089+0000","last_became_peered":"2023-10-10T20:36:20.503089+0000","last_unstale":"2023-10-10T20:36:21.511861+0000","last_undegraded":"2023-10-10T20:36:21.511861+0000","last_fullsized":"2023-10-10T20:36:21.511861+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:06.703672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.514118+0000","last_change":"2023-10-10T20:36:21.514118+0000","last_active":"2023-10-10T20:36:21.514118+0000","last_peered":"2023-10-10T20:36:21.514118+0000","last_clean":"2023-10-10T20:36:21.514118+0000","last_became_active":"2023-10-10T20:36:21.513869+0000","last_became_peered":"2023-10-10T20:36:21.513869+0000","last_unstale":"2023-10-10T20:36:21.514118+0000","last_undegraded":"2023-10-10T20:36:21.514118+0000","last_fullsized":"2023-10-10T20:36:21.514118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:42.776284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"61'8","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.533014+0000","last_change":"2023-10-10T20:36:20.499777+0000","last_active":"2023-10-10T20:36:21.533014+0000","last_peered":"2023-10-10T20:36:21.533014+0000","last_clean":"2023-10-10T20:36:21.533014+0000","last_became_active":"2023-10-10T20:36:20.499501+0000","last_became_peered":"2023-10-10T20:36:20.499501+0000","last_unstale":"2023-10-10T20:36:21.533014+0000","last_undegraded":"2023-10-10T20:36:21.533014+0000","last_fullsized":"2023-10-10T20:36:21.533014+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:35:50.152564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.521417+0000","last_change":"2023-10-10T20:36:21.518427+0000","last_active":"2023-10-10T20:36:21.521417+0000","last_peered":"2023-10-10T20:36:21.521417+0000","last_clean":"2023-10-10T20:36:21.521417+0000","last_became_active":"2023-10-10T20:36:21.518151+0000","last_became_peered":"2023-10-10T20:36:21.518151+0000","last_unstale":"2023-10-10T20:36:21.521417+0000","last_undegraded":"2023-10-10T20:36:21.521417+0000","last_fullsized":"2023-10-10T20:36:21.521417+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:57.315341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.511731+0000","last_change":"2023-10-10T20:36:20.509410+0000","last_active":"2023-10-10T20:36:21.511731+0000","last_peered":"2023-10-10T20:36:21.511731+0000","last_clean":"2023-10-10T20:36:21.511731+0000","last_became_active":"2023-10-10T20:36:20.509049+0000","last_became_peered":"2023-10-10T20:36:20.509049+0000","last_unstale":"2023-10-10T20:36:21.511731+0000","last_undegraded":"2023-10-10T20:36:21.511731+0000","last_fullsized":"2023-10-10T20:36:21.511731+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:38:55.245786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.522220+0000","last_change":"2023-10-10T20:36:21.513324+0000","last_active":"2023-10-10T20:36:21.522220+0000","last_peered":"2023-10-10T20:36:21.522220+0000","last_clean":"2023-10-10T20:36:21.522220+0000","last_became_active":"2023-10-10T20:36:21.512662+0000","last_became_peered":"2023-10-10T20:36:21.512662+0000","last_unstale":"2023-10-10T20:36:21.522220+0000","last_undegraded":"2023-10-10T20:36:21.522220+0000","last_fullsized":"2023-10-10T20:36:21.522220+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:24:56.826111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"61'2","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.532751+0000","last_change":"2023-10-10T20:36:20.504643+0000","last_active":"2023-10-10T20:36:21.532751+0000","last_peered":"2023-10-10T20:36:21.532751+0000","last_clean":"2023-10-10T20:36:21.532751+0000","last_became_active":"2023-10-10T20:36:20.504381+0000","last_became_peered":"2023-10-10T20:36:20.504381+0000","last_unstale":"2023-10-10T20:36:21.532751+0000","last_undegraded":"2023-10-10T20:36:21.532751+0000","last_fullsized":"2023-10-10T20:36:21.532751+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:35:08.260987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.510308+0000","last_change":"2023-10-10T20:36:21.510308+0000","last_active":"2023-10-10T20:36:21.510308+0000","last_peered":"2023-10-10T20:36:21.510308+0000","last_clean":"2023-10-10T20:36:21.510308+0000","last_became_active":"2023-10-10T20:36:21.510051+0000","last_became_peered":"2023-10-10T20:36:21.510051+0000","last_unstale":"2023-10-10T20:36:21.510308+0000","last_undegraded":"2023-10-10T20:36:21.510308+0000","last_fullsized":"2023-10-10T20:36:21.510308+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:07:19.870599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.514109+0000","last_change":"2023-10-10T20:36:20.500546+0000","last_active":"2023-10-10T20:36:21.514109+0000","last_peered":"2023-10-10T20:36:21.514109+0000","last_clean":"2023-10-10T20:36:21.514109+0000","last_became_active":"2023-10-10T20:36:20.500290+0000","last_became_peered":"2023-10-10T20:36:20.500290+0000","last_unstale":"2023-10-10T20:36:21.514109+0000","last_undegraded":"2023-10-10T20:36:21.514109+0000","last_fullsized":"2023-10-10T20:36:21.514109+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:25.236700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.517145+0000","last_change":"2023-10-10T20:36:21.517145+0000","last_active":"2023-10-10T20:36:21.517145+0000","last_peered":"2023-10-10T20:36:21.517145+0000","last_clean":"2023-10-10T20:36:21.517145+0000","last_became_active":"2023-10-10T20:36:21.516865+0000","last_became_peered":"2023-10-10T20:36:21.516865+0000","last_unstale":"2023-10-10T20:36:21.517145+0000","last_undegraded":"2023-10-10T20:36:21.517145+0000","last_fullsized":"2023-10-10T20:36:21.517145+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:20:36.461834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.502130+0000","last_change":"2023-10-10T20:36:20.501499+0000","last_active":"2023-10-10T20:36:21.502130+0000","last_peered":"2023-10-10T20:36:21.502130+0000","last_clean":"2023-10-10T20:36:21.502130+0000","last_became_active":"2023-10-10T20:36:20.501235+0000","last_became_peered":"2023-10-10T20:36:20.501235+0000","last_unstale":"2023-10-10T20:36:21.502130+0000","last_undegraded":"2023-10-10T20:36:21.502130+0000","last_fullsized":"2023-10-10T20:36:21.502130+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:08.864753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.518260+0000","last_change":"2023-10-10T20:36:21.518260+0000","last_active":"2023-10-10T20:36:21.518260+0000","last_peered":"2023-10-10T20:36:21.518260+0000","last_clean":"2023-10-10T20:36:21.518260+0000","last_became_active":"2023-10-10T20:36:21.513050+0000","last_became_peered":"2023-10-10T20:36:21.513050+0000","last_unstale":"2023-10-10T20:36:21.518260+0000","last_undegraded":"2023-10-10T20:36:21.518260+0000","last_fullsized":"2023-10-10T20:36:21.518260+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:51:30.117443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.504194+0000","last_change":"2023-10-10T20:36:20.502598+0000","last_active":"2023-10-10T20:36:20.504194+0000","last_peered":"2023-10-10T20:36:20.504194+0000","last_clean":"2023-10-10T20:36:20.504194+0000","last_became_active":"2023-10-10T20:36:20.502263+0000","last_became_peered":"2023-10-10T20:36:20.502263+0000","last_unstale":"2023-10-10T20:36:20.504194+0000","last_undegraded":"2023-10-10T20:36:20.504194+0000","last_fullsized":"2023-10-10T20:36:20.504194+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:56:05.686047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.518667+0000","last_change":"2023-10-10T20:36:21.518667+0000","last_active":"2023-10-10T20:36:21.518667+0000","last_peered":"2023-10-10T20:36:21.518667+0000","last_clean":"2023-10-10T20:36:21.518667+0000","last_became_active":"2023-10-10T20:36:21.518488+0000","last_became_peered":"2023-10-10T20:36:21.518488+0000","last_unstale":"2023-10-10T20:36:21.518667+0000","last_undegraded":"2023-10-10T20:36:21.518667+0000","last_fullsized":"2023-10-10T20:36:21.518667+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:52:20.259891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.511953+0000","last_change":"2023-10-10T20:36:20.504838+0000","last_active":"2023-10-10T20:36:21.511953+0000","last_peered":"2023-10-10T20:36:21.511953+0000","last_clean":"2023-10-10T20:36:21.511953+0000","last_became_active":"2023-10-10T20:36:20.504509+0000","last_became_peered":"2023-10-10T20:36:20.504509+0000","last_unstale":"2023-10-10T20:36:21.511953+0000","last_undegraded":"2023-10-10T20:36:21.511953+0000","last_fullsized":"2023-10-10T20:36:21.511953+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:45:38.046706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.517502+0000","last_change":"2023-10-10T20:36:21.517502+0000","last_active":"2023-10-10T20:36:21.517502+0000","last_peered":"2023-10-10T20:36:21.517502+0000","last_clean":"2023-10-10T20:36:21.517502+0000","last_became_active":"2023-10-10T20:36:21.517203+0000","last_became_peered":"2023-10-10T20:36:21.517203+0000","last_unstale":"2023-10-10T20:36:21.517502+0000","last_undegraded":"2023-10-10T20:36:21.517502+0000","last_fullsized":"2023-10-10T20:36:21.517502+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:10:15.579634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.508524+0000","last_change":"2023-10-10T20:36:20.504694+0000","last_active":"2023-10-10T20:36:20.508524+0000","last_peered":"2023-10-10T20:36:20.508524+0000","last_clean":"2023-10-10T20:36:20.508524+0000","last_became_active":"2023-10-10T20:36:20.504440+0000","last_became_peered":"2023-10-10T20:36:20.504440+0000","last_unstale":"2023-10-10T20:36:20.508524+0000","last_undegraded":"2023-10-10T20:36:20.508524+0000","last_fullsized":"2023-10-10T20:36:20.508524+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:11:35.393353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.509824+0000","last_change":"2023-10-10T20:36:21.509824+0000","last_active":"2023-10-10T20:36:21.509824+0000","last_peered":"2023-10-10T20:36:21.509824+0000","last_clean":"2023-10-10T20:36:21.509824+0000","last_became_active":"2023-10-10T20:36:21.509556+0000","last_became_peered":"2023-10-10T20:36:21.509556+0000","last_unstale":"2023-10-10T20:36:21.509824+0000","last_undegraded":"2023-10-10T20:36:21.509824+0000","last_fullsized":"2023-10-10T20:36:21.509824+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:27.371302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.532437+0000","last_change":"2023-10-10T20:36:20.509385+0000","last_active":"2023-10-10T20:36:21.532437+0000","last_peered":"2023-10-10T20:36:21.532437+0000","last_clean":"2023-10-10T20:36:21.532437+0000","last_became_active":"2023-10-10T20:36:20.509054+0000","last_became_peered":"2023-10-10T20:36:20.509054+0000","last_unstale":"2023-10-10T20:36:21.532437+0000","last_undegraded":"2023-10-10T20:36:21.532437+0000","last_fullsized":"2023-10-10T20:36:21.532437+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:38:05.142702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.519328+0000","last_change":"2023-10-10T20:36:21.514636+0000","last_active":"2023-10-10T20:36:21.519328+0000","last_peered":"2023-10-10T20:36:21.519328+0000","last_clean":"2023-10-10T20:36:21.519328+0000","last_became_active":"2023-10-10T20:36:21.514299+0000","last_became_peered":"2023-10-10T20:36:21.514299+0000","last_unstale":"2023-10-10T20:36:21.519328+0000","last_undegraded":"2023-10-10T20:36:21.519328+0000","last_fullsized":"2023-10-10T20:36:21.519328+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:28:02.757484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.511506+0000","last_change":"2023-10-10T20:36:20.503318+0000","last_active":"2023-10-10T20:36:21.511506+0000","last_peered":"2023-10-10T20:36:21.511506+0000","last_clean":"2023-10-10T20:36:21.511506+0000","last_became_active":"2023-10-10T20:36:20.503066+0000","last_became_peered":"2023-10-10T20:36:20.503066+0000","last_unstale":"2023-10-10T20:36:21.511506+0000","last_undegraded":"2023-10-10T20:36:21.511506+0000","last_fullsized":"2023-10-10T20:36:21.511506+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:08.864753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.527677+0000","last_change":"2023-10-10T20:36:21.517948+0000","last_active":"2023-10-10T20:36:21.527677+0000","last_peered":"2023-10-10T20:36:21.527677+0000","last_clean":"2023-10-10T20:36:21.527677+0000","last_became_active":"2023-10-10T20:36:21.516384+0000","last_became_peered":"2023-10-10T20:36:21.516384+0000","last_unstale":"2023-10-10T20:36:21.527677+0000","last_undegraded":"2023-10-10T20:36:21.527677+0000","last_fullsized":"2023-10-10T20:36:21.527677+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:54:47.058919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.506269+0000","last_change":"2023-10-10T20:36:20.502283+0000","last_active":"2023-10-10T20:36:20.506269+0000","last_peered":"2023-10-10T20:36:20.506269+0000","last_clean":"2023-10-10T20:36:20.506269+0000","last_became_active":"2023-10-10T20:36:20.502048+0000","last_became_peered":"2023-10-10T20:36:20.502048+0000","last_unstale":"2023-10-10T20:36:20.506269+0000","last_undegraded":"2023-10-10T20:36:20.506269+0000","last_fullsized":"2023-10-10T20:36:20.506269+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:19:38.829714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.527729+0000","last_change":"2023-10-10T20:36:21.517626+0000","last_active":"2023-10-10T20:36:21.527729+0000","last_peered":"2023-10-10T20:36:21.527729+0000","last_clean":"2023-10-10T20:36:21.527729+0000","last_became_active":"2023-10-10T20:36:21.517353+0000","last_became_peered":"2023-10-10T20:36:21.517353+0000","last_unstale":"2023-10-10T20:36:21.527729+0000","last_undegraded":"2023-10-10T20:36:21.527729+0000","last_fullsized":"2023-10-10T20:36:21.527729+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:09:10.544623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.508567+0000","last_change":"2023-10-10T20:36:20.505099+0000","last_active":"2023-10-10T20:36:20.508567+0000","last_peered":"2023-10-10T20:36:20.508567+0000","last_clean":"2023-10-10T20:36:20.508567+0000","last_became_active":"2023-10-10T20:36:20.504778+0000","last_became_peered":"2023-10-10T20:36:20.504778+0000","last_unstale":"2023-10-10T20:36:20.508567+0000","last_undegraded":"2023-10-10T20:36:20.508567+0000","last_fullsized":"2023-10-10T20:36:20.508567+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:39:13.485398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.513102+0000","last_change":"2023-10-10T20:36:21.513102+0000","last_active":"2023-10-10T20:36:21.513102+0000","last_peered":"2023-10-10T20:36:21.513102+0000","last_clean":"2023-10-10T20:36:21.513102+0000","last_became_active":"2023-10-10T20:36:21.512779+0000","last_became_peered":"2023-10-10T20:36:21.512779+0000","last_unstale":"2023-10-10T20:36:21.513102+0000","last_undegraded":"2023-10-10T20:36:21.513102+0000","last_fullsized":"2023-10-10T20:36:21.513102+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:57:56.698645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.497171+0000","last_change":"2023-10-10T20:36:20.494842+0000","last_active":"2023-10-10T20:36:20.497171+0000","last_peered":"2023-10-10T20:36:20.497171+0000","last_clean":"2023-10-10T20:36:20.497171+0000","last_became_active":"2023-10-10T20:36:20.494633+0000","last_became_peered":"2023-10-10T20:36:20.494633+0000","last_unstale":"2023-10-10T20:36:20.497171+0000","last_undegraded":"2023-10-10T20:36:20.497171+0000","last_fullsized":"2023-10-10T20:36:20.497171+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:54:20.021115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.523032+0000","last_change":"2023-10-10T20:36:21.523032+0000","last_active":"2023-10-10T20:36:21.523032+0000","last_peered":"2023-10-10T20:36:21.523032+0000","last_clean":"2023-10-10T20:36:21.523032+0000","last_became_active":"2023-10-10T20:36:21.522782+0000","last_became_peered":"2023-10-10T20:36:21.522782+0000","last_unstale":"2023-10-10T20:36:21.523032+0000","last_undegraded":"2023-10-10T20:36:21.523032+0000","last_fullsized":"2023-10-10T20:36:21.523032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:27:20.416644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.499878+0000","last_change":"2023-10-10T20:36:20.496726+0000","last_active":"2023-10-10T20:36:20.499878+0000","last_peered":"2023-10-10T20:36:20.499878+0000","last_clean":"2023-10-10T20:36:20.499878+0000","last_became_active":"2023-10-10T20:36:20.496480+0000","last_became_peered":"2023-10-10T20:36:20.496480+0000","last_unstale":"2023-10-10T20:36:20.499878+0000","last_undegraded":"2023-10-10T20:36:20.499878+0000","last_fullsized":"2023-10-10T20:36:20.499878+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:22:59.981260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.513112+0000","last_change":"2023-10-10T20:36:21.513112+0000","last_active":"2023-10-10T20:36:21.513112+0000","last_peered":"2023-10-10T20:36:21.513112+0000","last_clean":"2023-10-10T20:36:21.513112+0000","last_became_active":"2023-10-10T20:36:21.512752+0000","last_became_peered":"2023-10-10T20:36:21.512752+0000","last_unstale":"2023-10-10T20:36:21.513112+0000","last_undegraded":"2023-10-10T20:36:21.513112+0000","last_fullsized":"2023-10-10T20:36:21.513112+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:37.824587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.508221+0000","last_change":"2023-10-10T20:36:20.499139+0000","last_active":"2023-10-10T20:36:21.508221+0000","last_peered":"2023-10-10T20:36:21.508221+0000","last_clean":"2023-10-10T20:36:21.508221+0000","last_became_active":"2023-10-10T20:36:20.498860+0000","last_became_peered":"2023-10-10T20:36:20.498860+0000","last_unstale":"2023-10-10T20:36:21.508221+0000","last_undegraded":"2023-10-10T20:36:21.508221+0000","last_fullsized":"2023-10-10T20:36:21.508221+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:22:59.981260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.521093+0000","last_change":"2023-10-10T20:36:21.521093+0000","last_active":"2023-10-10T20:36:21.521093+0000","last_peered":"2023-10-10T20:36:21.521093+0000","last_clean":"2023-10-10T20:36:21.521093+0000","last_became_active":"2023-10-10T20:36:21.520739+0000","last_became_peered":"2023-10-10T20:36:21.520739+0000","last_unstale":"2023-10-10T20:36:21.521093+0000","last_undegraded":"2023-10-10T20:36:21.521093+0000","last_fullsized":"2023-10-10T20:36:21.521093+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:11:36.397020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.504869+0000","last_change":"2023-10-10T20:36:20.502795+0000","last_active":"2023-10-10T20:36:20.504869+0000","last_peered":"2023-10-10T20:36:20.504869+0000","last_clean":"2023-10-10T20:36:20.504869+0000","last_became_active":"2023-10-10T20:36:20.502459+0000","last_became_peered":"2023-10-10T20:36:20.502459+0000","last_unstale":"2023-10-10T20:36:20.504869+0000","last_undegraded":"2023-10-10T20:36:20.504869+0000","last_fullsized":"2023-10-10T20:36:20.504869+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:50:26.158577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.515077+0000","last_change":"2023-10-10T20:36:21.515077+0000","last_active":"2023-10-10T20:36:21.515077+0000","last_peered":"2023-10-10T20:36:21.515077+0000","last_clean":"2023-10-10T20:36:21.515077+0000","last_became_active":"2023-10-10T20:36:21.514835+0000","last_became_peered":"2023-10-10T20:36:21.514835+0000","last_unstale":"2023-10-10T20:36:21.515077+0000","last_undegraded":"2023-10-10T20:36:21.515077+0000","last_fullsized":"2023-10-10T20:36:21.515077+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:46:55.099687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.530872+0000","last_change":"2023-10-10T20:36:20.500621+0000","last_active":"2023-10-10T20:36:21.530872+0000","last_peered":"2023-10-10T20:36:21.530872+0000","last_clean":"2023-10-10T20:36:21.530872+0000","last_became_active":"2023-10-10T20:36:20.500328+0000","last_became_peered":"2023-10-10T20:36:20.500328+0000","last_unstale":"2023-10-10T20:36:21.530872+0000","last_undegraded":"2023-10-10T20:36:21.530872+0000","last_fullsized":"2023-10-10T20:36:21.530872+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:30.267821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":205,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.501984+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:36:21.501984+0000","last_peered":"2023-10-10T20:36:21.501984+0000","last_clean":"2023-10-10T20:36:21.501984+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:36:21.501984+0000","last_undegraded":"2023-10-10T20:36:21.501984+0000","last_fullsized":"2023-10-10T20:36:21.501984+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298600,"kb_used_data":1312,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073240,"statfs":{"total":96636764160,"available":96330997760,"internally_reserved":0,"allocated":1343488,"data_stored":905080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":6.8630000000000004,"5min":6.8630000000000004,"15min":6.8630000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":4.1959999999999997,"5min":4.1959999999999997,"15min":4.1959999999999997},"last":0.33700000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":10.1,"5min":10.1,"15min":10.1},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":6.806,"5min":6.806,"15min":6.806},"last":0.437}]},{"osd":2,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":6.9340000000000002,"5min":6.9340000000000002,"15min":6.9340000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":6.9870000000000001,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.38}]},{"osd":3,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":6.6719999999999997,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":4.1609999999999996,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.39000000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.32000000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.44800000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55600000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298020,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073820,"statfs":{"total":96636764160,"available":96331591680,"internally_reserved":0,"allocated":749568,"data_stored":314216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":7.7590000000000003,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":7.8090000000000002,"5min":7.8090000000000002,"15min":7.8090000000000002},"last":0.46600000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":7.867,"5min":7.867,"15min":7.867},"last":0.37},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":9.7089999999999996,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.50900000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":4.7999999999999998,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":6.7629999999999999,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":0.55000000000000004}]},{"osd":3,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":4.8620000000000001,"5min":4.8620000000000001,"15min":4.8620000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":7.9009999999999998,"5min":7.9009999999999998,"15min":7.9009999999999998},"last":0.40799999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.44}]},{"osd":5,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.48999999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.44800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298032,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073808,"statfs":{"total":96636764160,"available":96331579392,"internally_reserved":0,"allocated":761856,"data_stored":316248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":5.0380000000000003,"5min":5.0380000000000003,"15min":5.0380000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.48599999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":5.0919999999999996,"5min":5.0919999999999996,"15min":5.0919999999999996},"last":0.40300000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.1159999999999997,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.45700000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":4.9850000000000003,"5min":4.9850000000000003,"15min":4.9850000000000003},"last":0.65600000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.67000000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":5.0659999999999998,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.70899999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0499999999999998,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.74399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298588,"kb_used_data":1300,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073252,"statfs":{"total":96636764160,"available":96331010048,"internally_reserved":0,"allocated":1331200,"data_stored":903531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":7.9249999999999998,"5min":7.9249999999999998,"15min":7.9249999999999998},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":6.6299999999999999,"5min":6.6299999999999999,"15min":6.6299999999999999},"last":0.54900000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":8.2509999999999994,"5min":8.2509999999999994,"15min":8.2509999999999994},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":6.7370000000000001,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.51700000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":7.7960000000000003,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.505},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":7.8929999999999998,"5min":7.8929999999999998,"15min":7.8929999999999998},"last":0.53200000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":6.6929999999999996,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":7.6920000000000002,"5min":7.6920000000000002,"15min":7.6920000000000002},"last":0.73199999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":7.7690000000000001,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.8659999999999997,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":0.79000000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":7.8360000000000003,"5min":7.8360000000000003,"15min":7.8360000000000003},"last":0.753},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":6.6660000000000004,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.65600000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":6.7160000000000002,"5min":6.7160000000000002,"15min":6.7160000000000002},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":7.7370000000000001,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.71099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298024,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073816,"statfs":{"total":96636764160,"available":96331587584,"internally_reserved":0,"allocated":753664,"data_stored":315660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.435},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.5}]},{"osd":1,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":11.967000000000001,"5min":11.967000000000001,"15min":11.967000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.1100000000000003,"5min":4.1100000000000003,"15min":4.1100000000000003},"last":0.46800000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":6.3040000000000003,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":11.935,"5min":11.935,"15min":11.935},"last":0.52500000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.77700000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.72299999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.747},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.71199999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":6.2649999999999997,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.80600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298020,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073820,"statfs":{"total":96636764160,"available":96331591680,"internally_reserved":0,"allocated":749568,"data_stored":314236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.317,"15min":0.317},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.436,"5min":0.436,"15min":0.436},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.317}]},{"osd":1,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.35099999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.307}]},{"osd":4,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.48699999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.437},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.51200000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.498}]},{"osd":7,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.41899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298028,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073812,"statfs":{"total":96636764160,"available":96331583488,"internally_reserved":0,"allocated":757760,"data_stored":320482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":7.6719999999999997,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.55500000000000005}]},{"osd":1,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":7.7160000000000002,"5min":7.7160000000000002,"15min":7.7160000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.442}]},{"osd":2,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":7.7839999999999998,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.59099999999999997}]},{"osd":3,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.49299999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.374},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.63300000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.623}]},{"osd":7,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298028,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073812,"statfs":{"total":96636764160,"available":96331583488,"internally_reserved":0,"allocated":757760,"data_stored":320502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.443}]},{"osd":1,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":5.2679999999999998,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":5.3109999999999999,"5min":5.3109999999999999,"15min":5.3109999999999999},"last":0.48599999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.2960000000000003,"5min":5.2960000000000003,"15min":5.2960000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.47199999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":5.3330000000000002,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.2850000000000001,"5min":5.2850000000000001,"15min":5.2850000000000001},"last":0.40699999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.501}]},{"osd":6,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.53200000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.60399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:36:25.561 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:36:25.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.707+0000 7fd1033b5700 1 Processor -- start 2023-10-10T20:36:25.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.707+0000 7fd1033b5700 1 -- start start 2023-10-10T20:36:25.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.708+0000 7fd1033b5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd0fc159890 0x7fd0fc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:25.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.708+0000 7fd1033b5700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd0fc159c60 0x7fd0fc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:25.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.708+0000 7fd1033b5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0fc15a160 con 0x7fd0fc15db10 2023-10-10T20:36:25.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.708+0000 7fd1033b5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0fc15df50 con 0x7fd0fc159890 2023-10-10T20:36:25.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.708+0000 7fd1033b5700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0fc0d13a0 con 0x7fd0fc159c60 2023-10-10T20:36:25.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.708+0000 7fd101151700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd0fc159890 0x7fd0fc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:25.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.708+0000 7fd101952700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd0fc15db10 0x7fd0fc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56640/0 (socket says 172.21.15.89:56640) 2023-10-10T20:36:25.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.708+0000 7fd101952700 1 -- 172.21.15.89:0/1639783425 learned_addr learned my addr 172.21.15.89:0/1639783425 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:25.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.708+0000 7fd100950700 1 --2- 172.21.15.89:0/1639783425 >> v2:172.21.15.111:3300/0 conn(0x7fd0fc159c60 0x7fd0fc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:25.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.709+0000 7fd101151700 1 -- 172.21.15.89:0/1639783425 >> v2:172.21.15.111:3300/0 conn(0x7fd0fc159c60 msgr2=0x7fd0fc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:25.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.709+0000 7fd101151700 1 --2- 172.21.15.89:0/1639783425 >> v2:172.21.15.111:3300/0 conn(0x7fd0fc159c60 0x7fd0fc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.709+0000 7fd101151700 1 -- 172.21.15.89:0/1639783425 >> v1:172.21.15.89:6789/0 conn(0x7fd0fc15db10 legacy=0x7fd0fc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:25.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.709+0000 7fd101151700 1 -- 172.21.15.89:0/1639783425 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0fc15e140 con 0x7fd0fc159890 2023-10-10T20:36:25.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.709+0000 7fd0f3fff700 1 -- 172.21.15.89:0/1639783425 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871560472 0 0) 0x7fd0fc15a160 con 0x7fd0fc15db10 2023-10-10T20:36:25.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.709+0000 7fd100950700 1 --2- 172.21.15.89:0/1639783425 >> v2:172.21.15.111:3300/0 conn(0x7fd0fc159c60 0x7fd0fc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:25.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.709+0000 7fd101151700 1 --2- 172.21.15.89:0/1639783425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0fc159890 0x7fd0fc15abd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0b67f0 tx=0x7fd0f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49594dc1e69c11ea server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:25.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.709+0000 7fd0f3fff700 1 -- 172.21.15.89:0/1639783425 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f800b6b0 con 0x7fd0fc159890 2023-10-10T20:36:25.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.709+0000 7fd0f3fff700 1 -- 172.21.15.89:0/1639783425 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f800b810 con 0x7fd0fc159890 2023-10-10T20:36:25.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.709+0000 7fd0f3fff700 1 -- 172.21.15.89:0/1639783425 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f800bbb0 con 0x7fd0fc159890 2023-10-10T20:36:25.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.709+0000 7fd1033b5700 1 -- 172.21.15.89:0/1639783425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0fc159890 msgr2=0x7fd0fc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:25.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.709+0000 7fd1033b5700 1 --2- 172.21.15.89:0/1639783425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0fc159890 0x7fd0fc15abd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0b67f0 tx=0x7fd0f80091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.709+0000 7fd1033b5700 1 -- 172.21.15.89:0/1639783425 shutdown_connections 2023-10-10T20:36:25.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.709+0000 7fd1033b5700 1 --2- 172.21.15.89:0/1639783425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0fc159890 0x7fd0fc15abd0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.709+0000 7fd1033b5700 1 --2- 172.21.15.89:0/1639783425 >> v2:172.21.15.111:3300/0 conn(0x7fd0fc159c60 0x7fd0fc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.709+0000 7fd1033b5700 1 -- 172.21.15.89:0/1639783425 >> 172.21.15.89:0/1639783425 conn(0x7fd0fc068e20 msgr2=0x7fd0fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:25.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.709+0000 7fd1033b5700 1 -- 172.21.15.89:0/1639783425 shutdown_connections 2023-10-10T20:36:25.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.709+0000 7fd1033b5700 1 -- 172.21.15.89:0/1639783425 wait complete. 2023-10-10T20:36:25.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.711+0000 7fd1033b5700 1 Processor -- start 2023-10-10T20:36:25.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.711+0000 7fd1033b5700 1 -- start start 2023-10-10T20:36:25.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.711+0000 7fd1033b5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd0fc159890 0x7fd0fc14b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:25.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.711+0000 7fd1033b5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0fc159c60 0x7fd0fc0b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:25.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.711+0000 7fd1033b5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0fc15db10 0x7fd0fc0b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:25.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.711+0000 7fd1033b5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc14bac0 con 0x7fd0fc15db10 2023-10-10T20:36:25.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.711+0000 7fd1033b5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc0b4ad0 con 0x7fd0fc159c60 2023-10-10T20:36:25.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.711+0000 7fd1033b5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc16d630 con 0x7fd0fc159890 2023-10-10T20:36:25.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.712+0000 7fd101952700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0fc15db10 0x7fd0fc0b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:25.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.712+0000 7fd101952700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0fc15db10 0x7fd0fc0b1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46854/0 (socket says 172.21.15.89:46854) 2023-10-10T20:36:25.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.712+0000 7fd101952700 1 -- 172.21.15.89:0/3577685135 learned_addr learned my addr 172.21.15.89:0/3577685135 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:25.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.712+0000 7fd100950700 1 --2- 172.21.15.89:0/3577685135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0fc159c60 0x7fd0fc0b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:25.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.712+0000 7fd101952700 1 -- 172.21.15.89:0/3577685135 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd0fc159890 msgr2=0x7fd0fc14b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:25.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.712+0000 7fd101151700 1 --2- 172.21.15.89:0/3577685135 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd0fc159890 0x7fd0fc14b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:25.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.712+0000 7fd101952700 1 --2- 172.21.15.89:0/3577685135 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd0fc159890 0x7fd0fc14b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.712+0000 7fd101952700 1 -- 172.21.15.89:0/3577685135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0fc159c60 msgr2=0x7fd0fc0b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:25.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.712+0000 7fd101952700 1 --2- 172.21.15.89:0/3577685135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0fc159c60 0x7fd0fc0b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.712+0000 7fd101952700 1 -- 172.21.15.89:0/3577685135 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f8009420 con 0x7fd0fc15db10 2023-10-10T20:36:25.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.712+0000 7fd100950700 1 --2- 172.21.15.89:0/3577685135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0fc159c60 0x7fd0fc0b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:25.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.712+0000 7fd101151700 1 --2- 172.21.15.89:0/3577685135 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd0fc159890 0x7fd0fc14b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:25.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.712+0000 7fd101952700 1 --2- 172.21.15.89:0/3577685135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0fc15db10 0x7fd0fc0b1ce0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400cbf0 tx=0x7fd0f40019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:25.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.712+0000 7fd0f17fa700 1 -- 172.21.15.89:0/3577685135 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f400a9c0 con 0x7fd0fc15db10 2023-10-10T20:36:25.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.712+0000 7fd1033b5700 1 -- 172.21.15.89:0/3577685135 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0fc16d910 con 0x7fd0fc15db10 2023-10-10T20:36:25.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.712+0000 7fd0f17fa700 1 -- 172.21.15.89:0/3577685135 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f400ad30 con 0x7fd0fc15db10 2023-10-10T20:36:25.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.712+0000 7fd1033b5700 1 -- 172.21.15.89:0/3577685135 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0fc16ded0 con 0x7fd0fc15db10 2023-10-10T20:36:25.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.712+0000 7fd0f17fa700 1 -- 172.21.15.89:0/3577685135 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f400aea0 con 0x7fd0fc15db10 2023-10-10T20:36:25.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.713+0000 7fd1033b5700 1 -- 172.21.15.89:0/3577685135 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0fc0b4c90 con 0x7fd0fc15db10 2023-10-10T20:36:25.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.713+0000 7fd0f17fa700 1 -- 172.21.15.89:0/3577685135 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fd0f4007300 con 0x7fd0fc15db10 2023-10-10T20:36:25.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.714+0000 7fd0f17fa700 1 --2- 172.21.15.89:0/3577685135 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd0e0066fc0 0x7fd0e0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:25.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.714+0000 7fd0f17fa700 1 -- 172.21.15.89:0/3577685135 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd0f4084970 con 0x7fd0fc15db10 2023-10-10T20:36:25.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.714+0000 7fd101151700 1 --2- 172.21.15.89:0/3577685135 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd0e0066fc0 0x7fd0e0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:25.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.714+0000 7fd101151700 1 --2- 172.21.15.89:0/3577685135 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd0e0066fc0 0x7fd0e0069470 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8003080 tx=0x7fd0f8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:25.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.718+0000 7fd0f17fa700 1 -- 172.21.15.89:0/3577685135 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0f4051bf0 con 0x7fd0fc15db10 2023-10-10T20:36:25.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.895+0000 7fd1033b5700 1 -- 172.21.15.89:0/3577685135 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd0fc04c3c0 con 0x7fd0e0066fc0 2023-10-10T20:36:25.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.903+0000 7fd0f17fa700 1 -- 172.21.15.89:0/3577685135 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289503 (secure 0 0 0) 0x7fd0fc04c3c0 con 0x7fd0e0066fc0 2023-10-10T20:36:25.909 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:25.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.906+0000 7fd1033b5700 1 -- 172.21.15.89:0/3577685135 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd0e0066fc0 msgr2=0x7fd0e0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:25.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.906+0000 7fd1033b5700 1 --2- 172.21.15.89:0/3577685135 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd0e0066fc0 0x7fd0e0069470 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8003080 tx=0x7fd0f8017000 comp rx=0 tx=0).stop 2023-10-10T20:36:25.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.906+0000 7fd1033b5700 1 -- 172.21.15.89:0/3577685135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0fc15db10 msgr2=0x7fd0fc0b1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:25.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.906+0000 7fd1033b5700 1 --2- 172.21.15.89:0/3577685135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0fc15db10 0x7fd0fc0b1ce0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400cbf0 tx=0x7fd0f40019b0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.906+0000 7fd1033b5700 1 -- 172.21.15.89:0/3577685135 shutdown_connections 2023-10-10T20:36:25.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.906+0000 7fd1033b5700 1 --2- 172.21.15.89:0/3577685135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0fc15db10 0x7fd0fc0b1ce0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.906+0000 7fd1033b5700 1 --2- 172.21.15.89:0/3577685135 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd0fc159890 0x7fd0fc14b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.906+0000 7fd1033b5700 1 --2- 172.21.15.89:0/3577685135 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd0e0066fc0 0x7fd0e0069470 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.906+0000 7fd1033b5700 1 --2- 172.21.15.89:0/3577685135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0fc159c60 0x7fd0fc0b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:25.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.906+0000 7fd1033b5700 1 -- 172.21.15.89:0/3577685135 >> 172.21.15.89:0/3577685135 conn(0x7fd0fc068e20 msgr2=0x7fd0fc14d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:25.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.906+0000 7fd1033b5700 1 -- 172.21.15.89:0/3577685135 shutdown_connections 2023-10-10T20:36:25.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:25.906+0000 7fd1033b5700 1 -- 172.21.15.89:0/3577685135 wait complete. 2023-10-10T20:36:25.914 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:36:25.920 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2023-10-10T20:36:24.672494+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385340,"kb_used_data":7036,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752589380,"statfs":{"total":773094113280,"available":770651525120,"internally_reserved":0,"allocated":7204864,"data_stored":3709955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.190750"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.521217+0000","last_change":"2023-10-10T20:36:21.511544+0000","last_active":"2023-10-10T20:36:21.521217+0000","last_peered":"2023-10-10T20:36:21.521217+0000","last_clean":"2023-10-10T20:36:21.521217+0000","last_became_active":"2023-10-10T20:36:21.511287+0000","last_became_peered":"2023-10-10T20:36:21.511287+0000","last_unstale":"2023-10-10T20:36:21.521217+0000","last_undegraded":"2023-10-10T20:36:21.521217+0000","last_fullsized":"2023-10-10T20:36:21.521217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:56:38.109485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.519727+0000","last_change":"2023-10-10T20:36:21.519727+0000","last_active":"2023-10-10T20:36:21.519727+0000","last_peered":"2023-10-10T20:36:21.519727+0000","last_clean":"2023-10-10T20:36:21.519727+0000","last_became_active":"2023-10-10T20:36:21.519455+0000","last_became_peered":"2023-10-10T20:36:21.519455+0000","last_unstale":"2023-10-10T20:36:21.519727+0000","last_undegraded":"2023-10-10T20:36:21.519727+0000","last_fullsized":"2023-10-10T20:36:21.519727+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:01:01.948430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.515655+0000","last_change":"2023-10-10T20:36:21.515655+0000","last_active":"2023-10-10T20:36:21.515655+0000","last_peered":"2023-10-10T20:36:21.515655+0000","last_clean":"2023-10-10T20:36:21.515655+0000","last_became_active":"2023-10-10T20:36:21.515419+0000","last_became_peered":"2023-10-10T20:36:21.515419+0000","last_unstale":"2023-10-10T20:36:21.515655+0000","last_undegraded":"2023-10-10T20:36:21.515655+0000","last_fullsized":"2023-10-10T20:36:21.515655+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:21:52.585716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.518305+0000","last_change":"2023-10-10T20:36:21.518305+0000","last_active":"2023-10-10T20:36:21.518305+0000","last_peered":"2023-10-10T20:36:21.518305+0000","last_clean":"2023-10-10T20:36:21.518305+0000","last_became_active":"2023-10-10T20:36:21.518057+0000","last_became_peered":"2023-10-10T20:36:21.518057+0000","last_unstale":"2023-10-10T20:36:21.518305+0000","last_undegraded":"2023-10-10T20:36:21.518305+0000","last_fullsized":"2023-10-10T20:36:21.518305+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:57.315341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.504118+0000","last_change":"2023-10-10T20:36:21.504118+0000","last_active":"2023-10-10T20:36:21.504118+0000","last_peered":"2023-10-10T20:36:21.504118+0000","last_clean":"2023-10-10T20:36:21.504118+0000","last_became_active":"2023-10-10T20:36:21.503804+0000","last_became_peered":"2023-10-10T20:36:21.503804+0000","last_unstale":"2023-10-10T20:36:21.504118+0000","last_undegraded":"2023-10-10T20:36:21.504118+0000","last_fullsized":"2023-10-10T20:36:21.504118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:34:04.208699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.515472+0000","last_change":"2023-10-10T20:36:21.515472+0000","last_active":"2023-10-10T20:36:21.515472+0000","last_peered":"2023-10-10T20:36:21.515472+0000","last_clean":"2023-10-10T20:36:21.515472+0000","last_became_active":"2023-10-10T20:36:21.515186+0000","last_became_peered":"2023-10-10T20:36:21.515186+0000","last_unstale":"2023-10-10T20:36:21.515472+0000","last_undegraded":"2023-10-10T20:36:21.515472+0000","last_fullsized":"2023-10-10T20:36:21.515472+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:07:19.870599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.526249+0000","last_change":"2023-10-10T20:36:21.520963+0000","last_active":"2023-10-10T20:36:21.526249+0000","last_peered":"2023-10-10T20:36:21.526249+0000","last_clean":"2023-10-10T20:36:21.526249+0000","last_became_active":"2023-10-10T20:36:21.520699+0000","last_became_peered":"2023-10-10T20:36:21.520699+0000","last_unstale":"2023-10-10T20:36:21.526249+0000","last_undegraded":"2023-10-10T20:36:21.526249+0000","last_fullsized":"2023-10-10T20:36:21.526249+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:57:03.984128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.516964+0000","last_change":"2023-10-10T20:36:21.516964+0000","last_active":"2023-10-10T20:36:21.516964+0000","last_peered":"2023-10-10T20:36:21.516964+0000","last_clean":"2023-10-10T20:36:21.516964+0000","last_became_active":"2023-10-10T20:36:21.516644+0000","last_became_peered":"2023-10-10T20:36:21.516644+0000","last_unstale":"2023-10-10T20:36:21.516964+0000","last_undegraded":"2023-10-10T20:36:21.516964+0000","last_fullsized":"2023-10-10T20:36:21.516964+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:14.041794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.515833+0000","last_change":"2023-10-10T20:36:21.515833+0000","last_active":"2023-10-10T20:36:21.515833+0000","last_peered":"2023-10-10T20:36:21.515833+0000","last_clean":"2023-10-10T20:36:21.515833+0000","last_became_active":"2023-10-10T20:36:21.515566+0000","last_became_peered":"2023-10-10T20:36:21.515566+0000","last_unstale":"2023-10-10T20:36:21.515833+0000","last_undegraded":"2023-10-10T20:36:21.515833+0000","last_fullsized":"2023-10-10T20:36:21.515833+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:28.201245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.532818+0000","last_change":"2023-10-10T20:36:21.525476+0000","last_active":"2023-10-10T20:36:21.532818+0000","last_peered":"2023-10-10T20:36:21.532818+0000","last_clean":"2023-10-10T20:36:21.532818+0000","last_became_active":"2023-10-10T20:36:21.525200+0000","last_became_peered":"2023-10-10T20:36:21.525200+0000","last_unstale":"2023-10-10T20:36:21.532818+0000","last_undegraded":"2023-10-10T20:36:21.532818+0000","last_fullsized":"2023-10-10T20:36:21.532818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:45:39.050374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.521912+0000","last_change":"2023-10-10T20:36:21.513019+0000","last_active":"2023-10-10T20:36:21.521912+0000","last_peered":"2023-10-10T20:36:21.521912+0000","last_clean":"2023-10-10T20:36:21.521912+0000","last_became_active":"2023-10-10T20:36:21.512756+0000","last_became_peered":"2023-10-10T20:36:21.512756+0000","last_unstale":"2023-10-10T20:36:21.521912+0000","last_undegraded":"2023-10-10T20:36:21.521912+0000","last_fullsized":"2023-10-10T20:36:21.521912+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:36:10.486811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.510556+0000","last_change":"2023-10-10T20:36:21.510556+0000","last_active":"2023-10-10T20:36:21.510556+0000","last_peered":"2023-10-10T20:36:21.510556+0000","last_clean":"2023-10-10T20:36:21.510556+0000","last_became_active":"2023-10-10T20:36:21.510245+0000","last_became_peered":"2023-10-10T20:36:21.510245+0000","last_unstale":"2023-10-10T20:36:21.510556+0000","last_undegraded":"2023-10-10T20:36:21.510556+0000","last_fullsized":"2023-10-10T20:36:21.510556+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:27:20.416644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.515701+0000","last_change":"2023-10-10T20:36:21.515701+0000","last_active":"2023-10-10T20:36:21.515701+0000","last_peered":"2023-10-10T20:36:21.515701+0000","last_clean":"2023-10-10T20:36:21.515701+0000","last_became_active":"2023-10-10T20:36:21.515442+0000","last_became_peered":"2023-10-10T20:36:21.515442+0000","last_unstale":"2023-10-10T20:36:21.515701+0000","last_undegraded":"2023-10-10T20:36:21.515701+0000","last_fullsized":"2023-10-10T20:36:21.515701+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:07.707339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.516371+0000","last_change":"2023-10-10T20:36:21.516371+0000","last_active":"2023-10-10T20:36:21.516371+0000","last_peered":"2023-10-10T20:36:21.516371+0000","last_clean":"2023-10-10T20:36:21.516371+0000","last_became_active":"2023-10-10T20:36:21.516074+0000","last_became_peered":"2023-10-10T20:36:21.516074+0000","last_unstale":"2023-10-10T20:36:21.516371+0000","last_undegraded":"2023-10-10T20:36:21.516371+0000","last_fullsized":"2023-10-10T20:36:21.516371+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:20.928729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.507633+0000","last_change":"2023-10-10T20:36:21.507633+0000","last_active":"2023-10-10T20:36:21.507633+0000","last_peered":"2023-10-10T20:36:21.507633+0000","last_clean":"2023-10-10T20:36:21.507633+0000","last_became_active":"2023-10-10T20:36:21.507359+0000","last_became_peered":"2023-10-10T20:36:21.507359+0000","last_unstale":"2023-10-10T20:36:21.507633+0000","last_undegraded":"2023-10-10T20:36:21.507633+0000","last_fullsized":"2023-10-10T20:36:21.507633+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:31.961936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.531588+0000","last_change":"2023-10-10T20:36:21.522891+0000","last_active":"2023-10-10T20:36:21.531588+0000","last_peered":"2023-10-10T20:36:21.531588+0000","last_clean":"2023-10-10T20:36:21.531588+0000","last_became_active":"2023-10-10T20:36:21.522646+0000","last_became_peered":"2023-10-10T20:36:21.522646+0000","last_unstale":"2023-10-10T20:36:21.531588+0000","last_undegraded":"2023-10-10T20:36:21.531588+0000","last_fullsized":"2023-10-10T20:36:21.531588+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:39:14.489065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.516141+0000","last_change":"2023-10-10T20:36:21.516141+0000","last_active":"2023-10-10T20:36:21.516141+0000","last_peered":"2023-10-10T20:36:21.516141+0000","last_clean":"2023-10-10T20:36:21.516141+0000","last_became_active":"2023-10-10T20:36:21.515786+0000","last_became_peered":"2023-10-10T20:36:21.515786+0000","last_unstale":"2023-10-10T20:36:21.516141+0000","last_undegraded":"2023-10-10T20:36:21.516141+0000","last_fullsized":"2023-10-10T20:36:21.516141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:14:47.981506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.517645+0000","last_change":"2023-10-10T20:36:21.517645+0000","last_active":"2023-10-10T20:36:21.517645+0000","last_peered":"2023-10-10T20:36:21.517645+0000","last_clean":"2023-10-10T20:36:21.517645+0000","last_became_active":"2023-10-10T20:36:21.517409+0000","last_became_peered":"2023-10-10T20:36:21.517409+0000","last_unstale":"2023-10-10T20:36:21.517645+0000","last_undegraded":"2023-10-10T20:36:21.517645+0000","last_fullsized":"2023-10-10T20:36:21.517645+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:14.041794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.517258+0000","last_change":"2023-10-10T20:36:21.517258+0000","last_active":"2023-10-10T20:36:21.517258+0000","last_peered":"2023-10-10T20:36:21.517258+0000","last_clean":"2023-10-10T20:36:21.517258+0000","last_became_active":"2023-10-10T20:36:21.516848+0000","last_became_peered":"2023-10-10T20:36:21.516848+0000","last_unstale":"2023-10-10T20:36:21.517258+0000","last_undegraded":"2023-10-10T20:36:21.517258+0000","last_fullsized":"2023-10-10T20:36:21.517258+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:37.824587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.515143+0000","last_change":"2023-10-10T20:36:21.515143+0000","last_active":"2023-10-10T20:36:21.515143+0000","last_peered":"2023-10-10T20:36:21.515143+0000","last_clean":"2023-10-10T20:36:21.515143+0000","last_became_active":"2023-10-10T20:36:21.514886+0000","last_became_peered":"2023-10-10T20:36:21.514886+0000","last_unstale":"2023-10-10T20:36:21.515143+0000","last_undegraded":"2023-10-10T20:36:21.515143+0000","last_fullsized":"2023-10-10T20:36:21.515143+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:28:02.757484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.524465+0000","last_change":"2023-10-10T20:36:21.521017+0000","last_active":"2023-10-10T20:36:21.524465+0000","last_peered":"2023-10-10T20:36:21.524465+0000","last_clean":"2023-10-10T20:36:21.524465+0000","last_became_active":"2023-10-10T20:36:21.520761+0000","last_became_peered":"2023-10-10T20:36:21.520761+0000","last_unstale":"2023-10-10T20:36:21.524465+0000","last_undegraded":"2023-10-10T20:36:21.524465+0000","last_fullsized":"2023-10-10T20:36:21.524465+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:07.707339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.524481+0000","last_change":"2023-10-10T20:36:21.522024+0000","last_active":"2023-10-10T20:36:21.524481+0000","last_peered":"2023-10-10T20:36:21.524481+0000","last_clean":"2023-10-10T20:36:21.524481+0000","last_became_active":"2023-10-10T20:36:21.521753+0000","last_became_peered":"2023-10-10T20:36:21.521753+0000","last_unstale":"2023-10-10T20:36:21.524481+0000","last_undegraded":"2023-10-10T20:36:21.524481+0000","last_fullsized":"2023-10-10T20:36:21.524481+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:01:01.948430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.528031+0000","last_change":"2023-10-10T20:36:21.519328+0000","last_active":"2023-10-10T20:36:21.528031+0000","last_peered":"2023-10-10T20:36:21.528031+0000","last_clean":"2023-10-10T20:36:21.528031+0000","last_became_active":"2023-10-10T20:36:21.519068+0000","last_became_peered":"2023-10-10T20:36:21.519068+0000","last_unstale":"2023-10-10T20:36:21.528031+0000","last_undegraded":"2023-10-10T20:36:21.528031+0000","last_fullsized":"2023-10-10T20:36:21.528031+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:23:47.669757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.528039+0000","last_change":"2023-10-10T20:36:21.521383+0000","last_active":"2023-10-10T20:36:21.528039+0000","last_peered":"2023-10-10T20:36:21.528039+0000","last_clean":"2023-10-10T20:36:21.528039+0000","last_became_active":"2023-10-10T20:36:21.517956+0000","last_became_peered":"2023-10-10T20:36:21.517956+0000","last_unstale":"2023-10-10T20:36:21.528039+0000","last_undegraded":"2023-10-10T20:36:21.528039+0000","last_fullsized":"2023-10-10T20:36:21.528039+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:00:10.479829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.532734+0000","last_change":"2023-10-10T20:36:21.525888+0000","last_active":"2023-10-10T20:36:21.532734+0000","last_peered":"2023-10-10T20:36:21.532734+0000","last_clean":"2023-10-10T20:36:21.532734+0000","last_became_active":"2023-10-10T20:36:21.525043+0000","last_became_peered":"2023-10-10T20:36:21.525043+0000","last_unstale":"2023-10-10T20:36:21.532734+0000","last_undegraded":"2023-10-10T20:36:21.532734+0000","last_fullsized":"2023-10-10T20:36:21.532734+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:52:24.203489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.522142+0000","last_change":"2023-10-10T20:36:21.517680+0000","last_active":"2023-10-10T20:36:21.522142+0000","last_peered":"2023-10-10T20:36:21.522142+0000","last_clean":"2023-10-10T20:36:21.522142+0000","last_became_active":"2023-10-10T20:36:21.517432+0000","last_became_peered":"2023-10-10T20:36:21.517432+0000","last_unstale":"2023-10-10T20:36:21.522142+0000","last_undegraded":"2023-10-10T20:36:21.522142+0000","last_fullsized":"2023-10-10T20:36:21.522142+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:44:10.575443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.519544+0000","last_change":"2023-10-10T20:36:21.519544+0000","last_active":"2023-10-10T20:36:21.519544+0000","last_peered":"2023-10-10T20:36:21.519544+0000","last_clean":"2023-10-10T20:36:21.519544+0000","last_became_active":"2023-10-10T20:36:21.519243+0000","last_became_peered":"2023-10-10T20:36:21.519243+0000","last_unstale":"2023-10-10T20:36:21.519544+0000","last_undegraded":"2023-10-10T20:36:21.519544+0000","last_fullsized":"2023-10-10T20:36:21.519544+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:09:19.504447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.532687+0000","last_change":"2023-10-10T20:36:21.525603+0000","last_active":"2023-10-10T20:36:21.532687+0000","last_peered":"2023-10-10T20:36:21.532687+0000","last_clean":"2023-10-10T20:36:21.532687+0000","last_became_active":"2023-10-10T20:36:21.525318+0000","last_became_peered":"2023-10-10T20:36:21.525318+0000","last_unstale":"2023-10-10T20:36:21.532687+0000","last_undegraded":"2023-10-10T20:36:21.532687+0000","last_fullsized":"2023-10-10T20:36:21.532687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:09.868421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.518446+0000","last_change":"2023-10-10T20:36:21.518446+0000","last_active":"2023-10-10T20:36:21.518446+0000","last_peered":"2023-10-10T20:36:21.518446+0000","last_clean":"2023-10-10T20:36:21.518446+0000","last_became_active":"2023-10-10T20:36:21.518099+0000","last_became_peered":"2023-10-10T20:36:21.518099+0000","last_unstale":"2023-10-10T20:36:21.518446+0000","last_undegraded":"2023-10-10T20:36:21.518446+0000","last_fullsized":"2023-10-10T20:36:21.518446+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:45:20.306180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.520878+0000","last_change":"2023-10-10T20:36:21.515422+0000","last_active":"2023-10-10T20:36:21.520878+0000","last_peered":"2023-10-10T20:36:21.520878+0000","last_clean":"2023-10-10T20:36:21.520878+0000","last_became_active":"2023-10-10T20:36:21.515179+0000","last_became_peered":"2023-10-10T20:36:21.515179+0000","last_unstale":"2023-10-10T20:36:21.520878+0000","last_undegraded":"2023-10-10T20:36:21.520878+0000","last_fullsized":"2023-10-10T20:36:21.520878+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:36:10.486811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.526272+0000","last_change":"2023-10-10T20:36:21.523186+0000","last_active":"2023-10-10T20:36:21.526272+0000","last_peered":"2023-10-10T20:36:21.526272+0000","last_clean":"2023-10-10T20:36:21.526272+0000","last_became_active":"2023-10-10T20:36:21.522938+0000","last_became_peered":"2023-10-10T20:36:21.522938+0000","last_unstale":"2023-10-10T20:36:21.526272+0000","last_undegraded":"2023-10-10T20:36:21.526272+0000","last_fullsized":"2023-10-10T20:36:21.526272+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:56:38.109485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.521756+0000","last_change":"2023-10-10T20:36:21.521756+0000","last_active":"2023-10-10T20:36:21.521756+0000","last_peered":"2023-10-10T20:36:21.521756+0000","last_clean":"2023-10-10T20:36:21.521756+0000","last_became_active":"2023-10-10T20:36:21.521484+0000","last_became_peered":"2023-10-10T20:36:21.521484+0000","last_unstale":"2023-10-10T20:36:21.521756+0000","last_undegraded":"2023-10-10T20:36:21.521756+0000","last_fullsized":"2023-10-10T20:36:21.521756+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:39:14.489065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.523993+0000","last_change":"2023-10-10T20:36:21.515995+0000","last_active":"2023-10-10T20:36:21.523993+0000","last_peered":"2023-10-10T20:36:21.523993+0000","last_clean":"2023-10-10T20:36:21.523993+0000","last_became_active":"2023-10-10T20:36:21.515463+0000","last_became_peered":"2023-10-10T20:36:21.515463+0000","last_unstale":"2023-10-10T20:36:21.523993+0000","last_undegraded":"2023-10-10T20:36:21.523993+0000","last_fullsized":"2023-10-10T20:36:21.523993+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:48:14.169260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"61'4","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.531211+0000","last_change":"2023-10-10T20:36:20.498938+0000","last_active":"2023-10-10T20:36:21.531211+0000","last_peered":"2023-10-10T20:36:21.531211+0000","last_clean":"2023-10-10T20:36:21.531211+0000","last_became_active":"2023-10-10T20:36:20.498679+0000","last_became_peered":"2023-10-10T20:36:20.498679+0000","last_unstale":"2023-10-10T20:36:21.531211+0000","last_undegraded":"2023-10-10T20:36:21.531211+0000","last_fullsized":"2023-10-10T20:36:21.531211+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:43:19.431501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.518818+0000","last_change":"2023-10-10T20:36:21.518818+0000","last_active":"2023-10-10T20:36:21.518818+0000","last_peered":"2023-10-10T20:36:21.518818+0000","last_clean":"2023-10-10T20:36:21.518818+0000","last_became_active":"2023-10-10T20:36:21.518607+0000","last_became_peered":"2023-10-10T20:36:21.518607+0000","last_unstale":"2023-10-10T20:36:21.518818+0000","last_undegraded":"2023-10-10T20:36:21.518818+0000","last_fullsized":"2023-10-10T20:36:21.518818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:10:51.485286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.502947+0000","last_change":"2023-10-10T20:36:20.506790+0000","last_active":"2023-10-10T20:36:21.502947+0000","last_peered":"2023-10-10T20:36:21.502947+0000","last_clean":"2023-10-10T20:36:21.502947+0000","last_became_active":"2023-10-10T20:36:20.506501+0000","last_became_peered":"2023-10-10T20:36:20.506501+0000","last_unstale":"2023-10-10T20:36:21.502947+0000","last_undegraded":"2023-10-10T20:36:21.502947+0000","last_fullsized":"2023-10-10T20:36:21.502947+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:56.311673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.525588+0000","last_change":"2023-10-10T20:36:21.525588+0000","last_active":"2023-10-10T20:36:21.525588+0000","last_peered":"2023-10-10T20:36:21.525588+0000","last_clean":"2023-10-10T20:36:21.525588+0000","last_became_active":"2023-10-10T20:36:21.525338+0000","last_became_peered":"2023-10-10T20:36:21.525338+0000","last_unstale":"2023-10-10T20:36:21.525588+0000","last_undegraded":"2023-10-10T20:36:21.525588+0000","last_fullsized":"2023-10-10T20:36:21.525588+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:52:24.203489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"61'2","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.532332+0000","last_change":"2023-10-10T20:36:20.500244+0000","last_active":"2023-10-10T20:36:21.532332+0000","last_peered":"2023-10-10T20:36:21.532332+0000","last_clean":"2023-10-10T20:36:21.532332+0000","last_became_active":"2023-10-10T20:36:20.499989+0000","last_became_peered":"2023-10-10T20:36:20.499989+0000","last_unstale":"2023-10-10T20:36:21.532332+0000","last_undegraded":"2023-10-10T20:36:21.532332+0000","last_fullsized":"2023-10-10T20:36:21.532332+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:27:22.089809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.517730+0000","last_change":"2023-10-10T20:36:21.510979+0000","last_active":"2023-10-10T20:36:21.517730+0000","last_peered":"2023-10-10T20:36:21.517730+0000","last_clean":"2023-10-10T20:36:21.517730+0000","last_became_active":"2023-10-10T20:36:21.510713+0000","last_became_peered":"2023-10-10T20:36:21.510713+0000","last_unstale":"2023-10-10T20:36:21.517730+0000","last_undegraded":"2023-10-10T20:36:21.517730+0000","last_fullsized":"2023-10-10T20:36:21.517730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:33:07.729616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.530864+0000","last_change":"2023-10-10T20:36:20.500071+0000","last_active":"2023-10-10T20:36:21.530864+0000","last_peered":"2023-10-10T20:36:21.530864+0000","last_clean":"2023-10-10T20:36:21.530864+0000","last_became_active":"2023-10-10T20:36:20.499578+0000","last_became_peered":"2023-10-10T20:36:20.499578+0000","last_unstale":"2023-10-10T20:36:21.530864+0000","last_undegraded":"2023-10-10T20:36:21.530864+0000","last_fullsized":"2023-10-10T20:36:21.530864+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:41.117726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.520858+0000","last_change":"2023-10-10T20:36:21.514919+0000","last_active":"2023-10-10T20:36:21.520858+0000","last_peered":"2023-10-10T20:36:21.520858+0000","last_clean":"2023-10-10T20:36:21.520858+0000","last_became_active":"2023-10-10T20:36:21.514650+0000","last_became_peered":"2023-10-10T20:36:21.514650+0000","last_unstale":"2023-10-10T20:36:21.520858+0000","last_undegraded":"2023-10-10T20:36:21.520858+0000","last_fullsized":"2023-10-10T20:36:21.520858+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:45:07.874792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.508985+0000","last_change":"2023-10-10T20:36:20.498535+0000","last_active":"2023-10-10T20:36:21.508985+0000","last_peered":"2023-10-10T20:36:21.508985+0000","last_clean":"2023-10-10T20:36:21.508985+0000","last_became_active":"2023-10-10T20:36:20.498275+0000","last_became_peered":"2023-10-10T20:36:20.498275+0000","last_unstale":"2023-10-10T20:36:21.508985+0000","last_undegraded":"2023-10-10T20:36:21.508985+0000","last_fullsized":"2023-10-10T20:36:21.508985+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:23:46.666090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.532798+0000","last_change":"2023-10-10T20:36:21.527199+0000","last_active":"2023-10-10T20:36:21.532798+0000","last_peered":"2023-10-10T20:36:21.532798+0000","last_clean":"2023-10-10T20:36:21.532798+0000","last_became_active":"2023-10-10T20:36:21.526474+0000","last_became_peered":"2023-10-10T20:36:21.526474+0000","last_unstale":"2023-10-10T20:36:21.532798+0000","last_undegraded":"2023-10-10T20:36:21.532798+0000","last_fullsized":"2023-10-10T20:36:21.532798+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:11:36.397020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.511709+0000","last_change":"2023-10-10T20:36:20.501670+0000","last_active":"2023-10-10T20:36:21.511709+0000","last_peered":"2023-10-10T20:36:21.511709+0000","last_clean":"2023-10-10T20:36:21.511709+0000","last_became_active":"2023-10-10T20:36:20.501390+0000","last_became_peered":"2023-10-10T20:36:20.501390+0000","last_unstale":"2023-10-10T20:36:21.511709+0000","last_undegraded":"2023-10-10T20:36:21.511709+0000","last_fullsized":"2023-10-10T20:36:21.511709+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:50.029620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.515235+0000","last_change":"2023-10-10T20:36:21.515235+0000","last_active":"2023-10-10T20:36:21.515235+0000","last_peered":"2023-10-10T20:36:21.515235+0000","last_clean":"2023-10-10T20:36:21.515235+0000","last_became_active":"2023-10-10T20:36:21.514989+0000","last_became_peered":"2023-10-10T20:36:21.514989+0000","last_unstale":"2023-10-10T20:36:21.515235+0000","last_undegraded":"2023-10-10T20:36:21.515235+0000","last_fullsized":"2023-10-10T20:36:21.515235+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:44:25.114526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.499925+0000","last_change":"2023-10-10T20:36:20.497222+0000","last_active":"2023-10-10T20:36:20.499925+0000","last_peered":"2023-10-10T20:36:20.499925+0000","last_clean":"2023-10-10T20:36:20.499925+0000","last_became_active":"2023-10-10T20:36:20.496978+0000","last_became_peered":"2023-10-10T20:36:20.496978+0000","last_unstale":"2023-10-10T20:36:20.499925+0000","last_undegraded":"2023-10-10T20:36:20.499925+0000","last_fullsized":"2023-10-10T20:36:20.499925+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:53:12.126771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.509669+0000","last_change":"2023-10-10T20:36:21.509669+0000","last_active":"2023-10-10T20:36:21.509669+0000","last_peered":"2023-10-10T20:36:21.509669+0000","last_clean":"2023-10-10T20:36:21.509669+0000","last_became_active":"2023-10-10T20:36:21.509433+0000","last_became_peered":"2023-10-10T20:36:21.509433+0000","last_unstale":"2023-10-10T20:36:21.509669+0000","last_undegraded":"2023-10-10T20:36:21.509669+0000","last_fullsized":"2023-10-10T20:36:21.509669+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:24:21.171886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.506166+0000","last_change":"2023-10-10T20:36:20.500783+0000","last_active":"2023-10-10T20:36:20.506166+0000","last_peered":"2023-10-10T20:36:20.506166+0000","last_clean":"2023-10-10T20:36:20.506166+0000","last_became_active":"2023-10-10T20:36:20.500503+0000","last_became_peered":"2023-10-10T20:36:20.500503+0000","last_unstale":"2023-10-10T20:36:20.506166+0000","last_undegraded":"2023-10-10T20:36:20.506166+0000","last_fullsized":"2023-10-10T20:36:20.506166+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:19:05.349730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.522122+0000","last_change":"2023-10-10T20:36:21.516803+0000","last_active":"2023-10-10T20:36:21.522122+0000","last_peered":"2023-10-10T20:36:21.522122+0000","last_clean":"2023-10-10T20:36:21.522122+0000","last_became_active":"2023-10-10T20:36:21.516512+0000","last_became_peered":"2023-10-10T20:36:21.516512+0000","last_unstale":"2023-10-10T20:36:21.522122+0000","last_undegraded":"2023-10-10T20:36:21.522122+0000","last_fullsized":"2023-10-10T20:36:21.522122+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:33:13.394154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.512956+0000","last_change":"2023-10-10T20:36:20.498783+0000","last_active":"2023-10-10T20:36:21.512956+0000","last_peered":"2023-10-10T20:36:21.512956+0000","last_clean":"2023-10-10T20:36:21.512956+0000","last_became_active":"2023-10-10T20:36:20.497548+0000","last_became_peered":"2023-10-10T20:36:20.497548+0000","last_unstale":"2023-10-10T20:36:21.512956+0000","last_undegraded":"2023-10-10T20:36:21.512956+0000","last_fullsized":"2023-10-10T20:36:21.512956+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:53:12.126771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.524056+0000","last_change":"2023-10-10T20:36:21.516709+0000","last_active":"2023-10-10T20:36:21.524056+0000","last_peered":"2023-10-10T20:36:21.524056+0000","last_clean":"2023-10-10T20:36:21.524056+0000","last_became_active":"2023-10-10T20:36:21.516423+0000","last_became_peered":"2023-10-10T20:36:21.516423+0000","last_unstale":"2023-10-10T20:36:21.524056+0000","last_undegraded":"2023-10-10T20:36:21.524056+0000","last_fullsized":"2023-10-10T20:36:21.524056+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:11:23.892822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.506162+0000","last_change":"2023-10-10T20:36:20.504774+0000","last_active":"2023-10-10T20:36:20.506162+0000","last_peered":"2023-10-10T20:36:20.506162+0000","last_clean":"2023-10-10T20:36:20.506162+0000","last_became_active":"2023-10-10T20:36:20.504518+0000","last_became_peered":"2023-10-10T20:36:20.504518+0000","last_unstale":"2023-10-10T20:36:20.506162+0000","last_undegraded":"2023-10-10T20:36:20.506162+0000","last_fullsized":"2023-10-10T20:36:20.506162+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:17:20.214488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.520222+0000","last_change":"2023-10-10T20:36:21.520222+0000","last_active":"2023-10-10T20:36:21.520222+0000","last_peered":"2023-10-10T20:36:21.520222+0000","last_clean":"2023-10-10T20:36:21.520222+0000","last_became_active":"2023-10-10T20:36:21.519816+0000","last_became_peered":"2023-10-10T20:36:21.519816+0000","last_unstale":"2023-10-10T20:36:21.520222+0000","last_undegraded":"2023-10-10T20:36:21.520222+0000","last_fullsized":"2023-10-10T20:36:21.520222+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:57.315341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"61'3","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.530707+0000","last_change":"2023-10-10T20:36:20.494259+0000","last_active":"2023-10-10T20:36:21.530707+0000","last_peered":"2023-10-10T20:36:21.530707+0000","last_clean":"2023-10-10T20:36:21.530707+0000","last_became_active":"2023-10-10T20:36:20.493992+0000","last_became_peered":"2023-10-10T20:36:20.493992+0000","last_unstale":"2023-10-10T20:36:21.530707+0000","last_undegraded":"2023-10-10T20:36:21.530707+0000","last_fullsized":"2023-10-10T20:36:21.530707+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:09:09.540955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.511431+0000","last_change":"2023-10-10T20:36:21.511431+0000","last_active":"2023-10-10T20:36:21.511431+0000","last_peered":"2023-10-10T20:36:21.511431+0000","last_clean":"2023-10-10T20:36:21.511431+0000","last_became_active":"2023-10-10T20:36:21.511162+0000","last_became_peered":"2023-10-10T20:36:21.511162+0000","last_unstale":"2023-10-10T20:36:21.511431+0000","last_undegraded":"2023-10-10T20:36:21.511431+0000","last_fullsized":"2023-10-10T20:36:21.511431+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:21:57.926200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"61'6","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.530767+0000","last_change":"2023-10-10T20:36:20.498323+0000","last_active":"2023-10-10T20:36:21.530767+0000","last_peered":"2023-10-10T20:36:21.530767+0000","last_clean":"2023-10-10T20:36:21.530767+0000","last_became_active":"2023-10-10T20:36:20.497587+0000","last_became_peered":"2023-10-10T20:36:20.497587+0000","last_unstale":"2023-10-10T20:36:21.530767+0000","last_undegraded":"2023-10-10T20:36:21.530767+0000","last_fullsized":"2023-10-10T20:36:21.530767+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:50.029620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.521183+0000","last_change":"2023-10-10T20:36:21.511754+0000","last_active":"2023-10-10T20:36:21.521183+0000","last_peered":"2023-10-10T20:36:21.521183+0000","last_clean":"2023-10-10T20:36:21.521183+0000","last_became_active":"2023-10-10T20:36:21.511357+0000","last_became_peered":"2023-10-10T20:36:21.511357+0000","last_unstale":"2023-10-10T20:36:21.521183+0000","last_undegraded":"2023-10-10T20:36:21.521183+0000","last_fullsized":"2023-10-10T20:36:21.521183+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:33:13.394154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.529310+0000","last_change":"2023-10-10T20:36:20.501259+0000","last_active":"2023-10-10T20:36:21.529310+0000","last_peered":"2023-10-10T20:36:21.529310+0000","last_clean":"2023-10-10T20:36:21.529310+0000","last_became_active":"2023-10-10T20:36:20.497900+0000","last_became_peered":"2023-10-10T20:36:20.497900+0000","last_unstale":"2023-10-10T20:36:21.529310+0000","last_undegraded":"2023-10-10T20:36:21.529310+0000","last_fullsized":"2023-10-10T20:36:21.529310+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:35:50.152564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.521193+0000","last_change":"2023-10-10T20:36:21.516087+0000","last_active":"2023-10-10T20:36:21.521193+0000","last_peered":"2023-10-10T20:36:21.521193+0000","last_clean":"2023-10-10T20:36:21.521193+0000","last_became_active":"2023-10-10T20:36:21.515800+0000","last_became_peered":"2023-10-10T20:36:21.515800+0000","last_unstale":"2023-10-10T20:36:21.521193+0000","last_undegraded":"2023-10-10T20:36:21.521193+0000","last_fullsized":"2023-10-10T20:36:21.521193+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:24:56.826111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.508889+0000","last_change":"2023-10-10T20:36:20.508382+0000","last_active":"2023-10-10T20:36:20.508889+0000","last_peered":"2023-10-10T20:36:20.508889+0000","last_clean":"2023-10-10T20:36:20.508889+0000","last_became_active":"2023-10-10T20:36:20.508154+0000","last_became_peered":"2023-10-10T20:36:20.508154+0000","last_unstale":"2023-10-10T20:36:20.508889+0000","last_undegraded":"2023-10-10T20:36:20.508889+0000","last_fullsized":"2023-10-10T20:36:20.508889+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:24:41.733760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.517285+0000","last_change":"2023-10-10T20:36:21.517285+0000","last_active":"2023-10-10T20:36:21.517285+0000","last_peered":"2023-10-10T20:36:21.517285+0000","last_clean":"2023-10-10T20:36:21.517285+0000","last_became_active":"2023-10-10T20:36:21.517040+0000","last_became_peered":"2023-10-10T20:36:21.517040+0000","last_unstale":"2023-10-10T20:36:21.517285+0000","last_undegraded":"2023-10-10T20:36:21.517285+0000","last_fullsized":"2023-10-10T20:36:21.517285+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:13:56.076698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.511861+0000","last_change":"2023-10-10T20:36:20.503387+0000","last_active":"2023-10-10T20:36:21.511861+0000","last_peered":"2023-10-10T20:36:21.511861+0000","last_clean":"2023-10-10T20:36:21.511861+0000","last_became_active":"2023-10-10T20:36:20.503089+0000","last_became_peered":"2023-10-10T20:36:20.503089+0000","last_unstale":"2023-10-10T20:36:21.511861+0000","last_undegraded":"2023-10-10T20:36:21.511861+0000","last_fullsized":"2023-10-10T20:36:21.511861+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:06.703672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.514118+0000","last_change":"2023-10-10T20:36:21.514118+0000","last_active":"2023-10-10T20:36:21.514118+0000","last_peered":"2023-10-10T20:36:21.514118+0000","last_clean":"2023-10-10T20:36:21.514118+0000","last_became_active":"2023-10-10T20:36:21.513869+0000","last_became_peered":"2023-10-10T20:36:21.513869+0000","last_unstale":"2023-10-10T20:36:21.514118+0000","last_undegraded":"2023-10-10T20:36:21.514118+0000","last_fullsized":"2023-10-10T20:36:21.514118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:42.776284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"61'8","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.533014+0000","last_change":"2023-10-10T20:36:20.499777+0000","last_active":"2023-10-10T20:36:21.533014+0000","last_peered":"2023-10-10T20:36:21.533014+0000","last_clean":"2023-10-10T20:36:21.533014+0000","last_became_active":"2023-10-10T20:36:20.499501+0000","last_became_peered":"2023-10-10T20:36:20.499501+0000","last_unstale":"2023-10-10T20:36:21.533014+0000","last_undegraded":"2023-10-10T20:36:21.533014+0000","last_fullsized":"2023-10-10T20:36:21.533014+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:35:50.152564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.521417+0000","last_change":"2023-10-10T20:36:21.518427+0000","last_active":"2023-10-10T20:36:21.521417+0000","last_peered":"2023-10-10T20:36:21.521417+0000","last_clean":"2023-10-10T20:36:21.521417+0000","last_became_active":"2023-10-10T20:36:21.518151+0000","last_became_peered":"2023-10-10T20:36:21.518151+0000","last_unstale":"2023-10-10T20:36:21.521417+0000","last_undegraded":"2023-10-10T20:36:21.521417+0000","last_fullsized":"2023-10-10T20:36:21.521417+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:57.315341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.511731+0000","last_change":"2023-10-10T20:36:20.509410+0000","last_active":"2023-10-10T20:36:21.511731+0000","last_peered":"2023-10-10T20:36:21.511731+0000","last_clean":"2023-10-10T20:36:21.511731+0000","last_became_active":"2023-10-10T20:36:20.509049+0000","last_became_peered":"2023-10-10T20:36:20.509049+0000","last_unstale":"2023-10-10T20:36:21.511731+0000","last_undegraded":"2023-10-10T20:36:21.511731+0000","last_fullsized":"2023-10-10T20:36:21.511731+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:38:55.245786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.522220+0000","last_change":"2023-10-10T20:36:21.513324+0000","last_active":"2023-10-10T20:36:21.522220+0000","last_peered":"2023-10-10T20:36:21.522220+0000","last_clean":"2023-10-10T20:36:21.522220+0000","last_became_active":"2023-10-10T20:36:21.512662+0000","last_became_peered":"2023-10-10T20:36:21.512662+0000","last_unstale":"2023-10-10T20:36:21.522220+0000","last_undegraded":"2023-10-10T20:36:21.522220+0000","last_fullsized":"2023-10-10T20:36:21.522220+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:24:56.826111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"61'2","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.532751+0000","last_change":"2023-10-10T20:36:20.504643+0000","last_active":"2023-10-10T20:36:21.532751+0000","last_peered":"2023-10-10T20:36:21.532751+0000","last_clean":"2023-10-10T20:36:21.532751+0000","last_became_active":"2023-10-10T20:36:20.504381+0000","last_became_peered":"2023-10-10T20:36:20.504381+0000","last_unstale":"2023-10-10T20:36:21.532751+0000","last_undegraded":"2023-10-10T20:36:21.532751+0000","last_fullsized":"2023-10-10T20:36:21.532751+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:35:08.260987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.510308+0000","last_change":"2023-10-10T20:36:21.510308+0000","last_active":"2023-10-10T20:36:21.510308+0000","last_peered":"2023-10-10T20:36:21.510308+0000","last_clean":"2023-10-10T20:36:21.510308+0000","last_became_active":"2023-10-10T20:36:21.510051+0000","last_became_peered":"2023-10-10T20:36:21.510051+0000","last_unstale":"2023-10-10T20:36:21.510308+0000","last_undegraded":"2023-10-10T20:36:21.510308+0000","last_fullsized":"2023-10-10T20:36:21.510308+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:07:19.870599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.514109+0000","last_change":"2023-10-10T20:36:20.500546+0000","last_active":"2023-10-10T20:36:21.514109+0000","last_peered":"2023-10-10T20:36:21.514109+0000","last_clean":"2023-10-10T20:36:21.514109+0000","last_became_active":"2023-10-10T20:36:20.500290+0000","last_became_peered":"2023-10-10T20:36:20.500290+0000","last_unstale":"2023-10-10T20:36:21.514109+0000","last_undegraded":"2023-10-10T20:36:21.514109+0000","last_fullsized":"2023-10-10T20:36:21.514109+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:25.236700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.517145+0000","last_change":"2023-10-10T20:36:21.517145+0000","last_active":"2023-10-10T20:36:21.517145+0000","last_peered":"2023-10-10T20:36:21.517145+0000","last_clean":"2023-10-10T20:36:21.517145+0000","last_became_active":"2023-10-10T20:36:21.516865+0000","last_became_peered":"2023-10-10T20:36:21.516865+0000","last_unstale":"2023-10-10T20:36:21.517145+0000","last_undegraded":"2023-10-10T20:36:21.517145+0000","last_fullsized":"2023-10-10T20:36:21.517145+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:20:36.461834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.502130+0000","last_change":"2023-10-10T20:36:20.501499+0000","last_active":"2023-10-10T20:36:21.502130+0000","last_peered":"2023-10-10T20:36:21.502130+0000","last_clean":"2023-10-10T20:36:21.502130+0000","last_became_active":"2023-10-10T20:36:20.501235+0000","last_became_peered":"2023-10-10T20:36:20.501235+0000","last_unstale":"2023-10-10T20:36:21.502130+0000","last_undegraded":"2023-10-10T20:36:21.502130+0000","last_fullsized":"2023-10-10T20:36:21.502130+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:08.864753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.518260+0000","last_change":"2023-10-10T20:36:21.518260+0000","last_active":"2023-10-10T20:36:21.518260+0000","last_peered":"2023-10-10T20:36:21.518260+0000","last_clean":"2023-10-10T20:36:21.518260+0000","last_became_active":"2023-10-10T20:36:21.513050+0000","last_became_peered":"2023-10-10T20:36:21.513050+0000","last_unstale":"2023-10-10T20:36:21.518260+0000","last_undegraded":"2023-10-10T20:36:21.518260+0000","last_fullsized":"2023-10-10T20:36:21.518260+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:51:30.117443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.504194+0000","last_change":"2023-10-10T20:36:20.502598+0000","last_active":"2023-10-10T20:36:20.504194+0000","last_peered":"2023-10-10T20:36:20.504194+0000","last_clean":"2023-10-10T20:36:20.504194+0000","last_became_active":"2023-10-10T20:36:20.502263+0000","last_became_peered":"2023-10-10T20:36:20.502263+0000","last_unstale":"2023-10-10T20:36:20.504194+0000","last_undegraded":"2023-10-10T20:36:20.504194+0000","last_fullsized":"2023-10-10T20:36:20.504194+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:56:05.686047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.518667+0000","last_change":"2023-10-10T20:36:21.518667+0000","last_active":"2023-10-10T20:36:21.518667+0000","last_peered":"2023-10-10T20:36:21.518667+0000","last_clean":"2023-10-10T20:36:21.518667+0000","last_became_active":"2023-10-10T20:36:21.518488+0000","last_became_peered":"2023-10-10T20:36:21.518488+0000","last_unstale":"2023-10-10T20:36:21.518667+0000","last_undegraded":"2023-10-10T20:36:21.518667+0000","last_fullsized":"2023-10-10T20:36:21.518667+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:52:20.259891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.511953+0000","last_change":"2023-10-10T20:36:20.504838+0000","last_active":"2023-10-10T20:36:21.511953+0000","last_peered":"2023-10-10T20:36:21.511953+0000","last_clean":"2023-10-10T20:36:21.511953+0000","last_became_active":"2023-10-10T20:36:20.504509+0000","last_became_peered":"2023-10-10T20:36:20.504509+0000","last_unstale":"2023-10-10T20:36:21.511953+0000","last_undegraded":"2023-10-10T20:36:21.511953+0000","last_fullsized":"2023-10-10T20:36:21.511953+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:45:38.046706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.517502+0000","last_change":"2023-10-10T20:36:21.517502+0000","last_active":"2023-10-10T20:36:21.517502+0000","last_peered":"2023-10-10T20:36:21.517502+0000","last_clean":"2023-10-10T20:36:21.517502+0000","last_became_active":"2023-10-10T20:36:21.517203+0000","last_became_peered":"2023-10-10T20:36:21.517203+0000","last_unstale":"2023-10-10T20:36:21.517502+0000","last_undegraded":"2023-10-10T20:36:21.517502+0000","last_fullsized":"2023-10-10T20:36:21.517502+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:10:15.579634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.508524+0000","last_change":"2023-10-10T20:36:20.504694+0000","last_active":"2023-10-10T20:36:20.508524+0000","last_peered":"2023-10-10T20:36:20.508524+0000","last_clean":"2023-10-10T20:36:20.508524+0000","last_became_active":"2023-10-10T20:36:20.504440+0000","last_became_peered":"2023-10-10T20:36:20.504440+0000","last_unstale":"2023-10-10T20:36:20.508524+0000","last_undegraded":"2023-10-10T20:36:20.508524+0000","last_fullsized":"2023-10-10T20:36:20.508524+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:11:35.393353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.509824+0000","last_change":"2023-10-10T20:36:21.509824+0000","last_active":"2023-10-10T20:36:21.509824+0000","last_peered":"2023-10-10T20:36:21.509824+0000","last_clean":"2023-10-10T20:36:21.509824+0000","last_became_active":"2023-10-10T20:36:21.509556+0000","last_became_peered":"2023-10-10T20:36:21.509556+0000","last_unstale":"2023-10-10T20:36:21.509824+0000","last_undegraded":"2023-10-10T20:36:21.509824+0000","last_fullsized":"2023-10-10T20:36:21.509824+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:27.371302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.532437+0000","last_change":"2023-10-10T20:36:20.509385+0000","last_active":"2023-10-10T20:36:21.532437+0000","last_peered":"2023-10-10T20:36:21.532437+0000","last_clean":"2023-10-10T20:36:21.532437+0000","last_became_active":"2023-10-10T20:36:20.509054+0000","last_became_peered":"2023-10-10T20:36:20.509054+0000","last_unstale":"2023-10-10T20:36:21.532437+0000","last_undegraded":"2023-10-10T20:36:21.532437+0000","last_fullsized":"2023-10-10T20:36:21.532437+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:38:05.142702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.519328+0000","last_change":"2023-10-10T20:36:21.514636+0000","last_active":"2023-10-10T20:36:21.519328+0000","last_peered":"2023-10-10T20:36:21.519328+0000","last_clean":"2023-10-10T20:36:21.519328+0000","last_became_active":"2023-10-10T20:36:21.514299+0000","last_became_peered":"2023-10-10T20:36:21.514299+0000","last_unstale":"2023-10-10T20:36:21.519328+0000","last_undegraded":"2023-10-10T20:36:21.519328+0000","last_fullsized":"2023-10-10T20:36:21.519328+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:28:02.757484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.511506+0000","last_change":"2023-10-10T20:36:20.503318+0000","last_active":"2023-10-10T20:36:21.511506+0000","last_peered":"2023-10-10T20:36:21.511506+0000","last_clean":"2023-10-10T20:36:21.511506+0000","last_became_active":"2023-10-10T20:36:20.503066+0000","last_became_peered":"2023-10-10T20:36:20.503066+0000","last_unstale":"2023-10-10T20:36:21.511506+0000","last_undegraded":"2023-10-10T20:36:21.511506+0000","last_fullsized":"2023-10-10T20:36:21.511506+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:08.864753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.527677+0000","last_change":"2023-10-10T20:36:21.517948+0000","last_active":"2023-10-10T20:36:21.527677+0000","last_peered":"2023-10-10T20:36:21.527677+0000","last_clean":"2023-10-10T20:36:21.527677+0000","last_became_active":"2023-10-10T20:36:21.516384+0000","last_became_peered":"2023-10-10T20:36:21.516384+0000","last_unstale":"2023-10-10T20:36:21.527677+0000","last_undegraded":"2023-10-10T20:36:21.527677+0000","last_fullsized":"2023-10-10T20:36:21.527677+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:54:47.058919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.506269+0000","last_change":"2023-10-10T20:36:20.502283+0000","last_active":"2023-10-10T20:36:20.506269+0000","last_peered":"2023-10-10T20:36:20.506269+0000","last_clean":"2023-10-10T20:36:20.506269+0000","last_became_active":"2023-10-10T20:36:20.502048+0000","last_became_peered":"2023-10-10T20:36:20.502048+0000","last_unstale":"2023-10-10T20:36:20.506269+0000","last_undegraded":"2023-10-10T20:36:20.506269+0000","last_fullsized":"2023-10-10T20:36:20.506269+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:19:38.829714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.527729+0000","last_change":"2023-10-10T20:36:21.517626+0000","last_active":"2023-10-10T20:36:21.527729+0000","last_peered":"2023-10-10T20:36:21.527729+0000","last_clean":"2023-10-10T20:36:21.527729+0000","last_became_active":"2023-10-10T20:36:21.517353+0000","last_became_peered":"2023-10-10T20:36:21.517353+0000","last_unstale":"2023-10-10T20:36:21.527729+0000","last_undegraded":"2023-10-10T20:36:21.527729+0000","last_fullsized":"2023-10-10T20:36:21.527729+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:09:10.544623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.508567+0000","last_change":"2023-10-10T20:36:20.505099+0000","last_active":"2023-10-10T20:36:20.508567+0000","last_peered":"2023-10-10T20:36:20.508567+0000","last_clean":"2023-10-10T20:36:20.508567+0000","last_became_active":"2023-10-10T20:36:20.504778+0000","last_became_peered":"2023-10-10T20:36:20.504778+0000","last_unstale":"2023-10-10T20:36:20.508567+0000","last_undegraded":"2023-10-10T20:36:20.508567+0000","last_fullsized":"2023-10-10T20:36:20.508567+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:39:13.485398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.513102+0000","last_change":"2023-10-10T20:36:21.513102+0000","last_active":"2023-10-10T20:36:21.513102+0000","last_peered":"2023-10-10T20:36:21.513102+0000","last_clean":"2023-10-10T20:36:21.513102+0000","last_became_active":"2023-10-10T20:36:21.512779+0000","last_became_peered":"2023-10-10T20:36:21.512779+0000","last_unstale":"2023-10-10T20:36:21.513102+0000","last_undegraded":"2023-10-10T20:36:21.513102+0000","last_fullsized":"2023-10-10T20:36:21.513102+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:57:56.698645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.497171+0000","last_change":"2023-10-10T20:36:20.494842+0000","last_active":"2023-10-10T20:36:20.497171+0000","last_peered":"2023-10-10T20:36:20.497171+0000","last_clean":"2023-10-10T20:36:20.497171+0000","last_became_active":"2023-10-10T20:36:20.494633+0000","last_became_peered":"2023-10-10T20:36:20.494633+0000","last_unstale":"2023-10-10T20:36:20.497171+0000","last_undegraded":"2023-10-10T20:36:20.497171+0000","last_fullsized":"2023-10-10T20:36:20.497171+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:54:20.021115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.523032+0000","last_change":"2023-10-10T20:36:21.523032+0000","last_active":"2023-10-10T20:36:21.523032+0000","last_peered":"2023-10-10T20:36:21.523032+0000","last_clean":"2023-10-10T20:36:21.523032+0000","last_became_active":"2023-10-10T20:36:21.522782+0000","last_became_peered":"2023-10-10T20:36:21.522782+0000","last_unstale":"2023-10-10T20:36:21.523032+0000","last_undegraded":"2023-10-10T20:36:21.523032+0000","last_fullsized":"2023-10-10T20:36:21.523032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:27:20.416644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.499878+0000","last_change":"2023-10-10T20:36:20.496726+0000","last_active":"2023-10-10T20:36:20.499878+0000","last_peered":"2023-10-10T20:36:20.499878+0000","last_clean":"2023-10-10T20:36:20.499878+0000","last_became_active":"2023-10-10T20:36:20.496480+0000","last_became_peered":"2023-10-10T20:36:20.496480+0000","last_unstale":"2023-10-10T20:36:20.499878+0000","last_undegraded":"2023-10-10T20:36:20.499878+0000","last_fullsized":"2023-10-10T20:36:20.499878+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:22:59.981260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.513112+0000","last_change":"2023-10-10T20:36:21.513112+0000","last_active":"2023-10-10T20:36:21.513112+0000","last_peered":"2023-10-10T20:36:21.513112+0000","last_clean":"2023-10-10T20:36:21.513112+0000","last_became_active":"2023-10-10T20:36:21.512752+0000","last_became_peered":"2023-10-10T20:36:21.512752+0000","last_unstale":"2023-10-10T20:36:21.513112+0000","last_undegraded":"2023-10-10T20:36:21.513112+0000","last_fullsized":"2023-10-10T20:36:21.513112+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:37.824587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.508221+0000","last_change":"2023-10-10T20:36:20.499139+0000","last_active":"2023-10-10T20:36:21.508221+0000","last_peered":"2023-10-10T20:36:21.508221+0000","last_clean":"2023-10-10T20:36:21.508221+0000","last_became_active":"2023-10-10T20:36:20.498860+0000","last_became_peered":"2023-10-10T20:36:20.498860+0000","last_unstale":"2023-10-10T20:36:21.508221+0000","last_undegraded":"2023-10-10T20:36:21.508221+0000","last_fullsized":"2023-10-10T20:36:21.508221+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:22:59.981260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.521093+0000","last_change":"2023-10-10T20:36:21.521093+0000","last_active":"2023-10-10T20:36:21.521093+0000","last_peered":"2023-10-10T20:36:21.521093+0000","last_clean":"2023-10-10T20:36:21.521093+0000","last_became_active":"2023-10-10T20:36:21.520739+0000","last_became_peered":"2023-10-10T20:36:21.520739+0000","last_unstale":"2023-10-10T20:36:21.521093+0000","last_undegraded":"2023-10-10T20:36:21.521093+0000","last_fullsized":"2023-10-10T20:36:21.521093+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:11:36.397020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-10T20:36:20.504869+0000","last_change":"2023-10-10T20:36:20.502795+0000","last_active":"2023-10-10T20:36:20.504869+0000","last_peered":"2023-10-10T20:36:20.504869+0000","last_clean":"2023-10-10T20:36:20.504869+0000","last_became_active":"2023-10-10T20:36:20.502459+0000","last_became_peered":"2023-10-10T20:36:20.502459+0000","last_unstale":"2023-10-10T20:36:20.504869+0000","last_undegraded":"2023-10-10T20:36:20.504869+0000","last_fullsized":"2023-10-10T20:36:20.504869+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:50:26.158577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.515077+0000","last_change":"2023-10-10T20:36:21.515077+0000","last_active":"2023-10-10T20:36:21.515077+0000","last_peered":"2023-10-10T20:36:21.515077+0000","last_clean":"2023-10-10T20:36:21.515077+0000","last_became_active":"2023-10-10T20:36:21.514835+0000","last_became_peered":"2023-10-10T20:36:21.514835+0000","last_unstale":"2023-10-10T20:36:21.515077+0000","last_undegraded":"2023-10-10T20:36:21.515077+0000","last_fullsized":"2023-10-10T20:36:21.515077+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:20.481438+0000","last_clean_scrub_stamp":"2023-10-10T20:36:20.481438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:46:55.099687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.530872+0000","last_change":"2023-10-10T20:36:20.500621+0000","last_active":"2023-10-10T20:36:21.530872+0000","last_peered":"2023-10-10T20:36:21.530872+0000","last_clean":"2023-10-10T20:36:21.530872+0000","last_became_active":"2023-10-10T20:36:20.500328+0000","last_became_peered":"2023-10-10T20:36:20.500328+0000","last_unstale":"2023-10-10T20:36:21.530872+0000","last_undegraded":"2023-10-10T20:36:21.530872+0000","last_fullsized":"2023-10-10T20:36:21.530872+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:36:19.477770+0000","last_clean_scrub_stamp":"2023-10-10T20:36:19.477770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:30.267821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":205,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-10T20:36:21.501984+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:36:21.501984+0000","last_peered":"2023-10-10T20:36:21.501984+0000","last_clean":"2023-10-10T20:36:21.501984+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:36:21.501984+0000","last_undegraded":"2023-10-10T20:36:21.501984+0000","last_fullsized":"2023-10-10T20:36:21.501984+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298600,"kb_used_data":1312,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073240,"statfs":{"total":96636764160,"available":96330997760,"internally_reserved":0,"allocated":1343488,"data_stored":905080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":6.8630000000000004,"5min":6.8630000000000004,"15min":6.8630000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":4.1959999999999997,"5min":4.1959999999999997,"15min":4.1959999999999997},"last":0.33700000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":10.1,"5min":10.1,"15min":10.1},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":6.806,"5min":6.806,"15min":6.806},"last":0.437}]},{"osd":2,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":6.9340000000000002,"5min":6.9340000000000002,"15min":6.9340000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":6.9870000000000001,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.38}]},{"osd":3,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":6.6719999999999997,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":4.1609999999999996,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.39000000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.32000000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.44800000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55600000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298020,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073820,"statfs":{"total":96636764160,"available":96331591680,"internally_reserved":0,"allocated":749568,"data_stored":314216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":7.7590000000000003,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":7.8090000000000002,"5min":7.8090000000000002,"15min":7.8090000000000002},"last":0.46600000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":7.867,"5min":7.867,"15min":7.867},"last":0.37},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":9.7089999999999996,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.50900000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":4.7999999999999998,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":6.7629999999999999,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":0.55000000000000004}]},{"osd":3,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":4.8620000000000001,"5min":4.8620000000000001,"15min":4.8620000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":7.9009999999999998,"5min":7.9009999999999998,"15min":7.9009999999999998},"last":0.40799999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.44}]},{"osd":5,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.48999999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.44800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298032,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073808,"statfs":{"total":96636764160,"available":96331579392,"internally_reserved":0,"allocated":761856,"data_stored":316248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":5.0380000000000003,"5min":5.0380000000000003,"15min":5.0380000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.48599999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":5.0919999999999996,"5min":5.0919999999999996,"15min":5.0919999999999996},"last":0.40300000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.1159999999999997,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.45700000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":4.9850000000000003,"5min":4.9850000000000003,"15min":4.9850000000000003},"last":0.65600000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.67000000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":5.0659999999999998,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.70899999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:35:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0499999999999998,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.74399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298588,"kb_used_data":1300,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073252,"statfs":{"total":96636764160,"available":96331010048,"internally_reserved":0,"allocated":1331200,"data_stored":903531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":7.9249999999999998,"5min":7.9249999999999998,"15min":7.9249999999999998},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":6.6299999999999999,"5min":6.6299999999999999,"15min":6.6299999999999999},"last":0.54900000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":8.2509999999999994,"5min":8.2509999999999994,"15min":8.2509999999999994},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":6.7370000000000001,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.51700000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":7.7960000000000003,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.505},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":7.8929999999999998,"5min":7.8929999999999998,"15min":7.8929999999999998},"last":0.53200000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":6.6929999999999996,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":7.6920000000000002,"5min":7.6920000000000002,"15min":7.6920000000000002},"last":0.73199999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":7.7690000000000001,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.8659999999999997,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":0.79000000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":7.8360000000000003,"5min":7.8360000000000003,"15min":7.8360000000000003},"last":0.753},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":6.6660000000000004,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.65600000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:35:32 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":6.7160000000000002,"5min":6.7160000000000002,"15min":6.7160000000000002},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":7.7370000000000001,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.71099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298024,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073816,"statfs":{"total":96636764160,"available":96331587584,"internally_reserved":0,"allocated":753664,"data_stored":315660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.435},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.5}]},{"osd":1,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":11.967000000000001,"5min":11.967000000000001,"15min":11.967000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.1100000000000003,"5min":4.1100000000000003,"15min":4.1100000000000003},"last":0.46800000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":6.3040000000000003,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":11.935,"5min":11.935,"15min":11.935},"last":0.52500000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.77700000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.72299999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.747},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.71199999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:35:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":6.2649999999999997,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.80600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298020,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073820,"statfs":{"total":96636764160,"available":96331591680,"internally_reserved":0,"allocated":749568,"data_stored":314236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.317,"15min":0.317},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.436,"5min":0.436,"15min":0.436},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.317}]},{"osd":1,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.35099999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.307}]},{"osd":4,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.48699999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.437},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.51200000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.498}]},{"osd":7,"last update":"Tue Oct 10 20:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.41899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298028,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073812,"statfs":{"total":96636764160,"available":96331583488,"internally_reserved":0,"allocated":757760,"data_stored":320482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":7.6719999999999997,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.55500000000000005}]},{"osd":1,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":7.7160000000000002,"5min":7.7160000000000002,"15min":7.7160000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.442}]},{"osd":2,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":7.7839999999999998,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.59099999999999997}]},{"osd":3,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.49299999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.374},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.63300000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.623}]},{"osd":7,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298028,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073812,"statfs":{"total":96636764160,"available":96331583488,"internally_reserved":0,"allocated":757760,"data_stored":320502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.443}]},{"osd":1,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":5.2679999999999998,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":5.3109999999999999,"5min":5.3109999999999999,"15min":5.3109999999999999},"last":0.48599999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.2960000000000003,"5min":5.2960000000000003,"15min":5.2960000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.47199999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":5.3330000000000002,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.2850000000000001,"5min":5.2850000000000001,"15min":5.2850000000000001},"last":0.40699999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.501}]},{"osd":6,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.53200000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:35:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.60399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:36:25.927 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:36:25.928 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:36:26.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.080+0000 7fefbffff700 1 Processor -- start 2023-10-10T20:36:26.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.080+0000 7fefbffff700 1 -- start start 2023-10-10T20:36:26.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.081+0000 7fefbffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fefc00cf480 0x7fefc015d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:26.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.081+0000 7fefbffff700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefc00b10f0 con 0x7fefc015d760 2023-10-10T20:36:26.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.081+0000 7fefbffff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefc015db80 con 0x7fefc01595d0 2023-10-10T20:36:26.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.081+0000 7fefbffff700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefc015e000 con 0x7fefc00cf480 2023-10-10T20:36:26.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.081+0000 7fefb67fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fefc00cf480 0x7fefc015d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:26.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.081+0000 7fefbf7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fefc015d760 0x7fefc00b1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56642/0 (socket says 172.21.15.89:56642) 2023-10-10T20:36:26.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.081+0000 7fefbf7fe700 1 -- 172.21.15.89:0/465802770 learned_addr learned my addr 172.21.15.89:0/465802770 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:26.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.081+0000 7fefb67fc700 1 -- 172.21.15.89:0/465802770 >> v1:172.21.15.111:6789/0 conn(0x7fefc01595d0 legacy=0x7fefc00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:26.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.081+0000 7fefb67fc700 1 -- 172.21.15.89:0/465802770 >> v1:172.21.15.89:6789/0 conn(0x7fefc015d760 legacy=0x7fefc00b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:26.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.081+0000 7fefbe7fc700 1 -- 172.21.15.89:0/465802770 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289923856 0 0) 0x7fefc00b10f0 con 0x7fefc015d760 2023-10-10T20:36:26.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.081+0000 7fefb67fc700 1 -- 172.21.15.89:0/465802770 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc00cf990 con 0x7fefc00cf480 2023-10-10T20:36:26.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.082+0000 7fefb67fc700 1 --2- 172.21.15.89:0/465802770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc00cf480 0x7fefc015d220 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fefa80098e0 tx=0x7fefa80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e476ec8f65714ad1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:26.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.082+0000 7fefbe7fc700 1 -- 172.21.15.89:0/465802770 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa8009e80 con 0x7fefc00cf480 2023-10-10T20:36:26.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.082+0000 7fefbe7fc700 1 -- 172.21.15.89:0/465802770 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa800b820 con 0x7fefc00cf480 2023-10-10T20:36:26.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.082+0000 7fefbe7fc700 1 -- 172.21.15.89:0/465802770 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa800bba0 con 0x7fefc00cf480 2023-10-10T20:36:26.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.082+0000 7fefbffff700 1 -- 172.21.15.89:0/465802770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc00cf480 msgr2=0x7fefc015d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:26.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.082+0000 7fefbffff700 1 --2- 172.21.15.89:0/465802770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc00cf480 0x7fefc015d220 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fefa80098e0 tx=0x7fefa80091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:26.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.082+0000 7fefbffff700 1 -- 172.21.15.89:0/465802770 shutdown_connections 2023-10-10T20:36:26.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.082+0000 7fefbffff700 1 --2- 172.21.15.89:0/465802770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc00cf480 0x7fefc015d220 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:26.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.082+0000 7fefbffff700 1 -- 172.21.15.89:0/465802770 >> 172.21.15.89:0/465802770 conn(0x7fefc00d05f0 msgr2=0x7fefc00d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:26.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.082+0000 7fefbffff700 1 -- 172.21.15.89:0/465802770 shutdown_connections 2023-10-10T20:36:26.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.082+0000 7fefbffff700 1 -- 172.21.15.89:0/465802770 wait complete. 2023-10-10T20:36:26.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.083+0000 7fefbffff700 1 Processor -- start 2023-10-10T20:36:26.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.084+0000 7fefbffff700 1 -- start start 2023-10-10T20:36:26.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.084+0000 7fefbffff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fefc00cf480 0x7fefc0154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:26.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.084+0000 7fefbffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc01595d0 0x7fefc016a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:26.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.084+0000 7fefbffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fefc015d760 0x7fefc0152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:26.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.084+0000 7fefbffff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc016a9e0 con 0x7fefc01595d0 2023-10-10T20:36:26.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.084+0000 7fefbffff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc0152780 con 0x7fefc015d760 2023-10-10T20:36:26.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.084+0000 7fefbffff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc01553d0 con 0x7fefc00cf480 2023-10-10T20:36:26.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.084+0000 7fefb67fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc01595d0 0x7fefc016a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:26.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.084+0000 7fefb67fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc01595d0 0x7fefc016a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46862/0 (socket says 172.21.15.89:46862) 2023-10-10T20:36:26.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.084+0000 7fefb67fc700 1 -- 172.21.15.89:0/3278594323 learned_addr learned my addr 172.21.15.89:0/3278594323 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:26.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.085+0000 7fefbf7fe700 1 --2- 172.21.15.89:0/3278594323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fefc015d760 0x7fefc0152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:26.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.085+0000 7fefbeffd700 1 --2- 172.21.15.89:0/3278594323 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fefc00cf480 0x7fefc0154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:26.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.085+0000 7fefb67fc700 1 -- 172.21.15.89:0/3278594323 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fefc00cf480 msgr2=0x7fefc0154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:26.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.085+0000 7fefb67fc700 1 --2- 172.21.15.89:0/3278594323 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fefc00cf480 0x7fefc0154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:26.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.085+0000 7fefb67fc700 1 -- 172.21.15.89:0/3278594323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fefc015d760 msgr2=0x7fefc0152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:26.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.085+0000 7fefb67fc700 1 --2- 172.21.15.89:0/3278594323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fefc015d760 0x7fefc0152240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:26.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.085+0000 7fefb67fc700 1 -- 172.21.15.89:0/3278594323 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa8009420 con 0x7fefc01595d0 2023-10-10T20:36:26.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.085+0000 7fefbf7fe700 1 --2- 172.21.15.89:0/3278594323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fefc015d760 0x7fefc0152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:26.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.085+0000 7fefb67fc700 1 --2- 172.21.15.89:0/3278594323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc01595d0 0x7fefc016a4a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fefa80030c0 tx=0x7fefa8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:26.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.085+0000 7fefb77fe700 1 -- 172.21.15.89:0/3278594323 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa801b470 con 0x7fefc01595d0 2023-10-10T20:36:26.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.085+0000 7fefb77fe700 1 -- 172.21.15.89:0/3278594323 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa801b5d0 con 0x7fefc01595d0 2023-10-10T20:36:26.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.085+0000 7fefbffff700 1 -- 172.21.15.89:0/3278594323 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc01556b0 con 0x7fefc01595d0 2023-10-10T20:36:26.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.085+0000 7fefb77fe700 1 -- 172.21.15.89:0/3278594323 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa801b990 con 0x7fefc01595d0 2023-10-10T20:36:26.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.085+0000 7fefbffff700 1 -- 172.21.15.89:0/3278594323 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc0155c70 con 0x7fefc01595d0 2023-10-10T20:36:26.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.086+0000 7fefbffff700 1 -- 172.21.15.89:0/3278594323 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc016ac60 con 0x7fefc01595d0 2023-10-10T20:36:26.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.086+0000 7fefb77fe700 1 -- 172.21.15.89:0/3278594323 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fefa801c450 con 0x7fefc01595d0 2023-10-10T20:36:26.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.087+0000 7fefb77fe700 1 --2- 172.21.15.89:0/3278594323 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fefa0066fc0 0x7fefa0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:26.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.087+0000 7fefb77fe700 1 -- 172.21.15.89:0/3278594323 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fefa8086470 con 0x7fefc01595d0 2023-10-10T20:36:26.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.087+0000 7fefbeffd700 1 --2- 172.21.15.89:0/3278594323 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fefa0066fc0 0x7fefa0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:26.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.087+0000 7fefbeffd700 1 --2- 172.21.15.89:0/3278594323 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fefa0066fc0 0x7fefa0069470 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fefc016cf60 tx=0x7fefb0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:26.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.091+0000 7fefb77fe700 1 -- 172.21.15.89:0/3278594323 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fefc016ac60 con 0x7fefc01595d0 2023-10-10T20:36:26.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.348+0000 7fefbffff700 1 -- 172.21.15.89:0/3278594323 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fefc016ac60 con 0x7fefc01595d0 2023-10-10T20:36:26.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.352+0000 7fefb77fe700 1 -- 172.21.15.89:0/3278594323 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7fefc016ac60 con 0x7fefc01595d0 2023-10-10T20:36:26.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.354+0000 7fefbffff700 1 -- 172.21.15.89:0/3278594323 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fefa0066fc0 msgr2=0x7fefa0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:26.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.354+0000 7fefbffff700 1 --2- 172.21.15.89:0/3278594323 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fefa0066fc0 0x7fefa0069470 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fefc016cf60 tx=0x7fefb0009040 comp rx=0 tx=0).stop 2023-10-10T20:36:26.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.354+0000 7fefbffff700 1 -- 172.21.15.89:0/3278594323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc01595d0 msgr2=0x7fefc016a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:26.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.354+0000 7fefbffff700 1 --2- 172.21.15.89:0/3278594323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc01595d0 0x7fefc016a4a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fefa80030c0 tx=0x7fefa8016000 comp rx=0 tx=0).stop 2023-10-10T20:36:26.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.354+0000 7fefbffff700 1 -- 172.21.15.89:0/3278594323 shutdown_connections 2023-10-10T20:36:26.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.354+0000 7fefbffff700 1 --2- 172.21.15.89:0/3278594323 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fefa0066fc0 0x7fefa0069470 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:26.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.354+0000 7fefbffff700 1 --2- 172.21.15.89:0/3278594323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fefc015d760 0x7fefc0152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:26.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.354+0000 7fefbffff700 1 --2- 172.21.15.89:0/3278594323 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fefc00cf480 0x7fefc0154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:26.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.354+0000 7fefbffff700 1 --2- 172.21.15.89:0/3278594323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc01595d0 0x7fefc016a4a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:26.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.354+0000 7fefbffff700 1 -- 172.21.15.89:0/3278594323 >> 172.21.15.89:0/3278594323 conn(0x7fefc00d05f0 msgr2=0x7fefc015a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:26.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.354+0000 7fefbffff700 1 -- 172.21.15.89:0/3278594323 shutdown_connections 2023-10-10T20:36:26.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.354+0000 7fefbffff700 1 -- 172.21.15.89:0/3278594323 wait complete. 2023-10-10T20:36:26.362 INFO:teuthology.orchestra.run.smithi089.stderr:updated caps for client.0 2023-10-10T20:36:26.368 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:36:26.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.481+0000 7f73da707700 1 Processor -- start 2023-10-10T20:36:26.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.481+0000 7f73da707700 1 -- start start 2023-10-10T20:36:26.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.482+0000 7f73da707700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f73d41597f0 0x7f73d40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:26.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.482+0000 7f73da707700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d40b11d0 con 0x7f73d4159420 2023-10-10T20:36:26.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.482+0000 7f73da707700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d40b14b0 con 0x7f73d415d230 2023-10-10T20:36:26.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.482+0000 7f73da707700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d40b1790 con 0x7f73d41597f0 2023-10-10T20:36:26.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.482+0000 7f73d9705700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f73d4159420 0x7f73d40b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56646/0 (socket says 172.21.15.89:56646) 2023-10-10T20:36:26.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.482+0000 7f73d9705700 1 -- 172.21.15.89:0/2675498628 learned_addr learned my addr 172.21.15.89:0/2675498628 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:26.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.482+0000 7f73d8f04700 1 --2- 172.21.15.89:0/2675498628 >> v2:172.21.15.111:3300/0 conn(0x7f73d41597f0 0x7f73d40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:26.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.482+0000 7f73cbfff700 1 -- 172.21.15.89:0/2675498628 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 337264298 0 0) 0x7f73d40b11d0 con 0x7f73d4159420 2023-10-10T20:36:26.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.482+0000 7f73cbfff700 1 -- 172.21.15.89:0/2675498628 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73c0003570 con 0x7f73d4159420 2023-10-10T20:36:26.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73cbfff700 1 -- 172.21.15.89:0/2675498628 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2268862200 0 0) 0x7f73c0003570 con 0x7f73d4159420 2023-10-10T20:36:26.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73cbfff700 1 -- 172.21.15.89:0/2675498628 >> v2:172.21.15.111:3300/0 conn(0x7f73d41597f0 msgr2=0x7f73d40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:26.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73cbfff700 1 --2- 172.21.15.89:0/2675498628 >> v2:172.21.15.111:3300/0 conn(0x7f73d41597f0 0x7f73d40b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:26.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73cbfff700 1 -- 172.21.15.89:0/2675498628 >> v1:172.21.15.111:6789/0 conn(0x7f73d415d230 legacy=0x7f73d415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:26.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73cbfff700 1 -- 172.21.15.89:0/2675498628 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d40b1a70 con 0x7f73d4159420 2023-10-10T20:36:26.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73cbfff700 1 -- 172.21.15.89:0/2675498628 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f73d0001470 con 0x7f73d4159420 2023-10-10T20:36:26.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73cbfff700 1 -- 172.21.15.89:0/2675498628 >> v1:172.21.15.89:6789/0 conn(0x7f73d4159420 legacy=0x7f73d40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:26.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73cbfff700 1 --2- 172.21.15.89:0/2675498628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73c0003a60 0x7f73c0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:26.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73cbfff700 1 --2- 172.21.15.89:0/2675498628 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73c0004810 0x7f73c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:26.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73cbfff700 1 --2- 172.21.15.89:0/2675498628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73c00051c0 0x7f73c000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:26.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73cbfff700 1 -- 172.21.15.89:0/2675498628 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d40b1790 con 0x7f73c00051c0 2023-10-10T20:36:26.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73cbfff700 1 -- 172.21.15.89:0/2675498628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c000c0d0 con 0x7f73c0003a60 2023-10-10T20:36:26.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73cbfff700 1 -- 172.21.15.89:0/2675498628 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c000c290 con 0x7f73c0004810 2023-10-10T20:36:26.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73d9705700 1 --2- 172.21.15.89:0/2675498628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73c0003a60 0x7f73c0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:26.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73d9f06700 1 --2- 172.21.15.89:0/2675498628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73c00051c0 0x7f73c000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:26.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73d8f04700 1 --2- 172.21.15.89:0/2675498628 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73c0004810 0x7f73c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:26.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73d9f06700 1 -- 172.21.15.89:0/2675498628 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73c0004810 msgr2=0x7f73c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:26.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73d9f06700 1 --2- 172.21.15.89:0/2675498628 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73c0004810 0x7f73c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:26.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73d9f06700 1 -- 172.21.15.89:0/2675498628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73c0003a60 msgr2=0x7f73c0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:26.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73d9f06700 1 --2- 172.21.15.89:0/2675498628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73c0003a60 0x7f73c0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:26.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73d9f06700 1 -- 172.21.15.89:0/2675498628 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c0003570 con 0x7f73c00051c0 2023-10-10T20:36:26.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73d9f06700 1 -- 172.21.15.89:0/2675498628 reap_dead start 2023-10-10T20:36:26.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73d9f06700 1 --2- 172.21.15.89:0/2675498628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73c00051c0 0x7f73c000ba60 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f73cc001f90 tx=0x7f73cc001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26019eb1d0e7945 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:26.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73cbfff700 1 -- 172.21.15.89:0/2675498628 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc001950 con 0x7f73c00051c0 2023-10-10T20:36:26.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73cbfff700 1 -- 172.21.15.89:0/2675498628 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73cc00c8b0 con 0x7f73c00051c0 2023-10-10T20:36:26.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.483+0000 7f73cbfff700 1 -- 172.21.15.89:0/2675498628 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc00cc50 con 0x7f73c00051c0 2023-10-10T20:36:26.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.484+0000 7f73da707700 1 -- 172.21.15.89:0/2675498628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73c00051c0 msgr2=0x7f73c000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:26.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.484+0000 7f73da707700 1 --2- 172.21.15.89:0/2675498628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73c00051c0 0x7f73c000ba60 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f73cc001f90 tx=0x7f73cc001640 comp rx=0 tx=0).stop 2023-10-10T20:36:26.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.484+0000 7f73da707700 1 -- 172.21.15.89:0/2675498628 shutdown_connections 2023-10-10T20:36:26.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.484+0000 7f73da707700 1 --2- 172.21.15.89:0/2675498628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73c00051c0 0x7f73c000ba60 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:26.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.484+0000 7f73da707700 1 -- 172.21.15.89:0/2675498628 >> 172.21.15.89:0/2675498628 conn(0x7f73d40d0660 msgr2=0x7f73d40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:26.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.484+0000 7f73da707700 1 -- 172.21.15.89:0/2675498628 shutdown_connections 2023-10-10T20:36:26.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.484+0000 7f73da707700 1 -- 172.21.15.89:0/2675498628 wait complete. 2023-10-10T20:36:26.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.485+0000 7f73da707700 1 Processor -- start 2023-10-10T20:36:26.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.485+0000 7f73da707700 1 -- start start 2023-10-10T20:36:26.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.486+0000 7f73da707700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73c00051c0 0x7f73d4153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:26.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.486+0000 7f73da707700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73d41538b0 0x7f73d4155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:26.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.486+0000 7f73da707700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73d416a220 0x7f73d4156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:26.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.486+0000 7f73da707700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d416cd80 con 0x7f73c00051c0 2023-10-10T20:36:26.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.486+0000 7f73da707700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d416d8c0 con 0x7f73d416a220 2023-10-10T20:36:26.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.486+0000 7f73da707700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d41540d0 con 0x7f73d41538b0 2023-10-10T20:36:26.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.486+0000 7f73d9705700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73c00051c0 0x7f73d4153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:26.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.486+0000 7f73d9705700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73c00051c0 0x7f73d4153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46876/0 (socket says 172.21.15.89:46876) 2023-10-10T20:36:26.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.486+0000 7f73d9705700 1 -- 172.21.15.89:0/1760696213 learned_addr learned my addr 172.21.15.89:0/1760696213 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:26.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.486+0000 7f73d8f04700 1 --2- 172.21.15.89:0/1760696213 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73d41538b0 0x7f73d4155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:26.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.486+0000 7f73d9f06700 1 --2- 172.21.15.89:0/1760696213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73d416a220 0x7f73d4156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:26.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.486+0000 7f73d9705700 1 -- 172.21.15.89:0/1760696213 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73d41538b0 msgr2=0x7f73d4155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:26.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.486+0000 7f73d9705700 1 --2- 172.21.15.89:0/1760696213 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73d41538b0 0x7f73d4155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:26.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.486+0000 7f73d9705700 1 -- 172.21.15.89:0/1760696213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73d416a220 msgr2=0x7f73d4156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:26.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.486+0000 7f73d9705700 1 --2- 172.21.15.89:0/1760696213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73d416a220 0x7f73d4156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:26.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.486+0000 7f73d9705700 1 -- 172.21.15.89:0/1760696213 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d4153690 con 0x7f73c00051c0 2023-10-10T20:36:26.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.486+0000 7f73d9f06700 1 --2- 172.21.15.89:0/1760696213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73d416a220 0x7f73d4156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:26.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.486+0000 7f73d9705700 1 --2- 172.21.15.89:0/1760696213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73c00051c0 0x7f73d4153150 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f73d0007390 tx=0x7f73d0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:26.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.487+0000 7f73c97fa700 1 -- 172.21.15.89:0/1760696213 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d0018030 con 0x7f73c00051c0 2023-10-10T20:36:26.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.487+0000 7f73c97fa700 1 -- 172.21.15.89:0/1760696213 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d000f8d0 con 0x7f73c00051c0 2023-10-10T20:36:26.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.487+0000 7f73da707700 1 -- 172.21.15.89:0/1760696213 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d416cf90 con 0x7f73c00051c0 2023-10-10T20:36:26.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.487+0000 7f73c97fa700 1 -- 172.21.15.89:0/1760696213 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d000fc70 con 0x7f73c00051c0 2023-10-10T20:36:26.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.487+0000 7f73da707700 1 -- 172.21.15.89:0/1760696213 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d416d550 con 0x7f73c00051c0 2023-10-10T20:36:26.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.488+0000 7f73da707700 1 -- 172.21.15.89:0/1760696213 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d416dba0 con 0x7f73c00051c0 2023-10-10T20:36:26.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.488+0000 7f73c97fa700 1 -- 172.21.15.89:0/1760696213 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f73d001b070 con 0x7f73c00051c0 2023-10-10T20:36:26.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.488+0000 7f73c97fa700 1 --2- 172.21.15.89:0/1760696213 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f73b8066fc0 0x7f73b8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:26.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.488+0000 7f73c97fa700 1 -- 172.21.15.89:0/1760696213 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f73d0019070 con 0x7f73c00051c0 2023-10-10T20:36:26.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.488+0000 7f73d8f04700 1 --2- 172.21.15.89:0/1760696213 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f73b8066fc0 0x7f73b8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:26.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.489+0000 7f73d8f04700 1 --2- 172.21.15.89:0/1760696213 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f73b8066fc0 0x7f73b8069470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f73d40687e0 tx=0x7f73c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:26.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.492+0000 7f73c97fa700 1 -- 172.21.15.89:0/1760696213 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73d00516f0 con 0x7f73c00051c0 2023-10-10T20:36:26.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.718+0000 7f73da707700 1 -- 172.21.15.89:0/1760696213 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73d404c3c0 con 0x7f73c00051c0 2023-10-10T20:36:26.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.718+0000 7f73c97fa700 1 -- 172.21.15.89:0/1760696213 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f73d0054d10 con 0x7f73c00051c0 2023-10-10T20:36:26.721 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:36:26.721 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/3362909084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":3362909084},{"type":"v1","addr":"172.21.15.111:6835","nonce":3362909084}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5102,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1472423656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1472423656},{"type":"v1","addr":"172.21.15.111:6834","nonce":1472423656}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/3183325104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3183325104},{"type":"v1","addr":"172.21.15.89:6836","nonce":3183325104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:36:20.867329+0000","modified":"2023-10-10T20:36:24.518115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/862652054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":862652054},{"type":"v1","addr":"172.21.15.89:6837","nonce":862652054}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-10T20:36:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.720+0000 7f73da707700 1 -- 172.21.15.89:0/1760696213 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f73b8066fc0 msgr2=0x7f73b8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.720+0000 7f73da707700 1 --2- 172.21.15.89:0/1760696213 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f73b8066fc0 0x7f73b8069470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f73d40687e0 tx=0x7f73c4008040 comp rx=0 tx=0).stop 2023-10-10T20:36:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.720+0000 7f73da707700 1 -- 172.21.15.89:0/1760696213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73c00051c0 msgr2=0x7f73d4153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.720+0000 7f73da707700 1 --2- 172.21.15.89:0/1760696213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73c00051c0 0x7f73d4153150 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f73d0007390 tx=0x7f73d0001e50 comp rx=0 tx=0).stop 2023-10-10T20:36:26.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.720+0000 7f73da707700 1 -- 172.21.15.89:0/1760696213 shutdown_connections 2023-10-10T20:36:26.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.720+0000 7f73da707700 1 --2- 172.21.15.89:0/1760696213 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f73b8066fc0 0x7f73b8069470 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:26.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.720+0000 7f73da707700 1 --2- 172.21.15.89:0/1760696213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73d416a220 0x7f73d4156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:26.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.720+0000 7f73da707700 1 --2- 172.21.15.89:0/1760696213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73c00051c0 0x7f73d4153150 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:26.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.720+0000 7f73da707700 1 --2- 172.21.15.89:0/1760696213 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73d41538b0 0x7f73d4155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:26.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.720+0000 7f73da707700 1 -- 172.21.15.89:0/1760696213 >> 172.21.15.89:0/1760696213 conn(0x7f73d40d0660 msgr2=0x7f73d40d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:26.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.721+0000 7f73da707700 1 -- 172.21.15.89:0/1760696213 shutdown_connections 2023-10-10T20:36:26.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:26.721+0000 7f73da707700 1 -- 172.21.15.89:0/1760696213 wait complete. 2023-10-10T20:36:26.725 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 37 2023-10-10T20:36:26.796 INFO:tasks.ceph.mds.c.smithi089.stderr:2023-10-10T20:36:26.794+0000 7f60b7bbb700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:36:27.138 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:36:20.867329+0000', 'modified': '2023-10-10T20:36:24.518115+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5697}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5697': {'gid': 5697, 'name': 'c', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6837/862652054', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6835', 'nonce': 862652054}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 862652054}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:36:27.138 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:36:27.139 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-10T20:36:27.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.252+0000 7fb7a973f700 1 Processor -- start 2023-10-10T20:36:27.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.252+0000 7fb7a973f700 1 -- start start 2023-10-10T20:36:27.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.253+0000 7fb7a973f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb7a415d230 0x7fb7a415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.253+0000 7fb7a973f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a40b2980 con 0x7fb7a4159420 2023-10-10T20:36:27.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.253+0000 7fb7a973f700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a415dbb0 con 0x7fb7a41597f0 2023-10-10T20:36:27.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.253+0000 7fb7a973f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a40b11d0 con 0x7fb7a415d230 2023-10-10T20:36:27.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.253+0000 7fb7a3fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb7a4159420 0x7fb7a40b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56652/0 (socket says 172.21.15.89:56652) 2023-10-10T20:36:27.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.253+0000 7fb7a3fff700 1 -- 172.21.15.89:0/4240445645 learned_addr learned my addr 172.21.15.89:0/4240445645 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:27.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.253+0000 7fb7a2ffd700 1 -- 172.21.15.89:0/4240445645 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992566767 0 0) 0x7fb7a40b2980 con 0x7fb7a4159420 2023-10-10T20:36:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.253+0000 7fb7a8f3e700 1 --2- 172.21.15.89:0/4240445645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb7a415d230 0x7fb7a415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.253+0000 7fb7a2ffd700 1 -- 172.21.15.89:0/4240445645 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb78c003570 con 0x7fb7a4159420 2023-10-10T20:36:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.253+0000 7fb7a2ffd700 1 -- 172.21.15.89:0/4240445645 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2720874032 0 0) 0x7fb78c003570 con 0x7fb7a4159420 2023-10-10T20:36:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.253+0000 7fb7a2ffd700 1 -- 172.21.15.89:0/4240445645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb7a415d230 msgr2=0x7fb7a415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a2ffd700 1 --2- 172.21.15.89:0/4240445645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb7a415d230 0x7fb7a415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a2ffd700 1 -- 172.21.15.89:0/4240445645 >> v1:172.21.15.111:6789/0 conn(0x7fb7a41597f0 legacy=0x7fb7a40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a2ffd700 1 -- 172.21.15.89:0/4240445645 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a40b14b0 con 0x7fb7a4159420 2023-10-10T20:36:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a2ffd700 1 -- 172.21.15.89:0/4240445645 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fb794001470 con 0x7fb7a4159420 2023-10-10T20:36:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a2ffd700 1 -- 172.21.15.89:0/4240445645 >> v1:172.21.15.89:6789/0 conn(0x7fb7a4159420 legacy=0x7fb7a40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a2ffd700 1 --2- 172.21.15.89:0/4240445645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb78c003a90 0x7fb78c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a2ffd700 1 --2- 172.21.15.89:0/4240445645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb78c004810 0x7fb78c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a2ffd700 1 --2- 172.21.15.89:0/4240445645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb78c005210 0x7fb78c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a2ffd700 1 -- 172.21.15.89:0/4240445645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a40b11d0 con 0x7fb78c005210 2023-10-10T20:36:27.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a2ffd700 1 -- 172.21.15.89:0/4240445645 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c00c0e0 con 0x7fb78c004810 2023-10-10T20:36:27.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a2ffd700 1 -- 172.21.15.89:0/4240445645 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c00c2a0 con 0x7fb78c003a90 2023-10-10T20:36:27.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a8f3e700 1 --2- 172.21.15.89:0/4240445645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb7a415d230 0x7fb7a415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:27.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a8f3e700 1 --2- 172.21.15.89:0/4240445645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb78c005210 0x7fb78c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a3fff700 1 --2- 172.21.15.89:0/4240445645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb78c003a90 0x7fb78c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a37fe700 1 --2- 172.21.15.89:0/4240445645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb78c004810 0x7fb78c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a8f3e700 1 -- 172.21.15.89:0/4240445645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb78c003a90 msgr2=0x7fb78c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:27.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a8f3e700 1 --2- 172.21.15.89:0/4240445645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb78c003a90 0x7fb78c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a8f3e700 1 -- 172.21.15.89:0/4240445645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb78c004810 msgr2=0x7fb78c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:27.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a8f3e700 1 --2- 172.21.15.89:0/4240445645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb78c004810 0x7fb78c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a8f3e700 1 -- 172.21.15.89:0/4240445645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78c003570 con 0x7fb78c005210 2023-10-10T20:36:27.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a8f3e700 1 --2- 172.21.15.89:0/4240445645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb78c005210 0x7fb78c00ba60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb7a415d600 tx=0x7fb79c0097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1397aea848390bf5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:27.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a2ffd700 1 -- 172.21.15.89:0/4240445645 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c007bc0 con 0x7fb78c005210 2023-10-10T20:36:27.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a2ffd700 1 -- 172.21.15.89:0/4240445645 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79c00f040 con 0x7fb78c005210 2023-10-10T20:36:27.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.254+0000 7fb7a2ffd700 1 -- 172.21.15.89:0/4240445645 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c0133f0 con 0x7fb78c005210 2023-10-10T20:36:27.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.255+0000 7fb7a973f700 1 -- 172.21.15.89:0/4240445645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb78c005210 msgr2=0x7fb78c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.255+0000 7fb7a973f700 1 --2- 172.21.15.89:0/4240445645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb78c005210 0x7fb78c00ba60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb7a415d600 tx=0x7fb79c0097b0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.255+0000 7fb7a8f3e700 1 -- 172.21.15.89:0/4240445645 reap_dead start 2023-10-10T20:36:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.255+0000 7fb7a973f700 1 -- 172.21.15.89:0/4240445645 shutdown_connections 2023-10-10T20:36:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.255+0000 7fb7a973f700 1 -- 172.21.15.89:0/4240445645 >> 172.21.15.89:0/4240445645 conn(0x7fb7a40d0660 msgr2=0x7fb7a40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.255+0000 7fb7a973f700 1 -- 172.21.15.89:0/4240445645 shutdown_connections 2023-10-10T20:36:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.255+0000 7fb7a973f700 1 -- 172.21.15.89:0/4240445645 wait complete. 2023-10-10T20:36:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.256+0000 7fb7a973f700 1 Processor -- start 2023-10-10T20:36:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.256+0000 7fb7a973f700 1 -- start start 2023-10-10T20:36:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.257+0000 7fb7a973f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb78c005210 0x7fb7a413a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.257+0000 7fb7a973f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb7a40b4e20 0x7fb7a40b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.257+0000 7fb7a973f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7a414dc10 0x7fb7a414e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.257+0000 7fb7a973f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4159ca0 con 0x7fb78c005210 2023-10-10T20:36:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.257+0000 7fb7a973f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a40b57d0 con 0x7fb7a414dc10 2023-10-10T20:36:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.257+0000 7fb7a973f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a415a220 con 0x7fb7a40b4e20 2023-10-10T20:36:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.257+0000 7fb7a3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb78c005210 0x7fb7a413a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.257+0000 7fb7a3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb78c005210 0x7fb7a413a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46892/0 (socket says 172.21.15.89:46892) 2023-10-10T20:36:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.257+0000 7fb7a3fff700 1 -- 172.21.15.89:0/1612900528 learned_addr learned my addr 172.21.15.89:0/1612900528 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.257+0000 7fb7a37fe700 1 --2- 172.21.15.89:0/1612900528 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb7a40b4e20 0x7fb7a40b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.257+0000 7fb7a8f3e700 1 --2- 172.21.15.89:0/1612900528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7a414dc10 0x7fb7a414e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.257+0000 7fb7a3fff700 1 -- 172.21.15.89:0/1612900528 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb7a40b4e20 msgr2=0x7fb7a40b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:27.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.257+0000 7fb7a3fff700 1 --2- 172.21.15.89:0/1612900528 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb7a40b4e20 0x7fb7a40b5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.257+0000 7fb7a3fff700 1 -- 172.21.15.89:0/1612900528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7a414dc10 msgr2=0x7fb7a414e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:27.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.257+0000 7fb7a3fff700 1 --2- 172.21.15.89:0/1612900528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7a414dc10 0x7fb7a414e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.257+0000 7fb7a3fff700 1 -- 172.21.15.89:0/1612900528 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a414ea00 con 0x7fb78c005210 2023-10-10T20:36:27.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.257+0000 7fb7a3fff700 1 --2- 172.21.15.89:0/1612900528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb78c005210 0x7fb7a413a140 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb794007370 tx=0x7fb794001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:27.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.258+0000 7fb78bfff700 1 -- 172.21.15.89:0/1612900528 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794018030 con 0x7fb78c005210 2023-10-10T20:36:27.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.258+0000 7fb78bfff700 1 -- 172.21.15.89:0/1612900528 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb794007840 con 0x7fb78c005210 2023-10-10T20:36:27.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.258+0000 7fb7a973f700 1 -- 172.21.15.89:0/1612900528 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a414ece0 con 0x7fb78c005210 2023-10-10T20:36:27.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.258+0000 7fb78bfff700 1 -- 172.21.15.89:0/1612900528 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79400fb00 con 0x7fb78c005210 2023-10-10T20:36:27.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.258+0000 7fb7a973f700 1 -- 172.21.15.89:0/1612900528 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a413a680 con 0x7fb78c005210 2023-10-10T20:36:27.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.259+0000 7fb78bfff700 1 -- 172.21.15.89:0/1612900528 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fb79401b070 con 0x7fb78c005210 2023-10-10T20:36:27.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.259+0000 7fb78bfff700 1 --2- 172.21.15.89:0/1612900528 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb790066fc0 0x7fb790069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.259+0000 7fb78bfff700 1 -- 172.21.15.89:0/1612900528 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb794019070 con 0x7fb78c005210 2023-10-10T20:36:27.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.260+0000 7fb7a37fe700 1 --2- 172.21.15.89:0/1612900528 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb790066fc0 0x7fb790069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.260+0000 7fb7a37fe700 1 --2- 172.21.15.89:0/1612900528 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb790066fc0 0x7fb790069470 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb798007e40 tx=0x7fb798009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:27.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.261+0000 7fb7b3c9ab80 1 Processor -- start 2023-10-10T20:36:27.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.261+0000 7fb7b3c9ab80 1 -- start start 2023-10-10T20:36:27.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.262+0000 7fb7b3c9ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c40567480 0x561c405467d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.262+0000 7fb7b3c9ab80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c4051a5f0 0x561c40567850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.262+0000 7fb7b3c9ab80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c4051cb80 0x561c4051cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.262+0000 7fb7b3c9ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c40473a70 con 0x561c40567480 2023-10-10T20:36:27.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.262+0000 7fb7b3c9ab80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c40471270 con 0x561c4051a5f0 2023-10-10T20:36:27.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.262+0000 7fb7b3c9ab80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c4046d060 con 0x561c4051cb80 2023-10-10T20:36:27.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.262+0000 7fb7a3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c40567480 0x561c405467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.262+0000 7fb7a3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c40567480 0x561c405467d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46898/0 (socket says 172.21.15.89:46898) 2023-10-10T20:36:27.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.262+0000 7fb7a3fff700 1 -- 172.21.15.89:0/4276638070 learned_addr learned my addr 172.21.15.89:0/4276638070 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.262+0000 7fb7a37fe700 1 --2- 172.21.15.89:0/4276638070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c4051a5f0 0x561c40567850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.262+0000 7fb7a8f3e700 1 --2- 172.21.15.89:0/4276638070 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c4051cb80 0x561c4051cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.263+0000 7fb7a3fff700 1 -- 172.21.15.89:0/4276638070 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c4051cb80 msgr2=0x561c4051cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.263+0000 7fb7a3fff700 1 --2- 172.21.15.89:0/4276638070 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c4051cb80 0x561c4051cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.263+0000 7fb7a3fff700 1 -- 172.21.15.89:0/4276638070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c4051a5f0 msgr2=0x561c40567850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.263+0000 7fb7a3fff700 1 --2- 172.21.15.89:0/4276638070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c4051a5f0 0x561c40567850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.263+0000 7fb7a3fff700 1 -- 172.21.15.89:0/4276638070 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c40307270 con 0x561c40567480 2023-10-10T20:36:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.263+0000 7fb7a37fe700 1 --2- 172.21.15.89:0/4276638070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c4051a5f0 0x561c40567850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.263+0000 7fb7a3fff700 1 --2- 172.21.15.89:0/4276638070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c40567480 0x561c405467d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb794082ad0 tx=0x7fb794006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a880c8fa1c14c04c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.263+0000 7fb7897fa700 1 -- 172.21.15.89:0/4276638070 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794055c00 con 0x561c40567480 2023-10-10T20:36:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.263+0000 7fb7897fa700 1 -- 172.21.15.89:0/4276638070 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7940554a0 con 0x561c40567480 2023-10-10T20:36:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.263+0000 7fb7897fa700 1 -- 172.21.15.89:0/4276638070 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7940586f0 con 0x561c40567480 2023-10-10T20:36:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.263+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/4276638070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c40567480 msgr2=0x561c405467d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.263+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/4276638070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c40567480 0x561c405467d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb794082ad0 tx=0x7fb794006040 comp rx=0 tx=0).stop 2023-10-10T20:36:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.264+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/4276638070 shutdown_connections 2023-10-10T20:36:27.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.264+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/4276638070 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c4051cb80 0x561c4051cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.264+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/4276638070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c40567480 0x561c405467d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.264+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/4276638070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c4051a5f0 0x561c40567850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.264+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/4276638070 >> 172.21.15.89:0/4276638070 conn(0x561c4051e0c0 msgr2=0x561c4051e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:27.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.264+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/4276638070 shutdown_connections 2023-10-10T20:36:27.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.264+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/4276638070 wait complete. 2023-10-10T20:36:27.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.264+0000 7fb7b3c9ab80 1 -- start start 2023-10-10T20:36:27.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.264+0000 7fb7b3c9ab80 1 Processor -- start 2023-10-10T20:36:27.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.265+0000 7fb7b3c9ab80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c4051a5f0 0x561c40540d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.265+0000 7fb7b3c9ab80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c4051cb80 0x561c40545640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.265+0000 7fb7b3c9ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c40567480 0x561c405631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.265+0000 7fb7b3c9ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c40418890 con 0x561c40567480 2023-10-10T20:36:27.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.265+0000 7fb7b3c9ab80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c40433ea0 con 0x561c4051cb80 2023-10-10T20:36:27.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.265+0000 7fb7b3c9ab80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c402efe30 con 0x561c4051a5f0 2023-10-10T20:36:27.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.265+0000 7fb7b3c9ab80 10 client.0 _ll_register_callbacks cb 0x561c404e02c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:36:27.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.265+0000 7fb7a8f3e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c40567480 0x561c405631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.265+0000 7fb7a8f3e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c40567480 0x561c405631d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46904/0 (socket says 172.21.15.89:46904) 2023-10-10T20:36:27.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.265+0000 7fb7a8f3e700 1 -- 172.21.15.89:0/3183078703 learned_addr learned my addr 172.21.15.89:0/3183078703 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:27.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.265+0000 7fb7a3fff700 1 --2- 172.21.15.89:0/3183078703 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c4051a5f0 0x561c40540d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.265+0000 7fb7a37fe700 1 --2- 172.21.15.89:0/3183078703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c4051cb80 0x561c40545640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.265+0000 7fb7a8f3e700 1 -- 172.21.15.89:0/3183078703 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c4051a5f0 msgr2=0x561c40540d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:27.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.265+0000 7fb7a8f3e700 1 --2- 172.21.15.89:0/3183078703 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c4051a5f0 0x561c40540d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.265+0000 7fb7a8f3e700 1 -- 172.21.15.89:0/3183078703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c4051cb80 msgr2=0x561c40545640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:27.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.265+0000 7fb7a8f3e700 1 --2- 172.21.15.89:0/3183078703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c4051cb80 0x561c40545640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.265+0000 7fb7a8f3e700 1 -- 172.21.15.89:0/3183078703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a413a680 con 0x561c40567480 2023-10-10T20:36:27.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.265+0000 7fb7a37fe700 1 --2- 172.21.15.89:0/3183078703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c4051cb80 0x561c40545640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:27.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.265+0000 7fb7a8f3e700 1 --2- 172.21.15.89:0/3183078703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c40567480 0x561c405631d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb79c007900 tx=0x7fb79c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161d34ecd686ebd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:27.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.266+0000 7fb7827fc700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:36:27.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.266+0000 7fb7827fc700 1 -- 172.21.15.89:0/3183078703 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c0133f0 con 0x561c40567480 2023-10-10T20:36:27.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.266+0000 7fb7827fc700 1 -- 172.21.15.89:0/3183078703 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb79c013970 con 0x561c40567480 2023-10-10T20:36:27.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.266+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3183078703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c40511cd0 con 0x561c40567480 2023-10-10T20:36:27.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.266+0000 7fb7827fc700 1 -- 172.21.15.89:0/3183078703 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb79c009aa0 con 0x561c40567480 2023-10-10T20:36:27.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.266+0000 7fb7b3c9ab80 10 client.5847 fetch_fsmap learned FSMap version 37 2023-10-10T20:36:27.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.266+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3183078703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x561c402a9fe0 con 0x561c40567480 2023-10-10T20:36:27.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.266+0000 7fb7827fc700 1 -- 172.21.15.89:0/3183078703 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561c40511cd0 con 0x561c40567480 2023-10-10T20:36:27.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.266+0000 7fb7827fc700 1 -- 172.21.15.89:0/3183078703 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 37) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fb79c007340 con 0x561c40567480 2023-10-10T20:36:27.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.266+0000 7fb7b3c9ab80 10 client.5847 fetch_fsmap finished waiting for FSMap version 37 2023-10-10T20:36:27.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.266+0000 7fb7b3c9ab80 10 client.5847 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-10T20:36:27.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.266+0000 7fb7b3c9ab80 20 client.5847 populate_metadata read hostname 'smithi089' 2023-10-10T20:36:27.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.266+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/3183078703 >> [v2:172.21.15.89:6835/862652054,v1:172.21.15.89:6837/862652054] conn(0x561c40566200 0x561c405665d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.266+0000 7fb7b3c9ab80 4 client.5847 mds_command: new command op to 5697 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:36:27.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.266+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3183078703 --> [v2:172.21.15.89:6835/862652054,v1:172.21.15.89:6837/862652054] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c404eeaa0 con 0x561c40566200 2023-10-10T20:36:27.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.266+0000 7fb7a3fff700 1 --2- 172.21.15.89:0/3183078703 >> [v2:172.21.15.89:6835/862652054,v1:172.21.15.89:6837/862652054] conn(0x561c40566200 0x561c405665d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.266+0000 7fb7827fc700 10 client.5847 ms_handle_connect on v2:172.21.15.89:6835/862652054 2023-10-10T20:36:27.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.266+0000 7fb7a3fff700 1 --2- 172.21.15.89:0/3183078703 >> [v2:172.21.15.89:6835/862652054,v1:172.21.15.89:6837/862652054] conn(0x561c40566200 0x561c405665d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f29e846c3f9997de server_cookie=bfdc96ddd6fa082a in_seq=0 out_seq=0 2023-10-10T20:36:27.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.267+0000 7fb7827fc700 1 -- 172.21.15.89:0/3183078703 <== mds.0 v2:172.21.15.89:6835/862652054 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x561c404eeaa0 con 0x561c40566200 2023-10-10T20:36:27.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.267+0000 7fb7827fc700 10 client.5847 handle_command_reply: tid=0 2023-10-10T20:36:27.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.267+0000 7fb7b3c9ab80 1 client.5847 shutdown 2023-10-10T20:36:27.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.267+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3183078703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c40567480 msgr2=0x561c405631d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:27.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.267+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/3183078703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c40567480 0x561c405631d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb79c007900 tx=0x7fb79c017000 comp rx=0 tx=0).stop 2023-10-10T20:36:27.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.267+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3183078703 shutdown_connections 2023-10-10T20:36:27.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.267+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/3183078703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c40567480 0x561c405631d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.267+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/3183078703 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c4051a5f0 0x561c40540d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.267+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/3183078703 >> [v2:172.21.15.89:6835/862652054,v1:172.21.15.89:6837/862652054] conn(0x561c40566200 0x561c405665d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.267+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/3183078703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c4051cb80 0x561c40545640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.267+0000 7fb7827fc700 0 client.5847 ms_handle_reset on v2:172.21.15.89:6835/862652054 2023-10-10T20:36:27.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.267+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3183078703 >> 172.21.15.89:0/3183078703 conn(0x561c4051e0c0 msgr2=0x561c4051cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:27.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.267+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3183078703 shutdown_connections 2023-10-10T20:36:27.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.267+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3183078703 wait complete. 2023-10-10T20:36:27.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.267+0000 7fb7b3c9ab80 20 client.5847 trim_cache size 0 max 16384 2023-10-10T20:36:27.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.284+0000 7fb7b3c9ab80 1 Processor -- start 2023-10-10T20:36:27.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.284+0000 7fb7b3c9ab80 1 -- start start 2023-10-10T20:36:27.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb7b3c9ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c4051a5f0 0x561c4051cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb7b3c9ab80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c4051cb80 0x561c40519c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb7b3c9ab80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c40567480 0x561c40541f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb7b3c9ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c40549a40 con 0x561c4051a5f0 2023-10-10T20:36:27.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb7b3c9ab80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c4054b1e0 con 0x561c40567480 2023-10-10T20:36:27.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb7b3c9ab80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c404f5410 con 0x561c4051cb80 2023-10-10T20:36:27.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb7a3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c4051a5f0 0x561c4051cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb7a3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c4051a5f0 0x561c4051cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46906/0 (socket says 172.21.15.89:46906) 2023-10-10T20:36:27.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb7a3fff700 1 -- 172.21.15.89:0/3475452420 learned_addr learned my addr 172.21.15.89:0/3475452420 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:27.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb7a8f3e700 1 --2- 172.21.15.89:0/3475452420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c40567480 0x561c40541f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb7a37fe700 1 --2- 172.21.15.89:0/3475452420 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c4051cb80 0x561c40519c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb7a3fff700 1 -- 172.21.15.89:0/3475452420 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c4051cb80 msgr2=0x561c40519c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:27.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb7a3fff700 1 --2- 172.21.15.89:0/3475452420 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c4051cb80 0x561c40519c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb7a3fff700 1 -- 172.21.15.89:0/3475452420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c40567480 msgr2=0x561c40541f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:27.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb7a3fff700 1 --2- 172.21.15.89:0/3475452420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c40567480 0x561c40541f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb7a3fff700 1 -- 172.21.15.89:0/3475452420 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c4043b780 con 0x561c4051a5f0 2023-10-10T20:36:27.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb7a37fe700 1 --2- 172.21.15.89:0/3475452420 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c4051cb80 0x561c40519c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:27.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb7a3fff700 1 --2- 172.21.15.89:0/3475452420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c4051a5f0 0x561c4051cf50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb79400f4b0 tx=0x7fb794020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06c38222005061b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:27.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb789ffb700 1 -- 172.21.15.89:0/3475452420 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794055270 con 0x561c4051a5f0 2023-10-10T20:36:27.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb789ffb700 1 -- 172.21.15.89:0/3475452420 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79405c440 con 0x561c4051a5f0 2023-10-10T20:36:27.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.285+0000 7fb789ffb700 1 -- 172.21.15.89:0/3475452420 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7940586f0 con 0x561c4051a5f0 2023-10-10T20:36:27.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.286+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3475452420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c4051a5f0 msgr2=0x561c4051cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:27.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.286+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/3475452420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c4051a5f0 0x561c4051cf50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb79400f4b0 tx=0x7fb794020000 comp rx=0 tx=0).stop 2023-10-10T20:36:27.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.286+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3475452420 shutdown_connections 2023-10-10T20:36:27.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.286+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/3475452420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c40567480 0x561c40541f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.286+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/3475452420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c4051a5f0 0x561c4051cf50 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.286+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/3475452420 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c4051cb80 0x561c40519c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.286+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3475452420 >> 172.21.15.89:0/3475452420 conn(0x561c4051e0c0 msgr2=0x561c405665d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:27.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.286+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3475452420 shutdown_connections 2023-10-10T20:36:27.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.286+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3475452420 wait complete. 2023-10-10T20:36:27.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.287+0000 7fb7b3c9ab80 1 -- start start 2023-10-10T20:36:27.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.287+0000 7fb7b3c9ab80 1 Processor -- start 2023-10-10T20:36:27.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.287+0000 7fb7b3c9ab80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c4051a5f0 0x561c405aec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.287+0000 7fb7b3c9ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c4051cb80 0x561c40574580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.287+0000 7fb7b3c9ab80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c40567480 0x561c40546d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.287+0000 7fb7b3c9ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c405c9f40 con 0x561c4051cb80 2023-10-10T20:36:27.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.287+0000 7fb7b3c9ab80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c405c8d70 con 0x561c4051a5f0 2023-10-10T20:36:27.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.287+0000 7fb7b3c9ab80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c405c8fb0 con 0x561c40567480 2023-10-10T20:36:27.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.287+0000 7fb7b3c9ab80 10 client.0 _ll_register_callbacks cb 0x561c404e02c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:36:27.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7a37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c4051cb80 0x561c40574580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7a37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c4051cb80 0x561c40574580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46922/0 (socket says 172.21.15.89:46922) 2023-10-10T20:36:27.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7a37fe700 1 -- 172.21.15.89:0/3313063397 learned_addr learned my addr 172.21.15.89:0/3313063397 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:27.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7a3fff700 1 --2- 172.21.15.89:0/3313063397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c4051a5f0 0x561c405aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7a8f3e700 1 --2- 172.21.15.89:0/3313063397 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c40567480 0x561c40546d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7a37fe700 1 -- 172.21.15.89:0/3313063397 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c40567480 msgr2=0x561c40546d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:27.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7a37fe700 1 --2- 172.21.15.89:0/3313063397 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c40567480 0x561c40546d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7a37fe700 1 -- 172.21.15.89:0/3313063397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c4051a5f0 msgr2=0x561c405aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:27.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7a37fe700 1 --2- 172.21.15.89:0/3313063397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c4051a5f0 0x561c405aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7a37fe700 1 -- 172.21.15.89:0/3313063397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb79406f040 con 0x561c4051cb80 2023-10-10T20:36:27.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7a37fe700 1 --2- 172.21.15.89:0/3313063397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c4051cb80 0x561c40574580 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb7a41579a0 tx=0x7fb798017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eea6429e104e0b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:27.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7827fc700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:36:27.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7827fc700 1 -- 172.21.15.89:0/3313063397 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb798006820 con 0x561c4051cb80 2023-10-10T20:36:27.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7827fc700 1 -- 172.21.15.89:0/3313063397 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb79801a040 con 0x561c4051cb80 2023-10-10T20:36:27.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3313063397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7940554a0 con 0x561c4051cb80 2023-10-10T20:36:27.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7827fc700 1 -- 172.21.15.89:0/3313063397 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb798006b90 con 0x561c4051cb80 2023-10-10T20:36:27.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7b3c9ab80 10 client.5853 fetch_fsmap learned FSMap version 37 2023-10-10T20:36:27.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3313063397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x561c40422470 con 0x561c4051cb80 2023-10-10T20:36:27.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7827fc700 1 -- 172.21.15.89:0/3313063397 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7940554a0 con 0x561c4051cb80 2023-10-10T20:36:27.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7827fc700 1 -- 172.21.15.89:0/3313063397 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 37) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fb798006cf0 con 0x561c4051cb80 2023-10-10T20:36:27.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7b3c9ab80 10 client.5853 fetch_fsmap finished waiting for FSMap version 37 2023-10-10T20:36:27.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7b3c9ab80 10 client.5853 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-10T20:36:27.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7b3c9ab80 20 client.5853 populate_metadata read hostname 'smithi089' 2023-10-10T20:36:27.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.288+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/3313063397 >> [v2:172.21.15.89:6835/862652054,v1:172.21.15.89:6837/862652054] conn(0x561c405436f0 0x561c40543ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7b3c9ab80 4 client.5853 mds_command: new command op to 5697 tid=0[{"prefix": "status"}] 2023-10-10T20:36:27.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3313063397 --> [v2:172.21.15.89:6835/862652054,v1:172.21.15.89:6837/862652054] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561c4040fe00 con 0x561c405436f0 2023-10-10T20:36:27.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7a3fff700 1 --2- 172.21.15.89:0/3313063397 >> [v2:172.21.15.89:6835/862652054,v1:172.21.15.89:6837/862652054] conn(0x561c405436f0 0x561c40543ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7827fc700 10 client.5853 ms_handle_connect on v2:172.21.15.89:6835/862652054 2023-10-10T20:36:27.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7a3fff700 1 --2- 172.21.15.89:0/3313063397 >> [v2:172.21.15.89:6835/862652054,v1:172.21.15.89:6837/862652054] conn(0x561c405436f0 0x561c40543ac0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb9bd474f902183 server_cookie=c3e27f2b868f5e7e in_seq=0 out_seq=0 2023-10-10T20:36:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7827fc700 1 -- 172.21.15.89:0/3313063397 <== mds.0 v2:172.21.15.89:6835/862652054 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x561c4040fe00 con 0x561c405436f0 2023-10-10T20:36:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7827fc700 10 client.5853 handle_command_reply: tid=0 2023-10-10T20:36:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7b3c9ab80 1 client.5853 shutdown 2023-10-10T20:36:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3313063397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c4051cb80 msgr2=0x561c40574580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/3313063397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c4051cb80 0x561c40574580 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb7a41579a0 tx=0x7fb798017000 comp rx=0 tx=0).stop 2023-10-10T20:36:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3313063397 shutdown_connections 2023-10-10T20:36:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/3313063397 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561c40567480 0x561c40546d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/3313063397 >> [v2:172.21.15.89:6835/862652054,v1:172.21.15.89:6837/862652054] conn(0x561c405436f0 0x561c40543ac0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/3313063397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561c4051a5f0 0x561c405aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7827fc700 0 client.5853 ms_handle_reset on v2:172.21.15.89:6835/862652054 2023-10-10T20:36:27.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7b3c9ab80 1 --2- 172.21.15.89:0/3313063397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c4051cb80 0x561c40574580 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3313063397 >> 172.21.15.89:0/3313063397 conn(0x561c4051e0c0 msgr2=0x561c40562be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:27.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3313063397 shutdown_connections 2023-10-10T20:36:27.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7b3c9ab80 1 -- 172.21.15.89:0/3313063397 wait complete. 2023-10-10T20:36:27.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.289+0000 7fb7b3c9ab80 20 client.5853 trim_cache size 0 max 16384 2023-10-10T20:36:27.322 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:36:27.322 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "5dd675b8-9026-4d8a-a1bf-62fdc5e7de67", 2023-10-10T20:36:27.322 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-10-10T20:36:27.322 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 5697, 2023-10-10T20:36:27.323 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-10-10T20:36:27.323 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-10-10T20:36:27.323 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-10-10T20:36:27.323 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 5.7765602720000002, 2023-10-10T20:36:27.324 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 37, 2023-10-10T20:36:27.324 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 62, 2023-10-10T20:36:27.324 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 62, 2023-10-10T20:36:27.324 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 10.906487592 2023-10-10T20:36:27.324 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:36:27.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.290+0000 7fb7a973f700 1 -- 172.21.15.89:0/1612900528 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb790066fc0 msgr2=0x7fb790069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:27.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.290+0000 7fb7a973f700 1 --2- 172.21.15.89:0/1612900528 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb790066fc0 0x7fb790069470 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb798007e40 tx=0x7fb798009040 comp rx=0 tx=0).stop 2023-10-10T20:36:27.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.290+0000 7fb7a973f700 1 -- 172.21.15.89:0/1612900528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb78c005210 msgr2=0x7fb7a413a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:27.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.290+0000 7fb7a973f700 1 --2- 172.21.15.89:0/1612900528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb78c005210 0x7fb7a413a140 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb794007370 tx=0x7fb794001e50 comp rx=0 tx=0).stop 2023-10-10T20:36:27.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.290+0000 7fb7a973f700 1 -- 172.21.15.89:0/1612900528 shutdown_connections 2023-10-10T20:36:27.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.290+0000 7fb7a973f700 1 --2- 172.21.15.89:0/1612900528 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb790066fc0 0x7fb790069470 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.290+0000 7fb7a973f700 1 --2- 172.21.15.89:0/1612900528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7a414dc10 0x7fb7a414e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.290+0000 7fb7a973f700 1 --2- 172.21.15.89:0/1612900528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb78c005210 0x7fb7a413a140 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.290+0000 7fb7a973f700 1 --2- 172.21.15.89:0/1612900528 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb7a40b4e20 0x7fb7a40b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.290+0000 7fb7a973f700 1 -- 172.21.15.89:0/1612900528 >> 172.21.15.89:0/1612900528 conn(0x7fb7a40d0660 msgr2=0x7fb7a4156eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:27.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.290+0000 7fb7a973f700 1 -- 172.21.15.89:0/1612900528 shutdown_connections 2023-10-10T20:36:27.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.290+0000 7fb7a973f700 1 -- 172.21.15.89:0/1612900528 wait complete. 2023-10-10T20:36:27.329 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:36:27.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.441+0000 7f5f79666700 1 Processor -- start 2023-10-10T20:36:27.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.441+0000 7f5f79666700 1 -- start start 2023-10-10T20:36:27.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f79666700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f74159890 0x7f5f7415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f79666700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7415a160 con 0x7f5f74069560 2023-10-10T20:36:27.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f79666700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f740d13a0 con 0x7f5f74159c60 2023-10-10T20:36:27.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f79666700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7415e140 con 0x7f5f74159890 2023-10-10T20:36:27.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f737fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5f74069560 0x7f5f740ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56668/0 (socket says 172.21.15.89:56668) 2023-10-10T20:36:27.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f737fe700 1 -- 172.21.15.89:0/722445721 learned_addr learned my addr 172.21.15.89:0/722445721 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:27.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f72ffd700 1 --2- 172.21.15.89:0/722445721 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f74159890 0x7f5f7415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f71ffb700 1 -- 172.21.15.89:0/722445721 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 313467168 0 0) 0x7f5f7415a160 con 0x7f5f74069560 2023-10-10T20:36:27.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f71ffb700 1 -- 172.21.15.89:0/722445721 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f60003570 con 0x7f5f74069560 2023-10-10T20:36:27.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f71ffb700 1 -- 172.21.15.89:0/722445721 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903500711 0 0) 0x7f5f740d13a0 con 0x7f5f74159c60 2023-10-10T20:36:27.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f71ffb700 1 -- 172.21.15.89:0/722445721 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f7415a160 con 0x7f5f74159c60 2023-10-10T20:36:27.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f71ffb700 1 -- 172.21.15.89:0/722445721 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2513355388 0 0) 0x7f5f60003570 con 0x7f5f74069560 2023-10-10T20:36:27.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f71ffb700 1 -- 172.21.15.89:0/722445721 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f74159890 msgr2=0x7f5f7415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:27.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f71ffb700 1 --2- 172.21.15.89:0/722445721 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f74159890 0x7f5f7415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f71ffb700 1 -- 172.21.15.89:0/722445721 >> v1:172.21.15.111:6789/0 conn(0x7f5f74159c60 legacy=0x7f5f7415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:27.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f71ffb700 1 -- 172.21.15.89:0/722445721 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7415a3d0 con 0x7f5f74069560 2023-10-10T20:36:27.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f71ffb700 1 -- 172.21.15.89:0/722445721 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f5f64001470 con 0x7f5f74069560 2023-10-10T20:36:27.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f71ffb700 1 -- 172.21.15.89:0/722445721 >> v1:172.21.15.89:6789/0 conn(0x7f5f74069560 legacy=0x7f5f740ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:27.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f71ffb700 1 --2- 172.21.15.89:0/722445721 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f74159890 0x7f5f60003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f71ffb700 1 --2- 172.21.15.89:0/722445721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f600041d0 0x7f5f60004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f71ffb700 1 --2- 172.21.15.89:0/722445721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f60004d50 0x7f5f600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f71ffb700 1 -- 172.21.15.89:0/722445721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7415e140 con 0x7f5f600041d0 2023-10-10T20:36:27.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f71ffb700 1 -- 172.21.15.89:0/722445721 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6000bbf0 con 0x7f5f60004d50 2023-10-10T20:36:27.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.442+0000 7f5f71ffb700 1 -- 172.21.15.89:0/722445721 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6000bdb0 con 0x7f5f74159890 2023-10-10T20:36:27.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.443+0000 7f5f727fc700 1 --2- 172.21.15.89:0/722445721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f600041d0 0x7f5f60004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.443+0000 7f5f72ffd700 1 --2- 172.21.15.89:0/722445721 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f74159890 0x7f5f60003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.443+0000 7f5f737fe700 1 --2- 172.21.15.89:0/722445721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f60004d50 0x7f5f600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.443+0000 7f5f727fc700 1 -- 172.21.15.89:0/722445721 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f74159890 msgr2=0x7f5f60003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:27.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.443+0000 7f5f727fc700 1 --2- 172.21.15.89:0/722445721 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f74159890 0x7f5f60003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.443+0000 7f5f727fc700 1 -- 172.21.15.89:0/722445721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f60004d50 msgr2=0x7f5f600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:27.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.443+0000 7f5f727fc700 1 --2- 172.21.15.89:0/722445721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f60004d50 0x7f5f600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.443+0000 7f5f727fc700 1 -- 172.21.15.89:0/722445721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f60003570 con 0x7f5f600041d0 2023-10-10T20:36:27.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.443+0000 7f5f72ffd700 1 --2- 172.21.15.89:0/722445721 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f74159890 0x7f5f60003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:27.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.443+0000 7f5f737fe700 1 --2- 172.21.15.89:0/722445721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f60004d50 0x7f5f600051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:27.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.443+0000 7f5f727fc700 1 --2- 172.21.15.89:0/722445721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f600041d0 0x7f5f60004810 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c0019e0 tx=0x7f5f5c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0e3b0a5856d47 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:27.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.443+0000 7f5f71ffb700 1 -- 172.21.15.89:0/722445721 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c00c5d0 con 0x7f5f600041d0 2023-10-10T20:36:27.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.443+0000 7f5f71ffb700 1 -- 172.21.15.89:0/722445721 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5c00c730 con 0x7f5f600041d0 2023-10-10T20:36:27.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.443+0000 7f5f71ffb700 1 -- 172.21.15.89:0/722445721 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c00cad0 con 0x7f5f600041d0 2023-10-10T20:36:27.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.444+0000 7f5f79666700 1 -- 172.21.15.89:0/722445721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f600041d0 msgr2=0x7f5f60004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:27.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.444+0000 7f5f79666700 1 --2- 172.21.15.89:0/722445721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f600041d0 0x7f5f60004810 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c0019e0 tx=0x7f5f5c007d30 comp rx=0 tx=0).stop 2023-10-10T20:36:27.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.444+0000 7f5f737fe700 1 -- 172.21.15.89:0/722445721 reap_dead start 2023-10-10T20:36:27.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.444+0000 7f5f79666700 1 -- 172.21.15.89:0/722445721 shutdown_connections 2023-10-10T20:36:27.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.444+0000 7f5f79666700 1 -- 172.21.15.89:0/722445721 >> 172.21.15.89:0/722445721 conn(0x7f5f74068e20 msgr2=0x7f5f740d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:27.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.444+0000 7f5f79666700 1 -- 172.21.15.89:0/722445721 shutdown_connections 2023-10-10T20:36:27.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.444+0000 7f5f79666700 1 -- 172.21.15.89:0/722445721 wait complete. 2023-10-10T20:36:27.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.445+0000 7f5f79666700 1 Processor -- start 2023-10-10T20:36:27.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.445+0000 7f5f79666700 1 -- start start 2023-10-10T20:36:27.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f79666700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f740cf2f0 0x7f5f740b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f79666700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f740b4d80 0x7f5f740cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f79666700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f740cfc00 0x7f5f740b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f79666700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740b3ad0 con 0x7f5f740cf2f0 2023-10-10T20:36:27.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f79666700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740b48f0 con 0x7f5f740b4d80 2023-10-10T20:36:27.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f79666700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740b3ce0 con 0x7f5f740cfc00 2023-10-10T20:36:27.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f72ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f740cf2f0 0x7f5f740b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f72ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f740cf2f0 0x7f5f740b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46940/0 (socket says 172.21.15.89:46940) 2023-10-10T20:36:27.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f72ffd700 1 -- 172.21.15.89:0/67514066 learned_addr learned my addr 172.21.15.89:0/67514066 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:27.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f737fe700 1 --2- 172.21.15.89:0/67514066 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f740cfc00 0x7f5f740b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f727fc700 1 --2- 172.21.15.89:0/67514066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f740b4d80 0x7f5f740cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f72ffd700 1 -- 172.21.15.89:0/67514066 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f740cfc00 msgr2=0x7f5f740b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:27.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f72ffd700 1 --2- 172.21.15.89:0/67514066 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f740cfc00 0x7f5f740b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f72ffd700 1 -- 172.21.15.89:0/67514066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f740b4d80 msgr2=0x7f5f740cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:27.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f72ffd700 1 --2- 172.21.15.89:0/67514066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f740b4d80 0x7f5f740cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f72ffd700 1 -- 172.21.15.89:0/67514066 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f740b3fc0 con 0x7f5f740cf2f0 2023-10-10T20:36:27.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f72ffd700 1 --2- 172.21.15.89:0/67514066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f740cf2f0 0x7f5f740b3590 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0105a0 tx=0x7f5f6c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:27.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f6affd700 1 -- 172.21.15.89:0/67514066 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c00e050 con 0x7f5f740cf2f0 2023-10-10T20:36:27.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f6affd700 1 -- 172.21.15.89:0/67514066 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c00f040 con 0x7f5f740cf2f0 2023-10-10T20:36:27.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f79666700 1 -- 172.21.15.89:0/67514066 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f740b42a0 con 0x7f5f740cf2f0 2023-10-10T20:36:27.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f6affd700 1 -- 172.21.15.89:0/67514066 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c015860 con 0x7f5f740cf2f0 2023-10-10T20:36:27.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.446+0000 7f5f79666700 1 -- 172.21.15.89:0/67514066 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f740b51f0 con 0x7f5f740cf2f0 2023-10-10T20:36:27.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.447+0000 7f5f79666700 1 -- 172.21.15.89:0/67514066 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7404c3c0 con 0x7f5f740cf2f0 2023-10-10T20:36:27.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.448+0000 7f5f6affd700 1 -- 172.21.15.89:0/67514066 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f5f6c003020 con 0x7f5f740cf2f0 2023-10-10T20:36:27.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.448+0000 7f5f6affd700 1 --2- 172.21.15.89:0/67514066 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f54066fc0 0x7f5f54069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:27.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.448+0000 7f5f6affd700 1 -- 172.21.15.89:0/67514066 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5f6c085220 con 0x7f5f740cf2f0 2023-10-10T20:36:27.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.448+0000 7f5f727fc700 1 --2- 172.21.15.89:0/67514066 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f54066fc0 0x7f5f54069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:27.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.449+0000 7f5f727fc700 1 --2- 172.21.15.89:0/67514066 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f54066fc0 0x7f5f54069470 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5f7416d300 tx=0x7f5f5c015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:27.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.452+0000 7f5f6affd700 1 -- 172.21.15.89:0/67514066 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f6c052390 con 0x7f5f740cf2f0 2023-10-10T20:36:27.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.680+0000 7f5f79666700 1 -- 172.21.15.89:0/67514066 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5f740b4bd0 con 0x7f5f740cf2f0 2023-10-10T20:36:27.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.680+0000 7f5f6affd700 1 -- 172.21.15.89:0/67514066 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5f6c0559b0 con 0x7f5f740cf2f0 2023-10-10T20:36:27.683 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:36:27.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.682+0000 7f5f79666700 1 -- 172.21.15.89:0/67514066 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f54066fc0 msgr2=0x7f5f54069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:27.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.682+0000 7f5f79666700 1 --2- 172.21.15.89:0/67514066 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f54066fc0 0x7f5f54069470 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5f7416d300 tx=0x7f5f5c015000 comp rx=0 tx=0).stop 2023-10-10T20:36:27.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.682+0000 7f5f79666700 1 -- 172.21.15.89:0/67514066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f740cf2f0 msgr2=0x7f5f740b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:27.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.682+0000 7f5f79666700 1 --2- 172.21.15.89:0/67514066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f740cf2f0 0x7f5f740b3590 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0105a0 tx=0x7f5f6c00b040 comp rx=0 tx=0).stop 2023-10-10T20:36:27.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.682+0000 7f5f79666700 1 -- 172.21.15.89:0/67514066 shutdown_connections 2023-10-10T20:36:27.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.682+0000 7f5f79666700 1 --2- 172.21.15.89:0/67514066 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f740cfc00 0x7f5f740b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.682+0000 7f5f79666700 1 --2- 172.21.15.89:0/67514066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f740cf2f0 0x7f5f740b3590 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.682+0000 7f5f79666700 1 --2- 172.21.15.89:0/67514066 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f54066fc0 0x7f5f54069470 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.682+0000 7f5f79666700 1 --2- 172.21.15.89:0/67514066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f740b4d80 0x7f5f740cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:27.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.682+0000 7f5f79666700 1 -- 172.21.15.89:0/67514066 >> 172.21.15.89:0/67514066 conn(0x7f5f74068e20 msgr2=0x7f5f7415d590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:27.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.682+0000 7f5f79666700 1 -- 172.21.15.89:0/67514066 shutdown_connections 2023-10-10T20:36:27.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:27.682+0000 7f5f79666700 1 -- 172.21.15.89:0/67514066 wait complete. 2023-10-10T20:36:27.696 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:36:27.697 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:36:27.697 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-10T20:36:27.697 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:36:27.698 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:36:27.698 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:36:27.699 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:36:27.699 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:36:27.699 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:36:27.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:27.700 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-10T20:36:27.715 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:36:27.715 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:36:27.716 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:36:27.716 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:27.716 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-10T20:36:27.716 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:27.716 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:36:27.717 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:27.717 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-10T20:36:27.717 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:36:27.717 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:27.717 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-10T20:36:27.718 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:36:27.718 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:27.718 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:36:27.718 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:27.719 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:36:27.719 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42378sec preferred_lft 42378sec 2023-10-10T20:36:27.719 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2023-10-10T20:36:27.719 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:27.719 INFO:teuthology.orchestra.run.smithi089.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:36:27.720 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:27.720 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:36:27.720 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:27.720 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::98a3:d4ff:fec1:83bb/64 scope link 2023-10-10T20:36:27.720 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:27.721 INFO:teuthology.orchestra.run.smithi089.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:36:27.721 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether ee:16:22:10:ca:9a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:36:27.721 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec16:22ff:fe10:ca9a/64 scope link 2023-10-10T20:36:27.721 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:27.721 INFO:teuthology.orchestra.run.smithi089.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:36:27.722 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:36:27.722 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::6864:2bff:fe5c:71d4/64 scope link 2023-10-10T20:36:27.722 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:27.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:27.723 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:36:27.778 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-10T20:36:27.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:27.780 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-10T20:36:27.836 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-10T20:36:27.836 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:36:27.837 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:36:27.837 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:36:27.837 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:36:27.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:27.838 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:36:27.839 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:36:27.840 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:36:27.840 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:36:27.841 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:36:27.841 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:36:27.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:27.936 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-10T20:36:27.951 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:36:27.951 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:36:27.952 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:36:27.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:27.953 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-10-10T20:36:28.015 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:36:28.015 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:36:28.069 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:36:28.069 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:36:28.070 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:36:28.124 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:36:28.124 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:36:28.125 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:36:28.203 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:36:28.203 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:36:28.204 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:36:28.211 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:36:28.212 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:36:28.212 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:36:28.212 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:36:28.212 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:36:28.233 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:36:28.233 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:36:28.234 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:36:28.234 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:36:28.234 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:36:28.234 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:36:28.235 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:36:28.235 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:36:28.236 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:36:28.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:28.236 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:36:28.300 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:36:28.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.452+0000 7f4b6080e700 1 Processor -- start 2023-10-10T20:36:28.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.452+0000 7f4b6080e700 1 -- start start 2023-10-10T20:36:28.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.453+0000 7f4b6080e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4b5c143050 0x7f4b5c159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:28.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.453+0000 7f4b6080e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b5c143420 con 0x7f4b5c06b9e0 2023-10-10T20:36:28.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.453+0000 7f4b6080e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b5c06a1e0 con 0x7f4b5c0b3f20 2023-10-10T20:36:28.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.453+0000 7f4b6080e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5c0695d0 con 0x7f4b5c143050 2023-10-10T20:36:28.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.453+0000 7f4b5a59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4b5c06b9e0 0x7f4b5c0b3810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37238/0 (socket says 172.21.15.89:37238) 2023-10-10T20:36:28.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.453+0000 7f4b5a59c700 1 -- 172.21.15.89:0/577646357 learned_addr learned my addr 172.21.15.89:0/577646357 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:28.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.453+0000 7f4b5ad9d700 1 --2- 172.21.15.89:0/577646357 >> v2:172.21.15.111:3300/0 conn(0x7f4b5c143050 0x7f4b5c159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:28.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.453+0000 7f4b59d9b700 1 -- 172.21.15.89:0/577646357 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264221012 0 0) 0x7f4b5c143420 con 0x7f4b5c06b9e0 2023-10-10T20:36:28.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.453+0000 7f4b59d9b700 1 -- 172.21.15.89:0/577646357 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b48003570 con 0x7f4b5c06b9e0 2023-10-10T20:36:28.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.453+0000 7f4b5ad9d700 1 -- 172.21.15.89:0/577646357 >> v1:172.21.15.111:6789/0 conn(0x7f4b5c0b3f20 legacy=0x7f4b5c06a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:28.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.453+0000 7f4b59d9b700 1 -- 172.21.15.89:0/577646357 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2703872707 0 0) 0x7f4b48003570 con 0x7f4b5c06b9e0 2023-10-10T20:36:28.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.453+0000 7f4b5ad9d700 1 -- 172.21.15.89:0/577646357 >> v1:172.21.15.89:6789/0 conn(0x7f4b5c06b9e0 legacy=0x7f4b5c0b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:28.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.453+0000 7f4b5ad9d700 1 -- 172.21.15.89:0/577646357 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5c0698b0 con 0x7f4b5c143050 2023-10-10T20:36:28.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.453+0000 7f4b5ad9d700 1 --2- 172.21.15.89:0/577646357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b5c143050 0x7f4b5c159c90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009eb0 tx=0x7f4b54009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=809e2b941e07a441 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:28.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.454+0000 7f4b59d9b700 1 -- 172.21.15.89:0/577646357 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5400e070 con 0x7f4b5c143050 2023-10-10T20:36:28.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.454+0000 7f4b59d9b700 1 -- 172.21.15.89:0/577646357 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5400b920 con 0x7f4b5c143050 2023-10-10T20:36:28.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.454+0000 7f4b59d9b700 1 -- 172.21.15.89:0/577646357 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44001470 con 0x7f4b5c143050 2023-10-10T20:36:28.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.454+0000 7f4b6080e700 1 -- 172.21.15.89:0/577646357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b5c143050 msgr2=0x7f4b5c159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:28.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.454+0000 7f4b6080e700 1 --2- 172.21.15.89:0/577646357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b5c143050 0x7f4b5c159c90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009eb0 tx=0x7f4b54009780 comp rx=0 tx=0).stop 2023-10-10T20:36:28.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.454+0000 7f4b6080e700 1 -- 172.21.15.89:0/577646357 shutdown_connections 2023-10-10T20:36:28.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.454+0000 7f4b6080e700 1 --2- 172.21.15.89:0/577646357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b5c143050 0x7f4b5c159c90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:28.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.454+0000 7f4b6080e700 1 -- 172.21.15.89:0/577646357 >> 172.21.15.89:0/577646357 conn(0x7f4b5c0ce430 msgr2=0x7f4b5c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:28.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.454+0000 7f4b6080e700 1 -- 172.21.15.89:0/577646357 shutdown_connections 2023-10-10T20:36:28.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.454+0000 7f4b6080e700 1 -- 172.21.15.89:0/577646357 wait complete. 2023-10-10T20:36:28.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.455+0000 7f4b6080e700 1 Processor -- start 2023-10-10T20:36:28.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.455+0000 7f4b6080e700 1 -- start start 2023-10-10T20:36:28.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.456+0000 7f4b6080e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b5c06b9e0 0x7f4b5c16d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:28.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.456+0000 7f4b6080e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5c0b3f20 0x7f4b5c155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:28.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.456+0000 7f4b6080e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b5c143050 0x7f4b5c1566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:28.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.456+0000 7f4b6080e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c15bec0 con 0x7f4b5c06b9e0 2023-10-10T20:36:28.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.456+0000 7f4b6080e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c16da70 con 0x7f4b5c143050 2023-10-10T20:36:28.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.456+0000 7f4b6080e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c16dd50 con 0x7f4b5c0b3f20 2023-10-10T20:36:28.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.456+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b5c06b9e0 0x7f4b5c16d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:28.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.456+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b5c06b9e0 0x7f4b5c16d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35230/0 (socket says 172.21.15.89:35230) 2023-10-10T20:36:28.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.456+0000 7f4b5ad9d700 1 -- 172.21.15.89:0/2447527846 learned_addr learned my addr 172.21.15.89:0/2447527846 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:28.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.456+0000 7f4b5b59e700 1 --2- 172.21.15.89:0/2447527846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b5c143050 0x7f4b5c1566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:28.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.456+0000 7f4b5a59c700 1 --2- 172.21.15.89:0/2447527846 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5c0b3f20 0x7f4b5c155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:28.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.456+0000 7f4b5ad9d700 1 -- 172.21.15.89:0/2447527846 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5c0b3f20 msgr2=0x7f4b5c155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:28.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.456+0000 7f4b5ad9d700 1 --2- 172.21.15.89:0/2447527846 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5c0b3f20 0x7f4b5c155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:28.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.456+0000 7f4b5ad9d700 1 -- 172.21.15.89:0/2447527846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b5c143050 msgr2=0x7f4b5c1566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:28.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.456+0000 7f4b5ad9d700 1 --2- 172.21.15.89:0/2447527846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b5c143050 0x7f4b5c1566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:28.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.456+0000 7f4b5ad9d700 1 -- 172.21.15.89:0/2447527846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b54009a20 con 0x7f4b5c06b9e0 2023-10-10T20:36:28.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.457+0000 7f4b5ad9d700 1 --2- 172.21.15.89:0/2447527846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b5c06b9e0 0x7f4b5c16d1f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400b6c0 tx=0x7f4b54017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:28.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.457+0000 7f4b52ffd700 1 -- 172.21.15.89:0/2447527846 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b540096f0 con 0x7f4b5c06b9e0 2023-10-10T20:36:28.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.457+0000 7f4b52ffd700 1 -- 172.21.15.89:0/2447527846 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b54026400 con 0x7f4b5c06b9e0 2023-10-10T20:36:28.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.457+0000 7f4b6080e700 1 -- 172.21.15.89:0/2447527846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5c15c160 con 0x7f4b5c06b9e0 2023-10-10T20:36:28.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.457+0000 7f4b52ffd700 1 -- 172.21.15.89:0/2447527846 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b540267c0 con 0x7f4b5c06b9e0 2023-10-10T20:36:28.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.457+0000 7f4b6080e700 1 -- 172.21.15.89:0/2447527846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5c155e80 con 0x7f4b5c06b9e0 2023-10-10T20:36:28.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.458+0000 7f4b6080e700 1 -- 172.21.15.89:0/2447527846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5c04c3c0 con 0x7f4b5c06b9e0 2023-10-10T20:36:28.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.458+0000 7f4b52ffd700 1 -- 172.21.15.89:0/2447527846 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f4b54019030 con 0x7f4b5c06b9e0 2023-10-10T20:36:28.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.458+0000 7f4b52ffd700 1 --2- 172.21.15.89:0/2447527846 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4b3c066fc0 0x7f4b3c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:28.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.459+0000 7f4b52ffd700 1 -- 172.21.15.89:0/2447527846 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4b54086680 con 0x7f4b5c06b9e0 2023-10-10T20:36:28.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.459+0000 7f4b5a59c700 1 --2- 172.21.15.89:0/2447527846 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4b3c066fc0 0x7f4b3c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:28.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.459+0000 7f4b5a59c700 1 --2- 172.21.15.89:0/2447527846 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4b3c066fc0 0x7f4b3c069470 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0cdb30 tx=0x7f4b44000bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:28.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.463+0000 7f4b52ffd700 1 -- 172.21.15.89:0/2447527846 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b540538b0 con 0x7f4b5c06b9e0 2023-10-10T20:36:28.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.693+0000 7f4b6080e700 1 -- 172.21.15.89:0/2447527846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4b5c0adad0 con 0x7f4b5c06b9e0 2023-10-10T20:36:28.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.694+0000 7f4b52ffd700 1 -- 172.21.15.89:0/2447527846 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4b54056ed0 con 0x7f4b5c06b9e0 2023-10-10T20:36:28.696 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:36:28.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.696+0000 7f4b6080e700 1 -- 172.21.15.89:0/2447527846 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4b3c066fc0 msgr2=0x7f4b3c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:28.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.696+0000 7f4b6080e700 1 --2- 172.21.15.89:0/2447527846 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4b3c066fc0 0x7f4b3c069470 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0cdb30 tx=0x7f4b44000bc0 comp rx=0 tx=0).stop 2023-10-10T20:36:28.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.696+0000 7f4b6080e700 1 -- 172.21.15.89:0/2447527846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b5c06b9e0 msgr2=0x7f4b5c16d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:28.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.696+0000 7f4b6080e700 1 --2- 172.21.15.89:0/2447527846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b5c06b9e0 0x7f4b5c16d1f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400b6c0 tx=0x7f4b54017000 comp rx=0 tx=0).stop 2023-10-10T20:36:28.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.696+0000 7f4b6080e700 1 -- 172.21.15.89:0/2447527846 shutdown_connections 2023-10-10T20:36:28.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.696+0000 7f4b6080e700 1 --2- 172.21.15.89:0/2447527846 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4b3c066fc0 0x7f4b3c069470 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:28.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.696+0000 7f4b6080e700 1 --2- 172.21.15.89:0/2447527846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b5c143050 0x7f4b5c1566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:28.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.696+0000 7f4b6080e700 1 --2- 172.21.15.89:0/2447527846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b5c06b9e0 0x7f4b5c16d1f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:28.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.696+0000 7f4b6080e700 1 --2- 172.21.15.89:0/2447527846 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5c0b3f20 0x7f4b5c155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:28.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.696+0000 7f4b6080e700 1 -- 172.21.15.89:0/2447527846 >> 172.21.15.89:0/2447527846 conn(0x7f4b5c0ce430 msgr2=0x7f4b5c15b300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:28.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.696+0000 7f4b6080e700 1 -- 172.21.15.89:0/2447527846 shutdown_connections 2023-10-10T20:36:28.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:28.696+0000 7f4b6080e700 1 -- 172.21.15.89:0/2447527846 wait complete. 2023-10-10T20:36:28.710 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:36:28.710 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:36:28.710 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-10-10T20:36:28.711 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:36:28.711 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:36:28.711 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:36:28.712 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:36:28.712 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:36:28.712 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:36:28.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:28.713 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-10-10T20:36:28.731 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:36:28.731 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:36:28.732 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:36:28.732 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:28.732 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-10-10T20:36:28.732 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:28.732 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:36:28.732 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:28.733 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-10-10T20:36:28.733 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:36:28.733 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:28.733 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-10-10T20:36:28.733 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:36:28.734 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:28.734 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:36:28.734 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:28.734 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:36:28.734 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42394sec preferred_lft 42394sec 2023-10-10T20:36:28.735 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-10-10T20:36:28.735 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:28.735 INFO:teuthology.orchestra.run.smithi111.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:36:28.735 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:36:28.735 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:36:28.736 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:28.736 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::a486:a6ff:fe5f:6021/64 scope link 2023-10-10T20:36:28.736 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:28.736 INFO:teuthology.orchestra.run.smithi111.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:36:28.736 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 1e:d0:81:1b:e2:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:36:28.737 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::1cd0:81ff:fe1b:e26b/64 scope link 2023-10-10T20:36:28.737 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:28.737 INFO:teuthology.orchestra.run.smithi111.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:36:28.737 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:36:28.737 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::88d:cdff:fe75:455e/64 scope link 2023-10-10T20:36:28.737 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:36:28.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:28.738 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:36:28.797 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-10-10T20:36:28.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:28.799 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-10-10T20:36:28.857 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-10-10T20:36:28.857 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:36:28.857 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:36:28.857 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:36:28.858 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:36:28.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:28.858 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:36:28.859 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:36:28.859 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:36:28.859 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:36:28.860 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:36:28.860 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:36:28.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:28.963 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-10-10T20:36:28.978 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:36:28.978 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:36:28.978 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:36:28.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:28.979 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-10-10T20:36:29.042 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:36:29.043 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:36:29.098 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:36:29.098 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:36:29.098 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:36:29.152 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:36:29.152 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:36:29.153 DEBUG:teuthology.orchestra.run.smithi111:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:36:29.231 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:36:29.232 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:36:29.232 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:36:29.239 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:36:29.240 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:36:29.240 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:36:29.240 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:36:29.240 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:36:29.261 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:36:29.261 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:36:29.262 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:36:29.262 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:36:29.262 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:36:29.262 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:36:29.262 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:36:29.263 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:36:29.264 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:36:29.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:36:29.264 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:36:29.330 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-10T20:36:29.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.448+0000 7f6c58154700 1 Processor -- start 2023-10-10T20:36:29.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.448+0000 7f6c58154700 1 -- start start 2023-10-10T20:36:29.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.449+0000 7f6c58154700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6c50159ec0 0x7f6c500b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:29.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.449+0000 7f6c58154700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6c500cf350 0x7f6c500cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:29.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.449+0000 7f6c58154700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c500b5b00 con 0x7f6c500b53d0 2023-10-10T20:36:29.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.449+0000 7f6c58154700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c500b5850 con 0x7f6c50159ec0 2023-10-10T20:36:29.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.449+0000 7f6c58154700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c500b5da0 con 0x7f6c500cf350 2023-10-10T20:36:29.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.449+0000 7f6c55ef0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6c50159ec0 0x7f6c500b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:29.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.449+0000 7f6c55ef0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6c50159ec0 0x7f6c500b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35238/0 (socket says 172.21.15.89:35238) 2023-10-10T20:36:29.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.449+0000 7f6c55ef0700 1 -- 172.21.15.89:0/3674874263 learned_addr learned my addr 172.21.15.89:0/3674874263 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:29.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.450+0000 7f6c566f1700 1 --2- 172.21.15.89:0/3674874263 >> v2:172.21.15.111:3300/0 conn(0x7f6c500cf350 0x7f6c500cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:29.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.450+0000 7f6c55ef0700 1 -- 172.21.15.89:0/3674874263 >> v2:172.21.15.111:3300/0 conn(0x7f6c500cf350 msgr2=0x7f6c500cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:29.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.450+0000 7f6c55ef0700 1 --2- 172.21.15.89:0/3674874263 >> v2:172.21.15.111:3300/0 conn(0x7f6c500cf350 0x7f6c500cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:29.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.450+0000 7f6c55ef0700 1 -- 172.21.15.89:0/3674874263 >> v1:172.21.15.111:6789/0 conn(0x7f6c500b53d0 legacy=0x7f6c500cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:29.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.450+0000 7f6c55ef0700 1 -- 172.21.15.89:0/3674874263 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c500b6080 con 0x7f6c50159ec0 2023-10-10T20:36:29.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.450+0000 7f6c566f1700 1 --2- 172.21.15.89:0/3674874263 >> v2:172.21.15.111:3300/0 conn(0x7f6c500cf350 0x7f6c500cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:36:29.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.450+0000 7f6c55ef0700 1 --2- 172.21.15.89:0/3674874263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c50159ec0 0x7f6c500b4e90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6c500cb4a0 tx=0x7f6c4c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8e7c67fed9f187 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:29.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.450+0000 7f6c54eee700 1 -- 172.21.15.89:0/3674874263 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c005c10 con 0x7f6c50159ec0 2023-10-10T20:36:29.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.450+0000 7f6c54eee700 1 -- 172.21.15.89:0/3674874263 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c005d70 con 0x7f6c50159ec0 2023-10-10T20:36:29.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.450+0000 7f6c54eee700 1 -- 172.21.15.89:0/3674874263 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c00b840 con 0x7f6c50159ec0 2023-10-10T20:36:29.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.450+0000 7f6c58154700 1 -- 172.21.15.89:0/3674874263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c50159ec0 msgr2=0x7f6c500b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:29.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.450+0000 7f6c58154700 1 --2- 172.21.15.89:0/3674874263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c50159ec0 0x7f6c500b4e90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6c500cb4a0 tx=0x7f6c4c009220 comp rx=0 tx=0).stop 2023-10-10T20:36:29.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.450+0000 7f6c58154700 1 -- 172.21.15.89:0/3674874263 shutdown_connections 2023-10-10T20:36:29.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.450+0000 7f6c58154700 1 --2- 172.21.15.89:0/3674874263 >> v2:172.21.15.111:3300/0 conn(0x7f6c500cf350 0x7f6c500cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:29.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.450+0000 7f6c58154700 1 --2- 172.21.15.89:0/3674874263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c50159ec0 0x7f6c500b4e90 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:29.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.450+0000 7f6c58154700 1 -- 172.21.15.89:0/3674874263 >> 172.21.15.89:0/3674874263 conn(0x7f6c500999f0 msgr2=0x7f6c500b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:29.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.451+0000 7f6c58154700 1 -- 172.21.15.89:0/3674874263 shutdown_connections 2023-10-10T20:36:29.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.451+0000 7f6c58154700 1 -- 172.21.15.89:0/3674874263 wait complete. 2023-10-10T20:36:29.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.452+0000 7f6c58154700 1 Processor -- start 2023-10-10T20:36:29.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.452+0000 7f6c58154700 1 -- start start 2023-10-10T20:36:29.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.452+0000 7f6c58154700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c500b53d0 0x7f6c50145570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:29.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.452+0000 7f6c58154700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c500cf350 0x7f6c5009c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:29.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.452+0000 7f6c58154700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6c50159ec0 0x7f6c5009cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:29.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.452+0000 7f6c58154700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5009e220 con 0x7f6c500cf350 2023-10-10T20:36:29.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.452+0000 7f6c58154700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c50145ab0 con 0x7f6c500b53d0 2023-10-10T20:36:29.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.452+0000 7f6c58154700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5009dd90 con 0x7f6c50159ec0 2023-10-10T20:36:29.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.452+0000 7f6c556ef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c500cf350 0x7f6c5009c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:29.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.452+0000 7f6c556ef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c500cf350 0x7f6c5009c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35240/0 (socket says 172.21.15.89:35240) 2023-10-10T20:36:29.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.452+0000 7f6c556ef700 1 -- 172.21.15.89:0/3072329952 learned_addr learned my addr 172.21.15.89:0/3072329952 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:29.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.453+0000 7f6c55ef0700 1 --2- 172.21.15.89:0/3072329952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c500b53d0 0x7f6c50145570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:29.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.453+0000 7f6c566f1700 1 --2- 172.21.15.89:0/3072329952 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6c50159ec0 0x7f6c5009cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:29.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.453+0000 7f6c556ef700 1 -- 172.21.15.89:0/3072329952 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6c50159ec0 msgr2=0x7f6c5009cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:29.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.453+0000 7f6c556ef700 1 --2- 172.21.15.89:0/3072329952 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6c50159ec0 0x7f6c5009cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:29.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.453+0000 7f6c556ef700 1 -- 172.21.15.89:0/3072329952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c500b53d0 msgr2=0x7f6c50145570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:29.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.453+0000 7f6c556ef700 1 --2- 172.21.15.89:0/3072329952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c500b53d0 0x7f6c50145570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:29.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.453+0000 7f6c556ef700 1 -- 172.21.15.89:0/3072329952 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c009480 con 0x7f6c500cf350 2023-10-10T20:36:29.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.453+0000 7f6c556ef700 1 --2- 172.21.15.89:0/3072329952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c500cf350 0x7f6c5009c650 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000b610 tx=0x7f6c40009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:29.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.453+0000 7f6c45ffb700 1 -- 172.21.15.89:0/3072329952 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4000f510 con 0x7f6c500cf350 2023-10-10T20:36:29.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.453+0000 7f6c45ffb700 1 -- 172.21.15.89:0/3072329952 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4000f670 con 0x7f6c500cf350 2023-10-10T20:36:29.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.453+0000 7f6c58154700 1 -- 172.21.15.89:0/3072329952 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5009d180 con 0x7f6c500cf350 2023-10-10T20:36:29.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.453+0000 7f6c45ffb700 1 -- 172.21.15.89:0/3072329952 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4000fa30 con 0x7f6c500cf350 2023-10-10T20:36:29.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.453+0000 7f6c58154700 1 -- 172.21.15.89:0/3072329952 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5009d740 con 0x7f6c500cf350 2023-10-10T20:36:29.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.454+0000 7f6c58154700 1 -- 172.21.15.89:0/3072329952 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c50063c90 con 0x7f6c500cf350 2023-10-10T20:36:29.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.454+0000 7f6c45ffb700 1 -- 172.21.15.89:0/3072329952 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f6c4001a400 con 0x7f6c500cf350 2023-10-10T20:36:29.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.455+0000 7f6c45ffb700 1 --2- 172.21.15.89:0/3072329952 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6c34066fc0 0x7f6c34069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:29.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.455+0000 7f6c45ffb700 1 -- 172.21.15.89:0/3072329952 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6c40083fe0 con 0x7f6c500cf350 2023-10-10T20:36:29.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.455+0000 7f6c55ef0700 1 --2- 172.21.15.89:0/3072329952 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6c34066fc0 0x7f6c34069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:29.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.455+0000 7f6c55ef0700 1 --2- 172.21.15.89:0/3072329952 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6c34066fc0 0x7f6c34069470 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0030c0 tx=0x7f6c4c015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:29.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.459+0000 7f6c45ffb700 1 -- 172.21.15.89:0/3072329952 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c400512e0 con 0x7f6c500cf350 2023-10-10T20:36:29.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:29.693+0000 7f6c58154700 1 -- 172.21.15.89:0/3072329952 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6c500b4e90 con 0x7f6c500cf350 2023-10-10T20:36:30.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.545+0000 7f6c45ffb700 1 -- 172.21.15.89:0/3072329952 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v38) v1 ==== 136+0+0 (secure 0 0 0) 0x7f6c40054900 con 0x7f6c500cf350 2023-10-10T20:36:30.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.548+0000 7f6c58154700 1 -- 172.21.15.89:0/3072329952 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6c34066fc0 msgr2=0x7f6c34069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:30.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.548+0000 7f6c58154700 1 --2- 172.21.15.89:0/3072329952 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6c34066fc0 0x7f6c34069470 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0030c0 tx=0x7f6c4c015000 comp rx=0 tx=0).stop 2023-10-10T20:36:30.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.548+0000 7f6c58154700 1 -- 172.21.15.89:0/3072329952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c500cf350 msgr2=0x7f6c5009c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:30.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.548+0000 7f6c58154700 1 --2- 172.21.15.89:0/3072329952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c500cf350 0x7f6c5009c650 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000b610 tx=0x7f6c40009120 comp rx=0 tx=0).stop 2023-10-10T20:36:30.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.548+0000 7f6c58154700 1 -- 172.21.15.89:0/3072329952 shutdown_connections 2023-10-10T20:36:30.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.548+0000 7f6c58154700 1 --2- 172.21.15.89:0/3072329952 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6c50159ec0 0x7f6c5009cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:30.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.548+0000 7f6c58154700 1 --2- 172.21.15.89:0/3072329952 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6c34066fc0 0x7f6c34069470 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.548+0000 7f6c58154700 1 --2- 172.21.15.89:0/3072329952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c500b53d0 0x7f6c50145570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.548+0000 7f6c58154700 1 --2- 172.21.15.89:0/3072329952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c500cf350 0x7f6c5009c650 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.548+0000 7f6c58154700 1 -- 172.21.15.89:0/3072329952 >> 172.21.15.89:0/3072329952 conn(0x7f6c500999f0 msgr2=0x7f6c500c5510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.548+0000 7f6c58154700 1 -- 172.21.15.89:0/3072329952 shutdown_connections 2023-10-10T20:36:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.548+0000 7f6c58154700 1 -- 172.21.15.89:0/3072329952 wait complete. 2023-10-10T20:36:30.563 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-10T20:36:30.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.678+0000 7fe3cf10e700 1 Processor -- start 2023-10-10T20:36:30.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.678+0000 7fe3cf10e700 1 -- start start 2023-10-10T20:36:30.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.679+0000 7fe3cf10e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe3c8069560 0x7fe3c80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:30.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.679+0000 7fe3cf10e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c8069ee0 con 0x7fe3c8159c60 2023-10-10T20:36:30.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.679+0000 7fe3cf10e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c815a160 con 0x7fe3c8159890 2023-10-10T20:36:30.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.679+0000 7fe3cf10e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c80d13a0 con 0x7fe3c8069560 2023-10-10T20:36:30.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.679+0000 7fe3bffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe3c8159c60 0x7fe3c815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37242/0 (socket says 172.21.15.89:37242) 2023-10-10T20:36:30.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.679+0000 7fe3bffff700 1 -- 172.21.15.89:0/2721512551 learned_addr learned my addr 172.21.15.89:0/2721512551 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:30.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.679+0000 7fe3cd6ab700 1 --2- 172.21.15.89:0/2721512551 >> v2:172.21.15.111:3300/0 conn(0x7fe3c8069560 0x7fe3c80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:30.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.679+0000 7fe3bf7fe700 1 -- 172.21.15.89:0/2721512551 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559346527 0 0) 0x7fe3c8069ee0 con 0x7fe3c8159c60 2023-10-10T20:36:30.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.679+0000 7fe3bf7fe700 1 -- 172.21.15.89:0/2721512551 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3b0003570 con 0x7fe3c8159c60 2023-10-10T20:36:30.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.680+0000 7fe3bf7fe700 1 -- 172.21.15.89:0/2721512551 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1315822689 0 0) 0x7fe3b0003570 con 0x7fe3c8159c60 2023-10-10T20:36:30.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.680+0000 7fe3cd6ab700 1 -- 172.21.15.89:0/2721512551 >> v1:172.21.15.111:6789/0 conn(0x7fe3c8159890 legacy=0x7fe3c815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:30.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.680+0000 7fe3cd6ab700 1 -- 172.21.15.89:0/2721512551 >> v1:172.21.15.89:6789/0 conn(0x7fe3c8159c60 legacy=0x7fe3c815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:30.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.680+0000 7fe3cd6ab700 1 -- 172.21.15.89:0/2721512551 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c815e140 con 0x7fe3c8069560 2023-10-10T20:36:30.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.680+0000 7fe3cd6ab700 1 --2- 172.21.15.89:0/2721512551 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3c8069560 0x7fe3c80699a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009e60 tx=0x7fe3c0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a051ab4c2246199b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:30.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.680+0000 7fe3bf7fe700 1 -- 172.21.15.89:0/2721512551 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c000e070 con 0x7fe3c8069560 2023-10-10T20:36:30.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.680+0000 7fe3bf7fe700 1 -- 172.21.15.89:0/2721512551 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c000b850 con 0x7fe3c8069560 2023-10-10T20:36:30.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.680+0000 7fe3bf7fe700 1 -- 172.21.15.89:0/2721512551 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8001470 con 0x7fe3c8069560 2023-10-10T20:36:30.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.680+0000 7fe3cf10e700 1 -- 172.21.15.89:0/2721512551 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3c8069560 msgr2=0x7fe3c80699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:30.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.680+0000 7fe3cf10e700 1 --2- 172.21.15.89:0/2721512551 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3c8069560 0x7fe3c80699a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009e60 tx=0x7fe3c0009780 comp rx=0 tx=0).stop 2023-10-10T20:36:30.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.680+0000 7fe3cf10e700 1 -- 172.21.15.89:0/2721512551 shutdown_connections 2023-10-10T20:36:30.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.680+0000 7fe3cf10e700 1 --2- 172.21.15.89:0/2721512551 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3c8069560 0x7fe3c80699a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:30.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.680+0000 7fe3cf10e700 1 -- 172.21.15.89:0/2721512551 >> 172.21.15.89:0/2721512551 conn(0x7fe3c8068e20 msgr2=0x7fe3c80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:30.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.681+0000 7fe3cf10e700 1 -- 172.21.15.89:0/2721512551 shutdown_connections 2023-10-10T20:36:30.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.681+0000 7fe3cf10e700 1 -- 172.21.15.89:0/2721512551 wait complete. 2023-10-10T20:36:30.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.682+0000 7fe3cf10e700 1 Processor -- start 2023-10-10T20:36:30.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.682+0000 7fe3cf10e700 1 -- start start 2023-10-10T20:36:30.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.682+0000 7fe3cf10e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3c8069560 0x7fe3c816dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:30.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.682+0000 7fe3cf10e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe3c8159890 0x7fe3c80b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:30.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.682+0000 7fe3cf10e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3c8159c60 0x7fe3c8140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:30.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.682+0000 7fe3cf10e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c816e250 con 0x7fe3c8069560 2023-10-10T20:36:30.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.682+0000 7fe3cf10e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c80b6340 con 0x7fe3c8159c60 2023-10-10T20:36:30.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.682+0000 7fe3cf10e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c80b5730 con 0x7fe3c8159890 2023-10-10T20:36:30.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.682+0000 7fe3cceaa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3c8069560 0x7fe3c816dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:30.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.683+0000 7fe3cceaa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3c8069560 0x7fe3c816dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35242/0 (socket says 172.21.15.89:35242) 2023-10-10T20:36:30.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.683+0000 7fe3cceaa700 1 -- 172.21.15.89:0/3054902577 learned_addr learned my addr 172.21.15.89:0/3054902577 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:30.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.683+0000 7fe3cd6ab700 1 --2- 172.21.15.89:0/3054902577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3c8159c60 0x7fe3c8140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:30.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.683+0000 7fe3bffff700 1 --2- 172.21.15.89:0/3054902577 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe3c8159890 0x7fe3c80b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:30.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.683+0000 7fe3cceaa700 1 -- 172.21.15.89:0/3054902577 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe3c8159890 msgr2=0x7fe3c80b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:30.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.683+0000 7fe3cceaa700 1 --2- 172.21.15.89:0/3054902577 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe3c8159890 0x7fe3c80b0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:30.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.683+0000 7fe3cceaa700 1 -- 172.21.15.89:0/3054902577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3c8159c60 msgr2=0x7fe3c8140940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:30.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.683+0000 7fe3cceaa700 1 --2- 172.21.15.89:0/3054902577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3c8159c60 0x7fe3c8140940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:30.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.683+0000 7fe3cceaa700 1 -- 172.21.15.89:0/3054902577 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c0009a20 con 0x7fe3c8069560 2023-10-10T20:36:30.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.683+0000 7fe3cceaa700 1 --2- 172.21.15.89:0/3054902577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3c8069560 0x7fe3c816dd10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400c810 tx=0x7fe3c4007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:30.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.683+0000 7fe3bcff9700 1 -- 172.21.15.89:0/3054902577 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4011590 con 0x7fe3c8069560 2023-10-10T20:36:30.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.683+0000 7fe3bcff9700 1 -- 172.21.15.89:0/3054902577 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c4001e90 con 0x7fe3c8069560 2023-10-10T20:36:30.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.683+0000 7fe3cf10e700 1 -- 172.21.15.89:0/3054902577 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c80b5a10 con 0x7fe3c8069560 2023-10-10T20:36:30.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.683+0000 7fe3bcff9700 1 -- 172.21.15.89:0/3054902577 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4011960 con 0x7fe3c8069560 2023-10-10T20:36:30.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.683+0000 7fe3cf10e700 1 -- 172.21.15.89:0/3054902577 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c80b5fd0 con 0x7fe3c8069560 2023-10-10T20:36:30.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.684+0000 7fe3cf10e700 1 -- 172.21.15.89:0/3054902577 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c804c3c0 con 0x7fe3c8069560 2023-10-10T20:36:30.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.684+0000 7fe3bcff9700 1 -- 172.21.15.89:0/3054902577 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fe3c4011d50 con 0x7fe3c8069560 2023-10-10T20:36:30.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.685+0000 7fe3bcff9700 1 --2- 172.21.15.89:0/3054902577 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe3b4066fc0 0x7fe3b4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:30.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.685+0000 7fe3bffff700 1 --2- 172.21.15.89:0/3054902577 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe3b4066fc0 0x7fe3b4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:30.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.685+0000 7fe3bcff9700 1 -- 172.21.15.89:0/3054902577 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe3c4083950 con 0x7fe3c8069560 2023-10-10T20:36:30.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.685+0000 7fe3bffff700 1 --2- 172.21.15.89:0/3054902577 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe3b4066fc0 0x7fe3b4069470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80d0d60 tx=0x7fe3b8000bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:30.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.689+0000 7fe3bcff9700 1 -- 172.21.15.89:0/3054902577 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3c4050bd0 con 0x7fe3c8069560 2023-10-10T20:36:30.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.897+0000 7fe3cf10e700 1 -- 172.21.15.89:0/3054902577 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3c806a5b0 con 0x7fe3b4066fc0 2023-10-10T20:36:30.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.898+0000 7fe3bcff9700 1 -- 172.21.15.89:0/3054902577 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fe3c806a5b0 con 0x7fe3b4066fc0 2023-10-10T20:36:30.901 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:36:30.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:36:30.901 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs" 2023-10-10T20:36:30.901 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:36:30.901 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:36:30.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.901+0000 7fe3cf10e700 1 -- 172.21.15.89:0/3054902577 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe3b4066fc0 msgr2=0x7fe3b4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:30.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.901+0000 7fe3cf10e700 1 --2- 172.21.15.89:0/3054902577 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe3b4066fc0 0x7fe3b4069470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80d0d60 tx=0x7fe3b8000bc0 comp rx=0 tx=0).stop 2023-10-10T20:36:30.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.901+0000 7fe3cf10e700 1 -- 172.21.15.89:0/3054902577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3c8069560 msgr2=0x7fe3c816dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:30.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.901+0000 7fe3cf10e700 1 --2- 172.21.15.89:0/3054902577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3c8069560 0x7fe3c816dd10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400c810 tx=0x7fe3c4007f00 comp rx=0 tx=0).stop 2023-10-10T20:36:30.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.901+0000 7fe3cf10e700 1 -- 172.21.15.89:0/3054902577 shutdown_connections 2023-10-10T20:36:30.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.901+0000 7fe3cf10e700 1 --2- 172.21.15.89:0/3054902577 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe3b4066fc0 0x7fe3b4069470 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:30.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.901+0000 7fe3cf10e700 1 --2- 172.21.15.89:0/3054902577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3c8159c60 0x7fe3c8140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:30.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.901+0000 7fe3cf10e700 1 --2- 172.21.15.89:0/3054902577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3c8069560 0x7fe3c816dd10 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:30.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.901+0000 7fe3cf10e700 1 --2- 172.21.15.89:0/3054902577 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe3c8159890 0x7fe3c80b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:30.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.901+0000 7fe3cf10e700 1 -- 172.21.15.89:0/3054902577 >> 172.21.15.89:0/3054902577 conn(0x7fe3c8068e20 msgr2=0x7fe3c8156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:30.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.901+0000 7fe3cf10e700 1 -- 172.21.15.89:0/3054902577 shutdown_connections 2023-10-10T20:36:30.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:30.901+0000 7fe3cf10e700 1 -- 172.21.15.89:0/3054902577 wait complete. 2023-10-10T20:36:30.915 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:36:31.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.030+0000 7fd86d946700 1 Processor -- start 2023-10-10T20:36:31.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.030+0000 7fd86d946700 1 -- start start 2023-10-10T20:36:31.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.031+0000 7fd86d946700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd8680685e0 0x7fd8680b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.031+0000 7fd86d946700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd868159970 0x7fd8680b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.031+0000 7fd86d946700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86806cff0 con 0x7fd8680689b0 2023-10-10T20:36:31.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.031+0000 7fd86d946700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd86806bf50 con 0x7fd8680685e0 2023-10-10T20:36:31.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.031+0000 7fd86d946700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86806c230 con 0x7fd868159970 2023-10-10T20:36:31.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.031+0000 7fd866ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd8680685e0 0x7fd8680b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.031+0000 7fd866ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd8680685e0 0x7fd8680b0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35246/0 (socket says 172.21.15.89:35246) 2023-10-10T20:36:31.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.031+0000 7fd866ffd700 1 -- 172.21.15.89:0/3456348407 learned_addr learned my addr 172.21.15.89:0/3456348407 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:31.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.031+0000 7fd8677fe700 1 --2- 172.21.15.89:0/3456348407 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd868159970 0x7fd8680b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.031+0000 7fd866ffd700 1 -- 172.21.15.89:0/3456348407 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd868159970 msgr2=0x7fd8680b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.031+0000 7fd866ffd700 1 --2- 172.21.15.89:0/3456348407 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd868159970 0x7fd8680b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.031+0000 7fd866ffd700 1 -- 172.21.15.89:0/3456348407 >> v1:172.21.15.111:6789/0 conn(0x7fd8680689b0 legacy=0x7fd868159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.031+0000 7fd866ffd700 1 -- 172.21.15.89:0/3456348407 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86806c510 con 0x7fd8680685e0 2023-10-10T20:36:31.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.032+0000 7fd866ffd700 1 --2- 172.21.15.89:0/3456348407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8680685e0 0x7fd8680b0b60 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd8580099b0 tx=0x7fd858009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79b0504c66379e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:31.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.032+0000 7fd8667fc700 1 -- 172.21.15.89:0/3456348407 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85800b860 con 0x7fd8680685e0 2023-10-10T20:36:31.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.032+0000 7fd8667fc700 1 -- 172.21.15.89:0/3456348407 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85800b9c0 con 0x7fd8680685e0 2023-10-10T20:36:31.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.032+0000 7fd8667fc700 1 -- 172.21.15.89:0/3456348407 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85800bd60 con 0x7fd8680685e0 2023-10-10T20:36:31.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.032+0000 7fd86d946700 1 -- 172.21.15.89:0/3456348407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8680685e0 msgr2=0x7fd8680b0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:31.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.032+0000 7fd86d946700 1 --2- 172.21.15.89:0/3456348407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8680685e0 0x7fd8680b0b60 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd8580099b0 tx=0x7fd858009250 comp rx=0 tx=0).stop 2023-10-10T20:36:31.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.032+0000 7fd86d946700 1 -- 172.21.15.89:0/3456348407 shutdown_connections 2023-10-10T20:36:31.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.032+0000 7fd86d946700 1 --2- 172.21.15.89:0/3456348407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8680685e0 0x7fd8680b0b60 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.032+0000 7fd86d946700 1 --2- 172.21.15.89:0/3456348407 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd868159970 0x7fd8680b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.032+0000 7fd86d946700 1 -- 172.21.15.89:0/3456348407 >> 172.21.15.89:0/3456348407 conn(0x7fd86806a940 msgr2=0x7fd86806ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:31.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.032+0000 7fd86d946700 1 -- 172.21.15.89:0/3456348407 shutdown_connections 2023-10-10T20:36:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.032+0000 7fd86d946700 1 -- 172.21.15.89:0/3456348407 wait complete. 2023-10-10T20:36:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.033+0000 7fd86d946700 1 Processor -- start 2023-10-10T20:36:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.034+0000 7fd86d946700 1 -- start start 2023-10-10T20:36:31.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.034+0000 7fd86d946700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8680685e0 0x7fd86816aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.034+0000 7fd86d946700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd8680689b0 0x7fd868151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.034+0000 7fd86d946700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd868159970 0x7fd86814e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.034+0000 7fd86d946700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86814ced0 con 0x7fd8680685e0 2023-10-10T20:36:31.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.034+0000 7fd86d946700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86815ae90 con 0x7fd868159970 2023-10-10T20:36:31.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.034+0000 7fd86d946700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd868152520 con 0x7fd8680689b0 2023-10-10T20:36:31.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.034+0000 7fd866ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8680685e0 0x7fd86816aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.034+0000 7fd866ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8680685e0 0x7fd86816aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35258/0 (socket says 172.21.15.89:35258) 2023-10-10T20:36:31.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.034+0000 7fd866ffd700 1 -- 172.21.15.89:0/3435629573 learned_addr learned my addr 172.21.15.89:0/3435629573 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:31.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.034+0000 7fd8677fe700 1 --2- 172.21.15.89:0/3435629573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd868159970 0x7fd86814e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.034+0000 7fd85e7fc700 1 --2- 172.21.15.89:0/3435629573 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd8680689b0 0x7fd868151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.035+0000 7fd866ffd700 1 -- 172.21.15.89:0/3435629573 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd8680689b0 msgr2=0x7fd868151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:31.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.035+0000 7fd866ffd700 1 --2- 172.21.15.89:0/3435629573 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd8680689b0 0x7fd868151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.035+0000 7fd866ffd700 1 -- 172.21.15.89:0/3435629573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd868159970 msgr2=0x7fd86814e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:31.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.035+0000 7fd866ffd700 1 --2- 172.21.15.89:0/3435629573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd868159970 0x7fd86814e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.035+0000 7fd866ffd700 1 -- 172.21.15.89:0/3435629573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8580094f0 con 0x7fd8680685e0 2023-10-10T20:36:31.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.035+0000 7fd8677fe700 1 --2- 172.21.15.89:0/3435629573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd868159970 0x7fd86814e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:31.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.035+0000 7fd866ffd700 1 --2- 172.21.15.89:0/3435629573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8680685e0 0x7fd86816aa20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd86806b630 tx=0x7fd858017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:31.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.035+0000 7fd85f7fe700 1 -- 172.21.15.89:0/3435629573 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858024440 con 0x7fd8680685e0 2023-10-10T20:36:31.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.035+0000 7fd85f7fe700 1 -- 172.21.15.89:0/3435629573 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8580245c0 con 0x7fd8680685e0 2023-10-10T20:36:31.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.035+0000 7fd86d946700 1 -- 172.21.15.89:0/3435629573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd86814c630 con 0x7fd8680685e0 2023-10-10T20:36:31.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.035+0000 7fd85f7fe700 1 -- 172.21.15.89:0/3435629573 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858024980 con 0x7fd8680685e0 2023-10-10T20:36:31.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.035+0000 7fd86d946700 1 -- 172.21.15.89:0/3435629573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd86814cbf0 con 0x7fd8680685e0 2023-10-10T20:36:31.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.036+0000 7fd86d946700 1 -- 172.21.15.89:0/3435629573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd86806d840 con 0x7fd8680685e0 2023-10-10T20:36:31.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.036+0000 7fd85f7fe700 1 -- 172.21.15.89:0/3435629573 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fd85801a420 con 0x7fd8680685e0 2023-10-10T20:36:31.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.037+0000 7fd85f7fe700 1 --2- 172.21.15.89:0/3435629573 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd848066fc0 0x7fd848069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.037+0000 7fd85e7fc700 1 --2- 172.21.15.89:0/3435629573 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd848066fc0 0x7fd848069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.037+0000 7fd85f7fe700 1 -- 172.21.15.89:0/3435629573 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd8580863d0 con 0x7fd8680685e0 2023-10-10T20:36:31.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.037+0000 7fd85e7fc700 1 --2- 172.21.15.89:0/3435629573 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd848066fc0 0x7fd848069470 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd86814f160 tx=0x7fd850009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:31.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.041+0000 7fd85f7fe700 1 -- 172.21.15.89:0/3435629573 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd858053650 con 0x7fd8680685e0 2023-10-10T20:36:31.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.224+0000 7fd86d946700 1 -- 172.21.15.89:0/3435629573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd86806c7f0 con 0x7fd8680685e0 2023-10-10T20:36:31.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.225+0000 7fd85f7fe700 1 -- 172.21.15.89:0/3435629573 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd858056c70 con 0x7fd8680685e0 2023-10-10T20:36:31.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.227+0000 7fd86d946700 1 -- 172.21.15.89:0/3435629573 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd848066fc0 msgr2=0x7fd848069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:31.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.227+0000 7fd86d946700 1 --2- 172.21.15.89:0/3435629573 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd848066fc0 0x7fd848069470 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd86814f160 tx=0x7fd850009040 comp rx=0 tx=0).stop 2023-10-10T20:36:31.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.227+0000 7fd86d946700 1 -- 172.21.15.89:0/3435629573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8680685e0 msgr2=0x7fd86816aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:31.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.227+0000 7fd86d946700 1 --2- 172.21.15.89:0/3435629573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8680685e0 0x7fd86816aa20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd86806b630 tx=0x7fd858017000 comp rx=0 tx=0).stop 2023-10-10T20:36:31.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.227+0000 7fd86d946700 1 -- 172.21.15.89:0/3435629573 shutdown_connections 2023-10-10T20:36:31.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.227+0000 7fd86d946700 1 --2- 172.21.15.89:0/3435629573 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd848066fc0 0x7fd848069470 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.227+0000 7fd86d946700 1 --2- 172.21.15.89:0/3435629573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd868159970 0x7fd86814e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.227+0000 7fd86d946700 1 --2- 172.21.15.89:0/3435629573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8680685e0 0x7fd86816aa20 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.227+0000 7fd86d946700 1 --2- 172.21.15.89:0/3435629573 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd8680689b0 0x7fd868151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.227+0000 7fd86d946700 1 -- 172.21.15.89:0/3435629573 >> 172.21.15.89:0/3435629573 conn(0x7fd86806a940 msgr2=0x7fd868156de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:31.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.227+0000 7fd86d946700 1 -- 172.21.15.89:0/3435629573 shutdown_connections 2023-10-10T20:36:31.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.227+0000 7fd86d946700 1 -- 172.21.15.89:0/3435629573 wait complete. 2023-10-10T20:36:31.241 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000078554 --mode=777 2023-10-10T20:36:31.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.356+0000 7f251604d700 1 Processor -- start 2023-10-10T20:36:31.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.357+0000 7f251604d700 1 -- start start 2023-10-10T20:36:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.357+0000 7f251604d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25100b03c0 0x7f25100cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.357+0000 7f251604d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f25100b0790 0x7f25100b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.357+0000 7f251604d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f25101591e0 0x7f2510159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.357+0000 7f251604d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25100cfa00 con 0x7f25100b0790 2023-10-10T20:36:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.357+0000 7f251604d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25100cb580 con 0x7f25101591e0 2023-10-10T20:36:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.357+0000 7f251604d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25100b2de0 con 0x7f25100b03c0 2023-10-10T20:36:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.358+0000 7f251484a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f25100b0790 0x7f25100b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.358+0000 7f251484a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f25100b0790 0x7f25100b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35266/0 (socket says 172.21.15.89:35266) 2023-10-10T20:36:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.358+0000 7f251484a700 1 -- 172.21.15.89:0/362451591 learned_addr learned my addr 172.21.15.89:0/362451591 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:31.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.358+0000 7f251584c700 1 --2- 172.21.15.89:0/362451591 >> v2:172.21.15.111:3300/0 conn(0x7f25101591e0 0x7f2510159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.358+0000 7f251504b700 1 --2- 172.21.15.89:0/362451591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25100b03c0 0x7f25100cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.358+0000 7f251484a700 1 -- 172.21.15.89:0/362451591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25100b03c0 msgr2=0x7f25100cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:31.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.358+0000 7f251484a700 1 --2- 172.21.15.89:0/362451591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25100b03c0 0x7f25100cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.358+0000 7f251484a700 1 -- 172.21.15.89:0/362451591 >> v2:172.21.15.111:3300/0 conn(0x7f25101591e0 msgr2=0x7f2510159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:31.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.358+0000 7f251484a700 1 --2- 172.21.15.89:0/362451591 >> v2:172.21.15.111:3300/0 conn(0x7f25101591e0 0x7f2510159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.358+0000 7f251484a700 1 -- 172.21.15.89:0/362451591 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25100cf160 con 0x7f25100b0790 2023-10-10T20:36:31.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.358+0000 7f251484a700 1 --2- 172.21.15.89:0/362451591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25100b0790 0x7f25100b28a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f25100cb4f0 tx=0x7f2500009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37242844f10faece server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:31.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.358+0000 7f2507fff700 1 -- 172.21.15.89:0/362451591 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250000b640 con 0x7f25100b0790 2023-10-10T20:36:31.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.358+0000 7f2507fff700 1 -- 172.21.15.89:0/362451591 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250000b7a0 con 0x7f25100b0790 2023-10-10T20:36:31.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.358+0000 7f2507fff700 1 -- 172.21.15.89:0/362451591 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250000bb60 con 0x7f25100b0790 2023-10-10T20:36:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.359+0000 7f251604d700 1 -- 172.21.15.89:0/362451591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25100b0790 msgr2=0x7f25100b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.359+0000 7f251604d700 1 --2- 172.21.15.89:0/362451591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25100b0790 0x7f25100b28a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f25100cb4f0 tx=0x7f2500009250 comp rx=0 tx=0).stop 2023-10-10T20:36:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.359+0000 7f251604d700 1 -- 172.21.15.89:0/362451591 shutdown_connections 2023-10-10T20:36:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.359+0000 7f251604d700 1 --2- 172.21.15.89:0/362451591 >> v2:172.21.15.111:3300/0 conn(0x7f25101591e0 0x7f2510159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.359+0000 7f251604d700 1 --2- 172.21.15.89:0/362451591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25100b03c0 0x7f25100cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.359+0000 7f251604d700 1 --2- 172.21.15.89:0/362451591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25100b0790 0x7f25100b28a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.359+0000 7f251604d700 1 -- 172.21.15.89:0/362451591 >> 172.21.15.89:0/362451591 conn(0x7f2510063230 msgr2=0x7f2510063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:31.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.359+0000 7f251604d700 1 -- 172.21.15.89:0/362451591 shutdown_connections 2023-10-10T20:36:31.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.359+0000 7f251604d700 1 -- 172.21.15.89:0/362451591 wait complete. 2023-10-10T20:36:31.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.360+0000 7f251604d700 1 Processor -- start 2023-10-10T20:36:31.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.360+0000 7f251604d700 1 -- start start 2023-10-10T20:36:31.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.361+0000 7f251604d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25100b03c0 0x7f25100c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.361+0000 7f251604d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25100b0790 0x7f251015c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.361+0000 7f251604d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25101591e0 0x7f251015cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.361+0000 7f251604d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25100c95d0 con 0x7f25100b03c0 2023-10-10T20:36:31.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.361+0000 7f251604d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251015d960 con 0x7f25101591e0 2023-10-10T20:36:31.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.361+0000 7f251604d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251015d0c0 con 0x7f25100b0790 2023-10-10T20:36:31.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.361+0000 7f251504b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25100b03c0 0x7f25100c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.361+0000 7f251504b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25100b03c0 0x7f25100c5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35274/0 (socket says 172.21.15.89:35274) 2023-10-10T20:36:31.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.361+0000 7f251504b700 1 -- 172.21.15.89:0/1647508620 learned_addr learned my addr 172.21.15.89:0/1647508620 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:31.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.361+0000 7f251584c700 1 --2- 172.21.15.89:0/1647508620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25101591e0 0x7f251015cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.361+0000 7f251484a700 1 --2- 172.21.15.89:0/1647508620 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25100b0790 0x7f251015c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.361+0000 7f251504b700 1 -- 172.21.15.89:0/1647508620 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25100b0790 msgr2=0x7f251015c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:31.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.361+0000 7f251504b700 1 --2- 172.21.15.89:0/1647508620 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25100b0790 0x7f251015c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.361+0000 7f251504b700 1 -- 172.21.15.89:0/1647508620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25101591e0 msgr2=0x7f251015cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:31.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.361+0000 7f251504b700 1 --2- 172.21.15.89:0/1647508620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25101591e0 0x7f251015cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.361+0000 7f251504b700 1 -- 172.21.15.89:0/1647508620 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25000094f0 con 0x7f25100b03c0 2023-10-10T20:36:31.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.361+0000 7f251504b700 1 --2- 172.21.15.89:0/1647508620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25100b03c0 0x7f25100c5490 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f250c009ef0 tx=0x7f250c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:31.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.362+0000 7f25057fa700 1 -- 172.21.15.89:0/1647508620 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c00cc00 con 0x7f25100b03c0 2023-10-10T20:36:31.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.362+0000 7f25057fa700 1 -- 172.21.15.89:0/1647508620 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250c00f040 con 0x7f25100b03c0 2023-10-10T20:36:31.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.362+0000 7f25057fa700 1 -- 172.21.15.89:0/1647508620 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c014480 con 0x7f25100b03c0 2023-10-10T20:36:31.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.362+0000 7f251604d700 1 -- 172.21.15.89:0/1647508620 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251015d3a0 con 0x7f25100b03c0 2023-10-10T20:36:31.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.362+0000 7f251604d700 1 -- 172.21.15.89:0/1647508620 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25101597e0 con 0x7f25100b03c0 2023-10-10T20:36:31.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.363+0000 7f251604d700 1 -- 172.21.15.89:0/1647508620 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251004c3c0 con 0x7f25100b03c0 2023-10-10T20:36:31.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.363+0000 7f25057fa700 1 -- 172.21.15.89:0/1647508620 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f250c0147f0 con 0x7f25100b03c0 2023-10-10T20:36:31.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.363+0000 7f25057fa700 1 --2- 172.21.15.89:0/1647508620 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f24f4066fc0 0x7f24f4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.363+0000 7f25057fa700 1 -- 172.21.15.89:0/1647508620 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f250c085050 con 0x7f25100b03c0 2023-10-10T20:36:31.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.363+0000 7f251484a700 1 --2- 172.21.15.89:0/1647508620 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f24f4066fc0 0x7f24f4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.364+0000 7f251484a700 1 --2- 172.21.15.89:0/1647508620 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f24f4066fc0 0x7f24f4069470 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f25100cb4f0 tx=0x7f2500017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:31.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.368+0000 7f25057fa700 1 -- 172.21.15.89:0/1647508620 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f250c0521c0 con 0x7f25100b03c0 2023-10-10T20:36:31.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.587+0000 7f251604d700 1 -- 172.21.15.89:0/1647508620 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000078554", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7f25100c9710 con 0x7f24f4066fc0 2023-10-10T20:36:31.590 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:36:31.589+0000 7f023b8aa700 -1 client.5538 remove_session_caps still has dirty|flushing caps on 0x10000000001.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2023-10-10T20:36:06.033931+0000 mtime=2023-10-10T20:36:06.033931+0000 ctime=2023-10-10T20:36:06.039545+0000 change_attr=2 caps=- flushing_caps=Ax COMPLETE parents=0x10000000000.head["group_0000000000303131"] has_dir_layout 0x55a1c6946000) 2023-10-10T20:36:31.594 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:36:31.592+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:36:31.594 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:36:31.592+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:36:31.595 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:36:31.592+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:36:31.595 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:36:31.592+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:36:31.595 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:36:31.592+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:36:31.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.617+0000 7f25057fa700 1 -- 172.21.15.89:0/1647508620 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2500000bc0 con 0x7f24f4066fc0 2023-10-10T20:36:31.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.619+0000 7f251604d700 1 -- 172.21.15.89:0/1647508620 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f24f4066fc0 msgr2=0x7f24f4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:31.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.619+0000 7f251604d700 1 --2- 172.21.15.89:0/1647508620 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f24f4066fc0 0x7f24f4069470 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f25100cb4f0 tx=0x7f2500017000 comp rx=0 tx=0).stop 2023-10-10T20:36:31.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.619+0000 7f251604d700 1 -- 172.21.15.89:0/1647508620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25100b03c0 msgr2=0x7f25100c5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:31.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.619+0000 7f251604d700 1 --2- 172.21.15.89:0/1647508620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25100b03c0 0x7f25100c5490 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f250c009ef0 tx=0x7f250c00b040 comp rx=0 tx=0).stop 2023-10-10T20:36:31.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.619+0000 7f251604d700 1 -- 172.21.15.89:0/1647508620 shutdown_connections 2023-10-10T20:36:31.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.619+0000 7f251604d700 1 --2- 172.21.15.89:0/1647508620 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f24f4066fc0 0x7f24f4069470 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.619+0000 7f251604d700 1 --2- 172.21.15.89:0/1647508620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25101591e0 0x7f251015cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.619+0000 7f251604d700 1 --2- 172.21.15.89:0/1647508620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25100b03c0 0x7f25100c5490 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.619+0000 7f251604d700 1 --2- 172.21.15.89:0/1647508620 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25100b0790 0x7f251015c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.619+0000 7f251604d700 1 -- 172.21.15.89:0/1647508620 >> 172.21.15.89:0/1647508620 conn(0x7f2510063230 msgr2=0x7f25100b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:31.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.619+0000 7f251604d700 1 -- 172.21.15.89:0/1647508620 shutdown_connections 2023-10-10T20:36:31.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.619+0000 7f251604d700 1 -- 172.21.15.89:0/1647508620 wait complete. 2023-10-10T20:36:31.633 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000078554 2023-10-10T20:36:31.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.748+0000 7f328ccd0700 1 Processor -- start 2023-10-10T20:36:31.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.748+0000 7f328ccd0700 1 -- start start 2023-10-10T20:36:31.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f328ccd0700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f32880cb1a0 0x7f328815a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f328ccd0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328806ebd0 con 0x7f328815a8a0 2023-10-10T20:36:31.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f328ccd0700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328815ad20 con 0x7f32880ae0d0 2023-10-10T20:36:31.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f328ccd0700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f328815b2e0 con 0x7f32880cb1a0 2023-10-10T20:36:31.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f3286ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f328815a8a0 0x7f328806f060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37254/0 (socket says 172.21.15.89:37254) 2023-10-10T20:36:31.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f3286ffd700 1 -- 172.21.15.89:0/1101747134 learned_addr learned my addr 172.21.15.89:0/1101747134 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:36:31.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f32867fc700 1 -- 172.21.15.89:0/1101747134 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1059248869 0 0) 0x7f328806ebd0 con 0x7f328815a8a0 2023-10-10T20:36:31.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f32877fe700 1 --2- 172.21.15.89:0/1101747134 >> v2:172.21.15.111:3300/0 conn(0x7f32880cb1a0 0x7f328815a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f32867fc700 1 -- 172.21.15.89:0/1101747134 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3274003570 con 0x7f328815a8a0 2023-10-10T20:36:31.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f32867fc700 1 -- 172.21.15.89:0/1101747134 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2267623377 0 0) 0x7f3274003570 con 0x7f328815a8a0 2023-10-10T20:36:31.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f32867fc700 1 -- 172.21.15.89:0/1101747134 >> v2:172.21.15.111:3300/0 conn(0x7f32880cb1a0 msgr2=0x7f328815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:31.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f32867fc700 1 --2- 172.21.15.89:0/1101747134 >> v2:172.21.15.111:3300/0 conn(0x7f32880cb1a0 0x7f328815a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f32867fc700 1 -- 172.21.15.89:0/1101747134 >> v1:172.21.15.111:6789/0 conn(0x7f32880ae0d0 legacy=0x7f32880ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:31.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f32867fc700 1 -- 172.21.15.89:0/1101747134 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328815b000 con 0x7f328815a8a0 2023-10-10T20:36:31.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f32867fc700 1 -- 172.21.15.89:0/1101747134 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f3278001470 con 0x7f328815a8a0 2023-10-10T20:36:31.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f32867fc700 1 -- 172.21.15.89:0/1101747134 >> v1:172.21.15.89:6789/0 conn(0x7f328815a8a0 legacy=0x7f328806f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:31.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f32867fc700 1 --2- 172.21.15.89:0/1101747134 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3274003a60 0x7f3274003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f32867fc700 1 --2- 172.21.15.89:0/1101747134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3274004810 0x7f3274004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f32867fc700 1 --2- 172.21.15.89:0/1101747134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32740051c0 0x7f327400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f32867fc700 1 -- 172.21.15.89:0/1101747134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328815b2e0 con 0x7f3274004810 2023-10-10T20:36:31.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f32867fc700 1 -- 172.21.15.89:0/1101747134 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327400c0d0 con 0x7f32740051c0 2023-10-10T20:36:31.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f32867fc700 1 -- 172.21.15.89:0/1101747134 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327400c290 con 0x7f3274003a60 2023-10-10T20:36:31.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.749+0000 7f3286ffd700 1 --2- 172.21.15.89:0/1101747134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3274004810 0x7f3274004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.750+0000 7f32877fe700 1 --2- 172.21.15.89:0/1101747134 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3274003a60 0x7f3274003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.750+0000 7f3287fff700 1 --2- 172.21.15.89:0/1101747134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32740051c0 0x7f327400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.750+0000 7f3286ffd700 1 -- 172.21.15.89:0/1101747134 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3274003a60 msgr2=0x7f3274003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:31.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.750+0000 7f3286ffd700 1 --2- 172.21.15.89:0/1101747134 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3274003a60 0x7f3274003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.750+0000 7f3286ffd700 1 -- 172.21.15.89:0/1101747134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32740051c0 msgr2=0x7f327400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:36:31.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.750+0000 7f3286ffd700 1 --2- 172.21.15.89:0/1101747134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32740051c0 0x7f327400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.750+0000 7f3286ffd700 1 -- 172.21.15.89:0/1101747134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3274003570 con 0x7f3274004810 2023-10-10T20:36:31.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.750+0000 7f3287fff700 1 --2- 172.21.15.89:0/1101747134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32740051c0 0x7f327400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:31.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.750+0000 7f3287fff700 1 -- 172.21.15.89:0/1101747134 reap_dead start 2023-10-10T20:36:31.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.750+0000 7f3286ffd700 1 --2- 172.21.15.89:0/1101747134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3274004810 0x7f3274004c80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f327800c390 tx=0x7f32880c8be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0b7db3e8a8e22e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:31.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.750+0000 7f32867fc700 1 -- 172.21.15.89:0/1101747134 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327800cc60 con 0x7f3274004810 2023-10-10T20:36:31.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.750+0000 7f32867fc700 1 -- 172.21.15.89:0/1101747134 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327800cdc0 con 0x7f3274004810 2023-10-10T20:36:31.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.750+0000 7f32867fc700 1 -- 172.21.15.89:0/1101747134 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3278005b70 con 0x7f3274004810 2023-10-10T20:36:31.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.751+0000 7f328ccd0700 1 -- 172.21.15.89:0/1101747134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3274004810 msgr2=0x7f3274004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:31.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.751+0000 7f328ccd0700 1 --2- 172.21.15.89:0/1101747134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3274004810 0x7f3274004c80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f327800c390 tx=0x7f32880c8be0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.751+0000 7f328ccd0700 1 -- 172.21.15.89:0/1101747134 shutdown_connections 2023-10-10T20:36:31.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.751+0000 7f328ccd0700 1 --2- 172.21.15.89:0/1101747134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3274004810 0x7f3274004c80 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.751+0000 7f328ccd0700 1 -- 172.21.15.89:0/1101747134 >> 172.21.15.89:0/1101747134 conn(0x7f32880cc1e0 msgr2=0x7f32880cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:31.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.751+0000 7f328ccd0700 1 -- 172.21.15.89:0/1101747134 shutdown_connections 2023-10-10T20:36:31.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.751+0000 7f328ccd0700 1 -- 172.21.15.89:0/1101747134 wait complete. 2023-10-10T20:36:31.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.752+0000 7f328ccd0700 1 Processor -- start 2023-10-10T20:36:31.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.752+0000 7f328ccd0700 1 -- start start 2023-10-10T20:36:31.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f328ccd0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3274004810 0x7f328806ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f328ccd0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f328816b240 0x7f328814e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f328ccd0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f32880af230 0x7f32880af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f328ccd0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32880afbe0 con 0x7f3274004810 2023-10-10T20:36:31.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f328ccd0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328814eac0 con 0x7f328816b240 2023-10-10T20:36:31.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f328ccd0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328816b7b0 con 0x7f32880af230 2023-10-10T20:36:31.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f32877fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3274004810 0x7f328806ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f32877fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3274004810 0x7f328806ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35320/0 (socket says 172.21.15.89:35320) 2023-10-10T20:36:31.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f32877fe700 1 -- 172.21.15.89:0/102038567 learned_addr learned my addr 172.21.15.89:0/102038567 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:36:31.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f3286ffd700 1 --2- 172.21.15.89:0/102038567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f328816b240 0x7f328814e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f3287fff700 1 --2- 172.21.15.89:0/102038567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f32880af230 0x7f32880af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f32877fe700 1 -- 172.21.15.89:0/102038567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f32880af230 msgr2=0x7f32880af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:31.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f32877fe700 1 --2- 172.21.15.89:0/102038567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f32880af230 0x7f32880af6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f32877fe700 1 -- 172.21.15.89:0/102038567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f328816b240 msgr2=0x7f328814e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:31.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f32877fe700 1 --2- 172.21.15.89:0/102038567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f328816b240 0x7f328814e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f32877fe700 1 -- 172.21.15.89:0/102038567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328816ba90 con 0x7f3274004810 2023-10-10T20:36:31.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f3286ffd700 1 --2- 172.21.15.89:0/102038567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f328816b240 0x7f328814e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:36:31.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f32877fe700 1 --2- 172.21.15.89:0/102038567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3274004810 0x7f328806ab90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3270011d40 tx=0x7f32700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:31.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f327f7fe700 1 -- 172.21.15.89:0/102038567 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327000fac0 con 0x7f3274004810 2023-10-10T20:36:31.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f327f7fe700 1 -- 172.21.15.89:0/102038567 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327000fc20 con 0x7f3274004810 2023-10-10T20:36:31.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f328ccd0700 1 -- 172.21.15.89:0/102038567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328816bd70 con 0x7f3274004810 2023-10-10T20:36:31.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f327f7fe700 1 -- 172.21.15.89:0/102038567 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3270014440 con 0x7f3274004810 2023-10-10T20:36:31.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.753+0000 7f328ccd0700 1 -- 172.21.15.89:0/102038567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32881537a0 con 0x7f3274004810 2023-10-10T20:36:31.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.754+0000 7f328ccd0700 1 -- 172.21.15.89:0/102038567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f328814eda0 con 0x7f3274004810 2023-10-10T20:36:31.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.755+0000 7f327f7fe700 1 -- 172.21.15.89:0/102038567 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f3270014a80 con 0x7f3274004810 2023-10-10T20:36:31.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.755+0000 7f327f7fe700 1 --2- 172.21.15.89:0/102038567 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3268066fc0 0x7f3268069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:36:31.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.755+0000 7f327f7fe700 1 -- 172.21.15.89:0/102038567 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3270018030 con 0x7f3274004810 2023-10-10T20:36:31.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.755+0000 7f3286ffd700 1 --2- 172.21.15.89:0/102038567 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3268066fc0 0x7f3268069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:36:31.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.755+0000 7f3286ffd700 1 --2- 172.21.15.89:0/102038567 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3268066fc0 0x7f3268069470 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3278007cd0 tx=0x7f3278015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:36:31.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.759+0000 7f327f7fe700 1 -- 172.21.15.89:0/102038567 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3270052550 con 0x7f3274004810 2023-10-10T20:36:31.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.974+0000 7f328ccd0700 1 -- 172.21.15.89:0/102038567 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000078554", "target": ["mon-mgr", ""]}) v1 -- 0x7f328806e860 con 0x7f3268066fc0 2023-10-10T20:36:31.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.979+0000 7f327f7fe700 1 -- 172.21.15.89:0/102038567 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f328806e860 con 0x7f3268066fc0 2023-10-10T20:36:31.981 INFO:teuthology.orchestra.run.smithi089.stdout:/volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89 2023-10-10T20:36:31.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.980+0000 7f328ccd0700 1 -- 172.21.15.89:0/102038567 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3268066fc0 msgr2=0x7f3268069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:31.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.980+0000 7f328ccd0700 1 --2- 172.21.15.89:0/102038567 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3268066fc0 0x7f3268069470 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3278007cd0 tx=0x7f3278015000 comp rx=0 tx=0).stop 2023-10-10T20:36:31.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.980+0000 7f328ccd0700 1 -- 172.21.15.89:0/102038567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3274004810 msgr2=0x7f328806ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:36:31.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.980+0000 7f328ccd0700 1 --2- 172.21.15.89:0/102038567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3274004810 0x7f328806ab90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3270011d40 tx=0x7f32700091a0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.981+0000 7f328ccd0700 1 -- 172.21.15.89:0/102038567 shutdown_connections 2023-10-10T20:36:31.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.981+0000 7f328ccd0700 1 --2- 172.21.15.89:0/102038567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f32880af230 0x7f32880af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.981+0000 7f328ccd0700 1 --2- 172.21.15.89:0/102038567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3274004810 0x7f328806ab90 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.981+0000 7f328ccd0700 1 --2- 172.21.15.89:0/102038567 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3268066fc0 0x7f3268069470 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.981+0000 7f328ccd0700 1 --2- 172.21.15.89:0/102038567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f328816b240 0x7f328814e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:36:31.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.981+0000 7f328ccd0700 1 -- 172.21.15.89:0/102038567 >> 172.21.15.89:0/102038567 conn(0x7f32880cc1e0 msgr2=0x7f328806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:36:31.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.981+0000 7f328ccd0700 1 -- 172.21.15.89:0/102038567 shutdown_connections 2023-10-10T20:36:31.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:36:31.981+0000 7f328ccd0700 1 -- 172.21.15.89:0/102038567 wait complete. 2023-10-10T20:36:31.995 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000078554 with 3000 files each 0.1MB size under directory volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89 2023-10-10T20:36:31.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:31.995 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.0 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:32.021 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:32.021 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:32.021 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.5894e-05 s, 0.0 kB/s 2023-10-10T20:36:32.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:32.022 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:32.078 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:32.078 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:32.078 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8238e-05 s, 0.0 kB/s 2023-10-10T20:36:32.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:32.079 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:32.132 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:32.133 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:32.133 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2666e-05 s, 0.0 kB/s 2023-10-10T20:36:32.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:32.134 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.3 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:32.188 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:32.188 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:32.188 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.291e-05 s, 0.0 kB/s 2023-10-10T20:36:32.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:32.189 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.4 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9182e-05 s, 0.0 kB/s 2023-10-10T20:36:32.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:32.244 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.5 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:32.297 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:32.298 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:32.298 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9095e-05 s, 0.0 kB/s 2023-10-10T20:36:32.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:32.299 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.6 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:32.352 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0943e-05 s, 0.0 kB/s 2023-10-10T20:36:32.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:32.353 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.7 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:32.407 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8929e-05 s, 0.0 kB/s 2023-10-10T20:36:32.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:32.409 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.8 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:32.462 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:32.463 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:32.463 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6933e-05 s, 0.0 kB/s 2023-10-10T20:36:32.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:32.464 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.9 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9159e-05 s, 0.0 kB/s 2023-10-10T20:36:32.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:32.519 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.10 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:32.572 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:32.572 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:32.572 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.935e-05 s, 0.0 kB/s 2023-10-10T20:36:32.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:32.574 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.11 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:32.630 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:32.630 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:32.630 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0636e-05 s, 0.0 kB/s 2023-10-10T20:36:32.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:32.631 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.12 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:32.685 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:32.685 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:32.685 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2719e-05 s, 0.0 kB/s 2023-10-10T20:36:32.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:32.686 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.13 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:32.741 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0962e-05 s, 0.0 kB/s 2023-10-10T20:36:32.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:32.741 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.14 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:32.796 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2056e-05 s, 0.0 kB/s 2023-10-10T20:36:32.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:32.798 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.15 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:32.852 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:32.852 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:32.852 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2105e-05 s, 0.0 kB/s 2023-10-10T20:36:32.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:32.853 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.16 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:32.907 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:32.907 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:32.908 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3989e-05 s, 0.0 kB/s 2023-10-10T20:36:32.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:32.908 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.17 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2027e-05 s, 0.0 kB/s 2023-10-10T20:36:32.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:32.964 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.18 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:33.022 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:33.022 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:33.022 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1145e-05 s, 0.0 kB/s 2023-10-10T20:36:33.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:33.023 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.19 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:33.078 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9349e-05 s, 0.0 kB/s 2023-10-10T20:36:33.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:33.078 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.20 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2351e-05 s, 0.0 kB/s 2023-10-10T20:36:33.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:33.134 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.21 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:33.188 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:33.188 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:33.189 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2974e-05 s, 0.0 kB/s 2023-10-10T20:36:33.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:33.189 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.22 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1876e-05 s, 0.0 kB/s 2023-10-10T20:36:33.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:33.245 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.23 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:33.299 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:33.299 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:33.299 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7011e-05 s, 0.0 kB/s 2023-10-10T20:36:33.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:33.300 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.24 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9357e-05 s, 0.0 kB/s 2023-10-10T20:36:33.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:33.355 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.25 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:33.408 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:33.409 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:33.409 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7319e-05 s, 0.0 kB/s 2023-10-10T20:36:33.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:33.410 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.26 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:33.464 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:33.464 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:33.464 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0519e-05 s, 0.0 kB/s 2023-10-10T20:36:33.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:33.465 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.27 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:33.519 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:33.519 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:33.519 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9589e-05 s, 0.0 kB/s 2023-10-10T20:36:33.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:33.520 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.28 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6642e-05 s, 0.0 kB/s 2023-10-10T20:36:33.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:33.575 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.29 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9709e-05 s, 0.0 kB/s 2023-10-10T20:36:33.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:33.630 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.30 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:33.684 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:33.685 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:33.685 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1561e-05 s, 0.0 kB/s 2023-10-10T20:36:33.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:33.686 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.31 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:33.740 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:33.740 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:33.740 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7078e-05 s, 0.0 kB/s 2023-10-10T20:36:33.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:33.741 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.32 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:33.795 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:33.795 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:33.795 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7337e-05 s, 0.0 kB/s 2023-10-10T20:36:33.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:33.796 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.33 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:33.850 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:33.850 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:33.850 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.092e-05 s, 0.0 kB/s 2023-10-10T20:36:33.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:33.851 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.34 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:33.905 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:33.905 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:33.905 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7807e-05 s, 0.0 kB/s 2023-10-10T20:36:33.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:33.906 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.35 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:33.960 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:33.960 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:33.960 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9218e-05 s, 0.0 kB/s 2023-10-10T20:36:33.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:33.961 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.36 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:34.016 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:34.016 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:34.016 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8993e-05 s, 0.0 kB/s 2023-10-10T20:36:34.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:34.017 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.37 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:34.071 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:34.071 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:34.071 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9201e-05 s, 0.0 kB/s 2023-10-10T20:36:34.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:34.072 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.38 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:34.126 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:34.126 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:34.126 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9249e-05 s, 0.0 kB/s 2023-10-10T20:36:34.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:34.127 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.39 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:34.181 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:34.181 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:34.181 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8728e-05 s, 0.0 kB/s 2023-10-10T20:36:34.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:34.182 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.40 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.112e-05 s, 0.0 kB/s 2023-10-10T20:36:34.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:34.237 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.41 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:34.291 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:34.291 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:34.292 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8232e-05 s, 0.0 kB/s 2023-10-10T20:36:34.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:34.292 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.42 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:34.346 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:34.346 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:34.347 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.941e-05 s, 0.0 kB/s 2023-10-10T20:36:34.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:34.347 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.43 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:34.402 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:34.403 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:34.403 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8407e-05 s, 0.0 kB/s 2023-10-10T20:36:34.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:34.404 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.44 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:34.458 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:34.458 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:34.458 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8384e-05 s, 0.0 kB/s 2023-10-10T20:36:34.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:34.459 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.45 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:34.512 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:34.513 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:34.513 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1589e-05 s, 0.0 kB/s 2023-10-10T20:36:34.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:34.514 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.46 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:34.567 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:34.567 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:34.567 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8621e-05 s, 0.0 kB/s 2023-10-10T20:36:34.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:34.569 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.47 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:34.624 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:34.624 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:34.624 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1396e-05 s, 0.0 kB/s 2023-10-10T20:36:34.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:34.625 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.48 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:34.679 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:34.679 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:34.680 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3352e-05 s, 0.0 kB/s 2023-10-10T20:36:34.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:34.680 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.49 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:34.734 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:34.734 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:34.735 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9037e-05 s, 0.0 kB/s 2023-10-10T20:36:34.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:34.736 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.50 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:34.790 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:34.790 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:34.790 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0964e-05 s, 0.0 kB/s 2023-10-10T20:36:34.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:34.791 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.51 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:34.845 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:34.845 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:34.845 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9775e-05 s, 0.0 kB/s 2023-10-10T20:36:34.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:34.846 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.52 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:34.899 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:34.900 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:34.900 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8489e-05 s, 0.0 kB/s 2023-10-10T20:36:34.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:34.901 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.53 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:34.955 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:34.955 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:34.956 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0142e-05 s, 0.0 kB/s 2023-10-10T20:36:34.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:34.956 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.54 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:35.011 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:35.011 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:35.012 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.003e-05 s, 0.0 kB/s 2023-10-10T20:36:35.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:35.013 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.55 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:35.066 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:35.066 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:35.066 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9395e-05 s, 0.0 kB/s 2023-10-10T20:36:35.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:35.067 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.56 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:35.121 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:35.121 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:35.122 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7406e-05 s, 0.0 kB/s 2023-10-10T20:36:35.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:35.123 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.57 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:35.177 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:35.177 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:35.178 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7807e-05 s, 0.0 kB/s 2023-10-10T20:36:35.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:35.178 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.58 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:35.232 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:35.232 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:35.232 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1196e-05 s, 0.0 kB/s 2023-10-10T20:36:35.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:35.233 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.59 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:35.287 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:35.287 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:35.288 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.717e-05 s, 0.0 kB/s 2023-10-10T20:36:35.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:35.288 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.60 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:35.341 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:35.342 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:35.342 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9315e-05 s, 0.0 kB/s 2023-10-10T20:36:35.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:35.343 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.61 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0723e-05 s, 0.0 kB/s 2023-10-10T20:36:35.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:35.398 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.62 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:35.452 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:35.452 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:35.452 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8123e-05 s, 0.0 kB/s 2023-10-10T20:36:35.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:35.453 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.63 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:35.508 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:35.508 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:35.508 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.018e-05 s, 0.0 kB/s 2023-10-10T20:36:35.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:35.509 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.64 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:35.563 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:35.563 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:35.563 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0242e-05 s, 0.0 kB/s 2023-10-10T20:36:35.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:35.564 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.65 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8659e-05 s, 0.0 kB/s 2023-10-10T20:36:35.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:35.619 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.66 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:35.673 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:35.673 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:35.673 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9155e-05 s, 0.0 kB/s 2023-10-10T20:36:35.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:35.674 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.67 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:35.728 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:35.728 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:35.728 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1268e-05 s, 0.0 kB/s 2023-10-10T20:36:35.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:35.729 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.68 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:35.782 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5756e-05 s, 0.0 kB/s 2023-10-10T20:36:35.784 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:35.784 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.69 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:35.839 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:35.839 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:35.839 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9163e-05 s, 0.0 kB/s 2023-10-10T20:36:35.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:35.840 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.70 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:35.896 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:35.896 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:35.896 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9408e-05 s, 0.0 kB/s 2023-10-10T20:36:35.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:35.897 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.71 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:35.951 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:35.951 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:35.951 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7461e-05 s, 0.0 kB/s 2023-10-10T20:36:35.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:35.952 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.72 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:36.006 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:36.006 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:36.006 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0866e-05 s, 0.0 kB/s 2023-10-10T20:36:36.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:36.007 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.73 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:36.060 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:36.060 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:36.061 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2359e-05 s, 0.0 kB/s 2023-10-10T20:36:36.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:36.063 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.74 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:36.118 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:36.119 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:36.119 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0451e-05 s, 0.0 kB/s 2023-10-10T20:36:36.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:36.120 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.75 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:36.174 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:36.174 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:36.175 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8679e-05 s, 0.0 kB/s 2023-10-10T20:36:36.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:36.176 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.76 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:36.229 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:36.230 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:36.230 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9529e-05 s, 0.0 kB/s 2023-10-10T20:36:36.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:36.230 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.77 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:36.284 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:36.284 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:36.285 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.822e-05 s, 0.0 kB/s 2023-10-10T20:36:36.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:36.285 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.78 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:36.340 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:36.340 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:36.340 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9669e-05 s, 0.0 kB/s 2023-10-10T20:36:36.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:36.341 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.79 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:36.395 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:36.395 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:36.396 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9033e-05 s, 0.0 kB/s 2023-10-10T20:36:36.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:36.396 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.80 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:36.452 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:36.452 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:36.452 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3213e-05 s, 0.0 kB/s 2023-10-10T20:36:36.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:36.453 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.81 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:36.507 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:36.507 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:36.508 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9367e-05 s, 0.0 kB/s 2023-10-10T20:36:36.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:36.508 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.82 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:36.563 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:36.563 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:36.563 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9926e-05 s, 0.0 kB/s 2023-10-10T20:36:36.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:36.564 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.83 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:36.618 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:36.618 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:36.618 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2709e-05 s, 0.0 kB/s 2023-10-10T20:36:36.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:36.619 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.84 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:36.672 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:36.673 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:36.673 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4715e-05 s, 0.0 kB/s 2023-10-10T20:36:36.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:36.674 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.85 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:36.729 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:36.729 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:36.729 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.296e-05 s, 0.0 kB/s 2023-10-10T20:36:36.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:36.730 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.86 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:36.784 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:36.785 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:36.785 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9891e-05 s, 0.0 kB/s 2023-10-10T20:36:36.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:36.786 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.87 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:36.840 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:36.840 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:36.840 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.118e-05 s, 0.0 kB/s 2023-10-10T20:36:36.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:36.841 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.88 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6322e-05 s, 0.0 kB/s 2023-10-10T20:36:36.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:36.897 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.89 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.193e-05 s, 0.0 kB/s 2023-10-10T20:36:36.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:36.953 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.90 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:37.008 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:37.008 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:37.008 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2511e-05 s, 0.0 kB/s 2023-10-10T20:36:37.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:37.009 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.91 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:37.063 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:37.064 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:37.064 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8765e-05 s, 0.0 kB/s 2023-10-10T20:36:37.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:37.064 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.92 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:37.119 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:37.119 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:37.119 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1987e-05 s, 0.0 kB/s 2023-10-10T20:36:37.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:37.120 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.93 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:37.173 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:37.174 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:37.174 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0647e-05 s, 0.0 kB/s 2023-10-10T20:36:37.174 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:37.175 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.94 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:37.229 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:37.229 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:37.229 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1136e-05 s, 0.0 kB/s 2023-10-10T20:36:37.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:37.230 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.95 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:37.284 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:37.284 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:37.284 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9466e-05 s, 0.0 kB/s 2023-10-10T20:36:37.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:37.285 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.96 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:37.339 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:37.339 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:37.339 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.079e-05 s, 0.0 kB/s 2023-10-10T20:36:37.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:37.340 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.97 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:37.394 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:37.394 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:37.394 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0189e-05 s, 0.0 kB/s 2023-10-10T20:36:37.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:37.395 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.98 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:37.449 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:37.449 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:37.449 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7592e-05 s, 0.0 kB/s 2023-10-10T20:36:37.450 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:37.450 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.99 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:37.503 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:37.504 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:37.504 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2503e-05 s, 0.0 kB/s 2023-10-10T20:36:37.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:37.505 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.100 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:37.559 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:37.560 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:37.560 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7774e-05 s, 0.0 kB/s 2023-10-10T20:36:37.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:37.561 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.101 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:37.615 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:37.615 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:37.615 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7885e-05 s, 0.0 kB/s 2023-10-10T20:36:37.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:37.616 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.102 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:37.670 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:37.670 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:37.670 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7914e-05 s, 0.0 kB/s 2023-10-10T20:36:37.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:37.671 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.103 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:37.725 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:37.725 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:37.725 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7109e-05 s, 0.0 kB/s 2023-10-10T20:36:37.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:37.726 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.104 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:37.779 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:37.779 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:37.780 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.809e-05 s, 0.0 kB/s 2023-10-10T20:36:37.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:37.781 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.105 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:37.836 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:37.836 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:37.836 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7941e-05 s, 0.0 kB/s 2023-10-10T20:36:37.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:37.837 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.106 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:37.891 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:37.891 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:37.891 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0157e-05 s, 0.0 kB/s 2023-10-10T20:36:37.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:37.892 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.107 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:37.947 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1069e-05 s, 0.0 kB/s 2023-10-10T20:36:37.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:37.949 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.108 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:38.003 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:38.004 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:38.004 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0003e-05 s, 0.0 kB/s 2023-10-10T20:36:38.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:38.006 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.109 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:38.059 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:38.059 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:38.059 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2503e-05 s, 0.0 kB/s 2023-10-10T20:36:38.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:38.060 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.110 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:38.114 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:38.114 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:38.114 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0801e-05 s, 0.0 kB/s 2023-10-10T20:36:38.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:38.115 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.111 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:38.168 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:38.168 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:38.169 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7276e-05 s, 0.0 kB/s 2023-10-10T20:36:38.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:38.170 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.112 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:38.225 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:38.225 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:38.225 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7608e-05 s, 0.0 kB/s 2023-10-10T20:36:38.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:38.226 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.113 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:38.280 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:38.280 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:38.280 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7511e-05 s, 0.0 kB/s 2023-10-10T20:36:38.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:38.281 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.114 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:38.335 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:38.336 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:38.336 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.261e-05 s, 0.0 kB/s 2023-10-10T20:36:38.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:38.337 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.115 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8173e-05 s, 0.0 kB/s 2023-10-10T20:36:38.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:38.393 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.116 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:38.447 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:38.447 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:38.447 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8135e-05 s, 0.0 kB/s 2023-10-10T20:36:38.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:38.448 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.117 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2869e-05 s, 0.0 kB/s 2023-10-10T20:36:38.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:38.503 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.118 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:38.558 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:38.558 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:38.558 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0889e-05 s, 0.0 kB/s 2023-10-10T20:36:38.558 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:38.559 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.119 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:38.613 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:38.613 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:38.614 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3012e-05 s, 0.0 kB/s 2023-10-10T20:36:38.614 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:38.614 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.120 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:38.669 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:38.669 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:38.670 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.777e-05 s, 0.0 kB/s 2023-10-10T20:36:38.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:38.670 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.121 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:38.724 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:38.725 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:38.725 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3113e-05 s, 0.0 kB/s 2023-10-10T20:36:38.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:38.726 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.122 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:38.780 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:38.780 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:38.780 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1753e-05 s, 0.0 kB/s 2023-10-10T20:36:38.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:38.781 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.123 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:38.835 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:38.835 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:38.835 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2315e-05 s, 0.0 kB/s 2023-10-10T20:36:38.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:38.836 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.124 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:38.889 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:38.890 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:38.890 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.845e-05 s, 0.0 kB/s 2023-10-10T20:36:38.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:38.891 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.125 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:38.945 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:38.945 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:38.946 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9544e-05 s, 0.0 kB/s 2023-10-10T20:36:38.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:38.946 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.126 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:39.001 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:39.001 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:39.001 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1136e-05 s, 0.0 kB/s 2023-10-10T20:36:39.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:39.002 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.127 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:39.056 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:39.056 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:39.056 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.881e-05 s, 0.0 kB/s 2023-10-10T20:36:39.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:39.057 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.128 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:39.111 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:39.111 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:39.111 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3769e-05 s, 0.0 kB/s 2023-10-10T20:36:39.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:39.112 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.129 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:39.166 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:39.166 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:39.167 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7864e-05 s, 0.0 kB/s 2023-10-10T20:36:39.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:39.167 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.130 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:39.221 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:39.221 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:39.221 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1509e-05 s, 0.0 kB/s 2023-10-10T20:36:39.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:39.222 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.131 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:39.276 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:39.276 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:39.276 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6713e-05 s, 0.0 kB/s 2023-10-10T20:36:39.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:39.277 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.132 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:39.331 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:39.331 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:39.331 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0991e-05 s, 0.0 kB/s 2023-10-10T20:36:39.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:39.332 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.133 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:39.386 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:39.386 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:39.386 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.813e-05 s, 0.0 kB/s 2023-10-10T20:36:39.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:39.387 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.134 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:39.441 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:39.441 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:39.441 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0415e-05 s, 0.0 kB/s 2023-10-10T20:36:39.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:39.442 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.135 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:39.496 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:39.496 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:39.497 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9467e-05 s, 0.0 kB/s 2023-10-10T20:36:39.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:39.498 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.136 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:39.551 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:39.551 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:39.552 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.859e-05 s, 0.0 kB/s 2023-10-10T20:36:39.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:39.552 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.137 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0907e-05 s, 0.0 kB/s 2023-10-10T20:36:39.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:39.607 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.138 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:39.661 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:39.662 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:39.662 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7554e-05 s, 0.0 kB/s 2023-10-10T20:36:39.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:39.663 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.139 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:39.716 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:39.717 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:39.717 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.886e-05 s, 0.0 kB/s 2023-10-10T20:36:39.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:39.718 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.140 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:39.772 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:39.773 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:39.773 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8915e-05 s, 0.0 kB/s 2023-10-10T20:36:39.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:39.774 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.141 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:39.829 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9321e-05 s, 0.0 kB/s 2023-10-10T20:36:39.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:39.829 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.142 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:39.882 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:39.883 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:39.883 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0499e-05 s, 0.0 kB/s 2023-10-10T20:36:39.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:39.884 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.143 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:39.938 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:39.939 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:39.939 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1321e-05 s, 0.0 kB/s 2023-10-10T20:36:39.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:39.940 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.144 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:39.994 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:39.994 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:39.995 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2309e-05 s, 0.0 kB/s 2023-10-10T20:36:39.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:39.995 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.145 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:40.051 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:40.051 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:40.052 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5148e-05 s, 0.0 kB/s 2023-10-10T20:36:40.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:40.052 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.146 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:40.106 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:40.107 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:40.107 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8596e-05 s, 0.0 kB/s 2023-10-10T20:36:40.107 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:40.108 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.147 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7627e-05 s, 0.0 kB/s 2023-10-10T20:36:40.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:40.163 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.148 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:40.217 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:40.217 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:40.218 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3476e-05 s, 0.0 kB/s 2023-10-10T20:36:40.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:40.218 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.149 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:40.272 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:40.273 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:40.273 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8204e-05 s, 0.0 kB/s 2023-10-10T20:36:40.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:40.274 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.150 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:40.327 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:40.328 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:40.328 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1938e-05 s, 0.0 kB/s 2023-10-10T20:36:40.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:40.329 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.151 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:40.383 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:40.383 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:40.383 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7322e-05 s, 0.0 kB/s 2023-10-10T20:36:40.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:40.384 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.152 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:40.438 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:40.438 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:40.438 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0173e-05 s, 0.0 kB/s 2023-10-10T20:36:40.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:40.439 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.153 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:40.492 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:40.493 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:40.493 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0803e-05 s, 0.0 kB/s 2023-10-10T20:36:40.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:40.494 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.154 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:40.547 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:40.548 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:40.548 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8178e-05 s, 0.0 kB/s 2023-10-10T20:36:40.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:40.549 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.155 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:40.602 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:40.603 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:40.603 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8656e-05 s, 0.0 kB/s 2023-10-10T20:36:40.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:40.604 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.156 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:40.658 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:40.658 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:40.658 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7381e-05 s, 0.0 kB/s 2023-10-10T20:36:40.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:40.659 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.157 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:40.713 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:40.713 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:40.713 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2212e-05 s, 0.0 kB/s 2023-10-10T20:36:40.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:40.714 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.158 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:40.768 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:40.768 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:40.768 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2414e-05 s, 0.0 kB/s 2023-10-10T20:36:40.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:40.769 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.159 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:40.823 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:40.823 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:40.823 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.283e-05 s, 0.0 kB/s 2023-10-10T20:36:40.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:40.824 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.160 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:40.878 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:40.878 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:40.879 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7172e-05 s, 0.0 kB/s 2023-10-10T20:36:40.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:40.879 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.161 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:40.934 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:40.934 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:40.934 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7742e-05 s, 0.0 kB/s 2023-10-10T20:36:40.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:40.935 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.162 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:40.989 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:40.989 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:40.989 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0454e-05 s, 0.0 kB/s 2023-10-10T20:36:40.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:40.990 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.163 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:41.044 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:41.044 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:41.044 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8544e-05 s, 0.0 kB/s 2023-10-10T20:36:41.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:41.045 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.164 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:41.098 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:41.099 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:41.099 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8971e-05 s, 0.0 kB/s 2023-10-10T20:36:41.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:41.100 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.165 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:41.154 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:41.154 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:41.155 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.712e-05 s, 0.0 kB/s 2023-10-10T20:36:41.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:41.155 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.166 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:41.211 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:41.212 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:41.212 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.982e-05 s, 0.0 kB/s 2023-10-10T20:36:41.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:41.212 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.167 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:41.267 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:41.267 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:41.268 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9827e-05 s, 0.0 kB/s 2023-10-10T20:36:41.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:41.269 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.168 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:41.323 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:41.324 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:41.324 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9829e-05 s, 0.0 kB/s 2023-10-10T20:36:41.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:41.325 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.169 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:41.379 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:41.379 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:41.379 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0048e-05 s, 0.0 kB/s 2023-10-10T20:36:41.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:41.380 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.170 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:41.435 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:41.436 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:41.436 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7494e-05 s, 0.0 kB/s 2023-10-10T20:36:41.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:41.437 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.171 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:41.491 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:41.491 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:41.491 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2787e-05 s, 0.0 kB/s 2023-10-10T20:36:41.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:41.492 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.172 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8687e-05 s, 0.0 kB/s 2023-10-10T20:36:41.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:41.547 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.173 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:41.601 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:41.602 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:41.602 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2265e-05 s, 0.0 kB/s 2023-10-10T20:36:41.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:41.603 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.174 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:41.657 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:41.658 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:41.658 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9882e-05 s, 0.0 kB/s 2023-10-10T20:36:41.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:41.659 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.175 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:41.712 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:41.713 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:41.713 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.013e-05 s, 0.0 kB/s 2023-10-10T20:36:41.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:41.714 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.176 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:41.768 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:41.768 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:41.768 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1581e-05 s, 0.0 kB/s 2023-10-10T20:36:41.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:41.769 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.177 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:41.823 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:41.823 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:41.823 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0173e-05 s, 0.0 kB/s 2023-10-10T20:36:41.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:41.824 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.178 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:41.831 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:36:41.866 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:36:41.878 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:41.878 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:41.879 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8866e-05 s, 0.0 kB/s 2023-10-10T20:36:41.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:41.879 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.179 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:41.933 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:41.934 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:41.934 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1451e-05 s, 0.0 kB/s 2023-10-10T20:36:41.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:41.935 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.180 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:42.009 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:42.009 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:42.010 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1433e-05 s, 0.0 kB/s 2023-10-10T20:36:42.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:42.011 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.181 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:42.066 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:42.067 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:42.067 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8943e-05 s, 0.0 kB/s 2023-10-10T20:36:42.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:42.067 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.182 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:42.122 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:42.122 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:42.122 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8465e-05 s, 0.0 kB/s 2023-10-10T20:36:42.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:42.123 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.183 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:42.177 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:42.177 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:42.177 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7956e-05 s, 0.0 kB/s 2023-10-10T20:36:42.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:42.178 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.184 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:42.232 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:42.232 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:42.232 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0987e-05 s, 0.0 kB/s 2023-10-10T20:36:42.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:42.233 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.185 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:42.287 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:42.287 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:42.287 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8481e-05 s, 0.0 kB/s 2023-10-10T20:36:42.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:42.288 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.186 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:42.342 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:42.342 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:42.342 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8022e-05 s, 0.0 kB/s 2023-10-10T20:36:42.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:42.343 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.187 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:42.397 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:42.397 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:42.397 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1831e-05 s, 0.0 kB/s 2023-10-10T20:36:42.397 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:42.398 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.188 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:42.452 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:42.452 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:42.452 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8442e-05 s, 0.0 kB/s 2023-10-10T20:36:42.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:42.453 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.189 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:42.507 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:42.507 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:42.508 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.138e-05 s, 0.0 kB/s 2023-10-10T20:36:42.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:42.508 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.190 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:42.562 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:42.562 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:42.562 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.961e-05 s, 0.0 kB/s 2023-10-10T20:36:42.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:42.563 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.191 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:42.617 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:42.617 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:42.617 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.79e-05 s, 0.0 kB/s 2023-10-10T20:36:42.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:42.618 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.192 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:42.671 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:42.671 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:42.672 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0787e-05 s, 0.0 kB/s 2023-10-10T20:36:42.672 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:42.672 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.193 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:42.727 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:42.727 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:42.727 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8456e-05 s, 0.0 kB/s 2023-10-10T20:36:42.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:42.728 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.194 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:42.782 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:42.782 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:42.782 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7485e-05 s, 0.0 kB/s 2023-10-10T20:36:42.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:42.783 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.195 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:42.837 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:42.838 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:42.838 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9199e-05 s, 0.0 kB/s 2023-10-10T20:36:42.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:42.839 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.196 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:42.893 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:42.893 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:42.893 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9589e-05 s, 0.0 kB/s 2023-10-10T20:36:42.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:42.894 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.197 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:42.948 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:42.949 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:42.949 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7415e-05 s, 0.0 kB/s 2023-10-10T20:36:42.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:42.949 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.198 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:43.004 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:43.004 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:43.004 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0965e-05 s, 0.0 kB/s 2023-10-10T20:36:43.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:43.005 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.199 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:43.059 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:43.059 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:43.060 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7474e-05 s, 0.0 kB/s 2023-10-10T20:36:43.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:43.061 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.200 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:43.115 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:43.115 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:43.115 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.789e-05 s, 0.0 kB/s 2023-10-10T20:36:43.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:43.116 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.201 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:43.170 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:43.170 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:43.170 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7627e-05 s, 0.0 kB/s 2023-10-10T20:36:43.171 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:43.171 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.202 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:43.224 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:43.225 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:43.225 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.762e-05 s, 0.0 kB/s 2023-10-10T20:36:43.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:43.226 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.203 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:43.281 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:43.281 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:43.281 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2226e-05 s, 0.0 kB/s 2023-10-10T20:36:43.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:43.282 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.204 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:43.336 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:43.336 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:43.336 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9326e-05 s, 0.0 kB/s 2023-10-10T20:36:43.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:43.337 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.205 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:43.391 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:43.391 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:43.392 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9095e-05 s, 0.0 kB/s 2023-10-10T20:36:43.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:43.392 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.206 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:43.447 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:43.447 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:43.447 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0983e-05 s, 0.0 kB/s 2023-10-10T20:36:43.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:43.448 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.207 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:43.502 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:43.502 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:43.503 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0152e-05 s, 0.0 kB/s 2023-10-10T20:36:43.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:43.503 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.208 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7808e-05 s, 0.0 kB/s 2023-10-10T20:36:43.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:43.559 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.209 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:43.613 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:43.613 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:43.613 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9865e-05 s, 0.0 kB/s 2023-10-10T20:36:43.613 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:43.614 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.210 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:43.667 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:43.667 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:43.668 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6904e-05 s, 0.0 kB/s 2023-10-10T20:36:43.669 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:43.669 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.211 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:43.724 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:43.724 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:43.724 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1289e-05 s, 0.0 kB/s 2023-10-10T20:36:43.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:43.725 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.212 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:43.779 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:43.779 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:43.779 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.841e-05 s, 0.0 kB/s 2023-10-10T20:36:43.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:43.780 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.213 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:43.833 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:43.834 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:43.834 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8974e-05 s, 0.0 kB/s 2023-10-10T20:36:43.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:43.835 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.214 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:43.890 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:43.890 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:43.890 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7199e-05 s, 0.0 kB/s 2023-10-10T20:36:43.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:43.891 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.215 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:43.945 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:43.945 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:43.945 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2882e-05 s, 0.0 kB/s 2023-10-10T20:36:43.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:43.946 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.216 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:44.000 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:44.000 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:44.000 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8522e-05 s, 0.0 kB/s 2023-10-10T20:36:44.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:44.001 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.217 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:44.055 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:44.055 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:44.055 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7938e-05 s, 0.0 kB/s 2023-10-10T20:36:44.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:44.056 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.218 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:44.110 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:44.110 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:44.110 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8449e-05 s, 0.0 kB/s 2023-10-10T20:36:44.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:44.112 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.219 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:44.165 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:44.165 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:44.165 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8658e-05 s, 0.0 kB/s 2023-10-10T20:36:44.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:44.166 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.220 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:44.220 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:44.220 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:44.220 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7497e-05 s, 0.0 kB/s 2023-10-10T20:36:44.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:44.221 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.221 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:44.276 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:44.276 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:44.277 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7226e-05 s, 0.0 kB/s 2023-10-10T20:36:44.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:44.277 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.222 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:44.331 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:44.331 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:44.332 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.67e-05 s, 0.0 kB/s 2023-10-10T20:36:44.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:44.332 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.223 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:44.386 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:44.386 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:44.386 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6913e-05 s, 0.0 kB/s 2023-10-10T20:36:44.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:44.387 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.224 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:44.440 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:44.440 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:44.441 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0643e-05 s, 0.0 kB/s 2023-10-10T20:36:44.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:44.442 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.225 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:44.497 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:44.497 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:44.497 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8625e-05 s, 0.0 kB/s 2023-10-10T20:36:44.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:44.498 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.226 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:44.552 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:44.552 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:44.552 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1088e-05 s, 0.0 kB/s 2023-10-10T20:36:44.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:44.553 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.227 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:44.607 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:44.607 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:44.607 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1976e-05 s, 0.0 kB/s 2023-10-10T20:36:44.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:44.608 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.228 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:44.662 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:44.662 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:44.663 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.796e-05 s, 0.0 kB/s 2023-10-10T20:36:44.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:44.663 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.229 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:44.717 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:44.717 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:44.717 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1839e-05 s, 0.0 kB/s 2023-10-10T20:36:44.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:44.718 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.230 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:44.772 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:44.772 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:44.772 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8489e-05 s, 0.0 kB/s 2023-10-10T20:36:44.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:44.773 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.231 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:44.826 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:44.827 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:44.827 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7869e-05 s, 0.0 kB/s 2023-10-10T20:36:44.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:44.828 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.232 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:44.883 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0603e-05 s, 0.0 kB/s 2023-10-10T20:36:44.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:44.883 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.233 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:44.938 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:44.938 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:44.938 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9643e-05 s, 0.0 kB/s 2023-10-10T20:36:44.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:44.939 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.234 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:44.993 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:44.993 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:44.994 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8336e-05 s, 0.0 kB/s 2023-10-10T20:36:44.994 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:44.994 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.235 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:45.047 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:45.048 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:45.048 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.062e-05 s, 0.0 kB/s 2023-10-10T20:36:45.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:45.048 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.236 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:45.103 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:45.103 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:45.103 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8539e-05 s, 0.0 kB/s 2023-10-10T20:36:45.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:45.104 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.237 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9294e-05 s, 0.0 kB/s 2023-10-10T20:36:45.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:45.159 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.238 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:45.213 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:45.213 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:45.213 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2028e-05 s, 0.0 kB/s 2023-10-10T20:36:45.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:45.214 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.239 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:45.268 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:45.268 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:45.268 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7469e-05 s, 0.0 kB/s 2023-10-10T20:36:45.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:45.269 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.240 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:45.323 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:45.323 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:45.324 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8615e-05 s, 0.0 kB/s 2023-10-10T20:36:45.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:45.324 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.241 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:45.379 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:45.379 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:45.379 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7414e-05 s, 0.0 kB/s 2023-10-10T20:36:45.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:45.380 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.242 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:45.433 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:45.434 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:45.434 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2376e-05 s, 0.0 kB/s 2023-10-10T20:36:45.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:45.435 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.243 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:45.489 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:45.489 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:45.489 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0018e-05 s, 0.0 kB/s 2023-10-10T20:36:45.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:45.490 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.244 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:45.545 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:45.545 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:45.546 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.354e-05 s, 0.0 kB/s 2023-10-10T20:36:45.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:45.547 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.245 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:45.601 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:45.601 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:45.601 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.26e-05 s, 0.0 kB/s 2023-10-10T20:36:45.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:45.602 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.246 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:45.656 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:45.656 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:45.657 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6491e-05 s, 0.0 kB/s 2023-10-10T20:36:45.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:45.657 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.247 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:45.711 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:45.712 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:45.712 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.102e-05 s, 0.0 kB/s 2023-10-10T20:36:45.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:45.713 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.248 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:45.766 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:45.767 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:45.767 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1655e-05 s, 0.0 kB/s 2023-10-10T20:36:45.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:45.768 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.249 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:45.821 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:45.822 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:45.822 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7461e-05 s, 0.0 kB/s 2023-10-10T20:36:45.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:45.823 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.250 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:45.876 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:45.876 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:45.876 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9613e-05 s, 0.0 kB/s 2023-10-10T20:36:45.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:45.878 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.251 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:45.933 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:45.933 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:45.933 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.918e-05 s, 0.0 kB/s 2023-10-10T20:36:45.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:45.934 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.252 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:45.988 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:45.988 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:45.989 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8273e-05 s, 0.0 kB/s 2023-10-10T20:36:45.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:45.989 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.253 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:46.044 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:46.044 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:46.045 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8648e-05 s, 0.0 kB/s 2023-10-10T20:36:46.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:46.046 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.254 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:46.099 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:46.099 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:46.100 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9674e-05 s, 0.0 kB/s 2023-10-10T20:36:46.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:46.100 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.255 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:46.153 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:46.154 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:46.154 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3006e-05 s, 0.0 kB/s 2023-10-10T20:36:46.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:46.155 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.256 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:46.209 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:46.209 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:46.209 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1869e-05 s, 0.0 kB/s 2023-10-10T20:36:46.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:46.210 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.257 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:46.264 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:46.264 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:46.264 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7824e-05 s, 0.0 kB/s 2023-10-10T20:36:46.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:46.265 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.258 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:46.319 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:46.319 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:46.319 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9934e-05 s, 0.0 kB/s 2023-10-10T20:36:46.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:46.320 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.259 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:46.373 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:46.374 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:46.374 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7712e-05 s, 0.0 kB/s 2023-10-10T20:36:46.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:46.375 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.260 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:46.429 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:46.430 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:46.430 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9313e-05 s, 0.0 kB/s 2023-10-10T20:36:46.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:46.431 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.261 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:46.484 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:46.485 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:46.485 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.7735e-05 s, 0.0 kB/s 2023-10-10T20:36:46.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:46.486 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.262 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:46.540 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:46.541 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:46.541 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9985e-05 s, 0.0 kB/s 2023-10-10T20:36:46.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:46.542 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.263 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:46.596 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:46.596 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:46.596 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6463e-05 s, 0.0 kB/s 2023-10-10T20:36:46.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:46.597 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.264 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0391e-05 s, 0.0 kB/s 2023-10-10T20:36:46.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:46.652 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.265 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:46.706 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:46.706 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:46.707 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7178e-05 s, 0.0 kB/s 2023-10-10T20:36:46.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:46.707 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.266 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:46.762 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:46.762 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:46.762 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7709e-05 s, 0.0 kB/s 2023-10-10T20:36:46.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:46.763 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.267 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3306e-05 s, 0.0 kB/s 2023-10-10T20:36:46.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:46.818 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.268 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:46.872 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:46.872 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:46.872 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1874e-05 s, 0.0 kB/s 2023-10-10T20:36:46.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:46.873 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.269 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:46.927 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:46.928 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:46.928 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6724e-05 s, 0.0 kB/s 2023-10-10T20:36:46.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:46.929 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.270 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:46.985 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:46.985 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:46.985 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2983e-05 s, 0.0 kB/s 2023-10-10T20:36:46.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:46.986 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.271 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:47.040 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:47.040 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:47.040 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2641e-05 s, 0.0 kB/s 2023-10-10T20:36:47.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:47.041 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.272 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:47.095 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:47.095 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:47.096 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.999e-05 s, 0.0 kB/s 2023-10-10T20:36:47.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:47.096 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.273 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:47.150 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:47.150 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:47.151 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.072e-05 s, 0.0 kB/s 2023-10-10T20:36:47.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:47.151 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.274 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:47.205 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:47.205 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:47.205 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0083e-05 s, 0.0 kB/s 2023-10-10T20:36:47.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:47.207 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.275 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:47.262 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:47.262 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:47.262 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9282e-05 s, 0.0 kB/s 2023-10-10T20:36:47.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:47.263 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.276 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:47.317 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:47.317 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:47.317 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1844e-05 s, 0.0 kB/s 2023-10-10T20:36:47.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:47.318 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.277 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:47.372 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:47.372 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:47.372 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1201e-05 s, 0.0 kB/s 2023-10-10T20:36:47.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:47.373 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.278 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:47.427 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:47.427 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:47.427 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0226e-05 s, 0.0 kB/s 2023-10-10T20:36:47.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:47.428 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.279 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:47.482 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:47.482 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:47.482 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8327e-05 s, 0.0 kB/s 2023-10-10T20:36:47.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:47.483 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.280 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:47.537 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:47.538 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:47.538 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.237e-05 s, 0.0 kB/s 2023-10-10T20:36:47.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:47.539 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.281 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:47.593 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:47.593 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:47.593 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9895e-05 s, 0.0 kB/s 2023-10-10T20:36:47.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:47.594 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.282 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:47.648 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:47.648 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:47.649 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1103e-05 s, 0.0 kB/s 2023-10-10T20:36:47.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:47.649 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.283 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:47.704 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:47.704 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:47.704 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3945e-05 s, 0.0 kB/s 2023-10-10T20:36:47.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:47.705 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.284 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:47.759 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:47.759 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:47.760 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8919e-05 s, 0.0 kB/s 2023-10-10T20:36:47.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:47.760 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.285 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:47.815 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:47.815 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:47.815 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1931e-05 s, 0.0 kB/s 2023-10-10T20:36:47.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:47.816 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.286 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:47.870 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:47.870 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:47.870 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1299e-05 s, 0.0 kB/s 2023-10-10T20:36:47.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:47.871 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.287 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:47.925 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:47.925 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:47.925 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6112e-05 s, 0.0 kB/s 2023-10-10T20:36:47.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:47.927 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.288 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:47.983 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:47.983 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:47.983 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1458e-05 s, 0.0 kB/s 2023-10-10T20:36:47.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:47.984 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.289 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:48.039 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:48.039 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:48.039 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0261e-05 s, 0.0 kB/s 2023-10-10T20:36:48.040 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:48.040 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.290 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:48.094 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:48.094 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:48.094 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8654e-05 s, 0.0 kB/s 2023-10-10T20:36:48.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:48.095 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.291 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:48.149 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:48.149 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:48.150 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0417e-05 s, 0.0 kB/s 2023-10-10T20:36:48.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:48.150 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.292 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:48.205 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:48.206 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:48.206 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7561e-05 s, 0.0 kB/s 2023-10-10T20:36:48.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:48.207 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.293 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:48.261 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:48.261 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:48.261 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7968e-05 s, 0.0 kB/s 2023-10-10T20:36:48.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:48.262 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.294 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:48.316 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:48.316 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:48.316 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9632e-05 s, 0.0 kB/s 2023-10-10T20:36:48.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:48.317 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.295 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:48.371 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:48.371 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:48.371 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1323e-05 s, 0.0 kB/s 2023-10-10T20:36:48.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:48.372 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.296 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:48.426 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:48.426 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:48.426 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0668e-05 s, 0.0 kB/s 2023-10-10T20:36:48.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:48.428 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.297 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:48.483 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:48.483 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:48.483 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7603e-05 s, 0.0 kB/s 2023-10-10T20:36:48.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:48.484 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.298 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:48.538 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:48.538 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:48.538 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2063e-05 s, 0.0 kB/s 2023-10-10T20:36:48.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:48.539 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.299 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:48.593 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:48.594 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:48.594 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0422e-05 s, 0.0 kB/s 2023-10-10T20:36:48.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:48.595 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.300 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:48.649 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:48.650 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:48.650 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6596e-05 s, 0.0 kB/s 2023-10-10T20:36:48.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:48.651 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.301 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:48.705 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:48.706 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:48.706 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.071e-05 s, 0.0 kB/s 2023-10-10T20:36:48.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:48.707 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.302 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:48.761 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:48.762 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:48.762 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0802e-05 s, 0.0 kB/s 2023-10-10T20:36:48.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:48.763 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.303 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:48.817 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:48.817 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:48.817 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0925e-05 s, 0.0 kB/s 2023-10-10T20:36:48.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:48.818 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.304 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:48.872 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:48.872 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:48.872 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8448e-05 s, 0.0 kB/s 2023-10-10T20:36:48.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:48.873 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.305 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:48.927 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:48.927 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:48.927 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1515e-05 s, 0.0 kB/s 2023-10-10T20:36:48.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:48.928 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.306 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:48.983 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:48.983 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:48.983 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.109e-05 s, 0.0 kB/s 2023-10-10T20:36:48.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:48.984 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.307 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:49.038 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:49.038 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:49.038 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.837e-05 s, 0.0 kB/s 2023-10-10T20:36:49.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:49.039 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.308 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:49.094 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:49.094 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:49.094 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3594e-05 s, 0.0 kB/s 2023-10-10T20:36:49.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:49.095 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.309 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:49.149 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:49.149 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:49.149 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9981e-05 s, 0.0 kB/s 2023-10-10T20:36:49.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:49.150 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.310 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:49.204 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:49.204 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:49.204 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0264e-05 s, 0.0 kB/s 2023-10-10T20:36:49.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:49.205 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.311 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:49.259 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:49.260 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:49.260 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1209e-05 s, 0.0 kB/s 2023-10-10T20:36:49.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:49.261 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.312 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:49.315 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:49.315 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:49.316 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2546e-05 s, 0.0 kB/s 2023-10-10T20:36:49.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:49.316 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.313 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:49.371 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:49.371 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:49.372 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9834e-05 s, 0.0 kB/s 2023-10-10T20:36:49.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:49.372 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.314 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:49.427 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:49.428 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:49.428 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.862e-05 s, 0.0 kB/s 2023-10-10T20:36:49.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:49.428 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.315 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:49.483 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:49.484 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:49.484 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8469e-05 s, 0.0 kB/s 2023-10-10T20:36:49.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:49.485 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.316 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:49.539 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:49.539 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:49.539 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.071e-05 s, 0.0 kB/s 2023-10-10T20:36:49.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:49.540 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.317 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:49.594 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:49.594 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:49.594 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1686e-05 s, 0.0 kB/s 2023-10-10T20:36:49.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:49.595 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.318 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:49.648 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:49.649 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:49.649 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0615e-05 s, 0.0 kB/s 2023-10-10T20:36:49.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:49.650 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.319 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:49.704 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:49.704 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:49.705 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3616e-05 s, 0.0 kB/s 2023-10-10T20:36:49.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:49.705 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.320 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:49.759 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:49.759 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:49.759 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0316e-05 s, 0.0 kB/s 2023-10-10T20:36:49.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:49.760 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.321 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:49.814 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:49.814 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:49.814 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0474e-05 s, 0.0 kB/s 2023-10-10T20:36:49.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:49.815 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.322 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:49.869 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:49.869 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:49.870 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0074e-05 s, 0.0 kB/s 2023-10-10T20:36:49.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:49.870 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.323 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:49.924 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:49.924 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:49.924 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0266e-05 s, 0.0 kB/s 2023-10-10T20:36:49.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:49.925 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.324 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:49.979 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:49.979 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:49.979 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.84e-05 s, 0.0 kB/s 2023-10-10T20:36:49.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:49.980 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.325 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:50.034 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:50.034 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:50.035 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8923e-05 s, 0.0 kB/s 2023-10-10T20:36:50.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:50.035 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.326 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:50.089 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:50.090 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:50.090 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9824e-05 s, 0.0 kB/s 2023-10-10T20:36:50.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:50.091 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.327 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:50.145 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:50.145 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:50.145 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8479e-05 s, 0.0 kB/s 2023-10-10T20:36:50.146 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:50.146 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.328 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:50.200 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:50.200 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:50.200 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8547e-05 s, 0.0 kB/s 2023-10-10T20:36:50.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:50.201 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.329 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:50.255 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:50.255 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:50.256 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8291e-05 s, 0.0 kB/s 2023-10-10T20:36:50.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:50.256 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.330 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:50.311 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:50.311 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:50.311 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5723e-05 s, 0.0 kB/s 2023-10-10T20:36:50.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:50.312 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.331 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:50.365 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:50.366 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:50.366 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8109e-05 s, 0.0 kB/s 2023-10-10T20:36:50.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:50.367 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.332 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:50.421 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:50.421 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:50.421 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2673e-05 s, 0.0 kB/s 2023-10-10T20:36:50.422 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:50.422 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.333 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:50.475 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:50.476 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:50.476 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6358e-05 s, 0.0 kB/s 2023-10-10T20:36:50.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:50.477 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.334 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:50.530 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:50.531 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:50.531 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6348e-05 s, 0.0 kB/s 2023-10-10T20:36:50.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:50.532 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.335 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:50.586 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:50.586 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:50.586 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7162e-05 s, 0.0 kB/s 2023-10-10T20:36:50.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:50.587 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.336 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:50.641 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:50.641 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:50.641 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.692e-05 s, 0.0 kB/s 2023-10-10T20:36:50.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:50.642 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.337 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:50.696 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:50.696 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:50.696 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5897e-05 s, 0.0 kB/s 2023-10-10T20:36:50.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:50.697 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.338 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:50.750 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:50.751 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:50.751 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9773e-05 s, 0.0 kB/s 2023-10-10T20:36:50.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:50.752 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.339 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:50.806 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:50.806 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:50.806 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8777e-05 s, 0.0 kB/s 2023-10-10T20:36:50.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:50.807 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.340 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:50.861 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:50.862 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:50.862 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9281e-05 s, 0.0 kB/s 2023-10-10T20:36:50.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:50.862 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.341 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:50.917 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:50.917 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:50.917 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8243e-05 s, 0.0 kB/s 2023-10-10T20:36:50.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:50.918 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.342 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:50.972 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0813e-05 s, 0.0 kB/s 2023-10-10T20:36:50.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:50.974 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.343 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:51.028 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:51.028 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:51.028 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8628e-05 s, 0.0 kB/s 2023-10-10T20:36:51.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:51.029 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.344 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:51.083 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:51.083 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:51.083 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7449e-05 s, 0.0 kB/s 2023-10-10T20:36:51.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:51.084 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.345 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:51.138 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:51.138 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:51.138 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4303e-05 s, 0.0 kB/s 2023-10-10T20:36:51.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:51.139 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.346 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:51.193 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:51.193 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:51.193 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0345e-05 s, 0.0 kB/s 2023-10-10T20:36:51.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:51.194 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.347 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:51.248 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:51.248 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:51.249 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8105e-05 s, 0.0 kB/s 2023-10-10T20:36:51.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:51.249 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.348 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:51.304 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:51.304 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:51.304 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0401e-05 s, 0.0 kB/s 2023-10-10T20:36:51.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:51.305 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.349 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:51.359 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:51.359 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:51.359 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.741e-05 s, 0.0 kB/s 2023-10-10T20:36:51.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:51.360 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.350 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:51.415 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:51.415 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:51.415 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9997e-05 s, 0.0 kB/s 2023-10-10T20:36:51.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:51.416 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.351 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:51.470 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:51.471 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:51.471 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3495e-05 s, 0.0 kB/s 2023-10-10T20:36:51.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:51.472 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.352 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:51.526 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:51.526 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:51.527 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8628e-05 s, 0.0 kB/s 2023-10-10T20:36:51.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:51.527 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.353 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:51.582 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:51.582 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:51.583 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8346e-05 s, 0.0 kB/s 2023-10-10T20:36:51.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:51.583 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.354 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:51.638 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:51.638 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:51.638 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7571e-05 s, 0.0 kB/s 2023-10-10T20:36:51.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:51.639 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.355 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:51.693 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:51.693 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:51.693 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0112e-05 s, 0.0 kB/s 2023-10-10T20:36:51.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:51.694 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.356 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2053e-05 s, 0.0 kB/s 2023-10-10T20:36:51.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:51.749 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.357 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:51.803 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:51.803 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:51.803 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9797e-05 s, 0.0 kB/s 2023-10-10T20:36:51.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:51.804 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.358 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:51.858 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:51.859 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:51.859 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.681e-05 s, 0.0 kB/s 2023-10-10T20:36:51.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:51.859 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.359 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:51.916 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:51.916 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:51.916 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1116e-05 s, 0.0 kB/s 2023-10-10T20:36:51.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:51.917 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.360 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8533e-05 s, 0.0 kB/s 2023-10-10T20:36:51.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:51.973 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.361 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:52.030 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:52.031 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:52.031 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0175e-05 s, 0.0 kB/s 2023-10-10T20:36:52.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:52.032 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.362 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:52.085 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:52.085 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:52.085 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9329e-05 s, 0.0 kB/s 2023-10-10T20:36:52.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:52.086 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.363 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1384e-05 s, 0.0 kB/s 2023-10-10T20:36:52.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:52.141 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.364 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:52.195 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:52.195 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:52.195 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2171e-05 s, 0.0 kB/s 2023-10-10T20:36:52.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:52.196 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.365 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:52.250 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:52.250 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:52.251 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1645e-05 s, 0.0 kB/s 2023-10-10T20:36:52.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:52.251 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.366 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3077e-05 s, 0.0 kB/s 2023-10-10T20:36:52.307 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:52.307 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.367 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:52.361 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:52.361 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:52.361 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8719e-05 s, 0.0 kB/s 2023-10-10T20:36:52.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:52.362 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.368 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:52.416 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:52.416 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:52.416 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.79e-05 s, 0.0 kB/s 2023-10-10T20:36:52.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:52.417 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.369 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:52.471 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:52.471 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:52.471 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1439e-05 s, 0.0 kB/s 2023-10-10T20:36:52.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:52.472 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.370 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:52.525 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:52.526 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:52.526 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6352e-05 s, 0.0 kB/s 2023-10-10T20:36:52.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:52.527 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.371 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:52.581 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:52.581 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:52.581 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7779e-05 s, 0.0 kB/s 2023-10-10T20:36:52.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:52.582 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.372 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:52.636 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:52.636 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:52.636 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6596e-05 s, 0.0 kB/s 2023-10-10T20:36:52.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:52.637 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.373 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:52.691 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:52.691 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:52.692 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0436e-05 s, 0.0 kB/s 2023-10-10T20:36:52.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:52.692 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.374 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:52.747 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:52.748 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:52.748 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7669e-05 s, 0.0 kB/s 2023-10-10T20:36:52.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:52.748 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.375 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:52.803 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:52.803 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:52.804 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7498e-05 s, 0.0 kB/s 2023-10-10T20:36:52.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:52.804 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.376 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:52.857 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:52.858 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:52.858 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5711e-05 s, 0.0 kB/s 2023-10-10T20:36:52.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:52.859 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.377 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:52.913 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:52.913 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:52.914 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1555e-05 s, 0.0 kB/s 2023-10-10T20:36:52.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:52.914 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.378 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:52.968 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:52.968 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:52.969 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.009e-05 s, 0.0 kB/s 2023-10-10T20:36:52.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:52.969 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.379 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:53.023 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:53.023 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:53.023 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6863e-05 s, 0.0 kB/s 2023-10-10T20:36:53.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:53.024 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.380 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:53.078 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:53.078 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:53.079 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7298e-05 s, 0.0 kB/s 2023-10-10T20:36:53.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:53.079 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.381 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:53.135 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:53.136 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:53.136 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6855e-05 s, 0.0 kB/s 2023-10-10T20:36:53.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:53.137 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.382 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:53.191 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:53.191 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:53.192 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.929e-05 s, 0.0 kB/s 2023-10-10T20:36:53.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:53.193 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.383 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:53.248 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:53.275 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:53.275 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9034e-05 s, 0.0 kB/s 2023-10-10T20:36:53.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:53.276 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.384 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:53.303 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:53.303 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:53.303 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2238e-05 s, 0.0 kB/s 2023-10-10T20:36:53.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:53.304 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.385 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:53.357 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:53.358 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:53.358 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0488e-05 s, 0.0 kB/s 2023-10-10T20:36:53.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:53.359 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.386 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:53.414 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:53.415 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:53.415 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8332e-05 s, 0.0 kB/s 2023-10-10T20:36:53.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:53.416 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.387 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:53.470 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:53.470 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:53.470 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7616e-05 s, 0.0 kB/s 2023-10-10T20:36:53.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:53.471 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.388 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:53.525 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:53.526 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:53.526 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2098e-05 s, 0.0 kB/s 2023-10-10T20:36:53.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:53.527 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.389 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:53.581 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:53.581 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:53.582 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1019e-05 s, 0.0 kB/s 2023-10-10T20:36:53.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:53.582 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.390 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:53.637 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:53.637 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:53.637 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.184e-05 s, 0.0 kB/s 2023-10-10T20:36:53.638 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:53.638 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.391 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:53.692 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:53.692 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:53.692 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4309e-05 s, 0.0 kB/s 2023-10-10T20:36:53.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:53.693 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.392 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:53.746 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:53.746 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:53.747 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.123e-05 s, 0.0 kB/s 2023-10-10T20:36:53.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:53.748 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.393 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:53.803 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:53.803 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:53.803 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1556e-05 s, 0.0 kB/s 2023-10-10T20:36:53.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:53.804 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.394 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:53.858 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:53.858 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:53.858 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6366e-05 s, 0.0 kB/s 2023-10-10T20:36:53.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:53.859 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.395 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:53.912 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:53.913 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:53.913 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9695e-05 s, 0.0 kB/s 2023-10-10T20:36:53.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:53.914 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.396 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:53.968 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:53.968 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:53.969 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9192e-05 s, 0.0 kB/s 2023-10-10T20:36:53.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:53.969 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.397 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:54.024 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:54.024 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:54.024 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7748e-05 s, 0.0 kB/s 2023-10-10T20:36:54.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:54.025 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.398 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:54.079 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:54.080 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:54.080 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.183e-05 s, 0.0 kB/s 2023-10-10T20:36:54.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:54.081 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.399 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:54.134 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:54.135 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:54.135 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9736e-05 s, 0.0 kB/s 2023-10-10T20:36:54.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:54.136 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.400 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:54.189 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:54.190 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:54.190 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8611e-05 s, 0.0 kB/s 2023-10-10T20:36:54.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:54.191 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.401 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:54.245 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:54.245 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:54.245 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7573e-05 s, 0.0 kB/s 2023-10-10T20:36:54.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:54.246 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.402 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:54.300 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:54.300 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:54.300 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0386e-05 s, 0.0 kB/s 2023-10-10T20:36:54.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:54.301 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.403 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:54.354 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:54.355 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:54.355 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.141e-05 s, 0.0 kB/s 2023-10-10T20:36:54.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:54.356 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.404 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:54.410 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:54.410 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:54.411 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8779e-05 s, 0.0 kB/s 2023-10-10T20:36:54.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:54.411 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.405 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:54.466 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:54.466 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:54.467 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7213e-05 s, 0.0 kB/s 2023-10-10T20:36:54.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:54.467 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.406 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:54.521 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:54.521 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:54.521 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0947e-05 s, 0.0 kB/s 2023-10-10T20:36:54.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:54.522 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.407 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:54.576 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:54.576 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:54.576 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1111e-05 s, 0.0 kB/s 2023-10-10T20:36:54.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:54.577 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.408 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:54.631 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:54.631 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:54.631 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6891e-05 s, 0.0 kB/s 2023-10-10T20:36:54.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:54.632 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.409 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:54.686 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:54.686 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:54.686 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2938e-05 s, 0.0 kB/s 2023-10-10T20:36:54.687 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:54.687 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.410 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:54.741 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:54.741 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:54.741 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0254e-05 s, 0.0 kB/s 2023-10-10T20:36:54.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:54.742 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.411 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:54.795 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:54.796 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:54.796 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7831e-05 s, 0.0 kB/s 2023-10-10T20:36:54.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:54.797 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.412 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:54.851 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:54.851 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:54.851 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1119e-05 s, 0.0 kB/s 2023-10-10T20:36:54.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:54.852 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.413 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:54.906 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:54.906 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:54.906 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1281e-05 s, 0.0 kB/s 2023-10-10T20:36:54.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:54.907 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.414 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7781e-05 s, 0.0 kB/s 2023-10-10T20:36:54.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:54.962 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.415 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:55.015 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:55.015 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:55.015 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8584e-05 s, 0.0 kB/s 2023-10-10T20:36:55.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:55.017 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.416 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.5691e-05 s, 0.0 kB/s 2023-10-10T20:36:55.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:55.073 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.417 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:55.127 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:55.127 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:55.127 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0855e-05 s, 0.0 kB/s 2023-10-10T20:36:55.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:55.128 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.418 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:55.182 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:55.182 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:55.183 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0203e-05 s, 0.0 kB/s 2023-10-10T20:36:55.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:55.183 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.419 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:55.238 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:55.239 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:55.239 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7691e-05 s, 0.0 kB/s 2023-10-10T20:36:55.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:55.240 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.420 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:55.294 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:55.294 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:55.294 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4447e-05 s, 0.0 kB/s 2023-10-10T20:36:55.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:55.295 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.421 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:55.349 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:55.350 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:55.350 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1005e-05 s, 0.0 kB/s 2023-10-10T20:36:55.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:55.351 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.422 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:55.405 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:55.405 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:55.405 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0635e-05 s, 0.0 kB/s 2023-10-10T20:36:55.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:55.406 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.423 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:55.460 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:55.460 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:55.461 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6892e-05 s, 0.0 kB/s 2023-10-10T20:36:55.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:55.461 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.424 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:55.515 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:55.516 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:55.516 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7896e-05 s, 0.0 kB/s 2023-10-10T20:36:55.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:55.517 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.425 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:55.571 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:55.571 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:55.571 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.7666e-05 s, 0.0 kB/s 2023-10-10T20:36:55.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:55.572 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.426 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1688e-05 s, 0.0 kB/s 2023-10-10T20:36:55.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:55.628 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.427 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:55.682 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:55.682 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:55.682 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8832e-05 s, 0.0 kB/s 2023-10-10T20:36:55.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:55.683 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.428 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:55.736 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:55.737 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:55.737 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7526e-05 s, 0.0 kB/s 2023-10-10T20:36:55.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:55.738 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.429 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:55.792 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:55.792 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:55.793 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.7219e-05 s, 0.0 kB/s 2023-10-10T20:36:55.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:55.793 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.430 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:55.848 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:55.848 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:55.848 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6925e-05 s, 0.0 kB/s 2023-10-10T20:36:55.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:55.849 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.431 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:55.903 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:55.903 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:55.903 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6265e-05 s, 0.0 kB/s 2023-10-10T20:36:55.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:55.904 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.432 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:55.958 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:55.959 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:55.959 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8135e-05 s, 0.0 kB/s 2023-10-10T20:36:55.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:55.960 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.433 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:56.014 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:56.014 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:56.015 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8362e-05 s, 0.0 kB/s 2023-10-10T20:36:56.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:56.015 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.434 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:56.070 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:56.070 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:56.071 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7633e-05 s, 0.0 kB/s 2023-10-10T20:36:56.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:56.071 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.435 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:56.125 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:56.126 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:56.126 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8617e-05 s, 0.0 kB/s 2023-10-10T20:36:56.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:56.127 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.436 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:56.181 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:56.182 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:56.182 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4135e-05 s, 0.0 kB/s 2023-10-10T20:36:56.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:56.183 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.437 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:56.237 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:56.237 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:56.237 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8651e-05 s, 0.0 kB/s 2023-10-10T20:36:56.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:56.239 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.438 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:56.294 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:56.294 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:56.295 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0775e-05 s, 0.0 kB/s 2023-10-10T20:36:56.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:56.295 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.439 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:56.349 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:56.349 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:56.349 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.719e-05 s, 0.0 kB/s 2023-10-10T20:36:56.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:56.350 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.440 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:56.405 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:56.406 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:56.406 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2553e-05 s, 0.0 kB/s 2023-10-10T20:36:56.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:56.407 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.441 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9994e-05 s, 0.0 kB/s 2023-10-10T20:36:56.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:56.463 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.442 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:56.517 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:56.518 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:56.518 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9488e-05 s, 0.0 kB/s 2023-10-10T20:36:56.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:56.519 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.443 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1309e-05 s, 0.0 kB/s 2023-10-10T20:36:56.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:56.574 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.444 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:56.628 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:56.628 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:56.628 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.081e-05 s, 0.0 kB/s 2023-10-10T20:36:56.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:56.629 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.445 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:56.683 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:56.684 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:56.684 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1672e-05 s, 0.0 kB/s 2023-10-10T20:36:56.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:56.685 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.446 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:56.739 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:56.739 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:56.740 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0459e-05 s, 0.0 kB/s 2023-10-10T20:36:56.740 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:56.740 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.447 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9756e-05 s, 0.0 kB/s 2023-10-10T20:36:56.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:56.795 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.448 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:56.850 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:56.850 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:56.851 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0998e-05 s, 0.0 kB/s 2023-10-10T20:36:56.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:56.851 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.449 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:56.906 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:56.906 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:56.906 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3225e-05 s, 0.0 kB/s 2023-10-10T20:36:56.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:56.907 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.450 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:56.961 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:56.962 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:56.962 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8691e-05 s, 0.0 kB/s 2023-10-10T20:36:56.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:56.963 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.451 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:57.017 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:57.017 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:57.018 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8997e-05 s, 0.0 kB/s 2023-10-10T20:36:57.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:57.018 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.452 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:57.073 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:57.073 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:57.073 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0449e-05 s, 0.0 kB/s 2023-10-10T20:36:57.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:57.074 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.453 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:57.128 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:57.128 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6162e-05 s, 0.0 kB/s 2023-10-10T20:36:57.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:57.129 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.454 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:57.184 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:57.184 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:57.184 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8099e-05 s, 0.0 kB/s 2023-10-10T20:36:57.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:57.185 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.455 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:57.239 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:57.240 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:57.240 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7891e-05 s, 0.0 kB/s 2023-10-10T20:36:57.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:57.241 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.456 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:57.295 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:57.295 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:57.295 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7945e-05 s, 0.0 kB/s 2023-10-10T20:36:57.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:57.296 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.457 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:57.350 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:57.350 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:57.350 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2002e-05 s, 0.0 kB/s 2023-10-10T20:36:57.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:57.351 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.458 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:57.405 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:57.405 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:57.405 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9958e-05 s, 0.0 kB/s 2023-10-10T20:36:57.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:57.406 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.459 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:57.462 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:57.462 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:57.462 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8222e-05 s, 0.0 kB/s 2023-10-10T20:36:57.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:57.463 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.460 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:57.517 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:57.518 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:57.518 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7186e-05 s, 0.0 kB/s 2023-10-10T20:36:57.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:57.519 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.461 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:57.573 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:57.574 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:57.574 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0855e-05 s, 0.0 kB/s 2023-10-10T20:36:57.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:57.575 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.462 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:57.629 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:57.629 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:57.629 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.113e-05 s, 0.0 kB/s 2023-10-10T20:36:57.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:57.630 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.463 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:57.684 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:57.684 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:57.684 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9909e-05 s, 0.0 kB/s 2023-10-10T20:36:57.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:57.685 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.464 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:57.738 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:57.739 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:57.739 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0183e-05 s, 0.0 kB/s 2023-10-10T20:36:57.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:57.740 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.465 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:57.794 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:57.794 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:57.795 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8912e-05 s, 0.0 kB/s 2023-10-10T20:36:57.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:57.796 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.466 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:57.850 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:57.850 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:57.850 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0012e-05 s, 0.0 kB/s 2023-10-10T20:36:57.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:57.851 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.467 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:57.904 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:57.904 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:57.904 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.846e-05 s, 0.0 kB/s 2023-10-10T20:36:57.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:57.906 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.468 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:57.961 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:57.961 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:57.961 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9863e-05 s, 0.0 kB/s 2023-10-10T20:36:57.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:57.962 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.469 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:58.015 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:58.016 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:58.016 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1098e-05 s, 0.0 kB/s 2023-10-10T20:36:58.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:58.016 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.470 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:58.071 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:58.071 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:58.071 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8278e-05 s, 0.0 kB/s 2023-10-10T20:36:58.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:58.072 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.471 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:58.126 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:58.126 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:58.126 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0225e-05 s, 0.0 kB/s 2023-10-10T20:36:58.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:58.127 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.472 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:58.180 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:58.181 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:58.181 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9212e-05 s, 0.0 kB/s 2023-10-10T20:36:58.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:58.182 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.473 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:58.236 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:58.236 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:58.236 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0166e-05 s, 0.0 kB/s 2023-10-10T20:36:58.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:58.237 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.474 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:58.291 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:58.292 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:58.292 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8893e-05 s, 0.0 kB/s 2023-10-10T20:36:58.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:58.293 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.475 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:58.347 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:58.347 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:58.347 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7617e-05 s, 0.0 kB/s 2023-10-10T20:36:58.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:58.348 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.476 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:58.401 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:58.402 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:58.402 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6909e-05 s, 0.0 kB/s 2023-10-10T20:36:58.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:58.403 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.477 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:58.456 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:58.456 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:58.457 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3445e-05 s, 0.0 kB/s 2023-10-10T20:36:58.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:58.458 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.478 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:58.513 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:58.513 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:58.513 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.006e-05 s, 0.0 kB/s 2023-10-10T20:36:58.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:58.514 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.479 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:58.568 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:58.568 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:58.568 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9733e-05 s, 0.0 kB/s 2023-10-10T20:36:58.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:58.569 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.480 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:58.623 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:58.623 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:58.623 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7801e-05 s, 0.0 kB/s 2023-10-10T20:36:58.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:58.624 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.481 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:58.679 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:58.679 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:58.679 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7651e-05 s, 0.0 kB/s 2023-10-10T20:36:58.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:58.680 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.482 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1523e-05 s, 0.0 kB/s 2023-10-10T20:36:58.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:58.735 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.483 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:58.789 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:58.789 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:58.789 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8802e-05 s, 0.0 kB/s 2023-10-10T20:36:58.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:58.792 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.484 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:58.844 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:58.845 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:58.845 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0364e-05 s, 0.0 kB/s 2023-10-10T20:36:58.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:58.846 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.485 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:58.900 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:58.900 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:58.900 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.402e-05 s, 0.0 kB/s 2023-10-10T20:36:58.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:58.901 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.486 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:58.955 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:58.955 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:58.956 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0319e-05 s, 0.0 kB/s 2023-10-10T20:36:58.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:58.956 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.487 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6809e-05 s, 0.0 kB/s 2023-10-10T20:36:59.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:59.012 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.488 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:59.066 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:59.066 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:59.066 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9766e-05 s, 0.0 kB/s 2023-10-10T20:36:59.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:59.067 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.489 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:59.121 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:59.122 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:59.122 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2695e-05 s, 0.0 kB/s 2023-10-10T20:36:59.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:59.123 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.490 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:59.177 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:59.177 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:59.177 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0654e-05 s, 0.0 kB/s 2023-10-10T20:36:59.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:59.178 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.491 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:59.232 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:59.232 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:59.232 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8376e-05 s, 0.0 kB/s 2023-10-10T20:36:59.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:59.233 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.492 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:59.287 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:59.287 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:59.287 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9549e-05 s, 0.0 kB/s 2023-10-10T20:36:59.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:59.288 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.493 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:59.342 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:59.342 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:59.342 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4319e-05 s, 0.0 kB/s 2023-10-10T20:36:59.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:59.343 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.494 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:59.397 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:59.397 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:59.397 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.7069e-05 s, 0.0 kB/s 2023-10-10T20:36:59.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:59.398 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.495 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:59.452 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:59.452 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:59.452 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8695e-05 s, 0.0 kB/s 2023-10-10T20:36:59.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:59.453 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.496 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:59.507 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:59.507 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:59.507 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6546e-05 s, 0.0 kB/s 2023-10-10T20:36:59.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:59.508 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.497 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:59.563 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:59.563 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:59.563 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1069e-05 s, 0.0 kB/s 2023-10-10T20:36:59.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:59.564 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.498 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:59.618 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:59.618 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:59.618 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8946e-05 s, 0.0 kB/s 2023-10-10T20:36:59.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:59.619 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.499 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:59.673 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:59.673 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:59.673 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.702e-05 s, 0.0 kB/s 2023-10-10T20:36:59.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:59.674 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.500 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:59.728 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:59.728 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:59.729 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8445e-05 s, 0.0 kB/s 2023-10-10T20:36:59.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:59.730 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.501 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8527e-05 s, 0.0 kB/s 2023-10-10T20:36:59.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:59.785 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.502 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:59.839 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:59.840 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:59.840 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9884e-05 s, 0.0 kB/s 2023-10-10T20:36:59.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:59.841 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.503 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:59.895 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:59.895 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:59.895 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7365e-05 s, 0.0 kB/s 2023-10-10T20:36:59.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:59.896 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.504 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:36:59.950 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:36:59.950 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:36:59.950 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.725e-05 s, 0.0 kB/s 2023-10-10T20:36:59.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:36:59.951 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.505 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:00.005 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:00.005 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:00.006 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9461e-05 s, 0.0 kB/s 2023-10-10T20:37:00.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:00.006 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.506 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:00.061 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:00.061 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:00.061 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.036e-05 s, 0.0 kB/s 2023-10-10T20:37:00.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:00.062 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.507 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:00.116 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:00.117 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:00.117 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4367e-05 s, 0.0 kB/s 2023-10-10T20:37:00.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:00.118 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.508 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:00.171 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:00.171 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:00.171 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7935e-05 s, 0.0 kB/s 2023-10-10T20:37:00.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:00.172 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.509 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:00.225 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:00.226 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:00.226 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9738e-05 s, 0.0 kB/s 2023-10-10T20:37:00.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:00.226 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.510 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:00.281 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:00.281 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:00.282 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9011e-05 s, 0.0 kB/s 2023-10-10T20:37:00.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:00.282 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.511 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8247e-05 s, 0.0 kB/s 2023-10-10T20:37:00.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:00.338 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.512 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:00.392 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:00.392 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:00.392 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0827e-05 s, 0.0 kB/s 2023-10-10T20:37:00.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:00.393 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.513 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:00.447 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:00.447 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:00.448 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9589e-05 s, 0.0 kB/s 2023-10-10T20:37:00.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:00.448 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.514 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:00.503 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:00.503 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:00.504 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1034e-05 s, 0.0 kB/s 2023-10-10T20:37:00.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:00.504 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.515 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:00.559 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:00.560 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:00.560 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0346e-05 s, 0.0 kB/s 2023-10-10T20:37:00.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:00.561 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.516 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:00.615 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:00.615 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:00.616 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7203e-05 s, 0.0 kB/s 2023-10-10T20:37:00.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:00.616 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.517 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:00.671 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:00.671 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:00.671 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.184e-05 s, 0.0 kB/s 2023-10-10T20:37:00.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:00.672 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.518 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:00.726 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:00.726 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:00.726 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7755e-05 s, 0.0 kB/s 2023-10-10T20:37:00.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:00.727 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.519 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:00.781 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:00.781 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:00.781 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0471e-05 s, 0.0 kB/s 2023-10-10T20:37:00.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:00.782 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.520 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7895e-05 s, 0.0 kB/s 2023-10-10T20:37:00.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:00.837 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.521 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:00.891 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:00.891 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:00.891 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7581e-05 s, 0.0 kB/s 2023-10-10T20:37:00.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:00.892 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.522 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:00.946 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:00.946 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:00.946 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2483e-05 s, 0.0 kB/s 2023-10-10T20:37:00.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:00.947 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.523 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:01.001 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:01.002 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:01.002 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3606e-05 s, 0.0 kB/s 2023-10-10T20:37:01.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:01.003 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.524 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:01.056 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:01.057 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:01.057 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8806e-05 s, 0.0 kB/s 2023-10-10T20:37:01.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:01.058 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.525 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:01.111 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:01.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:01.112 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9479e-05 s, 0.0 kB/s 2023-10-10T20:37:01.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:01.113 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.526 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:01.167 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:01.167 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:01.167 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8879e-05 s, 0.0 kB/s 2023-10-10T20:37:01.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:01.168 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.527 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:01.221 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:01.222 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:01.222 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9633e-05 s, 0.0 kB/s 2023-10-10T20:37:01.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:01.223 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.528 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:01.276 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:01.277 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:01.277 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8606e-05 s, 0.0 kB/s 2023-10-10T20:37:01.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:01.278 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.529 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:01.332 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:01.332 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:01.332 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3271e-05 s, 0.0 kB/s 2023-10-10T20:37:01.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:01.333 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.530 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:01.387 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:01.387 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:01.387 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0707e-05 s, 0.0 kB/s 2023-10-10T20:37:01.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:01.388 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.531 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:01.442 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:01.443 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:01.443 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.904e-05 s, 0.0 kB/s 2023-10-10T20:37:01.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:01.444 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.532 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:01.498 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:01.498 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:01.498 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.684e-05 s, 0.0 kB/s 2023-10-10T20:37:01.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:01.499 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.533 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:01.553 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:01.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:01.554 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6841e-05 s, 0.0 kB/s 2023-10-10T20:37:01.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:01.555 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.534 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:01.609 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:01.609 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:01.610 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9695e-05 s, 0.0 kB/s 2023-10-10T20:37:01.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:01.610 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.535 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:01.664 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:01.664 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:01.665 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8405e-05 s, 0.0 kB/s 2023-10-10T20:37:01.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:01.665 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.536 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:01.720 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:01.720 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:01.721 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8841e-05 s, 0.0 kB/s 2023-10-10T20:37:01.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:01.721 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.537 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:01.776 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:01.776 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:01.776 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9496e-05 s, 0.0 kB/s 2023-10-10T20:37:01.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:01.777 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.538 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:01.831 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:01.832 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:01.832 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8697e-05 s, 0.0 kB/s 2023-10-10T20:37:01.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:01.833 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.539 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:01.887 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:01.887 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:01.887 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0489e-05 s, 0.0 kB/s 2023-10-10T20:37:01.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:01.888 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.540 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:01.942 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:01.942 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:01.942 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8691e-05 s, 0.0 kB/s 2023-10-10T20:37:01.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:01.944 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.541 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:01.999 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:01.999 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:02.000 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4672e-05 s, 0.0 kB/s 2023-10-10T20:37:02.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:02.000 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.542 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:02.055 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:02.056 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:02.056 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8218e-05 s, 0.0 kB/s 2023-10-10T20:37:02.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:02.057 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.543 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:02.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:02.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:02.112 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2661e-05 s, 0.0 kB/s 2023-10-10T20:37:02.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:02.113 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.544 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:02.168 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:02.168 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:02.168 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7007e-05 s, 0.0 kB/s 2023-10-10T20:37:02.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:02.169 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.545 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:02.223 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:02.223 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:02.224 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3182e-05 s, 0.0 kB/s 2023-10-10T20:37:02.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:02.224 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.546 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:02.279 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:02.279 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:02.279 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7857e-05 s, 0.0 kB/s 2023-10-10T20:37:02.280 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:02.280 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.547 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:02.334 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:02.334 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:02.334 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.8299e-05 s, 0.0 kB/s 2023-10-10T20:37:02.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:02.335 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.548 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:02.389 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:02.389 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:02.389 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9477e-05 s, 0.0 kB/s 2023-10-10T20:37:02.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:02.390 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.549 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:02.444 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:02.444 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:02.444 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1005e-05 s, 0.0 kB/s 2023-10-10T20:37:02.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:02.445 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.550 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:02.499 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:02.499 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:02.499 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0886e-05 s, 0.0 kB/s 2023-10-10T20:37:02.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:02.500 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.551 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:02.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:02.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:02.554 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.767e-05 s, 0.0 kB/s 2023-10-10T20:37:02.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:02.555 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.552 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:02.609 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:02.609 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:02.609 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8401e-05 s, 0.0 kB/s 2023-10-10T20:37:02.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:02.610 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.553 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:02.664 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:02.665 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:02.665 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0795e-05 s, 0.0 kB/s 2023-10-10T20:37:02.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:02.666 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.554 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:02.720 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:02.720 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:02.720 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1688e-05 s, 0.0 kB/s 2023-10-10T20:37:02.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:02.721 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.555 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:02.775 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:02.775 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:02.776 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1181e-05 s, 0.0 kB/s 2023-10-10T20:37:02.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:02.777 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.556 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:02.831 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:02.831 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:02.832 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.024e-05 s, 0.0 kB/s 2023-10-10T20:37:02.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:02.832 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.557 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:02.886 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:02.886 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:02.887 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1032e-05 s, 0.0 kB/s 2023-10-10T20:37:02.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:02.887 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.558 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:02.942 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:02.943 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:02.943 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9479e-05 s, 0.0 kB/s 2023-10-10T20:37:02.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:02.943 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.559 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:02.998 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:02.999 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:02.999 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8047e-05 s, 0.0 kB/s 2023-10-10T20:37:03.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:03.001 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.560 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:03.054 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:03.055 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:03.055 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7397e-05 s, 0.0 kB/s 2023-10-10T20:37:03.055 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:03.056 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.561 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:03.110 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:03.110 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:03.110 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0241e-05 s, 0.0 kB/s 2023-10-10T20:37:03.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:03.111 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.562 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:03.165 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:03.165 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:03.166 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9038e-05 s, 0.0 kB/s 2023-10-10T20:37:03.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:03.166 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.563 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2532e-05 s, 0.0 kB/s 2023-10-10T20:37:03.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:03.222 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.564 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:03.276 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:03.277 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:03.277 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7359e-05 s, 0.0 kB/s 2023-10-10T20:37:03.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:03.278 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.565 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:03.332 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:03.332 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:03.332 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7604e-05 s, 0.0 kB/s 2023-10-10T20:37:03.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:03.333 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.566 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:03.387 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:03.387 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:03.387 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7439e-05 s, 0.0 kB/s 2023-10-10T20:37:03.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:03.388 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.567 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:03.442 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:03.443 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:03.443 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8287e-05 s, 0.0 kB/s 2023-10-10T20:37:03.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:03.444 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.568 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:03.498 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:03.499 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:03.499 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1013e-05 s, 0.0 kB/s 2023-10-10T20:37:03.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:03.500 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.569 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7559e-05 s, 0.0 kB/s 2023-10-10T20:37:03.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:03.555 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.570 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:03.611 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:03.611 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:03.611 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7729e-05 s, 0.0 kB/s 2023-10-10T20:37:03.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:03.612 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.571 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:03.667 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:03.667 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:03.667 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3062e-05 s, 0.0 kB/s 2023-10-10T20:37:03.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:03.668 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.572 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:03.722 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:03.722 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5306e-05 s, 0.0 kB/s 2023-10-10T20:37:03.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:03.723 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.573 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:03.778 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:03.778 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:03.778 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1204e-05 s, 0.0 kB/s 2023-10-10T20:37:03.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:03.779 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.574 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:03.834 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:03.834 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:03.834 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7035e-05 s, 0.0 kB/s 2023-10-10T20:37:03.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:03.835 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.575 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:03.889 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:03.890 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:03.890 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.899e-05 s, 0.0 kB/s 2023-10-10T20:37:03.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:03.891 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.576 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:03.946 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:03.946 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:03.947 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3472e-05 s, 0.0 kB/s 2023-10-10T20:37:03.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:03.947 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.577 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:04.001 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:04.001 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:04.001 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0445e-05 s, 0.0 kB/s 2023-10-10T20:37:04.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:04.002 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.578 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:04.057 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:04.057 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:04.057 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.7937e-05 s, 0.0 kB/s 2023-10-10T20:37:04.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:04.058 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.579 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:04.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:04.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:04.112 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9773e-05 s, 0.0 kB/s 2023-10-10T20:37:04.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:04.113 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.580 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:04.167 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:04.167 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:04.167 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8639e-05 s, 0.0 kB/s 2023-10-10T20:37:04.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:04.168 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.581 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:04.222 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:04.222 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:04.222 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0182e-05 s, 0.0 kB/s 2023-10-10T20:37:04.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:04.223 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.582 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2306e-05 s, 0.0 kB/s 2023-10-10T20:37:04.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:04.278 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.583 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:04.332 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:04.332 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:04.332 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.919e-05 s, 0.0 kB/s 2023-10-10T20:37:04.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:04.333 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.584 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:04.387 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:04.387 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:04.387 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1672e-05 s, 0.0 kB/s 2023-10-10T20:37:04.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:04.388 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.585 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8556e-05 s, 0.0 kB/s 2023-10-10T20:37:04.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:04.443 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.586 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:04.498 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:04.498 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.045e-05 s, 0.0 kB/s 2023-10-10T20:37:04.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:04.499 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.587 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:04.553 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:04.553 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:04.553 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.857e-05 s, 0.0 kB/s 2023-10-10T20:37:04.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:04.554 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.588 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:04.607 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:04.608 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:04.608 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8198e-05 s, 0.0 kB/s 2023-10-10T20:37:04.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:04.609 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.589 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:04.663 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:04.663 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:04.663 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6628e-05 s, 0.0 kB/s 2023-10-10T20:37:04.664 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:04.664 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.590 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:04.718 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:04.719 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:04.719 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.943e-05 s, 0.0 kB/s 2023-10-10T20:37:04.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:04.720 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.591 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:04.774 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:04.775 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:04.775 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8839e-05 s, 0.0 kB/s 2023-10-10T20:37:04.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:04.776 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.592 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7009e-05 s, 0.0 kB/s 2023-10-10T20:37:04.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:04.831 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.593 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:04.885 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:04.885 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:04.885 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9739e-05 s, 0.0 kB/s 2023-10-10T20:37:04.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:04.886 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.594 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:04.940 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:04.940 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:04.940 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8997e-05 s, 0.0 kB/s 2023-10-10T20:37:04.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:04.941 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.595 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:04.995 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:04.995 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:04.995 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8595e-05 s, 0.0 kB/s 2023-10-10T20:37:04.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:04.996 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.596 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:05.050 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:05.050 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:05.050 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1585e-05 s, 0.0 kB/s 2023-10-10T20:37:05.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:05.051 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.597 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:05.105 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:05.105 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:05.105 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9729e-05 s, 0.0 kB/s 2023-10-10T20:37:05.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:05.106 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.598 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:05.160 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:05.160 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:05.161 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0624e-05 s, 0.0 kB/s 2023-10-10T20:37:05.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:05.161 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.599 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:05.216 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:05.216 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:05.216 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4268e-05 s, 0.0 kB/s 2023-10-10T20:37:05.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:05.217 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.600 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:05.271 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:05.271 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:05.271 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7767e-05 s, 0.0 kB/s 2023-10-10T20:37:05.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:05.272 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.601 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:05.326 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:05.326 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:05.326 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.5828e-05 s, 0.0 kB/s 2023-10-10T20:37:05.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:05.327 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.602 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:05.381 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:05.381 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:05.381 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7876e-05 s, 0.0 kB/s 2023-10-10T20:37:05.382 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:05.382 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.603 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:05.436 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:05.436 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:05.436 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8159e-05 s, 0.0 kB/s 2023-10-10T20:37:05.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:05.437 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.604 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:05.491 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:05.491 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:05.491 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8829e-05 s, 0.0 kB/s 2023-10-10T20:37:05.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:05.492 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.605 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:05.546 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:05.546 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:05.546 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6781e-05 s, 0.0 kB/s 2023-10-10T20:37:05.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:05.547 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.606 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:05.601 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:05.601 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:05.601 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1497e-05 s, 0.0 kB/s 2023-10-10T20:37:05.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:05.602 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.607 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:05.656 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:05.656 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6862e-05 s, 0.0 kB/s 2023-10-10T20:37:05.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:05.657 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.608 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:05.711 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:05.711 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:05.712 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8878e-05 s, 0.0 kB/s 2023-10-10T20:37:05.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:05.712 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.609 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:05.767 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:05.768 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:05.768 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0814e-05 s, 0.0 kB/s 2023-10-10T20:37:05.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:05.769 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.610 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:05.823 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:05.823 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:05.823 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6963e-05 s, 0.0 kB/s 2023-10-10T20:37:05.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:05.824 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.611 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:05.878 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:05.878 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:05.878 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0143e-05 s, 0.0 kB/s 2023-10-10T20:37:05.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:05.879 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.612 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:05.933 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:05.933 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:05.933 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8859e-05 s, 0.0 kB/s 2023-10-10T20:37:05.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:05.935 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.613 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:05.988 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:05.989 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:05.989 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.307e-05 s, 0.0 kB/s 2023-10-10T20:37:05.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:05.990 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.614 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:06.044 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:06.044 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:06.044 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6261e-05 s, 0.0 kB/s 2023-10-10T20:37:06.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:06.045 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.615 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:06.099 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:06.099 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:06.099 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2656e-05 s, 0.0 kB/s 2023-10-10T20:37:06.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:06.100 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.616 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:06.155 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:06.156 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:06.156 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0191e-05 s, 0.0 kB/s 2023-10-10T20:37:06.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:06.157 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.617 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:06.211 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:06.211 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:06.212 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9841e-05 s, 0.0 kB/s 2023-10-10T20:37:06.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:06.212 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.618 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:06.267 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:06.267 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:06.268 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2059e-05 s, 0.0 kB/s 2023-10-10T20:37:06.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:06.268 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.619 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:06.323 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:06.323 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:06.323 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.855e-05 s, 0.0 kB/s 2023-10-10T20:37:06.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:06.324 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.620 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:06.378 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:06.378 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:06.378 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3908e-05 s, 0.0 kB/s 2023-10-10T20:37:06.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:06.379 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.621 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:06.433 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:06.434 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:06.434 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2129e-05 s, 0.0 kB/s 2023-10-10T20:37:06.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:06.435 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.622 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:06.489 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:06.489 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:06.490 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9539e-05 s, 0.0 kB/s 2023-10-10T20:37:06.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:06.490 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.623 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7852e-05 s, 0.0 kB/s 2023-10-10T20:37:06.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:06.547 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.624 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:06.601 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:06.602 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:06.602 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0041e-05 s, 0.0 kB/s 2023-10-10T20:37:06.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:06.603 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.625 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:06.657 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:06.657 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:06.657 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6985e-05 s, 0.0 kB/s 2023-10-10T20:37:06.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:06.658 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.626 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:06.712 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:06.712 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:06.712 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.111e-05 s, 0.0 kB/s 2023-10-10T20:37:06.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:06.713 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.627 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:06.767 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:06.767 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:06.767 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.5328e-05 s, 0.0 kB/s 2023-10-10T20:37:06.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:06.768 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.628 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:06.822 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:06.823 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:06.823 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8737e-05 s, 0.0 kB/s 2023-10-10T20:37:06.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:06.824 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.629 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:06.878 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:06.878 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:06.879 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7195e-05 s, 0.0 kB/s 2023-10-10T20:37:06.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:06.879 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.630 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:06.933 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:06.934 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:06.934 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8438e-05 s, 0.0 kB/s 2023-10-10T20:37:06.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:06.935 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.631 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:06.989 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:06.989 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:06.989 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2083e-05 s, 0.0 kB/s 2023-10-10T20:37:06.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:06.990 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.632 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:07.044 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:07.044 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:07.044 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7756e-05 s, 0.0 kB/s 2023-10-10T20:37:07.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:07.045 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.633 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:07.099 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:07.099 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:07.099 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3562e-05 s, 0.0 kB/s 2023-10-10T20:37:07.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:07.100 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.634 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:07.154 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:07.154 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:07.154 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8447e-05 s, 0.0 kB/s 2023-10-10T20:37:07.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:07.155 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.635 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:07.209 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:07.209 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:07.210 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0643e-05 s, 0.0 kB/s 2023-10-10T20:37:07.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:07.211 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.636 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:07.264 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:07.265 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:07.265 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8933e-05 s, 0.0 kB/s 2023-10-10T20:37:07.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:07.266 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.637 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:07.322 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:07.323 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:07.323 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0011e-05 s, 0.0 kB/s 2023-10-10T20:37:07.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:07.324 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.638 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:07.378 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:07.379 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:07.379 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8533e-05 s, 0.0 kB/s 2023-10-10T20:37:07.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:07.381 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.639 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:07.434 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:07.435 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:07.435 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9152e-05 s, 0.0 kB/s 2023-10-10T20:37:07.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:07.436 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.640 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:07.490 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:07.491 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:07.491 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0571e-05 s, 0.0 kB/s 2023-10-10T20:37:07.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:07.492 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.641 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:07.546 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:07.546 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:07.546 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.5769e-05 s, 0.0 kB/s 2023-10-10T20:37:07.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:07.548 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.642 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:07.601 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:07.601 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:07.601 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0679e-05 s, 0.0 kB/s 2023-10-10T20:37:07.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:07.602 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.643 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:07.656 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:07.657 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:07.657 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0742e-05 s, 0.0 kB/s 2023-10-10T20:37:07.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:07.658 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.644 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:07.712 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:07.712 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:07.712 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.708e-05 s, 0.0 kB/s 2023-10-10T20:37:07.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:07.713 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.645 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:07.767 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:07.767 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:07.768 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7958e-05 s, 0.0 kB/s 2023-10-10T20:37:07.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:07.768 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.646 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:07.823 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:07.823 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:07.823 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9422e-05 s, 0.0 kB/s 2023-10-10T20:37:07.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:07.824 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.647 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:07.878 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:07.878 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:07.878 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9247e-05 s, 0.0 kB/s 2023-10-10T20:37:07.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:07.879 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.648 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:07.933 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:07.933 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:07.933 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5285e-05 s, 0.0 kB/s 2023-10-10T20:37:07.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:07.934 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.649 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:07.988 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:07.988 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:07.989 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4579e-05 s, 0.0 kB/s 2023-10-10T20:37:07.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:07.989 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.650 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:08.043 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:08.044 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:08.044 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0647e-05 s, 0.0 kB/s 2023-10-10T20:37:08.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:08.044 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.651 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:08.099 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:08.099 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:08.100 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1005e-05 s, 0.0 kB/s 2023-10-10T20:37:08.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:08.100 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.652 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:08.155 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:08.155 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:08.156 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1504e-05 s, 0.0 kB/s 2023-10-10T20:37:08.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:08.156 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.653 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:08.211 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:08.212 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:08.212 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2204e-05 s, 0.0 kB/s 2023-10-10T20:37:08.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:08.213 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.654 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:08.267 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:08.267 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:08.268 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0451e-05 s, 0.0 kB/s 2023-10-10T20:37:08.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:08.268 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.655 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:08.323 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:08.323 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:08.323 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9192e-05 s, 0.0 kB/s 2023-10-10T20:37:08.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:08.324 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.656 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:08.378 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:08.378 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:08.379 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7831e-05 s, 0.0 kB/s 2023-10-10T20:37:08.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:08.379 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.657 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:08.434 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:08.434 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:08.434 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0491e-05 s, 0.0 kB/s 2023-10-10T20:37:08.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:08.435 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.658 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:08.488 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:08.489 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:08.489 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2165e-05 s, 0.0 kB/s 2023-10-10T20:37:08.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:08.490 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.659 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:08.544 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:08.545 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:08.545 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7684e-05 s, 0.0 kB/s 2023-10-10T20:37:08.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:08.545 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.660 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:08.600 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:08.600 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:08.600 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6211e-05 s, 0.0 kB/s 2023-10-10T20:37:08.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:08.601 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.661 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:08.658 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:08.658 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:08.659 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8821e-05 s, 0.0 kB/s 2023-10-10T20:37:08.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:08.660 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.662 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:08.716 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:08.716 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:08.716 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0491e-05 s, 0.0 kB/s 2023-10-10T20:37:08.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:08.718 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.663 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:08.774 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:08.774 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:08.774 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8738e-05 s, 0.0 kB/s 2023-10-10T20:37:08.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:08.776 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.664 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:08.832 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:08.833 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:08.833 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9995e-05 s, 0.0 kB/s 2023-10-10T20:37:08.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:08.834 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.665 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:08.889 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:08.889 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:08.889 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7864e-05 s, 0.0 kB/s 2023-10-10T20:37:08.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:08.890 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.666 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:08.944 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:08.945 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:08.945 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.5807e-05 s, 0.0 kB/s 2023-10-10T20:37:08.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:08.946 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.667 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:09.000 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:09.001 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:09.001 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7733e-05 s, 0.0 kB/s 2023-10-10T20:37:09.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:09.002 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.668 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:09.056 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:09.056 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:09.056 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2484e-05 s, 0.0 kB/s 2023-10-10T20:37:09.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:09.057 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.669 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:09.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:09.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:09.112 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7228e-05 s, 0.0 kB/s 2023-10-10T20:37:09.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:09.113 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.670 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:09.167 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:09.167 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:09.167 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9353e-05 s, 0.0 kB/s 2023-10-10T20:37:09.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:09.168 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.671 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:09.222 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:09.223 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:09.223 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.7105e-05 s, 0.0 kB/s 2023-10-10T20:37:09.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:09.224 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.672 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6949e-05 s, 0.0 kB/s 2023-10-10T20:37:09.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:09.279 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.673 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:09.333 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:09.333 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:09.334 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6821e-05 s, 0.0 kB/s 2023-10-10T20:37:09.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:09.334 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.674 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:09.389 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:09.389 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:09.389 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7941e-05 s, 0.0 kB/s 2023-10-10T20:37:09.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:09.390 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.675 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:09.444 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:09.445 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:09.445 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8933e-05 s, 0.0 kB/s 2023-10-10T20:37:09.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:09.447 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.676 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:09.500 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:09.500 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:09.501 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6293e-05 s, 0.0 kB/s 2023-10-10T20:37:09.501 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:09.501 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.677 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:09.556 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:09.556 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:09.556 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9697e-05 s, 0.0 kB/s 2023-10-10T20:37:09.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:09.557 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.678 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:09.611 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:09.611 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:09.611 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7008e-05 s, 0.0 kB/s 2023-10-10T20:37:09.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:09.612 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.679 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:09.667 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:09.667 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:09.668 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9305e-05 s, 0.0 kB/s 2023-10-10T20:37:09.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:09.668 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.680 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:09.724 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:09.724 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:09.724 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0319e-05 s, 0.0 kB/s 2023-10-10T20:37:09.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:09.725 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.681 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:09.779 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:09.779 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:09.779 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6945e-05 s, 0.0 kB/s 2023-10-10T20:37:09.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:09.780 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.682 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:09.834 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:09.834 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:09.835 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8252e-05 s, 0.0 kB/s 2023-10-10T20:37:09.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:09.835 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.683 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:09.890 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:09.890 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:09.890 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1754e-05 s, 0.0 kB/s 2023-10-10T20:37:09.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:09.891 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.684 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:09.945 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:09.945 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:09.945 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3791e-05 s, 0.0 kB/s 2023-10-10T20:37:09.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:09.946 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.685 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:10.000 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:10.000 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:10.000 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0039e-05 s, 0.0 kB/s 2023-10-10T20:37:10.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:10.002 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.686 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:10.057 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:10.057 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:10.057 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1464e-05 s, 0.0 kB/s 2023-10-10T20:37:10.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:10.058 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.687 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:10.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:10.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:10.112 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.699e-05 s, 0.0 kB/s 2023-10-10T20:37:10.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:10.113 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.688 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:10.167 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:10.167 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:10.168 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9843e-05 s, 0.0 kB/s 2023-10-10T20:37:10.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:10.168 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.689 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:10.222 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:10.222 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:10.222 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9855e-05 s, 0.0 kB/s 2023-10-10T20:37:10.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:10.223 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.690 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:10.277 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:10.277 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:10.277 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.873e-05 s, 0.0 kB/s 2023-10-10T20:37:10.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:10.278 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.691 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:10.332 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:10.332 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:10.332 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7499e-05 s, 0.0 kB/s 2023-10-10T20:37:10.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:10.334 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.692 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:10.389 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:10.389 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:10.389 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9939e-05 s, 0.0 kB/s 2023-10-10T20:37:10.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:10.390 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.693 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:10.444 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:10.444 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:10.444 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7799e-05 s, 0.0 kB/s 2023-10-10T20:37:10.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:10.445 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.694 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:10.499 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:10.499 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:10.500 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3193e-05 s, 0.0 kB/s 2023-10-10T20:37:10.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:10.501 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.695 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:10.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:10.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:10.554 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9478e-05 s, 0.0 kB/s 2023-10-10T20:37:10.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:10.555 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.696 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:10.609 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:10.609 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:10.609 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8955e-05 s, 0.0 kB/s 2023-10-10T20:37:10.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:10.610 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.697 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:10.664 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:10.664 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:10.665 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9653e-05 s, 0.0 kB/s 2023-10-10T20:37:10.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:10.665 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.698 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:10.720 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:10.720 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:10.720 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7296e-05 s, 0.0 kB/s 2023-10-10T20:37:10.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:10.721 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.699 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:10.775 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:10.775 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:10.775 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0773e-05 s, 0.0 kB/s 2023-10-10T20:37:10.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:10.776 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.700 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:10.830 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:10.830 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:10.830 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.627e-05 s, 0.0 kB/s 2023-10-10T20:37:10.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:10.831 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.701 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:10.886 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:10.886 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:10.886 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9819e-05 s, 0.0 kB/s 2023-10-10T20:37:10.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:10.887 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.702 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:10.941 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:10.941 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:10.941 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2071e-05 s, 0.0 kB/s 2023-10-10T20:37:10.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:10.942 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.703 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:10.996 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:10.996 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:10.997 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2521e-05 s, 0.0 kB/s 2023-10-10T20:37:10.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:10.997 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.704 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:11.052 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:11.052 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:11.052 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0642e-05 s, 0.0 kB/s 2023-10-10T20:37:11.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:11.053 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.705 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:11.107 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:11.107 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:11.107 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7559e-05 s, 0.0 kB/s 2023-10-10T20:37:11.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:11.108 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.706 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:11.162 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:11.162 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:11.162 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9231e-05 s, 0.0 kB/s 2023-10-10T20:37:11.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:11.163 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.707 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:11.217 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:11.217 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:11.217 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0685e-05 s, 0.0 kB/s 2023-10-10T20:37:11.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:11.218 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.708 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:11.272 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:11.273 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:11.273 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0871e-05 s, 0.0 kB/s 2023-10-10T20:37:11.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:11.274 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.709 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:11.328 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:11.328 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:11.329 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8649e-05 s, 0.0 kB/s 2023-10-10T20:37:11.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:11.329 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.710 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:11.383 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:11.384 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:11.384 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8782e-05 s, 0.0 kB/s 2023-10-10T20:37:11.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:11.385 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.711 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:11.439 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:11.439 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:11.440 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0124e-05 s, 0.0 kB/s 2023-10-10T20:37:11.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:11.440 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.712 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:11.494 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:11.494 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:11.494 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.203e-05 s, 0.0 kB/s 2023-10-10T20:37:11.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:11.495 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.713 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:11.549 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:11.549 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:11.549 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1206e-05 s, 0.0 kB/s 2023-10-10T20:37:11.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:11.550 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.714 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:11.604 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:11.604 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:11.604 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1281e-05 s, 0.0 kB/s 2023-10-10T20:37:11.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:11.605 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.715 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:11.659 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:11.659 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:11.660 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8375e-05 s, 0.0 kB/s 2023-10-10T20:37:11.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:11.660 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.716 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:11.714 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:11.715 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:11.715 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7723e-05 s, 0.0 kB/s 2023-10-10T20:37:11.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:11.716 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.717 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:11.770 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:11.770 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:11.771 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.856e-05 s, 0.0 kB/s 2023-10-10T20:37:11.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:11.771 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.718 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:11.825 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:11.825 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:11.825 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9687e-05 s, 0.0 kB/s 2023-10-10T20:37:11.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:11.826 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.719 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8012e-05 s, 0.0 kB/s 2023-10-10T20:37:11.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:11.881 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.720 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:11.935 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:11.935 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:11.936 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2212e-05 s, 0.0 kB/s 2023-10-10T20:37:11.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:11.936 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.721 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:11.956 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:37:11.979 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:37:11.991 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:11.991 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:11.991 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9609e-05 s, 0.0 kB/s 2023-10-10T20:37:11.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:11.992 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.722 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:12.047 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:12.048 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:12.048 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0294e-05 s, 0.0 kB/s 2023-10-10T20:37:12.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:12.049 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.723 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:12.103 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:12.103 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:12.104 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8484e-05 s, 0.0 kB/s 2023-10-10T20:37:12.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:12.104 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.724 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:12.158 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:12.159 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:12.159 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9644e-05 s, 0.0 kB/s 2023-10-10T20:37:12.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:12.160 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.725 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:12.214 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:12.214 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:12.215 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9834e-05 s, 0.0 kB/s 2023-10-10T20:37:12.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:12.215 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.726 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:12.270 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:12.270 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:12.271 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6833e-05 s, 0.0 kB/s 2023-10-10T20:37:12.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:12.271 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.727 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:12.326 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:12.326 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:12.326 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9566e-05 s, 0.0 kB/s 2023-10-10T20:37:12.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:12.327 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.728 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:12.381 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:12.381 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:12.382 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9537e-05 s, 0.0 kB/s 2023-10-10T20:37:12.382 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:12.382 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.729 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:12.436 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:12.436 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:12.436 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2904e-05 s, 0.0 kB/s 2023-10-10T20:37:12.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:12.437 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.730 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:12.491 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:12.491 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8249e-05 s, 0.0 kB/s 2023-10-10T20:37:12.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:12.492 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.731 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:12.546 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:12.547 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:12.547 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2887e-05 s, 0.0 kB/s 2023-10-10T20:37:12.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:12.548 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.732 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:12.602 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:12.602 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:12.602 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8394e-05 s, 0.0 kB/s 2023-10-10T20:37:12.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:12.603 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.733 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:12.657 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:12.657 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:12.658 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6578e-05 s, 0.0 kB/s 2023-10-10T20:37:12.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:12.659 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.734 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:12.712 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:12.712 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:12.712 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0514e-05 s, 0.0 kB/s 2023-10-10T20:37:12.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:12.713 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.735 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:12.767 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:12.767 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:12.767 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0742e-05 s, 0.0 kB/s 2023-10-10T20:37:12.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:12.768 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.736 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.377e-05 s, 0.0 kB/s 2023-10-10T20:37:12.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:12.823 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.737 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:12.877 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:12.877 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:12.877 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1665e-05 s, 0.0 kB/s 2023-10-10T20:37:12.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:12.878 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.738 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:12.932 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:12.932 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:12.933 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.908e-05 s, 0.0 kB/s 2023-10-10T20:37:12.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:12.933 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.739 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:12.987 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:12.988 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:12.988 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9178e-05 s, 0.0 kB/s 2023-10-10T20:37:12.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:12.989 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.740 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:13.043 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:13.043 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:13.043 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0169e-05 s, 0.0 kB/s 2023-10-10T20:37:13.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:13.044 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.741 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:13.098 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:13.098 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:13.098 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8788e-05 s, 0.0 kB/s 2023-10-10T20:37:13.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:13.099 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.742 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:13.154 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:13.154 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:13.154 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0406e-05 s, 0.0 kB/s 2023-10-10T20:37:13.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:13.155 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.743 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:13.209 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:13.210 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:13.210 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.806e-05 s, 0.0 kB/s 2023-10-10T20:37:13.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:13.211 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.744 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:13.265 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:13.265 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:13.265 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8666e-05 s, 0.0 kB/s 2023-10-10T20:37:13.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:13.266 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.745 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:13.320 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:13.320 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:13.321 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8641e-05 s, 0.0 kB/s 2023-10-10T20:37:13.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:13.321 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.746 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:13.376 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:13.376 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:13.376 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1241e-05 s, 0.0 kB/s 2023-10-10T20:37:13.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:13.377 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.747 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:13.431 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:13.431 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:13.431 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7846e-05 s, 0.0 kB/s 2023-10-10T20:37:13.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:13.432 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.748 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:13.486 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:13.486 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:13.486 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1074e-05 s, 0.0 kB/s 2023-10-10T20:37:13.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:13.487 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.749 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:13.541 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:13.541 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:13.541 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7608e-05 s, 0.0 kB/s 2023-10-10T20:37:13.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:13.542 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.750 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9203e-05 s, 0.0 kB/s 2023-10-10T20:37:13.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:13.597 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.751 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:13.651 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:13.652 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:13.652 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9241e-05 s, 0.0 kB/s 2023-10-10T20:37:13.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:13.653 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.752 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:13.707 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:13.708 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:13.708 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.06e-05 s, 0.0 kB/s 2023-10-10T20:37:13.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:13.709 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.753 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:13.763 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:13.763 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:13.764 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0776e-05 s, 0.0 kB/s 2023-10-10T20:37:13.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:13.764 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.754 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:13.819 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:13.819 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:13.819 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9955e-05 s, 0.0 kB/s 2023-10-10T20:37:13.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:13.820 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.755 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:13.874 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:13.874 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:13.875 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.268e-05 s, 0.0 kB/s 2023-10-10T20:37:13.875 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:13.875 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.756 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:13.930 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:13.930 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:13.930 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0649e-05 s, 0.0 kB/s 2023-10-10T20:37:13.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:13.931 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.757 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:13.985 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:13.985 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:13.985 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.8013e-05 s, 0.0 kB/s 2023-10-10T20:37:13.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:13.986 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.758 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:14.042 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:14.043 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:14.043 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0899e-05 s, 0.0 kB/s 2023-10-10T20:37:14.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:14.044 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.759 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:14.102 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:14.103 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:14.103 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.763e-05 s, 0.0 kB/s 2023-10-10T20:37:14.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:14.104 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.760 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:14.161 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:14.162 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:14.162 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0163e-05 s, 0.0 kB/s 2023-10-10T20:37:14.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:14.163 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.761 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:14.220 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6446e-05 s, 0.0 kB/s 2023-10-10T20:37:14.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:14.222 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.762 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:14.276 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:14.277 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:14.277 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1946e-05 s, 0.0 kB/s 2023-10-10T20:37:14.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:14.278 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.763 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:14.333 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:14.334 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:14.334 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1703e-05 s, 0.0 kB/s 2023-10-10T20:37:14.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:14.335 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.764 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:14.389 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:14.389 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:14.390 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0823e-05 s, 0.0 kB/s 2023-10-10T20:37:14.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:14.391 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.765 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1761e-05 s, 0.0 kB/s 2023-10-10T20:37:14.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:14.447 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.766 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9101e-05 s, 0.0 kB/s 2023-10-10T20:37:14.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:14.503 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.767 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8988e-05 s, 0.0 kB/s 2023-10-10T20:37:14.558 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:14.558 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.768 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.644e-05 s, 0.0 kB/s 2023-10-10T20:37:14.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:14.612 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.769 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:14.666 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7209e-05 s, 0.0 kB/s 2023-10-10T20:37:14.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:14.667 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.770 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8834e-05 s, 0.0 kB/s 2023-10-10T20:37:14.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:14.724 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.771 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:14.778 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:14.778 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:14.778 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9384e-05 s, 0.0 kB/s 2023-10-10T20:37:14.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:14.779 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.772 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:14.833 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:14.833 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:14.833 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.93e-05 s, 0.0 kB/s 2023-10-10T20:37:14.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:14.834 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.773 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:14.888 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:14.889 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:14.889 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.5566e-05 s, 0.0 kB/s 2023-10-10T20:37:14.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:14.890 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.774 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:14.945 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:14.945 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:14.945 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7756e-05 s, 0.0 kB/s 2023-10-10T20:37:14.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:14.947 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.775 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:15.000 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0488e-05 s, 0.0 kB/s 2023-10-10T20:37:15.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:15.002 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.776 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:15.056 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:15.057 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:15.057 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9035e-05 s, 0.0 kB/s 2023-10-10T20:37:15.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:15.058 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.777 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:15.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:15.113 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:15.113 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6226e-05 s, 0.0 kB/s 2023-10-10T20:37:15.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:15.114 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.778 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:15.168 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:15.168 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:15.168 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1275e-05 s, 0.0 kB/s 2023-10-10T20:37:15.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:15.169 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.779 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:15.223 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:15.223 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:15.223 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7295e-05 s, 0.0 kB/s 2023-10-10T20:37:15.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:15.224 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.780 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:15.278 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:15.278 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:15.278 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4281e-05 s, 0.0 kB/s 2023-10-10T20:37:15.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:15.279 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.781 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:15.333 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:15.333 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:15.334 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2054e-05 s, 0.0 kB/s 2023-10-10T20:37:15.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:15.335 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.782 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:15.388 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:15.388 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:15.388 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0404e-05 s, 0.0 kB/s 2023-10-10T20:37:15.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:15.389 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.783 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8369e-05 s, 0.0 kB/s 2023-10-10T20:37:15.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:15.444 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.784 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:15.498 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:15.498 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:15.499 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8704e-05 s, 0.0 kB/s 2023-10-10T20:37:15.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:15.499 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.785 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:15.553 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:15.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:15.554 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8462e-05 s, 0.0 kB/s 2023-10-10T20:37:15.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:15.555 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.786 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1458e-05 s, 0.0 kB/s 2023-10-10T20:37:15.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:15.610 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.787 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:15.664 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:15.665 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:15.665 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4013e-05 s, 0.0 kB/s 2023-10-10T20:37:15.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:15.666 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.788 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:15.722 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:15.722 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:15.722 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.809e-05 s, 0.0 kB/s 2023-10-10T20:37:15.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:15.723 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.789 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:15.777 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:15.777 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:15.777 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0198e-05 s, 0.0 kB/s 2023-10-10T20:37:15.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:15.778 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.790 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:15.832 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:15.832 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:15.832 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9234e-05 s, 0.0 kB/s 2023-10-10T20:37:15.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:15.833 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.791 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:15.887 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:15.887 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:15.887 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0815e-05 s, 0.0 kB/s 2023-10-10T20:37:15.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:15.888 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.792 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:15.942 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:15.942 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:15.942 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2438e-05 s, 0.0 kB/s 2023-10-10T20:37:15.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:15.943 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.793 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:15.997 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:15.997 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:15.998 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2785e-05 s, 0.0 kB/s 2023-10-10T20:37:15.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:15.998 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.794 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:16.053 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:16.053 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:16.054 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8181e-05 s, 0.0 kB/s 2023-10-10T20:37:16.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:16.054 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.795 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:16.108 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:16.109 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:16.109 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8467e-05 s, 0.0 kB/s 2023-10-10T20:37:16.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:16.110 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.796 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:16.164 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:16.164 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:16.164 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0996e-05 s, 0.0 kB/s 2023-10-10T20:37:16.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:16.165 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.797 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:16.219 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:16.219 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:16.220 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8061e-05 s, 0.0 kB/s 2023-10-10T20:37:16.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:16.220 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.798 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:16.275 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:16.275 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:16.275 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1973e-05 s, 0.0 kB/s 2023-10-10T20:37:16.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:16.276 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.799 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:16.330 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:16.331 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:16.331 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9326e-05 s, 0.0 kB/s 2023-10-10T20:37:16.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:16.332 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.800 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7912e-05 s, 0.0 kB/s 2023-10-10T20:37:16.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:16.387 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.801 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:16.441 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:16.441 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:16.442 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8975e-05 s, 0.0 kB/s 2023-10-10T20:37:16.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:16.442 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.802 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:16.497 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:16.498 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:16.498 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.1314e-05 s, 0.0 kB/s 2023-10-10T20:37:16.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:16.499 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.803 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0666e-05 s, 0.0 kB/s 2023-10-10T20:37:16.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:16.555 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.804 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:16.610 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:16.610 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:16.611 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.138e-05 s, 0.0 kB/s 2023-10-10T20:37:16.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:16.611 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.805 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:16.666 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:16.667 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:16.667 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2254e-05 s, 0.0 kB/s 2023-10-10T20:37:16.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:16.668 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.806 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:16.722 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:16.722 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:16.722 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.822e-05 s, 0.0 kB/s 2023-10-10T20:37:16.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:16.723 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.807 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:16.777 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:16.777 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:16.777 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9374e-05 s, 0.0 kB/s 2023-10-10T20:37:16.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:16.778 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.808 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:16.832 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:16.832 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:16.832 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9436e-05 s, 0.0 kB/s 2023-10-10T20:37:16.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:16.833 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.809 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:16.887 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:16.887 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:16.887 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9338e-05 s, 0.0 kB/s 2023-10-10T20:37:16.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:16.888 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.810 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:16.943 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:16.943 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1953e-05 s, 0.0 kB/s 2023-10-10T20:37:16.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:16.946 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.811 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:16.998 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:16.999 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:16.999 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0884e-05 s, 0.0 kB/s 2023-10-10T20:37:16.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:17.000 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.812 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:17.054 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:17.054 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:17.054 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7844e-05 s, 0.0 kB/s 2023-10-10T20:37:17.055 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:17.055 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.813 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:17.109 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:17.109 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:17.109 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2832e-05 s, 0.0 kB/s 2023-10-10T20:37:17.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:17.110 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.814 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:17.164 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:17.164 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:17.165 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9931e-05 s, 0.0 kB/s 2023-10-10T20:37:17.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:17.165 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.815 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:17.220 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:17.220 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:17.221 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7975e-05 s, 0.0 kB/s 2023-10-10T20:37:17.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:17.221 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.816 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:17.276 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:17.276 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:17.276 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.5977e-05 s, 0.0 kB/s 2023-10-10T20:37:17.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:17.277 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.817 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:17.331 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:17.332 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:17.332 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8743e-05 s, 0.0 kB/s 2023-10-10T20:37:17.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:17.333 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.818 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:17.387 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:17.387 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:17.387 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9431e-05 s, 0.0 kB/s 2023-10-10T20:37:17.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:17.388 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.819 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7497e-05 s, 0.0 kB/s 2023-10-10T20:37:17.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:17.443 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.820 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:17.498 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:17.499 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:17.499 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7514e-05 s, 0.0 kB/s 2023-10-10T20:37:17.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:17.500 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.821 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:17.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:17.555 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:17.555 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8562e-05 s, 0.0 kB/s 2023-10-10T20:37:17.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:17.556 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.822 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:17.610 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:17.610 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:17.610 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8219e-05 s, 0.0 kB/s 2023-10-10T20:37:17.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:17.611 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.823 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:17.665 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:17.665 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:17.666 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2498e-05 s, 0.0 kB/s 2023-10-10T20:37:17.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:17.666 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.824 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:17.721 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:17.722 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:17.722 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7288e-05 s, 0.0 kB/s 2023-10-10T20:37:17.722 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:17.723 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.825 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:17.777 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:17.777 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:17.777 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9764e-05 s, 0.0 kB/s 2023-10-10T20:37:17.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:17.778 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.826 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:17.832 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:17.832 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:17.832 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0648e-05 s, 0.0 kB/s 2023-10-10T20:37:17.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:17.833 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.827 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:17.886 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:17.887 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:17.887 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2806e-05 s, 0.0 kB/s 2023-10-10T20:37:17.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:17.888 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.828 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:17.943 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:17.943 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:17.944 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8639e-05 s, 0.0 kB/s 2023-10-10T20:37:17.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:17.944 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.829 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:17.999 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:18.000 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:18.000 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1466e-05 s, 0.0 kB/s 2023-10-10T20:37:18.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:18.001 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.830 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:18.055 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:18.055 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:18.055 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0038e-05 s, 0.0 kB/s 2023-10-10T20:37:18.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:18.056 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.831 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:18.110 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:18.111 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:18.111 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0178e-05 s, 0.0 kB/s 2023-10-10T20:37:18.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:18.111 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.832 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:18.166 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:18.166 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:18.166 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8851e-05 s, 0.0 kB/s 2023-10-10T20:37:18.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:18.167 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.833 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:18.221 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:18.222 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:18.222 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8677e-05 s, 0.0 kB/s 2023-10-10T20:37:18.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:18.223 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.834 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:18.277 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:18.277 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:18.277 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8691e-05 s, 0.0 kB/s 2023-10-10T20:37:18.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:18.278 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.835 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:18.332 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:18.332 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:18.332 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0599e-05 s, 0.0 kB/s 2023-10-10T20:37:18.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:18.333 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.836 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:18.387 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:18.387 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:18.387 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8031e-05 s, 0.0 kB/s 2023-10-10T20:37:18.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:18.388 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.837 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:18.442 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:18.442 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:18.443 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8192e-05 s, 0.0 kB/s 2023-10-10T20:37:18.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:18.444 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.838 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:18.499 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:18.499 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:18.499 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8772e-05 s, 0.0 kB/s 2023-10-10T20:37:18.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:18.500 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.839 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:18.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:18.555 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:18.555 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9522e-05 s, 0.0 kB/s 2023-10-10T20:37:18.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:18.556 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.840 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:18.610 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:18.610 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:18.610 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0735e-05 s, 0.0 kB/s 2023-10-10T20:37:18.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:18.611 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.841 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:18.665 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:18.665 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:18.665 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9477e-05 s, 0.0 kB/s 2023-10-10T20:37:18.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:18.666 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.842 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:18.722 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:18.722 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9116e-05 s, 0.0 kB/s 2023-10-10T20:37:18.722 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:18.723 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.843 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:18.776 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:18.777 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:18.777 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.141e-05 s, 0.0 kB/s 2023-10-10T20:37:18.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:18.778 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.844 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:18.832 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:18.833 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:18.833 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6796e-05 s, 0.0 kB/s 2023-10-10T20:37:18.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:18.834 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.845 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:18.888 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:18.888 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:18.889 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7227e-05 s, 0.0 kB/s 2023-10-10T20:37:18.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:18.889 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.846 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:18.944 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:18.944 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:18.944 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.5845e-05 s, 0.0 kB/s 2023-10-10T20:37:18.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:18.945 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.847 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:18.999 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:19.000 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:19.000 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7801e-05 s, 0.0 kB/s 2023-10-10T20:37:19.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:19.001 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.848 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:19.055 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:19.056 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:19.056 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2774e-05 s, 0.0 kB/s 2023-10-10T20:37:19.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:19.057 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.849 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:19.111 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:19.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:19.112 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8398e-05 s, 0.0 kB/s 2023-10-10T20:37:19.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:19.113 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.850 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:19.168 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:19.168 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:19.168 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9259e-05 s, 0.0 kB/s 2023-10-10T20:37:19.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:19.169 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.851 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:19.223 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:19.224 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:19.224 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9871e-05 s, 0.0 kB/s 2023-10-10T20:37:19.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:19.224 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.852 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:19.280 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:19.280 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:19.281 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6806e-05 s, 0.0 kB/s 2023-10-10T20:37:19.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:19.281 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.853 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:19.336 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:19.337 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:19.337 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1348e-05 s, 0.0 kB/s 2023-10-10T20:37:19.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:19.338 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.854 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:19.392 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:19.392 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:19.392 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0962e-05 s, 0.0 kB/s 2023-10-10T20:37:19.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:19.393 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.855 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:19.447 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:19.447 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:19.448 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9741e-05 s, 0.0 kB/s 2023-10-10T20:37:19.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:19.448 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.856 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:19.502 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:19.502 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:19.502 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0277e-05 s, 0.0 kB/s 2023-10-10T20:37:19.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:19.503 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.857 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:19.557 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:19.558 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:19.558 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7158e-05 s, 0.0 kB/s 2023-10-10T20:37:19.558 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:19.559 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.858 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:19.614 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7151e-05 s, 0.0 kB/s 2023-10-10T20:37:19.614 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:19.614 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.859 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:19.670 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:19.670 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:19.671 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8264e-05 s, 0.0 kB/s 2023-10-10T20:37:19.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:19.671 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.860 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:19.726 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:19.727 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:19.727 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7133e-05 s, 0.0 kB/s 2023-10-10T20:37:19.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:19.728 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.861 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:19.782 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:19.782 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:19.782 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7073e-05 s, 0.0 kB/s 2023-10-10T20:37:19.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:19.783 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.862 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:19.837 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:19.837 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:19.837 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7745e-05 s, 0.0 kB/s 2023-10-10T20:37:19.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:19.838 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.863 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:19.892 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:19.892 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:19.892 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1265e-05 s, 0.0 kB/s 2023-10-10T20:37:19.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:19.893 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.864 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:19.947 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:19.947 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:19.947 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.584e-05 s, 0.0 kB/s 2023-10-10T20:37:19.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:19.948 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.865 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:20.003 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:20.004 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:20.004 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1937e-05 s, 0.0 kB/s 2023-10-10T20:37:20.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:20.005 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.866 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:20.059 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:20.059 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:20.059 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8909e-05 s, 0.0 kB/s 2023-10-10T20:37:20.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:20.060 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.867 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:20.114 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:20.115 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:20.115 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2095e-05 s, 0.0 kB/s 2023-10-10T20:37:20.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:20.116 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.868 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:20.170 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:20.170 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:20.170 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0603e-05 s, 0.0 kB/s 2023-10-10T20:37:20.171 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:20.171 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.869 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:20.225 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:20.225 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:20.225 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0694e-05 s, 0.0 kB/s 2023-10-10T20:37:20.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:20.226 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.870 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:20.280 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:20.280 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:20.281 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1212e-05 s, 0.0 kB/s 2023-10-10T20:37:20.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:20.281 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.871 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:20.336 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:20.336 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:20.336 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8673e-05 s, 0.0 kB/s 2023-10-10T20:37:20.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:20.337 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.872 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9191e-05 s, 0.0 kB/s 2023-10-10T20:37:20.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:20.392 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.873 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:20.446 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:20.446 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:20.446 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7635e-05 s, 0.0 kB/s 2023-10-10T20:37:20.447 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:20.447 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.874 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0062e-05 s, 0.0 kB/s 2023-10-10T20:37:20.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:20.502 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.875 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:20.556 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:20.556 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:20.556 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1858e-05 s, 0.0 kB/s 2023-10-10T20:37:20.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:20.557 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.876 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:20.611 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:20.611 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:20.612 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8779e-05 s, 0.0 kB/s 2023-10-10T20:37:20.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:20.613 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.877 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:20.667 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:20.668 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:20.668 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8724e-05 s, 0.0 kB/s 2023-10-10T20:37:20.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:20.669 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.878 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:20.723 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:20.723 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:20.723 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8929e-05 s, 0.0 kB/s 2023-10-10T20:37:20.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:20.724 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.879 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:20.778 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:20.778 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:20.778 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6994e-05 s, 0.0 kB/s 2023-10-10T20:37:20.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:20.779 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.880 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:20.833 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:20.833 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:20.833 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7486e-05 s, 0.0 kB/s 2023-10-10T20:37:20.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:20.834 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.881 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:20.888 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:20.889 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:20.889 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0789e-05 s, 0.0 kB/s 2023-10-10T20:37:20.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:20.890 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.882 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:20.944 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:20.944 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:20.945 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9677e-05 s, 0.0 kB/s 2023-10-10T20:37:20.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:20.945 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.883 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:20.999 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:20.999 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:21.000 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3803e-05 s, 0.0 kB/s 2023-10-10T20:37:21.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:21.001 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.884 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:21.055 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:21.055 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:21.055 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9026e-05 s, 0.0 kB/s 2023-10-10T20:37:21.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:21.056 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.885 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:21.110 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:21.110 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:21.110 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8565e-05 s, 0.0 kB/s 2023-10-10T20:37:21.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:21.111 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.886 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:21.165 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:21.166 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:21.166 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7633e-05 s, 0.0 kB/s 2023-10-10T20:37:21.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:21.167 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.887 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:21.221 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:21.221 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:21.221 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9194e-05 s, 0.0 kB/s 2023-10-10T20:37:21.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:21.222 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.888 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1679e-05 s, 0.0 kB/s 2023-10-10T20:37:21.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:21.278 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.889 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:21.332 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:21.333 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:21.333 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1545e-05 s, 0.0 kB/s 2023-10-10T20:37:21.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:21.334 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.890 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:21.388 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:21.388 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:21.388 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6945e-05 s, 0.0 kB/s 2023-10-10T20:37:21.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:21.389 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.891 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:21.443 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:21.444 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:21.444 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8719e-05 s, 0.0 kB/s 2023-10-10T20:37:21.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:21.445 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.892 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:21.500 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9511e-05 s, 0.0 kB/s 2023-10-10T20:37:21.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:21.501 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.893 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:21.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:21.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:21.555 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.196e-05 s, 0.0 kB/s 2023-10-10T20:37:21.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:21.555 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.894 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:21.609 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:21.609 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:21.610 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6936e-05 s, 0.0 kB/s 2023-10-10T20:37:21.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:21.610 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.895 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9569e-05 s, 0.0 kB/s 2023-10-10T20:37:21.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:21.666 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.896 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:21.720 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:21.721 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:21.721 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8795e-05 s, 0.0 kB/s 2023-10-10T20:37:21.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:21.722 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.897 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:21.776 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:21.776 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:21.776 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9919e-05 s, 0.0 kB/s 2023-10-10T20:37:21.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:21.777 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.898 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:21.831 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:21.831 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:21.831 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0515e-05 s, 0.0 kB/s 2023-10-10T20:37:21.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:21.832 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.899 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:21.886 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:21.887 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:21.887 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9061e-05 s, 0.0 kB/s 2023-10-10T20:37:21.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:21.888 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.900 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:21.942 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:21.942 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:21.943 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.2039e-05 s, 0.0 kB/s 2023-10-10T20:37:21.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:21.943 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.901 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:21.998 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:21.998 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:21.998 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2748e-05 s, 0.0 kB/s 2023-10-10T20:37:21.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:21.999 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.902 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:22.054 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:22.054 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:22.054 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0356e-05 s, 0.0 kB/s 2023-10-10T20:37:22.055 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:22.055 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.903 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:22.109 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:22.109 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:22.109 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8042e-05 s, 0.0 kB/s 2023-10-10T20:37:22.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:22.110 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.904 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:22.164 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:22.165 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:22.165 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.793e-05 s, 0.0 kB/s 2023-10-10T20:37:22.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:22.166 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.905 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:22.220 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:22.220 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:22.220 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0554e-05 s, 0.0 kB/s 2023-10-10T20:37:22.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:22.221 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.906 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:22.275 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:22.275 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:22.276 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9239e-05 s, 0.0 kB/s 2023-10-10T20:37:22.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:22.276 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.907 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:22.330 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:22.331 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:22.331 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.7539e-05 s, 0.0 kB/s 2023-10-10T20:37:22.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:22.332 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.908 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:22.386 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:22.386 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:22.386 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0012e-05 s, 0.0 kB/s 2023-10-10T20:37:22.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:22.387 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.909 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:22.441 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:22.441 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:22.441 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7548e-05 s, 0.0 kB/s 2023-10-10T20:37:22.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:22.442 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.910 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:22.496 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:22.496 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:22.497 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8037e-05 s, 0.0 kB/s 2023-10-10T20:37:22.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:22.498 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.911 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8368e-05 s, 0.0 kB/s 2023-10-10T20:37:22.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:22.553 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.912 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:22.607 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:22.608 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:22.608 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.052e-05 s, 0.0 kB/s 2023-10-10T20:37:22.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:22.609 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.913 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:22.663 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:22.663 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:22.664 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8174e-05 s, 0.0 kB/s 2023-10-10T20:37:22.664 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:22.664 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.914 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:22.720 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:22.720 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7735e-05 s, 0.0 kB/s 2023-10-10T20:37:22.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:22.721 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.915 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:22.776 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:22.776 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:22.776 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.7829e-05 s, 0.0 kB/s 2023-10-10T20:37:22.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:22.777 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.916 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:22.831 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:22.831 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:22.831 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8626e-05 s, 0.0 kB/s 2023-10-10T20:37:22.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:22.832 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.917 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:22.887 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:22.888 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:22.888 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0441e-05 s, 0.0 kB/s 2023-10-10T20:37:22.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:22.889 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.918 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:22.942 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:22.943 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:22.943 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2412e-05 s, 0.0 kB/s 2023-10-10T20:37:22.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:22.944 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.919 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0446e-05 s, 0.0 kB/s 2023-10-10T20:37:23.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:23.001 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.920 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:23.055 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:23.056 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:23.056 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0841e-05 s, 0.0 kB/s 2023-10-10T20:37:23.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:23.057 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.921 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:23.111 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:23.111 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:23.112 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9535e-05 s, 0.0 kB/s 2023-10-10T20:37:23.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:23.112 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.922 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:23.167 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:23.168 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:23.168 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7947e-05 s, 0.0 kB/s 2023-10-10T20:37:23.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:23.169 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.923 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:23.223 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:23.223 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:23.224 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0014e-05 s, 0.0 kB/s 2023-10-10T20:37:23.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:23.224 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.924 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8869e-05 s, 0.0 kB/s 2023-10-10T20:37:23.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:23.280 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.925 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:23.334 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:23.334 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:23.334 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.921e-05 s, 0.0 kB/s 2023-10-10T20:37:23.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:23.335 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.926 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:23.389 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:23.389 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:23.389 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2844e-05 s, 0.0 kB/s 2023-10-10T20:37:23.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:23.390 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.927 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:23.445 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:23.445 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:23.445 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0409e-05 s, 0.0 kB/s 2023-10-10T20:37:23.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:23.446 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.928 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:23.500 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:23.500 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:23.500 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0446e-05 s, 0.0 kB/s 2023-10-10T20:37:23.501 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:23.501 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.929 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:23.555 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:23.556 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:23.556 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0122e-05 s, 0.0 kB/s 2023-10-10T20:37:23.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:23.557 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.930 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:23.611 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:23.611 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:23.612 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8897e-05 s, 0.0 kB/s 2023-10-10T20:37:23.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:23.612 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.931 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:23.666 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:23.666 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:23.667 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1128e-05 s, 0.0 kB/s 2023-10-10T20:37:23.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:23.667 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.932 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:23.721 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2606e-05 s, 0.0 kB/s 2023-10-10T20:37:23.722 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:23.723 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.933 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:23.777 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:23.778 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:23.778 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9065e-05 s, 0.0 kB/s 2023-10-10T20:37:23.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:23.779 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.934 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:23.833 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:23.833 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:23.833 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7681e-05 s, 0.0 kB/s 2023-10-10T20:37:23.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:23.834 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.935 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:23.888 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:23.888 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:23.888 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9986e-05 s, 0.0 kB/s 2023-10-10T20:37:23.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:23.889 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.936 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:23.943 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:23.943 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:23.943 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8984e-05 s, 0.0 kB/s 2023-10-10T20:37:23.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:23.944 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.937 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:23.999 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:23.999 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:23.999 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1124e-05 s, 0.0 kB/s 2023-10-10T20:37:24.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:24.000 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.938 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.4687e-05 s, 0.0 kB/s 2023-10-10T20:37:24.055 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:24.056 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.939 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:24.110 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:24.110 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:24.110 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6156e-05 s, 0.0 kB/s 2023-10-10T20:37:24.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:24.113 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.940 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:24.165 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:24.165 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:24.165 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.9527e-05 s, 0.0 kB/s 2023-10-10T20:37:24.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:24.166 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.941 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:24.220 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:24.221 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:24.221 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.107e-05 s, 0.0 kB/s 2023-10-10T20:37:24.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:24.222 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.942 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:24.276 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:24.338 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:24.338 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9384e-05 s, 0.0 kB/s 2023-10-10T20:37:24.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:24.339 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.943 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:24.354 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:24.354 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:24.354 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1155e-05 s, 0.0 kB/s 2023-10-10T20:37:24.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:24.355 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.944 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:24.410 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:24.411 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:24.411 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.26e-05 s, 0.0 kB/s 2023-10-10T20:37:24.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:24.412 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.945 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:24.467 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:24.467 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:24.467 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9502e-05 s, 0.0 kB/s 2023-10-10T20:37:24.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:24.469 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.946 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:24.522 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:24.523 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:24.523 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.759e-05 s, 0.0 kB/s 2023-10-10T20:37:24.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:24.525 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.947 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:24.578 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:24.578 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:24.579 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1279e-05 s, 0.0 kB/s 2023-10-10T20:37:24.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:24.579 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.948 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:24.634 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:24.635 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:24.635 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8449e-05 s, 0.0 kB/s 2023-10-10T20:37:24.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:24.636 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.949 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:24.690 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:24.690 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:24.690 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0776e-05 s, 0.0 kB/s 2023-10-10T20:37:24.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:24.691 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.950 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:24.745 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:24.745 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:24.745 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7275e-05 s, 0.0 kB/s 2023-10-10T20:37:24.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:24.746 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.951 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:24.800 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:24.800 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:24.800 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6059e-05 s, 0.0 kB/s 2023-10-10T20:37:24.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:24.801 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.952 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:24.855 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:24.856 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:24.856 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8911e-05 s, 0.0 kB/s 2023-10-10T20:37:24.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:24.857 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.953 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:24.910 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:24.911 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:24.911 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.7253e-05 s, 0.0 kB/s 2023-10-10T20:37:24.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:24.912 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.954 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:24.967 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:24.967 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:24.967 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2319e-05 s, 0.0 kB/s 2023-10-10T20:37:24.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:24.968 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.955 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:25.022 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6981e-05 s, 0.0 kB/s 2023-10-10T20:37:25.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:25.024 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.956 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:25.078 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:25.079 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:25.079 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9887e-05 s, 0.0 kB/s 2023-10-10T20:37:25.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:25.080 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.957 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:25.134 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:25.134 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:25.135 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2438e-05 s, 0.0 kB/s 2023-10-10T20:37:25.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:25.135 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.958 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:25.190 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:25.191 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:25.191 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7139e-05 s, 0.0 kB/s 2023-10-10T20:37:25.191 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:25.191 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.959 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:25.246 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:25.246 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:25.246 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1521e-05 s, 0.0 kB/s 2023-10-10T20:37:25.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:25.247 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.960 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:25.301 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:25.302 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:25.302 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6794e-05 s, 0.0 kB/s 2023-10-10T20:37:25.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:25.303 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.961 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:25.357 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:25.357 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:25.357 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.5551e-05 s, 0.0 kB/s 2023-10-10T20:37:25.358 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:25.358 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.962 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:25.412 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:25.413 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:25.413 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0377e-05 s, 0.0 kB/s 2023-10-10T20:37:25.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:25.414 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.963 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:25.468 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:25.468 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:25.468 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6566e-05 s, 0.0 kB/s 2023-10-10T20:37:25.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:25.469 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.964 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:25.522 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:25.523 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:25.523 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.1391e-05 s, 0.0 kB/s 2023-10-10T20:37:25.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:25.524 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.965 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:25.579 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:25.579 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:25.580 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6507e-05 s, 0.0 kB/s 2023-10-10T20:37:25.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:25.580 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.966 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:25.635 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:25.635 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:25.636 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9206e-05 s, 0.0 kB/s 2023-10-10T20:37:25.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:25.636 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.967 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:25.690 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:25.691 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:25.691 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0762e-05 s, 0.0 kB/s 2023-10-10T20:37:25.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:25.692 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.968 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:25.746 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:25.747 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:25.749 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.954e-05 s, 0.0 kB/s 2023-10-10T20:37:25.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:25.751 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.969 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:25.805 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:25.805 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:25.805 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8058e-05 s, 0.0 kB/s 2023-10-10T20:37:25.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:25.806 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.970 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:25.860 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:25.860 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:25.860 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8578e-05 s, 0.0 kB/s 2023-10-10T20:37:25.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:25.861 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.971 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:25.915 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:25.916 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:25.916 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0078e-05 s, 0.0 kB/s 2023-10-10T20:37:25.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:25.917 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.972 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:25.971 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:25.971 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:25.971 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2452e-05 s, 0.0 kB/s 2023-10-10T20:37:25.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:25.972 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.973 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:26.026 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:26.027 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:26.027 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9493e-05 s, 0.0 kB/s 2023-10-10T20:37:26.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:26.028 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.974 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:26.082 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:26.082 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:26.082 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8147e-05 s, 0.0 kB/s 2023-10-10T20:37:26.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:26.083 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.975 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:26.137 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:26.137 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:26.137 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8767e-05 s, 0.0 kB/s 2023-10-10T20:37:26.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:26.138 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.976 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:26.192 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:26.192 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:26.193 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2871e-05 s, 0.0 kB/s 2023-10-10T20:37:26.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:26.193 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.977 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:26.247 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:26.247 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:26.247 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1134e-05 s, 0.0 kB/s 2023-10-10T20:37:26.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:26.248 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.978 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:26.303 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:26.303 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:26.303 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1142e-05 s, 0.0 kB/s 2023-10-10T20:37:26.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:26.304 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.979 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:26.358 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:26.359 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:26.359 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9349e-05 s, 0.0 kB/s 2023-10-10T20:37:26.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:26.360 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.980 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:26.414 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:26.415 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:26.415 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9382e-05 s, 0.0 kB/s 2023-10-10T20:37:26.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:26.416 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.981 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:26.470 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:26.470 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:26.470 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1527e-05 s, 0.0 kB/s 2023-10-10T20:37:26.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:26.471 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.982 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0894e-05 s, 0.0 kB/s 2023-10-10T20:37:26.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:26.526 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.983 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:26.580 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:26.581 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:26.582 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6707e-05 s, 0.0 kB/s 2023-10-10T20:37:26.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:26.583 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.984 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:26.636 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:26.636 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:26.636 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9002e-05 s, 0.0 kB/s 2023-10-10T20:37:26.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:26.637 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.985 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:26.691 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:26.691 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:26.691 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7856e-05 s, 0.0 kB/s 2023-10-10T20:37:26.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:26.692 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.986 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:26.746 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:26.746 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:26.746 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0316e-05 s, 0.0 kB/s 2023-10-10T20:37:26.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:26.747 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.987 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:26.801 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:26.802 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:26.802 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.367e-05 s, 0.0 kB/s 2023-10-10T20:37:26.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:26.803 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.988 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:26.857 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:26.857 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:26.857 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.936e-05 s, 0.0 kB/s 2023-10-10T20:37:26.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:26.858 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.989 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:26.912 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:26.912 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:26.912 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0807e-05 s, 0.0 kB/s 2023-10-10T20:37:26.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:26.913 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.990 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:26.967 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.863e-05 s, 0.0 kB/s 2023-10-10T20:37:26.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:26.969 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.991 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0229e-05 s, 0.0 kB/s 2023-10-10T20:37:27.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:27.025 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.992 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:27.079 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:27.079 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:27.079 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9446e-05 s, 0.0 kB/s 2023-10-10T20:37:27.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:27.080 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.993 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:27.134 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:27.134 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:27.134 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0093e-05 s, 0.0 kB/s 2023-10-10T20:37:27.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:27.135 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.994 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:27.189 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:27.190 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:27.190 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.947e-05 s, 0.0 kB/s 2023-10-10T20:37:27.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:27.191 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.995 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1271e-05 s, 0.0 kB/s 2023-10-10T20:37:27.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:27.246 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.996 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:27.300 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:27.300 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:27.301 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.9439e-05 s, 0.0 kB/s 2023-10-10T20:37:27.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:27.301 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.997 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:27.357 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:27.357 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0719e-05 s, 0.0 kB/s 2023-10-10T20:37:27.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:27.358 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.998 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:27.412 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:27.413 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:27.413 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1887e-05 s, 0.0 kB/s 2023-10-10T20:37:27.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:27.414 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.999 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:27.468 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:27.468 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:27.468 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2819e-05 s, 0.0 kB/s 2023-10-10T20:37:27.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:27.469 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1000 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:27.523 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:27.523 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:27.523 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0345e-05 s, 0.0 kB/s 2023-10-10T20:37:27.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:27.524 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1001 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:27.578 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:27.578 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:27.579 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8331e-05 s, 0.0 kB/s 2023-10-10T20:37:27.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:27.579 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1002 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:27.634 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:27.635 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:27.635 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9118e-05 s, 0.0 kB/s 2023-10-10T20:37:27.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:27.636 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1003 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:27.690 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:27.690 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:27.690 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7481e-05 s, 0.0 kB/s 2023-10-10T20:37:27.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:27.691 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1004 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:27.745 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:27.745 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:27.745 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.138e-05 s, 0.0 kB/s 2023-10-10T20:37:27.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:27.746 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1005 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:27.800 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:27.800 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:27.800 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2588e-05 s, 0.0 kB/s 2023-10-10T20:37:27.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:27.801 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1006 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:27.855 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:27.855 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:27.855 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0234e-05 s, 0.0 kB/s 2023-10-10T20:37:27.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:27.856 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1007 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:27.911 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:27.911 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:27.911 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9039e-05 s, 0.0 kB/s 2023-10-10T20:37:27.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:27.912 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1008 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:27.966 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:27.966 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:27.967 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.971e-05 s, 0.0 kB/s 2023-10-10T20:37:27.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:27.967 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1009 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:28.022 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:28.022 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:28.022 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1776e-05 s, 0.0 kB/s 2023-10-10T20:37:28.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:28.023 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1010 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:28.077 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:28.077 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:28.077 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7458e-05 s, 0.0 kB/s 2023-10-10T20:37:28.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:28.078 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1011 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0945e-05 s, 0.0 kB/s 2023-10-10T20:37:28.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:28.133 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1012 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:28.187 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:28.187 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:28.187 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.899e-05 s, 0.0 kB/s 2023-10-10T20:37:28.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:28.188 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1013 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:28.242 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:28.242 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:28.243 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9422e-05 s, 0.0 kB/s 2023-10-10T20:37:28.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:28.243 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1014 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:28.297 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:28.297 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:28.298 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1141e-05 s, 0.0 kB/s 2023-10-10T20:37:28.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:28.298 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1015 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:28.354 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0242e-05 s, 0.0 kB/s 2023-10-10T20:37:28.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:28.354 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1016 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:28.409 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:28.409 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:28.410 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0954e-05 s, 0.0 kB/s 2023-10-10T20:37:28.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:28.410 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1017 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:28.465 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:28.465 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:28.465 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7146e-05 s, 0.0 kB/s 2023-10-10T20:37:28.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:28.466 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1018 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:28.520 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:28.520 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:28.520 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.949e-05 s, 0.0 kB/s 2023-10-10T20:37:28.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:28.521 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1019 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:28.575 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:28.575 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:28.575 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7419e-05 s, 0.0 kB/s 2023-10-10T20:37:28.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:28.576 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1020 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:28.630 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:28.630 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:28.631 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.0128e-05 s, 0.0 kB/s 2023-10-10T20:37:28.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:28.631 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1021 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:28.685 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:28.685 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:28.685 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.714e-05 s, 0.0 kB/s 2023-10-10T20:37:28.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:28.686 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1022 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:28.740 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:28.740 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:28.741 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1263e-05 s, 0.0 kB/s 2023-10-10T20:37:28.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:28.742 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1023 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:28.796 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:28.796 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:28.797 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7626e-05 s, 0.0 kB/s 2023-10-10T20:37:28.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:28.797 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1024 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:28.852 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:28.852 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:28.852 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1086e-05 s, 0.0 kB/s 2023-10-10T20:37:28.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:28.853 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1025 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:28.907 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:28.908 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:28.908 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7659e-05 s, 0.0 kB/s 2023-10-10T20:37:28.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:28.909 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1026 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:28.963 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:28.963 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:28.963 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.882e-05 s, 0.0 kB/s 2023-10-10T20:37:28.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:28.964 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1027 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:29.018 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:29.018 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:29.021 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6939e-05 s, 0.0 kB/s 2023-10-10T20:37:29.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:29.022 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1028 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.747e-05 s, 0.0 kB/s 2023-10-10T20:37:29.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:29.074 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1029 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:29.128 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:29.128 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:29.128 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5872e-05 s, 0.0 kB/s 2023-10-10T20:37:29.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:29.129 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1030 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:29.183 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:29.184 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:29.184 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7074e-05 s, 0.0 kB/s 2023-10-10T20:37:29.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:29.184 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1031 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:29.239 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:29.240 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:29.240 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1707e-05 s, 0.0 kB/s 2023-10-10T20:37:29.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:29.241 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1032 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:29.295 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:29.296 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:29.296 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9573e-05 s, 0.0 kB/s 2023-10-10T20:37:29.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:29.296 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1033 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:29.351 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:29.351 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:29.351 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1379e-05 s, 0.0 kB/s 2023-10-10T20:37:29.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:29.352 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1034 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:29.411 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:29.412 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:29.413 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0594e-05 s, 0.0 kB/s 2023-10-10T20:37:29.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:29.413 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1035 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:29.470 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:29.470 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:29.471 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7231e-05 s, 0.0 kB/s 2023-10-10T20:37:29.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:29.472 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1036 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:29.525 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:29.525 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:29.525 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0761e-05 s, 0.0 kB/s 2023-10-10T20:37:29.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:29.526 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1037 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:29.580 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:29.580 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:29.580 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8043e-05 s, 0.0 kB/s 2023-10-10T20:37:29.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:29.581 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1038 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:29.635 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:29.635 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:29.635 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0427e-05 s, 0.0 kB/s 2023-10-10T20:37:29.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:29.636 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1039 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:29.690 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:29.690 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:29.690 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.094e-05 s, 0.0 kB/s 2023-10-10T20:37:29.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:29.691 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1040 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:29.745 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:29.746 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:29.746 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8552e-05 s, 0.0 kB/s 2023-10-10T20:37:29.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:29.747 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1041 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:29.801 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:29.801 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:29.801 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0176e-05 s, 0.0 kB/s 2023-10-10T20:37:29.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:29.802 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1042 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:29.856 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:29.856 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:29.857 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.828e-05 s, 0.0 kB/s 2023-10-10T20:37:29.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:29.857 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1043 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:29.911 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:29.912 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:29.912 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1455e-05 s, 0.0 kB/s 2023-10-10T20:37:29.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:29.913 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1044 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:29.968 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:29.968 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:29.968 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9895e-05 s, 0.0 kB/s 2023-10-10T20:37:29.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:29.969 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1045 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:30.023 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:30.023 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:30.023 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0595e-05 s, 0.0 kB/s 2023-10-10T20:37:30.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:30.024 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1046 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:30.078 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:30.078 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:30.078 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0291e-05 s, 0.0 kB/s 2023-10-10T20:37:30.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:30.079 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1047 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:30.133 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:30.133 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:30.133 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2073e-05 s, 0.0 kB/s 2023-10-10T20:37:30.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:30.134 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1048 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:30.188 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:30.189 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:30.189 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9123e-05 s, 0.0 kB/s 2023-10-10T20:37:30.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:30.189 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1049 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:30.246 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:30.246 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:30.246 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2293e-05 s, 0.0 kB/s 2023-10-10T20:37:30.247 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:30.247 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1050 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:30.301 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:30.301 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:30.301 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5618e-05 s, 0.0 kB/s 2023-10-10T20:37:30.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:30.302 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1051 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:30.355 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:30.356 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:30.356 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9873e-05 s, 0.0 kB/s 2023-10-10T20:37:30.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:30.356 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1052 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:30.412 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:30.412 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:30.412 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8385e-05 s, 0.0 kB/s 2023-10-10T20:37:30.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:30.413 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1053 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:30.467 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:30.468 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:30.468 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7787e-05 s, 0.0 kB/s 2023-10-10T20:37:30.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:30.469 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1054 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:30.523 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:30.523 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:30.524 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0251e-05 s, 0.0 kB/s 2023-10-10T20:37:30.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:30.524 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1055 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:30.578 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:30.578 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:30.578 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0446e-05 s, 0.0 kB/s 2023-10-10T20:37:30.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:30.579 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1056 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:30.633 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:30.633 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:30.633 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9488e-05 s, 0.0 kB/s 2023-10-10T20:37:30.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:30.634 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1057 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:30.688 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:30.688 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:30.688 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6772e-05 s, 0.0 kB/s 2023-10-10T20:37:30.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:30.689 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1058 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:30.743 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:30.743 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:30.743 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9491e-05 s, 0.0 kB/s 2023-10-10T20:37:30.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:30.744 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1059 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:30.798 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:30.798 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:30.799 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9996e-05 s, 0.0 kB/s 2023-10-10T20:37:30.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:30.799 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1060 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:30.853 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:30.854 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:30.854 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9311e-05 s, 0.0 kB/s 2023-10-10T20:37:30.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:30.854 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1061 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:30.909 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:30.909 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:30.909 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.8193e-05 s, 0.0 kB/s 2023-10-10T20:37:30.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:30.910 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1062 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1899e-05 s, 0.0 kB/s 2023-10-10T20:37:30.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:30.965 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1063 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:31.019 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:31.019 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:31.020 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0567e-05 s, 0.0 kB/s 2023-10-10T20:37:31.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:31.020 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1064 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:31.074 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:31.074 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:31.075 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6996e-05 s, 0.0 kB/s 2023-10-10T20:37:31.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:31.075 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1065 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:31.130 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:31.130 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:31.131 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9309e-05 s, 0.0 kB/s 2023-10-10T20:37:31.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:31.131 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1066 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:31.186 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:31.186 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:31.186 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9448e-05 s, 0.0 kB/s 2023-10-10T20:37:31.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:31.187 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1067 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:31.241 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.405e-05 s, 0.0 kB/s 2023-10-10T20:37:31.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:31.243 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1068 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:31.298 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:31.299 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:31.299 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0736e-05 s, 0.0 kB/s 2023-10-10T20:37:31.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:31.300 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1069 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:31.354 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:31.354 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:31.354 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6883e-05 s, 0.0 kB/s 2023-10-10T20:37:31.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:31.355 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1070 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:31.409 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:31.410 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:31.410 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.174e-05 s, 0.0 kB/s 2023-10-10T20:37:31.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:31.411 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1071 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:31.465 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:31.466 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:31.466 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9587e-05 s, 0.0 kB/s 2023-10-10T20:37:31.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:31.467 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1072 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:31.521 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:31.521 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:31.521 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7362e-05 s, 0.0 kB/s 2023-10-10T20:37:31.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:31.522 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1073 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:31.575 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:31.576 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:31.576 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1322e-05 s, 0.0 kB/s 2023-10-10T20:37:31.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:31.577 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1074 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:31.631 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:31.631 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:31.631 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1663e-05 s, 0.0 kB/s 2023-10-10T20:37:31.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:31.632 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1075 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:31.688 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:31.689 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:31.689 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6984e-05 s, 0.0 kB/s 2023-10-10T20:37:31.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:31.690 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1076 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:31.743 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:31.743 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:31.744 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8342e-05 s, 0.0 kB/s 2023-10-10T20:37:31.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:31.744 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1077 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:31.798 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:31.798 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:31.798 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1143e-05 s, 0.0 kB/s 2023-10-10T20:37:31.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:31.799 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1078 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:31.853 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:31.853 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:31.853 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0112e-05 s, 0.0 kB/s 2023-10-10T20:37:31.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:31.854 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1079 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7023e-05 s, 0.0 kB/s 2023-10-10T20:37:31.909 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:31.909 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1080 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:31.964 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:31.964 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:31.965 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0286e-05 s, 0.0 kB/s 2023-10-10T20:37:31.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:31.965 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1081 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:32.020 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:32.020 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:32.021 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1094e-05 s, 0.0 kB/s 2023-10-10T20:37:32.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:32.021 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1082 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:32.076 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:32.076 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:32.076 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.155e-05 s, 0.0 kB/s 2023-10-10T20:37:32.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:32.077 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1083 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:32.131 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:32.131 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:32.131 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8951e-05 s, 0.0 kB/s 2023-10-10T20:37:32.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:32.132 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1084 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:32.185 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:32.185 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:32.186 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1945e-05 s, 0.0 kB/s 2023-10-10T20:37:32.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:32.187 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1085 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.711e-05 s, 0.0 kB/s 2023-10-10T20:37:32.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:32.243 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1086 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:32.299 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:32.300 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:32.300 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1999e-05 s, 0.0 kB/s 2023-10-10T20:37:32.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:32.301 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1087 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:32.355 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:32.355 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:32.355 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9071e-05 s, 0.0 kB/s 2023-10-10T20:37:32.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:32.356 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1088 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:32.411 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:32.411 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2128e-05 s, 0.0 kB/s 2023-10-10T20:37:32.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:32.412 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1089 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:32.467 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:32.467 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:32.467 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1107e-05 s, 0.0 kB/s 2023-10-10T20:37:32.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:32.468 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1090 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2182e-05 s, 0.0 kB/s 2023-10-10T20:37:32.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:32.523 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1091 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5071e-05 s, 0.0 kB/s 2023-10-10T20:37:32.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:32.578 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1092 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:32.632 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:32.632 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:32.632 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9232e-05 s, 0.0 kB/s 2023-10-10T20:37:32.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:32.633 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1093 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:32.687 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:32.687 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:32.688 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2815e-05 s, 0.0 kB/s 2023-10-10T20:37:32.688 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:32.688 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1094 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:32.742 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:32.743 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:32.743 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8262e-05 s, 0.0 kB/s 2023-10-10T20:37:32.743 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:32.744 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1095 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7046e-05 s, 0.0 kB/s 2023-10-10T20:37:32.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:32.799 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1096 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:32.854 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:32.854 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:32.854 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.24e-05 s, 0.0 kB/s 2023-10-10T20:37:32.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:32.855 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1097 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:32.909 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:32.909 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:32.910 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0453e-05 s, 0.0 kB/s 2023-10-10T20:37:32.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:32.910 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1098 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7647e-05 s, 0.0 kB/s 2023-10-10T20:37:32.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:32.966 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1099 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:33.021 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:33.021 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0021e-05 s, 0.0 kB/s 2023-10-10T20:37:33.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:33.022 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1100 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:33.076 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:33.076 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:33.076 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9077e-05 s, 0.0 kB/s 2023-10-10T20:37:33.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:33.077 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1101 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:33.131 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:33.132 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:33.132 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9025e-05 s, 0.0 kB/s 2023-10-10T20:37:33.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:33.133 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1102 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:33.187 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:33.187 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:33.187 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3364e-05 s, 0.0 kB/s 2023-10-10T20:37:33.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:33.188 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1103 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:33.242 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:33.243 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:33.243 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8771e-05 s, 0.0 kB/s 2023-10-10T20:37:33.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:33.244 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1104 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:33.298 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:33.298 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:33.298 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8153e-05 s, 0.0 kB/s 2023-10-10T20:37:33.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:33.299 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1105 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9208e-05 s, 0.0 kB/s 2023-10-10T20:37:33.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:33.354 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1106 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:33.408 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:33.409 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:33.409 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7607e-05 s, 0.0 kB/s 2023-10-10T20:37:33.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:33.410 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1107 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:33.464 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:33.464 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:33.465 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8329e-05 s, 0.0 kB/s 2023-10-10T20:37:33.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:33.465 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1108 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:33.520 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:33.520 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:33.521 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0196e-05 s, 0.0 kB/s 2023-10-10T20:37:33.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:33.521 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1109 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:33.576 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:33.576 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2101e-05 s, 0.0 kB/s 2023-10-10T20:37:33.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:33.577 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1110 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:33.631 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:33.632 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:33.632 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.912e-05 s, 0.0 kB/s 2023-10-10T20:37:33.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:33.632 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1111 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:33.687 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:33.688 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:33.688 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8653e-05 s, 0.0 kB/s 2023-10-10T20:37:33.688 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:33.689 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1112 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:33.743 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:33.743 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:33.743 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9638e-05 s, 0.0 kB/s 2023-10-10T20:37:33.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:33.744 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1113 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:33.798 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:33.799 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:33.799 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8155e-05 s, 0.0 kB/s 2023-10-10T20:37:33.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:33.800 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1114 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:33.853 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:33.853 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:33.853 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0183e-05 s, 0.0 kB/s 2023-10-10T20:37:33.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:33.854 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1115 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:33.908 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:33.909 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:33.909 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2568e-05 s, 0.0 kB/s 2023-10-10T20:37:33.909 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:33.910 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1116 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:33.964 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:33.964 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:33.964 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8204e-05 s, 0.0 kB/s 2023-10-10T20:37:33.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:33.965 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1117 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:34.019 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:34.019 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:34.020 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9314e-05 s, 0.0 kB/s 2023-10-10T20:37:34.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:34.020 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1118 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:34.075 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:34.075 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:34.075 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0446e-05 s, 0.0 kB/s 2023-10-10T20:37:34.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:34.076 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1119 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:34.130 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:34.130 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:34.130 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.261e-05 s, 0.0 kB/s 2023-10-10T20:37:34.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:34.131 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1120 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:34.184 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:34.185 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:34.185 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2545e-05 s, 0.0 kB/s 2023-10-10T20:37:34.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:34.186 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1121 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:34.240 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:34.241 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:34.241 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0139e-05 s, 0.0 kB/s 2023-10-10T20:37:34.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:34.242 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1122 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:34.295 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:34.296 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:34.296 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7228e-05 s, 0.0 kB/s 2023-10-10T20:37:34.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:34.297 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1123 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:34.351 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:34.351 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:34.351 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9168e-05 s, 0.0 kB/s 2023-10-10T20:37:34.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:34.352 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1124 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:34.407 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:34.407 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:34.407 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2483e-05 s, 0.0 kB/s 2023-10-10T20:37:34.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:34.408 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1125 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:34.462 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:34.462 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:34.463 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.7794e-05 s, 0.0 kB/s 2023-10-10T20:37:34.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:34.463 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1126 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:34.517 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:34.517 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:34.518 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6232e-05 s, 0.0 kB/s 2023-10-10T20:37:34.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:34.518 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1127 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:34.573 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:34.574 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:34.574 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7305e-05 s, 0.0 kB/s 2023-10-10T20:37:34.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:34.575 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1128 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:34.629 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8695e-05 s, 0.0 kB/s 2023-10-10T20:37:34.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:34.630 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1129 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:34.685 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:34.686 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:34.686 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9466e-05 s, 0.0 kB/s 2023-10-10T20:37:34.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:34.687 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1130 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:34.741 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:34.741 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:34.742 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9673e-05 s, 0.0 kB/s 2023-10-10T20:37:34.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:34.742 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1131 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:34.797 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:34.797 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:34.797 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8174e-05 s, 0.0 kB/s 2023-10-10T20:37:34.798 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:34.798 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1132 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:34.852 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:34.852 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:34.852 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8316e-05 s, 0.0 kB/s 2023-10-10T20:37:34.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:34.853 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1133 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:34.907 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:34.907 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:34.907 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9764e-05 s, 0.0 kB/s 2023-10-10T20:37:34.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:34.908 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1134 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:34.962 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:34.963 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:34.963 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7247e-05 s, 0.0 kB/s 2023-10-10T20:37:34.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:34.964 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1135 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:35.018 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:35.018 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:35.018 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.851e-05 s, 0.0 kB/s 2023-10-10T20:37:35.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:35.019 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1136 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:35.073 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:35.073 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:35.073 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.882e-05 s, 0.0 kB/s 2023-10-10T20:37:35.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:35.074 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1137 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:35.128 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:35.128 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:35.129 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0005e-05 s, 0.0 kB/s 2023-10-10T20:37:35.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:35.131 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1138 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:35.184 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:35.184 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:35.185 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9359e-05 s, 0.0 kB/s 2023-10-10T20:37:35.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:35.185 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1139 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:35.240 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:35.240 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:35.240 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9096e-05 s, 0.0 kB/s 2023-10-10T20:37:35.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:35.241 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1140 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:35.295 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:35.296 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:35.296 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3226e-05 s, 0.0 kB/s 2023-10-10T20:37:35.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:35.297 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1141 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:35.351 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:35.352 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:35.352 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7939e-05 s, 0.0 kB/s 2023-10-10T20:37:35.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:35.353 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1142 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:35.407 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:35.408 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:35.408 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9993e-05 s, 0.0 kB/s 2023-10-10T20:37:35.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:35.409 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1143 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:35.462 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:35.463 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:35.463 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.809e-05 s, 0.0 kB/s 2023-10-10T20:37:35.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:35.464 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1144 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:35.518 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:35.519 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:35.519 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.786e-05 s, 0.0 kB/s 2023-10-10T20:37:35.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:35.520 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1145 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:35.575 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:35.575 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7785e-05 s, 0.0 kB/s 2023-10-10T20:37:35.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:35.576 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1146 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:35.630 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:35.631 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:35.631 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.706e-05 s, 0.0 kB/s 2023-10-10T20:37:35.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:35.632 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1147 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:35.686 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:35.686 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:35.686 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9851e-05 s, 0.0 kB/s 2023-10-10T20:37:35.688 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:35.688 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1148 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:35.743 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:35.743 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:35.743 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8739e-05 s, 0.0 kB/s 2023-10-10T20:37:35.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:35.744 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1149 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:35.798 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:35.798 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2987e-05 s, 0.0 kB/s 2023-10-10T20:37:35.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:35.799 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1150 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:35.853 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:35.853 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:35.854 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8401e-05 s, 0.0 kB/s 2023-10-10T20:37:35.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:35.854 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1151 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:35.909 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:35.910 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:35.910 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7631e-05 s, 0.0 kB/s 2023-10-10T20:37:35.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:35.911 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1152 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:35.965 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:35.967 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:35.967 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6607e-05 s, 0.0 kB/s 2023-10-10T20:37:35.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:35.968 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1153 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:36.021 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:36.021 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:36.022 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.1829e-05 s, 0.0 kB/s 2023-10-10T20:37:36.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:36.022 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1154 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:36.077 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:36.078 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:36.078 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0865e-05 s, 0.0 kB/s 2023-10-10T20:37:36.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:36.079 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1155 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:36.133 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:36.133 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:36.133 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7719e-05 s, 0.0 kB/s 2023-10-10T20:37:36.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:36.134 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1156 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:36.187 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:36.188 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:36.188 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6895e-05 s, 0.0 kB/s 2023-10-10T20:37:36.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:36.189 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1157 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:36.243 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:36.244 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:36.244 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6883e-05 s, 0.0 kB/s 2023-10-10T20:37:36.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:36.245 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1158 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:36.299 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:36.299 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:36.300 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6802e-05 s, 0.0 kB/s 2023-10-10T20:37:36.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:36.300 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1159 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:36.355 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:36.355 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:36.355 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9869e-05 s, 0.0 kB/s 2023-10-10T20:37:36.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:36.356 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1160 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:36.411 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:36.411 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:36.412 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3378e-05 s, 0.0 kB/s 2023-10-10T20:37:36.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:36.412 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1161 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:36.467 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:36.468 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:36.468 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9922e-05 s, 0.0 kB/s 2023-10-10T20:37:36.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:36.469 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1162 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:36.522 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:36.523 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:36.523 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4063e-05 s, 0.0 kB/s 2023-10-10T20:37:36.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:36.524 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1163 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:36.579 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:36.580 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:36.580 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9349e-05 s, 0.0 kB/s 2023-10-10T20:37:36.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:36.581 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1164 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:36.635 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:36.635 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:36.635 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9251e-05 s, 0.0 kB/s 2023-10-10T20:37:36.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:36.636 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1165 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:36.690 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:36.690 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:36.690 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0746e-05 s, 0.0 kB/s 2023-10-10T20:37:36.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:36.691 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1166 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:36.745 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:36.746 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:36.746 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9488e-05 s, 0.0 kB/s 2023-10-10T20:37:36.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:36.747 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1167 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:36.801 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:36.801 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:36.801 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5484e-05 s, 0.0 kB/s 2023-10-10T20:37:36.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:36.802 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1168 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:36.856 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:36.856 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:36.857 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0735e-05 s, 0.0 kB/s 2023-10-10T20:37:36.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:36.857 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1169 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:36.911 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:36.911 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:36.912 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3089e-05 s, 0.0 kB/s 2023-10-10T20:37:36.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:36.912 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1170 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:36.967 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:36.967 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:36.968 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0194e-05 s, 0.0 kB/s 2023-10-10T20:37:36.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:36.968 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1171 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:37.023 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:37.024 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:37.024 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9589e-05 s, 0.0 kB/s 2023-10-10T20:37:37.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:37.025 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1172 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:37.080 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:37.080 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:37.080 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2132e-05 s, 0.0 kB/s 2023-10-10T20:37:37.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:37.081 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1173 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:37.135 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:37.136 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:37.136 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9143e-05 s, 0.0 kB/s 2023-10-10T20:37:37.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:37.136 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1174 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:37.191 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:37.191 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:37.191 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.996e-05 s, 0.0 kB/s 2023-10-10T20:37:37.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:37.192 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1175 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:37.246 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:37.246 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:37.246 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1612e-05 s, 0.0 kB/s 2023-10-10T20:37:37.247 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:37.247 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1176 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:37.301 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:37.301 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:37.302 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0221e-05 s, 0.0 kB/s 2023-10-10T20:37:37.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:37.302 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1177 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:37.357 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:37.357 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:37.357 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7196e-05 s, 0.0 kB/s 2023-10-10T20:37:37.358 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:37.358 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1178 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:37.412 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:37.412 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:37.412 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9391e-05 s, 0.0 kB/s 2023-10-10T20:37:37.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:37.413 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1179 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:37.467 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:37.467 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:37.467 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1735e-05 s, 0.0 kB/s 2023-10-10T20:37:37.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:37.468 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1180 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:37.522 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:37.522 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:37.522 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0655e-05 s, 0.0 kB/s 2023-10-10T20:37:37.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:37.523 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1181 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:37.577 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:37.577 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:37.578 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.114e-05 s, 0.0 kB/s 2023-10-10T20:37:37.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:37.578 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1182 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:37.633 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:37.634 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:37.634 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.014e-05 s, 0.0 kB/s 2023-10-10T20:37:37.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:37.635 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1183 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:37.689 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:37.689 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:37.689 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8228e-05 s, 0.0 kB/s 2023-10-10T20:37:37.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:37.690 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1184 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:37.744 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:37.744 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:37.744 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8756e-05 s, 0.0 kB/s 2023-10-10T20:37:37.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:37.746 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1185 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:37.801 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:37.802 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:37.802 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3037e-05 s, 0.0 kB/s 2023-10-10T20:37:37.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:37.803 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1186 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:37.857 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:37.857 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:37.857 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7594e-05 s, 0.0 kB/s 2023-10-10T20:37:37.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:37.858 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1187 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:37.912 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:37.912 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:37.913 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1801e-05 s, 0.0 kB/s 2023-10-10T20:37:37.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:37.913 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1188 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:37.968 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:37.968 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:37.968 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1212e-05 s, 0.0 kB/s 2023-10-10T20:37:37.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:37.969 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1189 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:38.023 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:38.024 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:38.024 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9619e-05 s, 0.0 kB/s 2023-10-10T20:37:38.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:38.025 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1190 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:38.079 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:38.079 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:38.079 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8359e-05 s, 0.0 kB/s 2023-10-10T20:37:38.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:38.080 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1191 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:38.134 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:38.134 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:38.135 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9117e-05 s, 0.0 kB/s 2023-10-10T20:37:38.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:38.135 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1192 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:38.189 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:38.189 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:38.189 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4371e-05 s, 0.0 kB/s 2023-10-10T20:37:38.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:38.190 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1193 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:38.244 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:38.244 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:38.245 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0298e-05 s, 0.0 kB/s 2023-10-10T20:37:38.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:38.245 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1194 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:38.300 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:38.300 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:38.301 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9599e-05 s, 0.0 kB/s 2023-10-10T20:37:38.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:38.301 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1195 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:38.356 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:38.356 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:38.356 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0665e-05 s, 0.0 kB/s 2023-10-10T20:37:38.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:38.357 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1196 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.858e-05 s, 0.0 kB/s 2023-10-10T20:37:38.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:38.413 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1197 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:38.467 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:38.467 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:38.467 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7096e-05 s, 0.0 kB/s 2023-10-10T20:37:38.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:38.468 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1198 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:38.522 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:38.522 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:38.522 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.309e-05 s, 0.0 kB/s 2023-10-10T20:37:38.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:38.523 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1199 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:38.577 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:38.578 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:38.578 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1287e-05 s, 0.0 kB/s 2023-10-10T20:37:38.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:38.579 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1200 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:38.633 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:38.634 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:38.634 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0421e-05 s, 0.0 kB/s 2023-10-10T20:37:38.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:38.635 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1201 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:38.689 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:38.689 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0434e-05 s, 0.0 kB/s 2023-10-10T20:37:38.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:38.691 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1202 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:38.745 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:38.746 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:38.746 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9518e-05 s, 0.0 kB/s 2023-10-10T20:37:38.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:38.747 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1203 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:38.800 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:38.801 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:38.801 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8644e-05 s, 0.0 kB/s 2023-10-10T20:37:38.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:38.802 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1204 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:38.856 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:38.856 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:38.857 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.21e-05 s, 0.0 kB/s 2023-10-10T20:37:38.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:38.857 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1205 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:38.912 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:38.912 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:38.912 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.843e-05 s, 0.0 kB/s 2023-10-10T20:37:38.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:38.913 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1206 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:38.967 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:38.967 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:38.967 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0946e-05 s, 0.0 kB/s 2023-10-10T20:37:38.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:38.968 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1207 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:39.022 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:39.023 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:39.023 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9829e-05 s, 0.0 kB/s 2023-10-10T20:37:39.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:39.024 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1208 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:39.078 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:39.079 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:39.079 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8627e-05 s, 0.0 kB/s 2023-10-10T20:37:39.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:39.080 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1209 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:39.134 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:39.134 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:39.135 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9798e-05 s, 0.0 kB/s 2023-10-10T20:37:39.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:39.135 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1210 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:39.189 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:39.190 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:39.190 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6797e-05 s, 0.0 kB/s 2023-10-10T20:37:39.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:39.191 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1211 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:39.245 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:39.245 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9955e-05 s, 0.0 kB/s 2023-10-10T20:37:39.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:39.246 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1212 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:39.301 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:39.301 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:39.301 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1043e-05 s, 0.0 kB/s 2023-10-10T20:37:39.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:39.302 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1213 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:39.371 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:39.371 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:39.371 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.473e-05 s, 0.0 kB/s 2023-10-10T20:37:39.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:39.372 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1214 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8401e-05 s, 0.0 kB/s 2023-10-10T20:37:39.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:39.427 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1215 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:39.482 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:39.482 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:39.483 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0507e-05 s, 0.0 kB/s 2023-10-10T20:37:39.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:39.484 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1216 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:39.538 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:39.538 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:39.539 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0369e-05 s, 0.0 kB/s 2023-10-10T20:37:39.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:39.539 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1217 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:39.594 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:39.594 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:39.595 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7742e-05 s, 0.0 kB/s 2023-10-10T20:37:39.595 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:39.595 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1218 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:39.651 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:39.651 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:39.651 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.157e-05 s, 0.0 kB/s 2023-10-10T20:37:39.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:39.652 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1219 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:39.706 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:39.706 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:39.707 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8069e-05 s, 0.0 kB/s 2023-10-10T20:37:39.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:39.707 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1220 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:39.762 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:39.763 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:39.763 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8478e-05 s, 0.0 kB/s 2023-10-10T20:37:39.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:39.764 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1221 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:39.818 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2413e-05 s, 0.0 kB/s 2023-10-10T20:37:39.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:39.820 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1222 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:39.874 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:39.875 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:39.875 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0024e-05 s, 0.0 kB/s 2023-10-10T20:37:39.875 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:39.876 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1223 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:39.930 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:39.931 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:39.931 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.921e-05 s, 0.0 kB/s 2023-10-10T20:37:39.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:39.932 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1224 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:39.986 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:39.986 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:39.987 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2166e-05 s, 0.0 kB/s 2023-10-10T20:37:39.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:39.987 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1225 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:40.043 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:40.043 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:40.043 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8097e-05 s, 0.0 kB/s 2023-10-10T20:37:40.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:40.044 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1226 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:40.098 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:40.098 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:40.098 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.661e-05 s, 0.0 kB/s 2023-10-10T20:37:40.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:40.099 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1227 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6477e-05 s, 0.0 kB/s 2023-10-10T20:37:40.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:40.154 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1228 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:40.208 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:40.208 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:40.208 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0141e-05 s, 0.0 kB/s 2023-10-10T20:37:40.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:40.209 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1229 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:40.263 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:40.263 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:40.263 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9166e-05 s, 0.0 kB/s 2023-10-10T20:37:40.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:40.264 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1230 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:40.318 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:40.318 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:40.318 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9979e-05 s, 0.0 kB/s 2023-10-10T20:37:40.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:40.319 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1231 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:40.373 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:40.373 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:40.373 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.049e-05 s, 0.0 kB/s 2023-10-10T20:37:40.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:40.374 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1232 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:40.428 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2199e-05 s, 0.0 kB/s 2023-10-10T20:37:40.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:40.430 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1233 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:40.484 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:40.484 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:40.485 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2139e-05 s, 0.0 kB/s 2023-10-10T20:37:40.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:40.485 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1234 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:40.540 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:40.540 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:40.541 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8162e-05 s, 0.0 kB/s 2023-10-10T20:37:40.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:40.541 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1235 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:40.596 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:40.596 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:40.596 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1487e-05 s, 0.0 kB/s 2023-10-10T20:37:40.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:40.597 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1236 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7882e-05 s, 0.0 kB/s 2023-10-10T20:37:40.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:40.653 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1237 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:40.707 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:40.708 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:40.708 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8199e-05 s, 0.0 kB/s 2023-10-10T20:37:40.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:40.709 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1238 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:40.763 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:40.763 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:40.763 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0267e-05 s, 0.0 kB/s 2023-10-10T20:37:40.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:40.764 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1239 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:40.818 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:40.818 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.917e-05 s, 0.0 kB/s 2023-10-10T20:37:40.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:40.819 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1240 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:40.873 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:40.873 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:40.873 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.044e-05 s, 0.0 kB/s 2023-10-10T20:37:40.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:40.874 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1241 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:40.928 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:40.928 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:40.929 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9619e-05 s, 0.0 kB/s 2023-10-10T20:37:40.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:40.929 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1242 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:40.984 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:40.984 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:40.984 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1499e-05 s, 0.0 kB/s 2023-10-10T20:37:40.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:40.985 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1243 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:41.039 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:41.040 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:41.040 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0172e-05 s, 0.0 kB/s 2023-10-10T20:37:41.040 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:41.040 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1244 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:41.095 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:41.095 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:41.095 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7673e-05 s, 0.0 kB/s 2023-10-10T20:37:41.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:41.097 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1245 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:41.150 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:41.150 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:41.150 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8511e-05 s, 0.0 kB/s 2023-10-10T20:37:41.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:41.151 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1246 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7698e-05 s, 0.0 kB/s 2023-10-10T20:37:41.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:41.206 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1247 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:41.261 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:41.261 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:41.261 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8099e-05 s, 0.0 kB/s 2023-10-10T20:37:41.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:41.262 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1248 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:41.316 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:41.316 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:41.316 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.59e-05 s, 0.0 kB/s 2023-10-10T20:37:41.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:41.317 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1249 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:41.371 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:41.371 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:41.371 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4559e-05 s, 0.0 kB/s 2023-10-10T20:37:41.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:41.372 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1250 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:41.426 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:41.426 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:41.426 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0327e-05 s, 0.0 kB/s 2023-10-10T20:37:41.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:41.428 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1251 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:41.484 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:41.484 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:41.484 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9725e-05 s, 0.0 kB/s 2023-10-10T20:37:41.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:41.485 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1252 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:41.540 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:41.540 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2144e-05 s, 0.0 kB/s 2023-10-10T20:37:41.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:41.541 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1253 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:41.595 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:41.595 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:41.596 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.87e-05 s, 0.0 kB/s 2023-10-10T20:37:41.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:41.597 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1254 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:41.650 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:41.650 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:41.650 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0057e-05 s, 0.0 kB/s 2023-10-10T20:37:41.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:41.651 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1255 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8413e-05 s, 0.0 kB/s 2023-10-10T20:37:41.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:41.707 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1256 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:41.761 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:41.761 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:41.761 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7556e-05 s, 0.0 kB/s 2023-10-10T20:37:41.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:41.762 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1257 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:41.816 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:41.816 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:41.816 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2762e-05 s, 0.0 kB/s 2023-10-10T20:37:41.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:41.817 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1258 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:41.871 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:41.871 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:41.872 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1384e-05 s, 0.0 kB/s 2023-10-10T20:37:41.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:41.872 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1259 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:41.927 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:41.927 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:41.928 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1168e-05 s, 0.0 kB/s 2023-10-10T20:37:41.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:41.928 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1260 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:41.983 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:41.984 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:41.984 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9548e-05 s, 0.0 kB/s 2023-10-10T20:37:41.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:41.985 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1261 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:42.037 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:37:42.039 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:42.039 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:42.039 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9549e-05 s, 0.0 kB/s 2023-10-10T20:37:42.040 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:42.040 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1262 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:42.041 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:37:42.098 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:42.098 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:42.098 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9088e-05 s, 0.0 kB/s 2023-10-10T20:37:42.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:42.099 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1263 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:42.153 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:42.153 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:42.154 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9652e-05 s, 0.0 kB/s 2023-10-10T20:37:42.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:42.155 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1264 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:42.209 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:42.209 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:42.210 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0365e-05 s, 0.0 kB/s 2023-10-10T20:37:42.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:42.210 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1265 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:42.264 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:42.265 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:42.265 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9839e-05 s, 0.0 kB/s 2023-10-10T20:37:42.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:42.266 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1266 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:42.321 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:42.321 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:42.321 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8321e-05 s, 0.0 kB/s 2023-10-10T20:37:42.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:42.322 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1267 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:42.376 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:42.377 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:42.377 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0142e-05 s, 0.0 kB/s 2023-10-10T20:37:42.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:42.378 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1268 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:42.432 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:42.432 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:42.432 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8236e-05 s, 0.0 kB/s 2023-10-10T20:37:42.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:42.433 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1269 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:42.487 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:42.487 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9905e-05 s, 0.0 kB/s 2023-10-10T20:37:42.488 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:42.488 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1270 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:42.543 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:42.543 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:42.543 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9796e-05 s, 0.0 kB/s 2023-10-10T20:37:42.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:42.544 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1271 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:42.598 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:42.598 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:42.598 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.703e-05 s, 0.0 kB/s 2023-10-10T20:37:42.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:42.599 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1272 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:42.654 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:42.655 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:42.655 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0542e-05 s, 0.0 kB/s 2023-10-10T20:37:42.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:42.656 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1273 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:42.710 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:42.711 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:42.711 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9611e-05 s, 0.0 kB/s 2023-10-10T20:37:42.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:42.712 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1274 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:42.766 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:42.767 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:42.767 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0426e-05 s, 0.0 kB/s 2023-10-10T20:37:42.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:42.768 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1275 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:42.822 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:42.822 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:42.822 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9975e-05 s, 0.0 kB/s 2023-10-10T20:37:42.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:42.823 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1276 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:42.877 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:42.878 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:42.878 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7477e-05 s, 0.0 kB/s 2023-10-10T20:37:42.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:42.879 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1277 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:42.934 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:42.935 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:42.935 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9858e-05 s, 0.0 kB/s 2023-10-10T20:37:42.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:42.936 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1278 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:42.990 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:42.991 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:42.991 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9837e-05 s, 0.0 kB/s 2023-10-10T20:37:42.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:42.992 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1279 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:43.047 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:43.047 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:43.047 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1301e-05 s, 0.0 kB/s 2023-10-10T20:37:43.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:43.048 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1280 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:43.102 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:43.102 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:43.102 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.945e-05 s, 0.0 kB/s 2023-10-10T20:37:43.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:43.103 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1281 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:43.157 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:43.157 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:43.157 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.7319e-05 s, 0.0 kB/s 2023-10-10T20:37:43.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:43.158 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1282 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:43.213 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:43.213 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1259e-05 s, 0.0 kB/s 2023-10-10T20:37:43.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:43.214 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1283 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:43.268 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:43.268 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:43.269 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7791e-05 s, 0.0 kB/s 2023-10-10T20:37:43.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:43.269 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1284 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:43.327 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:43.327 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:43.328 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3313e-05 s, 0.0 kB/s 2023-10-10T20:37:43.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:43.328 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1285 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:43.383 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7767e-05 s, 0.0 kB/s 2023-10-10T20:37:43.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:43.386 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1286 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:43.439 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:43.440 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:43.440 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0889e-05 s, 0.0 kB/s 2023-10-10T20:37:43.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:43.441 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1287 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:43.495 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:43.495 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:43.495 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2856e-05 s, 0.0 kB/s 2023-10-10T20:37:43.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:43.496 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1288 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:43.550 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:43.550 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:43.551 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1891e-05 s, 0.0 kB/s 2023-10-10T20:37:43.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:43.552 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1289 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1829e-05 s, 0.0 kB/s 2023-10-10T20:37:43.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:43.608 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1290 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:43.662 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:43.662 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:43.662 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8991e-05 s, 0.0 kB/s 2023-10-10T20:37:43.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:43.663 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1291 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:43.718 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:43.718 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:43.718 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1616e-05 s, 0.0 kB/s 2023-10-10T20:37:43.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:43.719 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1292 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:43.773 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:43.773 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:43.773 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8475e-05 s, 0.0 kB/s 2023-10-10T20:37:43.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:43.774 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1293 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:43.828 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:43.828 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:43.829 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9688e-05 s, 0.0 kB/s 2023-10-10T20:37:43.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:43.829 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1294 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:43.884 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:43.884 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:43.884 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9483e-05 s, 0.0 kB/s 2023-10-10T20:37:43.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:43.885 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1295 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0446e-05 s, 0.0 kB/s 2023-10-10T20:37:43.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:43.941 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1296 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:43.995 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:43.995 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:43.995 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2966e-05 s, 0.0 kB/s 2023-10-10T20:37:43.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:43.996 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1297 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:44.050 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:44.051 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:44.051 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2468e-05 s, 0.0 kB/s 2023-10-10T20:37:44.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:44.052 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1298 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:44.106 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:44.106 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:44.106 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7882e-05 s, 0.0 kB/s 2023-10-10T20:37:44.107 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:44.107 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1299 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:44.161 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:44.162 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:44.162 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0167e-05 s, 0.0 kB/s 2023-10-10T20:37:44.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:44.163 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1300 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:44.217 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:44.218 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:44.218 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0568e-05 s, 0.0 kB/s 2023-10-10T20:37:44.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:44.219 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1301 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:44.273 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:44.273 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:44.273 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7511e-05 s, 0.0 kB/s 2023-10-10T20:37:44.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:44.274 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1302 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:44.329 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:44.329 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:44.329 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2574e-05 s, 0.0 kB/s 2023-10-10T20:37:44.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:44.330 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1303 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:44.384 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:44.385 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:44.385 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9279e-05 s, 0.0 kB/s 2023-10-10T20:37:44.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:44.386 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1304 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:44.440 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:44.440 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:44.441 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1086e-05 s, 0.0 kB/s 2023-10-10T20:37:44.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:44.441 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1305 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:44.499 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:44.499 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4313e-05 s, 0.0 kB/s 2023-10-10T20:37:44.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:44.500 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1306 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:44.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:44.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:44.555 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9251e-05 s, 0.0 kB/s 2023-10-10T20:37:44.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:44.555 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1307 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:44.610 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:44.610 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:44.610 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2326e-05 s, 0.0 kB/s 2023-10-10T20:37:44.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:44.611 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1308 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:44.666 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8854e-05 s, 0.0 kB/s 2023-10-10T20:37:44.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:44.666 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1309 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:44.722 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:44.722 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:44.722 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1459e-05 s, 0.0 kB/s 2023-10-10T20:37:44.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:44.723 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1310 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:44.777 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:44.778 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:44.778 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1331e-05 s, 0.0 kB/s 2023-10-10T20:37:44.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:44.779 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1311 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:44.832 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:44.833 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:44.833 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3006e-05 s, 0.0 kB/s 2023-10-10T20:37:44.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:44.834 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1312 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:44.888 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:44.889 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:44.889 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1352e-05 s, 0.0 kB/s 2023-10-10T20:37:44.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:44.890 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1313 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:44.944 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:44.945 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:44.945 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9839e-05 s, 0.0 kB/s 2023-10-10T20:37:44.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:44.946 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1314 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:45.000 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:45.001 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:45.001 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9299e-05 s, 0.0 kB/s 2023-10-10T20:37:45.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:45.002 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1315 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:45.056 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:45.057 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:45.057 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2044e-05 s, 0.0 kB/s 2023-10-10T20:37:45.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:45.058 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1316 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:45.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:45.113 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:45.113 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1508e-05 s, 0.0 kB/s 2023-10-10T20:37:45.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:45.114 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1317 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:45.168 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:45.169 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:45.169 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.247e-05 s, 0.0 kB/s 2023-10-10T20:37:45.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:45.170 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1318 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:45.224 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:45.224 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:45.224 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.712e-05 s, 0.0 kB/s 2023-10-10T20:37:45.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:45.225 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1319 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:45.279 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:45.280 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:45.280 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9172e-05 s, 0.0 kB/s 2023-10-10T20:37:45.280 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:45.280 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1320 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:45.335 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:45.335 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:45.335 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8484e-05 s, 0.0 kB/s 2023-10-10T20:37:45.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:45.336 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1321 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1016e-05 s, 0.0 kB/s 2023-10-10T20:37:45.391 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:45.391 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1322 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:45.445 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:45.445 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:45.446 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9597e-05 s, 0.0 kB/s 2023-10-10T20:37:45.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:45.446 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1323 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:45.501 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:45.501 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:45.501 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3574e-05 s, 0.0 kB/s 2023-10-10T20:37:45.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:45.502 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1324 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:45.556 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:45.556 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:45.557 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9522e-05 s, 0.0 kB/s 2023-10-10T20:37:45.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:45.557 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1325 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:45.612 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:45.612 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:45.612 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3998e-05 s, 0.0 kB/s 2023-10-10T20:37:45.613 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:45.613 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1326 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:45.667 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:45.667 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:45.667 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4086e-05 s, 0.0 kB/s 2023-10-10T20:37:45.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:45.668 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1327 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:45.722 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:45.723 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:45.723 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1014e-05 s, 0.0 kB/s 2023-10-10T20:37:45.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:45.724 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1328 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:45.778 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:45.779 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:45.779 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0727e-05 s, 0.0 kB/s 2023-10-10T20:37:45.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:45.780 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1329 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:45.834 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:45.835 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:45.835 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0951e-05 s, 0.0 kB/s 2023-10-10T20:37:45.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:45.836 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1330 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:45.890 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:45.890 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:45.891 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.036e-05 s, 0.0 kB/s 2023-10-10T20:37:45.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:45.891 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1331 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:45.946 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:45.946 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:45.946 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8653e-05 s, 0.0 kB/s 2023-10-10T20:37:45.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:45.947 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1332 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:46.001 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:46.001 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:46.001 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1223e-05 s, 0.0 kB/s 2023-10-10T20:37:46.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:46.002 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1333 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:46.056 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:46.057 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:46.057 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1026e-05 s, 0.0 kB/s 2023-10-10T20:37:46.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:46.057 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1334 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:46.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:46.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:46.113 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.7608e-05 s, 0.0 kB/s 2023-10-10T20:37:46.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:46.113 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1335 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:46.168 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:46.169 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:46.169 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0056e-05 s, 0.0 kB/s 2023-10-10T20:37:46.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:46.169 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1336 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:46.224 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:46.225 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:46.225 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2541e-05 s, 0.0 kB/s 2023-10-10T20:37:46.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:46.226 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1337 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:46.280 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:46.280 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:46.280 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6659e-05 s, 0.0 kB/s 2023-10-10T20:37:46.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:46.281 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1338 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:46.335 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:46.336 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:46.336 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0111e-05 s, 0.0 kB/s 2023-10-10T20:37:46.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:46.337 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1339 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:46.391 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:46.392 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:46.392 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.9394e-05 s, 0.0 kB/s 2023-10-10T20:37:46.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:46.393 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1340 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:46.448 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:46.448 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:46.448 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0769e-05 s, 0.0 kB/s 2023-10-10T20:37:46.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:46.449 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1341 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:46.503 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:46.503 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:46.503 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8235e-05 s, 0.0 kB/s 2023-10-10T20:37:46.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:46.504 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1342 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:46.558 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:46.558 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:46.559 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7906e-05 s, 0.0 kB/s 2023-10-10T20:37:46.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:46.560 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1343 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:46.614 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:46.614 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:46.615 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8311e-05 s, 0.0 kB/s 2023-10-10T20:37:46.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:46.616 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1344 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:46.670 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:46.670 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:46.670 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2394e-05 s, 0.0 kB/s 2023-10-10T20:37:46.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:46.671 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1345 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:46.725 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:46.725 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:46.726 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9757e-05 s, 0.0 kB/s 2023-10-10T20:37:46.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:46.726 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1346 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:46.781 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:46.781 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:46.782 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2522e-05 s, 0.0 kB/s 2023-10-10T20:37:46.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:46.782 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1347 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:46.837 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:46.837 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:46.838 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.159e-05 s, 0.0 kB/s 2023-10-10T20:37:46.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:46.838 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1348 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:46.893 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:46.893 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:46.893 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4662e-05 s, 0.0 kB/s 2023-10-10T20:37:46.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:46.894 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1349 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:46.948 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:46.949 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:46.949 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8784e-05 s, 0.0 kB/s 2023-10-10T20:37:46.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:46.950 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1350 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:47.006 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:47.006 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:47.007 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9086e-05 s, 0.0 kB/s 2023-10-10T20:37:47.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:47.007 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1351 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:47.061 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:47.062 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:47.062 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.928e-05 s, 0.0 kB/s 2023-10-10T20:37:47.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:47.063 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1352 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:47.118 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:47.118 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:47.118 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8846e-05 s, 0.0 kB/s 2023-10-10T20:37:47.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:47.120 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1353 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:47.173 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:47.173 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:47.174 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6469e-05 s, 0.0 kB/s 2023-10-10T20:37:47.174 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:47.174 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1354 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:47.228 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:47.228 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:47.229 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.652e-05 s, 0.0 kB/s 2023-10-10T20:37:47.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:47.229 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1355 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:47.284 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:47.285 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:47.285 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0387e-05 s, 0.0 kB/s 2023-10-10T20:37:47.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:47.286 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1356 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:47.340 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:47.340 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:47.340 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0082e-05 s, 0.0 kB/s 2023-10-10T20:37:47.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:47.341 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1357 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:47.395 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:47.396 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:47.396 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9578e-05 s, 0.0 kB/s 2023-10-10T20:37:47.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:47.397 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1358 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:47.451 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:47.451 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:47.451 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2199e-05 s, 0.0 kB/s 2023-10-10T20:37:47.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:47.452 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1359 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:47.506 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:47.507 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:47.507 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7107e-05 s, 0.0 kB/s 2023-10-10T20:37:47.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:47.508 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1360 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:47.562 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:47.563 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:47.563 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6908e-05 s, 0.0 kB/s 2023-10-10T20:37:47.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:47.564 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1361 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:47.618 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:47.618 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:47.618 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8718e-05 s, 0.0 kB/s 2023-10-10T20:37:47.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:47.619 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1362 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:47.673 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:47.674 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:47.674 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.063e-05 s, 0.0 kB/s 2023-10-10T20:37:47.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:47.675 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1363 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:47.729 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:47.730 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:47.730 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9549e-05 s, 0.0 kB/s 2023-10-10T20:37:47.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:47.731 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1364 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:47.785 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:47.785 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:47.785 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1605e-05 s, 0.0 kB/s 2023-10-10T20:37:47.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:47.786 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1365 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8123e-05 s, 0.0 kB/s 2023-10-10T20:37:47.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:47.841 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1366 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:47.895 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:47.895 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:47.895 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2025e-05 s, 0.0 kB/s 2023-10-10T20:37:47.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:47.897 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1367 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:47.952 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:47.953 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:47.953 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9264e-05 s, 0.0 kB/s 2023-10-10T20:37:47.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:47.954 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1368 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:48.008 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0073e-05 s, 0.0 kB/s 2023-10-10T20:37:48.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:48.010 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1369 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:48.064 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:48.065 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:48.065 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.923e-05 s, 0.0 kB/s 2023-10-10T20:37:48.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:48.066 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1370 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:48.120 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:48.120 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:48.121 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9e-05 s, 0.0 kB/s 2023-10-10T20:37:48.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:48.121 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1371 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:48.175 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9291e-05 s, 0.0 kB/s 2023-10-10T20:37:48.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:48.177 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1372 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:48.231 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:48.231 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:48.231 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9427e-05 s, 0.0 kB/s 2023-10-10T20:37:48.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:48.232 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1373 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:48.286 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:48.287 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:48.287 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9272e-05 s, 0.0 kB/s 2023-10-10T20:37:48.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:48.288 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1374 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:48.342 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:48.342 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:48.343 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8533e-05 s, 0.0 kB/s 2023-10-10T20:37:48.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:48.343 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1375 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:48.397 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:48.398 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:48.398 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8733e-05 s, 0.0 kB/s 2023-10-10T20:37:48.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:48.399 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1376 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:48.453 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:48.454 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:48.454 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8746e-05 s, 0.0 kB/s 2023-10-10T20:37:48.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:48.455 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1377 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:48.509 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:48.509 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:48.510 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0799e-05 s, 0.0 kB/s 2023-10-10T20:37:48.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:48.510 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1378 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1125e-05 s, 0.0 kB/s 2023-10-10T20:37:48.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:48.567 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1379 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:48.621 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:48.621 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:48.622 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9896e-05 s, 0.0 kB/s 2023-10-10T20:37:48.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:48.622 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1380 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:48.676 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:48.676 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:48.677 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9717e-05 s, 0.0 kB/s 2023-10-10T20:37:48.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:48.677 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1381 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:48.733 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:48.733 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:48.733 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7084e-05 s, 0.0 kB/s 2023-10-10T20:37:48.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:48.734 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1382 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:48.788 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:48.788 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:48.788 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.892e-05 s, 0.0 kB/s 2023-10-10T20:37:48.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:48.789 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1383 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:48.843 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:48.844 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:48.844 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7267e-05 s, 0.0 kB/s 2023-10-10T20:37:48.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:48.845 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1384 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:48.899 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:48.899 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:48.899 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8731e-05 s, 0.0 kB/s 2023-10-10T20:37:48.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:48.900 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1385 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:48.954 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:48.954 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:48.954 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0173e-05 s, 0.0 kB/s 2023-10-10T20:37:48.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:48.955 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1386 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:49.009 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:49.010 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:49.010 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0674e-05 s, 0.0 kB/s 2023-10-10T20:37:49.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:49.011 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1387 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:49.066 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:49.067 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:49.067 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0572e-05 s, 0.0 kB/s 2023-10-10T20:37:49.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:49.068 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1388 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:49.122 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:49.122 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:49.122 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8186e-05 s, 0.0 kB/s 2023-10-10T20:37:49.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:49.123 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1389 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:49.177 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:49.177 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:49.178 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6669e-05 s, 0.0 kB/s 2023-10-10T20:37:49.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:49.178 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1390 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:49.233 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:49.234 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:49.234 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0566e-05 s, 0.0 kB/s 2023-10-10T20:37:49.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:49.235 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1391 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:49.289 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:49.290 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:49.290 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7942e-05 s, 0.0 kB/s 2023-10-10T20:37:49.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:49.291 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1392 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:49.346 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:49.346 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9444e-05 s, 0.0 kB/s 2023-10-10T20:37:49.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:49.348 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1393 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:49.401 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:49.401 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:49.402 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1245e-05 s, 0.0 kB/s 2023-10-10T20:37:49.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:49.403 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1394 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:49.457 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:49.457 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:49.458 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6828e-05 s, 0.0 kB/s 2023-10-10T20:37:49.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:49.458 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1395 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:49.513 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:49.514 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:49.514 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.361e-05 s, 0.0 kB/s 2023-10-10T20:37:49.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:49.515 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1396 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:49.569 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:49.570 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:49.570 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1068e-05 s, 0.0 kB/s 2023-10-10T20:37:49.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:49.571 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1397 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:49.625 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:49.626 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:49.626 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1002e-05 s, 0.0 kB/s 2023-10-10T20:37:49.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:49.627 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1398 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:49.681 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:49.681 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:49.682 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7463e-05 s, 0.0 kB/s 2023-10-10T20:37:49.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:49.682 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1399 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:49.737 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:49.738 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:49.738 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9781e-05 s, 0.0 kB/s 2023-10-10T20:37:49.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:49.739 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1400 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:49.793 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:49.793 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:49.793 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8285e-05 s, 0.0 kB/s 2023-10-10T20:37:49.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:49.794 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1401 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:49.848 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:49.849 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:49.849 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8502e-05 s, 0.0 kB/s 2023-10-10T20:37:49.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:49.850 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1402 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:49.904 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:49.904 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:49.905 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7749e-05 s, 0.0 kB/s 2023-10-10T20:37:49.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:49.905 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1403 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:49.960 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:49.960 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:49.961 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8006e-05 s, 0.0 kB/s 2023-10-10T20:37:49.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:49.961 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1404 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:50.015 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:50.016 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:50.016 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.867e-05 s, 0.0 kB/s 2023-10-10T20:37:50.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:50.017 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1405 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:50.072 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:50.072 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:50.072 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0445e-05 s, 0.0 kB/s 2023-10-10T20:37:50.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:50.073 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1406 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:50.127 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:50.128 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:50.128 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7658e-05 s, 0.0 kB/s 2023-10-10T20:37:50.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:50.129 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1407 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:50.184 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:50.184 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:50.184 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6511e-05 s, 0.0 kB/s 2023-10-10T20:37:50.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:50.185 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1408 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:50.239 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:50.239 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:50.240 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.151e-05 s, 0.0 kB/s 2023-10-10T20:37:50.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:50.240 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1409 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:50.295 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:50.295 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:50.296 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9151e-05 s, 0.0 kB/s 2023-10-10T20:37:50.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:50.296 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1410 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:50.350 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:50.350 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:50.350 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7164e-05 s, 0.0 kB/s 2023-10-10T20:37:50.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:50.351 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1411 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:50.405 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:50.405 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:50.405 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9021e-05 s, 0.0 kB/s 2023-10-10T20:37:50.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:50.406 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1412 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:50.460 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:50.460 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:50.461 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7414e-05 s, 0.0 kB/s 2023-10-10T20:37:50.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:50.461 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1413 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:50.516 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:50.516 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:50.517 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9704e-05 s, 0.0 kB/s 2023-10-10T20:37:50.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:50.517 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1414 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:50.572 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:50.572 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:50.573 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9725e-05 s, 0.0 kB/s 2023-10-10T20:37:50.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:50.573 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1415 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:50.641 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:50.642 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:50.642 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6744e-05 s, 0.0 kB/s 2023-10-10T20:37:50.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:50.643 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1416 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:50.697 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:50.698 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:50.698 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3249e-05 s, 0.0 kB/s 2023-10-10T20:37:50.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:50.699 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1417 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:50.754 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:50.754 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:50.754 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2008e-05 s, 0.0 kB/s 2023-10-10T20:37:50.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:50.755 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1418 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:50.809 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:50.809 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:50.810 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0956e-05 s, 0.0 kB/s 2023-10-10T20:37:50.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:50.810 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1419 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:50.864 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:50.864 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:50.865 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0955e-05 s, 0.0 kB/s 2023-10-10T20:37:50.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:50.865 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1420 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7935e-05 s, 0.0 kB/s 2023-10-10T20:37:50.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:50.920 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1421 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:50.974 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:50.974 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:50.974 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6978e-05 s, 0.0 kB/s 2023-10-10T20:37:50.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:50.975 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1422 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8404e-05 s, 0.0 kB/s 2023-10-10T20:37:51.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:51.030 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1423 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:51.084 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:51.085 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:51.085 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6885e-05 s, 0.0 kB/s 2023-10-10T20:37:51.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:51.086 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1424 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:51.140 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:51.140 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:51.140 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0587e-05 s, 0.0 kB/s 2023-10-10T20:37:51.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:51.141 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1425 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:51.195 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:51.195 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:51.195 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2461e-05 s, 0.0 kB/s 2023-10-10T20:37:51.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:51.196 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1426 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:51.251 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:51.251 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:51.251 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1262e-05 s, 0.0 kB/s 2023-10-10T20:37:51.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:51.252 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1427 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:51.308 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:51.309 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:51.309 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9729e-05 s, 0.0 kB/s 2023-10-10T20:37:51.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:51.309 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1428 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:51.364 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:51.364 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:51.364 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5898e-05 s, 0.0 kB/s 2023-10-10T20:37:51.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:51.365 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1429 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:51.419 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:51.419 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:51.419 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2023e-05 s, 0.0 kB/s 2023-10-10T20:37:51.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:51.421 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1430 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:51.475 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:51.475 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:51.475 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1801e-05 s, 0.0 kB/s 2023-10-10T20:37:51.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:51.476 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1431 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:51.531 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:51.531 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:51.531 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9401e-05 s, 0.0 kB/s 2023-10-10T20:37:51.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:51.532 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1432 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:51.586 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:51.587 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:51.587 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1439e-05 s, 0.0 kB/s 2023-10-10T20:37:51.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:51.588 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1433 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:51.643 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:51.643 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:51.643 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2708e-05 s, 0.0 kB/s 2023-10-10T20:37:51.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:51.644 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1434 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:51.702 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:51.702 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:51.702 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0578e-05 s, 0.0 kB/s 2023-10-10T20:37:51.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:51.703 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1435 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:51.758 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:51.759 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:51.759 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9023e-05 s, 0.0 kB/s 2023-10-10T20:37:51.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:51.760 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1436 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:51.814 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:51.815 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:51.815 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9679e-05 s, 0.0 kB/s 2023-10-10T20:37:51.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:51.816 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1437 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:51.870 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:51.870 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:51.871 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.902e-05 s, 0.0 kB/s 2023-10-10T20:37:51.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:51.871 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1438 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9292e-05 s, 0.0 kB/s 2023-10-10T20:37:51.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:51.928 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1439 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:51.982 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:51.983 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:51.983 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0924e-05 s, 0.0 kB/s 2023-10-10T20:37:51.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:51.984 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1440 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:52.038 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0834e-05 s, 0.0 kB/s 2023-10-10T20:37:52.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:52.040 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1441 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:52.094 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:52.095 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:52.095 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9729e-05 s, 0.0 kB/s 2023-10-10T20:37:52.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:52.095 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1442 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:52.150 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:52.151 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:52.151 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9091e-05 s, 0.0 kB/s 2023-10-10T20:37:52.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:52.151 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1443 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:52.206 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:52.206 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:52.207 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2933e-05 s, 0.0 kB/s 2023-10-10T20:37:52.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:52.207 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1444 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:52.262 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:52.262 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:52.263 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0636e-05 s, 0.0 kB/s 2023-10-10T20:37:52.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:52.263 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1445 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:52.318 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:52.319 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:52.319 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2237e-05 s, 0.0 kB/s 2023-10-10T20:37:52.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:52.320 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1446 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:52.374 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:52.374 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:52.375 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9558e-05 s, 0.0 kB/s 2023-10-10T20:37:52.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:52.375 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1447 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:52.430 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:52.430 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:52.431 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2284e-05 s, 0.0 kB/s 2023-10-10T20:37:52.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:52.431 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1448 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:52.487 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:52.487 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:52.487 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0464e-05 s, 0.0 kB/s 2023-10-10T20:37:52.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:52.488 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1449 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:52.542 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:52.542 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:52.543 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2907e-05 s, 0.0 kB/s 2023-10-10T20:37:52.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:52.544 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1450 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:52.597 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:52.598 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:52.598 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8591e-05 s, 0.0 kB/s 2023-10-10T20:37:52.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:52.599 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1451 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:52.653 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:52.654 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:52.654 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0132e-05 s, 0.0 kB/s 2023-10-10T20:37:52.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:52.655 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1452 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:52.727 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:52.727 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:52.727 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.8217e-05 s, 0.0 kB/s 2023-10-10T20:37:52.728 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:52.728 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1453 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:52.783 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:52.783 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:52.783 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3372e-05 s, 0.0 kB/s 2023-10-10T20:37:52.784 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:52.784 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1454 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:52.838 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:52.838 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:52.838 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0668e-05 s, 0.0 kB/s 2023-10-10T20:37:52.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:52.839 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1455 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:52.892 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:52.893 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:52.893 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8487e-05 s, 0.0 kB/s 2023-10-10T20:37:52.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:52.894 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1456 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:52.949 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:52.949 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:52.949 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9999e-05 s, 0.0 kB/s 2023-10-10T20:37:52.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:52.950 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1457 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:53.004 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:53.005 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:53.005 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8828e-05 s, 0.0 kB/s 2023-10-10T20:37:53.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:53.007 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1458 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:53.060 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:53.061 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:53.061 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2717e-05 s, 0.0 kB/s 2023-10-10T20:37:53.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:53.062 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1459 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:53.116 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:53.117 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:53.117 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9028e-05 s, 0.0 kB/s 2023-10-10T20:37:53.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:53.118 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1460 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:53.172 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:53.172 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:53.172 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4267e-05 s, 0.0 kB/s 2023-10-10T20:37:53.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:53.173 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1461 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:53.227 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:53.227 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:53.228 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0572e-05 s, 0.0 kB/s 2023-10-10T20:37:53.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:53.228 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1462 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:53.284 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:53.284 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:53.284 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9621e-05 s, 0.0 kB/s 2023-10-10T20:37:53.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:53.285 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1463 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:53.339 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:53.339 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:53.340 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0151e-05 s, 0.0 kB/s 2023-10-10T20:37:53.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:53.340 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1464 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:53.395 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:53.395 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:53.396 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9349e-05 s, 0.0 kB/s 2023-10-10T20:37:53.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:53.396 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1465 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:53.451 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:53.451 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:53.451 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0482e-05 s, 0.0 kB/s 2023-10-10T20:37:53.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:53.452 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1466 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:53.506 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:53.506 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:53.506 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8339e-05 s, 0.0 kB/s 2023-10-10T20:37:53.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:53.507 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1467 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:53.561 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:53.562 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:53.562 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0881e-05 s, 0.0 kB/s 2023-10-10T20:37:53.562 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:53.563 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1468 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:53.617 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:53.618 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:53.618 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7957e-05 s, 0.0 kB/s 2023-10-10T20:37:53.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:53.619 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1469 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:53.673 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:53.674 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:53.674 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8121e-05 s, 0.0 kB/s 2023-10-10T20:37:53.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:53.675 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1470 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:53.729 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:53.729 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:53.730 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2611e-05 s, 0.0 kB/s 2023-10-10T20:37:53.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:53.730 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1471 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:53.785 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:53.785 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:53.786 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9081e-05 s, 0.0 kB/s 2023-10-10T20:37:53.786 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:53.786 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1472 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:53.841 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:53.841 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:53.841 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1237e-05 s, 0.0 kB/s 2023-10-10T20:37:53.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:53.842 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1473 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:53.896 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:53.896 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:53.897 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0657e-05 s, 0.0 kB/s 2023-10-10T20:37:53.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:53.897 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1474 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:53.952 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:53.952 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:53.952 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0046e-05 s, 0.0 kB/s 2023-10-10T20:37:53.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:53.953 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1475 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:54.007 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:54.007 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:54.007 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4667e-05 s, 0.0 kB/s 2023-10-10T20:37:54.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:54.008 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1476 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:54.064 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:54.064 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:54.064 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7936e-05 s, 0.0 kB/s 2023-10-10T20:37:54.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:54.065 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1477 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:54.119 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:54.120 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:54.120 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0882e-05 s, 0.0 kB/s 2023-10-10T20:37:54.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:54.121 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1478 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:54.175 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:54.176 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:54.176 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.263e-05 s, 0.0 kB/s 2023-10-10T20:37:54.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:54.177 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1479 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:54.231 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:54.231 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:54.231 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.015e-05 s, 0.0 kB/s 2023-10-10T20:37:54.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:54.232 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1480 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:54.286 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:54.286 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:54.287 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0126e-05 s, 0.0 kB/s 2023-10-10T20:37:54.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:54.287 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1481 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:54.342 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:54.342 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:54.342 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9226e-05 s, 0.0 kB/s 2023-10-10T20:37:54.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:54.343 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1482 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:54.397 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:54.398 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:54.398 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.9344e-05 s, 0.0 kB/s 2023-10-10T20:37:54.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:54.399 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1483 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:54.453 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:54.453 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:54.454 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.1849e-05 s, 0.0 kB/s 2023-10-10T20:37:54.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:54.455 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1484 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:54.509 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:54.509 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:54.509 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5513e-05 s, 0.0 kB/s 2023-10-10T20:37:54.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:54.510 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1485 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:54.564 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:54.564 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:54.564 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8107e-05 s, 0.0 kB/s 2023-10-10T20:37:54.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:54.565 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1486 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:54.618 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:54.619 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:54.619 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0592e-05 s, 0.0 kB/s 2023-10-10T20:37:54.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:54.620 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1487 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:54.674 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:54.675 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:54.675 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7461e-05 s, 0.0 kB/s 2023-10-10T20:37:54.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:54.676 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1488 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:54.730 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:54.731 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:54.731 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9734e-05 s, 0.0 kB/s 2023-10-10T20:37:54.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:54.732 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1489 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:54.786 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:54.786 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:54.787 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1351e-05 s, 0.0 kB/s 2023-10-10T20:37:54.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:54.787 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1490 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:54.842 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:54.842 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:54.843 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0157e-05 s, 0.0 kB/s 2023-10-10T20:37:54.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:54.843 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1491 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:54.899 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:54.899 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:54.900 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.808e-05 s, 0.0 kB/s 2023-10-10T20:37:54.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:54.900 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1492 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:55.259 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:55.260 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7592e-05 s, 0.0 kB/s 2023-10-10T20:37:55.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:55.261 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1493 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:55.278 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2563e-05 s, 0.0 kB/s 2023-10-10T20:37:55.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:55.278 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1494 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:55.334 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:55.334 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:55.334 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1444e-05 s, 0.0 kB/s 2023-10-10T20:37:55.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:55.335 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1495 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:55.391 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:55.391 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:55.392 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1359e-05 s, 0.0 kB/s 2023-10-10T20:37:55.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:55.392 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1496 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:55.448 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:55.448 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:55.448 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7326e-05 s, 0.0 kB/s 2023-10-10T20:37:55.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:55.450 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1497 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:55.505 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:55.505 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:55.506 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9868e-05 s, 0.0 kB/s 2023-10-10T20:37:55.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:55.507 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1498 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:55.564 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:55.565 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:55.565 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8449e-05 s, 0.0 kB/s 2023-10-10T20:37:55.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:55.566 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1499 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9119e-05 s, 0.0 kB/s 2023-10-10T20:37:55.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:55.625 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1500 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:55.681 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:55.681 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:55.682 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6915e-05 s, 0.0 kB/s 2023-10-10T20:37:55.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:55.683 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1501 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:55.738 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:55.739 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:55.739 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.914e-05 s, 0.0 kB/s 2023-10-10T20:37:55.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:55.740 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1502 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:55.794 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:55.794 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0851e-05 s, 0.0 kB/s 2023-10-10T20:37:55.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:55.795 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1503 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:55.850 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:55.851 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:55.851 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5744e-05 s, 0.0 kB/s 2023-10-10T20:37:55.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:55.852 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1504 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:55.906 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:55.906 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:55.906 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4045e-05 s, 0.0 kB/s 2023-10-10T20:37:55.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:55.907 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1505 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:55.961 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:55.961 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:55.961 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6973e-05 s, 0.0 kB/s 2023-10-10T20:37:55.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:55.962 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1506 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:56.016 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:56.017 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:56.017 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9621e-05 s, 0.0 kB/s 2023-10-10T20:37:56.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:56.018 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1507 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:56.072 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:56.073 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:56.073 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0703e-05 s, 0.0 kB/s 2023-10-10T20:37:56.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:56.074 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1508 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:56.128 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:56.128 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:56.129 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8628e-05 s, 0.0 kB/s 2023-10-10T20:37:56.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:56.129 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1509 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:56.184 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:56.184 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:56.184 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7928e-05 s, 0.0 kB/s 2023-10-10T20:37:56.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:56.185 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1510 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:56.291 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:56.292 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:56.292 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 6.0078e-05 s, 0.0 kB/s 2023-10-10T20:37:56.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:56.293 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1511 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:56.307 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:56.308 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:56.308 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.378e-05 s, 0.0 kB/s 2023-10-10T20:37:56.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:56.309 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1512 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2629e-05 s, 0.0 kB/s 2023-10-10T20:37:56.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:56.364 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1513 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:56.418 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:56.419 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:56.419 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9539e-05 s, 0.0 kB/s 2023-10-10T20:37:56.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:56.420 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1514 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:56.474 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:56.475 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:56.475 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0188e-05 s, 0.0 kB/s 2023-10-10T20:37:56.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:56.476 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1515 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:56.531 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:56.531 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:56.532 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8979e-05 s, 0.0 kB/s 2023-10-10T20:37:56.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:56.532 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1516 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:56.587 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5753e-05 s, 0.0 kB/s 2023-10-10T20:37:56.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:56.588 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1517 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:56.641 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:56.641 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:56.641 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9999e-05 s, 0.0 kB/s 2023-10-10T20:37:56.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:56.642 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1518 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:56.696 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:56.697 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:56.697 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.962e-05 s, 0.0 kB/s 2023-10-10T20:37:56.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:56.698 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1519 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:56.753 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:56.753 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:56.753 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9158e-05 s, 0.0 kB/s 2023-10-10T20:37:56.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:56.754 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1520 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:56.809 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:56.809 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:56.809 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8466e-05 s, 0.0 kB/s 2023-10-10T20:37:56.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:56.810 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1521 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:56.864 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:56.864 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:56.864 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9198e-05 s, 0.0 kB/s 2023-10-10T20:37:56.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:56.865 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1522 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:56.919 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:56.919 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:56.920 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1193e-05 s, 0.0 kB/s 2023-10-10T20:37:56.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:56.920 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1523 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:56.976 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:56.976 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1541e-05 s, 0.0 kB/s 2023-10-10T20:37:56.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:56.976 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1524 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:57.031 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:57.031 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:57.031 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9454e-05 s, 0.0 kB/s 2023-10-10T20:37:57.032 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:57.032 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1525 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:57.086 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:57.087 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:57.087 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9379e-05 s, 0.0 kB/s 2023-10-10T20:37:57.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:57.087 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1526 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:57.142 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:57.142 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:57.143 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8613e-05 s, 0.0 kB/s 2023-10-10T20:37:57.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:57.144 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1527 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:57.198 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:57.198 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:57.199 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9373e-05 s, 0.0 kB/s 2023-10-10T20:37:57.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:57.199 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1528 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:57.254 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:57.254 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:57.255 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0869e-05 s, 0.0 kB/s 2023-10-10T20:37:57.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:57.256 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1529 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:57.310 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:57.310 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7287e-05 s, 0.0 kB/s 2023-10-10T20:37:57.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:57.311 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1530 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:57.366 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:57.366 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:57.366 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9382e-05 s, 0.0 kB/s 2023-10-10T20:37:57.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:57.367 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1531 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:57.420 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:57.421 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:57.421 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7719e-05 s, 0.0 kB/s 2023-10-10T20:37:57.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:57.422 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1532 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:57.476 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:57.476 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:57.476 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.7376e-05 s, 0.0 kB/s 2023-10-10T20:37:57.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:57.477 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1533 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:57.530 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:57.531 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:57.531 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9979e-05 s, 0.0 kB/s 2023-10-10T20:37:57.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:57.532 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1534 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:57.587 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:57.587 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:57.588 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8688e-05 s, 0.0 kB/s 2023-10-10T20:37:57.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:57.588 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1535 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:57.643 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:57.643 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:57.643 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3167e-05 s, 0.0 kB/s 2023-10-10T20:37:57.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:57.644 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1536 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0135e-05 s, 0.0 kB/s 2023-10-10T20:37:57.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:57.699 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1537 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:57.753 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:57.753 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:57.753 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1281e-05 s, 0.0 kB/s 2023-10-10T20:37:57.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:57.754 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1538 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:57.808 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:57.808 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:57.809 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9764e-05 s, 0.0 kB/s 2023-10-10T20:37:57.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:57.809 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1539 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:57.863 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:57.864 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:57.864 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1366e-05 s, 0.0 kB/s 2023-10-10T20:37:57.864 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:57.865 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1540 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:57.920 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.908e-05 s, 0.0 kB/s 2023-10-10T20:37:57.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:57.920 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1541 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7483e-05 s, 0.0 kB/s 2023-10-10T20:37:57.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:57.976 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1542 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:58.031 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:58.032 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:58.032 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7953e-05 s, 0.0 kB/s 2023-10-10T20:37:58.032 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:58.033 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1543 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:58.087 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:58.088 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:58.088 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3185e-05 s, 0.0 kB/s 2023-10-10T20:37:58.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:58.089 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1544 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:58.143 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:58.143 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:58.143 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0082e-05 s, 0.0 kB/s 2023-10-10T20:37:58.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:58.145 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1545 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:58.200 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:58.200 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:58.200 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1597e-05 s, 0.0 kB/s 2023-10-10T20:37:58.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:58.201 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1546 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:58.255 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:58.255 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:58.255 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8562e-05 s, 0.0 kB/s 2023-10-10T20:37:58.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:58.256 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1547 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:58.310 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:58.310 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:58.310 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6429e-05 s, 0.0 kB/s 2023-10-10T20:37:58.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:58.311 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1548 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:58.365 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:58.365 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:58.366 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0246e-05 s, 0.0 kB/s 2023-10-10T20:37:58.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:58.366 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1549 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:58.420 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:58.420 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:58.420 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2302e-05 s, 0.0 kB/s 2023-10-10T20:37:58.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:58.421 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1550 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:58.475 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:58.475 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:58.475 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3194e-05 s, 0.0 kB/s 2023-10-10T20:37:58.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:58.476 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1551 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:58.530 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:58.531 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:58.531 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6733e-05 s, 0.0 kB/s 2023-10-10T20:37:58.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:58.531 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1552 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:58.586 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:58.586 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:58.587 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9969e-05 s, 0.0 kB/s 2023-10-10T20:37:58.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:58.587 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1553 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:58.641 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:58.641 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:58.641 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0035e-05 s, 0.0 kB/s 2023-10-10T20:37:58.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:58.642 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1554 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:58.696 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:58.696 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:58.697 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0108e-05 s, 0.0 kB/s 2023-10-10T20:37:58.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:58.697 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1555 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:58.752 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:58.752 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:58.752 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3052e-05 s, 0.0 kB/s 2023-10-10T20:37:58.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:58.753 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1556 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:58.807 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:58.807 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:58.807 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1522e-05 s, 0.0 kB/s 2023-10-10T20:37:58.808 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:58.808 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1557 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:58.862 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:58.863 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:58.863 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8723e-05 s, 0.0 kB/s 2023-10-10T20:37:58.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:58.864 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1558 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:58.918 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:58.919 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:58.919 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1859e-05 s, 0.0 kB/s 2023-10-10T20:37:58.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:58.920 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1559 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:58.974 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:58.974 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2196e-05 s, 0.0 kB/s 2023-10-10T20:37:58.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:58.975 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1560 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:59.029 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:59.029 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:59.030 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.896e-05 s, 0.0 kB/s 2023-10-10T20:37:59.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:59.031 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1561 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:59.085 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:59.086 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:59.086 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.373e-05 s, 0.0 kB/s 2023-10-10T20:37:59.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:59.087 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1562 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:59.141 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:59.141 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:59.142 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9207e-05 s, 0.0 kB/s 2023-10-10T20:37:59.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:59.142 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1563 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:59.196 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:59.197 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:59.197 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0898e-05 s, 0.0 kB/s 2023-10-10T20:37:59.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:59.198 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1564 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:59.254 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:59.254 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:59.254 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1604e-05 s, 0.0 kB/s 2023-10-10T20:37:59.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:59.255 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1565 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:59.309 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:59.309 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:59.309 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1089e-05 s, 0.0 kB/s 2023-10-10T20:37:59.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:59.310 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1566 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:59.364 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:59.364 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:59.365 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7725e-05 s, 0.0 kB/s 2023-10-10T20:37:59.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:59.365 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1567 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:59.419 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:59.420 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:59.420 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1512e-05 s, 0.0 kB/s 2023-10-10T20:37:59.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:59.421 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1568 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:59.475 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:59.476 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:59.476 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1193e-05 s, 0.0 kB/s 2023-10-10T20:37:59.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:59.477 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1569 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:59.531 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:59.531 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:59.532 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7813e-05 s, 0.0 kB/s 2023-10-10T20:37:59.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:59.533 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1570 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:59.587 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:59.588 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:59.588 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9876e-05 s, 0.0 kB/s 2023-10-10T20:37:59.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:59.589 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1571 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:59.643 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:59.643 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:59.643 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2758e-05 s, 0.0 kB/s 2023-10-10T20:37:59.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:59.644 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1572 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:59.698 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:59.698 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:59.699 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8127e-05 s, 0.0 kB/s 2023-10-10T20:37:59.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:59.699 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1573 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:59.753 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:59.753 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:59.753 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8684e-05 s, 0.0 kB/s 2023-10-10T20:37:59.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:59.754 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1574 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:59.809 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8668e-05 s, 0.0 kB/s 2023-10-10T20:37:59.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:59.810 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1575 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:59.863 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:59.863 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:59.864 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8627e-05 s, 0.0 kB/s 2023-10-10T20:37:59.864 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:59.864 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1576 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:59.919 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:59.920 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:59.920 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0863e-05 s, 0.0 kB/s 2023-10-10T20:37:59.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:59.920 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1577 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:37:59.975 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:37:59.975 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:37:59.975 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9519e-05 s, 0.0 kB/s 2023-10-10T20:37:59.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:37:59.976 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1578 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:00.030 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:00.031 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:00.031 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5636e-05 s, 0.0 kB/s 2023-10-10T20:38:00.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:00.032 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1579 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:00.087 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:00.087 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:00.087 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9634e-05 s, 0.0 kB/s 2023-10-10T20:38:00.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:00.088 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1580 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:00.142 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:00.142 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:00.142 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9168e-05 s, 0.0 kB/s 2023-10-10T20:38:00.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:00.143 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1581 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:00.197 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:00.198 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:00.198 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6039e-05 s, 0.0 kB/s 2023-10-10T20:38:00.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:00.199 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1582 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:00.254 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9735e-05 s, 0.0 kB/s 2023-10-10T20:38:00.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:00.255 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1583 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:00.309 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:00.309 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:00.309 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2533e-05 s, 0.0 kB/s 2023-10-10T20:38:00.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:00.310 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1584 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:00.364 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:00.364 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:00.364 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.306e-05 s, 0.0 kB/s 2023-10-10T20:38:00.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:00.365 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1585 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:00.419 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:00.420 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:00.420 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.2076e-05 s, 0.0 kB/s 2023-10-10T20:38:00.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:00.421 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1586 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:00.477 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:00.478 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:00.478 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1094e-05 s, 0.0 kB/s 2023-10-10T20:38:00.478 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:00.479 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1587 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.673e-05 s, 0.0 kB/s 2023-10-10T20:38:00.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:00.534 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1588 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:00.589 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:00.589 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:00.589 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9898e-05 s, 0.0 kB/s 2023-10-10T20:38:00.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:00.590 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1589 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:00.644 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:00.644 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:00.645 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.801e-05 s, 0.0 kB/s 2023-10-10T20:38:00.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:00.645 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1590 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:00.700 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:00.701 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:00.701 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1495e-05 s, 0.0 kB/s 2023-10-10T20:38:00.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:00.702 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1591 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:00.756 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:00.756 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:00.757 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9091e-05 s, 0.0 kB/s 2023-10-10T20:38:00.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:00.757 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1592 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0285e-05 s, 0.0 kB/s 2023-10-10T20:38:00.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:00.813 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1593 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:00.867 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:00.867 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:00.867 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1472e-05 s, 0.0 kB/s 2023-10-10T20:38:00.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:00.868 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1594 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:00.922 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:00.923 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:00.923 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0905e-05 s, 0.0 kB/s 2023-10-10T20:38:00.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:00.924 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1595 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:00.978 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:00.979 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:00.979 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8435e-05 s, 0.0 kB/s 2023-10-10T20:38:00.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:00.980 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1596 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:01.034 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:01.034 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:01.035 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0994e-05 s, 0.0 kB/s 2023-10-10T20:38:01.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:01.036 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1597 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:01.090 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:01.091 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:01.091 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1901e-05 s, 0.0 kB/s 2023-10-10T20:38:01.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:01.092 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1598 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:01.146 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:01.147 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:01.147 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7764e-05 s, 0.0 kB/s 2023-10-10T20:38:01.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:01.148 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1599 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:01.202 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:01.202 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:01.203 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0964e-05 s, 0.0 kB/s 2023-10-10T20:38:01.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:01.203 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1600 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:01.258 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:01.258 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:01.258 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8455e-05 s, 0.0 kB/s 2023-10-10T20:38:01.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:01.259 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1601 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:01.313 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:01.313 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:01.313 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0166e-05 s, 0.0 kB/s 2023-10-10T20:38:01.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:01.314 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1602 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:01.387 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:01.388 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:01.388 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.978e-05 s, 0.0 kB/s 2023-10-10T20:38:01.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:01.389 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1603 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:01.442 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:01.443 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:01.443 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1291e-05 s, 0.0 kB/s 2023-10-10T20:38:01.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:01.444 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1604 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:01.498 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:01.498 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:01.499 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3709e-05 s, 0.0 kB/s 2023-10-10T20:38:01.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:01.499 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1605 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:01.553 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:01.553 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:01.553 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1376e-05 s, 0.0 kB/s 2023-10-10T20:38:01.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:01.554 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1606 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:01.608 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:01.608 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:01.608 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8438e-05 s, 0.0 kB/s 2023-10-10T20:38:01.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:01.609 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1607 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:01.663 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:01.663 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:01.664 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0654e-05 s, 0.0 kB/s 2023-10-10T20:38:01.664 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:01.664 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1608 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:01.719 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:01.719 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:01.719 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0645e-05 s, 0.0 kB/s 2023-10-10T20:38:01.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:01.720 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1609 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:01.774 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:01.774 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:01.774 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7302e-05 s, 0.0 kB/s 2023-10-10T20:38:01.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:01.775 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1610 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:01.829 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:01.829 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:01.829 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5532e-05 s, 0.0 kB/s 2023-10-10T20:38:01.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:01.830 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1611 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:01.884 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:01.884 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:01.884 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8128e-05 s, 0.0 kB/s 2023-10-10T20:38:01.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:01.885 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1612 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:01.940 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:01.940 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:01.940 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3856e-05 s, 0.0 kB/s 2023-10-10T20:38:01.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:01.941 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1613 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:01.995 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:01.995 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:01.996 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0603e-05 s, 0.0 kB/s 2023-10-10T20:38:01.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:01.996 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1614 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:02.051 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:02.051 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:02.051 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0096e-05 s, 0.0 kB/s 2023-10-10T20:38:02.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:02.052 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1615 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:02.106 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8986e-05 s, 0.0 kB/s 2023-10-10T20:38:02.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:02.110 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1616 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:02.162 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:02.162 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:02.162 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8584e-05 s, 0.0 kB/s 2023-10-10T20:38:02.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:02.163 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1617 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:02.217 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:02.217 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:02.217 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.015e-05 s, 0.0 kB/s 2023-10-10T20:38:02.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:02.218 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1618 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:02.271 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:02.272 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:02.272 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.18e-05 s, 0.0 kB/s 2023-10-10T20:38:02.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:02.273 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1619 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:02.328 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:02.329 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:02.329 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9012e-05 s, 0.0 kB/s 2023-10-10T20:38:02.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:02.330 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1620 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:02.384 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:02.384 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:02.384 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6865e-05 s, 0.0 kB/s 2023-10-10T20:38:02.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:02.385 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1621 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:02.439 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:02.439 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:02.439 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0372e-05 s, 0.0 kB/s 2023-10-10T20:38:02.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:02.440 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1622 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:02.494 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:02.494 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:02.494 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9214e-05 s, 0.0 kB/s 2023-10-10T20:38:02.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:02.495 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1623 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:02.549 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:02.549 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:02.549 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0892e-05 s, 0.0 kB/s 2023-10-10T20:38:02.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:02.550 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1624 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:02.604 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:02.604 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:02.605 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8417e-05 s, 0.0 kB/s 2023-10-10T20:38:02.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:02.605 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1625 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:02.660 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:02.660 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:02.660 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.07e-05 s, 0.0 kB/s 2023-10-10T20:38:02.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:02.661 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1626 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:02.716 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:02.716 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:02.717 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1491e-05 s, 0.0 kB/s 2023-10-10T20:38:02.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:02.717 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1627 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:02.772 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:02.772 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:02.772 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0442e-05 s, 0.0 kB/s 2023-10-10T20:38:02.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:02.773 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1628 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:02.827 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:02.827 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:02.827 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8757e-05 s, 0.0 kB/s 2023-10-10T20:38:02.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:02.828 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1629 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:02.881 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:02.881 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:02.882 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9363e-05 s, 0.0 kB/s 2023-10-10T20:38:02.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:02.883 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1630 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:02.938 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:02.939 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:02.939 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1668e-05 s, 0.0 kB/s 2023-10-10T20:38:02.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:02.940 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1631 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:02.994 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:02.995 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:02.995 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.221e-05 s, 0.0 kB/s 2023-10-10T20:38:02.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:02.996 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1632 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:03.050 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:03.050 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:03.050 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3909e-05 s, 0.0 kB/s 2023-10-10T20:38:03.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:03.051 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1633 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:03.105 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:03.106 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:03.106 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2819e-05 s, 0.0 kB/s 2023-10-10T20:38:03.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:03.107 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1634 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:03.161 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:03.161 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:03.162 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0043e-05 s, 0.0 kB/s 2023-10-10T20:38:03.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:03.162 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1635 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:03.217 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:03.217 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:03.217 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3031e-05 s, 0.0 kB/s 2023-10-10T20:38:03.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:03.218 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1636 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:03.272 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:03.272 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:03.272 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2355e-05 s, 0.0 kB/s 2023-10-10T20:38:03.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:03.273 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1637 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:03.327 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:03.327 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:03.327 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9671e-05 s, 0.0 kB/s 2023-10-10T20:38:03.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:03.328 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1638 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:03.382 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:03.382 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:03.383 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1171e-05 s, 0.0 kB/s 2023-10-10T20:38:03.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:03.383 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1639 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:03.438 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:03.438 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:03.438 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.204e-05 s, 0.0 kB/s 2023-10-10T20:38:03.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:03.439 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1640 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:03.493 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:03.493 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:03.493 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7093e-05 s, 0.0 kB/s 2023-10-10T20:38:03.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:03.494 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1641 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:03.549 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:03.549 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9617e-05 s, 0.0 kB/s 2023-10-10T20:38:03.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:03.550 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1642 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:03.604 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:03.604 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:03.604 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.571e-05 s, 0.0 kB/s 2023-10-10T20:38:03.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:03.605 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1643 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.589e-05 s, 0.0 kB/s 2023-10-10T20:38:03.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:03.660 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1644 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:03.714 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:03.715 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:03.715 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9983e-05 s, 0.0 kB/s 2023-10-10T20:38:03.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:03.715 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1645 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:03.771 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:03.772 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:03.772 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9766e-05 s, 0.0 kB/s 2023-10-10T20:38:03.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:03.773 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1646 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:03.827 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:03.827 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:03.828 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0269e-05 s, 0.0 kB/s 2023-10-10T20:38:03.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:03.828 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1647 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:03.883 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:03.883 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:03.883 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9182e-05 s, 0.0 kB/s 2023-10-10T20:38:03.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:03.884 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1648 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:03.938 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:03.938 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:03.938 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8898e-05 s, 0.0 kB/s 2023-10-10T20:38:03.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:03.939 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1649 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:03.994 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:03.994 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:03.994 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6331e-05 s, 0.0 kB/s 2023-10-10T20:38:03.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:03.995 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1650 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1042e-05 s, 0.0 kB/s 2023-10-10T20:38:04.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:04.051 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1651 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:04.105 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:04.105 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:04.106 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9825e-05 s, 0.0 kB/s 2023-10-10T20:38:04.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:04.106 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1652 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:04.161 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:04.161 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:04.161 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9872e-05 s, 0.0 kB/s 2023-10-10T20:38:04.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:04.162 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1653 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:04.216 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:04.218 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:04.218 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8775e-05 s, 0.0 kB/s 2023-10-10T20:38:04.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:04.219 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1654 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8317e-05 s, 0.0 kB/s 2023-10-10T20:38:04.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:04.274 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1655 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:04.328 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:04.328 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:04.329 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9636e-05 s, 0.0 kB/s 2023-10-10T20:38:04.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:04.329 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1656 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:04.384 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:04.384 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:04.385 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0004e-05 s, 0.0 kB/s 2023-10-10T20:38:04.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:04.386 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1657 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8714e-05 s, 0.0 kB/s 2023-10-10T20:38:04.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:04.440 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1658 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0422e-05 s, 0.0 kB/s 2023-10-10T20:38:04.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:04.495 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1659 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:04.551 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:04.551 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:04.551 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.672e-05 s, 0.0 kB/s 2023-10-10T20:38:04.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:04.552 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1660 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:04.605 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:04.606 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:04.606 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0616e-05 s, 0.0 kB/s 2023-10-10T20:38:04.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:04.607 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1661 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:04.661 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:04.661 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:04.661 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7559e-05 s, 0.0 kB/s 2023-10-10T20:38:04.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:04.662 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1662 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:04.716 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:04.716 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:04.716 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3876e-05 s, 0.0 kB/s 2023-10-10T20:38:04.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:04.717 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1663 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:04.772 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:04.772 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:04.772 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8658e-05 s, 0.0 kB/s 2023-10-10T20:38:04.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:04.773 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1664 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0499e-05 s, 0.0 kB/s 2023-10-10T20:38:04.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:04.829 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1665 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:04.883 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:04.883 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:04.884 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8794e-05 s, 0.0 kB/s 2023-10-10T20:38:04.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:04.884 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1666 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:04.939 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:04.939 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:04.940 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5743e-05 s, 0.0 kB/s 2023-10-10T20:38:04.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:04.940 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1667 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:04.994 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:04.994 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:04.994 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2142e-05 s, 0.0 kB/s 2023-10-10T20:38:04.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:04.995 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1668 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:05.048 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:05.049 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:05.049 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.036e-05 s, 0.0 kB/s 2023-10-10T20:38:05.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:05.050 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1669 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:05.105 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:05.105 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:05.105 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2431e-05 s, 0.0 kB/s 2023-10-10T20:38:05.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:05.106 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1670 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:05.159 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:05.160 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:05.160 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0221e-05 s, 0.0 kB/s 2023-10-10T20:38:05.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:05.161 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1671 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:05.215 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:05.215 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:05.215 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.831e-05 s, 0.0 kB/s 2023-10-10T20:38:05.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:05.216 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1672 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:05.270 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:05.270 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:05.270 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.042e-05 s, 0.0 kB/s 2023-10-10T20:38:05.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:05.271 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1673 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:05.325 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:05.325 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:05.326 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9556e-05 s, 0.0 kB/s 2023-10-10T20:38:05.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:05.326 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1674 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:05.380 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:05.380 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:05.380 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2323e-05 s, 0.0 kB/s 2023-10-10T20:38:05.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:05.381 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1675 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:05.435 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:05.435 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:05.435 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1635e-05 s, 0.0 kB/s 2023-10-10T20:38:05.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:05.437 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1676 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2966e-05 s, 0.0 kB/s 2023-10-10T20:38:05.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:05.491 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1677 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:05.545 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:05.546 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:05.546 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1345e-05 s, 0.0 kB/s 2023-10-10T20:38:05.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:05.546 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1678 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:05.601 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:05.602 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:05.602 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9519e-05 s, 0.0 kB/s 2023-10-10T20:38:05.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:05.603 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1679 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:05.658 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4046e-05 s, 0.0 kB/s 2023-10-10T20:38:05.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:05.658 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1680 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:05.712 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:05.713 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:05.713 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7928e-05 s, 0.0 kB/s 2023-10-10T20:38:05.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:05.714 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1681 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:05.767 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:05.767 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:05.768 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9849e-05 s, 0.0 kB/s 2023-10-10T20:38:05.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:05.769 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1682 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:05.824 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:05.824 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:05.825 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1036e-05 s, 0.0 kB/s 2023-10-10T20:38:05.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:05.825 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1683 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:05.880 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:05.880 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:05.881 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9251e-05 s, 0.0 kB/s 2023-10-10T20:38:05.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:05.881 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1684 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:05.935 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:05.936 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:05.936 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2136e-05 s, 0.0 kB/s 2023-10-10T20:38:05.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:05.938 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1685 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:05.993 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:05.993 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1229e-05 s, 0.0 kB/s 2023-10-10T20:38:05.994 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:05.994 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1686 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:06.049 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:06.049 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:06.049 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.128e-05 s, 0.0 kB/s 2023-10-10T20:38:06.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:06.050 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1687 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:06.105 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:06.105 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:06.105 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0922e-05 s, 0.0 kB/s 2023-10-10T20:38:06.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:06.106 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1688 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:06.161 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:06.161 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:06.161 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9896e-05 s, 0.0 kB/s 2023-10-10T20:38:06.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:06.162 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1689 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:06.216 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:06.216 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:06.216 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9795e-05 s, 0.0 kB/s 2023-10-10T20:38:06.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:06.217 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1690 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:06.271 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:06.271 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:06.271 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3639e-05 s, 0.0 kB/s 2023-10-10T20:38:06.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:06.272 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1691 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:06.326 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:06.326 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:06.327 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9047e-05 s, 0.0 kB/s 2023-10-10T20:38:06.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:06.328 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1692 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0344e-05 s, 0.0 kB/s 2023-10-10T20:38:06.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:06.383 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1693 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:06.437 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:06.438 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:06.438 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2239e-05 s, 0.0 kB/s 2023-10-10T20:38:06.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:06.438 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1694 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:06.507 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:06.507 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:06.507 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.595e-05 s, 0.0 kB/s 2023-10-10T20:38:06.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:06.508 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1695 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:06.562 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:06.563 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:06.563 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2213e-05 s, 0.0 kB/s 2023-10-10T20:38:06.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:06.564 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1696 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:06.618 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:06.619 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:06.619 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8524e-05 s, 0.0 kB/s 2023-10-10T20:38:06.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:06.620 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1697 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:06.674 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:06.674 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:06.674 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9492e-05 s, 0.0 kB/s 2023-10-10T20:38:06.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:06.675 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1698 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:06.729 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:06.729 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:06.729 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.986e-05 s, 0.0 kB/s 2023-10-10T20:38:06.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:06.730 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1699 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:06.784 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:06.785 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:06.785 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9906e-05 s, 0.0 kB/s 2023-10-10T20:38:06.786 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:06.787 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1700 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:06.842 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:06.842 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:06.843 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6512e-05 s, 0.0 kB/s 2023-10-10T20:38:06.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:06.844 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1701 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:06.898 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:06.898 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:06.899 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.868e-05 s, 0.0 kB/s 2023-10-10T20:38:06.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:06.899 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1702 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:06.953 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:06.954 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:06.954 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3094e-05 s, 0.0 kB/s 2023-10-10T20:38:06.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:06.955 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1703 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:07.009 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:07.009 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:07.010 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9013e-05 s, 0.0 kB/s 2023-10-10T20:38:07.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:07.010 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1704 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:07.065 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:07.066 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:07.066 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9501e-05 s, 0.0 kB/s 2023-10-10T20:38:07.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:07.067 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1705 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:07.122 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:07.122 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:07.122 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9642e-05 s, 0.0 kB/s 2023-10-10T20:38:07.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:07.123 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1706 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:07.177 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:07.177 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:07.177 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6723e-05 s, 0.0 kB/s 2023-10-10T20:38:07.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:07.178 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1707 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:07.232 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:07.232 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:07.232 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.7989e-05 s, 0.0 kB/s 2023-10-10T20:38:07.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:07.233 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1708 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:07.288 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:07.289 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:07.289 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1538e-05 s, 0.0 kB/s 2023-10-10T20:38:07.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:07.290 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1709 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:07.344 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:07.344 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:07.345 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7752e-05 s, 0.0 kB/s 2023-10-10T20:38:07.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:07.345 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1710 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:07.400 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:07.400 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:07.400 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6506e-05 s, 0.0 kB/s 2023-10-10T20:38:07.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:07.401 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1711 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:07.455 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:07.455 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:07.455 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5112e-05 s, 0.0 kB/s 2023-10-10T20:38:07.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:07.456 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1712 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:07.510 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:07.511 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:07.511 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8266e-05 s, 0.0 kB/s 2023-10-10T20:38:07.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:07.512 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1713 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:07.566 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:07.567 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:07.567 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.835e-05 s, 0.0 kB/s 2023-10-10T20:38:07.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:07.568 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1714 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:07.622 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:07.622 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:07.623 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2969e-05 s, 0.0 kB/s 2023-10-10T20:38:07.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:07.624 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1715 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:07.678 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9802e-05 s, 0.0 kB/s 2023-10-10T20:38:07.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:07.681 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1716 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:07.734 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:07.735 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:07.735 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1224e-05 s, 0.0 kB/s 2023-10-10T20:38:07.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:07.737 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1717 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:07.790 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:07.790 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:07.791 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4589e-05 s, 0.0 kB/s 2023-10-10T20:38:07.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:07.791 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1718 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:07.846 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:07.846 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:07.846 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7948e-05 s, 0.0 kB/s 2023-10-10T20:38:07.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:07.847 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1719 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:07.901 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:07.901 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:07.901 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8884e-05 s, 0.0 kB/s 2023-10-10T20:38:07.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:07.902 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1720 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:07.957 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:07.957 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:07.957 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8773e-05 s, 0.0 kB/s 2023-10-10T20:38:07.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:07.958 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1721 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:08.012 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:08.013 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:08.013 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0195e-05 s, 0.0 kB/s 2023-10-10T20:38:08.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:08.013 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1722 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:08.068 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:08.069 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:08.069 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1037e-05 s, 0.0 kB/s 2023-10-10T20:38:08.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:08.070 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1723 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:08.125 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:08.125 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:08.125 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8403e-05 s, 0.0 kB/s 2023-10-10T20:38:08.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:08.126 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1724 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:08.180 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:08.181 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:08.181 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0988e-05 s, 0.0 kB/s 2023-10-10T20:38:08.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:08.182 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1725 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:08.236 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:08.236 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:08.237 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1043e-05 s, 0.0 kB/s 2023-10-10T20:38:08.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:08.238 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1726 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:08.292 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:08.292 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:08.292 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3386e-05 s, 0.0 kB/s 2023-10-10T20:38:08.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:08.294 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1727 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:08.349 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:08.349 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:08.349 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9223e-05 s, 0.0 kB/s 2023-10-10T20:38:08.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:08.350 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1728 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:08.405 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:08.405 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:08.405 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8815e-05 s, 0.0 kB/s 2023-10-10T20:38:08.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:08.406 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1729 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:08.460 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:08.460 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:08.461 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.8457e-05 s, 0.0 kB/s 2023-10-10T20:38:08.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:08.461 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1730 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:08.516 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:08.517 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:08.517 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9679e-05 s, 0.0 kB/s 2023-10-10T20:38:08.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:08.518 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1731 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:08.572 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:08.572 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:08.573 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1676e-05 s, 0.0 kB/s 2023-10-10T20:38:08.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:08.573 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1732 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:08.627 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:08.627 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:08.628 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1735e-05 s, 0.0 kB/s 2023-10-10T20:38:08.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:08.628 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1733 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.888e-05 s, 0.0 kB/s 2023-10-10T20:38:08.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:08.684 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1734 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:08.738 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:08.739 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:08.739 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1289e-05 s, 0.0 kB/s 2023-10-10T20:38:08.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:08.740 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1735 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:08.795 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:08.795 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:08.795 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0898e-05 s, 0.0 kB/s 2023-10-10T20:38:08.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:08.797 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1736 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:08.850 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:08.851 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:08.851 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8912e-05 s, 0.0 kB/s 2023-10-10T20:38:08.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:08.852 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1737 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:08.907 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:08.907 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:08.907 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9265e-05 s, 0.0 kB/s 2023-10-10T20:38:08.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:08.908 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1738 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:08.962 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:08.963 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:08.963 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8955e-05 s, 0.0 kB/s 2023-10-10T20:38:08.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:08.964 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1739 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:09.019 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:09.019 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0449e-05 s, 0.0 kB/s 2023-10-10T20:38:09.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:09.020 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1740 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:09.074 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.592e-05 s, 0.0 kB/s 2023-10-10T20:38:09.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:09.076 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1741 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:09.131 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:09.131 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:09.131 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2216e-05 s, 0.0 kB/s 2023-10-10T20:38:09.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:09.132 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1742 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:09.186 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:09.187 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:09.187 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8018e-05 s, 0.0 kB/s 2023-10-10T20:38:09.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:09.188 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1743 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:09.242 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:09.242 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:09.242 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8863e-05 s, 0.0 kB/s 2023-10-10T20:38:09.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:09.243 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1744 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:09.297 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:09.298 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:09.298 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1135e-05 s, 0.0 kB/s 2023-10-10T20:38:09.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:09.299 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1745 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:09.353 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:09.353 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:09.354 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1303e-05 s, 0.0 kB/s 2023-10-10T20:38:09.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:09.354 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1746 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:09.408 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:09.408 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:09.409 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7773e-05 s, 0.0 kB/s 2023-10-10T20:38:09.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:09.409 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1747 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:09.467 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:09.467 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:09.467 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0956e-05 s, 0.0 kB/s 2023-10-10T20:38:09.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:09.468 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1748 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:09.522 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:09.523 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:09.523 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0909e-05 s, 0.0 kB/s 2023-10-10T20:38:09.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:09.523 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1749 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:09.578 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:09.578 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:09.579 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8019e-05 s, 0.0 kB/s 2023-10-10T20:38:09.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:09.579 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1750 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:09.633 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:09.633 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:09.634 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9644e-05 s, 0.0 kB/s 2023-10-10T20:38:09.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:09.634 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1751 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:09.688 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:09.689 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:09.689 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7498e-05 s, 0.0 kB/s 2023-10-10T20:38:09.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:09.690 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1752 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:09.744 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:09.744 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:09.744 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8942e-05 s, 0.0 kB/s 2023-10-10T20:38:09.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:09.745 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1753 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:09.800 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:09.800 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9172e-05 s, 0.0 kB/s 2023-10-10T20:38:09.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:09.801 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1754 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:09.855 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:09.856 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:09.856 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9751e-05 s, 0.0 kB/s 2023-10-10T20:38:09.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:09.858 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1755 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:09.911 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:09.912 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:09.912 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.207e-05 s, 0.0 kB/s 2023-10-10T20:38:09.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:09.913 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1756 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:09.967 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:09.967 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:09.968 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0493e-05 s, 0.0 kB/s 2023-10-10T20:38:09.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:09.968 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1757 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:10.023 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:10.024 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:10.024 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0799e-05 s, 0.0 kB/s 2023-10-10T20:38:10.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:10.025 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1758 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:10.079 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:10.080 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:10.080 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0222e-05 s, 0.0 kB/s 2023-10-10T20:38:10.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:10.081 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1759 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:10.136 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:10.136 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:10.136 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2728e-05 s, 0.0 kB/s 2023-10-10T20:38:10.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:10.137 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1760 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:10.191 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:10.191 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:10.192 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0708e-05 s, 0.0 kB/s 2023-10-10T20:38:10.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:10.192 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1761 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:10.247 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:10.248 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:10.248 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.025e-05 s, 0.0 kB/s 2023-10-10T20:38:10.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:10.249 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1762 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:10.303 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:10.304 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:10.304 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1482e-05 s, 0.0 kB/s 2023-10-10T20:38:10.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:10.305 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1763 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:10.359 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:10.360 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:10.360 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8333e-05 s, 0.0 kB/s 2023-10-10T20:38:10.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:10.361 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1764 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:10.415 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:10.416 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:10.416 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.4638e-05 s, 0.0 kB/s 2023-10-10T20:38:10.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:10.417 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1765 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:10.471 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:10.471 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:10.472 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7028e-05 s, 0.0 kB/s 2023-10-10T20:38:10.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:10.472 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1766 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:10.527 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:10.527 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:10.528 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8417e-05 s, 0.0 kB/s 2023-10-10T20:38:10.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:10.528 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1767 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:10.583 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:10.583 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:10.584 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0003e-05 s, 0.0 kB/s 2023-10-10T20:38:10.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:10.584 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1768 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:10.640 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:10.640 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:10.640 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8333e-05 s, 0.0 kB/s 2023-10-10T20:38:10.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:10.641 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1769 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:10.696 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:10.697 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:10.697 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3782e-05 s, 0.0 kB/s 2023-10-10T20:38:10.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:10.698 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1770 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:10.753 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:10.754 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:10.754 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.133e-05 s, 0.0 kB/s 2023-10-10T20:38:10.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:10.755 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1771 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:10.813 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:10.814 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:10.814 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.7356e-05 s, 0.0 kB/s 2023-10-10T20:38:10.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:10.815 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1772 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:10.871 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:10.871 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:10.871 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0615e-05 s, 0.0 kB/s 2023-10-10T20:38:10.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:10.872 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1773 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:10.929 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:10.929 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:10.929 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0695e-05 s, 0.0 kB/s 2023-10-10T20:38:10.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:10.930 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1774 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:10.986 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:10.986 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:10.987 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7968e-05 s, 0.0 kB/s 2023-10-10T20:38:10.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:10.988 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1775 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:11.044 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:11.044 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:11.044 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8152e-05 s, 0.0 kB/s 2023-10-10T20:38:11.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:11.045 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1776 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:11.100 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:11.100 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:11.101 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.574e-05 s, 0.0 kB/s 2023-10-10T20:38:11.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:11.101 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1777 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:11.157 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:11.157 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:11.158 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9991e-05 s, 0.0 kB/s 2023-10-10T20:38:11.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:11.159 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1778 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:11.214 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:11.215 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:11.215 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7818e-05 s, 0.0 kB/s 2023-10-10T20:38:11.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:11.216 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1779 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:11.272 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:11.272 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:11.273 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2073e-05 s, 0.0 kB/s 2023-10-10T20:38:11.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:11.274 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1780 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:11.329 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:11.329 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:11.330 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9223e-05 s, 0.0 kB/s 2023-10-10T20:38:11.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:11.330 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1781 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:11.385 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:11.386 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:11.386 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1749e-05 s, 0.0 kB/s 2023-10-10T20:38:11.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:11.387 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1782 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:11.442 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:11.443 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:11.443 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.174e-05 s, 0.0 kB/s 2023-10-10T20:38:11.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:11.444 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1783 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:11.498 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:11.499 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:11.499 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1367e-05 s, 0.0 kB/s 2023-10-10T20:38:11.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:11.500 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1784 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:11.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:11.555 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:11.555 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9575e-05 s, 0.0 kB/s 2023-10-10T20:38:11.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:11.556 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1785 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:11.610 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:11.611 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:11.611 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1556e-05 s, 0.0 kB/s 2023-10-10T20:38:11.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:11.612 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1786 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:11.667 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:11.667 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:11.667 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3417e-05 s, 0.0 kB/s 2023-10-10T20:38:11.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:11.668 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1787 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:11.723 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:11.723 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:11.723 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0536e-05 s, 0.0 kB/s 2023-10-10T20:38:11.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:11.724 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1788 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:11.779 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:11.779 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:11.779 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7836e-05 s, 0.0 kB/s 2023-10-10T20:38:11.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:11.780 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1789 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:11.835 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:11.835 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:11.835 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9701e-05 s, 0.0 kB/s 2023-10-10T20:38:11.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:11.836 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1790 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:11.891 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:11.891 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:11.891 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4948e-05 s, 0.0 kB/s 2023-10-10T20:38:11.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:11.892 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1791 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:11.947 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:11.947 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:11.947 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1929e-05 s, 0.0 kB/s 2023-10-10T20:38:11.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:11.948 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1792 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:12.003 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:12.003 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:12.003 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0295e-05 s, 0.0 kB/s 2023-10-10T20:38:12.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:12.004 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1793 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:12.059 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:12.059 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:12.059 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3425e-05 s, 0.0 kB/s 2023-10-10T20:38:12.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:12.060 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1794 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:12.115 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:12.115 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:12.115 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1834e-05 s, 0.0 kB/s 2023-10-10T20:38:12.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:12.116 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1795 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:12.153 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:38:12.158 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:38:12.170 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:12.170 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:12.170 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4074e-05 s, 0.0 kB/s 2023-10-10T20:38:12.171 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:12.171 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1796 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:12.228 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:12.228 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:12.228 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2708e-05 s, 0.0 kB/s 2023-10-10T20:38:12.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:12.229 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1797 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:12.283 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:12.284 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:12.284 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3553e-05 s, 0.0 kB/s 2023-10-10T20:38:12.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:12.285 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1798 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:12.341 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:12.341 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:12.341 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1341e-05 s, 0.0 kB/s 2023-10-10T20:38:12.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:12.342 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1799 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:12.397 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:12.397 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:12.397 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.968e-05 s, 0.0 kB/s 2023-10-10T20:38:12.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:12.398 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1800 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:12.453 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:12.454 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:12.454 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1879e-05 s, 0.0 kB/s 2023-10-10T20:38:12.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:12.455 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1801 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:12.509 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:12.510 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:12.510 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.9073e-05 s, 0.0 kB/s 2023-10-10T20:38:12.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:12.514 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1802 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:12.565 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:12.566 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:12.566 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9537e-05 s, 0.0 kB/s 2023-10-10T20:38:12.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:12.567 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1803 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:12.683 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:12.683 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:12.683 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.429e-05 s, 0.0 kB/s 2023-10-10T20:38:12.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:12.684 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1804 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 5.3407e-05 s, 0.0 kB/s 2023-10-10T20:38:12.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:12.701 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1805 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:12.755 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:12.755 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:12.756 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1322e-05 s, 0.0 kB/s 2023-10-10T20:38:12.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:12.756 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1806 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:12.812 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:12.812 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0275e-05 s, 0.0 kB/s 2023-10-10T20:38:12.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:12.813 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1807 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1762e-05 s, 0.0 kB/s 2023-10-10T20:38:12.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:12.869 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1808 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:12.923 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:12.924 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:12.924 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1529e-05 s, 0.0 kB/s 2023-10-10T20:38:12.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:12.925 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1809 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:12.980 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:12.980 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:12.980 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1813e-05 s, 0.0 kB/s 2023-10-10T20:38:12.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:12.981 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1810 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:13.035 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:13.035 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:13.036 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0215e-05 s, 0.0 kB/s 2023-10-10T20:38:13.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:13.036 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1811 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:13.091 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:13.092 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:13.092 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4247e-05 s, 0.0 kB/s 2023-10-10T20:38:13.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:13.093 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1812 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:13.147 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:13.148 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:13.148 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8538e-05 s, 0.0 kB/s 2023-10-10T20:38:13.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:13.149 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1813 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:13.204 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:13.205 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:13.205 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3153e-05 s, 0.0 kB/s 2023-10-10T20:38:13.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:13.206 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1814 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:13.260 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:13.261 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:13.261 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9025e-05 s, 0.0 kB/s 2023-10-10T20:38:13.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:13.262 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1815 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:13.316 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:13.316 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:13.317 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2665e-05 s, 0.0 kB/s 2023-10-10T20:38:13.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:13.318 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1816 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:13.373 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:13.374 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:13.374 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.23e-05 s, 0.0 kB/s 2023-10-10T20:38:13.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:13.375 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1817 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:13.430 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:13.430 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:13.430 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0524e-05 s, 0.0 kB/s 2023-10-10T20:38:13.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:13.431 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1818 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:13.486 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:13.486 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:13.486 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1276e-05 s, 0.0 kB/s 2023-10-10T20:38:13.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:13.487 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1819 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:13.541 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:13.542 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:13.542 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1804e-05 s, 0.0 kB/s 2023-10-10T20:38:13.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:13.543 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1820 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:13.598 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:13.598 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1957e-05 s, 0.0 kB/s 2023-10-10T20:38:13.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:13.599 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1821 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:13.654 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:13.654 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:13.654 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0198e-05 s, 0.0 kB/s 2023-10-10T20:38:13.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:13.655 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1822 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:13.709 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:13.710 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:13.710 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0419e-05 s, 0.0 kB/s 2023-10-10T20:38:13.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:13.711 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1823 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:13.765 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:13.766 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:13.766 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1018e-05 s, 0.0 kB/s 2023-10-10T20:38:13.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:13.767 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1824 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:13.822 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:13.822 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:13.822 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1595e-05 s, 0.0 kB/s 2023-10-10T20:38:13.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:13.823 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1825 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:13.877 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:13.877 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:13.878 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1639e-05 s, 0.0 kB/s 2023-10-10T20:38:13.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:13.879 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1826 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1244e-05 s, 0.0 kB/s 2023-10-10T20:38:13.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:13.935 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1827 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:13.989 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:13.989 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:13.990 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3268e-05 s, 0.0 kB/s 2023-10-10T20:38:13.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:13.990 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1828 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:14.046 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:14.046 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:14.046 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0676e-05 s, 0.0 kB/s 2023-10-10T20:38:14.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:14.047 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1829 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:14.102 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:14.103 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:14.103 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3097e-05 s, 0.0 kB/s 2023-10-10T20:38:14.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:14.104 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1830 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:14.158 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:14.159 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:14.159 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2361e-05 s, 0.0 kB/s 2023-10-10T20:38:14.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:14.160 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1831 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:14.215 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:14.216 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:14.216 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0854e-05 s, 0.0 kB/s 2023-10-10T20:38:14.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:14.217 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1832 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:14.272 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:14.272 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:14.272 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9799e-05 s, 0.0 kB/s 2023-10-10T20:38:14.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:14.273 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1833 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:14.327 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:14.328 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:14.328 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0216e-05 s, 0.0 kB/s 2023-10-10T20:38:14.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:14.329 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1834 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:14.383 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:14.384 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:14.384 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1352e-05 s, 0.0 kB/s 2023-10-10T20:38:14.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:14.385 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1835 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:14.439 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0861e-05 s, 0.0 kB/s 2023-10-10T20:38:14.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:14.441 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1836 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0708e-05 s, 0.0 kB/s 2023-10-10T20:38:14.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:14.496 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1837 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:14.551 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:14.551 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1347e-05 s, 0.0 kB/s 2023-10-10T20:38:14.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:14.552 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1838 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3137e-05 s, 0.0 kB/s 2023-10-10T20:38:14.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:14.610 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1839 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:14.664 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:14.664 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:14.665 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1478e-05 s, 0.0 kB/s 2023-10-10T20:38:14.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:14.665 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1840 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:14.720 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1865e-05 s, 0.0 kB/s 2023-10-10T20:38:14.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:14.722 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1841 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:14.776 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:14.777 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:14.777 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2672e-05 s, 0.0 kB/s 2023-10-10T20:38:14.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:14.778 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1842 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:14.832 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:14.833 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:14.833 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1111e-05 s, 0.0 kB/s 2023-10-10T20:38:14.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:14.834 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1843 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:14.888 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:14.889 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:14.889 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9936e-05 s, 0.0 kB/s 2023-10-10T20:38:14.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:14.890 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1844 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:14.944 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:14.945 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:14.945 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0368e-05 s, 0.0 kB/s 2023-10-10T20:38:14.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:14.946 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1845 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0652e-05 s, 0.0 kB/s 2023-10-10T20:38:15.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:15.002 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1846 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:15.057 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:15.057 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:15.058 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0716e-05 s, 0.0 kB/s 2023-10-10T20:38:15.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:15.059 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1847 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:15.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:15.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:15.113 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1516e-05 s, 0.0 kB/s 2023-10-10T20:38:15.114 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:15.114 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1848 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:15.169 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9629e-05 s, 0.0 kB/s 2023-10-10T20:38:15.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:15.171 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1849 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:15.227 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:15.227 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:15.227 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8543e-05 s, 0.0 kB/s 2023-10-10T20:38:15.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:15.229 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1850 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:15.282 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:15.283 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:15.283 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0456e-05 s, 0.0 kB/s 2023-10-10T20:38:15.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:15.284 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1851 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:15.339 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:15.339 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:15.340 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1799e-05 s, 0.0 kB/s 2023-10-10T20:38:15.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:15.341 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1852 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:15.395 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:15.395 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:15.395 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9526e-05 s, 0.0 kB/s 2023-10-10T20:38:15.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:15.396 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1853 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:15.451 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:15.452 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:15.452 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0458e-05 s, 0.0 kB/s 2023-10-10T20:38:15.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:15.453 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1854 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:15.507 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:15.508 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:15.508 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2453e-05 s, 0.0 kB/s 2023-10-10T20:38:15.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:15.509 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1855 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:15.563 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:15.563 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:15.563 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.212e-05 s, 0.0 kB/s 2023-10-10T20:38:15.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:15.565 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1856 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:15.620 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:15.620 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:15.621 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1133e-05 s, 0.0 kB/s 2023-10-10T20:38:15.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:15.621 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1857 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:15.676 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:15.677 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:15.677 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1525e-05 s, 0.0 kB/s 2023-10-10T20:38:15.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:15.677 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1858 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:15.732 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:15.733 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:15.733 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1668e-05 s, 0.0 kB/s 2023-10-10T20:38:15.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:15.734 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1859 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:15.788 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:15.789 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:15.789 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2177e-05 s, 0.0 kB/s 2023-10-10T20:38:15.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:15.790 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1860 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:15.844 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:15.844 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:15.845 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.124e-05 s, 0.0 kB/s 2023-10-10T20:38:15.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:15.845 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1861 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:15.900 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:15.900 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:15.901 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1184e-05 s, 0.0 kB/s 2023-10-10T20:38:15.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:15.901 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1862 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:15.956 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:15.957 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:15.957 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3359e-05 s, 0.0 kB/s 2023-10-10T20:38:15.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:15.958 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1863 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:16.013 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.279e-05 s, 0.0 kB/s 2023-10-10T20:38:16.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:16.013 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1864 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:16.068 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:16.069 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:16.069 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7919e-05 s, 0.0 kB/s 2023-10-10T20:38:16.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:16.070 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1865 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:16.125 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:16.125 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:16.125 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1729e-05 s, 0.0 kB/s 2023-10-10T20:38:16.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:16.126 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1866 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:16.180 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:16.181 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:16.181 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.864e-05 s, 0.0 kB/s 2023-10-10T20:38:16.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:16.182 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1867 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:16.237 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:16.237 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:16.237 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.347e-05 s, 0.0 kB/s 2023-10-10T20:38:16.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:16.239 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1868 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:16.295 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:16.295 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:16.295 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3091e-05 s, 0.0 kB/s 2023-10-10T20:38:16.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:16.296 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1869 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:16.350 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7059e-05 s, 0.0 kB/s 2023-10-10T20:38:16.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:16.352 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1870 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:16.408 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9942e-05 s, 0.0 kB/s 2023-10-10T20:38:16.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:16.408 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1871 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:16.463 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:16.463 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:16.463 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1532e-05 s, 0.0 kB/s 2023-10-10T20:38:16.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:16.464 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1872 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:16.518 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:16.519 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:16.519 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9652e-05 s, 0.0 kB/s 2023-10-10T20:38:16.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:16.520 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1873 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:16.575 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:16.575 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:16.575 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5359e-05 s, 0.0 kB/s 2023-10-10T20:38:16.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:16.576 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1874 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:16.630 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:16.631 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:16.631 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8647e-05 s, 0.0 kB/s 2023-10-10T20:38:16.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:16.632 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1875 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:16.686 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:16.686 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:16.686 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0626e-05 s, 0.0 kB/s 2023-10-10T20:38:16.687 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:16.688 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1876 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:16.741 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:16.741 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:16.741 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8568e-05 s, 0.0 kB/s 2023-10-10T20:38:16.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:16.742 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1877 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:16.797 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:16.797 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:16.797 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0136e-05 s, 0.0 kB/s 2023-10-10T20:38:16.798 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:16.798 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1878 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:16.852 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:16.852 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:16.853 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9591e-05 s, 0.0 kB/s 2023-10-10T20:38:16.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:16.853 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1879 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:16.907 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:16.907 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:16.908 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2197e-05 s, 0.0 kB/s 2023-10-10T20:38:16.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:16.908 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1880 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:16.963 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:16.964 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:16.964 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8829e-05 s, 0.0 kB/s 2023-10-10T20:38:16.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:16.965 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1881 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:17.019 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:17.019 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5311e-05 s, 0.0 kB/s 2023-10-10T20:38:17.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:17.020 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1882 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:17.075 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:17.075 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:17.075 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2476e-05 s, 0.0 kB/s 2023-10-10T20:38:17.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:17.076 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1883 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:17.130 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:17.130 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:17.131 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1039e-05 s, 0.0 kB/s 2023-10-10T20:38:17.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:17.131 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1884 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:17.186 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:17.186 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:17.187 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9657e-05 s, 0.0 kB/s 2023-10-10T20:38:17.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:17.187 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1885 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:17.243 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:17.243 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:17.243 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1312e-05 s, 0.0 kB/s 2023-10-10T20:38:17.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:17.244 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1886 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:17.298 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:17.298 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:17.299 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8712e-05 s, 0.0 kB/s 2023-10-10T20:38:17.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:17.299 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1887 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:17.354 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:17.355 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:17.355 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1646e-05 s, 0.0 kB/s 2023-10-10T20:38:17.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:17.356 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1888 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:17.409 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:17.410 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:17.410 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8639e-05 s, 0.0 kB/s 2023-10-10T20:38:17.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:17.411 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1889 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:17.466 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:17.467 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:17.467 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.879e-05 s, 0.0 kB/s 2023-10-10T20:38:17.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:17.468 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1890 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:17.522 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:17.523 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:17.523 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8389e-05 s, 0.0 kB/s 2023-10-10T20:38:17.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:17.524 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1891 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8973e-05 s, 0.0 kB/s 2023-10-10T20:38:17.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:17.579 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1892 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0124e-05 s, 0.0 kB/s 2023-10-10T20:38:17.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:17.635 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1893 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:17.689 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:17.689 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:17.690 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9859e-05 s, 0.0 kB/s 2023-10-10T20:38:17.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:17.690 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1894 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:17.745 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:17.745 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:17.745 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8625e-05 s, 0.0 kB/s 2023-10-10T20:38:17.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:17.746 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1895 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:17.800 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:17.800 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:17.801 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.822e-05 s, 0.0 kB/s 2023-10-10T20:38:17.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:17.801 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1896 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:17.856 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:17.856 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:17.856 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.751e-05 s, 0.0 kB/s 2023-10-10T20:38:17.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:17.857 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1897 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:17.911 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:17.911 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:17.912 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8756e-05 s, 0.0 kB/s 2023-10-10T20:38:17.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:17.912 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1898 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:17.967 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:17.967 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:17.968 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9196e-05 s, 0.0 kB/s 2023-10-10T20:38:17.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:17.968 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1899 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:18.023 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:18.023 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:18.024 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9674e-05 s, 0.0 kB/s 2023-10-10T20:38:18.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:18.024 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1900 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:18.080 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:18.080 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:18.080 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7737e-05 s, 0.0 kB/s 2023-10-10T20:38:18.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:18.081 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1901 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:18.135 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:18.136 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:18.136 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4613e-05 s, 0.0 kB/s 2023-10-10T20:38:18.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:18.137 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1902 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:18.192 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.045e-05 s, 0.0 kB/s 2023-10-10T20:38:18.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:18.192 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1903 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:18.248 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:18.248 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:18.248 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7926e-05 s, 0.0 kB/s 2023-10-10T20:38:18.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:18.249 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1904 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:18.303 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:18.303 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:18.304 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3845e-05 s, 0.0 kB/s 2023-10-10T20:38:18.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:18.304 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1905 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:18.358 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:18.359 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:18.359 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8102e-05 s, 0.0 kB/s 2023-10-10T20:38:18.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:18.360 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1906 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:18.414 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:18.414 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:18.414 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.457e-05 s, 0.0 kB/s 2023-10-10T20:38:18.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:18.415 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1907 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:18.469 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:18.469 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:18.469 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8928e-05 s, 0.0 kB/s 2023-10-10T20:38:18.470 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:18.470 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1908 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:18.524 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:18.525 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:18.525 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7386e-05 s, 0.0 kB/s 2023-10-10T20:38:18.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:18.526 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1909 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1719e-05 s, 0.0 kB/s 2023-10-10T20:38:18.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:18.582 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1910 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:18.635 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:18.636 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:18.636 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1636e-05 s, 0.0 kB/s 2023-10-10T20:38:18.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:18.638 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1911 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:18.693 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:18.694 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:18.694 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8743e-05 s, 0.0 kB/s 2023-10-10T20:38:18.694 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:18.695 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1912 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:18.749 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:18.749 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:18.749 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0928e-05 s, 0.0 kB/s 2023-10-10T20:38:18.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:18.750 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1913 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:18.804 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:18.804 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:18.805 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1427e-05 s, 0.0 kB/s 2023-10-10T20:38:18.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:18.805 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1914 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:18.860 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:18.860 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:18.860 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1897e-05 s, 0.0 kB/s 2023-10-10T20:38:18.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:18.861 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1915 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:18.914 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:18.915 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:18.915 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8482e-05 s, 0.0 kB/s 2023-10-10T20:38:18.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:18.916 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1916 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:18.971 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:18.972 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:18.972 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.833e-05 s, 0.0 kB/s 2023-10-10T20:38:18.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:18.973 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1917 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:19.027 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:19.028 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:19.028 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9193e-05 s, 0.0 kB/s 2023-10-10T20:38:19.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:19.029 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1918 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:19.084 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:19.084 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:19.084 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8603e-05 s, 0.0 kB/s 2023-10-10T20:38:19.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:19.085 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1919 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:19.139 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:19.139 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:19.140 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.772e-05 s, 0.0 kB/s 2023-10-10T20:38:19.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:19.141 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1920 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:19.196 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:19.197 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:19.197 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2811e-05 s, 0.0 kB/s 2023-10-10T20:38:19.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:19.198 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1921 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:19.253 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:19.253 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:19.253 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0013e-05 s, 0.0 kB/s 2023-10-10T20:38:19.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:19.254 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1922 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:19.309 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:19.309 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:19.310 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2271e-05 s, 0.0 kB/s 2023-10-10T20:38:19.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:19.311 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1923 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:19.364 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:19.365 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:19.365 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8997e-05 s, 0.0 kB/s 2023-10-10T20:38:19.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:19.366 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1924 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:19.420 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:19.421 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:19.421 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8025e-05 s, 0.0 kB/s 2023-10-10T20:38:19.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:19.422 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1925 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:19.477 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:19.477 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:19.477 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0381e-05 s, 0.0 kB/s 2023-10-10T20:38:19.478 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:19.478 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1926 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:19.532 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:19.532 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:19.533 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7752e-05 s, 0.0 kB/s 2023-10-10T20:38:19.533 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:19.533 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1927 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7793e-05 s, 0.0 kB/s 2023-10-10T20:38:19.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:19.591 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1928 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:19.645 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:19.645 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:19.646 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0993e-05 s, 0.0 kB/s 2023-10-10T20:38:19.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:19.647 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1929 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:19.701 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:19.701 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:19.702 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3046e-05 s, 0.0 kB/s 2023-10-10T20:38:19.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:19.702 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1930 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:19.757 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:19.758 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:19.758 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0515e-05 s, 0.0 kB/s 2023-10-10T20:38:19.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:19.759 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1931 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:19.814 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:19.814 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:19.814 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0286e-05 s, 0.0 kB/s 2023-10-10T20:38:19.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:19.815 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1932 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:19.870 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:19.870 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:19.871 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8301e-05 s, 0.0 kB/s 2023-10-10T20:38:19.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:19.872 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1933 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:19.925 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:19.925 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:19.926 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1146e-05 s, 0.0 kB/s 2023-10-10T20:38:19.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:19.926 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1934 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:19.981 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:19.981 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:19.981 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0872e-05 s, 0.0 kB/s 2023-10-10T20:38:19.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:19.982 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1935 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:20.036 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:20.036 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:20.036 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9329e-05 s, 0.0 kB/s 2023-10-10T20:38:20.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:20.037 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1936 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:20.091 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:20.092 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:20.092 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0871e-05 s, 0.0 kB/s 2023-10-10T20:38:20.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:20.093 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1937 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:20.147 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:20.147 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:20.148 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9013e-05 s, 0.0 kB/s 2023-10-10T20:38:20.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:20.148 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1938 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:20.203 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:20.203 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:20.204 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1299e-05 s, 0.0 kB/s 2023-10-10T20:38:20.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:20.204 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1939 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:20.260 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:20.260 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:20.260 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1311e-05 s, 0.0 kB/s 2023-10-10T20:38:20.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:20.261 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1940 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:20.316 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:20.316 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:20.316 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0443e-05 s, 0.0 kB/s 2023-10-10T20:38:20.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:20.317 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1941 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:20.371 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:20.372 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:20.372 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9323e-05 s, 0.0 kB/s 2023-10-10T20:38:20.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:20.373 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1942 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:20.427 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:20.428 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:20.428 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.218e-05 s, 0.0 kB/s 2023-10-10T20:38:20.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:20.429 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1943 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:20.484 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:20.485 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:20.485 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.5603e-05 s, 0.0 kB/s 2023-10-10T20:38:20.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:20.486 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1944 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:20.540 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:20.541 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:20.541 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7985e-05 s, 0.0 kB/s 2023-10-10T20:38:20.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:20.542 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1945 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:20.596 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:20.597 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:20.597 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1934e-05 s, 0.0 kB/s 2023-10-10T20:38:20.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:20.598 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1946 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:20.652 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:20.653 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:20.653 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.954e-05 s, 0.0 kB/s 2023-10-10T20:38:20.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:20.654 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1947 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:20.709 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:20.709 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:20.709 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9227e-05 s, 0.0 kB/s 2023-10-10T20:38:20.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:20.710 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1948 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:20.764 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:20.764 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:20.765 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.9512e-05 s, 0.0 kB/s 2023-10-10T20:38:20.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:20.766 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1949 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:20.820 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:20.820 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:20.821 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3115e-05 s, 0.0 kB/s 2023-10-10T20:38:20.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:20.822 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1950 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:20.875 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:20.875 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:20.876 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8218e-05 s, 0.0 kB/s 2023-10-10T20:38:20.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:20.876 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1951 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:20.930 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:20.931 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:20.931 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8901e-05 s, 0.0 kB/s 2023-10-10T20:38:20.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:20.932 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1952 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:20.986 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:20.986 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:20.986 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1319e-05 s, 0.0 kB/s 2023-10-10T20:38:20.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:20.987 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1953 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:21.041 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:21.041 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:21.042 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9357e-05 s, 0.0 kB/s 2023-10-10T20:38:21.042 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:21.043 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1954 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:21.097 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:21.098 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:21.098 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1678e-05 s, 0.0 kB/s 2023-10-10T20:38:21.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:21.099 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1955 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:21.153 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:21.154 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:21.154 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0055e-05 s, 0.0 kB/s 2023-10-10T20:38:21.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:21.155 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1956 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:21.209 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:21.209 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:21.210 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7783e-05 s, 0.0 kB/s 2023-10-10T20:38:21.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:21.210 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1957 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:21.265 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:21.266 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:21.266 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1799e-05 s, 0.0 kB/s 2023-10-10T20:38:21.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:21.267 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1958 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:21.321 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:21.322 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:21.322 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0047e-05 s, 0.0 kB/s 2023-10-10T20:38:21.322 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:21.323 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1959 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:21.376 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:21.377 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:21.377 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9992e-05 s, 0.0 kB/s 2023-10-10T20:38:21.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:21.378 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1960 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:21.434 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:21.434 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:21.434 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2819e-05 s, 0.0 kB/s 2023-10-10T20:38:21.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:21.435 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1961 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1567e-05 s, 0.0 kB/s 2023-10-10T20:38:21.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:21.491 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1962 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:21.546 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:21.546 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:21.546 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.131e-05 s, 0.0 kB/s 2023-10-10T20:38:21.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:21.547 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1963 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:21.601 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:21.602 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:21.602 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8054e-05 s, 0.0 kB/s 2023-10-10T20:38:21.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:21.603 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1964 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:21.657 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:21.658 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:21.658 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0172e-05 s, 0.0 kB/s 2023-10-10T20:38:21.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:21.659 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1965 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:21.713 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:21.713 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:21.713 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0536e-05 s, 0.0 kB/s 2023-10-10T20:38:21.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:21.714 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1966 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:21.769 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:21.769 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0968e-05 s, 0.0 kB/s 2023-10-10T20:38:21.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:21.770 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1967 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:21.824 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:21.824 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:21.825 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4058e-05 s, 0.0 kB/s 2023-10-10T20:38:21.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:21.825 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1968 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:21.882 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:21.882 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:21.882 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9641e-05 s, 0.0 kB/s 2023-10-10T20:38:21.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:21.884 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1969 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:21.938 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:21.939 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:21.939 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3751e-05 s, 0.0 kB/s 2023-10-10T20:38:21.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:21.940 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1970 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:21.994 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:21.995 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:21.995 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.803e-05 s, 0.0 kB/s 2023-10-10T20:38:21.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:21.996 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1971 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:22.051 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:22.051 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:22.051 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0745e-05 s, 0.0 kB/s 2023-10-10T20:38:22.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:22.052 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1972 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:22.106 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:22.107 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:22.107 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1632e-05 s, 0.0 kB/s 2023-10-10T20:38:22.107 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:22.108 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1973 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:22.163 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:22.163 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:22.164 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0572e-05 s, 0.0 kB/s 2023-10-10T20:38:22.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:22.164 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1974 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:22.218 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:22.218 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:22.219 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8738e-05 s, 0.0 kB/s 2023-10-10T20:38:22.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:22.219 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1975 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:22.275 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:22.275 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:22.275 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8993e-05 s, 0.0 kB/s 2023-10-10T20:38:22.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:22.276 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1976 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:22.330 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:22.331 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:22.331 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8539e-05 s, 0.0 kB/s 2023-10-10T20:38:22.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:22.332 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1977 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:22.387 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:22.387 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:22.388 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2319e-05 s, 0.0 kB/s 2023-10-10T20:38:22.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:22.389 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1978 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:22.442 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:22.442 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:22.443 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.24e-05 s, 0.0 kB/s 2023-10-10T20:38:22.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:22.443 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1979 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:22.499 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:22.499 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:22.500 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8528e-05 s, 0.0 kB/s 2023-10-10T20:38:22.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:22.500 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1980 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8986e-05 s, 0.0 kB/s 2023-10-10T20:38:22.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:22.556 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1981 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:22.610 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:22.610 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:22.611 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1094e-05 s, 0.0 kB/s 2023-10-10T20:38:22.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:22.611 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1982 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9165e-05 s, 0.0 kB/s 2023-10-10T20:38:22.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:22.669 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1983 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:22.722 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:22.723 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:22.723 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.96e-05 s, 0.0 kB/s 2023-10-10T20:38:22.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:22.724 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1984 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:22.778 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:22.779 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:22.779 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0425e-05 s, 0.0 kB/s 2023-10-10T20:38:22.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:22.780 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1985 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:22.835 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:22.835 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:22.836 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7311e-05 s, 0.0 kB/s 2023-10-10T20:38:22.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:22.837 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1986 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:22.890 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:22.890 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:22.890 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8967e-05 s, 0.0 kB/s 2023-10-10T20:38:22.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:22.891 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1987 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:22.946 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:22.946 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9895e-05 s, 0.0 kB/s 2023-10-10T20:38:22.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:22.947 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1988 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1581e-05 s, 0.0 kB/s 2023-10-10T20:38:23.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:23.003 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1989 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:23.058 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:23.058 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:23.058 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.144e-05 s, 0.0 kB/s 2023-10-10T20:38:23.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:23.059 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1990 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:23.113 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:23.114 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:23.114 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.922e-05 s, 0.0 kB/s 2023-10-10T20:38:23.114 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:23.115 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1991 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:23.169 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:23.170 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:23.170 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2041e-05 s, 0.0 kB/s 2023-10-10T20:38:23.171 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:23.171 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1992 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:23.225 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:23.225 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:23.225 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0646e-05 s, 0.0 kB/s 2023-10-10T20:38:23.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:23.226 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1993 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:23.281 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:23.281 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2161e-05 s, 0.0 kB/s 2023-10-10T20:38:23.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:23.282 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1994 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1066e-05 s, 0.0 kB/s 2023-10-10T20:38:23.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:23.337 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1995 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:23.405 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:23.405 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:23.405 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.313e-05 s, 0.0 kB/s 2023-10-10T20:38:23.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:23.406 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1996 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:23.460 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:23.460 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:23.461 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1415e-05 s, 0.0 kB/s 2023-10-10T20:38:23.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:23.461 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1997 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:23.516 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:23.516 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:23.516 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3422e-05 s, 0.0 kB/s 2023-10-10T20:38:23.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:23.517 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1998 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:23.571 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:23.571 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:23.572 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8807e-05 s, 0.0 kB/s 2023-10-10T20:38:23.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:23.572 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.1999 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:23.627 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:23.627 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:23.627 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0151e-05 s, 0.0 kB/s 2023-10-10T20:38:23.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:23.628 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2000 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:23.682 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:23.682 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:23.682 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1313e-05 s, 0.0 kB/s 2023-10-10T20:38:23.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:23.683 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2001 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:23.737 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:23.737 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:23.738 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1993e-05 s, 0.0 kB/s 2023-10-10T20:38:23.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:23.738 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2002 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:23.793 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:23.794 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:23.794 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2844e-05 s, 0.0 kB/s 2023-10-10T20:38:23.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:23.795 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2003 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:23.849 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:23.849 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:23.850 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9998e-05 s, 0.0 kB/s 2023-10-10T20:38:23.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:23.850 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2004 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6985e-05 s, 0.0 kB/s 2023-10-10T20:38:23.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:23.906 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2005 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:23.960 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:23.961 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:23.961 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8094e-05 s, 0.0 kB/s 2023-10-10T20:38:23.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:23.962 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2006 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:24.016 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:24.016 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:24.016 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.783e-05 s, 0.0 kB/s 2023-10-10T20:38:24.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:24.017 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2007 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:24.071 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1566e-05 s, 0.0 kB/s 2023-10-10T20:38:24.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:24.073 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2008 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:24.127 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:24.127 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:24.128 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8221e-05 s, 0.0 kB/s 2023-10-10T20:38:24.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:24.128 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2009 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:24.184 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:24.184 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:24.184 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4149e-05 s, 0.0 kB/s 2023-10-10T20:38:24.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:24.185 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2010 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:24.239 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:24.239 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8257e-05 s, 0.0 kB/s 2023-10-10T20:38:24.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:24.240 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2011 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:24.296 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:24.297 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:24.297 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.823e-05 s, 0.0 kB/s 2023-10-10T20:38:24.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:24.298 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2012 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:24.352 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:24.352 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:24.353 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0214e-05 s, 0.0 kB/s 2023-10-10T20:38:24.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:24.353 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2013 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:24.408 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:24.408 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:24.409 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8077e-05 s, 0.0 kB/s 2023-10-10T20:38:24.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:24.409 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2014 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:24.464 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:24.465 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:24.465 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8129e-05 s, 0.0 kB/s 2023-10-10T20:38:24.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:24.466 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2015 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:24.520 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:24.520 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:24.521 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0446e-05 s, 0.0 kB/s 2023-10-10T20:38:24.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:24.521 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2016 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:24.576 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:24.576 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:24.577 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0034e-05 s, 0.0 kB/s 2023-10-10T20:38:24.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:24.577 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2017 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:24.632 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:24.633 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:24.633 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7436e-05 s, 0.0 kB/s 2023-10-10T20:38:24.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:24.634 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2018 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:24.688 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:24.688 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:24.688 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8839e-05 s, 0.0 kB/s 2023-10-10T20:38:24.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:24.689 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2019 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:24.743 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:24.744 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:24.744 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3159e-05 s, 0.0 kB/s 2023-10-10T20:38:24.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:24.744 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2020 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:24.799 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:24.799 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:24.799 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3618e-05 s, 0.0 kB/s 2023-10-10T20:38:24.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:24.800 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2021 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:24.855 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:24.855 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:24.856 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8263e-05 s, 0.0 kB/s 2023-10-10T20:38:24.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:24.857 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2022 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:24.910 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:24.910 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:24.911 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0371e-05 s, 0.0 kB/s 2023-10-10T20:38:24.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:24.912 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2023 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:24.966 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:24.967 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:24.967 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9311e-05 s, 0.0 kB/s 2023-10-10T20:38:24.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:24.968 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2024 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:25.024 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1675e-05 s, 0.0 kB/s 2023-10-10T20:38:25.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:25.025 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2025 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:25.079 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:25.080 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:25.080 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.077e-05 s, 0.0 kB/s 2023-10-10T20:38:25.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:25.081 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2026 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:25.135 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:25.136 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:25.136 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9209e-05 s, 0.0 kB/s 2023-10-10T20:38:25.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:25.137 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2027 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:25.191 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:25.191 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:25.191 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0626e-05 s, 0.0 kB/s 2023-10-10T20:38:25.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:25.195 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2028 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:25.247 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:25.247 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:25.247 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2722e-05 s, 0.0 kB/s 2023-10-10T20:38:25.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:25.248 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2029 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:25.303 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:25.303 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:25.303 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0714e-05 s, 0.0 kB/s 2023-10-10T20:38:25.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:25.304 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2030 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:25.358 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:25.358 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:25.359 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2898e-05 s, 0.0 kB/s 2023-10-10T20:38:25.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:25.359 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2031 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:25.413 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:25.413 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:25.413 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9372e-05 s, 0.0 kB/s 2023-10-10T20:38:25.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:25.414 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2032 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:25.468 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:25.468 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:25.468 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9516e-05 s, 0.0 kB/s 2023-10-10T20:38:25.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:25.469 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2033 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:25.523 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:25.523 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:25.524 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0111e-05 s, 0.0 kB/s 2023-10-10T20:38:25.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:25.524 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2034 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:25.579 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:25.580 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:25.580 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0904e-05 s, 0.0 kB/s 2023-10-10T20:38:25.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:25.581 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2035 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:25.636 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:25.636 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:25.637 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9703e-05 s, 0.0 kB/s 2023-10-10T20:38:25.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:25.637 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2036 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:25.692 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:25.692 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:25.693 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7159e-05 s, 0.0 kB/s 2023-10-10T20:38:25.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:25.693 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2037 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:25.748 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:25.748 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:25.748 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3323e-05 s, 0.0 kB/s 2023-10-10T20:38:25.749 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:25.749 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2038 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:25.803 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:25.804 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:25.804 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1569e-05 s, 0.0 kB/s 2023-10-10T20:38:25.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:25.805 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2039 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:25.859 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:25.859 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:25.860 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.098e-05 s, 0.0 kB/s 2023-10-10T20:38:25.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:25.860 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2040 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:25.915 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:25.915 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:25.916 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9118e-05 s, 0.0 kB/s 2023-10-10T20:38:25.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:25.916 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2041 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:25.971 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:25.972 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:25.972 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9397e-05 s, 0.0 kB/s 2023-10-10T20:38:25.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:25.973 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2042 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:26.027 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:26.027 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:26.027 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.857e-05 s, 0.0 kB/s 2023-10-10T20:38:26.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:26.028 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2043 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:26.082 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:26.083 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:26.083 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.166e-05 s, 0.0 kB/s 2023-10-10T20:38:26.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:26.084 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2044 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:26.138 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9941e-05 s, 0.0 kB/s 2023-10-10T20:38:26.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:26.140 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2045 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:26.195 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:26.216 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:26.217 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3242e-05 s, 0.0 kB/s 2023-10-10T20:38:26.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:26.218 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2046 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:26.255 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:26.256 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:26.256 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7199e-05 s, 0.0 kB/s 2023-10-10T20:38:26.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:26.257 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2047 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:26.318 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:26.318 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:26.318 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4189e-05 s, 0.0 kB/s 2023-10-10T20:38:26.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:26.319 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2048 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:26.374 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8132e-05 s, 0.0 kB/s 2023-10-10T20:38:26.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:26.376 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2049 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:26.442 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:26.444 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:26.444 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.947e-05 s, 0.0 kB/s 2023-10-10T20:38:26.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:26.445 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2050 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:26.498 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:26.498 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:26.499 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3133e-05 s, 0.0 kB/s 2023-10-10T20:38:26.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:26.499 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2051 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:26.559 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:26.560 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:26.560 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0683e-05 s, 0.0 kB/s 2023-10-10T20:38:26.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:26.561 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2052 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:26.617 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:26.617 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:26.618 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2944e-05 s, 0.0 kB/s 2023-10-10T20:38:26.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:26.618 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2053 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:26.673 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:26.673 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:26.674 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3309e-05 s, 0.0 kB/s 2023-10-10T20:38:26.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:26.674 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2054 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:26.729 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:26.730 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:26.730 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.5863e-05 s, 0.0 kB/s 2023-10-10T20:38:26.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:26.731 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2055 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:26.785 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:26.786 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:26.786 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0411e-05 s, 0.0 kB/s 2023-10-10T20:38:26.786 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:26.787 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2056 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:26.841 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:26.841 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:26.842 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8084e-05 s, 0.0 kB/s 2023-10-10T20:38:26.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:26.843 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2057 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:26.897 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:26.897 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:26.898 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8532e-05 s, 0.0 kB/s 2023-10-10T20:38:26.898 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:26.898 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2058 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:26.953 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:26.954 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:26.954 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8792e-05 s, 0.0 kB/s 2023-10-10T20:38:26.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:26.955 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2059 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:27.010 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:27.010 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0204e-05 s, 0.0 kB/s 2023-10-10T20:38:27.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:27.011 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2060 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:27.068 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:27.068 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:27.068 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8919e-05 s, 0.0 kB/s 2023-10-10T20:38:27.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:27.069 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2061 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:27.123 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:27.123 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:27.124 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.141e-05 s, 0.0 kB/s 2023-10-10T20:38:27.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:27.125 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2062 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:27.179 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:27.179 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:27.180 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2857e-05 s, 0.0 kB/s 2023-10-10T20:38:27.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:27.180 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2063 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:27.236 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0886e-05 s, 0.0 kB/s 2023-10-10T20:38:27.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:27.236 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2064 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:27.291 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:27.291 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:27.292 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9657e-05 s, 0.0 kB/s 2023-10-10T20:38:27.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:27.293 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2065 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.97e-05 s, 0.0 kB/s 2023-10-10T20:38:27.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:27.349 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2066 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:27.403 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:27.403 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:27.404 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8529e-05 s, 0.0 kB/s 2023-10-10T20:38:27.404 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:27.404 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2067 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:27.458 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:27.458 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:27.458 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8264e-05 s, 0.0 kB/s 2023-10-10T20:38:27.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:27.459 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2068 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:27.514 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:27.514 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:27.514 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.153e-05 s, 0.0 kB/s 2023-10-10T20:38:27.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:27.515 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2069 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:27.569 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:27.570 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:27.570 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2368e-05 s, 0.0 kB/s 2023-10-10T20:38:27.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:27.571 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2070 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:27.625 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:27.626 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:27.626 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8302e-05 s, 0.0 kB/s 2023-10-10T20:38:27.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:27.627 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2071 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:27.681 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:27.681 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:27.681 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0396e-05 s, 0.0 kB/s 2023-10-10T20:38:27.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:27.682 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2072 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:27.736 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:27.737 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:27.737 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8703e-05 s, 0.0 kB/s 2023-10-10T20:38:27.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:27.738 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2073 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:27.792 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:27.793 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:27.793 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.779e-05 s, 0.0 kB/s 2023-10-10T20:38:27.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:27.794 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2074 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:27.848 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:27.848 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:27.849 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1658e-05 s, 0.0 kB/s 2023-10-10T20:38:27.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:27.850 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2075 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:27.904 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:27.904 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:27.904 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0317e-05 s, 0.0 kB/s 2023-10-10T20:38:27.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:27.905 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2076 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:27.959 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:27.960 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:27.960 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1332e-05 s, 0.0 kB/s 2023-10-10T20:38:27.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:27.961 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2077 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:28.016 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:28.016 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:28.016 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6615e-05 s, 0.0 kB/s 2023-10-10T20:38:28.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:28.017 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2078 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:28.071 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:28.071 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:28.072 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2271e-05 s, 0.0 kB/s 2023-10-10T20:38:28.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:28.072 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2079 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8731e-05 s, 0.0 kB/s 2023-10-10T20:38:28.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:28.129 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2080 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:28.183 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:28.183 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:28.184 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9515e-05 s, 0.0 kB/s 2023-10-10T20:38:28.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:28.184 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2081 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:28.239 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:28.239 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:28.240 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.228e-05 s, 0.0 kB/s 2023-10-10T20:38:28.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:28.240 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2082 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:28.295 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:28.295 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:28.296 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1389e-05 s, 0.0 kB/s 2023-10-10T20:38:28.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:28.296 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2083 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:28.351 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9248e-05 s, 0.0 kB/s 2023-10-10T20:38:28.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:28.353 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2084 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:28.407 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:28.408 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:28.408 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2684e-05 s, 0.0 kB/s 2023-10-10T20:38:28.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:28.409 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2085 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:28.463 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:28.463 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:28.464 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9351e-05 s, 0.0 kB/s 2023-10-10T20:38:28.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:28.464 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2086 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:28.526 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:28.526 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:28.526 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9361e-05 s, 0.0 kB/s 2023-10-10T20:38:28.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:28.527 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2087 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:28.581 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:28.581 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:28.582 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.988e-05 s, 0.0 kB/s 2023-10-10T20:38:28.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:28.583 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2088 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:28.637 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:28.638 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:28.638 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9051e-05 s, 0.0 kB/s 2023-10-10T20:38:28.638 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:28.639 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2089 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:28.693 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:28.693 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:28.694 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8605e-05 s, 0.0 kB/s 2023-10-10T20:38:28.694 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:28.694 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2090 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:28.749 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:28.749 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:28.750 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0683e-05 s, 0.0 kB/s 2023-10-10T20:38:28.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:28.750 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2091 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:28.805 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:28.806 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:28.806 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.044e-05 s, 0.0 kB/s 2023-10-10T20:38:28.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:28.807 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2092 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:28.862 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:28.862 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:28.862 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0491e-05 s, 0.0 kB/s 2023-10-10T20:38:28.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:28.863 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2093 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:28.918 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:28.918 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:28.918 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8408e-05 s, 0.0 kB/s 2023-10-10T20:38:28.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:28.919 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2094 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:28.973 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:28.973 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:28.973 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0008e-05 s, 0.0 kB/s 2023-10-10T20:38:28.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:28.974 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2095 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:29.028 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:29.028 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:29.029 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2269e-05 s, 0.0 kB/s 2023-10-10T20:38:29.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:29.029 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2096 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:29.084 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:29.085 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:29.085 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3282e-05 s, 0.0 kB/s 2023-10-10T20:38:29.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:29.086 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2097 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:29.140 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:29.140 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:29.140 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4104e-05 s, 0.0 kB/s 2023-10-10T20:38:29.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:29.142 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2098 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:29.197 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:29.198 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:29.198 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1669e-05 s, 0.0 kB/s 2023-10-10T20:38:29.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:29.199 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2099 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:29.254 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:29.254 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:29.255 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.916e-05 s, 0.0 kB/s 2023-10-10T20:38:29.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:29.255 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2100 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:29.310 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:29.310 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:29.311 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1248e-05 s, 0.0 kB/s 2023-10-10T20:38:29.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:29.311 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2101 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:29.367 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:29.367 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:29.368 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4546e-05 s, 0.0 kB/s 2023-10-10T20:38:29.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:29.369 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2102 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:29.423 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:29.423 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:29.423 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0934e-05 s, 0.0 kB/s 2023-10-10T20:38:29.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:29.424 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2103 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8799e-05 s, 0.0 kB/s 2023-10-10T20:38:29.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:29.480 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2104 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:29.535 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:29.535 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:29.536 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3341e-05 s, 0.0 kB/s 2023-10-10T20:38:29.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:29.536 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2105 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:29.590 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:29.591 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:29.591 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3158e-05 s, 0.0 kB/s 2023-10-10T20:38:29.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:29.592 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2106 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:29.646 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:29.647 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:29.647 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8787e-05 s, 0.0 kB/s 2023-10-10T20:38:29.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:29.648 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2107 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:29.702 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:29.703 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:29.703 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9546e-05 s, 0.0 kB/s 2023-10-10T20:38:29.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:29.704 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2108 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:29.758 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:29.758 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:29.759 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8646e-05 s, 0.0 kB/s 2023-10-10T20:38:29.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:29.759 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2109 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:29.814 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:29.815 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:29.815 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0053e-05 s, 0.0 kB/s 2023-10-10T20:38:29.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:29.816 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2110 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:29.870 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:29.870 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:29.871 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0751e-05 s, 0.0 kB/s 2023-10-10T20:38:29.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:29.871 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2111 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:29.927 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:29.927 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:29.927 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2607e-05 s, 0.0 kB/s 2023-10-10T20:38:29.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:29.928 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2112 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:29.983 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8996e-05 s, 0.0 kB/s 2023-10-10T20:38:29.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:29.983 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2113 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:30.038 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:30.039 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:30.039 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1328e-05 s, 0.0 kB/s 2023-10-10T20:38:30.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:30.040 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2114 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:30.095 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:30.096 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:30.096 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0873e-05 s, 0.0 kB/s 2023-10-10T20:38:30.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:30.096 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2115 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:30.152 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:30.152 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:30.152 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8552e-05 s, 0.0 kB/s 2023-10-10T20:38:30.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:30.153 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2116 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:30.207 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:30.208 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:30.208 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8931e-05 s, 0.0 kB/s 2023-10-10T20:38:30.208 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:30.209 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2117 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:30.263 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:30.264 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:30.264 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.695e-05 s, 0.0 kB/s 2023-10-10T20:38:30.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:30.265 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2118 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:30.319 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:30.320 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:30.320 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8732e-05 s, 0.0 kB/s 2023-10-10T20:38:30.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:30.321 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2119 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:30.376 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:30.377 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:30.377 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7698e-05 s, 0.0 kB/s 2023-10-10T20:38:30.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:30.378 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2120 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:30.432 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:30.433 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:30.433 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7697e-05 s, 0.0 kB/s 2023-10-10T20:38:30.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:30.434 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2121 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:30.488 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:30.489 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:30.489 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.023e-05 s, 0.0 kB/s 2023-10-10T20:38:30.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:30.490 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2122 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:30.546 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:30.546 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:30.546 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9706e-05 s, 0.0 kB/s 2023-10-10T20:38:30.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:30.547 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2123 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:30.601 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:30.602 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:30.602 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2414e-05 s, 0.0 kB/s 2023-10-10T20:38:30.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:30.603 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2124 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:30.657 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:30.657 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:30.657 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8207e-05 s, 0.0 kB/s 2023-10-10T20:38:30.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:30.658 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2125 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:30.712 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:30.712 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:30.713 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8952e-05 s, 0.0 kB/s 2023-10-10T20:38:30.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:30.713 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2126 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:30.768 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:30.769 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:30.769 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8218e-05 s, 0.0 kB/s 2023-10-10T20:38:30.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:30.770 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2127 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:30.824 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:30.824 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:30.825 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3265e-05 s, 0.0 kB/s 2023-10-10T20:38:30.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:30.825 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2128 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:30.879 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:30.880 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:30.880 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0468e-05 s, 0.0 kB/s 2023-10-10T20:38:30.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:30.881 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2129 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:30.935 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:30.936 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:30.936 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0362e-05 s, 0.0 kB/s 2023-10-10T20:38:30.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:30.937 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2130 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:30.991 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:30.992 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:30.992 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1566e-05 s, 0.0 kB/s 2023-10-10T20:38:30.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:30.993 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2131 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:31.047 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:31.048 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:31.048 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0301e-05 s, 0.0 kB/s 2023-10-10T20:38:31.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:31.049 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2132 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:31.104 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:31.104 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:31.104 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0609e-05 s, 0.0 kB/s 2023-10-10T20:38:31.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:31.106 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2133 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:31.162 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:31.162 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:31.163 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2071e-05 s, 0.0 kB/s 2023-10-10T20:38:31.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:31.163 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2134 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:31.218 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:31.219 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:31.219 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1413e-05 s, 0.0 kB/s 2023-10-10T20:38:31.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:31.220 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2135 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:31.275 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:31.275 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:31.275 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0968e-05 s, 0.0 kB/s 2023-10-10T20:38:31.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:31.276 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2136 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:31.332 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.922e-05 s, 0.0 kB/s 2023-10-10T20:38:31.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:31.333 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2137 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:31.386 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:31.387 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:31.387 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7731e-05 s, 0.0 kB/s 2023-10-10T20:38:31.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:31.388 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2138 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:31.443 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:31.443 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:31.444 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.972e-05 s, 0.0 kB/s 2023-10-10T20:38:31.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:31.445 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2139 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:31.500 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:31.500 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:31.500 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9663e-05 s, 0.0 kB/s 2023-10-10T20:38:31.501 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:31.501 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2140 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:31.556 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:31.556 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:31.556 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1902e-05 s, 0.0 kB/s 2023-10-10T20:38:31.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:31.557 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2141 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:31.611 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:31.612 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:31.612 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3238e-05 s, 0.0 kB/s 2023-10-10T20:38:31.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:31.613 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2142 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:31.667 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:31.668 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:31.668 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6466e-05 s, 0.0 kB/s 2023-10-10T20:38:31.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:31.669 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2143 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:31.723 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:31.724 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:31.724 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0823e-05 s, 0.0 kB/s 2023-10-10T20:38:31.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:31.725 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2144 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:31.779 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:31.779 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:31.781 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8955e-05 s, 0.0 kB/s 2023-10-10T20:38:31.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:31.782 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2145 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:31.834 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:31.835 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:31.835 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.4882e-05 s, 0.0 kB/s 2023-10-10T20:38:31.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:31.836 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2146 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:31.890 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:31.890 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:31.891 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4773e-05 s, 0.0 kB/s 2023-10-10T20:38:31.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:31.891 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2147 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:31.946 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:31.946 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:31.947 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2725e-05 s, 0.0 kB/s 2023-10-10T20:38:31.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:31.947 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2148 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:32.001 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:32.002 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:32.002 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9895e-05 s, 0.0 kB/s 2023-10-10T20:38:32.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:32.003 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2149 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:32.057 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:32.057 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:32.058 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4116e-05 s, 0.0 kB/s 2023-10-10T20:38:32.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:32.058 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2150 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:32.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:32.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:32.112 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2785e-05 s, 0.0 kB/s 2023-10-10T20:38:32.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:32.113 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2151 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:32.167 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:32.167 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:32.168 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8629e-05 s, 0.0 kB/s 2023-10-10T20:38:32.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:32.168 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2152 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7563e-05 s, 0.0 kB/s 2023-10-10T20:38:32.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:32.225 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2153 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:32.279 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:32.279 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:32.280 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0224e-05 s, 0.0 kB/s 2023-10-10T20:38:32.280 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:32.280 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2154 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:32.335 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:32.335 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:32.336 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.753e-05 s, 0.0 kB/s 2023-10-10T20:38:32.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:32.337 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2155 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0366e-05 s, 0.0 kB/s 2023-10-10T20:38:32.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:32.393 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2156 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:32.449 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:32.449 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:32.449 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8542e-05 s, 0.0 kB/s 2023-10-10T20:38:32.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:32.450 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2157 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:32.505 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:32.505 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:32.505 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0424e-05 s, 0.0 kB/s 2023-10-10T20:38:32.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:32.506 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2158 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:32.560 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:32.560 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:32.561 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6839e-05 s, 0.0 kB/s 2023-10-10T20:38:32.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:32.561 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2159 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:32.616 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:32.617 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:32.617 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9201e-05 s, 0.0 kB/s 2023-10-10T20:38:32.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:32.618 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2160 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:32.672 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:32.672 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:32.673 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9586e-05 s, 0.0 kB/s 2023-10-10T20:38:32.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:32.673 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2161 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:32.728 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:32.728 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:32.728 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1821e-05 s, 0.0 kB/s 2023-10-10T20:38:32.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:32.729 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2162 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1325e-05 s, 0.0 kB/s 2023-10-10T20:38:32.784 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:32.784 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2163 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:32.839 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0205e-05 s, 0.0 kB/s 2023-10-10T20:38:32.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:32.841 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2164 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:32.895 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:32.896 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:32.896 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0764e-05 s, 0.0 kB/s 2023-10-10T20:38:32.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:32.897 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2165 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:32.952 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:32.952 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:32.952 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1321e-05 s, 0.0 kB/s 2023-10-10T20:38:32.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:32.953 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2166 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:33.009 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:33.010 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:33.010 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8304e-05 s, 0.0 kB/s 2023-10-10T20:38:33.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:33.011 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2167 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:33.065 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:33.065 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:33.066 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.7615e-05 s, 0.0 kB/s 2023-10-10T20:38:33.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:33.066 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2168 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:33.121 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:33.122 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:33.122 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9265e-05 s, 0.0 kB/s 2023-10-10T20:38:33.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:33.123 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2169 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:33.178 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:33.179 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:33.179 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0802e-05 s, 0.0 kB/s 2023-10-10T20:38:33.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:33.180 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2170 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:33.235 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:33.235 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:33.235 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0996e-05 s, 0.0 kB/s 2023-10-10T20:38:33.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:33.236 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2171 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6808e-05 s, 0.0 kB/s 2023-10-10T20:38:33.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:33.292 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2172 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:33.347 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:33.347 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:33.347 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.789e-05 s, 0.0 kB/s 2023-10-10T20:38:33.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:33.348 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2173 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:33.403 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:33.403 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:33.403 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.7692e-05 s, 0.0 kB/s 2023-10-10T20:38:33.404 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:33.404 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2174 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:33.459 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:33.459 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:33.459 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9848e-05 s, 0.0 kB/s 2023-10-10T20:38:33.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:33.460 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2175 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:33.515 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:33.515 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:33.515 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9993e-05 s, 0.0 kB/s 2023-10-10T20:38:33.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:33.516 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2176 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:33.570 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:33.571 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:33.571 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8509e-05 s, 0.0 kB/s 2023-10-10T20:38:33.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:33.572 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2177 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:33.626 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:33.627 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:33.627 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9369e-05 s, 0.0 kB/s 2023-10-10T20:38:33.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:33.629 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2178 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:33.683 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:33.683 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:33.683 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0109e-05 s, 0.0 kB/s 2023-10-10T20:38:33.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:33.684 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2179 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:33.738 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:33.739 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:33.739 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0366e-05 s, 0.0 kB/s 2023-10-10T20:38:33.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:33.740 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2180 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:33.794 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:33.795 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:33.795 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9273e-05 s, 0.0 kB/s 2023-10-10T20:38:33.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:33.796 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2181 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:33.850 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:33.850 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:33.851 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3789e-05 s, 0.0 kB/s 2023-10-10T20:38:33.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:33.851 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2182 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:33.905 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:33.906 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:33.906 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8567e-05 s, 0.0 kB/s 2023-10-10T20:38:33.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:33.907 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2183 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:33.961 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:33.962 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:33.962 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9064e-05 s, 0.0 kB/s 2023-10-10T20:38:33.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:33.963 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2184 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:34.017 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:34.017 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:34.018 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2757e-05 s, 0.0 kB/s 2023-10-10T20:38:34.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:34.019 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2185 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:34.073 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:34.073 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:34.074 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0275e-05 s, 0.0 kB/s 2023-10-10T20:38:34.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:34.075 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2186 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:34.130 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:34.130 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:34.130 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1982e-05 s, 0.0 kB/s 2023-10-10T20:38:34.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:34.131 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2187 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:34.185 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:34.186 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:34.186 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8645e-05 s, 0.0 kB/s 2023-10-10T20:38:34.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:34.187 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2188 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:34.241 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:34.242 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:34.242 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2063e-05 s, 0.0 kB/s 2023-10-10T20:38:34.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:34.243 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2189 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:34.297 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:34.298 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:34.298 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9002e-05 s, 0.0 kB/s 2023-10-10T20:38:34.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:34.299 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2190 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:34.354 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:34.354 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:34.354 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3994e-05 s, 0.0 kB/s 2023-10-10T20:38:34.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:34.355 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2191 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:34.410 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:34.410 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:34.410 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0671e-05 s, 0.0 kB/s 2023-10-10T20:38:34.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:34.411 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2192 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:34.465 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:34.466 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:34.466 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.756e-05 s, 0.0 kB/s 2023-10-10T20:38:34.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:34.467 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2193 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:34.521 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:34.521 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:34.522 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7605e-05 s, 0.0 kB/s 2023-10-10T20:38:34.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:34.522 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2194 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:34.577 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:34.577 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:34.577 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0123e-05 s, 0.0 kB/s 2023-10-10T20:38:34.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:34.578 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2195 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:34.672 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:34.673 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:34.673 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 5.5556e-05 s, 0.0 kB/s 2023-10-10T20:38:34.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:34.674 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2196 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:34.689 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:34.689 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:34.689 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4e-05 s, 0.0 kB/s 2023-10-10T20:38:34.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:34.690 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2197 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:34.744 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:34.745 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:34.745 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2416e-05 s, 0.0 kB/s 2023-10-10T20:38:34.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:34.746 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2198 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:34.800 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:34.801 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:34.801 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0152e-05 s, 0.0 kB/s 2023-10-10T20:38:34.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:34.802 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2199 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:34.857 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:34.857 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:34.857 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1158e-05 s, 0.0 kB/s 2023-10-10T20:38:34.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:34.858 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2200 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:34.912 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:34.913 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:34.913 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9838e-05 s, 0.0 kB/s 2023-10-10T20:38:34.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:34.914 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2201 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:34.968 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:34.968 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:34.969 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9651e-05 s, 0.0 kB/s 2023-10-10T20:38:34.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:34.969 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2202 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:35.024 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:35.025 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:35.025 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1147e-05 s, 0.0 kB/s 2023-10-10T20:38:35.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:35.026 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2203 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:35.081 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:35.081 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:35.082 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6661e-05 s, 0.0 kB/s 2023-10-10T20:38:35.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:35.082 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2204 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:35.137 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:35.137 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:35.137 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3365e-05 s, 0.0 kB/s 2023-10-10T20:38:35.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:35.138 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2205 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:35.193 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:35.193 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:35.193 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9461e-05 s, 0.0 kB/s 2023-10-10T20:38:35.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:35.194 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2206 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:35.248 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:35.249 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:35.249 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9879e-05 s, 0.0 kB/s 2023-10-10T20:38:35.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:35.250 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2207 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:35.304 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:35.304 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:35.304 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0882e-05 s, 0.0 kB/s 2023-10-10T20:38:35.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:35.305 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2208 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:35.359 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:35.360 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:35.360 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9132e-05 s, 0.0 kB/s 2023-10-10T20:38:35.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:35.361 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2209 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:35.416 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:35.416 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:35.416 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2717e-05 s, 0.0 kB/s 2023-10-10T20:38:35.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:35.417 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2210 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:35.472 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:35.472 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0478e-05 s, 0.0 kB/s 2023-10-10T20:38:35.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:35.473 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2211 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:35.527 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:35.527 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:35.528 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.2832e-05 s, 0.0 kB/s 2023-10-10T20:38:35.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:35.529 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2212 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:35.582 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:35.583 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:35.583 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8366e-05 s, 0.0 kB/s 2023-10-10T20:38:35.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:35.584 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2213 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:35.638 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:35.639 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:35.639 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1175e-05 s, 0.0 kB/s 2023-10-10T20:38:35.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:35.639 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2214 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:35.696 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:35.696 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:35.697 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9838e-05 s, 0.0 kB/s 2023-10-10T20:38:35.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:35.697 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2215 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:35.751 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:35.751 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:35.751 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1981e-05 s, 0.0 kB/s 2023-10-10T20:38:35.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:35.752 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2216 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7517e-05 s, 0.0 kB/s 2023-10-10T20:38:35.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:35.808 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2217 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:35.862 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:35.862 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:35.863 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0899e-05 s, 0.0 kB/s 2023-10-10T20:38:35.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:35.863 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2218 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:35.918 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:35.919 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:35.919 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0993e-05 s, 0.0 kB/s 2023-10-10T20:38:35.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:35.920 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2219 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:35.974 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:35.975 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:35.975 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9025e-05 s, 0.0 kB/s 2023-10-10T20:38:35.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:35.975 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2220 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:36.030 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:36.030 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:36.030 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2748e-05 s, 0.0 kB/s 2023-10-10T20:38:36.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:36.031 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2221 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:36.085 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:36.085 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:36.086 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9741e-05 s, 0.0 kB/s 2023-10-10T20:38:36.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:36.087 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2222 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:36.141 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:36.142 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:36.142 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3113e-05 s, 0.0 kB/s 2023-10-10T20:38:36.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:36.143 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2223 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:36.197 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:36.197 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:36.198 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9856e-05 s, 0.0 kB/s 2023-10-10T20:38:36.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:36.198 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2224 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:36.253 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:36.253 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:36.254 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.0194e-05 s, 0.0 kB/s 2023-10-10T20:38:36.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:36.254 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2225 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:36.312 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:36.312 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:36.313 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8122e-05 s, 0.0 kB/s 2023-10-10T20:38:36.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:36.313 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2226 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:36.369 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7376e-05 s, 0.0 kB/s 2023-10-10T20:38:36.369 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:36.369 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2227 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:36.426 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:36.426 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:36.427 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1942e-05 s, 0.0 kB/s 2023-10-10T20:38:36.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:36.427 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2228 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:36.485 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:36.485 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:36.485 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2249e-05 s, 0.0 kB/s 2023-10-10T20:38:36.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:36.486 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2229 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:36.551 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:36.551 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:36.551 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8775e-05 s, 0.0 kB/s 2023-10-10T20:38:36.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:36.555 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2230 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:36.611 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:36.612 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:36.612 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0547e-05 s, 0.0 kB/s 2023-10-10T20:38:36.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:36.613 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2231 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:36.668 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:36.668 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:36.668 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1954e-05 s, 0.0 kB/s 2023-10-10T20:38:36.669 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:36.669 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2232 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:36.728 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:36.728 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:36.729 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0046e-05 s, 0.0 kB/s 2023-10-10T20:38:36.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:36.729 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2233 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:36.789 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:36.789 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:36.789 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0884e-05 s, 0.0 kB/s 2023-10-10T20:38:36.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:36.790 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2234 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:36.845 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:36.845 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:36.845 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0808e-05 s, 0.0 kB/s 2023-10-10T20:38:36.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:36.846 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2235 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:36.903 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:36.903 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:36.903 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1918e-05 s, 0.0 kB/s 2023-10-10T20:38:36.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:36.904 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2236 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:36.963 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:36.963 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:36.963 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8942e-05 s, 0.0 kB/s 2023-10-10T20:38:36.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:36.964 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2237 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:37.018 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:37.019 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:37.019 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8581e-05 s, 0.0 kB/s 2023-10-10T20:38:37.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:37.020 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2238 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:37.074 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:37.075 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:37.075 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7439e-05 s, 0.0 kB/s 2023-10-10T20:38:37.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:37.076 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2239 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:37.131 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:37.131 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:37.131 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.972e-05 s, 0.0 kB/s 2023-10-10T20:38:37.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:37.132 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2240 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:37.188 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:37.189 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:37.189 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2033e-05 s, 0.0 kB/s 2023-10-10T20:38:37.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:37.191 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2241 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:37.245 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:37.245 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:37.246 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0661e-05 s, 0.0 kB/s 2023-10-10T20:38:37.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:37.247 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2242 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:37.301 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:37.301 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:37.301 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4064e-05 s, 0.0 kB/s 2023-10-10T20:38:37.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:37.302 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2243 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:37.356 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:37.356 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:37.357 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9595e-05 s, 0.0 kB/s 2023-10-10T20:38:37.358 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:37.358 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2244 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:37.413 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:37.413 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:37.414 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0081e-05 s, 0.0 kB/s 2023-10-10T20:38:37.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:37.415 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2245 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:37.469 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:37.470 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:37.470 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8436e-05 s, 0.0 kB/s 2023-10-10T20:38:37.470 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:37.471 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2246 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:37.526 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:37.526 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:37.526 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1708e-05 s, 0.0 kB/s 2023-10-10T20:38:37.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:37.527 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2247 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:37.581 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:37.582 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:37.582 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0486e-05 s, 0.0 kB/s 2023-10-10T20:38:37.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:37.583 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2248 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:37.637 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:37.637 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:37.638 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8589e-05 s, 0.0 kB/s 2023-10-10T20:38:37.638 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:37.638 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2249 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:37.693 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:37.693 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:37.694 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9342e-05 s, 0.0 kB/s 2023-10-10T20:38:37.694 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:37.695 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2250 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:37.749 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:37.749 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:37.750 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2044e-05 s, 0.0 kB/s 2023-10-10T20:38:37.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:37.750 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2251 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:37.804 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:37.805 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:37.805 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9416e-05 s, 0.0 kB/s 2023-10-10T20:38:37.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:37.806 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2252 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:37.861 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:37.861 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:37.861 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.723e-05 s, 0.0 kB/s 2023-10-10T20:38:37.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:37.862 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2253 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:37.916 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:37.916 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:37.916 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3094e-05 s, 0.0 kB/s 2023-10-10T20:38:37.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:37.917 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2254 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:37.971 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:37.972 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:37.972 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1367e-05 s, 0.0 kB/s 2023-10-10T20:38:37.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:37.973 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2255 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:38.027 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:38.028 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:38.028 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0634e-05 s, 0.0 kB/s 2023-10-10T20:38:38.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:38.029 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2256 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:38.083 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:38.083 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:38.084 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0063e-05 s, 0.0 kB/s 2023-10-10T20:38:38.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:38.084 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2257 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:38.140 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:38.140 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:38.140 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1626e-05 s, 0.0 kB/s 2023-10-10T20:38:38.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:38.141 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2258 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:38.195 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:38.196 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:38.196 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8019e-05 s, 0.0 kB/s 2023-10-10T20:38:38.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:38.197 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2259 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0869e-05 s, 0.0 kB/s 2023-10-10T20:38:38.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:38.253 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2260 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:38.307 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:38.308 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:38.308 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9618e-05 s, 0.0 kB/s 2023-10-10T20:38:38.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:38.309 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2261 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:38.364 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:38.364 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:38.364 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4619e-05 s, 0.0 kB/s 2023-10-10T20:38:38.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:38.365 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2262 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:38.419 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:38.419 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:38.419 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0569e-05 s, 0.0 kB/s 2023-10-10T20:38:38.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:38.420 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2263 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:38.475 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:38.475 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:38.475 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.11e-05 s, 0.0 kB/s 2023-10-10T20:38:38.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:38.476 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2264 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:38.530 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:38.531 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:38.531 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0244e-05 s, 0.0 kB/s 2023-10-10T20:38:38.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:38.532 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2265 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:38.587 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:38.587 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:38.587 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2031e-05 s, 0.0 kB/s 2023-10-10T20:38:38.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:38.588 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2266 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:38.642 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:38.642 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:38.643 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9495e-05 s, 0.0 kB/s 2023-10-10T20:38:38.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:38.643 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2267 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:38.699 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:38.699 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:38.699 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0056e-05 s, 0.0 kB/s 2023-10-10T20:38:38.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:38.700 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2268 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:38.754 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:38.755 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:38.755 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0184e-05 s, 0.0 kB/s 2023-10-10T20:38:38.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:38.756 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2269 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:38.810 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:38.810 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:38.811 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8049e-05 s, 0.0 kB/s 2023-10-10T20:38:38.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:38.812 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2270 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:38.866 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:38.866 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:38.867 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7905e-05 s, 0.0 kB/s 2023-10-10T20:38:38.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:38.867 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2271 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:38.923 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:38.923 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:38.923 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6909e-05 s, 0.0 kB/s 2023-10-10T20:38:38.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:38.924 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2272 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:38.978 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:38.978 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:38.979 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.386e-05 s, 0.0 kB/s 2023-10-10T20:38:38.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:38.979 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2273 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:39.034 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:39.035 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:39.035 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2355e-05 s, 0.0 kB/s 2023-10-10T20:38:39.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:39.036 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2274 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:39.090 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:39.091 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:39.091 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8534e-05 s, 0.0 kB/s 2023-10-10T20:38:39.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:39.092 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2275 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:39.146 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:39.147 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:39.147 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1936e-05 s, 0.0 kB/s 2023-10-10T20:38:39.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:39.148 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2276 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:39.203 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:39.203 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:39.203 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1411e-05 s, 0.0 kB/s 2023-10-10T20:38:39.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:39.204 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2277 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:39.258 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:39.258 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:39.259 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1137e-05 s, 0.0 kB/s 2023-10-10T20:38:39.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:39.260 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2278 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:39.316 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:39.316 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:39.316 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1239e-05 s, 0.0 kB/s 2023-10-10T20:38:39.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:39.317 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2279 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:39.372 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:39.372 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:39.372 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1106e-05 s, 0.0 kB/s 2023-10-10T20:38:39.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:39.373 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2280 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1111e-05 s, 0.0 kB/s 2023-10-10T20:38:39.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:39.429 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2281 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:39.484 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:39.484 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:39.485 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0536e-05 s, 0.0 kB/s 2023-10-10T20:38:39.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:39.485 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2282 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:39.539 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:39.539 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:39.540 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0805e-05 s, 0.0 kB/s 2023-10-10T20:38:39.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:39.540 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2283 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:39.595 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:39.595 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:39.595 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0617e-05 s, 0.0 kB/s 2023-10-10T20:38:39.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:39.597 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2284 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:39.652 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:39.653 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:39.653 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1387e-05 s, 0.0 kB/s 2023-10-10T20:38:39.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:39.654 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2285 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:39.709 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:39.709 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:39.709 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1091e-05 s, 0.0 kB/s 2023-10-10T20:38:39.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:39.710 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2286 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:39.765 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:39.765 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:39.765 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.256e-05 s, 0.0 kB/s 2023-10-10T20:38:39.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:39.766 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2287 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0804e-05 s, 0.0 kB/s 2023-10-10T20:38:39.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:39.822 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2288 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:39.877 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:39.877 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:39.878 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.202e-05 s, 0.0 kB/s 2023-10-10T20:38:39.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:39.879 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2289 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:39.933 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:39.933 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:39.933 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.087e-05 s, 0.0 kB/s 2023-10-10T20:38:39.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:39.934 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2290 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:39.988 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:39.989 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:39.989 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1505e-05 s, 0.0 kB/s 2023-10-10T20:38:39.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:39.990 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2291 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:40.045 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:40.045 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:40.046 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1576e-05 s, 0.0 kB/s 2023-10-10T20:38:40.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:40.046 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2292 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:40.102 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:40.102 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:40.103 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1279e-05 s, 0.0 kB/s 2023-10-10T20:38:40.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:40.104 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2293 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:40.158 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:40.159 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:40.159 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9806e-05 s, 0.0 kB/s 2023-10-10T20:38:40.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:40.160 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2294 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:40.215 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:40.215 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1241e-05 s, 0.0 kB/s 2023-10-10T20:38:40.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:40.216 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2295 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:40.335 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:40.336 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:40.336 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.6848e-05 s, 0.0 kB/s 2023-10-10T20:38:40.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:40.336 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2296 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:40.353 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:40.353 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:40.354 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1794e-05 s, 0.0 kB/s 2023-10-10T20:38:40.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:40.355 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2297 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:40.409 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:40.409 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:40.410 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0135e-05 s, 0.0 kB/s 2023-10-10T20:38:40.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:40.411 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2298 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:40.465 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:40.466 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:40.466 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1885e-05 s, 0.0 kB/s 2023-10-10T20:38:40.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:40.467 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2299 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:40.522 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:40.523 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:40.523 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9628e-05 s, 0.0 kB/s 2023-10-10T20:38:40.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:40.524 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2300 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:40.578 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:40.578 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:40.579 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3754e-05 s, 0.0 kB/s 2023-10-10T20:38:40.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:40.579 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2301 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:40.634 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:40.634 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:40.635 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0149e-05 s, 0.0 kB/s 2023-10-10T20:38:40.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:40.635 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2302 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:40.690 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:40.691 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:40.691 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3522e-05 s, 0.0 kB/s 2023-10-10T20:38:40.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:40.692 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2303 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:40.746 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:40.746 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:40.747 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.076e-05 s, 0.0 kB/s 2023-10-10T20:38:40.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:40.748 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2304 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:40.806 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:40.806 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:40.806 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2581e-05 s, 0.0 kB/s 2023-10-10T20:38:40.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:40.808 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2305 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:40.859 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:40.859 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:40.859 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9202e-05 s, 0.0 kB/s 2023-10-10T20:38:40.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:40.860 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2306 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:40.916 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:40.916 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:40.917 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1942e-05 s, 0.0 kB/s 2023-10-10T20:38:40.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:40.917 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2307 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:40.972 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:40.972 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:40.973 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.031e-05 s, 0.0 kB/s 2023-10-10T20:38:40.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:40.973 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2308 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:41.029 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:41.029 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:41.029 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1469e-05 s, 0.0 kB/s 2023-10-10T20:38:41.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:41.030 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2309 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:41.084 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:41.085 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:41.085 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2322e-05 s, 0.0 kB/s 2023-10-10T20:38:41.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:41.086 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2310 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:41.141 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:41.141 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:41.142 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3666e-05 s, 0.0 kB/s 2023-10-10T20:38:41.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:41.142 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2311 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1347e-05 s, 0.0 kB/s 2023-10-10T20:38:41.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:41.198 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2312 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:41.253 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:41.253 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:41.253 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2432e-05 s, 0.0 kB/s 2023-10-10T20:38:41.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:41.255 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2313 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:41.309 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:41.309 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:41.310 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1027e-05 s, 0.0 kB/s 2023-10-10T20:38:41.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:41.311 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2314 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:41.364 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:41.365 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:41.365 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1932e-05 s, 0.0 kB/s 2023-10-10T20:38:41.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:41.366 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2315 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:41.422 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1029e-05 s, 0.0 kB/s 2023-10-10T20:38:41.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:41.424 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2316 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:41.479 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:41.479 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:41.479 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1273e-05 s, 0.0 kB/s 2023-10-10T20:38:41.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:41.480 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2317 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:41.534 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:41.535 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:41.535 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.388e-05 s, 0.0 kB/s 2023-10-10T20:38:41.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:41.537 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2318 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:41.591 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:41.591 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:41.592 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3072e-05 s, 0.0 kB/s 2023-10-10T20:38:41.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:41.593 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2319 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:41.646 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:41.647 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:41.647 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9032e-05 s, 0.0 kB/s 2023-10-10T20:38:41.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:41.648 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2320 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:41.702 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:41.702 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9675e-05 s, 0.0 kB/s 2023-10-10T20:38:41.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:41.704 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2321 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:41.758 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:41.759 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:41.759 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0034e-05 s, 0.0 kB/s 2023-10-10T20:38:41.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:41.760 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2322 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:41.814 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:41.815 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:41.815 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8909e-05 s, 0.0 kB/s 2023-10-10T20:38:41.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:41.816 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2323 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:41.870 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:41.870 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:41.870 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8361e-05 s, 0.0 kB/s 2023-10-10T20:38:41.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:41.871 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2324 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:41.926 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:41.926 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1163e-05 s, 0.0 kB/s 2023-10-10T20:38:41.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:41.927 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2325 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:41.982 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:41.982 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0224e-05 s, 0.0 kB/s 2023-10-10T20:38:41.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:41.983 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2326 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:42.038 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:42.038 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:42.038 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7337e-05 s, 0.0 kB/s 2023-10-10T20:38:42.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:42.040 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2327 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:42.093 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:42.094 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:42.094 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9631e-05 s, 0.0 kB/s 2023-10-10T20:38:42.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:42.096 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2328 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:42.150 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:42.151 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:42.151 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8641e-05 s, 0.0 kB/s 2023-10-10T20:38:42.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:42.152 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2329 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:42.207 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:42.207 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:42.208 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1595e-05 s, 0.0 kB/s 2023-10-10T20:38:42.208 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:42.208 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2330 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:42.227 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:38:42.251 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:38:42.263 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:42.264 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:42.264 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0623e-05 s, 0.0 kB/s 2023-10-10T20:38:42.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:42.265 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2331 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:42.319 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:42.320 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:42.320 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4725e-05 s, 0.0 kB/s 2023-10-10T20:38:42.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:42.321 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2332 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:42.395 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:42.396 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:42.396 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2949e-05 s, 0.0 kB/s 2023-10-10T20:38:42.397 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:42.397 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2333 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:42.451 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:42.452 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:42.452 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7009e-05 s, 0.0 kB/s 2023-10-10T20:38:42.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:42.454 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2334 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:42.507 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:42.508 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:42.508 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1041e-05 s, 0.0 kB/s 2023-10-10T20:38:42.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:42.509 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2335 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:42.564 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:42.564 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:42.564 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.044e-05 s, 0.0 kB/s 2023-10-10T20:38:42.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:42.565 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2336 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:42.619 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:42.620 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:42.620 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9806e-05 s, 0.0 kB/s 2023-10-10T20:38:42.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:42.621 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2337 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:42.675 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:42.675 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:42.675 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1032e-05 s, 0.0 kB/s 2023-10-10T20:38:42.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:42.676 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2338 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:42.731 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:42.731 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:42.731 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0653e-05 s, 0.0 kB/s 2023-10-10T20:38:42.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:42.732 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2339 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:42.786 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:42.786 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:42.787 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8826e-05 s, 0.0 kB/s 2023-10-10T20:38:42.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:42.787 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2340 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:42.842 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:42.843 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:42.843 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1296e-05 s, 0.0 kB/s 2023-10-10T20:38:42.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:42.844 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2341 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:42.898 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:42.899 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:42.899 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0891e-05 s, 0.0 kB/s 2023-10-10T20:38:42.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:42.900 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2342 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:42.955 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:42.955 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:42.956 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3e-05 s, 0.0 kB/s 2023-10-10T20:38:42.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:42.957 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2343 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:43.010 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:43.011 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:43.011 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1552e-05 s, 0.0 kB/s 2023-10-10T20:38:43.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:43.012 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2344 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:43.067 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:43.067 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:43.067 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9875e-05 s, 0.0 kB/s 2023-10-10T20:38:43.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:43.068 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2345 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:43.122 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:43.123 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:43.123 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3463e-05 s, 0.0 kB/s 2023-10-10T20:38:43.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:43.124 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2346 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:43.179 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:43.180 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:43.180 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4091e-05 s, 0.0 kB/s 2023-10-10T20:38:43.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:43.180 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2347 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:43.235 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2073e-05 s, 0.0 kB/s 2023-10-10T20:38:43.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:43.237 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2348 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:43.291 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:43.291 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:43.291 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9733e-05 s, 0.0 kB/s 2023-10-10T20:38:43.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:43.292 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2349 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:43.346 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:43.347 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:43.347 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7348e-05 s, 0.0 kB/s 2023-10-10T20:38:43.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:43.348 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2350 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:43.402 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:43.402 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:43.402 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7721e-05 s, 0.0 kB/s 2023-10-10T20:38:43.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:43.404 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2351 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:43.459 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:43.459 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:43.459 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8365e-05 s, 0.0 kB/s 2023-10-10T20:38:43.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:43.460 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2352 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:43.514 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:43.514 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:43.515 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7792e-05 s, 0.0 kB/s 2023-10-10T20:38:43.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:43.515 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2353 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:43.570 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:43.570 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.015e-05 s, 0.0 kB/s 2023-10-10T20:38:43.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:43.571 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2354 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8824e-05 s, 0.0 kB/s 2023-10-10T20:38:43.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:43.626 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2355 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:43.681 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:43.682 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:43.682 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9978e-05 s, 0.0 kB/s 2023-10-10T20:38:43.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:43.683 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2356 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:43.737 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:43.737 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:43.738 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8805e-05 s, 0.0 kB/s 2023-10-10T20:38:43.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:43.738 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2357 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:43.793 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:43.793 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:43.794 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.822e-05 s, 0.0 kB/s 2023-10-10T20:38:43.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:43.794 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2358 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:43.849 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:43.849 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:43.850 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8318e-05 s, 0.0 kB/s 2023-10-10T20:38:43.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:43.850 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2359 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:43.905 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:43.906 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:43.906 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0068e-05 s, 0.0 kB/s 2023-10-10T20:38:43.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:43.907 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2360 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:43.961 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:43.962 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:43.962 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1115e-05 s, 0.0 kB/s 2023-10-10T20:38:43.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:43.963 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2361 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:44.017 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:44.018 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:44.018 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9651e-05 s, 0.0 kB/s 2023-10-10T20:38:44.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:44.019 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2362 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:44.073 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:44.074 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:44.074 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9261e-05 s, 0.0 kB/s 2023-10-10T20:38:44.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:44.075 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2363 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:44.129 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:44.130 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:44.130 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0648e-05 s, 0.0 kB/s 2023-10-10T20:38:44.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:44.131 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2364 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:44.185 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:44.185 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:44.186 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0118e-05 s, 0.0 kB/s 2023-10-10T20:38:44.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:44.186 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2365 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:44.241 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:44.242 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:44.242 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9538e-05 s, 0.0 kB/s 2023-10-10T20:38:44.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:44.243 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2366 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:44.297 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:44.297 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:44.297 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8244e-05 s, 0.0 kB/s 2023-10-10T20:38:44.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:44.298 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2367 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:44.353 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:44.353 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:44.353 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2023e-05 s, 0.0 kB/s 2023-10-10T20:38:44.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:44.355 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2368 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:44.408 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:44.408 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:44.408 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7737e-05 s, 0.0 kB/s 2023-10-10T20:38:44.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:44.409 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2369 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:44.463 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:44.464 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:44.464 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.8856e-05 s, 0.0 kB/s 2023-10-10T20:38:44.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:44.465 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2370 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:44.520 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:44.521 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:44.521 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3489e-05 s, 0.0 kB/s 2023-10-10T20:38:44.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:44.522 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2371 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:44.577 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:44.577 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:44.577 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8278e-05 s, 0.0 kB/s 2023-10-10T20:38:44.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:44.578 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2372 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:44.632 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:44.633 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:44.633 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8415e-05 s, 0.0 kB/s 2023-10-10T20:38:44.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:44.634 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2373 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:44.688 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7031e-05 s, 0.0 kB/s 2023-10-10T20:38:44.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:44.690 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2374 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:44.744 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:44.745 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:44.745 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8099e-05 s, 0.0 kB/s 2023-10-10T20:38:44.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:44.747 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2375 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:44.801 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:44.801 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1572e-05 s, 0.0 kB/s 2023-10-10T20:38:44.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:44.802 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2376 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:44.856 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:44.856 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:44.856 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8289e-05 s, 0.0 kB/s 2023-10-10T20:38:44.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:44.857 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2377 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:44.911 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:44.911 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:44.912 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1338e-05 s, 0.0 kB/s 2023-10-10T20:38:44.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:44.912 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2378 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:44.966 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:44.966 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:44.967 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7565e-05 s, 0.0 kB/s 2023-10-10T20:38:44.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:44.967 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2379 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:45.022 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:45.023 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:45.023 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0036e-05 s, 0.0 kB/s 2023-10-10T20:38:45.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:45.023 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2380 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:45.078 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:45.079 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:45.079 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.3428e-05 s, 0.0 kB/s 2023-10-10T20:38:45.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:45.080 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2381 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:45.134 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:45.135 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:45.135 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4709e-05 s, 0.0 kB/s 2023-10-10T20:38:45.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:45.135 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2382 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:45.190 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:45.191 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:45.191 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1999e-05 s, 0.0 kB/s 2023-10-10T20:38:45.191 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:45.192 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2383 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:45.246 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:45.247 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:45.247 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8384e-05 s, 0.0 kB/s 2023-10-10T20:38:45.247 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:45.248 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2384 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:45.302 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:45.302 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:45.303 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1547e-05 s, 0.0 kB/s 2023-10-10T20:38:45.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:45.303 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2385 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:45.358 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:45.358 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:45.359 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0211e-05 s, 0.0 kB/s 2023-10-10T20:38:45.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:45.359 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2386 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:45.422 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:45.422 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:45.422 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.63e-05 s, 0.0 kB/s 2023-10-10T20:38:45.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:45.423 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2387 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:45.477 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:45.477 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:45.478 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8443e-05 s, 0.0 kB/s 2023-10-10T20:38:45.478 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:45.478 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2388 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:45.533 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:45.533 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:45.533 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2503e-05 s, 0.0 kB/s 2023-10-10T20:38:45.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:45.534 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2389 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:45.590 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:45.590 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:45.590 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9399e-05 s, 0.0 kB/s 2023-10-10T20:38:45.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:45.591 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2390 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:45.645 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:45.646 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:45.646 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0575e-05 s, 0.0 kB/s 2023-10-10T20:38:45.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:45.647 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2391 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:45.701 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:45.702 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:45.702 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8959e-05 s, 0.0 kB/s 2023-10-10T20:38:45.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:45.703 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2392 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:45.757 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:45.757 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:45.758 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.078e-05 s, 0.0 kB/s 2023-10-10T20:38:45.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:45.758 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2393 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:45.812 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:45.813 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:45.813 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1052e-05 s, 0.0 kB/s 2023-10-10T20:38:45.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:45.814 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2394 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:45.868 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:45.869 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:45.869 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0423e-05 s, 0.0 kB/s 2023-10-10T20:38:45.869 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:45.870 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2395 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:45.927 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:45.927 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:45.927 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0091e-05 s, 0.0 kB/s 2023-10-10T20:38:45.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:45.928 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2396 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:45.982 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:45.982 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:45.983 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9422e-05 s, 0.0 kB/s 2023-10-10T20:38:45.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:45.983 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2397 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:46.038 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:46.039 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:46.039 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7288e-05 s, 0.0 kB/s 2023-10-10T20:38:46.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:46.040 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2398 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:46.094 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:46.095 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:46.095 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8603e-05 s, 0.0 kB/s 2023-10-10T20:38:46.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:46.096 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2399 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:46.151 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:46.151 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:46.151 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9239e-05 s, 0.0 kB/s 2023-10-10T20:38:46.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:46.152 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2400 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:46.206 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:46.207 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:46.207 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9501e-05 s, 0.0 kB/s 2023-10-10T20:38:46.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:46.208 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2401 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:46.262 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:46.263 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:46.263 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0328e-05 s, 0.0 kB/s 2023-10-10T20:38:46.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:46.264 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2402 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:46.318 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:46.319 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:46.319 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9123e-05 s, 0.0 kB/s 2023-10-10T20:38:46.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:46.320 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2403 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:46.374 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:46.374 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:46.375 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.951e-05 s, 0.0 kB/s 2023-10-10T20:38:46.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:46.375 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2404 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:46.430 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:46.430 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:46.430 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3398e-05 s, 0.0 kB/s 2023-10-10T20:38:46.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:46.431 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2405 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:46.485 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:46.485 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:46.485 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1221e-05 s, 0.0 kB/s 2023-10-10T20:38:46.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:46.486 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2406 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:46.540 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:46.540 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:46.541 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0852e-05 s, 0.0 kB/s 2023-10-10T20:38:46.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:46.541 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2407 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:46.596 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:46.597 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:46.597 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0513e-05 s, 0.0 kB/s 2023-10-10T20:38:46.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:46.598 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2408 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:46.653 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4509e-05 s, 0.0 kB/s 2023-10-10T20:38:46.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:46.653 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2409 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:46.708 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:46.708 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:46.709 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1329e-05 s, 0.0 kB/s 2023-10-10T20:38:46.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:46.710 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2410 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:46.765 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:46.765 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:46.766 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7047e-05 s, 0.0 kB/s 2023-10-10T20:38:46.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:46.766 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2411 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:46.821 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:46.821 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:46.822 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0103e-05 s, 0.0 kB/s 2023-10-10T20:38:46.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:46.822 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2412 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:46.877 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:46.878 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:46.878 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0301e-05 s, 0.0 kB/s 2023-10-10T20:38:46.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:46.879 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2413 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:46.934 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:46.934 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:46.934 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1843e-05 s, 0.0 kB/s 2023-10-10T20:38:46.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:46.935 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2414 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:46.989 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:46.990 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:46.990 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0859e-05 s, 0.0 kB/s 2023-10-10T20:38:46.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:46.991 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2415 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:47.045 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:47.046 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:47.046 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7794e-05 s, 0.0 kB/s 2023-10-10T20:38:47.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:47.047 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2416 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:47.101 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:47.101 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:47.102 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9477e-05 s, 0.0 kB/s 2023-10-10T20:38:47.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:47.102 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2417 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:47.158 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:47.158 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:47.158 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8677e-05 s, 0.0 kB/s 2023-10-10T20:38:47.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:47.159 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2418 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:47.213 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:47.214 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:47.214 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0453e-05 s, 0.0 kB/s 2023-10-10T20:38:47.214 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:47.215 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2419 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:47.269 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:47.269 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:47.270 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7715e-05 s, 0.0 kB/s 2023-10-10T20:38:47.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:47.270 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2420 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:47.325 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:47.325 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:47.325 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.125e-05 s, 0 B/s 2023-10-10T20:38:47.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:47.326 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2421 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:47.380 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:47.381 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:47.381 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7279e-05 s, 0.0 kB/s 2023-10-10T20:38:47.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:47.381 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2422 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:47.436 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:47.437 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:47.437 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.079e-05 s, 0.0 kB/s 2023-10-10T20:38:47.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:47.438 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2423 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:47.491 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:47.492 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:47.492 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1139e-05 s, 0.0 kB/s 2023-10-10T20:38:47.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:47.493 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2424 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:47.548 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:47.548 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:47.548 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0967e-05 s, 0.0 kB/s 2023-10-10T20:38:47.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:47.549 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2425 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:47.604 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:47.604 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:47.605 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1981e-05 s, 0.0 kB/s 2023-10-10T20:38:47.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:47.606 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2426 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:47.661 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:47.662 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:47.662 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1296e-05 s, 0.0 kB/s 2023-10-10T20:38:47.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:47.663 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2427 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:47.717 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:47.717 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:47.718 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0416e-05 s, 0.0 kB/s 2023-10-10T20:38:47.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:47.718 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2428 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:47.773 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:47.774 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:47.774 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2375e-05 s, 0.0 kB/s 2023-10-10T20:38:47.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:47.775 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2429 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:47.829 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:47.829 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:47.830 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2307e-05 s, 0.0 kB/s 2023-10-10T20:38:47.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:47.830 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2430 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:47.885 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:47.886 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:47.886 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8822e-05 s, 0.0 kB/s 2023-10-10T20:38:47.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:47.887 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2431 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:47.942 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:47.942 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:47.942 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1788e-05 s, 0.0 kB/s 2023-10-10T20:38:47.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:47.943 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2432 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:47.997 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:47.998 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:47.998 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8039e-05 s, 0.0 kB/s 2023-10-10T20:38:47.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:47.999 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2433 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9938e-05 s, 0.0 kB/s 2023-10-10T20:38:48.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:48.054 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2434 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:48.109 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:48.109 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:48.110 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0939e-05 s, 0.0 kB/s 2023-10-10T20:38:48.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:48.110 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2435 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:48.166 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:48.166 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:48.166 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2861e-05 s, 0.0 kB/s 2023-10-10T20:38:48.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:48.167 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2436 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:48.221 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:48.222 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:48.222 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0633e-05 s, 0.0 kB/s 2023-10-10T20:38:48.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:48.223 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2437 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:48.277 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:48.277 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:48.277 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0559e-05 s, 0.0 kB/s 2023-10-10T20:38:48.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:48.278 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2438 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:48.332 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:48.333 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:48.333 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.126e-05 s, 0.0 kB/s 2023-10-10T20:38:48.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:48.334 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2439 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:48.389 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:48.389 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:48.389 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9569e-05 s, 0.0 kB/s 2023-10-10T20:38:48.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:48.390 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2440 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:48.444 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:48.444 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:48.445 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2982e-05 s, 0.0 kB/s 2023-10-10T20:38:48.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:48.445 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2441 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:48.500 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:48.500 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:48.501 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0013e-05 s, 0.0 kB/s 2023-10-10T20:38:48.501 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:48.501 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2442 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:48.556 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:48.556 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:48.557 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0886e-05 s, 0.0 kB/s 2023-10-10T20:38:48.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:48.557 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2443 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:48.613 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:48.614 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:48.614 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0229e-05 s, 0.0 kB/s 2023-10-10T20:38:48.614 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:48.615 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2444 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:48.669 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:48.669 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:48.670 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9371e-05 s, 0.0 kB/s 2023-10-10T20:38:48.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:48.670 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2445 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:48.725 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:48.726 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:48.726 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9035e-05 s, 0.0 kB/s 2023-10-10T20:38:48.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:48.727 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2446 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:48.781 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:48.782 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:48.782 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8448e-05 s, 0.0 kB/s 2023-10-10T20:38:48.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:48.783 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2447 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:48.837 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:48.837 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:48.838 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8478e-05 s, 0.0 kB/s 2023-10-10T20:38:48.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:48.838 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2448 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:48.893 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:48.893 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:48.893 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5791e-05 s, 0.0 kB/s 2023-10-10T20:38:48.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:48.894 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2449 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4786e-05 s, 0.0 kB/s 2023-10-10T20:38:48.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:48.949 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2450 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:49.004 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:49.004 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:49.004 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.106e-05 s, 0.0 kB/s 2023-10-10T20:38:49.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:49.005 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2451 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:49.059 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:49.060 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:49.060 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2441e-05 s, 0.0 kB/s 2023-10-10T20:38:49.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:49.061 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2452 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:49.116 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:49.117 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:49.117 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1136e-05 s, 0.0 kB/s 2023-10-10T20:38:49.118 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:49.118 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2453 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:49.173 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:49.173 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:49.173 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1246e-05 s, 0.0 kB/s 2023-10-10T20:38:49.174 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:49.174 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2454 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:49.229 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:49.230 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:49.230 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.951e-05 s, 0.0 kB/s 2023-10-10T20:38:49.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:49.231 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2455 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:49.285 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:49.286 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:49.286 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0518e-05 s, 0.0 kB/s 2023-10-10T20:38:49.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:49.287 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2456 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1076e-05 s, 0.0 kB/s 2023-10-10T20:38:49.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:49.343 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2457 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:49.398 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:49.398 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:49.398 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0756e-05 s, 0.0 kB/s 2023-10-10T20:38:49.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:49.399 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2458 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:49.453 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:49.454 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:49.454 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9892e-05 s, 0.0 kB/s 2023-10-10T20:38:49.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:49.455 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2459 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:49.509 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:49.509 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:49.510 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0495e-05 s, 0.0 kB/s 2023-10-10T20:38:49.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:49.511 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2460 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:49.565 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:49.565 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:49.566 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.347e-05 s, 0.0 kB/s 2023-10-10T20:38:49.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:49.567 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2461 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:49.621 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:49.621 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:49.621 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1208e-05 s, 0.0 kB/s 2023-10-10T20:38:49.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:49.624 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2462 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:49.678 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:49.679 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:49.679 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8859e-05 s, 0.0 kB/s 2023-10-10T20:38:49.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:49.680 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2463 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:49.734 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:49.735 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:49.735 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.034e-05 s, 0.0 kB/s 2023-10-10T20:38:49.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:49.736 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2464 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:49.790 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:49.791 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:49.791 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1507e-05 s, 0.0 kB/s 2023-10-10T20:38:49.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:49.792 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2465 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:49.846 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:49.847 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:49.847 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9929e-05 s, 0.0 kB/s 2023-10-10T20:38:49.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:49.848 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2466 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:49.902 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.875e-05 s, 0.0 kB/s 2023-10-10T20:38:49.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:49.904 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2467 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:49.958 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:49.959 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:49.959 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1684e-05 s, 0.0 kB/s 2023-10-10T20:38:49.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:49.960 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2468 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:50.015 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:50.015 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:50.015 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0259e-05 s, 0.0 kB/s 2023-10-10T20:38:50.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:50.016 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2469 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:50.070 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:50.071 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:50.071 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1341e-05 s, 0.0 kB/s 2023-10-10T20:38:50.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:50.072 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2470 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:50.126 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:50.126 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:50.126 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0725e-05 s, 0.0 kB/s 2023-10-10T20:38:50.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:50.127 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2471 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:50.182 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:50.182 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:50.182 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0487e-05 s, 0.0 kB/s 2023-10-10T20:38:50.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:50.183 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2472 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:50.237 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:50.237 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:50.238 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2858e-05 s, 0.0 kB/s 2023-10-10T20:38:50.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:50.238 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2473 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1381e-05 s, 0.0 kB/s 2023-10-10T20:38:50.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:50.294 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2474 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:50.348 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:50.349 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:50.349 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1082e-05 s, 0.0 kB/s 2023-10-10T20:38:50.349 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:50.350 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2475 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:50.405 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:50.405 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:50.405 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3375e-05 s, 0.0 kB/s 2023-10-10T20:38:50.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:50.406 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2476 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:50.461 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:50.461 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:50.461 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0483e-05 s, 0.0 kB/s 2023-10-10T20:38:50.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:50.462 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2477 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:50.516 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:50.517 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:50.517 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9739e-05 s, 0.0 kB/s 2023-10-10T20:38:50.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:50.518 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2478 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:50.572 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:50.573 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:50.573 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.062e-05 s, 0.0 kB/s 2023-10-10T20:38:50.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:50.574 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2479 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:50.629 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:50.629 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:50.629 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9503e-05 s, 0.0 kB/s 2023-10-10T20:38:50.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:50.630 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2480 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:50.684 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:50.684 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:50.685 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0216e-05 s, 0.0 kB/s 2023-10-10T20:38:50.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:50.685 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2481 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:50.739 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:50.740 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:50.740 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.402e-05 s, 0.0 kB/s 2023-10-10T20:38:50.740 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:50.741 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2482 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2271e-05 s, 0.0 kB/s 2023-10-10T20:38:50.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:50.797 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2483 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:50.851 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:50.852 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:50.853 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8606e-05 s, 0.0 kB/s 2023-10-10T20:38:50.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:50.853 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2484 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:50.907 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:50.907 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:50.907 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0869e-05 s, 0.0 kB/s 2023-10-10T20:38:50.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:50.908 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2485 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:50.962 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:50.962 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:50.962 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1338e-05 s, 0.0 kB/s 2023-10-10T20:38:50.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:50.963 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2486 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:51.017 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:51.017 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:51.018 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4034e-05 s, 0.0 kB/s 2023-10-10T20:38:51.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:51.018 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2487 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:51.073 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:51.073 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:51.074 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3201e-05 s, 0.0 kB/s 2023-10-10T20:38:51.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:51.075 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2488 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:51.128 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:51.129 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:51.129 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1761e-05 s, 0.0 kB/s 2023-10-10T20:38:51.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:51.130 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2489 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:51.185 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:51.185 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:51.185 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2578e-05 s, 0.0 kB/s 2023-10-10T20:38:51.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:51.186 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2490 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:51.240 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:51.241 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:51.241 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8509e-05 s, 0.0 kB/s 2023-10-10T20:38:51.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:51.242 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2491 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:51.296 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:51.296 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:51.297 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8483e-05 s, 0.0 kB/s 2023-10-10T20:38:51.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:51.297 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2492 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:51.352 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:51.353 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:51.353 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4217e-05 s, 0.0 kB/s 2023-10-10T20:38:51.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:51.354 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2493 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:51.410 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:51.410 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:51.410 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1016e-05 s, 0.0 kB/s 2023-10-10T20:38:51.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:51.411 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2494 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:51.466 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:51.466 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:51.466 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8333e-05 s, 0.0 kB/s 2023-10-10T20:38:51.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:51.467 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2495 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:51.521 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:51.521 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:51.522 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7482e-05 s, 0.0 kB/s 2023-10-10T20:38:51.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:51.522 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2496 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:51.583 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:51.583 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:51.584 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3521e-05 s, 0.0 kB/s 2023-10-10T20:38:51.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:51.584 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2497 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.439e-05 s, 0.0 kB/s 2023-10-10T20:38:51.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:51.641 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2498 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:51.695 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:51.695 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:51.696 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2504e-05 s, 0.0 kB/s 2023-10-10T20:38:51.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:51.696 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2499 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8785e-05 s, 0.0 kB/s 2023-10-10T20:38:51.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:51.752 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2500 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:51.807 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:51.808 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:51.808 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1212e-05 s, 0.0 kB/s 2023-10-10T20:38:51.808 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:51.809 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2501 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:51.864 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:51.864 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:51.864 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8013e-05 s, 0.0 kB/s 2023-10-10T20:38:51.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:51.865 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2502 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:51.919 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:51.920 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:51.920 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0911e-05 s, 0.0 kB/s 2023-10-10T20:38:51.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:51.921 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2503 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:51.976 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:51.976 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:51.977 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.764e-05 s, 0.0 kB/s 2023-10-10T20:38:51.977 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:51.978 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2504 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:52.032 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:52.032 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:52.033 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.34e-05 s, 0.0 kB/s 2023-10-10T20:38:52.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:52.033 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2505 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:52.095 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3352e-05 s, 0.0 kB/s 2023-10-10T20:38:52.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:52.099 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2506 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:52.158 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:52.158 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:52.158 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1408e-05 s, 0.0 kB/s 2023-10-10T20:38:52.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:52.159 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2507 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.175e-05 s, 0.0 kB/s 2023-10-10T20:38:52.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:52.226 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2508 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:52.282 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:52.283 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:52.283 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0958e-05 s, 0.0 kB/s 2023-10-10T20:38:52.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:52.284 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2509 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2378e-05 s, 0.0 kB/s 2023-10-10T20:38:52.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:52.344 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2510 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:52.406 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:52.406 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:52.407 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4008e-05 s, 0.0 kB/s 2023-10-10T20:38:52.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:52.407 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2511 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3327e-05 s, 0.0 kB/s 2023-10-10T20:38:52.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:52.469 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2512 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:52.531 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:52.532 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:52.532 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3052e-05 s, 0.0 kB/s 2023-10-10T20:38:52.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:52.533 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2513 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:52.588 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:52.588 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:52.588 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3666e-05 s, 0.0 kB/s 2023-10-10T20:38:52.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:52.589 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2514 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:52.647 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:52.647 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:52.648 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0469e-05 s, 0.0 kB/s 2023-10-10T20:38:52.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:52.649 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2515 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:52.704 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:52.705 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:52.705 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0938e-05 s, 0.0 kB/s 2023-10-10T20:38:52.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:52.706 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2516 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:52.763 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:52.763 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:52.763 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1129e-05 s, 0.0 kB/s 2023-10-10T20:38:52.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:52.764 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2517 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:52.819 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:52.820 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:52.820 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0985e-05 s, 0.0 kB/s 2023-10-10T20:38:52.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:52.821 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2518 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:52.875 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:52.876 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:52.876 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.38e-05 s, 0.0 kB/s 2023-10-10T20:38:52.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:52.877 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2519 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:52.931 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:52.932 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:52.932 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5766e-05 s, 0.0 kB/s 2023-10-10T20:38:52.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:52.933 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2520 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:52.988 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:52.988 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:52.988 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8643e-05 s, 0.0 kB/s 2023-10-10T20:38:52.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:52.989 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2521 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:53.042 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:53.043 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:53.043 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8328e-05 s, 0.0 kB/s 2023-10-10T20:38:53.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:53.045 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2522 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:53.100 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:53.100 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:53.101 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3324e-05 s, 0.0 kB/s 2023-10-10T20:38:53.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:53.101 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2523 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:53.156 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:53.156 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:53.156 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2579e-05 s, 0.0 kB/s 2023-10-10T20:38:53.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:53.157 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2524 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:53.211 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:53.212 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:53.212 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2041e-05 s, 0.0 kB/s 2023-10-10T20:38:53.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:53.213 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2525 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:53.268 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:53.268 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:53.268 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0405e-05 s, 0.0 kB/s 2023-10-10T20:38:53.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:53.269 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2526 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:53.323 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:53.323 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:53.324 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7075e-05 s, 0.0 kB/s 2023-10-10T20:38:53.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:53.324 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2527 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:53.379 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:53.379 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:53.380 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0348e-05 s, 0.0 kB/s 2023-10-10T20:38:53.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:53.380 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2528 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:53.435 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:53.435 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:53.436 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1583e-05 s, 0.0 kB/s 2023-10-10T20:38:53.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:53.436 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2529 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.759e-05 s, 0.0 kB/s 2023-10-10T20:38:53.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:53.492 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2530 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:53.547 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:53.547 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:53.547 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3711e-05 s, 0.0 kB/s 2023-10-10T20:38:53.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:53.548 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2531 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:53.603 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:53.603 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:53.603 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3391e-05 s, 0.0 kB/s 2023-10-10T20:38:53.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:53.604 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2532 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:53.659 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:53.659 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:53.659 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2546e-05 s, 0.0 kB/s 2023-10-10T20:38:53.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:53.660 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2533 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:53.716 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:53.716 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:53.717 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0083e-05 s, 0.0 kB/s 2023-10-10T20:38:53.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:53.717 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2534 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:53.772 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:53.773 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:53.773 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9617e-05 s, 0.0 kB/s 2023-10-10T20:38:53.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:53.774 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2535 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:53.829 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:53.829 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9686e-05 s, 0.0 kB/s 2023-10-10T20:38:53.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:53.830 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2536 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:53.884 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:53.884 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:53.884 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1369e-05 s, 0.0 kB/s 2023-10-10T20:38:53.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:53.885 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2537 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:53.939 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:53.939 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:53.940 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1773e-05 s, 0.0 kB/s 2023-10-10T20:38:53.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:53.940 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2538 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:53.995 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:53.995 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:53.996 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1307e-05 s, 0.0 kB/s 2023-10-10T20:38:53.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:53.996 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2539 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:54.051 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:54.051 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:54.052 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0059e-05 s, 0.0 kB/s 2023-10-10T20:38:54.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:54.052 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2540 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:54.108 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:54.108 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:54.108 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7926e-05 s, 0.0 kB/s 2023-10-10T20:38:54.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:54.109 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2541 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:54.163 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:54.164 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:54.164 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2481e-05 s, 0.0 kB/s 2023-10-10T20:38:54.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:54.165 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2542 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:54.219 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:54.219 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:54.220 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.003e-05 s, 0.0 kB/s 2023-10-10T20:38:54.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:54.220 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2543 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:54.275 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:54.276 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:54.276 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0527e-05 s, 0.0 kB/s 2023-10-10T20:38:54.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:54.277 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2544 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:54.331 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:54.332 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:54.332 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1235e-05 s, 0.0 kB/s 2023-10-10T20:38:54.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:54.333 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2545 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:54.388 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:54.388 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:54.388 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3758e-05 s, 0.0 kB/s 2023-10-10T20:38:54.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:54.389 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2546 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:54.443 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:54.444 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:54.444 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0639e-05 s, 0.0 kB/s 2023-10-10T20:38:54.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:54.445 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2547 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:54.499 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:54.500 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:54.500 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2225e-05 s, 0.0 kB/s 2023-10-10T20:38:54.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:54.501 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2548 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:54.555 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:54.555 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:54.556 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1605e-05 s, 0.0 kB/s 2023-10-10T20:38:54.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:54.556 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2549 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:54.611 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:54.611 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:54.611 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9098e-05 s, 0.0 kB/s 2023-10-10T20:38:54.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:54.613 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2550 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:54.669 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:54.669 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:54.669 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1541e-05 s, 0.0 kB/s 2023-10-10T20:38:54.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:54.670 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2551 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:54.724 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:54.725 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:54.725 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.07e-05 s, 0.0 kB/s 2023-10-10T20:38:54.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:54.726 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2552 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:54.781 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:54.781 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:54.781 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8799e-05 s, 0.0 kB/s 2023-10-10T20:38:54.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:54.782 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2553 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6764e-05 s, 0.0 kB/s 2023-10-10T20:38:54.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:54.838 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2554 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:54.892 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:54.892 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:54.893 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1051e-05 s, 0.0 kB/s 2023-10-10T20:38:54.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:54.893 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2555 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:54.948 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:54.949 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:54.949 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0275e-05 s, 0.0 kB/s 2023-10-10T20:38:54.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:54.950 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2556 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:55.004 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:55.005 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:55.005 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9614e-05 s, 0.0 kB/s 2023-10-10T20:38:55.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:55.005 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2557 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:55.060 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:55.060 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:55.061 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0779e-05 s, 0.0 kB/s 2023-10-10T20:38:55.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:55.061 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2558 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1621e-05 s, 0.0 kB/s 2023-10-10T20:38:55.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:55.117 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2559 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:55.171 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:55.172 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:55.172 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.012e-05 s, 0.0 kB/s 2023-10-10T20:38:55.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:55.173 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2560 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:55.227 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:55.228 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:55.228 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8441e-05 s, 0.0 kB/s 2023-10-10T20:38:55.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:55.229 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2561 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:55.283 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:55.283 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:55.284 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9854e-05 s, 0.0 kB/s 2023-10-10T20:38:55.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:55.284 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2562 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:55.339 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:55.339 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:55.340 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1444e-05 s, 0.0 kB/s 2023-10-10T20:38:55.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:55.340 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2563 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:55.395 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:55.395 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:55.396 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1684e-05 s, 0.0 kB/s 2023-10-10T20:38:55.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:55.396 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2564 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:55.451 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:55.451 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:55.452 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.807e-05 s, 0.0 kB/s 2023-10-10T20:38:55.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:55.452 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2565 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:55.507 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:55.508 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:55.508 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3278e-05 s, 0.0 kB/s 2023-10-10T20:38:55.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:55.509 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2566 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:55.563 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:55.564 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:55.564 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0042e-05 s, 0.0 kB/s 2023-10-10T20:38:55.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:55.565 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2567 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:55.619 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:55.619 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:55.620 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8471e-05 s, 0.0 kB/s 2023-10-10T20:38:55.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:55.620 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2568 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:55.676 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:55.676 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:55.677 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5476e-05 s, 0.0 kB/s 2023-10-10T20:38:55.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:55.678 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2569 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:55.732 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:55.732 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:55.733 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2264e-05 s, 0.0 kB/s 2023-10-10T20:38:55.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:55.733 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2570 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:55.788 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:55.788 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:55.789 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1935e-05 s, 0.0 kB/s 2023-10-10T20:38:55.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:55.789 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2571 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:55.844 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:55.844 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:55.845 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3434e-05 s, 0.0 kB/s 2023-10-10T20:38:55.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:55.845 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2572 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:55.900 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:55.901 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:55.901 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8833e-05 s, 0.0 kB/s 2023-10-10T20:38:55.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:55.902 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2573 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:55.956 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:55.957 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:55.957 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9132e-05 s, 0.0 kB/s 2023-10-10T20:38:55.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:55.958 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2574 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:56.012 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:56.013 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:56.013 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0039e-05 s, 0.0 kB/s 2023-10-10T20:38:56.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:56.014 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2575 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:56.068 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:56.068 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:56.069 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9805e-05 s, 0.0 kB/s 2023-10-10T20:38:56.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:56.069 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2576 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:56.124 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:56.125 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:56.125 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8069e-05 s, 0.0 kB/s 2023-10-10T20:38:56.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:56.126 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2577 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:56.181 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:56.181 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:56.181 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9519e-05 s, 0.0 kB/s 2023-10-10T20:38:56.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:56.182 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2578 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:56.236 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:56.237 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:56.237 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.123e-05 s, 0.0 kB/s 2023-10-10T20:38:56.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:56.238 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2579 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:56.292 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:56.292 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:56.292 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8064e-05 s, 0.0 kB/s 2023-10-10T20:38:56.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:56.294 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2580 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:56.349 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:56.349 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:56.350 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1378e-05 s, 0.0 kB/s 2023-10-10T20:38:56.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:56.350 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2581 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:56.407 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:56.407 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:56.407 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2007e-05 s, 0.0 kB/s 2023-10-10T20:38:56.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:56.408 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2582 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0072e-05 s, 0.0 kB/s 2023-10-10T20:38:56.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:56.464 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2583 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:56.518 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:56.518 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:56.519 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9443e-05 s, 0.0 kB/s 2023-10-10T20:38:56.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:56.519 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2584 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:56.575 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:56.575 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3071e-05 s, 0.0 kB/s 2023-10-10T20:38:56.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:56.576 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2585 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:56.632 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:56.632 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:56.632 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1857e-05 s, 0.0 kB/s 2023-10-10T20:38:56.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:56.633 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2586 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:56.688 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:56.688 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:56.688 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0371e-05 s, 0.0 kB/s 2023-10-10T20:38:56.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:56.690 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2587 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:56.744 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:56.744 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:56.745 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9475e-05 s, 0.0 kB/s 2023-10-10T20:38:56.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:56.745 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2588 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:56.801 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:56.801 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:56.801 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1202e-05 s, 0.0 kB/s 2023-10-10T20:38:56.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:56.802 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2589 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:56.857 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:56.857 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:56.858 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0504e-05 s, 0.0 kB/s 2023-10-10T20:38:56.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:56.859 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2590 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:56.912 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:56.912 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:56.913 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5247e-05 s, 0.0 kB/s 2023-10-10T20:38:56.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:56.913 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2591 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9935e-05 s, 0.0 kB/s 2023-10-10T20:38:56.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:56.970 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2592 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:57.024 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:57.024 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:57.025 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0536e-05 s, 0.0 kB/s 2023-10-10T20:38:57.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:57.025 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2593 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:57.080 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:57.080 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:57.080 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7592e-05 s, 0.0 kB/s 2023-10-10T20:38:57.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:57.081 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2594 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:57.136 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:57.136 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1317e-05 s, 0.0 kB/s 2023-10-10T20:38:57.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:57.137 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2595 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:57.205 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:57.205 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:57.205 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.8634e-05 s, 0.0 kB/s 2023-10-10T20:38:57.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:57.206 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2596 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:57.260 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:57.261 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:57.261 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.076e-05 s, 0.0 kB/s 2023-10-10T20:38:57.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:57.262 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2597 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.158e-05 s, 0.0 kB/s 2023-10-10T20:38:57.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:57.317 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2598 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:57.371 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:57.371 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:57.372 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0587e-05 s, 0.0 kB/s 2023-10-10T20:38:57.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:57.372 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2599 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:57.427 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:57.431 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:57.431 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.959e-05 s, 0.0 kB/s 2023-10-10T20:38:57.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:57.432 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2600 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:57.483 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:57.483 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:57.484 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1461e-05 s, 0.0 kB/s 2023-10-10T20:38:57.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:57.484 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2601 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:57.539 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:57.539 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:57.540 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.277e-05 s, 0.0 kB/s 2023-10-10T20:38:57.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:57.540 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2602 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9422e-05 s, 0.0 kB/s 2023-10-10T20:38:57.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:57.597 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2603 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:57.650 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:57.651 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:57.651 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.249e-05 s, 0.0 kB/s 2023-10-10T20:38:57.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:57.651 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2604 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:57.707 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:57.707 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:57.707 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3785e-05 s, 0.0 kB/s 2023-10-10T20:38:57.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:57.708 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2605 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:57.763 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:57.763 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:57.763 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0115e-05 s, 0.0 kB/s 2023-10-10T20:38:57.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:57.764 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2606 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:57.819 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:57.819 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:57.819 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.772e-05 s, 0.0 kB/s 2023-10-10T20:38:57.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:57.820 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2607 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:57.875 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:57.875 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:57.875 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0367e-05 s, 0.0 kB/s 2023-10-10T20:38:57.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:57.876 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2608 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:57.930 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:57.930 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:57.930 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4044e-05 s, 0.0 kB/s 2023-10-10T20:38:57.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:57.932 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2609 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:57.988 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:57.988 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:57.988 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1369e-05 s, 0.0 kB/s 2023-10-10T20:38:57.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:57.989 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2610 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:58.043 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:58.043 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:58.044 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6643e-05 s, 0.0 kB/s 2023-10-10T20:38:58.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:58.044 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2611 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:58.099 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:58.099 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:58.099 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1081e-05 s, 0.0 kB/s 2023-10-10T20:38:58.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:58.100 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2612 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:58.154 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:58.155 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:58.155 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0062e-05 s, 0.0 kB/s 2023-10-10T20:38:58.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:58.156 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2613 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:58.210 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:58.211 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:58.211 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3447e-05 s, 0.0 kB/s 2023-10-10T20:38:58.211 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:58.212 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2614 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:58.266 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:58.267 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:58.267 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2437e-05 s, 0.0 kB/s 2023-10-10T20:38:58.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:58.268 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2615 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:58.323 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:58.323 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:58.323 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8086e-05 s, 0.0 kB/s 2023-10-10T20:38:58.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:58.324 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2616 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:58.379 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:58.379 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:58.379 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0235e-05 s, 0.0 kB/s 2023-10-10T20:38:58.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:58.380 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2617 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:58.434 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:58.435 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:58.435 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1178e-05 s, 0.0 kB/s 2023-10-10T20:38:58.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:58.436 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2618 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:58.490 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:58.491 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:58.491 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7289e-05 s, 0.0 kB/s 2023-10-10T20:38:58.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:58.492 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2619 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:58.546 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:58.547 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:58.547 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9913e-05 s, 0.0 kB/s 2023-10-10T20:38:58.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:58.548 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2620 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:58.602 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:58.602 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:58.603 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9238e-05 s, 0.0 kB/s 2023-10-10T20:38:58.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:58.603 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2621 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:58.659 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:58.659 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:58.659 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2137e-05 s, 0.0 kB/s 2023-10-10T20:38:58.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:58.660 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2622 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:58.715 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:58.715 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:58.716 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0865e-05 s, 0.0 kB/s 2023-10-10T20:38:58.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:58.716 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2623 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:58.771 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:58.772 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:58.772 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8332e-05 s, 0.0 kB/s 2023-10-10T20:38:58.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:58.774 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2624 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:58.827 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:58.827 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:58.828 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.985e-05 s, 0.0 kB/s 2023-10-10T20:38:58.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:58.828 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2625 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:58.883 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:58.884 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:58.884 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7448e-05 s, 0.0 kB/s 2023-10-10T20:38:58.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:58.885 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2626 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:58.939 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:58.940 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:58.940 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9447e-05 s, 0.0 kB/s 2023-10-10T20:38:58.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:58.941 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2627 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:58.996 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8738e-05 s, 0.0 kB/s 2023-10-10T20:38:58.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:58.996 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2628 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:59.051 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:59.052 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:59.052 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1366e-05 s, 0.0 kB/s 2023-10-10T20:38:59.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:59.053 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2629 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:59.108 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:59.108 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.15e-05 s, 0.0 kB/s 2023-10-10T20:38:59.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:59.109 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2630 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:59.164 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:59.164 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:59.164 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0949e-05 s, 0.0 kB/s 2023-10-10T20:38:59.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:59.165 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2631 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:59.219 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:59.219 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:59.219 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1527e-05 s, 0.0 kB/s 2023-10-10T20:38:59.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:59.220 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2632 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:59.274 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:59.275 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:59.275 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9991e-05 s, 0.0 kB/s 2023-10-10T20:38:59.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:59.276 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2633 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:59.331 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:59.331 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:59.332 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0723e-05 s, 0.0 kB/s 2023-10-10T20:38:59.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:59.332 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2634 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:59.387 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:59.387 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:59.387 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2263e-05 s, 0.0 kB/s 2023-10-10T20:38:59.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:59.388 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2635 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:59.443 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:59.443 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:59.443 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9751e-05 s, 0.0 kB/s 2023-10-10T20:38:59.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:59.444 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2636 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:59.498 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:59.499 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:59.499 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.996e-05 s, 0.0 kB/s 2023-10-10T20:38:59.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:59.499 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2637 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:59.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:59.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:59.554 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0587e-05 s, 0.0 kB/s 2023-10-10T20:38:59.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:59.555 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2638 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:59.610 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:59.610 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:59.610 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7883e-05 s, 0.0 kB/s 2023-10-10T20:38:59.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:59.611 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2639 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:59.666 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:59.666 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:59.666 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.7223e-05 s, 0.0 kB/s 2023-10-10T20:38:59.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:59.667 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2640 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:59.721 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:59.722 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:59.722 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8579e-05 s, 0.0 kB/s 2023-10-10T20:38:59.722 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:59.723 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2641 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:59.777 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:59.777 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:59.778 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7644e-05 s, 0.0 kB/s 2023-10-10T20:38:59.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:59.778 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2642 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:59.833 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:59.833 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:59.834 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7954e-05 s, 0.0 kB/s 2023-10-10T20:38:59.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:59.834 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2643 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:59.890 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:59.890 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:59.890 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9251e-05 s, 0.0 kB/s 2023-10-10T20:38:59.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:59.891 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2644 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:38:59.945 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:38:59.945 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:38:59.945 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3172e-05 s, 0.0 kB/s 2023-10-10T20:38:59.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:38:59.946 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2645 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:00.000 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:00.001 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:00.001 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8612e-05 s, 0.0 kB/s 2023-10-10T20:39:00.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:00.002 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2646 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:00.056 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:00.057 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:00.057 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1492e-05 s, 0.0 kB/s 2023-10-10T20:39:00.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:00.058 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2647 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:00.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:00.113 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:00.113 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8752e-05 s, 0.0 kB/s 2023-10-10T20:39:00.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:00.114 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2648 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:00.169 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:00.169 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:00.169 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9164e-05 s, 0.0 kB/s 2023-10-10T20:39:00.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:00.170 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2649 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:00.224 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:00.225 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:00.225 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9164e-05 s, 0.0 kB/s 2023-10-10T20:39:00.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:00.226 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2650 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:00.280 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:00.281 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:00.281 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.948e-05 s, 0.0 kB/s 2023-10-10T20:39:00.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:00.282 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2651 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8487e-05 s, 0.0 kB/s 2023-10-10T20:39:00.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:00.338 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2652 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:00.392 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:00.392 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:00.393 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.056e-05 s, 0.0 kB/s 2023-10-10T20:39:00.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:00.393 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2653 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2887e-05 s, 0.0 kB/s 2023-10-10T20:39:00.450 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:00.450 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2654 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:00.505 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:00.505 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:00.505 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2318e-05 s, 0.0 kB/s 2023-10-10T20:39:00.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:00.506 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2655 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:00.561 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:00.561 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:00.561 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.281e-05 s, 0.0 kB/s 2023-10-10T20:39:00.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:00.562 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2656 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1116e-05 s, 0.0 kB/s 2023-10-10T20:39:00.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:00.618 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2657 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:00.672 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.93e-05 s, 0.0 kB/s 2023-10-10T20:39:00.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:00.674 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2658 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:00.729 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:00.729 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:00.729 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2162e-05 s, 0.0 kB/s 2023-10-10T20:39:00.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:00.730 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2659 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:00.785 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:00.785 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:00.785 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.972e-05 s, 0.0 kB/s 2023-10-10T20:39:00.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:00.786 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2660 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:00.840 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:00.841 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:00.841 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9722e-05 s, 0.0 kB/s 2023-10-10T20:39:00.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:00.842 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2661 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:00.896 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:00.896 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:00.897 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9385e-05 s, 0.0 kB/s 2023-10-10T20:39:00.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:00.897 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2662 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:00.952 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:00.953 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:00.953 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9372e-05 s, 0.0 kB/s 2023-10-10T20:39:00.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:00.954 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2663 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:01.008 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:01.008 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:01.009 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9636e-05 s, 0.0 kB/s 2023-10-10T20:39:01.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:01.010 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2664 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:01.064 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:01.065 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:01.065 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9436e-05 s, 0.0 kB/s 2023-10-10T20:39:01.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:01.066 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2665 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:01.120 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:01.121 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:01.121 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8332e-05 s, 0.0 kB/s 2023-10-10T20:39:01.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:01.122 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2666 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:01.177 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:01.177 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:01.177 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.727e-05 s, 0.0 kB/s 2023-10-10T20:39:01.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:01.178 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2667 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:01.232 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:01.233 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:01.233 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7655e-05 s, 0.0 kB/s 2023-10-10T20:39:01.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:01.234 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2668 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:01.289 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:01.289 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:01.290 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9147e-05 s, 0.0 kB/s 2023-10-10T20:39:01.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:01.290 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2669 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:01.344 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:01.344 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:01.344 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1258e-05 s, 0.0 kB/s 2023-10-10T20:39:01.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:01.345 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2670 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:01.401 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:01.402 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:01.402 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6434e-05 s, 0.0 kB/s 2023-10-10T20:39:01.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:01.403 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2671 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:01.458 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:01.458 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:01.458 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8596e-05 s, 0.0 kB/s 2023-10-10T20:39:01.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:01.459 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2672 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:01.514 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:01.514 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:01.514 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1022e-05 s, 0.0 kB/s 2023-10-10T20:39:01.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:01.515 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2673 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:01.569 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:01.570 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:01.570 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.255e-05 s, 0.0 kB/s 2023-10-10T20:39:01.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:01.571 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2674 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:01.625 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:01.626 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:01.626 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9909e-05 s, 0.0 kB/s 2023-10-10T20:39:01.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:01.627 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2675 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:01.681 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:01.682 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:01.682 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.011e-05 s, 0.0 kB/s 2023-10-10T20:39:01.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:01.685 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2676 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:01.738 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:01.738 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:01.738 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1586e-05 s, 0.0 kB/s 2023-10-10T20:39:01.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:01.739 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2677 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:01.793 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:01.793 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:01.794 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8664e-05 s, 0.0 kB/s 2023-10-10T20:39:01.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:01.794 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2678 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:01.849 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:01.850 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:01.850 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8372e-05 s, 0.0 kB/s 2023-10-10T20:39:01.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:01.851 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2679 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:01.905 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:01.905 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:01.906 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1464e-05 s, 0.0 kB/s 2023-10-10T20:39:01.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:01.906 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2680 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:01.962 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:01.962 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:01.962 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.342e-05 s, 0.0 kB/s 2023-10-10T20:39:01.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:01.963 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2681 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:02.017 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:02.018 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:02.018 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8349e-05 s, 0.0 kB/s 2023-10-10T20:39:02.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:02.019 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2682 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:02.073 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:02.074 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:02.074 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4615e-05 s, 0.0 kB/s 2023-10-10T20:39:02.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:02.075 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2683 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:02.130 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:02.130 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2299e-05 s, 0.0 kB/s 2023-10-10T20:39:02.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:02.131 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2684 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:02.185 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:02.186 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:02.186 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9929e-05 s, 0.0 kB/s 2023-10-10T20:39:02.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:02.187 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2685 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:02.241 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:02.241 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:02.242 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5984e-05 s, 0.0 kB/s 2023-10-10T20:39:02.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:02.242 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2686 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:02.320 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:02.321 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:02.321 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.1157e-05 s, 0.0 kB/s 2023-10-10T20:39:02.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:02.322 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2687 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:02.376 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:02.377 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:02.377 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.182e-05 s, 0.0 kB/s 2023-10-10T20:39:02.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:02.378 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2688 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:02.432 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:02.433 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:02.433 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9954e-05 s, 0.0 kB/s 2023-10-10T20:39:02.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:02.434 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2689 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:02.488 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:02.488 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:02.488 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.392e-05 s, 0.0 kB/s 2023-10-10T20:39:02.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:02.489 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2690 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:02.543 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:02.544 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:02.544 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8187e-05 s, 0.0 kB/s 2023-10-10T20:39:02.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:02.545 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2691 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:02.599 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:02.599 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:02.600 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.727e-05 s, 0.0 kB/s 2023-10-10T20:39:02.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:02.601 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2692 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:02.655 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:02.655 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:02.655 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.966e-05 s, 0.0 kB/s 2023-10-10T20:39:02.656 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:02.656 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2693 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:02.710 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:02.710 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:02.711 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9914e-05 s, 0.0 kB/s 2023-10-10T20:39:02.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:02.711 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2694 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:02.767 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:02.767 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:02.767 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3501e-05 s, 0.0 kB/s 2023-10-10T20:39:02.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:02.768 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2695 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:02.822 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:02.823 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:02.823 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8755e-05 s, 0.0 kB/s 2023-10-10T20:39:02.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:02.824 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2696 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:02.878 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:02.878 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:02.879 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0571e-05 s, 0.0 kB/s 2023-10-10T20:39:02.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:02.879 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2697 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:02.934 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:02.934 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:02.934 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2538e-05 s, 0.0 kB/s 2023-10-10T20:39:02.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:02.935 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2698 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:02.990 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:02.990 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:02.990 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8019e-05 s, 0.0 kB/s 2023-10-10T20:39:02.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:02.991 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2699 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:03.045 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:03.046 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:03.046 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9081e-05 s, 0.0 kB/s 2023-10-10T20:39:03.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:03.047 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2700 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:03.101 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:03.101 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:03.102 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8959e-05 s, 0.0 kB/s 2023-10-10T20:39:03.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:03.102 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2701 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:03.158 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:03.159 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:03.159 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3446e-05 s, 0.0 kB/s 2023-10-10T20:39:03.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:03.161 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2702 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:03.215 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:03.215 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:03.215 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0348e-05 s, 0.0 kB/s 2023-10-10T20:39:03.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:03.216 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2703 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:03.270 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:03.271 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:03.271 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1396e-05 s, 0.0 kB/s 2023-10-10T20:39:03.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:03.272 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2704 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:03.326 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:03.326 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:03.327 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1032e-05 s, 0.0 kB/s 2023-10-10T20:39:03.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:03.329 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2705 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:03.382 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:03.383 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:03.383 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1409e-05 s, 0.0 kB/s 2023-10-10T20:39:03.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:03.384 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2706 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:03.438 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:03.438 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:03.439 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6347e-05 s, 0.0 kB/s 2023-10-10T20:39:03.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:03.439 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2707 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:03.494 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:03.494 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:03.494 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.003e-05 s, 0.0 kB/s 2023-10-10T20:39:03.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:03.495 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2708 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:03.549 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:03.549 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:03.550 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6302e-05 s, 0.0 kB/s 2023-10-10T20:39:03.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:03.550 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2709 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:03.605 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:03.605 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:03.606 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1564e-05 s, 0.0 kB/s 2023-10-10T20:39:03.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:03.606 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2710 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:03.662 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3075e-05 s, 0.0 kB/s 2023-10-10T20:39:03.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:03.663 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2711 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:03.717 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:03.717 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:03.718 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.855e-05 s, 0.0 kB/s 2023-10-10T20:39:03.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:03.718 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2712 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:03.772 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:03.773 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:03.773 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2849e-05 s, 0.0 kB/s 2023-10-10T20:39:03.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:03.774 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2713 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:03.828 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:03.828 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:03.828 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9466e-05 s, 0.0 kB/s 2023-10-10T20:39:03.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:03.829 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2714 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:03.883 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:03.883 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:03.884 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0536e-05 s, 0.0 kB/s 2023-10-10T20:39:03.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:03.884 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2715 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:03.939 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:03.939 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:03.940 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7665e-05 s, 0.0 kB/s 2023-10-10T20:39:03.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:03.940 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2716 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:03.995 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:03.995 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:03.995 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9765e-05 s, 0.0 kB/s 2023-10-10T20:39:03.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:03.996 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2717 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:04.051 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1975e-05 s, 0.0 kB/s 2023-10-10T20:39:04.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:04.051 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2718 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:04.106 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:04.106 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:04.106 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.078e-05 s, 0.0 kB/s 2023-10-10T20:39:04.107 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:04.107 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2719 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:04.161 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:04.162 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:04.162 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8537e-05 s, 0.0 kB/s 2023-10-10T20:39:04.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:04.163 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2720 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:04.217 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:04.218 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:04.218 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8281e-05 s, 0.0 kB/s 2023-10-10T20:39:04.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:04.219 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2721 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0869e-05 s, 0.0 kB/s 2023-10-10T20:39:04.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:04.274 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2722 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:04.329 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:04.330 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:04.330 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9931e-05 s, 0.0 kB/s 2023-10-10T20:39:04.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:04.331 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2723 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:04.385 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:04.386 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:04.386 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3287e-05 s, 0.0 kB/s 2023-10-10T20:39:04.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:04.387 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2724 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1052e-05 s, 0.0 kB/s 2023-10-10T20:39:04.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:04.443 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2725 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:04.498 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:04.498 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9712e-05 s, 0.0 kB/s 2023-10-10T20:39:04.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:04.499 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2726 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:04.553 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:04.553 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:04.554 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7943e-05 s, 0.0 kB/s 2023-10-10T20:39:04.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:04.555 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2727 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:04.609 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:04.609 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:04.609 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1747e-05 s, 0.0 kB/s 2023-10-10T20:39:04.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:04.610 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2728 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:04.664 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:04.665 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:04.665 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0889e-05 s, 0.0 kB/s 2023-10-10T20:39:04.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:04.666 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2729 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:04.721 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:04.721 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:04.721 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1183e-05 s, 0.0 kB/s 2023-10-10T20:39:04.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:04.722 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2730 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:04.777 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:04.777 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:04.777 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0876e-05 s, 0.0 kB/s 2023-10-10T20:39:04.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:04.778 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2731 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1562e-05 s, 0.0 kB/s 2023-10-10T20:39:04.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:04.833 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2732 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:04.888 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:04.889 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:04.889 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9648e-05 s, 0.0 kB/s 2023-10-10T20:39:04.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:04.890 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2733 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:04.944 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:04.944 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:04.945 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.7776e-05 s, 0.0 kB/s 2023-10-10T20:39:04.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:04.945 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2734 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:05.000 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:05.000 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:05.000 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1868e-05 s, 0.0 kB/s 2023-10-10T20:39:05.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:05.001 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2735 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:05.055 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:05.055 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:05.056 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9353e-05 s, 0.0 kB/s 2023-10-10T20:39:05.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:05.056 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2736 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:05.111 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:05.112 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:05.112 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0443e-05 s, 0.0 kB/s 2023-10-10T20:39:05.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:05.113 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2737 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:05.168 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:05.168 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:05.168 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0329e-05 s, 0.0 kB/s 2023-10-10T20:39:05.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:05.169 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2738 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:05.223 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:05.223 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:05.224 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8841e-05 s, 0.0 kB/s 2023-10-10T20:39:05.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:05.225 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2739 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:05.280 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:05.280 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:05.280 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1391e-05 s, 0.0 kB/s 2023-10-10T20:39:05.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:05.281 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2740 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:05.336 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:05.336 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:05.337 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.793e-05 s, 0.0 kB/s 2023-10-10T20:39:05.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:05.338 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2741 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:05.391 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:05.392 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:05.392 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.6972e-05 s, 0.0 kB/s 2023-10-10T20:39:05.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:05.393 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2742 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:05.448 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:05.448 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:05.448 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4109e-05 s, 0.0 kB/s 2023-10-10T20:39:05.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:05.449 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2743 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:05.503 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:05.503 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:05.504 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8689e-05 s, 0.0 kB/s 2023-10-10T20:39:05.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:05.505 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2744 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:05.559 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:05.559 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:05.559 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7792e-05 s, 0.0 kB/s 2023-10-10T20:39:05.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:05.560 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2745 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:05.615 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:05.615 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:05.615 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.766e-05 s, 0.0 kB/s 2023-10-10T20:39:05.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:05.616 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2746 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:05.671 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:05.671 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:05.672 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8192e-05 s, 0.0 kB/s 2023-10-10T20:39:05.672 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:05.673 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2747 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:05.726 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:05.726 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:05.727 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0461e-05 s, 0.0 kB/s 2023-10-10T20:39:05.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:05.727 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2748 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:05.783 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:05.783 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:05.784 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8316e-05 s, 0.0 kB/s 2023-10-10T20:39:05.784 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:05.784 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2749 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:05.839 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:05.839 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:05.839 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0242e-05 s, 0.0 kB/s 2023-10-10T20:39:05.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:05.840 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2750 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:05.894 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:05.895 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:05.895 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2504e-05 s, 0.0 kB/s 2023-10-10T20:39:05.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:05.896 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2751 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:05.950 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:05.951 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:05.951 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9376e-05 s, 0.0 kB/s 2023-10-10T20:39:05.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:05.952 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2752 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:06.008 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:06.008 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:06.008 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3942e-05 s, 0.0 kB/s 2023-10-10T20:39:06.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:06.009 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2753 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:06.063 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:06.063 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:06.063 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1996e-05 s, 0.0 kB/s 2023-10-10T20:39:06.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:06.064 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2754 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:06.118 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:06.118 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:06.119 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1017e-05 s, 0.0 kB/s 2023-10-10T20:39:06.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:06.119 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2755 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1191e-05 s, 0.0 kB/s 2023-10-10T20:39:06.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:06.178 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2756 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:06.230 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:06.231 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:06.231 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.043e-05 s, 0.0 kB/s 2023-10-10T20:39:06.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:06.233 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2757 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:06.289 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:06.289 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:06.290 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4008e-05 s, 0.0 kB/s 2023-10-10T20:39:06.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:06.291 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2758 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:06.344 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:06.345 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:06.345 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8964e-05 s, 0.0 kB/s 2023-10-10T20:39:06.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:06.346 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2759 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:06.402 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:06.402 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:06.402 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2595e-05 s, 0.0 kB/s 2023-10-10T20:39:06.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:06.403 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2760 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:06.458 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:06.458 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:06.458 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.12e-05 s, 0.0 kB/s 2023-10-10T20:39:06.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:06.459 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2761 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:06.513 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:06.513 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:06.513 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1939e-05 s, 0.0 kB/s 2023-10-10T20:39:06.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:06.514 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2762 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:06.568 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:06.569 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:06.569 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2888e-05 s, 0.0 kB/s 2023-10-10T20:39:06.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:06.569 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2763 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:06.624 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:06.624 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:06.624 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2495e-05 s, 0.0 kB/s 2023-10-10T20:39:06.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:06.625 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2764 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:06.679 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:06.680 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:06.680 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2058e-05 s, 0.0 kB/s 2023-10-10T20:39:06.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:06.681 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2765 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:06.735 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:06.736 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:06.736 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6601e-05 s, 0.0 kB/s 2023-10-10T20:39:06.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:06.737 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2766 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:06.792 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:06.792 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:06.792 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9075e-05 s, 0.0 kB/s 2023-10-10T20:39:06.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:06.793 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2767 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:06.848 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:06.848 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:06.848 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.24e-05 s, 0.0 kB/s 2023-10-10T20:39:06.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:06.849 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2768 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:06.903 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:06.904 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:06.904 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1196e-05 s, 0.0 kB/s 2023-10-10T20:39:06.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:06.905 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2769 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:06.960 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:06.960 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:06.960 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2212e-05 s, 0.0 kB/s 2023-10-10T20:39:06.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:06.961 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2770 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:07.015 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:07.016 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:07.016 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9047e-05 s, 0.0 kB/s 2023-10-10T20:39:07.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:07.017 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2771 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:07.072 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:07.072 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:07.072 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3864e-05 s, 0.0 kB/s 2023-10-10T20:39:07.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:07.073 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2772 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:07.127 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:07.128 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:07.128 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1051e-05 s, 0.0 kB/s 2023-10-10T20:39:07.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:07.129 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2773 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:07.184 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:07.185 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:07.185 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2928e-05 s, 0.0 kB/s 2023-10-10T20:39:07.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:07.186 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2774 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:07.241 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:07.241 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:07.241 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9754e-05 s, 0.0 kB/s 2023-10-10T20:39:07.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:07.242 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2775 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:07.296 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:07.297 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:07.297 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0121e-05 s, 0.0 kB/s 2023-10-10T20:39:07.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:07.298 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2776 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:07.352 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9033e-05 s, 0.0 kB/s 2023-10-10T20:39:07.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:07.354 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2777 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:07.446 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:07.447 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:07.447 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.6911e-05 s, 0.0 kB/s 2023-10-10T20:39:07.447 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:07.448 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2778 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:07.463 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:07.464 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:07.464 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1972e-05 s, 0.0 kB/s 2023-10-10T20:39:07.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:07.465 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2779 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:07.519 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:07.520 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:07.520 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3533e-05 s, 0.0 kB/s 2023-10-10T20:39:07.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:07.521 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2780 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:07.575 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:07.576 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:07.576 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.324e-05 s, 0.0 kB/s 2023-10-10T20:39:07.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:07.577 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2781 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:07.631 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:07.632 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:07.632 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4039e-05 s, 0.0 kB/s 2023-10-10T20:39:07.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:07.633 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2782 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:07.688 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:07.688 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:07.688 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1434e-05 s, 0.0 kB/s 2023-10-10T20:39:07.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:07.689 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2783 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:07.747 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:07.747 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:07.747 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1439e-05 s, 0.0 kB/s 2023-10-10T20:39:07.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:07.748 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2784 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:07.806 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:07.807 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:07.807 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2811e-05 s, 0.0 kB/s 2023-10-10T20:39:07.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:07.808 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2785 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:07.867 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:07.867 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:07.868 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.862e-05 s, 0.0 kB/s 2023-10-10T20:39:07.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:07.869 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2786 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:07.923 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:07.923 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:07.923 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3884e-05 s, 0.0 kB/s 2023-10-10T20:39:07.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:07.924 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2787 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:07.978 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:07.978 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:07.978 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3126e-05 s, 0.0 kB/s 2023-10-10T20:39:07.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:07.980 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2788 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:08.036 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:08.036 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:08.036 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2331e-05 s, 0.0 kB/s 2023-10-10T20:39:08.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:08.037 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2789 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:08.092 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:08.092 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:08.092 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3489e-05 s, 0.0 kB/s 2023-10-10T20:39:08.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:08.093 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2790 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:08.152 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:08.152 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:08.153 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.9707e-05 s, 0.0 kB/s 2023-10-10T20:39:08.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:08.153 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2791 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:08.209 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:08.209 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:08.210 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3422e-05 s, 0.0 kB/s 2023-10-10T20:39:08.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:08.210 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2792 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:08.268 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:08.268 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:08.268 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4148e-05 s, 0.0 kB/s 2023-10-10T20:39:08.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:08.269 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2793 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:08.324 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:08.325 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:08.325 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.124e-05 s, 0.0 kB/s 2023-10-10T20:39:08.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:08.325 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2794 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:08.381 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:08.382 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:08.382 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0883e-05 s, 0.0 kB/s 2023-10-10T20:39:08.382 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:08.383 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2795 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:08.440 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:08.440 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:08.440 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1172e-05 s, 0.0 kB/s 2023-10-10T20:39:08.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:08.441 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2796 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:08.497 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:08.497 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:08.497 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0893e-05 s, 0.0 kB/s 2023-10-10T20:39:08.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:08.498 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2797 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:08.553 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:08.553 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:08.553 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2236e-05 s, 0.0 kB/s 2023-10-10T20:39:08.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:08.554 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2798 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:08.608 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:08.609 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:08.609 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9459e-05 s, 0.0 kB/s 2023-10-10T20:39:08.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:08.610 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2799 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:08.665 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:08.666 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:08.666 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.014e-05 s, 0.0 kB/s 2023-10-10T20:39:08.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:08.667 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2800 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:08.721 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:08.721 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:08.721 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0335e-05 s, 0.0 kB/s 2023-10-10T20:39:08.722 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:08.722 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2801 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:08.777 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:08.777 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:08.778 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1154e-05 s, 0.0 kB/s 2023-10-10T20:39:08.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:08.779 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2802 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:08.834 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:08.835 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:08.835 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.975e-05 s, 0.0 kB/s 2023-10-10T20:39:08.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:08.835 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2803 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:08.891 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:08.892 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:08.892 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0911e-05 s, 0.0 kB/s 2023-10-10T20:39:08.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:08.894 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2804 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:08.948 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:08.949 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:08.949 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.189e-05 s, 0.0 kB/s 2023-10-10T20:39:08.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:08.950 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2805 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1682e-05 s, 0.0 kB/s 2023-10-10T20:39:09.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:09.007 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2806 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:09.064 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:09.064 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:09.065 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0305e-05 s, 0.0 kB/s 2023-10-10T20:39:09.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:09.066 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2807 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:09.119 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:09.120 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:09.120 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3932e-05 s, 0.0 kB/s 2023-10-10T20:39:09.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:09.121 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2808 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:09.176 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:09.176 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:09.176 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1675e-05 s, 0.0 kB/s 2023-10-10T20:39:09.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:09.177 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2809 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:09.231 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:09.232 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:09.232 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2347e-05 s, 0.0 kB/s 2023-10-10T20:39:09.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:09.233 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2810 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8115e-05 s, 0.0 kB/s 2023-10-10T20:39:09.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:09.289 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2811 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:09.344 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:09.344 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:09.345 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1518e-05 s, 0.0 kB/s 2023-10-10T20:39:09.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:09.345 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2812 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:09.400 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:09.400 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:09.400 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2259e-05 s, 0.0 kB/s 2023-10-10T20:39:09.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:09.401 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2813 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:09.456 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:09.456 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:09.456 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3415e-05 s, 0.0 kB/s 2023-10-10T20:39:09.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:09.457 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2814 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1468e-05 s, 0.0 kB/s 2023-10-10T20:39:09.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:09.513 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2815 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:09.568 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:09.568 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:09.568 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3328e-05 s, 0.0 kB/s 2023-10-10T20:39:09.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:09.569 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2816 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:09.623 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:09.624 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:09.624 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2877e-05 s, 0.0 kB/s 2023-10-10T20:39:09.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:09.625 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2817 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:09.679 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:09.680 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:09.680 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3795e-05 s, 0.0 kB/s 2023-10-10T20:39:09.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:09.680 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2818 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:09.735 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:09.735 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:09.736 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8637e-05 s, 0.0 kB/s 2023-10-10T20:39:09.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:09.736 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2819 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:09.792 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:09.792 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:09.792 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3364e-05 s, 0.0 kB/s 2023-10-10T20:39:09.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:09.793 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2820 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2066e-05 s, 0.0 kB/s 2023-10-10T20:39:09.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:09.850 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2821 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:09.904 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:09.905 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:09.905 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2658e-05 s, 0.0 kB/s 2023-10-10T20:39:09.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:09.906 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2822 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:09.961 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:09.961 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:09.961 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3563e-05 s, 0.0 kB/s 2023-10-10T20:39:09.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:09.962 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2823 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:10.016 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:10.017 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:10.017 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9782e-05 s, 0.0 kB/s 2023-10-10T20:39:10.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:10.018 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2824 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:10.072 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:10.073 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:10.073 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9751e-05 s, 0.0 kB/s 2023-10-10T20:39:10.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:10.074 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2825 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:10.129 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:10.129 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:10.129 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9383e-05 s, 0.0 kB/s 2023-10-10T20:39:10.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:10.130 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2826 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:10.185 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:10.185 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:10.185 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9559e-05 s, 0.0 kB/s 2023-10-10T20:39:10.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:10.186 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2827 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:10.240 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:10.240 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:10.240 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0533e-05 s, 0.0 kB/s 2023-10-10T20:39:10.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:10.242 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2828 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:10.298 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:10.298 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:10.298 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0027e-05 s, 0.0 kB/s 2023-10-10T20:39:10.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:10.299 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2829 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:10.353 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:10.353 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:10.354 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3605e-05 s, 0.0 kB/s 2023-10-10T20:39:10.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:10.354 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2830 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:10.409 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:10.409 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:10.410 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3316e-05 s, 0.0 kB/s 2023-10-10T20:39:10.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:10.410 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2831 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:10.465 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9347e-05 s, 0.0 kB/s 2023-10-10T20:39:10.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:10.467 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2832 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:10.521 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:10.521 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:10.522 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1728e-05 s, 0.0 kB/s 2023-10-10T20:39:10.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:10.522 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2833 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:10.577 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:10.578 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:10.578 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4118e-05 s, 0.0 kB/s 2023-10-10T20:39:10.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:10.579 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2834 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:10.633 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:10.634 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:10.634 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2796e-05 s, 0.0 kB/s 2023-10-10T20:39:10.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:10.635 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2835 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:10.689 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:10.690 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:10.690 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9511e-05 s, 0.0 kB/s 2023-10-10T20:39:10.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:10.691 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2836 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.388e-05 s, 0.0 kB/s 2023-10-10T20:39:10.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:10.747 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2837 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:10.801 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:10.802 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:10.802 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3087e-05 s, 0.0 kB/s 2023-10-10T20:39:10.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:10.803 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2838 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:10.858 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:10.858 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:10.859 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1338e-05 s, 0.0 kB/s 2023-10-10T20:39:10.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:10.859 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2839 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:10.915 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:10.915 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:10.915 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1055e-05 s, 0.0 kB/s 2023-10-10T20:39:10.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:10.916 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2840 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:10.971 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:10.971 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:10.971 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4051e-05 s, 0.0 kB/s 2023-10-10T20:39:10.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:10.972 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2841 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:11.027 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:11.027 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:11.027 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0999e-05 s, 0.0 kB/s 2023-10-10T20:39:11.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:11.028 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2842 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:11.082 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:11.082 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:11.082 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1824e-05 s, 0.0 kB/s 2023-10-10T20:39:11.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:11.084 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2843 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:11.140 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:11.140 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:11.140 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4122e-05 s, 0.0 kB/s 2023-10-10T20:39:11.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:11.141 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2844 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:11.196 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:11.196 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2329e-05 s, 0.0 kB/s 2023-10-10T20:39:11.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:11.197 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2845 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:11.252 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:11.253 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:11.253 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3641e-05 s, 0.0 kB/s 2023-10-10T20:39:11.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:11.254 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2846 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:11.309 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:11.309 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:11.309 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0796e-05 s, 0.0 kB/s 2023-10-10T20:39:11.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:11.310 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2847 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:11.365 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:11.365 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:11.365 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3942e-05 s, 0.0 kB/s 2023-10-10T20:39:11.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:11.366 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2848 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:11.421 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:11.421 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:11.421 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0763e-05 s, 0.0 kB/s 2023-10-10T20:39:11.422 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:11.423 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2849 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:11.477 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:11.477 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:11.477 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0744e-05 s, 0.0 kB/s 2023-10-10T20:39:11.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:11.478 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2850 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:11.533 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:11.533 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:11.533 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7298e-05 s, 0.0 kB/s 2023-10-10T20:39:11.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:11.534 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2851 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:11.588 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:11.589 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:11.589 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.164e-05 s, 0.0 kB/s 2023-10-10T20:39:11.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:11.590 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2852 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:11.645 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:11.646 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:11.646 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1477e-05 s, 0.0 kB/s 2023-10-10T20:39:11.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:11.647 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2853 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:11.701 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:11.702 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:11.702 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4821e-05 s, 0.0 kB/s 2023-10-10T20:39:11.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:11.703 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2854 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:11.757 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:11.758 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:11.758 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2375e-05 s, 0.0 kB/s 2023-10-10T20:39:11.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:11.759 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2855 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:11.814 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:11.814 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:11.814 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1678e-05 s, 0.0 kB/s 2023-10-10T20:39:11.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:11.815 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2856 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:11.870 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:11.871 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:11.871 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9126e-05 s, 0.0 kB/s 2023-10-10T20:39:11.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:11.872 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2857 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:11.926 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:11.926 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:11.927 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9124e-05 s, 0.0 kB/s 2023-10-10T20:39:11.927 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:11.927 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2858 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:11.983 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:11.983 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:11.983 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3245e-05 s, 0.0 kB/s 2023-10-10T20:39:11.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:11.984 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2859 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:12.038 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:12.039 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:12.039 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.023e-05 s, 0.0 kB/s 2023-10-10T20:39:12.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:12.040 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2860 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:12.093 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:12.094 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:12.094 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0265e-05 s, 0.0 kB/s 2023-10-10T20:39:12.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:12.095 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2861 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:12.150 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:12.151 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:12.151 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2534e-05 s, 0.0 kB/s 2023-10-10T20:39:12.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:12.152 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2862 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:12.207 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:12.207 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:12.207 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4129e-05 s, 0.0 kB/s 2023-10-10T20:39:12.208 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:12.208 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2863 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:12.262 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:12.262 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:12.263 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2302e-05 s, 0.0 kB/s 2023-10-10T20:39:12.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:12.263 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2864 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:12.318 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:12.319 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:12.319 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.037e-05 s, 0.0 kB/s 2023-10-10T20:39:12.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:12.320 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2865 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:12.341 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:39:12.362 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:39:12.375 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:12.375 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:12.375 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3529e-05 s, 0.0 kB/s 2023-10-10T20:39:12.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:12.376 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2866 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:12.467 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:12.467 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:12.468 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4261e-05 s, 0.0 kB/s 2023-10-10T20:39:12.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:12.468 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2867 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:12.523 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:12.524 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:12.524 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1164e-05 s, 0.0 kB/s 2023-10-10T20:39:12.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:12.525 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2868 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:12.579 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:12.580 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:12.580 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9819e-05 s, 0.0 kB/s 2023-10-10T20:39:12.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:12.581 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2869 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:12.636 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:12.636 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:12.636 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1428e-05 s, 0.0 kB/s 2023-10-10T20:39:12.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:12.637 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2870 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:12.691 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:12.691 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:12.691 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0662e-05 s, 0.0 kB/s 2023-10-10T20:39:12.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:12.693 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2871 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:12.749 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:12.749 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:12.750 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.114e-05 s, 0.0 kB/s 2023-10-10T20:39:12.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:12.750 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2872 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:12.805 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:12.806 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:12.806 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0753e-05 s, 0.0 kB/s 2023-10-10T20:39:12.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:12.807 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2873 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:12.862 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:12.862 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:12.862 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1869e-05 s, 0.0 kB/s 2023-10-10T20:39:12.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:12.863 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2874 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:12.919 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:12.919 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:12.919 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2184e-05 s, 0.0 kB/s 2023-10-10T20:39:12.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:12.920 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2875 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:12.975 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:12.975 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:12.976 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0369e-05 s, 0.0 kB/s 2023-10-10T20:39:12.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:12.977 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2876 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:13.030 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:13.031 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:13.031 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1808e-05 s, 0.0 kB/s 2023-10-10T20:39:13.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:13.032 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2877 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:13.087 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:13.087 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:13.087 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2107e-05 s, 0.0 kB/s 2023-10-10T20:39:13.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:13.088 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2878 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1762e-05 s, 0.0 kB/s 2023-10-10T20:39:13.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:13.144 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2879 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:13.199 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:13.199 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:13.200 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3966e-05 s, 0.0 kB/s 2023-10-10T20:39:13.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:13.200 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2880 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:13.256 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:13.256 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:13.256 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.399e-05 s, 0.0 kB/s 2023-10-10T20:39:13.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:13.257 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2881 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:13.311 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:13.312 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:13.312 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1613e-05 s, 0.0 kB/s 2023-10-10T20:39:13.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:13.313 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2882 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:13.368 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:13.368 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:13.368 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0969e-05 s, 0.0 kB/s 2023-10-10T20:39:13.369 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:13.369 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2883 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:13.423 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:13.424 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:13.424 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0483e-05 s, 0.0 kB/s 2023-10-10T20:39:13.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:13.425 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2884 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:13.480 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:13.480 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:13.480 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1399e-05 s, 0.0 kB/s 2023-10-10T20:39:13.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:13.481 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2885 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:13.535 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:13.536 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:13.536 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6974e-05 s, 0.0 kB/s 2023-10-10T20:39:13.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:13.537 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2886 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4162e-05 s, 0.0 kB/s 2023-10-10T20:39:13.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:13.598 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2887 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:13.653 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:13.653 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:13.653 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4245e-05 s, 0.0 kB/s 2023-10-10T20:39:13.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:13.654 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2888 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:13.709 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:13.709 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:13.709 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0187e-05 s, 0.0 kB/s 2023-10-10T20:39:13.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:13.710 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2889 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:13.767 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:13.767 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:13.768 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0535e-05 s, 0.0 kB/s 2023-10-10T20:39:13.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:13.770 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2890 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:13.823 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:13.824 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:13.824 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1212e-05 s, 0.0 kB/s 2023-10-10T20:39:13.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:13.825 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2891 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:13.880 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:13.880 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:13.880 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1931e-05 s, 0.0 kB/s 2023-10-10T20:39:13.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:13.881 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2892 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:13.935 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:13.936 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:13.936 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9672e-05 s, 0.0 kB/s 2023-10-10T20:39:13.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:13.937 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2893 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:13.991 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:13.992 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:13.992 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0654e-05 s, 0.0 kB/s 2023-10-10T20:39:13.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:13.993 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2894 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:14.046 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:14.047 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:14.047 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0258e-05 s, 0.0 kB/s 2023-10-10T20:39:14.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:14.048 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2895 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:14.104 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.054e-05 s, 0.0 kB/s 2023-10-10T20:39:14.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:14.106 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2896 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:14.161 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:14.161 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:14.162 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.382e-05 s, 0.0 kB/s 2023-10-10T20:39:14.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:14.162 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2897 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:14.218 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:14.218 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:14.218 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.962e-05 s, 0.0 kB/s 2023-10-10T20:39:14.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:14.219 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2898 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:14.273 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:14.273 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:14.274 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2537e-05 s, 0.0 kB/s 2023-10-10T20:39:14.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:14.274 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2899 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:14.330 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:14.330 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:14.330 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1522e-05 s, 0.0 kB/s 2023-10-10T20:39:14.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:14.331 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2900 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:14.385 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:14.386 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:14.386 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2048e-05 s, 0.0 kB/s 2023-10-10T20:39:14.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:14.387 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2901 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:14.442 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1918e-05 s, 0.0 kB/s 2023-10-10T20:39:14.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:14.444 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2902 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1701e-05 s, 0.0 kB/s 2023-10-10T20:39:14.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:14.500 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2903 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:14.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:14.554 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:14.555 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2426e-05 s, 0.0 kB/s 2023-10-10T20:39:14.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:14.555 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2904 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1651e-05 s, 0.0 kB/s 2023-10-10T20:39:14.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:14.612 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2905 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:14.666 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3558e-05 s, 0.0 kB/s 2023-10-10T20:39:14.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:14.668 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2906 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:14.722 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.076e-05 s, 0.0 kB/s 2023-10-10T20:39:14.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:14.724 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2907 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:14.778 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:14.778 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:14.779 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0368e-05 s, 0.0 kB/s 2023-10-10T20:39:14.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:14.779 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2908 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:14.834 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:14.835 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:14.835 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9175e-05 s, 0.0 kB/s 2023-10-10T20:39:14.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:14.836 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2909 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:14.890 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:14.890 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:14.891 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7988e-05 s, 0.0 kB/s 2023-10-10T20:39:14.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:14.891 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2910 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:14.947 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:14.947 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:14.947 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6511e-05 s, 0.0 kB/s 2023-10-10T20:39:14.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:14.948 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2911 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8306e-05 s, 0.0 kB/s 2023-10-10T20:39:15.003 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:15.004 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2912 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:15.058 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:15.059 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:15.059 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1407e-05 s, 0.0 kB/s 2023-10-10T20:39:15.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:15.060 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2913 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:15.114 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:15.114 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:15.115 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8681e-05 s, 0.0 kB/s 2023-10-10T20:39:15.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:15.115 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2914 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:15.171 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0075e-05 s, 0.0 kB/s 2023-10-10T20:39:15.171 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:15.171 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2915 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:15.227 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:15.227 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:15.227 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.403e-05 s, 0.0 kB/s 2023-10-10T20:39:15.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:15.228 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2916 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:15.282 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:15.283 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:15.283 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.929e-05 s, 0.0 kB/s 2023-10-10T20:39:15.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:15.284 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2917 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:15.338 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:15.338 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:15.339 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9878e-05 s, 0.0 kB/s 2023-10-10T20:39:15.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:15.339 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2918 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:15.393 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:15.394 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:15.394 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8777e-05 s, 0.0 kB/s 2023-10-10T20:39:15.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:15.395 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2919 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:15.450 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5617e-05 s, 0.0 kB/s 2023-10-10T20:39:15.450 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:15.450 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2920 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:15.504 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:15.505 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:15.505 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1163e-05 s, 0.0 kB/s 2023-10-10T20:39:15.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:15.506 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2921 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:15.560 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:15.560 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:15.561 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6485e-05 s, 0.0 kB/s 2023-10-10T20:39:15.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:15.561 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2922 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:15.616 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:15.616 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:15.616 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4369e-05 s, 0.0 kB/s 2023-10-10T20:39:15.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:15.617 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2923 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:15.671 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:15.671 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:15.671 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.081e-05 s, 0.0 kB/s 2023-10-10T20:39:15.672 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:15.672 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2924 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:15.726 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:15.727 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:15.727 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0338e-05 s, 0.0 kB/s 2023-10-10T20:39:15.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:15.728 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2925 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:15.782 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:15.783 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:15.783 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3634e-05 s, 0.0 kB/s 2023-10-10T20:39:15.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:15.784 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2926 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:15.838 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:15.838 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:15.839 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1723e-05 s, 0.0 kB/s 2023-10-10T20:39:15.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:15.839 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2927 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:15.894 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:15.895 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:15.895 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0175e-05 s, 0.0 kB/s 2023-10-10T20:39:15.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:15.896 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2928 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:15.951 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:15.951 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:15.951 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1646e-05 s, 0.0 kB/s 2023-10-10T20:39:15.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:15.952 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2929 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0538e-05 s, 0.0 kB/s 2023-10-10T20:39:16.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:16.008 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2930 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:16.062 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:16.062 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:16.062 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1841e-05 s, 0.0 kB/s 2023-10-10T20:39:16.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:16.063 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2931 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:16.117 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:16.117 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:16.117 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7467e-05 s, 0.0 kB/s 2023-10-10T20:39:16.118 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:16.118 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2932 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:16.171 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:16.171 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:16.172 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0858e-05 s, 0.0 kB/s 2023-10-10T20:39:16.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:16.173 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2933 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:16.228 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:16.228 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:16.228 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3856e-05 s, 0.0 kB/s 2023-10-10T20:39:16.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:16.229 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2934 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:16.284 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:16.285 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:16.285 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2121e-05 s, 0.0 kB/s 2023-10-10T20:39:16.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:16.286 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2935 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:16.340 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:16.340 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:16.341 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0809e-05 s, 0.0 kB/s 2023-10-10T20:39:16.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:16.341 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2936 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:16.396 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:16.397 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:16.397 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3755e-05 s, 0.0 kB/s 2023-10-10T20:39:16.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:16.399 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2937 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:16.454 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:16.454 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0943e-05 s, 0.0 kB/s 2023-10-10T20:39:16.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:16.455 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2938 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:16.509 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:16.510 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:16.510 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.8933e-05 s, 0.0 kB/s 2023-10-10T20:39:16.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:16.510 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2939 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:16.565 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:16.566 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:16.566 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3199e-05 s, 0.0 kB/s 2023-10-10T20:39:16.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:16.567 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2940 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1947e-05 s, 0.0 kB/s 2023-10-10T20:39:16.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:16.622 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2941 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:16.677 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:16.677 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:16.677 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7072e-05 s, 0.0 kB/s 2023-10-10T20:39:16.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:16.678 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2942 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:16.732 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:16.732 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:16.733 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0812e-05 s, 0.0 kB/s 2023-10-10T20:39:16.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:16.733 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2943 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:16.788 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:16.788 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:16.789 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1743e-05 s, 0.0 kB/s 2023-10-10T20:39:16.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:16.789 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2944 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:16.845 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:16.845 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:16.845 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2105e-05 s, 0.0 kB/s 2023-10-10T20:39:16.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:16.846 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2945 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.6866e-05 s, 0.0 kB/s 2023-10-10T20:39:16.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:16.901 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2946 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:16.956 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:16.956 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:16.956 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0297e-05 s, 0.0 kB/s 2023-10-10T20:39:16.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:16.957 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2947 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:17.010 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:17.011 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:17.011 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0091e-05 s, 0.0 kB/s 2023-10-10T20:39:17.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:17.013 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2948 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:17.068 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:17.068 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:17.069 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2823e-05 s, 0.0 kB/s 2023-10-10T20:39:17.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:17.069 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2949 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:17.124 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:17.124 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:17.124 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2878e-05 s, 0.0 kB/s 2023-10-10T20:39:17.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:17.125 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2950 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:17.179 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:17.181 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:17.181 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0811e-05 s, 0.0 kB/s 2023-10-10T20:39:17.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:17.182 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2951 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:17.235 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:17.235 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:17.235 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9974e-05 s, 0.0 kB/s 2023-10-10T20:39:17.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:17.236 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2952 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:17.290 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:17.290 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:17.291 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1068e-05 s, 0.0 kB/s 2023-10-10T20:39:17.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:17.292 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2953 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1402e-05 s, 0.0 kB/s 2023-10-10T20:39:17.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:17.348 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2954 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:17.402 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:17.403 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:17.403 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0432e-05 s, 0.0 kB/s 2023-10-10T20:39:17.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:17.404 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2955 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:17.461 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:17.461 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0833e-05 s, 0.0 kB/s 2023-10-10T20:39:17.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:17.462 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2956 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:17.516 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:17.517 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:17.517 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2163e-05 s, 0.0 kB/s 2023-10-10T20:39:17.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:17.518 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2957 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:17.572 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:17.573 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:17.573 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9989e-05 s, 0.0 kB/s 2023-10-10T20:39:17.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:17.574 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2958 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:17.628 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:17.629 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:17.629 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2074e-05 s, 0.0 kB/s 2023-10-10T20:39:17.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:17.630 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2959 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:17.684 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:17.685 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:17.685 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.15e-05 s, 0.0 kB/s 2023-10-10T20:39:17.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:17.686 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2960 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:17.740 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:17.740 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:17.741 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3655e-05 s, 0.0 kB/s 2023-10-10T20:39:17.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:17.742 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2961 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:17.796 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:17.797 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:17.797 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0103e-05 s, 0.0 kB/s 2023-10-10T20:39:17.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:17.798 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2962 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:17.854 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:17.855 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:17.855 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9004e-05 s, 0.0 kB/s 2023-10-10T20:39:17.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:17.856 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2963 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:17.910 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:17.910 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:17.910 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2417e-05 s, 0.0 kB/s 2023-10-10T20:39:17.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:17.912 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2964 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:17.968 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:17.969 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:17.969 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.056e-05 s, 0.0 kB/s 2023-10-10T20:39:17.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:17.971 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2965 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:18.025 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:18.025 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:18.026 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0395e-05 s, 0.0 kB/s 2023-10-10T20:39:18.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:18.027 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2966 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:18.081 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:18.082 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:18.082 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1209e-05 s, 0.0 kB/s 2023-10-10T20:39:18.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:18.083 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2967 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:18.137 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:18.138 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:18.138 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4128e-05 s, 0.0 kB/s 2023-10-10T20:39:18.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:18.139 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2968 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3888e-05 s, 0.0 kB/s 2023-10-10T20:39:18.195 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:18.195 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2969 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:18.249 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:18.250 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:18.250 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0888e-05 s, 0.0 kB/s 2023-10-10T20:39:18.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:18.251 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2970 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:18.305 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:18.306 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:18.306 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8942e-05 s, 0.0 kB/s 2023-10-10T20:39:18.306 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:18.307 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2971 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9343e-05 s, 0.0 kB/s 2023-10-10T20:39:18.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:18.363 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2972 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:18.417 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:18.418 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:18.418 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0201e-05 s, 0.0 kB/s 2023-10-10T20:39:18.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:18.419 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2973 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:18.473 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:18.474 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:18.474 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1037e-05 s, 0.0 kB/s 2023-10-10T20:39:18.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:18.475 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2974 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:18.529 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:18.530 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:18.530 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.338e-05 s, 0.0 kB/s 2023-10-10T20:39:18.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:18.531 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2975 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:18.586 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:18.586 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:18.586 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.3435e-05 s, 0.0 kB/s 2023-10-10T20:39:18.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:18.587 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2976 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:18.641 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:18.642 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:18.642 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0309e-05 s, 0.0 kB/s 2023-10-10T20:39:18.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:18.643 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2977 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:18.699 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.09e-05 s, 0.0 kB/s 2023-10-10T20:39:18.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:18.699 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2978 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:18.754 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:18.755 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:18.755 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.859e-05 s, 0.0 kB/s 2023-10-10T20:39:18.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:18.756 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2979 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:18.811 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:18.811 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:18.811 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2194e-05 s, 0.0 kB/s 2023-10-10T20:39:18.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:18.812 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2980 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:18.867 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:18.867 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:18.867 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.738e-05 s, 0.0 kB/s 2023-10-10T20:39:18.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:18.868 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2981 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:18.923 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:18.923 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:18.923 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2079e-05 s, 0.0 kB/s 2023-10-10T20:39:18.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:18.924 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2982 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.7835e-05 s, 0.0 kB/s 2023-10-10T20:39:18.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:18.981 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2983 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:19.035 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:19.036 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:19.036 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8351e-05 s, 0.0 kB/s 2023-10-10T20:39:19.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:19.037 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2984 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:19.092 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:19.092 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:19.093 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9706e-05 s, 0.0 kB/s 2023-10-10T20:39:19.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:19.093 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2985 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:19.148 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:19.148 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:19.149 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0308e-05 s, 0.0 kB/s 2023-10-10T20:39:19.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:19.149 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2986 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:19.224 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:19.225 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:19.225 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 4.1532e-05 s, 0.0 kB/s 2023-10-10T20:39:19.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:19.226 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2987 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:19.281 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:19.281 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:19.281 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9112e-05 s, 0.0 kB/s 2023-10-10T20:39:19.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:19.282 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2988 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:19.336 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:19.337 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:19.337 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0863e-05 s, 0.0 kB/s 2023-10-10T20:39:19.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:19.338 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2989 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:19.393 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:19.393 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:19.393 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8533e-05 s, 0.0 kB/s 2023-10-10T20:39:19.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:19.394 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2990 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:19.449 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:19.449 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:19.449 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.4748e-05 s, 0.0 kB/s 2023-10-10T20:39:19.450 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:19.451 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2991 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:19.504 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:19.505 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:19.505 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.2706e-05 s, 0.0 kB/s 2023-10-10T20:39:19.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:19.506 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2992 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:19.560 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:19.561 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:19.561 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8256e-05 s, 0.0 kB/s 2023-10-10T20:39:19.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:19.562 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2993 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:19.617 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:19.617 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:19.618 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1964e-05 s, 0.0 kB/s 2023-10-10T20:39:19.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:19.619 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2994 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:19.672 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:19.673 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:19.673 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.8318e-05 s, 0.0 kB/s 2023-10-10T20:39:19.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:19.674 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2995 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:19.729 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:19.729 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:19.730 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.1579e-05 s, 0.0 kB/s 2023-10-10T20:39:19.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:19.730 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2996 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:19.784 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:19.785 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:19.785 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9042e-05 s, 0.0 kB/s 2023-10-10T20:39:19.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:19.786 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2997 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:19.840 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:19.841 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:19.841 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.5392e-05 s, 0.0 kB/s 2023-10-10T20:39:19.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:19.842 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2998 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:19.896 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:19.897 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:19.897 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 2.9117e-05 s, 0.0 kB/s 2023-10-10T20:39:19.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:19.898 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000078554/1b777ba9-66f4-4270-ab78-0de9b7e9bf89/subvolume_file.2999 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:39:19.952 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records in 2023-10-10T20:39:19.952 INFO:teuthology.orchestra.run.smithi089.stderr:0+0 records out 2023-10-10T20:39:19.953 INFO:teuthology.orchestra.run.smithi089.stderr:0 bytes copied, 3.0616e-05 s, 0.0 kB/s 2023-10-10T20:39:19.953 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000078554 2023-10-10T20:39:20.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.107+0000 7f30b7b45700 1 Processor -- start 2023-10-10T20:39:20.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.107+0000 7f30b7b45700 1 -- start start 2023-10-10T20:39:20.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.108+0000 7f30b7b45700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f30b0155eb0 0x7f30b006e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:20.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.108+0000 7f30b7b45700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30b00c70b0 0x7f30b0069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:20.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.108+0000 7f30b7b45700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b006eb90 con 0x7f30b0069cb0 2023-10-10T20:39:20.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.108+0000 7f30b7b45700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b006a130 con 0x7f30b0155eb0 2023-10-10T20:39:20.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.108+0000 7f30b7b45700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b00cd350 con 0x7f30b00c70b0 2023-10-10T20:39:20.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.108+0000 7f30b58e1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f30b0155eb0 0x7f30b006e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:20.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.108+0000 7f30b60e2700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f30b0069cb0 0x7f30b00ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:32858/0 (socket says 172.21.15.89:32858) 2023-10-10T20:39:20.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.108+0000 7f30b60e2700 1 -- 172.21.15.89:0/3291626849 learned_addr learned my addr 172.21.15.89:0/3291626849 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:20.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.108+0000 7f30b58e1700 1 --2- 172.21.15.89:0/3291626849 >> v2:172.21.15.89:3300/0 conn(0x7f30b0155eb0 0x7f30b006e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43222/0 (socket says 172.21.15.89:43222) 2023-10-10T20:39:20.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.109+0000 7f30b50e0700 1 --2- 172.21.15.89:0/3291626849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30b00c70b0 0x7f30b0069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:20.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.109+0000 7f30b58e1700 1 -- 172.21.15.89:0/3291626849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30b00c70b0 msgr2=0x7f30b0069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:20.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.109+0000 7f30b58e1700 1 --2- 172.21.15.89:0/3291626849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30b00c70b0 0x7f30b0069770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:20.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.109+0000 7f30b58e1700 1 -- 172.21.15.89:0/3291626849 >> v1:172.21.15.89:6789/0 conn(0x7f30b0069cb0 legacy=0x7f30b00ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:20.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.109+0000 7f30b58e1700 1 -- 172.21.15.89:0/3291626849 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b006ee70 con 0x7f30b0155eb0 2023-10-10T20:39:20.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.109+0000 7f30b50e0700 1 --2- 172.21.15.89:0/3291626849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30b00c70b0 0x7f30b0069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:20.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.109+0000 7f30b58e1700 1 --2- 172.21.15.89:0/3291626849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30b0155eb0 0x7f30b006e240 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009e90 tx=0x7f30ac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4282af7e1afb7d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:20.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.109+0000 7f30b48df700 1 -- 172.21.15.89:0/3291626849 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac00e070 con 0x7f30b0155eb0 2023-10-10T20:39:20.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.109+0000 7f30b48df700 1 -- 172.21.15.89:0/3291626849 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac00bbf0 con 0x7f30b0155eb0 2023-10-10T20:39:20.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.109+0000 7f30b48df700 1 -- 172.21.15.89:0/3291626849 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac012430 con 0x7f30b0155eb0 2023-10-10T20:39:20.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.109+0000 7f30b7b45700 1 -- 172.21.15.89:0/3291626849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30b0155eb0 msgr2=0x7f30b006e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:20.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.109+0000 7f30b7b45700 1 --2- 172.21.15.89:0/3291626849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30b0155eb0 0x7f30b006e240 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009e90 tx=0x7f30ac009780 comp rx=0 tx=0).stop 2023-10-10T20:39:20.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.109+0000 7f30b7b45700 1 -- 172.21.15.89:0/3291626849 shutdown_connections 2023-10-10T20:39:20.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.109+0000 7f30b7b45700 1 --2- 172.21.15.89:0/3291626849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30b0155eb0 0x7f30b006e240 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:20.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.109+0000 7f30b7b45700 1 --2- 172.21.15.89:0/3291626849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30b00c70b0 0x7f30b0069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:20.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.109+0000 7f30b7b45700 1 -- 172.21.15.89:0/3291626849 >> 172.21.15.89:0/3291626849 conn(0x7f30b0068e10 msgr2=0x7f30b00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:20.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.110+0000 7f30b7b45700 1 -- 172.21.15.89:0/3291626849 shutdown_connections 2023-10-10T20:39:20.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.110+0000 7f30b7b45700 1 -- 172.21.15.89:0/3291626849 wait complete. 2023-10-10T20:39:20.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.111+0000 7f30b7b45700 1 Processor -- start 2023-10-10T20:39:20.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.111+0000 7f30b7b45700 1 -- start start 2023-10-10T20:39:20.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.111+0000 7f30b7b45700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30b0069cb0 0x7f30b00c4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:20.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.111+0000 7f30b7b45700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30b00c70b0 0x7f30b00c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:20.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.111+0000 7f30b7b45700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30b0155eb0 0x7f30b00ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:20.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.111+0000 7f30b7b45700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b00c54d0 con 0x7f30b0155eb0 2023-10-10T20:39:20.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.111+0000 7f30b7b45700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b0157940 con 0x7f30b00c70b0 2023-10-10T20:39:20.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.111+0000 7f30b7b45700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b00c6970 con 0x7f30b0069cb0 2023-10-10T20:39:20.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.112+0000 7f30b60e2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30b0155eb0 0x7f30b00ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:20.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.112+0000 7f30b60e2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30b0155eb0 0x7f30b00ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43228/0 (socket says 172.21.15.89:43228) 2023-10-10T20:39:20.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.112+0000 7f30b60e2700 1 -- 172.21.15.89:0/3114682694 learned_addr learned my addr 172.21.15.89:0/3114682694 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:20.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.112+0000 7f30b50e0700 1 --2- 172.21.15.89:0/3114682694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30b00c70b0 0x7f30b00c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:20.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.112+0000 7f30b58e1700 1 --2- 172.21.15.89:0/3114682694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30b0069cb0 0x7f30b00c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:20.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.112+0000 7f30b60e2700 1 -- 172.21.15.89:0/3114682694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30b0069cb0 msgr2=0x7f30b00c4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:20.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.112+0000 7f30b60e2700 1 --2- 172.21.15.89:0/3114682694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30b0069cb0 0x7f30b00c4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:20.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.112+0000 7f30b60e2700 1 -- 172.21.15.89:0/3114682694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30b00c70b0 msgr2=0x7f30b00c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:20.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.112+0000 7f30b60e2700 1 --2- 172.21.15.89:0/3114682694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30b00c70b0 0x7f30b00c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:20.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.112+0000 7f30b60e2700 1 -- 172.21.15.89:0/3114682694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ac009a20 con 0x7f30b0155eb0 2023-10-10T20:39:20.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.112+0000 7f30b50e0700 1 --2- 172.21.15.89:0/3114682694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30b00c70b0 0x7f30b00c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:20.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.112+0000 7f30b58e1700 1 --2- 172.21.15.89:0/3114682694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30b0069cb0 0x7f30b00c4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:20.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.112+0000 7f30b60e2700 1 --2- 172.21.15.89:0/3114682694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30b0155eb0 0x7f30b00ca710 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f30a800c590 tx=0x7f30a8007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:20.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.112+0000 7f30a5ffb700 1 -- 172.21.15.89:0/3114682694 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a8005b30 con 0x7f30b0155eb0 2023-10-10T20:39:20.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.112+0000 7f30a5ffb700 1 -- 172.21.15.89:0/3114682694 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a8005cb0 con 0x7f30b0155eb0 2023-10-10T20:39:20.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.112+0000 7f30b7b45700 1 -- 172.21.15.89:0/3114682694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b00c48c0 con 0x7f30b0155eb0 2023-10-10T20:39:20.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.112+0000 7f30a5ffb700 1 -- 172.21.15.89:0/3114682694 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a8007320 con 0x7f30b0155eb0 2023-10-10T20:39:20.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.112+0000 7f30b7b45700 1 -- 172.21.15.89:0/3114682694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b00c4c50 con 0x7f30b0155eb0 2023-10-10T20:39:20.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.113+0000 7f30b7b45700 1 -- 172.21.15.89:0/3114682694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b004c3c0 con 0x7f30b0155eb0 2023-10-10T20:39:20.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.113+0000 7f30a5ffb700 1 -- 172.21.15.89:0/3114682694 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 8) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f30a801b3b0 con 0x7f30b0155eb0 2023-10-10T20:39:20.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.114+0000 7f30a5ffb700 1 --2- 172.21.15.89:0/3114682694 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3094066fc0 0x7f3094069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:20.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.114+0000 7f30a5ffb700 1 -- 172.21.15.89:0/3114682694 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f30a8083b40 con 0x7f30b0155eb0 2023-10-10T20:39:20.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.114+0000 7f30b58e1700 1 --2- 172.21.15.89:0/3114682694 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3094066fc0 0x7f3094069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:20.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.114+0000 7f30b58e1700 1 --2- 172.21.15.89:0/3114682694 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3094066fc0 0x7f3094069470 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009fd0 tx=0x7f30ac019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:20.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.118+0000 7f30a5ffb700 1 -- 172.21.15.89:0/3114682694 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30a8050d80 con 0x7f30b0155eb0 2023-10-10T20:39:20.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.335+0000 7f30b7b45700 1 -- 172.21.15.89:0/3114682694 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000078554", "target": ["mon-mgr", ""]}) v1 -- 0x7f30b00b2050 con 0x7f3094066fc0 2023-10-10T20:39:20.342 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:20.339+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:20.342 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:20.339+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:20.342 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:20.339+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:20.342 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:20.339+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:20.342 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:20.339+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:20.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.399+0000 7f30a5ffb700 1 -- 172.21.15.89:0/3114682694 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f30b00b2050 con 0x7f3094066fc0 2023-10-10T20:39:20.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.402+0000 7f30b7b45700 1 -- 172.21.15.89:0/3114682694 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3094066fc0 msgr2=0x7f3094069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:20.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.402+0000 7f30b7b45700 1 --2- 172.21.15.89:0/3114682694 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3094066fc0 0x7f3094069470 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009fd0 tx=0x7f30ac019000 comp rx=0 tx=0).stop 2023-10-10T20:39:20.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.402+0000 7f30b7b45700 1 -- 172.21.15.89:0/3114682694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30b0155eb0 msgr2=0x7f30b00ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:20.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.402+0000 7f30b7b45700 1 --2- 172.21.15.89:0/3114682694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30b0155eb0 0x7f30b00ca710 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f30a800c590 tx=0x7f30a8007ef0 comp rx=0 tx=0).stop 2023-10-10T20:39:20.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.402+0000 7f30b7b45700 1 -- 172.21.15.89:0/3114682694 shutdown_connections 2023-10-10T20:39:20.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.402+0000 7f30b7b45700 1 --2- 172.21.15.89:0/3114682694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30b0155eb0 0x7f30b00ca710 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:20.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.402+0000 7f30b7b45700 1 --2- 172.21.15.89:0/3114682694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30b0069cb0 0x7f30b00c4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:20.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.402+0000 7f30b7b45700 1 --2- 172.21.15.89:0/3114682694 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3094066fc0 0x7f3094069470 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:20.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.402+0000 7f30b7b45700 1 --2- 172.21.15.89:0/3114682694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30b00c70b0 0x7f30b00c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:20.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.402+0000 7f30b7b45700 1 -- 172.21.15.89:0/3114682694 >> 172.21.15.89:0/3114682694 conn(0x7f30b0068e10 msgr2=0x7f30b00af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:20.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.402+0000 7f30b7b45700 1 -- 172.21.15.89:0/3114682694 shutdown_connections 2023-10-10T20:39:20.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.402+0000 7f30b7b45700 1 -- 172.21.15.89:0/3114682694 wait complete. 2023-10-10T20:39:20.407 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:20.402+0000 7f023e8b0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:20.407 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:20.402+0000 7f023e8b0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:20.408 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:20.402+0000 7f023e8b0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:20.408 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:20.402+0000 7f023e8b0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:20.408 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:20.402+0000 7f023e8b0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:20.417 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs 2023-10-10T20:39:20.420 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:20.417+0000 7f023d8ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:20.420 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:20.417+0000 7f023d8ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:20.420 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:20.417+0000 7f023d8ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:20.421 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:20.417+0000 7f023d8ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:20.421 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:20.417+0000 7f023d8ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:20.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.559+0000 7f934dfa9700 1 Processor -- start 2023-10-10T20:39:20.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.559+0000 7f934dfa9700 1 -- start start 2023-10-10T20:39:20.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.560+0000 7f934dfa9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f93480b1b40 0x7f93480b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:20.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.560+0000 7f934dfa9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93480cfb00 con 0x7f93480b16d0 2023-10-10T20:39:20.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.560+0000 7f934dfa9700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93480b5c40 con 0x7f93480b4e70 2023-10-10T20:39:20.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.560+0000 7f934dfa9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93480b5f20 con 0x7f93480b1b40 2023-10-10T20:39:20.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.560+0000 7f9347fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f93480b16d0 0x7f93480b33b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:32864/0 (socket says 172.21.15.89:32864) 2023-10-10T20:39:20.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.560+0000 7f9347fff700 1 -- 172.21.15.89:0/3962179552 learned_addr learned my addr 172.21.15.89:0/3962179552 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:20.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.561+0000 7f934d7a8700 1 --2- 172.21.15.89:0/3962179552 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f93480b1b40 0x7f93480b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:20.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.561+0000 7f93477fe700 1 -- 172.21.15.89:0/3962179552 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112151542 0 0) 0x7f93480cfb00 con 0x7f93480b16d0 2023-10-10T20:39:20.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.561+0000 7f93477fe700 1 -- 172.21.15.89:0/3962179552 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9338003570 con 0x7f93480b16d0 2023-10-10T20:39:20.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.561+0000 7f93477fe700 1 -- 172.21.15.89:0/3962179552 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175240557 0 0) 0x7f93480b5c40 con 0x7f93480b4e70 2023-10-10T20:39:20.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.561+0000 7f93477fe700 1 -- 172.21.15.89:0/3962179552 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93480cfb00 con 0x7f93480b4e70 2023-10-10T20:39:20.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.561+0000 7f934d7a8700 1 -- 172.21.15.89:0/3962179552 >> v1:172.21.15.111:6789/0 conn(0x7f93480b4e70 legacy=0x7f93480ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:20.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.561+0000 7f934d7a8700 1 -- 172.21.15.89:0/3962179552 >> v1:172.21.15.89:6789/0 conn(0x7f93480b16d0 legacy=0x7f93480b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:20.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.561+0000 7f934d7a8700 1 -- 172.21.15.89:0/3962179552 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93480b6200 con 0x7f93480b1b40 2023-10-10T20:39:20.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.561+0000 7f934d7a8700 1 --2- 172.21.15.89:0/3962179552 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f93480b1b40 0x7f93480b3ac0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9340009e90 tx=0x7f9340009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d55be3d3b249126 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:20.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.561+0000 7f93477fe700 1 -- 172.21.15.89:0/3962179552 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934000e070 con 0x7f93480b1b40 2023-10-10T20:39:20.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.561+0000 7f93477fe700 1 -- 172.21.15.89:0/3962179552 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934000bb80 con 0x7f93480b1b40 2023-10-10T20:39:20.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.561+0000 7f93477fe700 1 -- 172.21.15.89:0/3962179552 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340012430 con 0x7f93480b1b40 2023-10-10T20:39:20.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.562+0000 7f934dfa9700 1 -- 172.21.15.89:0/3962179552 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f93480b1b40 msgr2=0x7f93480b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:20.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.562+0000 7f934dfa9700 1 --2- 172.21.15.89:0/3962179552 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f93480b1b40 0x7f93480b3ac0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9340009e90 tx=0x7f9340009780 comp rx=0 tx=0).stop 2023-10-10T20:39:20.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.562+0000 7f934dfa9700 1 -- 172.21.15.89:0/3962179552 shutdown_connections 2023-10-10T20:39:20.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.562+0000 7f934dfa9700 1 --2- 172.21.15.89:0/3962179552 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f93480b1b40 0x7f93480b3ac0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:20.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.562+0000 7f934dfa9700 1 -- 172.21.15.89:0/3962179552 >> 172.21.15.89:0/3962179552 conn(0x7f93480d10c0 msgr2=0x7f93480affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:20.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.562+0000 7f934dfa9700 1 -- 172.21.15.89:0/3962179552 shutdown_connections 2023-10-10T20:39:20.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.562+0000 7f934dfa9700 1 -- 172.21.15.89:0/3962179552 wait complete. 2023-10-10T20:39:20.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.563+0000 7f934dfa9700 1 Processor -- start 2023-10-10T20:39:20.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.563+0000 7f934dfa9700 1 -- start start 2023-10-10T20:39:20.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.564+0000 7f934dfa9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f93480b16d0 0x7f93480c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:20.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.564+0000 7f934dfa9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93480b1b40 0x7f93480c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:20.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.564+0000 7f934dfa9700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93480b4e70 0x7f93480c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:20.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.564+0000 7f934dfa9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93480c6820 con 0x7f93480b1b40 2023-10-10T20:39:20.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.564+0000 7f934dfa9700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93480c7300 con 0x7f93480b4e70 2023-10-10T20:39:20.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.564+0000 7f934dfa9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93480c6a90 con 0x7f93480b16d0 2023-10-10T20:39:20.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.564+0000 7f9347fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93480b1b40 0x7f93480c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:20.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.564+0000 7f9347fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93480b1b40 0x7f93480c5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43278/0 (socket says 172.21.15.89:43278) 2023-10-10T20:39:20.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.564+0000 7f9347fff700 1 -- 172.21.15.89:0/3590321796 learned_addr learned my addr 172.21.15.89:0/3590321796 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:20.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.564+0000 7f934d7a8700 1 --2- 172.21.15.89:0/3590321796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93480b4e70 0x7f93480c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:20.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.564+0000 7f934cfa7700 1 --2- 172.21.15.89:0/3590321796 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f93480b16d0 0x7f93480c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:20.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.565+0000 7f9347fff700 1 -- 172.21.15.89:0/3590321796 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f93480b16d0 msgr2=0x7f93480c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:20.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.565+0000 7f9347fff700 1 --2- 172.21.15.89:0/3590321796 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f93480b16d0 0x7f93480c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:20.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.565+0000 7f9347fff700 1 -- 172.21.15.89:0/3590321796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93480b4e70 msgr2=0x7f93480c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:20.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.565+0000 7f9347fff700 1 --2- 172.21.15.89:0/3590321796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93480b4e70 0x7f93480c62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:20.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.565+0000 7f9347fff700 1 -- 172.21.15.89:0/3590321796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9340009a20 con 0x7f93480b1b40 2023-10-10T20:39:20.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.565+0000 7f9347fff700 1 --2- 172.21.15.89:0/3590321796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93480b1b40 0x7f93480c5da0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f933400c790 tx=0x7f9334007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:20.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.565+0000 7f9344ff9700 1 -- 172.21.15.89:0/3590321796 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334005b30 con 0x7f93480b1b40 2023-10-10T20:39:20.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.565+0000 7f9344ff9700 1 -- 172.21.15.89:0/3590321796 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9334005c90 con 0x7f93480b1b40 2023-10-10T20:39:20.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.565+0000 7f934dfa9700 1 -- 172.21.15.89:0/3590321796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93480c6d70 con 0x7f93480b1b40 2023-10-10T20:39:20.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.565+0000 7f9344ff9700 1 -- 172.21.15.89:0/3590321796 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334007300 con 0x7f93480b1b40 2023-10-10T20:39:20.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.565+0000 7f934dfa9700 1 -- 172.21.15.89:0/3590321796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93480c7880 con 0x7f93480b1b40 2023-10-10T20:39:20.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.566+0000 7f934dfa9700 1 -- 172.21.15.89:0/3590321796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9348088990 con 0x7f93480b1b40 2023-10-10T20:39:20.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.567+0000 7f9344ff9700 1 -- 172.21.15.89:0/3590321796 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 8) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f933401b3b0 con 0x7f93480b1b40 2023-10-10T20:39:20.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.567+0000 7f9344ff9700 1 --2- 172.21.15.89:0/3590321796 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f932c066fc0 0x7f932c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:20.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.567+0000 7f9344ff9700 1 -- 172.21.15.89:0/3590321796 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f9334083a40 con 0x7f93480b1b40 2023-10-10T20:39:20.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.567+0000 7f934cfa7700 1 --2- 172.21.15.89:0/3590321796 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f932c066fc0 0x7f932c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:20.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.568+0000 7f934cfa7700 1 --2- 172.21.15.89:0/3590321796 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f932c066fc0 0x7f932c069470 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f933c007fd0 tx=0x7f933c007d10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:20.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.573+0000 7f9344ff9700 1 -- 172.21.15.89:0/3590321796 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9334054710 con 0x7f93480b1b40 2023-10-10T20:39:20.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.858+0000 7f934dfa9700 1 -- 172.21.15.89:0/3590321796 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f934804bb80 con 0x7f932c066fc0 2023-10-10T20:39:20.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.869+0000 7f9344ff9700 1 -- 172.21.15.89:0/3590321796 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+538 (secure 0 0 0) 0x7f934804bb80 con 0x7f932c066fc0 2023-10-10T20:39:20.872 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:39:20.872 INFO:teuthology.orchestra.run.smithi089.stdout: "mon_addrs": [ 2023-10-10T20:39:20.872 INFO:teuthology.orchestra.run.smithi089.stdout: "172.21.15.89:6789", 2023-10-10T20:39:20.872 INFO:teuthology.orchestra.run.smithi089.stdout: "172.21.15.111:6789", 2023-10-10T20:39:20.873 INFO:teuthology.orchestra.run.smithi089.stdout: "172.21.15.111:6790" 2023-10-10T20:39:20.873 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-10T20:39:20.873 INFO:teuthology.orchestra.run.smithi089.stdout: "pending_subvolume_deletions": 1, 2023-10-10T20:39:20.873 INFO:teuthology.orchestra.run.smithi089.stdout: "pools": { 2023-10-10T20:39:20.873 INFO:teuthology.orchestra.run.smithi089.stdout: "data": [ 2023-10-10T20:39:20.874 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:20.874 INFO:teuthology.orchestra.run.smithi089.stdout: "avail": 365974454272, 2023-10-10T20:39:20.874 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs_data", 2023-10-10T20:39:20.874 INFO:teuthology.orchestra.run.smithi089.stdout: "used": 8192 2023-10-10T20:39:20.874 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:20.875 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-10T20:39:20.875 INFO:teuthology.orchestra.run.smithi089.stdout: "metadata": [ 2023-10-10T20:39:20.875 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:20.875 INFO:teuthology.orchestra.run.smithi089.stdout: "avail": 365974454272, 2023-10-10T20:39:20.875 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs_metadata", 2023-10-10T20:39:20.876 INFO:teuthology.orchestra.run.smithi089.stdout: "used": 20627456 2023-10-10T20:39:20.876 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:20.876 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-10T20:39:20.876 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:20.876 INFO:teuthology.orchestra.run.smithi089.stdout: "used_size": 145 2023-10-10T20:39:20.876 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:39:20.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.871+0000 7f934dfa9700 1 -- 172.21.15.89:0/3590321796 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f932c066fc0 msgr2=0x7f932c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:20.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.871+0000 7f934dfa9700 1 --2- 172.21.15.89:0/3590321796 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f932c066fc0 0x7f932c069470 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f933c007fd0 tx=0x7f933c007d10 comp rx=0 tx=0).stop 2023-10-10T20:39:20.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.871+0000 7f934dfa9700 1 -- 172.21.15.89:0/3590321796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93480b1b40 msgr2=0x7f93480c5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:20.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.871+0000 7f934dfa9700 1 --2- 172.21.15.89:0/3590321796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93480b1b40 0x7f93480c5da0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f933400c790 tx=0x7f9334007ea0 comp rx=0 tx=0).stop 2023-10-10T20:39:20.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.872+0000 7f934dfa9700 1 -- 172.21.15.89:0/3590321796 shutdown_connections 2023-10-10T20:39:20.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.872+0000 7f934dfa9700 1 --2- 172.21.15.89:0/3590321796 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f932c066fc0 0x7f932c069470 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:20.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.872+0000 7f934dfa9700 1 --2- 172.21.15.89:0/3590321796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93480b4e70 0x7f93480c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:20.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.872+0000 7f934dfa9700 1 --2- 172.21.15.89:0/3590321796 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f93480b16d0 0x7f93480c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:20.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.872+0000 7f934dfa9700 1 --2- 172.21.15.89:0/3590321796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93480b1b40 0x7f93480c5da0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:20.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.872+0000 7f934dfa9700 1 -- 172.21.15.89:0/3590321796 >> 172.21.15.89:0/3590321796 conn(0x7f93480d10c0 msgr2=0x7f9348159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:20.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.872+0000 7f934dfa9700 1 -- 172.21.15.89:0/3590321796 shutdown_connections 2023-10-10T20:39:20.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:20.872+0000 7f934dfa9700 1 -- 172.21.15.89:0/3590321796 wait complete. 2023-10-10T20:39:20.887 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:39:20.887 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:39:20.927 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:39:20.968 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:39:20.968 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:39:20.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:20.969 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:39:21.515 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:39:21.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:39:21.517 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:39:21.534 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:39:21.535 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:21.535 DEBUG:teuthology.orchestra.run.smithi111:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:39:22.203 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:39:22.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:39:22.204 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:39:22.222 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:39:22.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.369+0000 7f58f8e0a700 1 Processor -- start 2023-10-10T20:39:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.369+0000 7f58f8e0a700 1 -- start start 2023-10-10T20:39:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.370+0000 7f58f8e0a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f58f406e8c0 0x7f58f40c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.370+0000 7f58f8e0a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f58f40c8660 0x7f58f40b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.370+0000 7f58f8e0a700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f415aa30 con 0x7f58f4153df0 2023-10-10T20:39:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.370+0000 7f58f8e0a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f40c8a60 con 0x7f58f406e8c0 2023-10-10T20:39:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.370+0000 7f58f8e0a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f415a100 con 0x7f58f40c8660 2023-10-10T20:39:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.370+0000 7f58f2ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f58f406e8c0 0x7f58f40c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.370+0000 7f58f2ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f58f406e8c0 0x7f58f40c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43288/0 (socket says 172.21.15.89:43288) 2023-10-10T20:39:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.370+0000 7f58f2ffd700 1 -- 172.21.15.89:0/4153353190 learned_addr learned my addr 172.21.15.89:0/4153353190 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.370+0000 7f58f3fff700 1 --2- 172.21.15.89:0/4153353190 >> v2:172.21.15.111:3300/0 conn(0x7f58f40c8660 0x7f58f40b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.371+0000 7f58f2ffd700 1 -- 172.21.15.89:0/4153353190 >> v2:172.21.15.111:3300/0 conn(0x7f58f40c8660 msgr2=0x7f58f40b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.371+0000 7f58f2ffd700 1 --2- 172.21.15.89:0/4153353190 >> v2:172.21.15.111:3300/0 conn(0x7f58f40c8660 0x7f58f40b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.371+0000 7f58f2ffd700 1 -- 172.21.15.89:0/4153353190 >> v1:172.21.15.111:6789/0 conn(0x7f58f4153df0 legacy=0x7f58f4063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.371+0000 7f58f2ffd700 1 -- 172.21.15.89:0/4153353190 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f415a3e0 con 0x7f58f406e8c0 2023-10-10T20:39:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.371+0000 7f58f3fff700 1 --2- 172.21.15.89:0/4153353190 >> v2:172.21.15.111:3300/0 conn(0x7f58f40c8660 0x7f58f40b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:22.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.371+0000 7f58f2ffd700 1 --2- 172.21.15.89:0/4153353190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58f406e8c0 0x7f58f40c8120 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f58f406dd40 tx=0x7f58e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2912ebff0e3c6224 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:22.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.371+0000 7f58f27fc700 1 -- 172.21.15.89:0/4153353190 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e800b820 con 0x7f58f406e8c0 2023-10-10T20:39:22.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.371+0000 7f58f27fc700 1 -- 172.21.15.89:0/4153353190 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e800b980 con 0x7f58f406e8c0 2023-10-10T20:39:22.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.371+0000 7f58f27fc700 1 -- 172.21.15.89:0/4153353190 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e800bd20 con 0x7f58f406e8c0 2023-10-10T20:39:22.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.371+0000 7f58f8e0a700 1 -- 172.21.15.89:0/4153353190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58f406e8c0 msgr2=0x7f58f40c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:22.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.371+0000 7f58f8e0a700 1 --2- 172.21.15.89:0/4153353190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58f406e8c0 0x7f58f40c8120 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f58f406dd40 tx=0x7f58e8009250 comp rx=0 tx=0).stop 2023-10-10T20:39:22.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.372+0000 7f58f8e0a700 1 -- 172.21.15.89:0/4153353190 shutdown_connections 2023-10-10T20:39:22.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.372+0000 7f58f8e0a700 1 --2- 172.21.15.89:0/4153353190 >> v2:172.21.15.111:3300/0 conn(0x7f58f40c8660 0x7f58f40b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:22.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.372+0000 7f58f8e0a700 1 --2- 172.21.15.89:0/4153353190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58f406e8c0 0x7f58f40c8120 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:22.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.372+0000 7f58f8e0a700 1 -- 172.21.15.89:0/4153353190 >> 172.21.15.89:0/4153353190 conn(0x7f58f406ddb0 msgr2=0x7f58f406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:22.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.372+0000 7f58f8e0a700 1 -- 172.21.15.89:0/4153353190 shutdown_connections 2023-10-10T20:39:22.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.372+0000 7f58f8e0a700 1 -- 172.21.15.89:0/4153353190 wait complete. 2023-10-10T20:39:22.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.374+0000 7f58f8e0a700 1 Processor -- start 2023-10-10T20:39:22.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.374+0000 7f58f8e0a700 1 -- start start 2023-10-10T20:39:22.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.375+0000 7f58f8e0a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f58f406e8c0 0x7f58f416a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:22.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.375+0000 7f58f8e0a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58f40c8660 0x7f58f416ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:22.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.375+0000 7f58f8e0a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f4153df0 0x7f58f415bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:22.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.375+0000 7f58f8e0a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40c5130 con 0x7f58f40c8660 2023-10-10T20:39:22.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.375+0000 7f58f8e0a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f4154360 con 0x7f58f4153df0 2023-10-10T20:39:22.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.375+0000 7f58f8e0a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f416abe0 con 0x7f58f406e8c0 2023-10-10T20:39:22.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.375+0000 7f58f2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58f40c8660 0x7f58f416ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:22.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.375+0000 7f58f2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58f40c8660 0x7f58f416ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43300/0 (socket says 172.21.15.89:43300) 2023-10-10T20:39:22.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.375+0000 7f58f2ffd700 1 -- 172.21.15.89:0/2116644128 learned_addr learned my addr 172.21.15.89:0/2116644128 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:22.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.375+0000 7f58f3fff700 1 --2- 172.21.15.89:0/2116644128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f4153df0 0x7f58f415bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:22.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.375+0000 7f58f37fe700 1 --2- 172.21.15.89:0/2116644128 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f58f406e8c0 0x7f58f416a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:22.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.375+0000 7f58f2ffd700 1 -- 172.21.15.89:0/2116644128 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f58f406e8c0 msgr2=0x7f58f416a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:22.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.375+0000 7f58f2ffd700 1 --2- 172.21.15.89:0/2116644128 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f58f406e8c0 0x7f58f416a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:22.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.375+0000 7f58f2ffd700 1 -- 172.21.15.89:0/2116644128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f4153df0 msgr2=0x7f58f415bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:22.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.375+0000 7f58f2ffd700 1 --2- 172.21.15.89:0/2116644128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f4153df0 0x7f58f415bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:22.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.375+0000 7f58f2ffd700 1 -- 172.21.15.89:0/2116644128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e80094f0 con 0x7f58f40c8660 2023-10-10T20:39:22.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.375+0000 7f58f2ffd700 1 --2- 172.21.15.89:0/2116644128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58f40c8660 0x7f58f416ae40 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f58f406c7f0 tx=0x7f58e8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:22.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.375+0000 7f58db7fe700 1 -- 172.21.15.89:0/2116644128 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8011e20 con 0x7f58f40c8660 2023-10-10T20:39:22.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.376+0000 7f58db7fe700 1 -- 172.21.15.89:0/2116644128 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e8020070 con 0x7f58f40c8660 2023-10-10T20:39:22.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.376+0000 7f58db7fe700 1 -- 172.21.15.89:0/2116644128 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e801b6a0 con 0x7f58f40c8660 2023-10-10T20:39:22.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.376+0000 7f58f8e0a700 1 -- 172.21.15.89:0/2116644128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f40c2a50 con 0x7f58f40c8660 2023-10-10T20:39:22.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.376+0000 7f58f8e0a700 1 -- 172.21.15.89:0/2116644128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f40c3010 con 0x7f58f40c8660 2023-10-10T20:39:22.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.377+0000 7f58f8e0a700 1 -- 172.21.15.89:0/2116644128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f404c3c0 con 0x7f58f40c8660 2023-10-10T20:39:22.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.377+0000 7f58db7fe700 1 -- 172.21.15.89:0/2116644128 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f58e8022030 con 0x7f58f40c8660 2023-10-10T20:39:22.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.378+0000 7f58db7fe700 1 --2- 172.21.15.89:0/2116644128 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f58e0067050 0x7f58e0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:22.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.378+0000 7f58f37fe700 1 --2- 172.21.15.89:0/2116644128 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f58e0067050 0x7f58e0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:22.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.378+0000 7f58db7fe700 1 -- 172.21.15.89:0/2116644128 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f58e8086650 con 0x7f58f40c8660 2023-10-10T20:39:22.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.378+0000 7f58f37fe700 1 --2- 172.21.15.89:0/2116644128 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f58e0067050 0x7f58e0069500 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f58e4006fd0 tx=0x7f58e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:22.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.384+0000 7f58db7fe700 1 -- 172.21.15.89:0/2116644128 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f58e8053770 con 0x7f58f40c8660 2023-10-10T20:39:22.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.673+0000 7f58f8e0a700 1 -- 172.21.15.89:0/2116644128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58f404c3c0 con 0x7f58f40c8660 2023-10-10T20:39:22.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.674+0000 7f58db7fe700 1 -- 172.21.15.89:0/2116644128 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f58e8056d90 con 0x7f58f40c8660 2023-10-10T20:39:22.676 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:22.677 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/3362909084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":3362909084},{"type":"v1","addr":"172.21.15.111:6835","nonce":3362909084}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5102,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1472423656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1472423656},{"type":"v1","addr":"172.21.15.111:6834","nonce":1472423656}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/3183325104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3183325104},{"type":"v1","addr":"172.21.15.89:6836","nonce":3183325104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:36:20.867329+0000","modified":"2023-10-10T20:36:24.518115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/862652054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":862652054},{"type":"v1","addr":"172.21.15.89:6837","nonce":862652054}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-10T20:39:22.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.676+0000 7f58f8e0a700 1 -- 172.21.15.89:0/2116644128 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f58e0067050 msgr2=0x7f58e0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:22.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.676+0000 7f58f8e0a700 1 --2- 172.21.15.89:0/2116644128 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f58e0067050 0x7f58e0069500 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f58e4006fd0 tx=0x7f58e4008040 comp rx=0 tx=0).stop 2023-10-10T20:39:22.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.676+0000 7f58f8e0a700 1 -- 172.21.15.89:0/2116644128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58f40c8660 msgr2=0x7f58f416ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:22.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.676+0000 7f58f8e0a700 1 --2- 172.21.15.89:0/2116644128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58f40c8660 0x7f58f416ae40 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f58f406c7f0 tx=0x7f58e8016000 comp rx=0 tx=0).stop 2023-10-10T20:39:22.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.677+0000 7f58f8e0a700 1 -- 172.21.15.89:0/2116644128 shutdown_connections 2023-10-10T20:39:22.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.677+0000 7f58f8e0a700 1 --2- 172.21.15.89:0/2116644128 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f58e0067050 0x7f58e0069500 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:22.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.677+0000 7f58f8e0a700 1 --2- 172.21.15.89:0/2116644128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f4153df0 0x7f58f415bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:22.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.677+0000 7f58f8e0a700 1 --2- 172.21.15.89:0/2116644128 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f58f406e8c0 0x7f58f416a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:22.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.677+0000 7f58f8e0a700 1 --2- 172.21.15.89:0/2116644128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58f40c8660 0x7f58f416ae40 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:22.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.677+0000 7f58f8e0a700 1 -- 172.21.15.89:0/2116644128 >> 172.21.15.89:0/2116644128 conn(0x7f58f406ddb0 msgr2=0x7f58f415af40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:22.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.677+0000 7f58f8e0a700 1 -- 172.21.15.89:0/2116644128 shutdown_connections 2023-10-10T20:39:22.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.677+0000 7f58f8e0a700 1 -- 172.21.15.89:0/2116644128 wait complete. 2023-10-10T20:39:22.681 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 38 2023-10-10T20:39:22.692 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:39:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.821+0000 7f134f646700 1 Processor -- start 2023-10-10T20:39:22.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.821+0000 7f134f646700 1 -- start start 2023-10-10T20:39:22.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.822+0000 7f134f646700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1348159890 0x7f134815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:22.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.822+0000 7f134f646700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1348159c60 0x7f134815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:22.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.822+0000 7f134f646700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f134815a160 con 0x7f134815db10 2023-10-10T20:39:22.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.822+0000 7f134f646700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13480d13a0 con 0x7f1348159890 2023-10-10T20:39:22.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.822+0000 7f134f646700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134815e140 con 0x7f1348159c60 2023-10-10T20:39:22.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.823+0000 7f134cbe1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1348159c60 0x7f134815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:22.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.823+0000 7f134d3e2700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1348159890 0x7f134815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:22.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.823+0000 7f134cbe1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1348159c60 0x7f134815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:52010/0 (socket says 172.21.15.89:52010) 2023-10-10T20:39:22.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.823+0000 7f134cbe1700 1 -- 172.21.15.89:0/2716298473 learned_addr learned my addr 172.21.15.89:0/2716298473 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:22.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.823+0000 7f133ffff700 1 -- 172.21.15.89:0/2716298473 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3222332089 0 0) 0x7f134815a160 con 0x7f134815db10 2023-10-10T20:39:22.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.823+0000 7f133ffff700 1 -- 172.21.15.89:0/2716298473 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1334003570 con 0x7f134815db10 2023-10-10T20:39:22.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.823+0000 7f134d3e2700 1 -- 172.21.15.89:0/2716298473 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1348159c60 msgr2=0x7f134815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:22.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.823+0000 7f134d3e2700 1 --2- 172.21.15.89:0/2716298473 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1348159c60 0x7f134815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:22.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.823+0000 7f134d3e2700 1 -- 172.21.15.89:0/2716298473 >> v1:172.21.15.111:6789/0 conn(0x7f134815db10 legacy=0x7f1348069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:22.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.823+0000 7f134d3e2700 1 -- 172.21.15.89:0/2716298473 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134815a3d0 con 0x7f1348159890 2023-10-10T20:39:22.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.823+0000 7f134cbe1700 1 --2- 172.21.15.89:0/2716298473 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1348159c60 0x7f134815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:39:22.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.823+0000 7f134d3e2700 1 --2- 172.21.15.89:0/2716298473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1348159890 0x7f134815abd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f13440098e0 tx=0x7f13440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bee87502231ad13 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:22.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.824+0000 7f133ffff700 1 -- 172.21.15.89:0/2716298473 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1344011630 con 0x7f1348159890 2023-10-10T20:39:22.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.824+0000 7f133ffff700 1 -- 172.21.15.89:0/2716298473 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1344011790 con 0x7f1348159890 2023-10-10T20:39:22.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.824+0000 7f133ffff700 1 -- 172.21.15.89:0/2716298473 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1344011b10 con 0x7f1348159890 2023-10-10T20:39:22.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.824+0000 7f134f646700 1 -- 172.21.15.89:0/2716298473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1348159890 msgr2=0x7f134815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:22.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.824+0000 7f134f646700 1 --2- 172.21.15.89:0/2716298473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1348159890 0x7f134815abd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f13440098e0 tx=0x7f13440091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:22.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.824+0000 7f134f646700 1 -- 172.21.15.89:0/2716298473 shutdown_connections 2023-10-10T20:39:22.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.824+0000 7f134f646700 1 --2- 172.21.15.89:0/2716298473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1348159890 0x7f134815abd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:22.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.824+0000 7f134f646700 1 --2- 172.21.15.89:0/2716298473 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1348159c60 0x7f134815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:22.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.824+0000 7f134f646700 1 -- 172.21.15.89:0/2716298473 >> 172.21.15.89:0/2716298473 conn(0x7f1348068e20 msgr2=0x7f13480d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:22.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.824+0000 7f134f646700 1 -- 172.21.15.89:0/2716298473 shutdown_connections 2023-10-10T20:39:22.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.824+0000 7f134f646700 1 -- 172.21.15.89:0/2716298473 wait complete. 2023-10-10T20:39:22.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.826+0000 7f134f646700 1 Processor -- start 2023-10-10T20:39:22.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.826+0000 7f134f646700 1 -- start start 2023-10-10T20:39:22.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.827+0000 7f134f646700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1348159890 0x7f13480b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:22.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.827+0000 7f134f646700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1348159c60 0x7f13480b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:22.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.827+0000 7f134f646700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f134815db10 0x7f13480b6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:22.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.827+0000 7f134f646700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1348152d90 con 0x7f134815db10 2023-10-10T20:39:22.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.827+0000 7f134f646700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1348152180 con 0x7f1348159890 2023-10-10T20:39:22.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.827+0000 7f134f646700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1348152460 con 0x7f1348159c60 2023-10-10T20:39:22.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.827+0000 7f134dbe3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f134815db10 0x7f13480b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:22.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.827+0000 7f134dbe3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f134815db10 0x7f13480b6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43310/0 (socket says 172.21.15.89:43310) 2023-10-10T20:39:22.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.827+0000 7f134dbe3700 1 -- 172.21.15.89:0/3440753701 learned_addr learned my addr 172.21.15.89:0/3440753701 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:22.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.827+0000 7f134d3e2700 1 --2- 172.21.15.89:0/3440753701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1348159890 0x7f13480b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:22.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.827+0000 7f134cbe1700 1 --2- 172.21.15.89:0/3440753701 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1348159c60 0x7f13480b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:22.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.827+0000 7f134dbe3700 1 -- 172.21.15.89:0/3440753701 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1348159c60 msgr2=0x7f13480b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:22.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.827+0000 7f134dbe3700 1 --2- 172.21.15.89:0/3440753701 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1348159c60 0x7f13480b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:22.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.827+0000 7f134dbe3700 1 -- 172.21.15.89:0/3440753701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1348159890 msgr2=0x7f13480b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:22.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.827+0000 7f134dbe3700 1 --2- 172.21.15.89:0/3440753701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1348159890 0x7f13480b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:22.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.827+0000 7f134dbe3700 1 -- 172.21.15.89:0/3440753701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13380094f0 con 0x7f134815db10 2023-10-10T20:39:22.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.827+0000 7f134dbe3700 1 --2- 172.21.15.89:0/3440753701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f134815db10 0x7f13480b6020 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f134000c860 tx=0x7f1340002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:22.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.827+0000 7f133d7fa700 1 -- 172.21.15.89:0/3440753701 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340002030 con 0x7f134815db10 2023-10-10T20:39:22.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.828+0000 7f133d7fa700 1 -- 172.21.15.89:0/3440753701 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13400021b0 con 0x7f134815db10 2023-10-10T20:39:22.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.828+0000 7f133d7fa700 1 -- 172.21.15.89:0/3440753701 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340012630 con 0x7f134815db10 2023-10-10T20:39:22.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.828+0000 7f134f646700 1 -- 172.21.15.89:0/3440753701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1344009420 con 0x7f134815db10 2023-10-10T20:39:22.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.828+0000 7f134f646700 1 -- 172.21.15.89:0/3440753701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1348152a20 con 0x7f134815db10 2023-10-10T20:39:22.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.829+0000 7f134f646700 1 -- 172.21.15.89:0/3440753701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f134804c3c0 con 0x7f134815db10 2023-10-10T20:39:22.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.829+0000 7f133d7fa700 1 -- 172.21.15.89:0/3440753701 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f1340012a20 con 0x7f134815db10 2023-10-10T20:39:22.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.830+0000 7f133d7fa700 1 --2- 172.21.15.89:0/3440753701 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f132c067050 0x7f132c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:22.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.830+0000 7f133d7fa700 1 -- 172.21.15.89:0/3440753701 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f1340084750 con 0x7f134815db10 2023-10-10T20:39:22.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.830+0000 7f134d3e2700 1 --2- 172.21.15.89:0/3440753701 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f132c067050 0x7f132c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:22.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.830+0000 7f134d3e2700 1 --2- 172.21.15.89:0/3440753701 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f132c067050 0x7f132c069500 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f13440091e0 tx=0x7f1344017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:22.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:22.835+0000 7f133d7fa700 1 -- 172.21.15.89:0/3440753701 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1340051870 con 0x7f134815db10 2023-10-10T20:39:23.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.063+0000 7f134f646700 1 -- 172.21.15.89:0/3440753701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13480c7660 con 0x7f134815db10 2023-10-10T20:39:23.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.064+0000 7f133d7fa700 1 -- 172.21.15.89:0/3440753701 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+15451 (secure 0 0 0) 0x7f1340054e90 con 0x7f134815db10 2023-10-10T20:39:23.067 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:23.067 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":64,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:37:23.989105+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:36:19.221740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-10T20:36:19.834213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.30","mappings":[{"from":6,"to":0}]},{"pgid":"11.3c","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:39:23.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.067+0000 7f134f646700 1 -- 172.21.15.89:0/3440753701 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f132c067050 msgr2=0x7f132c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:23.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.067+0000 7f134f646700 1 --2- 172.21.15.89:0/3440753701 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f132c067050 0x7f132c069500 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f13440091e0 tx=0x7f1344017000 comp rx=0 tx=0).stop 2023-10-10T20:39:23.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.067+0000 7f134f646700 1 -- 172.21.15.89:0/3440753701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f134815db10 msgr2=0x7f13480b6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:23.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.067+0000 7f134f646700 1 --2- 172.21.15.89:0/3440753701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f134815db10 0x7f13480b6020 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f134000c860 tx=0x7f1340002700 comp rx=0 tx=0).stop 2023-10-10T20:39:23.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.067+0000 7f134f646700 1 -- 172.21.15.89:0/3440753701 shutdown_connections 2023-10-10T20:39:23.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.067+0000 7f134f646700 1 --2- 172.21.15.89:0/3440753701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f134815db10 0x7f13480b6020 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.067+0000 7f134f646700 1 --2- 172.21.15.89:0/3440753701 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f132c067050 0x7f132c069500 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.067+0000 7f134f646700 1 --2- 172.21.15.89:0/3440753701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1348159890 0x7f13480b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.067+0000 7f134f646700 1 --2- 172.21.15.89:0/3440753701 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1348159c60 0x7f13480b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.067+0000 7f134f646700 1 -- 172.21.15.89:0/3440753701 >> 172.21.15.89:0/3440753701 conn(0x7f1348068e20 msgr2=0x7f13480c6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:23.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.067+0000 7f134f646700 1 -- 172.21.15.89:0/3440753701 shutdown_connections 2023-10-10T20:39:23.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.067+0000 7f134f646700 1 -- 172.21.15.89:0/3440753701 wait complete. 2023-10-10T20:39:23.083 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:36:19.221740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:36:19.834213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:39:23.084 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:39:23.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.212+0000 7f3fe66d3700 1 Processor -- start 2023-10-10T20:39:23.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.212+0000 7f3fe66d3700 1 -- start start 2023-10-10T20:39:23.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.213+0000 7f3fe66d3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3fe0159a60 0x7f3fe00c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:23.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.213+0000 7f3fe66d3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3fe00ccc00 0x7f3fe0069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:23.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.213+0000 7f3fe66d3700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe00c9cb0 con 0x7f3fe006a020 2023-10-10T20:39:23.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.213+0000 7f3fe66d3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe006dfc0 con 0x7f3fe00ccc00 2023-10-10T20:39:23.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.213+0000 7f3fe66d3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00c9f30 con 0x7f3fe0159a60 2023-10-10T20:39:23.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.213+0000 7f3fe4ed0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3fe00ccc00 0x7f3fe0069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:23.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.213+0000 7f3fe4ed0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3fe00ccc00 0x7f3fe0069ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43312/0 (socket says 172.21.15.89:43312) 2023-10-10T20:39:23.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.213+0000 7f3fe4ed0700 1 -- 172.21.15.89:0/3713008518 learned_addr learned my addr 172.21.15.89:0/3713008518 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:23.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.213+0000 7f3fe56d1700 1 --2- 172.21.15.89:0/3713008518 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3fe0159a60 0x7f3fe00c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:23.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.213+0000 7f3fe4ed0700 1 -- 172.21.15.89:0/3713008518 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3fe0159a60 msgr2=0x7f3fe00c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:23.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.213+0000 7f3fe4ed0700 1 --2- 172.21.15.89:0/3713008518 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3fe0159a60 0x7f3fe00c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.213+0000 7f3fe4ed0700 1 -- 172.21.15.89:0/3713008518 >> v1:172.21.15.111:6789/0 conn(0x7f3fe006a020 legacy=0x7f3fe00cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:23.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.213+0000 7f3fe4ed0700 1 -- 172.21.15.89:0/3713008518 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe00ca470 con 0x7f3fe00ccc00 2023-10-10T20:39:23.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.213+0000 7f3fe56d1700 1 --2- 172.21.15.89:0/3713008518 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3fe0159a60 0x7f3fe00c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:23.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.214+0000 7f3fe4ed0700 1 --2- 172.21.15.89:0/3713008518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fe00ccc00 0x7f3fe0069ae0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00099b0 tx=0x7f3fd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b2205c040d24de server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:23.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.214+0000 7f3fd7fff700 1 -- 172.21.15.89:0/3713008518 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd000b8b0 con 0x7f3fe00ccc00 2023-10-10T20:39:23.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.214+0000 7f3fd7fff700 1 -- 172.21.15.89:0/3713008518 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd000ba10 con 0x7f3fe00ccc00 2023-10-10T20:39:23.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.214+0000 7f3fd7fff700 1 -- 172.21.15.89:0/3713008518 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd000bdb0 con 0x7f3fe00ccc00 2023-10-10T20:39:23.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.214+0000 7f3fe66d3700 1 -- 172.21.15.89:0/3713008518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fe00ccc00 msgr2=0x7f3fe0069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:23.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.214+0000 7f3fe66d3700 1 --2- 172.21.15.89:0/3713008518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fe00ccc00 0x7f3fe0069ae0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00099b0 tx=0x7f3fd0009250 comp rx=0 tx=0).stop 2023-10-10T20:39:23.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.214+0000 7f3fe66d3700 1 -- 172.21.15.89:0/3713008518 shutdown_connections 2023-10-10T20:39:23.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.214+0000 7f3fe66d3700 1 --2- 172.21.15.89:0/3713008518 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3fe0159a60 0x7f3fe00c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.214+0000 7f3fe66d3700 1 --2- 172.21.15.89:0/3713008518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fe00ccc00 0x7f3fe0069ae0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.214+0000 7f3fe66d3700 1 -- 172.21.15.89:0/3713008518 >> 172.21.15.89:0/3713008518 conn(0x7f3fe006baa0 msgr2=0x7f3fe006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:23.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.215+0000 7f3fe66d3700 1 -- 172.21.15.89:0/3713008518 shutdown_connections 2023-10-10T20:39:23.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.215+0000 7f3fe66d3700 1 -- 172.21.15.89:0/3713008518 wait complete. 2023-10-10T20:39:23.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.216+0000 7f3fe66d3700 1 Processor -- start 2023-10-10T20:39:23.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.216+0000 7f3fe66d3700 1 -- start start 2023-10-10T20:39:23.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.217+0000 7f3fe66d3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fe006a020 0x7f3fe00b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:23.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.217+0000 7f3fe66d3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fe00ccc00 0x7f3fe00b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:23.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.217+0000 7f3fe66d3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3fe0159a60 0x7f3fe014cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:23.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.217+0000 7f3fe66d3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe014ca60 con 0x7f3fe00ccc00 2023-10-10T20:39:23.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.217+0000 7f3fe66d3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00b36d0 con 0x7f3fe006a020 2023-10-10T20:39:23.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.217+0000 7f3fe66d3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00b1a80 con 0x7f3fe0159a60 2023-10-10T20:39:23.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.217+0000 7f3fe4ed0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fe00ccc00 0x7f3fe00b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:23.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.217+0000 7f3fe4ed0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fe00ccc00 0x7f3fe00b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43328/0 (socket says 172.21.15.89:43328) 2023-10-10T20:39:23.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.217+0000 7f3fe4ed0700 1 -- 172.21.15.89:0/1002032361 learned_addr learned my addr 172.21.15.89:0/1002032361 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:23.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.218+0000 7f3fe56d1700 1 --2- 172.21.15.89:0/1002032361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fe006a020 0x7f3fe00b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:23.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.218+0000 7f3fe5ed2700 1 --2- 172.21.15.89:0/1002032361 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3fe0159a60 0x7f3fe014cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:23.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.218+0000 7f3fe4ed0700 1 -- 172.21.15.89:0/1002032361 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3fe0159a60 msgr2=0x7f3fe014cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:23.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.218+0000 7f3fe4ed0700 1 --2- 172.21.15.89:0/1002032361 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3fe0159a60 0x7f3fe014cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.218+0000 7f3fe4ed0700 1 -- 172.21.15.89:0/1002032361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fe006a020 msgr2=0x7f3fe00b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:23.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.218+0000 7f3fe4ed0700 1 --2- 172.21.15.89:0/1002032361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fe006a020 0x7f3fe00b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.218+0000 7f3fe4ed0700 1 -- 172.21.15.89:0/1002032361 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd00094f0 con 0x7f3fe00ccc00 2023-10-10T20:39:23.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.218+0000 7f3fe4ed0700 1 --2- 172.21.15.89:0/1002032361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fe00ccc00 0x7f3fe00b3190 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00030c0 tx=0x7f3fd0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:23.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.218+0000 7f3fd57fa700 1 -- 172.21.15.89:0/1002032361 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd001b450 con 0x7f3fe00ccc00 2023-10-10T20:39:23.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.218+0000 7f3fd57fa700 1 -- 172.21.15.89:0/1002032361 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd0020070 con 0x7f3fe00ccc00 2023-10-10T20:39:23.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.218+0000 7f3fd57fa700 1 -- 172.21.15.89:0/1002032361 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd001b800 con 0x7f3fe00ccc00 2023-10-10T20:39:23.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.218+0000 7f3fe66d3700 1 -- 172.21.15.89:0/1002032361 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe00b1d60 con 0x7f3fe00ccc00 2023-10-10T20:39:23.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.218+0000 7f3fe66d3700 1 -- 172.21.15.89:0/1002032361 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe0154b80 con 0x7f3fe00ccc00 2023-10-10T20:39:23.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.219+0000 7f3fe66d3700 1 -- 172.21.15.89:0/1002032361 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe00b2150 con 0x7f3fe00ccc00 2023-10-10T20:39:23.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.219+0000 7f3fd57fa700 1 -- 172.21.15.89:0/1002032361 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f3fd0022030 con 0x7f3fe00ccc00 2023-10-10T20:39:23.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.220+0000 7f3fd57fa700 1 --2- 172.21.15.89:0/1002032361 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3fc4067170 0x7f3fc4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:23.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.220+0000 7f3fd57fa700 1 -- 172.21.15.89:0/1002032361 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f3fd0086500 con 0x7f3fe00ccc00 2023-10-10T20:39:23.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.220+0000 7f3fe56d1700 1 --2- 172.21.15.89:0/1002032361 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3fc4067170 0x7f3fc4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:23.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.221+0000 7f3fe56d1700 1 --2- 172.21.15.89:0/1002032361 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3fc4067170 0x7f3fc4069620 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc006fd0 tx=0x7f3fdc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:23.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.225+0000 7f3fd57fa700 1 -- 172.21.15.89:0/1002032361 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3fd00534a0 con 0x7f3fe00ccc00 2023-10-10T20:39:23.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.445+0000 7f3fe66d3700 1 -- 172.21.15.89:0/1002032361 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3fe0154d70 con 0x7f3fe00ccc00 2023-10-10T20:39:23.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.445+0000 7f3fd57fa700 1 -- 172.21.15.89:0/1002032361 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3fd0056ac0 con 0x7f3fe00ccc00 2023-10-10T20:39:23.448 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:39:23.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.448+0000 7f3fe66d3700 1 -- 172.21.15.89:0/1002032361 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3fc4067170 msgr2=0x7f3fc4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:23.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.448+0000 7f3fe66d3700 1 --2- 172.21.15.89:0/1002032361 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3fc4067170 0x7f3fc4069620 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc006fd0 tx=0x7f3fdc008040 comp rx=0 tx=0).stop 2023-10-10T20:39:23.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.448+0000 7f3fe66d3700 1 -- 172.21.15.89:0/1002032361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fe00ccc00 msgr2=0x7f3fe00b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:23.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.448+0000 7f3fe66d3700 1 --2- 172.21.15.89:0/1002032361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fe00ccc00 0x7f3fe00b3190 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00030c0 tx=0x7f3fd0016000 comp rx=0 tx=0).stop 2023-10-10T20:39:23.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.448+0000 7f3fe66d3700 1 -- 172.21.15.89:0/1002032361 shutdown_connections 2023-10-10T20:39:23.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.448+0000 7f3fe66d3700 1 --2- 172.21.15.89:0/1002032361 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3fe0159a60 0x7f3fe014cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.448+0000 7f3fe66d3700 1 --2- 172.21.15.89:0/1002032361 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3fc4067170 0x7f3fc4069620 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.448+0000 7f3fe66d3700 1 --2- 172.21.15.89:0/1002032361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fe006a020 0x7f3fe00b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.448+0000 7f3fe66d3700 1 --2- 172.21.15.89:0/1002032361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fe00ccc00 0x7f3fe00b3190 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.448+0000 7f3fe66d3700 1 -- 172.21.15.89:0/1002032361 >> 172.21.15.89:0/1002032361 conn(0x7f3fe006baa0 msgr2=0x7f3fe00c4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:23.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.448+0000 7f3fe66d3700 1 -- 172.21.15.89:0/1002032361 shutdown_connections 2023-10-10T20:39:23.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.448+0000 7f3fe66d3700 1 -- 172.21.15.89:0/1002032361 wait complete. 2023-10-10T20:39:23.466 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:39:23.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.596+0000 7f839e78e700 1 Processor -- start 2023-10-10T20:39:23.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.596+0000 7f839e78e700 1 -- start start 2023-10-10T20:39:23.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.596+0000 7f839e78e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83980c93a0 0x7f839806c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:23.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.596+0000 7f839e78e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f83980c9770 0x7f83980caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:23.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.596+0000 7f839e78e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f83980cb400 0x7f83980c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:23.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.596+0000 7f839e78e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83980af490 con 0x7f83980c9770 2023-10-10T20:39:23.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.597+0000 7f839e78e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83980af770 con 0x7f83980cb400 2023-10-10T20:39:23.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.597+0000 7f839e78e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83980afa50 con 0x7f83980c93a0 2023-10-10T20:39:23.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.597+0000 7f839cf8b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f83980c9770 0x7f83980caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:23.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.597+0000 7f839cf8b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f83980c9770 0x7f83980caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43338/0 (socket says 172.21.15.89:43338) 2023-10-10T20:39:23.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.597+0000 7f839cf8b700 1 -- 172.21.15.89:0/1347219182 learned_addr learned my addr 172.21.15.89:0/1347219182 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:23.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.597+0000 7f839d78c700 1 --2- 172.21.15.89:0/1347219182 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83980c93a0 0x7f839806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:23.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.597+0000 7f839df8d700 1 --2- 172.21.15.89:0/1347219182 >> v2:172.21.15.111:3300/0 conn(0x7f83980cb400 0x7f83980c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:23.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.597+0000 7f839cf8b700 1 -- 172.21.15.89:0/1347219182 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83980c93a0 msgr2=0x7f839806c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:23.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.597+0000 7f839cf8b700 1 --2- 172.21.15.89:0/1347219182 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83980c93a0 0x7f839806c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.597+0000 7f839cf8b700 1 -- 172.21.15.89:0/1347219182 >> v2:172.21.15.111:3300/0 conn(0x7f83980cb400 msgr2=0x7f83980c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:23.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.597+0000 7f839cf8b700 1 --2- 172.21.15.89:0/1347219182 >> v2:172.21.15.111:3300/0 conn(0x7f83980cb400 0x7f83980c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.597+0000 7f839cf8b700 1 -- 172.21.15.89:0/1347219182 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83980c7ab0 con 0x7f83980c9770 2023-10-10T20:39:23.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.597+0000 7f839df8d700 1 --2- 172.21.15.89:0/1347219182 >> v2:172.21.15.111:3300/0 conn(0x7f83980cb400 0x7f83980c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:23.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.597+0000 7f839cf8b700 1 --2- 172.21.15.89:0/1347219182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83980c9770 0x7f83980caec0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f83980cca30 tx=0x7f838800b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840393a0fd96627f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:23.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.597+0000 7f838ffff700 1 -- 172.21.15.89:0/1347219182 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388005c10 con 0x7f83980c9770 2023-10-10T20:39:23.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.597+0000 7f838ffff700 1 -- 172.21.15.89:0/1347219182 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8388005d70 con 0x7f83980c9770 2023-10-10T20:39:23.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.597+0000 7f838ffff700 1 -- 172.21.15.89:0/1347219182 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838800dc30 con 0x7f83980c9770 2023-10-10T20:39:23.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.598+0000 7f839e78e700 1 -- 172.21.15.89:0/1347219182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83980c9770 msgr2=0x7f83980caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:23.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.598+0000 7f839e78e700 1 --2- 172.21.15.89:0/1347219182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83980c9770 0x7f83980caec0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f83980cca30 tx=0x7f838800b4c0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.598+0000 7f839e78e700 1 -- 172.21.15.89:0/1347219182 shutdown_connections 2023-10-10T20:39:23.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.598+0000 7f839e78e700 1 --2- 172.21.15.89:0/1347219182 >> v2:172.21.15.111:3300/0 conn(0x7f83980cb400 0x7f83980c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.598+0000 7f839e78e700 1 --2- 172.21.15.89:0/1347219182 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83980c93a0 0x7f839806c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.598+0000 7f839e78e700 1 --2- 172.21.15.89:0/1347219182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83980c9770 0x7f83980caec0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.598+0000 7f839e78e700 1 -- 172.21.15.89:0/1347219182 >> 172.21.15.89:0/1347219182 conn(0x7f83980cddc0 msgr2=0x7f83980c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:23.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.598+0000 7f839e78e700 1 -- 172.21.15.89:0/1347219182 shutdown_connections 2023-10-10T20:39:23.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.598+0000 7f839e78e700 1 -- 172.21.15.89:0/1347219182 wait complete. 2023-10-10T20:39:23.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.599+0000 7f839e78e700 1 Processor -- start 2023-10-10T20:39:23.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.600+0000 7f839e78e700 1 -- start start 2023-10-10T20:39:23.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.600+0000 7f839e78e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83980c93a0 0x7f8398142ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:23.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.600+0000 7f839e78e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83980c9770 0x7f8398154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:23.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.600+0000 7f839e78e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83980cb400 0x7f83981550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:23.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.600+0000 7f839e78e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8398143530 con 0x7f83980c93a0 2023-10-10T20:39:23.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.600+0000 7f839e78e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8398157950 con 0x7f83980c9770 2023-10-10T20:39:23.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.600+0000 7f839e78e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8398157020 con 0x7f83980cb400 2023-10-10T20:39:23.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.600+0000 7f839d78c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83980c93a0 0x7f8398142ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:23.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.601+0000 7f839d78c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83980c93a0 0x7f8398142ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43350/0 (socket says 172.21.15.89:43350) 2023-10-10T20:39:23.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.601+0000 7f839d78c700 1 -- 172.21.15.89:0/3899021022 learned_addr learned my addr 172.21.15.89:0/3899021022 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:23.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.601+0000 7f839df8d700 1 --2- 172.21.15.89:0/3899021022 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83980cb400 0x7f83981550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:23.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.601+0000 7f839cf8b700 1 --2- 172.21.15.89:0/3899021022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83980c9770 0x7f8398154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:23.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.601+0000 7f839d78c700 1 -- 172.21.15.89:0/3899021022 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83980cb400 msgr2=0x7f83981550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:23.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.601+0000 7f839d78c700 1 --2- 172.21.15.89:0/3899021022 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83980cb400 0x7f83981550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.601+0000 7f839d78c700 1 -- 172.21.15.89:0/3899021022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83980c9770 msgr2=0x7f8398154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:23.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.601+0000 7f839d78c700 1 --2- 172.21.15.89:0/3899021022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83980c9770 0x7f8398154b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.601+0000 7f839d78c700 1 -- 172.21.15.89:0/3899021022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838800b720 con 0x7f83980c93a0 2023-10-10T20:39:23.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.601+0000 7f839d78c700 1 --2- 172.21.15.89:0/3899021022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83980c93a0 0x7f8398142ff0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f839400c7d0 tx=0x7f8394008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:23.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.601+0000 7f838d7fa700 1 -- 172.21.15.89:0/3899021022 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8394011600 con 0x7f83980c93a0 2023-10-10T20:39:23.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.601+0000 7f838d7fa700 1 -- 172.21.15.89:0/3899021022 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8394011760 con 0x7f83980c93a0 2023-10-10T20:39:23.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.601+0000 7f839e78e700 1 -- 172.21.15.89:0/3899021022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8398157300 con 0x7f83980c93a0 2023-10-10T20:39:23.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.601+0000 7f838d7fa700 1 -- 172.21.15.89:0/3899021022 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8394011b20 con 0x7f83980c93a0 2023-10-10T20:39:23.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.601+0000 7f839e78e700 1 -- 172.21.15.89:0/3899021022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8398156d10 con 0x7f83980c93a0 2023-10-10T20:39:23.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.603+0000 7f839e78e700 1 -- 172.21.15.89:0/3899021022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f839806d560 con 0x7f83980c93a0 2023-10-10T20:39:23.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.603+0000 7f838d7fa700 1 -- 172.21.15.89:0/3899021022 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f83940193f0 con 0x7f83980c93a0 2023-10-10T20:39:23.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.603+0000 7f838d7fa700 1 --2- 172.21.15.89:0/3899021022 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f837c067170 0x7f837c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:23.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.603+0000 7f838d7fa700 1 -- 172.21.15.89:0/3899021022 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f8394084d50 con 0x7f83980c93a0 2023-10-10T20:39:23.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.603+0000 7f839cf8b700 1 --2- 172.21.15.89:0/3899021022 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f837c067170 0x7f837c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:23.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.604+0000 7f839cf8b700 1 --2- 172.21.15.89:0/3899021022 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f837c067170 0x7f837c069620 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f838800b500 tx=0x7f838801a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:23.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.609+0000 7f838d7fa700 1 -- 172.21.15.89:0/3899021022 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8394088030 con 0x7f83980c93a0 2023-10-10T20:39:23.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.834+0000 7f839e78e700 1 -- 172.21.15.89:0/3899021022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f839806ac60 con 0x7f83980c93a0 2023-10-10T20:39:23.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.835+0000 7f838d7fa700 1 -- 172.21.15.89:0/3899021022 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v64) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8394051be0 con 0x7f83980c93a0 2023-10-10T20:39:23.838 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:39:23.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.837+0000 7f839e78e700 1 -- 172.21.15.89:0/3899021022 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f837c067170 msgr2=0x7f837c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:23.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.837+0000 7f839e78e700 1 --2- 172.21.15.89:0/3899021022 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f837c067170 0x7f837c069620 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f838800b500 tx=0x7f838801a000 comp rx=0 tx=0).stop 2023-10-10T20:39:23.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.837+0000 7f839e78e700 1 -- 172.21.15.89:0/3899021022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83980c93a0 msgr2=0x7f8398142ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:23.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.837+0000 7f839e78e700 1 --2- 172.21.15.89:0/3899021022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83980c93a0 0x7f8398142ff0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f839400c7d0 tx=0x7f8394008040 comp rx=0 tx=0).stop 2023-10-10T20:39:23.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.837+0000 7f839e78e700 1 -- 172.21.15.89:0/3899021022 shutdown_connections 2023-10-10T20:39:23.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.837+0000 7f839e78e700 1 --2- 172.21.15.89:0/3899021022 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83980cb400 0x7f83981550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.837+0000 7f839e78e700 1 --2- 172.21.15.89:0/3899021022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83980c93a0 0x7f8398142ff0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.837+0000 7f839e78e700 1 --2- 172.21.15.89:0/3899021022 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f837c067170 0x7f837c069620 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.837+0000 7f839e78e700 1 --2- 172.21.15.89:0/3899021022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83980c9770 0x7f8398154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:23.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.837+0000 7f839e78e700 1 -- 172.21.15.89:0/3899021022 >> 172.21.15.89:0/3899021022 conn(0x7f83980cddc0 msgr2=0x7f83980c2090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:23.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.837+0000 7f839e78e700 1 -- 172.21.15.89:0/3899021022 shutdown_connections 2023-10-10T20:39:23.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:23.837+0000 7f839e78e700 1 -- 172.21.15.89:0/3899021022 wait complete. 2023-10-10T20:39:23.856 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:39:24.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.039+0000 7f5a6f429700 1 Processor -- start 2023-10-10T20:39:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.040+0000 7f5a6f429700 1 -- start start 2023-10-10T20:39:24.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.040+0000 7f5a6f429700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5a681575a0 0x7f5a680cb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.040+0000 7f5a6f429700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a68063580 0x7f5a6806e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.040+0000 7f5a6f429700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a6806ee10 con 0x7f5a6806e9b0 2023-10-10T20:39:24.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.040+0000 7f5a6f429700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a680b1100 con 0x7f5a681575a0 2023-10-10T20:39:24.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.040+0000 7f5a6f429700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680c3ae0 con 0x7f5a68063580 2023-10-10T20:39:24.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6ec28700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5a6806e9b0 0x7f5a680ac9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:32874/0 (socket says 172.21.15.89:32874) 2023-10-10T20:39:24.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6ec28700 1 -- 172.21.15.89:0/1963239595 learned_addr learned my addr 172.21.15.89:0/1963239595 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:24.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6e427700 1 --2- 172.21.15.89:0/1963239595 >> v2:172.21.15.111:3300/0 conn(0x7f5a681575a0 0x7f5a680cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6d425700 1 -- 172.21.15.89:0/1963239595 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2183334525 0 0) 0x7f5a60001490 con 0x7f5a6806e9b0 2023-10-10T20:39:24.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6dc26700 1 --2- 172.21.15.89:0/1963239595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a68063580 0x7f5a6806e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6d425700 1 -- 172.21.15.89:0/1963239595 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a54003570 con 0x7f5a6806e9b0 2023-10-10T20:39:24.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6d425700 1 -- 172.21.15.89:0/1963239595 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 149378716 0 0) 0x7f5a54003570 con 0x7f5a6806e9b0 2023-10-10T20:39:24.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6d425700 1 -- 172.21.15.89:0/1963239595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a68063580 msgr2=0x7f5a6806e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:24.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6d425700 1 --2- 172.21.15.89:0/1963239595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a68063580 0x7f5a6806e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6d425700 1 -- 172.21.15.89:0/1963239595 >> v2:172.21.15.111:3300/0 conn(0x7f5a681575a0 msgr2=0x7f5a680cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:24.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6d425700 1 --2- 172.21.15.89:0/1963239595 >> v2:172.21.15.111:3300/0 conn(0x7f5a681575a0 0x7f5a680cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6d425700 1 -- 172.21.15.89:0/1963239595 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a680cbe60 con 0x7f5a6806e9b0 2023-10-10T20:39:24.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6d425700 1 -- 172.21.15.89:0/1963239595 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f5a60001980 con 0x7f5a6806e9b0 2023-10-10T20:39:24.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6d425700 1 -- 172.21.15.89:0/1963239595 >> v1:172.21.15.89:6789/0 conn(0x7f5a6806e9b0 legacy=0x7f5a680ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:24.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6d425700 1 --2- 172.21.15.89:0/1963239595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a68063580 0x7f5a54003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6d425700 1 --2- 172.21.15.89:0/1963239595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a54003fe0 0x7f5a54004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6d425700 1 --2- 172.21.15.89:0/1963239595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a54004d50 0x7f5a540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6d425700 1 -- 172.21.15.89:0/1963239595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680b1100 con 0x7f5a54004d50 2023-10-10T20:39:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6d425700 1 -- 172.21.15.89:0/1963239595 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680c3ae0 con 0x7f5a54003fe0 2023-10-10T20:39:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6d425700 1 -- 172.21.15.89:0/1963239595 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5400ba90 con 0x7f5a68063580 2023-10-10T20:39:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6e427700 1 --2- 172.21.15.89:0/1963239595 >> v2:172.21.15.111:3300/0 conn(0x7f5a681575a0 0x7f5a680cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:39:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.041+0000 7f5a6ec28700 1 --2- 172.21.15.89:0/1963239595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a54004d50 0x7f5a540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.042+0000 7f5a6dc26700 1 --2- 172.21.15.89:0/1963239595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a54003fe0 0x7f5a54004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.042+0000 7f5a6e427700 1 --2- 172.21.15.89:0/1963239595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a68063580 0x7f5a54003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.042+0000 7f5a6ec28700 1 -- 172.21.15.89:0/1963239595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a68063580 msgr2=0x7f5a54003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.042+0000 7f5a6ec28700 1 --2- 172.21.15.89:0/1963239595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a68063580 0x7f5a54003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.042+0000 7f5a6ec28700 1 -- 172.21.15.89:0/1963239595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a54003fe0 msgr2=0x7f5a54004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.042+0000 7f5a6ec28700 1 --2- 172.21.15.89:0/1963239595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a54003fe0 0x7f5a54004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.042+0000 7f5a6ec28700 1 -- 172.21.15.89:0/1963239595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a54003570 con 0x7f5a54004d50 2023-10-10T20:39:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.042+0000 7f5a6ec28700 1 --2- 172.21.15.89:0/1963239595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a54004d50 0x7f5a540051c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000ab00 tx=0x7f5a6000a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae82b16952996e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.042+0000 7f5a6d425700 1 -- 172.21.15.89:0/1963239595 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000a490 con 0x7f5a54004d50 2023-10-10T20:39:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.042+0000 7f5a6d425700 1 -- 172.21.15.89:0/1963239595 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6000c570 con 0x7f5a54004d50 2023-10-10T20:39:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.042+0000 7f5a6d425700 1 -- 172.21.15.89:0/1963239595 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000c930 con 0x7f5a54004d50 2023-10-10T20:39:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.043+0000 7f5a6f429700 1 -- 172.21.15.89:0/1963239595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a54004d50 msgr2=0x7f5a540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.043+0000 7f5a6f429700 1 --2- 172.21.15.89:0/1963239595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a54004d50 0x7f5a540051c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000ab00 tx=0x7f5a6000a1a0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.043+0000 7f5a6ec28700 1 -- 172.21.15.89:0/1963239595 reap_dead start 2023-10-10T20:39:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.043+0000 7f5a6f429700 1 -- 172.21.15.89:0/1963239595 shutdown_connections 2023-10-10T20:39:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.043+0000 7f5a6f429700 1 -- 172.21.15.89:0/1963239595 >> 172.21.15.89:0/1963239595 conn(0x7f5a680c9f70 msgr2=0x7f5a6806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.043+0000 7f5a6f429700 1 -- 172.21.15.89:0/1963239595 shutdown_connections 2023-10-10T20:39:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.043+0000 7f5a6f429700 1 -- 172.21.15.89:0/1963239595 wait complete. 2023-10-10T20:39:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.044+0000 7f5a6f429700 1 Processor -- start 2023-10-10T20:39:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.044+0000 7f5a6f429700 1 -- start start 2023-10-10T20:39:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.045+0000 7f5a6f429700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a54004d50 0x7f5a68168470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.045+0000 7f5a6f429700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a68152200 0x7f5a680c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.045+0000 7f5a6f429700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a68155130 0x7f5a681555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.045+0000 7f5a6f429700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a68155af0 con 0x7f5a54004d50 2023-10-10T20:39:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.045+0000 7f5a6f429700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a68168a60 con 0x7f5a68155130 2023-10-10T20:39:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.045+0000 7f5a6f429700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a68157970 con 0x7f5a68152200 2023-10-10T20:39:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.045+0000 7f5a6e427700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a54004d50 0x7f5a68168470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.045+0000 7f5a6e427700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a54004d50 0x7f5a68168470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43374/0 (socket says 172.21.15.89:43374) 2023-10-10T20:39:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.045+0000 7f5a6e427700 1 -- 172.21.15.89:0/937987128 learned_addr learned my addr 172.21.15.89:0/937987128 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.045+0000 7f5a6ec28700 1 --2- 172.21.15.89:0/937987128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a68155130 0x7f5a681555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.045+0000 7f5a6dc26700 1 --2- 172.21.15.89:0/937987128 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a68152200 0x7f5a680c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.045+0000 7f5a6e427700 1 -- 172.21.15.89:0/937987128 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a68152200 msgr2=0x7f5a680c5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:24.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.045+0000 7f5a6e427700 1 --2- 172.21.15.89:0/937987128 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a68152200 0x7f5a680c5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.045+0000 7f5a6e427700 1 -- 172.21.15.89:0/937987128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a68155130 msgr2=0x7f5a681555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:24.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.045+0000 7f5a6e427700 1 --2- 172.21.15.89:0/937987128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a68155130 0x7f5a681555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.045+0000 7f5a6e427700 1 -- 172.21.15.89:0/937987128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a681584f0 con 0x7f5a54004d50 2023-10-10T20:39:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.045+0000 7f5a6ec28700 1 --2- 172.21.15.89:0/937987128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a68155130 0x7f5a681555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.045+0000 7f5a6e427700 1 --2- 172.21.15.89:0/937987128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a54004d50 0x7f5a68168470 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5a64011990 tx=0x7f5a64008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.046+0000 7f5a5e7fc700 1 -- 172.21.15.89:0/937987128 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a64011ed0 con 0x7f5a54004d50 2023-10-10T20:39:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.046+0000 7f5a5e7fc700 1 -- 172.21.15.89:0/937987128 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6400f8e0 con 0x7f5a54004d50 2023-10-10T20:39:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.046+0000 7f5a6f429700 1 -- 172.21.15.89:0/937987128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a680c5fe0 con 0x7f5a54004d50 2023-10-10T20:39:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.046+0000 7f5a5e7fc700 1 -- 172.21.15.89:0/937987128 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6400fc60 con 0x7f5a54004d50 2023-10-10T20:39:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.046+0000 7f5a6f429700 1 -- 172.21.15.89:0/937987128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a68157ea0 con 0x7f5a54004d50 2023-10-10T20:39:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.047+0000 7f5a6f429700 1 -- 172.21.15.89:0/937987128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a681525d0 con 0x7f5a54004d50 2023-10-10T20:39:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.047+0000 7f5a5e7fc700 1 -- 172.21.15.89:0/937987128 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f5a64015670 con 0x7f5a54004d50 2023-10-10T20:39:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.048+0000 7f5a5e7fc700 1 --2- 172.21.15.89:0/937987128 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a4c067170 0x7f5a4c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.048+0000 7f5a5e7fc700 1 -- 172.21.15.89:0/937987128 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f5a640851d0 con 0x7f5a54004d50 2023-10-10T20:39:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.048+0000 7f5a6dc26700 1 --2- 172.21.15.89:0/937987128 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a4c067170 0x7f5a4c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.048+0000 7f5a6dc26700 1 --2- 172.21.15.89:0/937987128 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a4c067170 0x7f5a4c069620 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5a58006f00 tx=0x7f5a58008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.052+0000 7f5a5e7fc700 1 -- 172.21.15.89:0/937987128 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a64052170 con 0x7f5a54004d50 2023-10-10T20:39:24.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.261+0000 7f5a6f429700 1 -- 172.21.15.89:0/937987128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a680cc140 con 0x7f5a54004d50 2023-10-10T20:39:24.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.261+0000 7f5a5e7fc700 1 -- 172.21.15.89:0/937987128 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v64) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a64055790 con 0x7f5a54004d50 2023-10-10T20:39:24.264 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-10T20:39:24.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.263+0000 7f5a6f429700 1 -- 172.21.15.89:0/937987128 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a4c067170 msgr2=0x7f5a4c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:24.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.263+0000 7f5a6f429700 1 --2- 172.21.15.89:0/937987128 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a4c067170 0x7f5a4c069620 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5a58006f00 tx=0x7f5a58008040 comp rx=0 tx=0).stop 2023-10-10T20:39:24.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.263+0000 7f5a6f429700 1 -- 172.21.15.89:0/937987128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a54004d50 msgr2=0x7f5a68168470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:24.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.263+0000 7f5a6f429700 1 --2- 172.21.15.89:0/937987128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a54004d50 0x7f5a68168470 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5a64011990 tx=0x7f5a64008040 comp rx=0 tx=0).stop 2023-10-10T20:39:24.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.263+0000 7f5a6f429700 1 -- 172.21.15.89:0/937987128 shutdown_connections 2023-10-10T20:39:24.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.263+0000 7f5a6f429700 1 --2- 172.21.15.89:0/937987128 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a4c067170 0x7f5a4c069620 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.263+0000 7f5a6f429700 1 --2- 172.21.15.89:0/937987128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a68155130 0x7f5a681555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.263+0000 7f5a6f429700 1 --2- 172.21.15.89:0/937987128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a54004d50 0x7f5a68168470 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.263+0000 7f5a6f429700 1 --2- 172.21.15.89:0/937987128 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a68152200 0x7f5a680c5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.263+0000 7f5a6f429700 1 -- 172.21.15.89:0/937987128 >> 172.21.15.89:0/937987128 conn(0x7f5a680c9f70 msgr2=0x7f5a680c33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:24.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.263+0000 7f5a6f429700 1 -- 172.21.15.89:0/937987128 shutdown_connections 2023-10-10T20:39:24.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.263+0000 7f5a6f429700 1 -- 172.21.15.89:0/937987128 wait complete. 2023-10-10T20:39:24.279 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:39:24.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.415+0000 7ff43102a700 1 Processor -- start 2023-10-10T20:39:24.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.415+0000 7ff43102a700 1 -- start start 2023-10-10T20:39:24.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.416+0000 7ff43102a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff42c159c60 0x7ff42c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.416+0000 7ff43102a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff42c15db10 0x7ff42c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.416+0000 7ff43102a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff42c15a160 con 0x7ff42c159890 2023-10-10T20:39:24.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.416+0000 7ff43102a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff42c15df50 con 0x7ff42c159c60 2023-10-10T20:39:24.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.416+0000 7ff43102a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c0d13a0 con 0x7ff42c15db10 2023-10-10T20:39:24.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.416+0000 7ff42ad9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff42c159890 0x7ff42c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:32882/0 (socket says 172.21.15.89:32882) 2023-10-10T20:39:24.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.416+0000 7ff42ad9d700 1 -- 172.21.15.89:0/3439856204 learned_addr learned my addr 172.21.15.89:0/3439856204 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:24.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.416+0000 7ff42a59c700 1 --2- 172.21.15.89:0/3439856204 >> v2:172.21.15.111:3300/0 conn(0x7ff42c159c60 0x7ff42c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.416+0000 7ff42b59e700 1 --2- 172.21.15.89:0/3439856204 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff42c15db10 0x7ff42c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.416+0000 7ff429d9b700 1 -- 172.21.15.89:0/3439856204 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2014420760 0 0) 0x7ff42c15a160 con 0x7ff42c159890 2023-10-10T20:39:24.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.416+0000 7ff429d9b700 1 -- 172.21.15.89:0/3439856204 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff414003570 con 0x7ff42c159890 2023-10-10T20:39:24.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff429d9b700 1 -- 172.21.15.89:0/3439856204 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3865835224 0 0) 0x7ff414003570 con 0x7ff42c159890 2023-10-10T20:39:24.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff429d9b700 1 -- 172.21.15.89:0/3439856204 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff42c15db10 msgr2=0x7ff42c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:24.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff429d9b700 1 --2- 172.21.15.89:0/3439856204 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff42c15db10 0x7ff42c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff429d9b700 1 -- 172.21.15.89:0/3439856204 >> v2:172.21.15.111:3300/0 conn(0x7ff42c159c60 msgr2=0x7ff42c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:24.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff429d9b700 1 --2- 172.21.15.89:0/3439856204 >> v2:172.21.15.111:3300/0 conn(0x7ff42c159c60 0x7ff42c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff429d9b700 1 -- 172.21.15.89:0/3439856204 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42c15e140 con 0x7ff42c159890 2023-10-10T20:39:24.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff429d9b700 1 -- 172.21.15.89:0/3439856204 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7ff41c001470 con 0x7ff42c159890 2023-10-10T20:39:24.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff429d9b700 1 -- 172.21.15.89:0/3439856204 >> v1:172.21.15.89:6789/0 conn(0x7ff42c159890 legacy=0x7ff42c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:24.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff429d9b700 1 --2- 172.21.15.89:0/3439856204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff414003a60 0x7ff414003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff429d9b700 1 --2- 172.21.15.89:0/3439856204 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff414004810 0x7ff414004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff429d9b700 1 --2- 172.21.15.89:0/3439856204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4140051c0 0x7ff41400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff429d9b700 1 -- 172.21.15.89:0/3439856204 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c0d13a0 con 0x7ff4140051c0 2023-10-10T20:39:24.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff429d9b700 1 -- 172.21.15.89:0/3439856204 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff414004680 con 0x7ff414003a60 2023-10-10T20:39:24.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff429d9b700 1 -- 172.21.15.89:0/3439856204 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41400c020 con 0x7ff414004810 2023-10-10T20:39:24.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff42a59c700 1 --2- 172.21.15.89:0/3439856204 >> v2:172.21.15.111:3300/0 conn(0x7ff42c159c60 0x7ff42c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:39:24.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff42b59e700 1 --2- 172.21.15.89:0/3439856204 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff42c15db10 0x7ff42c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:39:24.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff42ad9d700 1 --2- 172.21.15.89:0/3439856204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff414003a60 0x7ff414003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff42a59c700 1 --2- 172.21.15.89:0/3439856204 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff414004810 0x7ff414004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff42b59e700 1 --2- 172.21.15.89:0/3439856204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4140051c0 0x7ff41400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff42b59e700 1 -- 172.21.15.89:0/3439856204 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff414004810 msgr2=0x7ff414004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:24.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff42b59e700 1 --2- 172.21.15.89:0/3439856204 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff414004810 0x7ff414004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff42b59e700 1 -- 172.21.15.89:0/3439856204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff414003a60 msgr2=0x7ff414003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:24.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff42b59e700 1 --2- 172.21.15.89:0/3439856204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff414003a60 0x7ff414003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff42b59e700 1 -- 172.21.15.89:0/3439856204 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff414003570 con 0x7ff4140051c0 2023-10-10T20:39:24.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.417+0000 7ff42ad9d700 1 --2- 172.21.15.89:0/3439856204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff414003a60 0x7ff414003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:39:24.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.418+0000 7ff42b59e700 1 --2- 172.21.15.89:0/3439856204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4140051c0 0x7ff41400ba60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff42400e280 tx=0x7ff42400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d598b75ce4c610 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:24.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.418+0000 7ff429d9b700 1 -- 172.21.15.89:0/3439856204 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42400ead0 con 0x7ff4140051c0 2023-10-10T20:39:24.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.418+0000 7ff429d9b700 1 -- 172.21.15.89:0/3439856204 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42400f040 con 0x7ff4140051c0 2023-10-10T20:39:24.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.418+0000 7ff429d9b700 1 -- 172.21.15.89:0/3439856204 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42400ee60 con 0x7ff4140051c0 2023-10-10T20:39:24.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.418+0000 7ff43102a700 1 -- 172.21.15.89:0/3439856204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4140051c0 msgr2=0x7ff41400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:24.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.418+0000 7ff43102a700 1 --2- 172.21.15.89:0/3439856204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4140051c0 0x7ff41400ba60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff42400e280 tx=0x7ff42400b040 comp rx=0 tx=0).stop 2023-10-10T20:39:24.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.418+0000 7ff42b59e700 1 -- 172.21.15.89:0/3439856204 reap_dead start 2023-10-10T20:39:24.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.418+0000 7ff43102a700 1 -- 172.21.15.89:0/3439856204 shutdown_connections 2023-10-10T20:39:24.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.418+0000 7ff43102a700 1 -- 172.21.15.89:0/3439856204 >> 172.21.15.89:0/3439856204 conn(0x7ff42c068e20 msgr2=0x7ff42c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:24.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.418+0000 7ff43102a700 1 -- 172.21.15.89:0/3439856204 shutdown_connections 2023-10-10T20:39:24.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.418+0000 7ff43102a700 1 -- 172.21.15.89:0/3439856204 wait complete. 2023-10-10T20:39:24.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.420+0000 7ff43102a700 1 Processor -- start 2023-10-10T20:39:24.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.420+0000 7ff43102a700 1 -- start start 2023-10-10T20:39:24.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.420+0000 7ff43102a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4140051c0 0x7ff42c0ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.420+0000 7ff43102a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff42c169d70 0x7ff42c0b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.420+0000 7ff43102a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff42c0b61c0 0x7ff42c158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.420+0000 7ff43102a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c1576c0 con 0x7ff4140051c0 2023-10-10T20:39:24.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.420+0000 7ff43102a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c157c80 con 0x7ff42c169d70 2023-10-10T20:39:24.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.420+0000 7ff43102a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c0b4310 con 0x7ff42c0b61c0 2023-10-10T20:39:24.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.421+0000 7ff42ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4140051c0 0x7ff42c0ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.421+0000 7ff42ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4140051c0 0x7ff42c0ce4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43392/0 (socket says 172.21.15.89:43392) 2023-10-10T20:39:24.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.421+0000 7ff42ad9d700 1 -- 172.21.15.89:0/83250832 learned_addr learned my addr 172.21.15.89:0/83250832 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:24.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.421+0000 7ff42a59c700 1 --2- 172.21.15.89:0/83250832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff42c169d70 0x7ff42c0b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.421+0000 7ff42b59e700 1 --2- 172.21.15.89:0/83250832 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff42c0b61c0 0x7ff42c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.421+0000 7ff42ad9d700 1 -- 172.21.15.89:0/83250832 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff42c0b61c0 msgr2=0x7ff42c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:24.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.421+0000 7ff42ad9d700 1 --2- 172.21.15.89:0/83250832 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff42c0b61c0 0x7ff42c158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.421+0000 7ff42ad9d700 1 -- 172.21.15.89:0/83250832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff42c169d70 msgr2=0x7ff42c0b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:24.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.421+0000 7ff42ad9d700 1 --2- 172.21.15.89:0/83250832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff42c169d70 0x7ff42c0b5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.421+0000 7ff42ad9d700 1 -- 172.21.15.89:0/83250832 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42c0b45f0 con 0x7ff4140051c0 2023-10-10T20:39:24.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.421+0000 7ff42ad9d700 1 --2- 172.21.15.89:0/83250832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4140051c0 0x7ff42c0ce4d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff41c001010 tx=0x7ff41c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:24.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.421+0000 7ff412ffd700 1 -- 172.21.15.89:0/83250832 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c018030 con 0x7ff4140051c0 2023-10-10T20:39:24.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.421+0000 7ff412ffd700 1 -- 172.21.15.89:0/83250832 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41c00f9a0 con 0x7ff4140051c0 2023-10-10T20:39:24.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.421+0000 7ff43102a700 1 -- 172.21.15.89:0/83250832 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42c157910 con 0x7ff4140051c0 2023-10-10T20:39:24.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.422+0000 7ff412ffd700 1 -- 172.21.15.89:0/83250832 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c00fd40 con 0x7ff4140051c0 2023-10-10T20:39:24.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.422+0000 7ff43102a700 1 -- 172.21.15.89:0/83250832 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff42c0cea10 con 0x7ff4140051c0 2023-10-10T20:39:24.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.423+0000 7ff43102a700 1 -- 172.21.15.89:0/83250832 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff42c0cec30 con 0x7ff4140051c0 2023-10-10T20:39:24.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.423+0000 7ff412ffd700 1 -- 172.21.15.89:0/83250832 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7ff41c01b070 con 0x7ff4140051c0 2023-10-10T20:39:24.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.423+0000 7ff412ffd700 1 --2- 172.21.15.89:0/83250832 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff418067170 0x7ff418069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.423+0000 7ff412ffd700 1 -- 172.21.15.89:0/83250832 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5763+0+0 (secure 0 0 0) 0x7ff41c019070 con 0x7ff4140051c0 2023-10-10T20:39:24.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.423+0000 7ff42a59c700 1 --2- 172.21.15.89:0/83250832 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff418067170 0x7ff418069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.424+0000 7ff42a59c700 1 --2- 172.21.15.89:0/83250832 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff418067170 0x7ff418069620 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff42c15aeb0 tx=0x7ff4200091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:24.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.431+0000 7ff412ffd700 1 -- 172.21.15.89:0/83250832 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff41c051840 con 0x7ff4140051c0 2023-10-10T20:39:24.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.735+0000 7ff43102a700 1 -- 172.21.15.89:0/83250832 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff42c04c3c0 con 0x7ff4140051c0 2023-10-10T20:39:24.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.736+0000 7ff412ffd700 1 -- 172.21.15.89:0/83250832 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4602 (secure 0 0 0) 0x7ff41c00fea0 con 0x7ff4140051c0 2023-10-10T20:39:24.739 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:24.739 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/3362909084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":3362909084},{"type":"v1","addr":"172.21.15.111:6835","nonce":3362909084}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5102,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1472423656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1472423656},{"type":"v1","addr":"172.21.15.111:6834","nonce":1472423656}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/3183325104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3183325104},{"type":"v1","addr":"172.21.15.89:6836","nonce":3183325104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:36:20.867329+0000","modified":"2023-10-10T20:36:24.518115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/862652054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":862652054},{"type":"v1","addr":"172.21.15.89:6837","nonce":862652054}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-10T20:39:24.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.739+0000 7ff43102a700 1 -- 172.21.15.89:0/83250832 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff418067170 msgr2=0x7ff418069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:24.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.739+0000 7ff43102a700 1 --2- 172.21.15.89:0/83250832 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff418067170 0x7ff418069620 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff42c15aeb0 tx=0x7ff4200091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.739+0000 7ff43102a700 1 -- 172.21.15.89:0/83250832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4140051c0 msgr2=0x7ff42c0ce4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:24.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.739+0000 7ff43102a700 1 --2- 172.21.15.89:0/83250832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4140051c0 0x7ff42c0ce4d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff41c001010 tx=0x7ff41c001e90 comp rx=0 tx=0).stop 2023-10-10T20:39:24.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.739+0000 7ff43102a700 1 -- 172.21.15.89:0/83250832 shutdown_connections 2023-10-10T20:39:24.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.739+0000 7ff43102a700 1 --2- 172.21.15.89:0/83250832 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff42c0b61c0 0x7ff42c158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.739+0000 7ff43102a700 1 --2- 172.21.15.89:0/83250832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4140051c0 0x7ff42c0ce4d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.739+0000 7ff43102a700 1 --2- 172.21.15.89:0/83250832 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff418067170 0x7ff418069620 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.739+0000 7ff43102a700 1 --2- 172.21.15.89:0/83250832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff42c169d70 0x7ff42c0b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.739+0000 7ff43102a700 1 -- 172.21.15.89:0/83250832 >> 172.21.15.89:0/83250832 conn(0x7ff42c068e20 msgr2=0x7ff42c152b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:24.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.739+0000 7ff43102a700 1 -- 172.21.15.89:0/83250832 shutdown_connections 2023-10-10T20:39:24.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.739+0000 7ff43102a700 1 -- 172.21.15.89:0/83250832 wait complete. 2023-10-10T20:39:24.744 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 38 2023-10-10T20:39:24.754 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:39:24.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.875+0000 7f8b83a6f700 1 Processor -- start 2023-10-10T20:39:24.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.875+0000 7f8b83a6f700 1 -- start start 2023-10-10T20:39:24.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.876+0000 7f8b83a6f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7c06c0a0 0x7f8b7c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.876+0000 7f8b83a6f700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8b7c15df50 0x7f8b7c15e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.876+0000 7f8b83a6f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7c15d7e0 con 0x7f8b7c06bcd0 2023-10-10T20:39:24.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.876+0000 7f8b83a6f700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b7c15dac0 con 0x7f8b7c15df50 2023-10-10T20:39:24.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.876+0000 7f8b83a6f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c15aa30 con 0x7f8b7c06c0a0 2023-10-10T20:39:24.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.876+0000 7f8b8180b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8b7c06bcd0 0x7f8b7c15a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:32886/0 (socket says 172.21.15.89:32886) 2023-10-10T20:39:24.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.876+0000 7f8b8180b700 1 -- 172.21.15.89:0/716645452 learned_addr learned my addr 172.21.15.89:0/716645452 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:24.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.876+0000 7f8b8200c700 1 --2- 172.21.15.89:0/716645452 >> v2:172.21.15.111:3300/0 conn(0x7f8b7c15df50 0x7f8b7c15e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.876+0000 7f8b80809700 1 -- 172.21.15.89:0/716645452 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557661844 0 0) 0x7f8b7c15d7e0 con 0x7f8b7c06bcd0 2023-10-10T20:39:24.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.876+0000 7f8b80809700 1 -- 172.21.15.89:0/716645452 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b68003570 con 0x7f8b7c06bcd0 2023-10-10T20:39:24.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.876+0000 7f8b8100a700 1 --2- 172.21.15.89:0/716645452 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7c06c0a0 0x7f8b7c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b80809700 1 -- 172.21.15.89:0/716645452 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2306310463 0 0) 0x7f8b68003570 con 0x7f8b7c06bcd0 2023-10-10T20:39:24.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b80809700 1 -- 172.21.15.89:0/716645452 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7c06c0a0 msgr2=0x7f8b7c0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:24.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b80809700 1 --2- 172.21.15.89:0/716645452 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7c06c0a0 0x7f8b7c0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b80809700 1 -- 172.21.15.89:0/716645452 >> v2:172.21.15.111:3300/0 conn(0x7f8b7c15df50 msgr2=0x7f8b7c15e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:24.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b80809700 1 --2- 172.21.15.89:0/716645452 >> v2:172.21.15.111:3300/0 conn(0x7f8b7c15df50 0x7f8b7c15e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b80809700 1 -- 172.21.15.89:0/716645452 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c15ad10 con 0x7f8b7c06bcd0 2023-10-10T20:39:24.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b80809700 1 -- 172.21.15.89:0/716645452 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f8b78001470 con 0x7f8b7c06bcd0 2023-10-10T20:39:24.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b80809700 1 -- 172.21.15.89:0/716645452 >> v1:172.21.15.89:6789/0 conn(0x7f8b7c06bcd0 legacy=0x7f8b7c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:24.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b80809700 1 --2- 172.21.15.89:0/716645452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b68003a60 0x7f8b68003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b80809700 1 --2- 172.21.15.89:0/716645452 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7c06c0a0 0x7f8b68004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b80809700 1 --2- 172.21.15.89:0/716645452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b68004d50 0x7f8b680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b80809700 1 -- 172.21.15.89:0/716645452 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c15dac0 con 0x7f8b68003a60 2023-10-10T20:39:24.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b80809700 1 -- 172.21.15.89:0/716645452 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c15aa30 con 0x7f8b68004d50 2023-10-10T20:39:24.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b80809700 1 -- 172.21.15.89:0/716645452 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6800bae0 con 0x7f8b7c06c0a0 2023-10-10T20:39:24.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b8180b700 1 --2- 172.21.15.89:0/716645452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b68003a60 0x7f8b68003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b8100a700 1 --2- 172.21.15.89:0/716645452 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7c06c0a0 0x7f8b68004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b8200c700 1 --2- 172.21.15.89:0/716645452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b68004d50 0x7f8b680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b8180b700 1 -- 172.21.15.89:0/716645452 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7c06c0a0 msgr2=0x7f8b68004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:24.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b8180b700 1 --2- 172.21.15.89:0/716645452 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7c06c0a0 0x7f8b68004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b8180b700 1 -- 172.21.15.89:0/716645452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b68004d50 msgr2=0x7f8b680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:24.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b8180b700 1 --2- 172.21.15.89:0/716645452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b68004d50 0x7f8b680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b8180b700 1 -- 172.21.15.89:0/716645452 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b68003570 con 0x7f8b68003a60 2023-10-10T20:39:24.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b8200c700 1 --2- 172.21.15.89:0/716645452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b68004d50 0x7f8b680051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:24.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.877+0000 7f8b8180b700 1 --2- 172.21.15.89:0/716645452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b68003a60 0x7f8b68003e70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800c3c0 tx=0x7f8b78001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b587ef9688e80c66 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:24.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.878+0000 7f8b80809700 1 -- 172.21.15.89:0/716645452 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7800cc90 con 0x7f8b68003a60 2023-10-10T20:39:24.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.878+0000 7f8b80809700 1 -- 172.21.15.89:0/716645452 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7800ce10 con 0x7f8b68003a60 2023-10-10T20:39:24.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.878+0000 7f8b80809700 1 -- 172.21.15.89:0/716645452 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78005b70 con 0x7f8b68003a60 2023-10-10T20:39:24.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.878+0000 7f8b83a6f700 1 -- 172.21.15.89:0/716645452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b68003a60 msgr2=0x7f8b68003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:24.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.878+0000 7f8b83a6f700 1 --2- 172.21.15.89:0/716645452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b68003a60 0x7f8b68003e70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800c3c0 tx=0x7f8b78001e90 comp rx=0 tx=0).stop 2023-10-10T20:39:24.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.878+0000 7f8b8200c700 1 -- 172.21.15.89:0/716645452 reap_dead start 2023-10-10T20:39:24.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.878+0000 7f8b83a6f700 1 -- 172.21.15.89:0/716645452 shutdown_connections 2023-10-10T20:39:24.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.878+0000 7f8b83a6f700 1 -- 172.21.15.89:0/716645452 >> 172.21.15.89:0/716645452 conn(0x7f8b7c0ce600 msgr2=0x7f8b7c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:24.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.878+0000 7f8b83a6f700 1 -- 172.21.15.89:0/716645452 shutdown_connections 2023-10-10T20:39:24.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.878+0000 7f8b83a6f700 1 -- 172.21.15.89:0/716645452 wait complete. 2023-10-10T20:39:24.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.879+0000 7f8b83a6f700 1 Processor -- start 2023-10-10T20:39:24.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.879+0000 7f8b83a6f700 1 -- start start 2023-10-10T20:39:24.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.880+0000 7f8b83a6f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b68003a60 0x7f8b7c16b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.880+0000 7f8b83a6f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7c154060 0x7f8b7c0cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.880+0000 7f8b83a6f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b7c0cba80 0x7f8b7c0b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.880+0000 7f8b83a6f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c0cbf20 con 0x7f8b68003a60 2023-10-10T20:39:24.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.880+0000 7f8b83a6f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c155780 con 0x7f8b7c0cba80 2023-10-10T20:39:24.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.880+0000 7f8b83a6f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c1565e0 con 0x7f8b7c154060 2023-10-10T20:39:24.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.880+0000 7f8b8180b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b68003a60 0x7f8b7c16b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.880+0000 7f8b8180b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b68003a60 0x7f8b7c16b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43418/0 (socket says 172.21.15.89:43418) 2023-10-10T20:39:24.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.880+0000 7f8b8180b700 1 -- 172.21.15.89:0/3941053468 learned_addr learned my addr 172.21.15.89:0/3941053468 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:24.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.880+0000 7f8b8100a700 1 --2- 172.21.15.89:0/3941053468 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7c154060 0x7f8b7c0cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.880+0000 7f8b8180b700 1 -- 172.21.15.89:0/3941053468 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7c154060 msgr2=0x7f8b7c0cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:24.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.880+0000 7f8b8180b700 1 --2- 172.21.15.89:0/3941053468 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7c154060 0x7f8b7c0cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.880+0000 7f8b8200c700 1 --2- 172.21.15.89:0/3941053468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b7c0cba80 0x7f8b7c0b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.880+0000 7f8b8180b700 1 -- 172.21.15.89:0/3941053468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b7c0cba80 msgr2=0x7f8b7c0b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:24.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.880+0000 7f8b8180b700 1 --2- 172.21.15.89:0/3941053468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b7c0cba80 0x7f8b7c0b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:24.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.881+0000 7f8b8180b700 1 -- 172.21.15.89:0/3941053468 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c1544d0 con 0x7f8b68003a60 2023-10-10T20:39:24.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.881+0000 7f8b8200c700 1 --2- 172.21.15.89:0/3941053468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b7c0cba80 0x7f8b7c0b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:24.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.881+0000 7f8b8180b700 1 --2- 172.21.15.89:0/3941053468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b68003a60 0x7f8b7c16b010 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800c9b0 tx=0x7f8b78015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:24.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.881+0000 7f8b71ffb700 1 -- 172.21.15.89:0/3941053468 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78001430 con 0x7f8b68003a60 2023-10-10T20:39:24.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.881+0000 7f8b71ffb700 1 -- 172.21.15.89:0/3941053468 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b780072e0 con 0x7f8b68003a60 2023-10-10T20:39:24.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.881+0000 7f8b83a6f700 1 -- 172.21.15.89:0/3941053468 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7c1559d0 con 0x7f8b68003a60 2023-10-10T20:39:24.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.881+0000 7f8b71ffb700 1 -- 172.21.15.89:0/3941053468 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b780076a0 con 0x7f8b68003a60 2023-10-10T20:39:24.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.881+0000 7f8b83a6f700 1 -- 172.21.15.89:0/3941053468 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7c155f90 con 0x7f8b68003a60 2023-10-10T20:39:24.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.882+0000 7f8b83a6f700 1 -- 172.21.15.89:0/3941053468 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7c04c3c0 con 0x7f8b68003a60 2023-10-10T20:39:24.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.883+0000 7f8b71ffb700 1 -- 172.21.15.89:0/3941053468 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f8b78019030 con 0x7f8b68003a60 2023-10-10T20:39:24.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.883+0000 7f8b71ffb700 1 --2- 172.21.15.89:0/3941053468 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b60067170 0x7f8b60069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:24.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.884+0000 7f8b71ffb700 1 -- 172.21.15.89:0/3941053468 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f8b780859a0 con 0x7f8b68003a60 2023-10-10T20:39:24.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.884+0000 7f8b8100a700 1 --2- 172.21.15.89:0/3941053468 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b60067170 0x7f8b60069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:24.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.884+0000 7f8b8100a700 1 --2- 172.21.15.89:0/3941053468 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b60067170 0x7f8b60069620 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0093f0 tx=0x7f8b6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:24.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:24.888+0000 7f8b71ffb700 1 -- 172.21.15.89:0/3941053468 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b78052940 con 0x7f8b68003a60 2023-10-10T20:39:25.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.119+0000 7f8b83a6f700 1 -- 172.21.15.89:0/3941053468 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b7c15aff0 con 0x7f8b68003a60 2023-10-10T20:39:25.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.120+0000 7f8b71ffb700 1 -- 172.21.15.89:0/3941053468 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+15451 (secure 0 0 0) 0x7f8b78055f60 con 0x7f8b68003a60 2023-10-10T20:39:25.123 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:25.123 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":64,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:37:23.989105+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:36:19.221740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-10T20:36:19.834213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.30","mappings":[{"from":6,"to":0}]},{"pgid":"11.3c","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:39:25.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.122+0000 7f8b83a6f700 1 -- 172.21.15.89:0/3941053468 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b60067170 msgr2=0x7f8b60069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:25.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.122+0000 7f8b83a6f700 1 --2- 172.21.15.89:0/3941053468 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b60067170 0x7f8b60069620 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0093f0 tx=0x7f8b6c008040 comp rx=0 tx=0).stop 2023-10-10T20:39:25.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.122+0000 7f8b83a6f700 1 -- 172.21.15.89:0/3941053468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b68003a60 msgr2=0x7f8b7c16b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:25.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.122+0000 7f8b83a6f700 1 --2- 172.21.15.89:0/3941053468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b68003a60 0x7f8b7c16b010 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800c9b0 tx=0x7f8b78015000 comp rx=0 tx=0).stop 2023-10-10T20:39:25.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.123+0000 7f8b83a6f700 1 -- 172.21.15.89:0/3941053468 shutdown_connections 2023-10-10T20:39:25.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.123+0000 7f8b83a6f700 1 --2- 172.21.15.89:0/3941053468 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b60067170 0x7f8b60069620 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.123+0000 7f8b83a6f700 1 --2- 172.21.15.89:0/3941053468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b7c0cba80 0x7f8b7c0b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.123+0000 7f8b83a6f700 1 --2- 172.21.15.89:0/3941053468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b68003a60 0x7f8b7c16b010 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.123+0000 7f8b83a6f700 1 --2- 172.21.15.89:0/3941053468 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7c154060 0x7f8b7c0cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.123+0000 7f8b83a6f700 1 -- 172.21.15.89:0/3941053468 >> 172.21.15.89:0/3941053468 conn(0x7f8b7c0ce600 msgr2=0x7f8b7c154cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:25.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.123+0000 7f8b83a6f700 1 -- 172.21.15.89:0/3941053468 shutdown_connections 2023-10-10T20:39:25.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.123+0000 7f8b83a6f700 1 -- 172.21.15.89:0/3941053468 wait complete. 2023-10-10T20:39:25.138 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:39:25.139 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:39:25.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.263+0000 7f6b2a168700 1 Processor -- start 2023-10-10T20:39:25.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.263+0000 7f6b2a168700 1 -- start start 2023-10-10T20:39:25.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.264+0000 7f6b2a168700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6b24159420 0x7f6b240b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:25.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.264+0000 7f6b2a168700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6b2415d230 0x7f6b2415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:25.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.264+0000 7f6b2a168700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b240b2980 con 0x7f6b241597f0 2023-10-10T20:39:25.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.264+0000 7f6b2a168700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2415db70 con 0x7f6b24159420 2023-10-10T20:39:25.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.264+0000 7f6b2a168700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b240b11d0 con 0x7f6b2415d230 2023-10-10T20:39:25.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.264+0000 7f6b29166700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6b24159420 0x7f6b240b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:25.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.264+0000 7f6b29166700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6b24159420 0x7f6b240b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43430/0 (socket says 172.21.15.89:43430) 2023-10-10T20:39:25.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.264+0000 7f6b29166700 1 -- 172.21.15.89:0/568333296 learned_addr learned my addr 172.21.15.89:0/568333296 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:25.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.265+0000 7f6b29967700 1 --2- 172.21.15.89:0/568333296 >> v2:172.21.15.111:3300/0 conn(0x7f6b2415d230 0x7f6b2415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:25.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.265+0000 7f6b29166700 1 -- 172.21.15.89:0/568333296 >> v2:172.21.15.111:3300/0 conn(0x7f6b2415d230 msgr2=0x7f6b2415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:25.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.265+0000 7f6b29166700 1 --2- 172.21.15.89:0/568333296 >> v2:172.21.15.111:3300/0 conn(0x7f6b2415d230 0x7f6b2415d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.265+0000 7f6b29166700 1 -- 172.21.15.89:0/568333296 >> v1:172.21.15.89:6789/0 conn(0x7f6b241597f0 legacy=0x7f6b240b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:25.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.265+0000 7f6b1bfff700 1 -- 172.21.15.89:0/568333296 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1421993849 0 0) 0x7f6b240b2980 con 0x7f6b241597f0 2023-10-10T20:39:25.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.265+0000 7f6b29166700 1 -- 172.21.15.89:0/568333296 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b240b14b0 con 0x7f6b24159420 2023-10-10T20:39:25.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.265+0000 7f6b29967700 1 --2- 172.21.15.89:0/568333296 >> v2:172.21.15.111:3300/0 conn(0x7f6b2415d230 0x7f6b2415d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:25.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.265+0000 7f6b29166700 1 --2- 172.21.15.89:0/568333296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b24159420 0x7f6b240b5f40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6b2415d170 tx=0x7f6b20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ce4f7aec4f50ef server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:25.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.265+0000 7f6b1bfff700 1 -- 172.21.15.89:0/568333296 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000b860 con 0x7f6b24159420 2023-10-10T20:39:25.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.265+0000 7f6b1bfff700 1 -- 172.21.15.89:0/568333296 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2000b9c0 con 0x7f6b24159420 2023-10-10T20:39:25.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.265+0000 7f6b1bfff700 1 -- 172.21.15.89:0/568333296 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000bd60 con 0x7f6b24159420 2023-10-10T20:39:25.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.266+0000 7f6b2a168700 1 -- 172.21.15.89:0/568333296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b24159420 msgr2=0x7f6b240b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:25.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.266+0000 7f6b2a168700 1 --2- 172.21.15.89:0/568333296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b24159420 0x7f6b240b5f40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6b2415d170 tx=0x7f6b20009250 comp rx=0 tx=0).stop 2023-10-10T20:39:25.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.266+0000 7f6b2a168700 1 -- 172.21.15.89:0/568333296 shutdown_connections 2023-10-10T20:39:25.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.266+0000 7f6b2a168700 1 --2- 172.21.15.89:0/568333296 >> v2:172.21.15.111:3300/0 conn(0x7f6b2415d230 0x7f6b2415d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.266+0000 7f6b2a168700 1 --2- 172.21.15.89:0/568333296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b24159420 0x7f6b240b5f40 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.266+0000 7f6b2a168700 1 -- 172.21.15.89:0/568333296 >> 172.21.15.89:0/568333296 conn(0x7f6b240d0660 msgr2=0x7f6b240d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:25.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.266+0000 7f6b2a168700 1 -- 172.21.15.89:0/568333296 shutdown_connections 2023-10-10T20:39:25.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.266+0000 7f6b2a168700 1 -- 172.21.15.89:0/568333296 wait complete. 2023-10-10T20:39:25.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.267+0000 7f6b2a168700 1 Processor -- start 2023-10-10T20:39:25.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.267+0000 7f6b2a168700 1 -- start start 2023-10-10T20:39:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.268+0000 7f6b2a168700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b24159420 0x7f6b2416a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.268+0000 7f6b2a168700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6b241597f0 0x7f6b2415c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.268+0000 7f6b2a168700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b2415d230 0x7f6b240c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.268+0000 7f6b2a168700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2416b8b0 con 0x7f6b2415d230 2023-10-10T20:39:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.268+0000 7f6b2a168700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2416aca0 con 0x7f6b24159420 2023-10-10T20:39:25.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.268+0000 7f6b2a168700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2416af80 con 0x7f6b241597f0 2023-10-10T20:39:25.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.268+0000 7f6b29967700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b2415d230 0x7f6b240c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:25.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.268+0000 7f6b29967700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b2415d230 0x7f6b240c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43440/0 (socket says 172.21.15.89:43440) 2023-10-10T20:39:25.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.268+0000 7f6b29967700 1 -- 172.21.15.89:0/3564330403 learned_addr learned my addr 172.21.15.89:0/3564330403 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:25.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.268+0000 7f6b29166700 1 --2- 172.21.15.89:0/3564330403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b24159420 0x7f6b2416a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:25.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.268+0000 7f6b28965700 1 --2- 172.21.15.89:0/3564330403 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6b241597f0 0x7f6b2415c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:25.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.269+0000 7f6b29967700 1 -- 172.21.15.89:0/3564330403 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6b241597f0 msgr2=0x7f6b2415c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:25.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.269+0000 7f6b29967700 1 --2- 172.21.15.89:0/3564330403 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6b241597f0 0x7f6b2415c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.269+0000 7f6b29967700 1 -- 172.21.15.89:0/3564330403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b24159420 msgr2=0x7f6b2416a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:25.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.269+0000 7f6b29967700 1 --2- 172.21.15.89:0/3564330403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b24159420 0x7f6b2416a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.269+0000 7f6b29967700 1 -- 172.21.15.89:0/3564330403 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b200094f0 con 0x7f6b2415d230 2023-10-10T20:39:25.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.269+0000 7f6b29166700 1 --2- 172.21.15.89:0/3564330403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b24159420 0x7f6b2416a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:25.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.269+0000 7f6b29967700 1 --2- 172.21.15.89:0/3564330403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b2415d230 0x7f6b240c9af0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00cef0 tx=0x7f6b1c007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:25.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.269+0000 7f6b197fa700 1 -- 172.21.15.89:0/3564330403 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c00ae00 con 0x7f6b2415d230 2023-10-10T20:39:25.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.269+0000 7f6b197fa700 1 -- 172.21.15.89:0/3564330403 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1c016420 con 0x7f6b2415d230 2023-10-10T20:39:25.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.269+0000 7f6b2a168700 1 -- 172.21.15.89:0/3564330403 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2416b260 con 0x7f6b2415d230 2023-10-10T20:39:25.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.269+0000 7f6b197fa700 1 -- 172.21.15.89:0/3564330403 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c0167e0 con 0x7f6b2415d230 2023-10-10T20:39:25.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.269+0000 7f6b2a168700 1 -- 172.21.15.89:0/3564330403 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2415c580 con 0x7f6b2415d230 2023-10-10T20:39:25.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.270+0000 7f6b2a168700 1 -- 172.21.15.89:0/3564330403 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2404c3c0 con 0x7f6b2415d230 2023-10-10T20:39:25.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.271+0000 7f6b197fa700 1 -- 172.21.15.89:0/3564330403 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f6b1c016bd0 con 0x7f6b2415d230 2023-10-10T20:39:25.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.271+0000 7f6b197fa700 1 --2- 172.21.15.89:0/3564330403 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6b08067170 0x7f6b08069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:25.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.271+0000 7f6b197fa700 1 -- 172.21.15.89:0/3564330403 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f6b1c012070 con 0x7f6b2415d230 2023-10-10T20:39:25.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.271+0000 7f6b29166700 1 --2- 172.21.15.89:0/3564330403 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6b08067170 0x7f6b08069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:25.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.272+0000 7f6b29166700 1 --2- 172.21.15.89:0/3564330403 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6b08067170 0x7f6b08069620 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6b2415d170 tx=0x7f6b20017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:25.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.276+0000 7f6b197fa700 1 -- 172.21.15.89:0/3564330403 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b1c04e5a0 con 0x7f6b2415d230 2023-10-10T20:39:25.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.577+0000 7f6b2a168700 1 -- 172.21.15.89:0/3564330403 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b2416b9f0 con 0x7f6b2415d230 2023-10-10T20:39:25.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.578+0000 7f6b197fa700 1 -- 172.21.15.89:0/3564330403 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f6b1c052200 con 0x7f6b2415d230 2023-10-10T20:39:25.581 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:25.581 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/3362909084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":3362909084},{"type":"v1","addr":"172.21.15.111:6835","nonce":3362909084}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5102,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1472423656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1472423656},{"type":"v1","addr":"172.21.15.111:6834","nonce":1472423656}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/3183325104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3183325104},{"type":"v1","addr":"172.21.15.89:6836","nonce":3183325104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:36:20.867329+0000","modified":"2023-10-10T20:36:24.518115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/862652054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":862652054},{"type":"v1","addr":"172.21.15.89:6837","nonce":862652054}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-10T20:39:25.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.580+0000 7f6b2a168700 1 -- 172.21.15.89:0/3564330403 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6b08067170 msgr2=0x7f6b08069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:25.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.580+0000 7f6b2a168700 1 --2- 172.21.15.89:0/3564330403 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6b08067170 0x7f6b08069620 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6b2415d170 tx=0x7f6b20017000 comp rx=0 tx=0).stop 2023-10-10T20:39:25.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.580+0000 7f6b2a168700 1 -- 172.21.15.89:0/3564330403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b2415d230 msgr2=0x7f6b240c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:25.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.580+0000 7f6b2a168700 1 --2- 172.21.15.89:0/3564330403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b2415d230 0x7f6b240c9af0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00cef0 tx=0x7f6b1c007860 comp rx=0 tx=0).stop 2023-10-10T20:39:25.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.580+0000 7f6b2a168700 1 -- 172.21.15.89:0/3564330403 shutdown_connections 2023-10-10T20:39:25.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.580+0000 7f6b2a168700 1 --2- 172.21.15.89:0/3564330403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b2415d230 0x7f6b240c9af0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.580+0000 7f6b2a168700 1 --2- 172.21.15.89:0/3564330403 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6b08067170 0x7f6b08069620 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.580+0000 7f6b2a168700 1 --2- 172.21.15.89:0/3564330403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b24159420 0x7f6b2416a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.580+0000 7f6b2a168700 1 --2- 172.21.15.89:0/3564330403 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6b241597f0 0x7f6b2415c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.580+0000 7f6b2a168700 1 -- 172.21.15.89:0/3564330403 >> 172.21.15.89:0/3564330403 conn(0x7f6b240d0660 msgr2=0x7f6b240cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:25.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.580+0000 7f6b2a168700 1 -- 172.21.15.89:0/3564330403 shutdown_connections 2023-10-10T20:39:25.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.580+0000 7f6b2a168700 1 -- 172.21.15.89:0/3564330403 wait complete. 2023-10-10T20:39:25.585 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 38 2023-10-10T20:39:25.598 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:39:25.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.723+0000 7f9bd3fff700 1 Processor -- start 2023-10-10T20:39:25.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.723+0000 7f9bd3fff700 1 -- start start 2023-10-10T20:39:25.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.723+0000 7f9bd3fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bd40c6c60 0x7f9bd40c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:25.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.723+0000 7f9bd3fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9bd406e8c0 0x7f9bd40c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:25.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.724+0000 7f9bd3fff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd40b1410 con 0x7f9bd41595a0 2023-10-10T20:39:25.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.724+0000 7f9bd3fff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd40b2b90 con 0x7f9bd406e8c0 2023-10-10T20:39:25.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.724+0000 7f9bd3fff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd4063230 con 0x7f9bd40c6c60 2023-10-10T20:39:25.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.724+0000 7f9bd37fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9bd406e8c0 0x7f9bd40c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:25.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.724+0000 7f9bd27fc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bd40c6c60 0x7f9bd40c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:25.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.724+0000 7f9bd37fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9bd406e8c0 0x7f9bd40c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:41710/0 (socket says 172.21.15.89:41710) 2023-10-10T20:39:25.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.724+0000 7f9bd27fc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bd40c6c60 0x7f9bd40c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:52132/0 (socket says 172.21.15.89:52132) 2023-10-10T20:39:25.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.724+0000 7f9bd2ffd700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f9bd41595a0 0x7f9bd40b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:40560/0 (socket says 172.21.15.89:40560) 2023-10-10T20:39:25.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.724+0000 7f9bd37fe700 1 -- 172.21.15.89:0/1276494320 learned_addr learned my addr 172.21.15.89:0/1276494320 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:25.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.724+0000 7f9bd37fe700 1 -- 172.21.15.89:0/1276494320 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bd40c6c60 msgr2=0x7f9bd40c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:25.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.724+0000 7f9bd37fe700 1 --2- 172.21.15.89:0/1276494320 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bd40c6c60 0x7f9bd40c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.724+0000 7f9bd37fe700 1 -- 172.21.15.89:0/1276494320 >> v1:172.21.15.111:6789/0 conn(0x7f9bd41595a0 legacy=0x7f9bd40b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:25.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.724+0000 7f9bd37fe700 1 -- 172.21.15.89:0/1276494320 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd40634f0 con 0x7f9bd406e8c0 2023-10-10T20:39:25.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.724+0000 7f9bd1ffb700 1 -- 172.21.15.89:0/1276494320 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604206160 0 0) 0x7f9bd40b1410 con 0x7f9bd41595a0 2023-10-10T20:39:25.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.725+0000 7f9bd37fe700 1 --2- 172.21.15.89:0/1276494320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bd406e8c0 0x7f9bd40c8660 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc009f20 tx=0x7f9bcc009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14541210658c00c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:25.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.725+0000 7f9bd1ffb700 1 -- 172.21.15.89:0/1276494320 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc00e070 con 0x7f9bd406e8c0 2023-10-10T20:39:25.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.725+0000 7f9bd1ffb700 1 -- 172.21.15.89:0/1276494320 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bcc00b9d0 con 0x7f9bd406e8c0 2023-10-10T20:39:25.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.725+0000 7f9bd1ffb700 1 -- 172.21.15.89:0/1276494320 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc0092e0 con 0x7f9bd406e8c0 2023-10-10T20:39:25.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.725+0000 7f9bd3fff700 1 -- 172.21.15.89:0/1276494320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bd406e8c0 msgr2=0x7f9bd40c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:25.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.725+0000 7f9bd3fff700 1 --2- 172.21.15.89:0/1276494320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bd406e8c0 0x7f9bd40c8660 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc009f20 tx=0x7f9bcc009840 comp rx=0 tx=0).stop 2023-10-10T20:39:25.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.725+0000 7f9bd3fff700 1 -- 172.21.15.89:0/1276494320 shutdown_connections 2023-10-10T20:39:25.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.725+0000 7f9bd3fff700 1 --2- 172.21.15.89:0/1276494320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bd406e8c0 0x7f9bd40c8660 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.725+0000 7f9bd3fff700 1 --2- 172.21.15.89:0/1276494320 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bd40c6c60 0x7f9bd40c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.725+0000 7f9bd3fff700 1 -- 172.21.15.89:0/1276494320 >> 172.21.15.89:0/1276494320 conn(0x7f9bd406ddb0 msgr2=0x7f9bd406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:25.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.725+0000 7f9bd3fff700 1 -- 172.21.15.89:0/1276494320 shutdown_connections 2023-10-10T20:39:25.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.725+0000 7f9bd3fff700 1 -- 172.21.15.89:0/1276494320 wait complete. 2023-10-10T20:39:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.727+0000 7f9bd3fff700 1 Processor -- start 2023-10-10T20:39:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.727+0000 7f9bd3fff700 1 -- start start 2023-10-10T20:39:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.727+0000 7f9bd3fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bd406e8c0 0x7f9bd40c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.727+0000 7f9bd3fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bd40c6c60 0x7f9bd40c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.727+0000 7f9bd3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd41595a0 0x7f9bd4150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:25.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.727+0000 7f9bd3fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd40c58c0 con 0x7f9bd41595a0 2023-10-10T20:39:25.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.727+0000 7f9bd3fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd40c4cb0 con 0x7f9bd40c6c60 2023-10-10T20:39:25.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.727+0000 7f9bd3fff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd40c4f90 con 0x7f9bd406e8c0 2023-10-10T20:39:25.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.727+0000 7f9bd37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd41595a0 0x7f9bd4150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:25.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.727+0000 7f9bd37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd41595a0 0x7f9bd4150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43454/0 (socket says 172.21.15.89:43454) 2023-10-10T20:39:25.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.728+0000 7f9bd37fe700 1 -- 172.21.15.89:0/1309213450 learned_addr learned my addr 172.21.15.89:0/1309213450 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:25.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.728+0000 7f9bd27fc700 1 --2- 172.21.15.89:0/1309213450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bd40c6c60 0x7f9bd40c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:25.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.728+0000 7f9bd2ffd700 1 --2- 172.21.15.89:0/1309213450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bd406e8c0 0x7f9bd40c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:25.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.728+0000 7f9bd37fe700 1 -- 172.21.15.89:0/1309213450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bd406e8c0 msgr2=0x7f9bd40c45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:25.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.728+0000 7f9bd37fe700 1 --2- 172.21.15.89:0/1309213450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bd406e8c0 0x7f9bd40c45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.728+0000 7f9bd37fe700 1 -- 172.21.15.89:0/1309213450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bd40c6c60 msgr2=0x7f9bd40c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:25.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.728+0000 7f9bd37fe700 1 --2- 172.21.15.89:0/1309213450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bd40c6c60 0x7f9bd40c1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.728+0000 7f9bd37fe700 1 -- 172.21.15.89:0/1309213450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc009ae0 con 0x7f9bd41595a0 2023-10-10T20:39:25.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.728+0000 7f9bd37fe700 1 --2- 172.21.15.89:0/1309213450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd41595a0 0x7f9bd4150960 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4064340 tx=0x7f9bcc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.728+0000 7f9bbaffd700 1 -- 172.21.15.89:0/1309213450 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc008b30 con 0x7f9bd41595a0 2023-10-10T20:39:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.728+0000 7f9bbaffd700 1 -- 172.21.15.89:0/1309213450 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bcc008c90 con 0x7f9bd41595a0 2023-10-10T20:39:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.728+0000 7f9bd3fff700 1 -- 172.21.15.89:0/1309213450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd40c5270 con 0x7f9bd41595a0 2023-10-10T20:39:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.728+0000 7f9bbaffd700 1 -- 172.21.15.89:0/1309213450 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc0266b0 con 0x7f9bd41595a0 2023-10-10T20:39:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.728+0000 7f9bd3fff700 1 -- 172.21.15.89:0/1309213450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd40c2090 con 0x7f9bd41595a0 2023-10-10T20:39:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.729+0000 7f9bd3fff700 1 -- 172.21.15.89:0/1309213450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd404c3c0 con 0x7f9bd41595a0 2023-10-10T20:39:25.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.730+0000 7f9bbaffd700 1 -- 172.21.15.89:0/1309213450 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f9bcc003040 con 0x7f9bd41595a0 2023-10-10T20:39:25.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.730+0000 7f9bbaffd700 1 --2- 172.21.15.89:0/1309213450 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9bc0067170 0x7f9bc0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:25.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.730+0000 7f9bd2ffd700 1 --2- 172.21.15.89:0/1309213450 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9bc0067170 0x7f9bc0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:25.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.730+0000 7f9bbaffd700 1 -- 172.21.15.89:0/1309213450 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f9bcc0867d0 con 0x7f9bd41595a0 2023-10-10T20:39:25.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.731+0000 7f9bd2ffd700 1 --2- 172.21.15.89:0/1309213450 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9bc0067170 0x7f9bc0069620 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4007180 tx=0x7f9bc4007ef0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:25.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.735+0000 7f9bbaffd700 1 -- 172.21.15.89:0/1309213450 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bcc053660 con 0x7f9bd41595a0 2023-10-10T20:39:25.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.990+0000 7f9bd3fff700 1 -- 172.21.15.89:0/1309213450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9bd40637d0 con 0x7f9bd41595a0 2023-10-10T20:39:25.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.990+0000 7f9bbaffd700 1 -- 172.21.15.89:0/1309213450 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f9bcc056c80 con 0x7f9bd41595a0 2023-10-10T20:39:25.993 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:25.993 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/3362909084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":3362909084},{"type":"v1","addr":"172.21.15.111:6835","nonce":3362909084}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5102,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1472423656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1472423656},{"type":"v1","addr":"172.21.15.111:6834","nonce":1472423656}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/3183325104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3183325104},{"type":"v1","addr":"172.21.15.89:6836","nonce":3183325104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:36:20.867329+0000","modified":"2023-10-10T20:36:24.518115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/862652054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":862652054},{"type":"v1","addr":"172.21.15.89:6837","nonce":862652054}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-10T20:39:25.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.992+0000 7f9bd3fff700 1 -- 172.21.15.89:0/1309213450 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9bc0067170 msgr2=0x7f9bc0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:25.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.992+0000 7f9bd3fff700 1 --2- 172.21.15.89:0/1309213450 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9bc0067170 0x7f9bc0069620 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4007180 tx=0x7f9bc4007ef0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.992+0000 7f9bd3fff700 1 -- 172.21.15.89:0/1309213450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd41595a0 msgr2=0x7f9bd4150960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:25.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.992+0000 7f9bd3fff700 1 --2- 172.21.15.89:0/1309213450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd41595a0 0x7f9bd4150960 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4064340 tx=0x7f9bcc018000 comp rx=0 tx=0).stop 2023-10-10T20:39:25.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.992+0000 7f9bd3fff700 1 -- 172.21.15.89:0/1309213450 shutdown_connections 2023-10-10T20:39:25.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.992+0000 7f9bd3fff700 1 --2- 172.21.15.89:0/1309213450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd41595a0 0x7f9bd4150960 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.992+0000 7f9bd3fff700 1 --2- 172.21.15.89:0/1309213450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bd406e8c0 0x7f9bd40c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.992+0000 7f9bd3fff700 1 --2- 172.21.15.89:0/1309213450 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9bc0067170 0x7f9bc0069620 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.992+0000 7f9bd3fff700 1 --2- 172.21.15.89:0/1309213450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bd40c6c60 0x7f9bd40c1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:25.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.992+0000 7f9bd3fff700 1 -- 172.21.15.89:0/1309213450 >> 172.21.15.89:0/1309213450 conn(0x7f9bd406ddb0 msgr2=0x7f9bd415cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:25.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.993+0000 7f9bd3fff700 1 -- 172.21.15.89:0/1309213450 shutdown_connections 2023-10-10T20:39:25.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:25.993+0000 7f9bd3fff700 1 -- 172.21.15.89:0/1309213450 wait complete. 2023-10-10T20:39:25.997 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 38 2023-10-10T20:39:26.007 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:39:26.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.141+0000 7f817b59e700 1 Processor -- start 2023-10-10T20:39:26.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.141+0000 7f817b59e700 1 -- start start 2023-10-10T20:39:26.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.142+0000 7f817b59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f817c13f620 0x7f817c0af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:26.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.142+0000 7f817b59e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f817c0c8120 0x7f817c0c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:26.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.142+0000 7f817b59e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f817c156270 0x7f817c1566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:26.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.142+0000 7f817b59e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f817c0c8ad0 con 0x7f817c0c8120 2023-10-10T20:39:26.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.142+0000 7f817b59e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f817c0b2080 con 0x7f817c156270 2023-10-10T20:39:26.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.142+0000 7f817b59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817c06eb10 con 0x7f817c13f620 2023-10-10T20:39:26.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.142+0000 7f8179d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f817c0c8120 0x7f817c0c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:26.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.142+0000 7f8179d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f817c0c8120 0x7f817c0c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43462/0 (socket says 172.21.15.89:43462) 2023-10-10T20:39:26.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.142+0000 7f8179d9b700 1 -- 172.21.15.89:0/701750200 learned_addr learned my addr 172.21.15.89:0/701750200 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:26.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.142+0000 7f817ad9d700 1 --2- 172.21.15.89:0/701750200 >> v2:172.21.15.111:3300/0 conn(0x7f817c156270 0x7f817c1566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:26.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.142+0000 7f817a59c700 1 --2- 172.21.15.89:0/701750200 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f817c13f620 0x7f817c0af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:26.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.142+0000 7f8179d9b700 1 -- 172.21.15.89:0/701750200 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f817c13f620 msgr2=0x7f817c0af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:26.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.142+0000 7f8179d9b700 1 --2- 172.21.15.89:0/701750200 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f817c13f620 0x7f817c0af9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.142+0000 7f8179d9b700 1 -- 172.21.15.89:0/701750200 >> v2:172.21.15.111:3300/0 conn(0x7f817c156270 msgr2=0x7f817c1566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:26.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.142+0000 7f8179d9b700 1 --2- 172.21.15.89:0/701750200 >> v2:172.21.15.111:3300/0 conn(0x7f817c156270 0x7f817c1566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.142+0000 7f8179d9b700 1 -- 172.21.15.89:0/701750200 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817c0b2320 con 0x7f817c0c8120 2023-10-10T20:39:26.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.142+0000 7f817a59c700 1 --2- 172.21.15.89:0/701750200 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f817c13f620 0x7f817c0af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:26.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.142+0000 7f8179d9b700 1 --2- 172.21.15.89:0/701750200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f817c0c8120 0x7f817c0c8590 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f817c159790 tx=0x7f817000a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0047fdfcec8a44 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:26.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.143+0000 7f817959a700 1 -- 172.21.15.89:0/701750200 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817000ca40 con 0x7f817c0c8120 2023-10-10T20:39:26.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.143+0000 7f817959a700 1 -- 172.21.15.89:0/701750200 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817000cba0 con 0x7f817c0c8120 2023-10-10T20:39:26.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.143+0000 7f817959a700 1 -- 172.21.15.89:0/701750200 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170018400 con 0x7f817c0c8120 2023-10-10T20:39:26.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.143+0000 7f817b59e700 1 -- 172.21.15.89:0/701750200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f817c0c8120 msgr2=0x7f817c0c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:26.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.143+0000 7f817b59e700 1 --2- 172.21.15.89:0/701750200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f817c0c8120 0x7f817c0c8590 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f817c159790 tx=0x7f817000a3f0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.143+0000 7f817b59e700 1 -- 172.21.15.89:0/701750200 shutdown_connections 2023-10-10T20:39:26.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.143+0000 7f817b59e700 1 --2- 172.21.15.89:0/701750200 >> v2:172.21.15.111:3300/0 conn(0x7f817c156270 0x7f817c1566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.143+0000 7f817b59e700 1 --2- 172.21.15.89:0/701750200 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f817c13f620 0x7f817c0af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.143+0000 7f817b59e700 1 --2- 172.21.15.89:0/701750200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f817c0c8120 0x7f817c0c8590 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.143+0000 7f817b59e700 1 -- 172.21.15.89:0/701750200 >> 172.21.15.89:0/701750200 conn(0x7f817c06ddb0 msgr2=0x7f817c06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:26.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.143+0000 7f817b59e700 1 -- 172.21.15.89:0/701750200 shutdown_connections 2023-10-10T20:39:26.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.143+0000 7f817b59e700 1 -- 172.21.15.89:0/701750200 wait complete. 2023-10-10T20:39:26.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.144+0000 7f817b59e700 1 Processor -- start 2023-10-10T20:39:26.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.145+0000 7f817b59e700 1 -- start start 2023-10-10T20:39:26.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.145+0000 7f817b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f817c0c8120 0x7f817c167910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:26.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.145+0000 7f817b59e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f817c13f620 0x7f817c0aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:26.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.145+0000 7f817b59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f817c156270 0x7f817c155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:26.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.145+0000 7f817b59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c0c57f0 con 0x7f817c0c8120 2023-10-10T20:39:26.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.145+0000 7f817b59e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c167e50 con 0x7f817c13f620 2023-10-10T20:39:26.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.145+0000 7f817b59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817c0c4be0 con 0x7f817c156270 2023-10-10T20:39:26.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.145+0000 7f817a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f817c0c8120 0x7f817c167910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:26.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.145+0000 7f817a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f817c0c8120 0x7f817c167910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43470/0 (socket says 172.21.15.89:43470) 2023-10-10T20:39:26.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.145+0000 7f817a59c700 1 -- 172.21.15.89:0/2037308958 learned_addr learned my addr 172.21.15.89:0/2037308958 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:26.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.146+0000 7f8179d9b700 1 --2- 172.21.15.89:0/2037308958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f817c13f620 0x7f817c0aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:26.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.146+0000 7f817ad9d700 1 --2- 172.21.15.89:0/2037308958 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f817c156270 0x7f817c155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:26.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.146+0000 7f817a59c700 1 -- 172.21.15.89:0/2037308958 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f817c156270 msgr2=0x7f817c155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:26.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.146+0000 7f817a59c700 1 --2- 172.21.15.89:0/2037308958 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f817c156270 0x7f817c155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.146+0000 7f817a59c700 1 -- 172.21.15.89:0/2037308958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f817c13f620 msgr2=0x7f817c0aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:26.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.146+0000 7f817a59c700 1 --2- 172.21.15.89:0/2037308958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f817c13f620 0x7f817c0aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.146+0000 7f817a59c700 1 -- 172.21.15.89:0/2037308958 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817000a690 con 0x7f817c0c8120 2023-10-10T20:39:26.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.146+0000 7f8179d9b700 1 --2- 172.21.15.89:0/2037308958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f817c13f620 0x7f817c0aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:26.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.146+0000 7f817a59c700 1 --2- 172.21.15.89:0/2037308958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f817c0c8120 0x7f817c167910 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f816c00c850 tx=0x7f816c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:26.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.146+0000 7f816a7fc700 1 -- 172.21.15.89:0/2037308958 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c011600 con 0x7f817c0c8120 2023-10-10T20:39:26.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.146+0000 7f816a7fc700 1 -- 172.21.15.89:0/2037308958 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816c011760 con 0x7f817c0c8120 2023-10-10T20:39:26.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.146+0000 7f817b59e700 1 -- 172.21.15.89:0/2037308958 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817c0c4ec0 con 0x7f817c0c8120 2023-10-10T20:39:26.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.146+0000 7f816a7fc700 1 -- 172.21.15.89:0/2037308958 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c011b00 con 0x7f817c0c8120 2023-10-10T20:39:26.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.146+0000 7f817b59e700 1 -- 172.21.15.89:0/2037308958 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817c0c5480 con 0x7f817c0c8120 2023-10-10T20:39:26.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.147+0000 7f817b59e700 1 -- 172.21.15.89:0/2037308958 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f817c04c3c0 con 0x7f817c0c8120 2023-10-10T20:39:26.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.148+0000 7f816a7fc700 1 -- 172.21.15.89:0/2037308958 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f816c0073b0 con 0x7f817c0c8120 2023-10-10T20:39:26.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.148+0000 7f816a7fc700 1 --2- 172.21.15.89:0/2037308958 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f815c067170 0x7f815c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:26.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.148+0000 7f816a7fc700 1 -- 172.21.15.89:0/2037308958 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f816c084980 con 0x7f817c0c8120 2023-10-10T20:39:26.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.148+0000 7f8179d9b700 1 --2- 172.21.15.89:0/2037308958 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f815c067170 0x7f815c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:26.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.148+0000 7f8179d9b700 1 --2- 172.21.15.89:0/2037308958 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f815c067170 0x7f815c069620 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f817c159790 tx=0x7f8170018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:26.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.152+0000 7f816a7fc700 1 -- 172.21.15.89:0/2037308958 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f816c051810 con 0x7f817c0c8120 2023-10-10T20:39:26.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.378+0000 7f817b59e700 1 -- 172.21.15.89:0/2037308958 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f817c0b2600 con 0x7f817c0c8120 2023-10-10T20:39:26.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.379+0000 7f816a7fc700 1 -- 172.21.15.89:0/2037308958 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+15451 (secure 0 0 0) 0x7f816c01e090 con 0x7f817c0c8120 2023-10-10T20:39:26.381 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:26.382 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":64,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:37:23.989105+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:36:19.221740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-10T20:36:19.834213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.30","mappings":[{"from":6,"to":0}]},{"pgid":"11.3c","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:39:26.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.381+0000 7f817b59e700 1 -- 172.21.15.89:0/2037308958 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f815c067170 msgr2=0x7f815c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:26.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.381+0000 7f817b59e700 1 --2- 172.21.15.89:0/2037308958 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f815c067170 0x7f815c069620 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f817c159790 tx=0x7f8170018000 comp rx=0 tx=0).stop 2023-10-10T20:39:26.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.381+0000 7f817b59e700 1 -- 172.21.15.89:0/2037308958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f817c0c8120 msgr2=0x7f817c167910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:26.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.381+0000 7f817b59e700 1 --2- 172.21.15.89:0/2037308958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f817c0c8120 0x7f817c167910 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f816c00c850 tx=0x7f816c008040 comp rx=0 tx=0).stop 2023-10-10T20:39:26.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.381+0000 7f817b59e700 1 -- 172.21.15.89:0/2037308958 shutdown_connections 2023-10-10T20:39:26.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.381+0000 7f817b59e700 1 --2- 172.21.15.89:0/2037308958 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f817c156270 0x7f817c155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.381+0000 7f817b59e700 1 --2- 172.21.15.89:0/2037308958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f817c0c8120 0x7f817c167910 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.381+0000 7f817b59e700 1 --2- 172.21.15.89:0/2037308958 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f815c067170 0x7f815c069620 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.381+0000 7f817b59e700 1 --2- 172.21.15.89:0/2037308958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f817c13f620 0x7f817c0aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.381+0000 7f817b59e700 1 -- 172.21.15.89:0/2037308958 >> 172.21.15.89:0/2037308958 conn(0x7f817c06ddb0 msgr2=0x7f817c0af9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:26.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.381+0000 7f817b59e700 1 -- 172.21.15.89:0/2037308958 shutdown_connections 2023-10-10T20:39:26.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.381+0000 7f817b59e700 1 -- 172.21.15.89:0/2037308958 wait complete. 2023-10-10T20:39:26.396 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:39:26.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.522+0000 7f52e5c57700 1 Processor -- start 2023-10-10T20:39:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.522+0000 7f52e5c57700 1 -- start start 2023-10-10T20:39:26.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.523+0000 7f52e5c57700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52e0069560 0x7f52e00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:26.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.523+0000 7f52e5c57700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e0069ee0 con 0x7f52e0159c60 2023-10-10T20:39:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.523+0000 7f52e5c57700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e015a160 con 0x7f52e0159890 2023-10-10T20:39:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.523+0000 7f52e5c57700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e00d13a0 con 0x7f52e0069560 2023-10-10T20:39:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.523+0000 7f52deffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f52e0159c60 0x7f52e015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:32914/0 (socket says 172.21.15.89:32914) 2023-10-10T20:39:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.523+0000 7f52deffd700 1 -- 172.21.15.89:0/626834079 learned_addr learned my addr 172.21.15.89:0/626834079 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.523+0000 7f52dffff700 1 --2- 172.21.15.89:0/626834079 >> v2:172.21.15.111:3300/0 conn(0x7f52e0069560 0x7f52e00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.523+0000 7f52de7fc700 1 -- 172.21.15.89:0/626834079 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591878531 0 0) 0x7f52e0069ee0 con 0x7f52e0159c60 2023-10-10T20:39:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.523+0000 7f52de7fc700 1 -- 172.21.15.89:0/626834079 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52c8003570 con 0x7f52e0159c60 2023-10-10T20:39:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.524+0000 7f52dffff700 1 -- 172.21.15.89:0/626834079 >> v1:172.21.15.111:6789/0 conn(0x7f52e0159890 legacy=0x7f52e015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.524+0000 7f52dffff700 1 -- 172.21.15.89:0/626834079 >> v1:172.21.15.89:6789/0 conn(0x7f52e0159c60 legacy=0x7f52e015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.524+0000 7f52dffff700 1 -- 172.21.15.89:0/626834079 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e015e140 con 0x7f52e0069560 2023-10-10T20:39:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.524+0000 7f52dffff700 1 --2- 172.21.15.89:0/626834079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52e0069560 0x7f52e00699a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009e60 tx=0x7f52d8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e9d04fe32f20e1f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.524+0000 7f52de7fc700 1 -- 172.21.15.89:0/626834079 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d800e070 con 0x7f52e0069560 2023-10-10T20:39:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.524+0000 7f52de7fc700 1 -- 172.21.15.89:0/626834079 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d800bb60 con 0x7f52e0069560 2023-10-10T20:39:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.524+0000 7f52de7fc700 1 -- 172.21.15.89:0/626834079 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d8012430 con 0x7f52e0069560 2023-10-10T20:39:26.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.525+0000 7f52e5c57700 1 -- 172.21.15.89:0/626834079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52e0069560 msgr2=0x7f52e00699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:26.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.525+0000 7f52e5c57700 1 --2- 172.21.15.89:0/626834079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52e0069560 0x7f52e00699a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009e60 tx=0x7f52d8009780 comp rx=0 tx=0).stop 2023-10-10T20:39:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.525+0000 7f52e5c57700 1 -- 172.21.15.89:0/626834079 shutdown_connections 2023-10-10T20:39:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.525+0000 7f52e5c57700 1 --2- 172.21.15.89:0/626834079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52e0069560 0x7f52e00699a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.525+0000 7f52e5c57700 1 -- 172.21.15.89:0/626834079 >> 172.21.15.89:0/626834079 conn(0x7f52e0068e20 msgr2=0x7f52e00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.525+0000 7f52e5c57700 1 -- 172.21.15.89:0/626834079 shutdown_connections 2023-10-10T20:39:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.525+0000 7f52e5c57700 1 -- 172.21.15.89:0/626834079 wait complete. 2023-10-10T20:39:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.527+0000 7f52e5c57700 1 Processor -- start 2023-10-10T20:39:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.527+0000 7f52e5c57700 1 -- start start 2023-10-10T20:39:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.528+0000 7f52e5c57700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f52e0069560 0x7f52e00b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.528+0000 7f52e5c57700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52e0159890 0x7f52e00b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.528+0000 7f52e5c57700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52e0159c60 0x7f52e00b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.528+0000 7f52e5c57700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e00cffc0 con 0x7f52e0159c60 2023-10-10T20:39:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.528+0000 7f52e5c57700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e00b6350 con 0x7f52e0159890 2023-10-10T20:39:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.528+0000 7f52e5c57700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e00cd2f0 con 0x7f52e0069560 2023-10-10T20:39:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.528+0000 7f52dffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52e0159c60 0x7f52e00b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.528+0000 7f52deffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52e0159890 0x7f52e00b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.528+0000 7f52df7fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f52e0069560 0x7f52e00b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.528+0000 7f52dffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52e0159c60 0x7f52e00b5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43474/0 (socket says 172.21.15.89:43474) 2023-10-10T20:39:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.528+0000 7f52deffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52e0159890 0x7f52e00b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:41740/0 (socket says 172.21.15.89:41740) 2023-10-10T20:39:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.528+0000 7f52dffff700 1 -- 172.21.15.89:0/3681670163 learned_addr learned my addr 172.21.15.89:0/3681670163 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.529+0000 7f52dffff700 1 -- 172.21.15.89:0/3681670163 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f52e0069560 msgr2=0x7f52e00b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.529+0000 7f52dffff700 1 --2- 172.21.15.89:0/3681670163 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f52e0069560 0x7f52e00b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.529+0000 7f52dffff700 1 -- 172.21.15.89:0/3681670163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52e0159890 msgr2=0x7f52e00b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:26.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.529+0000 7f52dffff700 1 --2- 172.21.15.89:0/3681670163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52e0159890 0x7f52e00b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.529+0000 7f52dffff700 1 -- 172.21.15.89:0/3681670163 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d8009a20 con 0x7f52e0159c60 2023-10-10T20:39:26.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.529+0000 7f52deffd700 1 --2- 172.21.15.89:0/3681670163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52e0159890 0x7f52e00b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:39:26.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.529+0000 7f52dffff700 1 --2- 172.21.15.89:0/3681670163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52e0159c60 0x7f52e00b5440 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f52e006a070 tx=0x7f52d8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:26.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.529+0000 7f52c77fe700 1 -- 172.21.15.89:0/3681670163 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d8027470 con 0x7f52e0159c60 2023-10-10T20:39:26.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.529+0000 7f52c77fe700 1 -- 172.21.15.89:0/3681670163 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d80275f0 con 0x7f52e0159c60 2023-10-10T20:39:26.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.529+0000 7f52e5c57700 1 -- 172.21.15.89:0/3681670163 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e00cd5d0 con 0x7f52e0159c60 2023-10-10T20:39:26.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.529+0000 7f52c77fe700 1 -- 172.21.15.89:0/3681670163 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d80279b0 con 0x7f52e0159c60 2023-10-10T20:39:26.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.529+0000 7f52e5c57700 1 -- 172.21.15.89:0/3681670163 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e00cdb90 con 0x7f52e0159c60 2023-10-10T20:39:26.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.530+0000 7f52e5c57700 1 -- 172.21.15.89:0/3681670163 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e004c3c0 con 0x7f52e0159c60 2023-10-10T20:39:26.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.531+0000 7f52c77fe700 1 -- 172.21.15.89:0/3681670163 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f52d8024070 con 0x7f52e0159c60 2023-10-10T20:39:26.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.531+0000 7f52c77fe700 1 --2- 172.21.15.89:0/3681670163 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f52cc067170 0x7f52cc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:26.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.532+0000 7f52c77fe700 1 -- 172.21.15.89:0/3681670163 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f52d8086f50 con 0x7f52e0159c60 2023-10-10T20:39:26.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.532+0000 7f52df7fe700 1 --2- 172.21.15.89:0/3681670163 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f52cc067170 0x7f52cc069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:26.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.532+0000 7f52df7fe700 1 --2- 172.21.15.89:0/3681670163 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f52cc067170 0x7f52cc069620 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f52d0003a80 tx=0x7f52d0007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:26.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.536+0000 7f52c77fe700 1 -- 172.21.15.89:0/3681670163 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52d8053ef0 con 0x7f52e0159c60 2023-10-10T20:39:26.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.861+0000 7f52e5c57700 1 -- 172.21.15.89:0/3681670163 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f52e00b2780 con 0x7f52e0159c60 2023-10-10T20:39:26.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-10-10T20:39:26.862+0000 7fd0184ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:39:26.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.867+0000 7f52c77fe700 1 -- 172.21.15.89:0/3681670163 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) v1 ==== 161+0+0 (secure 0 0 0) 0x7f52d8057510 con 0x7f52e0159c60 2023-10-10T20:39:26.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.869+0000 7f52e5c57700 1 -- 172.21.15.89:0/3681670163 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f52cc067170 msgr2=0x7f52cc069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:26.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.869+0000 7f52e5c57700 1 --2- 172.21.15.89:0/3681670163 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f52cc067170 0x7f52cc069620 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f52d0003a80 tx=0x7f52d0007ec0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.869+0000 7f52e5c57700 1 -- 172.21.15.89:0/3681670163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52e0159c60 msgr2=0x7f52e00b5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:26.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.869+0000 7f52e5c57700 1 --2- 172.21.15.89:0/3681670163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52e0159c60 0x7f52e00b5440 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f52e006a070 tx=0x7f52d8019000 comp rx=0 tx=0).stop 2023-10-10T20:39:26.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.869+0000 7f52e5c57700 1 -- 172.21.15.89:0/3681670163 shutdown_connections 2023-10-10T20:39:26.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.869+0000 7f52e5c57700 1 --2- 172.21.15.89:0/3681670163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52e0159c60 0x7f52e00b5440 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.869+0000 7f52e5c57700 1 --2- 172.21.15.89:0/3681670163 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f52e0069560 0x7f52e00b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.869+0000 7f52e5c57700 1 --2- 172.21.15.89:0/3681670163 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f52cc067170 0x7f52cc069620 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.869+0000 7f52e5c57700 1 --2- 172.21.15.89:0/3681670163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52e0159890 0x7f52e00b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:26.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.869+0000 7f52e5c57700 1 -- 172.21.15.89:0/3681670163 >> 172.21.15.89:0/3681670163 conn(0x7f52e0068e20 msgr2=0x7f52e01553c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:26.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.869+0000 7f52e5c57700 1 -- 172.21.15.89:0/3681670163 shutdown_connections 2023-10-10T20:39:26.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:26.869+0000 7f52e5c57700 1 -- 172.21.15.89:0/3681670163 wait complete. 2023-10-10T20:39:26.874 INFO:teuthology.orchestra.run.smithi089.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:39:26.884 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:39:26.904 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-10T20:39:27.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.000+0000 7f02c6ea5700 1 Processor -- start 2023-10-10T20:39:27.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.000+0000 7f02c6ea5700 1 -- start start 2023-10-10T20:39:27.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.001+0000 7f02c6ea5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02c00aca80 0x7f02c006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:27.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.001+0000 7f02c6ea5700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f02c0069d60 0x7f02c00c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:27.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.001+0000 7f02c6ea5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c006bdb0 con 0x7f02c00c81f0 2023-10-10T20:39:27.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.001+0000 7f02c6ea5700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02c00c8670 con 0x7f02c0069d60 2023-10-10T20:39:27.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.001+0000 7f02c6ea5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c00cb370 con 0x7f02c00aca80 2023-10-10T20:39:27.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.001+0000 7f02c5442700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f02c00c81f0 0x7f02c0068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:32930/0 (socket says 172.21.15.89:32930) 2023-10-10T20:39:27.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.001+0000 7f02c5442700 1 -- 172.21.15.89:0/3343638913 learned_addr learned my addr 172.21.15.89:0/3343638913 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:27.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.001+0000 7f02c4c41700 1 --2- 172.21.15.89:0/3343638913 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02c00aca80 0x7f02c006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:27.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.001+0000 7f02b7fff700 1 --2- 172.21.15.89:0/3343638913 >> v2:172.21.15.111:3300/0 conn(0x7f02c0069d60 0x7f02c00c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:27.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.002+0000 7f02b77fe700 1 -- 172.21.15.89:0/3343638913 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280149724 0 0) 0x7f02c006bdb0 con 0x7f02c00c81f0 2023-10-10T20:39:27.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.002+0000 7f02b77fe700 1 -- 172.21.15.89:0/3343638913 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02a8003570 con 0x7f02c00c81f0 2023-10-10T20:39:27.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.002+0000 7f02c4c41700 1 -- 172.21.15.89:0/3343638913 >> v2:172.21.15.111:3300/0 conn(0x7f02c0069d60 msgr2=0x7f02c00c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.002+0000 7f02c4c41700 1 --2- 172.21.15.89:0/3343638913 >> v2:172.21.15.111:3300/0 conn(0x7f02c0069d60 0x7f02c00c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.002+0000 7f02c4c41700 1 -- 172.21.15.89:0/3343638913 >> v1:172.21.15.89:6789/0 conn(0x7f02c00c81f0 legacy=0x7f02c0068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.002+0000 7f02c4c41700 1 -- 172.21.15.89:0/3343638913 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c0159f60 con 0x7f02c00aca80 2023-10-10T20:39:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.002+0000 7f02c4c41700 1 --2- 172.21.15.89:0/3343638913 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02c00aca80 0x7f02c006aba0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009eb0 tx=0x7f02bc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=449c4583ccbfaf7c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.002+0000 7f02b77fe700 1 -- 172.21.15.89:0/3343638913 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc00e070 con 0x7f02c00aca80 2023-10-10T20:39:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.002+0000 7f02b77fe700 1 -- 172.21.15.89:0/3343638913 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02bc00bb80 con 0x7f02c00aca80 2023-10-10T20:39:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.002+0000 7f02b77fe700 1 -- 172.21.15.89:0/3343638913 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc012430 con 0x7f02c00aca80 2023-10-10T20:39:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.003+0000 7f02c6ea5700 1 -- 172.21.15.89:0/3343638913 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02c00aca80 msgr2=0x7f02c006aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.003+0000 7f02c6ea5700 1 --2- 172.21.15.89:0/3343638913 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02c00aca80 0x7f02c006aba0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009eb0 tx=0x7f02bc009780 comp rx=0 tx=0).stop 2023-10-10T20:39:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.003+0000 7f02c6ea5700 1 -- 172.21.15.89:0/3343638913 shutdown_connections 2023-10-10T20:39:27.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.003+0000 7f02c6ea5700 1 --2- 172.21.15.89:0/3343638913 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02c00aca80 0x7f02c006aba0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:27.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.003+0000 7f02c6ea5700 1 --2- 172.21.15.89:0/3343638913 >> v2:172.21.15.111:3300/0 conn(0x7f02c0069d60 0x7f02c00c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:27.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.003+0000 7f02c6ea5700 1 -- 172.21.15.89:0/3343638913 >> 172.21.15.89:0/3343638913 conn(0x7f02c00cd670 msgr2=0x7f02c00cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:27.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.003+0000 7f02c6ea5700 1 -- 172.21.15.89:0/3343638913 shutdown_connections 2023-10-10T20:39:27.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.003+0000 7f02c6ea5700 1 -- 172.21.15.89:0/3343638913 wait complete. 2023-10-10T20:39:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.004+0000 7f02c6ea5700 1 Processor -- start 2023-10-10T20:39:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.004+0000 7f02c6ea5700 1 -- start start 2023-10-10T20:39:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02c6ea5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02c0069d60 0x7f02c00c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02c6ea5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02c00aca80 0x7f02c00ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02c6ea5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02c00c81f0 0x7f02c00ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:27.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02c6ea5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c00c6c50 con 0x7f02c00aca80 2023-10-10T20:39:27.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02c6ea5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c00c5670 con 0x7f02c00c81f0 2023-10-10T20:39:27.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02c6ea5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c00c5950 con 0x7f02c0069d60 2023-10-10T20:39:27.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02b7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02c00aca80 0x7f02c00ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:27.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02b7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02c00aca80 0x7f02c00ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43500/0 (socket says 172.21.15.89:43500) 2023-10-10T20:39:27.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02b7fff700 1 -- 172.21.15.89:0/2397392978 learned_addr learned my addr 172.21.15.89:0/2397392978 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:27.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02c4c41700 1 --2- 172.21.15.89:0/2397392978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02c0069d60 0x7f02c00c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:27.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02c5442700 1 --2- 172.21.15.89:0/2397392978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02c00c81f0 0x7f02c00ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:27.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02b7fff700 1 -- 172.21.15.89:0/2397392978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02c0069d60 msgr2=0x7f02c00c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:27.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02b7fff700 1 --2- 172.21.15.89:0/2397392978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02c0069d60 0x7f02c00c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:27.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02b7fff700 1 -- 172.21.15.89:0/2397392978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02c00c81f0 msgr2=0x7f02c00ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:27.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02b7fff700 1 --2- 172.21.15.89:0/2397392978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02c00c81f0 0x7f02c00ad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:27.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02b7fff700 1 -- 172.21.15.89:0/2397392978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc009a20 con 0x7f02c00aca80 2023-10-10T20:39:27.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02b7fff700 1 --2- 172.21.15.89:0/2397392978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02c00aca80 0x7f02c00ccaf0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f02b000ec50 tx=0x7f02b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:27.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02b4ff9700 1 -- 172.21.15.89:0/2397392978 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b000c850 con 0x7f02c00aca80 2023-10-10T20:39:27.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02b4ff9700 1 -- 172.21.15.89:0/2397392978 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b000c9b0 con 0x7f02c00aca80 2023-10-10T20:39:27.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02c6ea5700 1 -- 172.21.15.89:0/2397392978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c00c5c30 con 0x7f02c00aca80 2023-10-10T20:39:27.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02b4ff9700 1 -- 172.21.15.89:0/2397392978 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b000cd70 con 0x7f02c00aca80 2023-10-10T20:39:27.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.005+0000 7f02c6ea5700 1 -- 172.21.15.89:0/2397392978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c00c61f0 con 0x7f02c00aca80 2023-10-10T20:39:27.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.006+0000 7f02c6ea5700 1 -- 172.21.15.89:0/2397392978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c004c3c0 con 0x7f02c00aca80 2023-10-10T20:39:27.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.007+0000 7f02b4ff9700 1 -- 172.21.15.89:0/2397392978 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f02b0013680 con 0x7f02c00aca80 2023-10-10T20:39:27.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.007+0000 7f02b4ff9700 1 --2- 172.21.15.89:0/2397392978 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f02ac067170 0x7f02ac069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:27.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.007+0000 7f02c4c41700 1 --2- 172.21.15.89:0/2397392978 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f02ac067170 0x7f02ac069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:27.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.007+0000 7f02b4ff9700 1 -- 172.21.15.89:0/2397392978 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f02b0084aa0 con 0x7f02c00aca80 2023-10-10T20:39:27.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.008+0000 7f02c4c41700 1 --2- 172.21.15.89:0/2397392978 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f02ac067170 0x7f02ac069620 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0097c0 tx=0x7f02bc019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:27.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.012+0000 7f02b4ff9700 1 -- 172.21.15.89:0/2397392978 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02b00519f0 con 0x7f02c00aca80 2023-10-10T20:39:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.238+0000 7f02c6ea5700 1 -- 172.21.15.89:0/2397392978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f02c006b510 con 0x7f02c00aca80 2023-10-10T20:39:27.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.867+0000 7f02b4ff9700 1 -- 172.21.15.89:0/2397392978 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) v1 ==== 104+0+0 (secure 0 0 0) 0x7f02b0055010 con 0x7f02c00aca80 2023-10-10T20:39:27.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.870+0000 7f02c6ea5700 1 -- 172.21.15.89:0/2397392978 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f02ac067170 msgr2=0x7f02ac069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:27.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.870+0000 7f02c6ea5700 1 --2- 172.21.15.89:0/2397392978 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f02ac067170 0x7f02ac069620 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0097c0 tx=0x7f02bc019000 comp rx=0 tx=0).stop 2023-10-10T20:39:27.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.870+0000 7f02c6ea5700 1 -- 172.21.15.89:0/2397392978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02c00aca80 msgr2=0x7f02c00ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:27.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.870+0000 7f02c6ea5700 1 --2- 172.21.15.89:0/2397392978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02c00aca80 0x7f02c00ccaf0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f02b000ec50 tx=0x7f02b00091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:27.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.870+0000 7f02c6ea5700 1 -- 172.21.15.89:0/2397392978 shutdown_connections 2023-10-10T20:39:27.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.870+0000 7f02c6ea5700 1 --2- 172.21.15.89:0/2397392978 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f02ac067170 0x7f02ac069620 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:27.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.870+0000 7f02c6ea5700 1 --2- 172.21.15.89:0/2397392978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02c00c81f0 0x7f02c00ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:27.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.870+0000 7f02c6ea5700 1 --2- 172.21.15.89:0/2397392978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02c0069d60 0x7f02c00c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:27.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.870+0000 7f02c6ea5700 1 --2- 172.21.15.89:0/2397392978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02c00aca80 0x7f02c00ccaf0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:27.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.870+0000 7f02c6ea5700 1 -- 172.21.15.89:0/2397392978 >> 172.21.15.89:0/2397392978 conn(0x7f02c00cd670 msgr2=0x7f02c006aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:27.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.870+0000 7f02c6ea5700 1 -- 172.21.15.89:0/2397392978 shutdown_connections 2023-10-10T20:39:27.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:27.870+0000 7f02c6ea5700 1 -- 172.21.15.89:0/2397392978 wait complete. 2023-10-10T20:39:27.885 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:39:28.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.045+0000 7f6837fff700 1 Processor -- start 2023-10-10T20:39:28.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.045+0000 7f6837fff700 1 -- start start 2023-10-10T20:39:28.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.045+0000 7f6837fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f68380c94c0 0x7f68380cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:28.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.045+0000 7f6837fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6838157910 0x7f68380cb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:28.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.045+0000 7f6837fff700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683815b560 con 0x7f6838157540 2023-10-10T20:39:28.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.045+0000 7f6837fff700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68380cbbe0 con 0x7f6838157910 2023-10-10T20:39:28.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.045+0000 7f6837fff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683815b840 con 0x7f68380c94c0 2023-10-10T20:39:28.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.046+0000 7f68377fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6838157910 0x7f68380cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:28.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.046+0000 7f6836ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6838157540 0x7f68380c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:32942/0 (socket says 172.21.15.89:32942) 2023-10-10T20:39:28.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.046+0000 7f68377fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6838157910 0x7f68380cb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43504/0 (socket says 172.21.15.89:43504) 2023-10-10T20:39:28.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.046+0000 7f6836ffd700 1 -- 172.21.15.89:0/2863482350 learned_addr learned my addr 172.21.15.89:0/2863482350 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:28.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.046+0000 7f682e7fc700 1 --2- 172.21.15.89:0/2863482350 >> v2:172.21.15.111:3300/0 conn(0x7f68380c94c0 0x7f68380cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:28.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.046+0000 7f68377fe700 1 -- 172.21.15.89:0/2863482350 >> v2:172.21.15.111:3300/0 conn(0x7f68380c94c0 msgr2=0x7f68380cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:28.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.046+0000 7f68377fe700 1 --2- 172.21.15.89:0/2863482350 >> v2:172.21.15.111:3300/0 conn(0x7f68380c94c0 0x7f68380cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:28.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.046+0000 7f68377fe700 1 -- 172.21.15.89:0/2863482350 >> v1:172.21.15.89:6789/0 conn(0x7f6838157540 legacy=0x7f68380c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:28.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.046+0000 7f68377fe700 1 -- 172.21.15.89:0/2863482350 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683815bb20 con 0x7f6838157910 2023-10-10T20:39:28.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.046+0000 7f68367fc700 1 -- 172.21.15.89:0/2863482350 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263150727 0 0) 0x7f683815b560 con 0x7f6838157540 2023-10-10T20:39:28.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.046+0000 7f682e7fc700 1 --2- 172.21.15.89:0/2863482350 >> v2:172.21.15.111:3300/0 conn(0x7f68380c94c0 0x7f68380cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:28.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.046+0000 7f68377fe700 1 --2- 172.21.15.89:0/2863482350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6838157910 0x7f68380cb310 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f683806c680 tx=0x7f6830009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7571a69f1e6dca server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:28.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.046+0000 7f68367fc700 1 -- 172.21.15.89:0/2863482350 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683000e070 con 0x7f6838157910 2023-10-10T20:39:28.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.046+0000 7f68367fc700 1 -- 172.21.15.89:0/2863482350 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683000bc30 con 0x7f6838157910 2023-10-10T20:39:28.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.046+0000 7f68367fc700 1 -- 172.21.15.89:0/2863482350 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6830012430 con 0x7f6838157910 2023-10-10T20:39:28.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.047+0000 7f6837fff700 1 -- 172.21.15.89:0/2863482350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6838157910 msgr2=0x7f68380cb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:28.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.047+0000 7f6837fff700 1 --2- 172.21.15.89:0/2863482350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6838157910 0x7f68380cb310 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f683806c680 tx=0x7f6830009780 comp rx=0 tx=0).stop 2023-10-10T20:39:28.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.047+0000 7f6837fff700 1 -- 172.21.15.89:0/2863482350 shutdown_connections 2023-10-10T20:39:28.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.047+0000 7f6837fff700 1 --2- 172.21.15.89:0/2863482350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6838157910 0x7f68380cb310 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:28.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.047+0000 7f6837fff700 1 --2- 172.21.15.89:0/2863482350 >> v2:172.21.15.111:3300/0 conn(0x7f68380c94c0 0x7f68380cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:28.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.047+0000 7f6837fff700 1 -- 172.21.15.89:0/2863482350 >> 172.21.15.89:0/2863482350 conn(0x7f683806baf0 msgr2=0x7f683806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:28.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.047+0000 7f6837fff700 1 -- 172.21.15.89:0/2863482350 shutdown_connections 2023-10-10T20:39:28.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.047+0000 7f6837fff700 1 -- 172.21.15.89:0/2863482350 wait complete. 2023-10-10T20:39:28.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.048+0000 7f6837fff700 1 Processor -- start 2023-10-10T20:39:28.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.048+0000 7f6837fff700 1 -- start start 2023-10-10T20:39:28.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f6837fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f68380c94c0 0x7f68380b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:28.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f6837fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6838157540 0x7f683814a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:28.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f6837fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6838157910 0x7f68380c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:28.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f6837fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6838149c20 con 0x7f6838157910 2023-10-10T20:39:28.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f6837fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683814a6b0 con 0x7f6838157540 2023-10-10T20:39:28.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f6837fff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68380cd2d0 con 0x7f68380c94c0 2023-10-10T20:39:28.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f68377fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6838157910 0x7f68380c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:28.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f68377fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6838157910 0x7f68380c7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43518/0 (socket says 172.21.15.89:43518) 2023-10-10T20:39:28.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f68377fe700 1 -- 172.21.15.89:0/1297675070 learned_addr learned my addr 172.21.15.89:0/1297675070 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:28.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f682e7fc700 1 --2- 172.21.15.89:0/1297675070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6838157540 0x7f683814a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:28.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f6836ffd700 1 --2- 172.21.15.89:0/1297675070 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f68380c94c0 0x7f68380b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:28.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f68377fe700 1 -- 172.21.15.89:0/1297675070 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f68380c94c0 msgr2=0x7f68380b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:28.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f68377fe700 1 --2- 172.21.15.89:0/1297675070 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f68380c94c0 0x7f68380b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:28.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f68377fe700 1 -- 172.21.15.89:0/1297675070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6838157540 msgr2=0x7f683814a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:28.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f68377fe700 1 --2- 172.21.15.89:0/1297675070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6838157540 0x7f683814a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:28.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f68377fe700 1 -- 172.21.15.89:0/1297675070 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6830009a20 con 0x7f6838157910 2023-10-10T20:39:28.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f6836ffd700 1 --2- 172.21.15.89:0/1297675070 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f68380c94c0 0x7f68380b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:28.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f68377fe700 1 --2- 172.21.15.89:0/1297675070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6838157910 0x7f68380c7920 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f683806ce10 tx=0x7f6830019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:28.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f682f7fe700 1 -- 172.21.15.89:0/1297675070 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683001d630 con 0x7f6838157910 2023-10-10T20:39:28.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f682f7fe700 1 -- 172.21.15.89:0/1297675070 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683001d790 con 0x7f6838157910 2023-10-10T20:39:28.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f6837fff700 1 -- 172.21.15.89:0/1297675070 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68380b1eb0 con 0x7f6838157910 2023-10-10T20:39:28.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.049+0000 7f682f7fe700 1 -- 172.21.15.89:0/1297675070 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683001db50 con 0x7f6838157910 2023-10-10T20:39:28.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.050+0000 7f6837fff700 1 -- 172.21.15.89:0/1297675070 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6838149db0 con 0x7f6838157910 2023-10-10T20:39:28.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.050+0000 7f6837fff700 1 -- 172.21.15.89:0/1297675070 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68380cd440 con 0x7f6838157910 2023-10-10T20:39:28.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.051+0000 7f682f7fe700 1 -- 172.21.15.89:0/1297675070 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f6830024070 con 0x7f6838157910 2023-10-10T20:39:28.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.051+0000 7f682f7fe700 1 --2- 172.21.15.89:0/1297675070 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6818067170 0x7f6818069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:28.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.051+0000 7f682f7fe700 1 -- 172.21.15.89:0/1297675070 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f6830087220 con 0x7f6838157910 2023-10-10T20:39:28.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.051+0000 7f6836ffd700 1 --2- 172.21.15.89:0/1297675070 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6818067170 0x7f6818069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:28.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.051+0000 7f6836ffd700 1 --2- 172.21.15.89:0/1297675070 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6818067170 0x7f6818069620 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f68380cf640 tx=0x7f6828009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:28.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.055+0000 7f682f7fe700 1 -- 172.21.15.89:0/1297675070 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6830054170 con 0x7f6838157910 2023-10-10T20:39:28.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:28.240+0000 7f6837fff700 1 -- 172.21.15.89:0/1297675070 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f683804c3c0 con 0x7f6838157910 2023-10-10T20:39:29.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.199+0000 7f682f7fe700 1 -- 172.21.15.89:0/1297675070 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v66) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6830009880 con 0x7f6838157910 2023-10-10T20:39:29.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.202+0000 7f6837fff700 1 -- 172.21.15.89:0/1297675070 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6818067170 msgr2=0x7f6818069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:29.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.202+0000 7f6837fff700 1 --2- 172.21.15.89:0/1297675070 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6818067170 0x7f6818069620 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f68380cf640 tx=0x7f6828009040 comp rx=0 tx=0).stop 2023-10-10T20:39:29.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.202+0000 7f6837fff700 1 -- 172.21.15.89:0/1297675070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6838157910 msgr2=0x7f68380c7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:29.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.202+0000 7f6837fff700 1 --2- 172.21.15.89:0/1297675070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6838157910 0x7f68380c7920 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f683806ce10 tx=0x7f6830019000 comp rx=0 tx=0).stop 2023-10-10T20:39:29.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.202+0000 7f6837fff700 1 -- 172.21.15.89:0/1297675070 shutdown_connections 2023-10-10T20:39:29.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.202+0000 7f6837fff700 1 --2- 172.21.15.89:0/1297675070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6838157910 0x7f68380c7920 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:29.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.202+0000 7f6837fff700 1 --2- 172.21.15.89:0/1297675070 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f68380c94c0 0x7f68380b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:29.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.202+0000 7f6837fff700 1 --2- 172.21.15.89:0/1297675070 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6818067170 0x7f6818069620 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:29.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.202+0000 7f6837fff700 1 --2- 172.21.15.89:0/1297675070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6838157540 0x7f683814a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:29.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.202+0000 7f6837fff700 1 -- 172.21.15.89:0/1297675070 >> 172.21.15.89:0/1297675070 conn(0x7f683806baf0 msgr2=0x7f6838159260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:29.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.202+0000 7f6837fff700 1 -- 172.21.15.89:0/1297675070 shutdown_connections 2023-10-10T20:39:29.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.202+0000 7f6837fff700 1 -- 172.21.15.89:0/1297675070 wait complete. 2023-10-10T20:39:29.207 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:39:29.220 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:39:29.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.341+0000 7ff0c49d5700 1 Processor -- start 2023-10-10T20:39:29.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.341+0000 7ff0c49d5700 1 -- start start 2023-10-10T20:39:29.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.342+0000 7ff0c49d5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff0c00ced30 0x7ff0c0162910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:29.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.342+0000 7ff0c49d5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c015c320 con 0x7ff0c0155360 2023-10-10T20:39:29.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.342+0000 7ff0c49d5700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c015b710 con 0x7ff0c00b42c0 2023-10-10T20:39:29.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.342+0000 7ff0c49d5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c015b9f0 con 0x7ff0c00ced30 2023-10-10T20:39:29.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.342+0000 7ff0be59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff0c0155360 0x7ff0c00b3bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38894/0 (socket says 172.21.15.89:38894) 2023-10-10T20:39:29.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.342+0000 7ff0be59c700 1 -- 172.21.15.89:0/3901336464 learned_addr learned my addr 172.21.15.89:0/3901336464 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:29.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.342+0000 7ff0bed9d700 1 --2- 172.21.15.89:0/3901336464 >> v2:172.21.15.89:3300/0 conn(0x7ff0c00ced30 0x7ff0c0162910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:29.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.343+0000 7ff0bed9d700 1 -- 172.21.15.89:0/3901336464 >> v1:172.21.15.111:6789/0 conn(0x7ff0c00b42c0 legacy=0x7ff0c00ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:29.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.343+0000 7ff0bed9d700 1 -- 172.21.15.89:0/3901336464 >> v1:172.21.15.89:6789/0 conn(0x7ff0c0155360 legacy=0x7ff0c00b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:29.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.343+0000 7ff0bed9d700 1 -- 172.21.15.89:0/3901336464 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c015bcd0 con 0x7ff0c00ced30 2023-10-10T20:39:29.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.343+0000 7ff0bed9d700 1 --2- 172.21.15.89:0/3901336464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00ced30 0x7ff0c0162910 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8009e60 tx=0x7ff0b8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4aa3de856e6a537 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:29.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.343+0000 7ff0bdd9b700 1 -- 172.21.15.89:0/3901336464 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b800e070 con 0x7ff0c00ced30 2023-10-10T20:39:29.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.343+0000 7ff0bdd9b700 1 -- 172.21.15.89:0/3901336464 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b800bc30 con 0x7ff0c00ced30 2023-10-10T20:39:29.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.343+0000 7ff0bdd9b700 1 -- 172.21.15.89:0/3901336464 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8012430 con 0x7ff0c00ced30 2023-10-10T20:39:29.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.343+0000 7ff0c49d5700 1 -- 172.21.15.89:0/3901336464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00ced30 msgr2=0x7ff0c0162910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:29.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.343+0000 7ff0c49d5700 1 --2- 172.21.15.89:0/3901336464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00ced30 0x7ff0c0162910 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8009e60 tx=0x7ff0b8009780 comp rx=0 tx=0).stop 2023-10-10T20:39:29.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.343+0000 7ff0c49d5700 1 -- 172.21.15.89:0/3901336464 shutdown_connections 2023-10-10T20:39:29.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.343+0000 7ff0c49d5700 1 --2- 172.21.15.89:0/3901336464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00ced30 0x7ff0c0162910 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:29.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.343+0000 7ff0c49d5700 1 -- 172.21.15.89:0/3901336464 >> 172.21.15.89:0/3901336464 conn(0x7ff0c006baf0 msgr2=0x7ff0c006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:29.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.343+0000 7ff0c49d5700 1 -- 172.21.15.89:0/3901336464 shutdown_connections 2023-10-10T20:39:29.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.343+0000 7ff0c49d5700 1 -- 172.21.15.89:0/3901336464 wait complete. 2023-10-10T20:39:29.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.344+0000 7ff0c49d5700 1 Processor -- start 2023-10-10T20:39:29.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.345+0000 7ff0c49d5700 1 -- start start 2023-10-10T20:39:29.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.345+0000 7ff0c49d5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00b42c0 0x7ff0c01472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:29.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.345+0000 7ff0c49d5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0c00ced30 0x7ff0c0069e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:29.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.345+0000 7ff0c49d5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff0c0155360 0x7ff0c00c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:29.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.345+0000 7ff0c49d5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c006b4f0 con 0x7ff0c00b42c0 2023-10-10T20:39:29.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.345+0000 7ff0c49d5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c006a970 con 0x7ff0c00ced30 2023-10-10T20:39:29.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.345+0000 7ff0c49d5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c006ac50 con 0x7ff0c0155360 2023-10-10T20:39:29.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.345+0000 7ff0be59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00b42c0 0x7ff0c01472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:29.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.345+0000 7ff0be59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00b42c0 0x7ff0c01472e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50762/0 (socket says 172.21.15.89:50762) 2023-10-10T20:39:29.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.345+0000 7ff0be59c700 1 -- 172.21.15.89:0/3350703938 learned_addr learned my addr 172.21.15.89:0/3350703938 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:29.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.346+0000 7ff0b7fff700 1 --2- 172.21.15.89:0/3350703938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0c00ced30 0x7ff0c0069e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:29.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.346+0000 7ff0bed9d700 1 --2- 172.21.15.89:0/3350703938 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff0c0155360 0x7ff0c00c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:29.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.346+0000 7ff0be59c700 1 -- 172.21.15.89:0/3350703938 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff0c0155360 msgr2=0x7ff0c00c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:29.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.346+0000 7ff0be59c700 1 --2- 172.21.15.89:0/3350703938 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff0c0155360 0x7ff0c00c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:29.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.346+0000 7ff0be59c700 1 -- 172.21.15.89:0/3350703938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0c00ced30 msgr2=0x7ff0c0069e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:29.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.346+0000 7ff0be59c700 1 --2- 172.21.15.89:0/3350703938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0c00ced30 0x7ff0c0069e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:29.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.346+0000 7ff0be59c700 1 -- 172.21.15.89:0/3350703938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b8009a20 con 0x7ff0c00b42c0 2023-10-10T20:39:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.346+0000 7ff0b7fff700 1 --2- 172.21.15.89:0/3350703938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0c00ced30 0x7ff0c0069e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.346+0000 7ff0be59c700 1 --2- 172.21.15.89:0/3350703938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00b42c0 0x7ff0c01472e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000c560 tx=0x7ff0b0001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.346+0000 7ff0b67fc700 1 -- 172.21.15.89:0/3350703938 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0005b30 con 0x7ff0c00b42c0 2023-10-10T20:39:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.346+0000 7ff0b67fc700 1 -- 172.21.15.89:0/3350703938 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b0005cb0 con 0x7ff0c00b42c0 2023-10-10T20:39:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.346+0000 7ff0c49d5700 1 -- 172.21.15.89:0/3350703938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c006af30 con 0x7ff0c00b42c0 2023-10-10T20:39:29.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.346+0000 7ff0b67fc700 1 -- 172.21.15.89:0/3350703938 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0007320 con 0x7ff0c00b42c0 2023-10-10T20:39:29.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.346+0000 7ff0c49d5700 1 -- 172.21.15.89:0/3350703938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c00c79e0 con 0x7ff0c00b42c0 2023-10-10T20:39:29.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.347+0000 7ff0c49d5700 1 -- 172.21.15.89:0/3350703938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c00718b0 con 0x7ff0c00b42c0 2023-10-10T20:39:29.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.347+0000 7ff0b67fc700 1 -- 172.21.15.89:0/3350703938 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7ff0b001b3b0 con 0x7ff0c00b42c0 2023-10-10T20:39:29.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.348+0000 7ff0b67fc700 1 --2- 172.21.15.89:0/3350703938 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff0a0067170 0x7ff0a0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:29.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.348+0000 7ff0b67fc700 1 -- 172.21.15.89:0/3350703938 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5411+0+0 (secure 0 0 0) 0x7ff0b0083bf0 con 0x7ff0c00b42c0 2023-10-10T20:39:29.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.348+0000 7ff0b7fff700 1 --2- 172.21.15.89:0/3350703938 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff0a0067170 0x7ff0a0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:29.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.348+0000 7ff0b7fff700 1 --2- 172.21.15.89:0/3350703938 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff0a0067170 0x7ff0a0069620 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff0c01542c0 tx=0x7ff0a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:29.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.352+0000 7ff0b67fc700 1 -- 172.21.15.89:0/3350703938 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff0b0050cf0 con 0x7ff0c00b42c0 2023-10-10T20:39:29.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:29.541+0000 7ff0c49d5700 1 -- 172.21.15.89:0/3350703938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff0c015bfb0 con 0x7ff0c00b42c0 2023-10-10T20:39:30.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.202+0000 7ff0b67fc700 1 -- 172.21.15.89:0/3350703938 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v67) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff0b0017020 con 0x7ff0c00b42c0 2023-10-10T20:39:30.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.210+0000 7ff0c49d5700 1 -- 172.21.15.89:0/3350703938 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff0a0067170 msgr2=0x7ff0a0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:30.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.210+0000 7ff0c49d5700 1 --2- 172.21.15.89:0/3350703938 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff0a0067170 0x7ff0a0069620 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff0c01542c0 tx=0x7ff0a8009040 comp rx=0 tx=0).stop 2023-10-10T20:39:30.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.210+0000 7ff0c49d5700 1 -- 172.21.15.89:0/3350703938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00b42c0 msgr2=0x7ff0c01472e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:30.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.210+0000 7ff0c49d5700 1 --2- 172.21.15.89:0/3350703938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00b42c0 0x7ff0c01472e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000c560 tx=0x7ff0b0001020 comp rx=0 tx=0).stop 2023-10-10T20:39:30.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.212+0000 7ff0c49d5700 1 -- 172.21.15.89:0/3350703938 shutdown_connections 2023-10-10T20:39:30.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.212+0000 7ff0c49d5700 1 --2- 172.21.15.89:0/3350703938 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff0c0155360 0x7ff0c00c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:30.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.212+0000 7ff0c49d5700 1 --2- 172.21.15.89:0/3350703938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00b42c0 0x7ff0c01472e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:30.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.212+0000 7ff0c49d5700 1 --2- 172.21.15.89:0/3350703938 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff0a0067170 0x7ff0a0069620 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:30.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.212+0000 7ff0c49d5700 1 --2- 172.21.15.89:0/3350703938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0c00ced30 0x7ff0c0069e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:30.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.212+0000 7ff0c49d5700 1 -- 172.21.15.89:0/3350703938 >> 172.21.15.89:0/3350703938 conn(0x7ff0c006baf0 msgr2=0x7ff0c0154490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:30.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.212+0000 7ff0c49d5700 1 -- 172.21.15.89:0/3350703938 shutdown_connections 2023-10-10T20:39:30.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.212+0000 7ff0c49d5700 1 -- 172.21.15.89:0/3350703938 wait complete. 2023-10-10T20:39:30.216 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' removed 2023-10-10T20:39:30.230 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:39:30.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.350+0000 7fdbc2184700 1 Processor -- start 2023-10-10T20:39:30.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.350+0000 7fdbc2184700 1 -- start start 2023-10-10T20:39:30.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.351+0000 7fdbc2184700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdbbc0b16d0 0x7fdbbc0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:30.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.351+0000 7fdbc2184700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdbbc0b33b0 0x7fdbbc0b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:30.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.351+0000 7fdbc2184700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbbc0b2080 con 0x7fdbbc0b4e70 2023-10-10T20:39:30.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.351+0000 7fdbc2184700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbbc0cfb00 con 0x7fdbbc0b33b0 2023-10-10T20:39:30.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.351+0000 7fdbc2184700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbbc0b5c40 con 0x7fdbbc0b16d0 2023-10-10T20:39:30.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.351+0000 7fdbc1983700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdbbc0b33b0 0x7fdbbc0b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:30.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.351+0000 7fdbc1983700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdbbc0b33b0 0x7fdbbc0b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50768/0 (socket says 172.21.15.89:50768) 2023-10-10T20:39:30.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.351+0000 7fdbc1983700 1 -- 172.21.15.89:0/3454607367 learned_addr learned my addr 172.21.15.89:0/3454607367 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:30.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.351+0000 7fdbc1983700 1 -- 172.21.15.89:0/3454607367 >> v2:172.21.15.111:3300/0 conn(0x7fdbbc0b16d0 msgr2=0x7fdbbc0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:30.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.351+0000 7fdbc0981700 1 --2- 172.21.15.89:0/3454607367 >> v2:172.21.15.111:3300/0 conn(0x7fdbbc0b16d0 0x7fdbbc0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:30.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.351+0000 7fdbc1983700 1 --2- 172.21.15.89:0/3454607367 >> v2:172.21.15.111:3300/0 conn(0x7fdbbc0b16d0 0x7fdbbc0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:30.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.351+0000 7fdbc1983700 1 -- 172.21.15.89:0/3454607367 >> v1:172.21.15.111:6789/0 conn(0x7fdbbc0b4e70 legacy=0x7fdbbc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:30.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.351+0000 7fdbc1983700 1 -- 172.21.15.89:0/3454607367 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbbc0b5f20 con 0x7fdbbc0b33b0 2023-10-10T20:39:30.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.351+0000 7fdbc0981700 1 --2- 172.21.15.89:0/3454607367 >> v2:172.21.15.111:3300/0 conn(0x7fdbbc0b16d0 0x7fdbbc0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:30.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.352+0000 7fdbc1983700 1 --2- 172.21.15.89:0/3454607367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbbc0b33b0 0x7fdbbc0b37d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc089830 tx=0x7fdbb4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fb8a513e8075de server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:30.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.352+0000 7fdbb3fff700 1 -- 172.21.15.89:0/3454607367 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb400e070 con 0x7fdbbc0b33b0 2023-10-10T20:39:30.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.352+0000 7fdbb3fff700 1 -- 172.21.15.89:0/3454607367 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb400ba10 con 0x7fdbbc0b33b0 2023-10-10T20:39:30.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.352+0000 7fdbb3fff700 1 -- 172.21.15.89:0/3454607367 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb40092e0 con 0x7fdbbc0b33b0 2023-10-10T20:39:30.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.352+0000 7fdbc2184700 1 -- 172.21.15.89:0/3454607367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbbc0b33b0 msgr2=0x7fdbbc0b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:30.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.352+0000 7fdbc2184700 1 --2- 172.21.15.89:0/3454607367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbbc0b33b0 0x7fdbbc0b37d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc089830 tx=0x7fdbb4009840 comp rx=0 tx=0).stop 2023-10-10T20:39:30.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.352+0000 7fdbc2184700 1 -- 172.21.15.89:0/3454607367 shutdown_connections 2023-10-10T20:39:30.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.352+0000 7fdbc2184700 1 --2- 172.21.15.89:0/3454607367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbbc0b33b0 0x7fdbbc0b37d0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:30.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.352+0000 7fdbc2184700 1 --2- 172.21.15.89:0/3454607367 >> v2:172.21.15.111:3300/0 conn(0x7fdbbc0b16d0 0x7fdbbc0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:30.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.352+0000 7fdbc2184700 1 -- 172.21.15.89:0/3454607367 >> 172.21.15.89:0/3454607367 conn(0x7fdbbc0d10c0 msgr2=0x7fdbbc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:30.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.352+0000 7fdbc2184700 1 -- 172.21.15.89:0/3454607367 shutdown_connections 2023-10-10T20:39:30.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.352+0000 7fdbc2184700 1 -- 172.21.15.89:0/3454607367 wait complete. 2023-10-10T20:39:30.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.354+0000 7fdbc2184700 1 Processor -- start 2023-10-10T20:39:30.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.354+0000 7fdbc2184700 1 -- start start 2023-10-10T20:39:30.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.354+0000 7fdbc2184700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbbc0b16d0 0x7fdbbc0a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:30.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.354+0000 7fdbc2184700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbbc0b33b0 0x7fdbbc0cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:30.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.354+0000 7fdbc2184700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdbbc0b4e70 0x7fdbbc16d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:30.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.354+0000 7fdbc2184700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc0c8020 con 0x7fdbbc0b33b0 2023-10-10T20:39:30.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.354+0000 7fdbc2184700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc0a75e0 con 0x7fdbbc0b16d0 2023-10-10T20:39:30.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.354+0000 7fdbc2184700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc0c9730 con 0x7fdbbc0b4e70 2023-10-10T20:39:30.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.355+0000 7fdbc0981700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbbc0b33b0 0x7fdbbc0cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:30.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.355+0000 7fdbc0981700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbbc0b33b0 0x7fdbbc0cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50774/0 (socket says 172.21.15.89:50774) 2023-10-10T20:39:30.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.355+0000 7fdbc0981700 1 -- 172.21.15.89:0/714155205 learned_addr learned my addr 172.21.15.89:0/714155205 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:30.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.355+0000 7fdbc1182700 1 --2- 172.21.15.89:0/714155205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbbc0b16d0 0x7fdbbc0a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:30.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.355+0000 7fdbc1983700 1 --2- 172.21.15.89:0/714155205 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdbbc0b4e70 0x7fdbbc16d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:30.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.355+0000 7fdbc0981700 1 -- 172.21.15.89:0/714155205 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdbbc0b4e70 msgr2=0x7fdbbc16d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:30.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.355+0000 7fdbc0981700 1 --2- 172.21.15.89:0/714155205 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdbbc0b4e70 0x7fdbbc16d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:30.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.355+0000 7fdbc0981700 1 -- 172.21.15.89:0/714155205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbbc0b16d0 msgr2=0x7fdbbc0a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:30.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.355+0000 7fdbc0981700 1 --2- 172.21.15.89:0/714155205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbbc0b16d0 0x7fdbbc0a6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:30.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.355+0000 7fdbc0981700 1 -- 172.21.15.89:0/714155205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb4009ae0 con 0x7fdbbc0b33b0 2023-10-10T20:39:30.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.355+0000 7fdbc0981700 1 --2- 172.21.15.89:0/714155205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbbc0b33b0 0x7fdbbc0cb0d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00c850 tx=0x7fdbac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:30.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.355+0000 7fdbb17fa700 1 -- 172.21.15.89:0/714155205 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac011600 con 0x7fdbbc0b33b0 2023-10-10T20:39:30.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.355+0000 7fdbb17fa700 1 -- 172.21.15.89:0/714155205 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbac011760 con 0x7fdbbc0b33b0 2023-10-10T20:39:30.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.355+0000 7fdbc2184700 1 -- 172.21.15.89:0/714155205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbbc0c9a10 con 0x7fdbbc0b33b0 2023-10-10T20:39:30.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.355+0000 7fdbb17fa700 1 -- 172.21.15.89:0/714155205 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac011b00 con 0x7fdbbc0b33b0 2023-10-10T20:39:30.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.355+0000 7fdbc2184700 1 -- 172.21.15.89:0/714155205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbbc16dc70 con 0x7fdbbc0b33b0 2023-10-10T20:39:30.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.356+0000 7fdbc2184700 1 -- 172.21.15.89:0/714155205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbbc088990 con 0x7fdbbc0b33b0 2023-10-10T20:39:30.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.356+0000 7fdbb17fa700 1 -- 172.21.15.89:0/714155205 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fdbac0073b0 con 0x7fdbbc0b33b0 2023-10-10T20:39:30.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.357+0000 7fdbb17fa700 1 --2- 172.21.15.89:0/714155205 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdba0067170 0x7fdba0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:30.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.357+0000 7fdbb17fa700 1 -- 172.21.15.89:0/714155205 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fdbac084600 con 0x7fdbbc0b33b0 2023-10-10T20:39:30.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.357+0000 7fdbc1182700 1 --2- 172.21.15.89:0/714155205 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdba0067170 0x7fdba0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:30.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.357+0000 7fdbc1182700 1 --2- 172.21.15.89:0/714155205 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdba0067170 0x7fdba0069620 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8003870 tx=0x7fdbb8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:30.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.361+0000 7fdbb17fa700 1 -- 172.21.15.89:0/714155205 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdbac055260 con 0x7fdbbc0b33b0 2023-10-10T20:39:30.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.547+0000 7fdbc2184700 1 -- 172.21.15.89:0/714155205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fdbbc04bb80 con 0x7fdbbc0b33b0 2023-10-10T20:39:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.550+0000 7fdbb17fa700 1 -- 172.21.15.89:0/714155205 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdbac054df0 con 0x7fdbbc0b33b0 2023-10-10T20:39:30.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.553+0000 7fdbc2184700 1 -- 172.21.15.89:0/714155205 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdba0067170 msgr2=0x7fdba0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:30.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.553+0000 7fdbc2184700 1 --2- 172.21.15.89:0/714155205 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdba0067170 0x7fdba0069620 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8003870 tx=0x7fdbb8009040 comp rx=0 tx=0).stop 2023-10-10T20:39:30.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.553+0000 7fdbc2184700 1 -- 172.21.15.89:0/714155205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbbc0b33b0 msgr2=0x7fdbbc0cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:30.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.553+0000 7fdbc2184700 1 --2- 172.21.15.89:0/714155205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbbc0b33b0 0x7fdbbc0cb0d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00c850 tx=0x7fdbac008040 comp rx=0 tx=0).stop 2023-10-10T20:39:30.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.553+0000 7fdbc2184700 1 -- 172.21.15.89:0/714155205 shutdown_connections 2023-10-10T20:39:30.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.553+0000 7fdbc2184700 1 --2- 172.21.15.89:0/714155205 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdbbc0b4e70 0x7fdbbc16d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:30.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.553+0000 7fdbc2184700 1 --2- 172.21.15.89:0/714155205 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdba0067170 0x7fdba0069620 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:30.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.553+0000 7fdbc2184700 1 --2- 172.21.15.89:0/714155205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbbc0b16d0 0x7fdbbc0a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:30.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.553+0000 7fdbc2184700 1 --2- 172.21.15.89:0/714155205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbbc0b33b0 0x7fdbbc0cb0d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:30.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.553+0000 7fdbc2184700 1 -- 172.21.15.89:0/714155205 >> 172.21.15.89:0/714155205 conn(0x7fdbbc0d10c0 msgr2=0x7fdbbc0ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:30.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.553+0000 7fdbc2184700 1 -- 172.21.15.89:0/714155205 shutdown_connections 2023-10-10T20:39:30.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.553+0000 7fdbc2184700 1 -- 172.21.15.89:0/714155205 wait complete. 2023-10-10T20:39:30.567 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions' 2023-10-10T20:39:30.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.680+0000 7f060e5ce700 1 Processor -- start 2023-10-10T20:39:30.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.680+0000 7f060e5ce700 1 -- start start 2023-10-10T20:39:30.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.681+0000 7f060e5ce700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f060806aa40 0x7f06080cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:30.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.681+0000 7f060e5ce700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06080c91e0 0x7f0608160390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:30.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.681+0000 7f060e5ce700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06080cc210 con 0x7f060806bac0 2023-10-10T20:39:30.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.681+0000 7f060e5ce700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06080cc4f0 con 0x7f060806aa40 2023-10-10T20:39:30.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.681+0000 7f060e5ce700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06080cc7d0 con 0x7f06080c91e0 2023-10-10T20:39:30.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.681+0000 7f060cb6b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06080c91e0 0x7f0608160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:30.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.681+0000 7f06077fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f060806bac0 0x7f060806ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:58618/0 (socket says 172.21.15.89:58618) 2023-10-10T20:39:30.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.681+0000 7f06077fe700 1 -- 172.21.15.89:0/2264548169 learned_addr learned my addr 172.21.15.89:0/2264548169 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:30.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.681+0000 7f0607fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f060806aa40 0x7f06080cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:30.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.681+0000 7f060cb6b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06080c91e0 0x7f0608160390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:37674/0 (socket says 172.21.15.89:37674) 2023-10-10T20:39:30.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.682+0000 7f0606ffd700 1 -- 172.21.15.89:0/2264548169 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436041297 0 0) 0x7f06080cc210 con 0x7f060806bac0 2023-10-10T20:39:30.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.682+0000 7f0606ffd700 1 -- 172.21.15.89:0/2264548169 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05f0003570 con 0x7f060806bac0 2023-10-10T20:39:30.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.682+0000 7f060cb6b700 1 -- 172.21.15.89:0/2264548169 >> v2:172.21.15.111:3300/0 conn(0x7f060806aa40 msgr2=0x7f06080cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:30.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.682+0000 7f060cb6b700 1 --2- 172.21.15.89:0/2264548169 >> v2:172.21.15.111:3300/0 conn(0x7f060806aa40 0x7f06080cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:30.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.682+0000 7f060cb6b700 1 -- 172.21.15.89:0/2264548169 >> v1:172.21.15.111:6789/0 conn(0x7f060806bac0 legacy=0x7f060806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:30.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.682+0000 7f060cb6b700 1 -- 172.21.15.89:0/2264548169 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06080ccab0 con 0x7f06080c91e0 2023-10-10T20:39:30.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.682+0000 7f0607fff700 1 --2- 172.21.15.89:0/2264548169 >> v2:172.21.15.111:3300/0 conn(0x7f060806aa40 0x7f06080cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:39:30.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.682+0000 7f060cb6b700 1 --2- 172.21.15.89:0/2264548169 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06080c91e0 0x7f0608160390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0600009e60 tx=0x7f0600009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3ee0a85b234106e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:30.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.682+0000 7f0606ffd700 1 -- 172.21.15.89:0/2264548169 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060000e070 con 0x7f06080c91e0 2023-10-10T20:39:30.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.682+0000 7f0606ffd700 1 -- 172.21.15.89:0/2264548169 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060000bab0 con 0x7f06080c91e0 2023-10-10T20:39:30.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.682+0000 7f0606ffd700 1 -- 172.21.15.89:0/2264548169 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060000be70 con 0x7f06080c91e0 2023-10-10T20:39:30.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.683+0000 7f060e5ce700 1 -- 172.21.15.89:0/2264548169 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06080c91e0 msgr2=0x7f0608160390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:30.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.683+0000 7f060e5ce700 1 --2- 172.21.15.89:0/2264548169 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06080c91e0 0x7f0608160390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0600009e60 tx=0x7f0600009780 comp rx=0 tx=0).stop 2023-10-10T20:39:30.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.683+0000 7f060e5ce700 1 -- 172.21.15.89:0/2264548169 shutdown_connections 2023-10-10T20:39:30.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.683+0000 7f060e5ce700 1 --2- 172.21.15.89:0/2264548169 >> v2:172.21.15.111:3300/0 conn(0x7f060806aa40 0x7f06080cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:30.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.683+0000 7f060e5ce700 1 --2- 172.21.15.89:0/2264548169 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06080c91e0 0x7f0608160390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:30.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.683+0000 7f060e5ce700 1 -- 172.21.15.89:0/2264548169 >> 172.21.15.89:0/2264548169 conn(0x7f06080cd670 msgr2=0x7f06080cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:30.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.683+0000 7f060e5ce700 1 -- 172.21.15.89:0/2264548169 shutdown_connections 2023-10-10T20:39:30.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.683+0000 7f060e5ce700 1 -- 172.21.15.89:0/2264548169 wait complete. 2023-10-10T20:39:30.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.684+0000 7f060e5ce700 1 Processor -- start 2023-10-10T20:39:30.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.684+0000 7f060e5ce700 1 -- start start 2023-10-10T20:39:30.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.685+0000 7f060e5ce700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f060806aa40 0x7f060813ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:30.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.685+0000 7f060e5ce700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f060806bac0 0x7f060815b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:30.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.685+0000 7f060e5ce700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06080c91e0 0x7f060815b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:30.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.685+0000 7f060e5ce700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060814f650 con 0x7f06080c91e0 2023-10-10T20:39:30.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.685+0000 7f060e5ce700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060814ea40 con 0x7f060806bac0 2023-10-10T20:39:30.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.685+0000 7f060e5ce700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060814ed20 con 0x7f060806aa40 2023-10-10T20:39:30.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.685+0000 7f060cb6b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06080c91e0 0x7f060815b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:30.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.685+0000 7f060cb6b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06080c91e0 0x7f060815b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50780/0 (socket says 172.21.15.89:50780) 2023-10-10T20:39:30.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.685+0000 7f060cb6b700 1 -- 172.21.15.89:0/3284227753 learned_addr learned my addr 172.21.15.89:0/3284227753 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:30.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.685+0000 7f0607fff700 1 --2- 172.21.15.89:0/3284227753 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f060806aa40 0x7f060813ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:30.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.685+0000 7f06077fe700 1 --2- 172.21.15.89:0/3284227753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f060806bac0 0x7f060815b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:30.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.685+0000 7f060cb6b700 1 -- 172.21.15.89:0/3284227753 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f060806aa40 msgr2=0x7f060813ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:30.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.685+0000 7f060cb6b700 1 --2- 172.21.15.89:0/3284227753 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f060806aa40 0x7f060813ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:30.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.685+0000 7f060cb6b700 1 -- 172.21.15.89:0/3284227753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f060806bac0 msgr2=0x7f060815b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:30.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.685+0000 7f060cb6b700 1 --2- 172.21.15.89:0/3284227753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f060806bac0 0x7f060815b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:30.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.685+0000 7f060cb6b700 1 -- 172.21.15.89:0/3284227753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f80093c0 con 0x7f06080c91e0 2023-10-10T20:39:30.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.686+0000 7f060cb6b700 1 --2- 172.21.15.89:0/3284227753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06080c91e0 0x7f060815b5f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f060806c6a0 tx=0x7f0600019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:30.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.686+0000 7f05effff700 1 -- 172.21.15.89:0/3284227753 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600012e10 con 0x7f06080c91e0 2023-10-10T20:39:30.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.686+0000 7f05effff700 1 -- 172.21.15.89:0/3284227753 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0600027470 con 0x7f06080c91e0 2023-10-10T20:39:30.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.686+0000 7f060e5ce700 1 -- 172.21.15.89:0/3284227753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0600009a20 con 0x7f06080c91e0 2023-10-10T20:39:30.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.686+0000 7f05effff700 1 -- 172.21.15.89:0/3284227753 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600027830 con 0x7f06080c91e0 2023-10-10T20:39:30.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.686+0000 7f060e5ce700 1 -- 172.21.15.89:0/3284227753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060814f2e0 con 0x7f06080c91e0 2023-10-10T20:39:30.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.687+0000 7f060e5ce700 1 -- 172.21.15.89:0/3284227753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060806f7e0 con 0x7f06080c91e0 2023-10-10T20:39:30.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.687+0000 7f05effff700 1 -- 172.21.15.89:0/3284227753 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f0600024070 con 0x7f06080c91e0 2023-10-10T20:39:30.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.687+0000 7f05effff700 1 --2- 172.21.15.89:0/3284227753 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f05f4067170 0x7f05f4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:30.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.688+0000 7f0607fff700 1 --2- 172.21.15.89:0/3284227753 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f05f4067170 0x7f05f4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:30.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.688+0000 7f05effff700 1 -- 172.21.15.89:0/3284227753 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0600086ff0 con 0x7f06080c91e0 2023-10-10T20:39:30.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.688+0000 7f0607fff700 1 --2- 172.21.15.89:0/3284227753 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f05f4067170 0x7f05f4069620 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009f30 tx=0x7f05f8009120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:30.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.692+0000 7f05effff700 1 -- 172.21.15.89:0/3284227753 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0600054250 con 0x7f06080c91e0 2023-10-10T20:39:30.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:30.948+0000 7f060e5ce700 1 -- 172.21.15.89:0/3284227753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]} v 0) v1 -- 0x7f060804c3c0 con 0x7f06080c91e0 2023-10-10T20:39:31.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.211+0000 7f05effff700 1 -- 172.21.15.89:0/3284227753 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]}]=0 v227) v1 ==== 158+0+0 (secure 0 0 0) 0x7f0600057870 con 0x7f06080c91e0 2023-10-10T20:39:31.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.213+0000 7f060e5ce700 1 -- 172.21.15.89:0/3284227753 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f05f4067170 msgr2=0x7f05f4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:31.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.213+0000 7f060e5ce700 1 --2- 172.21.15.89:0/3284227753 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f05f4067170 0x7f05f4069620 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009f30 tx=0x7f05f8009120 comp rx=0 tx=0).stop 2023-10-10T20:39:31.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.213+0000 7f060e5ce700 1 -- 172.21.15.89:0/3284227753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06080c91e0 msgr2=0x7f060815b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:31.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.213+0000 7f060e5ce700 1 --2- 172.21.15.89:0/3284227753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06080c91e0 0x7f060815b5f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f060806c6a0 tx=0x7f0600019000 comp rx=0 tx=0).stop 2023-10-10T20:39:31.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.213+0000 7f060e5ce700 1 -- 172.21.15.89:0/3284227753 shutdown_connections 2023-10-10T20:39:31.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.213+0000 7f060e5ce700 1 --2- 172.21.15.89:0/3284227753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06080c91e0 0x7f060815b5f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:31.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.213+0000 7f060e5ce700 1 --2- 172.21.15.89:0/3284227753 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f060806aa40 0x7f060813ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:31.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.213+0000 7f060e5ce700 1 --2- 172.21.15.89:0/3284227753 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f05f4067170 0x7f05f4069620 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:31.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.213+0000 7f060e5ce700 1 --2- 172.21.15.89:0/3284227753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f060806bac0 0x7f060815b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:31.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.213+0000 7f060e5ce700 1 -- 172.21.15.89:0/3284227753 >> 172.21.15.89:0/3284227753 conn(0x7f06080cd670 msgr2=0x7f060814fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:31.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.213+0000 7f060e5ce700 1 -- 172.21.15.89:0/3284227753 shutdown_connections 2023-10-10T20:39:31.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.213+0000 7f060e5ce700 1 -- 172.21.15.89:0/3284227753 wait complete. 2023-10-10T20:39:31.228 INFO:tasks.cephfs_test_runner:Tests the pending_subvolume_deletions in 'fs volume info' command ... ok 2023-10-10T20:39:31.228 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_with_human_readable_flag (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info --human_readable' command 2023-10-10T20:39:31.229 INFO:tasks.cephfs_test_runner:test_volume_info_with_human_readable_flag (tasks.cephfs.test_volumes.TestVolumes) 2023-10-10T20:39:31.230 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag' 2023-10-10T20:39:31.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.343+0000 7f19d9436700 1 Processor -- start 2023-10-10T20:39:31.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.343+0000 7f19d9436700 1 -- start start 2023-10-10T20:39:31.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.343+0000 7f19d9436700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f19d4159c60 0x7f19d415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:31.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.343+0000 7f19d9436700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f19d415db10 0x7f19d4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:31.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.343+0000 7f19d9436700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d415a160 con 0x7f19d4159890 2023-10-10T20:39:31.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.343+0000 7f19d9436700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d415df50 con 0x7f19d415db10 2023-10-10T20:39:31.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.343+0000 7f19d9436700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d40d13a0 con 0x7f19d4159c60 2023-10-10T20:39:31.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.344+0000 7f19d37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f19d415db10 0x7f19d4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:31.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.344+0000 7f19d37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f19d415db10 0x7f19d4069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50786/0 (socket says 172.21.15.89:50786) 2023-10-10T20:39:31.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.344+0000 7f19d37fe700 1 -- 172.21.15.89:0/3092277550 learned_addr learned my addr 172.21.15.89:0/3092277550 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:31.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.344+0000 7f19ca7fc700 1 --2- 172.21.15.89:0/3092277550 >> v2:172.21.15.111:3300/0 conn(0x7f19d4159c60 0x7f19d415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:31.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.344+0000 7f19d37fe700 1 -- 172.21.15.89:0/3092277550 >> v2:172.21.15.111:3300/0 conn(0x7f19d4159c60 msgr2=0x7f19d415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:31.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.344+0000 7f19d37fe700 1 --2- 172.21.15.89:0/3092277550 >> v2:172.21.15.111:3300/0 conn(0x7f19d4159c60 0x7f19d415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:31.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.344+0000 7f19d37fe700 1 -- 172.21.15.89:0/3092277550 >> v1:172.21.15.111:6789/0 conn(0x7f19d4159890 legacy=0x7f19d415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:31.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.344+0000 7f19d37fe700 1 -- 172.21.15.89:0/3092277550 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d415e140 con 0x7f19d415db10 2023-10-10T20:39:31.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.344+0000 7f19ca7fc700 1 --2- 172.21.15.89:0/3092277550 >> v2:172.21.15.111:3300/0 conn(0x7f19d4159c60 0x7f19d415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:31.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.344+0000 7f19d37fe700 1 --2- 172.21.15.89:0/3092277550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19d415db10 0x7f19d4069560 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f19d40b67f0 tx=0x7f19cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4527fe0d7850fd server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:31.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.344+0000 7f19d27fc700 1 -- 172.21.15.89:0/3092277550 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc00e070 con 0x7f19d415db10 2023-10-10T20:39:31.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.344+0000 7f19d27fc700 1 -- 172.21.15.89:0/3092277550 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc00ba10 con 0x7f19d415db10 2023-10-10T20:39:31.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.344+0000 7f19d27fc700 1 -- 172.21.15.89:0/3092277550 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0092e0 con 0x7f19d415db10 2023-10-10T20:39:31.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.345+0000 7f19d9436700 1 -- 172.21.15.89:0/3092277550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19d415db10 msgr2=0x7f19d4069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:31.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.345+0000 7f19d9436700 1 --2- 172.21.15.89:0/3092277550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19d415db10 0x7f19d4069560 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f19d40b67f0 tx=0x7f19cc009840 comp rx=0 tx=0).stop 2023-10-10T20:39:31.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.345+0000 7f19d9436700 1 -- 172.21.15.89:0/3092277550 shutdown_connections 2023-10-10T20:39:31.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.345+0000 7f19d9436700 1 --2- 172.21.15.89:0/3092277550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19d415db10 0x7f19d4069560 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:31.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.345+0000 7f19d9436700 1 --2- 172.21.15.89:0/3092277550 >> v2:172.21.15.111:3300/0 conn(0x7f19d4159c60 0x7f19d415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:31.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.345+0000 7f19d9436700 1 -- 172.21.15.89:0/3092277550 >> 172.21.15.89:0/3092277550 conn(0x7f19d4068e20 msgr2=0x7f19d40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:31.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.345+0000 7f19d9436700 1 -- 172.21.15.89:0/3092277550 shutdown_connections 2023-10-10T20:39:31.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.345+0000 7f19d9436700 1 -- 172.21.15.89:0/3092277550 wait complete. 2023-10-10T20:39:31.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.346+0000 7f19d9436700 1 Processor -- start 2023-10-10T20:39:31.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.346+0000 7f19d9436700 1 -- start start 2023-10-10T20:39:31.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19d9436700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19d4159890 0x7f19d40b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:31.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19d9436700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19d4159c60 0x7f19d4154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:31.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19d9436700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19d415db10 0x7f19d4144120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:31.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19d9436700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4153cb0 con 0x7f19d415db10 2023-10-10T20:39:31.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19d9436700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d41552a0 con 0x7f19d4159c60 2023-10-10T20:39:31.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19d9436700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4155580 con 0x7f19d4159890 2023-10-10T20:39:31.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19d37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19d415db10 0x7f19d4144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:31.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19d37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19d415db10 0x7f19d4144120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50800/0 (socket says 172.21.15.89:50800) 2023-10-10T20:39:31.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19d37fe700 1 -- 172.21.15.89:0/847252782 learned_addr learned my addr 172.21.15.89:0/847252782 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:31.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19ca7fc700 1 --2- 172.21.15.89:0/847252782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19d4159c60 0x7f19d4154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:31.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19d2ffd700 1 --2- 172.21.15.89:0/847252782 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19d4159890 0x7f19d40b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:31.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19d37fe700 1 -- 172.21.15.89:0/847252782 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19d4159890 msgr2=0x7f19d40b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:31.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19d37fe700 1 --2- 172.21.15.89:0/847252782 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19d4159890 0x7f19d40b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:31.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19d37fe700 1 -- 172.21.15.89:0/847252782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19d4159c60 msgr2=0x7f19d4154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:31.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19d37fe700 1 --2- 172.21.15.89:0/847252782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19d4159c60 0x7f19d4154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:31.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19d37fe700 1 -- 172.21.15.89:0/847252782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19cc009ae0 con 0x7f19d415db10 2023-10-10T20:39:31.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19ca7fc700 1 --2- 172.21.15.89:0/847252782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19d4159c60 0x7f19d4154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:31.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19d2ffd700 1 --2- 172.21.15.89:0/847252782 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19d4159890 0x7f19d40b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:31.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19d37fe700 1 --2- 172.21.15.89:0/847252782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19d415db10 0x7f19d4144120 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f19d4070e20 tx=0x7f19cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:31.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.347+0000 7f19cb7fe700 1 -- 172.21.15.89:0/847252782 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0032f0 con 0x7f19d415db10 2023-10-10T20:39:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.348+0000 7f19cb7fe700 1 -- 172.21.15.89:0/847252782 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc012e60 con 0x7f19d415db10 2023-10-10T20:39:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.348+0000 7f19d9436700 1 -- 172.21.15.89:0/847252782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d4155860 con 0x7f19d415db10 2023-10-10T20:39:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.348+0000 7f19cb7fe700 1 -- 172.21.15.89:0/847252782 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc01d670 con 0x7f19d415db10 2023-10-10T20:39:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.348+0000 7f19d9436700 1 -- 172.21.15.89:0/847252782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d40b2860 con 0x7f19d415db10 2023-10-10T20:39:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.349+0000 7f19d9436700 1 -- 172.21.15.89:0/847252782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d4153e60 con 0x7f19d415db10 2023-10-10T20:39:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.349+0000 7f19cb7fe700 1 -- 172.21.15.89:0/847252782 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f19cc01daa0 con 0x7f19d415db10 2023-10-10T20:39:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.349+0000 7f19cb7fe700 1 --2- 172.21.15.89:0/847252782 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f19b4067170 0x7f19b4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.349+0000 7f19cb7fe700 1 -- 172.21.15.89:0/847252782 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f19cc086010 con 0x7f19d415db10 2023-10-10T20:39:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.349+0000 7f19d2ffd700 1 --2- 172.21.15.89:0/847252782 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f19b4067170 0x7f19b4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.350+0000 7f19d2ffd700 1 --2- 172.21.15.89:0/847252782 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f19b4067170 0x7f19b4069620 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f19c4007fd0 tx=0x7f19c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:31.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.353+0000 7f19cb7fe700 1 -- 172.21.15.89:0/847252782 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19d4153e60 con 0x7f19d415db10 2023-10-10T20:39:31.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:31.610+0000 7f19d9436700 1 -- 172.21.15.89:0/847252782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]} v 0) v1 -- 0x7f19d4155a50 con 0x7f19d415db10 2023-10-10T20:39:32.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.212+0000 7f19cb7fe700 1 -- 172.21.15.89:0/847252782 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]}]=0 v228) v1 ==== 161+0+0 (secure 0 0 0) 0x7f19cc0531e0 con 0x7f19d415db10 2023-10-10T20:39:32.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.214+0000 7f19d9436700 1 -- 172.21.15.89:0/847252782 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f19b4067170 msgr2=0x7f19b4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:32.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.214+0000 7f19d9436700 1 --2- 172.21.15.89:0/847252782 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f19b4067170 0x7f19b4069620 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f19c4007fd0 tx=0x7f19c4009040 comp rx=0 tx=0).stop 2023-10-10T20:39:32.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.214+0000 7f19d9436700 1 -- 172.21.15.89:0/847252782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19d415db10 msgr2=0x7f19d4144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:32.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.214+0000 7f19d9436700 1 --2- 172.21.15.89:0/847252782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19d415db10 0x7f19d4144120 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f19d4070e20 tx=0x7f19cc017000 comp rx=0 tx=0).stop 2023-10-10T20:39:32.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.214+0000 7f19d9436700 1 -- 172.21.15.89:0/847252782 shutdown_connections 2023-10-10T20:39:32.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.214+0000 7f19d9436700 1 --2- 172.21.15.89:0/847252782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19d415db10 0x7f19d4144120 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.214+0000 7f19d9436700 1 --2- 172.21.15.89:0/847252782 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19d4159890 0x7f19d40b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.214+0000 7f19d9436700 1 --2- 172.21.15.89:0/847252782 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f19b4067170 0x7f19b4069620 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.214+0000 7f19d9436700 1 --2- 172.21.15.89:0/847252782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19d4159c60 0x7f19d4154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.214+0000 7f19d9436700 1 -- 172.21.15.89:0/847252782 >> 172.21.15.89:0/847252782 conn(0x7f19d4068e20 msgr2=0x7f19d4157020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:32.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.214+0000 7f19d9436700 1 -- 172.21.15.89:0/847252782 shutdown_connections 2023-10-10T20:39:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.214+0000 7f19d9436700 1 -- 172.21.15.89:0/847252782 wait complete. 2023-10-10T20:39:32.229 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:39:32.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.345+0000 7f61e7365700 1 Processor -- start 2023-10-10T20:39:32.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.346+0000 7f61e7365700 1 -- start start 2023-10-10T20:39:32.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.346+0000 7f61e7365700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f61e0159890 0x7f61e015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:32.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.346+0000 7f61e7365700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e015a160 con 0x7f61e0069560 2023-10-10T20:39:32.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.346+0000 7f61e7365700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e00d13a0 con 0x7f61e0159c60 2023-10-10T20:39:32.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.346+0000 7f61e7365700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e015e140 con 0x7f61e0159890 2023-10-10T20:39:32.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.346+0000 7f61e5101700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f61e0159890 0x7f61e015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:32.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.347+0000 7f61e5902700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f61e0069560 0x7f61e00ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38900/0 (socket says 172.21.15.89:38900) 2023-10-10T20:39:32.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.347+0000 7f61e5902700 1 -- 172.21.15.89:0/3219289600 learned_addr learned my addr 172.21.15.89:0/3219289600 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:32.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.347+0000 7f61e5101700 1 -- 172.21.15.89:0/3219289600 >> v1:172.21.15.111:6789/0 conn(0x7f61e0159c60 legacy=0x7f61e015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:32.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.347+0000 7f61e5101700 1 -- 172.21.15.89:0/3219289600 >> v1:172.21.15.89:6789/0 conn(0x7f61e0069560 legacy=0x7f61e00ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:32.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.347+0000 7f61e5101700 1 -- 172.21.15.89:0/3219289600 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e015a3d0 con 0x7f61e0159890 2023-10-10T20:39:32.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.347+0000 7f61d7fff700 1 -- 172.21.15.89:0/3219289600 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 722051211 0 0) 0x7f61e015a160 con 0x7f61e0069560 2023-10-10T20:39:32.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.347+0000 7f61e5101700 1 --2- 172.21.15.89:0/3219289600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61e0159890 0x7f61e015abd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009e60 tx=0x7f61dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d43daf72eaa51b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:32.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.347+0000 7f61d7fff700 1 -- 172.21.15.89:0/3219289600 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc00e070 con 0x7f61e0159890 2023-10-10T20:39:32.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.347+0000 7f61d7fff700 1 -- 172.21.15.89:0/3219289600 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc00bbf0 con 0x7f61e0159890 2023-10-10T20:39:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.347+0000 7f61d7fff700 1 -- 172.21.15.89:0/3219289600 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc012430 con 0x7f61e0159890 2023-10-10T20:39:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.347+0000 7f61e7365700 1 -- 172.21.15.89:0/3219289600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61e0159890 msgr2=0x7f61e015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.348+0000 7f61e7365700 1 --2- 172.21.15.89:0/3219289600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61e0159890 0x7f61e015abd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009e60 tx=0x7f61dc009780 comp rx=0 tx=0).stop 2023-10-10T20:39:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.348+0000 7f61e7365700 1 -- 172.21.15.89:0/3219289600 shutdown_connections 2023-10-10T20:39:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.348+0000 7f61e7365700 1 --2- 172.21.15.89:0/3219289600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61e0159890 0x7f61e015abd0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.348+0000 7f61e7365700 1 -- 172.21.15.89:0/3219289600 >> 172.21.15.89:0/3219289600 conn(0x7f61e0068e20 msgr2=0x7f61e00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:32.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.348+0000 7f61e7365700 1 -- 172.21.15.89:0/3219289600 shutdown_connections 2023-10-10T20:39:32.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.348+0000 7f61e7365700 1 -- 172.21.15.89:0/3219289600 wait complete. 2023-10-10T20:39:32.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.349+0000 7f61e7365700 1 Processor -- start 2023-10-10T20:39:32.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.349+0000 7f61e7365700 1 -- start start 2023-10-10T20:39:32.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e7365700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61e0069560 0x7f61e014d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:32.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e7365700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61e0159890 0x7f61e0152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e7365700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61e0159c60 0x7f61e00b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e7365700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e014d660 con 0x7f61e0159c60 2023-10-10T20:39:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e7365700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e0152790 con 0x7f61e0159890 2023-10-10T20:39:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e7365700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e0152a30 con 0x7f61e0069560 2023-10-10T20:39:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e5902700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61e0159c60 0x7f61e00b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e5902700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61e0159c60 0x7f61e00b4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50810/0 (socket says 172.21.15.89:50810) 2023-10-10T20:39:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e5902700 1 -- 172.21.15.89:0/50306037 learned_addr learned my addr 172.21.15.89:0/50306037 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e4900700 1 --2- 172.21.15.89:0/50306037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61e0159890 0x7f61e0152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e5101700 1 --2- 172.21.15.89:0/50306037 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61e0069560 0x7f61e014d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e5902700 1 -- 172.21.15.89:0/50306037 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61e0069560 msgr2=0x7f61e014d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e5902700 1 --2- 172.21.15.89:0/50306037 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61e0069560 0x7f61e014d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e5902700 1 -- 172.21.15.89:0/50306037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61e0159890 msgr2=0x7f61e0152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e5902700 1 --2- 172.21.15.89:0/50306037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61e0159890 0x7f61e0152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e5902700 1 -- 172.21.15.89:0/50306037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61dc009a20 con 0x7f61e0159c60 2023-10-10T20:39:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e5101700 1 --2- 172.21.15.89:0/50306037 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61e0069560 0x7f61e014d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:32.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e5902700 1 --2- 172.21.15.89:0/50306037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61e0159c60 0x7f61e00b4a50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f61d800c530 tx=0x7f61d8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:32.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61d57fa700 1 -- 172.21.15.89:0/50306037 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d8005b30 con 0x7f61e0159c60 2023-10-10T20:39:32.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61d57fa700 1 -- 172.21.15.89:0/50306037 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d8005c90 con 0x7f61e0159c60 2023-10-10T20:39:32.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e7365700 1 -- 172.21.15.89:0/50306037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e0152d10 con 0x7f61e0159c60 2023-10-10T20:39:32.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61d57fa700 1 -- 172.21.15.89:0/50306037 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d8007320 con 0x7f61e0159c60 2023-10-10T20:39:32.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.350+0000 7f61e7365700 1 -- 172.21.15.89:0/50306037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e01532d0 con 0x7f61e0159c60 2023-10-10T20:39:32.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.351+0000 7f61e7365700 1 -- 172.21.15.89:0/50306037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e004c3c0 con 0x7f61e0159c60 2023-10-10T20:39:32.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.352+0000 7f61d57fa700 1 -- 172.21.15.89:0/50306037 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f61d801b3b0 con 0x7f61e0159c60 2023-10-10T20:39:32.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.352+0000 7f61d57fa700 1 --2- 172.21.15.89:0/50306037 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f61c4067170 0x7f61c4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:32.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.352+0000 7f61d57fa700 1 -- 172.21.15.89:0/50306037 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f61d80839e0 con 0x7f61e0159c60 2023-10-10T20:39:32.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.352+0000 7f61e5101700 1 --2- 172.21.15.89:0/50306037 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f61c4067170 0x7f61c4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:32.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.352+0000 7f61e5101700 1 --2- 172.21.15.89:0/50306037 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f61c4067170 0x7f61c4069620 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f61e00c7b50 tx=0x7f61dc018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:32.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.356+0000 7f61d57fa700 1 -- 172.21.15.89:0/50306037 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61d8050cc0 con 0x7f61e0159c60 2023-10-10T20:39:32.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.541+0000 7f61e7365700 1 -- 172.21.15.89:0/50306037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f61e006a5b0 con 0x7f61e0159c60 2023-10-10T20:39:32.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.545+0000 7f61d57fa700 1 -- 172.21.15.89:0/50306037 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f61d80170c0 con 0x7f61e0159c60 2023-10-10T20:39:32.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.547+0000 7f61e7365700 1 -- 172.21.15.89:0/50306037 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f61c4067170 msgr2=0x7f61c4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:32.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.547+0000 7f61e7365700 1 --2- 172.21.15.89:0/50306037 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f61c4067170 0x7f61c4069620 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f61e00c7b50 tx=0x7f61dc018000 comp rx=0 tx=0).stop 2023-10-10T20:39:32.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.547+0000 7f61e7365700 1 -- 172.21.15.89:0/50306037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61e0159c60 msgr2=0x7f61e00b4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:32.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.547+0000 7f61e7365700 1 --2- 172.21.15.89:0/50306037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61e0159c60 0x7f61e00b4a50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f61d800c530 tx=0x7f61d8007f90 comp rx=0 tx=0).stop 2023-10-10T20:39:32.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.547+0000 7f61e7365700 1 -- 172.21.15.89:0/50306037 shutdown_connections 2023-10-10T20:39:32.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.547+0000 7f61e7365700 1 --2- 172.21.15.89:0/50306037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61e0159c60 0x7f61e00b4a50 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.547+0000 7f61e7365700 1 --2- 172.21.15.89:0/50306037 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61e0069560 0x7f61e014d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.547+0000 7f61e7365700 1 --2- 172.21.15.89:0/50306037 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f61c4067170 0x7f61c4069620 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.547+0000 7f61e7365700 1 --2- 172.21.15.89:0/50306037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61e0159890 0x7f61e0152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.547+0000 7f61e7365700 1 -- 172.21.15.89:0/50306037 >> 172.21.15.89:0/50306037 conn(0x7f61e0068e20 msgr2=0x7f61e00cf650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:32.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.547+0000 7f61e7365700 1 -- 172.21.15.89:0/50306037 shutdown_connections 2023-10-10T20:39:32.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.547+0000 7f61e7365700 1 -- 172.21.15.89:0/50306037 wait complete. 2023-10-10T20:39:32.561 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:39:32.562 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:39:32.602 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:39:32.645 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:39:32.645 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:39:32.646 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:39:32.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.758+0000 7fe9f51c8700 1 Processor -- start 2023-10-10T20:39:32.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.758+0000 7fe9f51c8700 1 -- start start 2023-10-10T20:39:32.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.758+0000 7fe9f51c8700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe9f00b5ea0 0x7fe9f0068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:32.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.758+0000 7fe9f51c8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe9f0068bb0 0x7fe9f015c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:32.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.758+0000 7fe9f51c8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f015c480 con 0x7fe9f0159c10 2023-10-10T20:39:32.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.758+0000 7fe9f51c8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f00b2a40 con 0x7fe9f0068bb0 2023-10-10T20:39:32.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.758+0000 7fe9f51c8700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f015b870 con 0x7fe9f00b5ea0 2023-10-10T20:39:32.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.759+0000 7fe9ef59e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe9f0068bb0 0x7fe9f015c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:32.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.759+0000 7fe9eed9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe9f0159c10 0x7fe9f015b090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38912/0 (socket says 172.21.15.89:38912) 2023-10-10T20:39:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.759+0000 7fe9ef59e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe9f0068bb0 0x7fe9f015c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50824/0 (socket says 172.21.15.89:50824) 2023-10-10T20:39:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.759+0000 7fe9eed9d700 1 -- 172.21.15.89:0/4199426944 learned_addr learned my addr 172.21.15.89:0/4199426944 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.759+0000 7fe9ee59c700 1 --2- 172.21.15.89:0/4199426944 >> v2:172.21.15.111:3300/0 conn(0x7fe9f00b5ea0 0x7fe9f0068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.759+0000 7fe9ef59e700 1 -- 172.21.15.89:0/4199426944 >> v2:172.21.15.111:3300/0 conn(0x7fe9f00b5ea0 msgr2=0x7fe9f0068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.759+0000 7fe9ef59e700 1 --2- 172.21.15.89:0/4199426944 >> v2:172.21.15.111:3300/0 conn(0x7fe9f00b5ea0 0x7fe9f0068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.759+0000 7fe9ef59e700 1 -- 172.21.15.89:0/4199426944 >> v1:172.21.15.89:6789/0 conn(0x7fe9f0159c10 legacy=0x7fe9f015b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.759+0000 7fe9ef59e700 1 -- 172.21.15.89:0/4199426944 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f015bb50 con 0x7fe9f0068bb0 2023-10-10T20:39:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.759+0000 7fe9edd9b700 1 -- 172.21.15.89:0/4199426944 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3920489380 0 0) 0x7fe9f015c480 con 0x7fe9f0159c10 2023-10-10T20:39:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.759+0000 7fe9ef59e700 1 --2- 172.21.15.89:0/4199426944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f0068bb0 0x7fe9f015c910 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00d14f0 tx=0x7fe9e8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41c8e69ed24d2b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.759+0000 7fe9edd9b700 1 -- 172.21.15.89:0/4199426944 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e800e070 con 0x7fe9f0068bb0 2023-10-10T20:39:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.759+0000 7fe9edd9b700 1 -- 172.21.15.89:0/4199426944 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e800bc30 con 0x7fe9f0068bb0 2023-10-10T20:39:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.759+0000 7fe9edd9b700 1 -- 172.21.15.89:0/4199426944 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8012430 con 0x7fe9f0068bb0 2023-10-10T20:39:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.760+0000 7fe9f51c8700 1 -- 172.21.15.89:0/4199426944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f0068bb0 msgr2=0x7fe9f015c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.760+0000 7fe9f51c8700 1 --2- 172.21.15.89:0/4199426944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f0068bb0 0x7fe9f015c910 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00d14f0 tx=0x7fe9e8009780 comp rx=0 tx=0).stop 2023-10-10T20:39:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.760+0000 7fe9f51c8700 1 -- 172.21.15.89:0/4199426944 shutdown_connections 2023-10-10T20:39:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.760+0000 7fe9f51c8700 1 --2- 172.21.15.89:0/4199426944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f0068bb0 0x7fe9f015c910 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.760+0000 7fe9f51c8700 1 --2- 172.21.15.89:0/4199426944 >> v2:172.21.15.111:3300/0 conn(0x7fe9f00b5ea0 0x7fe9f0068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.760+0000 7fe9f51c8700 1 -- 172.21.15.89:0/4199426944 >> 172.21.15.89:0/4199426944 conn(0x7fe9f0099a20 msgr2=0x7fe9f00af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.760+0000 7fe9f51c8700 1 -- 172.21.15.89:0/4199426944 shutdown_connections 2023-10-10T20:39:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.760+0000 7fe9f51c8700 1 -- 172.21.15.89:0/4199426944 wait complete. 2023-10-10T20:39:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.761+0000 7fe9f51c8700 1 Processor -- start 2023-10-10T20:39:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.761+0000 7fe9f51c8700 1 -- start start 2023-10-10T20:39:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9f51c8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9f0068bb0 0x7fe9f0167070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9f51c8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f00b5ea0 0x7fe9f0169e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9f51c8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9f0159c10 0x7fe9f0168d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9f51c8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f016b580 con 0x7fe9f00b5ea0 2023-10-10T20:39:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9f51c8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f016a3a0 con 0x7fe9f0159c10 2023-10-10T20:39:32.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9f51c8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f0169280 con 0x7fe9f0068bb0 2023-10-10T20:39:32.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9ee59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f00b5ea0 0x7fe9f0169e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:32.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9ee59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f00b5ea0 0x7fe9f0169e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50840/0 (socket says 172.21.15.89:50840) 2023-10-10T20:39:32.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9ee59c700 1 -- 172.21.15.89:0/1597653136 learned_addr learned my addr 172.21.15.89:0/1597653136 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:32.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9ef59e700 1 --2- 172.21.15.89:0/1597653136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9f0159c10 0x7fe9f0168d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9eed9d700 1 --2- 172.21.15.89:0/1597653136 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9f0068bb0 0x7fe9f0167070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9ee59c700 1 -- 172.21.15.89:0/1597653136 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9f0068bb0 msgr2=0x7fe9f0167070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9ee59c700 1 --2- 172.21.15.89:0/1597653136 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9f0068bb0 0x7fe9f0167070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9ee59c700 1 -- 172.21.15.89:0/1597653136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9f0159c10 msgr2=0x7fe9f0168d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9ee59c700 1 --2- 172.21.15.89:0/1597653136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9f0159c10 0x7fe9f0168d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9ee59c700 1 -- 172.21.15.89:0/1597653136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e8009a20 con 0x7fe9f00b5ea0 2023-10-10T20:39:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9ee59c700 1 --2- 172.21.15.89:0/1597653136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f00b5ea0 0x7fe9f0169e60 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400d890 tx=0x7fe9e4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9d6ffd700 1 -- 172.21.15.89:0/1597653136 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40125f0 con 0x7fe9f00b5ea0 2023-10-10T20:39:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9d6ffd700 1 -- 172.21.15.89:0/1597653136 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e4012750 con 0x7fe9f00b5ea0 2023-10-10T20:39:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9f51c8700 1 -- 172.21.15.89:0/1597653136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f016a970 con 0x7fe9f00b5ea0 2023-10-10T20:39:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9d6ffd700 1 -- 172.21.15.89:0/1597653136 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4012ad0 con 0x7fe9f00b5ea0 2023-10-10T20:39:32.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.762+0000 7fe9f51c8700 1 -- 172.21.15.89:0/1597653136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f016af30 con 0x7fe9f00b5ea0 2023-10-10T20:39:32.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.763+0000 7fe9f51c8700 1 -- 172.21.15.89:0/1597653136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f016b6f0 con 0x7fe9f00b5ea0 2023-10-10T20:39:32.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.764+0000 7fe9d6ffd700 1 -- 172.21.15.89:0/1597653136 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fe9e401e460 con 0x7fe9f00b5ea0 2023-10-10T20:39:32.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.764+0000 7fe9d6ffd700 1 --2- 172.21.15.89:0/1597653136 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe9dc067170 0x7fe9dc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:32.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.764+0000 7fe9d6ffd700 1 -- 172.21.15.89:0/1597653136 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fe9e4084b40 con 0x7fe9f00b5ea0 2023-10-10T20:39:32.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.764+0000 7fe9eed9d700 1 --2- 172.21.15.89:0/1597653136 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe9dc067170 0x7fe9dc069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.765+0000 7fe9eed9d700 1 --2- 172.21.15.89:0/1597653136 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe9dc067170 0x7fe9dc069620 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0148be0 tx=0x7fe9e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.768+0000 7fe9d6ffd700 1 -- 172.21.15.89:0/1597653136 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe9e4051e20 con 0x7fe9f00b5ea0 2023-10-10T20:39:32.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.993+0000 7fe9f51c8700 1 -- 172.21.15.89:0/1597653136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9f004c3c0 con 0x7fe9f00b5ea0 2023-10-10T20:39:32.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.993+0000 7fe9d6ffd700 1 -- 172.21.15.89:0/1597653136 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fe9e4055440 con 0x7fe9f00b5ea0 2023-10-10T20:39:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:32.996 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":40,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/3362909084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":3362909084},{"type":"v1","addr":"172.21.15.111:6835","nonce":3362909084}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5102,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1472423656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1472423656},{"type":"v1","addr":"172.21.15.111:6834","nonce":1472423656}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/3183325104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3183325104},{"type":"v1","addr":"172.21.15.89:6836","nonce":3183325104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":6003,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/4288009381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":4288009381},{"type":"v1","addr":"172.21.15.89:6837","nonce":4288009381}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[]} 2023-10-10T20:39:32.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.995+0000 7fe9f51c8700 1 -- 172.21.15.89:0/1597653136 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe9dc067170 msgr2=0x7fe9dc069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:32.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.995+0000 7fe9f51c8700 1 --2- 172.21.15.89:0/1597653136 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe9dc067170 0x7fe9dc069620 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0148be0 tx=0x7fe9e0009040 comp rx=0 tx=0).stop 2023-10-10T20:39:32.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.995+0000 7fe9f51c8700 1 -- 172.21.15.89:0/1597653136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f00b5ea0 msgr2=0x7fe9f0169e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:32.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.995+0000 7fe9f51c8700 1 --2- 172.21.15.89:0/1597653136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f00b5ea0 0x7fe9f0169e60 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400d890 tx=0x7fe9e4008040 comp rx=0 tx=0).stop 2023-10-10T20:39:32.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.996+0000 7fe9f51c8700 1 -- 172.21.15.89:0/1597653136 shutdown_connections 2023-10-10T20:39:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.996+0000 7fe9f51c8700 1 --2- 172.21.15.89:0/1597653136 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe9dc067170 0x7fe9dc069620 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.996+0000 7fe9f51c8700 1 --2- 172.21.15.89:0/1597653136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9f0159c10 0x7fe9f0168d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.996+0000 7fe9f51c8700 1 --2- 172.21.15.89:0/1597653136 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe9f0068bb0 0x7fe9f0167070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.996+0000 7fe9f51c8700 1 --2- 172.21.15.89:0/1597653136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f00b5ea0 0x7fe9f0169e60 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.996+0000 7fe9f51c8700 1 -- 172.21.15.89:0/1597653136 >> 172.21.15.89:0/1597653136 conn(0x7fe9f0099a20 msgr2=0x7fe9f009a520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:33.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.996+0000 7fe9f51c8700 1 -- 172.21.15.89:0/1597653136 shutdown_connections 2023-10-10T20:39:33.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:32.996+0000 7fe9f51c8700 1 -- 172.21.15.89:0/1597653136 wait complete. 2023-10-10T20:39:33.000 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 40 2023-10-10T20:39:33.010 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:39:33.010 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:39:33.011 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:39:33.011 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:39:33.011 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:39:33.011 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:39:33.012 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:39:33.012 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:39:33.012 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:39:33.012 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:39:33.013 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:39:33.013 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:39:33.013 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:39:33.013 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:39:33.014 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:39:33.014 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:39:33.019 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:39:33.019 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:39:33.021 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:39:33.021 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:39:33.023 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:39:33.024 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:39:33.025 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:39:33.025 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:39:33.062 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:39:33.063 DEBUG:teuthology.parallel:result is None 2023-10-10T20:39:33.064 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:39:33.064 DEBUG:teuthology.parallel:result is None 2023-10-10T20:39:33.066 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:39:33.067 DEBUG:teuthology.parallel:result is None 2023-10-10T20:39:33.067 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:39:33.068 DEBUG:teuthology.parallel:result is None 2023-10-10T20:39:33.068 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:39:33.140 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:39:33.140 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-10T20:39:33.150 INFO:tasks.ceph.mds.b.smithi111.stdout:starting mds.b at 2023-10-10T20:39:33.150 INFO:tasks.ceph.mds.d.smithi111.stdout:starting mds.d at 2023-10-10T20:39:33.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.192+0000 7f28edac0700 1 Processor -- start 2023-10-10T20:39:33.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.192+0000 7f28edac0700 1 -- start start 2023-10-10T20:39:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.192+0000 7f28edac0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f28e8159c60 0x7f28e815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.192+0000 7f28edac0700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f28e815db10 0x7f28e8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.192+0000 7f28edac0700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e815a160 con 0x7f28e8159890 2023-10-10T20:39:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.192+0000 7f28edac0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e815df50 con 0x7f28e8159c60 2023-10-10T20:39:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.192+0000 7f28edac0700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e80d13a0 con 0x7f28e815db10 2023-10-10T20:39:33.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.193+0000 7f28e6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f28e8159c60 0x7f28e815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:33.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.193+0000 7f28e6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f28e8159c60 0x7f28e815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50866/0 (socket says 172.21.15.89:50866) 2023-10-10T20:39:33.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.193+0000 7f28e6ffd700 1 -- 172.21.15.89:0/3943853820 learned_addr learned my addr 172.21.15.89:0/3943853820 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:33.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.193+0000 7f28e7fff700 1 --2- 172.21.15.89:0/3943853820 >> v2:172.21.15.111:3300/0 conn(0x7f28e815db10 0x7f28e8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:33.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.193+0000 7f28e6ffd700 1 -- 172.21.15.89:0/3943853820 >> v2:172.21.15.111:3300/0 conn(0x7f28e815db10 msgr2=0x7f28e8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:33.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.193+0000 7f28e6ffd700 1 --2- 172.21.15.89:0/3943853820 >> v2:172.21.15.111:3300/0 conn(0x7f28e815db10 0x7f28e8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:33.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.193+0000 7f28e6ffd700 1 -- 172.21.15.89:0/3943853820 >> v1:172.21.15.111:6789/0 conn(0x7f28e8159890 legacy=0x7f28e815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:33.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.193+0000 7f28e6ffd700 1 -- 172.21.15.89:0/3943853820 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e815e140 con 0x7f28e8159c60 2023-10-10T20:39:33.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.193+0000 7f28e6ffd700 1 --2- 172.21.15.89:0/3943853820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28e8159c60 0x7f28e815d5d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f28e806a890 tx=0x7f28dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6ab5091b7fa4db server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:33.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.193+0000 7f28e67fc700 1 -- 172.21.15.89:0/3943853820 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc009e70 con 0x7f28e8159c60 2023-10-10T20:39:33.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.193+0000 7f28e67fc700 1 -- 172.21.15.89:0/3943853820 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc00b860 con 0x7f28e8159c60 2023-10-10T20:39:33.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.193+0000 7f28e67fc700 1 -- 172.21.15.89:0/3943853820 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc00bc00 con 0x7f28e8159c60 2023-10-10T20:39:33.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.194+0000 7f28edac0700 1 -- 172.21.15.89:0/3943853820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28e8159c60 msgr2=0x7f28e815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:33.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.194+0000 7f28edac0700 1 --2- 172.21.15.89:0/3943853820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28e8159c60 0x7f28e815d5d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f28e806a890 tx=0x7f28dc009250 comp rx=0 tx=0).stop 2023-10-10T20:39:33.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.194+0000 7f28edac0700 1 -- 172.21.15.89:0/3943853820 shutdown_connections 2023-10-10T20:39:33.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.194+0000 7f28edac0700 1 --2- 172.21.15.89:0/3943853820 >> v2:172.21.15.111:3300/0 conn(0x7f28e815db10 0x7f28e8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:33.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.194+0000 7f28edac0700 1 --2- 172.21.15.89:0/3943853820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28e8159c60 0x7f28e815d5d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:33.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.194+0000 7f28edac0700 1 -- 172.21.15.89:0/3943853820 >> 172.21.15.89:0/3943853820 conn(0x7f28e8068e20 msgr2=0x7f28e80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:33.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.194+0000 7f28edac0700 1 -- 172.21.15.89:0/3943853820 shutdown_connections 2023-10-10T20:39:33.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.194+0000 7f28edac0700 1 -- 172.21.15.89:0/3943853820 wait complete. 2023-10-10T20:39:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.195+0000 7f28edac0700 1 Processor -- start 2023-10-10T20:39:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.195+0000 7f28edac0700 1 -- start start 2023-10-10T20:39:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.196+0000 7f28edac0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28e8159890 0x7f28e8140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.196+0000 7f28edac0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28e8159c60 0x7f28e80b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.196+0000 7f28edac0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28e815db10 0x7f28e80b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.196+0000 7f28edac0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e8141300 con 0x7f28e8159c60 2023-10-10T20:39:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.196+0000 7f28edac0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80cd6d0 con 0x7f28e815db10 2023-10-10T20:39:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.196+0000 7f28edac0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80cd9b0 con 0x7f28e8159890 2023-10-10T20:39:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.196+0000 7f28e6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28e8159c60 0x7f28e80b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.196+0000 7f28e6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28e8159c60 0x7f28e80b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50868/0 (socket says 172.21.15.89:50868) 2023-10-10T20:39:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.196+0000 7f28e6ffd700 1 -- 172.21.15.89:0/2184275947 learned_addr learned my addr 172.21.15.89:0/2184275947 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.196+0000 7f28e7fff700 1 --2- 172.21.15.89:0/2184275947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28e815db10 0x7f28e80b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.196+0000 7f28e77fe700 1 --2- 172.21.15.89:0/2184275947 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28e8159890 0x7f28e8140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.196+0000 7f28e6ffd700 1 -- 172.21.15.89:0/2184275947 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28e8159890 msgr2=0x7f28e8140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.196+0000 7f28e6ffd700 1 --2- 172.21.15.89:0/2184275947 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28e8159890 0x7f28e8140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.196+0000 7f28e6ffd700 1 -- 172.21.15.89:0/2184275947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28e815db10 msgr2=0x7f28e80b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.196+0000 7f28e6ffd700 1 --2- 172.21.15.89:0/2184275947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28e815db10 0x7f28e80b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.196+0000 7f28e6ffd700 1 -- 172.21.15.89:0/2184275947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc0094f0 con 0x7f28e8159c60 2023-10-10T20:39:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.196+0000 7f28e6ffd700 1 --2- 172.21.15.89:0/2184275947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28e8159c60 0x7f28e80b3590 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f28e8070e20 tx=0x7f28dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.197+0000 7f28cf7fe700 1 -- 172.21.15.89:0/2184275947 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc00bbe0 con 0x7f28e8159c60 2023-10-10T20:39:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.197+0000 7f28cf7fe700 1 -- 172.21.15.89:0/2184275947 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc011420 con 0x7f28e8159c60 2023-10-10T20:39:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.197+0000 7f28cf7fe700 1 -- 172.21.15.89:0/2184275947 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc0117e0 con 0x7f28e8159c60 2023-10-10T20:39:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.197+0000 7f28edac0700 1 -- 172.21.15.89:0/2184275947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e80cdc90 con 0x7f28e8159c60 2023-10-10T20:39:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.197+0000 7f28edac0700 1 -- 172.21.15.89:0/2184275947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e80ce250 con 0x7f28e8159c60 2023-10-10T20:39:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.198+0000 7f28edac0700 1 -- 172.21.15.89:0/2184275947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e80ce440 con 0x7f28e8159c60 2023-10-10T20:39:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.198+0000 7f28cf7fe700 1 -- 172.21.15.89:0/2184275947 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f28dc011c10 con 0x7f28e8159c60 2023-10-10T20:39:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.198+0000 7f28cf7fe700 1 --2- 172.21.15.89:0/2184275947 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f28d4067170 0x7f28d4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.198+0000 7f28cf7fe700 1 -- 172.21.15.89:0/2184275947 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f28dc085dd0 con 0x7f28e8159c60 2023-10-10T20:39:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.198+0000 7f28e77fe700 1 --2- 172.21.15.89:0/2184275947 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f28d4067170 0x7f28d4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.199+0000 7f28e77fe700 1 --2- 172.21.15.89:0/2184275947 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f28d4067170 0x7f28d4069620 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f28d8003a80 tx=0x7f28d8007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.202+0000 7f28cf7fe700 1 -- 172.21.15.89:0/2184275947 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28dc0530b0 con 0x7f28e8159c60 2023-10-10T20:39:33.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:33.404+0000 7f28edac0700 1 -- 172.21.15.89:0/2184275947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f28e804c3c0 con 0x7f28e8159c60 2023-10-10T20:39:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.229+0000 7f28cf7fe700 1 -- 172.21.15.89:0/2184275947 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v72) v1 ==== 97+0+0 (secure 0 0 0) 0x7f28dc0566d0 con 0x7f28e8159c60 2023-10-10T20:39:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.231+0000 7f28edac0700 1 -- 172.21.15.89:0/2184275947 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f28d4067170 msgr2=0x7f28d4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:34.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.231+0000 7f28edac0700 1 --2- 172.21.15.89:0/2184275947 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f28d4067170 0x7f28d4069620 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f28d8003a80 tx=0x7f28d8007ec0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.231+0000 7f28edac0700 1 -- 172.21.15.89:0/2184275947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28e8159c60 msgr2=0x7f28e80b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:34.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.231+0000 7f28edac0700 1 --2- 172.21.15.89:0/2184275947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28e8159c60 0x7f28e80b3590 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f28e8070e20 tx=0x7f28dc017000 comp rx=0 tx=0).stop 2023-10-10T20:39:34.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.231+0000 7f28edac0700 1 -- 172.21.15.89:0/2184275947 shutdown_connections 2023-10-10T20:39:34.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.231+0000 7f28edac0700 1 --2- 172.21.15.89:0/2184275947 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f28d4067170 0x7f28d4069620 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.231+0000 7f28edac0700 1 --2- 172.21.15.89:0/2184275947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28e815db10 0x7f28e80b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.231+0000 7f28edac0700 1 --2- 172.21.15.89:0/2184275947 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28e8159890 0x7f28e8140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.231+0000 7f28edac0700 1 --2- 172.21.15.89:0/2184275947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28e8159c60 0x7f28e80b3590 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.231+0000 7f28edac0700 1 -- 172.21.15.89:0/2184275947 >> 172.21.15.89:0/2184275947 conn(0x7f28e8068e20 msgr2=0x7f28e8157230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.231+0000 7f28edac0700 1 -- 172.21.15.89:0/2184275947 shutdown_connections 2023-10-10T20:39:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.231+0000 7f28edac0700 1 -- 172.21.15.89:0/2184275947 wait complete. 2023-10-10T20:39:34.236 INFO:teuthology.orchestra.run.smithi089.stderr: removed all blocklist entries 2023-10-10T20:39:34.246 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:39:34.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.360+0000 7f5fc7a3e700 1 Processor -- start 2023-10-10T20:39:34.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.360+0000 7f5fc7a3e700 1 -- start start 2023-10-10T20:39:34.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.361+0000 7f5fc7a3e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5fc0159c60 0x7f5fc015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:34.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.361+0000 7f5fc7a3e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5fc015db10 0x7f5fc0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:34.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.361+0000 7f5fc7a3e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc015a160 con 0x7f5fc0159890 2023-10-10T20:39:34.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.361+0000 7f5fc7a3e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc00d13a0 con 0x7f5fc015db10 2023-10-10T20:39:34.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.361+0000 7f5fc7a3e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc015e140 con 0x7f5fc0159c60 2023-10-10T20:39:34.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.361+0000 7f5fc5fdb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5fc015db10 0x7f5fc0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:34.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.361+0000 7f5fc5fdb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5fc015db10 0x7f5fc0069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50878/0 (socket says 172.21.15.89:50878) 2023-10-10T20:39:34.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.361+0000 7f5fc5fdb700 1 -- 172.21.15.89:0/783298795 learned_addr learned my addr 172.21.15.89:0/783298795 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:34.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.361+0000 7f5fc4fd9700 1 --2- 172.21.15.89:0/783298795 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5fc0159c60 0x7f5fc015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:34.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.362+0000 7f5fc5fdb700 1 -- 172.21.15.89:0/783298795 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5fc0159c60 msgr2=0x7f5fc015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:34.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.362+0000 7f5fc5fdb700 1 --2- 172.21.15.89:0/783298795 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5fc0159c60 0x7f5fc015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.362+0000 7f5fc5fdb700 1 -- 172.21.15.89:0/783298795 >> v1:172.21.15.111:6789/0 conn(0x7f5fc0159890 legacy=0x7f5fc015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:34.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.362+0000 7f5fc5fdb700 1 -- 172.21.15.89:0/783298795 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc015a3d0 con 0x7f5fc015db10 2023-10-10T20:39:34.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.362+0000 7f5fc5fdb700 1 --2- 172.21.15.89:0/783298795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc015db10 0x7f5fc0069560 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80099b0 tx=0x7f5fb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c71a891c230fa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:34.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.362+0000 7f5fb7fff700 1 -- 172.21.15.89:0/783298795 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8009ea0 con 0x7f5fc015db10 2023-10-10T20:39:34.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.362+0000 7f5fb7fff700 1 -- 172.21.15.89:0/783298795 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb800b860 con 0x7f5fc015db10 2023-10-10T20:39:34.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.362+0000 7f5fb7fff700 1 -- 172.21.15.89:0/783298795 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb800bc00 con 0x7f5fc015db10 2023-10-10T20:39:34.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.362+0000 7f5fc7a3e700 1 -- 172.21.15.89:0/783298795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc015db10 msgr2=0x7f5fc0069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:34.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.362+0000 7f5fc7a3e700 1 --2- 172.21.15.89:0/783298795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc015db10 0x7f5fc0069560 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80099b0 tx=0x7f5fb8009250 comp rx=0 tx=0).stop 2023-10-10T20:39:34.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.362+0000 7f5fc7a3e700 1 -- 172.21.15.89:0/783298795 shutdown_connections 2023-10-10T20:39:34.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.362+0000 7f5fc7a3e700 1 --2- 172.21.15.89:0/783298795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc015db10 0x7f5fc0069560 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.362+0000 7f5fc7a3e700 1 --2- 172.21.15.89:0/783298795 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5fc0159c60 0x7f5fc015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.362+0000 7f5fc7a3e700 1 -- 172.21.15.89:0/783298795 >> 172.21.15.89:0/783298795 conn(0x7f5fc0068e20 msgr2=0x7f5fc00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:34.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.363+0000 7f5fc7a3e700 1 -- 172.21.15.89:0/783298795 shutdown_connections 2023-10-10T20:39:34.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.363+0000 7f5fc7a3e700 1 -- 172.21.15.89:0/783298795 wait complete. 2023-10-10T20:39:34.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.364+0000 7f5fc7a3e700 1 Processor -- start 2023-10-10T20:39:34.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.364+0000 7f5fc7a3e700 1 -- start start 2023-10-10T20:39:34.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.364+0000 7f5fc7a3e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fc0159890 0x7f5fc00ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:34.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fc7a3e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc0159c60 0x7f5fc014be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:34.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fc7a3e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5fc015db10 0x7f5fc015cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:34.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fc7a3e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00ce550 con 0x7f5fc0159c60 2023-10-10T20:39:34.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fc7a3e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc015ba40 con 0x7f5fc0159890 2023-10-10T20:39:34.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fc7a3e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc015bd20 con 0x7f5fc015db10 2023-10-10T20:39:34.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fc4fd9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc0159c60 0x7f5fc014be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:34.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fc4fd9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc0159c60 0x7f5fc014be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50884/0 (socket says 172.21.15.89:50884) 2023-10-10T20:39:34.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fc4fd9700 1 -- 172.21.15.89:0/1616537001 learned_addr learned my addr 172.21.15.89:0/1616537001 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:34.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fc5fdb700 1 --2- 172.21.15.89:0/1616537001 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5fc015db10 0x7f5fc015cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:34.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fc57da700 1 --2- 172.21.15.89:0/1616537001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fc0159890 0x7f5fc00ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:34.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fc4fd9700 1 -- 172.21.15.89:0/1616537001 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5fc015db10 msgr2=0x7f5fc015cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:34.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fc4fd9700 1 --2- 172.21.15.89:0/1616537001 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5fc015db10 0x7f5fc015cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fc4fd9700 1 -- 172.21.15.89:0/1616537001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fc0159890 msgr2=0x7f5fc00ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:34.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fc4fd9700 1 --2- 172.21.15.89:0/1616537001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fc0159890 0x7f5fc00ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fc4fd9700 1 -- 172.21.15.89:0/1616537001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb80094f0 con 0x7f5fc0159c60 2023-10-10T20:39:34.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fc4fd9700 1 --2- 172.21.15.89:0/1616537001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc0159c60 0x7f5fc014be70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000ec50 tx=0x7f5fb00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:34.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fb57fa700 1 -- 172.21.15.89:0/1616537001 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb000c850 con 0x7f5fc0159c60 2023-10-10T20:39:34.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fb57fa700 1 -- 172.21.15.89:0/1616537001 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb000c9b0 con 0x7f5fc0159c60 2023-10-10T20:39:34.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fc7a3e700 1 -- 172.21.15.89:0/1616537001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc015c000 con 0x7f5fc0159c60 2023-10-10T20:39:34.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fb57fa700 1 -- 172.21.15.89:0/1616537001 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb000cd70 con 0x7f5fc0159c60 2023-10-10T20:39:34.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.365+0000 7f5fc7a3e700 1 -- 172.21.15.89:0/1616537001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc015c5c0 con 0x7f5fc0159c60 2023-10-10T20:39:34.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.366+0000 7f5fc7a3e700 1 -- 172.21.15.89:0/1616537001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc004c3c0 con 0x7f5fc0159c60 2023-10-10T20:39:34.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.367+0000 7f5fb57fa700 1 -- 172.21.15.89:0/1616537001 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f5fb0013680 con 0x7f5fc0159c60 2023-10-10T20:39:34.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.367+0000 7f5fb57fa700 1 --2- 172.21.15.89:0/1616537001 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5fa4067170 0x7f5fa4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:34.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.367+0000 7f5fb57fa700 1 -- 172.21.15.89:0/1616537001 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5fb0084740 con 0x7f5fc0159c60 2023-10-10T20:39:34.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.367+0000 7f5fc57da700 1 --2- 172.21.15.89:0/1616537001 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5fa4067170 0x7f5fa4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:34.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.368+0000 7f5fc57da700 1 --2- 172.21.15.89:0/1616537001 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5fa4067170 0x7f5fa4069620 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc007fd0 tx=0x7f5fbc0016f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:34.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.371+0000 7f5fb57fa700 1 -- 172.21.15.89:0/1616537001 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5fb0051a70 con 0x7f5fc0159c60 2023-10-10T20:39:34.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.632+0000 7f5fc7a3e700 1 -- 172.21.15.89:0/1616537001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5fc006a5b0 con 0x7f5fc0159c60 2023-10-10T20:39:34.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.632+0000 7f5fb57fa700 1 -- 172.21.15.89:0/1616537001 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5fb0055090 con 0x7f5fc0159c60 2023-10-10T20:39:34.635 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:34.635 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:39:34.636 INFO:teuthology.orchestra.run.smithi089.stdout: "auth_dump": [ 2023-10-10T20:39:34.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:34.636 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.a", 2023-10-10T20:39:34.636 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlZLSNLhAA8WZTrsht6tAOP+gxaTXKMg==", 2023-10-10T20:39:34.637 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:39:34.637 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:39:34.637 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:39:34.637 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:39:34.638 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:39:34.638 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:34.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:34.639 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.b", 2023-10-10T20:39:34.639 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgyDEORAAMMoSxPFZc0bCIwWy8UsXgA==", 2023-10-10T20:39:34.639 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:39:34.639 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:39:34.640 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:39:34.640 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:39:34.640 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:39:34.640 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:34.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:34.641 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.c", 2023-10-10T20:39:34.641 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlMFxSNRAASuQEJFghLIOkTQWa4ckTvg==", 2023-10-10T20:39:34.641 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:39:34.642 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:39:34.642 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:39:34.642 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:39:34.642 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:39:34.642 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:34.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:34.643 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.d", 2023-10-10T20:39:34.643 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlPkvrBBAA2VLJmv2t8szTEGtCVwalJw==", 2023-10-10T20:39:34.643 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:39:34.644 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:39:34.644 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:39:34.644 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:39:34.644 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:39:34.644 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:34.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:34.645 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.0", 2023-10-10T20:39:34.645 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQArtSVltOcKHRAAk8+zUus04oTVI9KvfhjlXg==", 2023-10-10T20:39:34.645 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:39:34.645 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:39:34.646 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:39:34.646 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:39:34.646 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:34.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:34.647 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.1", 2023-10-10T20:39:34.647 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAutSVly8P0GhAAe1bvR63+ptJg84lizMNdQw==", 2023-10-10T20:39:34.647 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:39:34.647 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:39:34.647 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:39:34.648 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:39:34.648 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:34.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:34.648 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.2", 2023-10-10T20:39:34.649 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAxtSVl0x9fGRAAEKi7+6EW5mzPbebmz0+sgQ==", 2023-10-10T20:39:34.649 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:39:34.649 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:39:34.649 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:39:34.649 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:39:34.649 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:34.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:34.650 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.3", 2023-10-10T20:39:34.650 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA0tSVlfJyaGBAAa+8w54U6v7+npbdufKJruA==", 2023-10-10T20:39:34.650 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:39:34.651 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:39:34.651 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:39:34.651 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:39:34.651 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:34.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:34.652 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.4", 2023-10-10T20:39:34.652 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA6tSVlEQ/QFxAA/PRB3c90BuAlF6xekGfX9A==", 2023-10-10T20:39:34.652 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:39:34.652 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:39:34.652 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:39:34.653 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:39:34.653 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:34.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:34.653 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.5", 2023-10-10T20:39:34.653 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA9tSVlMOKrFRAAuBnIsEz2VmeSZi9XWzjlzA==", 2023-10-10T20:39:34.654 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:39:34.654 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:39:34.654 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:39:34.654 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:39:34.654 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:34.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:34.655 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.6", 2023-10-10T20:39:34.655 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBAtSVlH7PQFBAAHr6c7d4TYJEu/KSFEf+EOA==", 2023-10-10T20:39:34.655 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:39:34.656 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:39:34.656 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:39:34.656 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:39:34.656 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:34.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:34.657 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.7", 2023-10-10T20:39:34.657 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBDtSVlAhC1FBAAuE2BwAA/f7VOkZfVdqAG7A==", 2023-10-10T20:39:34.657 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:39:34.657 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:39:34.657 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:39:34.658 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:39:34.658 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:34.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:34.658 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.0", 2023-10-10T20:39:34.659 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlRd3HCBAAb0QP2VfTS7aelno1mFpGAA==", 2023-10-10T20:39:34.659 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:39:34.659 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:39:34.659 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow r", 2023-10-10T20:39:34.659 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:39:34.660 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:34.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:34.660 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.1", 2023-10-10T20:39:34.660 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlgBouERAAqPdE0KvJU3M53B6Y6AH5GQ==", 2023-10-10T20:39:34.661 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:39:34.661 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:39:34.661 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:39:34.661 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:39:34.661 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:39:34.661 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:34.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:34.662 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.2", 2023-10-10T20:39:34.663 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlXA4tDhAAoPVYQjECpzD+C67mnsY5IA==", 2023-10-10T20:39:34.663 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:39:34.663 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:39:34.663 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:39:34.663 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:39:34.663 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:39:34.664 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:34.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:34.664 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.3", 2023-10-10T20:39:34.664 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlZ5KGFhAAuKmZwbOfmmw1nAj5ry6tHw==", 2023-10-10T20:39:34.665 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:39:34.665 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:39:34.665 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:39:34.666 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:39:34.666 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:39:34.666 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:34.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:34.667 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.admin", 2023-10-10T20:39:34.667 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgpL2EhAAoDOki002rfBrauTgfu481Q==", 2023-10-10T20:39:34.667 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:39:34.667 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:39:34.668 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:39:34.668 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:39:34.668 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:39:34.668 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:34.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:34.669 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.x", 2023-10-10T20:39:34.669 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlBhaJKxAArraBxT5xWg515LfpJ7MlbA==", 2023-10-10T20:39:34.669 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:39:34.670 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:39:34.670 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:39:34.670 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:39:34.670 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:34.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:34.671 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.y", 2023-10-10T20:39:34.671 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlTZIyJxAAWAOsitHwsIxJWwp3b1Z98Q==", 2023-10-10T20:39:34.672 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:39:34.672 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:39:34.672 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:39:34.672 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:39:34.673 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.673 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:34.673 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-10T20:39:34.673 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:39:34.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.634+0000 7f5fc7a3e700 1 -- 172.21.15.89:0/1616537001 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5fa4067170 msgr2=0x7f5fa4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:34.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.634+0000 7f5fc7a3e700 1 --2- 172.21.15.89:0/1616537001 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5fa4067170 0x7f5fa4069620 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc007fd0 tx=0x7f5fbc0016f0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.634+0000 7f5fc7a3e700 1 -- 172.21.15.89:0/1616537001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc0159c60 msgr2=0x7f5fc014be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:34.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.634+0000 7f5fc7a3e700 1 --2- 172.21.15.89:0/1616537001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc0159c60 0x7f5fc014be70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000ec50 tx=0x7f5fb00091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.634+0000 7f5fc7a3e700 1 -- 172.21.15.89:0/1616537001 shutdown_connections 2023-10-10T20:39:34.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.634+0000 7f5fc7a3e700 1 --2- 172.21.15.89:0/1616537001 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5fc015db10 0x7f5fc015cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.634+0000 7f5fc7a3e700 1 --2- 172.21.15.89:0/1616537001 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5fa4067170 0x7f5fa4069620 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.634+0000 7f5fc7a3e700 1 --2- 172.21.15.89:0/1616537001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fc0159890 0x7f5fc00ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.634+0000 7f5fc7a3e700 1 --2- 172.21.15.89:0/1616537001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc0159c60 0x7f5fc014be70 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.635+0000 7f5fc7a3e700 1 -- 172.21.15.89:0/1616537001 >> 172.21.15.89:0/1616537001 conn(0x7f5fc0068e20 msgr2=0x7f5fc014d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:34.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.635+0000 7f5fc7a3e700 1 -- 172.21.15.89:0/1616537001 shutdown_connections 2023-10-10T20:39:34.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.635+0000 7f5fc7a3e700 1 -- 172.21.15.89:0/1616537001 wait complete. 2023-10-10T20:39:34.678 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:39:34.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.791+0000 7fefb3f5c700 1 Processor -- start 2023-10-10T20:39:34.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.791+0000 7fefb3f5c700 1 -- start start 2023-10-10T20:39:34.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.792+0000 7fefb3f5c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fefac068550 0x7fefac068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:34.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.792+0000 7fefb3f5c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefac06ec70 con 0x7fefac15e010 2023-10-10T20:39:34.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.792+0000 7fefb3f5c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefac0b2e70 con 0x7fefac06f810 2023-10-10T20:39:34.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.792+0000 7fefb3f5c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefac0b3430 con 0x7fefac068550 2023-10-10T20:39:34.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.792+0000 7fefb14f7700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fefac15e010 0x7fefac06f100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38936/0 (socket says 172.21.15.89:38936) 2023-10-10T20:39:34.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.792+0000 7fefb14f7700 1 -- 172.21.15.89:0/1631164210 learned_addr learned my addr 172.21.15.89:0/1631164210 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:34.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.792+0000 7fefb1cf8700 1 --2- 172.21.15.89:0/1631164210 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fefac068550 0x7fefac068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:34.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.792+0000 7fefb0cf6700 1 -- 172.21.15.89:0/1631164210 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307596919 0 0) 0x7fefac06ec70 con 0x7fefac15e010 2023-10-10T20:39:34.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb0cf6700 1 -- 172.21.15.89:0/1631164210 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef98003570 con 0x7fefac15e010 2023-10-10T20:39:34.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb0cf6700 1 -- 172.21.15.89:0/1631164210 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3652406518 0 0) 0x7fefac0b2e70 con 0x7fefac06f810 2023-10-10T20:39:34.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb0cf6700 1 -- 172.21.15.89:0/1631164210 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefac06ec70 con 0x7fefac06f810 2023-10-10T20:39:34.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb0cf6700 1 -- 172.21.15.89:0/1631164210 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3191107225 0 0) 0x7fef98003570 con 0x7fefac15e010 2023-10-10T20:39:34.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb0cf6700 1 -- 172.21.15.89:0/1631164210 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fefac068550 msgr2=0x7fefac068920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:34.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb0cf6700 1 --2- 172.21.15.89:0/1631164210 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fefac068550 0x7fefac068920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb0cf6700 1 -- 172.21.15.89:0/1631164210 >> v1:172.21.15.111:6789/0 conn(0x7fefac06f810 legacy=0x7fefac159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:34.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb0cf6700 1 -- 172.21.15.89:0/1631164210 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac0b3150 con 0x7fefac15e010 2023-10-10T20:39:34.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb0cf6700 1 -- 172.21.15.89:0/1631164210 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fef9c001470 con 0x7fefac15e010 2023-10-10T20:39:34.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb0cf6700 1 -- 172.21.15.89:0/1631164210 >> v1:172.21.15.89:6789/0 conn(0x7fefac15e010 legacy=0x7fefac06f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:34.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb0cf6700 1 --2- 172.21.15.89:0/1631164210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef98003c20 0x7fef98004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:34.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb0cf6700 1 --2- 172.21.15.89:0/1631164210 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fefac068550 0x7fef98004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:34.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb0cf6700 1 --2- 172.21.15.89:0/1631164210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef98004d50 0x7fef980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:34.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb0cf6700 1 -- 172.21.15.89:0/1631164210 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9800bb40 con 0x7fef98003c20 2023-10-10T20:39:34.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb0cf6700 1 -- 172.21.15.89:0/1631164210 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9800bc80 con 0x7fef98004d50 2023-10-10T20:39:34.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb0cf6700 1 -- 172.21.15.89:0/1631164210 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9800be40 con 0x7fefac068550 2023-10-10T20:39:34.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb1cf8700 1 --2- 172.21.15.89:0/1631164210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef98003c20 0x7fef98004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:34.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb14f7700 1 --2- 172.21.15.89:0/1631164210 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fefac068550 0x7fef98004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:34.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb24f9700 1 --2- 172.21.15.89:0/1631164210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef98004d50 0x7fef980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:34.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb1cf8700 1 -- 172.21.15.89:0/1631164210 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fefac068550 msgr2=0x7fef98004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:34.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb1cf8700 1 --2- 172.21.15.89:0/1631164210 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fefac068550 0x7fef98004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb1cf8700 1 -- 172.21.15.89:0/1631164210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef98004d50 msgr2=0x7fef980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:34.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb1cf8700 1 --2- 172.21.15.89:0/1631164210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef98004d50 0x7fef980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb1cf8700 1 -- 172.21.15.89:0/1631164210 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef98003570 con 0x7fef98003c20 2023-10-10T20:39:34.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb14f7700 1 --2- 172.21.15.89:0/1631164210 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fefac068550 0x7fef98004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:34.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb24f9700 1 --2- 172.21.15.89:0/1631164210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef98004d50 0x7fef980051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:34.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb1cf8700 1 --2- 172.21.15.89:0/1631164210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef98003c20 0x7fef98004010 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fefa800e210 tx=0x7fefa800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb0fbac2ff5dcc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:34.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.793+0000 7fefb0cf6700 1 -- 172.21.15.89:0/1631164210 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa800eac0 con 0x7fef98003c20 2023-10-10T20:39:34.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.794+0000 7fefb0cf6700 1 -- 172.21.15.89:0/1631164210 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa800f040 con 0x7fef98003c20 2023-10-10T20:39:34.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.794+0000 7fefb0cf6700 1 -- 172.21.15.89:0/1631164210 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa800ee70 con 0x7fef98003c20 2023-10-10T20:39:34.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.794+0000 7fefb3f5c700 1 -- 172.21.15.89:0/1631164210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef98003c20 msgr2=0x7fef98004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:34.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.794+0000 7fefb3f5c700 1 --2- 172.21.15.89:0/1631164210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef98003c20 0x7fef98004010 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fefa800e210 tx=0x7fefa800b040 comp rx=0 tx=0).stop 2023-10-10T20:39:34.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.794+0000 7fefb24f9700 1 -- 172.21.15.89:0/1631164210 reap_dead start 2023-10-10T20:39:34.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.794+0000 7fefb3f5c700 1 -- 172.21.15.89:0/1631164210 shutdown_connections 2023-10-10T20:39:34.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.794+0000 7fefb3f5c700 1 -- 172.21.15.89:0/1631164210 >> 172.21.15.89:0/1631164210 conn(0x7fefac0cd480 msgr2=0x7fefac0cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:34.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.794+0000 7fefb3f5c700 1 -- 172.21.15.89:0/1631164210 shutdown_connections 2023-10-10T20:39:34.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.794+0000 7fefb3f5c700 1 -- 172.21.15.89:0/1631164210 wait complete. 2023-10-10T20:39:34.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.795+0000 7fefb3f5c700 1 Processor -- start 2023-10-10T20:39:34.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.795+0000 7fefb3f5c700 1 -- start start 2023-10-10T20:39:34.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.796+0000 7fefb3f5c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fef98003c20 0x7fefac16adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:34.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.796+0000 7fefb3f5c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefac15a6b0 0x7fefac0cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:34.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.796+0000 7fefb3f5c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fefac15c6b0 0x7fefac15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:34.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.796+0000 7fefb3f5c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac15a070 con 0x7fefac15a6b0 2023-10-10T20:39:34.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.796+0000 7fefb3f5c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac15d060 con 0x7fefac15c6b0 2023-10-10T20:39:34.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.796+0000 7fefb3f5c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefac16b2f0 con 0x7fef98003c20 2023-10-10T20:39:34.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.796+0000 7fefb14f7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefac15a6b0 0x7fefac0cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:34.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.796+0000 7fefb14f7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefac15a6b0 0x7fefac0cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50904/0 (socket says 172.21.15.89:50904) 2023-10-10T20:39:34.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.796+0000 7fefb14f7700 1 -- 172.21.15.89:0/2061558019 learned_addr learned my addr 172.21.15.89:0/2061558019 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:34.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.796+0000 7fefb1cf8700 1 --2- 172.21.15.89:0/2061558019 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fef98003c20 0x7fefac16adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:34.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.796+0000 7fefb24f9700 1 --2- 172.21.15.89:0/2061558019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fefac15c6b0 0x7fefac15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:34.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.796+0000 7fefb14f7700 1 -- 172.21.15.89:0/2061558019 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fef98003c20 msgr2=0x7fefac16adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:34.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.796+0000 7fefb14f7700 1 --2- 172.21.15.89:0/2061558019 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fef98003c20 0x7fefac16adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.796+0000 7fefb14f7700 1 -- 172.21.15.89:0/2061558019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fefac15c6b0 msgr2=0x7fefac15cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:34.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.796+0000 7fefb14f7700 1 --2- 172.21.15.89:0/2061558019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fefac15c6b0 0x7fefac15cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.796+0000 7fefb14f7700 1 -- 172.21.15.89:0/2061558019 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac15abd0 con 0x7fefac15a6b0 2023-10-10T20:39:34.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.796+0000 7fefb1cf8700 1 --2- 172.21.15.89:0/2061558019 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fef98003c20 0x7fefac16adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:34.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.796+0000 7fefb24f9700 1 --2- 172.21.15.89:0/2061558019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fefac15c6b0 0x7fefac15cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:34.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.796+0000 7fefb14f7700 1 --2- 172.21.15.89:0/2061558019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefac15a6b0 0x7fefac0cccc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00ebc0 tx=0x7fef9c0010f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:34.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.797+0000 7fefa1ffb700 1 -- 172.21.15.89:0/2061558019 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00c700 con 0x7fefac15a6b0 2023-10-10T20:39:34.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.797+0000 7fefa1ffb700 1 -- 172.21.15.89:0/2061558019 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9c00c860 con 0x7fefac15a6b0 2023-10-10T20:39:34.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.797+0000 7fefb3f5c700 1 -- 172.21.15.89:0/2061558019 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefac15aeb0 con 0x7fefac15a6b0 2023-10-10T20:39:34.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.797+0000 7fefa1ffb700 1 -- 172.21.15.89:0/2061558019 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00cc00 con 0x7fefac15a6b0 2023-10-10T20:39:34.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.797+0000 7fefb3f5c700 1 -- 172.21.15.89:0/2061558019 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefac15b470 con 0x7fefac15a6b0 2023-10-10T20:39:34.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.798+0000 7fefb3f5c700 1 -- 172.21.15.89:0/2061558019 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefac06c4c0 con 0x7fefac15a6b0 2023-10-10T20:39:34.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.798+0000 7fefa1ffb700 1 -- 172.21.15.89:0/2061558019 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fef9c01d460 con 0x7fefac15a6b0 2023-10-10T20:39:34.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.798+0000 7fefa1ffb700 1 --2- 172.21.15.89:0/2061558019 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fef90067170 0x7fef90069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:34.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.798+0000 7fefa1ffb700 1 -- 172.21.15.89:0/2061558019 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fef9c0844b0 con 0x7fefac15a6b0 2023-10-10T20:39:34.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.798+0000 7fefb1cf8700 1 --2- 172.21.15.89:0/2061558019 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fef90067170 0x7fef90069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:34.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.799+0000 7fefb1cf8700 1 --2- 172.21.15.89:0/2061558019 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fef90067170 0x7fef90069620 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fefa800afd0 tx=0x7fefa801a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:34.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.803+0000 7fefa1ffb700 1 -- 172.21.15.89:0/2061558019 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef9c0517e0 con 0x7fefac15a6b0 2023-10-10T20:39:34.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.988+0000 7fefb3f5c700 1 -- 172.21.15.89:0/2061558019 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefac04c3c0 con 0x7fefac15a6b0 2023-10-10T20:39:34.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.989+0000 7fefa1ffb700 1 -- 172.21.15.89:0/2061558019 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+12135 (secure 0 0 0) 0x7fef9c054e00 con 0x7fefac15a6b0 2023-10-10T20:39:34.991 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:34.992 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":72,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:39:34.225302+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:39:34.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.991+0000 7fefb3f5c700 1 -- 172.21.15.89:0/2061558019 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fef90067170 msgr2=0x7fef90069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:34.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.991+0000 7fefb3f5c700 1 --2- 172.21.15.89:0/2061558019 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fef90067170 0x7fef90069620 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fefa800afd0 tx=0x7fefa801a000 comp rx=0 tx=0).stop 2023-10-10T20:39:34.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.991+0000 7fefb3f5c700 1 -- 172.21.15.89:0/2061558019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefac15a6b0 msgr2=0x7fefac0cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:34.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.991+0000 7fefb3f5c700 1 --2- 172.21.15.89:0/2061558019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefac15a6b0 0x7fefac0cccc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00ebc0 tx=0x7fef9c0010f0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.991+0000 7fefb3f5c700 1 -- 172.21.15.89:0/2061558019 shutdown_connections 2023-10-10T20:39:34.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.991+0000 7fefb3f5c700 1 --2- 172.21.15.89:0/2061558019 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fef90067170 0x7fef90069620 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.991+0000 7fefb3f5c700 1 --2- 172.21.15.89:0/2061558019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fefac15c6b0 0x7fefac15cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.991+0000 7fefb3f5c700 1 --2- 172.21.15.89:0/2061558019 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fef98003c20 0x7fefac16adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.991+0000 7fefb3f5c700 1 --2- 172.21.15.89:0/2061558019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefac15a6b0 0x7fefac0cccc0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:34.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.991+0000 7fefb3f5c700 1 -- 172.21.15.89:0/2061558019 >> 172.21.15.89:0/2061558019 conn(0x7fefac0cd480 msgr2=0x7fefac06f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:34.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.991+0000 7fefb3f5c700 1 -- 172.21.15.89:0/2061558019 shutdown_connections 2023-10-10T20:39:34.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:34.991+0000 7fefb3f5c700 1 -- 172.21.15.89:0/2061558019 wait complete. 2023-10-10T20:39:35.006 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:39:35.006 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:39:35.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.123+0000 7fb47d9ec700 1 Processor -- start 2023-10-10T20:39:35.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.123+0000 7fb47d9ec700 1 -- start start 2023-10-10T20:39:35.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.124+0000 7fb47d9ec700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4780cc600 0x7fb4780b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.124+0000 7fb47d9ec700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb4780cc9d0 0x7fb47815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.124+0000 7fb47d9ec700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb478150b50 con 0x7fb4780cd0f0 2023-10-10T20:39:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.124+0000 7fb47d9ec700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4780cede0 con 0x7fb4780cc9d0 2023-10-10T20:39:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.124+0000 7fb47d9ec700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780b2260 con 0x7fb4780cc600 2023-10-10T20:39:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.124+0000 7fb47d1eb700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb4780cd0f0 0x7fb478159c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38946/0 (socket says 172.21.15.89:38946) 2023-10-10T20:39:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.124+0000 7fb47d1eb700 1 -- 172.21.15.89:0/1970925774 learned_addr learned my addr 172.21.15.89:0/1970925774 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:35.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.124+0000 7fb46f7fe700 1 -- 172.21.15.89:0/1970925774 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3248430645 0 0) 0x7fb478150b50 con 0x7fb4780cd0f0 2023-10-10T20:39:35.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.124+0000 7fb46f7fe700 1 -- 172.21.15.89:0/1970925774 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb460003570 con 0x7fb4780cd0f0 2023-10-10T20:39:35.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.124+0000 7fb46ffff700 1 --2- 172.21.15.89:0/1970925774 >> v2:172.21.15.111:3300/0 conn(0x7fb4780cc9d0 0x7fb47815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.124+0000 7fb47c9ea700 1 --2- 172.21.15.89:0/1970925774 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4780cc600 0x7fb4780b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb46f7fe700 1 -- 172.21.15.89:0/1970925774 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 205342529 0 0) 0x7fb460003570 con 0x7fb4780cd0f0 2023-10-10T20:39:35.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb46f7fe700 1 -- 172.21.15.89:0/1970925774 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4780cc600 msgr2=0x7fb4780b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:35.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb46f7fe700 1 --2- 172.21.15.89:0/1970925774 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4780cc600 0x7fb4780b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb46f7fe700 1 -- 172.21.15.89:0/1970925774 >> v2:172.21.15.111:3300/0 conn(0x7fb4780cc9d0 msgr2=0x7fb47815d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:35.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb46f7fe700 1 --2- 172.21.15.89:0/1970925774 >> v2:172.21.15.111:3300/0 conn(0x7fb4780cc9d0 0x7fb47815d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb46f7fe700 1 -- 172.21.15.89:0/1970925774 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4780b2540 con 0x7fb4780cd0f0 2023-10-10T20:39:35.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb46f7fe700 1 -- 172.21.15.89:0/1970925774 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fb470001470 con 0x7fb4780cd0f0 2023-10-10T20:39:35.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb46f7fe700 1 -- 172.21.15.89:0/1970925774 >> v1:172.21.15.89:6789/0 conn(0x7fb4780cd0f0 legacy=0x7fb478159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:35.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb46f7fe700 1 --2- 172.21.15.89:0/1970925774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb460003a60 0x7fb460003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb46f7fe700 1 --2- 172.21.15.89:0/1970925774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb460004810 0x7fb460004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb46f7fe700 1 --2- 172.21.15.89:0/1970925774 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4780cc600 0x7fb4600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb46f7fe700 1 -- 172.21.15.89:0/1970925774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780cede0 con 0x7fb460003a60 2023-10-10T20:39:35.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb46f7fe700 1 -- 172.21.15.89:0/1970925774 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780b2260 con 0x7fb460004810 2023-10-10T20:39:35.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb46f7fe700 1 -- 172.21.15.89:0/1970925774 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46000ba60 con 0x7fb4780cc600 2023-10-10T20:39:35.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb47c9ea700 1 --2- 172.21.15.89:0/1970925774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb460003a60 0x7fb460003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb47d1eb700 1 --2- 172.21.15.89:0/1970925774 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4780cc600 0x7fb4600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb46ffff700 1 --2- 172.21.15.89:0/1970925774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb460004810 0x7fb460004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb47c9ea700 1 -- 172.21.15.89:0/1970925774 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4780cc600 msgr2=0x7fb4600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:35.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb47c9ea700 1 --2- 172.21.15.89:0/1970925774 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4780cc600 0x7fb4600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb47c9ea700 1 -- 172.21.15.89:0/1970925774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb460004810 msgr2=0x7fb460004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:35.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb47c9ea700 1 --2- 172.21.15.89:0/1970925774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb460004810 0x7fb460004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb47c9ea700 1 -- 172.21.15.89:0/1970925774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb460003570 con 0x7fb460003a60 2023-10-10T20:39:35.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb47d1eb700 1 --2- 172.21.15.89:0/1970925774 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb4780cc600 0x7fb4600051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:35.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb46ffff700 1 --2- 172.21.15.89:0/1970925774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb460004810 0x7fb460004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:35.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb47c9ea700 1 --2- 172.21.15.89:0/1970925774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb460003a60 0x7fb460003e70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb47400e240 tx=0x7fb47400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e320b83165c619c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:35.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb46f7fe700 1 -- 172.21.15.89:0/1970925774 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47400eaf0 con 0x7fb460003a60 2023-10-10T20:39:35.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb46f7fe700 1 -- 172.21.15.89:0/1970925774 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47400f040 con 0x7fb460003a60 2023-10-10T20:39:35.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.125+0000 7fb46f7fe700 1 -- 172.21.15.89:0/1970925774 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47400eea0 con 0x7fb460003a60 2023-10-10T20:39:35.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.126+0000 7fb47d9ec700 1 -- 172.21.15.89:0/1970925774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb460003a60 msgr2=0x7fb460003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:35.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.126+0000 7fb47d9ec700 1 --2- 172.21.15.89:0/1970925774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb460003a60 0x7fb460003e70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb47400e240 tx=0x7fb47400b040 comp rx=0 tx=0).stop 2023-10-10T20:39:35.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.126+0000 7fb47d1eb700 1 -- 172.21.15.89:0/1970925774 reap_dead start 2023-10-10T20:39:35.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.126+0000 7fb47d9ec700 1 -- 172.21.15.89:0/1970925774 shutdown_connections 2023-10-10T20:39:35.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.126+0000 7fb47d9ec700 1 -- 172.21.15.89:0/1970925774 >> 172.21.15.89:0/1970925774 conn(0x7fb4780d0630 msgr2=0x7fb4780d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:35.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.126+0000 7fb47d9ec700 1 -- 172.21.15.89:0/1970925774 shutdown_connections 2023-10-10T20:39:35.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.126+0000 7fb47d9ec700 1 -- 172.21.15.89:0/1970925774 wait complete. 2023-10-10T20:39:35.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.127+0000 7fb47d9ec700 1 Processor -- start 2023-10-10T20:39:35.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.127+0000 7fb47d9ec700 1 -- start start 2023-10-10T20:39:35.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.128+0000 7fb47d9ec700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb460003a60 0x7fb478153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.128+0000 7fb47d9ec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4781565f0 0x7fb478156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.128+0000 7fb47d9ec700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4781545a0 0x7fb478154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.128+0000 7fb47d9ec700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb478155f90 con 0x7fb4781565f0 2023-10-10T20:39:35.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.128+0000 7fb47d9ec700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb478156fa0 con 0x7fb4781545a0 2023-10-10T20:39:35.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.128+0000 7fb47d9ec700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb478155310 con 0x7fb460003a60 2023-10-10T20:39:35.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.128+0000 7fb46ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4781565f0 0x7fb478156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.128+0000 7fb46ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4781565f0 0x7fb478156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50922/0 (socket says 172.21.15.89:50922) 2023-10-10T20:39:35.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.128+0000 7fb46ffff700 1 -- 172.21.15.89:0/422040649 learned_addr learned my addr 172.21.15.89:0/422040649 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:35.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.128+0000 7fb47c9ea700 1 --2- 172.21.15.89:0/422040649 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb460003a60 0x7fb478153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.128+0000 7fb47d1eb700 1 --2- 172.21.15.89:0/422040649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4781545a0 0x7fb478154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.128+0000 7fb46ffff700 1 -- 172.21.15.89:0/422040649 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb460003a60 msgr2=0x7fb478153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:35.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.128+0000 7fb46ffff700 1 --2- 172.21.15.89:0/422040649 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb460003a60 0x7fb478153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.128+0000 7fb46ffff700 1 -- 172.21.15.89:0/422040649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4781545a0 msgr2=0x7fb478154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:35.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.128+0000 7fb46ffff700 1 --2- 172.21.15.89:0/422040649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4781545a0 0x7fb478154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.128+0000 7fb46ffff700 1 -- 172.21.15.89:0/422040649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4781555f0 con 0x7fb4781565f0 2023-10-10T20:39:35.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.128+0000 7fb46ffff700 1 --2- 172.21.15.89:0/422040649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4781565f0 0x7fb478156a60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb4680118e0 tx=0x7fb468008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:35.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.129+0000 7fb46cff9700 1 -- 172.21.15.89:0/422040649 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468011e80 con 0x7fb4781565f0 2023-10-10T20:39:35.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.129+0000 7fb46cff9700 1 -- 172.21.15.89:0/422040649 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46800f8e0 con 0x7fb4781565f0 2023-10-10T20:39:35.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.129+0000 7fb47d9ec700 1 -- 172.21.15.89:0/422040649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4781558d0 con 0x7fb4781565f0 2023-10-10T20:39:35.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.129+0000 7fb46cff9700 1 -- 172.21.15.89:0/422040649 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46800fc80 con 0x7fb4781565f0 2023-10-10T20:39:35.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.129+0000 7fb47d9ec700 1 -- 172.21.15.89:0/422040649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb478153f90 con 0x7fb4781565f0 2023-10-10T20:39:35.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.130+0000 7fb47d9ec700 1 -- 172.21.15.89:0/422040649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4780889d0 con 0x7fb4781565f0 2023-10-10T20:39:35.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.130+0000 7fb46cff9700 1 -- 172.21.15.89:0/422040649 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fb468015460 con 0x7fb4781565f0 2023-10-10T20:39:35.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.130+0000 7fb46cff9700 1 --2- 172.21.15.89:0/422040649 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb464067170 0x7fb464069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.131+0000 7fb47c9ea700 1 --2- 172.21.15.89:0/422040649 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb464067170 0x7fb464069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.131+0000 7fb46cff9700 1 -- 172.21.15.89:0/422040649 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb468084d00 con 0x7fb4781565f0 2023-10-10T20:39:35.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.131+0000 7fb47c9ea700 1 --2- 172.21.15.89:0/422040649 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb464067170 0x7fb464069620 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb47400e240 tx=0x7fb47401a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:35.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.135+0000 7fb46cff9700 1 -- 172.21.15.89:0/422040649 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb468052030 con 0x7fb4781565f0 2023-10-10T20:39:35.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.319+0000 7fb47d9ec700 1 -- 172.21.15.89:0/422040649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4780b2820 con 0x7fb4781565f0 2023-10-10T20:39:35.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.319+0000 7fb46cff9700 1 -- 172.21.15.89:0/422040649 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb468055650 con 0x7fb4781565f0 2023-10-10T20:39:35.322 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:39:35.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.321+0000 7fb47d9ec700 1 -- 172.21.15.89:0/422040649 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb464067170 msgr2=0x7fb464069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:35.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.321+0000 7fb47d9ec700 1 --2- 172.21.15.89:0/422040649 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb464067170 0x7fb464069620 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb47400e240 tx=0x7fb47401a000 comp rx=0 tx=0).stop 2023-10-10T20:39:35.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.321+0000 7fb47d9ec700 1 -- 172.21.15.89:0/422040649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4781565f0 msgr2=0x7fb478156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:35.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.321+0000 7fb47d9ec700 1 --2- 172.21.15.89:0/422040649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4781565f0 0x7fb478156a60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb4680118e0 tx=0x7fb468008040 comp rx=0 tx=0).stop 2023-10-10T20:39:35.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.321+0000 7fb47d9ec700 1 -- 172.21.15.89:0/422040649 shutdown_connections 2023-10-10T20:39:35.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.321+0000 7fb47d9ec700 1 --2- 172.21.15.89:0/422040649 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb464067170 0x7fb464069620 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.321+0000 7fb47d9ec700 1 --2- 172.21.15.89:0/422040649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4781545a0 0x7fb478154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.321+0000 7fb47d9ec700 1 --2- 172.21.15.89:0/422040649 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb460003a60 0x7fb478153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.321+0000 7fb47d9ec700 1 --2- 172.21.15.89:0/422040649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4781565f0 0x7fb478156a60 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.321+0000 7fb47d9ec700 1 -- 172.21.15.89:0/422040649 >> 172.21.15.89:0/422040649 conn(0x7fb4780d0630 msgr2=0x7fb4780b5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:35.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.321+0000 7fb47d9ec700 1 -- 172.21.15.89:0/422040649 shutdown_connections 2023-10-10T20:39:35.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.321+0000 7fb47d9ec700 1 -- 172.21.15.89:0/422040649 wait complete. 2023-10-10T20:39:35.335 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:39:35.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.448+0000 7f67fb5b3700 1 Processor -- start 2023-10-10T20:39:35.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.448+0000 7f67fb5b3700 1 -- start start 2023-10-10T20:39:35.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.449+0000 7f67fb5b3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f67f4159c60 0x7f67f415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.449+0000 7f67fb5b3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67f415db10 0x7f67f4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.449+0000 7f67fb5b3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f415a160 con 0x7f67f4159890 2023-10-10T20:39:35.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.449+0000 7f67fb5b3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f415df50 con 0x7f67f4159c60 2023-10-10T20:39:35.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.449+0000 7f67fb5b3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f40d13a0 con 0x7f67f415db10 2023-10-10T20:39:35.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.449+0000 7f67f8b4e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f67f4159c60 0x7f67f415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.449+0000 7f67f934f700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f67f4159890 0x7f67f415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38960/0 (socket says 172.21.15.89:38960) 2023-10-10T20:39:35.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.449+0000 7f67f934f700 1 -- 172.21.15.89:0/1587243567 learned_addr learned my addr 172.21.15.89:0/1587243567 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:35.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.449+0000 7f67f8b4e700 1 -- 172.21.15.89:0/1587243567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67f415db10 msgr2=0x7f67f4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:35.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.449+0000 7f67f9b50700 1 --2- 172.21.15.89:0/1587243567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67f415db10 0x7f67f4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.449+0000 7f67f8b4e700 1 --2- 172.21.15.89:0/1587243567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67f415db10 0x7f67f4069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.449+0000 7f67f3fff700 1 -- 172.21.15.89:0/1587243567 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430894600 0 0) 0x7f67f415a160 con 0x7f67f4159890 2023-10-10T20:39:35.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.449+0000 7f67f8b4e700 1 -- 172.21.15.89:0/1587243567 >> v1:172.21.15.89:6789/0 conn(0x7f67f4159890 legacy=0x7f67f415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:35.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.449+0000 7f67f8b4e700 1 -- 172.21.15.89:0/1587243567 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f415e140 con 0x7f67f4159c60 2023-10-10T20:39:35.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.449+0000 7f67f9b50700 1 --2- 172.21.15.89:0/1587243567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67f415db10 0x7f67f4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:35.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.449+0000 7f67f8b4e700 1 --2- 172.21.15.89:0/1587243567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f4159c60 0x7f67f415d5d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f67e80098e0 tx=0x7f67e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202c8848969d22c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:35.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.449+0000 7f67f3fff700 1 -- 172.21.15.89:0/1587243567 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8009e80 con 0x7f67f4159c60 2023-10-10T20:39:35.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.450+0000 7f67f3fff700 1 -- 172.21.15.89:0/1587243567 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e800b860 con 0x7f67f4159c60 2023-10-10T20:39:35.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.450+0000 7f67f3fff700 1 -- 172.21.15.89:0/1587243567 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e800bbe0 con 0x7f67f4159c60 2023-10-10T20:39:35.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.450+0000 7f67fb5b3700 1 -- 172.21.15.89:0/1587243567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f4159c60 msgr2=0x7f67f415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:35.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.450+0000 7f67fb5b3700 1 --2- 172.21.15.89:0/1587243567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f4159c60 0x7f67f415d5d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f67e80098e0 tx=0x7f67e80091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.450+0000 7f67fb5b3700 1 -- 172.21.15.89:0/1587243567 shutdown_connections 2023-10-10T20:39:35.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.450+0000 7f67fb5b3700 1 --2- 172.21.15.89:0/1587243567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67f415db10 0x7f67f4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.450+0000 7f67fb5b3700 1 --2- 172.21.15.89:0/1587243567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f4159c60 0x7f67f415d5d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.450+0000 7f67fb5b3700 1 -- 172.21.15.89:0/1587243567 >> 172.21.15.89:0/1587243567 conn(0x7f67f4068e20 msgr2=0x7f67f40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:35.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.450+0000 7f67fb5b3700 1 -- 172.21.15.89:0/1587243567 shutdown_connections 2023-10-10T20:39:35.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.450+0000 7f67fb5b3700 1 -- 172.21.15.89:0/1587243567 wait complete. 2023-10-10T20:39:35.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.451+0000 7f67fb5b3700 1 Processor -- start 2023-10-10T20:39:35.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.451+0000 7f67fb5b3700 1 -- start start 2023-10-10T20:39:35.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.452+0000 7f67fb5b3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67f4159890 0x7f67f40b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.452+0000 7f67fb5b3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67f4159c60 0x7f67f40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.452+0000 7f67fb5b3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f415db10 0x7f67f40cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.452+0000 7f67fb5b3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f40b36c0 con 0x7f67f415db10 2023-10-10T20:39:35.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.452+0000 7f67fb5b3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f40b39a0 con 0x7f67f4159890 2023-10-10T20:39:35.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.452+0000 7f67fb5b3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f40b0ff0 con 0x7f67f4159c60 2023-10-10T20:39:35.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.452+0000 7f67f9b50700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f415db10 0x7f67f40cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.452+0000 7f67f9b50700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f415db10 0x7f67f40cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50934/0 (socket says 172.21.15.89:50934) 2023-10-10T20:39:35.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.452+0000 7f67f9b50700 1 -- 172.21.15.89:0/2719904647 learned_addr learned my addr 172.21.15.89:0/2719904647 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:35.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.452+0000 7f67f934f700 1 --2- 172.21.15.89:0/2719904647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67f4159890 0x7f67f40b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.452+0000 7f67f8b4e700 1 --2- 172.21.15.89:0/2719904647 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67f4159c60 0x7f67f40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.452+0000 7f67f9b50700 1 -- 172.21.15.89:0/2719904647 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67f4159c60 msgr2=0x7f67f40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:35.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.452+0000 7f67f9b50700 1 --2- 172.21.15.89:0/2719904647 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67f4159c60 0x7f67f40cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.452+0000 7f67f9b50700 1 -- 172.21.15.89:0/2719904647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67f4159890 msgr2=0x7f67f40b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:35.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.452+0000 7f67f9b50700 1 --2- 172.21.15.89:0/2719904647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67f4159890 0x7f67f40b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.452+0000 7f67f9b50700 1 -- 172.21.15.89:0/2719904647 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e8009420 con 0x7f67f415db10 2023-10-10T20:39:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.452+0000 7f67f934f700 1 --2- 172.21.15.89:0/2719904647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67f4159890 0x7f67f40b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.453+0000 7f67f9b50700 1 --2- 172.21.15.89:0/2719904647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f415db10 0x7f67f40cd100 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f67e400c850 tx=0x7f67e4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.453+0000 7f67f17fa700 1 -- 172.21.15.89:0/2719904647 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e4011600 con 0x7f67f415db10 2023-10-10T20:39:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.453+0000 7f67f17fa700 1 -- 172.21.15.89:0/2719904647 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e40117c0 con 0x7f67f415db10 2023-10-10T20:39:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.453+0000 7f67fb5b3700 1 -- 172.21.15.89:0/2719904647 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f40b12d0 con 0x7f67f415db10 2023-10-10T20:39:35.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.453+0000 7f67f17fa700 1 -- 172.21.15.89:0/2719904647 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e4011b80 con 0x7f67f415db10 2023-10-10T20:39:35.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.453+0000 7f67fb5b3700 1 -- 172.21.15.89:0/2719904647 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f40cfad0 con 0x7f67f415db10 2023-10-10T20:39:35.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.454+0000 7f67fb5b3700 1 -- 172.21.15.89:0/2719904647 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f404c3c0 con 0x7f67f415db10 2023-10-10T20:39:35.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.454+0000 7f67f17fa700 1 -- 172.21.15.89:0/2719904647 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f67e4018350 con 0x7f67f415db10 2023-10-10T20:39:35.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.454+0000 7f67f17fa700 1 --2- 172.21.15.89:0/2719904647 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f67e0067170 0x7f67e0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.455+0000 7f67f17fa700 1 -- 172.21.15.89:0/2719904647 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f67e4084570 con 0x7f67f415db10 2023-10-10T20:39:35.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.455+0000 7f67f934f700 1 --2- 172.21.15.89:0/2719904647 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f67e0067170 0x7f67e0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.455+0000 7f67f934f700 1 --2- 172.21.15.89:0/2719904647 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f67e0067170 0x7f67e0069620 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f67ec001560 tx=0x7f67ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:35.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.459+0000 7f67f17fa700 1 -- 172.21.15.89:0/2719904647 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67e4051790 con 0x7f67f415db10 2023-10-10T20:39:35.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.644+0000 7f67fb5b3700 1 -- 172.21.15.89:0/2719904647 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f67f406a5b0 con 0x7f67f415db10 2023-10-10T20:39:35.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.645+0000 7f67f17fa700 1 -- 172.21.15.89:0/2719904647 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) v1 ==== 84+0+65 (secure 0 0 0) 0x7f67e4054db0 con 0x7f67f415db10 2023-10-10T20:39:35.647 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:35.647 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:39:35.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:35.648 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2023-10-10T20:39:35.648 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2023-10-10T20:39:35.648 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:35.648 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:39:35.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.647+0000 7f67fb5b3700 1 -- 172.21.15.89:0/2719904647 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f67e0067170 msgr2=0x7f67e0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:35.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.647+0000 7f67fb5b3700 1 --2- 172.21.15.89:0/2719904647 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f67e0067170 0x7f67e0069620 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f67ec001560 tx=0x7f67ec009040 comp rx=0 tx=0).stop 2023-10-10T20:39:35.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.647+0000 7f67fb5b3700 1 -- 172.21.15.89:0/2719904647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f415db10 msgr2=0x7f67f40cd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:35.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.647+0000 7f67fb5b3700 1 --2- 172.21.15.89:0/2719904647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f415db10 0x7f67f40cd100 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f67e400c850 tx=0x7f67e4008040 comp rx=0 tx=0).stop 2023-10-10T20:39:35.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.647+0000 7f67fb5b3700 1 -- 172.21.15.89:0/2719904647 shutdown_connections 2023-10-10T20:39:35.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.647+0000 7f67fb5b3700 1 --2- 172.21.15.89:0/2719904647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f415db10 0x7f67f40cd100 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.647+0000 7f67fb5b3700 1 --2- 172.21.15.89:0/2719904647 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f67e0067170 0x7f67e0069620 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.647+0000 7f67fb5b3700 1 --2- 172.21.15.89:0/2719904647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67f4159890 0x7f67f40b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.647+0000 7f67fb5b3700 1 --2- 172.21.15.89:0/2719904647 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f67f4159c60 0x7f67f40cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.647+0000 7f67fb5b3700 1 -- 172.21.15.89:0/2719904647 >> 172.21.15.89:0/2719904647 conn(0x7f67f4068e20 msgr2=0x7f67f4155310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:35.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.647+0000 7f67fb5b3700 1 -- 172.21.15.89:0/2719904647 shutdown_connections 2023-10-10T20:39:35.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.647+0000 7f67fb5b3700 1 -- 172.21.15.89:0/2719904647 wait complete. 2023-10-10T20:39:35.661 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:39:35.661 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:39:35.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.778+0000 7f1e00878700 1 Processor -- start 2023-10-10T20:39:35.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.778+0000 7f1e00878700 1 -- start start 2023-10-10T20:39:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.779+0000 7f1e00878700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1dfc0b03c0 0x7f1dfc0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.779+0000 7f1e00878700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dfc0cfa00 con 0x7f1dfc0b0790 2023-10-10T20:39:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.779+0000 7f1e00878700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dfc0cb580 con 0x7f1dfc1591e0 2023-10-10T20:39:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.779+0000 7f1e00878700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dfc0cf160 con 0x7f1dfc0b03c0 2023-10-10T20:39:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.779+0000 7f1dfb7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1dfc0b03c0 0x7f1dfc0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.779+0000 7f1dfb7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1dfc0b03c0 0x7f1dfc0cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50944/0 (socket says 172.21.15.89:50944) 2023-10-10T20:39:35.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.779+0000 7f1dfb7fe700 1 -- 172.21.15.89:0/567015289 learned_addr learned my addr 172.21.15.89:0/567015289 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:35.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.779+0000 7f1dfaffd700 1 --1- 172.21.15.89:0/567015289 >> v1:172.21.15.89:6789/0 conn(0x7f1dfc0b0790 0x7f1dfc0b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38972/0 (socket says 172.21.15.89:38972) 2023-10-10T20:39:35.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.779+0000 7f1dfb7fe700 1 -- 172.21.15.89:0/567015289 >> v1:172.21.15.111:6789/0 conn(0x7f1dfc1591e0 legacy=0x7f1dfc159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:35.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.779+0000 7f1dfb7fe700 1 -- 172.21.15.89:0/567015289 >> v1:172.21.15.89:6789/0 conn(0x7f1dfc0b0790 legacy=0x7f1dfc0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:35.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.779+0000 7f1dfa7fc700 1 -- 172.21.15.89:0/567015289 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315002003 0 0) 0x7f1dfc0cfa00 con 0x7f1dfc0b0790 2023-10-10T20:39:35.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.779+0000 7f1dfb7fe700 1 -- 172.21.15.89:0/567015289 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc0cf440 con 0x7f1dfc0b03c0 2023-10-10T20:39:35.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.779+0000 7f1dfb7fe700 1 --2- 172.21.15.89:0/567015289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc0b03c0 0x7f1dfc0cea70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0099e0 tx=0x7f1dec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce842c71d4f69347 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:35.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.779+0000 7f1dfa7fc700 1 -- 172.21.15.89:0/567015289 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec00b820 con 0x7f1dfc0b03c0 2023-10-10T20:39:35.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.779+0000 7f1dfa7fc700 1 -- 172.21.15.89:0/567015289 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dec00b980 con 0x7f1dfc0b03c0 2023-10-10T20:39:35.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.779+0000 7f1dfa7fc700 1 -- 172.21.15.89:0/567015289 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec00bd40 con 0x7f1dfc0b03c0 2023-10-10T20:39:35.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.780+0000 7f1e00878700 1 -- 172.21.15.89:0/567015289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc0b03c0 msgr2=0x7f1dfc0cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:35.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.780+0000 7f1e00878700 1 --2- 172.21.15.89:0/567015289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc0b03c0 0x7f1dfc0cea70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0099e0 tx=0x7f1dec009250 comp rx=0 tx=0).stop 2023-10-10T20:39:35.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.780+0000 7f1e00878700 1 -- 172.21.15.89:0/567015289 shutdown_connections 2023-10-10T20:39:35.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.780+0000 7f1e00878700 1 --2- 172.21.15.89:0/567015289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc0b03c0 0x7f1dfc0cea70 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.780+0000 7f1e00878700 1 -- 172.21.15.89:0/567015289 >> 172.21.15.89:0/567015289 conn(0x7f1dfc063230 msgr2=0x7f1dfc063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:35.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.780+0000 7f1e00878700 1 -- 172.21.15.89:0/567015289 shutdown_connections 2023-10-10T20:39:35.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.780+0000 7f1e00878700 1 -- 172.21.15.89:0/567015289 wait complete. 2023-10-10T20:39:35.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.781+0000 7f1e00878700 1 Processor -- start 2023-10-10T20:39:35.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.781+0000 7f1e00878700 1 -- start start 2023-10-10T20:39:35.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.782+0000 7f1e00878700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1dfc0b03c0 0x7f1dfc159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.782+0000 7f1e00878700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dfc0b0790 0x7f1dfc157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.782+0000 7f1e00878700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc1591e0 0x7f1dfc0c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.782+0000 7f1e00878700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc0c7640 con 0x7f1dfc1591e0 2023-10-10T20:39:35.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.782+0000 7f1e00878700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc0c7920 con 0x7f1dfc0b0790 2023-10-10T20:39:35.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.782+0000 7f1e00878700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc0c7c00 con 0x7f1dfc0b03c0 2023-10-10T20:39:35.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.782+0000 7f1dfbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc1591e0 0x7f1dfc0c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.782+0000 7f1dfbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc1591e0 0x7f1dfc0c5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50952/0 (socket says 172.21.15.89:50952) 2023-10-10T20:39:35.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.782+0000 7f1dfbfff700 1 -- 172.21.15.89:0/2912692788 learned_addr learned my addr 172.21.15.89:0/2912692788 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:35.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.782+0000 7f1dfaffd700 1 --2- 172.21.15.89:0/2912692788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dfc0b0790 0x7f1dfc157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.782+0000 7f1dfb7fe700 1 --2- 172.21.15.89:0/2912692788 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1dfc0b03c0 0x7f1dfc159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.782+0000 7f1dfbfff700 1 -- 172.21.15.89:0/2912692788 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1dfc0b03c0 msgr2=0x7f1dfc159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:35.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.782+0000 7f1dfbfff700 1 --2- 172.21.15.89:0/2912692788 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1dfc0b03c0 0x7f1dfc159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.782+0000 7f1dfbfff700 1 -- 172.21.15.89:0/2912692788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dfc0b0790 msgr2=0x7f1dfc157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:35.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.782+0000 7f1dfbfff700 1 --2- 172.21.15.89:0/2912692788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dfc0b0790 0x7f1dfc157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:35.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.782+0000 7f1dfbfff700 1 -- 172.21.15.89:0/2912692788 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec0094f0 con 0x7f1dfc1591e0 2023-10-10T20:39:35.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.782+0000 7f1dfbfff700 1 --2- 172.21.15.89:0/2912692788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc1591e0 0x7f1dfc0c5550 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1df400c840 tx=0x7f1df4001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:35.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.783+0000 7f1de37fe700 1 -- 172.21.15.89:0/2912692788 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4012630 con 0x7f1dfc1591e0 2023-10-10T20:39:35.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.783+0000 7f1de37fe700 1 -- 172.21.15.89:0/2912692788 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df40127b0 con 0x7f1dfc1591e0 2023-10-10T20:39:35.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.783+0000 7f1e00878700 1 -- 172.21.15.89:0/2912692788 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dfc0c7ee0 con 0x7f1dfc1591e0 2023-10-10T20:39:35.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.783+0000 7f1de37fe700 1 -- 172.21.15.89:0/2912692788 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4012b70 con 0x7f1dfc1591e0 2023-10-10T20:39:35.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.783+0000 7f1e00878700 1 -- 172.21.15.89:0/2912692788 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc15a360 con 0x7f1dfc1591e0 2023-10-10T20:39:35.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.784+0000 7f1e00878700 1 -- 172.21.15.89:0/2912692788 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dfc0c7a60 con 0x7f1dfc1591e0 2023-10-10T20:39:35.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.784+0000 7f1de37fe700 1 -- 172.21.15.89:0/2912692788 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f1df4003030 con 0x7f1dfc1591e0 2023-10-10T20:39:35.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.784+0000 7f1de37fe700 1 --2- 172.21.15.89:0/2912692788 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1de8067170 0x7f1de8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:35.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.784+0000 7f1de37fe700 1 -- 172.21.15.89:0/2912692788 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1df40844e0 con 0x7f1dfc1591e0 2023-10-10T20:39:35.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.784+0000 7f1dfb7fe700 1 --2- 172.21.15.89:0/2912692788 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1de8067170 0x7f1de8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:35.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.785+0000 7f1dfb7fe700 1 --2- 172.21.15.89:0/2912692788 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1de8067170 0x7f1de8069620 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0099e0 tx=0x7f1dec017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:35.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.789+0000 7f1de37fe700 1 -- 172.21.15.89:0/2912692788 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1dfc0c7a60 con 0x7f1dfc1591e0 2023-10-10T20:39:35.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:35.971+0000 7f1e00878700 1 -- 172.21.15.89:0/2912692788 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1dfc0c7a60 con 0x7f1dfc1591e0 2023-10-10T20:39:36.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.244+0000 7f1de37fe700 1 -- 172.21.15.89:0/2912692788 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1dfc0c7a60 con 0x7f1dfc1591e0 2023-10-10T20:39:36.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.247+0000 7f1e00878700 1 -- 172.21.15.89:0/2912692788 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1de8067170 msgr2=0x7f1de8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:36.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.247+0000 7f1e00878700 1 --2- 172.21.15.89:0/2912692788 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1de8067170 0x7f1de8069620 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0099e0 tx=0x7f1dec017000 comp rx=0 tx=0).stop 2023-10-10T20:39:36.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.247+0000 7f1e00878700 1 -- 172.21.15.89:0/2912692788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc1591e0 msgr2=0x7f1dfc0c5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:36.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.247+0000 7f1e00878700 1 --2- 172.21.15.89:0/2912692788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc1591e0 0x7f1dfc0c5550 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1df400c840 tx=0x7f1df4001670 comp rx=0 tx=0).stop 2023-10-10T20:39:36.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.247+0000 7f1e00878700 1 -- 172.21.15.89:0/2912692788 shutdown_connections 2023-10-10T20:39:36.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.247+0000 7f1e00878700 1 --2- 172.21.15.89:0/2912692788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dfc1591e0 0x7f1dfc0c5550 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:36.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.247+0000 7f1e00878700 1 --2- 172.21.15.89:0/2912692788 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1dfc0b03c0 0x7f1dfc159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:36.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.247+0000 7f1e00878700 1 --2- 172.21.15.89:0/2912692788 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1de8067170 0x7f1de8069620 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:36.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.247+0000 7f1e00878700 1 --2- 172.21.15.89:0/2912692788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dfc0b0790 0x7f1dfc157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:36.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.247+0000 7f1e00878700 1 -- 172.21.15.89:0/2912692788 >> 172.21.15.89:0/2912692788 conn(0x7f1dfc063230 msgr2=0x7f1dfc159600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:36.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.247+0000 7f1e00878700 1 -- 172.21.15.89:0/2912692788 shutdown_connections 2023-10-10T20:39:36.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.247+0000 7f1e00878700 1 -- 172.21.15.89:0/2912692788 wait complete. 2023-10-10T20:39:36.252 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2023-10-10T20:39:36.263 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:39:36.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.382+0000 7fa73c33d700 1 Processor -- start 2023-10-10T20:39:36.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.382+0000 7fa73c33d700 1 -- start start 2023-10-10T20:39:36.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.383+0000 7fa73c33d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa734159950 0x7fa7340cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:36.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.383+0000 7fa73c33d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa734069c70 0x7fa7340cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:36.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.383+0000 7fa73c33d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7340ce380 con 0x7fa7340af320 2023-10-10T20:39:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.383+0000 7fa73c33d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7340cec20 con 0x7fa734069c70 2023-10-10T20:39:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.383+0000 7fa73c33d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7340afa90 con 0x7fa734159950 2023-10-10T20:39:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.383+0000 7fa73a0d9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa734159950 0x7fa7340cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.384+0000 7fa73a8da700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa734069c70 0x7fa7340cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.384+0000 7fa7398d8700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa7340af320 0x7fa734069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:58684/0 (socket says 172.21.15.89:58684) 2023-10-10T20:39:36.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.384+0000 7fa7398d8700 1 -- 172.21.15.89:0/2528884411 learned_addr learned my addr 172.21.15.89:0/2528884411 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:36.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.384+0000 7fa73a8da700 1 -- 172.21.15.89:0/2528884411 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa734159950 msgr2=0x7fa7340cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:36.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.384+0000 7fa73a8da700 1 --2- 172.21.15.89:0/2528884411 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa734159950 0x7fa7340cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:36.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.384+0000 7fa73a8da700 1 -- 172.21.15.89:0/2528884411 >> v1:172.21.15.111:6789/0 conn(0x7fa7340af320 legacy=0x7fa734069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:36.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.384+0000 7fa73a8da700 1 -- 172.21.15.89:0/2528884411 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7340ce5d0 con 0x7fa734069c70 2023-10-10T20:39:36.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.384+0000 7fa73a8da700 1 --2- 172.21.15.89:0/2528884411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa734069c70 0x7fa7340cf0b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009e60 tx=0x7fa72c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cdd7f42f026d419 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:36.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.385+0000 7fa7390d7700 1 -- 172.21.15.89:0/2528884411 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c00e070 con 0x7fa734069c70 2023-10-10T20:39:36.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.385+0000 7fa7390d7700 1 -- 172.21.15.89:0/2528884411 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c00bb60 con 0x7fa734069c70 2023-10-10T20:39:36.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.385+0000 7fa7390d7700 1 -- 172.21.15.89:0/2528884411 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c012430 con 0x7fa734069c70 2023-10-10T20:39:36.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.385+0000 7fa73c33d700 1 -- 172.21.15.89:0/2528884411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa734069c70 msgr2=0x7fa7340cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:36.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.385+0000 7fa73c33d700 1 --2- 172.21.15.89:0/2528884411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa734069c70 0x7fa7340cf0b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009e60 tx=0x7fa72c009780 comp rx=0 tx=0).stop 2023-10-10T20:39:36.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.385+0000 7fa73c33d700 1 -- 172.21.15.89:0/2528884411 shutdown_connections 2023-10-10T20:39:36.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.385+0000 7fa73c33d700 1 --2- 172.21.15.89:0/2528884411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa734069c70 0x7fa7340cf0b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:36.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.385+0000 7fa73c33d700 1 --2- 172.21.15.89:0/2528884411 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa734159950 0x7fa7340cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:36.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.385+0000 7fa73c33d700 1 -- 172.21.15.89:0/2528884411 >> 172.21.15.89:0/2528884411 conn(0x7fa734068e00 msgr2=0x7fa7340d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:36.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.385+0000 7fa73c33d700 1 -- 172.21.15.89:0/2528884411 shutdown_connections 2023-10-10T20:39:36.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.385+0000 7fa73c33d700 1 -- 172.21.15.89:0/2528884411 wait complete. 2023-10-10T20:39:36.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.386+0000 7fa73c33d700 1 Processor -- start 2023-10-10T20:39:36.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.386+0000 7fa73c33d700 1 -- start start 2023-10-10T20:39:36.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.387+0000 7fa73c33d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa734069c70 0x7fa7340b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:36.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.387+0000 7fa73c33d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7340af320 0x7fa734151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:36.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.387+0000 7fa73c33d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa734159950 0x7fa734151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:36.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.387+0000 7fa73c33d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa734157fe0 con 0x7fa7340af320 2023-10-10T20:39:36.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.387+0000 7fa73c33d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7341571f0 con 0x7fa734069c70 2023-10-10T20:39:36.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.387+0000 7fa73c33d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7341581f0 con 0x7fa734159950 2023-10-10T20:39:36.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.387+0000 7fa7398d8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7340af320 0x7fa734151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:36.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.387+0000 7fa7398d8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7340af320 0x7fa734151960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50966/0 (socket says 172.21.15.89:50966) 2023-10-10T20:39:36.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.387+0000 7fa7398d8700 1 -- 172.21.15.89:0/4033864591 learned_addr learned my addr 172.21.15.89:0/4033864591 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:36.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.387+0000 7fa73a8da700 1 --2- 172.21.15.89:0/4033864591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa734159950 0x7fa734151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:36.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.387+0000 7fa73a0d9700 1 --2- 172.21.15.89:0/4033864591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa734069c70 0x7fa7340b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:36.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.387+0000 7fa7398d8700 1 -- 172.21.15.89:0/4033864591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa734159950 msgr2=0x7fa734151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:36.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.387+0000 7fa7398d8700 1 --2- 172.21.15.89:0/4033864591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa734159950 0x7fa734151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:36.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.387+0000 7fa7398d8700 1 -- 172.21.15.89:0/4033864591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa734069c70 msgr2=0x7fa7340b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:36.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.387+0000 7fa7398d8700 1 --2- 172.21.15.89:0/4033864591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa734069c70 0x7fa7340b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:36.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.387+0000 7fa7398d8700 1 -- 172.21.15.89:0/4033864591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c009a20 con 0x7fa7340af320 2023-10-10T20:39:36.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.387+0000 7fa73a0d9700 1 --2- 172.21.15.89:0/4033864591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa734069c70 0x7fa7340b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:36.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.388+0000 7fa7398d8700 1 --2- 172.21.15.89:0/4033864591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7340af320 0x7fa734151960 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa72400c7c0 tx=0x7fa724007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:36.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.388+0000 7fa72a7fc700 1 -- 172.21.15.89:0/4033864591 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724005b30 con 0x7fa7340af320 2023-10-10T20:39:36.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.388+0000 7fa72a7fc700 1 -- 172.21.15.89:0/4033864591 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa724005c90 con 0x7fa7340af320 2023-10-10T20:39:36.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.388+0000 7fa73c33d700 1 -- 172.21.15.89:0/4033864591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7341584d0 con 0x7fa7340af320 2023-10-10T20:39:36.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.388+0000 7fa72a7fc700 1 -- 172.21.15.89:0/4033864591 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724004a80 con 0x7fa7340af320 2023-10-10T20:39:36.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.388+0000 7fa73c33d700 1 -- 172.21.15.89:0/4033864591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa734156e00 con 0x7fa7340af320 2023-10-10T20:39:36.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.389+0000 7fa73c33d700 1 -- 172.21.15.89:0/4033864591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7340718b0 con 0x7fa7340af320 2023-10-10T20:39:36.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.389+0000 7fa72a7fc700 1 -- 172.21.15.89:0/4033864591 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fa7240070d0 con 0x7fa7340af320 2023-10-10T20:39:36.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.389+0000 7fa72a7fc700 1 --2- 172.21.15.89:0/4033864591 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa718067170 0x7fa718069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:36.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.390+0000 7fa72a7fc700 1 -- 172.21.15.89:0/4033864591 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fa724083a60 con 0x7fa7340af320 2023-10-10T20:39:36.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.390+0000 7fa73a0d9700 1 --2- 172.21.15.89:0/4033864591 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa718067170 0x7fa718069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:36.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.390+0000 7fa73a0d9700 1 --2- 172.21.15.89:0/4033864591 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa718067170 0x7fa718069620 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa730009ce0 tx=0x7fa730008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:36.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.394+0000 7fa72a7fc700 1 -- 172.21.15.89:0/4033864591 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa72404b5a0 con 0x7fa7340af320 2023-10-10T20:39:36.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:36.577+0000 7fa73c33d700 1 -- 172.21.15.89:0/4033864591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa7341586c0 con 0x7fa7340af320 2023-10-10T20:39:37.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.253+0000 7fa72a7fc700 1 -- 172.21.15.89:0/4033864591 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v74) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa724050c20 con 0x7fa7340af320 2023-10-10T20:39:37.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.258+0000 7fa73c33d700 1 -- 172.21.15.89:0/4033864591 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa718067170 msgr2=0x7fa718069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:37.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.258+0000 7fa73c33d700 1 --2- 172.21.15.89:0/4033864591 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa718067170 0x7fa718069620 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa730009ce0 tx=0x7fa730008040 comp rx=0 tx=0).stop 2023-10-10T20:39:37.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.258+0000 7fa73c33d700 1 -- 172.21.15.89:0/4033864591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7340af320 msgr2=0x7fa734151960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:37.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.258+0000 7fa73c33d700 1 --2- 172.21.15.89:0/4033864591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7340af320 0x7fa734151960 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa72400c7c0 tx=0x7fa724007f70 comp rx=0 tx=0).stop 2023-10-10T20:39:37.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.258+0000 7fa73c33d700 1 -- 172.21.15.89:0/4033864591 shutdown_connections 2023-10-10T20:39:37.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.258+0000 7fa73c33d700 1 --2- 172.21.15.89:0/4033864591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa734159950 0x7fa734151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:37.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.258+0000 7fa73c33d700 1 --2- 172.21.15.89:0/4033864591 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa718067170 0x7fa718069620 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:37.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.258+0000 7fa73c33d700 1 --2- 172.21.15.89:0/4033864591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa734069c70 0x7fa7340b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:37.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.258+0000 7fa73c33d700 1 --2- 172.21.15.89:0/4033864591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7340af320 0x7fa734151960 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:37.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.258+0000 7fa73c33d700 1 -- 172.21.15.89:0/4033864591 >> 172.21.15.89:0/4033864591 conn(0x7fa734068e00 msgr2=0x7fa7340c5550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:37.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.258+0000 7fa73c33d700 1 -- 172.21.15.89:0/4033864591 shutdown_connections 2023-10-10T20:39:37.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.259+0000 7fa73c33d700 1 -- 172.21.15.89:0/4033864591 wait complete. 2023-10-10T20:39:37.264 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2023-10-10T20:39:37.280 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:39:37.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.397+0000 7ff935579700 1 Processor -- start 2023-10-10T20:39:37.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.397+0000 7ff935579700 1 -- start start 2023-10-10T20:39:37.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.398+0000 7ff935579700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff930159c60 0x7ff93015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:37.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.398+0000 7ff935579700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff93015db10 0x7ff930069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:37.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.398+0000 7ff935579700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff93015a160 con 0x7ff930159890 2023-10-10T20:39:37.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.398+0000 7ff935579700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff93015df50 con 0x7ff930159c60 2023-10-10T20:39:37.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.398+0000 7ff935579700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9300d13a0 con 0x7ff93015db10 2023-10-10T20:39:37.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.398+0000 7ff92effd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff930159890 0x7ff93015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38984/0 (socket says 172.21.15.89:38984) 2023-10-10T20:39:37.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.398+0000 7ff9267fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff930159c60 0x7ff93015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:37.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.398+0000 7ff92effd700 1 -- 172.21.15.89:0/1602467795 learned_addr learned my addr 172.21.15.89:0/1602467795 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:37.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.398+0000 7ff92f7fe700 1 --2- 172.21.15.89:0/1602467795 >> v2:172.21.15.111:3300/0 conn(0x7ff93015db10 0x7ff930069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:37.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.398+0000 7ff9267fc700 1 -- 172.21.15.89:0/1602467795 >> v2:172.21.15.111:3300/0 conn(0x7ff93015db10 msgr2=0x7ff930069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:37.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.398+0000 7ff9267fc700 1 --2- 172.21.15.89:0/1602467795 >> v2:172.21.15.111:3300/0 conn(0x7ff93015db10 0x7ff930069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:37.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.398+0000 7ff9267fc700 1 -- 172.21.15.89:0/1602467795 >> v1:172.21.15.89:6789/0 conn(0x7ff930159890 legacy=0x7ff93015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:37.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.398+0000 7ff9267fc700 1 -- 172.21.15.89:0/1602467795 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93015e140 con 0x7ff930159c60 2023-10-10T20:39:37.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.398+0000 7ff92e7fc700 1 -- 172.21.15.89:0/1602467795 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915954868 0 0) 0x7ff93015a160 con 0x7ff930159890 2023-10-10T20:39:37.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.398+0000 7ff92f7fe700 1 --2- 172.21.15.89:0/1602467795 >> v2:172.21.15.111:3300/0 conn(0x7ff93015db10 0x7ff930069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:37.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.399+0000 7ff9267fc700 1 --2- 172.21.15.89:0/1602467795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff930159c60 0x7ff93015d5d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff93006a890 tx=0x7ff9180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac36b035c416f45 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:37.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.399+0000 7ff92e7fc700 1 -- 172.21.15.89:0/1602467795 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91800b6b0 con 0x7ff930159c60 2023-10-10T20:39:37.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.399+0000 7ff92e7fc700 1 -- 172.21.15.89:0/1602467795 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91800b810 con 0x7ff930159c60 2023-10-10T20:39:37.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.399+0000 7ff92e7fc700 1 -- 172.21.15.89:0/1602467795 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91800bbb0 con 0x7ff930159c60 2023-10-10T20:39:37.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.399+0000 7ff935579700 1 -- 172.21.15.89:0/1602467795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff930159c60 msgr2=0x7ff93015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:37.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.399+0000 7ff935579700 1 --2- 172.21.15.89:0/1602467795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff930159c60 0x7ff93015d5d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff93006a890 tx=0x7ff9180091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:37.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.399+0000 7ff935579700 1 -- 172.21.15.89:0/1602467795 shutdown_connections 2023-10-10T20:39:37.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.399+0000 7ff935579700 1 --2- 172.21.15.89:0/1602467795 >> v2:172.21.15.111:3300/0 conn(0x7ff93015db10 0x7ff930069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:37.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.399+0000 7ff935579700 1 --2- 172.21.15.89:0/1602467795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff930159c60 0x7ff93015d5d0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:37.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.399+0000 7ff935579700 1 -- 172.21.15.89:0/1602467795 >> 172.21.15.89:0/1602467795 conn(0x7ff930068e20 msgr2=0x7ff9300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:37.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.399+0000 7ff935579700 1 -- 172.21.15.89:0/1602467795 shutdown_connections 2023-10-10T20:39:37.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.399+0000 7ff935579700 1 -- 172.21.15.89:0/1602467795 wait complete. 2023-10-10T20:39:37.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.400+0000 7ff935579700 1 Processor -- start 2023-10-10T20:39:37.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.401+0000 7ff935579700 1 -- start start 2023-10-10T20:39:37.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.401+0000 7ff935579700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff930159890 0x7ff9301470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:37.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.401+0000 7ff935579700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff930159c60 0x7ff9300cd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:37.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.401+0000 7ff935579700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff93015db10 0x7ff9300cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:37.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.401+0000 7ff935579700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9300cec70 con 0x7ff930159c60 2023-10-10T20:39:37.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.401+0000 7ff935579700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9300b06e0 con 0x7ff93015db10 2023-10-10T20:39:37.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.401+0000 7ff935579700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9300ce6b0 con 0x7ff930159890 2023-10-10T20:39:37.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.401+0000 7ff9267fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff930159c60 0x7ff9300cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:37.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.401+0000 7ff9267fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff930159c60 0x7ff9300cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50986/0 (socket says 172.21.15.89:50986) 2023-10-10T20:39:37.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.401+0000 7ff9267fc700 1 -- 172.21.15.89:0/2082339623 learned_addr learned my addr 172.21.15.89:0/2082339623 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:37.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.402+0000 7ff92f7fe700 1 --2- 172.21.15.89:0/2082339623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff93015db10 0x7ff9300cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:37.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.402+0000 7ff92effd700 1 --2- 172.21.15.89:0/2082339623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff930159890 0x7ff9301470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:37.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.402+0000 7ff9267fc700 1 -- 172.21.15.89:0/2082339623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff930159890 msgr2=0x7ff9301470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:37.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.402+0000 7ff9267fc700 1 --2- 172.21.15.89:0/2082339623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff930159890 0x7ff9301470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:37.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.402+0000 7ff9267fc700 1 -- 172.21.15.89:0/2082339623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff93015db10 msgr2=0x7ff9300cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:37.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.402+0000 7ff9267fc700 1 --2- 172.21.15.89:0/2082339623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff93015db10 0x7ff9300cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:37.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.402+0000 7ff9267fc700 1 -- 172.21.15.89:0/2082339623 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff918009420 con 0x7ff930159c60 2023-10-10T20:39:37.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.402+0000 7ff9267fc700 1 --2- 172.21.15.89:0/2082339623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff930159c60 0x7ff9300cd770 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff930070e20 tx=0x7ff918017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:37.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.402+0000 7ff9277fe700 1 -- 172.21.15.89:0/2082339623 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918011420 con 0x7ff930159c60 2023-10-10T20:39:37.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.402+0000 7ff9277fe700 1 -- 172.21.15.89:0/2082339623 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff918011580 con 0x7ff930159c60 2023-10-10T20:39:37.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.402+0000 7ff935579700 1 -- 172.21.15.89:0/2082339623 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9300ce990 con 0x7ff930159c60 2023-10-10T20:39:37.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.402+0000 7ff9277fe700 1 -- 172.21.15.89:0/2082339623 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918011920 con 0x7ff930159c60 2023-10-10T20:39:37.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.402+0000 7ff935579700 1 -- 172.21.15.89:0/2082339623 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff930147620 con 0x7ff930159c60 2023-10-10T20:39:37.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.403+0000 7ff935579700 1 -- 172.21.15.89:0/2082339623 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9300b0540 con 0x7ff930159c60 2023-10-10T20:39:37.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.404+0000 7ff9277fe700 1 -- 172.21.15.89:0/2082339623 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7ff918011cd0 con 0x7ff930159c60 2023-10-10T20:39:37.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.404+0000 7ff9277fe700 1 --2- 172.21.15.89:0/2082339623 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff910067170 0x7ff910069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:37.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.404+0000 7ff9277fe700 1 -- 172.21.15.89:0/2082339623 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5605+0+0 (secure 0 0 0) 0x7ff918086670 con 0x7ff930159c60 2023-10-10T20:39:37.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.404+0000 7ff92effd700 1 --2- 172.21.15.89:0/2082339623 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff910067170 0x7ff910069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:37.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.404+0000 7ff92effd700 1 --2- 172.21.15.89:0/2082339623 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff910067170 0x7ff910069620 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff9200011e0 tx=0x7ff920009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:37.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.408+0000 7ff9277fe700 1 -- 172.21.15.89:0/2082339623 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff918053810 con 0x7ff930159c60 2023-10-10T20:39:37.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:37.633+0000 7ff935579700 1 -- 172.21.15.89:0/2082339623 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff9300b0540 con 0x7ff930159c60 2023-10-10T20:39:38.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.253+0000 7ff9277fe700 1 -- 172.21.15.89:0/2082339623 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v42) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff9300b0540 con 0x7ff930159c60 2023-10-10T20:39:38.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.260+0000 7ff935579700 1 -- 172.21.15.89:0/2082339623 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff910067170 msgr2=0x7ff910069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:38.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.260+0000 7ff935579700 1 --2- 172.21.15.89:0/2082339623 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff910067170 0x7ff910069620 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff9200011e0 tx=0x7ff920009040 comp rx=0 tx=0).stop 2023-10-10T20:39:38.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.260+0000 7ff935579700 1 -- 172.21.15.89:0/2082339623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff930159c60 msgr2=0x7ff9300cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:38.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.260+0000 7ff935579700 1 --2- 172.21.15.89:0/2082339623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff930159c60 0x7ff9300cd770 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff930070e20 tx=0x7ff918017000 comp rx=0 tx=0).stop 2023-10-10T20:39:38.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.261+0000 7ff935579700 1 -- 172.21.15.89:0/2082339623 shutdown_connections 2023-10-10T20:39:38.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.261+0000 7ff935579700 1 --2- 172.21.15.89:0/2082339623 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff910067170 0x7ff910069620 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.261+0000 7ff935579700 1 --2- 172.21.15.89:0/2082339623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff93015db10 0x7ff9300cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.262+0000 7ff935579700 1 --2- 172.21.15.89:0/2082339623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff930159890 0x7ff9301470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.262+0000 7ff935579700 1 --2- 172.21.15.89:0/2082339623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff930159c60 0x7ff9300cd770 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.262+0000 7ff935579700 1 -- 172.21.15.89:0/2082339623 >> 172.21.15.89:0/2082339623 conn(0x7ff930068e20 msgr2=0x7ff930152fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:38.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.262+0000 7ff935579700 1 -- 172.21.15.89:0/2082339623 shutdown_connections 2023-10-10T20:39:38.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.262+0000 7ff935579700 1 -- 172.21.15.89:0/2082339623 wait complete. 2023-10-10T20:39:38.274 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 12 and data pool 13 2023-10-10T20:39:38.288 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:39:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.401+0000 7f7bebfff700 1 Processor -- start 2023-10-10T20:39:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.401+0000 7f7bebfff700 1 -- start start 2023-10-10T20:39:38.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.402+0000 7f7bebfff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bec1597f0 0x7f7bec0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:38.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.402+0000 7f7bebfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7bec15d230 0x7f7bec15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:38.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.402+0000 7f7bebfff700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bec15dbd0 con 0x7f7bec159420 2023-10-10T20:39:38.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.402+0000 7f7bebfff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bec0b11d0 con 0x7f7bec15d230 2023-10-10T20:39:38.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.402+0000 7f7bebfff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec0b14b0 con 0x7f7bec1597f0 2023-10-10T20:39:38.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.402+0000 7f7beaffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7bec159420 0x7f7bec0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48110/0 (socket says 172.21.15.89:48110) 2023-10-10T20:39:38.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.402+0000 7f7beaffd700 1 -- 172.21.15.89:0/3641996359 learned_addr learned my addr 172.21.15.89:0/3641996359 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:38.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.402+0000 7f7bea7fc700 1 -- 172.21.15.89:0/3641996359 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1781429393 0 0) 0x7f7bdc001110 con 0x7f7bec159420 2023-10-10T20:39:38.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.402+0000 7f7be27fc700 1 --2- 172.21.15.89:0/3641996359 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bec1597f0 0x7f7bec0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:38.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.402+0000 7f7bea7fc700 1 -- 172.21.15.89:0/3641996359 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bd8003570 con 0x7f7bec159420 2023-10-10T20:39:38.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.402+0000 7f7beb7fe700 1 --2- 172.21.15.89:0/3641996359 >> v2:172.21.15.111:3300/0 conn(0x7f7bec15d230 0x7f7bec15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:38.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7bea7fc700 1 -- 172.21.15.89:0/3641996359 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 832883472 0 0) 0x7f7bd8003570 con 0x7f7bec159420 2023-10-10T20:39:38.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7bea7fc700 1 -- 172.21.15.89:0/3641996359 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bec1597f0 msgr2=0x7f7bec0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:38.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7bea7fc700 1 --2- 172.21.15.89:0/3641996359 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bec1597f0 0x7f7bec0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7bea7fc700 1 -- 172.21.15.89:0/3641996359 >> v2:172.21.15.111:3300/0 conn(0x7f7bec15d230 msgr2=0x7f7bec15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:38.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7bea7fc700 1 --2- 172.21.15.89:0/3641996359 >> v2:172.21.15.111:3300/0 conn(0x7f7bec15d230 0x7f7bec15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7bea7fc700 1 -- 172.21.15.89:0/3641996359 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec0b1790 con 0x7f7bec159420 2023-10-10T20:39:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7bea7fc700 1 -- 172.21.15.89:0/3641996359 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f7bdc001600 con 0x7f7bec159420 2023-10-10T20:39:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7bea7fc700 1 -- 172.21.15.89:0/3641996359 >> v1:172.21.15.89:6789/0 conn(0x7f7bec159420 legacy=0x7f7bec0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7bea7fc700 1 --2- 172.21.15.89:0/3641996359 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bec1597f0 0x7f7bd8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7bea7fc700 1 --2- 172.21.15.89:0/3641996359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bd8003fe0 0x7f7bd8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7bea7fc700 1 --2- 172.21.15.89:0/3641996359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bd8004d50 0x7f7bd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7bea7fc700 1 -- 172.21.15.89:0/3641996359 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec0b11d0 con 0x7f7bd8003fe0 2023-10-10T20:39:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7bea7fc700 1 -- 172.21.15.89:0/3641996359 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec0b14b0 con 0x7f7bd8004d50 2023-10-10T20:39:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7bea7fc700 1 -- 172.21.15.89:0/3641996359 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd800ba90 con 0x7f7bec1597f0 2023-10-10T20:39:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7be27fc700 1 --2- 172.21.15.89:0/3641996359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bd8003fe0 0x7f7bd8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7beaffd700 1 --2- 172.21.15.89:0/3641996359 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bec1597f0 0x7f7bd8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7beb7fe700 1 --2- 172.21.15.89:0/3641996359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bd8004d50 0x7f7bd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7be27fc700 1 -- 172.21.15.89:0/3641996359 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bec1597f0 msgr2=0x7f7bd8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7be27fc700 1 --2- 172.21.15.89:0/3641996359 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bec1597f0 0x7f7bd8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7be27fc700 1 -- 172.21.15.89:0/3641996359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bd8004d50 msgr2=0x7f7bd80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7be27fc700 1 --2- 172.21.15.89:0/3641996359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bd8004d50 0x7f7bd80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7be27fc700 1 -- 172.21.15.89:0/3641996359 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd8003570 con 0x7f7bd8003fe0 2023-10-10T20:39:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7be27fc700 1 --2- 172.21.15.89:0/3641996359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bd8003fe0 0x7f7bd8004810 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009c00 tx=0x7f7bd40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e64dde706b60ca9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7bea7fc700 1 -- 172.21.15.89:0/3641996359 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd400e6b0 con 0x7f7bd8003fe0 2023-10-10T20:39:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7bea7fc700 1 -- 172.21.15.89:0/3641996359 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd400e810 con 0x7f7bd8003fe0 2023-10-10T20:39:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.403+0000 7f7bea7fc700 1 -- 172.21.15.89:0/3641996359 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd400ebd0 con 0x7f7bd8003fe0 2023-10-10T20:39:38.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.404+0000 7f7bebfff700 1 -- 172.21.15.89:0/3641996359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bd8003fe0 msgr2=0x7f7bd8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:38.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.404+0000 7f7bebfff700 1 --2- 172.21.15.89:0/3641996359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bd8003fe0 0x7f7bd8004810 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009c00 tx=0x7f7bd40091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.404+0000 7f7beb7fe700 1 -- 172.21.15.89:0/3641996359 reap_dead start 2023-10-10T20:39:38.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.404+0000 7f7bebfff700 1 -- 172.21.15.89:0/3641996359 shutdown_connections 2023-10-10T20:39:38.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.404+0000 7f7bebfff700 1 -- 172.21.15.89:0/3641996359 >> 172.21.15.89:0/3641996359 conn(0x7f7bec0d0660 msgr2=0x7f7bec0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:38.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.404+0000 7f7bebfff700 1 -- 172.21.15.89:0/3641996359 shutdown_connections 2023-10-10T20:39:38.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.404+0000 7f7bebfff700 1 -- 172.21.15.89:0/3641996359 wait complete. 2023-10-10T20:39:38.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.405+0000 7f7bebfff700 1 Processor -- start 2023-10-10T20:39:38.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.405+0000 7f7bebfff700 1 -- start start 2023-10-10T20:39:38.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.406+0000 7f7bebfff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bd8003fe0 0x7f7bec0c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:38.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.406+0000 7f7bebfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bec15bae0 0x7f7bec0c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:38.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.406+0000 7f7bebfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bec0c70e0 0x7f7bec169f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:38.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.406+0000 7f7bebfff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec0c6710 con 0x7f7bec15bae0 2023-10-10T20:39:38.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.406+0000 7f7bebfff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec15bf50 con 0x7f7bec0c70e0 2023-10-10T20:39:38.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.406+0000 7f7bebfff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec0c5b00 con 0x7f7bd8003fe0 2023-10-10T20:39:38.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.406+0000 7f7be27fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bec15bae0 0x7f7bec0c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:38.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.406+0000 7f7be27fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bec15bae0 0x7f7bec0c6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41736/0 (socket says 172.21.15.89:41736) 2023-10-10T20:39:38.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.406+0000 7f7be27fc700 1 -- 172.21.15.89:0/4265107925 learned_addr learned my addr 172.21.15.89:0/4265107925 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:38.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.406+0000 7f7beaffd700 1 --2- 172.21.15.89:0/4265107925 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bd8003fe0 0x7f7bec0c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:38.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.406+0000 7f7be27fc700 1 -- 172.21.15.89:0/4265107925 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bd8003fe0 msgr2=0x7f7bec0c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:38.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.406+0000 7f7be27fc700 1 --2- 172.21.15.89:0/4265107925 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bd8003fe0 0x7f7bec0c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.406+0000 7f7be27fc700 1 -- 172.21.15.89:0/4265107925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bec0c70e0 msgr2=0x7f7bec169f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:38.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.406+0000 7f7beb7fe700 1 --2- 172.21.15.89:0/4265107925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bec0c70e0 0x7f7bec169f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:38.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.406+0000 7f7be27fc700 1 --2- 172.21.15.89:0/4265107925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bec0c70e0 0x7f7bec169f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.406+0000 7f7be27fc700 1 -- 172.21.15.89:0/4265107925 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec0c5de0 con 0x7f7bec15bae0 2023-10-10T20:39:38.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.407+0000 7f7be27fc700 1 --2- 172.21.15.89:0/4265107925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bec15bae0 0x7f7bec0c6ba0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8004730 tx=0x7f7bd401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:38.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.407+0000 7f7be37fe700 1 -- 172.21.15.89:0/4265107925 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4022070 con 0x7f7bec15bae0 2023-10-10T20:39:38.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.407+0000 7f7be37fe700 1 -- 172.21.15.89:0/4265107925 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd400e980 con 0x7f7bec15bae0 2023-10-10T20:39:38.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.407+0000 7f7be37fe700 1 -- 172.21.15.89:0/4265107925 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4027470 con 0x7f7bec15bae0 2023-10-10T20:39:38.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.407+0000 7f7bebfff700 1 -- 172.21.15.89:0/4265107925 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bec0c60c0 con 0x7f7bec15bae0 2023-10-10T20:39:38.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.407+0000 7f7bebfff700 1 -- 172.21.15.89:0/4265107925 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bec15b540 con 0x7f7bec15bae0 2023-10-10T20:39:38.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.408+0000 7f7bebfff700 1 -- 172.21.15.89:0/4265107925 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bec0c74b0 con 0x7f7bec15bae0 2023-10-10T20:39:38.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.408+0000 7f7be37fe700 1 -- 172.21.15.89:0/4265107925 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f7bd401c450 con 0x7f7bec15bae0 2023-10-10T20:39:38.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.408+0000 7f7be37fe700 1 --2- 172.21.15.89:0/4265107925 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7bcc067170 0x7f7bcc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:38.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.409+0000 7f7beaffd700 1 --2- 172.21.15.89:0/4265107925 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7bcc067170 0x7f7bcc069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:38.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.409+0000 7f7be37fe700 1 -- 172.21.15.89:0/4265107925 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7bd4086460 con 0x7f7bec15bae0 2023-10-10T20:39:38.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.409+0000 7f7beaffd700 1 --2- 172.21.15.89:0/4265107925 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7bcc067170 0x7f7bcc069620 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc001ed0 tx=0x7f7bdc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:38.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.413+0000 7f7be37fe700 1 -- 172.21.15.89:0/4265107925 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7bd40533a0 con 0x7f7bec15bae0 2023-10-10T20:39:38.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.598+0000 7f7bebfff700 1 -- 172.21.15.89:0/4265107925 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bec04c3c0 con 0x7f7bec15bae0 2023-10-10T20:39:38.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.599+0000 7f7be37fe700 1 -- 172.21.15.89:0/4265107925 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+15366 (secure 0 0 0) 0x7f7bd40569c0 con 0x7f7bec15bae0 2023-10-10T20:39:38.601 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:38.602 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":75,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:39:37.634838+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:39:35.989512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-10T20:39:36.595551+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:39:38.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.601+0000 7f7bebfff700 1 -- 172.21.15.89:0/4265107925 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7bcc067170 msgr2=0x7f7bcc069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:38.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.601+0000 7f7bebfff700 1 --2- 172.21.15.89:0/4265107925 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7bcc067170 0x7f7bcc069620 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc001ed0 tx=0x7f7bdc009040 comp rx=0 tx=0).stop 2023-10-10T20:39:38.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.601+0000 7f7bebfff700 1 -- 172.21.15.89:0/4265107925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bec15bae0 msgr2=0x7f7bec0c6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:38.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.601+0000 7f7bebfff700 1 --2- 172.21.15.89:0/4265107925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bec15bae0 0x7f7bec0c6ba0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8004730 tx=0x7f7bd401a000 comp rx=0 tx=0).stop 2023-10-10T20:39:38.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.601+0000 7f7bebfff700 1 -- 172.21.15.89:0/4265107925 shutdown_connections 2023-10-10T20:39:38.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.601+0000 7f7bebfff700 1 --2- 172.21.15.89:0/4265107925 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7bcc067170 0x7f7bcc069620 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.601+0000 7f7bebfff700 1 --2- 172.21.15.89:0/4265107925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bec0c70e0 0x7f7bec169f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.601+0000 7f7bebfff700 1 --2- 172.21.15.89:0/4265107925 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7bd8003fe0 0x7f7bec0c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.601+0000 7f7bebfff700 1 --2- 172.21.15.89:0/4265107925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bec15bae0 0x7f7bec0c6ba0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.601+0000 7f7bebfff700 1 -- 172.21.15.89:0/4265107925 >> 172.21.15.89:0/4265107925 conn(0x7f7bec0d0660 msgr2=0x7f7bec0b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:38.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.601+0000 7f7bebfff700 1 -- 172.21.15.89:0/4265107925 shutdown_connections 2023-10-10T20:39:38.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.601+0000 7f7bebfff700 1 -- 172.21.15.89:0/4265107925 wait complete. 2023-10-10T20:39:38.616 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:39:38.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.728+0000 7fbb92fe6700 1 Processor -- start 2023-10-10T20:39:38.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.728+0000 7fbb92fe6700 1 -- start start 2023-10-10T20:39:38.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.729+0000 7fbb92fe6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb8c159890 0x7fbb8c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:38.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.729+0000 7fbb92fe6700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbb8c069560 0x7fbb8c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:38.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.729+0000 7fbb92fe6700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb8c069ee0 con 0x7fbb8c159c60 2023-10-10T20:39:38.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.729+0000 7fbb92fe6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb8c15a160 con 0x7fbb8c159890 2023-10-10T20:39:38.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.729+0000 7fbb92fe6700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb8c0d13a0 con 0x7fbb8c069560 2023-10-10T20:39:38.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.729+0000 7fbb90d82700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb8c159890 0x7fbb8c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:38.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.729+0000 7fbb90d82700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb8c159890 0x7fbb8c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41750/0 (socket says 172.21.15.89:41750) 2023-10-10T20:39:38.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.729+0000 7fbb90d82700 1 -- 172.21.15.89:0/3174082766 learned_addr learned my addr 172.21.15.89:0/3174082766 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:38.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.730+0000 7fbb91583700 1 --2- 172.21.15.89:0/3174082766 >> v2:172.21.15.111:3300/0 conn(0x7fbb8c069560 0x7fbb8c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:38.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.730+0000 7fbb90d82700 1 -- 172.21.15.89:0/3174082766 >> v2:172.21.15.111:3300/0 conn(0x7fbb8c069560 msgr2=0x7fbb8c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:38.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.730+0000 7fbb90d82700 1 --2- 172.21.15.89:0/3174082766 >> v2:172.21.15.111:3300/0 conn(0x7fbb8c069560 0x7fbb8c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.730+0000 7fbb90d82700 1 -- 172.21.15.89:0/3174082766 >> v1:172.21.15.111:6789/0 conn(0x7fbb8c159c60 legacy=0x7fbb8c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:38.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.730+0000 7fbb90d82700 1 -- 172.21.15.89:0/3174082766 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c15e140 con 0x7fbb8c159890 2023-10-10T20:39:38.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.730+0000 7fbb90d82700 1 --2- 172.21.15.89:0/3174082766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c159890 0x7fbb8c15abd0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c06a890 tx=0x7fbb80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423d520586b3d0a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:38.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.730+0000 7fbb8b7fe700 1 -- 172.21.15.89:0/3174082766 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8000b820 con 0x7fbb8c159890 2023-10-10T20:39:38.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.730+0000 7fbb8b7fe700 1 -- 172.21.15.89:0/3174082766 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8000b980 con 0x7fbb8c159890 2023-10-10T20:39:38.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.730+0000 7fbb8b7fe700 1 -- 172.21.15.89:0/3174082766 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8000bd20 con 0x7fbb8c159890 2023-10-10T20:39:38.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.731+0000 7fbb92fe6700 1 -- 172.21.15.89:0/3174082766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c159890 msgr2=0x7fbb8c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:38.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.731+0000 7fbb92fe6700 1 --2- 172.21.15.89:0/3174082766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c159890 0x7fbb8c15abd0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c06a890 tx=0x7fbb80009250 comp rx=0 tx=0).stop 2023-10-10T20:39:38.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.731+0000 7fbb92fe6700 1 -- 172.21.15.89:0/3174082766 shutdown_connections 2023-10-10T20:39:38.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.731+0000 7fbb92fe6700 1 --2- 172.21.15.89:0/3174082766 >> v2:172.21.15.111:3300/0 conn(0x7fbb8c069560 0x7fbb8c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.731+0000 7fbb92fe6700 1 --2- 172.21.15.89:0/3174082766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c159890 0x7fbb8c15abd0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.731+0000 7fbb92fe6700 1 -- 172.21.15.89:0/3174082766 >> 172.21.15.89:0/3174082766 conn(0x7fbb8c068e20 msgr2=0x7fbb8c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:38.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.731+0000 7fbb92fe6700 1 -- 172.21.15.89:0/3174082766 shutdown_connections 2023-10-10T20:39:38.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.731+0000 7fbb92fe6700 1 -- 172.21.15.89:0/3174082766 wait complete. 2023-10-10T20:39:38.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.732+0000 7fbb92fe6700 1 Processor -- start 2023-10-10T20:39:38.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.732+0000 7fbb92fe6700 1 -- start start 2023-10-10T20:39:38.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb92fe6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb8c069560 0x7fbb8c0cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:38.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb92fe6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb8c159890 0x7fbb8c14d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:38.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb92fe6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c159c60 0x7fbb8c0b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:38.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb92fe6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c0cfa30 con 0x7fbb8c159c60 2023-10-10T20:39:38.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb92fe6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c0cfec0 con 0x7fbb8c069560 2023-10-10T20:39:38.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb92fe6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c0cee20 con 0x7fbb8c159890 2023-10-10T20:39:38.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb91583700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c159c60 0x7fbb8c0b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:38.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb91583700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c159c60 0x7fbb8c0b1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41760/0 (socket says 172.21.15.89:41760) 2023-10-10T20:39:38.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb91583700 1 -- 172.21.15.89:0/1740140997 learned_addr learned my addr 172.21.15.89:0/1740140997 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:38.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb91583700 1 -- 172.21.15.89:0/1740140997 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb8c159890 msgr2=0x7fbb8c14d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:38.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb91583700 1 --2- 172.21.15.89:0/1740140997 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb8c159890 0x7fbb8c14d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb91583700 1 -- 172.21.15.89:0/1740140997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb8c069560 msgr2=0x7fbb8c0cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:38.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb90d82700 1 --2- 172.21.15.89:0/1740140997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb8c069560 0x7fbb8c0cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:38.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb91583700 1 --2- 172.21.15.89:0/1740140997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb8c069560 0x7fbb8c0cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb91583700 1 -- 172.21.15.89:0/1740140997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb800094f0 con 0x7fbb8c159c60 2023-10-10T20:39:38.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb91583700 1 --2- 172.21.15.89:0/1740140997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c159c60 0x7fbb8c0b1dc0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009fa0 tx=0x7fbb84009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:38.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb88ff9700 1 -- 172.21.15.89:0/1740140997 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8400ed20 con 0x7fbb8c159c60 2023-10-10T20:39:38.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb88ff9700 1 -- 172.21.15.89:0/1740140997 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8400ee80 con 0x7fbb8c159c60 2023-10-10T20:39:38.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb92fe6700 1 -- 172.21.15.89:0/1740140997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8c0cf100 con 0x7fbb8c159c60 2023-10-10T20:39:38.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb88ff9700 1 -- 172.21.15.89:0/1740140997 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84014670 con 0x7fbb8c159c60 2023-10-10T20:39:38.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.733+0000 7fbb92fe6700 1 -- 172.21.15.89:0/1740140997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8c0cf6c0 con 0x7fbb8c159c60 2023-10-10T20:39:38.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.734+0000 7fbb92fe6700 1 -- 172.21.15.89:0/1740140997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8c0cfbe0 con 0x7fbb8c159c60 2023-10-10T20:39:38.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.735+0000 7fbb88ff9700 1 -- 172.21.15.89:0/1740140997 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fbb840149e0 con 0x7fbb8c159c60 2023-10-10T20:39:38.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.735+0000 7fbb88ff9700 1 --2- 172.21.15.89:0/1740140997 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb70067170 0x7fbb70069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:38.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.735+0000 7fbb88ff9700 1 -- 172.21.15.89:0/1740140997 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbb84085020 con 0x7fbb8c159c60 2023-10-10T20:39:38.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.735+0000 7fbb90d82700 1 --2- 172.21.15.89:0/1740140997 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb70067170 0x7fbb70069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:38.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.736+0000 7fbb90d82700 1 --2- 172.21.15.89:0/1740140997 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb70067170 0x7fbb70069620 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c06a890 tx=0x7fbb80016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:38.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.739+0000 7fbb88ff9700 1 -- 172.21.15.89:0/1740140997 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb8c0cfbe0 con 0x7fbb8c159c60 2023-10-10T20:39:38.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.926+0000 7fbb92fe6700 1 -- 172.21.15.89:0/1740140997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb8c04c3c0 con 0x7fbb8c159c60 2023-10-10T20:39:38.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.927+0000 7fbb88ff9700 1 -- 172.21.15.89:0/1740140997 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+15366 (secure 0 0 0) 0x7fbb84052000 con 0x7fbb8c159c60 2023-10-10T20:39:38.929 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:38.929 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":75,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:39:37.634838+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:39:35.989512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-10T20:39:36.595551+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:39:38.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.929+0000 7fbb92fe6700 1 -- 172.21.15.89:0/1740140997 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb70067170 msgr2=0x7fbb70069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:38.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.929+0000 7fbb92fe6700 1 --2- 172.21.15.89:0/1740140997 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb70067170 0x7fbb70069620 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c06a890 tx=0x7fbb80016000 comp rx=0 tx=0).stop 2023-10-10T20:39:38.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.929+0000 7fbb92fe6700 1 -- 172.21.15.89:0/1740140997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c159c60 msgr2=0x7fbb8c0b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:38.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.929+0000 7fbb92fe6700 1 --2- 172.21.15.89:0/1740140997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c159c60 0x7fbb8c0b1dc0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009fa0 tx=0x7fbb84009900 comp rx=0 tx=0).stop 2023-10-10T20:39:38.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.929+0000 7fbb92fe6700 1 -- 172.21.15.89:0/1740140997 shutdown_connections 2023-10-10T20:39:38.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.929+0000 7fbb92fe6700 1 --2- 172.21.15.89:0/1740140997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c159c60 0x7fbb8c0b1dc0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.929+0000 7fbb92fe6700 1 --2- 172.21.15.89:0/1740140997 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb70067170 0x7fbb70069620 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.929+0000 7fbb92fe6700 1 --2- 172.21.15.89:0/1740140997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb8c069560 0x7fbb8c0cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.929+0000 7fbb92fe6700 1 --2- 172.21.15.89:0/1740140997 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb8c159890 0x7fbb8c14d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:38.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.929+0000 7fbb92fe6700 1 -- 172.21.15.89:0/1740140997 >> 172.21.15.89:0/1740140997 conn(0x7fbb8c068e20 msgr2=0x7fbb8c0b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:38.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.929+0000 7fbb92fe6700 1 -- 172.21.15.89:0/1740140997 shutdown_connections 2023-10-10T20:39:38.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:38.929+0000 7fbb92fe6700 1 -- 172.21.15.89:0/1740140997 wait complete. 2023-10-10T20:39:38.944 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:39:39.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.062+0000 7fcb34165700 1 Processor -- start 2023-10-10T20:39:39.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.062+0000 7fcb34165700 1 -- start start 2023-10-10T20:39:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.063+0000 7fcb34165700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcb2c159890 0x7fcb2c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.063+0000 7fcb34165700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb2c159c60 0x7fcb2c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.063+0000 7fcb34165700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb2c15a160 con 0x7fcb2c15db10 2023-10-10T20:39:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.063+0000 7fcb34165700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb2c0d13a0 con 0x7fcb2c159890 2023-10-10T20:39:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.063+0000 7fcb34165700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c15e140 con 0x7fcb2c159c60 2023-10-10T20:39:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.063+0000 7fcb32702700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcb2c15db10 0x7fcb2c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48124/0 (socket says 172.21.15.89:48124) 2023-10-10T20:39:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.063+0000 7fcb32702700 1 -- 172.21.15.89:0/1676030647 learned_addr learned my addr 172.21.15.89:0/1676030647 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.063+0000 7fcb30eff700 1 -- 172.21.15.89:0/1676030647 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 898746121 0 0) 0x7fcb2c15a160 con 0x7fcb2c15db10 2023-10-10T20:39:39.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb31f01700 1 --2- 172.21.15.89:0/1676030647 >> v2:172.21.15.111:3300/0 conn(0x7fcb2c159890 0x7fcb2c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:39.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb31700700 1 --2- 172.21.15.89:0/1676030647 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb2c159c60 0x7fcb2c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:39.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb30eff700 1 -- 172.21.15.89:0/1676030647 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb18003570 con 0x7fcb2c15db10 2023-10-10T20:39:39.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb30eff700 1 -- 172.21.15.89:0/1676030647 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2628899012 0 0) 0x7fcb18003570 con 0x7fcb2c15db10 2023-10-10T20:39:39.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb30eff700 1 -- 172.21.15.89:0/1676030647 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb2c159c60 msgr2=0x7fcb2c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:39.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb30eff700 1 --2- 172.21.15.89:0/1676030647 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb2c159c60 0x7fcb2c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:39.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb30eff700 1 -- 172.21.15.89:0/1676030647 >> v2:172.21.15.111:3300/0 conn(0x7fcb2c159890 msgr2=0x7fcb2c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:39.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb30eff700 1 --2- 172.21.15.89:0/1676030647 >> v2:172.21.15.111:3300/0 conn(0x7fcb2c159890 0x7fcb2c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:39.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb30eff700 1 -- 172.21.15.89:0/1676030647 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c15a3d0 con 0x7fcb2c15db10 2023-10-10T20:39:39.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb30eff700 1 -- 172.21.15.89:0/1676030647 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fcb24001470 con 0x7fcb2c15db10 2023-10-10T20:39:39.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb30eff700 1 -- 172.21.15.89:0/1676030647 >> v1:172.21.15.89:6789/0 conn(0x7fcb2c15db10 legacy=0x7fcb2c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:39.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb30eff700 1 --2- 172.21.15.89:0/1676030647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb18003a60 0x7fcb18003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:39.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb30eff700 1 --2- 172.21.15.89:0/1676030647 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb2c159c60 0x7fcb18004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:39.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb30eff700 1 --2- 172.21.15.89:0/1676030647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb18004d50 0x7fcb180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb30eff700 1 -- 172.21.15.89:0/1676030647 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c0d13a0 con 0x7fcb18004d50 2023-10-10T20:39:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb30eff700 1 -- 172.21.15.89:0/1676030647 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c15e140 con 0x7fcb18003a60 2023-10-10T20:39:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb30eff700 1 -- 172.21.15.89:0/1676030647 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1800ba60 con 0x7fcb2c159c60 2023-10-10T20:39:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb32702700 1 --2- 172.21.15.89:0/1676030647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb18004d50 0x7fcb180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb31700700 1 --2- 172.21.15.89:0/1676030647 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb2c159c60 0x7fcb18004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb31f01700 1 --2- 172.21.15.89:0/1676030647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb18003a60 0x7fcb18003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb32702700 1 -- 172.21.15.89:0/1676030647 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb2c159c60 msgr2=0x7fcb18004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb32702700 1 --2- 172.21.15.89:0/1676030647 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb2c159c60 0x7fcb18004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb32702700 1 -- 172.21.15.89:0/1676030647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb18003a60 msgr2=0x7fcb18003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb32702700 1 --2- 172.21.15.89:0/1676030647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb18003a60 0x7fcb18003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb32702700 1 -- 172.21.15.89:0/1676030647 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb18003570 con 0x7fcb18004d50 2023-10-10T20:39:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.064+0000 7fcb32702700 1 --2- 172.21.15.89:0/1676030647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb18004d50 0x7fcb180051c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400c3c0 tx=0x7fcb24001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86bdcff014484121 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.065+0000 7fcb30eff700 1 -- 172.21.15.89:0/1676030647 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2400cc30 con 0x7fcb18004d50 2023-10-10T20:39:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.065+0000 7fcb30eff700 1 -- 172.21.15.89:0/1676030647 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2400cdb0 con 0x7fcb18004d50 2023-10-10T20:39:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.065+0000 7fcb30eff700 1 -- 172.21.15.89:0/1676030647 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24005b70 con 0x7fcb18004d50 2023-10-10T20:39:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.065+0000 7fcb34165700 1 -- 172.21.15.89:0/1676030647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb18004d50 msgr2=0x7fcb180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.065+0000 7fcb34165700 1 --2- 172.21.15.89:0/1676030647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb18004d50 0x7fcb180051c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400c3c0 tx=0x7fcb24001e90 comp rx=0 tx=0).stop 2023-10-10T20:39:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.065+0000 7fcb32702700 1 -- 172.21.15.89:0/1676030647 reap_dead start 2023-10-10T20:39:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.065+0000 7fcb34165700 1 -- 172.21.15.89:0/1676030647 shutdown_connections 2023-10-10T20:39:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.065+0000 7fcb34165700 1 -- 172.21.15.89:0/1676030647 >> 172.21.15.89:0/1676030647 conn(0x7fcb2c068e20 msgr2=0x7fcb2c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:39.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.065+0000 7fcb34165700 1 -- 172.21.15.89:0/1676030647 shutdown_connections 2023-10-10T20:39:39.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.065+0000 7fcb34165700 1 -- 172.21.15.89:0/1676030647 wait complete. 2023-10-10T20:39:39.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.066+0000 7fcb34165700 1 Processor -- start 2023-10-10T20:39:39.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.066+0000 7fcb34165700 1 -- start start 2023-10-10T20:39:39.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.067+0000 7fcb34165700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb2c148c10 0x7fcb2c169ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:39.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.067+0000 7fcb34165700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb2c0b45c0 0x7fcb2c0ce120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:39.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.067+0000 7fcb34165700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb2c0ce660 0x7fcb2c1521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:39.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.067+0000 7fcb34165700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c16d310 con 0x7fcb2c148c10 2023-10-10T20:39:39.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.067+0000 7fcb34165700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c149540 con 0x7fcb2c0b45c0 2023-10-10T20:39:39.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.067+0000 7fcb34165700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c0b4a30 con 0x7fcb2c0ce660 2023-10-10T20:39:39.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.067+0000 7fcb31f01700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb2c148c10 0x7fcb2c169ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:39.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.067+0000 7fcb31f01700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb2c148c10 0x7fcb2c169ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41768/0 (socket says 172.21.15.89:41768) 2023-10-10T20:39:39.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.067+0000 7fcb31f01700 1 -- 172.21.15.89:0/2699727085 learned_addr learned my addr 172.21.15.89:0/2699727085 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:39.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.067+0000 7fcb32702700 1 --2- 172.21.15.89:0/2699727085 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb2c0ce660 0x7fcb2c1521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:39.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.067+0000 7fcb31700700 1 --2- 172.21.15.89:0/2699727085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb2c0b45c0 0x7fcb2c0ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:39.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.067+0000 7fcb31f01700 1 -- 172.21.15.89:0/2699727085 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb2c0ce660 msgr2=0x7fcb2c1521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:39.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.067+0000 7fcb31f01700 1 --2- 172.21.15.89:0/2699727085 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb2c0ce660 0x7fcb2c1521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:39.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.067+0000 7fcb31f01700 1 -- 172.21.15.89:0/2699727085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb2c0b45c0 msgr2=0x7fcb2c0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:39.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.067+0000 7fcb31f01700 1 --2- 172.21.15.89:0/2699727085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb2c0b45c0 0x7fcb2c0ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:39.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.067+0000 7fcb31f01700 1 -- 172.21.15.89:0/2699727085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c16a230 con 0x7fcb2c148c10 2023-10-10T20:39:39.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.067+0000 7fcb32702700 1 --2- 172.21.15.89:0/2699727085 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb2c0ce660 0x7fcb2c1521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:39.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.067+0000 7fcb31f01700 1 --2- 172.21.15.89:0/2699727085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb2c148c10 0x7fcb2c169ba0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009fd0 tx=0x7fcb2800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:39.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.068+0000 7fcb21ffb700 1 -- 172.21.15.89:0/2699727085 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2800e040 con 0x7fcb2c148c10 2023-10-10T20:39:39.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.068+0000 7fcb21ffb700 1 -- 172.21.15.89:0/2699727085 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2800f040 con 0x7fcb2c148c10 2023-10-10T20:39:39.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.068+0000 7fcb34165700 1 -- 172.21.15.89:0/2699727085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2c16a510 con 0x7fcb2c148c10 2023-10-10T20:39:39.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.068+0000 7fcb21ffb700 1 -- 172.21.15.89:0/2699727085 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28015430 con 0x7fcb2c148c10 2023-10-10T20:39:39.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.068+0000 7fcb34165700 1 -- 172.21.15.89:0/2699727085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2c16aad0 con 0x7fcb2c148c10 2023-10-10T20:39:39.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.069+0000 7fcb34165700 1 -- 172.21.15.89:0/2699727085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2c16d5f0 con 0x7fcb2c148c10 2023-10-10T20:39:39.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.069+0000 7fcb21ffb700 1 -- 172.21.15.89:0/2699727085 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fcb28015860 con 0x7fcb2c148c10 2023-10-10T20:39:39.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.070+0000 7fcb21ffb700 1 --2- 172.21.15.89:0/2699727085 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb10067170 0x7fcb10069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:39.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.070+0000 7fcb21ffb700 1 -- 172.21.15.89:0/2699727085 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcb28085690 con 0x7fcb2c148c10 2023-10-10T20:39:39.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.070+0000 7fcb31700700 1 --2- 172.21.15.89:0/2699727085 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb10067170 0x7fcb10069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:39.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.070+0000 7fcb31700700 1 --2- 172.21.15.89:0/2699727085 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb10067170 0x7fcb10069620 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c16d140 tx=0x7fcb1c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:39.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.074+0000 7fcb21ffb700 1 -- 172.21.15.89:0/2699727085 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb28052560 con 0x7fcb2c148c10 2023-10-10T20:39:39.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:39.302+0000 7fcb34165700 1 -- 172.21.15.89:0/2699727085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcb2c04c3c0 con 0x7fcb2c148c10 2023-10-10T20:39:40.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.271+0000 7fcb21ffb700 1 -- 172.21.15.89:0/2699727085 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcb28055b80 con 0x7fcb2c148c10 2023-10-10T20:39:40.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.274+0000 7fcb34165700 1 -- 172.21.15.89:0/2699727085 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb10067170 msgr2=0x7fcb10069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:40.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.274+0000 7fcb34165700 1 --2- 172.21.15.89:0/2699727085 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb10067170 0x7fcb10069620 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c16d140 tx=0x7fcb1c008040 comp rx=0 tx=0).stop 2023-10-10T20:39:40.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.274+0000 7fcb34165700 1 -- 172.21.15.89:0/2699727085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb2c148c10 msgr2=0x7fcb2c169ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:40.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.274+0000 7fcb34165700 1 --2- 172.21.15.89:0/2699727085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb2c148c10 0x7fcb2c169ba0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009fd0 tx=0x7fcb2800b040 comp rx=0 tx=0).stop 2023-10-10T20:39:40.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.274+0000 7fcb34165700 1 -- 172.21.15.89:0/2699727085 shutdown_connections 2023-10-10T20:39:40.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.274+0000 7fcb34165700 1 --2- 172.21.15.89:0/2699727085 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb2c0ce660 0x7fcb2c1521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:40.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.274+0000 7fcb34165700 1 --2- 172.21.15.89:0/2699727085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb2c148c10 0x7fcb2c169ba0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:40.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.274+0000 7fcb34165700 1 --2- 172.21.15.89:0/2699727085 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb10067170 0x7fcb10069620 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:40.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.274+0000 7fcb34165700 1 --2- 172.21.15.89:0/2699727085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb2c0b45c0 0x7fcb2c0ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:40.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.274+0000 7fcb34165700 1 -- 172.21.15.89:0/2699727085 >> 172.21.15.89:0/2699727085 conn(0x7fcb2c068e20 msgr2=0x7fcb2c15bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:40.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.274+0000 7fcb34165700 1 -- 172.21.15.89:0/2699727085 shutdown_connections 2023-10-10T20:39:40.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.274+0000 7fcb34165700 1 -- 172.21.15.89:0/2699727085 wait complete. 2023-10-10T20:39:40.291 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:39:40.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.412+0000 7f442e564700 1 Processor -- start 2023-10-10T20:39:40.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.412+0000 7f442e564700 1 -- start start 2023-10-10T20:39:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.413+0000 7f442e564700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4428069560 0x7f44280699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.413+0000 7f442e564700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4428069ee0 con 0x7f4428159c60 2023-10-10T20:39:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.413+0000 7f442e564700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f442815a160 con 0x7f4428159890 2023-10-10T20:39:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.413+0000 7f442e564700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44280d13a0 con 0x7f4428069560 2023-10-10T20:39:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.413+0000 7f44277fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4428159c60 0x7f442815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48140/0 (socket says 172.21.15.89:48140) 2023-10-10T20:39:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.413+0000 7f44277fe700 1 -- 172.21.15.89:0/738324867 learned_addr learned my addr 172.21.15.89:0/738324867 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:40.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.413+0000 7f442cb01700 1 --2- 172.21.15.89:0/738324867 >> v2:172.21.15.111:3300/0 conn(0x7f4428069560 0x7f44280699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:40.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.413+0000 7f4426ffd700 1 -- 172.21.15.89:0/738324867 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3215537309 0 0) 0x7f4428069ee0 con 0x7f4428159c60 2023-10-10T20:39:40.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.413+0000 7f4426ffd700 1 -- 172.21.15.89:0/738324867 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4410003570 con 0x7f4428159c60 2023-10-10T20:39:40.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.414+0000 7f4426ffd700 1 -- 172.21.15.89:0/738324867 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875450977 0 0) 0x7f442815a160 con 0x7f4428159890 2023-10-10T20:39:40.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.414+0000 7f4426ffd700 1 -- 172.21.15.89:0/738324867 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4428069ee0 con 0x7f4428159890 2023-10-10T20:39:40.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.414+0000 7f4426ffd700 1 -- 172.21.15.89:0/738324867 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 353097117 0 0) 0x7f4410003570 con 0x7f4428159c60 2023-10-10T20:39:40.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.414+0000 7f442cb01700 1 -- 172.21.15.89:0/738324867 >> v1:172.21.15.111:6789/0 conn(0x7f4428159890 legacy=0x7f442815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:40.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.414+0000 7f442cb01700 1 -- 172.21.15.89:0/738324867 >> v1:172.21.15.89:6789/0 conn(0x7f4428159c60 legacy=0x7f442815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:40.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.414+0000 7f442cb01700 1 -- 172.21.15.89:0/738324867 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442815e140 con 0x7f4428069560 2023-10-10T20:39:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.414+0000 7f442cb01700 1 --2- 172.21.15.89:0/738324867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4428069560 0x7f44280699a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4420009e60 tx=0x7f4420009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=391d82621c40d5dc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.414+0000 7f4426ffd700 1 -- 172.21.15.89:0/738324867 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442000e070 con 0x7f4428069560 2023-10-10T20:39:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.414+0000 7f4426ffd700 1 -- 172.21.15.89:0/738324867 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442000b7a0 con 0x7f4428069560 2023-10-10T20:39:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.414+0000 7f4426ffd700 1 -- 172.21.15.89:0/738324867 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c001470 con 0x7f4428069560 2023-10-10T20:39:40.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.415+0000 7f442e564700 1 -- 172.21.15.89:0/738324867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4428069560 msgr2=0x7f44280699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:40.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.415+0000 7f442e564700 1 --2- 172.21.15.89:0/738324867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4428069560 0x7f44280699a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4420009e60 tx=0x7f4420009780 comp rx=0 tx=0).stop 2023-10-10T20:39:40.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.415+0000 7f442e564700 1 -- 172.21.15.89:0/738324867 shutdown_connections 2023-10-10T20:39:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.415+0000 7f442e564700 1 --2- 172.21.15.89:0/738324867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4428069560 0x7f44280699a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.415+0000 7f442e564700 1 -- 172.21.15.89:0/738324867 >> 172.21.15.89:0/738324867 conn(0x7f4428068e20 msgr2=0x7f44280d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.415+0000 7f442e564700 1 -- 172.21.15.89:0/738324867 shutdown_connections 2023-10-10T20:39:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.415+0000 7f442e564700 1 -- 172.21.15.89:0/738324867 wait complete. 2023-10-10T20:39:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.416+0000 7f442e564700 1 Processor -- start 2023-10-10T20:39:40.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.416+0000 7f442e564700 1 -- start start 2023-10-10T20:39:40.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f442e564700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4428069560 0x7f4428153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:40.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f442e564700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4428159890 0x7f44280cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:40.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f442e564700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4428159c60 0x7f44280cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:40.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f442e564700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4428153aa0 con 0x7f4428159890 2023-10-10T20:39:40.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f442e564700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442815d680 con 0x7f4428069560 2023-10-10T20:39:40.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f442e564700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442815b4d0 con 0x7f4428159c60 2023-10-10T20:39:40.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f44277fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4428159890 0x7f44280cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:40.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f44277fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4428159890 0x7f44280cf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41780/0 (socket says 172.21.15.89:41780) 2023-10-10T20:39:40.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f44277fe700 1 -- 172.21.15.89:0/2702682761 learned_addr learned my addr 172.21.15.89:0/2702682761 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:40.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f442cb01700 1 --2- 172.21.15.89:0/2702682761 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4428159c60 0x7f44280cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:40.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f4427fff700 1 --2- 172.21.15.89:0/2702682761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4428069560 0x7f4428153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:40.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f44277fe700 1 -- 172.21.15.89:0/2702682761 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4428159c60 msgr2=0x7f44280cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:40.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f44277fe700 1 --2- 172.21.15.89:0/2702682761 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4428159c60 0x7f44280cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:40.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f44277fe700 1 -- 172.21.15.89:0/2702682761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4428069560 msgr2=0x7f4428153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:40.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f44277fe700 1 --2- 172.21.15.89:0/2702682761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4428069560 0x7f4428153560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:40.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f44277fe700 1 -- 172.21.15.89:0/2702682761 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4420009a20 con 0x7f4428159890 2023-10-10T20:39:40.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f44277fe700 1 --2- 172.21.15.89:0/2702682761 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4428159890 0x7f44280cf520 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f441c00cb70 tx=0x7f441c001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:40.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f440ffff700 1 -- 172.21.15.89:0/2702682761 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c00ce80 con 0x7f4428159890 2023-10-10T20:39:40.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f442e564700 1 -- 172.21.15.89:0/2702682761 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f442815b7b0 con 0x7f4428159890 2023-10-10T20:39:40.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f440ffff700 1 -- 172.21.15.89:0/2702682761 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441c007150 con 0x7f4428159890 2023-10-10T20:39:40.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f440ffff700 1 -- 172.21.15.89:0/2702682761 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c0074f0 con 0x7f4428159890 2023-10-10T20:39:40.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.417+0000 7f442e564700 1 -- 172.21.15.89:0/2702682761 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442815d8d0 con 0x7f4428159890 2023-10-10T20:39:40.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.418+0000 7f442e564700 1 -- 172.21.15.89:0/2702682761 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f442815b940 con 0x7f4428159890 2023-10-10T20:39:40.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.419+0000 7f440ffff700 1 -- 172.21.15.89:0/2702682761 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f441c004830 con 0x7f4428159890 2023-10-10T20:39:40.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.419+0000 7f440ffff700 1 --2- 172.21.15.89:0/2702682761 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4414067170 0x7f4414069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:40.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.419+0000 7f4427fff700 1 --2- 172.21.15.89:0/2702682761 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4414067170 0x7f4414069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:40.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.419+0000 7f440ffff700 1 -- 172.21.15.89:0/2702682761 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f441c083a30 con 0x7f4428159890 2023-10-10T20:39:40.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.420+0000 7f4427fff700 1 --2- 172.21.15.89:0/2702682761 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4414067170 0x7f4414069620 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4418007fd0 tx=0x7f4418007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:40.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.423+0000 7f440ffff700 1 -- 172.21.15.89:0/2702682761 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f441c050a10 con 0x7f4428159890 2023-10-10T20:39:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:40.648+0000 7f442e564700 1 -- 172.21.15.89:0/2702682761 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f442804c3c0 con 0x7f4428159890 2023-10-10T20:39:41.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.276+0000 7f440ffff700 1 -- 172.21.15.89:0/2702682761 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7f441c018020 con 0x7f4428159890 2023-10-10T20:39:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.279+0000 7f442e564700 1 -- 172.21.15.89:0/2702682761 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4414067170 msgr2=0x7f4414069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.279+0000 7f442e564700 1 --2- 172.21.15.89:0/2702682761 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4414067170 0x7f4414069620 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4418007fd0 tx=0x7f4418007d50 comp rx=0 tx=0).stop 2023-10-10T20:39:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.279+0000 7f442e564700 1 -- 172.21.15.89:0/2702682761 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4428159890 msgr2=0x7f44280cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:41.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.279+0000 7f442e564700 1 --2- 172.21.15.89:0/2702682761 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4428159890 0x7f44280cf520 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f441c00cb70 tx=0x7f441c001ab0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.279+0000 7f442e564700 1 -- 172.21.15.89:0/2702682761 shutdown_connections 2023-10-10T20:39:41.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.279+0000 7f442e564700 1 --2- 172.21.15.89:0/2702682761 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4428159c60 0x7f44280cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.279+0000 7f442e564700 1 --2- 172.21.15.89:0/2702682761 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4414067170 0x7f4414069620 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.279+0000 7f442e564700 1 --2- 172.21.15.89:0/2702682761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4428069560 0x7f4428153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.279+0000 7f442e564700 1 --2- 172.21.15.89:0/2702682761 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4428159890 0x7f44280cf520 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.279+0000 7f442e564700 1 -- 172.21.15.89:0/2702682761 >> 172.21.15.89:0/2702682761 conn(0x7f4428068e20 msgr2=0x7f44280c6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:41.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.279+0000 7f442e564700 1 -- 172.21.15.89:0/2702682761 shutdown_connections 2023-10-10T20:39:41.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.279+0000 7f442e564700 1 -- 172.21.15.89:0/2702682761 wait complete. 2023-10-10T20:39:41.295 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:39:41.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.411+0000 7f54a26e7700 1 Processor -- start 2023-10-10T20:39:41.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.411+0000 7f54a26e7700 1 -- start start 2023-10-10T20:39:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.412+0000 7f54a26e7700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f549c159890 0x7f549c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:41.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.412+0000 7f54a26e7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549c15a160 con 0x7f549c069560 2023-10-10T20:39:41.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.412+0000 7f54a26e7700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549c0d13a0 con 0x7f549c159c60 2023-10-10T20:39:41.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.412+0000 7f54a26e7700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549c15e140 con 0x7f549c159890 2023-10-10T20:39:41.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.412+0000 7f54a0c84700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f549c069560 0x7f549c0ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48154/0 (socket says 172.21.15.89:48154) 2023-10-10T20:39:41.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.412+0000 7f54a0c84700 1 -- 172.21.15.89:0/366098046 learned_addr learned my addr 172.21.15.89:0/366098046 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:41.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.412+0000 7f549affd700 1 -- 172.21.15.89:0/366098046 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 250485835 0 0) 0x7f549c15a160 con 0x7f549c069560 2023-10-10T20:39:41.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549affd700 1 -- 172.21.15.89:0/366098046 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5484003570 con 0x7f549c069560 2023-10-10T20:39:41.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549bfff700 1 --2- 172.21.15.89:0/366098046 >> v2:172.21.15.111:3300/0 conn(0x7f549c159890 0x7f549c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:41.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549affd700 1 -- 172.21.15.89:0/366098046 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 25905029 0 0) 0x7f5484003570 con 0x7f549c069560 2023-10-10T20:39:41.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549affd700 1 -- 172.21.15.89:0/366098046 >> v2:172.21.15.111:3300/0 conn(0x7f549c159890 msgr2=0x7f549c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549affd700 1 --2- 172.21.15.89:0/366098046 >> v2:172.21.15.111:3300/0 conn(0x7f549c159890 0x7f549c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549affd700 1 -- 172.21.15.89:0/366098046 >> v1:172.21.15.111:6789/0 conn(0x7f549c159c60 legacy=0x7f549c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549affd700 1 -- 172.21.15.89:0/366098046 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c15a3d0 con 0x7f549c069560 2023-10-10T20:39:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549affd700 1 -- 172.21.15.89:0/366098046 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f54940017f0 con 0x7f549c069560 2023-10-10T20:39:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549affd700 1 -- 172.21.15.89:0/366098046 >> v1:172.21.15.89:6789/0 conn(0x7f549c069560 legacy=0x7f549c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:41.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549affd700 1 --2- 172.21.15.89:0/366098046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5484003a60 0x7f5484003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:41.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549affd700 1 --2- 172.21.15.89:0/366098046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5484004810 0x7f5484004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:41.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549affd700 1 --2- 172.21.15.89:0/366098046 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54840051c0 0x7f548400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:41.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549affd700 1 -- 172.21.15.89:0/366098046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c15e140 con 0x7f5484003a60 2023-10-10T20:39:41.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549affd700 1 -- 172.21.15.89:0/366098046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548400c0d0 con 0x7f5484004810 2023-10-10T20:39:41.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549affd700 1 -- 172.21.15.89:0/366098046 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548400c290 con 0x7f54840051c0 2023-10-10T20:39:41.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549bfff700 1 --2- 172.21.15.89:0/366098046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5484003a60 0x7f5484003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:41.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549b7fe700 1 --2- 172.21.15.89:0/366098046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5484004810 0x7f5484004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:41.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f54a0c84700 1 --2- 172.21.15.89:0/366098046 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54840051c0 0x7f548400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:41.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549bfff700 1 -- 172.21.15.89:0/366098046 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54840051c0 msgr2=0x7f548400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:41.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549bfff700 1 --2- 172.21.15.89:0/366098046 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54840051c0 0x7f548400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549bfff700 1 -- 172.21.15.89:0/366098046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5484004810 msgr2=0x7f5484004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:41.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549bfff700 1 --2- 172.21.15.89:0/366098046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5484004810 0x7f5484004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549bfff700 1 -- 172.21.15.89:0/366098046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5484003570 con 0x7f5484003a60 2023-10-10T20:39:41.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f54a0c84700 1 -- 172.21.15.89:0/366098046 reap_dead start 2023-10-10T20:39:41.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.413+0000 7f549bfff700 1 --2- 172.21.15.89:0/366098046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5484003a60 0x7f5484003e70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f548c009c00 tx=0x7f548c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567aea96e59473f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:41.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.414+0000 7f549affd700 1 -- 172.21.15.89:0/366098046 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c00e710 con 0x7f5484003a60 2023-10-10T20:39:41.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.414+0000 7f549affd700 1 -- 172.21.15.89:0/366098046 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548c00e870 con 0x7f5484003a60 2023-10-10T20:39:41.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.414+0000 7f549affd700 1 -- 172.21.15.89:0/366098046 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c00ec30 con 0x7f5484003a60 2023-10-10T20:39:41.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.414+0000 7f54a26e7700 1 -- 172.21.15.89:0/366098046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5484003a60 msgr2=0x7f5484003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:41.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.414+0000 7f54a26e7700 1 --2- 172.21.15.89:0/366098046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5484003a60 0x7f5484003e70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f548c009c00 tx=0x7f548c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.414+0000 7f54a26e7700 1 -- 172.21.15.89:0/366098046 shutdown_connections 2023-10-10T20:39:41.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.414+0000 7f54a26e7700 1 --2- 172.21.15.89:0/366098046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5484003a60 0x7f5484003e70 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.414+0000 7f54a26e7700 1 -- 172.21.15.89:0/366098046 >> 172.21.15.89:0/366098046 conn(0x7f549c068e20 msgr2=0x7f549c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:41.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.414+0000 7f54a26e7700 1 -- 172.21.15.89:0/366098046 shutdown_connections 2023-10-10T20:39:41.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.414+0000 7f54a26e7700 1 -- 172.21.15.89:0/366098046 wait complete. 2023-10-10T20:39:41.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.415+0000 7f54a26e7700 1 Processor -- start 2023-10-10T20:39:41.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.415+0000 7f54a26e7700 1 -- start start 2023-10-10T20:39:41.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.416+0000 7f54a26e7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5484003a60 0x7f549c16f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:41.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.416+0000 7f54a26e7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f549c0ca600 0x7f549c0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:41.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.416+0000 7f54a26e7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f549c1523d0 0x7f549c170b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:41.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.416+0000 7f54a26e7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0b6300 con 0x7f549c0ca600 2023-10-10T20:39:41.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.416+0000 7f54a26e7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0b59d0 con 0x7f5484003a60 2023-10-10T20:39:41.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.416+0000 7f54a26e7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0b5cb0 con 0x7f549c1523d0 2023-10-10T20:39:41.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.416+0000 7f549b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f549c0ca600 0x7f549c0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:41.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.416+0000 7f549b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f549c0ca600 0x7f549c0af950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41802/0 (socket says 172.21.15.89:41802) 2023-10-10T20:39:41.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.416+0000 7f549b7fe700 1 -- 172.21.15.89:0/1055684613 learned_addr learned my addr 172.21.15.89:0/1055684613 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:41.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.416+0000 7f54a0c84700 1 --2- 172.21.15.89:0/1055684613 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f549c1523d0 0x7f549c170b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:41.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.416+0000 7f549bfff700 1 --2- 172.21.15.89:0/1055684613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5484003a60 0x7f549c16f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:41.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.416+0000 7f549b7fe700 1 -- 172.21.15.89:0/1055684613 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f549c1523d0 msgr2=0x7f549c170b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:41.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.416+0000 7f549b7fe700 1 --2- 172.21.15.89:0/1055684613 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f549c1523d0 0x7f549c170b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.416+0000 7f549b7fe700 1 -- 172.21.15.89:0/1055684613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5484003a60 msgr2=0x7f549c16f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:41.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.416+0000 7f549b7fe700 1 --2- 172.21.15.89:0/1055684613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5484003a60 0x7f549c16f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.416+0000 7f549b7fe700 1 -- 172.21.15.89:0/1055684613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c0b5f90 con 0x7f549c0ca600 2023-10-10T20:39:41.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.417+0000 7f549b7fe700 1 --2- 172.21.15.89:0/1055684613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f549c0ca600 0x7f549c0af950 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5490002950 tx=0x7f5490007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:41.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.417+0000 7f5483fff700 1 -- 172.21.15.89:0/1055684613 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5490003390 con 0x7f549c0ca600 2023-10-10T20:39:41.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.417+0000 7f5483fff700 1 -- 172.21.15.89:0/1055684613 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5490006820 con 0x7f549c0ca600 2023-10-10T20:39:41.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.417+0000 7f5483fff700 1 -- 172.21.15.89:0/1055684613 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5490006c00 con 0x7f549c0ca600 2023-10-10T20:39:41.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.417+0000 7f54a26e7700 1 -- 172.21.15.89:0/1055684613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c0afe90 con 0x7f549c0ca600 2023-10-10T20:39:41.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.417+0000 7f54a26e7700 1 -- 172.21.15.89:0/1055684613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c152840 con 0x7f549c0ca600 2023-10-10T20:39:41.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.418+0000 7f54a26e7700 1 -- 172.21.15.89:0/1055684613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549c0b6440 con 0x7f549c0ca600 2023-10-10T20:39:41.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.418+0000 7f5483fff700 1 -- 172.21.15.89:0/1055684613 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f5490007230 con 0x7f549c0ca600 2023-10-10T20:39:41.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.418+0000 7f5483fff700 1 --2- 172.21.15.89:0/1055684613 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5488067170 0x7f5488069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:41.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.418+0000 7f5483fff700 1 -- 172.21.15.89:0/1055684613 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f54900847c0 con 0x7f549c0ca600 2023-10-10T20:39:41.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.418+0000 7f549bfff700 1 --2- 172.21.15.89:0/1055684613 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5488067170 0x7f5488069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:41.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.419+0000 7f549bfff700 1 --2- 172.21.15.89:0/1055684613 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5488067170 0x7f5488069620 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f548c0091e0 tx=0x7f548c01a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:41.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.423+0000 7f5483fff700 1 -- 172.21.15.89:0/1055684613 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f549c0b6440 con 0x7f549c0ca600 2023-10-10T20:39:41.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.647+0000 7f54a26e7700 1 -- 172.21.15.89:0/1055684613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f549c0b6440 con 0x7f549c0ca600 2023-10-10T20:39:41.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.648+0000 7f5483fff700 1 -- 172.21.15.89:0/1055684613 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f549c0b6440 con 0x7f549c0ca600 2023-10-10T20:39:41.650 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:41.650 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5324,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/1607893755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":1607893755},{"type":"v1","addr":"172.21.15.111:6835","nonce":1607893755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6069,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/1480864169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":1480864169},{"type":"v1","addr":"172.21.15.89:6836","nonce":1480864169}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2347053559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2347053559},{"type":"v1","addr":"172.21.15.89:6837","nonce":2347053559}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:39:37.635170+0000","modified":"2023-10-10T20:39:41.271571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6087},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6087":{"gid":6087,"name":"b","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1070916496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1070916496},{"type":"v1","addr":"172.21.15.111:6834","nonce":1070916496}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-10T20:39:41.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.650+0000 7f54a26e7700 1 -- 172.21.15.89:0/1055684613 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5488067170 msgr2=0x7f5488069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:41.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.650+0000 7f54a26e7700 1 --2- 172.21.15.89:0/1055684613 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5488067170 0x7f5488069620 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f548c0091e0 tx=0x7f548c01a000 comp rx=0 tx=0).stop 2023-10-10T20:39:41.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.650+0000 7f54a26e7700 1 -- 172.21.15.89:0/1055684613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f549c0ca600 msgr2=0x7f549c0af950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:41.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.650+0000 7f54a26e7700 1 --2- 172.21.15.89:0/1055684613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f549c0ca600 0x7f549c0af950 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5490002950 tx=0x7f5490007d30 comp rx=0 tx=0).stop 2023-10-10T20:39:41.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.650+0000 7f54a26e7700 1 -- 172.21.15.89:0/1055684613 shutdown_connections 2023-10-10T20:39:41.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.650+0000 7f54a26e7700 1 --2- 172.21.15.89:0/1055684613 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f549c1523d0 0x7f549c170b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.650+0000 7f54a26e7700 1 --2- 172.21.15.89:0/1055684613 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5488067170 0x7f5488069620 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.650+0000 7f54a26e7700 1 --2- 172.21.15.89:0/1055684613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5484003a60 0x7f549c16f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.650+0000 7f54a26e7700 1 --2- 172.21.15.89:0/1055684613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f549c0ca600 0x7f549c0af950 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.650+0000 7f54a26e7700 1 -- 172.21.15.89:0/1055684613 >> 172.21.15.89:0/1055684613 conn(0x7f549c068e20 msgr2=0x7f549c15cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:41.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.650+0000 7f54a26e7700 1 -- 172.21.15.89:0/1055684613 shutdown_connections 2023-10-10T20:39:41.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.650+0000 7f54a26e7700 1 -- 172.21.15.89:0/1055684613 wait complete. 2023-10-10T20:39:41.655 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 46 2023-10-10T20:39:41.664 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:39:41.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.776+0000 7fa7499a6700 1 Processor -- start 2023-10-10T20:39:41.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.776+0000 7fa7499a6700 1 -- start start 2023-10-10T20:39:41.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.777+0000 7fa7499a6700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa744159890 0x7fa74415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:41.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.777+0000 7fa7499a6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa74415a160 con 0x7fa744069560 2023-10-10T20:39:41.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.777+0000 7fa7499a6700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7440d13a0 con 0x7fa744159c60 2023-10-10T20:39:41.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.777+0000 7fa7499a6700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa74415e140 con 0x7fa744159890 2023-10-10T20:39:41.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.777+0000 7fa7437fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa744069560 0x7fa7440ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48170/0 (socket says 172.21.15.89:48170) 2023-10-10T20:39:41.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.777+0000 7fa7437fe700 1 -- 172.21.15.89:0/3703565662 learned_addr learned my addr 172.21.15.89:0/3703565662 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:41.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa742ffd700 1 --2- 172.21.15.89:0/3703565662 >> v2:172.21.15.111:3300/0 conn(0x7fa744159890 0x7fa74415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:41.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa7427fc700 1 -- 172.21.15.89:0/3703565662 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283834100 0 0) 0x7fa74415a160 con 0x7fa744069560 2023-10-10T20:39:41.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa7427fc700 1 -- 172.21.15.89:0/3703565662 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa730003570 con 0x7fa744069560 2023-10-10T20:39:41.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa7427fc700 1 -- 172.21.15.89:0/3703565662 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1836145747 0 0) 0x7fa730003570 con 0x7fa744069560 2023-10-10T20:39:41.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa7427fc700 1 -- 172.21.15.89:0/3703565662 >> v2:172.21.15.111:3300/0 conn(0x7fa744159890 msgr2=0x7fa74415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:41.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa7427fc700 1 --2- 172.21.15.89:0/3703565662 >> v2:172.21.15.111:3300/0 conn(0x7fa744159890 0x7fa74415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa7427fc700 1 -- 172.21.15.89:0/3703565662 >> v1:172.21.15.111:6789/0 conn(0x7fa744159c60 legacy=0x7fa74415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:41.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa7427fc700 1 -- 172.21.15.89:0/3703565662 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74415a3d0 con 0x7fa744069560 2023-10-10T20:39:41.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa7427fc700 1 -- 172.21.15.89:0/3703565662 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fa73c0017f0 con 0x7fa744069560 2023-10-10T20:39:41.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa7427fc700 1 -- 172.21.15.89:0/3703565662 >> v1:172.21.15.89:6789/0 conn(0x7fa744069560 legacy=0x7fa7440ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:41.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa7427fc700 1 --2- 172.21.15.89:0/3703565662 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa730003a60 0x7fa730003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:41.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa7427fc700 1 --2- 172.21.15.89:0/3703565662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa730004810 0x7fa730004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:41.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa7427fc700 1 --2- 172.21.15.89:0/3703565662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7300051c0 0x7fa73000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:41.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa7427fc700 1 -- 172.21.15.89:0/3703565662 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74415e140 con 0x7fa7300051c0 2023-10-10T20:39:41.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa7427fc700 1 -- 172.21.15.89:0/3703565662 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73000c0d0 con 0x7fa730004810 2023-10-10T20:39:41.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa7427fc700 1 -- 172.21.15.89:0/3703565662 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73000c290 con 0x7fa730003a60 2023-10-10T20:39:41.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa7437fe700 1 --2- 172.21.15.89:0/3703565662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7300051c0 0x7fa73000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:41.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa73a7fc700 1 --2- 172.21.15.89:0/3703565662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa730004810 0x7fa730004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:41.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa742ffd700 1 --2- 172.21.15.89:0/3703565662 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa730003a60 0x7fa730003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:41.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.778+0000 7fa7437fe700 1 -- 172.21.15.89:0/3703565662 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa730003a60 msgr2=0x7fa730003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:41.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.779+0000 7fa7437fe700 1 --2- 172.21.15.89:0/3703565662 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa730003a60 0x7fa730003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.779+0000 7fa7437fe700 1 -- 172.21.15.89:0/3703565662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa730004810 msgr2=0x7fa730004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:41.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.779+0000 7fa7437fe700 1 --2- 172.21.15.89:0/3703565662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa730004810 0x7fa730004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.779+0000 7fa7437fe700 1 -- 172.21.15.89:0/3703565662 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa730003570 con 0x7fa7300051c0 2023-10-10T20:39:41.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.779+0000 7fa7437fe700 1 -- 172.21.15.89:0/3703565662 reap_dead start 2023-10-10T20:39:41.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.779+0000 7fa7437fe700 1 --2- 172.21.15.89:0/3703565662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7300051c0 0x7fa73000ba60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00a930 tx=0x7fa73c0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44e70df227279c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:41.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.779+0000 7fa7427fc700 1 -- 172.21.15.89:0/3703565662 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c0070d0 con 0x7fa7300051c0 2023-10-10T20:39:41.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.779+0000 7fa7427fc700 1 -- 172.21.15.89:0/3703565662 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73c007230 con 0x7fa7300051c0 2023-10-10T20:39:41.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.779+0000 7fa7427fc700 1 -- 172.21.15.89:0/3703565662 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c0075d0 con 0x7fa7300051c0 2023-10-10T20:39:41.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.779+0000 7fa7499a6700 1 -- 172.21.15.89:0/3703565662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7300051c0 msgr2=0x7fa73000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:41.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.779+0000 7fa7499a6700 1 --2- 172.21.15.89:0/3703565662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7300051c0 0x7fa73000ba60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00a930 tx=0x7fa73c0021d0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.779+0000 7fa7499a6700 1 -- 172.21.15.89:0/3703565662 shutdown_connections 2023-10-10T20:39:41.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.779+0000 7fa7499a6700 1 --2- 172.21.15.89:0/3703565662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7300051c0 0x7fa73000ba60 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.779+0000 7fa7499a6700 1 -- 172.21.15.89:0/3703565662 >> 172.21.15.89:0/3703565662 conn(0x7fa744068e20 msgr2=0x7fa7440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:41.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.779+0000 7fa7499a6700 1 -- 172.21.15.89:0/3703565662 shutdown_connections 2023-10-10T20:39:41.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.779+0000 7fa7499a6700 1 -- 172.21.15.89:0/3703565662 wait complete. 2023-10-10T20:39:41.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.781+0000 7fa7499a6700 1 Processor -- start 2023-10-10T20:39:41.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.781+0000 7fa7499a6700 1 -- start start 2023-10-10T20:39:41.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.781+0000 7fa7499a6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7300051c0 0x7fa7440cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:41.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.781+0000 7fa7499a6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7440ce210 0x7fa7440b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:41.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.781+0000 7fa7499a6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa7440b1d80 0x7fa744155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:41.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.781+0000 7fa7499a6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7440b6400 con 0x7fa7300051c0 2023-10-10T20:39:41.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.781+0000 7fa7499a6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7440ce680 con 0x7fa7440ce210 2023-10-10T20:39:41.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.781+0000 7fa7499a6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7440b3630 con 0x7fa7440b1d80 2023-10-10T20:39:41.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.781+0000 7fa742ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7300051c0 0x7fa7440cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:41.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.781+0000 7fa742ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7300051c0 0x7fa7440cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41832/0 (socket says 172.21.15.89:41832) 2023-10-10T20:39:41.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.782+0000 7fa742ffd700 1 -- 172.21.15.89:0/2135270885 learned_addr learned my addr 172.21.15.89:0/2135270885 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:41.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.782+0000 7fa7437fe700 1 --2- 172.21.15.89:0/2135270885 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa7440b1d80 0x7fa744155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.782+0000 7fa73a7fc700 1 --2- 172.21.15.89:0/2135270885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7440ce210 0x7fa7440b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.782+0000 7fa742ffd700 1 -- 172.21.15.89:0/2135270885 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa7440b1d80 msgr2=0x7fa744155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.782+0000 7fa742ffd700 1 --2- 172.21.15.89:0/2135270885 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa7440b1d80 0x7fa744155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.782+0000 7fa742ffd700 1 -- 172.21.15.89:0/2135270885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7440ce210 msgr2=0x7fa7440b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.782+0000 7fa742ffd700 1 --2- 172.21.15.89:0/2135270885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7440ce210 0x7fa7440b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.782+0000 7fa742ffd700 1 -- 172.21.15.89:0/2135270885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7440b3910 con 0x7fa7300051c0 2023-10-10T20:39:41.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.782+0000 7fa73a7fc700 1 --2- 172.21.15.89:0/2135270885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7440ce210 0x7fa7440b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:41.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.782+0000 7fa742ffd700 1 --2- 172.21.15.89:0/2135270885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7300051c0 0x7fa7440cf280 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa734011b10 tx=0x7fa7340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:41.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.782+0000 7fa73b7fe700 1 -- 172.21.15.89:0/2135270885 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73400f9d0 con 0x7fa7300051c0 2023-10-10T20:39:41.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.782+0000 7fa73b7fe700 1 -- 172.21.15.89:0/2135270885 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73400fb30 con 0x7fa7300051c0 2023-10-10T20:39:41.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.782+0000 7fa7499a6700 1 -- 172.21.15.89:0/2135270885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7440b3bf0 con 0x7fa7300051c0 2023-10-10T20:39:41.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.782+0000 7fa73b7fe700 1 -- 172.21.15.89:0/2135270885 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73400feb0 con 0x7fa7300051c0 2023-10-10T20:39:41.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.782+0000 7fa7499a6700 1 -- 172.21.15.89:0/2135270885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7440cf860 con 0x7fa7300051c0 2023-10-10T20:39:41.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.783+0000 7fa7499a6700 1 -- 172.21.15.89:0/2135270885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7440b4110 con 0x7fa7300051c0 2023-10-10T20:39:41.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.784+0000 7fa73b7fe700 1 -- 172.21.15.89:0/2135270885 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fa734014650 con 0x7fa7300051c0 2023-10-10T20:39:41.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.784+0000 7fa73b7fe700 1 --2- 172.21.15.89:0/2135270885 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa724067170 0x7fa724069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:41.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.784+0000 7fa73b7fe700 1 -- 172.21.15.89:0/2135270885 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa734018030 con 0x7fa7300051c0 2023-10-10T20:39:41.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.784+0000 7fa73a7fc700 1 --2- 172.21.15.89:0/2135270885 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa724067170 0x7fa724069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:41.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.784+0000 7fa73a7fc700 1 --2- 172.21.15.89:0/2135270885 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa724067170 0x7fa724069620 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa74415d3d0 tx=0x7fa72c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:41.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.788+0000 7fa73b7fe700 1 -- 172.21.15.89:0/2135270885 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7440b4110 con 0x7fa7300051c0 2023-10-10T20:39:41.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.974+0000 7fa7499a6700 1 -- 172.21.15.89:0/2135270885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7440b4110 con 0x7fa7300051c0 2023-10-10T20:39:41.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.975+0000 7fa73b7fe700 1 -- 172.21.15.89:0/2135270885 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+15353 (secure 0 0 0) 0x7fa7440b4110 con 0x7fa7300051c0 2023-10-10T20:39:41.977 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:41.978 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":76,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:39:39.250746+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:39:35.989512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-10T20:39:36.595551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:39:41.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.977+0000 7fa7499a6700 1 -- 172.21.15.89:0/2135270885 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa724067170 msgr2=0x7fa724069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:41.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.977+0000 7fa7499a6700 1 --2- 172.21.15.89:0/2135270885 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa724067170 0x7fa724069620 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa74415d3d0 tx=0x7fa72c009040 comp rx=0 tx=0).stop 2023-10-10T20:39:41.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.977+0000 7fa7499a6700 1 -- 172.21.15.89:0/2135270885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7300051c0 msgr2=0x7fa7440cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:41.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.977+0000 7fa7499a6700 1 --2- 172.21.15.89:0/2135270885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7300051c0 0x7fa7440cf280 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa734011b10 tx=0x7fa7340091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.977+0000 7fa7499a6700 1 -- 172.21.15.89:0/2135270885 shutdown_connections 2023-10-10T20:39:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.977+0000 7fa7499a6700 1 --2- 172.21.15.89:0/2135270885 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa7440b1d80 0x7fa744155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.977+0000 7fa7499a6700 1 --2- 172.21.15.89:0/2135270885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7300051c0 0x7fa7440cf280 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.977+0000 7fa7499a6700 1 --2- 172.21.15.89:0/2135270885 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa724067170 0x7fa724069620 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.977+0000 7fa7499a6700 1 --2- 172.21.15.89:0/2135270885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7440ce210 0x7fa7440b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:41.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.977+0000 7fa7499a6700 1 -- 172.21.15.89:0/2135270885 >> 172.21.15.89:0/2135270885 conn(0x7fa744068e20 msgr2=0x7fa744153730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:41.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.977+0000 7fa7499a6700 1 -- 172.21.15.89:0/2135270885 shutdown_connections 2023-10-10T20:39:41.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:41.977+0000 7fa7499a6700 1 -- 172.21.15.89:0/2135270885 wait complete. 2023-10-10T20:39:41.992 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:39:41.993 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:39:42.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.104+0000 7f160d9fe700 1 Processor -- start 2023-10-10T20:39:42.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.104+0000 7f160d9fe700 1 -- start start 2023-10-10T20:39:42.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.105+0000 7f160d9fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1608159420 0x7f16080b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:42.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.105+0000 7f160d9fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f160815d230 0x7f160815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:42.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.105+0000 7f160d9fe700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f160815dbd0 con 0x7f16081597f0 2023-10-10T20:39:42.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.105+0000 7f160d9fe700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16080b2980 con 0x7f160815d230 2023-10-10T20:39:42.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.105+0000 7f160d9fe700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16080b11d0 con 0x7f1608159420 2023-10-10T20:39:42.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.105+0000 7f160d1fd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f160815d230 0x7f160815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:42.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.105+0000 7f15fffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f16081597f0 0x7f16080b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48186/0 (socket says 172.21.15.89:48186) 2023-10-10T20:39:42.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.105+0000 7f15fffff700 1 -- 172.21.15.89:0/3850780577 learned_addr learned my addr 172.21.15.89:0/3850780577 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:42.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.105+0000 7f160c9fc700 1 --2- 172.21.15.89:0/3850780577 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1608159420 0x7f16080b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:42.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.106+0000 7f160d1fd700 1 -- 172.21.15.89:0/3850780577 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1608159420 msgr2=0x7f16080b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:42.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.106+0000 7f160d1fd700 1 --2- 172.21.15.89:0/3850780577 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1608159420 0x7f16080b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.106+0000 7f160d1fd700 1 -- 172.21.15.89:0/3850780577 >> v1:172.21.15.89:6789/0 conn(0x7f16081597f0 legacy=0x7f16080b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:42.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.106+0000 7f160d1fd700 1 -- 172.21.15.89:0/3850780577 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16080b14b0 con 0x7f160815d230 2023-10-10T20:39:42.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.106+0000 7f160d1fd700 1 --2- 172.21.15.89:0/3850780577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f160815d230 0x7f160815d690 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f16000098e0 tx=0x7f16000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9d40f79db89c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:42.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.106+0000 7f15ff7fe700 1 -- 172.21.15.89:0/3850780577 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1600009e80 con 0x7f160815d230 2023-10-10T20:39:42.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.106+0000 7f15ff7fe700 1 -- 172.21.15.89:0/3850780577 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160000b860 con 0x7f160815d230 2023-10-10T20:39:42.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.106+0000 7f15ff7fe700 1 -- 172.21.15.89:0/3850780577 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160000bbe0 con 0x7f160815d230 2023-10-10T20:39:42.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.106+0000 7f160d9fe700 1 -- 172.21.15.89:0/3850780577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f160815d230 msgr2=0x7f160815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:42.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.106+0000 7f160d9fe700 1 --2- 172.21.15.89:0/3850780577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f160815d230 0x7f160815d690 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f16000098e0 tx=0x7f16000091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.106+0000 7f160d9fe700 1 -- 172.21.15.89:0/3850780577 shutdown_connections 2023-10-10T20:39:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.106+0000 7f160d9fe700 1 --2- 172.21.15.89:0/3850780577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f160815d230 0x7f160815d690 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.106+0000 7f160d9fe700 1 --2- 172.21.15.89:0/3850780577 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1608159420 0x7f16080b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.106+0000 7f160d9fe700 1 -- 172.21.15.89:0/3850780577 >> 172.21.15.89:0/3850780577 conn(0x7f16080d0660 msgr2=0x7f16080d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.106+0000 7f160d9fe700 1 -- 172.21.15.89:0/3850780577 shutdown_connections 2023-10-10T20:39:42.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.106+0000 7f160d9fe700 1 -- 172.21.15.89:0/3850780577 wait complete. 2023-10-10T20:39:42.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.108+0000 7f160d9fe700 1 Processor -- start 2023-10-10T20:39:42.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.108+0000 7f160d9fe700 1 -- start start 2023-10-10T20:39:42.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.108+0000 7f160d9fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1608159420 0x7f16080ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:42.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.108+0000 7f160d9fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16081597f0 0x7f16080cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:42.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.108+0000 7f160d9fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f160815d230 0x7f160816a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:42.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.108+0000 7f160d9fe700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16080caf70 con 0x7f160815d230 2023-10-10T20:39:42.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.108+0000 7f160d9fe700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16080cb250 con 0x7f16081597f0 2023-10-10T20:39:42.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.108+0000 7f160d9fe700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16080cb530 con 0x7f1608159420 2023-10-10T20:39:42.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.109+0000 7f160d1fd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f160815d230 0x7f160816a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:42.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.109+0000 7f160d1fd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f160815d230 0x7f160816a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41852/0 (socket says 172.21.15.89:41852) 2023-10-10T20:39:42.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.109+0000 7f160d1fd700 1 -- 172.21.15.89:0/3358749619 learned_addr learned my addr 172.21.15.89:0/3358749619 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:42.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.109+0000 7f160c9fc700 1 --2- 172.21.15.89:0/3358749619 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1608159420 0x7f16080ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:42.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.109+0000 7f15fffff700 1 --2- 172.21.15.89:0/3358749619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16081597f0 0x7f16080cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:42.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.109+0000 7f160d1fd700 1 -- 172.21.15.89:0/3358749619 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1608159420 msgr2=0x7f16080ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:42.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.109+0000 7f160d1fd700 1 --2- 172.21.15.89:0/3358749619 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1608159420 0x7f16080ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.109+0000 7f160d1fd700 1 -- 172.21.15.89:0/3358749619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16081597f0 msgr2=0x7f16080cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:42.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.109+0000 7f160d1fd700 1 --2- 172.21.15.89:0/3358749619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16081597f0 0x7f16080cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.109+0000 7f160d1fd700 1 -- 172.21.15.89:0/3358749619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1600009420 con 0x7f160815d230 2023-10-10T20:39:42.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.109+0000 7f160c9fc700 1 --2- 172.21.15.89:0/3358749619 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1608159420 0x7f16080ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:42.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.109+0000 7f160d1fd700 1 --2- 172.21.15.89:0/3358749619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f160815d230 0x7f160816a010 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1608089340 tx=0x7f1600017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:42.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.109+0000 7f15fcff9700 1 -- 172.21.15.89:0/3358749619 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1600011630 con 0x7f160815d230 2023-10-10T20:39:42.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.109+0000 7f15fcff9700 1 -- 172.21.15.89:0/3358749619 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16000117b0 con 0x7f160815d230 2023-10-10T20:39:42.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.109+0000 7f15fcff9700 1 -- 172.21.15.89:0/3358749619 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1600011b70 con 0x7f160815d230 2023-10-10T20:39:42.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.109+0000 7f160d9fe700 1 -- 172.21.15.89:0/3358749619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16080cb810 con 0x7f160815d230 2023-10-10T20:39:42.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.109+0000 7f160d9fe700 1 -- 172.21.15.89:0/3358749619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1608155920 con 0x7f160815d230 2023-10-10T20:39:42.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.110+0000 7f160d9fe700 1 -- 172.21.15.89:0/3358749619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16080cbd10 con 0x7f160815d230 2023-10-10T20:39:42.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.110+0000 7f15fcff9700 1 -- 172.21.15.89:0/3358749619 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f16000253f0 con 0x7f160815d230 2023-10-10T20:39:42.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.111+0000 7f15fcff9700 1 --2- 172.21.15.89:0/3358749619 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f15f4067170 0x7f15f4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:42.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.111+0000 7f15fcff9700 1 -- 172.21.15.89:0/3358749619 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1600086620 con 0x7f160815d230 2023-10-10T20:39:42.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.111+0000 7f160c9fc700 1 --2- 172.21.15.89:0/3358749619 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f15f4067170 0x7f15f4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:42.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.111+0000 7f160c9fc700 1 --2- 172.21.15.89:0/3358749619 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f15f4067170 0x7f15f4069620 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1608156080 tx=0x7f1604007710 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:42.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.115+0000 7f15fcff9700 1 -- 172.21.15.89:0/3358749619 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1600053600 con 0x7f160815d230 2023-10-10T20:39:42.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.293+0000 7f160d9fe700 1 -- 172.21.15.89:0/3358749619 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f16080cb3c0 con 0x7f15f4067170 2023-10-10T20:39:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.300+0000 7f15fcff9700 1 -- 172.21.15.89:0/3358749619 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286833 (secure 0 0 0) 0x7f16080cb3c0 con 0x7f15f4067170 2023-10-10T20:39:42.306 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:42.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.303+0000 7f160d9fe700 1 -- 172.21.15.89:0/3358749619 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f15f4067170 msgr2=0x7f15f4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:42.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.303+0000 7f160d9fe700 1 --2- 172.21.15.89:0/3358749619 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f15f4067170 0x7f15f4069620 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1608156080 tx=0x7f1604007710 comp rx=0 tx=0).stop 2023-10-10T20:39:42.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.303+0000 7f160d9fe700 1 -- 172.21.15.89:0/3358749619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f160815d230 msgr2=0x7f160816a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:42.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.303+0000 7f160d9fe700 1 --2- 172.21.15.89:0/3358749619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f160815d230 0x7f160816a010 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1608089340 tx=0x7f1600017000 comp rx=0 tx=0).stop 2023-10-10T20:39:42.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.303+0000 7f160d9fe700 1 -- 172.21.15.89:0/3358749619 shutdown_connections 2023-10-10T20:39:42.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.303+0000 7f160d9fe700 1 --2- 172.21.15.89:0/3358749619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f160815d230 0x7f160816a010 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.304+0000 7f160d9fe700 1 --2- 172.21.15.89:0/3358749619 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1608159420 0x7f16080ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.304+0000 7f160d9fe700 1 --2- 172.21.15.89:0/3358749619 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f15f4067170 0x7f15f4069620 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.304+0000 7f160d9fe700 1 --2- 172.21.15.89:0/3358749619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16081597f0 0x7f16080cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.304+0000 7f160d9fe700 1 -- 172.21.15.89:0/3358749619 >> 172.21.15.89:0/3358749619 conn(0x7f16080d0660 msgr2=0x7f16081564f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:42.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.304+0000 7f160d9fe700 1 -- 172.21.15.89:0/3358749619 shutdown_connections 2023-10-10T20:39:42.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.304+0000 7f160d9fe700 1 -- 172.21.15.89:0/3358749619 wait complete. 2023-10-10T20:39:42.312 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:39:42.318 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2023-10-10T20:39:40.713576+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2390648,"kb_used_data":8248,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752584072,"statfs":{"total":773094113280,"available":770646089728,"internally_reserved":0,"allocated":8445952,"data_stored":4276440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.471190"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.262078+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:18:44.235474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261385+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:32:45.306980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253087+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:55:15.047824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258454+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:10:36.151908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254702+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:17.810574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258770+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:49:29.215389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257832+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:12:11.985278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257660+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258331+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:55:15.047824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261413+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261949+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:51:25.472971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259733+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:51.489549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253308+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:49:29.215389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.271693+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:27.148718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257856+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:56:02.724230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252996+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:22.796992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252806+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:21:45.349150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257789+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:56:39.215508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260126+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:09.824624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257801+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:55:16.441311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258944+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:45:29.295885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.266901+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:22.796992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258929+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:51:00.261125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.271424+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:13:05.408163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261055+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:10:42.178506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253538+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:45.015636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254993+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261378+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.263848+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:24.508374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.266498+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:32.888171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254896+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258291+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:44.990290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.255157+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.275787+0000","last_change":"2023-10-10T20:39:37.275787+0000","last_active":"2023-10-10T20:39:37.275787+0000","last_peered":"2023-10-10T20:39:37.275787+0000","last_clean":"2023-10-10T20:39:37.275787+0000","last_became_active":"2023-10-10T20:39:37.275528+0000","last_became_peered":"2023-10-10T20:39:37.275528+0000","last_unstale":"2023-10-10T20:39:37.275787+0000","last_undegraded":"2023-10-10T20:39:37.275787+0000","last_fullsized":"2023-10-10T20:39:37.275787+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:23.501900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260922+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.262406+0000","last_change":"2023-10-10T20:39:37.262406+0000","last_active":"2023-10-10T20:39:37.262406+0000","last_peered":"2023-10-10T20:39:37.262406+0000","last_clean":"2023-10-10T20:39:37.262406+0000","last_became_active":"2023-10-10T20:39:37.261561+0000","last_became_peered":"2023-10-10T20:39:37.261561+0000","last_unstale":"2023-10-10T20:39:37.262406+0000","last_undegraded":"2023-10-10T20:39:37.262406+0000","last_fullsized":"2023-10-10T20:39:37.262406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:19:53.834466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260876+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:17.810574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.276745+0000","last_change":"2023-10-10T20:39:37.276745+0000","last_active":"2023-10-10T20:39:37.276745+0000","last_peered":"2023-10-10T20:39:37.276745+0000","last_clean":"2023-10-10T20:39:37.276745+0000","last_became_active":"2023-10-10T20:39:37.276469+0000","last_became_peered":"2023-10-10T20:39:37.276469+0000","last_unstale":"2023-10-10T20:39:37.276745+0000","last_undegraded":"2023-10-10T20:39:37.276745+0000","last_fullsized":"2023-10-10T20:39:37.276745+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:50.483075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252593+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:18:44.235474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.275690+0000","last_change":"2023-10-10T20:39:37.275690+0000","last_active":"2023-10-10T20:39:37.275690+0000","last_peered":"2023-10-10T20:39:37.275690+0000","last_clean":"2023-10-10T20:39:37.275690+0000","last_became_active":"2023-10-10T20:39:37.275412+0000","last_became_peered":"2023-10-10T20:39:37.275412+0000","last_unstale":"2023-10-10T20:39:37.275690+0000","last_undegraded":"2023-10-10T20:39:37.275690+0000","last_fullsized":"2023-10-10T20:39:37.275690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:50:59.254651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.266049+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:16.564322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.267783+0000","last_change":"2023-10-10T20:39:37.267783+0000","last_active":"2023-10-10T20:39:37.267783+0000","last_peered":"2023-10-10T20:39:37.267783+0000","last_clean":"2023-10-10T20:39:37.267783+0000","last_became_active":"2023-10-10T20:39:37.267436+0000","last_became_peered":"2023-10-10T20:39:37.267436+0000","last_unstale":"2023-10-10T20:39:37.267783+0000","last_undegraded":"2023-10-10T20:39:37.267783+0000","last_fullsized":"2023-10-10T20:39:37.267783+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:42.059614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254194+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:04.993021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.267358+0000","last_change":"2023-10-10T20:39:37.267358+0000","last_active":"2023-10-10T20:39:37.267358+0000","last_peered":"2023-10-10T20:39:37.267358+0000","last_clean":"2023-10-10T20:39:37.267358+0000","last_became_active":"2023-10-10T20:39:37.267054+0000","last_became_peered":"2023-10-10T20:39:37.267054+0000","last_unstale":"2023-10-10T20:39:37.267358+0000","last_undegraded":"2023-10-10T20:39:37.267358+0000","last_fullsized":"2023-10-10T20:39:37.267358+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:31.881697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253409+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:49.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.265548+0000","last_change":"2023-10-10T20:39:37.265548+0000","last_active":"2023-10-10T20:39:37.265548+0000","last_peered":"2023-10-10T20:39:37.265548+0000","last_clean":"2023-10-10T20:39:37.265548+0000","last_became_active":"2023-10-10T20:39:37.265284+0000","last_became_peered":"2023-10-10T20:39:37.265284+0000","last_unstale":"2023-10-10T20:39:37.265548+0000","last_undegraded":"2023-10-10T20:39:37.265548+0000","last_fullsized":"2023-10-10T20:39:37.265548+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:53:22.002691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257399+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:02:13.214283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.270081+0000","last_change":"2023-10-10T20:39:37.270081+0000","last_active":"2023-10-10T20:39:37.270081+0000","last_peered":"2023-10-10T20:39:37.270081+0000","last_clean":"2023-10-10T20:39:37.270081+0000","last_became_active":"2023-10-10T20:39:37.269828+0000","last_became_peered":"2023-10-10T20:39:37.269828+0000","last_unstale":"2023-10-10T20:39:37.270081+0000","last_undegraded":"2023-10-10T20:39:37.270081+0000","last_fullsized":"2023-10-10T20:39:37.270081+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:16:06.819183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257391+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:41:32.637048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264681+0000","last_change":"2023-10-10T20:39:37.264681+0000","last_active":"2023-10-10T20:39:37.264681+0000","last_peered":"2023-10-10T20:39:37.264681+0000","last_clean":"2023-10-10T20:39:37.264681+0000","last_became_active":"2023-10-10T20:39:37.264173+0000","last_became_peered":"2023-10-10T20:39:37.264173+0000","last_unstale":"2023-10-10T20:39:37.264681+0000","last_undegraded":"2023-10-10T20:39:37.264681+0000","last_fullsized":"2023-10-10T20:39:37.264681+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:31.881697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257813+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:09.824624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.270474+0000","last_change":"2023-10-10T20:39:37.270474+0000","last_active":"2023-10-10T20:39:37.270474+0000","last_peered":"2023-10-10T20:39:37.270474+0000","last_clean":"2023-10-10T20:39:37.270474+0000","last_became_active":"2023-10-10T20:39:37.270158+0000","last_became_peered":"2023-10-10T20:39:37.270158+0000","last_unstale":"2023-10-10T20:39:37.270474+0000","last_undegraded":"2023-10-10T20:39:37.270474+0000","last_fullsized":"2023-10-10T20:39:37.270474+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:19:53.834466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259123+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272341+0000","last_change":"2023-10-10T20:39:37.272341+0000","last_active":"2023-10-10T20:39:37.272341+0000","last_peered":"2023-10-10T20:39:37.272341+0000","last_clean":"2023-10-10T20:39:37.272341+0000","last_became_active":"2023-10-10T20:39:37.272070+0000","last_became_peered":"2023-10-10T20:39:37.272070+0000","last_unstale":"2023-10-10T20:39:37.272341+0000","last_undegraded":"2023-10-10T20:39:37.272341+0000","last_fullsized":"2023-10-10T20:39:37.272341+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:42.059614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254514+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:26.586272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264016+0000","last_change":"2023-10-10T20:39:37.264016+0000","last_active":"2023-10-10T20:39:37.264016+0000","last_peered":"2023-10-10T20:39:37.264016+0000","last_clean":"2023-10-10T20:39:37.264016+0000","last_became_active":"2023-10-10T20:39:37.263724+0000","last_became_peered":"2023-10-10T20:39:37.263724+0000","last_unstale":"2023-10-10T20:39:37.264016+0000","last_undegraded":"2023-10-10T20:39:37.264016+0000","last_fullsized":"2023-10-10T20:39:37.264016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:25.579798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258964+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272988+0000","last_change":"2023-10-10T20:39:37.272988+0000","last_active":"2023-10-10T20:39:37.272988+0000","last_peered":"2023-10-10T20:39:37.272988+0000","last_clean":"2023-10-10T20:39:37.272988+0000","last_became_active":"2023-10-10T20:39:37.272715+0000","last_became_peered":"2023-10-10T20:39:37.272715+0000","last_unstale":"2023-10-10T20:39:37.272988+0000","last_undegraded":"2023-10-10T20:39:37.272988+0000","last_fullsized":"2023-10-10T20:39:37.272988+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:10:21.213337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252142+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.268230+0000","last_change":"2023-10-10T20:39:37.268230+0000","last_active":"2023-10-10T20:39:37.268230+0000","last_peered":"2023-10-10T20:39:37.268230+0000","last_clean":"2023-10-10T20:39:37.268230+0000","last_became_active":"2023-10-10T20:39:37.267986+0000","last_became_peered":"2023-10-10T20:39:37.267986+0000","last_unstale":"2023-10-10T20:39:37.268230+0000","last_undegraded":"2023-10-10T20:39:37.268230+0000","last_fullsized":"2023-10-10T20:39:37.268230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:13:05.731066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.264627+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:24:00.218312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.269974+0000","last_change":"2023-10-10T20:39:37.269974+0000","last_active":"2023-10-10T20:39:37.269974+0000","last_peered":"2023-10-10T20:39:37.269974+0000","last_clean":"2023-10-10T20:39:37.269974+0000","last_became_active":"2023-10-10T20:39:37.269650+0000","last_became_peered":"2023-10-10T20:39:37.269650+0000","last_unstale":"2023-10-10T20:39:37.269974+0000","last_undegraded":"2023-10-10T20:39:37.269974+0000","last_fullsized":"2023-10-10T20:39:37.269974+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:16.804099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260831+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:55:56.167036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.266948+0000","last_change":"2023-10-10T20:39:37.266948+0000","last_active":"2023-10-10T20:39:37.266948+0000","last_peered":"2023-10-10T20:39:37.266948+0000","last_clean":"2023-10-10T20:39:37.266948+0000","last_became_active":"2023-10-10T20:39:37.266149+0000","last_became_peered":"2023-10-10T20:39:37.266149+0000","last_unstale":"2023-10-10T20:39:37.266948+0000","last_undegraded":"2023-10-10T20:39:37.266948+0000","last_fullsized":"2023-10-10T20:39:37.266948+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:25.579798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252135+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.265234+0000","last_change":"2023-10-10T20:39:37.265234+0000","last_active":"2023-10-10T20:39:37.265234+0000","last_peered":"2023-10-10T20:39:37.265234+0000","last_clean":"2023-10-10T20:39:37.265234+0000","last_became_active":"2023-10-10T20:39:37.264953+0000","last_became_peered":"2023-10-10T20:39:37.264953+0000","last_unstale":"2023-10-10T20:39:37.265234+0000","last_undegraded":"2023-10-10T20:39:37.265234+0000","last_fullsized":"2023-10-10T20:39:37.265234+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:53:22.002691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260398+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:26.586272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.269676+0000","last_change":"2023-10-10T20:39:37.269676+0000","last_active":"2023-10-10T20:39:37.269676+0000","last_peered":"2023-10-10T20:39:37.269676+0000","last_clean":"2023-10-10T20:39:37.269676+0000","last_became_active":"2023-10-10T20:39:37.269437+0000","last_became_peered":"2023-10-10T20:39:37.269437+0000","last_unstale":"2023-10-10T20:39:37.269676+0000","last_undegraded":"2023-10-10T20:39:37.269676+0000","last_fullsized":"2023-10-10T20:39:37.269676+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:43.983815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253984+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:51:00.261125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264681+0000","last_change":"2023-10-10T20:39:37.264681+0000","last_active":"2023-10-10T20:39:37.264681+0000","last_peered":"2023-10-10T20:39:37.264681+0000","last_clean":"2023-10-10T20:39:37.264681+0000","last_became_active":"2023-10-10T20:39:37.264410+0000","last_became_peered":"2023-10-10T20:39:37.264410+0000","last_unstale":"2023-10-10T20:39:37.264681+0000","last_undegraded":"2023-10-10T20:39:37.264681+0000","last_fullsized":"2023-10-10T20:39:37.264681+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:21.790518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259115+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:02:53.563766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272951+0000","last_change":"2023-10-10T20:39:37.272951+0000","last_active":"2023-10-10T20:39:37.272951+0000","last_peered":"2023-10-10T20:39:37.272951+0000","last_clean":"2023-10-10T20:39:37.272951+0000","last_became_active":"2023-10-10T20:39:37.272680+0000","last_became_peered":"2023-10-10T20:39:37.272680+0000","last_unstale":"2023-10-10T20:39:37.272951+0000","last_undegraded":"2023-10-10T20:39:37.272951+0000","last_fullsized":"2023-10-10T20:39:37.272951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:09:37.235149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257781+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272869+0000","last_change":"2023-10-10T20:39:37.272869+0000","last_active":"2023-10-10T20:39:37.272869+0000","last_peered":"2023-10-10T20:39:37.272869+0000","last_clean":"2023-10-10T20:39:37.272869+0000","last_became_active":"2023-10-10T20:39:37.272418+0000","last_became_peered":"2023-10-10T20:39:37.272418+0000","last_unstale":"2023-10-10T20:39:37.272869+0000","last_undegraded":"2023-10-10T20:39:37.272869+0000","last_fullsized":"2023-10-10T20:39:37.272869+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:20:16.442207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259051+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:57.313252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272054+0000","last_change":"2023-10-10T20:39:37.272054+0000","last_active":"2023-10-10T20:39:37.272054+0000","last_peered":"2023-10-10T20:39:37.272054+0000","last_clean":"2023-10-10T20:39:37.272054+0000","last_became_active":"2023-10-10T20:39:37.271818+0000","last_became_peered":"2023-10-10T20:39:37.271818+0000","last_unstale":"2023-10-10T20:39:37.272054+0000","last_undegraded":"2023-10-10T20:39:37.272054+0000","last_fullsized":"2023-10-10T20:39:37.272054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:57:37.129049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258156+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:18:44.235474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.275925+0000","last_change":"2023-10-10T20:39:37.275925+0000","last_active":"2023-10-10T20:39:37.275925+0000","last_peered":"2023-10-10T20:39:37.275925+0000","last_clean":"2023-10-10T20:39:37.275925+0000","last_became_active":"2023-10-10T20:39:37.275666+0000","last_became_peered":"2023-10-10T20:39:37.275666+0000","last_unstale":"2023-10-10T20:39:37.275925+0000","last_undegraded":"2023-10-10T20:39:37.275925+0000","last_fullsized":"2023-10-10T20:39:37.275925+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:01:58.347464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254173+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:51.489549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.265642+0000","last_change":"2023-10-10T20:39:37.265642+0000","last_active":"2023-10-10T20:39:37.265642+0000","last_peered":"2023-10-10T20:39:37.265642+0000","last_clean":"2023-10-10T20:39:37.265642+0000","last_became_active":"2023-10-10T20:39:37.265399+0000","last_became_peered":"2023-10-10T20:39:37.265399+0000","last_unstale":"2023-10-10T20:39:37.265642+0000","last_undegraded":"2023-10-10T20:39:37.265642+0000","last_fullsized":"2023-10-10T20:39:37.265642+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:53:41.456643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253320+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:09.824624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272730+0000","last_change":"2023-10-10T20:39:37.272730+0000","last_active":"2023-10-10T20:39:37.272730+0000","last_peered":"2023-10-10T20:39:37.272730+0000","last_clean":"2023-10-10T20:39:37.272730+0000","last_became_active":"2023-10-10T20:39:37.272487+0000","last_became_peered":"2023-10-10T20:39:37.272487+0000","last_unstale":"2023-10-10T20:39:37.272730+0000","last_undegraded":"2023-10-10T20:39:37.272730+0000","last_fullsized":"2023-10-10T20:39:37.272730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:40.276426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260216+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:04.993021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.275883+0000","last_change":"2023-10-10T20:39:37.275883+0000","last_active":"2023-10-10T20:39:37.275883+0000","last_peered":"2023-10-10T20:39:37.275883+0000","last_clean":"2023-10-10T20:39:37.275883+0000","last_became_active":"2023-10-10T20:39:37.275635+0000","last_became_peered":"2023-10-10T20:39:37.275635+0000","last_unstale":"2023-10-10T20:39:37.275883+0000","last_undegraded":"2023-10-10T20:39:37.275883+0000","last_fullsized":"2023-10-10T20:39:37.275883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:23:59.211838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253448+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:44.990290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264870+0000","last_change":"2023-10-10T20:39:37.264870+0000","last_active":"2023-10-10T20:39:37.264870+0000","last_peered":"2023-10-10T20:39:37.264870+0000","last_clean":"2023-10-10T20:39:37.264870+0000","last_became_active":"2023-10-10T20:39:37.264606+0000","last_became_peered":"2023-10-10T20:39:37.264606+0000","last_unstale":"2023-10-10T20:39:37.264870+0000","last_undegraded":"2023-10-10T20:39:37.264870+0000","last_fullsized":"2023-10-10T20:39:37.264870+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:02.132949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258303+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:45.015636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.267077+0000","last_change":"2023-10-10T20:39:37.267077+0000","last_active":"2023-10-10T20:39:37.267077+0000","last_peered":"2023-10-10T20:39:37.267077+0000","last_clean":"2023-10-10T20:39:37.267077+0000","last_became_active":"2023-10-10T20:39:37.266325+0000","last_became_peered":"2023-10-10T20:39:37.266325+0000","last_unstale":"2023-10-10T20:39:37.267077+0000","last_undegraded":"2023-10-10T20:39:37.267077+0000","last_fullsized":"2023-10-10T20:39:37.267077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:03.986546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259437+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.262060+0000","last_change":"2023-10-10T20:39:37.262060+0000","last_active":"2023-10-10T20:39:37.262060+0000","last_peered":"2023-10-10T20:39:37.262060+0000","last_clean":"2023-10-10T20:39:37.262060+0000","last_became_active":"2023-10-10T20:39:37.261636+0000","last_became_peered":"2023-10-10T20:39:37.261636+0000","last_unstale":"2023-10-10T20:39:37.262060+0000","last_undegraded":"2023-10-10T20:39:37.262060+0000","last_fullsized":"2023-10-10T20:39:37.262060+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:15.557848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252172+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264795+0000","last_change":"2023-10-10T20:39:37.264795+0000","last_active":"2023-10-10T20:39:37.264795+0000","last_peered":"2023-10-10T20:39:37.264795+0000","last_clean":"2023-10-10T20:39:37.264795+0000","last_became_active":"2023-10-10T20:39:37.263322+0000","last_became_peered":"2023-10-10T20:39:37.263322+0000","last_unstale":"2023-10-10T20:39:37.264795+0000","last_undegraded":"2023-10-10T20:39:37.264795+0000","last_fullsized":"2023-10-10T20:39:37.264795+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:15.557848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258150+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:22.796992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.263674+0000","last_change":"2023-10-10T20:39:37.263674+0000","last_active":"2023-10-10T20:39:37.263674+0000","last_peered":"2023-10-10T20:39:37.263674+0000","last_clean":"2023-10-10T20:39:37.263674+0000","last_became_active":"2023-10-10T20:39:37.263294+0000","last_became_peered":"2023-10-10T20:39:37.263294+0000","last_unstale":"2023-10-10T20:39:37.263674+0000","last_undegraded":"2023-10-10T20:39:37.263674+0000","last_fullsized":"2023-10-10T20:39:37.263674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:02.132949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.265134+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:43.066089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264553+0000","last_change":"2023-10-10T20:39:37.264553+0000","last_active":"2023-10-10T20:39:37.264553+0000","last_peered":"2023-10-10T20:39:37.264553+0000","last_clean":"2023-10-10T20:39:37.264553+0000","last_became_active":"2023-10-10T20:39:37.264274+0000","last_became_peered":"2023-10-10T20:39:37.264274+0000","last_unstale":"2023-10-10T20:39:37.264553+0000","last_undegraded":"2023-10-10T20:39:37.264553+0000","last_fullsized":"2023-10-10T20:39:37.264553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:25.579798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":217,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-10T20:39:36.243840+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:39:36.243840+0000","last_peered":"2023-10-10T20:39:36.243840+0000","last_clean":"2023-10-10T20:39:36.243840+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:39:36.243840+0000","last_undegraded":"2023-10-10T20:39:36.243840+0000","last_fullsized":"2023-10-10T20:39:36.243840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298756,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073084,"statfs":{"total":96636764160,"available":96330838016,"internally_reserved":0,"allocated":1503232,"data_stored":978234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.69699999999999995,"15min":0.88100000000000001},"min":{"1min":0.32500000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.68500000000000005,"5min":6.8630000000000004,"15min":6.8630000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.59899999999999998,"15min":0.65000000000000002},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.752,"5min":4.1959999999999997,"15min":4.1959999999999997},"last":0.41899999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.79800000000000004,"15min":1.0980000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.70099999999999996,"5min":10.1,"15min":10.1},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.66700000000000004,"15min":0.76700000000000002},"min":{"1min":0.42999999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71199999999999997,"5min":6.806,"15min":6.806},"last":0.498}]},{"osd":2,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.70699999999999996,"15min":0.82399999999999995},"min":{"1min":0.42899999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70899999999999996,"5min":6.9340000000000002,"15min":6.9340000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.503,"5min":0.65400000000000003,"15min":0.77800000000000002},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69599999999999995,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.436}]},{"osd":3,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.67700000000000005,"15min":0.79700000000000004},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.72199999999999998,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63400000000000001,"15min":0.69199999999999995},"min":{"1min":0.42799999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76200000000000001,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.53900000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.314,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.65900000000000003,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.32900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69199999999999995,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.52900000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.376,"5min":0.314,"15min":0.314},"max":{"1min":0.78900000000000003,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53200000000000003,"15min":0.52500000000000002},"min":{"1min":0.42899999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68000000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.51600000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73799999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.51300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73699999999999999,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.46800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.59399999999999997,"15min":0.70099999999999996},"min":{"1min":0.36499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.72899999999999998,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.629,"15min":0.749},"min":{"1min":0.35299999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66000000000000003,"5min":7.8090000000000002,"15min":7.8090000000000002},"last":0.45100000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.628,"15min":0.746},"min":{"1min":0.36899999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":7.867,"15min":7.867},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.73999999999999999,"15min":0.95399999999999996},"min":{"1min":0.36299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70199999999999996,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.57799999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.55600000000000005,"15min":0.60999999999999999},"min":{"1min":0.371,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.65900000000000003,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.71499999999999997,"15min":0.90100000000000002},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.80000000000000004,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":0.53200000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.58599999999999997,"15min":0.63400000000000001},"min":{"1min":0.38600000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.65800000000000003,"5min":4.8620000000000001,"15min":4.8620000000000001},"last":0.501},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.64700000000000002,"15min":0.75},"min":{"1min":0.439,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68899999999999995,"5min":7.9009999999999998,"15min":7.9009999999999998},"last":0.55300000000000005}]},{"osd":4,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48599999999999999,"15min":0.45800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68400000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.438,"5min":0.37,"15min":0.37},"max":{"1min":0.73999999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.62}]},{"osd":5,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.5,"15min":0.48199999999999998},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.68899999999999995,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71199999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.48599999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52500000000000002,"15min":0.51100000000000001},"min":{"1min":0.377,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73299999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.38600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.745,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.59299999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.40600000000000003,"15min":0.505},"min":{"1min":0.19600000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.40200000000000002,"5min":5.0380000000000003,"15min":5.0380000000000003},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.34000000000000002,"15min":0.36799999999999999},"min":{"1min":0.224,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45200000000000001,"5min":1.194,"15min":1.194},"last":0.35899999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.34100000000000003,"15min":0.36899999999999999},"min":{"1min":0.215,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.42599999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.307,"5min":0.42399999999999999,"15min":0.53400000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45900000000000002,"5min":5.0919999999999996,"15min":5.0919999999999996},"last":0.5}]},{"osd":3,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.45200000000000001,"15min":0.55800000000000005},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.442,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.376},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.38900000000000001,"15min":0.41499999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.48999999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.44500000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57599999999999996,"15min":0.56499999999999995},"min":{"1min":0.38800000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73899999999999999,"5min":1.254,"15min":1.254},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.64500000000000002,"15min":0.73199999999999998},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82699999999999996,"5min":4.9850000000000003,"15min":4.9850000000000003},"last":0.63}]},{"osd":5,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.58199999999999996,"15min":0.59599999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72699999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59999999999999998,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83599999999999997,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.73999999999999999,"15min":0.88800000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89100000000000001,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.41799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89800000000000002,"5min":1.171,"15min":1.171},"last":0.70199999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.93300000000000005,"5min":1.214,"15min":1.214},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.67500000000000004,"15min":0.75},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.748,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.73199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302848,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068992,"statfs":{"total":96636764160,"available":96326647808,"internally_reserved":0,"allocated":1499136,"data_stored":977273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.44,"15min":0.57199999999999995},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.50900000000000001,"5min":7.9249999999999998,"15min":7.9249999999999998},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.42899999999999999,"15min":0.54200000000000004},"min":{"1min":0.22500000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.47199999999999998,"5min":6.6299999999999999,"15min":6.6299999999999999},"last":0.31900000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.45900000000000002,"15min":0.59699999999999998},"min":{"1min":0.23999999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.55400000000000005,"5min":8.2509999999999994,"15min":8.2509999999999994},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.45300000000000001,"15min":0.56599999999999995},"min":{"1min":0.223,"5min":0.222,"15min":0.222},"max":{"1min":0.54100000000000004,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.35299999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.46700000000000003,"15min":0.59799999999999998},"min":{"1min":0.23100000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54100000000000004,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.51800000000000002,"15min":0.65500000000000003},"min":{"1min":0.29199999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53300000000000003,"5min":7.8929999999999998,"15min":7.8929999999999998},"last":0.33300000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.65600000000000003,"15min":0.747},"min":{"1min":0.39400000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.88900000000000001,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.68700000000000006,"15min":0.81299999999999994},"min":{"1min":0.374,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78700000000000003,"5min":7.6920000000000002,"15min":7.6920000000000002},"last":0.54300000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.67400000000000004,"15min":0.79200000000000004},"min":{"1min":0.376,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90200000000000002,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.69499999999999995,"15min":0.81599999999999995},"min":{"1min":0.38100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79900000000000004,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":0.63100000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.72599999999999998,"15min":0.85899999999999999},"min":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81399999999999995,"5min":7.8360000000000003,"15min":7.8360000000000003},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70299999999999996,"15min":0.79200000000000004},"min":{"1min":0.46100000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92000000000000004,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.64500000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.68600000000000005,"15min":0.78400000000000003},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86399999999999999,"5min":6.7160000000000002,"15min":6.7160000000000002},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.70799999999999996,"15min":0.82499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93100000000000005,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.61299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.31900000000000001,"15min":0.32800000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.59699999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.33600000000000002,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45400000000000001,"5min":0.746,"15min":0.746},"last":0.38200000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.45400000000000001,"15min":0.77100000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.63,"5min":11.967000000000001,"15min":11.967000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39000000000000001,"15min":0.498},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52700000000000002,"5min":4.1100000000000003,"15min":4.1100000000000003},"last":0.34200000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.41199999999999998,"15min":0.57899999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48799999999999999,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.50900000000000001,"15min":0.82199999999999995},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65900000000000003,"5min":11.935,"15min":11.935},"last":0.45600000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.65900000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.80100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.58999999999999997},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.495}]},{"osd":6,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.33400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83599999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63700000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81599999999999995,"5min":1.052,"15min":1.052},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64500000000000002,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85999999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.62}]}]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298164,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073676,"statfs":{"total":96636764160,"available":96331444224,"internally_reserved":0,"allocated":897024,"data_stored":381603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.309,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44400000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.317},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48699999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.307}]},{"osd":1,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33600000000000002,"15min":0.33000000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.49199999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38200000000000001,"15min":0.376},"min":{"1min":0.307,"5min":0.251,"15min":0.251},"max":{"1min":0.53900000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.38200000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.34899999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35499999999999998,"15min":0.36099999999999999},"min":{"1min":0.25900000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.51600000000000001,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.32600000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.38100000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83299999999999996,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.54500000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.39400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78700000000000003,"5min":1.05,"15min":1.05},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86799999999999999,"5min":1.01,"15min":1.01},"last":0.441}]},{"osd":6,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92000000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.877,"5min":1.026,"15min":1.026},"last":0.47599999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77100000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.46600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89400000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.52500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58199999999999996,"5min":0.747,"15min":0.747},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.59799999999999998,"15min":0.79200000000000004},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69699999999999995,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.69699999999999995}]},{"osd":1,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.60299999999999998,"15min":0.79900000000000004},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60299999999999998,"5min":7.7160000000000002,"15min":7.7160000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54900000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68100000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63600000000000001,"15min":0.81799999999999995},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.48699999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.58899999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.57299999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.502,"15min":0.48399999999999999},"min":{"1min":0.34599999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.622},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.38400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77300000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.77300000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.41699999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72099999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.52600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.67400000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.58199999999999996},"min":{"1min":0.44700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.27900000000000003,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.59699999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.32000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.73299999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.70399999999999996}]},{"osd":1,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.60199999999999998,"15min":0.66600000000000004},"min":{"1min":0.38500000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.66300000000000003,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.58499999999999996,"15min":0.65100000000000002},"min":{"1min":0.33500000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65200000000000002,"5min":5.3109999999999999,"15min":5.3109999999999999},"last":0.68200000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.60799999999999998,"15min":0.67400000000000004},"min":{"1min":0.373,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72799999999999998,"5min":5.2960000000000003,"15min":5.2960000000000003},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.33900000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71999999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.68999999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.64100000000000001,"15min":0.70699999999999996},"min":{"1min":0.39900000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80000000000000004,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.63700000000000001,"15min":0.71699999999999997},"min":{"1min":0.439,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69499999999999995,"5min":5.2850000000000001,"15min":5.2850000000000001},"last":0.61799999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48799999999999999,"15min":0.499},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64200000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.35099999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.501}]},{"osd":6,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.86099999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.753,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.71299999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.36499999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84799999999999998,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82199999999999995,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.72699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:39:42.327 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:39:42.422 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:39:42.424 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:39:42.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.473+0000 7fed43b29700 1 Processor -- start 2023-10-10T20:39:42.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.473+0000 7fed43b29700 1 -- start start 2023-10-10T20:39:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.474+0000 7fed43b29700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fed3c159890 0x7fed3c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.474+0000 7fed43b29700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fed3c159c60 0x7fed3c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.474+0000 7fed43b29700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed3c15db10 0x7fed3c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.474+0000 7fed43b29700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed3c15df50 con 0x7fed3c159c60 2023-10-10T20:39:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.474+0000 7fed43b29700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed3c15a160 con 0x7fed3c159890 2023-10-10T20:39:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.474+0000 7fed43b29700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c0d13a0 con 0x7fed3c15db10 2023-10-10T20:39:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.474+0000 7fed410c4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fed3c159c60 0x7fed3c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.474+0000 7fed410c4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fed3c159c60 0x7fed3c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41868/0 (socket says 172.21.15.89:41868) 2023-10-10T20:39:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.474+0000 7fed410c4700 1 -- 172.21.15.89:0/3442089559 learned_addr learned my addr 172.21.15.89:0/3442089559 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.474+0000 7fed418c5700 1 --2- 172.21.15.89:0/3442089559 >> v2:172.21.15.111:3300/0 conn(0x7fed3c159890 0x7fed3c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.474+0000 7fed420c6700 1 --2- 172.21.15.89:0/3442089559 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed3c15db10 0x7fed3c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.475+0000 7fed410c4700 1 -- 172.21.15.89:0/3442089559 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed3c15db10 msgr2=0x7fed3c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.475+0000 7fed410c4700 1 --2- 172.21.15.89:0/3442089559 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed3c15db10 0x7fed3c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.475+0000 7fed410c4700 1 -- 172.21.15.89:0/3442089559 >> v2:172.21.15.111:3300/0 conn(0x7fed3c159890 msgr2=0x7fed3c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.475+0000 7fed410c4700 1 --2- 172.21.15.89:0/3442089559 >> v2:172.21.15.111:3300/0 conn(0x7fed3c159890 0x7fed3c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.475+0000 7fed410c4700 1 -- 172.21.15.89:0/3442089559 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c15e140 con 0x7fed3c159c60 2023-10-10T20:39:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.475+0000 7fed410c4700 1 --2- 172.21.15.89:0/3442089559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed3c159c60 0x7fed3c15d5d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fed3c0d1090 tx=0x7fed38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1a53efd693784d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.475+0000 7fed408c3700 1 -- 172.21.15.89:0/3442089559 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3800b640 con 0x7fed3c159c60 2023-10-10T20:39:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.475+0000 7fed408c3700 1 -- 172.21.15.89:0/3442089559 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3800b7a0 con 0x7fed3c159c60 2023-10-10T20:39:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.475+0000 7fed408c3700 1 -- 172.21.15.89:0/3442089559 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3800bb60 con 0x7fed3c159c60 2023-10-10T20:39:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.475+0000 7fed43b29700 1 -- 172.21.15.89:0/3442089559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed3c159c60 msgr2=0x7fed3c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.475+0000 7fed43b29700 1 --2- 172.21.15.89:0/3442089559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed3c159c60 0x7fed3c15d5d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fed3c0d1090 tx=0x7fed38009250 comp rx=0 tx=0).stop 2023-10-10T20:39:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.475+0000 7fed43b29700 1 -- 172.21.15.89:0/3442089559 shutdown_connections 2023-10-10T20:39:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.476+0000 7fed43b29700 1 --2- 172.21.15.89:0/3442089559 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed3c15db10 0x7fed3c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.476+0000 7fed43b29700 1 --2- 172.21.15.89:0/3442089559 >> v2:172.21.15.111:3300/0 conn(0x7fed3c159890 0x7fed3c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.476+0000 7fed43b29700 1 --2- 172.21.15.89:0/3442089559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed3c159c60 0x7fed3c15d5d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.476+0000 7fed43b29700 1 -- 172.21.15.89:0/3442089559 >> 172.21.15.89:0/3442089559 conn(0x7fed3c068e20 msgr2=0x7fed3c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.476+0000 7fed43b29700 1 -- 172.21.15.89:0/3442089559 shutdown_connections 2023-10-10T20:39:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.476+0000 7fed43b29700 1 -- 172.21.15.89:0/3442089559 wait complete. 2023-10-10T20:39:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.477+0000 7fed43b29700 1 Processor -- start 2023-10-10T20:39:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.477+0000 7fed43b29700 1 -- start start 2023-10-10T20:39:42.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.477+0000 7fed43b29700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed3c159890 0x7fed3c0c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:42.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.477+0000 7fed43b29700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed3c159c60 0x7fed3c0afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.477+0000 7fed43b29700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed3c15db10 0x7fed3c157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.477+0000 7fed43b29700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c0c95d0 con 0x7fed3c159c60 2023-10-10T20:39:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.477+0000 7fed43b29700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c0b37c0 con 0x7fed3c159890 2023-10-10T20:39:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.477+0000 7fed43b29700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c0b0010 con 0x7fed3c15db10 2023-10-10T20:39:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.477+0000 7fed410c4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed3c159c60 0x7fed3c0afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:42.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.478+0000 7fed410c4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed3c159c60 0x7fed3c0afad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41874/0 (socket says 172.21.15.89:41874) 2023-10-10T20:39:42.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.478+0000 7fed410c4700 1 -- 172.21.15.89:0/2594519692 learned_addr learned my addr 172.21.15.89:0/2594519692 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:42.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.478+0000 7fed420c6700 1 --2- 172.21.15.89:0/2594519692 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed3c15db10 0x7fed3c157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:42.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.478+0000 7fed418c5700 1 --2- 172.21.15.89:0/2594519692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed3c159890 0x7fed3c0c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:42.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.478+0000 7fed410c4700 1 -- 172.21.15.89:0/2594519692 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed3c15db10 msgr2=0x7fed3c157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:42.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.478+0000 7fed410c4700 1 --2- 172.21.15.89:0/2594519692 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed3c15db10 0x7fed3c157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.478+0000 7fed410c4700 1 -- 172.21.15.89:0/2594519692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed3c159890 msgr2=0x7fed3c0c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:42.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.478+0000 7fed410c4700 1 --2- 172.21.15.89:0/2594519692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed3c159890 0x7fed3c0c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.478+0000 7fed410c4700 1 -- 172.21.15.89:0/2594519692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed380094f0 con 0x7fed3c159c60 2023-10-10T20:39:42.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.478+0000 7fed410c4700 1 --2- 172.21.15.89:0/2594519692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed3c159c60 0x7fed3c0afad0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fed3c070e20 tx=0x7fed38017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.478+0000 7fed2dffb700 1 -- 172.21.15.89:0/2594519692 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3800b910 con 0x7fed3c159c60 2023-10-10T20:39:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.478+0000 7fed2dffb700 1 -- 172.21.15.89:0/2594519692 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3800ba70 con 0x7fed3c159c60 2023-10-10T20:39:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.478+0000 7fed2dffb700 1 -- 172.21.15.89:0/2594519692 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38011630 con 0x7fed3c159c60 2023-10-10T20:39:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.478+0000 7fed43b29700 1 -- 172.21.15.89:0/2594519692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3c0c8d30 con 0x7fed3c159c60 2023-10-10T20:39:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.478+0000 7fed43b29700 1 -- 172.21.15.89:0/2594519692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3c0c92f0 con 0x7fed3c159c60 2023-10-10T20:39:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.479+0000 7fed43b29700 1 -- 172.21.15.89:0/2594519692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed3c0b3950 con 0x7fed3c159c60 2023-10-10T20:39:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.480+0000 7fed2dffb700 1 -- 172.21.15.89:0/2594519692 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fed380119a0 con 0x7fed3c159c60 2023-10-10T20:39:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.480+0000 7fed2dffb700 1 --2- 172.21.15.89:0/2594519692 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fed20067170 0x7fed20069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.480+0000 7fed2dffb700 1 -- 172.21.15.89:0/2594519692 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fed38085f90 con 0x7fed3c159c60 2023-10-10T20:39:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.480+0000 7fed418c5700 1 --2- 172.21.15.89:0/2594519692 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fed20067170 0x7fed20069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:42.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.480+0000 7fed418c5700 1 --2- 172.21.15.89:0/2594519692 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fed20067170 0x7fed20069620 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fed30009d10 tx=0x7fed300091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:42.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.484+0000 7fed2dffb700 1 -- 172.21.15.89:0/2594519692 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed380b75a0 con 0x7fed3c159c60 2023-10-10T20:39:42.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.663+0000 7fed43b29700 1 -- 172.21.15.89:0/2594519692 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fed3c04c3c0 con 0x7fed20067170 2023-10-10T20:39:42.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.671+0000 7fed2dffb700 1 -- 172.21.15.89:0/2594519692 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286833 (secure 0 0 0) 0x7fed3c04c3c0 con 0x7fed20067170 2023-10-10T20:39:42.677 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:42.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.674+0000 7fed43b29700 1 -- 172.21.15.89:0/2594519692 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fed20067170 msgr2=0x7fed20069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:42.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.674+0000 7fed43b29700 1 --2- 172.21.15.89:0/2594519692 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fed20067170 0x7fed20069620 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fed30009d10 tx=0x7fed300091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.674+0000 7fed43b29700 1 -- 172.21.15.89:0/2594519692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed3c159c60 msgr2=0x7fed3c0afad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:42.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.674+0000 7fed43b29700 1 --2- 172.21.15.89:0/2594519692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed3c159c60 0x7fed3c0afad0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fed3c070e20 tx=0x7fed38017000 comp rx=0 tx=0).stop 2023-10-10T20:39:42.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.674+0000 7fed43b29700 1 -- 172.21.15.89:0/2594519692 shutdown_connections 2023-10-10T20:39:42.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.674+0000 7fed43b29700 1 --2- 172.21.15.89:0/2594519692 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed3c15db10 0x7fed3c157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.674+0000 7fed43b29700 1 --2- 172.21.15.89:0/2594519692 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fed20067170 0x7fed20069620 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.674+0000 7fed43b29700 1 --2- 172.21.15.89:0/2594519692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed3c159890 0x7fed3c0c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.674+0000 7fed43b29700 1 --2- 172.21.15.89:0/2594519692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed3c159c60 0x7fed3c0afad0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.674+0000 7fed43b29700 1 -- 172.21.15.89:0/2594519692 >> 172.21.15.89:0/2594519692 conn(0x7fed3c068e20 msgr2=0x7fed3c158940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:42.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.674+0000 7fed43b29700 1 -- 172.21.15.89:0/2594519692 shutdown_connections 2023-10-10T20:39:42.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.674+0000 7fed43b29700 1 -- 172.21.15.89:0/2594519692 wait complete. 2023-10-10T20:39:42.682 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:39:42.688 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2023-10-10T20:39:40.713576+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2390648,"kb_used_data":8248,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752584072,"statfs":{"total":773094113280,"available":770646089728,"internally_reserved":0,"allocated":8445952,"data_stored":4276440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.471190"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.262078+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:18:44.235474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261385+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:32:45.306980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253087+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:55:15.047824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258454+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:10:36.151908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254702+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:17.810574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258770+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:49:29.215389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257832+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:12:11.985278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257660+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258331+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:55:15.047824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261413+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261949+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:51:25.472971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259733+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:51.489549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253308+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:49:29.215389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.271693+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:27.148718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257856+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:56:02.724230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252996+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:22.796992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252806+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:21:45.349150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257789+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:56:39.215508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260126+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:09.824624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257801+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:55:16.441311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258944+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:45:29.295885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.266901+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:22.796992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258929+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:51:00.261125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.271424+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:13:05.408163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261055+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:10:42.178506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253538+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:45.015636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254993+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261378+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.263848+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:24.508374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.266498+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:32.888171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254896+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258291+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:44.990290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.255157+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.275787+0000","last_change":"2023-10-10T20:39:37.275787+0000","last_active":"2023-10-10T20:39:37.275787+0000","last_peered":"2023-10-10T20:39:37.275787+0000","last_clean":"2023-10-10T20:39:37.275787+0000","last_became_active":"2023-10-10T20:39:37.275528+0000","last_became_peered":"2023-10-10T20:39:37.275528+0000","last_unstale":"2023-10-10T20:39:37.275787+0000","last_undegraded":"2023-10-10T20:39:37.275787+0000","last_fullsized":"2023-10-10T20:39:37.275787+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:23.501900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260922+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.262406+0000","last_change":"2023-10-10T20:39:37.262406+0000","last_active":"2023-10-10T20:39:37.262406+0000","last_peered":"2023-10-10T20:39:37.262406+0000","last_clean":"2023-10-10T20:39:37.262406+0000","last_became_active":"2023-10-10T20:39:37.261561+0000","last_became_peered":"2023-10-10T20:39:37.261561+0000","last_unstale":"2023-10-10T20:39:37.262406+0000","last_undegraded":"2023-10-10T20:39:37.262406+0000","last_fullsized":"2023-10-10T20:39:37.262406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:19:53.834466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260876+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:17.810574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.276745+0000","last_change":"2023-10-10T20:39:37.276745+0000","last_active":"2023-10-10T20:39:37.276745+0000","last_peered":"2023-10-10T20:39:37.276745+0000","last_clean":"2023-10-10T20:39:37.276745+0000","last_became_active":"2023-10-10T20:39:37.276469+0000","last_became_peered":"2023-10-10T20:39:37.276469+0000","last_unstale":"2023-10-10T20:39:37.276745+0000","last_undegraded":"2023-10-10T20:39:37.276745+0000","last_fullsized":"2023-10-10T20:39:37.276745+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:50.483075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252593+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:18:44.235474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.275690+0000","last_change":"2023-10-10T20:39:37.275690+0000","last_active":"2023-10-10T20:39:37.275690+0000","last_peered":"2023-10-10T20:39:37.275690+0000","last_clean":"2023-10-10T20:39:37.275690+0000","last_became_active":"2023-10-10T20:39:37.275412+0000","last_became_peered":"2023-10-10T20:39:37.275412+0000","last_unstale":"2023-10-10T20:39:37.275690+0000","last_undegraded":"2023-10-10T20:39:37.275690+0000","last_fullsized":"2023-10-10T20:39:37.275690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:50:59.254651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.266049+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:16.564322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.267783+0000","last_change":"2023-10-10T20:39:37.267783+0000","last_active":"2023-10-10T20:39:37.267783+0000","last_peered":"2023-10-10T20:39:37.267783+0000","last_clean":"2023-10-10T20:39:37.267783+0000","last_became_active":"2023-10-10T20:39:37.267436+0000","last_became_peered":"2023-10-10T20:39:37.267436+0000","last_unstale":"2023-10-10T20:39:37.267783+0000","last_undegraded":"2023-10-10T20:39:37.267783+0000","last_fullsized":"2023-10-10T20:39:37.267783+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:42.059614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254194+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:04.993021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.267358+0000","last_change":"2023-10-10T20:39:37.267358+0000","last_active":"2023-10-10T20:39:37.267358+0000","last_peered":"2023-10-10T20:39:37.267358+0000","last_clean":"2023-10-10T20:39:37.267358+0000","last_became_active":"2023-10-10T20:39:37.267054+0000","last_became_peered":"2023-10-10T20:39:37.267054+0000","last_unstale":"2023-10-10T20:39:37.267358+0000","last_undegraded":"2023-10-10T20:39:37.267358+0000","last_fullsized":"2023-10-10T20:39:37.267358+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:31.881697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253409+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:49.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.265548+0000","last_change":"2023-10-10T20:39:37.265548+0000","last_active":"2023-10-10T20:39:37.265548+0000","last_peered":"2023-10-10T20:39:37.265548+0000","last_clean":"2023-10-10T20:39:37.265548+0000","last_became_active":"2023-10-10T20:39:37.265284+0000","last_became_peered":"2023-10-10T20:39:37.265284+0000","last_unstale":"2023-10-10T20:39:37.265548+0000","last_undegraded":"2023-10-10T20:39:37.265548+0000","last_fullsized":"2023-10-10T20:39:37.265548+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:53:22.002691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257399+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:02:13.214283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.270081+0000","last_change":"2023-10-10T20:39:37.270081+0000","last_active":"2023-10-10T20:39:37.270081+0000","last_peered":"2023-10-10T20:39:37.270081+0000","last_clean":"2023-10-10T20:39:37.270081+0000","last_became_active":"2023-10-10T20:39:37.269828+0000","last_became_peered":"2023-10-10T20:39:37.269828+0000","last_unstale":"2023-10-10T20:39:37.270081+0000","last_undegraded":"2023-10-10T20:39:37.270081+0000","last_fullsized":"2023-10-10T20:39:37.270081+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:16:06.819183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257391+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:41:32.637048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264681+0000","last_change":"2023-10-10T20:39:37.264681+0000","last_active":"2023-10-10T20:39:37.264681+0000","last_peered":"2023-10-10T20:39:37.264681+0000","last_clean":"2023-10-10T20:39:37.264681+0000","last_became_active":"2023-10-10T20:39:37.264173+0000","last_became_peered":"2023-10-10T20:39:37.264173+0000","last_unstale":"2023-10-10T20:39:37.264681+0000","last_undegraded":"2023-10-10T20:39:37.264681+0000","last_fullsized":"2023-10-10T20:39:37.264681+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:31.881697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257813+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:09.824624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.270474+0000","last_change":"2023-10-10T20:39:37.270474+0000","last_active":"2023-10-10T20:39:37.270474+0000","last_peered":"2023-10-10T20:39:37.270474+0000","last_clean":"2023-10-10T20:39:37.270474+0000","last_became_active":"2023-10-10T20:39:37.270158+0000","last_became_peered":"2023-10-10T20:39:37.270158+0000","last_unstale":"2023-10-10T20:39:37.270474+0000","last_undegraded":"2023-10-10T20:39:37.270474+0000","last_fullsized":"2023-10-10T20:39:37.270474+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:19:53.834466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259123+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272341+0000","last_change":"2023-10-10T20:39:37.272341+0000","last_active":"2023-10-10T20:39:37.272341+0000","last_peered":"2023-10-10T20:39:37.272341+0000","last_clean":"2023-10-10T20:39:37.272341+0000","last_became_active":"2023-10-10T20:39:37.272070+0000","last_became_peered":"2023-10-10T20:39:37.272070+0000","last_unstale":"2023-10-10T20:39:37.272341+0000","last_undegraded":"2023-10-10T20:39:37.272341+0000","last_fullsized":"2023-10-10T20:39:37.272341+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:42.059614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254514+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:26.586272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264016+0000","last_change":"2023-10-10T20:39:37.264016+0000","last_active":"2023-10-10T20:39:37.264016+0000","last_peered":"2023-10-10T20:39:37.264016+0000","last_clean":"2023-10-10T20:39:37.264016+0000","last_became_active":"2023-10-10T20:39:37.263724+0000","last_became_peered":"2023-10-10T20:39:37.263724+0000","last_unstale":"2023-10-10T20:39:37.264016+0000","last_undegraded":"2023-10-10T20:39:37.264016+0000","last_fullsized":"2023-10-10T20:39:37.264016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:25.579798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258964+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272988+0000","last_change":"2023-10-10T20:39:37.272988+0000","last_active":"2023-10-10T20:39:37.272988+0000","last_peered":"2023-10-10T20:39:37.272988+0000","last_clean":"2023-10-10T20:39:37.272988+0000","last_became_active":"2023-10-10T20:39:37.272715+0000","last_became_peered":"2023-10-10T20:39:37.272715+0000","last_unstale":"2023-10-10T20:39:37.272988+0000","last_undegraded":"2023-10-10T20:39:37.272988+0000","last_fullsized":"2023-10-10T20:39:37.272988+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:10:21.213337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252142+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.268230+0000","last_change":"2023-10-10T20:39:37.268230+0000","last_active":"2023-10-10T20:39:37.268230+0000","last_peered":"2023-10-10T20:39:37.268230+0000","last_clean":"2023-10-10T20:39:37.268230+0000","last_became_active":"2023-10-10T20:39:37.267986+0000","last_became_peered":"2023-10-10T20:39:37.267986+0000","last_unstale":"2023-10-10T20:39:37.268230+0000","last_undegraded":"2023-10-10T20:39:37.268230+0000","last_fullsized":"2023-10-10T20:39:37.268230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:13:05.731066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.264627+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:24:00.218312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.269974+0000","last_change":"2023-10-10T20:39:37.269974+0000","last_active":"2023-10-10T20:39:37.269974+0000","last_peered":"2023-10-10T20:39:37.269974+0000","last_clean":"2023-10-10T20:39:37.269974+0000","last_became_active":"2023-10-10T20:39:37.269650+0000","last_became_peered":"2023-10-10T20:39:37.269650+0000","last_unstale":"2023-10-10T20:39:37.269974+0000","last_undegraded":"2023-10-10T20:39:37.269974+0000","last_fullsized":"2023-10-10T20:39:37.269974+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:16.804099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260831+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:55:56.167036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.266948+0000","last_change":"2023-10-10T20:39:37.266948+0000","last_active":"2023-10-10T20:39:37.266948+0000","last_peered":"2023-10-10T20:39:37.266948+0000","last_clean":"2023-10-10T20:39:37.266948+0000","last_became_active":"2023-10-10T20:39:37.266149+0000","last_became_peered":"2023-10-10T20:39:37.266149+0000","last_unstale":"2023-10-10T20:39:37.266948+0000","last_undegraded":"2023-10-10T20:39:37.266948+0000","last_fullsized":"2023-10-10T20:39:37.266948+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:25.579798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252135+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.265234+0000","last_change":"2023-10-10T20:39:37.265234+0000","last_active":"2023-10-10T20:39:37.265234+0000","last_peered":"2023-10-10T20:39:37.265234+0000","last_clean":"2023-10-10T20:39:37.265234+0000","last_became_active":"2023-10-10T20:39:37.264953+0000","last_became_peered":"2023-10-10T20:39:37.264953+0000","last_unstale":"2023-10-10T20:39:37.265234+0000","last_undegraded":"2023-10-10T20:39:37.265234+0000","last_fullsized":"2023-10-10T20:39:37.265234+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:53:22.002691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260398+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:26.586272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.269676+0000","last_change":"2023-10-10T20:39:37.269676+0000","last_active":"2023-10-10T20:39:37.269676+0000","last_peered":"2023-10-10T20:39:37.269676+0000","last_clean":"2023-10-10T20:39:37.269676+0000","last_became_active":"2023-10-10T20:39:37.269437+0000","last_became_peered":"2023-10-10T20:39:37.269437+0000","last_unstale":"2023-10-10T20:39:37.269676+0000","last_undegraded":"2023-10-10T20:39:37.269676+0000","last_fullsized":"2023-10-10T20:39:37.269676+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:43.983815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253984+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:51:00.261125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264681+0000","last_change":"2023-10-10T20:39:37.264681+0000","last_active":"2023-10-10T20:39:37.264681+0000","last_peered":"2023-10-10T20:39:37.264681+0000","last_clean":"2023-10-10T20:39:37.264681+0000","last_became_active":"2023-10-10T20:39:37.264410+0000","last_became_peered":"2023-10-10T20:39:37.264410+0000","last_unstale":"2023-10-10T20:39:37.264681+0000","last_undegraded":"2023-10-10T20:39:37.264681+0000","last_fullsized":"2023-10-10T20:39:37.264681+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:21.790518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259115+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:02:53.563766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272951+0000","last_change":"2023-10-10T20:39:37.272951+0000","last_active":"2023-10-10T20:39:37.272951+0000","last_peered":"2023-10-10T20:39:37.272951+0000","last_clean":"2023-10-10T20:39:37.272951+0000","last_became_active":"2023-10-10T20:39:37.272680+0000","last_became_peered":"2023-10-10T20:39:37.272680+0000","last_unstale":"2023-10-10T20:39:37.272951+0000","last_undegraded":"2023-10-10T20:39:37.272951+0000","last_fullsized":"2023-10-10T20:39:37.272951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:09:37.235149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257781+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272869+0000","last_change":"2023-10-10T20:39:37.272869+0000","last_active":"2023-10-10T20:39:37.272869+0000","last_peered":"2023-10-10T20:39:37.272869+0000","last_clean":"2023-10-10T20:39:37.272869+0000","last_became_active":"2023-10-10T20:39:37.272418+0000","last_became_peered":"2023-10-10T20:39:37.272418+0000","last_unstale":"2023-10-10T20:39:37.272869+0000","last_undegraded":"2023-10-10T20:39:37.272869+0000","last_fullsized":"2023-10-10T20:39:37.272869+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:20:16.442207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259051+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:57.313252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272054+0000","last_change":"2023-10-10T20:39:37.272054+0000","last_active":"2023-10-10T20:39:37.272054+0000","last_peered":"2023-10-10T20:39:37.272054+0000","last_clean":"2023-10-10T20:39:37.272054+0000","last_became_active":"2023-10-10T20:39:37.271818+0000","last_became_peered":"2023-10-10T20:39:37.271818+0000","last_unstale":"2023-10-10T20:39:37.272054+0000","last_undegraded":"2023-10-10T20:39:37.272054+0000","last_fullsized":"2023-10-10T20:39:37.272054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:57:37.129049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258156+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:18:44.235474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.275925+0000","last_change":"2023-10-10T20:39:37.275925+0000","last_active":"2023-10-10T20:39:37.275925+0000","last_peered":"2023-10-10T20:39:37.275925+0000","last_clean":"2023-10-10T20:39:37.275925+0000","last_became_active":"2023-10-10T20:39:37.275666+0000","last_became_peered":"2023-10-10T20:39:37.275666+0000","last_unstale":"2023-10-10T20:39:37.275925+0000","last_undegraded":"2023-10-10T20:39:37.275925+0000","last_fullsized":"2023-10-10T20:39:37.275925+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:01:58.347464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254173+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:51.489549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.265642+0000","last_change":"2023-10-10T20:39:37.265642+0000","last_active":"2023-10-10T20:39:37.265642+0000","last_peered":"2023-10-10T20:39:37.265642+0000","last_clean":"2023-10-10T20:39:37.265642+0000","last_became_active":"2023-10-10T20:39:37.265399+0000","last_became_peered":"2023-10-10T20:39:37.265399+0000","last_unstale":"2023-10-10T20:39:37.265642+0000","last_undegraded":"2023-10-10T20:39:37.265642+0000","last_fullsized":"2023-10-10T20:39:37.265642+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:53:41.456643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253320+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:09.824624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272730+0000","last_change":"2023-10-10T20:39:37.272730+0000","last_active":"2023-10-10T20:39:37.272730+0000","last_peered":"2023-10-10T20:39:37.272730+0000","last_clean":"2023-10-10T20:39:37.272730+0000","last_became_active":"2023-10-10T20:39:37.272487+0000","last_became_peered":"2023-10-10T20:39:37.272487+0000","last_unstale":"2023-10-10T20:39:37.272730+0000","last_undegraded":"2023-10-10T20:39:37.272730+0000","last_fullsized":"2023-10-10T20:39:37.272730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:40.276426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260216+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:04.993021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.275883+0000","last_change":"2023-10-10T20:39:37.275883+0000","last_active":"2023-10-10T20:39:37.275883+0000","last_peered":"2023-10-10T20:39:37.275883+0000","last_clean":"2023-10-10T20:39:37.275883+0000","last_became_active":"2023-10-10T20:39:37.275635+0000","last_became_peered":"2023-10-10T20:39:37.275635+0000","last_unstale":"2023-10-10T20:39:37.275883+0000","last_undegraded":"2023-10-10T20:39:37.275883+0000","last_fullsized":"2023-10-10T20:39:37.275883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:23:59.211838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253448+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:44.990290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264870+0000","last_change":"2023-10-10T20:39:37.264870+0000","last_active":"2023-10-10T20:39:37.264870+0000","last_peered":"2023-10-10T20:39:37.264870+0000","last_clean":"2023-10-10T20:39:37.264870+0000","last_became_active":"2023-10-10T20:39:37.264606+0000","last_became_peered":"2023-10-10T20:39:37.264606+0000","last_unstale":"2023-10-10T20:39:37.264870+0000","last_undegraded":"2023-10-10T20:39:37.264870+0000","last_fullsized":"2023-10-10T20:39:37.264870+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:02.132949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258303+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:45.015636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.267077+0000","last_change":"2023-10-10T20:39:37.267077+0000","last_active":"2023-10-10T20:39:37.267077+0000","last_peered":"2023-10-10T20:39:37.267077+0000","last_clean":"2023-10-10T20:39:37.267077+0000","last_became_active":"2023-10-10T20:39:37.266325+0000","last_became_peered":"2023-10-10T20:39:37.266325+0000","last_unstale":"2023-10-10T20:39:37.267077+0000","last_undegraded":"2023-10-10T20:39:37.267077+0000","last_fullsized":"2023-10-10T20:39:37.267077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:03.986546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259437+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.262060+0000","last_change":"2023-10-10T20:39:37.262060+0000","last_active":"2023-10-10T20:39:37.262060+0000","last_peered":"2023-10-10T20:39:37.262060+0000","last_clean":"2023-10-10T20:39:37.262060+0000","last_became_active":"2023-10-10T20:39:37.261636+0000","last_became_peered":"2023-10-10T20:39:37.261636+0000","last_unstale":"2023-10-10T20:39:37.262060+0000","last_undegraded":"2023-10-10T20:39:37.262060+0000","last_fullsized":"2023-10-10T20:39:37.262060+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:15.557848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252172+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264795+0000","last_change":"2023-10-10T20:39:37.264795+0000","last_active":"2023-10-10T20:39:37.264795+0000","last_peered":"2023-10-10T20:39:37.264795+0000","last_clean":"2023-10-10T20:39:37.264795+0000","last_became_active":"2023-10-10T20:39:37.263322+0000","last_became_peered":"2023-10-10T20:39:37.263322+0000","last_unstale":"2023-10-10T20:39:37.264795+0000","last_undegraded":"2023-10-10T20:39:37.264795+0000","last_fullsized":"2023-10-10T20:39:37.264795+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:15.557848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258150+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:22.796992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.263674+0000","last_change":"2023-10-10T20:39:37.263674+0000","last_active":"2023-10-10T20:39:37.263674+0000","last_peered":"2023-10-10T20:39:37.263674+0000","last_clean":"2023-10-10T20:39:37.263674+0000","last_became_active":"2023-10-10T20:39:37.263294+0000","last_became_peered":"2023-10-10T20:39:37.263294+0000","last_unstale":"2023-10-10T20:39:37.263674+0000","last_undegraded":"2023-10-10T20:39:37.263674+0000","last_fullsized":"2023-10-10T20:39:37.263674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:02.132949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.265134+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:43.066089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264553+0000","last_change":"2023-10-10T20:39:37.264553+0000","last_active":"2023-10-10T20:39:37.264553+0000","last_peered":"2023-10-10T20:39:37.264553+0000","last_clean":"2023-10-10T20:39:37.264553+0000","last_became_active":"2023-10-10T20:39:37.264274+0000","last_became_peered":"2023-10-10T20:39:37.264274+0000","last_unstale":"2023-10-10T20:39:37.264553+0000","last_undegraded":"2023-10-10T20:39:37.264553+0000","last_fullsized":"2023-10-10T20:39:37.264553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:25.579798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":217,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-10T20:39:36.243840+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:39:36.243840+0000","last_peered":"2023-10-10T20:39:36.243840+0000","last_clean":"2023-10-10T20:39:36.243840+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:39:36.243840+0000","last_undegraded":"2023-10-10T20:39:36.243840+0000","last_fullsized":"2023-10-10T20:39:36.243840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298756,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073084,"statfs":{"total":96636764160,"available":96330838016,"internally_reserved":0,"allocated":1503232,"data_stored":978234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.69699999999999995,"15min":0.88100000000000001},"min":{"1min":0.32500000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.68500000000000005,"5min":6.8630000000000004,"15min":6.8630000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.59899999999999998,"15min":0.65000000000000002},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.752,"5min":4.1959999999999997,"15min":4.1959999999999997},"last":0.41899999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.79800000000000004,"15min":1.0980000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.70099999999999996,"5min":10.1,"15min":10.1},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.66700000000000004,"15min":0.76700000000000002},"min":{"1min":0.42999999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71199999999999997,"5min":6.806,"15min":6.806},"last":0.498}]},{"osd":2,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.70699999999999996,"15min":0.82399999999999995},"min":{"1min":0.42899999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70899999999999996,"5min":6.9340000000000002,"15min":6.9340000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.503,"5min":0.65400000000000003,"15min":0.77800000000000002},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69599999999999995,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.436}]},{"osd":3,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.67700000000000005,"15min":0.79700000000000004},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.72199999999999998,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63400000000000001,"15min":0.69199999999999995},"min":{"1min":0.42799999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76200000000000001,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.53900000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.314,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.65900000000000003,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.32900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69199999999999995,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.52900000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.376,"5min":0.314,"15min":0.314},"max":{"1min":0.78900000000000003,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53200000000000003,"15min":0.52500000000000002},"min":{"1min":0.42899999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68000000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.51600000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73799999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.51300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73699999999999999,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.46800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.59399999999999997,"15min":0.70099999999999996},"min":{"1min":0.36499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.72899999999999998,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.629,"15min":0.749},"min":{"1min":0.35299999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66000000000000003,"5min":7.8090000000000002,"15min":7.8090000000000002},"last":0.45100000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.628,"15min":0.746},"min":{"1min":0.36899999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":7.867,"15min":7.867},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.73999999999999999,"15min":0.95399999999999996},"min":{"1min":0.36299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70199999999999996,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.57799999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.55600000000000005,"15min":0.60999999999999999},"min":{"1min":0.371,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.65900000000000003,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.71499999999999997,"15min":0.90100000000000002},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.80000000000000004,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":0.53200000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.58599999999999997,"15min":0.63400000000000001},"min":{"1min":0.38600000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.65800000000000003,"5min":4.8620000000000001,"15min":4.8620000000000001},"last":0.501},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.64700000000000002,"15min":0.75},"min":{"1min":0.439,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68899999999999995,"5min":7.9009999999999998,"15min":7.9009999999999998},"last":0.55300000000000005}]},{"osd":4,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48599999999999999,"15min":0.45800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68400000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.438,"5min":0.37,"15min":0.37},"max":{"1min":0.73999999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.62}]},{"osd":5,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.5,"15min":0.48199999999999998},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.68899999999999995,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71199999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.48599999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52500000000000002,"15min":0.51100000000000001},"min":{"1min":0.377,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73299999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.38600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.745,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.59299999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.40600000000000003,"15min":0.505},"min":{"1min":0.19600000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.40200000000000002,"5min":5.0380000000000003,"15min":5.0380000000000003},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.34000000000000002,"15min":0.36799999999999999},"min":{"1min":0.224,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45200000000000001,"5min":1.194,"15min":1.194},"last":0.35899999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.34100000000000003,"15min":0.36899999999999999},"min":{"1min":0.215,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.42599999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.307,"5min":0.42399999999999999,"15min":0.53400000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45900000000000002,"5min":5.0919999999999996,"15min":5.0919999999999996},"last":0.5}]},{"osd":3,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.45200000000000001,"15min":0.55800000000000005},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.442,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.376},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.38900000000000001,"15min":0.41499999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.48999999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.44500000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57599999999999996,"15min":0.56499999999999995},"min":{"1min":0.38800000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73899999999999999,"5min":1.254,"15min":1.254},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.64500000000000002,"15min":0.73199999999999998},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82699999999999996,"5min":4.9850000000000003,"15min":4.9850000000000003},"last":0.63}]},{"osd":5,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.58199999999999996,"15min":0.59599999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72699999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59999999999999998,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83599999999999997,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.73999999999999999,"15min":0.88800000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89100000000000001,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.41799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89800000000000002,"5min":1.171,"15min":1.171},"last":0.70199999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.93300000000000005,"5min":1.214,"15min":1.214},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.67500000000000004,"15min":0.75},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.748,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.73199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302848,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068992,"statfs":{"total":96636764160,"available":96326647808,"internally_reserved":0,"allocated":1499136,"data_stored":977273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.44,"15min":0.57199999999999995},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.50900000000000001,"5min":7.9249999999999998,"15min":7.9249999999999998},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.42899999999999999,"15min":0.54200000000000004},"min":{"1min":0.22500000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.47199999999999998,"5min":6.6299999999999999,"15min":6.6299999999999999},"last":0.31900000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.45900000000000002,"15min":0.59699999999999998},"min":{"1min":0.23999999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.55400000000000005,"5min":8.2509999999999994,"15min":8.2509999999999994},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.45300000000000001,"15min":0.56599999999999995},"min":{"1min":0.223,"5min":0.222,"15min":0.222},"max":{"1min":0.54100000000000004,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.35299999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.46700000000000003,"15min":0.59799999999999998},"min":{"1min":0.23100000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54100000000000004,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.51800000000000002,"15min":0.65500000000000003},"min":{"1min":0.29199999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53300000000000003,"5min":7.8929999999999998,"15min":7.8929999999999998},"last":0.33300000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.65600000000000003,"15min":0.747},"min":{"1min":0.39400000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.88900000000000001,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.68700000000000006,"15min":0.81299999999999994},"min":{"1min":0.374,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78700000000000003,"5min":7.6920000000000002,"15min":7.6920000000000002},"last":0.54300000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.67400000000000004,"15min":0.79200000000000004},"min":{"1min":0.376,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90200000000000002,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.69499999999999995,"15min":0.81599999999999995},"min":{"1min":0.38100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79900000000000004,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":0.63100000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.72599999999999998,"15min":0.85899999999999999},"min":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81399999999999995,"5min":7.8360000000000003,"15min":7.8360000000000003},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70299999999999996,"15min":0.79200000000000004},"min":{"1min":0.46100000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92000000000000004,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.64500000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.68600000000000005,"15min":0.78400000000000003},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86399999999999999,"5min":6.7160000000000002,"15min":6.7160000000000002},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.70799999999999996,"15min":0.82499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93100000000000005,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.61299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.31900000000000001,"15min":0.32800000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.59699999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.33600000000000002,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45400000000000001,"5min":0.746,"15min":0.746},"last":0.38200000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.45400000000000001,"15min":0.77100000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.63,"5min":11.967000000000001,"15min":11.967000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39000000000000001,"15min":0.498},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52700000000000002,"5min":4.1100000000000003,"15min":4.1100000000000003},"last":0.34200000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.41199999999999998,"15min":0.57899999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48799999999999999,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.50900000000000001,"15min":0.82199999999999995},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65900000000000003,"5min":11.935,"15min":11.935},"last":0.45600000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.65900000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.80100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.58999999999999997},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.495}]},{"osd":6,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.33400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83599999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63700000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81599999999999995,"5min":1.052,"15min":1.052},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64500000000000002,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85999999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.62}]}]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298164,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073676,"statfs":{"total":96636764160,"available":96331444224,"internally_reserved":0,"allocated":897024,"data_stored":381603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.309,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44400000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.317},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48699999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.307}]},{"osd":1,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33600000000000002,"15min":0.33000000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.49199999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38200000000000001,"15min":0.376},"min":{"1min":0.307,"5min":0.251,"15min":0.251},"max":{"1min":0.53900000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.38200000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.34899999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35499999999999998,"15min":0.36099999999999999},"min":{"1min":0.25900000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.51600000000000001,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.32600000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.38100000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83299999999999996,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.54500000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.39400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78700000000000003,"5min":1.05,"15min":1.05},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86799999999999999,"5min":1.01,"15min":1.01},"last":0.441}]},{"osd":6,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92000000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.877,"5min":1.026,"15min":1.026},"last":0.47599999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77100000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.46600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89400000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.52500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58199999999999996,"5min":0.747,"15min":0.747},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.59799999999999998,"15min":0.79200000000000004},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69699999999999995,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.69699999999999995}]},{"osd":1,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.60299999999999998,"15min":0.79900000000000004},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60299999999999998,"5min":7.7160000000000002,"15min":7.7160000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54900000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68100000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63600000000000001,"15min":0.81799999999999995},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.48699999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.58899999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.57299999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.502,"15min":0.48399999999999999},"min":{"1min":0.34599999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.622},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.38400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77300000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.77300000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.41699999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72099999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.52600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.67400000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.58199999999999996},"min":{"1min":0.44700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.27900000000000003,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.59699999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.32000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.73299999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.70399999999999996}]},{"osd":1,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.60199999999999998,"15min":0.66600000000000004},"min":{"1min":0.38500000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.66300000000000003,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.58499999999999996,"15min":0.65100000000000002},"min":{"1min":0.33500000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65200000000000002,"5min":5.3109999999999999,"15min":5.3109999999999999},"last":0.68200000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.60799999999999998,"15min":0.67400000000000004},"min":{"1min":0.373,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72799999999999998,"5min":5.2960000000000003,"15min":5.2960000000000003},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.33900000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71999999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.68999999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.64100000000000001,"15min":0.70699999999999996},"min":{"1min":0.39900000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80000000000000004,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.63700000000000001,"15min":0.71699999999999997},"min":{"1min":0.439,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69499999999999995,"5min":5.2850000000000001,"15min":5.2850000000000001},"last":0.61799999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48799999999999999,"15min":0.499},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64200000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.35099999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.501}]},{"osd":6,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.86099999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.753,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.71299999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.36499999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84799999999999998,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82199999999999995,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.72699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:39:42.695 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:39:42.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.841+0000 7fed58c13700 1 Processor -- start 2023-10-10T20:39:42.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.841+0000 7fed58c13700 1 -- start start 2023-10-10T20:39:42.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.842+0000 7fed58c13700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fed5406c0a0 0x7fed540caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:42.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.842+0000 7fed58c13700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed5415df50 0x7fed5415e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:42.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.842+0000 7fed58c13700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed5415d7e0 con 0x7fed5406bcd0 2023-10-10T20:39:42.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.842+0000 7fed58c13700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed5415dac0 con 0x7fed5406c0a0 2023-10-10T20:39:42.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.842+0000 7fed58c13700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed5415aa30 con 0x7fed5415df50 2023-10-10T20:39:42.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.843+0000 7fed52d9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed5415df50 0x7fed5415e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:42.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.843+0000 7fed49d9b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fed5406c0a0 0x7fed540caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.843+0000 7fed52d9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed5415df50 0x7fed5415e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:33924/0 (socket says 172.21.15.89:33924) 2023-10-10T20:39:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.843+0000 7fed52d9d700 1 -- 172.21.15.89:0/2400322638 learned_addr learned my addr 172.21.15.89:0/2400322638 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.843+0000 7fed52d9d700 1 -- 172.21.15.89:0/2400322638 >> v2:172.21.15.111:3300/0 conn(0x7fed5406c0a0 msgr2=0x7fed540caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.843+0000 7fed52d9d700 1 --2- 172.21.15.89:0/2400322638 >> v2:172.21.15.111:3300/0 conn(0x7fed5406c0a0 0x7fed540caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.843+0000 7fed52d9d700 1 -- 172.21.15.89:0/2400322638 >> v1:172.21.15.111:6789/0 conn(0x7fed5406bcd0 legacy=0x7fed5415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.843+0000 7fed52d9d700 1 -- 172.21.15.89:0/2400322638 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5415ad10 con 0x7fed5415df50 2023-10-10T20:39:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.843+0000 7fed49d9b700 1 --2- 172.21.15.89:0/2400322638 >> v2:172.21.15.111:3300/0 conn(0x7fed5406c0a0 0x7fed540caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:39:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.843+0000 7fed52d9d700 1 --2- 172.21.15.89:0/2400322638 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed5415df50 0x7fed5415e3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009f20 tx=0x7fed4c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6a200c2f13b6e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.843+0000 7fed51d9b700 1 -- 172.21.15.89:0/2400322638 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c00e070 con 0x7fed5415df50 2023-10-10T20:39:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.843+0000 7fed51d9b700 1 -- 172.21.15.89:0/2400322638 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4c00b9d0 con 0x7fed5415df50 2023-10-10T20:39:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.843+0000 7fed51d9b700 1 -- 172.21.15.89:0/2400322638 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c0092e0 con 0x7fed5415df50 2023-10-10T20:39:42.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.844+0000 7fed58c13700 1 -- 172.21.15.89:0/2400322638 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed5415df50 msgr2=0x7fed5415e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:42.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.844+0000 7fed58c13700 1 --2- 172.21.15.89:0/2400322638 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed5415df50 0x7fed5415e3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009f20 tx=0x7fed4c009840 comp rx=0 tx=0).stop 2023-10-10T20:39:42.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.844+0000 7fed58c13700 1 -- 172.21.15.89:0/2400322638 shutdown_connections 2023-10-10T20:39:42.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.844+0000 7fed58c13700 1 --2- 172.21.15.89:0/2400322638 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed5415df50 0x7fed5415e3f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.844+0000 7fed58c13700 1 --2- 172.21.15.89:0/2400322638 >> v2:172.21.15.111:3300/0 conn(0x7fed5406c0a0 0x7fed540caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.844+0000 7fed58c13700 1 -- 172.21.15.89:0/2400322638 >> 172.21.15.89:0/2400322638 conn(0x7fed540ce600 msgr2=0x7fed540cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:42.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.844+0000 7fed58c13700 1 -- 172.21.15.89:0/2400322638 shutdown_connections 2023-10-10T20:39:42.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.844+0000 7fed58c13700 1 -- 172.21.15.89:0/2400322638 wait complete. 2023-10-10T20:39:42.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.845+0000 7fed58c13700 1 Processor -- start 2023-10-10T20:39:42.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.845+0000 7fed58c13700 1 -- start start 2023-10-10T20:39:42.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.846+0000 7fed58c13700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed5406bcd0 0x7fed541587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:42.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.846+0000 7fed58c13700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed5406c0a0 0x7fed54158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:42.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.846+0000 7fed58c13700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed5415df50 0x7fed540c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:42.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.846+0000 7fed58c13700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed540b1990 con 0x7fed5406bcd0 2023-10-10T20:39:42.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.846+0000 7fed58c13700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed540b13a0 con 0x7fed5406c0a0 2023-10-10T20:39:42.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.846+0000 7fed58c13700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed54159540 con 0x7fed5415df50 2023-10-10T20:39:42.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.846+0000 7fed5259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed5406bcd0 0x7fed541587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:42.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.846+0000 7fed5259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed5406bcd0 0x7fed541587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41878/0 (socket says 172.21.15.89:41878) 2023-10-10T20:39:42.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.846+0000 7fed5259c700 1 -- 172.21.15.89:0/1555616057 learned_addr learned my addr 172.21.15.89:0/1555616057 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:42.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.846+0000 7fed49d9b700 1 --2- 172.21.15.89:0/1555616057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed5406c0a0 0x7fed54158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:42.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.846+0000 7fed52d9d700 1 --2- 172.21.15.89:0/1555616057 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed5415df50 0x7fed540c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:42.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.846+0000 7fed5259c700 1 -- 172.21.15.89:0/1555616057 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed5415df50 msgr2=0x7fed540c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:42.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.846+0000 7fed5259c700 1 --2- 172.21.15.89:0/1555616057 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed5415df50 0x7fed540c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.846+0000 7fed5259c700 1 -- 172.21.15.89:0/1555616057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed5406c0a0 msgr2=0x7fed54158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:42.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.846+0000 7fed5259c700 1 --2- 172.21.15.89:0/1555616057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed5406c0a0 0x7fed54158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:42.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.846+0000 7fed5259c700 1 -- 172.21.15.89:0/1555616057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c009420 con 0x7fed5406bcd0 2023-10-10T20:39:42.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.846+0000 7fed49d9b700 1 --2- 172.21.15.89:0/1555616057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed5406c0a0 0x7fed54158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:39:42.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.847+0000 7fed5259c700 1 --2- 172.21.15.89:0/1555616057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed5406bcd0 0x7fed541587a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fed4400c4e0 tx=0x7fed4400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:42.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.847+0000 7fed4affd700 1 -- 172.21.15.89:0/1555616057 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44011590 con 0x7fed5406bcd0 2023-10-10T20:39:42.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.847+0000 7fed4affd700 1 -- 172.21.15.89:0/1555616057 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed44011710 con 0x7fed5406bcd0 2023-10-10T20:39:42.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.847+0000 7fed58c13700 1 -- 172.21.15.89:0/1555616057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4c009ae0 con 0x7fed5406bcd0 2023-10-10T20:39:42.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.847+0000 7fed4affd700 1 -- 172.21.15.89:0/1555616057 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44011ad0 con 0x7fed5406bcd0 2023-10-10T20:39:42.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.847+0000 7fed58c13700 1 -- 172.21.15.89:0/1555616057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed541598d0 con 0x7fed5406bcd0 2023-10-10T20:39:42.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.848+0000 7fed58c13700 1 -- 172.21.15.89:0/1555616057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed5404c3c0 con 0x7fed5406bcd0 2023-10-10T20:39:42.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.848+0000 7fed4affd700 1 -- 172.21.15.89:0/1555616057 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fed4401b3b0 con 0x7fed5406bcd0 2023-10-10T20:39:42.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.848+0000 7fed4affd700 1 --2- 172.21.15.89:0/1555616057 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fed34067170 0x7fed34069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:42.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.849+0000 7fed4affd700 1 -- 172.21.15.89:0/1555616057 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fed44083c40 con 0x7fed5406bcd0 2023-10-10T20:39:42.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.849+0000 7fed49d9b700 1 --2- 172.21.15.89:0/1555616057 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fed34067170 0x7fed34069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:42.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.849+0000 7fed49d9b700 1 --2- 172.21.15.89:0/1555616057 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fed34067170 0x7fed34069620 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fed54155870 tx=0x7fed3c0091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:42.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:42.853+0000 7fed4affd700 1 -- 172.21.15.89:0/1555616057 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed44050c20 con 0x7fed5406bcd0 2023-10-10T20:39:43.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.033+0000 7fed58c13700 1 -- 172.21.15.89:0/1555616057 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fed54159a60 con 0x7fed34067170 2023-10-10T20:39:43.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.041+0000 7fed4affd700 1 -- 172.21.15.89:0/1555616057 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286833 (secure 0 0 0) 0x7fed54159a60 con 0x7fed34067170 2023-10-10T20:39:43.046 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:43.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.044+0000 7fed58c13700 1 -- 172.21.15.89:0/1555616057 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fed34067170 msgr2=0x7fed34069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:43.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.044+0000 7fed58c13700 1 --2- 172.21.15.89:0/1555616057 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fed34067170 0x7fed34069620 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fed54155870 tx=0x7fed3c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.044+0000 7fed58c13700 1 -- 172.21.15.89:0/1555616057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed5406bcd0 msgr2=0x7fed541587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:43.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.044+0000 7fed58c13700 1 --2- 172.21.15.89:0/1555616057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed5406bcd0 0x7fed541587a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fed4400c4e0 tx=0x7fed4400c7f0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.044+0000 7fed58c13700 1 -- 172.21.15.89:0/1555616057 shutdown_connections 2023-10-10T20:39:43.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.044+0000 7fed58c13700 1 --2- 172.21.15.89:0/1555616057 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fed5415df50 0x7fed540c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.044+0000 7fed58c13700 1 --2- 172.21.15.89:0/1555616057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed5406bcd0 0x7fed541587a0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.044+0000 7fed58c13700 1 --2- 172.21.15.89:0/1555616057 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fed34067170 0x7fed34069620 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.044+0000 7fed58c13700 1 --2- 172.21.15.89:0/1555616057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed5406c0a0 0x7fed54158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.044+0000 7fed58c13700 1 -- 172.21.15.89:0/1555616057 >> 172.21.15.89:0/1555616057 conn(0x7fed540ce600 msgr2=0x7fed5406cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:43.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.044+0000 7fed58c13700 1 -- 172.21.15.89:0/1555616057 shutdown_connections 2023-10-10T20:39:43.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.044+0000 7fed58c13700 1 -- 172.21.15.89:0/1555616057 wait complete. 2023-10-10T20:39:43.052 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:39:43.058 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2023-10-10T20:39:42.714005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2390648,"kb_used_data":8248,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752584072,"statfs":{"total":773094113280,"available":770646089728,"internally_reserved":0,"allocated":8445952,"data_stored":4276440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001227"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.262078+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:18:44.235474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261385+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:32:45.306980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253087+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:55:15.047824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258454+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:10:36.151908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254702+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:17.810574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258770+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:49:29.215389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257832+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:12:11.985278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257660+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258331+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:55:15.047824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261413+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261949+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:51:25.472971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259733+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:51.489549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253308+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:49:29.215389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.271693+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:27.148718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257856+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:56:02.724230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252996+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:22.796992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252806+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:21:45.349150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257789+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:56:39.215508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260126+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:09.824624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257801+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:55:16.441311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258944+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:45:29.295885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.266901+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:22.796992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258929+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:51:00.261125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.271424+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:13:05.408163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261055+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:10:42.178506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253538+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:45.015636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254993+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261378+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.263848+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:24.508374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.266498+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:32.888171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254896+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258291+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:44.990290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.255157+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.275787+0000","last_change":"2023-10-10T20:39:37.275787+0000","last_active":"2023-10-10T20:39:37.275787+0000","last_peered":"2023-10-10T20:39:37.275787+0000","last_clean":"2023-10-10T20:39:37.275787+0000","last_became_active":"2023-10-10T20:39:37.275528+0000","last_became_peered":"2023-10-10T20:39:37.275528+0000","last_unstale":"2023-10-10T20:39:37.275787+0000","last_undegraded":"2023-10-10T20:39:37.275787+0000","last_fullsized":"2023-10-10T20:39:37.275787+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:23.501900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260922+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.262406+0000","last_change":"2023-10-10T20:39:37.262406+0000","last_active":"2023-10-10T20:39:37.262406+0000","last_peered":"2023-10-10T20:39:37.262406+0000","last_clean":"2023-10-10T20:39:37.262406+0000","last_became_active":"2023-10-10T20:39:37.261561+0000","last_became_peered":"2023-10-10T20:39:37.261561+0000","last_unstale":"2023-10-10T20:39:37.262406+0000","last_undegraded":"2023-10-10T20:39:37.262406+0000","last_fullsized":"2023-10-10T20:39:37.262406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:19:53.834466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260876+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:17.810574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.276745+0000","last_change":"2023-10-10T20:39:37.276745+0000","last_active":"2023-10-10T20:39:37.276745+0000","last_peered":"2023-10-10T20:39:37.276745+0000","last_clean":"2023-10-10T20:39:37.276745+0000","last_became_active":"2023-10-10T20:39:37.276469+0000","last_became_peered":"2023-10-10T20:39:37.276469+0000","last_unstale":"2023-10-10T20:39:37.276745+0000","last_undegraded":"2023-10-10T20:39:37.276745+0000","last_fullsized":"2023-10-10T20:39:37.276745+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:50.483075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252593+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:18:44.235474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.275690+0000","last_change":"2023-10-10T20:39:37.275690+0000","last_active":"2023-10-10T20:39:37.275690+0000","last_peered":"2023-10-10T20:39:37.275690+0000","last_clean":"2023-10-10T20:39:37.275690+0000","last_became_active":"2023-10-10T20:39:37.275412+0000","last_became_peered":"2023-10-10T20:39:37.275412+0000","last_unstale":"2023-10-10T20:39:37.275690+0000","last_undegraded":"2023-10-10T20:39:37.275690+0000","last_fullsized":"2023-10-10T20:39:37.275690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:50:59.254651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.266049+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:16.564322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.267783+0000","last_change":"2023-10-10T20:39:37.267783+0000","last_active":"2023-10-10T20:39:37.267783+0000","last_peered":"2023-10-10T20:39:37.267783+0000","last_clean":"2023-10-10T20:39:37.267783+0000","last_became_active":"2023-10-10T20:39:37.267436+0000","last_became_peered":"2023-10-10T20:39:37.267436+0000","last_unstale":"2023-10-10T20:39:37.267783+0000","last_undegraded":"2023-10-10T20:39:37.267783+0000","last_fullsized":"2023-10-10T20:39:37.267783+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:42.059614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254194+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:04.993021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.267358+0000","last_change":"2023-10-10T20:39:37.267358+0000","last_active":"2023-10-10T20:39:37.267358+0000","last_peered":"2023-10-10T20:39:37.267358+0000","last_clean":"2023-10-10T20:39:37.267358+0000","last_became_active":"2023-10-10T20:39:37.267054+0000","last_became_peered":"2023-10-10T20:39:37.267054+0000","last_unstale":"2023-10-10T20:39:37.267358+0000","last_undegraded":"2023-10-10T20:39:37.267358+0000","last_fullsized":"2023-10-10T20:39:37.267358+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:31.881697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253409+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:49.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.265548+0000","last_change":"2023-10-10T20:39:37.265548+0000","last_active":"2023-10-10T20:39:37.265548+0000","last_peered":"2023-10-10T20:39:37.265548+0000","last_clean":"2023-10-10T20:39:37.265548+0000","last_became_active":"2023-10-10T20:39:37.265284+0000","last_became_peered":"2023-10-10T20:39:37.265284+0000","last_unstale":"2023-10-10T20:39:37.265548+0000","last_undegraded":"2023-10-10T20:39:37.265548+0000","last_fullsized":"2023-10-10T20:39:37.265548+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:53:22.002691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257399+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:02:13.214283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.270081+0000","last_change":"2023-10-10T20:39:37.270081+0000","last_active":"2023-10-10T20:39:37.270081+0000","last_peered":"2023-10-10T20:39:37.270081+0000","last_clean":"2023-10-10T20:39:37.270081+0000","last_became_active":"2023-10-10T20:39:37.269828+0000","last_became_peered":"2023-10-10T20:39:37.269828+0000","last_unstale":"2023-10-10T20:39:37.270081+0000","last_undegraded":"2023-10-10T20:39:37.270081+0000","last_fullsized":"2023-10-10T20:39:37.270081+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:16:06.819183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257391+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:41:32.637048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264681+0000","last_change":"2023-10-10T20:39:37.264681+0000","last_active":"2023-10-10T20:39:37.264681+0000","last_peered":"2023-10-10T20:39:37.264681+0000","last_clean":"2023-10-10T20:39:37.264681+0000","last_became_active":"2023-10-10T20:39:37.264173+0000","last_became_peered":"2023-10-10T20:39:37.264173+0000","last_unstale":"2023-10-10T20:39:37.264681+0000","last_undegraded":"2023-10-10T20:39:37.264681+0000","last_fullsized":"2023-10-10T20:39:37.264681+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:31.881697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257813+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:09.824624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.270474+0000","last_change":"2023-10-10T20:39:37.270474+0000","last_active":"2023-10-10T20:39:37.270474+0000","last_peered":"2023-10-10T20:39:37.270474+0000","last_clean":"2023-10-10T20:39:37.270474+0000","last_became_active":"2023-10-10T20:39:37.270158+0000","last_became_peered":"2023-10-10T20:39:37.270158+0000","last_unstale":"2023-10-10T20:39:37.270474+0000","last_undegraded":"2023-10-10T20:39:37.270474+0000","last_fullsized":"2023-10-10T20:39:37.270474+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:19:53.834466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259123+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272341+0000","last_change":"2023-10-10T20:39:37.272341+0000","last_active":"2023-10-10T20:39:37.272341+0000","last_peered":"2023-10-10T20:39:37.272341+0000","last_clean":"2023-10-10T20:39:37.272341+0000","last_became_active":"2023-10-10T20:39:37.272070+0000","last_became_peered":"2023-10-10T20:39:37.272070+0000","last_unstale":"2023-10-10T20:39:37.272341+0000","last_undegraded":"2023-10-10T20:39:37.272341+0000","last_fullsized":"2023-10-10T20:39:37.272341+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:42.059614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254514+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:26.586272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264016+0000","last_change":"2023-10-10T20:39:37.264016+0000","last_active":"2023-10-10T20:39:37.264016+0000","last_peered":"2023-10-10T20:39:37.264016+0000","last_clean":"2023-10-10T20:39:37.264016+0000","last_became_active":"2023-10-10T20:39:37.263724+0000","last_became_peered":"2023-10-10T20:39:37.263724+0000","last_unstale":"2023-10-10T20:39:37.264016+0000","last_undegraded":"2023-10-10T20:39:37.264016+0000","last_fullsized":"2023-10-10T20:39:37.264016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:25.579798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258964+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272988+0000","last_change":"2023-10-10T20:39:37.272988+0000","last_active":"2023-10-10T20:39:37.272988+0000","last_peered":"2023-10-10T20:39:37.272988+0000","last_clean":"2023-10-10T20:39:37.272988+0000","last_became_active":"2023-10-10T20:39:37.272715+0000","last_became_peered":"2023-10-10T20:39:37.272715+0000","last_unstale":"2023-10-10T20:39:37.272988+0000","last_undegraded":"2023-10-10T20:39:37.272988+0000","last_fullsized":"2023-10-10T20:39:37.272988+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:10:21.213337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252142+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.268230+0000","last_change":"2023-10-10T20:39:37.268230+0000","last_active":"2023-10-10T20:39:37.268230+0000","last_peered":"2023-10-10T20:39:37.268230+0000","last_clean":"2023-10-10T20:39:37.268230+0000","last_became_active":"2023-10-10T20:39:37.267986+0000","last_became_peered":"2023-10-10T20:39:37.267986+0000","last_unstale":"2023-10-10T20:39:37.268230+0000","last_undegraded":"2023-10-10T20:39:37.268230+0000","last_fullsized":"2023-10-10T20:39:37.268230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:13:05.731066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.264627+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:24:00.218312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.269974+0000","last_change":"2023-10-10T20:39:37.269974+0000","last_active":"2023-10-10T20:39:37.269974+0000","last_peered":"2023-10-10T20:39:37.269974+0000","last_clean":"2023-10-10T20:39:37.269974+0000","last_became_active":"2023-10-10T20:39:37.269650+0000","last_became_peered":"2023-10-10T20:39:37.269650+0000","last_unstale":"2023-10-10T20:39:37.269974+0000","last_undegraded":"2023-10-10T20:39:37.269974+0000","last_fullsized":"2023-10-10T20:39:37.269974+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:16.804099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260831+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:55:56.167036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.266948+0000","last_change":"2023-10-10T20:39:37.266948+0000","last_active":"2023-10-10T20:39:37.266948+0000","last_peered":"2023-10-10T20:39:37.266948+0000","last_clean":"2023-10-10T20:39:37.266948+0000","last_became_active":"2023-10-10T20:39:37.266149+0000","last_became_peered":"2023-10-10T20:39:37.266149+0000","last_unstale":"2023-10-10T20:39:37.266948+0000","last_undegraded":"2023-10-10T20:39:37.266948+0000","last_fullsized":"2023-10-10T20:39:37.266948+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:25.579798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252135+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.265234+0000","last_change":"2023-10-10T20:39:37.265234+0000","last_active":"2023-10-10T20:39:37.265234+0000","last_peered":"2023-10-10T20:39:37.265234+0000","last_clean":"2023-10-10T20:39:37.265234+0000","last_became_active":"2023-10-10T20:39:37.264953+0000","last_became_peered":"2023-10-10T20:39:37.264953+0000","last_unstale":"2023-10-10T20:39:37.265234+0000","last_undegraded":"2023-10-10T20:39:37.265234+0000","last_fullsized":"2023-10-10T20:39:37.265234+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:53:22.002691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260398+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:26.586272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.269676+0000","last_change":"2023-10-10T20:39:37.269676+0000","last_active":"2023-10-10T20:39:37.269676+0000","last_peered":"2023-10-10T20:39:37.269676+0000","last_clean":"2023-10-10T20:39:37.269676+0000","last_became_active":"2023-10-10T20:39:37.269437+0000","last_became_peered":"2023-10-10T20:39:37.269437+0000","last_unstale":"2023-10-10T20:39:37.269676+0000","last_undegraded":"2023-10-10T20:39:37.269676+0000","last_fullsized":"2023-10-10T20:39:37.269676+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:43.983815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253984+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:51:00.261125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264681+0000","last_change":"2023-10-10T20:39:37.264681+0000","last_active":"2023-10-10T20:39:37.264681+0000","last_peered":"2023-10-10T20:39:37.264681+0000","last_clean":"2023-10-10T20:39:37.264681+0000","last_became_active":"2023-10-10T20:39:37.264410+0000","last_became_peered":"2023-10-10T20:39:37.264410+0000","last_unstale":"2023-10-10T20:39:37.264681+0000","last_undegraded":"2023-10-10T20:39:37.264681+0000","last_fullsized":"2023-10-10T20:39:37.264681+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:21.790518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259115+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:02:53.563766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272951+0000","last_change":"2023-10-10T20:39:37.272951+0000","last_active":"2023-10-10T20:39:37.272951+0000","last_peered":"2023-10-10T20:39:37.272951+0000","last_clean":"2023-10-10T20:39:37.272951+0000","last_became_active":"2023-10-10T20:39:37.272680+0000","last_became_peered":"2023-10-10T20:39:37.272680+0000","last_unstale":"2023-10-10T20:39:37.272951+0000","last_undegraded":"2023-10-10T20:39:37.272951+0000","last_fullsized":"2023-10-10T20:39:37.272951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:09:37.235149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257781+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272869+0000","last_change":"2023-10-10T20:39:37.272869+0000","last_active":"2023-10-10T20:39:37.272869+0000","last_peered":"2023-10-10T20:39:37.272869+0000","last_clean":"2023-10-10T20:39:37.272869+0000","last_became_active":"2023-10-10T20:39:37.272418+0000","last_became_peered":"2023-10-10T20:39:37.272418+0000","last_unstale":"2023-10-10T20:39:37.272869+0000","last_undegraded":"2023-10-10T20:39:37.272869+0000","last_fullsized":"2023-10-10T20:39:37.272869+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:20:16.442207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259051+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:57.313252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272054+0000","last_change":"2023-10-10T20:39:37.272054+0000","last_active":"2023-10-10T20:39:37.272054+0000","last_peered":"2023-10-10T20:39:37.272054+0000","last_clean":"2023-10-10T20:39:37.272054+0000","last_became_active":"2023-10-10T20:39:37.271818+0000","last_became_peered":"2023-10-10T20:39:37.271818+0000","last_unstale":"2023-10-10T20:39:37.272054+0000","last_undegraded":"2023-10-10T20:39:37.272054+0000","last_fullsized":"2023-10-10T20:39:37.272054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:57:37.129049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258156+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:18:44.235474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.275925+0000","last_change":"2023-10-10T20:39:37.275925+0000","last_active":"2023-10-10T20:39:37.275925+0000","last_peered":"2023-10-10T20:39:37.275925+0000","last_clean":"2023-10-10T20:39:37.275925+0000","last_became_active":"2023-10-10T20:39:37.275666+0000","last_became_peered":"2023-10-10T20:39:37.275666+0000","last_unstale":"2023-10-10T20:39:37.275925+0000","last_undegraded":"2023-10-10T20:39:37.275925+0000","last_fullsized":"2023-10-10T20:39:37.275925+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:01:58.347464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254173+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:51.489549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.265642+0000","last_change":"2023-10-10T20:39:37.265642+0000","last_active":"2023-10-10T20:39:37.265642+0000","last_peered":"2023-10-10T20:39:37.265642+0000","last_clean":"2023-10-10T20:39:37.265642+0000","last_became_active":"2023-10-10T20:39:37.265399+0000","last_became_peered":"2023-10-10T20:39:37.265399+0000","last_unstale":"2023-10-10T20:39:37.265642+0000","last_undegraded":"2023-10-10T20:39:37.265642+0000","last_fullsized":"2023-10-10T20:39:37.265642+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:53:41.456643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253320+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:09.824624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272730+0000","last_change":"2023-10-10T20:39:37.272730+0000","last_active":"2023-10-10T20:39:37.272730+0000","last_peered":"2023-10-10T20:39:37.272730+0000","last_clean":"2023-10-10T20:39:37.272730+0000","last_became_active":"2023-10-10T20:39:37.272487+0000","last_became_peered":"2023-10-10T20:39:37.272487+0000","last_unstale":"2023-10-10T20:39:37.272730+0000","last_undegraded":"2023-10-10T20:39:37.272730+0000","last_fullsized":"2023-10-10T20:39:37.272730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:40.276426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260216+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:04.993021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.275883+0000","last_change":"2023-10-10T20:39:37.275883+0000","last_active":"2023-10-10T20:39:37.275883+0000","last_peered":"2023-10-10T20:39:37.275883+0000","last_clean":"2023-10-10T20:39:37.275883+0000","last_became_active":"2023-10-10T20:39:37.275635+0000","last_became_peered":"2023-10-10T20:39:37.275635+0000","last_unstale":"2023-10-10T20:39:37.275883+0000","last_undegraded":"2023-10-10T20:39:37.275883+0000","last_fullsized":"2023-10-10T20:39:37.275883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:23:59.211838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253448+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:44.990290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264870+0000","last_change":"2023-10-10T20:39:37.264870+0000","last_active":"2023-10-10T20:39:37.264870+0000","last_peered":"2023-10-10T20:39:37.264870+0000","last_clean":"2023-10-10T20:39:37.264870+0000","last_became_active":"2023-10-10T20:39:37.264606+0000","last_became_peered":"2023-10-10T20:39:37.264606+0000","last_unstale":"2023-10-10T20:39:37.264870+0000","last_undegraded":"2023-10-10T20:39:37.264870+0000","last_fullsized":"2023-10-10T20:39:37.264870+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:02.132949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258303+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:45.015636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.267077+0000","last_change":"2023-10-10T20:39:37.267077+0000","last_active":"2023-10-10T20:39:37.267077+0000","last_peered":"2023-10-10T20:39:37.267077+0000","last_clean":"2023-10-10T20:39:37.267077+0000","last_became_active":"2023-10-10T20:39:37.266325+0000","last_became_peered":"2023-10-10T20:39:37.266325+0000","last_unstale":"2023-10-10T20:39:37.267077+0000","last_undegraded":"2023-10-10T20:39:37.267077+0000","last_fullsized":"2023-10-10T20:39:37.267077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:03.986546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259437+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.262060+0000","last_change":"2023-10-10T20:39:37.262060+0000","last_active":"2023-10-10T20:39:37.262060+0000","last_peered":"2023-10-10T20:39:37.262060+0000","last_clean":"2023-10-10T20:39:37.262060+0000","last_became_active":"2023-10-10T20:39:37.261636+0000","last_became_peered":"2023-10-10T20:39:37.261636+0000","last_unstale":"2023-10-10T20:39:37.262060+0000","last_undegraded":"2023-10-10T20:39:37.262060+0000","last_fullsized":"2023-10-10T20:39:37.262060+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:15.557848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252172+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264795+0000","last_change":"2023-10-10T20:39:37.264795+0000","last_active":"2023-10-10T20:39:37.264795+0000","last_peered":"2023-10-10T20:39:37.264795+0000","last_clean":"2023-10-10T20:39:37.264795+0000","last_became_active":"2023-10-10T20:39:37.263322+0000","last_became_peered":"2023-10-10T20:39:37.263322+0000","last_unstale":"2023-10-10T20:39:37.264795+0000","last_undegraded":"2023-10-10T20:39:37.264795+0000","last_fullsized":"2023-10-10T20:39:37.264795+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:15.557848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258150+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:22.796992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.263674+0000","last_change":"2023-10-10T20:39:37.263674+0000","last_active":"2023-10-10T20:39:37.263674+0000","last_peered":"2023-10-10T20:39:37.263674+0000","last_clean":"2023-10-10T20:39:37.263674+0000","last_became_active":"2023-10-10T20:39:37.263294+0000","last_became_peered":"2023-10-10T20:39:37.263294+0000","last_unstale":"2023-10-10T20:39:37.263674+0000","last_undegraded":"2023-10-10T20:39:37.263674+0000","last_fullsized":"2023-10-10T20:39:37.263674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:02.132949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.265134+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:43.066089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264553+0000","last_change":"2023-10-10T20:39:37.264553+0000","last_active":"2023-10-10T20:39:37.264553+0000","last_peered":"2023-10-10T20:39:37.264553+0000","last_clean":"2023-10-10T20:39:37.264553+0000","last_became_active":"2023-10-10T20:39:37.264274+0000","last_became_peered":"2023-10-10T20:39:37.264274+0000","last_unstale":"2023-10-10T20:39:37.264553+0000","last_undegraded":"2023-10-10T20:39:37.264553+0000","last_fullsized":"2023-10-10T20:39:37.264553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:25.579798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":217,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-10T20:39:36.243840+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:39:36.243840+0000","last_peered":"2023-10-10T20:39:36.243840+0000","last_clean":"2023-10-10T20:39:36.243840+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:39:36.243840+0000","last_undegraded":"2023-10-10T20:39:36.243840+0000","last_fullsized":"2023-10-10T20:39:36.243840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298756,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073084,"statfs":{"total":96636764160,"available":96330838016,"internally_reserved":0,"allocated":1503232,"data_stored":978234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.69699999999999995,"15min":0.88100000000000001},"min":{"1min":0.32500000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.68500000000000005,"5min":6.8630000000000004,"15min":6.8630000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.59899999999999998,"15min":0.65000000000000002},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.752,"5min":4.1959999999999997,"15min":4.1959999999999997},"last":0.41899999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.79800000000000004,"15min":1.0980000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.70099999999999996,"5min":10.1,"15min":10.1},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.66700000000000004,"15min":0.76700000000000002},"min":{"1min":0.42999999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71199999999999997,"5min":6.806,"15min":6.806},"last":0.498}]},{"osd":2,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.70699999999999996,"15min":0.82399999999999995},"min":{"1min":0.42899999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70899999999999996,"5min":6.9340000000000002,"15min":6.9340000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.503,"5min":0.65400000000000003,"15min":0.77800000000000002},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69599999999999995,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.436}]},{"osd":3,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.67700000000000005,"15min":0.79700000000000004},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.72199999999999998,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63400000000000001,"15min":0.69199999999999995},"min":{"1min":0.42799999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76200000000000001,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.53900000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.314,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.65900000000000003,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.32900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69199999999999995,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.52900000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.376,"5min":0.314,"15min":0.314},"max":{"1min":0.78900000000000003,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53200000000000003,"15min":0.52500000000000002},"min":{"1min":0.42899999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68000000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.51600000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73799999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.51300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73699999999999999,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.46800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.59399999999999997,"15min":0.70099999999999996},"min":{"1min":0.36499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.72899999999999998,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.629,"15min":0.749},"min":{"1min":0.35299999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66000000000000003,"5min":7.8090000000000002,"15min":7.8090000000000002},"last":0.45100000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.628,"15min":0.746},"min":{"1min":0.36899999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":7.867,"15min":7.867},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.73999999999999999,"15min":0.95399999999999996},"min":{"1min":0.36299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70199999999999996,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.57799999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.55600000000000005,"15min":0.60999999999999999},"min":{"1min":0.371,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.65900000000000003,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.71499999999999997,"15min":0.90100000000000002},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.80000000000000004,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":0.53200000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.58599999999999997,"15min":0.63400000000000001},"min":{"1min":0.38600000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.65800000000000003,"5min":4.8620000000000001,"15min":4.8620000000000001},"last":0.501},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.64700000000000002,"15min":0.75},"min":{"1min":0.439,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68899999999999995,"5min":7.9009999999999998,"15min":7.9009999999999998},"last":0.55300000000000005}]},{"osd":4,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48599999999999999,"15min":0.45800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68400000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.438,"5min":0.37,"15min":0.37},"max":{"1min":0.73999999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.62}]},{"osd":5,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.5,"15min":0.48199999999999998},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.68899999999999995,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71199999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.48599999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52500000000000002,"15min":0.51100000000000001},"min":{"1min":0.377,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73299999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.38600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.745,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.59299999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.40600000000000003,"15min":0.505},"min":{"1min":0.19600000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.40200000000000002,"5min":5.0380000000000003,"15min":5.0380000000000003},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.34000000000000002,"15min":0.36799999999999999},"min":{"1min":0.224,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45200000000000001,"5min":1.194,"15min":1.194},"last":0.35899999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.34100000000000003,"15min":0.36899999999999999},"min":{"1min":0.215,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.42599999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.307,"5min":0.42399999999999999,"15min":0.53400000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45900000000000002,"5min":5.0919999999999996,"15min":5.0919999999999996},"last":0.5}]},{"osd":3,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.45200000000000001,"15min":0.55800000000000005},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.442,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.376},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.38900000000000001,"15min":0.41499999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.48999999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.44500000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57599999999999996,"15min":0.56499999999999995},"min":{"1min":0.38800000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73899999999999999,"5min":1.254,"15min":1.254},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.64500000000000002,"15min":0.73199999999999998},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82699999999999996,"5min":4.9850000000000003,"15min":4.9850000000000003},"last":0.63}]},{"osd":5,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.58199999999999996,"15min":0.59599999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72699999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59999999999999998,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83599999999999997,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.73999999999999999,"15min":0.88800000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89100000000000001,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.41799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89800000000000002,"5min":1.171,"15min":1.171},"last":0.70199999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.93300000000000005,"5min":1.214,"15min":1.214},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.67500000000000004,"15min":0.75},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.748,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.73199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302848,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068992,"statfs":{"total":96636764160,"available":96326647808,"internally_reserved":0,"allocated":1499136,"data_stored":977273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.44,"15min":0.57199999999999995},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.50900000000000001,"5min":7.9249999999999998,"15min":7.9249999999999998},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.42899999999999999,"15min":0.54200000000000004},"min":{"1min":0.22500000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.47199999999999998,"5min":6.6299999999999999,"15min":6.6299999999999999},"last":0.31900000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.45900000000000002,"15min":0.59699999999999998},"min":{"1min":0.23999999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.55400000000000005,"5min":8.2509999999999994,"15min":8.2509999999999994},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.45300000000000001,"15min":0.56599999999999995},"min":{"1min":0.223,"5min":0.222,"15min":0.222},"max":{"1min":0.54100000000000004,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.35299999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.46700000000000003,"15min":0.59799999999999998},"min":{"1min":0.23100000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54100000000000004,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.51800000000000002,"15min":0.65500000000000003},"min":{"1min":0.29199999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53300000000000003,"5min":7.8929999999999998,"15min":7.8929999999999998},"last":0.33300000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.65600000000000003,"15min":0.747},"min":{"1min":0.39400000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.88900000000000001,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.68700000000000006,"15min":0.81299999999999994},"min":{"1min":0.374,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78700000000000003,"5min":7.6920000000000002,"15min":7.6920000000000002},"last":0.54300000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.67400000000000004,"15min":0.79200000000000004},"min":{"1min":0.376,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90200000000000002,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.69499999999999995,"15min":0.81599999999999995},"min":{"1min":0.38100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79900000000000004,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":0.63100000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.72599999999999998,"15min":0.85899999999999999},"min":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81399999999999995,"5min":7.8360000000000003,"15min":7.8360000000000003},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70299999999999996,"15min":0.79200000000000004},"min":{"1min":0.46100000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92000000000000004,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.64500000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.68600000000000005,"15min":0.78400000000000003},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86399999999999999,"5min":6.7160000000000002,"15min":6.7160000000000002},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.70799999999999996,"15min":0.82499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93100000000000005,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.61299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.31900000000000001,"15min":0.32800000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.59699999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.33600000000000002,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45400000000000001,"5min":0.746,"15min":0.746},"last":0.38200000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.45400000000000001,"15min":0.77100000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.63,"5min":11.967000000000001,"15min":11.967000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39000000000000001,"15min":0.498},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52700000000000002,"5min":4.1100000000000003,"15min":4.1100000000000003},"last":0.34200000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.41199999999999998,"15min":0.57899999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48799999999999999,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.50900000000000001,"15min":0.82199999999999995},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65900000000000003,"5min":11.935,"15min":11.935},"last":0.45600000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.65900000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.80100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.58999999999999997},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.495}]},{"osd":6,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.33400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83599999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63700000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81599999999999995,"5min":1.052,"15min":1.052},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64500000000000002,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85999999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.62}]}]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298164,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073676,"statfs":{"total":96636764160,"available":96331444224,"internally_reserved":0,"allocated":897024,"data_stored":381603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.309,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44400000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.317},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48699999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.307}]},{"osd":1,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33600000000000002,"15min":0.33000000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.49199999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38200000000000001,"15min":0.376},"min":{"1min":0.307,"5min":0.251,"15min":0.251},"max":{"1min":0.53900000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.38200000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.34899999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35499999999999998,"15min":0.36099999999999999},"min":{"1min":0.25900000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.51600000000000001,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.32600000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.38100000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83299999999999996,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.54500000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.39400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78700000000000003,"5min":1.05,"15min":1.05},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86799999999999999,"5min":1.01,"15min":1.01},"last":0.441}]},{"osd":6,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92000000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.877,"5min":1.026,"15min":1.026},"last":0.47599999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77100000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.46600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89400000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.52500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58199999999999996,"5min":0.747,"15min":0.747},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.59799999999999998,"15min":0.79200000000000004},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69699999999999995,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.69699999999999995}]},{"osd":1,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.60299999999999998,"15min":0.79900000000000004},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60299999999999998,"5min":7.7160000000000002,"15min":7.7160000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54900000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68100000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63600000000000001,"15min":0.81799999999999995},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.48699999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.58899999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.57299999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.502,"15min":0.48399999999999999},"min":{"1min":0.34599999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.622},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.38400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77300000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.77300000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.41699999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72099999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.52600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.67400000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.58199999999999996},"min":{"1min":0.44700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.27900000000000003,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.59699999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.32000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.73299999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.70399999999999996}]},{"osd":1,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.60199999999999998,"15min":0.66600000000000004},"min":{"1min":0.38500000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.66300000000000003,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.58499999999999996,"15min":0.65100000000000002},"min":{"1min":0.33500000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65200000000000002,"5min":5.3109999999999999,"15min":5.3109999999999999},"last":0.68200000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.60799999999999998,"15min":0.67400000000000004},"min":{"1min":0.373,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72799999999999998,"5min":5.2960000000000003,"15min":5.2960000000000003},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.33900000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71999999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.68999999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.64100000000000001,"15min":0.70699999999999996},"min":{"1min":0.39900000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80000000000000004,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.63700000000000001,"15min":0.71699999999999997},"min":{"1min":0.439,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69499999999999995,"5min":5.2850000000000001,"15min":5.2850000000000001},"last":0.61799999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48799999999999999,"15min":0.499},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64200000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.35099999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.501}]},{"osd":6,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.86099999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.753,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.71299999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.36499999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84799999999999998,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82199999999999995,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.72699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:39:43.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3f is not active+clean 2023-10-10T20:39:43.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.262078+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:18:44.235474+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:39:43.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3e is not active+clean 2023-10-10T20:39:43.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.261385+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:32:45.306980+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:39:43.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3d is not active+clean 2023-10-10T20:39:43.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.253087+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:55:15.047824+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:39:43.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3c is not active+clean 2023-10-10T20:39:43.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.258454+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:10:36.151908+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:39:43.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3b is not active+clean 2023-10-10T20:39:43.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.254702+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:52:17.810574+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:39:43.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3a is not active+clean 2023-10-10T20:39:43.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.258770+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:49:29.215389+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:39:43.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.39 is not active+clean 2023-10-10T20:39:43.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.257832+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:12:11.985278+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:39:43.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.38 is not active+clean 2023-10-10T20:39:43.070 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.257660+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:39:43.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.37 is not active+clean 2023-10-10T20:39:43.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.258331+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:55:15.047824+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:39:43.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.36 is not active+clean 2023-10-10T20:39:43.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.261413+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:39:43.072 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.35 is not active+clean 2023-10-10T20:39:43.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.261949+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:51:25.472971+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:39:43.073 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.34 is not active+clean 2023-10-10T20:39:43.073 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.259733+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T20:50:51.489549+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:39:43.073 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.33 is not active+clean 2023-10-10T20:39:43.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.253308+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:49:29.215389+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:39:43.074 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.32 is not active+clean 2023-10-10T20:39:43.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.271693+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:52:27.148718+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:39:43.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.31 is not active+clean 2023-10-10T20:39:43.075 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.257856+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:56:02.724230+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:39:43.076 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.30 is not active+clean 2023-10-10T20:39:43.076 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.252996+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:30:22.796992+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:39:43.076 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2f is not active+clean 2023-10-10T20:39:43.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.252806+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:21:45.349150+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:39:43.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2e is not active+clean 2023-10-10T20:39:43.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.257789+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:56:39.215508+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:39:43.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2d is not active+clean 2023-10-10T20:39:43.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.260126+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:41:09.824624+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:39:43.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2c is not active+clean 2023-10-10T20:39:43.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.257801+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:55:16.441311+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:39:43.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2b is not active+clean 2023-10-10T20:39:43.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.258944+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:45:29.295885+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:39:43.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2a is not active+clean 2023-10-10T20:39:43.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.266901+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:30:22.796992+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:39:43.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.29 is not active+clean 2023-10-10T20:39:43.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.258929+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:51:00.261125+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:39:43.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.28 is not active+clean 2023-10-10T20:39:43.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.271424+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:13:05.408163+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:39:43.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.27 is not active+clean 2023-10-10T20:39:43.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.261055+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:10:42.178506+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:39:43.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.26 is not active+clean 2023-10-10T20:39:43.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.253538+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:21:45.015636+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:39:43.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.25 is not active+clean 2023-10-10T20:39:43.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.254993+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:39:43.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.24 is not active+clean 2023-10-10T20:39:43.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.261378+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:39:43.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.23 is not active+clean 2023-10-10T20:39:43.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.263848+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:48:24.508374+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:39:43.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.22 is not active+clean 2023-10-10T20:39:43.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.266498+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:17:32.888171+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:39:43.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.21 is not active+clean 2023-10-10T20:39:43.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.254896+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:39:43.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.20 is not active+clean 2023-10-10T20:39:43.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.258291+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:35:44.990290+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:39:43.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1e is not active+clean 2023-10-10T20:39:43.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.255157+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:39:43.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1f is not active+clean 2023-10-10T20:39:43.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.260922+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:39:43.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1c is not active+clean 2023-10-10T20:39:43.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.260876+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:52:17.810574+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:39:43.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1d is not active+clean 2023-10-10T20:39:43.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.252593+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:18:44.235474+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:39:43.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1a is not active+clean 2023-10-10T20:39:43.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.266049+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:54:16.564322+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:39:43.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1b is not active+clean 2023-10-10T20:39:43.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.254194+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:03:04.993021+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:39:43.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.18 is not active+clean 2023-10-10T20:39:43.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.253409+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:34:49.058619+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:39:43.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.19 is not active+clean 2023-10-10T20:39:43.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.257399+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:02:13.214283+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:39:43.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.16 is not active+clean 2023-10-10T20:39:43.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.257391+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:41:32.637048+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:39:43.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.17 is not active+clean 2023-10-10T20:39:43.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.257813+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:41:09.824624+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:39:43.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.14 is not active+clean 2023-10-10T20:39:43.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.259123+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:39:43.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.15 is not active+clean 2023-10-10T20:39:43.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.254514+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:51:26.586272+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:39:43.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.12 is not active+clean 2023-10-10T20:39:43.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.258964+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:39:43.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.13 is not active+clean 2023-10-10T20:39:43.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.252142+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:39:43.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.10 is not active+clean 2023-10-10T20:39:43.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.264627+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:24:00.218312+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:39:43.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.11 is not active+clean 2023-10-10T20:39:43.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.260831+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T20:55:56.167036+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:39:43.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.e is not active+clean 2023-10-10T20:39:43.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.252135+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:39:43.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.f is not active+clean 2023-10-10T20:39:43.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.260398+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:51:26.586272+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:39:43.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.d is not active+clean 2023-10-10T20:39:43.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.253984+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:51:00.261125+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:39:43.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.a is not active+clean 2023-10-10T20:39:43.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.259115+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:02:53.563766+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:39:43.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.b is not active+clean 2023-10-10T20:39:43.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.257781+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:39:43.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.8 is not active+clean 2023-10-10T20:39:43.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.259051+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:48:57.313252+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:39:43.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.9 is not active+clean 2023-10-10T20:39:43.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.258156+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:18:44.235474+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:39:43.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.6 is not active+clean 2023-10-10T20:39:43.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.254173+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T20:50:51.489549+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:39:43.108 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.7 is not active+clean 2023-10-10T20:39:43.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.253320+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:41:09.824624+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:39:43.109 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.4 is not active+clean 2023-10-10T20:39:43.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.260216+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:03:04.993021+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:39:43.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.5 is not active+clean 2023-10-10T20:39:43.110 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.253448+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:35:44.990290+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:39:43.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2 is not active+clean 2023-10-10T20:39:43.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.258303+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:21:45.015636+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:39:43.111 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3 is not active+clean 2023-10-10T20:39:43.112 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.259437+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:39:43.112 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.0 is not active+clean 2023-10-10T20:39:43.112 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.252172+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:39:43.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1 is not active+clean 2023-10-10T20:39:43.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.258150+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:30:22.796992+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:39:43.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.c is not active+clean 2023-10-10T20:39:43.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:39:37.239917+0000', 'last_change': '2023-10-10T20:39:37.265134+0000', 'last_active': '2023-10-10T20:39:37.239917+0000', 'last_peered': '2023-10-10T20:39:37.239917+0000', 'last_clean': '2023-10-10T20:39:37.239917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:39:37.239917+0000', 'last_undegraded': '2023-10-10T20:39:37.239917+0000', 'last_fullsized': '2023-10-10T20:39:37.239917+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'last_clean_scrub_stamp': '2023-10-10T20:39:37.239917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:58:43.066089+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:39:43.115 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-10T20:39:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.226+0000 7f6030956700 1 Processor -- start 2023-10-10T20:39:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.226+0000 7f6030956700 1 -- start start 2023-10-10T20:39:43.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.227+0000 7f6030956700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f602c155360 0x7f602c0b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.227+0000 7f6030956700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602c15c320 con 0x7f602c0ced30 2023-10-10T20:39:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.227+0000 7f6030956700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602c15b710 con 0x7f602c0b40f0 2023-10-10T20:39:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.227+0000 7f6030956700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f602c0b4590 con 0x7f602c155360 2023-10-10T20:39:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.227+0000 7f602ad9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f602c0ced30 0x7f602c0cc9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48196/0 (socket says 172.21.15.89:48196) 2023-10-10T20:39:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.227+0000 7f602ad9d700 1 -- 172.21.15.89:0/3476791495 learned_addr learned my addr 172.21.15.89:0/3476791495 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:43.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f602a59c700 1 --2- 172.21.15.89:0/3476791495 >> v2:172.21.15.111:3300/0 conn(0x7f602c155360 0x7f602c0b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:43.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f6029d9b700 1 -- 172.21.15.89:0/3476791495 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035193131 0 0) 0x7f602c15c320 con 0x7f602c0ced30 2023-10-10T20:39:43.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f6029d9b700 1 -- 172.21.15.89:0/3476791495 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6018003570 con 0x7f602c0ced30 2023-10-10T20:39:43.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f6029d9b700 1 -- 172.21.15.89:0/3476791495 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3963492282 0 0) 0x7f6018003570 con 0x7f602c0ced30 2023-10-10T20:39:43.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f6029d9b700 1 -- 172.21.15.89:0/3476791495 >> v2:172.21.15.111:3300/0 conn(0x7f602c155360 msgr2=0x7f602c0b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:43.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f6029d9b700 1 --2- 172.21.15.89:0/3476791495 >> v2:172.21.15.111:3300/0 conn(0x7f602c155360 0x7f602c0b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f6029d9b700 1 -- 172.21.15.89:0/3476791495 >> v1:172.21.15.111:6789/0 conn(0x7f602c0b40f0 legacy=0x7f602c0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:43.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f6029d9b700 1 -- 172.21.15.89:0/3476791495 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602c15b9f0 con 0x7f602c0ced30 2023-10-10T20:39:43.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f6029d9b700 1 -- 172.21.15.89:0/3476791495 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f60240017f0 con 0x7f602c0ced30 2023-10-10T20:39:43.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f6029d9b700 1 -- 172.21.15.89:0/3476791495 >> v1:172.21.15.89:6789/0 conn(0x7f602c0ced30 legacy=0x7f602c0cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f6029d9b700 1 --2- 172.21.15.89:0/3476791495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6018003a60 0x7f6018003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f6029d9b700 1 --2- 172.21.15.89:0/3476791495 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6018004810 0x7f6018004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f6029d9b700 1 --2- 172.21.15.89:0/3476791495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60180051c0 0x7f601800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f6029d9b700 1 -- 172.21.15.89:0/3476791495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602c0b4590 con 0x7f6018003a60 2023-10-10T20:39:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f6029d9b700 1 -- 172.21.15.89:0/3476791495 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601800c0d0 con 0x7f60180051c0 2023-10-10T20:39:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f6029d9b700 1 -- 172.21.15.89:0/3476791495 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601800c290 con 0x7f6018004810 2023-10-10T20:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f602a59c700 1 --2- 172.21.15.89:0/3476791495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6018003a60 0x7f6018003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f6021d9b700 1 --2- 172.21.15.89:0/3476791495 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6018004810 0x7f6018004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f602ad9d700 1 --2- 172.21.15.89:0/3476791495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60180051c0 0x7f601800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f602a59c700 1 -- 172.21.15.89:0/3476791495 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6018004810 msgr2=0x7f6018004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f602a59c700 1 --2- 172.21.15.89:0/3476791495 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6018004810 0x7f6018004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f602a59c700 1 -- 172.21.15.89:0/3476791495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60180051c0 msgr2=0x7f601800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f602a59c700 1 --2- 172.21.15.89:0/3476791495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60180051c0 0x7f601800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f602a59c700 1 -- 172.21.15.89:0/3476791495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6018003570 con 0x7f6018003a60 2023-10-10T20:39:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f602ad9d700 1 -- 172.21.15.89:0/3476791495 reap_dead start 2023-10-10T20:39:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f6021d9b700 1 --2- 172.21.15.89:0/3476791495 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6018004810 0x7f6018004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.228+0000 7f602a59c700 1 --2- 172.21.15.89:0/3476791495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6018003a60 0x7f6018003e70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f601c009c00 tx=0x7f601c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57822158e2e35c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.229+0000 7f6029d9b700 1 -- 172.21.15.89:0/3476791495 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c00e710 con 0x7f6018003a60 2023-10-10T20:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.229+0000 7f6029d9b700 1 -- 172.21.15.89:0/3476791495 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c00e870 con 0x7f6018003a60 2023-10-10T20:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.229+0000 7f6029d9b700 1 -- 172.21.15.89:0/3476791495 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c00ec30 con 0x7f6018003a60 2023-10-10T20:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.229+0000 7f6030956700 1 -- 172.21.15.89:0/3476791495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6018003a60 msgr2=0x7f6018003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.229+0000 7f6030956700 1 --2- 172.21.15.89:0/3476791495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6018003a60 0x7f6018003e70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f601c009c00 tx=0x7f601c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.229+0000 7f6030956700 1 -- 172.21.15.89:0/3476791495 shutdown_connections 2023-10-10T20:39:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.229+0000 7f6030956700 1 --2- 172.21.15.89:0/3476791495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6018003a60 0x7f6018003e70 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.229+0000 7f6030956700 1 -- 172.21.15.89:0/3476791495 >> 172.21.15.89:0/3476791495 conn(0x7f602c06baf0 msgr2=0x7f602c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.229+0000 7f6030956700 1 -- 172.21.15.89:0/3476791495 shutdown_connections 2023-10-10T20:39:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.229+0000 7f6030956700 1 -- 172.21.15.89:0/3476791495 wait complete. 2023-10-10T20:39:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.230+0000 7f6030956700 1 Processor -- start 2023-10-10T20:39:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.230+0000 7f6030956700 1 -- start start 2023-10-10T20:39:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.231+0000 7f6030956700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6018003a60 0x7f602c1401b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:43.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.231+0000 7f6030956700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f602c0b0670 0x7f602c142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:43.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.231+0000 7f6030956700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f602c16b5d0 0x7f602c16ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:43.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.231+0000 7f6030956700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602c14fb30 con 0x7f602c0b0670 2023-10-10T20:39:43.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.231+0000 7f6030956700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602c16bfb0 con 0x7f602c16b5d0 2023-10-10T20:39:43.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.231+0000 7f6030956700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602c13ec00 con 0x7f6018003a60 2023-10-10T20:39:43.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.231+0000 7f6021d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f602c0b0670 0x7f602c142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:43.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.231+0000 7f6021d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f602c0b0670 0x7f602c142ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41882/0 (socket says 172.21.15.89:41882) 2023-10-10T20:39:43.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.231+0000 7f6021d9b700 1 -- 172.21.15.89:0/3114595235 learned_addr learned my addr 172.21.15.89:0/3114595235 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:43.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.231+0000 7f602a59c700 1 --2- 172.21.15.89:0/3114595235 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6018003a60 0x7f602c1401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.231+0000 7f602ad9d700 1 --2- 172.21.15.89:0/3114595235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f602c16b5d0 0x7f602c16ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.231+0000 7f6021d9b700 1 -- 172.21.15.89:0/3114595235 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6018003a60 msgr2=0x7f602c1401b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.231+0000 7f6021d9b700 1 --2- 172.21.15.89:0/3114595235 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6018003a60 0x7f602c1401b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.232+0000 7f6021d9b700 1 -- 172.21.15.89:0/3114595235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f602c16b5d0 msgr2=0x7f602c16ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.232+0000 7f6021d9b700 1 --2- 172.21.15.89:0/3114595235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f602c16b5d0 0x7f602c16ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.232+0000 7f6021d9b700 1 -- 172.21.15.89:0/3114595235 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602c13dff0 con 0x7f602c0b0670 2023-10-10T20:39:43.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.232+0000 7f602ad9d700 1 --2- 172.21.15.89:0/3114595235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f602c16b5d0 0x7f602c16ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:43.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.232+0000 7f6021d9b700 1 --2- 172.21.15.89:0/3114595235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f602c0b0670 0x7f602c142ba0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6014000c00 tx=0x7f6014009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:43.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.232+0000 7f6022ffd700 1 -- 172.21.15.89:0/3114595235 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60140115c0 con 0x7f602c0b0670 2023-10-10T20:39:43.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.232+0000 7f6022ffd700 1 -- 172.21.15.89:0/3114595235 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6014011720 con 0x7f602c0b0670 2023-10-10T20:39:43.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.232+0000 7f6030956700 1 -- 172.21.15.89:0/3114595235 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602c13e2d0 con 0x7f602c0b0670 2023-10-10T20:39:43.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.232+0000 7f6022ffd700 1 -- 172.21.15.89:0/3114595235 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6014011ac0 con 0x7f602c0b0670 2023-10-10T20:39:43.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.232+0000 7f6030956700 1 -- 172.21.15.89:0/3114595235 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602c13e890 con 0x7f602c0b0670 2023-10-10T20:39:43.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.233+0000 7f6030956700 1 -- 172.21.15.89:0/3114595235 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f602c13edb0 con 0x7f602c0b0670 2023-10-10T20:39:43.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.233+0000 7f6022ffd700 1 -- 172.21.15.89:0/3114595235 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f6014014480 con 0x7f602c0b0670 2023-10-10T20:39:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.234+0000 7f6022ffd700 1 --2- 172.21.15.89:0/3114595235 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f600c067170 0x7f600c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.234+0000 7f6022ffd700 1 -- 172.21.15.89:0/3114595235 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f60140841d0 con 0x7f602c0b0670 2023-10-10T20:39:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.234+0000 7f602a59c700 1 --2- 172.21.15.89:0/3114595235 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f600c067170 0x7f600c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.234+0000 7f602a59c700 1 --2- 172.21.15.89:0/3114595235 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f600c067170 0x7f600c069620 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f601c0091e0 tx=0x7f601c01a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.238+0000 7f6022ffd700 1 -- 172.21.15.89:0/3114595235 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f601404d5a0 con 0x7f602c0b0670 2023-10-10T20:39:43.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.500+0000 7f6030956700 1 -- 172.21.15.89:0/3114595235 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f602c04c3c0 con 0x7f602c0b0670 2023-10-10T20:39:43.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.500+0000 7f6022ffd700 1 -- 172.21.15.89:0/3114595235 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1036 (secure 0 0 0) 0x7f602c13edb0 con 0x7f602c0b0670 2023-10-10T20:39:43.503 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:43.503 INFO:teuthology.orchestra.run.smithi089.stdout:{"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":326,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":76,"num_osds":8,"num_up_osds":8,"osd_up_since":1696970067,"num_in_osds":8,"osd_in_since":1696970059,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2448023552,"bytes_avail":770646089728,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":46,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":6087}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-10-10T20:37:28.685504+0000","services":{}},"progress_events":{}} 2023-10-10T20:39:43.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.502+0000 7f6030956700 1 -- 172.21.15.89:0/3114595235 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f600c067170 msgr2=0x7f600c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:43.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.502+0000 7f6030956700 1 --2- 172.21.15.89:0/3114595235 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f600c067170 0x7f600c069620 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f601c0091e0 tx=0x7f601c01a000 comp rx=0 tx=0).stop 2023-10-10T20:39:43.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.502+0000 7f6030956700 1 -- 172.21.15.89:0/3114595235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f602c0b0670 msgr2=0x7f602c142ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:43.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.502+0000 7f6030956700 1 --2- 172.21.15.89:0/3114595235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f602c0b0670 0x7f602c142ba0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6014000c00 tx=0x7f6014009040 comp rx=0 tx=0).stop 2023-10-10T20:39:43.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.502+0000 7f6030956700 1 -- 172.21.15.89:0/3114595235 shutdown_connections 2023-10-10T20:39:43.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.502+0000 7f6030956700 1 --2- 172.21.15.89:0/3114595235 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f600c067170 0x7f600c069620 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.502+0000 7f6030956700 1 --2- 172.21.15.89:0/3114595235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f602c16b5d0 0x7f602c16ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.502+0000 7f6030956700 1 --2- 172.21.15.89:0/3114595235 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6018003a60 0x7f602c1401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.502+0000 7f6030956700 1 --2- 172.21.15.89:0/3114595235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f602c0b0670 0x7f602c142ba0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.502+0000 7f6030956700 1 -- 172.21.15.89:0/3114595235 >> 172.21.15.89:0/3114595235 conn(0x7f602c06baf0 msgr2=0x7f602c06d130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:43.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.502+0000 7f6030956700 1 -- 172.21.15.89:0/3114595235 shutdown_connections 2023-10-10T20:39:43.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.502+0000 7f6030956700 1 -- 172.21.15.89:0/3114595235 wait complete. 2023-10-10T20:39:43.517 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-10T20:39:43.517 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:39:43.557 INFO:tasks.ceph.mds.b.smithi111.stderr:2023-10-10T20:39:43.556+0000 7f03c0505700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:39:43.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.629+0000 7f7c2c896700 1 Processor -- start 2023-10-10T20:39:43.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.629+0000 7f7c2c896700 1 -- start start 2023-10-10T20:39:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.630+0000 7f7c2c896700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7c28069560 0x7f7c280699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.630+0000 7f7c2c896700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c28069ee0 con 0x7f7c28159c60 2023-10-10T20:39:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.630+0000 7f7c2c896700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2815a160 con 0x7f7c28159890 2023-10-10T20:39:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.630+0000 7f7c2c896700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c280d13a0 con 0x7f7c28069560 2023-10-10T20:39:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.630+0000 7f7c1dd9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7c28159c60 0x7f7c2815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48204/0 (socket says 172.21.15.89:48204) 2023-10-10T20:39:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.630+0000 7f7c1dd9b700 1 -- 172.21.15.89:0/278126929 learned_addr learned my addr 172.21.15.89:0/278126929 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.630+0000 7f7c26d9d700 1 --2- 172.21.15.89:0/278126929 >> v2:172.21.15.111:3300/0 conn(0x7f7c28069560 0x7f7c280699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.630+0000 7f7c25d9b700 1 -- 172.21.15.89:0/278126929 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251543505 0 0) 0x7f7c28069ee0 con 0x7f7c28159c60 2023-10-10T20:39:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.630+0000 7f7c25d9b700 1 -- 172.21.15.89:0/278126929 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c14003570 con 0x7f7c28159c60 2023-10-10T20:39:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c25d9b700 1 -- 172.21.15.89:0/278126929 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2806819021 0 0) 0x7f7c14003570 con 0x7f7c28159c60 2023-10-10T20:39:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c25d9b700 1 -- 172.21.15.89:0/278126929 >> v2:172.21.15.111:3300/0 conn(0x7f7c28069560 msgr2=0x7f7c280699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c25d9b700 1 --2- 172.21.15.89:0/278126929 >> v2:172.21.15.111:3300/0 conn(0x7f7c28069560 0x7f7c280699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c25d9b700 1 -- 172.21.15.89:0/278126929 >> v1:172.21.15.111:6789/0 conn(0x7f7c28159890 legacy=0x7f7c2815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c25d9b700 1 -- 172.21.15.89:0/278126929 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2815e140 con 0x7f7c28159c60 2023-10-10T20:39:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c25d9b700 1 -- 172.21.15.89:0/278126929 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f7c10001470 con 0x7f7c28159c60 2023-10-10T20:39:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c25d9b700 1 -- 172.21.15.89:0/278126929 >> v1:172.21.15.89:6789/0 conn(0x7f7c28159c60 legacy=0x7f7c2815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c25d9b700 1 --2- 172.21.15.89:0/278126929 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7c14003a60 0x7f7c14003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c25d9b700 1 --2- 172.21.15.89:0/278126929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c14004810 0x7f7c14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c25d9b700 1 --2- 172.21.15.89:0/278126929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c140051c0 0x7f7c1400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c25d9b700 1 -- 172.21.15.89:0/278126929 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c280d13a0 con 0x7f7c140051c0 2023-10-10T20:39:43.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c25d9b700 1 -- 172.21.15.89:0/278126929 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1400c0d0 con 0x7f7c14004810 2023-10-10T20:39:43.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c25d9b700 1 -- 172.21.15.89:0/278126929 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1400c290 con 0x7f7c14003a60 2023-10-10T20:39:43.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c26d9d700 1 --2- 172.21.15.89:0/278126929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c140051c0 0x7f7c1400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:43.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c1dd9b700 1 --2- 172.21.15.89:0/278126929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c14004810 0x7f7c14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:43.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c2659c700 1 --2- 172.21.15.89:0/278126929 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7c14003a60 0x7f7c14003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:43.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c26d9d700 1 -- 172.21.15.89:0/278126929 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7c14003a60 msgr2=0x7f7c14003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:43.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c26d9d700 1 --2- 172.21.15.89:0/278126929 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7c14003a60 0x7f7c14003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c26d9d700 1 -- 172.21.15.89:0/278126929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c14004810 msgr2=0x7f7c14004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:43.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c26d9d700 1 --2- 172.21.15.89:0/278126929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c14004810 0x7f7c14004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c26d9d700 1 -- 172.21.15.89:0/278126929 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c14003570 con 0x7f7c140051c0 2023-10-10T20:39:43.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c26d9d700 1 -- 172.21.15.89:0/278126929 reap_dead start 2023-10-10T20:39:43.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c26d9d700 1 --2- 172.21.15.89:0/278126929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c140051c0 0x7f7c1400ba60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000e210 tx=0x7f7c2000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc46a4ebd2d3e2a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:43.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c25d9b700 1 -- 172.21.15.89:0/278126929 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2000eac0 con 0x7f7c140051c0 2023-10-10T20:39:43.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c25d9b700 1 -- 172.21.15.89:0/278126929 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2000f040 con 0x7f7c140051c0 2023-10-10T20:39:43.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.631+0000 7f7c25d9b700 1 -- 172.21.15.89:0/278126929 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2000ee70 con 0x7f7c140051c0 2023-10-10T20:39:43.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.632+0000 7f7c2c896700 1 -- 172.21.15.89:0/278126929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c140051c0 msgr2=0x7f7c1400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:43.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.632+0000 7f7c2c896700 1 --2- 172.21.15.89:0/278126929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c140051c0 0x7f7c1400ba60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000e210 tx=0x7f7c2000b040 comp rx=0 tx=0).stop 2023-10-10T20:39:43.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.632+0000 7f7c2c896700 1 -- 172.21.15.89:0/278126929 shutdown_connections 2023-10-10T20:39:43.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.632+0000 7f7c2c896700 1 --2- 172.21.15.89:0/278126929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c140051c0 0x7f7c1400ba60 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.632+0000 7f7c2c896700 1 -- 172.21.15.89:0/278126929 >> 172.21.15.89:0/278126929 conn(0x7f7c28068e20 msgr2=0x7f7c280d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:43.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.632+0000 7f7c2c896700 1 -- 172.21.15.89:0/278126929 shutdown_connections 2023-10-10T20:39:43.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.632+0000 7f7c2c896700 1 -- 172.21.15.89:0/278126929 wait complete. 2023-10-10T20:39:43.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.633+0000 7f7c2c896700 1 Processor -- start 2023-10-10T20:39:43.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.633+0000 7f7c2c896700 1 -- start start 2023-10-10T20:39:43.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.634+0000 7f7c2c896700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7c140051c0 0x7f7c2816e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:43.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.634+0000 7f7c2c896700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c280cd0c0 0x7f7c2815eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:43.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.634+0000 7f7c2c896700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c280c6210 0x7f7c2815f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:43.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.634+0000 7f7c2c896700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c280cd8a0 con 0x7f7c280cd0c0 2023-10-10T20:39:43.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.634+0000 7f7c2c896700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c280c7470 con 0x7f7c280c6210 2023-10-10T20:39:43.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.634+0000 7f7c2c896700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c280c5d80 con 0x7f7c140051c0 2023-10-10T20:39:43.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.634+0000 7f7c1dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c280cd0c0 0x7f7c2815eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:43.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.634+0000 7f7c1dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c280cd0c0 0x7f7c2815eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41914/0 (socket says 172.21.15.89:41914) 2023-10-10T20:39:43.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.634+0000 7f7c1dd9b700 1 -- 172.21.15.89:0/568069988 learned_addr learned my addr 172.21.15.89:0/568069988 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:43.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.634+0000 7f7c26d9d700 1 --2- 172.21.15.89:0/568069988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c280c6210 0x7f7c2815f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:43.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.634+0000 7f7c2659c700 1 --2- 172.21.15.89:0/568069988 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7c140051c0 0x7f7c2816e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:43.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.634+0000 7f7c1dd9b700 1 -- 172.21.15.89:0/568069988 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7c140051c0 msgr2=0x7f7c2816e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:43.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.634+0000 7f7c1dd9b700 1 --2- 172.21.15.89:0/568069988 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7c140051c0 0x7f7c2816e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.634+0000 7f7c1dd9b700 1 -- 172.21.15.89:0/568069988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c280c6210 msgr2=0x7f7c2815f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:43.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.634+0000 7f7c1dd9b700 1 --2- 172.21.15.89:0/568069988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c280c6210 0x7f7c2815f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.634+0000 7f7c1dd9b700 1 -- 172.21.15.89:0/568069988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2816e6f0 con 0x7f7c280cd0c0 2023-10-10T20:39:43.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.634+0000 7f7c2659c700 1 --2- 172.21.15.89:0/568069988 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7c140051c0 0x7f7c2816e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:43.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.634+0000 7f7c1dd9b700 1 --2- 172.21.15.89:0/568069988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c280cd0c0 0x7f7c2815eee0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7c10007390 tx=0x7f7c10001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:43.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.635+0000 7f7c1effd700 1 -- 172.21.15.89:0/568069988 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c10018030 con 0x7f7c280cd0c0 2023-10-10T20:39:43.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.635+0000 7f7c1effd700 1 -- 172.21.15.89:0/568069988 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1000f8d0 con 0x7f7c280cd0c0 2023-10-10T20:39:43.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.635+0000 7f7c2c896700 1 -- 172.21.15.89:0/568069988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c280cd5c0 con 0x7f7c280cd0c0 2023-10-10T20:39:43.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.635+0000 7f7c1effd700 1 -- 172.21.15.89:0/568069988 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1000fcb0 con 0x7f7c280cd0c0 2023-10-10T20:39:43.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.635+0000 7f7c2c896700 1 -- 172.21.15.89:0/568069988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c280c5a10 con 0x7f7c280cd0c0 2023-10-10T20:39:43.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.636+0000 7f7c2c896700 1 -- 172.21.15.89:0/568069988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c280c5590 con 0x7f7c280cd0c0 2023-10-10T20:39:43.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.636+0000 7f7c1effd700 1 -- 172.21.15.89:0/568069988 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f7c1001b070 con 0x7f7c280cd0c0 2023-10-10T20:39:43.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.636+0000 7f7c1effd700 1 --2- 172.21.15.89:0/568069988 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7c08067170 0x7f7c08069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:43.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.636+0000 7f7c1effd700 1 -- 172.21.15.89:0/568069988 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7c10019070 con 0x7f7c280cd0c0 2023-10-10T20:39:43.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.636+0000 7f7c2659c700 1 --2- 172.21.15.89:0/568069988 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7c08067170 0x7f7c08069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:43.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.637+0000 7f7c2659c700 1 --2- 172.21.15.89:0/568069988 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7c08067170 0x7f7c08069620 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7c18000f60 tx=0x7f7c18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:43.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.640+0000 7f7c1effd700 1 -- 172.21.15.89:0/568069988 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c280c5590 con 0x7f7c280cd0c0 2023-10-10T20:39:43.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.819+0000 7f7c2c896700 1 -- 172.21.15.89:0/568069988 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7c280c5590 con 0x7f7c08067170 2023-10-10T20:39:43.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.827+0000 7f7c1effd700 1 -- 172.21.15.89:0/568069988 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286833 (secure 0 0 0) 0x7f7c280c5590 con 0x7f7c08067170 2023-10-10T20:39:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:43.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.830+0000 7f7c2c896700 1 -- 172.21.15.89:0/568069988 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7c08067170 msgr2=0x7f7c08069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:43.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.830+0000 7f7c2c896700 1 --2- 172.21.15.89:0/568069988 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7c08067170 0x7f7c08069620 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7c18000f60 tx=0x7f7c18009040 comp rx=0 tx=0).stop 2023-10-10T20:39:43.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.830+0000 7f7c2c896700 1 -- 172.21.15.89:0/568069988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c280cd0c0 msgr2=0x7f7c2815eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:43.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.830+0000 7f7c2c896700 1 --2- 172.21.15.89:0/568069988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c280cd0c0 0x7f7c2815eee0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7c10007390 tx=0x7f7c10001e50 comp rx=0 tx=0).stop 2023-10-10T20:39:43.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.830+0000 7f7c2c896700 1 -- 172.21.15.89:0/568069988 shutdown_connections 2023-10-10T20:39:43.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.830+0000 7f7c2c896700 1 --2- 172.21.15.89:0/568069988 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7c08067170 0x7f7c08069620 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.830+0000 7f7c2c896700 1 --2- 172.21.15.89:0/568069988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c280c6210 0x7f7c2815f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.830+0000 7f7c2c896700 1 --2- 172.21.15.89:0/568069988 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7c140051c0 0x7f7c2816e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.830+0000 7f7c2c896700 1 --2- 172.21.15.89:0/568069988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c280cd0c0 0x7f7c2815eee0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:43.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.830+0000 7f7c2c896700 1 -- 172.21.15.89:0/568069988 >> 172.21.15.89:0/568069988 conn(0x7f7c28068e20 msgr2=0x7f7c28150820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:43.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.830+0000 7f7c2c896700 1 -- 172.21.15.89:0/568069988 shutdown_connections 2023-10-10T20:39:43.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:43.830+0000 7f7c2c896700 1 -- 172.21.15.89:0/568069988 wait complete. 2023-10-10T20:39:43.838 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:39:43.844 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2023-10-10T20:39:42.714005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2390648,"kb_used_data":8248,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752584072,"statfs":{"total":773094113280,"available":770646089728,"internally_reserved":0,"allocated":8445952,"data_stored":4276440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001227"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.262078+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:18:44.235474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261385+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:32:45.306980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253087+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:55:15.047824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258454+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:10:36.151908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254702+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:17.810574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258770+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:49:29.215389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257832+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:12:11.985278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257660+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258331+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:55:15.047824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261413+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261949+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:51:25.472971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259733+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:51.489549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253308+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:49:29.215389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.271693+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:27.148718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257856+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:56:02.724230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252996+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:22.796992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252806+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:21:45.349150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257789+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:56:39.215508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260126+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:09.824624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257801+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:55:16.441311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258944+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:45:29.295885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.266901+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:22.796992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258929+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:51:00.261125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.271424+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:13:05.408163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261055+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:10:42.178506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253538+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:45.015636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254993+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.261378+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.263848+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:24.508374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.266498+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:32.888171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254896+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258291+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:44.990290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.255157+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.275787+0000","last_change":"2023-10-10T20:39:37.275787+0000","last_active":"2023-10-10T20:39:37.275787+0000","last_peered":"2023-10-10T20:39:37.275787+0000","last_clean":"2023-10-10T20:39:37.275787+0000","last_became_active":"2023-10-10T20:39:37.275528+0000","last_became_peered":"2023-10-10T20:39:37.275528+0000","last_unstale":"2023-10-10T20:39:37.275787+0000","last_undegraded":"2023-10-10T20:39:37.275787+0000","last_fullsized":"2023-10-10T20:39:37.275787+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:23.501900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260922+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.262406+0000","last_change":"2023-10-10T20:39:37.262406+0000","last_active":"2023-10-10T20:39:37.262406+0000","last_peered":"2023-10-10T20:39:37.262406+0000","last_clean":"2023-10-10T20:39:37.262406+0000","last_became_active":"2023-10-10T20:39:37.261561+0000","last_became_peered":"2023-10-10T20:39:37.261561+0000","last_unstale":"2023-10-10T20:39:37.262406+0000","last_undegraded":"2023-10-10T20:39:37.262406+0000","last_fullsized":"2023-10-10T20:39:37.262406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:19:53.834466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260876+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:17.810574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.276745+0000","last_change":"2023-10-10T20:39:37.276745+0000","last_active":"2023-10-10T20:39:37.276745+0000","last_peered":"2023-10-10T20:39:37.276745+0000","last_clean":"2023-10-10T20:39:37.276745+0000","last_became_active":"2023-10-10T20:39:37.276469+0000","last_became_peered":"2023-10-10T20:39:37.276469+0000","last_unstale":"2023-10-10T20:39:37.276745+0000","last_undegraded":"2023-10-10T20:39:37.276745+0000","last_fullsized":"2023-10-10T20:39:37.276745+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:50.483075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252593+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:18:44.235474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.275690+0000","last_change":"2023-10-10T20:39:37.275690+0000","last_active":"2023-10-10T20:39:37.275690+0000","last_peered":"2023-10-10T20:39:37.275690+0000","last_clean":"2023-10-10T20:39:37.275690+0000","last_became_active":"2023-10-10T20:39:37.275412+0000","last_became_peered":"2023-10-10T20:39:37.275412+0000","last_unstale":"2023-10-10T20:39:37.275690+0000","last_undegraded":"2023-10-10T20:39:37.275690+0000","last_fullsized":"2023-10-10T20:39:37.275690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:50:59.254651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.266049+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:16.564322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.267783+0000","last_change":"2023-10-10T20:39:37.267783+0000","last_active":"2023-10-10T20:39:37.267783+0000","last_peered":"2023-10-10T20:39:37.267783+0000","last_clean":"2023-10-10T20:39:37.267783+0000","last_became_active":"2023-10-10T20:39:37.267436+0000","last_became_peered":"2023-10-10T20:39:37.267436+0000","last_unstale":"2023-10-10T20:39:37.267783+0000","last_undegraded":"2023-10-10T20:39:37.267783+0000","last_fullsized":"2023-10-10T20:39:37.267783+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:42.059614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254194+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:04.993021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.267358+0000","last_change":"2023-10-10T20:39:37.267358+0000","last_active":"2023-10-10T20:39:37.267358+0000","last_peered":"2023-10-10T20:39:37.267358+0000","last_clean":"2023-10-10T20:39:37.267358+0000","last_became_active":"2023-10-10T20:39:37.267054+0000","last_became_peered":"2023-10-10T20:39:37.267054+0000","last_unstale":"2023-10-10T20:39:37.267358+0000","last_undegraded":"2023-10-10T20:39:37.267358+0000","last_fullsized":"2023-10-10T20:39:37.267358+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:31.881697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253409+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:49.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.265548+0000","last_change":"2023-10-10T20:39:37.265548+0000","last_active":"2023-10-10T20:39:37.265548+0000","last_peered":"2023-10-10T20:39:37.265548+0000","last_clean":"2023-10-10T20:39:37.265548+0000","last_became_active":"2023-10-10T20:39:37.265284+0000","last_became_peered":"2023-10-10T20:39:37.265284+0000","last_unstale":"2023-10-10T20:39:37.265548+0000","last_undegraded":"2023-10-10T20:39:37.265548+0000","last_fullsized":"2023-10-10T20:39:37.265548+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:53:22.002691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257399+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:02:13.214283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.270081+0000","last_change":"2023-10-10T20:39:37.270081+0000","last_active":"2023-10-10T20:39:37.270081+0000","last_peered":"2023-10-10T20:39:37.270081+0000","last_clean":"2023-10-10T20:39:37.270081+0000","last_became_active":"2023-10-10T20:39:37.269828+0000","last_became_peered":"2023-10-10T20:39:37.269828+0000","last_unstale":"2023-10-10T20:39:37.270081+0000","last_undegraded":"2023-10-10T20:39:37.270081+0000","last_fullsized":"2023-10-10T20:39:37.270081+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:16:06.819183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257391+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:41:32.637048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264681+0000","last_change":"2023-10-10T20:39:37.264681+0000","last_active":"2023-10-10T20:39:37.264681+0000","last_peered":"2023-10-10T20:39:37.264681+0000","last_clean":"2023-10-10T20:39:37.264681+0000","last_became_active":"2023-10-10T20:39:37.264173+0000","last_became_peered":"2023-10-10T20:39:37.264173+0000","last_unstale":"2023-10-10T20:39:37.264681+0000","last_undegraded":"2023-10-10T20:39:37.264681+0000","last_fullsized":"2023-10-10T20:39:37.264681+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:31.881697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257813+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:09.824624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.270474+0000","last_change":"2023-10-10T20:39:37.270474+0000","last_active":"2023-10-10T20:39:37.270474+0000","last_peered":"2023-10-10T20:39:37.270474+0000","last_clean":"2023-10-10T20:39:37.270474+0000","last_became_active":"2023-10-10T20:39:37.270158+0000","last_became_peered":"2023-10-10T20:39:37.270158+0000","last_unstale":"2023-10-10T20:39:37.270474+0000","last_undegraded":"2023-10-10T20:39:37.270474+0000","last_fullsized":"2023-10-10T20:39:37.270474+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:19:53.834466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259123+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272341+0000","last_change":"2023-10-10T20:39:37.272341+0000","last_active":"2023-10-10T20:39:37.272341+0000","last_peered":"2023-10-10T20:39:37.272341+0000","last_clean":"2023-10-10T20:39:37.272341+0000","last_became_active":"2023-10-10T20:39:37.272070+0000","last_became_peered":"2023-10-10T20:39:37.272070+0000","last_unstale":"2023-10-10T20:39:37.272341+0000","last_undegraded":"2023-10-10T20:39:37.272341+0000","last_fullsized":"2023-10-10T20:39:37.272341+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:42.059614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254514+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:26.586272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264016+0000","last_change":"2023-10-10T20:39:37.264016+0000","last_active":"2023-10-10T20:39:37.264016+0000","last_peered":"2023-10-10T20:39:37.264016+0000","last_clean":"2023-10-10T20:39:37.264016+0000","last_became_active":"2023-10-10T20:39:37.263724+0000","last_became_peered":"2023-10-10T20:39:37.263724+0000","last_unstale":"2023-10-10T20:39:37.264016+0000","last_undegraded":"2023-10-10T20:39:37.264016+0000","last_fullsized":"2023-10-10T20:39:37.264016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:25.579798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258964+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272988+0000","last_change":"2023-10-10T20:39:37.272988+0000","last_active":"2023-10-10T20:39:37.272988+0000","last_peered":"2023-10-10T20:39:37.272988+0000","last_clean":"2023-10-10T20:39:37.272988+0000","last_became_active":"2023-10-10T20:39:37.272715+0000","last_became_peered":"2023-10-10T20:39:37.272715+0000","last_unstale":"2023-10-10T20:39:37.272988+0000","last_undegraded":"2023-10-10T20:39:37.272988+0000","last_fullsized":"2023-10-10T20:39:37.272988+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:10:21.213337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252142+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:11.320173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.268230+0000","last_change":"2023-10-10T20:39:37.268230+0000","last_active":"2023-10-10T20:39:37.268230+0000","last_peered":"2023-10-10T20:39:37.268230+0000","last_clean":"2023-10-10T20:39:37.268230+0000","last_became_active":"2023-10-10T20:39:37.267986+0000","last_became_peered":"2023-10-10T20:39:37.267986+0000","last_unstale":"2023-10-10T20:39:37.268230+0000","last_undegraded":"2023-10-10T20:39:37.268230+0000","last_fullsized":"2023-10-10T20:39:37.268230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:13:05.731066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.264627+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:24:00.218312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.269974+0000","last_change":"2023-10-10T20:39:37.269974+0000","last_active":"2023-10-10T20:39:37.269974+0000","last_peered":"2023-10-10T20:39:37.269974+0000","last_clean":"2023-10-10T20:39:37.269974+0000","last_became_active":"2023-10-10T20:39:37.269650+0000","last_became_peered":"2023-10-10T20:39:37.269650+0000","last_unstale":"2023-10-10T20:39:37.269974+0000","last_undegraded":"2023-10-10T20:39:37.269974+0000","last_fullsized":"2023-10-10T20:39:37.269974+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:16.804099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260831+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:55:56.167036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.266948+0000","last_change":"2023-10-10T20:39:37.266948+0000","last_active":"2023-10-10T20:39:37.266948+0000","last_peered":"2023-10-10T20:39:37.266948+0000","last_clean":"2023-10-10T20:39:37.266948+0000","last_became_active":"2023-10-10T20:39:37.266149+0000","last_became_peered":"2023-10-10T20:39:37.266149+0000","last_unstale":"2023-10-10T20:39:37.266948+0000","last_undegraded":"2023-10-10T20:39:37.266948+0000","last_fullsized":"2023-10-10T20:39:37.266948+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:25.579798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252135+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:34:31.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.265234+0000","last_change":"2023-10-10T20:39:37.265234+0000","last_active":"2023-10-10T20:39:37.265234+0000","last_peered":"2023-10-10T20:39:37.265234+0000","last_clean":"2023-10-10T20:39:37.265234+0000","last_became_active":"2023-10-10T20:39:37.264953+0000","last_became_peered":"2023-10-10T20:39:37.264953+0000","last_unstale":"2023-10-10T20:39:37.265234+0000","last_undegraded":"2023-10-10T20:39:37.265234+0000","last_fullsized":"2023-10-10T20:39:37.265234+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:53:22.002691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260398+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:26.586272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.269676+0000","last_change":"2023-10-10T20:39:37.269676+0000","last_active":"2023-10-10T20:39:37.269676+0000","last_peered":"2023-10-10T20:39:37.269676+0000","last_clean":"2023-10-10T20:39:37.269676+0000","last_became_active":"2023-10-10T20:39:37.269437+0000","last_became_peered":"2023-10-10T20:39:37.269437+0000","last_unstale":"2023-10-10T20:39:37.269676+0000","last_undegraded":"2023-10-10T20:39:37.269676+0000","last_fullsized":"2023-10-10T20:39:37.269676+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:43.983815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253984+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:51:00.261125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264681+0000","last_change":"2023-10-10T20:39:37.264681+0000","last_active":"2023-10-10T20:39:37.264681+0000","last_peered":"2023-10-10T20:39:37.264681+0000","last_clean":"2023-10-10T20:39:37.264681+0000","last_became_active":"2023-10-10T20:39:37.264410+0000","last_became_peered":"2023-10-10T20:39:37.264410+0000","last_unstale":"2023-10-10T20:39:37.264681+0000","last_undegraded":"2023-10-10T20:39:37.264681+0000","last_fullsized":"2023-10-10T20:39:37.264681+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:21.790518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259115+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:02:53.563766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272951+0000","last_change":"2023-10-10T20:39:37.272951+0000","last_active":"2023-10-10T20:39:37.272951+0000","last_peered":"2023-10-10T20:39:37.272951+0000","last_clean":"2023-10-10T20:39:37.272951+0000","last_became_active":"2023-10-10T20:39:37.272680+0000","last_became_peered":"2023-10-10T20:39:37.272680+0000","last_unstale":"2023-10-10T20:39:37.272951+0000","last_undegraded":"2023-10-10T20:39:37.272951+0000","last_fullsized":"2023-10-10T20:39:37.272951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:09:37.235149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.257781+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272869+0000","last_change":"2023-10-10T20:39:37.272869+0000","last_active":"2023-10-10T20:39:37.272869+0000","last_peered":"2023-10-10T20:39:37.272869+0000","last_clean":"2023-10-10T20:39:37.272869+0000","last_became_active":"2023-10-10T20:39:37.272418+0000","last_became_peered":"2023-10-10T20:39:37.272418+0000","last_unstale":"2023-10-10T20:39:37.272869+0000","last_undegraded":"2023-10-10T20:39:37.272869+0000","last_fullsized":"2023-10-10T20:39:37.272869+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:20:16.442207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259051+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:57.313252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272054+0000","last_change":"2023-10-10T20:39:37.272054+0000","last_active":"2023-10-10T20:39:37.272054+0000","last_peered":"2023-10-10T20:39:37.272054+0000","last_clean":"2023-10-10T20:39:37.272054+0000","last_became_active":"2023-10-10T20:39:37.271818+0000","last_became_peered":"2023-10-10T20:39:37.271818+0000","last_unstale":"2023-10-10T20:39:37.272054+0000","last_undegraded":"2023-10-10T20:39:37.272054+0000","last_fullsized":"2023-10-10T20:39:37.272054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:57:37.129049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258156+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:18:44.235474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.275925+0000","last_change":"2023-10-10T20:39:37.275925+0000","last_active":"2023-10-10T20:39:37.275925+0000","last_peered":"2023-10-10T20:39:37.275925+0000","last_clean":"2023-10-10T20:39:37.275925+0000","last_became_active":"2023-10-10T20:39:37.275666+0000","last_became_peered":"2023-10-10T20:39:37.275666+0000","last_unstale":"2023-10-10T20:39:37.275925+0000","last_undegraded":"2023-10-10T20:39:37.275925+0000","last_fullsized":"2023-10-10T20:39:37.275925+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:01:58.347464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.254173+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:51.489549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.265642+0000","last_change":"2023-10-10T20:39:37.265642+0000","last_active":"2023-10-10T20:39:37.265642+0000","last_peered":"2023-10-10T20:39:37.265642+0000","last_clean":"2023-10-10T20:39:37.265642+0000","last_became_active":"2023-10-10T20:39:37.265399+0000","last_became_peered":"2023-10-10T20:39:37.265399+0000","last_unstale":"2023-10-10T20:39:37.265642+0000","last_undegraded":"2023-10-10T20:39:37.265642+0000","last_fullsized":"2023-10-10T20:39:37.265642+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:53:41.456643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253320+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:09.824624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.272730+0000","last_change":"2023-10-10T20:39:37.272730+0000","last_active":"2023-10-10T20:39:37.272730+0000","last_peered":"2023-10-10T20:39:37.272730+0000","last_clean":"2023-10-10T20:39:37.272730+0000","last_became_active":"2023-10-10T20:39:37.272487+0000","last_became_peered":"2023-10-10T20:39:37.272487+0000","last_unstale":"2023-10-10T20:39:37.272730+0000","last_undegraded":"2023-10-10T20:39:37.272730+0000","last_fullsized":"2023-10-10T20:39:37.272730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:40.276426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.260216+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:04.993021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.275883+0000","last_change":"2023-10-10T20:39:37.275883+0000","last_active":"2023-10-10T20:39:37.275883+0000","last_peered":"2023-10-10T20:39:37.275883+0000","last_clean":"2023-10-10T20:39:37.275883+0000","last_became_active":"2023-10-10T20:39:37.275635+0000","last_became_peered":"2023-10-10T20:39:37.275635+0000","last_unstale":"2023-10-10T20:39:37.275883+0000","last_undegraded":"2023-10-10T20:39:37.275883+0000","last_fullsized":"2023-10-10T20:39:37.275883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:23:59.211838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.253448+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:44.990290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264870+0000","last_change":"2023-10-10T20:39:37.264870+0000","last_active":"2023-10-10T20:39:37.264870+0000","last_peered":"2023-10-10T20:39:37.264870+0000","last_clean":"2023-10-10T20:39:37.264870+0000","last_became_active":"2023-10-10T20:39:37.264606+0000","last_became_peered":"2023-10-10T20:39:37.264606+0000","last_unstale":"2023-10-10T20:39:37.264870+0000","last_undegraded":"2023-10-10T20:39:37.264870+0000","last_fullsized":"2023-10-10T20:39:37.264870+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:02.132949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258303+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:45.015636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.267077+0000","last_change":"2023-10-10T20:39:37.267077+0000","last_active":"2023-10-10T20:39:37.267077+0000","last_peered":"2023-10-10T20:39:37.267077+0000","last_clean":"2023-10-10T20:39:37.267077+0000","last_became_active":"2023-10-10T20:39:37.266325+0000","last_became_peered":"2023-10-10T20:39:37.266325+0000","last_unstale":"2023-10-10T20:39:37.267077+0000","last_undegraded":"2023-10-10T20:39:37.267077+0000","last_fullsized":"2023-10-10T20:39:37.267077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:03.986546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.259437+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.262060+0000","last_change":"2023-10-10T20:39:37.262060+0000","last_active":"2023-10-10T20:39:37.262060+0000","last_peered":"2023-10-10T20:39:37.262060+0000","last_clean":"2023-10-10T20:39:37.262060+0000","last_became_active":"2023-10-10T20:39:37.261636+0000","last_became_peered":"2023-10-10T20:39:37.261636+0000","last_unstale":"2023-10-10T20:39:37.262060+0000","last_undegraded":"2023-10-10T20:39:37.262060+0000","last_fullsized":"2023-10-10T20:39:37.262060+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:15.557848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.252172+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:14.429896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264795+0000","last_change":"2023-10-10T20:39:37.264795+0000","last_active":"2023-10-10T20:39:37.264795+0000","last_peered":"2023-10-10T20:39:37.264795+0000","last_clean":"2023-10-10T20:39:37.264795+0000","last_became_active":"2023-10-10T20:39:37.263322+0000","last_became_peered":"2023-10-10T20:39:37.263322+0000","last_unstale":"2023-10-10T20:39:37.264795+0000","last_undegraded":"2023-10-10T20:39:37.264795+0000","last_fullsized":"2023-10-10T20:39:37.264795+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:15.557848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.258150+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:22.796992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.263674+0000","last_change":"2023-10-10T20:39:37.263674+0000","last_active":"2023-10-10T20:39:37.263674+0000","last_peered":"2023-10-10T20:39:37.263674+0000","last_clean":"2023-10-10T20:39:37.263674+0000","last_became_active":"2023-10-10T20:39:37.263294+0000","last_became_peered":"2023-10-10T20:39:37.263294+0000","last_unstale":"2023-10-10T20:39:37.263674+0000","last_undegraded":"2023-10-10T20:39:37.263674+0000","last_fullsized":"2023-10-10T20:39:37.263674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:02.132949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:39:37.239917+0000","last_change":"2023-10-10T20:39:37.265134+0000","last_active":"2023-10-10T20:39:37.239917+0000","last_peered":"2023-10-10T20:39:37.239917+0000","last_clean":"2023-10-10T20:39:37.239917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:39:37.239917+0000","last_undegraded":"2023-10-10T20:39:37.239917+0000","last_fullsized":"2023-10-10T20:39:37.239917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:43.066089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-10T20:39:37.264553+0000","last_change":"2023-10-10T20:39:37.264553+0000","last_active":"2023-10-10T20:39:37.264553+0000","last_peered":"2023-10-10T20:39:37.264553+0000","last_clean":"2023-10-10T20:39:37.264553+0000","last_became_active":"2023-10-10T20:39:37.264274+0000","last_became_peered":"2023-10-10T20:39:37.264274+0000","last_unstale":"2023-10-10T20:39:37.264553+0000","last_undegraded":"2023-10-10T20:39:37.264553+0000","last_fullsized":"2023-10-10T20:39:37.264553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:25.579798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":217,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-10T20:39:36.243840+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:39:36.243840+0000","last_peered":"2023-10-10T20:39:36.243840+0000","last_clean":"2023-10-10T20:39:36.243840+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:39:36.243840+0000","last_undegraded":"2023-10-10T20:39:36.243840+0000","last_fullsized":"2023-10-10T20:39:36.243840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298756,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073084,"statfs":{"total":96636764160,"available":96330838016,"internally_reserved":0,"allocated":1503232,"data_stored":978234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.69699999999999995,"15min":0.88100000000000001},"min":{"1min":0.32500000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.68500000000000005,"5min":6.8630000000000004,"15min":6.8630000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.59899999999999998,"15min":0.65000000000000002},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.752,"5min":4.1959999999999997,"15min":4.1959999999999997},"last":0.41899999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.79800000000000004,"15min":1.0980000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.70099999999999996,"5min":10.1,"15min":10.1},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.66700000000000004,"15min":0.76700000000000002},"min":{"1min":0.42999999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71199999999999997,"5min":6.806,"15min":6.806},"last":0.498}]},{"osd":2,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.70699999999999996,"15min":0.82399999999999995},"min":{"1min":0.42899999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70899999999999996,"5min":6.9340000000000002,"15min":6.9340000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.503,"5min":0.65400000000000003,"15min":0.77800000000000002},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69599999999999995,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.436}]},{"osd":3,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.67700000000000005,"15min":0.79700000000000004},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.72199999999999998,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63400000000000001,"15min":0.69199999999999995},"min":{"1min":0.42799999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76200000000000001,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.53900000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.314,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.65900000000000003,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.32900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69199999999999995,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.52900000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.376,"5min":0.314,"15min":0.314},"max":{"1min":0.78900000000000003,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53200000000000003,"15min":0.52500000000000002},"min":{"1min":0.42899999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68000000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.51600000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73799999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.51300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73699999999999999,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.46800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.59399999999999997,"15min":0.70099999999999996},"min":{"1min":0.36499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.72899999999999998,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.629,"15min":0.749},"min":{"1min":0.35299999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66000000000000003,"5min":7.8090000000000002,"15min":7.8090000000000002},"last":0.45100000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.628,"15min":0.746},"min":{"1min":0.36899999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":7.867,"15min":7.867},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.73999999999999999,"15min":0.95399999999999996},"min":{"1min":0.36299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70199999999999996,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.57799999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.55600000000000005,"15min":0.60999999999999999},"min":{"1min":0.371,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.65900000000000003,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.71499999999999997,"15min":0.90100000000000002},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.80000000000000004,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":0.53200000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.58599999999999997,"15min":0.63400000000000001},"min":{"1min":0.38600000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.65800000000000003,"5min":4.8620000000000001,"15min":4.8620000000000001},"last":0.501},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.64700000000000002,"15min":0.75},"min":{"1min":0.439,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68899999999999995,"5min":7.9009999999999998,"15min":7.9009999999999998},"last":0.55300000000000005}]},{"osd":4,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48599999999999999,"15min":0.45800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68400000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.438,"5min":0.37,"15min":0.37},"max":{"1min":0.73999999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.62}]},{"osd":5,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.5,"15min":0.48199999999999998},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.68899999999999995,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71199999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.48599999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52500000000000002,"15min":0.51100000000000001},"min":{"1min":0.377,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73299999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.38600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.745,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.59299999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.40600000000000003,"15min":0.505},"min":{"1min":0.19600000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.40200000000000002,"5min":5.0380000000000003,"15min":5.0380000000000003},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.34000000000000002,"15min":0.36799999999999999},"min":{"1min":0.224,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45200000000000001,"5min":1.194,"15min":1.194},"last":0.35899999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.34100000000000003,"15min":0.36899999999999999},"min":{"1min":0.215,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.42599999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.307,"5min":0.42399999999999999,"15min":0.53400000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45900000000000002,"5min":5.0919999999999996,"15min":5.0919999999999996},"last":0.5}]},{"osd":3,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.45200000000000001,"15min":0.55800000000000005},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.442,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.376},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.38900000000000001,"15min":0.41499999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.48999999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.44500000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57599999999999996,"15min":0.56499999999999995},"min":{"1min":0.38800000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73899999999999999,"5min":1.254,"15min":1.254},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.64500000000000002,"15min":0.73199999999999998},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82699999999999996,"5min":4.9850000000000003,"15min":4.9850000000000003},"last":0.63}]},{"osd":5,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.58199999999999996,"15min":0.59599999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72699999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59999999999999998,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83599999999999997,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.73999999999999999,"15min":0.88800000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89100000000000001,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.41799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89800000000000002,"5min":1.171,"15min":1.171},"last":0.70199999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.93300000000000005,"5min":1.214,"15min":1.214},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.67500000000000004,"15min":0.75},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.748,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.73199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302848,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068992,"statfs":{"total":96636764160,"available":96326647808,"internally_reserved":0,"allocated":1499136,"data_stored":977273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.44,"15min":0.57199999999999995},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.50900000000000001,"5min":7.9249999999999998,"15min":7.9249999999999998},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.42899999999999999,"15min":0.54200000000000004},"min":{"1min":0.22500000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.47199999999999998,"5min":6.6299999999999999,"15min":6.6299999999999999},"last":0.31900000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.45900000000000002,"15min":0.59699999999999998},"min":{"1min":0.23999999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.55400000000000005,"5min":8.2509999999999994,"15min":8.2509999999999994},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.45300000000000001,"15min":0.56599999999999995},"min":{"1min":0.223,"5min":0.222,"15min":0.222},"max":{"1min":0.54100000000000004,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.35299999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.46700000000000003,"15min":0.59799999999999998},"min":{"1min":0.23100000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54100000000000004,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.51800000000000002,"15min":0.65500000000000003},"min":{"1min":0.29199999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53300000000000003,"5min":7.8929999999999998,"15min":7.8929999999999998},"last":0.33300000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.65600000000000003,"15min":0.747},"min":{"1min":0.39400000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.88900000000000001,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.68700000000000006,"15min":0.81299999999999994},"min":{"1min":0.374,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78700000000000003,"5min":7.6920000000000002,"15min":7.6920000000000002},"last":0.54300000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.67400000000000004,"15min":0.79200000000000004},"min":{"1min":0.376,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90200000000000002,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.69499999999999995,"15min":0.81599999999999995},"min":{"1min":0.38100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79900000000000004,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":0.63100000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.72599999999999998,"15min":0.85899999999999999},"min":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81399999999999995,"5min":7.8360000000000003,"15min":7.8360000000000003},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70299999999999996,"15min":0.79200000000000004},"min":{"1min":0.46100000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92000000000000004,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.64500000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.68600000000000005,"15min":0.78400000000000003},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86399999999999999,"5min":6.7160000000000002,"15min":6.7160000000000002},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.70799999999999996,"15min":0.82499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93100000000000005,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.61299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.31900000000000001,"15min":0.32800000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.59699999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.33600000000000002,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45400000000000001,"5min":0.746,"15min":0.746},"last":0.38200000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.45400000000000001,"15min":0.77100000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.63,"5min":11.967000000000001,"15min":11.967000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39000000000000001,"15min":0.498},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52700000000000002,"5min":4.1100000000000003,"15min":4.1100000000000003},"last":0.34200000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.41199999999999998,"15min":0.57899999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48799999999999999,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.50900000000000001,"15min":0.82199999999999995},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65900000000000003,"5min":11.935,"15min":11.935},"last":0.45600000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.65900000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.80100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.58999999999999997},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.495}]},{"osd":6,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.33400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83599999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63700000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81599999999999995,"5min":1.052,"15min":1.052},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64500000000000002,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85999999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.62}]}]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298164,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073676,"statfs":{"total":96636764160,"available":96331444224,"internally_reserved":0,"allocated":897024,"data_stored":381603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.309,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44400000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.317},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48699999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.307}]},{"osd":1,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33600000000000002,"15min":0.33000000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.49199999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38200000000000001,"15min":0.376},"min":{"1min":0.307,"5min":0.251,"15min":0.251},"max":{"1min":0.53900000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.38200000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.34899999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35499999999999998,"15min":0.36099999999999999},"min":{"1min":0.25900000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.51600000000000001,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.32600000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.38100000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83299999999999996,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.54500000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.39400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78700000000000003,"5min":1.05,"15min":1.05},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86799999999999999,"5min":1.01,"15min":1.01},"last":0.441}]},{"osd":6,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92000000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.877,"5min":1.026,"15min":1.026},"last":0.47599999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77100000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.46600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89400000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.52500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58199999999999996,"5min":0.747,"15min":0.747},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.59799999999999998,"15min":0.79200000000000004},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69699999999999995,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.69699999999999995}]},{"osd":1,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.60299999999999998,"15min":0.79900000000000004},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60299999999999998,"5min":7.7160000000000002,"15min":7.7160000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54900000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68100000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63600000000000001,"15min":0.81799999999999995},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.48699999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.58899999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.57299999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.502,"15min":0.48399999999999999},"min":{"1min":0.34599999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.622},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.38400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77300000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.77300000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.41699999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72099999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.52600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.67400000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.58199999999999996},"min":{"1min":0.44700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":387866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.27900000000000003,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.59699999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.32000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.73299999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.70399999999999996}]},{"osd":1,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.60199999999999998,"15min":0.66600000000000004},"min":{"1min":0.38500000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.66300000000000003,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.58499999999999996,"15min":0.65100000000000002},"min":{"1min":0.33500000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65200000000000002,"5min":5.3109999999999999,"15min":5.3109999999999999},"last":0.68200000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.60799999999999998,"15min":0.67400000000000004},"min":{"1min":0.373,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72799999999999998,"5min":5.2960000000000003,"15min":5.2960000000000003},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.33900000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71999999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.68999999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.64100000000000001,"15min":0.70699999999999996},"min":{"1min":0.39900000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80000000000000004,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.63700000000000001,"15min":0.71699999999999997},"min":{"1min":0.439,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69499999999999995,"5min":5.2850000000000001,"15min":5.2850000000000001},"last":0.61799999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48799999999999999,"15min":0.499},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64200000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.35099999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.501}]},{"osd":6,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.86099999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.753,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.71299999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.36499999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84799999999999998,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82199999999999995,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.72699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:39:46.851 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:39:46.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.977+0000 7f4e013dc700 1 Processor -- start 2023-10-10T20:39:46.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.977+0000 7f4e013dc700 1 -- start start 2023-10-10T20:39:46.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.978+0000 7f4e013dc700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4dfc159890 0x7f4dfc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:46.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.978+0000 7f4e013dc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4dfc159c60 0x7f4dfc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:46.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.978+0000 7f4e013dc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dfc15db10 0x7f4dfc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:46.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.978+0000 7f4e013dc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dfc15df50 con 0x7f4dfc159c60 2023-10-10T20:39:46.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.978+0000 7f4e013dc700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dfc15a160 con 0x7f4dfc159890 2023-10-10T20:39:46.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.978+0000 7f4e013dc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc0d13a0 con 0x7f4dfc15db10 2023-10-10T20:39:46.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.978+0000 7f4dfa7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4dfc159c60 0x7f4dfc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:46.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.978+0000 7f4dfa7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4dfc159c60 0x7f4dfc15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41924/0 (socket says 172.21.15.89:41924) 2023-10-10T20:39:46.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.978+0000 7f4dfa7fc700 1 -- 172.21.15.89:0/2169899263 learned_addr learned my addr 172.21.15.89:0/2169899263 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:46.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.978+0000 7f4dfb7fe700 1 --2- 172.21.15.89:0/2169899263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dfc15db10 0x7f4dfc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:46.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.978+0000 7f4dfaffd700 1 --2- 172.21.15.89:0/2169899263 >> v2:172.21.15.111:3300/0 conn(0x7f4dfc159890 0x7f4dfc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:46.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.978+0000 7f4dfa7fc700 1 -- 172.21.15.89:0/2169899263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dfc15db10 msgr2=0x7f4dfc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:46.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.978+0000 7f4dfa7fc700 1 --2- 172.21.15.89:0/2169899263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dfc15db10 0x7f4dfc069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:46.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.978+0000 7f4dfa7fc700 1 -- 172.21.15.89:0/2169899263 >> v2:172.21.15.111:3300/0 conn(0x7f4dfc159890 msgr2=0x7f4dfc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:46.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.978+0000 7f4dfa7fc700 1 --2- 172.21.15.89:0/2169899263 >> v2:172.21.15.111:3300/0 conn(0x7f4dfc159890 0x7f4dfc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:46.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.978+0000 7f4dfa7fc700 1 -- 172.21.15.89:0/2169899263 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dfc15e140 con 0x7f4dfc159c60 2023-10-10T20:39:46.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.978+0000 7f4dfa7fc700 1 --2- 172.21.15.89:0/2169899263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dfc159c60 0x7f4dfc15d5d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0d1090 tx=0x7f4de4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ca45bdf829b23 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:46.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.978+0000 7f4df9ffb700 1 -- 172.21.15.89:0/2169899263 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de400b640 con 0x7f4dfc159c60 2023-10-10T20:39:46.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.978+0000 7f4df9ffb700 1 -- 172.21.15.89:0/2169899263 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de400b7a0 con 0x7f4dfc159c60 2023-10-10T20:39:46.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.979+0000 7f4df9ffb700 1 -- 172.21.15.89:0/2169899263 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de400bb60 con 0x7f4dfc159c60 2023-10-10T20:39:46.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.979+0000 7f4e013dc700 1 -- 172.21.15.89:0/2169899263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dfc159c60 msgr2=0x7f4dfc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:46.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.979+0000 7f4e013dc700 1 --2- 172.21.15.89:0/2169899263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dfc159c60 0x7f4dfc15d5d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0d1090 tx=0x7f4de4009250 comp rx=0 tx=0).stop 2023-10-10T20:39:46.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.979+0000 7f4e013dc700 1 -- 172.21.15.89:0/2169899263 shutdown_connections 2023-10-10T20:39:46.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.979+0000 7f4e013dc700 1 --2- 172.21.15.89:0/2169899263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dfc15db10 0x7f4dfc069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:46.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.979+0000 7f4e013dc700 1 --2- 172.21.15.89:0/2169899263 >> v2:172.21.15.111:3300/0 conn(0x7f4dfc159890 0x7f4dfc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:46.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.979+0000 7f4e013dc700 1 --2- 172.21.15.89:0/2169899263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dfc159c60 0x7f4dfc15d5d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:46.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.979+0000 7f4e013dc700 1 -- 172.21.15.89:0/2169899263 >> 172.21.15.89:0/2169899263 conn(0x7f4dfc068e20 msgr2=0x7f4dfc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:46.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.979+0000 7f4e013dc700 1 -- 172.21.15.89:0/2169899263 shutdown_connections 2023-10-10T20:39:46.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.979+0000 7f4e013dc700 1 -- 172.21.15.89:0/2169899263 wait complete. 2023-10-10T20:39:46.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.980+0000 7f4e013dc700 1 Processor -- start 2023-10-10T20:39:46.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.980+0000 7f4e013dc700 1 -- start start 2023-10-10T20:39:46.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.981+0000 7f4e013dc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dfc159890 0x7f4dfc0b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:46.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.981+0000 7f4e013dc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dfc159c60 0x7f4dfc0b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:46.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.981+0000 7f4e013dc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dfc15db10 0x7f4dfc0b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:46.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.981+0000 7f4e013dc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc0b05d0 con 0x7f4dfc15db10 2023-10-10T20:39:46.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.981+0000 7f4e013dc700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc0b46a0 con 0x7f4dfc159890 2023-10-10T20:39:46.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.981+0000 7f4e013dc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc0b4980 con 0x7f4dfc159c60 2023-10-10T20:39:46.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.981+0000 7f4dfb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dfc15db10 0x7f4dfc0b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:46.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.981+0000 7f4dfb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dfc15db10 0x7f4dfc0b5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41940/0 (socket says 172.21.15.89:41940) 2023-10-10T20:39:46.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.981+0000 7f4dfb7fe700 1 -- 172.21.15.89:0/3136706717 learned_addr learned my addr 172.21.15.89:0/3136706717 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:46.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.981+0000 7f4dfa7fc700 1 --2- 172.21.15.89:0/3136706717 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dfc159c60 0x7f4dfc0b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:46.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.981+0000 7f4dfaffd700 1 --2- 172.21.15.89:0/3136706717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dfc159890 0x7f4dfc0b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:46.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.982+0000 7f4dfb7fe700 1 -- 172.21.15.89:0/3136706717 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dfc159c60 msgr2=0x7f4dfc0b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:46.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.982+0000 7f4dfb7fe700 1 --2- 172.21.15.89:0/3136706717 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dfc159c60 0x7f4dfc0b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:46.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.982+0000 7f4dfb7fe700 1 -- 172.21.15.89:0/3136706717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dfc159890 msgr2=0x7f4dfc0b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:46.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.982+0000 7f4dfb7fe700 1 --2- 172.21.15.89:0/3136706717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dfc159890 0x7f4dfc0b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:46.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.982+0000 7f4dfb7fe700 1 -- 172.21.15.89:0/3136706717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de40094f0 con 0x7f4dfc15db10 2023-10-10T20:39:46.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.982+0000 7f4dfb7fe700 1 --2- 172.21.15.89:0/3136706717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dfc15db10 0x7f4dfc0b5550 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00ea50 tx=0x7f4dec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:46.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.982+0000 7f4df2ffd700 1 -- 172.21.15.89:0/3136706717 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec00ee90 con 0x7f4dfc15db10 2023-10-10T20:39:46.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.982+0000 7f4df2ffd700 1 -- 172.21.15.89:0/3136706717 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec00c850 con 0x7f4dfc15db10 2023-10-10T20:39:46.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.982+0000 7f4e013dc700 1 -- 172.21.15.89:0/3136706717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dfc0b07e0 con 0x7f4dfc15db10 2023-10-10T20:39:46.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.982+0000 7f4df2ffd700 1 -- 172.21.15.89:0/3136706717 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec00cc10 con 0x7f4dfc15db10 2023-10-10T20:39:46.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.982+0000 7f4e013dc700 1 -- 172.21.15.89:0/3136706717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dfc0c8560 con 0x7f4dfc15db10 2023-10-10T20:39:46.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.983+0000 7f4e013dc700 1 -- 172.21.15.89:0/3136706717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dfc0b4ba0 con 0x7f4dfc15db10 2023-10-10T20:39:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.983+0000 7f4df2ffd700 1 -- 172.21.15.89:0/3136706717 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f4dec013470 con 0x7f4dfc15db10 2023-10-10T20:39:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.984+0000 7f4df2ffd700 1 --2- 172.21.15.89:0/3136706717 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4ddc067170 0x7f4ddc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.984+0000 7f4df2ffd700 1 -- 172.21.15.89:0/3136706717 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4dec084a40 con 0x7f4dfc15db10 2023-10-10T20:39:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.984+0000 7f4dfaffd700 1 --2- 172.21.15.89:0/3136706717 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4ddc067170 0x7f4ddc069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.984+0000 7f4dfaffd700 1 --2- 172.21.15.89:0/3136706717 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4ddc067170 0x7f4ddc069620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4df4009fd0 tx=0x7f4df400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:46.988+0000 7f4df2ffd700 1 -- 172.21.15.89:0/3136706717 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4dec086e60 con 0x7f4dfc15db10 2023-10-10T20:39:47.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.168+0000 7f4e013dc700 1 -- 172.21.15.89:0/3136706717 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4dfc0b0d00 con 0x7f4ddc067170 2023-10-10T20:39:47.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.176+0000 7f4df2ffd700 1 -- 172.21.15.89:0/3136706717 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289938 (secure 0 0 0) 0x7f4dfc0b0d00 con 0x7f4ddc067170 2023-10-10T20:39:47.181 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:47.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.179+0000 7f4e013dc700 1 -- 172.21.15.89:0/3136706717 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4ddc067170 msgr2=0x7f4ddc069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:47.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.179+0000 7f4e013dc700 1 --2- 172.21.15.89:0/3136706717 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4ddc067170 0x7f4ddc069620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4df4009fd0 tx=0x7f4df400b040 comp rx=0 tx=0).stop 2023-10-10T20:39:47.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.179+0000 7f4e013dc700 1 -- 172.21.15.89:0/3136706717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dfc15db10 msgr2=0x7f4dfc0b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:47.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.179+0000 7f4e013dc700 1 --2- 172.21.15.89:0/3136706717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dfc15db10 0x7f4dfc0b5550 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00ea50 tx=0x7f4dec0091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.179+0000 7f4e013dc700 1 -- 172.21.15.89:0/3136706717 shutdown_connections 2023-10-10T20:39:47.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.179+0000 7f4e013dc700 1 --2- 172.21.15.89:0/3136706717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dfc15db10 0x7f4dfc0b5550 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.179+0000 7f4e013dc700 1 --2- 172.21.15.89:0/3136706717 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4ddc067170 0x7f4ddc069620 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.179+0000 7f4e013dc700 1 --2- 172.21.15.89:0/3136706717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dfc159890 0x7f4dfc0b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.179+0000 7f4e013dc700 1 --2- 172.21.15.89:0/3136706717 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4dfc159c60 0x7f4dfc0b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.179+0000 7f4e013dc700 1 -- 172.21.15.89:0/3136706717 >> 172.21.15.89:0/3136706717 conn(0x7f4dfc068e20 msgr2=0x7f4dfc0cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:47.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.179+0000 7f4e013dc700 1 -- 172.21.15.89:0/3136706717 shutdown_connections 2023-10-10T20:39:47.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.179+0000 7f4e013dc700 1 -- 172.21.15.89:0/3136706717 wait complete. 2023-10-10T20:39:47.187 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:39:47.194 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2023-10-10T20:39:46.715117+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2390916,"kb_used_data":8516,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752583804,"statfs":{"total":773094113280,"available":770645815296,"internally_reserved":0,"allocated":8720384,"data_stored":4391723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.463563"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.277693+0000","last_change":"2023-10-10T20:39:38.271237+0000","last_active":"2023-10-10T20:39:38.277693+0000","last_peered":"2023-10-10T20:39:38.277693+0000","last_clean":"2023-10-10T20:39:38.277693+0000","last_became_active":"2023-10-10T20:39:38.270985+0000","last_became_peered":"2023-10-10T20:39:38.270985+0000","last_unstale":"2023-10-10T20:39:38.277693+0000","last_undegraded":"2023-10-10T20:39:38.277693+0000","last_fullsized":"2023-10-10T20:39:38.277693+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:42.906143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.280330+0000","last_change":"2023-10-10T20:39:38.274683+0000","last_active":"2023-10-10T20:39:38.280330+0000","last_peered":"2023-10-10T20:39:38.280330+0000","last_clean":"2023-10-10T20:39:38.280330+0000","last_became_active":"2023-10-10T20:39:38.274460+0000","last_became_peered":"2023-10-10T20:39:38.274460+0000","last_unstale":"2023-10-10T20:39:38.280330+0000","last_undegraded":"2023-10-10T20:39:38.280330+0000","last_fullsized":"2023-10-10T20:39:38.280330+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:44.990290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.275476+0000","last_change":"2023-10-10T20:39:38.269953+0000","last_active":"2023-10-10T20:39:38.275476+0000","last_peered":"2023-10-10T20:39:38.275476+0000","last_clean":"2023-10-10T20:39:38.275476+0000","last_became_active":"2023-10-10T20:39:38.266498+0000","last_became_peered":"2023-10-10T20:39:38.266498+0000","last_unstale":"2023-10-10T20:39:38.275476+0000","last_undegraded":"2023-10-10T20:39:38.275476+0000","last_fullsized":"2023-10-10T20:39:38.275476+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:01:59.353938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.283873+0000","last_change":"2023-10-10T20:39:38.275556+0000","last_active":"2023-10-10T20:39:38.283873+0000","last_peered":"2023-10-10T20:39:38.283873+0000","last_clean":"2023-10-10T20:39:38.283873+0000","last_became_active":"2023-10-10T20:39:38.275297+0000","last_became_peered":"2023-10-10T20:39:38.275297+0000","last_unstale":"2023-10-10T20:39:38.283873+0000","last_undegraded":"2023-10-10T20:39:38.283873+0000","last_fullsized":"2023-10-10T20:39:38.283873+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:00:05.932494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.279337+0000","last_change":"2023-10-10T20:39:38.276381+0000","last_active":"2023-10-10T20:39:38.279337+0000","last_peered":"2023-10-10T20:39:38.279337+0000","last_clean":"2023-10-10T20:39:38.279337+0000","last_became_active":"2023-10-10T20:39:38.276130+0000","last_became_peered":"2023-10-10T20:39:38.276130+0000","last_unstale":"2023-10-10T20:39:38.279337+0000","last_undegraded":"2023-10-10T20:39:38.279337+0000","last_fullsized":"2023-10-10T20:39:38.279337+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:00:50.973187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.273882+0000","last_change":"2023-10-10T20:39:38.270898+0000","last_active":"2023-10-10T20:39:38.273882+0000","last_peered":"2023-10-10T20:39:38.273882+0000","last_clean":"2023-10-10T20:39:38.273882+0000","last_became_active":"2023-10-10T20:39:38.270622+0000","last_became_peered":"2023-10-10T20:39:38.270622+0000","last_unstale":"2023-10-10T20:39:38.273882+0000","last_undegraded":"2023-10-10T20:39:38.273882+0000","last_fullsized":"2023-10-10T20:39:38.273882+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:00.911369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.273382+0000","last_change":"2023-10-10T20:39:38.265641+0000","last_active":"2023-10-10T20:39:38.273382+0000","last_peered":"2023-10-10T20:39:38.273382+0000","last_clean":"2023-10-10T20:39:38.273382+0000","last_became_active":"2023-10-10T20:39:38.265254+0000","last_became_peered":"2023-10-10T20:39:38.265254+0000","last_unstale":"2023-10-10T20:39:38.273382+0000","last_undegraded":"2023-10-10T20:39:38.273382+0000","last_fullsized":"2023-10-10T20:39:38.273382+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:35:31.862545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.278421+0000","last_change":"2023-10-10T20:39:38.276987+0000","last_active":"2023-10-10T20:39:38.278421+0000","last_peered":"2023-10-10T20:39:38.278421+0000","last_clean":"2023-10-10T20:39:38.278421+0000","last_became_active":"2023-10-10T20:39:38.276698+0000","last_became_peered":"2023-10-10T20:39:38.276698+0000","last_unstale":"2023-10-10T20:39:38.278421+0000","last_undegraded":"2023-10-10T20:39:38.278421+0000","last_fullsized":"2023-10-10T20:39:38.278421+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:00:50.973187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.281322+0000","last_change":"2023-10-10T20:39:38.279573+0000","last_active":"2023-10-10T20:39:38.281322+0000","last_peered":"2023-10-10T20:39:38.281322+0000","last_clean":"2023-10-10T20:39:38.281322+0000","last_became_active":"2023-10-10T20:39:38.279300+0000","last_became_peered":"2023-10-10T20:39:38.279300+0000","last_unstale":"2023-10-10T20:39:38.281322+0000","last_undegraded":"2023-10-10T20:39:38.281322+0000","last_fullsized":"2023-10-10T20:39:38.281322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:05:53.997189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.263775+0000","last_change":"2023-10-10T20:39:38.260454+0000","last_active":"2023-10-10T20:39:38.263775+0000","last_peered":"2023-10-10T20:39:38.263775+0000","last_clean":"2023-10-10T20:39:38.263775+0000","last_became_active":"2023-10-10T20:39:38.260223+0000","last_became_peered":"2023-10-10T20:39:38.260223+0000","last_unstale":"2023-10-10T20:39:38.263775+0000","last_undegraded":"2023-10-10T20:39:38.263775+0000","last_fullsized":"2023-10-10T20:39:38.263775+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:55:15.047824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.280798+0000","last_change":"2023-10-10T20:39:38.277873+0000","last_active":"2023-10-10T20:39:38.280798+0000","last_peered":"2023-10-10T20:39:38.280798+0000","last_clean":"2023-10-10T20:39:38.280798+0000","last_became_active":"2023-10-10T20:39:38.277637+0000","last_became_peered":"2023-10-10T20:39:38.277637+0000","last_unstale":"2023-10-10T20:39:38.280798+0000","last_undegraded":"2023-10-10T20:39:38.280798+0000","last_fullsized":"2023-10-10T20:39:38.280798+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:27:44.154097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.265472+0000","last_change":"2023-10-10T20:39:38.260878+0000","last_active":"2023-10-10T20:39:38.265472+0000","last_peered":"2023-10-10T20:39:38.265472+0000","last_clean":"2023-10-10T20:39:38.265472+0000","last_became_active":"2023-10-10T20:39:38.260582+0000","last_became_peered":"2023-10-10T20:39:38.260582+0000","last_unstale":"2023-10-10T20:39:38.265472+0000","last_undegraded":"2023-10-10T20:39:38.265472+0000","last_fullsized":"2023-10-10T20:39:38.265472+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:45.015636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.278870+0000","last_change":"2023-10-10T20:39:38.273583+0000","last_active":"2023-10-10T20:39:38.278870+0000","last_peered":"2023-10-10T20:39:38.278870+0000","last_clean":"2023-10-10T20:39:38.278870+0000","last_became_active":"2023-10-10T20:39:38.273092+0000","last_became_peered":"2023-10-10T20:39:38.273092+0000","last_unstale":"2023-10-10T20:39:38.278870+0000","last_undegraded":"2023-10-10T20:39:38.278870+0000","last_fullsized":"2023-10-10T20:39:38.278870+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:42.906143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.284031+0000","last_change":"2023-10-10T20:39:38.277181+0000","last_active":"2023-10-10T20:39:38.284031+0000","last_peered":"2023-10-10T20:39:38.284031+0000","last_clean":"2023-10-10T20:39:38.284031+0000","last_became_active":"2023-10-10T20:39:38.276452+0000","last_became_peered":"2023-10-10T20:39:38.276452+0000","last_unstale":"2023-10-10T20:39:38.284031+0000","last_undegraded":"2023-10-10T20:39:38.284031+0000","last_fullsized":"2023-10-10T20:39:38.284031+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:03.565064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.269187+0000","last_change":"2023-10-10T20:39:38.265251+0000","last_active":"2023-10-10T20:39:38.269187+0000","last_peered":"2023-10-10T20:39:38.269187+0000","last_clean":"2023-10-10T20:39:38.269187+0000","last_became_active":"2023-10-10T20:39:38.264874+0000","last_became_peered":"2023-10-10T20:39:38.264874+0000","last_unstale":"2023-10-10T20:39:38.269187+0000","last_undegraded":"2023-10-10T20:39:38.269187+0000","last_fullsized":"2023-10-10T20:39:38.269187+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:58.771339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.273842+0000","last_change":"2023-10-10T20:39:38.269009+0000","last_active":"2023-10-10T20:39:38.273842+0000","last_peered":"2023-10-10T20:39:38.273842+0000","last_clean":"2023-10-10T20:39:38.273842+0000","last_became_active":"2023-10-10T20:39:38.267142+0000","last_became_peered":"2023-10-10T20:39:38.267142+0000","last_unstale":"2023-10-10T20:39:38.273842+0000","last_undegraded":"2023-10-10T20:39:38.273842+0000","last_fullsized":"2023-10-10T20:39:38.273842+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:49:29.215389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.278508+0000","last_change":"2023-10-10T20:39:38.269410+0000","last_active":"2023-10-10T20:39:38.278508+0000","last_peered":"2023-10-10T20:39:38.278508+0000","last_clean":"2023-10-10T20:39:38.278508+0000","last_became_active":"2023-10-10T20:39:38.269134+0000","last_became_peered":"2023-10-10T20:39:38.269134+0000","last_unstale":"2023-10-10T20:39:38.278508+0000","last_undegraded":"2023-10-10T20:39:38.278508+0000","last_fullsized":"2023-10-10T20:39:38.278508+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:18:44.235474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.284080+0000","last_change":"2023-10-10T20:39:38.279345+0000","last_active":"2023-10-10T20:39:38.284080+0000","last_peered":"2023-10-10T20:39:38.284080+0000","last_clean":"2023-10-10T20:39:38.284080+0000","last_became_active":"2023-10-10T20:39:38.279102+0000","last_became_peered":"2023-10-10T20:39:38.279102+0000","last_unstale":"2023-10-10T20:39:38.284080+0000","last_undegraded":"2023-10-10T20:39:38.284080+0000","last_fullsized":"2023-10-10T20:39:38.284080+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:53:19.449643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.261007+0000","last_change":"2023-10-10T20:39:38.260003+0000","last_active":"2023-10-10T20:39:38.261007+0000","last_peered":"2023-10-10T20:39:38.261007+0000","last_clean":"2023-10-10T20:39:38.261007+0000","last_became_active":"2023-10-10T20:39:38.259664+0000","last_became_peered":"2023-10-10T20:39:38.259664+0000","last_unstale":"2023-10-10T20:39:38.261007+0000","last_undegraded":"2023-10-10T20:39:38.261007+0000","last_fullsized":"2023-10-10T20:39:38.261007+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:45.015636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.284051+0000","last_change":"2023-10-10T20:39:38.278666+0000","last_active":"2023-10-10T20:39:38.284051+0000","last_peered":"2023-10-10T20:39:38.284051+0000","last_clean":"2023-10-10T20:39:38.284051+0000","last_became_active":"2023-10-10T20:39:38.278403+0000","last_became_peered":"2023-10-10T20:39:38.278403+0000","last_unstale":"2023-10-10T20:39:38.284051+0000","last_undegraded":"2023-10-10T20:39:38.284051+0000","last_fullsized":"2023-10-10T20:39:38.284051+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:06:57.242628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.274370+0000","last_change":"2023-10-10T20:39:38.267852+0000","last_active":"2023-10-10T20:39:38.274370+0000","last_peered":"2023-10-10T20:39:38.274370+0000","last_clean":"2023-10-10T20:39:38.274370+0000","last_became_active":"2023-10-10T20:39:38.267614+0000","last_became_peered":"2023-10-10T20:39:38.267614+0000","last_unstale":"2023-10-10T20:39:38.274370+0000","last_undegraded":"2023-10-10T20:39:38.274370+0000","last_fullsized":"2023-10-10T20:39:38.274370+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:59.221041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.283684+0000","last_change":"2023-10-10T20:39:38.279068+0000","last_active":"2023-10-10T20:39:38.283684+0000","last_peered":"2023-10-10T20:39:38.283684+0000","last_clean":"2023-10-10T20:39:38.283684+0000","last_became_active":"2023-10-10T20:39:38.278513+0000","last_became_peered":"2023-10-10T20:39:38.278513+0000","last_unstale":"2023-10-10T20:39:38.283684+0000","last_undegraded":"2023-10-10T20:39:38.283684+0000","last_fullsized":"2023-10-10T20:39:38.283684+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:51.489549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.277411+0000","last_change":"2023-10-10T20:39:38.268228+0000","last_active":"2023-10-10T20:39:38.277411+0000","last_peered":"2023-10-10T20:39:38.277411+0000","last_clean":"2023-10-10T20:39:38.277411+0000","last_became_active":"2023-10-10T20:39:38.266221+0000","last_became_peered":"2023-10-10T20:39:38.266221+0000","last_unstale":"2023-10-10T20:39:38.277411+0000","last_undegraded":"2023-10-10T20:39:38.277411+0000","last_fullsized":"2023-10-10T20:39:38.277411+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:09.824624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.267555+0000","last_change":"2023-10-10T20:39:38.263539+0000","last_active":"2023-10-10T20:39:38.267555+0000","last_peered":"2023-10-10T20:39:38.267555+0000","last_clean":"2023-10-10T20:39:38.267555+0000","last_became_active":"2023-10-10T20:39:38.262568+0000","last_became_peered":"2023-10-10T20:39:38.262568+0000","last_unstale":"2023-10-10T20:39:38.267555+0000","last_undegraded":"2023-10-10T20:39:38.267555+0000","last_fullsized":"2023-10-10T20:39:38.267555+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:41:03.560942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.273565+0000","last_change":"2023-10-10T20:39:38.267235+0000","last_active":"2023-10-10T20:39:38.273565+0000","last_peered":"2023-10-10T20:39:38.273565+0000","last_clean":"2023-10-10T20:39:38.273565+0000","last_became_active":"2023-10-10T20:39:38.266964+0000","last_became_peered":"2023-10-10T20:39:38.266964+0000","last_unstale":"2023-10-10T20:39:38.273565+0000","last_undegraded":"2023-10-10T20:39:38.273565+0000","last_fullsized":"2023-10-10T20:39:38.273565+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:21:45.349150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.279294+0000","last_change":"2023-10-10T20:39:38.275314+0000","last_active":"2023-10-10T20:39:38.279294+0000","last_peered":"2023-10-10T20:39:38.279294+0000","last_clean":"2023-10-10T20:39:38.279294+0000","last_became_active":"2023-10-10T20:39:38.275012+0000","last_became_peered":"2023-10-10T20:39:38.275012+0000","last_unstale":"2023-10-10T20:39:38.279294+0000","last_undegraded":"2023-10-10T20:39:38.279294+0000","last_fullsized":"2023-10-10T20:39:38.279294+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:00.911369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.278811+0000","last_change":"2023-10-10T20:39:38.270399+0000","last_active":"2023-10-10T20:39:38.278811+0000","last_peered":"2023-10-10T20:39:38.278811+0000","last_clean":"2023-10-10T20:39:38.278811+0000","last_became_active":"2023-10-10T20:39:38.270174+0000","last_became_peered":"2023-10-10T20:39:38.270174+0000","last_unstale":"2023-10-10T20:39:38.278811+0000","last_undegraded":"2023-10-10T20:39:38.278811+0000","last_fullsized":"2023-10-10T20:39:38.278811+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:09.824624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.277660+0000","last_change":"2023-10-10T20:39:38.270234+0000","last_active":"2023-10-10T20:39:38.277660+0000","last_peered":"2023-10-10T20:39:38.277660+0000","last_clean":"2023-10-10T20:39:38.277660+0000","last_became_active":"2023-10-10T20:39:38.269903+0000","last_became_peered":"2023-10-10T20:39:38.269903+0000","last_unstale":"2023-10-10T20:39:38.277660+0000","last_undegraded":"2023-10-10T20:39:38.277660+0000","last_fullsized":"2023-10-10T20:39:38.277660+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:05:48.142508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.280562+0000","last_change":"2023-10-10T20:39:38.275768+0000","last_active":"2023-10-10T20:39:38.280562+0000","last_peered":"2023-10-10T20:39:38.280562+0000","last_clean":"2023-10-10T20:39:38.280562+0000","last_became_active":"2023-10-10T20:39:38.274777+0000","last_became_peered":"2023-10-10T20:39:38.274777+0000","last_unstale":"2023-10-10T20:39:38.280562+0000","last_undegraded":"2023-10-10T20:39:38.280562+0000","last_fullsized":"2023-10-10T20:39:38.280562+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:19:54.840940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.280653+0000","last_change":"2023-10-10T20:39:38.274392+0000","last_active":"2023-10-10T20:39:38.280653+0000","last_peered":"2023-10-10T20:39:38.280653+0000","last_clean":"2023-10-10T20:39:38.280653+0000","last_became_active":"2023-10-10T20:39:38.273789+0000","last_became_peered":"2023-10-10T20:39:38.273789+0000","last_unstale":"2023-10-10T20:39:38.280653+0000","last_undegraded":"2023-10-10T20:39:38.280653+0000","last_fullsized":"2023-10-10T20:39:38.280653+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:26.586272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.279432+0000","last_change":"2023-10-10T20:39:38.273796+0000","last_active":"2023-10-10T20:39:38.279432+0000","last_peered":"2023-10-10T20:39:38.279432+0000","last_clean":"2023-10-10T20:39:38.279432+0000","last_became_active":"2023-10-10T20:39:38.273522+0000","last_became_peered":"2023-10-10T20:39:38.273522+0000","last_unstale":"2023-10-10T20:39:38.279432+0000","last_undegraded":"2023-10-10T20:39:38.279432+0000","last_fullsized":"2023-10-10T20:39:38.279432+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:55:15.047824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.269512+0000","last_change":"2023-10-10T20:39:38.267021+0000","last_active":"2023-10-10T20:39:38.269512+0000","last_peered":"2023-10-10T20:39:38.269512+0000","last_clean":"2023-10-10T20:39:38.269512+0000","last_became_active":"2023-10-10T20:39:38.266761+0000","last_became_peered":"2023-10-10T20:39:38.266761+0000","last_unstale":"2023-10-10T20:39:38.269512+0000","last_undegraded":"2023-10-10T20:39:38.269512+0000","last_fullsized":"2023-10-10T20:39:38.269512+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:00:50.973187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.279352+0000","last_change":"2023-10-10T20:39:38.274536+0000","last_active":"2023-10-10T20:39:38.279352+0000","last_peered":"2023-10-10T20:39:38.279352+0000","last_clean":"2023-10-10T20:39:38.279352+0000","last_became_active":"2023-10-10T20:39:38.274307+0000","last_became_peered":"2023-10-10T20:39:38.274307+0000","last_unstale":"2023-10-10T20:39:38.279352+0000","last_undegraded":"2023-10-10T20:39:38.279352+0000","last_fullsized":"2023-10-10T20:39:38.279352+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:05:53.997189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"75'4","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.281606+0000","last_change":"2023-10-10T20:39:37.275787+0000","last_active":"2023-10-10T20:39:38.281606+0000","last_peered":"2023-10-10T20:39:38.281606+0000","last_clean":"2023-10-10T20:39:38.281606+0000","last_became_active":"2023-10-10T20:39:37.275528+0000","last_became_peered":"2023-10-10T20:39:37.275528+0000","last_unstale":"2023-10-10T20:39:38.281606+0000","last_undegraded":"2023-10-10T20:39:38.281606+0000","last_fullsized":"2023-10-10T20:39:38.281606+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:23.501900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.269360+0000","last_change":"2023-10-10T20:39:38.265643+0000","last_active":"2023-10-10T20:39:38.269360+0000","last_peered":"2023-10-10T20:39:38.269360+0000","last_clean":"2023-10-10T20:39:38.269360+0000","last_became_active":"2023-10-10T20:39:38.265034+0000","last_became_peered":"2023-10-10T20:39:38.265034+0000","last_unstale":"2023-10-10T20:39:38.269360+0000","last_undegraded":"2023-10-10T20:39:38.269360+0000","last_fullsized":"2023-10-10T20:39:38.269360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:49:29.215389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"75'1","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.270065+0000","last_change":"2023-10-10T20:39:37.262406+0000","last_active":"2023-10-10T20:39:38.270065+0000","last_peered":"2023-10-10T20:39:38.270065+0000","last_clean":"2023-10-10T20:39:38.270065+0000","last_became_active":"2023-10-10T20:39:37.261561+0000","last_became_peered":"2023-10-10T20:39:37.261561+0000","last_unstale":"2023-10-10T20:39:38.270065+0000","last_undegraded":"2023-10-10T20:39:38.270065+0000","last_fullsized":"2023-10-10T20:39:38.270065+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:19:53.834466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.278617+0000","last_change":"2023-10-10T20:39:38.276732+0000","last_active":"2023-10-10T20:39:38.278617+0000","last_peered":"2023-10-10T20:39:38.278617+0000","last_clean":"2023-10-10T20:39:38.278617+0000","last_became_active":"2023-10-10T20:39:38.276426+0000","last_became_peered":"2023-10-10T20:39:38.276426+0000","last_unstale":"2023-10-10T20:39:38.278617+0000","last_undegraded":"2023-10-10T20:39:38.278617+0000","last_fullsized":"2023-10-10T20:39:38.278617+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:36.375542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"75'2","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.284118+0000","last_change":"2023-10-10T20:39:37.276745+0000","last_active":"2023-10-10T20:39:38.284118+0000","last_peered":"2023-10-10T20:39:38.284118+0000","last_clean":"2023-10-10T20:39:38.284118+0000","last_became_active":"2023-10-10T20:39:37.276469+0000","last_became_peered":"2023-10-10T20:39:37.276469+0000","last_unstale":"2023-10-10T20:39:38.284118+0000","last_undegraded":"2023-10-10T20:39:38.284118+0000","last_fullsized":"2023-10-10T20:39:38.284118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:50.483075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.275499+0000","last_change":"2023-10-10T20:39:38.271235+0000","last_active":"2023-10-10T20:39:38.275499+0000","last_peered":"2023-10-10T20:39:38.275499+0000","last_clean":"2023-10-10T20:39:38.275499+0000","last_became_active":"2023-10-10T20:39:38.270986+0000","last_became_peered":"2023-10-10T20:39:38.270986+0000","last_unstale":"2023-10-10T20:39:38.275499+0000","last_undegraded":"2023-10-10T20:39:38.275499+0000","last_fullsized":"2023-10-10T20:39:38.275499+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:05:48.142508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"75'2","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.284315+0000","last_change":"2023-10-10T20:39:37.275690+0000","last_active":"2023-10-10T20:39:38.284315+0000","last_peered":"2023-10-10T20:39:38.284315+0000","last_clean":"2023-10-10T20:39:38.284315+0000","last_became_active":"2023-10-10T20:39:37.275412+0000","last_became_peered":"2023-10-10T20:39:37.275412+0000","last_unstale":"2023-10-10T20:39:38.284315+0000","last_undegraded":"2023-10-10T20:39:38.284315+0000","last_fullsized":"2023-10-10T20:39:38.284315+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:50:59.254651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.280631+0000","last_change":"2023-10-10T20:39:38.274835+0000","last_active":"2023-10-10T20:39:38.280631+0000","last_peered":"2023-10-10T20:39:38.280631+0000","last_clean":"2023-10-10T20:39:38.280631+0000","last_became_active":"2023-10-10T20:39:38.274095+0000","last_became_peered":"2023-10-10T20:39:38.274095+0000","last_unstale":"2023-10-10T20:39:38.280631+0000","last_undegraded":"2023-10-10T20:39:38.280631+0000","last_fullsized":"2023-10-10T20:39:38.280631+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:04.993021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"75'1","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.275842+0000","last_change":"2023-10-10T20:39:37.267783+0000","last_active":"2023-10-10T20:39:38.275842+0000","last_peered":"2023-10-10T20:39:38.275842+0000","last_clean":"2023-10-10T20:39:38.275842+0000","last_became_active":"2023-10-10T20:39:37.267436+0000","last_became_peered":"2023-10-10T20:39:37.267436+0000","last_unstale":"2023-10-10T20:39:38.275842+0000","last_undegraded":"2023-10-10T20:39:38.275842+0000","last_fullsized":"2023-10-10T20:39:38.275842+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:42.059614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.278519+0000","last_change":"2023-10-10T20:39:38.269840+0000","last_active":"2023-10-10T20:39:38.278519+0000","last_peered":"2023-10-10T20:39:38.278519+0000","last_clean":"2023-10-10T20:39:38.278519+0000","last_became_active":"2023-10-10T20:39:38.269121+0000","last_became_peered":"2023-10-10T20:39:38.269121+0000","last_unstale":"2023-10-10T20:39:38.278519+0000","last_undegraded":"2023-10-10T20:39:38.278519+0000","last_fullsized":"2023-10-10T20:39:38.278519+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:36.375542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"75'1","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.271168+0000","last_change":"2023-10-10T20:39:37.267358+0000","last_active":"2023-10-10T20:39:38.271168+0000","last_peered":"2023-10-10T20:39:38.271168+0000","last_clean":"2023-10-10T20:39:38.271168+0000","last_became_active":"2023-10-10T20:39:37.267054+0000","last_became_peered":"2023-10-10T20:39:37.267054+0000","last_unstale":"2023-10-10T20:39:38.271168+0000","last_undegraded":"2023-10-10T20:39:38.271168+0000","last_fullsized":"2023-10-10T20:39:38.271168+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:31.881697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.275258+0000","last_change":"2023-10-10T20:39:38.265695+0000","last_active":"2023-10-10T20:39:38.275258+0000","last_peered":"2023-10-10T20:39:38.275258+0000","last_clean":"2023-10-10T20:39:38.275258+0000","last_became_active":"2023-10-10T20:39:38.265085+0000","last_became_peered":"2023-10-10T20:39:38.265085+0000","last_unstale":"2023-10-10T20:39:38.275258+0000","last_undegraded":"2023-10-10T20:39:38.275258+0000","last_fullsized":"2023-10-10T20:39:38.275258+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:36.375542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.259205+0000","last_change":"2023-10-10T20:39:37.265548+0000","last_active":"2023-10-10T20:39:38.259205+0000","last_peered":"2023-10-10T20:39:38.259205+0000","last_clean":"2023-10-10T20:39:38.259205+0000","last_became_active":"2023-10-10T20:39:37.265284+0000","last_became_peered":"2023-10-10T20:39:37.265284+0000","last_unstale":"2023-10-10T20:39:38.259205+0000","last_undegraded":"2023-10-10T20:39:38.259205+0000","last_fullsized":"2023-10-10T20:39:38.259205+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:53:22.002691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.269167+0000","last_change":"2023-10-10T20:39:38.265111+0000","last_active":"2023-10-10T20:39:38.269167+0000","last_peered":"2023-10-10T20:39:38.269167+0000","last_clean":"2023-10-10T20:39:38.269167+0000","last_became_active":"2023-10-10T20:39:38.264564+0000","last_became_peered":"2023-10-10T20:39:38.264564+0000","last_unstale":"2023-10-10T20:39:38.269167+0000","last_undegraded":"2023-10-10T20:39:38.269167+0000","last_fullsized":"2023-10-10T20:39:38.269167+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:23:40.746261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.257771+0000","last_change":"2023-10-10T20:39:37.270081+0000","last_active":"2023-10-10T20:39:38.257771+0000","last_peered":"2023-10-10T20:39:38.257771+0000","last_clean":"2023-10-10T20:39:38.257771+0000","last_became_active":"2023-10-10T20:39:37.269828+0000","last_became_peered":"2023-10-10T20:39:37.269828+0000","last_unstale":"2023-10-10T20:39:38.257771+0000","last_undegraded":"2023-10-10T20:39:38.257771+0000","last_fullsized":"2023-10-10T20:39:38.257771+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:16:06.819183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.277985+0000","last_change":"2023-10-10T20:39:38.275395+0000","last_active":"2023-10-10T20:39:38.277985+0000","last_peered":"2023-10-10T20:39:38.277985+0000","last_clean":"2023-10-10T20:39:38.277985+0000","last_became_active":"2023-10-10T20:39:38.275125+0000","last_became_peered":"2023-10-10T20:39:38.275125+0000","last_unstale":"2023-10-10T20:39:38.277985+0000","last_undegraded":"2023-10-10T20:39:38.277985+0000","last_fullsized":"2023-10-10T20:39:38.277985+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:52:10.474603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.257995+0000","last_change":"2023-10-10T20:39:37.264681+0000","last_active":"2023-10-10T20:39:38.257995+0000","last_peered":"2023-10-10T20:39:38.257995+0000","last_clean":"2023-10-10T20:39:38.257995+0000","last_became_active":"2023-10-10T20:39:37.264173+0000","last_became_peered":"2023-10-10T20:39:37.264173+0000","last_unstale":"2023-10-10T20:39:38.257995+0000","last_undegraded":"2023-10-10T20:39:38.257995+0000","last_fullsized":"2023-10-10T20:39:38.257995+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:31.881697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.273732+0000","last_change":"2023-10-10T20:39:38.269076+0000","last_active":"2023-10-10T20:39:38.273732+0000","last_peered":"2023-10-10T20:39:38.273732+0000","last_clean":"2023-10-10T20:39:38.273732+0000","last_became_active":"2023-10-10T20:39:38.268736+0000","last_became_peered":"2023-10-10T20:39:38.268736+0000","last_unstale":"2023-10-10T20:39:38.273732+0000","last_undegraded":"2023-10-10T20:39:38.273732+0000","last_fullsized":"2023-10-10T20:39:38.273732+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:36.375542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.256061+0000","last_change":"2023-10-10T20:39:37.270474+0000","last_active":"2023-10-10T20:39:38.256061+0000","last_peered":"2023-10-10T20:39:38.256061+0000","last_clean":"2023-10-10T20:39:38.256061+0000","last_became_active":"2023-10-10T20:39:37.270158+0000","last_became_peered":"2023-10-10T20:39:37.270158+0000","last_unstale":"2023-10-10T20:39:38.256061+0000","last_undegraded":"2023-10-10T20:39:38.256061+0000","last_fullsized":"2023-10-10T20:39:38.256061+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:19:53.834466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.277399+0000","last_change":"2023-10-10T20:39:38.270402+0000","last_active":"2023-10-10T20:39:38.277399+0000","last_peered":"2023-10-10T20:39:38.277399+0000","last_clean":"2023-10-10T20:39:38.277399+0000","last_became_active":"2023-10-10T20:39:38.270069+0000","last_became_peered":"2023-10-10T20:39:38.270069+0000","last_unstale":"2023-10-10T20:39:38.277399+0000","last_undegraded":"2023-10-10T20:39:38.277399+0000","last_fullsized":"2023-10-10T20:39:38.277399+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:49:29.215389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"75'3","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.274260+0000","last_change":"2023-10-10T20:39:37.272341+0000","last_active":"2023-10-10T20:39:38.274260+0000","last_peered":"2023-10-10T20:39:38.274260+0000","last_clean":"2023-10-10T20:39:38.274260+0000","last_became_active":"2023-10-10T20:39:37.272070+0000","last_became_peered":"2023-10-10T20:39:37.272070+0000","last_unstale":"2023-10-10T20:39:38.274260+0000","last_undegraded":"2023-10-10T20:39:38.274260+0000","last_fullsized":"2023-10-10T20:39:38.274260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:42.059614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.279308+0000","last_change":"2023-10-10T20:39:38.277428+0000","last_active":"2023-10-10T20:39:38.279308+0000","last_peered":"2023-10-10T20:39:38.279308+0000","last_clean":"2023-10-10T20:39:38.279308+0000","last_became_active":"2023-10-10T20:39:38.277005+0000","last_became_peered":"2023-10-10T20:39:38.277005+0000","last_unstale":"2023-10-10T20:39:38.279308+0000","last_undegraded":"2023-10-10T20:39:38.279308+0000","last_fullsized":"2023-10-10T20:39:38.279308+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:42.906143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"75'6","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.281399+0000","last_change":"2023-10-10T20:39:37.264016+0000","last_active":"2023-10-10T20:39:38.281399+0000","last_peered":"2023-10-10T20:39:38.281399+0000","last_clean":"2023-10-10T20:39:38.281399+0000","last_became_active":"2023-10-10T20:39:37.263724+0000","last_became_peered":"2023-10-10T20:39:37.263724+0000","last_unstale":"2023-10-10T20:39:38.281399+0000","last_undegraded":"2023-10-10T20:39:38.281399+0000","last_fullsized":"2023-10-10T20:39:38.281399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:25.579798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.277313+0000","last_change":"2023-10-10T20:39:38.273085+0000","last_active":"2023-10-10T20:39:38.277313+0000","last_peered":"2023-10-10T20:39:38.277313+0000","last_clean":"2023-10-10T20:39:38.277313+0000","last_became_active":"2023-10-10T20:39:38.270412+0000","last_became_peered":"2023-10-10T20:39:38.270412+0000","last_unstale":"2023-10-10T20:39:38.277313+0000","last_undegraded":"2023-10-10T20:39:38.277313+0000","last_fullsized":"2023-10-10T20:39:38.277313+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:55:15.047824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"75'2","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.279921+0000","last_change":"2023-10-10T20:39:37.272988+0000","last_active":"2023-10-10T20:39:38.279921+0000","last_peered":"2023-10-10T20:39:38.279921+0000","last_clean":"2023-10-10T20:39:38.279921+0000","last_became_active":"2023-10-10T20:39:37.272715+0000","last_became_peered":"2023-10-10T20:39:37.272715+0000","last_unstale":"2023-10-10T20:39:38.279921+0000","last_undegraded":"2023-10-10T20:39:38.279921+0000","last_fullsized":"2023-10-10T20:39:38.279921+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:10:21.213337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.275121+0000","last_change":"2023-10-10T20:39:38.265739+0000","last_active":"2023-10-10T20:39:38.275121+0000","last_peered":"2023-10-10T20:39:38.275121+0000","last_clean":"2023-10-10T20:39:38.275121+0000","last_became_active":"2023-10-10T20:39:38.265420+0000","last_became_peered":"2023-10-10T20:39:38.265420+0000","last_unstale":"2023-10-10T20:39:38.275121+0000","last_undegraded":"2023-10-10T20:39:38.275121+0000","last_fullsized":"2023-10-10T20:39:38.275121+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:09:38.241624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.253237+0000","last_change":"2023-10-10T20:39:37.268230+0000","last_active":"2023-10-10T20:39:38.253237+0000","last_peered":"2023-10-10T20:39:38.253237+0000","last_clean":"2023-10-10T20:39:38.253237+0000","last_became_active":"2023-10-10T20:39:37.267986+0000","last_became_peered":"2023-10-10T20:39:37.267986+0000","last_unstale":"2023-10-10T20:39:38.253237+0000","last_undegraded":"2023-10-10T20:39:38.253237+0000","last_fullsized":"2023-10-10T20:39:38.253237+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:13:05.731066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.275723+0000","last_change":"2023-10-10T20:39:38.271304+0000","last_active":"2023-10-10T20:39:38.275723+0000","last_peered":"2023-10-10T20:39:38.275723+0000","last_clean":"2023-10-10T20:39:38.275723+0000","last_became_active":"2023-10-10T20:39:38.271054+0000","last_became_peered":"2023-10-10T20:39:38.271054+0000","last_unstale":"2023-10-10T20:39:38.275723+0000","last_undegraded":"2023-10-10T20:39:38.275723+0000","last_fullsized":"2023-10-10T20:39:38.275723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:51:00.261125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.264570+0000","last_change":"2023-10-10T20:39:37.269974+0000","last_active":"2023-10-10T20:39:38.264570+0000","last_peered":"2023-10-10T20:39:38.264570+0000","last_clean":"2023-10-10T20:39:38.264570+0000","last_became_active":"2023-10-10T20:39:37.269650+0000","last_became_peered":"2023-10-10T20:39:37.269650+0000","last_unstale":"2023-10-10T20:39:38.264570+0000","last_undegraded":"2023-10-10T20:39:38.264570+0000","last_fullsized":"2023-10-10T20:39:38.264570+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:16.804099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.280266+0000","last_change":"2023-10-10T20:39:38.274204+0000","last_active":"2023-10-10T20:39:38.280266+0000","last_peered":"2023-10-10T20:39:38.280266+0000","last_clean":"2023-10-10T20:39:38.280266+0000","last_became_active":"2023-10-10T20:39:38.273967+0000","last_became_peered":"2023-10-10T20:39:38.273967+0000","last_unstale":"2023-10-10T20:39:38.280266+0000","last_undegraded":"2023-10-10T20:39:38.280266+0000","last_fullsized":"2023-10-10T20:39:38.280266+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:53:23.009166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"75'8","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.281359+0000","last_change":"2023-10-10T20:39:37.266948+0000","last_active":"2023-10-10T20:39:38.281359+0000","last_peered":"2023-10-10T20:39:38.281359+0000","last_clean":"2023-10-10T20:39:38.281359+0000","last_became_active":"2023-10-10T20:39:37.266149+0000","last_became_peered":"2023-10-10T20:39:37.266149+0000","last_unstale":"2023-10-10T20:39:38.281359+0000","last_undegraded":"2023-10-10T20:39:38.281359+0000","last_fullsized":"2023-10-10T20:39:38.281359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:25.579798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.275554+0000","last_change":"2023-10-10T20:39:38.272169+0000","last_active":"2023-10-10T20:39:38.275554+0000","last_peered":"2023-10-10T20:39:38.275554+0000","last_clean":"2023-10-10T20:39:38.275554+0000","last_became_active":"2023-10-10T20:39:38.271894+0000","last_became_peered":"2023-10-10T20:39:38.271894+0000","last_unstale":"2023-10-10T20:39:38.275554+0000","last_undegraded":"2023-10-10T20:39:38.275554+0000","last_fullsized":"2023-10-10T20:39:38.275554+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:05:53.997189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"75'1","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.272629+0000","last_change":"2023-10-10T20:39:37.265234+0000","last_active":"2023-10-10T20:39:38.272629+0000","last_peered":"2023-10-10T20:39:38.272629+0000","last_clean":"2023-10-10T20:39:38.272629+0000","last_became_active":"2023-10-10T20:39:37.264953+0000","last_became_peered":"2023-10-10T20:39:37.264953+0000","last_unstale":"2023-10-10T20:39:38.272629+0000","last_undegraded":"2023-10-10T20:39:38.272629+0000","last_fullsized":"2023-10-10T20:39:38.272629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:53:22.002691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.269380+0000","last_change":"2023-10-10T20:39:38.265818+0000","last_active":"2023-10-10T20:39:38.269380+0000","last_peered":"2023-10-10T20:39:38.269380+0000","last_clean":"2023-10-10T20:39:38.269380+0000","last_became_active":"2023-10-10T20:39:38.265370+0000","last_became_peered":"2023-10-10T20:39:38.265370+0000","last_unstale":"2023-10-10T20:39:38.269380+0000","last_undegraded":"2023-10-10T20:39:38.269380+0000","last_fullsized":"2023-10-10T20:39:38.269380+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:05:53.997189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"75'2","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.279240+0000","last_change":"2023-10-10T20:39:37.269676+0000","last_active":"2023-10-10T20:39:38.279240+0000","last_peered":"2023-10-10T20:39:38.279240+0000","last_clean":"2023-10-10T20:39:38.279240+0000","last_became_active":"2023-10-10T20:39:37.269437+0000","last_became_peered":"2023-10-10T20:39:37.269437+0000","last_unstale":"2023-10-10T20:39:38.279240+0000","last_undegraded":"2023-10-10T20:39:38.279240+0000","last_fullsized":"2023-10-10T20:39:38.279240+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:43.983815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.279271+0000","last_change":"2023-10-10T20:39:38.274899+0000","last_active":"2023-10-10T20:39:38.279271+0000","last_peered":"2023-10-10T20:39:38.279271+0000","last_clean":"2023-10-10T20:39:38.279271+0000","last_became_active":"2023-10-10T20:39:38.274659+0000","last_became_peered":"2023-10-10T20:39:38.274659+0000","last_unstale":"2023-10-10T20:39:38.279271+0000","last_undegraded":"2023-10-10T20:39:38.279271+0000","last_fullsized":"2023-10-10T20:39:38.279271+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:49.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.260276+0000","last_change":"2023-10-10T20:39:37.264681+0000","last_active":"2023-10-10T20:39:38.260276+0000","last_peered":"2023-10-10T20:39:38.260276+0000","last_clean":"2023-10-10T20:39:38.260276+0000","last_became_active":"2023-10-10T20:39:37.264410+0000","last_became_peered":"2023-10-10T20:39:37.264410+0000","last_unstale":"2023-10-10T20:39:38.260276+0000","last_undegraded":"2023-10-10T20:39:38.260276+0000","last_fullsized":"2023-10-10T20:39:38.260276+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:21.790518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.284001+0000","last_change":"2023-10-10T20:39:38.277019+0000","last_active":"2023-10-10T20:39:38.284001+0000","last_peered":"2023-10-10T20:39:38.284001+0000","last_clean":"2023-10-10T20:39:38.284001+0000","last_became_active":"2023-10-10T20:39:38.276134+0000","last_became_peered":"2023-10-10T20:39:38.276134+0000","last_unstale":"2023-10-10T20:39:38.284001+0000","last_undegraded":"2023-10-10T20:39:38.284001+0000","last_fullsized":"2023-10-10T20:39:38.284001+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:52.151912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"75'2","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.281468+0000","last_change":"2023-10-10T20:39:37.272951+0000","last_active":"2023-10-10T20:39:38.281468+0000","last_peered":"2023-10-10T20:39:38.281468+0000","last_clean":"2023-10-10T20:39:38.281468+0000","last_became_active":"2023-10-10T20:39:37.272680+0000","last_became_peered":"2023-10-10T20:39:37.272680+0000","last_unstale":"2023-10-10T20:39:38.281468+0000","last_undegraded":"2023-10-10T20:39:38.281468+0000","last_fullsized":"2023-10-10T20:39:38.281468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:09:37.235149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.273518+0000","last_change":"2023-10-10T20:39:38.271421+0000","last_active":"2023-10-10T20:39:38.273518+0000","last_peered":"2023-10-10T20:39:38.273518+0000","last_clean":"2023-10-10T20:39:38.273518+0000","last_became_active":"2023-10-10T20:39:38.271180+0000","last_became_peered":"2023-10-10T20:39:38.271180+0000","last_unstale":"2023-10-10T20:39:38.273518+0000","last_undegraded":"2023-10-10T20:39:38.273518+0000","last_fullsized":"2023-10-10T20:39:38.273518+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:49.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.263318+0000","last_change":"2023-10-10T20:39:37.272869+0000","last_active":"2023-10-10T20:39:38.263318+0000","last_peered":"2023-10-10T20:39:38.263318+0000","last_clean":"2023-10-10T20:39:38.263318+0000","last_became_active":"2023-10-10T20:39:37.272418+0000","last_became_peered":"2023-10-10T20:39:37.272418+0000","last_unstale":"2023-10-10T20:39:38.263318+0000","last_undegraded":"2023-10-10T20:39:38.263318+0000","last_fullsized":"2023-10-10T20:39:38.263318+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:20:16.442207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.273621+0000","last_change":"2023-10-10T20:39:38.269196+0000","last_active":"2023-10-10T20:39:38.273621+0000","last_peered":"2023-10-10T20:39:38.273621+0000","last_clean":"2023-10-10T20:39:38.273621+0000","last_became_active":"2023-10-10T20:39:38.268930+0000","last_became_peered":"2023-10-10T20:39:38.268930+0000","last_unstale":"2023-10-10T20:39:38.273621+0000","last_undegraded":"2023-10-10T20:39:38.273621+0000","last_fullsized":"2023-10-10T20:39:38.273621+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:11:03.307599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.262940+0000","last_change":"2023-10-10T20:39:37.272054+0000","last_active":"2023-10-10T20:39:38.262940+0000","last_peered":"2023-10-10T20:39:38.262940+0000","last_clean":"2023-10-10T20:39:38.262940+0000","last_became_active":"2023-10-10T20:39:37.271818+0000","last_became_peered":"2023-10-10T20:39:37.271818+0000","last_unstale":"2023-10-10T20:39:38.262940+0000","last_undegraded":"2023-10-10T20:39:38.262940+0000","last_fullsized":"2023-10-10T20:39:38.262940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:57:37.129049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.281345+0000","last_change":"2023-10-10T20:39:38.279650+0000","last_active":"2023-10-10T20:39:38.281345+0000","last_peered":"2023-10-10T20:39:38.281345+0000","last_clean":"2023-10-10T20:39:38.281345+0000","last_became_active":"2023-10-10T20:39:38.279332+0000","last_became_peered":"2023-10-10T20:39:38.279332+0000","last_unstale":"2023-10-10T20:39:38.281345+0000","last_undegraded":"2023-10-10T20:39:38.281345+0000","last_fullsized":"2023-10-10T20:39:38.281345+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:05:48.142508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.253102+0000","last_change":"2023-10-10T20:39:37.275925+0000","last_active":"2023-10-10T20:39:38.253102+0000","last_peered":"2023-10-10T20:39:38.253102+0000","last_clean":"2023-10-10T20:39:38.253102+0000","last_became_active":"2023-10-10T20:39:37.275666+0000","last_became_peered":"2023-10-10T20:39:37.275666+0000","last_unstale":"2023-10-10T20:39:38.253102+0000","last_undegraded":"2023-10-10T20:39:38.253102+0000","last_fullsized":"2023-10-10T20:39:38.253102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:01:58.347464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.278940+0000","last_change":"2023-10-10T20:39:38.271236+0000","last_active":"2023-10-10T20:39:38.278940+0000","last_peered":"2023-10-10T20:39:38.278940+0000","last_clean":"2023-10-10T20:39:38.278940+0000","last_became_active":"2023-10-10T20:39:38.271023+0000","last_became_peered":"2023-10-10T20:39:38.271023+0000","last_unstale":"2023-10-10T20:39:38.278940+0000","last_undegraded":"2023-10-10T20:39:38.278940+0000","last_fullsized":"2023-10-10T20:39:38.278940+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:45.015636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"75'2","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.280492+0000","last_change":"2023-10-10T20:39:37.265642+0000","last_active":"2023-10-10T20:39:38.280492+0000","last_peered":"2023-10-10T20:39:38.280492+0000","last_clean":"2023-10-10T20:39:38.280492+0000","last_became_active":"2023-10-10T20:39:37.265399+0000","last_became_peered":"2023-10-10T20:39:37.265399+0000","last_unstale":"2023-10-10T20:39:38.280492+0000","last_undegraded":"2023-10-10T20:39:38.280492+0000","last_fullsized":"2023-10-10T20:39:38.280492+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:53:41.456643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.275523+0000","last_change":"2023-10-10T20:39:38.267513+0000","last_active":"2023-10-10T20:39:38.275523+0000","last_peered":"2023-10-10T20:39:38.275523+0000","last_clean":"2023-10-10T20:39:38.275523+0000","last_became_active":"2023-10-10T20:39:38.266509+0000","last_became_peered":"2023-10-10T20:39:38.266509+0000","last_unstale":"2023-10-10T20:39:38.275523+0000","last_undegraded":"2023-10-10T20:39:38.275523+0000","last_fullsized":"2023-10-10T20:39:38.275523+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:00:50.973187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.261956+0000","last_change":"2023-10-10T20:39:37.272730+0000","last_active":"2023-10-10T20:39:38.261956+0000","last_peered":"2023-10-10T20:39:38.261956+0000","last_clean":"2023-10-10T20:39:38.261956+0000","last_became_active":"2023-10-10T20:39:37.272487+0000","last_became_peered":"2023-10-10T20:39:37.272487+0000","last_unstale":"2023-10-10T20:39:38.261956+0000","last_undegraded":"2023-10-10T20:39:38.261956+0000","last_fullsized":"2023-10-10T20:39:38.261956+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:40.276426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.277882+0000","last_change":"2023-10-10T20:39:38.273704+0000","last_active":"2023-10-10T20:39:38.277882+0000","last_peered":"2023-10-10T20:39:38.277882+0000","last_clean":"2023-10-10T20:39:38.277882+0000","last_became_active":"2023-10-10T20:39:38.273452+0000","last_became_peered":"2023-10-10T20:39:38.273452+0000","last_unstale":"2023-10-10T20:39:38.277882+0000","last_undegraded":"2023-10-10T20:39:38.277882+0000","last_fullsized":"2023-10-10T20:39:38.277882+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:00.911369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.255343+0000","last_change":"2023-10-10T20:39:37.275883+0000","last_active":"2023-10-10T20:39:38.255343+0000","last_peered":"2023-10-10T20:39:38.255343+0000","last_clean":"2023-10-10T20:39:38.255343+0000","last_became_active":"2023-10-10T20:39:37.275635+0000","last_became_peered":"2023-10-10T20:39:37.275635+0000","last_unstale":"2023-10-10T20:39:38.255343+0000","last_undegraded":"2023-10-10T20:39:38.255343+0000","last_fullsized":"2023-10-10T20:39:38.255343+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:23:59.211838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.279292+0000","last_change":"2023-10-10T20:39:38.277262+0000","last_active":"2023-10-10T20:39:38.279292+0000","last_peered":"2023-10-10T20:39:38.279292+0000","last_clean":"2023-10-10T20:39:38.279292+0000","last_became_active":"2023-10-10T20:39:38.276700+0000","last_became_peered":"2023-10-10T20:39:38.276700+0000","last_unstale":"2023-10-10T20:39:38.279292+0000","last_undegraded":"2023-10-10T20:39:38.279292+0000","last_fullsized":"2023-10-10T20:39:38.279292+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:05:48.142508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.259518+0000","last_change":"2023-10-10T20:39:37.264870+0000","last_active":"2023-10-10T20:39:38.259518+0000","last_peered":"2023-10-10T20:39:38.259518+0000","last_clean":"2023-10-10T20:39:38.259518+0000","last_became_active":"2023-10-10T20:39:37.264606+0000","last_became_peered":"2023-10-10T20:39:37.264606+0000","last_unstale":"2023-10-10T20:39:38.259518+0000","last_undegraded":"2023-10-10T20:39:38.259518+0000","last_fullsized":"2023-10-10T20:39:38.259518+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:02.132949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.270426+0000","last_change":"2023-10-10T20:39:38.268001+0000","last_active":"2023-10-10T20:39:38.270426+0000","last_peered":"2023-10-10T20:39:38.270426+0000","last_clean":"2023-10-10T20:39:38.270426+0000","last_became_active":"2023-10-10T20:39:38.267724+0000","last_became_peered":"2023-10-10T20:39:38.267724+0000","last_unstale":"2023-10-10T20:39:38.270426+0000","last_undegraded":"2023-10-10T20:39:38.270426+0000","last_fullsized":"2023-10-10T20:39:38.270426+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:42.906143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.259552+0000","last_change":"2023-10-10T20:39:37.267077+0000","last_active":"2023-10-10T20:39:38.259552+0000","last_peered":"2023-10-10T20:39:38.259552+0000","last_clean":"2023-10-10T20:39:38.259552+0000","last_became_active":"2023-10-10T20:39:37.266325+0000","last_became_peered":"2023-10-10T20:39:37.266325+0000","last_unstale":"2023-10-10T20:39:38.259552+0000","last_undegraded":"2023-10-10T20:39:38.259552+0000","last_fullsized":"2023-10-10T20:39:38.259552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:03.986546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.277463+0000","last_change":"2023-10-10T20:39:38.273591+0000","last_active":"2023-10-10T20:39:38.277463+0000","last_peered":"2023-10-10T20:39:38.277463+0000","last_clean":"2023-10-10T20:39:38.277463+0000","last_became_active":"2023-10-10T20:39:38.273200+0000","last_became_peered":"2023-10-10T20:39:38.273200+0000","last_unstale":"2023-10-10T20:39:38.277463+0000","last_undegraded":"2023-10-10T20:39:38.277463+0000","last_fullsized":"2023-10-10T20:39:38.277463+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:18:44.235474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.265976+0000","last_change":"2023-10-10T20:39:37.262060+0000","last_active":"2023-10-10T20:39:38.265976+0000","last_peered":"2023-10-10T20:39:38.265976+0000","last_clean":"2023-10-10T20:39:38.265976+0000","last_became_active":"2023-10-10T20:39:37.261636+0000","last_became_peered":"2023-10-10T20:39:37.261636+0000","last_unstale":"2023-10-10T20:39:38.265976+0000","last_undegraded":"2023-10-10T20:39:38.265976+0000","last_fullsized":"2023-10-10T20:39:38.265976+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:15.557848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.279367+0000","last_change":"2023-10-10T20:39:38.275698+0000","last_active":"2023-10-10T20:39:38.279367+0000","last_peered":"2023-10-10T20:39:38.279367+0000","last_clean":"2023-10-10T20:39:38.279367+0000","last_became_active":"2023-10-10T20:39:38.275471+0000","last_became_peered":"2023-10-10T20:39:38.275471+0000","last_unstale":"2023-10-10T20:39:38.279367+0000","last_undegraded":"2023-10-10T20:39:38.279367+0000","last_fullsized":"2023-10-10T20:39:38.279367+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:13:06.737541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.256020+0000","last_change":"2023-10-10T20:39:37.264795+0000","last_active":"2023-10-10T20:39:38.256020+0000","last_peered":"2023-10-10T20:39:38.256020+0000","last_clean":"2023-10-10T20:39:38.256020+0000","last_became_active":"2023-10-10T20:39:37.263322+0000","last_became_peered":"2023-10-10T20:39:37.263322+0000","last_unstale":"2023-10-10T20:39:38.256020+0000","last_undegraded":"2023-10-10T20:39:38.256020+0000","last_fullsized":"2023-10-10T20:39:38.256020+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:54:15.557848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.272397+0000","last_change":"2023-10-10T20:39:38.266199+0000","last_active":"2023-10-10T20:39:38.272397+0000","last_peered":"2023-10-10T20:39:38.272397+0000","last_clean":"2023-10-10T20:39:38.272397+0000","last_became_active":"2023-10-10T20:39:38.263733+0000","last_became_peered":"2023-10-10T20:39:38.263733+0000","last_unstale":"2023-10-10T20:39:38.272397+0000","last_undegraded":"2023-10-10T20:39:38.272397+0000","last_fullsized":"2023-10-10T20:39:38.272397+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:49.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.265309+0000","last_change":"2023-10-10T20:39:37.263674+0000","last_active":"2023-10-10T20:39:38.265309+0000","last_peered":"2023-10-10T20:39:38.265309+0000","last_clean":"2023-10-10T20:39:38.265309+0000","last_became_active":"2023-10-10T20:39:37.263294+0000","last_became_peered":"2023-10-10T20:39:37.263294+0000","last_unstale":"2023-10-10T20:39:38.265309+0000","last_undegraded":"2023-10-10T20:39:38.265309+0000","last_fullsized":"2023-10-10T20:39:38.265309+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:02.132949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.280680+0000","last_change":"2023-10-10T20:39:38.277081+0000","last_active":"2023-10-10T20:39:38.280680+0000","last_peered":"2023-10-10T20:39:38.280680+0000","last_clean":"2023-10-10T20:39:38.280680+0000","last_became_active":"2023-10-10T20:39:38.276516+0000","last_became_peered":"2023-10-10T20:39:38.276516+0000","last_unstale":"2023-10-10T20:39:38.280680+0000","last_undegraded":"2023-10-10T20:39:38.280680+0000","last_fullsized":"2023-10-10T20:39:38.280680+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:37.239917+0000","last_clean_scrub_stamp":"2023-10-10T20:39:37.239917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:03.139424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"75'1","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.280847+0000","last_change":"2023-10-10T20:39:37.264553+0000","last_active":"2023-10-10T20:39:38.280847+0000","last_peered":"2023-10-10T20:39:38.280847+0000","last_clean":"2023-10-10T20:39:38.280847+0000","last_became_active":"2023-10-10T20:39:37.264274+0000","last_became_peered":"2023-10-10T20:39:37.264274+0000","last_unstale":"2023-10-10T20:39:38.280847+0000","last_undegraded":"2023-10-10T20:39:38.280847+0000","last_fullsized":"2023-10-10T20:39:38.280847+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:39:36.233442+0000","last_clean_scrub_stamp":"2023-10-10T20:39:36.233442+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:25.579798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":219,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-10T20:39:38.256974+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:39:38.256974+0000","last_peered":"2023-10-10T20:39:38.256974+0000","last_clean":"2023-10-10T20:39:38.256974+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:39:38.256974+0000","last_undegraded":"2023-10-10T20:39:38.256974+0000","last_fullsized":"2023-10-10T20:39:38.256974+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298784,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073056,"statfs":{"total":96636764160,"available":96330809344,"internally_reserved":0,"allocated":1531904,"data_stored":991355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.60499999999999998,"15min":0.84999999999999998},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.747,"5min":6.8630000000000004,"15min":6.8630000000000004},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.56299999999999994,"15min":0.63800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.629,"5min":4.1959999999999997,"15min":4.1959999999999997},"last":0.46600000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.65300000000000002,"15min":1.0489999999999999},"min":{"1min":0.39900000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.78800000000000003,"5min":10.1,"15min":10.1},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.61299999999999999,"15min":0.749},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66400000000000003,"5min":6.806,"15min":6.806},"last":0.54400000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.64000000000000001,"15min":0.80200000000000005},"min":{"1min":0.41599999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":6.9340000000000002,"15min":6.9340000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59499999999999997,"15min":0.75900000000000001},"min":{"1min":0.38800000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73199999999999998,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.48399999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61799999999999999,"15min":0.77800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70899999999999996,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.68100000000000005},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71899999999999997,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.65000000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.34599999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.626,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.374,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78400000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.78400000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.73299999999999998,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72699999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.72699999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58899999999999997,"15min":0.57199999999999995},"min":{"1min":0.48699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77100000000000002,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.46800000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75600000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.746}]}]},{"osd":6,"up_from":13,"seq":55834574916,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298212,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073628,"statfs":{"total":96636764160,"available":96331395072,"internally_reserved":0,"allocated":946176,"data_stored":401583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55700000000000005,"15min":0.68899999999999995},"min":{"1min":0.40799999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.73099999999999998,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57899999999999996,"15min":0.73299999999999998},"min":{"1min":0.372,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70799999999999996,"5min":7.8090000000000002,"15min":7.8090000000000002},"last":0.63200000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.72899999999999998},"min":{"1min":0.39700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71899999999999997,"5min":7.867,"15min":7.867},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.64200000000000002,"15min":0.92100000000000004},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75700000000000001,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.54200000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.747,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.63400000000000001,"15min":0.874},"min":{"1min":0.48399999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.71199999999999997,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":0.61599999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.628},"min":{"1min":0.439,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79000000000000004,"5min":4.8620000000000001,"15min":4.8620000000000001},"last":0.496},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.73499999999999999},"min":{"1min":0.47199999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.755,"5min":7.9009999999999998,"15min":7.9009999999999998},"last":0.50700000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.504,"15min":0.46400000000000002},"min":{"1min":0.42799999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71799999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.56499999999999995},"min":{"1min":0.54200000000000004,"5min":0.37,"15min":0.37},"max":{"1min":0.69899999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.60099999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51400000000000001,"15min":0.48599999999999999},"min":{"1min":0.40400000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55200000000000005,"15min":0.52600000000000002},"min":{"1min":0.47799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.57099999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.42799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74399999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56899999999999995,"15min":0.54100000000000004},"min":{"1min":0.52700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.747,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.58799999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298212,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073628,"statfs":{"total":96636764160,"available":96331395072,"internally_reserved":0,"allocated":946176,"data_stored":401111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.36399999999999999,"15min":0.49099999999999999},"min":{"1min":0.219,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.56799999999999995,"5min":5.0380000000000003,"15min":5.0380000000000003},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33100000000000002,"15min":0.36499999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.47299999999999998,"5min":1.194,"15min":1.194},"last":0.41399999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.36799999999999999},"min":{"1min":0.27800000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.51400000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.379},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.52000000000000002},"min":{"1min":0.26000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52500000000000002,"5min":5.0919999999999996,"15min":5.0919999999999996},"last":0.47699999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.40799999999999997,"15min":0.54300000000000004},"min":{"1min":0.28000000000000003,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.59899999999999998,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38100000000000001,"15min":0.41299999999999998},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55500000000000005,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.36199999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.40000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92400000000000004,"5min":1.254,"15min":1.254},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60599999999999998,"15min":0.71899999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96499999999999997,"5min":4.9850000000000003,"15min":4.9850000000000003},"last":0.69899999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97599999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.42899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.878,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.68600000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67600000000000005,"15min":0.86699999999999999},"min":{"1min":0.46500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.995,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.40699999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85799999999999998,"5min":1.171,"15min":1.171},"last":0.70799999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60099999999999998,"15min":0.62},"min":{"1min":0.443,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.004,"5min":1.214,"15min":1.214},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63700000000000001,"15min":0.73699999999999999},"min":{"1min":0.41999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90500000000000003,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.64500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302876,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068964,"statfs":{"total":96636764160,"available":96326619136,"internally_reserved":0,"allocated":1527808,"data_stored":990358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.377,"15min":0.55100000000000005},"min":{"1min":0.20699999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.51600000000000001,"5min":7.9249999999999998,"15min":7.9249999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.374,"15min":0.52400000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45800000000000002,"5min":6.6299999999999999,"15min":6.6299999999999999},"last":0.41999999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.39000000000000001,"15min":0.57399999999999995},"min":{"1min":0.23100000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.47499999999999998,"5min":8.2509999999999994,"15min":8.2509999999999994},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39800000000000002,"15min":0.54800000000000004},"min":{"1min":0.245,"5min":0.222,"15min":0.222},"max":{"1min":0.52400000000000002,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.48099999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.40500000000000003,"15min":0.57699999999999996},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.45400000000000001,"15min":0.63400000000000001},"min":{"1min":0.31900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57899999999999996,"5min":7.8929999999999998,"15min":7.8929999999999998},"last":0.56699999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.61099999999999999,"15min":0.73199999999999998},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78000000000000003,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.63600000000000001,"15min":0.79600000000000004},"min":{"1min":0.45100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.95399999999999996,"5min":7.6920000000000002,"15min":7.6920000000000002},"last":0.85599999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.77500000000000002},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92400000000000004,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.64300000000000002,"15min":0.79900000000000004},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91400000000000003,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":0.84099999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67000000000000004,"15min":0.83999999999999997},"min":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94499999999999995,"5min":7.8360000000000003,"15min":7.8360000000000003},"last":0.88},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66400000000000003,"15min":0.77900000000000003},"min":{"1min":0.49199999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90300000000000002,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.86899999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64000000000000001,"15min":0.76900000000000002},"min":{"1min":0.45900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93300000000000005,"5min":6.7160000000000002,"15min":6.7160000000000002},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66100000000000003,"15min":0.80900000000000005},"min":{"1min":0.44600000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.878,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.80600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298204,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073636,"statfs":{"total":96636764160,"available":96331403264,"internally_reserved":0,"allocated":937984,"data_stored":401471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.31900000000000001,"15min":0.32800000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.59699999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.245},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.33600000000000002,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45400000000000001,"5min":0.746,"15min":0.746},"last":0.26400000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.45400000000000001,"15min":0.77100000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.63,"5min":11.967000000000001,"15min":11.967000000000001},"last":0.311},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39000000000000001,"15min":0.498},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52700000000000002,"5min":4.1100000000000003,"15min":4.1100000000000003},"last":0.32500000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.41199999999999998,"15min":0.57899999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48799999999999999,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.50900000000000001,"15min":0.82199999999999995},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65900000000000003,"5min":11.935,"15min":11.935},"last":0.34799999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54400000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.80100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.58999999999999997},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.41499999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.33400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83599999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.53100000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81599999999999995,"5min":1.052,"15min":1.052},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64500000000000002,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85999999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.61599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574916,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298212,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073628,"statfs":{"total":96636764160,"available":96331395072,"internally_reserved":0,"allocated":946176,"data_stored":402453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.309,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44400000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.317},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48699999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.55400000000000005}]},{"osd":1,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33600000000000002,"15min":0.33000000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.49199999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38200000000000001,"15min":0.376},"min":{"1min":0.307,"5min":0.251,"15min":0.251},"max":{"1min":0.53900000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.57099999999999995}]},{"osd":2,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.34899999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35499999999999998,"15min":0.36099999999999999},"min":{"1min":0.25900000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.51600000000000001,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.64300000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.38100000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83299999999999996,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.82299999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.39400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78700000000000003,"5min":1.05,"15min":1.05},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86799999999999999,"5min":1.01,"15min":1.01},"last":0.76200000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92000000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.877,"5min":1.026,"15min":1.026},"last":0.85499999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77100000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.46600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89400000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.88400000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298200,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073640,"statfs":{"total":96636764160,"available":96331407360,"internally_reserved":0,"allocated":933888,"data_stored":400917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58199999999999996,"5min":0.747,"15min":0.747},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.59799999999999998,"15min":0.79200000000000004},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69699999999999995,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.46300000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.60299999999999998,"15min":0.79900000000000004},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60299999999999998,"5min":7.7160000000000002,"15min":7.7160000000000002},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.39700000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68100000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63600000000000001,"15min":0.81799999999999995},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.50600000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.58899999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.436}]},{"osd":5,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.502,"15min":0.48399999999999999},"min":{"1min":0.34599999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.38400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77300000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.63400000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.41699999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72099999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.52600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.626}]},{"osd":7,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.58199999999999996},"min":{"1min":0.44700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.58999999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574916,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298216,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073624,"statfs":{"total":96636764160,"available":96331390976,"internally_reserved":0,"allocated":950272,"data_stored":402475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.27900000000000003,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.59699999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.32000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.73299999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.47599999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.60199999999999998,"15min":0.66600000000000004},"min":{"1min":0.38500000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.66300000000000003,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.58499999999999996,"15min":0.65100000000000002},"min":{"1min":0.33500000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65200000000000002,"5min":5.3109999999999999,"15min":5.3109999999999999},"last":0.45000000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.60799999999999998,"15min":0.67400000000000004},"min":{"1min":0.373,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72799999999999998,"5min":5.2960000000000003,"15min":5.2960000000000003},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.33900000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71999999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.54800000000000004}]},{"osd":3,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.64100000000000001,"15min":0.70699999999999996},"min":{"1min":0.39900000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80000000000000004,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.63700000000000001,"15min":0.71699999999999997},"min":{"1min":0.439,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69499999999999995,"5min":5.2850000000000001,"15min":5.2850000000000001},"last":0.505}]},{"osd":4,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48799999999999999,"15min":0.499},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64200000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.35099999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.56200000000000006}]},{"osd":6,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.86099999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.753,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.59199999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.36499999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84799999999999998,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82199999999999995,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.60899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:39:47.202 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:39:47.202 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:39:47.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.353+0000 7f6f7b76a700 1 Processor -- start 2023-10-10T20:39:47.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.353+0000 7f6f7b76a700 1 -- start start 2023-10-10T20:39:47.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.354+0000 7f6f7b76a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6f74069c70 0x7f6f740cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:47.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.354+0000 7f6f7b76a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6f740af320 0x7f6f740cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:47.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.354+0000 7f6f7b76a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f740ce380 con 0x7f6f74159950 2023-10-10T20:39:47.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.354+0000 7f6f7b76a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f740cec20 con 0x7f6f74069c70 2023-10-10T20:39:47.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.354+0000 7f6f7b76a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740afa90 con 0x7f6f740af320 2023-10-10T20:39:47.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.354+0000 7f6f78d05700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6f74069c70 0x7f6f740cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:47.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.354+0000 7f6f79506700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6f74159950 0x7f6f74069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48216/0 (socket says 172.21.15.89:48216) 2023-10-10T20:39:47.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.354+0000 7f6f79506700 1 -- 172.21.15.89:0/3447847073 learned_addr learned my addr 172.21.15.89:0/3447847073 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:47.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.354+0000 7f6f78d05700 1 -- 172.21.15.89:0/3447847073 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6f740af320 msgr2=0x7f6f740cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:47.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.354+0000 7f6f79d07700 1 --2- 172.21.15.89:0/3447847073 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6f740af320 0x7f6f740cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:47.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.354+0000 7f6f78d05700 1 --2- 172.21.15.89:0/3447847073 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6f740af320 0x7f6f740cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.354+0000 7f6f6bfff700 1 -- 172.21.15.89:0/3447847073 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2988787655 0 0) 0x7f6f740ce380 con 0x7f6f74159950 2023-10-10T20:39:47.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.354+0000 7f6f78d05700 1 -- 172.21.15.89:0/3447847073 >> v1:172.21.15.89:6789/0 conn(0x7f6f74159950 legacy=0x7f6f74069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:47.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.354+0000 7f6f78d05700 1 -- 172.21.15.89:0/3447847073 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f740ce5d0 con 0x7f6f74069c70 2023-10-10T20:39:47.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.355+0000 7f6f78d05700 1 --2- 172.21.15.89:0/3447847073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f74069c70 0x7f6f740cbb90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6f640098e0 tx=0x7f6f640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0946e241f47986f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:47.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.355+0000 7f6f6bfff700 1 -- 172.21.15.89:0/3447847073 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64009e80 con 0x7f6f74069c70 2023-10-10T20:39:47.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.355+0000 7f6f6bfff700 1 -- 172.21.15.89:0/3447847073 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6400b860 con 0x7f6f74069c70 2023-10-10T20:39:47.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.355+0000 7f6f6bfff700 1 -- 172.21.15.89:0/3447847073 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400bbe0 con 0x7f6f74069c70 2023-10-10T20:39:47.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.355+0000 7f6f7b76a700 1 -- 172.21.15.89:0/3447847073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f74069c70 msgr2=0x7f6f740cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:47.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.355+0000 7f6f7b76a700 1 --2- 172.21.15.89:0/3447847073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f74069c70 0x7f6f740cbb90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6f640098e0 tx=0x7f6f640091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.355+0000 7f6f7b76a700 1 -- 172.21.15.89:0/3447847073 shutdown_connections 2023-10-10T20:39:47.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.355+0000 7f6f7b76a700 1 --2- 172.21.15.89:0/3447847073 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6f740af320 0x7f6f740cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.355+0000 7f6f7b76a700 1 --2- 172.21.15.89:0/3447847073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f74069c70 0x7f6f740cbb90 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.355+0000 7f6f7b76a700 1 -- 172.21.15.89:0/3447847073 >> 172.21.15.89:0/3447847073 conn(0x7f6f74068e00 msgr2=0x7f6f740d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:47.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.355+0000 7f6f7b76a700 1 -- 172.21.15.89:0/3447847073 shutdown_connections 2023-10-10T20:39:47.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.355+0000 7f6f7b76a700 1 -- 172.21.15.89:0/3447847073 wait complete. 2023-10-10T20:39:47.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.356+0000 7f6f7b76a700 1 Processor -- start 2023-10-10T20:39:47.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.356+0000 7f6f7b76a700 1 -- start start 2023-10-10T20:39:47.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.357+0000 7f6f7b76a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f74069c70 0x7f6f74158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:47.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.357+0000 7f6f7b76a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f740af320 0x7f6f74151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:47.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.357+0000 7f6f7b76a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6f74159950 0x7f6f740cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:47.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.357+0000 7f6f7b76a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f741587d0 con 0x7f6f74069c70 2023-10-10T20:39:47.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.357+0000 7f6f7b76a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f74157c00 con 0x7f6f740af320 2023-10-10T20:39:47.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.357+0000 7f6f7b76a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f74157ea0 con 0x7f6f74159950 2023-10-10T20:39:47.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.357+0000 7f6f79506700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f74069c70 0x7f6f74158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:47.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.357+0000 7f6f79506700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f74069c70 0x7f6f74158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41960/0 (socket says 172.21.15.89:41960) 2023-10-10T20:39:47.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.357+0000 7f6f79506700 1 -- 172.21.15.89:0/2591049148 learned_addr learned my addr 172.21.15.89:0/2591049148 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:47.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.358+0000 7f6f79d07700 1 --2- 172.21.15.89:0/2591049148 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6f74159950 0x7f6f740cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:47.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.358+0000 7f6f79506700 1 -- 172.21.15.89:0/2591049148 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6f74159950 msgr2=0x7f6f740cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:47.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.358+0000 7f6f79506700 1 --2- 172.21.15.89:0/2591049148 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6f74159950 0x7f6f740cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.358+0000 7f6f78d05700 1 --2- 172.21.15.89:0/2591049148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f740af320 0x7f6f74151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:47.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.358+0000 7f6f79506700 1 -- 172.21.15.89:0/2591049148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f740af320 msgr2=0x7f6f74151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:47.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.358+0000 7f6f79506700 1 --2- 172.21.15.89:0/2591049148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f740af320 0x7f6f74151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.358+0000 7f6f79506700 1 -- 172.21.15.89:0/2591049148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f64009420 con 0x7f6f74069c70 2023-10-10T20:39:47.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.358+0000 7f6f78d05700 1 --2- 172.21.15.89:0/2591049148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f740af320 0x7f6f74151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:47.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.358+0000 7f6f79506700 1 --2- 172.21.15.89:0/2591049148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f74069c70 0x7f6f74158c60 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000c5b0 tx=0x7f6f70001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:47.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.358+0000 7f6f697fa700 1 -- 172.21.15.89:0/2591049148 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70005b30 con 0x7f6f74069c70 2023-10-10T20:39:47.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.358+0000 7f6f697fa700 1 -- 172.21.15.89:0/2591049148 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f70005c90 con 0x7f6f74069c70 2023-10-10T20:39:47.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.358+0000 7f6f697fa700 1 -- 172.21.15.89:0/2591049148 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70007300 con 0x7f6f74069c70 2023-10-10T20:39:47.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.358+0000 7f6f7b76a700 1 -- 172.21.15.89:0/2591049148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f74158180 con 0x7f6f74069c70 2023-10-10T20:39:47.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.358+0000 7f6f7b76a700 1 -- 172.21.15.89:0/2591049148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f741523d0 con 0x7f6f74069c70 2023-10-10T20:39:47.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.359+0000 7f6f7b76a700 1 -- 172.21.15.89:0/2591049148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f740718b0 con 0x7f6f74069c70 2023-10-10T20:39:47.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.359+0000 7f6f697fa700 1 -- 172.21.15.89:0/2591049148 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f6f7001b3b0 con 0x7f6f74069c70 2023-10-10T20:39:47.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.360+0000 7f6f697fa700 1 --2- 172.21.15.89:0/2591049148 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6f58067170 0x7f6f58069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:47.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.360+0000 7f6f697fa700 1 -- 172.21.15.89:0/2591049148 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6f70083da0 con 0x7f6f74069c70 2023-10-10T20:39:47.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.360+0000 7f6f78d05700 1 --2- 172.21.15.89:0/2591049148 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6f58067170 0x7f6f58069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:47.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.360+0000 7f6f78d05700 1 --2- 172.21.15.89:0/2591049148 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6f58067170 0x7f6f58069620 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6f74154ec0 tx=0x7f6f64017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:47.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.364+0000 7f6f697fa700 1 -- 172.21.15.89:0/2591049148 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f70050d80 con 0x7f6f74069c70 2023-10-10T20:39:47.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.627+0000 7f6f7b76a700 1 -- 172.21.15.89:0/2591049148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6f74158ab0 con 0x7f6f74069c70 2023-10-10T20:39:47.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.631+0000 7f6f697fa700 1 -- 172.21.15.89:0/2591049148 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f6f700170e0 con 0x7f6f74069c70 2023-10-10T20:39:47.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.633+0000 7f6f7b76a700 1 -- 172.21.15.89:0/2591049148 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6f58067170 msgr2=0x7f6f58069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:47.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.633+0000 7f6f7b76a700 1 --2- 172.21.15.89:0/2591049148 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6f58067170 0x7f6f58069620 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6f74154ec0 tx=0x7f6f64017000 comp rx=0 tx=0).stop 2023-10-10T20:39:47.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.633+0000 7f6f7b76a700 1 -- 172.21.15.89:0/2591049148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f74069c70 msgr2=0x7f6f74158c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:47.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.633+0000 7f6f7b76a700 1 --2- 172.21.15.89:0/2591049148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f74069c70 0x7f6f74158c60 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000c5b0 tx=0x7f6f70001070 comp rx=0 tx=0).stop 2023-10-10T20:39:47.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.633+0000 7f6f7b76a700 1 -- 172.21.15.89:0/2591049148 shutdown_connections 2023-10-10T20:39:47.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.633+0000 7f6f7b76a700 1 --2- 172.21.15.89:0/2591049148 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6f74159950 0x7f6f740cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.633+0000 7f6f7b76a700 1 --2- 172.21.15.89:0/2591049148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f74069c70 0x7f6f74158c60 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.633+0000 7f6f7b76a700 1 --2- 172.21.15.89:0/2591049148 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6f58067170 0x7f6f58069620 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.633+0000 7f6f7b76a700 1 --2- 172.21.15.89:0/2591049148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f740af320 0x7f6f74151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.633+0000 7f6f7b76a700 1 -- 172.21.15.89:0/2591049148 >> 172.21.15.89:0/2591049148 conn(0x7f6f74068e00 msgr2=0x7f6f74069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:47.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.633+0000 7f6f7b76a700 1 -- 172.21.15.89:0/2591049148 shutdown_connections 2023-10-10T20:39:47.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.633+0000 7f6f7b76a700 1 -- 172.21.15.89:0/2591049148 wait complete. 2023-10-10T20:39:47.638 INFO:teuthology.orchestra.run.smithi089.stderr:updated caps for client.0 2023-10-10T20:39:47.647 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:39:47.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.760+0000 7f70784fb700 1 Processor -- start 2023-10-10T20:39:47.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.760+0000 7f70784fb700 1 -- start start 2023-10-10T20:39:47.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.761+0000 7f70784fb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7070159c60 0x7f707015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:47.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.761+0000 7f70784fb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f707015db10 0x7f7070069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:47.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.762+0000 7f70784fb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707015a160 con 0x7f7070159890 2023-10-10T20:39:47.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.762+0000 7f70784fb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70700d13a0 con 0x7f707015db10 2023-10-10T20:39:47.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.762+0000 7f70784fb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707015e140 con 0x7f7070159c60 2023-10-10T20:39:47.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.762+0000 7f7076a98700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f707015db10 0x7f7070069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:47.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.762+0000 7f7076297700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7070159890 0x7f707015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48232/0 (socket says 172.21.15.89:48232) 2023-10-10T20:39:47.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.762+0000 7f7076297700 1 -- 172.21.15.89:0/1049463322 learned_addr learned my addr 172.21.15.89:0/1049463322 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:47.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.762+0000 7f7075a96700 1 --2- 172.21.15.89:0/1049463322 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7070159c60 0x7f707015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:47.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.762+0000 7f7076a98700 1 -- 172.21.15.89:0/1049463322 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7070159c60 msgr2=0x7f707015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:47.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.762+0000 7f7076a98700 1 --2- 172.21.15.89:0/1049463322 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7070159c60 0x7f707015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.762+0000 7f7076a98700 1 -- 172.21.15.89:0/1049463322 >> v1:172.21.15.89:6789/0 conn(0x7f7070159890 legacy=0x7f707015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:47.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.762+0000 7f7075295700 1 -- 172.21.15.89:0/1049463322 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622726612 0 0) 0x7f707015a160 con 0x7f7070159890 2023-10-10T20:39:47.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.762+0000 7f7076a98700 1 -- 172.21.15.89:0/1049463322 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707015a3d0 con 0x7f707015db10 2023-10-10T20:39:47.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.762+0000 7f7076a98700 1 --2- 172.21.15.89:0/1049463322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f707015db10 0x7f7070069560 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f70680098e0 tx=0x7f70680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4103960ea0d414fa server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:47.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.762+0000 7f7075295700 1 -- 172.21.15.89:0/1049463322 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068009e80 con 0x7f707015db10 2023-10-10T20:39:47.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.762+0000 7f7075295700 1 -- 172.21.15.89:0/1049463322 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706800b860 con 0x7f707015db10 2023-10-10T20:39:47.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.762+0000 7f7075295700 1 -- 172.21.15.89:0/1049463322 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706800bbe0 con 0x7f707015db10 2023-10-10T20:39:47.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.763+0000 7f70784fb700 1 -- 172.21.15.89:0/1049463322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f707015db10 msgr2=0x7f7070069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:47.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.763+0000 7f70784fb700 1 --2- 172.21.15.89:0/1049463322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f707015db10 0x7f7070069560 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f70680098e0 tx=0x7f70680091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.763+0000 7f70784fb700 1 -- 172.21.15.89:0/1049463322 shutdown_connections 2023-10-10T20:39:47.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.763+0000 7f70784fb700 1 --2- 172.21.15.89:0/1049463322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f707015db10 0x7f7070069560 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.763+0000 7f70784fb700 1 --2- 172.21.15.89:0/1049463322 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7070159c60 0x7f707015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.763+0000 7f70784fb700 1 -- 172.21.15.89:0/1049463322 >> 172.21.15.89:0/1049463322 conn(0x7f7070068e20 msgr2=0x7f70700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:47.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.763+0000 7f70784fb700 1 -- 172.21.15.89:0/1049463322 shutdown_connections 2023-10-10T20:39:47.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.763+0000 7f70784fb700 1 -- 172.21.15.89:0/1049463322 wait complete. 2023-10-10T20:39:47.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.764+0000 7f70784fb700 1 Processor -- start 2023-10-10T20:39:47.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.764+0000 7f70784fb700 1 -- start start 2023-10-10T20:39:47.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f70784fb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7070159890 0x7f70700b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:47.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f70784fb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7070159c60 0x7f70700b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:47.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f70784fb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f707015db10 0x7f70700b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:47.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f70784fb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70700cd0d0 con 0x7f7070159c60 2023-10-10T20:39:47.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f70784fb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70700cd690 con 0x7f7070159890 2023-10-10T20:39:47.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f70784fb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707016a5e0 con 0x7f707015db10 2023-10-10T20:39:47.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f7075a96700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7070159c60 0x7f70700b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:47.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f7075a96700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7070159c60 0x7f70700b1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41980/0 (socket says 172.21.15.89:41980) 2023-10-10T20:39:47.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f7075a96700 1 -- 172.21.15.89:0/2711074955 learned_addr learned my addr 172.21.15.89:0/2711074955 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:47.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f7076a98700 1 --2- 172.21.15.89:0/2711074955 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f707015db10 0x7f70700b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:47.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f7076297700 1 --2- 172.21.15.89:0/2711074955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7070159890 0x7f70700b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:47.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f7075a96700 1 -- 172.21.15.89:0/2711074955 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f707015db10 msgr2=0x7f70700b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:47.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f7075a96700 1 --2- 172.21.15.89:0/2711074955 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f707015db10 0x7f70700b20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f7075a96700 1 -- 172.21.15.89:0/2711074955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7070159890 msgr2=0x7f70700b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:47.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f7075a96700 1 --2- 172.21.15.89:0/2711074955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7070159890 0x7f70700b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:47.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f7075a96700 1 -- 172.21.15.89:0/2711074955 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7068009420 con 0x7f7070159c60 2023-10-10T20:39:47.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f7075a96700 1 --2- 172.21.15.89:0/2711074955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7070159c60 0x7f70700b1b80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f706000c880 tx=0x7f7060008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:47.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f70667fc700 1 -- 172.21.15.89:0/2711074955 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060011600 con 0x7f7070159c60 2023-10-10T20:39:47.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f70667fc700 1 -- 172.21.15.89:0/2711074955 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7060011780 con 0x7f7070159c60 2023-10-10T20:39:47.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f70667fc700 1 -- 172.21.15.89:0/2711074955 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060011b40 con 0x7f7070159c60 2023-10-10T20:39:47.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f70784fb700 1 -- 172.21.15.89:0/2711074955 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707016a8c0 con 0x7f7070159c60 2023-10-10T20:39:47.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.765+0000 7f70784fb700 1 -- 172.21.15.89:0/2711074955 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70700b4e80 con 0x7f7070159c60 2023-10-10T20:39:47.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.766+0000 7f70784fb700 1 -- 172.21.15.89:0/2711074955 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70700cd840 con 0x7f7070159c60 2023-10-10T20:39:47.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.767+0000 7f70667fc700 1 -- 172.21.15.89:0/2711074955 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f7060010350 con 0x7f7070159c60 2023-10-10T20:39:47.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.767+0000 7f70667fc700 1 --2- 172.21.15.89:0/2711074955 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7054067170 0x7f7054069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:47.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.767+0000 7f7076297700 1 --2- 172.21.15.89:0/2711074955 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7054067170 0x7f7054069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:47.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.767+0000 7f70667fc700 1 -- 172.21.15.89:0/2711074955 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f70600846c0 con 0x7f7070159c60 2023-10-10T20:39:47.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.768+0000 7f7076297700 1 --2- 172.21.15.89:0/2711074955 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7054067170 0x7f7054069620 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f706c002220 tx=0x7f706c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:47.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:47.772+0000 7f70667fc700 1 -- 172.21.15.89:0/2711074955 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70700cd840 con 0x7f7070159c60 2023-10-10T20:39:48.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.004+0000 7f70784fb700 1 -- 172.21.15.89:0/2711074955 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70700cd800 con 0x7f7070159c60 2023-10-10T20:39:48.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.004+0000 7f70667fc700 1 -- 172.21.15.89:0/2711074955 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f7060051590 con 0x7f7070159c60 2023-10-10T20:39:48.007 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:48.007 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5324,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/1607893755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":1607893755},{"type":"v1","addr":"172.21.15.111:6835","nonce":1607893755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6069,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/1480864169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":1480864169},{"type":"v1","addr":"172.21.15.89:6836","nonce":1480864169}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2347053559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2347053559},{"type":"v1","addr":"172.21.15.89:6837","nonce":2347053559}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:39:37.635170+0000","modified":"2023-10-10T20:39:41.271571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6087},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6087":{"gid":6087,"name":"b","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1070916496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1070916496},{"type":"v1","addr":"172.21.15.111:6834","nonce":1070916496}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-10T20:39:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.006+0000 7f70784fb700 1 -- 172.21.15.89:0/2711074955 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7054067170 msgr2=0x7f7054069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.006+0000 7f70784fb700 1 --2- 172.21.15.89:0/2711074955 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7054067170 0x7f7054069620 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f706c002220 tx=0x7f706c009040 comp rx=0 tx=0).stop 2023-10-10T20:39:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.006+0000 7f70784fb700 1 -- 172.21.15.89:0/2711074955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7070159c60 msgr2=0x7f70700b1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.006+0000 7f70784fb700 1 --2- 172.21.15.89:0/2711074955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7070159c60 0x7f70700b1b80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f706000c880 tx=0x7f7060008040 comp rx=0 tx=0).stop 2023-10-10T20:39:48.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.007+0000 7f70784fb700 1 -- 172.21.15.89:0/2711074955 shutdown_connections 2023-10-10T20:39:48.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.007+0000 7f70784fb700 1 --2- 172.21.15.89:0/2711074955 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f707015db10 0x7f70700b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.007+0000 7f70784fb700 1 --2- 172.21.15.89:0/2711074955 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7054067170 0x7f7054069620 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.007+0000 7f70784fb700 1 --2- 172.21.15.89:0/2711074955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7070159890 0x7f70700b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.007+0000 7f70784fb700 1 --2- 172.21.15.89:0/2711074955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7070159c60 0x7f70700b1b80 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.007+0000 7f70784fb700 1 -- 172.21.15.89:0/2711074955 >> 172.21.15.89:0/2711074955 conn(0x7f7070068e20 msgr2=0x7f70700c7130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:48.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.007+0000 7f70784fb700 1 -- 172.21.15.89:0/2711074955 shutdown_connections 2023-10-10T20:39:48.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.007+0000 7f70784fb700 1 -- 172.21.15.89:0/2711074955 wait complete. 2023-10-10T20:39:48.011 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 46 2023-10-10T20:39:48.423 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:39:37.635170+0000', 'modified': '2023-10-10T20:39:41.271571+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6087}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6087': {'gid': 6087, 'name': 'b', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6834/1070916496', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6832', 'nonce': 1070916496}, {'type': 'v1', 'addr': '172.21.15.111:6834', 'nonce': 1070916496}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:39:48.423 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:39:48.424 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-10T20:39:48.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.542+0000 7f320210f700 1 Processor -- start 2023-10-10T20:39:48.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.542+0000 7f320210f700 1 -- start start 2023-10-10T20:39:48.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.543+0000 7f320210f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f31fc159420 0x7f31fc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.543+0000 7f320210f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31fc15d230 0x7f31fc15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.543+0000 7f320210f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31fc0b2980 con 0x7f31fc1597f0 2023-10-10T20:39:48.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.543+0000 7f320210f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31fc15dbb0 con 0x7f31fc159420 2023-10-10T20:39:48.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.543+0000 7f320210f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc0b11d0 con 0x7f31fc15d230 2023-10-10T20:39:48.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.543+0000 7f320110d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f31fc159420 0x7f31fc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.543+0000 7f320090c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f31fc1597f0 0x7f31fc0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:60764/0 (socket says 172.21.15.89:60764) 2023-10-10T20:39:48.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.543+0000 7f320090c700 1 -- 172.21.15.89:0/2175552094 learned_addr learned my addr 172.21.15.89:0/2175552094 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:48.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.543+0000 7f320190e700 1 --2- 172.21.15.89:0/2175552094 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31fc15d230 0x7f31fc15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.543+0000 7f320110d700 1 -- 172.21.15.89:0/2175552094 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31fc15d230 msgr2=0x7f31fc15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:48.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.543+0000 7f320110d700 1 --2- 172.21.15.89:0/2175552094 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31fc15d230 0x7f31fc15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.543+0000 7f320110d700 1 -- 172.21.15.89:0/2175552094 >> v1:172.21.15.89:6789/0 conn(0x7f31fc1597f0 legacy=0x7f31fc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:48.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.543+0000 7f31f3fff700 1 -- 172.21.15.89:0/2175552094 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1707939333 0 0) 0x7f31fc0b2980 con 0x7f31fc1597f0 2023-10-10T20:39:48.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.543+0000 7f320110d700 1 -- 172.21.15.89:0/2175552094 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31fc0b14b0 con 0x7f31fc159420 2023-10-10T20:39:48.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.543+0000 7f320110d700 1 --2- 172.21.15.89:0/2175552094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31fc159420 0x7f31fc0b5f40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f31f80098e0 tx=0x7f31f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b05507a27cc615 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:48.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.543+0000 7f31f3fff700 1 -- 172.21.15.89:0/2175552094 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f8009e80 con 0x7f31fc159420 2023-10-10T20:39:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.544+0000 7f31f3fff700 1 -- 172.21.15.89:0/2175552094 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f800b860 con 0x7f31fc159420 2023-10-10T20:39:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.544+0000 7f31f3fff700 1 -- 172.21.15.89:0/2175552094 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f800bbe0 con 0x7f31fc159420 2023-10-10T20:39:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.544+0000 7f320210f700 1 -- 172.21.15.89:0/2175552094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31fc159420 msgr2=0x7f31fc0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.544+0000 7f320210f700 1 --2- 172.21.15.89:0/2175552094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31fc159420 0x7f31fc0b5f40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f31f80098e0 tx=0x7f31f80091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.544+0000 7f320210f700 1 -- 172.21.15.89:0/2175552094 shutdown_connections 2023-10-10T20:39:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.544+0000 7f320210f700 1 --2- 172.21.15.89:0/2175552094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31fc159420 0x7f31fc0b5f40 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.544+0000 7f320210f700 1 --2- 172.21.15.89:0/2175552094 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31fc15d230 0x7f31fc15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.544+0000 7f320210f700 1 -- 172.21.15.89:0/2175552094 >> 172.21.15.89:0/2175552094 conn(0x7f31fc0d0660 msgr2=0x7f31fc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.544+0000 7f320210f700 1 -- 172.21.15.89:0/2175552094 shutdown_connections 2023-10-10T20:39:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.544+0000 7f320210f700 1 -- 172.21.15.89:0/2175552094 wait complete. 2023-10-10T20:39:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.545+0000 7f320210f700 1 Processor -- start 2023-10-10T20:39:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.545+0000 7f320210f700 1 -- start start 2023-10-10T20:39:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.546+0000 7f320210f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31fc159420 0x7f31fc141100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.546+0000 7f320210f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31fc1597f0 0x7f31fc153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.546+0000 7f320210f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31fc15d230 0x7f31fc1539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.546+0000 7f320210f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc141640 con 0x7f31fc159420 2023-10-10T20:39:48.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.546+0000 7f320210f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc1545f0 con 0x7f31fc15d230 2023-10-10T20:39:48.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.546+0000 7f320210f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc154030 con 0x7f31fc1597f0 2023-10-10T20:39:48.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.546+0000 7f320110d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31fc159420 0x7f31fc141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.546+0000 7f320110d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31fc159420 0x7f31fc141100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48862/0 (socket says 172.21.15.89:48862) 2023-10-10T20:39:48.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.546+0000 7f320110d700 1 -- 172.21.15.89:0/1298947107 learned_addr learned my addr 172.21.15.89:0/1298947107 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:48.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.546+0000 7f320090c700 1 --2- 172.21.15.89:0/1298947107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31fc1597f0 0x7f31fc153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.546+0000 7f320110d700 1 -- 172.21.15.89:0/1298947107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31fc1597f0 msgr2=0x7f31fc153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:48.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.546+0000 7f320110d700 1 --2- 172.21.15.89:0/1298947107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31fc1597f0 0x7f31fc153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.546+0000 7f320190e700 1 --2- 172.21.15.89:0/1298947107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31fc15d230 0x7f31fc1539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.546+0000 7f320110d700 1 -- 172.21.15.89:0/1298947107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31fc15d230 msgr2=0x7f31fc1539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.546+0000 7f320110d700 1 --2- 172.21.15.89:0/1298947107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31fc15d230 0x7f31fc1539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.546+0000 7f320110d700 1 -- 172.21.15.89:0/1298947107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f8009420 con 0x7f31fc159420 2023-10-10T20:39:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.546+0000 7f320190e700 1 --2- 172.21.15.89:0/1298947107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31fc15d230 0x7f31fc1539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.547+0000 7f320110d700 1 --2- 172.21.15.89:0/1298947107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31fc159420 0x7f31fc141100 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f31f800b3c0 tx=0x7f31f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.547+0000 7f31f17fa700 1 -- 172.21.15.89:0/1298947107 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f800bbe0 con 0x7f31fc159420 2023-10-10T20:39:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.547+0000 7f31f17fa700 1 -- 172.21.15.89:0/1298947107 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f8011420 con 0x7f31fc159420 2023-10-10T20:39:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.547+0000 7f320210f700 1 -- 172.21.15.89:0/1298947107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31fc154310 con 0x7f31fc159420 2023-10-10T20:39:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.547+0000 7f31f17fa700 1 -- 172.21.15.89:0/1298947107 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f80117a0 con 0x7f31fc159420 2023-10-10T20:39:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.547+0000 7f320210f700 1 -- 172.21.15.89:0/1298947107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31fc15a0b0 con 0x7f31fc159420 2023-10-10T20:39:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.548+0000 7f31f17fa700 1 -- 172.21.15.89:0/1298947107 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f31f8011b90 con 0x7f31fc159420 2023-10-10T20:39:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.549+0000 7f31f17fa700 1 --2- 172.21.15.89:0/1298947107 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31e0067170 0x7f31e0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.549+0000 7f31f17fa700 1 -- 172.21.15.89:0/1298947107 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f31f8085d90 con 0x7f31fc159420 2023-10-10T20:39:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.549+0000 7f320090c700 1 --2- 172.21.15.89:0/1298947107 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31e0067170 0x7f31e0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.549+0000 7f320090c700 1 --2- 172.21.15.89:0/1298947107 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31e0067170 0x7f31e0069620 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f31fc15cdf0 tx=0x7f31ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.551+0000 7f320c66ab80 1 Processor -- start 2023-10-10T20:39:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.551+0000 7f320c66ab80 1 -- start start 2023-10-10T20:39:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.551+0000 7f320c66ab80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561565cad480 0x561565c8c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.551+0000 7f320c66ab80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565c605f0 0x561565cad850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.551+0000 7f320c66ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c62b80 0x561565c62ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.551+0000 7f320c66ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561565bb9a70 con 0x561565c62b80 2023-10-10T20:39:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.551+0000 7f320c66ab80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561565bb7270 con 0x561565c605f0 2023-10-10T20:39:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.551+0000 7f320c66ab80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x561565bb3060 con 0x561565cad480 2023-10-10T20:39:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.551+0000 7f320190e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c62b80 0x561565c62ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.551+0000 7f320190e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c62b80 0x561565c62ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48872/0 (socket says 172.21.15.89:48872) 2023-10-10T20:39:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.551+0000 7f320190e700 1 -- 172.21.15.89:0/4129417108 learned_addr learned my addr 172.21.15.89:0/4129417108 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.552+0000 7f320190e700 1 -- 172.21.15.89:0/4129417108 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561565cad480 msgr2=0x561565c8c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.552+0000 7f320190e700 1 --2- 172.21.15.89:0/4129417108 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561565cad480 0x561565c8c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.552+0000 7f320190e700 1 -- 172.21.15.89:0/4129417108 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565c605f0 msgr2=0x561565cad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.552+0000 7f320190e700 1 --2- 172.21.15.89:0/4129417108 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565c605f0 0x561565cad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.552+0000 7f320190e700 1 -- 172.21.15.89:0/4129417108 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561565a4d270 con 0x561565c62b80 2023-10-10T20:39:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.552+0000 7f320190e700 1 --2- 172.21.15.89:0/4129417108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c62b80 0x561565c62ff0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f31f4007370 tx=0x7f31f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0693712b59b1494 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.552+0000 7f31e67fc700 1 -- 172.21.15.89:0/4129417108 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4007e00 con 0x561565c62b80 2023-10-10T20:39:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.552+0000 7f31e67fc700 1 -- 172.21.15.89:0/4129417108 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f400f050 con 0x561565c62b80 2023-10-10T20:39:48.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.552+0000 7f31e67fc700 1 -- 172.21.15.89:0/4129417108 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f40126f0 con 0x561565c62b80 2023-10-10T20:39:48.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.552+0000 7f320c66ab80 1 -- 172.21.15.89:0/4129417108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c62b80 msgr2=0x561565c62ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:48.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.552+0000 7f320c66ab80 1 --2- 172.21.15.89:0/4129417108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c62b80 0x561565c62ff0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f31f4007370 tx=0x7f31f400b040 comp rx=0 tx=0).stop 2023-10-10T20:39:48.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.553+0000 7f320c66ab80 1 -- 172.21.15.89:0/4129417108 shutdown_connections 2023-10-10T20:39:48.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.553+0000 7f320c66ab80 1 --2- 172.21.15.89:0/4129417108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c62b80 0x561565c62ff0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.553+0000 7f320c66ab80 1 --2- 172.21.15.89:0/4129417108 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561565cad480 0x561565c8c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.553+0000 7f320c66ab80 1 --2- 172.21.15.89:0/4129417108 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565c605f0 0x561565cad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.553+0000 7f320c66ab80 1 -- 172.21.15.89:0/4129417108 >> 172.21.15.89:0/4129417108 conn(0x561565c640c0 msgr2=0x561565c648f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:48.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.553+0000 7f320c66ab80 1 -- 172.21.15.89:0/4129417108 shutdown_connections 2023-10-10T20:39:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.553+0000 7f320c66ab80 1 -- 172.21.15.89:0/4129417108 wait complete. 2023-10-10T20:39:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.553+0000 7f320c66ab80 1 -- start start 2023-10-10T20:39:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.553+0000 7f320c66ab80 1 Processor -- start 2023-10-10T20:39:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.554+0000 7f320c66ab80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565c605f0 0x561565caa2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.554+0000 7f320c66ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c62b80 0x561565c8eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.554+0000 7f320c66ab80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561565cad480 0x561565c92490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.554+0000 7f320c66ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561565b5e890 con 0x561565c62b80 2023-10-10T20:39:48.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.554+0000 7f320c66ab80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561565b79ea0 con 0x561565c605f0 2023-10-10T20:39:48.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.554+0000 7f320c66ab80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x561565a35e30 con 0x561565cad480 2023-10-10T20:39:48.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.554+0000 7f320c66ab80 10 client.0 _ll_register_callbacks cb 0x561565c262c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:39:48.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.554+0000 7f320090c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c62b80 0x561565c8eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.554+0000 7f320090c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c62b80 0x561565c8eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48874/0 (socket says 172.21.15.89:48874) 2023-10-10T20:39:48.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.554+0000 7f320090c700 1 -- 172.21.15.89:0/2635893217 learned_addr learned my addr 172.21.15.89:0/2635893217 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:48.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.554+0000 7f320110d700 1 --2- 172.21.15.89:0/2635893217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565c605f0 0x561565caa2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.554+0000 7f320190e700 1 --2- 172.21.15.89:0/2635893217 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561565cad480 0x561565c92490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.554+0000 7f320090c700 1 -- 172.21.15.89:0/2635893217 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561565cad480 msgr2=0x561565c92490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:48.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f320090c700 1 --2- 172.21.15.89:0/2635893217 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561565cad480 0x561565c92490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f320090c700 1 -- 172.21.15.89:0/2635893217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565c605f0 msgr2=0x561565caa2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:48.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f320090c700 1 --2- 172.21.15.89:0/2635893217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565c605f0 0x561565caa2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f320090c700 1 -- 172.21.15.89:0/2635893217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f4009cb0 con 0x561565c62b80 2023-10-10T20:39:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f320090c700 1 --2- 172.21.15.89:0/2635893217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c62b80 0x561565c8eaf0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f31ec007780 tx=0x7f31ec012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9075805a62a7c962 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f31db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:39:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f31db7fe700 1 -- 172.21.15.89:0/2635893217 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec0077b0 con 0x561565c62b80 2023-10-10T20:39:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f31db7fe700 1 -- 172.21.15.89:0/2635893217 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f31ec014c50 con 0x561565c62b80 2023-10-10T20:39:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f320c66ab80 1 -- 172.21.15.89:0/2635893217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561565c57cd0 con 0x561565c62b80 2023-10-10T20:39:48.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f31db7fe700 1 -- 172.21.15.89:0/2635893217 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31ec014e10 con 0x561565c62b80 2023-10-10T20:39:48.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f320c66ab80 10 client.6249 fetch_fsmap learned FSMap version 46 2023-10-10T20:39:48.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f320c66ab80 1 -- 172.21.15.89:0/2635893217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x5615659effe0 con 0x561565c62b80 2023-10-10T20:39:48.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f31db7fe700 1 -- 172.21.15.89:0/2635893217 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561565c57cd0 con 0x561565c62b80 2023-10-10T20:39:48.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f31db7fe700 1 -- 172.21.15.89:0/2635893217 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 46) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f31ec020660 con 0x561565c62b80 2023-10-10T20:39:48.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f320c66ab80 10 client.6249 fetch_fsmap finished waiting for FSMap version 46 2023-10-10T20:39:48.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f320c66ab80 10 client.6249 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-10T20:39:48.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f320c66ab80 20 client.6249 populate_metadata read hostname 'smithi089' 2023-10-10T20:39:48.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f320c66ab80 1 --2- 172.21.15.89:0/2635893217 >> [v2:172.21.15.111:6832/1070916496,v1:172.21.15.111:6834/1070916496] conn(0x561565cb9180 0x561565cb9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f320c66ab80 4 client.6249 mds_command: new command op to 6087 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:39:48.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.555+0000 7f320c66ab80 1 -- 172.21.15.89:0/2635893217 --> [v2:172.21.15.111:6832/1070916496,v1:172.21.15.111:6834/1070916496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561565c34aa0 con 0x561565cb9180 2023-10-10T20:39:48.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.556+0000 7f320110d700 1 --2- 172.21.15.89:0/2635893217 >> [v2:172.21.15.111:6832/1070916496,v1:172.21.15.111:6834/1070916496] conn(0x561565cb9180 0x561565cb9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.556+0000 7f320110d700 1 --2- 172.21.15.89:0/2635893217 >> [v2:172.21.15.111:6832/1070916496,v1:172.21.15.111:6834/1070916496] conn(0x561565cb9180 0x561565cb9550 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b373b1181325b4f server_cookie=248f6b007536d872 in_seq=0 out_seq=0 2023-10-10T20:39:48.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.556+0000 7f31db7fe700 10 client.6249 ms_handle_connect on v2:172.21.15.111:6832/1070916496 2023-10-10T20:39:48.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.557+0000 7f31db7fe700 1 -- 172.21.15.89:0/2635893217 <== mds.0 v2:172.21.15.111:6832/1070916496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x561565c34aa0 con 0x561565cb9180 2023-10-10T20:39:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.557+0000 7f31db7fe700 10 client.6249 handle_command_reply: tid=0 2023-10-10T20:39:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.557+0000 7f320c66ab80 1 client.6249 shutdown 2023-10-10T20:39:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.557+0000 7f320c66ab80 1 -- 172.21.15.89:0/2635893217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c62b80 msgr2=0x561565c8eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.557+0000 7f320c66ab80 1 --2- 172.21.15.89:0/2635893217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c62b80 0x561565c8eaf0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f31ec007780 tx=0x7f31ec012000 comp rx=0 tx=0).stop 2023-10-10T20:39:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.557+0000 7f320c66ab80 1 -- 172.21.15.89:0/2635893217 shutdown_connections 2023-10-10T20:39:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.557+0000 7f320c66ab80 1 --2- 172.21.15.89:0/2635893217 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561565cad480 0x561565c92490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.557+0000 7f320c66ab80 1 --2- 172.21.15.89:0/2635893217 >> [v2:172.21.15.111:6832/1070916496,v1:172.21.15.111:6834/1070916496] conn(0x561565cb9180 0x561565cb9550 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.557+0000 7f320c66ab80 1 --2- 172.21.15.89:0/2635893217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565c605f0 0x561565caa2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.557+0000 7f31db7fe700 0 client.6249 ms_handle_reset on v2:172.21.15.111:6832/1070916496 2023-10-10T20:39:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.557+0000 7f320c66ab80 1 --2- 172.21.15.89:0/2635893217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c62b80 0x561565c8eaf0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.557+0000 7f320c66ab80 1 -- 172.21.15.89:0/2635893217 >> 172.21.15.89:0/2635893217 conn(0x561565c640c0 msgr2=0x561565c648d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:48.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.557+0000 7f320c66ab80 1 -- 172.21.15.89:0/2635893217 shutdown_connections 2023-10-10T20:39:48.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.557+0000 7f320c66ab80 1 -- 172.21.15.89:0/2635893217 wait complete. 2023-10-10T20:39:48.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.557+0000 7f320c66ab80 20 client.6249 trim_cache size 0 max 16384 2023-10-10T20:39:48.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.574+0000 7f320c66ab80 1 Processor -- start 2023-10-10T20:39:48.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.574+0000 7f320c66ab80 1 -- start start 2023-10-10T20:39:48.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.574+0000 7f320c66ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c605f0 0x561565c86410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.574+0000 7f320c66ab80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561565c62b80 0x561565cb8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.574+0000 7f320c66ab80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565cad480 0x561565cb8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.574+0000 7f320c66ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561565c59fe0 con 0x561565c605f0 2023-10-10T20:39:48.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.574+0000 7f320c66ab80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561565c5f740 con 0x561565cad480 2023-10-10T20:39:48.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.574+0000 7f320c66ab80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x561565c65180 con 0x561565c62b80 2023-10-10T20:39:48.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.574+0000 7f320110d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c605f0 0x561565c86410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.574+0000 7f320110d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c605f0 0x561565c86410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48884/0 (socket says 172.21.15.89:48884) 2023-10-10T20:39:48.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.574+0000 7f320110d700 1 -- 172.21.15.89:0/4277308523 learned_addr learned my addr 172.21.15.89:0/4277308523 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:48.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.575+0000 7f320110d700 1 -- 172.21.15.89:0/4277308523 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561565c62b80 msgr2=0x561565cb8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:48.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.575+0000 7f320090c700 1 --2- 172.21.15.89:0/4277308523 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561565c62b80 0x561565cb8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.575+0000 7f320190e700 1 --2- 172.21.15.89:0/4277308523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565cad480 0x561565cb8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.575+0000 7f320110d700 1 --2- 172.21.15.89:0/4277308523 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561565c62b80 0x561565cb8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.575+0000 7f320110d700 1 -- 172.21.15.89:0/4277308523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565cad480 msgr2=0x561565cb8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:48.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.575+0000 7f320110d700 1 --2- 172.21.15.89:0/4277308523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565cad480 0x561565cb8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.575+0000 7f320110d700 1 -- 172.21.15.89:0/4277308523 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561565b81780 con 0x561565c605f0 2023-10-10T20:39:48.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.575+0000 7f320090c700 1 --2- 172.21.15.89:0/4277308523 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561565c62b80 0x561565cb8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:48.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.575+0000 7f320110d700 1 --2- 172.21.15.89:0/4277308523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c605f0 0x561565c86410 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f31f8006d80 tx=0x7f31f802f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd4cd0d9074f271 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:48.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.575+0000 7f31e6ffd700 1 -- 172.21.15.89:0/4277308523 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f805da50 con 0x561565c605f0 2023-10-10T20:39:48.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.575+0000 7f31e6ffd700 1 -- 172.21.15.89:0/4277308523 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f8054ca0 con 0x561565c605f0 2023-10-10T20:39:48.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.575+0000 7f31e6ffd700 1 -- 172.21.15.89:0/4277308523 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f8056050 con 0x561565c605f0 2023-10-10T20:39:48.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.575+0000 7f320c66ab80 1 -- 172.21.15.89:0/4277308523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c605f0 msgr2=0x561565c86410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:48.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.575+0000 7f320c66ab80 1 --2- 172.21.15.89:0/4277308523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c605f0 0x561565c86410 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f31f8006d80 tx=0x7f31f802f040 comp rx=0 tx=0).stop 2023-10-10T20:39:48.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.576+0000 7f320c66ab80 1 -- 172.21.15.89:0/4277308523 shutdown_connections 2023-10-10T20:39:48.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.576+0000 7f320c66ab80 1 --2- 172.21.15.89:0/4277308523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565cad480 0x561565cb8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.576+0000 7f320c66ab80 1 --2- 172.21.15.89:0/4277308523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c605f0 0x561565c86410 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.576+0000 7f320c66ab80 1 --2- 172.21.15.89:0/4277308523 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561565c62b80 0x561565cb8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.576+0000 7f320c66ab80 1 -- 172.21.15.89:0/4277308523 >> 172.21.15.89:0/4277308523 conn(0x561565c640c0 msgr2=0x561565c5fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:48.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.576+0000 7f320c66ab80 1 -- 172.21.15.89:0/4277308523 shutdown_connections 2023-10-10T20:39:48.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.576+0000 7f320c66ab80 1 -- 172.21.15.89:0/4277308523 wait complete. 2023-10-10T20:39:48.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.576+0000 7f320c66ab80 1 -- start start 2023-10-10T20:39:48.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.576+0000 7f320c66ab80 1 Processor -- start 2023-10-10T20:39:48.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.577+0000 7f320c66ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c605f0 0x561565c8d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.577+0000 7f320c66ab80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561565c62b80 0x561565cac850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.577+0000 7f320c66ab80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565cad480 0x561565d0f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.577+0000 7f320c66ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561565cc6920 con 0x561565c605f0 2023-10-10T20:39:48.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.577+0000 7f320c66ab80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561565c8d670 con 0x561565cad480 2023-10-10T20:39:48.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.577+0000 7f320c66ab80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x561565d10100 con 0x561565c62b80 2023-10-10T20:39:48.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.577+0000 7f320c66ab80 10 client.0 _ll_register_callbacks cb 0x561565c262c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:39:48.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.577+0000 7f320110d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c605f0 0x561565c8d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.577+0000 7f320110d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c605f0 0x561565c8d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48894/0 (socket says 172.21.15.89:48894) 2023-10-10T20:39:48.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.577+0000 7f320110d700 1 -- 172.21.15.89:0/3444851791 learned_addr learned my addr 172.21.15.89:0/3444851791 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:48.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.577+0000 7f320110d700 1 -- 172.21.15.89:0/3444851791 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561565c62b80 msgr2=0x561565cac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:48.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.577+0000 7f320110d700 1 --2- 172.21.15.89:0/3444851791 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561565c62b80 0x561565cac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.577+0000 7f320110d700 1 -- 172.21.15.89:0/3444851791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565cad480 msgr2=0x561565d0f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:48.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.577+0000 7f320110d700 1 --2- 172.21.15.89:0/3444851791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565cad480 0x561565d0f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.577+0000 7f320110d700 1 -- 172.21.15.89:0/3444851791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31fc15a0b0 con 0x561565c605f0 2023-10-10T20:39:48.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.578+0000 7f320110d700 1 --2- 172.21.15.89:0/3444851791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c605f0 0x561565c8d130 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f31f8056d20 tx=0x7f31f8059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c711ac2b963f49b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:48.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.578+0000 7f31db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:39:48.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.578+0000 7f31db7fe700 1 -- 172.21.15.89:0/3444851791 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f805da50 con 0x561565c605f0 2023-10-10T20:39:48.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.578+0000 7f31db7fe700 1 -- 172.21.15.89:0/3444851791 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f31f8011e40 con 0x561565c605f0 2023-10-10T20:39:48.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.578+0000 7f320c66ab80 1 -- 172.21.15.89:0/3444851791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31f400f050 con 0x561565c605f0 2023-10-10T20:39:48.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.578+0000 7f31db7fe700 1 -- 172.21.15.89:0/3444851791 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31f805d7d0 con 0x561565c605f0 2023-10-10T20:39:48.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.578+0000 7f320c66ab80 10 client.6255 fetch_fsmap learned FSMap version 46 2023-10-10T20:39:48.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.578+0000 7f320c66ab80 1 -- 172.21.15.89:0/3444851791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x561565b68470 con 0x561565c605f0 2023-10-10T20:39:48.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.578+0000 7f31db7fe700 1 -- 172.21.15.89:0/3444851791 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f400f050 con 0x561565c605f0 2023-10-10T20:39:48.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.578+0000 7f31db7fe700 1 -- 172.21.15.89:0/3444851791 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 46) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f31f8084130 con 0x561565c605f0 2023-10-10T20:39:48.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.578+0000 7f320c66ab80 10 client.6255 fetch_fsmap finished waiting for FSMap version 46 2023-10-10T20:39:48.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.578+0000 7f320c66ab80 10 client.6255 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-10T20:39:48.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.578+0000 7f320c66ab80 20 client.6255 populate_metadata read hostname 'smithi089' 2023-10-10T20:39:48.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.578+0000 7f320c66ab80 1 --2- 172.21.15.89:0/3444851791 >> [v2:172.21.15.111:6832/1070916496,v1:172.21.15.111:6834/1070916496] conn(0x561565d0f950 0x561565c876c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.578+0000 7f320c66ab80 4 client.6255 mds_command: new command op to 6087 tid=0[{"prefix": "status"}] 2023-10-10T20:39:48.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.578+0000 7f320c66ab80 1 -- 172.21.15.89:0/3444851791 --> [v2:172.21.15.111:6832/1070916496,v1:172.21.15.111:6834/1070916496] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561565b55e00 con 0x561565d0f950 2023-10-10T20:39:48.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.579+0000 7f320090c700 1 --2- 172.21.15.89:0/3444851791 >> [v2:172.21.15.111:6832/1070916496,v1:172.21.15.111:6834/1070916496] conn(0x561565d0f950 0x561565c876c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.579+0000 7f31db7fe700 10 client.6255 ms_handle_connect on v2:172.21.15.111:6832/1070916496 2023-10-10T20:39:48.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.579+0000 7f320090c700 1 --2- 172.21.15.89:0/3444851791 >> [v2:172.21.15.111:6832/1070916496,v1:172.21.15.111:6834/1070916496] conn(0x561565d0f950 0x561565c876c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96326300b33c86e6 server_cookie=b164d6c9e1133e3 in_seq=0 out_seq=0 2023-10-10T20:39:48.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.579+0000 7f31db7fe700 1 -- 172.21.15.89:0/3444851791 <== mds.0 v2:172.21.15.111:6832/1070916496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x561565b55e00 con 0x561565d0f950 2023-10-10T20:39:48.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.579+0000 7f31db7fe700 10 client.6255 handle_command_reply: tid=0 2023-10-10T20:39:48.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.579+0000 7f320c66ab80 1 client.6255 shutdown 2023-10-10T20:39:48.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.579+0000 7f320c66ab80 1 -- 172.21.15.89:0/3444851791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c605f0 msgr2=0x561565c8d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:48.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.579+0000 7f320c66ab80 1 --2- 172.21.15.89:0/3444851791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c605f0 0x561565c8d130 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f31f8056d20 tx=0x7f31f8059000 comp rx=0 tx=0).stop 2023-10-10T20:39:48.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.579+0000 7f320c66ab80 1 -- 172.21.15.89:0/3444851791 shutdown_connections 2023-10-10T20:39:48.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.579+0000 7f320c66ab80 1 --2- 172.21.15.89:0/3444851791 >> [v2:172.21.15.111:6832/1070916496,v1:172.21.15.111:6834/1070916496] conn(0x561565d0f950 0x561565c876c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.579+0000 7f320c66ab80 1 --2- 172.21.15.89:0/3444851791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565cad480 0x561565d0f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.579+0000 7f31db7fe700 0 client.6255 ms_handle_reset on v2:172.21.15.111:6832/1070916496 2023-10-10T20:39:48.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.579+0000 7f320c66ab80 1 --2- 172.21.15.89:0/3444851791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561565c605f0 0x561565c8d130 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.579+0000 7f320c66ab80 1 --2- 172.21.15.89:0/3444851791 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x561565c62b80 0x561565cac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.579+0000 7f320c66ab80 1 -- 172.21.15.89:0/3444851791 >> 172.21.15.89:0/3444851791 conn(0x561565c640c0 msgr2=0x561565cb8570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:48.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.579+0000 7f320c66ab80 1 -- 172.21.15.89:0/3444851791 shutdown_connections 2023-10-10T20:39:48.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.579+0000 7f320c66ab80 1 -- 172.21.15.89:0/3444851791 wait complete. 2023-10-10T20:39:48.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.580+0000 7f320c66ab80 20 client.6255 trim_cache size 0 max 16384 2023-10-10T20:39:48.608 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:39:48.609 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "5dd675b8-9026-4d8a-a1bf-62fdc5e7de67", 2023-10-10T20:39:48.609 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-10-10T20:39:48.609 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 6087, 2023-10-10T20:39:48.609 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-10-10T20:39:48.609 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-10-10T20:39:48.610 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-10-10T20:39:48.610 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 10.329044948, 2023-10-10T20:39:48.610 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 46, 2023-10-10T20:39:48.610 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 76, 2023-10-10T20:39:48.610 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 76, 2023-10-10T20:39:48.611 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 15.429981266 2023-10-10T20:39:48.611 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:39:48.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.580+0000 7f320210f700 1 -- 172.21.15.89:0/1298947107 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31e0067170 msgr2=0x7f31e0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:48.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.580+0000 7f320210f700 1 --2- 172.21.15.89:0/1298947107 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31e0067170 0x7f31e0069620 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f31fc15cdf0 tx=0x7f31ec009040 comp rx=0 tx=0).stop 2023-10-10T20:39:48.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.580+0000 7f320210f700 1 -- 172.21.15.89:0/1298947107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31fc159420 msgr2=0x7f31fc141100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:48.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.580+0000 7f320210f700 1 --2- 172.21.15.89:0/1298947107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31fc159420 0x7f31fc141100 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f31f800b3c0 tx=0x7f31f8017000 comp rx=0 tx=0).stop 2023-10-10T20:39:48.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.580+0000 7f320210f700 1 -- 172.21.15.89:0/1298947107 shutdown_connections 2023-10-10T20:39:48.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.580+0000 7f320210f700 1 --2- 172.21.15.89:0/1298947107 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31e0067170 0x7f31e0069620 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.580+0000 7f320210f700 1 --2- 172.21.15.89:0/1298947107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31fc15d230 0x7f31fc1539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.580+0000 7f320210f700 1 --2- 172.21.15.89:0/1298947107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31fc159420 0x7f31fc141100 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.580+0000 7f320210f700 1 --2- 172.21.15.89:0/1298947107 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31fc1597f0 0x7f31fc153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.580+0000 7f320210f700 1 -- 172.21.15.89:0/1298947107 >> 172.21.15.89:0/1298947107 conn(0x7f31fc0d0660 msgr2=0x7f31fc0ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:48.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.580+0000 7f320210f700 1 -- 172.21.15.89:0/1298947107 shutdown_connections 2023-10-10T20:39:48.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.580+0000 7f320210f700 1 -- 172.21.15.89:0/1298947107 wait complete. 2023-10-10T20:39:48.615 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:39:48.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.728+0000 7fab72981700 1 Processor -- start 2023-10-10T20:39:48.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.728+0000 7fab72981700 1 -- start start 2023-10-10T20:39:48.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.729+0000 7fab72981700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fab6c159890 0x7fab6c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.729+0000 7fab72981700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab6c069560 0x7fab6c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.729+0000 7fab72981700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6c069ee0 con 0x7fab6c159c60 2023-10-10T20:39:48.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.729+0000 7fab72981700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6c15a160 con 0x7fab6c159890 2023-10-10T20:39:48.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.729+0000 7fab72981700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0d13a0 con 0x7fab6c069560 2023-10-10T20:39:48.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.729+0000 7fab6bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fab6c159890 0x7fab6c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.729+0000 7fab6bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fab6c159890 0x7fab6c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48896/0 (socket says 172.21.15.89:48896) 2023-10-10T20:39:48.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.729+0000 7fab6bfff700 1 -- 172.21.15.89:0/3695742137 learned_addr learned my addr 172.21.15.89:0/3695742137 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:48.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.730+0000 7fab70f1e700 1 --2- 172.21.15.89:0/3695742137 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab6c069560 0x7fab6c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.730+0000 7fab6bfff700 1 -- 172.21.15.89:0/3695742137 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab6c069560 msgr2=0x7fab6c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:48.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.730+0000 7fab6bfff700 1 --2- 172.21.15.89:0/3695742137 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab6c069560 0x7fab6c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.730+0000 7fab6bfff700 1 -- 172.21.15.89:0/3695742137 >> v1:172.21.15.111:6789/0 conn(0x7fab6c159c60 legacy=0x7fab6c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:48.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.730+0000 7fab6bfff700 1 -- 172.21.15.89:0/3695742137 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c15e140 con 0x7fab6c159890 2023-10-10T20:39:48.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.730+0000 7fab70f1e700 1 --2- 172.21.15.89:0/3695742137 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab6c069560 0x7fab6c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:48.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.730+0000 7fab6bfff700 1 --2- 172.21.15.89:0/3695742137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab6c159890 0x7fab6c15abd0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0099b0 tx=0x7fab5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3627eba1151e3a63 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:48.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.730+0000 7fab6affd700 1 -- 172.21.15.89:0/3695742137 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c00b860 con 0x7fab6c159890 2023-10-10T20:39:48.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.730+0000 7fab6affd700 1 -- 172.21.15.89:0/3695742137 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c00b9c0 con 0x7fab6c159890 2023-10-10T20:39:48.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.730+0000 7fab6affd700 1 -- 172.21.15.89:0/3695742137 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c00bd60 con 0x7fab6c159890 2023-10-10T20:39:48.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.730+0000 7fab72981700 1 -- 172.21.15.89:0/3695742137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab6c159890 msgr2=0x7fab6c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:48.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.730+0000 7fab72981700 1 --2- 172.21.15.89:0/3695742137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab6c159890 0x7fab6c15abd0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0099b0 tx=0x7fab5c009250 comp rx=0 tx=0).stop 2023-10-10T20:39:48.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.730+0000 7fab72981700 1 -- 172.21.15.89:0/3695742137 shutdown_connections 2023-10-10T20:39:48.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.730+0000 7fab72981700 1 --2- 172.21.15.89:0/3695742137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab6c159890 0x7fab6c15abd0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.730+0000 7fab72981700 1 --2- 172.21.15.89:0/3695742137 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab6c069560 0x7fab6c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.731+0000 7fab72981700 1 -- 172.21.15.89:0/3695742137 >> 172.21.15.89:0/3695742137 conn(0x7fab6c068e20 msgr2=0x7fab6c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:48.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.731+0000 7fab72981700 1 -- 172.21.15.89:0/3695742137 shutdown_connections 2023-10-10T20:39:48.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.731+0000 7fab72981700 1 -- 172.21.15.89:0/3695742137 wait complete. 2023-10-10T20:39:48.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.732+0000 7fab72981700 1 Processor -- start 2023-10-10T20:39:48.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.732+0000 7fab72981700 1 -- start start 2023-10-10T20:39:48.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.732+0000 7fab72981700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab6c069560 0x7fab6c0c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.732+0000 7fab72981700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab6c159890 0x7fab6c0b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.732+0000 7fab72981700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab6c159c60 0x7fab6c148500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.732+0000 7fab72981700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0c8b00 con 0x7fab6c159c60 2023-10-10T20:39:48.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.732+0000 7fab72981700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0c7e80 con 0x7fab6c159890 2023-10-10T20:39:48.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.732+0000 7fab72981700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0c8160 con 0x7fab6c069560 2023-10-10T20:39:48.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.733+0000 7fab70f1e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab6c159c60 0x7fab6c148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.733+0000 7fab70f1e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab6c159c60 0x7fab6c148500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48904/0 (socket says 172.21.15.89:48904) 2023-10-10T20:39:48.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.733+0000 7fab70f1e700 1 -- 172.21.15.89:0/1334859537 learned_addr learned my addr 172.21.15.89:0/1334859537 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:48.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.733+0000 7fab6bfff700 1 --2- 172.21.15.89:0/1334859537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab6c069560 0x7fab6c0c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.733+0000 7fab6b7fe700 1 --2- 172.21.15.89:0/1334859537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab6c159890 0x7fab6c0b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.733+0000 7fab70f1e700 1 -- 172.21.15.89:0/1334859537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab6c069560 msgr2=0x7fab6c0c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:48.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.733+0000 7fab70f1e700 1 --2- 172.21.15.89:0/1334859537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab6c069560 0x7fab6c0c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.733+0000 7fab70f1e700 1 -- 172.21.15.89:0/1334859537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab6c159890 msgr2=0x7fab6c0b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:48.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.733+0000 7fab70f1e700 1 --2- 172.21.15.89:0/1334859537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab6c159890 0x7fab6c0b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.733+0000 7fab70f1e700 1 -- 172.21.15.89:0/1334859537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c0094f0 con 0x7fab6c159c60 2023-10-10T20:39:48.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.733+0000 7fab70f1e700 1 --2- 172.21.15.89:0/1334859537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab6c159c60 0x7fab6c148500 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fab6400cef0 tx=0x7fab64007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:48.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.733+0000 7fab53fff700 1 -- 172.21.15.89:0/1334859537 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6400ad20 con 0x7fab6c159c60 2023-10-10T20:39:48.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.733+0000 7fab53fff700 1 -- 172.21.15.89:0/1334859537 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab64016420 con 0x7fab6c159c60 2023-10-10T20:39:48.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.733+0000 7fab72981700 1 -- 172.21.15.89:0/1334859537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6c0c8440 con 0x7fab6c159c60 2023-10-10T20:39:48.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.733+0000 7fab53fff700 1 -- 172.21.15.89:0/1334859537 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6400aea0 con 0x7fab6c159c60 2023-10-10T20:39:48.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.733+0000 7fab72981700 1 -- 172.21.15.89:0/1334859537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c148a40 con 0x7fab6c159c60 2023-10-10T20:39:48.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.734+0000 7fab72981700 1 -- 172.21.15.89:0/1334859537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6c0c8c40 con 0x7fab6c159c60 2023-10-10T20:39:48.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.734+0000 7fab53fff700 1 -- 172.21.15.89:0/1334859537 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fab64016a70 con 0x7fab6c159c60 2023-10-10T20:39:48.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.735+0000 7fab53fff700 1 --2- 172.21.15.89:0/1334859537 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab58067170 0x7fab58069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:48.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.735+0000 7fab53fff700 1 -- 172.21.15.89:0/1334859537 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fab64012070 con 0x7fab6c159c60 2023-10-10T20:39:48.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.735+0000 7fab6bfff700 1 --2- 172.21.15.89:0/1334859537 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab58067170 0x7fab58069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:48.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.735+0000 7fab6bfff700 1 --2- 172.21.15.89:0/1334859537 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab58067170 0x7fab58069620 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0099b0 tx=0x7fab5c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:48.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.739+0000 7fab53fff700 1 -- 172.21.15.89:0/1334859537 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab6404e5a0 con 0x7fab6c159c60 2023-10-10T20:39:48.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.965+0000 7fab72981700 1 -- 172.21.15.89:0/1334859537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fab6c04c3c0 con 0x7fab6c159c60 2023-10-10T20:39:48.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.965+0000 7fab53fff700 1 -- 172.21.15.89:0/1334859537 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) v1 ==== 53+0+73 (secure 0 0 0) 0x7fab6c0c8c40 con 0x7fab6c159c60 2023-10-10T20:39:48.968 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:39:48.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.967+0000 7fab72981700 1 -- 172.21.15.89:0/1334859537 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab58067170 msgr2=0x7fab58069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:48.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.967+0000 7fab72981700 1 --2- 172.21.15.89:0/1334859537 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab58067170 0x7fab58069620 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0099b0 tx=0x7fab5c017000 comp rx=0 tx=0).stop 2023-10-10T20:39:48.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.967+0000 7fab72981700 1 -- 172.21.15.89:0/1334859537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab6c159c60 msgr2=0x7fab6c148500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:48.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.967+0000 7fab72981700 1 --2- 172.21.15.89:0/1334859537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab6c159c60 0x7fab6c148500 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fab6400cef0 tx=0x7fab64007860 comp rx=0 tx=0).stop 2023-10-10T20:39:48.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.967+0000 7fab72981700 1 -- 172.21.15.89:0/1334859537 shutdown_connections 2023-10-10T20:39:48.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.967+0000 7fab72981700 1 --2- 172.21.15.89:0/1334859537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab6c159c60 0x7fab6c148500 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.967+0000 7fab72981700 1 --2- 172.21.15.89:0/1334859537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fab6c069560 0x7fab6c0c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.967+0000 7fab72981700 1 --2- 172.21.15.89:0/1334859537 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fab58067170 0x7fab58069620 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.967+0000 7fab72981700 1 --2- 172.21.15.89:0/1334859537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab6c159890 0x7fab6c0b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:48.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.967+0000 7fab72981700 1 -- 172.21.15.89:0/1334859537 >> 172.21.15.89:0/1334859537 conn(0x7fab6c068e20 msgr2=0x7fab6c154760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:48.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.967+0000 7fab72981700 1 -- 172.21.15.89:0/1334859537 shutdown_connections 2023-10-10T20:39:48.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:48.967+0000 7fab72981700 1 -- 172.21.15.89:0/1334859537 wait complete. 2023-10-10T20:39:48.981 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:39:48.982 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:39:48.982 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-10T20:39:48.985 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:39:48.986 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:39:48.986 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:39:48.986 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:39:48.987 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:39:48.987 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:39:48.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:39:48.988 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-10T20:39:49.003 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:39:49.004 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:39:49.004 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:39:49.004 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:39:49.004 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-10T20:39:49.005 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:39:49.005 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:39:49.005 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:39:49.005 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-10T20:39:49.005 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:39:49.006 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:39:49.006 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-10T20:39:49.006 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:39:49.006 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:39:49.006 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:39:49.007 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:39:49.007 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:39:49.007 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42177sec preferred_lft 42177sec 2023-10-10T20:39:49.007 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2023-10-10T20:39:49.008 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:39:49.008 INFO:teuthology.orchestra.run.smithi089.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:39:49.008 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:39:49.008 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:39:49.008 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:39:49.008 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::98a3:d4ff:fec1:83bb/64 scope link 2023-10-10T20:39:49.009 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:39:49.009 INFO:teuthology.orchestra.run.smithi089.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:39:49.009 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether ee:16:22:10:ca:9a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:39:49.009 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec16:22ff:fe10:ca9a/64 scope link 2023-10-10T20:39:49.009 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:39:49.010 INFO:teuthology.orchestra.run.smithi089.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:39:49.010 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:39:49.010 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::6864:2bff:fe5c:71d4/64 scope link 2023-10-10T20:39:49.010 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:39:49.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:39:49.011 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:39:49.066 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-10T20:39:49.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:39:49.068 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-10T20:39:49.123 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-10T20:39:49.124 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:39:49.124 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:39:49.124 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:39:49.124 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:39:49.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:39:49.125 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:39:49.126 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:39:49.126 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:39:49.126 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:39:49.127 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:39:49.127 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:39:49.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:39:49.224 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-10T20:39:49.239 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:39:49.239 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:39:49.240 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:39:49.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:39:49.240 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-10-10T20:39:49.303 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:39:49.303 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:39:49.359 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:39:49.359 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:39:49.360 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:39:49.413 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:39:49.414 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:39:49.414 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:39:49.494 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:39:49.494 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:39:49.494 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:39:49.502 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:39:49.503 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:39:49.503 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:39:49.503 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:39:49.503 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:39:49.525 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:39:49.526 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:39:49.526 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:39:49.526 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:39:49.526 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:39:49.527 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:39:49.527 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:39:49.527 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:39:49.528 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:39:49.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:39:49.529 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:39:49.594 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:39:49.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.746+0000 7fb6a257a700 1 Processor -- start 2023-10-10T20:39:49.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.746+0000 7fb6a257a700 1 -- start start 2023-10-10T20:39:49.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.746+0000 7fb6a257a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb69c0b5ea0 0x7fb69c068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:49.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.746+0000 7fb6a257a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb69c068bb0 0x7fb69c15c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:49.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.746+0000 7fb6a257a700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb69c15c480 con 0x7fb69c159c10 2023-10-10T20:39:49.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.746+0000 7fb6a257a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69c0b2a40 con 0x7fb69c068bb0 2023-10-10T20:39:49.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.746+0000 7fb6a257a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c15b870 con 0x7fb69c0b5ea0 2023-10-10T20:39:49.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.747+0000 7fb6a0b17700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb69c068bb0 0x7fb69c15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:49.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.747+0000 7fb69b7fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb69c0b5ea0 0x7fb69c068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:49.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.747+0000 7fb6a0b17700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb69c068bb0 0x7fb69c15c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:33940/0 (socket says 172.21.15.89:33940) 2023-10-10T20:39:49.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.747+0000 7fb6a0b17700 1 -- 172.21.15.89:0/2093406123 learned_addr learned my addr 172.21.15.89:0/2093406123 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:49.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.747+0000 7fb69b7fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb69c0b5ea0 0x7fb69c068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:37456/0 (socket says 172.21.15.89:37456) 2023-10-10T20:39:49.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.747+0000 7fb69bfff700 1 --1- 172.21.15.89:0/2093406123 >> v1:172.21.15.111:6789/0 conn(0x7fb69c159c10 0x7fb69c15b090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:43924/0 (socket says 172.21.15.89:43924) 2023-10-10T20:39:49.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.747+0000 7fb69b7fe700 1 -- 172.21.15.89:0/2093406123 >> v2:172.21.15.111:3300/0 conn(0x7fb69c068bb0 msgr2=0x7fb69c15c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:49.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.747+0000 7fb69b7fe700 1 --2- 172.21.15.89:0/2093406123 >> v2:172.21.15.111:3300/0 conn(0x7fb69c068bb0 0x7fb69c15c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:49.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.747+0000 7fb69b7fe700 1 -- 172.21.15.89:0/2093406123 >> v1:172.21.15.111:6789/0 conn(0x7fb69c159c10 legacy=0x7fb69c15b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:49.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.747+0000 7fb69b7fe700 1 -- 172.21.15.89:0/2093406123 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c15bb50 con 0x7fb69c0b5ea0 2023-10-10T20:39:49.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.747+0000 7fb69affd700 1 -- 172.21.15.89:0/2093406123 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3484365350 0 0) 0x7fb69c15c480 con 0x7fb69c159c10 2023-10-10T20:39:49.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.747+0000 7fb6a0b17700 1 --2- 172.21.15.89:0/2093406123 >> v2:172.21.15.111:3300/0 conn(0x7fb69c068bb0 0x7fb69c15c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:39:49.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.748+0000 7fb69b7fe700 1 --2- 172.21.15.89:0/2093406123 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb69c0b5ea0 0x7fb69c068670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009910 tx=0x7fb68c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbbbe4598a2d2fd7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:49.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.748+0000 7fb69affd700 1 -- 172.21.15.89:0/2093406123 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c00b640 con 0x7fb69c0b5ea0 2023-10-10T20:39:49.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.748+0000 7fb69affd700 1 -- 172.21.15.89:0/2093406123 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68c00b7a0 con 0x7fb69c0b5ea0 2023-10-10T20:39:49.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.748+0000 7fb69affd700 1 -- 172.21.15.89:0/2093406123 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c00bb60 con 0x7fb69c0b5ea0 2023-10-10T20:39:49.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.748+0000 7fb6a257a700 1 -- 172.21.15.89:0/2093406123 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb69c0b5ea0 msgr2=0x7fb69c068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:49.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.748+0000 7fb6a257a700 1 --2- 172.21.15.89:0/2093406123 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb69c0b5ea0 0x7fb69c068670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009910 tx=0x7fb68c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:49.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.748+0000 7fb6a257a700 1 -- 172.21.15.89:0/2093406123 shutdown_connections 2023-10-10T20:39:49.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.748+0000 7fb6a257a700 1 --2- 172.21.15.89:0/2093406123 >> v2:172.21.15.111:3300/0 conn(0x7fb69c068bb0 0x7fb69c15c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:49.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.748+0000 7fb6a257a700 1 --2- 172.21.15.89:0/2093406123 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb69c0b5ea0 0x7fb69c068670 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:49.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.748+0000 7fb6a257a700 1 -- 172.21.15.89:0/2093406123 >> 172.21.15.89:0/2093406123 conn(0x7fb69c099a20 msgr2=0x7fb69c0af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:49.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.748+0000 7fb6a257a700 1 -- 172.21.15.89:0/2093406123 shutdown_connections 2023-10-10T20:39:49.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.748+0000 7fb6a257a700 1 -- 172.21.15.89:0/2093406123 wait complete. 2023-10-10T20:39:49.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.749+0000 7fb6a257a700 1 Processor -- start 2023-10-10T20:39:49.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.750+0000 7fb6a257a700 1 -- start start 2023-10-10T20:39:49.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.750+0000 7fb6a257a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb69c068bb0 0x7fb69c0c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:49.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.750+0000 7fb6a257a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb69c0b5ea0 0x7fb69c15fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:49.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.750+0000 7fb6a257a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb69c159c10 0x7fb69c147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:49.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.750+0000 7fb6a257a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c0c5a60 con 0x7fb69c159c10 2023-10-10T20:39:49.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.750+0000 7fb6a257a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c0b0450 con 0x7fb69c068bb0 2023-10-10T20:39:49.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.750+0000 7fb6a257a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c0af840 con 0x7fb69c0b5ea0 2023-10-10T20:39:49.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.750+0000 7fb6a0b17700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb69c159c10 0x7fb69c147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:49.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.750+0000 7fb6a0b17700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb69c159c10 0x7fb69c147e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48906/0 (socket says 172.21.15.89:48906) 2023-10-10T20:39:49.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.750+0000 7fb6a0b17700 1 -- 172.21.15.89:0/3446190172 learned_addr learned my addr 172.21.15.89:0/3446190172 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:49.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.751+0000 7fb69b7fe700 1 --2- 172.21.15.89:0/3446190172 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb69c0b5ea0 0x7fb69c15fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:49.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.751+0000 7fb69bfff700 1 --2- 172.21.15.89:0/3446190172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb69c068bb0 0x7fb69c0c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:49.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.751+0000 7fb6a0b17700 1 -- 172.21.15.89:0/3446190172 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb69c0b5ea0 msgr2=0x7fb69c15fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:49.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.751+0000 7fb6a0b17700 1 --2- 172.21.15.89:0/3446190172 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb69c0b5ea0 0x7fb69c15fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:49.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.751+0000 7fb6a0b17700 1 -- 172.21.15.89:0/3446190172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb69c068bb0 msgr2=0x7fb69c0c5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:49.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.751+0000 7fb6a0b17700 1 --2- 172.21.15.89:0/3446190172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb69c068bb0 0x7fb69c0c5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:49.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.751+0000 7fb6a0b17700 1 -- 172.21.15.89:0/3446190172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb694009ae0 con 0x7fb69c159c10 2023-10-10T20:39:49.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.751+0000 7fb69bfff700 1 --2- 172.21.15.89:0/3446190172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb69c068bb0 0x7fb69c0c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:49.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.751+0000 7fb6a0b17700 1 --2- 172.21.15.89:0/3446190172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb69c159c10 0x7fb69c147e00 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb694009fd0 tx=0x7fb69400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:49.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.751+0000 7fb683fff700 1 -- 172.21.15.89:0/3446190172 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb694018070 con 0x7fb69c159c10 2023-10-10T20:39:49.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.751+0000 7fb683fff700 1 -- 172.21.15.89:0/3446190172 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69400f040 con 0x7fb69c159c10 2023-10-10T20:39:49.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.751+0000 7fb6a257a700 1 -- 172.21.15.89:0/3446190172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68c009420 con 0x7fb69c159c10 2023-10-10T20:39:49.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.751+0000 7fb683fff700 1 -- 172.21.15.89:0/3446190172 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69400cd30 con 0x7fb69c159c10 2023-10-10T20:39:49.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.751+0000 7fb6a257a700 1 -- 172.21.15.89:0/3446190172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69c0afe00 con 0x7fb69c159c10 2023-10-10T20:39:49.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.752+0000 7fb6a257a700 1 -- 172.21.15.89:0/3446190172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69c0b66a0 con 0x7fb69c159c10 2023-10-10T20:39:49.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.753+0000 7fb683fff700 1 -- 172.21.15.89:0/3446190172 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fb694003020 con 0x7fb69c159c10 2023-10-10T20:39:49.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.753+0000 7fb683fff700 1 --2- 172.21.15.89:0/3446190172 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb688067170 0x7fb688069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:49.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.753+0000 7fb683fff700 1 -- 172.21.15.89:0/3446190172 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb694084990 con 0x7fb69c159c10 2023-10-10T20:39:49.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.753+0000 7fb69bfff700 1 --2- 172.21.15.89:0/3446190172 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb688067170 0x7fb688069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:49.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.753+0000 7fb69bfff700 1 --2- 172.21.15.89:0/3446190172 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb688067170 0x7fb688069620 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb690007fd0 tx=0x7fb690007e30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:49.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.757+0000 7fb683fff700 1 -- 172.21.15.89:0/3446190172 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb69404d5a0 con 0x7fb69c159c10 2023-10-10T20:39:49.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.982+0000 7fb6a257a700 1 -- 172.21.15.89:0/3446190172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb69c0b2d90 con 0x7fb69c159c10 2023-10-10T20:39:49.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.982+0000 7fb683fff700 1 -- 172.21.15.89:0/3446190172 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb694051860 con 0x7fb69c159c10 2023-10-10T20:39:49.985 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:39:49.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.984+0000 7fb6a257a700 1 -- 172.21.15.89:0/3446190172 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb688067170 msgr2=0x7fb688069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:49.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.984+0000 7fb6a257a700 1 --2- 172.21.15.89:0/3446190172 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb688067170 0x7fb688069620 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb690007fd0 tx=0x7fb690007e30 comp rx=0 tx=0).stop 2023-10-10T20:39:49.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.984+0000 7fb6a257a700 1 -- 172.21.15.89:0/3446190172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb69c159c10 msgr2=0x7fb69c147e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:49.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.984+0000 7fb6a257a700 1 --2- 172.21.15.89:0/3446190172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb69c159c10 0x7fb69c147e00 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb694009fd0 tx=0x7fb69400b040 comp rx=0 tx=0).stop 2023-10-10T20:39:49.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.984+0000 7fb6a257a700 1 -- 172.21.15.89:0/3446190172 shutdown_connections 2023-10-10T20:39:49.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.984+0000 7fb6a257a700 1 --2- 172.21.15.89:0/3446190172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb69c159c10 0x7fb69c147e00 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:49.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.984+0000 7fb6a257a700 1 --2- 172.21.15.89:0/3446190172 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb688067170 0x7fb688069620 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:49.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.984+0000 7fb6a257a700 1 --2- 172.21.15.89:0/3446190172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb69c068bb0 0x7fb69c0c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:49.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.984+0000 7fb6a257a700 1 --2- 172.21.15.89:0/3446190172 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb69c0b5ea0 0x7fb69c15fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:49.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.984+0000 7fb6a257a700 1 -- 172.21.15.89:0/3446190172 >> 172.21.15.89:0/3446190172 conn(0x7fb69c099a20 msgr2=0x7fb69c09ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:49.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.984+0000 7fb6a257a700 1 -- 172.21.15.89:0/3446190172 shutdown_connections 2023-10-10T20:39:49.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:49.985+0000 7fb6a257a700 1 -- 172.21.15.89:0/3446190172 wait complete. 2023-10-10T20:39:49.999 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:39:49.999 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:39:49.999 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-10-10T20:39:50.000 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:39:50.000 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:39:50.000 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:39:50.001 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:39:50.001 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:39:50.001 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:39:50.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:39:50.002 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-10-10T20:39:50.024 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:39:50.024 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:39:50.024 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:39:50.025 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:39:50.025 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-10-10T20:39:50.025 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:39:50.025 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:39:50.026 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:39:50.026 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-10-10T20:39:50.026 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:39:50.026 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:39:50.026 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-10-10T20:39:50.027 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:39:50.027 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:39:50.027 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:39:50.027 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-10-10T20:39:50.028 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:39:50.028 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42192sec preferred_lft 42192sec 2023-10-10T20:39:50.028 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-10-10T20:39:50.028 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:39:50.028 INFO:teuthology.orchestra.run.smithi111.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:39:50.029 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:39:50.029 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:39:50.029 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:39:50.029 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::a486:a6ff:fe5f:6021/64 scope link 2023-10-10T20:39:50.029 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:39:50.030 INFO:teuthology.orchestra.run.smithi111.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:39:50.030 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 1e:d0:81:1b:e2:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:39:50.030 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::1cd0:81ff:fe1b:e26b/64 scope link 2023-10-10T20:39:50.030 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:39:50.030 INFO:teuthology.orchestra.run.smithi111.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:39:50.031 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:39:50.031 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::88d:cdff:fe75:455e/64 scope link 2023-10-10T20:39:50.031 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:39:50.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:39:50.032 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:39:50.091 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-10-10T20:39:50.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:39:50.093 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-10-10T20:39:50.152 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-10-10T20:39:50.152 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:39:50.152 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:39:50.153 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:39:50.153 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:39:50.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:39:50.154 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:39:50.154 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:39:50.155 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:39:50.155 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:39:50.155 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:39:50.156 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:39:50.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:39:50.259 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-10-10T20:39:50.276 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:39:50.276 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:39:50.276 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:39:50.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:39:50.277 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-10-10T20:39:50.341 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:39:50.341 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:39:50.396 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:39:50.397 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:39:50.398 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:39:50.451 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:39:50.452 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:39:50.452 DEBUG:teuthology.orchestra.run.smithi111:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:39:50.530 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:39:50.531 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:39:50.531 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:39:50.539 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:39:50.539 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:39:50.539 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:39:50.539 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:39:50.540 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:39:50.561 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:39:50.562 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:39:50.562 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:39:50.562 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:39:50.563 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:39:50.563 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:39:50.563 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:39:50.563 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:39:50.564 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:39:50.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:39:50.565 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:39:50.630 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-10T20:39:50.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.753+0000 7f80e3c87700 1 Processor -- start 2023-10-10T20:39:50.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.753+0000 7f80e3c87700 1 -- start start 2023-10-10T20:39:50.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.754+0000 7f80e3c87700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f80dc0cd910 0x7f80dc0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:50.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.754+0000 7f80e3c87700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f80dc0b55a0 0x7f80dc0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:50.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.754+0000 7f80e3c87700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80dc0b5b00 con 0x7f80dc159ec0 2023-10-10T20:39:50.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.754+0000 7f80e3c87700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80dc0b5da0 con 0x7f80dc0b55a0 2023-10-10T20:39:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.754+0000 7f80e3c87700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc0b6080 con 0x7f80dc0cd910 2023-10-10T20:39:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.754+0000 7f80e2224700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f80dc0b55a0 0x7f80dc0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.754+0000 7f80e2224700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f80dc0b55a0 0x7f80dc0ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48922/0 (socket says 172.21.15.89:48922) 2023-10-10T20:39:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.754+0000 7f80e2224700 1 -- 172.21.15.89:0/396246552 learned_addr learned my addr 172.21.15.89:0/396246552 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.754+0000 7f80e1222700 1 --2- 172.21.15.89:0/396246552 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f80dc0cd910 0x7f80dc0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:50.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.755+0000 7f80e2224700 1 -- 172.21.15.89:0/396246552 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f80dc0cd910 msgr2=0x7f80dc0ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:50.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.755+0000 7f80e2224700 1 --2- 172.21.15.89:0/396246552 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f80dc0cd910 0x7f80dc0ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:50.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.755+0000 7f80e2224700 1 -- 172.21.15.89:0/396246552 >> v1:172.21.15.111:6789/0 conn(0x7f80dc159ec0 legacy=0x7f80dc0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:50.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.755+0000 7f80e2224700 1 -- 172.21.15.89:0/396246552 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80dc0cc100 con 0x7f80dc0b55a0 2023-10-10T20:39:50.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.755+0000 7f80e2224700 1 --2- 172.21.15.89:0/396246552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80dc0b55a0 0x7f80dc0ce880 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f80d40099b0 tx=0x7f80d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988c545ee7bf687b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.755+0000 7f80e0a21700 1 -- 172.21.15.89:0/396246552 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d400b820 con 0x7f80dc0b55a0 2023-10-10T20:39:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.755+0000 7f80e0a21700 1 -- 172.21.15.89:0/396246552 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d400b980 con 0x7f80dc0b55a0 2023-10-10T20:39:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.755+0000 7f80e0a21700 1 -- 172.21.15.89:0/396246552 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d400bd20 con 0x7f80dc0b55a0 2023-10-10T20:39:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.755+0000 7f80e3c87700 1 -- 172.21.15.89:0/396246552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80dc0b55a0 msgr2=0x7f80dc0ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.755+0000 7f80e3c87700 1 --2- 172.21.15.89:0/396246552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80dc0b55a0 0x7f80dc0ce880 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f80d40099b0 tx=0x7f80d4009250 comp rx=0 tx=0).stop 2023-10-10T20:39:50.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.755+0000 7f80e3c87700 1 -- 172.21.15.89:0/396246552 shutdown_connections 2023-10-10T20:39:50.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.755+0000 7f80e3c87700 1 --2- 172.21.15.89:0/396246552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80dc0b55a0 0x7f80dc0ce880 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:50.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.755+0000 7f80e3c87700 1 --2- 172.21.15.89:0/396246552 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f80dc0cd910 0x7f80dc0ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:50.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.755+0000 7f80e3c87700 1 -- 172.21.15.89:0/396246552 >> 172.21.15.89:0/396246552 conn(0x7f80dc0999f0 msgr2=0x7f80dc0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:50.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.756+0000 7f80e3c87700 1 -- 172.21.15.89:0/396246552 shutdown_connections 2023-10-10T20:39:50.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.756+0000 7f80e3c87700 1 -- 172.21.15.89:0/396246552 wait complete. 2023-10-10T20:39:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.757+0000 7f80e3c87700 1 Processor -- start 2023-10-10T20:39:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.757+0000 7f80e3c87700 1 -- start start 2023-10-10T20:39:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.757+0000 7f80e3c87700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80dc0b55a0 0x7f80dc149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:50.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.757+0000 7f80e3c87700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f80dc159ec0 0x7f80dc09dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:50.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.757+0000 7f80e3c87700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80dc16ade0 0x7f80dc09ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:50.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.757+0000 7f80e3c87700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc149890 con 0x7f80dc0b55a0 2023-10-10T20:39:50.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.757+0000 7f80e3c87700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc09cf80 con 0x7f80dc16ade0 2023-10-10T20:39:50.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.757+0000 7f80e3c87700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc09d220 con 0x7f80dc159ec0 2023-10-10T20:39:50.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.757+0000 7f80e1a23700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80dc0b55a0 0x7f80dc149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:50.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.757+0000 7f80e1a23700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80dc0b55a0 0x7f80dc149350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48930/0 (socket says 172.21.15.89:48930) 2023-10-10T20:39:50.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.757+0000 7f80e1a23700 1 -- 172.21.15.89:0/1653050891 learned_addr learned my addr 172.21.15.89:0/1653050891 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:50.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.757+0000 7f80e1222700 1 --2- 172.21.15.89:0/1653050891 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f80dc159ec0 0x7f80dc09dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:50.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.757+0000 7f80e2224700 1 --2- 172.21.15.89:0/1653050891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80dc16ade0 0x7f80dc09ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.758+0000 7f80e1a23700 1 -- 172.21.15.89:0/1653050891 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f80dc159ec0 msgr2=0x7f80dc09dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.758+0000 7f80e1a23700 1 --2- 172.21.15.89:0/1653050891 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f80dc159ec0 0x7f80dc09dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.758+0000 7f80e1a23700 1 -- 172.21.15.89:0/1653050891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80dc16ade0 msgr2=0x7f80dc09ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.758+0000 7f80e1a23700 1 --2- 172.21.15.89:0/1653050891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80dc16ade0 0x7f80dc09ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.758+0000 7f80e1a23700 1 -- 172.21.15.89:0/1653050891 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d40094f0 con 0x7f80dc0b55a0 2023-10-10T20:39:50.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.758+0000 7f80e1a23700 1 --2- 172.21.15.89:0/1653050891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80dc0b55a0 0x7f80dc149350 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f80d800cc90 tx=0x7f80d8002af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:50.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.758+0000 7f80d1ffb700 1 -- 172.21.15.89:0/1653050891 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d800a8b0 con 0x7f80dc0b55a0 2023-10-10T20:39:50.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.758+0000 7f80d1ffb700 1 -- 172.21.15.89:0/1653050891 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d800aa10 con 0x7f80dc0b55a0 2023-10-10T20:39:50.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.758+0000 7f80e3c87700 1 -- 172.21.15.89:0/1653050891 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80dc09bae0 con 0x7f80dc0b55a0 2023-10-10T20:39:50.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.758+0000 7f80d1ffb700 1 -- 172.21.15.89:0/1653050891 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d800add0 con 0x7f80dc0b55a0 2023-10-10T20:39:50.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.758+0000 7f80e3c87700 1 -- 172.21.15.89:0/1653050891 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80dc09d540 con 0x7f80dc0b55a0 2023-10-10T20:39:50.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.759+0000 7f80e3c87700 1 -- 172.21.15.89:0/1653050891 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80dc063c90 con 0x7f80dc0b55a0 2023-10-10T20:39:50.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.759+0000 7f80d1ffb700 1 -- 172.21.15.89:0/1653050891 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f80d8012630 con 0x7f80dc0b55a0 2023-10-10T20:39:50.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.760+0000 7f80d1ffb700 1 --2- 172.21.15.89:0/1653050891 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f80c0067170 0x7f80c0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:50.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.760+0000 7f80d1ffb700 1 -- 172.21.15.89:0/1653050891 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f80d8084ce0 con 0x7f80dc0b55a0 2023-10-10T20:39:50.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.760+0000 7f80e1222700 1 --2- 172.21.15.89:0/1653050891 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f80c0067170 0x7f80c0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:50.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.760+0000 7f80e1222700 1 --2- 172.21.15.89:0/1653050891 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f80c0067170 0x7f80c0069620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f80dc09cd70 tx=0x7f80cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:50.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.764+0000 7f80d1ffb700 1 -- 172.21.15.89:0/1653050891 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80d8051cc0 con 0x7f80dc0b55a0 2023-10-10T20:39:50.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:50.991+0000 7f80e3c87700 1 -- 172.21.15.89:0/1653050891 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f80dc0718b0 con 0x7f80dc0b55a0 2023-10-10T20:39:51.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:51.950+0000 7f80d1ffb700 1 -- 172.21.15.89:0/1653050891 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v47) v1 ==== 136+0+0 (secure 0 0 0) 0x7f80d80552e0 con 0x7f80dc0b55a0 2023-10-10T20:39:51.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:51.953+0000 7f80e3c87700 1 -- 172.21.15.89:0/1653050891 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f80c0067170 msgr2=0x7f80c0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:51.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:51.953+0000 7f80e3c87700 1 --2- 172.21.15.89:0/1653050891 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f80c0067170 0x7f80c0069620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f80dc09cd70 tx=0x7f80cc008040 comp rx=0 tx=0).stop 2023-10-10T20:39:51.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:51.953+0000 7f80e3c87700 1 -- 172.21.15.89:0/1653050891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80dc0b55a0 msgr2=0x7f80dc149350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:51.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:51.953+0000 7f80e3c87700 1 --2- 172.21.15.89:0/1653050891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80dc0b55a0 0x7f80dc149350 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f80d800cc90 tx=0x7f80d8002af0 comp rx=0 tx=0).stop 2023-10-10T20:39:51.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:51.953+0000 7f80e3c87700 1 -- 172.21.15.89:0/1653050891 shutdown_connections 2023-10-10T20:39:51.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:51.953+0000 7f80e3c87700 1 --2- 172.21.15.89:0/1653050891 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f80c0067170 0x7f80c0069620 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:51.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:51.953+0000 7f80e3c87700 1 --2- 172.21.15.89:0/1653050891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80dc16ade0 0x7f80dc09ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:51.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:51.953+0000 7f80e3c87700 1 --2- 172.21.15.89:0/1653050891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80dc0b55a0 0x7f80dc149350 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:51.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:51.953+0000 7f80e3c87700 1 --2- 172.21.15.89:0/1653050891 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f80dc159ec0 0x7f80dc09dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:51.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:51.953+0000 7f80e3c87700 1 -- 172.21.15.89:0/1653050891 >> 172.21.15.89:0/1653050891 conn(0x7f80dc0999f0 msgr2=0x7f80dc158fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:51.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:51.953+0000 7f80e3c87700 1 -- 172.21.15.89:0/1653050891 shutdown_connections 2023-10-10T20:39:51.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:51.953+0000 7f80e3c87700 1 -- 172.21.15.89:0/1653050891 wait complete. 2023-10-10T20:39:51.968 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-10T20:39:52.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.083+0000 7fac733b5700 1 Processor -- start 2023-10-10T20:39:52.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.083+0000 7fac733b5700 1 -- start start 2023-10-10T20:39:52.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.084+0000 7fac733b5700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fac6c159890 0x7fac6c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.084+0000 7fac733b5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fac6c159c60 0x7fac6c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.084+0000 7fac733b5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac6c15db10 0x7fac6c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.084+0000 7fac733b5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac6c15df50 con 0x7fac6c159c60 2023-10-10T20:39:52.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.084+0000 7fac733b5700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac6c15a160 con 0x7fac6c159890 2023-10-10T20:39:52.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.084+0000 7fac733b5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c0d13a0 con 0x7fac6c15db10 2023-10-10T20:39:52.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.084+0000 7fac70950700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fac6c159c60 0x7fac6c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:52.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.084+0000 7fac70950700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fac6c159c60 0x7fac6c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48936/0 (socket says 172.21.15.89:48936) 2023-10-10T20:39:52.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.084+0000 7fac70950700 1 -- 172.21.15.89:0/52399171 learned_addr learned my addr 172.21.15.89:0/52399171 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:52.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.084+0000 7fac71151700 1 --2- 172.21.15.89:0/52399171 >> v2:172.21.15.111:3300/0 conn(0x7fac6c159890 0x7fac6c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:52.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.084+0000 7fac71952700 1 --2- 172.21.15.89:0/52399171 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac6c15db10 0x7fac6c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:52.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.085+0000 7fac70950700 1 -- 172.21.15.89:0/52399171 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac6c15db10 msgr2=0x7fac6c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:52.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.085+0000 7fac70950700 1 --2- 172.21.15.89:0/52399171 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac6c15db10 0x7fac6c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.085+0000 7fac70950700 1 -- 172.21.15.89:0/52399171 >> v2:172.21.15.111:3300/0 conn(0x7fac6c159890 msgr2=0x7fac6c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.085+0000 7fac70950700 1 --2- 172.21.15.89:0/52399171 >> v2:172.21.15.111:3300/0 conn(0x7fac6c159890 0x7fac6c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.085+0000 7fac70950700 1 -- 172.21.15.89:0/52399171 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c15e140 con 0x7fac6c159c60 2023-10-10T20:39:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.085+0000 7fac71151700 1 --2- 172.21.15.89:0/52399171 >> v2:172.21.15.111:3300/0 conn(0x7fac6c159890 0x7fac6c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:52.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.085+0000 7fac70950700 1 --2- 172.21.15.89:0/52399171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac6c159c60 0x7fac6c15d5d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0d1090 tx=0x7fac5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e898f6a38275543 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:52.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.085+0000 7fac63fff700 1 -- 172.21.15.89:0/52399171 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c00b860 con 0x7fac6c159c60 2023-10-10T20:39:52.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.085+0000 7fac63fff700 1 -- 172.21.15.89:0/52399171 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5c00b9c0 con 0x7fac6c159c60 2023-10-10T20:39:52.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.085+0000 7fac63fff700 1 -- 172.21.15.89:0/52399171 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c00bd60 con 0x7fac6c159c60 2023-10-10T20:39:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.085+0000 7fac733b5700 1 -- 172.21.15.89:0/52399171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac6c159c60 msgr2=0x7fac6c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.085+0000 7fac733b5700 1 --2- 172.21.15.89:0/52399171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac6c159c60 0x7fac6c15d5d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0d1090 tx=0x7fac5c009250 comp rx=0 tx=0).stop 2023-10-10T20:39:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.085+0000 7fac733b5700 1 -- 172.21.15.89:0/52399171 shutdown_connections 2023-10-10T20:39:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.085+0000 7fac733b5700 1 --2- 172.21.15.89:0/52399171 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac6c15db10 0x7fac6c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.085+0000 7fac733b5700 1 --2- 172.21.15.89:0/52399171 >> v2:172.21.15.111:3300/0 conn(0x7fac6c159890 0x7fac6c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.085+0000 7fac733b5700 1 --2- 172.21.15.89:0/52399171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac6c159c60 0x7fac6c15d5d0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.085+0000 7fac733b5700 1 -- 172.21.15.89:0/52399171 >> 172.21.15.89:0/52399171 conn(0x7fac6c068e20 msgr2=0x7fac6c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.086+0000 7fac733b5700 1 -- 172.21.15.89:0/52399171 shutdown_connections 2023-10-10T20:39:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.086+0000 7fac733b5700 1 -- 172.21.15.89:0/52399171 wait complete. 2023-10-10T20:39:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.087+0000 7fac733b5700 1 Processor -- start 2023-10-10T20:39:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.087+0000 7fac733b5700 1 -- start start 2023-10-10T20:39:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac733b5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac6c159890 0x7fac6c14c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac733b5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac6c159c60 0x7fac6c0b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac733b5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac6c15db10 0x7fac6c15c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac733b5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c16abe0 con 0x7fac6c159c60 2023-10-10T20:39:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac733b5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c16a2b0 con 0x7fac6c15db10 2023-10-10T20:39:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac733b5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c16a590 con 0x7fac6c159890 2023-10-10T20:39:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac70950700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac6c159c60 0x7fac6c0b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac70950700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac6c159c60 0x7fac6c0b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48946/0 (socket says 172.21.15.89:48946) 2023-10-10T20:39:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac70950700 1 -- 172.21.15.89:0/1949342069 learned_addr learned my addr 172.21.15.89:0/1949342069 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac71151700 1 --2- 172.21.15.89:0/1949342069 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac6c159890 0x7fac6c14c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac71952700 1 --2- 172.21.15.89:0/1949342069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac6c15db10 0x7fac6c15c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac70950700 1 -- 172.21.15.89:0/1949342069 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac6c159890 msgr2=0x7fac6c14c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac70950700 1 --2- 172.21.15.89:0/1949342069 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac6c159890 0x7fac6c14c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac70950700 1 -- 172.21.15.89:0/1949342069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac6c15db10 msgr2=0x7fac6c15c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac70950700 1 --2- 172.21.15.89:0/1949342069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac6c15db10 0x7fac6c15c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac70950700 1 -- 172.21.15.89:0/1949342069 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac5c0094f0 con 0x7fac6c159c60 2023-10-10T20:39:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac70950700 1 --2- 172.21.15.89:0/1949342069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac6c159c60 0x7fac6c0b2050 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009290 tx=0x7fac5c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac617fa700 1 -- 172.21.15.89:0/1949342069 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c00bcd0 con 0x7fac6c159c60 2023-10-10T20:39:52.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac617fa700 1 -- 172.21.15.89:0/1949342069 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5c01f070 con 0x7fac6c159c60 2023-10-10T20:39:52.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac733b5700 1 -- 172.21.15.89:0/1949342069 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac6c16a870 con 0x7fac6c159c60 2023-10-10T20:39:52.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac617fa700 1 -- 172.21.15.89:0/1949342069 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c01a890 con 0x7fac6c159c60 2023-10-10T20:39:52.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.088+0000 7fac733b5700 1 -- 172.21.15.89:0/1949342069 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6c0cd250 con 0x7fac6c159c60 2023-10-10T20:39:52.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.089+0000 7fac733b5700 1 -- 172.21.15.89:0/1949342069 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6c16af70 con 0x7fac6c159c60 2023-10-10T20:39:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.090+0000 7fac617fa700 1 -- 172.21.15.89:0/1949342069 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fac5c01ac00 con 0x7fac6c159c60 2023-10-10T20:39:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.090+0000 7fac617fa700 1 --2- 172.21.15.89:0/1949342069 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fac50067170 0x7fac50069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.090+0000 7fac617fa700 1 -- 172.21.15.89:0/1949342069 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fac5c021070 con 0x7fac6c159c60 2023-10-10T20:39:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.090+0000 7fac71151700 1 --2- 172.21.15.89:0/1949342069 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fac50067170 0x7fac50069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.090+0000 7fac71151700 1 --2- 172.21.15.89:0/1949342069 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fac50067170 0x7fac50069620 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fac68007460 tx=0x7fac6800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.094+0000 7fac617fa700 1 -- 172.21.15.89:0/1949342069 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac6c16af70 con 0x7fac6c159c60 2023-10-10T20:39:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.307+0000 7fac733b5700 1 -- 172.21.15.89:0/1949342069 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fac6c16af70 con 0x7fac50067170 2023-10-10T20:39:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.309+0000 7fac617fa700 1 -- 172.21.15.89:0/1949342069 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fac6c16af70 con 0x7fac50067170 2023-10-10T20:39:52.311 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:39:52.311 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:52.311 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs" 2023-10-10T20:39:52.311 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:52.312 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:39:52.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.311+0000 7fac733b5700 1 -- 172.21.15.89:0/1949342069 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fac50067170 msgr2=0x7fac50069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:52.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.311+0000 7fac733b5700 1 --2- 172.21.15.89:0/1949342069 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fac50067170 0x7fac50069620 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fac68007460 tx=0x7fac6800b040 comp rx=0 tx=0).stop 2023-10-10T20:39:52.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.311+0000 7fac733b5700 1 -- 172.21.15.89:0/1949342069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac6c159c60 msgr2=0x7fac6c0b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:52.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.311+0000 7fac733b5700 1 --2- 172.21.15.89:0/1949342069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac6c159c60 0x7fac6c0b2050 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009290 tx=0x7fac5c017000 comp rx=0 tx=0).stop 2023-10-10T20:39:52.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.311+0000 7fac733b5700 1 -- 172.21.15.89:0/1949342069 shutdown_connections 2023-10-10T20:39:52.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.311+0000 7fac733b5700 1 --2- 172.21.15.89:0/1949342069 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fac50067170 0x7fac50069620 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.311+0000 7fac733b5700 1 --2- 172.21.15.89:0/1949342069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac6c15db10 0x7fac6c15c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.311+0000 7fac733b5700 1 --2- 172.21.15.89:0/1949342069 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac6c159890 0x7fac6c14c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.311+0000 7fac733b5700 1 --2- 172.21.15.89:0/1949342069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac6c159c60 0x7fac6c0b2050 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.311+0000 7fac733b5700 1 -- 172.21.15.89:0/1949342069 >> 172.21.15.89:0/1949342069 conn(0x7fac6c068e20 msgr2=0x7fac6c154f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:52.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.311+0000 7fac733b5700 1 -- 172.21.15.89:0/1949342069 shutdown_connections 2023-10-10T20:39:52.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.311+0000 7fac733b5700 1 -- 172.21.15.89:0/1949342069 wait complete. 2023-10-10T20:39:52.325 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:39:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.440+0000 7fb8c3740700 1 Processor -- start 2023-10-10T20:39:52.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.440+0000 7fb8c3740700 1 -- start start 2023-10-10T20:39:52.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.441+0000 7fb8c3740700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb8bc06bcd0 0x7fb8bc0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.441+0000 7fb8c3740700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb8bc06c0a0 0x7fb8bc15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.441+0000 7fb8c3740700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8bc15df50 0x7fb8bc15e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.441+0000 7fb8c3740700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8bc15d7e0 con 0x7fb8bc06bcd0 2023-10-10T20:39:52.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.441+0000 7fb8c3740700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8bc15dac0 con 0x7fb8bc06c0a0 2023-10-10T20:39:52.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.441+0000 7fb8c3740700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc15aa30 con 0x7fb8bc15df50 2023-10-10T20:39:52.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.441+0000 7fb8c14dc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb8bc06bcd0 0x7fb8bc0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:52.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.441+0000 7fb8c14dc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb8bc06bcd0 0x7fb8bc0caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48948/0 (socket says 172.21.15.89:48948) 2023-10-10T20:39:52.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.441+0000 7fb8c14dc700 1 -- 172.21.15.89:0/2284451681 learned_addr learned my addr 172.21.15.89:0/2284451681 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:52.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.442+0000 7fb8c14dc700 1 -- 172.21.15.89:0/2284451681 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8bc15df50 msgr2=0x7fb8bc15e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:52.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.442+0000 7fb8c14dc700 1 --2- 172.21.15.89:0/2284451681 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8bc15df50 0x7fb8bc15e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.442+0000 7fb8c14dc700 1 -- 172.21.15.89:0/2284451681 >> v2:172.21.15.111:3300/0 conn(0x7fb8bc06c0a0 msgr2=0x7fb8bc15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:52.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.442+0000 7fb8c14dc700 1 --2- 172.21.15.89:0/2284451681 >> v2:172.21.15.111:3300/0 conn(0x7fb8bc06c0a0 0x7fb8bc15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.442+0000 7fb8c14dc700 1 -- 172.21.15.89:0/2284451681 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8bc15ad10 con 0x7fb8bc06bcd0 2023-10-10T20:39:52.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.442+0000 7fb8c14dc700 1 --2- 172.21.15.89:0/2284451681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8bc06bcd0 0x7fb8bc0caa90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc06b4e0 tx=0x7fb8b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b63058b768e238 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:52.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.442+0000 7fb8b3fff700 1 -- 172.21.15.89:0/2284451681 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b800e070 con 0x7fb8bc06bcd0 2023-10-10T20:39:52.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.442+0000 7fb8b3fff700 1 -- 172.21.15.89:0/2284451681 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b800ba50 con 0x7fb8bc06bcd0 2023-10-10T20:39:52.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.442+0000 7fb8b3fff700 1 -- 172.21.15.89:0/2284451681 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b80092e0 con 0x7fb8bc06bcd0 2023-10-10T20:39:52.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.442+0000 7fb8c3740700 1 -- 172.21.15.89:0/2284451681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8bc06bcd0 msgr2=0x7fb8bc0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:52.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.442+0000 7fb8c3740700 1 --2- 172.21.15.89:0/2284451681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8bc06bcd0 0x7fb8bc0caa90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc06b4e0 tx=0x7fb8b8009840 comp rx=0 tx=0).stop 2023-10-10T20:39:52.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.442+0000 7fb8c3740700 1 -- 172.21.15.89:0/2284451681 shutdown_connections 2023-10-10T20:39:52.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.442+0000 7fb8c3740700 1 --2- 172.21.15.89:0/2284451681 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8bc15df50 0x7fb8bc15e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.442+0000 7fb8c3740700 1 --2- 172.21.15.89:0/2284451681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8bc06bcd0 0x7fb8bc0caa90 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.442+0000 7fb8c3740700 1 --2- 172.21.15.89:0/2284451681 >> v2:172.21.15.111:3300/0 conn(0x7fb8bc06c0a0 0x7fb8bc15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.442+0000 7fb8c3740700 1 -- 172.21.15.89:0/2284451681 >> 172.21.15.89:0/2284451681 conn(0x7fb8bc0ce600 msgr2=0x7fb8bc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:52.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.443+0000 7fb8c3740700 1 -- 172.21.15.89:0/2284451681 shutdown_connections 2023-10-10T20:39:52.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.443+0000 7fb8c3740700 1 -- 172.21.15.89:0/2284451681 wait complete. 2023-10-10T20:39:52.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.444+0000 7fb8c3740700 1 Processor -- start 2023-10-10T20:39:52.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.444+0000 7fb8c3740700 1 -- start start 2023-10-10T20:39:52.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.444+0000 7fb8c3740700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8bc06bcd0 0x7fb8bc14a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.444+0000 7fb8c3740700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8bc06c0a0 0x7fb8bc15c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.444+0000 7fb8c3740700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8bc15df50 0x7fb8bc16d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.444+0000 7fb8c3740700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc0c6080 con 0x7fb8bc06bcd0 2023-10-10T20:39:52.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8c3740700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc0c62d0 con 0x7fb8bc06c0a0 2023-10-10T20:39:52.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8c3740700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc0c65b0 con 0x7fb8bc15df50 2023-10-10T20:39:52.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8c14dc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8bc06bcd0 0x7fb8bc14a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:52.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8c14dc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8bc06bcd0 0x7fb8bc14a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48956/0 (socket says 172.21.15.89:48956) 2023-10-10T20:39:52.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8c14dc700 1 -- 172.21.15.89:0/3780180450 learned_addr learned my addr 172.21.15.89:0/3780180450 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:52.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8c0cdb700 1 --2- 172.21.15.89:0/3780180450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8bc06c0a0 0x7fb8bc15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:52.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8c1cdd700 1 --2- 172.21.15.89:0/3780180450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8bc15df50 0x7fb8bc16d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:52.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8c14dc700 1 -- 172.21.15.89:0/3780180450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8bc15df50 msgr2=0x7fb8bc16d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:52.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8c14dc700 1 --2- 172.21.15.89:0/3780180450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8bc15df50 0x7fb8bc16d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8c14dc700 1 -- 172.21.15.89:0/3780180450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8bc06c0a0 msgr2=0x7fb8bc15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:52.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8c14dc700 1 --2- 172.21.15.89:0/3780180450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8bc06c0a0 0x7fb8bc15c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8c14dc700 1 -- 172.21.15.89:0/3780180450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b8009ae0 con 0x7fb8bc06bcd0 2023-10-10T20:39:52.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8c0cdb700 1 --2- 172.21.15.89:0/3780180450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8bc06c0a0 0x7fb8bc15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:52.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8c1cdd700 1 --2- 172.21.15.89:0/3780180450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8bc15df50 0x7fb8bc16d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:52.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8c14dc700 1 --2- 172.21.15.89:0/3780180450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8bc06bcd0 0x7fb8bc14a2f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009fd0 tx=0x7fb8b8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:52.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8b17fa700 1 -- 172.21.15.89:0/3780180450 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8007bd0 con 0x7fb8bc06bcd0 2023-10-10T20:39:52.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8b17fa700 1 -- 172.21.15.89:0/3780180450 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b8007d30 con 0x7fb8bc06bcd0 2023-10-10T20:39:52.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8c3740700 1 -- 172.21.15.89:0/3780180450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8bc15bf60 con 0x7fb8bc06bcd0 2023-10-10T20:39:52.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8b17fa700 1 -- 172.21.15.89:0/3780180450 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8007ea0 con 0x7fb8bc06bcd0 2023-10-10T20:39:52.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.445+0000 7fb8c3740700 1 -- 172.21.15.89:0/3780180450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8bc0c6890 con 0x7fb8bc06bcd0 2023-10-10T20:39:52.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.446+0000 7fb8c3740700 1 -- 172.21.15.89:0/3780180450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8bc04c3c0 con 0x7fb8bc06bcd0 2023-10-10T20:39:52.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.447+0000 7fb8b17fa700 1 -- 172.21.15.89:0/3780180450 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fb8b801d420 con 0x7fb8bc06bcd0 2023-10-10T20:39:52.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.447+0000 7fb8b17fa700 1 --2- 172.21.15.89:0/3780180450 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb8a0067170 0x7fb8a0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.447+0000 7fb8b17fa700 1 -- 172.21.15.89:0/3780180450 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb8b8086790 con 0x7fb8bc06bcd0 2023-10-10T20:39:52.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.447+0000 7fb8c0cdb700 1 --2- 172.21.15.89:0/3780180450 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb8a0067170 0x7fb8a0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:52.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.447+0000 7fb8c0cdb700 1 --2- 172.21.15.89:0/3780180450 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb8a0067170 0x7fb8a0069620 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc06e070 tx=0x7fb8ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:52.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.451+0000 7fb8b17fa700 1 -- 172.21.15.89:0/3780180450 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8b8053660 con 0x7fb8bc06bcd0 2023-10-10T20:39:52.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.635+0000 7fb8c3740700 1 -- 172.21.15.89:0/3780180450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb8bc15aff0 con 0x7fb8bc06bcd0 2023-10-10T20:39:52.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.635+0000 7fb8b17fa700 1 -- 172.21.15.89:0/3780180450 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb8b8056c80 con 0x7fb8bc06bcd0 2023-10-10T20:39:52.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.637+0000 7fb8c3740700 1 -- 172.21.15.89:0/3780180450 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb8a0067170 msgr2=0x7fb8a0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:52.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.637+0000 7fb8c3740700 1 --2- 172.21.15.89:0/3780180450 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb8a0067170 0x7fb8a0069620 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc06e070 tx=0x7fb8ac008040 comp rx=0 tx=0).stop 2023-10-10T20:39:52.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.637+0000 7fb8c3740700 1 -- 172.21.15.89:0/3780180450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8bc06bcd0 msgr2=0x7fb8bc14a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:52.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.637+0000 7fb8c3740700 1 --2- 172.21.15.89:0/3780180450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8bc06bcd0 0x7fb8bc14a2f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009fd0 tx=0x7fb8b8018000 comp rx=0 tx=0).stop 2023-10-10T20:39:52.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.637+0000 7fb8c3740700 1 -- 172.21.15.89:0/3780180450 shutdown_connections 2023-10-10T20:39:52.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.637+0000 7fb8c3740700 1 --2- 172.21.15.89:0/3780180450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8bc15df50 0x7fb8bc16d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.637+0000 7fb8c3740700 1 --2- 172.21.15.89:0/3780180450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8bc06bcd0 0x7fb8bc14a2f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.637+0000 7fb8c3740700 1 --2- 172.21.15.89:0/3780180450 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb8a0067170 0x7fb8a0069620 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.637+0000 7fb8c3740700 1 --2- 172.21.15.89:0/3780180450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8bc06c0a0 0x7fb8bc15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.637+0000 7fb8c3740700 1 -- 172.21.15.89:0/3780180450 >> 172.21.15.89:0/3780180450 conn(0x7fb8bc0ce600 msgr2=0x7fb8bc155f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:52.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.637+0000 7fb8c3740700 1 -- 172.21.15.89:0/3780180450 shutdown_connections 2023-10-10T20:39:52.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.637+0000 7fb8c3740700 1 -- 172.21.15.89:0/3780180450 wait complete. 2023-10-10T20:39:52.651 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000078545 2023-10-10T20:39:52.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.768+0000 7f9011b27700 1 Processor -- start 2023-10-10T20:39:52.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.768+0000 7f9011b27700 1 -- start start 2023-10-10T20:39:52.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.769+0000 7f9011b27700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f900c0b4e70 0x7f900c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.769+0000 7f9011b27700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f900c0b1b40 0x7f900c15d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.769+0000 7f9011b27700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f900c0cfb00 con 0x7f900c0b16d0 2023-10-10T20:39:52.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.769+0000 7f9011b27700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900c0b5c40 con 0x7f900c0b4e70 2023-10-10T20:39:52.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.769+0000 7f9011b27700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0b5f20 con 0x7f900c0b1b40 2023-10-10T20:39:52.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.769+0000 7f9010b25700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f900c0b4e70 0x7f900c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:52.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.769+0000 7f9010b25700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f900c0b4e70 0x7f900c0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48972/0 (socket says 172.21.15.89:48972) 2023-10-10T20:39:52.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.769+0000 7f9010b25700 1 -- 172.21.15.89:0/545788905 learned_addr learned my addr 172.21.15.89:0/545788905 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:52.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.769+0000 7f9011326700 1 --2- 172.21.15.89:0/545788905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f900c0b1b40 0x7f900c15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:52.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.769+0000 7f9010b25700 1 -- 172.21.15.89:0/545788905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f900c0b1b40 msgr2=0x7f900c15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:52.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.769+0000 7f9010b25700 1 --2- 172.21.15.89:0/545788905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f900c0b1b40 0x7f900c15d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.769+0000 7f9010b25700 1 -- 172.21.15.89:0/545788905 >> v1:172.21.15.111:6789/0 conn(0x7f900c0b16d0 legacy=0x7f900c15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:52.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.769+0000 7f9010b25700 1 -- 172.21.15.89:0/545788905 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c0b6200 con 0x7f900c0b4e70 2023-10-10T20:39:52.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.769+0000 7f9011326700 1 --2- 172.21.15.89:0/545788905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f900c0b1b40 0x7f900c15d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:52.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.770+0000 7f9010b25700 1 --2- 172.21.15.89:0/545788905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f900c0b4e70 0x7f900c0ccff0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f90000099b0 tx=0x7f9000009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c83f0a3e2482c5e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:52.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.770+0000 7f900b7fe700 1 -- 172.21.15.89:0/545788905 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900000b860 con 0x7f900c0b4e70 2023-10-10T20:39:52.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.770+0000 7f900b7fe700 1 -- 172.21.15.89:0/545788905 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900000b9c0 con 0x7f900c0b4e70 2023-10-10T20:39:52.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.770+0000 7f900b7fe700 1 -- 172.21.15.89:0/545788905 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900000bd60 con 0x7f900c0b4e70 2023-10-10T20:39:52.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.770+0000 7f9011b27700 1 -- 172.21.15.89:0/545788905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f900c0b4e70 msgr2=0x7f900c0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:52.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.770+0000 7f9011b27700 1 --2- 172.21.15.89:0/545788905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f900c0b4e70 0x7f900c0ccff0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f90000099b0 tx=0x7f9000009250 comp rx=0 tx=0).stop 2023-10-10T20:39:52.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.770+0000 7f9011b27700 1 -- 172.21.15.89:0/545788905 shutdown_connections 2023-10-10T20:39:52.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.770+0000 7f9011b27700 1 --2- 172.21.15.89:0/545788905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f900c0b4e70 0x7f900c0ccff0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.770+0000 7f9011b27700 1 --2- 172.21.15.89:0/545788905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f900c0b1b40 0x7f900c15d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.770+0000 7f9011b27700 1 -- 172.21.15.89:0/545788905 >> 172.21.15.89:0/545788905 conn(0x7f900c0d10c0 msgr2=0x7f900c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:52.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.770+0000 7f9011b27700 1 -- 172.21.15.89:0/545788905 shutdown_connections 2023-10-10T20:39:52.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.770+0000 7f9011b27700 1 -- 172.21.15.89:0/545788905 wait complete. 2023-10-10T20:39:52.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.771+0000 7f9011b27700 1 Processor -- start 2023-10-10T20:39:52.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.772+0000 7f9011b27700 1 -- start start 2023-10-10T20:39:52.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.772+0000 7f9011b27700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f900c0b16d0 0x7f900c147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.772+0000 7f9011b27700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f900c0b1b40 0x7f900c16d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.772+0000 7f9011b27700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f900c0b4e70 0x7f900c16db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.772+0000 7f9011b27700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c169f80 con 0x7f900c0b4e70 2023-10-10T20:39:52.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.772+0000 7f9011b27700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0ca7c0 con 0x7f900c0b16d0 2023-10-10T20:39:52.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.772+0000 7f9011b27700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0caaa0 con 0x7f900c0b1b40 2023-10-10T20:39:52.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.772+0000 7f9011326700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f900c0b4e70 0x7f900c16db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:52.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.772+0000 7f9011326700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f900c0b4e70 0x7f900c16db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48988/0 (socket says 172.21.15.89:48988) 2023-10-10T20:39:52.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.772+0000 7f9011326700 1 -- 172.21.15.89:0/1792128518 learned_addr learned my addr 172.21.15.89:0/1792128518 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:52.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.773+0000 7f9011326700 1 -- 172.21.15.89:0/1792128518 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f900c0b1b40 msgr2=0x7f900c16d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:52.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.773+0000 7f900bfff700 1 --2- 172.21.15.89:0/1792128518 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f900c0b1b40 0x7f900c16d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:52.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.773+0000 7f9010b25700 1 --2- 172.21.15.89:0/1792128518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f900c0b16d0 0x7f900c147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:52.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.773+0000 7f9011326700 1 --2- 172.21.15.89:0/1792128518 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f900c0b1b40 0x7f900c16d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.773+0000 7f9011326700 1 -- 172.21.15.89:0/1792128518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f900c0b16d0 msgr2=0x7f900c147ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:52.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.773+0000 7f9011326700 1 --2- 172.21.15.89:0/1792128518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f900c0b16d0 0x7f900c147ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:52.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.773+0000 7f9011326700 1 -- 172.21.15.89:0/1792128518 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90000094f0 con 0x7f900c0b4e70 2023-10-10T20:39:52.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.773+0000 7f9011326700 1 --2- 172.21.15.89:0/1792128518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f900c0b4e70 0x7f900c16db30 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f900400d180 tx=0x7f9004009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:52.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.773+0000 7f9008ff9700 1 -- 172.21.15.89:0/1792128518 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900400de20 con 0x7f900c0b4e70 2023-10-10T20:39:52.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.773+0000 7f9008ff9700 1 -- 172.21.15.89:0/1792128518 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900400f040 con 0x7f900c0b4e70 2023-10-10T20:39:52.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.773+0000 7f9011b27700 1 -- 172.21.15.89:0/1792128518 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c0cad80 con 0x7f900c0b4e70 2023-10-10T20:39:52.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.773+0000 7f9008ff9700 1 -- 172.21.15.89:0/1792128518 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9004015690 con 0x7f900c0b4e70 2023-10-10T20:39:52.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.773+0000 7f9011b27700 1 -- 172.21.15.89:0/1792128518 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900c0cb340 con 0x7f900c0b4e70 2023-10-10T20:39:52.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.774+0000 7f9011b27700 1 -- 172.21.15.89:0/1792128518 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900c0cb560 con 0x7f900c0b4e70 2023-10-10T20:39:52.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.774+0000 7f9008ff9700 1 -- 172.21.15.89:0/1792128518 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f900401e3b0 con 0x7f900c0b4e70 2023-10-10T20:39:52.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.775+0000 7f9008ff9700 1 --2- 172.21.15.89:0/1792128518 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8ff0067170 0x7f8ff0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:52.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.775+0000 7f9008ff9700 1 -- 172.21.15.89:0/1792128518 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9004084f90 con 0x7f900c0b4e70 2023-10-10T20:39:52.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.775+0000 7f9010b25700 1 --2- 172.21.15.89:0/1792128518 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8ff0067170 0x7f8ff0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:52.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.775+0000 7f9010b25700 1 --2- 172.21.15.89:0/1792128518 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8ff0067170 0x7f8ff0069620 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f90000099b0 tx=0x7f9000017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:52.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.779+0000 7f9008ff9700 1 -- 172.21.15.89:0/1792128518 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9004051e60 con 0x7f900c0b4e70 2023-10-10T20:39:52.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:52.990+0000 7f9011b27700 1 -- 172.21.15.89:0/1792128518 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group_0000000000078545", "target": ["mon-mgr", ""]}) v1 -- 0x7f900c069b20 con 0x7f8ff0067170 2023-10-10T20:39:52.997 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:52.995+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:52.997 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:52.995+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:52.997 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:52.995+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:52.997 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:52.995+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:52.998 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:39:52.995+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:39:53.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.008+0000 7f9008ff9700 1 -- 172.21.15.89:0/1792128518 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f900c069b20 con 0x7f8ff0067170 2023-10-10T20:39:53.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.010+0000 7f9011b27700 1 -- 172.21.15.89:0/1792128518 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8ff0067170 msgr2=0x7f8ff0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:53.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.010+0000 7f9011b27700 1 --2- 172.21.15.89:0/1792128518 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8ff0067170 0x7f8ff0069620 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f90000099b0 tx=0x7f9000017000 comp rx=0 tx=0).stop 2023-10-10T20:39:53.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.010+0000 7f9011b27700 1 -- 172.21.15.89:0/1792128518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f900c0b4e70 msgr2=0x7f900c16db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:53.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.010+0000 7f9011b27700 1 --2- 172.21.15.89:0/1792128518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f900c0b4e70 0x7f900c16db30 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f900400d180 tx=0x7f9004009a50 comp rx=0 tx=0).stop 2023-10-10T20:39:53.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.010+0000 7f9011b27700 1 -- 172.21.15.89:0/1792128518 shutdown_connections 2023-10-10T20:39:53.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.010+0000 7f9011b27700 1 --2- 172.21.15.89:0/1792128518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f900c0b4e70 0x7f900c16db30 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.010+0000 7f9011b27700 1 --2- 172.21.15.89:0/1792128518 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8ff0067170 0x7f8ff0069620 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.010+0000 7f9011b27700 1 --2- 172.21.15.89:0/1792128518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f900c0b16d0 0x7f900c147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.010+0000 7f9011b27700 1 --2- 172.21.15.89:0/1792128518 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f900c0b1b40 0x7f900c16d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.010+0000 7f9011b27700 1 -- 172.21.15.89:0/1792128518 >> 172.21.15.89:0/1792128518 conn(0x7f900c0d10c0 msgr2=0x7f900c0b5480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:53.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.010+0000 7f9011b27700 1 -- 172.21.15.89:0/1792128518 shutdown_connections 2023-10-10T20:39:53.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.010+0000 7f9011b27700 1 -- 172.21.15.89:0/1792128518 wait complete. 2023-10-10T20:39:53.024 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs --human_readable 2023-10-10T20:39:53.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.139+0000 7fcf4759e700 1 Processor -- start 2023-10-10T20:39:53.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.139+0000 7fcf4759e700 1 -- start start 2023-10-10T20:39:53.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.140+0000 7fcf4759e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcf48159a60 0x7fcf480c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:53.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.140+0000 7fcf4759e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf480c9cb0 con 0x7fcf480cb230 2023-10-10T20:39:53.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.140+0000 7fcf4759e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf480c9f30 con 0x7fcf480ccc00 2023-10-10T20:39:53.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.140+0000 7fcf4759e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf4806dfc0 con 0x7fcf48159a60 2023-10-10T20:39:53.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.140+0000 7fcf4659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcf48159a60 0x7fcf480c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:53.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.140+0000 7fcf46d9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcf480cb230 0x7fcf4806d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:60770/0 (socket says 172.21.15.89:60770) 2023-10-10T20:39:53.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.140+0000 7fcf46d9d700 1 -- 172.21.15.89:0/89745075 learned_addr learned my addr 172.21.15.89:0/89745075 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:53.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.141+0000 7fcf4659c700 1 -- 172.21.15.89:0/89745075 >> v1:172.21.15.111:6789/0 conn(0x7fcf480ccc00 legacy=0x7fcf48069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:53.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.141+0000 7fcf4659c700 1 -- 172.21.15.89:0/89745075 >> v1:172.21.15.89:6789/0 conn(0x7fcf480cb230 legacy=0x7fcf4806d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:53.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.141+0000 7fcf4659c700 1 -- 172.21.15.89:0/89745075 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf480ca470 con 0x7fcf48159a60 2023-10-10T20:39:53.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.141+0000 7fcf4559a700 1 -- 172.21.15.89:0/89745075 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1006829062 0 0) 0x7fcf480c9cb0 con 0x7fcf480cb230 2023-10-10T20:39:53.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.141+0000 7fcf4659c700 1 --2- 172.21.15.89:0/89745075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf48159a60 0x7fcf480c9770 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fcf380098e0 tx=0x7fcf380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492fe8302baaff8b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:53.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.141+0000 7fcf4559a700 1 -- 172.21.15.89:0/89745075 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38009e80 con 0x7fcf48159a60 2023-10-10T20:39:53.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.141+0000 7fcf4559a700 1 -- 172.21.15.89:0/89745075 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3800b8f0 con 0x7fcf48159a60 2023-10-10T20:39:53.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.141+0000 7fcf4559a700 1 -- 172.21.15.89:0/89745075 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3800bc70 con 0x7fcf48159a60 2023-10-10T20:39:53.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.141+0000 7fcf4759e700 1 -- 172.21.15.89:0/89745075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf48159a60 msgr2=0x7fcf480c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:53.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.141+0000 7fcf4759e700 1 --2- 172.21.15.89:0/89745075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf48159a60 0x7fcf480c9770 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fcf380098e0 tx=0x7fcf380091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.142+0000 7fcf4759e700 1 -- 172.21.15.89:0/89745075 shutdown_connections 2023-10-10T20:39:53.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.142+0000 7fcf4759e700 1 --2- 172.21.15.89:0/89745075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf48159a60 0x7fcf480c9770 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.142+0000 7fcf4759e700 1 -- 172.21.15.89:0/89745075 >> 172.21.15.89:0/89745075 conn(0x7fcf4806baa0 msgr2=0x7fcf4806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:53.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.142+0000 7fcf4759e700 1 -- 172.21.15.89:0/89745075 shutdown_connections 2023-10-10T20:39:53.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.142+0000 7fcf4759e700 1 -- 172.21.15.89:0/89745075 wait complete. 2023-10-10T20:39:53.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.143+0000 7fcf4759e700 1 Processor -- start 2023-10-10T20:39:53.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.143+0000 7fcf4759e700 1 -- start start 2023-10-10T20:39:53.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.143+0000 7fcf4759e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcf480cb230 0x7fcf48168010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:53.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.143+0000 7fcf4759e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf480ccc00 0x7fcf4816ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:53.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.143+0000 7fcf4759e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf48159a60 0x7fcf48169ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:53.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.143+0000 7fcf4759e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4816c640 con 0x7fcf48159a60 2023-10-10T20:39:53.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.143+0000 7fcf4759e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4816b340 con 0x7fcf480ccc00 2023-10-10T20:39:53.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.143+0000 7fcf4759e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf480aa5a0 con 0x7fcf480cb230 2023-10-10T20:39:53.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.144+0000 7fcf46d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf48159a60 0x7fcf48169ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:53.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.144+0000 7fcf46d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf48159a60 0x7fcf48169ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49020/0 (socket says 172.21.15.89:49020) 2023-10-10T20:39:53.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.144+0000 7fcf46d9d700 1 -- 172.21.15.89:0/3768273751 learned_addr learned my addr 172.21.15.89:0/3768273751 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:53.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.144+0000 7fcf45d9b700 1 --2- 172.21.15.89:0/3768273751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf480ccc00 0x7fcf4816ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:53.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.144+0000 7fcf4659c700 1 --2- 172.21.15.89:0/3768273751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcf480cb230 0x7fcf48168010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:53.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.144+0000 7fcf46d9d700 1 -- 172.21.15.89:0/3768273751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcf480cb230 msgr2=0x7fcf48168010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:53.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.144+0000 7fcf46d9d700 1 --2- 172.21.15.89:0/3768273751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcf480cb230 0x7fcf48168010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.144+0000 7fcf46d9d700 1 -- 172.21.15.89:0/3768273751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf480ccc00 msgr2=0x7fcf4816ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:53.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.144+0000 7fcf46d9d700 1 --2- 172.21.15.89:0/3768273751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf480ccc00 0x7fcf4816ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.144+0000 7fcf46d9d700 1 -- 172.21.15.89:0/3768273751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf38009420 con 0x7fcf48159a60 2023-10-10T20:39:53.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.144+0000 7fcf45d9b700 1 --2- 172.21.15.89:0/3768273751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf480ccc00 0x7fcf4816ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:53.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.144+0000 7fcf46d9d700 1 --2- 172.21.15.89:0/3768273751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf48159a60 0x7fcf48169ca0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000cc90 tx=0x7fcf40001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:53.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.144+0000 7fcf367fc700 1 -- 172.21.15.89:0/3768273751 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4000cea0 con 0x7fcf48159a60 2023-10-10T20:39:53.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.144+0000 7fcf367fc700 1 -- 172.21.15.89:0/3768273751 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4000ab30 con 0x7fcf48159a60 2023-10-10T20:39:53.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.144+0000 7fcf4759e700 1 -- 172.21.15.89:0/3768273751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf4816ba30 con 0x7fcf48159a60 2023-10-10T20:39:53.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.144+0000 7fcf367fc700 1 -- 172.21.15.89:0/3768273751 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf400070f0 con 0x7fcf48159a60 2023-10-10T20:39:53.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.144+0000 7fcf4759e700 1 -- 172.21.15.89:0/3768273751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf4816bff0 con 0x7fcf48159a60 2023-10-10T20:39:53.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.145+0000 7fcf4759e700 1 -- 172.21.15.89:0/3768273751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf4816c7b0 con 0x7fcf48159a60 2023-10-10T20:39:53.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.145+0000 7fcf367fc700 1 -- 172.21.15.89:0/3768273751 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fcf400074a0 con 0x7fcf48159a60 2023-10-10T20:39:53.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.146+0000 7fcf367fc700 1 --2- 172.21.15.89:0/3768273751 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcf28067170 0x7fcf28069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:53.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.146+0000 7fcf367fc700 1 -- 172.21.15.89:0/3768273751 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcf40084dc0 con 0x7fcf48159a60 2023-10-10T20:39:53.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.146+0000 7fcf4659c700 1 --2- 172.21.15.89:0/3768273751 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcf28067170 0x7fcf28069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:53.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.146+0000 7fcf4659c700 1 --2- 172.21.15.89:0/3768273751 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcf28067170 0x7fcf28069620 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcf380093f0 tx=0x7fcf38017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:53.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.150+0000 7fcf367fc700 1 -- 172.21.15.89:0/3768273751 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf40051d10 con 0x7fcf48159a60 2023-10-10T20:39:53.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.362+0000 7fcf4759e700 1 -- 172.21.15.89:0/3768273751 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "human_readable": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fcf480ca750 con 0x7fcf28067170 2023-10-10T20:39:53.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.365+0000 7fcf367fc700 1 -- 172.21.15.89:0/3768273751 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+534 (secure 0 0 0) 0x7fcf480ca750 con 0x7fcf28067170 2023-10-10T20:39:53.367 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:39:53.368 INFO:teuthology.orchestra.run.smithi089.stdout: "mon_addrs": [ 2023-10-10T20:39:53.368 INFO:teuthology.orchestra.run.smithi089.stdout: "172.21.15.89:6789", 2023-10-10T20:39:53.368 INFO:teuthology.orchestra.run.smithi089.stdout: "172.21.15.111:6789", 2023-10-10T20:39:53.369 INFO:teuthology.orchestra.run.smithi089.stdout: "172.21.15.111:6790" 2023-10-10T20:39:53.369 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-10T20:39:53.369 INFO:teuthology.orchestra.run.smithi089.stdout: "pending_subvolume_deletions": 0, 2023-10-10T20:39:53.370 INFO:teuthology.orchestra.run.smithi089.stdout: "pools": { 2023-10-10T20:39:53.370 INFO:teuthology.orchestra.run.smithi089.stdout: "data": [ 2023-10-10T20:39:53.370 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:53.371 INFO:teuthology.orchestra.run.smithi089.stdout: "avail": " 340G", 2023-10-10T20:39:53.371 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs_data", 2023-10-10T20:39:53.372 INFO:teuthology.orchestra.run.smithi089.stdout: "used": " 0 " 2023-10-10T20:39:53.372 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:53.372 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-10T20:39:53.373 INFO:teuthology.orchestra.run.smithi089.stdout: "metadata": [ 2023-10-10T20:39:53.373 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:39:53.373 INFO:teuthology.orchestra.run.smithi089.stdout: "avail": " 340G", 2023-10-10T20:39:53.374 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs_metadata", 2023-10-10T20:39:53.374 INFO:teuthology.orchestra.run.smithi089.stdout: "used": "64.0k" 2023-10-10T20:39:53.374 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:39:53.374 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-10T20:39:53.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:39:53.375 INFO:teuthology.orchestra.run.smithi089.stdout: "used_size": " 0 " 2023-10-10T20:39:53.375 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:39:53.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.367+0000 7fcf4759e700 1 -- 172.21.15.89:0/3768273751 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcf28067170 msgr2=0x7fcf28069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:53.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.367+0000 7fcf4759e700 1 --2- 172.21.15.89:0/3768273751 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcf28067170 0x7fcf28069620 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcf380093f0 tx=0x7fcf38017000 comp rx=0 tx=0).stop 2023-10-10T20:39:53.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.367+0000 7fcf4759e700 1 -- 172.21.15.89:0/3768273751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf48159a60 msgr2=0x7fcf48169ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:53.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.367+0000 7fcf4759e700 1 --2- 172.21.15.89:0/3768273751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf48159a60 0x7fcf48169ca0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000cc90 tx=0x7fcf40001a20 comp rx=0 tx=0).stop 2023-10-10T20:39:53.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.367+0000 7fcf4759e700 1 -- 172.21.15.89:0/3768273751 shutdown_connections 2023-10-10T20:39:53.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.367+0000 7fcf4759e700 1 --2- 172.21.15.89:0/3768273751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf48159a60 0x7fcf48169ca0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.367+0000 7fcf4759e700 1 --2- 172.21.15.89:0/3768273751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcf480cb230 0x7fcf48168010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.367+0000 7fcf4759e700 1 --2- 172.21.15.89:0/3768273751 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcf28067170 0x7fcf28069620 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.367+0000 7fcf4759e700 1 --2- 172.21.15.89:0/3768273751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf480ccc00 0x7fcf4816ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.367+0000 7fcf4759e700 1 -- 172.21.15.89:0/3768273751 >> 172.21.15.89:0/3768273751 conn(0x7fcf4806baa0 msgr2=0x7fcf480c3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:53.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.367+0000 7fcf4759e700 1 -- 172.21.15.89:0/3768273751 shutdown_connections 2023-10-10T20:39:53.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.367+0000 7fcf4759e700 1 -- 172.21.15.89:0/3768273751 wait complete. 2023-10-10T20:39:53.381 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:39:53.381 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:39:53.414 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:39:53.460 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:39:53.461 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:39:53.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:53.462 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:39:53.508 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:39:53.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:39:53.509 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:39:53.524 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:39:53.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:39:53.525 DEBUG:teuthology.orchestra.run.smithi111:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:39:53.568 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:39:53.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:39:53.569 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:39:53.624 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:39:53.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.737+0000 7f03a1c96700 1 Processor -- start 2023-10-10T20:39:53.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.737+0000 7f03a1c96700 1 -- start start 2023-10-10T20:39:53.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.738+0000 7f03a1c96700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f039c159420 0x7f039c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:53.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.738+0000 7f03a1c96700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f039c1597f0 0x7f039c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:53.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.738+0000 7f03a1c96700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f039c0b11d0 con 0x7f039c15d230 2023-10-10T20:39:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.738+0000 7f03a1c96700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f039c0b14b0 con 0x7f039c1597f0 2023-10-10T20:39:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.738+0000 7f03a1c96700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c0b1790 con 0x7f039c159420 2023-10-10T20:39:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.738+0000 7f03a0c94700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f039c159420 0x7f039c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.738+0000 7f03a0c94700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f039c159420 0x7f039c0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:37584/0 (socket says 172.21.15.89:37584) 2023-10-10T20:39:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.738+0000 7f03a0c94700 1 -- 172.21.15.89:0/2425660589 learned_addr learned my addr 172.21.15.89:0/2425660589 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.738+0000 7f039bfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f039c1597f0 0x7f039c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.738+0000 7f03a0c94700 1 -- 172.21.15.89:0/2425660589 >> v2:172.21.15.111:3300/0 conn(0x7f039c1597f0 msgr2=0x7f039c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.738+0000 7f03a0c94700 1 --2- 172.21.15.89:0/2425660589 >> v2:172.21.15.111:3300/0 conn(0x7f039c1597f0 0x7f039c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.738+0000 7f03a0c94700 1 -- 172.21.15.89:0/2425660589 >> v1:172.21.15.111:6789/0 conn(0x7f039c15d230 legacy=0x7f039c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.738+0000 7f03a0c94700 1 -- 172.21.15.89:0/2425660589 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c0b1a70 con 0x7f039c159420 2023-10-10T20:39:53.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.738+0000 7f039bfff700 1 --2- 172.21.15.89:0/2425660589 >> v2:172.21.15.111:3300/0 conn(0x7f039c1597f0 0x7f039c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:39:53.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.739+0000 7f03a0c94700 1 --2- 172.21.15.89:0/2425660589 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f039c159420 0x7f039c0b5f40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0394009f20 tx=0x7f0394009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b58fe2491c7da68 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:53.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.739+0000 7f039b7fe700 1 -- 172.21.15.89:0/2425660589 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039400e070 con 0x7f039c159420 2023-10-10T20:39:53.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.739+0000 7f039b7fe700 1 -- 172.21.15.89:0/2425660589 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039400b9d0 con 0x7f039c159420 2023-10-10T20:39:53.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.739+0000 7f039b7fe700 1 -- 172.21.15.89:0/2425660589 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03940092e0 con 0x7f039c159420 2023-10-10T20:39:53.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.739+0000 7f03a1c96700 1 -- 172.21.15.89:0/2425660589 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f039c159420 msgr2=0x7f039c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:53.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.739+0000 7f03a1c96700 1 --2- 172.21.15.89:0/2425660589 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f039c159420 0x7f039c0b5f40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0394009f20 tx=0x7f0394009840 comp rx=0 tx=0).stop 2023-10-10T20:39:53.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.739+0000 7f03a1c96700 1 -- 172.21.15.89:0/2425660589 shutdown_connections 2023-10-10T20:39:53.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.739+0000 7f03a1c96700 1 --2- 172.21.15.89:0/2425660589 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f039c159420 0x7f039c0b5f40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.739+0000 7f03a1c96700 1 --2- 172.21.15.89:0/2425660589 >> v2:172.21.15.111:3300/0 conn(0x7f039c1597f0 0x7f039c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.739+0000 7f03a1c96700 1 -- 172.21.15.89:0/2425660589 >> 172.21.15.89:0/2425660589 conn(0x7f039c0d0660 msgr2=0x7f039c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:53.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.739+0000 7f03a1c96700 1 -- 172.21.15.89:0/2425660589 shutdown_connections 2023-10-10T20:39:53.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.739+0000 7f03a1c96700 1 -- 172.21.15.89:0/2425660589 wait complete. 2023-10-10T20:39:53.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.740+0000 7f03a1c96700 1 Processor -- start 2023-10-10T20:39:53.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.741+0000 7f03a1c96700 1 -- start start 2023-10-10T20:39:53.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.741+0000 7f03a1c96700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c159420 0x7f039c14d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:53.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.741+0000 7f03a1c96700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f039c1597f0 0x7f039c14d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:53.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.741+0000 7f03a1c96700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f039c15d230 0x7f039c14de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:53.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.741+0000 7f03a1c96700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c15cfb0 con 0x7f039c159420 2023-10-10T20:39:53.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.741+0000 7f03a1c96700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c152ba0 con 0x7f039c15d230 2023-10-10T20:39:53.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.741+0000 7f03a1c96700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c152d30 con 0x7f039c1597f0 2023-10-10T20:39:53.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.741+0000 7f03a0c94700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c159420 0x7f039c14d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:53.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.741+0000 7f03a0c94700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c159420 0x7f039c14d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49026/0 (socket says 172.21.15.89:49026) 2023-10-10T20:39:53.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.741+0000 7f03a0c94700 1 -- 172.21.15.89:0/1283984384 learned_addr learned my addr 172.21.15.89:0/1283984384 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:53.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.742+0000 7f039bfff700 1 --2- 172.21.15.89:0/1283984384 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f039c1597f0 0x7f039c14d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:53.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.742+0000 7f03a1495700 1 --2- 172.21.15.89:0/1283984384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f039c15d230 0x7f039c14de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:53.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.742+0000 7f03a0c94700 1 -- 172.21.15.89:0/1283984384 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f039c1597f0 msgr2=0x7f039c14d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:53.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.742+0000 7f03a0c94700 1 --2- 172.21.15.89:0/1283984384 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f039c1597f0 0x7f039c14d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.742+0000 7f03a0c94700 1 -- 172.21.15.89:0/1283984384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f039c15d230 msgr2=0x7f039c14de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:53.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.742+0000 7f03a0c94700 1 --2- 172.21.15.89:0/1283984384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f039c15d230 0x7f039c14de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.742+0000 7f03a0c94700 1 -- 172.21.15.89:0/1283984384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0390009420 con 0x7f039c159420 2023-10-10T20:39:53.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.742+0000 7f03a1495700 1 --2- 172.21.15.89:0/1283984384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f039c15d230 0x7f039c14de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:53.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.742+0000 7f03a0c94700 1 --2- 172.21.15.89:0/1283984384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c159420 0x7f039c14d0c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f039400b610 tx=0x7f0394018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:53.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.742+0000 7f0398ff9700 1 -- 172.21.15.89:0/1283984384 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03940089a0 con 0x7f039c159420 2023-10-10T20:39:53.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.742+0000 7f0398ff9700 1 -- 172.21.15.89:0/1283984384 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0394008b00 con 0x7f039c159420 2023-10-10T20:39:53.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.742+0000 7f03a1c96700 1 -- 172.21.15.89:0/1283984384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0394009ae0 con 0x7f039c159420 2023-10-10T20:39:53.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.742+0000 7f0398ff9700 1 -- 172.21.15.89:0/1283984384 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0394026460 con 0x7f039c159420 2023-10-10T20:39:53.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.742+0000 7f03a1c96700 1 -- 172.21.15.89:0/1283984384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039c14f880 con 0x7f039c159420 2023-10-10T20:39:53.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.743+0000 7f03a1c96700 1 -- 172.21.15.89:0/1283984384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039c04c3c0 con 0x7f039c159420 2023-10-10T20:39:53.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.743+0000 7f0398ff9700 1 -- 172.21.15.89:0/1283984384 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f0394003040 con 0x7f039c159420 2023-10-10T20:39:53.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.744+0000 7f0398ff9700 1 --2- 172.21.15.89:0/1283984384 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0388067170 0x7f0388069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:53.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.744+0000 7f0398ff9700 1 -- 172.21.15.89:0/1283984384 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0394086780 con 0x7f039c159420 2023-10-10T20:39:53.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.744+0000 7f039bfff700 1 --2- 172.21.15.89:0/1283984384 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0388067170 0x7f0388069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:53.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.744+0000 7f039bfff700 1 --2- 172.21.15.89:0/1283984384 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0388067170 0x7f0388069620 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f039c0af920 tx=0x7f03900091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:53.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.748+0000 7f0398ff9700 1 -- 172.21.15.89:0/1283984384 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0394053700 con 0x7f039c159420 2023-10-10T20:39:53.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.973+0000 7f03a1c96700 1 -- 172.21.15.89:0/1283984384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f039c0b1d50 con 0x7f039c159420 2023-10-10T20:39:53.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.973+0000 7f0398ff9700 1 -- 172.21.15.89:0/1283984384 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f0394056d20 con 0x7f039c159420 2023-10-10T20:39:53.976 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:53.976 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5324,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/1607893755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":1607893755},{"type":"v1","addr":"172.21.15.111:6835","nonce":1607893755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6069,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/1480864169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":1480864169},{"type":"v1","addr":"172.21.15.89:6836","nonce":1480864169}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2347053559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2347053559},{"type":"v1","addr":"172.21.15.89:6837","nonce":2347053559}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:39:37.635170+0000","modified":"2023-10-10T20:39:41.271571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6087},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6087":{"gid":6087,"name":"b","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1070916496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1070916496},{"type":"v1","addr":"172.21.15.111:6834","nonce":1070916496}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-10T20:39:53.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.975+0000 7f03a1c96700 1 -- 172.21.15.89:0/1283984384 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0388067170 msgr2=0x7f0388069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:53.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.975+0000 7f03a1c96700 1 --2- 172.21.15.89:0/1283984384 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0388067170 0x7f0388069620 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f039c0af920 tx=0x7f03900091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.975+0000 7f03a1c96700 1 -- 172.21.15.89:0/1283984384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c159420 msgr2=0x7f039c14d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:53.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.975+0000 7f03a1c96700 1 --2- 172.21.15.89:0/1283984384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c159420 0x7f039c14d0c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f039400b610 tx=0x7f0394018000 comp rx=0 tx=0).stop 2023-10-10T20:39:53.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.975+0000 7f03a1c96700 1 -- 172.21.15.89:0/1283984384 shutdown_connections 2023-10-10T20:39:53.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.975+0000 7f03a1c96700 1 --2- 172.21.15.89:0/1283984384 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0388067170 0x7f0388069620 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.975+0000 7f03a1c96700 1 --2- 172.21.15.89:0/1283984384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f039c15d230 0x7f039c14de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.975+0000 7f03a1c96700 1 --2- 172.21.15.89:0/1283984384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c159420 0x7f039c14d0c0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.975+0000 7f03a1c96700 1 --2- 172.21.15.89:0/1283984384 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f039c1597f0 0x7f039c14d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:53.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.975+0000 7f03a1c96700 1 -- 172.21.15.89:0/1283984384 >> 172.21.15.89:0/1283984384 conn(0x7f039c0d0660 msgr2=0x7f039c0b5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:53.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.975+0000 7f03a1c96700 1 -- 172.21.15.89:0/1283984384 shutdown_connections 2023-10-10T20:39:53.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:53.975+0000 7f03a1c96700 1 -- 172.21.15.89:0/1283984384 wait complete. 2023-10-10T20:39:53.981 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 47 2023-10-10T20:39:53.990 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:39:54.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.102+0000 7fdde21ed700 1 Processor -- start 2023-10-10T20:39:54.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.102+0000 7fdde21ed700 1 -- start start 2023-10-10T20:39:54.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.103+0000 7fdde21ed700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdddc159420 0x7fdddc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.103+0000 7fdde21ed700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdddc1597f0 0x7fdddc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.103+0000 7fdde21ed700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdddc0b11d0 con 0x7fdddc15d230 2023-10-10T20:39:54.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.103+0000 7fdde21ed700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdddc0b14b0 con 0x7fdddc159420 2023-10-10T20:39:54.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.103+0000 7fdde21ed700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdddc0b1790 con 0x7fdddc1597f0 2023-10-10T20:39:54.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.103+0000 7fdde11eb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdddc159420 0x7fdddc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.103+0000 7fdde11eb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdddc159420 0x7fdddc0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49040/0 (socket says 172.21.15.89:49040) 2023-10-10T20:39:54.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.103+0000 7fdde11eb700 1 -- 172.21.15.89:0/1114992447 learned_addr learned my addr 172.21.15.89:0/1114992447 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:54.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.103+0000 7fdde09ea700 1 --2- 172.21.15.89:0/1114992447 >> v2:172.21.15.111:3300/0 conn(0x7fdddc1597f0 0x7fdddc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.103+0000 7fdde11eb700 1 -- 172.21.15.89:0/1114992447 >> v2:172.21.15.111:3300/0 conn(0x7fdddc1597f0 msgr2=0x7fdddc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:54.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.103+0000 7fdde11eb700 1 --2- 172.21.15.89:0/1114992447 >> v2:172.21.15.111:3300/0 conn(0x7fdddc1597f0 0x7fdddc0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.103+0000 7fdde11eb700 1 -- 172.21.15.89:0/1114992447 >> v1:172.21.15.111:6789/0 conn(0x7fdddc15d230 legacy=0x7fdddc15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:54.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.103+0000 7fdde11eb700 1 -- 172.21.15.89:0/1114992447 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc0b1a70 con 0x7fdddc159420 2023-10-10T20:39:54.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.104+0000 7fdde11eb700 1 --2- 172.21.15.89:0/1114992447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc159420 0x7fdddc0b5f40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fdddc089750 tx=0x7fddd8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237d3fb459ce2dbb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:54.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.104+0000 7fddd3fff700 1 -- 172.21.15.89:0/1114992447 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd800e070 con 0x7fdddc159420 2023-10-10T20:39:54.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.104+0000 7fddd3fff700 1 -- 172.21.15.89:0/1114992447 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd800ba50 con 0x7fdddc159420 2023-10-10T20:39:54.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.104+0000 7fddd3fff700 1 -- 172.21.15.89:0/1114992447 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd80092e0 con 0x7fdddc159420 2023-10-10T20:39:54.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.104+0000 7fdde21ed700 1 -- 172.21.15.89:0/1114992447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc159420 msgr2=0x7fdddc0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:54.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.104+0000 7fdde21ed700 1 --2- 172.21.15.89:0/1114992447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc159420 0x7fdddc0b5f40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fdddc089750 tx=0x7fddd8009840 comp rx=0 tx=0).stop 2023-10-10T20:39:54.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.104+0000 7fdde21ed700 1 -- 172.21.15.89:0/1114992447 shutdown_connections 2023-10-10T20:39:54.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.104+0000 7fdde21ed700 1 --2- 172.21.15.89:0/1114992447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc159420 0x7fdddc0b5f40 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.104+0000 7fdde21ed700 1 --2- 172.21.15.89:0/1114992447 >> v2:172.21.15.111:3300/0 conn(0x7fdddc1597f0 0x7fdddc0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.104+0000 7fdde21ed700 1 -- 172.21.15.89:0/1114992447 >> 172.21.15.89:0/1114992447 conn(0x7fdddc0d0660 msgr2=0x7fdddc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:54.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.104+0000 7fdde21ed700 1 -- 172.21.15.89:0/1114992447 shutdown_connections 2023-10-10T20:39:54.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.104+0000 7fdde21ed700 1 -- 172.21.15.89:0/1114992447 wait complete. 2023-10-10T20:39:54.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.106+0000 7fdde21ed700 1 Processor -- start 2023-10-10T20:39:54.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.106+0000 7fdde21ed700 1 -- start start 2023-10-10T20:39:54.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.106+0000 7fdde21ed700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdddc159420 0x7fdddc147180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.106+0000 7fdde21ed700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdddc1597f0 0x7fdddc0c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.106+0000 7fdde21ed700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc15d230 0x7fdddc0c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.106+0000 7fdde21ed700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc1476c0 con 0x7fdddc15d230 2023-10-10T20:39:54.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.106+0000 7fdde21ed700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc0ca9e0 con 0x7fdddc1597f0 2023-10-10T20:39:54.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.106+0000 7fdde21ed700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc0cacc0 con 0x7fdddc159420 2023-10-10T20:39:54.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.107+0000 7fdde19ec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc15d230 0x7fdddc0c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.107+0000 7fdde19ec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc15d230 0x7fdddc0c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49048/0 (socket says 172.21.15.89:49048) 2023-10-10T20:39:54.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.107+0000 7fdde11eb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdddc159420 0x7fdddc147180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.107+0000 7fdde19ec700 1 -- 172.21.15.89:0/1870944143 learned_addr learned my addr 172.21.15.89:0/1870944143 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:54.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.107+0000 7fdde09ea700 1 --2- 172.21.15.89:0/1870944143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdddc1597f0 0x7fdddc0c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.107+0000 7fdde19ec700 1 -- 172.21.15.89:0/1870944143 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdddc159420 msgr2=0x7fdddc147180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:54.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.107+0000 7fdde19ec700 1 --2- 172.21.15.89:0/1870944143 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdddc159420 0x7fdddc147180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.107+0000 7fdde19ec700 1 -- 172.21.15.89:0/1870944143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdddc1597f0 msgr2=0x7fdddc0c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:54.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.107+0000 7fdde19ec700 1 --2- 172.21.15.89:0/1870944143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdddc1597f0 0x7fdddc0c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.107+0000 7fdde19ec700 1 -- 172.21.15.89:0/1870944143 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd8009ae0 con 0x7fdddc15d230 2023-10-10T20:39:54.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.107+0000 7fdde19ec700 1 --2- 172.21.15.89:0/1870944143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc15d230 0x7fdddc0c85e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fddd400c490 tx=0x7fddd4007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:54.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.107+0000 7fddd17fa700 1 -- 172.21.15.89:0/1870944143 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd400ce40 con 0x7fdddc15d230 2023-10-10T20:39:54.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.107+0000 7fddd17fa700 1 -- 172.21.15.89:0/1870944143 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd4011590 con 0x7fdddc15d230 2023-10-10T20:39:54.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.107+0000 7fdde21ed700 1 -- 172.21.15.89:0/1870944143 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdddc0cafa0 con 0x7fdddc15d230 2023-10-10T20:39:54.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.107+0000 7fddd17fa700 1 -- 172.21.15.89:0/1870944143 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4011950 con 0x7fdddc15d230 2023-10-10T20:39:54.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.107+0000 7fdde21ed700 1 -- 172.21.15.89:0/1870944143 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdddc0c6130 con 0x7fdddc15d230 2023-10-10T20:39:54.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.108+0000 7fdde21ed700 1 -- 172.21.15.89:0/1870944143 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdddc0cb4c0 con 0x7fdddc15d230 2023-10-10T20:39:54.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.108+0000 7fddd17fa700 1 -- 172.21.15.89:0/1870944143 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fddd4011cc0 con 0x7fdddc15d230 2023-10-10T20:39:54.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.109+0000 7fddd17fa700 1 --2- 172.21.15.89:0/1870944143 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fddc0067170 0x7fddc0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.109+0000 7fdde11eb700 1 --2- 172.21.15.89:0/1870944143 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fddc0067170 0x7fddc0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.109+0000 7fddd17fa700 1 -- 172.21.15.89:0/1870944143 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fddd4083b10 con 0x7fdddc15d230 2023-10-10T20:39:54.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.109+0000 7fdde11eb700 1 --2- 172.21.15.89:0/1870944143 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fddc0067170 0x7fddc0069620 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009880 tx=0x7fddd8018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:54.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.113+0000 7fddd17fa700 1 -- 172.21.15.89:0/1870944143 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdddc0cb4c0 con 0x7fdddc15d230 2023-10-10T20:39:54.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.301+0000 7fdde21ed700 1 -- 172.21.15.89:0/1870944143 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdddc0cb4c0 con 0x7fdddc15d230 2023-10-10T20:39:54.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.302+0000 7fddd17fa700 1 -- 172.21.15.89:0/1870944143 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+15353 (secure 0 0 0) 0x7fdddc0cb4c0 con 0x7fdddc15d230 2023-10-10T20:39:54.304 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:54.304 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":76,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:39:39.250746+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:39:35.989512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-10T20:39:36.595551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:39:54.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.304+0000 7fdde21ed700 1 -- 172.21.15.89:0/1870944143 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fddc0067170 msgr2=0x7fddc0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:54.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.304+0000 7fdde21ed700 1 --2- 172.21.15.89:0/1870944143 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fddc0067170 0x7fddc0069620 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009880 tx=0x7fddd8018000 comp rx=0 tx=0).stop 2023-10-10T20:39:54.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.304+0000 7fdde21ed700 1 -- 172.21.15.89:0/1870944143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc15d230 msgr2=0x7fdddc0c85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:54.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.304+0000 7fdde21ed700 1 --2- 172.21.15.89:0/1870944143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc15d230 0x7fdddc0c85e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fddd400c490 tx=0x7fddd4007f20 comp rx=0 tx=0).stop 2023-10-10T20:39:54.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.304+0000 7fdde21ed700 1 -- 172.21.15.89:0/1870944143 shutdown_connections 2023-10-10T20:39:54.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.304+0000 7fdde21ed700 1 --2- 172.21.15.89:0/1870944143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc15d230 0x7fdddc0c85e0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.304+0000 7fdde21ed700 1 --2- 172.21.15.89:0/1870944143 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdddc159420 0x7fdddc147180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.304+0000 7fdde21ed700 1 --2- 172.21.15.89:0/1870944143 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fddc0067170 0x7fddc0069620 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.304+0000 7fdde21ed700 1 --2- 172.21.15.89:0/1870944143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdddc1597f0 0x7fdddc0c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.304+0000 7fdde21ed700 1 -- 172.21.15.89:0/1870944143 >> 172.21.15.89:0/1870944143 conn(0x7fdddc0d0660 msgr2=0x7fdddc15d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:54.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.304+0000 7fdde21ed700 1 -- 172.21.15.89:0/1870944143 shutdown_connections 2023-10-10T20:39:54.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.304+0000 7fdde21ed700 1 -- 172.21.15.89:0/1870944143 wait complete. 2023-10-10T20:39:54.318 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:39:35.989512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:39:36.595551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:39:54.319 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:39:54.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.434+0000 7feb90ff8700 1 Processor -- start 2023-10-10T20:39:54.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.434+0000 7feb90ff8700 1 -- start start 2023-10-10T20:39:54.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.435+0000 7feb90ff8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feb8c159890 0x7feb8c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.435+0000 7feb90ff8700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7feb8c159c60 0x7feb8c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.435+0000 7feb90ff8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7feb8c15db10 0x7feb8c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.435+0000 7feb90ff8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb8c15a160 con 0x7feb8c15db10 2023-10-10T20:39:54.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.435+0000 7feb90ff8700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb8c0d13a0 con 0x7feb8c159c60 2023-10-10T20:39:54.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.435+0000 7feb90ff8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c15e140 con 0x7feb8c159890 2023-10-10T20:39:54.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.435+0000 7feb8ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7feb8c15db10 0x7feb8c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.435+0000 7feb8ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7feb8c15db10 0x7feb8c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49056/0 (socket says 172.21.15.89:49056) 2023-10-10T20:39:54.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.435+0000 7feb8ad9d700 1 -- 172.21.15.89:0/1645949455 learned_addr learned my addr 172.21.15.89:0/1645949455 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:54.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.435+0000 7feb8a59c700 1 --2- 172.21.15.89:0/1645949455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feb8c159890 0x7feb8c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.435+0000 7feb81d9b700 1 --2- 172.21.15.89:0/1645949455 >> v2:172.21.15.111:3300/0 conn(0x7feb8c159c60 0x7feb8c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.435+0000 7feb8ad9d700 1 -- 172.21.15.89:0/1645949455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feb8c159890 msgr2=0x7feb8c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:54.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.435+0000 7feb8ad9d700 1 --2- 172.21.15.89:0/1645949455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feb8c159890 0x7feb8c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.435+0000 7feb8ad9d700 1 -- 172.21.15.89:0/1645949455 >> v2:172.21.15.111:3300/0 conn(0x7feb8c159c60 msgr2=0x7feb8c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:54.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.435+0000 7feb8ad9d700 1 --2- 172.21.15.89:0/1645949455 >> v2:172.21.15.111:3300/0 conn(0x7feb8c159c60 0x7feb8c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.435+0000 7feb8ad9d700 1 -- 172.21.15.89:0/1645949455 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8c15a3d0 con 0x7feb8c15db10 2023-10-10T20:39:54.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.435+0000 7feb81d9b700 1 --2- 172.21.15.89:0/1645949455 >> v2:172.21.15.111:3300/0 conn(0x7feb8c159c60 0x7feb8c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:54.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.435+0000 7feb8a59c700 1 --2- 172.21.15.89:0/1645949455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feb8c159890 0x7feb8c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:54.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.435+0000 7feb8ad9d700 1 --2- 172.21.15.89:0/1645949455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb8c15db10 0x7feb8c069560 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0650f0 tx=0x7feb84009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb281e67c12b214b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:54.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.436+0000 7feb89d9b700 1 -- 172.21.15.89:0/1645949455 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8400e070 con 0x7feb8c15db10 2023-10-10T20:39:54.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.436+0000 7feb89d9b700 1 -- 172.21.15.89:0/1645949455 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8400ba50 con 0x7feb8c15db10 2023-10-10T20:39:54.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.436+0000 7feb89d9b700 1 -- 172.21.15.89:0/1645949455 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb840092e0 con 0x7feb8c15db10 2023-10-10T20:39:54.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.436+0000 7feb90ff8700 1 -- 172.21.15.89:0/1645949455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb8c15db10 msgr2=0x7feb8c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:54.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.436+0000 7feb90ff8700 1 --2- 172.21.15.89:0/1645949455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb8c15db10 0x7feb8c069560 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0650f0 tx=0x7feb84009840 comp rx=0 tx=0).stop 2023-10-10T20:39:54.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.436+0000 7feb90ff8700 1 -- 172.21.15.89:0/1645949455 shutdown_connections 2023-10-10T20:39:54.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.436+0000 7feb90ff8700 1 --2- 172.21.15.89:0/1645949455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb8c15db10 0x7feb8c069560 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.436+0000 7feb90ff8700 1 --2- 172.21.15.89:0/1645949455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feb8c159890 0x7feb8c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.436+0000 7feb90ff8700 1 --2- 172.21.15.89:0/1645949455 >> v2:172.21.15.111:3300/0 conn(0x7feb8c159c60 0x7feb8c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.436+0000 7feb90ff8700 1 -- 172.21.15.89:0/1645949455 >> 172.21.15.89:0/1645949455 conn(0x7feb8c068e20 msgr2=0x7feb8c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:54.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.436+0000 7feb90ff8700 1 -- 172.21.15.89:0/1645949455 shutdown_connections 2023-10-10T20:39:54.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.436+0000 7feb90ff8700 1 -- 172.21.15.89:0/1645949455 wait complete. 2023-10-10T20:39:54.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.437+0000 7feb90ff8700 1 Processor -- start 2023-10-10T20:39:54.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.437+0000 7feb90ff8700 1 -- start start 2023-10-10T20:39:54.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.438+0000 7feb90ff8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb8c159890 0x7feb8c143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.438+0000 7feb90ff8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feb8c159c60 0x7feb8c0b5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.438+0000 7feb90ff8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb8c15db10 0x7feb8c0b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.438+0000 7feb90ff8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c0ce9c0 con 0x7feb8c159890 2023-10-10T20:39:54.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.438+0000 7feb90ff8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c0b62a0 con 0x7feb8c15db10 2023-10-10T20:39:54.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.438+0000 7feb90ff8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c0ce0d0 con 0x7feb8c159c60 2023-10-10T20:39:54.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.438+0000 7feb8a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb8c159890 0x7feb8c143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.438+0000 7feb8a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb8c159890 0x7feb8c143240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49068/0 (socket says 172.21.15.89:49068) 2023-10-10T20:39:54.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.438+0000 7feb8a59c700 1 -- 172.21.15.89:0/991206405 learned_addr learned my addr 172.21.15.89:0/991206405 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:54.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.438+0000 7feb81d9b700 1 --2- 172.21.15.89:0/991206405 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feb8c159c60 0x7feb8c0b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.438+0000 7feb8ad9d700 1 --2- 172.21.15.89:0/991206405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb8c15db10 0x7feb8c0b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.438+0000 7feb8a59c700 1 -- 172.21.15.89:0/991206405 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feb8c159c60 msgr2=0x7feb8c0b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:54.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.438+0000 7feb8a59c700 1 --2- 172.21.15.89:0/991206405 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feb8c159c60 0x7feb8c0b5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.438+0000 7feb8a59c700 1 -- 172.21.15.89:0/991206405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb8c15db10 msgr2=0x7feb8c0b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:54.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.438+0000 7feb8a59c700 1 --2- 172.21.15.89:0/991206405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb8c15db10 0x7feb8c0b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.438+0000 7feb8a59c700 1 -- 172.21.15.89:0/991206405 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb84009ae0 con 0x7feb8c159890 2023-10-10T20:39:54.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.439+0000 7feb8a59c700 1 --2- 172.21.15.89:0/991206405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb8c159890 0x7feb8c143240 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7feb7c00e8d0 tx=0x7feb7c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:54.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.439+0000 7feb82ffd700 1 -- 172.21.15.89:0/991206405 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c00ee70 con 0x7feb8c159890 2023-10-10T20:39:54.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.439+0000 7feb82ffd700 1 -- 172.21.15.89:0/991206405 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7c00c5e0 con 0x7feb8c159890 2023-10-10T20:39:54.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.439+0000 7feb90ff8700 1 -- 172.21.15.89:0/991206405 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb8c0ce370 con 0x7feb8c159890 2023-10-10T20:39:54.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.439+0000 7feb82ffd700 1 -- 172.21.15.89:0/991206405 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c00c980 con 0x7feb8c159890 2023-10-10T20:39:54.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.439+0000 7feb90ff8700 1 -- 172.21.15.89:0/991206405 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb8c0c78a0 con 0x7feb8c159890 2023-10-10T20:39:54.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.439+0000 7feb90ff8700 1 -- 172.21.15.89:0/991206405 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8c0ceca0 con 0x7feb8c159890 2023-10-10T20:39:54.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.441+0000 7feb82ffd700 1 -- 172.21.15.89:0/991206405 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7feb7c025430 con 0x7feb8c159890 2023-10-10T20:39:54.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.441+0000 7feb82ffd700 1 --2- 172.21.15.89:0/991206405 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7feb6c067170 0x7feb6c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.441+0000 7feb82ffd700 1 -- 172.21.15.89:0/991206405 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feb7c084b70 con 0x7feb8c159890 2023-10-10T20:39:54.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.441+0000 7feb81d9b700 1 --2- 172.21.15.89:0/991206405 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7feb6c067170 0x7feb6c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.442+0000 7feb81d9b700 1 --2- 172.21.15.89:0/991206405 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7feb6c067170 0x7feb6c069620 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0b0290 tx=0x7feb74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:54.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.445+0000 7feb82ffd700 1 -- 172.21.15.89:0/991206405 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb7c055550 con 0x7feb8c159890 2023-10-10T20:39:54.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.626+0000 7feb90ff8700 1 -- 172.21.15.89:0/991206405 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb8c04c3c0 con 0x7feb8c159890 2023-10-10T20:39:54.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.626+0000 7feb82ffd700 1 -- 172.21.15.89:0/991206405 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb7c0200a0 con 0x7feb8c159890 2023-10-10T20:39:54.629 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:39:54.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.628+0000 7feb90ff8700 1 -- 172.21.15.89:0/991206405 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7feb6c067170 msgr2=0x7feb6c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:54.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.628+0000 7feb90ff8700 1 --2- 172.21.15.89:0/991206405 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7feb6c067170 0x7feb6c069620 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0b0290 tx=0x7feb74008040 comp rx=0 tx=0).stop 2023-10-10T20:39:54.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.628+0000 7feb90ff8700 1 -- 172.21.15.89:0/991206405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb8c159890 msgr2=0x7feb8c143240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:54.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.628+0000 7feb90ff8700 1 --2- 172.21.15.89:0/991206405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb8c159890 0x7feb8c143240 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7feb7c00e8d0 tx=0x7feb7c008040 comp rx=0 tx=0).stop 2023-10-10T20:39:54.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.628+0000 7feb90ff8700 1 -- 172.21.15.89:0/991206405 shutdown_connections 2023-10-10T20:39:54.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.628+0000 7feb90ff8700 1 --2- 172.21.15.89:0/991206405 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7feb6c067170 0x7feb6c069620 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.628+0000 7feb90ff8700 1 --2- 172.21.15.89:0/991206405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb8c15db10 0x7feb8c0b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.628+0000 7feb90ff8700 1 --2- 172.21.15.89:0/991206405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb8c159890 0x7feb8c143240 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.628+0000 7feb90ff8700 1 --2- 172.21.15.89:0/991206405 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feb8c159c60 0x7feb8c0b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.628+0000 7feb90ff8700 1 -- 172.21.15.89:0/991206405 >> 172.21.15.89:0/991206405 conn(0x7feb8c068e20 msgr2=0x7feb8c15b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:54.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.628+0000 7feb90ff8700 1 -- 172.21.15.89:0/991206405 shutdown_connections 2023-10-10T20:39:54.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.628+0000 7feb90ff8700 1 -- 172.21.15.89:0/991206405 wait complete. 2023-10-10T20:39:54.643 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:39:54.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.764+0000 7fadca86c700 1 Processor -- start 2023-10-10T20:39:54.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.764+0000 7fadca86c700 1 -- start start 2023-10-10T20:39:54.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.765+0000 7fadca86c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fadc40b4e70 0x7fadc40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.765+0000 7fadca86c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc40b16d0 0x7fadc40b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.765+0000 7fadca86c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fadc415d240 0x7fadc415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.765+0000 7fadca86c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc40b2080 con 0x7fadc415d240 2023-10-10T20:39:54.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.765+0000 7fadca86c700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc40cfb00 con 0x7fadc40b4e70 2023-10-10T20:39:54.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.765+0000 7fadca86c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc40b5c40 con 0x7fadc40b16d0 2023-10-10T20:39:54.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.765+0000 7fadca06b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fadc415d240 0x7fadc415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.765+0000 7fadca06b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fadc415d240 0x7fadc415d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49078/0 (socket says 172.21.15.89:49078) 2023-10-10T20:39:54.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.765+0000 7fadca06b700 1 -- 172.21.15.89:0/1663606391 learned_addr learned my addr 172.21.15.89:0/1663606391 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:54.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.765+0000 7fadc9069700 1 --2- 172.21.15.89:0/1663606391 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc40b16d0 0x7fadc40b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.765+0000 7fadc986a700 1 --2- 172.21.15.89:0/1663606391 >> v2:172.21.15.111:3300/0 conn(0x7fadc40b4e70 0x7fadc40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.765+0000 7fadca06b700 1 -- 172.21.15.89:0/1663606391 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc40b16d0 msgr2=0x7fadc40b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:54.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.765+0000 7fadca06b700 1 --2- 172.21.15.89:0/1663606391 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc40b16d0 0x7fadc40b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.765+0000 7fadca06b700 1 -- 172.21.15.89:0/1663606391 >> v2:172.21.15.111:3300/0 conn(0x7fadc40b4e70 msgr2=0x7fadc40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:54.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.765+0000 7fadca06b700 1 --2- 172.21.15.89:0/1663606391 >> v2:172.21.15.111:3300/0 conn(0x7fadc40b4e70 0x7fadc40ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.765+0000 7fadca06b700 1 -- 172.21.15.89:0/1663606391 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc40b5f20 con 0x7fadc415d240 2023-10-10T20:39:54.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.765+0000 7fadc9069700 1 --2- 172.21.15.89:0/1663606391 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc40b16d0 0x7fadc40b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:54.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.765+0000 7fadc986a700 1 --2- 172.21.15.89:0/1663606391 >> v2:172.21.15.111:3300/0 conn(0x7fadc40b4e70 0x7fadc40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:54.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.766+0000 7fadca06b700 1 --2- 172.21.15.89:0/1663606391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415d240 0x7fadc415d6c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fadc4064910 tx=0x7fadbc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929b4100d326af37 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:54.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.766+0000 7fadc8868700 1 -- 172.21.15.89:0/1663606391 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc00b860 con 0x7fadc415d240 2023-10-10T20:39:54.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.766+0000 7fadc8868700 1 -- 172.21.15.89:0/1663606391 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc00b9c0 con 0x7fadc415d240 2023-10-10T20:39:54.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.766+0000 7fadc8868700 1 -- 172.21.15.89:0/1663606391 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc00bd60 con 0x7fadc415d240 2023-10-10T20:39:54.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.766+0000 7fadca86c700 1 -- 172.21.15.89:0/1663606391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415d240 msgr2=0x7fadc415d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:54.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.766+0000 7fadca86c700 1 --2- 172.21.15.89:0/1663606391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415d240 0x7fadc415d6c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fadc4064910 tx=0x7fadbc009250 comp rx=0 tx=0).stop 2023-10-10T20:39:54.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.766+0000 7fadca86c700 1 -- 172.21.15.89:0/1663606391 shutdown_connections 2023-10-10T20:39:54.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.766+0000 7fadca86c700 1 --2- 172.21.15.89:0/1663606391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415d240 0x7fadc415d6c0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.766+0000 7fadca86c700 1 --2- 172.21.15.89:0/1663606391 >> v2:172.21.15.111:3300/0 conn(0x7fadc40b4e70 0x7fadc40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.766+0000 7fadca86c700 1 --2- 172.21.15.89:0/1663606391 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc40b16d0 0x7fadc40b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.766+0000 7fadca86c700 1 -- 172.21.15.89:0/1663606391 >> 172.21.15.89:0/1663606391 conn(0x7fadc40d10c0 msgr2=0x7fadc40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:54.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.766+0000 7fadca86c700 1 -- 172.21.15.89:0/1663606391 shutdown_connections 2023-10-10T20:39:54.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.766+0000 7fadca86c700 1 -- 172.21.15.89:0/1663606391 wait complete. 2023-10-10T20:39:54.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.767+0000 7fadca86c700 1 Processor -- start 2023-10-10T20:39:54.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.768+0000 7fadca86c700 1 -- start start 2023-10-10T20:39:54.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.768+0000 7fadca86c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc40b16d0 0x7fadc40ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.768+0000 7fadca86c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc40b4e70 0x7fadc4159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.768+0000 7fadca86c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415d240 0x7fadc40c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.768+0000 7fadca86c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc4169af0 con 0x7fadc415d240 2023-10-10T20:39:54.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.768+0000 7fadca86c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc4159540 con 0x7fadc40b4e70 2023-10-10T20:39:54.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.768+0000 7fadca86c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc416a330 con 0x7fadc40b16d0 2023-10-10T20:39:54.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.768+0000 7fadca06b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415d240 0x7fadc40c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.768+0000 7fadca06b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415d240 0x7fadc40c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49088/0 (socket says 172.21.15.89:49088) 2023-10-10T20:39:54.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.768+0000 7fadca06b700 1 -- 172.21.15.89:0/3314940573 learned_addr learned my addr 172.21.15.89:0/3314940573 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:54.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.768+0000 7fadc9069700 1 --2- 172.21.15.89:0/3314940573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc40b4e70 0x7fadc4159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.768+0000 7fadc986a700 1 --2- 172.21.15.89:0/3314940573 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc40b16d0 0x7fadc40ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.769+0000 7fadca06b700 1 -- 172.21.15.89:0/3314940573 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc40b16d0 msgr2=0x7fadc40ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:54.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.769+0000 7fadca06b700 1 --2- 172.21.15.89:0/3314940573 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc40b16d0 0x7fadc40ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.769+0000 7fadca06b700 1 -- 172.21.15.89:0/3314940573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc40b4e70 msgr2=0x7fadc4159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:54.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.769+0000 7fadca06b700 1 --2- 172.21.15.89:0/3314940573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc40b4e70 0x7fadc4159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.769+0000 7fadca06b700 1 -- 172.21.15.89:0/3314940573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc0094f0 con 0x7fadc415d240 2023-10-10T20:39:54.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.769+0000 7fadca06b700 1 --2- 172.21.15.89:0/3314940573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415d240 0x7fadc40c76b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fadc4070dd0 tx=0x7fadbc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:54.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.769+0000 7fadb9ffb700 1 -- 172.21.15.89:0/3314940573 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc011ea0 con 0x7fadc415d240 2023-10-10T20:39:54.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.769+0000 7fadb9ffb700 1 -- 172.21.15.89:0/3314940573 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc01a630 con 0x7fadc415d240 2023-10-10T20:39:54.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.769+0000 7fadca86c700 1 -- 172.21.15.89:0/3314940573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc40cab90 con 0x7fadc415d240 2023-10-10T20:39:54.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.769+0000 7fadb9ffb700 1 -- 172.21.15.89:0/3314940573 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc01aa10 con 0x7fadc415d240 2023-10-10T20:39:54.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.769+0000 7fadca86c700 1 -- 172.21.15.89:0/3314940573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc40ca930 con 0x7fadc415d240 2023-10-10T20:39:54.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.770+0000 7fadca86c700 1 -- 172.21.15.89:0/3314940573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc4088990 con 0x7fadc415d240 2023-10-10T20:39:54.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.770+0000 7fadb9ffb700 1 -- 172.21.15.89:0/3314940573 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fadbc02d440 con 0x7fadc415d240 2023-10-10T20:39:54.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.771+0000 7fadb9ffb700 1 --2- 172.21.15.89:0/3314940573 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fada8067170 0x7fada8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:54.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.771+0000 7fadb9ffb700 1 -- 172.21.15.89:0/3314940573 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fadbc086390 con 0x7fadc415d240 2023-10-10T20:39:54.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.771+0000 7fadc986a700 1 --2- 172.21.15.89:0/3314940573 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fada8067170 0x7fada8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:54.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.771+0000 7fadc986a700 1 --2- 172.21.15.89:0/3314940573 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fada8067170 0x7fada8069620 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fadc000d6e0 tx=0x7fadc0007850 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:54.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.775+0000 7fadb9ffb700 1 -- 172.21.15.89:0/3314940573 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fadbc053370 con 0x7fadc415d240 2023-10-10T20:39:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.956+0000 7fadca86c700 1 -- 172.21.15.89:0/3314940573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fadc40b1e60 con 0x7fadc415d240 2023-10-10T20:39:54.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.958+0000 7fadb9ffb700 1 -- 172.21.15.89:0/3314940573 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+11 (secure 0 0 0) 0x7fadbc006d70 con 0x7fadc415d240 2023-10-10T20:39:54.960 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:39:54.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.961+0000 7fadca86c700 1 -- 172.21.15.89:0/3314940573 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fada8067170 msgr2=0x7fada8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.961+0000 7fadca86c700 1 --2- 172.21.15.89:0/3314940573 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fada8067170 0x7fada8069620 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fadc000d6e0 tx=0x7fadc0007850 comp rx=0 tx=0).stop 2023-10-10T20:39:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.961+0000 7fadca86c700 1 -- 172.21.15.89:0/3314940573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415d240 msgr2=0x7fadc40c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.961+0000 7fadca86c700 1 --2- 172.21.15.89:0/3314940573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415d240 0x7fadc40c76b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fadc4070dd0 tx=0x7fadbc017000 comp rx=0 tx=0).stop 2023-10-10T20:39:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.961+0000 7fadca86c700 1 -- 172.21.15.89:0/3314940573 shutdown_connections 2023-10-10T20:39:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.961+0000 7fadca86c700 1 --2- 172.21.15.89:0/3314940573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415d240 0x7fadc40c76b0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.961+0000 7fadca86c700 1 --2- 172.21.15.89:0/3314940573 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc40b16d0 0x7fadc40ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.961+0000 7fadca86c700 1 --2- 172.21.15.89:0/3314940573 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fada8067170 0x7fada8069620 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.961+0000 7fadca86c700 1 --2- 172.21.15.89:0/3314940573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc40b4e70 0x7fadc4159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.961+0000 7fadca86c700 1 -- 172.21.15.89:0/3314940573 >> 172.21.15.89:0/3314940573 conn(0x7fadc40d10c0 msgr2=0x7fadc40ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.961+0000 7fadca86c700 1 -- 172.21.15.89:0/3314940573 shutdown_connections 2023-10-10T20:39:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:54.961+0000 7fadca86c700 1 -- 172.21.15.89:0/3314940573 wait complete. 2023-10-10T20:39:54.979 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:39:55.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.095+0000 7f3f13d36700 1 Processor -- start 2023-10-10T20:39:55.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.095+0000 7f3f13d36700 1 -- start start 2023-10-10T20:39:55.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.096+0000 7f3f13d36700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f0c159c60 0x7f3f0c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.096+0000 7f3f13d36700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c15a160 con 0x7f3f0c159890 2023-10-10T20:39:55.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.096+0000 7f3f13d36700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c0d13a0 con 0x7f3f0c15db10 2023-10-10T20:39:55.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.096+0000 7f3f13d36700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c15e140 con 0x7f3f0c159c60 2023-10-10T20:39:55.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.096+0000 7f3f11ad2700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3f0c159890 0x7f3f0c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:60780/0 (socket says 172.21.15.89:60780) 2023-10-10T20:39:55.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.096+0000 7f3f11ad2700 1 -- 172.21.15.89:0/3354775710 learned_addr learned my addr 172.21.15.89:0/3354775710 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:55.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.096+0000 7f3f10ad0700 1 -- 172.21.15.89:0/3354775710 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1207900236 0 0) 0x7f3f0c15a160 con 0x7f3f0c159890 2023-10-10T20:39:55.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.096+0000 7f3f10ad0700 1 -- 172.21.15.89:0/3354775710 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ef8003570 con 0x7f3f0c159890 2023-10-10T20:39:55.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.096+0000 7f3f112d1700 1 --2- 172.21.15.89:0/3354775710 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f0c159c60 0x7f3f0c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f10ad0700 1 -- 172.21.15.89:0/3354775710 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2728486200 0 0) 0x7f3ef8003570 con 0x7f3f0c159890 2023-10-10T20:39:55.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f10ad0700 1 -- 172.21.15.89:0/3354775710 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f0c159c60 msgr2=0x7f3f0c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:55.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f10ad0700 1 --2- 172.21.15.89:0/3354775710 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f0c159c60 0x7f3f0c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f10ad0700 1 -- 172.21.15.89:0/3354775710 >> v1:172.21.15.111:6789/0 conn(0x7f3f0c15db10 legacy=0x7f3f0c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:55.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f10ad0700 1 -- 172.21.15.89:0/3354775710 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c15a3d0 con 0x7f3f0c159890 2023-10-10T20:39:55.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f10ad0700 1 -- 172.21.15.89:0/3354775710 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f3f080017f0 con 0x7f3f0c159890 2023-10-10T20:39:55.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f10ad0700 1 -- 172.21.15.89:0/3354775710 >> v1:172.21.15.89:6789/0 conn(0x7f3f0c159890 legacy=0x7f3f0c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:55.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f10ad0700 1 --2- 172.21.15.89:0/3354775710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ef8003a90 0x7f3ef8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f10ad0700 1 --2- 172.21.15.89:0/3354775710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ef8004810 0x7f3ef8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f10ad0700 1 --2- 172.21.15.89:0/3354775710 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f0c159c60 0x7f3ef8005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f10ad0700 1 -- 172.21.15.89:0/3354775710 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c15e140 con 0x7f3ef8004810 2023-10-10T20:39:55.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f10ad0700 1 -- 172.21.15.89:0/3354775710 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef800bbf0 con 0x7f3ef8003a90 2023-10-10T20:39:55.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f10ad0700 1 -- 172.21.15.89:0/3354775710 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef800bd80 con 0x7f3f0c159c60 2023-10-10T20:39:55.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f112d1700 1 --2- 172.21.15.89:0/3354775710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ef8004810 0x7f3ef8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f11ad2700 1 --2- 172.21.15.89:0/3354775710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ef8003a90 0x7f3ef8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f122d3700 1 --2- 172.21.15.89:0/3354775710 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f0c159c60 0x7f3ef8005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f112d1700 1 -- 172.21.15.89:0/3354775710 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f0c159c60 msgr2=0x7f3ef8005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:55.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f112d1700 1 --2- 172.21.15.89:0/3354775710 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f0c159c60 0x7f3ef8005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f112d1700 1 -- 172.21.15.89:0/3354775710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ef8003a90 msgr2=0x7f3ef8003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:55.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f112d1700 1 --2- 172.21.15.89:0/3354775710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ef8003a90 0x7f3ef8003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f112d1700 1 -- 172.21.15.89:0/3354775710 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef8003570 con 0x7f3ef8004810 2023-10-10T20:39:55.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f122d3700 1 --2- 172.21.15.89:0/3354775710 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f0c159c60 0x7f3ef8005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f112d1700 1 --2- 172.21.15.89:0/3354775710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ef8004810 0x7f3ef8004c80 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009c00 tx=0x7f3efc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c51bac2024b040 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f10ad0700 1 -- 172.21.15.89:0/3354775710 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc00e710 con 0x7f3ef8004810 2023-10-10T20:39:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f10ad0700 1 -- 172.21.15.89:0/3354775710 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3efc00e870 con 0x7f3ef8004810 2023-10-10T20:39:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.097+0000 7f3f10ad0700 1 -- 172.21.15.89:0/3354775710 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc00ec30 con 0x7f3ef8004810 2023-10-10T20:39:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.098+0000 7f3f13d36700 1 -- 172.21.15.89:0/3354775710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ef8004810 msgr2=0x7f3ef8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:55.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.098+0000 7f3f13d36700 1 --2- 172.21.15.89:0/3354775710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ef8004810 0x7f3ef8004c80 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009c00 tx=0x7f3efc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.098+0000 7f3f122d3700 1 -- 172.21.15.89:0/3354775710 reap_dead start 2023-10-10T20:39:55.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.098+0000 7f3f13d36700 1 -- 172.21.15.89:0/3354775710 shutdown_connections 2023-10-10T20:39:55.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.098+0000 7f3f13d36700 1 -- 172.21.15.89:0/3354775710 >> 172.21.15.89:0/3354775710 conn(0x7f3f0c068e20 msgr2=0x7f3f0c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:55.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.098+0000 7f3f13d36700 1 -- 172.21.15.89:0/3354775710 shutdown_connections 2023-10-10T20:39:55.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.098+0000 7f3f13d36700 1 -- 172.21.15.89:0/3354775710 wait complete. 2023-10-10T20:39:55.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.099+0000 7f3f13d36700 1 Processor -- start 2023-10-10T20:39:55.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.099+0000 7f3f13d36700 1 -- start start 2023-10-10T20:39:55.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.100+0000 7f3f13d36700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ef8004810 0x7f3f0c0b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.100+0000 7f3f13d36700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f0c0c6b70 0x7f3f0c143d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.100+0000 7f3f13d36700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f0c0c6fe0 0x7f3f0c1442a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.100+0000 7f3f13d36700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0b2590 con 0x7f3f0c0c6b70 2023-10-10T20:39:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.100+0000 7f3f13d36700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0b38a0 con 0x7f3ef8004810 2023-10-10T20:39:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.100+0000 7f3f13d36700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c1447e0 con 0x7f3f0c0c6fe0 2023-10-10T20:39:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.100+0000 7f3f112d1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f0c0c6b70 0x7f3f0c143d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.100+0000 7f3f112d1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f0c0c6b70 0x7f3f0c143d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49104/0 (socket says 172.21.15.89:49104) 2023-10-10T20:39:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.100+0000 7f3f112d1700 1 -- 172.21.15.89:0/4251599708 learned_addr learned my addr 172.21.15.89:0/4251599708 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.100+0000 7f3f11ad2700 1 --2- 172.21.15.89:0/4251599708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ef8004810 0x7f3f0c0b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.100+0000 7f3f122d3700 1 --2- 172.21.15.89:0/4251599708 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f0c0c6fe0 0x7f3f0c1442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.100+0000 7f3f112d1700 1 -- 172.21.15.89:0/4251599708 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f0c0c6fe0 msgr2=0x7f3f0c1442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.100+0000 7f3f112d1700 1 --2- 172.21.15.89:0/4251599708 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f0c0c6fe0 0x7f3f0c1442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.100+0000 7f3f112d1700 1 -- 172.21.15.89:0/4251599708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ef8004810 msgr2=0x7f3f0c0b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:55.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.100+0000 7f3f112d1700 1 --2- 172.21.15.89:0/4251599708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ef8004810 0x7f3f0c0b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.100+0000 7f3f112d1700 1 -- 172.21.15.89:0/4251599708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c0b1980 con 0x7f3f0c0c6b70 2023-10-10T20:39:55.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.100+0000 7f3f112d1700 1 --2- 172.21.15.89:0/4251599708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f0c0c6b70 0x7f3f0c143d60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80057b0 tx=0x7f3efc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:55.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.101+0000 7f3f01ffb700 1 -- 172.21.15.89:0/4251599708 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc022070 con 0x7f3f0c0c6b70 2023-10-10T20:39:55.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.101+0000 7f3f01ffb700 1 -- 172.21.15.89:0/4251599708 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3efc00e9e0 con 0x7f3f0c0c6b70 2023-10-10T20:39:55.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.101+0000 7f3f13d36700 1 -- 172.21.15.89:0/4251599708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0c0b1c60 con 0x7f3f0c0c6b70 2023-10-10T20:39:55.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.101+0000 7f3f01ffb700 1 -- 172.21.15.89:0/4251599708 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc01c450 con 0x7f3f0c0c6b70 2023-10-10T20:39:55.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.101+0000 7f3f13d36700 1 -- 172.21.15.89:0/4251599708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c0b2220 con 0x7f3f0c0c6b70 2023-10-10T20:39:55.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.102+0000 7f3f13d36700 1 -- 172.21.15.89:0/4251599708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0c04c3c0 con 0x7f3f0c0c6b70 2023-10-10T20:39:55.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.102+0000 7f3f01ffb700 1 -- 172.21.15.89:0/4251599708 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f3efc01c880 con 0x7f3f0c0c6b70 2023-10-10T20:39:55.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.103+0000 7f3f01ffb700 1 --2- 172.21.15.89:0/4251599708 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3ef0067280 0x7f3ef0069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.103+0000 7f3f11ad2700 1 --2- 172.21.15.89:0/4251599708 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3ef0067280 0x7f3ef0069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.103+0000 7f3f01ffb700 1 -- 172.21.15.89:0/4251599708 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3efc0860c0 con 0x7f3f0c0c6b70 2023-10-10T20:39:55.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.103+0000 7f3f11ad2700 1 --2- 172.21.15.89:0/4251599708 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3ef0067280 0x7f3ef0069730 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3f08001920 tx=0x7f3f08006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:55.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.107+0000 7f3f01ffb700 1 -- 172.21.15.89:0/4251599708 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3efc053060 con 0x7f3f0c0c6b70 2023-10-10T20:39:55.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.290+0000 7f3f13d36700 1 -- 172.21.15.89:0/4251599708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3f0c0cb7e0 con 0x7f3f0c0c6b70 2023-10-10T20:39:55.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.290+0000 7f3f01ffb700 1 -- 172.21.15.89:0/4251599708 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3efc017020 con 0x7f3f0c0c6b70 2023-10-10T20:39:55.293 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-10T20:39:55.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.292+0000 7f3f13d36700 1 -- 172.21.15.89:0/4251599708 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3ef0067280 msgr2=0x7f3ef0069730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:55.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.292+0000 7f3f13d36700 1 --2- 172.21.15.89:0/4251599708 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3ef0067280 0x7f3ef0069730 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3f08001920 tx=0x7f3f08006040 comp rx=0 tx=0).stop 2023-10-10T20:39:55.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.292+0000 7f3f13d36700 1 -- 172.21.15.89:0/4251599708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f0c0c6b70 msgr2=0x7f3f0c143d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:55.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.292+0000 7f3f13d36700 1 --2- 172.21.15.89:0/4251599708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f0c0c6b70 0x7f3f0c143d60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80057b0 tx=0x7f3efc01a000 comp rx=0 tx=0).stop 2023-10-10T20:39:55.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.292+0000 7f3f13d36700 1 -- 172.21.15.89:0/4251599708 shutdown_connections 2023-10-10T20:39:55.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.292+0000 7f3f13d36700 1 --2- 172.21.15.89:0/4251599708 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3f0c0c6fe0 0x7f3f0c1442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.292+0000 7f3f13d36700 1 --2- 172.21.15.89:0/4251599708 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3ef0067280 0x7f3ef0069730 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.292+0000 7f3f13d36700 1 --2- 172.21.15.89:0/4251599708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ef8004810 0x7f3f0c0b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.292+0000 7f3f13d36700 1 --2- 172.21.15.89:0/4251599708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f0c0c6b70 0x7f3f0c143d60 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.292+0000 7f3f13d36700 1 -- 172.21.15.89:0/4251599708 >> 172.21.15.89:0/4251599708 conn(0x7f3f0c068e20 msgr2=0x7f3f0c152ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:55.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.292+0000 7f3f13d36700 1 -- 172.21.15.89:0/4251599708 shutdown_connections 2023-10-10T20:39:55.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.292+0000 7f3f13d36700 1 -- 172.21.15.89:0/4251599708 wait complete. 2023-10-10T20:39:55.307 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:39:55.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.420+0000 7fadcb6f2700 1 Processor -- start 2023-10-10T20:39:55.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.420+0000 7fadcb6f2700 1 -- start start 2023-10-10T20:39:55.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.421+0000 7fadcb6f2700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fadc4159890 0x7fadc415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.421+0000 7fadcb6f2700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc4159c60 0x7fadc415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.421+0000 7fadcb6f2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fadc415db10 0x7fadc4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.421+0000 7fadcb6f2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc415a160 con 0x7fadc415db10 2023-10-10T20:39:55.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.421+0000 7fadcb6f2700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc40d13a0 con 0x7fadc4159890 2023-10-10T20:39:55.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.421+0000 7fadcb6f2700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc415e140 con 0x7fadc4159c60 2023-10-10T20:39:55.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.421+0000 7fadc9c8f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fadc415db10 0x7fadc4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.421+0000 7fadc9c8f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fadc415db10 0x7fadc4069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49116/0 (socket says 172.21.15.89:49116) 2023-10-10T20:39:55.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.421+0000 7fadc9c8f700 1 -- 172.21.15.89:0/682449057 learned_addr learned my addr 172.21.15.89:0/682449057 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:55.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.421+0000 7fadc8c8d700 1 --2- 172.21.15.89:0/682449057 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc4159c60 0x7fadc415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.421+0000 7fadc948e700 1 --2- 172.21.15.89:0/682449057 >> v2:172.21.15.111:3300/0 conn(0x7fadc4159890 0x7fadc415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.421+0000 7fadc9c8f700 1 -- 172.21.15.89:0/682449057 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc4159c60 msgr2=0x7fadc415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:55.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.421+0000 7fadc9c8f700 1 --2- 172.21.15.89:0/682449057 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc4159c60 0x7fadc415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.421+0000 7fadc9c8f700 1 -- 172.21.15.89:0/682449057 >> v2:172.21.15.111:3300/0 conn(0x7fadc4159890 msgr2=0x7fadc415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:55.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.421+0000 7fadc9c8f700 1 --2- 172.21.15.89:0/682449057 >> v2:172.21.15.111:3300/0 conn(0x7fadc4159890 0x7fadc415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.421+0000 7fadc9c8f700 1 -- 172.21.15.89:0/682449057 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc415a3d0 con 0x7fadc415db10 2023-10-10T20:39:55.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.421+0000 7fadc948e700 1 --2- 172.21.15.89:0/682449057 >> v2:172.21.15.111:3300/0 conn(0x7fadc4159890 0x7fadc415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:55.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.421+0000 7fadc8c8d700 1 --2- 172.21.15.89:0/682449057 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc4159c60 0x7fadc415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:55.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.422+0000 7fadc9c8f700 1 --2- 172.21.15.89:0/682449057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415db10 0x7fadc4069560 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fadc40d1090 tx=0x7fadbc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7418e2024ada5b8e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:55.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.422+0000 7fadbbfff700 1 -- 172.21.15.89:0/682449057 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc00e070 con 0x7fadc415db10 2023-10-10T20:39:55.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.422+0000 7fadbbfff700 1 -- 172.21.15.89:0/682449057 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc00ba50 con 0x7fadc415db10 2023-10-10T20:39:55.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.422+0000 7fadbbfff700 1 -- 172.21.15.89:0/682449057 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0092e0 con 0x7fadc415db10 2023-10-10T20:39:55.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.422+0000 7fadcb6f2700 1 -- 172.21.15.89:0/682449057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415db10 msgr2=0x7fadc4069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:55.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.422+0000 7fadcb6f2700 1 --2- 172.21.15.89:0/682449057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415db10 0x7fadc4069560 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fadc40d1090 tx=0x7fadbc009840 comp rx=0 tx=0).stop 2023-10-10T20:39:55.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.422+0000 7fadcb6f2700 1 -- 172.21.15.89:0/682449057 shutdown_connections 2023-10-10T20:39:55.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.422+0000 7fadcb6f2700 1 --2- 172.21.15.89:0/682449057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415db10 0x7fadc4069560 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.422+0000 7fadcb6f2700 1 --2- 172.21.15.89:0/682449057 >> v2:172.21.15.111:3300/0 conn(0x7fadc4159890 0x7fadc415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.422+0000 7fadcb6f2700 1 --2- 172.21.15.89:0/682449057 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc4159c60 0x7fadc415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.422+0000 7fadcb6f2700 1 -- 172.21.15.89:0/682449057 >> 172.21.15.89:0/682449057 conn(0x7fadc4068e20 msgr2=0x7fadc40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:55.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.422+0000 7fadcb6f2700 1 -- 172.21.15.89:0/682449057 shutdown_connections 2023-10-10T20:39:55.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.422+0000 7fadcb6f2700 1 -- 172.21.15.89:0/682449057 wait complete. 2023-10-10T20:39:55.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.423+0000 7fadcb6f2700 1 Processor -- start 2023-10-10T20:39:55.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.424+0000 7fadcb6f2700 1 -- start start 2023-10-10T20:39:55.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.424+0000 7fadcb6f2700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc4159890 0x7fadc40b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.424+0000 7fadcb6f2700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc4159c60 0x7fadc40cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.424+0000 7fadcb6f2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415db10 0x7fadc4154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.424+0000 7fadcb6f2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc4153ad0 con 0x7fadc415db10 2023-10-10T20:39:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.424+0000 7fadcb6f2700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc41548c0 con 0x7fadc4159890 2023-10-10T20:39:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.424+0000 7fadcb6f2700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc4153cb0 con 0x7fadc4159c60 2023-10-10T20:39:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.424+0000 7fadc9c8f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415db10 0x7fadc4154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.424+0000 7fadc9c8f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415db10 0x7fadc4154d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49130/0 (socket says 172.21.15.89:49130) 2023-10-10T20:39:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.424+0000 7fadc9c8f700 1 -- 172.21.15.89:0/2281808031 learned_addr learned my addr 172.21.15.89:0/2281808031 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.425+0000 7fadc9c8f700 1 -- 172.21.15.89:0/2281808031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc4159c60 msgr2=0x7fadc40cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.425+0000 7fadc9c8f700 1 --2- 172.21.15.89:0/2281808031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc4159c60 0x7fadc40cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.425+0000 7fadc9c8f700 1 -- 172.21.15.89:0/2281808031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc4159890 msgr2=0x7fadc40b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.425+0000 7fadc948e700 1 --2- 172.21.15.89:0/2281808031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc4159890 0x7fadc40b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.425+0000 7fadc9c8f700 1 --2- 172.21.15.89:0/2281808031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc4159890 0x7fadc40b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.425+0000 7fadc9c8f700 1 -- 172.21.15.89:0/2281808031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc009ae0 con 0x7fadc415db10 2023-10-10T20:39:55.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.425+0000 7fadc948e700 1 --2- 172.21.15.89:0/2281808031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc4159890 0x7fadc40b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:55.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.425+0000 7fadc9c8f700 1 --2- 172.21.15.89:0/2281808031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415db10 0x7fadc4154d50 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fadc4070e20 tx=0x7fadbc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:55.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.425+0000 7fadb97fa700 1 -- 172.21.15.89:0/2281808031 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0032f0 con 0x7fadc415db10 2023-10-10T20:39:55.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.425+0000 7fadb97fa700 1 -- 172.21.15.89:0/2281808031 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc008e40 con 0x7fadc415db10 2023-10-10T20:39:55.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.425+0000 7fadcb6f2700 1 -- 172.21.15.89:0/2281808031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc4153f90 con 0x7fadc415db10 2023-10-10T20:39:55.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.425+0000 7fadb97fa700 1 -- 172.21.15.89:0/2281808031 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0256b0 con 0x7fadc415db10 2023-10-10T20:39:55.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.425+0000 7fadcb6f2700 1 -- 172.21.15.89:0/2281808031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc4154550 con 0x7fadc415db10 2023-10-10T20:39:55.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.426+0000 7fadcb6f2700 1 -- 172.21.15.89:0/2281808031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc404c3c0 con 0x7fadc415db10 2023-10-10T20:39:55.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.426+0000 7fadb97fa700 1 -- 172.21.15.89:0/2281808031 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fadbc003040 con 0x7fadc415db10 2023-10-10T20:39:55.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.427+0000 7fadb97fa700 1 --2- 172.21.15.89:0/2281808031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fada8067200 0x7fada80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.427+0000 7fadc948e700 1 --2- 172.21.15.89:0/2281808031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fada8067200 0x7fada80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.427+0000 7fadb97fa700 1 -- 172.21.15.89:0/2281808031 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fadbc086110 con 0x7fadc415db10 2023-10-10T20:39:55.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.427+0000 7fadc948e700 1 --2- 172.21.15.89:0/2281808031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fada8067200 0x7fada80696b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fadc0006f00 tx=0x7fadc0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:55.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.431+0000 7fadb97fa700 1 -- 172.21.15.89:0/2281808031 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fadbc052fd0 con 0x7fadc415db10 2023-10-10T20:39:55.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.658+0000 7fadcb6f2700 1 -- 172.21.15.89:0/2281808031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadc4154a00 con 0x7fadc415db10 2023-10-10T20:39:55.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.659+0000 7fadb97fa700 1 -- 172.21.15.89:0/2281808031 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fadbc0565f0 con 0x7fadc415db10 2023-10-10T20:39:55.661 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:55.662 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5324,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/1607893755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":1607893755},{"type":"v1","addr":"172.21.15.111:6835","nonce":1607893755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6069,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/1480864169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":1480864169},{"type":"v1","addr":"172.21.15.89:6836","nonce":1480864169}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2347053559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2347053559},{"type":"v1","addr":"172.21.15.89:6837","nonce":2347053559}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:39:37.635170+0000","modified":"2023-10-10T20:39:41.271571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6087},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6087":{"gid":6087,"name":"b","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1070916496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1070916496},{"type":"v1","addr":"172.21.15.111:6834","nonce":1070916496}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-10T20:39:55.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.661+0000 7fadcb6f2700 1 -- 172.21.15.89:0/2281808031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fada8067200 msgr2=0x7fada80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:55.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.661+0000 7fadcb6f2700 1 --2- 172.21.15.89:0/2281808031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fada8067200 0x7fada80696b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fadc0006f00 tx=0x7fadc0008040 comp rx=0 tx=0).stop 2023-10-10T20:39:55.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.661+0000 7fadcb6f2700 1 -- 172.21.15.89:0/2281808031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415db10 msgr2=0x7fadc4154d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:55.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.661+0000 7fadcb6f2700 1 --2- 172.21.15.89:0/2281808031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415db10 0x7fadc4154d50 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fadc4070e20 tx=0x7fadbc018000 comp rx=0 tx=0).stop 2023-10-10T20:39:55.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.661+0000 7fadcb6f2700 1 -- 172.21.15.89:0/2281808031 shutdown_connections 2023-10-10T20:39:55.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.661+0000 7fadcb6f2700 1 --2- 172.21.15.89:0/2281808031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc415db10 0x7fadc4154d50 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.661+0000 7fadcb6f2700 1 --2- 172.21.15.89:0/2281808031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fada8067200 0x7fada80696b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.661+0000 7fadcb6f2700 1 --2- 172.21.15.89:0/2281808031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc4159890 0x7fadc40b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.661+0000 7fadcb6f2700 1 --2- 172.21.15.89:0/2281808031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fadc4159c60 0x7fadc40cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.661+0000 7fadcb6f2700 1 -- 172.21.15.89:0/2281808031 >> 172.21.15.89:0/2281808031 conn(0x7fadc4068e20 msgr2=0x7fadc4152db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:55.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.661+0000 7fadcb6f2700 1 -- 172.21.15.89:0/2281808031 shutdown_connections 2023-10-10T20:39:55.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.661+0000 7fadcb6f2700 1 -- 172.21.15.89:0/2281808031 wait complete. 2023-10-10T20:39:55.666 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 47 2023-10-10T20:39:55.675 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:39:55.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.788+0000 7f8c3117c700 1 Processor -- start 2023-10-10T20:39:55.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.788+0000 7f8c3117c700 1 -- start start 2023-10-10T20:39:55.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.789+0000 7f8c3117c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8c2c1597f0 0x7f8c2c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.789+0000 7f8c3117c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8c2c15d230 0x7f8c2c15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.789+0000 7f8c3117c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c2c15db70 con 0x7f8c2c159420 2023-10-10T20:39:55.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.789+0000 7f8c3117c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c2c0b11d0 con 0x7f8c2c15d230 2023-10-10T20:39:55.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.789+0000 7f8c3117c700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c2c0b14b0 con 0x7f8c2c1597f0 2023-10-10T20:39:55.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.789+0000 7f8c3097b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8c2c15d230 0x7f8c2c15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.789+0000 7f8c3097b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8c2c15d230 0x7f8c2c15d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49144/0 (socket says 172.21.15.89:49144) 2023-10-10T20:39:55.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.789+0000 7f8c3097b700 1 -- 172.21.15.89:0/2721681946 learned_addr learned my addr 172.21.15.89:0/2721681946 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:55.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.789+0000 7f8c2b7fe700 1 --2- 172.21.15.89:0/2721681946 >> v2:172.21.15.111:3300/0 conn(0x7f8c2c1597f0 0x7f8c2c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.789+0000 7f8c3097b700 1 -- 172.21.15.89:0/2721681946 >> v2:172.21.15.111:3300/0 conn(0x7f8c2c1597f0 msgr2=0x7f8c2c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.789+0000 7f8c3097b700 1 --2- 172.21.15.89:0/2721681946 >> v2:172.21.15.111:3300/0 conn(0x7f8c2c1597f0 0x7f8c2c0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.789+0000 7f8c3097b700 1 -- 172.21.15.89:0/2721681946 >> v1:172.21.15.111:6789/0 conn(0x7f8c2c159420 legacy=0x7f8c2c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.789+0000 7f8c3097b700 1 -- 172.21.15.89:0/2721681946 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c0b1790 con 0x7f8c2c15d230 2023-10-10T20:39:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.789+0000 7f8c3097b700 1 --2- 172.21.15.89:0/2721681946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c2c15d230 0x7f8c2c15d630 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c089750 tx=0x7f8c24009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94a6a0e83cace88 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.790+0000 7f8c2affd700 1 -- 172.21.15.89:0/2721681946 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2400e070 con 0x7f8c2c15d230 2023-10-10T20:39:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.790+0000 7f8c2affd700 1 -- 172.21.15.89:0/2721681946 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2400ba10 con 0x7f8c2c15d230 2023-10-10T20:39:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.790+0000 7f8c2affd700 1 -- 172.21.15.89:0/2721681946 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c240092e0 con 0x7f8c2c15d230 2023-10-10T20:39:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.790+0000 7f8c3117c700 1 -- 172.21.15.89:0/2721681946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c2c15d230 msgr2=0x7f8c2c15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.790+0000 7f8c3117c700 1 --2- 172.21.15.89:0/2721681946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c2c15d230 0x7f8c2c15d630 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c089750 tx=0x7f8c24009840 comp rx=0 tx=0).stop 2023-10-10T20:39:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.790+0000 7f8c3117c700 1 -- 172.21.15.89:0/2721681946 shutdown_connections 2023-10-10T20:39:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.790+0000 7f8c3117c700 1 --2- 172.21.15.89:0/2721681946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c2c15d230 0x7f8c2c15d630 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.790+0000 7f8c3117c700 1 --2- 172.21.15.89:0/2721681946 >> v2:172.21.15.111:3300/0 conn(0x7f8c2c1597f0 0x7f8c2c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.790+0000 7f8c3117c700 1 -- 172.21.15.89:0/2721681946 >> 172.21.15.89:0/2721681946 conn(0x7f8c2c0d0660 msgr2=0x7f8c2c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.790+0000 7f8c3117c700 1 -- 172.21.15.89:0/2721681946 shutdown_connections 2023-10-10T20:39:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.790+0000 7f8c3117c700 1 -- 172.21.15.89:0/2721681946 wait complete. 2023-10-10T20:39:55.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.791+0000 7f8c3117c700 1 Processor -- start 2023-10-10T20:39:55.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.791+0000 7f8c3117c700 1 -- start start 2023-10-10T20:39:55.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.792+0000 7f8c3117c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c2c159420 0x7f8c2c0c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.792+0000 7f8c3117c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c2c1597f0 0x7f8c2c0b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.792+0000 7f8c3117c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c2c15d230 0x7f8c2c16cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.792+0000 7f8c3117c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c0b5dd0 con 0x7f8c2c1597f0 2023-10-10T20:39:55.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.792+0000 7f8c3117c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c16a960 con 0x7f8c2c159420 2023-10-10T20:39:55.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.792+0000 7f8c3117c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c0b5530 con 0x7f8c2c15d230 2023-10-10T20:39:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.792+0000 7f8c2b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c2c1597f0 0x7f8c2c0b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.792+0000 7f8c2b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c2c1597f0 0x7f8c2c0b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49146/0 (socket says 172.21.15.89:49146) 2023-10-10T20:39:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.792+0000 7f8c2b7fe700 1 -- 172.21.15.89:0/828188228 learned_addr learned my addr 172.21.15.89:0/828188228 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.792+0000 7f8c2bfff700 1 --2- 172.21.15.89:0/828188228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c2c159420 0x7f8c2c0c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.792+0000 7f8c3097b700 1 --2- 172.21.15.89:0/828188228 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c2c15d230 0x7f8c2c16cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.792+0000 7f8c2b7fe700 1 -- 172.21.15.89:0/828188228 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c2c15d230 msgr2=0x7f8c2c16cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.792+0000 7f8c2b7fe700 1 --2- 172.21.15.89:0/828188228 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c2c15d230 0x7f8c2c16cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.792+0000 7f8c2b7fe700 1 -- 172.21.15.89:0/828188228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c2c159420 msgr2=0x7f8c2c0c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.792+0000 7f8c2b7fe700 1 --2- 172.21.15.89:0/828188228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c2c159420 0x7f8c2c0c5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.792+0000 7f8c2b7fe700 1 -- 172.21.15.89:0/828188228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c24009ae0 con 0x7f8c2c1597f0 2023-10-10T20:39:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.793+0000 7f8c2b7fe700 1 --2- 172.21.15.89:0/828188228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c2c1597f0 0x7f8c2c0b4ce0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000c880 tx=0x7f8c20008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.793+0000 7f8c13fff700 1 -- 172.21.15.89:0/828188228 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20011600 con 0x7f8c2c1597f0 2023-10-10T20:39:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.793+0000 7f8c13fff700 1 -- 172.21.15.89:0/828188228 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c20011780 con 0x7f8c2c1597f0 2023-10-10T20:39:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.793+0000 7f8c3117c700 1 -- 172.21.15.89:0/828188228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c2c0b5810 con 0x7f8c2c1597f0 2023-10-10T20:39:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.793+0000 7f8c13fff700 1 -- 172.21.15.89:0/828188228 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20011b40 con 0x7f8c2c1597f0 2023-10-10T20:39:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.793+0000 7f8c3117c700 1 -- 172.21.15.89:0/828188228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2c0b5220 con 0x7f8c2c1597f0 2023-10-10T20:39:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.794+0000 7f8c3117c700 1 -- 172.21.15.89:0/828188228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2c16aaf0 con 0x7f8c2c1597f0 2023-10-10T20:39:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.794+0000 7f8c13fff700 1 -- 172.21.15.89:0/828188228 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f8c20010350 con 0x7f8c2c1597f0 2023-10-10T20:39:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.794+0000 7f8c13fff700 1 --2- 172.21.15.89:0/828188228 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c18067280 0x7f8c18069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.795+0000 7f8c13fff700 1 -- 172.21.15.89:0/828188228 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8c200848d0 con 0x7f8c2c1597f0 2023-10-10T20:39:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.795+0000 7f8c2bfff700 1 --2- 172.21.15.89:0/828188228 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c18067280 0x7f8c18069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.795+0000 7f8c2bfff700 1 --2- 172.21.15.89:0/828188228 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c18067280 0x7f8c18069730 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c003870 tx=0x7f8c1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.799+0000 7f8c13fff700 1 -- 172.21.15.89:0/828188228 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c200516a0 con 0x7f8c2c1597f0 2023-10-10T20:39:55.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.985+0000 7f8c3117c700 1 -- 172.21.15.89:0/828188228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c2c04c3c0 con 0x7f8c2c1597f0 2023-10-10T20:39:55.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.985+0000 7f8c13fff700 1 -- 172.21.15.89:0/828188228 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+15353 (secure 0 0 0) 0x7f8c20015090 con 0x7f8c2c1597f0 2023-10-10T20:39:55.988 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:55.988 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":76,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:39:39.250746+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:39:35.989512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-10T20:39:36.595551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:39:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.987+0000 7f8c3117c700 1 -- 172.21.15.89:0/828188228 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c18067280 msgr2=0x7f8c18069730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.987+0000 7f8c3117c700 1 --2- 172.21.15.89:0/828188228 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c18067280 0x7f8c18069730 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c003870 tx=0x7f8c1c009040 comp rx=0 tx=0).stop 2023-10-10T20:39:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.988+0000 7f8c3117c700 1 -- 172.21.15.89:0/828188228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c2c1597f0 msgr2=0x7f8c2c0b4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.988+0000 7f8c3117c700 1 --2- 172.21.15.89:0/828188228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c2c1597f0 0x7f8c2c0b4ce0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000c880 tx=0x7f8c20008040 comp rx=0 tx=0).stop 2023-10-10T20:39:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.988+0000 7f8c3117c700 1 -- 172.21.15.89:0/828188228 shutdown_connections 2023-10-10T20:39:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.988+0000 7f8c3117c700 1 --2- 172.21.15.89:0/828188228 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c2c15d230 0x7f8c2c16cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.988+0000 7f8c3117c700 1 --2- 172.21.15.89:0/828188228 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c18067280 0x7f8c18069730 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.988+0000 7f8c3117c700 1 --2- 172.21.15.89:0/828188228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c2c159420 0x7f8c2c0c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.988+0000 7f8c3117c700 1 --2- 172.21.15.89:0/828188228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c2c1597f0 0x7f8c2c0b4ce0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:55.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.988+0000 7f8c3117c700 1 -- 172.21.15.89:0/828188228 >> 172.21.15.89:0/828188228 conn(0x7f8c2c0d0660 msgr2=0x7f8c2c069b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:55.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.988+0000 7f8c3117c700 1 -- 172.21.15.89:0/828188228 shutdown_connections 2023-10-10T20:39:55.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:55.988+0000 7f8c3117c700 1 -- 172.21.15.89:0/828188228 wait complete. 2023-10-10T20:39:56.002 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:39:56.003 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:39:56.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.115+0000 7f725bfff700 1 Processor -- start 2023-10-10T20:39:56.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.115+0000 7f725bfff700 1 -- start start 2023-10-10T20:39:56.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.116+0000 7f725bfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f725c159420 0x7f725c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.116+0000 7f725bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f725c15d230 0x7f725c15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.116+0000 7f725bfff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725c0b2980 con 0x7f725c1597f0 2023-10-10T20:39:56.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.116+0000 7f725bfff700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725c15db70 con 0x7f725c15d230 2023-10-10T20:39:56.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.116+0000 7f725bfff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725c0b11d0 con 0x7f725c159420 2023-10-10T20:39:56.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.116+0000 7f725b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f725c15d230 0x7f725c15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:56.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.116+0000 7f725b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f725c15d230 0x7f725c15d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49148/0 (socket says 172.21.15.89:49148) 2023-10-10T20:39:56.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.116+0000 7f725b7fe700 1 -- 172.21.15.89:0/2741206896 learned_addr learned my addr 172.21.15.89:0/2741206896 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:56.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.116+0000 7f725affd700 1 --2- 172.21.15.89:0/2741206896 >> v2:172.21.15.111:3300/0 conn(0x7f725c159420 0x7f725c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:56.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.117+0000 7f725b7fe700 1 -- 172.21.15.89:0/2741206896 >> v2:172.21.15.111:3300/0 conn(0x7f725c159420 msgr2=0x7f725c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:56.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.117+0000 7f725b7fe700 1 --2- 172.21.15.89:0/2741206896 >> v2:172.21.15.111:3300/0 conn(0x7f725c159420 0x7f725c0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.117+0000 7f725b7fe700 1 -- 172.21.15.89:0/2741206896 >> v1:172.21.15.111:6789/0 conn(0x7f725c1597f0 legacy=0x7f725c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:56.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.117+0000 7f725b7fe700 1 -- 172.21.15.89:0/2741206896 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c0b14b0 con 0x7f725c15d230 2023-10-10T20:39:56.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.117+0000 7f725b7fe700 1 --2- 172.21.15.89:0/2741206896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725c15d230 0x7f725c15d630 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f725c0b2f20 tx=0x7f7254009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4e30f9afab8f09 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:56.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.117+0000 7f725a7fc700 1 -- 172.21.15.89:0/2741206896 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725400e070 con 0x7f725c15d230 2023-10-10T20:39:56.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.117+0000 7f725a7fc700 1 -- 172.21.15.89:0/2741206896 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725400ba50 con 0x7f725c15d230 2023-10-10T20:39:56.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.117+0000 7f725a7fc700 1 -- 172.21.15.89:0/2741206896 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72540092e0 con 0x7f725c15d230 2023-10-10T20:39:56.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.117+0000 7f725bfff700 1 -- 172.21.15.89:0/2741206896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725c15d230 msgr2=0x7f725c15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:56.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.117+0000 7f725bfff700 1 --2- 172.21.15.89:0/2741206896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725c15d230 0x7f725c15d630 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f725c0b2f20 tx=0x7f7254009840 comp rx=0 tx=0).stop 2023-10-10T20:39:56.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.117+0000 7f725bfff700 1 -- 172.21.15.89:0/2741206896 shutdown_connections 2023-10-10T20:39:56.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.117+0000 7f725bfff700 1 --2- 172.21.15.89:0/2741206896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725c15d230 0x7f725c15d630 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.117+0000 7f725bfff700 1 --2- 172.21.15.89:0/2741206896 >> v2:172.21.15.111:3300/0 conn(0x7f725c159420 0x7f725c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.117+0000 7f725bfff700 1 -- 172.21.15.89:0/2741206896 >> 172.21.15.89:0/2741206896 conn(0x7f725c0d0660 msgr2=0x7f725c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:56.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.117+0000 7f725bfff700 1 -- 172.21.15.89:0/2741206896 shutdown_connections 2023-10-10T20:39:56.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.117+0000 7f725bfff700 1 -- 172.21.15.89:0/2741206896 wait complete. 2023-10-10T20:39:56.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.119+0000 7f725bfff700 1 Processor -- start 2023-10-10T20:39:56.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.119+0000 7f725bfff700 1 -- start start 2023-10-10T20:39:56.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.119+0000 7f725bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725c159420 0x7f725c155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.119+0000 7f725bfff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f725c1597f0 0x7f725c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.119+0000 7f725bfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725c15d230 0x7f725c0c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.119+0000 7f725bfff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c0c9740 con 0x7f725c159420 2023-10-10T20:39:56.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.119+0000 7f725bfff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c0c5400 con 0x7f725c15d230 2023-10-10T20:39:56.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.119+0000 7f725bfff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725c0c5650 con 0x7f725c1597f0 2023-10-10T20:39:56.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.119+0000 7f725affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725c159420 0x7f725c155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:56.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.119+0000 7f725affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725c159420 0x7f725c155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49164/0 (socket says 172.21.15.89:49164) 2023-10-10T20:39:56.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.119+0000 7f725affd700 1 -- 172.21.15.89:0/3791686860 learned_addr learned my addr 172.21.15.89:0/3791686860 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:56.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.120+0000 7f72527fc700 1 --2- 172.21.15.89:0/3791686860 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f725c1597f0 0x7f725c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:56.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.120+0000 7f725b7fe700 1 --2- 172.21.15.89:0/3791686860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725c15d230 0x7f725c0c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:56.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.120+0000 7f725affd700 1 -- 172.21.15.89:0/3791686860 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f725c1597f0 msgr2=0x7f725c0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:56.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.120+0000 7f725affd700 1 --2- 172.21.15.89:0/3791686860 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f725c1597f0 0x7f725c0c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.120+0000 7f725affd700 1 -- 172.21.15.89:0/3791686860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725c15d230 msgr2=0x7f725c0c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:56.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.120+0000 7f725affd700 1 --2- 172.21.15.89:0/3791686860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725c15d230 0x7f725c0c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.120+0000 7f725affd700 1 -- 172.21.15.89:0/3791686860 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7254009ae0 con 0x7f725c159420 2023-10-10T20:39:56.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.120+0000 7f725b7fe700 1 --2- 172.21.15.89:0/3791686860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725c15d230 0x7f725c0c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:56.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.120+0000 7f72527fc700 1 --2- 172.21.15.89:0/3791686860 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f725c1597f0 0x7f725c0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:56.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.120+0000 7f725affd700 1 --2- 172.21.15.89:0/3791686860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725c159420 0x7f725c155480 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f724c00d890 tx=0x7f724c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:56.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.120+0000 7f72537fe700 1 -- 172.21.15.89:0/3791686860 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c00b600 con 0x7f725c159420 2023-10-10T20:39:56.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.120+0000 7f72537fe700 1 -- 172.21.15.89:0/3791686860 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c00b760 con 0x7f725c159420 2023-10-10T20:39:56.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.120+0000 7f725bfff700 1 -- 172.21.15.89:0/3791686860 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725c0c5930 con 0x7f725c159420 2023-10-10T20:39:56.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.120+0000 7f72537fe700 1 -- 172.21.15.89:0/3791686860 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c00bae0 con 0x7f725c159420 2023-10-10T20:39:56.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.120+0000 7f725bfff700 1 -- 172.21.15.89:0/3791686860 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725c0c5ef0 con 0x7f725c159420 2023-10-10T20:39:56.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.121+0000 7f725bfff700 1 -- 172.21.15.89:0/3791686860 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f725c0c6540 con 0x7f725c159420 2023-10-10T20:39:56.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.121+0000 7f72537fe700 1 -- 172.21.15.89:0/3791686860 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f724c01e460 con 0x7f725c159420 2023-10-10T20:39:56.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.122+0000 7f72537fe700 1 --2- 172.21.15.89:0/3791686860 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f723c067200 0x7f723c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.122+0000 7f72537fe700 1 -- 172.21.15.89:0/3791686860 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f724c085060 con 0x7f725c159420 2023-10-10T20:39:56.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.122+0000 7f72527fc700 1 --2- 172.21.15.89:0/3791686860 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f723c067200 0x7f723c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:56.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.122+0000 7f72527fc700 1 --2- 172.21.15.89:0/3791686860 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f723c067200 0x7f723c0696b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7244007fd0 tx=0x7f7244009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:56.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.126+0000 7f72537fe700 1 -- 172.21.15.89:0/3791686860 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f724c051f80 con 0x7f725c159420 2023-10-10T20:39:56.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.351+0000 7f725bfff700 1 -- 172.21.15.89:0/3791686860 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f725c04c3c0 con 0x7f725c159420 2023-10-10T20:39:56.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.351+0000 7f72537fe700 1 -- 172.21.15.89:0/3791686860 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f724c0555a0 con 0x7f725c159420 2023-10-10T20:39:56.354 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:56.354 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5324,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/1607893755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":1607893755},{"type":"v1","addr":"172.21.15.111:6835","nonce":1607893755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6069,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/1480864169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":1480864169},{"type":"v1","addr":"172.21.15.89:6836","nonce":1480864169}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2347053559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2347053559},{"type":"v1","addr":"172.21.15.89:6837","nonce":2347053559}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:39:37.635170+0000","modified":"2023-10-10T20:39:41.271571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6087},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6087":{"gid":6087,"name":"b","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1070916496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1070916496},{"type":"v1","addr":"172.21.15.111:6834","nonce":1070916496}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-10T20:39:56.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.353+0000 7f725bfff700 1 -- 172.21.15.89:0/3791686860 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f723c067200 msgr2=0x7f723c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:56.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.353+0000 7f725bfff700 1 --2- 172.21.15.89:0/3791686860 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f723c067200 0x7f723c0696b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7244007fd0 tx=0x7f7244009040 comp rx=0 tx=0).stop 2023-10-10T20:39:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.353+0000 7f725bfff700 1 -- 172.21.15.89:0/3791686860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725c159420 msgr2=0x7f725c155480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.353+0000 7f725bfff700 1 --2- 172.21.15.89:0/3791686860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725c159420 0x7f725c155480 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f724c00d890 tx=0x7f724c008040 comp rx=0 tx=0).stop 2023-10-10T20:39:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.353+0000 7f725bfff700 1 -- 172.21.15.89:0/3791686860 shutdown_connections 2023-10-10T20:39:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.353+0000 7f725bfff700 1 --2- 172.21.15.89:0/3791686860 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f723c067200 0x7f723c0696b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.353+0000 7f725bfff700 1 --2- 172.21.15.89:0/3791686860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725c15d230 0x7f725c0c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.353+0000 7f725bfff700 1 --2- 172.21.15.89:0/3791686860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f725c159420 0x7f725c155480 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.353+0000 7f725bfff700 1 --2- 172.21.15.89:0/3791686860 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f725c1597f0 0x7f725c0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.353+0000 7f725bfff700 1 -- 172.21.15.89:0/3791686860 >> 172.21.15.89:0/3791686860 conn(0x7f725c0d0660 msgr2=0x7f725c1586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.354+0000 7f725bfff700 1 -- 172.21.15.89:0/3791686860 shutdown_connections 2023-10-10T20:39:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.354+0000 7f725bfff700 1 -- 172.21.15.89:0/3791686860 wait complete. 2023-10-10T20:39:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 47 2023-10-10T20:39:56.368 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:39:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.480+0000 7fcf04a94700 1 Processor -- start 2023-10-10T20:39:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.480+0000 7fcf04a94700 1 -- start start 2023-10-10T20:39:56.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.481+0000 7fcf04a94700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcf00159c60 0x7fcf0015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.481+0000 7fcf04a94700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf0015a160 con 0x7fcf00159890 2023-10-10T20:39:56.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.481+0000 7fcf04a94700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf000d13a0 con 0x7fcf0015db10 2023-10-10T20:39:56.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.481+0000 7fcf04a94700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf0015df50 con 0x7fcf00159c60 2023-10-10T20:39:56.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.481+0000 7fcefe59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcf00159890 0x7fcf0015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:60786/0 (socket says 172.21.15.89:60786) 2023-10-10T20:39:56.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.481+0000 7fcefe59c700 1 -- 172.21.15.89:0/1551843722 learned_addr learned my addr 172.21.15.89:0/1551843722 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:56.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefd59a700 1 -- 172.21.15.89:0/1551843722 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1565845033 0 0) 0x7fcf0015a160 con 0x7fcf00159890 2023-10-10T20:39:56.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefd59a700 1 -- 172.21.15.89:0/1551843722 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fceec003570 con 0x7fcf00159890 2023-10-10T20:39:56.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefdd9b700 1 --2- 172.21.15.89:0/1551843722 >> v2:172.21.15.111:3300/0 conn(0x7fcf00159c60 0x7fcf0015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:56.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefd59a700 1 -- 172.21.15.89:0/1551843722 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2469854797 0 0) 0x7fceec003570 con 0x7fcf00159890 2023-10-10T20:39:56.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefd59a700 1 -- 172.21.15.89:0/1551843722 >> v2:172.21.15.111:3300/0 conn(0x7fcf00159c60 msgr2=0x7fcf0015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:56.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefd59a700 1 --2- 172.21.15.89:0/1551843722 >> v2:172.21.15.111:3300/0 conn(0x7fcf00159c60 0x7fcf0015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefd59a700 1 -- 172.21.15.89:0/1551843722 >> v1:172.21.15.111:6789/0 conn(0x7fcf0015db10 legacy=0x7fcf00069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:56.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefd59a700 1 -- 172.21.15.89:0/1551843722 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0015e140 con 0x7fcf00159890 2023-10-10T20:39:56.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefd59a700 1 -- 172.21.15.89:0/1551843722 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fcef8001470 con 0x7fcf00159890 2023-10-10T20:39:56.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefd59a700 1 -- 172.21.15.89:0/1551843722 >> v1:172.21.15.89:6789/0 conn(0x7fcf00159890 legacy=0x7fcf0015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:56.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefd59a700 1 --2- 172.21.15.89:0/1551843722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fceec003a60 0x7fceec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefd59a700 1 --2- 172.21.15.89:0/1551843722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fceec004810 0x7fceec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefd59a700 1 --2- 172.21.15.89:0/1551843722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fceec0051c0 0x7fceec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefd59a700 1 -- 172.21.15.89:0/1551843722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec00c0a0 con 0x7fceec003a60 2023-10-10T20:39:56.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefd59a700 1 -- 172.21.15.89:0/1551843722 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec00c210 con 0x7fceec0051c0 2023-10-10T20:39:56.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefd59a700 1 -- 172.21.15.89:0/1551843722 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceec00c3d0 con 0x7fceec004810 2023-10-10T20:39:56.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefe59c700 1 --2- 172.21.15.89:0/1551843722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fceec003a60 0x7fceec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:56.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefed9d700 1 --2- 172.21.15.89:0/1551843722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fceec0051c0 0x7fceec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:56.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefdd9b700 1 --2- 172.21.15.89:0/1551843722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fceec004810 0x7fceec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:56.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefe59c700 1 -- 172.21.15.89:0/1551843722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fceec004810 msgr2=0x7fceec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:56.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefe59c700 1 --2- 172.21.15.89:0/1551843722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fceec004810 0x7fceec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefe59c700 1 -- 172.21.15.89:0/1551843722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fceec0051c0 msgr2=0x7fceec00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:56.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefe59c700 1 --2- 172.21.15.89:0/1551843722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fceec0051c0 0x7fceec00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefed9d700 1 -- 172.21.15.89:0/1551843722 reap_dead start 2023-10-10T20:39:56.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefe59c700 1 -- 172.21.15.89:0/1551843722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceec003570 con 0x7fceec003a60 2023-10-10T20:39:56.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefdd9b700 1 --2- 172.21.15.89:0/1551843722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fceec004810 0x7fceec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:56.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.482+0000 7fcefe59c700 1 --2- 172.21.15.89:0/1551843722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fceec003a60 0x7fceec003e70 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcef800c3c0 tx=0x7fcef8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677ec0ad473deb69 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:56.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.483+0000 7fcefd59a700 1 -- 172.21.15.89:0/1551843722 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef800cc90 con 0x7fceec003a60 2023-10-10T20:39:56.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.483+0000 7fcefd59a700 1 -- 172.21.15.89:0/1551843722 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef800cdf0 con 0x7fceec003a60 2023-10-10T20:39:56.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.483+0000 7fcefd59a700 1 -- 172.21.15.89:0/1551843722 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef8005b50 con 0x7fceec003a60 2023-10-10T20:39:56.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.483+0000 7fcf04a94700 1 -- 172.21.15.89:0/1551843722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fceec003a60 msgr2=0x7fceec003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:56.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.483+0000 7fcf04a94700 1 --2- 172.21.15.89:0/1551843722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fceec003a60 0x7fceec003e70 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcef800c3c0 tx=0x7fcef8001e50 comp rx=0 tx=0).stop 2023-10-10T20:39:56.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.483+0000 7fcf04a94700 1 -- 172.21.15.89:0/1551843722 shutdown_connections 2023-10-10T20:39:56.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.483+0000 7fcf04a94700 1 --2- 172.21.15.89:0/1551843722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fceec003a60 0x7fceec003e70 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.483+0000 7fcf04a94700 1 -- 172.21.15.89:0/1551843722 >> 172.21.15.89:0/1551843722 conn(0x7fcf00068e20 msgr2=0x7fcf000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:56.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.483+0000 7fcf04a94700 1 -- 172.21.15.89:0/1551843722 shutdown_connections 2023-10-10T20:39:56.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.483+0000 7fcf04a94700 1 -- 172.21.15.89:0/1551843722 wait complete. 2023-10-10T20:39:56.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.484+0000 7fcf04a94700 1 Processor -- start 2023-10-10T20:39:56.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.484+0000 7fcf04a94700 1 -- start start 2023-10-10T20:39:56.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.485+0000 7fcf04a94700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fceec003a60 0x7fcf00151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.485+0000 7fcf04a94700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcf0015f680 0x7fcf0015faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.485+0000 7fcf04a94700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf000b41d0 0x7fcf000b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.485+0000 7fcf04a94700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000c7f10 con 0x7fceec003a60 2023-10-10T20:39:56.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.485+0000 7fcf04a94700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf00160030 con 0x7fcf000b41d0 2023-10-10T20:39:56.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.485+0000 7fcf04a94700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000b57a0 con 0x7fcf0015f680 2023-10-10T20:39:56.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.485+0000 7fcefe59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fceec003a60 0x7fcf00151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:56.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.485+0000 7fcefe59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fceec003a60 0x7fcf00151f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49178/0 (socket says 172.21.15.89:49178) 2023-10-10T20:39:56.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.485+0000 7fcefe59c700 1 -- 172.21.15.89:0/2150752430 learned_addr learned my addr 172.21.15.89:0/2150752430 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:56.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.485+0000 7fcefe59c700 1 -- 172.21.15.89:0/2150752430 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcf0015f680 msgr2=0x7fcf0015faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:56.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.485+0000 7fcefe59c700 1 --2- 172.21.15.89:0/2150752430 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcf0015f680 0x7fcf0015faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.485+0000 7fcefe59c700 1 -- 172.21.15.89:0/2150752430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf000b41d0 msgr2=0x7fcf000b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:56.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.485+0000 7fcefe59c700 1 --2- 172.21.15.89:0/2150752430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf000b41d0 0x7fcf000b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.485+0000 7fcefe59c700 1 -- 172.21.15.89:0/2150752430 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf000b4f00 con 0x7fceec003a60 2023-10-10T20:39:56.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.485+0000 7fcefe59c700 1 --2- 172.21.15.89:0/2150752430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fceec003a60 0x7fcf00151f60 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fceec00c1e0 tx=0x7fcef8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:56.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.486+0000 7fcef67fc700 1 -- 172.21.15.89:0/2150752430 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef80070d0 con 0x7fceec003a60 2023-10-10T20:39:56.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.486+0000 7fcef67fc700 1 -- 172.21.15.89:0/2150752430 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef8007290 con 0x7fceec003a60 2023-10-10T20:39:56.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.486+0000 7fcef67fc700 1 -- 172.21.15.89:0/2150752430 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef8007650 con 0x7fceec003a60 2023-10-10T20:39:56.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.486+0000 7fcf04a94700 1 -- 172.21.15.89:0/2150752430 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf000b51e0 con 0x7fceec003a60 2023-10-10T20:39:56.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.486+0000 7fcf04a94700 1 -- 172.21.15.89:0/2150752430 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf000c8140 con 0x7fceec003a60 2023-10-10T20:39:56.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.487+0000 7fcf04a94700 1 -- 172.21.15.89:0/2150752430 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf0004c3c0 con 0x7fceec003a60 2023-10-10T20:39:56.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.487+0000 7fcef67fc700 1 -- 172.21.15.89:0/2150752430 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fcef8028460 con 0x7fceec003a60 2023-10-10T20:39:56.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.487+0000 7fcef67fc700 1 --2- 172.21.15.89:0/2150752430 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcee0067200 0x7fcee00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.487+0000 7fcefdd9b700 1 --2- 172.21.15.89:0/2150752430 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcee0067200 0x7fcee00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:56.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.487+0000 7fcef67fc700 1 -- 172.21.15.89:0/2150752430 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcef8086220 con 0x7fceec003a60 2023-10-10T20:39:56.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.488+0000 7fcefdd9b700 1 --2- 172.21.15.89:0/2150752430 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcee0067200 0x7fcee00696b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcee8006dc0 tx=0x7fcee8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:56.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.492+0000 7fcef67fc700 1 -- 172.21.15.89:0/2150752430 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcef8052f30 con 0x7fceec003a60 2023-10-10T20:39:56.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.716+0000 7fcf04a94700 1 -- 172.21.15.89:0/2150752430 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf000896d0 con 0x7fceec003a60 2023-10-10T20:39:56.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.716+0000 7fcef67fc700 1 -- 172.21.15.89:0/2150752430 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fcef8056550 con 0x7fceec003a60 2023-10-10T20:39:56.719 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:56.719 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5324,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/1607893755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":1607893755},{"type":"v1","addr":"172.21.15.111:6835","nonce":1607893755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6069,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/1480864169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":1480864169},{"type":"v1","addr":"172.21.15.89:6836","nonce":1480864169}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2347053559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2347053559},{"type":"v1","addr":"172.21.15.89:6837","nonce":2347053559}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:39:37.635170+0000","modified":"2023-10-10T20:39:41.271571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6087},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6087":{"gid":6087,"name":"b","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1070916496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1070916496},{"type":"v1","addr":"172.21.15.111:6834","nonce":1070916496}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-10T20:39:56.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.718+0000 7fcf04a94700 1 -- 172.21.15.89:0/2150752430 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcee0067200 msgr2=0x7fcee00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:56.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.718+0000 7fcf04a94700 1 --2- 172.21.15.89:0/2150752430 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcee0067200 0x7fcee00696b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcee8006dc0 tx=0x7fcee8008040 comp rx=0 tx=0).stop 2023-10-10T20:39:56.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.718+0000 7fcf04a94700 1 -- 172.21.15.89:0/2150752430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fceec003a60 msgr2=0x7fcf00151f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:56.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.718+0000 7fcf04a94700 1 --2- 172.21.15.89:0/2150752430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fceec003a60 0x7fcf00151f60 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fceec00c1e0 tx=0x7fcef8015000 comp rx=0 tx=0).stop 2023-10-10T20:39:56.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.718+0000 7fcf04a94700 1 -- 172.21.15.89:0/2150752430 shutdown_connections 2023-10-10T20:39:56.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.718+0000 7fcf04a94700 1 --2- 172.21.15.89:0/2150752430 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcee0067200 0x7fcee00696b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.718+0000 7fcf04a94700 1 --2- 172.21.15.89:0/2150752430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf000b41d0 0x7fcf000b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.718+0000 7fcf04a94700 1 --2- 172.21.15.89:0/2150752430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fceec003a60 0x7fcf00151f60 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.718+0000 7fcf04a94700 1 --2- 172.21.15.89:0/2150752430 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcf0015f680 0x7fcf0015faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.718+0000 7fcf04a94700 1 -- 172.21.15.89:0/2150752430 >> 172.21.15.89:0/2150752430 conn(0x7fcf00068e20 msgr2=0x7fcf000c6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:56.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.718+0000 7fcf04a94700 1 -- 172.21.15.89:0/2150752430 shutdown_connections 2023-10-10T20:39:56.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.719+0000 7fcf04a94700 1 -- 172.21.15.89:0/2150752430 wait complete. 2023-10-10T20:39:56.723 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 47 2023-10-10T20:39:56.733 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:39:56.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.845+0000 7f4c528bd700 1 Processor -- start 2023-10-10T20:39:56.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.845+0000 7f4c528bd700 1 -- start start 2023-10-10T20:39:56.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.846+0000 7f4c528bd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4c4c159c60 0x7f4c4c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.846+0000 7f4c528bd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4c4c15db10 0x7f4c4c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.846+0000 7f4c528bd700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4c15a160 con 0x7f4c4c159890 2023-10-10T20:39:56.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.846+0000 7f4c528bd700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c4c15df50 con 0x7f4c4c15db10 2023-10-10T20:39:56.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.846+0000 7f4c528bd700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c4c0d13a0 con 0x7f4c4c159c60 2023-10-10T20:39:56.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.846+0000 7f4c4bfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4c4c159890 0x7f4c4c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:60800/0 (socket says 172.21.15.89:60800) 2023-10-10T20:39:56.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.846+0000 7f4c4bfff700 1 -- 172.21.15.89:0/541158638 learned_addr learned my addr 172.21.15.89:0/541158638 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:56.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.846+0000 7f4c50e5a700 1 --2- 172.21.15.89:0/541158638 >> v2:172.21.15.89:3300/0 conn(0x7f4c4c15db10 0x7f4c4c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:56.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.847+0000 7f4c4b7fe700 1 --2- 172.21.15.89:0/541158638 >> v2:172.21.15.111:3300/0 conn(0x7f4c4c159c60 0x7f4c4c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:56.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.847+0000 7f4c4affd700 1 -- 172.21.15.89:0/541158638 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3834474077 0 0) 0x7f4c4c15a160 con 0x7f4c4c159890 2023-10-10T20:39:56.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.847+0000 7f4c4affd700 1 -- 172.21.15.89:0/541158638 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c34003570 con 0x7f4c4c159890 2023-10-10T20:39:56.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.847+0000 7f4c50e5a700 1 -- 172.21.15.89:0/541158638 >> v2:172.21.15.111:3300/0 conn(0x7f4c4c159c60 msgr2=0x7f4c4c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:56.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.847+0000 7f4c50e5a700 1 --2- 172.21.15.89:0/541158638 >> v2:172.21.15.111:3300/0 conn(0x7f4c4c159c60 0x7f4c4c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.847+0000 7f4c50e5a700 1 -- 172.21.15.89:0/541158638 >> v1:172.21.15.89:6789/0 conn(0x7f4c4c159890 legacy=0x7f4c4c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:56.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.847+0000 7f4c50e5a700 1 -- 172.21.15.89:0/541158638 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c15e140 con 0x7f4c4c15db10 2023-10-10T20:39:56.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.847+0000 7f4c50e5a700 1 --2- 172.21.15.89:0/541158638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c4c15db10 0x7f4c4c069560 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0b67f0 tx=0x7f4c44009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4a8819dd315432 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:56.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.847+0000 7f4c4affd700 1 -- 172.21.15.89:0/541158638 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4400e070 con 0x7f4c4c15db10 2023-10-10T20:39:56.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.847+0000 7f4c4affd700 1 -- 172.21.15.89:0/541158638 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4400bb60 con 0x7f4c4c15db10 2023-10-10T20:39:56.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.847+0000 7f4c4affd700 1 -- 172.21.15.89:0/541158638 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c44012430 con 0x7f4c4c15db10 2023-10-10T20:39:56.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.848+0000 7f4c528bd700 1 -- 172.21.15.89:0/541158638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c4c15db10 msgr2=0x7f4c4c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:56.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.848+0000 7f4c528bd700 1 --2- 172.21.15.89:0/541158638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c4c15db10 0x7f4c4c069560 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0b67f0 tx=0x7f4c44009780 comp rx=0 tx=0).stop 2023-10-10T20:39:56.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.848+0000 7f4c528bd700 1 -- 172.21.15.89:0/541158638 shutdown_connections 2023-10-10T20:39:56.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.848+0000 7f4c528bd700 1 --2- 172.21.15.89:0/541158638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c4c15db10 0x7f4c4c069560 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.848+0000 7f4c528bd700 1 --2- 172.21.15.89:0/541158638 >> v2:172.21.15.111:3300/0 conn(0x7f4c4c159c60 0x7f4c4c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.848+0000 7f4c528bd700 1 -- 172.21.15.89:0/541158638 >> 172.21.15.89:0/541158638 conn(0x7f4c4c068e20 msgr2=0x7f4c4c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:56.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.848+0000 7f4c528bd700 1 -- 172.21.15.89:0/541158638 shutdown_connections 2023-10-10T20:39:56.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.848+0000 7f4c528bd700 1 -- 172.21.15.89:0/541158638 wait complete. 2023-10-10T20:39:56.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.849+0000 7f4c528bd700 1 Processor -- start 2023-10-10T20:39:56.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.849+0000 7f4c528bd700 1 -- start start 2023-10-10T20:39:56.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c528bd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c4c159890 0x7f4c4c0b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c528bd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c4c159c60 0x7f4c4c0c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c528bd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c4c15db10 0x7f4c4c0c7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c528bd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c16cdc0 con 0x7f4c4c159c60 2023-10-10T20:39:56.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c528bd700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c0cd430 con 0x7f4c4c15db10 2023-10-10T20:39:56.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c528bd700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c0cd710 con 0x7f4c4c159890 2023-10-10T20:39:56.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c4b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c4c159c60 0x7f4c4c0c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:56.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c4b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c4c159c60 0x7f4c4c0c6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49192/0 (socket says 172.21.15.89:49192) 2023-10-10T20:39:56.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c4b7fe700 1 -- 172.21.15.89:0/3774845441 learned_addr learned my addr 172.21.15.89:0/3774845441 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:56.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c50e5a700 1 --2- 172.21.15.89:0/3774845441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c4c15db10 0x7f4c4c0c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:56.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c4bfff700 1 --2- 172.21.15.89:0/3774845441 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c4c159890 0x7f4c4c0b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:56.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c4b7fe700 1 -- 172.21.15.89:0/3774845441 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c4c159890 msgr2=0x7f4c4c0b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:56.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c4b7fe700 1 --2- 172.21.15.89:0/3774845441 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c4c159890 0x7f4c4c0b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c4b7fe700 1 -- 172.21.15.89:0/3774845441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c4c15db10 msgr2=0x7f4c4c0c7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:56.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c4b7fe700 1 --2- 172.21.15.89:0/3774845441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c4c15db10 0x7f4c4c0c7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:56.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c4b7fe700 1 -- 172.21.15.89:0/3774845441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c44009a20 con 0x7f4c4c159c60 2023-10-10T20:39:56.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c4b7fe700 1 --2- 172.21.15.89:0/3774845441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c4c159c60 0x7f4c4c0c6b50 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000d890 tx=0x7f4c40008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:56.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c33fff700 1 -- 172.21.15.89:0/3774845441 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4000b600 con 0x7f4c4c159c60 2023-10-10T20:39:56.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c33fff700 1 -- 172.21.15.89:0/3774845441 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4000b760 con 0x7f4c4c159c60 2023-10-10T20:39:56.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c528bd700 1 -- 172.21.15.89:0/3774845441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4c0cd9f0 con 0x7f4c4c159c60 2023-10-10T20:39:56.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c33fff700 1 -- 172.21.15.89:0/3774845441 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4000bae0 con 0x7f4c4c159c60 2023-10-10T20:39:56.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.850+0000 7f4c528bd700 1 -- 172.21.15.89:0/3774845441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4c0cd0c0 con 0x7f4c4c159c60 2023-10-10T20:39:56.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.851+0000 7f4c528bd700 1 -- 172.21.15.89:0/3774845441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4c04c3c0 con 0x7f4c4c159c60 2023-10-10T20:39:56.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.852+0000 7f4c33fff700 1 -- 172.21.15.89:0/3774845441 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f4c4001e460 con 0x7f4c4c159c60 2023-10-10T20:39:56.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.852+0000 7f4c33fff700 1 --2- 172.21.15.89:0/3774845441 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c38067200 0x7f4c380696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:56.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.852+0000 7f4c4bfff700 1 --2- 172.21.15.89:0/3774845441 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c38067200 0x7f4c380696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:56.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.852+0000 7f4c33fff700 1 -- 172.21.15.89:0/3774845441 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4c40085060 con 0x7f4c4c159c60 2023-10-10T20:39:56.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.853+0000 7f4c4bfff700 1 --2- 172.21.15.89:0/3774845441 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c38067200 0x7f4c380696b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0058d0 tx=0x7f4c3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:56.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:56.856+0000 7f4c33fff700 1 -- 172.21.15.89:0/3774845441 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c40051f80 con 0x7f4c4c159c60 2023-10-10T20:39:57.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.044+0000 7f4c528bd700 1 -- 172.21.15.89:0/3774845441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c4c154ac0 con 0x7f4c4c159c60 2023-10-10T20:39:57.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.044+0000 7f4c33fff700 1 -- 172.21.15.89:0/3774845441 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+15353 (secure 0 0 0) 0x7f4c400555a0 con 0x7f4c4c159c60 2023-10-10T20:39:57.047 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:39:57.047 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":76,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:39:39.250746+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:39:35.989512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-10T20:39:36.595551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:39:57.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.046+0000 7f4c528bd700 1 -- 172.21.15.89:0/3774845441 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c38067200 msgr2=0x7f4c380696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:57.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.046+0000 7f4c528bd700 1 --2- 172.21.15.89:0/3774845441 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c38067200 0x7f4c380696b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0058d0 tx=0x7f4c3c009040 comp rx=0 tx=0).stop 2023-10-10T20:39:57.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.046+0000 7f4c528bd700 1 -- 172.21.15.89:0/3774845441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c4c159c60 msgr2=0x7f4c4c0c6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:57.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.046+0000 7f4c528bd700 1 --2- 172.21.15.89:0/3774845441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c4c159c60 0x7f4c4c0c6b50 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000d890 tx=0x7f4c40008040 comp rx=0 tx=0).stop 2023-10-10T20:39:57.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.046+0000 7f4c528bd700 1 -- 172.21.15.89:0/3774845441 shutdown_connections 2023-10-10T20:39:57.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.046+0000 7f4c528bd700 1 --2- 172.21.15.89:0/3774845441 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c38067200 0x7f4c380696b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.046+0000 7f4c528bd700 1 --2- 172.21.15.89:0/3774845441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c4c15db10 0x7f4c4c0c7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.046+0000 7f4c528bd700 1 --2- 172.21.15.89:0/3774845441 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c4c159890 0x7f4c4c0b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.046+0000 7f4c528bd700 1 --2- 172.21.15.89:0/3774845441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c4c159c60 0x7f4c4c0c6b50 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.046+0000 7f4c528bd700 1 -- 172.21.15.89:0/3774845441 >> 172.21.15.89:0/3774845441 conn(0x7f4c4c068e20 msgr2=0x7f4c4c149c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:57.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.046+0000 7f4c528bd700 1 -- 172.21.15.89:0/3774845441 shutdown_connections 2023-10-10T20:39:57.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.046+0000 7f4c528bd700 1 -- 172.21.15.89:0/3774845441 wait complete. 2023-10-10T20:39:57.061 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:39:57.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.174+0000 7f6bc0759700 1 Processor -- start 2023-10-10T20:39:57.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.174+0000 7f6bc0759700 1 -- start start 2023-10-10T20:39:57.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.175+0000 7f6bc0759700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bb80aca80 0x7f6bb806aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:57.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.175+0000 7f6bc0759700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb806bdb0 con 0x7f6bb80c83c0 2023-10-10T20:39:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bc0759700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb80cb370 con 0x7f6bb8069d60 2023-10-10T20:39:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bc0759700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb8159f60 con 0x7f6bb80aca80 2023-10-10T20:39:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbecf6700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6bb80c83c0 0x7f6bb80cb800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:60802/0 (socket says 172.21.15.89:60802) 2023-10-10T20:39:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbecf6700 1 -- 172.21.15.89:0/813738 learned_addr learned my addr 172.21.15.89:0/813738 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbe4f5700 1 --2- 172.21.15.89:0/813738 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bb80aca80 0x7f6bb806aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbd4f3700 1 -- 172.21.15.89:0/813738 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221101813 0 0) 0x7f6bb806bdb0 con 0x7f6bb80c83c0 2023-10-10T20:39:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbd4f3700 1 -- 172.21.15.89:0/813738 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ba4003570 con 0x7f6bb80c83c0 2023-10-10T20:39:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbd4f3700 1 -- 172.21.15.89:0/813738 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2092480758 0 0) 0x7f6bb80cb370 con 0x7f6bb8069d60 2023-10-10T20:39:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbd4f3700 1 -- 172.21.15.89:0/813738 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bb806bdb0 con 0x7f6bb8069d60 2023-10-10T20:39:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbd4f3700 1 -- 172.21.15.89:0/813738 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2598231278 0 0) 0x7f6ba4003570 con 0x7f6bb80c83c0 2023-10-10T20:39:57.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbd4f3700 1 -- 172.21.15.89:0/813738 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bb80aca80 msgr2=0x7f6bb806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:57.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbd4f3700 1 --2- 172.21.15.89:0/813738 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bb80aca80 0x7f6bb806aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbd4f3700 1 -- 172.21.15.89:0/813738 >> v1:172.21.15.111:6789/0 conn(0x7f6bb8069d60 legacy=0x7f6bb80c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:57.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbd4f3700 1 -- 172.21.15.89:0/813738 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb806b510 con 0x7f6bb80c83c0 2023-10-10T20:39:57.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbd4f3700 1 -- 172.21.15.89:0/813738 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f6bb00017f0 con 0x7f6bb80c83c0 2023-10-10T20:39:57.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbd4f3700 1 -- 172.21.15.89:0/813738 >> v1:172.21.15.89:6789/0 conn(0x7f6bb80c83c0 legacy=0x7f6bb80cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:57.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbd4f3700 1 --2- 172.21.15.89:0/813738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ba4003c20 0x7f6ba4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:57.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbd4f3700 1 --2- 172.21.15.89:0/813738 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bb80aca80 0x7f6ba4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:57.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbd4f3700 1 --2- 172.21.15.89:0/813738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba4004d50 0x7f6ba40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:57.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbd4f3700 1 -- 172.21.15.89:0/813738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb8159f60 con 0x7f6ba4004d50 2023-10-10T20:39:57.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbd4f3700 1 -- 172.21.15.89:0/813738 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba400bbf0 con 0x7f6ba4003c20 2023-10-10T20:39:57.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbd4f3700 1 -- 172.21.15.89:0/813738 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba400bd60 con 0x7f6bb80aca80 2023-10-10T20:39:57.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbecf6700 1 --2- 172.21.15.89:0/813738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba4004d50 0x7f6ba40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:57.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbe4f5700 1 --2- 172.21.15.89:0/813738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ba4003c20 0x7f6ba4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:57.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.176+0000 7f6bbdcf4700 1 --2- 172.21.15.89:0/813738 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bb80aca80 0x7f6ba4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:57.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.177+0000 7f6bbecf6700 1 -- 172.21.15.89:0/813738 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bb80aca80 msgr2=0x7f6ba4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:57.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.177+0000 7f6bbecf6700 1 --2- 172.21.15.89:0/813738 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bb80aca80 0x7f6ba4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.177+0000 7f6bbecf6700 1 -- 172.21.15.89:0/813738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ba4003c20 msgr2=0x7f6ba4004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:57.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.177+0000 7f6bbecf6700 1 --2- 172.21.15.89:0/813738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ba4003c20 0x7f6ba4004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.177+0000 7f6bbecf6700 1 -- 172.21.15.89:0/813738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba4003570 con 0x7f6ba4004d50 2023-10-10T20:39:57.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.177+0000 7f6bbecf6700 1 --2- 172.21.15.89:0/813738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba4004d50 0x7f6ba40051c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6bb806f890 tx=0x7f6bb00020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad9d0e158dc8220 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:57.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.177+0000 7f6bbd4f3700 1 -- 172.21.15.89:0/813738 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb00070d0 con 0x7f6ba4004d50 2023-10-10T20:39:57.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.177+0000 7f6bbd4f3700 1 -- 172.21.15.89:0/813738 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb0007230 con 0x7f6ba4004d50 2023-10-10T20:39:57.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.177+0000 7f6bbd4f3700 1 -- 172.21.15.89:0/813738 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb00075b0 con 0x7f6ba4004d50 2023-10-10T20:39:57.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.177+0000 7f6bc0759700 1 -- 172.21.15.89:0/813738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba4004d50 msgr2=0x7f6ba40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:57.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.177+0000 7f6bc0759700 1 --2- 172.21.15.89:0/813738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba4004d50 0x7f6ba40051c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6bb806f890 tx=0x7f6bb00020c0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.177+0000 7f6bbecf6700 1 -- 172.21.15.89:0/813738 reap_dead start 2023-10-10T20:39:57.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.178+0000 7f6bc0759700 1 -- 172.21.15.89:0/813738 shutdown_connections 2023-10-10T20:39:57.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.178+0000 7f6bc0759700 1 -- 172.21.15.89:0/813738 >> 172.21.15.89:0/813738 conn(0x7f6bb80cd670 msgr2=0x7f6bb80cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:57.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.178+0000 7f6bc0759700 1 -- 172.21.15.89:0/813738 shutdown_connections 2023-10-10T20:39:57.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.178+0000 7f6bc0759700 1 -- 172.21.15.89:0/813738 wait complete. 2023-10-10T20:39:57.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.179+0000 7f6bc0759700 1 Processor -- start 2023-10-10T20:39:57.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.179+0000 7f6bc0759700 1 -- start start 2023-10-10T20:39:57.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.179+0000 7f6bc0759700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba4004d50 0x7f6bb8155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:57.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.179+0000 7f6bc0759700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bb81561f0 0x7f6bb8156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:57.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.179+0000 7f6bc0759700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bb8157ce0 0x7f6bb8158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:57.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.179+0000 7f6bc0759700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb8156070 con 0x7f6ba4004d50 2023-10-10T20:39:57.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.179+0000 7f6bc0759700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb81586c0 con 0x7f6bb8157ce0 2023-10-10T20:39:57.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.179+0000 7f6bc0759700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb8156c70 con 0x7f6bb81561f0 2023-10-10T20:39:57.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.179+0000 7f6bbe4f5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba4004d50 0x7f6bb8155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:57.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.179+0000 7f6bbe4f5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba4004d50 0x7f6bb8155b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49210/0 (socket says 172.21.15.89:49210) 2023-10-10T20:39:57.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.179+0000 7f6bbe4f5700 1 -- 172.21.15.89:0/2164807442 learned_addr learned my addr 172.21.15.89:0/2164807442 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:57.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.180+0000 7f6bbecf6700 1 --2- 172.21.15.89:0/2164807442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bb8157ce0 0x7f6bb8158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:57.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.180+0000 7f6bbe4f5700 1 -- 172.21.15.89:0/2164807442 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bb81561f0 msgr2=0x7f6bb8156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:57.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.180+0000 7f6bbe4f5700 1 --2- 172.21.15.89:0/2164807442 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bb81561f0 0x7f6bb8156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.180+0000 7f6bbe4f5700 1 -- 172.21.15.89:0/2164807442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bb8157ce0 msgr2=0x7f6bb8158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:57.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.180+0000 7f6bbe4f5700 1 --2- 172.21.15.89:0/2164807442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bb8157ce0 0x7f6bb8158180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.180+0000 7f6bbe4f5700 1 -- 172.21.15.89:0/2164807442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb8156f50 con 0x7f6ba4004d50 2023-10-10T20:39:57.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.180+0000 7f6bbecf6700 1 --2- 172.21.15.89:0/2164807442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bb8157ce0 0x7f6bb8158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:57.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.180+0000 7f6bbe4f5700 1 --2- 172.21.15.89:0/2164807442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba4004d50 0x7f6bb8155b30 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4011b10 tx=0x7f6bb4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:57.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.180+0000 7f6bae7fc700 1 -- 172.21.15.89:0/2164807442 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4005c10 con 0x7f6ba4004d50 2023-10-10T20:39:57.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.180+0000 7f6bae7fc700 1 -- 172.21.15.89:0/2164807442 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb4005d70 con 0x7f6ba4004d50 2023-10-10T20:39:57.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.180+0000 7f6bc0759700 1 -- 172.21.15.89:0/2164807442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb8157230 con 0x7f6ba4004d50 2023-10-10T20:39:57.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.180+0000 7f6bae7fc700 1 -- 172.21.15.89:0/2164807442 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb400fa10 con 0x7f6ba4004d50 2023-10-10T20:39:57.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.180+0000 7f6bc0759700 1 -- 172.21.15.89:0/2164807442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb80c7b50 con 0x7f6ba4004d50 2023-10-10T20:39:57.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.181+0000 7f6bc0759700 1 -- 172.21.15.89:0/2164807442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb8157750 con 0x7f6ba4004d50 2023-10-10T20:39:57.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.181+0000 7f6bae7fc700 1 -- 172.21.15.89:0/2164807442 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f6bb4006050 con 0x7f6ba4004d50 2023-10-10T20:39:57.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.182+0000 7f6bae7fc700 1 --2- 172.21.15.89:0/2164807442 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6b9c067200 0x7f6b9c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:57.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.182+0000 7f6bae7fc700 1 -- 172.21.15.89:0/2164807442 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6bb4085510 con 0x7f6ba4004d50 2023-10-10T20:39:57.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.182+0000 7f6bbdcf4700 1 --2- 172.21.15.89:0/2164807442 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6b9c067200 0x7f6b9c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:57.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.182+0000 7f6bbdcf4700 1 --2- 172.21.15.89:0/2164807442 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6b9c067200 0x7f6b9c0696b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8007e40 tx=0x7f6ba8007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:57.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.186+0000 7f6bae7fc700 1 -- 172.21.15.89:0/2164807442 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6bb8157750 con 0x7f6ba4004d50 2023-10-10T20:39:57.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.411+0000 7f6bc0759700 1 -- 172.21.15.89:0/2164807442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6bb8157750 con 0x7f6ba4004d50 2023-10-10T20:39:57.413 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-10-10T20:39:57.411+0000 7fd0184ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:39:57.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.415+0000 7f6bae7fc700 1 -- 172.21.15.89:0/2164807442 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6bb8157750 con 0x7f6ba4004d50 2023-10-10T20:39:57.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.417+0000 7f6bc0759700 1 -- 172.21.15.89:0/2164807442 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6b9c067200 msgr2=0x7f6b9c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:57.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.417+0000 7f6bc0759700 1 --2- 172.21.15.89:0/2164807442 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6b9c067200 0x7f6b9c0696b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8007e40 tx=0x7f6ba8007d30 comp rx=0 tx=0).stop 2023-10-10T20:39:57.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.417+0000 7f6bc0759700 1 -- 172.21.15.89:0/2164807442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba4004d50 msgr2=0x7f6bb8155b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:57.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.417+0000 7f6bc0759700 1 --2- 172.21.15.89:0/2164807442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba4004d50 0x7f6bb8155b30 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4011b10 tx=0x7f6bb4009120 comp rx=0 tx=0).stop 2023-10-10T20:39:57.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.417+0000 7f6bc0759700 1 -- 172.21.15.89:0/2164807442 shutdown_connections 2023-10-10T20:39:57.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.417+0000 7f6bc0759700 1 --2- 172.21.15.89:0/2164807442 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6b9c067200 0x7f6b9c0696b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.417+0000 7f6bc0759700 1 --2- 172.21.15.89:0/2164807442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bb8157ce0 0x7f6bb8158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.417+0000 7f6bc0759700 1 --2- 172.21.15.89:0/2164807442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba4004d50 0x7f6bb8155b30 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.417+0000 7f6bc0759700 1 --2- 172.21.15.89:0/2164807442 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bb81561f0 0x7f6bb8156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.417+0000 7f6bc0759700 1 -- 172.21.15.89:0/2164807442 >> 172.21.15.89:0/2164807442 conn(0x7f6bb80cd670 msgr2=0x7f6bb80cce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:57.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.417+0000 7f6bc0759700 1 -- 172.21.15.89:0/2164807442 shutdown_connections 2023-10-10T20:39:57.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.417+0000 7f6bc0759700 1 -- 172.21.15.89:0/2164807442 wait complete. 2023-10-10T20:39:57.422 INFO:tasks.ceph.mds.b.smithi111.stderr: -1887> 2023-10-10T20:39:43.556+0000 7f03c0505700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:39:57.422 INFO:teuthology.orchestra.run.smithi089.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:39:57.431 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:39:57.442 INFO:tasks.ceph.mds.b.smithi111.stdout:starting mds.b at 2023-10-10T20:39:57.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.546+0000 7f20d694c700 1 Processor -- start 2023-10-10T20:39:57.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.546+0000 7f20d694c700 1 -- start start 2023-10-10T20:39:57.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.547+0000 7f20d694c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20d00b4e70 0x7f20d00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:57.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.547+0000 7f20d694c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f20d00b16d0 0x7f20d00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:57.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.547+0000 7f20d694c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f20d015d240 0x7f20d015d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:57.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.547+0000 7f20d694c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d00b2080 con 0x7f20d00b4e70 2023-10-10T20:39:57.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.547+0000 7f20d694c700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d00cfb00 con 0x7f20d015d240 2023-10-10T20:39:57.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.547+0000 7f20d694c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d00b5c40 con 0x7f20d00b16d0 2023-10-10T20:39:57.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.547+0000 7f20d594a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20d00b4e70 0x7f20d00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:57.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.547+0000 7f20d594a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20d00b4e70 0x7f20d00ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49220/0 (socket says 172.21.15.89:49220) 2023-10-10T20:39:57.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.547+0000 7f20d594a700 1 -- 172.21.15.89:0/3588431693 learned_addr learned my addr 172.21.15.89:0/3588431693 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:57.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.547+0000 7f20d5149700 1 --2- 172.21.15.89:0/3588431693 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f20d00b16d0 0x7f20d00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:57.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.547+0000 7f20d614b700 1 --2- 172.21.15.89:0/3588431693 >> v2:172.21.15.111:3300/0 conn(0x7f20d015d240 0x7f20d015d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:57.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.547+0000 7f20d594a700 1 -- 172.21.15.89:0/3588431693 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f20d00b16d0 msgr2=0x7f20d00b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:57.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.547+0000 7f20d594a700 1 --2- 172.21.15.89:0/3588431693 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f20d00b16d0 0x7f20d00b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.547+0000 7f20d594a700 1 -- 172.21.15.89:0/3588431693 >> v2:172.21.15.111:3300/0 conn(0x7f20d015d240 msgr2=0x7f20d015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:57.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.547+0000 7f20d594a700 1 --2- 172.21.15.89:0/3588431693 >> v2:172.21.15.111:3300/0 conn(0x7f20d015d240 0x7f20d015d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.547+0000 7f20d594a700 1 -- 172.21.15.89:0/3588431693 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d00b5f20 con 0x7f20d00b4e70 2023-10-10T20:39:57.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.547+0000 7f20d614b700 1 --2- 172.21.15.89:0/3588431693 >> v2:172.21.15.111:3300/0 conn(0x7f20d015d240 0x7f20d015d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:39:57.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.548+0000 7f20d594a700 1 --2- 172.21.15.89:0/3588431693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20d00b4e70 0x7f20d00ccff0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f20d015d180 tx=0x7f20cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ba54ef0955a5c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:57.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.548+0000 7f20d4948700 1 -- 172.21.15.89:0/3588431693 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc00b860 con 0x7f20d00b4e70 2023-10-10T20:39:57.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.548+0000 7f20d4948700 1 -- 172.21.15.89:0/3588431693 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20cc00b9c0 con 0x7f20d00b4e70 2023-10-10T20:39:57.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.548+0000 7f20d4948700 1 -- 172.21.15.89:0/3588431693 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc00bd60 con 0x7f20d00b4e70 2023-10-10T20:39:57.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.548+0000 7f20d694c700 1 -- 172.21.15.89:0/3588431693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20d00b4e70 msgr2=0x7f20d00ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:57.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.548+0000 7f20d694c700 1 --2- 172.21.15.89:0/3588431693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20d00b4e70 0x7f20d00ccff0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f20d015d180 tx=0x7f20cc009250 comp rx=0 tx=0).stop 2023-10-10T20:39:57.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.548+0000 7f20d694c700 1 -- 172.21.15.89:0/3588431693 shutdown_connections 2023-10-10T20:39:57.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.548+0000 7f20d694c700 1 --2- 172.21.15.89:0/3588431693 >> v2:172.21.15.111:3300/0 conn(0x7f20d015d240 0x7f20d015d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.548+0000 7f20d694c700 1 --2- 172.21.15.89:0/3588431693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20d00b4e70 0x7f20d00ccff0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.548+0000 7f20d694c700 1 --2- 172.21.15.89:0/3588431693 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f20d00b16d0 0x7f20d00b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.548+0000 7f20d694c700 1 -- 172.21.15.89:0/3588431693 >> 172.21.15.89:0/3588431693 conn(0x7f20d00d10c0 msgr2=0x7f20d00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:57.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.548+0000 7f20d694c700 1 -- 172.21.15.89:0/3588431693 shutdown_connections 2023-10-10T20:39:57.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.548+0000 7f20d694c700 1 -- 172.21.15.89:0/3588431693 wait complete. 2023-10-10T20:39:57.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.549+0000 7f20d694c700 1 Processor -- start 2023-10-10T20:39:57.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.549+0000 7f20d694c700 1 -- start start 2023-10-10T20:39:57.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.550+0000 7f20d694c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20d00b16d0 0x7f20d0149130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:57.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.550+0000 7f20d694c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20d00b4e70 0x7f20d00c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:57.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.550+0000 7f20d694c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f20d015d240 0x7f20d00ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:57.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.550+0000 7f20d694c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d0149670 con 0x7f20d00b16d0 2023-10-10T20:39:57.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.550+0000 7f20d694c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d016b300 con 0x7f20d00b4e70 2023-10-10T20:39:57.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.550+0000 7f20d694c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d016b5e0 con 0x7f20d015d240 2023-10-10T20:39:57.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.550+0000 7f20d594a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20d00b16d0 0x7f20d0149130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:57.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.550+0000 7f20d594a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20d00b16d0 0x7f20d0149130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49228/0 (socket says 172.21.15.89:49228) 2023-10-10T20:39:57.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.550+0000 7f20d594a700 1 -- 172.21.15.89:0/1568467894 learned_addr learned my addr 172.21.15.89:0/1568467894 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:57.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.550+0000 7f20d5149700 1 --2- 172.21.15.89:0/1568467894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20d00b4e70 0x7f20d00c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:57.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.550+0000 7f20d614b700 1 --2- 172.21.15.89:0/1568467894 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f20d015d240 0x7f20d00ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:57.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.551+0000 7f20d594a700 1 -- 172.21.15.89:0/1568467894 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f20d015d240 msgr2=0x7f20d00ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:57.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.551+0000 7f20d594a700 1 --2- 172.21.15.89:0/1568467894 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f20d015d240 0x7f20d00ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.551+0000 7f20d594a700 1 -- 172.21.15.89:0/1568467894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20d00b4e70 msgr2=0x7f20d00c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:57.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.551+0000 7f20d594a700 1 --2- 172.21.15.89:0/1568467894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20d00b4e70 0x7f20d00c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:57.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.551+0000 7f20d594a700 1 -- 172.21.15.89:0/1568467894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc0094f0 con 0x7f20d00b16d0 2023-10-10T20:39:57.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.551+0000 7f20d614b700 1 --2- 172.21.15.89:0/1568467894 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f20d015d240 0x7f20d00ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:57.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.551+0000 7f20d594a700 1 --2- 172.21.15.89:0/1568467894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20d00b16d0 0x7f20d0149130 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f20d0070dd0 tx=0x7f20cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:57.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.551+0000 7f20c1ffb700 1 -- 172.21.15.89:0/1568467894 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc011e70 con 0x7f20d00b16d0 2023-10-10T20:39:57.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.551+0000 7f20c1ffb700 1 -- 172.21.15.89:0/1568467894 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20cc024440 con 0x7f20d00b16d0 2023-10-10T20:39:57.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.551+0000 7f20d694c700 1 -- 172.21.15.89:0/1568467894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d016b8c0 con 0x7f20d00b16d0 2023-10-10T20:39:57.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.551+0000 7f20c1ffb700 1 -- 172.21.15.89:0/1568467894 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc024800 con 0x7f20d00b16d0 2023-10-10T20:39:57.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.551+0000 7f20d694c700 1 -- 172.21.15.89:0/1568467894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d00c61d0 con 0x7f20d00b16d0 2023-10-10T20:39:57.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.552+0000 7f20d694c700 1 -- 172.21.15.89:0/1568467894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d0088990 con 0x7f20d00b16d0 2023-10-10T20:39:57.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.553+0000 7f20c1ffb700 1 -- 172.21.15.89:0/1568467894 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f20cc01a420 con 0x7f20d00b16d0 2023-10-10T20:39:57.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.553+0000 7f20c1ffb700 1 --2- 172.21.15.89:0/1568467894 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f20b4067200 0x7f20b40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:57.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.553+0000 7f20c1ffb700 1 -- 172.21.15.89:0/1568467894 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f20cc087010 con 0x7f20d00b16d0 2023-10-10T20:39:57.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.553+0000 7f20d5149700 1 --2- 172.21.15.89:0/1568467894 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f20b4067200 0x7f20b40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:57.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.554+0000 7f20d5149700 1 --2- 172.21.15.89:0/1568467894 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f20b4067200 0x7f20b40696b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f20c4006fd0 tx=0x7f20c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:57.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.557+0000 7f20c1ffb700 1 -- 172.21.15.89:0/1568467894 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20cc053f20 con 0x7f20d00b16d0 2023-10-10T20:39:57.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:57.782+0000 7f20d694c700 1 -- 172.21.15.89:0/1568467894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f20d0158b90 con 0x7f20d00b16d0 2023-10-10T20:39:58.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.414+0000 7f20c1ffb700 1 -- 172.21.15.89:0/1568467894 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f20cc057540 con 0x7f20d00b16d0 2023-10-10T20:39:58.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.417+0000 7f20d694c700 1 -- 172.21.15.89:0/1568467894 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f20b4067200 msgr2=0x7f20b40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:58.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.417+0000 7f20d694c700 1 --2- 172.21.15.89:0/1568467894 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f20b4067200 0x7f20b40696b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f20c4006fd0 tx=0x7f20c4008040 comp rx=0 tx=0).stop 2023-10-10T20:39:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.417+0000 7f20d694c700 1 -- 172.21.15.89:0/1568467894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20d00b16d0 msgr2=0x7f20d0149130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.417+0000 7f20d694c700 1 --2- 172.21.15.89:0/1568467894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20d00b16d0 0x7f20d0149130 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f20d0070dd0 tx=0x7f20cc017000 comp rx=0 tx=0).stop 2023-10-10T20:39:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.417+0000 7f20d694c700 1 -- 172.21.15.89:0/1568467894 shutdown_connections 2023-10-10T20:39:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.417+0000 7f20d694c700 1 --2- 172.21.15.89:0/1568467894 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f20d015d240 0x7f20d00ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.417+0000 7f20d694c700 1 --2- 172.21.15.89:0/1568467894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20d00b16d0 0x7f20d0149130 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:58.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.417+0000 7f20d694c700 1 --2- 172.21.15.89:0/1568467894 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f20b4067200 0x7f20b40696b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:58.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.417+0000 7f20d694c700 1 --2- 172.21.15.89:0/1568467894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20d00b4e70 0x7f20d00c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:58.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.417+0000 7f20d694c700 1 -- 172.21.15.89:0/1568467894 >> 172.21.15.89:0/1568467894 conn(0x7f20d00d10c0 msgr2=0x7f20d0158f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:58.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.417+0000 7f20d694c700 1 -- 172.21.15.89:0/1568467894 shutdown_connections 2023-10-10T20:39:58.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.417+0000 7f20d694c700 1 -- 172.21.15.89:0/1568467894 wait complete. 2023-10-10T20:39:58.433 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:39:58.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.549+0000 7f2b03fff700 1 Processor -- start 2023-10-10T20:39:58.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.549+0000 7f2b03fff700 1 -- start start 2023-10-10T20:39:58.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.550+0000 7f2b03fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2b040b0790 0x7f2b040cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:58.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.550+0000 7f2b03fff700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b040cfa00 con 0x7f2b040b03c0 2023-10-10T20:39:58.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.550+0000 7f2b03fff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b040cb580 con 0x7f2b041591e0 2023-10-10T20:39:58.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.550+0000 7f2b03fff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b040cf160 con 0x7f2b040b0790 2023-10-10T20:39:58.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.550+0000 7f2b02ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2b040b03c0 0x7f2b040b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59366/0 (socket says 172.21.15.89:59366) 2023-10-10T20:39:58.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.550+0000 7f2b02ffd700 1 -- 172.21.15.89:0/3249479467 learned_addr learned my addr 172.21.15.89:0/3249479467 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:58.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.550+0000 7f2afa7fc700 1 --2- 172.21.15.89:0/3249479467 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2b040b0790 0x7f2b040cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:58.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.550+0000 7f2b027fc700 1 -- 172.21.15.89:0/3249479467 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2671691127 0 0) 0x7f2b040cfa00 con 0x7f2b040b03c0 2023-10-10T20:39:58.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.550+0000 7f2b027fc700 1 -- 172.21.15.89:0/3249479467 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2af0003570 con 0x7f2b040b03c0 2023-10-10T20:39:58.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b027fc700 1 -- 172.21.15.89:0/3249479467 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2205023308 0 0) 0x7f2af0003570 con 0x7f2b040b03c0 2023-10-10T20:39:58.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b027fc700 1 -- 172.21.15.89:0/3249479467 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2b040b0790 msgr2=0x7f2b040cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:58.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b027fc700 1 --2- 172.21.15.89:0/3249479467 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2b040b0790 0x7f2b040cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:58.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b027fc700 1 -- 172.21.15.89:0/3249479467 >> v1:172.21.15.111:6789/0 conn(0x7f2b041591e0 legacy=0x7f2b04159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:58.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b027fc700 1 -- 172.21.15.89:0/3249479467 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b040cf440 con 0x7f2b040b03c0 2023-10-10T20:39:58.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b027fc700 1 -- 172.21.15.89:0/3249479467 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f2af4001470 con 0x7f2b040b03c0 2023-10-10T20:39:58.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b027fc700 1 -- 172.21.15.89:0/3249479467 >> v1:172.21.15.89:6789/0 conn(0x7f2b040b03c0 legacy=0x7f2b040b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:58.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b027fc700 1 --2- 172.21.15.89:0/3249479467 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2b040b0790 0x7f2af0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:58.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b027fc700 1 --2- 172.21.15.89:0/3249479467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2af0003fd0 0x7f2af0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:58.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b027fc700 1 --2- 172.21.15.89:0/3249479467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af0004d50 0x7f2af00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:58.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b027fc700 1 -- 172.21.15.89:0/3249479467 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b040cf160 con 0x7f2af0004d50 2023-10-10T20:39:58.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b027fc700 1 -- 172.21.15.89:0/3249479467 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af000bb40 con 0x7f2af0003fd0 2023-10-10T20:39:58.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b027fc700 1 -- 172.21.15.89:0/3249479467 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af000bd00 con 0x7f2b040b0790 2023-10-10T20:39:58.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b037fe700 1 --2- 172.21.15.89:0/3249479467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af0004d50 0x7f2af00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:58.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b02ffd700 1 --2- 172.21.15.89:0/3249479467 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2b040b0790 0x7f2af0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:58.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2afa7fc700 1 --2- 172.21.15.89:0/3249479467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2af0003fd0 0x7f2af0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:58.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b037fe700 1 -- 172.21.15.89:0/3249479467 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2b040b0790 msgr2=0x7f2af0003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:58.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b037fe700 1 --2- 172.21.15.89:0/3249479467 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2b040b0790 0x7f2af0003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:58.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b037fe700 1 -- 172.21.15.89:0/3249479467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2af0003fd0 msgr2=0x7f2af0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:58.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b037fe700 1 --2- 172.21.15.89:0/3249479467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2af0003fd0 0x7f2af0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:58.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b037fe700 1 -- 172.21.15.89:0/3249479467 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af0003570 con 0x7f2af0004d50 2023-10-10T20:39:58.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b037fe700 1 --2- 172.21.15.89:0/3249479467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af0004d50 0x7f2af00051c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0020a0 tx=0x7f2afc0016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8e9bf7670a8200 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:58.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b027fc700 1 -- 172.21.15.89:0/3249479467 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc0019e0 con 0x7f2af0004d50 2023-10-10T20:39:58.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b027fc700 1 -- 172.21.15.89:0/3249479467 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2afc00c8b0 con 0x7f2af0004d50 2023-10-10T20:39:58.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.551+0000 7f2b027fc700 1 -- 172.21.15.89:0/3249479467 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc00cc50 con 0x7f2af0004d50 2023-10-10T20:39:58.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.552+0000 7f2b03fff700 1 -- 172.21.15.89:0/3249479467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af0004d50 msgr2=0x7f2af00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:58.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.552+0000 7f2b03fff700 1 --2- 172.21.15.89:0/3249479467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af0004d50 0x7f2af00051c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0020a0 tx=0x7f2afc0016d0 comp rx=0 tx=0).stop 2023-10-10T20:39:58.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.552+0000 7f2b037fe700 1 -- 172.21.15.89:0/3249479467 reap_dead start 2023-10-10T20:39:58.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.552+0000 7f2b03fff700 1 -- 172.21.15.89:0/3249479467 shutdown_connections 2023-10-10T20:39:58.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.552+0000 7f2b03fff700 1 -- 172.21.15.89:0/3249479467 >> 172.21.15.89:0/3249479467 conn(0x7f2b04063230 msgr2=0x7f2b04063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:58.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.552+0000 7f2b03fff700 1 -- 172.21.15.89:0/3249479467 shutdown_connections 2023-10-10T20:39:58.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.552+0000 7f2b03fff700 1 -- 172.21.15.89:0/3249479467 wait complete. 2023-10-10T20:39:58.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.553+0000 7f2b03fff700 1 Processor -- start 2023-10-10T20:39:58.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.553+0000 7f2b03fff700 1 -- start start 2023-10-10T20:39:58.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.554+0000 7f2b03fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2b04069a00 0x7f2b0416d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:58.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.554+0000 7f2b03fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b040c75a0 0x7f2b0415ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:58.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.554+0000 7f2b03fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b0415b160 0x7f2b0415c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:58.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.554+0000 7f2b03fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b040c8ff0 con 0x7f2b0415b160 2023-10-10T20:39:58.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.554+0000 7f2b03fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0416d7f0 con 0x7f2b040c75a0 2023-10-10T20:39:58.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.554+0000 7f2b03fff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b040c7b40 con 0x7f2b04069a00 2023-10-10T20:39:58.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.554+0000 7f2b037fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b0415b160 0x7f2b0415c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:58.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.554+0000 7f2b037fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b0415b160 0x7f2b0415c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54098/0 (socket says 172.21.15.89:54098) 2023-10-10T20:39:58.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.554+0000 7f2b037fe700 1 -- 172.21.15.89:0/632648710 learned_addr learned my addr 172.21.15.89:0/632648710 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:58.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.554+0000 7f2b037fe700 1 -- 172.21.15.89:0/632648710 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2b04069a00 msgr2=0x7f2b0416d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:58.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.554+0000 7f2b02ffd700 1 --2- 172.21.15.89:0/632648710 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2b04069a00 0x7f2b0416d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:58.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.554+0000 7f2b037fe700 1 --2- 172.21.15.89:0/632648710 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2b04069a00 0x7f2b0416d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:58.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.554+0000 7f2afa7fc700 1 --2- 172.21.15.89:0/632648710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b040c75a0 0x7f2b0415ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:58.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.554+0000 7f2b037fe700 1 -- 172.21.15.89:0/632648710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b040c75a0 msgr2=0x7f2b0415ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:58.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.554+0000 7f2b037fe700 1 --2- 172.21.15.89:0/632648710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b040c75a0 0x7f2b0415ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:58.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.554+0000 7f2b037fe700 1 -- 172.21.15.89:0/632648710 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b040c7e20 con 0x7f2b0415b160 2023-10-10T20:39:58.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.555+0000 7f2b037fe700 1 --2- 172.21.15.89:0/632648710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b0415b160 0x7f2b0415c650 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f2afc012000 tx=0x7f2afc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:58.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.555+0000 7f2afb7fe700 1 -- 172.21.15.89:0/632648710 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc0128a0 con 0x7f2b0415b160 2023-10-10T20:39:58.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.555+0000 7f2afb7fe700 1 -- 172.21.15.89:0/632648710 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2afc012a60 con 0x7f2b0415b160 2023-10-10T20:39:58.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.555+0000 7f2b03fff700 1 -- 172.21.15.89:0/632648710 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b040c8100 con 0x7f2b0415b160 2023-10-10T20:39:58.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.555+0000 7f2afb7fe700 1 -- 172.21.15.89:0/632648710 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc012e20 con 0x7f2b0415b160 2023-10-10T20:39:58.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.555+0000 7f2b03fff700 1 -- 172.21.15.89:0/632648710 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b04069dd0 con 0x7f2b0415b160 2023-10-10T20:39:58.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.556+0000 7f2b03fff700 1 -- 172.21.15.89:0/632648710 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b040c9160 con 0x7f2b0415b160 2023-10-10T20:39:58.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.556+0000 7f2afb7fe700 1 -- 172.21.15.89:0/632648710 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f2afc01e640 con 0x7f2b0415b160 2023-10-10T20:39:58.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.556+0000 7f2afb7fe700 1 --2- 172.21.15.89:0/632648710 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2ae4067200 0x7f2ae40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:58.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.557+0000 7f2afb7fe700 1 -- 172.21.15.89:0/632648710 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f2afc086840 con 0x7f2b0415b160 2023-10-10T20:39:58.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.557+0000 7f2b02ffd700 1 --2- 172.21.15.89:0/632648710 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2ae4067200 0x7f2ae40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:58.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.557+0000 7f2b02ffd700 1 --2- 172.21.15.89:0/632648710 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2ae4067200 0x7f2ae40696b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2af40010e0 tx=0x7f2af4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:58.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.561+0000 7f2afb7fe700 1 -- 172.21.15.89:0/632648710 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2afc053710 con 0x7f2b0415b160 2023-10-10T20:39:58.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:58.748+0000 7f2b03fff700 1 -- 172.21.15.89:0/632648710 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2b0404c3c0 con 0x7f2b0415b160 2023-10-10T20:39:59.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.416+0000 7f2afb7fe700 1 -- 172.21.15.89:0/632648710 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v78) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2afc056d30 con 0x7f2b0415b160 2023-10-10T20:39:59.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.418+0000 7f2b03fff700 1 -- 172.21.15.89:0/632648710 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2ae4067200 msgr2=0x7f2ae40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:59.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.418+0000 7f2b03fff700 1 --2- 172.21.15.89:0/632648710 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2ae4067200 0x7f2ae40696b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2af40010e0 tx=0x7f2af4009040 comp rx=0 tx=0).stop 2023-10-10T20:39:59.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.418+0000 7f2b03fff700 1 -- 172.21.15.89:0/632648710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b0415b160 msgr2=0x7f2b0415c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:59.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.418+0000 7f2b03fff700 1 --2- 172.21.15.89:0/632648710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b0415b160 0x7f2b0415c650 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f2afc012000 tx=0x7f2afc016000 comp rx=0 tx=0).stop 2023-10-10T20:39:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.418+0000 7f2b03fff700 1 -- 172.21.15.89:0/632648710 shutdown_connections 2023-10-10T20:39:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.418+0000 7f2b03fff700 1 --2- 172.21.15.89:0/632648710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b0415b160 0x7f2b0415c650 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.418+0000 7f2b03fff700 1 --2- 172.21.15.89:0/632648710 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2b04069a00 0x7f2b0416d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.418+0000 7f2b03fff700 1 --2- 172.21.15.89:0/632648710 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2ae4067200 0x7f2ae40696b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.418+0000 7f2b03fff700 1 --2- 172.21.15.89:0/632648710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b040c75a0 0x7f2b0415ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.418+0000 7f2b03fff700 1 -- 172.21.15.89:0/632648710 >> 172.21.15.89:0/632648710 conn(0x7f2b04063230 msgr2=0x7f2b04068c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.418+0000 7f2b03fff700 1 -- 172.21.15.89:0/632648710 shutdown_connections 2023-10-10T20:39:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.419+0000 7f2b03fff700 1 -- 172.21.15.89:0/632648710 wait complete. 2023-10-10T20:39:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:39:59.435 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:39:59.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.552+0000 7fc202a15700 1 Processor -- start 2023-10-10T20:39:59.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.552+0000 7fc202a15700 1 -- start start 2023-10-10T20:39:59.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.553+0000 7fc202a15700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc1fc157720 0x7fc1fc1589f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:59.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.553+0000 7fc202a15700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1fc158560 con 0x7fc1fc0ad620 2023-10-10T20:39:59.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.553+0000 7fc202a15700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1fc06f320 con 0x7fc1fc06e7b0 2023-10-10T20:39:59.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.553+0000 7fc202a15700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1fc06f600 con 0x7fc1fc157720 2023-10-10T20:39:59.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.553+0000 7fc200fb2700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc1fc0ad620 0x7fc1fc15b3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59372/0 (socket says 172.21.15.89:59372) 2023-10-10T20:39:59.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.553+0000 7fc200fb2700 1 -- 172.21.15.89:0/2082784763 learned_addr learned my addr 172.21.15.89:0/2082784763 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:39:59.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.553+0000 7fc1fbfff700 1 --2- 172.21.15.89:0/2082784763 >> v2:172.21.15.111:3300/0 conn(0x7fc1fc157720 0x7fc1fc1589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:59.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.553+0000 7fc1faffd700 1 -- 172.21.15.89:0/2082784763 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612625081 0 0) 0x7fc1fc158560 con 0x7fc1fc0ad620 2023-10-10T20:39:59.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.553+0000 7fc1faffd700 1 -- 172.21.15.89:0/2082784763 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1e8003570 con 0x7fc1fc0ad620 2023-10-10T20:39:59.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1faffd700 1 -- 172.21.15.89:0/2082784763 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2335488515 0 0) 0x7fc1e8003570 con 0x7fc1fc0ad620 2023-10-10T20:39:59.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1faffd700 1 -- 172.21.15.89:0/2082784763 >> v2:172.21.15.111:3300/0 conn(0x7fc1fc157720 msgr2=0x7fc1fc1589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:59.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1faffd700 1 --2- 172.21.15.89:0/2082784763 >> v2:172.21.15.111:3300/0 conn(0x7fc1fc157720 0x7fc1fc1589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:59.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1faffd700 1 -- 172.21.15.89:0/2082784763 >> v1:172.21.15.111:6789/0 conn(0x7fc1fc06e7b0 legacy=0x7fc1fc0acf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:59.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1faffd700 1 -- 172.21.15.89:0/2082784763 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc157f10 con 0x7fc1fc0ad620 2023-10-10T20:39:59.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1faffd700 1 -- 172.21.15.89:0/2082784763 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fc1f40017f0 con 0x7fc1fc0ad620 2023-10-10T20:39:59.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1faffd700 1 -- 172.21.15.89:0/2082784763 >> v1:172.21.15.89:6789/0 conn(0x7fc1fc0ad620 legacy=0x7fc1fc15b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:59.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1faffd700 1 --2- 172.21.15.89:0/2082784763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e8003a60 0x7fc1e8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:59.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1faffd700 1 --2- 172.21.15.89:0/2082784763 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc1e8004810 0x7fc1e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:59.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1faffd700 1 --2- 172.21.15.89:0/2082784763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1e80051c0 0x7fc1e800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:59.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1faffd700 1 -- 172.21.15.89:0/2082784763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc06f600 con 0x7fc1e8003a60 2023-10-10T20:39:59.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1faffd700 1 -- 172.21.15.89:0/2082784763 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e800c0d0 con 0x7fc1e80051c0 2023-10-10T20:39:59.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1faffd700 1 -- 172.21.15.89:0/2082784763 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e800c290 con 0x7fc1e8004810 2023-10-10T20:39:59.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1fbfff700 1 --2- 172.21.15.89:0/2082784763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e8003a60 0x7fc1e8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:59.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1fb7fe700 1 --2- 172.21.15.89:0/2082784763 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc1e8004810 0x7fc1e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:59.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc200fb2700 1 --2- 172.21.15.89:0/2082784763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1e80051c0 0x7fc1e800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:59.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1fbfff700 1 -- 172.21.15.89:0/2082784763 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc1e8004810 msgr2=0x7fc1e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:59.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1fbfff700 1 --2- 172.21.15.89:0/2082784763 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc1e8004810 0x7fc1e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:59.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1fbfff700 1 -- 172.21.15.89:0/2082784763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1e80051c0 msgr2=0x7fc1e800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:39:59.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1fbfff700 1 --2- 172.21.15.89:0/2082784763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1e80051c0 0x7fc1e800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:59.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1fbfff700 1 -- 172.21.15.89:0/2082784763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e8003570 con 0x7fc1e8003a60 2023-10-10T20:39:59.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc200fb2700 1 -- 172.21.15.89:0/2082784763 reap_dead start 2023-10-10T20:39:59.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1fb7fe700 1 --2- 172.21.15.89:0/2082784763 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc1e8004810 0x7fc1e8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:39:59.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1fbfff700 1 --2- 172.21.15.89:0/2082784763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e8003a60 0x7fc1e8003e70 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009c00 tx=0x7fc1ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686f2bf393a20241 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:59.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1faffd700 1 -- 172.21.15.89:0/2082784763 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec00e710 con 0x7fc1e8003a60 2023-10-10T20:39:59.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1faffd700 1 -- 172.21.15.89:0/2082784763 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ec00e870 con 0x7fc1e8003a60 2023-10-10T20:39:59.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.554+0000 7fc1faffd700 1 -- 172.21.15.89:0/2082784763 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec00ec30 con 0x7fc1e8003a60 2023-10-10T20:39:59.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.555+0000 7fc202a15700 1 -- 172.21.15.89:0/2082784763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e8003a60 msgr2=0x7fc1e8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:59.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.555+0000 7fc202a15700 1 --2- 172.21.15.89:0/2082784763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e8003a60 0x7fc1e8003e70 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009c00 tx=0x7fc1ec0091a0 comp rx=0 tx=0).stop 2023-10-10T20:39:59.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.555+0000 7fc202a15700 1 -- 172.21.15.89:0/2082784763 shutdown_connections 2023-10-10T20:39:59.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.555+0000 7fc202a15700 1 --2- 172.21.15.89:0/2082784763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e8003a60 0x7fc1e8003e70 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:59.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.555+0000 7fc202a15700 1 -- 172.21.15.89:0/2082784763 >> 172.21.15.89:0/2082784763 conn(0x7fc1fc0cd640 msgr2=0x7fc1fc0cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:39:59.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.555+0000 7fc202a15700 1 -- 172.21.15.89:0/2082784763 shutdown_connections 2023-10-10T20:39:59.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.555+0000 7fc202a15700 1 -- 172.21.15.89:0/2082784763 wait complete. 2023-10-10T20:39:59.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.556+0000 7fc202a15700 1 Processor -- start 2023-10-10T20:39:59.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.556+0000 7fc202a15700 1 -- start start 2023-10-10T20:39:59.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.557+0000 7fc202a15700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc1e8003a60 0x7fc1fc0c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:59.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.557+0000 7fc202a15700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1fc155e70 0x7fc1fc0c2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:59.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.557+0000 7fc202a15700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1fc0b18d0 0x7fc1fc0b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:59.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.557+0000 7fc202a15700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc1689b0 con 0x7fc1fc0b18d0 2023-10-10T20:39:59.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.557+0000 7fc202a15700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc167b20 con 0x7fc1fc155e70 2023-10-10T20:39:59.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.557+0000 7fc202a15700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc0b2280 con 0x7fc1e8003a60 2023-10-10T20:39:59.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.557+0000 7fc200fb2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1fc0b18d0 0x7fc1fc0b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:59.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.557+0000 7fc200fb2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1fc0b18d0 0x7fc1fc0b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54118/0 (socket says 172.21.15.89:54118) 2023-10-10T20:39:59.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.557+0000 7fc200fb2700 1 -- 172.21.15.89:0/3527682028 learned_addr learned my addr 172.21.15.89:0/3527682028 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:39:59.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.557+0000 7fc1fbfff700 1 --2- 172.21.15.89:0/3527682028 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc1e8003a60 0x7fc1fc0c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:59.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.557+0000 7fc1fb7fe700 1 --2- 172.21.15.89:0/3527682028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1fc155e70 0x7fc1fc0c2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:59.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.557+0000 7fc200fb2700 1 -- 172.21.15.89:0/3527682028 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc1e8003a60 msgr2=0x7fc1fc0c4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:59.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.557+0000 7fc200fb2700 1 --2- 172.21.15.89:0/3527682028 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc1e8003a60 0x7fc1fc0c4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:59.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.557+0000 7fc200fb2700 1 -- 172.21.15.89:0/3527682028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1fc155e70 msgr2=0x7fc1fc0c2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:39:59.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.557+0000 7fc200fb2700 1 --2- 172.21.15.89:0/3527682028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1fc155e70 0x7fc1fc0c2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:39:59.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.557+0000 7fc200fb2700 1 -- 172.21.15.89:0/3527682028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc167d30 con 0x7fc1fc0b18d0 2023-10-10T20:39:59.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.558+0000 7fc200fb2700 1 --2- 172.21.15.89:0/3527682028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1fc0b18d0 0x7fc1fc0b1d40 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0acb50 tx=0x7fc1f4001330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:59.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.558+0000 7fc1f3fff700 1 -- 172.21.15.89:0/3527682028 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f400ebc0 con 0x7fc1fc0b18d0 2023-10-10T20:39:59.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.558+0000 7fc1f3fff700 1 -- 172.21.15.89:0/3527682028 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f400ed20 con 0x7fc1fc0b18d0 2023-10-10T20:39:59.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.558+0000 7fc1f3fff700 1 -- 172.21.15.89:0/3527682028 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4013670 con 0x7fc1fc0b18d0 2023-10-10T20:39:59.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.558+0000 7fc202a15700 1 -- 172.21.15.89:0/3527682028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1fc168010 con 0x7fc1fc0b18d0 2023-10-10T20:39:59.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.558+0000 7fc202a15700 1 -- 172.21.15.89:0/3527682028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1fc1685d0 con 0x7fc1fc0b18d0 2023-10-10T20:39:59.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.559+0000 7fc202a15700 1 -- 172.21.15.89:0/3527682028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1fc0718b0 con 0x7fc1fc0b18d0 2023-10-10T20:39:59.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.559+0000 7fc1f3fff700 1 -- 172.21.15.89:0/3527682028 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fc1f40139e0 con 0x7fc1fc0b18d0 2023-10-10T20:39:59.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.560+0000 7fc1f3fff700 1 --2- 172.21.15.89:0/3527682028 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc1dc067200 0x7fc1dc0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:39:59.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.560+0000 7fc1fbfff700 1 --2- 172.21.15.89:0/3527682028 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc1dc067200 0x7fc1dc0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:39:59.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.560+0000 7fc1f3fff700 1 -- 172.21.15.89:0/3527682028 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fc1f4084d30 con 0x7fc1fc0b18d0 2023-10-10T20:39:59.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.560+0000 7fc1fbfff700 1 --2- 172.21.15.89:0/3527682028 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc1dc067200 0x7fc1dc0696b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0091e0 tx=0x7fc1ec01a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:39:59.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.564+0000 7fc1f3fff700 1 -- 172.21.15.89:0/3527682028 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1f4089070 con 0x7fc1fc0b18d0 2023-10-10T20:39:59.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:39:59.759+0000 7fc202a15700 1 -- 172.21.15.89:0/3527682028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc1fc1581f0 con 0x7fc1fc0b18d0 2023-10-10T20:40:00.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.423+0000 7fc1f3fff700 1 -- 172.21.15.89:0/3527682028 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v79) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc1f4055900 con 0x7fc1fc0b18d0 2023-10-10T20:40:00.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.427+0000 7fc202a15700 1 -- 172.21.15.89:0/3527682028 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc1dc067200 msgr2=0x7fc1dc0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:00.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.427+0000 7fc202a15700 1 --2- 172.21.15.89:0/3527682028 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc1dc067200 0x7fc1dc0696b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0091e0 tx=0x7fc1ec01a000 comp rx=0 tx=0).stop 2023-10-10T20:40:00.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.427+0000 7fc202a15700 1 -- 172.21.15.89:0/3527682028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1fc0b18d0 msgr2=0x7fc1fc0b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:00.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.427+0000 7fc202a15700 1 --2- 172.21.15.89:0/3527682028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1fc0b18d0 0x7fc1fc0b1d40 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0acb50 tx=0x7fc1f4001330 comp rx=0 tx=0).stop 2023-10-10T20:40:00.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.427+0000 7fc202a15700 1 -- 172.21.15.89:0/3527682028 shutdown_connections 2023-10-10T20:40:00.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.427+0000 7fc202a15700 1 --2- 172.21.15.89:0/3527682028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1fc0b18d0 0x7fc1fc0b1d40 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:00.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.427+0000 7fc202a15700 1 --2- 172.21.15.89:0/3527682028 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc1e8003a60 0x7fc1fc0c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:00.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.427+0000 7fc202a15700 1 --2- 172.21.15.89:0/3527682028 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc1dc067200 0x7fc1dc0696b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:00.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.427+0000 7fc202a15700 1 --2- 172.21.15.89:0/3527682028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1fc155e70 0x7fc1fc0c2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:00.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.427+0000 7fc202a15700 1 -- 172.21.15.89:0/3527682028 >> 172.21.15.89:0/3527682028 conn(0x7fc1fc0cd640 msgr2=0x7fc1fc0acb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:00.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.427+0000 7fc202a15700 1 -- 172.21.15.89:0/3527682028 shutdown_connections 2023-10-10T20:40:00.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.427+0000 7fc202a15700 1 -- 172.21.15.89:0/3527682028 wait complete. 2023-10-10T20:40:00.433 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' removed 2023-10-10T20:40:00.444 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:40:00.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.562+0000 7f92c9bf4700 1 Processor -- start 2023-10-10T20:40:00.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.562+0000 7f92c9bf4700 1 -- start start 2023-10-10T20:40:00.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.563+0000 7f92c9bf4700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f92c4159c60 0x7f92c415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:00.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.563+0000 7f92c9bf4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f92c415db10 0x7f92c4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:00.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.563+0000 7f92c9bf4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c415a160 con 0x7f92c4159890 2023-10-10T20:40:00.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.563+0000 7f92c9bf4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c415df50 con 0x7f92c415db10 2023-10-10T20:40:00.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.563+0000 7f92c9bf4700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c40d13a0 con 0x7f92c4159c60 2023-10-10T20:40:00.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.563+0000 7f92c3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f92c415db10 0x7f92c4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:00.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.563+0000 7f92c37fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f92c4159890 0x7f92c415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59386/0 (socket says 172.21.15.89:59386) 2023-10-10T20:40:00.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.563+0000 7f92c3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f92c415db10 0x7f92c4069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54122/0 (socket says 172.21.15.89:54122) 2023-10-10T20:40:00.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.563+0000 7f92c37fe700 1 -- 172.21.15.89:0/3722194429 learned_addr learned my addr 172.21.15.89:0/3722194429 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:00.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.563+0000 7f92c2ffd700 1 --2- 172.21.15.89:0/3722194429 >> v2:172.21.15.111:3300/0 conn(0x7f92c4159c60 0x7f92c415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:00.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.563+0000 7f92c3fff700 1 -- 172.21.15.89:0/3722194429 >> v2:172.21.15.111:3300/0 conn(0x7f92c4159c60 msgr2=0x7f92c415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:00.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.563+0000 7f92c3fff700 1 --2- 172.21.15.89:0/3722194429 >> v2:172.21.15.111:3300/0 conn(0x7f92c4159c60 0x7f92c415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:00.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.563+0000 7f92c3fff700 1 -- 172.21.15.89:0/3722194429 >> v1:172.21.15.89:6789/0 conn(0x7f92c4159890 legacy=0x7f92c415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:00.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.563+0000 7f92c27fc700 1 -- 172.21.15.89:0/3722194429 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1338033674 0 0) 0x7f92c415a160 con 0x7f92c4159890 2023-10-10T20:40:00.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.563+0000 7f92c3fff700 1 -- 172.21.15.89:0/3722194429 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c415e140 con 0x7f92c415db10 2023-10-10T20:40:00.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.563+0000 7f92c3fff700 1 --2- 172.21.15.89:0/3722194429 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92c415db10 0x7f92c4069560 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f92c40b67f0 tx=0x7f92bc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728fafa0a469a3a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:00.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.563+0000 7f92c27fc700 1 -- 172.21.15.89:0/3722194429 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc00e070 con 0x7f92c415db10 2023-10-10T20:40:00.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.563+0000 7f92c27fc700 1 -- 172.21.15.89:0/3722194429 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92bc00bc30 con 0x7f92c415db10 2023-10-10T20:40:00.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.563+0000 7f92c27fc700 1 -- 172.21.15.89:0/3722194429 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc012430 con 0x7f92c415db10 2023-10-10T20:40:00.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.564+0000 7f92c9bf4700 1 -- 172.21.15.89:0/3722194429 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92c415db10 msgr2=0x7f92c4069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:00.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.564+0000 7f92c9bf4700 1 --2- 172.21.15.89:0/3722194429 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92c415db10 0x7f92c4069560 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f92c40b67f0 tx=0x7f92bc009780 comp rx=0 tx=0).stop 2023-10-10T20:40:00.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.564+0000 7f92c9bf4700 1 -- 172.21.15.89:0/3722194429 shutdown_connections 2023-10-10T20:40:00.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.564+0000 7f92c9bf4700 1 --2- 172.21.15.89:0/3722194429 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92c415db10 0x7f92c4069560 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:00.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.564+0000 7f92c9bf4700 1 --2- 172.21.15.89:0/3722194429 >> v2:172.21.15.111:3300/0 conn(0x7f92c4159c60 0x7f92c415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:00.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.564+0000 7f92c9bf4700 1 -- 172.21.15.89:0/3722194429 >> 172.21.15.89:0/3722194429 conn(0x7f92c4068e20 msgr2=0x7f92c40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:00.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.564+0000 7f92c9bf4700 1 -- 172.21.15.89:0/3722194429 shutdown_connections 2023-10-10T20:40:00.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.564+0000 7f92c9bf4700 1 -- 172.21.15.89:0/3722194429 wait complete. 2023-10-10T20:40:00.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.565+0000 7f92c9bf4700 1 Processor -- start 2023-10-10T20:40:00.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.565+0000 7f92c9bf4700 1 -- start start 2023-10-10T20:40:00.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.566+0000 7f92c9bf4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92c4159890 0x7f92c4151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:00.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.566+0000 7f92c9bf4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92c4159c60 0x7f92c40cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:00.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.566+0000 7f92c9bf4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92c415db10 0x7f92c415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:00.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.566+0000 7f92c9bf4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c40cff60 con 0x7f92c415db10 2023-10-10T20:40:00.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.566+0000 7f92c9bf4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c40cfad0 con 0x7f92c4159890 2023-10-10T20:40:00.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.566+0000 7f92c9bf4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c40cec20 con 0x7f92c4159c60 2023-10-10T20:40:00.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.566+0000 7f92c3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92c415db10 0x7f92c415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:00.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.566+0000 7f92c3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92c415db10 0x7f92c415f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54134/0 (socket says 172.21.15.89:54134) 2023-10-10T20:40:00.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.566+0000 7f92c3fff700 1 -- 172.21.15.89:0/2761432232 learned_addr learned my addr 172.21.15.89:0/2761432232 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:00.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.566+0000 7f92c2ffd700 1 --2- 172.21.15.89:0/2761432232 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92c4159c60 0x7f92c40cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:00.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.566+0000 7f92c37fe700 1 --2- 172.21.15.89:0/2761432232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92c4159890 0x7f92c4151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:00.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.566+0000 7f92c3fff700 1 -- 172.21.15.89:0/2761432232 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92c4159c60 msgr2=0x7f92c40cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:00.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.566+0000 7f92c3fff700 1 --2- 172.21.15.89:0/2761432232 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92c4159c60 0x7f92c40cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:00.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.566+0000 7f92c3fff700 1 -- 172.21.15.89:0/2761432232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92c4159890 msgr2=0x7f92c4151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:00.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.566+0000 7f92c3fff700 1 --2- 172.21.15.89:0/2761432232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92c4159890 0x7f92c4151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:00.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.566+0000 7f92c3fff700 1 -- 172.21.15.89:0/2761432232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc009a20 con 0x7f92c415db10 2023-10-10T20:40:00.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.566+0000 7f92c3fff700 1 --2- 172.21.15.89:0/2761432232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92c415db10 0x7f92c415f3c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f92c4070e20 tx=0x7f92bc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:00.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.567+0000 7f92ab7fe700 1 -- 172.21.15.89:0/2761432232 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc01d630 con 0x7f92c415db10 2023-10-10T20:40:00.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.567+0000 7f92ab7fe700 1 -- 172.21.15.89:0/2761432232 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92bc01d790 con 0x7f92c415db10 2023-10-10T20:40:00.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.567+0000 7f92ab7fe700 1 -- 172.21.15.89:0/2761432232 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc01db50 con 0x7f92c415db10 2023-10-10T20:40:00.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.567+0000 7f92c9bf4700 1 -- 172.21.15.89:0/2761432232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c40ceec0 con 0x7f92c415db10 2023-10-10T20:40:00.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.567+0000 7f92c9bf4700 1 -- 172.21.15.89:0/2761432232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c40cf480 con 0x7f92c415db10 2023-10-10T20:40:00.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.568+0000 7f92c9bf4700 1 -- 172.21.15.89:0/2761432232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c404c3c0 con 0x7f92c415db10 2023-10-10T20:40:00.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.568+0000 7f92ab7fe700 1 -- 172.21.15.89:0/2761432232 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f92bc024070 con 0x7f92c415db10 2023-10-10T20:40:00.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.568+0000 7f92ab7fe700 1 --2- 172.21.15.89:0/2761432232 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f92b0067200 0x7f92b00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:00.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.569+0000 7f92c37fe700 1 --2- 172.21.15.89:0/2761432232 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f92b0067200 0x7f92b00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:00.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.569+0000 7f92ab7fe700 1 -- 172.21.15.89:0/2761432232 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f92bc086f90 con 0x7f92c415db10 2023-10-10T20:40:00.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.569+0000 7f92c37fe700 1 --2- 172.21.15.89:0/2761432232 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f92b0067200 0x7f92b00696b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f92b80011e0 tx=0x7f92b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:00.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.573+0000 7f92ab7fe700 1 -- 172.21.15.89:0/2761432232 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92bc0541b0 con 0x7f92c415db10 2023-10-10T20:40:00.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.756+0000 7f92c9bf4700 1 -- 172.21.15.89:0/2761432232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f92c40cfc10 con 0x7f92c415db10 2023-10-10T20:40:00.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.760+0000 7f92ab7fe700 1 -- 172.21.15.89:0/2761432232 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f92bc009880 con 0x7f92c415db10 2023-10-10T20:40:00.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.762+0000 7f92c9bf4700 1 -- 172.21.15.89:0/2761432232 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f92b0067200 msgr2=0x7f92b00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:00.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.762+0000 7f92c9bf4700 1 --2- 172.21.15.89:0/2761432232 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f92b0067200 0x7f92b00696b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f92b80011e0 tx=0x7f92b8009040 comp rx=0 tx=0).stop 2023-10-10T20:40:00.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.762+0000 7f92c9bf4700 1 -- 172.21.15.89:0/2761432232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92c415db10 msgr2=0x7f92c415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:00.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.762+0000 7f92c9bf4700 1 --2- 172.21.15.89:0/2761432232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92c415db10 0x7f92c415f3c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f92c4070e20 tx=0x7f92bc019000 comp rx=0 tx=0).stop 2023-10-10T20:40:00.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.762+0000 7f92c9bf4700 1 -- 172.21.15.89:0/2761432232 shutdown_connections 2023-10-10T20:40:00.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.762+0000 7f92c9bf4700 1 --2- 172.21.15.89:0/2761432232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92c415db10 0x7f92c415f3c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:00.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.762+0000 7f92c9bf4700 1 --2- 172.21.15.89:0/2761432232 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f92b0067200 0x7f92b00696b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:00.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.762+0000 7f92c9bf4700 1 --2- 172.21.15.89:0/2761432232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92c4159890 0x7f92c4151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:00.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.762+0000 7f92c9bf4700 1 --2- 172.21.15.89:0/2761432232 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f92c4159c60 0x7f92c40cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:00.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.762+0000 7f92c9bf4700 1 -- 172.21.15.89:0/2761432232 >> 172.21.15.89:0/2761432232 conn(0x7f92c4068e20 msgr2=0x7f92c40c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:00.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.762+0000 7f92c9bf4700 1 -- 172.21.15.89:0/2761432232 shutdown_connections 2023-10-10T20:40:00.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.762+0000 7f92c9bf4700 1 -- 172.21.15.89:0/2761432232 wait complete. 2023-10-10T20:40:00.781 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag' 2023-10-10T20:40:00.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.895+0000 7f19e3cab700 1 Processor -- start 2023-10-10T20:40:00.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.895+0000 7f19e3cab700 1 -- start start 2023-10-10T20:40:00.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.895+0000 7f19e3cab700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f19dc159c60 0x7f19dc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:00.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.895+0000 7f19e3cab700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f19dc15db10 0x7f19dc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:00.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.895+0000 7f19e3cab700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19dc15a160 con 0x7f19dc159890 2023-10-10T20:40:00.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.895+0000 7f19e3cab700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19dc15df50 con 0x7f19dc159c60 2023-10-10T20:40:00.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.895+0000 7f19e3cab700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19dc0d13a0 con 0x7f19dc15db10 2023-10-10T20:40:00.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.896+0000 7f19e1246700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f19dc159c60 0x7f19dc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:00.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.896+0000 7f19e1246700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f19dc159c60 0x7f19dc15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54144/0 (socket says 172.21.15.89:54144) 2023-10-10T20:40:00.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.896+0000 7f19e1246700 1 -- 172.21.15.89:0/2378881648 learned_addr learned my addr 172.21.15.89:0/2378881648 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:00.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.896+0000 7f19e1246700 1 -- 172.21.15.89:0/2378881648 >> v2:172.21.15.111:3300/0 conn(0x7f19dc15db10 msgr2=0x7f19dc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:00.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.896+0000 7f19e1246700 1 --2- 172.21.15.89:0/2378881648 >> v2:172.21.15.111:3300/0 conn(0x7f19dc15db10 0x7f19dc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:00.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.896+0000 7f19e1246700 1 -- 172.21.15.89:0/2378881648 >> v1:172.21.15.111:6789/0 conn(0x7f19dc159890 legacy=0x7f19dc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:00.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.896+0000 7f19e1246700 1 -- 172.21.15.89:0/2378881648 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc15e140 con 0x7f19dc159c60 2023-10-10T20:40:00.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.896+0000 7f19e1246700 1 --2- 172.21.15.89:0/2378881648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19dc159c60 0x7f19dc15d5d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f19dc06a890 tx=0x7f19cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9eb9c2bef34a09 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:00.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.896+0000 7f19e0a45700 1 -- 172.21.15.89:0/2378881648 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc00b820 con 0x7f19dc159c60 2023-10-10T20:40:00.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.896+0000 7f19e0a45700 1 -- 172.21.15.89:0/2378881648 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc00b980 con 0x7f19dc159c60 2023-10-10T20:40:00.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.896+0000 7f19e0a45700 1 -- 172.21.15.89:0/2378881648 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc00bd20 con 0x7f19dc159c60 2023-10-10T20:40:00.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.897+0000 7f19e3cab700 1 -- 172.21.15.89:0/2378881648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19dc159c60 msgr2=0x7f19dc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:00.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.897+0000 7f19e3cab700 1 --2- 172.21.15.89:0/2378881648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19dc159c60 0x7f19dc15d5d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f19dc06a890 tx=0x7f19cc009250 comp rx=0 tx=0).stop 2023-10-10T20:40:00.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.897+0000 7f19e3cab700 1 -- 172.21.15.89:0/2378881648 shutdown_connections 2023-10-10T20:40:00.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.897+0000 7f19e3cab700 1 --2- 172.21.15.89:0/2378881648 >> v2:172.21.15.111:3300/0 conn(0x7f19dc15db10 0x7f19dc069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:00.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.897+0000 7f19e3cab700 1 --2- 172.21.15.89:0/2378881648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19dc159c60 0x7f19dc15d5d0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:00.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.897+0000 7f19e3cab700 1 -- 172.21.15.89:0/2378881648 >> 172.21.15.89:0/2378881648 conn(0x7f19dc068e20 msgr2=0x7f19dc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:00.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.897+0000 7f19e3cab700 1 -- 172.21.15.89:0/2378881648 shutdown_connections 2023-10-10T20:40:00.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.897+0000 7f19e3cab700 1 -- 172.21.15.89:0/2378881648 wait complete. 2023-10-10T20:40:00.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.898+0000 7f19e3cab700 1 Processor -- start 2023-10-10T20:40:00.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.898+0000 7f19e3cab700 1 -- start start 2023-10-10T20:40:00.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e3cab700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19dc159890 0x7f19dc151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:00.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e3cab700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19dc159c60 0x7f19dc0c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:00.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e3cab700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19dc15db10 0x7f19dc149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e3cab700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc0ca760 con 0x7f19dc159c60 2023-10-10T20:40:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e3cab700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc151060 con 0x7f19dc159890 2023-10-10T20:40:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e3cab700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc1518b0 con 0x7f19dc15db10 2023-10-10T20:40:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e1246700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19dc159c60 0x7f19dc0c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e1246700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19dc159c60 0x7f19dc0c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54160/0 (socket says 172.21.15.89:54160) 2023-10-10T20:40:00.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e1246700 1 -- 172.21.15.89:0/1694865132 learned_addr learned my addr 172.21.15.89:0/1694865132 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:00.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e1a47700 1 --2- 172.21.15.89:0/1694865132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19dc159890 0x7f19dc151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:00.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e2248700 1 --2- 172.21.15.89:0/1694865132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19dc15db10 0x7f19dc149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:00.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e1246700 1 -- 172.21.15.89:0/1694865132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19dc15db10 msgr2=0x7f19dc149440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:00.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e1246700 1 --2- 172.21.15.89:0/1694865132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19dc15db10 0x7f19dc149440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:00.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e1246700 1 -- 172.21.15.89:0/1694865132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19dc159890 msgr2=0x7f19dc151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:00.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e1246700 1 --2- 172.21.15.89:0/1694865132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19dc159890 0x7f19dc151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:00.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e1246700 1 -- 172.21.15.89:0/1694865132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19cc0094f0 con 0x7f19dc159c60 2023-10-10T20:40:00.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e2248700 1 --2- 172.21.15.89:0/1694865132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19dc15db10 0x7f19dc149440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:00.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e1246700 1 --2- 172.21.15.89:0/1694865132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19dc159c60 0x7f19dc0c9e10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f19cc003080 tx=0x7f19cc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:00.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19d1ffb700 1 -- 172.21.15.89:0/1694865132 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc011e20 con 0x7f19dc159c60 2023-10-10T20:40:00.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19d1ffb700 1 -- 172.21.15.89:0/1694865132 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc01f070 con 0x7f19dc159c60 2023-10-10T20:40:00.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19d1ffb700 1 -- 172.21.15.89:0/1694865132 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc01a8a0 con 0x7f19dc159c60 2023-10-10T20:40:00.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e3cab700 1 -- 172.21.15.89:0/1694865132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19dc149980 con 0x7f19dc159c60 2023-10-10T20:40:00.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.899+0000 7f19e3cab700 1 -- 172.21.15.89:0/1694865132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19dc151260 con 0x7f19dc159c60 2023-10-10T20:40:00.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.900+0000 7f19e3cab700 1 -- 172.21.15.89:0/1694865132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19dc04c3c0 con 0x7f19dc159c60 2023-10-10T20:40:00.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.901+0000 7f19d1ffb700 1 -- 172.21.15.89:0/1694865132 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f19cc01ac50 con 0x7f19dc159c60 2023-10-10T20:40:00.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.901+0000 7f19d1ffb700 1 --2- 172.21.15.89:0/1694865132 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f19c0067200 0x7f19c00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:00.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.901+0000 7f19d1ffb700 1 -- 172.21.15.89:0/1694865132 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f19cc021070 con 0x7f19dc159c60 2023-10-10T20:40:00.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.901+0000 7f19e1a47700 1 --2- 172.21.15.89:0/1694865132 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f19c0067200 0x7f19c00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:00.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.902+0000 7f19e1a47700 1 --2- 172.21.15.89:0/1694865132 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f19c0067200 0x7f19c00696b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f19d8006fd0 tx=0x7f19d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:00.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:00.905+0000 7f19d1ffb700 1 -- 172.21.15.89:0/1694865132 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19cc053c20 con 0x7f19dc159c60 2023-10-10T20:40:01.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:01.165+0000 7f19e3cab700 1 -- 172.21.15.89:0/1694865132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]} v 0) v1 -- 0x7f19dc06a5b0 con 0x7f19dc159c60 2023-10-10T20:40:01.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:01.970+0000 7f19d1ffb700 1 -- 172.21.15.89:0/1694865132 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]}]=0 v256) v1 ==== 158+0+0 (secure 0 0 0) 0x7f19cc057240 con 0x7f19dc159c60 2023-10-10T20:40:01.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:01.973+0000 7f19e3cab700 1 -- 172.21.15.89:0/1694865132 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f19c0067200 msgr2=0x7f19c00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:01.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:01.973+0000 7f19e3cab700 1 --2- 172.21.15.89:0/1694865132 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f19c0067200 0x7f19c00696b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f19d8006fd0 tx=0x7f19d8008040 comp rx=0 tx=0).stop 2023-10-10T20:40:01.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:01.973+0000 7f19e3cab700 1 -- 172.21.15.89:0/1694865132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19dc159c60 msgr2=0x7f19dc0c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:01.973+0000 7f19e3cab700 1 --2- 172.21.15.89:0/1694865132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19dc159c60 0x7f19dc0c9e10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f19cc003080 tx=0x7f19cc016000 comp rx=0 tx=0).stop 2023-10-10T20:40:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:01.973+0000 7f19e3cab700 1 -- 172.21.15.89:0/1694865132 shutdown_connections 2023-10-10T20:40:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:01.973+0000 7f19e3cab700 1 --2- 172.21.15.89:0/1694865132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19dc15db10 0x7f19dc149440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:01.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:01.973+0000 7f19e3cab700 1 --2- 172.21.15.89:0/1694865132 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f19c0067200 0x7f19c00696b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:01.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:01.973+0000 7f19e3cab700 1 --2- 172.21.15.89:0/1694865132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19dc159890 0x7f19dc151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:01.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:01.973+0000 7f19e3cab700 1 --2- 172.21.15.89:0/1694865132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19dc159c60 0x7f19dc0c9e10 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:01.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:01.973+0000 7f19e3cab700 1 -- 172.21.15.89:0/1694865132 >> 172.21.15.89:0/1694865132 conn(0x7f19dc068e20 msgr2=0x7f19dc153190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:01.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:01.973+0000 7f19e3cab700 1 -- 172.21.15.89:0/1694865132 shutdown_connections 2023-10-10T20:40:01.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:01.973+0000 7f19e3cab700 1 -- 172.21.15.89:0/1694865132 wait complete. 2023-10-10T20:40:01.991 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info --human_readable' command ... ok 2023-10-10T20:40:01.992 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_with_human_readable_flag_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info --human_readable' command without subvolume group 2023-10-10T20:40:01.992 INFO:tasks.cephfs_test_runner:test_volume_info_with_human_readable_flag_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) 2023-10-10T20:40:01.993 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup' 2023-10-10T20:40:02.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.113+0000 7fd80d7fd700 1 Processor -- start 2023-10-10T20:40:02.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.113+0000 7fd80d7fd700 1 -- start start 2023-10-10T20:40:02.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.113+0000 7fd80d7fd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd8081578b0 0x7fd808155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:02.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.113+0000 7fd80d7fd700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8080ad930 con 0x7fd8080af2f0 2023-10-10T20:40:02.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.113+0000 7fd80d7fd700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8080acd20 con 0x7fd808155c10 2023-10-10T20:40:02.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.113+0000 7fd80d7fd700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8080ad000 con 0x7fd8081578b0 2023-10-10T20:40:02.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.113+0000 7fd8067fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd8081578b0 0x7fd808155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:02.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.113+0000 7fd8067fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd8081578b0 0x7fd808155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54168/0 (socket says 172.21.15.89:54168) 2023-10-10T20:40:02.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.113+0000 7fd8077fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd8080af2f0 0x7fd8080af730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59392/0 (socket says 172.21.15.89:59392) 2023-10-10T20:40:02.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.113+0000 7fd8067fc700 1 -- 172.21.15.89:0/750775507 learned_addr learned my addr 172.21.15.89:0/750775507 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:02.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.113+0000 7fd8067fc700 1 -- 172.21.15.89:0/750775507 >> v1:172.21.15.111:6789/0 conn(0x7fd808155c10 legacy=0x7fd8080ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:02.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.113+0000 7fd8067fc700 1 -- 172.21.15.89:0/750775507 >> v1:172.21.15.89:6789/0 conn(0x7fd8080af2f0 legacy=0x7fd8080af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:02.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.113+0000 7fd8067fc700 1 -- 172.21.15.89:0/750775507 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8080ad2e0 con 0x7fd8081578b0 2023-10-10T20:40:02.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.113+0000 7fd8067fc700 1 --2- 172.21.15.89:0/750775507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8081578b0 0x7fd808155fe0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0099b0 tx=0x7fd7fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ba84bb11d7697c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:02.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.113+0000 7fd805ffb700 1 -- 172.21.15.89:0/750775507 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc00b820 con 0x7fd8081578b0 2023-10-10T20:40:02.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.113+0000 7fd805ffb700 1 -- 172.21.15.89:0/750775507 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc00b980 con 0x7fd8081578b0 2023-10-10T20:40:02.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.113+0000 7fd805ffb700 1 -- 172.21.15.89:0/750775507 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc00bd20 con 0x7fd8081578b0 2023-10-10T20:40:02.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.114+0000 7fd80d7fd700 1 -- 172.21.15.89:0/750775507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8081578b0 msgr2=0x7fd808155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:02.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.114+0000 7fd80d7fd700 1 --2- 172.21.15.89:0/750775507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8081578b0 0x7fd808155fe0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0099b0 tx=0x7fd7fc009250 comp rx=0 tx=0).stop 2023-10-10T20:40:02.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.114+0000 7fd80d7fd700 1 -- 172.21.15.89:0/750775507 shutdown_connections 2023-10-10T20:40:02.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.114+0000 7fd80d7fd700 1 --2- 172.21.15.89:0/750775507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8081578b0 0x7fd808155fe0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:02.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.114+0000 7fd80d7fd700 1 -- 172.21.15.89:0/750775507 >> 172.21.15.89:0/750775507 conn(0x7fd80806a320 msgr2=0x7fd8080cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:02.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.114+0000 7fd80d7fd700 1 -- 172.21.15.89:0/750775507 shutdown_connections 2023-10-10T20:40:02.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.114+0000 7fd80d7fd700 1 -- 172.21.15.89:0/750775507 wait complete. 2023-10-10T20:40:02.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.115+0000 7fd80d7fd700 1 Processor -- start 2023-10-10T20:40:02.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.115+0000 7fd80d7fd700 1 -- start start 2023-10-10T20:40:02.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd80d7fd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd8080af2f0 0x7fd808168e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:02.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd80d7fd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd808155c10 0x7fd8080caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:02.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd80d7fd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8081578b0 0x7fd808150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:02.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd80d7fd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080cb860 con 0x7fd8081578b0 2023-10-10T20:40:02.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd80d7fd700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080cbb40 con 0x7fd808155c10 2023-10-10T20:40:02.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd80d7fd700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080cbe20 con 0x7fd8080af2f0 2023-10-10T20:40:02.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd8077fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8081578b0 0x7fd808150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:02.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd8077fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8081578b0 0x7fd808150ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54170/0 (socket says 172.21.15.89:54170) 2023-10-10T20:40:02.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd8077fe700 1 -- 172.21.15.89:0/610744174 learned_addr learned my addr 172.21.15.89:0/610744174 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:02.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd8067fc700 1 --2- 172.21.15.89:0/610744174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd808155c10 0x7fd8080caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd806ffd700 1 --2- 172.21.15.89:0/610744174 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd8080af2f0 0x7fd808168e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd8077fe700 1 -- 172.21.15.89:0/610744174 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd8080af2f0 msgr2=0x7fd808168e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd8077fe700 1 --2- 172.21.15.89:0/610744174 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd8080af2f0 0x7fd808168e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd8077fe700 1 -- 172.21.15.89:0/610744174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd808155c10 msgr2=0x7fd8080caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd8077fe700 1 --2- 172.21.15.89:0/610744174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd808155c10 0x7fd8080caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:02.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd8077fe700 1 -- 172.21.15.89:0/610744174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7fc0094f0 con 0x7fd8081578b0 2023-10-10T20:40:02.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd8067fc700 1 --2- 172.21.15.89:0/610744174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd808155c10 0x7fd8080caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:02.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd8077fe700 1 --2- 172.21.15.89:0/610744174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8081578b0 0x7fd808150ba0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd80000cd00 tx=0x7fd800001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:02.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd7eeffd700 1 -- 172.21.15.89:0/610744174 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd800005df0 con 0x7fd8081578b0 2023-10-10T20:40:02.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.116+0000 7fd7eeffd700 1 -- 172.21.15.89:0/610744174 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80000a960 con 0x7fd8081578b0 2023-10-10T20:40:02.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.117+0000 7fd7eeffd700 1 -- 172.21.15.89:0/610744174 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80000ad00 con 0x7fd8081578b0 2023-10-10T20:40:02.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.117+0000 7fd80d7fd700 1 -- 172.21.15.89:0/610744174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8080c9030 con 0x7fd8081578b0 2023-10-10T20:40:02.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.117+0000 7fd80d7fd700 1 -- 172.21.15.89:0/610744174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd808169390 con 0x7fd8081578b0 2023-10-10T20:40:02.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.117+0000 7fd80d7fd700 1 -- 172.21.15.89:0/610744174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8080c9550 con 0x7fd8081578b0 2023-10-10T20:40:02.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.118+0000 7fd7eeffd700 1 -- 172.21.15.89:0/610744174 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fd800007300 con 0x7fd8081578b0 2023-10-10T20:40:02.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.118+0000 7fd7eeffd700 1 --2- 172.21.15.89:0/610744174 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd7f4067200 0x7fd7f40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:02.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.118+0000 7fd7eeffd700 1 -- 172.21.15.89:0/610744174 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd8000847c0 con 0x7fd8081578b0 2023-10-10T20:40:02.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.118+0000 7fd806ffd700 1 --2- 172.21.15.89:0/610744174 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd7f4067200 0x7fd7f40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:02.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.119+0000 7fd806ffd700 1 --2- 172.21.15.89:0/610744174 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd7f4067200 0x7fd7f40696b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80027e0 tx=0x7fd7f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:02.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.123+0000 7fd7eeffd700 1 -- 172.21.15.89:0/610744174 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8080c9550 con 0x7fd8081578b0 2023-10-10T20:40:02.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.386+0000 7fd80d7fd700 1 -- 172.21.15.89:0/610744174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]} v 0) v1 -- 0x7fd8080c9550 con 0x7fd8081578b0 2023-10-10T20:40:02.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.976+0000 7fd7eeffd700 1 -- 172.21.15.89:0/610744174 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]}]=0 v257) v1 ==== 184+0+0 (secure 0 0 0) 0x7fd8080c9550 con 0x7fd8081578b0 2023-10-10T20:40:02.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.978+0000 7fd80d7fd700 1 -- 172.21.15.89:0/610744174 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd7f4067200 msgr2=0x7fd7f40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:02.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.978+0000 7fd80d7fd700 1 --2- 172.21.15.89:0/610744174 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd7f4067200 0x7fd7f40696b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80027e0 tx=0x7fd7f8009040 comp rx=0 tx=0).stop 2023-10-10T20:40:02.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.978+0000 7fd80d7fd700 1 -- 172.21.15.89:0/610744174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8081578b0 msgr2=0x7fd808150ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:02.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.978+0000 7fd80d7fd700 1 --2- 172.21.15.89:0/610744174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8081578b0 0x7fd808150ba0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd80000cd00 tx=0x7fd800001b10 comp rx=0 tx=0).stop 2023-10-10T20:40:02.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.978+0000 7fd80d7fd700 1 -- 172.21.15.89:0/610744174 shutdown_connections 2023-10-10T20:40:02.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.978+0000 7fd80d7fd700 1 --2- 172.21.15.89:0/610744174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8081578b0 0x7fd808150ba0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:02.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.978+0000 7fd80d7fd700 1 --2- 172.21.15.89:0/610744174 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd8080af2f0 0x7fd808168e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:02.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.978+0000 7fd80d7fd700 1 --2- 172.21.15.89:0/610744174 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd7f4067200 0x7fd7f40696b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:02.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.978+0000 7fd80d7fd700 1 --2- 172.21.15.89:0/610744174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd808155c10 0x7fd8080caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:02.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.978+0000 7fd80d7fd700 1 -- 172.21.15.89:0/610744174 >> 172.21.15.89:0/610744174 conn(0x7fd80806a320 msgr2=0x7fd8080b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:02.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.978+0000 7fd80d7fd700 1 -- 172.21.15.89:0/610744174 shutdown_connections 2023-10-10T20:40:02.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:02.978+0000 7fd80d7fd700 1 -- 172.21.15.89:0/610744174 wait complete. 2023-10-10T20:40:02.993 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:40:03.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.109+0000 7f1ca523c700 1 Processor -- start 2023-10-10T20:40:03.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.109+0000 7f1ca523c700 1 -- start start 2023-10-10T20:40:03.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.110+0000 7f1ca523c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1ca00b4e70 0x7f1ca00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:03.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.110+0000 7f1ca523c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1ca00b16d0 0x7f1ca00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:03.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.110+0000 7f1ca523c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca00b2080 con 0x7f1ca015d240 2023-10-10T20:40:03.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.110+0000 7f1ca523c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca00cfb00 con 0x7f1ca00b16d0 2023-10-10T20:40:03.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.110+0000 7f1ca523c700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca00b5c40 con 0x7f1ca00b4e70 2023-10-10T20:40:03.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.110+0000 7f1c9f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1ca00b16d0 0x7f1ca00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:03.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.110+0000 7f1c9f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1ca00b16d0 0x7f1ca00b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54180/0 (socket says 172.21.15.89:54180) 2023-10-10T20:40:03.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.110+0000 7f1c9f7fe700 1 -- 172.21.15.89:0/1319694543 learned_addr learned my addr 172.21.15.89:0/1319694543 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:03.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.110+0000 7f1c9ffff700 1 --2- 172.21.15.89:0/1319694543 >> v2:172.21.15.111:3300/0 conn(0x7f1ca00b4e70 0x7f1ca00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:03.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.110+0000 7f1c9f7fe700 1 -- 172.21.15.89:0/1319694543 >> v2:172.21.15.111:3300/0 conn(0x7f1ca00b4e70 msgr2=0x7f1ca00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:03.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.110+0000 7f1c9f7fe700 1 --2- 172.21.15.89:0/1319694543 >> v2:172.21.15.111:3300/0 conn(0x7f1ca00b4e70 0x7f1ca00ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.110+0000 7f1c9f7fe700 1 -- 172.21.15.89:0/1319694543 >> v1:172.21.15.111:6789/0 conn(0x7f1ca015d240 legacy=0x7f1ca015d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:03.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.110+0000 7f1c9f7fe700 1 -- 172.21.15.89:0/1319694543 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca00b5f20 con 0x7f1ca00b16d0 2023-10-10T20:40:03.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.111+0000 7f1c9f7fe700 1 --2- 172.21.15.89:0/1319694543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ca00b16d0 0x7f1ca00b1b40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0064910 tx=0x7f1c94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f473aee3dbf306ca server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:03.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.111+0000 7f1c9effd700 1 -- 172.21.15.89:0/1319694543 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9400b860 con 0x7f1ca00b16d0 2023-10-10T20:40:03.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.111+0000 7f1c9effd700 1 -- 172.21.15.89:0/1319694543 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9400b9c0 con 0x7f1ca00b16d0 2023-10-10T20:40:03.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.111+0000 7f1c9effd700 1 -- 172.21.15.89:0/1319694543 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9400bd60 con 0x7f1ca00b16d0 2023-10-10T20:40:03.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.111+0000 7f1ca523c700 1 -- 172.21.15.89:0/1319694543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ca00b16d0 msgr2=0x7f1ca00b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:03.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.111+0000 7f1ca523c700 1 --2- 172.21.15.89:0/1319694543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ca00b16d0 0x7f1ca00b1b40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0064910 tx=0x7f1c94009250 comp rx=0 tx=0).stop 2023-10-10T20:40:03.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.111+0000 7f1ca523c700 1 -- 172.21.15.89:0/1319694543 shutdown_connections 2023-10-10T20:40:03.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.111+0000 7f1ca523c700 1 --2- 172.21.15.89:0/1319694543 >> v2:172.21.15.111:3300/0 conn(0x7f1ca00b4e70 0x7f1ca00ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.111+0000 7f1ca523c700 1 --2- 172.21.15.89:0/1319694543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ca00b16d0 0x7f1ca00b1b40 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.111+0000 7f1ca523c700 1 -- 172.21.15.89:0/1319694543 >> 172.21.15.89:0/1319694543 conn(0x7f1ca00d10c0 msgr2=0x7f1ca00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:03.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.111+0000 7f1ca523c700 1 -- 172.21.15.89:0/1319694543 shutdown_connections 2023-10-10T20:40:03.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.111+0000 7f1ca523c700 1 -- 172.21.15.89:0/1319694543 wait complete. 2023-10-10T20:40:03.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.112+0000 7f1ca523c700 1 Processor -- start 2023-10-10T20:40:03.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.113+0000 7f1ca523c700 1 -- start start 2023-10-10T20:40:03.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.113+0000 7f1ca523c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1ca00b16d0 0x7f1ca014b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:03.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.113+0000 7f1ca523c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ca00b4e70 0x7f1ca015c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:03.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.113+0000 7f1ca523c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ca015d240 0x7f1ca015cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:03.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.113+0000 7f1ca523c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca00c6280 con 0x7f1ca00b4e70 2023-10-10T20:40:03.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.113+0000 7f1ca523c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca014b690 con 0x7f1ca015d240 2023-10-10T20:40:03.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.113+0000 7f1ca523c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca015c1c0 con 0x7f1ca00b16d0 2023-10-10T20:40:03.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.113+0000 7f1c9f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ca00b4e70 0x7f1ca015c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:03.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.113+0000 7f1c9f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ca00b4e70 0x7f1ca015c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54182/0 (socket says 172.21.15.89:54182) 2023-10-10T20:40:03.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.113+0000 7f1c9f7fe700 1 -- 172.21.15.89:0/2256755321 learned_addr learned my addr 172.21.15.89:0/2256755321 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:03.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.114+0000 7f1c9ffff700 1 --2- 172.21.15.89:0/2256755321 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1ca00b16d0 0x7f1ca014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:03.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.114+0000 7f1ca4a3b700 1 --2- 172.21.15.89:0/2256755321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ca015d240 0x7f1ca015cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:03.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.114+0000 7f1c9f7fe700 1 -- 172.21.15.89:0/2256755321 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1ca00b16d0 msgr2=0x7f1ca014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:03.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.114+0000 7f1c9f7fe700 1 --2- 172.21.15.89:0/2256755321 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1ca00b16d0 0x7f1ca014b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.114+0000 7f1c9f7fe700 1 -- 172.21.15.89:0/2256755321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ca015d240 msgr2=0x7f1ca015cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:03.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.114+0000 7f1c9f7fe700 1 --2- 172.21.15.89:0/2256755321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ca015d240 0x7f1ca015cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.114+0000 7f1c9f7fe700 1 -- 172.21.15.89:0/2256755321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c940094f0 con 0x7f1ca00b4e70 2023-10-10T20:40:03.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.114+0000 7f1c9f7fe700 1 --2- 172.21.15.89:0/2256755321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ca00b4e70 0x7f1ca015c650 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1c94006fd0 tx=0x7f1c94017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.114+0000 7f1c87fff700 1 -- 172.21.15.89:0/2256755321 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94011e20 con 0x7f1ca00b4e70 2023-10-10T20:40:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.114+0000 7f1c87fff700 1 -- 172.21.15.89:0/2256755321 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9401a630 con 0x7f1ca00b4e70 2023-10-10T20:40:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.114+0000 7f1ca523c700 1 -- 172.21.15.89:0/2256755321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca015b5b0 con 0x7f1ca00b4e70 2023-10-10T20:40:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.114+0000 7f1c87fff700 1 -- 172.21.15.89:0/2256755321 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9401a9f0 con 0x7f1ca00b4e70 2023-10-10T20:40:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.114+0000 7f1ca523c700 1 -- 172.21.15.89:0/2256755321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca015bbf0 con 0x7f1ca00b4e70 2023-10-10T20:40:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.115+0000 7f1ca523c700 1 -- 172.21.15.89:0/2256755321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca0088990 con 0x7f1ca00b4e70 2023-10-10T20:40:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.115+0000 7f1c87fff700 1 -- 172.21.15.89:0/2256755321 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f1c9402d440 con 0x7f1ca00b4e70 2023-10-10T20:40:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.116+0000 7f1c87fff700 1 --2- 172.21.15.89:0/2256755321 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1c8c067200 0x7f1c8c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.116+0000 7f1c87fff700 1 -- 172.21.15.89:0/2256755321 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1c94086200 con 0x7f1ca00b4e70 2023-10-10T20:40:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.116+0000 7f1c9ffff700 1 --2- 172.21.15.89:0/2256755321 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1c8c067200 0x7f1c8c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:03.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.116+0000 7f1c9ffff700 1 --2- 172.21.15.89:0/2256755321 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1c8c067200 0x7f1c8c0696b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1c90006fd0 tx=0x7f1c90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:03.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.120+0000 7f1c87fff700 1 -- 172.21.15.89:0/2256755321 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c940534a0 con 0x7f1ca00b4e70 2023-10-10T20:40:03.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.304+0000 7f1ca523c700 1 -- 172.21.15.89:0/2256755321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1ca015c300 con 0x7f1ca00b4e70 2023-10-10T20:40:03.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.307+0000 7f1c87fff700 1 -- 172.21.15.89:0/2256755321 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1c94056ac0 con 0x7f1ca00b4e70 2023-10-10T20:40:03.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.310+0000 7f1ca523c700 1 -- 172.21.15.89:0/2256755321 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1c8c067200 msgr2=0x7f1c8c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:03.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.310+0000 7f1ca523c700 1 --2- 172.21.15.89:0/2256755321 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1c8c067200 0x7f1c8c0696b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1c90006fd0 tx=0x7f1c90008040 comp rx=0 tx=0).stop 2023-10-10T20:40:03.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.310+0000 7f1ca523c700 1 -- 172.21.15.89:0/2256755321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ca00b4e70 msgr2=0x7f1ca015c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:03.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.310+0000 7f1ca523c700 1 --2- 172.21.15.89:0/2256755321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ca00b4e70 0x7f1ca015c650 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1c94006fd0 tx=0x7f1c94017000 comp rx=0 tx=0).stop 2023-10-10T20:40:03.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.310+0000 7f1ca523c700 1 -- 172.21.15.89:0/2256755321 shutdown_connections 2023-10-10T20:40:03.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.310+0000 7f1ca523c700 1 --2- 172.21.15.89:0/2256755321 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1c8c067200 0x7f1c8c0696b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.310+0000 7f1ca523c700 1 --2- 172.21.15.89:0/2256755321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ca015d240 0x7f1ca015cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.310+0000 7f1ca523c700 1 --2- 172.21.15.89:0/2256755321 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1ca00b16d0 0x7f1ca014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.310+0000 7f1ca523c700 1 --2- 172.21.15.89:0/2256755321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ca00b4e70 0x7f1ca015c650 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.310+0000 7f1ca523c700 1 -- 172.21.15.89:0/2256755321 >> 172.21.15.89:0/2256755321 conn(0x7f1ca00d10c0 msgr2=0x7f1ca0158fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:03.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.310+0000 7f1ca523c700 1 -- 172.21.15.89:0/2256755321 shutdown_connections 2023-10-10T20:40:03.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.310+0000 7f1ca523c700 1 -- 172.21.15.89:0/2256755321 wait complete. 2023-10-10T20:40:03.325 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:40:03.326 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:40:03.367 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:40:03.417 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:40:03.417 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:40:03.418 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:40:03.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.530+0000 7f7a4413d700 1 Processor -- start 2023-10-10T20:40:03.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.531+0000 7f7a4413d700 1 -- start start 2023-10-10T20:40:03.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.531+0000 7f7a4413d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7a3c159c60 0x7f7a3c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:03.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.531+0000 7f7a4413d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7a3c15db10 0x7f7a3c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:03.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.531+0000 7f7a4413d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3c15a160 con 0x7f7a3c159890 2023-10-10T20:40:03.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.531+0000 7f7a4413d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3c15df50 con 0x7f7a3c159c60 2023-10-10T20:40:03.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.531+0000 7f7a4413d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3c0d13a0 con 0x7f7a3c15db10 2023-10-10T20:40:03.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.531+0000 7f7a416d8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7a3c159c60 0x7f7a3c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:03.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.532+0000 7f7a416d8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7a3c159c60 0x7f7a3c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54198/0 (socket says 172.21.15.89:54198) 2023-10-10T20:40:03.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.532+0000 7f7a416d8700 1 -- 172.21.15.89:0/677349514 learned_addr learned my addr 172.21.15.89:0/677349514 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.532+0000 7f7a426da700 1 --2- 172.21.15.89:0/677349514 >> v2:172.21.15.111:3300/0 conn(0x7f7a3c15db10 0x7f7a3c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.532+0000 7f7a416d8700 1 -- 172.21.15.89:0/677349514 >> v2:172.21.15.111:3300/0 conn(0x7f7a3c15db10 msgr2=0x7f7a3c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.532+0000 7f7a416d8700 1 --2- 172.21.15.89:0/677349514 >> v2:172.21.15.111:3300/0 conn(0x7f7a3c15db10 0x7f7a3c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.532+0000 7f7a416d8700 1 -- 172.21.15.89:0/677349514 >> v1:172.21.15.111:6789/0 conn(0x7f7a3c159890 legacy=0x7f7a3c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.532+0000 7f7a416d8700 1 -- 172.21.15.89:0/677349514 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c15e140 con 0x7f7a3c159c60 2023-10-10T20:40:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.532+0000 7f7a416d8700 1 --2- 172.21.15.89:0/677349514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a3c159c60 0x7f7a3c15d5d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c06a890 tx=0x7f7a30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96c59f30b5c24a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.532+0000 7f7a40ed7700 1 -- 172.21.15.89:0/677349514 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3000b820 con 0x7f7a3c159c60 2023-10-10T20:40:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.532+0000 7f7a40ed7700 1 -- 172.21.15.89:0/677349514 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3000b980 con 0x7f7a3c159c60 2023-10-10T20:40:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.532+0000 7f7a40ed7700 1 -- 172.21.15.89:0/677349514 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3000bd20 con 0x7f7a3c159c60 2023-10-10T20:40:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.533+0000 7f7a4413d700 1 -- 172.21.15.89:0/677349514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a3c159c60 msgr2=0x7f7a3c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.533+0000 7f7a4413d700 1 --2- 172.21.15.89:0/677349514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a3c159c60 0x7f7a3c15d5d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c06a890 tx=0x7f7a30009250 comp rx=0 tx=0).stop 2023-10-10T20:40:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.533+0000 7f7a4413d700 1 -- 172.21.15.89:0/677349514 shutdown_connections 2023-10-10T20:40:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.533+0000 7f7a4413d700 1 --2- 172.21.15.89:0/677349514 >> v2:172.21.15.111:3300/0 conn(0x7f7a3c15db10 0x7f7a3c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.533+0000 7f7a4413d700 1 --2- 172.21.15.89:0/677349514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a3c159c60 0x7f7a3c15d5d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.533+0000 7f7a4413d700 1 -- 172.21.15.89:0/677349514 >> 172.21.15.89:0/677349514 conn(0x7f7a3c068e20 msgr2=0x7f7a3c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.533+0000 7f7a4413d700 1 -- 172.21.15.89:0/677349514 shutdown_connections 2023-10-10T20:40:03.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.533+0000 7f7a4413d700 1 -- 172.21.15.89:0/677349514 wait complete. 2023-10-10T20:40:03.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.534+0000 7f7a4413d700 1 Processor -- start 2023-10-10T20:40:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.534+0000 7f7a4413d700 1 -- start start 2023-10-10T20:40:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a4413d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a3c159890 0x7f7a3c0cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a4413d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a3c159c60 0x7f7a3c150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a4413d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a3c15db10 0x7f7a3c0c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a4413d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c151e10 con 0x7f7a3c159c60 2023-10-10T20:40:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a4413d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c0cfef0 con 0x7f7a3c15db10 2023-10-10T20:40:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a4413d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c151200 con 0x7f7a3c159890 2023-10-10T20:40:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a416d8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a3c159c60 0x7f7a3c150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a416d8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a3c159c60 0x7f7a3c150820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54200/0 (socket says 172.21.15.89:54200) 2023-10-10T20:40:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a416d8700 1 -- 172.21.15.89:0/496041319 learned_addr learned my addr 172.21.15.89:0/496041319 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a426da700 1 --2- 172.21.15.89:0/496041319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a3c15db10 0x7f7a3c0c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a41ed9700 1 --2- 172.21.15.89:0/496041319 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a3c159890 0x7f7a3c0cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a416d8700 1 -- 172.21.15.89:0/496041319 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a3c159890 msgr2=0x7f7a3c0cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a416d8700 1 --2- 172.21.15.89:0/496041319 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a3c159890 0x7f7a3c0cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a416d8700 1 -- 172.21.15.89:0/496041319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a3c15db10 msgr2=0x7f7a3c0c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a416d8700 1 --2- 172.21.15.89:0/496041319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a3c15db10 0x7f7a3c0c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a416d8700 1 -- 172.21.15.89:0/496041319 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a300094f0 con 0x7f7a3c159c60 2023-10-10T20:40:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a426da700 1 --2- 172.21.15.89:0/496041319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a3c15db10 0x7f7a3c0c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a416d8700 1 --2- 172.21.15.89:0/496041319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a3c159c60 0x7f7a3c150820 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7a300030c0 tx=0x7f7a30016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a39ffb700 1 -- 172.21.15.89:0/496041319 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a30011e60 con 0x7f7a3c159c60 2023-10-10T20:40:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a39ffb700 1 -- 172.21.15.89:0/496041319 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a30020070 con 0x7f7a3c159c60 2023-10-10T20:40:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a4413d700 1 -- 172.21.15.89:0/496041319 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3c1514e0 con 0x7f7a3c159c60 2023-10-10T20:40:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a39ffb700 1 -- 172.21.15.89:0/496041319 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3001b6a0 con 0x7f7a3c159c60 2023-10-10T20:40:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.535+0000 7f7a4413d700 1 -- 172.21.15.89:0/496041319 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3c151aa0 con 0x7f7a3c159c60 2023-10-10T20:40:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.536+0000 7f7a4413d700 1 -- 172.21.15.89:0/496041319 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3c151f80 con 0x7f7a3c159c60 2023-10-10T20:40:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.537+0000 7f7a39ffb700 1 -- 172.21.15.89:0/496041319 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f7a30022030 con 0x7f7a3c159c60 2023-10-10T20:40:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.537+0000 7f7a39ffb700 1 --2- 172.21.15.89:0/496041319 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7a28067200 0x7f7a280696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.537+0000 7f7a39ffb700 1 -- 172.21.15.89:0/496041319 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7a300866a0 con 0x7f7a3c159c60 2023-10-10T20:40:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.537+0000 7f7a41ed9700 1 --2- 172.21.15.89:0/496041319 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7a28067200 0x7f7a280696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.538+0000 7f7a41ed9700 1 --2- 172.21.15.89:0/496041319 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7a28067200 0x7f7a280696b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c007fd0 tx=0x7f7a2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:03.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.541+0000 7f7a39ffb700 1 -- 172.21.15.89:0/496041319 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a300538c0 con 0x7f7a3c159c60 2023-10-10T20:40:03.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.770+0000 7f7a4413d700 1 -- 172.21.15.89:0/496041319 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a3c151f50 con 0x7f7a3c159c60 2023-10-10T20:40:03.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.771+0000 7f7a39ffb700 1 -- 172.21.15.89:0/496041319 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f7a30056ee0 con 0x7f7a3c159c60 2023-10-10T20:40:03.773 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:03.774 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5324,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/1607893755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":1607893755},{"type":"v1","addr":"172.21.15.111:6835","nonce":1607893755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5938,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/3221626574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3221626574},{"type":"v1","addr":"172.21.15.111:6834","nonce":3221626574}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6069,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/1480864169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":1480864169},{"type":"v1","addr":"172.21.15.89:6836","nonce":1480864169}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2347053559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2347053559},{"type":"v1","addr":"172.21.15.89:6837","nonce":2347053559}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[]} 2023-10-10T20:40:03.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.772+0000 7f7a4413d700 1 -- 172.21.15.89:0/496041319 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7a28067200 msgr2=0x7f7a280696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:03.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.772+0000 7f7a4413d700 1 --2- 172.21.15.89:0/496041319 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7a28067200 0x7f7a280696b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c007fd0 tx=0x7f7a2c009040 comp rx=0 tx=0).stop 2023-10-10T20:40:03.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.773+0000 7f7a4413d700 1 -- 172.21.15.89:0/496041319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a3c159c60 msgr2=0x7f7a3c150820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:03.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.773+0000 7f7a4413d700 1 --2- 172.21.15.89:0/496041319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a3c159c60 0x7f7a3c150820 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7a300030c0 tx=0x7f7a30016000 comp rx=0 tx=0).stop 2023-10-10T20:40:03.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.773+0000 7f7a4413d700 1 -- 172.21.15.89:0/496041319 shutdown_connections 2023-10-10T20:40:03.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.773+0000 7f7a4413d700 1 --2- 172.21.15.89:0/496041319 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7a28067200 0x7f7a280696b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.773+0000 7f7a4413d700 1 --2- 172.21.15.89:0/496041319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a3c15db10 0x7f7a3c0c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.773+0000 7f7a4413d700 1 --2- 172.21.15.89:0/496041319 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a3c159890 0x7f7a3c0cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.773+0000 7f7a4413d700 1 --2- 172.21.15.89:0/496041319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a3c159c60 0x7f7a3c150820 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.773+0000 7f7a4413d700 1 -- 172.21.15.89:0/496041319 >> 172.21.15.89:0/496041319 conn(0x7f7a3c068e20 msgr2=0x7f7a3c1577b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:03.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.773+0000 7f7a4413d700 1 -- 172.21.15.89:0/496041319 shutdown_connections 2023-10-10T20:40:03.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.773+0000 7f7a4413d700 1 -- 172.21.15.89:0/496041319 wait complete. 2023-10-10T20:40:03.777 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 49 2023-10-10T20:40:03.787 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:40:03.787 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:40:03.788 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:40:03.788 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:40:03.788 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:40:03.788 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:40:03.789 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:40:03.789 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:40:03.789 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:40:03.789 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:40:03.790 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:40:03.790 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:40:03.790 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:40:03.791 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:40:03.791 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:40:03.791 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:40:03.796 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:40:03.796 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:40:03.797 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:40:03.798 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:40:03.801 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:40:03.801 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:40:03.802 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:40:03.803 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:40:03.840 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:40:03.841 DEBUG:teuthology.parallel:result is None 2023-10-10T20:40:03.841 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:40:03.842 DEBUG:teuthology.parallel:result is None 2023-10-10T20:40:03.844 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:40:03.844 DEBUG:teuthology.parallel:result is None 2023-10-10T20:40:03.845 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:40:03.845 DEBUG:teuthology.parallel:result is None 2023-10-10T20:40:03.846 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:40:03.913 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:40:03.913 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-10T20:40:03.925 INFO:tasks.ceph.mds.b.smithi111.stdout:starting mds.b at 2023-10-10T20:40:03.926 INFO:tasks.ceph.mds.d.smithi111.stdout:starting mds.d at 2023-10-10T20:40:03.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.965+0000 7ff5483e5700 1 Processor -- start 2023-10-10T20:40:03.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.965+0000 7ff5483e5700 1 -- start start 2023-10-10T20:40:03.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.966+0000 7ff5483e5700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff540159c60 0x7ff54015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:03.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.966+0000 7ff5483e5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff54015db10 0x7ff540069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:03.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.966+0000 7ff5483e5700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54015a160 con 0x7ff540159890 2023-10-10T20:40:03.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.966+0000 7ff5483e5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54015df50 con 0x7ff54015db10 2023-10-10T20:40:03.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.966+0000 7ff5483e5700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5400d13a0 con 0x7ff540159c60 2023-10-10T20:40:03.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.966+0000 7ff546982700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff54015db10 0x7ff540069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:03.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.966+0000 7ff545980700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff540159c60 0x7ff54015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:03.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.966+0000 7ff546982700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff54015db10 0x7ff540069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54238/0 (socket says 172.21.15.89:54238) 2023-10-10T20:40:03.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.966+0000 7ff546982700 1 -- 172.21.15.89:0/1273473840 learned_addr learned my addr 172.21.15.89:0/1273473840 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:03.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.966+0000 7ff545980700 1 -- 172.21.15.89:0/1273473840 >> v2:172.21.15.89:3300/0 conn(0x7ff54015db10 msgr2=0x7ff540069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:03.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.966+0000 7ff545980700 1 --2- 172.21.15.89:0/1273473840 >> v2:172.21.15.89:3300/0 conn(0x7ff54015db10 0x7ff540069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.966+0000 7ff545980700 1 -- 172.21.15.89:0/1273473840 >> v1:172.21.15.111:6789/0 conn(0x7ff540159890 legacy=0x7ff54015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:03.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.966+0000 7ff545980700 1 -- 172.21.15.89:0/1273473840 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54015e140 con 0x7ff540159c60 2023-10-10T20:40:03.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.966+0000 7ff54517f700 1 -- 172.21.15.89:0/1273473840 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859674319 0 0) 0x7ff54015a160 con 0x7ff540159890 2023-10-10T20:40:03.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.966+0000 7ff545980700 1 --2- 172.21.15.89:0/1273473840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff540159c60 0x7ff54015d5d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff5400b67f0 tx=0x7ff53c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=857ed72c7ec84cd7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:03.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.966+0000 7ff54517f700 1 -- 172.21.15.89:0/1273473840 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c011630 con 0x7ff540159c60 2023-10-10T20:40:03.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.966+0000 7ff54517f700 1 -- 172.21.15.89:0/1273473840 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c011790 con 0x7ff540159c60 2023-10-10T20:40:03.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.966+0000 7ff54517f700 1 -- 172.21.15.89:0/1273473840 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c011b30 con 0x7ff540159c60 2023-10-10T20:40:03.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.967+0000 7ff5483e5700 1 -- 172.21.15.89:0/1273473840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff540159c60 msgr2=0x7ff54015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:03.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.967+0000 7ff5483e5700 1 --2- 172.21.15.89:0/1273473840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff540159c60 0x7ff54015d5d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff5400b67f0 tx=0x7ff53c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.967+0000 7ff5483e5700 1 -- 172.21.15.89:0/1273473840 shutdown_connections 2023-10-10T20:40:03.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.967+0000 7ff5483e5700 1 --2- 172.21.15.89:0/1273473840 >> v2:172.21.15.89:3300/0 conn(0x7ff54015db10 0x7ff540069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.967+0000 7ff5483e5700 1 --2- 172.21.15.89:0/1273473840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff540159c60 0x7ff54015d5d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.967+0000 7ff5483e5700 1 -- 172.21.15.89:0/1273473840 >> 172.21.15.89:0/1273473840 conn(0x7ff540068e20 msgr2=0x7ff5400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:03.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.967+0000 7ff5483e5700 1 -- 172.21.15.89:0/1273473840 shutdown_connections 2023-10-10T20:40:03.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.967+0000 7ff5483e5700 1 -- 172.21.15.89:0/1273473840 wait complete. 2023-10-10T20:40:03.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.968+0000 7ff5483e5700 1 Processor -- start 2023-10-10T20:40:03.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.968+0000 7ff5483e5700 1 -- start start 2023-10-10T20:40:03.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.969+0000 7ff5483e5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff540159890 0x7ff5400b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:03.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.969+0000 7ff5483e5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff540159c60 0x7ff5400c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:03.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.969+0000 7ff5483e5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff54015db10 0x7ff54014b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:03.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.969+0000 7ff5483e5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5400c6610 con 0x7ff54015db10 2023-10-10T20:40:03.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.969+0000 7ff5483e5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54015c1e0 con 0x7ff540159c60 2023-10-10T20:40:03.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.969+0000 7ff5483e5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5400c5ad0 con 0x7ff540159890 2023-10-10T20:40:03.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.969+0000 7ff546982700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff54015db10 0x7ff54014b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:03.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.969+0000 7ff546982700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff54015db10 0x7ff54014b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54250/0 (socket says 172.21.15.89:54250) 2023-10-10T20:40:03.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.969+0000 7ff546982700 1 -- 172.21.15.89:0/1436528154 learned_addr learned my addr 172.21.15.89:0/1436528154 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:03.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.969+0000 7ff546181700 1 --2- 172.21.15.89:0/1436528154 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff540159890 0x7ff5400b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:03.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.969+0000 7ff545980700 1 --2- 172.21.15.89:0/1436528154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff540159c60 0x7ff5400c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:03.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.969+0000 7ff546982700 1 -- 172.21.15.89:0/1436528154 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff540159890 msgr2=0x7ff5400b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:03.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.969+0000 7ff546982700 1 --2- 172.21.15.89:0/1436528154 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff540159890 0x7ff5400b1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.969+0000 7ff546982700 1 -- 172.21.15.89:0/1436528154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff540159c60 msgr2=0x7ff5400c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:03.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.969+0000 7ff546982700 1 --2- 172.21.15.89:0/1436528154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff540159c60 0x7ff5400c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:03.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.969+0000 7ff546982700 1 -- 172.21.15.89:0/1436528154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c009420 con 0x7ff54015db10 2023-10-10T20:40:03.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.969+0000 7ff545980700 1 --2- 172.21.15.89:0/1436528154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff540159c60 0x7ff5400c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:03.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.969+0000 7ff546982700 1 --2- 172.21.15.89:0/1436528154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff54015db10 0x7ff54014b800 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff538009de0 tx=0x7ff53800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:03.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.969+0000 7ff5367fc700 1 -- 172.21.15.89:0/1436528154 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53800c940 con 0x7ff54015db10 2023-10-10T20:40:03.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.970+0000 7ff5367fc700 1 -- 172.21.15.89:0/1436528154 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53800f040 con 0x7ff54015db10 2023-10-10T20:40:03.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.970+0000 7ff5367fc700 1 -- 172.21.15.89:0/1436528154 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53800cd10 con 0x7ff54015db10 2023-10-10T20:40:03.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.970+0000 7ff5483e5700 1 -- 172.21.15.89:0/1436528154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5400c5d70 con 0x7ff54015db10 2023-10-10T20:40:03.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.970+0000 7ff5483e5700 1 -- 172.21.15.89:0/1436528154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5400c6330 con 0x7ff54015db10 2023-10-10T20:40:03.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.970+0000 7ff5483e5700 1 -- 172.21.15.89:0/1436528154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54004c3c0 con 0x7ff54015db10 2023-10-10T20:40:03.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.971+0000 7ff5367fc700 1 -- 172.21.15.89:0/1436528154 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7ff538014650 con 0x7ff54015db10 2023-10-10T20:40:03.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.971+0000 7ff5367fc700 1 --2- 172.21.15.89:0/1436528154 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff524067200 0x7ff5240696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:03.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.972+0000 7ff5367fc700 1 -- 172.21.15.89:0/1436528154 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff538084f80 con 0x7ff54015db10 2023-10-10T20:40:03.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.972+0000 7ff546181700 1 --2- 172.21.15.89:0/1436528154 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff524067200 0x7ff5240696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:03.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.972+0000 7ff546181700 1 --2- 172.21.15.89:0/1436528154 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff524067200 0x7ff5240696b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff5300011e0 tx=0x7ff530009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:03.976+0000 7ff5367fc700 1 -- 172.21.15.89:0/1436528154 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff538055b20 con 0x7ff54015db10 2023-10-10T20:40:04.004 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-10-10T20:40:04.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:04.175+0000 7ff5483e5700 1 -- 172.21.15.89:0/1436528154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff54015c320 con 0x7ff54015db10 2023-10-10T20:40:04.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:04.990+0000 7ff5367fc700 1 -- 172.21.15.89:0/1436528154 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v84) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff5380556b0 con 0x7ff54015db10 2023-10-10T20:40:04.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:04.992+0000 7ff5483e5700 1 -- 172.21.15.89:0/1436528154 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff524067200 msgr2=0x7ff5240696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:04.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:04.992+0000 7ff5483e5700 1 --2- 172.21.15.89:0/1436528154 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff524067200 0x7ff5240696b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff5300011e0 tx=0x7ff530009040 comp rx=0 tx=0).stop 2023-10-10T20:40:04.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:04.992+0000 7ff5483e5700 1 -- 172.21.15.89:0/1436528154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff54015db10 msgr2=0x7ff54014b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:04.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:04.992+0000 7ff5483e5700 1 --2- 172.21.15.89:0/1436528154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff54015db10 0x7ff54014b800 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff538009de0 tx=0x7ff53800b040 comp rx=0 tx=0).stop 2023-10-10T20:40:04.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:04.992+0000 7ff5483e5700 1 -- 172.21.15.89:0/1436528154 shutdown_connections 2023-10-10T20:40:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:04.992+0000 7ff5483e5700 1 --2- 172.21.15.89:0/1436528154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff54015db10 0x7ff54014b800 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:04.992+0000 7ff5483e5700 1 --2- 172.21.15.89:0/1436528154 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff540159890 0x7ff5400b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:04.992+0000 7ff5483e5700 1 --2- 172.21.15.89:0/1436528154 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff524067200 0x7ff5240696b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:04.992+0000 7ff5483e5700 1 --2- 172.21.15.89:0/1436528154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff540159c60 0x7ff5400c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:04.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:04.992+0000 7ff5483e5700 1 -- 172.21.15.89:0/1436528154 >> 172.21.15.89:0/1436528154 conn(0x7ff540068e20 msgr2=0x7ff54014d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:04.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:04.993+0000 7ff5483e5700 1 -- 172.21.15.89:0/1436528154 shutdown_connections 2023-10-10T20:40:04.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:04.993+0000 7ff5483e5700 1 -- 172.21.15.89:0/1436528154 wait complete. 2023-10-10T20:40:04.998 INFO:teuthology.orchestra.run.smithi089.stderr: removed all blocklist entries 2023-10-10T20:40:05.008 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:40:05.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.124+0000 7fe149682700 1 Processor -- start 2023-10-10T20:40:05.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.124+0000 7fe149682700 1 -- start start 2023-10-10T20:40:05.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.125+0000 7fe149682700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe1441597f0 0x7fe1440b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:05.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.125+0000 7fe149682700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe14415d230 0x7fe14415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:05.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.125+0000 7fe149682700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe14415dbb0 con 0x7fe144159420 2023-10-10T20:40:05.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.125+0000 7fe149682700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1440b11d0 con 0x7fe1441597f0 2023-10-10T20:40:05.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.125+0000 7fe149682700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440b14b0 con 0x7fe14415d230 2023-10-10T20:40:05.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.125+0000 7fe1437fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe1441597f0 0x7fe1440b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:05.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.125+0000 7fe143fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe144159420 0x7fe1440b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59420/0 (socket says 172.21.15.89:59420) 2023-10-10T20:40:05.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.125+0000 7fe143fff700 1 -- 172.21.15.89:0/3439715182 learned_addr learned my addr 172.21.15.89:0/3439715182 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:05.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.125+0000 7fe148e81700 1 --2- 172.21.15.89:0/3439715182 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe14415d230 0x7fe14415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:05.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.125+0000 7fe1437fe700 1 -- 172.21.15.89:0/3439715182 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe14415d230 msgr2=0x7fe14415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:05.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.125+0000 7fe1437fe700 1 --2- 172.21.15.89:0/3439715182 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe14415d230 0x7fe14415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.125+0000 7fe1437fe700 1 -- 172.21.15.89:0/3439715182 >> v1:172.21.15.89:6789/0 conn(0x7fe144159420 legacy=0x7fe1440b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:05.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.125+0000 7fe1437fe700 1 -- 172.21.15.89:0/3439715182 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1440b1790 con 0x7fe1441597f0 2023-10-10T20:40:05.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.125+0000 7fe142ffd700 1 -- 172.21.15.89:0/3439715182 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083012233 0 0) 0x7fe14415dbb0 con 0x7fe144159420 2023-10-10T20:40:05.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.125+0000 7fe1437fe700 1 --2- 172.21.15.89:0/3439715182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1441597f0 0x7fe1440b2270 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe1380098e0 tx=0x7fe1380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504d86a1624ee827 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:05.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.126+0000 7fe142ffd700 1 -- 172.21.15.89:0/3439715182 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138009e80 con 0x7fe1441597f0 2023-10-10T20:40:05.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.126+0000 7fe142ffd700 1 -- 172.21.15.89:0/3439715182 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13800b860 con 0x7fe1441597f0 2023-10-10T20:40:05.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.126+0000 7fe142ffd700 1 -- 172.21.15.89:0/3439715182 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13800bbe0 con 0x7fe1441597f0 2023-10-10T20:40:05.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.126+0000 7fe149682700 1 -- 172.21.15.89:0/3439715182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1441597f0 msgr2=0x7fe1440b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:05.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.126+0000 7fe149682700 1 --2- 172.21.15.89:0/3439715182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1441597f0 0x7fe1440b2270 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe1380098e0 tx=0x7fe1380091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.126+0000 7fe149682700 1 -- 172.21.15.89:0/3439715182 shutdown_connections 2023-10-10T20:40:05.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.126+0000 7fe149682700 1 --2- 172.21.15.89:0/3439715182 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe14415d230 0x7fe14415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.126+0000 7fe149682700 1 --2- 172.21.15.89:0/3439715182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1441597f0 0x7fe1440b2270 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.126+0000 7fe149682700 1 -- 172.21.15.89:0/3439715182 >> 172.21.15.89:0/3439715182 conn(0x7fe1440d0660 msgr2=0x7fe1440d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:05.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.126+0000 7fe149682700 1 -- 172.21.15.89:0/3439715182 shutdown_connections 2023-10-10T20:40:05.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.126+0000 7fe149682700 1 -- 172.21.15.89:0/3439715182 wait complete. 2023-10-10T20:40:05.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.127+0000 7fe149682700 1 Processor -- start 2023-10-10T20:40:05.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.127+0000 7fe149682700 1 -- start start 2023-10-10T20:40:05.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.128+0000 7fe149682700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe144159420 0x7fe144154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:05.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.128+0000 7fe149682700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1441597f0 0x7fe1441572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:05.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.128+0000 7fe149682700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe14415d230 0x7fe144157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.128+0000 7fe149682700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14415cc50 con 0x7fe1441597f0 2023-10-10T20:40:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.128+0000 7fe149682700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14415bbb0 con 0x7fe14415d230 2023-10-10T20:40:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.128+0000 7fe149682700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14415be90 con 0x7fe144159420 2023-10-10T20:40:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.128+0000 7fe1437fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1441597f0 0x7fe1441572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:05.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.128+0000 7fe1437fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1441597f0 0x7fe1441572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54274/0 (socket says 172.21.15.89:54274) 2023-10-10T20:40:05.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.128+0000 7fe1437fe700 1 -- 172.21.15.89:0/139629605 learned_addr learned my addr 172.21.15.89:0/139629605 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:05.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.128+0000 7fe143fff700 1 --2- 172.21.15.89:0/139629605 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe144159420 0x7fe144154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:05.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.129+0000 7fe1437fe700 1 -- 172.21.15.89:0/139629605 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe144159420 msgr2=0x7fe144154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:05.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.129+0000 7fe1437fe700 1 --2- 172.21.15.89:0/139629605 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe144159420 0x7fe144154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.129+0000 7fe1437fe700 1 -- 172.21.15.89:0/139629605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe14415d230 msgr2=0x7fe144157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:05.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.129+0000 7fe1437fe700 1 --2- 172.21.15.89:0/139629605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe14415d230 0x7fe144157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.129+0000 7fe1437fe700 1 -- 172.21.15.89:0/139629605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe138009420 con 0x7fe1441597f0 2023-10-10T20:40:05.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.129+0000 7fe1437fe700 1 --2- 172.21.15.89:0/139629605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1441597f0 0x7fe1441572c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe138013000 tx=0x7fe138017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:05.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.129+0000 7fe12bfff700 1 -- 172.21.15.89:0/139629605 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13800b9d0 con 0x7fe1441597f0 2023-10-10T20:40:05.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.129+0000 7fe12bfff700 1 -- 172.21.15.89:0/139629605 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13800bb30 con 0x7fe1441597f0 2023-10-10T20:40:05.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.129+0000 7fe149682700 1 -- 172.21.15.89:0/139629605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe14415c170 con 0x7fe1441597f0 2023-10-10T20:40:05.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.129+0000 7fe12bfff700 1 -- 172.21.15.89:0/139629605 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138025670 con 0x7fe1441597f0 2023-10-10T20:40:05.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.129+0000 7fe149682700 1 -- 172.21.15.89:0/139629605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14415c730 con 0x7fe1441597f0 2023-10-10T20:40:05.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.130+0000 7fe149682700 1 -- 172.21.15.89:0/139629605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe14415c920 con 0x7fe1441597f0 2023-10-10T20:40:05.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.130+0000 7fe12bfff700 1 -- 172.21.15.89:0/139629605 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fe138023070 con 0x7fe1441597f0 2023-10-10T20:40:05.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.130+0000 7fe12bfff700 1 --2- 172.21.15.89:0/139629605 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe130067230 0x7fe1300696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:05.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.131+0000 7fe12bfff700 1 -- 172.21.15.89:0/139629605 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe138085be0 con 0x7fe1441597f0 2023-10-10T20:40:05.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.131+0000 7fe143fff700 1 --2- 172.21.15.89:0/139629605 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe130067230 0x7fe1300696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:05.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.131+0000 7fe143fff700 1 --2- 172.21.15.89:0/139629605 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe130067230 0x7fe1300696e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe1340011e0 tx=0x7fe134009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:05.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.135+0000 7fe12bfff700 1 -- 172.21.15.89:0/139629605 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe138052e50 con 0x7fe1441597f0 2023-10-10T20:40:05.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.390+0000 7fe149682700 1 -- 172.21.15.89:0/139629605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe14404c3c0 con 0x7fe1441597f0 2023-10-10T20:40:05.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.391+0000 7fe12bfff700 1 -- 172.21.15.89:0/139629605 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe138056470 con 0x7fe1441597f0 2023-10-10T20:40:05.393 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:05.394 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:40:05.394 INFO:teuthology.orchestra.run.smithi089.stdout: "auth_dump": [ 2023-10-10T20:40:05.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:05.394 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.a", 2023-10-10T20:40:05.395 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlZLSNLhAA8WZTrsht6tAOP+gxaTXKMg==", 2023-10-10T20:40:05.395 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:05.395 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:40:05.395 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:40:05.395 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:40:05.396 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:05.396 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:05.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:05.396 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.b", 2023-10-10T20:40:05.397 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgyDEORAAMMoSxPFZc0bCIwWy8UsXgA==", 2023-10-10T20:40:05.397 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:05.397 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:40:05.397 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:40:05.397 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:40:05.398 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:05.398 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:05.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:05.399 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.c", 2023-10-10T20:40:05.399 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlMFxSNRAASuQEJFghLIOkTQWa4ckTvg==", 2023-10-10T20:40:05.399 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:05.400 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:40:05.400 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:40:05.400 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:40:05.400 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:05.400 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:05.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:05.401 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.d", 2023-10-10T20:40:05.401 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlPkvrBBAA2VLJmv2t8szTEGtCVwalJw==", 2023-10-10T20:40:05.401 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:05.402 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:40:05.402 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:40:05.402 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:40:05.403 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:05.403 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:05.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:05.404 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.0", 2023-10-10T20:40:05.404 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQArtSVltOcKHRAAk8+zUus04oTVI9KvfhjlXg==", 2023-10-10T20:40:05.405 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:05.405 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:40:05.405 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:40:05.405 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:05.406 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:05.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:05.406 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.1", 2023-10-10T20:40:05.406 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAutSVly8P0GhAAe1bvR63+ptJg84lizMNdQw==", 2023-10-10T20:40:05.407 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:05.407 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:40:05.407 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:40:05.407 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:05.407 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:05.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:05.408 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.2", 2023-10-10T20:40:05.408 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAxtSVl0x9fGRAAEKi7+6EW5mzPbebmz0+sgQ==", 2023-10-10T20:40:05.408 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:05.408 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:40:05.409 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:40:05.409 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:05.409 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:05.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:05.410 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.3", 2023-10-10T20:40:05.410 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA0tSVlfJyaGBAAa+8w54U6v7+npbdufKJruA==", 2023-10-10T20:40:05.410 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:05.411 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:40:05.411 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:40:05.411 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:05.411 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:05.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:05.412 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.4", 2023-10-10T20:40:05.412 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA6tSVlEQ/QFxAA/PRB3c90BuAlF6xekGfX9A==", 2023-10-10T20:40:05.412 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:05.413 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:40:05.413 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:40:05.413 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:05.413 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:05.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:05.414 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.5", 2023-10-10T20:40:05.414 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA9tSVlMOKrFRAAuBnIsEz2VmeSZi9XWzjlzA==", 2023-10-10T20:40:05.415 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:05.415 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:40:05.415 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:40:05.415 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:05.415 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:05.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:05.416 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.6", 2023-10-10T20:40:05.416 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBAtSVlH7PQFBAAHr6c7d4TYJEu/KSFEf+EOA==", 2023-10-10T20:40:05.416 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:05.416 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:40:05.417 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:40:05.417 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:05.417 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:05.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:05.418 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.7", 2023-10-10T20:40:05.418 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBDtSVlAhC1FBAAuE2BwAA/f7VOkZfVdqAG7A==", 2023-10-10T20:40:05.418 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:05.418 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:40:05.419 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:40:05.419 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:05.419 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:05.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:05.420 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.0", 2023-10-10T20:40:05.420 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlRd3HCBAAb0QP2VfTS7aelno1mFpGAA==", 2023-10-10T20:40:05.420 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:05.420 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:40:05.420 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow r", 2023-10-10T20:40:05.420 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:40:05.421 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:05.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:05.421 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.1", 2023-10-10T20:40:05.421 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlgBouERAAqPdE0KvJU3M53B6Y6AH5GQ==", 2023-10-10T20:40:05.422 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:05.422 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:40:05.422 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:40:05.422 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:40:05.422 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:40:05.423 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:05.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:05.423 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.2", 2023-10-10T20:40:05.423 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlXA4tDhAAoPVYQjECpzD+C67mnsY5IA==", 2023-10-10T20:40:05.423 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:05.424 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:40:05.424 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:40:05.424 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:40:05.424 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:40:05.424 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:05.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:05.425 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.3", 2023-10-10T20:40:05.425 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlZ5KGFhAAuKmZwbOfmmw1nAj5ry6tHw==", 2023-10-10T20:40:05.425 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:05.426 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:40:05.426 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:40:05.426 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:40:05.426 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:40:05.427 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:05.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:05.427 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.admin", 2023-10-10T20:40:05.427 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgpL2EhAAoDOki002rfBrauTgfu481Q==", 2023-10-10T20:40:05.428 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:05.428 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:40:05.428 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:40:05.428 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:40:05.428 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:05.428 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:05.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:05.429 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.x", 2023-10-10T20:40:05.429 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlBhaJKxAArraBxT5xWg515LfpJ7MlbA==", 2023-10-10T20:40:05.429 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:05.430 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:40:05.430 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:40:05.430 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:05.430 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:05.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:05.431 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.y", 2023-10-10T20:40:05.431 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlTZIyJxAAWAOsitHwsIxJWwp3b1Z98Q==", 2023-10-10T20:40:05.431 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:05.431 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:40:05.432 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:40:05.432 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:05.432 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.432 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:05.433 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-10T20:40:05.433 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:40:05.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.393+0000 7fe149682700 1 -- 172.21.15.89:0/139629605 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe130067230 msgr2=0x7fe1300696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:05.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.393+0000 7fe149682700 1 --2- 172.21.15.89:0/139629605 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe130067230 0x7fe1300696e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe1340011e0 tx=0x7fe134009040 comp rx=0 tx=0).stop 2023-10-10T20:40:05.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.393+0000 7fe149682700 1 -- 172.21.15.89:0/139629605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1441597f0 msgr2=0x7fe1441572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:05.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.393+0000 7fe149682700 1 --2- 172.21.15.89:0/139629605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1441597f0 0x7fe1441572c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe138013000 tx=0x7fe138017000 comp rx=0 tx=0).stop 2023-10-10T20:40:05.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.393+0000 7fe149682700 1 -- 172.21.15.89:0/139629605 shutdown_connections 2023-10-10T20:40:05.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.393+0000 7fe149682700 1 --2- 172.21.15.89:0/139629605 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe130067230 0x7fe1300696e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.393+0000 7fe149682700 1 --2- 172.21.15.89:0/139629605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe14415d230 0x7fe144157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.393+0000 7fe149682700 1 --2- 172.21.15.89:0/139629605 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe144159420 0x7fe144154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.393+0000 7fe149682700 1 --2- 172.21.15.89:0/139629605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1441597f0 0x7fe1441572c0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.393+0000 7fe149682700 1 -- 172.21.15.89:0/139629605 >> 172.21.15.89:0/139629605 conn(0x7fe1440d0660 msgr2=0x7fe1440ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:05.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.393+0000 7fe149682700 1 -- 172.21.15.89:0/139629605 shutdown_connections 2023-10-10T20:40:05.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.393+0000 7fe149682700 1 -- 172.21.15.89:0/139629605 wait complete. 2023-10-10T20:40:05.440 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:40:05.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.553+0000 7fa72043b700 1 Processor -- start 2023-10-10T20:40:05.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.553+0000 7fa72043b700 1 -- start start 2023-10-10T20:40:05.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.554+0000 7fa72043b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa718159890 0x7fa71815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:05.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.554+0000 7fa72043b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa718159c60 0x7fa71815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:05.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.554+0000 7fa72043b700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71815a160 con 0x7fa71815db10 2023-10-10T20:40:05.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.554+0000 7fa72043b700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7180d13a0 con 0x7fa718159c60 2023-10-10T20:40:05.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.554+0000 7fa72043b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71815e140 con 0x7fa718159890 2023-10-10T20:40:05.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.554+0000 7fa71d9d6700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa718159c60 0x7fa71815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:05.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.554+0000 7fa71e1d7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa718159890 0x7fa71815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:05.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.554+0000 7fa71e1d7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa718159890 0x7fa71815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:56912/0 (socket says 172.21.15.89:56912) 2023-10-10T20:40:05.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.554+0000 7fa71e1d7700 1 -- 172.21.15.89:0/3497850347 learned_addr learned my addr 172.21.15.89:0/3497850347 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:05.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.554+0000 7fa71d9d6700 1 -- 172.21.15.89:0/3497850347 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa718159890 msgr2=0x7fa71815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:05.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.554+0000 7fa71d9d6700 1 --2- 172.21.15.89:0/3497850347 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa718159890 0x7fa71815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.554+0000 7fa71d9d6700 1 -- 172.21.15.89:0/3497850347 >> v1:172.21.15.111:6789/0 conn(0x7fa71815db10 legacy=0x7fa718069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:05.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.554+0000 7fa71d9d6700 1 -- 172.21.15.89:0/3497850347 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71815a3d0 con 0x7fa718159c60 2023-10-10T20:40:05.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.554+0000 7fa71e1d7700 1 --2- 172.21.15.89:0/3497850347 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa718159890 0x7fa71815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:40:05.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.555+0000 7fa71d9d6700 1 --2- 172.21.15.89:0/3497850347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa718159c60 0x7fa71815d5d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa7080098e0 tx=0x7fa7080091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb4eb2706a12f9ef server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:05.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.555+0000 7fa71d1d5700 1 -- 172.21.15.89:0/3497850347 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70800b640 con 0x7fa718159c60 2023-10-10T20:40:05.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.555+0000 7fa71d1d5700 1 -- 172.21.15.89:0/3497850347 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70800b7a0 con 0x7fa718159c60 2023-10-10T20:40:05.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.555+0000 7fa71d1d5700 1 -- 172.21.15.89:0/3497850347 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70800bb40 con 0x7fa718159c60 2023-10-10T20:40:05.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.555+0000 7fa72043b700 1 -- 172.21.15.89:0/3497850347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa718159c60 msgr2=0x7fa71815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:05.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.555+0000 7fa72043b700 1 --2- 172.21.15.89:0/3497850347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa718159c60 0x7fa71815d5d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa7080098e0 tx=0x7fa7080091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.555+0000 7fa72043b700 1 -- 172.21.15.89:0/3497850347 shutdown_connections 2023-10-10T20:40:05.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.555+0000 7fa72043b700 1 --2- 172.21.15.89:0/3497850347 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa718159890 0x7fa71815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.555+0000 7fa72043b700 1 --2- 172.21.15.89:0/3497850347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa718159c60 0x7fa71815d5d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.555+0000 7fa72043b700 1 -- 172.21.15.89:0/3497850347 >> 172.21.15.89:0/3497850347 conn(0x7fa718068e20 msgr2=0x7fa7180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:05.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.555+0000 7fa72043b700 1 -- 172.21.15.89:0/3497850347 shutdown_connections 2023-10-10T20:40:05.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.555+0000 7fa72043b700 1 -- 172.21.15.89:0/3497850347 wait complete. 2023-10-10T20:40:05.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.557+0000 7fa72043b700 1 Processor -- start 2023-10-10T20:40:05.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.557+0000 7fa72043b700 1 -- start start 2023-10-10T20:40:05.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.557+0000 7fa72043b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa718159890 0x7fa7181527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:05.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.557+0000 7fa72043b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa718159c60 0x7fa718152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:05.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.557+0000 7fa72043b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa71815db10 0x7fa7180cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:05.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.557+0000 7fa72043b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180ce830 con 0x7fa718159c60 2023-10-10T20:40:05.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.557+0000 7fa72043b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180b3590 con 0x7fa718159890 2023-10-10T20:40:05.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.557+0000 7fa72043b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180ce270 con 0x7fa71815db10 2023-10-10T20:40:05.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.557+0000 7fa71d9d6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa718159c60 0x7fa718152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:05.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.558+0000 7fa71d9d6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa718159c60 0x7fa718152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54290/0 (socket says 172.21.15.89:54290) 2023-10-10T20:40:05.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.558+0000 7fa71d9d6700 1 -- 172.21.15.89:0/1704342175 learned_addr learned my addr 172.21.15.89:0/1704342175 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:05.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.558+0000 7fa71e1d7700 1 --2- 172.21.15.89:0/1704342175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa718159890 0x7fa7181527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:05.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.558+0000 7fa71e9d8700 1 --2- 172.21.15.89:0/1704342175 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa71815db10 0x7fa7180cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:05.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.558+0000 7fa71d9d6700 1 -- 172.21.15.89:0/1704342175 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa71815db10 msgr2=0x7fa7180cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:05.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.558+0000 7fa71d9d6700 1 --2- 172.21.15.89:0/1704342175 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa71815db10 0x7fa7180cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.558+0000 7fa71d9d6700 1 -- 172.21.15.89:0/1704342175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa718159890 msgr2=0x7fa7181527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:05.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.558+0000 7fa71d9d6700 1 --2- 172.21.15.89:0/1704342175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa718159890 0x7fa7181527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.558+0000 7fa71d9d6700 1 -- 172.21.15.89:0/1704342175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7140094f0 con 0x7fa718159c60 2023-10-10T20:40:05.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.558+0000 7fa71e9d8700 1 --2- 172.21.15.89:0/1704342175 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa71815db10 0x7fa7180cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:05.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.558+0000 7fa71d9d6700 1 --2- 172.21.15.89:0/1704342175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa718159c60 0x7fa718152cf0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa708013000 tx=0x7fa708017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:05.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.558+0000 7fa70e7fc700 1 -- 172.21.15.89:0/1704342175 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70801bda0 con 0x7fa718159c60 2023-10-10T20:40:05.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.558+0000 7fa70e7fc700 1 -- 172.21.15.89:0/1704342175 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa708011420 con 0x7fa718159c60 2023-10-10T20:40:05.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.558+0000 7fa70e7fc700 1 -- 172.21.15.89:0/1704342175 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7080117e0 con 0x7fa718159c60 2023-10-10T20:40:05.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.558+0000 7fa72043b700 1 -- 172.21.15.89:0/1704342175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa708009420 con 0x7fa718159c60 2023-10-10T20:40:05.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.558+0000 7fa72043b700 1 -- 172.21.15.89:0/1704342175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7180b0e80 con 0x7fa718159c60 2023-10-10T20:40:05.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.559+0000 7fa72043b700 1 -- 172.21.15.89:0/1704342175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7180b0ce0 con 0x7fa718159c60 2023-10-10T20:40:05.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.559+0000 7fa70e7fc700 1 -- 172.21.15.89:0/1704342175 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fa708011b50 con 0x7fa718159c60 2023-10-10T20:40:05.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.560+0000 7fa70e7fc700 1 --2- 172.21.15.89:0/1704342175 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa6fc067200 0x7fa6fc0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:05.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.560+0000 7fa71e1d7700 1 --2- 172.21.15.89:0/1704342175 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa6fc067200 0x7fa6fc0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:05.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.560+0000 7fa70e7fc700 1 -- 172.21.15.89:0/1704342175 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa708085da0 con 0x7fa718159c60 2023-10-10T20:40:05.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.560+0000 7fa71e1d7700 1 --2- 172.21.15.89:0/1704342175 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa6fc067200 0x7fa6fc0696b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa714006fd0 tx=0x7fa714008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:05.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.564+0000 7fa70e7fc700 1 -- 172.21.15.89:0/1704342175 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7180b0ce0 con 0x7fa718159c60 2023-10-10T20:40:05.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.748+0000 7fa72043b700 1 -- 172.21.15.89:0/1704342175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7180b0ce0 con 0x7fa718159c60 2023-10-10T20:40:05.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.749+0000 7fa70e7fc700 1 -- 172.21.15.89:0/1704342175 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+12135 (secure 0 0 0) 0x7fa7180b0ce0 con 0x7fa718159c60 2023-10-10T20:40:05.752 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:05.752 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":84,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:40:04.987972+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:40:05.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.751+0000 7fa72043b700 1 -- 172.21.15.89:0/1704342175 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa6fc067200 msgr2=0x7fa6fc0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:05.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.751+0000 7fa72043b700 1 --2- 172.21.15.89:0/1704342175 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa6fc067200 0x7fa6fc0696b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa714006fd0 tx=0x7fa714008040 comp rx=0 tx=0).stop 2023-10-10T20:40:05.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.751+0000 7fa72043b700 1 -- 172.21.15.89:0/1704342175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa718159c60 msgr2=0x7fa718152cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:05.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.751+0000 7fa72043b700 1 --2- 172.21.15.89:0/1704342175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa718159c60 0x7fa718152cf0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa708013000 tx=0x7fa708017000 comp rx=0 tx=0).stop 2023-10-10T20:40:05.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.751+0000 7fa72043b700 1 -- 172.21.15.89:0/1704342175 shutdown_connections 2023-10-10T20:40:05.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.751+0000 7fa72043b700 1 --2- 172.21.15.89:0/1704342175 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa71815db10 0x7fa7180cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.751+0000 7fa72043b700 1 --2- 172.21.15.89:0/1704342175 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa6fc067200 0x7fa6fc0696b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.751+0000 7fa72043b700 1 --2- 172.21.15.89:0/1704342175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa718159890 0x7fa7181527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.751+0000 7fa72043b700 1 --2- 172.21.15.89:0/1704342175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa718159c60 0x7fa718152cf0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.751+0000 7fa72043b700 1 -- 172.21.15.89:0/1704342175 >> 172.21.15.89:0/1704342175 conn(0x7fa718068e20 msgr2=0x7fa71814a390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:05.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.751+0000 7fa72043b700 1 -- 172.21.15.89:0/1704342175 shutdown_connections 2023-10-10T20:40:05.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.751+0000 7fa72043b700 1 -- 172.21.15.89:0/1704342175 wait complete. 2023-10-10T20:40:05.765 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:40:05.766 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:40:05.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.881+0000 7f7a074e3700 1 Processor -- start 2023-10-10T20:40:05.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.881+0000 7f7a074e3700 1 -- start start 2023-10-10T20:40:05.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.882+0000 7f7a074e3700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7a00141500 0x7f7a000c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:05.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.882+0000 7f7a074e3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a0015c2a0 con 0x7f7a0006bcd0 2023-10-10T20:40:05.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.882+0000 7f7a074e3700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a0015b690 con 0x7f7a0006c0a0 2023-10-10T20:40:05.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.882+0000 7f7a074e3700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a0015b970 con 0x7f7a00141500 2023-10-10T20:40:05.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.882+0000 7f7a0527f700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7a0006bcd0 0x7f7a00140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59430/0 (socket says 172.21.15.89:59430) 2023-10-10T20:40:05.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.882+0000 7f7a0527f700 1 -- 172.21.15.89:0/2503011299 learned_addr learned my addr 172.21.15.89:0/2503011299 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:05.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.882+0000 7f7a04a7e700 1 --2- 172.21.15.89:0/2503011299 >> v2:172.21.15.111:3300/0 conn(0x7f7a00141500 0x7f7a000c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:05.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.883+0000 7f79f7fff700 1 -- 172.21.15.89:0/2503011299 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272553229 0 0) 0x7f7a0015c2a0 con 0x7f7a0006bcd0 2023-10-10T20:40:05.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.883+0000 7f79f7fff700 1 -- 172.21.15.89:0/2503011299 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79ec003570 con 0x7f7a0006bcd0 2023-10-10T20:40:05.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.883+0000 7f7a04a7e700 1 -- 172.21.15.89:0/2503011299 >> v1:172.21.15.111:6789/0 conn(0x7f7a0006c0a0 legacy=0x7f7a000c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:05.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.883+0000 7f79f7fff700 1 -- 172.21.15.89:0/2503011299 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1478782921 0 0) 0x7f79ec003570 con 0x7f7a0006bcd0 2023-10-10T20:40:05.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.883+0000 7f7a04a7e700 1 -- 172.21.15.89:0/2503011299 >> v1:172.21.15.89:6789/0 conn(0x7f7a0006bcd0 legacy=0x7f7a00140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:05.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.883+0000 7f7a04a7e700 1 -- 172.21.15.89:0/2503011299 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0015bc50 con 0x7f7a00141500 2023-10-10T20:40:05.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.883+0000 7f7a04a7e700 1 --2- 172.21.15.89:0/2503011299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a00141500 0x7f7a000c8e00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f79f00098e0 tx=0x7f79f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92ccb1662df4db9b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:05.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.883+0000 7f79f7fff700 1 -- 172.21.15.89:0/2503011299 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc001470 con 0x7f7a00141500 2023-10-10T20:40:05.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.883+0000 7f79f7fff700 1 -- 172.21.15.89:0/2503011299 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f0011420 con 0x7f7a00141500 2023-10-10T20:40:05.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.883+0000 7f79f7fff700 1 -- 172.21.15.89:0/2503011299 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f00117c0 con 0x7f7a00141500 2023-10-10T20:40:05.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.884+0000 7f7a074e3700 1 -- 172.21.15.89:0/2503011299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a00141500 msgr2=0x7f7a000c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:05.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.884+0000 7f7a074e3700 1 --2- 172.21.15.89:0/2503011299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a00141500 0x7f7a000c8e00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f79f00098e0 tx=0x7f79f00091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.884+0000 7f7a074e3700 1 -- 172.21.15.89:0/2503011299 shutdown_connections 2023-10-10T20:40:05.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.884+0000 7f7a074e3700 1 --2- 172.21.15.89:0/2503011299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a00141500 0x7f7a000c8e00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.884+0000 7f7a074e3700 1 -- 172.21.15.89:0/2503011299 >> 172.21.15.89:0/2503011299 conn(0x7f7a000ce600 msgr2=0x7f7a000cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:05.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.884+0000 7f7a074e3700 1 -- 172.21.15.89:0/2503011299 shutdown_connections 2023-10-10T20:40:05.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.884+0000 7f7a074e3700 1 -- 172.21.15.89:0/2503011299 wait complete. 2023-10-10T20:40:05.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.885+0000 7f7a074e3700 1 Processor -- start 2023-10-10T20:40:05.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.885+0000 7f7a074e3700 1 -- start start 2023-10-10T20:40:05.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a074e3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a0006bcd0 0x7f7a0013b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:05.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a074e3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a0006c0a0 0x7f7a000c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:05.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a074e3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a00141500 0x7f7a0006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:05.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a074e3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a000b16d0 con 0x7f7a0006c0a0 2023-10-10T20:40:05.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a074e3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a000b19b0 con 0x7f7a00141500 2023-10-10T20:40:05.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a074e3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a000b1c90 con 0x7f7a0006bcd0 2023-10-10T20:40:05.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a04a7e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a0006c0a0 0x7f7a000c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:05.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a04a7e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a0006c0a0 0x7f7a000c5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54296/0 (socket says 172.21.15.89:54296) 2023-10-10T20:40:05.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a04a7e700 1 -- 172.21.15.89:0/3280266263 learned_addr learned my addr 172.21.15.89:0/3280266263 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:05.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a05a80700 1 --2- 172.21.15.89:0/3280266263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a00141500 0x7f7a0006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:05.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a0527f700 1 --2- 172.21.15.89:0/3280266263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a0006bcd0 0x7f7a0013b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:05.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a04a7e700 1 -- 172.21.15.89:0/3280266263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a0006bcd0 msgr2=0x7f7a0013b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:05.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a04a7e700 1 --2- 172.21.15.89:0/3280266263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a0006bcd0 0x7f7a0013b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a04a7e700 1 -- 172.21.15.89:0/3280266263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a00141500 msgr2=0x7f7a0006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:05.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a04a7e700 1 --2- 172.21.15.89:0/3280266263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a00141500 0x7f7a0006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a04a7e700 1 -- 172.21.15.89:0/3280266263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f0009420 con 0x7f7a0006c0a0 2023-10-10T20:40:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a0527f700 1 --2- 172.21.15.89:0/3280266263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a0006bcd0 0x7f7a0013b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a04a7e700 1 --2- 172.21.15.89:0/3280266263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a0006c0a0 0x7f7a000c5990 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f79f00091e0 tx=0x7f79f0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f79f57fa700 1 -- 172.21.15.89:0/3280266263 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f001cbe0 con 0x7f7a0006c0a0 2023-10-10T20:40:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f79f57fa700 1 -- 172.21.15.89:0/3280266263 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f001cd40 con 0x7f7a0006c0a0 2023-10-10T20:40:05.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a074e3700 1 -- 172.21.15.89:0/3280266263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a000b1f70 con 0x7f7a0006c0a0 2023-10-10T20:40:05.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f79f57fa700 1 -- 172.21.15.89:0/3280266263 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f001a6a0 con 0x7f7a0006c0a0 2023-10-10T20:40:05.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.886+0000 7f7a074e3700 1 -- 172.21.15.89:0/3280266263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a000b13a0 con 0x7f7a0006c0a0 2023-10-10T20:40:05.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.887+0000 7f7a074e3700 1 -- 172.21.15.89:0/3280266263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a0004c3c0 con 0x7f7a0006c0a0 2023-10-10T20:40:05.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.888+0000 7f79f57fa700 1 -- 172.21.15.89:0/3280266263 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f79f001aa90 con 0x7f7a0006c0a0 2023-10-10T20:40:05.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.888+0000 7f79f57fa700 1 --2- 172.21.15.89:0/3280266263 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f79e4067200 0x7f79e40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:05.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.888+0000 7f79f57fa700 1 -- 172.21.15.89:0/3280266263 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f79f0086000 con 0x7f7a0006c0a0 2023-10-10T20:40:05.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.888+0000 7f7a0527f700 1 --2- 172.21.15.89:0/3280266263 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f79e4067200 0x7f79e40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:05.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.889+0000 7f7a0527f700 1 --2- 172.21.15.89:0/3280266263 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f79e4067200 0x7f79e40696b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f79fc001ed0 tx=0x7f79fc000bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:05.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:05.892+0000 7f79f57fa700 1 -- 172.21.15.89:0/3280266263 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79f00b75a0 con 0x7f7a0006c0a0 2023-10-10T20:40:06.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.075+0000 7f7a074e3700 1 -- 172.21.15.89:0/3280266263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7a0015bf30 con 0x7f7a0006c0a0 2023-10-10T20:40:06.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.076+0000 7f79f57fa700 1 -- 172.21.15.89:0/3280266263 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79f0029020 con 0x7f7a0006c0a0 2023-10-10T20:40:06.078 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:40:06.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.078+0000 7f7a074e3700 1 -- 172.21.15.89:0/3280266263 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f79e4067200 msgr2=0x7f79e40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:06.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.078+0000 7f7a074e3700 1 --2- 172.21.15.89:0/3280266263 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f79e4067200 0x7f79e40696b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f79fc001ed0 tx=0x7f79fc000bc0 comp rx=0 tx=0).stop 2023-10-10T20:40:06.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.078+0000 7f7a074e3700 1 -- 172.21.15.89:0/3280266263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a0006c0a0 msgr2=0x7f7a000c5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:06.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.078+0000 7f7a074e3700 1 --2- 172.21.15.89:0/3280266263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a0006c0a0 0x7f7a000c5990 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f79f00091e0 tx=0x7f79f0016000 comp rx=0 tx=0).stop 2023-10-10T20:40:06.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.078+0000 7f7a074e3700 1 -- 172.21.15.89:0/3280266263 shutdown_connections 2023-10-10T20:40:06.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.078+0000 7f7a074e3700 1 --2- 172.21.15.89:0/3280266263 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f79e4067200 0x7f79e40696b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:06.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.078+0000 7f7a074e3700 1 --2- 172.21.15.89:0/3280266263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a00141500 0x7f7a0006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:06.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.078+0000 7f7a074e3700 1 --2- 172.21.15.89:0/3280266263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a0006bcd0 0x7f7a0013b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:06.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.078+0000 7f7a074e3700 1 --2- 172.21.15.89:0/3280266263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a0006c0a0 0x7f7a000c5990 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:06.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.078+0000 7f7a074e3700 1 -- 172.21.15.89:0/3280266263 >> 172.21.15.89:0/3280266263 conn(0x7f7a000ce600 msgr2=0x7f7a000c3780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:06.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.078+0000 7f7a074e3700 1 -- 172.21.15.89:0/3280266263 shutdown_connections 2023-10-10T20:40:06.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.078+0000 7f7a074e3700 1 -- 172.21.15.89:0/3280266263 wait complete. 2023-10-10T20:40:06.092 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:40:06.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.204+0000 7f03fe588700 1 Processor -- start 2023-10-10T20:40:06.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.204+0000 7f03fe588700 1 -- start start 2023-10-10T20:40:06.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.205+0000 7f03fe588700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f03f8159c60 0x7f03f815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:06.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.205+0000 7f03fe588700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f03f815db10 0x7f03f8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:06.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.205+0000 7f03fe588700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f815a160 con 0x7f03f8159890 2023-10-10T20:40:06.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.205+0000 7f03fe588700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03f815df50 con 0x7f03f8159c60 2023-10-10T20:40:06.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.205+0000 7f03fe588700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03f80d13a0 con 0x7f03f815db10 2023-10-10T20:40:06.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.205+0000 7f03f77fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f03f8159c60 0x7f03f815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:06.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.205+0000 7f03f77fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f03f8159c60 0x7f03f815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54310/0 (socket says 172.21.15.89:54310) 2023-10-10T20:40:06.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.205+0000 7f03f77fe700 1 -- 172.21.15.89:0/3148667404 learned_addr learned my addr 172.21.15.89:0/3148667404 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:06.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.205+0000 7f03fcb25700 1 --2- 172.21.15.89:0/3148667404 >> v2:172.21.15.111:3300/0 conn(0x7f03f815db10 0x7f03f8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:06.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.206+0000 7f03f77fe700 1 -- 172.21.15.89:0/3148667404 >> v2:172.21.15.111:3300/0 conn(0x7f03f815db10 msgr2=0x7f03f8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:06.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.206+0000 7f03f77fe700 1 --2- 172.21.15.89:0/3148667404 >> v2:172.21.15.111:3300/0 conn(0x7f03f815db10 0x7f03f8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:06.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.206+0000 7f03f77fe700 1 -- 172.21.15.89:0/3148667404 >> v1:172.21.15.111:6789/0 conn(0x7f03f8159890 legacy=0x7f03f815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:06.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.206+0000 7f03f77fe700 1 -- 172.21.15.89:0/3148667404 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f815e140 con 0x7f03f8159c60 2023-10-10T20:40:06.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.206+0000 7f03fcb25700 1 --2- 172.21.15.89:0/3148667404 >> v2:172.21.15.111:3300/0 conn(0x7f03f815db10 0x7f03f8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:06.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.206+0000 7f03f77fe700 1 --2- 172.21.15.89:0/3148667404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03f8159c60 0x7f03f815d5d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f03f806a890 tx=0x7f03ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd2947556d8d2bf server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:06.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.206+0000 7f03f6ffd700 1 -- 172.21.15.89:0/3148667404 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec00b860 con 0x7f03f8159c60 2023-10-10T20:40:06.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.206+0000 7f03f6ffd700 1 -- 172.21.15.89:0/3148667404 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ec00b9c0 con 0x7f03f8159c60 2023-10-10T20:40:06.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.206+0000 7f03f6ffd700 1 -- 172.21.15.89:0/3148667404 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec00bd60 con 0x7f03f8159c60 2023-10-10T20:40:06.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.206+0000 7f03fe588700 1 -- 172.21.15.89:0/3148667404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03f8159c60 msgr2=0x7f03f815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:06.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.206+0000 7f03fe588700 1 --2- 172.21.15.89:0/3148667404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03f8159c60 0x7f03f815d5d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f03f806a890 tx=0x7f03ec009250 comp rx=0 tx=0).stop 2023-10-10T20:40:06.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.206+0000 7f03fe588700 1 -- 172.21.15.89:0/3148667404 shutdown_connections 2023-10-10T20:40:06.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.206+0000 7f03fe588700 1 --2- 172.21.15.89:0/3148667404 >> v2:172.21.15.111:3300/0 conn(0x7f03f815db10 0x7f03f8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:06.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.206+0000 7f03fe588700 1 --2- 172.21.15.89:0/3148667404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03f8159c60 0x7f03f815d5d0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:06.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.206+0000 7f03fe588700 1 -- 172.21.15.89:0/3148667404 >> 172.21.15.89:0/3148667404 conn(0x7f03f8068e20 msgr2=0x7f03f80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:06.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.206+0000 7f03fe588700 1 -- 172.21.15.89:0/3148667404 shutdown_connections 2023-10-10T20:40:06.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.207+0000 7f03fe588700 1 -- 172.21.15.89:0/3148667404 wait complete. 2023-10-10T20:40:06.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.208+0000 7f03fe588700 1 Processor -- start 2023-10-10T20:40:06.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.208+0000 7f03fe588700 1 -- start start 2023-10-10T20:40:06.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03fe588700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03f8159890 0x7f03f814caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:06.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03fe588700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f03f8159c60 0x7f03f80cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:06.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03fe588700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03f815db10 0x7f03f815caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:06.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03fe588700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f816bde0 con 0x7f03f8159890 2023-10-10T20:40:06.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03fe588700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f816b1d0 con 0x7f03f815db10 2023-10-10T20:40:06.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03fe588700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f816b4b0 con 0x7f03f8159c60 2023-10-10T20:40:06.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03f7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03f8159890 0x7f03f814caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:06.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03f7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03f8159890 0x7f03f814caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54320/0 (socket says 172.21.15.89:54320) 2023-10-10T20:40:06.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03f7fff700 1 -- 172.21.15.89:0/3663487593 learned_addr learned my addr 172.21.15.89:0/3663487593 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:06.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03fcb25700 1 --2- 172.21.15.89:0/3663487593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03f815db10 0x7f03f815caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:06.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03f77fe700 1 --2- 172.21.15.89:0/3663487593 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f03f8159c60 0x7f03f80cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:06.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03f7fff700 1 -- 172.21.15.89:0/3663487593 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f03f8159c60 msgr2=0x7f03f80cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:06.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03f7fff700 1 --2- 172.21.15.89:0/3663487593 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f03f8159c60 0x7f03f80cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:06.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03f7fff700 1 -- 172.21.15.89:0/3663487593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03f815db10 msgr2=0x7f03f815caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:06.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03f7fff700 1 --2- 172.21.15.89:0/3663487593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03f815db10 0x7f03f815caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:06.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03f7fff700 1 -- 172.21.15.89:0/3663487593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec0094f0 con 0x7f03f8159890 2023-10-10T20:40:06.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03fcb25700 1 --2- 172.21.15.89:0/3663487593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03f815db10 0x7f03f815caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:06.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03f7fff700 1 --2- 172.21.15.89:0/3663487593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03f8159890 0x7f03f814caf0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f03e800c590 tx=0x7f03e800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:06.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03dffff700 1 -- 172.21.15.89:0/3663487593 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e8011590 con 0x7f03f8159890 2023-10-10T20:40:06.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03dffff700 1 -- 172.21.15.89:0/3663487593 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e80116f0 con 0x7f03f8159890 2023-10-10T20:40:06.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03fe588700 1 -- 172.21.15.89:0/3663487593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f816b790 con 0x7f03f8159890 2023-10-10T20:40:06.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03dffff700 1 -- 172.21.15.89:0/3663487593 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e8011ab0 con 0x7f03f8159890 2023-10-10T20:40:06.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.209+0000 7f03fe588700 1 -- 172.21.15.89:0/3663487593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f80b3590 con 0x7f03f8159890 2023-10-10T20:40:06.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.210+0000 7f03fe588700 1 -- 172.21.15.89:0/3663487593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f804c3c0 con 0x7f03f8159890 2023-10-10T20:40:06.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.210+0000 7f03dffff700 1 -- 172.21.15.89:0/3663487593 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f03e801b3b0 con 0x7f03f8159890 2023-10-10T20:40:06.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.211+0000 7f03dffff700 1 --2- 172.21.15.89:0/3663487593 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f03e4067200 0x7f03e40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:06.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.211+0000 7f03dffff700 1 -- 172.21.15.89:0/3663487593 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f03e8083be0 con 0x7f03f8159890 2023-10-10T20:40:06.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.211+0000 7f03f77fe700 1 --2- 172.21.15.89:0/3663487593 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f03e4067200 0x7f03e40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:06.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.211+0000 7f03f77fe700 1 --2- 172.21.15.89:0/3663487593 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f03e4067200 0x7f03e40696b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f03f806a890 tx=0x7f03ec017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:06.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.215+0000 7f03dffff700 1 -- 172.21.15.89:0/3663487593 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03e8050e50 con 0x7f03f8159890 2023-10-10T20:40:06.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.399+0000 7f03fe588700 1 -- 172.21.15.89:0/3663487593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f03f80b1aa0 con 0x7f03f8159890 2023-10-10T20:40:06.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.399+0000 7f03dffff700 1 -- 172.21.15.89:0/3663487593 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v84) v1 ==== 84+0+65 (secure 0 0 0) 0x7f03e8017070 con 0x7f03f8159890 2023-10-10T20:40:06.402 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:06.402 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:40:06.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:06.403 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2023-10-10T20:40:06.403 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2023-10-10T20:40:06.403 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:06.403 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:40:06.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.401+0000 7f03fe588700 1 -- 172.21.15.89:0/3663487593 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f03e4067200 msgr2=0x7f03e40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:06.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.401+0000 7f03fe588700 1 --2- 172.21.15.89:0/3663487593 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f03e4067200 0x7f03e40696b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f03f806a890 tx=0x7f03ec017000 comp rx=0 tx=0).stop 2023-10-10T20:40:06.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.401+0000 7f03fe588700 1 -- 172.21.15.89:0/3663487593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03f8159890 msgr2=0x7f03f814caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:06.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.401+0000 7f03fe588700 1 --2- 172.21.15.89:0/3663487593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03f8159890 0x7f03f814caf0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f03e800c590 tx=0x7f03e800c950 comp rx=0 tx=0).stop 2023-10-10T20:40:06.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.401+0000 7f03fe588700 1 -- 172.21.15.89:0/3663487593 shutdown_connections 2023-10-10T20:40:06.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.401+0000 7f03fe588700 1 --2- 172.21.15.89:0/3663487593 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f03e4067200 0x7f03e40696b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:06.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.401+0000 7f03fe588700 1 --2- 172.21.15.89:0/3663487593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03f815db10 0x7f03f815caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:06.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.401+0000 7f03fe588700 1 --2- 172.21.15.89:0/3663487593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03f8159890 0x7f03f814caf0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:06.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.401+0000 7f03fe588700 1 --2- 172.21.15.89:0/3663487593 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f03f8159c60 0x7f03f80cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:06.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.401+0000 7f03fe588700 1 -- 172.21.15.89:0/3663487593 >> 172.21.15.89:0/3663487593 conn(0x7f03f8068e20 msgr2=0x7f03f80c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:06.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.401+0000 7f03fe588700 1 -- 172.21.15.89:0/3663487593 shutdown_connections 2023-10-10T20:40:06.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.401+0000 7f03fe588700 1 -- 172.21.15.89:0/3663487593 wait complete. 2023-10-10T20:40:06.415 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:40:06.416 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:40:06.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.532+0000 7fe4c8c95700 1 Processor -- start 2023-10-10T20:40:06.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.532+0000 7fe4c8c95700 1 -- start start 2023-10-10T20:40:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.533+0000 7fe4c8c95700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe4c40b03c0 0x7fe4c40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.533+0000 7fe4c8c95700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4c40b0790 0x7fe4c40b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.533+0000 7fe4c8c95700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c40cfa00 con 0x7fe4c41591e0 2023-10-10T20:40:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.533+0000 7fe4c8c95700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c40cb580 con 0x7fe4c40b03c0 2023-10-10T20:40:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.533+0000 7fe4c8c95700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c40b2de0 con 0x7fe4c40b0790 2023-10-10T20:40:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.533+0000 7fe4c37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe4c40b03c0 0x7fe4c40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.533+0000 7fe4c37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe4c40b03c0 0x7fe4c40cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54328/0 (socket says 172.21.15.89:54328) 2023-10-10T20:40:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.533+0000 7fe4c37fe700 1 -- 172.21.15.89:0/2182727405 learned_addr learned my addr 172.21.15.89:0/2182727405 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.533+0000 7fe4c2ffd700 1 --2- 172.21.15.89:0/2182727405 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4c40b0790 0x7fe4c40b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.533+0000 7fe4c37fe700 1 -- 172.21.15.89:0/2182727405 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4c40b0790 msgr2=0x7fe4c40b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.533+0000 7fe4c37fe700 1 --2- 172.21.15.89:0/2182727405 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4c40b0790 0x7fe4c40b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.533+0000 7fe4c37fe700 1 -- 172.21.15.89:0/2182727405 >> v1:172.21.15.111:6789/0 conn(0x7fe4c41591e0 legacy=0x7fe4c4159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.533+0000 7fe4c37fe700 1 -- 172.21.15.89:0/2182727405 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c40cf160 con 0x7fe4c40b03c0 2023-10-10T20:40:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.533+0000 7fe4c37fe700 1 --2- 172.21.15.89:0/2182727405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4c40b03c0 0x7fe4c40cea70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4009f20 tx=0x7fe4b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5be761aff9cf1c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.534+0000 7fe4c27fc700 1 -- 172.21.15.89:0/2182727405 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b400e070 con 0x7fe4c40b03c0 2023-10-10T20:40:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.534+0000 7fe4c27fc700 1 -- 172.21.15.89:0/2182727405 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b400ba80 con 0x7fe4c40b03c0 2023-10-10T20:40:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.534+0000 7fe4c27fc700 1 -- 172.21.15.89:0/2182727405 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b40092e0 con 0x7fe4c40b03c0 2023-10-10T20:40:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.534+0000 7fe4c8c95700 1 -- 172.21.15.89:0/2182727405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4c40b03c0 msgr2=0x7fe4c40cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.534+0000 7fe4c8c95700 1 --2- 172.21.15.89:0/2182727405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4c40b03c0 0x7fe4c40cea70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4009f20 tx=0x7fe4b4009840 comp rx=0 tx=0).stop 2023-10-10T20:40:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.534+0000 7fe4c8c95700 1 -- 172.21.15.89:0/2182727405 shutdown_connections 2023-10-10T20:40:06.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.534+0000 7fe4c8c95700 1 --2- 172.21.15.89:0/2182727405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4c40b03c0 0x7fe4c40cea70 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:06.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.534+0000 7fe4c8c95700 1 --2- 172.21.15.89:0/2182727405 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4c40b0790 0x7fe4c40b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:06.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.534+0000 7fe4c8c95700 1 -- 172.21.15.89:0/2182727405 >> 172.21.15.89:0/2182727405 conn(0x7fe4c4063230 msgr2=0x7fe4c4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:06.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.534+0000 7fe4c8c95700 1 -- 172.21.15.89:0/2182727405 shutdown_connections 2023-10-10T20:40:06.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.534+0000 7fe4c8c95700 1 -- 172.21.15.89:0/2182727405 wait complete. 2023-10-10T20:40:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.535+0000 7fe4c8c95700 1 Processor -- start 2023-10-10T20:40:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.536+0000 7fe4c8c95700 1 -- start start 2023-10-10T20:40:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.536+0000 7fe4c8c95700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4c40b03c0 0x7fe4c40ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.536+0000 7fe4c8c95700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c40b0790 0x7fe4c40b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.536+0000 7fe4c8c95700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4c41591e0 0x7fe4c40b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.536+0000 7fe4c8c95700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c415d340 con 0x7fe4c40b03c0 2023-10-10T20:40:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.536+0000 7fe4c8c95700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c40c7d60 con 0x7fe4c40b0790 2023-10-10T20:40:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.536+0000 7fe4c8c95700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c40c8040 con 0x7fe4c41591e0 2023-10-10T20:40:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.536+0000 7fe4c37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4c40b03c0 0x7fe4c40ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.536+0000 7fe4c37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4c40b03c0 0x7fe4c40ca230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54340/0 (socket says 172.21.15.89:54340) 2023-10-10T20:40:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.536+0000 7fe4c37fe700 1 -- 172.21.15.89:0/1305118448 learned_addr learned my addr 172.21.15.89:0/1305118448 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.537+0000 7fe4c37fe700 1 -- 172.21.15.89:0/1305118448 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4c41591e0 msgr2=0x7fe4c40b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.537+0000 7fe4c37fe700 1 --2- 172.21.15.89:0/1305118448 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4c41591e0 0x7fe4c40b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.537+0000 7fe4c37fe700 1 -- 172.21.15.89:0/1305118448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c40b0790 msgr2=0x7fe4c40b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.537+0000 7fe4c37fe700 1 --2- 172.21.15.89:0/1305118448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c40b0790 0x7fe4c40b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.537+0000 7fe4c37fe700 1 -- 172.21.15.89:0/1305118448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b4009ae0 con 0x7fe4c40b03c0 2023-10-10T20:40:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.537+0000 7fe4c37fe700 1 --2- 172.21.15.89:0/1305118448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4c40b03c0 0x7fe4c40ca230 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400b6c0 tx=0x7fe4b4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.537+0000 7fe4ab7fe700 1 -- 172.21.15.89:0/1305118448 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4008ae0 con 0x7fe4c40b03c0 2023-10-10T20:40:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.537+0000 7fe4ab7fe700 1 -- 172.21.15.89:0/1305118448 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b4008c60 con 0x7fe4c40b03c0 2023-10-10T20:40:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.537+0000 7fe4ab7fe700 1 -- 172.21.15.89:0/1305118448 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4025460 con 0x7fe4c40b03c0 2023-10-10T20:40:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.537+0000 7fe4c8c95700 1 -- 172.21.15.89:0/1305118448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c40c8320 con 0x7fe4c40b03c0 2023-10-10T20:40:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.537+0000 7fe4c8c95700 1 -- 172.21.15.89:0/1305118448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c40ca9b0 con 0x7fe4c40b03c0 2023-10-10T20:40:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.538+0000 7fe4c8c95700 1 -- 172.21.15.89:0/1305118448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c404c3c0 con 0x7fe4c40b03c0 2023-10-10T20:40:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.538+0000 7fe4ab7fe700 1 -- 172.21.15.89:0/1305118448 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fe4b4003040 con 0x7fe4c40b03c0 2023-10-10T20:40:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.539+0000 7fe4ab7fe700 1 --2- 172.21.15.89:0/1305118448 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe4b0067200 0x7fe4b00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.539+0000 7fe4ab7fe700 1 -- 172.21.15.89:0/1305118448 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe4b4086310 con 0x7fe4c40b03c0 2023-10-10T20:40:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.539+0000 7fe4c2ffd700 1 --2- 172.21.15.89:0/1305118448 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe4b0067200 0x7fe4b00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.539+0000 7fe4c2ffd700 1 --2- 172.21.15.89:0/1305118448 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe4b0067200 0x7fe4b00696b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009a60 tx=0x7fe4b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.543+0000 7fe4ab7fe700 1 -- 172.21.15.89:0/1305118448 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4b4053390 con 0x7fe4c40b03c0 2023-10-10T20:40:06.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.729+0000 7fe4c8c95700 1 -- 172.21.15.89:0/1305118448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe4c40cad50 con 0x7fe4c40b03c0 2023-10-10T20:40:06.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.995+0000 7fe4ab7fe700 1 -- 172.21.15.89:0/1305118448 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v85) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe4b40569b0 con 0x7fe4c40b03c0 2023-10-10T20:40:07.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.998+0000 7fe4c8c95700 1 -- 172.21.15.89:0/1305118448 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe4b0067200 msgr2=0x7fe4b00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:07.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.998+0000 7fe4c8c95700 1 --2- 172.21.15.89:0/1305118448 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe4b0067200 0x7fe4b00696b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009a60 tx=0x7fe4b8008040 comp rx=0 tx=0).stop 2023-10-10T20:40:07.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.998+0000 7fe4c8c95700 1 -- 172.21.15.89:0/1305118448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4c40b03c0 msgr2=0x7fe4c40ca230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:07.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.998+0000 7fe4c8c95700 1 --2- 172.21.15.89:0/1305118448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4c40b03c0 0x7fe4c40ca230 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400b6c0 tx=0x7fe4b4018000 comp rx=0 tx=0).stop 2023-10-10T20:40:07.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.998+0000 7fe4c8c95700 1 -- 172.21.15.89:0/1305118448 shutdown_connections 2023-10-10T20:40:07.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.998+0000 7fe4c8c95700 1 --2- 172.21.15.89:0/1305118448 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4c41591e0 0x7fe4c40b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:07.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.998+0000 7fe4c8c95700 1 --2- 172.21.15.89:0/1305118448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4c40b03c0 0x7fe4c40ca230 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:07.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.998+0000 7fe4c8c95700 1 --2- 172.21.15.89:0/1305118448 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe4b0067200 0x7fe4b00696b0 secure :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009a60 tx=0x7fe4b8008040 comp rx=0 tx=0).stop 2023-10-10T20:40:07.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.998+0000 7fe4c8c95700 1 --2- 172.21.15.89:0/1305118448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c40b0790 0x7fe4c40b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:07.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.999+0000 7fe4c8c95700 1 -- 172.21.15.89:0/1305118448 >> 172.21.15.89:0/1305118448 conn(0x7fe4c4063230 msgr2=0x7fe4c4159620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:07.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:06.999+0000 7fe4c8c95700 1 -- 172.21.15.89:0/1305118448 shutdown_connections 2023-10-10T20:40:07.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.000+0000 7fe4c8c95700 1 -- 172.21.15.89:0/1305118448 wait complete. 2023-10-10T20:40:07.004 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2023-10-10T20:40:07.016 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:40:07.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.136+0000 7f55355de700 1 Processor -- start 2023-10-10T20:40:07.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.136+0000 7f55355de700 1 -- start start 2023-10-10T20:40:07.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.136+0000 7f55355de700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f55300caab0 0x7f553006ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:07.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.136+0000 7f55355de700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5530157c50 con 0x7f55300cca50 2023-10-10T20:40:07.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.136+0000 7f55355de700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f553015d7d0 con 0x7f55300cce20 2023-10-10T20:40:07.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.136+0000 7f55355de700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f553015aa20 con 0x7f55300caab0 2023-10-10T20:40:07.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.137+0000 7f552f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f55300caab0 0x7f553006ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:07.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.137+0000 7f552effd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f55300cca50 0x7f553015a1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59444/0 (socket says 172.21.15.89:59444) 2023-10-10T20:40:07.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.137+0000 7f552effd700 1 -- 172.21.15.89:0/1807749652 learned_addr learned my addr 172.21.15.89:0/1807749652 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:07.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.137+0000 7f552f7fe700 1 -- 172.21.15.89:0/1807749652 >> v1:172.21.15.111:6789/0 conn(0x7f55300cce20 legacy=0x7f553015df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:07.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.137+0000 7f552f7fe700 1 -- 172.21.15.89:0/1807749652 >> v1:172.21.15.89:6789/0 conn(0x7f55300cca50 legacy=0x7f553015a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:07.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.137+0000 7f552f7fe700 1 -- 172.21.15.89:0/1807749652 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553015ad00 con 0x7f55300caab0 2023-10-10T20:40:07.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.137+0000 7f552e7fc700 1 -- 172.21.15.89:0/1807749652 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025079014 0 0) 0x7f5520001490 con 0x7f55300cca50 2023-10-10T20:40:07.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.137+0000 7f552f7fe700 1 --2- 172.21.15.89:0/1807749652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55300caab0 0x7f553006ba80 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f55280098e0 tx=0x7f55280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd87f113d9e8821 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:07.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.137+0000 7f552e7fc700 1 -- 172.21.15.89:0/1807749652 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528009e80 con 0x7f55300caab0 2023-10-10T20:40:07.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.137+0000 7f552e7fc700 1 -- 172.21.15.89:0/1807749652 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552800b820 con 0x7f55300caab0 2023-10-10T20:40:07.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.137+0000 7f552e7fc700 1 -- 172.21.15.89:0/1807749652 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552800bba0 con 0x7f55300caab0 2023-10-10T20:40:07.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.138+0000 7f55355de700 1 -- 172.21.15.89:0/1807749652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55300caab0 msgr2=0x7f553006ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:07.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.138+0000 7f55355de700 1 --2- 172.21.15.89:0/1807749652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55300caab0 0x7f553006ba80 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f55280098e0 tx=0x7f55280091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.138+0000 7f55355de700 1 -- 172.21.15.89:0/1807749652 shutdown_connections 2023-10-10T20:40:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.138+0000 7f55355de700 1 --2- 172.21.15.89:0/1807749652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55300caab0 0x7f553006ba80 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.138+0000 7f55355de700 1 -- 172.21.15.89:0/1807749652 >> 172.21.15.89:0/1807749652 conn(0x7f55300ce5c0 msgr2=0x7f55300ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.138+0000 7f55355de700 1 -- 172.21.15.89:0/1807749652 shutdown_connections 2023-10-10T20:40:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.138+0000 7f55355de700 1 -- 172.21.15.89:0/1807749652 wait complete. 2023-10-10T20:40:07.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.139+0000 7f55355de700 1 Processor -- start 2023-10-10T20:40:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.139+0000 7f55355de700 1 -- start start 2023-10-10T20:40:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f55355de700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f55300caab0 0x7f5530167450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f55355de700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55300cca50 0x7f55300c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f55355de700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55300cce20 0x7f55300c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f55355de700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5530167990 con 0x7f55300cca50 2023-10-10T20:40:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f55355de700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55300b31b0 con 0x7f55300cce20 2023-10-10T20:40:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f55355de700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55300b3490 con 0x7f55300caab0 2023-10-10T20:40:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f55267fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55300cca50 0x7f55300c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f55267fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55300cca50 0x7f55300c7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54366/0 (socket says 172.21.15.89:54366) 2023-10-10T20:40:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f55267fc700 1 -- 172.21.15.89:0/2057680055 learned_addr learned my addr 172.21.15.89:0/2057680055 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:07.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f552effd700 1 --2- 172.21.15.89:0/2057680055 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f55300caab0 0x7f5530167450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:07.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f552f7fe700 1 --2- 172.21.15.89:0/2057680055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55300cce20 0x7f55300c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:07.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f55267fc700 1 -- 172.21.15.89:0/2057680055 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f55300caab0 msgr2=0x7f5530167450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:07.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f55267fc700 1 --2- 172.21.15.89:0/2057680055 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f55300caab0 0x7f5530167450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:07.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f55267fc700 1 -- 172.21.15.89:0/2057680055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55300cce20 msgr2=0x7f55300c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:07.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f55267fc700 1 --2- 172.21.15.89:0/2057680055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55300cce20 0x7f55300c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:07.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f55267fc700 1 -- 172.21.15.89:0/2057680055 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5528009420 con 0x7f55300cca50 2023-10-10T20:40:07.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f552f7fe700 1 --2- 172.21.15.89:0/2057680055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55300cce20 0x7f55300c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:07.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f55267fc700 1 --2- 172.21.15.89:0/2057680055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55300cca50 0x7f55300c7e20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f551800b5b0 tx=0x7f55180090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:07.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f55277fe700 1 -- 172.21.15.89:0/2057680055 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551800f510 con 0x7f55300cca50 2023-10-10T20:40:07.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.140+0000 7f55277fe700 1 -- 172.21.15.89:0/2057680055 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551800f670 con 0x7f55300cca50 2023-10-10T20:40:07.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.141+0000 7f55355de700 1 -- 172.21.15.89:0/2057680055 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55300b3770 con 0x7f55300cca50 2023-10-10T20:40:07.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.141+0000 7f55277fe700 1 -- 172.21.15.89:0/2057680055 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551800fa10 con 0x7f55300cca50 2023-10-10T20:40:07.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.141+0000 7f55355de700 1 -- 172.21.15.89:0/2057680055 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55300b2450 con 0x7f55300cca50 2023-10-10T20:40:07.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.141+0000 7f55355de700 1 -- 172.21.15.89:0/2057680055 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55300cd7d0 con 0x7f55300cca50 2023-10-10T20:40:07.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.142+0000 7f55277fe700 1 -- 172.21.15.89:0/2057680055 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f551801a400 con 0x7f55300cca50 2023-10-10T20:40:07.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.142+0000 7f55277fe700 1 --2- 172.21.15.89:0/2057680055 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5510067200 0x7f55100696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:07.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.142+0000 7f55277fe700 1 -- 172.21.15.89:0/2057680055 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f5518084200 con 0x7f55300cca50 2023-10-10T20:40:07.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.142+0000 7f552effd700 1 --2- 172.21.15.89:0/2057680055 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5510067200 0x7f55100696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:07.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.143+0000 7f552effd700 1 --2- 172.21.15.89:0/2057680055 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5510067200 0x7f55100696b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f55200072e0 tx=0x7f5520001a10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:07.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.147+0000 7f55277fe700 1 -- 172.21.15.89:0/2057680055 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5518054e10 con 0x7f55300cca50 2023-10-10T20:40:07.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.329+0000 7f55355de700 1 -- 172.21.15.89:0/2057680055 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f55300718b0 con 0x7f55300cca50 2023-10-10T20:40:08.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:07.999+0000 7f55277fe700 1 -- 172.21.15.89:0/2057680055 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v86) v1 ==== 170+0+0 (secure 0 0 0) 0x7f55180549a0 con 0x7f55300cca50 2023-10-10T20:40:08.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.003+0000 7f55355de700 1 -- 172.21.15.89:0/2057680055 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5510067200 msgr2=0x7f55100696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:08.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.003+0000 7f55355de700 1 --2- 172.21.15.89:0/2057680055 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5510067200 0x7f55100696b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f55200072e0 tx=0x7f5520001a10 comp rx=0 tx=0).stop 2023-10-10T20:40:08.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.003+0000 7f55355de700 1 -- 172.21.15.89:0/2057680055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55300cca50 msgr2=0x7f55300c7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:08.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.003+0000 7f55355de700 1 --2- 172.21.15.89:0/2057680055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55300cca50 0x7f55300c7e20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f551800b5b0 tx=0x7f55180090f0 comp rx=0 tx=0).stop 2023-10-10T20:40:08.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.003+0000 7f55355de700 1 -- 172.21.15.89:0/2057680055 shutdown_connections 2023-10-10T20:40:08.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.003+0000 7f55355de700 1 --2- 172.21.15.89:0/2057680055 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5510067200 0x7f55100696b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:08.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.003+0000 7f55355de700 1 --2- 172.21.15.89:0/2057680055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55300cce20 0x7f55300c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:08.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.003+0000 7f55355de700 1 --2- 172.21.15.89:0/2057680055 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f55300caab0 0x7f5530167450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:08.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.003+0000 7f55355de700 1 --2- 172.21.15.89:0/2057680055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55300cca50 0x7f55300c7e20 secure :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f551800b5b0 tx=0x7f55180090f0 comp rx=0 tx=0).stop 2023-10-10T20:40:08.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.003+0000 7f55355de700 1 -- 172.21.15.89:0/2057680055 >> 172.21.15.89:0/2057680055 conn(0x7f55300ce5c0 msgr2=0x7f553006ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:08.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.004+0000 7f55355de700 1 -- 172.21.15.89:0/2057680055 shutdown_connections 2023-10-10T20:40:08.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.004+0000 7f55355de700 1 -- 172.21.15.89:0/2057680055 wait complete. 2023-10-10T20:40:08.016 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2023-10-10T20:40:08.031 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:40:08.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.148+0000 7f5f82098700 1 Processor -- start 2023-10-10T20:40:08.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.148+0000 7f5f82098700 1 -- start start 2023-10-10T20:40:08.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.149+0000 7f5f82098700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f7c159c60 0x7f5f7c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:08.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.149+0000 7f5f82098700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5f7c15db10 0x7f5f7c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:08.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.149+0000 7f5f82098700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7c15a160 con 0x7f5f7c159890 2023-10-10T20:40:08.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.149+0000 7f5f82098700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7c0d13a0 con 0x7f5f7c15db10 2023-10-10T20:40:08.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.149+0000 7f5f82098700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c15e140 con 0x7f5f7c159c60 2023-10-10T20:40:08.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.149+0000 7f5f7bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5f7c15db10 0x7f5f7c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:08.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.149+0000 7f5f7bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5f7c15db10 0x7f5f7c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43282/0 (socket says 172.21.15.89:43282) 2023-10-10T20:40:08.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.149+0000 7f5f7bfff700 1 -- 172.21.15.89:0/203985087 learned_addr learned my addr 172.21.15.89:0/203985087 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:08.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.149+0000 7f5f7affd700 1 --2- 172.21.15.89:0/203985087 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f7c159c60 0x7f5f7c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:08.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.149+0000 7f5f7bfff700 1 -- 172.21.15.89:0/203985087 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f7c159c60 msgr2=0x7f5f7c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:08.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.149+0000 7f5f7bfff700 1 --2- 172.21.15.89:0/203985087 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f7c159c60 0x7f5f7c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:08.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.150+0000 7f5f7bfff700 1 -- 172.21.15.89:0/203985087 >> v1:172.21.15.111:6789/0 conn(0x7f5f7c159890 legacy=0x7f5f7c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:08.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.150+0000 7f5f7bfff700 1 -- 172.21.15.89:0/203985087 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c15a3d0 con 0x7f5f7c15db10 2023-10-10T20:40:08.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.150+0000 7f5f7bfff700 1 --2- 172.21.15.89:0/203985087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7c15db10 0x7f5f7c069560 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5f740099b0 tx=0x7f5f74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c8c91b38f397bb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:08.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.150+0000 7f5f7a7fc700 1 -- 172.21.15.89:0/203985087 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7400b860 con 0x7f5f7c15db10 2023-10-10T20:40:08.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.150+0000 7f5f7a7fc700 1 -- 172.21.15.89:0/203985087 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7400b9c0 con 0x7f5f7c15db10 2023-10-10T20:40:08.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.150+0000 7f5f7a7fc700 1 -- 172.21.15.89:0/203985087 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7400bd60 con 0x7f5f7c15db10 2023-10-10T20:40:08.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.150+0000 7f5f82098700 1 -- 172.21.15.89:0/203985087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7c15db10 msgr2=0x7f5f7c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:08.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.150+0000 7f5f82098700 1 --2- 172.21.15.89:0/203985087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7c15db10 0x7f5f7c069560 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5f740099b0 tx=0x7f5f74009250 comp rx=0 tx=0).stop 2023-10-10T20:40:08.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.150+0000 7f5f82098700 1 -- 172.21.15.89:0/203985087 shutdown_connections 2023-10-10T20:40:08.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.150+0000 7f5f82098700 1 --2- 172.21.15.89:0/203985087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7c15db10 0x7f5f7c069560 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:08.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.150+0000 7f5f82098700 1 --2- 172.21.15.89:0/203985087 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f7c159c60 0x7f5f7c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:08.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.150+0000 7f5f82098700 1 -- 172.21.15.89:0/203985087 >> 172.21.15.89:0/203985087 conn(0x7f5f7c068e20 msgr2=0x7f5f7c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:08.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.150+0000 7f5f82098700 1 -- 172.21.15.89:0/203985087 shutdown_connections 2023-10-10T20:40:08.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.150+0000 7f5f82098700 1 -- 172.21.15.89:0/203985087 wait complete. 2023-10-10T20:40:08.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.152+0000 7f5f82098700 1 Processor -- start 2023-10-10T20:40:08.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.152+0000 7f5f82098700 1 -- start start 2023-10-10T20:40:08.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.152+0000 7f5f82098700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f7c159890 0x7f5f7c0b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:08.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.152+0000 7f5f82098700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7c159c60 0x7f5f7c0cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:08.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.152+0000 7f5f82098700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f7c15db10 0x7f5f7c0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:08.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.152+0000 7f5f82098700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c0b2770 con 0x7f5f7c159c60 2023-10-10T20:40:08.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.152+0000 7f5f82098700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c169880 con 0x7f5f7c15db10 2023-10-10T20:40:08.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.152+0000 7f5f82098700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c16a120 con 0x7f5f7c159890 2023-10-10T20:40:08.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.152+0000 7f5f7affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7c159c60 0x7f5f7c0cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:08.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.153+0000 7f5f7affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7c159c60 0x7f5f7c0cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43292/0 (socket says 172.21.15.89:43292) 2023-10-10T20:40:08.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.153+0000 7f5f7affd700 1 -- 172.21.15.89:0/1829623759 learned_addr learned my addr 172.21.15.89:0/1829623759 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:08.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.153+0000 7f5f7bfff700 1 --2- 172.21.15.89:0/1829623759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f7c15db10 0x7f5f7c0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:08.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.153+0000 7f5f7b7fe700 1 --2- 172.21.15.89:0/1829623759 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f7c159890 0x7f5f7c0b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:08.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.153+0000 7f5f7affd700 1 -- 172.21.15.89:0/1829623759 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f7c159890 msgr2=0x7f5f7c0b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:08.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.153+0000 7f5f7affd700 1 --2- 172.21.15.89:0/1829623759 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f7c159890 0x7f5f7c0b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:08.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.153+0000 7f5f7affd700 1 -- 172.21.15.89:0/1829623759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f7c15db10 msgr2=0x7f5f7c0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:08.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.153+0000 7f5f7affd700 1 --2- 172.21.15.89:0/1829623759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f7c15db10 0x7f5f7c0cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:08.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.153+0000 7f5f7affd700 1 -- 172.21.15.89:0/1829623759 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f740094f0 con 0x7f5f7c159c60 2023-10-10T20:40:08.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.153+0000 7f5f7b7fe700 1 --2- 172.21.15.89:0/1829623759 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f7c159890 0x7f5f7c0b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:08.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.153+0000 7f5f7affd700 1 --2- 172.21.15.89:0/1829623759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7c159c60 0x7f5f7c0cd2d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000c8b0 tx=0x7f5f70008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:08.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.153+0000 7f5f637fe700 1 -- 172.21.15.89:0/1829623759 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f70011600 con 0x7f5f7c159c60 2023-10-10T20:40:08.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.153+0000 7f5f637fe700 1 -- 172.21.15.89:0/1829623759 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f70011760 con 0x7f5f7c159c60 2023-10-10T20:40:08.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.153+0000 7f5f82098700 1 -- 172.21.15.89:0/1829623759 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f7c0b2010 con 0x7f5f7c159c60 2023-10-10T20:40:08.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.153+0000 7f5f637fe700 1 -- 172.21.15.89:0/1829623759 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f70011b00 con 0x7f5f7c159c60 2023-10-10T20:40:08.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.153+0000 7f5f82098700 1 -- 172.21.15.89:0/1829623759 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7c169ad0 con 0x7f5f7c159c60 2023-10-10T20:40:08.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.154+0000 7f5f82098700 1 -- 172.21.15.89:0/1829623759 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7c04c3c0 con 0x7f5f7c159c60 2023-10-10T20:40:08.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.155+0000 7f5f637fe700 1 -- 172.21.15.89:0/1829623759 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f5f700223a0 con 0x7f5f7c159c60 2023-10-10T20:40:08.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.155+0000 7f5f637fe700 1 --2- 172.21.15.89:0/1829623759 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f68067200 0x7f5f680696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:08.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.155+0000 7f5f7b7fe700 1 --2- 172.21.15.89:0/1829623759 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f68067200 0x7f5f680696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:08.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.155+0000 7f5f637fe700 1 -- 172.21.15.89:0/1829623759 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f5f70084ac0 con 0x7f5f7c159c60 2023-10-10T20:40:08.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.156+0000 7f5f7b7fe700 1 --2- 172.21.15.89:0/1829623759 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f68067200 0x7f5f680696b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c007fd0 tx=0x7f5f6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:08.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.160+0000 7f5f637fe700 1 -- 172.21.15.89:0/1829623759 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f70051930 con 0x7f5f7c159c60 2023-10-10T20:40:08.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:08.386+0000 7f5f82098700 1 -- 172.21.15.89:0/1829623759 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5f7c16a260 con 0x7f5f7c159c60 2023-10-10T20:40:09.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.027+0000 7f5f637fe700 1 -- 172.21.15.89:0/1829623759 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v51) v1 ==== 174+0+0 (secure 0 0 0) 0x7f5f7001e0c0 con 0x7f5f7c159c60 2023-10-10T20:40:09.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.039+0000 7f5f82098700 1 -- 172.21.15.89:0/1829623759 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f68067200 msgr2=0x7f5f680696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:09.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.039+0000 7f5f82098700 1 --2- 172.21.15.89:0/1829623759 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f68067200 0x7f5f680696b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c007fd0 tx=0x7f5f6c009040 comp rx=0 tx=0).stop 2023-10-10T20:40:09.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.039+0000 7f5f82098700 1 -- 172.21.15.89:0/1829623759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7c159c60 msgr2=0x7f5f7c0cd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:09.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.039+0000 7f5f82098700 1 --2- 172.21.15.89:0/1829623759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7c159c60 0x7f5f7c0cd2d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000c8b0 tx=0x7f5f70008040 comp rx=0 tx=0).stop 2023-10-10T20:40:09.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.039+0000 7f5f82098700 1 -- 172.21.15.89:0/1829623759 shutdown_connections 2023-10-10T20:40:09.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.039+0000 7f5f82098700 1 --2- 172.21.15.89:0/1829623759 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5f68067200 0x7f5f680696b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.039+0000 7f5f82098700 1 --2- 172.21.15.89:0/1829623759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f7c15db10 0x7f5f7c0cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.039+0000 7f5f82098700 1 --2- 172.21.15.89:0/1829623759 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5f7c159890 0x7f5f7c0b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.039+0000 7f5f82098700 1 --2- 172.21.15.89:0/1829623759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7c159c60 0x7f5f7c0cd2d0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.039+0000 7f5f82098700 1 -- 172.21.15.89:0/1829623759 >> 172.21.15.89:0/1829623759 conn(0x7f5f7c068e20 msgr2=0x7f5f7c156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:09.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.039+0000 7f5f82098700 1 -- 172.21.15.89:0/1829623759 shutdown_connections 2023-10-10T20:40:09.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.039+0000 7f5f82098700 1 -- 172.21.15.89:0/1829623759 wait complete. 2023-10-10T20:40:09.044 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 14 and data pool 15 2023-10-10T20:40:09.055 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:40:09.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.170+0000 7f9cc4f17700 1 Processor -- start 2023-10-10T20:40:09.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.170+0000 7f9cc4f17700 1 -- start start 2023-10-10T20:40:09.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.171+0000 7f9cc4f17700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9cc0069560 0x7f9cc00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:09.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.171+0000 7f9cc4f17700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc0069ee0 con 0x7f9cc0159890 2023-10-10T20:40:09.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.171+0000 7f9cc4f17700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc015a160 con 0x7f9cc0159c60 2023-10-10T20:40:09.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.171+0000 7f9cc4f17700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc00d13a0 con 0x7f9cc0069560 2023-10-10T20:40:09.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.171+0000 7f9cbe59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9cc0159890 0x7f9cc015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55702/0 (socket says 172.21.15.89:55702) 2023-10-10T20:40:09.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.171+0000 7f9cbe59c700 1 -- 172.21.15.89:0/3667425660 learned_addr learned my addr 172.21.15.89:0/3667425660 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:09.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbed9d700 1 --2- 172.21.15.89:0/3667425660 >> v2:172.21.15.111:3300/0 conn(0x7f9cc0069560 0x7f9cc00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:09.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbdd9b700 1 -- 172.21.15.89:0/3667425660 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706814479 0 0) 0x7f9cc0069ee0 con 0x7f9cc0159890 2023-10-10T20:40:09.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbdd9b700 1 -- 172.21.15.89:0/3667425660 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cac003570 con 0x7f9cc0159890 2023-10-10T20:40:09.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbdd9b700 1 -- 172.21.15.89:0/3667425660 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3490229805 0 0) 0x7f9cac003570 con 0x7f9cc0159890 2023-10-10T20:40:09.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbdd9b700 1 -- 172.21.15.89:0/3667425660 >> v2:172.21.15.111:3300/0 conn(0x7f9cc0069560 msgr2=0x7f9cc00699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:09.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbdd9b700 1 --2- 172.21.15.89:0/3667425660 >> v2:172.21.15.111:3300/0 conn(0x7f9cc0069560 0x7f9cc00699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbdd9b700 1 -- 172.21.15.89:0/3667425660 >> v1:172.21.15.111:6789/0 conn(0x7f9cc0159c60 legacy=0x7f9cc015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:09.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbdd9b700 1 -- 172.21.15.89:0/3667425660 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc015e140 con 0x7f9cc0159890 2023-10-10T20:40:09.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbdd9b700 1 -- 172.21.15.89:0/3667425660 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f9cb0001470 con 0x7f9cc0159890 2023-10-10T20:40:09.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbdd9b700 1 -- 172.21.15.89:0/3667425660 >> v1:172.21.15.89:6789/0 conn(0x7f9cc0159890 legacy=0x7f9cc015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:09.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbdd9b700 1 --2- 172.21.15.89:0/3667425660 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9cac003a60 0x7f9cac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:09.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbdd9b700 1 --2- 172.21.15.89:0/3667425660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cac004810 0x7f9cac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:09.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbdd9b700 1 --2- 172.21.15.89:0/3667425660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cac0051c0 0x7f9cac00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:09.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbdd9b700 1 -- 172.21.15.89:0/3667425660 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc00d13a0 con 0x7f9cac004810 2023-10-10T20:40:09.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbdd9b700 1 -- 172.21.15.89:0/3667425660 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac00c0d0 con 0x7f9cac0051c0 2023-10-10T20:40:09.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbdd9b700 1 -- 172.21.15.89:0/3667425660 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac00c290 con 0x7f9cac003a60 2023-10-10T20:40:09.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cb5d9b700 1 --2- 172.21.15.89:0/3667425660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cac004810 0x7f9cac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:09.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbe59c700 1 --2- 172.21.15.89:0/3667425660 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9cac003a60 0x7f9cac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:09.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbed9d700 1 --2- 172.21.15.89:0/3667425660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cac0051c0 0x7f9cac00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:09.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cb5d9b700 1 -- 172.21.15.89:0/3667425660 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9cac003a60 msgr2=0x7f9cac003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:09.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cb5d9b700 1 --2- 172.21.15.89:0/3667425660 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9cac003a60 0x7f9cac003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cb5d9b700 1 -- 172.21.15.89:0/3667425660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cac0051c0 msgr2=0x7f9cac00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:09.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cb5d9b700 1 --2- 172.21.15.89:0/3667425660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cac0051c0 0x7f9cac00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cb5d9b700 1 -- 172.21.15.89:0/3667425660 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac003570 con 0x7f9cac004810 2023-10-10T20:40:09.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbed9d700 1 -- 172.21.15.89:0/3667425660 reap_dead start 2023-10-10T20:40:09.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.172+0000 7f9cbe59c700 1 --2- 172.21.15.89:0/3667425660 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9cac003a60 0x7f9cac003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:09.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.173+0000 7f9cb5d9b700 1 --2- 172.21.15.89:0/3667425660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cac004810 0x7f9cac004c80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9ca80018b0 tx=0x7f9ca8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd142ed5fbcfdaff server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:09.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.173+0000 7f9cbdd9b700 1 -- 172.21.15.89:0/3667425660 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8012420 con 0x7f9cac004810 2023-10-10T20:40:09.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.173+0000 7f9cbdd9b700 1 -- 172.21.15.89:0/3667425660 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca80125a0 con 0x7f9cac004810 2023-10-10T20:40:09.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.173+0000 7f9cbdd9b700 1 -- 172.21.15.89:0/3667425660 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8012960 con 0x7f9cac004810 2023-10-10T20:40:09.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.173+0000 7f9cc4f17700 1 -- 172.21.15.89:0/3667425660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cac004810 msgr2=0x7f9cac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:09.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.173+0000 7f9cc4f17700 1 --2- 172.21.15.89:0/3667425660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cac004810 0x7f9cac004c80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9ca80018b0 tx=0x7f9ca8007d30 comp rx=0 tx=0).stop 2023-10-10T20:40:09.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.173+0000 7f9cc4f17700 1 -- 172.21.15.89:0/3667425660 shutdown_connections 2023-10-10T20:40:09.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.173+0000 7f9cc4f17700 1 --2- 172.21.15.89:0/3667425660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cac004810 0x7f9cac004c80 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.173+0000 7f9cc4f17700 1 -- 172.21.15.89:0/3667425660 >> 172.21.15.89:0/3667425660 conn(0x7f9cc0068e20 msgr2=0x7f9cc00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:09.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.173+0000 7f9cc4f17700 1 -- 172.21.15.89:0/3667425660 shutdown_connections 2023-10-10T20:40:09.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.173+0000 7f9cc4f17700 1 -- 172.21.15.89:0/3667425660 wait complete. 2023-10-10T20:40:09.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.174+0000 7f9cc4f17700 1 Processor -- start 2023-10-10T20:40:09.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.174+0000 7f9cc4f17700 1 -- start start 2023-10-10T20:40:09.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.175+0000 7f9cc4f17700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9cac004810 0x7f9cc0169da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:09.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.175+0000 7f9cc4f17700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cc016afc0 0x7f9cc0153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:09.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.175+0000 7f9cc4f17700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cc016b430 0x7f9cc0154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:09.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.175+0000 7f9cc4f17700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc014ae20 con 0x7f9cc016b430 2023-10-10T20:40:09.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.175+0000 7f9cc4f17700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc016a690 con 0x7f9cc016afc0 2023-10-10T20:40:09.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.175+0000 7f9cc4f17700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc016a970 con 0x7f9cac004810 2023-10-10T20:40:09.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.175+0000 7f9cbed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cc016b430 0x7f9cc0154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:09.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.175+0000 7f9cbed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cc016b430 0x7f9cc0154220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43314/0 (socket says 172.21.15.89:43314) 2023-10-10T20:40:09.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.175+0000 7f9cbed9d700 1 -- 172.21.15.89:0/2397778355 learned_addr learned my addr 172.21.15.89:0/2397778355 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:09.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.175+0000 7f9cbe59c700 1 --2- 172.21.15.89:0/2397778355 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9cac004810 0x7f9cc0169da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:09.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.175+0000 7f9cbed9d700 1 -- 172.21.15.89:0/2397778355 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9cac004810 msgr2=0x7f9cc0169da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:09.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.176+0000 7f9cbed9d700 1 --2- 172.21.15.89:0/2397778355 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9cac004810 0x7f9cc0169da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.176+0000 7f9cbed9d700 1 -- 172.21.15.89:0/2397778355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cc016afc0 msgr2=0x7f9cc0153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:09.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.176+0000 7f9cb5d9b700 1 --2- 172.21.15.89:0/2397778355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cc016afc0 0x7f9cc0153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:09.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.176+0000 7f9cbed9d700 1 --2- 172.21.15.89:0/2397778355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cc016afc0 0x7f9cc0153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.176+0000 7f9cbed9d700 1 -- 172.21.15.89:0/2397778355 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc016ac50 con 0x7f9cc016b430 2023-10-10T20:40:09.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.176+0000 7f9cb5d9b700 1 --2- 172.21.15.89:0/2397778355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cc016afc0 0x7f9cc0153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:09.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.176+0000 7f9cbed9d700 1 --2- 172.21.15.89:0/2397778355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cc016b430 0x7f9cc0154220 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9cc006a300 tx=0x7f9cb800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:09.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.176+0000 7f9cb6ffd700 1 -- 172.21.15.89:0/2397778355 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb800e040 con 0x7f9cc016b430 2023-10-10T20:40:09.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.176+0000 7f9cb6ffd700 1 -- 172.21.15.89:0/2397778355 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb800f040 con 0x7f9cc016b430 2023-10-10T20:40:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.176+0000 7f9cb6ffd700 1 -- 172.21.15.89:0/2397778355 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb800fe20 con 0x7f9cc016b430 2023-10-10T20:40:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.176+0000 7f9cc4f17700 1 -- 172.21.15.89:0/2397778355 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc014b070 con 0x7f9cc016b430 2023-10-10T20:40:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.176+0000 7f9cc4f17700 1 -- 172.21.15.89:0/2397778355 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc0152460 con 0x7f9cc016b430 2023-10-10T20:40:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.177+0000 7f9cc4f17700 1 -- 172.21.15.89:0/2397778355 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc014dc90 con 0x7f9cc016b430 2023-10-10T20:40:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.177+0000 7f9cb6ffd700 1 -- 172.21.15.89:0/2397778355 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f9cb8015640 con 0x7f9cc016b430 2023-10-10T20:40:09.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.177+0000 7f9cb6ffd700 1 --2- 172.21.15.89:0/2397778355 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ca0067200 0x7f9ca00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:09.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.178+0000 7f9cb6ffd700 1 -- 172.21.15.89:0/2397778355 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9cb8085480 con 0x7f9cc016b430 2023-10-10T20:40:09.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.178+0000 7f9cbe59c700 1 --2- 172.21.15.89:0/2397778355 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ca0067200 0x7f9ca00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:09.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.178+0000 7f9cbe59c700 1 --2- 172.21.15.89:0/2397778355 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ca0067200 0x7f9ca00696b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0001200 tx=0x7f9cb0001110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:09.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.182+0000 7f9cb6ffd700 1 -- 172.21.15.89:0/2397778355 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9cb8052290 con 0x7f9cc016b430 2023-10-10T20:40:09.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.366+0000 7f9cc4f17700 1 -- 172.21.15.89:0/2397778355 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cc016ade0 con 0x7f9cc016b430 2023-10-10T20:40:09.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.367+0000 7f9cb6ffd700 1 -- 172.21.15.89:0/2397778355 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+15366 (secure 0 0 0) 0x7f9cb80558b0 con 0x7f9cc016b430 2023-10-10T20:40:09.369 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:09.369 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":87,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:40:08.387550+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:40:06.747813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-10T20:40:07.347913+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:40:09.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.369+0000 7f9cc4f17700 1 -- 172.21.15.89:0/2397778355 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ca0067200 msgr2=0x7f9ca00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:09.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.369+0000 7f9cc4f17700 1 --2- 172.21.15.89:0/2397778355 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ca0067200 0x7f9ca00696b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0001200 tx=0x7f9cb0001110 comp rx=0 tx=0).stop 2023-10-10T20:40:09.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.369+0000 7f9cc4f17700 1 -- 172.21.15.89:0/2397778355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cc016b430 msgr2=0x7f9cc0154220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:09.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.369+0000 7f9cc4f17700 1 --2- 172.21.15.89:0/2397778355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cc016b430 0x7f9cc0154220 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9cc006a300 tx=0x7f9cb800b040 comp rx=0 tx=0).stop 2023-10-10T20:40:09.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.369+0000 7f9cc4f17700 1 -- 172.21.15.89:0/2397778355 shutdown_connections 2023-10-10T20:40:09.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.369+0000 7f9cc4f17700 1 --2- 172.21.15.89:0/2397778355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cc016b430 0x7f9cc0154220 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.369+0000 7f9cc4f17700 1 --2- 172.21.15.89:0/2397778355 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9cac004810 0x7f9cc0169da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.369+0000 7f9cc4f17700 1 --2- 172.21.15.89:0/2397778355 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ca0067200 0x7f9ca00696b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.369+0000 7f9cc4f17700 1 --2- 172.21.15.89:0/2397778355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cc016afc0 0x7f9cc0153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.369+0000 7f9cc4f17700 1 -- 172.21.15.89:0/2397778355 >> 172.21.15.89:0/2397778355 conn(0x7f9cc0068e20 msgr2=0x7f9cc0069960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:09.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.369+0000 7f9cc4f17700 1 -- 172.21.15.89:0/2397778355 shutdown_connections 2023-10-10T20:40:09.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.369+0000 7f9cc4f17700 1 -- 172.21.15.89:0/2397778355 wait complete. 2023-10-10T20:40:09.383 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:40:09.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.496+0000 7f617e875700 1 Processor -- start 2023-10-10T20:40:09.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.496+0000 7f617e875700 1 -- start start 2023-10-10T20:40:09.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.497+0000 7f617e875700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6178159890 0x7f617815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:09.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.497+0000 7f617e875700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f617815a160 con 0x7f6178159c60 2023-10-10T20:40:09.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.497+0000 7f617e875700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61780d13a0 con 0x7f6178069560 2023-10-10T20:40:09.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.497+0000 7f617e875700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f617815e140 con 0x7f6178159890 2023-10-10T20:40:09.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.497+0000 7f6177fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6178159890 0x7f617815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:09.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.497+0000 7f61777fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6178159c60 0x7f617815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55708/0 (socket says 172.21.15.89:55708) 2023-10-10T20:40:09.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.497+0000 7f6177fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6178159890 0x7f617815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43330/0 (socket says 172.21.15.89:43330) 2023-10-10T20:40:09.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.497+0000 7f61777fe700 1 -- 172.21.15.89:0/1138786930 learned_addr learned my addr 172.21.15.89:0/1138786930 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:09.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.497+0000 7f6177fff700 1 -- 172.21.15.89:0/1138786930 >> v1:172.21.15.111:6789/0 conn(0x7f6178069560 legacy=0x7f61780ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:09.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.497+0000 7f6177fff700 1 -- 172.21.15.89:0/1138786930 >> v1:172.21.15.89:6789/0 conn(0x7f6178159c60 legacy=0x7f617815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:09.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.497+0000 7f6177fff700 1 -- 172.21.15.89:0/1138786930 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617815a3d0 con 0x7f6178159890 2023-10-10T20:40:09.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.497+0000 7f6176ffd700 1 -- 172.21.15.89:0/1138786930 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4208490153 0 0) 0x7f617815a160 con 0x7f6178159c60 2023-10-10T20:40:09.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.497+0000 7f6177fff700 1 --2- 172.21.15.89:0/1138786930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6178159890 0x7f617815abd0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f61680098e0 tx=0x7f61680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7bbbd4bee2609bf server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:09.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.497+0000 7f6176ffd700 1 -- 172.21.15.89:0/1138786930 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168009e80 con 0x7f6178159890 2023-10-10T20:40:09.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.498+0000 7f6176ffd700 1 -- 172.21.15.89:0/1138786930 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616800b820 con 0x7f6178159890 2023-10-10T20:40:09.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.498+0000 7f6176ffd700 1 -- 172.21.15.89:0/1138786930 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800bba0 con 0x7f6178159890 2023-10-10T20:40:09.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.498+0000 7f617e875700 1 -- 172.21.15.89:0/1138786930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6178159890 msgr2=0x7f617815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:09.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.498+0000 7f617e875700 1 --2- 172.21.15.89:0/1138786930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6178159890 0x7f617815abd0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f61680098e0 tx=0x7f61680091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.498+0000 7f617e875700 1 -- 172.21.15.89:0/1138786930 shutdown_connections 2023-10-10T20:40:09.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.498+0000 7f617e875700 1 --2- 172.21.15.89:0/1138786930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6178159890 0x7f617815abd0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.498+0000 7f617e875700 1 -- 172.21.15.89:0/1138786930 >> 172.21.15.89:0/1138786930 conn(0x7f6178068e20 msgr2=0x7f61780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:09.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.498+0000 7f617e875700 1 -- 172.21.15.89:0/1138786930 shutdown_connections 2023-10-10T20:40:09.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.498+0000 7f617e875700 1 -- 172.21.15.89:0/1138786930 wait complete. 2023-10-10T20:40:09.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.499+0000 7f617e875700 1 Processor -- start 2023-10-10T20:40:09.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.499+0000 7f617e875700 1 -- start start 2023-10-10T20:40:09.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.500+0000 7f617e875700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6178069560 0x7f6178140c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:09.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.500+0000 7f617e875700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6178159890 0x7f6178155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:09.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.500+0000 7f617e875700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6178159c60 0x7f6178156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:09.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.500+0000 7f617e875700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6178152af0 con 0x7f6178159c60 2023-10-10T20:40:09.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.500+0000 7f617e875700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61780cede0 con 0x7f6178159890 2023-10-10T20:40:09.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.500+0000 7f617e875700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61780ce1d0 con 0x7f6178069560 2023-10-10T20:40:09.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.500+0000 7f617ce12700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6178159c60 0x7f6178156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:09.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.500+0000 7f617ce12700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6178159c60 0x7f6178156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43340/0 (socket says 172.21.15.89:43340) 2023-10-10T20:40:09.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.500+0000 7f617ce12700 1 -- 172.21.15.89:0/3431871135 learned_addr learned my addr 172.21.15.89:0/3431871135 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:09.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.500+0000 7f61777fe700 1 --2- 172.21.15.89:0/3431871135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6178159890 0x7f6178155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:09.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.500+0000 7f6177fff700 1 --2- 172.21.15.89:0/3431871135 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6178069560 0x7f6178140c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.500+0000 7f617ce12700 1 -- 172.21.15.89:0/3431871135 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6178069560 msgr2=0x7f6178140c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.500+0000 7f617ce12700 1 --2- 172.21.15.89:0/3431871135 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6178069560 0x7f6178140c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.500+0000 7f617ce12700 1 -- 172.21.15.89:0/3431871135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6178159890 msgr2=0x7f6178155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.500+0000 7f617ce12700 1 --2- 172.21.15.89:0/3431871135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6178159890 0x7f6178155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.501+0000 7f617ce12700 1 -- 172.21.15.89:0/3431871135 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6168009420 con 0x7f6178159c60 2023-10-10T20:40:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.501+0000 7f617ce12700 1 --2- 172.21.15.89:0/3431871135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6178159c60 0x7f6178156150 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f617000bbb0 tx=0x7f6170009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.501+0000 7f615ffff700 1 -- 172.21.15.89:0/3431871135 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700092e0 con 0x7f6178159c60 2023-10-10T20:40:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.501+0000 7f615ffff700 1 -- 172.21.15.89:0/3431871135 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617000c7a0 con 0x7f6178159c60 2023-10-10T20:40:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.501+0000 7f617e875700 1 -- 172.21.15.89:0/3431871135 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61780ce4b0 con 0x7f6178159c60 2023-10-10T20:40:09.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.501+0000 7f615ffff700 1 -- 172.21.15.89:0/3431871135 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617000cb40 con 0x7f6178159c60 2023-10-10T20:40:09.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.501+0000 7f617e875700 1 -- 172.21.15.89:0/3431871135 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61780cea70 con 0x7f6178159c60 2023-10-10T20:40:09.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.502+0000 7f617e875700 1 -- 172.21.15.89:0/3431871135 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f617804c3c0 con 0x7f6178159c60 2023-10-10T20:40:09.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.502+0000 7f615ffff700 1 -- 172.21.15.89:0/3431871135 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f6170012070 con 0x7f6178159c60 2023-10-10T20:40:09.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.502+0000 7f615ffff700 1 --2- 172.21.15.89:0/3431871135 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6164067200 0x7f61640696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:09.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.503+0000 7f615ffff700 1 -- 172.21.15.89:0/3431871135 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f61700846f0 con 0x7f6178159c60 2023-10-10T20:40:09.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.503+0000 7f6177fff700 1 --2- 172.21.15.89:0/3431871135 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6164067200 0x7f61640696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:09.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.503+0000 7f6177fff700 1 --2- 172.21.15.89:0/3431871135 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6164067200 0x7f61640696b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f61781540a0 tx=0x7f6168016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:09.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.507+0000 7f615ffff700 1 -- 172.21.15.89:0/3431871135 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6170051530 con 0x7f6178159c60 2023-10-10T20:40:09.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.693+0000 7f617e875700 1 -- 172.21.15.89:0/3431871135 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61780b2f10 con 0x7f6178159c60 2023-10-10T20:40:09.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.693+0000 7f615ffff700 1 -- 172.21.15.89:0/3431871135 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+15366 (secure 0 0 0) 0x7f6170054b50 con 0x7f6178159c60 2023-10-10T20:40:09.696 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:09.696 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":87,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:40:08.387550+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:40:06.747813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-10T20:40:07.347913+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:40:09.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.695+0000 7f617e875700 1 -- 172.21.15.89:0/3431871135 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6164067200 msgr2=0x7f61640696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:09.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.695+0000 7f617e875700 1 --2- 172.21.15.89:0/3431871135 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6164067200 0x7f61640696b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f61781540a0 tx=0x7f6168016000 comp rx=0 tx=0).stop 2023-10-10T20:40:09.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.695+0000 7f617e875700 1 -- 172.21.15.89:0/3431871135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6178159c60 msgr2=0x7f6178156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:09.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.695+0000 7f617e875700 1 --2- 172.21.15.89:0/3431871135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6178159c60 0x7f6178156150 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f617000bbb0 tx=0x7f6170009840 comp rx=0 tx=0).stop 2023-10-10T20:40:09.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.695+0000 7f617e875700 1 -- 172.21.15.89:0/3431871135 shutdown_connections 2023-10-10T20:40:09.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.695+0000 7f617e875700 1 --2- 172.21.15.89:0/3431871135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6178159c60 0x7f6178156150 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.695+0000 7f617e875700 1 --2- 172.21.15.89:0/3431871135 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6178069560 0x7f6178140c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.695+0000 7f617e875700 1 --2- 172.21.15.89:0/3431871135 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6164067200 0x7f61640696b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.695+0000 7f617e875700 1 --2- 172.21.15.89:0/3431871135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6178159890 0x7f6178155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.695+0000 7f617e875700 1 -- 172.21.15.89:0/3431871135 >> 172.21.15.89:0/3431871135 conn(0x7f6178068e20 msgr2=0x7f61780b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:09.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.696+0000 7f617e875700 1 -- 172.21.15.89:0/3431871135 shutdown_connections 2023-10-10T20:40:09.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.696+0000 7f617e875700 1 -- 172.21.15.89:0/3431871135 wait complete. 2023-10-10T20:40:09.710 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:40:09.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.825+0000 7fca1a86f700 1 Processor -- start 2023-10-10T20:40:09.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.825+0000 7fca1a86f700 1 -- start start 2023-10-10T20:40:09.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.826+0000 7fca1a86f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fca140b16d0 0x7fca140b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:09.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.826+0000 7fca1a86f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fca140b33b0 0x7fca140b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:09.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.826+0000 7fca1a86f700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca140b2080 con 0x7fca140b4e70 2023-10-10T20:40:09.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.826+0000 7fca1a86f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca140cfb00 con 0x7fca140b16d0 2023-10-10T20:40:09.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.826+0000 7fca1a86f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca140b5c40 con 0x7fca140b33b0 2023-10-10T20:40:09.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.826+0000 7fca1906c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fca140b16d0 0x7fca140b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:09.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.826+0000 7fca1906c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fca140b16d0 0x7fca140b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43342/0 (socket says 172.21.15.89:43342) 2023-10-10T20:40:09.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.826+0000 7fca1906c700 1 -- 172.21.15.89:0/1410511476 learned_addr learned my addr 172.21.15.89:0/1410511476 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.826+0000 7fca1906c700 1 -- 172.21.15.89:0/1410511476 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fca140b33b0 msgr2=0x7fca140b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.826+0000 7fca1906c700 1 --2- 172.21.15.89:0/1410511476 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fca140b33b0 0x7fca140b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.826+0000 7fca1906c700 1 -- 172.21.15.89:0/1410511476 >> v1:172.21.15.111:6789/0 conn(0x7fca140b4e70 legacy=0x7fca140ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.826+0000 7fca1906c700 1 -- 172.21.15.89:0/1410511476 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca140b5f20 con 0x7fca140b16d0 2023-10-10T20:40:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.826+0000 7fca1906c700 1 --2- 172.21.15.89:0/1410511476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca140b16d0 0x7fca140b1b40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fca040099b0 tx=0x7fca04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6304451fd2a19d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.826+0000 7fca1886b700 1 -- 172.21.15.89:0/1410511476 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0400b820 con 0x7fca140b16d0 2023-10-10T20:40:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.826+0000 7fca1886b700 1 -- 172.21.15.89:0/1410511476 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0400b980 con 0x7fca140b16d0 2023-10-10T20:40:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.826+0000 7fca1886b700 1 -- 172.21.15.89:0/1410511476 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0400bd40 con 0x7fca140b16d0 2023-10-10T20:40:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.827+0000 7fca1a86f700 1 -- 172.21.15.89:0/1410511476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca140b16d0 msgr2=0x7fca140b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:09.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.827+0000 7fca1a86f700 1 --2- 172.21.15.89:0/1410511476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca140b16d0 0x7fca140b1b40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fca040099b0 tx=0x7fca04009250 comp rx=0 tx=0).stop 2023-10-10T20:40:09.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.827+0000 7fca1a86f700 1 -- 172.21.15.89:0/1410511476 shutdown_connections 2023-10-10T20:40:09.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.827+0000 7fca1a86f700 1 --2- 172.21.15.89:0/1410511476 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fca140b33b0 0x7fca140b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.827+0000 7fca1a86f700 1 --2- 172.21.15.89:0/1410511476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca140b16d0 0x7fca140b1b40 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.827+0000 7fca1a86f700 1 -- 172.21.15.89:0/1410511476 >> 172.21.15.89:0/1410511476 conn(0x7fca140d10c0 msgr2=0x7fca140affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:09.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.827+0000 7fca1a86f700 1 -- 172.21.15.89:0/1410511476 shutdown_connections 2023-10-10T20:40:09.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.827+0000 7fca1a86f700 1 -- 172.21.15.89:0/1410511476 wait complete. 2023-10-10T20:40:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.828+0000 7fca1a86f700 1 Processor -- start 2023-10-10T20:40:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.828+0000 7fca1a86f700 1 -- start start 2023-10-10T20:40:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca1a86f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fca140b16d0 0x7fca1416d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca1a86f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca140b33b0 0x7fca1416dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca1a86f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca140b4e70 0x7fca1416a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca1a86f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1416bc30 con 0x7fca140b33b0 2023-10-10T20:40:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca1a86f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca140c90e0 con 0x7fca140b4e70 2023-10-10T20:40:09.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca1a86f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1416b300 con 0x7fca140b16d0 2023-10-10T20:40:09.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca1906c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca140b33b0 0x7fca1416dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:09.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca1906c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca140b33b0 0x7fca1416dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43356/0 (socket says 172.21.15.89:43356) 2023-10-10T20:40:09.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca1906c700 1 -- 172.21.15.89:0/3231415854 learned_addr learned my addr 172.21.15.89:0/3231415854 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:09.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca1986d700 1 --2- 172.21.15.89:0/3231415854 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fca140b16d0 0x7fca1416d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:09.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca1a06e700 1 --2- 172.21.15.89:0/3231415854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca140b4e70 0x7fca1416a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:09.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca1906c700 1 -- 172.21.15.89:0/3231415854 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fca140b16d0 msgr2=0x7fca1416d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:09.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca1906c700 1 --2- 172.21.15.89:0/3231415854 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fca140b16d0 0x7fca1416d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca1906c700 1 -- 172.21.15.89:0/3231415854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca140b4e70 msgr2=0x7fca1416a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:09.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca1906c700 1 --2- 172.21.15.89:0/3231415854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca140b4e70 0x7fca1416a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:09.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca1906c700 1 -- 172.21.15.89:0/3231415854 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca040094f0 con 0x7fca140b33b0 2023-10-10T20:40:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca1a06e700 1 --2- 172.21.15.89:0/3231415854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca140b4e70 0x7fca1416a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca1906c700 1 --2- 172.21.15.89:0/3231415854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca140b33b0 0x7fca1416dbd0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fca14070dd0 tx=0x7fca04016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca09ffb700 1 -- 172.21.15.89:0/3231415854 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca04011da0 con 0x7fca140b33b0 2023-10-10T20:40:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca09ffb700 1 -- 172.21.15.89:0/3231415854 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0401f070 con 0x7fca140b33b0 2023-10-10T20:40:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca09ffb700 1 -- 172.21.15.89:0/3231415854 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca040266d0 con 0x7fca140b33b0 2023-10-10T20:40:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.829+0000 7fca1a86f700 1 -- 172.21.15.89:0/3231415854 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca1416b5e0 con 0x7fca140b33b0 2023-10-10T20:40:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.830+0000 7fca1a86f700 1 -- 172.21.15.89:0/3231415854 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca14147a00 con 0x7fca140b33b0 2023-10-10T20:40:09.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.830+0000 7fca1a86f700 1 -- 172.21.15.89:0/3231415854 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca1416bda0 con 0x7fca140b33b0 2023-10-10T20:40:09.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.831+0000 7fca09ffb700 1 -- 172.21.15.89:0/3231415854 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fca0401a440 con 0x7fca140b33b0 2023-10-10T20:40:09.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.831+0000 7fca09ffb700 1 --2- 172.21.15.89:0/3231415854 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc9f8067200 0x7fc9f80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:09.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.831+0000 7fca09ffb700 1 -- 172.21.15.89:0/3231415854 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fca04021070 con 0x7fca140b33b0 2023-10-10T20:40:09.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.831+0000 7fca1986d700 1 --2- 172.21.15.89:0/3231415854 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc9f8067200 0x7fc9f80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:09.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.832+0000 7fca1986d700 1 --2- 172.21.15.89:0/3231415854 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc9f8067200 0x7fc9f80696b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fca10006fd0 tx=0x7fca100090f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:09.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:09.835+0000 7fca09ffb700 1 -- 172.21.15.89:0/3231415854 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca04053860 con 0x7fca140b33b0 2023-10-10T20:40:10.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:10.065+0000 7fca1a86f700 1 -- 172.21.15.89:0/3231415854 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fca14147bf0 con 0x7fca140b33b0 2023-10-10T20:40:10.724 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-10-10T20:40:11.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.040+0000 7fca09ffb700 1 -- 172.21.15.89:0/3231415854 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) v1 ==== 118+0+0 (secure 0 0 0) 0x7fca0402b020 con 0x7fca140b33b0 2023-10-10T20:40:11.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.043+0000 7fca1a86f700 1 -- 172.21.15.89:0/3231415854 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc9f8067200 msgr2=0x7fc9f80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:11.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.043+0000 7fca1a86f700 1 --2- 172.21.15.89:0/3231415854 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc9f8067200 0x7fc9f80696b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fca10006fd0 tx=0x7fca100090f0 comp rx=0 tx=0).stop 2023-10-10T20:40:11.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.043+0000 7fca1a86f700 1 -- 172.21.15.89:0/3231415854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca140b33b0 msgr2=0x7fca1416dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:11.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.043+0000 7fca1a86f700 1 --2- 172.21.15.89:0/3231415854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca140b33b0 0x7fca1416dbd0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fca14070dd0 tx=0x7fca04016000 comp rx=0 tx=0).stop 2023-10-10T20:40:11.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.043+0000 7fca1a86f700 1 -- 172.21.15.89:0/3231415854 shutdown_connections 2023-10-10T20:40:11.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.043+0000 7fca1a86f700 1 --2- 172.21.15.89:0/3231415854 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc9f8067200 0x7fc9f80696b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:11.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.043+0000 7fca1a86f700 1 --2- 172.21.15.89:0/3231415854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca140b4e70 0x7fca1416a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:11.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.043+0000 7fca1a86f700 1 --2- 172.21.15.89:0/3231415854 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fca140b16d0 0x7fca1416d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:11.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.043+0000 7fca1a86f700 1 --2- 172.21.15.89:0/3231415854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca140b33b0 0x7fca1416dbd0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:11.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.043+0000 7fca1a86f700 1 -- 172.21.15.89:0/3231415854 >> 172.21.15.89:0/3231415854 conn(0x7fca140d10c0 msgr2=0x7fca14159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:11.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.043+0000 7fca1a86f700 1 -- 172.21.15.89:0/3231415854 shutdown_connections 2023-10-10T20:40:11.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.043+0000 7fca1a86f700 1 -- 172.21.15.89:0/3231415854 wait complete. 2023-10-10T20:40:11.059 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:40:11.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.180+0000 7f53ee147700 1 Processor -- start 2023-10-10T20:40:11.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.180+0000 7f53ee147700 1 -- start start 2023-10-10T20:40:11.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.181+0000 7f53ee147700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f53e80b4e70 0x7f53e80ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:11.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.181+0000 7f53ee147700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f53e80b16d0 0x7f53e80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:11.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.181+0000 7f53ee147700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e80b2080 con 0x7f53e815d240 2023-10-10T20:40:11.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.181+0000 7f53ee147700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e80cfb00 con 0x7f53e80b4e70 2023-10-10T20:40:11.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.181+0000 7f53ee147700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e80b5c40 con 0x7f53e80b16d0 2023-10-10T20:40:11.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.181+0000 7f53ed145700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f53e80b4e70 0x7f53e80ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:11.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.181+0000 7f53ed145700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f53e80b4e70 0x7f53e80ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43366/0 (socket says 172.21.15.89:43366) 2023-10-10T20:40:11.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.181+0000 7f53ed145700 1 -- 172.21.15.89:0/365826005 learned_addr learned my addr 172.21.15.89:0/365826005 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:11.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.181+0000 7f53ec944700 1 --2- 172.21.15.89:0/365826005 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f53e80b16d0 0x7f53e80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:11.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.181+0000 7f53ed145700 1 -- 172.21.15.89:0/365826005 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f53e80b16d0 msgr2=0x7f53e80b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:11.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.181+0000 7f53ed145700 1 --2- 172.21.15.89:0/365826005 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f53e80b16d0 0x7f53e80b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:11.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.181+0000 7f53ed145700 1 -- 172.21.15.89:0/365826005 >> v1:172.21.15.111:6789/0 conn(0x7f53e815d240 legacy=0x7f53e815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:11.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.181+0000 7f53ed145700 1 -- 172.21.15.89:0/365826005 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e80b5f20 con 0x7f53e80b4e70 2023-10-10T20:40:11.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.181+0000 7f53ec944700 1 --2- 172.21.15.89:0/365826005 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f53e80b16d0 0x7f53e80b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:11.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.181+0000 7f53ed145700 1 --2- 172.21.15.89:0/365826005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e80b4e70 0x7f53e80ccff0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009f20 tx=0x7f53e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5930e4734d0a85f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:11.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.182+0000 7f53e7fff700 1 -- 172.21.15.89:0/365826005 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e000e070 con 0x7f53e80b4e70 2023-10-10T20:40:11.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.182+0000 7f53e7fff700 1 -- 172.21.15.89:0/365826005 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e000ba40 con 0x7f53e80b4e70 2023-10-10T20:40:11.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.182+0000 7f53e7fff700 1 -- 172.21.15.89:0/365826005 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e00092e0 con 0x7f53e80b4e70 2023-10-10T20:40:11.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.182+0000 7f53ee147700 1 -- 172.21.15.89:0/365826005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e80b4e70 msgr2=0x7f53e80ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:11.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.182+0000 7f53ee147700 1 --2- 172.21.15.89:0/365826005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e80b4e70 0x7f53e80ccff0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009f20 tx=0x7f53e0009840 comp rx=0 tx=0).stop 2023-10-10T20:40:11.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.182+0000 7f53ee147700 1 -- 172.21.15.89:0/365826005 shutdown_connections 2023-10-10T20:40:11.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.182+0000 7f53ee147700 1 --2- 172.21.15.89:0/365826005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e80b4e70 0x7f53e80ccff0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:11.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.182+0000 7f53ee147700 1 --2- 172.21.15.89:0/365826005 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f53e80b16d0 0x7f53e80b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:11.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.182+0000 7f53ee147700 1 -- 172.21.15.89:0/365826005 >> 172.21.15.89:0/365826005 conn(0x7f53e80d10c0 msgr2=0x7f53e80affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:11.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.182+0000 7f53ee147700 1 -- 172.21.15.89:0/365826005 shutdown_connections 2023-10-10T20:40:11.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.182+0000 7f53ee147700 1 -- 172.21.15.89:0/365826005 wait complete. 2023-10-10T20:40:11.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.183+0000 7f53ee147700 1 Processor -- start 2023-10-10T20:40:11.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.183+0000 7f53ee147700 1 -- start start 2023-10-10T20:40:11.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.184+0000 7f53ee147700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53e80b16d0 0x7f53e80c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:11.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.184+0000 7f53ee147700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e80b4e70 0x7f53e8148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:11.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.184+0000 7f53ee147700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f53e815d240 0x7f53e80ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:11.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.184+0000 7f53ee147700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e80c9520 con 0x7f53e80b4e70 2023-10-10T20:40:11.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.184+0000 7f53ee147700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e80c9ef0 con 0x7f53e80b16d0 2023-10-10T20:40:11.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.184+0000 7f53ee147700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e80c8910 con 0x7f53e815d240 2023-10-10T20:40:11.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.184+0000 7f53ec944700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e80b4e70 0x7f53e8148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:11.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.184+0000 7f53ec944700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e80b4e70 0x7f53e8148f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43380/0 (socket says 172.21.15.89:43380) 2023-10-10T20:40:11.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.184+0000 7f53ec944700 1 -- 172.21.15.89:0/3333920376 learned_addr learned my addr 172.21.15.89:0/3333920376 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:11.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.185+0000 7f53ed145700 1 --2- 172.21.15.89:0/3333920376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53e80b16d0 0x7f53e80c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:11.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.185+0000 7f53ed946700 1 --2- 172.21.15.89:0/3333920376 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f53e815d240 0x7f53e80ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:11.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.185+0000 7f53ec944700 1 -- 172.21.15.89:0/3333920376 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f53e815d240 msgr2=0x7f53e80ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:11.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.185+0000 7f53ec944700 1 --2- 172.21.15.89:0/3333920376 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f53e815d240 0x7f53e80ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:11.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.185+0000 7f53ec944700 1 -- 172.21.15.89:0/3333920376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53e80b16d0 msgr2=0x7f53e80c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:11.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.185+0000 7f53ec944700 1 --2- 172.21.15.89:0/3333920376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53e80b16d0 0x7f53e80c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:11.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.185+0000 7f53ec944700 1 -- 172.21.15.89:0/3333920376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e0009ae0 con 0x7f53e80b4e70 2023-10-10T20:40:11.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.185+0000 7f53ec944700 1 --2- 172.21.15.89:0/3333920376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e80b4e70 0x7f53e8148f00 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00c830 tx=0x7f53dc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:11.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.185+0000 7f53e57fa700 1 -- 172.21.15.89:0/3333920376 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc011600 con 0x7f53e80b4e70 2023-10-10T20:40:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.185+0000 7f53e57fa700 1 -- 172.21.15.89:0/3333920376 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc011760 con 0x7f53e80b4e70 2023-10-10T20:40:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.185+0000 7f53e57fa700 1 -- 172.21.15.89:0/3333920376 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc011b20 con 0x7f53e80b4e70 2023-10-10T20:40:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.185+0000 7f53ee147700 1 -- 172.21.15.89:0/3333920376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e80c8bf0 con 0x7f53e80b4e70 2023-10-10T20:40:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.185+0000 7f53ee147700 1 -- 172.21.15.89:0/3333920376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e80c91b0 con 0x7f53e80b4e70 2023-10-10T20:40:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.186+0000 7f53ee147700 1 -- 172.21.15.89:0/3333920376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e80c96d0 con 0x7f53e80b4e70 2023-10-10T20:40:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.186+0000 7f53e57fa700 1 -- 172.21.15.89:0/3333920376 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f53dc00a3f0 con 0x7f53e80b4e70 2023-10-10T20:40:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.187+0000 7f53e57fa700 1 --2- 172.21.15.89:0/3333920376 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f53d4067200 0x7f53d40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.187+0000 7f53e57fa700 1 -- 172.21.15.89:0/3333920376 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f53dc084b60 con 0x7f53e80b4e70 2023-10-10T20:40:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.187+0000 7f53ed145700 1 --2- 172.21.15.89:0/3333920376 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f53d4067200 0x7f53d40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.187+0000 7f53ed145700 1 --2- 172.21.15.89:0/3333920376 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f53d4067200 0x7f53d40696b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009880 tx=0x7f53e0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:11.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.191+0000 7f53e57fa700 1 -- 172.21.15.89:0/3333920376 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53e80c96d0 con 0x7f53e80b4e70 2023-10-10T20:40:11.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:11.415+0000 7f53ee147700 1 -- 172.21.15.89:0/3333920376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f53e80c9660 con 0x7f53e80b4e70 2023-10-10T20:40:12.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.041+0000 7f53e57fa700 1 -- 172.21.15.89:0/3333920376 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v55) v1 ==== 122+0+0 (secure 0 0 0) 0x7f53dc0150c0 con 0x7f53e80b4e70 2023-10-10T20:40:12.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.043+0000 7f53ee147700 1 -- 172.21.15.89:0/3333920376 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f53d4067200 msgr2=0x7f53d40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:12.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.043+0000 7f53ee147700 1 --2- 172.21.15.89:0/3333920376 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f53d4067200 0x7f53d40696b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009880 tx=0x7f53e0017000 comp rx=0 tx=0).stop 2023-10-10T20:40:12.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.043+0000 7f53ee147700 1 -- 172.21.15.89:0/3333920376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e80b4e70 msgr2=0x7f53e8148f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:12.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.043+0000 7f53ee147700 1 --2- 172.21.15.89:0/3333920376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e80b4e70 0x7f53e8148f00 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00c830 tx=0x7f53dc008040 comp rx=0 tx=0).stop 2023-10-10T20:40:12.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.044+0000 7f53ee147700 1 -- 172.21.15.89:0/3333920376 shutdown_connections 2023-10-10T20:40:12.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.044+0000 7f53ee147700 1 --2- 172.21.15.89:0/3333920376 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f53e815d240 0x7f53e80ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.044+0000 7f53ee147700 1 --2- 172.21.15.89:0/3333920376 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f53d4067200 0x7f53d40696b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.044+0000 7f53ee147700 1 --2- 172.21.15.89:0/3333920376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53e80b16d0 0x7f53e80c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.044+0000 7f53ee147700 1 --2- 172.21.15.89:0/3333920376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e80b4e70 0x7f53e8148f00 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.044+0000 7f53ee147700 1 -- 172.21.15.89:0/3333920376 >> 172.21.15.89:0/3333920376 conn(0x7f53e80d10c0 msgr2=0x7f53e80b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:12.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.044+0000 7f53ee147700 1 -- 172.21.15.89:0/3333920376 shutdown_connections 2023-10-10T20:40:12.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.044+0000 7f53ee147700 1 -- 172.21.15.89:0/3333920376 wait complete. 2023-10-10T20:40:12.059 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:40:12.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.174+0000 7f12d8955700 1 Processor -- start 2023-10-10T20:40:12.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.174+0000 7f12d8955700 1 -- start start 2023-10-10T20:40:12.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.175+0000 7f12d8955700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f12d4159890 0x7f12d415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:12.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.175+0000 7f12d8955700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f12d4159c60 0x7f12d415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:12.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.175+0000 7f12d8955700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d415a160 con 0x7f12d415db10 2023-10-10T20:40:12.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.175+0000 7f12d8955700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d40d13a0 con 0x7f12d4159890 2023-10-10T20:40:12.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.175+0000 7f12d8955700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d415e140 con 0x7f12d4159c60 2023-10-10T20:40:12.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.175+0000 7f12d259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f12d4159890 0x7f12d415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:12.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.175+0000 7f12d259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f12d4159890 0x7f12d415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43384/0 (socket says 172.21.15.89:43384) 2023-10-10T20:40:12.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.175+0000 7f12d2d9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f12d415db10 0x7f12d4069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55724/0 (socket says 172.21.15.89:55724) 2023-10-10T20:40:12.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.175+0000 7f12d259c700 1 -- 172.21.15.89:0/1228128335 learned_addr learned my addr 172.21.15.89:0/1228128335 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:12.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.175+0000 7f12d1d9b700 1 --2- 172.21.15.89:0/1228128335 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f12d4159c60 0x7f12d415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:12.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.175+0000 7f12d259c700 1 -- 172.21.15.89:0/1228128335 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f12d4159c60 msgr2=0x7f12d415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:12.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.175+0000 7f12d259c700 1 --2- 172.21.15.89:0/1228128335 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f12d4159c60 0x7f12d415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.175+0000 7f12d259c700 1 -- 172.21.15.89:0/1228128335 >> v1:172.21.15.89:6789/0 conn(0x7f12d415db10 legacy=0x7f12d4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:12.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.175+0000 7f12d259c700 1 -- 172.21.15.89:0/1228128335 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d415a3d0 con 0x7f12d4159890 2023-10-10T20:40:12.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.175+0000 7f12d159a700 1 -- 172.21.15.89:0/1228128335 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2464938864 0 0) 0x7f12d415a160 con 0x7f12d415db10 2023-10-10T20:40:12.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.175+0000 7f12d259c700 1 --2- 172.21.15.89:0/1228128335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12d4159890 0x7f12d415abd0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f12c40099b0 tx=0x7f12c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a0703987d195ab server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:12.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.176+0000 7f12d159a700 1 -- 172.21.15.89:0/1228128335 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4009ea0 con 0x7f12d4159890 2023-10-10T20:40:12.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.176+0000 7f12d159a700 1 -- 172.21.15.89:0/1228128335 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c400b860 con 0x7f12d4159890 2023-10-10T20:40:12.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.176+0000 7f12d159a700 1 -- 172.21.15.89:0/1228128335 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c400bc00 con 0x7f12d4159890 2023-10-10T20:40:12.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.176+0000 7f12d8955700 1 -- 172.21.15.89:0/1228128335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12d4159890 msgr2=0x7f12d415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:12.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.176+0000 7f12d8955700 1 --2- 172.21.15.89:0/1228128335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12d4159890 0x7f12d415abd0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f12c40099b0 tx=0x7f12c4009250 comp rx=0 tx=0).stop 2023-10-10T20:40:12.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.176+0000 7f12d8955700 1 -- 172.21.15.89:0/1228128335 shutdown_connections 2023-10-10T20:40:12.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.176+0000 7f12d8955700 1 --2- 172.21.15.89:0/1228128335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12d4159890 0x7f12d415abd0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.176+0000 7f12d8955700 1 --2- 172.21.15.89:0/1228128335 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f12d4159c60 0x7f12d415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.176+0000 7f12d8955700 1 -- 172.21.15.89:0/1228128335 >> 172.21.15.89:0/1228128335 conn(0x7f12d4068e20 msgr2=0x7f12d40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:12.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.176+0000 7f12d8955700 1 -- 172.21.15.89:0/1228128335 shutdown_connections 2023-10-10T20:40:12.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.176+0000 7f12d8955700 1 -- 172.21.15.89:0/1228128335 wait complete. 2023-10-10T20:40:12.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.177+0000 7f12d8955700 1 Processor -- start 2023-10-10T20:40:12.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.177+0000 7f12d8955700 1 -- start start 2023-10-10T20:40:12.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.178+0000 7f12d8955700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12d4159890 0x7f12d40cf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:12.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.178+0000 7f12d8955700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f12d4159c60 0x7f12d40c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:12.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.178+0000 7f12d8955700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12d415db10 0x7f12d40cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:12.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.178+0000 7f12d8955700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40ce140 con 0x7f12d4159890 2023-10-10T20:40:12.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.178+0000 7f12d8955700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40b36c0 con 0x7f12d415db10 2023-10-10T20:40:12.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.178+0000 7f12d8955700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40cd600 con 0x7f12d4159c60 2023-10-10T20:40:12.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.178+0000 7f12d259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12d4159890 0x7f12d40cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:12.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.178+0000 7f12d259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12d4159890 0x7f12d40cf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43388/0 (socket says 172.21.15.89:43388) 2023-10-10T20:40:12.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.178+0000 7f12d259c700 1 -- 172.21.15.89:0/1044746698 learned_addr learned my addr 172.21.15.89:0/1044746698 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:12.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.178+0000 7f12d259c700 1 -- 172.21.15.89:0/1044746698 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f12d4159c60 msgr2=0x7f12d40c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:12.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.178+0000 7f12d259c700 1 --2- 172.21.15.89:0/1044746698 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f12d4159c60 0x7f12d40c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.178+0000 7f12d259c700 1 -- 172.21.15.89:0/1044746698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12d415db10 msgr2=0x7f12d40cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:12.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.178+0000 7f12d259c700 1 --2- 172.21.15.89:0/1044746698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12d415db10 0x7f12d40cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.178+0000 7f12d259c700 1 -- 172.21.15.89:0/1044746698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c40094f0 con 0x7f12d4159890 2023-10-10T20:40:12.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.178+0000 7f12d259c700 1 --2- 172.21.15.89:0/1044746698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12d4159890 0x7f12d40cf120 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f12c4006fd0 tx=0x7f12c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:12.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.179+0000 7f12c27fc700 1 -- 172.21.15.89:0/1044746698 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c400bbe0 con 0x7f12d4159890 2023-10-10T20:40:12.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.179+0000 7f12c27fc700 1 -- 172.21.15.89:0/1044746698 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c4011420 con 0x7f12d4159890 2023-10-10T20:40:12.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.179+0000 7f12d8955700 1 -- 172.21.15.89:0/1044746698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d40cd8a0 con 0x7f12d4159890 2023-10-10T20:40:12.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.179+0000 7f12c27fc700 1 -- 172.21.15.89:0/1044746698 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c40117e0 con 0x7f12d4159890 2023-10-10T20:40:12.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.179+0000 7f12d8955700 1 -- 172.21.15.89:0/1044746698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d40cde60 con 0x7f12d4159890 2023-10-10T20:40:12.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.179+0000 7f12d8955700 1 -- 172.21.15.89:0/1044746698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d404c3c0 con 0x7f12d4159890 2023-10-10T20:40:12.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.180+0000 7f12c27fc700 1 -- 172.21.15.89:0/1044746698 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f12c4011bd0 con 0x7f12d4159890 2023-10-10T20:40:12.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.180+0000 7f12c27fc700 1 --2- 172.21.15.89:0/1044746698 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f12b4067200 0x7f12b40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:12.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.180+0000 7f12c27fc700 1 -- 172.21.15.89:0/1044746698 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f12c4086280 con 0x7f12d4159890 2023-10-10T20:40:12.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.180+0000 7f12d1d9b700 1 --2- 172.21.15.89:0/1044746698 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f12b4067200 0x7f12b40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:12.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.181+0000 7f12d1d9b700 1 --2- 172.21.15.89:0/1044746698 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f12b4067200 0x7f12b40696b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f12d40c7440 tx=0x7f12c80091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:12.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.185+0000 7f12c27fc700 1 -- 172.21.15.89:0/1044746698 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12c40531a0 con 0x7f12d4159890 2023-10-10T20:40:12.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.411+0000 7f12d8955700 1 -- 172.21.15.89:0/1044746698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12d40d1040 con 0x7f12d4159890 2023-10-10T20:40:12.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.412+0000 7f12c27fc700 1 -- 172.21.15.89:0/1044746698 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f12c40567c0 con 0x7f12d4159890 2023-10-10T20:40:12.414 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:12.414 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":55,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5995,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/3194805494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3194805494},{"type":"v1","addr":"172.21.15.111:6834","nonce":3194805494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6447,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/2766962252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2766962252},{"type":"v1","addr":"172.21.15.89:6836","nonce":2766962252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6450,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1141782720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1141782720},{"type":"v1","addr":"172.21.15.89:6837","nonce":1141782720}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:40:08.387849+0000","modified":"2023-10-10T20:40:12.040215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6462":{"gid":6462,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/4069200228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":4069200228},{"type":"v1","addr":"172.21.15.111:6835","nonce":4069200228}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-10T20:40:12.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.414+0000 7f12d8955700 1 -- 172.21.15.89:0/1044746698 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f12b4067200 msgr2=0x7f12b40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:12.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.414+0000 7f12d8955700 1 --2- 172.21.15.89:0/1044746698 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f12b4067200 0x7f12b40696b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f12d40c7440 tx=0x7f12c80091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.414+0000 7f12d8955700 1 -- 172.21.15.89:0/1044746698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12d4159890 msgr2=0x7f12d40cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:12.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.414+0000 7f12d8955700 1 --2- 172.21.15.89:0/1044746698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12d4159890 0x7f12d40cf120 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f12c4006fd0 tx=0x7f12c4017000 comp rx=0 tx=0).stop 2023-10-10T20:40:12.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.414+0000 7f12d8955700 1 -- 172.21.15.89:0/1044746698 shutdown_connections 2023-10-10T20:40:12.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.414+0000 7f12d8955700 1 --2- 172.21.15.89:0/1044746698 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f12b4067200 0x7f12b40696b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.414+0000 7f12d8955700 1 --2- 172.21.15.89:0/1044746698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12d415db10 0x7f12d40cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.414+0000 7f12d8955700 1 --2- 172.21.15.89:0/1044746698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12d4159890 0x7f12d40cf120 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.414+0000 7f12d8955700 1 --2- 172.21.15.89:0/1044746698 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f12d4159c60 0x7f12d40c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.414+0000 7f12d8955700 1 -- 172.21.15.89:0/1044746698 >> 172.21.15.89:0/1044746698 conn(0x7f12d4068e20 msgr2=0x7f12d414d680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:12.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.414+0000 7f12d8955700 1 -- 172.21.15.89:0/1044746698 shutdown_connections 2023-10-10T20:40:12.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.414+0000 7f12d8955700 1 -- 172.21.15.89:0/1044746698 wait complete. 2023-10-10T20:40:12.419 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 55 2023-10-10T20:40:12.428 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:40:12.494 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:40:12.497 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:40:12.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.543+0000 7f49f19d0700 1 Processor -- start 2023-10-10T20:40:12.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.543+0000 7f49f19d0700 1 -- start start 2023-10-10T20:40:12.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.544+0000 7f49f19d0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f49ec159890 0x7f49ec15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:12.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.544+0000 7f49f19d0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f49ec159c60 0x7f49ec15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:12.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.544+0000 7f49f19d0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49ec15a160 con 0x7f49ec15db10 2023-10-10T20:40:12.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.544+0000 7f49f19d0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ec0d13a0 con 0x7f49ec159c60 2023-10-10T20:40:12.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.544+0000 7f49f19d0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec15e140 con 0x7f49ec159890 2023-10-10T20:40:12.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.544+0000 7f49e27fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f49ec159c60 0x7f49ec15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:12.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.544+0000 7f49eb7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f49ec15db10 0x7f49ec069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55730/0 (socket says 172.21.15.89:55730) 2023-10-10T20:40:12.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.544+0000 7f49eb7fe700 1 -- 172.21.15.89:0/2495936927 learned_addr learned my addr 172.21.15.89:0/2495936927 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:12.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.544+0000 7f49eaffd700 1 --2- 172.21.15.89:0/2495936927 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f49ec159890 0x7f49ec15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:12.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.544+0000 7f49e27fc700 1 -- 172.21.15.89:0/2495936927 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f49ec159890 msgr2=0x7f49ec15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:12.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.544+0000 7f49e27fc700 1 --2- 172.21.15.89:0/2495936927 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f49ec159890 0x7f49ec15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.544+0000 7f49e27fc700 1 -- 172.21.15.89:0/2495936927 >> v1:172.21.15.89:6789/0 conn(0x7f49ec15db10 legacy=0x7f49ec069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:12.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.544+0000 7f49e27fc700 1 -- 172.21.15.89:0/2495936927 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec15a3d0 con 0x7f49ec159c60 2023-10-10T20:40:12.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.544+0000 7f49ea7fc700 1 -- 172.21.15.89:0/2495936927 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3786362111 0 0) 0x7f49ec15a160 con 0x7f49ec15db10 2023-10-10T20:40:12.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.544+0000 7f49e27fc700 1 --2- 172.21.15.89:0/2495936927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49ec159c60 0x7f49ec15d5d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f49d40098e0 tx=0x7f49d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5316d01eb1192024 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:12.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.545+0000 7f49ea7fc700 1 -- 172.21.15.89:0/2495936927 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d400b640 con 0x7f49ec159c60 2023-10-10T20:40:12.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.545+0000 7f49ea7fc700 1 -- 172.21.15.89:0/2495936927 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d400b7a0 con 0x7f49ec159c60 2023-10-10T20:40:12.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.545+0000 7f49ea7fc700 1 -- 172.21.15.89:0/2495936927 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d400bb40 con 0x7f49ec159c60 2023-10-10T20:40:12.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.545+0000 7f49f19d0700 1 -- 172.21.15.89:0/2495936927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49ec159c60 msgr2=0x7f49ec15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:12.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.545+0000 7f49f19d0700 1 --2- 172.21.15.89:0/2495936927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49ec159c60 0x7f49ec15d5d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f49d40098e0 tx=0x7f49d40091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.545+0000 7f49f19d0700 1 -- 172.21.15.89:0/2495936927 shutdown_connections 2023-10-10T20:40:12.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.545+0000 7f49f19d0700 1 --2- 172.21.15.89:0/2495936927 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f49ec159890 0x7f49ec15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.545+0000 7f49f19d0700 1 --2- 172.21.15.89:0/2495936927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49ec159c60 0x7f49ec15d5d0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.545+0000 7f49f19d0700 1 -- 172.21.15.89:0/2495936927 >> 172.21.15.89:0/2495936927 conn(0x7f49ec068e20 msgr2=0x7f49ec0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:12.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.545+0000 7f49f19d0700 1 -- 172.21.15.89:0/2495936927 shutdown_connections 2023-10-10T20:40:12.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.545+0000 7f49f19d0700 1 -- 172.21.15.89:0/2495936927 wait complete. 2023-10-10T20:40:12.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.547+0000 7f49f19d0700 1 Processor -- start 2023-10-10T20:40:12.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.547+0000 7f49f19d0700 1 -- start start 2023-10-10T20:40:12.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.547+0000 7f49f19d0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49ec159890 0x7f49ec0c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:12.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.547+0000 7f49f19d0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49ec159c60 0x7f49ec149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:12.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.547+0000 7f49f19d0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f49ec15db10 0x7f49ec148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:12.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.547+0000 7f49f19d0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec0c6160 con 0x7f49ec159c60 2023-10-10T20:40:12.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.547+0000 7f49f19d0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec0c6440 con 0x7f49ec159890 2023-10-10T20:40:12.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.547+0000 7f49f19d0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec0c6720 con 0x7f49ec15db10 2023-10-10T20:40:12.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.547+0000 7f49e27fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49ec159c60 0x7f49ec149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:12.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.547+0000 7f49e27fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49ec159c60 0x7f49ec149ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43400/0 (socket says 172.21.15.89:43400) 2023-10-10T20:40:12.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.547+0000 7f49e27fc700 1 -- 172.21.15.89:0/1369021173 learned_addr learned my addr 172.21.15.89:0/1369021173 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:12.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.548+0000 7f49eb7fe700 1 --2- 172.21.15.89:0/1369021173 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f49ec15db10 0x7f49ec148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:12.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.548+0000 7f49eaffd700 1 --2- 172.21.15.89:0/1369021173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49ec159890 0x7f49ec0c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:12.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.548+0000 7f49e27fc700 1 -- 172.21.15.89:0/1369021173 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f49ec15db10 msgr2=0x7f49ec148fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:12.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.548+0000 7f49e27fc700 1 --2- 172.21.15.89:0/1369021173 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f49ec15db10 0x7f49ec148fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.548+0000 7f49e27fc700 1 -- 172.21.15.89:0/1369021173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49ec159890 msgr2=0x7f49ec0c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:12.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.548+0000 7f49e27fc700 1 --2- 172.21.15.89:0/1369021173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49ec159890 0x7f49ec0c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.548+0000 7f49e27fc700 1 -- 172.21.15.89:0/1369021173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d4009420 con 0x7f49ec159c60 2023-10-10T20:40:12.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.548+0000 7f49eaffd700 1 --2- 172.21.15.89:0/1369021173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49ec159890 0x7f49ec0c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:12.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.548+0000 7f49e27fc700 1 --2- 172.21.15.89:0/1369021173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49ec159c60 0x7f49ec149ec0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f49d4013000 tx=0x7f49d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:12.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.548+0000 7f49e37fe700 1 -- 172.21.15.89:0/1369021173 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d400bb20 con 0x7f49ec159c60 2023-10-10T20:40:12.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.548+0000 7f49e37fe700 1 -- 172.21.15.89:0/1369021173 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d4011420 con 0x7f49ec159c60 2023-10-10T20:40:12.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.548+0000 7f49f19d0700 1 -- 172.21.15.89:0/1369021173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ec0c6a00 con 0x7f49ec159c60 2023-10-10T20:40:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.548+0000 7f49e37fe700 1 -- 172.21.15.89:0/1369021173 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d40117c0 con 0x7f49ec159c60 2023-10-10T20:40:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.548+0000 7f49f19d0700 1 -- 172.21.15.89:0/1369021173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ec0c5d70 con 0x7f49ec159c60 2023-10-10T20:40:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.549+0000 7f49f19d0700 1 -- 172.21.15.89:0/1369021173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ec0893c0 con 0x7f49ec159c60 2023-10-10T20:40:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.549+0000 7f49e37fe700 1 -- 172.21.15.89:0/1369021173 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f49d4011b70 con 0x7f49ec159c60 2023-10-10T20:40:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.550+0000 7f49e37fe700 1 --2- 172.21.15.89:0/1369021173 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f49cc067200 0x7f49cc0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:12.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.550+0000 7f49e37fe700 1 -- 172.21.15.89:0/1369021173 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f49d4086030 con 0x7f49ec159c60 2023-10-10T20:40:12.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.550+0000 7f49eaffd700 1 --2- 172.21.15.89:0/1369021173 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f49cc067200 0x7f49cc0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:12.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.550+0000 7f49eaffd700 1 --2- 172.21.15.89:0/1369021173 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f49cc067200 0x7f49cc0696b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f49dc006f00 tx=0x7f49dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:12.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.554+0000 7f49e37fe700 1 -- 172.21.15.89:0/1369021173 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49d4052f50 con 0x7f49ec159c60 2023-10-10T20:40:12.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.737+0000 7f49f19d0700 1 -- 172.21.15.89:0/1369021173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49ec0890e0 con 0x7f49ec159c60 2023-10-10T20:40:12.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.738+0000 7f49e37fe700 1 -- 172.21.15.89:0/1369021173 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+15353 (secure 0 0 0) 0x7f49d4056570 con 0x7f49ec159c60 2023-10-10T20:40:12.740 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:12.740 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":88,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:40:10.022096+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:40:06.747813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-10T20:40:07.347913+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:40:12.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.740+0000 7f49f19d0700 1 -- 172.21.15.89:0/1369021173 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f49cc067200 msgr2=0x7f49cc0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:12.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.740+0000 7f49f19d0700 1 --2- 172.21.15.89:0/1369021173 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f49cc067200 0x7f49cc0696b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f49dc006f00 tx=0x7f49dc008040 comp rx=0 tx=0).stop 2023-10-10T20:40:12.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.740+0000 7f49f19d0700 1 -- 172.21.15.89:0/1369021173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49ec159c60 msgr2=0x7f49ec149ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:12.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.740+0000 7f49f19d0700 1 --2- 172.21.15.89:0/1369021173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49ec159c60 0x7f49ec149ec0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f49d4013000 tx=0x7f49d4017000 comp rx=0 tx=0).stop 2023-10-10T20:40:12.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.740+0000 7f49f19d0700 1 -- 172.21.15.89:0/1369021173 shutdown_connections 2023-10-10T20:40:12.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.740+0000 7f49f19d0700 1 --2- 172.21.15.89:0/1369021173 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f49ec15db10 0x7f49ec148fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.740+0000 7f49f19d0700 1 --2- 172.21.15.89:0/1369021173 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f49cc067200 0x7f49cc0696b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.740+0000 7f49f19d0700 1 --2- 172.21.15.89:0/1369021173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49ec159890 0x7f49ec0c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.740+0000 7f49f19d0700 1 --2- 172.21.15.89:0/1369021173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49ec159c60 0x7f49ec149ec0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.740+0000 7f49f19d0700 1 -- 172.21.15.89:0/1369021173 >> 172.21.15.89:0/1369021173 conn(0x7f49ec068e20 msgr2=0x7f49ec15fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:12.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.740+0000 7f49f19d0700 1 -- 172.21.15.89:0/1369021173 shutdown_connections 2023-10-10T20:40:12.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.740+0000 7f49f19d0700 1 -- 172.21.15.89:0/1369021173 wait complete. 2023-10-10T20:40:12.754 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:40:12.755 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:40:12.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.906+0000 7f8b8d858700 1 Processor -- start 2023-10-10T20:40:12.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.906+0000 7f8b8d858700 1 -- start start 2023-10-10T20:40:12.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.907+0000 7f8b8d858700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b881593b0 0x7f8b88159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:12.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.907+0000 7f8b8d858700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8b880b1e50 0x7f8b880b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:12.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.907+0000 7f8b8d858700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b880b6570 con 0x7f8b880ceaa0 2023-10-10T20:40:12.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.907+0000 7f8b8d858700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b880cfe20 con 0x7f8b880b1e50 2023-10-10T20:40:12.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.907+0000 7f8b8d858700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b880cf210 con 0x7f8b881593b0 2023-10-10T20:40:12.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.907+0000 7f8b8d057700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8b880b1e50 0x7f8b880b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:12.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.907+0000 7f8b7ffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8b880ceaa0 0x7f8b880b1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55734/0 (socket says 172.21.15.89:55734) 2023-10-10T20:40:12.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.907+0000 7f8b7ffff700 1 -- 172.21.15.89:0/3279225968 learned_addr learned my addr 172.21.15.89:0/3279225968 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:12.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.907+0000 7f8b8c856700 1 --2- 172.21.15.89:0/3279225968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b881593b0 0x7f8b88159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:12.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.907+0000 7f8b7f7fe700 1 -- 172.21.15.89:0/3279225968 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989254724 0 0) 0x7f8b78001110 con 0x7f8b880ceaa0 2023-10-10T20:40:12.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.907+0000 7f8b7f7fe700 1 -- 172.21.15.89:0/3279225968 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b70003570 con 0x7f8b880ceaa0 2023-10-10T20:40:12.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.908+0000 7f8b8d057700 1 -- 172.21.15.89:0/3279225968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b881593b0 msgr2=0x7f8b88159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:12.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.908+0000 7f8b8d057700 1 --2- 172.21.15.89:0/3279225968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b881593b0 0x7f8b88159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.908+0000 7f8b8d057700 1 -- 172.21.15.89:0/3279225968 >> v1:172.21.15.89:6789/0 conn(0x7f8b880ceaa0 legacy=0x7f8b880b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:12.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.908+0000 7f8b8d057700 1 -- 172.21.15.89:0/3279225968 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b880cf4f0 con 0x7f8b880b1e50 2023-10-10T20:40:12.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.908+0000 7f8b8c856700 1 --2- 172.21.15.89:0/3279225968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b881593b0 0x7f8b88159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:12.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.908+0000 7f8b8d057700 1 --2- 172.21.15.89:0/3279225968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b880b1e50 0x7f8b880b6030 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009e60 tx=0x7f8b80009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ad50f28086257e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:12.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.908+0000 7f8b7f7fe700 1 -- 172.21.15.89:0/3279225968 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8000e070 con 0x7f8b880b1e50 2023-10-10T20:40:12.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.908+0000 7f8b7f7fe700 1 -- 172.21.15.89:0/3279225968 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8000bb60 con 0x7f8b880b1e50 2023-10-10T20:40:12.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.908+0000 7f8b7f7fe700 1 -- 172.21.15.89:0/3279225968 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80012430 con 0x7f8b880b1e50 2023-10-10T20:40:12.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.908+0000 7f8b8d858700 1 -- 172.21.15.89:0/3279225968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b880b1e50 msgr2=0x7f8b880b6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:12.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.908+0000 7f8b8d858700 1 --2- 172.21.15.89:0/3279225968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b880b1e50 0x7f8b880b6030 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009e60 tx=0x7f8b80009780 comp rx=0 tx=0).stop 2023-10-10T20:40:12.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.908+0000 7f8b8d858700 1 -- 172.21.15.89:0/3279225968 shutdown_connections 2023-10-10T20:40:12.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.908+0000 7f8b8d858700 1 --2- 172.21.15.89:0/3279225968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b880b1e50 0x7f8b880b6030 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.908+0000 7f8b8d858700 1 --2- 172.21.15.89:0/3279225968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b881593b0 0x7f8b88159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.908+0000 7f8b8d858700 1 -- 172.21.15.89:0/3279225968 >> 172.21.15.89:0/3279225968 conn(0x7f8b880d1130 msgr2=0x7f8b880b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:12.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.908+0000 7f8b8d858700 1 -- 172.21.15.89:0/3279225968 shutdown_connections 2023-10-10T20:40:12.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.908+0000 7f8b8d858700 1 -- 172.21.15.89:0/3279225968 wait complete. 2023-10-10T20:40:12.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.909+0000 7f8b8d858700 1 Processor -- start 2023-10-10T20:40:12.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.910+0000 7f8b8d858700 1 -- start start 2023-10-10T20:40:12.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.910+0000 7f8b8d858700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b880b1e50 0x7f8b880cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:12.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.910+0000 7f8b8d858700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b880ceaa0 0x7f8b88155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:12.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.910+0000 7f8b8d858700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b881593b0 0x7f8b8815b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:12.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.910+0000 7f8b8d858700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8815a750 con 0x7f8b880b1e50 2023-10-10T20:40:12.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.910+0000 7f8b8d858700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b88156160 con 0x7f8b880ceaa0 2023-10-10T20:40:12.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.910+0000 7f8b8d858700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b880cb210 con 0x7f8b881593b0 2023-10-10T20:40:12.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.910+0000 7f8b8c856700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b880b1e50 0x7f8b880cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:12.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.911+0000 7f8b8c856700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b880b1e50 0x7f8b880cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43432/0 (socket says 172.21.15.89:43432) 2023-10-10T20:40:12.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.911+0000 7f8b8c856700 1 -- 172.21.15.89:0/1122028442 learned_addr learned my addr 172.21.15.89:0/1122028442 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:12.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.911+0000 7f8b8d057700 1 --2- 172.21.15.89:0/1122028442 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b881593b0 0x7f8b8815b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:12.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.911+0000 7f8b7ffff700 1 --2- 172.21.15.89:0/1122028442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b880ceaa0 0x7f8b88155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:12.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.911+0000 7f8b8c856700 1 -- 172.21.15.89:0/1122028442 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b881593b0 msgr2=0x7f8b8815b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:12.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.911+0000 7f8b8c856700 1 --2- 172.21.15.89:0/1122028442 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b881593b0 0x7f8b8815b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.911+0000 7f8b8c856700 1 -- 172.21.15.89:0/1122028442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b880ceaa0 msgr2=0x7f8b88155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:12.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.911+0000 7f8b8c856700 1 --2- 172.21.15.89:0/1122028442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b880ceaa0 0x7f8b88155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:12.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.911+0000 7f8b8c856700 1 -- 172.21.15.89:0/1122028442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b80009a20 con 0x7f8b880b1e50 2023-10-10T20:40:12.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.911+0000 7f8b7ffff700 1 --2- 172.21.15.89:0/1122028442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b880ceaa0 0x7f8b88155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:12.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.911+0000 7f8b8c856700 1 --2- 172.21.15.89:0/1122028442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b880b1e50 0x7f8b880cab30 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400d890 tx=0x7f8b84008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:12.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.911+0000 7f8b7cff9700 1 -- 172.21.15.89:0/1122028442 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8400b600 con 0x7f8b880b1e50 2023-10-10T20:40:12.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.911+0000 7f8b7cff9700 1 -- 172.21.15.89:0/1122028442 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8400b760 con 0x7f8b880b1e50 2023-10-10T20:40:12.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.911+0000 7f8b8d858700 1 -- 172.21.15.89:0/1122028442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b880cb4f0 con 0x7f8b880b1e50 2023-10-10T20:40:12.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.911+0000 7f8b7cff9700 1 -- 172.21.15.89:0/1122028442 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8400bb00 con 0x7f8b880b1e50 2023-10-10T20:40:12.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.911+0000 7f8b8d858700 1 -- 172.21.15.89:0/1122028442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8815ac80 con 0x7f8b880b1e50 2023-10-10T20:40:12.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.912+0000 7f8b8d858700 1 -- 172.21.15.89:0/1122028442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8815aff0 con 0x7f8b880b1e50 2023-10-10T20:40:12.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.912+0000 7f8b7cff9700 1 -- 172.21.15.89:0/1122028442 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f8b84014430 con 0x7f8b880b1e50 2023-10-10T20:40:12.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.913+0000 7f8b7cff9700 1 --2- 172.21.15.89:0/1122028442 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b74067200 0x7f8b740696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:12.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.913+0000 7f8b7cff9700 1 -- 172.21.15.89:0/1122028442 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8b84084dd0 con 0x7f8b880b1e50 2023-10-10T20:40:12.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.913+0000 7f8b7ffff700 1 --2- 172.21.15.89:0/1122028442 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b74067200 0x7f8b740696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:12.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.913+0000 7f8b7ffff700 1 --2- 172.21.15.89:0/1122028442 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b74067200 0x7f8b740696b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8b78001270 tx=0x7f8b78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:12.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:12.917+0000 7f8b7cff9700 1 -- 172.21.15.89:0/1122028442 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b84051da0 con 0x7f8b880b1e50 2023-10-10T20:40:13.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.097+0000 7f8b8d858700 1 -- 172.21.15.89:0/1122028442 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b8804c3c0 con 0x7f8b74067200 2023-10-10T20:40:13.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.105+0000 7f8b7cff9700 1 -- 172.21.15.89:0/1122028442 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279189 (secure 0 0 0) 0x7f8b8804c3c0 con 0x7f8b74067200 2023-10-10T20:40:13.110 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:13.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.108+0000 7f8b8d858700 1 -- 172.21.15.89:0/1122028442 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b74067200 msgr2=0x7f8b740696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:13.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.108+0000 7f8b8d858700 1 --2- 172.21.15.89:0/1122028442 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b74067200 0x7f8b740696b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8b78001270 tx=0x7f8b78009040 comp rx=0 tx=0).stop 2023-10-10T20:40:13.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.108+0000 7f8b8d858700 1 -- 172.21.15.89:0/1122028442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b880b1e50 msgr2=0x7f8b880cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:13.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.108+0000 7f8b8d858700 1 --2- 172.21.15.89:0/1122028442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b880b1e50 0x7f8b880cab30 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400d890 tx=0x7f8b84008040 comp rx=0 tx=0).stop 2023-10-10T20:40:13.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.108+0000 7f8b8d858700 1 -- 172.21.15.89:0/1122028442 shutdown_connections 2023-10-10T20:40:13.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.108+0000 7f8b8d858700 1 --2- 172.21.15.89:0/1122028442 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b881593b0 0x7f8b8815b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.108+0000 7f8b8d858700 1 --2- 172.21.15.89:0/1122028442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b880b1e50 0x7f8b880cab30 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.108+0000 7f8b8d858700 1 --2- 172.21.15.89:0/1122028442 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b74067200 0x7f8b740696b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.108+0000 7f8b8d858700 1 --2- 172.21.15.89:0/1122028442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b880ceaa0 0x7f8b88155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.108+0000 7f8b8d858700 1 -- 172.21.15.89:0/1122028442 >> 172.21.15.89:0/1122028442 conn(0x7f8b880d1130 msgr2=0x7f8b880b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:13.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.108+0000 7f8b8d858700 1 -- 172.21.15.89:0/1122028442 shutdown_connections 2023-10-10T20:40:13.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.108+0000 7f8b8d858700 1 -- 172.21.15.89:0/1122028442 wait complete. 2023-10-10T20:40:13.115 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:40:13.122 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2023-10-10T20:40:12.720026+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395876,"kb_used_data":9380,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752578844,"statfs":{"total":773094113280,"available":770640736256,"internally_reserved":0,"allocated":9605120,"data_stored":4779567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.725282"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999363+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:59:07.783164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997293+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:10:02.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000303+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:15:33.267457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000522+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:00:28.956602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998301+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:56:06.818452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000192+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:54:48.893540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999923+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998663+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:26:36.884070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998957+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:31.591354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.996718+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:17:57.416786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997206+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:03:11.358023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997183+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000423+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:06:23.747999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000508+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:01:20.723997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997163+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:00:28.956602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998250+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:56:17.858084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998262+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:28:06.502123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000675+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:32:46.206016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999401+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:14:30.053146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000613+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.001511+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:28:06.502123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.001160+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:00:28.956602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000745+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:55:55.484134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.001441+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999584+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:25.450278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998537+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:56:28.068099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999909+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:28:06.502123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999320+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:12.919153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.001698+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:04:07.489913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.996623+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:43:58.261869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000340+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:06:17.893318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997358+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:35:02.298470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":228,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-10T20:40:04.992183+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:40:04.992183+0000","last_peered":"2023-10-10T20:40:04.992183+0000","last_clean":"2023-10-10T20:40:04.992183+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:40:04.992183+0000","last_undegraded":"2023-10-10T20:40:04.992183+0000","last_fullsized":"2023-10-10T20:40:04.992183+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298896,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072944,"statfs":{"total":96636764160,"available":96330694656,"internally_reserved":0,"allocated":1646592,"data_stored":1040342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.60499999999999998,"15min":0.84999999999999998},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.747,"5min":6.8630000000000004,"15min":6.8630000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.499,"5min":0.56299999999999994,"15min":0.63800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.629,"5min":4.1959999999999997,"15min":4.1959999999999997},"last":0.49299999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.65300000000000002,"15min":1.0489999999999999},"min":{"1min":0.39900000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.78800000000000003,"5min":10.1,"15min":10.1},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.61299999999999999,"15min":0.749},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66400000000000003,"5min":6.806,"15min":6.806},"last":0.63}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.64000000000000001,"15min":0.80200000000000005},"min":{"1min":0.41599999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":6.9340000000000002,"15min":6.9340000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59499999999999997,"15min":0.75900000000000001},"min":{"1min":0.38800000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73199999999999998,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.55100000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61799999999999999,"15min":0.77800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70899999999999996,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.68100000000000005},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71899999999999997,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.56499999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.34599999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.626,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.374,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78400000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.40000000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.73299999999999998,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72699999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.38200000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58899999999999997,"15min":0.57199999999999995},"min":{"1min":0.48699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77100000000000002,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.46800000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75600000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.45900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55700000000000005,"15min":0.68899999999999995},"min":{"1min":0.40799999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.73099999999999998,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":0.432},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57899999999999996,"15min":0.73299999999999998},"min":{"1min":0.372,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70799999999999996,"5min":7.8090000000000002,"15min":7.8090000000000002},"last":0.45500000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.72899999999999998},"min":{"1min":0.39700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71899999999999997,"5min":7.867,"15min":7.867},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.64200000000000002,"15min":0.92100000000000004},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75700000000000001,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.497}]},{"osd":2,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.747,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.63400000000000001,"15min":0.874},"min":{"1min":0.48399999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.71199999999999997,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":0.57499999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.628},"min":{"1min":0.439,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79000000000000004,"5min":4.8620000000000001,"15min":4.8620000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.73499999999999999},"min":{"1min":0.47199999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.755,"5min":7.9009999999999998,"15min":7.9009999999999998},"last":0.59899999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.504,"15min":0.46400000000000002},"min":{"1min":0.42799999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71799999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.56499999999999995},"min":{"1min":0.54200000000000004,"5min":0.37,"15min":0.37},"max":{"1min":0.69899999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67600000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51400000000000001,"15min":0.48599999999999999},"min":{"1min":0.40400000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55200000000000005,"15min":0.52600000000000002},"min":{"1min":0.47799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.70999999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.42799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74399999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56899999999999995,"15min":0.54100000000000004},"min":{"1min":0.52700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.747,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.64100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.36399999999999999,"15min":0.49099999999999999},"min":{"1min":0.219,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.56799999999999995,"5min":5.0380000000000003,"15min":5.0380000000000003},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33100000000000002,"15min":0.36499999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.47299999999999998,"5min":1.194,"15min":1.194},"last":0.29299999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.36799999999999999},"min":{"1min":0.27800000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.51400000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.52000000000000002},"min":{"1min":0.26000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52500000000000002,"5min":5.0919999999999996,"15min":5.0919999999999996},"last":0.27800000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.40799999999999997,"15min":0.54300000000000004},"min":{"1min":0.28000000000000003,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.59899999999999998,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.314},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38100000000000001,"15min":0.41299999999999998},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55500000000000005,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.19900000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.40000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92400000000000004,"5min":1.254,"15min":1.254},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60599999999999998,"15min":0.71899999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96499999999999997,"5min":4.9850000000000003,"15min":4.9850000000000003},"last":0.43099999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97599999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.42899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.878,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.51500000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67600000000000005,"15min":0.86699999999999999},"min":{"1min":0.46500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.995,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.40699999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85799999999999998,"5min":1.171,"15min":1.171},"last":0.36299999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60099999999999998,"15min":0.62},"min":{"1min":0.443,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.004,"5min":1.214,"15min":1.214},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63700000000000001,"15min":0.73699999999999999},"min":{"1min":0.41999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90500000000000003,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.47699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302988,"kb_used_data":1604,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068852,"statfs":{"total":96636764160,"available":96326504448,"internally_reserved":0,"allocated":1642496,"data_stored":1039381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.377,"15min":0.55100000000000005},"min":{"1min":0.20699999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.51600000000000001,"5min":7.9249999999999998,"15min":7.9249999999999998},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.374,"15min":0.52400000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45800000000000002,"5min":6.6299999999999999,"15min":6.6299999999999999},"last":0.26400000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.39000000000000001,"15min":0.57399999999999995},"min":{"1min":0.23100000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.47499999999999998,"5min":8.2509999999999994,"15min":8.2509999999999994},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39800000000000002,"15min":0.54800000000000004},"min":{"1min":0.245,"5min":0.222,"15min":0.222},"max":{"1min":0.52400000000000002,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.33800000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.40500000000000003,"15min":0.57699999999999996},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.45400000000000001,"15min":0.63400000000000001},"min":{"1min":0.31900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57899999999999996,"5min":7.8929999999999998,"15min":7.8929999999999998},"last":0.379}]},{"osd":4,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.61099999999999999,"15min":0.73199999999999998},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78000000000000003,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.63600000000000001,"15min":0.79600000000000004},"min":{"1min":0.45100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.95399999999999996,"5min":7.6920000000000002,"15min":7.6920000000000002},"last":0.54500000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.77500000000000002},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92400000000000004,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.64300000000000002,"15min":0.79900000000000004},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91400000000000003,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":0.621}]},{"osd":6,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67000000000000004,"15min":0.83999999999999997},"min":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94499999999999995,"5min":7.8360000000000003,"15min":7.8360000000000003},"last":0.495},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66400000000000003,"15min":0.77900000000000003},"min":{"1min":0.49199999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90300000000000002,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.58799999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64000000000000001,"15min":0.76900000000000002},"min":{"1min":0.45900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93300000000000005,"5min":6.7160000000000002,"15min":6.7160000000000002},"last":0.627},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66100000000000003,"15min":0.80900000000000005},"min":{"1min":0.44600000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.878,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.56699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.31900000000000001,"15min":0.32800000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.59699999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.33600000000000002,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45400000000000001,"5min":0.746,"15min":0.746},"last":0.30499999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.45400000000000001,"15min":0.77100000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.63,"5min":11.967000000000001,"15min":11.967000000000001},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39000000000000001,"15min":0.498},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52700000000000002,"5min":4.1100000000000003,"15min":4.1100000000000003},"last":0.39800000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.41199999999999998,"15min":0.57899999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48799999999999999,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.50900000000000001,"15min":0.82199999999999995},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65900000000000003,"5min":11.935,"15min":11.935},"last":0.38300000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.40600000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.80100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.58999999999999997},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.45100000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.33400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83599999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42099999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81599999999999995,"5min":1.052,"15min":1.052},"last":0.433},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64500000000000002,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85999999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.48999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302412,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069428,"statfs":{"total":96636764160,"available":96327094272,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.309,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44400000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.317},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48699999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.35499999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33600000000000002,"15min":0.33000000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.49199999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38200000000000001,"15min":0.376},"min":{"1min":0.307,"5min":0.251,"15min":0.251},"max":{"1min":0.53900000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.34899999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.309},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35499999999999998,"15min":0.36099999999999999},"min":{"1min":0.25900000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.51600000000000001,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.28999999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.38100000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83299999999999996,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60699999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.39400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78700000000000003,"5min":1.05,"15min":1.05},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86799999999999999,"5min":1.01,"15min":1.01},"last":0.55500000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92000000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.877,"5min":1.026,"15min":1.026},"last":0.71499999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77100000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.46600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89400000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.66000000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58199999999999996,"5min":0.747,"15min":0.747},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.59799999999999998,"15min":0.79200000000000004},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69699999999999995,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.46400000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.60299999999999998,"15min":0.79900000000000004},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60299999999999998,"5min":7.7160000000000002,"15min":7.7160000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68100000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.379},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63600000000000001,"15min":0.81799999999999995},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.441}]},{"osd":3,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.58899999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48399999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.502,"15min":0.48399999999999999},"min":{"1min":0.34599999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.38400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77300000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.41699999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72099999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.52600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.501}]},{"osd":7,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.58199999999999996},"min":{"1min":0.44700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.63600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.307,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63600000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.442},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.34300000000000003,"5min":0.215,"15min":0.215},"max":{"1min":0.70399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.48499999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.56200000000000006,"15min":0.65300000000000002},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55300000000000005,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68200000000000005,"5min":5.3109999999999999,"15min":5.3109999999999999},"last":0.52100000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56899999999999995,"15min":0.66100000000000003},"min":{"1min":0.32900000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64700000000000002,"5min":5.2960000000000003,"15min":5.2960000000000003},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72399999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.46400000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59499999999999997,"15min":0.69099999999999995},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70399999999999996,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59699999999999998,"15min":0.70399999999999996},"min":{"1min":0.40300000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70599999999999996,"5min":5.2850000000000001,"15min":5.2850000000000001},"last":0.39300000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48299999999999998,"15min":0.497},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.63800000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.626},{"interface":"front","average":{"1min":0.505,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65000000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.54400000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.65100000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71299999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58299999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.33000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72699999999999998,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.60099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:40:13.128 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:40:13.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.275+0000 7f0d2b3f8700 1 Processor -- start 2023-10-10T20:40:13.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.275+0000 7f0d2b3f8700 1 -- start start 2023-10-10T20:40:13.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.276+0000 7f0d2b3f8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d24159890 0x7f0d2415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:13.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.276+0000 7f0d2b3f8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0d24159c60 0x7f0d2415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:13.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.276+0000 7f0d2b3f8700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0d2415db10 0x7f0d24069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:13.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.276+0000 7f0d2b3f8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d2415a160 con 0x7f0d24159c60 2023-10-10T20:40:13.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.276+0000 7f0d2b3f8700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d240d13a0 con 0x7f0d2415db10 2023-10-10T20:40:13.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.276+0000 7f0d2b3f8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2415e140 con 0x7f0d24159890 2023-10-10T20:40:13.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.276+0000 7f0d28993700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0d24159c60 0x7f0d2415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:13.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.276+0000 7f0d28993700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0d24159c60 0x7f0d2415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43444/0 (socket says 172.21.15.89:43444) 2023-10-10T20:40:13.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.276+0000 7f0d28993700 1 -- 172.21.15.89:0/1313919959 learned_addr learned my addr 172.21.15.89:0/1313919959 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:13.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.276+0000 7f0d28993700 1 -- 172.21.15.89:0/1313919959 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d24159890 msgr2=0x7f0d2415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:13.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.276+0000 7f0d29995700 1 --2- 172.21.15.89:0/1313919959 >> v2:172.21.15.111:3300/0 conn(0x7f0d2415db10 0x7f0d24069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:13.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.276+0000 7f0d29194700 1 --2- 172.21.15.89:0/1313919959 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d24159890 0x7f0d2415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:13.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.276+0000 7f0d28993700 1 --2- 172.21.15.89:0/1313919959 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d24159890 0x7f0d2415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.276+0000 7f0d28993700 1 -- 172.21.15.89:0/1313919959 >> v2:172.21.15.111:3300/0 conn(0x7f0d2415db10 msgr2=0x7f0d24069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:13.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.276+0000 7f0d28993700 1 --2- 172.21.15.89:0/1313919959 >> v2:172.21.15.111:3300/0 conn(0x7f0d2415db10 0x7f0d24069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.276+0000 7f0d28993700 1 -- 172.21.15.89:0/1313919959 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d2415a3d0 con 0x7f0d24159c60 2023-10-10T20:40:13.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.276+0000 7f0d29995700 1 --2- 172.21.15.89:0/1313919959 >> v2:172.21.15.111:3300/0 conn(0x7f0d2415db10 0x7f0d24069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:13.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.276+0000 7f0d29194700 1 --2- 172.21.15.89:0/1313919959 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d24159890 0x7f0d2415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:13.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.277+0000 7f0d28993700 1 --2- 172.21.15.89:0/1313919959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d24159c60 0x7f0d2415d5d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f0d24089b50 tx=0x7f0d14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec39e9a7eb524702 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:13.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.277+0000 7f0d1bfff700 1 -- 172.21.15.89:0/1313919959 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1400b860 con 0x7f0d24159c60 2023-10-10T20:40:13.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.277+0000 7f0d1bfff700 1 -- 172.21.15.89:0/1313919959 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1400b9c0 con 0x7f0d24159c60 2023-10-10T20:40:13.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.277+0000 7f0d1bfff700 1 -- 172.21.15.89:0/1313919959 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1400bd60 con 0x7f0d24159c60 2023-10-10T20:40:13.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.277+0000 7f0d2b3f8700 1 -- 172.21.15.89:0/1313919959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d24159c60 msgr2=0x7f0d2415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:13.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.277+0000 7f0d2b3f8700 1 --2- 172.21.15.89:0/1313919959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d24159c60 0x7f0d2415d5d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f0d24089b50 tx=0x7f0d14009250 comp rx=0 tx=0).stop 2023-10-10T20:40:13.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.277+0000 7f0d2b3f8700 1 -- 172.21.15.89:0/1313919959 shutdown_connections 2023-10-10T20:40:13.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.277+0000 7f0d2b3f8700 1 --2- 172.21.15.89:0/1313919959 >> v2:172.21.15.111:3300/0 conn(0x7f0d2415db10 0x7f0d24069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.277+0000 7f0d2b3f8700 1 --2- 172.21.15.89:0/1313919959 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d24159890 0x7f0d2415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.277+0000 7f0d2b3f8700 1 --2- 172.21.15.89:0/1313919959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d24159c60 0x7f0d2415d5d0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.277+0000 7f0d2b3f8700 1 -- 172.21.15.89:0/1313919959 >> 172.21.15.89:0/1313919959 conn(0x7f0d24068e20 msgr2=0x7f0d240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:13.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.277+0000 7f0d2b3f8700 1 -- 172.21.15.89:0/1313919959 shutdown_connections 2023-10-10T20:40:13.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.277+0000 7f0d2b3f8700 1 -- 172.21.15.89:0/1313919959 wait complete. 2023-10-10T20:40:13.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.278+0000 7f0d2b3f8700 1 Processor -- start 2023-10-10T20:40:13.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.278+0000 7f0d2b3f8700 1 -- start start 2023-10-10T20:40:13.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.279+0000 7f0d2b3f8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d24159890 0x7f0d2414ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:13.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.279+0000 7f0d2b3f8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d24159c60 0x7f0d24151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:13.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.279+0000 7f0d2b3f8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d2415db10 0x7f0d240cd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:13.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.279+0000 7f0d2b3f8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d24152ad0 con 0x7f0d2415db10 2023-10-10T20:40:13.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.279+0000 7f0d2b3f8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d24151ec0 con 0x7f0d24159c60 2023-10-10T20:40:13.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.279+0000 7f0d2b3f8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d241521a0 con 0x7f0d24159890 2023-10-10T20:40:13.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.279+0000 7f0d29995700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d2415db10 0x7f0d240cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:13.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.279+0000 7f0d29995700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d2415db10 0x7f0d240cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43446/0 (socket says 172.21.15.89:43446) 2023-10-10T20:40:13.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.279+0000 7f0d29995700 1 -- 172.21.15.89:0/3488801968 learned_addr learned my addr 172.21.15.89:0/3488801968 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:13.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.280+0000 7f0d28993700 1 --2- 172.21.15.89:0/3488801968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d24159c60 0x7f0d24151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:13.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.280+0000 7f0d29194700 1 --2- 172.21.15.89:0/3488801968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d24159890 0x7f0d2414ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:13.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.280+0000 7f0d29995700 1 -- 172.21.15.89:0/3488801968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d24159890 msgr2=0x7f0d2414ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:13.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.280+0000 7f0d29995700 1 --2- 172.21.15.89:0/3488801968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d24159890 0x7f0d2414ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.280+0000 7f0d29995700 1 -- 172.21.15.89:0/3488801968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d24159c60 msgr2=0x7f0d24151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:13.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.280+0000 7f0d29995700 1 --2- 172.21.15.89:0/3488801968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d24159c60 0x7f0d24151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.280+0000 7f0d29995700 1 -- 172.21.15.89:0/3488801968 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d140094f0 con 0x7f0d2415db10 2023-10-10T20:40:13.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.280+0000 7f0d29194700 1 --2- 172.21.15.89:0/3488801968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d24159890 0x7f0d2414ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:13.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.280+0000 7f0d28993700 1 --2- 172.21.15.89:0/3488801968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d24159c60 0x7f0d24151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:13.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.280+0000 7f0d29995700 1 --2- 172.21.15.89:0/3488801968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d2415db10 0x7f0d240cd470 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00c850 tx=0x7f0d1c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:13.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.280+0000 7f0d197fa700 1 -- 172.21.15.89:0/3488801968 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c011600 con 0x7f0d2415db10 2023-10-10T20:40:13.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.280+0000 7f0d197fa700 1 -- 172.21.15.89:0/3488801968 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1c011760 con 0x7f0d2415db10 2023-10-10T20:40:13.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.280+0000 7f0d2b3f8700 1 -- 172.21.15.89:0/3488801968 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d24152480 con 0x7f0d2415db10 2023-10-10T20:40:13.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.280+0000 7f0d197fa700 1 -- 172.21.15.89:0/3488801968 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c011b20 con 0x7f0d2415db10 2023-10-10T20:40:13.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.280+0000 7f0d2b3f8700 1 -- 172.21.15.89:0/3488801968 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d240cfae0 con 0x7f0d2415db10 2023-10-10T20:40:13.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.281+0000 7f0d2b3f8700 1 -- 172.21.15.89:0/3488801968 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d2404c3c0 con 0x7f0d2415db10 2023-10-10T20:40:13.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.281+0000 7f0d197fa700 1 -- 172.21.15.89:0/3488801968 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f0d1c018350 con 0x7f0d2415db10 2023-10-10T20:40:13.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.282+0000 7f0d197fa700 1 --2- 172.21.15.89:0/3488801968 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0d08067200 0x7f0d080696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:13.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.282+0000 7f0d197fa700 1 -- 172.21.15.89:0/3488801968 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0d1c084ad0 con 0x7f0d2415db10 2023-10-10T20:40:13.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.282+0000 7f0d29194700 1 --2- 172.21.15.89:0/3488801968 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0d08067200 0x7f0d080696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:13.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.282+0000 7f0d29194700 1 --2- 172.21.15.89:0/3488801968 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0d08067200 0x7f0d080696b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0d2000d510 tx=0x7f0d200075e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:13.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.286+0000 7f0d197fa700 1 -- 172.21.15.89:0/3488801968 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d1c0518e0 con 0x7f0d2415db10 2023-10-10T20:40:13.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.464+0000 7f0d2b3f8700 1 -- 172.21.15.89:0/3488801968 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d2406a5b0 con 0x7f0d08067200 2023-10-10T20:40:13.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.471+0000 7f0d197fa700 1 -- 172.21.15.89:0/3488801968 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279189 (secure 0 0 0) 0x7f0d2406a5b0 con 0x7f0d08067200 2023-10-10T20:40:13.477 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:13.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.474+0000 7f0d2b3f8700 1 -- 172.21.15.89:0/3488801968 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0d08067200 msgr2=0x7f0d080696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:13.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.474+0000 7f0d2b3f8700 1 --2- 172.21.15.89:0/3488801968 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0d08067200 0x7f0d080696b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0d2000d510 tx=0x7f0d200075e0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.474+0000 7f0d2b3f8700 1 -- 172.21.15.89:0/3488801968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d2415db10 msgr2=0x7f0d240cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:13.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.474+0000 7f0d2b3f8700 1 --2- 172.21.15.89:0/3488801968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d2415db10 0x7f0d240cd470 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00c850 tx=0x7f0d1c008040 comp rx=0 tx=0).stop 2023-10-10T20:40:13.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.474+0000 7f0d2b3f8700 1 -- 172.21.15.89:0/3488801968 shutdown_connections 2023-10-10T20:40:13.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.474+0000 7f0d2b3f8700 1 --2- 172.21.15.89:0/3488801968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d2415db10 0x7f0d240cd470 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.474+0000 7f0d2b3f8700 1 --2- 172.21.15.89:0/3488801968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d24159890 0x7f0d2414ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.474+0000 7f0d2b3f8700 1 --2- 172.21.15.89:0/3488801968 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0d08067200 0x7f0d080696b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.474+0000 7f0d2b3f8700 1 --2- 172.21.15.89:0/3488801968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d24159c60 0x7f0d24151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.474+0000 7f0d2b3f8700 1 -- 172.21.15.89:0/3488801968 >> 172.21.15.89:0/3488801968 conn(0x7f0d24068e20 msgr2=0x7f0d24069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:13.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.474+0000 7f0d2b3f8700 1 -- 172.21.15.89:0/3488801968 shutdown_connections 2023-10-10T20:40:13.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.474+0000 7f0d2b3f8700 1 -- 172.21.15.89:0/3488801968 wait complete. 2023-10-10T20:40:13.484 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:40:13.488 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2023-10-10T20:40:12.720026+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395876,"kb_used_data":9380,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752578844,"statfs":{"total":773094113280,"available":770640736256,"internally_reserved":0,"allocated":9605120,"data_stored":4779567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.725282"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999363+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:59:07.783164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997293+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:10:02.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000303+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:15:33.267457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000522+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:00:28.956602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998301+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:56:06.818452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000192+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:54:48.893540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999923+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998663+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:26:36.884070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998957+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:31.591354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.996718+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:17:57.416786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997206+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:03:11.358023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997183+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000423+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:06:23.747999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000508+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:01:20.723997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997163+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:00:28.956602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998250+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:56:17.858084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998262+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:28:06.502123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000675+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:32:46.206016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999401+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:14:30.053146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000613+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.001511+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:28:06.502123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.001160+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:00:28.956602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000745+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:55:55.484134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.001441+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999584+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:25.450278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998537+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:56:28.068099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999909+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:28:06.502123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999320+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:12.919153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.001698+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:04:07.489913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.996623+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:43:58.261869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000340+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:06:17.893318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997358+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:35:02.298470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":228,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-10T20:40:04.992183+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:40:04.992183+0000","last_peered":"2023-10-10T20:40:04.992183+0000","last_clean":"2023-10-10T20:40:04.992183+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:40:04.992183+0000","last_undegraded":"2023-10-10T20:40:04.992183+0000","last_fullsized":"2023-10-10T20:40:04.992183+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298896,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072944,"statfs":{"total":96636764160,"available":96330694656,"internally_reserved":0,"allocated":1646592,"data_stored":1040342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.60499999999999998,"15min":0.84999999999999998},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.747,"5min":6.8630000000000004,"15min":6.8630000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.499,"5min":0.56299999999999994,"15min":0.63800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.629,"5min":4.1959999999999997,"15min":4.1959999999999997},"last":0.49299999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.65300000000000002,"15min":1.0489999999999999},"min":{"1min":0.39900000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.78800000000000003,"5min":10.1,"15min":10.1},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.61299999999999999,"15min":0.749},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66400000000000003,"5min":6.806,"15min":6.806},"last":0.63}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.64000000000000001,"15min":0.80200000000000005},"min":{"1min":0.41599999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":6.9340000000000002,"15min":6.9340000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59499999999999997,"15min":0.75900000000000001},"min":{"1min":0.38800000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73199999999999998,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.55100000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61799999999999999,"15min":0.77800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70899999999999996,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.68100000000000005},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71899999999999997,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.56499999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.34599999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.626,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.374,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78400000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.40000000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.73299999999999998,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72699999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.38200000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58899999999999997,"15min":0.57199999999999995},"min":{"1min":0.48699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77100000000000002,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.46800000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75600000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.45900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55700000000000005,"15min":0.68899999999999995},"min":{"1min":0.40799999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.73099999999999998,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":0.432},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57899999999999996,"15min":0.73299999999999998},"min":{"1min":0.372,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70799999999999996,"5min":7.8090000000000002,"15min":7.8090000000000002},"last":0.45500000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.72899999999999998},"min":{"1min":0.39700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71899999999999997,"5min":7.867,"15min":7.867},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.64200000000000002,"15min":0.92100000000000004},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75700000000000001,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.497}]},{"osd":2,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.747,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.63400000000000001,"15min":0.874},"min":{"1min":0.48399999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.71199999999999997,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":0.57499999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.628},"min":{"1min":0.439,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79000000000000004,"5min":4.8620000000000001,"15min":4.8620000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.73499999999999999},"min":{"1min":0.47199999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.755,"5min":7.9009999999999998,"15min":7.9009999999999998},"last":0.59899999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.504,"15min":0.46400000000000002},"min":{"1min":0.42799999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71799999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.56499999999999995},"min":{"1min":0.54200000000000004,"5min":0.37,"15min":0.37},"max":{"1min":0.69899999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67600000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51400000000000001,"15min":0.48599999999999999},"min":{"1min":0.40400000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55200000000000005,"15min":0.52600000000000002},"min":{"1min":0.47799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.70999999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.42799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74399999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56899999999999995,"15min":0.54100000000000004},"min":{"1min":0.52700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.747,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.64100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.36399999999999999,"15min":0.49099999999999999},"min":{"1min":0.219,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.56799999999999995,"5min":5.0380000000000003,"15min":5.0380000000000003},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33100000000000002,"15min":0.36499999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.47299999999999998,"5min":1.194,"15min":1.194},"last":0.29299999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.36799999999999999},"min":{"1min":0.27800000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.51400000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.52000000000000002},"min":{"1min":0.26000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52500000000000002,"5min":5.0919999999999996,"15min":5.0919999999999996},"last":0.27800000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.40799999999999997,"15min":0.54300000000000004},"min":{"1min":0.28000000000000003,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.59899999999999998,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.314},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38100000000000001,"15min":0.41299999999999998},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55500000000000005,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.19900000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.40000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92400000000000004,"5min":1.254,"15min":1.254},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60599999999999998,"15min":0.71899999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96499999999999997,"5min":4.9850000000000003,"15min":4.9850000000000003},"last":0.43099999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97599999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.42899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.878,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.51500000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67600000000000005,"15min":0.86699999999999999},"min":{"1min":0.46500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.995,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.40699999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85799999999999998,"5min":1.171,"15min":1.171},"last":0.36299999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60099999999999998,"15min":0.62},"min":{"1min":0.443,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.004,"5min":1.214,"15min":1.214},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63700000000000001,"15min":0.73699999999999999},"min":{"1min":0.41999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90500000000000003,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.47699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302988,"kb_used_data":1604,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068852,"statfs":{"total":96636764160,"available":96326504448,"internally_reserved":0,"allocated":1642496,"data_stored":1039381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.377,"15min":0.55100000000000005},"min":{"1min":0.20699999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.51600000000000001,"5min":7.9249999999999998,"15min":7.9249999999999998},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.374,"15min":0.52400000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45800000000000002,"5min":6.6299999999999999,"15min":6.6299999999999999},"last":0.26400000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.39000000000000001,"15min":0.57399999999999995},"min":{"1min":0.23100000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.47499999999999998,"5min":8.2509999999999994,"15min":8.2509999999999994},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39800000000000002,"15min":0.54800000000000004},"min":{"1min":0.245,"5min":0.222,"15min":0.222},"max":{"1min":0.52400000000000002,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.33800000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.40500000000000003,"15min":0.57699999999999996},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.45400000000000001,"15min":0.63400000000000001},"min":{"1min":0.31900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57899999999999996,"5min":7.8929999999999998,"15min":7.8929999999999998},"last":0.379}]},{"osd":4,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.61099999999999999,"15min":0.73199999999999998},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78000000000000003,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.63600000000000001,"15min":0.79600000000000004},"min":{"1min":0.45100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.95399999999999996,"5min":7.6920000000000002,"15min":7.6920000000000002},"last":0.54500000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.77500000000000002},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92400000000000004,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.64300000000000002,"15min":0.79900000000000004},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91400000000000003,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":0.621}]},{"osd":6,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67000000000000004,"15min":0.83999999999999997},"min":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94499999999999995,"5min":7.8360000000000003,"15min":7.8360000000000003},"last":0.495},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66400000000000003,"15min":0.77900000000000003},"min":{"1min":0.49199999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90300000000000002,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.58799999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64000000000000001,"15min":0.76900000000000002},"min":{"1min":0.45900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93300000000000005,"5min":6.7160000000000002,"15min":6.7160000000000002},"last":0.627},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66100000000000003,"15min":0.80900000000000005},"min":{"1min":0.44600000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.878,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.56699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.31900000000000001,"15min":0.32800000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.59699999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.33600000000000002,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45400000000000001,"5min":0.746,"15min":0.746},"last":0.30499999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.45400000000000001,"15min":0.77100000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.63,"5min":11.967000000000001,"15min":11.967000000000001},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39000000000000001,"15min":0.498},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52700000000000002,"5min":4.1100000000000003,"15min":4.1100000000000003},"last":0.39800000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.41199999999999998,"15min":0.57899999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48799999999999999,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.50900000000000001,"15min":0.82199999999999995},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65900000000000003,"5min":11.935,"15min":11.935},"last":0.38300000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.40600000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.80100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.58999999999999997},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.45100000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.33400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83599999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42099999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81599999999999995,"5min":1.052,"15min":1.052},"last":0.433},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64500000000000002,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85999999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.48999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302412,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069428,"statfs":{"total":96636764160,"available":96327094272,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.309,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44400000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.317},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48699999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.35499999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33600000000000002,"15min":0.33000000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.49199999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38200000000000001,"15min":0.376},"min":{"1min":0.307,"5min":0.251,"15min":0.251},"max":{"1min":0.53900000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.34899999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.309},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35499999999999998,"15min":0.36099999999999999},"min":{"1min":0.25900000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.51600000000000001,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.28999999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.38100000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83299999999999996,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60699999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.39400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78700000000000003,"5min":1.05,"15min":1.05},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86799999999999999,"5min":1.01,"15min":1.01},"last":0.55500000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92000000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.877,"5min":1.026,"15min":1.026},"last":0.71499999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77100000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.46600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89400000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.66000000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58199999999999996,"5min":0.747,"15min":0.747},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.59799999999999998,"15min":0.79200000000000004},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69699999999999995,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.46400000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.60299999999999998,"15min":0.79900000000000004},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60299999999999998,"5min":7.7160000000000002,"15min":7.7160000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68100000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.379},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63600000000000001,"15min":0.81799999999999995},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.441}]},{"osd":3,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.58899999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48399999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.502,"15min":0.48399999999999999},"min":{"1min":0.34599999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.38400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77300000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.41699999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72099999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.52600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.501}]},{"osd":7,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.58199999999999996},"min":{"1min":0.44700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.63600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.307,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63600000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.442},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.34300000000000003,"5min":0.215,"15min":0.215},"max":{"1min":0.70399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.48499999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.56200000000000006,"15min":0.65300000000000002},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55300000000000005,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68200000000000005,"5min":5.3109999999999999,"15min":5.3109999999999999},"last":0.52100000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56899999999999995,"15min":0.66100000000000003},"min":{"1min":0.32900000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64700000000000002,"5min":5.2960000000000003,"15min":5.2960000000000003},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72399999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.46400000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59499999999999997,"15min":0.69099999999999995},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70399999999999996,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59699999999999998,"15min":0.70399999999999996},"min":{"1min":0.40300000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70599999999999996,"5min":5.2850000000000001,"15min":5.2850000000000001},"last":0.39300000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48299999999999998,"15min":0.497},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.63800000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.626},{"interface":"front","average":{"1min":0.505,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65000000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.54400000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.65100000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71299999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58299999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.33000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72699999999999998,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.60099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:40:13.494 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:40:13.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.641+0000 7fe4b8347700 1 Processor -- start 2023-10-10T20:40:13.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.641+0000 7fe4b8347700 1 -- start start 2023-10-10T20:40:13.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.642+0000 7fe4b8347700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b0159c10 0x7fe4b015b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:13.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.642+0000 7fe4b8347700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe4b015c910 0x7fe4b015cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:13.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.642+0000 7fe4b8347700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b015d2b0 con 0x7fe4b00b5ea0 2023-10-10T20:40:13.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.642+0000 7fe4b8347700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4b00b2a40 con 0x7fe4b015c910 2023-10-10T20:40:13.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.642+0000 7fe4b8347700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b015c480 con 0x7fe4b0159c10 2023-10-10T20:40:13.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.642+0000 7fe4b60e3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b0159c10 0x7fe4b015b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:13.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.642+0000 7fe4b68e4700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe4b015c910 0x7fe4b015cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:13.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.642+0000 7fe4b60e3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b0159c10 0x7fe4b015b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:40608/0 (socket says 172.21.15.89:40608) 2023-10-10T20:40:13.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.642+0000 7fe4b60e3700 1 -- 172.21.15.89:0/222697632 learned_addr learned my addr 172.21.15.89:0/222697632 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:13.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.642+0000 7fe4b68e4700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe4b015c910 0x7fe4b015cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:33028/0 (socket says 172.21.15.89:33028) 2023-10-10T20:40:13.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.642+0000 7fe4b58e2700 1 --1- 172.21.15.89:0/222697632 >> v1:172.21.15.111:6789/0 conn(0x7fe4b00b5ea0 0x7fe4b0068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:35294/0 (socket says 172.21.15.89:35294) 2023-10-10T20:40:13.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.642+0000 7fe4b60e3700 1 -- 172.21.15.89:0/222697632 >> v2:172.21.15.111:3300/0 conn(0x7fe4b015c910 msgr2=0x7fe4b015cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:13.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.642+0000 7fe4b60e3700 1 --2- 172.21.15.89:0/222697632 >> v2:172.21.15.111:3300/0 conn(0x7fe4b015c910 0x7fe4b015cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.642+0000 7fe4b60e3700 1 -- 172.21.15.89:0/222697632 >> v1:172.21.15.111:6789/0 conn(0x7fe4b00b5ea0 legacy=0x7fe4b0068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:13.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.642+0000 7fe4b60e3700 1 -- 172.21.15.89:0/222697632 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b015b5d0 con 0x7fe4b0159c10 2023-10-10T20:40:13.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.642+0000 7fe4b60e3700 1 --2- 172.21.15.89:0/222697632 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b0159c10 0x7fe4b015b090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009f20 tx=0x7fe4a8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11600094a60c7be6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:13.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.643+0000 7fe4b50e1700 1 -- 172.21.15.89:0/222697632 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a800e070 con 0x7fe4b0159c10 2023-10-10T20:40:13.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.643+0000 7fe4b50e1700 1 -- 172.21.15.89:0/222697632 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a800b9d0 con 0x7fe4b0159c10 2023-10-10T20:40:13.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.643+0000 7fe4b50e1700 1 -- 172.21.15.89:0/222697632 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a80092e0 con 0x7fe4b0159c10 2023-10-10T20:40:13.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.643+0000 7fe4b8347700 1 -- 172.21.15.89:0/222697632 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b0159c10 msgr2=0x7fe4b015b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:13.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.643+0000 7fe4b8347700 1 --2- 172.21.15.89:0/222697632 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b0159c10 0x7fe4b015b090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009f20 tx=0x7fe4a8009840 comp rx=0 tx=0).stop 2023-10-10T20:40:13.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.643+0000 7fe4b8347700 1 -- 172.21.15.89:0/222697632 shutdown_connections 2023-10-10T20:40:13.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.643+0000 7fe4b8347700 1 --2- 172.21.15.89:0/222697632 >> v2:172.21.15.111:3300/0 conn(0x7fe4b015c910 0x7fe4b015cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.643+0000 7fe4b8347700 1 --2- 172.21.15.89:0/222697632 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b0159c10 0x7fe4b015b090 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.643+0000 7fe4b8347700 1 -- 172.21.15.89:0/222697632 >> 172.21.15.89:0/222697632 conn(0x7fe4b0099a20 msgr2=0x7fe4b00af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:13.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.643+0000 7fe4b8347700 1 -- 172.21.15.89:0/222697632 shutdown_connections 2023-10-10T20:40:13.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.643+0000 7fe4b8347700 1 -- 172.21.15.89:0/222697632 wait complete. 2023-10-10T20:40:13.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.644+0000 7fe4b8347700 1 Processor -- start 2023-10-10T20:40:13.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.644+0000 7fe4b8347700 1 -- start start 2023-10-10T20:40:13.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.645+0000 7fe4b8347700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b00b5ea0 0x7fe4b00b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:13.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.645+0000 7fe4b8347700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4b0159c10 0x7fe4b009db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:13.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.645+0000 7fe4b8347700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b015c910 0x7fe4b0169890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:13.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.645+0000 7fe4b8347700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b00afd90 con 0x7fe4b0159c10 2023-10-10T20:40:13.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.645+0000 7fe4b8347700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b00ad770 con 0x7fe4b015c910 2023-10-10T20:40:13.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.645+0000 7fe4b8347700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b00ada50 con 0x7fe4b00b5ea0 2023-10-10T20:40:13.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.645+0000 7fe4b58e2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4b0159c10 0x7fe4b009db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:13.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.645+0000 7fe4b58e2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4b0159c10 0x7fe4b009db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43464/0 (socket says 172.21.15.89:43464) 2023-10-10T20:40:13.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.645+0000 7fe4b58e2700 1 -- 172.21.15.89:0/2531846819 learned_addr learned my addr 172.21.15.89:0/2531846819 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:13.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.645+0000 7fe4b60e3700 1 --2- 172.21.15.89:0/2531846819 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b00b5ea0 0x7fe4b00b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:13.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.645+0000 7fe4b68e4700 1 --2- 172.21.15.89:0/2531846819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b015c910 0x7fe4b0169890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:13.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.645+0000 7fe4b58e2700 1 -- 172.21.15.89:0/2531846819 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b00b5ea0 msgr2=0x7fe4b00b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:13.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.645+0000 7fe4b58e2700 1 --2- 172.21.15.89:0/2531846819 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b00b5ea0 0x7fe4b00b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.645+0000 7fe4b58e2700 1 -- 172.21.15.89:0/2531846819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b015c910 msgr2=0x7fe4b0169890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:13.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.645+0000 7fe4b58e2700 1 --2- 172.21.15.89:0/2531846819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b015c910 0x7fe4b0169890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.645+0000 7fe4b58e2700 1 -- 172.21.15.89:0/2531846819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a8009ae0 con 0x7fe4b0159c10 2023-10-10T20:40:13.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.646+0000 7fe4b58e2700 1 --2- 172.21.15.89:0/2531846819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4b0159c10 0x7fe4b009db80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400c7c0 tx=0x7fe4a4007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:13.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.646+0000 7fe4ae7fc700 1 -- 172.21.15.89:0/2531846819 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4005a70 con 0x7fe4b0159c10 2023-10-10T20:40:13.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.646+0000 7fe4ae7fc700 1 -- 172.21.15.89:0/2531846819 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a4005bd0 con 0x7fe4b0159c10 2023-10-10T20:40:13.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.646+0000 7fe4ae7fc700 1 -- 172.21.15.89:0/2531846819 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40070d0 con 0x7fe4b0159c10 2023-10-10T20:40:13.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.646+0000 7fe4b8347700 1 -- 172.21.15.89:0/2531846819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b00add30 con 0x7fe4b0159c10 2023-10-10T20:40:13.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.646+0000 7fe4b8347700 1 -- 172.21.15.89:0/2531846819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b00b0600 con 0x7fe4b0159c10 2023-10-10T20:40:13.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.647+0000 7fe4b8347700 1 -- 172.21.15.89:0/2531846819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b00ae120 con 0x7fe4b0159c10 2023-10-10T20:40:13.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.647+0000 7fe4ae7fc700 1 -- 172.21.15.89:0/2531846819 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fe4a4007480 con 0x7fe4b0159c10 2023-10-10T20:40:13.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.648+0000 7fe4ae7fc700 1 --2- 172.21.15.89:0/2531846819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe49c067200 0x7fe49c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:13.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.648+0000 7fe4ae7fc700 1 -- 172.21.15.89:0/2531846819 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe4a4083c20 con 0x7fe4b0159c10 2023-10-10T20:40:13.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.648+0000 7fe4b60e3700 1 --2- 172.21.15.89:0/2531846819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe49c067200 0x7fe49c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:13.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.648+0000 7fe4b60e3700 1 --2- 172.21.15.89:0/2531846819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe49c067200 0x7fe49c0696b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009880 tx=0x7fe4a8018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:13.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.652+0000 7fe4ae7fc700 1 -- 172.21.15.89:0/2531846819 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4a4050b40 con 0x7fe4b0159c10 2023-10-10T20:40:13.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.829+0000 7fe4b8347700 1 -- 172.21.15.89:0/2531846819 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4b004c3c0 con 0x7fe49c067200 2023-10-10T20:40:13.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.837+0000 7fe4ae7fc700 1 -- 172.21.15.89:0/2531846819 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279189 (secure 0 0 0) 0x7fe4b004c3c0 con 0x7fe49c067200 2023-10-10T20:40:13.842 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:13.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.839+0000 7fe4b8347700 1 -- 172.21.15.89:0/2531846819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe49c067200 msgr2=0x7fe49c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:13.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.839+0000 7fe4b8347700 1 --2- 172.21.15.89:0/2531846819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe49c067200 0x7fe49c0696b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009880 tx=0x7fe4a8018000 comp rx=0 tx=0).stop 2023-10-10T20:40:13.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.839+0000 7fe4b8347700 1 -- 172.21.15.89:0/2531846819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4b0159c10 msgr2=0x7fe4b009db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:13.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.839+0000 7fe4b8347700 1 --2- 172.21.15.89:0/2531846819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4b0159c10 0x7fe4b009db80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400c7c0 tx=0x7fe4a4007f70 comp rx=0 tx=0).stop 2023-10-10T20:40:13.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.839+0000 7fe4b8347700 1 -- 172.21.15.89:0/2531846819 shutdown_connections 2023-10-10T20:40:13.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.839+0000 7fe4b8347700 1 --2- 172.21.15.89:0/2531846819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe49c067200 0x7fe49c0696b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.840+0000 7fe4b8347700 1 --2- 172.21.15.89:0/2531846819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b015c910 0x7fe4b0169890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.840+0000 7fe4b8347700 1 --2- 172.21.15.89:0/2531846819 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b00b5ea0 0x7fe4b00b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.840+0000 7fe4b8347700 1 --2- 172.21.15.89:0/2531846819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4b0159c10 0x7fe4b009db80 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:13.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.840+0000 7fe4b8347700 1 -- 172.21.15.89:0/2531846819 >> 172.21.15.89:0/2531846819 conn(0x7fe4b0099a20 msgr2=0x7fe4b009aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:13.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.840+0000 7fe4b8347700 1 -- 172.21.15.89:0/2531846819 shutdown_connections 2023-10-10T20:40:13.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:13.840+0000 7fe4b8347700 1 -- 172.21.15.89:0/2531846819 wait complete. 2023-10-10T20:40:13.847 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:40:13.853 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2023-10-10T20:40:12.720026+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395876,"kb_used_data":9380,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752578844,"statfs":{"total":773094113280,"available":770640736256,"internally_reserved":0,"allocated":9605120,"data_stored":4779567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.725282"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999363+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:59:07.783164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997293+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:10:02.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000303+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:15:33.267457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000522+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:00:28.956602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998301+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:56:06.818452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000192+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:54:48.893540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999923+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998663+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:26:36.884070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998957+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:31.591354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.996718+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:17:57.416786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997206+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:03:11.358023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997183+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000423+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:06:23.747999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000508+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:01:20.723997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997163+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:00:28.956602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998250+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:56:17.858084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998262+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:28:06.502123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000675+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:32:46.206016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999401+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:14:30.053146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000613+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.001511+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:28:06.502123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.001160+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:00:28.956602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000745+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:55:55.484134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.001441+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999584+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:25.450278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998537+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:56:28.068099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999909+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:28:06.502123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999320+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:12.919153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.001698+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:04:07.489913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.996623+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:43:58.261869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000340+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:06:17.893318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997358+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:35:02.298470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":228,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-10T20:40:04.992183+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:40:04.992183+0000","last_peered":"2023-10-10T20:40:04.992183+0000","last_clean":"2023-10-10T20:40:04.992183+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:40:04.992183+0000","last_undegraded":"2023-10-10T20:40:04.992183+0000","last_fullsized":"2023-10-10T20:40:04.992183+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298896,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072944,"statfs":{"total":96636764160,"available":96330694656,"internally_reserved":0,"allocated":1646592,"data_stored":1040342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.60499999999999998,"15min":0.84999999999999998},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.747,"5min":6.8630000000000004,"15min":6.8630000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.499,"5min":0.56299999999999994,"15min":0.63800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.629,"5min":4.1959999999999997,"15min":4.1959999999999997},"last":0.49299999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.65300000000000002,"15min":1.0489999999999999},"min":{"1min":0.39900000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.78800000000000003,"5min":10.1,"15min":10.1},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.61299999999999999,"15min":0.749},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66400000000000003,"5min":6.806,"15min":6.806},"last":0.63}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.64000000000000001,"15min":0.80200000000000005},"min":{"1min":0.41599999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":6.9340000000000002,"15min":6.9340000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59499999999999997,"15min":0.75900000000000001},"min":{"1min":0.38800000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73199999999999998,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.55100000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61799999999999999,"15min":0.77800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70899999999999996,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.68100000000000005},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71899999999999997,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.56499999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.34599999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.626,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.374,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78400000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.40000000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.73299999999999998,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72699999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.38200000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58899999999999997,"15min":0.57199999999999995},"min":{"1min":0.48699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77100000000000002,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.46800000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75600000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.45900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55700000000000005,"15min":0.68899999999999995},"min":{"1min":0.40799999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.73099999999999998,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":0.432},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57899999999999996,"15min":0.73299999999999998},"min":{"1min":0.372,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70799999999999996,"5min":7.8090000000000002,"15min":7.8090000000000002},"last":0.45500000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.72899999999999998},"min":{"1min":0.39700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71899999999999997,"5min":7.867,"15min":7.867},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.64200000000000002,"15min":0.92100000000000004},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75700000000000001,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.497}]},{"osd":2,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.747,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.63400000000000001,"15min":0.874},"min":{"1min":0.48399999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.71199999999999997,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":0.57499999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.628},"min":{"1min":0.439,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79000000000000004,"5min":4.8620000000000001,"15min":4.8620000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.73499999999999999},"min":{"1min":0.47199999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.755,"5min":7.9009999999999998,"15min":7.9009999999999998},"last":0.59899999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.504,"15min":0.46400000000000002},"min":{"1min":0.42799999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71799999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.56499999999999995},"min":{"1min":0.54200000000000004,"5min":0.37,"15min":0.37},"max":{"1min":0.69899999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67600000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51400000000000001,"15min":0.48599999999999999},"min":{"1min":0.40400000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55200000000000005,"15min":0.52600000000000002},"min":{"1min":0.47799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.70999999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.42799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74399999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56899999999999995,"15min":0.54100000000000004},"min":{"1min":0.52700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.747,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.64100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.36399999999999999,"15min":0.49099999999999999},"min":{"1min":0.219,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.56799999999999995,"5min":5.0380000000000003,"15min":5.0380000000000003},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33100000000000002,"15min":0.36499999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.47299999999999998,"5min":1.194,"15min":1.194},"last":0.29299999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.36799999999999999},"min":{"1min":0.27800000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.51400000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.52000000000000002},"min":{"1min":0.26000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52500000000000002,"5min":5.0919999999999996,"15min":5.0919999999999996},"last":0.27800000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.40799999999999997,"15min":0.54300000000000004},"min":{"1min":0.28000000000000003,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.59899999999999998,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.314},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38100000000000001,"15min":0.41299999999999998},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55500000000000005,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.19900000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.40000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92400000000000004,"5min":1.254,"15min":1.254},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60599999999999998,"15min":0.71899999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96499999999999997,"5min":4.9850000000000003,"15min":4.9850000000000003},"last":0.43099999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97599999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.42899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.878,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.51500000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67600000000000005,"15min":0.86699999999999999},"min":{"1min":0.46500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.995,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.40699999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85799999999999998,"5min":1.171,"15min":1.171},"last":0.36299999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60099999999999998,"15min":0.62},"min":{"1min":0.443,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.004,"5min":1.214,"15min":1.214},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63700000000000001,"15min":0.73699999999999999},"min":{"1min":0.41999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90500000000000003,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.47699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302988,"kb_used_data":1604,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068852,"statfs":{"total":96636764160,"available":96326504448,"internally_reserved":0,"allocated":1642496,"data_stored":1039381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.377,"15min":0.55100000000000005},"min":{"1min":0.20699999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.51600000000000001,"5min":7.9249999999999998,"15min":7.9249999999999998},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.374,"15min":0.52400000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45800000000000002,"5min":6.6299999999999999,"15min":6.6299999999999999},"last":0.26400000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.39000000000000001,"15min":0.57399999999999995},"min":{"1min":0.23100000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.47499999999999998,"5min":8.2509999999999994,"15min":8.2509999999999994},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39800000000000002,"15min":0.54800000000000004},"min":{"1min":0.245,"5min":0.222,"15min":0.222},"max":{"1min":0.52400000000000002,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.33800000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.40500000000000003,"15min":0.57699999999999996},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.45400000000000001,"15min":0.63400000000000001},"min":{"1min":0.31900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57899999999999996,"5min":7.8929999999999998,"15min":7.8929999999999998},"last":0.379}]},{"osd":4,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.61099999999999999,"15min":0.73199999999999998},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78000000000000003,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.63600000000000001,"15min":0.79600000000000004},"min":{"1min":0.45100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.95399999999999996,"5min":7.6920000000000002,"15min":7.6920000000000002},"last":0.54500000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.77500000000000002},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92400000000000004,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.64300000000000002,"15min":0.79900000000000004},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91400000000000003,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":0.621}]},{"osd":6,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67000000000000004,"15min":0.83999999999999997},"min":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94499999999999995,"5min":7.8360000000000003,"15min":7.8360000000000003},"last":0.495},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66400000000000003,"15min":0.77900000000000003},"min":{"1min":0.49199999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90300000000000002,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.58799999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64000000000000001,"15min":0.76900000000000002},"min":{"1min":0.45900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93300000000000005,"5min":6.7160000000000002,"15min":6.7160000000000002},"last":0.627},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66100000000000003,"15min":0.80900000000000005},"min":{"1min":0.44600000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.878,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.56699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.31900000000000001,"15min":0.32800000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.59699999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.33600000000000002,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45400000000000001,"5min":0.746,"15min":0.746},"last":0.30499999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.45400000000000001,"15min":0.77100000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.63,"5min":11.967000000000001,"15min":11.967000000000001},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39000000000000001,"15min":0.498},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52700000000000002,"5min":4.1100000000000003,"15min":4.1100000000000003},"last":0.39800000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.41199999999999998,"15min":0.57899999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48799999999999999,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.50900000000000001,"15min":0.82199999999999995},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65900000000000003,"5min":11.935,"15min":11.935},"last":0.38300000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.40600000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.80100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.58999999999999997},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.45100000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.33400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83599999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42099999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81599999999999995,"5min":1.052,"15min":1.052},"last":0.433},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64500000000000002,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85999999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.48999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302412,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069428,"statfs":{"total":96636764160,"available":96327094272,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.309,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44400000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.317},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48699999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.35499999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33600000000000002,"15min":0.33000000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.49199999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38200000000000001,"15min":0.376},"min":{"1min":0.307,"5min":0.251,"15min":0.251},"max":{"1min":0.53900000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.34899999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.309},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35499999999999998,"15min":0.36099999999999999},"min":{"1min":0.25900000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.51600000000000001,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.28999999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.38100000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83299999999999996,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60699999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.39400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78700000000000003,"5min":1.05,"15min":1.05},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86799999999999999,"5min":1.01,"15min":1.01},"last":0.55500000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92000000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.877,"5min":1.026,"15min":1.026},"last":0.71499999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77100000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.46600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89400000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.66000000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58199999999999996,"5min":0.747,"15min":0.747},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.59799999999999998,"15min":0.79200000000000004},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69699999999999995,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.46400000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.60299999999999998,"15min":0.79900000000000004},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60299999999999998,"5min":7.7160000000000002,"15min":7.7160000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68100000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.379},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63600000000000001,"15min":0.81799999999999995},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.441}]},{"osd":3,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.58899999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48399999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.502,"15min":0.48399999999999999},"min":{"1min":0.34599999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.38400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77300000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.41699999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72099999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.52600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.501}]},{"osd":7,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.58199999999999996},"min":{"1min":0.44700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.63600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.307,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63600000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.442},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.34300000000000003,"5min":0.215,"15min":0.215},"max":{"1min":0.70399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.48499999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.56200000000000006,"15min":0.65300000000000002},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55300000000000005,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68200000000000005,"5min":5.3109999999999999,"15min":5.3109999999999999},"last":0.52100000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56899999999999995,"15min":0.66100000000000003},"min":{"1min":0.32900000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64700000000000002,"5min":5.2960000000000003,"15min":5.2960000000000003},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72399999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.46400000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59499999999999997,"15min":0.69099999999999995},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70399999999999996,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59699999999999998,"15min":0.70399999999999996},"min":{"1min":0.40300000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70599999999999996,"5min":5.2850000000000001,"15min":5.2850000000000001},"last":0.39300000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48299999999999998,"15min":0.497},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.63800000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.626},{"interface":"front","average":{"1min":0.505,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65000000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.54400000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.65100000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71299999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58299999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.33000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72699999999999998,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.60099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:40:13.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3f is not active+clean 2023-10-10T20:40:13.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3e is not active+clean 2023-10-10T20:40:13.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3d is not active+clean 2023-10-10T20:40:13.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3c is not active+clean 2023-10-10T20:40:13.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3b is not active+clean 2023-10-10T20:40:13.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3a is not active+clean 2023-10-10T20:40:13.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.39 is not active+clean 2023-10-10T20:40:13.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.38 is not active+clean 2023-10-10T20:40:13.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.37 is not active+clean 2023-10-10T20:40:13.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.36 is not active+clean 2023-10-10T20:40:13.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.35 is not active+clean 2023-10-10T20:40:13.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.34 is not active+clean 2023-10-10T20:40:13.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.33 is not active+clean 2023-10-10T20:40:13.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.32 is not active+clean 2023-10-10T20:40:13.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.31 is not active+clean 2023-10-10T20:40:13.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.30 is not active+clean 2023-10-10T20:40:13.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2f is not active+clean 2023-10-10T20:40:13.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2e is not active+clean 2023-10-10T20:40:13.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2d is not active+clean 2023-10-10T20:40:13.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2c is not active+clean 2023-10-10T20:40:13.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2b is not active+clean 2023-10-10T20:40:13.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2a is not active+clean 2023-10-10T20:40:13.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.29 is not active+clean 2023-10-10T20:40:13.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.28 is not active+clean 2023-10-10T20:40:13.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.27 is not active+clean 2023-10-10T20:40:13.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.26 is not active+clean 2023-10-10T20:40:13.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.25 is not active+clean 2023-10-10T20:40:13.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.24 is not active+clean 2023-10-10T20:40:13.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.23 is not active+clean 2023-10-10T20:40:13.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.22 is not active+clean 2023-10-10T20:40:13.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.21 is not active+clean 2023-10-10T20:40:13.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.20 is not active+clean 2023-10-10T20:40:13.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2023-10-10T20:40:13.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1f is not active+clean 2023-10-10T20:40:13.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:06.999363+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:59:07.783164+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:13.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2023-10-10T20:40:13.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2023-10-10T20:40:13.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:06.997293+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:10:02.557707+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:40:13.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1c is not active+clean 2023-10-10T20:40:13.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1d is not active+clean 2023-10-10T20:40:13.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:07.000303+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:15:33.267457+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:13.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2023-10-10T20:40:13.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1c is not active+clean 2023-10-10T20:40:13.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:07.000522+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:00:28.956602+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:13.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2023-10-10T20:40:13.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1b is not active+clean 2023-10-10T20:40:13.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:06.998301+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T20:56:06.818452+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:40:13.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2023-10-10T20:40:13.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1a is not active+clean 2023-10-10T20:40:13.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:07.000192+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:54:48.893540+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:13.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2023-10-10T20:40:13.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2023-10-10T20:40:13.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:06.999923+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:40:13.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2023-10-10T20:40:13.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.18 is not active+clean 2023-10-10T20:40:13.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:06.998663+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:26:36.884070+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:40:13.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2023-10-10T20:40:13.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.17 is not active+clean 2023-10-10T20:40:13.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:06.998957+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:17:31.591354+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:40:13.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2023-10-10T20:40:13.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.16 is not active+clean 2023-10-10T20:40:13.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:06.996718+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:17:57.416786+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:40:13.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2023-10-10T20:40:13.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2023-10-10T20:40:13.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:06.997206+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:03:11.358023+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:40:13.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2023-10-10T20:40:13.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.14 is not active+clean 2023-10-10T20:40:13.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:06.997183+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:40:13.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2023-10-10T20:40:13.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.13 is not active+clean 2023-10-10T20:40:13.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:07.000423+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:06:23.747999+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:40:13.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.13 is not active+clean 2023-10-10T20:40:13.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.12 is not active+clean 2023-10-10T20:40:13.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:07.000508+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:01:20.723997+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:40:13.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2023-10-10T20:40:13.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.11 is not active+clean 2023-10-10T20:40:13.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:06.997163+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:00:28.956602+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:40:13.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2023-10-10T20:40:13.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.10 is not active+clean 2023-10-10T20:40:13.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:06.998250+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:56:17.858084+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:40:13.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2023-10-10T20:40:13.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.913 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.e is not active+clean 2023-10-10T20:40:13.913 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:06.998262+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:28:06.502123+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:40:13.913 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2023-10-10T20:40:13.914 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.914 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2023-10-10T20:40:13.915 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:07.000675+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:32:46.206016+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:40:13.915 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2023-10-10T20:40:13.915 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.c is not active+clean 2023-10-10T20:40:13.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:06.999401+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:14:30.053146+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:13.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2023-10-10T20:40:13.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.b is not active+clean 2023-10-10T20:40:13.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:07.000613+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:13.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2023-10-10T20:40:13.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2023-10-10T20:40:13.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:07.001511+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:28:06.502123+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:40:13.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2023-10-10T20:40:13.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2023-10-10T20:40:13.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:07.001160+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:00:28.956602+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:40:13.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2023-10-10T20:40:13.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.8 is not active+clean 2023-10-10T20:40:13.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:07.000745+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T20:55:55.484134+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:13.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2023-10-10T20:40:13.927 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.928 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2023-10-10T20:40:13.928 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:07.001441+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:40:13.929 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.7 is not active+clean 2023-10-10T20:40:13.929 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.930 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.6 is not active+clean 2023-10-10T20:40:13.931 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:06.999584+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:16:25.450278+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:13.932 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.4 is not active+clean 2023-10-10T20:40:13.932 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.933 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.5 is not active+clean 2023-10-10T20:40:13.933 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:06.998537+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:56:28.068099+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:13.934 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2023-10-10T20:40:13.934 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.935 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2023-10-10T20:40:13.935 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:06.999909+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:28:06.502123+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:40:13.935 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2023-10-10T20:40:13.936 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.936 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3 is not active+clean 2023-10-10T20:40:13.936 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:06.999320+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:32:12.919153+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:13.937 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2023-10-10T20:40:13.937 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.938 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2023-10-10T20:40:13.938 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:07.001698+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:04:07.489913+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:40:13.939 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2023-10-10T20:40:13.939 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.939 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1 is not active+clean 2023-10-10T20:40:13.940 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:06.996623+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:43:58.261869+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:40:13.940 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2023-10-10T20:40:13.940 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.941 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.0 is not active+clean 2023-10-10T20:40:13.941 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:07.000340+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:06:17.893318+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:40:13.942 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2023-10-10T20:40:13.942 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:40:07.994486+0000', 'last_change': '2023-10-10T20:40:07.994486+0000', 'last_active': '2023-10-10T20:40:07.994486+0000', 'last_peered': '2023-10-10T20:40:07.994486+0000', 'last_clean': '2023-10-10T20:40:07.994486+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:07.994486+0000', 'last_undegraded': '2023-10-10T20:40:07.994486+0000', 'last_fullsized': '2023-10-10T20:40:07.994486+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:07.994486+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:40:13.942 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2023-10-10T20:40:13.943 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:06.990727+0000', 'last_change': '2023-10-10T20:40:06.997358+0000', 'last_active': '2023-10-10T20:40:06.990727+0000', 'last_peered': '2023-10-10T20:40:06.990727+0000', 'last_clean': '2023-10-10T20:40:06.990727+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:06.990727+0000', 'last_undegraded': '2023-10-10T20:40:06.990727+0000', 'last_fullsized': '2023-10-10T20:40:06.990727+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:06.990727+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:35:02.298470+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:40:13.944 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-10T20:40:14.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.056+0000 7f4c60b54700 1 Processor -- start 2023-10-10T20:40:14.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.056+0000 7f4c60b54700 1 -- start start 2023-10-10T20:40:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.057+0000 7f4c60b54700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4c5c06ab40 0x7f4c5c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.057+0000 7f4c60b54700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4c5c0af8c0 0x7f4c5c159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.057+0000 7f4c60b54700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c5c068900 con 0x7f4c5c068390 2023-10-10T20:40:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.057+0000 7f4c60b54700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c5c068be0 con 0x7f4c5c0af8c0 2023-10-10T20:40:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.057+0000 7f4c60b54700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c5c068ec0 con 0x7f4c5c06ab40 2023-10-10T20:40:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.057+0000 7f4c5ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4c5c0af8c0 0x7f4c5c159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.057+0000 7f4c5ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4c5c0af8c0 0x7f4c5c159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43478/0 (socket says 172.21.15.89:43478) 2023-10-10T20:40:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.057+0000 7f4c5ad9d700 1 -- 172.21.15.89:0/3656191124 learned_addr learned my addr 172.21.15.89:0/3656191124 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.058+0000 7f4c5a59c700 1 --2- 172.21.15.89:0/3656191124 >> v2:172.21.15.111:3300/0 conn(0x7f4c5c06ab40 0x7f4c5c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.058+0000 7f4c5ad9d700 1 -- 172.21.15.89:0/3656191124 >> v2:172.21.15.111:3300/0 conn(0x7f4c5c06ab40 msgr2=0x7f4c5c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.058+0000 7f4c5ad9d700 1 --2- 172.21.15.89:0/3656191124 >> v2:172.21.15.111:3300/0 conn(0x7f4c5c06ab40 0x7f4c5c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.058+0000 7f4c5ad9d700 1 -- 172.21.15.89:0/3656191124 >> v1:172.21.15.111:6789/0 conn(0x7f4c5c068390 legacy=0x7f4c5c0c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.058+0000 7f4c5ad9d700 1 -- 172.21.15.89:0/3656191124 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5c0691a0 con 0x7f4c5c0af8c0 2023-10-10T20:40:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.058+0000 7f4c5ad9d700 1 --2- 172.21.15.89:0/3656191124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c5c0af8c0 0x7f4c5c159620 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0ce6b0 tx=0x7f4c54009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd852f5d0fbfab2f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.058+0000 7f4c59d9b700 1 -- 172.21.15.89:0/3656191124 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5400e070 con 0x7f4c5c0af8c0 2023-10-10T20:40:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.058+0000 7f4c59d9b700 1 -- 172.21.15.89:0/3656191124 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5400bae0 con 0x7f4c5c0af8c0 2023-10-10T20:40:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.058+0000 7f4c59d9b700 1 -- 172.21.15.89:0/3656191124 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c540092e0 con 0x7f4c5c0af8c0 2023-10-10T20:40:14.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.058+0000 7f4c60b54700 1 -- 172.21.15.89:0/3656191124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c5c0af8c0 msgr2=0x7f4c5c159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:14.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.058+0000 7f4c60b54700 1 --2- 172.21.15.89:0/3656191124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c5c0af8c0 0x7f4c5c159620 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0ce6b0 tx=0x7f4c54009840 comp rx=0 tx=0).stop 2023-10-10T20:40:14.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.058+0000 7f4c60b54700 1 -- 172.21.15.89:0/3656191124 shutdown_connections 2023-10-10T20:40:14.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.058+0000 7f4c60b54700 1 --2- 172.21.15.89:0/3656191124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c5c0af8c0 0x7f4c5c159620 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.058+0000 7f4c60b54700 1 --2- 172.21.15.89:0/3656191124 >> v2:172.21.15.111:3300/0 conn(0x7f4c5c06ab40 0x7f4c5c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.058+0000 7f4c60b54700 1 -- 172.21.15.89:0/3656191124 >> 172.21.15.89:0/3656191124 conn(0x7f4c5c0cd640 msgr2=0x7f4c5c0cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:14.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.058+0000 7f4c60b54700 1 -- 172.21.15.89:0/3656191124 shutdown_connections 2023-10-10T20:40:14.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.058+0000 7f4c60b54700 1 -- 172.21.15.89:0/3656191124 wait complete. 2023-10-10T20:40:14.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.060+0000 7f4c60b54700 1 Processor -- start 2023-10-10T20:40:14.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.060+0000 7f4c60b54700 1 -- start start 2023-10-10T20:40:14.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.060+0000 7f4c60b54700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c5c068390 0x7f4c5c1586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:14.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.060+0000 7f4c60b54700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c5c06ab40 0x7f4c5c0ae070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:14.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.060+0000 7f4c60b54700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c5c0af8c0 0x7f4c5c15a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:14.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.060+0000 7f4c60b54700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c0adbe0 con 0x7f4c5c0af8c0 2023-10-10T20:40:14.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.060+0000 7f4c60b54700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c0acfd0 con 0x7f4c5c06ab40 2023-10-10T20:40:14.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.060+0000 7f4c60b54700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c0ad2b0 con 0x7f4c5c068390 2023-10-10T20:40:14.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.060+0000 7f4c5ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c5c0af8c0 0x7f4c5c15a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:14.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.060+0000 7f4c5ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c5c0af8c0 0x7f4c5c15a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43484/0 (socket says 172.21.15.89:43484) 2023-10-10T20:40:14.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.060+0000 7f4c5ad9d700 1 -- 172.21.15.89:0/4224966701 learned_addr learned my addr 172.21.15.89:0/4224966701 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:14.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.061+0000 7f4c5a59c700 1 --2- 172.21.15.89:0/4224966701 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c5c068390 0x7f4c5c1586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:14.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.061+0000 7f4c51d9b700 1 --2- 172.21.15.89:0/4224966701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c5c06ab40 0x7f4c5c0ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:14.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.061+0000 7f4c5ad9d700 1 -- 172.21.15.89:0/4224966701 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c5c068390 msgr2=0x7f4c5c1586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:14.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.061+0000 7f4c5ad9d700 1 --2- 172.21.15.89:0/4224966701 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c5c068390 0x7f4c5c1586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.061+0000 7f4c5ad9d700 1 -- 172.21.15.89:0/4224966701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c5c06ab40 msgr2=0x7f4c5c0ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.061+0000 7f4c5ad9d700 1 --2- 172.21.15.89:0/4224966701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c5c06ab40 0x7f4c5c0ae070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.061+0000 7f4c5ad9d700 1 -- 172.21.15.89:0/4224966701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c54009ae0 con 0x7f4c5c0af8c0 2023-10-10T20:40:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.061+0000 7f4c51d9b700 1 --2- 172.21.15.89:0/4224966701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c5c06ab40 0x7f4c5c0ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.061+0000 7f4c5ad9d700 1 --2- 172.21.15.89:0/4224966701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c5c0af8c0 0x7f4c5c15a4f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c071430 tx=0x7f4c54018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.061+0000 7f4c52ffd700 1 -- 172.21.15.89:0/4224966701 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c540032f0 con 0x7f4c5c0af8c0 2023-10-10T20:40:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.061+0000 7f4c52ffd700 1 -- 172.21.15.89:0/4224966701 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c54008dd0 con 0x7f4c5c0af8c0 2023-10-10T20:40:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.061+0000 7f4c60b54700 1 -- 172.21.15.89:0/4224966701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5c0ad590 con 0x7f4c5c0af8c0 2023-10-10T20:40:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.061+0000 7f4c52ffd700 1 -- 172.21.15.89:0/4224966701 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5401c640 con 0x7f4c5c0af8c0 2023-10-10T20:40:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.061+0000 7f4c60b54700 1 -- 172.21.15.89:0/4224966701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5c158bf0 con 0x7f4c5c0af8c0 2023-10-10T20:40:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.062+0000 7f4c60b54700 1 -- 172.21.15.89:0/4224966701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c5c04c3c0 con 0x7f4c5c0af8c0 2023-10-10T20:40:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.062+0000 7f4c52ffd700 1 -- 172.21.15.89:0/4224966701 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f4c54003040 con 0x7f4c5c0af8c0 2023-10-10T20:40:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.063+0000 7f4c52ffd700 1 --2- 172.21.15.89:0/4224966701 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c3c067200 0x7f4c3c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.063+0000 7f4c5a59c700 1 --2- 172.21.15.89:0/4224966701 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c3c067200 0x7f4c3c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.063+0000 7f4c52ffd700 1 -- 172.21.15.89:0/4224966701 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4c54086310 con 0x7f4c5c0af8c0 2023-10-10T20:40:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.063+0000 7f4c5a59c700 1 --2- 172.21.15.89:0/4224966701 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c3c067200 0x7f4c3c0696b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c006fd0 tx=0x7f4c4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.067+0000 7f4c52ffd700 1 -- 172.21.15.89:0/4224966701 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c54053120 con 0x7f4c5c0af8c0 2023-10-10T20:40:14.316 INFO:tasks.ceph.mds.d.smithi111.stderr:2023-10-10T20:40:14.314+0000 7f5b6f48c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:40:14.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.322+0000 7f4c60b54700 1 -- 172.21.15.89:0/4224966701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f4c5c0add20 con 0x7f4c5c0af8c0 2023-10-10T20:40:14.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.323+0000 7f4c52ffd700 1 -- 172.21.15.89:0/4224966701 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1111 (secure 0 0 0) 0x7f4c54056740 con 0x7f4c5c0af8c0 2023-10-10T20:40:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:14.326 INFO:teuthology.orchestra.run.smithi089.stdout:{"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":357,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":88,"num_osds":8,"num_up_osds":8,"osd_up_since":1696970067,"num_in_osds":8,"osd_in_since":1696970059,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2453377024,"bytes_avail":770640736256,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":55,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":6462}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-10-10T20:37:28.685504+0000","services":{}},"progress_events":{}} 2023-10-10T20:40:14.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.325+0000 7f4c60b54700 1 -- 172.21.15.89:0/4224966701 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c3c067200 msgr2=0x7f4c3c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:14.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.325+0000 7f4c60b54700 1 --2- 172.21.15.89:0/4224966701 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c3c067200 0x7f4c3c0696b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c006fd0 tx=0x7f4c4c008040 comp rx=0 tx=0).stop 2023-10-10T20:40:14.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.325+0000 7f4c60b54700 1 -- 172.21.15.89:0/4224966701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c5c0af8c0 msgr2=0x7f4c5c15a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:14.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.325+0000 7f4c60b54700 1 --2- 172.21.15.89:0/4224966701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c5c0af8c0 0x7f4c5c15a4f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c071430 tx=0x7f4c54018000 comp rx=0 tx=0).stop 2023-10-10T20:40:14.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.325+0000 7f4c60b54700 1 -- 172.21.15.89:0/4224966701 shutdown_connections 2023-10-10T20:40:14.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.325+0000 7f4c60b54700 1 --2- 172.21.15.89:0/4224966701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c5c0af8c0 0x7f4c5c15a4f0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.325+0000 7f4c60b54700 1 --2- 172.21.15.89:0/4224966701 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c5c068390 0x7f4c5c1586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.325+0000 7f4c60b54700 1 --2- 172.21.15.89:0/4224966701 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c3c067200 0x7f4c3c0696b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.325+0000 7f4c60b54700 1 --2- 172.21.15.89:0/4224966701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c5c06ab40 0x7f4c5c0ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.325+0000 7f4c60b54700 1 -- 172.21.15.89:0/4224966701 >> 172.21.15.89:0/4224966701 conn(0x7f4c5c0cd640 msgr2=0x7f4c5c0b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:14.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.325+0000 7f4c60b54700 1 -- 172.21.15.89:0/4224966701 shutdown_connections 2023-10-10T20:40:14.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.325+0000 7f4c60b54700 1 -- 172.21.15.89:0/4224966701 wait complete. 2023-10-10T20:40:14.339 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-10T20:40:14.339 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:40:14.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.452+0000 7fea235ca700 1 Processor -- start 2023-10-10T20:40:14.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.452+0000 7fea235ca700 1 -- start start 2023-10-10T20:40:14.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.453+0000 7fea235ca700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fea1c159890 0x7fea1c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:14.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.453+0000 7fea235ca700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fea1c159c60 0x7fea1c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:14.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.453+0000 7fea235ca700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1c15a160 con 0x7fea1c15db10 2023-10-10T20:40:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.453+0000 7fea235ca700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1c0d13a0 con 0x7fea1c159890 2023-10-10T20:40:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.453+0000 7fea235ca700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c15e140 con 0x7fea1c159c60 2023-10-10T20:40:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.453+0000 7fea21b67700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fea1c15db10 0x7fea1c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55738/0 (socket says 172.21.15.89:55738) 2023-10-10T20:40:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.453+0000 7fea21b67700 1 -- 172.21.15.89:0/779095306 learned_addr learned my addr 172.21.15.89:0/779095306 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.453+0000 7fea20b65700 1 --2- 172.21.15.89:0/779095306 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fea1c159c60 0x7fea1c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.453+0000 7fea21366700 1 --2- 172.21.15.89:0/779095306 >> v2:172.21.15.111:3300/0 conn(0x7fea1c159890 0x7fea1c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.453+0000 7fea13fff700 1 -- 172.21.15.89:0/779095306 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 807548238 0 0) 0x7fea1c15a160 con 0x7fea1c15db10 2023-10-10T20:40:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.453+0000 7fea13fff700 1 -- 172.21.15.89:0/779095306 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea08003570 con 0x7fea1c15db10 2023-10-10T20:40:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.453+0000 7fea13fff700 1 -- 172.21.15.89:0/779095306 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3724254667 0 0) 0x7fea08003570 con 0x7fea1c15db10 2023-10-10T20:40:14.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.453+0000 7fea20b65700 1 -- 172.21.15.89:0/779095306 >> v2:172.21.15.111:3300/0 conn(0x7fea1c159890 msgr2=0x7fea1c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:14.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.453+0000 7fea20b65700 1 --2- 172.21.15.89:0/779095306 >> v2:172.21.15.111:3300/0 conn(0x7fea1c159890 0x7fea1c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.453+0000 7fea20b65700 1 -- 172.21.15.89:0/779095306 >> v1:172.21.15.89:6789/0 conn(0x7fea1c15db10 legacy=0x7fea1c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:14.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.453+0000 7fea20b65700 1 -- 172.21.15.89:0/779095306 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c15a3d0 con 0x7fea1c159c60 2023-10-10T20:40:14.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.453+0000 7fea21366700 1 --2- 172.21.15.89:0/779095306 >> v2:172.21.15.111:3300/0 conn(0x7fea1c159890 0x7fea1c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:40:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.454+0000 7fea20b65700 1 --2- 172.21.15.89:0/779095306 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fea1c159c60 0x7fea1c15d5d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0098e0 tx=0x7fea0c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5d999a8ea8bb42b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.454+0000 7fea13fff700 1 -- 172.21.15.89:0/779095306 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea140017f0 con 0x7fea1c159c60 2023-10-10T20:40:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.454+0000 7fea13fff700 1 -- 172.21.15.89:0/779095306 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0c011420 con 0x7fea1c159c60 2023-10-10T20:40:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.454+0000 7fea13fff700 1 -- 172.21.15.89:0/779095306 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c0117a0 con 0x7fea1c159c60 2023-10-10T20:40:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.454+0000 7fea235ca700 1 -- 172.21.15.89:0/779095306 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fea1c159c60 msgr2=0x7fea1c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:14.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.454+0000 7fea235ca700 1 --2- 172.21.15.89:0/779095306 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fea1c159c60 0x7fea1c15d5d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0098e0 tx=0x7fea0c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.454+0000 7fea235ca700 1 -- 172.21.15.89:0/779095306 shutdown_connections 2023-10-10T20:40:14.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.454+0000 7fea235ca700 1 --2- 172.21.15.89:0/779095306 >> v2:172.21.15.111:3300/0 conn(0x7fea1c159890 0x7fea1c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.454+0000 7fea235ca700 1 --2- 172.21.15.89:0/779095306 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fea1c159c60 0x7fea1c15d5d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.454+0000 7fea235ca700 1 -- 172.21.15.89:0/779095306 >> 172.21.15.89:0/779095306 conn(0x7fea1c068e20 msgr2=0x7fea1c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:14.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.454+0000 7fea235ca700 1 -- 172.21.15.89:0/779095306 shutdown_connections 2023-10-10T20:40:14.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.454+0000 7fea235ca700 1 -- 172.21.15.89:0/779095306 wait complete. 2023-10-10T20:40:14.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.456+0000 7fea235ca700 1 Processor -- start 2023-10-10T20:40:14.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.456+0000 7fea235ca700 1 -- start start 2023-10-10T20:40:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.456+0000 7fea235ca700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea1c159890 0x7fea1c0b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.456+0000 7fea235ca700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fea1c159c60 0x7fea1c0c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.456+0000 7fea235ca700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea1c15db10 0x7fea1c1523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.456+0000 7fea235ca700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c0b6120 con 0x7fea1c159890 2023-10-10T20:40:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.456+0000 7fea235ca700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c0cdaf0 con 0x7fea1c15db10 2023-10-10T20:40:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.456+0000 7fea235ca700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c0cddd0 con 0x7fea1c159c60 2023-10-10T20:40:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.457+0000 7fea21366700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea1c159890 0x7fea1c0b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.457+0000 7fea21366700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea1c159890 0x7fea1c0b5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43498/0 (socket says 172.21.15.89:43498) 2023-10-10T20:40:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.457+0000 7fea21366700 1 -- 172.21.15.89:0/4054231840 learned_addr learned my addr 172.21.15.89:0/4054231840 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:14.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.457+0000 7fea20b65700 1 --2- 172.21.15.89:0/4054231840 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fea1c159c60 0x7fea1c0c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:14.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.457+0000 7fea21b67700 1 --2- 172.21.15.89:0/4054231840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea1c15db10 0x7fea1c1523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:14.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.457+0000 7fea21366700 1 -- 172.21.15.89:0/4054231840 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fea1c159c60 msgr2=0x7fea1c0c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:14.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.457+0000 7fea21366700 1 --2- 172.21.15.89:0/4054231840 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fea1c159c60 0x7fea1c0c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.457+0000 7fea21366700 1 -- 172.21.15.89:0/4054231840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea1c15db10 msgr2=0x7fea1c1523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.457+0000 7fea21366700 1 --2- 172.21.15.89:0/4054231840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea1c15db10 0x7fea1c1523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.457+0000 7fea21366700 1 -- 172.21.15.89:0/4054231840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea18009420 con 0x7fea1c159890 2023-10-10T20:40:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.457+0000 7fea20b65700 1 --2- 172.21.15.89:0/4054231840 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fea1c159c60 0x7fea1c0c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.457+0000 7fea21b67700 1 --2- 172.21.15.89:0/4054231840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea1c15db10 0x7fea1c1523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.457+0000 7fea21366700 1 --2- 172.21.15.89:0/4054231840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea1c159890 0x7fea1c0b5b00 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fea1800ec80 tx=0x7fea180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.457+0000 7fea117fa700 1 -- 172.21.15.89:0/4054231840 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1800c9d0 con 0x7fea1c159890 2023-10-10T20:40:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.457+0000 7fea117fa700 1 -- 172.21.15.89:0/4054231840 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1800cb30 con 0x7fea1c159890 2023-10-10T20:40:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.457+0000 7fea235ca700 1 -- 172.21.15.89:0/4054231840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0c009420 con 0x7fea1c159890 2023-10-10T20:40:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.457+0000 7fea117fa700 1 -- 172.21.15.89:0/4054231840 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1800ceb0 con 0x7fea1c159890 2023-10-10T20:40:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.457+0000 7fea235ca700 1 -- 172.21.15.89:0/4054231840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1c0ce160 con 0x7fea1c159890 2023-10-10T20:40:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.458+0000 7fea235ca700 1 -- 172.21.15.89:0/4054231840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea1c0c8020 con 0x7fea1c159890 2023-10-10T20:40:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.459+0000 7fea117fa700 1 -- 172.21.15.89:0/4054231840 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fea18013680 con 0x7fea1c159890 2023-10-10T20:40:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.459+0000 7fea117fa700 1 --2- 172.21.15.89:0/4054231840 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fea00067200 0x7fea000696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.459+0000 7fea117fa700 1 -- 172.21.15.89:0/4054231840 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fea18084eb0 con 0x7fea1c159890 2023-10-10T20:40:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.459+0000 7fea20b65700 1 --2- 172.21.15.89:0/4054231840 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fea00067200 0x7fea000696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.459+0000 7fea20b65700 1 --2- 172.21.15.89:0/4054231840 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fea00067200 0x7fea000696b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fea1c15a090 tx=0x7fea0c016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.463+0000 7fea117fa700 1 -- 172.21.15.89:0/4054231840 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea18051cf0 con 0x7fea1c159890 2023-10-10T20:40:14.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.642+0000 7fea235ca700 1 -- 172.21.15.89:0/4054231840 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fea1c0cdc60 con 0x7fea00067200 2023-10-10T20:40:14.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.649+0000 7fea117fa700 1 -- 172.21.15.89:0/4054231840 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279189 (secure 0 0 0) 0x7fea1c0cdc60 con 0x7fea00067200 2023-10-10T20:40:14.655 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:14.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.652+0000 7fea235ca700 1 -- 172.21.15.89:0/4054231840 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fea00067200 msgr2=0x7fea000696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:14.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.652+0000 7fea235ca700 1 --2- 172.21.15.89:0/4054231840 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fea00067200 0x7fea000696b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fea1c15a090 tx=0x7fea0c016000 comp rx=0 tx=0).stop 2023-10-10T20:40:14.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.652+0000 7fea235ca700 1 -- 172.21.15.89:0/4054231840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea1c159890 msgr2=0x7fea1c0b5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:14.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.652+0000 7fea235ca700 1 --2- 172.21.15.89:0/4054231840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea1c159890 0x7fea1c0b5b00 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fea1800ec80 tx=0x7fea180091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.652+0000 7fea235ca700 1 -- 172.21.15.89:0/4054231840 shutdown_connections 2023-10-10T20:40:14.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.652+0000 7fea235ca700 1 --2- 172.21.15.89:0/4054231840 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fea00067200 0x7fea000696b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.652+0000 7fea235ca700 1 --2- 172.21.15.89:0/4054231840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea1c15db10 0x7fea1c1523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.652+0000 7fea235ca700 1 --2- 172.21.15.89:0/4054231840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea1c159890 0x7fea1c0b5b00 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.652+0000 7fea235ca700 1 --2- 172.21.15.89:0/4054231840 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fea1c159c60 0x7fea1c0c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.652+0000 7fea235ca700 1 -- 172.21.15.89:0/4054231840 >> 172.21.15.89:0/4054231840 conn(0x7fea1c068e20 msgr2=0x7fea1c156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.652+0000 7fea235ca700 1 -- 172.21.15.89:0/4054231840 shutdown_connections 2023-10-10T20:40:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:14.652+0000 7fea235ca700 1 -- 172.21.15.89:0/4054231840 wait complete. 2023-10-10T20:40:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:40:14.666 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2023-10-10T20:40:12.720026+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395876,"kb_used_data":9380,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752578844,"statfs":{"total":773094113280,"available":770640736256,"internally_reserved":0,"allocated":9605120,"data_stored":4779567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.725282"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999363+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:59:07.783164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997293+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:10:02.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000303+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:15:33.267457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000522+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:00:28.956602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998301+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:56:06.818452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000192+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:54:48.893540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999923+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998663+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:26:36.884070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998957+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:31.591354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.996718+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:17:57.416786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997206+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:03:11.358023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997183+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000423+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:06:23.747999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000508+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:01:20.723997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997163+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:00:28.956602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998250+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:56:17.858084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998262+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:28:06.502123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000675+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:32:46.206016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999401+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:14:30.053146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000613+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.001511+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:28:06.502123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.001160+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:00:28.956602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000745+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:55:55.484134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.001441+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:10.424942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999584+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:25.450278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.998537+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:56:28.068099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999909+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:28:06.502123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.999320+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:12.919153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.001698+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:04:07.489913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.996623+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:43:58.261869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:07.000340+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:06:17.893318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:40:07.994486+0000","last_change":"2023-10-10T20:40:07.994486+0000","last_active":"2023-10-10T20:40:07.994486+0000","last_peered":"2023-10-10T20:40:07.994486+0000","last_clean":"2023-10-10T20:40:07.994486+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:07.994486+0000","last_undegraded":"2023-10-10T20:40:07.994486+0000","last_fullsized":"2023-10-10T20:40:07.994486+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:06.990727+0000","last_change":"2023-10-10T20:40:06.997358+0000","last_active":"2023-10-10T20:40:06.990727+0000","last_peered":"2023-10-10T20:40:06.990727+0000","last_clean":"2023-10-10T20:40:06.990727+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:06.990727+0000","last_undegraded":"2023-10-10T20:40:06.990727+0000","last_fullsized":"2023-10-10T20:40:06.990727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:35:02.298470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":228,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-10T20:40:04.992183+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:40:04.992183+0000","last_peered":"2023-10-10T20:40:04.992183+0000","last_clean":"2023-10-10T20:40:04.992183+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:40:04.992183+0000","last_undegraded":"2023-10-10T20:40:04.992183+0000","last_fullsized":"2023-10-10T20:40:04.992183+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298896,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072944,"statfs":{"total":96636764160,"available":96330694656,"internally_reserved":0,"allocated":1646592,"data_stored":1040342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.60499999999999998,"15min":0.84999999999999998},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.747,"5min":6.8630000000000004,"15min":6.8630000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.499,"5min":0.56299999999999994,"15min":0.63800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.629,"5min":4.1959999999999997,"15min":4.1959999999999997},"last":0.49299999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.65300000000000002,"15min":1.0489999999999999},"min":{"1min":0.39900000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.78800000000000003,"5min":10.1,"15min":10.1},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.61299999999999999,"15min":0.749},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66400000000000003,"5min":6.806,"15min":6.806},"last":0.63}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.64000000000000001,"15min":0.80200000000000005},"min":{"1min":0.41599999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":6.9340000000000002,"15min":6.9340000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59499999999999997,"15min":0.75900000000000001},"min":{"1min":0.38800000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73199999999999998,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.55100000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61799999999999999,"15min":0.77800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70899999999999996,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.68100000000000005},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71899999999999997,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.56499999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.34599999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.626,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.374,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78400000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.40000000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.73299999999999998,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72699999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.38200000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58899999999999997,"15min":0.57199999999999995},"min":{"1min":0.48699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77100000000000002,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.46800000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75600000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.45900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55700000000000005,"15min":0.68899999999999995},"min":{"1min":0.40799999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.73099999999999998,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":0.432},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57899999999999996,"15min":0.73299999999999998},"min":{"1min":0.372,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70799999999999996,"5min":7.8090000000000002,"15min":7.8090000000000002},"last":0.45500000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.72899999999999998},"min":{"1min":0.39700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71899999999999997,"5min":7.867,"15min":7.867},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.64200000000000002,"15min":0.92100000000000004},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75700000000000001,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.497}]},{"osd":2,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.747,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.63400000000000001,"15min":0.874},"min":{"1min":0.48399999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.71199999999999997,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":0.57499999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.628},"min":{"1min":0.439,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79000000000000004,"5min":4.8620000000000001,"15min":4.8620000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.73499999999999999},"min":{"1min":0.47199999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.755,"5min":7.9009999999999998,"15min":7.9009999999999998},"last":0.59899999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.504,"15min":0.46400000000000002},"min":{"1min":0.42799999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71799999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.56499999999999995},"min":{"1min":0.54200000000000004,"5min":0.37,"15min":0.37},"max":{"1min":0.69899999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67600000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51400000000000001,"15min":0.48599999999999999},"min":{"1min":0.40400000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55200000000000005,"15min":0.52600000000000002},"min":{"1min":0.47799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.70999999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.42799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74399999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56899999999999995,"15min":0.54100000000000004},"min":{"1min":0.52700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.747,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.64100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.36399999999999999,"15min":0.49099999999999999},"min":{"1min":0.219,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.56799999999999995,"5min":5.0380000000000003,"15min":5.0380000000000003},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33100000000000002,"15min":0.36499999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.47299999999999998,"5min":1.194,"15min":1.194},"last":0.29299999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.36799999999999999},"min":{"1min":0.27800000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.51400000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.52000000000000002},"min":{"1min":0.26000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52500000000000002,"5min":5.0919999999999996,"15min":5.0919999999999996},"last":0.27800000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.40799999999999997,"15min":0.54300000000000004},"min":{"1min":0.28000000000000003,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.59899999999999998,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.314},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38100000000000001,"15min":0.41299999999999998},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55500000000000005,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.19900000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.40000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92400000000000004,"5min":1.254,"15min":1.254},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60599999999999998,"15min":0.71899999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96499999999999997,"5min":4.9850000000000003,"15min":4.9850000000000003},"last":0.43099999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97599999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.42899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.878,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.51500000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67600000000000005,"15min":0.86699999999999999},"min":{"1min":0.46500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.995,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.40699999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85799999999999998,"5min":1.171,"15min":1.171},"last":0.36299999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60099999999999998,"15min":0.62},"min":{"1min":0.443,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.004,"5min":1.214,"15min":1.214},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63700000000000001,"15min":0.73699999999999999},"min":{"1min":0.41999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90500000000000003,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.47699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302988,"kb_used_data":1604,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068852,"statfs":{"total":96636764160,"available":96326504448,"internally_reserved":0,"allocated":1642496,"data_stored":1039381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.377,"15min":0.55100000000000005},"min":{"1min":0.20699999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.51600000000000001,"5min":7.9249999999999998,"15min":7.9249999999999998},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.374,"15min":0.52400000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45800000000000002,"5min":6.6299999999999999,"15min":6.6299999999999999},"last":0.26400000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.39000000000000001,"15min":0.57399999999999995},"min":{"1min":0.23100000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.47499999999999998,"5min":8.2509999999999994,"15min":8.2509999999999994},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39800000000000002,"15min":0.54800000000000004},"min":{"1min":0.245,"5min":0.222,"15min":0.222},"max":{"1min":0.52400000000000002,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.33800000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.40500000000000003,"15min":0.57699999999999996},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.45400000000000001,"15min":0.63400000000000001},"min":{"1min":0.31900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57899999999999996,"5min":7.8929999999999998,"15min":7.8929999999999998},"last":0.379}]},{"osd":4,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.61099999999999999,"15min":0.73199999999999998},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78000000000000003,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.63600000000000001,"15min":0.79600000000000004},"min":{"1min":0.45100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.95399999999999996,"5min":7.6920000000000002,"15min":7.6920000000000002},"last":0.54500000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.77500000000000002},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92400000000000004,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.64300000000000002,"15min":0.79900000000000004},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91400000000000003,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":0.621}]},{"osd":6,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67000000000000004,"15min":0.83999999999999997},"min":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94499999999999995,"5min":7.8360000000000003,"15min":7.8360000000000003},"last":0.495},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66400000000000003,"15min":0.77900000000000003},"min":{"1min":0.49199999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90300000000000002,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.58799999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64000000000000001,"15min":0.76900000000000002},"min":{"1min":0.45900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93300000000000005,"5min":6.7160000000000002,"15min":6.7160000000000002},"last":0.627},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66100000000000003,"15min":0.80900000000000005},"min":{"1min":0.44600000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.878,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.56699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.31900000000000001,"15min":0.32800000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.59699999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.33600000000000002,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45400000000000001,"5min":0.746,"15min":0.746},"last":0.30499999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.45400000000000001,"15min":0.77100000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.63,"5min":11.967000000000001,"15min":11.967000000000001},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39000000000000001,"15min":0.498},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52700000000000002,"5min":4.1100000000000003,"15min":4.1100000000000003},"last":0.39800000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.41199999999999998,"15min":0.57899999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48799999999999999,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.50900000000000001,"15min":0.82199999999999995},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65900000000000003,"5min":11.935,"15min":11.935},"last":0.38300000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.40600000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.80100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.58999999999999997},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.45100000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.33400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83599999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42099999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81599999999999995,"5min":1.052,"15min":1.052},"last":0.433},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64500000000000002,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85999999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.48999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302412,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069428,"statfs":{"total":96636764160,"available":96327094272,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.309,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44400000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.317},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48699999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.35499999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33600000000000002,"15min":0.33000000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.49199999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38200000000000001,"15min":0.376},"min":{"1min":0.307,"5min":0.251,"15min":0.251},"max":{"1min":0.53900000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.34899999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.309},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35499999999999998,"15min":0.36099999999999999},"min":{"1min":0.25900000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.51600000000000001,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.28999999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.38100000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83299999999999996,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60699999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.39400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78700000000000003,"5min":1.05,"15min":1.05},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86799999999999999,"5min":1.01,"15min":1.01},"last":0.55500000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92000000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.877,"5min":1.026,"15min":1.026},"last":0.71499999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77100000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.46600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89400000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.66000000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58199999999999996,"5min":0.747,"15min":0.747},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.59799999999999998,"15min":0.79200000000000004},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69699999999999995,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.46400000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.60299999999999998,"15min":0.79900000000000004},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60299999999999998,"5min":7.7160000000000002,"15min":7.7160000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68100000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.379},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63600000000000001,"15min":0.81799999999999995},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.441}]},{"osd":3,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.58899999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48399999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.502,"15min":0.48399999999999999},"min":{"1min":0.34599999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.38400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77300000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.41699999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72099999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.52600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.501}]},{"osd":7,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.58199999999999996},"min":{"1min":0.44700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.63600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073524,"statfs":{"total":96636764160,"available":96331288576,"internally_reserved":0,"allocated":1052672,"data_stored":449974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.307,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63600000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.442},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.34300000000000003,"5min":0.215,"15min":0.215},"max":{"1min":0.70399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.48499999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.56200000000000006,"15min":0.65300000000000002},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55300000000000005,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68200000000000005,"5min":5.3109999999999999,"15min":5.3109999999999999},"last":0.52100000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56899999999999995,"15min":0.66100000000000003},"min":{"1min":0.32900000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64700000000000002,"5min":5.2960000000000003,"15min":5.2960000000000003},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72399999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.46400000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59499999999999997,"15min":0.69099999999999995},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70399999999999996,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59699999999999998,"15min":0.70399999999999996},"min":{"1min":0.40300000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70599999999999996,"5min":5.2850000000000001,"15min":5.2850000000000001},"last":0.39300000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48299999999999998,"15min":0.497},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.63800000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.626},{"interface":"front","average":{"1min":0.505,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65000000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.54400000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.65100000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71299999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58299999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.33000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72699999999999998,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.60099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:40:17.672 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:40:17.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.797+0000 7f7056543700 1 Processor -- start 2023-10-10T20:40:17.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.797+0000 7f7056543700 1 -- start start 2023-10-10T20:40:17.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.798+0000 7f7056543700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7050159890 0x7f705015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:17.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.798+0000 7f7056543700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7050159c60 0x7f705015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:17.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.798+0000 7f7056543700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705015a160 con 0x7f705015db10 2023-10-10T20:40:17.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.798+0000 7f7056543700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70500d13a0 con 0x7f7050159c60 2023-10-10T20:40:17.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.798+0000 7f7056543700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705015e140 con 0x7f7050159890 2023-10-10T20:40:17.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.798+0000 7f7054ae0700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f705015db10 0x7f7050069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55754/0 (socket says 172.21.15.89:55754) 2023-10-10T20:40:17.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.798+0000 7f7054ae0700 1 -- 172.21.15.89:0/2904947585 learned_addr learned my addr 172.21.15.89:0/2904947585 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:17.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.798+0000 7f704ffff700 1 --2- 172.21.15.89:0/2904947585 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7050159890 0x7f705015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:17.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.798+0000 7f704f7fe700 1 --2- 172.21.15.89:0/2904947585 >> v2:172.21.15.111:3300/0 conn(0x7f7050159c60 0x7f705015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:17.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.798+0000 7f704effd700 1 -- 172.21.15.89:0/2904947585 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 664220195 0 0) 0x7f705015a160 con 0x7f705015db10 2023-10-10T20:40:17.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.798+0000 7f704effd700 1 -- 172.21.15.89:0/2904947585 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7038003570 con 0x7f705015db10 2023-10-10T20:40:17.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.798+0000 7f704effd700 1 -- 172.21.15.89:0/2904947585 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4129895240 0 0) 0x7f7038003570 con 0x7f705015db10 2023-10-10T20:40:17.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.798+0000 7f704effd700 1 -- 172.21.15.89:0/2904947585 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7050159890 msgr2=0x7f705015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:17.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.798+0000 7f704effd700 1 --2- 172.21.15.89:0/2904947585 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7050159890 0x7f705015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:17.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.798+0000 7f704effd700 1 -- 172.21.15.89:0/2904947585 >> v2:172.21.15.111:3300/0 conn(0x7f7050159c60 msgr2=0x7f705015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:17.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.798+0000 7f704effd700 1 --2- 172.21.15.89:0/2904947585 >> v2:172.21.15.111:3300/0 conn(0x7f7050159c60 0x7f705015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:17.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.798+0000 7f704effd700 1 -- 172.21.15.89:0/2904947585 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705015a3d0 con 0x7f705015db10 2023-10-10T20:40:17.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.798+0000 7f704effd700 1 -- 172.21.15.89:0/2904947585 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f70480017f0 con 0x7f705015db10 2023-10-10T20:40:17.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f704effd700 1 -- 172.21.15.89:0/2904947585 >> v1:172.21.15.89:6789/0 conn(0x7f705015db10 legacy=0x7f7050069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:17.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f704effd700 1 --2- 172.21.15.89:0/2904947585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7038003a60 0x7f7038003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:17.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f704effd700 1 --2- 172.21.15.89:0/2904947585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7038004810 0x7f7038004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:17.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f704effd700 1 --2- 172.21.15.89:0/2904947585 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7050159890 0x7f70380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:17.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f704effd700 1 -- 172.21.15.89:0/2904947585 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70500d13a0 con 0x7f7038003a60 2023-10-10T20:40:17.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f704effd700 1 -- 172.21.15.89:0/2904947585 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705015e140 con 0x7f7038004810 2023-10-10T20:40:17.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f704effd700 1 -- 172.21.15.89:0/2904947585 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703800ba60 con 0x7f7050159890 2023-10-10T20:40:17.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f704ffff700 1 --2- 172.21.15.89:0/2904947585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7038003a60 0x7f7038003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:17.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f704f7fe700 1 --2- 172.21.15.89:0/2904947585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7038004810 0x7f7038004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:17.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f7054ae0700 1 --2- 172.21.15.89:0/2904947585 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7050159890 0x7f70380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:17.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f704ffff700 1 -- 172.21.15.89:0/2904947585 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7050159890 msgr2=0x7f70380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:17.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f704ffff700 1 --2- 172.21.15.89:0/2904947585 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7050159890 0x7f70380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:17.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f704ffff700 1 -- 172.21.15.89:0/2904947585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7038004810 msgr2=0x7f7038004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:17.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f704ffff700 1 --2- 172.21.15.89:0/2904947585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7038004810 0x7f7038004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:17.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f704ffff700 1 -- 172.21.15.89:0/2904947585 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7038003570 con 0x7f7038003a60 2023-10-10T20:40:17.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f7054ae0700 1 --2- 172.21.15.89:0/2904947585 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7050159890 0x7f70380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:17.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f704ffff700 1 --2- 172.21.15.89:0/2904947585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7038003a60 0x7f7038003e70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7040009c30 tx=0x7f70400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208d9b715f7a665a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:17.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f704effd700 1 -- 172.21.15.89:0/2904947585 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704000e710 con 0x7f7038003a60 2023-10-10T20:40:17.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f704effd700 1 -- 172.21.15.89:0/2904947585 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704000e870 con 0x7f7038003a60 2023-10-10T20:40:17.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.799+0000 7f704effd700 1 -- 172.21.15.89:0/2904947585 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704000ec30 con 0x7f7038003a60 2023-10-10T20:40:17.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.800+0000 7f7056543700 1 -- 172.21.15.89:0/2904947585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7038003a60 msgr2=0x7f7038003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:17.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.800+0000 7f7056543700 1 --2- 172.21.15.89:0/2904947585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7038003a60 0x7f7038003e70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7040009c30 tx=0x7f70400091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:17.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.800+0000 7f7054ae0700 1 -- 172.21.15.89:0/2904947585 reap_dead start 2023-10-10T20:40:17.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.800+0000 7f7056543700 1 -- 172.21.15.89:0/2904947585 shutdown_connections 2023-10-10T20:40:17.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.800+0000 7f7056543700 1 -- 172.21.15.89:0/2904947585 >> 172.21.15.89:0/2904947585 conn(0x7f7050068e20 msgr2=0x7f70500d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:17.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.800+0000 7f7056543700 1 -- 172.21.15.89:0/2904947585 shutdown_connections 2023-10-10T20:40:17.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.800+0000 7f7056543700 1 -- 172.21.15.89:0/2904947585 wait complete. 2023-10-10T20:40:17.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.801+0000 7f7056543700 1 Processor -- start 2023-10-10T20:40:17.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.801+0000 7f7056543700 1 -- start start 2023-10-10T20:40:17.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.802+0000 7f7056543700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7038003a60 0x7f7050169bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:17.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.802+0000 7f7056543700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70501471f0 0x7f70500b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:17.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.802+0000 7f7056543700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70500b3ad0 0x7f70500c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:17.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.802+0000 7f7056543700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705016a100 con 0x7f7038003a60 2023-10-10T20:40:17.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.802+0000 7f7056543700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70500b4bf0 con 0x7f70501471f0 2023-10-10T20:40:17.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.802+0000 7f7056543700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70500b3fe0 con 0x7f70500b3ad0 2023-10-10T20:40:17.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.802+0000 7f704ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7038003a60 0x7f7050169bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:17.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.802+0000 7f704ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7038003a60 0x7f7050169bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43512/0 (socket says 172.21.15.89:43512) 2023-10-10T20:40:17.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.802+0000 7f704ffff700 1 -- 172.21.15.89:0/1838986477 learned_addr learned my addr 172.21.15.89:0/1838986477 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:17.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.802+0000 7f704f7fe700 1 --2- 172.21.15.89:0/1838986477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70501471f0 0x7f70500b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:17.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.802+0000 7f7054ae0700 1 --2- 172.21.15.89:0/1838986477 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70500b3ad0 0x7f70500c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:17.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.802+0000 7f704ffff700 1 -- 172.21.15.89:0/1838986477 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70500b3ad0 msgr2=0x7f70500c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:17.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.802+0000 7f704ffff700 1 --2- 172.21.15.89:0/1838986477 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70500b3ad0 0x7f70500c69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:17.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.802+0000 7f704ffff700 1 -- 172.21.15.89:0/1838986477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70501471f0 msgr2=0x7f70500b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:17.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.802+0000 7f704ffff700 1 --2- 172.21.15.89:0/1838986477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70501471f0 0x7f70500b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:17.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.802+0000 7f704ffff700 1 -- 172.21.15.89:0/1838986477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70500b42c0 con 0x7f7038003a60 2023-10-10T20:40:17.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.802+0000 7f704f7fe700 1 --2- 172.21.15.89:0/1838986477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70501471f0 0x7f70500b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:17.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.802+0000 7f704ffff700 1 --2- 172.21.15.89:0/1838986477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7038003a60 0x7f7050169bc0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f70380046d0 tx=0x7f704001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:17.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.803+0000 7f7037fff700 1 -- 172.21.15.89:0/1838986477 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7040022070 con 0x7f7038003a60 2023-10-10T20:40:17.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.803+0000 7f7037fff700 1 -- 172.21.15.89:0/1838986477 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704000e9e0 con 0x7f7038003a60 2023-10-10T20:40:17.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.803+0000 7f7056543700 1 -- 172.21.15.89:0/1838986477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70500b45a0 con 0x7f7038003a60 2023-10-10T20:40:17.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.803+0000 7f7037fff700 1 -- 172.21.15.89:0/1838986477 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704001c450 con 0x7f7038003a60 2023-10-10T20:40:17.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.803+0000 7f7056543700 1 -- 172.21.15.89:0/1838986477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70500cfcf0 con 0x7f7038003a60 2023-10-10T20:40:17.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.804+0000 7f7056543700 1 -- 172.21.15.89:0/1838986477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705004c3c0 con 0x7f7038003a60 2023-10-10T20:40:17.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.804+0000 7f7037fff700 1 -- 172.21.15.89:0/1838986477 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f704001c840 con 0x7f7038003a60 2023-10-10T20:40:17.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.804+0000 7f7037fff700 1 --2- 172.21.15.89:0/1838986477 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f703c067200 0x7f703c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:17.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.805+0000 7f7037fff700 1 -- 172.21.15.89:0/1838986477 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7040086240 con 0x7f7038003a60 2023-10-10T20:40:17.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.805+0000 7f704f7fe700 1 --2- 172.21.15.89:0/1838986477 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f703c067200 0x7f703c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:17.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.805+0000 7f704f7fe700 1 --2- 172.21.15.89:0/1838986477 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f703c067200 0x7f703c0696b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7044006dc0 tx=0x7f7044008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:17.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.809+0000 7f7037fff700 1 -- 172.21.15.89:0/1838986477 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7040053160 con 0x7f7038003a60 2023-10-10T20:40:17.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.989+0000 7f7056543700 1 -- 172.21.15.89:0/1838986477 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f705006a5b0 con 0x7f703c067200 2023-10-10T20:40:17.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:17.997+0000 7f7037fff700 1 -- 172.21.15.89:0/1838986477 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289893 (secure 0 0 0) 0x7f705006a5b0 con 0x7f703c067200 2023-10-10T20:40:18.004 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:18.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.000+0000 7f7056543700 1 -- 172.21.15.89:0/1838986477 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f703c067200 msgr2=0x7f703c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:18.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.000+0000 7f7056543700 1 --2- 172.21.15.89:0/1838986477 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f703c067200 0x7f703c0696b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7044006dc0 tx=0x7f7044008040 comp rx=0 tx=0).stop 2023-10-10T20:40:18.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.000+0000 7f7056543700 1 -- 172.21.15.89:0/1838986477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7038003a60 msgr2=0x7f7050169bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:18.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.000+0000 7f7056543700 1 --2- 172.21.15.89:0/1838986477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7038003a60 0x7f7050169bc0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f70380046d0 tx=0x7f704001a000 comp rx=0 tx=0).stop 2023-10-10T20:40:18.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.000+0000 7f7056543700 1 -- 172.21.15.89:0/1838986477 shutdown_connections 2023-10-10T20:40:18.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.000+0000 7f7056543700 1 --2- 172.21.15.89:0/1838986477 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70500b3ad0 0x7f70500c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.000+0000 7f7056543700 1 --2- 172.21.15.89:0/1838986477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7038003a60 0x7f7050169bc0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.000+0000 7f7056543700 1 --2- 172.21.15.89:0/1838986477 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f703c067200 0x7f703c0696b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.000+0000 7f7056543700 1 --2- 172.21.15.89:0/1838986477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70501471f0 0x7f70500b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.000+0000 7f7056543700 1 -- 172.21.15.89:0/1838986477 >> 172.21.15.89:0/1838986477 conn(0x7f7050068e20 msgr2=0x7f7050152c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:18.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.000+0000 7f7056543700 1 -- 172.21.15.89:0/1838986477 shutdown_connections 2023-10-10T20:40:18.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.000+0000 7f7056543700 1 -- 172.21.15.89:0/1838986477 wait complete. 2023-10-10T20:40:18.009 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:40:18.014 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2023-10-10T20:40:16.720830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2400324,"kb_used_data":9732,"kb_used_omap":0,"kb_used_meta":2390528,"kb_avail":752574396,"statfs":{"total":773094113280,"available":770636181504,"internally_reserved":0,"allocated":9965568,"data_stored":4938691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2447900672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001797"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.039533+0000","last_change":"2023-10-10T20:40:09.035722+0000","last_active":"2023-10-10T20:40:09.039533+0000","last_peered":"2023-10-10T20:40:09.039533+0000","last_clean":"2023-10-10T20:40:09.039533+0000","last_became_active":"2023-10-10T20:40:09.035445+0000","last_became_peered":"2023-10-10T20:40:09.035445+0000","last_unstale":"2023-10-10T20:40:09.039533+0000","last_undegraded":"2023-10-10T20:40:09.039533+0000","last_fullsized":"2023-10-10T20:40:09.039533+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:32:47.209775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.037763+0000","last_change":"2023-10-10T20:40:09.035065+0000","last_active":"2023-10-10T20:40:09.037763+0000","last_peered":"2023-10-10T20:40:09.037763+0000","last_clean":"2023-10-10T20:40:09.037763+0000","last_became_active":"2023-10-10T20:40:09.034846+0000","last_became_peered":"2023-10-10T20:40:09.034846+0000","last_unstale":"2023-10-10T20:40:09.037763+0000","last_undegraded":"2023-10-10T20:40:09.037763+0000","last_fullsized":"2023-10-10T20:40:09.037763+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:56:38.502579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.040566+0000","last_change":"2023-10-10T20:40:09.037493+0000","last_active":"2023-10-10T20:40:09.040566+0000","last_peered":"2023-10-10T20:40:09.040566+0000","last_clean":"2023-10-10T20:40:09.040566+0000","last_became_active":"2023-10-10T20:40:09.037245+0000","last_became_peered":"2023-10-10T20:40:09.037245+0000","last_unstale":"2023-10-10T20:40:09.040566+0000","last_undegraded":"2023-10-10T20:40:09.040566+0000","last_fullsized":"2023-10-10T20:40:09.040566+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:04:08.493673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.030762+0000","last_change":"2023-10-10T20:40:09.024568+0000","last_active":"2023-10-10T20:40:09.030762+0000","last_peered":"2023-10-10T20:40:09.030762+0000","last_clean":"2023-10-10T20:40:09.030762+0000","last_became_active":"2023-10-10T20:40:09.024314+0000","last_became_peered":"2023-10-10T20:40:09.024314+0000","last_unstale":"2023-10-10T20:40:09.030762+0000","last_undegraded":"2023-10-10T20:40:09.030762+0000","last_fullsized":"2023-10-10T20:40:09.030762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:36:16.536704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.041074+0000","last_change":"2023-10-10T20:40:09.033664+0000","last_active":"2023-10-10T20:40:09.041074+0000","last_peered":"2023-10-10T20:40:09.041074+0000","last_clean":"2023-10-10T20:40:09.041074+0000","last_became_active":"2023-10-10T20:40:09.033009+0000","last_became_peered":"2023-10-10T20:40:09.033009+0000","last_unstale":"2023-10-10T20:40:09.041074+0000","last_undegraded":"2023-10-10T20:40:09.041074+0000","last_fullsized":"2023-10-10T20:40:09.041074+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:48:42.895368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.030392+0000","last_change":"2023-10-10T20:40:09.023968+0000","last_active":"2023-10-10T20:40:09.030392+0000","last_peered":"2023-10-10T20:40:09.030392+0000","last_clean":"2023-10-10T20:40:09.030392+0000","last_became_active":"2023-10-10T20:40:09.023710+0000","last_became_peered":"2023-10-10T20:40:09.023710+0000","last_unstale":"2023-10-10T20:40:09.030392+0000","last_undegraded":"2023-10-10T20:40:09.030392+0000","last_fullsized":"2023-10-10T20:40:09.030392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:03:24.798161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.026787+0000","last_change":"2023-10-10T20:40:09.021038+0000","last_active":"2023-10-10T20:40:09.026787+0000","last_peered":"2023-10-10T20:40:09.026787+0000","last_clean":"2023-10-10T20:40:09.026787+0000","last_became_active":"2023-10-10T20:40:09.020788+0000","last_became_peered":"2023-10-10T20:40:09.020788+0000","last_unstale":"2023-10-10T20:40:09.026787+0000","last_undegraded":"2023-10-10T20:40:09.026787+0000","last_fullsized":"2023-10-10T20:40:09.026787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:36:16.536704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.033420+0000","last_change":"2023-10-10T20:40:09.027929+0000","last_active":"2023-10-10T20:40:09.033420+0000","last_peered":"2023-10-10T20:40:09.033420+0000","last_clean":"2023-10-10T20:40:09.033420+0000","last_became_active":"2023-10-10T20:40:09.027647+0000","last_became_peered":"2023-10-10T20:40:09.027647+0000","last_unstale":"2023-10-10T20:40:09.033420+0000","last_undegraded":"2023-10-10T20:40:09.033420+0000","last_fullsized":"2023-10-10T20:40:09.033420+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:36:16.536704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.036749+0000","last_change":"2023-10-10T20:40:09.030540+0000","last_active":"2023-10-10T20:40:09.036749+0000","last_peered":"2023-10-10T20:40:09.036749+0000","last_clean":"2023-10-10T20:40:09.036749+0000","last_became_active":"2023-10-10T20:40:09.030271+0000","last_became_peered":"2023-10-10T20:40:09.030271+0000","last_unstale":"2023-10-10T20:40:09.036749+0000","last_undegraded":"2023-10-10T20:40:09.036749+0000","last_fullsized":"2023-10-10T20:40:09.036749+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:11:46.763981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.041151+0000","last_change":"2023-10-10T20:40:09.036650+0000","last_active":"2023-10-10T20:40:09.041151+0000","last_peered":"2023-10-10T20:40:09.041151+0000","last_clean":"2023-10-10T20:40:09.041151+0000","last_became_active":"2023-10-10T20:40:09.036399+0000","last_became_peered":"2023-10-10T20:40:09.036399+0000","last_unstale":"2023-10-10T20:40:09.041151+0000","last_undegraded":"2023-10-10T20:40:09.041151+0000","last_fullsized":"2023-10-10T20:40:09.041151+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:00:08.059251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.037920+0000","last_change":"2023-10-10T20:40:09.035577+0000","last_active":"2023-10-10T20:40:09.037920+0000","last_peered":"2023-10-10T20:40:09.037920+0000","last_clean":"2023-10-10T20:40:09.037920+0000","last_became_active":"2023-10-10T20:40:09.035272+0000","last_became_peered":"2023-10-10T20:40:09.035272+0000","last_unstale":"2023-10-10T20:40:09.037920+0000","last_undegraded":"2023-10-10T20:40:09.037920+0000","last_fullsized":"2023-10-10T20:40:09.037920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:00:08.059251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.033777+0000","last_change":"2023-10-10T20:40:09.028245+0000","last_active":"2023-10-10T20:40:09.033777+0000","last_peered":"2023-10-10T20:40:09.033777+0000","last_clean":"2023-10-10T20:40:09.033777+0000","last_became_active":"2023-10-10T20:40:09.027967+0000","last_became_peered":"2023-10-10T20:40:09.027967+0000","last_unstale":"2023-10-10T20:40:09.033777+0000","last_undegraded":"2023-10-10T20:40:09.033777+0000","last_fullsized":"2023-10-10T20:40:09.033777+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:48.018528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.031979+0000","last_change":"2023-10-10T20:40:09.028678+0000","last_active":"2023-10-10T20:40:09.031979+0000","last_peered":"2023-10-10T20:40:09.031979+0000","last_clean":"2023-10-10T20:40:09.031979+0000","last_became_active":"2023-10-10T20:40:09.028399+0000","last_became_peered":"2023-10-10T20:40:09.028399+0000","last_unstale":"2023-10-10T20:40:09.031979+0000","last_undegraded":"2023-10-10T20:40:09.031979+0000","last_fullsized":"2023-10-10T20:40:09.031979+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:19:08.851688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.039065+0000","last_change":"2023-10-10T20:40:09.032942+0000","last_active":"2023-10-10T20:40:09.039065+0000","last_peered":"2023-10-10T20:40:09.039065+0000","last_clean":"2023-10-10T20:40:09.039065+0000","last_became_active":"2023-10-10T20:40:09.032654+0000","last_became_peered":"2023-10-10T20:40:09.032654+0000","last_unstale":"2023-10-10T20:40:09.039065+0000","last_undegraded":"2023-10-10T20:40:09.039065+0000","last_fullsized":"2023-10-10T20:40:09.039065+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:04.944188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.039638+0000","last_change":"2023-10-10T20:40:09.032821+0000","last_active":"2023-10-10T20:40:09.039638+0000","last_peered":"2023-10-10T20:40:09.039638+0000","last_clean":"2023-10-10T20:40:09.039638+0000","last_became_active":"2023-10-10T20:40:09.032549+0000","last_became_peered":"2023-10-10T20:40:09.032549+0000","last_unstale":"2023-10-10T20:40:09.039638+0000","last_undegraded":"2023-10-10T20:40:09.039638+0000","last_fullsized":"2023-10-10T20:40:09.039638+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:31:28.232375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.030708+0000","last_change":"2023-10-10T20:40:09.025551+0000","last_active":"2023-10-10T20:40:09.030708+0000","last_peered":"2023-10-10T20:40:09.030708+0000","last_clean":"2023-10-10T20:40:09.030708+0000","last_became_active":"2023-10-10T20:40:09.025240+0000","last_became_peered":"2023-10-10T20:40:09.025240+0000","last_unstale":"2023-10-10T20:40:09.030708+0000","last_undegraded":"2023-10-10T20:40:09.030708+0000","last_fullsized":"2023-10-10T20:40:09.030708+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:59:58.077174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.035159+0000","last_change":"2023-10-10T20:40:09.032005+0000","last_active":"2023-10-10T20:40:09.035159+0000","last_peered":"2023-10-10T20:40:09.035159+0000","last_clean":"2023-10-10T20:40:09.035159+0000","last_became_active":"2023-10-10T20:40:09.031692+0000","last_became_peered":"2023-10-10T20:40:09.031692+0000","last_unstale":"2023-10-10T20:40:09.035159+0000","last_undegraded":"2023-10-10T20:40:09.035159+0000","last_fullsized":"2023-10-10T20:40:09.035159+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:54:35.128397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.030717+0000","last_change":"2023-10-10T20:40:09.024047+0000","last_active":"2023-10-10T20:40:09.030717+0000","last_peered":"2023-10-10T20:40:09.030717+0000","last_clean":"2023-10-10T20:40:09.030717+0000","last_became_active":"2023-10-10T20:40:09.023781+0000","last_became_peered":"2023-10-10T20:40:09.023781+0000","last_unstale":"2023-10-10T20:40:09.030717+0000","last_undegraded":"2023-10-10T20:40:09.030717+0000","last_fullsized":"2023-10-10T20:40:09.030717+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:03:24.798161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.031243+0000","last_change":"2023-10-10T20:40:09.025660+0000","last_active":"2023-10-10T20:40:09.031243+0000","last_peered":"2023-10-10T20:40:09.031243+0000","last_clean":"2023-10-10T20:40:09.031243+0000","last_became_active":"2023-10-10T20:40:09.025403+0000","last_became_peered":"2023-10-10T20:40:09.025403+0000","last_unstale":"2023-10-10T20:40:09.031243+0000","last_undegraded":"2023-10-10T20:40:09.031243+0000","last_fullsized":"2023-10-10T20:40:09.031243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:29:56.919641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.039047+0000","last_change":"2023-10-10T20:40:09.030059+0000","last_active":"2023-10-10T20:40:09.039047+0000","last_peered":"2023-10-10T20:40:09.039047+0000","last_clean":"2023-10-10T20:40:09.039047+0000","last_became_active":"2023-10-10T20:40:09.029784+0000","last_became_peered":"2023-10-10T20:40:09.029784+0000","last_unstale":"2023-10-10T20:40:09.039047+0000","last_undegraded":"2023-10-10T20:40:09.039047+0000","last_fullsized":"2023-10-10T20:40:09.039047+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:38:11.556616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.021363+0000","last_change":"2023-10-10T20:40:09.018714+0000","last_active":"2023-10-10T20:40:09.021363+0000","last_peered":"2023-10-10T20:40:09.021363+0000","last_clean":"2023-10-10T20:40:09.021363+0000","last_became_active":"2023-10-10T20:40:09.018461+0000","last_became_peered":"2023-10-10T20:40:09.018461+0000","last_unstale":"2023-10-10T20:40:09.021363+0000","last_undegraded":"2023-10-10T20:40:09.021363+0000","last_fullsized":"2023-10-10T20:40:09.021363+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:12:35.387750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.035168+0000","last_change":"2023-10-10T20:40:09.029764+0000","last_active":"2023-10-10T20:40:09.035168+0000","last_peered":"2023-10-10T20:40:09.035168+0000","last_clean":"2023-10-10T20:40:09.035168+0000","last_became_active":"2023-10-10T20:40:09.029305+0000","last_became_peered":"2023-10-10T20:40:09.029305+0000","last_unstale":"2023-10-10T20:40:09.035168+0000","last_undegraded":"2023-10-10T20:40:09.035168+0000","last_fullsized":"2023-10-10T20:40:09.035168+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:03:24.798161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.036969+0000","last_change":"2023-10-10T20:40:09.034290+0000","last_active":"2023-10-10T20:40:09.036969+0000","last_peered":"2023-10-10T20:40:09.036969+0000","last_clean":"2023-10-10T20:40:09.036969+0000","last_became_active":"2023-10-10T20:40:09.033979+0000","last_became_peered":"2023-10-10T20:40:09.033979+0000","last_unstale":"2023-10-10T20:40:09.036969+0000","last_undegraded":"2023-10-10T20:40:09.036969+0000","last_fullsized":"2023-10-10T20:40:09.036969+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:48:42.895368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.030689+0000","last_change":"2023-10-10T20:40:09.025512+0000","last_active":"2023-10-10T20:40:09.030689+0000","last_peered":"2023-10-10T20:40:09.030689+0000","last_clean":"2023-10-10T20:40:09.030689+0000","last_became_active":"2023-10-10T20:40:09.025211+0000","last_became_peered":"2023-10-10T20:40:09.025211+0000","last_unstale":"2023-10-10T20:40:09.030689+0000","last_undegraded":"2023-10-10T20:40:09.030689+0000","last_fullsized":"2023-10-10T20:40:09.030689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:48:42.895368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.033812+0000","last_change":"2023-10-10T20:40:09.026443+0000","last_active":"2023-10-10T20:40:09.033812+0000","last_peered":"2023-10-10T20:40:09.033812+0000","last_clean":"2023-10-10T20:40:09.033812+0000","last_became_active":"2023-10-10T20:40:09.026047+0000","last_became_peered":"2023-10-10T20:40:09.026047+0000","last_unstale":"2023-10-10T20:40:09.033812+0000","last_undegraded":"2023-10-10T20:40:09.033812+0000","last_fullsized":"2023-10-10T20:40:09.033812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:19:08.851688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.028997+0000","last_change":"2023-10-10T20:40:09.027157+0000","last_active":"2023-10-10T20:40:09.028997+0000","last_peered":"2023-10-10T20:40:09.028997+0000","last_clean":"2023-10-10T20:40:09.028997+0000","last_became_active":"2023-10-10T20:40:09.026864+0000","last_became_peered":"2023-10-10T20:40:09.026864+0000","last_unstale":"2023-10-10T20:40:09.028997+0000","last_undegraded":"2023-10-10T20:40:09.028997+0000","last_fullsized":"2023-10-10T20:40:09.028997+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:19:08.851688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.032338+0000","last_change":"2023-10-10T20:40:09.024838+0000","last_active":"2023-10-10T20:40:09.032338+0000","last_peered":"2023-10-10T20:40:09.032338+0000","last_clean":"2023-10-10T20:40:09.032338+0000","last_became_active":"2023-10-10T20:40:09.024555+0000","last_became_peered":"2023-10-10T20:40:09.024555+0000","last_unstale":"2023-10-10T20:40:09.032338+0000","last_undegraded":"2023-10-10T20:40:09.032338+0000","last_fullsized":"2023-10-10T20:40:09.032338+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:56:38.502579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.038395+0000","last_change":"2023-10-10T20:40:09.032710+0000","last_active":"2023-10-10T20:40:09.038395+0000","last_peered":"2023-10-10T20:40:09.038395+0000","last_clean":"2023-10-10T20:40:09.038395+0000","last_became_active":"2023-10-10T20:40:09.032465+0000","last_became_peered":"2023-10-10T20:40:09.032465+0000","last_unstale":"2023-10-10T20:40:09.038395+0000","last_undegraded":"2023-10-10T20:40:09.038395+0000","last_fullsized":"2023-10-10T20:40:09.038395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:59:58.077174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.035243+0000","last_change":"2023-10-10T20:40:09.032416+0000","last_active":"2023-10-10T20:40:09.035243+0000","last_peered":"2023-10-10T20:40:09.035243+0000","last_clean":"2023-10-10T20:40:09.035243+0000","last_became_active":"2023-10-10T20:40:09.032154+0000","last_became_peered":"2023-10-10T20:40:09.032154+0000","last_unstale":"2023-10-10T20:40:09.035243+0000","last_undegraded":"2023-10-10T20:40:09.035243+0000","last_fullsized":"2023-10-10T20:40:09.035243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:59:58.077174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.038969+0000","last_change":"2023-10-10T20:40:09.030269+0000","last_active":"2023-10-10T20:40:09.038969+0000","last_peered":"2023-10-10T20:40:09.038969+0000","last_clean":"2023-10-10T20:40:09.038969+0000","last_became_active":"2023-10-10T20:40:09.027408+0000","last_became_peered":"2023-10-10T20:40:09.027408+0000","last_unstale":"2023-10-10T20:40:09.038969+0000","last_undegraded":"2023-10-10T20:40:09.038969+0000","last_fullsized":"2023-10-10T20:40:09.038969+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:56:18.861844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.031335+0000","last_change":"2023-10-10T20:40:09.027394+0000","last_active":"2023-10-10T20:40:09.031335+0000","last_peered":"2023-10-10T20:40:09.031335+0000","last_clean":"2023-10-10T20:40:09.031335+0000","last_became_active":"2023-10-10T20:40:09.027172+0000","last_became_peered":"2023-10-10T20:40:09.027172+0000","last_unstale":"2023-10-10T20:40:09.031335+0000","last_undegraded":"2023-10-10T20:40:09.031335+0000","last_fullsized":"2023-10-10T20:40:09.031335+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:48:42.895368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.033279+0000","last_change":"2023-10-10T20:40:09.028939+0000","last_active":"2023-10-10T20:40:09.033279+0000","last_peered":"2023-10-10T20:40:09.033279+0000","last_clean":"2023-10-10T20:40:09.033279+0000","last_became_active":"2023-10-10T20:40:09.028700+0000","last_became_peered":"2023-10-10T20:40:09.028700+0000","last_unstale":"2023-10-10T20:40:09.033279+0000","last_undegraded":"2023-10-10T20:40:09.033279+0000","last_fullsized":"2023-10-10T20:40:09.033279+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:11.428701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.031854+0000","last_change":"2023-10-10T20:40:09.025820+0000","last_active":"2023-10-10T20:40:09.031854+0000","last_peered":"2023-10-10T20:40:09.031854+0000","last_clean":"2023-10-10T20:40:09.031854+0000","last_became_active":"2023-10-10T20:40:09.025058+0000","last_became_peered":"2023-10-10T20:40:09.025058+0000","last_unstale":"2023-10-10T20:40:09.031854+0000","last_undegraded":"2023-10-10T20:40:09.031854+0000","last_fullsized":"2023-10-10T20:40:09.031854+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:52:41.229173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"87'4","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.042163+0000","last_change":"2023-10-10T20:40:08.017435+0000","last_active":"2023-10-10T20:40:09.042163+0000","last_peered":"2023-10-10T20:40:09.042163+0000","last_clean":"2023-10-10T20:40:09.042163+0000","last_became_active":"2023-10-10T20:40:08.016794+0000","last_became_peered":"2023-10-10T20:40:08.016794+0000","last_unstale":"2023-10-10T20:40:09.042163+0000","last_undegraded":"2023-10-10T20:40:09.042163+0000","last_fullsized":"2023-10-10T20:40:09.042163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:52:56.099397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.031263+0000","last_change":"2023-10-10T20:40:09.027386+0000","last_active":"2023-10-10T20:40:09.031263+0000","last_peered":"2023-10-10T20:40:09.031263+0000","last_clean":"2023-10-10T20:40:09.031263+0000","last_became_active":"2023-10-10T20:40:09.027141+0000","last_became_peered":"2023-10-10T20:40:09.027141+0000","last_unstale":"2023-10-10T20:40:09.031263+0000","last_undegraded":"2023-10-10T20:40:09.031263+0000","last_fullsized":"2023-10-10T20:40:09.031263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:56:38.502579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"87'1","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.040829+0000","last_change":"2023-10-10T20:40:08.011048+0000","last_active":"2023-10-10T20:40:09.040829+0000","last_peered":"2023-10-10T20:40:09.040829+0000","last_clean":"2023-10-10T20:40:09.040829+0000","last_became_active":"2023-10-10T20:40:08.010764+0000","last_became_peered":"2023-10-10T20:40:08.010764+0000","last_unstale":"2023-10-10T20:40:09.040829+0000","last_undegraded":"2023-10-10T20:40:09.040829+0000","last_fullsized":"2023-10-10T20:40:09.040829+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:52:56.099397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.033438+0000","last_change":"2023-10-10T20:40:09.030971+0000","last_active":"2023-10-10T20:40:09.033438+0000","last_peered":"2023-10-10T20:40:09.033438+0000","last_clean":"2023-10-10T20:40:09.033438+0000","last_became_active":"2023-10-10T20:40:09.030692+0000","last_became_peered":"2023-10-10T20:40:09.030692+0000","last_unstale":"2023-10-10T20:40:09.033438+0000","last_undegraded":"2023-10-10T20:40:09.033438+0000","last_fullsized":"2023-10-10T20:40:09.033438+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:55:56.487894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"87'2","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.040834+0000","last_change":"2023-10-10T20:40:08.019586+0000","last_active":"2023-10-10T20:40:09.040834+0000","last_peered":"2023-10-10T20:40:09.040834+0000","last_clean":"2023-10-10T20:40:09.040834+0000","last_became_active":"2023-10-10T20:40:08.019243+0000","last_became_peered":"2023-10-10T20:40:08.019243+0000","last_unstale":"2023-10-10T20:40:09.040834+0000","last_undegraded":"2023-10-10T20:40:09.040834+0000","last_fullsized":"2023-10-10T20:40:09.040834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:04:01.312167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.038827+0000","last_change":"2023-10-10T20:40:09.034870+0000","last_active":"2023-10-10T20:40:09.038827+0000","last_peered":"2023-10-10T20:40:09.038827+0000","last_clean":"2023-10-10T20:40:09.038827+0000","last_became_active":"2023-10-10T20:40:09.034431+0000","last_became_peered":"2023-10-10T20:40:09.034431+0000","last_unstale":"2023-10-10T20:40:09.038827+0000","last_undegraded":"2023-10-10T20:40:09.038827+0000","last_fullsized":"2023-10-10T20:40:09.038827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:25:35.014775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"87'2","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.044427+0000","last_change":"2023-10-10T20:40:08.018241+0000","last_active":"2023-10-10T20:40:09.044427+0000","last_peered":"2023-10-10T20:40:09.044427+0000","last_clean":"2023-10-10T20:40:09.044427+0000","last_became_active":"2023-10-10T20:40:08.017083+0000","last_became_peered":"2023-10-10T20:40:08.017083+0000","last_unstale":"2023-10-10T20:40:09.044427+0000","last_undegraded":"2023-10-10T20:40:09.044427+0000","last_fullsized":"2023-10-10T20:40:09.044427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:03:11.358023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.035511+0000","last_change":"2023-10-10T20:40:09.030544+0000","last_active":"2023-10-10T20:40:09.035511+0000","last_peered":"2023-10-10T20:40:09.035511+0000","last_clean":"2023-10-10T20:40:09.035511+0000","last_became_active":"2023-10-10T20:40:09.030272+0000","last_became_peered":"2023-10-10T20:40:09.030272+0000","last_unstale":"2023-10-10T20:40:09.035511+0000","last_undegraded":"2023-10-10T20:40:09.035511+0000","last_fullsized":"2023-10-10T20:40:09.035511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:52:57.103156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"87'1","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.041327+0000","last_change":"2023-10-10T20:40:08.010904+0000","last_active":"2023-10-10T20:40:09.041327+0000","last_peered":"2023-10-10T20:40:09.041327+0000","last_clean":"2023-10-10T20:40:09.041327+0000","last_became_active":"2023-10-10T20:40:08.010633+0000","last_became_peered":"2023-10-10T20:40:08.010633+0000","last_unstale":"2023-10-10T20:40:09.041327+0000","last_undegraded":"2023-10-10T20:40:09.041327+0000","last_fullsized":"2023-10-10T20:40:09.041327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:47:32.593061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.033627+0000","last_change":"2023-10-10T20:40:09.027615+0000","last_active":"2023-10-10T20:40:09.033627+0000","last_peered":"2023-10-10T20:40:09.033627+0000","last_clean":"2023-10-10T20:40:09.033627+0000","last_became_active":"2023-10-10T20:40:09.027336+0000","last_became_peered":"2023-10-10T20:40:09.027336+0000","last_unstale":"2023-10-10T20:40:09.033627+0000","last_undegraded":"2023-10-10T20:40:09.033627+0000","last_fullsized":"2023-10-10T20:40:09.033627+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:54:35.128397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"87'1","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.040808+0000","last_change":"2023-10-10T20:40:08.016825+0000","last_active":"2023-10-10T20:40:09.040808+0000","last_peered":"2023-10-10T20:40:09.040808+0000","last_clean":"2023-10-10T20:40:09.040808+0000","last_became_active":"2023-10-10T20:40:08.016560+0000","last_became_peered":"2023-10-10T20:40:08.016560+0000","last_unstale":"2023-10-10T20:40:09.040808+0000","last_undegraded":"2023-10-10T20:40:09.040808+0000","last_fullsized":"2023-10-10T20:40:09.040808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:32:46.206016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.030543+0000","last_change":"2023-10-10T20:40:09.025547+0000","last_active":"2023-10-10T20:40:09.030543+0000","last_peered":"2023-10-10T20:40:09.030543+0000","last_clean":"2023-10-10T20:40:09.030543+0000","last_became_active":"2023-10-10T20:40:09.025324+0000","last_became_peered":"2023-10-10T20:40:09.025324+0000","last_unstale":"2023-10-10T20:40:09.030543+0000","last_undegraded":"2023-10-10T20:40:09.030543+0000","last_fullsized":"2023-10-10T20:40:09.030543+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:54:35.128397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.021397+0000","last_change":"2023-10-10T20:40:08.017181+0000","last_active":"2023-10-10T20:40:09.021397+0000","last_peered":"2023-10-10T20:40:09.021397+0000","last_clean":"2023-10-10T20:40:09.021397+0000","last_became_active":"2023-10-10T20:40:08.016853+0000","last_became_peered":"2023-10-10T20:40:08.016853+0000","last_unstale":"2023-10-10T20:40:09.021397+0000","last_undegraded":"2023-10-10T20:40:09.021397+0000","last_fullsized":"2023-10-10T20:40:09.021397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:03:11.358023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.036690+0000","last_change":"2023-10-10T20:40:09.032036+0000","last_active":"2023-10-10T20:40:09.036690+0000","last_peered":"2023-10-10T20:40:09.036690+0000","last_clean":"2023-10-10T20:40:09.036690+0000","last_became_active":"2023-10-10T20:40:09.030342+0000","last_became_peered":"2023-10-10T20:40:09.030342+0000","last_unstale":"2023-10-10T20:40:09.036690+0000","last_undegraded":"2023-10-10T20:40:09.036690+0000","last_fullsized":"2023-10-10T20:40:09.036690+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:43:46.981879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.021481+0000","last_change":"2023-10-10T20:40:08.009774+0000","last_active":"2023-10-10T20:40:09.021481+0000","last_peered":"2023-10-10T20:40:09.021481+0000","last_clean":"2023-10-10T20:40:09.021481+0000","last_became_active":"2023-10-10T20:40:08.009493+0000","last_became_peered":"2023-10-10T20:40:08.009493+0000","last_unstale":"2023-10-10T20:40:09.021481+0000","last_undegraded":"2023-10-10T20:40:09.021481+0000","last_fullsized":"2023-10-10T20:40:09.021481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:41:00.555522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.035387+0000","last_change":"2023-10-10T20:40:09.030126+0000","last_active":"2023-10-10T20:40:09.035387+0000","last_peered":"2023-10-10T20:40:09.035387+0000","last_clean":"2023-10-10T20:40:09.035387+0000","last_became_active":"2023-10-10T20:40:09.029845+0000","last_became_peered":"2023-10-10T20:40:09.029845+0000","last_unstale":"2023-10-10T20:40:09.035387+0000","last_undegraded":"2023-10-10T20:40:09.035387+0000","last_fullsized":"2023-10-10T20:40:09.035387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:24:41.449979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.021478+0000","last_change":"2023-10-10T20:40:08.011149+0000","last_active":"2023-10-10T20:40:09.021478+0000","last_peered":"2023-10-10T20:40:09.021478+0000","last_clean":"2023-10-10T20:40:09.021478+0000","last_became_active":"2023-10-10T20:40:08.010885+0000","last_became_peered":"2023-10-10T20:40:08.010885+0000","last_unstale":"2023-10-10T20:40:09.021478+0000","last_undegraded":"2023-10-10T20:40:09.021478+0000","last_fullsized":"2023-10-10T20:40:09.021478+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:56:28.068099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.036729+0000","last_change":"2023-10-10T20:40:09.029693+0000","last_active":"2023-10-10T20:40:09.036729+0000","last_peered":"2023-10-10T20:40:09.036729+0000","last_clean":"2023-10-10T20:40:09.036729+0000","last_became_active":"2023-10-10T20:40:09.029296+0000","last_became_peered":"2023-10-10T20:40:09.029296+0000","last_unstale":"2023-10-10T20:40:09.036729+0000","last_undegraded":"2023-10-10T20:40:09.036729+0000","last_fullsized":"2023-10-10T20:40:09.036729+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:56:36.966922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.014166+0000","last_change":"2023-10-10T20:40:08.017421+0000","last_active":"2023-10-10T20:40:09.014166+0000","last_peered":"2023-10-10T20:40:09.014166+0000","last_clean":"2023-10-10T20:40:09.014166+0000","last_became_active":"2023-10-10T20:40:08.017152+0000","last_became_peered":"2023-10-10T20:40:08.017152+0000","last_unstale":"2023-10-10T20:40:09.014166+0000","last_undegraded":"2023-10-10T20:40:09.014166+0000","last_fullsized":"2023-10-10T20:40:09.014166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:33:47.187024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.034077+0000","last_change":"2023-10-10T20:40:09.026748+0000","last_active":"2023-10-10T20:40:09.034077+0000","last_peered":"2023-10-10T20:40:09.034077+0000","last_clean":"2023-10-10T20:40:09.034077+0000","last_became_active":"2023-10-10T20:40:09.026475+0000","last_became_peered":"2023-10-10T20:40:09.026475+0000","last_unstale":"2023-10-10T20:40:09.034077+0000","last_undegraded":"2023-10-10T20:40:09.034077+0000","last_fullsized":"2023-10-10T20:40:09.034077+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:36:16.536704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"87'3","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.042146+0000","last_change":"2023-10-10T20:40:08.008122+0000","last_active":"2023-10-10T20:40:09.042146+0000","last_peered":"2023-10-10T20:40:09.042146+0000","last_clean":"2023-10-10T20:40:09.042146+0000","last_became_active":"2023-10-10T20:40:08.007832+0000","last_became_peered":"2023-10-10T20:40:08.007832+0000","last_unstale":"2023-10-10T20:40:09.042146+0000","last_undegraded":"2023-10-10T20:40:09.042146+0000","last_fullsized":"2023-10-10T20:40:09.042146+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:07:29.691713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.032361+0000","last_change":"2023-10-10T20:40:09.025333+0000","last_active":"2023-10-10T20:40:09.032361+0000","last_peered":"2023-10-10T20:40:09.032361+0000","last_clean":"2023-10-10T20:40:09.032361+0000","last_became_active":"2023-10-10T20:40:09.025062+0000","last_became_peered":"2023-10-10T20:40:09.025062+0000","last_unstale":"2023-10-10T20:40:09.032361+0000","last_undegraded":"2023-10-10T20:40:09.032361+0000","last_fullsized":"2023-10-10T20:40:09.032361+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:19:08.851688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"87'6","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.050533+0000","last_change":"2023-10-10T20:40:08.006299+0000","last_active":"2023-10-10T20:40:09.050533+0000","last_peered":"2023-10-10T20:40:09.050533+0000","last_clean":"2023-10-10T20:40:09.050533+0000","last_became_active":"2023-10-10T20:40:08.005962+0000","last_became_peered":"2023-10-10T20:40:08.005962+0000","last_unstale":"2023-10-10T20:40:09.050533+0000","last_undegraded":"2023-10-10T20:40:09.050533+0000","last_fullsized":"2023-10-10T20:40:09.050533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:55:55.484134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.030572+0000","last_change":"2023-10-10T20:40:09.026462+0000","last_active":"2023-10-10T20:40:09.030572+0000","last_peered":"2023-10-10T20:40:09.030572+0000","last_clean":"2023-10-10T20:40:09.030572+0000","last_became_active":"2023-10-10T20:40:09.026185+0000","last_became_peered":"2023-10-10T20:40:09.026185+0000","last_unstale":"2023-10-10T20:40:09.030572+0000","last_undegraded":"2023-10-10T20:40:09.030572+0000","last_fullsized":"2023-10-10T20:40:09.030572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:43:46.981879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"87'2","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.042114+0000","last_change":"2023-10-10T20:40:08.016530+0000","last_active":"2023-10-10T20:40:09.042114+0000","last_peered":"2023-10-10T20:40:09.042114+0000","last_clean":"2023-10-10T20:40:09.042114+0000","last_became_active":"2023-10-10T20:40:08.016256+0000","last_became_peered":"2023-10-10T20:40:08.016256+0000","last_unstale":"2023-10-10T20:40:09.042114+0000","last_undegraded":"2023-10-10T20:40:09.042114+0000","last_fullsized":"2023-10-10T20:40:09.042114+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:30.662179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.032807+0000","last_change":"2023-10-10T20:40:09.024413+0000","last_active":"2023-10-10T20:40:09.032807+0000","last_peered":"2023-10-10T20:40:09.032807+0000","last_clean":"2023-10-10T20:40:09.032807+0000","last_became_active":"2023-10-10T20:40:09.024143+0000","last_became_peered":"2023-10-10T20:40:09.024143+0000","last_unstale":"2023-10-10T20:40:09.032807+0000","last_undegraded":"2023-10-10T20:40:09.032807+0000","last_fullsized":"2023-10-10T20:40:09.032807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:35:03.302229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.017562+0000","last_change":"2023-10-10T20:40:08.018565+0000","last_active":"2023-10-10T20:40:09.017562+0000","last_peered":"2023-10-10T20:40:09.017562+0000","last_clean":"2023-10-10T20:40:09.017562+0000","last_became_active":"2023-10-10T20:40:08.018317+0000","last_became_peered":"2023-10-10T20:40:08.018317+0000","last_unstale":"2023-10-10T20:40:09.017562+0000","last_undegraded":"2023-10-10T20:40:09.017562+0000","last_fullsized":"2023-10-10T20:40:09.017562+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:18:06.126353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.036794+0000","last_change":"2023-10-10T20:40:09.031472+0000","last_active":"2023-10-10T20:40:09.036794+0000","last_peered":"2023-10-10T20:40:09.036794+0000","last_clean":"2023-10-10T20:40:09.036794+0000","last_became_active":"2023-10-10T20:40:09.030821+0000","last_became_peered":"2023-10-10T20:40:09.030821+0000","last_unstale":"2023-10-10T20:40:09.036794+0000","last_undegraded":"2023-10-10T20:40:09.036794+0000","last_fullsized":"2023-10-10T20:40:09.036794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:49:11.364608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.021871+0000","last_change":"2023-10-10T20:40:08.011325+0000","last_active":"2023-10-10T20:40:09.021871+0000","last_peered":"2023-10-10T20:40:09.021871+0000","last_clean":"2023-10-10T20:40:09.021871+0000","last_became_active":"2023-10-10T20:40:08.011091+0000","last_became_peered":"2023-10-10T20:40:08.011091+0000","last_unstale":"2023-10-10T20:40:09.021871+0000","last_undegraded":"2023-10-10T20:40:09.021871+0000","last_fullsized":"2023-10-10T20:40:09.021871+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:04:07.489913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.039785+0000","last_change":"2023-10-10T20:40:09.034501+0000","last_active":"2023-10-10T20:40:09.039785+0000","last_peered":"2023-10-10T20:40:09.039785+0000","last_clean":"2023-10-10T20:40:09.039785+0000","last_became_active":"2023-10-10T20:40:09.034208+0000","last_became_peered":"2023-10-10T20:40:09.034208+0000","last_unstale":"2023-10-10T20:40:09.039785+0000","last_undegraded":"2023-10-10T20:40:09.039785+0000","last_fullsized":"2023-10-10T20:40:09.039785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:53:29.525908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"87'8","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.044004+0000","last_change":"2023-10-10T20:40:08.011477+0000","last_active":"2023-10-10T20:40:09.044004+0000","last_peered":"2023-10-10T20:40:09.044004+0000","last_clean":"2023-10-10T20:40:09.044004+0000","last_became_active":"2023-10-10T20:40:08.011221+0000","last_became_peered":"2023-10-10T20:40:08.011221+0000","last_unstale":"2023-10-10T20:40:09.044004+0000","last_undegraded":"2023-10-10T20:40:09.044004+0000","last_fullsized":"2023-10-10T20:40:09.044004+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:34:05.954078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.037022+0000","last_change":"2023-10-10T20:40:09.030930+0000","last_active":"2023-10-10T20:40:09.037022+0000","last_peered":"2023-10-10T20:40:09.037022+0000","last_clean":"2023-10-10T20:40:09.037022+0000","last_became_active":"2023-10-10T20:40:09.030699+0000","last_became_peered":"2023-10-10T20:40:09.030699+0000","last_unstale":"2023-10-10T20:40:09.037022+0000","last_undegraded":"2023-10-10T20:40:09.037022+0000","last_fullsized":"2023-10-10T20:40:09.037022+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:43:46.981879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"87'2","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.041914+0000","last_change":"2023-10-10T20:40:08.011010+0000","last_active":"2023-10-10T20:40:09.041914+0000","last_peered":"2023-10-10T20:40:09.041914+0000","last_clean":"2023-10-10T20:40:09.041914+0000","last_became_active":"2023-10-10T20:40:08.010739+0000","last_became_peered":"2023-10-10T20:40:08.010739+0000","last_unstale":"2023-10-10T20:40:09.041914+0000","last_undegraded":"2023-10-10T20:40:09.041914+0000","last_fullsized":"2023-10-10T20:40:09.041914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:32:46.206016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.035487+0000","last_change":"2023-10-10T20:40:09.027255+0000","last_active":"2023-10-10T20:40:09.035487+0000","last_peered":"2023-10-10T20:40:09.035487+0000","last_clean":"2023-10-10T20:40:09.035487+0000","last_became_active":"2023-10-10T20:40:09.026934+0000","last_became_peered":"2023-10-10T20:40:09.026934+0000","last_unstale":"2023-10-10T20:40:09.035487+0000","last_undegraded":"2023-10-10T20:40:09.035487+0000","last_fullsized":"2023-10-10T20:40:09.035487+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:04:02.315927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"87'1","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.042483+0000","last_change":"2023-10-10T20:40:08.012275+0000","last_active":"2023-10-10T20:40:09.042483+0000","last_peered":"2023-10-10T20:40:09.042483+0000","last_clean":"2023-10-10T20:40:09.042483+0000","last_became_active":"2023-10-10T20:40:08.011982+0000","last_became_peered":"2023-10-10T20:40:08.011982+0000","last_unstale":"2023-10-10T20:40:09.042483+0000","last_undegraded":"2023-10-10T20:40:09.042483+0000","last_fullsized":"2023-10-10T20:40:09.042483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:10:02.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.034413+0000","last_change":"2023-10-10T20:40:09.028441+0000","last_active":"2023-10-10T20:40:09.034413+0000","last_peered":"2023-10-10T20:40:09.034413+0000","last_clean":"2023-10-10T20:40:09.034413+0000","last_became_active":"2023-10-10T20:40:09.028149+0000","last_became_peered":"2023-10-10T20:40:09.028149+0000","last_unstale":"2023-10-10T20:40:09.034413+0000","last_undegraded":"2023-10-10T20:40:09.034413+0000","last_fullsized":"2023-10-10T20:40:09.034413+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:56:36.966922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.018843+0000","last_change":"2023-10-10T20:40:08.017320+0000","last_active":"2023-10-10T20:40:09.018843+0000","last_peered":"2023-10-10T20:40:09.018843+0000","last_clean":"2023-10-10T20:40:09.018843+0000","last_became_active":"2023-10-10T20:40:08.017047+0000","last_became_peered":"2023-10-10T20:40:08.017047+0000","last_unstale":"2023-10-10T20:40:09.018843+0000","last_undegraded":"2023-10-10T20:40:09.018843+0000","last_fullsized":"2023-10-10T20:40:09.018843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:04:07.489913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.039038+0000","last_change":"2023-10-10T20:40:09.030527+0000","last_active":"2023-10-10T20:40:09.039038+0000","last_peered":"2023-10-10T20:40:09.039038+0000","last_clean":"2023-10-10T20:40:09.039038+0000","last_became_active":"2023-10-10T20:40:09.030229+0000","last_became_peered":"2023-10-10T20:40:09.030229+0000","last_unstale":"2023-10-10T20:40:09.039038+0000","last_undegraded":"2023-10-10T20:40:09.039038+0000","last_fullsized":"2023-10-10T20:40:09.039038+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:05:09.311895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"87'2","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.040817+0000","last_change":"2023-10-10T20:40:08.019439+0000","last_active":"2023-10-10T20:40:09.040817+0000","last_peered":"2023-10-10T20:40:09.040817+0000","last_clean":"2023-10-10T20:40:09.040817+0000","last_became_active":"2023-10-10T20:40:08.019180+0000","last_became_peered":"2023-10-10T20:40:08.019180+0000","last_unstale":"2023-10-10T20:40:09.040817+0000","last_undegraded":"2023-10-10T20:40:09.040817+0000","last_fullsized":"2023-10-10T20:40:09.040817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:07:29.691713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.030538+0000","last_change":"2023-10-10T20:40:09.025562+0000","last_active":"2023-10-10T20:40:09.030538+0000","last_peered":"2023-10-10T20:40:09.030538+0000","last_clean":"2023-10-10T20:40:09.030538+0000","last_became_active":"2023-10-10T20:40:09.025262+0000","last_became_peered":"2023-10-10T20:40:09.025262+0000","last_unstale":"2023-10-10T20:40:09.030538+0000","last_undegraded":"2023-10-10T20:40:09.030538+0000","last_fullsized":"2023-10-10T20:40:09.030538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:00:21.130578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.018135+0000","last_change":"2023-10-10T20:40:08.014985+0000","last_active":"2023-10-10T20:40:09.018135+0000","last_peered":"2023-10-10T20:40:09.018135+0000","last_clean":"2023-10-10T20:40:09.018135+0000","last_became_active":"2023-10-10T20:40:08.014719+0000","last_became_peered":"2023-10-10T20:40:08.014719+0000","last_unstale":"2023-10-10T20:40:09.018135+0000","last_undegraded":"2023-10-10T20:40:09.018135+0000","last_fullsized":"2023-10-10T20:40:09.018135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:32:46.206016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.034166+0000","last_change":"2023-10-10T20:40:09.029693+0000","last_active":"2023-10-10T20:40:09.034166+0000","last_peered":"2023-10-10T20:40:09.034166+0000","last_clean":"2023-10-10T20:40:09.034166+0000","last_became_active":"2023-10-10T20:40:09.029426+0000","last_became_peered":"2023-10-10T20:40:09.029426+0000","last_unstale":"2023-10-10T20:40:09.034166+0000","last_undegraded":"2023-10-10T20:40:09.034166+0000","last_fullsized":"2023-10-10T20:40:09.034166+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:04:43.775540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.017546+0000","last_change":"2023-10-10T20:40:08.011862+0000","last_active":"2023-10-10T20:40:09.017546+0000","last_peered":"2023-10-10T20:40:09.017546+0000","last_clean":"2023-10-10T20:40:09.017546+0000","last_became_active":"2023-10-10T20:40:08.011579+0000","last_became_peered":"2023-10-10T20:40:08.011579+0000","last_unstale":"2023-10-10T20:40:09.017546+0000","last_undegraded":"2023-10-10T20:40:09.017546+0000","last_fullsized":"2023-10-10T20:40:09.017546+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:55:55.484134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.041205+0000","last_change":"2023-10-10T20:40:09.036718+0000","last_active":"2023-10-10T20:40:09.041205+0000","last_peered":"2023-10-10T20:40:09.041205+0000","last_clean":"2023-10-10T20:40:09.041205+0000","last_became_active":"2023-10-10T20:40:09.036419+0000","last_became_peered":"2023-10-10T20:40:09.036419+0000","last_unstale":"2023-10-10T20:40:09.041205+0000","last_undegraded":"2023-10-10T20:40:09.041205+0000","last_fullsized":"2023-10-10T20:40:09.041205+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:56:36.966922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.022300+0000","last_change":"2023-10-10T20:40:08.018961+0000","last_active":"2023-10-10T20:40:09.022300+0000","last_peered":"2023-10-10T20:40:09.022300+0000","last_clean":"2023-10-10T20:40:09.022300+0000","last_became_active":"2023-10-10T20:40:08.016455+0000","last_became_peered":"2023-10-10T20:40:08.016455+0000","last_unstale":"2023-10-10T20:40:09.022300+0000","last_undegraded":"2023-10-10T20:40:09.022300+0000","last_fullsized":"2023-10-10T20:40:09.022300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:10:02.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.037783+0000","last_change":"2023-10-10T20:40:09.035028+0000","last_active":"2023-10-10T20:40:09.037783+0000","last_peered":"2023-10-10T20:40:09.037783+0000","last_clean":"2023-10-10T20:40:09.037783+0000","last_became_active":"2023-10-10T20:40:09.034579+0000","last_became_peered":"2023-10-10T20:40:09.034579+0000","last_unstale":"2023-10-10T20:40:09.037783+0000","last_undegraded":"2023-10-10T20:40:09.037783+0000","last_fullsized":"2023-10-10T20:40:09.037783+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:11:46.763981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"87'2","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.044495+0000","last_change":"2023-10-10T20:40:08.012505+0000","last_active":"2023-10-10T20:40:09.044495+0000","last_peered":"2023-10-10T20:40:09.044495+0000","last_clean":"2023-10-10T20:40:09.044495+0000","last_became_active":"2023-10-10T20:40:08.012221+0000","last_became_peered":"2023-10-10T20:40:08.012221+0000","last_unstale":"2023-10-10T20:40:09.044495+0000","last_undegraded":"2023-10-10T20:40:09.044495+0000","last_fullsized":"2023-10-10T20:40:09.044495+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:04:07.489913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.032777+0000","last_change":"2023-10-10T20:40:09.024924+0000","last_active":"2023-10-10T20:40:09.032777+0000","last_peered":"2023-10-10T20:40:09.032777+0000","last_clean":"2023-10-10T20:40:09.032777+0000","last_became_active":"2023-10-10T20:40:09.024628+0000","last_became_peered":"2023-10-10T20:40:09.024628+0000","last_unstale":"2023-10-10T20:40:09.032777+0000","last_undegraded":"2023-10-10T20:40:09.032777+0000","last_fullsized":"2023-10-10T20:40:09.032777+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:10:03.561466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.019937+0000","last_change":"2023-10-10T20:40:08.018305+0000","last_active":"2023-10-10T20:40:09.019937+0000","last_peered":"2023-10-10T20:40:09.019937+0000","last_clean":"2023-10-10T20:40:09.019937+0000","last_became_active":"2023-10-10T20:40:08.017551+0000","last_became_peered":"2023-10-10T20:40:08.017551+0000","last_unstale":"2023-10-10T20:40:09.019937+0000","last_undegraded":"2023-10-10T20:40:09.019937+0000","last_fullsized":"2023-10-10T20:40:09.019937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:35:02.298470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.040630+0000","last_change":"2023-10-10T20:40:09.037789+0000","last_active":"2023-10-10T20:40:09.040630+0000","last_peered":"2023-10-10T20:40:09.040630+0000","last_clean":"2023-10-10T20:40:09.040630+0000","last_became_active":"2023-10-10T20:40:09.037489+0000","last_became_peered":"2023-10-10T20:40:09.037489+0000","last_unstale":"2023-10-10T20:40:09.040630+0000","last_undegraded":"2023-10-10T20:40:09.040630+0000","last_fullsized":"2023-10-10T20:40:09.040630+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:03:12.361782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.020544+0000","last_change":"2023-10-10T20:40:08.018155+0000","last_active":"2023-10-10T20:40:09.020544+0000","last_peered":"2023-10-10T20:40:09.020544+0000","last_clean":"2023-10-10T20:40:09.020544+0000","last_became_active":"2023-10-10T20:40:08.017421+0000","last_became_peered":"2023-10-10T20:40:08.017421+0000","last_unstale":"2023-10-10T20:40:09.020544+0000","last_undegraded":"2023-10-10T20:40:09.020544+0000","last_fullsized":"2023-10-10T20:40:09.020544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:28:06.502123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.038356+0000","last_change":"2023-10-10T20:40:09.032166+0000","last_active":"2023-10-10T20:40:09.038356+0000","last_peered":"2023-10-10T20:40:09.038356+0000","last_clean":"2023-10-10T20:40:09.038356+0000","last_became_active":"2023-10-10T20:40:09.031913+0000","last_became_peered":"2023-10-10T20:40:09.031913+0000","last_unstale":"2023-10-10T20:40:09.038356+0000","last_undegraded":"2023-10-10T20:40:09.038356+0000","last_fullsized":"2023-10-10T20:40:09.038356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:49:11.364608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.024391+0000","last_change":"2023-10-10T20:40:08.011090+0000","last_active":"2023-10-10T20:40:09.024391+0000","last_peered":"2023-10-10T20:40:09.024391+0000","last_clean":"2023-10-10T20:40:09.024391+0000","last_became_active":"2023-10-10T20:40:08.010837+0000","last_became_peered":"2023-10-10T20:40:08.010837+0000","last_unstale":"2023-10-10T20:40:09.024391+0000","last_undegraded":"2023-10-10T20:40:09.024391+0000","last_fullsized":"2023-10-10T20:40:09.024391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:55:55.484134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.038990+0000","last_change":"2023-10-10T20:40:09.030442+0000","last_active":"2023-10-10T20:40:09.038990+0000","last_peered":"2023-10-10T20:40:09.038990+0000","last_clean":"2023-10-10T20:40:09.038990+0000","last_became_active":"2023-10-10T20:40:09.029968+0000","last_became_peered":"2023-10-10T20:40:09.029968+0000","last_unstale":"2023-10-10T20:40:09.038990+0000","last_undegraded":"2023-10-10T20:40:09.038990+0000","last_fullsized":"2023-10-10T20:40:09.038990+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:28:13.195637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.021628+0000","last_change":"2023-10-10T20:40:08.018718+0000","last_active":"2023-10-10T20:40:09.021628+0000","last_peered":"2023-10-10T20:40:09.021628+0000","last_clean":"2023-10-10T20:40:09.021628+0000","last_became_active":"2023-10-10T20:40:08.018156+0000","last_became_peered":"2023-10-10T20:40:08.018156+0000","last_unstale":"2023-10-10T20:40:09.021628+0000","last_undegraded":"2023-10-10T20:40:09.021628+0000","last_fullsized":"2023-10-10T20:40:09.021628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:43:58.261869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.033310+0000","last_change":"2023-10-10T20:40:09.027538+0000","last_active":"2023-10-10T20:40:09.033310+0000","last_peered":"2023-10-10T20:40:09.033310+0000","last_clean":"2023-10-10T20:40:09.033310+0000","last_became_active":"2023-10-10T20:40:09.027254+0000","last_became_peered":"2023-10-10T20:40:09.027254+0000","last_unstale":"2023-10-10T20:40:09.033310+0000","last_undegraded":"2023-10-10T20:40:09.033310+0000","last_fullsized":"2023-10-10T20:40:09.033310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:24:41.449979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.025908+0000","last_change":"2023-10-10T20:40:08.014797+0000","last_active":"2023-10-10T20:40:09.025908+0000","last_peered":"2023-10-10T20:40:09.025908+0000","last_clean":"2023-10-10T20:40:09.025908+0000","last_became_active":"2023-10-10T20:40:08.014437+0000","last_became_peered":"2023-10-10T20:40:08.014437+0000","last_unstale":"2023-10-10T20:40:09.025908+0000","last_undegraded":"2023-10-10T20:40:09.025908+0000","last_fullsized":"2023-10-10T20:40:09.025908+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:35:02.298470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.032115+0000","last_change":"2023-10-10T20:40:09.028755+0000","last_active":"2023-10-10T20:40:09.032115+0000","last_peered":"2023-10-10T20:40:09.032115+0000","last_clean":"2023-10-10T20:40:09.032115+0000","last_became_active":"2023-10-10T20:40:09.028498+0000","last_became_peered":"2023-10-10T20:40:09.028498+0000","last_unstale":"2023-10-10T20:40:09.032115+0000","last_undegraded":"2023-10-10T20:40:09.032115+0000","last_fullsized":"2023-10-10T20:40:09.032115+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:35:34.522039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.014501+0000","last_change":"2023-10-10T20:40:08.011687+0000","last_active":"2023-10-10T20:40:09.014501+0000","last_peered":"2023-10-10T20:40:09.014501+0000","last_clean":"2023-10-10T20:40:09.014501+0000","last_became_active":"2023-10-10T20:40:08.011214+0000","last_became_peered":"2023-10-10T20:40:08.011214+0000","last_unstale":"2023-10-10T20:40:09.014501+0000","last_undegraded":"2023-10-10T20:40:09.014501+0000","last_fullsized":"2023-10-10T20:40:09.014501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:55:34.892864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.038271+0000","last_change":"2023-10-10T20:40:09.032099+0000","last_active":"2023-10-10T20:40:09.038271+0000","last_peered":"2023-10-10T20:40:09.038271+0000","last_clean":"2023-10-10T20:40:09.038271+0000","last_became_active":"2023-10-10T20:40:09.031833+0000","last_became_peered":"2023-10-10T20:40:09.031833+0000","last_unstale":"2023-10-10T20:40:09.038271+0000","last_undegraded":"2023-10-10T20:40:09.038271+0000","last_fullsized":"2023-10-10T20:40:09.038271+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:25:35.014775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.016867+0000","last_change":"2023-10-10T20:40:08.014834+0000","last_active":"2023-10-10T20:40:09.016867+0000","last_peered":"2023-10-10T20:40:09.016867+0000","last_clean":"2023-10-10T20:40:09.016867+0000","last_became_active":"2023-10-10T20:40:08.014555+0000","last_became_peered":"2023-10-10T20:40:08.014555+0000","last_unstale":"2023-10-10T20:40:09.016867+0000","last_undegraded":"2023-10-10T20:40:09.016867+0000","last_fullsized":"2023-10-10T20:40:09.016867+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:10:07.992434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.039076+0000","last_change":"2023-10-10T20:40:09.030461+0000","last_active":"2023-10-10T20:40:09.039076+0000","last_peered":"2023-10-10T20:40:09.039076+0000","last_clean":"2023-10-10T20:40:09.039076+0000","last_became_active":"2023-10-10T20:40:09.030181+0000","last_became_peered":"2023-10-10T20:40:09.030181+0000","last_unstale":"2023-10-10T20:40:09.039076+0000","last_undegraded":"2023-10-10T20:40:09.039076+0000","last_fullsized":"2023-10-10T20:40:09.039076+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:07.994486+0000","last_clean_scrub_stamp":"2023-10-10T20:40:07.994486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:17:24.346198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"87'1","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.040803+0000","last_change":"2023-10-10T20:40:08.009862+0000","last_active":"2023-10-10T20:40:09.040803+0000","last_peered":"2023-10-10T20:40:09.040803+0000","last_clean":"2023-10-10T20:40:09.040803+0000","last_became_active":"2023-10-10T20:40:08.009520+0000","last_became_peered":"2023-10-10T20:40:08.009520+0000","last_unstale":"2023-10-10T20:40:09.040803+0000","last_undegraded":"2023-10-10T20:40:09.040803+0000","last_fullsized":"2023-10-10T20:40:09.040803+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:06.990727+0000","last_clean_scrub_stamp":"2023-10-10T20:40:06.990727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:04:01.312167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":231,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-10T20:40:09.014999+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:40:09.014999+0000","last_peered":"2023-10-10T20:40:09.014999+0000","last_clean":"2023-10-10T20:40:09.014999+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:40:09.014999+0000","last_undegraded":"2023-10-10T20:40:09.014999+0000","last_fullsized":"2023-10-10T20:40:09.014999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574921,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303036,"kb_used_data":1652,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068804,"statfs":{"total":96636764160,"available":96326455296,"internally_reserved":0,"allocated":1691648,"data_stored":1059768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.60499999999999998,"15min":0.84999999999999998},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.747,"5min":6.8630000000000004,"15min":6.8630000000000004},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.56299999999999994,"15min":0.63800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.629,"5min":4.1959999999999997,"15min":4.1959999999999997},"last":0.48099999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.65300000000000002,"15min":1.0489999999999999},"min":{"1min":0.39900000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.78800000000000003,"5min":10.1,"15min":10.1},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.61299999999999999,"15min":0.749},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66400000000000003,"5min":6.806,"15min":6.806},"last":0.54900000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.64000000000000001,"15min":0.80200000000000005},"min":{"1min":0.41599999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":6.9340000000000002,"15min":6.9340000000000002},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59499999999999997,"15min":0.75900000000000001},"min":{"1min":0.38800000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73199999999999998,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.437}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61799999999999999,"15min":0.77800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70899999999999996,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.68100000000000005},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71899999999999997,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.41499999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.34599999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.626,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.374,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78400000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.63200000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.73299999999999998,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72699999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.61799999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58899999999999997,"15min":0.57199999999999995},"min":{"1min":0.48699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77100000000000002,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.46800000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75600000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.66400000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298360,"kb_used_data":1072,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073480,"statfs":{"total":96636764160,"available":96331243520,"internally_reserved":0,"allocated":1097728,"data_stored":469912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55700000000000005,"15min":0.68899999999999995},"min":{"1min":0.40799999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.73099999999999998,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":0.432},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57899999999999996,"15min":0.73299999999999998},"min":{"1min":0.372,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70799999999999996,"5min":7.8090000000000002,"15min":7.8090000000000002},"last":0.45500000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.72899999999999998},"min":{"1min":0.39700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71899999999999997,"5min":7.867,"15min":7.867},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.64200000000000002,"15min":0.92100000000000004},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75700000000000001,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.497}]},{"osd":2,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.747,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.63400000000000001,"15min":0.874},"min":{"1min":0.48399999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.71199999999999997,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":0.57499999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.628},"min":{"1min":0.439,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79000000000000004,"5min":4.8620000000000001,"15min":4.8620000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.73499999999999999},"min":{"1min":0.47199999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.755,"5min":7.9009999999999998,"15min":7.9009999999999998},"last":0.59899999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.504,"15min":0.46400000000000002},"min":{"1min":0.42799999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71799999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.56499999999999995},"min":{"1min":0.54200000000000004,"5min":0.37,"15min":0.37},"max":{"1min":0.69899999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67600000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51400000000000001,"15min":0.48599999999999999},"min":{"1min":0.40400000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55200000000000005,"15min":0.52600000000000002},"min":{"1min":0.47799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.70999999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.42799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74399999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56899999999999995,"15min":0.54100000000000004},"min":{"1min":0.52700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.747,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.64100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298352,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073488,"statfs":{"total":96636764160,"available":96331251712,"internally_reserved":0,"allocated":1089536,"data_stored":469288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.36399999999999999,"15min":0.49099999999999999},"min":{"1min":0.219,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.56799999999999995,"5min":5.0380000000000003,"15min":5.0380000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33100000000000002,"15min":0.36499999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.47299999999999998,"5min":1.194,"15min":1.194},"last":0.28299999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.36799999999999999},"min":{"1min":0.27800000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.51400000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.52000000000000002},"min":{"1min":0.26000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52500000000000002,"5min":5.0919999999999996,"15min":5.0919999999999996},"last":0.35899999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.40799999999999997,"15min":0.54300000000000004},"min":{"1min":0.28000000000000003,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.59899999999999998,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38100000000000001,"15min":0.41299999999999998},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55500000000000005,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.371}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.40000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92400000000000004,"5min":1.254,"15min":1.254},"last":0.495},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60599999999999998,"15min":0.71899999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96499999999999997,"5min":4.9850000000000003,"15min":4.9850000000000003},"last":0.58799999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97599999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.42899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.878,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.435}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67600000000000005,"15min":0.86699999999999999},"min":{"1min":0.46500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.995,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.621},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.40699999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85799999999999998,"5min":1.171,"15min":1.171},"last":0.54200000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60099999999999998,"15min":0.62},"min":{"1min":0.443,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.004,"5min":1.214,"15min":1.214},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63700000000000001,"15min":0.73699999999999999},"min":{"1min":0.41999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90500000000000003,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.45900000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574920,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":1059249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.377,"15min":0.55100000000000005},"min":{"1min":0.20699999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.51600000000000001,"5min":7.9249999999999998,"15min":7.9249999999999998},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.374,"15min":0.52400000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45800000000000002,"5min":6.6299999999999999,"15min":6.6299999999999999},"last":0.35899999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.39000000000000001,"15min":0.57399999999999995},"min":{"1min":0.23100000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.47499999999999998,"5min":8.2509999999999994,"15min":8.2509999999999994},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39800000000000002,"15min":0.54800000000000004},"min":{"1min":0.245,"5min":0.222,"15min":0.222},"max":{"1min":0.52400000000000002,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.32800000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.40500000000000003,"15min":0.57699999999999996},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.45400000000000001,"15min":0.63400000000000001},"min":{"1min":0.31900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57899999999999996,"5min":7.8929999999999998,"15min":7.8929999999999998},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.61099999999999999,"15min":0.73199999999999998},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78000000000000003,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.63600000000000001,"15min":0.79600000000000004},"min":{"1min":0.45100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.95399999999999996,"5min":7.6920000000000002,"15min":7.6920000000000002},"last":0.67600000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.77500000000000002},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92400000000000004,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.64300000000000002,"15min":0.79900000000000004},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91400000000000003,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":0.68300000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67000000000000004,"15min":0.83999999999999997},"min":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94499999999999995,"5min":7.8360000000000003,"15min":7.8360000000000003},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66400000000000003,"15min":0.77900000000000003},"min":{"1min":0.49199999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90300000000000002,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.72299999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64000000000000001,"15min":0.76900000000000002},"min":{"1min":0.45900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93300000000000005,"5min":6.7160000000000002,"15min":6.7160000000000002},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66100000000000003,"15min":0.80900000000000005},"min":{"1min":0.44600000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.878,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.65700000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298356,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073484,"statfs":{"total":96636764160,"available":96331247616,"internally_reserved":0,"allocated":1093632,"data_stored":469322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.31900000000000001,"15min":0.32800000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.59699999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.312},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.33600000000000002,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45400000000000001,"5min":0.746,"15min":0.746},"last":0.215}]},{"osd":1,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.45400000000000001,"15min":0.77100000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.63,"5min":11.967000000000001,"15min":11.967000000000001},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39000000000000001,"15min":0.498},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52700000000000002,"5min":4.1100000000000003,"15min":4.1100000000000003},"last":0.28799999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.41199999999999998,"15min":0.57899999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48799999999999999,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.50900000000000001,"15min":0.82199999999999995},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65900000000000003,"5min":11.935,"15min":11.935},"last":0.34200000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.60299999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.80100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.58999999999999997},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.53800000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.33400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83599999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.52300000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81599999999999995,"5min":1.052,"15min":1.052},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64500000000000002,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85999999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.66000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302464,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069376,"statfs":{"total":96636764160,"available":96327041024,"internally_reserved":0,"allocated":1105920,"data_stored":470846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.309,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44400000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.317},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48699999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.32100000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33600000000000002,"15min":0.33000000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.49199999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38200000000000001,"15min":0.376},"min":{"1min":0.307,"5min":0.251,"15min":0.251},"max":{"1min":0.53900000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.55200000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.34899999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35499999999999998,"15min":0.36099999999999999},"min":{"1min":0.25900000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.51600000000000001,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.44800000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.38100000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83299999999999996,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.79000000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.39400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78700000000000003,"5min":1.05,"15min":1.05},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86799999999999999,"5min":1.01,"15min":1.01},"last":0.61599999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92000000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.877,"5min":1.026,"15min":1.026},"last":0.71199999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77100000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.46600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89400000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.81699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298372,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073468,"statfs":{"total":96636764160,"available":96331231232,"internally_reserved":0,"allocated":1110016,"data_stored":470948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58199999999999996,"5min":0.747,"15min":0.747},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.59799999999999998,"15min":0.79200000000000004},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69699999999999995,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.52700000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.60299999999999998,"15min":0.79900000000000004},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60299999999999998,"5min":7.7160000000000002,"15min":7.7160000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54000000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68100000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63600000000000001,"15min":0.81799999999999995},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.57599999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.58899999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.41199999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.502,"15min":0.48399999999999999},"min":{"1min":0.34599999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.437},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.38400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77300000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.59499999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.41699999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72099999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.52600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.61799999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.58199999999999996},"min":{"1min":0.44700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.629},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.60499999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298356,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073484,"statfs":{"total":96636764160,"available":96331247616,"internally_reserved":0,"allocated":1093632,"data_stored":469358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.307,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63600000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.34300000000000003,"5min":0.215,"15min":0.215},"max":{"1min":0.70399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.76700000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.56200000000000006,"15min":0.65300000000000002},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":0.629},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55300000000000005,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68200000000000005,"5min":5.3109999999999999,"15min":5.3109999999999999},"last":0.70299999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56899999999999995,"15min":0.66100000000000003},"min":{"1min":0.32900000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64700000000000002,"5min":5.2960000000000003,"15min":5.2960000000000003},"last":0.746},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72399999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.67900000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59499999999999997,"15min":0.69099999999999995},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70399999999999996,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59699999999999998,"15min":0.70399999999999996},"min":{"1min":0.40300000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70599999999999996,"5min":5.2850000000000001,"15min":5.2850000000000001},"last":0.65600000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48299999999999998,"15min":0.497},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.63800000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65000000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.60899999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.65100000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71299999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58599999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.33000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72699999999999998,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.56000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:40:18.021 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:40:18.021 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:40:18.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.174+0000 7f91e5072700 1 Processor -- start 2023-10-10T20:40:18.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.174+0000 7f91e5072700 1 -- start start 2023-10-10T20:40:18.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.175+0000 7f91e5072700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91e01594e0 0x7f91e00ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:18.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.175+0000 7f91e5072700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f91e01598b0 0x7f91e00b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:18.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.175+0000 7f91e5072700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e00cf1f0 con 0x7f91e00b60e0 2023-10-10T20:40:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.175+0000 7f91e5072700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e00cf4d0 con 0x7f91e01598b0 2023-10-10T20:40:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.175+0000 7f91e5072700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00cf7b0 con 0x7f91e01594e0 2023-10-10T20:40:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.175+0000 7f91df7fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f91e01598b0 0x7f91e00b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.175+0000 7f91dffff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91e01594e0 0x7f91e00ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.175+0000 7f91df7fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f91e01598b0 0x7f91e00b1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:52726/0 (socket says 172.21.15.89:52726) 2023-10-10T20:40:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.175+0000 7f91dffff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91e01594e0 0x7f91e00ceb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:37210/0 (socket says 172.21.15.89:37210) 2023-10-10T20:40:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.175+0000 7f91df7fe700 1 -- 172.21.15.89:0/2405708206 learned_addr learned my addr 172.21.15.89:0/2405708206 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.175+0000 7f91e4871700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f91e00b60e0 0x7f91e015d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:60114/0 (socket says 172.21.15.89:60114) 2023-10-10T20:40:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.176+0000 7f91df7fe700 1 -- 172.21.15.89:0/2405708206 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91e01594e0 msgr2=0x7f91e00ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.176+0000 7f91df7fe700 1 --2- 172.21.15.89:0/2405708206 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91e01594e0 0x7f91e00ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.176+0000 7f91df7fe700 1 -- 172.21.15.89:0/2405708206 >> v1:172.21.15.111:6789/0 conn(0x7f91e00b60e0 legacy=0x7f91e015d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.176+0000 7f91df7fe700 1 -- 172.21.15.89:0/2405708206 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e00cfa90 con 0x7f91e01598b0 2023-10-10T20:40:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.176+0000 7f91df7fe700 1 --2- 172.21.15.89:0/2405708206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91e01598b0 0x7f91e00b1700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f91d40099b0 tx=0x7f91d4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=951940ed05886ffd server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.176+0000 7f91deffd700 1 -- 172.21.15.89:0/2405708206 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4009ea0 con 0x7f91e01598b0 2023-10-10T20:40:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.176+0000 7f91deffd700 1 -- 172.21.15.89:0/2405708206 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d400b860 con 0x7f91e01598b0 2023-10-10T20:40:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.176+0000 7f91deffd700 1 -- 172.21.15.89:0/2405708206 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d400bc00 con 0x7f91e01598b0 2023-10-10T20:40:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.176+0000 7f91e5072700 1 -- 172.21.15.89:0/2405708206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91e01598b0 msgr2=0x7f91e00b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.176+0000 7f91e5072700 1 --2- 172.21.15.89:0/2405708206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91e01598b0 0x7f91e00b1700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f91d40099b0 tx=0x7f91d4009250 comp rx=0 tx=0).stop 2023-10-10T20:40:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.177+0000 7f91e5072700 1 -- 172.21.15.89:0/2405708206 shutdown_connections 2023-10-10T20:40:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.177+0000 7f91e5072700 1 --2- 172.21.15.89:0/2405708206 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91e01594e0 0x7f91e00ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.177+0000 7f91e5072700 1 --2- 172.21.15.89:0/2405708206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91e01598b0 0x7f91e00b1700 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.177+0000 7f91e5072700 1 -- 172.21.15.89:0/2405708206 >> 172.21.15.89:0/2405708206 conn(0x7f91e00d10f0 msgr2=0x7f91e00affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.177+0000 7f91e5072700 1 -- 172.21.15.89:0/2405708206 shutdown_connections 2023-10-10T20:40:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.177+0000 7f91e5072700 1 -- 172.21.15.89:0/2405708206 wait complete. 2023-10-10T20:40:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.178+0000 7f91e5072700 1 Processor -- start 2023-10-10T20:40:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.178+0000 7f91e5072700 1 -- start start 2023-10-10T20:40:18.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.178+0000 7f91e5072700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91e00b60e0 0x7f91e015db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:18.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.178+0000 7f91e5072700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91e01594e0 0x7f91e0154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:18.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.178+0000 7f91e5072700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91e01598b0 0x7f91e0146df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:18.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.178+0000 7f91e5072700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e015cdb0 con 0x7f91e00b60e0 2023-10-10T20:40:18.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.178+0000 7f91e5072700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e0152700 con 0x7f91e01598b0 2023-10-10T20:40:18.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.178+0000 7f91e5072700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e015e090 con 0x7f91e01594e0 2023-10-10T20:40:18.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.179+0000 7f91dffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91e00b60e0 0x7f91e015db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:18.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.179+0000 7f91dffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91e00b60e0 0x7f91e015db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56434/0 (socket says 172.21.15.89:56434) 2023-10-10T20:40:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.179+0000 7f91dffff700 1 -- 172.21.15.89:0/3940738586 learned_addr learned my addr 172.21.15.89:0/3940738586 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.179+0000 7f91df7fe700 1 --2- 172.21.15.89:0/3940738586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91e01594e0 0x7f91e0154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.179+0000 7f91e4871700 1 --2- 172.21.15.89:0/3940738586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91e01598b0 0x7f91e0146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.179+0000 7f91dffff700 1 -- 172.21.15.89:0/3940738586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91e01594e0 msgr2=0x7f91e0154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.179+0000 7f91dffff700 1 --2- 172.21.15.89:0/3940738586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91e01594e0 0x7f91e0154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.179+0000 7f91dffff700 1 -- 172.21.15.89:0/3940738586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91e01598b0 msgr2=0x7f91e0146df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:18.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.179+0000 7f91dffff700 1 --2- 172.21.15.89:0/3940738586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91e01598b0 0x7f91e0146df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.179+0000 7f91dffff700 1 -- 172.21.15.89:0/3940738586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d40094f0 con 0x7f91e00b60e0 2023-10-10T20:40:18.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.179+0000 7f91dffff700 1 --2- 172.21.15.89:0/3940738586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91e00b60e0 0x7f91e015db50 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f91d000ec80 tx=0x7f91d0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.179+0000 7f91c7fff700 1 -- 172.21.15.89:0/3940738586 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d000c850 con 0x7f91e00b60e0 2023-10-10T20:40:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.179+0000 7f91c7fff700 1 -- 172.21.15.89:0/3940738586 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d000c9b0 con 0x7f91e00b60e0 2023-10-10T20:40:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.179+0000 7f91e5072700 1 -- 172.21.15.89:0/3940738586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e015c1a0 con 0x7f91e00b60e0 2023-10-10T20:40:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.179+0000 7f91c7fff700 1 -- 172.21.15.89:0/3940738586 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d000cd30 con 0x7f91e00b60e0 2023-10-10T20:40:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.179+0000 7f91e5072700 1 -- 172.21.15.89:0/3940738586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e015c760 con 0x7f91e00b60e0 2023-10-10T20:40:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.180+0000 7f91e5072700 1 -- 172.21.15.89:0/3940738586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e004c3c0 con 0x7f91e00b60e0 2023-10-10T20:40:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.181+0000 7f91c7fff700 1 -- 172.21.15.89:0/3940738586 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f91d0013680 con 0x7f91e00b60e0 2023-10-10T20:40:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.181+0000 7f91c7fff700 1 --2- 172.21.15.89:0/3940738586 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f91cc067200 0x7f91cc0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.181+0000 7f91c7fff700 1 -- 172.21.15.89:0/3940738586 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f91d0084f80 con 0x7f91e00b60e0 2023-10-10T20:40:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.181+0000 7f91df7fe700 1 --2- 172.21.15.89:0/3940738586 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f91cc067200 0x7f91cc0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.181+0000 7f91df7fe700 1 --2- 172.21.15.89:0/3940738586 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f91cc067200 0x7f91cc0696b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f91e0159d20 tx=0x7f91d4017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.185+0000 7f91c7fff700 1 -- 172.21.15.89:0/3940738586 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91d0051ea0 con 0x7f91e00b60e0 2023-10-10T20:40:18.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.445+0000 7f91e5072700 1 -- 172.21.15.89:0/3940738586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f91e015d090 con 0x7f91e00b60e0 2023-10-10T20:40:18.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.449+0000 7f91c7fff700 1 -- 172.21.15.89:0/3940738586 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7f91d00210e0 con 0x7f91e00b60e0 2023-10-10T20:40:18.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.451+0000 7f91e5072700 1 -- 172.21.15.89:0/3940738586 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f91cc067200 msgr2=0x7f91cc0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:18.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.451+0000 7f91e5072700 1 --2- 172.21.15.89:0/3940738586 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f91cc067200 0x7f91cc0696b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f91e0159d20 tx=0x7f91d4017000 comp rx=0 tx=0).stop 2023-10-10T20:40:18.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.451+0000 7f91e5072700 1 -- 172.21.15.89:0/3940738586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91e00b60e0 msgr2=0x7f91e015db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:18.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.451+0000 7f91e5072700 1 --2- 172.21.15.89:0/3940738586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91e00b60e0 0x7f91e015db50 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f91d000ec80 tx=0x7f91d0009120 comp rx=0 tx=0).stop 2023-10-10T20:40:18.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.451+0000 7f91e5072700 1 -- 172.21.15.89:0/3940738586 shutdown_connections 2023-10-10T20:40:18.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.451+0000 7f91e5072700 1 --2- 172.21.15.89:0/3940738586 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f91cc067200 0x7f91cc0696b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.451+0000 7f91e5072700 1 --2- 172.21.15.89:0/3940738586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91e01598b0 0x7f91e0146df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.451+0000 7f91e5072700 1 --2- 172.21.15.89:0/3940738586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91e00b60e0 0x7f91e015db50 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.451+0000 7f91e5072700 1 --2- 172.21.15.89:0/3940738586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f91e01594e0 0x7f91e0154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.451+0000 7f91e5072700 1 -- 172.21.15.89:0/3940738586 >> 172.21.15.89:0/3940738586 conn(0x7f91e00d10f0 msgr2=0x7f91e00b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:18.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.451+0000 7f91e5072700 1 -- 172.21.15.89:0/3940738586 shutdown_connections 2023-10-10T20:40:18.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.451+0000 7f91e5072700 1 -- 172.21.15.89:0/3940738586 wait complete. 2023-10-10T20:40:18.456 INFO:teuthology.orchestra.run.smithi089.stderr:updated caps for client.0 2023-10-10T20:40:18.465 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:40:18.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.578+0000 7fb9dca11700 1 Processor -- start 2023-10-10T20:40:18.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.578+0000 7fb9dca11700 1 -- start start 2023-10-10T20:40:18.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.579+0000 7fb9dca11700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb9d806bcd0 0x7fb9d8140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:18.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.579+0000 7fb9dca11700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d815c2a0 con 0x7fb9d80c9510 2023-10-10T20:40:18.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.579+0000 7fb9dca11700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d815b690 con 0x7fb9d806c0a0 2023-10-10T20:40:18.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.579+0000 7fb9dca11700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d80c9900 con 0x7fb9d806bcd0 2023-10-10T20:40:18.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.579+0000 7fb9d659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb9d806bcd0 0x7fb9d8140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:18.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.579+0000 7fb9d6d9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb9d80c9510 0x7fb9d80cc1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36386/0 (socket says 172.21.15.89:36386) 2023-10-10T20:40:18.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.579+0000 7fb9d6d9d700 1 -- 172.21.15.89:0/3486591860 learned_addr learned my addr 172.21.15.89:0/3486591860 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:18.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.580+0000 7fb9d659c700 1 -- 172.21.15.89:0/3486591860 >> v1:172.21.15.111:6789/0 conn(0x7fb9d806c0a0 legacy=0x7fb9d80c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:18.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.580+0000 7fb9d659c700 1 -- 172.21.15.89:0/3486591860 >> v1:172.21.15.89:6789/0 conn(0x7fb9d80c9510 legacy=0x7fb9d80cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:18.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.580+0000 7fb9d659c700 1 -- 172.21.15.89:0/3486591860 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d815b970 con 0x7fb9d806bcd0 2023-10-10T20:40:18.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.580+0000 7fb9d5d9b700 1 -- 172.21.15.89:0/3486591860 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259915697 0 0) 0x7fb9d815c2a0 con 0x7fb9d80c9510 2023-10-10T20:40:18.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.580+0000 7fb9d659c700 1 --2- 172.21.15.89:0/3486591860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9d806bcd0 0x7fb9d8140df0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80098e0 tx=0x7fb9c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26d700c73fb3b44 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:18.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.580+0000 7fb9d5d9b700 1 -- 172.21.15.89:0/3486591860 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8009e80 con 0x7fb9d806bcd0 2023-10-10T20:40:18.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.580+0000 7fb9d5d9b700 1 -- 172.21.15.89:0/3486591860 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c800b860 con 0x7fb9d806bcd0 2023-10-10T20:40:18.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.580+0000 7fb9d5d9b700 1 -- 172.21.15.89:0/3486591860 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c800bbe0 con 0x7fb9d806bcd0 2023-10-10T20:40:18.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.580+0000 7fb9dca11700 1 -- 172.21.15.89:0/3486591860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9d806bcd0 msgr2=0x7fb9d8140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:18.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.580+0000 7fb9dca11700 1 --2- 172.21.15.89:0/3486591860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9d806bcd0 0x7fb9d8140df0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80098e0 tx=0x7fb9c80091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.580+0000 7fb9dca11700 1 -- 172.21.15.89:0/3486591860 shutdown_connections 2023-10-10T20:40:18.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.580+0000 7fb9dca11700 1 --2- 172.21.15.89:0/3486591860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9d806bcd0 0x7fb9d8140df0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.580+0000 7fb9dca11700 1 -- 172.21.15.89:0/3486591860 >> 172.21.15.89:0/3486591860 conn(0x7fb9d80ce600 msgr2=0x7fb9d80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:18.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.580+0000 7fb9dca11700 1 -- 172.21.15.89:0/3486591860 shutdown_connections 2023-10-10T20:40:18.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.580+0000 7fb9dca11700 1 -- 172.21.15.89:0/3486591860 wait complete. 2023-10-10T20:40:18.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.582+0000 7fb9dca11700 1 Processor -- start 2023-10-10T20:40:18.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.582+0000 7fb9dca11700 1 -- start start 2023-10-10T20:40:18.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.582+0000 7fb9dca11700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb9d806bcd0 0x7fb9d80c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:18.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.582+0000 7fb9dca11700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9d806c0a0 0x7fb9d813fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:18.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.582+0000 7fb9dca11700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9d80c9510 0x7fb9d8154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:18.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.582+0000 7fb9dca11700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80c61c0 con 0x7fb9d80c9510 2023-10-10T20:40:18.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.582+0000 7fb9dca11700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80c64a0 con 0x7fb9d806c0a0 2023-10-10T20:40:18.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.582+0000 7fb9dca11700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80c6780 con 0x7fb9d806bcd0 2023-10-10T20:40:18.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.582+0000 7fb9d6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9d80c9510 0x7fb9d8154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:18.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.583+0000 7fb9d6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9d80c9510 0x7fb9d8154460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56458/0 (socket says 172.21.15.89:56458) 2023-10-10T20:40:18.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.583+0000 7fb9d6d9d700 1 -- 172.21.15.89:0/342321080 learned_addr learned my addr 172.21.15.89:0/342321080 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:18.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.583+0000 7fb9d659c700 1 --2- 172.21.15.89:0/342321080 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb9d806bcd0 0x7fb9d80c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:18.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.583+0000 7fb9cdd9b700 1 --2- 172.21.15.89:0/342321080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9d806c0a0 0x7fb9d813fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:18.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.583+0000 7fb9d6d9d700 1 -- 172.21.15.89:0/342321080 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb9d806bcd0 msgr2=0x7fb9d80c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:18.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.583+0000 7fb9d6d9d700 1 --2- 172.21.15.89:0/342321080 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb9d806bcd0 0x7fb9d80c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.583+0000 7fb9d6d9d700 1 -- 172.21.15.89:0/342321080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9d806c0a0 msgr2=0x7fb9d813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:18.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.583+0000 7fb9d6d9d700 1 --2- 172.21.15.89:0/342321080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9d806c0a0 0x7fb9d813fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.583+0000 7fb9d6d9d700 1 -- 172.21.15.89:0/342321080 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c8009420 con 0x7fb9d80c9510 2023-10-10T20:40:18.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.583+0000 7fb9cdd9b700 1 --2- 172.21.15.89:0/342321080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9d806c0a0 0x7fb9d813fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:18.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.583+0000 7fb9d6d9d700 1 --2- 172.21.15.89:0/342321080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9d80c9510 0x7fb9d8154460 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000cc10 tx=0x7fb9d0001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:18.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.583+0000 7fb9ceffd700 1 -- 172.21.15.89:0/342321080 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d000a930 con 0x7fb9d80c9510 2023-10-10T20:40:18.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.583+0000 7fb9ceffd700 1 -- 172.21.15.89:0/342321080 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d000aab0 con 0x7fb9d80c9510 2023-10-10T20:40:18.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.583+0000 7fb9dca11700 1 -- 172.21.15.89:0/342321080 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d80c6a60 con 0x7fb9d80c9510 2023-10-10T20:40:18.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.583+0000 7fb9ceffd700 1 -- 172.21.15.89:0/342321080 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d000ae70 con 0x7fb9d80c9510 2023-10-10T20:40:18.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.583+0000 7fb9dca11700 1 -- 172.21.15.89:0/342321080 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d80c5e00 con 0x7fb9d80c9510 2023-10-10T20:40:18.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.584+0000 7fb9dca11700 1 -- 172.21.15.89:0/342321080 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d80b26a0 con 0x7fb9d80c9510 2023-10-10T20:40:18.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.584+0000 7fb9ceffd700 1 -- 172.21.15.89:0/342321080 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fb9d0007300 con 0x7fb9d80c9510 2023-10-10T20:40:18.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.585+0000 7fb9ceffd700 1 --2- 172.21.15.89:0/342321080 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb9b8067200 0x7fb9b80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:18.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.585+0000 7fb9ceffd700 1 -- 172.21.15.89:0/342321080 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb9d0084ea0 con 0x7fb9d80c9510 2023-10-10T20:40:18.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.585+0000 7fb9d659c700 1 --2- 172.21.15.89:0/342321080 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb9b8067200 0x7fb9b80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:18.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.585+0000 7fb9d659c700 1 --2- 172.21.15.89:0/342321080 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb9b8067200 0x7fb9b80696b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80091e0 tx=0x7fb9c8016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:18.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.589+0000 7fb9ceffd700 1 -- 172.21.15.89:0/342321080 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9d80b26a0 con 0x7fb9d80c9510 2023-10-10T20:40:18.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.815+0000 7fb9dca11700 1 -- 172.21.15.89:0/342321080 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9d80b26a0 con 0x7fb9d80c9510 2023-10-10T20:40:18.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.815+0000 7fb9ceffd700 1 -- 172.21.15.89:0/342321080 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fb9d80b26a0 con 0x7fb9d80c9510 2023-10-10T20:40:18.818 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:18.818 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":55,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5995,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/3194805494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3194805494},{"type":"v1","addr":"172.21.15.111:6834","nonce":3194805494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6447,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/2766962252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2766962252},{"type":"v1","addr":"172.21.15.89:6836","nonce":2766962252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6450,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1141782720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1141782720},{"type":"v1","addr":"172.21.15.89:6837","nonce":1141782720}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:40:08.387849+0000","modified":"2023-10-10T20:40:12.040215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6462":{"gid":6462,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/4069200228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":4069200228},{"type":"v1","addr":"172.21.15.111:6835","nonce":4069200228}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-10T20:40:18.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.817+0000 7fb9dca11700 1 -- 172.21.15.89:0/342321080 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb9b8067200 msgr2=0x7fb9b80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:18.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.817+0000 7fb9dca11700 1 --2- 172.21.15.89:0/342321080 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb9b8067200 0x7fb9b80696b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80091e0 tx=0x7fb9c8016000 comp rx=0 tx=0).stop 2023-10-10T20:40:18.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.817+0000 7fb9dca11700 1 -- 172.21.15.89:0/342321080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9d80c9510 msgr2=0x7fb9d8154460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:18.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.817+0000 7fb9dca11700 1 --2- 172.21.15.89:0/342321080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9d80c9510 0x7fb9d8154460 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000cc10 tx=0x7fb9d0001970 comp rx=0 tx=0).stop 2023-10-10T20:40:18.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.817+0000 7fb9dca11700 1 -- 172.21.15.89:0/342321080 shutdown_connections 2023-10-10T20:40:18.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.817+0000 7fb9dca11700 1 --2- 172.21.15.89:0/342321080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9d80c9510 0x7fb9d8154460 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.817+0000 7fb9dca11700 1 --2- 172.21.15.89:0/342321080 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb9d806bcd0 0x7fb9d80c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.817+0000 7fb9dca11700 1 --2- 172.21.15.89:0/342321080 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb9b8067200 0x7fb9b80696b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.817+0000 7fb9dca11700 1 --2- 172.21.15.89:0/342321080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9d806c0a0 0x7fb9d813fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:18.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.817+0000 7fb9dca11700 1 -- 172.21.15.89:0/342321080 >> 172.21.15.89:0/342321080 conn(0x7fb9d80ce600 msgr2=0x7fb9d80c4450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:18.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.817+0000 7fb9dca11700 1 -- 172.21.15.89:0/342321080 shutdown_connections 2023-10-10T20:40:18.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:18.817+0000 7fb9dca11700 1 -- 172.21.15.89:0/342321080 wait complete. 2023-10-10T20:40:18.822 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 55 2023-10-10T20:40:19.234 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:40:08.387849+0000', 'modified': '2023-10-10T20:40:12.040215+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6462}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6462': {'gid': 6462, 'name': 'd', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6835/4069200228', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6833', 'nonce': 4069200228}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 4069200228}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:40:19.234 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:40:19.235 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-10T20:40:19.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.351+0000 7f56468b4700 1 Processor -- start 2023-10-10T20:40:19.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.351+0000 7f56468b4700 1 -- start start 2023-10-10T20:40:19.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.351+0000 7f56468b4700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5640159890 0x7f564015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.351+0000 7f56468b4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5640159c60 0x7f564015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.351+0000 7f56468b4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f564015a160 con 0x7f564015db10 2023-10-10T20:40:19.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.351+0000 7f56468b4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f564015df50 con 0x7f5640159c60 2023-10-10T20:40:19.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.351+0000 7f56468b4700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56400d13a0 con 0x7f5640159890 2023-10-10T20:40:19.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.352+0000 7f563f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5640159c60 0x7f564015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.352+0000 7f5644e51700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f564015db10 0x7f5640069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36396/0 (socket says 172.21.15.89:36396) 2023-10-10T20:40:19.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.352+0000 7f5644e51700 1 -- 172.21.15.89:0/1614330937 learned_addr learned my addr 172.21.15.89:0/1614330937 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:19.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.352+0000 7f563ffff700 1 --2- 172.21.15.89:0/1614330937 >> v2:172.21.15.111:3300/0 conn(0x7f5640159890 0x7f564015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.352+0000 7f563f7fe700 1 -- 172.21.15.89:0/1614330937 >> v2:172.21.15.111:3300/0 conn(0x7f5640159890 msgr2=0x7f564015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:19.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.352+0000 7f563f7fe700 1 --2- 172.21.15.89:0/1614330937 >> v2:172.21.15.111:3300/0 conn(0x7f5640159890 0x7f564015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.352+0000 7f563f7fe700 1 -- 172.21.15.89:0/1614330937 >> v1:172.21.15.89:6789/0 conn(0x7f564015db10 legacy=0x7f5640069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:19.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.352+0000 7f563effd700 1 -- 172.21.15.89:0/1614330937 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342807243 0 0) 0x7f564015a160 con 0x7f564015db10 2023-10-10T20:40:19.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.352+0000 7f563f7fe700 1 -- 172.21.15.89:0/1614330937 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564015e140 con 0x7f5640159c60 2023-10-10T20:40:19.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.352+0000 7f563ffff700 1 --2- 172.21.15.89:0/1614330937 >> v2:172.21.15.111:3300/0 conn(0x7f5640159890 0x7f564015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:19.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.352+0000 7f563f7fe700 1 --2- 172.21.15.89:0/1614330937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5640159c60 0x7f564015d5d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f56400d1090 tx=0x7f56340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e0a02af0e505f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:19.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.352+0000 7f563effd700 1 -- 172.21.15.89:0/1614330937 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563400b6b0 con 0x7f5640159c60 2023-10-10T20:40:19.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.352+0000 7f563effd700 1 -- 172.21.15.89:0/1614330937 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563400b810 con 0x7f5640159c60 2023-10-10T20:40:19.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.352+0000 7f563effd700 1 -- 172.21.15.89:0/1614330937 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563400bbb0 con 0x7f5640159c60 2023-10-10T20:40:19.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.353+0000 7f56468b4700 1 -- 172.21.15.89:0/1614330937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5640159c60 msgr2=0x7f564015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:19.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.353+0000 7f56468b4700 1 --2- 172.21.15.89:0/1614330937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5640159c60 0x7f564015d5d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f56400d1090 tx=0x7f56340091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.353+0000 7f56468b4700 1 -- 172.21.15.89:0/1614330937 shutdown_connections 2023-10-10T20:40:19.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.353+0000 7f56468b4700 1 --2- 172.21.15.89:0/1614330937 >> v2:172.21.15.111:3300/0 conn(0x7f5640159890 0x7f564015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.353+0000 7f56468b4700 1 --2- 172.21.15.89:0/1614330937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5640159c60 0x7f564015d5d0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.353+0000 7f56468b4700 1 -- 172.21.15.89:0/1614330937 >> 172.21.15.89:0/1614330937 conn(0x7f5640068e20 msgr2=0x7f56400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:19.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.353+0000 7f56468b4700 1 -- 172.21.15.89:0/1614330937 shutdown_connections 2023-10-10T20:40:19.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.353+0000 7f56468b4700 1 -- 172.21.15.89:0/1614330937 wait complete. 2023-10-10T20:40:19.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.354+0000 7f56468b4700 1 Processor -- start 2023-10-10T20:40:19.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.354+0000 7f56468b4700 1 -- start start 2023-10-10T20:40:19.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f56468b4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5640159890 0x7f5640144350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f56468b4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5640159c60 0x7f56400c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f56468b4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f564015db10 0x7f56400b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f56468b4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56400cff70 con 0x7f564015db10 2023-10-10T20:40:19.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f56468b4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56400c7de0 con 0x7f5640159c60 2023-10-10T20:40:19.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f56468b4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56400c8a60 con 0x7f5640159890 2023-10-10T20:40:19.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f5644e51700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f564015db10 0x7f56400b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f5644e51700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f564015db10 0x7f56400b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56464/0 (socket says 172.21.15.89:56464) 2023-10-10T20:40:19.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f5644e51700 1 -- 172.21.15.89:0/2679335803 learned_addr learned my addr 172.21.15.89:0/2679335803 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:19.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f563ffff700 1 --2- 172.21.15.89:0/2679335803 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5640159890 0x7f5640144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f563f7fe700 1 --2- 172.21.15.89:0/2679335803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5640159c60 0x7f56400c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f5644e51700 1 -- 172.21.15.89:0/2679335803 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5640159890 msgr2=0x7f5640144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:19.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f5644e51700 1 --2- 172.21.15.89:0/2679335803 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5640159890 0x7f5640144350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f5644e51700 1 -- 172.21.15.89:0/2679335803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5640159c60 msgr2=0x7f56400c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:19.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f5644e51700 1 --2- 172.21.15.89:0/2679335803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5640159c60 0x7f56400c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f5644e51700 1 -- 172.21.15.89:0/2679335803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5634009420 con 0x7f564015db10 2023-10-10T20:40:19.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f5644e51700 1 --2- 172.21.15.89:0/2679335803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f564015db10 0x7f56400b05c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f563800cc10 tx=0x7f5638001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:19.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f5627fff700 1 -- 172.21.15.89:0/2679335803 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563800a930 con 0x7f564015db10 2023-10-10T20:40:19.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f5627fff700 1 -- 172.21.15.89:0/2679335803 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563800aab0 con 0x7f564015db10 2023-10-10T20:40:19.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f56468b4700 1 -- 172.21.15.89:0/2679335803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56400cfc90 con 0x7f564015db10 2023-10-10T20:40:19.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f5627fff700 1 -- 172.21.15.89:0/2679335803 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563800ae70 con 0x7f564015db10 2023-10-10T20:40:19.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.355+0000 7f56468b4700 1 -- 172.21.15.89:0/2679335803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56400ca520 con 0x7f564015db10 2023-10-10T20:40:19.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.357+0000 7f5627fff700 1 -- 172.21.15.89:0/2679335803 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f5638007300 con 0x7f564015db10 2023-10-10T20:40:19.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.357+0000 7f5627fff700 1 --2- 172.21.15.89:0/2679335803 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f562c067200 0x7f562c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.357+0000 7f5627fff700 1 -- 172.21.15.89:0/2679335803 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5638084e10 con 0x7f564015db10 2023-10-10T20:40:19.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.357+0000 7f563ffff700 1 --2- 172.21.15.89:0/2679335803 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f562c067200 0x7f562c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.357+0000 7f563ffff700 1 --2- 172.21.15.89:0/2679335803 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f562c067200 0x7f562c0696b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f563000ac00 tx=0x7f5630006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:19.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.359+0000 7f564fbadb80 1 Processor -- start 2023-10-10T20:40:19.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.359+0000 7f564fbadb80 1 -- start start 2023-10-10T20:40:19.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f564fbadb80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35c8fa50 0x559d35c8fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f564fbadb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cd9d00 0x559d35cb8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f564fbadb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35cb8f60 0x559d35c8bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f564fbadb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d35be6a70 con 0x559d35cb8f60 2023-10-10T20:40:19.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f564fbadb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d35be4270 con 0x559d35cd9d00 2023-10-10T20:40:19.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f564fbadb80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d35be0060 con 0x559d35c8fa50 2023-10-10T20:40:19.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f5644e51700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35cb8f60 0x559d35c8bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f5644e51700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35cb8f60 0x559d35c8bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56476/0 (socket says 172.21.15.89:56476) 2023-10-10T20:40:19.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f5644e51700 1 -- 172.21.15.89:0/2442213719 learned_addr learned my addr 172.21.15.89:0/2442213719 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:19.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f563ffff700 1 --2- 172.21.15.89:0/2442213719 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35c8fa50 0x559d35c8fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f563f7fe700 1 --2- 172.21.15.89:0/2442213719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cd9d00 0x559d35cb8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f5644e51700 1 -- 172.21.15.89:0/2442213719 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35c8fa50 msgr2=0x559d35c8fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:19.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f5644e51700 1 --2- 172.21.15.89:0/2442213719 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35c8fa50 0x559d35c8fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f5644e51700 1 -- 172.21.15.89:0/2442213719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cd9d00 msgr2=0x559d35cb8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:19.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f5644e51700 1 --2- 172.21.15.89:0/2442213719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cd9d00 0x559d35cb8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f5644e51700 1 -- 172.21.15.89:0/2442213719 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d35a7a270 con 0x559d35cb8f60 2023-10-10T20:40:19.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f563f7fe700 1 --2- 172.21.15.89:0/2442213719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cd9d00 0x559d35cb8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:19.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f5644e51700 1 --2- 172.21.15.89:0/2442213719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35cb8f60 0x559d35c8bcf0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5638082f10 tx=0x7f5638055840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afbc5b602f5a2b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:19.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f56257fa700 1 -- 172.21.15.89:0/2442213719 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638058c30 con 0x559d35cb8f60 2023-10-10T20:40:19.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.360+0000 7f56257fa700 1 -- 172.21.15.89:0/2442213719 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563805c980 con 0x559d35cb8f60 2023-10-10T20:40:19.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.361+0000 7f56257fa700 1 -- 172.21.15.89:0/2442213719 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638055c40 con 0x559d35cb8f60 2023-10-10T20:40:19.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.361+0000 7f564fbadb80 1 -- 172.21.15.89:0/2442213719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35cb8f60 msgr2=0x559d35c8bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:19.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.361+0000 7f564fbadb80 1 --2- 172.21.15.89:0/2442213719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35cb8f60 0x559d35c8bcf0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5638082f10 tx=0x7f5638055840 comp rx=0 tx=0).stop 2023-10-10T20:40:19.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.361+0000 7f564fbadb80 1 -- 172.21.15.89:0/2442213719 shutdown_connections 2023-10-10T20:40:19.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.361+0000 7f564fbadb80 1 --2- 172.21.15.89:0/2442213719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35cb8f60 0x559d35c8bcf0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.361+0000 7f564fbadb80 1 --2- 172.21.15.89:0/2442213719 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35c8fa50 0x559d35c8fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.361+0000 7f564fbadb80 1 --2- 172.21.15.89:0/2442213719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cd9d00 0x559d35cb8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.361+0000 7f564fbadb80 1 -- 172.21.15.89:0/2442213719 >> 172.21.15.89:0/2442213719 conn(0x559d35c91370 msgr2=0x559d35c91740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:19.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.361+0000 7f564fbadb80 1 -- 172.21.15.89:0/2442213719 shutdown_connections 2023-10-10T20:40:19.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.361+0000 7f564fbadb80 1 -- 172.21.15.89:0/2442213719 wait complete. 2023-10-10T20:40:19.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.362+0000 7f564fbadb80 1 -- start start 2023-10-10T20:40:19.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.362+0000 7f564fbadb80 1 Processor -- start 2023-10-10T20:40:19.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.362+0000 7f564fbadb80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35c8fa50 0x559d35ce6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f564fbadb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cb8f60 0x559d35c8c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f564fbadb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35cd9d00 0x559d35c8cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f564fbadb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d35b8b890 con 0x559d35cd9d00 2023-10-10T20:40:19.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f564fbadb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d35ba6ea0 con 0x559d35cb8f60 2023-10-10T20:40:19.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f564fbadb80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d35a62e30 con 0x559d35c8fa50 2023-10-10T20:40:19.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f564fbadb80 10 client.0 _ll_register_callbacks cb 0x559d35c53110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:40:19.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f5644e51700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35cd9d00 0x559d35c8cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f5644e51700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35cd9d00 0x559d35c8cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56490/0 (socket says 172.21.15.89:56490) 2023-10-10T20:40:19.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f5644e51700 1 -- 172.21.15.89:0/1508535966 learned_addr learned my addr 172.21.15.89:0/1508535966 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:19.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f563ffff700 1 --2- 172.21.15.89:0/1508535966 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35c8fa50 0x559d35ce6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f563f7fe700 1 --2- 172.21.15.89:0/1508535966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cb8f60 0x559d35c8c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f5644e51700 1 -- 172.21.15.89:0/1508535966 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35c8fa50 msgr2=0x559d35ce6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:19.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f5644e51700 1 --2- 172.21.15.89:0/1508535966 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35c8fa50 0x559d35ce6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f5644e51700 1 -- 172.21.15.89:0/1508535966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cb8f60 msgr2=0x559d35c8c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:19.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f5644e51700 1 --2- 172.21.15.89:0/1508535966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cb8f60 0x559d35c8c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f5644e51700 1 -- 172.21.15.89:0/1508535966 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56400ca520 con 0x559d35cd9d00 2023-10-10T20:40:19.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f5644e51700 1 --2- 172.21.15.89:0/1508535966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35cd9d00 0x559d35c8cc50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5638051ee0 tx=0x7f5638082000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0b0da93732d1b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:19.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f561e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:40:19.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f561e7fc700 1 -- 172.21.15.89:0/1508535966 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563805cbd0 con 0x559d35cd9d00 2023-10-10T20:40:19.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f561e7fc700 1 -- 172.21.15.89:0/1508535966 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f563805cd30 con 0x559d35cd9d00 2023-10-10T20:40:19.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f564fbadb80 1 -- 172.21.15.89:0/1508535966 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d35c84b70 con 0x559d35cd9d00 2023-10-10T20:40:19.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f561e7fc700 1 -- 172.21.15.89:0/1508535966 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f563805c700 con 0x559d35cd9d00 2023-10-10T20:40:19.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f564fbadb80 10 client.6621 fetch_fsmap learned FSMap version 55 2023-10-10T20:40:19.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.363+0000 7f564fbadb80 1 -- 172.21.15.89:0/1508535966 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x559d35a1cfe0 con 0x559d35cd9d00 2023-10-10T20:40:19.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.364+0000 7f561e7fc700 1 -- 172.21.15.89:0/1508535966 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559d35c84b70 con 0x559d35cd9d00 2023-10-10T20:40:19.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.364+0000 7f561e7fc700 1 -- 172.21.15.89:0/1508535966 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 55) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f56380517d0 con 0x559d35cd9d00 2023-10-10T20:40:19.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.364+0000 7f564fbadb80 10 client.6621 fetch_fsmap finished waiting for FSMap version 55 2023-10-10T20:40:19.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.364+0000 7f564fbadb80 10 client.6621 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-10T20:40:19.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.364+0000 7f564fbadb80 20 client.6621 populate_metadata read hostname 'smithi089' 2023-10-10T20:40:19.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.364+0000 7f564fbadb80 1 --2- 172.21.15.89:0/1508535966 >> [v2:172.21.15.111:6833/4069200228,v1:172.21.15.111:6835/4069200228] conn(0x559d35cd83b0 0x559d35cd8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.364+0000 7f564fbadb80 4 client.6621 mds_command: new command op to 6462 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:40:19.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.364+0000 7f564fbadb80 1 -- 172.21.15.89:0/1508535966 --> [v2:172.21.15.111:6833/4069200228,v1:172.21.15.111:6835/4069200228] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d35c61aa0 con 0x559d35cd83b0 2023-10-10T20:40:19.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.364+0000 7f563ffff700 1 --2- 172.21.15.89:0/1508535966 >> [v2:172.21.15.111:6833/4069200228,v1:172.21.15.111:6835/4069200228] conn(0x559d35cd83b0 0x559d35cd8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.364+0000 7f561e7fc700 10 client.6621 ms_handle_connect on v2:172.21.15.111:6833/4069200228 2023-10-10T20:40:19.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.364+0000 7f563ffff700 1 --2- 172.21.15.89:0/1508535966 >> [v2:172.21.15.111:6833/4069200228,v1:172.21.15.111:6835/4069200228] conn(0x559d35cd83b0 0x559d35cd8780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2e09d5405733123 server_cookie=e0e9d10ca9938bf4 in_seq=0 out_seq=0 2023-10-10T20:40:19.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.365+0000 7f561e7fc700 1 -- 172.21.15.89:0/1508535966 <== mds.0 v2:172.21.15.111:6833/4069200228 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x559d35c61aa0 con 0x559d35cd83b0 2023-10-10T20:40:19.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.365+0000 7f561e7fc700 10 client.6621 handle_command_reply: tid=0 2023-10-10T20:40:19.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.365+0000 7f564fbadb80 1 client.6621 shutdown 2023-10-10T20:40:19.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.365+0000 7f564fbadb80 1 -- 172.21.15.89:0/1508535966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35cd9d00 msgr2=0x559d35c8cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:19.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.365+0000 7f564fbadb80 1 --2- 172.21.15.89:0/1508535966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35cd9d00 0x559d35c8cc50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5638051ee0 tx=0x7f5638082000 comp rx=0 tx=0).stop 2023-10-10T20:40:19.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.365+0000 7f564fbadb80 1 -- 172.21.15.89:0/1508535966 shutdown_connections 2023-10-10T20:40:19.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.365+0000 7f564fbadb80 1 --2- 172.21.15.89:0/1508535966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35cd9d00 0x559d35c8cc50 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.365+0000 7f564fbadb80 1 --2- 172.21.15.89:0/1508535966 >> [v2:172.21.15.111:6833/4069200228,v1:172.21.15.111:6835/4069200228] conn(0x559d35cd83b0 0x559d35cd8780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.365+0000 7f564fbadb80 1 --2- 172.21.15.89:0/1508535966 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35c8fa50 0x559d35ce6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.365+0000 7f561e7fc700 0 client.6621 ms_handle_reset on v2:172.21.15.111:6833/4069200228 2023-10-10T20:40:19.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.365+0000 7f564fbadb80 1 --2- 172.21.15.89:0/1508535966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cb8f60 0x559d35c8c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.365+0000 7f564fbadb80 1 -- 172.21.15.89:0/1508535966 >> 172.21.15.89:0/1508535966 conn(0x559d35c91370 msgr2=0x559d35c8bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:19.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.365+0000 7f564fbadb80 1 -- 172.21.15.89:0/1508535966 shutdown_connections 2023-10-10T20:40:19.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.365+0000 7f564fbadb80 1 -- 172.21.15.89:0/1508535966 wait complete. 2023-10-10T20:40:19.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.365+0000 7f564fbadb80 20 client.6621 trim_cache size 0 max 16384 2023-10-10T20:40:19.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.382+0000 7f564fbadb80 1 Processor -- start 2023-10-10T20:40:19.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.382+0000 7f564fbadb80 1 -- start start 2023-10-10T20:40:19.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.383+0000 7f564fbadb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35c8fa50 0x559d35c8cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.383+0000 7f564fbadb80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35cb8f60 0x559d35c8b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.383+0000 7f564fbadb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cd9d00 0x559d35cd02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.383+0000 7f564fbadb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d35c8d5e0 con 0x559d35c8fa50 2023-10-10T20:40:19.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.383+0000 7f564fbadb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d35c8a5a0 con 0x559d35cd9d00 2023-10-10T20:40:19.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.383+0000 7f564fbadb80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d35cb81a0 con 0x559d35cb8f60 2023-10-10T20:40:19.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.383+0000 7f563ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35c8fa50 0x559d35c8cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.383+0000 7f563ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35c8fa50 0x559d35c8cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56504/0 (socket says 172.21.15.89:56504) 2023-10-10T20:40:19.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.383+0000 7f563ffff700 1 -- 172.21.15.89:0/736562063 learned_addr learned my addr 172.21.15.89:0/736562063 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:19.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.383+0000 7f563f7fe700 1 --2- 172.21.15.89:0/736562063 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35cb8f60 0x559d35c8b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.383+0000 7f563ffff700 1 -- 172.21.15.89:0/736562063 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35cb8f60 msgr2=0x559d35c8b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:19.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.383+0000 7f5644e51700 1 --2- 172.21.15.89:0/736562063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cd9d00 0x559d35cd02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.383+0000 7f563ffff700 1 --2- 172.21.15.89:0/736562063 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35cb8f60 0x559d35c8b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.383+0000 7f563ffff700 1 -- 172.21.15.89:0/736562063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cd9d00 msgr2=0x559d35cd02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:19.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.383+0000 7f563ffff700 1 --2- 172.21.15.89:0/736562063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cd9d00 0x559d35cd02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.383+0000 7f563ffff700 1 -- 172.21.15.89:0/736562063 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d35bae780 con 0x559d35c8fa50 2023-10-10T20:40:19.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.383+0000 7f563f7fe700 1 --2- 172.21.15.89:0/736562063 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35cb8f60 0x559d35c8b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:19.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.383+0000 7f5644e51700 1 --2- 172.21.15.89:0/736562063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cd9d00 0x559d35cd02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:19.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.383+0000 7f563ffff700 1 --2- 172.21.15.89:0/736562063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35c8fa50 0x559d35c8cc40 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f563000a5c0 tx=0x7f5630025040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbea4399b2522676 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:19.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.384+0000 7f5625ffb700 1 -- 172.21.15.89:0/736562063 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630028040 con 0x559d35c8fa50 2023-10-10T20:40:19.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.384+0000 7f5625ffb700 1 -- 172.21.15.89:0/736562063 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5630014c80 con 0x559d35c8fa50 2023-10-10T20:40:19.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.384+0000 7f5625ffb700 1 -- 172.21.15.89:0/736562063 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630014df0 con 0x559d35c8fa50 2023-10-10T20:40:19.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.384+0000 7f564fbadb80 1 -- 172.21.15.89:0/736562063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35c8fa50 msgr2=0x559d35c8cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:19.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.384+0000 7f564fbadb80 1 --2- 172.21.15.89:0/736562063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35c8fa50 0x559d35c8cc40 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f563000a5c0 tx=0x7f5630025040 comp rx=0 tx=0).stop 2023-10-10T20:40:19.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.384+0000 7f564fbadb80 1 -- 172.21.15.89:0/736562063 shutdown_connections 2023-10-10T20:40:19.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.384+0000 7f564fbadb80 1 --2- 172.21.15.89:0/736562063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cd9d00 0x559d35cd02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.384+0000 7f564fbadb80 1 --2- 172.21.15.89:0/736562063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35c8fa50 0x559d35c8cc40 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.384+0000 7f564fbadb80 1 --2- 172.21.15.89:0/736562063 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35cb8f60 0x559d35c8b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.384+0000 7f564fbadb80 1 -- 172.21.15.89:0/736562063 >> 172.21.15.89:0/736562063 conn(0x559d35c91370 msgr2=0x559d35cd79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:19.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.384+0000 7f564fbadb80 1 -- 172.21.15.89:0/736562063 shutdown_connections 2023-10-10T20:40:19.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.384+0000 7f564fbadb80 1 -- 172.21.15.89:0/736562063 wait complete. 2023-10-10T20:40:19.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.385+0000 7f564fbadb80 1 -- start start 2023-10-10T20:40:19.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.385+0000 7f564fbadb80 1 Processor -- start 2023-10-10T20:40:19.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f564fbadb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35c8fa50 0x559d35cb4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f564fbadb80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35cb8f60 0x559d35ccd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f564fbadb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cd9d00 0x559d35cf5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f564fbadb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d35cf6830 con 0x559d35c8fa50 2023-10-10T20:40:19.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f564fbadb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d35cf8520 con 0x559d35cd9d00 2023-10-10T20:40:19.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f564fbadb80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d35cef9f0 con 0x559d35cb8f60 2023-10-10T20:40:19.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f564fbadb80 10 client.0 _ll_register_callbacks cb 0x559d35c53110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:40:19.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f563ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35c8fa50 0x559d35cb4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f563ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35c8fa50 0x559d35cb4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56520/0 (socket says 172.21.15.89:56520) 2023-10-10T20:40:19.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f563ffff700 1 -- 172.21.15.89:0/2425165016 learned_addr learned my addr 172.21.15.89:0/2425165016 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:19.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f563f7fe700 1 --2- 172.21.15.89:0/2425165016 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35cb8f60 0x559d35ccd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f5644e51700 1 --2- 172.21.15.89:0/2425165016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cd9d00 0x559d35cf5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f563ffff700 1 -- 172.21.15.89:0/2425165016 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35cb8f60 msgr2=0x559d35ccd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:19.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f563ffff700 1 --2- 172.21.15.89:0/2425165016 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35cb8f60 0x559d35ccd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f563ffff700 1 -- 172.21.15.89:0/2425165016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cd9d00 msgr2=0x559d35cf5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:19.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f563ffff700 1 --2- 172.21.15.89:0/2425165016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cd9d00 0x559d35cf5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f563ffff700 1 -- 172.21.15.89:0/2425165016 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563000e040 con 0x559d35c8fa50 2023-10-10T20:40:19.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f563ffff700 1 --2- 172.21.15.89:0/2425165016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35c8fa50 0x559d35cb4040 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f5630008010 tx=0x7f5630029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb749f7ebd713e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:19.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f561e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:40:19.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f561e7fc700 1 -- 172.21.15.89:0/2425165016 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630028050 con 0x559d35c8fa50 2023-10-10T20:40:19.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f561e7fc700 1 -- 172.21.15.89:0/2425165016 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f563002ca60 con 0x559d35c8fa50 2023-10-10T20:40:19.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f564fbadb80 1 -- 172.21.15.89:0/2425165016 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f563805c980 con 0x559d35c8fa50 2023-10-10T20:40:19.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f561e7fc700 1 -- 172.21.15.89:0/2425165016 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f563002ce30 con 0x559d35c8fa50 2023-10-10T20:40:19.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f564fbadb80 10 client.6627 fetch_fsmap learned FSMap version 55 2023-10-10T20:40:19.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f564fbadb80 1 -- 172.21.15.89:0/2425165016 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x559d35b95470 con 0x559d35c8fa50 2023-10-10T20:40:19.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.386+0000 7f561e7fc700 1 -- 172.21.15.89:0/2425165016 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563805c980 con 0x559d35c8fa50 2023-10-10T20:40:19.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.387+0000 7f561e7fc700 1 -- 172.21.15.89:0/2425165016 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 55) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5630033030 con 0x559d35c8fa50 2023-10-10T20:40:19.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.387+0000 7f564fbadb80 10 client.6627 fetch_fsmap finished waiting for FSMap version 55 2023-10-10T20:40:19.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.387+0000 7f564fbadb80 10 client.6627 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-10T20:40:19.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.387+0000 7f564fbadb80 20 client.6627 populate_metadata read hostname 'smithi089' 2023-10-10T20:40:19.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.387+0000 7f564fbadb80 1 --2- 172.21.15.89:0/2425165016 >> [v2:172.21.15.111:6833/4069200228,v1:172.21.15.111:6835/4069200228] conn(0x559d35d3b450 0x559d35d3b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.387+0000 7f564fbadb80 4 client.6627 mds_command: new command op to 6462 tid=0[{"prefix": "status"}] 2023-10-10T20:40:19.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.387+0000 7f564fbadb80 1 -- 172.21.15.89:0/2425165016 --> [v2:172.21.15.111:6833/4069200228,v1:172.21.15.111:6835/4069200228] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559d35b82e00 con 0x559d35d3b450 2023-10-10T20:40:19.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.387+0000 7f563f7fe700 1 --2- 172.21.15.89:0/2425165016 >> [v2:172.21.15.111:6833/4069200228,v1:172.21.15.111:6835/4069200228] conn(0x559d35d3b450 0x559d35d3b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.387+0000 7f561e7fc700 10 client.6627 ms_handle_connect on v2:172.21.15.111:6833/4069200228 2023-10-10T20:40:19.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.387+0000 7f563f7fe700 1 --2- 172.21.15.89:0/2425165016 >> [v2:172.21.15.111:6833/4069200228,v1:172.21.15.111:6835/4069200228] conn(0x559d35d3b450 0x559d35d3b820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac468497623e9832 server_cookie=df759c2446e945c1 in_seq=0 out_seq=0 2023-10-10T20:40:19.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.387+0000 7f561e7fc700 1 -- 172.21.15.89:0/2425165016 <== mds.0 v2:172.21.15.111:6833/4069200228 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x559d35b82e00 con 0x559d35d3b450 2023-10-10T20:40:19.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.387+0000 7f561e7fc700 10 client.6627 handle_command_reply: tid=0 2023-10-10T20:40:19.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.387+0000 7f564fbadb80 1 client.6627 shutdown 2023-10-10T20:40:19.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.387+0000 7f564fbadb80 1 -- 172.21.15.89:0/2425165016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35c8fa50 msgr2=0x559d35cb4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:19.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.387+0000 7f564fbadb80 1 --2- 172.21.15.89:0/2425165016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35c8fa50 0x559d35cb4040 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f5630008010 tx=0x7f5630029000 comp rx=0 tx=0).stop 2023-10-10T20:40:19.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f564fbadb80 1 -- 172.21.15.89:0/2425165016 shutdown_connections 2023-10-10T20:40:19.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f564fbadb80 1 --2- 172.21.15.89:0/2425165016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d35cd9d00 0x559d35cf5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f564fbadb80 1 --2- 172.21.15.89:0/2425165016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d35c8fa50 0x559d35cb4040 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f564fbadb80 1 --2- 172.21.15.89:0/2425165016 >> [v2:172.21.15.111:6833/4069200228,v1:172.21.15.111:6835/4069200228] conn(0x559d35d3b450 0x559d35d3b820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f564fbadb80 1 --2- 172.21.15.89:0/2425165016 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x559d35cb8f60 0x559d35ccd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f561e7fc700 0 client.6627 ms_handle_reset on v2:172.21.15.111:6833/4069200228 2023-10-10T20:40:19.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f564fbadb80 1 -- 172.21.15.89:0/2425165016 >> 172.21.15.89:0/2425165016 conn(0x559d35c91370 msgr2=0x559d35cd8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:19.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f564fbadb80 1 -- 172.21.15.89:0/2425165016 shutdown_connections 2023-10-10T20:40:19.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f564fbadb80 1 -- 172.21.15.89:0/2425165016 wait complete. 2023-10-10T20:40:19.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f564fbadb80 20 client.6627 trim_cache size 0 max 16384 2023-10-10T20:40:19.414 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:40:19.415 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "5dd675b8-9026-4d8a-a1bf-62fdc5e7de67", 2023-10-10T20:40:19.415 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-10-10T20:40:19.415 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 6462, 2023-10-10T20:40:19.415 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-10-10T20:40:19.415 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-10-10T20:40:19.416 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-10-10T20:40:19.416 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 10.365718675, 2023-10-10T20:40:19.416 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 55, 2023-10-10T20:40:19.416 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 88, 2023-10-10T20:40:19.416 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 88, 2023-10-10T20:40:19.417 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 15.462341979 2023-10-10T20:40:19.417 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:40:19.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f56468b4700 1 -- 172.21.15.89:0/2679335803 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f562c067200 msgr2=0x7f562c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f56468b4700 1 --2- 172.21.15.89:0/2679335803 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f562c067200 0x7f562c0696b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f563000ac00 tx=0x7f5630006f90 comp rx=0 tx=0).stop 2023-10-10T20:40:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f56468b4700 1 -- 172.21.15.89:0/2679335803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f564015db10 msgr2=0x7f56400b05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f56468b4700 1 --2- 172.21.15.89:0/2679335803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f564015db10 0x7f56400b05c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f563800cc10 tx=0x7f5638001970 comp rx=0 tx=0).stop 2023-10-10T20:40:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f56468b4700 1 -- 172.21.15.89:0/2679335803 shutdown_connections 2023-10-10T20:40:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f56468b4700 1 --2- 172.21.15.89:0/2679335803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f564015db10 0x7f56400b05c0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f56468b4700 1 --2- 172.21.15.89:0/2679335803 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5640159890 0x7f5640144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f56468b4700 1 --2- 172.21.15.89:0/2679335803 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f562c067200 0x7f562c0696b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f56468b4700 1 --2- 172.21.15.89:0/2679335803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5640159c60 0x7f56400c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f56468b4700 1 -- 172.21.15.89:0/2679335803 >> 172.21.15.89:0/2679335803 conn(0x7f5640068e20 msgr2=0x7f5640153170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:19.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f56468b4700 1 -- 172.21.15.89:0/2679335803 shutdown_connections 2023-10-10T20:40:19.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.388+0000 7f56468b4700 1 -- 172.21.15.89:0/2679335803 wait complete. 2023-10-10T20:40:19.421 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:40:19.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.534+0000 7f14280b3700 1 Processor -- start 2023-10-10T20:40:19.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.534+0000 7f14280b3700 1 -- start start 2023-10-10T20:40:19.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.535+0000 7f14280b3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1420156690 0x7f14200ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.535+0000 7f14280b3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1420156a60 0x7f14200694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.535+0000 7f14280b3700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1420157e30 con 0x7f14200af480 2023-10-10T20:40:19.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.535+0000 7f14280b3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f142006aa00 con 0x7f1420156a60 2023-10-10T20:40:19.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.535+0000 7f14280b3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14200699e0 con 0x7f1420156690 2023-10-10T20:40:19.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.535+0000 7f1426650700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1420156a60 0x7f14200694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.535+0000 7f1426650700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1420156a60 0x7f14200694a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56526/0 (socket says 172.21.15.89:56526) 2023-10-10T20:40:19.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.535+0000 7f1426650700 1 -- 172.21.15.89:0/2942408788 learned_addr learned my addr 172.21.15.89:0/2942408788 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:19.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.535+0000 7f1425e4f700 1 --2- 172.21.15.89:0/2942408788 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1420156690 0x7f14200ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.535+0000 7f1426650700 1 -- 172.21.15.89:0/2942408788 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1420156690 msgr2=0x7f14200ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:19.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.535+0000 7f1426650700 1 --2- 172.21.15.89:0/2942408788 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1420156690 0x7f14200ca9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.535+0000 7f1426650700 1 -- 172.21.15.89:0/2942408788 >> v1:172.21.15.111:6789/0 conn(0x7f14200af480 legacy=0x7f14200c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:19.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.535+0000 7f1426650700 1 -- 172.21.15.89:0/2942408788 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14201578b0 con 0x7f1420156a60 2023-10-10T20:40:19.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.535+0000 7f1426650700 1 --2- 172.21.15.89:0/2942408788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1420156a60 0x7f14200694a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f141800d1a0 tx=0x7f141800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0c51b51d61fd0a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:19.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.536+0000 7f1424e4d700 1 -- 172.21.15.89:0/2942408788 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1418010070 con 0x7f1420156a60 2023-10-10T20:40:19.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.536+0000 7f1424e4d700 1 -- 172.21.15.89:0/2942408788 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141800de80 con 0x7f1420156a60 2023-10-10T20:40:19.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.536+0000 7f1424e4d700 1 -- 172.21.15.89:0/2942408788 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141800b320 con 0x7f1420156a60 2023-10-10T20:40:19.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.536+0000 7f14280b3700 1 -- 172.21.15.89:0/2942408788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1420156a60 msgr2=0x7f14200694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:19.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.536+0000 7f14280b3700 1 --2- 172.21.15.89:0/2942408788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1420156a60 0x7f14200694a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f141800d1a0 tx=0x7f141800ba50 comp rx=0 tx=0).stop 2023-10-10T20:40:19.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.536+0000 7f14280b3700 1 -- 172.21.15.89:0/2942408788 shutdown_connections 2023-10-10T20:40:19.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.536+0000 7f14280b3700 1 --2- 172.21.15.89:0/2942408788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1420156a60 0x7f14200694a0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.536+0000 7f14280b3700 1 --2- 172.21.15.89:0/2942408788 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1420156690 0x7f14200ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.536+0000 7f14280b3700 1 -- 172.21.15.89:0/2942408788 >> 172.21.15.89:0/2942408788 conn(0x7f14200c9870 msgr2=0x7f14200c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:19.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.536+0000 7f14280b3700 1 -- 172.21.15.89:0/2942408788 shutdown_connections 2023-10-10T20:40:19.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.536+0000 7f14280b3700 1 -- 172.21.15.89:0/2942408788 wait complete. 2023-10-10T20:40:19.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.537+0000 7f14280b3700 1 Processor -- start 2023-10-10T20:40:19.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.537+0000 7f14280b3700 1 -- start start 2023-10-10T20:40:19.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.538+0000 7f14280b3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14200af480 0x7f14201478d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.538+0000 7f14280b3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1420156690 0x7f1420168f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.538+0000 7f14280b3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1420156a60 0x7f14201554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.538+0000 7f14280b3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14200c4350 con 0x7f14200af480 2023-10-10T20:40:19.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.538+0000 7f14280b3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1420147e10 con 0x7f1420156a60 2023-10-10T20:40:19.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.538+0000 7f14280b3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1420169840 con 0x7f1420156690 2023-10-10T20:40:19.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.538+0000 7f1425e4f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14200af480 0x7f14201478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.538+0000 7f1425e4f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14200af480 0x7f14201478d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56536/0 (socket says 172.21.15.89:56536) 2023-10-10T20:40:19.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.538+0000 7f1425e4f700 1 -- 172.21.15.89:0/705535422 learned_addr learned my addr 172.21.15.89:0/705535422 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:19.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.538+0000 7f142564e700 1 --2- 172.21.15.89:0/705535422 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1420156690 0x7f1420168f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.538+0000 7f1426650700 1 --2- 172.21.15.89:0/705535422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1420156a60 0x7f14201554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.538+0000 7f1425e4f700 1 -- 172.21.15.89:0/705535422 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1420156690 msgr2=0x7f1420168f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:19.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.538+0000 7f1425e4f700 1 --2- 172.21.15.89:0/705535422 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1420156690 0x7f1420168f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.538+0000 7f1425e4f700 1 -- 172.21.15.89:0/705535422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1420156a60 msgr2=0x7f14201554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:19.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.538+0000 7f1425e4f700 1 --2- 172.21.15.89:0/705535422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1420156a60 0x7f14201554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.538+0000 7f1425e4f700 1 -- 172.21.15.89:0/705535422 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141800bcf0 con 0x7f14200af480 2023-10-10T20:40:19.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.539+0000 7f1425e4f700 1 --2- 172.21.15.89:0/705535422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14200af480 0x7f14201478d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f141c00c7d0 tx=0x7f141c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:19.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.539+0000 7f1415ffb700 1 -- 172.21.15.89:0/705535422 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c011600 con 0x7f14200af480 2023-10-10T20:40:19.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.539+0000 7f1415ffb700 1 -- 172.21.15.89:0/705535422 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141c011760 con 0x7f14200af480 2023-10-10T20:40:19.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.539+0000 7f14280b3700 1 -- 172.21.15.89:0/705535422 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1420169b20 con 0x7f14200af480 2023-10-10T20:40:19.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.539+0000 7f1415ffb700 1 -- 172.21.15.89:0/705535422 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c011b20 con 0x7f14200af480 2023-10-10T20:40:19.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.539+0000 7f14280b3700 1 -- 172.21.15.89:0/705535422 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1420169480 con 0x7f14200af480 2023-10-10T20:40:19.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.540+0000 7f14280b3700 1 -- 172.21.15.89:0/705535422 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f142004c3c0 con 0x7f14200af480 2023-10-10T20:40:19.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.540+0000 7f1415ffb700 1 -- 172.21.15.89:0/705535422 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f141c0193f0 con 0x7f14200af480 2023-10-10T20:40:19.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.540+0000 7f1415ffb700 1 --2- 172.21.15.89:0/705535422 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1404067200 0x7f14040696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:19.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.540+0000 7f1415ffb700 1 -- 172.21.15.89:0/705535422 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f141c084e90 con 0x7f14200af480 2023-10-10T20:40:19.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.540+0000 7f142564e700 1 --2- 172.21.15.89:0/705535422 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1404067200 0x7f14040696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:19.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.541+0000 7f142564e700 1 --2- 172.21.15.89:0/705535422 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1404067200 0x7f14040696b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1410006fd0 tx=0x7f1410008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:19.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.544+0000 7f1415ffb700 1 -- 172.21.15.89:0/705535422 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f141c055730 con 0x7f14200af480 2023-10-10T20:40:19.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.771+0000 7f14280b3700 1 -- 172.21.15.89:0/705535422 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f142006f270 con 0x7f14200af480 2023-10-10T20:40:19.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.771+0000 7f1415ffb700 1 -- 172.21.15.89:0/705535422 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) v1 ==== 53+0+73 (secure 0 0 0) 0x7f141c0552c0 con 0x7f14200af480 2023-10-10T20:40:19.773 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:40:19.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.773+0000 7f14280b3700 1 -- 172.21.15.89:0/705535422 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1404067200 msgr2=0x7f14040696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:19.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.773+0000 7f14280b3700 1 --2- 172.21.15.89:0/705535422 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1404067200 0x7f14040696b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1410006fd0 tx=0x7f1410008040 comp rx=0 tx=0).stop 2023-10-10T20:40:19.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.773+0000 7f14280b3700 1 -- 172.21.15.89:0/705535422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14200af480 msgr2=0x7f14201478d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:19.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.773+0000 7f14280b3700 1 --2- 172.21.15.89:0/705535422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14200af480 0x7f14201478d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f141c00c7d0 tx=0x7f141c008040 comp rx=0 tx=0).stop 2023-10-10T20:40:19.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.773+0000 7f14280b3700 1 -- 172.21.15.89:0/705535422 shutdown_connections 2023-10-10T20:40:19.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.773+0000 7f14280b3700 1 --2- 172.21.15.89:0/705535422 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1404067200 0x7f14040696b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.773+0000 7f14280b3700 1 --2- 172.21.15.89:0/705535422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1420156a60 0x7f14201554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.773+0000 7f14280b3700 1 --2- 172.21.15.89:0/705535422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14200af480 0x7f14201478d0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.773+0000 7f14280b3700 1 --2- 172.21.15.89:0/705535422 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1420156690 0x7f1420168f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:19.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.773+0000 7f14280b3700 1 -- 172.21.15.89:0/705535422 >> 172.21.15.89:0/705535422 conn(0x7f14200c9870 msgr2=0x7f1420153370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:19.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.773+0000 7f14280b3700 1 -- 172.21.15.89:0/705535422 shutdown_connections 2023-10-10T20:40:19.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:19.773+0000 7f14280b3700 1 -- 172.21.15.89:0/705535422 wait complete. 2023-10-10T20:40:19.787 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:40:19.787 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:40:19.787 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-10T20:40:19.788 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:40:19.788 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:40:19.788 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:40:19.789 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:40:19.789 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:40:19.789 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:40:19.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:19.790 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-10T20:40:19.805 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:40:19.806 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:40:19.806 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:40:19.806 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:19.806 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-10T20:40:19.806 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:19.807 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:40:19.807 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:19.807 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-10T20:40:19.807 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:40:19.807 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:19.808 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-10T20:40:19.808 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:40:19.808 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:19.808 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:40:19.808 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:19.809 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:40:19.809 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42146sec preferred_lft 42146sec 2023-10-10T20:40:19.809 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2023-10-10T20:40:19.809 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:19.809 INFO:teuthology.orchestra.run.smithi089.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:40:19.809 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:19.810 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:40:19.810 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:19.810 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::98a3:d4ff:fec1:83bb/64 scope link 2023-10-10T20:40:19.810 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:19.810 INFO:teuthology.orchestra.run.smithi089.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:40:19.811 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether ee:16:22:10:ca:9a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:40:19.811 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec16:22ff:fe10:ca9a/64 scope link 2023-10-10T20:40:19.811 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:19.811 INFO:teuthology.orchestra.run.smithi089.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:40:19.811 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:40:19.811 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::6864:2bff:fe5c:71d4/64 scope link 2023-10-10T20:40:19.812 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:19.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:19.812 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:40:19.868 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-10T20:40:19.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:19.870 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-10T20:40:19.926 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-10T20:40:19.927 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:40:19.927 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:40:19.927 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:40:19.927 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:40:19.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:19.928 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:40:19.929 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:40:19.929 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:40:19.929 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:40:19.930 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:40:19.930 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:40:20.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:20.027 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-10T20:40:20.042 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:40:20.042 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:40:20.042 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:40:20.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:20.043 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-10-10T20:40:20.106 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:40:20.107 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:40:20.161 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:40:20.162 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:40:20.163 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:40:20.216 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:40:20.217 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:40:20.217 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:40:20.296 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:40:20.297 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:40:20.297 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:40:20.305 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:40:20.305 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:40:20.305 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:40:20.305 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:40:20.306 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:40:20.327 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:40:20.327 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:40:20.328 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:40:20.328 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:40:20.328 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:40:20.328 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:40:20.328 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:40:20.329 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:40:20.330 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:40:20.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:20.330 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:40:20.396 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:40:20.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.548+0000 7f38ea4ba700 1 Processor -- start 2023-10-10T20:40:20.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.548+0000 7f38ea4ba700 1 -- start start 2023-10-10T20:40:20.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.549+0000 7f38ea4ba700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f38e406c0a0 0x7f38e40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:20.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.549+0000 7f38ea4ba700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f38e415df50 0x7f38e415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:20.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.549+0000 7f38ea4ba700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e415d7e0 con 0x7f38e406bcd0 2023-10-10T20:40:20.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.549+0000 7f38ea4ba700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e415dac0 con 0x7f38e406c0a0 2023-10-10T20:40:20.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.549+0000 7f38ea4ba700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e415aa30 con 0x7f38e415df50 2023-10-10T20:40:20.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.549+0000 7f38e37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f38e406c0a0 0x7f38e40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:20.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.549+0000 7f38e3fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f38e406bcd0 0x7f38e415a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36398/0 (socket says 172.21.15.89:36398) 2023-10-10T20:40:20.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.549+0000 7f38e3fff700 1 -- 172.21.15.89:0/888938443 learned_addr learned my addr 172.21.15.89:0/888938443 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:20.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.549+0000 7f38e8a57700 1 --2- 172.21.15.89:0/888938443 >> v2:172.21.15.111:3300/0 conn(0x7f38e415df50 0x7f38e415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:20.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.549+0000 7f38e2ffd700 1 -- 172.21.15.89:0/888938443 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830640209 0 0) 0x7f38e415d7e0 con 0x7f38e406bcd0 2023-10-10T20:40:20.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.549+0000 7f38e2ffd700 1 -- 172.21.15.89:0/888938443 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38cc003570 con 0x7f38e406bcd0 2023-10-10T20:40:20.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.549+0000 7f38e37fe700 1 -- 172.21.15.89:0/888938443 >> v2:172.21.15.111:3300/0 conn(0x7f38e415df50 msgr2=0x7f38e415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:20.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.549+0000 7f38e37fe700 1 --2- 172.21.15.89:0/888938443 >> v2:172.21.15.111:3300/0 conn(0x7f38e415df50 0x7f38e415e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:20.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.549+0000 7f38e37fe700 1 -- 172.21.15.89:0/888938443 >> v1:172.21.15.89:6789/0 conn(0x7f38e406bcd0 legacy=0x7f38e415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:20.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.549+0000 7f38e37fe700 1 -- 172.21.15.89:0/888938443 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e415ad10 con 0x7f38e406c0a0 2023-10-10T20:40:20.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.549+0000 7f38e37fe700 1 --2- 172.21.15.89:0/888938443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e406c0a0 0x7f38e40caa90 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f38e415d720 tx=0x7f38d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df3ec951a935481 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:20.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.549+0000 7f38e2ffd700 1 -- 172.21.15.89:0/888938443 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4011630 con 0x7f38e406c0a0 2023-10-10T20:40:20.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.549+0000 7f38e2ffd700 1 -- 172.21.15.89:0/888938443 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d4011790 con 0x7f38e406c0a0 2023-10-10T20:40:20.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.549+0000 7f38e2ffd700 1 -- 172.21.15.89:0/888938443 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4011b30 con 0x7f38e406c0a0 2023-10-10T20:40:20.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.550+0000 7f38ea4ba700 1 -- 172.21.15.89:0/888938443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e406c0a0 msgr2=0x7f38e40caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:20.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.550+0000 7f38ea4ba700 1 --2- 172.21.15.89:0/888938443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e406c0a0 0x7f38e40caa90 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f38e415d720 tx=0x7f38d40091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:20.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.550+0000 7f38ea4ba700 1 -- 172.21.15.89:0/888938443 shutdown_connections 2023-10-10T20:40:20.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.550+0000 7f38ea4ba700 1 --2- 172.21.15.89:0/888938443 >> v2:172.21.15.111:3300/0 conn(0x7f38e415df50 0x7f38e415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:20.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.550+0000 7f38ea4ba700 1 --2- 172.21.15.89:0/888938443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e406c0a0 0x7f38e40caa90 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:20.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.550+0000 7f38ea4ba700 1 -- 172.21.15.89:0/888938443 >> 172.21.15.89:0/888938443 conn(0x7f38e40ce600 msgr2=0x7f38e40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:20.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.550+0000 7f38ea4ba700 1 -- 172.21.15.89:0/888938443 shutdown_connections 2023-10-10T20:40:20.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.550+0000 7f38ea4ba700 1 -- 172.21.15.89:0/888938443 wait complete. 2023-10-10T20:40:20.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.551+0000 7f38ea4ba700 1 Processor -- start 2023-10-10T20:40:20.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.551+0000 7f38ea4ba700 1 -- start start 2023-10-10T20:40:20.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.552+0000 7f38ea4ba700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38e406bcd0 0x7f38e40aa120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:20.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.552+0000 7f38ea4ba700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e406c0a0 0x7f38e40c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:20.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.552+0000 7f38ea4ba700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e415df50 0x7f38e4169f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:20.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.552+0000 7f38ea4ba700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e40aa660 con 0x7f38e406c0a0 2023-10-10T20:40:20.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.552+0000 7f38ea4ba700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e40c5fb0 con 0x7f38e415df50 2023-10-10T20:40:20.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.552+0000 7f38ea4ba700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e40c6290 con 0x7f38e406bcd0 2023-10-10T20:40:20.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.552+0000 7f38e37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e406c0a0 0x7f38e40c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:20.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.552+0000 7f38e37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e406c0a0 0x7f38e40c56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56558/0 (socket says 172.21.15.89:56558) 2023-10-10T20:40:20.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.552+0000 7f38e37fe700 1 -- 172.21.15.89:0/4035037725 learned_addr learned my addr 172.21.15.89:0/4035037725 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:20.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.552+0000 7f38e8a57700 1 --2- 172.21.15.89:0/4035037725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e415df50 0x7f38e4169f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:20.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.552+0000 7f38e3fff700 1 --2- 172.21.15.89:0/4035037725 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38e406bcd0 0x7f38e40aa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:20.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.552+0000 7f38e37fe700 1 -- 172.21.15.89:0/4035037725 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38e406bcd0 msgr2=0x7f38e40aa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:20.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.552+0000 7f38e37fe700 1 --2- 172.21.15.89:0/4035037725 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38e406bcd0 0x7f38e40aa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:20.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.552+0000 7f38e37fe700 1 -- 172.21.15.89:0/4035037725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e415df50 msgr2=0x7f38e4169f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:20.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.552+0000 7f38e37fe700 1 --2- 172.21.15.89:0/4035037725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e415df50 0x7f38e4169f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:20.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.552+0000 7f38e37fe700 1 -- 172.21.15.89:0/4035037725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d4009420 con 0x7f38e406c0a0 2023-10-10T20:40:20.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.553+0000 7f38e37fe700 1 --2- 172.21.15.89:0/4035037725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e406c0a0 0x7f38e40c56b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f38e4069220 tx=0x7f38d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:20.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.553+0000 7f38cbfff700 1 -- 172.21.15.89:0/4035037725 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4011900 con 0x7f38e406c0a0 2023-10-10T20:40:20.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.553+0000 7f38cbfff700 1 -- 172.21.15.89:0/4035037725 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d4011a80 con 0x7f38e406c0a0 2023-10-10T20:40:20.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.553+0000 7f38ea4ba700 1 -- 172.21.15.89:0/4035037725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e40c6570 con 0x7f38e406c0a0 2023-10-10T20:40:20.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.553+0000 7f38cbfff700 1 -- 172.21.15.89:0/4035037725 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4025640 con 0x7f38e406c0a0 2023-10-10T20:40:20.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.553+0000 7f38ea4ba700 1 -- 172.21.15.89:0/4035037725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e40c6b30 con 0x7f38e406c0a0 2023-10-10T20:40:20.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.554+0000 7f38ea4ba700 1 -- 172.21.15.89:0/4035037725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e40c6d50 con 0x7f38e406c0a0 2023-10-10T20:40:20.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.554+0000 7f38cbfff700 1 -- 172.21.15.89:0/4035037725 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f38d4025a70 con 0x7f38e406c0a0 2023-10-10T20:40:20.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.555+0000 7f38cbfff700 1 --2- 172.21.15.89:0/4035037725 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f38d0067200 0x7f38d00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:20.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.555+0000 7f38cbfff700 1 -- 172.21.15.89:0/4035037725 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f38d4085f60 con 0x7f38e406c0a0 2023-10-10T20:40:20.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.555+0000 7f38e3fff700 1 --2- 172.21.15.89:0/4035037725 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f38d0067200 0x7f38d00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:20.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.555+0000 7f38e3fff700 1 --2- 172.21.15.89:0/4035037725 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f38d0067200 0x7f38d00696b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f38d8007fd0 tx=0x7f38d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:20.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.559+0000 7f38cbfff700 1 -- 172.21.15.89:0/4035037725 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38d4052e80 con 0x7f38e406c0a0 2023-10-10T20:40:20.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.785+0000 7f38ea4ba700 1 -- 172.21.15.89:0/4035037725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f38e404c3c0 con 0x7f38e406c0a0 2023-10-10T20:40:20.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.786+0000 7f38cbfff700 1 -- 172.21.15.89:0/4035037725 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) v1 ==== 53+0+73 (secure 0 0 0) 0x7f38d4015070 con 0x7f38e406c0a0 2023-10-10T20:40:20.789 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:40:20.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.788+0000 7f38ea4ba700 1 -- 172.21.15.89:0/4035037725 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f38d0067200 msgr2=0x7f38d00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:20.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.788+0000 7f38ea4ba700 1 --2- 172.21.15.89:0/4035037725 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f38d0067200 0x7f38d00696b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f38d8007fd0 tx=0x7f38d8009040 comp rx=0 tx=0).stop 2023-10-10T20:40:20.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.788+0000 7f38ea4ba700 1 -- 172.21.15.89:0/4035037725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e406c0a0 msgr2=0x7f38e40c56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:20.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.788+0000 7f38ea4ba700 1 --2- 172.21.15.89:0/4035037725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e406c0a0 0x7f38e40c56b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f38e4069220 tx=0x7f38d4017000 comp rx=0 tx=0).stop 2023-10-10T20:40:20.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.788+0000 7f38ea4ba700 1 -- 172.21.15.89:0/4035037725 shutdown_connections 2023-10-10T20:40:20.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.788+0000 7f38ea4ba700 1 --2- 172.21.15.89:0/4035037725 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f38d0067200 0x7f38d00696b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:20.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.788+0000 7f38ea4ba700 1 --2- 172.21.15.89:0/4035037725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e415df50 0x7f38e4169f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:20.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.788+0000 7f38ea4ba700 1 --2- 172.21.15.89:0/4035037725 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38e406bcd0 0x7f38e40aa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:20.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.788+0000 7f38ea4ba700 1 --2- 172.21.15.89:0/4035037725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e406c0a0 0x7f38e40c56b0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:20.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.788+0000 7f38ea4ba700 1 -- 172.21.15.89:0/4035037725 >> 172.21.15.89:0/4035037725 conn(0x7f38e40ce600 msgr2=0x7f38e4069480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:20.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.788+0000 7f38ea4ba700 1 -- 172.21.15.89:0/4035037725 shutdown_connections 2023-10-10T20:40:20.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:20.788+0000 7f38ea4ba700 1 -- 172.21.15.89:0/4035037725 wait complete. 2023-10-10T20:40:20.802 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:40:20.803 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:40:20.803 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-10-10T20:40:20.803 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:40:20.804 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:40:20.804 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:40:20.804 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:40:20.805 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:40:20.805 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:40:20.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:20.806 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-10-10T20:40:20.827 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:40:20.827 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:40:20.827 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:40:20.828 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:20.828 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-10-10T20:40:20.828 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:20.828 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:40:20.828 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:20.829 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-10-10T20:40:20.829 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:40:20.829 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:20.829 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-10-10T20:40:20.829 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:40:20.830 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:20.830 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:40:20.830 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:20.830 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:40:20.831 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42161sec preferred_lft 42161sec 2023-10-10T20:40:20.831 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-10-10T20:40:20.831 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:20.831 INFO:teuthology.orchestra.run.smithi111.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:40:20.831 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:20.832 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:40:20.832 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:20.832 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::a486:a6ff:fe5f:6021/64 scope link 2023-10-10T20:40:20.832 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:20.832 INFO:teuthology.orchestra.run.smithi111.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:40:20.832 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 1e:d0:81:1b:e2:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:40:20.833 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::1cd0:81ff:fe1b:e26b/64 scope link 2023-10-10T20:40:20.833 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:20.833 INFO:teuthology.orchestra.run.smithi111.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:40:20.833 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:40:20.833 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::88d:cdff:fe75:455e/64 scope link 2023-10-10T20:40:20.834 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:20.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:20.834 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:40:20.893 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-10-10T20:40:20.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:20.894 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-10-10T20:40:20.951 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-10-10T20:40:20.952 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:40:20.952 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:40:20.952 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:40:20.952 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:40:20.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:20.953 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:40:20.953 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:40:20.954 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:40:20.954 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:40:20.954 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:40:20.955 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:40:21.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:21.055 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-10-10T20:40:21.070 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:40:21.070 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:40:21.070 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:40:21.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:21.071 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-10-10T20:40:21.134 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:40:21.134 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:40:21.188 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:40:21.188 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:40:21.189 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:40:21.241 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:40:21.242 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:40:21.242 DEBUG:teuthology.orchestra.run.smithi111:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:40:21.320 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:40:21.321 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:40:21.321 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:40:21.329 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:40:21.329 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:40:21.329 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:40:21.329 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:40:21.330 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:40:21.351 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:40:21.351 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:40:21.351 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:40:21.351 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:40:21.351 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:40:21.352 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:40:21.352 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:40:21.352 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:40:21.353 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:40:21.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:21.354 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:40:21.417 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-10T20:40:21.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.535+0000 7fbee5332700 1 Processor -- start 2023-10-10T20:40:21.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.535+0000 7fbee5332700 1 -- start start 2023-10-10T20:40:21.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.536+0000 7fbee5332700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbee00b55a0 0x7fbee0164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:21.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.536+0000 7fbee5332700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee00cfa60 con 0x7fbee0159ec0 2023-10-10T20:40:21.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.536+0000 7fbee5332700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee00b5b00 con 0x7fbee00cd910 2023-10-10T20:40:21.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.536+0000 7fbee5332700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee00b5da0 con 0x7fbee00b55a0 2023-10-10T20:40:21.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.536+0000 7fbedeffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbee0159ec0 0x7fbee00b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36414/0 (socket says 172.21.15.89:36414) 2023-10-10T20:40:21.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.536+0000 7fbedeffd700 1 -- 172.21.15.89:0/4009425953 learned_addr learned my addr 172.21.15.89:0/4009425953 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:21.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.536+0000 7fbedf7fe700 1 --2- 172.21.15.89:0/4009425953 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbee00b55a0 0x7fbee0164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:21.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.536+0000 7fbede7fc700 1 -- 172.21.15.89:0/4009425953 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2918209868 0 0) 0x7fbee00cfa60 con 0x7fbee0159ec0 2023-10-10T20:40:21.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbede7fc700 1 -- 172.21.15.89:0/4009425953 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbecc003570 con 0x7fbee0159ec0 2023-10-10T20:40:21.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbede7fc700 1 -- 172.21.15.89:0/4009425953 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2337663428 0 0) 0x7fbecc003570 con 0x7fbee0159ec0 2023-10-10T20:40:21.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbede7fc700 1 -- 172.21.15.89:0/4009425953 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbee00b55a0 msgr2=0x7fbee0164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:21.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbede7fc700 1 --2- 172.21.15.89:0/4009425953 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbee00b55a0 0x7fbee0164400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:21.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbede7fc700 1 -- 172.21.15.89:0/4009425953 >> v1:172.21.15.111:6789/0 conn(0x7fbee00cd910 legacy=0x7fbee00ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:21.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbede7fc700 1 -- 172.21.15.89:0/4009425953 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee00b6080 con 0x7fbee0159ec0 2023-10-10T20:40:21.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbede7fc700 1 -- 172.21.15.89:0/4009425953 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fbed00015f0 con 0x7fbee0159ec0 2023-10-10T20:40:21.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbede7fc700 1 -- 172.21.15.89:0/4009425953 >> v1:172.21.15.89:6789/0 conn(0x7fbee0159ec0 legacy=0x7fbee00b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:21.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbede7fc700 1 --2- 172.21.15.89:0/4009425953 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbee00b55a0 0x7fbecc003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:21.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbede7fc700 1 --2- 172.21.15.89:0/4009425953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbecc004010 0x7fbecc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:21.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbede7fc700 1 --2- 172.21.15.89:0/4009425953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc004d50 0x7fbecc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:21.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbede7fc700 1 -- 172.21.15.89:0/4009425953 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee00b5da0 con 0x7fbecc004010 2023-10-10T20:40:21.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbede7fc700 1 -- 172.21.15.89:0/4009425953 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc00bbf0 con 0x7fbecc004d50 2023-10-10T20:40:21.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbede7fc700 1 -- 172.21.15.89:0/4009425953 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc00bdb0 con 0x7fbee00b55a0 2023-10-10T20:40:21.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbed67fc700 1 --2- 172.21.15.89:0/4009425953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbecc004010 0x7fbecc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:21.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbedeffd700 1 --2- 172.21.15.89:0/4009425953 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbee00b55a0 0x7fbecc003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:21.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbedf7fe700 1 --2- 172.21.15.89:0/4009425953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc004d50 0x7fbecc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:21.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbed67fc700 1 -- 172.21.15.89:0/4009425953 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbee00b55a0 msgr2=0x7fbecc003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:21.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbed67fc700 1 --2- 172.21.15.89:0/4009425953 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbee00b55a0 0x7fbecc003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:21.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbed67fc700 1 -- 172.21.15.89:0/4009425953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc004d50 msgr2=0x7fbecc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:21.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbed67fc700 1 --2- 172.21.15.89:0/4009425953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc004d50 0x7fbecc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:21.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbed67fc700 1 -- 172.21.15.89:0/4009425953 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc003570 con 0x7fbecc004010 2023-10-10T20:40:21.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.537+0000 7fbed67fc700 1 --2- 172.21.15.89:0/4009425953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbecc004010 0x7fbecc004810 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fbee00c75d0 tx=0x7fbec8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f9a05deb3b3f13 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:21.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.538+0000 7fbede7fc700 1 -- 172.21.15.89:0/4009425953 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8012420 con 0x7fbecc004010 2023-10-10T20:40:21.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.538+0000 7fbede7fc700 1 -- 172.21.15.89:0/4009425953 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec8012580 con 0x7fbecc004010 2023-10-10T20:40:21.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.538+0000 7fbede7fc700 1 -- 172.21.15.89:0/4009425953 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8012920 con 0x7fbecc004010 2023-10-10T20:40:21.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.538+0000 7fbee5332700 1 -- 172.21.15.89:0/4009425953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbecc004010 msgr2=0x7fbecc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:21.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.538+0000 7fbee5332700 1 --2- 172.21.15.89:0/4009425953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbecc004010 0x7fbecc004810 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fbee00c75d0 tx=0x7fbec8007d30 comp rx=0 tx=0).stop 2023-10-10T20:40:21.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.538+0000 7fbedf7fe700 1 -- 172.21.15.89:0/4009425953 reap_dead start 2023-10-10T20:40:21.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.538+0000 7fbee5332700 1 -- 172.21.15.89:0/4009425953 shutdown_connections 2023-10-10T20:40:21.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.538+0000 7fbee5332700 1 -- 172.21.15.89:0/4009425953 >> 172.21.15.89:0/4009425953 conn(0x7fbee00999f0 msgr2=0x7fbee00b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:21.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.538+0000 7fbee5332700 1 -- 172.21.15.89:0/4009425953 shutdown_connections 2023-10-10T20:40:21.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.538+0000 7fbee5332700 1 -- 172.21.15.89:0/4009425953 wait complete. 2023-10-10T20:40:21.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.539+0000 7fbee5332700 1 Processor -- start 2023-10-10T20:40:21.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.539+0000 7fbee5332700 1 -- start start 2023-10-10T20:40:21.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.540+0000 7fbee5332700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc004010 0x7fbee006af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:21.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.540+0000 7fbee5332700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbee013cad0 0x7fbee015f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:21.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.540+0000 7fbee5332700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee015fc30 0x7fbee0157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:21.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.540+0000 7fbee5332700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee01600d0 con 0x7fbee015fc30 2023-10-10T20:40:21.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.540+0000 7fbee5332700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee006ab00 con 0x7fbecc004010 2023-10-10T20:40:21.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.540+0000 7fbee5332700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee0069ef0 con 0x7fbee013cad0 2023-10-10T20:40:21.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.540+0000 7fbedf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee015fc30 0x7fbee0157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:21.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.540+0000 7fbedf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee015fc30 0x7fbee0157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56578/0 (socket says 172.21.15.89:56578) 2023-10-10T20:40:21.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.540+0000 7fbedf7fe700 1 -- 172.21.15.89:0/4222874730 learned_addr learned my addr 172.21.15.89:0/4222874730 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:21.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.540+0000 7fbed67fc700 1 --2- 172.21.15.89:0/4222874730 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbee013cad0 0x7fbee015f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:21.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.540+0000 7fbedf7fe700 1 -- 172.21.15.89:0/4222874730 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbee013cad0 msgr2=0x7fbee015f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:21.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.540+0000 7fbedf7fe700 1 --2- 172.21.15.89:0/4222874730 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbee013cad0 0x7fbee015f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:21.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.540+0000 7fbedeffd700 1 --2- 172.21.15.89:0/4222874730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc004010 0x7fbee006af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:21.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.540+0000 7fbedf7fe700 1 -- 172.21.15.89:0/4222874730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc004010 msgr2=0x7fbee006af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:21.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.540+0000 7fbedf7fe700 1 --2- 172.21.15.89:0/4222874730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc004010 0x7fbee006af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:21.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.540+0000 7fbedf7fe700 1 -- 172.21.15.89:0/4222874730 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee006a1d0 con 0x7fbee015fc30 2023-10-10T20:40:21.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.540+0000 7fbed67fc700 1 --2- 172.21.15.89:0/4222874730 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbee013cad0 0x7fbee015f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:21.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.540+0000 7fbedeffd700 1 --2- 172.21.15.89:0/4222874730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc004010 0x7fbee006af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:21.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.541+0000 7fbedf7fe700 1 --2- 172.21.15.89:0/4222874730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee015fc30 0x7fbee0157360 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fbed80094d0 tx=0x7fbed800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:21.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.541+0000 7fbed77fe700 1 -- 172.21.15.89:0/4222874730 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed800e040 con 0x7fbee015fc30 2023-10-10T20:40:21.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.541+0000 7fbed77fe700 1 -- 172.21.15.89:0/4222874730 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed800f040 con 0x7fbee015fc30 2023-10-10T20:40:21.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.541+0000 7fbee5332700 1 -- 172.21.15.89:0/4222874730 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee006a4b0 con 0x7fbee015fc30 2023-10-10T20:40:21.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.541+0000 7fbed77fe700 1 -- 172.21.15.89:0/4222874730 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed800fe70 con 0x7fbee015fc30 2023-10-10T20:40:21.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.541+0000 7fbee5332700 1 -- 172.21.15.89:0/4222874730 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee013cf40 con 0x7fbee015fc30 2023-10-10T20:40:21.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.542+0000 7fbee5332700 1 -- 172.21.15.89:0/4222874730 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee0063c90 con 0x7fbee015fc30 2023-10-10T20:40:21.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.542+0000 7fbed77fe700 1 -- 172.21.15.89:0/4222874730 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fbed8015640 con 0x7fbee015fc30 2023-10-10T20:40:21.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.542+0000 7fbed77fe700 1 --2- 172.21.15.89:0/4222874730 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbec0067230 0x7fbec00696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:21.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.543+0000 7fbed77fe700 1 -- 172.21.15.89:0/4222874730 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbed80855d0 con 0x7fbee015fc30 2023-10-10T20:40:21.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.543+0000 7fbedeffd700 1 --2- 172.21.15.89:0/4222874730 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbec0067230 0x7fbec00696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:21.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.543+0000 7fbedeffd700 1 --2- 172.21.15.89:0/4222874730 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbec0067230 0x7fbec00696e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbed0000e10 tx=0x7fbed0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:21.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.547+0000 7fbed77fe700 1 -- 172.21.15.89:0/4222874730 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbed8052300 con 0x7fbee015fc30 2023-10-10T20:40:21.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:21.774+0000 7fbee5332700 1 -- 172.21.15.89:0/4222874730 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbee006ac90 con 0x7fbee015fc30 2023-10-10T20:40:22.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.066+0000 7fbed77fe700 1 -- 172.21.15.89:0/4222874730 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 136+0+0 (secure 0 0 0) 0x7fbed8055920 con 0x7fbee015fc30 2023-10-10T20:40:22.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.068+0000 7fbee5332700 1 -- 172.21.15.89:0/4222874730 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbec0067230 msgr2=0x7fbec00696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:22.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.068+0000 7fbee5332700 1 --2- 172.21.15.89:0/4222874730 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbec0067230 0x7fbec00696e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbed0000e10 tx=0x7fbed0009040 comp rx=0 tx=0).stop 2023-10-10T20:40:22.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.068+0000 7fbee5332700 1 -- 172.21.15.89:0/4222874730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee015fc30 msgr2=0x7fbee0157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:22.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.068+0000 7fbee5332700 1 --2- 172.21.15.89:0/4222874730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee015fc30 0x7fbee0157360 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fbed80094d0 tx=0x7fbed800b040 comp rx=0 tx=0).stop 2023-10-10T20:40:22.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.068+0000 7fbee5332700 1 -- 172.21.15.89:0/4222874730 shutdown_connections 2023-10-10T20:40:22.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.068+0000 7fbee5332700 1 --2- 172.21.15.89:0/4222874730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee015fc30 0x7fbee0157360 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.068+0000 7fbee5332700 1 --2- 172.21.15.89:0/4222874730 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbec0067230 0x7fbec00696e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.068+0000 7fbee5332700 1 --2- 172.21.15.89:0/4222874730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc004010 0x7fbee006af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.068+0000 7fbee5332700 1 --2- 172.21.15.89:0/4222874730 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbee013cad0 0x7fbee015f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.068+0000 7fbee5332700 1 -- 172.21.15.89:0/4222874730 >> 172.21.15.89:0/4222874730 conn(0x7fbee00999f0 msgr2=0x7fbee00c5680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:22.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.068+0000 7fbee5332700 1 -- 172.21.15.89:0/4222874730 shutdown_connections 2023-10-10T20:40:22.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.068+0000 7fbee5332700 1 -- 172.21.15.89:0/4222874730 wait complete. 2023-10-10T20:40:22.083 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-10T20:40:22.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.196+0000 7faa8247d700 1 Processor -- start 2023-10-10T20:40:22.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.196+0000 7faa8247d700 1 -- start start 2023-10-10T20:40:22.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.197+0000 7faa8247d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa7c159420 0x7faa7c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:22.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.197+0000 7faa8247d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa7c1597f0 0x7faa7c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:22.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.197+0000 7faa8247d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faa7c15d230 0x7faa7c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:22.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.197+0000 7faa8247d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7c15dbd0 con 0x7faa7c1597f0 2023-10-10T20:40:22.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.197+0000 7faa8247d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7c0b11d0 con 0x7faa7c15d230 2023-10-10T20:40:22.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.197+0000 7faa8247d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c0b14b0 con 0x7faa7c159420 2023-10-10T20:40:22.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.197+0000 7faa80c7a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa7c1597f0 0x7faa7c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:22.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.197+0000 7faa80c7a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa7c1597f0 0x7faa7c0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56582/0 (socket says 172.21.15.89:56582) 2023-10-10T20:40:22.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.197+0000 7faa80c7a700 1 -- 172.21.15.89:0/808979576 learned_addr learned my addr 172.21.15.89:0/808979576 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:22.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.197+0000 7faa80c7a700 1 -- 172.21.15.89:0/808979576 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa7c159420 msgr2=0x7faa7c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:22.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.197+0000 7faa8147b700 1 --2- 172.21.15.89:0/808979576 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa7c159420 0x7faa7c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:22.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.197+0000 7faa81c7c700 1 --2- 172.21.15.89:0/808979576 >> v2:172.21.15.111:3300/0 conn(0x7faa7c15d230 0x7faa7c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:22.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.197+0000 7faa80c7a700 1 --2- 172.21.15.89:0/808979576 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa7c159420 0x7faa7c0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.197+0000 7faa80c7a700 1 -- 172.21.15.89:0/808979576 >> v2:172.21.15.111:3300/0 conn(0x7faa7c15d230 msgr2=0x7faa7c15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:22.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.197+0000 7faa80c7a700 1 --2- 172.21.15.89:0/808979576 >> v2:172.21.15.111:3300/0 conn(0x7faa7c15d230 0x7faa7c15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.197+0000 7faa80c7a700 1 -- 172.21.15.89:0/808979576 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c0b1790 con 0x7faa7c1597f0 2023-10-10T20:40:22.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.197+0000 7faa8147b700 1 --2- 172.21.15.89:0/808979576 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa7c159420 0x7faa7c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:22.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.197+0000 7faa81c7c700 1 --2- 172.21.15.89:0/808979576 >> v2:172.21.15.111:3300/0 conn(0x7faa7c15d230 0x7faa7c15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:22.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.197+0000 7faa80c7a700 1 --2- 172.21.15.89:0/808979576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa7c1597f0 0x7faa7c0b2270 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0d1720 tx=0x7faa6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf3ae2ec692c4e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:22.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.198+0000 7faa73fff700 1 -- 172.21.15.89:0/808979576 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c00b860 con 0x7faa7c1597f0 2023-10-10T20:40:22.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.198+0000 7faa73fff700 1 -- 172.21.15.89:0/808979576 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6c00b9c0 con 0x7faa7c1597f0 2023-10-10T20:40:22.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.198+0000 7faa73fff700 1 -- 172.21.15.89:0/808979576 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c00bd60 con 0x7faa7c1597f0 2023-10-10T20:40:22.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.198+0000 7faa8247d700 1 -- 172.21.15.89:0/808979576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa7c1597f0 msgr2=0x7faa7c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:22.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.198+0000 7faa8247d700 1 --2- 172.21.15.89:0/808979576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa7c1597f0 0x7faa7c0b2270 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0d1720 tx=0x7faa6c009250 comp rx=0 tx=0).stop 2023-10-10T20:40:22.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.198+0000 7faa8247d700 1 -- 172.21.15.89:0/808979576 shutdown_connections 2023-10-10T20:40:22.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.198+0000 7faa8247d700 1 --2- 172.21.15.89:0/808979576 >> v2:172.21.15.111:3300/0 conn(0x7faa7c15d230 0x7faa7c15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.198+0000 7faa8247d700 1 --2- 172.21.15.89:0/808979576 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa7c159420 0x7faa7c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.198+0000 7faa8247d700 1 --2- 172.21.15.89:0/808979576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa7c1597f0 0x7faa7c0b2270 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.198+0000 7faa8247d700 1 -- 172.21.15.89:0/808979576 >> 172.21.15.89:0/808979576 conn(0x7faa7c0d0660 msgr2=0x7faa7c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:22.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.198+0000 7faa8247d700 1 -- 172.21.15.89:0/808979576 shutdown_connections 2023-10-10T20:40:22.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.198+0000 7faa8247d700 1 -- 172.21.15.89:0/808979576 wait complete. 2023-10-10T20:40:22.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.199+0000 7faa8247d700 1 Processor -- start 2023-10-10T20:40:22.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.199+0000 7faa8247d700 1 -- start start 2023-10-10T20:40:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.200+0000 7faa8247d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa7c159420 0x7faa7c1694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.200+0000 7faa8247d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa7c1597f0 0x7faa7c15b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.200+0000 7faa8247d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa7c15d230 0x7faa7c0c6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.200+0000 7faa8247d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c0c63f0 con 0x7faa7c1597f0 2023-10-10T20:40:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.200+0000 7faa8247d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c0c57e0 con 0x7faa7c159420 2023-10-10T20:40:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.200+0000 7faa8247d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c0c5ac0 con 0x7faa7c15d230 2023-10-10T20:40:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.200+0000 7faa80c7a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa7c1597f0 0x7faa7c15b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.200+0000 7faa80c7a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa7c1597f0 0x7faa7c15b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56594/0 (socket says 172.21.15.89:56594) 2023-10-10T20:40:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.200+0000 7faa80c7a700 1 -- 172.21.15.89:0/2994998298 learned_addr learned my addr 172.21.15.89:0/2994998298 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.200+0000 7faa80c7a700 1 -- 172.21.15.89:0/2994998298 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa7c15d230 msgr2=0x7faa7c0c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:22.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.200+0000 7faa80c7a700 1 --2- 172.21.15.89:0/2994998298 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa7c15d230 0x7faa7c0c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.200+0000 7faa80c7a700 1 -- 172.21.15.89:0/2994998298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa7c159420 msgr2=0x7faa7c1694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:22.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.200+0000 7faa80c7a700 1 --2- 172.21.15.89:0/2994998298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa7c159420 0x7faa7c1694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.200+0000 7faa80c7a700 1 -- 172.21.15.89:0/2994998298 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6c0094f0 con 0x7faa7c1597f0 2023-10-10T20:40:22.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.201+0000 7faa80c7a700 1 --2- 172.21.15.89:0/2994998298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa7c1597f0 0x7faa7c15b370 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7faa7c089340 tx=0x7faa6c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:22.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.201+0000 7faa717fa700 1 -- 172.21.15.89:0/2994998298 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c00bcd0 con 0x7faa7c1597f0 2023-10-10T20:40:22.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.201+0000 7faa717fa700 1 -- 172.21.15.89:0/2994998298 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6c01a630 con 0x7faa7c1597f0 2023-10-10T20:40:22.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.201+0000 7faa8247d700 1 -- 172.21.15.89:0/2994998298 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7c0c5da0 con 0x7faa7c1597f0 2023-10-10T20:40:22.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.201+0000 7faa717fa700 1 -- 172.21.15.89:0/2994998298 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c01a9f0 con 0x7faa7c1597f0 2023-10-10T20:40:22.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.201+0000 7faa8247d700 1 -- 172.21.15.89:0/2994998298 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7c0c9b90 con 0x7faa7c1597f0 2023-10-10T20:40:22.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.202+0000 7faa8247d700 1 -- 172.21.15.89:0/2994998298 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa7c04c3c0 con 0x7faa7c1597f0 2023-10-10T20:40:22.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.202+0000 7faa717fa700 1 -- 172.21.15.89:0/2994998298 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7faa6c0233e0 con 0x7faa7c1597f0 2023-10-10T20:40:22.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.203+0000 7faa717fa700 1 --2- 172.21.15.89:0/2994998298 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faa60067200 0x7faa600696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:22.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.203+0000 7faa717fa700 1 -- 172.21.15.89:0/2994998298 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faa6c086690 con 0x7faa7c1597f0 2023-10-10T20:40:22.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.203+0000 7faa8147b700 1 --2- 172.21.15.89:0/2994998298 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faa60067200 0x7faa600696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:22.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.203+0000 7faa8147b700 1 --2- 172.21.15.89:0/2994998298 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faa60067200 0x7faa600696b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7faa78007970 tx=0x7faa78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:22.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.207+0000 7faa717fa700 1 -- 172.21.15.89:0/2994998298 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa6c0535b0 con 0x7faa7c1597f0 2023-10-10T20:40:22.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.416+0000 7faa8247d700 1 -- 172.21.15.89:0/2994998298 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7faa7c0c6530 con 0x7faa60067200 2023-10-10T20:40:22.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.418+0000 7faa717fa700 1 -- 172.21.15.89:0/2994998298 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7faa7c0c6530 con 0x7faa60067200 2023-10-10T20:40:22.420 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:40:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs" 2023-10-10T20:40:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:22.421 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:40:22.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.420+0000 7faa8247d700 1 -- 172.21.15.89:0/2994998298 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faa60067200 msgr2=0x7faa600696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:22.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.420+0000 7faa8247d700 1 --2- 172.21.15.89:0/2994998298 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faa60067200 0x7faa600696b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7faa78007970 tx=0x7faa78008040 comp rx=0 tx=0).stop 2023-10-10T20:40:22.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.420+0000 7faa8247d700 1 -- 172.21.15.89:0/2994998298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa7c1597f0 msgr2=0x7faa7c15b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:22.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.420+0000 7faa8247d700 1 --2- 172.21.15.89:0/2994998298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa7c1597f0 0x7faa7c15b370 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7faa7c089340 tx=0x7faa6c017000 comp rx=0 tx=0).stop 2023-10-10T20:40:22.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.420+0000 7faa8247d700 1 -- 172.21.15.89:0/2994998298 shutdown_connections 2023-10-10T20:40:22.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.420+0000 7faa8247d700 1 --2- 172.21.15.89:0/2994998298 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faa7c15d230 0x7faa7c0c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.420+0000 7faa8247d700 1 --2- 172.21.15.89:0/2994998298 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faa60067200 0x7faa600696b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.420+0000 7faa8247d700 1 --2- 172.21.15.89:0/2994998298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa7c159420 0x7faa7c1694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.420+0000 7faa8247d700 1 --2- 172.21.15.89:0/2994998298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa7c1597f0 0x7faa7c15b370 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.420+0000 7faa8247d700 1 -- 172.21.15.89:0/2994998298 >> 172.21.15.89:0/2994998298 conn(0x7faa7c0d0660 msgr2=0x7faa7c156ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:22.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.420+0000 7faa8247d700 1 -- 172.21.15.89:0/2994998298 shutdown_connections 2023-10-10T20:40:22.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.420+0000 7faa8247d700 1 -- 172.21.15.89:0/2994998298 wait complete. 2023-10-10T20:40:22.435 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:40:22.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.551+0000 7fcbf949e700 1 Processor -- start 2023-10-10T20:40:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.552+0000 7fcbf949e700 1 -- start start 2023-10-10T20:40:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.552+0000 7fcbf949e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcbf406bcd0 0x7fcbf40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.552+0000 7fcbf949e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcbf415df50 0x7fcbf415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.552+0000 7fcbf949e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf415d7e0 con 0x7fcbf406c0a0 2023-10-10T20:40:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.552+0000 7fcbf949e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf415dac0 con 0x7fcbf406bcd0 2023-10-10T20:40:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.552+0000 7fcbf949e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf415aa30 con 0x7fcbf415df50 2023-10-10T20:40:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.553+0000 7fcbf2ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcbf406bcd0 0x7fcbf40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:22.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.553+0000 7fcbf2ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcbf406bcd0 0x7fcbf40caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56604/0 (socket says 172.21.15.89:56604) 2023-10-10T20:40:22.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.553+0000 7fcbea7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcbf406c0a0 0x7fcbf415a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36428/0 (socket says 172.21.15.89:36428) 2023-10-10T20:40:22.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.553+0000 7fcbf2ffd700 1 -- 172.21.15.89:0/2323961107 learned_addr learned my addr 172.21.15.89:0/2323961107 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:22.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.553+0000 7fcbf2ffd700 1 -- 172.21.15.89:0/2323961107 >> v2:172.21.15.111:3300/0 conn(0x7fcbf415df50 msgr2=0x7fcbf415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:22.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.553+0000 7fcbf37fe700 1 --2- 172.21.15.89:0/2323961107 >> v2:172.21.15.111:3300/0 conn(0x7fcbf415df50 0x7fcbf415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:22.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.553+0000 7fcbf2ffd700 1 --2- 172.21.15.89:0/2323961107 >> v2:172.21.15.111:3300/0 conn(0x7fcbf415df50 0x7fcbf415e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.553+0000 7fcbf27fc700 1 -- 172.21.15.89:0/2323961107 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609869679 0 0) 0x7fcbf415d7e0 con 0x7fcbf406c0a0 2023-10-10T20:40:22.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.553+0000 7fcbf2ffd700 1 -- 172.21.15.89:0/2323961107 >> v1:172.21.15.89:6789/0 conn(0x7fcbf406c0a0 legacy=0x7fcbf415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:22.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.553+0000 7fcbf2ffd700 1 -- 172.21.15.89:0/2323961107 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf415ad10 con 0x7fcbf406bcd0 2023-10-10T20:40:22.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.553+0000 7fcbf37fe700 1 --2- 172.21.15.89:0/2323961107 >> v2:172.21.15.111:3300/0 conn(0x7fcbf415df50 0x7fcbf415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.553+0000 7fcbf2ffd700 1 --2- 172.21.15.89:0/2323961107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf406bcd0 0x7fcbf40caa90 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fcbf415d720 tx=0x7fcbe4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982d13228dd4971c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.553+0000 7fcbf27fc700 1 -- 172.21.15.89:0/2323961107 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe400b860 con 0x7fcbf406bcd0 2023-10-10T20:40:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.553+0000 7fcbf27fc700 1 -- 172.21.15.89:0/2323961107 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe400b9c0 con 0x7fcbf406bcd0 2023-10-10T20:40:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.553+0000 7fcbf27fc700 1 -- 172.21.15.89:0/2323961107 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe400bd80 con 0x7fcbf406bcd0 2023-10-10T20:40:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.554+0000 7fcbf949e700 1 -- 172.21.15.89:0/2323961107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf406bcd0 msgr2=0x7fcbf40caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.554+0000 7fcbf949e700 1 --2- 172.21.15.89:0/2323961107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf406bcd0 0x7fcbf40caa90 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fcbf415d720 tx=0x7fcbe4009250 comp rx=0 tx=0).stop 2023-10-10T20:40:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.554+0000 7fcbf949e700 1 -- 172.21.15.89:0/2323961107 shutdown_connections 2023-10-10T20:40:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.554+0000 7fcbf949e700 1 --2- 172.21.15.89:0/2323961107 >> v2:172.21.15.111:3300/0 conn(0x7fcbf415df50 0x7fcbf415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.554+0000 7fcbf949e700 1 --2- 172.21.15.89:0/2323961107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf406bcd0 0x7fcbf40caa90 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.554+0000 7fcbf949e700 1 -- 172.21.15.89:0/2323961107 >> 172.21.15.89:0/2323961107 conn(0x7fcbf40ce600 msgr2=0x7fcbf40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:22.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.554+0000 7fcbf949e700 1 -- 172.21.15.89:0/2323961107 shutdown_connections 2023-10-10T20:40:22.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.554+0000 7fcbf949e700 1 -- 172.21.15.89:0/2323961107 wait complete. 2023-10-10T20:40:22.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.555+0000 7fcbf949e700 1 Processor -- start 2023-10-10T20:40:22.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.555+0000 7fcbf949e700 1 -- start start 2023-10-10T20:40:22.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.556+0000 7fcbf949e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbf406bcd0 0x7fcbf4153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:22.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.556+0000 7fcbf949e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcbf406c0a0 0x7fcbf40c6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:22.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.556+0000 7fcbf949e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf415df50 0x7fcbf406d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:22.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.556+0000 7fcbf949e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf415a070 con 0x7fcbf415df50 2023-10-10T20:40:22.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.556+0000 7fcbf949e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf40c6080 con 0x7fcbf406bcd0 2023-10-10T20:40:22.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.556+0000 7fcbf949e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf40c5750 con 0x7fcbf406c0a0 2023-10-10T20:40:22.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.556+0000 7fcbf37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf415df50 0x7fcbf406d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:22.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.556+0000 7fcbf37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf415df50 0x7fcbf406d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56610/0 (socket says 172.21.15.89:56610) 2023-10-10T20:40:22.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.556+0000 7fcbf37fe700 1 -- 172.21.15.89:0/1051900110 learned_addr learned my addr 172.21.15.89:0/1051900110 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:22.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.556+0000 7fcbf2ffd700 1 --2- 172.21.15.89:0/1051900110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbf406bcd0 0x7fcbf4153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:22.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.556+0000 7fcbea7fc700 1 --2- 172.21.15.89:0/1051900110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcbf406c0a0 0x7fcbf40c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:22.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.556+0000 7fcbf37fe700 1 -- 172.21.15.89:0/1051900110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcbf406c0a0 msgr2=0x7fcbf40c6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:22.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.556+0000 7fcbf37fe700 1 --2- 172.21.15.89:0/1051900110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcbf406c0a0 0x7fcbf40c6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.556+0000 7fcbf37fe700 1 -- 172.21.15.89:0/1051900110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbf406bcd0 msgr2=0x7fcbf4153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:22.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.556+0000 7fcbf37fe700 1 --2- 172.21.15.89:0/1051900110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbf406bcd0 0x7fcbf4153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.556+0000 7fcbf37fe700 1 -- 172.21.15.89:0/1051900110 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe40094f0 con 0x7fcbf415df50 2023-10-10T20:40:22.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.556+0000 7fcbea7fc700 1 --2- 172.21.15.89:0/1051900110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcbf406c0a0 0x7fcbf40c6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:22.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.556+0000 7fcbf37fe700 1 --2- 172.21.15.89:0/1051900110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf415df50 0x7fcbf406d240 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00cf20 tx=0x7fcbec007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:22.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.557+0000 7fcbeb7fe700 1 -- 172.21.15.89:0/1051900110 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec00ad40 con 0x7fcbf415df50 2023-10-10T20:40:22.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.557+0000 7fcbeb7fe700 1 -- 172.21.15.89:0/1051900110 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbec016420 con 0x7fcbf415df50 2023-10-10T20:40:22.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.557+0000 7fcbf949e700 1 -- 172.21.15.89:0/1051900110 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf40c5a30 con 0x7fcbf415df50 2023-10-10T20:40:22.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.557+0000 7fcbeb7fe700 1 -- 172.21.15.89:0/1051900110 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec0167e0 con 0x7fcbf415df50 2023-10-10T20:40:22.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.557+0000 7fcbf949e700 1 -- 172.21.15.89:0/1051900110 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf4153dc0 con 0x7fcbf415df50 2023-10-10T20:40:22.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.558+0000 7fcbf949e700 1 -- 172.21.15.89:0/1051900110 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf40c6210 con 0x7fcbf415df50 2023-10-10T20:40:22.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.558+0000 7fcbeb7fe700 1 -- 172.21.15.89:0/1051900110 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fcbec016bd0 con 0x7fcbf415df50 2023-10-10T20:40:22.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.558+0000 7fcbeb7fe700 1 --2- 172.21.15.89:0/1051900110 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcbd4067200 0x7fcbd40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:22.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.559+0000 7fcbf2ffd700 1 --2- 172.21.15.89:0/1051900110 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcbd4067200 0x7fcbd40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:22.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.559+0000 7fcbeb7fe700 1 -- 172.21.15.89:0/1051900110 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcbec012070 con 0x7fcbf415df50 2023-10-10T20:40:22.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.559+0000 7fcbf2ffd700 1 --2- 172.21.15.89:0/1051900110 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcbd4067200 0x7fcbd40696b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcbf415d720 tx=0x7fcbe4017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:22.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.563+0000 7fcbeb7fe700 1 -- 172.21.15.89:0/1051900110 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcbec04e5a0 con 0x7fcbf415df50 2023-10-10T20:40:22.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.747+0000 7fcbf949e700 1 -- 172.21.15.89:0/1051900110 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcbf404c3c0 con 0x7fcbf415df50 2023-10-10T20:40:22.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.747+0000 7fcbeb7fe700 1 -- 172.21.15.89:0/1051900110 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcbec052180 con 0x7fcbf415df50 2023-10-10T20:40:22.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.749+0000 7fcbf949e700 1 -- 172.21.15.89:0/1051900110 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcbd4067200 msgr2=0x7fcbd40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:22.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.749+0000 7fcbf949e700 1 --2- 172.21.15.89:0/1051900110 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcbd4067200 0x7fcbd40696b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcbf415d720 tx=0x7fcbe4017000 comp rx=0 tx=0).stop 2023-10-10T20:40:22.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.749+0000 7fcbf949e700 1 -- 172.21.15.89:0/1051900110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf415df50 msgr2=0x7fcbf406d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:22.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.749+0000 7fcbf949e700 1 --2- 172.21.15.89:0/1051900110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf415df50 0x7fcbf406d240 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00cf20 tx=0x7fcbec007860 comp rx=0 tx=0).stop 2023-10-10T20:40:22.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.749+0000 7fcbf949e700 1 -- 172.21.15.89:0/1051900110 shutdown_connections 2023-10-10T20:40:22.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.749+0000 7fcbf949e700 1 --2- 172.21.15.89:0/1051900110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf415df50 0x7fcbf406d240 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.749+0000 7fcbf949e700 1 --2- 172.21.15.89:0/1051900110 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcbd4067200 0x7fcbd40696b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.749+0000 7fcbf949e700 1 --2- 172.21.15.89:0/1051900110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbf406bcd0 0x7fcbf4153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.749+0000 7fcbf949e700 1 --2- 172.21.15.89:0/1051900110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcbf406c0a0 0x7fcbf40c6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.749+0000 7fcbf949e700 1 -- 172.21.15.89:0/1051900110 >> 172.21.15.89:0/1051900110 conn(0x7fcbf40ce600 msgr2=0x7fcbf406a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:22.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.749+0000 7fcbf949e700 1 -- 172.21.15.89:0/1051900110 shutdown_connections 2023-10-10T20:40:22.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.749+0000 7fcbf949e700 1 -- 172.21.15.89:0/1051900110 wait complete. 2023-10-10T20:40:22.763 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs --human_readable 2023-10-10T20:40:22.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.878+0000 7fe84a0d7700 1 Processor -- start 2023-10-10T20:40:22.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.878+0000 7fe84a0d7700 1 -- start start 2023-10-10T20:40:22.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.879+0000 7fe84a0d7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8440b16d0 0x7fe8440b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:22.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.879+0000 7fe84a0d7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe8440b33b0 0x7fe8440b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:22.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.879+0000 7fe84a0d7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8440b2080 con 0x7fe8440b4e70 2023-10-10T20:40:22.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.879+0000 7fe84a0d7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8440cfb00 con 0x7fe8440b33b0 2023-10-10T20:40:22.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.879+0000 7fe84a0d7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8440b5c40 con 0x7fe8440b16d0 2023-10-10T20:40:22.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.879+0000 7fe8490d5700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe8440b4e70 0x7fe8440ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36430/0 (socket says 172.21.15.89:36430) 2023-10-10T20:40:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.879+0000 7fe8490d5700 1 -- 172.21.15.89:0/189933568 learned_addr learned my addr 172.21.15.89:0/189933568 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.879+0000 7fe8498d6700 1 --2- 172.21.15.89:0/189933568 >> v2:172.21.15.89:3300/0 conn(0x7fe8440b33b0 0x7fe8440b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.879+0000 7fe8488d4700 1 --2- 172.21.15.89:0/189933568 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8440b16d0 0x7fe8440b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.879+0000 7fe83bfff700 1 -- 172.21.15.89:0/189933568 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077378747 0 0) 0x7fe8440b2080 con 0x7fe8440b4e70 2023-10-10T20:40:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.879+0000 7fe83bfff700 1 -- 172.21.15.89:0/189933568 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe830003570 con 0x7fe8440b4e70 2023-10-10T20:40:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.879+0000 7fe8498d6700 1 -- 172.21.15.89:0/189933568 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8440b16d0 msgr2=0x7fe8440b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:22.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.879+0000 7fe8498d6700 1 --2- 172.21.15.89:0/189933568 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8440b16d0 0x7fe8440b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.879+0000 7fe8498d6700 1 -- 172.21.15.89:0/189933568 >> v1:172.21.15.89:6789/0 conn(0x7fe8440b4e70 legacy=0x7fe8440ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:22.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.880+0000 7fe8498d6700 1 -- 172.21.15.89:0/189933568 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8440b5f20 con 0x7fe8440b33b0 2023-10-10T20:40:22.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.880+0000 7fe8498d6700 1 --2- 172.21.15.89:0/189933568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8440b33b0 0x7fe8440b3830 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe83c009e60 tx=0x7fe83c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed0a816ce561590 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:22.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.880+0000 7fe83bfff700 1 -- 172.21.15.89:0/189933568 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c00e070 con 0x7fe8440b33b0 2023-10-10T20:40:22.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.880+0000 7fe83bfff700 1 -- 172.21.15.89:0/189933568 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83c00bab0 con 0x7fe8440b33b0 2023-10-10T20:40:22.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.880+0000 7fe83bfff700 1 -- 172.21.15.89:0/189933568 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c00be50 con 0x7fe8440b33b0 2023-10-10T20:40:22.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.880+0000 7fe84a0d7700 1 -- 172.21.15.89:0/189933568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8440b33b0 msgr2=0x7fe8440b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:22.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.880+0000 7fe84a0d7700 1 --2- 172.21.15.89:0/189933568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8440b33b0 0x7fe8440b3830 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe83c009e60 tx=0x7fe83c009780 comp rx=0 tx=0).stop 2023-10-10T20:40:22.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.880+0000 7fe84a0d7700 1 -- 172.21.15.89:0/189933568 shutdown_connections 2023-10-10T20:40:22.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.880+0000 7fe84a0d7700 1 --2- 172.21.15.89:0/189933568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8440b33b0 0x7fe8440b3830 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.880+0000 7fe84a0d7700 1 --2- 172.21.15.89:0/189933568 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8440b16d0 0x7fe8440b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.880+0000 7fe84a0d7700 1 -- 172.21.15.89:0/189933568 >> 172.21.15.89:0/189933568 conn(0x7fe8440d10c0 msgr2=0x7fe8440affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:22.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.880+0000 7fe84a0d7700 1 -- 172.21.15.89:0/189933568 shutdown_connections 2023-10-10T20:40:22.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.880+0000 7fe84a0d7700 1 -- 172.21.15.89:0/189933568 wait complete. 2023-10-10T20:40:22.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.881+0000 7fe84a0d7700 1 Processor -- start 2023-10-10T20:40:22.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.881+0000 7fe84a0d7700 1 -- start start 2023-10-10T20:40:22.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.882+0000 7fe84a0d7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8440b16d0 0x7fe8440c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:22.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.882+0000 7fe84a0d7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8440b33b0 0x7fe8440ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:22.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.882+0000 7fe84a0d7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8440b4e70 0x7fe8440c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:22.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.882+0000 7fe84a0d7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8440c8220 con 0x7fe8440b16d0 2023-10-10T20:40:22.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.882+0000 7fe84a0d7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8440c8500 con 0x7fe8440b33b0 2023-10-10T20:40:22.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.882+0000 7fe84a0d7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8440c87e0 con 0x7fe8440b4e70 2023-10-10T20:40:22.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.882+0000 7fe8490d5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8440b16d0 0x7fe8440c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:22.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.882+0000 7fe8490d5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8440b16d0 0x7fe8440c7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56636/0 (socket says 172.21.15.89:56636) 2023-10-10T20:40:22.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.882+0000 7fe8490d5700 1 -- 172.21.15.89:0/2862770964 learned_addr learned my addr 172.21.15.89:0/2862770964 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:22.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.883+0000 7fe8498d6700 1 --2- 172.21.15.89:0/2862770964 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8440b4e70 0x7fe8440c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:22.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.883+0000 7fe8488d4700 1 --2- 172.21.15.89:0/2862770964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8440b33b0 0x7fe8440ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:22.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.883+0000 7fe8490d5700 1 -- 172.21.15.89:0/2862770964 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8440b4e70 msgr2=0x7fe8440c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:22.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.883+0000 7fe8490d5700 1 --2- 172.21.15.89:0/2862770964 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8440b4e70 0x7fe8440c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.883+0000 7fe8490d5700 1 -- 172.21.15.89:0/2862770964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8440b33b0 msgr2=0x7fe8440ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.883+0000 7fe8490d5700 1 --2- 172.21.15.89:0/2862770964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8440b33b0 0x7fe8440ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.883+0000 7fe8490d5700 1 -- 172.21.15.89:0/2862770964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c009a20 con 0x7fe8440b16d0 2023-10-10T20:40:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.883+0000 7fe8490d5700 1 --2- 172.21.15.89:0/2862770964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8440b16d0 0x7fe8440c7b80 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe84000c590 tx=0x7fe840007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.883+0000 7fe8397fa700 1 -- 172.21.15.89:0/2862770964 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe840005b30 con 0x7fe8440b16d0 2023-10-10T20:40:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.883+0000 7fe8397fa700 1 -- 172.21.15.89:0/2862770964 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe840005cb0 con 0x7fe8440b16d0 2023-10-10T20:40:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.883+0000 7fe84a0d7700 1 -- 172.21.15.89:0/2862770964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8440c8ac0 con 0x7fe8440b16d0 2023-10-10T20:40:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.883+0000 7fe8397fa700 1 -- 172.21.15.89:0/2862770964 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe840007320 con 0x7fe8440b16d0 2023-10-10T20:40:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.883+0000 7fe84a0d7700 1 -- 172.21.15.89:0/2862770964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8440ca680 con 0x7fe8440b16d0 2023-10-10T20:40:22.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.884+0000 7fe84a0d7700 1 -- 172.21.15.89:0/2862770964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe844088990 con 0x7fe8440b16d0 2023-10-10T20:40:22.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.884+0000 7fe8397fa700 1 -- 172.21.15.89:0/2862770964 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fe84001b3b0 con 0x7fe8440b16d0 2023-10-10T20:40:22.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.885+0000 7fe8397fa700 1 --2- 172.21.15.89:0/2862770964 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe828067200 0x7fe8280696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:22.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.885+0000 7fe8397fa700 1 -- 172.21.15.89:0/2862770964 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe840083f40 con 0x7fe8440b16d0 2023-10-10T20:40:22.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.885+0000 7fe8488d4700 1 --2- 172.21.15.89:0/2862770964 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe828067200 0x7fe8280696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:22.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.885+0000 7fe8488d4700 1 --2- 172.21.15.89:0/2862770964 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe828067200 0x7fe8280696b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe8440c9bb0 tx=0x7fe834008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:22.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:22.889+0000 7fe8397fa700 1 -- 172.21.15.89:0/2862770964 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe840050e60 con 0x7fe8440b16d0 2023-10-10T20:40:23.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.102+0000 7fe84a0d7700 1 -- 172.21.15.89:0/2862770964 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "human_readable": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe8440c8670 con 0x7fe828067200 2023-10-10T20:40:23.106 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:40:23.104+0000 7f023b8aa700 -1 client.6297 remove_session_caps still has dirty|flushing caps on 0x10000000001.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2023-10-10T20:39:53.003837+0000 mtime=2023-10-10T20:39:53.003837+0000 ctime=2023-10-10T20:39:53.009100+0000 change_attr=2 caps=- flushing_caps=Ax COMPLETE parents=0x10000000000.head["group_0000000000078545"] has_dir_layout 0x55a1c6047600) 2023-10-10T20:40:23.109 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:40:23.107+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:40:23.110 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:40:23.107+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:40:23.110 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:40:23.107+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:40:23.110 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:40:23.107+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:40:23.110 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:40:23.107+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:40:23.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.113+0000 7fe8397fa700 1 -- 172.21.15.89:0/2862770964 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+470 (secure 0 0 0) 0x7fe8440c8670 con 0x7fe828067200 2023-10-10T20:40:23.115 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:40:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "mon_addrs": [ 2023-10-10T20:40:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "172.21.15.89:6789", 2023-10-10T20:40:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "172.21.15.111:6789", 2023-10-10T20:40:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "172.21.15.111:6790" 2023-10-10T20:40:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-10T20:40:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "pools": { 2023-10-10T20:40:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "data": [ 2023-10-10T20:40:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "avail": " 340G", 2023-10-10T20:40:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs_data", 2023-10-10T20:40:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "used": " 0 " 2023-10-10T20:40:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-10T20:40:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "metadata": [ 2023-10-10T20:40:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "avail": " 340G", 2023-10-10T20:40:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs_metadata", 2023-10-10T20:40:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "used": "64.0k" 2023-10-10T20:40:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-10T20:40:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:23.120 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:40:23.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.115+0000 7fe84a0d7700 1 -- 172.21.15.89:0/2862770964 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe828067200 msgr2=0x7fe8280696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:23.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.115+0000 7fe84a0d7700 1 --2- 172.21.15.89:0/2862770964 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe828067200 0x7fe8280696b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe8440c9bb0 tx=0x7fe834008040 comp rx=0 tx=0).stop 2023-10-10T20:40:23.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.115+0000 7fe84a0d7700 1 -- 172.21.15.89:0/2862770964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8440b16d0 msgr2=0x7fe8440c7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:23.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.115+0000 7fe84a0d7700 1 --2- 172.21.15.89:0/2862770964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8440b16d0 0x7fe8440c7b80 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe84000c590 tx=0x7fe840007e00 comp rx=0 tx=0).stop 2023-10-10T20:40:23.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.115+0000 7fe84a0d7700 1 -- 172.21.15.89:0/2862770964 shutdown_connections 2023-10-10T20:40:23.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.115+0000 7fe84a0d7700 1 --2- 172.21.15.89:0/2862770964 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe8440b4e70 0x7fe8440c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.115+0000 7fe84a0d7700 1 --2- 172.21.15.89:0/2862770964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8440b16d0 0x7fe8440c7b80 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.115+0000 7fe84a0d7700 1 --2- 172.21.15.89:0/2862770964 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe828067200 0x7fe8280696b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.115+0000 7fe84a0d7700 1 --2- 172.21.15.89:0/2862770964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8440b33b0 0x7fe8440ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.115+0000 7fe84a0d7700 1 -- 172.21.15.89:0/2862770964 >> 172.21.15.89:0/2862770964 conn(0x7fe8440d10c0 msgr2=0x7fe84415f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:23.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.115+0000 7fe84a0d7700 1 -- 172.21.15.89:0/2862770964 shutdown_connections 2023-10-10T20:40:23.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.115+0000 7fe84a0d7700 1 -- 172.21.15.89:0/2862770964 wait complete. 2023-10-10T20:40:23.129 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:40:23.129 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:40:23.162 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:40:23.203 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:40:23.204 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:40:23.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:40:23.204 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:40:23.247 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:40:23.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:23.248 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:40:23.303 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:40:23.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:40:23.304 DEBUG:teuthology.orchestra.run.smithi111:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:40:23.346 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:40:23.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:23.347 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:40:23.406 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:40:23.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.519+0000 7f025dd2a700 1 Processor -- start 2023-10-10T20:40:23.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.519+0000 7f025dd2a700 1 -- start start 2023-10-10T20:40:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.520+0000 7f025dd2a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0258159420 0x7f02580b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.520+0000 7f025dd2a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02581597f0 0x7f02580b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.520+0000 7f025dd2a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f025815d230 0x7f025815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.520+0000 7f025dd2a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025815dbd0 con 0x7f025815d230 2023-10-10T20:40:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.520+0000 7f025dd2a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02580b11d0 con 0x7f0258159420 2023-10-10T20:40:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.520+0000 7f025dd2a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02580b14b0 con 0x7f02581597f0 2023-10-10T20:40:23.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.520+0000 7f025d529700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f025815d230 0x7f025815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:23.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.520+0000 7f025d529700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f025815d230 0x7f025815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56666/0 (socket says 172.21.15.89:56666) 2023-10-10T20:40:23.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.520+0000 7f025d529700 1 -- 172.21.15.89:0/1406127298 learned_addr learned my addr 172.21.15.89:0/1406127298 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:23.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.520+0000 7f0257fff700 1 --2- 172.21.15.89:0/1406127298 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02581597f0 0x7f02580b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:23.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.520+0000 7f025cd28700 1 --2- 172.21.15.89:0/1406127298 >> v2:172.21.15.111:3300/0 conn(0x7f0258159420 0x7f02580b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:23.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.520+0000 7f025d529700 1 -- 172.21.15.89:0/1406127298 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02581597f0 msgr2=0x7f02580b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:23.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.520+0000 7f025d529700 1 --2- 172.21.15.89:0/1406127298 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02581597f0 0x7f02580b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.520+0000 7f025d529700 1 -- 172.21.15.89:0/1406127298 >> v2:172.21.15.111:3300/0 conn(0x7f0258159420 msgr2=0x7f02580b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:23.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.520+0000 7f025d529700 1 --2- 172.21.15.89:0/1406127298 >> v2:172.21.15.111:3300/0 conn(0x7f0258159420 0x7f02580b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.520+0000 7f025d529700 1 -- 172.21.15.89:0/1406127298 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02580b1790 con 0x7f025815d230 2023-10-10T20:40:23.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.520+0000 7f025cd28700 1 --2- 172.21.15.89:0/1406127298 >> v2:172.21.15.111:3300/0 conn(0x7f0258159420 0x7f02580b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:23.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.520+0000 7f025d529700 1 --2- 172.21.15.89:0/1406127298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025815d230 0x7f025815d690 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f02580b2f20 tx=0x7f0250009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3392209c2eedf94a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:23.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.521+0000 7f02577fe700 1 -- 172.21.15.89:0/1406127298 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025000e070 con 0x7f025815d230 2023-10-10T20:40:23.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.521+0000 7f02577fe700 1 -- 172.21.15.89:0/1406127298 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025000ba50 con 0x7f025815d230 2023-10-10T20:40:23.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.521+0000 7f02577fe700 1 -- 172.21.15.89:0/1406127298 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02500092e0 con 0x7f025815d230 2023-10-10T20:40:23.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.521+0000 7f025dd2a700 1 -- 172.21.15.89:0/1406127298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025815d230 msgr2=0x7f025815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:23.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.521+0000 7f025dd2a700 1 --2- 172.21.15.89:0/1406127298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025815d230 0x7f025815d690 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f02580b2f20 tx=0x7f0250009840 comp rx=0 tx=0).stop 2023-10-10T20:40:23.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.521+0000 7f025dd2a700 1 -- 172.21.15.89:0/1406127298 shutdown_connections 2023-10-10T20:40:23.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.521+0000 7f025dd2a700 1 --2- 172.21.15.89:0/1406127298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025815d230 0x7f025815d690 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.521+0000 7f025dd2a700 1 --2- 172.21.15.89:0/1406127298 >> v2:172.21.15.111:3300/0 conn(0x7f0258159420 0x7f02580b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.521+0000 7f025dd2a700 1 --2- 172.21.15.89:0/1406127298 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02581597f0 0x7f02580b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.521+0000 7f025dd2a700 1 -- 172.21.15.89:0/1406127298 >> 172.21.15.89:0/1406127298 conn(0x7f02580d0660 msgr2=0x7f02580d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:23.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.521+0000 7f025dd2a700 1 -- 172.21.15.89:0/1406127298 shutdown_connections 2023-10-10T20:40:23.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.521+0000 7f025dd2a700 1 -- 172.21.15.89:0/1406127298 wait complete. 2023-10-10T20:40:23.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.522+0000 7f025dd2a700 1 Processor -- start 2023-10-10T20:40:23.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.523+0000 7f025dd2a700 1 -- start start 2023-10-10T20:40:23.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.523+0000 7f025dd2a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0258159420 0x7f02580c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:23.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.523+0000 7f025dd2a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02581597f0 0x7f02580c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:23.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.523+0000 7f025dd2a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f025815d230 0x7f02581545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:23.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.523+0000 7f025dd2a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02580c6600 con 0x7f0258159420 2023-10-10T20:40:23.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.523+0000 7f025dd2a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02580c6330 con 0x7f025815d230 2023-10-10T20:40:23.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.523+0000 7f025dd2a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025815b990 con 0x7f02581597f0 2023-10-10T20:40:23.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.523+0000 7f025cd28700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0258159420 0x7f02580c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:23.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.523+0000 7f025cd28700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0258159420 0x7f02580c5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56676/0 (socket says 172.21.15.89:56676) 2023-10-10T20:40:23.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.523+0000 7f025cd28700 1 -- 172.21.15.89:0/2736956948 learned_addr learned my addr 172.21.15.89:0/2736956948 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:23.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.523+0000 7f0257fff700 1 --2- 172.21.15.89:0/2736956948 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02581597f0 0x7f02580c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:23.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.524+0000 7f025d529700 1 --2- 172.21.15.89:0/2736956948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f025815d230 0x7f02581545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:23.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.524+0000 7f025cd28700 1 -- 172.21.15.89:0/2736956948 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02581597f0 msgr2=0x7f02580c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:23.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.524+0000 7f025cd28700 1 --2- 172.21.15.89:0/2736956948 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02581597f0 0x7f02580c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.524+0000 7f025cd28700 1 -- 172.21.15.89:0/2736956948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f025815d230 msgr2=0x7f02581545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:23.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.524+0000 7f025cd28700 1 --2- 172.21.15.89:0/2736956948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f025815d230 0x7f02581545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.524+0000 7f025cd28700 1 -- 172.21.15.89:0/2736956948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0250009ae0 con 0x7f0258159420 2023-10-10T20:40:23.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.524+0000 7f025d529700 1 --2- 172.21.15.89:0/2736956948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f025815d230 0x7f02581545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:23.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.524+0000 7f025cd28700 1 --2- 172.21.15.89:0/2736956948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0258159420 0x7f02580c5890 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f024000e8d0 tx=0x7f0240008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:23.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.524+0000 7f0254ff9700 1 -- 172.21.15.89:0/2736956948 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240013410 con 0x7f0258159420 2023-10-10T20:40:23.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.524+0000 7f0254ff9700 1 -- 172.21.15.89:0/2736956948 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0240013570 con 0x7f0258159420 2023-10-10T20:40:23.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.524+0000 7f025dd2a700 1 -- 172.21.15.89:0/2736956948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f025815bc70 con 0x7f0258159420 2023-10-10T20:40:23.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.524+0000 7f0254ff9700 1 -- 172.21.15.89:0/2736956948 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240013910 con 0x7f0258159420 2023-10-10T20:40:23.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.524+0000 7f025dd2a700 1 -- 172.21.15.89:0/2736956948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025815c230 con 0x7f0258159420 2023-10-10T20:40:23.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.525+0000 7f025dd2a700 1 -- 172.21.15.89:0/2736956948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025815c5a0 con 0x7f0258159420 2023-10-10T20:40:23.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.525+0000 7f0254ff9700 1 -- 172.21.15.89:0/2736956948 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f0240013d00 con 0x7f0258159420 2023-10-10T20:40:23.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.525+0000 7f0254ff9700 1 --2- 172.21.15.89:0/2736956948 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0238067200 0x7f02380696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:23.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.526+0000 7f0254ff9700 1 -- 172.21.15.89:0/2736956948 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0240084800 con 0x7f0258159420 2023-10-10T20:40:23.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.526+0000 7f0257fff700 1 --2- 172.21.15.89:0/2736956948 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0238067200 0x7f02380696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:23.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.526+0000 7f0257fff700 1 --2- 172.21.15.89:0/2736956948 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0238067200 0x7f02380696b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0248006fd0 tx=0x7f0248008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:23.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.530+0000 7f0254ff9700 1 -- 172.21.15.89:0/2736956948 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02400550a0 con 0x7f0258159420 2023-10-10T20:40:23.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.757+0000 7f025dd2a700 1 -- 172.21.15.89:0/2736956948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f025804c3c0 con 0x7f0258159420 2023-10-10T20:40:23.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.757+0000 7f0254ff9700 1 -- 172.21.15.89:0/2736956948 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f0240054c30 con 0x7f0258159420 2023-10-10T20:40:23.760 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:23.760 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5995,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/3194805494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3194805494},{"type":"v1","addr":"172.21.15.111:6834","nonce":3194805494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6447,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/2766962252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2766962252},{"type":"v1","addr":"172.21.15.89:6836","nonce":2766962252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6450,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1141782720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1141782720},{"type":"v1","addr":"172.21.15.89:6837","nonce":1141782720}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:40:08.387849+0000","modified":"2023-10-10T20:40:12.040215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6462":{"gid":6462,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/4069200228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":4069200228},{"type":"v1","addr":"172.21.15.111:6835","nonce":4069200228}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-10T20:40:23.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.759+0000 7f025dd2a700 1 -- 172.21.15.89:0/2736956948 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0238067200 msgr2=0x7f02380696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:23.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.759+0000 7f025dd2a700 1 --2- 172.21.15.89:0/2736956948 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0238067200 0x7f02380696b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0248006fd0 tx=0x7f0248008040 comp rx=0 tx=0).stop 2023-10-10T20:40:23.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.759+0000 7f025dd2a700 1 -- 172.21.15.89:0/2736956948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0258159420 msgr2=0x7f02580c5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:23.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.759+0000 7f025dd2a700 1 --2- 172.21.15.89:0/2736956948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0258159420 0x7f02580c5890 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f024000e8d0 tx=0x7f0240008040 comp rx=0 tx=0).stop 2023-10-10T20:40:23.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.759+0000 7f025dd2a700 1 -- 172.21.15.89:0/2736956948 shutdown_connections 2023-10-10T20:40:23.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.759+0000 7f025dd2a700 1 --2- 172.21.15.89:0/2736956948 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0238067200 0x7f02380696b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.759+0000 7f025dd2a700 1 --2- 172.21.15.89:0/2736956948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f025815d230 0x7f02581545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.759+0000 7f025dd2a700 1 --2- 172.21.15.89:0/2736956948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0258159420 0x7f02580c5890 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.759+0000 7f025dd2a700 1 --2- 172.21.15.89:0/2736956948 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02581597f0 0x7f02580c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.759+0000 7f025dd2a700 1 -- 172.21.15.89:0/2736956948 >> 172.21.15.89:0/2736956948 conn(0x7f02580d0660 msgr2=0x7f0258157190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:23.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.759+0000 7f025dd2a700 1 -- 172.21.15.89:0/2736956948 shutdown_connections 2023-10-10T20:40:23.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.759+0000 7f025dd2a700 1 -- 172.21.15.89:0/2736956948 wait complete. 2023-10-10T20:40:23.764 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 56 2023-10-10T20:40:23.774 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:40:23.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.887+0000 7f84cc8bb700 1 Processor -- start 2023-10-10T20:40:23.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.887+0000 7f84cc8bb700 1 -- start start 2023-10-10T20:40:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84cc8bb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84c8159420 0x7f84c80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84cc8bb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f84c81597f0 0x7f84c80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84cc8bb700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f84c815d230 0x7f84c815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84cc8bb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c815dbd0 con 0x7f84c81597f0 2023-10-10T20:40:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84cc8bb700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c80b11d0 con 0x7f84c815d230 2023-10-10T20:40:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84cc8bb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80b14b0 con 0x7f84c8159420 2023-10-10T20:40:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84c659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f84c81597f0 0x7f84c80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84c659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f84c81597f0 0x7f84c80b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56686/0 (socket says 172.21.15.89:56686) 2023-10-10T20:40:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84c659c700 1 -- 172.21.15.89:0/3341176185 learned_addr learned my addr 172.21.15.89:0/3341176185 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84c6d9d700 1 --2- 172.21.15.89:0/3341176185 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84c8159420 0x7f84c80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84c659c700 1 -- 172.21.15.89:0/3341176185 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84c8159420 msgr2=0x7f84c80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84c759e700 1 --2- 172.21.15.89:0/3341176185 >> v2:172.21.15.111:3300/0 conn(0x7f84c815d230 0x7f84c815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84c659c700 1 --2- 172.21.15.89:0/3341176185 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84c8159420 0x7f84c80b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84c659c700 1 -- 172.21.15.89:0/3341176185 >> v2:172.21.15.111:3300/0 conn(0x7f84c815d230 msgr2=0x7f84c815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84c659c700 1 --2- 172.21.15.89:0/3341176185 >> v2:172.21.15.111:3300/0 conn(0x7f84c815d230 0x7f84c815d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84c659c700 1 -- 172.21.15.89:0/3341176185 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c80b1790 con 0x7f84c81597f0 2023-10-10T20:40:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84c6d9d700 1 --2- 172.21.15.89:0/3341176185 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84c8159420 0x7f84c80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84c759e700 1 --2- 172.21.15.89:0/3341176185 >> v2:172.21.15.111:3300/0 conn(0x7f84c815d230 0x7f84c815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84c659c700 1 --2- 172.21.15.89:0/3341176185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84c81597f0 0x7f84c80b2270 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f84c80d1720 tx=0x7f84b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40598f3bee30224d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84c5d9b700 1 -- 172.21.15.89:0/3341176185 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b000b860 con 0x7f84c81597f0 2023-10-10T20:40:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84c5d9b700 1 -- 172.21.15.89:0/3341176185 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b000b9c0 con 0x7f84c81597f0 2023-10-10T20:40:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.888+0000 7f84c5d9b700 1 -- 172.21.15.89:0/3341176185 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b000bd60 con 0x7f84c81597f0 2023-10-10T20:40:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.889+0000 7f84cc8bb700 1 -- 172.21.15.89:0/3341176185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84c81597f0 msgr2=0x7f84c80b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.889+0000 7f84cc8bb700 1 --2- 172.21.15.89:0/3341176185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84c81597f0 0x7f84c80b2270 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f84c80d1720 tx=0x7f84b0009250 comp rx=0 tx=0).stop 2023-10-10T20:40:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.889+0000 7f84cc8bb700 1 -- 172.21.15.89:0/3341176185 shutdown_connections 2023-10-10T20:40:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.889+0000 7f84cc8bb700 1 --2- 172.21.15.89:0/3341176185 >> v2:172.21.15.111:3300/0 conn(0x7f84c815d230 0x7f84c815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.889+0000 7f84cc8bb700 1 --2- 172.21.15.89:0/3341176185 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84c8159420 0x7f84c80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.889+0000 7f84cc8bb700 1 --2- 172.21.15.89:0/3341176185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84c81597f0 0x7f84c80b2270 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.889+0000 7f84cc8bb700 1 -- 172.21.15.89:0/3341176185 >> 172.21.15.89:0/3341176185 conn(0x7f84c80d0660 msgr2=0x7f84c80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.889+0000 7f84cc8bb700 1 -- 172.21.15.89:0/3341176185 shutdown_connections 2023-10-10T20:40:23.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.889+0000 7f84cc8bb700 1 -- 172.21.15.89:0/3341176185 wait complete. 2023-10-10T20:40:23.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.890+0000 7f84cc8bb700 1 Processor -- start 2023-10-10T20:40:23.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.890+0000 7f84cc8bb700 1 -- start start 2023-10-10T20:40:23.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.891+0000 7f84cc8bb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84c8159420 0x7f84c8169e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:23.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.891+0000 7f84cc8bb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84c81597f0 0x7f84c80b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:23.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.891+0000 7f84cc8bb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84c815d230 0x7f84c80b4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:23.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.891+0000 7f84cc8bb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c8151810 con 0x7f84c815d230 2023-10-10T20:40:23.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.891+0000 7f84cc8bb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c816a950 con 0x7f84c8159420 2023-10-10T20:40:23.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.891+0000 7f84cc8bb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c816ac30 con 0x7f84c81597f0 2023-10-10T20:40:23.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.891+0000 7f84c759e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84c815d230 0x7f84c80b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:23.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.891+0000 7f84c759e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84c815d230 0x7f84c80b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56690/0 (socket says 172.21.15.89:56690) 2023-10-10T20:40:23.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.891+0000 7f84c759e700 1 -- 172.21.15.89:0/2670750289 learned_addr learned my addr 172.21.15.89:0/2670750289 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:23.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.891+0000 7f84c6d9d700 1 --2- 172.21.15.89:0/2670750289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84c8159420 0x7f84c8169e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:23.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.891+0000 7f84c659c700 1 --2- 172.21.15.89:0/2670750289 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84c81597f0 0x7f84c80b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:23.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.891+0000 7f84c759e700 1 -- 172.21.15.89:0/2670750289 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84c81597f0 msgr2=0x7f84c80b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:23.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.891+0000 7f84c759e700 1 --2- 172.21.15.89:0/2670750289 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84c81597f0 0x7f84c80b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.891+0000 7f84c759e700 1 -- 172.21.15.89:0/2670750289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84c8159420 msgr2=0x7f84c8169e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.891+0000 7f84c759e700 1 --2- 172.21.15.89:0/2670750289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84c8159420 0x7f84c8169e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.891+0000 7f84c759e700 1 -- 172.21.15.89:0/2670750289 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b00094f0 con 0x7f84c815d230 2023-10-10T20:40:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.891+0000 7f84c6d9d700 1 --2- 172.21.15.89:0/2670750289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84c8159420 0x7f84c8169e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.892+0000 7f84c759e700 1 --2- 172.21.15.89:0/2670750289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84c815d230 0x7f84c80b4970 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f84b800c850 tx=0x7f84b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.892+0000 7f84beffd700 1 -- 172.21.15.89:0/2670750289 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8011600 con 0x7f84c815d230 2023-10-10T20:40:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.892+0000 7f84beffd700 1 -- 172.21.15.89:0/2670750289 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b8011760 con 0x7f84c815d230 2023-10-10T20:40:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.892+0000 7f84cc8bb700 1 -- 172.21.15.89:0/2670750289 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c816af10 con 0x7f84c815d230 2023-10-10T20:40:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.892+0000 7f84beffd700 1 -- 172.21.15.89:0/2670750289 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8011b00 con 0x7f84c815d230 2023-10-10T20:40:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.892+0000 7f84cc8bb700 1 -- 172.21.15.89:0/2670750289 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c815cc80 con 0x7f84c815d230 2023-10-10T20:40:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.893+0000 7f84cc8bb700 1 -- 172.21.15.89:0/2670750289 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c8151980 con 0x7f84c815d230 2023-10-10T20:40:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.893+0000 7f84beffd700 1 -- 172.21.15.89:0/2670750289 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f84b8018350 con 0x7f84c815d230 2023-10-10T20:40:23.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.893+0000 7f84beffd700 1 --2- 172.21.15.89:0/2670750289 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84a8067200 0x7f84a80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:23.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.894+0000 7f84beffd700 1 -- 172.21.15.89:0/2670750289 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f84b8084900 con 0x7f84c815d230 2023-10-10T20:40:23.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.894+0000 7f84c6d9d700 1 --2- 172.21.15.89:0/2670750289 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84a8067200 0x7f84a80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:23.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.894+0000 7f84c6d9d700 1 --2- 172.21.15.89:0/2670750289 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84a8067200 0x7f84a80696b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f84c000e650 tx=0x7f84c0007710 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:23.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:23.898+0000 7f84beffd700 1 -- 172.21.15.89:0/2670750289 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84b8051710 con 0x7f84c815d230 2023-10-10T20:40:24.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.088+0000 7f84cc8bb700 1 -- 172.21.15.89:0/2670750289 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84c804c3c0 con 0x7f84c815d230 2023-10-10T20:40:24.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.088+0000 7f84beffd700 1 -- 172.21.15.89:0/2670750289 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+15353 (secure 0 0 0) 0x7f84b801e090 con 0x7f84c815d230 2023-10-10T20:40:24.091 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:24.091 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":88,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:40:10.022096+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:40:06.747813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-10T20:40:07.347913+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:40:24.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.090+0000 7f84cc8bb700 1 -- 172.21.15.89:0/2670750289 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84a8067200 msgr2=0x7f84a80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:24.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.090+0000 7f84cc8bb700 1 --2- 172.21.15.89:0/2670750289 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84a8067200 0x7f84a80696b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f84c000e650 tx=0x7f84c0007710 comp rx=0 tx=0).stop 2023-10-10T20:40:24.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.090+0000 7f84cc8bb700 1 -- 172.21.15.89:0/2670750289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84c815d230 msgr2=0x7f84c80b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:24.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.091+0000 7f84cc8bb700 1 --2- 172.21.15.89:0/2670750289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84c815d230 0x7f84c80b4970 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f84b800c850 tx=0x7f84b8008040 comp rx=0 tx=0).stop 2023-10-10T20:40:24.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.091+0000 7f84cc8bb700 1 -- 172.21.15.89:0/2670750289 shutdown_connections 2023-10-10T20:40:24.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.091+0000 7f84cc8bb700 1 --2- 172.21.15.89:0/2670750289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84c815d230 0x7f84c80b4970 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.091+0000 7f84cc8bb700 1 --2- 172.21.15.89:0/2670750289 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84a8067200 0x7f84a80696b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.091+0000 7f84cc8bb700 1 --2- 172.21.15.89:0/2670750289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84c8159420 0x7f84c8169e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.091+0000 7f84cc8bb700 1 --2- 172.21.15.89:0/2670750289 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84c81597f0 0x7f84c80b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.091+0000 7f84cc8bb700 1 -- 172.21.15.89:0/2670750289 >> 172.21.15.89:0/2670750289 conn(0x7f84c80d0660 msgr2=0x7f84c8069850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:24.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.091+0000 7f84cc8bb700 1 -- 172.21.15.89:0/2670750289 shutdown_connections 2023-10-10T20:40:24.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.091+0000 7f84cc8bb700 1 -- 172.21.15.89:0/2670750289 wait complete. 2023-10-10T20:40:24.105 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:40:06.747813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:40:07.347913+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:40:24.106 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:40:24.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.221+0000 7f3e4c45d700 1 Processor -- start 2023-10-10T20:40:24.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.221+0000 7f3e4c45d700 1 -- start start 2023-10-10T20:40:24.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.222+0000 7f3e4c45d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e44159890 0x7f3e4415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.222+0000 7f3e4c45d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3e44069560 0x7f3e440699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.222+0000 7f3e4c45d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e4415a160 con 0x7f3e44159c60 2023-10-10T20:40:24.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.222+0000 7f3e4c45d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e44069f20 con 0x7f3e44069560 2023-10-10T20:40:24.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.222+0000 7f3e4c45d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e440d13a0 con 0x7f3e44159890 2023-10-10T20:40:24.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.222+0000 7f3e499f8700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3e44159c60 0x7f3e4415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36444/0 (socket says 172.21.15.89:36444) 2023-10-10T20:40:24.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.222+0000 7f3e499f8700 1 -- 172.21.15.89:0/1624864497 learned_addr learned my addr 172.21.15.89:0/1624864497 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:24.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.222+0000 7f3e4a1f9700 1 --2- 172.21.15.89:0/1624864497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e44159890 0x7f3e4415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.222+0000 7f3e4a9fa700 1 --2- 172.21.15.89:0/1624864497 >> v2:172.21.15.111:3300/0 conn(0x7f3e44069560 0x7f3e440699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.222+0000 7f3e491f7700 1 -- 172.21.15.89:0/1624864497 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884655750 0 0) 0x7f3e4415a160 con 0x7f3e44159c60 2023-10-10T20:40:24.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.222+0000 7f3e491f7700 1 -- 172.21.15.89:0/1624864497 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e30003570 con 0x7f3e44159c60 2023-10-10T20:40:24.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e491f7700 1 -- 172.21.15.89:0/1624864497 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2565640601 0 0) 0x7f3e30003570 con 0x7f3e44159c60 2023-10-10T20:40:24.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e491f7700 1 -- 172.21.15.89:0/1624864497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e44159890 msgr2=0x7f3e4415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:24.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e491f7700 1 --2- 172.21.15.89:0/1624864497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e44159890 0x7f3e4415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e491f7700 1 -- 172.21.15.89:0/1624864497 >> v2:172.21.15.111:3300/0 conn(0x7f3e44069560 msgr2=0x7f3e440699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:24.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e491f7700 1 --2- 172.21.15.89:0/1624864497 >> v2:172.21.15.111:3300/0 conn(0x7f3e44069560 0x7f3e440699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e491f7700 1 -- 172.21.15.89:0/1624864497 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4415e140 con 0x7f3e44159c60 2023-10-10T20:40:24.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e491f7700 1 -- 172.21.15.89:0/1624864497 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f3e34001470 con 0x7f3e44159c60 2023-10-10T20:40:24.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e491f7700 1 -- 172.21.15.89:0/1624864497 >> v1:172.21.15.89:6789/0 conn(0x7f3e44159c60 legacy=0x7f3e4415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:24.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e491f7700 1 --2- 172.21.15.89:0/1624864497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e30003a60 0x7f3e30003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e491f7700 1 --2- 172.21.15.89:0/1624864497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e30004810 0x7f3e30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e491f7700 1 --2- 172.21.15.89:0/1624864497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e300051c0 0x7f3e3000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e491f7700 1 -- 172.21.15.89:0/1624864497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e44069f20 con 0x7f3e30004810 2023-10-10T20:40:24.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e491f7700 1 -- 172.21.15.89:0/1624864497 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e440d13a0 con 0x7f3e300051c0 2023-10-10T20:40:24.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e491f7700 1 -- 172.21.15.89:0/1624864497 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3000bfa0 con 0x7f3e30003a60 2023-10-10T20:40:24.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e4a9fa700 1 --2- 172.21.15.89:0/1624864497 >> v2:172.21.15.111:3300/0 conn(0x7f3e44069560 0x7f3e440699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:40:24.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e4a1f9700 1 --2- 172.21.15.89:0/1624864497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e44159890 0x7f3e4415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:40:24.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e499f8700 1 --2- 172.21.15.89:0/1624864497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e30004810 0x7f3e30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e4a1f9700 1 --2- 172.21.15.89:0/1624864497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e30003a60 0x7f3e30003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e4a9fa700 1 --2- 172.21.15.89:0/1624864497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e300051c0 0x7f3e3000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e499f8700 1 -- 172.21.15.89:0/1624864497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e30003a60 msgr2=0x7f3e30003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:24.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e499f8700 1 --2- 172.21.15.89:0/1624864497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e30003a60 0x7f3e30003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e499f8700 1 -- 172.21.15.89:0/1624864497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e300051c0 msgr2=0x7f3e3000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e499f8700 1 --2- 172.21.15.89:0/1624864497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e300051c0 0x7f3e3000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e499f8700 1 -- 172.21.15.89:0/1624864497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e30003570 con 0x7f3e30004810 2023-10-10T20:40:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e4a1f9700 1 --2- 172.21.15.89:0/1624864497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e30003a60 0x7f3e30003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e499f8700 1 --2- 172.21.15.89:0/1624864497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e30004810 0x7f3e30004c80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400c390 tx=0x7f3e34001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f746c032aed656e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:24.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e491f7700 1 -- 172.21.15.89:0/1624864497 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3400cc60 con 0x7f3e30004810 2023-10-10T20:40:24.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e491f7700 1 -- 172.21.15.89:0/1624864497 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3400ce00 con 0x7f3e30004810 2023-10-10T20:40:24.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.223+0000 7f3e491f7700 1 -- 172.21.15.89:0/1624864497 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34005b70 con 0x7f3e30004810 2023-10-10T20:40:24.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.224+0000 7f3e4c45d700 1 -- 172.21.15.89:0/1624864497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e30004810 msgr2=0x7f3e30004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:24.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.224+0000 7f3e4c45d700 1 --2- 172.21.15.89:0/1624864497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e30004810 0x7f3e30004c80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400c390 tx=0x7f3e34001e90 comp rx=0 tx=0).stop 2023-10-10T20:40:24.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.224+0000 7f3e4a9fa700 1 -- 172.21.15.89:0/1624864497 reap_dead start 2023-10-10T20:40:24.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.224+0000 7f3e4c45d700 1 -- 172.21.15.89:0/1624864497 shutdown_connections 2023-10-10T20:40:24.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.224+0000 7f3e4c45d700 1 -- 172.21.15.89:0/1624864497 >> 172.21.15.89:0/1624864497 conn(0x7f3e44068e20 msgr2=0x7f3e440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:24.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.224+0000 7f3e4c45d700 1 -- 172.21.15.89:0/1624864497 shutdown_connections 2023-10-10T20:40:24.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.224+0000 7f3e4c45d700 1 -- 172.21.15.89:0/1624864497 wait complete. 2023-10-10T20:40:24.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.225+0000 7f3e4c45d700 1 Processor -- start 2023-10-10T20:40:24.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.225+0000 7f3e4c45d700 1 -- start start 2023-10-10T20:40:24.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.226+0000 7f3e4c45d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e30004810 0x7f3e440cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.226+0000 7f3e4c45d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e440c78a0 0x7f3e440c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.226+0000 7f3e4c45d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e440ca0d0 0x7f3e440ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.226+0000 7f3e4c45d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e440c8250 con 0x7f3e440c78a0 2023-10-10T20:40:24.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.226+0000 7f3e4c45d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e440cebe0 con 0x7f3e440ca0d0 2023-10-10T20:40:24.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.226+0000 7f3e4c45d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e440cdd00 con 0x7f3e30004810 2023-10-10T20:40:24.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.226+0000 7f3e499f8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e440c78a0 0x7f3e440c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.226+0000 7f3e499f8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e440c78a0 0x7f3e440c7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56702/0 (socket says 172.21.15.89:56702) 2023-10-10T20:40:24.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.226+0000 7f3e499f8700 1 -- 172.21.15.89:0/1950197652 learned_addr learned my addr 172.21.15.89:0/1950197652 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:24.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.226+0000 7f3e4a1f9700 1 --2- 172.21.15.89:0/1950197652 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e30004810 0x7f3e440cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.226+0000 7f3e4a9fa700 1 --2- 172.21.15.89:0/1950197652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e440ca0d0 0x7f3e440ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.226+0000 7f3e499f8700 1 -- 172.21.15.89:0/1950197652 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e30004810 msgr2=0x7f3e440cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:24.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.226+0000 7f3e499f8700 1 --2- 172.21.15.89:0/1950197652 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e30004810 0x7f3e440cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.226+0000 7f3e499f8700 1 -- 172.21.15.89:0/1950197652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e440ca0d0 msgr2=0x7f3e440ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:24.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.226+0000 7f3e499f8700 1 --2- 172.21.15.89:0/1950197652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e440ca0d0 0x7f3e440ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.226+0000 7f3e499f8700 1 -- 172.21.15.89:0/1950197652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e440cdfd0 con 0x7f3e440c78a0 2023-10-10T20:40:24.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.226+0000 7f3e499f8700 1 --2- 172.21.15.89:0/1950197652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e440c78a0 0x7f3e440c7d10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3e30004650 tx=0x7f3e34015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:24.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.227+0000 7f3e3a7fc700 1 -- 172.21.15.89:0/1950197652 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e340073b0 con 0x7f3e440c78a0 2023-10-10T20:40:24.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.227+0000 7f3e3a7fc700 1 -- 172.21.15.89:0/1950197652 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e34007510 con 0x7f3e440c78a0 2023-10-10T20:40:24.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.227+0000 7f3e4c45d700 1 -- 172.21.15.89:0/1950197652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e440ce2b0 con 0x7f3e440c78a0 2023-10-10T20:40:24.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.227+0000 7f3e3a7fc700 1 -- 172.21.15.89:0/1950197652 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34007890 con 0x7f3e440c78a0 2023-10-10T20:40:24.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.227+0000 7f3e4c45d700 1 -- 172.21.15.89:0/1950197652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e440ce870 con 0x7f3e440c78a0 2023-10-10T20:40:24.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.227+0000 7f3e4c45d700 1 -- 172.21.15.89:0/1950197652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e44069960 con 0x7f3e440c78a0 2023-10-10T20:40:24.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.228+0000 7f3e3a7fc700 1 -- 172.21.15.89:0/1950197652 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f3e34019070 con 0x7f3e440c78a0 2023-10-10T20:40:24.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.228+0000 7f3e3a7fc700 1 --2- 172.21.15.89:0/1950197652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3e28067200 0x7f3e280696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.228+0000 7f3e3a7fc700 1 -- 172.21.15.89:0/1950197652 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3e34085af0 con 0x7f3e440c78a0 2023-10-10T20:40:24.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.228+0000 7f3e4a1f9700 1 --2- 172.21.15.89:0/1950197652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3e28067200 0x7f3e280696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.229+0000 7f3e4a1f9700 1 --2- 172.21.15.89:0/1950197652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3e28067200 0x7f3e280696b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009fd0 tx=0x7f3e4000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:24.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.232+0000 7f3e3a7fc700 1 -- 172.21.15.89:0/1950197652 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e34052a10 con 0x7f3e440c78a0 2023-10-10T20:40:24.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.414+0000 7f3e4c45d700 1 -- 172.21.15.89:0/1950197652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e440cc000 con 0x7f3e440c78a0 2023-10-10T20:40:24.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.414+0000 7f3e3a7fc700 1 -- 172.21.15.89:0/1950197652 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e34056030 con 0x7f3e440c78a0 2023-10-10T20:40:24.417 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:40:24.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.416+0000 7f3e4c45d700 1 -- 172.21.15.89:0/1950197652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3e28067200 msgr2=0x7f3e280696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:24.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.416+0000 7f3e4c45d700 1 --2- 172.21.15.89:0/1950197652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3e28067200 0x7f3e280696b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009fd0 tx=0x7f3e4000b040 comp rx=0 tx=0).stop 2023-10-10T20:40:24.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.416+0000 7f3e4c45d700 1 -- 172.21.15.89:0/1950197652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e440c78a0 msgr2=0x7f3e440c7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:24.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.416+0000 7f3e4c45d700 1 --2- 172.21.15.89:0/1950197652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e440c78a0 0x7f3e440c7d10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3e30004650 tx=0x7f3e34015000 comp rx=0 tx=0).stop 2023-10-10T20:40:24.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.416+0000 7f3e4c45d700 1 -- 172.21.15.89:0/1950197652 shutdown_connections 2023-10-10T20:40:24.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.416+0000 7f3e4c45d700 1 --2- 172.21.15.89:0/1950197652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3e28067200 0x7f3e280696b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.416+0000 7f3e4c45d700 1 --2- 172.21.15.89:0/1950197652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e440ca0d0 0x7f3e440ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.416+0000 7f3e4c45d700 1 --2- 172.21.15.89:0/1950197652 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e30004810 0x7f3e440cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.416+0000 7f3e4c45d700 1 --2- 172.21.15.89:0/1950197652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e440c78a0 0x7f3e440c7d10 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.416+0000 7f3e4c45d700 1 -- 172.21.15.89:0/1950197652 >> 172.21.15.89:0/1950197652 conn(0x7f3e44068e20 msgr2=0x7f3e44152da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:24.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.416+0000 7f3e4c45d700 1 -- 172.21.15.89:0/1950197652 shutdown_connections 2023-10-10T20:40:24.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.416+0000 7f3e4c45d700 1 -- 172.21.15.89:0/1950197652 wait complete. 2023-10-10T20:40:24.431 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:40:24.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.548+0000 7ff6fa4aa700 1 Processor -- start 2023-10-10T20:40:24.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.548+0000 7ff6fa4aa700 1 -- start start 2023-10-10T20:40:24.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6fa4aa700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6f4159890 0x7ff6f415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6fa4aa700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f415a160 con 0x7ff6f4159c60 2023-10-10T20:40:24.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6fa4aa700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f40d13a0 con 0x7ff6f4069560 2023-10-10T20:40:24.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6fa4aa700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f415e140 con 0x7ff6f4159890 2023-10-10T20:40:24.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6f37fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff6f4159c60 0x7ff6f415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36452/0 (socket says 172.21.15.89:36452) 2023-10-10T20:40:24.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6f37fe700 1 -- 172.21.15.89:0/364925530 learned_addr learned my addr 172.21.15.89:0/364925530 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:24.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6f3fff700 1 --2- 172.21.15.89:0/364925530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6f4159890 0x7ff6f415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6f2ffd700 1 -- 172.21.15.89:0/364925530 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 272957502 0 0) 0x7ff6f415a160 con 0x7ff6f4159c60 2023-10-10T20:40:24.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6f2ffd700 1 -- 172.21.15.89:0/364925530 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6dc003570 con 0x7ff6f4159c60 2023-10-10T20:40:24.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6f2ffd700 1 -- 172.21.15.89:0/364925530 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3987696551 0 0) 0x7ff6dc003570 con 0x7ff6f4159c60 2023-10-10T20:40:24.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6f2ffd700 1 -- 172.21.15.89:0/364925530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6f4159890 msgr2=0x7ff6f415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:24.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6f2ffd700 1 --2- 172.21.15.89:0/364925530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6f4159890 0x7ff6f415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6f2ffd700 1 -- 172.21.15.89:0/364925530 >> v1:172.21.15.111:6789/0 conn(0x7ff6f4069560 legacy=0x7ff6f40ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:24.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6f2ffd700 1 -- 172.21.15.89:0/364925530 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f415a3d0 con 0x7ff6f4159c60 2023-10-10T20:40:24.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6f2ffd700 1 -- 172.21.15.89:0/364925530 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7ff6e8001470 con 0x7ff6f4159c60 2023-10-10T20:40:24.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6f2ffd700 1 -- 172.21.15.89:0/364925530 >> v1:172.21.15.89:6789/0 conn(0x7ff6f4159c60 legacy=0x7ff6f415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:24.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6f2ffd700 1 --2- 172.21.15.89:0/364925530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6f4159890 0x7ff6dc003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6f2ffd700 1 --2- 172.21.15.89:0/364925530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6dc003fd0 0x7ff6dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6f2ffd700 1 --2- 172.21.15.89:0/364925530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6dc004d50 0x7ff6dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6f2ffd700 1 -- 172.21.15.89:0/364925530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f415e140 con 0x7ff6dc004d50 2023-10-10T20:40:24.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6f2ffd700 1 -- 172.21.15.89:0/364925530 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc00bb40 con 0x7ff6dc003fd0 2023-10-10T20:40:24.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6f2ffd700 1 -- 172.21.15.89:0/364925530 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc00bd00 con 0x7ff6f4159890 2023-10-10T20:40:24.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.549+0000 7ff6f8a47700 1 --2- 172.21.15.89:0/364925530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6dc004d50 0x7ff6dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.550+0000 7ff6f3fff700 1 --2- 172.21.15.89:0/364925530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6f4159890 0x7ff6dc003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.550+0000 7ff6f37fe700 1 --2- 172.21.15.89:0/364925530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6dc003fd0 0x7ff6dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.550+0000 7ff6f8a47700 1 -- 172.21.15.89:0/364925530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6f4159890 msgr2=0x7ff6dc003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:24.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.550+0000 7ff6f8a47700 1 --2- 172.21.15.89:0/364925530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6f4159890 0x7ff6dc003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.550+0000 7ff6f8a47700 1 -- 172.21.15.89:0/364925530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6dc003fd0 msgr2=0x7ff6dc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:24.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.550+0000 7ff6f8a47700 1 --2- 172.21.15.89:0/364925530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6dc003fd0 0x7ff6dc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.550+0000 7ff6f8a47700 1 -- 172.21.15.89:0/364925530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6dc003570 con 0x7ff6dc004d50 2023-10-10T20:40:24.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.550+0000 7ff6f8a47700 1 --2- 172.21.15.89:0/364925530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6dc004d50 0x7ff6dc0051c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0020a0 tx=0x7ff6ec0016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51997da121dcdae server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:24.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.550+0000 7ff6f2ffd700 1 -- 172.21.15.89:0/364925530 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec0019e0 con 0x7ff6dc004d50 2023-10-10T20:40:24.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.550+0000 7ff6f2ffd700 1 -- 172.21.15.89:0/364925530 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ec00c8b0 con 0x7ff6dc004d50 2023-10-10T20:40:24.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.550+0000 7ff6f2ffd700 1 -- 172.21.15.89:0/364925530 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec00cc50 con 0x7ff6dc004d50 2023-10-10T20:40:24.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.551+0000 7ff6fa4aa700 1 -- 172.21.15.89:0/364925530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6dc004d50 msgr2=0x7ff6dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:24.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.551+0000 7ff6fa4aa700 1 --2- 172.21.15.89:0/364925530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6dc004d50 0x7ff6dc0051c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0020a0 tx=0x7ff6ec0016d0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.551+0000 7ff6f8a47700 1 -- 172.21.15.89:0/364925530 reap_dead start 2023-10-10T20:40:24.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.551+0000 7ff6fa4aa700 1 -- 172.21.15.89:0/364925530 shutdown_connections 2023-10-10T20:40:24.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.551+0000 7ff6fa4aa700 1 -- 172.21.15.89:0/364925530 >> 172.21.15.89:0/364925530 conn(0x7ff6f4068e20 msgr2=0x7ff6f40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:24.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.551+0000 7ff6fa4aa700 1 -- 172.21.15.89:0/364925530 shutdown_connections 2023-10-10T20:40:24.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.551+0000 7ff6fa4aa700 1 -- 172.21.15.89:0/364925530 wait complete. 2023-10-10T20:40:24.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.552+0000 7ff6fa4aa700 1 Processor -- start 2023-10-10T20:40:24.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.552+0000 7ff6fa4aa700 1 -- start start 2023-10-10T20:40:24.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6fa4aa700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6f4158070 0x7ff6f40b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6fa4aa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f4158440 0x7ff6f40b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6fa4aa700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6f40b3e30 0x7ff6f40b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6fa4aa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40b63c0 con 0x7ff6f4158440 2023-10-10T20:40:24.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6fa4aa700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40ce480 con 0x7ff6f4158070 2023-10-10T20:40:24.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6fa4aa700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40b4350 con 0x7ff6f40b3e30 2023-10-10T20:40:24.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6f37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f4158440 0x7ff6f40b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6f37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f4158440 0x7ff6f40b5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56726/0 (socket says 172.21.15.89:56726) 2023-10-10T20:40:24.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6f37fe700 1 -- 172.21.15.89:0/2350139008 learned_addr learned my addr 172.21.15.89:0/2350139008 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:24.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6f3fff700 1 --2- 172.21.15.89:0/2350139008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6f4158070 0x7ff6f40b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6f8a47700 1 --2- 172.21.15.89:0/2350139008 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6f40b3e30 0x7ff6f40b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6f37fe700 1 -- 172.21.15.89:0/2350139008 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6f40b3e30 msgr2=0x7ff6f40b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:24.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6f37fe700 1 --2- 172.21.15.89:0/2350139008 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6f40b3e30 0x7ff6f40b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6f37fe700 1 -- 172.21.15.89:0/2350139008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6f4158070 msgr2=0x7ff6f40b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:24.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6f37fe700 1 --2- 172.21.15.89:0/2350139008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6f4158070 0x7ff6f40b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6f37fe700 1 -- 172.21.15.89:0/2350139008 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f40cdb50 con 0x7ff6f4158440 2023-10-10T20:40:24.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6f37fe700 1 --2- 172.21.15.89:0/2350139008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f4158440 0x7ff6f40b5940 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80011e0 tx=0x7ff6e8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:24.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6dbfff700 1 -- 172.21.15.89:0/2350139008 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8018030 con 0x7ff6f4158440 2023-10-10T20:40:24.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6dbfff700 1 -- 172.21.15.89:0/2350139008 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e800f870 con 0x7ff6f4158440 2023-10-10T20:40:24.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6fa4aa700 1 -- 172.21.15.89:0/2350139008 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f40cde30 con 0x7ff6f4158440 2023-10-10T20:40:24.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6dbfff700 1 -- 172.21.15.89:0/2350139008 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e800fc10 con 0x7ff6f4158440 2023-10-10T20:40:24.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.553+0000 7ff6fa4aa700 1 -- 172.21.15.89:0/2350139008 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f41435e0 con 0x7ff6f4158440 2023-10-10T20:40:24.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.554+0000 7ff6fa4aa700 1 -- 172.21.15.89:0/2350139008 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f404c3c0 con 0x7ff6f4158440 2023-10-10T20:40:24.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.554+0000 7ff6dbfff700 1 -- 172.21.15.89:0/2350139008 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7ff6e801b070 con 0x7ff6f4158440 2023-10-10T20:40:24.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.555+0000 7ff6dbfff700 1 --2- 172.21.15.89:0/2350139008 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff6e0067200 0x7ff6e00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.555+0000 7ff6dbfff700 1 -- 172.21.15.89:0/2350139008 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff6e8019070 con 0x7ff6f4158440 2023-10-10T20:40:24.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.555+0000 7ff6f3fff700 1 --2- 172.21.15.89:0/2350139008 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff6e0067200 0x7ff6e00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.555+0000 7ff6f3fff700 1 --2- 172.21.15.89:0/2350139008 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff6e0067200 0x7ff6e00696b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4006f00 tx=0x7ff6e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:24.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.559+0000 7ff6dbfff700 1 -- 172.21.15.89:0/2350139008 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6e8051750 con 0x7ff6f4158440 2023-10-10T20:40:24.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.741+0000 7ff6fa4aa700 1 -- 172.21.15.89:0/2350139008 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff6f4155810 con 0x7ff6f4158440 2023-10-10T20:40:24.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.742+0000 7ff6dbfff700 1 -- 172.21.15.89:0/2350139008 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff6e8054d70 con 0x7ff6f4158440 2023-10-10T20:40:24.744 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:40:24.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.744+0000 7ff6fa4aa700 1 -- 172.21.15.89:0/2350139008 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff6e0067200 msgr2=0x7ff6e00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:24.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.744+0000 7ff6fa4aa700 1 --2- 172.21.15.89:0/2350139008 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff6e0067200 0x7ff6e00696b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4006f00 tx=0x7ff6e4008040 comp rx=0 tx=0).stop 2023-10-10T20:40:24.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.744+0000 7ff6fa4aa700 1 -- 172.21.15.89:0/2350139008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f4158440 msgr2=0x7ff6f40b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:24.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.744+0000 7ff6fa4aa700 1 --2- 172.21.15.89:0/2350139008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f4158440 0x7ff6f40b5940 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80011e0 tx=0x7ff6e8001e70 comp rx=0 tx=0).stop 2023-10-10T20:40:24.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.744+0000 7ff6fa4aa700 1 -- 172.21.15.89:0/2350139008 shutdown_connections 2023-10-10T20:40:24.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.744+0000 7ff6fa4aa700 1 --2- 172.21.15.89:0/2350139008 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6f40b3e30 0x7ff6f40b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.744+0000 7ff6fa4aa700 1 --2- 172.21.15.89:0/2350139008 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff6e0067200 0x7ff6e00696b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.744+0000 7ff6fa4aa700 1 --2- 172.21.15.89:0/2350139008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6f4158070 0x7ff6f40b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.744+0000 7ff6fa4aa700 1 --2- 172.21.15.89:0/2350139008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f4158440 0x7ff6f40b5940 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.744+0000 7ff6fa4aa700 1 -- 172.21.15.89:0/2350139008 >> 172.21.15.89:0/2350139008 conn(0x7ff6f4068e20 msgr2=0x7ff6f415d120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:24.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.744+0000 7ff6fa4aa700 1 -- 172.21.15.89:0/2350139008 shutdown_connections 2023-10-10T20:40:24.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.744+0000 7ff6fa4aa700 1 -- 172.21.15.89:0/2350139008 wait complete. 2023-10-10T20:40:24.758 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:40:24.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.875+0000 7f314df11700 1 Processor -- start 2023-10-10T20:40:24.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.875+0000 7f314df11700 1 -- start start 2023-10-10T20:40:24.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.876+0000 7f314df11700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f314806c0a0 0x7f31480caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.876+0000 7f314df11700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f314815df50 0x7f314815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.876+0000 7f314df11700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314815d7e0 con 0x7f314806bcd0 2023-10-10T20:40:24.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.876+0000 7f314df11700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314815dac0 con 0x7f314815df50 2023-10-10T20:40:24.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.876+0000 7f314df11700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314815aa30 con 0x7f314806c0a0 2023-10-10T20:40:24.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.876+0000 7f3147fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f314815df50 0x7f314815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.876+0000 7f3147fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f314815df50 0x7f314815e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56730/0 (socket says 172.21.15.89:56730) 2023-10-10T20:40:24.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.876+0000 7f3147fff700 1 -- 172.21.15.89:0/3452603639 learned_addr learned my addr 172.21.15.89:0/3452603639 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:24.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.876+0000 7f3146ffd700 1 --2- 172.21.15.89:0/3452603639 >> v2:172.21.15.111:3300/0 conn(0x7f314806c0a0 0x7f31480caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.876+0000 7f3147fff700 1 -- 172.21.15.89:0/3452603639 >> v2:172.21.15.111:3300/0 conn(0x7f314806c0a0 msgr2=0x7f31480caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:24.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.876+0000 7f3147fff700 1 --2- 172.21.15.89:0/3452603639 >> v2:172.21.15.111:3300/0 conn(0x7f314806c0a0 0x7f31480caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.876+0000 7f3147fff700 1 -- 172.21.15.89:0/3452603639 >> v1:172.21.15.111:6789/0 conn(0x7f314806bcd0 legacy=0x7f314815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:24.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.876+0000 7f3147fff700 1 -- 172.21.15.89:0/3452603639 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314815ad10 con 0x7f314815df50 2023-10-10T20:40:24.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.876+0000 7f3147fff700 1 --2- 172.21.15.89:0/3452603639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f314815df50 0x7f314815e3b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f314806b4e0 tx=0x7f3138009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b422d956674cdc78 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:24.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.876+0000 7f31467fc700 1 -- 172.21.15.89:0/3452603639 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313800b820 con 0x7f314815df50 2023-10-10T20:40:24.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.876+0000 7f31467fc700 1 -- 172.21.15.89:0/3452603639 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313800b980 con 0x7f314815df50 2023-10-10T20:40:24.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.876+0000 7f31467fc700 1 -- 172.21.15.89:0/3452603639 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313800bd40 con 0x7f314815df50 2023-10-10T20:40:24.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.877+0000 7f314df11700 1 -- 172.21.15.89:0/3452603639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f314815df50 msgr2=0x7f314815e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:24.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.877+0000 7f314df11700 1 --2- 172.21.15.89:0/3452603639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f314815df50 0x7f314815e3b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f314806b4e0 tx=0x7f3138009250 comp rx=0 tx=0).stop 2023-10-10T20:40:24.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.877+0000 7f314df11700 1 -- 172.21.15.89:0/3452603639 shutdown_connections 2023-10-10T20:40:24.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.877+0000 7f314df11700 1 --2- 172.21.15.89:0/3452603639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f314815df50 0x7f314815e3b0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.877+0000 7f314df11700 1 --2- 172.21.15.89:0/3452603639 >> v2:172.21.15.111:3300/0 conn(0x7f314806c0a0 0x7f31480caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.877+0000 7f314df11700 1 -- 172.21.15.89:0/3452603639 >> 172.21.15.89:0/3452603639 conn(0x7f31480ce600 msgr2=0x7f31480cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:24.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.877+0000 7f314df11700 1 -- 172.21.15.89:0/3452603639 shutdown_connections 2023-10-10T20:40:24.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.877+0000 7f314df11700 1 -- 172.21.15.89:0/3452603639 wait complete. 2023-10-10T20:40:24.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.878+0000 7f314df11700 1 Processor -- start 2023-10-10T20:40:24.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.878+0000 7f314df11700 1 -- start start 2023-10-10T20:40:24.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.879+0000 7f314df11700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f314806bcd0 0x7f31480c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.879+0000 7f314df11700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f314806c0a0 0x7f314815c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.879+0000 7f314df11700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f314815df50 0x7f3148154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.879+0000 7f314df11700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31480b4250 con 0x7f314815df50 2023-10-10T20:40:24.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.879+0000 7f314df11700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31480c5f40 con 0x7f314806bcd0 2023-10-10T20:40:24.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.879+0000 7f314df11700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31480b39b0 con 0x7f314806c0a0 2023-10-10T20:40:24.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.879+0000 7f3147fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f314815df50 0x7f3148154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.879+0000 7f3147fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f314815df50 0x7f3148154630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56734/0 (socket says 172.21.15.89:56734) 2023-10-10T20:40:24.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.879+0000 7f3147fff700 1 -- 172.21.15.89:0/1078650911 learned_addr learned my addr 172.21.15.89:0/1078650911 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:24.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.879+0000 7f31477fe700 1 --2- 172.21.15.89:0/1078650911 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f314806bcd0 0x7f31480c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.879+0000 7f3146ffd700 1 --2- 172.21.15.89:0/1078650911 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f314806c0a0 0x7f314815c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.879+0000 7f3147fff700 1 -- 172.21.15.89:0/1078650911 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f314806c0a0 msgr2=0x7f314815c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:24.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.879+0000 7f3147fff700 1 --2- 172.21.15.89:0/1078650911 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f314806c0a0 0x7f314815c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.879+0000 7f3147fff700 1 -- 172.21.15.89:0/1078650911 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f314806bcd0 msgr2=0x7f31480c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:24.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.879+0000 7f3147fff700 1 --2- 172.21.15.89:0/1078650911 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f314806bcd0 0x7f31480c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:24.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.879+0000 7f3147fff700 1 -- 172.21.15.89:0/1078650911 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31380094f0 con 0x7f314815df50 2023-10-10T20:40:24.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.879+0000 7f3146ffd700 1 --2- 172.21.15.89:0/1078650911 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f314806c0a0 0x7f314815c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:24.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.879+0000 7f3147fff700 1 --2- 172.21.15.89:0/1078650911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f314815df50 0x7f3148154630 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f3138006fd0 tx=0x7f3138016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:24.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.880+0000 7f313f7fe700 1 -- 172.21.15.89:0/1078650911 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313800b820 con 0x7f314815df50 2023-10-10T20:40:24.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.880+0000 7f313f7fe700 1 -- 172.21.15.89:0/1078650911 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3138020070 con 0x7f314815df50 2023-10-10T20:40:24.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.880+0000 7f314df11700 1 -- 172.21.15.89:0/1078650911 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31480b3c90 con 0x7f314815df50 2023-10-10T20:40:24.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.880+0000 7f313f7fe700 1 -- 172.21.15.89:0/1078650911 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313801b680 con 0x7f314815df50 2023-10-10T20:40:24.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.880+0000 7f314df11700 1 -- 172.21.15.89:0/1078650911 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31480c8470 con 0x7f314815df50 2023-10-10T20:40:24.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.881+0000 7f314df11700 1 -- 172.21.15.89:0/1078650911 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f314804c3c0 con 0x7f314815df50 2023-10-10T20:40:24.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.881+0000 7f313f7fe700 1 -- 172.21.15.89:0/1078650911 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 11) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f3138022030 con 0x7f314815df50 2023-10-10T20:40:24.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.881+0000 7f313f7fe700 1 --2- 172.21.15.89:0/1078650911 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3128067200 0x7f31280696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:24.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.882+0000 7f31477fe700 1 --2- 172.21.15.89:0/1078650911 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3128067200 0x7f31280696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:24.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.882+0000 7f313f7fe700 1 -- 172.21.15.89:0/1078650911 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3138086a80 con 0x7f314815df50 2023-10-10T20:40:24.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.882+0000 7f31477fe700 1 --2- 172.21.15.89:0/1078650911 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3128067200 0x7f31280696b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3140009fd0 tx=0x7f3140009600 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:24.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:24.886+0000 7f313f7fe700 1 -- 172.21.15.89:0/1078650911 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3138088e60 con 0x7f314815df50 2023-10-10T20:40:25.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.071+0000 7f314df11700 1 -- 172.21.15.89:0/1078650911 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f314804c3c0 con 0x7f314815df50 2023-10-10T20:40:25.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.071+0000 7f313f7fe700 1 -- 172.21.15.89:0/1078650911 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f31380539a0 con 0x7f314815df50 2023-10-10T20:40:25.074 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-10T20:40:25.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.074+0000 7f314df11700 1 -- 172.21.15.89:0/1078650911 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3128067200 msgr2=0x7f31280696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:25.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.074+0000 7f314df11700 1 --2- 172.21.15.89:0/1078650911 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3128067200 0x7f31280696b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3140009fd0 tx=0x7f3140009600 comp rx=0 tx=0).stop 2023-10-10T20:40:25.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.074+0000 7f314df11700 1 -- 172.21.15.89:0/1078650911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f314815df50 msgr2=0x7f3148154630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:25.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.074+0000 7f314df11700 1 --2- 172.21.15.89:0/1078650911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f314815df50 0x7f3148154630 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f3138006fd0 tx=0x7f3138016000 comp rx=0 tx=0).stop 2023-10-10T20:40:25.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.074+0000 7f314df11700 1 -- 172.21.15.89:0/1078650911 shutdown_connections 2023-10-10T20:40:25.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.074+0000 7f314df11700 1 --2- 172.21.15.89:0/1078650911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f314815df50 0x7f3148154630 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.074+0000 7f314df11700 1 --2- 172.21.15.89:0/1078650911 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3128067200 0x7f31280696b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.074+0000 7f314df11700 1 --2- 172.21.15.89:0/1078650911 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f314806bcd0 0x7f31480c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.074+0000 7f314df11700 1 --2- 172.21.15.89:0/1078650911 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f314806c0a0 0x7f314815c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.074+0000 7f314df11700 1 -- 172.21.15.89:0/1078650911 >> 172.21.15.89:0/1078650911 conn(0x7f31480ce600 msgr2=0x7f314815a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:25.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.074+0000 7f314df11700 1 -- 172.21.15.89:0/1078650911 shutdown_connections 2023-10-10T20:40:25.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.074+0000 7f314df11700 1 -- 172.21.15.89:0/1078650911 wait complete. 2023-10-10T20:40:25.091 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:40:25.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.204+0000 7fb69e735700 1 Processor -- start 2023-10-10T20:40:25.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.204+0000 7fb69e735700 1 -- start start 2023-10-10T20:40:25.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.205+0000 7fb69e735700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb698159c60 0x7fb69815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:25.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.205+0000 7fb69e735700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb69815db10 0x7fb698069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:25.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.205+0000 7fb69e735700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb69815a160 con 0x7fb698159890 2023-10-10T20:40:25.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.205+0000 7fb69e735700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69815df50 con 0x7fb69815db10 2023-10-10T20:40:25.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.205+0000 7fb69e735700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6980d13a0 con 0x7fb698159c60 2023-10-10T20:40:25.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.205+0000 7fb69ccd2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb69815db10 0x7fb698069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:25.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.205+0000 7fb697fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb698159890 0x7fb69815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36456/0 (socket says 172.21.15.89:36456) 2023-10-10T20:40:25.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.205+0000 7fb697fff700 1 -- 172.21.15.89:0/3363240638 learned_addr learned my addr 172.21.15.89:0/3363240638 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:25.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.205+0000 7fb6977fe700 1 --2- 172.21.15.89:0/3363240638 >> v2:172.21.15.111:3300/0 conn(0x7fb698159c60 0x7fb69815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:25.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.205+0000 7fb69ccd2700 1 -- 172.21.15.89:0/3363240638 >> v2:172.21.15.111:3300/0 conn(0x7fb698159c60 msgr2=0x7fb69815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:25.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.205+0000 7fb69ccd2700 1 --2- 172.21.15.89:0/3363240638 >> v2:172.21.15.111:3300/0 conn(0x7fb698159c60 0x7fb69815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.205+0000 7fb69ccd2700 1 -- 172.21.15.89:0/3363240638 >> v1:172.21.15.89:6789/0 conn(0x7fb698159890 legacy=0x7fb69815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:25.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.205+0000 7fb696ffd700 1 -- 172.21.15.89:0/3363240638 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2302679332 0 0) 0x7fb69815a160 con 0x7fb698159890 2023-10-10T20:40:25.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.205+0000 7fb69ccd2700 1 -- 172.21.15.89:0/3363240638 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69815e140 con 0x7fb69815db10 2023-10-10T20:40:25.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.205+0000 7fb6977fe700 1 --2- 172.21.15.89:0/3363240638 >> v2:172.21.15.111:3300/0 conn(0x7fb698159c60 0x7fb69815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:25.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.205+0000 7fb69ccd2700 1 --2- 172.21.15.89:0/3363240638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb69815db10 0x7fb698069560 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb6980b67f0 tx=0x7fb690009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2590899bd53413 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:25.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.205+0000 7fb696ffd700 1 -- 172.21.15.89:0/3363240638 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69000e070 con 0x7fb69815db10 2023-10-10T20:40:25.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.205+0000 7fb696ffd700 1 -- 172.21.15.89:0/3363240638 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69000bc30 con 0x7fb69815db10 2023-10-10T20:40:25.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.205+0000 7fb696ffd700 1 -- 172.21.15.89:0/3363240638 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb690012430 con 0x7fb69815db10 2023-10-10T20:40:25.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.206+0000 7fb69e735700 1 -- 172.21.15.89:0/3363240638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb69815db10 msgr2=0x7fb698069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:25.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.206+0000 7fb69e735700 1 --2- 172.21.15.89:0/3363240638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb69815db10 0x7fb698069560 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb6980b67f0 tx=0x7fb690009780 comp rx=0 tx=0).stop 2023-10-10T20:40:25.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.206+0000 7fb69e735700 1 -- 172.21.15.89:0/3363240638 shutdown_connections 2023-10-10T20:40:25.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.206+0000 7fb69e735700 1 --2- 172.21.15.89:0/3363240638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb69815db10 0x7fb698069560 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.206+0000 7fb69e735700 1 --2- 172.21.15.89:0/3363240638 >> v2:172.21.15.111:3300/0 conn(0x7fb698159c60 0x7fb69815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.206+0000 7fb69e735700 1 -- 172.21.15.89:0/3363240638 >> 172.21.15.89:0/3363240638 conn(0x7fb698068e20 msgr2=0x7fb6980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:25.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.206+0000 7fb69e735700 1 -- 172.21.15.89:0/3363240638 shutdown_connections 2023-10-10T20:40:25.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.206+0000 7fb69e735700 1 -- 172.21.15.89:0/3363240638 wait complete. 2023-10-10T20:40:25.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.207+0000 7fb69e735700 1 Processor -- start 2023-10-10T20:40:25.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.207+0000 7fb69e735700 1 -- start start 2023-10-10T20:40:25.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.208+0000 7fb69e735700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb698159890 0x7fb6981508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:25.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.208+0000 7fb69e735700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb698159c60 0x7fb698154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:25.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.208+0000 7fb69e735700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb69815db10 0x7fb698151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:25.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.208+0000 7fb69e735700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb698153d20 con 0x7fb698159890 2023-10-10T20:40:25.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.208+0000 7fb69e735700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb698154b40 con 0x7fb698159c60 2023-10-10T20:40:25.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.208+0000 7fb69e735700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb698153f30 con 0x7fb69815db10 2023-10-10T20:40:25.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.208+0000 7fb697fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb698159890 0x7fb6981508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:25.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.208+0000 7fb697fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb698159890 0x7fb6981508e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56756/0 (socket says 172.21.15.89:56756) 2023-10-10T20:40:25.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.208+0000 7fb697fff700 1 -- 172.21.15.89:0/1826599200 learned_addr learned my addr 172.21.15.89:0/1826599200 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:25.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.208+0000 7fb6977fe700 1 --2- 172.21.15.89:0/1826599200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb698159c60 0x7fb698154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:25.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.208+0000 7fb69ccd2700 1 --2- 172.21.15.89:0/1826599200 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb69815db10 0x7fb698151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:25.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.208+0000 7fb697fff700 1 -- 172.21.15.89:0/1826599200 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb69815db10 msgr2=0x7fb698151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:25.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.208+0000 7fb697fff700 1 --2- 172.21.15.89:0/1826599200 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb69815db10 0x7fb698151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.208+0000 7fb697fff700 1 -- 172.21.15.89:0/1826599200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb698159c60 msgr2=0x7fb698154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:25.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.208+0000 7fb697fff700 1 --2- 172.21.15.89:0/1826599200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb698159c60 0x7fb698154fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.208+0000 7fb697fff700 1 -- 172.21.15.89:0/1826599200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb690009a20 con 0x7fb698159890 2023-10-10T20:40:25.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.208+0000 7fb6977fe700 1 --2- 172.21.15.89:0/1826599200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb698159c60 0x7fb698154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:25.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.208+0000 7fb697fff700 1 --2- 172.21.15.89:0/1826599200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb698159890 0x7fb6981508e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00c7e0 tx=0x7fb68c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:25.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.209+0000 7fb67ffff700 1 -- 172.21.15.89:0/1826599200 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c005b30 con 0x7fb698159890 2023-10-10T20:40:25.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.209+0000 7fb67ffff700 1 -- 172.21.15.89:0/1826599200 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68c005c90 con 0x7fb698159890 2023-10-10T20:40:25.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.209+0000 7fb69e735700 1 -- 172.21.15.89:0/1826599200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb698154210 con 0x7fb698159890 2023-10-10T20:40:25.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.209+0000 7fb67ffff700 1 -- 172.21.15.89:0/1826599200 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c007300 con 0x7fb698159890 2023-10-10T20:40:25.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.209+0000 7fb69e735700 1 -- 172.21.15.89:0/1826599200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6981547d0 con 0x7fb698159890 2023-10-10T20:40:25.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.209+0000 7fb69e735700 1 -- 172.21.15.89:0/1826599200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69804c3c0 con 0x7fb698159890 2023-10-10T20:40:25.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.210+0000 7fb67ffff700 1 -- 172.21.15.89:0/1826599200 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fb68c01b400 con 0x7fb698159890 2023-10-10T20:40:25.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.210+0000 7fb67ffff700 1 --2- 172.21.15.89:0/1826599200 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb684067200 0x7fb6840696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:25.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.210+0000 7fb67ffff700 1 -- 172.21.15.89:0/1826599200 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb68c084600 con 0x7fb698159890 2023-10-10T20:40:25.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.210+0000 7fb6977fe700 1 --2- 172.21.15.89:0/1826599200 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb684067200 0x7fb6840696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:25.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.211+0000 7fb6977fe700 1 --2- 172.21.15.89:0/1826599200 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb684067200 0x7fb6840696b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb6980b0d80 tx=0x7fb688006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:25.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.215+0000 7fb67ffff700 1 -- 172.21.15.89:0/1826599200 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb68c051440 con 0x7fb698159890 2023-10-10T20:40:25.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.440+0000 7fb69e735700 1 -- 172.21.15.89:0/1826599200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb698069e50 con 0x7fb698159890 2023-10-10T20:40:25.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.441+0000 7fb67ffff700 1 -- 172.21.15.89:0/1826599200 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fb68c054a60 con 0x7fb698159890 2023-10-10T20:40:25.443 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:25.444 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5995,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/3194805494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3194805494},{"type":"v1","addr":"172.21.15.111:6834","nonce":3194805494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6447,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/2766962252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2766962252},{"type":"v1","addr":"172.21.15.89:6836","nonce":2766962252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6450,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1141782720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1141782720},{"type":"v1","addr":"172.21.15.89:6837","nonce":1141782720}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:40:08.387849+0000","modified":"2023-10-10T20:40:12.040215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6462":{"gid":6462,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/4069200228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":4069200228},{"type":"v1","addr":"172.21.15.111:6835","nonce":4069200228}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-10T20:40:25.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.443+0000 7fb69e735700 1 -- 172.21.15.89:0/1826599200 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb684067200 msgr2=0x7fb6840696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:25.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.443+0000 7fb69e735700 1 --2- 172.21.15.89:0/1826599200 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb684067200 0x7fb6840696b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb6980b0d80 tx=0x7fb688006f90 comp rx=0 tx=0).stop 2023-10-10T20:40:25.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.443+0000 7fb69e735700 1 -- 172.21.15.89:0/1826599200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb698159890 msgr2=0x7fb6981508e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:25.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.443+0000 7fb69e735700 1 --2- 172.21.15.89:0/1826599200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb698159890 0x7fb6981508e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00c7e0 tx=0x7fb68c007df0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.443+0000 7fb69e735700 1 -- 172.21.15.89:0/1826599200 shutdown_connections 2023-10-10T20:40:25.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.443+0000 7fb69e735700 1 --2- 172.21.15.89:0/1826599200 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb69815db10 0x7fb698151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.443+0000 7fb69e735700 1 --2- 172.21.15.89:0/1826599200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb698159890 0x7fb6981508e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.443+0000 7fb69e735700 1 --2- 172.21.15.89:0/1826599200 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb684067200 0x7fb6840696b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.443+0000 7fb69e735700 1 --2- 172.21.15.89:0/1826599200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb698159c60 0x7fb698154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.443+0000 7fb69e735700 1 -- 172.21.15.89:0/1826599200 >> 172.21.15.89:0/1826599200 conn(0x7fb698068e20 msgr2=0x7fb69815ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:25.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.443+0000 7fb69e735700 1 -- 172.21.15.89:0/1826599200 shutdown_connections 2023-10-10T20:40:25.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.443+0000 7fb69e735700 1 -- 172.21.15.89:0/1826599200 wait complete. 2023-10-10T20:40:25.448 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 56 2023-10-10T20:40:25.457 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:40:25.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.570+0000 7fd6c24a9700 1 Processor -- start 2023-10-10T20:40:25.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.570+0000 7fd6c24a9700 1 -- start start 2023-10-10T20:40:25.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.571+0000 7fd6c24a9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6bc1578b0 0x7fd6bc155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:25.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.571+0000 7fd6c24a9700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd6bc0af2f0 0x7fd6bc0af710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:25.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.571+0000 7fd6c24a9700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6bc0ad930 con 0x7fd6bc155c10 2023-10-10T20:40:25.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.571+0000 7fd6c24a9700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6bc0b0c60 con 0x7fd6bc0af2f0 2023-10-10T20:40:25.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.571+0000 7fd6c24a9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc0acd20 con 0x7fd6bc1578b0 2023-10-10T20:40:25.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.571+0000 7fd6c0a46700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd6bc0af2f0 0x7fd6bc0af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:25.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.571+0000 7fd6bb7fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6bc1578b0 0x7fd6bc155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:25.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.571+0000 7fd6bbfff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fd6bc155c10 0x7fd6bc0ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:60158/0 (socket says 172.21.15.89:60158) 2023-10-10T20:40:25.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.571+0000 7fd6bbfff700 1 -- 172.21.15.89:0/3589740686 learned_addr learned my addr 172.21.15.89:0/3589740686 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:25.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.572+0000 7fd6bb7fe700 1 -- 172.21.15.89:0/3589740686 >> v2:172.21.15.111:3300/0 conn(0x7fd6bc0af2f0 msgr2=0x7fd6bc0af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:25.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.572+0000 7fd6bb7fe700 1 --2- 172.21.15.89:0/3589740686 >> v2:172.21.15.111:3300/0 conn(0x7fd6bc0af2f0 0x7fd6bc0af710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.572+0000 7fd6bb7fe700 1 -- 172.21.15.89:0/3589740686 >> v1:172.21.15.111:6789/0 conn(0x7fd6bc155c10 legacy=0x7fd6bc0ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:25.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.572+0000 7fd6bb7fe700 1 -- 172.21.15.89:0/3589740686 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6bc0ad000 con 0x7fd6bc1578b0 2023-10-10T20:40:25.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.572+0000 7fd6c0a46700 1 --2- 172.21.15.89:0/3589740686 >> v2:172.21.15.111:3300/0 conn(0x7fd6bc0af2f0 0x7fd6bc0af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:40:25.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.572+0000 7fd6bb7fe700 1 --2- 172.21.15.89:0/3589740686 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6bc1578b0 0x7fd6bc155fe0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00098e0 tx=0x7fd6b00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9be457f66906fc34 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:25.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.572+0000 7fd6baffd700 1 -- 172.21.15.89:0/3589740686 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b000b640 con 0x7fd6bc1578b0 2023-10-10T20:40:25.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.572+0000 7fd6baffd700 1 -- 172.21.15.89:0/3589740686 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b000b7a0 con 0x7fd6bc1578b0 2023-10-10T20:40:25.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.572+0000 7fd6baffd700 1 -- 172.21.15.89:0/3589740686 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b000bb40 con 0x7fd6bc1578b0 2023-10-10T20:40:25.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.573+0000 7fd6c24a9700 1 -- 172.21.15.89:0/3589740686 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6bc1578b0 msgr2=0x7fd6bc155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:25.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.573+0000 7fd6c24a9700 1 --2- 172.21.15.89:0/3589740686 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6bc1578b0 0x7fd6bc155fe0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00098e0 tx=0x7fd6b00091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.573+0000 7fd6c24a9700 1 -- 172.21.15.89:0/3589740686 shutdown_connections 2023-10-10T20:40:25.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.573+0000 7fd6c24a9700 1 --2- 172.21.15.89:0/3589740686 >> v2:172.21.15.111:3300/0 conn(0x7fd6bc0af2f0 0x7fd6bc0af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.573+0000 7fd6c24a9700 1 --2- 172.21.15.89:0/3589740686 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6bc1578b0 0x7fd6bc155fe0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.573+0000 7fd6c24a9700 1 -- 172.21.15.89:0/3589740686 >> 172.21.15.89:0/3589740686 conn(0x7fd6bc06a320 msgr2=0x7fd6bc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:25.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.573+0000 7fd6c24a9700 1 -- 172.21.15.89:0/3589740686 shutdown_connections 2023-10-10T20:40:25.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.573+0000 7fd6c24a9700 1 -- 172.21.15.89:0/3589740686 wait complete. 2023-10-10T20:40:25.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.574+0000 7fd6c24a9700 1 Processor -- start 2023-10-10T20:40:25.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.574+0000 7fd6c24a9700 1 -- start start 2023-10-10T20:40:25.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.575+0000 7fd6c24a9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc0af2f0 0x7fd6bc0a6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:25.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.575+0000 7fd6c24a9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6bc155c10 0x7fd6bc148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:25.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.575+0000 7fd6c24a9700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6bc1578b0 0x7fd6bc0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:25.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.575+0000 7fd6c24a9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc165940 con 0x7fd6bc0af2f0 2023-10-10T20:40:25.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.575+0000 7fd6c24a9700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc165b50 con 0x7fd6bc1578b0 2023-10-10T20:40:25.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.575+0000 7fd6c24a9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc165e30 con 0x7fd6bc155c10 2023-10-10T20:40:25.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.575+0000 7fd6bbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc0af2f0 0x7fd6bc0a6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:25.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.575+0000 7fd6bbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc0af2f0 0x7fd6bc0a6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56772/0 (socket says 172.21.15.89:56772) 2023-10-10T20:40:25.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.575+0000 7fd6bbfff700 1 -- 172.21.15.89:0/2527001798 learned_addr learned my addr 172.21.15.89:0/2527001798 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:25.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.575+0000 7fd6bb7fe700 1 --2- 172.21.15.89:0/2527001798 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6bc155c10 0x7fd6bc148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:25.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.575+0000 7fd6c0a46700 1 --2- 172.21.15.89:0/2527001798 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6bc1578b0 0x7fd6bc0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:25.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.575+0000 7fd6bbfff700 1 -- 172.21.15.89:0/2527001798 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6bc155c10 msgr2=0x7fd6bc148fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:25.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.575+0000 7fd6bbfff700 1 --2- 172.21.15.89:0/2527001798 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6bc155c10 0x7fd6bc148fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.575+0000 7fd6bbfff700 1 -- 172.21.15.89:0/2527001798 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6bc1578b0 msgr2=0x7fd6bc0a6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:25.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.575+0000 7fd6bbfff700 1 --2- 172.21.15.89:0/2527001798 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6bc1578b0 0x7fd6bc0a6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.575+0000 7fd6bbfff700 1 -- 172.21.15.89:0/2527001798 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b400ab30 con 0x7fd6bc0af2f0 2023-10-10T20:40:25.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.575+0000 7fd6bbfff700 1 --2- 172.21.15.89:0/2527001798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc0af2f0 0x7fd6bc0a6c20 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00c7c0 tx=0x7fd6ac007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:25.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.575+0000 7fd6a3fff700 1 -- 172.21.15.89:0/2527001798 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac005b30 con 0x7fd6bc0af2f0 2023-10-10T20:40:25.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.576+0000 7fd6a3fff700 1 -- 172.21.15.89:0/2527001798 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ac005c90 con 0x7fd6bc0af2f0 2023-10-10T20:40:25.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.576+0000 7fd6c24a9700 1 -- 172.21.15.89:0/2527001798 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b0009420 con 0x7fd6bc0af2f0 2023-10-10T20:40:25.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.576+0000 7fd6a3fff700 1 -- 172.21.15.89:0/2527001798 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac007320 con 0x7fd6bc0af2f0 2023-10-10T20:40:25.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.576+0000 7fd6c24a9700 1 -- 172.21.15.89:0/2527001798 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6bc1663f0 con 0x7fd6bc0af2f0 2023-10-10T20:40:25.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.576+0000 7fd6c24a9700 1 -- 172.21.15.89:0/2527001798 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6bc168f20 con 0x7fd6bc0af2f0 2023-10-10T20:40:25.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.577+0000 7fd6a3fff700 1 -- 172.21.15.89:0/2527001798 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fd6ac007690 con 0x7fd6bc0af2f0 2023-10-10T20:40:25.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.577+0000 7fd6a3fff700 1 --2- 172.21.15.89:0/2527001798 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd6a8067310 0x7fd6a80697c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:25.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.577+0000 7fd6bb7fe700 1 --2- 172.21.15.89:0/2527001798 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd6a8067310 0x7fd6a80697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:25.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.577+0000 7fd6a3fff700 1 -- 172.21.15.89:0/2527001798 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd6ac083eb0 con 0x7fd6bc0af2f0 2023-10-10T20:40:25.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.578+0000 7fd6bb7fe700 1 --2- 172.21.15.89:0/2527001798 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd6a8067310 0x7fd6a80697c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00091e0 tx=0x7fd6b0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:25.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.581+0000 7fd6a3fff700 1 -- 172.21.15.89:0/2527001798 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6ac050cd0 con 0x7fd6bc0af2f0 2023-10-10T20:40:25.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.764+0000 7fd6c24a9700 1 -- 172.21.15.89:0/2527001798 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6bc166740 con 0x7fd6bc0af2f0 2023-10-10T20:40:25.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.765+0000 7fd6a3fff700 1 -- 172.21.15.89:0/2527001798 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+15353 (secure 0 0 0) 0x7fd6ac017020 con 0x7fd6bc0af2f0 2023-10-10T20:40:25.767 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:25.767 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":88,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:40:10.022096+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:40:06.747813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-10T20:40:07.347913+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:40:25.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.767+0000 7fd6c24a9700 1 -- 172.21.15.89:0/2527001798 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd6a8067310 msgr2=0x7fd6a80697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:25.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.767+0000 7fd6c24a9700 1 --2- 172.21.15.89:0/2527001798 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd6a8067310 0x7fd6a80697c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00091e0 tx=0x7fd6b0017000 comp rx=0 tx=0).stop 2023-10-10T20:40:25.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.767+0000 7fd6c24a9700 1 -- 172.21.15.89:0/2527001798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc0af2f0 msgr2=0x7fd6bc0a6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:25.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.767+0000 7fd6c24a9700 1 --2- 172.21.15.89:0/2527001798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc0af2f0 0x7fd6bc0a6c20 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00c7c0 tx=0x7fd6ac007f30 comp rx=0 tx=0).stop 2023-10-10T20:40:25.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.767+0000 7fd6c24a9700 1 -- 172.21.15.89:0/2527001798 shutdown_connections 2023-10-10T20:40:25.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.767+0000 7fd6c24a9700 1 --2- 172.21.15.89:0/2527001798 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd6a8067310 0x7fd6a80697c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.767+0000 7fd6c24a9700 1 --2- 172.21.15.89:0/2527001798 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6bc1578b0 0x7fd6bc0a6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.767+0000 7fd6c24a9700 1 --2- 172.21.15.89:0/2527001798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc0af2f0 0x7fd6bc0a6c20 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.767+0000 7fd6c24a9700 1 --2- 172.21.15.89:0/2527001798 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6bc155c10 0x7fd6bc148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.767+0000 7fd6c24a9700 1 -- 172.21.15.89:0/2527001798 >> 172.21.15.89:0/2527001798 conn(0x7fd6bc06a320 msgr2=0x7fd6bc154b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:25.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.767+0000 7fd6c24a9700 1 -- 172.21.15.89:0/2527001798 shutdown_connections 2023-10-10T20:40:25.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.767+0000 7fd6c24a9700 1 -- 172.21.15.89:0/2527001798 wait complete. 2023-10-10T20:40:25.782 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:40:25.783 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:40:25.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.895+0000 7f9cb16df700 1 Processor -- start 2023-10-10T20:40:25.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.895+0000 7f9cb16df700 1 -- start start 2023-10-10T20:40:25.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.896+0000 7f9cb16df700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9cac142ea0 0x7f9cac0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:25.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.896+0000 7f9cb16df700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cac0b40f0 con 0x7f9cac0ad870 2023-10-10T20:40:25.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.896+0000 7f9cb16df700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cac069640 con 0x7f9cac06c0f0 2023-10-10T20:40:25.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.896+0000 7f9cb16df700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cac069920 con 0x7f9cac142ea0 2023-10-10T20:40:25.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.896+0000 7f9cabfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9cac142ea0 0x7f9cac0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:25.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.896+0000 7f9cab7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9cac0ad870 0x7f9cac06b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36468/0 (socket says 172.21.15.89:36468) 2023-10-10T20:40:25.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.896+0000 7f9cab7fe700 1 -- 172.21.15.89:0/2324020255 learned_addr learned my addr 172.21.15.89:0/2324020255 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:25.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.896+0000 7f9cabfff700 1 -- 172.21.15.89:0/2324020255 >> v1:172.21.15.111:6789/0 conn(0x7f9cac06c0f0 legacy=0x7f9cac06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:25.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.896+0000 7f9cabfff700 1 -- 172.21.15.89:0/2324020255 >> v1:172.21.15.89:6789/0 conn(0x7f9cac0ad870 legacy=0x7f9cac06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:25.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.896+0000 7f9cabfff700 1 -- 172.21.15.89:0/2324020255 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac069c00 con 0x7f9cac142ea0 2023-10-10T20:40:25.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.897+0000 7f9cabfff700 1 --2- 172.21.15.89:0/2324020255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cac142ea0 0x7f9cac0b3630 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0098e0 tx=0x7f9c9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87717a19384a6cb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:25.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.897+0000 7f9caaffd700 1 -- 172.21.15.89:0/2324020255 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c009e80 con 0x7f9cac142ea0 2023-10-10T20:40:25.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.897+0000 7f9caaffd700 1 -- 172.21.15.89:0/2324020255 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9c00b860 con 0x7f9cac142ea0 2023-10-10T20:40:25.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.897+0000 7f9caaffd700 1 -- 172.21.15.89:0/2324020255 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c00bbe0 con 0x7f9cac142ea0 2023-10-10T20:40:25.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.897+0000 7f9cb16df700 1 -- 172.21.15.89:0/2324020255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cac142ea0 msgr2=0x7f9cac0b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:25.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.897+0000 7f9cb16df700 1 --2- 172.21.15.89:0/2324020255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cac142ea0 0x7f9cac0b3630 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0098e0 tx=0x7f9c9c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.897+0000 7f9cb16df700 1 -- 172.21.15.89:0/2324020255 shutdown_connections 2023-10-10T20:40:25.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.897+0000 7f9cb16df700 1 --2- 172.21.15.89:0/2324020255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cac142ea0 0x7f9cac0b3630 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.897+0000 7f9cb16df700 1 -- 172.21.15.89:0/2324020255 >> 172.21.15.89:0/2324020255 conn(0x7f9cac0ce430 msgr2=0x7f9cac0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:25.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.897+0000 7f9cb16df700 1 -- 172.21.15.89:0/2324020255 shutdown_connections 2023-10-10T20:40:25.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.897+0000 7f9cb16df700 1 -- 172.21.15.89:0/2324020255 wait complete. 2023-10-10T20:40:25.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.898+0000 7f9cb16df700 1 Processor -- start 2023-10-10T20:40:25.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.899+0000 7f9cb16df700 1 -- start start 2023-10-10T20:40:25.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.899+0000 7f9cb16df700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9cac06c0f0 0x7f9cac15caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:25.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.899+0000 7f9cb16df700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cac142ea0 0x7f9cac158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:25.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.899+0000 7f9cb16df700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cac0ca020 0x7f9cac0c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:25.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.899+0000 7f9cb16df700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac15d3a0 con 0x7f9cac0ca020 2023-10-10T20:40:25.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.899+0000 7f9cb16df700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0c75e0 con 0x7f9cac142ea0 2023-10-10T20:40:25.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.899+0000 7f9cb16df700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0c78c0 con 0x7f9cac06c0f0 2023-10-10T20:40:25.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.899+0000 7f9cb0ede700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cac0ca020 0x7f9cac0c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:25.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.899+0000 7f9cb0ede700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cac0ca020 0x7f9cac0c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56780/0 (socket says 172.21.15.89:56780) 2023-10-10T20:40:25.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.899+0000 7f9cb0ede700 1 -- 172.21.15.89:0/71782543 learned_addr learned my addr 172.21.15.89:0/71782543 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:25.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.899+0000 7f9cabfff700 1 --2- 172.21.15.89:0/71782543 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9cac06c0f0 0x7f9cac15caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:25.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.899+0000 7f9cab7fe700 1 --2- 172.21.15.89:0/71782543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cac142ea0 0x7f9cac158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:25.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.900+0000 7f9cb0ede700 1 -- 172.21.15.89:0/71782543 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9cac06c0f0 msgr2=0x7f9cac15caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:25.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.900+0000 7f9cb0ede700 1 --2- 172.21.15.89:0/71782543 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9cac06c0f0 0x7f9cac15caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.900+0000 7f9cb0ede700 1 -- 172.21.15.89:0/71782543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cac142ea0 msgr2=0x7f9cac158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:25.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.900+0000 7f9cb0ede700 1 --2- 172.21.15.89:0/71782543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cac142ea0 0x7f9cac158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:25.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.900+0000 7f9cb0ede700 1 -- 172.21.15.89:0/71782543 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9c009420 con 0x7f9cac0ca020 2023-10-10T20:40:25.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.900+0000 7f9cb0ede700 1 --2- 172.21.15.89:0/71782543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cac0ca020 0x7f9cac0c83a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400c910 tx=0x7f9ca40016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:25.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.900+0000 7f9c93fff700 1 -- 172.21.15.89:0/71782543 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca400a7e0 con 0x7f9cac0ca020 2023-10-10T20:40:25.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.900+0000 7f9c93fff700 1 -- 172.21.15.89:0/71782543 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca400a940 con 0x7f9cac0ca020 2023-10-10T20:40:25.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.900+0000 7f9cb16df700 1 -- 172.21.15.89:0/71782543 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cac0c7ba0 con 0x7f9cac0ca020 2023-10-10T20:40:25.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.900+0000 7f9c93fff700 1 -- 172.21.15.89:0/71782543 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca400ad00 con 0x7f9cac0ca020 2023-10-10T20:40:25.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.900+0000 7f9cb16df700 1 -- 172.21.15.89:0/71782543 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cac15cfe0 con 0x7f9cac0ca020 2023-10-10T20:40:25.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.901+0000 7f9cb16df700 1 -- 172.21.15.89:0/71782543 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cac04c3c0 con 0x7f9cac0ca020 2023-10-10T20:40:25.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.901+0000 7f9c93fff700 1 -- 172.21.15.89:0/71782543 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f9ca4012630 con 0x7f9cac0ca020 2023-10-10T20:40:25.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.901+0000 7f9c93fff700 1 --2- 172.21.15.89:0/71782543 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9c98067290 0x7f9c98069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:25.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.902+0000 7f9c93fff700 1 -- 172.21.15.89:0/71782543 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9ca4084ea0 con 0x7f9cac0ca020 2023-10-10T20:40:25.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.902+0000 7f9cabfff700 1 --2- 172.21.15.89:0/71782543 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9c98067290 0x7f9c98069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:25.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.902+0000 7f9cabfff700 1 --2- 172.21.15.89:0/71782543 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9c98067290 0x7f9c98069740 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0093f0 tx=0x7f9c9c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:25.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:25.906+0000 7f9c93fff700 1 -- 172.21.15.89:0/71782543 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ca4051c00 con 0x7f9cac0ca020 2023-10-10T20:40:26.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.131+0000 7f9cb16df700 1 -- 172.21.15.89:0/71782543 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cac143270 con 0x7f9cac0ca020 2023-10-10T20:40:26.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.131+0000 7f9c93fff700 1 -- 172.21.15.89:0/71782543 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f9ca4016020 con 0x7f9cac0ca020 2023-10-10T20:40:26.134 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:26.134 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5995,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/3194805494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3194805494},{"type":"v1","addr":"172.21.15.111:6834","nonce":3194805494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6447,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/2766962252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2766962252},{"type":"v1","addr":"172.21.15.89:6836","nonce":2766962252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6450,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1141782720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1141782720},{"type":"v1","addr":"172.21.15.89:6837","nonce":1141782720}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:40:08.387849+0000","modified":"2023-10-10T20:40:12.040215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6462":{"gid":6462,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/4069200228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":4069200228},{"type":"v1","addr":"172.21.15.111:6835","nonce":4069200228}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-10T20:40:26.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.134+0000 7f9cb16df700 1 -- 172.21.15.89:0/71782543 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9c98067290 msgr2=0x7f9c98069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:26.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.134+0000 7f9cb16df700 1 --2- 172.21.15.89:0/71782543 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9c98067290 0x7f9c98069740 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0093f0 tx=0x7f9c9c017000 comp rx=0 tx=0).stop 2023-10-10T20:40:26.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.134+0000 7f9cb16df700 1 -- 172.21.15.89:0/71782543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cac0ca020 msgr2=0x7f9cac0c83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:26.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.134+0000 7f9cb16df700 1 --2- 172.21.15.89:0/71782543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cac0ca020 0x7f9cac0c83a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400c910 tx=0x7f9ca40016f0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.134+0000 7f9cb16df700 1 -- 172.21.15.89:0/71782543 shutdown_connections 2023-10-10T20:40:26.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.134+0000 7f9cb16df700 1 --2- 172.21.15.89:0/71782543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cac0ca020 0x7f9cac0c83a0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.134+0000 7f9cb16df700 1 --2- 172.21.15.89:0/71782543 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9cac06c0f0 0x7f9cac15caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.134+0000 7f9cb16df700 1 --2- 172.21.15.89:0/71782543 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9c98067290 0x7f9c98069740 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.134+0000 7f9cb16df700 1 --2- 172.21.15.89:0/71782543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cac142ea0 0x7f9cac158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.134+0000 7f9cb16df700 1 -- 172.21.15.89:0/71782543 >> 172.21.15.89:0/71782543 conn(0x7f9cac0ce430 msgr2=0x7f9cac141e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:26.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.134+0000 7f9cb16df700 1 -- 172.21.15.89:0/71782543 shutdown_connections 2023-10-10T20:40:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.134+0000 7f9cb16df700 1 -- 172.21.15.89:0/71782543 wait complete. 2023-10-10T20:40:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 56 2023-10-10T20:40:26.148 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:40:26.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.260+0000 7f181f16c700 1 Processor -- start 2023-10-10T20:40:26.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.260+0000 7f181f16c700 1 -- start start 2023-10-10T20:40:26.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.261+0000 7f181f16c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1818159890 0x7f181815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:26.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.261+0000 7f181f16c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1818069560 0x7f18180699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:26.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.261+0000 7f181f16c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1818069ee0 con 0x7f1818159c60 2023-10-10T20:40:26.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.261+0000 7f181f16c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f181815a160 con 0x7f1818159890 2023-10-10T20:40:26.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.261+0000 7f181f16c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18180d13a0 con 0x7f1818069560 2023-10-10T20:40:26.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.261+0000 7f181cf08700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1818159890 0x7f181815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:26.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.261+0000 7f181cf08700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1818159890 0x7f181815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56784/0 (socket says 172.21.15.89:56784) 2023-10-10T20:40:26.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.261+0000 7f181cf08700 1 -- 172.21.15.89:0/673764368 learned_addr learned my addr 172.21.15.89:0/673764368 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:26.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.261+0000 7f181d709700 1 --2- 172.21.15.89:0/673764368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1818069560 0x7f18180699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:26.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.261+0000 7f181cf08700 1 -- 172.21.15.89:0/673764368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1818069560 msgr2=0x7f18180699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:26.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.261+0000 7f181cf08700 1 --2- 172.21.15.89:0/673764368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1818069560 0x7f18180699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.261+0000 7f181cf08700 1 -- 172.21.15.89:0/673764368 >> v1:172.21.15.111:6789/0 conn(0x7f1818159c60 legacy=0x7f181815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:26.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.261+0000 7f181cf08700 1 -- 172.21.15.89:0/673764368 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181815e140 con 0x7f1818159890 2023-10-10T20:40:26.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.261+0000 7f181d709700 1 --2- 172.21.15.89:0/673764368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1818069560 0x7f18180699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:26.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.262+0000 7f181cf08700 1 --2- 172.21.15.89:0/673764368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1818159890 0x7f181815abd0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f18080099b0 tx=0x7f1808009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29619d33e9a9bac7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:26.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.262+0000 7f18177fe700 1 -- 172.21.15.89:0/673764368 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180800b860 con 0x7f1818159890 2023-10-10T20:40:26.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.262+0000 7f18177fe700 1 -- 172.21.15.89:0/673764368 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180800b9c0 con 0x7f1818159890 2023-10-10T20:40:26.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.262+0000 7f18177fe700 1 -- 172.21.15.89:0/673764368 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180800bd60 con 0x7f1818159890 2023-10-10T20:40:26.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.262+0000 7f181f16c700 1 -- 172.21.15.89:0/673764368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1818159890 msgr2=0x7f181815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:26.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.262+0000 7f181f16c700 1 --2- 172.21.15.89:0/673764368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1818159890 0x7f181815abd0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f18080099b0 tx=0x7f1808009250 comp rx=0 tx=0).stop 2023-10-10T20:40:26.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.262+0000 7f181f16c700 1 -- 172.21.15.89:0/673764368 shutdown_connections 2023-10-10T20:40:26.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.262+0000 7f181f16c700 1 --2- 172.21.15.89:0/673764368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1818159890 0x7f181815abd0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.262+0000 7f181f16c700 1 --2- 172.21.15.89:0/673764368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1818069560 0x7f18180699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.262+0000 7f181f16c700 1 -- 172.21.15.89:0/673764368 >> 172.21.15.89:0/673764368 conn(0x7f1818068e20 msgr2=0x7f18180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:26.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.262+0000 7f181f16c700 1 -- 172.21.15.89:0/673764368 shutdown_connections 2023-10-10T20:40:26.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.262+0000 7f181f16c700 1 -- 172.21.15.89:0/673764368 wait complete. 2023-10-10T20:40:26.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.263+0000 7f181f16c700 1 Processor -- start 2023-10-10T20:40:26.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.264+0000 7f181f16c700 1 -- start start 2023-10-10T20:40:26.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.264+0000 7f181f16c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1818069560 0x7f18181524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:26.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.264+0000 7f181f16c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1818159890 0x7f18180cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:26.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.264+0000 7f181f16c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1818159c60 0x7f18180b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:26.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.264+0000 7f181f16c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18180b3a20 con 0x7f1818159890 2023-10-10T20:40:26.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.264+0000 7f181f16c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18180b3d00 con 0x7f1818159c60 2023-10-10T20:40:26.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.264+0000 7f181f16c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18180b3fe0 con 0x7f1818069560 2023-10-10T20:40:26.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.264+0000 7f1817fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1818159890 0x7f18180cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:26.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.265+0000 7f1817fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1818159890 0x7f18180cf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56792/0 (socket says 172.21.15.89:56792) 2023-10-10T20:40:26.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.265+0000 7f1817fff700 1 -- 172.21.15.89:0/3524256120 learned_addr learned my addr 172.21.15.89:0/3524256120 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:26.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.265+0000 7f181cf08700 1 --2- 172.21.15.89:0/3524256120 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1818069560 0x7f18181524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:26.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.265+0000 7f181d709700 1 --2- 172.21.15.89:0/3524256120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1818159c60 0x7f18180b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:26.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.265+0000 7f1817fff700 1 -- 172.21.15.89:0/3524256120 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1818069560 msgr2=0x7f18181524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:26.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.265+0000 7f1817fff700 1 --2- 172.21.15.89:0/3524256120 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1818069560 0x7f18181524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.265+0000 7f1817fff700 1 -- 172.21.15.89:0/3524256120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1818159c60 msgr2=0x7f18180b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:26.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.265+0000 7f1817fff700 1 --2- 172.21.15.89:0/3524256120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1818159c60 0x7f18180b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.265+0000 7f1817fff700 1 -- 172.21.15.89:0/3524256120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18080094f0 con 0x7f1818159890 2023-10-10T20:40:26.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.265+0000 7f181cf08700 1 --2- 172.21.15.89:0/3524256120 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1818069560 0x7f18181524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:26.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.265+0000 7f1817fff700 1 --2- 172.21.15.89:0/3524256120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1818159890 0x7f18180cf7f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f180c00c590 tx=0x7f180c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:26.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.265+0000 7f1814ff9700 1 -- 172.21.15.89:0/3524256120 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c011590 con 0x7f1818159890 2023-10-10T20:40:26.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.265+0000 7f1814ff9700 1 -- 172.21.15.89:0/3524256120 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180c0116f0 con 0x7f1818159890 2023-10-10T20:40:26.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.265+0000 7f181f16c700 1 -- 172.21.15.89:0/3524256120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18180b42c0 con 0x7f1818159890 2023-10-10T20:40:26.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.265+0000 7f1814ff9700 1 -- 172.21.15.89:0/3524256120 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c011ab0 con 0x7f1818159890 2023-10-10T20:40:26.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.265+0000 7f181f16c700 1 -- 172.21.15.89:0/3524256120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18181529f0 con 0x7f1818159890 2023-10-10T20:40:26.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.266+0000 7f181f16c700 1 -- 172.21.15.89:0/3524256120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1818169b90 con 0x7f1818159890 2023-10-10T20:40:26.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.267+0000 7f1814ff9700 1 -- 172.21.15.89:0/3524256120 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f180c01b3b0 con 0x7f1818159890 2023-10-10T20:40:26.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.267+0000 7f1814ff9700 1 --2- 172.21.15.89:0/3524256120 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1804067290 0x7f1804069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:26.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.267+0000 7f1814ff9700 1 -- 172.21.15.89:0/3524256120 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f180c083ff0 con 0x7f1818159890 2023-10-10T20:40:26.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.267+0000 7f181cf08700 1 --2- 172.21.15.89:0/3524256120 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1804067290 0x7f1804069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:26.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.267+0000 7f181cf08700 1 --2- 172.21.15.89:0/3524256120 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1804067290 0x7f1804069740 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f18080099b0 tx=0x7f1808017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:26.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.271+0000 7f1814ff9700 1 -- 172.21.15.89:0/3524256120 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1818169b90 con 0x7f1818159890 2023-10-10T20:40:26.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.496+0000 7f181f16c700 1 -- 172.21.15.89:0/3524256120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1818169b90 con 0x7f1818159890 2023-10-10T20:40:26.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.496+0000 7f1814ff9700 1 -- 172.21.15.89:0/3524256120 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f1818169b90 con 0x7f1818159890 2023-10-10T20:40:26.499 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:26.499 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5995,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/3194805494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3194805494},{"type":"v1","addr":"172.21.15.111:6834","nonce":3194805494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6447,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/2766962252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2766962252},{"type":"v1","addr":"172.21.15.89:6836","nonce":2766962252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6450,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1141782720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1141782720},{"type":"v1","addr":"172.21.15.89:6837","nonce":1141782720}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:40:08.387849+0000","modified":"2023-10-10T20:40:12.040215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6462":{"gid":6462,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/4069200228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":4069200228},{"type":"v1","addr":"172.21.15.111:6835","nonce":4069200228}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-10T20:40:26.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.498+0000 7f181f16c700 1 -- 172.21.15.89:0/3524256120 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1804067290 msgr2=0x7f1804069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:26.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.498+0000 7f181f16c700 1 --2- 172.21.15.89:0/3524256120 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1804067290 0x7f1804069740 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f18080099b0 tx=0x7f1808017000 comp rx=0 tx=0).stop 2023-10-10T20:40:26.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.498+0000 7f181f16c700 1 -- 172.21.15.89:0/3524256120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1818159890 msgr2=0x7f18180cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:26.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.498+0000 7f181f16c700 1 --2- 172.21.15.89:0/3524256120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1818159890 0x7f18180cf7f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f180c00c590 tx=0x7f180c00c950 comp rx=0 tx=0).stop 2023-10-10T20:40:26.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.498+0000 7f181f16c700 1 -- 172.21.15.89:0/3524256120 shutdown_connections 2023-10-10T20:40:26.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.498+0000 7f181f16c700 1 --2- 172.21.15.89:0/3524256120 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1804067290 0x7f1804069740 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.498+0000 7f181f16c700 1 --2- 172.21.15.89:0/3524256120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1818159c60 0x7f18180b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.498+0000 7f181f16c700 1 --2- 172.21.15.89:0/3524256120 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1818069560 0x7f18181524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.498+0000 7f181f16c700 1 --2- 172.21.15.89:0/3524256120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1818159890 0x7f18180cf7f0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.498+0000 7f181f16c700 1 -- 172.21.15.89:0/3524256120 >> 172.21.15.89:0/3524256120 conn(0x7f1818068e20 msgr2=0x7f18180b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:26.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.498+0000 7f181f16c700 1 -- 172.21.15.89:0/3524256120 shutdown_connections 2023-10-10T20:40:26.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.498+0000 7f181f16c700 1 -- 172.21.15.89:0/3524256120 wait complete. 2023-10-10T20:40:26.503 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 56 2023-10-10T20:40:26.512 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:40:26.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.625+0000 7fd19116b700 1 Processor -- start 2023-10-10T20:40:26.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.625+0000 7fd19116b700 1 -- start start 2023-10-10T20:40:26.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.626+0000 7fd19116b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd18c06bcd0 0x7fd18c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:26.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.626+0000 7fd19116b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd18c15df50 0x7fd18c15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:26.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.626+0000 7fd19116b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18c15d7e0 con 0x7fd18c06c0a0 2023-10-10T20:40:26.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.626+0000 7fd19116b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18c15dac0 con 0x7fd18c06bcd0 2023-10-10T20:40:26.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.626+0000 7fd19116b700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18c15aa30 con 0x7fd18c15df50 2023-10-10T20:40:26.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.626+0000 7fd18ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd18c06bcd0 0x7fd18c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:26.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.626+0000 7fd18259c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd18c06c0a0 0x7fd18c15a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36470/0 (socket says 172.21.15.89:36470) 2023-10-10T20:40:26.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.626+0000 7fd18259c700 1 -- 172.21.15.89:0/921397565 learned_addr learned my addr 172.21.15.89:0/921397565 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:26.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.626+0000 7fd18b59e700 1 --2- 172.21.15.89:0/921397565 >> v2:172.21.15.111:3300/0 conn(0x7fd18c15df50 0x7fd18c15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:26.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.627+0000 7fd18ad9d700 1 -- 172.21.15.89:0/921397565 >> v2:172.21.15.111:3300/0 conn(0x7fd18c15df50 msgr2=0x7fd18c15e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:26.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.627+0000 7fd18ad9d700 1 --2- 172.21.15.89:0/921397565 >> v2:172.21.15.111:3300/0 conn(0x7fd18c15df50 0x7fd18c15e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.627+0000 7fd18ad9d700 1 -- 172.21.15.89:0/921397565 >> v1:172.21.15.89:6789/0 conn(0x7fd18c06c0a0 legacy=0x7fd18c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:26.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.627+0000 7fd18ad9d700 1 -- 172.21.15.89:0/921397565 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c15ad10 con 0x7fd18c06bcd0 2023-10-10T20:40:26.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.627+0000 7fd18ad9d700 1 --2- 172.21.15.89:0/921397565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd18c06bcd0 0x7fd18c0caa90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd18c15d720 tx=0x7fd17c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518e964c0a21b039 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:26.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.627+0000 7fd18a59c700 1 -- 172.21.15.89:0/921397565 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c00b6b0 con 0x7fd18c06bcd0 2023-10-10T20:40:26.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.627+0000 7fd18a59c700 1 -- 172.21.15.89:0/921397565 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c00b810 con 0x7fd18c06bcd0 2023-10-10T20:40:26.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.627+0000 7fd18a59c700 1 -- 172.21.15.89:0/921397565 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c00bbb0 con 0x7fd18c06bcd0 2023-10-10T20:40:26.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.627+0000 7fd19116b700 1 -- 172.21.15.89:0/921397565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd18c06bcd0 msgr2=0x7fd18c0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:26.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.627+0000 7fd19116b700 1 --2- 172.21.15.89:0/921397565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd18c06bcd0 0x7fd18c0caa90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd18c15d720 tx=0x7fd17c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.627+0000 7fd19116b700 1 -- 172.21.15.89:0/921397565 shutdown_connections 2023-10-10T20:40:26.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.627+0000 7fd19116b700 1 --2- 172.21.15.89:0/921397565 >> v2:172.21.15.111:3300/0 conn(0x7fd18c15df50 0x7fd18c15e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.627+0000 7fd19116b700 1 --2- 172.21.15.89:0/921397565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd18c06bcd0 0x7fd18c0caa90 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.627+0000 7fd19116b700 1 -- 172.21.15.89:0/921397565 >> 172.21.15.89:0/921397565 conn(0x7fd18c0ce600 msgr2=0x7fd18c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:26.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.627+0000 7fd19116b700 1 -- 172.21.15.89:0/921397565 shutdown_connections 2023-10-10T20:40:26.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.627+0000 7fd19116b700 1 -- 172.21.15.89:0/921397565 wait complete. 2023-10-10T20:40:26.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.629+0000 7fd19116b700 1 Processor -- start 2023-10-10T20:40:26.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.629+0000 7fd19116b700 1 -- start start 2023-10-10T20:40:26.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.629+0000 7fd19116b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd18c06bcd0 0x7fd18c147d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:26.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.629+0000 7fd19116b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd18c06c0a0 0x7fd18c0b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:26.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.629+0000 7fd19116b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd18c15df50 0x7fd18c0b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:26.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.629+0000 7fd19116b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c0b25f0 con 0x7fd18c06bcd0 2023-10-10T20:40:26.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.629+0000 7fd19116b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c0b3440 con 0x7fd18c06c0a0 2023-10-10T20:40:26.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.629+0000 7fd19116b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c0b2830 con 0x7fd18c15df50 2023-10-10T20:40:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.629+0000 7fd18ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd18c06bcd0 0x7fd18c147d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.629+0000 7fd18ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd18c06bcd0 0x7fd18c147d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56816/0 (socket says 172.21.15.89:56816) 2023-10-10T20:40:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.629+0000 7fd18ad9d700 1 -- 172.21.15.89:0/3875366819 learned_addr learned my addr 172.21.15.89:0/3875366819 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.630+0000 7fd18b59e700 1 --2- 172.21.15.89:0/3875366819 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd18c15df50 0x7fd18c0b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.630+0000 7fd18259c700 1 --2- 172.21.15.89:0/3875366819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd18c06c0a0 0x7fd18c0b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:26.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.630+0000 7fd18ad9d700 1 -- 172.21.15.89:0/3875366819 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd18c15df50 msgr2=0x7fd18c0b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:26.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.630+0000 7fd18ad9d700 1 --2- 172.21.15.89:0/3875366819 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd18c15df50 0x7fd18c0b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.630+0000 7fd18ad9d700 1 -- 172.21.15.89:0/3875366819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd18c06c0a0 msgr2=0x7fd18c0b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:26.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.630+0000 7fd18ad9d700 1 --2- 172.21.15.89:0/3875366819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd18c06c0a0 0x7fd18c0b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.630+0000 7fd18ad9d700 1 -- 172.21.15.89:0/3875366819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c009420 con 0x7fd18c06bcd0 2023-10-10T20:40:26.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.630+0000 7fd18ad9d700 1 --2- 172.21.15.89:0/3875366819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd18c06bcd0 0x7fd18c147d20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fd18c069220 tx=0x7fd17c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:26.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.630+0000 7fd1837fe700 1 -- 172.21.15.89:0/3875366819 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c00bb90 con 0x7fd18c06bcd0 2023-10-10T20:40:26.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.630+0000 7fd1837fe700 1 -- 172.21.15.89:0/3875366819 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c011420 con 0x7fd18c06bcd0 2023-10-10T20:40:26.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.630+0000 7fd19116b700 1 -- 172.21.15.89:0/3875366819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18c0b2b10 con 0x7fd18c06bcd0 2023-10-10T20:40:26.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.630+0000 7fd1837fe700 1 -- 172.21.15.89:0/3875366819 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c011840 con 0x7fd18c06bcd0 2023-10-10T20:40:26.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.630+0000 7fd19116b700 1 -- 172.21.15.89:0/3875366819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18c0b30d0 con 0x7fd18c06bcd0 2023-10-10T20:40:26.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.631+0000 7fd19116b700 1 -- 172.21.15.89:0/3875366819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd18c04c3c0 con 0x7fd18c06bcd0 2023-10-10T20:40:26.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.631+0000 7fd1837fe700 1 -- 172.21.15.89:0/3875366819 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fd17c011c70 con 0x7fd18c06bcd0 2023-10-10T20:40:26.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.632+0000 7fd1837fe700 1 --2- 172.21.15.89:0/3875366819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd16c067290 0x7fd16c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:26.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.632+0000 7fd1837fe700 1 -- 172.21.15.89:0/3875366819 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd17c086350 con 0x7fd18c06bcd0 2023-10-10T20:40:26.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.632+0000 7fd18259c700 1 --2- 172.21.15.89:0/3875366819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd16c067290 0x7fd16c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:26.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.632+0000 7fd18259c700 1 --2- 172.21.15.89:0/3875366819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd16c067290 0x7fd16c069740 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd18c16d1c0 tx=0x7fd174009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:26.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.636+0000 7fd1837fe700 1 -- 172.21.15.89:0/3875366819 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd17c0531f0 con 0x7fd18c06bcd0 2023-10-10T20:40:26.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.824+0000 7fd19116b700 1 -- 172.21.15.89:0/3875366819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd18c15aff0 con 0x7fd18c06bcd0 2023-10-10T20:40:26.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.825+0000 7fd1837fe700 1 -- 172.21.15.89:0/3875366819 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+15353 (secure 0 0 0) 0x7fd17c0567d0 con 0x7fd18c06bcd0 2023-10-10T20:40:26.827 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:26.827 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":88,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:40:10.022096+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:40:06.747813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-10T20:40:07.347913+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:40:26.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.827+0000 7fd19116b700 1 -- 172.21.15.89:0/3875366819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd16c067290 msgr2=0x7fd16c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:26.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.827+0000 7fd19116b700 1 --2- 172.21.15.89:0/3875366819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd16c067290 0x7fd16c069740 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd18c16d1c0 tx=0x7fd174009040 comp rx=0 tx=0).stop 2023-10-10T20:40:26.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.827+0000 7fd19116b700 1 -- 172.21.15.89:0/3875366819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd18c06bcd0 msgr2=0x7fd18c147d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:26.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.827+0000 7fd19116b700 1 --2- 172.21.15.89:0/3875366819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd18c06bcd0 0x7fd18c147d20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fd18c069220 tx=0x7fd17c017000 comp rx=0 tx=0).stop 2023-10-10T20:40:26.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.827+0000 7fd19116b700 1 -- 172.21.15.89:0/3875366819 shutdown_connections 2023-10-10T20:40:26.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.827+0000 7fd19116b700 1 --2- 172.21.15.89:0/3875366819 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd18c15df50 0x7fd18c0b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.827+0000 7fd19116b700 1 --2- 172.21.15.89:0/3875366819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd18c06bcd0 0x7fd18c147d20 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.827+0000 7fd19116b700 1 --2- 172.21.15.89:0/3875366819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd16c067290 0x7fd16c069740 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.827+0000 7fd19116b700 1 --2- 172.21.15.89:0/3875366819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd18c06c0a0 0x7fd18c0b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.827+0000 7fd19116b700 1 -- 172.21.15.89:0/3875366819 >> 172.21.15.89:0/3875366819 conn(0x7fd18c0ce600 msgr2=0x7fd18c0c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:26.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.827+0000 7fd19116b700 1 -- 172.21.15.89:0/3875366819 shutdown_connections 2023-10-10T20:40:26.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.827+0000 7fd19116b700 1 -- 172.21.15.89:0/3875366819 wait complete. 2023-10-10T20:40:26.841 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:40:26.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.954+0000 7f78426a6700 1 Processor -- start 2023-10-10T20:40:26.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.955+0000 7f78426a6700 1 -- start start 2023-10-10T20:40:26.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.955+0000 7f78426a6700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f783c159890 0x7f783c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:26.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.955+0000 7f78426a6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783c15a160 con 0x7f783c069560 2023-10-10T20:40:26.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.955+0000 7f78426a6700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783c0d13a0 con 0x7f783c159c60 2023-10-10T20:40:26.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.955+0000 7f78426a6700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f783c15e140 con 0x7f783c159890 2023-10-10T20:40:26.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.955+0000 7f7840c43700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f783c069560 0x7f783c0ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36480/0 (socket says 172.21.15.89:36480) 2023-10-10T20:40:26.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.955+0000 7f7840c43700 1 -- 172.21.15.89:0/2103984217 learned_addr learned my addr 172.21.15.89:0/2103984217 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:26.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.956+0000 7f783bfff700 1 --2- 172.21.15.89:0/2103984217 >> v2:172.21.15.111:3300/0 conn(0x7f783c159890 0x7f783c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:26.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.956+0000 7f783affd700 1 -- 172.21.15.89:0/2103984217 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805925059 0 0) 0x7f783c15a160 con 0x7f783c069560 2023-10-10T20:40:26.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.956+0000 7f783affd700 1 -- 172.21.15.89:0/2103984217 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7824003570 con 0x7f783c069560 2023-10-10T20:40:26.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.956+0000 7f783bfff700 1 -- 172.21.15.89:0/2103984217 >> v1:172.21.15.111:6789/0 conn(0x7f783c159c60 legacy=0x7f783c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:26.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.956+0000 7f783affd700 1 -- 172.21.15.89:0/2103984217 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 837675151 0 0) 0x7f7824003570 con 0x7f783c069560 2023-10-10T20:40:26.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.956+0000 7f783bfff700 1 -- 172.21.15.89:0/2103984217 >> v1:172.21.15.89:6789/0 conn(0x7f783c069560 legacy=0x7f783c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:26.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.956+0000 7f783bfff700 1 -- 172.21.15.89:0/2103984217 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783c15a3d0 con 0x7f783c159890 2023-10-10T20:40:26.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.956+0000 7f783bfff700 1 --2- 172.21.15.89:0/2103984217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f783c159890 0x7f783c15abd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f782c0098e0 tx=0x7f782c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53068f88db59f45d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:26.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.956+0000 7f783affd700 1 -- 172.21.15.89:0/2103984217 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78340017f0 con 0x7f783c159890 2023-10-10T20:40:26.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.956+0000 7f783affd700 1 -- 172.21.15.89:0/2103984217 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782c011420 con 0x7f783c159890 2023-10-10T20:40:26.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.956+0000 7f783affd700 1 -- 172.21.15.89:0/2103984217 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c0117c0 con 0x7f783c159890 2023-10-10T20:40:26.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.957+0000 7f78426a6700 1 -- 172.21.15.89:0/2103984217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f783c159890 msgr2=0x7f783c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:26.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.957+0000 7f78426a6700 1 --2- 172.21.15.89:0/2103984217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f783c159890 0x7f783c15abd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f782c0098e0 tx=0x7f782c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.957+0000 7f78426a6700 1 -- 172.21.15.89:0/2103984217 shutdown_connections 2023-10-10T20:40:26.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.957+0000 7f78426a6700 1 --2- 172.21.15.89:0/2103984217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f783c159890 0x7f783c15abd0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.957+0000 7f78426a6700 1 -- 172.21.15.89:0/2103984217 >> 172.21.15.89:0/2103984217 conn(0x7f783c068e20 msgr2=0x7f783c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:26.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.957+0000 7f78426a6700 1 -- 172.21.15.89:0/2103984217 shutdown_connections 2023-10-10T20:40:26.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.957+0000 7f78426a6700 1 -- 172.21.15.89:0/2103984217 wait complete. 2023-10-10T20:40:26.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.958+0000 7f78426a6700 1 Processor -- start 2023-10-10T20:40:26.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.958+0000 7f78426a6700 1 -- start start 2023-10-10T20:40:26.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.959+0000 7f78426a6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f783c069560 0x7f783c140980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:26.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.959+0000 7f78426a6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f783c159890 0x7f783c15cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.959+0000 7f78426a6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f783c159c60 0x7f783c15d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.959+0000 7f78426a6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c0b4ae0 con 0x7f783c159890 2023-10-10T20:40:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.959+0000 7f78426a6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c0b3bf0 con 0x7f783c069560 2023-10-10T20:40:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.959+0000 7f78426a6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f783c0b3ed0 con 0x7f783c159c60 2023-10-10T20:40:26.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.959+0000 7f783b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f783c159890 0x7f783c15cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:26.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.959+0000 7f783b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f783c159890 0x7f783c15cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56822/0 (socket says 172.21.15.89:56822) 2023-10-10T20:40:26.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.959+0000 7f783b7fe700 1 -- 172.21.15.89:0/1338948509 learned_addr learned my addr 172.21.15.89:0/1338948509 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:26.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.959+0000 7f783bfff700 1 --2- 172.21.15.89:0/1338948509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f783c069560 0x7f783c140980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:26.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.959+0000 7f7840c43700 1 --2- 172.21.15.89:0/1338948509 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f783c159c60 0x7f783c15d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:26.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.959+0000 7f783b7fe700 1 -- 172.21.15.89:0/1338948509 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f783c159c60 msgr2=0x7f783c15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:26.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.959+0000 7f783b7fe700 1 --2- 172.21.15.89:0/1338948509 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f783c159c60 0x7f783c15d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.959+0000 7f783b7fe700 1 -- 172.21.15.89:0/1338948509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f783c069560 msgr2=0x7f783c140980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:26.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.959+0000 7f783b7fe700 1 --2- 172.21.15.89:0/1338948509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f783c069560 0x7f783c140980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:26.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.959+0000 7f783b7fe700 1 -- 172.21.15.89:0/1338948509 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782c009420 con 0x7f783c159890 2023-10-10T20:40:26.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.959+0000 7f783b7fe700 1 --2- 172.21.15.89:0/1338948509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f783c159890 0x7f783c15cc00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f783000c810 tx=0x7f7830007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:26.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.959+0000 7f7823fff700 1 -- 172.21.15.89:0/1338948509 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7830012630 con 0x7f783c159890 2023-10-10T20:40:26.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.960+0000 7f7823fff700 1 -- 172.21.15.89:0/1338948509 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7830012790 con 0x7f783c159890 2023-10-10T20:40:26.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.960+0000 7f78426a6700 1 -- 172.21.15.89:0/1338948509 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f783c0b41b0 con 0x7f783c159890 2023-10-10T20:40:26.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.960+0000 7f7823fff700 1 -- 172.21.15.89:0/1338948509 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7830012b50 con 0x7f783c159890 2023-10-10T20:40:26.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.960+0000 7f78426a6700 1 -- 172.21.15.89:0/1338948509 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f783c0b4770 con 0x7f783c159890 2023-10-10T20:40:26.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.961+0000 7f78426a6700 1 -- 172.21.15.89:0/1338948509 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f783c0b2aa0 con 0x7f783c159890 2023-10-10T20:40:26.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.961+0000 7f7823fff700 1 -- 172.21.15.89:0/1338948509 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f783001b400 con 0x7f783c159890 2023-10-10T20:40:26.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.961+0000 7f7823fff700 1 --2- 172.21.15.89:0/1338948509 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7828067290 0x7f7828069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:26.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.961+0000 7f7823fff700 1 -- 172.21.15.89:0/1338948509 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7830084710 con 0x7f783c159890 2023-10-10T20:40:26.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.961+0000 7f783bfff700 1 --2- 172.21.15.89:0/1338948509 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7828067290 0x7f7828069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:26.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.962+0000 7f783bfff700 1 --2- 172.21.15.89:0/1338948509 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7828067290 0x7f7828069740 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f782c0030c0 tx=0x7f782c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:26.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:26.966+0000 7f7823fff700 1 -- 172.21.15.89:0/1338948509 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78300514f0 con 0x7f783c159890 2023-10-10T20:40:27.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.192+0000 7f78426a6700 1 -- 172.21.15.89:0/1338948509 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f783c04c3c0 con 0x7f783c159890 2023-10-10T20:40:27.195 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-10-10T20:40:27.193+0000 7fd0184ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:40:27.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.196+0000 7f7823fff700 1 -- 172.21.15.89:0/1338948509 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v57) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7830054b10 con 0x7f783c159890 2023-10-10T20:40:27.199 INFO:tasks.ceph.mds.d.smithi111.stderr: -900> 2023-10-10T20:40:14.314+0000 7f5b6f48c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:40:27.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.198+0000 7f78426a6700 1 -- 172.21.15.89:0/1338948509 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7828067290 msgr2=0x7f7828069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:27.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.198+0000 7f78426a6700 1 --2- 172.21.15.89:0/1338948509 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7828067290 0x7f7828069740 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f782c0030c0 tx=0x7f782c017000 comp rx=0 tx=0).stop 2023-10-10T20:40:27.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.198+0000 7f78426a6700 1 -- 172.21.15.89:0/1338948509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f783c159890 msgr2=0x7f783c15cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:27.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.198+0000 7f78426a6700 1 --2- 172.21.15.89:0/1338948509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f783c159890 0x7f783c15cc00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f783000c810 tx=0x7f7830007f00 comp rx=0 tx=0).stop 2023-10-10T20:40:27.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.198+0000 7f78426a6700 1 -- 172.21.15.89:0/1338948509 shutdown_connections 2023-10-10T20:40:27.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.198+0000 7f78426a6700 1 --2- 172.21.15.89:0/1338948509 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f783c159c60 0x7f783c15d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:27.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.198+0000 7f78426a6700 1 --2- 172.21.15.89:0/1338948509 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7828067290 0x7f7828069740 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:27.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.198+0000 7f78426a6700 1 --2- 172.21.15.89:0/1338948509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f783c069560 0x7f783c140980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:27.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.198+0000 7f78426a6700 1 --2- 172.21.15.89:0/1338948509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f783c159890 0x7f783c15cc00 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:27.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.198+0000 7f78426a6700 1 -- 172.21.15.89:0/1338948509 >> 172.21.15.89:0/1338948509 conn(0x7f783c068e20 msgr2=0x7f783c0c6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:27.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.198+0000 7f78426a6700 1 -- 172.21.15.89:0/1338948509 shutdown_connections 2023-10-10T20:40:27.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.198+0000 7f78426a6700 1 -- 172.21.15.89:0/1338948509 wait complete. 2023-10-10T20:40:27.203 INFO:teuthology.orchestra.run.smithi089.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:40:27.213 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:40:27.222 INFO:tasks.ceph.mds.d.smithi111.stdout:starting mds.d at 2023-10-10T20:40:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.329+0000 7fa7038ad700 1 Processor -- start 2023-10-10T20:40:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.329+0000 7fa7038ad700 1 -- start start 2023-10-10T20:40:27.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.330+0000 7fa7038ad700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa6fc159890 0x7fa6fc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:27.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.330+0000 7fa7038ad700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa6fc159c60 0x7fa6fc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.330+0000 7fa7038ad700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc15a160 con 0x7fa6fc15db10 2023-10-10T20:40:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.330+0000 7fa7038ad700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6fc0d13a0 con 0x7fa6fc159c60 2023-10-10T20:40:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.330+0000 7fa7038ad700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc15e140 con 0x7fa6fc159890 2023-10-10T20:40:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.330+0000 7fa701e4a700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa6fc15db10 0x7fa6fc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36492/0 (socket says 172.21.15.89:36492) 2023-10-10T20:40:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.330+0000 7fa701e4a700 1 -- 172.21.15.89:0/3266521864 learned_addr learned my addr 172.21.15.89:0/3266521864 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.330+0000 7fa700e48700 1 --2- 172.21.15.89:0/3266521864 >> v2:172.21.15.89:3300/0 conn(0x7fa6fc159c60 0x7fa6fc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.331+0000 7fa701649700 1 --2- 172.21.15.89:0/3266521864 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa6fc159890 0x7fa6fc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.331+0000 7fa700e48700 1 -- 172.21.15.89:0/3266521864 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa6fc159890 msgr2=0x7fa6fc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.331+0000 7fa700e48700 1 --2- 172.21.15.89:0/3266521864 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa6fc159890 0x7fa6fc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.331+0000 7fa700e48700 1 -- 172.21.15.89:0/3266521864 >> v1:172.21.15.89:6789/0 conn(0x7fa6fc15db10 legacy=0x7fa6fc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:27.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.331+0000 7fa6f3fff700 1 -- 172.21.15.89:0/3266521864 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 738748571 0 0) 0x7fa6fc15a160 con 0x7fa6fc15db10 2023-10-10T20:40:27.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.331+0000 7fa700e48700 1 -- 172.21.15.89:0/3266521864 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc15a3d0 con 0x7fa6fc159c60 2023-10-10T20:40:27.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.331+0000 7fa700e48700 1 --2- 172.21.15.89:0/3266521864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6fc159c60 0x7fa6fc15d5d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0098e0 tx=0x7fa6ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effe5376843d8c10 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:27.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.331+0000 7fa6f3fff700 1 -- 172.21.15.89:0/3266521864 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec009e80 con 0x7fa6fc159c60 2023-10-10T20:40:27.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.331+0000 7fa6f3fff700 1 -- 172.21.15.89:0/3266521864 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ec00b860 con 0x7fa6fc159c60 2023-10-10T20:40:27.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.331+0000 7fa6f3fff700 1 -- 172.21.15.89:0/3266521864 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec00bbe0 con 0x7fa6fc159c60 2023-10-10T20:40:27.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.332+0000 7fa7038ad700 1 -- 172.21.15.89:0/3266521864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6fc159c60 msgr2=0x7fa6fc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:27.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.332+0000 7fa7038ad700 1 --2- 172.21.15.89:0/3266521864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6fc159c60 0x7fa6fc15d5d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0098e0 tx=0x7fa6ec0091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:27.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.332+0000 7fa7038ad700 1 -- 172.21.15.89:0/3266521864 shutdown_connections 2023-10-10T20:40:27.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.332+0000 7fa7038ad700 1 --2- 172.21.15.89:0/3266521864 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa6fc159890 0x7fa6fc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:27.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.332+0000 7fa7038ad700 1 --2- 172.21.15.89:0/3266521864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6fc159c60 0x7fa6fc15d5d0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:27.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.332+0000 7fa7038ad700 1 -- 172.21.15.89:0/3266521864 >> 172.21.15.89:0/3266521864 conn(0x7fa6fc068e20 msgr2=0x7fa6fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:27.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.332+0000 7fa7038ad700 1 -- 172.21.15.89:0/3266521864 shutdown_connections 2023-10-10T20:40:27.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.332+0000 7fa7038ad700 1 -- 172.21.15.89:0/3266521864 wait complete. 2023-10-10T20:40:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.333+0000 7fa7038ad700 1 Processor -- start 2023-10-10T20:40:27.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.333+0000 7fa7038ad700 1 -- start start 2023-10-10T20:40:27.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa7038ad700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6fc159890 0x7fa6fc0b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:27.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa7038ad700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa6fc159c60 0x7fa6fc1472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:27.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa7038ad700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6fc15db10 0x7fa6fc0cf810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:27.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa7038ad700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc147830 con 0x7fa6fc15db10 2023-10-10T20:40:27.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa7038ad700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0cfd50 con 0x7fa6fc159890 2023-10-10T20:40:27.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa7038ad700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0b5b00 con 0x7fa6fc159c60 2023-10-10T20:40:27.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa701e4a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6fc15db10 0x7fa6fc0cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:27.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa701e4a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6fc15db10 0x7fa6fc0cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56830/0 (socket says 172.21.15.89:56830) 2023-10-10T20:40:27.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa701e4a700 1 -- 172.21.15.89:0/511153822 learned_addr learned my addr 172.21.15.89:0/511153822 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:27.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa700e48700 1 --2- 172.21.15.89:0/511153822 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa6fc159c60 0x7fa6fc1472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:27.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa701649700 1 --2- 172.21.15.89:0/511153822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6fc159890 0x7fa6fc0b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:27.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa701e4a700 1 -- 172.21.15.89:0/511153822 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa6fc159c60 msgr2=0x7fa6fc1472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:27.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa701e4a700 1 --2- 172.21.15.89:0/511153822 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa6fc159c60 0x7fa6fc1472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:27.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa701e4a700 1 -- 172.21.15.89:0/511153822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6fc159890 msgr2=0x7fa6fc0b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa701e4a700 1 --2- 172.21.15.89:0/511153822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6fc159890 0x7fa6fc0b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa701e4a700 1 -- 172.21.15.89:0/511153822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec009420 con 0x7fa6fc15db10 2023-10-10T20:40:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa701e4a700 1 --2- 172.21.15.89:0/511153822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6fc15db10 0x7fa6fc0cf810 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400cbf0 tx=0x7fa6f40019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa6f17fa700 1 -- 172.21.15.89:0/511153822 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400a850 con 0x7fa6fc15db10 2023-10-10T20:40:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa6f17fa700 1 -- 172.21.15.89:0/511153822 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f400a9b0 con 0x7fa6fc15db10 2023-10-10T20:40:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa7038ad700 1 -- 172.21.15.89:0/511153822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6fc0b51d0 con 0x7fa6fc15db10 2023-10-10T20:40:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa6f17fa700 1 -- 172.21.15.89:0/511153822 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400ad70 con 0x7fa6fc15db10 2023-10-10T20:40:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.334+0000 7fa7038ad700 1 -- 172.21.15.89:0/511153822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6fc0b5790 con 0x7fa6fc15db10 2023-10-10T20:40:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.335+0000 7fa7038ad700 1 -- 172.21.15.89:0/511153822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6fc0b64d0 con 0x7fa6fc15db10 2023-10-10T20:40:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.336+0000 7fa6f17fa700 1 -- 172.21.15.89:0/511153822 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fa6f4007300 con 0x7fa6fc15db10 2023-10-10T20:40:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.336+0000 7fa6f17fa700 1 --2- 172.21.15.89:0/511153822 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa6e0067290 0x7fa6e0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.336+0000 7fa6f17fa700 1 -- 172.21.15.89:0/511153822 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fa6f4084e00 con 0x7fa6fc15db10 2023-10-10T20:40:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.336+0000 7fa701649700 1 --2- 172.21.15.89:0/511153822 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa6e0067290 0x7fa6e0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.336+0000 7fa701649700 1 --2- 172.21.15.89:0/511153822 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa6e0067290 0x7fa6e0069740 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8006fd0 tx=0x7fa6f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:27.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.340+0000 7fa6f17fa700 1 -- 172.21.15.89:0/511153822 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6f4051b90 con 0x7fa6fc15db10 2023-10-10T20:40:27.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:27.567+0000 7fa7038ad700 1 -- 172.21.15.89:0/511153822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa6fc0b5c70 con 0x7fa6fc15db10 2023-10-10T20:40:28.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.197+0000 7fa6f17fa700 1 -- 172.21.15.89:0/511153822 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v58) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa6f40551b0 con 0x7fa6fc15db10 2023-10-10T20:40:28.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.199+0000 7fa7038ad700 1 -- 172.21.15.89:0/511153822 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa6e0067290 msgr2=0x7fa6e0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:28.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.199+0000 7fa7038ad700 1 --2- 172.21.15.89:0/511153822 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa6e0067290 0x7fa6e0069740 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8006fd0 tx=0x7fa6f8008040 comp rx=0 tx=0).stop 2023-10-10T20:40:28.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.199+0000 7fa7038ad700 1 -- 172.21.15.89:0/511153822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6fc15db10 msgr2=0x7fa6fc0cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:28.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.199+0000 7fa7038ad700 1 --2- 172.21.15.89:0/511153822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6fc15db10 0x7fa6fc0cf810 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400cbf0 tx=0x7fa6f40019b0 comp rx=0 tx=0).stop 2023-10-10T20:40:28.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.199+0000 7fa7038ad700 1 -- 172.21.15.89:0/511153822 shutdown_connections 2023-10-10T20:40:28.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.199+0000 7fa7038ad700 1 --2- 172.21.15.89:0/511153822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6fc15db10 0x7fa6fc0cf810 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:28.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.199+0000 7fa7038ad700 1 --2- 172.21.15.89:0/511153822 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa6e0067290 0x7fa6e0069740 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:28.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.199+0000 7fa7038ad700 1 --2- 172.21.15.89:0/511153822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6fc159890 0x7fa6fc0b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:28.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.199+0000 7fa7038ad700 1 --2- 172.21.15.89:0/511153822 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa6fc159c60 0x7fa6fc1472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:28.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.199+0000 7fa7038ad700 1 -- 172.21.15.89:0/511153822 >> 172.21.15.89:0/511153822 conn(0x7fa6fc068e20 msgr2=0x7fa6fc1530e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:28.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.199+0000 7fa7038ad700 1 -- 172.21.15.89:0/511153822 shutdown_connections 2023-10-10T20:40:28.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.199+0000 7fa7038ad700 1 -- 172.21.15.89:0/511153822 wait complete. 2023-10-10T20:40:28.214 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:40:28.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.332+0000 7f827eb4c700 1 Processor -- start 2023-10-10T20:40:28.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.332+0000 7f827eb4c700 1 -- start start 2023-10-10T20:40:28.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.333+0000 7f827eb4c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8278068aa0 0x7f8278162460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:28.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.333+0000 7f827eb4c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82780ae6d0 con 0x7f8278157740 2023-10-10T20:40:28.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.333+0000 7f827eb4c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f827806dc20 con 0x7f82780b2100 2023-10-10T20:40:28.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.333+0000 7f827eb4c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82780acc00 con 0x7f8278068aa0 2023-10-10T20:40:28.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.333+0000 7f827c8e8700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8278157740 0x7f82780adfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38996/0 (socket says 172.21.15.89:38996) 2023-10-10T20:40:28.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.333+0000 7f827c8e8700 1 -- 172.21.15.89:0/1112742820 learned_addr learned my addr 172.21.15.89:0/1112742820 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:28.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.333+0000 7f827d0e9700 1 --2- 172.21.15.89:0/1112742820 >> v2:172.21.15.89:3300/0 conn(0x7f8278068aa0 0x7f8278162460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:28.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.334+0000 7f826f7fe700 1 -- 172.21.15.89:0/1112742820 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3810947288 0 0) 0x7f8274001110 con 0x7f8278157740 2023-10-10T20:40:28.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.334+0000 7f826f7fe700 1 -- 172.21.15.89:0/1112742820 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8260003570 con 0x7f8278157740 2023-10-10T20:40:28.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.334+0000 7f827d0e9700 1 -- 172.21.15.89:0/1112742820 >> v1:172.21.15.111:6789/0 conn(0x7f82780b2100 legacy=0x7f8278068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:28.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.334+0000 7f827d0e9700 1 -- 172.21.15.89:0/1112742820 >> v1:172.21.15.89:6789/0 conn(0x7f8278157740 legacy=0x7f82780adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:28.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.334+0000 7f827d0e9700 1 -- 172.21.15.89:0/1112742820 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82780ae910 con 0x7f8278068aa0 2023-10-10T20:40:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.334+0000 7f827d0e9700 1 --2- 172.21.15.89:0/1112742820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8278068aa0 0x7f8278162460 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8270009e60 tx=0x7f8270009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c271732c466f63 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.334+0000 7f826f7fe700 1 -- 172.21.15.89:0/1112742820 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827000e070 con 0x7f8278068aa0 2023-10-10T20:40:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.334+0000 7f826f7fe700 1 -- 172.21.15.89:0/1112742820 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827000bb20 con 0x7f8278068aa0 2023-10-10T20:40:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.334+0000 7f826f7fe700 1 -- 172.21.15.89:0/1112742820 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8270012430 con 0x7f8278068aa0 2023-10-10T20:40:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.334+0000 7f827eb4c700 1 -- 172.21.15.89:0/1112742820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8278068aa0 msgr2=0x7f8278162460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:28.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.334+0000 7f827eb4c700 1 --2- 172.21.15.89:0/1112742820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8278068aa0 0x7f8278162460 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8270009e60 tx=0x7f8270009780 comp rx=0 tx=0).stop 2023-10-10T20:40:28.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.334+0000 7f827eb4c700 1 -- 172.21.15.89:0/1112742820 shutdown_connections 2023-10-10T20:40:28.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.334+0000 7f827eb4c700 1 --2- 172.21.15.89:0/1112742820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8278068aa0 0x7f8278162460 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:28.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.334+0000 7f827eb4c700 1 -- 172.21.15.89:0/1112742820 >> 172.21.15.89:0/1112742820 conn(0x7f82780c8f90 msgr2=0x7f82780c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:28.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.334+0000 7f827eb4c700 1 -- 172.21.15.89:0/1112742820 shutdown_connections 2023-10-10T20:40:28.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.334+0000 7f827eb4c700 1 -- 172.21.15.89:0/1112742820 wait complete. 2023-10-10T20:40:28.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.335+0000 7f827eb4c700 1 Processor -- start 2023-10-10T20:40:28.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.336+0000 7f827eb4c700 1 -- start start 2023-10-10T20:40:28.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.336+0000 7f827eb4c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8278068aa0 0x7f827815af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:28.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.336+0000 7f827eb4c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82780b2100 0x7f8278149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:28.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.336+0000 7f827eb4c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8278157740 0x7f82780cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:28.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.336+0000 7f827eb4c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827815c390 con 0x7f82780b2100 2023-10-10T20:40:28.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.336+0000 7f827eb4c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827815b4b0 con 0x7f8278068aa0 2023-10-10T20:40:28.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.336+0000 7f827eb4c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827815b780 con 0x7f8278157740 2023-10-10T20:40:28.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.337+0000 7f826ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82780b2100 0x7f8278149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:28.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.337+0000 7f826ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82780b2100 0x7f8278149870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34050/0 (socket says 172.21.15.89:34050) 2023-10-10T20:40:28.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.337+0000 7f826ffff700 1 -- 172.21.15.89:0/1009868449 learned_addr learned my addr 172.21.15.89:0/1009868449 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.337+0000 7f826ffff700 1 -- 172.21.15.89:0/1009868449 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8278157740 msgr2=0x7f82780cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.337+0000 7f827c8e8700 1 --2- 172.21.15.89:0/1009868449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8278068aa0 0x7f827815af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.337+0000 7f827d0e9700 1 --2- 172.21.15.89:0/1009868449 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8278157740 0x7f82780cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.337+0000 7f826ffff700 1 --2- 172.21.15.89:0/1009868449 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8278157740 0x7f82780cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.337+0000 7f826ffff700 1 -- 172.21.15.89:0/1009868449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8278068aa0 msgr2=0x7f827815af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:28.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.337+0000 7f826ffff700 1 --2- 172.21.15.89:0/1009868449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8278068aa0 0x7f827815af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:28.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.337+0000 7f826ffff700 1 -- 172.21.15.89:0/1009868449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8270009a20 con 0x7f82780b2100 2023-10-10T20:40:28.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.337+0000 7f826ffff700 1 --2- 172.21.15.89:0/1009868449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82780b2100 0x7f8278149870 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f826800c5d0 tx=0x7f8268007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:28.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.337+0000 7f826cff9700 1 -- 172.21.15.89:0/1009868449 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8268011590 con 0x7f82780b2100 2023-10-10T20:40:28.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.337+0000 7f826cff9700 1 -- 172.21.15.89:0/1009868449 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8268011710 con 0x7f82780b2100 2023-10-10T20:40:28.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.337+0000 7f826cff9700 1 -- 172.21.15.89:0/1009868449 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8268011ad0 con 0x7f82780b2100 2023-10-10T20:40:28.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.337+0000 7f827eb4c700 1 -- 172.21.15.89:0/1009868449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827815ba60 con 0x7f82780b2100 2023-10-10T20:40:28.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.337+0000 7f827eb4c700 1 -- 172.21.15.89:0/1009868449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827815c020 con 0x7f82780b2100 2023-10-10T20:40:28.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.338+0000 7f827eb4c700 1 -- 172.21.15.89:0/1009868449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82780718b0 con 0x7f82780b2100 2023-10-10T20:40:28.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.339+0000 7f826cff9700 1 -- 172.21.15.89:0/1009868449 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f826801c460 con 0x7f82780b2100 2023-10-10T20:40:28.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.339+0000 7f826cff9700 1 --2- 172.21.15.89:0/1009868449 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8264067290 0x7f8264069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:28.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.339+0000 7f827c8e8700 1 --2- 172.21.15.89:0/1009868449 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8264067290 0x7f8264069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:28.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.339+0000 7f826cff9700 1 -- 172.21.15.89:0/1009868449 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f82680848c0 con 0x7f82780b2100 2023-10-10T20:40:28.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.339+0000 7f827c8e8700 1 --2- 172.21.15.89:0/1009868449 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8264067290 0x7f8264069740 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f82740058d0 tx=0x7f8274009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:28.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.343+0000 7f826cff9700 1 -- 172.21.15.89:0/1009868449 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82680516d0 con 0x7f82780b2100 2023-10-10T20:40:28.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:28.526+0000 7f827eb4c700 1 -- 172.21.15.89:0/1009868449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f827806f810 con 0x7f82780b2100 2023-10-10T20:40:29.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.200+0000 7f826cff9700 1 -- 172.21.15.89:0/1009868449 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8268054cf0 con 0x7f82780b2100 2023-10-10T20:40:29.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.203+0000 7f827eb4c700 1 -- 172.21.15.89:0/1009868449 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8264067290 msgr2=0x7f8264069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:29.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.203+0000 7f827eb4c700 1 --2- 172.21.15.89:0/1009868449 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8264067290 0x7f8264069740 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f82740058d0 tx=0x7f8274009040 comp rx=0 tx=0).stop 2023-10-10T20:40:29.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.203+0000 7f827eb4c700 1 -- 172.21.15.89:0/1009868449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82780b2100 msgr2=0x7f8278149870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:29.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.203+0000 7f827eb4c700 1 --2- 172.21.15.89:0/1009868449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82780b2100 0x7f8278149870 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f826800c5d0 tx=0x7f8268007f60 comp rx=0 tx=0).stop 2023-10-10T20:40:29.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.203+0000 7f827eb4c700 1 -- 172.21.15.89:0/1009868449 shutdown_connections 2023-10-10T20:40:29.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.203+0000 7f827eb4c700 1 --2- 172.21.15.89:0/1009868449 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8278157740 0x7f82780cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:29.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.203+0000 7f827eb4c700 1 --2- 172.21.15.89:0/1009868449 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8264067290 0x7f8264069740 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:29.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.203+0000 7f827eb4c700 1 --2- 172.21.15.89:0/1009868449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8278068aa0 0x7f827815af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:29.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.203+0000 7f827eb4c700 1 --2- 172.21.15.89:0/1009868449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82780b2100 0x7f8278149870 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:29.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.203+0000 7f827eb4c700 1 -- 172.21.15.89:0/1009868449 >> 172.21.15.89:0/1009868449 conn(0x7f82780c8f90 msgr2=0x7f82780cb350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:29.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.203+0000 7f827eb4c700 1 -- 172.21.15.89:0/1009868449 shutdown_connections 2023-10-10T20:40:29.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.203+0000 7f827eb4c700 1 -- 172.21.15.89:0/1009868449 wait complete. 2023-10-10T20:40:29.208 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:40:29.219 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:40:29.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.340+0000 7f0b7f41a700 1 Processor -- start 2023-10-10T20:40:29.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.341+0000 7f0b7f41a700 1 -- start start 2023-10-10T20:40:29.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.341+0000 7f0b7f41a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0b78159ec0 0x7f0b780b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:29.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.341+0000 7f0b7f41a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0b780cf350 0x7f0b780cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:29.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.341+0000 7f0b7f41a700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b780b5b00 con 0x7f0b780b53d0 2023-10-10T20:40:29.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.341+0000 7f0b7f41a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b780b5850 con 0x7f0b78159ec0 2023-10-10T20:40:29.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.341+0000 7f0b7f41a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b780b5da0 con 0x7f0b780cf350 2023-10-10T20:40:29.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.342+0000 7f0b7d1b6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0b78159ec0 0x7f0b780b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:29.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.342+0000 7f0b7d1b6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0b78159ec0 0x7f0b780b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34052/0 (socket says 172.21.15.89:34052) 2023-10-10T20:40:29.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.342+0000 7f0b7d1b6700 1 -- 172.21.15.89:0/1399760159 learned_addr learned my addr 172.21.15.89:0/1399760159 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:29.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.342+0000 7f0b7d9b7700 1 --2- 172.21.15.89:0/1399760159 >> v2:172.21.15.111:3300/0 conn(0x7f0b780cf350 0x7f0b780cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:29.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.342+0000 7f0b7d1b6700 1 -- 172.21.15.89:0/1399760159 >> v2:172.21.15.111:3300/0 conn(0x7f0b780cf350 msgr2=0x7f0b780cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:29.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.342+0000 7f0b7d1b6700 1 --2- 172.21.15.89:0/1399760159 >> v2:172.21.15.111:3300/0 conn(0x7f0b780cf350 0x7f0b780cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:29.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.342+0000 7f0b7d1b6700 1 -- 172.21.15.89:0/1399760159 >> v1:172.21.15.111:6789/0 conn(0x7f0b780b53d0 legacy=0x7f0b780cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:29.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.342+0000 7f0b7d1b6700 1 -- 172.21.15.89:0/1399760159 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b780b6080 con 0x7f0b78159ec0 2023-10-10T20:40:29.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.342+0000 7f0b7d1b6700 1 --2- 172.21.15.89:0/1399760159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b78159ec0 0x7f0b780b4e90 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0b780cb4a0 tx=0x7f0b74009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91e8dfa657fa91c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:29.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.342+0000 7f0b6ffff700 1 -- 172.21.15.89:0/1399760159 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74005c10 con 0x7f0b78159ec0 2023-10-10T20:40:29.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.342+0000 7f0b6ffff700 1 -- 172.21.15.89:0/1399760159 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b74005d70 con 0x7f0b78159ec0 2023-10-10T20:40:29.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.342+0000 7f0b6ffff700 1 -- 172.21.15.89:0/1399760159 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7400b880 con 0x7f0b78159ec0 2023-10-10T20:40:29.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.343+0000 7f0b7f41a700 1 -- 172.21.15.89:0/1399760159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b78159ec0 msgr2=0x7f0b780b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:29.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.343+0000 7f0b7f41a700 1 --2- 172.21.15.89:0/1399760159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b78159ec0 0x7f0b780b4e90 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0b780cb4a0 tx=0x7f0b74009220 comp rx=0 tx=0).stop 2023-10-10T20:40:29.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.343+0000 7f0b7f41a700 1 -- 172.21.15.89:0/1399760159 shutdown_connections 2023-10-10T20:40:29.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.343+0000 7f0b7f41a700 1 --2- 172.21.15.89:0/1399760159 >> v2:172.21.15.111:3300/0 conn(0x7f0b780cf350 0x7f0b780cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:29.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.343+0000 7f0b7f41a700 1 --2- 172.21.15.89:0/1399760159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b78159ec0 0x7f0b780b4e90 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:29.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.343+0000 7f0b7f41a700 1 -- 172.21.15.89:0/1399760159 >> 172.21.15.89:0/1399760159 conn(0x7f0b780999f0 msgr2=0x7f0b780b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:29.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.343+0000 7f0b7f41a700 1 -- 172.21.15.89:0/1399760159 shutdown_connections 2023-10-10T20:40:29.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.343+0000 7f0b7f41a700 1 -- 172.21.15.89:0/1399760159 wait complete. 2023-10-10T20:40:29.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.344+0000 7f0b7f41a700 1 Processor -- start 2023-10-10T20:40:29.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.344+0000 7f0b7f41a700 1 -- start start 2023-10-10T20:40:29.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b7f41a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0b780b53d0 0x7f0b78158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:29.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b7f41a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b780cf350 0x7f0b78159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:29.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b7f41a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b78159ec0 0x7f0b78159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:29.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b7f41a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b78158890 con 0x7f0b78159ec0 2023-10-10T20:40:29.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b7f41a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7815dc30 con 0x7f0b780cf350 2023-10-10T20:40:29.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b7f41a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7815df10 con 0x7f0b780b53d0 2023-10-10T20:40:29.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b7d9b7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b78159ec0 0x7f0b78159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:29.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b7d9b7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b78159ec0 0x7f0b78159570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34058/0 (socket says 172.21.15.89:34058) 2023-10-10T20:40:29.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b7d9b7700 1 -- 172.21.15.89:0/333168354 learned_addr learned my addr 172.21.15.89:0/333168354 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:29.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b7d1b6700 1 --2- 172.21.15.89:0/333168354 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0b780b53d0 0x7f0b78158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:29.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b7c9b5700 1 --2- 172.21.15.89:0/333168354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b780cf350 0x7f0b78159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:29.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b7d9b7700 1 -- 172.21.15.89:0/333168354 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0b780b53d0 msgr2=0x7f0b78158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:29.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b7d9b7700 1 --2- 172.21.15.89:0/333168354 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0b780b53d0 0x7f0b78158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b7d9b7700 1 -- 172.21.15.89:0/333168354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b780cf350 msgr2=0x7f0b78159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b7d9b7700 1 --2- 172.21.15.89:0/333168354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b780cf350 0x7f0b78159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b7d9b7700 1 -- 172.21.15.89:0/333168354 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b74009480 con 0x7f0b78159ec0 2023-10-10T20:40:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b7d9b7700 1 --2- 172.21.15.89:0/333168354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b78159ec0 0x7f0b78159570 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009fa0 tx=0x7f0b70009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:29.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b6d7fa700 1 -- 172.21.15.89:0/333168354 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7000ebc0 con 0x7f0b78159ec0 2023-10-10T20:40:29.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b6d7fa700 1 -- 172.21.15.89:0/333168354 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7000ed20 con 0x7f0b78159ec0 2023-10-10T20:40:29.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b6d7fa700 1 -- 172.21.15.89:0/333168354 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b70014670 con 0x7f0b78159ec0 2023-10-10T20:40:29.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.345+0000 7f0b7f41a700 1 -- 172.21.15.89:0/333168354 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b7815e1f0 con 0x7f0b78159ec0 2023-10-10T20:40:29.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.346+0000 7f0b7f41a700 1 -- 172.21.15.89:0/333168354 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b7815ced0 con 0x7f0b78159ec0 2023-10-10T20:40:29.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.346+0000 7f0b7f41a700 1 -- 172.21.15.89:0/333168354 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b78063c90 con 0x7f0b78159ec0 2023-10-10T20:40:29.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.347+0000 7f0b6d7fa700 1 -- 172.21.15.89:0/333168354 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f0b700149e0 con 0x7f0b78159ec0 2023-10-10T20:40:29.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.347+0000 7f0b6d7fa700 1 --2- 172.21.15.89:0/333168354 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0b5c067290 0x7f0b5c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:29.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.347+0000 7f0b6d7fa700 1 -- 172.21.15.89:0/333168354 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f0b70084ff0 con 0x7f0b78159ec0 2023-10-10T20:40:29.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.347+0000 7f0b7d1b6700 1 --2- 172.21.15.89:0/333168354 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0b5c067290 0x7f0b5c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:29.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.348+0000 7f0b7d1b6700 1 --2- 172.21.15.89:0/333168354 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0b5c067290 0x7f0b5c069740 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0b78063780 tx=0x7f0b74016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:29.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.351+0000 7f0b6d7fa700 1 -- 172.21.15.89:0/333168354 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b70051f70 con 0x7f0b78159ec0 2023-10-10T20:40:29.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:29.534+0000 7f0b7f41a700 1 -- 172.21.15.89:0/333168354 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0b780718b0 con 0x7f0b78159ec0 2023-10-10T20:40:30.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.218+0000 7f0b6d7fa700 1 -- 172.21.15.89:0/333168354 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0b70055590 con 0x7f0b78159ec0 2023-10-10T20:40:30.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.221+0000 7f0b7f41a700 1 -- 172.21.15.89:0/333168354 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0b5c067290 msgr2=0x7f0b5c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:30.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.221+0000 7f0b7f41a700 1 --2- 172.21.15.89:0/333168354 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0b5c067290 0x7f0b5c069740 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0b78063780 tx=0x7f0b74016000 comp rx=0 tx=0).stop 2023-10-10T20:40:30.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.221+0000 7f0b7f41a700 1 -- 172.21.15.89:0/333168354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b78159ec0 msgr2=0x7f0b78159570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:30.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.221+0000 7f0b7f41a700 1 --2- 172.21.15.89:0/333168354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b78159ec0 0x7f0b78159570 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009fa0 tx=0x7f0b70009900 comp rx=0 tx=0).stop 2023-10-10T20:40:30.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.222+0000 7f0b7f41a700 1 -- 172.21.15.89:0/333168354 shutdown_connections 2023-10-10T20:40:30.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.222+0000 7f0b7f41a700 1 --2- 172.21.15.89:0/333168354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b78159ec0 0x7f0b78159570 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:30.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.222+0000 7f0b7f41a700 1 --2- 172.21.15.89:0/333168354 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0b780b53d0 0x7f0b78158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:30.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.222+0000 7f0b7f41a700 1 --2- 172.21.15.89:0/333168354 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0b5c067290 0x7f0b5c069740 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:30.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.222+0000 7f0b7f41a700 1 --2- 172.21.15.89:0/333168354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b780cf350 0x7f0b78159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:30.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.222+0000 7f0b7f41a700 1 -- 172.21.15.89:0/333168354 >> 172.21.15.89:0/333168354 conn(0x7f0b780999f0 msgr2=0x7f0b780cdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:30.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.222+0000 7f0b7f41a700 1 -- 172.21.15.89:0/333168354 shutdown_connections 2023-10-10T20:40:30.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.222+0000 7f0b7f41a700 1 -- 172.21.15.89:0/333168354 wait complete. 2023-10-10T20:40:30.227 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' removed 2023-10-10T20:40:30.237 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:40:30.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.352+0000 7f0a5d7e9700 1 Processor -- start 2023-10-10T20:40:30.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.353+0000 7f0a5d7e9700 1 -- start start 2023-10-10T20:40:30.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.353+0000 7f0a5d7e9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a58159c60 0x7f0a5815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:30.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.353+0000 7f0a5d7e9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a5815a160 con 0x7f0a58159890 2023-10-10T20:40:30.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.353+0000 7f0a5d7e9700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a580d13a0 con 0x7f0a5815db10 2023-10-10T20:40:30.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.353+0000 7f0a5d7e9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5815e140 con 0x7f0a58159c60 2023-10-10T20:40:30.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.353+0000 7f0a56ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0a58159890 0x7f0a5815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39002/0 (socket says 172.21.15.89:39002) 2023-10-10T20:40:30.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.353+0000 7f0a56ffd700 1 -- 172.21.15.89:0/2969769342 learned_addr learned my addr 172.21.15.89:0/2969769342 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:30.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a4e7fc700 1 --2- 172.21.15.89:0/2969769342 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a58159c60 0x7f0a5815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:30.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a567fc700 1 -- 172.21.15.89:0/2969769342 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073349071 0 0) 0x7f0a5815a160 con 0x7f0a58159890 2023-10-10T20:40:30.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a567fc700 1 -- 172.21.15.89:0/2969769342 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a44003570 con 0x7f0a58159890 2023-10-10T20:40:30.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a567fc700 1 -- 172.21.15.89:0/2969769342 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3281879082 0 0) 0x7f0a580d13a0 con 0x7f0a5815db10 2023-10-10T20:40:30.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a567fc700 1 -- 172.21.15.89:0/2969769342 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a5815a160 con 0x7f0a5815db10 2023-10-10T20:40:30.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a567fc700 1 -- 172.21.15.89:0/2969769342 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 783961815 0 0) 0x7f0a44003570 con 0x7f0a58159890 2023-10-10T20:40:30.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a567fc700 1 -- 172.21.15.89:0/2969769342 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a58159c60 msgr2=0x7f0a5815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:30.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a567fc700 1 --2- 172.21.15.89:0/2969769342 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a58159c60 0x7f0a5815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:30.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a567fc700 1 -- 172.21.15.89:0/2969769342 >> v1:172.21.15.111:6789/0 conn(0x7f0a5815db10 legacy=0x7f0a58069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:30.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a567fc700 1 -- 172.21.15.89:0/2969769342 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5815a3d0 con 0x7f0a58159890 2023-10-10T20:40:30.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a567fc700 1 -- 172.21.15.89:0/2969769342 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f0a48001470 con 0x7f0a58159890 2023-10-10T20:40:30.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a567fc700 1 -- 172.21.15.89:0/2969769342 >> v1:172.21.15.89:6789/0 conn(0x7f0a58159890 legacy=0x7f0a5815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:30.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a567fc700 1 --2- 172.21.15.89:0/2969769342 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a58159c60 0x7f0a44003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:30.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a567fc700 1 --2- 172.21.15.89:0/2969769342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a44004180 0x7f0a44004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:30.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a567fc700 1 --2- 172.21.15.89:0/2969769342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a44004d50 0x7f0a440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:30.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a567fc700 1 -- 172.21.15.89:0/2969769342 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5815e140 con 0x7f0a44004180 2023-10-10T20:40:30.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a567fc700 1 -- 172.21.15.89:0/2969769342 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4400bba0 con 0x7f0a44004d50 2023-10-10T20:40:30.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a567fc700 1 -- 172.21.15.89:0/2969769342 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4400bd60 con 0x7f0a58159c60 2023-10-10T20:40:30.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a4e7fc700 1 --2- 172.21.15.89:0/2969769342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a44004180 0x7f0a44004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:30.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a56ffd700 1 --2- 172.21.15.89:0/2969769342 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a58159c60 0x7f0a44003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:30.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a577fe700 1 --2- 172.21.15.89:0/2969769342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a44004d50 0x7f0a440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:30.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a4e7fc700 1 -- 172.21.15.89:0/2969769342 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a58159c60 msgr2=0x7f0a44003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:30.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a4e7fc700 1 --2- 172.21.15.89:0/2969769342 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a58159c60 0x7f0a44003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:30.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a4e7fc700 1 -- 172.21.15.89:0/2969769342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a44004d50 msgr2=0x7f0a440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:30.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a4e7fc700 1 --2- 172.21.15.89:0/2969769342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a44004d50 0x7f0a440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:30.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a4e7fc700 1 -- 172.21.15.89:0/2969769342 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a44003570 con 0x7f0a44004180 2023-10-10T20:40:30.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.354+0000 7f0a577fe700 1 --2- 172.21.15.89:0/2969769342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a44004d50 0x7f0a440051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:30.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.355+0000 7f0a4e7fc700 1 --2- 172.21.15.89:0/2969769342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a44004180 0x7f0a44004810 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f0a5815a090 tx=0x7f0a400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6b12c13f8f9409 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:30.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.355+0000 7f0a567fc700 1 -- 172.21.15.89:0/2969769342 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4000e710 con 0x7f0a44004180 2023-10-10T20:40:30.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.355+0000 7f0a567fc700 1 -- 172.21.15.89:0/2969769342 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4000e870 con 0x7f0a44004180 2023-10-10T20:40:30.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.355+0000 7f0a567fc700 1 -- 172.21.15.89:0/2969769342 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4000ec30 con 0x7f0a44004180 2023-10-10T20:40:30.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.355+0000 7f0a5d7e9700 1 -- 172.21.15.89:0/2969769342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a44004180 msgr2=0x7f0a44004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:30.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.355+0000 7f0a5d7e9700 1 --2- 172.21.15.89:0/2969769342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a44004180 0x7f0a44004810 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f0a5815a090 tx=0x7f0a400091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:30.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.355+0000 7f0a577fe700 1 -- 172.21.15.89:0/2969769342 reap_dead start 2023-10-10T20:40:30.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.355+0000 7f0a5d7e9700 1 -- 172.21.15.89:0/2969769342 shutdown_connections 2023-10-10T20:40:30.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.355+0000 7f0a5d7e9700 1 -- 172.21.15.89:0/2969769342 >> 172.21.15.89:0/2969769342 conn(0x7f0a58068e20 msgr2=0x7f0a580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:30.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.355+0000 7f0a5d7e9700 1 -- 172.21.15.89:0/2969769342 shutdown_connections 2023-10-10T20:40:30.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.355+0000 7f0a5d7e9700 1 -- 172.21.15.89:0/2969769342 wait complete. 2023-10-10T20:40:30.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.356+0000 7f0a5d7e9700 1 Processor -- start 2023-10-10T20:40:30.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.357+0000 7f0a5d7e9700 1 -- start start 2023-10-10T20:40:30.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.357+0000 7f0a5d7e9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a44004180 0x7f0a5816a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:30.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.357+0000 7f0a5d7e9700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a580cd0c0 0x7f0a580b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:30.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.357+0000 7f0a5d7e9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a5816db00 0x7f0a5816df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:30.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.357+0000 7f0a5d7e9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5816adb0 con 0x7f0a44004180 2023-10-10T20:40:30.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.357+0000 7f0a5d7e9700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a580c7950 con 0x7f0a580cd0c0 2023-10-10T20:40:30.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.357+0000 7f0a5d7e9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5816d390 con 0x7f0a5816db00 2023-10-10T20:40:30.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.357+0000 7f0a56ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a44004180 0x7f0a5816a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:30.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.357+0000 7f0a56ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a44004180 0x7f0a5816a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34068/0 (socket says 172.21.15.89:34068) 2023-10-10T20:40:30.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.357+0000 7f0a56ffd700 1 -- 172.21.15.89:0/2089952329 learned_addr learned my addr 172.21.15.89:0/2089952329 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:30.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.357+0000 7f0a577fe700 1 --2- 172.21.15.89:0/2089952329 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a5816db00 0x7f0a5816df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:30.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.358+0000 7f0a4e7fc700 1 --2- 172.21.15.89:0/2089952329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a580cd0c0 0x7f0a580b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:30.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.358+0000 7f0a56ffd700 1 -- 172.21.15.89:0/2089952329 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a5816db00 msgr2=0x7f0a5816df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:30.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.358+0000 7f0a56ffd700 1 --2- 172.21.15.89:0/2089952329 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a5816db00 0x7f0a5816df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:30.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.358+0000 7f0a56ffd700 1 -- 172.21.15.89:0/2089952329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a580cd0c0 msgr2=0x7f0a580b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:30.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.358+0000 7f0a56ffd700 1 --2- 172.21.15.89:0/2089952329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a580cd0c0 0x7f0a580b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:30.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.358+0000 7f0a56ffd700 1 -- 172.21.15.89:0/2089952329 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5816d670 con 0x7f0a44004180 2023-10-10T20:40:30.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.358+0000 7f0a4e7fc700 1 --2- 172.21.15.89:0/2089952329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a580cd0c0 0x7f0a580b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:30.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.358+0000 7f0a56ffd700 1 --2- 172.21.15.89:0/2089952329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a44004180 0x7f0a5816a870 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f0a480073f0 tx=0x7f0a48001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:30.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.358+0000 7f0a4f7fe700 1 -- 172.21.15.89:0/2089952329 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48018030 con 0x7f0a44004180 2023-10-10T20:40:30.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.358+0000 7f0a4f7fe700 1 -- 172.21.15.89:0/2089952329 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a48001040 con 0x7f0a44004180 2023-10-10T20:40:30.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.358+0000 7f0a5d7e9700 1 -- 172.21.15.89:0/2089952329 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a580b5f70 con 0x7f0a44004180 2023-10-10T20:40:30.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.358+0000 7f0a4f7fe700 1 -- 172.21.15.89:0/2089952329 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4800fb20 con 0x7f0a44004180 2023-10-10T20:40:30.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.358+0000 7f0a5d7e9700 1 -- 172.21.15.89:0/2089952329 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a580c7e80 con 0x7f0a44004180 2023-10-10T20:40:30.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.359+0000 7f0a5d7e9700 1 -- 172.21.15.89:0/2089952329 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a5804c3c0 con 0x7f0a44004180 2023-10-10T20:40:30.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.359+0000 7f0a4f7fe700 1 -- 172.21.15.89:0/2089952329 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f0a4801b070 con 0x7f0a44004180 2023-10-10T20:40:30.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.360+0000 7f0a4f7fe700 1 --2- 172.21.15.89:0/2089952329 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0a38067290 0x7f0a38069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:30.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.360+0000 7f0a4f7fe700 1 -- 172.21.15.89:0/2089952329 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0a48019070 con 0x7f0a44004180 2023-10-10T20:40:30.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.360+0000 7f0a4e7fc700 1 --2- 172.21.15.89:0/2089952329 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0a38067290 0x7f0a38069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:30.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.360+0000 7f0a4e7fc700 1 --2- 172.21.15.89:0/2089952329 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0a38067290 0x7f0a38069740 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0a580ca340 tx=0x7f0a4001a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:30.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.364+0000 7f0a4f7fe700 1 -- 172.21.15.89:0/2089952329 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a48051720 con 0x7f0a44004180 2023-10-10T20:40:30.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.552+0000 7f0a5d7e9700 1 -- 172.21.15.89:0/2089952329 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0a580ca8b0 con 0x7f0a44004180 2023-10-10T20:40:30.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.556+0000 7f0a4f7fe700 1 -- 172.21.15.89:0/2089952329 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0a48054d40 con 0x7f0a44004180 2023-10-10T20:40:30.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.558+0000 7f0a5d7e9700 1 -- 172.21.15.89:0/2089952329 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0a38067290 msgr2=0x7f0a38069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:30.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.558+0000 7f0a5d7e9700 1 --2- 172.21.15.89:0/2089952329 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0a38067290 0x7f0a38069740 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0a580ca340 tx=0x7f0a4001a000 comp rx=0 tx=0).stop 2023-10-10T20:40:30.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.558+0000 7f0a5d7e9700 1 -- 172.21.15.89:0/2089952329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a44004180 msgr2=0x7f0a5816a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:30.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.558+0000 7f0a5d7e9700 1 --2- 172.21.15.89:0/2089952329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a44004180 0x7f0a5816a870 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f0a480073f0 tx=0x7f0a48001e10 comp rx=0 tx=0).stop 2023-10-10T20:40:30.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.558+0000 7f0a5d7e9700 1 -- 172.21.15.89:0/2089952329 shutdown_connections 2023-10-10T20:40:30.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.558+0000 7f0a5d7e9700 1 --2- 172.21.15.89:0/2089952329 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a5816db00 0x7f0a5816df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:30.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.558+0000 7f0a5d7e9700 1 --2- 172.21.15.89:0/2089952329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a44004180 0x7f0a5816a870 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:30.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.558+0000 7f0a5d7e9700 1 --2- 172.21.15.89:0/2089952329 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0a38067290 0x7f0a38069740 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:30.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.558+0000 7f0a5d7e9700 1 --2- 172.21.15.89:0/2089952329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a580cd0c0 0x7f0a580b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:30.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.558+0000 7f0a5d7e9700 1 -- 172.21.15.89:0/2089952329 >> 172.21.15.89:0/2089952329 conn(0x7f0a58068e20 msgr2=0x7f0a580d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:30.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.558+0000 7f0a5d7e9700 1 -- 172.21.15.89:0/2089952329 shutdown_connections 2023-10-10T20:40:30.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.558+0000 7f0a5d7e9700 1 -- 172.21.15.89:0/2089952329 wait complete. 2023-10-10T20:40:30.573 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup' 2023-10-10T20:40:30.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.685+0000 7fa2a1d14700 1 Processor -- start 2023-10-10T20:40:30.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.685+0000 7fa2a1d14700 1 -- start start 2023-10-10T20:40:30.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.686+0000 7fa2a1d14700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa29c159890 0x7fa29c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:30.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.686+0000 7fa2a1d14700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa29c159c60 0x7fa29c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:30.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.686+0000 7fa2a1d14700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa29c15a160 con 0x7fa29c15db10 2023-10-10T20:40:30.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.686+0000 7fa2a1d14700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa29c0d13a0 con 0x7fa29c159c60 2023-10-10T20:40:30.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.686+0000 7fa2a1d14700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c15e140 con 0x7fa29c159890 2023-10-10T20:40:30.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.686+0000 7fa29b7fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa29c159890 0x7fa29c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:30.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.686+0000 7fa29affd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa29c159c60 0x7fa29c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:30.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.687+0000 7fa29affd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa29c159c60 0x7fa29c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:40016/0 (socket says 172.21.15.89:40016) 2023-10-10T20:40:30.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.687+0000 7fa29affd700 1 -- 172.21.15.89:0/2225130878 learned_addr learned my addr 172.21.15.89:0/2225130878 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:30.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.687+0000 7fa29b7fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa29c159890 0x7fa29c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:45606/0 (socket says 172.21.15.89:45606) 2023-10-10T20:40:30.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.687+0000 7fa29bfff700 1 --1- 172.21.15.89:0/2225130878 >> v1:172.21.15.111:6789/0 conn(0x7fa29c15db10 0x7fa29c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:57372/0 (socket says 172.21.15.89:57372) 2023-10-10T20:40:30.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.687+0000 7fa29affd700 1 -- 172.21.15.89:0/2225130878 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa29c159890 msgr2=0x7fa29c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:30.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.687+0000 7fa29affd700 1 --2- 172.21.15.89:0/2225130878 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa29c159890 0x7fa29c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:30.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.687+0000 7fa29affd700 1 -- 172.21.15.89:0/2225130878 >> v1:172.21.15.111:6789/0 conn(0x7fa29c15db10 legacy=0x7fa29c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:30.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.687+0000 7fa29affd700 1 -- 172.21.15.89:0/2225130878 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29c15a3d0 con 0x7fa29c159c60 2023-10-10T20:40:30.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.687+0000 7fa29b7fe700 1 --2- 172.21.15.89:0/2225130878 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa29c159890 0x7fa29c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:40:30.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.687+0000 7fa29affd700 1 --2- 172.21.15.89:0/2225130878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa29c159c60 0x7fa29c15d5d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa2900099b0 tx=0x7fa290009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7efb9e637702e87 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:30.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.687+0000 7fa29a7fc700 1 -- 172.21.15.89:0/2225130878 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa290009ea0 con 0x7fa29c159c60 2023-10-10T20:40:30.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.687+0000 7fa29a7fc700 1 -- 172.21.15.89:0/2225130878 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29000b860 con 0x7fa29c159c60 2023-10-10T20:40:30.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.687+0000 7fa29a7fc700 1 -- 172.21.15.89:0/2225130878 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29000bc00 con 0x7fa29c159c60 2023-10-10T20:40:30.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.688+0000 7fa2a1d14700 1 -- 172.21.15.89:0/2225130878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa29c159c60 msgr2=0x7fa29c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:30.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.688+0000 7fa2a1d14700 1 --2- 172.21.15.89:0/2225130878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa29c159c60 0x7fa29c15d5d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa2900099b0 tx=0x7fa290009250 comp rx=0 tx=0).stop 2023-10-10T20:40:30.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.688+0000 7fa2a1d14700 1 -- 172.21.15.89:0/2225130878 shutdown_connections 2023-10-10T20:40:30.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.688+0000 7fa2a1d14700 1 --2- 172.21.15.89:0/2225130878 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa29c159890 0x7fa29c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:30.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.688+0000 7fa2a1d14700 1 --2- 172.21.15.89:0/2225130878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa29c159c60 0x7fa29c15d5d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:30.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.688+0000 7fa2a1d14700 1 -- 172.21.15.89:0/2225130878 >> 172.21.15.89:0/2225130878 conn(0x7fa29c068e20 msgr2=0x7fa29c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:30.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.688+0000 7fa2a1d14700 1 -- 172.21.15.89:0/2225130878 shutdown_connections 2023-10-10T20:40:30.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.688+0000 7fa2a1d14700 1 -- 172.21.15.89:0/2225130878 wait complete. 2023-10-10T20:40:30.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.689+0000 7fa2a1d14700 1 Processor -- start 2023-10-10T20:40:30.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.689+0000 7fa2a1d14700 1 -- start start 2023-10-10T20:40:30.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.690+0000 7fa2a1d14700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa29c159890 0x7fa29c0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:30.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.690+0000 7fa2a1d14700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa29c159c60 0x7fa29c0b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:30.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.690+0000 7fa2a1d14700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa29c15db10 0x7fa29c0cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:30.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.690+0000 7fa2a1d14700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c0b4010 con 0x7fa29c15db10 2023-10-10T20:40:30.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.690+0000 7fa2a1d14700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c0ce220 con 0x7fa29c159c60 2023-10-10T20:40:30.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.690+0000 7fa2a1d14700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c0b28a0 con 0x7fa29c159890 2023-10-10T20:40:30.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.690+0000 7fa29bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa29c15db10 0x7fa29c0cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:30.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.690+0000 7fa29bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa29c15db10 0x7fa29c0cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34078/0 (socket says 172.21.15.89:34078) 2023-10-10T20:40:30.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.690+0000 7fa29bfff700 1 -- 172.21.15.89:0/1043884347 learned_addr learned my addr 172.21.15.89:0/1043884347 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:30.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.690+0000 7fa29affd700 1 --2- 172.21.15.89:0/1043884347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa29c159c60 0x7fa29c0b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:30.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.690+0000 7fa29b7fe700 1 --2- 172.21.15.89:0/1043884347 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa29c159890 0x7fa29c0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:30.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.690+0000 7fa29bfff700 1 -- 172.21.15.89:0/1043884347 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa29c159890 msgr2=0x7fa29c0cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:30.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.690+0000 7fa29bfff700 1 --2- 172.21.15.89:0/1043884347 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa29c159890 0x7fa29c0cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:30.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.690+0000 7fa29bfff700 1 -- 172.21.15.89:0/1043884347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa29c159c60 msgr2=0x7fa29c0b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:30.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.690+0000 7fa29bfff700 1 --2- 172.21.15.89:0/1043884347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa29c159c60 0x7fa29c0b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:30.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.690+0000 7fa29bfff700 1 -- 172.21.15.89:0/1043884347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c009a20 con 0x7fa29c15db10 2023-10-10T20:40:30.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.690+0000 7fa29bfff700 1 --2- 172.21.15.89:0/1043884347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa29c15db10 0x7fa29c0cf0b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fa29400c510 tx=0x7fa294007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:30.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.691+0000 7fa2837fe700 1 -- 172.21.15.89:0/1043884347 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29400ce80 con 0x7fa29c15db10 2023-10-10T20:40:30.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.691+0000 7fa2837fe700 1 -- 172.21.15.89:0/1043884347 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa294005a70 con 0x7fa29c15db10 2023-10-10T20:40:30.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.691+0000 7fa2a1d14700 1 -- 172.21.15.89:0/1043884347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2900094f0 con 0x7fa29c15db10 2023-10-10T20:40:30.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.691+0000 7fa2837fe700 1 -- 172.21.15.89:0/1043884347 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa294005df0 con 0x7fa29c15db10 2023-10-10T20:40:30.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.691+0000 7fa2a1d14700 1 -- 172.21.15.89:0/1043884347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa29c0cda00 con 0x7fa29c15db10 2023-10-10T20:40:30.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.692+0000 7fa2a1d14700 1 -- 172.21.15.89:0/1043884347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa29c0b4180 con 0x7fa29c15db10 2023-10-10T20:40:30.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.692+0000 7fa2837fe700 1 -- 172.21.15.89:0/1043884347 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fa2940072e0 con 0x7fa29c15db10 2023-10-10T20:40:30.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.692+0000 7fa2837fe700 1 --2- 172.21.15.89:0/1043884347 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa288067290 0x7fa288069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:30.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.692+0000 7fa2837fe700 1 -- 172.21.15.89:0/1043884347 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa294083a60 con 0x7fa29c15db10 2023-10-10T20:40:30.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.692+0000 7fa29b7fe700 1 --2- 172.21.15.89:0/1043884347 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa288067290 0x7fa288069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:30.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.693+0000 7fa29b7fe700 1 --2- 172.21.15.89:0/1043884347 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa288067290 0x7fa288069740 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009fd0 tx=0x7fa28c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:30.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.696+0000 7fa2837fe700 1 -- 172.21.15.89:0/1043884347 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa294050bc0 con 0x7fa29c15db10 2023-10-10T20:40:30.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:30.953+0000 7fa2a1d14700 1 -- 172.21.15.89:0/1043884347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]} v 0) v1 -- 0x7fa29c04c3c0 con 0x7fa29c15db10 2023-10-10T20:40:31.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.214+0000 7fa2837fe700 1 -- 172.21.15.89:0/1043884347 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]}]=0 v284) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa294017020 con 0x7fa29c15db10 2023-10-10T20:40:31.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.216+0000 7fa2a1d14700 1 -- 172.21.15.89:0/1043884347 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa288067290 msgr2=0x7fa288069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:31.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.216+0000 7fa2a1d14700 1 --2- 172.21.15.89:0/1043884347 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa288067290 0x7fa288069740 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009fd0 tx=0x7fa28c00b040 comp rx=0 tx=0).stop 2023-10-10T20:40:31.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.216+0000 7fa2a1d14700 1 -- 172.21.15.89:0/1043884347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa29c15db10 msgr2=0x7fa29c0cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:31.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.216+0000 7fa2a1d14700 1 --2- 172.21.15.89:0/1043884347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa29c15db10 0x7fa29c0cf0b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fa29400c510 tx=0x7fa294007f50 comp rx=0 tx=0).stop 2023-10-10T20:40:31.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.217+0000 7fa2a1d14700 1 -- 172.21.15.89:0/1043884347 shutdown_connections 2023-10-10T20:40:31.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.217+0000 7fa2a1d14700 1 --2- 172.21.15.89:0/1043884347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa29c15db10 0x7fa29c0cf0b0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:31.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.217+0000 7fa2a1d14700 1 --2- 172.21.15.89:0/1043884347 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa29c159890 0x7fa29c0cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:31.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.217+0000 7fa2a1d14700 1 --2- 172.21.15.89:0/1043884347 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa288067290 0x7fa288069740 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:31.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.217+0000 7fa2a1d14700 1 --2- 172.21.15.89:0/1043884347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa29c159c60 0x7fa29c0b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:31.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.217+0000 7fa2a1d14700 1 -- 172.21.15.89:0/1043884347 >> 172.21.15.89:0/1043884347 conn(0x7fa29c068e20 msgr2=0x7fa29c154110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:31.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.217+0000 7fa2a1d14700 1 -- 172.21.15.89:0/1043884347 shutdown_connections 2023-10-10T20:40:31.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.217+0000 7fa2a1d14700 1 -- 172.21.15.89:0/1043884347 wait complete. 2023-10-10T20:40:31.231 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info --human_readable' command without subvolume group ... ok 2023-10-10T20:40:31.232 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info' command without subvolume group 2023-10-10T20:40:31.232 INFO:tasks.cephfs_test_runner:test_volume_info_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) 2023-10-10T20:40:31.233 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup' 2023-10-10T20:40:31.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.346+0000 7f00fcf09700 1 Processor -- start 2023-10-10T20:40:31.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.347+0000 7f00fcf09700 1 -- start start 2023-10-10T20:40:31.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.347+0000 7f00fcf09700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f00f81598d0 0x7f00f815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:31.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.347+0000 7f00fcf09700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00f80c7bd0 0x7f00f806ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:31.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.347+0000 7f00fcf09700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f00f806f350 0x7f00f806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:31.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.347+0000 7f00fcf09700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00f815af50 con 0x7f00f806f350 2023-10-10T20:40:31.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.347+0000 7f00fcf09700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00f806e980 con 0x7f00f81598d0 2023-10-10T20:40:31.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.347+0000 7f00fcf09700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f80ce180 con 0x7f00f80c7bd0 2023-10-10T20:40:31.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.347+0000 7f00f7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f00f806f350 0x7f00f806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:31.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.348+0000 7f00f7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f00f806f350 0x7f00f806bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34088/0 (socket says 172.21.15.89:34088) 2023-10-10T20:40:31.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.348+0000 7f00f7fff700 1 -- 172.21.15.89:0/553824368 learned_addr learned my addr 172.21.15.89:0/553824368 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:31.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.348+0000 7f00f77fe700 1 --2- 172.21.15.89:0/553824368 >> v2:172.21.15.111:3300/0 conn(0x7f00f81598d0 0x7f00f815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:31.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.348+0000 7f00f6ffd700 1 --2- 172.21.15.89:0/553824368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00f80c7bd0 0x7f00f806ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:31.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.348+0000 7f00f7fff700 1 -- 172.21.15.89:0/553824368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00f80c7bd0 msgr2=0x7f00f806ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:31.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.348+0000 7f00f7fff700 1 --2- 172.21.15.89:0/553824368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00f80c7bd0 0x7f00f806ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:31.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.348+0000 7f00f7fff700 1 -- 172.21.15.89:0/553824368 >> v2:172.21.15.111:3300/0 conn(0x7f00f81598d0 msgr2=0x7f00f815aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:31.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.348+0000 7f00f7fff700 1 --2- 172.21.15.89:0/553824368 >> v2:172.21.15.111:3300/0 conn(0x7f00f81598d0 0x7f00f815aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:31.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.348+0000 7f00f7fff700 1 -- 172.21.15.89:0/553824368 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f80c80f0 con 0x7f00f806f350 2023-10-10T20:40:31.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.348+0000 7f00f7fff700 1 --2- 172.21.15.89:0/553824368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f806f350 0x7f00f806bbe0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f00f8064fe0 tx=0x7f00f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f0c0705811e785 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:31.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.348+0000 7f00f67fc700 1 -- 172.21.15.89:0/553824368 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f000b640 con 0x7f00f806f350 2023-10-10T20:40:31.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.348+0000 7f00f67fc700 1 -- 172.21.15.89:0/553824368 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f000b7a0 con 0x7f00f806f350 2023-10-10T20:40:31.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.348+0000 7f00f67fc700 1 -- 172.21.15.89:0/553824368 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f000bb60 con 0x7f00f806f350 2023-10-10T20:40:31.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.349+0000 7f00fcf09700 1 -- 172.21.15.89:0/553824368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f806f350 msgr2=0x7f00f806bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:31.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.349+0000 7f00fcf09700 1 --2- 172.21.15.89:0/553824368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f806f350 0x7f00f806bbe0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f00f8064fe0 tx=0x7f00f0009250 comp rx=0 tx=0).stop 2023-10-10T20:40:31.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.349+0000 7f00fcf09700 1 -- 172.21.15.89:0/553824368 shutdown_connections 2023-10-10T20:40:31.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.349+0000 7f00fcf09700 1 --2- 172.21.15.89:0/553824368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f806f350 0x7f00f806bbe0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:31.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.349+0000 7f00fcf09700 1 --2- 172.21.15.89:0/553824368 >> v2:172.21.15.111:3300/0 conn(0x7f00f81598d0 0x7f00f815aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:31.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.349+0000 7f00fcf09700 1 --2- 172.21.15.89:0/553824368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00f80c7bd0 0x7f00f806ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:31.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.349+0000 7f00fcf09700 1 -- 172.21.15.89:0/553824368 >> 172.21.15.89:0/553824368 conn(0x7f00f806b490 msgr2=0x7f00f80cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:31.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.349+0000 7f00fcf09700 1 -- 172.21.15.89:0/553824368 shutdown_connections 2023-10-10T20:40:31.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.349+0000 7f00fcf09700 1 -- 172.21.15.89:0/553824368 wait complete. 2023-10-10T20:40:31.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.350+0000 7f00fcf09700 1 Processor -- start 2023-10-10T20:40:31.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.350+0000 7f00fcf09700 1 -- start start 2023-10-10T20:40:31.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00fcf09700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00f806f350 0x7f00f80c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:31.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00fcf09700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00f80c7bd0 0x7f00f81545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:31.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00fcf09700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f81598d0 0x7f00f816a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00fcf09700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f8154140 con 0x7f00f81598d0 2023-10-10T20:40:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00fcf09700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f815b850 con 0x7f00f806f350 2023-10-10T20:40:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00fcf09700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f815bb30 con 0x7f00f80c7bd0 2023-10-10T20:40:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00f7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f81598d0 0x7f00f816a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00f7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f81598d0 0x7f00f816a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34100/0 (socket says 172.21.15.89:34100) 2023-10-10T20:40:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00f7fff700 1 -- 172.21.15.89:0/2908330993 learned_addr learned my addr 172.21.15.89:0/2908330993 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00f77fe700 1 --2- 172.21.15.89:0/2908330993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00f806f350 0x7f00f80c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00f6ffd700 1 --2- 172.21.15.89:0/2908330993 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00f80c7bd0 0x7f00f81545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00f7fff700 1 -- 172.21.15.89:0/2908330993 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00f80c7bd0 msgr2=0x7f00f81545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00f7fff700 1 --2- 172.21.15.89:0/2908330993 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00f80c7bd0 0x7f00f81545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00f7fff700 1 -- 172.21.15.89:0/2908330993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00f806f350 msgr2=0x7f00f80c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:31.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00f7fff700 1 --2- 172.21.15.89:0/2908330993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00f806f350 0x7f00f80c3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:31.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00f7fff700 1 -- 172.21.15.89:0/2908330993 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f00094f0 con 0x7f00f81598d0 2023-10-10T20:40:31.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00f7fff700 1 --2- 172.21.15.89:0/2908330993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f81598d0 0x7f00f816a7a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f00f00094c0 tx=0x7f00f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:31.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00df7fe700 1 -- 172.21.15.89:0/2908330993 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f000bb20 con 0x7f00f81598d0 2023-10-10T20:40:31.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00df7fe700 1 -- 172.21.15.89:0/2908330993 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f0011420 con 0x7f00f81598d0 2023-10-10T20:40:31.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00df7fe700 1 -- 172.21.15.89:0/2908330993 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f00117e0 con 0x7f00f81598d0 2023-10-10T20:40:31.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.351+0000 7f00fcf09700 1 -- 172.21.15.89:0/2908330993 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f8153dd0 con 0x7f00f81598d0 2023-10-10T20:40:31.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.352+0000 7f00fcf09700 1 -- 172.21.15.89:0/2908330993 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f815b490 con 0x7f00f81598d0 2023-10-10T20:40:31.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.352+0000 7f00fcf09700 1 -- 172.21.15.89:0/2908330993 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f81542b0 con 0x7f00f81598d0 2023-10-10T20:40:31.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.353+0000 7f00df7fe700 1 -- 172.21.15.89:0/2908330993 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f00f0011b50 con 0x7f00f81598d0 2023-10-10T20:40:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.353+0000 7f00df7fe700 1 --2- 172.21.15.89:0/2908330993 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f00e4067290 0x7f00e4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.353+0000 7f00df7fe700 1 -- 172.21.15.89:0/2908330993 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f00f0085f50 con 0x7f00f81598d0 2023-10-10T20:40:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.353+0000 7f00f77fe700 1 --2- 172.21.15.89:0/2908330993 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f00e4067290 0x7f00e4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.354+0000 7f00f77fe700 1 --2- 172.21.15.89:0/2908330993 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f00e4067290 0x7f00e4069740 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f00e800f570 tx=0x7f00e800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.357+0000 7f00df7fe700 1 -- 172.21.15.89:0/2908330993 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00f0053130 con 0x7f00f81598d0 2023-10-10T20:40:31.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:31.615+0000 7f00fcf09700 1 -- 172.21.15.89:0/2908330993 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]} v 0) v1 -- 0x7f00f804c3c0 con 0x7f00f81598d0 2023-10-10T20:40:32.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.216+0000 7f00df7fe700 1 -- 172.21.15.89:0/2908330993 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]}]=0 v285) v1 ==== 159+0+0 (secure 0 0 0) 0x7f00f0056750 con 0x7f00f81598d0 2023-10-10T20:40:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.218+0000 7f00fcf09700 1 -- 172.21.15.89:0/2908330993 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f00e4067290 msgr2=0x7f00e4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.218+0000 7f00fcf09700 1 --2- 172.21.15.89:0/2908330993 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f00e4067290 0x7f00e4069740 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f00e800f570 tx=0x7f00e800b040 comp rx=0 tx=0).stop 2023-10-10T20:40:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.218+0000 7f00fcf09700 1 -- 172.21.15.89:0/2908330993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f81598d0 msgr2=0x7f00f816a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.218+0000 7f00fcf09700 1 --2- 172.21.15.89:0/2908330993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f81598d0 0x7f00f816a7a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f00f00094c0 tx=0x7f00f0017000 comp rx=0 tx=0).stop 2023-10-10T20:40:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.218+0000 7f00fcf09700 1 -- 172.21.15.89:0/2908330993 shutdown_connections 2023-10-10T20:40:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.218+0000 7f00fcf09700 1 --2- 172.21.15.89:0/2908330993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f81598d0 0x7f00f816a7a0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.218+0000 7f00fcf09700 1 --2- 172.21.15.89:0/2908330993 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f00e4067290 0x7f00e4069740 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.218+0000 7f00fcf09700 1 --2- 172.21.15.89:0/2908330993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00f806f350 0x7f00f80c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.218+0000 7f00fcf09700 1 --2- 172.21.15.89:0/2908330993 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f00f80c7bd0 0x7f00f81545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.218+0000 7f00fcf09700 1 -- 172.21.15.89:0/2908330993 >> 172.21.15.89:0/2908330993 conn(0x7f00f806b490 msgr2=0x7f00f80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.218+0000 7f00fcf09700 1 -- 172.21.15.89:0/2908330993 shutdown_connections 2023-10-10T20:40:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.218+0000 7f00fcf09700 1 -- 172.21.15.89:0/2908330993 wait complete. 2023-10-10T20:40:32.234 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:40:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.351+0000 7f2a91d75700 1 Processor -- start 2023-10-10T20:40:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.351+0000 7f2a91d75700 1 -- start start 2023-10-10T20:40:32.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.352+0000 7f2a91d75700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2a8c0b16d0 0x7f2a8c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:32.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.352+0000 7f2a91d75700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2a8c0b33b0 0x7f2a8c0b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:32.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.352+0000 7f2a91d75700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a8c0b2080 con 0x7f2a8c0b4e70 2023-10-10T20:40:32.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.352+0000 7f2a91d75700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a8c0cfb00 con 0x7f2a8c0b16d0 2023-10-10T20:40:32.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.352+0000 7f2a91d75700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a8c0b5c40 con 0x7f2a8c0b33b0 2023-10-10T20:40:32.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.352+0000 7f2a90d73700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2a8c0b4e70 0x7f2a8c0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39004/0 (socket says 172.21.15.89:39004) 2023-10-10T20:40:32.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.352+0000 7f2a90d73700 1 -- 172.21.15.89:0/4105363460 learned_addr learned my addr 172.21.15.89:0/4105363460 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:32.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.352+0000 7f2a8bfff700 1 --2- 172.21.15.89:0/4105363460 >> v2:172.21.15.89:3300/0 conn(0x7f2a8c0b16d0 0x7f2a8c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:32.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.352+0000 7f2a8b7fe700 1 -- 172.21.15.89:0/4105363460 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1523532829 0 0) 0x7f2a8c0b2080 con 0x7f2a8c0b4e70 2023-10-10T20:40:32.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.352+0000 7f2a8b7fe700 1 -- 172.21.15.89:0/4105363460 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a78003570 con 0x7f2a8c0b4e70 2023-10-10T20:40:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.352+0000 7f2a8bfff700 1 -- 172.21.15.89:0/4105363460 >> v2:172.21.15.111:3300/0 conn(0x7f2a8c0b33b0 msgr2=0x7f2a8c0b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.352+0000 7f2a8bfff700 1 --2- 172.21.15.89:0/4105363460 >> v2:172.21.15.111:3300/0 conn(0x7f2a8c0b33b0 0x7f2a8c0b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.352+0000 7f2a8bfff700 1 -- 172.21.15.89:0/4105363460 >> v1:172.21.15.89:6789/0 conn(0x7f2a8c0b4e70 legacy=0x7f2a8c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.352+0000 7f2a8bfff700 1 -- 172.21.15.89:0/4105363460 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8c0b5f20 con 0x7f2a8c0b16d0 2023-10-10T20:40:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.352+0000 7f2a8bfff700 1 --2- 172.21.15.89:0/4105363460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a8c0b16d0 0x7f2a8c0b1b40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c071380 tx=0x7f2a7c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b3825a2b98b912 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.352+0000 7f2a8b7fe700 1 -- 172.21.15.89:0/4105363460 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c00e070 con 0x7f2a8c0b16d0 2023-10-10T20:40:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.353+0000 7f2a8b7fe700 1 -- 172.21.15.89:0/4105363460 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c00bb60 con 0x7f2a8c0b16d0 2023-10-10T20:40:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.353+0000 7f2a8b7fe700 1 -- 172.21.15.89:0/4105363460 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c012430 con 0x7f2a8c0b16d0 2023-10-10T20:40:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.353+0000 7f2a91d75700 1 -- 172.21.15.89:0/4105363460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a8c0b16d0 msgr2=0x7f2a8c0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.353+0000 7f2a91d75700 1 --2- 172.21.15.89:0/4105363460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a8c0b16d0 0x7f2a8c0b1b40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c071380 tx=0x7f2a7c009780 comp rx=0 tx=0).stop 2023-10-10T20:40:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.353+0000 7f2a91d75700 1 -- 172.21.15.89:0/4105363460 shutdown_connections 2023-10-10T20:40:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.353+0000 7f2a91d75700 1 --2- 172.21.15.89:0/4105363460 >> v2:172.21.15.111:3300/0 conn(0x7f2a8c0b33b0 0x7f2a8c0b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.353+0000 7f2a91d75700 1 --2- 172.21.15.89:0/4105363460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a8c0b16d0 0x7f2a8c0b1b40 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.353+0000 7f2a91d75700 1 -- 172.21.15.89:0/4105363460 >> 172.21.15.89:0/4105363460 conn(0x7f2a8c0d10c0 msgr2=0x7f2a8c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.353+0000 7f2a91d75700 1 -- 172.21.15.89:0/4105363460 shutdown_connections 2023-10-10T20:40:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.353+0000 7f2a91d75700 1 -- 172.21.15.89:0/4105363460 wait complete. 2023-10-10T20:40:32.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.354+0000 7f2a91d75700 1 Processor -- start 2023-10-10T20:40:32.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.354+0000 7f2a91d75700 1 -- start start 2023-10-10T20:40:32.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.355+0000 7f2a91d75700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2a8c0b16d0 0x7f2a8c0a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:32.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.355+0000 7f2a91d75700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8c0b33b0 0x7f2a8c0c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:32.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.355+0000 7f2a91d75700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a8c0b4e70 0x7f2a8c169fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:32.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.355+0000 7f2a91d75700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c16aed0 con 0x7f2a8c0b4e70 2023-10-10T20:40:32.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.355+0000 7f2a91d75700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c0c83e0 con 0x7f2a8c0b33b0 2023-10-10T20:40:32.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.355+0000 7f2a91d75700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c0c8660 con 0x7f2a8c0b16d0 2023-10-10T20:40:32.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.355+0000 7f2a91574700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a8c0b4e70 0x7f2a8c169fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:32.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.355+0000 7f2a91574700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a8c0b4e70 0x7f2a8c169fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34122/0 (socket says 172.21.15.89:34122) 2023-10-10T20:40:32.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.355+0000 7f2a91574700 1 -- 172.21.15.89:0/4029052620 learned_addr learned my addr 172.21.15.89:0/4029052620 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:32.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.355+0000 7f2a90d73700 1 --2- 172.21.15.89:0/4029052620 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2a8c0b16d0 0x7f2a8c0a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:32.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.355+0000 7f2a8bfff700 1 --2- 172.21.15.89:0/4029052620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8c0b33b0 0x7f2a8c0c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:32.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.355+0000 7f2a91574700 1 -- 172.21.15.89:0/4029052620 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2a8c0b16d0 msgr2=0x7f2a8c0a73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:32.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.355+0000 7f2a91574700 1 --2- 172.21.15.89:0/4029052620 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2a8c0b16d0 0x7f2a8c0a73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:32.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.355+0000 7f2a91574700 1 -- 172.21.15.89:0/4029052620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8c0b33b0 msgr2=0x7f2a8c0c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:32.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.355+0000 7f2a91574700 1 --2- 172.21.15.89:0/4029052620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8c0b33b0 0x7f2a8c0c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:32.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.355+0000 7f2a91574700 1 -- 172.21.15.89:0/4029052620 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c009a20 con 0x7f2a8c0b4e70 2023-10-10T20:40:32.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.356+0000 7f2a91574700 1 --2- 172.21.15.89:0/4029052620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a8c0b4e70 0x7f2a8c169fe0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400b810 tx=0x7f2a740091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:32.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.356+0000 7f2a88ff9700 1 -- 172.21.15.89:0/4029052620 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7400f570 con 0x7f2a8c0b4e70 2023-10-10T20:40:32.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.356+0000 7f2a88ff9700 1 -- 172.21.15.89:0/4029052620 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7400f6f0 con 0x7f2a8c0b4e70 2023-10-10T20:40:32.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.356+0000 7f2a91d75700 1 -- 172.21.15.89:0/4029052620 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8c0c8940 con 0x7f2a8c0b4e70 2023-10-10T20:40:32.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.356+0000 7f2a88ff9700 1 -- 172.21.15.89:0/4029052620 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7400fab0 con 0x7f2a8c0b4e70 2023-10-10T20:40:32.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.356+0000 7f2a91d75700 1 -- 172.21.15.89:0/4029052620 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8c0c8f00 con 0x7f2a8c0b4e70 2023-10-10T20:40:32.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.357+0000 7f2a91d75700 1 -- 172.21.15.89:0/4029052620 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8c16b080 con 0x7f2a8c0b4e70 2023-10-10T20:40:32.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.357+0000 7f2a88ff9700 1 -- 172.21.15.89:0/4029052620 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f2a7401a460 con 0x7f2a8c0b4e70 2023-10-10T20:40:32.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.357+0000 7f2a88ff9700 1 --2- 172.21.15.89:0/4029052620 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2a6c067290 0x7f2a6c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:32.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.357+0000 7f2a88ff9700 1 -- 172.21.15.89:0/4029052620 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2a74084420 con 0x7f2a8c0b4e70 2023-10-10T20:40:32.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.357+0000 7f2a90d73700 1 --2- 172.21.15.89:0/4029052620 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2a6c067290 0x7f2a6c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:32.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.358+0000 7f2a90d73700 1 --2- 172.21.15.89:0/4029052620 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2a6c067290 0x7f2a6c069740 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2a840058d0 tx=0x7f2a84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:32.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.362+0000 7f2a88ff9700 1 -- 172.21.15.89:0/4029052620 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a8c16b080 con 0x7f2a8c0b4e70 2023-10-10T20:40:32.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.546+0000 7f2a91d75700 1 -- 172.21.15.89:0/4029052620 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2a8c16b080 con 0x7f2a8c0b4e70 2023-10-10T20:40:32.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.550+0000 7f2a88ff9700 1 -- 172.21.15.89:0/4029052620 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2a8c16b080 con 0x7f2a8c0b4e70 2023-10-10T20:40:32.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.552+0000 7f2a91d75700 1 -- 172.21.15.89:0/4029052620 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2a6c067290 msgr2=0x7f2a6c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:32.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.552+0000 7f2a91d75700 1 --2- 172.21.15.89:0/4029052620 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2a6c067290 0x7f2a6c069740 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2a840058d0 tx=0x7f2a84009040 comp rx=0 tx=0).stop 2023-10-10T20:40:32.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.552+0000 7f2a91d75700 1 -- 172.21.15.89:0/4029052620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a8c0b4e70 msgr2=0x7f2a8c169fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:32.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.552+0000 7f2a91d75700 1 --2- 172.21.15.89:0/4029052620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a8c0b4e70 0x7f2a8c169fe0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400b810 tx=0x7f2a740091d0 comp rx=0 tx=0).stop 2023-10-10T20:40:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.552+0000 7f2a91d75700 1 -- 172.21.15.89:0/4029052620 shutdown_connections 2023-10-10T20:40:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.552+0000 7f2a91d75700 1 --2- 172.21.15.89:0/4029052620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a8c0b4e70 0x7f2a8c169fe0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.552+0000 7f2a91d75700 1 --2- 172.21.15.89:0/4029052620 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2a8c0b16d0 0x7f2a8c0a73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.552+0000 7f2a91d75700 1 --2- 172.21.15.89:0/4029052620 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2a6c067290 0x7f2a6c069740 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:32.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.552+0000 7f2a91d75700 1 --2- 172.21.15.89:0/4029052620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8c0b33b0 0x7f2a8c0c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:32.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.552+0000 7f2a91d75700 1 -- 172.21.15.89:0/4029052620 >> 172.21.15.89:0/4029052620 conn(0x7f2a8c0d10c0 msgr2=0x7f2a8c069a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:32.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.552+0000 7f2a91d75700 1 -- 172.21.15.89:0/4029052620 shutdown_connections 2023-10-10T20:40:32.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.552+0000 7f2a91d75700 1 -- 172.21.15.89:0/4029052620 wait complete. 2023-10-10T20:40:32.566 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:40:32.566 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:40:32.602 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:40:32.649 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:40:32.649 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:40:32.650 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:40:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.762+0000 7f6eeeaaa700 1 Processor -- start 2023-10-10T20:40:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.762+0000 7f6eeeaaa700 1 -- start start 2023-10-10T20:40:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.763+0000 7f6eeeaaa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6ee8159c60 0x7f6ee815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.763+0000 7f6eeeaaa700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee815a160 con 0x7f6ee8159890 2023-10-10T20:40:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.763+0000 7f6eeeaaa700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee80d13a0 con 0x7f6ee815db10 2023-10-10T20:40:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.763+0000 7f6eeeaaa700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee815df50 con 0x7f6ee8159c60 2023-10-10T20:40:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.763+0000 7f6edffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6ee8159c60 0x7f6ee815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.763+0000 7f6eec846700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6ee8159890 0x7f6ee815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39018/0 (socket says 172.21.15.89:39018) 2023-10-10T20:40:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.763+0000 7f6eec846700 1 -- 172.21.15.89:0/1057112946 learned_addr learned my addr 172.21.15.89:0/1057112946 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.764+0000 7f6edffff700 1 -- 172.21.15.89:0/1057112946 >> v1:172.21.15.111:6789/0 conn(0x7f6ee815db10 legacy=0x7f6ee8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.764+0000 7f6edffff700 1 -- 172.21.15.89:0/1057112946 >> v1:172.21.15.89:6789/0 conn(0x7f6ee8159890 legacy=0x7f6ee815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.764+0000 7f6edf7fe700 1 -- 172.21.15.89:0/1057112946 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241559077 0 0) 0x7f6ee815a160 con 0x7f6ee8159890 2023-10-10T20:40:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.764+0000 7f6edffff700 1 -- 172.21.15.89:0/1057112946 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee815e140 con 0x7f6ee8159c60 2023-10-10T20:40:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.764+0000 7f6edffff700 1 --2- 172.21.15.89:0/1057112946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee8159c60 0x7f6ee815d5d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80098e0 tx=0x7f6ed80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31ac6f2798f238 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.764+0000 7f6edf7fe700 1 -- 172.21.15.89:0/1057112946 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8009e80 con 0x7f6ee8159c60 2023-10-10T20:40:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.764+0000 7f6edf7fe700 1 -- 172.21.15.89:0/1057112946 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed800b820 con 0x7f6ee8159c60 2023-10-10T20:40:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.764+0000 7f6edf7fe700 1 -- 172.21.15.89:0/1057112946 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed800bba0 con 0x7f6ee8159c60 2023-10-10T20:40:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.765+0000 7f6eeeaaa700 1 -- 172.21.15.89:0/1057112946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee8159c60 msgr2=0x7f6ee815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.765+0000 7f6eeeaaa700 1 --2- 172.21.15.89:0/1057112946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee8159c60 0x7f6ee815d5d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80098e0 tx=0x7f6ed80091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.765+0000 7f6eeeaaa700 1 -- 172.21.15.89:0/1057112946 shutdown_connections 2023-10-10T20:40:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.765+0000 7f6eeeaaa700 1 --2- 172.21.15.89:0/1057112946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee8159c60 0x7f6ee815d5d0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.765+0000 7f6eeeaaa700 1 -- 172.21.15.89:0/1057112946 >> 172.21.15.89:0/1057112946 conn(0x7f6ee8068e20 msgr2=0x7f6ee80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.765+0000 7f6eeeaaa700 1 -- 172.21.15.89:0/1057112946 shutdown_connections 2023-10-10T20:40:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.765+0000 7f6eeeaaa700 1 -- 172.21.15.89:0/1057112946 wait complete. 2023-10-10T20:40:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.766+0000 7f6eeeaaa700 1 Processor -- start 2023-10-10T20:40:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.766+0000 7f6eeeaaa700 1 -- start start 2023-10-10T20:40:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.767+0000 7f6eeeaaa700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ee8159890 0x7f6ee81501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.767+0000 7f6eeeaaa700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ee8159c60 0x7f6ee80cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:32.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.767+0000 7f6eeeaaa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee815db10 0x7f6ee80c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:32.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.767+0000 7f6eeeaaa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee816a6a0 con 0x7f6ee815db10 2023-10-10T20:40:32.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.767+0000 7f6eeeaaa700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee8159060 con 0x7f6ee8159c60 2023-10-10T20:40:32.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.767+0000 7f6eeeaaa700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80cdba0 con 0x7f6ee8159890 2023-10-10T20:40:32.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.767+0000 7f6eed047700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee815db10 0x7f6ee80c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:32.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.767+0000 7f6eed047700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee815db10 0x7f6ee80c67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34136/0 (socket says 172.21.15.89:34136) 2023-10-10T20:40:32.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.767+0000 7f6eed047700 1 -- 172.21.15.89:0/3538323164 learned_addr learned my addr 172.21.15.89:0/3538323164 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:32.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.767+0000 7f6eec846700 1 --2- 172.21.15.89:0/3538323164 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ee8159890 0x7f6ee81501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:32.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.767+0000 7f6eed047700 1 -- 172.21.15.89:0/3538323164 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ee8159890 msgr2=0x7f6ee81501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.767+0000 7f6eed047700 1 --2- 172.21.15.89:0/3538323164 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ee8159890 0x7f6ee81501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.767+0000 7f6eed047700 1 -- 172.21.15.89:0/3538323164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ee8159c60 msgr2=0x7f6ee80cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.767+0000 7f6edffff700 1 --2- 172.21.15.89:0/3538323164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ee8159c60 0x7f6ee80cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.767+0000 7f6eed047700 1 --2- 172.21.15.89:0/3538323164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ee8159c60 0x7f6ee80cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.767+0000 7f6eed047700 1 -- 172.21.15.89:0/3538323164 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed8009420 con 0x7f6ee815db10 2023-10-10T20:40:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.767+0000 7f6eed047700 1 --2- 172.21.15.89:0/3538323164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee815db10 0x7f6ee80c67b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000bbb0 tx=0x7f6ee0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.768+0000 7f6edcff9700 1 -- 172.21.15.89:0/3538323164 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00092e0 con 0x7f6ee815db10 2023-10-10T20:40:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.768+0000 7f6edcff9700 1 -- 172.21.15.89:0/3538323164 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee000c7a0 con 0x7f6ee815db10 2023-10-10T20:40:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.768+0000 7f6eeeaaa700 1 -- 172.21.15.89:0/3538323164 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee8169d70 con 0x7f6ee815db10 2023-10-10T20:40:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.768+0000 7f6edcff9700 1 -- 172.21.15.89:0/3538323164 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee000cb40 con 0x7f6ee815db10 2023-10-10T20:40:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.768+0000 7f6eeeaaa700 1 -- 172.21.15.89:0/3538323164 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee816a330 con 0x7f6ee815db10 2023-10-10T20:40:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.769+0000 7f6eeeaaa700 1 -- 172.21.15.89:0/3538323164 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee804c3c0 con 0x7f6ee815db10 2023-10-10T20:40:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.769+0000 7f6edcff9700 1 -- 172.21.15.89:0/3538323164 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f6ee0012070 con 0x7f6ee815db10 2023-10-10T20:40:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.769+0000 7f6edcff9700 1 --2- 172.21.15.89:0/3538323164 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ed4067290 0x7f6ed4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.769+0000 7f6edcff9700 1 -- 172.21.15.89:0/3538323164 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6ee00845b0 con 0x7f6ee815db10 2023-10-10T20:40:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.769+0000 7f6eec846700 1 --2- 172.21.15.89:0/3538323164 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ed4067290 0x7f6ed4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.770+0000 7f6eec846700 1 --2- 172.21.15.89:0/3538323164 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ed4067290 0x7f6ed4069740 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8157090 tx=0x7f6ee4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:32.773+0000 7f6edcff9700 1 -- 172.21.15.89:0/3538323164 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ee0051610 con 0x7f6ee815db10 2023-10-10T20:40:33.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.001+0000 7f6eeeaaa700 1 -- 172.21.15.89:0/3538323164 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ee806a5b0 con 0x7f6ee815db10 2023-10-10T20:40:33.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.001+0000 7f6edcff9700 1 -- 172.21.15.89:0/3538323164 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f6ee0054c30 con 0x7f6ee815db10 2023-10-10T20:40:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:33.004 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":58,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5678,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/709452565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":709452565},{"type":"v1","addr":"172.21.15.111:6835","nonce":709452565}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":5995,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/3194805494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3194805494},{"type":"v1","addr":"172.21.15.111:6834","nonce":3194805494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6447,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/2766962252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2766962252},{"type":"v1","addr":"172.21.15.89:6836","nonce":2766962252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6450,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1141782720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1141782720},{"type":"v1","addr":"172.21.15.89:6837","nonce":1141782720}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[]} 2023-10-10T20:40:33.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.003+0000 7f6eeeaaa700 1 -- 172.21.15.89:0/3538323164 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ed4067290 msgr2=0x7f6ed4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:33.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.003+0000 7f6eeeaaa700 1 --2- 172.21.15.89:0/3538323164 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ed4067290 0x7f6ed4069740 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8157090 tx=0x7f6ee4009040 comp rx=0 tx=0).stop 2023-10-10T20:40:33.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.003+0000 7f6eeeaaa700 1 -- 172.21.15.89:0/3538323164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee815db10 msgr2=0x7f6ee80c67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:33.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.003+0000 7f6eeeaaa700 1 --2- 172.21.15.89:0/3538323164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee815db10 0x7f6ee80c67b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000bbb0 tx=0x7f6ee0009840 comp rx=0 tx=0).stop 2023-10-10T20:40:33.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.003+0000 7f6eeeaaa700 1 -- 172.21.15.89:0/3538323164 shutdown_connections 2023-10-10T20:40:33.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.003+0000 7f6eeeaaa700 1 --2- 172.21.15.89:0/3538323164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee815db10 0x7f6ee80c67b0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:33.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.003+0000 7f6eeeaaa700 1 --2- 172.21.15.89:0/3538323164 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ee8159890 0x7f6ee81501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:33.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.003+0000 7f6eeeaaa700 1 --2- 172.21.15.89:0/3538323164 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ed4067290 0x7f6ed4069740 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:33.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.003+0000 7f6eeeaaa700 1 --2- 172.21.15.89:0/3538323164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ee8159c60 0x7f6ee80cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:33.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.003+0000 7f6eeeaaa700 1 -- 172.21.15.89:0/3538323164 >> 172.21.15.89:0/3538323164 conn(0x7f6ee8068e20 msgr2=0x7f6ee81570c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:33.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.003+0000 7f6eeeaaa700 1 -- 172.21.15.89:0/3538323164 shutdown_connections 2023-10-10T20:40:33.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.004+0000 7f6eeeaaa700 1 -- 172.21.15.89:0/3538323164 wait complete. 2023-10-10T20:40:33.008 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 58 2023-10-10T20:40:33.018 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:40:33.018 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:40:33.018 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:40:33.019 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:40:33.019 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:40:33.019 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:40:33.020 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:40:33.020 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:40:33.020 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:40:33.020 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:40:33.021 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:40:33.021 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:40:33.021 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:40:33.021 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:40:33.022 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:40:33.022 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:40:33.027 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:40:33.027 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:40:33.029 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:40:33.029 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:40:33.034 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:40:33.035 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:40:33.036 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:40:33.036 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:40:33.070 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:40:33.071 DEBUG:teuthology.parallel:result is None 2023-10-10T20:40:33.072 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:40:33.072 DEBUG:teuthology.parallel:result is None 2023-10-10T20:40:33.077 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:40:33.078 DEBUG:teuthology.parallel:result is None 2023-10-10T20:40:33.078 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:40:33.079 DEBUG:teuthology.parallel:result is None 2023-10-10T20:40:33.079 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:40:33.144 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:40:33.144 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-10T20:40:33.161 INFO:tasks.ceph.mds.b.smithi111.stdout:starting mds.b at 2023-10-10T20:40:33.161 INFO:tasks.ceph.mds.d.smithi111.stdout:starting mds.d at 2023-10-10T20:40:33.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.199+0000 7fbf63fff700 1 Processor -- start 2023-10-10T20:40:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.199+0000 7fbf63fff700 1 -- start start 2023-10-10T20:40:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.200+0000 7fbf63fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbf6406c2c0 0x7fbf640b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.200+0000 7fbf63fff700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6406eaf0 con 0x7fbf64156590 2023-10-10T20:40:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.200+0000 7fbf63fff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6415b080 con 0x7fbf640b23d0 2023-10-10T20:40:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.200+0000 7fbf63fff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6415ab00 con 0x7fbf6406c2c0 2023-10-10T20:40:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.200+0000 7fbf62ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbf64156590 0x7fbf640b04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39034/0 (socket says 172.21.15.89:39034) 2023-10-10T20:40:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.200+0000 7fbf62ffd700 1 -- 172.21.15.89:0/443210228 learned_addr learned my addr 172.21.15.89:0/443210228 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.200+0000 7fbf61ffb700 1 -- 172.21.15.89:0/443210228 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2681674865 0 0) 0x7fbf6406eaf0 con 0x7fbf64156590 2023-10-10T20:40:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.200+0000 7fbf627fc700 1 --2- 172.21.15.89:0/443210228 >> v2:172.21.15.111:3300/0 conn(0x7fbf6406c2c0 0x7fbf640b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.200+0000 7fbf61ffb700 1 -- 172.21.15.89:0/443210228 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf4c005780 con 0x7fbf64156590 2023-10-10T20:40:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf61ffb700 1 -- 172.21.15.89:0/443210228 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1639464461 0 0) 0x7fbf4c005780 con 0x7fbf64156590 2023-10-10T20:40:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf61ffb700 1 -- 172.21.15.89:0/443210228 >> v2:172.21.15.111:3300/0 conn(0x7fbf6406c2c0 msgr2=0x7fbf640b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf61ffb700 1 --2- 172.21.15.89:0/443210228 >> v2:172.21.15.111:3300/0 conn(0x7fbf6406c2c0 0x7fbf640b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf61ffb700 1 -- 172.21.15.89:0/443210228 >> v1:172.21.15.111:6789/0 conn(0x7fbf640b23d0 legacy=0x7fbf641581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf61ffb700 1 -- 172.21.15.89:0/443210228 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf640afa70 con 0x7fbf64156590 2023-10-10T20:40:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf61ffb700 1 -- 172.21.15.89:0/443210228 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fbf5c0017f0 con 0x7fbf64156590 2023-10-10T20:40:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf61ffb700 1 -- 172.21.15.89:0/443210228 >> v1:172.21.15.89:6789/0 conn(0x7fbf64156590 legacy=0x7fbf640b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf61ffb700 1 --2- 172.21.15.89:0/443210228 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf4c005c70 0x7fbf4c006080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf61ffb700 1 --2- 172.21.15.89:0/443210228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf4c006a20 0x7fbf4c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf61ffb700 1 --2- 172.21.15.89:0/443210228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf4c0073d0 0x7fbf4c00dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf61ffb700 1 -- 172.21.15.89:0/443210228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6415ab00 con 0x7fbf4c006a20 2023-10-10T20:40:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf61ffb700 1 -- 172.21.15.89:0/443210228 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c00e2e0 con 0x7fbf4c0073d0 2023-10-10T20:40:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf61ffb700 1 -- 172.21.15.89:0/443210228 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c00e4a0 con 0x7fbf4c005c70 2023-10-10T20:40:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf627fc700 1 --2- 172.21.15.89:0/443210228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf4c006a20 0x7fbf4c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf62ffd700 1 --2- 172.21.15.89:0/443210228 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf4c005c70 0x7fbf4c006080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf637fe700 1 --2- 172.21.15.89:0/443210228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf4c0073d0 0x7fbf4c00dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf627fc700 1 -- 172.21.15.89:0/443210228 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf4c005c70 msgr2=0x7fbf4c006080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf627fc700 1 --2- 172.21.15.89:0/443210228 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf4c005c70 0x7fbf4c006080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf627fc700 1 -- 172.21.15.89:0/443210228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf4c0073d0 msgr2=0x7fbf4c00dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf627fc700 1 --2- 172.21.15.89:0/443210228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf4c0073d0 0x7fbf4c00dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf637fe700 1 -- 172.21.15.89:0/443210228 reap_dead start 2023-10-10T20:40:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf627fc700 1 -- 172.21.15.89:0/443210228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c005780 con 0x7fbf4c006a20 2023-10-10T20:40:33.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf627fc700 1 --2- 172.21.15.89:0/443210228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf4c006a20 0x7fbf4c006e90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009c00 tx=0x7fbf580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8623abc107b5b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:33.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf61ffb700 1 -- 172.21.15.89:0/443210228 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5800e710 con 0x7fbf4c006a20 2023-10-10T20:40:33.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf61ffb700 1 -- 172.21.15.89:0/443210228 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5800e870 con 0x7fbf4c006a20 2023-10-10T20:40:33.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.201+0000 7fbf61ffb700 1 -- 172.21.15.89:0/443210228 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5800ec30 con 0x7fbf4c006a20 2023-10-10T20:40:33.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.202+0000 7fbf63fff700 1 -- 172.21.15.89:0/443210228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf4c006a20 msgr2=0x7fbf4c006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:33.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.202+0000 7fbf63fff700 1 --2- 172.21.15.89:0/443210228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf4c006a20 0x7fbf4c006e90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009c00 tx=0x7fbf580091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:33.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.202+0000 7fbf63fff700 1 -- 172.21.15.89:0/443210228 shutdown_connections 2023-10-10T20:40:33.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.202+0000 7fbf63fff700 1 --2- 172.21.15.89:0/443210228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf4c006a20 0x7fbf4c006e90 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:33.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.202+0000 7fbf63fff700 1 -- 172.21.15.89:0/443210228 >> 172.21.15.89:0/443210228 conn(0x7fbf6406ddb0 msgr2=0x7fbf6406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:33.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.202+0000 7fbf63fff700 1 -- 172.21.15.89:0/443210228 shutdown_connections 2023-10-10T20:40:33.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.202+0000 7fbf63fff700 1 -- 172.21.15.89:0/443210228 wait complete. 2023-10-10T20:40:33.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.203+0000 7fbf63fff700 1 Processor -- start 2023-10-10T20:40:33.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.203+0000 7fbf63fff700 1 -- start start 2023-10-10T20:40:33.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.204+0000 7fbf63fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf4c006a20 0x7fbf641496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:33.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.204+0000 7fbf63fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf64069ca0 0x7fbf640a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:33.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.204+0000 7fbf63fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf640c5d60 0x7fbf640c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:33.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.204+0000 7fbf63fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640c6720 con 0x7fbf640c5d60 2023-10-10T20:40:33.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.204+0000 7fbf63fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6416a540 con 0x7fbf64069ca0 2023-10-10T20:40:33.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.204+0000 7fbf63fff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6406b000 con 0x7fbf4c006a20 2023-10-10T20:40:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.204+0000 7fbf637fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf640c5d60 0x7fbf640c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.204+0000 7fbf637fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf640c5d60 0x7fbf640c61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34168/0 (socket says 172.21.15.89:34168) 2023-10-10T20:40:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.204+0000 7fbf637fe700 1 -- 172.21.15.89:0/2947354338 learned_addr learned my addr 172.21.15.89:0/2947354338 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.204+0000 7fbf62ffd700 1 --2- 172.21.15.89:0/2947354338 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf4c006a20 0x7fbf641496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.204+0000 7fbf627fc700 1 --2- 172.21.15.89:0/2947354338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf64069ca0 0x7fbf640a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.204+0000 7fbf637fe700 1 -- 172.21.15.89:0/2947354338 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf4c006a20 msgr2=0x7fbf641496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.204+0000 7fbf637fe700 1 --2- 172.21.15.89:0/2947354338 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf4c006a20 0x7fbf641496c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.204+0000 7fbf637fe700 1 -- 172.21.15.89:0/2947354338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf64069ca0 msgr2=0x7fbf640a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.204+0000 7fbf637fe700 1 --2- 172.21.15.89:0/2947354338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf64069ca0 0x7fbf640a5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.204+0000 7fbf637fe700 1 -- 172.21.15.89:0/2947354338 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6406b2e0 con 0x7fbf640c5d60 2023-10-10T20:40:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.204+0000 7fbf637fe700 1 --2- 172.21.15.89:0/2947354338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf640c5d60 0x7fbf640c61e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fbf640642f0 tx=0x7fbf54009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.205+0000 7fbf4affd700 1 -- 172.21.15.89:0/2947354338 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54006820 con 0x7fbf640c5d60 2023-10-10T20:40:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.205+0000 7fbf4affd700 1 -- 172.21.15.89:0/2947354338 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf540069c0 con 0x7fbf640c5d60 2023-10-10T20:40:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.205+0000 7fbf63fff700 1 -- 172.21.15.89:0/2947354338 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6406b5c0 con 0x7fbf640c5d60 2023-10-10T20:40:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.205+0000 7fbf4affd700 1 -- 172.21.15.89:0/2947354338 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54006d80 con 0x7fbf640c5d60 2023-10-10T20:40:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.205+0000 7fbf63fff700 1 -- 172.21.15.89:0/2947354338 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf640a63a0 con 0x7fbf640c5d60 2023-10-10T20:40:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.206+0000 7fbf63fff700 1 -- 172.21.15.89:0/2947354338 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf640c90f0 con 0x7fbf640c5d60 2023-10-10T20:40:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.206+0000 7fbf4affd700 1 -- 172.21.15.89:0/2947354338 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fbf54007240 con 0x7fbf640c5d60 2023-10-10T20:40:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.206+0000 7fbf4affd700 1 --2- 172.21.15.89:0/2947354338 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbf50067290 0x7fbf50069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.206+0000 7fbf4affd700 1 -- 172.21.15.89:0/2947354338 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fbf54083a30 con 0x7fbf640c5d60 2023-10-10T20:40:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.206+0000 7fbf62ffd700 1 --2- 172.21.15.89:0/2947354338 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbf50067290 0x7fbf50069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:33.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.207+0000 7fbf62ffd700 1 --2- 172.21.15.89:0/2947354338 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbf50067290 0x7fbf50069740 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c001920 tx=0x7fbf5c006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:33.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.211+0000 7fbf4affd700 1 -- 172.21.15.89:0/2947354338 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf5404d5a0 con 0x7fbf640c5d60 2023-10-10T20:40:33.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:33.413+0000 7fbf63fff700 1 -- 172.21.15.89:0/2947354338 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbf6404c3c0 con 0x7fbf640c5d60 2023-10-10T20:40:34.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.230+0000 7fbf4affd700 1 -- 172.21.15.89:0/2947354338 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v96) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbf54019070 con 0x7fbf640c5d60 2023-10-10T20:40:34.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.233+0000 7fbf63fff700 1 -- 172.21.15.89:0/2947354338 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbf50067290 msgr2=0x7fbf50069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.233+0000 7fbf63fff700 1 --2- 172.21.15.89:0/2947354338 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbf50067290 0x7fbf50069740 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c001920 tx=0x7fbf5c006040 comp rx=0 tx=0).stop 2023-10-10T20:40:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.233+0000 7fbf63fff700 1 -- 172.21.15.89:0/2947354338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf640c5d60 msgr2=0x7fbf640c61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.233+0000 7fbf63fff700 1 --2- 172.21.15.89:0/2947354338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf640c5d60 0x7fbf640c61e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fbf640642f0 tx=0x7fbf54009040 comp rx=0 tx=0).stop 2023-10-10T20:40:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.233+0000 7fbf63fff700 1 -- 172.21.15.89:0/2947354338 shutdown_connections 2023-10-10T20:40:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.233+0000 7fbf63fff700 1 --2- 172.21.15.89:0/2947354338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf640c5d60 0x7fbf640c61e0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.233+0000 7fbf63fff700 1 --2- 172.21.15.89:0/2947354338 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbf4c006a20 0x7fbf641496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.233+0000 7fbf63fff700 1 --2- 172.21.15.89:0/2947354338 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbf50067290 0x7fbf50069740 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.233+0000 7fbf63fff700 1 --2- 172.21.15.89:0/2947354338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf64069ca0 0x7fbf640a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.233+0000 7fbf63fff700 1 -- 172.21.15.89:0/2947354338 >> 172.21.15.89:0/2947354338 conn(0x7fbf6406ddb0 msgr2=0x7fbf640c86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.233+0000 7fbf63fff700 1 -- 172.21.15.89:0/2947354338 shutdown_connections 2023-10-10T20:40:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.233+0000 7fbf63fff700 1 -- 172.21.15.89:0/2947354338 wait complete. 2023-10-10T20:40:34.238 INFO:teuthology.orchestra.run.smithi089.stderr: removed all blocklist entries 2023-10-10T20:40:34.248 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:40:34.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.362+0000 7f30761ad700 1 Processor -- start 2023-10-10T20:40:34.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.362+0000 7f30761ad700 1 -- start start 2023-10-10T20:40:34.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.363+0000 7f30761ad700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3070159840 0x7f30700cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:34.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.363+0000 7f30761ad700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f30700cba70 0x7f30700694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:34.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.363+0000 7f30761ad700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30700cfb30 con 0x7f3070159c10 2023-10-10T20:40:34.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.363+0000 7f30761ad700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30700cbf10 con 0x7f30700cba70 2023-10-10T20:40:34.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.363+0000 7f30761ad700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30700cf4e0 con 0x7f3070159840 2023-10-10T20:40:34.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.363+0000 7f306effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f30700cba70 0x7f30700694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:34.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.363+0000 7f306ffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3070159c10 0x7f30700ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39044/0 (socket says 172.21.15.89:39044) 2023-10-10T20:40:34.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.363+0000 7f306ffff700 1 -- 172.21.15.89:0/3689653877 learned_addr learned my addr 172.21.15.89:0/3689653877 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:34.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.363+0000 7f306f7fe700 1 --2- 172.21.15.89:0/3689653877 >> v2:172.21.15.111:3300/0 conn(0x7f3070159840 0x7f30700cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:34.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.363+0000 7f306effd700 1 -- 172.21.15.89:0/3689653877 >> v2:172.21.15.111:3300/0 conn(0x7f3070159840 msgr2=0x7f30700cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:34.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.363+0000 7f306effd700 1 --2- 172.21.15.89:0/3689653877 >> v2:172.21.15.111:3300/0 conn(0x7f3070159840 0x7f30700cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.363+0000 7f306effd700 1 -- 172.21.15.89:0/3689653877 >> v1:172.21.15.89:6789/0 conn(0x7f3070159c10 legacy=0x7f30700ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:34.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.363+0000 7f306effd700 1 -- 172.21.15.89:0/3689653877 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30700cf7c0 con 0x7f30700cba70 2023-10-10T20:40:34.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.363+0000 7f306e7fc700 1 -- 172.21.15.89:0/3689653877 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136308743 0 0) 0x7f30700cfb30 con 0x7f3070159c10 2023-10-10T20:40:34.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.363+0000 7f306effd700 1 --2- 172.21.15.89:0/3689653877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30700cba70 0x7f30700694a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f30640098e0 tx=0x7f30640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fceab5cdb6e4cd1d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:34.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.363+0000 7f306e7fc700 1 -- 172.21.15.89:0/3689653877 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3064011630 con 0x7f30700cba70 2023-10-10T20:40:34.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.363+0000 7f306e7fc700 1 -- 172.21.15.89:0/3689653877 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3064011790 con 0x7f30700cba70 2023-10-10T20:40:34.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.363+0000 7f306e7fc700 1 -- 172.21.15.89:0/3689653877 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3064011b30 con 0x7f30700cba70 2023-10-10T20:40:34.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.364+0000 7f30761ad700 1 -- 172.21.15.89:0/3689653877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30700cba70 msgr2=0x7f30700694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:34.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.364+0000 7f30761ad700 1 --2- 172.21.15.89:0/3689653877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30700cba70 0x7f30700694a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f30640098e0 tx=0x7f30640091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.364+0000 7f30761ad700 1 -- 172.21.15.89:0/3689653877 shutdown_connections 2023-10-10T20:40:34.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.364+0000 7f30761ad700 1 --2- 172.21.15.89:0/3689653877 >> v2:172.21.15.111:3300/0 conn(0x7f3070159840 0x7f30700cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.364+0000 7f30761ad700 1 --2- 172.21.15.89:0/3689653877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30700cba70 0x7f30700694a0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.364+0000 7f30761ad700 1 -- 172.21.15.89:0/3689653877 >> 172.21.15.89:0/3689653877 conn(0x7f3070068e10 msgr2=0x7f30700d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:34.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.364+0000 7f30761ad700 1 -- 172.21.15.89:0/3689653877 shutdown_connections 2023-10-10T20:40:34.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.364+0000 7f30761ad700 1 -- 172.21.15.89:0/3689653877 wait complete. 2023-10-10T20:40:34.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.365+0000 7f30761ad700 1 Processor -- start 2023-10-10T20:40:34.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.365+0000 7f30761ad700 1 -- start start 2023-10-10T20:40:34.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f30761ad700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30700cba70 0x7f307016d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:34.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f30761ad700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3070159840 0x7f30700c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:34.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f30761ad700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3070159c10 0x7f3070155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:34.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f30761ad700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30700c91f0 con 0x7f30700cba70 2023-10-10T20:40:34.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f30761ad700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30700c8720 con 0x7f3070159c10 2023-10-10T20:40:34.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f30761ad700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3070156190 con 0x7f3070159840 2023-10-10T20:40:34.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f306f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30700cba70 0x7f307016d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:34.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f306f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30700cba70 0x7f307016d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34192/0 (socket says 172.21.15.89:34192) 2023-10-10T20:40:34.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f306f7fe700 1 -- 172.21.15.89:0/540999524 learned_addr learned my addr 172.21.15.89:0/540999524 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:34.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f306effd700 1 --2- 172.21.15.89:0/540999524 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3070159840 0x7f30700c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:34.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f306ffff700 1 --2- 172.21.15.89:0/540999524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3070159c10 0x7f3070155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:34.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f306f7fe700 1 -- 172.21.15.89:0/540999524 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3070159840 msgr2=0x7f30700c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:34.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f306f7fe700 1 --2- 172.21.15.89:0/540999524 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3070159840 0x7f30700c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f306f7fe700 1 -- 172.21.15.89:0/540999524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3070159c10 msgr2=0x7f3070155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:34.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f306f7fe700 1 --2- 172.21.15.89:0/540999524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3070159c10 0x7f3070155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f306f7fe700 1 -- 172.21.15.89:0/540999524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3064009420 con 0x7f30700cba70 2023-10-10T20:40:34.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f306effd700 1 --2- 172.21.15.89:0/540999524 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3070159840 0x7f30700c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:34.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f306ffff700 1 --2- 172.21.15.89:0/540999524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3070159c10 0x7f3070155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:34.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f306f7fe700 1 --2- 172.21.15.89:0/540999524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30700cba70 0x7f307016d9f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f306000ec80 tx=0x7f30600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:34.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.366+0000 7f30577fe700 1 -- 172.21.15.89:0/540999524 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306000c9d0 con 0x7f30700cba70 2023-10-10T20:40:34.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.367+0000 7f30577fe700 1 -- 172.21.15.89:0/540999524 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306000cb30 con 0x7f30700cba70 2023-10-10T20:40:34.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.367+0000 7f30761ad700 1 -- 172.21.15.89:0/540999524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30700c8950 con 0x7f30700cba70 2023-10-10T20:40:34.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.367+0000 7f30577fe700 1 -- 172.21.15.89:0/540999524 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060013470 con 0x7f30700cba70 2023-10-10T20:40:34.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.367+0000 7f30761ad700 1 -- 172.21.15.89:0/540999524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30700c8f10 con 0x7f30700cba70 2023-10-10T20:40:34.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.368+0000 7f30761ad700 1 -- 172.21.15.89:0/540999524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30700ca050 con 0x7f30700cba70 2023-10-10T20:40:34.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.368+0000 7f30577fe700 1 -- 172.21.15.89:0/540999524 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f3060013820 con 0x7f30700cba70 2023-10-10T20:40:34.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.368+0000 7f30577fe700 1 --2- 172.21.15.89:0/540999524 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f305c067290 0x7f305c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:34.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.368+0000 7f306effd700 1 --2- 172.21.15.89:0/540999524 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f305c067290 0x7f305c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:34.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.369+0000 7f30577fe700 1 -- 172.21.15.89:0/540999524 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3060084ab0 con 0x7f30700cba70 2023-10-10T20:40:34.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.369+0000 7f306effd700 1 --2- 172.21.15.89:0/540999524 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f305c067290 0x7f305c069740 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f306400a010 tx=0x7f3064017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:34.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.373+0000 7f30577fe700 1 -- 172.21.15.89:0/540999524 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3060051c60 con 0x7f30700cba70 2023-10-10T20:40:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.628+0000 7f30761ad700 1 -- 172.21.15.89:0/540999524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f30700b1270 con 0x7f30700cba70 2023-10-10T20:40:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.628+0000 7f30577fe700 1 -- 172.21.15.89:0/540999524 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3060055280 con 0x7f30700cba70 2023-10-10T20:40:34.631 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:34.631 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:40:34.632 INFO:teuthology.orchestra.run.smithi089.stdout: "auth_dump": [ 2023-10-10T20:40:34.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:34.632 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.a", 2023-10-10T20:40:34.633 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlZLSNLhAA8WZTrsht6tAOP+gxaTXKMg==", 2023-10-10T20:40:34.633 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:34.633 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:40:34.634 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:40:34.634 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:40:34.635 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:34.635 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:34.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:34.636 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.b", 2023-10-10T20:40:34.636 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgyDEORAAMMoSxPFZc0bCIwWy8UsXgA==", 2023-10-10T20:40:34.637 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:34.637 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:40:34.637 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:40:34.638 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:40:34.638 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:34.638 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:34.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:34.639 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.c", 2023-10-10T20:40:34.640 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlMFxSNRAASuQEJFghLIOkTQWa4ckTvg==", 2023-10-10T20:40:34.640 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:34.640 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:40:34.641 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:40:34.641 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:40:34.641 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:34.641 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:34.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:34.642 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.d", 2023-10-10T20:40:34.642 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlPkvrBBAA2VLJmv2t8szTEGtCVwalJw==", 2023-10-10T20:40:34.642 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:34.643 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:40:34.643 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:40:34.643 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:40:34.643 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:34.643 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:34.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:34.644 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.0", 2023-10-10T20:40:34.644 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQArtSVltOcKHRAAk8+zUus04oTVI9KvfhjlXg==", 2023-10-10T20:40:34.645 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:34.645 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:40:34.645 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:40:34.645 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:34.645 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:34.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:34.646 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.1", 2023-10-10T20:40:34.646 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAutSVly8P0GhAAe1bvR63+ptJg84lizMNdQw==", 2023-10-10T20:40:34.646 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:34.647 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:40:34.647 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:40:34.647 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:34.647 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:34.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:34.648 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.2", 2023-10-10T20:40:34.648 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAxtSVl0x9fGRAAEKi7+6EW5mzPbebmz0+sgQ==", 2023-10-10T20:40:34.648 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:34.648 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:40:34.648 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:40:34.649 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:34.649 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:34.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:34.649 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.3", 2023-10-10T20:40:34.650 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA0tSVlfJyaGBAAa+8w54U6v7+npbdufKJruA==", 2023-10-10T20:40:34.650 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:34.650 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:40:34.650 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:40:34.651 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:34.651 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:34.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:34.652 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.4", 2023-10-10T20:40:34.652 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA6tSVlEQ/QFxAA/PRB3c90BuAlF6xekGfX9A==", 2023-10-10T20:40:34.652 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:34.652 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:40:34.652 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:40:34.653 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:34.653 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:34.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:34.653 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.5", 2023-10-10T20:40:34.654 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA9tSVlMOKrFRAAuBnIsEz2VmeSZi9XWzjlzA==", 2023-10-10T20:40:34.654 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:34.654 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:40:34.654 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:40:34.655 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:34.655 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:34.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:34.656 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.6", 2023-10-10T20:40:34.656 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBAtSVlH7PQFBAAHr6c7d4TYJEu/KSFEf+EOA==", 2023-10-10T20:40:34.657 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:34.657 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:40:34.658 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:40:34.658 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:34.658 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:34.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:34.659 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.7", 2023-10-10T20:40:34.660 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBDtSVlAhC1FBAAuE2BwAA/f7VOkZfVdqAG7A==", 2023-10-10T20:40:34.660 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:34.660 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:40:34.661 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:40:34.661 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:34.662 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:34.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:34.663 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.0", 2023-10-10T20:40:34.663 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlRd3HCBAAb0QP2VfTS7aelno1mFpGAA==", 2023-10-10T20:40:34.663 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:34.664 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:40:34.664 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow r", 2023-10-10T20:40:34.664 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:40:34.665 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:34.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:34.666 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.1", 2023-10-10T20:40:34.666 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlgBouERAAqPdE0KvJU3M53B6Y6AH5GQ==", 2023-10-10T20:40:34.667 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:34.667 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:40:34.667 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:40:34.667 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:40:34.668 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:40:34.668 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:34.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:34.669 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.2", 2023-10-10T20:40:34.670 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlXA4tDhAAoPVYQjECpzD+C67mnsY5IA==", 2023-10-10T20:40:34.670 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:34.670 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:40:34.671 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:40:34.671 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:40:34.672 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:40:34.672 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:34.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:34.673 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.3", 2023-10-10T20:40:34.673 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlZ5KGFhAAuKmZwbOfmmw1nAj5ry6tHw==", 2023-10-10T20:40:34.674 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:34.674 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:40:34.674 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:40:34.675 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:40:34.675 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:40:34.675 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:34.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:34.676 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.admin", 2023-10-10T20:40:34.677 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgpL2EhAAoDOki002rfBrauTgfu481Q==", 2023-10-10T20:40:34.677 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:34.678 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:40:34.678 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:40:34.678 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:40:34.679 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:34.679 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:34.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:34.680 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.x", 2023-10-10T20:40:34.680 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlBhaJKxAArraBxT5xWg515LfpJ7MlbA==", 2023-10-10T20:40:34.681 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:34.681 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:40:34.681 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:40:34.682 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:34.682 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:40:34.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:34.683 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.y", 2023-10-10T20:40:34.684 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlTZIyJxAAWAOsitHwsIxJWwp3b1Z98Q==", 2023-10-10T20:40:34.684 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:40:34.684 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:40:34.685 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:40:34.685 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:40:34.685 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.686 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:34.686 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-10T20:40:34.687 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:40:34.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.630+0000 7f30761ad700 1 -- 172.21.15.89:0/540999524 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f305c067290 msgr2=0x7f305c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:34.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.630+0000 7f30761ad700 1 --2- 172.21.15.89:0/540999524 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f305c067290 0x7f305c069740 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f306400a010 tx=0x7f3064017000 comp rx=0 tx=0).stop 2023-10-10T20:40:34.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.630+0000 7f30761ad700 1 -- 172.21.15.89:0/540999524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30700cba70 msgr2=0x7f307016d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:34.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.630+0000 7f30761ad700 1 --2- 172.21.15.89:0/540999524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30700cba70 0x7f307016d9f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f306000ec80 tx=0x7f30600091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.630+0000 7f30761ad700 1 -- 172.21.15.89:0/540999524 shutdown_connections 2023-10-10T20:40:34.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.630+0000 7f30761ad700 1 --2- 172.21.15.89:0/540999524 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f305c067290 0x7f305c069740 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.630+0000 7f30761ad700 1 --2- 172.21.15.89:0/540999524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3070159c10 0x7f3070155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.630+0000 7f30761ad700 1 --2- 172.21.15.89:0/540999524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30700cba70 0x7f307016d9f0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.630+0000 7f30761ad700 1 --2- 172.21.15.89:0/540999524 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3070159840 0x7f30700c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.630+0000 7f30761ad700 1 -- 172.21.15.89:0/540999524 >> 172.21.15.89:0/540999524 conn(0x7f3070068e10 msgr2=0x7f3070158950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:34.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.631+0000 7f30761ad700 1 -- 172.21.15.89:0/540999524 shutdown_connections 2023-10-10T20:40:34.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.631+0000 7f30761ad700 1 -- 172.21.15.89:0/540999524 wait complete. 2023-10-10T20:40:34.693 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:40:34.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.806+0000 7f2de2493700 1 Processor -- start 2023-10-10T20:40:34.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.806+0000 7f2de2493700 1 -- start start 2023-10-10T20:40:34.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.807+0000 7f2de2493700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2ddc159890 0x7f2ddc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:34.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.807+0000 7f2de2493700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ddc159c60 0x7f2ddc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:34.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.807+0000 7f2de2493700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ddc15a160 con 0x7f2ddc15db10 2023-10-10T20:40:34.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.807+0000 7f2de2493700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ddc0d13a0 con 0x7f2ddc159890 2023-10-10T20:40:34.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.807+0000 7f2de2493700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc15e140 con 0x7f2ddc159c60 2023-10-10T20:40:34.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.807+0000 7f2ddbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2ddc159890 0x7f2ddc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:34.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.807+0000 7f2ddbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2ddc159890 0x7f2ddc15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34202/0 (socket says 172.21.15.89:34202) 2023-10-10T20:40:34.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.807+0000 7f2ddbfff700 1 -- 172.21.15.89:0/2849642877 learned_addr learned my addr 172.21.15.89:0/2849642877 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:34.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.807+0000 7f2ddb7fe700 1 --2- 172.21.15.89:0/2849642877 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ddc159c60 0x7f2ddc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:34.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.808+0000 7f2ddbfff700 1 -- 172.21.15.89:0/2849642877 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ddc159c60 msgr2=0x7f2ddc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:34.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.808+0000 7f2ddbfff700 1 --2- 172.21.15.89:0/2849642877 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ddc159c60 0x7f2ddc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.808+0000 7f2ddbfff700 1 -- 172.21.15.89:0/2849642877 >> v1:172.21.15.111:6789/0 conn(0x7f2ddc15db10 legacy=0x7f2ddc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:34.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.808+0000 7f2ddbfff700 1 -- 172.21.15.89:0/2849642877 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc15a3d0 con 0x7f2ddc159890 2023-10-10T20:40:34.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.808+0000 7f2ddbfff700 1 --2- 172.21.15.89:0/2849642877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ddc159890 0x7f2ddc15abd0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0099b0 tx=0x7f2dcc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e6aad0ec18b397 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:34.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.808+0000 7f2ddaffd700 1 -- 172.21.15.89:0/2849642877 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc009ea0 con 0x7f2ddc159890 2023-10-10T20:40:34.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.808+0000 7f2ddaffd700 1 -- 172.21.15.89:0/2849642877 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dcc00b860 con 0x7f2ddc159890 2023-10-10T20:40:34.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.808+0000 7f2ddaffd700 1 -- 172.21.15.89:0/2849642877 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc00bc00 con 0x7f2ddc159890 2023-10-10T20:40:34.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.808+0000 7f2de2493700 1 -- 172.21.15.89:0/2849642877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ddc159890 msgr2=0x7f2ddc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:34.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.808+0000 7f2de2493700 1 --2- 172.21.15.89:0/2849642877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ddc159890 0x7f2ddc15abd0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0099b0 tx=0x7f2dcc009250 comp rx=0 tx=0).stop 2023-10-10T20:40:34.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.808+0000 7f2de2493700 1 -- 172.21.15.89:0/2849642877 shutdown_connections 2023-10-10T20:40:34.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.808+0000 7f2de2493700 1 --2- 172.21.15.89:0/2849642877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ddc159890 0x7f2ddc15abd0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.808+0000 7f2de2493700 1 --2- 172.21.15.89:0/2849642877 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ddc159c60 0x7f2ddc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.808+0000 7f2de2493700 1 -- 172.21.15.89:0/2849642877 >> 172.21.15.89:0/2849642877 conn(0x7f2ddc068e20 msgr2=0x7f2ddc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:34.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.808+0000 7f2de2493700 1 -- 172.21.15.89:0/2849642877 shutdown_connections 2023-10-10T20:40:34.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.809+0000 7f2de2493700 1 -- 172.21.15.89:0/2849642877 wait complete. 2023-10-10T20:40:34.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.810+0000 7f2de2493700 1 Processor -- start 2023-10-10T20:40:34.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.810+0000 7f2de2493700 1 -- start start 2023-10-10T20:40:34.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.810+0000 7f2de2493700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ddc159890 0x7f2ddc16e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:34.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.810+0000 7f2de2493700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ddc159c60 0x7f2ddc0b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:34.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.810+0000 7f2de2493700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ddc15db10 0x7f2ddc0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:34.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.810+0000 7f2de2493700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc0b3590 con 0x7f2ddc159890 2023-10-10T20:40:34.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.810+0000 7f2de2493700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc0c79b0 con 0x7f2ddc159c60 2023-10-10T20:40:34.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.810+0000 7f2de2493700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc0b43b0 con 0x7f2ddc15db10 2023-10-10T20:40:34.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.811+0000 7f2ddbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ddc159890 0x7f2ddc16e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:34.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.811+0000 7f2ddbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ddc159890 0x7f2ddc16e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34208/0 (socket says 172.21.15.89:34208) 2023-10-10T20:40:34.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.811+0000 7f2ddbfff700 1 -- 172.21.15.89:0/1990994766 learned_addr learned my addr 172.21.15.89:0/1990994766 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:34.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.811+0000 7f2de0a30700 1 --2- 172.21.15.89:0/1990994766 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ddc15db10 0x7f2ddc0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:34.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.811+0000 7f2ddb7fe700 1 --2- 172.21.15.89:0/1990994766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ddc159c60 0x7f2ddc0b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:34.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.811+0000 7f2ddbfff700 1 -- 172.21.15.89:0/1990994766 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ddc15db10 msgr2=0x7f2ddc0b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:34.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.811+0000 7f2ddbfff700 1 --2- 172.21.15.89:0/1990994766 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ddc15db10 0x7f2ddc0b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.811+0000 7f2ddbfff700 1 -- 172.21.15.89:0/1990994766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ddc159c60 msgr2=0x7f2ddc0b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:34.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.811+0000 7f2ddbfff700 1 --2- 172.21.15.89:0/1990994766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ddc159c60 0x7f2ddc0b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:34.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.811+0000 7f2ddbfff700 1 -- 172.21.15.89:0/1990994766 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc0094f0 con 0x7f2ddc159890 2023-10-10T20:40:34.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.811+0000 7f2ddbfff700 1 --2- 172.21.15.89:0/1990994766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ddc159890 0x7f2ddc16e2e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc0899c0 tx=0x7f2dcc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:34.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.811+0000 7f2dc3fff700 1 -- 172.21.15.89:0/1990994766 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc000d10 con 0x7f2ddc159890 2023-10-10T20:40:34.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.811+0000 7f2dc3fff700 1 -- 172.21.15.89:0/1990994766 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dcc00bbe0 con 0x7f2ddc159890 2023-10-10T20:40:34.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.811+0000 7f2de2493700 1 -- 172.21.15.89:0/1990994766 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ddc16e820 con 0x7f2ddc159890 2023-10-10T20:40:34.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.811+0000 7f2dc3fff700 1 -- 172.21.15.89:0/1990994766 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc011670 con 0x7f2ddc159890 2023-10-10T20:40:34.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.811+0000 7f2de2493700 1 -- 172.21.15.89:0/1990994766 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ddc0b3a80 con 0x7f2ddc159890 2023-10-10T20:40:34.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.812+0000 7f2de2493700 1 -- 172.21.15.89:0/1990994766 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ddc0b4690 con 0x7f2ddc159890 2023-10-10T20:40:34.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.813+0000 7f2dc3fff700 1 -- 172.21.15.89:0/1990994766 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f2dcc011aa0 con 0x7f2ddc159890 2023-10-10T20:40:34.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.813+0000 7f2dc3fff700 1 --2- 172.21.15.89:0/1990994766 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2dc8067290 0x7f2dc8069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:34.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.813+0000 7f2dc3fff700 1 -- 172.21.15.89:0/1990994766 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2dcc085f80 con 0x7f2ddc159890 2023-10-10T20:40:34.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.813+0000 7f2ddb7fe700 1 --2- 172.21.15.89:0/1990994766 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2dc8067290 0x7f2dc8069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:34.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.814+0000 7f2ddb7fe700 1 --2- 172.21.15.89:0/1990994766 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2dc8067290 0x7f2dc8069740 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc15b7b0 tx=0x7f2dd00091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:34.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:34.817+0000 7f2dc3fff700 1 -- 172.21.15.89:0/1990994766 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2dcc053130 con 0x7f2ddc159890 2023-10-10T20:40:35.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.003+0000 7f2de2493700 1 -- 172.21.15.89:0/1990994766 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ddc1590e0 con 0x7f2ddc159890 2023-10-10T20:40:35.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.004+0000 7f2dc3fff700 1 -- 172.21.15.89:0/1990994766 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+12135 (secure 0 0 0) 0x7f2dcc056750 con 0x7f2ddc159890 2023-10-10T20:40:35.006 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:35.007 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":96,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:40:34.228474+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:40:35.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.005+0000 7f2de2493700 1 -- 172.21.15.89:0/1990994766 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2dc8067290 msgr2=0x7f2dc8069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:35.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.005+0000 7f2de2493700 1 --2- 172.21.15.89:0/1990994766 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2dc8067290 0x7f2dc8069740 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc15b7b0 tx=0x7f2dd00091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.005+0000 7f2de2493700 1 -- 172.21.15.89:0/1990994766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ddc159890 msgr2=0x7f2ddc16e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:35.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.005+0000 7f2de2493700 1 --2- 172.21.15.89:0/1990994766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ddc159890 0x7f2ddc16e2e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc0899c0 tx=0x7f2dcc017000 comp rx=0 tx=0).stop 2023-10-10T20:40:35.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.005+0000 7f2de2493700 1 -- 172.21.15.89:0/1990994766 shutdown_connections 2023-10-10T20:40:35.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.005+0000 7f2de2493700 1 --2- 172.21.15.89:0/1990994766 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ddc15db10 0x7f2ddc0b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.005+0000 7f2de2493700 1 --2- 172.21.15.89:0/1990994766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ddc159890 0x7f2ddc16e2e0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.005+0000 7f2de2493700 1 --2- 172.21.15.89:0/1990994766 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2dc8067290 0x7f2dc8069740 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.005+0000 7f2de2493700 1 --2- 172.21.15.89:0/1990994766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ddc159c60 0x7f2ddc0b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.006+0000 7f2de2493700 1 -- 172.21.15.89:0/1990994766 >> 172.21.15.89:0/1990994766 conn(0x7f2ddc068e20 msgr2=0x7f2ddc15bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:35.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.006+0000 7f2de2493700 1 -- 172.21.15.89:0/1990994766 shutdown_connections 2023-10-10T20:40:35.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.006+0000 7f2de2493700 1 -- 172.21.15.89:0/1990994766 wait complete. 2023-10-10T20:40:35.020 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:40:35.020 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:40:35.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.136+0000 7f4c2b59e700 1 Processor -- start 2023-10-10T20:40:35.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.136+0000 7f4c2b59e700 1 -- start start 2023-10-10T20:40:35.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.137+0000 7f4c2b59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c2c0b4e70 0x7f4c2c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:35.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.137+0000 7f4c2b59e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4c2c0b1b40 0x7f4c2c15d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:35.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.137+0000 7f4c2b59e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c2c0cfb00 con 0x7f4c2c0b16d0 2023-10-10T20:40:35.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.137+0000 7f4c2b59e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c2c0b5c40 con 0x7f4c2c0b1b40 2023-10-10T20:40:35.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.137+0000 7f4c2b59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c0b5f20 con 0x7f4c2c0b4e70 2023-10-10T20:40:35.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.137+0000 7f4c21d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4c2c0b16d0 0x7f4c2c15d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39048/0 (socket says 172.21.15.89:39048) 2023-10-10T20:40:35.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.137+0000 7f4c21d9b700 1 -- 172.21.15.89:0/3148340590 learned_addr learned my addr 172.21.15.89:0/3148340590 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:35.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.137+0000 7f4c2a59c700 1 --2- 172.21.15.89:0/3148340590 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c2c0b4e70 0x7f4c2c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:35.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.137+0000 7f4c2ad9d700 1 --2- 172.21.15.89:0/3148340590 >> v2:172.21.15.111:3300/0 conn(0x7f4c2c0b1b40 0x7f4c2c15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:35.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.137+0000 7f4c29d9b700 1 -- 172.21.15.89:0/3148340590 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592577369 0 0) 0x7f4c2c0cfb00 con 0x7f4c2c0b16d0 2023-10-10T20:40:35.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.137+0000 7f4c29d9b700 1 -- 172.21.15.89:0/3148340590 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c18003570 con 0x7f4c2c0b16d0 2023-10-10T20:40:35.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.137+0000 7f4c2a59c700 1 -- 172.21.15.89:0/3148340590 >> v2:172.21.15.111:3300/0 conn(0x7f4c2c0b1b40 msgr2=0x7f4c2c15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:35.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.137+0000 7f4c2a59c700 1 --2- 172.21.15.89:0/3148340590 >> v2:172.21.15.111:3300/0 conn(0x7f4c2c0b1b40 0x7f4c2c15d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.137+0000 7f4c2a59c700 1 -- 172.21.15.89:0/3148340590 >> v1:172.21.15.89:6789/0 conn(0x7f4c2c0b16d0 legacy=0x7f4c2c15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:35.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.137+0000 7f4c2a59c700 1 -- 172.21.15.89:0/3148340590 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c0b6200 con 0x7f4c2c0b4e70 2023-10-10T20:40:35.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.138+0000 7f4c2a59c700 1 --2- 172.21.15.89:0/3148340590 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c2c0b4e70 0x7f4c2c0ccff0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c0098e0 tx=0x7f4c1c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23ff9ae63427a41 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:35.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.138+0000 7f4c29d9b700 1 -- 172.21.15.89:0/3148340590 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c011630 con 0x7f4c2c0b4e70 2023-10-10T20:40:35.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.138+0000 7f4c29d9b700 1 -- 172.21.15.89:0/3148340590 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c011790 con 0x7f4c2c0b4e70 2023-10-10T20:40:35.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.138+0000 7f4c29d9b700 1 -- 172.21.15.89:0/3148340590 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c011b30 con 0x7f4c2c0b4e70 2023-10-10T20:40:35.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.138+0000 7f4c2b59e700 1 -- 172.21.15.89:0/3148340590 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c2c0b4e70 msgr2=0x7f4c2c0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:35.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.138+0000 7f4c2b59e700 1 --2- 172.21.15.89:0/3148340590 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c2c0b4e70 0x7f4c2c0ccff0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c0098e0 tx=0x7f4c1c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.138+0000 7f4c2b59e700 1 -- 172.21.15.89:0/3148340590 shutdown_connections 2023-10-10T20:40:35.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.138+0000 7f4c2b59e700 1 --2- 172.21.15.89:0/3148340590 >> v2:172.21.15.111:3300/0 conn(0x7f4c2c0b1b40 0x7f4c2c15d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.138+0000 7f4c2b59e700 1 --2- 172.21.15.89:0/3148340590 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c2c0b4e70 0x7f4c2c0ccff0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.138+0000 7f4c2b59e700 1 -- 172.21.15.89:0/3148340590 >> 172.21.15.89:0/3148340590 conn(0x7f4c2c0d10c0 msgr2=0x7f4c2c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:35.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.138+0000 7f4c2b59e700 1 -- 172.21.15.89:0/3148340590 shutdown_connections 2023-10-10T20:40:35.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.138+0000 7f4c2b59e700 1 -- 172.21.15.89:0/3148340590 wait complete. 2023-10-10T20:40:35.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.139+0000 7f4c2b59e700 1 Processor -- start 2023-10-10T20:40:35.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.139+0000 7f4c2b59e700 1 -- start start 2023-10-10T20:40:35.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.140+0000 7f4c2b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c2c0b16d0 0x7f4c2c0c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:35.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.140+0000 7f4c2b59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c2c0b1b40 0x7f4c2c145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:35.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.140+0000 7f4c2b59e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c2c0b4e70 0x7f4c2c16a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:35.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.140+0000 7f4c2b59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c0c6560 con 0x7f4c2c0b16d0 2023-10-10T20:40:35.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.140+0000 7f4c2b59e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c16ad10 con 0x7f4c2c0b4e70 2023-10-10T20:40:35.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.140+0000 7f4c2b59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c0c5950 con 0x7f4c2c0b1b40 2023-10-10T20:40:35.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.140+0000 7f4c2a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c2c0b16d0 0x7f4c2c0c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:35.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.140+0000 7f4c2a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c2c0b16d0 0x7f4c2c0c69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34222/0 (socket says 172.21.15.89:34222) 2023-10-10T20:40:35.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.140+0000 7f4c2a59c700 1 -- 172.21.15.89:0/1331085616 learned_addr learned my addr 172.21.15.89:0/1331085616 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:35.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.140+0000 7f4c21d9b700 1 --2- 172.21.15.89:0/1331085616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c2c0b1b40 0x7f4c2c145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:35.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.140+0000 7f4c2ad9d700 1 --2- 172.21.15.89:0/1331085616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c2c0b4e70 0x7f4c2c16a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:35.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.140+0000 7f4c2a59c700 1 -- 172.21.15.89:0/1331085616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c2c0b1b40 msgr2=0x7f4c2c145c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:35.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.140+0000 7f4c2a59c700 1 --2- 172.21.15.89:0/1331085616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c2c0b1b40 0x7f4c2c145c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.140+0000 7f4c2a59c700 1 -- 172.21.15.89:0/1331085616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c2c0b4e70 msgr2=0x7f4c2c16a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:35.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.140+0000 7f4c2a59c700 1 --2- 172.21.15.89:0/1331085616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c2c0b4e70 0x7f4c2c16a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.140+0000 7f4c2a59c700 1 -- 172.21.15.89:0/1331085616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c009420 con 0x7f4c2c0b16d0 2023-10-10T20:40:35.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.140+0000 7f4c21d9b700 1 --2- 172.21.15.89:0/1331085616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c2c0b1b40 0x7f4c2c145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:35.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.141+0000 7f4c2a59c700 1 --2- 172.21.15.89:0/1331085616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c2c0b16d0 0x7f4c2c0c69f0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0b6060 tx=0x7f4c1c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:35.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.141+0000 7f4c22ffd700 1 -- 172.21.15.89:0/1331085616 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c011b10 con 0x7f4c2c0b16d0 2023-10-10T20:40:35.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.141+0000 7f4c22ffd700 1 -- 172.21.15.89:0/1331085616 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c0253f0 con 0x7f4c2c0b16d0 2023-10-10T20:40:35.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.141+0000 7f4c2b59e700 1 -- 172.21.15.89:0/1331085616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2c0c5c30 con 0x7f4c2c0b16d0 2023-10-10T20:40:35.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.141+0000 7f4c22ffd700 1 -- 172.21.15.89:0/1331085616 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0257b0 con 0x7f4c2c0b16d0 2023-10-10T20:40:35.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.141+0000 7f4c2b59e700 1 -- 172.21.15.89:0/1331085616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c2c0c61f0 con 0x7f4c2c0b16d0 2023-10-10T20:40:35.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.142+0000 7f4c2b59e700 1 -- 172.21.15.89:0/1331085616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c2c0ce180 con 0x7f4c2c0b16d0 2023-10-10T20:40:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.142+0000 7f4c22ffd700 1 -- 172.21.15.89:0/1331085616 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f4c1c025be0 con 0x7f4c2c0b16d0 2023-10-10T20:40:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.143+0000 7f4c22ffd700 1 --2- 172.21.15.89:0/1331085616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c0c067310 0x7f4c0c0697c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.143+0000 7f4c22ffd700 1 -- 172.21.15.89:0/1331085616 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4c1c085f80 con 0x7f4c2c0b16d0 2023-10-10T20:40:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.143+0000 7f4c21d9b700 1 --2- 172.21.15.89:0/1331085616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c0c067310 0x7f4c0c0697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.143+0000 7f4c21d9b700 1 --2- 172.21.15.89:0/1331085616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c0c067310 0x7f4c0c0697c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4c140058d0 tx=0x7f4c14009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.147+0000 7f4c22ffd700 1 -- 172.21.15.89:0/1331085616 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c1c0530f0 con 0x7f4c2c0b16d0 2023-10-10T20:40:35.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.331+0000 7f4c2b59e700 1 -- 172.21.15.89:0/1331085616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4c2c0c6840 con 0x7f4c2c0b16d0 2023-10-10T20:40:35.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.332+0000 7f4c22ffd700 1 -- 172.21.15.89:0/1331085616 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4c1c056710 con 0x7f4c2c0b16d0 2023-10-10T20:40:35.334 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:40:35.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.334+0000 7f4c2b59e700 1 -- 172.21.15.89:0/1331085616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c0c067310 msgr2=0x7f4c0c0697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:35.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.334+0000 7f4c2b59e700 1 --2- 172.21.15.89:0/1331085616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c0c067310 0x7f4c0c0697c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4c140058d0 tx=0x7f4c14009040 comp rx=0 tx=0).stop 2023-10-10T20:40:35.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.334+0000 7f4c2b59e700 1 -- 172.21.15.89:0/1331085616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c2c0b16d0 msgr2=0x7f4c2c0c69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:35.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.334+0000 7f4c2b59e700 1 --2- 172.21.15.89:0/1331085616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c2c0b16d0 0x7f4c2c0c69f0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0b6060 tx=0x7f4c1c017000 comp rx=0 tx=0).stop 2023-10-10T20:40:35.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.334+0000 7f4c2b59e700 1 -- 172.21.15.89:0/1331085616 shutdown_connections 2023-10-10T20:40:35.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.334+0000 7f4c2b59e700 1 --2- 172.21.15.89:0/1331085616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c0c067310 0x7f4c0c0697c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.334+0000 7f4c2b59e700 1 --2- 172.21.15.89:0/1331085616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c2c0b4e70 0x7f4c2c16a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.334+0000 7f4c2b59e700 1 --2- 172.21.15.89:0/1331085616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c2c0b16d0 0x7f4c2c0c69f0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.334+0000 7f4c2b59e700 1 --2- 172.21.15.89:0/1331085616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c2c0b1b40 0x7f4c2c145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.334+0000 7f4c2b59e700 1 -- 172.21.15.89:0/1331085616 >> 172.21.15.89:0/1331085616 conn(0x7f4c2c0d10c0 msgr2=0x7f4c2c0b53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:35.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.334+0000 7f4c2b59e700 1 -- 172.21.15.89:0/1331085616 shutdown_connections 2023-10-10T20:40:35.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.334+0000 7f4c2b59e700 1 -- 172.21.15.89:0/1331085616 wait complete. 2023-10-10T20:40:35.349 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:40:35.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.463+0000 7f38f451d700 1 Processor -- start 2023-10-10T20:40:35.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.463+0000 7f38f451d700 1 -- start start 2023-10-10T20:40:35.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.464+0000 7f38f451d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38ec0b1c20 0x7f38ec1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:35.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.464+0000 7f38f451d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f38ec157820 0x7f38ec0abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:35.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.464+0000 7f38f451d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38ec157c80 con 0x7f38ec0b1850 2023-10-10T20:40:35.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.464+0000 7f38f451d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ec06fc30 con 0x7f38ec157820 2023-10-10T20:40:35.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.464+0000 7f38f451d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0cc260 con 0x7f38ec0b1c20 2023-10-10T20:40:35.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.464+0000 7f38f1ab8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38ec0b1c20 0x7f38ec1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:35.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.464+0000 7f38f2aba700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f38ec157820 0x7f38ec0abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:35.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.464+0000 7f38f1ab8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38ec0b1c20 0x7f38ec1572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:45756/0 (socket says 172.21.15.89:45756) 2023-10-10T20:40:35.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.464+0000 7f38f1ab8700 1 -- 172.21.15.89:0/2844564780 learned_addr learned my addr 172.21.15.89:0/2844564780 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:35.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.464+0000 7f38f1ab8700 1 -- 172.21.15.89:0/2844564780 >> v2:172.21.15.111:3300/0 conn(0x7f38ec157820 msgr2=0x7f38ec0abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:35.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.464+0000 7f38f1ab8700 1 --2- 172.21.15.89:0/2844564780 >> v2:172.21.15.111:3300/0 conn(0x7f38ec157820 0x7f38ec0abf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.464+0000 7f38f1ab8700 1 -- 172.21.15.89:0/2844564780 >> v1:172.21.15.111:6789/0 conn(0x7f38ec0b1850 legacy=0x7f38ec0ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:35.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.464+0000 7f38f1ab8700 1 -- 172.21.15.89:0/2844564780 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec0cb300 con 0x7f38ec0b1c20 2023-10-10T20:40:35.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.465+0000 7f38f1ab8700 1 --2- 172.21.15.89:0/2844564780 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38ec0b1c20 0x7f38ec1572e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0099b0 tx=0x7f38dc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3159d98bbe4c3c64 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:35.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.465+0000 7f38f12b7700 1 -- 172.21.15.89:0/2844564780 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc009ea0 con 0x7f38ec0b1c20 2023-10-10T20:40:35.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.465+0000 7f38f12b7700 1 -- 172.21.15.89:0/2844564780 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc00b860 con 0x7f38ec0b1c20 2023-10-10T20:40:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.465+0000 7f38f12b7700 1 -- 172.21.15.89:0/2844564780 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc00bc00 con 0x7f38ec0b1c20 2023-10-10T20:40:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.465+0000 7f38f451d700 1 -- 172.21.15.89:0/2844564780 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38ec0b1c20 msgr2=0x7f38ec1572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.465+0000 7f38f451d700 1 --2- 172.21.15.89:0/2844564780 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38ec0b1c20 0x7f38ec1572e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0099b0 tx=0x7f38dc009250 comp rx=0 tx=0).stop 2023-10-10T20:40:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.465+0000 7f38f451d700 1 -- 172.21.15.89:0/2844564780 shutdown_connections 2023-10-10T20:40:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.465+0000 7f38f451d700 1 --2- 172.21.15.89:0/2844564780 >> v2:172.21.15.111:3300/0 conn(0x7f38ec157820 0x7f38ec0abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.465+0000 7f38f451d700 1 --2- 172.21.15.89:0/2844564780 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38ec0b1c20 0x7f38ec1572e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.465+0000 7f38f451d700 1 -- 172.21.15.89:0/2844564780 >> 172.21.15.89:0/2844564780 conn(0x7f38ec06a320 msgr2=0x7f38ec0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:35.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.465+0000 7f38f451d700 1 -- 172.21.15.89:0/2844564780 shutdown_connections 2023-10-10T20:40:35.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.465+0000 7f38f451d700 1 -- 172.21.15.89:0/2844564780 wait complete. 2023-10-10T20:40:35.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.466+0000 7f38f451d700 1 Processor -- start 2023-10-10T20:40:35.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.466+0000 7f38f451d700 1 -- start start 2023-10-10T20:40:35.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.467+0000 7f38f451d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38ec0b1850 0x7f38ec16b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:35.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.467+0000 7f38f451d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38ec0b1c20 0x7f38ec144eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:35.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.467+0000 7f38f451d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38ec157820 0x7f38ec157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:35.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.467+0000 7f38f451d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec15b7c0 con 0x7f38ec0b1850 2023-10-10T20:40:35.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.467+0000 7f38f451d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec167ba0 con 0x7f38ec157820 2023-10-10T20:40:35.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.467+0000 7f38f451d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec16ba20 con 0x7f38ec0b1c20 2023-10-10T20:40:35.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.467+0000 7f38f22b9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38ec0b1850 0x7f38ec16b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:35.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.467+0000 7f38f22b9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38ec0b1850 0x7f38ec16b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34228/0 (socket says 172.21.15.89:34228) 2023-10-10T20:40:35.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.467+0000 7f38f22b9700 1 -- 172.21.15.89:0/1160694703 learned_addr learned my addr 172.21.15.89:0/1160694703 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:35.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.467+0000 7f38f1ab8700 1 --2- 172.21.15.89:0/1160694703 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38ec0b1c20 0x7f38ec144eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:35.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.467+0000 7f38f2aba700 1 --2- 172.21.15.89:0/1160694703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38ec157820 0x7f38ec157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:35.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.468+0000 7f38f22b9700 1 -- 172.21.15.89:0/1160694703 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38ec0b1c20 msgr2=0x7f38ec144eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:35.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.468+0000 7f38f22b9700 1 --2- 172.21.15.89:0/1160694703 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38ec0b1c20 0x7f38ec144eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.468+0000 7f38f22b9700 1 -- 172.21.15.89:0/1160694703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38ec157820 msgr2=0x7f38ec157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:35.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.468+0000 7f38f22b9700 1 --2- 172.21.15.89:0/1160694703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38ec157820 0x7f38ec157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.468+0000 7f38f22b9700 1 -- 172.21.15.89:0/1160694703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc0094f0 con 0x7f38ec0b1850 2023-10-10T20:40:35.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.468+0000 7f38f1ab8700 1 --2- 172.21.15.89:0/1160694703 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38ec0b1c20 0x7f38ec144eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:35.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.468+0000 7f38f2aba700 1 --2- 172.21.15.89:0/1160694703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38ec157820 0x7f38ec157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:35.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.468+0000 7f38f22b9700 1 --2- 172.21.15.89:0/1160694703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38ec0b1850 0x7f38ec16b120 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f38e800c790 tx=0x7f38e8007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:35.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.468+0000 7f38e27fc700 1 -- 172.21.15.89:0/1160694703 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8011590 con 0x7f38ec0b1850 2023-10-10T20:40:35.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.468+0000 7f38e27fc700 1 -- 172.21.15.89:0/1160694703 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e8011710 con 0x7f38ec0b1850 2023-10-10T20:40:35.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.468+0000 7f38f451d700 1 -- 172.21.15.89:0/1160694703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ec16bd00 con 0x7f38ec0b1850 2023-10-10T20:40:35.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.468+0000 7f38e27fc700 1 -- 172.21.15.89:0/1160694703 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8011ad0 con 0x7f38ec0b1850 2023-10-10T20:40:35.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.468+0000 7f38f451d700 1 -- 172.21.15.89:0/1160694703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ec15b450 con 0x7f38ec0b1850 2023-10-10T20:40:35.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.469+0000 7f38f451d700 1 -- 172.21.15.89:0/1160694703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ec04c3c0 con 0x7f38ec0b1850 2023-10-10T20:40:35.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.469+0000 7f38e27fc700 1 -- 172.21.15.89:0/1160694703 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f38e801b3b0 con 0x7f38ec0b1850 2023-10-10T20:40:35.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.470+0000 7f38e27fc700 1 --2- 172.21.15.89:0/1160694703 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f38d0067290 0x7f38d0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:35.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.470+0000 7f38e27fc700 1 -- 172.21.15.89:0/1160694703 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f38e8083cc0 con 0x7f38ec0b1850 2023-10-10T20:40:35.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.470+0000 7f38f1ab8700 1 --2- 172.21.15.89:0/1160694703 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f38d0067290 0x7f38d0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:35.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.470+0000 7f38f1ab8700 1 --2- 172.21.15.89:0/1160694703 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f38d0067290 0x7f38d0069740 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0099b0 tx=0x7f38dc017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:35.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.474+0000 7f38e27fc700 1 -- 172.21.15.89:0/1160694703 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38e808a080 con 0x7f38ec0b1850 2023-10-10T20:40:35.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.657+0000 7f38f451d700 1 -- 172.21.15.89:0/1160694703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f38ec0cb5e0 con 0x7f38ec0b1850 2023-10-10T20:40:35.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.657+0000 7f38e27fc700 1 -- 172.21.15.89:0/1160694703 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v96) v1 ==== 84+0+65 (secure 0 0 0) 0x7f38e8017070 con 0x7f38ec0b1850 2023-10-10T20:40:35.660 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:35.660 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:40:35.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:35.661 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2023-10-10T20:40:35.661 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2023-10-10T20:40:35.661 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:35.661 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:40:35.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.659+0000 7f38f451d700 1 -- 172.21.15.89:0/1160694703 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f38d0067290 msgr2=0x7f38d0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:35.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.659+0000 7f38f451d700 1 --2- 172.21.15.89:0/1160694703 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f38d0067290 0x7f38d0069740 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0099b0 tx=0x7f38dc017000 comp rx=0 tx=0).stop 2023-10-10T20:40:35.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.659+0000 7f38f451d700 1 -- 172.21.15.89:0/1160694703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38ec0b1850 msgr2=0x7f38ec16b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:35.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.659+0000 7f38f451d700 1 --2- 172.21.15.89:0/1160694703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38ec0b1850 0x7f38ec16b120 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f38e800c790 tx=0x7f38e8007f40 comp rx=0 tx=0).stop 2023-10-10T20:40:35.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.659+0000 7f38f451d700 1 -- 172.21.15.89:0/1160694703 shutdown_connections 2023-10-10T20:40:35.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.659+0000 7f38f451d700 1 --2- 172.21.15.89:0/1160694703 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f38d0067290 0x7f38d0069740 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.659+0000 7f38f451d700 1 --2- 172.21.15.89:0/1160694703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38ec157820 0x7f38ec157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.659+0000 7f38f451d700 1 --2- 172.21.15.89:0/1160694703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38ec0b1850 0x7f38ec16b120 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.659+0000 7f38f451d700 1 --2- 172.21.15.89:0/1160694703 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f38ec0b1c20 0x7f38ec144eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.659+0000 7f38f451d700 1 -- 172.21.15.89:0/1160694703 >> 172.21.15.89:0/1160694703 conn(0x7f38ec06a320 msgr2=0x7f38ec0abf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:35.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.659+0000 7f38f451d700 1 -- 172.21.15.89:0/1160694703 shutdown_connections 2023-10-10T20:40:35.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.659+0000 7f38f451d700 1 -- 172.21.15.89:0/1160694703 wait complete. 2023-10-10T20:40:35.673 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:40:35.674 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:40:35.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.790+0000 7fb5fa73a700 1 Processor -- start 2023-10-10T20:40:35.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.790+0000 7fb5fa73a700 1 -- start start 2023-10-10T20:40:35.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.791+0000 7fb5fa73a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb5f4069a50 0x7fb5f40c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:35.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.791+0000 7fb5fa73a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb5f4069e20 0x7fb5f41569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:35.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.791+0000 7fb5fa73a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f40ac030 con 0x7fb5f40c8490 2023-10-10T20:40:35.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.791+0000 7fb5fa73a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f40ab820 con 0x7fb5f4069e20 2023-10-10T20:40:35.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.791+0000 7fb5fa73a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f406dbb0 con 0x7fb5f4069a50 2023-10-10T20:40:35.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.791+0000 7fb5f8cd7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb5f4069e20 0x7fb5f41569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:35.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.791+0000 7fb5f37fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb5f40c8490 0x7fb5f415a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39058/0 (socket says 172.21.15.89:39058) 2023-10-10T20:40:35.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.791+0000 7fb5f37fe700 1 -- 172.21.15.89:0/1281768083 learned_addr learned my addr 172.21.15.89:0/1281768083 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:35.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.792+0000 7fb5f3fff700 1 --2- 172.21.15.89:0/1281768083 >> v2:172.21.15.111:3300/0 conn(0x7fb5f4069a50 0x7fb5f40c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:35.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.792+0000 7fb5f8cd7700 1 -- 172.21.15.89:0/1281768083 >> v2:172.21.15.111:3300/0 conn(0x7fb5f4069a50 msgr2=0x7fb5f40c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:35.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.792+0000 7fb5f8cd7700 1 --2- 172.21.15.89:0/1281768083 >> v2:172.21.15.111:3300/0 conn(0x7fb5f4069a50 0x7fb5f40c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.792+0000 7fb5f8cd7700 1 -- 172.21.15.89:0/1281768083 >> v1:172.21.15.89:6789/0 conn(0x7fb5f40c8490 legacy=0x7fb5f415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:35.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.792+0000 7fb5f8cd7700 1 -- 172.21.15.89:0/1281768083 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f40aba70 con 0x7fb5f4069e20 2023-10-10T20:40:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.792+0000 7fb5f8cd7700 1 --2- 172.21.15.89:0/1281768083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f4069e20 0x7fb5f41569b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb5f406b970 tx=0x7fb5ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b673fef2192229f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.792+0000 7fb5f2ffd700 1 -- 172.21.15.89:0/1281768083 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec00e070 con 0x7fb5f4069e20 2023-10-10T20:40:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.792+0000 7fb5f2ffd700 1 -- 172.21.15.89:0/1281768083 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ec00bc30 con 0x7fb5f4069e20 2023-10-10T20:40:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.792+0000 7fb5f2ffd700 1 -- 172.21.15.89:0/1281768083 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec012430 con 0x7fb5f4069e20 2023-10-10T20:40:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.792+0000 7fb5fa73a700 1 -- 172.21.15.89:0/1281768083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f4069e20 msgr2=0x7fb5f41569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.792+0000 7fb5fa73a700 1 --2- 172.21.15.89:0/1281768083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f4069e20 0x7fb5f41569b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb5f406b970 tx=0x7fb5ec009780 comp rx=0 tx=0).stop 2023-10-10T20:40:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.792+0000 7fb5fa73a700 1 -- 172.21.15.89:0/1281768083 shutdown_connections 2023-10-10T20:40:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.792+0000 7fb5fa73a700 1 --2- 172.21.15.89:0/1281768083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f4069e20 0x7fb5f41569b0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.793+0000 7fb5fa73a700 1 --2- 172.21.15.89:0/1281768083 >> v2:172.21.15.111:3300/0 conn(0x7fb5f4069a50 0x7fb5f40c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.793+0000 7fb5fa73a700 1 -- 172.21.15.89:0/1281768083 >> 172.21.15.89:0/1281768083 conn(0x7fb5f40c8f90 msgr2=0x7fb5f40c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.793+0000 7fb5fa73a700 1 -- 172.21.15.89:0/1281768083 shutdown_connections 2023-10-10T20:40:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.793+0000 7fb5fa73a700 1 -- 172.21.15.89:0/1281768083 wait complete. 2023-10-10T20:40:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.794+0000 7fb5fa73a700 1 Processor -- start 2023-10-10T20:40:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.794+0000 7fb5fa73a700 1 -- start start 2023-10-10T20:40:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.794+0000 7fb5fa73a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5f4069a50 0x7fb5f40c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.794+0000 7fb5fa73a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5f4069e20 0x7fb5f41469f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.794+0000 7fb5fa73a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f40c8490 0x7fb5f414d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5fa73a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f40c07a0 con 0x7fb5f40c8490 2023-10-10T20:40:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5fa73a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f414ade0 con 0x7fb5f4069e20 2023-10-10T20:40:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5fa73a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f414b3a0 con 0x7fb5f4069a50 2023-10-10T20:40:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5f8cd7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f40c8490 0x7fb5f414d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5f8cd7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f40c8490 0x7fb5f414d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34244/0 (socket says 172.21.15.89:34244) 2023-10-10T20:40:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5f8cd7700 1 -- 172.21.15.89:0/145404248 learned_addr learned my addr 172.21.15.89:0/145404248 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5f37fe700 1 --2- 172.21.15.89:0/145404248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5f4069e20 0x7fb5f41469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5f8cd7700 1 -- 172.21.15.89:0/145404248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5f4069a50 msgr2=0x7fb5f40c1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5f3fff700 1 --2- 172.21.15.89:0/145404248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5f4069a50 0x7fb5f40c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5f8cd7700 1 --2- 172.21.15.89:0/145404248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5f4069a50 0x7fb5f40c1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5f8cd7700 1 -- 172.21.15.89:0/145404248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5f4069e20 msgr2=0x7fb5f41469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5f8cd7700 1 --2- 172.21.15.89:0/145404248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5f4069e20 0x7fb5f41469f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5f8cd7700 1 -- 172.21.15.89:0/145404248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec009a20 con 0x7fb5f40c8490 2023-10-10T20:40:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5f37fe700 1 --2- 172.21.15.89:0/145404248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5f4069e20 0x7fb5f41469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5f3fff700 1 --2- 172.21.15.89:0/145404248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5f4069a50 0x7fb5f40c1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5f8cd7700 1 --2- 172.21.15.89:0/145404248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f40c8490 0x7fb5f414d0e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec013000 tx=0x7fb5ec019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5dbfff700 1 -- 172.21.15.89:0/145404248 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec01e630 con 0x7fb5f40c8490 2023-10-10T20:40:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5dbfff700 1 -- 172.21.15.89:0/145404248 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ec01e790 con 0x7fb5f40c8490 2023-10-10T20:40:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5fa73a700 1 -- 172.21.15.89:0/145404248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f40c21c0 con 0x7fb5f40c8490 2023-10-10T20:40:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5dbfff700 1 -- 172.21.15.89:0/145404248 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec01eb50 con 0x7fb5f40c8490 2023-10-10T20:40:35.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.795+0000 7fb5fa73a700 1 -- 172.21.15.89:0/145404248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f40c2760 con 0x7fb5f40c8490 2023-10-10T20:40:35.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.796+0000 7fb5fa73a700 1 -- 172.21.15.89:0/145404248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f414b550 con 0x7fb5f40c8490 2023-10-10T20:40:35.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.797+0000 7fb5dbfff700 1 -- 172.21.15.89:0/145404248 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fb5ec024070 con 0x7fb5f40c8490 2023-10-10T20:40:35.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.797+0000 7fb5dbfff700 1 --2- 172.21.15.89:0/145404248 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb5e0067290 0x7fb5e0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:35.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.797+0000 7fb5dbfff700 1 -- 172.21.15.89:0/145404248 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb5ec087020 con 0x7fb5f40c8490 2023-10-10T20:40:35.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.797+0000 7fb5f3fff700 1 --2- 172.21.15.89:0/145404248 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb5e0067290 0x7fb5e0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:35.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.797+0000 7fb5f3fff700 1 --2- 172.21.15.89:0/145404248 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb5e0067290 0x7fb5e0069740 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4006fd0 tx=0x7fb5e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:35.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.801+0000 7fb5dbfff700 1 -- 172.21.15.89:0/145404248 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5ec0b85a0 con 0x7fb5f40c8490 2023-10-10T20:40:35.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:35.983+0000 7fb5fa73a700 1 -- 172.21.15.89:0/145404248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb5f40718b0 con 0x7fb5f40c8490 2023-10-10T20:40:36.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.239+0000 7fb5dbfff700 1 -- 172.21.15.89:0/145404248 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v97) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb5f414b550 con 0x7fb5f40c8490 2023-10-10T20:40:36.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.241+0000 7fb5fa73a700 1 -- 172.21.15.89:0/145404248 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb5e0067290 msgr2=0x7fb5e0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:36.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.241+0000 7fb5fa73a700 1 --2- 172.21.15.89:0/145404248 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb5e0067290 0x7fb5e0069740 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4006fd0 tx=0x7fb5e4008040 comp rx=0 tx=0).stop 2023-10-10T20:40:36.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.242+0000 7fb5fa73a700 1 -- 172.21.15.89:0/145404248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f40c8490 msgr2=0x7fb5f414d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:36.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.242+0000 7fb5fa73a700 1 --2- 172.21.15.89:0/145404248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f40c8490 0x7fb5f414d0e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec013000 tx=0x7fb5ec019000 comp rx=0 tx=0).stop 2023-10-10T20:40:36.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.242+0000 7fb5fa73a700 1 -- 172.21.15.89:0/145404248 shutdown_connections 2023-10-10T20:40:36.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.242+0000 7fb5fa73a700 1 --2- 172.21.15.89:0/145404248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f40c8490 0x7fb5f414d0e0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:36.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.242+0000 7fb5fa73a700 1 --2- 172.21.15.89:0/145404248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5f4069a50 0x7fb5f40c1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:36.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.242+0000 7fb5fa73a700 1 --2- 172.21.15.89:0/145404248 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb5e0067290 0x7fb5e0069740 secure :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4006fd0 tx=0x7fb5e4008040 comp rx=0 tx=0).stop 2023-10-10T20:40:36.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.242+0000 7fb5fa73a700 1 --2- 172.21.15.89:0/145404248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5f4069e20 0x7fb5f41469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:36.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.242+0000 7fb5fa73a700 1 -- 172.21.15.89:0/145404248 >> 172.21.15.89:0/145404248 conn(0x7fb5f40c8f90 msgr2=0x7fb5f40cb270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:36.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.242+0000 7fb5fa73a700 1 -- 172.21.15.89:0/145404248 shutdown_connections 2023-10-10T20:40:36.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.242+0000 7fb5fa73a700 1 -- 172.21.15.89:0/145404248 wait complete. 2023-10-10T20:40:36.247 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2023-10-10T20:40:36.260 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:40:36.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.379+0000 7fcaaa6c1700 1 Processor -- start 2023-10-10T20:40:36.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.379+0000 7fcaaa6c1700 1 -- start start 2023-10-10T20:40:36.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.380+0000 7fcaaa6c1700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcaa40c9700 0x7fcaa40caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:36.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.380+0000 7fcaaa6c1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcaa40cb430 0x7fcaa40ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:36.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.380+0000 7fcaaa6c1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa40adf00 con 0x7fcaa40b0360 2023-10-10T20:40:36.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.380+0000 7fcaaa6c1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa40cd890 con 0x7fcaa40cb430 2023-10-10T20:40:36.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.380+0000 7fcaaa6c1700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa40cda80 con 0x7fcaa40c9700 2023-10-10T20:40:36.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.380+0000 7fcaa96bf700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcaa40b0360 0x7fcaa40cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39072/0 (socket says 172.21.15.89:39072) 2023-10-10T20:40:36.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.380+0000 7fcaa96bf700 1 -- 172.21.15.89:0/2258427647 learned_addr learned my addr 172.21.15.89:0/2258427647 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:36.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.380+0000 7fcaa9ec0700 1 --2- 172.21.15.89:0/2258427647 >> v2:172.21.15.89:3300/0 conn(0x7fcaa40cb430 0x7fcaa40ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:36.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.380+0000 7fcaa9ec0700 1 -- 172.21.15.89:0/2258427647 >> v2:172.21.15.111:3300/0 conn(0x7fcaa40c9700 msgr2=0x7fcaa40caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:36.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.380+0000 7fcaa8ebe700 1 --2- 172.21.15.89:0/2258427647 >> v2:172.21.15.111:3300/0 conn(0x7fcaa40c9700 0x7fcaa40caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:36.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.380+0000 7fcaa9ec0700 1 --2- 172.21.15.89:0/2258427647 >> v2:172.21.15.111:3300/0 conn(0x7fcaa40c9700 0x7fcaa40caef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:36.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.380+0000 7fcaa9ec0700 1 -- 172.21.15.89:0/2258427647 >> v1:172.21.15.89:6789/0 conn(0x7fcaa40b0360 legacy=0x7fcaa40cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:36.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.380+0000 7fcaa9ec0700 1 -- 172.21.15.89:0/2258427647 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa40cdd60 con 0x7fcaa40cb430 2023-10-10T20:40:36.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.380+0000 7fca9bfff700 1 -- 172.21.15.89:0/2258427647 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 11949856 0 0) 0x7fcaa40adf00 con 0x7fcaa40b0360 2023-10-10T20:40:36.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.381+0000 7fcaa9ec0700 1 --2- 172.21.15.89:0/2258427647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaa40cb430 0x7fcaa40ae390 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0098e0 tx=0x7fca9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c784976c85e86d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:36.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.381+0000 7fca9bfff700 1 -- 172.21.15.89:0/2258427647 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c009e80 con 0x7fcaa40cb430 2023-10-10T20:40:36.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.381+0000 7fca9bfff700 1 -- 172.21.15.89:0/2258427647 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c00b820 con 0x7fcaa40cb430 2023-10-10T20:40:36.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.381+0000 7fca9bfff700 1 -- 172.21.15.89:0/2258427647 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c00bba0 con 0x7fcaa40cb430 2023-10-10T20:40:36.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.381+0000 7fcaaa6c1700 1 -- 172.21.15.89:0/2258427647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaa40cb430 msgr2=0x7fcaa40ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:36.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.381+0000 7fcaaa6c1700 1 --2- 172.21.15.89:0/2258427647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaa40cb430 0x7fcaa40ae390 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0098e0 tx=0x7fca9c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.381+0000 7fcaaa6c1700 1 -- 172.21.15.89:0/2258427647 shutdown_connections 2023-10-10T20:40:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.381+0000 7fcaaa6c1700 1 --2- 172.21.15.89:0/2258427647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaa40cb430 0x7fcaa40ae390 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.381+0000 7fcaaa6c1700 1 --2- 172.21.15.89:0/2258427647 >> v2:172.21.15.111:3300/0 conn(0x7fcaa40c9700 0x7fcaa40caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.381+0000 7fcaaa6c1700 1 -- 172.21.15.89:0/2258427647 >> 172.21.15.89:0/2258427647 conn(0x7fcaa406baa0 msgr2=0x7fcaa406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.381+0000 7fcaaa6c1700 1 -- 172.21.15.89:0/2258427647 shutdown_connections 2023-10-10T20:40:36.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.381+0000 7fcaaa6c1700 1 -- 172.21.15.89:0/2258427647 wait complete. 2023-10-10T20:40:36.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.382+0000 7fcaaa6c1700 1 Processor -- start 2023-10-10T20:40:36.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.382+0000 7fcaaa6c1700 1 -- start start 2023-10-10T20:40:36.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.383+0000 7fcaaa6c1700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaa40b0360 0x7fcaa415dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:36.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.383+0000 7fcaaa6c1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaa40c9700 0x7fcaa414d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:36.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.383+0000 7fcaaa6c1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaa40cb430 0x7fcaa4157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:36.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.383+0000 7fcaaa6c1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa4152140 con 0x7fcaa40cb430 2023-10-10T20:40:36.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.383+0000 7fcaaa6c1700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa41529e0 con 0x7fcaa40b0360 2023-10-10T20:40:36.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.383+0000 7fcaaa6c1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa415e170 con 0x7fcaa40c9700 2023-10-10T20:40:36.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.383+0000 7fcaa9ec0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaa40cb430 0x7fcaa4157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:36.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.383+0000 7fcaa9ec0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaa40cb430 0x7fcaa4157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34248/0 (socket says 172.21.15.89:34248) 2023-10-10T20:40:36.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.383+0000 7fcaa9ec0700 1 -- 172.21.15.89:0/3385225607 learned_addr learned my addr 172.21.15.89:0/3385225607 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:36.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.384+0000 7fcaa8ebe700 1 --2- 172.21.15.89:0/3385225607 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaa40c9700 0x7fcaa414d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:36.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.384+0000 7fcaa96bf700 1 --2- 172.21.15.89:0/3385225607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaa40b0360 0x7fcaa415dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:36.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.384+0000 7fcaa9ec0700 1 -- 172.21.15.89:0/3385225607 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaa40c9700 msgr2=0x7fcaa414d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:36.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.384+0000 7fcaa9ec0700 1 --2- 172.21.15.89:0/3385225607 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaa40c9700 0x7fcaa414d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:36.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.384+0000 7fcaa9ec0700 1 -- 172.21.15.89:0/3385225607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaa40b0360 msgr2=0x7fcaa415dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:36.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.384+0000 7fcaa9ec0700 1 --2- 172.21.15.89:0/3385225607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaa40b0360 0x7fcaa415dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:36.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.384+0000 7fcaa9ec0700 1 -- 172.21.15.89:0/3385225607 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c009420 con 0x7fcaa40cb430 2023-10-10T20:40:36.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.384+0000 7fcaa8ebe700 1 --2- 172.21.15.89:0/3385225607 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaa40c9700 0x7fcaa414d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:36.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.384+0000 7fcaa96bf700 1 --2- 172.21.15.89:0/3385225607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaa40b0360 0x7fcaa415dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:36.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.384+0000 7fcaa9ec0700 1 --2- 172.21.15.89:0/3385225607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaa40cb430 0x7fcaa4157df0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0093f0 tx=0x7fca9c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:36.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.384+0000 7fca997fa700 1 -- 172.21.15.89:0/3385225607 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c011630 con 0x7fcaa40cb430 2023-10-10T20:40:36.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.384+0000 7fca997fa700 1 -- 172.21.15.89:0/3385225607 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c011790 con 0x7fcaa40cb430 2023-10-10T20:40:36.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.384+0000 7fca997fa700 1 -- 172.21.15.89:0/3385225607 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c011b50 con 0x7fcaa40cb430 2023-10-10T20:40:36.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.384+0000 7fcaaa6c1700 1 -- 172.21.15.89:0/3385225607 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa40c8ee0 con 0x7fcaa40cb430 2023-10-10T20:40:36.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.384+0000 7fcaaa6c1700 1 -- 172.21.15.89:0/3385225607 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa4152670 con 0x7fcaa40cb430 2023-10-10T20:40:36.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.385+0000 7fcaaa6c1700 1 -- 172.21.15.89:0/3385225607 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa4152b50 con 0x7fcaa40cb430 2023-10-10T20:40:36.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.386+0000 7fca997fa700 1 -- 172.21.15.89:0/3385225607 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fca9c0233f0 con 0x7fcaa40cb430 2023-10-10T20:40:36.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.386+0000 7fca997fa700 1 --2- 172.21.15.89:0/3385225607 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fca88067290 0x7fca88069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:36.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.386+0000 7fca997fa700 1 -- 172.21.15.89:0/3385225607 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fca9c085f70 con 0x7fcaa40cb430 2023-10-10T20:40:36.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.386+0000 7fcaa96bf700 1 --2- 172.21.15.89:0/3385225607 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fca88067290 0x7fca88069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:36.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.387+0000 7fcaa96bf700 1 --2- 172.21.15.89:0/3385225607 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fca88067290 0x7fca88069740 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000d270 tx=0x7fcaa0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:36.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.391+0000 7fca997fa700 1 -- 172.21.15.89:0/3385225607 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca9c053030 con 0x7fcaa40cb430 2023-10-10T20:40:36.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:36.573+0000 7fcaaa6c1700 1 -- 172.21.15.89:0/3385225607 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcaa404c3c0 con 0x7fcaa40cb430 2023-10-10T20:40:37.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.247+0000 7fca997fa700 1 -- 172.21.15.89:0/3385225607 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v98) v1 ==== 170+0+0 (secure 0 0 0) 0x7fca9c056650 con 0x7fcaa40cb430 2023-10-10T20:40:37.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.250+0000 7fcaaa6c1700 1 -- 172.21.15.89:0/3385225607 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fca88067290 msgr2=0x7fca88069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:37.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.250+0000 7fcaaa6c1700 1 --2- 172.21.15.89:0/3385225607 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fca88067290 0x7fca88069740 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000d270 tx=0x7fcaa0009040 comp rx=0 tx=0).stop 2023-10-10T20:40:37.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.250+0000 7fcaaa6c1700 1 -- 172.21.15.89:0/3385225607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaa40cb430 msgr2=0x7fcaa4157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:37.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.250+0000 7fcaaa6c1700 1 --2- 172.21.15.89:0/3385225607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaa40cb430 0x7fcaa4157df0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0093f0 tx=0x7fca9c017000 comp rx=0 tx=0).stop 2023-10-10T20:40:37.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.250+0000 7fcaaa6c1700 1 -- 172.21.15.89:0/3385225607 shutdown_connections 2023-10-10T20:40:37.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.250+0000 7fcaaa6c1700 1 --2- 172.21.15.89:0/3385225607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaa40cb430 0x7fcaa4157df0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:37.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.250+0000 7fcaaa6c1700 1 --2- 172.21.15.89:0/3385225607 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fca88067290 0x7fca88069740 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:37.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.250+0000 7fcaaa6c1700 1 --2- 172.21.15.89:0/3385225607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaa40b0360 0x7fcaa415dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:37.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.250+0000 7fcaaa6c1700 1 --2- 172.21.15.89:0/3385225607 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaa40c9700 0x7fcaa414d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:37.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.250+0000 7fcaaa6c1700 1 -- 172.21.15.89:0/3385225607 >> 172.21.15.89:0/3385225607 conn(0x7fcaa406baa0 msgr2=0x7fcaa40c4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:37.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.251+0000 7fcaaa6c1700 1 -- 172.21.15.89:0/3385225607 shutdown_connections 2023-10-10T20:40:37.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.251+0000 7fcaaa6c1700 1 -- 172.21.15.89:0/3385225607 wait complete. 2023-10-10T20:40:37.260 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2023-10-10T20:40:37.275 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:40:37.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.392+0000 7fce705e6700 1 Processor -- start 2023-10-10T20:40:37.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.392+0000 7fce705e6700 1 -- start start 2023-10-10T20:40:37.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.393+0000 7fce705e6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fce68159c60 0x7fce6815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:37.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.393+0000 7fce705e6700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fce6815db10 0x7fce68069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:37.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.393+0000 7fce705e6700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6815a160 con 0x7fce68159890 2023-10-10T20:40:37.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.393+0000 7fce705e6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce6815df50 con 0x7fce68159c60 2023-10-10T20:40:37.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.393+0000 7fce705e6700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce680d13a0 con 0x7fce6815db10 2023-10-10T20:40:37.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.393+0000 7fce6db81700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fce68159c60 0x7fce6815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:37.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.393+0000 7fce6db81700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fce68159c60 0x7fce6815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34256/0 (socket says 172.21.15.89:34256) 2023-10-10T20:40:37.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.393+0000 7fce6db81700 1 -- 172.21.15.89:0/2091407107 learned_addr learned my addr 172.21.15.89:0/2091407107 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:37.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.394+0000 7fce6eb83700 1 --2- 172.21.15.89:0/2091407107 >> v2:172.21.15.111:3300/0 conn(0x7fce6815db10 0x7fce68069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:37.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.394+0000 7fce6db81700 1 -- 172.21.15.89:0/2091407107 >> v2:172.21.15.111:3300/0 conn(0x7fce6815db10 msgr2=0x7fce68069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:37.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.394+0000 7fce6db81700 1 --2- 172.21.15.89:0/2091407107 >> v2:172.21.15.111:3300/0 conn(0x7fce6815db10 0x7fce68069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:37.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.394+0000 7fce6db81700 1 -- 172.21.15.89:0/2091407107 >> v1:172.21.15.111:6789/0 conn(0x7fce68159890 legacy=0x7fce6815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:37.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.394+0000 7fce6db81700 1 -- 172.21.15.89:0/2091407107 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6815e140 con 0x7fce68159c60 2023-10-10T20:40:37.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.394+0000 7fce6db81700 1 --2- 172.21.15.89:0/2091407107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce68159c60 0x7fce6815d5d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fce6806a890 tx=0x7fce58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a070762fee9392 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:37.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.394+0000 7fce6d380700 1 -- 172.21.15.89:0/2091407107 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5800b820 con 0x7fce68159c60 2023-10-10T20:40:37.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.394+0000 7fce6d380700 1 -- 172.21.15.89:0/2091407107 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5800b980 con 0x7fce68159c60 2023-10-10T20:40:37.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.394+0000 7fce6d380700 1 -- 172.21.15.89:0/2091407107 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5800bd20 con 0x7fce68159c60 2023-10-10T20:40:37.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.394+0000 7fce705e6700 1 -- 172.21.15.89:0/2091407107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce68159c60 msgr2=0x7fce6815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:37.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.394+0000 7fce705e6700 1 --2- 172.21.15.89:0/2091407107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce68159c60 0x7fce6815d5d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fce6806a890 tx=0x7fce58009250 comp rx=0 tx=0).stop 2023-10-10T20:40:37.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.394+0000 7fce705e6700 1 -- 172.21.15.89:0/2091407107 shutdown_connections 2023-10-10T20:40:37.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.394+0000 7fce705e6700 1 --2- 172.21.15.89:0/2091407107 >> v2:172.21.15.111:3300/0 conn(0x7fce6815db10 0x7fce68069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:37.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.394+0000 7fce705e6700 1 --2- 172.21.15.89:0/2091407107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce68159c60 0x7fce6815d5d0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:37.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.394+0000 7fce705e6700 1 -- 172.21.15.89:0/2091407107 >> 172.21.15.89:0/2091407107 conn(0x7fce68068e20 msgr2=0x7fce680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:37.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.394+0000 7fce705e6700 1 -- 172.21.15.89:0/2091407107 shutdown_connections 2023-10-10T20:40:37.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.395+0000 7fce705e6700 1 -- 172.21.15.89:0/2091407107 wait complete. 2023-10-10T20:40:37.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.396+0000 7fce705e6700 1 Processor -- start 2023-10-10T20:40:37.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.396+0000 7fce705e6700 1 -- start start 2023-10-10T20:40:37.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.396+0000 7fce705e6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fce68159890 0x7fce680cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:37.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.396+0000 7fce705e6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce68159c60 0x7fce680b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:37.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.396+0000 7fce705e6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce6815db10 0x7fce680c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:37.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.396+0000 7fce705e6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6816ad90 con 0x7fce6815db10 2023-10-10T20:40:37.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.396+0000 7fce705e6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6816a180 con 0x7fce68159c60 2023-10-10T20:40:37.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.396+0000 7fce705e6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6816a460 con 0x7fce68159890 2023-10-10T20:40:37.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.396+0000 7fce6eb83700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce6815db10 0x7fce680c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:37.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.396+0000 7fce6eb83700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce6815db10 0x7fce680c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34260/0 (socket says 172.21.15.89:34260) 2023-10-10T20:40:37.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.396+0000 7fce6eb83700 1 -- 172.21.15.89:0/745698281 learned_addr learned my addr 172.21.15.89:0/745698281 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:37.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.397+0000 7fce6e382700 1 --2- 172.21.15.89:0/745698281 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fce68159890 0x7fce680cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:37.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.397+0000 7fce6eb83700 1 -- 172.21.15.89:0/745698281 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fce68159890 msgr2=0x7fce680cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:37.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.397+0000 7fce6db81700 1 --2- 172.21.15.89:0/745698281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce68159c60 0x7fce680b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:37.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.397+0000 7fce6eb83700 1 --2- 172.21.15.89:0/745698281 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fce68159890 0x7fce680cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:37.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.397+0000 7fce6eb83700 1 -- 172.21.15.89:0/745698281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce68159c60 msgr2=0x7fce680b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:37.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.397+0000 7fce6eb83700 1 --2- 172.21.15.89:0/745698281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce68159c60 0x7fce680b37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:37.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.397+0000 7fce6eb83700 1 -- 172.21.15.89:0/745698281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce580094f0 con 0x7fce6815db10 2023-10-10T20:40:37.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.397+0000 7fce6eb83700 1 --2- 172.21.15.89:0/745698281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce6815db10 0x7fce680c9d20 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fce6000cdb0 tx=0x7fce600077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:37.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.397+0000 7fce5e7fc700 1 -- 172.21.15.89:0/745698281 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6000aa80 con 0x7fce6815db10 2023-10-10T20:40:37.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.397+0000 7fce5e7fc700 1 -- 172.21.15.89:0/745698281 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6000abe0 con 0x7fce6815db10 2023-10-10T20:40:37.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.397+0000 7fce705e6700 1 -- 172.21.15.89:0/745698281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6816a740 con 0x7fce6815db10 2023-10-10T20:40:37.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.397+0000 7fce5e7fc700 1 -- 172.21.15.89:0/745698281 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60017430 con 0x7fce6815db10 2023-10-10T20:40:37.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.397+0000 7fce705e6700 1 -- 172.21.15.89:0/745698281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce68169e90 con 0x7fce6815db10 2023-10-10T20:40:37.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.398+0000 7fce705e6700 1 -- 172.21.15.89:0/745698281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce680b0540 con 0x7fce6815db10 2023-10-10T20:40:37.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.398+0000 7fce5e7fc700 1 -- 172.21.15.89:0/745698281 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fce60003020 con 0x7fce6815db10 2023-10-10T20:40:37.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.399+0000 7fce5e7fc700 1 --2- 172.21.15.89:0/745698281 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fce4c0672c0 0x7fce4c069770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:37.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.399+0000 7fce5e7fc700 1 -- 172.21.15.89:0/745698281 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fce60013070 con 0x7fce6815db10 2023-10-10T20:40:37.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.399+0000 7fce6e382700 1 --2- 172.21.15.89:0/745698281 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fce4c0672c0 0x7fce4c069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:37.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.399+0000 7fce6e382700 1 --2- 172.21.15.89:0/745698281 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fce4c0672c0 0x7fce4c069770 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fce6400a8c0 tx=0x7fce64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:37.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.403+0000 7fce5e7fc700 1 -- 172.21.15.89:0/745698281 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce680b0540 con 0x7fce6815db10 2023-10-10T20:40:37.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:37.629+0000 7fce705e6700 1 -- 172.21.15.89:0/745698281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fce680b0540 con 0x7fce6815db10 2023-10-10T20:40:38.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.265+0000 7fce5e7fc700 1 -- 172.21.15.89:0/745698281 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v60) v1 ==== 174+0+0 (secure 0 0 0) 0x7fce680b0540 con 0x7fce6815db10 2023-10-10T20:40:38.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.273+0000 7fce705e6700 1 -- 172.21.15.89:0/745698281 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fce4c0672c0 msgr2=0x7fce4c069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:38.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.273+0000 7fce705e6700 1 --2- 172.21.15.89:0/745698281 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fce4c0672c0 0x7fce4c069770 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fce6400a8c0 tx=0x7fce64008040 comp rx=0 tx=0).stop 2023-10-10T20:40:38.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.273+0000 7fce705e6700 1 -- 172.21.15.89:0/745698281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce6815db10 msgr2=0x7fce680c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:38.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.273+0000 7fce705e6700 1 --2- 172.21.15.89:0/745698281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce6815db10 0x7fce680c9d20 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fce6000cdb0 tx=0x7fce600077d0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.274+0000 7fce705e6700 1 -- 172.21.15.89:0/745698281 shutdown_connections 2023-10-10T20:40:38.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.274+0000 7fce705e6700 1 --2- 172.21.15.89:0/745698281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce6815db10 0x7fce680c9d20 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.274+0000 7fce705e6700 1 --2- 172.21.15.89:0/745698281 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fce68159890 0x7fce680cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.274+0000 7fce705e6700 1 --2- 172.21.15.89:0/745698281 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fce4c0672c0 0x7fce4c069770 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.274+0000 7fce705e6700 1 --2- 172.21.15.89:0/745698281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce68159c60 0x7fce680b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.274+0000 7fce705e6700 1 -- 172.21.15.89:0/745698281 >> 172.21.15.89:0/745698281 conn(0x7fce68068e20 msgr2=0x7fce68153200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:38.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.275+0000 7fce705e6700 1 -- 172.21.15.89:0/745698281 shutdown_connections 2023-10-10T20:40:38.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.275+0000 7fce705e6700 1 -- 172.21.15.89:0/745698281 wait complete. 2023-10-10T20:40:38.281 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 16 and data pool 17 2023-10-10T20:40:38.292 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:40:38.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.406+0000 7f5cbd875700 1 Processor -- start 2023-10-10T20:40:38.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.406+0000 7f5cbd875700 1 -- start start 2023-10-10T20:40:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.407+0000 7f5cbd875700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5cb806c0a0 0x7f5cb80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.407+0000 7f5cbd875700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5cb815df50 0x7f5cb815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.407+0000 7f5cbd875700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb815d7e0 con 0x7f5cb806bcd0 2023-10-10T20:40:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.407+0000 7f5cbd875700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cb815dac0 con 0x7f5cb806c0a0 2023-10-10T20:40:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.407+0000 7f5cbd875700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb815aa30 con 0x7f5cb815df50 2023-10-10T20:40:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.407+0000 7f5cb67fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5cb806c0a0 0x7f5cb80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.407+0000 7f5cb6ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5cb806bcd0 0x7f5cb815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49152/0 (socket says 172.21.15.89:49152) 2023-10-10T20:40:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.407+0000 7f5cb6ffd700 1 -- 172.21.15.89:0/2145535579 learned_addr learned my addr 172.21.15.89:0/2145535579 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.407+0000 7f5cb77fe700 1 --2- 172.21.15.89:0/2145535579 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5cb815df50 0x7f5cb815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.407+0000 7f5cb67fc700 1 -- 172.21.15.89:0/2145535579 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5cb815df50 msgr2=0x7f5cb815e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.408+0000 7f5cb67fc700 1 --2- 172.21.15.89:0/2145535579 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5cb815df50 0x7f5cb815e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.408+0000 7f5cb67fc700 1 -- 172.21.15.89:0/2145535579 >> v1:172.21.15.89:6789/0 conn(0x7f5cb806bcd0 legacy=0x7f5cb815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.408+0000 7f5cb67fc700 1 -- 172.21.15.89:0/2145535579 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb815ad10 con 0x7f5cb806c0a0 2023-10-10T20:40:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.408+0000 7f5cb5ffb700 1 -- 172.21.15.89:0/2145535579 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790616294 0 0) 0x7f5cb815d7e0 con 0x7f5cb806bcd0 2023-10-10T20:40:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.408+0000 7f5cb77fe700 1 --2- 172.21.15.89:0/2145535579 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5cb815df50 0x7f5cb815e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.408+0000 7f5cb67fc700 1 --2- 172.21.15.89:0/2145535579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cb806c0a0 0x7f5cb80caa90 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0098e0 tx=0x7f5cac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d87adc4d74bd76 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.408+0000 7f5cb5ffb700 1 -- 172.21.15.89:0/2145535579 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac009e80 con 0x7f5cb806c0a0 2023-10-10T20:40:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.408+0000 7f5cb5ffb700 1 -- 172.21.15.89:0/2145535579 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac00b860 con 0x7f5cb806c0a0 2023-10-10T20:40:38.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.408+0000 7f5cb5ffb700 1 -- 172.21.15.89:0/2145535579 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac00bbe0 con 0x7f5cb806c0a0 2023-10-10T20:40:38.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.408+0000 7f5cbd875700 1 -- 172.21.15.89:0/2145535579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cb806c0a0 msgr2=0x7f5cb80caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:38.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.408+0000 7f5cbd875700 1 --2- 172.21.15.89:0/2145535579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cb806c0a0 0x7f5cb80caa90 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0098e0 tx=0x7f5cac0091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.408+0000 7f5cbd875700 1 -- 172.21.15.89:0/2145535579 shutdown_connections 2023-10-10T20:40:38.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.408+0000 7f5cbd875700 1 --2- 172.21.15.89:0/2145535579 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5cb815df50 0x7f5cb815e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.408+0000 7f5cbd875700 1 --2- 172.21.15.89:0/2145535579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cb806c0a0 0x7f5cb80caa90 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.408+0000 7f5cbd875700 1 -- 172.21.15.89:0/2145535579 >> 172.21.15.89:0/2145535579 conn(0x7f5cb80ce600 msgr2=0x7f5cb80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:38.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.408+0000 7f5cbd875700 1 -- 172.21.15.89:0/2145535579 shutdown_connections 2023-10-10T20:40:38.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.408+0000 7f5cbd875700 1 -- 172.21.15.89:0/2145535579 wait complete. 2023-10-10T20:40:38.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.410+0000 7f5cbd875700 1 Processor -- start 2023-10-10T20:40:38.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.410+0000 7f5cbd875700 1 -- start start 2023-10-10T20:40:38.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.410+0000 7f5cbd875700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5cb806bcd0 0x7f5cb816ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:38.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.410+0000 7f5cbd875700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cb806c0a0 0x7f5cb81568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:38.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.410+0000 7f5cbd875700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cb815df50 0x7f5cb80b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:38.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.410+0000 7f5cbd875700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb816b070 con 0x7f5cb815df50 2023-10-10T20:40:38.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.410+0000 7f5cbd875700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb8156de0 con 0x7f5cb806c0a0 2023-10-10T20:40:38.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.410+0000 7f5cbd875700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb80c5da0 con 0x7f5cb806bcd0 2023-10-10T20:40:38.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.411+0000 7f5cb77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cb815df50 0x7f5cb80b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:38.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.411+0000 7f5cb77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cb815df50 0x7f5cb80b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38878/0 (socket says 172.21.15.89:38878) 2023-10-10T20:40:38.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.411+0000 7f5cb77fe700 1 -- 172.21.15.89:0/4069801374 learned_addr learned my addr 172.21.15.89:0/4069801374 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:38.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.411+0000 7f5cb6ffd700 1 --2- 172.21.15.89:0/4069801374 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5cb806bcd0 0x7f5cb816ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:38.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.411+0000 7f5cb67fc700 1 --2- 172.21.15.89:0/4069801374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cb806c0a0 0x7f5cb81568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:38.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.411+0000 7f5cb77fe700 1 -- 172.21.15.89:0/4069801374 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5cb806bcd0 msgr2=0x7f5cb816ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:38.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.411+0000 7f5cb77fe700 1 --2- 172.21.15.89:0/4069801374 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5cb806bcd0 0x7f5cb816ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.411+0000 7f5cb77fe700 1 -- 172.21.15.89:0/4069801374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cb806c0a0 msgr2=0x7f5cb81568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:38.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.411+0000 7f5cb77fe700 1 --2- 172.21.15.89:0/4069801374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cb806c0a0 0x7f5cb81568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.411+0000 7f5cb77fe700 1 -- 172.21.15.89:0/4069801374 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac009420 con 0x7f5cb815df50 2023-10-10T20:40:38.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.411+0000 7f5cb77fe700 1 --2- 172.21.15.89:0/4069801374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cb815df50 0x7f5cb80b13a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000a380 tx=0x7f5cb0007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:38.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.411+0000 7f5c9effd700 1 -- 172.21.15.89:0/4069801374 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb000aea0 con 0x7f5cb815df50 2023-10-10T20:40:38.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.411+0000 7f5c9effd700 1 -- 172.21.15.89:0/4069801374 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb000d650 con 0x7f5cb815df50 2023-10-10T20:40:38.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.411+0000 7f5cbd875700 1 -- 172.21.15.89:0/4069801374 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb80b3080 con 0x7f5cb815df50 2023-10-10T20:40:38.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.411+0000 7f5c9effd700 1 -- 172.21.15.89:0/4069801374 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb000da10 con 0x7f5cb815df50 2023-10-10T20:40:38.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.411+0000 7f5cbd875700 1 -- 172.21.15.89:0/4069801374 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb80b3640 con 0x7f5cb815df50 2023-10-10T20:40:38.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.412+0000 7f5cbd875700 1 -- 172.21.15.89:0/4069801374 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb804c3c0 con 0x7f5cb815df50 2023-10-10T20:40:38.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.413+0000 7f5c9effd700 1 -- 172.21.15.89:0/4069801374 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f5cb001e3b0 con 0x7f5cb815df50 2023-10-10T20:40:38.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.413+0000 7f5c9effd700 1 --2- 172.21.15.89:0/4069801374 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5ca4067290 0x7f5ca4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:38.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.413+0000 7f5c9effd700 1 -- 172.21.15.89:0/4069801374 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5cb0085540 con 0x7f5cb815df50 2023-10-10T20:40:38.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.413+0000 7f5cb6ffd700 1 --2- 172.21.15.89:0/4069801374 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5ca4067290 0x7f5ca4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:38.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.414+0000 7f5cb6ffd700 1 --2- 172.21.15.89:0/4069801374 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5ca4067290 0x7f5ca4069740 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5cb806dc60 tx=0x7f5ca8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:38.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.417+0000 7f5c9effd700 1 -- 172.21.15.89:0/4069801374 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5cb00523a0 con 0x7f5cb815df50 2023-10-10T20:40:38.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.602+0000 7f5cbd875700 1 -- 172.21.15.89:0/4069801374 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cb80c5f10 con 0x7f5cb815df50 2023-10-10T20:40:38.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.603+0000 7f5c9effd700 1 -- 172.21.15.89:0/4069801374 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+15366 (secure 0 0 0) 0x7f5cb001a090 con 0x7f5cb815df50 2023-10-10T20:40:38.606 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:38.606 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":99,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:40:37.630528+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:40:36.002449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-10T20:40:36.591960+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:40:38.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.605+0000 7f5cbd875700 1 -- 172.21.15.89:0/4069801374 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5ca4067290 msgr2=0x7f5ca4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:38.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.605+0000 7f5cbd875700 1 --2- 172.21.15.89:0/4069801374 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5ca4067290 0x7f5ca4069740 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5cb806dc60 tx=0x7f5ca8009040 comp rx=0 tx=0).stop 2023-10-10T20:40:38.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.605+0000 7f5cbd875700 1 -- 172.21.15.89:0/4069801374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cb815df50 msgr2=0x7f5cb80b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:38.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.605+0000 7f5cbd875700 1 --2- 172.21.15.89:0/4069801374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cb815df50 0x7f5cb80b13a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000a380 tx=0x7f5cb0007900 comp rx=0 tx=0).stop 2023-10-10T20:40:38.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.605+0000 7f5cbd875700 1 -- 172.21.15.89:0/4069801374 shutdown_connections 2023-10-10T20:40:38.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.605+0000 7f5cbd875700 1 --2- 172.21.15.89:0/4069801374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cb815df50 0x7f5cb80b13a0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.605+0000 7f5cbd875700 1 --2- 172.21.15.89:0/4069801374 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5cb806bcd0 0x7f5cb816ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.605+0000 7f5cbd875700 1 --2- 172.21.15.89:0/4069801374 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5ca4067290 0x7f5ca4069740 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.605+0000 7f5cbd875700 1 --2- 172.21.15.89:0/4069801374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cb806c0a0 0x7f5cb81568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.605+0000 7f5cbd875700 1 -- 172.21.15.89:0/4069801374 >> 172.21.15.89:0/4069801374 conn(0x7f5cb80ce600 msgr2=0x7f5cb815bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:38.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.605+0000 7f5cbd875700 1 -- 172.21.15.89:0/4069801374 shutdown_connections 2023-10-10T20:40:38.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.605+0000 7f5cbd875700 1 -- 172.21.15.89:0/4069801374 wait complete. 2023-10-10T20:40:38.620 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:40:38.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.732+0000 7f7970973700 1 Processor -- start 2023-10-10T20:40:38.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.732+0000 7f7970973700 1 -- start start 2023-10-10T20:40:38.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.733+0000 7f7970973700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f796c069560 0x7f796c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:38.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.733+0000 7f7970973700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f796c069ee0 con 0x7f796c159c60 2023-10-10T20:40:38.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.733+0000 7f7970973700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f796c15a160 con 0x7f796c159890 2023-10-10T20:40:38.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.733+0000 7f7970973700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f796c0d13a0 con 0x7f796c069560 2023-10-10T20:40:38.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.733+0000 7f796ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f796c069560 0x7f796c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:38.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.733+0000 7f796ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f796c069560 0x7f796c0699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38892/0 (socket says 172.21.15.89:38892) 2023-10-10T20:40:38.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.733+0000 7f796ad9d700 1 -- 172.21.15.89:0/849534100 learned_addr learned my addr 172.21.15.89:0/849534100 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:38.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.733+0000 7f7961d9b700 1 --1- 172.21.15.89:0/849534100 >> v1:172.21.15.89:6789/0 conn(0x7f796c159c60 0x7f796c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49162/0 (socket says 172.21.15.89:49162) 2023-10-10T20:40:38.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.734+0000 7f796ad9d700 1 -- 172.21.15.89:0/849534100 >> v1:172.21.15.111:6789/0 conn(0x7f796c159890 legacy=0x7f796c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:38.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.734+0000 7f796ad9d700 1 -- 172.21.15.89:0/849534100 >> v1:172.21.15.89:6789/0 conn(0x7f796c159c60 legacy=0x7f796c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:38.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.734+0000 7f7969d9b700 1 -- 172.21.15.89:0/849534100 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1199711659 0 0) 0x7f796c069ee0 con 0x7f796c159c60 2023-10-10T20:40:38.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.734+0000 7f796ad9d700 1 -- 172.21.15.89:0/849534100 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c15e140 con 0x7f796c069560 2023-10-10T20:40:38.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.734+0000 7f796ad9d700 1 --2- 172.21.15.89:0/849534100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f796c069560 0x7f796c0699a0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f7964009f20 tx=0x7f7964009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0ade0828c60da5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:38.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.734+0000 7f7969d9b700 1 -- 172.21.15.89:0/849534100 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796400e070 con 0x7f796c069560 2023-10-10T20:40:38.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.734+0000 7f7969d9b700 1 -- 172.21.15.89:0/849534100 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796400ba40 con 0x7f796c069560 2023-10-10T20:40:38.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.734+0000 7f7969d9b700 1 -- 172.21.15.89:0/849534100 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79640092e0 con 0x7f796c069560 2023-10-10T20:40:38.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.734+0000 7f7970973700 1 -- 172.21.15.89:0/849534100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f796c069560 msgr2=0x7f796c0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:38.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.734+0000 7f7970973700 1 --2- 172.21.15.89:0/849534100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f796c069560 0x7f796c0699a0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f7964009f20 tx=0x7f7964009840 comp rx=0 tx=0).stop 2023-10-10T20:40:38.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.734+0000 7f7970973700 1 -- 172.21.15.89:0/849534100 shutdown_connections 2023-10-10T20:40:38.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.734+0000 7f7970973700 1 --2- 172.21.15.89:0/849534100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f796c069560 0x7f796c0699a0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.734+0000 7f7970973700 1 -- 172.21.15.89:0/849534100 >> 172.21.15.89:0/849534100 conn(0x7f796c068e20 msgr2=0x7f796c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:38.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.734+0000 7f7970973700 1 -- 172.21.15.89:0/849534100 shutdown_connections 2023-10-10T20:40:38.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.734+0000 7f7970973700 1 -- 172.21.15.89:0/849534100 wait complete. 2023-10-10T20:40:38.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.736+0000 7f7970973700 1 Processor -- start 2023-10-10T20:40:38.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.736+0000 7f7970973700 1 -- start start 2023-10-10T20:40:38.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.736+0000 7f7970973700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f796c069560 0x7f796c0ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:38.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.736+0000 7f7970973700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f796c159890 0x7f796c0b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:38.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.736+0000 7f7970973700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f796c159c60 0x7f796c157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:38.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.736+0000 7f7970973700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c0c5a10 con 0x7f796c159c60 2023-10-10T20:40:38.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.736+0000 7f7970973700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c0b6490 con 0x7f796c159890 2023-10-10T20:40:38.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.736+0000 7f7970973700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796c0b5ed0 con 0x7f796c069560 2023-10-10T20:40:38.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.737+0000 7f796ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f796c159c60 0x7f796c157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:38.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.737+0000 7f796ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f796c159c60 0x7f796c157460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38906/0 (socket says 172.21.15.89:38906) 2023-10-10T20:40:38.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.737+0000 7f796ad9d700 1 -- 172.21.15.89:0/3940540017 learned_addr learned my addr 172.21.15.89:0/3940540017 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:38.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.737+0000 7f7961d9b700 1 --2- 172.21.15.89:0/3940540017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f796c159890 0x7f796c0b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:38.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.737+0000 7f796a59c700 1 --2- 172.21.15.89:0/3940540017 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f796c069560 0x7f796c0ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:38.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.737+0000 7f796ad9d700 1 -- 172.21.15.89:0/3940540017 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f796c069560 msgr2=0x7f796c0ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:38.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.737+0000 7f796ad9d700 1 --2- 172.21.15.89:0/3940540017 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f796c069560 0x7f796c0ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.737+0000 7f796ad9d700 1 -- 172.21.15.89:0/3940540017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f796c159890 msgr2=0x7f796c0b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:38.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.737+0000 7f796ad9d700 1 --2- 172.21.15.89:0/3940540017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f796c159890 0x7f796c0b5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.737+0000 7f796ad9d700 1 -- 172.21.15.89:0/3940540017 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7964009ae0 con 0x7f796c159c60 2023-10-10T20:40:38.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.737+0000 7f796a59c700 1 --2- 172.21.15.89:0/3940540017 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f796c069560 0x7f796c0ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:38.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.737+0000 7f796ad9d700 1 --2- 172.21.15.89:0/3940540017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f796c159c60 0x7f796c157460 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f796c06a070 tx=0x7f7964017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:38.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.737+0000 7f7962ffd700 1 -- 172.21.15.89:0/3940540017 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7964012ea0 con 0x7f796c159c60 2023-10-10T20:40:38.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.737+0000 7f7962ffd700 1 -- 172.21.15.89:0/3940540017 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796400be50 con 0x7f796c159c60 2023-10-10T20:40:38.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.737+0000 7f7962ffd700 1 -- 172.21.15.89:0/3940540017 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796401e660 con 0x7f796c159c60 2023-10-10T20:40:38.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.737+0000 7f7970973700 1 -- 172.21.15.89:0/3940540017 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796c0b61b0 con 0x7f796c159c60 2023-10-10T20:40:38.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.737+0000 7f7970973700 1 -- 172.21.15.89:0/3940540017 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796c0b5ba0 con 0x7f796c159c60 2023-10-10T20:40:38.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.738+0000 7f7970973700 1 -- 172.21.15.89:0/3940540017 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f796c0c5b80 con 0x7f796c159c60 2023-10-10T20:40:38.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.738+0000 7f7962ffd700 1 -- 172.21.15.89:0/3940540017 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f796401ea90 con 0x7f796c159c60 2023-10-10T20:40:38.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.739+0000 7f7962ffd700 1 --2- 172.21.15.89:0/3940540017 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f794c067290 0x7f794c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:38.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.739+0000 7f7962ffd700 1 -- 172.21.15.89:0/3940540017 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f79640869a0 con 0x7f796c159c60 2023-10-10T20:40:38.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.739+0000 7f796a59c700 1 --2- 172.21.15.89:0/3940540017 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f794c067290 0x7f794c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:38.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.739+0000 7f796a59c700 1 --2- 172.21.15.89:0/3940540017 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f794c067290 0x7f794c069740 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f795c002880 tx=0x7f795c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:38.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.743+0000 7f7962ffd700 1 -- 172.21.15.89:0/3940540017 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79640536f0 con 0x7f796c159c60 2023-10-10T20:40:38.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.927+0000 7f7970973700 1 -- 172.21.15.89:0/3940540017 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f796c04c3c0 con 0x7f796c159c60 2023-10-10T20:40:38.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.928+0000 7f7962ffd700 1 -- 172.21.15.89:0/3940540017 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+15366 (secure 0 0 0) 0x7f7964056d10 con 0x7f796c159c60 2023-10-10T20:40:38.930 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:38.930 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":99,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:40:37.630528+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:40:36.002449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-10T20:40:36.591960+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:40:38.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.930+0000 7f7970973700 1 -- 172.21.15.89:0/3940540017 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f794c067290 msgr2=0x7f794c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:38.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.930+0000 7f7970973700 1 --2- 172.21.15.89:0/3940540017 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f794c067290 0x7f794c069740 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f795c002880 tx=0x7f795c009040 comp rx=0 tx=0).stop 2023-10-10T20:40:38.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.930+0000 7f7970973700 1 -- 172.21.15.89:0/3940540017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f796c159c60 msgr2=0x7f796c157460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:38.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.930+0000 7f7970973700 1 --2- 172.21.15.89:0/3940540017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f796c159c60 0x7f796c157460 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f796c06a070 tx=0x7f7964017000 comp rx=0 tx=0).stop 2023-10-10T20:40:38.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.930+0000 7f7970973700 1 -- 172.21.15.89:0/3940540017 shutdown_connections 2023-10-10T20:40:38.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.930+0000 7f7970973700 1 --2- 172.21.15.89:0/3940540017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f796c159c60 0x7f796c157460 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.930+0000 7f7970973700 1 --2- 172.21.15.89:0/3940540017 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f796c069560 0x7f796c0ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.930+0000 7f7970973700 1 --2- 172.21.15.89:0/3940540017 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f794c067290 0x7f794c069740 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.930+0000 7f7970973700 1 --2- 172.21.15.89:0/3940540017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f796c159890 0x7f796c0b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:38.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.930+0000 7f7970973700 1 -- 172.21.15.89:0/3940540017 >> 172.21.15.89:0/3940540017 conn(0x7f796c068e20 msgr2=0x7f796c152b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:38.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.930+0000 7f7970973700 1 -- 172.21.15.89:0/3940540017 shutdown_connections 2023-10-10T20:40:38.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:38.930+0000 7f7970973700 1 -- 172.21.15.89:0/3940540017 wait complete. 2023-10-10T20:40:38.945 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:40:39.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.061+0000 7f4991214700 1 Processor -- start 2023-10-10T20:40:39.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.061+0000 7f4991214700 1 -- start start 2023-10-10T20:40:39.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.062+0000 7f4991214700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f498c159890 0x7f498c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:39.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.062+0000 7f4991214700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f498c069560 0x7f498c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:39.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.062+0000 7f4991214700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498c15a160 con 0x7f498c159c60 2023-10-10T20:40:39.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.062+0000 7f4991214700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498c069f20 con 0x7f498c069560 2023-10-10T20:40:39.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.062+0000 7f4991214700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c0d13a0 con 0x7f498c159890 2023-10-10T20:40:39.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.062+0000 7f498b59e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f498c069560 0x7f498c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:39.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.062+0000 7f498b59e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f498c069560 0x7f498c0699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38916/0 (socket says 172.21.15.89:38916) 2023-10-10T20:40:39.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.062+0000 7f498b59e700 1 -- 172.21.15.89:0/2669881527 learned_addr learned my addr 172.21.15.89:0/2669881527 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:39.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.062+0000 7f498ad9d700 1 --2- 172.21.15.89:0/2669881527 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f498c159890 0x7f498c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:39.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.062+0000 7f498b59e700 1 -- 172.21.15.89:0/2669881527 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f498c159890 msgr2=0x7f498c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:39.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.062+0000 7f498b59e700 1 --2- 172.21.15.89:0/2669881527 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f498c159890 0x7f498c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.062+0000 7f498b59e700 1 -- 172.21.15.89:0/2669881527 >> v1:172.21.15.111:6789/0 conn(0x7f498c159c60 legacy=0x7f498c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.062+0000 7f498b59e700 1 -- 172.21.15.89:0/2669881527 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c15e140 con 0x7f498c069560 2023-10-10T20:40:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.062+0000 7f498ad9d700 1 --2- 172.21.15.89:0/2669881527 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f498c159890 0x7f498c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.062+0000 7f498b59e700 1 --2- 172.21.15.89:0/2669881527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f498c069560 0x7f498c0699e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f4984009f20 tx=0x7f4984009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd440e2a88eea71d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.062+0000 7f498a59c700 1 -- 172.21.15.89:0/2669881527 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498400e070 con 0x7f498c069560 2023-10-10T20:40:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.062+0000 7f498a59c700 1 -- 172.21.15.89:0/2669881527 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498400ba80 con 0x7f498c069560 2023-10-10T20:40:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.063+0000 7f498a59c700 1 -- 172.21.15.89:0/2669881527 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49840092e0 con 0x7f498c069560 2023-10-10T20:40:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.063+0000 7f4991214700 1 -- 172.21.15.89:0/2669881527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f498c069560 msgr2=0x7f498c0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.063+0000 7f4991214700 1 --2- 172.21.15.89:0/2669881527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f498c069560 0x7f498c0699e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f4984009f20 tx=0x7f4984009840 comp rx=0 tx=0).stop 2023-10-10T20:40:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.063+0000 7f4991214700 1 -- 172.21.15.89:0/2669881527 shutdown_connections 2023-10-10T20:40:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.063+0000 7f4991214700 1 --2- 172.21.15.89:0/2669881527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f498c069560 0x7f498c0699e0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:39.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.063+0000 7f4991214700 1 --2- 172.21.15.89:0/2669881527 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f498c159890 0x7f498c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:39.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.063+0000 7f4991214700 1 -- 172.21.15.89:0/2669881527 >> 172.21.15.89:0/2669881527 conn(0x7f498c068e20 msgr2=0x7f498c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:39.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.063+0000 7f4991214700 1 -- 172.21.15.89:0/2669881527 shutdown_connections 2023-10-10T20:40:39.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.063+0000 7f4991214700 1 -- 172.21.15.89:0/2669881527 wait complete. 2023-10-10T20:40:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.064+0000 7f4991214700 1 Processor -- start 2023-10-10T20:40:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.064+0000 7f4991214700 1 -- start start 2023-10-10T20:40:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.065+0000 7f4991214700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f498c069560 0x7f498c149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.065+0000 7f4991214700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f498c159890 0x7f498c15e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.065+0000 7f4991214700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f498c159c60 0x7f498c15eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.065+0000 7f4991214700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c0b5de0 con 0x7f498c159c60 2023-10-10T20:40:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.065+0000 7f4991214700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c169a30 con 0x7f498c069560 2023-10-10T20:40:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.065+0000 7f4991214700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c16a5a0 con 0x7f498c159890 2023-10-10T20:40:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.065+0000 7f498b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f498c159c60 0x7f498c15eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.065+0000 7f498b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f498c159c60 0x7f498c15eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38930/0 (socket says 172.21.15.89:38930) 2023-10-10T20:40:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.065+0000 7f498b59e700 1 -- 172.21.15.89:0/4027281498 learned_addr learned my addr 172.21.15.89:0/4027281498 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.065+0000 7f498ad9d700 1 --2- 172.21.15.89:0/4027281498 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f498c069560 0x7f498c149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.065+0000 7f4983fff700 1 --2- 172.21.15.89:0/4027281498 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f498c159890 0x7f498c15e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.065+0000 7f498b59e700 1 -- 172.21.15.89:0/4027281498 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f498c159890 msgr2=0x7f498c15e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.065+0000 7f498b59e700 1 --2- 172.21.15.89:0/4027281498 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f498c159890 0x7f498c15e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.065+0000 7f498b59e700 1 -- 172.21.15.89:0/4027281498 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f498c069560 msgr2=0x7f498c149170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.065+0000 7f498b59e700 1 --2- 172.21.15.89:0/4027281498 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f498c069560 0x7f498c149170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.065+0000 7f498b59e700 1 -- 172.21.15.89:0/4027281498 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4984009ae0 con 0x7f498c159c60 2023-10-10T20:40:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.065+0000 7f4983fff700 1 --2- 172.21.15.89:0/4027281498 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f498c159890 0x7f498c15e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.065+0000 7f498b59e700 1 --2- 172.21.15.89:0/4027281498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f498c159c60 0x7f498c15eba0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f498400b6c0 tx=0x7f4984018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:39.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.065+0000 7f4982ffd700 1 -- 172.21.15.89:0/4027281498 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49840032f0 con 0x7f498c159c60 2023-10-10T20:40:39.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.066+0000 7f4982ffd700 1 -- 172.21.15.89:0/4027281498 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4984008e00 con 0x7f498c159c60 2023-10-10T20:40:39.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.066+0000 7f4991214700 1 -- 172.21.15.89:0/4027281498 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498c169c70 con 0x7f498c159c60 2023-10-10T20:40:39.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.066+0000 7f4982ffd700 1 -- 172.21.15.89:0/4027281498 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498401c640 con 0x7f498c159c60 2023-10-10T20:40:39.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.066+0000 7f4991214700 1 -- 172.21.15.89:0/4027281498 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498c16a230 con 0x7f498c159c60 2023-10-10T20:40:39.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.066+0000 7f4991214700 1 -- 172.21.15.89:0/4027281498 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498c16a710 con 0x7f498c159c60 2023-10-10T20:40:39.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.067+0000 7f4982ffd700 1 -- 172.21.15.89:0/4027281498 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f4984003040 con 0x7f498c159c60 2023-10-10T20:40:39.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.067+0000 7f4982ffd700 1 --2- 172.21.15.89:0/4027281498 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f496c067290 0x7f496c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:39.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.067+0000 7f4982ffd700 1 -- 172.21.15.89:0/4027281498 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f49840863f0 con 0x7f498c159c60 2023-10-10T20:40:39.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.067+0000 7f498ad9d700 1 --2- 172.21.15.89:0/4027281498 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f496c067290 0x7f496c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:39.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.068+0000 7f498ad9d700 1 --2- 172.21.15.89:0/4027281498 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f496c067290 0x7f496c069740 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f497c006f00 tx=0x7f497c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:39.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.071+0000 7f4982ffd700 1 -- 172.21.15.89:0/4027281498 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4984053100 con 0x7f498c159c60 2023-10-10T20:40:39.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:39.302+0000 7f4991214700 1 -- 172.21.15.89:0/4027281498 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f498c04c3c0 con 0x7f498c159c60 2023-10-10T20:40:40.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.269+0000 7f4982ffd700 1 -- 172.21.15.89:0/4027281498 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v63) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4984056720 con 0x7f498c159c60 2023-10-10T20:40:40.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.272+0000 7f4991214700 1 -- 172.21.15.89:0/4027281498 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f496c067290 msgr2=0x7f496c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:40.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.272+0000 7f4991214700 1 --2- 172.21.15.89:0/4027281498 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f496c067290 0x7f496c069740 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f497c006f00 tx=0x7f497c008040 comp rx=0 tx=0).stop 2023-10-10T20:40:40.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.272+0000 7f4991214700 1 -- 172.21.15.89:0/4027281498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f498c159c60 msgr2=0x7f498c15eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:40.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.272+0000 7f4991214700 1 --2- 172.21.15.89:0/4027281498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f498c159c60 0x7f498c15eba0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f498400b6c0 tx=0x7f4984018000 comp rx=0 tx=0).stop 2023-10-10T20:40:40.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.272+0000 7f4991214700 1 -- 172.21.15.89:0/4027281498 shutdown_connections 2023-10-10T20:40:40.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.272+0000 7f4991214700 1 --2- 172.21.15.89:0/4027281498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f498c159c60 0x7f498c15eba0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:40.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.272+0000 7f4991214700 1 --2- 172.21.15.89:0/4027281498 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f496c067290 0x7f496c069740 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:40.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.272+0000 7f4991214700 1 --2- 172.21.15.89:0/4027281498 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f498c069560 0x7f498c149170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:40.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.272+0000 7f4991214700 1 --2- 172.21.15.89:0/4027281498 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f498c159890 0x7f498c15e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:40.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.272+0000 7f4991214700 1 -- 172.21.15.89:0/4027281498 >> 172.21.15.89:0/4027281498 conn(0x7f498c068e20 msgr2=0x7f498c0c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:40.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.272+0000 7f4991214700 1 -- 172.21.15.89:0/4027281498 shutdown_connections 2023-10-10T20:40:40.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.272+0000 7f4991214700 1 -- 172.21.15.89:0/4027281498 wait complete. 2023-10-10T20:40:40.289 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:40:40.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.408+0000 7ff15759e700 1 Processor -- start 2023-10-10T20:40:40.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.408+0000 7ff15759e700 1 -- start start 2023-10-10T20:40:40.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.409+0000 7ff15759e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff1580b4e70 0x7ff1580ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:40.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.409+0000 7ff15759e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1580b16d0 0x7ff1580b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:40.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.409+0000 7ff15759e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1580b2080 con 0x7ff15815d240 2023-10-10T20:40:40.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.409+0000 7ff15759e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1580cfb00 con 0x7ff1580b4e70 2023-10-10T20:40:40.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.409+0000 7ff15759e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580b5c40 con 0x7ff1580b16d0 2023-10-10T20:40:40.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.409+0000 7ff15659c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff1580b4e70 0x7ff1580ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:40.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.409+0000 7ff15659c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff1580b4e70 0x7ff1580ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:59350/0 (socket says 172.21.15.89:59350) 2023-10-10T20:40:40.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.409+0000 7ff15659c700 1 -- 172.21.15.89:0/3335113610 learned_addr learned my addr 172.21.15.89:0/3335113610 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:40.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.409+0000 7ff156d9d700 1 --1- 172.21.15.89:0/3335113610 >> v1:172.21.15.111:6789/0 conn(0x7ff15815d240 0x7ff15815d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:37806/0 (socket says 172.21.15.89:37806) 2023-10-10T20:40:40.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.409+0000 7ff14dd9b700 1 --2- 172.21.15.89:0/3335113610 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1580b16d0 0x7ff1580b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:40.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.410+0000 7ff15659c700 1 -- 172.21.15.89:0/3335113610 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1580b16d0 msgr2=0x7ff1580b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:40.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.410+0000 7ff15659c700 1 --2- 172.21.15.89:0/3335113610 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1580b16d0 0x7ff1580b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:40.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.410+0000 7ff15659c700 1 -- 172.21.15.89:0/3335113610 >> v1:172.21.15.111:6789/0 conn(0x7ff15815d240 legacy=0x7ff15815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:40.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.410+0000 7ff15659c700 1 -- 172.21.15.89:0/3335113610 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1580b5f20 con 0x7ff1580b4e70 2023-10-10T20:40:40.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.410+0000 7ff155d9b700 1 -- 172.21.15.89:0/3335113610 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924993689 0 0) 0x7ff1580b2080 con 0x7ff15815d240 2023-10-10T20:40:40.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.410+0000 7ff14dd9b700 1 --2- 172.21.15.89:0/3335113610 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1580b16d0 0x7ff1580b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:40:40.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.410+0000 7ff15659c700 1 --2- 172.21.15.89:0/3335113610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1580b4e70 0x7ff1580ccff0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff1480099b0 tx=0x7ff148009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed3399fc78ba7def server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:40.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.410+0000 7ff155d9b700 1 -- 172.21.15.89:0/3335113610 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff148009ea0 con 0x7ff1580b4e70 2023-10-10T20:40:40.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.410+0000 7ff155d9b700 1 -- 172.21.15.89:0/3335113610 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14800b860 con 0x7ff1580b4e70 2023-10-10T20:40:40.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.410+0000 7ff155d9b700 1 -- 172.21.15.89:0/3335113610 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14800bc00 con 0x7ff1580b4e70 2023-10-10T20:40:40.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.410+0000 7ff15759e700 1 -- 172.21.15.89:0/3335113610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1580b4e70 msgr2=0x7ff1580ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:40.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.411+0000 7ff15759e700 1 --2- 172.21.15.89:0/3335113610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1580b4e70 0x7ff1580ccff0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff1480099b0 tx=0x7ff148009250 comp rx=0 tx=0).stop 2023-10-10T20:40:40.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.411+0000 7ff15759e700 1 -- 172.21.15.89:0/3335113610 shutdown_connections 2023-10-10T20:40:40.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.411+0000 7ff15759e700 1 --2- 172.21.15.89:0/3335113610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1580b4e70 0x7ff1580ccff0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:40.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.411+0000 7ff15759e700 1 --2- 172.21.15.89:0/3335113610 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1580b16d0 0x7ff1580b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:40.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.411+0000 7ff15759e700 1 -- 172.21.15.89:0/3335113610 >> 172.21.15.89:0/3335113610 conn(0x7ff1580d10c0 msgr2=0x7ff1580affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:40.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.411+0000 7ff15759e700 1 -- 172.21.15.89:0/3335113610 shutdown_connections 2023-10-10T20:40:40.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.411+0000 7ff15759e700 1 -- 172.21.15.89:0/3335113610 wait complete. 2023-10-10T20:40:40.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.412+0000 7ff15759e700 1 Processor -- start 2023-10-10T20:40:40.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.412+0000 7ff15759e700 1 -- start start 2023-10-10T20:40:40.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff15759e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1580b16d0 0x7ff1580a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:40.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff15759e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1580b4e70 0x7ff1580ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:40.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff15759e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff15815d240 0x7ff1581698e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:40.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff15759e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580c9690 con 0x7ff15815d240 2023-10-10T20:40:40.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff15759e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580af8f0 con 0x7ff1580b16d0 2023-10-10T20:40:40.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff15759e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580c98e0 con 0x7ff1580b4e70 2023-10-10T20:40:40.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff156d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff15815d240 0x7ff1581698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:40.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff156d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff15815d240 0x7ff1581698e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38944/0 (socket says 172.21.15.89:38944) 2023-10-10T20:40:40.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff156d9d700 1 -- 172.21.15.89:0/3905562165 learned_addr learned my addr 172.21.15.89:0/3905562165 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:40.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff15659c700 1 --2- 172.21.15.89:0/3905562165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1580b16d0 0x7ff1580a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:40.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff14dd9b700 1 --2- 172.21.15.89:0/3905562165 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1580b4e70 0x7ff1580ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:40.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff156d9d700 1 -- 172.21.15.89:0/3905562165 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1580b4e70 msgr2=0x7ff1580ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:40.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff156d9d700 1 --2- 172.21.15.89:0/3905562165 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1580b4e70 0x7ff1580ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:40.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff156d9d700 1 -- 172.21.15.89:0/3905562165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1580b16d0 msgr2=0x7ff1580a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff156d9d700 1 --2- 172.21.15.89:0/3905562165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1580b16d0 0x7ff1580a6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff156d9d700 1 -- 172.21.15.89:0/3905562165 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff140009420 con 0x7ff15815d240 2023-10-10T20:40:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff156d9d700 1 --2- 172.21.15.89:0/3905562165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff15815d240 0x7ff1581698e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff15000ccc0 tx=0x7ff150001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff14effd700 1 -- 172.21.15.89:0/3905562165 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff150005df0 con 0x7ff15815d240 2023-10-10T20:40:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff14effd700 1 -- 172.21.15.89:0/3905562165 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15000a600 con 0x7ff15815d240 2023-10-10T20:40:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff15759e700 1 -- 172.21.15.89:0/3905562165 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1480094f0 con 0x7ff15815d240 2023-10-10T20:40:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff14effd700 1 -- 172.21.15.89:0/3905562165 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15000a9c0 con 0x7ff15815d240 2023-10-10T20:40:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.413+0000 7ff15759e700 1 -- 172.21.15.89:0/3905562165 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1580c9ea0 con 0x7ff15815d240 2023-10-10T20:40:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.414+0000 7ff15759e700 1 -- 172.21.15.89:0/3905562165 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff158088990 con 0x7ff15815d240 2023-10-10T20:40:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.415+0000 7ff14effd700 1 -- 172.21.15.89:0/3905562165 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7ff1500070f0 con 0x7ff15815d240 2023-10-10T20:40:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.415+0000 7ff14effd700 1 --2- 172.21.15.89:0/3905562165 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff138067290 0x7ff138069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.415+0000 7ff14effd700 1 -- 172.21.15.89:0/3905562165 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff150084ba0 con 0x7ff15815d240 2023-10-10T20:40:40.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.415+0000 7ff15659c700 1 --2- 172.21.15.89:0/3905562165 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff138067290 0x7ff138069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:40.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.415+0000 7ff15659c700 1 --2- 172.21.15.89:0/3905562165 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff138067290 0x7ff138069740 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff1480099b0 tx=0x7ff148017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:40.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.419+0000 7ff14effd700 1 -- 172.21.15.89:0/3905562165 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff150051a30 con 0x7ff15815d240 2023-10-10T20:40:40.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:40.645+0000 7ff15759e700 1 -- 172.21.15.89:0/3905562165 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff15804bb80 con 0x7ff15815d240 2023-10-10T20:40:41.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.274+0000 7ff14effd700 1 -- 172.21.15.89:0/3905562165 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff150055050 con 0x7ff15815d240 2023-10-10T20:40:41.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.277+0000 7ff15759e700 1 -- 172.21.15.89:0/3905562165 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff138067290 msgr2=0x7ff138069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:41.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.277+0000 7ff15759e700 1 --2- 172.21.15.89:0/3905562165 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff138067290 0x7ff138069740 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff1480099b0 tx=0x7ff148017000 comp rx=0 tx=0).stop 2023-10-10T20:40:41.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.277+0000 7ff15759e700 1 -- 172.21.15.89:0/3905562165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff15815d240 msgr2=0x7ff1581698e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:41.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.277+0000 7ff15759e700 1 --2- 172.21.15.89:0/3905562165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff15815d240 0x7ff1581698e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff15000ccc0 tx=0x7ff150001a70 comp rx=0 tx=0).stop 2023-10-10T20:40:41.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.277+0000 7ff15759e700 1 -- 172.21.15.89:0/3905562165 shutdown_connections 2023-10-10T20:40:41.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.277+0000 7ff15759e700 1 --2- 172.21.15.89:0/3905562165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff15815d240 0x7ff1581698e0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.277+0000 7ff15759e700 1 --2- 172.21.15.89:0/3905562165 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff138067290 0x7ff138069740 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.277+0000 7ff15759e700 1 --2- 172.21.15.89:0/3905562165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1580b16d0 0x7ff1580a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.277+0000 7ff15759e700 1 --2- 172.21.15.89:0/3905562165 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1580b4e70 0x7ff1580ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.277+0000 7ff15759e700 1 -- 172.21.15.89:0/3905562165 >> 172.21.15.89:0/3905562165 conn(0x7ff1580d10c0 msgr2=0x7ff158159050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.277+0000 7ff15759e700 1 -- 172.21.15.89:0/3905562165 shutdown_connections 2023-10-10T20:40:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.277+0000 7ff15759e700 1 -- 172.21.15.89:0/3905562165 wait complete. 2023-10-10T20:40:41.292 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:40:41.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.406+0000 7f465e8c6700 1 Processor -- start 2023-10-10T20:40:41.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.406+0000 7f465e8c6700 1 -- start start 2023-10-10T20:40:41.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.407+0000 7f465e8c6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46581597f0 0x7f46580b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:41.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.407+0000 7f465e8c6700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f465815d230 0x7f465815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:41.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.407+0000 7f465e8c6700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f465815dbd0 con 0x7f4658159420 2023-10-10T20:40:41.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.407+0000 7f465e8c6700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46580b11d0 con 0x7f465815d230 2023-10-10T20:40:41.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.407+0000 7f465e8c6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46580b14b0 con 0x7f46581597f0 2023-10-10T20:40:41.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.407+0000 7f465e0c5700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f465815d230 0x7f465815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:41.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.407+0000 7f465d0c3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46581597f0 0x7f46580b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:41.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.407+0000 7f465d0c3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46581597f0 0x7f46580b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:55140/0 (socket says 172.21.15.89:55140) 2023-10-10T20:40:41.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.407+0000 7f465d8c4700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f4658159420 0x7f46580b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:37814/0 (socket says 172.21.15.89:37814) 2023-10-10T20:40:41.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.407+0000 7f465d0c3700 1 -- 172.21.15.89:0/3033213071 learned_addr learned my addr 172.21.15.89:0/3033213071 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:41.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.408+0000 7f465c8c2700 1 -- 172.21.15.89:0/3033213071 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2254094602 0 0) 0x7f4654001490 con 0x7f4658159420 2023-10-10T20:40:41.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.408+0000 7f465c8c2700 1 -- 172.21.15.89:0/3033213071 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4644003570 con 0x7f4658159420 2023-10-10T20:40:41.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.408+0000 7f465e0c5700 1 -- 172.21.15.89:0/3033213071 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46581597f0 msgr2=0x7f46580b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:41.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.408+0000 7f465e0c5700 1 --2- 172.21.15.89:0/3033213071 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46581597f0 0x7f46580b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.408+0000 7f465e0c5700 1 -- 172.21.15.89:0/3033213071 >> v1:172.21.15.111:6789/0 conn(0x7f4658159420 legacy=0x7f46580b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:41.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.408+0000 7f465e0c5700 1 -- 172.21.15.89:0/3033213071 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46580b1790 con 0x7f465815d230 2023-10-10T20:40:41.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.408+0000 7f465d0c3700 1 --2- 172.21.15.89:0/3033213071 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46581597f0 0x7f46580b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:40:41.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.408+0000 7f465e0c5700 1 --2- 172.21.15.89:0/3033213071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f465815d230 0x7f465815d690 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f46500098e0 tx=0x7f46500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=215fe2df36d056fe server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:41.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.408+0000 7f465c8c2700 1 -- 172.21.15.89:0/3033213071 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650011630 con 0x7f465815d230 2023-10-10T20:40:41.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.408+0000 7f465c8c2700 1 -- 172.21.15.89:0/3033213071 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4650011790 con 0x7f465815d230 2023-10-10T20:40:41.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.408+0000 7f465c8c2700 1 -- 172.21.15.89:0/3033213071 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650011b10 con 0x7f465815d230 2023-10-10T20:40:41.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.409+0000 7f465e8c6700 1 -- 172.21.15.89:0/3033213071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f465815d230 msgr2=0x7f465815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:41.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.409+0000 7f465e8c6700 1 --2- 172.21.15.89:0/3033213071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f465815d230 0x7f465815d690 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f46500098e0 tx=0x7f46500091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.409+0000 7f465e8c6700 1 -- 172.21.15.89:0/3033213071 shutdown_connections 2023-10-10T20:40:41.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.409+0000 7f465e8c6700 1 --2- 172.21.15.89:0/3033213071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f465815d230 0x7f465815d690 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.409+0000 7f465e8c6700 1 --2- 172.21.15.89:0/3033213071 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46581597f0 0x7f46580b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.409+0000 7f465e8c6700 1 -- 172.21.15.89:0/3033213071 >> 172.21.15.89:0/3033213071 conn(0x7f46580d0660 msgr2=0x7f46580d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:41.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.409+0000 7f465e8c6700 1 -- 172.21.15.89:0/3033213071 shutdown_connections 2023-10-10T20:40:41.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.409+0000 7f465e8c6700 1 -- 172.21.15.89:0/3033213071 wait complete. 2023-10-10T20:40:41.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.410+0000 7f465e8c6700 1 Processor -- start 2023-10-10T20:40:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.410+0000 7f465e8c6700 1 -- start start 2023-10-10T20:40:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f465e8c6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4658159420 0x7f46580c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f465e8c6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46581597f0 0x7f465815bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f465e8c6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f465815d230 0x7f465815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f465e8c6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465815cb40 con 0x7f46581597f0 2023-10-10T20:40:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f465e8c6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46580c6800 con 0x7f465815d230 2023-10-10T20:40:41.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f465e8c6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46580c5f60 con 0x7f4658159420 2023-10-10T20:40:41.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f465d0c3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46581597f0 0x7f465815bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:41.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f465d0c3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46581597f0 0x7f465815bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38954/0 (socket says 172.21.15.89:38954) 2023-10-10T20:40:41.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f465d0c3700 1 -- 172.21.15.89:0/2717660462 learned_addr learned my addr 172.21.15.89:0/2717660462 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:41.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f465e0c5700 1 --2- 172.21.15.89:0/2717660462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f465815d230 0x7f465815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:41.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f465d8c4700 1 --2- 172.21.15.89:0/2717660462 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4658159420 0x7f46580c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:41.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f465d0c3700 1 -- 172.21.15.89:0/2717660462 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4658159420 msgr2=0x7f46580c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:41.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f465d0c3700 1 --2- 172.21.15.89:0/2717660462 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4658159420 0x7f46580c58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f465d0c3700 1 -- 172.21.15.89:0/2717660462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f465815d230 msgr2=0x7f465815ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:41.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f465d0c3700 1 --2- 172.21.15.89:0/2717660462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f465815d230 0x7f465815ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f465d0c3700 1 -- 172.21.15.89:0/2717660462 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46480094f0 con 0x7f46581597f0 2023-10-10T20:40:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f465d0c3700 1 --2- 172.21.15.89:0/2717660462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46581597f0 0x7f465815bbb0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f464800ec50 tx=0x7f4648009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f464dffb700 1 -- 172.21.15.89:0/2717660462 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464800c850 con 0x7f46581597f0 2023-10-10T20:40:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f464dffb700 1 -- 172.21.15.89:0/2717660462 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464800c9b0 con 0x7f46581597f0 2023-10-10T20:40:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f465e8c6700 1 -- 172.21.15.89:0/2717660462 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4650009420 con 0x7f46581597f0 2023-10-10T20:40:41.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.411+0000 7f464dffb700 1 -- 172.21.15.89:0/2717660462 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464800cd50 con 0x7f46581597f0 2023-10-10T20:40:41.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.412+0000 7f465e8c6700 1 -- 172.21.15.89:0/2717660462 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46580c6520 con 0x7f46581597f0 2023-10-10T20:40:41.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.413+0000 7f465e8c6700 1 -- 172.21.15.89:0/2717660462 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46580d01f0 con 0x7f46581597f0 2023-10-10T20:40:41.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.413+0000 7f464dffb700 1 -- 172.21.15.89:0/2717660462 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f4648013680 con 0x7f46581597f0 2023-10-10T20:40:41.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.413+0000 7f464dffb700 1 --2- 172.21.15.89:0/2717660462 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f463c067290 0x7f463c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:41.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.413+0000 7f464dffb700 1 -- 172.21.15.89:0/2717660462 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4648084d20 con 0x7f46581597f0 2023-10-10T20:40:41.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.413+0000 7f465d8c4700 1 --2- 172.21.15.89:0/2717660462 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f463c067290 0x7f463c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:41.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.414+0000 7f465d8c4700 1 --2- 172.21.15.89:0/2717660462 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f463c067290 0x7f463c069740 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4654002100 tx=0x7f4654000f40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:41.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.417+0000 7f464dffb700 1 -- 172.21.15.89:0/2717660462 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4648051b80 con 0x7f46581597f0 2023-10-10T20:40:41.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.642+0000 7f465e8c6700 1 -- 172.21.15.89:0/2717660462 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f465804c3c0 con 0x7f46581597f0 2023-10-10T20:40:41.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.643+0000 7f464dffb700 1 -- 172.21.15.89:0/2717660462 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f46480551a0 con 0x7f46581597f0 2023-10-10T20:40:41.646 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:41.646 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1635581143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1635581143},{"type":"v1","addr":"172.21.15.89:6837","nonce":1635581143}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5732,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/321638734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":321638734},{"type":"v1","addr":"172.21.15.111:6835","nonce":321638734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1070384555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1070384555},{"type":"v1","addr":"172.21.15.111:6834","nonce":1070384555}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:40:37.630844+0000","modified":"2023-10-10T20:40:41.269543+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6822},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6822":{"gid":6822,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6836/2274240619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2274240619},{"type":"v1","addr":"172.21.15.89:6836","nonce":2274240619}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-10T20:40:41.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.645+0000 7f465e8c6700 1 -- 172.21.15.89:0/2717660462 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f463c067290 msgr2=0x7f463c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:41.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.645+0000 7f465e8c6700 1 --2- 172.21.15.89:0/2717660462 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f463c067290 0x7f463c069740 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4654002100 tx=0x7f4654000f40 comp rx=0 tx=0).stop 2023-10-10T20:40:41.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.645+0000 7f465e8c6700 1 -- 172.21.15.89:0/2717660462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46581597f0 msgr2=0x7f465815bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:41.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.645+0000 7f465e8c6700 1 --2- 172.21.15.89:0/2717660462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46581597f0 0x7f465815bbb0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f464800ec50 tx=0x7f4648009250 comp rx=0 tx=0).stop 2023-10-10T20:40:41.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.645+0000 7f465e8c6700 1 -- 172.21.15.89:0/2717660462 shutdown_connections 2023-10-10T20:40:41.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.645+0000 7f465e8c6700 1 --2- 172.21.15.89:0/2717660462 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f463c067290 0x7f463c069740 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.645+0000 7f465e8c6700 1 --2- 172.21.15.89:0/2717660462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f465815d230 0x7f465815ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.645+0000 7f465e8c6700 1 --2- 172.21.15.89:0/2717660462 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4658159420 0x7f46580c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.645+0000 7f465e8c6700 1 --2- 172.21.15.89:0/2717660462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46581597f0 0x7f465815bbb0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.645+0000 7f465e8c6700 1 -- 172.21.15.89:0/2717660462 >> 172.21.15.89:0/2717660462 conn(0x7f46580d0660 msgr2=0x7f4658068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:41.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.645+0000 7f465e8c6700 1 -- 172.21.15.89:0/2717660462 shutdown_connections 2023-10-10T20:40:41.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.645+0000 7f465e8c6700 1 -- 172.21.15.89:0/2717660462 wait complete. 2023-10-10T20:40:41.650 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 64 2023-10-10T20:40:41.659 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:40:41.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.772+0000 7f5a6e8fc700 1 Processor -- start 2023-10-10T20:40:41.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.772+0000 7f5a6e8fc700 1 -- start start 2023-10-10T20:40:41.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.773+0000 7f5a6e8fc700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5a68159420 0x7f5a680b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:41.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.773+0000 7f5a6e8fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5a681597f0 0x7f5a680b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:41.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.773+0000 7f5a6e8fc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a680b11d0 con 0x7f5a6815d230 2023-10-10T20:40:41.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.773+0000 7f5a6e8fc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a680b14b0 con 0x7f5a681597f0 2023-10-10T20:40:41.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.773+0000 7f5a6e8fc700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a680b1790 con 0x7f5a68159420 2023-10-10T20:40:41.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.773+0000 7f5a6d0f9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5a681597f0 0x7f5a680b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:41.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.773+0000 7f5a6e0fb700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5a6815d230 0x7f5a6815d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49170/0 (socket says 172.21.15.89:49170) 2023-10-10T20:40:41.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.773+0000 7f5a6d0f9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5a681597f0 0x7f5a680b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38962/0 (socket says 172.21.15.89:38962) 2023-10-10T20:40:41.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.773+0000 7f5a6e0fb700 1 -- 172.21.15.89:0/1017806510 learned_addr learned my addr 172.21.15.89:0/1017806510 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:41.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.773+0000 7f5a6d8fa700 1 --2- 172.21.15.89:0/1017806510 >> v2:172.21.15.111:3300/0 conn(0x7f5a68159420 0x7f5a680b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:41.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.773+0000 7f5a6d0f9700 1 -- 172.21.15.89:0/1017806510 >> v2:172.21.15.111:3300/0 conn(0x7f5a68159420 msgr2=0x7f5a680b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:41.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.773+0000 7f5a6d0f9700 1 --2- 172.21.15.89:0/1017806510 >> v2:172.21.15.111:3300/0 conn(0x7f5a68159420 0x7f5a680b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.773+0000 7f5a6d0f9700 1 -- 172.21.15.89:0/1017806510 >> v1:172.21.15.89:6789/0 conn(0x7f5a6815d230 legacy=0x7f5a6815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:41.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.773+0000 7f5a6d0f9700 1 -- 172.21.15.89:0/1017806510 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a680b1a70 con 0x7f5a681597f0 2023-10-10T20:40:41.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.773+0000 7f5a6d0f9700 1 --2- 172.21.15.89:0/1017806510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a681597f0 0x7f5a680b2270 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f5a680b2f20 tx=0x7f5a580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7ed91fefc89058 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:41.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.774+0000 7f5a6c8f8700 1 -- 172.21.15.89:0/1017806510 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58011630 con 0x7f5a681597f0 2023-10-10T20:40:41.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.774+0000 7f5a6c8f8700 1 -- 172.21.15.89:0/1017806510 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a58011790 con 0x7f5a681597f0 2023-10-10T20:40:41.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.774+0000 7f5a6c8f8700 1 -- 172.21.15.89:0/1017806510 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58011b30 con 0x7f5a681597f0 2023-10-10T20:40:41.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.774+0000 7f5a6e8fc700 1 -- 172.21.15.89:0/1017806510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a681597f0 msgr2=0x7f5a680b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:41.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.774+0000 7f5a6e8fc700 1 --2- 172.21.15.89:0/1017806510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a681597f0 0x7f5a680b2270 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f5a680b2f20 tx=0x7f5a580091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.774+0000 7f5a6e8fc700 1 -- 172.21.15.89:0/1017806510 shutdown_connections 2023-10-10T20:40:41.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.774+0000 7f5a6e8fc700 1 --2- 172.21.15.89:0/1017806510 >> v2:172.21.15.111:3300/0 conn(0x7f5a68159420 0x7f5a680b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.774+0000 7f5a6e8fc700 1 --2- 172.21.15.89:0/1017806510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a681597f0 0x7f5a680b2270 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.774+0000 7f5a6e8fc700 1 -- 172.21.15.89:0/1017806510 >> 172.21.15.89:0/1017806510 conn(0x7f5a680d0660 msgr2=0x7f5a680d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:41.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.774+0000 7f5a6e8fc700 1 -- 172.21.15.89:0/1017806510 shutdown_connections 2023-10-10T20:40:41.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.774+0000 7f5a6e8fc700 1 -- 172.21.15.89:0/1017806510 wait complete. 2023-10-10T20:40:41.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.775+0000 7f5a6e8fc700 1 Processor -- start 2023-10-10T20:40:41.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.775+0000 7f5a6e8fc700 1 -- start start 2023-10-10T20:40:41.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.776+0000 7f5a6e8fc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a68159420 0x7f5a6814e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:41.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.776+0000 7f5a6e8fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a681597f0 0x7f5a68155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:41.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.776+0000 7f5a6e8fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a6815d230 0x7f5a68156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:41.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.776+0000 7f5a6e8fc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680b51f0 con 0x7f5a6815d230 2023-10-10T20:40:41.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.776+0000 7f5a6e8fc700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a681503f0 con 0x7f5a681597f0 2023-10-10T20:40:41.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.776+0000 7f5a6e8fc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6815ab80 con 0x7f5a68159420 2023-10-10T20:40:41.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.776+0000 7f5a6e0fb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a6815d230 0x7f5a68156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:41.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.776+0000 7f5a6e0fb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a6815d230 0x7f5a68156240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38968/0 (socket says 172.21.15.89:38968) 2023-10-10T20:40:41.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.776+0000 7f5a6e0fb700 1 -- 172.21.15.89:0/4056564839 learned_addr learned my addr 172.21.15.89:0/4056564839 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:41.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.776+0000 7f5a6d8fa700 1 --2- 172.21.15.89:0/4056564839 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a68159420 0x7f5a6814e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:41.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.776+0000 7f5a6d0f9700 1 --2- 172.21.15.89:0/4056564839 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a681597f0 0x7f5a68155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:41.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.776+0000 7f5a6e0fb700 1 -- 172.21.15.89:0/4056564839 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a68159420 msgr2=0x7f5a6814e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:41.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.776+0000 7f5a6e0fb700 1 --2- 172.21.15.89:0/4056564839 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a68159420 0x7f5a6814e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.776+0000 7f5a6e0fb700 1 -- 172.21.15.89:0/4056564839 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a681597f0 msgr2=0x7f5a68155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:41.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.776+0000 7f5a6e0fb700 1 --2- 172.21.15.89:0/4056564839 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a681597f0 0x7f5a68155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.776+0000 7f5a6e0fb700 1 -- 172.21.15.89:0/4056564839 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a58009420 con 0x7f5a6815d230 2023-10-10T20:40:41.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.777+0000 7f5a6e0fb700 1 --2- 172.21.15.89:0/4056564839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a6815d230 0x7f5a68156240 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000cbc0 tx=0x7f5a60001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:41.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.777+0000 7f5a5dffb700 1 -- 172.21.15.89:0/4056564839 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000a7a0 con 0x7f5a6815d230 2023-10-10T20:40:41.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.777+0000 7f5a5dffb700 1 -- 172.21.15.89:0/4056564839 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6000a900 con 0x7f5a6815d230 2023-10-10T20:40:41.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.777+0000 7f5a6e8fc700 1 -- 172.21.15.89:0/4056564839 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a6815ae60 con 0x7f5a6815d230 2023-10-10T20:40:41.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.777+0000 7f5a5dffb700 1 -- 172.21.15.89:0/4056564839 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000ac80 con 0x7f5a6815d230 2023-10-10T20:40:41.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.777+0000 7f5a6e8fc700 1 -- 172.21.15.89:0/4056564839 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6815b420 con 0x7f5a6815d230 2023-10-10T20:40:41.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.778+0000 7f5a6e8fc700 1 -- 172.21.15.89:0/4056564839 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a681505b0 con 0x7f5a6815d230 2023-10-10T20:40:41.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.778+0000 7f5a5dffb700 1 -- 172.21.15.89:0/4056564839 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f5a60007300 con 0x7f5a6815d230 2023-10-10T20:40:41.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.779+0000 7f5a5dffb700 1 --2- 172.21.15.89:0/4056564839 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a4c067290 0x7f5a4c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:41.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.779+0000 7f5a5dffb700 1 -- 172.21.15.89:0/4056564839 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5a60084e10 con 0x7f5a6815d230 2023-10-10T20:40:41.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.779+0000 7f5a6d8fa700 1 --2- 172.21.15.89:0/4056564839 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a4c067290 0x7f5a4c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:41.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.779+0000 7f5a6d8fa700 1 --2- 172.21.15.89:0/4056564839 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a4c067290 0x7f5a4c069740 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009db0 tx=0x7f5a640091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:41.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.783+0000 7f5a5dffb700 1 -- 172.21.15.89:0/4056564839 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a60051b70 con 0x7f5a6815d230 2023-10-10T20:40:41.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.965+0000 7f5a6e8fc700 1 -- 172.21.15.89:0/4056564839 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a6804c3c0 con 0x7f5a6815d230 2023-10-10T20:40:41.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.966+0000 7f5a5dffb700 1 -- 172.21.15.89:0/4056564839 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+15355 (secure 0 0 0) 0x7f5a60055190 con 0x7f5a6815d230 2023-10-10T20:40:41.969 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:41.969 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":100,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:40:39.247030+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:40:36.002449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-10T20:40:36.591960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:40:41.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.969+0000 7f5a6e8fc700 1 -- 172.21.15.89:0/4056564839 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a4c067290 msgr2=0x7f5a4c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:41.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.969+0000 7f5a6e8fc700 1 --2- 172.21.15.89:0/4056564839 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a4c067290 0x7f5a4c069740 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009db0 tx=0x7f5a640091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.969+0000 7f5a6e8fc700 1 -- 172.21.15.89:0/4056564839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a6815d230 msgr2=0x7f5a68156240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:41.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.969+0000 7f5a6e8fc700 1 --2- 172.21.15.89:0/4056564839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a6815d230 0x7f5a68156240 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000cbc0 tx=0x7f5a60001990 comp rx=0 tx=0).stop 2023-10-10T20:40:41.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.969+0000 7f5a6e8fc700 1 -- 172.21.15.89:0/4056564839 shutdown_connections 2023-10-10T20:40:41.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.969+0000 7f5a6e8fc700 1 --2- 172.21.15.89:0/4056564839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a6815d230 0x7f5a68156240 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.969+0000 7f5a6e8fc700 1 --2- 172.21.15.89:0/4056564839 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a68159420 0x7f5a6814e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.969+0000 7f5a6e8fc700 1 --2- 172.21.15.89:0/4056564839 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a4c067290 0x7f5a4c069740 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.969+0000 7f5a6e8fc700 1 --2- 172.21.15.89:0/4056564839 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a681597f0 0x7f5a68155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:41.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.969+0000 7f5a6e8fc700 1 -- 172.21.15.89:0/4056564839 >> 172.21.15.89:0/4056564839 conn(0x7f5a680d0660 msgr2=0x7f5a680b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:41.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.969+0000 7f5a6e8fc700 1 -- 172.21.15.89:0/4056564839 shutdown_connections 2023-10-10T20:40:41.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:41.969+0000 7f5a6e8fc700 1 -- 172.21.15.89:0/4056564839 wait complete. 2023-10-10T20:40:41.983 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:40:41.984 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:40:42.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.095+0000 7f8c1089d700 1 Processor -- start 2023-10-10T20:40:42.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.095+0000 7f8c1089d700 1 -- start start 2023-10-10T20:40:42.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.096+0000 7f8c1089d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c0c13f620 0x7f8c0c0af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:42.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.096+0000 7f8c1089d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8c0c0c8120 0x7f8c0c0c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:42.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.096+0000 7f8c1089d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c0c0c8ad0 con 0x7f8c0c156270 2023-10-10T20:40:42.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.096+0000 7f8c1089d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c0c0b2080 con 0x7f8c0c0c8120 2023-10-10T20:40:42.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.096+0000 7f8c1089d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c06eb10 con 0x7f8c0c13f620 2023-10-10T20:40:42.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.096+0000 7f8c0b59e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8c0c156270 0x7f8c0c1566d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49176/0 (socket says 172.21.15.89:49176) 2023-10-10T20:40:42.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.096+0000 7f8c0b59e700 1 -- 172.21.15.89:0/1473234407 learned_addr learned my addr 172.21.15.89:0/1473234407 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:42.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.096+0000 7f8c0259c700 1 --2- 172.21.15.89:0/1473234407 >> v2:172.21.15.111:3300/0 conn(0x7f8c0c0c8120 0x7f8c0c0c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:42.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.096+0000 7f8c0ad9d700 1 --2- 172.21.15.89:0/1473234407 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c0c13f620 0x7f8c0c0af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:42.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.096+0000 7f8c0a59c700 1 -- 172.21.15.89:0/1473234407 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2860812448 0 0) 0x7f8c0c0c8ad0 con 0x7f8c0c156270 2023-10-10T20:40:42.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.096+0000 7f8c0a59c700 1 -- 172.21.15.89:0/1473234407 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bf8003570 con 0x7f8c0c156270 2023-10-10T20:40:42.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.096+0000 7f8c0a59c700 1 -- 172.21.15.89:0/1473234407 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3067727505 0 0) 0x7f8bf8003570 con 0x7f8c0c156270 2023-10-10T20:40:42.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.096+0000 7f8c0259c700 1 -- 172.21.15.89:0/1473234407 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c0c13f620 msgr2=0x7f8c0c0af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:42.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.096+0000 7f8c0259c700 1 --2- 172.21.15.89:0/1473234407 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c0c13f620 0x7f8c0c0af9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.096+0000 7f8c0259c700 1 -- 172.21.15.89:0/1473234407 >> v1:172.21.15.89:6789/0 conn(0x7f8c0c156270 legacy=0x7f8c0c1566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:42.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.097+0000 7f8c0259c700 1 -- 172.21.15.89:0/1473234407 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0c0b2320 con 0x7f8c0c0c8120 2023-10-10T20:40:42.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.097+0000 7f8c0259c700 1 --2- 172.21.15.89:0/1473234407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c0c0c8120 0x7f8c0c0c8590 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40098e0 tx=0x7f8bf40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eab2b7c84bb83ab0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:42.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.097+0000 7f8c0a59c700 1 -- 172.21.15.89:0/1473234407 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c040017f0 con 0x7f8c0c0c8120 2023-10-10T20:40:42.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.097+0000 7f8c0a59c700 1 -- 172.21.15.89:0/1473234407 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf4011420 con 0x7f8c0c0c8120 2023-10-10T20:40:42.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.097+0000 7f8c0a59c700 1 -- 172.21.15.89:0/1473234407 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40117c0 con 0x7f8c0c0c8120 2023-10-10T20:40:42.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.097+0000 7f8c1089d700 1 -- 172.21.15.89:0/1473234407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c0c0c8120 msgr2=0x7f8c0c0c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:42.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.097+0000 7f8c1089d700 1 --2- 172.21.15.89:0/1473234407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c0c0c8120 0x7f8c0c0c8590 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40098e0 tx=0x7f8bf40091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.097+0000 7f8c1089d700 1 -- 172.21.15.89:0/1473234407 shutdown_connections 2023-10-10T20:40:42.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.097+0000 7f8c1089d700 1 --2- 172.21.15.89:0/1473234407 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c0c13f620 0x7f8c0c0af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.097+0000 7f8c1089d700 1 --2- 172.21.15.89:0/1473234407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c0c0c8120 0x7f8c0c0c8590 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.097+0000 7f8c1089d700 1 -- 172.21.15.89:0/1473234407 >> 172.21.15.89:0/1473234407 conn(0x7f8c0c06ddb0 msgr2=0x7f8c0c06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:42.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.097+0000 7f8c1089d700 1 -- 172.21.15.89:0/1473234407 shutdown_connections 2023-10-10T20:40:42.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.097+0000 7f8c1089d700 1 -- 172.21.15.89:0/1473234407 wait complete. 2023-10-10T20:40:42.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.099+0000 7f8c1089d700 1 Processor -- start 2023-10-10T20:40:42.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.099+0000 7f8c1089d700 1 -- start start 2023-10-10T20:40:42.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.099+0000 7f8c1089d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c0c0c8120 0x7f8c0c14ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:42.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.099+0000 7f8c1089d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c0c13f620 0x7f8c0c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:42.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.099+0000 7f8c1089d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c0c156270 0x7f8c0c06b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:42.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.099+0000 7f8c1089d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c06c2e0 con 0x7f8c0c156270 2023-10-10T20:40:42.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.099+0000 7f8c1089d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c06b6d0 con 0x7f8c0c13f620 2023-10-10T20:40:42.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.099+0000 7f8c1089d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c06b9b0 con 0x7f8c0c0c8120 2023-10-10T20:40:42.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.099+0000 7f8c0b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c0c156270 0x7f8c0c06b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:42.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.100+0000 7f8c0b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c0c156270 0x7f8c0c06b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38978/0 (socket says 172.21.15.89:38978) 2023-10-10T20:40:42.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.100+0000 7f8c0b59e700 1 -- 172.21.15.89:0/2082191172 learned_addr learned my addr 172.21.15.89:0/2082191172 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:42.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.100+0000 7f8c0259c700 1 --2- 172.21.15.89:0/2082191172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c0c13f620 0x7f8c0c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:42.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.100+0000 7f8c0ad9d700 1 --2- 172.21.15.89:0/2082191172 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c0c0c8120 0x7f8c0c14ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:42.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.100+0000 7f8c0b59e700 1 -- 172.21.15.89:0/2082191172 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c0c0c8120 msgr2=0x7f8c0c14ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:42.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.100+0000 7f8c0b59e700 1 --2- 172.21.15.89:0/2082191172 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c0c0c8120 0x7f8c0c14ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.100+0000 7f8c0b59e700 1 -- 172.21.15.89:0/2082191172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c0c13f620 msgr2=0x7f8c0c06ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:42.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.100+0000 7f8c0b59e700 1 --2- 172.21.15.89:0/2082191172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c0c13f620 0x7f8c0c06ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.100+0000 7f8c0b59e700 1 -- 172.21.15.89:0/2082191172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf4009420 con 0x7f8c0c156270 2023-10-10T20:40:42.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.100+0000 7f8c0b59e700 1 --2- 172.21.15.89:0/2082191172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c0c156270 0x7f8c0c06b050 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400cc20 tx=0x7f8c040015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:42.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.100+0000 7f8c037fe700 1 -- 172.21.15.89:0/2082191172 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c04004dc0 con 0x7f8c0c156270 2023-10-10T20:40:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.100+0000 7f8c037fe700 1 -- 172.21.15.89:0/2082191172 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0400a7b0 con 0x7f8c0c156270 2023-10-10T20:40:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.100+0000 7f8c1089d700 1 -- 172.21.15.89:0/2082191172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0c06bc90 con 0x7f8c0c156270 2023-10-10T20:40:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.100+0000 7f8c037fe700 1 -- 172.21.15.89:0/2082191172 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0400ab70 con 0x7f8c0c156270 2023-10-10T20:40:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.100+0000 7f8c1089d700 1 -- 172.21.15.89:0/2082191172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0c14fa80 con 0x7f8c0c156270 2023-10-10T20:40:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.101+0000 7f8c1089d700 1 -- 172.21.15.89:0/2082191172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c0c04c3c0 con 0x7f8c0c156270 2023-10-10T20:40:42.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.102+0000 7f8c037fe700 1 -- 172.21.15.89:0/2082191172 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f8c04007150 con 0x7f8c0c156270 2023-10-10T20:40:42.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.102+0000 7f8c037fe700 1 --2- 172.21.15.89:0/2082191172 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8bec067280 0x7f8bec069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:42.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.102+0000 7f8c037fe700 1 -- 172.21.15.89:0/2082191172 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8c040845f0 con 0x7f8c0c156270 2023-10-10T20:40:42.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.102+0000 7f8c0ad9d700 1 --2- 172.21.15.89:0/2082191172 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8bec067280 0x7f8bec069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:42.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.102+0000 7f8c0ad9d700 1 --2- 172.21.15.89:0/2082191172 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8bec067280 0x7f8bec069730 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0093f0 tx=0x7f8bfc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:42.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.106+0000 7f8c037fe700 1 -- 172.21.15.89:0/2082191172 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c040514b0 con 0x7f8c0c156270 2023-10-10T20:40:42.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.286+0000 7f8c1089d700 1 -- 172.21.15.89:0/2082191172 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8c0c0b2600 con 0x7f8bec067280 2023-10-10T20:40:42.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.294+0000 7f8c037fe700 1 -- 172.21.15.89:0/2082191172 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286966 (secure 0 0 0) 0x7f8c0c0b2600 con 0x7f8bec067280 2023-10-10T20:40:42.300 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:42.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.298+0000 7f8c1089d700 1 -- 172.21.15.89:0/2082191172 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8bec067280 msgr2=0x7f8bec069730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:42.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.298+0000 7f8c1089d700 1 --2- 172.21.15.89:0/2082191172 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8bec067280 0x7f8bec069730 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0093f0 tx=0x7f8bfc008040 comp rx=0 tx=0).stop 2023-10-10T20:40:42.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.298+0000 7f8c1089d700 1 -- 172.21.15.89:0/2082191172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c0c156270 msgr2=0x7f8c0c06b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:42.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.298+0000 7f8c1089d700 1 --2- 172.21.15.89:0/2082191172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c0c156270 0x7f8c0c06b050 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400cc20 tx=0x7f8c040015b0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.298+0000 7f8c1089d700 1 -- 172.21.15.89:0/2082191172 shutdown_connections 2023-10-10T20:40:42.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.298+0000 7f8c1089d700 1 --2- 172.21.15.89:0/2082191172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c0c156270 0x7f8c0c06b050 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.298+0000 7f8c1089d700 1 --2- 172.21.15.89:0/2082191172 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c0c0c8120 0x7f8c0c14ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.298+0000 7f8c1089d700 1 --2- 172.21.15.89:0/2082191172 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8bec067280 0x7f8bec069730 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.298+0000 7f8c1089d700 1 --2- 172.21.15.89:0/2082191172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c0c13f620 0x7f8c0c06ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.298+0000 7f8c1089d700 1 -- 172.21.15.89:0/2082191172 >> 172.21.15.89:0/2082191172 conn(0x7f8c0c06ddb0 msgr2=0x7f8c0c0c8530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:42.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.298+0000 7f8c1089d700 1 -- 172.21.15.89:0/2082191172 shutdown_connections 2023-10-10T20:40:42.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.298+0000 7f8c1089d700 1 -- 172.21.15.89:0/2082191172 wait complete. 2023-10-10T20:40:42.306 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:40:42.312 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2023-10-10T20:40:40.725402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2413536,"kb_used_data":10656,"kb_used_omap":0,"kb_used_meta":2402816,"kb_avail":752561184,"statfs":{"total":773094113280,"available":770622652416,"internally_reserved":0,"allocated":10911744,"data_stored":5357850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2460483584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.486409"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253558+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:25.230014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.260877+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:53:07.253745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255734+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:43.144258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.250640+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:28:37.819892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255881+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:36:00.607814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252745+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:15.831776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257694+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:45:44.964754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.258387+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:59:28.728877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253182+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:46:05.714540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252306+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:37:30.151565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261779+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:29:51.950959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.250202+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:39:38.169839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253730+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:03:14.859123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.258385+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:29:39.101585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254614+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:09:46.330074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254027+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:36.297633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.263244+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:43.144258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255797+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:27.914463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255314+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:27.914463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261622+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:14:29.705941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.263327+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:37:04.622821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.250382+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:59:28.728877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.263024+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:27.914463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.251359+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:39:38.169839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261486+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:07.822152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.251605+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:54.041748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254573+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:30:45.472893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.250590+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:23.463451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.262806+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:26.505825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253631+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:09:46.330074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252631+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:58:16.915031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252495+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:10:45.408128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253140+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:42:01.406431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265989+0000","last_change":"2023-10-10T20:40:37.265989+0000","last_active":"2023-10-10T20:40:37.265989+0000","last_peered":"2023-10-10T20:40:37.265989+0000","last_clean":"2023-10-10T20:40:37.265989+0000","last_became_active":"2023-10-10T20:40:37.265518+0000","last_became_peered":"2023-10-10T20:40:37.265518+0000","last_unstale":"2023-10-10T20:40:37.265989+0000","last_undegraded":"2023-10-10T20:40:37.265989+0000","last_fullsized":"2023-10-10T20:40:37.265989+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:10:44.403291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.256386+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:43.144258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.268252+0000","last_change":"2023-10-10T20:40:37.268252+0000","last_active":"2023-10-10T20:40:37.268252+0000","last_peered":"2023-10-10T20:40:37.268252+0000","last_clean":"2023-10-10T20:40:37.268252+0000","last_became_active":"2023-10-10T20:40:37.267982+0000","last_became_peered":"2023-10-10T20:40:37.267982+0000","last_unstale":"2023-10-10T20:40:37.268252+0000","last_undegraded":"2023-10-10T20:40:37.268252+0000","last_fullsized":"2023-10-10T20:40:37.268252+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:14:28.701104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254507+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:50:36.307293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263019+0000","last_change":"2023-10-10T20:40:37.263019+0000","last_active":"2023-10-10T20:40:37.263019+0000","last_peered":"2023-10-10T20:40:37.263019+0000","last_clean":"2023-10-10T20:40:37.263019+0000","last_became_active":"2023-10-10T20:40:37.262760+0000","last_became_peered":"2023-10-10T20:40:37.262760+0000","last_unstale":"2023-10-10T20:40:37.263019+0000","last_undegraded":"2023-10-10T20:40:37.263019+0000","last_fullsized":"2023-10-10T20:40:37.263019+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:12:55.235702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254807+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:26.505825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263048+0000","last_change":"2023-10-10T20:40:37.263048+0000","last_active":"2023-10-10T20:40:37.263048+0000","last_peered":"2023-10-10T20:40:37.263048+0000","last_clean":"2023-10-10T20:40:37.263048+0000","last_became_active":"2023-10-10T20:40:37.262792+0000","last_became_peered":"2023-10-10T20:40:37.262792+0000","last_unstale":"2023-10-10T20:40:37.263048+0000","last_undegraded":"2023-10-10T20:40:37.263048+0000","last_fullsized":"2023-10-10T20:40:37.263048+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:43:02.879509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249434+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:59:28.728877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.262612+0000","last_change":"2023-10-10T20:40:37.262612+0000","last_active":"2023-10-10T20:40:37.262612+0000","last_peered":"2023-10-10T20:40:37.262612+0000","last_clean":"2023-10-10T20:40:37.262612+0000","last_became_active":"2023-10-10T20:40:37.262298+0000","last_became_peered":"2023-10-10T20:40:37.262298+0000","last_unstale":"2023-10-10T20:40:37.262612+0000","last_undegraded":"2023-10-10T20:40:37.262612+0000","last_fullsized":"2023-10-10T20:40:37.262612+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:33:04.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.262303+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:03:14.859123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263207+0000","last_change":"2023-10-10T20:40:37.263207+0000","last_active":"2023-10-10T20:40:37.263207+0000","last_peered":"2023-10-10T20:40:37.263207+0000","last_clean":"2023-10-10T20:40:37.263207+0000","last_became_active":"2023-10-10T20:40:37.262945+0000","last_became_peered":"2023-10-10T20:40:37.262945+0000","last_unstale":"2023-10-10T20:40:37.263207+0000","last_undegraded":"2023-10-10T20:40:37.263207+0000","last_fullsized":"2023-10-10T20:40:37.263207+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:55:52.981786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253420+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:10:46.929737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.269620+0000","last_change":"2023-10-10T20:40:37.269620+0000","last_active":"2023-10-10T20:40:37.269620+0000","last_peered":"2023-10-10T20:40:37.269620+0000","last_clean":"2023-10-10T20:40:37.269620+0000","last_became_active":"2023-10-10T20:40:37.269347+0000","last_became_peered":"2023-10-10T20:40:37.269347+0000","last_unstale":"2023-10-10T20:40:37.269620+0000","last_undegraded":"2023-10-10T20:40:37.269620+0000","last_fullsized":"2023-10-10T20:40:37.269620+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:26:39.104435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257197+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:29:14.456681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264965+0000","last_change":"2023-10-10T20:40:37.264965+0000","last_active":"2023-10-10T20:40:37.264965+0000","last_peered":"2023-10-10T20:40:37.264965+0000","last_clean":"2023-10-10T20:40:37.264965+0000","last_became_active":"2023-10-10T20:40:37.264288+0000","last_became_peered":"2023-10-10T20:40:37.264288+0000","last_unstale":"2023-10-10T20:40:37.264965+0000","last_undegraded":"2023-10-10T20:40:37.264965+0000","last_fullsized":"2023-10-10T20:40:37.264965+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:09:19.490206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.256154+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:24:13.463965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264630+0000","last_change":"2023-10-10T20:40:37.264630+0000","last_active":"2023-10-10T20:40:37.264630+0000","last_peered":"2023-10-10T20:40:37.264630+0000","last_clean":"2023-10-10T20:40:37.264630+0000","last_became_active":"2023-10-10T20:40:37.264244+0000","last_became_peered":"2023-10-10T20:40:37.264244+0000","last_unstale":"2023-10-10T20:40:37.264630+0000","last_undegraded":"2023-10-10T20:40:37.264630+0000","last_fullsized":"2023-10-10T20:40:37.264630+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:06.742295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255716+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:26.505825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261554+0000","last_change":"2023-10-10T20:40:37.261554+0000","last_active":"2023-10-10T20:40:37.261554+0000","last_peered":"2023-10-10T20:40:37.261554+0000","last_clean":"2023-10-10T20:40:37.261554+0000","last_became_active":"2023-10-10T20:40:37.261298+0000","last_became_peered":"2023-10-10T20:40:37.261298+0000","last_unstale":"2023-10-10T20:40:37.261554+0000","last_undegraded":"2023-10-10T20:40:37.261554+0000","last_fullsized":"2023-10-10T20:40:37.261554+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:26.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249506+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:09:20.495043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261773+0000","last_change":"2023-10-10T20:40:37.261773+0000","last_active":"2023-10-10T20:40:37.261773+0000","last_peered":"2023-10-10T20:40:37.261773+0000","last_clean":"2023-10-10T20:40:37.261773+0000","last_became_active":"2023-10-10T20:40:37.261546+0000","last_became_peered":"2023-10-10T20:40:37.261546+0000","last_unstale":"2023-10-10T20:40:37.261773+0000","last_undegraded":"2023-10-10T20:40:37.261773+0000","last_fullsized":"2023-10-10T20:40:37.261773+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:24:15.190453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255131+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:13:53.110967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.258355+0000","last_change":"2023-10-10T20:40:37.258355+0000","last_active":"2023-10-10T20:40:37.258355+0000","last_peered":"2023-10-10T20:40:37.258355+0000","last_clean":"2023-10-10T20:40:37.258355+0000","last_became_active":"2023-10-10T20:40:37.258111+0000","last_became_peered":"2023-10-10T20:40:37.258111+0000","last_unstale":"2023-10-10T20:40:37.258355+0000","last_undegraded":"2023-10-10T20:40:37.258355+0000","last_fullsized":"2023-10-10T20:40:37.258355+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:56:53.783210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249907+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:29:39.101585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.266663+0000","last_change":"2023-10-10T20:40:37.266663+0000","last_active":"2023-10-10T20:40:37.266663+0000","last_peered":"2023-10-10T20:40:37.266663+0000","last_clean":"2023-10-10T20:40:37.266663+0000","last_became_active":"2023-10-10T20:40:37.266402+0000","last_became_peered":"2023-10-10T20:40:37.266402+0000","last_unstale":"2023-10-10T20:40:37.266663+0000","last_undegraded":"2023-10-10T20:40:37.266663+0000","last_fullsized":"2023-10-10T20:40:37.266663+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:43:14.351198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249725+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:56:54.788047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265713+0000","last_change":"2023-10-10T20:40:37.265713+0000","last_active":"2023-10-10T20:40:37.265713+0000","last_peered":"2023-10-10T20:40:37.265713+0000","last_clean":"2023-10-10T20:40:37.265713+0000","last_became_active":"2023-10-10T20:40:37.265287+0000","last_became_peered":"2023-10-10T20:40:37.265287+0000","last_unstale":"2023-10-10T20:40:37.265713+0000","last_undegraded":"2023-10-10T20:40:37.265713+0000","last_fullsized":"2023-10-10T20:40:37.265713+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:26.330192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249409+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:14.934213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264839+0000","last_change":"2023-10-10T20:40:37.264839+0000","last_active":"2023-10-10T20:40:37.264839+0000","last_peered":"2023-10-10T20:40:37.264839+0000","last_clean":"2023-10-10T20:40:37.264839+0000","last_became_active":"2023-10-10T20:40:37.264587+0000","last_became_peered":"2023-10-10T20:40:37.264587+0000","last_unstale":"2023-10-10T20:40:37.264839+0000","last_undegraded":"2023-10-10T20:40:37.264839+0000","last_fullsized":"2023-10-10T20:40:37.264839+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:24:35.687360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255974+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:32.259196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.260083+0000","last_change":"2023-10-10T20:40:37.260083+0000","last_active":"2023-10-10T20:40:37.260083+0000","last_peered":"2023-10-10T20:40:37.260083+0000","last_clean":"2023-10-10T20:40:37.260083+0000","last_became_active":"2023-10-10T20:40:37.259508+0000","last_became_peered":"2023-10-10T20:40:37.259508+0000","last_unstale":"2023-10-10T20:40:37.260083+0000","last_undegraded":"2023-10-10T20:40:37.260083+0000","last_fullsized":"2023-10-10T20:40:37.260083+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:13.929377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254091+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:33:05.914463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.266631+0000","last_change":"2023-10-10T20:40:37.266631+0000","last_active":"2023-10-10T20:40:37.266631+0000","last_peered":"2023-10-10T20:40:37.266631+0000","last_clean":"2023-10-10T20:40:37.266631+0000","last_became_active":"2023-10-10T20:40:37.266371+0000","last_became_peered":"2023-10-10T20:40:37.266371+0000","last_unstale":"2023-10-10T20:40:37.266631+0000","last_undegraded":"2023-10-10T20:40:37.266631+0000","last_fullsized":"2023-10-10T20:40:37.266631+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:36:44.776420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253499+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:29:51.950959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264623+0000","last_change":"2023-10-10T20:40:37.264623+0000","last_active":"2023-10-10T20:40:37.264623+0000","last_peered":"2023-10-10T20:40:37.264623+0000","last_clean":"2023-10-10T20:40:37.264623+0000","last_became_active":"2023-10-10T20:40:37.264322+0000","last_became_peered":"2023-10-10T20:40:37.264322+0000","last_unstale":"2023-10-10T20:40:37.264623+0000","last_undegraded":"2023-10-10T20:40:37.264623+0000","last_fullsized":"2023-10-10T20:40:37.264623+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:14:28.701104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254258+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:55:53.986623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265269+0000","last_change":"2023-10-10T20:40:37.265269+0000","last_active":"2023-10-10T20:40:37.265269+0000","last_peered":"2023-10-10T20:40:37.265269+0000","last_clean":"2023-10-10T20:40:37.265269+0000","last_became_active":"2023-10-10T20:40:37.264648+0000","last_became_peered":"2023-10-10T20:40:37.264648+0000","last_unstale":"2023-10-10T20:40:37.265269+0000","last_undegraded":"2023-10-10T20:40:37.265269+0000","last_fullsized":"2023-10-10T20:40:37.265269+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:14.826939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257410+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:39:12.325954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261923+0000","last_change":"2023-10-10T20:40:37.261923+0000","last_active":"2023-10-10T20:40:37.261923+0000","last_peered":"2023-10-10T20:40:37.261923+0000","last_clean":"2023-10-10T20:40:37.261923+0000","last_became_active":"2023-10-10T20:40:37.261631+0000","last_became_peered":"2023-10-10T20:40:37.261631+0000","last_unstale":"2023-10-10T20:40:37.261923+0000","last_undegraded":"2023-10-10T20:40:37.261923+0000","last_fullsized":"2023-10-10T20:40:37.261923+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:16:28.059016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.251150+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:38:00.681114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263329+0000","last_change":"2023-10-10T20:40:37.263329+0000","last_active":"2023-10-10T20:40:37.263329+0000","last_peered":"2023-10-10T20:40:37.263329+0000","last_clean":"2023-10-10T20:40:37.263329+0000","last_became_active":"2023-10-10T20:40:37.263072+0000","last_became_peered":"2023-10-10T20:40:37.263072+0000","last_unstale":"2023-10-10T20:40:37.263329+0000","last_undegraded":"2023-10-10T20:40:37.263329+0000","last_fullsized":"2023-10-10T20:40:37.263329+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:31:56.472091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257903+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:56:54.788047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.262448+0000","last_change":"2023-10-10T20:40:37.262448+0000","last_active":"2023-10-10T20:40:37.262448+0000","last_peered":"2023-10-10T20:40:37.262448+0000","last_clean":"2023-10-10T20:40:37.262448+0000","last_became_active":"2023-10-10T20:40:37.262193+0000","last_became_peered":"2023-10-10T20:40:37.262193+0000","last_unstale":"2023-10-10T20:40:37.262448+0000","last_undegraded":"2023-10-10T20:40:37.262448+0000","last_fullsized":"2023-10-10T20:40:37.262448+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:05.415898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.248907+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:24:16.195290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265730+0000","last_change":"2023-10-10T20:40:37.265730+0000","last_active":"2023-10-10T20:40:37.265730+0000","last_peered":"2023-10-10T20:40:37.265730+0000","last_clean":"2023-10-10T20:40:37.265730+0000","last_became_active":"2023-10-10T20:40:37.264840+0000","last_became_peered":"2023-10-10T20:40:37.264840+0000","last_unstale":"2023-10-10T20:40:37.265730+0000","last_undegraded":"2023-10-10T20:40:37.265730+0000","last_fullsized":"2023-10-10T20:40:37.265730+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:39:11.321117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255331+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:55:53.986623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261516+0000","last_change":"2023-10-10T20:40:37.261516+0000","last_active":"2023-10-10T20:40:37.261516+0000","last_peered":"2023-10-10T20:40:37.261516+0000","last_clean":"2023-10-10T20:40:37.261516+0000","last_became_active":"2023-10-10T20:40:37.261250+0000","last_became_peered":"2023-10-10T20:40:37.261250+0000","last_unstale":"2023-10-10T20:40:37.261516+0000","last_undegraded":"2023-10-10T20:40:37.261516+0000","last_fullsized":"2023-10-10T20:40:37.261516+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:37:03.617984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252774+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:20:35.134043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263365+0000","last_change":"2023-10-10T20:40:37.263365+0000","last_active":"2023-10-10T20:40:37.263365+0000","last_peered":"2023-10-10T20:40:37.263365+0000","last_clean":"2023-10-10T20:40:37.263365+0000","last_became_active":"2023-10-10T20:40:37.263032+0000","last_became_peered":"2023-10-10T20:40:37.263032+0000","last_unstale":"2023-10-10T20:40:37.263365+0000","last_undegraded":"2023-10-10T20:40:37.263365+0000","last_fullsized":"2023-10-10T20:40:37.263365+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:06.817315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249799+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:23.463451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264758+0000","last_change":"2023-10-10T20:40:37.264758+0000","last_active":"2023-10-10T20:40:37.264758+0000","last_peered":"2023-10-10T20:40:37.264758+0000","last_clean":"2023-10-10T20:40:37.264758+0000","last_became_active":"2023-10-10T20:40:37.264496+0000","last_became_peered":"2023-10-10T20:40:37.264496+0000","last_unstale":"2023-10-10T20:40:37.264758+0000","last_undegraded":"2023-10-10T20:40:37.264758+0000","last_fullsized":"2023-10-10T20:40:37.264758+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:43:18.734440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255210+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:03:14.859123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261011+0000","last_change":"2023-10-10T20:40:37.261011+0000","last_active":"2023-10-10T20:40:37.261011+0000","last_peered":"2023-10-10T20:40:37.261011+0000","last_clean":"2023-10-10T20:40:37.261011+0000","last_became_active":"2023-10-10T20:40:37.260760+0000","last_became_peered":"2023-10-10T20:40:37.260760+0000","last_unstale":"2023-10-10T20:40:37.261011+0000","last_undegraded":"2023-10-10T20:40:37.261011+0000","last_fullsized":"2023-10-10T20:40:37.261011+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:42.139421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261667+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:32:45.025265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.267680+0000","last_change":"2023-10-10T20:40:37.267680+0000","last_active":"2023-10-10T20:40:37.267680+0000","last_peered":"2023-10-10T20:40:37.267680+0000","last_clean":"2023-10-10T20:40:37.267680+0000","last_became_active":"2023-10-10T20:40:37.267373+0000","last_became_peered":"2023-10-10T20:40:37.267373+0000","last_unstale":"2023-10-10T20:40:37.267680+0000","last_undegraded":"2023-10-10T20:40:37.267680+0000","last_fullsized":"2023-10-10T20:40:37.267680+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:30:44.468056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261219+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:27.335029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265941+0000","last_change":"2023-10-10T20:40:37.265941+0000","last_active":"2023-10-10T20:40:37.265941+0000","last_peered":"2023-10-10T20:40:37.265941+0000","last_clean":"2023-10-10T20:40:37.265941+0000","last_became_active":"2023-10-10T20:40:37.265532+0000","last_became_peered":"2023-10-10T20:40:37.265532+0000","last_unstale":"2023-10-10T20:40:37.265941+0000","last_undegraded":"2023-10-10T20:40:37.265941+0000","last_fullsized":"2023-10-10T20:40:37.265941+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:02.961417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257410+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:14.934213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264545+0000","last_change":"2023-10-10T20:40:37.264545+0000","last_active":"2023-10-10T20:40:37.264545+0000","last_peered":"2023-10-10T20:40:37.264545+0000","last_clean":"2023-10-10T20:40:37.264545+0000","last_became_active":"2023-10-10T20:40:37.264300+0000","last_became_peered":"2023-10-10T20:40:37.264300+0000","last_unstale":"2023-10-10T20:40:37.264545+0000","last_undegraded":"2023-10-10T20:40:37.264545+0000","last_fullsized":"2023-10-10T20:40:37.264545+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:03:53.037877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252229+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:43:08.406828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265540+0000","last_change":"2023-10-10T20:40:37.265540+0000","last_active":"2023-10-10T20:40:37.265540+0000","last_peered":"2023-10-10T20:40:37.265540+0000","last_clean":"2023-10-10T20:40:37.265540+0000","last_became_active":"2023-10-10T20:40:37.264555+0000","last_became_peered":"2023-10-10T20:40:37.264555+0000","last_unstale":"2023-10-10T20:40:37.265540+0000","last_undegraded":"2023-10-10T20:40:37.265540+0000","last_fullsized":"2023-10-10T20:40:37.265540+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:37:03.617984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.248868+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:39:12.325954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265098+0000","last_change":"2023-10-10T20:40:37.265098+0000","last_active":"2023-10-10T20:40:37.265098+0000","last_peered":"2023-10-10T20:40:37.265098+0000","last_clean":"2023-10-10T20:40:37.265098+0000","last_became_active":"2023-10-10T20:40:37.264742+0000","last_became_peered":"2023-10-10T20:40:37.264742+0000","last_unstale":"2023-10-10T20:40:37.265098+0000","last_undegraded":"2023-10-10T20:40:37.265098+0000","last_fullsized":"2023-10-10T20:40:37.265098+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:42:00.401594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":241,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:40:36.243776+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:40:36.243776+0000","last_peered":"2023-10-10T20:40:36.243776+0000","last_clean":"2023-10-10T20:40:36.243776+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:40:36.243776+0000","last_undegraded":"2023-10-10T20:40:36.243776+0000","last_fullsized":"2023-10-10T20:40:36.243776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303156,"kb_used_data":1772,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068684,"statfs":{"total":96636764160,"available":96326332416,"internally_reserved":0,"allocated":1814528,"data_stored":1114976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.60499999999999998,"15min":0.84999999999999998},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.747,"5min":6.8630000000000004,"15min":6.8630000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.56299999999999994,"15min":0.63800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.629,"5min":4.1959999999999997,"15min":4.1959999999999997},"last":0.40699999999999997}]},{"osd":1,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.65300000000000002,"15min":1.0489999999999999},"min":{"1min":0.39900000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.78800000000000003,"5min":10.1,"15min":10.1},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.61299999999999999,"15min":0.749},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66400000000000003,"5min":6.806,"15min":6.806},"last":0.44900000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.64000000000000001,"15min":0.80200000000000005},"min":{"1min":0.41599999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":6.9340000000000002,"15min":6.9340000000000002},"last":0.498},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59499999999999997,"15min":0.75900000000000001},"min":{"1min":0.38800000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73199999999999998,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61799999999999999,"15min":0.77800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70899999999999996,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.68100000000000005},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71899999999999997,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.48099999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.34599999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.626,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.374,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78400000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.38900000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.73299999999999998,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72699999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.372}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58899999999999997,"15min":0.57199999999999995},"min":{"1min":0.48699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77100000000000002,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.434},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.46800000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75600000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.50600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574927,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302576,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069264,"statfs":{"total":96636764160,"available":96326926336,"internally_reserved":0,"allocated":1220608,"data_stored":524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55700000000000005,"15min":0.68899999999999995},"min":{"1min":0.40799999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.73099999999999998,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":0.627},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57899999999999996,"15min":0.73299999999999998},"min":{"1min":0.372,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70799999999999996,"5min":7.8090000000000002,"15min":7.8090000000000002},"last":0.64300000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.72899999999999998},"min":{"1min":0.39700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71899999999999997,"5min":7.867,"15min":7.867},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.64200000000000002,"15min":0.92100000000000004},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75700000000000001,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.60099999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.747,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.63400000000000001,"15min":0.874},"min":{"1min":0.48399999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.71199999999999997,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":0.58499999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.628},"min":{"1min":0.439,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79000000000000004,"5min":4.8620000000000001,"15min":4.8620000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.73499999999999999},"min":{"1min":0.47199999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.755,"5min":7.9009999999999998,"15min":7.9009999999999998},"last":0.56499999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.504,"15min":0.46400000000000002},"min":{"1min":0.42799999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71799999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.56499999999999995},"min":{"1min":0.54200000000000004,"5min":0.37,"15min":0.37},"max":{"1min":0.69899999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72099999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51400000000000001,"15min":0.48599999999999999},"min":{"1min":0.40400000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55200000000000005,"15min":0.52600000000000002},"min":{"1min":0.47799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.70299999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.42799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74399999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56899999999999995,"15min":0.54100000000000004},"min":{"1min":0.52700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.747,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.65600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302576,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069264,"statfs":{"total":96636764160,"available":96326926336,"internally_reserved":0,"allocated":1220608,"data_stored":524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.36399999999999999,"15min":0.49099999999999999},"min":{"1min":0.219,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.56799999999999995,"5min":5.0380000000000003,"15min":5.0380000000000003},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33100000000000002,"15min":0.36499999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.47299999999999998,"5min":1.194,"15min":1.194},"last":0.318}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.36799999999999999},"min":{"1min":0.27800000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.51400000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.52000000000000002},"min":{"1min":0.26000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52500000000000002,"5min":5.0919999999999996,"15min":5.0919999999999996},"last":0.29399999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.40799999999999997,"15min":0.54300000000000004},"min":{"1min":0.28000000000000003,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.59899999999999998,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38100000000000001,"15min":0.41299999999999998},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55500000000000005,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.26800000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.40000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92400000000000004,"5min":1.254,"15min":1.254},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60599999999999998,"15min":0.71899999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96499999999999997,"5min":4.9850000000000003,"15min":4.9850000000000003},"last":0.56899999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97599999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.42899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.878,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.55300000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67600000000000005,"15min":0.86699999999999999},"min":{"1min":0.46500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.995,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.40699999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85799999999999998,"5min":1.171,"15min":1.171},"last":0.67300000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60099999999999998,"15min":0.62},"min":{"1min":0.443,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.004,"5min":1.214,"15min":1.214},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63700000000000001,"15min":0.73699999999999999},"min":{"1min":0.41999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90500000000000003,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.68700000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303140,"kb_used_data":1756,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068700,"statfs":{"total":96636764160,"available":96326348800,"internally_reserved":0,"allocated":1798144,"data_stored":1107752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.377,"15min":0.55100000000000005},"min":{"1min":0.20699999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.51600000000000001,"5min":7.9249999999999998,"15min":7.9249999999999998},"last":0.255},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.374,"15min":0.52400000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45800000000000002,"5min":6.6299999999999999,"15min":6.6299999999999999},"last":0.33300000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.39000000000000001,"15min":0.57399999999999995},"min":{"1min":0.23100000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.47499999999999998,"5min":8.2509999999999994,"15min":8.2509999999999994},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39800000000000002,"15min":0.54800000000000004},"min":{"1min":0.245,"5min":0.222,"15min":0.222},"max":{"1min":0.52400000000000002,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.372}]},{"osd":3,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.40500000000000003,"15min":0.57699999999999996},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.45400000000000001,"15min":0.63400000000000001},"min":{"1min":0.31900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57899999999999996,"5min":7.8929999999999998,"15min":7.8929999999999998},"last":0.32200000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.61099999999999999,"15min":0.73199999999999998},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78000000000000003,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.63600000000000001,"15min":0.79600000000000004},"min":{"1min":0.45100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.95399999999999996,"5min":7.6920000000000002,"15min":7.6920000000000002},"last":0.49399999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.77500000000000002},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92400000000000004,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.64300000000000002,"15min":0.79900000000000004},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91400000000000003,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":0.47999999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67000000000000004,"15min":0.83999999999999997},"min":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94499999999999995,"5min":7.8360000000000003,"15min":7.8360000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66400000000000003,"15min":0.77900000000000003},"min":{"1min":0.49199999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90300000000000002,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.56699999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64000000000000001,"15min":0.76900000000000002},"min":{"1min":0.45900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93300000000000005,"5min":6.7160000000000002,"15min":6.7160000000000002},"last":0.503},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66100000000000003,"15min":0.80900000000000005},"min":{"1min":0.44600000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.878,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.45300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302576,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069264,"statfs":{"total":96636764160,"available":96326926336,"internally_reserved":0,"allocated":1220608,"data_stored":524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.31900000000000001,"15min":0.32800000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.59699999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.33600000000000002,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45400000000000001,"5min":0.746,"15min":0.746},"last":0.25800000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.45400000000000001,"15min":0.77100000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.63,"5min":11.967000000000001,"15min":11.967000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39000000000000001,"15min":0.498},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52700000000000002,"5min":4.1100000000000003,"15min":4.1100000000000003},"last":0.309}]},{"osd":3,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.41199999999999998,"15min":0.57899999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48799999999999999,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.50900000000000001,"15min":0.82199999999999995},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65900000000000003,"5min":11.935,"15min":11.935},"last":0.41199999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.80100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.58999999999999997},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46200000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.33400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83599999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.58399999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81599999999999995,"5min":1.052,"15min":1.052},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64500000000000002,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85999999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.59799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574927,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302564,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069276,"statfs":{"total":96636764160,"available":96326938624,"internally_reserved":0,"allocated":1208320,"data_stored":518345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.31,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.311},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.31900000000000001},"min":{"1min":0.25,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.32900000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.23499999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38600000000000001,"15min":0.377},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.57099999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.34999999999999998},"min":{"1min":0.23699999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.35699999999999998,"15min":0.36199999999999999},"min":{"1min":0.26300000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.36599999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69999999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.82299999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64500000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.35899999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55900000000000005,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.76900000000000002,"5min":1.01,"15min":1.01},"last":0.56699999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.42099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85699999999999998,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.441,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.496,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.88400000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298468,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073372,"statfs":{"total":96636764160,"available":96331132928,"internally_reserved":0,"allocated":1208320,"data_stored":518345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58199999999999996,"5min":0.747,"15min":0.747},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.59799999999999998,"15min":0.79200000000000004},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69699999999999995,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.48499999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.60299999999999998,"15min":0.79900000000000004},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60299999999999998,"5min":7.7160000000000002,"15min":7.7160000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.58599999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68100000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63600000000000001,"15min":0.81799999999999995},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.72799999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.58899999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61399999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.502,"15min":0.48399999999999999},"min":{"1min":0.34599999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.38400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77300000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.60399999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.41699999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72099999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.52600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.70599999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.58199999999999996},"min":{"1min":0.44700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298480,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073360,"statfs":{"total":96636764160,"available":96331120640,"internally_reserved":0,"allocated":1220608,"data_stored":524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.307,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63600000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.34300000000000003,"5min":0.215,"15min":0.215},"max":{"1min":0.70399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.46600000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.56200000000000006,"15min":0.65300000000000002},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55300000000000005,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68200000000000005,"5min":5.3109999999999999,"15min":5.3109999999999999},"last":0.55700000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56899999999999995,"15min":0.66100000000000003},"min":{"1min":0.32900000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64700000000000002,"5min":5.2960000000000003,"15min":5.2960000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72399999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.47799999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59499999999999997,"15min":0.69099999999999995},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70399999999999996,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":0.499},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59699999999999998,"15min":0.70399999999999996},"min":{"1min":0.40300000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70599999999999996,"5min":5.2850000000000001,"15min":5.2850000000000001},"last":0.56799999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48299999999999998,"15min":0.497},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.63800000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.311},{"interface":"front","average":{"1min":0.505,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65000000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.34499999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.65100000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71299999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.36199999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.33000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72699999999999998,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.38100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:40:42.318 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:40:42.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.464+0000 7f6e461c3700 1 Processor -- start 2023-10-10T20:40:42.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.464+0000 7f6e461c3700 1 -- start start 2023-10-10T20:40:42.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.465+0000 7f6e461c3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6e40159890 0x7f6e4015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:42.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.465+0000 7f6e461c3700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6e40159c60 0x7f6e4015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:42.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.465+0000 7f6e461c3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e4015a160 con 0x7f6e4015db10 2023-10-10T20:40:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.465+0000 7f6e461c3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4015df50 con 0x7f6e40159890 2023-10-10T20:40:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.465+0000 7f6e461c3700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e400d13a0 con 0x7f6e40159c60 2023-10-10T20:40:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.465+0000 7f6e3f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6e40159890 0x7f6e4015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.465+0000 7f6e3ffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6e4015db10 0x7f6e40069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49192/0 (socket says 172.21.15.89:49192) 2023-10-10T20:40:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.465+0000 7f6e3ffff700 1 -- 172.21.15.89:0/1279244107 learned_addr learned my addr 172.21.15.89:0/1279244107 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.465+0000 7f6e3effd700 1 --2- 172.21.15.89:0/1279244107 >> v2:172.21.15.111:3300/0 conn(0x7f6e40159c60 0x7f6e4015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.465+0000 7f6e3f7fe700 1 -- 172.21.15.89:0/1279244107 >> v2:172.21.15.111:3300/0 conn(0x7f6e40159c60 msgr2=0x7f6e4015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.465+0000 7f6e3f7fe700 1 --2- 172.21.15.89:0/1279244107 >> v2:172.21.15.111:3300/0 conn(0x7f6e40159c60 0x7f6e4015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.465+0000 7f6e3f7fe700 1 -- 172.21.15.89:0/1279244107 >> v1:172.21.15.89:6789/0 conn(0x7f6e4015db10 legacy=0x7f6e40069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:42.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.465+0000 7f6e3e7fc700 1 -- 172.21.15.89:0/1279244107 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179448710 0 0) 0x7f6e4015a160 con 0x7f6e4015db10 2023-10-10T20:40:42.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.465+0000 7f6e3f7fe700 1 -- 172.21.15.89:0/1279244107 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4015e140 con 0x7f6e40159890 2023-10-10T20:40:42.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.465+0000 7f6e3f7fe700 1 --2- 172.21.15.89:0/1279244107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e40159890 0x7f6e4015abd0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f6e400b67f0 tx=0x7f6e340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06b3a5f9de12461 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:42.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.465+0000 7f6e3e7fc700 1 -- 172.21.15.89:0/1279244107 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3400b6b0 con 0x7f6e40159890 2023-10-10T20:40:42.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.465+0000 7f6e3e7fc700 1 -- 172.21.15.89:0/1279244107 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3400b810 con 0x7f6e40159890 2023-10-10T20:40:42.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.465+0000 7f6e3e7fc700 1 -- 172.21.15.89:0/1279244107 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3400bbb0 con 0x7f6e40159890 2023-10-10T20:40:42.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.466+0000 7f6e461c3700 1 -- 172.21.15.89:0/1279244107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e40159890 msgr2=0x7f6e4015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:42.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.466+0000 7f6e461c3700 1 --2- 172.21.15.89:0/1279244107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e40159890 0x7f6e4015abd0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f6e400b67f0 tx=0x7f6e340091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.466+0000 7f6e461c3700 1 -- 172.21.15.89:0/1279244107 shutdown_connections 2023-10-10T20:40:42.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.466+0000 7f6e461c3700 1 --2- 172.21.15.89:0/1279244107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e40159890 0x7f6e4015abd0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.466+0000 7f6e461c3700 1 --2- 172.21.15.89:0/1279244107 >> v2:172.21.15.111:3300/0 conn(0x7f6e40159c60 0x7f6e4015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.466+0000 7f6e461c3700 1 -- 172.21.15.89:0/1279244107 >> 172.21.15.89:0/1279244107 conn(0x7f6e40068e20 msgr2=0x7f6e400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.466+0000 7f6e461c3700 1 -- 172.21.15.89:0/1279244107 shutdown_connections 2023-10-10T20:40:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.466+0000 7f6e461c3700 1 -- 172.21.15.89:0/1279244107 wait complete. 2023-10-10T20:40:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.467+0000 7f6e461c3700 1 Processor -- start 2023-10-10T20:40:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.467+0000 7f6e461c3700 1 -- start start 2023-10-10T20:40:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.468+0000 7f6e461c3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e40159890 0x7f6e400ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.468+0000 7f6e461c3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e40159c60 0x7f6e4016a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.468+0000 7f6e461c3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e4015db10 0x7f6e40152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.468+0000 7f6e461c3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e400cf3c0 con 0x7f6e40159c60 2023-10-10T20:40:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.468+0000 7f6e461c3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e400cf6a0 con 0x7f6e40159890 2023-10-10T20:40:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.468+0000 7f6e461c3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e400cf980 con 0x7f6e4015db10 2023-10-10T20:40:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.468+0000 7f6e3effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e40159c60 0x7f6e4016a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.468+0000 7f6e3effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e40159c60 0x7f6e4016a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38996/0 (socket says 172.21.15.89:38996) 2023-10-10T20:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.468+0000 7f6e3effd700 1 -- 172.21.15.89:0/4037757262 learned_addr learned my addr 172.21.15.89:0/4037757262 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.468+0000 7f6e3f7fe700 1 --2- 172.21.15.89:0/4037757262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e40159890 0x7f6e400ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.468+0000 7f6e3ffff700 1 --2- 172.21.15.89:0/4037757262 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e4015db10 0x7f6e40152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.468+0000 7f6e3effd700 1 -- 172.21.15.89:0/4037757262 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e4015db10 msgr2=0x7f6e40152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.468+0000 7f6e3effd700 1 --2- 172.21.15.89:0/4037757262 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e4015db10 0x7f6e40152580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.468+0000 7f6e3effd700 1 -- 172.21.15.89:0/4037757262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e40159890 msgr2=0x7f6e400ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.468+0000 7f6e3effd700 1 --2- 172.21.15.89:0/4037757262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e40159890 0x7f6e400ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.468+0000 7f6e3effd700 1 -- 172.21.15.89:0/4037757262 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e34009420 con 0x7f6e40159c60 2023-10-10T20:40:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.469+0000 7f6e3effd700 1 --2- 172.21.15.89:0/4037757262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e40159c60 0x7f6e4016a1d0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000c7d0 tx=0x7f6e30008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.469+0000 7f6e277fe700 1 -- 172.21.15.89:0/4037757262 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e30011600 con 0x7f6e40159c60 2023-10-10T20:40:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.469+0000 7f6e277fe700 1 -- 172.21.15.89:0/4037757262 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e30011760 con 0x7f6e40159c60 2023-10-10T20:40:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.469+0000 7f6e461c3700 1 -- 172.21.15.89:0/4037757262 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e400cfc60 con 0x7f6e40159c60 2023-10-10T20:40:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.469+0000 7f6e277fe700 1 -- 172.21.15.89:0/4037757262 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e30011b20 con 0x7f6e40159c60 2023-10-10T20:40:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.469+0000 7f6e461c3700 1 -- 172.21.15.89:0/4037757262 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4016a710 con 0x7f6e40159c60 2023-10-10T20:40:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.470+0000 7f6e461c3700 1 -- 172.21.15.89:0/4037757262 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e400b3620 con 0x7f6e40159c60 2023-10-10T20:40:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.470+0000 7f6e277fe700 1 -- 172.21.15.89:0/4037757262 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f6e300193f0 con 0x7f6e40159c60 2023-10-10T20:40:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.470+0000 7f6e277fe700 1 --2- 172.21.15.89:0/4037757262 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6e2c067290 0x7f6e2c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.470+0000 7f6e277fe700 1 -- 172.21.15.89:0/4037757262 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6e30084fd0 con 0x7f6e40159c60 2023-10-10T20:40:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.470+0000 7f6e3f7fe700 1 --2- 172.21.15.89:0/4037757262 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6e2c067290 0x7f6e2c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.471+0000 7f6e3f7fe700 1 --2- 172.21.15.89:0/4037757262 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6e2c067290 0x7f6e2c069740 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6e340093f0 tx=0x7f6e34017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.474+0000 7f6e277fe700 1 -- 172.21.15.89:0/4037757262 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e400b3620 con 0x7f6e40159c60 2023-10-10T20:40:42.565 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:40:42.568 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:40:42.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.657+0000 7f6e461c3700 1 -- 172.21.15.89:0/4037757262 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6e400b3620 con 0x7f6e2c067290 2023-10-10T20:40:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.664+0000 7f6e277fe700 1 -- 172.21.15.89:0/4037757262 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286966 (secure 0 0 0) 0x7f6e400b3620 con 0x7f6e2c067290 2023-10-10T20:40:42.670 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:42.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.668+0000 7f6e461c3700 1 -- 172.21.15.89:0/4037757262 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6e2c067290 msgr2=0x7f6e2c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:42.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.668+0000 7f6e461c3700 1 --2- 172.21.15.89:0/4037757262 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6e2c067290 0x7f6e2c069740 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6e340093f0 tx=0x7f6e34017000 comp rx=0 tx=0).stop 2023-10-10T20:40:42.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.668+0000 7f6e461c3700 1 -- 172.21.15.89:0/4037757262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e40159c60 msgr2=0x7f6e4016a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:42.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.668+0000 7f6e461c3700 1 --2- 172.21.15.89:0/4037757262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e40159c60 0x7f6e4016a1d0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000c7d0 tx=0x7f6e30008040 comp rx=0 tx=0).stop 2023-10-10T20:40:42.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.668+0000 7f6e461c3700 1 -- 172.21.15.89:0/4037757262 shutdown_connections 2023-10-10T20:40:42.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.668+0000 7f6e461c3700 1 --2- 172.21.15.89:0/4037757262 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e4015db10 0x7f6e40152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.668+0000 7f6e461c3700 1 --2- 172.21.15.89:0/4037757262 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6e2c067290 0x7f6e2c069740 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.668+0000 7f6e461c3700 1 --2- 172.21.15.89:0/4037757262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e40159890 0x7f6e400ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.668+0000 7f6e461c3700 1 --2- 172.21.15.89:0/4037757262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e40159c60 0x7f6e4016a1d0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.668+0000 7f6e461c3700 1 -- 172.21.15.89:0/4037757262 >> 172.21.15.89:0/4037757262 conn(0x7f6e40068e20 msgr2=0x7f6e401584a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:42.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.668+0000 7f6e461c3700 1 -- 172.21.15.89:0/4037757262 shutdown_connections 2023-10-10T20:40:42.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.668+0000 7f6e461c3700 1 -- 172.21.15.89:0/4037757262 wait complete. 2023-10-10T20:40:42.675 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:40:42.682 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2023-10-10T20:40:40.725402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2413536,"kb_used_data":10656,"kb_used_omap":0,"kb_used_meta":2402816,"kb_avail":752561184,"statfs":{"total":773094113280,"available":770622652416,"internally_reserved":0,"allocated":10911744,"data_stored":5357850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2460483584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.486409"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253558+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:25.230014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.260877+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:53:07.253745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255734+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:43.144258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.250640+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:28:37.819892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255881+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:36:00.607814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252745+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:15.831776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257694+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:45:44.964754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.258387+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:59:28.728877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253182+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:46:05.714540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252306+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:37:30.151565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261779+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:29:51.950959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.250202+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:39:38.169839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253730+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:03:14.859123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.258385+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:29:39.101585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254614+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:09:46.330074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254027+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:36.297633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.263244+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:43.144258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255797+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:27.914463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255314+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:27.914463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261622+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:14:29.705941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.263327+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:37:04.622821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.250382+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:59:28.728877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.263024+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:27.914463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.251359+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:39:38.169839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261486+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:07.822152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.251605+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:54.041748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254573+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:30:45.472893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.250590+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:23.463451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.262806+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:26.505825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253631+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:09:46.330074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252631+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:58:16.915031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252495+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:10:45.408128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253140+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:42:01.406431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265989+0000","last_change":"2023-10-10T20:40:37.265989+0000","last_active":"2023-10-10T20:40:37.265989+0000","last_peered":"2023-10-10T20:40:37.265989+0000","last_clean":"2023-10-10T20:40:37.265989+0000","last_became_active":"2023-10-10T20:40:37.265518+0000","last_became_peered":"2023-10-10T20:40:37.265518+0000","last_unstale":"2023-10-10T20:40:37.265989+0000","last_undegraded":"2023-10-10T20:40:37.265989+0000","last_fullsized":"2023-10-10T20:40:37.265989+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:10:44.403291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.256386+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:43.144258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.268252+0000","last_change":"2023-10-10T20:40:37.268252+0000","last_active":"2023-10-10T20:40:37.268252+0000","last_peered":"2023-10-10T20:40:37.268252+0000","last_clean":"2023-10-10T20:40:37.268252+0000","last_became_active":"2023-10-10T20:40:37.267982+0000","last_became_peered":"2023-10-10T20:40:37.267982+0000","last_unstale":"2023-10-10T20:40:37.268252+0000","last_undegraded":"2023-10-10T20:40:37.268252+0000","last_fullsized":"2023-10-10T20:40:37.268252+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:14:28.701104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254507+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:50:36.307293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263019+0000","last_change":"2023-10-10T20:40:37.263019+0000","last_active":"2023-10-10T20:40:37.263019+0000","last_peered":"2023-10-10T20:40:37.263019+0000","last_clean":"2023-10-10T20:40:37.263019+0000","last_became_active":"2023-10-10T20:40:37.262760+0000","last_became_peered":"2023-10-10T20:40:37.262760+0000","last_unstale":"2023-10-10T20:40:37.263019+0000","last_undegraded":"2023-10-10T20:40:37.263019+0000","last_fullsized":"2023-10-10T20:40:37.263019+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:12:55.235702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254807+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:26.505825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263048+0000","last_change":"2023-10-10T20:40:37.263048+0000","last_active":"2023-10-10T20:40:37.263048+0000","last_peered":"2023-10-10T20:40:37.263048+0000","last_clean":"2023-10-10T20:40:37.263048+0000","last_became_active":"2023-10-10T20:40:37.262792+0000","last_became_peered":"2023-10-10T20:40:37.262792+0000","last_unstale":"2023-10-10T20:40:37.263048+0000","last_undegraded":"2023-10-10T20:40:37.263048+0000","last_fullsized":"2023-10-10T20:40:37.263048+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:43:02.879509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249434+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:59:28.728877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.262612+0000","last_change":"2023-10-10T20:40:37.262612+0000","last_active":"2023-10-10T20:40:37.262612+0000","last_peered":"2023-10-10T20:40:37.262612+0000","last_clean":"2023-10-10T20:40:37.262612+0000","last_became_active":"2023-10-10T20:40:37.262298+0000","last_became_peered":"2023-10-10T20:40:37.262298+0000","last_unstale":"2023-10-10T20:40:37.262612+0000","last_undegraded":"2023-10-10T20:40:37.262612+0000","last_fullsized":"2023-10-10T20:40:37.262612+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:33:04.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.262303+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:03:14.859123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263207+0000","last_change":"2023-10-10T20:40:37.263207+0000","last_active":"2023-10-10T20:40:37.263207+0000","last_peered":"2023-10-10T20:40:37.263207+0000","last_clean":"2023-10-10T20:40:37.263207+0000","last_became_active":"2023-10-10T20:40:37.262945+0000","last_became_peered":"2023-10-10T20:40:37.262945+0000","last_unstale":"2023-10-10T20:40:37.263207+0000","last_undegraded":"2023-10-10T20:40:37.263207+0000","last_fullsized":"2023-10-10T20:40:37.263207+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:55:52.981786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253420+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:10:46.929737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.269620+0000","last_change":"2023-10-10T20:40:37.269620+0000","last_active":"2023-10-10T20:40:37.269620+0000","last_peered":"2023-10-10T20:40:37.269620+0000","last_clean":"2023-10-10T20:40:37.269620+0000","last_became_active":"2023-10-10T20:40:37.269347+0000","last_became_peered":"2023-10-10T20:40:37.269347+0000","last_unstale":"2023-10-10T20:40:37.269620+0000","last_undegraded":"2023-10-10T20:40:37.269620+0000","last_fullsized":"2023-10-10T20:40:37.269620+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:26:39.104435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257197+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:29:14.456681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264965+0000","last_change":"2023-10-10T20:40:37.264965+0000","last_active":"2023-10-10T20:40:37.264965+0000","last_peered":"2023-10-10T20:40:37.264965+0000","last_clean":"2023-10-10T20:40:37.264965+0000","last_became_active":"2023-10-10T20:40:37.264288+0000","last_became_peered":"2023-10-10T20:40:37.264288+0000","last_unstale":"2023-10-10T20:40:37.264965+0000","last_undegraded":"2023-10-10T20:40:37.264965+0000","last_fullsized":"2023-10-10T20:40:37.264965+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:09:19.490206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.256154+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:24:13.463965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264630+0000","last_change":"2023-10-10T20:40:37.264630+0000","last_active":"2023-10-10T20:40:37.264630+0000","last_peered":"2023-10-10T20:40:37.264630+0000","last_clean":"2023-10-10T20:40:37.264630+0000","last_became_active":"2023-10-10T20:40:37.264244+0000","last_became_peered":"2023-10-10T20:40:37.264244+0000","last_unstale":"2023-10-10T20:40:37.264630+0000","last_undegraded":"2023-10-10T20:40:37.264630+0000","last_fullsized":"2023-10-10T20:40:37.264630+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:06.742295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255716+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:26.505825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261554+0000","last_change":"2023-10-10T20:40:37.261554+0000","last_active":"2023-10-10T20:40:37.261554+0000","last_peered":"2023-10-10T20:40:37.261554+0000","last_clean":"2023-10-10T20:40:37.261554+0000","last_became_active":"2023-10-10T20:40:37.261298+0000","last_became_peered":"2023-10-10T20:40:37.261298+0000","last_unstale":"2023-10-10T20:40:37.261554+0000","last_undegraded":"2023-10-10T20:40:37.261554+0000","last_fullsized":"2023-10-10T20:40:37.261554+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:26.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249506+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:09:20.495043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261773+0000","last_change":"2023-10-10T20:40:37.261773+0000","last_active":"2023-10-10T20:40:37.261773+0000","last_peered":"2023-10-10T20:40:37.261773+0000","last_clean":"2023-10-10T20:40:37.261773+0000","last_became_active":"2023-10-10T20:40:37.261546+0000","last_became_peered":"2023-10-10T20:40:37.261546+0000","last_unstale":"2023-10-10T20:40:37.261773+0000","last_undegraded":"2023-10-10T20:40:37.261773+0000","last_fullsized":"2023-10-10T20:40:37.261773+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:24:15.190453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255131+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:13:53.110967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.258355+0000","last_change":"2023-10-10T20:40:37.258355+0000","last_active":"2023-10-10T20:40:37.258355+0000","last_peered":"2023-10-10T20:40:37.258355+0000","last_clean":"2023-10-10T20:40:37.258355+0000","last_became_active":"2023-10-10T20:40:37.258111+0000","last_became_peered":"2023-10-10T20:40:37.258111+0000","last_unstale":"2023-10-10T20:40:37.258355+0000","last_undegraded":"2023-10-10T20:40:37.258355+0000","last_fullsized":"2023-10-10T20:40:37.258355+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:56:53.783210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249907+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:29:39.101585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.266663+0000","last_change":"2023-10-10T20:40:37.266663+0000","last_active":"2023-10-10T20:40:37.266663+0000","last_peered":"2023-10-10T20:40:37.266663+0000","last_clean":"2023-10-10T20:40:37.266663+0000","last_became_active":"2023-10-10T20:40:37.266402+0000","last_became_peered":"2023-10-10T20:40:37.266402+0000","last_unstale":"2023-10-10T20:40:37.266663+0000","last_undegraded":"2023-10-10T20:40:37.266663+0000","last_fullsized":"2023-10-10T20:40:37.266663+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:43:14.351198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249725+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:56:54.788047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265713+0000","last_change":"2023-10-10T20:40:37.265713+0000","last_active":"2023-10-10T20:40:37.265713+0000","last_peered":"2023-10-10T20:40:37.265713+0000","last_clean":"2023-10-10T20:40:37.265713+0000","last_became_active":"2023-10-10T20:40:37.265287+0000","last_became_peered":"2023-10-10T20:40:37.265287+0000","last_unstale":"2023-10-10T20:40:37.265713+0000","last_undegraded":"2023-10-10T20:40:37.265713+0000","last_fullsized":"2023-10-10T20:40:37.265713+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:26.330192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249409+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:14.934213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264839+0000","last_change":"2023-10-10T20:40:37.264839+0000","last_active":"2023-10-10T20:40:37.264839+0000","last_peered":"2023-10-10T20:40:37.264839+0000","last_clean":"2023-10-10T20:40:37.264839+0000","last_became_active":"2023-10-10T20:40:37.264587+0000","last_became_peered":"2023-10-10T20:40:37.264587+0000","last_unstale":"2023-10-10T20:40:37.264839+0000","last_undegraded":"2023-10-10T20:40:37.264839+0000","last_fullsized":"2023-10-10T20:40:37.264839+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:24:35.687360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255974+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:32.259196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.260083+0000","last_change":"2023-10-10T20:40:37.260083+0000","last_active":"2023-10-10T20:40:37.260083+0000","last_peered":"2023-10-10T20:40:37.260083+0000","last_clean":"2023-10-10T20:40:37.260083+0000","last_became_active":"2023-10-10T20:40:37.259508+0000","last_became_peered":"2023-10-10T20:40:37.259508+0000","last_unstale":"2023-10-10T20:40:37.260083+0000","last_undegraded":"2023-10-10T20:40:37.260083+0000","last_fullsized":"2023-10-10T20:40:37.260083+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:13.929377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254091+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:33:05.914463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.266631+0000","last_change":"2023-10-10T20:40:37.266631+0000","last_active":"2023-10-10T20:40:37.266631+0000","last_peered":"2023-10-10T20:40:37.266631+0000","last_clean":"2023-10-10T20:40:37.266631+0000","last_became_active":"2023-10-10T20:40:37.266371+0000","last_became_peered":"2023-10-10T20:40:37.266371+0000","last_unstale":"2023-10-10T20:40:37.266631+0000","last_undegraded":"2023-10-10T20:40:37.266631+0000","last_fullsized":"2023-10-10T20:40:37.266631+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:36:44.776420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253499+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:29:51.950959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264623+0000","last_change":"2023-10-10T20:40:37.264623+0000","last_active":"2023-10-10T20:40:37.264623+0000","last_peered":"2023-10-10T20:40:37.264623+0000","last_clean":"2023-10-10T20:40:37.264623+0000","last_became_active":"2023-10-10T20:40:37.264322+0000","last_became_peered":"2023-10-10T20:40:37.264322+0000","last_unstale":"2023-10-10T20:40:37.264623+0000","last_undegraded":"2023-10-10T20:40:37.264623+0000","last_fullsized":"2023-10-10T20:40:37.264623+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:14:28.701104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254258+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:55:53.986623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265269+0000","last_change":"2023-10-10T20:40:37.265269+0000","last_active":"2023-10-10T20:40:37.265269+0000","last_peered":"2023-10-10T20:40:37.265269+0000","last_clean":"2023-10-10T20:40:37.265269+0000","last_became_active":"2023-10-10T20:40:37.264648+0000","last_became_peered":"2023-10-10T20:40:37.264648+0000","last_unstale":"2023-10-10T20:40:37.265269+0000","last_undegraded":"2023-10-10T20:40:37.265269+0000","last_fullsized":"2023-10-10T20:40:37.265269+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:14.826939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257410+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:39:12.325954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261923+0000","last_change":"2023-10-10T20:40:37.261923+0000","last_active":"2023-10-10T20:40:37.261923+0000","last_peered":"2023-10-10T20:40:37.261923+0000","last_clean":"2023-10-10T20:40:37.261923+0000","last_became_active":"2023-10-10T20:40:37.261631+0000","last_became_peered":"2023-10-10T20:40:37.261631+0000","last_unstale":"2023-10-10T20:40:37.261923+0000","last_undegraded":"2023-10-10T20:40:37.261923+0000","last_fullsized":"2023-10-10T20:40:37.261923+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:16:28.059016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.251150+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:38:00.681114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263329+0000","last_change":"2023-10-10T20:40:37.263329+0000","last_active":"2023-10-10T20:40:37.263329+0000","last_peered":"2023-10-10T20:40:37.263329+0000","last_clean":"2023-10-10T20:40:37.263329+0000","last_became_active":"2023-10-10T20:40:37.263072+0000","last_became_peered":"2023-10-10T20:40:37.263072+0000","last_unstale":"2023-10-10T20:40:37.263329+0000","last_undegraded":"2023-10-10T20:40:37.263329+0000","last_fullsized":"2023-10-10T20:40:37.263329+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:31:56.472091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257903+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:56:54.788047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.262448+0000","last_change":"2023-10-10T20:40:37.262448+0000","last_active":"2023-10-10T20:40:37.262448+0000","last_peered":"2023-10-10T20:40:37.262448+0000","last_clean":"2023-10-10T20:40:37.262448+0000","last_became_active":"2023-10-10T20:40:37.262193+0000","last_became_peered":"2023-10-10T20:40:37.262193+0000","last_unstale":"2023-10-10T20:40:37.262448+0000","last_undegraded":"2023-10-10T20:40:37.262448+0000","last_fullsized":"2023-10-10T20:40:37.262448+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:05.415898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.248907+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:24:16.195290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265730+0000","last_change":"2023-10-10T20:40:37.265730+0000","last_active":"2023-10-10T20:40:37.265730+0000","last_peered":"2023-10-10T20:40:37.265730+0000","last_clean":"2023-10-10T20:40:37.265730+0000","last_became_active":"2023-10-10T20:40:37.264840+0000","last_became_peered":"2023-10-10T20:40:37.264840+0000","last_unstale":"2023-10-10T20:40:37.265730+0000","last_undegraded":"2023-10-10T20:40:37.265730+0000","last_fullsized":"2023-10-10T20:40:37.265730+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:39:11.321117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255331+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:55:53.986623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261516+0000","last_change":"2023-10-10T20:40:37.261516+0000","last_active":"2023-10-10T20:40:37.261516+0000","last_peered":"2023-10-10T20:40:37.261516+0000","last_clean":"2023-10-10T20:40:37.261516+0000","last_became_active":"2023-10-10T20:40:37.261250+0000","last_became_peered":"2023-10-10T20:40:37.261250+0000","last_unstale":"2023-10-10T20:40:37.261516+0000","last_undegraded":"2023-10-10T20:40:37.261516+0000","last_fullsized":"2023-10-10T20:40:37.261516+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:37:03.617984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252774+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:20:35.134043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263365+0000","last_change":"2023-10-10T20:40:37.263365+0000","last_active":"2023-10-10T20:40:37.263365+0000","last_peered":"2023-10-10T20:40:37.263365+0000","last_clean":"2023-10-10T20:40:37.263365+0000","last_became_active":"2023-10-10T20:40:37.263032+0000","last_became_peered":"2023-10-10T20:40:37.263032+0000","last_unstale":"2023-10-10T20:40:37.263365+0000","last_undegraded":"2023-10-10T20:40:37.263365+0000","last_fullsized":"2023-10-10T20:40:37.263365+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:06.817315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249799+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:23.463451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264758+0000","last_change":"2023-10-10T20:40:37.264758+0000","last_active":"2023-10-10T20:40:37.264758+0000","last_peered":"2023-10-10T20:40:37.264758+0000","last_clean":"2023-10-10T20:40:37.264758+0000","last_became_active":"2023-10-10T20:40:37.264496+0000","last_became_peered":"2023-10-10T20:40:37.264496+0000","last_unstale":"2023-10-10T20:40:37.264758+0000","last_undegraded":"2023-10-10T20:40:37.264758+0000","last_fullsized":"2023-10-10T20:40:37.264758+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:43:18.734440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255210+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:03:14.859123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261011+0000","last_change":"2023-10-10T20:40:37.261011+0000","last_active":"2023-10-10T20:40:37.261011+0000","last_peered":"2023-10-10T20:40:37.261011+0000","last_clean":"2023-10-10T20:40:37.261011+0000","last_became_active":"2023-10-10T20:40:37.260760+0000","last_became_peered":"2023-10-10T20:40:37.260760+0000","last_unstale":"2023-10-10T20:40:37.261011+0000","last_undegraded":"2023-10-10T20:40:37.261011+0000","last_fullsized":"2023-10-10T20:40:37.261011+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:42.139421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261667+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:32:45.025265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.267680+0000","last_change":"2023-10-10T20:40:37.267680+0000","last_active":"2023-10-10T20:40:37.267680+0000","last_peered":"2023-10-10T20:40:37.267680+0000","last_clean":"2023-10-10T20:40:37.267680+0000","last_became_active":"2023-10-10T20:40:37.267373+0000","last_became_peered":"2023-10-10T20:40:37.267373+0000","last_unstale":"2023-10-10T20:40:37.267680+0000","last_undegraded":"2023-10-10T20:40:37.267680+0000","last_fullsized":"2023-10-10T20:40:37.267680+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:30:44.468056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261219+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:27.335029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265941+0000","last_change":"2023-10-10T20:40:37.265941+0000","last_active":"2023-10-10T20:40:37.265941+0000","last_peered":"2023-10-10T20:40:37.265941+0000","last_clean":"2023-10-10T20:40:37.265941+0000","last_became_active":"2023-10-10T20:40:37.265532+0000","last_became_peered":"2023-10-10T20:40:37.265532+0000","last_unstale":"2023-10-10T20:40:37.265941+0000","last_undegraded":"2023-10-10T20:40:37.265941+0000","last_fullsized":"2023-10-10T20:40:37.265941+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:02.961417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257410+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:14.934213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264545+0000","last_change":"2023-10-10T20:40:37.264545+0000","last_active":"2023-10-10T20:40:37.264545+0000","last_peered":"2023-10-10T20:40:37.264545+0000","last_clean":"2023-10-10T20:40:37.264545+0000","last_became_active":"2023-10-10T20:40:37.264300+0000","last_became_peered":"2023-10-10T20:40:37.264300+0000","last_unstale":"2023-10-10T20:40:37.264545+0000","last_undegraded":"2023-10-10T20:40:37.264545+0000","last_fullsized":"2023-10-10T20:40:37.264545+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:03:53.037877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252229+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:43:08.406828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265540+0000","last_change":"2023-10-10T20:40:37.265540+0000","last_active":"2023-10-10T20:40:37.265540+0000","last_peered":"2023-10-10T20:40:37.265540+0000","last_clean":"2023-10-10T20:40:37.265540+0000","last_became_active":"2023-10-10T20:40:37.264555+0000","last_became_peered":"2023-10-10T20:40:37.264555+0000","last_unstale":"2023-10-10T20:40:37.265540+0000","last_undegraded":"2023-10-10T20:40:37.265540+0000","last_fullsized":"2023-10-10T20:40:37.265540+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:37:03.617984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.248868+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:39:12.325954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265098+0000","last_change":"2023-10-10T20:40:37.265098+0000","last_active":"2023-10-10T20:40:37.265098+0000","last_peered":"2023-10-10T20:40:37.265098+0000","last_clean":"2023-10-10T20:40:37.265098+0000","last_became_active":"2023-10-10T20:40:37.264742+0000","last_became_peered":"2023-10-10T20:40:37.264742+0000","last_unstale":"2023-10-10T20:40:37.265098+0000","last_undegraded":"2023-10-10T20:40:37.265098+0000","last_fullsized":"2023-10-10T20:40:37.265098+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:42:00.401594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":241,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:40:36.243776+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:40:36.243776+0000","last_peered":"2023-10-10T20:40:36.243776+0000","last_clean":"2023-10-10T20:40:36.243776+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:40:36.243776+0000","last_undegraded":"2023-10-10T20:40:36.243776+0000","last_fullsized":"2023-10-10T20:40:36.243776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303156,"kb_used_data":1772,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068684,"statfs":{"total":96636764160,"available":96326332416,"internally_reserved":0,"allocated":1814528,"data_stored":1114976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.60499999999999998,"15min":0.84999999999999998},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.747,"5min":6.8630000000000004,"15min":6.8630000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.56299999999999994,"15min":0.63800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.629,"5min":4.1959999999999997,"15min":4.1959999999999997},"last":0.40699999999999997}]},{"osd":1,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.65300000000000002,"15min":1.0489999999999999},"min":{"1min":0.39900000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.78800000000000003,"5min":10.1,"15min":10.1},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.61299999999999999,"15min":0.749},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66400000000000003,"5min":6.806,"15min":6.806},"last":0.44900000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.64000000000000001,"15min":0.80200000000000005},"min":{"1min":0.41599999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":6.9340000000000002,"15min":6.9340000000000002},"last":0.498},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59499999999999997,"15min":0.75900000000000001},"min":{"1min":0.38800000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73199999999999998,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61799999999999999,"15min":0.77800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70899999999999996,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.68100000000000005},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71899999999999997,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.48099999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.34599999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.626,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.374,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78400000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.38900000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.73299999999999998,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72699999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.372}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58899999999999997,"15min":0.57199999999999995},"min":{"1min":0.48699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77100000000000002,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.434},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.46800000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75600000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.50600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574927,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302576,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069264,"statfs":{"total":96636764160,"available":96326926336,"internally_reserved":0,"allocated":1220608,"data_stored":524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55700000000000005,"15min":0.68899999999999995},"min":{"1min":0.40799999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.73099999999999998,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":0.627},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57899999999999996,"15min":0.73299999999999998},"min":{"1min":0.372,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70799999999999996,"5min":7.8090000000000002,"15min":7.8090000000000002},"last":0.64300000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.72899999999999998},"min":{"1min":0.39700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71899999999999997,"5min":7.867,"15min":7.867},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.64200000000000002,"15min":0.92100000000000004},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75700000000000001,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.60099999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.747,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.63400000000000001,"15min":0.874},"min":{"1min":0.48399999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.71199999999999997,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":0.58499999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.628},"min":{"1min":0.439,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79000000000000004,"5min":4.8620000000000001,"15min":4.8620000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.73499999999999999},"min":{"1min":0.47199999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.755,"5min":7.9009999999999998,"15min":7.9009999999999998},"last":0.56499999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.504,"15min":0.46400000000000002},"min":{"1min":0.42799999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71799999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.56499999999999995},"min":{"1min":0.54200000000000004,"5min":0.37,"15min":0.37},"max":{"1min":0.69899999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72099999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51400000000000001,"15min":0.48599999999999999},"min":{"1min":0.40400000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55200000000000005,"15min":0.52600000000000002},"min":{"1min":0.47799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.70299999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.42799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74399999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56899999999999995,"15min":0.54100000000000004},"min":{"1min":0.52700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.747,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.65600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302576,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069264,"statfs":{"total":96636764160,"available":96326926336,"internally_reserved":0,"allocated":1220608,"data_stored":524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.36399999999999999,"15min":0.49099999999999999},"min":{"1min":0.219,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.56799999999999995,"5min":5.0380000000000003,"15min":5.0380000000000003},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33100000000000002,"15min":0.36499999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.47299999999999998,"5min":1.194,"15min":1.194},"last":0.318}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.36799999999999999},"min":{"1min":0.27800000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.51400000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.52000000000000002},"min":{"1min":0.26000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52500000000000002,"5min":5.0919999999999996,"15min":5.0919999999999996},"last":0.29399999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.40799999999999997,"15min":0.54300000000000004},"min":{"1min":0.28000000000000003,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.59899999999999998,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38100000000000001,"15min":0.41299999999999998},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55500000000000005,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.26800000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.40000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92400000000000004,"5min":1.254,"15min":1.254},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60599999999999998,"15min":0.71899999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96499999999999997,"5min":4.9850000000000003,"15min":4.9850000000000003},"last":0.56899999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97599999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.42899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.878,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.55300000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67600000000000005,"15min":0.86699999999999999},"min":{"1min":0.46500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.995,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.40699999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85799999999999998,"5min":1.171,"15min":1.171},"last":0.67300000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60099999999999998,"15min":0.62},"min":{"1min":0.443,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.004,"5min":1.214,"15min":1.214},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63700000000000001,"15min":0.73699999999999999},"min":{"1min":0.41999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90500000000000003,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.68700000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303140,"kb_used_data":1756,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068700,"statfs":{"total":96636764160,"available":96326348800,"internally_reserved":0,"allocated":1798144,"data_stored":1107752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.377,"15min":0.55100000000000005},"min":{"1min":0.20699999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.51600000000000001,"5min":7.9249999999999998,"15min":7.9249999999999998},"last":0.255},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.374,"15min":0.52400000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45800000000000002,"5min":6.6299999999999999,"15min":6.6299999999999999},"last":0.33300000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.39000000000000001,"15min":0.57399999999999995},"min":{"1min":0.23100000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.47499999999999998,"5min":8.2509999999999994,"15min":8.2509999999999994},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39800000000000002,"15min":0.54800000000000004},"min":{"1min":0.245,"5min":0.222,"15min":0.222},"max":{"1min":0.52400000000000002,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.372}]},{"osd":3,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.40500000000000003,"15min":0.57699999999999996},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.45400000000000001,"15min":0.63400000000000001},"min":{"1min":0.31900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57899999999999996,"5min":7.8929999999999998,"15min":7.8929999999999998},"last":0.32200000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.61099999999999999,"15min":0.73199999999999998},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78000000000000003,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.63600000000000001,"15min":0.79600000000000004},"min":{"1min":0.45100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.95399999999999996,"5min":7.6920000000000002,"15min":7.6920000000000002},"last":0.49399999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.77500000000000002},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92400000000000004,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.64300000000000002,"15min":0.79900000000000004},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91400000000000003,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":0.47999999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67000000000000004,"15min":0.83999999999999997},"min":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94499999999999995,"5min":7.8360000000000003,"15min":7.8360000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66400000000000003,"15min":0.77900000000000003},"min":{"1min":0.49199999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90300000000000002,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.56699999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64000000000000001,"15min":0.76900000000000002},"min":{"1min":0.45900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93300000000000005,"5min":6.7160000000000002,"15min":6.7160000000000002},"last":0.503},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66100000000000003,"15min":0.80900000000000005},"min":{"1min":0.44600000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.878,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.45300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302576,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069264,"statfs":{"total":96636764160,"available":96326926336,"internally_reserved":0,"allocated":1220608,"data_stored":524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.31900000000000001,"15min":0.32800000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.59699999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.33600000000000002,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45400000000000001,"5min":0.746,"15min":0.746},"last":0.25800000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.45400000000000001,"15min":0.77100000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.63,"5min":11.967000000000001,"15min":11.967000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39000000000000001,"15min":0.498},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52700000000000002,"5min":4.1100000000000003,"15min":4.1100000000000003},"last":0.309}]},{"osd":3,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.41199999999999998,"15min":0.57899999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48799999999999999,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.50900000000000001,"15min":0.82199999999999995},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65900000000000003,"5min":11.935,"15min":11.935},"last":0.41199999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.80100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.58999999999999997},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46200000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.33400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83599999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.58399999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81599999999999995,"5min":1.052,"15min":1.052},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64500000000000002,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85999999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.59799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574927,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302564,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069276,"statfs":{"total":96636764160,"available":96326938624,"internally_reserved":0,"allocated":1208320,"data_stored":518345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.31,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.311},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.31900000000000001},"min":{"1min":0.25,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.32900000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.23499999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38600000000000001,"15min":0.377},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.57099999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.34999999999999998},"min":{"1min":0.23699999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.35699999999999998,"15min":0.36199999999999999},"min":{"1min":0.26300000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.36599999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69999999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.82299999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64500000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.35899999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55900000000000005,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.76900000000000002,"5min":1.01,"15min":1.01},"last":0.56699999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.42099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85699999999999998,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.441,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.496,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.88400000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298468,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073372,"statfs":{"total":96636764160,"available":96331132928,"internally_reserved":0,"allocated":1208320,"data_stored":518345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58199999999999996,"5min":0.747,"15min":0.747},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.59799999999999998,"15min":0.79200000000000004},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69699999999999995,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.48499999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.60299999999999998,"15min":0.79900000000000004},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60299999999999998,"5min":7.7160000000000002,"15min":7.7160000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.58599999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68100000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63600000000000001,"15min":0.81799999999999995},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.72799999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.58899999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61399999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.502,"15min":0.48399999999999999},"min":{"1min":0.34599999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.38400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77300000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.60399999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.41699999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72099999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.52600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.70599999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.58199999999999996},"min":{"1min":0.44700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298480,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073360,"statfs":{"total":96636764160,"available":96331120640,"internally_reserved":0,"allocated":1220608,"data_stored":524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.307,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63600000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.34300000000000003,"5min":0.215,"15min":0.215},"max":{"1min":0.70399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.46600000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.56200000000000006,"15min":0.65300000000000002},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55300000000000005,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68200000000000005,"5min":5.3109999999999999,"15min":5.3109999999999999},"last":0.55700000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56899999999999995,"15min":0.66100000000000003},"min":{"1min":0.32900000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64700000000000002,"5min":5.2960000000000003,"15min":5.2960000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72399999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.47799999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59499999999999997,"15min":0.69099999999999995},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70399999999999996,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":0.499},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59699999999999998,"15min":0.70399999999999996},"min":{"1min":0.40300000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70599999999999996,"5min":5.2850000000000001,"15min":5.2850000000000001},"last":0.56799999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48299999999999998,"15min":0.497},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.63800000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.311},{"interface":"front","average":{"1min":0.505,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65000000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.34499999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.65100000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71299999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.36199999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.33000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72699999999999998,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.38100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:40:42.690 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:40:42.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.834+0000 7f2ed1b89700 1 Processor -- start 2023-10-10T20:40:42.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.834+0000 7f2ed1b89700 1 -- start start 2023-10-10T20:40:42.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.835+0000 7f2ed1b89700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ecc159420 0x7f2ecc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:42.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.835+0000 7f2ed1b89700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2ecc15d230 0x7f2ecc15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:42.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.835+0000 7f2ed1b89700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ecc15dbd0 con 0x7f2ecc1597f0 2023-10-10T20:40:42.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.835+0000 7f2ed1b89700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ecc0b2980 con 0x7f2ecc15d230 2023-10-10T20:40:42.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.835+0000 7f2ed1b89700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc0b11d0 con 0x7f2ecc159420 2023-10-10T20:40:42.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.835+0000 7f2ed1388700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2ecc15d230 0x7f2ecc15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:42.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.835+0000 7f2ed1388700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2ecc15d230 0x7f2ecc15d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39002/0 (socket says 172.21.15.89:39002) 2023-10-10T20:40:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.835+0000 7f2ed1388700 1 -- 172.21.15.89:0/3306651028 learned_addr learned my addr 172.21.15.89:0/3306651028 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.835+0000 7f2ed0b87700 1 --2- 172.21.15.89:0/3306651028 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ecc159420 0x7f2ecc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.835+0000 7f2ed1388700 1 -- 172.21.15.89:0/3306651028 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ecc159420 msgr2=0x7f2ecc0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.835+0000 7f2ed1388700 1 --2- 172.21.15.89:0/3306651028 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ecc159420 0x7f2ecc0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.835+0000 7f2ed1388700 1 -- 172.21.15.89:0/3306651028 >> v1:172.21.15.111:6789/0 conn(0x7f2ecc1597f0 legacy=0x7f2ecc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:42.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.835+0000 7f2ed1388700 1 -- 172.21.15.89:0/3306651028 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc0b14b0 con 0x7f2ecc15d230 2023-10-10T20:40:42.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.835+0000 7f2ed0b87700 1 --2- 172.21.15.89:0/3306651028 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ecc159420 0x7f2ecc0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:42.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.836+0000 7f2ed1388700 1 --2- 172.21.15.89:0/3306651028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ecc15d230 0x7f2ecc15d690 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40099b0 tx=0x7f2ec4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b70a900d50e2427 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:42.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.836+0000 7f2ecb7fe700 1 -- 172.21.15.89:0/3306651028 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec400b820 con 0x7f2ecc15d230 2023-10-10T20:40:42.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.836+0000 7f2ecb7fe700 1 -- 172.21.15.89:0/3306651028 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec400b980 con 0x7f2ecc15d230 2023-10-10T20:40:42.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.836+0000 7f2ecb7fe700 1 -- 172.21.15.89:0/3306651028 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec400bd20 con 0x7f2ecc15d230 2023-10-10T20:40:42.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.836+0000 7f2ed1b89700 1 -- 172.21.15.89:0/3306651028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ecc15d230 msgr2=0x7f2ecc15d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:42.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.836+0000 7f2ed1b89700 1 --2- 172.21.15.89:0/3306651028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ecc15d230 0x7f2ecc15d690 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40099b0 tx=0x7f2ec4009250 comp rx=0 tx=0).stop 2023-10-10T20:40:42.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.836+0000 7f2ed1b89700 1 -- 172.21.15.89:0/3306651028 shutdown_connections 2023-10-10T20:40:42.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.836+0000 7f2ed1b89700 1 --2- 172.21.15.89:0/3306651028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ecc15d230 0x7f2ecc15d690 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.836+0000 7f2ed1b89700 1 --2- 172.21.15.89:0/3306651028 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ecc159420 0x7f2ecc0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.836+0000 7f2ed1b89700 1 -- 172.21.15.89:0/3306651028 >> 172.21.15.89:0/3306651028 conn(0x7f2ecc0d0660 msgr2=0x7f2ecc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:42.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.836+0000 7f2ed1b89700 1 -- 172.21.15.89:0/3306651028 shutdown_connections 2023-10-10T20:40:42.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.836+0000 7f2ed1b89700 1 -- 172.21.15.89:0/3306651028 wait complete. 2023-10-10T20:40:42.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.837+0000 7f2ed1b89700 1 Processor -- start 2023-10-10T20:40:42.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.838+0000 7f2ed1b89700 1 -- start start 2023-10-10T20:40:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.838+0000 7f2ed1b89700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ecc159420 0x7f2ecc15ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.838+0000 7f2ed1b89700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ecc1597f0 0x7f2ecc16d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.838+0000 7f2ed1b89700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ecc15d230 0x7f2ecc153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.838+0000 7f2ed1b89700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc15b210 con 0x7f2ecc159420 2023-10-10T20:40:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.838+0000 7f2ed1b89700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc15b4f0 con 0x7f2ecc1597f0 2023-10-10T20:40:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.838+0000 7f2ed1b89700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc15b7d0 con 0x7f2ecc15d230 2023-10-10T20:40:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.838+0000 7f2ed0b87700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ecc159420 0x7f2ecc15ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:42.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.838+0000 7f2ed0b87700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ecc159420 0x7f2ecc15ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39004/0 (socket says 172.21.15.89:39004) 2023-10-10T20:40:42.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.838+0000 7f2ed0b87700 1 -- 172.21.15.89:0/1905391789 learned_addr learned my addr 172.21.15.89:0/1905391789 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:42.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.839+0000 7f2ed0b87700 1 -- 172.21.15.89:0/1905391789 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ecc15d230 msgr2=0x7f2ecc153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:42.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.839+0000 7f2ed0b87700 1 --2- 172.21.15.89:0/1905391789 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ecc15d230 0x7f2ecc153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.839+0000 7f2ed0b87700 1 -- 172.21.15.89:0/1905391789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ecc1597f0 msgr2=0x7f2ecc16d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:42.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.839+0000 7f2ed0b87700 1 --2- 172.21.15.89:0/1905391789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ecc1597f0 0x7f2ecc16d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:42.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.839+0000 7f2ed0b87700 1 -- 172.21.15.89:0/1905391789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec40094f0 con 0x7f2ecc159420 2023-10-10T20:40:42.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.839+0000 7f2ed0b87700 1 --2- 172.21.15.89:0/1905391789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ecc159420 0x7f2ecc15ac30 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400c830 tx=0x7f2eb4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:42.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.839+0000 7f2ec8ff9700 1 -- 172.21.15.89:0/1905391789 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4011600 con 0x7f2ecc159420 2023-10-10T20:40:42.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.839+0000 7f2ec8ff9700 1 -- 172.21.15.89:0/1905391789 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb4011760 con 0x7f2ecc159420 2023-10-10T20:40:42.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.839+0000 7f2ed1b89700 1 -- 172.21.15.89:0/1905391789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ecc15bab0 con 0x7f2ecc159420 2023-10-10T20:40:42.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.839+0000 7f2ec8ff9700 1 -- 172.21.15.89:0/1905391789 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4011b20 con 0x7f2ecc159420 2023-10-10T20:40:42.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.839+0000 7f2ed1b89700 1 -- 172.21.15.89:0/1905391789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ecc0c5400 con 0x7f2ecc159420 2023-10-10T20:40:42.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.840+0000 7f2ed1b89700 1 -- 172.21.15.89:0/1905391789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ecc15bca0 con 0x7f2ecc159420 2023-10-10T20:40:42.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.840+0000 7f2ec8ff9700 1 -- 172.21.15.89:0/1905391789 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f2eb400a3e0 con 0x7f2ecc159420 2023-10-10T20:40:42.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.840+0000 7f2ec8ff9700 1 --2- 172.21.15.89:0/1905391789 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2eac067290 0x7f2eac069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:42.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.841+0000 7f2ec8ff9700 1 -- 172.21.15.89:0/1905391789 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2eb4085010 con 0x7f2ecc159420 2023-10-10T20:40:42.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.841+0000 7f2ecbfff700 1 --2- 172.21.15.89:0/1905391789 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2eac067290 0x7f2eac069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:42.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.841+0000 7f2ecbfff700 1 --2- 172.21.15.89:0/1905391789 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2eac067290 0x7f2eac069740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc069f50 tx=0x7f2ebc007ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:42.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:42.845+0000 7f2ec8ff9700 1 -- 172.21.15.89:0/1905391789 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2eb4051d60 con 0x7f2ecc159420 2023-10-10T20:40:43.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.025+0000 7f2ed1b89700 1 -- 172.21.15.89:0/1905391789 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ecc15b660 con 0x7f2eac067290 2023-10-10T20:40:43.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.032+0000 7f2ec8ff9700 1 -- 172.21.15.89:0/1905391789 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286966 (secure 0 0 0) 0x7f2ecc15b660 con 0x7f2eac067290 2023-10-10T20:40:43.037 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:43.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.035+0000 7f2ed1b89700 1 -- 172.21.15.89:0/1905391789 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2eac067290 msgr2=0x7f2eac069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:43.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.035+0000 7f2ed1b89700 1 --2- 172.21.15.89:0/1905391789 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2eac067290 0x7f2eac069740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc069f50 tx=0x7f2ebc007ea0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.035+0000 7f2ed1b89700 1 -- 172.21.15.89:0/1905391789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ecc159420 msgr2=0x7f2ecc15ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:43.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.035+0000 7f2ed1b89700 1 --2- 172.21.15.89:0/1905391789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ecc159420 0x7f2ecc15ac30 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400c830 tx=0x7f2eb4008040 comp rx=0 tx=0).stop 2023-10-10T20:40:43.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.035+0000 7f2ed1b89700 1 -- 172.21.15.89:0/1905391789 shutdown_connections 2023-10-10T20:40:43.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.035+0000 7f2ed1b89700 1 --2- 172.21.15.89:0/1905391789 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ecc15d230 0x7f2ecc153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.035+0000 7f2ed1b89700 1 --2- 172.21.15.89:0/1905391789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ecc159420 0x7f2ecc15ac30 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.035+0000 7f2ed1b89700 1 --2- 172.21.15.89:0/1905391789 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2eac067290 0x7f2eac069740 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.035+0000 7f2ed1b89700 1 --2- 172.21.15.89:0/1905391789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ecc1597f0 0x7f2ecc16d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.035+0000 7f2ed1b89700 1 -- 172.21.15.89:0/1905391789 >> 172.21.15.89:0/1905391789 conn(0x7f2ecc0d0660 msgr2=0x7f2ecc0b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:43.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.035+0000 7f2ed1b89700 1 -- 172.21.15.89:0/1905391789 shutdown_connections 2023-10-10T20:40:43.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.035+0000 7f2ed1b89700 1 -- 172.21.15.89:0/1905391789 wait complete. 2023-10-10T20:40:43.043 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:40:43.049 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2023-10-10T20:40:42.725680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2413536,"kb_used_data":10656,"kb_used_omap":0,"kb_used_meta":2402816,"kb_avail":752561184,"statfs":{"total":773094113280,"available":770622652416,"internally_reserved":0,"allocated":10911744,"data_stored":5357850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2460483584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000876"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253558+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:25.230014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.260877+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:53:07.253745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255734+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:43.144258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.250640+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:28:37.819892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255881+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:36:00.607814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252745+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:15.831776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257694+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:45:44.964754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.258387+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:59:28.728877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253182+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:46:05.714540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252306+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:37:30.151565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261779+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:29:51.950959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.250202+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:39:38.169839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253730+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:03:14.859123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.258385+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:29:39.101585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254614+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:09:46.330074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254027+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:36.297633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.263244+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:43.144258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255797+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:27.914463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255314+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:27.914463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261622+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:14:29.705941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.263327+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:37:04.622821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.250382+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:59:28.728877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.263024+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:27.914463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.251359+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:39:38.169839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261486+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:07.822152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.251605+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:54.041748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254573+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:30:45.472893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.250590+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:23.463451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.262806+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:26.505825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253631+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:09:46.330074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252631+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:58:16.915031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252495+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:10:45.408128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253140+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:42:01.406431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265989+0000","last_change":"2023-10-10T20:40:37.265989+0000","last_active":"2023-10-10T20:40:37.265989+0000","last_peered":"2023-10-10T20:40:37.265989+0000","last_clean":"2023-10-10T20:40:37.265989+0000","last_became_active":"2023-10-10T20:40:37.265518+0000","last_became_peered":"2023-10-10T20:40:37.265518+0000","last_unstale":"2023-10-10T20:40:37.265989+0000","last_undegraded":"2023-10-10T20:40:37.265989+0000","last_fullsized":"2023-10-10T20:40:37.265989+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:10:44.403291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.256386+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:43.144258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.268252+0000","last_change":"2023-10-10T20:40:37.268252+0000","last_active":"2023-10-10T20:40:37.268252+0000","last_peered":"2023-10-10T20:40:37.268252+0000","last_clean":"2023-10-10T20:40:37.268252+0000","last_became_active":"2023-10-10T20:40:37.267982+0000","last_became_peered":"2023-10-10T20:40:37.267982+0000","last_unstale":"2023-10-10T20:40:37.268252+0000","last_undegraded":"2023-10-10T20:40:37.268252+0000","last_fullsized":"2023-10-10T20:40:37.268252+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:14:28.701104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254507+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:50:36.307293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263019+0000","last_change":"2023-10-10T20:40:37.263019+0000","last_active":"2023-10-10T20:40:37.263019+0000","last_peered":"2023-10-10T20:40:37.263019+0000","last_clean":"2023-10-10T20:40:37.263019+0000","last_became_active":"2023-10-10T20:40:37.262760+0000","last_became_peered":"2023-10-10T20:40:37.262760+0000","last_unstale":"2023-10-10T20:40:37.263019+0000","last_undegraded":"2023-10-10T20:40:37.263019+0000","last_fullsized":"2023-10-10T20:40:37.263019+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:12:55.235702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254807+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:26.505825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263048+0000","last_change":"2023-10-10T20:40:37.263048+0000","last_active":"2023-10-10T20:40:37.263048+0000","last_peered":"2023-10-10T20:40:37.263048+0000","last_clean":"2023-10-10T20:40:37.263048+0000","last_became_active":"2023-10-10T20:40:37.262792+0000","last_became_peered":"2023-10-10T20:40:37.262792+0000","last_unstale":"2023-10-10T20:40:37.263048+0000","last_undegraded":"2023-10-10T20:40:37.263048+0000","last_fullsized":"2023-10-10T20:40:37.263048+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:43:02.879509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249434+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:59:28.728877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.262612+0000","last_change":"2023-10-10T20:40:37.262612+0000","last_active":"2023-10-10T20:40:37.262612+0000","last_peered":"2023-10-10T20:40:37.262612+0000","last_clean":"2023-10-10T20:40:37.262612+0000","last_became_active":"2023-10-10T20:40:37.262298+0000","last_became_peered":"2023-10-10T20:40:37.262298+0000","last_unstale":"2023-10-10T20:40:37.262612+0000","last_undegraded":"2023-10-10T20:40:37.262612+0000","last_fullsized":"2023-10-10T20:40:37.262612+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:33:04.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.262303+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:03:14.859123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263207+0000","last_change":"2023-10-10T20:40:37.263207+0000","last_active":"2023-10-10T20:40:37.263207+0000","last_peered":"2023-10-10T20:40:37.263207+0000","last_clean":"2023-10-10T20:40:37.263207+0000","last_became_active":"2023-10-10T20:40:37.262945+0000","last_became_peered":"2023-10-10T20:40:37.262945+0000","last_unstale":"2023-10-10T20:40:37.263207+0000","last_undegraded":"2023-10-10T20:40:37.263207+0000","last_fullsized":"2023-10-10T20:40:37.263207+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:55:52.981786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253420+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:10:46.929737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.269620+0000","last_change":"2023-10-10T20:40:37.269620+0000","last_active":"2023-10-10T20:40:37.269620+0000","last_peered":"2023-10-10T20:40:37.269620+0000","last_clean":"2023-10-10T20:40:37.269620+0000","last_became_active":"2023-10-10T20:40:37.269347+0000","last_became_peered":"2023-10-10T20:40:37.269347+0000","last_unstale":"2023-10-10T20:40:37.269620+0000","last_undegraded":"2023-10-10T20:40:37.269620+0000","last_fullsized":"2023-10-10T20:40:37.269620+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:26:39.104435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257197+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:29:14.456681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264965+0000","last_change":"2023-10-10T20:40:37.264965+0000","last_active":"2023-10-10T20:40:37.264965+0000","last_peered":"2023-10-10T20:40:37.264965+0000","last_clean":"2023-10-10T20:40:37.264965+0000","last_became_active":"2023-10-10T20:40:37.264288+0000","last_became_peered":"2023-10-10T20:40:37.264288+0000","last_unstale":"2023-10-10T20:40:37.264965+0000","last_undegraded":"2023-10-10T20:40:37.264965+0000","last_fullsized":"2023-10-10T20:40:37.264965+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:09:19.490206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.256154+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:24:13.463965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264630+0000","last_change":"2023-10-10T20:40:37.264630+0000","last_active":"2023-10-10T20:40:37.264630+0000","last_peered":"2023-10-10T20:40:37.264630+0000","last_clean":"2023-10-10T20:40:37.264630+0000","last_became_active":"2023-10-10T20:40:37.264244+0000","last_became_peered":"2023-10-10T20:40:37.264244+0000","last_unstale":"2023-10-10T20:40:37.264630+0000","last_undegraded":"2023-10-10T20:40:37.264630+0000","last_fullsized":"2023-10-10T20:40:37.264630+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:06.742295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255716+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:26.505825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261554+0000","last_change":"2023-10-10T20:40:37.261554+0000","last_active":"2023-10-10T20:40:37.261554+0000","last_peered":"2023-10-10T20:40:37.261554+0000","last_clean":"2023-10-10T20:40:37.261554+0000","last_became_active":"2023-10-10T20:40:37.261298+0000","last_became_peered":"2023-10-10T20:40:37.261298+0000","last_unstale":"2023-10-10T20:40:37.261554+0000","last_undegraded":"2023-10-10T20:40:37.261554+0000","last_fullsized":"2023-10-10T20:40:37.261554+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:26.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249506+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:09:20.495043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261773+0000","last_change":"2023-10-10T20:40:37.261773+0000","last_active":"2023-10-10T20:40:37.261773+0000","last_peered":"2023-10-10T20:40:37.261773+0000","last_clean":"2023-10-10T20:40:37.261773+0000","last_became_active":"2023-10-10T20:40:37.261546+0000","last_became_peered":"2023-10-10T20:40:37.261546+0000","last_unstale":"2023-10-10T20:40:37.261773+0000","last_undegraded":"2023-10-10T20:40:37.261773+0000","last_fullsized":"2023-10-10T20:40:37.261773+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:24:15.190453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255131+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:13:53.110967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.258355+0000","last_change":"2023-10-10T20:40:37.258355+0000","last_active":"2023-10-10T20:40:37.258355+0000","last_peered":"2023-10-10T20:40:37.258355+0000","last_clean":"2023-10-10T20:40:37.258355+0000","last_became_active":"2023-10-10T20:40:37.258111+0000","last_became_peered":"2023-10-10T20:40:37.258111+0000","last_unstale":"2023-10-10T20:40:37.258355+0000","last_undegraded":"2023-10-10T20:40:37.258355+0000","last_fullsized":"2023-10-10T20:40:37.258355+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:56:53.783210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249907+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:29:39.101585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.266663+0000","last_change":"2023-10-10T20:40:37.266663+0000","last_active":"2023-10-10T20:40:37.266663+0000","last_peered":"2023-10-10T20:40:37.266663+0000","last_clean":"2023-10-10T20:40:37.266663+0000","last_became_active":"2023-10-10T20:40:37.266402+0000","last_became_peered":"2023-10-10T20:40:37.266402+0000","last_unstale":"2023-10-10T20:40:37.266663+0000","last_undegraded":"2023-10-10T20:40:37.266663+0000","last_fullsized":"2023-10-10T20:40:37.266663+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:43:14.351198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249725+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:56:54.788047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265713+0000","last_change":"2023-10-10T20:40:37.265713+0000","last_active":"2023-10-10T20:40:37.265713+0000","last_peered":"2023-10-10T20:40:37.265713+0000","last_clean":"2023-10-10T20:40:37.265713+0000","last_became_active":"2023-10-10T20:40:37.265287+0000","last_became_peered":"2023-10-10T20:40:37.265287+0000","last_unstale":"2023-10-10T20:40:37.265713+0000","last_undegraded":"2023-10-10T20:40:37.265713+0000","last_fullsized":"2023-10-10T20:40:37.265713+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:26.330192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249409+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:14.934213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264839+0000","last_change":"2023-10-10T20:40:37.264839+0000","last_active":"2023-10-10T20:40:37.264839+0000","last_peered":"2023-10-10T20:40:37.264839+0000","last_clean":"2023-10-10T20:40:37.264839+0000","last_became_active":"2023-10-10T20:40:37.264587+0000","last_became_peered":"2023-10-10T20:40:37.264587+0000","last_unstale":"2023-10-10T20:40:37.264839+0000","last_undegraded":"2023-10-10T20:40:37.264839+0000","last_fullsized":"2023-10-10T20:40:37.264839+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:24:35.687360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255974+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:32.259196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.260083+0000","last_change":"2023-10-10T20:40:37.260083+0000","last_active":"2023-10-10T20:40:37.260083+0000","last_peered":"2023-10-10T20:40:37.260083+0000","last_clean":"2023-10-10T20:40:37.260083+0000","last_became_active":"2023-10-10T20:40:37.259508+0000","last_became_peered":"2023-10-10T20:40:37.259508+0000","last_unstale":"2023-10-10T20:40:37.260083+0000","last_undegraded":"2023-10-10T20:40:37.260083+0000","last_fullsized":"2023-10-10T20:40:37.260083+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:13.929377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254091+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:33:05.914463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.266631+0000","last_change":"2023-10-10T20:40:37.266631+0000","last_active":"2023-10-10T20:40:37.266631+0000","last_peered":"2023-10-10T20:40:37.266631+0000","last_clean":"2023-10-10T20:40:37.266631+0000","last_became_active":"2023-10-10T20:40:37.266371+0000","last_became_peered":"2023-10-10T20:40:37.266371+0000","last_unstale":"2023-10-10T20:40:37.266631+0000","last_undegraded":"2023-10-10T20:40:37.266631+0000","last_fullsized":"2023-10-10T20:40:37.266631+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:36:44.776420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253499+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:29:51.950959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264623+0000","last_change":"2023-10-10T20:40:37.264623+0000","last_active":"2023-10-10T20:40:37.264623+0000","last_peered":"2023-10-10T20:40:37.264623+0000","last_clean":"2023-10-10T20:40:37.264623+0000","last_became_active":"2023-10-10T20:40:37.264322+0000","last_became_peered":"2023-10-10T20:40:37.264322+0000","last_unstale":"2023-10-10T20:40:37.264623+0000","last_undegraded":"2023-10-10T20:40:37.264623+0000","last_fullsized":"2023-10-10T20:40:37.264623+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:14:28.701104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254258+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:55:53.986623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265269+0000","last_change":"2023-10-10T20:40:37.265269+0000","last_active":"2023-10-10T20:40:37.265269+0000","last_peered":"2023-10-10T20:40:37.265269+0000","last_clean":"2023-10-10T20:40:37.265269+0000","last_became_active":"2023-10-10T20:40:37.264648+0000","last_became_peered":"2023-10-10T20:40:37.264648+0000","last_unstale":"2023-10-10T20:40:37.265269+0000","last_undegraded":"2023-10-10T20:40:37.265269+0000","last_fullsized":"2023-10-10T20:40:37.265269+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:14.826939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257410+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:39:12.325954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261923+0000","last_change":"2023-10-10T20:40:37.261923+0000","last_active":"2023-10-10T20:40:37.261923+0000","last_peered":"2023-10-10T20:40:37.261923+0000","last_clean":"2023-10-10T20:40:37.261923+0000","last_became_active":"2023-10-10T20:40:37.261631+0000","last_became_peered":"2023-10-10T20:40:37.261631+0000","last_unstale":"2023-10-10T20:40:37.261923+0000","last_undegraded":"2023-10-10T20:40:37.261923+0000","last_fullsized":"2023-10-10T20:40:37.261923+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:16:28.059016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.251150+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:38:00.681114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263329+0000","last_change":"2023-10-10T20:40:37.263329+0000","last_active":"2023-10-10T20:40:37.263329+0000","last_peered":"2023-10-10T20:40:37.263329+0000","last_clean":"2023-10-10T20:40:37.263329+0000","last_became_active":"2023-10-10T20:40:37.263072+0000","last_became_peered":"2023-10-10T20:40:37.263072+0000","last_unstale":"2023-10-10T20:40:37.263329+0000","last_undegraded":"2023-10-10T20:40:37.263329+0000","last_fullsized":"2023-10-10T20:40:37.263329+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:31:56.472091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257903+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:56:54.788047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.262448+0000","last_change":"2023-10-10T20:40:37.262448+0000","last_active":"2023-10-10T20:40:37.262448+0000","last_peered":"2023-10-10T20:40:37.262448+0000","last_clean":"2023-10-10T20:40:37.262448+0000","last_became_active":"2023-10-10T20:40:37.262193+0000","last_became_peered":"2023-10-10T20:40:37.262193+0000","last_unstale":"2023-10-10T20:40:37.262448+0000","last_undegraded":"2023-10-10T20:40:37.262448+0000","last_fullsized":"2023-10-10T20:40:37.262448+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:05.415898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.248907+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:24:16.195290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265730+0000","last_change":"2023-10-10T20:40:37.265730+0000","last_active":"2023-10-10T20:40:37.265730+0000","last_peered":"2023-10-10T20:40:37.265730+0000","last_clean":"2023-10-10T20:40:37.265730+0000","last_became_active":"2023-10-10T20:40:37.264840+0000","last_became_peered":"2023-10-10T20:40:37.264840+0000","last_unstale":"2023-10-10T20:40:37.265730+0000","last_undegraded":"2023-10-10T20:40:37.265730+0000","last_fullsized":"2023-10-10T20:40:37.265730+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:39:11.321117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255331+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:55:53.986623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261516+0000","last_change":"2023-10-10T20:40:37.261516+0000","last_active":"2023-10-10T20:40:37.261516+0000","last_peered":"2023-10-10T20:40:37.261516+0000","last_clean":"2023-10-10T20:40:37.261516+0000","last_became_active":"2023-10-10T20:40:37.261250+0000","last_became_peered":"2023-10-10T20:40:37.261250+0000","last_unstale":"2023-10-10T20:40:37.261516+0000","last_undegraded":"2023-10-10T20:40:37.261516+0000","last_fullsized":"2023-10-10T20:40:37.261516+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:37:03.617984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252774+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:20:35.134043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263365+0000","last_change":"2023-10-10T20:40:37.263365+0000","last_active":"2023-10-10T20:40:37.263365+0000","last_peered":"2023-10-10T20:40:37.263365+0000","last_clean":"2023-10-10T20:40:37.263365+0000","last_became_active":"2023-10-10T20:40:37.263032+0000","last_became_peered":"2023-10-10T20:40:37.263032+0000","last_unstale":"2023-10-10T20:40:37.263365+0000","last_undegraded":"2023-10-10T20:40:37.263365+0000","last_fullsized":"2023-10-10T20:40:37.263365+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:06.817315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249799+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:23.463451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264758+0000","last_change":"2023-10-10T20:40:37.264758+0000","last_active":"2023-10-10T20:40:37.264758+0000","last_peered":"2023-10-10T20:40:37.264758+0000","last_clean":"2023-10-10T20:40:37.264758+0000","last_became_active":"2023-10-10T20:40:37.264496+0000","last_became_peered":"2023-10-10T20:40:37.264496+0000","last_unstale":"2023-10-10T20:40:37.264758+0000","last_undegraded":"2023-10-10T20:40:37.264758+0000","last_fullsized":"2023-10-10T20:40:37.264758+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:43:18.734440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255210+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:03:14.859123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261011+0000","last_change":"2023-10-10T20:40:37.261011+0000","last_active":"2023-10-10T20:40:37.261011+0000","last_peered":"2023-10-10T20:40:37.261011+0000","last_clean":"2023-10-10T20:40:37.261011+0000","last_became_active":"2023-10-10T20:40:37.260760+0000","last_became_peered":"2023-10-10T20:40:37.260760+0000","last_unstale":"2023-10-10T20:40:37.261011+0000","last_undegraded":"2023-10-10T20:40:37.261011+0000","last_fullsized":"2023-10-10T20:40:37.261011+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:42.139421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261667+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:32:45.025265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.267680+0000","last_change":"2023-10-10T20:40:37.267680+0000","last_active":"2023-10-10T20:40:37.267680+0000","last_peered":"2023-10-10T20:40:37.267680+0000","last_clean":"2023-10-10T20:40:37.267680+0000","last_became_active":"2023-10-10T20:40:37.267373+0000","last_became_peered":"2023-10-10T20:40:37.267373+0000","last_unstale":"2023-10-10T20:40:37.267680+0000","last_undegraded":"2023-10-10T20:40:37.267680+0000","last_fullsized":"2023-10-10T20:40:37.267680+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:30:44.468056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261219+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:27.335029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265941+0000","last_change":"2023-10-10T20:40:37.265941+0000","last_active":"2023-10-10T20:40:37.265941+0000","last_peered":"2023-10-10T20:40:37.265941+0000","last_clean":"2023-10-10T20:40:37.265941+0000","last_became_active":"2023-10-10T20:40:37.265532+0000","last_became_peered":"2023-10-10T20:40:37.265532+0000","last_unstale":"2023-10-10T20:40:37.265941+0000","last_undegraded":"2023-10-10T20:40:37.265941+0000","last_fullsized":"2023-10-10T20:40:37.265941+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:02.961417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257410+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:14.934213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264545+0000","last_change":"2023-10-10T20:40:37.264545+0000","last_active":"2023-10-10T20:40:37.264545+0000","last_peered":"2023-10-10T20:40:37.264545+0000","last_clean":"2023-10-10T20:40:37.264545+0000","last_became_active":"2023-10-10T20:40:37.264300+0000","last_became_peered":"2023-10-10T20:40:37.264300+0000","last_unstale":"2023-10-10T20:40:37.264545+0000","last_undegraded":"2023-10-10T20:40:37.264545+0000","last_fullsized":"2023-10-10T20:40:37.264545+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:03:53.037877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252229+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:43:08.406828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265540+0000","last_change":"2023-10-10T20:40:37.265540+0000","last_active":"2023-10-10T20:40:37.265540+0000","last_peered":"2023-10-10T20:40:37.265540+0000","last_clean":"2023-10-10T20:40:37.265540+0000","last_became_active":"2023-10-10T20:40:37.264555+0000","last_became_peered":"2023-10-10T20:40:37.264555+0000","last_unstale":"2023-10-10T20:40:37.265540+0000","last_undegraded":"2023-10-10T20:40:37.265540+0000","last_fullsized":"2023-10-10T20:40:37.265540+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:37:03.617984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.248868+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:39:12.325954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265098+0000","last_change":"2023-10-10T20:40:37.265098+0000","last_active":"2023-10-10T20:40:37.265098+0000","last_peered":"2023-10-10T20:40:37.265098+0000","last_clean":"2023-10-10T20:40:37.265098+0000","last_became_active":"2023-10-10T20:40:37.264742+0000","last_became_peered":"2023-10-10T20:40:37.264742+0000","last_unstale":"2023-10-10T20:40:37.265098+0000","last_undegraded":"2023-10-10T20:40:37.265098+0000","last_fullsized":"2023-10-10T20:40:37.265098+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:42:00.401594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":241,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:40:36.243776+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:40:36.243776+0000","last_peered":"2023-10-10T20:40:36.243776+0000","last_clean":"2023-10-10T20:40:36.243776+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:40:36.243776+0000","last_undegraded":"2023-10-10T20:40:36.243776+0000","last_fullsized":"2023-10-10T20:40:36.243776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303156,"kb_used_data":1772,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068684,"statfs":{"total":96636764160,"available":96326332416,"internally_reserved":0,"allocated":1814528,"data_stored":1114976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.60499999999999998,"15min":0.84999999999999998},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.747,"5min":6.8630000000000004,"15min":6.8630000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.56299999999999994,"15min":0.63800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.629,"5min":4.1959999999999997,"15min":4.1959999999999997},"last":0.40699999999999997}]},{"osd":1,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.65300000000000002,"15min":1.0489999999999999},"min":{"1min":0.39900000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.78800000000000003,"5min":10.1,"15min":10.1},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.61299999999999999,"15min":0.749},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66400000000000003,"5min":6.806,"15min":6.806},"last":0.44900000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.64000000000000001,"15min":0.80200000000000005},"min":{"1min":0.41599999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":6.9340000000000002,"15min":6.9340000000000002},"last":0.498},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59499999999999997,"15min":0.75900000000000001},"min":{"1min":0.38800000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73199999999999998,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61799999999999999,"15min":0.77800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70899999999999996,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.68100000000000005},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71899999999999997,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.48099999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.34599999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.626,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.374,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78400000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.38900000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.73299999999999998,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72699999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.372}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58899999999999997,"15min":0.57199999999999995},"min":{"1min":0.48699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77100000000000002,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.434},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.46800000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75600000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.50600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574927,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302576,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069264,"statfs":{"total":96636764160,"available":96326926336,"internally_reserved":0,"allocated":1220608,"data_stored":524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55700000000000005,"15min":0.68899999999999995},"min":{"1min":0.40799999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.73099999999999998,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":0.627},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57899999999999996,"15min":0.73299999999999998},"min":{"1min":0.372,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70799999999999996,"5min":7.8090000000000002,"15min":7.8090000000000002},"last":0.64300000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.72899999999999998},"min":{"1min":0.39700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71899999999999997,"5min":7.867,"15min":7.867},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.64200000000000002,"15min":0.92100000000000004},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75700000000000001,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.60099999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.747,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.63400000000000001,"15min":0.874},"min":{"1min":0.48399999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.71199999999999997,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":0.58499999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.628},"min":{"1min":0.439,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79000000000000004,"5min":4.8620000000000001,"15min":4.8620000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.73499999999999999},"min":{"1min":0.47199999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.755,"5min":7.9009999999999998,"15min":7.9009999999999998},"last":0.56499999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.504,"15min":0.46400000000000002},"min":{"1min":0.42799999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71799999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.56499999999999995},"min":{"1min":0.54200000000000004,"5min":0.37,"15min":0.37},"max":{"1min":0.69899999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72099999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51400000000000001,"15min":0.48599999999999999},"min":{"1min":0.40400000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55200000000000005,"15min":0.52600000000000002},"min":{"1min":0.47799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.70299999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.42799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74399999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56899999999999995,"15min":0.54100000000000004},"min":{"1min":0.52700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.747,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.65600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302576,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069264,"statfs":{"total":96636764160,"available":96326926336,"internally_reserved":0,"allocated":1220608,"data_stored":524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.36399999999999999,"15min":0.49099999999999999},"min":{"1min":0.219,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.56799999999999995,"5min":5.0380000000000003,"15min":5.0380000000000003},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33100000000000002,"15min":0.36499999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.47299999999999998,"5min":1.194,"15min":1.194},"last":0.318}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.36799999999999999},"min":{"1min":0.27800000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.51400000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.52000000000000002},"min":{"1min":0.26000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52500000000000002,"5min":5.0919999999999996,"15min":5.0919999999999996},"last":0.29399999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.40799999999999997,"15min":0.54300000000000004},"min":{"1min":0.28000000000000003,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.59899999999999998,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38100000000000001,"15min":0.41299999999999998},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55500000000000005,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.26800000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.40000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92400000000000004,"5min":1.254,"15min":1.254},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60599999999999998,"15min":0.71899999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96499999999999997,"5min":4.9850000000000003,"15min":4.9850000000000003},"last":0.56899999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97599999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.42899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.878,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.55300000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67600000000000005,"15min":0.86699999999999999},"min":{"1min":0.46500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.995,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.40699999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85799999999999998,"5min":1.171,"15min":1.171},"last":0.67300000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60099999999999998,"15min":0.62},"min":{"1min":0.443,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.004,"5min":1.214,"15min":1.214},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63700000000000001,"15min":0.73699999999999999},"min":{"1min":0.41999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90500000000000003,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.68700000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303140,"kb_used_data":1756,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068700,"statfs":{"total":96636764160,"available":96326348800,"internally_reserved":0,"allocated":1798144,"data_stored":1107752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.377,"15min":0.55100000000000005},"min":{"1min":0.20699999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.51600000000000001,"5min":7.9249999999999998,"15min":7.9249999999999998},"last":0.255},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.374,"15min":0.52400000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45800000000000002,"5min":6.6299999999999999,"15min":6.6299999999999999},"last":0.33300000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.39000000000000001,"15min":0.57399999999999995},"min":{"1min":0.23100000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.47499999999999998,"5min":8.2509999999999994,"15min":8.2509999999999994},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39800000000000002,"15min":0.54800000000000004},"min":{"1min":0.245,"5min":0.222,"15min":0.222},"max":{"1min":0.52400000000000002,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.372}]},{"osd":3,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.40500000000000003,"15min":0.57699999999999996},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.45400000000000001,"15min":0.63400000000000001},"min":{"1min":0.31900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57899999999999996,"5min":7.8929999999999998,"15min":7.8929999999999998},"last":0.32200000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.61099999999999999,"15min":0.73199999999999998},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78000000000000003,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.63600000000000001,"15min":0.79600000000000004},"min":{"1min":0.45100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.95399999999999996,"5min":7.6920000000000002,"15min":7.6920000000000002},"last":0.49399999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.77500000000000002},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92400000000000004,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.64300000000000002,"15min":0.79900000000000004},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91400000000000003,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":0.47999999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67000000000000004,"15min":0.83999999999999997},"min":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94499999999999995,"5min":7.8360000000000003,"15min":7.8360000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66400000000000003,"15min":0.77900000000000003},"min":{"1min":0.49199999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90300000000000002,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.56699999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64000000000000001,"15min":0.76900000000000002},"min":{"1min":0.45900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93300000000000005,"5min":6.7160000000000002,"15min":6.7160000000000002},"last":0.503},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66100000000000003,"15min":0.80900000000000005},"min":{"1min":0.44600000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.878,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.45300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302576,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069264,"statfs":{"total":96636764160,"available":96326926336,"internally_reserved":0,"allocated":1220608,"data_stored":524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.31900000000000001,"15min":0.32800000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.59699999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.33600000000000002,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45400000000000001,"5min":0.746,"15min":0.746},"last":0.25800000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.45400000000000001,"15min":0.77100000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.63,"5min":11.967000000000001,"15min":11.967000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39000000000000001,"15min":0.498},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52700000000000002,"5min":4.1100000000000003,"15min":4.1100000000000003},"last":0.309}]},{"osd":3,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.41199999999999998,"15min":0.57899999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48799999999999999,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.50900000000000001,"15min":0.82199999999999995},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65900000000000003,"5min":11.935,"15min":11.935},"last":0.41199999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.80100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.58999999999999997},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46200000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.33400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83599999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.58399999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81599999999999995,"5min":1.052,"15min":1.052},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64500000000000002,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85999999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.59799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574927,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302564,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069276,"statfs":{"total":96636764160,"available":96326938624,"internally_reserved":0,"allocated":1208320,"data_stored":518345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.31,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.311},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.31900000000000001},"min":{"1min":0.25,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.32900000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.23499999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38600000000000001,"15min":0.377},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.57099999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.34999999999999998},"min":{"1min":0.23699999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.35699999999999998,"15min":0.36199999999999999},"min":{"1min":0.26300000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.36599999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69999999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.82299999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64500000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.35899999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55900000000000005,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.76900000000000002,"5min":1.01,"15min":1.01},"last":0.56699999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.42099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85699999999999998,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.441,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.496,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.88400000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298468,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073372,"statfs":{"total":96636764160,"available":96331132928,"internally_reserved":0,"allocated":1208320,"data_stored":518345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58199999999999996,"5min":0.747,"15min":0.747},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.59799999999999998,"15min":0.79200000000000004},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69699999999999995,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.48499999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.60299999999999998,"15min":0.79900000000000004},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60299999999999998,"5min":7.7160000000000002,"15min":7.7160000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.58599999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68100000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63600000000000001,"15min":0.81799999999999995},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.72799999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.58899999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61399999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.502,"15min":0.48399999999999999},"min":{"1min":0.34599999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.38400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77300000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.60399999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.41699999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72099999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.52600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.70599999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.58199999999999996},"min":{"1min":0.44700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298480,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073360,"statfs":{"total":96636764160,"available":96331120640,"internally_reserved":0,"allocated":1220608,"data_stored":524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.307,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63600000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.34300000000000003,"5min":0.215,"15min":0.215},"max":{"1min":0.70399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.46600000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.56200000000000006,"15min":0.65300000000000002},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55300000000000005,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68200000000000005,"5min":5.3109999999999999,"15min":5.3109999999999999},"last":0.55700000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56899999999999995,"15min":0.66100000000000003},"min":{"1min":0.32900000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64700000000000002,"5min":5.2960000000000003,"15min":5.2960000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72399999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.47799999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59499999999999997,"15min":0.69099999999999995},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70399999999999996,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":0.499},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59699999999999998,"15min":0.70399999999999996},"min":{"1min":0.40300000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70599999999999996,"5min":5.2850000000000001,"15min":5.2850000000000001},"last":0.56799999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48299999999999998,"15min":0.497},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.63800000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.311},{"interface":"front","average":{"1min":0.505,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65000000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.34499999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.65100000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71299999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.36199999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.33000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72699999999999998,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.38100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:40:43.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3f is not active+clean 2023-10-10T20:40:43.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.253558+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:53:25.230014+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:43.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3e is not active+clean 2023-10-10T20:40:43.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.260877+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:53:07.253745+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:40:43.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3d is not active+clean 2023-10-10T20:40:43.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.255734+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:12:43.144258+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:40:43.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3c is not active+clean 2023-10-10T20:40:43.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.250640+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:28:37.819892+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:43.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3b is not active+clean 2023-10-10T20:40:43.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.255881+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:36:00.607814+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:40:43.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3a is not active+clean 2023-10-10T20:40:43.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.252745+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:13:15.831776+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:40:43.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.39 is not active+clean 2023-10-10T20:40:43.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.257694+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:45:44.964754+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:40:43.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.38 is not active+clean 2023-10-10T20:40:43.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.258387+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:59:28.728877+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:40:43.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.37 is not active+clean 2023-10-10T20:40:43.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.253182+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:46:05.714540+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:43.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.36 is not active+clean 2023-10-10T20:40:43.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.252306+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:37:30.151565+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:43.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.35 is not active+clean 2023-10-10T20:40:43.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.261779+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:29:51.950959+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:40:43.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.34 is not active+clean 2023-10-10T20:40:43.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.250202+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:39:38.169839+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:43.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.33 is not active+clean 2023-10-10T20:40:43.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.253730+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:03:14.859123+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:40:43.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.32 is not active+clean 2023-10-10T20:40:43.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.258385+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:29:39.101585+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:40:43.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.31 is not active+clean 2023-10-10T20:40:43.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.254614+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:09:46.330074+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:40:43.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.30 is not active+clean 2023-10-10T20:40:43.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.254027+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:08:36.297633+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:43.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2f is not active+clean 2023-10-10T20:40:43.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.263244+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:12:43.144258+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:40:43.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2e is not active+clean 2023-10-10T20:40:43.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.255797+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:07:27.914463+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:40:43.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2d is not active+clean 2023-10-10T20:40:43.070 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.255314+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:07:27.914463+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:40:43.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2c is not active+clean 2023-10-10T20:40:43.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.261622+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:14:29.705941+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:40:43.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2b is not active+clean 2023-10-10T20:40:43.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.263327+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:37:04.622821+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:40:43.073 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2a is not active+clean 2023-10-10T20:40:43.073 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.250382+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:59:28.728877+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:40:43.073 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.29 is not active+clean 2023-10-10T20:40:43.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.263024+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:07:27.914463+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:40:43.074 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.28 is not active+clean 2023-10-10T20:40:43.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.251359+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:39:38.169839+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:40:43.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.27 is not active+clean 2023-10-10T20:40:43.075 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.261486+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:03:07.822152+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:40:43.076 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.26 is not active+clean 2023-10-10T20:40:43.076 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.251605+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:12:54.041748+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:43.076 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.25 is not active+clean 2023-10-10T20:40:43.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.254573+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:30:45.472893+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:40:43.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.24 is not active+clean 2023-10-10T20:40:43.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.250590+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:49:23.463451+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:40:43.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.23 is not active+clean 2023-10-10T20:40:43.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.262806+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:34:26.505825+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:40:43.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.22 is not active+clean 2023-10-10T20:40:43.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.253631+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:09:46.330074+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:40:43.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.21 is not active+clean 2023-10-10T20:40:43.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.252631+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:58:16.915031+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:43.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.20 is not active+clean 2023-10-10T20:40:43.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.252495+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:10:45.408128+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:40:43.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1e is not active+clean 2023-10-10T20:40:43.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.253140+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:42:01.406431+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:40:43.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1f is not active+clean 2023-10-10T20:40:43.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.256386+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:12:43.144258+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:40:43.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1c is not active+clean 2023-10-10T20:40:43.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.254507+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:50:36.307293+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:40:43.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1d is not active+clean 2023-10-10T20:40:43.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.254807+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:34:26.505825+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:40:43.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1a is not active+clean 2023-10-10T20:40:43.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.249434+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:59:28.728877+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:43.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1b is not active+clean 2023-10-10T20:40:43.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.262303+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:03:14.859123+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:40:43.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.18 is not active+clean 2023-10-10T20:40:43.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.253420+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:10:46.929737+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:43.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.19 is not active+clean 2023-10-10T20:40:43.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.257197+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:29:14.456681+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:40:43.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.16 is not active+clean 2023-10-10T20:40:43.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.256154+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:24:13.463965+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:40:43.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.17 is not active+clean 2023-10-10T20:40:43.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.255716+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:34:26.505825+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:40:43.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.14 is not active+clean 2023-10-10T20:40:43.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.249506+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:09:20.495043+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:40:43.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.15 is not active+clean 2023-10-10T20:40:43.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.255131+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:13:53.110967+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:40:43.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.12 is not active+clean 2023-10-10T20:40:43.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.249907+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:29:39.101585+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:40:43.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.13 is not active+clean 2023-10-10T20:40:43.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.249725+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:56:54.788047+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:40:43.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.11 is not active+clean 2023-10-10T20:40:43.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.249409+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:28:14.934213+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:40:43.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.e is not active+clean 2023-10-10T20:40:43.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.255974+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:54:32.259196+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:40:43.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.f is not active+clean 2023-10-10T20:40:43.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.254091+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:33:05.914463+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:40:43.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.c is not active+clean 2023-10-10T20:40:43.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.253499+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:29:51.950959+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:40:43.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.d is not active+clean 2023-10-10T20:40:43.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.254258+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:55:53.986623+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:40:43.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.a is not active+clean 2023-10-10T20:40:43.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.257410+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:39:12.325954+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:40:43.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.b is not active+clean 2023-10-10T20:40:43.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.251150+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:38:00.681114+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:43.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.8 is not active+clean 2023-10-10T20:40:43.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.257903+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:56:54.788047+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:40:43.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.9 is not active+clean 2023-10-10T20:40:43.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.248907+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:24:16.195290+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:40:43.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.6 is not active+clean 2023-10-10T20:40:43.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.255331+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:55:53.986623+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:40:43.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.7 is not active+clean 2023-10-10T20:40:43.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.252774+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:20:35.134043+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:43.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.4 is not active+clean 2023-10-10T20:40:43.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.249799+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:49:23.463451+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:43.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.5 is not active+clean 2023-10-10T20:40:43.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.255210+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:03:14.859123+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:40:43.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2 is not active+clean 2023-10-10T20:40:43.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.261667+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:32:45.025265+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:40:43.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3 is not active+clean 2023-10-10T20:40:43.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.261219+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:13:27.335029+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:40:43.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.0 is not active+clean 2023-10-10T20:40:43.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.257410+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:28:14.934213+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:40:43.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1 is not active+clean 2023-10-10T20:40:43.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.252229+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:43:08.406828+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:40:43.108 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.10 is not active+clean 2023-10-10T20:40:43.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:40:37.239039+0000', 'last_change': '2023-10-10T20:40:37.248868+0000', 'last_active': '2023-10-10T20:40:37.239039+0000', 'last_peered': '2023-10-10T20:40:37.239039+0000', 'last_clean': '2023-10-10T20:40:37.239039+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:40:37.239039+0000', 'last_undegraded': '2023-10-10T20:40:37.239039+0000', 'last_fullsized': '2023-10-10T20:40:37.239039+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'last_clean_scrub_stamp': '2023-10-10T20:40:37.239039+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:39:12.325954+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:40:43.109 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-10T20:40:43.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.221+0000 7f6edd454700 1 Processor -- start 2023-10-10T20:40:43.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.221+0000 7f6edd454700 1 -- start start 2023-10-10T20:40:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.222+0000 7f6edd454700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ed815c9a0 0x7f6ed80cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.222+0000 7f6edd454700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6ed80ce900 0x7f6ed80cb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.222+0000 7f6edd454700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed80cfa60 con 0x7f6ed815cd70 2023-10-10T20:40:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.222+0000 7f6edd454700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ed80cee50 con 0x7f6ed80ce900 2023-10-10T20:40:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.222+0000 7f6edd454700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80cf130 con 0x7f6ed815c9a0 2023-10-10T20:40:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.222+0000 7f6ed77fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6ed80ce900 0x7f6ed80cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.222+0000 7f6ed7fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ed815c9a0 0x7f6ed80cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.222+0000 7f6ed77fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6ed80ce900 0x7f6ed80cb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:59464/0 (socket says 172.21.15.89:59464) 2023-10-10T20:40:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.222+0000 7f6ed77fe700 1 -- 172.21.15.89:0/3108116841 learned_addr learned my addr 172.21.15.89:0/3108116841 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.222+0000 7f6edcc53700 1 --1- 172.21.15.89:0/3108116841 >> v1:172.21.15.111:6789/0 conn(0x7f6ed815cd70 0x7f6ed815d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:37840/0 (socket says 172.21.15.89:37840) 2023-10-10T20:40:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.223+0000 7f6ed7fff700 1 -- 172.21.15.89:0/3108116841 >> v2:172.21.15.111:3300/0 conn(0x7f6ed80ce900 msgr2=0x7f6ed80cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.223+0000 7f6ed7fff700 1 --2- 172.21.15.89:0/3108116841 >> v2:172.21.15.111:3300/0 conn(0x7f6ed80ce900 0x7f6ed80cb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.223+0000 7f6ed7fff700 1 -- 172.21.15.89:0/3108116841 >> v1:172.21.15.111:6789/0 conn(0x7f6ed815cd70 legacy=0x7f6ed815d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.223+0000 7f6ed6ffd700 1 -- 172.21.15.89:0/3108116841 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661207151 0 0) 0x7f6ed80cfa60 con 0x7f6ed815cd70 2023-10-10T20:40:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.223+0000 7f6ed7fff700 1 -- 172.21.15.89:0/3108116841 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed80cf410 con 0x7f6ed815c9a0 2023-10-10T20:40:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.223+0000 7f6ed77fe700 1 --2- 172.21.15.89:0/3108116841 >> v2:172.21.15.111:3300/0 conn(0x7f6ed80ce900 0x7f6ed80cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:40:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.223+0000 7f6ed7fff700 1 --2- 172.21.15.89:0/3108116841 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ed815c9a0 0x7f6ed80cfef0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80098e0 tx=0x7f6ec80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31c82965d10bdfd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:43.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.223+0000 7f6ed6ffd700 1 -- 172.21.15.89:0/3108116841 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec800b640 con 0x7f6ed815c9a0 2023-10-10T20:40:43.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.223+0000 7f6ed6ffd700 1 -- 172.21.15.89:0/3108116841 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec800b7a0 con 0x7f6ed815c9a0 2023-10-10T20:40:43.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.223+0000 7f6ed6ffd700 1 -- 172.21.15.89:0/3108116841 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec800bb40 con 0x7f6ed815c9a0 2023-10-10T20:40:43.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.223+0000 7f6edd454700 1 -- 172.21.15.89:0/3108116841 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ed815c9a0 msgr2=0x7f6ed80cfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.223+0000 7f6edd454700 1 --2- 172.21.15.89:0/3108116841 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ed815c9a0 0x7f6ed80cfef0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80098e0 tx=0x7f6ec80091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.224+0000 7f6edd454700 1 -- 172.21.15.89:0/3108116841 shutdown_connections 2023-10-10T20:40:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.224+0000 7f6edd454700 1 --2- 172.21.15.89:0/3108116841 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ed815c9a0 0x7f6ed80cfef0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.224+0000 7f6edd454700 1 --2- 172.21.15.89:0/3108116841 >> v2:172.21.15.111:3300/0 conn(0x7f6ed80ce900 0x7f6ed80cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.224+0000 7f6edd454700 1 -- 172.21.15.89:0/3108116841 >> 172.21.15.89:0/3108116841 conn(0x7f6ed80d05c0 msgr2=0x7f6ed80d09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:43.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.224+0000 7f6edd454700 1 -- 172.21.15.89:0/3108116841 shutdown_connections 2023-10-10T20:40:43.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.224+0000 7f6edd454700 1 -- 172.21.15.89:0/3108116841 wait complete. 2023-10-10T20:40:43.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.225+0000 7f6edd454700 1 Processor -- start 2023-10-10T20:40:43.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.225+0000 7f6edd454700 1 -- start start 2023-10-10T20:40:43.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6edd454700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ed80ce900 0x7f6ed80c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6edd454700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ed815c9a0 0x7f6ed8154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6edd454700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ed815cd70 0x7f6ed8169d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6edd454700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80c8600 con 0x7f6ed815cd70 2023-10-10T20:40:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6edd454700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80b3740 con 0x7f6ed80ce900 2023-10-10T20:40:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6edd454700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80b3a20 con 0x7f6ed815c9a0 2023-10-10T20:40:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6edcc53700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ed815cd70 0x7f6ed8169d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6edcc53700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ed815cd70 0x7f6ed8169d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39018/0 (socket says 172.21.15.89:39018) 2023-10-10T20:40:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6edcc53700 1 -- 172.21.15.89:0/3714146044 learned_addr learned my addr 172.21.15.89:0/3714146044 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6ed7fff700 1 --2- 172.21.15.89:0/3714146044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ed80ce900 0x7f6ed80c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6ed77fe700 1 --2- 172.21.15.89:0/3714146044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ed815c9a0 0x7f6ed8154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6edcc53700 1 -- 172.21.15.89:0/3714146044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ed815c9a0 msgr2=0x7f6ed8154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6edcc53700 1 --2- 172.21.15.89:0/3714146044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ed815c9a0 0x7f6ed8154f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6edcc53700 1 -- 172.21.15.89:0/3714146044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ed80ce900 msgr2=0x7f6ed80c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6edcc53700 1 --2- 172.21.15.89:0/3714146044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ed80ce900 0x7f6ed80c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6edcc53700 1 -- 172.21.15.89:0/3714146044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ecc0094f0 con 0x7f6ed815cd70 2023-10-10T20:40:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6edcc53700 1 --2- 172.21.15.89:0/3714146044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ed815cd70 0x7f6ed8169d30 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000ccc0 tx=0x7f6ed0001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6ebffff700 1 -- 172.21.15.89:0/3714146044 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0005df0 con 0x7f6ed815cd70 2023-10-10T20:40:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6ebffff700 1 -- 172.21.15.89:0/3714146044 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed000a670 con 0x7f6ed815cd70 2023-10-10T20:40:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6edd454700 1 -- 172.21.15.89:0/3714146044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec8009420 con 0x7f6ed815cd70 2023-10-10T20:40:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6ebffff700 1 -- 172.21.15.89:0/3714146044 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed000aa30 con 0x7f6ed815cd70 2023-10-10T20:40:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.226+0000 7f6edd454700 1 -- 172.21.15.89:0/3714146044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed80b3fe0 con 0x7f6ed815cd70 2023-10-10T20:40:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.227+0000 7f6edd454700 1 -- 172.21.15.89:0/3714146044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed8088ea0 con 0x7f6ed815cd70 2023-10-10T20:40:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.228+0000 7f6ebffff700 1 -- 172.21.15.89:0/3714146044 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f6ed00070f0 con 0x7f6ed815cd70 2023-10-10T20:40:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.228+0000 7f6ebffff700 1 --2- 172.21.15.89:0/3714146044 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ec4067290 0x7f6ec4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.228+0000 7f6ebffff700 1 -- 172.21.15.89:0/3714146044 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6ed0084b20 con 0x7f6ed815cd70 2023-10-10T20:40:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.228+0000 7f6ed7fff700 1 --2- 172.21.15.89:0/3714146044 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ec4067290 0x7f6ec4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.229+0000 7f6ed7fff700 1 --2- 172.21.15.89:0/3714146044 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ec4067290 0x7f6ec4069740 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80091e0 tx=0x7f6ec8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.232+0000 7f6ebffff700 1 -- 172.21.15.89:0/3714146044 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ed0051900 con 0x7f6ed815cd70 2023-10-10T20:40:43.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.493+0000 7f6edd454700 1 -- 172.21.15.89:0/3714146044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6ed80cf6f0 con 0x7f6ed815cd70 2023-10-10T20:40:43.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.494+0000 7f6ebffff700 1 -- 172.21.15.89:0/3714146044 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1037 (secure 0 0 0) 0x7f6ed0054f20 con 0x7f6ed815cd70 2023-10-10T20:40:43.496 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:43.497 INFO:teuthology.orchestra.run.smithi089.stdout:{"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":386,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":100,"num_osds":8,"num_up_osds":8,"osd_up_since":1696970067,"num_in_osds":8,"osd_in_since":1696970059,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2471460864,"bytes_avail":770622652416,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":64,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":6822}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-10-10T20:37:28.685504+0000","services":{}},"progress_events":{}} 2023-10-10T20:40:43.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.495+0000 7f6edd454700 1 -- 172.21.15.89:0/3714146044 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ec4067290 msgr2=0x7f6ec4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:43.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.496+0000 7f6edd454700 1 --2- 172.21.15.89:0/3714146044 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ec4067290 0x7f6ec4069740 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80091e0 tx=0x7f6ec8017000 comp rx=0 tx=0).stop 2023-10-10T20:40:43.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.496+0000 7f6edd454700 1 -- 172.21.15.89:0/3714146044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ed815cd70 msgr2=0x7f6ed8169d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:43.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.496+0000 7f6edd454700 1 --2- 172.21.15.89:0/3714146044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ed815cd70 0x7f6ed8169d30 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000ccc0 tx=0x7f6ed0001a20 comp rx=0 tx=0).stop 2023-10-10T20:40:43.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.496+0000 7f6edd454700 1 -- 172.21.15.89:0/3714146044 shutdown_connections 2023-10-10T20:40:43.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.496+0000 7f6edd454700 1 --2- 172.21.15.89:0/3714146044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ed815cd70 0x7f6ed8169d30 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.496+0000 7f6edd454700 1 --2- 172.21.15.89:0/3714146044 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ec4067290 0x7f6ec4069740 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.496+0000 7f6edd454700 1 --2- 172.21.15.89:0/3714146044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ed80ce900 0x7f6ed80c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.496+0000 7f6edd454700 1 --2- 172.21.15.89:0/3714146044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ed815c9a0 0x7f6ed8154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.496+0000 7f6edd454700 1 -- 172.21.15.89:0/3714146044 >> 172.21.15.89:0/3714146044 conn(0x7f6ed80d05c0 msgr2=0x7f6ed80c6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:43.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.496+0000 7f6edd454700 1 -- 172.21.15.89:0/3714146044 shutdown_connections 2023-10-10T20:40:43.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.496+0000 7f6edd454700 1 -- 172.21.15.89:0/3714146044 wait complete. 2023-10-10T20:40:43.509 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-10T20:40:43.510 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:40:43.557 INFO:tasks.ceph.mds.a.smithi089.stderr:2023-10-10T20:40:43.555+0000 7fb1d1f81700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:40:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.623+0000 7f089c089700 1 Processor -- start 2023-10-10T20:40:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.623+0000 7f089c089700 1 -- start start 2023-10-10T20:40:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.624+0000 7f089c089700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0894159890 0x7f089415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.624+0000 7f089c089700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0894159c60 0x7f089415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.624+0000 7f089c089700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f089415a160 con 0x7f089415db10 2023-10-10T20:40:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.624+0000 7f089c089700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f089415df50 con 0x7f0894159c60 2023-10-10T20:40:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.624+0000 7f089c089700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08940d13a0 con 0x7f0894159890 2023-10-10T20:40:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.624+0000 7f0899624700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0894159c60 0x7f089415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.624+0000 7f089a626700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f089415db10 0x7f0894069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49194/0 (socket says 172.21.15.89:49194) 2023-10-10T20:40:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.624+0000 7f089a626700 1 -- 172.21.15.89:0/1238099532 learned_addr learned my addr 172.21.15.89:0/1238099532 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.624+0000 7f0899e25700 1 --2- 172.21.15.89:0/1238099532 >> v2:172.21.15.111:3300/0 conn(0x7f0894159890 0x7f089415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.624+0000 7f0899624700 1 -- 172.21.15.89:0/1238099532 >> v2:172.21.15.111:3300/0 conn(0x7f0894159890 msgr2=0x7f089415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.624+0000 7f0899624700 1 --2- 172.21.15.89:0/1238099532 >> v2:172.21.15.111:3300/0 conn(0x7f0894159890 0x7f089415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.624+0000 7f0899624700 1 -- 172.21.15.89:0/1238099532 >> v1:172.21.15.89:6789/0 conn(0x7f089415db10 legacy=0x7f0894069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.624+0000 7f0898e23700 1 -- 172.21.15.89:0/1238099532 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769754318 0 0) 0x7f089415a160 con 0x7f089415db10 2023-10-10T20:40:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.624+0000 7f0899624700 1 -- 172.21.15.89:0/1238099532 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089415e140 con 0x7f0894159c60 2023-10-10T20:40:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.624+0000 7f0899624700 1 --2- 172.21.15.89:0/1238099532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0894159c60 0x7f089415d5d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f08940d1090 tx=0x7f08840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefebe0db0e73d37 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:43.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.625+0000 7f0898e23700 1 -- 172.21.15.89:0/1238099532 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088400b6b0 con 0x7f0894159c60 2023-10-10T20:40:43.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.625+0000 7f0898e23700 1 -- 172.21.15.89:0/1238099532 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088400b810 con 0x7f0894159c60 2023-10-10T20:40:43.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.625+0000 7f0898e23700 1 -- 172.21.15.89:0/1238099532 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088400bbb0 con 0x7f0894159c60 2023-10-10T20:40:43.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.625+0000 7f089c089700 1 -- 172.21.15.89:0/1238099532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0894159c60 msgr2=0x7f089415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:43.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.625+0000 7f089c089700 1 --2- 172.21.15.89:0/1238099532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0894159c60 0x7f089415d5d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f08940d1090 tx=0x7f08840091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.625+0000 7f089c089700 1 -- 172.21.15.89:0/1238099532 shutdown_connections 2023-10-10T20:40:43.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.625+0000 7f089c089700 1 --2- 172.21.15.89:0/1238099532 >> v2:172.21.15.111:3300/0 conn(0x7f0894159890 0x7f089415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.625+0000 7f089c089700 1 --2- 172.21.15.89:0/1238099532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0894159c60 0x7f089415d5d0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.625+0000 7f089c089700 1 -- 172.21.15.89:0/1238099532 >> 172.21.15.89:0/1238099532 conn(0x7f0894068e20 msgr2=0x7f08940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:43.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.625+0000 7f089c089700 1 -- 172.21.15.89:0/1238099532 shutdown_connections 2023-10-10T20:40:43.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.625+0000 7f089c089700 1 -- 172.21.15.89:0/1238099532 wait complete. 2023-10-10T20:40:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.626+0000 7f089c089700 1 Processor -- start 2023-10-10T20:40:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.626+0000 7f089c089700 1 -- start start 2023-10-10T20:40:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.627+0000 7f089c089700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0894159890 0x7f08940cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.627+0000 7f089c089700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0894159c60 0x7f089414d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.627+0000 7f089c089700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f089415db10 0x7f0894152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.627+0000 7f089c089700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0894151ff0 con 0x7f0894159c60 2023-10-10T20:40:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.627+0000 7f089c089700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08940cdc20 con 0x7f089415db10 2023-10-10T20:40:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.627+0000 7f089c089700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08940cdec0 con 0x7f0894159890 2023-10-10T20:40:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.627+0000 7f0899624700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0894159c60 0x7f089414d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.627+0000 7f0899624700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0894159c60 0x7f089414d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39038/0 (socket says 172.21.15.89:39038) 2023-10-10T20:40:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.627+0000 7f0899624700 1 -- 172.21.15.89:0/2593956968 learned_addr learned my addr 172.21.15.89:0/2593956968 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.627+0000 7f089a626700 1 --2- 172.21.15.89:0/2593956968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f089415db10 0x7f0894152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.627+0000 7f0899e25700 1 --2- 172.21.15.89:0/2593956968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0894159890 0x7f08940cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.627+0000 7f0899624700 1 -- 172.21.15.89:0/2593956968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0894159890 msgr2=0x7f08940cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.627+0000 7f0899624700 1 --2- 172.21.15.89:0/2593956968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0894159890 0x7f08940cec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.627+0000 7f0899624700 1 -- 172.21.15.89:0/2593956968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f089415db10 msgr2=0x7f0894152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:43.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.627+0000 7f0899624700 1 --2- 172.21.15.89:0/2593956968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f089415db10 0x7f0894152300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.627+0000 7f0899624700 1 -- 172.21.15.89:0/2593956968 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0884009420 con 0x7f0894159c60 2023-10-10T20:40:43.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.627+0000 7f0899e25700 1 --2- 172.21.15.89:0/2593956968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0894159890 0x7f08940cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:43.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.627+0000 7f089a626700 1 --2- 172.21.15.89:0/2593956968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f089415db10 0x7f0894152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:43.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.628+0000 7f0899624700 1 --2- 172.21.15.89:0/2593956968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0894159c60 0x7f089414d460 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f0894070e20 tx=0x7f0884017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:43.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.628+0000 7f0889ffb700 1 -- 172.21.15.89:0/2593956968 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884011420 con 0x7f0894159c60 2023-10-10T20:40:43.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.628+0000 7f0889ffb700 1 -- 172.21.15.89:0/2593956968 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0884011580 con 0x7f0894159c60 2023-10-10T20:40:43.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.628+0000 7f0889ffb700 1 -- 172.21.15.89:0/2593956968 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884011940 con 0x7f0894159c60 2023-10-10T20:40:43.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.628+0000 7f089c089700 1 -- 172.21.15.89:0/2593956968 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0894152e50 con 0x7f0894159c60 2023-10-10T20:40:43.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.628+0000 7f089c089700 1 -- 172.21.15.89:0/2593956968 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08940ce440 con 0x7f0894159c60 2023-10-10T20:40:43.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.629+0000 7f089c089700 1 -- 172.21.15.89:0/2593956968 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0894152130 con 0x7f0894159c60 2023-10-10T20:40:43.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.629+0000 7f0889ffb700 1 -- 172.21.15.89:0/2593956968 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f0884011d30 con 0x7f0894159c60 2023-10-10T20:40:43.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.630+0000 7f0889ffb700 1 --2- 172.21.15.89:0/2593956968 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0878067290 0x7f0878069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:43.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.630+0000 7f0889ffb700 1 -- 172.21.15.89:0/2593956968 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f08840864d0 con 0x7f0894159c60 2023-10-10T20:40:43.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.630+0000 7f0899e25700 1 --2- 172.21.15.89:0/2593956968 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0878067290 0x7f0878069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:43.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.630+0000 7f0899e25700 1 --2- 172.21.15.89:0/2593956968 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0878067290 0x7f0878069740 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f088c007540 tx=0x7f088c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:43.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.634+0000 7f0889ffb700 1 -- 172.21.15.89:0/2593956968 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0884053330 con 0x7f0894159c60 2023-10-10T20:40:43.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.813+0000 7f089c089700 1 -- 172.21.15.89:0/2593956968 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f089404c3c0 con 0x7f0878067290 2023-10-10T20:40:43.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.821+0000 7f0889ffb700 1 -- 172.21.15.89:0/2593956968 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286966 (secure 0 0 0) 0x7f089404c3c0 con 0x7f0878067290 2023-10-10T20:40:43.826 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:43.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.824+0000 7f089c089700 1 -- 172.21.15.89:0/2593956968 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0878067290 msgr2=0x7f0878069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:43.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.824+0000 7f089c089700 1 --2- 172.21.15.89:0/2593956968 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0878067290 0x7f0878069740 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f088c007540 tx=0x7f088c00b040 comp rx=0 tx=0).stop 2023-10-10T20:40:43.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.824+0000 7f089c089700 1 -- 172.21.15.89:0/2593956968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0894159c60 msgr2=0x7f089414d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:43.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.824+0000 7f089c089700 1 --2- 172.21.15.89:0/2593956968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0894159c60 0x7f089414d460 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f0894070e20 tx=0x7f0884017000 comp rx=0 tx=0).stop 2023-10-10T20:40:43.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.824+0000 7f089c089700 1 -- 172.21.15.89:0/2593956968 shutdown_connections 2023-10-10T20:40:43.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.824+0000 7f089c089700 1 --2- 172.21.15.89:0/2593956968 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0878067290 0x7f0878069740 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.824+0000 7f089c089700 1 --2- 172.21.15.89:0/2593956968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f089415db10 0x7f0894152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.824+0000 7f089c089700 1 --2- 172.21.15.89:0/2593956968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0894159890 0x7f08940cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.824+0000 7f089c089700 1 --2- 172.21.15.89:0/2593956968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0894159c60 0x7f089414d460 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:43.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.824+0000 7f089c089700 1 -- 172.21.15.89:0/2593956968 >> 172.21.15.89:0/2593956968 conn(0x7f0894068e20 msgr2=0x7f0894069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:43.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.824+0000 7f089c089700 1 -- 172.21.15.89:0/2593956968 shutdown_connections 2023-10-10T20:40:43.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:43.824+0000 7f089c089700 1 -- 172.21.15.89:0/2593956968 wait complete. 2023-10-10T20:40:43.831 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:40:43.838 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2023-10-10T20:40:42.725680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2413536,"kb_used_data":10656,"kb_used_omap":0,"kb_used_meta":2402816,"kb_avail":752561184,"statfs":{"total":773094113280,"available":770622652416,"internally_reserved":0,"allocated":10911744,"data_stored":5357850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2460483584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000876"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253558+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:25.230014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.260877+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:53:07.253745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255734+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:43.144258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.250640+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:28:37.819892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255881+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:36:00.607814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252745+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:15.831776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257694+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:45:44.964754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.258387+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:59:28.728877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253182+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:46:05.714540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252306+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:37:30.151565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261779+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:29:51.950959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.250202+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:39:38.169839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253730+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:03:14.859123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.258385+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:29:39.101585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254614+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:09:46.330074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254027+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:36.297633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.263244+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:43.144258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255797+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:27.914463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255314+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:27.914463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261622+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:14:29.705941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.263327+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:37:04.622821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.250382+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:59:28.728877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.263024+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:27.914463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.251359+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:39:38.169839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261486+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:07.822152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.251605+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:54.041748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254573+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:30:45.472893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.250590+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:23.463451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.262806+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:26.505825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253631+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:09:46.330074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252631+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:58:16.915031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252495+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:10:45.408128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253140+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:42:01.406431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265989+0000","last_change":"2023-10-10T20:40:37.265989+0000","last_active":"2023-10-10T20:40:37.265989+0000","last_peered":"2023-10-10T20:40:37.265989+0000","last_clean":"2023-10-10T20:40:37.265989+0000","last_became_active":"2023-10-10T20:40:37.265518+0000","last_became_peered":"2023-10-10T20:40:37.265518+0000","last_unstale":"2023-10-10T20:40:37.265989+0000","last_undegraded":"2023-10-10T20:40:37.265989+0000","last_fullsized":"2023-10-10T20:40:37.265989+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:10:44.403291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.256386+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:43.144258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.268252+0000","last_change":"2023-10-10T20:40:37.268252+0000","last_active":"2023-10-10T20:40:37.268252+0000","last_peered":"2023-10-10T20:40:37.268252+0000","last_clean":"2023-10-10T20:40:37.268252+0000","last_became_active":"2023-10-10T20:40:37.267982+0000","last_became_peered":"2023-10-10T20:40:37.267982+0000","last_unstale":"2023-10-10T20:40:37.268252+0000","last_undegraded":"2023-10-10T20:40:37.268252+0000","last_fullsized":"2023-10-10T20:40:37.268252+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:14:28.701104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254507+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:50:36.307293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263019+0000","last_change":"2023-10-10T20:40:37.263019+0000","last_active":"2023-10-10T20:40:37.263019+0000","last_peered":"2023-10-10T20:40:37.263019+0000","last_clean":"2023-10-10T20:40:37.263019+0000","last_became_active":"2023-10-10T20:40:37.262760+0000","last_became_peered":"2023-10-10T20:40:37.262760+0000","last_unstale":"2023-10-10T20:40:37.263019+0000","last_undegraded":"2023-10-10T20:40:37.263019+0000","last_fullsized":"2023-10-10T20:40:37.263019+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:12:55.235702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254807+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:26.505825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263048+0000","last_change":"2023-10-10T20:40:37.263048+0000","last_active":"2023-10-10T20:40:37.263048+0000","last_peered":"2023-10-10T20:40:37.263048+0000","last_clean":"2023-10-10T20:40:37.263048+0000","last_became_active":"2023-10-10T20:40:37.262792+0000","last_became_peered":"2023-10-10T20:40:37.262792+0000","last_unstale":"2023-10-10T20:40:37.263048+0000","last_undegraded":"2023-10-10T20:40:37.263048+0000","last_fullsized":"2023-10-10T20:40:37.263048+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:43:02.879509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249434+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:59:28.728877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.262612+0000","last_change":"2023-10-10T20:40:37.262612+0000","last_active":"2023-10-10T20:40:37.262612+0000","last_peered":"2023-10-10T20:40:37.262612+0000","last_clean":"2023-10-10T20:40:37.262612+0000","last_became_active":"2023-10-10T20:40:37.262298+0000","last_became_peered":"2023-10-10T20:40:37.262298+0000","last_unstale":"2023-10-10T20:40:37.262612+0000","last_undegraded":"2023-10-10T20:40:37.262612+0000","last_fullsized":"2023-10-10T20:40:37.262612+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:33:04.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.262303+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:03:14.859123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263207+0000","last_change":"2023-10-10T20:40:37.263207+0000","last_active":"2023-10-10T20:40:37.263207+0000","last_peered":"2023-10-10T20:40:37.263207+0000","last_clean":"2023-10-10T20:40:37.263207+0000","last_became_active":"2023-10-10T20:40:37.262945+0000","last_became_peered":"2023-10-10T20:40:37.262945+0000","last_unstale":"2023-10-10T20:40:37.263207+0000","last_undegraded":"2023-10-10T20:40:37.263207+0000","last_fullsized":"2023-10-10T20:40:37.263207+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:55:52.981786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253420+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:10:46.929737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.269620+0000","last_change":"2023-10-10T20:40:37.269620+0000","last_active":"2023-10-10T20:40:37.269620+0000","last_peered":"2023-10-10T20:40:37.269620+0000","last_clean":"2023-10-10T20:40:37.269620+0000","last_became_active":"2023-10-10T20:40:37.269347+0000","last_became_peered":"2023-10-10T20:40:37.269347+0000","last_unstale":"2023-10-10T20:40:37.269620+0000","last_undegraded":"2023-10-10T20:40:37.269620+0000","last_fullsized":"2023-10-10T20:40:37.269620+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:26:39.104435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257197+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:29:14.456681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264965+0000","last_change":"2023-10-10T20:40:37.264965+0000","last_active":"2023-10-10T20:40:37.264965+0000","last_peered":"2023-10-10T20:40:37.264965+0000","last_clean":"2023-10-10T20:40:37.264965+0000","last_became_active":"2023-10-10T20:40:37.264288+0000","last_became_peered":"2023-10-10T20:40:37.264288+0000","last_unstale":"2023-10-10T20:40:37.264965+0000","last_undegraded":"2023-10-10T20:40:37.264965+0000","last_fullsized":"2023-10-10T20:40:37.264965+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:09:19.490206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.256154+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:24:13.463965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264630+0000","last_change":"2023-10-10T20:40:37.264630+0000","last_active":"2023-10-10T20:40:37.264630+0000","last_peered":"2023-10-10T20:40:37.264630+0000","last_clean":"2023-10-10T20:40:37.264630+0000","last_became_active":"2023-10-10T20:40:37.264244+0000","last_became_peered":"2023-10-10T20:40:37.264244+0000","last_unstale":"2023-10-10T20:40:37.264630+0000","last_undegraded":"2023-10-10T20:40:37.264630+0000","last_fullsized":"2023-10-10T20:40:37.264630+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:06.742295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255716+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:26.505825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261554+0000","last_change":"2023-10-10T20:40:37.261554+0000","last_active":"2023-10-10T20:40:37.261554+0000","last_peered":"2023-10-10T20:40:37.261554+0000","last_clean":"2023-10-10T20:40:37.261554+0000","last_became_active":"2023-10-10T20:40:37.261298+0000","last_became_peered":"2023-10-10T20:40:37.261298+0000","last_unstale":"2023-10-10T20:40:37.261554+0000","last_undegraded":"2023-10-10T20:40:37.261554+0000","last_fullsized":"2023-10-10T20:40:37.261554+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:26.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249506+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:09:20.495043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261773+0000","last_change":"2023-10-10T20:40:37.261773+0000","last_active":"2023-10-10T20:40:37.261773+0000","last_peered":"2023-10-10T20:40:37.261773+0000","last_clean":"2023-10-10T20:40:37.261773+0000","last_became_active":"2023-10-10T20:40:37.261546+0000","last_became_peered":"2023-10-10T20:40:37.261546+0000","last_unstale":"2023-10-10T20:40:37.261773+0000","last_undegraded":"2023-10-10T20:40:37.261773+0000","last_fullsized":"2023-10-10T20:40:37.261773+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:24:15.190453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255131+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:13:53.110967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.258355+0000","last_change":"2023-10-10T20:40:37.258355+0000","last_active":"2023-10-10T20:40:37.258355+0000","last_peered":"2023-10-10T20:40:37.258355+0000","last_clean":"2023-10-10T20:40:37.258355+0000","last_became_active":"2023-10-10T20:40:37.258111+0000","last_became_peered":"2023-10-10T20:40:37.258111+0000","last_unstale":"2023-10-10T20:40:37.258355+0000","last_undegraded":"2023-10-10T20:40:37.258355+0000","last_fullsized":"2023-10-10T20:40:37.258355+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:56:53.783210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249907+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:29:39.101585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.266663+0000","last_change":"2023-10-10T20:40:37.266663+0000","last_active":"2023-10-10T20:40:37.266663+0000","last_peered":"2023-10-10T20:40:37.266663+0000","last_clean":"2023-10-10T20:40:37.266663+0000","last_became_active":"2023-10-10T20:40:37.266402+0000","last_became_peered":"2023-10-10T20:40:37.266402+0000","last_unstale":"2023-10-10T20:40:37.266663+0000","last_undegraded":"2023-10-10T20:40:37.266663+0000","last_fullsized":"2023-10-10T20:40:37.266663+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:43:14.351198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249725+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:56:54.788047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265713+0000","last_change":"2023-10-10T20:40:37.265713+0000","last_active":"2023-10-10T20:40:37.265713+0000","last_peered":"2023-10-10T20:40:37.265713+0000","last_clean":"2023-10-10T20:40:37.265713+0000","last_became_active":"2023-10-10T20:40:37.265287+0000","last_became_peered":"2023-10-10T20:40:37.265287+0000","last_unstale":"2023-10-10T20:40:37.265713+0000","last_undegraded":"2023-10-10T20:40:37.265713+0000","last_fullsized":"2023-10-10T20:40:37.265713+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:26.330192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249409+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:14.934213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264839+0000","last_change":"2023-10-10T20:40:37.264839+0000","last_active":"2023-10-10T20:40:37.264839+0000","last_peered":"2023-10-10T20:40:37.264839+0000","last_clean":"2023-10-10T20:40:37.264839+0000","last_became_active":"2023-10-10T20:40:37.264587+0000","last_became_peered":"2023-10-10T20:40:37.264587+0000","last_unstale":"2023-10-10T20:40:37.264839+0000","last_undegraded":"2023-10-10T20:40:37.264839+0000","last_fullsized":"2023-10-10T20:40:37.264839+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:24:35.687360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255974+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:32.259196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.260083+0000","last_change":"2023-10-10T20:40:37.260083+0000","last_active":"2023-10-10T20:40:37.260083+0000","last_peered":"2023-10-10T20:40:37.260083+0000","last_clean":"2023-10-10T20:40:37.260083+0000","last_became_active":"2023-10-10T20:40:37.259508+0000","last_became_peered":"2023-10-10T20:40:37.259508+0000","last_unstale":"2023-10-10T20:40:37.260083+0000","last_undegraded":"2023-10-10T20:40:37.260083+0000","last_fullsized":"2023-10-10T20:40:37.260083+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:13.929377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254091+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:33:05.914463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.266631+0000","last_change":"2023-10-10T20:40:37.266631+0000","last_active":"2023-10-10T20:40:37.266631+0000","last_peered":"2023-10-10T20:40:37.266631+0000","last_clean":"2023-10-10T20:40:37.266631+0000","last_became_active":"2023-10-10T20:40:37.266371+0000","last_became_peered":"2023-10-10T20:40:37.266371+0000","last_unstale":"2023-10-10T20:40:37.266631+0000","last_undegraded":"2023-10-10T20:40:37.266631+0000","last_fullsized":"2023-10-10T20:40:37.266631+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:36:44.776420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.253499+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:29:51.950959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264623+0000","last_change":"2023-10-10T20:40:37.264623+0000","last_active":"2023-10-10T20:40:37.264623+0000","last_peered":"2023-10-10T20:40:37.264623+0000","last_clean":"2023-10-10T20:40:37.264623+0000","last_became_active":"2023-10-10T20:40:37.264322+0000","last_became_peered":"2023-10-10T20:40:37.264322+0000","last_unstale":"2023-10-10T20:40:37.264623+0000","last_undegraded":"2023-10-10T20:40:37.264623+0000","last_fullsized":"2023-10-10T20:40:37.264623+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:14:28.701104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.254258+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:55:53.986623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265269+0000","last_change":"2023-10-10T20:40:37.265269+0000","last_active":"2023-10-10T20:40:37.265269+0000","last_peered":"2023-10-10T20:40:37.265269+0000","last_clean":"2023-10-10T20:40:37.265269+0000","last_became_active":"2023-10-10T20:40:37.264648+0000","last_became_peered":"2023-10-10T20:40:37.264648+0000","last_unstale":"2023-10-10T20:40:37.265269+0000","last_undegraded":"2023-10-10T20:40:37.265269+0000","last_fullsized":"2023-10-10T20:40:37.265269+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:14.826939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257410+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:39:12.325954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261923+0000","last_change":"2023-10-10T20:40:37.261923+0000","last_active":"2023-10-10T20:40:37.261923+0000","last_peered":"2023-10-10T20:40:37.261923+0000","last_clean":"2023-10-10T20:40:37.261923+0000","last_became_active":"2023-10-10T20:40:37.261631+0000","last_became_peered":"2023-10-10T20:40:37.261631+0000","last_unstale":"2023-10-10T20:40:37.261923+0000","last_undegraded":"2023-10-10T20:40:37.261923+0000","last_fullsized":"2023-10-10T20:40:37.261923+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:16:28.059016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.251150+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:38:00.681114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263329+0000","last_change":"2023-10-10T20:40:37.263329+0000","last_active":"2023-10-10T20:40:37.263329+0000","last_peered":"2023-10-10T20:40:37.263329+0000","last_clean":"2023-10-10T20:40:37.263329+0000","last_became_active":"2023-10-10T20:40:37.263072+0000","last_became_peered":"2023-10-10T20:40:37.263072+0000","last_unstale":"2023-10-10T20:40:37.263329+0000","last_undegraded":"2023-10-10T20:40:37.263329+0000","last_fullsized":"2023-10-10T20:40:37.263329+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:31:56.472091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257903+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:56:54.788047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.262448+0000","last_change":"2023-10-10T20:40:37.262448+0000","last_active":"2023-10-10T20:40:37.262448+0000","last_peered":"2023-10-10T20:40:37.262448+0000","last_clean":"2023-10-10T20:40:37.262448+0000","last_became_active":"2023-10-10T20:40:37.262193+0000","last_became_peered":"2023-10-10T20:40:37.262193+0000","last_unstale":"2023-10-10T20:40:37.262448+0000","last_undegraded":"2023-10-10T20:40:37.262448+0000","last_fullsized":"2023-10-10T20:40:37.262448+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:05.415898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.248907+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:24:16.195290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265730+0000","last_change":"2023-10-10T20:40:37.265730+0000","last_active":"2023-10-10T20:40:37.265730+0000","last_peered":"2023-10-10T20:40:37.265730+0000","last_clean":"2023-10-10T20:40:37.265730+0000","last_became_active":"2023-10-10T20:40:37.264840+0000","last_became_peered":"2023-10-10T20:40:37.264840+0000","last_unstale":"2023-10-10T20:40:37.265730+0000","last_undegraded":"2023-10-10T20:40:37.265730+0000","last_fullsized":"2023-10-10T20:40:37.265730+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:39:11.321117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255331+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:55:53.986623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261516+0000","last_change":"2023-10-10T20:40:37.261516+0000","last_active":"2023-10-10T20:40:37.261516+0000","last_peered":"2023-10-10T20:40:37.261516+0000","last_clean":"2023-10-10T20:40:37.261516+0000","last_became_active":"2023-10-10T20:40:37.261250+0000","last_became_peered":"2023-10-10T20:40:37.261250+0000","last_unstale":"2023-10-10T20:40:37.261516+0000","last_undegraded":"2023-10-10T20:40:37.261516+0000","last_fullsized":"2023-10-10T20:40:37.261516+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:37:03.617984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252774+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:20:35.134043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.263365+0000","last_change":"2023-10-10T20:40:37.263365+0000","last_active":"2023-10-10T20:40:37.263365+0000","last_peered":"2023-10-10T20:40:37.263365+0000","last_clean":"2023-10-10T20:40:37.263365+0000","last_became_active":"2023-10-10T20:40:37.263032+0000","last_became_peered":"2023-10-10T20:40:37.263032+0000","last_unstale":"2023-10-10T20:40:37.263365+0000","last_undegraded":"2023-10-10T20:40:37.263365+0000","last_fullsized":"2023-10-10T20:40:37.263365+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:06.817315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.249799+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:23.463451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264758+0000","last_change":"2023-10-10T20:40:37.264758+0000","last_active":"2023-10-10T20:40:37.264758+0000","last_peered":"2023-10-10T20:40:37.264758+0000","last_clean":"2023-10-10T20:40:37.264758+0000","last_became_active":"2023-10-10T20:40:37.264496+0000","last_became_peered":"2023-10-10T20:40:37.264496+0000","last_unstale":"2023-10-10T20:40:37.264758+0000","last_undegraded":"2023-10-10T20:40:37.264758+0000","last_fullsized":"2023-10-10T20:40:37.264758+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:43:18.734440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.255210+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:03:14.859123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.261011+0000","last_change":"2023-10-10T20:40:37.261011+0000","last_active":"2023-10-10T20:40:37.261011+0000","last_peered":"2023-10-10T20:40:37.261011+0000","last_clean":"2023-10-10T20:40:37.261011+0000","last_became_active":"2023-10-10T20:40:37.260760+0000","last_became_peered":"2023-10-10T20:40:37.260760+0000","last_unstale":"2023-10-10T20:40:37.261011+0000","last_undegraded":"2023-10-10T20:40:37.261011+0000","last_fullsized":"2023-10-10T20:40:37.261011+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:42.139421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261667+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:32:45.025265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.267680+0000","last_change":"2023-10-10T20:40:37.267680+0000","last_active":"2023-10-10T20:40:37.267680+0000","last_peered":"2023-10-10T20:40:37.267680+0000","last_clean":"2023-10-10T20:40:37.267680+0000","last_became_active":"2023-10-10T20:40:37.267373+0000","last_became_peered":"2023-10-10T20:40:37.267373+0000","last_unstale":"2023-10-10T20:40:37.267680+0000","last_undegraded":"2023-10-10T20:40:37.267680+0000","last_fullsized":"2023-10-10T20:40:37.267680+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:30:44.468056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.261219+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:27.335029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265941+0000","last_change":"2023-10-10T20:40:37.265941+0000","last_active":"2023-10-10T20:40:37.265941+0000","last_peered":"2023-10-10T20:40:37.265941+0000","last_clean":"2023-10-10T20:40:37.265941+0000","last_became_active":"2023-10-10T20:40:37.265532+0000","last_became_peered":"2023-10-10T20:40:37.265532+0000","last_unstale":"2023-10-10T20:40:37.265941+0000","last_undegraded":"2023-10-10T20:40:37.265941+0000","last_fullsized":"2023-10-10T20:40:37.265941+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:02.961417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.257410+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:14.934213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.264545+0000","last_change":"2023-10-10T20:40:37.264545+0000","last_active":"2023-10-10T20:40:37.264545+0000","last_peered":"2023-10-10T20:40:37.264545+0000","last_clean":"2023-10-10T20:40:37.264545+0000","last_became_active":"2023-10-10T20:40:37.264300+0000","last_became_peered":"2023-10-10T20:40:37.264300+0000","last_unstale":"2023-10-10T20:40:37.264545+0000","last_undegraded":"2023-10-10T20:40:37.264545+0000","last_fullsized":"2023-10-10T20:40:37.264545+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:03:53.037877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.252229+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:43:08.406828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265540+0000","last_change":"2023-10-10T20:40:37.265540+0000","last_active":"2023-10-10T20:40:37.265540+0000","last_peered":"2023-10-10T20:40:37.265540+0000","last_clean":"2023-10-10T20:40:37.265540+0000","last_became_active":"2023-10-10T20:40:37.264555+0000","last_became_peered":"2023-10-10T20:40:37.264555+0000","last_unstale":"2023-10-10T20:40:37.265540+0000","last_undegraded":"2023-10-10T20:40:37.265540+0000","last_fullsized":"2023-10-10T20:40:37.265540+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:37:03.617984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:40:37.239039+0000","last_change":"2023-10-10T20:40:37.248868+0000","last_active":"2023-10-10T20:40:37.239039+0000","last_peered":"2023-10-10T20:40:37.239039+0000","last_clean":"2023-10-10T20:40:37.239039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:40:37.239039+0000","last_undegraded":"2023-10-10T20:40:37.239039+0000","last_fullsized":"2023-10-10T20:40:37.239039+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:39:12.325954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-10T20:40:37.265098+0000","last_change":"2023-10-10T20:40:37.265098+0000","last_active":"2023-10-10T20:40:37.265098+0000","last_peered":"2023-10-10T20:40:37.265098+0000","last_clean":"2023-10-10T20:40:37.265098+0000","last_became_active":"2023-10-10T20:40:37.264742+0000","last_became_peered":"2023-10-10T20:40:37.264742+0000","last_unstale":"2023-10-10T20:40:37.265098+0000","last_undegraded":"2023-10-10T20:40:37.265098+0000","last_fullsized":"2023-10-10T20:40:37.265098+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:42:00.401594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":241,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:40:36.243776+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:40:36.243776+0000","last_peered":"2023-10-10T20:40:36.243776+0000","last_clean":"2023-10-10T20:40:36.243776+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:40:36.243776+0000","last_undegraded":"2023-10-10T20:40:36.243776+0000","last_fullsized":"2023-10-10T20:40:36.243776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303156,"kb_used_data":1772,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068684,"statfs":{"total":96636764160,"available":96326332416,"internally_reserved":0,"allocated":1814528,"data_stored":1114976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.60499999999999998,"15min":0.84999999999999998},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.747,"5min":6.8630000000000004,"15min":6.8630000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.56299999999999994,"15min":0.63800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.629,"5min":4.1959999999999997,"15min":4.1959999999999997},"last":0.40699999999999997}]},{"osd":1,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.65300000000000002,"15min":1.0489999999999999},"min":{"1min":0.39900000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.78800000000000003,"5min":10.1,"15min":10.1},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.61299999999999999,"15min":0.749},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66400000000000003,"5min":6.806,"15min":6.806},"last":0.44900000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.64000000000000001,"15min":0.80200000000000005},"min":{"1min":0.41599999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":6.9340000000000002,"15min":6.9340000000000002},"last":0.498},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59499999999999997,"15min":0.75900000000000001},"min":{"1min":0.38800000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73199999999999998,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61799999999999999,"15min":0.77800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70899999999999996,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.68100000000000005},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71899999999999997,"5min":4.1609999999999996,"15min":4.1609999999999996},"last":0.48099999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.34599999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.626,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.374,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78400000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.38900000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.73299999999999998,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72699999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.372}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58899999999999997,"15min":0.57199999999999995},"min":{"1min":0.48699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77100000000000002,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.434},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.46800000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75600000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.50600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574927,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302576,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069264,"statfs":{"total":96636764160,"available":96326926336,"internally_reserved":0,"allocated":1220608,"data_stored":524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55700000000000005,"15min":0.68899999999999995},"min":{"1min":0.40799999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.73099999999999998,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":0.627},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57899999999999996,"15min":0.73299999999999998},"min":{"1min":0.372,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70799999999999996,"5min":7.8090000000000002,"15min":7.8090000000000002},"last":0.64300000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.72899999999999998},"min":{"1min":0.39700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71899999999999997,"5min":7.867,"15min":7.867},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.64200000000000002,"15min":0.92100000000000004},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75700000000000001,"5min":9.7089999999999996,"15min":9.7089999999999996},"last":0.60099999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.747,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.63400000000000001,"15min":0.874},"min":{"1min":0.48399999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.71199999999999997,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":0.58499999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.628},"min":{"1min":0.439,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79000000000000004,"5min":4.8620000000000001,"15min":4.8620000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.73499999999999999},"min":{"1min":0.47199999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.755,"5min":7.9009999999999998,"15min":7.9009999999999998},"last":0.56499999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.504,"15min":0.46400000000000002},"min":{"1min":0.42799999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71799999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.56499999999999995},"min":{"1min":0.54200000000000004,"5min":0.37,"15min":0.37},"max":{"1min":0.69899999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72099999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51400000000000001,"15min":0.48599999999999999},"min":{"1min":0.40400000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55200000000000005,"15min":0.52600000000000002},"min":{"1min":0.47799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.70299999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.42799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74399999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56899999999999995,"15min":0.54100000000000004},"min":{"1min":0.52700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.747,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.65600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302576,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069264,"statfs":{"total":96636764160,"available":96326926336,"internally_reserved":0,"allocated":1220608,"data_stored":524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.36399999999999999,"15min":0.49099999999999999},"min":{"1min":0.219,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.56799999999999995,"5min":5.0380000000000003,"15min":5.0380000000000003},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33100000000000002,"15min":0.36499999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.47299999999999998,"5min":1.194,"15min":1.194},"last":0.318}]},{"osd":2,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.36799999999999999},"min":{"1min":0.27800000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.51400000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.52000000000000002},"min":{"1min":0.26000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52500000000000002,"5min":5.0919999999999996,"15min":5.0919999999999996},"last":0.29399999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.40799999999999997,"15min":0.54300000000000004},"min":{"1min":0.28000000000000003,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.59899999999999998,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38100000000000001,"15min":0.41299999999999998},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55500000000000005,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.26800000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.40000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92400000000000004,"5min":1.254,"15min":1.254},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60599999999999998,"15min":0.71899999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96499999999999997,"5min":4.9850000000000003,"15min":4.9850000000000003},"last":0.56899999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97599999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.42899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.878,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.55300000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67600000000000005,"15min":0.86699999999999999},"min":{"1min":0.46500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.995,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.40699999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85799999999999998,"5min":1.171,"15min":1.171},"last":0.67300000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60099999999999998,"15min":0.62},"min":{"1min":0.443,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.004,"5min":1.214,"15min":1.214},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63700000000000001,"15min":0.73699999999999999},"min":{"1min":0.41999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90500000000000003,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.68700000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303140,"kb_used_data":1756,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068700,"statfs":{"total":96636764160,"available":96326348800,"internally_reserved":0,"allocated":1798144,"data_stored":1107752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.377,"15min":0.55100000000000005},"min":{"1min":0.20699999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.51600000000000001,"5min":7.9249999999999998,"15min":7.9249999999999998},"last":0.255},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.374,"15min":0.52400000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.45800000000000002,"5min":6.6299999999999999,"15min":6.6299999999999999},"last":0.33300000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.39000000000000001,"15min":0.57399999999999995},"min":{"1min":0.23100000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.47499999999999998,"5min":8.2509999999999994,"15min":8.2509999999999994},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39800000000000002,"15min":0.54800000000000004},"min":{"1min":0.245,"5min":0.222,"15min":0.222},"max":{"1min":0.52400000000000002,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.372}]},{"osd":3,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.40500000000000003,"15min":0.57699999999999996},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.45400000000000001,"15min":0.63400000000000001},"min":{"1min":0.31900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57899999999999996,"5min":7.8929999999999998,"15min":7.8929999999999998},"last":0.32200000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.61099999999999999,"15min":0.73199999999999998},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78000000000000003,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.63600000000000001,"15min":0.79600000000000004},"min":{"1min":0.45100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.95399999999999996,"5min":7.6920000000000002,"15min":7.6920000000000002},"last":0.49399999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.77500000000000002},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92400000000000004,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.64300000000000002,"15min":0.79900000000000004},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91400000000000003,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":0.47999999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67000000000000004,"15min":0.83999999999999997},"min":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94499999999999995,"5min":7.8360000000000003,"15min":7.8360000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66400000000000003,"15min":0.77900000000000003},"min":{"1min":0.49199999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90300000000000002,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.56699999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64000000000000001,"15min":0.76900000000000002},"min":{"1min":0.45900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93300000000000005,"5min":6.7160000000000002,"15min":6.7160000000000002},"last":0.503},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66100000000000003,"15min":0.80900000000000005},"min":{"1min":0.44600000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.878,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.45300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302576,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069264,"statfs":{"total":96636764160,"available":96326926336,"internally_reserved":0,"allocated":1220608,"data_stored":524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.31900000000000001,"15min":0.32800000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.59699999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.33600000000000002,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45400000000000001,"5min":0.746,"15min":0.746},"last":0.25800000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.45400000000000001,"15min":0.77100000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.63,"5min":11.967000000000001,"15min":11.967000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39000000000000001,"15min":0.498},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52700000000000002,"5min":4.1100000000000003,"15min":4.1100000000000003},"last":0.309}]},{"osd":3,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.41199999999999998,"15min":0.57899999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48799999999999999,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.50900000000000001,"15min":0.82199999999999995},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65900000000000003,"5min":11.935,"15min":11.935},"last":0.41199999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.80100000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.58999999999999997},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46200000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.33400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83599999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.58399999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81599999999999995,"5min":1.052,"15min":1.052},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64500000000000002,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85999999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.59799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574927,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302564,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069276,"statfs":{"total":96636764160,"available":96326938624,"internally_reserved":0,"allocated":1208320,"data_stored":518345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.31,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.311},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.31900000000000001},"min":{"1min":0.25,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.32900000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.23499999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38600000000000001,"15min":0.377},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.57099999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.34999999999999998},"min":{"1min":0.23699999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.35699999999999998,"15min":0.36199999999999999},"min":{"1min":0.26300000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.36599999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69999999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.82299999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64500000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.35899999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55900000000000005,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.76900000000000002,"5min":1.01,"15min":1.01},"last":0.56699999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.42099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85699999999999998,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.441,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.496,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.88400000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298468,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073372,"statfs":{"total":96636764160,"available":96331132928,"internally_reserved":0,"allocated":1208320,"data_stored":518345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58199999999999996,"5min":0.747,"15min":0.747},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.59799999999999998,"15min":0.79200000000000004},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69699999999999995,"5min":7.6719999999999997,"15min":7.6719999999999997},"last":0.48499999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.60299999999999998,"15min":0.79900000000000004},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60299999999999998,"5min":7.7160000000000002,"15min":7.7160000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.58599999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68100000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63600000000000001,"15min":0.81799999999999995},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":0.72799999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.58899999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61399999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.502,"15min":0.48399999999999999},"min":{"1min":0.34599999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.38400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77300000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.60399999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.41699999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72099999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.52600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.70599999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.58199999999999996},"min":{"1min":0.44700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298480,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073360,"statfs":{"total":96636764160,"available":96331120640,"internally_reserved":0,"allocated":1220608,"data_stored":524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.307,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63600000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.34300000000000003,"5min":0.215,"15min":0.215},"max":{"1min":0.70399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.46600000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.56200000000000006,"15min":0.65300000000000002},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55300000000000005,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68200000000000005,"5min":5.3109999999999999,"15min":5.3109999999999999},"last":0.55700000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56899999999999995,"15min":0.66100000000000003},"min":{"1min":0.32900000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64700000000000002,"5min":5.2960000000000003,"15min":5.2960000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72399999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.47799999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59499999999999997,"15min":0.69099999999999995},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70399999999999996,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":0.499},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59699999999999998,"15min":0.70399999999999996},"min":{"1min":0.40300000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70599999999999996,"5min":5.2850000000000001,"15min":5.2850000000000001},"last":0.56799999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48299999999999998,"15min":0.497},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.63800000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.311},{"interface":"front","average":{"1min":0.505,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65000000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.34499999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.65100000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71299999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.36199999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.33000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72699999999999998,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.38100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:40:46.846 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:40:46.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.970+0000 7f0d8901a700 1 Processor -- start 2023-10-10T20:40:46.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.970+0000 7f0d8901a700 1 -- start start 2023-10-10T20:40:46.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.971+0000 7f0d8901a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0d84159420 0x7f0d840b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:46.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.971+0000 7f0d8901a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d8415d230 0x7f0d8415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:46.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.971+0000 7f0d8901a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d840b2980 con 0x7f0d841597f0 2023-10-10T20:40:46.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.971+0000 7f0d8901a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d8415dbb0 con 0x7f0d84159420 2023-10-10T20:40:46.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.971+0000 7f0d8901a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d840b11d0 con 0x7f0d8415d230 2023-10-10T20:40:46.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.971+0000 7f0d837fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0d841597f0 0x7f0d840b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49200/0 (socket says 172.21.15.89:49200) 2023-10-10T20:40:46.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.971+0000 7f0d837fe700 1 -- 172.21.15.89:0/696160368 learned_addr learned my addr 172.21.15.89:0/696160368 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:46.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.971+0000 7f0d88819700 1 --2- 172.21.15.89:0/696160368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d8415d230 0x7f0d8415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:46.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.971+0000 7f0d83fff700 1 --2- 172.21.15.89:0/696160368 >> v2:172.21.15.111:3300/0 conn(0x7f0d84159420 0x7f0d840b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:46.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.971+0000 7f0d82ffd700 1 -- 172.21.15.89:0/696160368 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967936144 0 0) 0x7f0d840b2980 con 0x7f0d841597f0 2023-10-10T20:40:46.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.971+0000 7f0d82ffd700 1 -- 172.21.15.89:0/696160368 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d6c003570 con 0x7f0d841597f0 2023-10-10T20:40:46.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d82ffd700 1 -- 172.21.15.89:0/696160368 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 966407828 0 0) 0x7f0d6c003570 con 0x7f0d841597f0 2023-10-10T20:40:46.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d82ffd700 1 -- 172.21.15.89:0/696160368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d8415d230 msgr2=0x7f0d8415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:46.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d82ffd700 1 --2- 172.21.15.89:0/696160368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d8415d230 0x7f0d8415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:46.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d82ffd700 1 -- 172.21.15.89:0/696160368 >> v2:172.21.15.111:3300/0 conn(0x7f0d84159420 msgr2=0x7f0d840b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:46.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d82ffd700 1 --2- 172.21.15.89:0/696160368 >> v2:172.21.15.111:3300/0 conn(0x7f0d84159420 0x7f0d840b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:46.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d82ffd700 1 -- 172.21.15.89:0/696160368 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d840b14b0 con 0x7f0d841597f0 2023-10-10T20:40:46.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d82ffd700 1 -- 172.21.15.89:0/696160368 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f0d78001470 con 0x7f0d841597f0 2023-10-10T20:40:46.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d82ffd700 1 -- 172.21.15.89:0/696160368 >> v1:172.21.15.89:6789/0 conn(0x7f0d841597f0 legacy=0x7f0d840b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:46.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d82ffd700 1 --2- 172.21.15.89:0/696160368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d6c003a60 0x7f0d6c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:46.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d82ffd700 1 --2- 172.21.15.89:0/696160368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d8415d230 0x7f0d6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:46.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d82ffd700 1 --2- 172.21.15.89:0/696160368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d6c004d50 0x7f0d6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:46.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d82ffd700 1 -- 172.21.15.89:0/696160368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8415dbb0 con 0x7f0d6c003a60 2023-10-10T20:40:46.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d82ffd700 1 -- 172.21.15.89:0/696160368 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d840b11d0 con 0x7f0d6c004d50 2023-10-10T20:40:46.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d82ffd700 1 -- 172.21.15.89:0/696160368 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c00ba60 con 0x7f0d8415d230 2023-10-10T20:40:46.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d83fff700 1 --2- 172.21.15.89:0/696160368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d6c003a60 0x7f0d6c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:46.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d837fe700 1 --2- 172.21.15.89:0/696160368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d8415d230 0x7f0d6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:46.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d88819700 1 --2- 172.21.15.89:0/696160368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d6c004d50 0x7f0d6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:46.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d83fff700 1 -- 172.21.15.89:0/696160368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d8415d230 msgr2=0x7f0d6c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:46.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d83fff700 1 --2- 172.21.15.89:0/696160368 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d8415d230 0x7f0d6c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:46.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d83fff700 1 -- 172.21.15.89:0/696160368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d6c004d50 msgr2=0x7f0d6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:46.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d83fff700 1 --2- 172.21.15.89:0/696160368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d6c004d50 0x7f0d6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:46.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d83fff700 1 -- 172.21.15.89:0/696160368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c003570 con 0x7f0d6c003a60 2023-10-10T20:40:46.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d83fff700 1 --2- 172.21.15.89:0/696160368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d6c003a60 0x7f0d6c003e70 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009c00 tx=0x7f0d740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe024a0a18c1982 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:46.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d82ffd700 1 -- 172.21.15.89:0/696160368 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7400e6b0 con 0x7f0d6c003a60 2023-10-10T20:40:46.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d82ffd700 1 -- 172.21.15.89:0/696160368 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7400e810 con 0x7f0d6c003a60 2023-10-10T20:40:46.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.972+0000 7f0d82ffd700 1 -- 172.21.15.89:0/696160368 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7400ebd0 con 0x7f0d6c003a60 2023-10-10T20:40:46.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.973+0000 7f0d8901a700 1 -- 172.21.15.89:0/696160368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d6c003a60 msgr2=0x7f0d6c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:46.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.973+0000 7f0d8901a700 1 --2- 172.21.15.89:0/696160368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d6c003a60 0x7f0d6c003e70 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009c00 tx=0x7f0d740091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:46.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.973+0000 7f0d88819700 1 -- 172.21.15.89:0/696160368 reap_dead start 2023-10-10T20:40:46.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.973+0000 7f0d8901a700 1 -- 172.21.15.89:0/696160368 shutdown_connections 2023-10-10T20:40:46.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.973+0000 7f0d8901a700 1 -- 172.21.15.89:0/696160368 >> 172.21.15.89:0/696160368 conn(0x7f0d840d0660 msgr2=0x7f0d840d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:46.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.973+0000 7f0d8901a700 1 -- 172.21.15.89:0/696160368 shutdown_connections 2023-10-10T20:40:46.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.973+0000 7f0d8901a700 1 -- 172.21.15.89:0/696160368 wait complete. 2023-10-10T20:40:46.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.974+0000 7f0d8901a700 1 Processor -- start 2023-10-10T20:40:46.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.974+0000 7f0d8901a700 1 -- start start 2023-10-10T20:40:46.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.975+0000 7f0d8901a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d84149f00 0x7f0d8414a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:46.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.975+0000 7f0d8901a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d840c6c30 0x7f0d840c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:46.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.975+0000 7f0d8901a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d840c5400 0x7f0d840c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:46.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.975+0000 7f0d8901a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d840c6330 con 0x7f0d840c6c30 2023-10-10T20:40:46.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.975+0000 7f0d8901a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d840cb130 con 0x7f0d84149f00 2023-10-10T20:40:46.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.975+0000 7f0d8901a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d840cb6d0 con 0x7f0d840c5400 2023-10-10T20:40:46.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.975+0000 7f0d837fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d840c6c30 0x7f0d840c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:46.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.975+0000 7f0d837fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d840c6c30 0x7f0d840c70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39056/0 (socket says 172.21.15.89:39056) 2023-10-10T20:40:46.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.975+0000 7f0d837fe700 1 -- 172.21.15.89:0/2972173828 learned_addr learned my addr 172.21.15.89:0/2972173828 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:46.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.975+0000 7f0d83fff700 1 --2- 172.21.15.89:0/2972173828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d84149f00 0x7f0d8414a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:46.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.975+0000 7f0d88819700 1 --2- 172.21.15.89:0/2972173828 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d840c5400 0x7f0d840c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:46.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.975+0000 7f0d837fe700 1 -- 172.21.15.89:0/2972173828 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d840c5400 msgr2=0x7f0d840c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:46.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.975+0000 7f0d837fe700 1 --2- 172.21.15.89:0/2972173828 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d840c5400 0x7f0d840c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:46.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.975+0000 7f0d837fe700 1 -- 172.21.15.89:0/2972173828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d84149f00 msgr2=0x7f0d8414a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:46.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.975+0000 7f0d837fe700 1 --2- 172.21.15.89:0/2972173828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d84149f00 0x7f0d8414a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:46.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.975+0000 7f0d837fe700 1 -- 172.21.15.89:0/2972173828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d840c75f0 con 0x7f0d840c6c30 2023-10-10T20:40:46.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.975+0000 7f0d88819700 1 --2- 172.21.15.89:0/2972173828 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d840c5400 0x7f0d840c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:46.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.975+0000 7f0d837fe700 1 --2- 172.21.15.89:0/2972173828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d840c6c30 0x7f0d840c70b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f0d78000fd0 tx=0x7f0d780014c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:46.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.976+0000 7f0d6bfff700 1 -- 172.21.15.89:0/2972173828 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7800ee70 con 0x7f0d840c6c30 2023-10-10T20:40:46.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.976+0000 7f0d6bfff700 1 -- 172.21.15.89:0/2972173828 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7800c7e0 con 0x7f0d840c6c30 2023-10-10T20:40:46.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.976+0000 7f0d8901a700 1 -- 172.21.15.89:0/2972173828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d840c5d70 con 0x7f0d840c6c30 2023-10-10T20:40:46.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.976+0000 7f0d6bfff700 1 -- 172.21.15.89:0/2972173828 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7800cba0 con 0x7f0d840c6c30 2023-10-10T20:40:46.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.976+0000 7f0d8901a700 1 -- 172.21.15.89:0/2972173828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d840cb360 con 0x7f0d840c6c30 2023-10-10T20:40:46.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.977+0000 7f0d8901a700 1 -- 172.21.15.89:0/2972173828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8404c3c0 con 0x7f0d840c6c30 2023-10-10T20:40:46.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.977+0000 7f0d6bfff700 1 -- 172.21.15.89:0/2972173828 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f0d7801d460 con 0x7f0d840c6c30 2023-10-10T20:40:46.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.977+0000 7f0d6bfff700 1 --2- 172.21.15.89:0/2972173828 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0d70067290 0x7f0d70069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.977+0000 7f0d6bfff700 1 -- 172.21.15.89:0/2972173828 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0d78084a90 con 0x7f0d840c6c30 2023-10-10T20:40:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.977+0000 7f0d83fff700 1 --2- 172.21.15.89:0/2972173828 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0d70067290 0x7f0d70069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.978+0000 7f0d83fff700 1 --2- 172.21.15.89:0/2972173828 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0d70067290 0x7f0d70069740 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0d740091e0 tx=0x7f0d7401a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:46.982+0000 7f0d6bfff700 1 -- 172.21.15.89:0/2972173828 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d780518f0 con 0x7f0d840c6c30 2023-10-10T20:40:47.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.160+0000 7f0d8901a700 1 -- 172.21.15.89:0/2972173828 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d840cb810 con 0x7f0d70067290 2023-10-10T20:40:47.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.168+0000 7f0d6bfff700 1 -- 172.21.15.89:0/2972173828 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289774 (secure 0 0 0) 0x7f0d840cb810 con 0x7f0d70067290 2023-10-10T20:40:47.174 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:47.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.171+0000 7f0d8901a700 1 -- 172.21.15.89:0/2972173828 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0d70067290 msgr2=0x7f0d70069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:47.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.171+0000 7f0d8901a700 1 --2- 172.21.15.89:0/2972173828 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0d70067290 0x7f0d70069740 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0d740091e0 tx=0x7f0d7401a000 comp rx=0 tx=0).stop 2023-10-10T20:40:47.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.171+0000 7f0d8901a700 1 -- 172.21.15.89:0/2972173828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d840c6c30 msgr2=0x7f0d840c70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:47.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.171+0000 7f0d8901a700 1 --2- 172.21.15.89:0/2972173828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d840c6c30 0x7f0d840c70b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f0d78000fd0 tx=0x7f0d780014c0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.171+0000 7f0d8901a700 1 -- 172.21.15.89:0/2972173828 shutdown_connections 2023-10-10T20:40:47.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.171+0000 7f0d8901a700 1 --2- 172.21.15.89:0/2972173828 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0d840c5400 0x7f0d840c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.171+0000 7f0d8901a700 1 --2- 172.21.15.89:0/2972173828 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0d70067290 0x7f0d70069740 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.171+0000 7f0d8901a700 1 --2- 172.21.15.89:0/2972173828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d84149f00 0x7f0d8414a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.171+0000 7f0d8901a700 1 --2- 172.21.15.89:0/2972173828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d840c6c30 0x7f0d840c70b0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.171+0000 7f0d8901a700 1 -- 172.21.15.89:0/2972173828 >> 172.21.15.89:0/2972173828 conn(0x7f0d840d0660 msgr2=0x7f0d84156230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:47.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.171+0000 7f0d8901a700 1 -- 172.21.15.89:0/2972173828 shutdown_connections 2023-10-10T20:40:47.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.171+0000 7f0d8901a700 1 -- 172.21.15.89:0/2972173828 wait complete. 2023-10-10T20:40:47.179 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:40:47.186 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2023-10-10T20:40:46.726569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2417924,"kb_used_data":10948,"kb_used_omap":0,"kb_used_meta":2406912,"kb_avail":752556796,"statfs":{"total":773094113280,"available":770618159104,"internally_reserved":0,"allocated":11210752,"data_stored":5485659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2464677888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.479157"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.288234+0000","last_change":"2023-10-10T20:40:38.276332+0000","last_active":"2023-10-10T20:40:38.288234+0000","last_peered":"2023-10-10T20:40:38.288234+0000","last_clean":"2023-10-10T20:40:38.288234+0000","last_became_active":"2023-10-10T20:40:38.275790+0000","last_became_peered":"2023-10-10T20:40:38.275790+0000","last_unstale":"2023-10-10T20:40:38.288234+0000","last_undegraded":"2023-10-10T20:40:38.288234+0000","last_fullsized":"2023-10-10T20:40:38.288234+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:50.192386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.273632+0000","last_change":"2023-10-10T20:40:38.271170+0000","last_active":"2023-10-10T20:40:38.273632+0000","last_peered":"2023-10-10T20:40:38.273632+0000","last_clean":"2023-10-10T20:40:38.273632+0000","last_became_active":"2023-10-10T20:40:38.270905+0000","last_became_peered":"2023-10-10T20:40:38.270905+0000","last_unstale":"2023-10-10T20:40:38.273632+0000","last_undegraded":"2023-10-10T20:40:38.273632+0000","last_fullsized":"2023-10-10T20:40:38.273632+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:55:53.986623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.274418+0000","last_change":"2023-10-10T20:40:38.271515+0000","last_active":"2023-10-10T20:40:38.274418+0000","last_peered":"2023-10-10T20:40:38.274418+0000","last_clean":"2023-10-10T20:40:38.274418+0000","last_became_active":"2023-10-10T20:40:38.271206+0000","last_became_peered":"2023-10-10T20:40:38.271206+0000","last_unstale":"2023-10-10T20:40:38.274418+0000","last_undegraded":"2023-10-10T20:40:38.274418+0000","last_fullsized":"2023-10-10T20:40:38.274418+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:14.044818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.288079+0000","last_change":"2023-10-10T20:40:38.272264+0000","last_active":"2023-10-10T20:40:38.288079+0000","last_peered":"2023-10-10T20:40:38.288079+0000","last_clean":"2023-10-10T20:40:38.288079+0000","last_became_active":"2023-10-10T20:40:38.272009+0000","last_became_peered":"2023-10-10T20:40:38.272009+0000","last_unstale":"2023-10-10T20:40:38.288079+0000","last_undegraded":"2023-10-10T20:40:38.288079+0000","last_fullsized":"2023-10-10T20:40:38.288079+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:27.887540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.268943+0000","last_change":"2023-10-10T20:40:38.265220+0000","last_active":"2023-10-10T20:40:38.268943+0000","last_peered":"2023-10-10T20:40:38.268943+0000","last_clean":"2023-10-10T20:40:38.268943+0000","last_became_active":"2023-10-10T20:40:38.264954+0000","last_became_peered":"2023-10-10T20:40:38.264954+0000","last_unstale":"2023-10-10T20:40:38.268943+0000","last_undegraded":"2023-10-10T20:40:38.268943+0000","last_fullsized":"2023-10-10T20:40:38.268943+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:59:54.929275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.261767+0000","last_change":"2023-10-10T20:40:38.257802+0000","last_active":"2023-10-10T20:40:38.261767+0000","last_peered":"2023-10-10T20:40:38.261767+0000","last_clean":"2023-10-10T20:40:38.261767+0000","last_became_active":"2023-10-10T20:40:38.257566+0000","last_became_peered":"2023-10-10T20:40:38.257566+0000","last_unstale":"2023-10-10T20:40:38.261767+0000","last_undegraded":"2023-10-10T20:40:38.261767+0000","last_fullsized":"2023-10-10T20:40:38.261767+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:29:14.456681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.280698+0000","last_change":"2023-10-10T20:40:38.274271+0000","last_active":"2023-10-10T20:40:38.280698+0000","last_peered":"2023-10-10T20:40:38.280698+0000","last_clean":"2023-10-10T20:40:38.280698+0000","last_became_active":"2023-10-10T20:40:38.274014+0000","last_became_peered":"2023-10-10T20:40:38.274014+0000","last_unstale":"2023-10-10T20:40:38.280698+0000","last_undegraded":"2023-10-10T20:40:38.280698+0000","last_fullsized":"2023-10-10T20:40:38.280698+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:53:07.253745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.263209+0000","last_change":"2023-10-10T20:40:38.260510+0000","last_active":"2023-10-10T20:40:38.263209+0000","last_peered":"2023-10-10T20:40:38.263209+0000","last_clean":"2023-10-10T20:40:38.263209+0000","last_became_active":"2023-10-10T20:40:38.259825+0000","last_became_peered":"2023-10-10T20:40:38.259825+0000","last_unstale":"2023-10-10T20:40:38.263209+0000","last_undegraded":"2023-10-10T20:40:38.263209+0000","last_fullsized":"2023-10-10T20:40:38.263209+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:38:00.681114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.269218+0000","last_change":"2023-10-10T20:40:38.260762+0000","last_active":"2023-10-10T20:40:38.269218+0000","last_peered":"2023-10-10T20:40:38.269218+0000","last_clean":"2023-10-10T20:40:38.269218+0000","last_became_active":"2023-10-10T20:40:38.260474+0000","last_became_peered":"2023-10-10T20:40:38.260474+0000","last_unstale":"2023-10-10T20:40:38.269218+0000","last_undegraded":"2023-10-10T20:40:38.269218+0000","last_fullsized":"2023-10-10T20:40:38.269218+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:32:55.669003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.272891+0000","last_change":"2023-10-10T20:40:38.267087+0000","last_active":"2023-10-10T20:40:38.272891+0000","last_peered":"2023-10-10T20:40:38.272891+0000","last_clean":"2023-10-10T20:40:38.272891+0000","last_became_active":"2023-10-10T20:40:38.266801+0000","last_became_peered":"2023-10-10T20:40:38.266801+0000","last_unstale":"2023-10-10T20:40:38.272891+0000","last_undegraded":"2023-10-10T20:40:38.272891+0000","last_fullsized":"2023-10-10T20:40:38.272891+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:44:39.396659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.281473+0000","last_change":"2023-10-10T20:40:38.278270+0000","last_active":"2023-10-10T20:40:38.281473+0000","last_peered":"2023-10-10T20:40:38.281473+0000","last_clean":"2023-10-10T20:40:38.281473+0000","last_became_active":"2023-10-10T20:40:38.277948+0000","last_became_peered":"2023-10-10T20:40:38.277948+0000","last_unstale":"2023-10-10T20:40:38.281473+0000","last_undegraded":"2023-10-10T20:40:38.281473+0000","last_fullsized":"2023-10-10T20:40:38.281473+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:09:20.495043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.276638+0000","last_change":"2023-10-10T20:40:38.265231+0000","last_active":"2023-10-10T20:40:38.276638+0000","last_peered":"2023-10-10T20:40:38.276638+0000","last_clean":"2023-10-10T20:40:38.276638+0000","last_became_active":"2023-10-10T20:40:38.264966+0000","last_became_peered":"2023-10-10T20:40:38.264966+0000","last_unstale":"2023-10-10T20:40:38.276638+0000","last_undegraded":"2023-10-10T20:40:38.276638+0000","last_fullsized":"2023-10-10T20:40:38.276638+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:57:51.009533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.274683+0000","last_change":"2023-10-10T20:40:38.266936+0000","last_active":"2023-10-10T20:40:38.274683+0000","last_peered":"2023-10-10T20:40:38.274683+0000","last_clean":"2023-10-10T20:40:38.274683+0000","last_became_active":"2023-10-10T20:40:38.265966+0000","last_became_peered":"2023-10-10T20:40:38.265966+0000","last_unstale":"2023-10-10T20:40:38.274683+0000","last_undegraded":"2023-10-10T20:40:38.274683+0000","last_fullsized":"2023-10-10T20:40:38.274683+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:37:04.622821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.280643+0000","last_change":"2023-10-10T20:40:38.274986+0000","last_active":"2023-10-10T20:40:38.280643+0000","last_peered":"2023-10-10T20:40:38.280643+0000","last_clean":"2023-10-10T20:40:38.280643+0000","last_became_active":"2023-10-10T20:40:38.274567+0000","last_became_peered":"2023-10-10T20:40:38.274567+0000","last_unstale":"2023-10-10T20:40:38.280643+0000","last_undegraded":"2023-10-10T20:40:38.280643+0000","last_fullsized":"2023-10-10T20:40:38.280643+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:23.463451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.263857+0000","last_change":"2023-10-10T20:40:38.258968+0000","last_active":"2023-10-10T20:40:38.263857+0000","last_peered":"2023-10-10T20:40:38.263857+0000","last_clean":"2023-10-10T20:40:38.263857+0000","last_became_active":"2023-10-10T20:40:38.258690+0000","last_became_peered":"2023-10-10T20:40:38.258690+0000","last_unstale":"2023-10-10T20:40:38.263857+0000","last_undegraded":"2023-10-10T20:40:38.263857+0000","last_fullsized":"2023-10-10T20:40:38.263857+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:03.966254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.261458+0000","last_change":"2023-10-10T20:40:38.259958+0000","last_active":"2023-10-10T20:40:38.261458+0000","last_peered":"2023-10-10T20:40:38.261458+0000","last_clean":"2023-10-10T20:40:38.261458+0000","last_became_active":"2023-10-10T20:40:38.259733+0000","last_became_peered":"2023-10-10T20:40:38.259733+0000","last_unstale":"2023-10-10T20:40:38.261458+0000","last_undegraded":"2023-10-10T20:40:38.261458+0000","last_fullsized":"2023-10-10T20:40:38.261458+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:30.755918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.281488+0000","last_change":"2023-10-10T20:40:38.278274+0000","last_active":"2023-10-10T20:40:38.281488+0000","last_peered":"2023-10-10T20:40:38.281488+0000","last_clean":"2023-10-10T20:40:38.281488+0000","last_became_active":"2023-10-10T20:40:38.277969+0000","last_became_peered":"2023-10-10T20:40:38.277969+0000","last_unstale":"2023-10-10T20:40:38.281488+0000","last_undegraded":"2023-10-10T20:40:38.281488+0000","last_fullsized":"2023-10-10T20:40:38.281488+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:56:54.788047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.279577+0000","last_change":"2023-10-10T20:40:38.274206+0000","last_active":"2023-10-10T20:40:38.279577+0000","last_peered":"2023-10-10T20:40:38.279577+0000","last_clean":"2023-10-10T20:40:38.279577+0000","last_became_active":"2023-10-10T20:40:38.273768+0000","last_became_peered":"2023-10-10T20:40:38.273768+0000","last_unstale":"2023-10-10T20:40:38.279577+0000","last_undegraded":"2023-10-10T20:40:38.279577+0000","last_fullsized":"2023-10-10T20:40:38.279577+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:24:16.195290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.274401+0000","last_change":"2023-10-10T20:40:38.267023+0000","last_active":"2023-10-10T20:40:38.274401+0000","last_peered":"2023-10-10T20:40:38.274401+0000","last_clean":"2023-10-10T20:40:38.274401+0000","last_became_active":"2023-10-10T20:40:38.266760+0000","last_became_peered":"2023-10-10T20:40:38.266760+0000","last_unstale":"2023-10-10T20:40:38.274401+0000","last_undegraded":"2023-10-10T20:40:38.274401+0000","last_fullsized":"2023-10-10T20:40:38.274401+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:43:19.739277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.281314+0000","last_change":"2023-10-10T20:40:38.275304+0000","last_active":"2023-10-10T20:40:38.281314+0000","last_peered":"2023-10-10T20:40:38.281314+0000","last_clean":"2023-10-10T20:40:38.281314+0000","last_became_active":"2023-10-10T20:40:38.275014+0000","last_became_peered":"2023-10-10T20:40:38.275014+0000","last_unstale":"2023-10-10T20:40:38.281314+0000","last_undegraded":"2023-10-10T20:40:38.281314+0000","last_fullsized":"2023-10-10T20:40:38.281314+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:43:19.739277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.272860+0000","last_change":"2023-10-10T20:40:38.271029+0000","last_active":"2023-10-10T20:40:38.272860+0000","last_peered":"2023-10-10T20:40:38.272860+0000","last_clean":"2023-10-10T20:40:38.272860+0000","last_became_active":"2023-10-10T20:40:38.270769+0000","last_became_peered":"2023-10-10T20:40:38.270769+0000","last_unstale":"2023-10-10T20:40:38.272860+0000","last_undegraded":"2023-10-10T20:40:38.272860+0000","last_fullsized":"2023-10-10T20:40:38.272860+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:39:38.169839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.278053+0000","last_change":"2023-10-10T20:40:38.275313+0000","last_active":"2023-10-10T20:40:38.278053+0000","last_peered":"2023-10-10T20:40:38.278053+0000","last_clean":"2023-10-10T20:40:38.278053+0000","last_became_active":"2023-10-10T20:40:38.274659+0000","last_became_peered":"2023-10-10T20:40:38.274659+0000","last_unstale":"2023-10-10T20:40:38.278053+0000","last_undegraded":"2023-10-10T20:40:38.278053+0000","last_fullsized":"2023-10-10T20:40:38.278053+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:54.041748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.279767+0000","last_change":"2023-10-10T20:40:38.274500+0000","last_active":"2023-10-10T20:40:38.279767+0000","last_peered":"2023-10-10T20:40:38.279767+0000","last_clean":"2023-10-10T20:40:38.279767+0000","last_became_active":"2023-10-10T20:40:38.274227+0000","last_became_peered":"2023-10-10T20:40:38.274227+0000","last_unstale":"2023-10-10T20:40:38.279767+0000","last_undegraded":"2023-10-10T20:40:38.279767+0000","last_fullsized":"2023-10-10T20:40:38.279767+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:24:16.195290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.267714+0000","last_change":"2023-10-10T20:40:38.259852+0000","last_active":"2023-10-10T20:40:38.267714+0000","last_peered":"2023-10-10T20:40:38.267714+0000","last_clean":"2023-10-10T20:40:38.267714+0000","last_became_active":"2023-10-10T20:40:38.259385+0000","last_became_peered":"2023-10-10T20:40:38.259385+0000","last_unstale":"2023-10-10T20:40:38.267714+0000","last_undegraded":"2023-10-10T20:40:38.267714+0000","last_fullsized":"2023-10-10T20:40:38.267714+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:58:16.915031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.281499+0000","last_change":"2023-10-10T20:40:38.278565+0000","last_active":"2023-10-10T20:40:38.281499+0000","last_peered":"2023-10-10T20:40:38.281499+0000","last_clean":"2023-10-10T20:40:38.281499+0000","last_became_active":"2023-10-10T20:40:38.278281+0000","last_became_peered":"2023-10-10T20:40:38.278281+0000","last_unstale":"2023-10-10T20:40:38.281499+0000","last_undegraded":"2023-10-10T20:40:38.281499+0000","last_fullsized":"2023-10-10T20:40:38.281499+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:14.934213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.272401+0000","last_change":"2023-10-10T20:40:38.261551+0000","last_active":"2023-10-10T20:40:38.272401+0000","last_peered":"2023-10-10T20:40:38.272401+0000","last_clean":"2023-10-10T20:40:38.272401+0000","last_became_active":"2023-10-10T20:40:38.261287+0000","last_became_peered":"2023-10-10T20:40:38.261287+0000","last_unstale":"2023-10-10T20:40:38.272401+0000","last_undegraded":"2023-10-10T20:40:38.272401+0000","last_fullsized":"2023-10-10T20:40:38.272401+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:51:40.276320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.264737+0000","last_change":"2023-10-10T20:40:38.261677+0000","last_active":"2023-10-10T20:40:38.264737+0000","last_peered":"2023-10-10T20:40:38.264737+0000","last_clean":"2023-10-10T20:40:38.264737+0000","last_became_active":"2023-10-10T20:40:38.261425+0000","last_became_peered":"2023-10-10T20:40:38.261425+0000","last_unstale":"2023-10-10T20:40:38.264737+0000","last_undegraded":"2023-10-10T20:40:38.264737+0000","last_fullsized":"2023-10-10T20:40:38.264737+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:24:13.463965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.268966+0000","last_change":"2023-10-10T20:40:38.263621+0000","last_active":"2023-10-10T20:40:38.268966+0000","last_peered":"2023-10-10T20:40:38.268966+0000","last_clean":"2023-10-10T20:40:38.268966+0000","last_became_active":"2023-10-10T20:40:38.263363+0000","last_became_peered":"2023-10-10T20:40:38.263363+0000","last_unstale":"2023-10-10T20:40:38.268966+0000","last_undegraded":"2023-10-10T20:40:38.268966+0000","last_fullsized":"2023-10-10T20:40:38.268966+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:28:37.819892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.279742+0000","last_change":"2023-10-10T20:40:38.274428+0000","last_active":"2023-10-10T20:40:38.279742+0000","last_peered":"2023-10-10T20:40:38.279742+0000","last_clean":"2023-10-10T20:40:38.279742+0000","last_became_active":"2023-10-10T20:40:38.274143+0000","last_became_peered":"2023-10-10T20:40:38.274143+0000","last_unstale":"2023-10-10T20:40:38.279742+0000","last_undegraded":"2023-10-10T20:40:38.279742+0000","last_fullsized":"2023-10-10T20:40:38.279742+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:39:12.325954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.261209+0000","last_change":"2023-10-10T20:40:38.260231+0000","last_active":"2023-10-10T20:40:38.261209+0000","last_peered":"2023-10-10T20:40:38.261209+0000","last_clean":"2023-10-10T20:40:38.261209+0000","last_became_active":"2023-10-10T20:40:38.259869+0000","last_became_peered":"2023-10-10T20:40:38.259869+0000","last_unstale":"2023-10-10T20:40:38.261209+0000","last_undegraded":"2023-10-10T20:40:38.261209+0000","last_fullsized":"2023-10-10T20:40:38.261209+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:03.966254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.272408+0000","last_change":"2023-10-10T20:40:38.261329+0000","last_active":"2023-10-10T20:40:38.272408+0000","last_peered":"2023-10-10T20:40:38.272408+0000","last_clean":"2023-10-10T20:40:38.272408+0000","last_became_active":"2023-10-10T20:40:38.261065+0000","last_became_peered":"2023-10-10T20:40:38.261065+0000","last_unstale":"2023-10-10T20:40:38.272408+0000","last_undegraded":"2023-10-10T20:40:38.272408+0000","last_fullsized":"2023-10-10T20:40:38.272408+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:07.692167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.264092+0000","last_change":"2023-10-10T20:40:38.260642+0000","last_active":"2023-10-10T20:40:38.264092+0000","last_peered":"2023-10-10T20:40:38.264092+0000","last_clean":"2023-10-10T20:40:38.264092+0000","last_became_active":"2023-10-10T20:40:38.260394+0000","last_became_peered":"2023-10-10T20:40:38.260394+0000","last_unstale":"2023-10-10T20:40:38.264092+0000","last_undegraded":"2023-10-10T20:40:38.264092+0000","last_fullsized":"2023-10-10T20:40:38.264092+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:13:53.110967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.269005+0000","last_change":"2023-10-10T20:40:38.266686+0000","last_active":"2023-10-10T20:40:38.269005+0000","last_peered":"2023-10-10T20:40:38.269005+0000","last_clean":"2023-10-10T20:40:38.269005+0000","last_became_active":"2023-10-10T20:40:38.266348+0000","last_became_peered":"2023-10-10T20:40:38.266348+0000","last_unstale":"2023-10-10T20:40:38.269005+0000","last_undegraded":"2023-10-10T20:40:38.269005+0000","last_fullsized":"2023-10-10T20:40:38.269005+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:45:44.964754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"99'4","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.281190+0000","last_change":"2023-10-10T20:40:37.265989+0000","last_active":"2023-10-10T20:40:38.281190+0000","last_peered":"2023-10-10T20:40:38.281190+0000","last_clean":"2023-10-10T20:40:38.281190+0000","last_became_active":"2023-10-10T20:40:37.265518+0000","last_became_peered":"2023-10-10T20:40:37.265518+0000","last_unstale":"2023-10-10T20:40:38.281190+0000","last_undegraded":"2023-10-10T20:40:38.281190+0000","last_fullsized":"2023-10-10T20:40:38.281190+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:10:44.403291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.280999+0000","last_change":"2023-10-10T20:40:38.276504+0000","last_active":"2023-10-10T20:40:38.280999+0000","last_peered":"2023-10-10T20:40:38.280999+0000","last_clean":"2023-10-10T20:40:38.280999+0000","last_became_active":"2023-10-10T20:40:38.276231+0000","last_became_peered":"2023-10-10T20:40:38.276231+0000","last_unstale":"2023-10-10T20:40:38.280999+0000","last_undegraded":"2023-10-10T20:40:38.280999+0000","last_fullsized":"2023-10-10T20:40:38.280999+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:37:04.622821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"99'1","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.281125+0000","last_change":"2023-10-10T20:40:37.268252+0000","last_active":"2023-10-10T20:40:38.281125+0000","last_peered":"2023-10-10T20:40:38.281125+0000","last_clean":"2023-10-10T20:40:38.281125+0000","last_became_active":"2023-10-10T20:40:37.267982+0000","last_became_peered":"2023-10-10T20:40:37.267982+0000","last_unstale":"2023-10-10T20:40:38.281125+0000","last_undegraded":"2023-10-10T20:40:38.281125+0000","last_fullsized":"2023-10-10T20:40:38.281125+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:14:28.701104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.269024+0000","last_change":"2023-10-10T20:40:38.266824+0000","last_active":"2023-10-10T20:40:38.269024+0000","last_peered":"2023-10-10T20:40:38.269024+0000","last_clean":"2023-10-10T20:40:38.269024+0000","last_became_active":"2023-10-10T20:40:38.266532+0000","last_became_peered":"2023-10-10T20:40:38.266532+0000","last_unstale":"2023-10-10T20:40:38.269024+0000","last_undegraded":"2023-10-10T20:40:38.269024+0000","last_fullsized":"2023-10-10T20:40:38.269024+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:05:13.348836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"99'2","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.280075+0000","last_change":"2023-10-10T20:40:37.263019+0000","last_active":"2023-10-10T20:40:38.280075+0000","last_peered":"2023-10-10T20:40:38.280075+0000","last_clean":"2023-10-10T20:40:38.280075+0000","last_became_active":"2023-10-10T20:40:37.262760+0000","last_became_peered":"2023-10-10T20:40:37.262760+0000","last_unstale":"2023-10-10T20:40:38.280075+0000","last_undegraded":"2023-10-10T20:40:38.280075+0000","last_fullsized":"2023-10-10T20:40:38.280075+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:12:55.235702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.274379+0000","last_change":"2023-10-10T20:40:38.266290+0000","last_active":"2023-10-10T20:40:38.274379+0000","last_peered":"2023-10-10T20:40:38.274379+0000","last_clean":"2023-10-10T20:40:38.274379+0000","last_became_active":"2023-10-10T20:40:38.266008+0000","last_became_peered":"2023-10-10T20:40:38.266008+0000","last_unstale":"2023-10-10T20:40:38.274379+0000","last_undegraded":"2023-10-10T20:40:38.274379+0000","last_fullsized":"2023-10-10T20:40:38.274379+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:24:16.195290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"99'2","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.279904+0000","last_change":"2023-10-10T20:40:37.263048+0000","last_active":"2023-10-10T20:40:38.279904+0000","last_peered":"2023-10-10T20:40:38.279904+0000","last_clean":"2023-10-10T20:40:38.279904+0000","last_became_active":"2023-10-10T20:40:37.262792+0000","last_became_peered":"2023-10-10T20:40:37.262792+0000","last_unstale":"2023-10-10T20:40:38.279904+0000","last_undegraded":"2023-10-10T20:40:38.279904+0000","last_fullsized":"2023-10-10T20:40:38.279904+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:43:02.879509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.261464+0000","last_change":"2023-10-10T20:40:38.257475+0000","last_active":"2023-10-10T20:40:38.261464+0000","last_peered":"2023-10-10T20:40:38.261464+0000","last_clean":"2023-10-10T20:40:38.261464+0000","last_became_active":"2023-10-10T20:40:38.257245+0000","last_became_peered":"2023-10-10T20:40:38.257245+0000","last_unstale":"2023-10-10T20:40:38.261464+0000","last_undegraded":"2023-10-10T20:40:38.261464+0000","last_fullsized":"2023-10-10T20:40:38.261464+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:14.044818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"99'1","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.280684+0000","last_change":"2023-10-10T20:40:37.262612+0000","last_active":"2023-10-10T20:40:38.280684+0000","last_peered":"2023-10-10T20:40:38.280684+0000","last_clean":"2023-10-10T20:40:38.280684+0000","last_became_active":"2023-10-10T20:40:37.262298+0000","last_became_peered":"2023-10-10T20:40:37.262298+0000","last_unstale":"2023-10-10T20:40:38.280684+0000","last_undegraded":"2023-10-10T20:40:38.280684+0000","last_fullsized":"2023-10-10T20:40:38.280684+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:33:04.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.272742+0000","last_change":"2023-10-10T20:40:38.270541+0000","last_active":"2023-10-10T20:40:38.272742+0000","last_peered":"2023-10-10T20:40:38.272742+0000","last_clean":"2023-10-10T20:40:38.272742+0000","last_became_active":"2023-10-10T20:40:38.270251+0000","last_became_peered":"2023-10-10T20:40:38.270251+0000","last_unstale":"2023-10-10T20:40:38.272742+0000","last_undegraded":"2023-10-10T20:40:38.272742+0000","last_fullsized":"2023-10-10T20:40:38.272742+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:29:39.101585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"99'1","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.275635+0000","last_change":"2023-10-10T20:40:37.263207+0000","last_active":"2023-10-10T20:40:38.275635+0000","last_peered":"2023-10-10T20:40:38.275635+0000","last_clean":"2023-10-10T20:40:38.275635+0000","last_became_active":"2023-10-10T20:40:37.262945+0000","last_became_peered":"2023-10-10T20:40:37.262945+0000","last_unstale":"2023-10-10T20:40:38.275635+0000","last_undegraded":"2023-10-10T20:40:38.275635+0000","last_fullsized":"2023-10-10T20:40:38.275635+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:55:52.981786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.273212+0000","last_change":"2023-10-10T20:40:38.262840+0000","last_active":"2023-10-10T20:40:38.273212+0000","last_peered":"2023-10-10T20:40:38.273212+0000","last_clean":"2023-10-10T20:40:38.273212+0000","last_became_active":"2023-10-10T20:40:38.262552+0000","last_became_peered":"2023-10-10T20:40:38.262552+0000","last_unstale":"2023-10-10T20:40:38.273212+0000","last_undegraded":"2023-10-10T20:40:38.273212+0000","last_fullsized":"2023-10-10T20:40:38.273212+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:00:49.763784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.264459+0000","last_change":"2023-10-10T20:40:37.269620+0000","last_active":"2023-10-10T20:40:38.264459+0000","last_peered":"2023-10-10T20:40:38.264459+0000","last_clean":"2023-10-10T20:40:38.264459+0000","last_became_active":"2023-10-10T20:40:37.269347+0000","last_became_peered":"2023-10-10T20:40:37.269347+0000","last_unstale":"2023-10-10T20:40:38.264459+0000","last_undegraded":"2023-10-10T20:40:38.264459+0000","last_fullsized":"2023-10-10T20:40:38.264459+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:26:39.104435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.280560+0000","last_change":"2023-10-10T20:40:38.269635+0000","last_active":"2023-10-10T20:40:38.280560+0000","last_peered":"2023-10-10T20:40:38.280560+0000","last_clean":"2023-10-10T20:40:38.280560+0000","last_became_active":"2023-10-10T20:40:38.269364+0000","last_became_peered":"2023-10-10T20:40:38.269364+0000","last_unstale":"2023-10-10T20:40:38.280560+0000","last_undegraded":"2023-10-10T20:40:38.280560+0000","last_fullsized":"2023-10-10T20:40:38.280560+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:12.802014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.254415+0000","last_change":"2023-10-10T20:40:37.264965+0000","last_active":"2023-10-10T20:40:38.254415+0000","last_peered":"2023-10-10T20:40:38.254415+0000","last_clean":"2023-10-10T20:40:38.254415+0000","last_became_active":"2023-10-10T20:40:37.264288+0000","last_became_peered":"2023-10-10T20:40:37.264288+0000","last_unstale":"2023-10-10T20:40:38.254415+0000","last_undegraded":"2023-10-10T20:40:38.254415+0000","last_fullsized":"2023-10-10T20:40:38.254415+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:09:19.490206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.277277+0000","last_change":"2023-10-10T20:40:38.267441+0000","last_active":"2023-10-10T20:40:38.277277+0000","last_peered":"2023-10-10T20:40:38.277277+0000","last_clean":"2023-10-10T20:40:38.277277+0000","last_became_active":"2023-10-10T20:40:38.267163+0000","last_became_peered":"2023-10-10T20:40:38.267163+0000","last_unstale":"2023-10-10T20:40:38.277277+0000","last_undegraded":"2023-10-10T20:40:38.277277+0000","last_fullsized":"2023-10-10T20:40:38.277277+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:05:13.348836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.261014+0000","last_change":"2023-10-10T20:40:37.264630+0000","last_active":"2023-10-10T20:40:38.261014+0000","last_peered":"2023-10-10T20:40:38.261014+0000","last_clean":"2023-10-10T20:40:38.261014+0000","last_became_active":"2023-10-10T20:40:37.264244+0000","last_became_peered":"2023-10-10T20:40:37.264244+0000","last_unstale":"2023-10-10T20:40:38.261014+0000","last_undegraded":"2023-10-10T20:40:38.261014+0000","last_fullsized":"2023-10-10T20:40:38.261014+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:06.742295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.280859+0000","last_change":"2023-10-10T20:40:38.275281+0000","last_active":"2023-10-10T20:40:38.280859+0000","last_peered":"2023-10-10T20:40:38.280859+0000","last_clean":"2023-10-10T20:40:38.280859+0000","last_became_active":"2023-10-10T20:40:38.274983+0000","last_became_peered":"2023-10-10T20:40:38.274983+0000","last_unstale":"2023-10-10T20:40:38.280859+0000","last_undegraded":"2023-10-10T20:40:38.280859+0000","last_fullsized":"2023-10-10T20:40:38.280859+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:39:12.325954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.251780+0000","last_change":"2023-10-10T20:40:37.261554+0000","last_active":"2023-10-10T20:40:38.251780+0000","last_peered":"2023-10-10T20:40:38.251780+0000","last_clean":"2023-10-10T20:40:38.251780+0000","last_became_active":"2023-10-10T20:40:37.261298+0000","last_became_peered":"2023-10-10T20:40:37.261298+0000","last_unstale":"2023-10-10T20:40:38.251780+0000","last_undegraded":"2023-10-10T20:40:38.251780+0000","last_fullsized":"2023-10-10T20:40:38.251780+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:26.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.274931+0000","last_change":"2023-10-10T20:40:38.273399+0000","last_active":"2023-10-10T20:40:38.274931+0000","last_peered":"2023-10-10T20:40:38.274931+0000","last_clean":"2023-10-10T20:40:38.274931+0000","last_became_active":"2023-10-10T20:40:38.272777+0000","last_became_peered":"2023-10-10T20:40:38.272777+0000","last_unstale":"2023-10-10T20:40:38.274931+0000","last_undegraded":"2023-10-10T20:40:38.274931+0000","last_fullsized":"2023-10-10T20:40:38.274931+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:20:35.134043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"99'3","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.287342+0000","last_change":"2023-10-10T20:40:37.261773+0000","last_active":"2023-10-10T20:40:38.287342+0000","last_peered":"2023-10-10T20:40:38.287342+0000","last_clean":"2023-10-10T20:40:38.287342+0000","last_became_active":"2023-10-10T20:40:37.261546+0000","last_became_peered":"2023-10-10T20:40:37.261546+0000","last_unstale":"2023-10-10T20:40:38.287342+0000","last_undegraded":"2023-10-10T20:40:38.287342+0000","last_fullsized":"2023-10-10T20:40:38.287342+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:24:15.190453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.261016+0000","last_change":"2023-10-10T20:40:38.259860+0000","last_active":"2023-10-10T20:40:38.261016+0000","last_peered":"2023-10-10T20:40:38.261016+0000","last_clean":"2023-10-10T20:40:38.261016+0000","last_became_active":"2023-10-10T20:40:38.259466+0000","last_became_peered":"2023-10-10T20:40:38.259466+0000","last_unstale":"2023-10-10T20:40:38.261016+0000","last_undegraded":"2023-10-10T20:40:38.261016+0000","last_fullsized":"2023-10-10T20:40:38.261016+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:26:40.109272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"99'6","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.290890+0000","last_change":"2023-10-10T20:40:37.258355+0000","last_active":"2023-10-10T20:40:38.290890+0000","last_peered":"2023-10-10T20:40:38.290890+0000","last_clean":"2023-10-10T20:40:38.290890+0000","last_became_active":"2023-10-10T20:40:37.258111+0000","last_became_peered":"2023-10-10T20:40:37.258111+0000","last_unstale":"2023-10-10T20:40:38.290890+0000","last_undegraded":"2023-10-10T20:40:38.290890+0000","last_fullsized":"2023-10-10T20:40:38.290890+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:56:53.783210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.263338+0000","last_change":"2023-10-10T20:40:38.257179+0000","last_active":"2023-10-10T20:40:38.263338+0000","last_peered":"2023-10-10T20:40:38.263338+0000","last_clean":"2023-10-10T20:40:38.263338+0000","last_became_active":"2023-10-10T20:40:38.256898+0000","last_became_peered":"2023-10-10T20:40:38.256898+0000","last_unstale":"2023-10-10T20:40:38.263338+0000","last_undegraded":"2023-10-10T20:40:38.263338+0000","last_fullsized":"2023-10-10T20:40:38.263338+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:14.044818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"99'2","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.281513+0000","last_change":"2023-10-10T20:40:37.266663+0000","last_active":"2023-10-10T20:40:38.281513+0000","last_peered":"2023-10-10T20:40:38.281513+0000","last_clean":"2023-10-10T20:40:38.281513+0000","last_became_active":"2023-10-10T20:40:37.266402+0000","last_became_peered":"2023-10-10T20:40:37.266402+0000","last_unstale":"2023-10-10T20:40:38.281513+0000","last_undegraded":"2023-10-10T20:40:38.281513+0000","last_fullsized":"2023-10-10T20:40:38.281513+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:43:14.351198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.267637+0000","last_change":"2023-10-10T20:40:38.259703+0000","last_active":"2023-10-10T20:40:38.267637+0000","last_peered":"2023-10-10T20:40:38.267637+0000","last_clean":"2023-10-10T20:40:38.267637+0000","last_became_active":"2023-10-10T20:40:38.259223+0000","last_became_peered":"2023-10-10T20:40:38.259223+0000","last_unstale":"2023-10-10T20:40:38.267637+0000","last_undegraded":"2023-10-10T20:40:38.267637+0000","last_fullsized":"2023-10-10T20:40:38.267637+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:43:08.406828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.267913+0000","last_change":"2023-10-10T20:40:37.265713+0000","last_active":"2023-10-10T20:40:38.267913+0000","last_peered":"2023-10-10T20:40:38.267913+0000","last_clean":"2023-10-10T20:40:38.267913+0000","last_became_active":"2023-10-10T20:40:37.265287+0000","last_became_peered":"2023-10-10T20:40:37.265287+0000","last_unstale":"2023-10-10T20:40:38.267913+0000","last_undegraded":"2023-10-10T20:40:38.267913+0000","last_fullsized":"2023-10-10T20:40:38.267913+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:26.330192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.274818+0000","last_change":"2023-10-10T20:40:38.272942+0000","last_active":"2023-10-10T20:40:38.274818+0000","last_peered":"2023-10-10T20:40:38.274818+0000","last_clean":"2023-10-10T20:40:38.274818+0000","last_became_active":"2023-10-10T20:40:38.272674+0000","last_became_peered":"2023-10-10T20:40:38.272674+0000","last_unstale":"2023-10-10T20:40:38.274818+0000","last_undegraded":"2023-10-10T20:40:38.274818+0000","last_fullsized":"2023-10-10T20:40:38.274818+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:46:05.714540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"99'8","reported_seq":18,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.289537+0000","last_change":"2023-10-10T20:40:37.264839+0000","last_active":"2023-10-10T20:40:38.289537+0000","last_peered":"2023-10-10T20:40:38.289537+0000","last_clean":"2023-10-10T20:40:38.289537+0000","last_became_active":"2023-10-10T20:40:37.264587+0000","last_became_peered":"2023-10-10T20:40:37.264587+0000","last_unstale":"2023-10-10T20:40:38.289537+0000","last_undegraded":"2023-10-10T20:40:38.289537+0000","last_fullsized":"2023-10-10T20:40:38.289537+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:24:35.687360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.276984+0000","last_change":"2023-10-10T20:40:38.273592+0000","last_active":"2023-10-10T20:40:38.276984+0000","last_peered":"2023-10-10T20:40:38.276984+0000","last_clean":"2023-10-10T20:40:38.276984+0000","last_became_active":"2023-10-10T20:40:38.273356+0000","last_became_peered":"2023-10-10T20:40:38.273356+0000","last_unstale":"2023-10-10T20:40:38.276984+0000","last_undegraded":"2023-10-10T20:40:38.276984+0000","last_fullsized":"2023-10-10T20:40:38.276984+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:01:59.691235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"99'1","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.287199+0000","last_change":"2023-10-10T20:40:37.260083+0000","last_active":"2023-10-10T20:40:38.287199+0000","last_peered":"2023-10-10T20:40:38.287199+0000","last_clean":"2023-10-10T20:40:38.287199+0000","last_became_active":"2023-10-10T20:40:37.259508+0000","last_became_peered":"2023-10-10T20:40:37.259508+0000","last_unstale":"2023-10-10T20:40:38.287199+0000","last_undegraded":"2023-10-10T20:40:38.287199+0000","last_fullsized":"2023-10-10T20:40:38.287199+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:13.929377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.264824+0000","last_change":"2023-10-10T20:40:38.262888+0000","last_active":"2023-10-10T20:40:38.264824+0000","last_peered":"2023-10-10T20:40:38.264824+0000","last_clean":"2023-10-10T20:40:38.264824+0000","last_became_active":"2023-10-10T20:40:38.262635+0000","last_became_peered":"2023-10-10T20:40:38.262635+0000","last_unstale":"2023-10-10T20:40:38.264824+0000","last_undegraded":"2023-10-10T20:40:38.264824+0000","last_fullsized":"2023-10-10T20:40:38.264824+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:26:40.109272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"99'2","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.281703+0000","last_change":"2023-10-10T20:40:37.266631+0000","last_active":"2023-10-10T20:40:38.281703+0000","last_peered":"2023-10-10T20:40:38.281703+0000","last_clean":"2023-10-10T20:40:38.281703+0000","last_became_active":"2023-10-10T20:40:37.266371+0000","last_became_peered":"2023-10-10T20:40:37.266371+0000","last_unstale":"2023-10-10T20:40:38.281703+0000","last_undegraded":"2023-10-10T20:40:38.281703+0000","last_fullsized":"2023-10-10T20:40:38.281703+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:36:44.776420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.272074+0000","last_change":"2023-10-10T20:40:38.263032+0000","last_active":"2023-10-10T20:40:38.272074+0000","last_peered":"2023-10-10T20:40:38.272074+0000","last_clean":"2023-10-10T20:40:38.272074+0000","last_became_active":"2023-10-10T20:40:38.262761+0000","last_became_peered":"2023-10-10T20:40:38.262761+0000","last_unstale":"2023-10-10T20:40:38.272074+0000","last_undegraded":"2023-10-10T20:40:38.272074+0000","last_fullsized":"2023-10-10T20:40:38.272074+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:54.041748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"99'1","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.279676+0000","last_change":"2023-10-10T20:40:37.264623+0000","last_active":"2023-10-10T20:40:38.279676+0000","last_peered":"2023-10-10T20:40:38.279676+0000","last_clean":"2023-10-10T20:40:38.279676+0000","last_became_active":"2023-10-10T20:40:37.264322+0000","last_became_peered":"2023-10-10T20:40:37.264322+0000","last_unstale":"2023-10-10T20:40:38.279676+0000","last_undegraded":"2023-10-10T20:40:38.279676+0000","last_fullsized":"2023-10-10T20:40:38.279676+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:14:28.701104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.260428+0000","last_change":"2023-10-10T20:40:38.258756+0000","last_active":"2023-10-10T20:40:38.260428+0000","last_peered":"2023-10-10T20:40:38.260428+0000","last_clean":"2023-10-10T20:40:38.260428+0000","last_became_active":"2023-10-10T20:40:38.258436+0000","last_became_peered":"2023-10-10T20:40:38.258436+0000","last_unstale":"2023-10-10T20:40:38.260428+0000","last_undegraded":"2023-10-10T20:40:38.260428+0000","last_fullsized":"2023-10-10T20:40:38.260428+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:09:20.495043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.253322+0000","last_change":"2023-10-10T20:40:37.265269+0000","last_active":"2023-10-10T20:40:38.253322+0000","last_peered":"2023-10-10T20:40:38.253322+0000","last_clean":"2023-10-10T20:40:38.253322+0000","last_became_active":"2023-10-10T20:40:37.264648+0000","last_became_peered":"2023-10-10T20:40:37.264648+0000","last_unstale":"2023-10-10T20:40:38.253322+0000","last_undegraded":"2023-10-10T20:40:38.253322+0000","last_fullsized":"2023-10-10T20:40:38.253322+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:14.826939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.274541+0000","last_change":"2023-10-10T20:40:38.271373+0000","last_active":"2023-10-10T20:40:38.274541+0000","last_peered":"2023-10-10T20:40:38.274541+0000","last_clean":"2023-10-10T20:40:38.274541+0000","last_became_active":"2023-10-10T20:40:38.271076+0000","last_became_peered":"2023-10-10T20:40:38.271076+0000","last_unstale":"2023-10-10T20:40:38.274541+0000","last_undegraded":"2023-10-10T20:40:38.274541+0000","last_fullsized":"2023-10-10T20:40:38.274541+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:43:08.406828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"99'2","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.280885+0000","last_change":"2023-10-10T20:40:37.261923+0000","last_active":"2023-10-10T20:40:38.280885+0000","last_peered":"2023-10-10T20:40:38.280885+0000","last_clean":"2023-10-10T20:40:38.280885+0000","last_became_active":"2023-10-10T20:40:37.261631+0000","last_became_peered":"2023-10-10T20:40:37.261631+0000","last_unstale":"2023-10-10T20:40:38.280885+0000","last_undegraded":"2023-10-10T20:40:38.280885+0000","last_fullsized":"2023-10-10T20:40:38.280885+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:16:28.059016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.261438+0000","last_change":"2023-10-10T20:40:38.259196+0000","last_active":"2023-10-10T20:40:38.261438+0000","last_peered":"2023-10-10T20:40:38.261438+0000","last_clean":"2023-10-10T20:40:38.261438+0000","last_became_active":"2023-10-10T20:40:38.258943+0000","last_became_peered":"2023-10-10T20:40:38.258943+0000","last_unstale":"2023-10-10T20:40:38.261438+0000","last_undegraded":"2023-10-10T20:40:38.261438+0000","last_fullsized":"2023-10-10T20:40:38.261438+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:07.660102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.261112+0000","last_change":"2023-10-10T20:40:37.263329+0000","last_active":"2023-10-10T20:40:38.261112+0000","last_peered":"2023-10-10T20:40:38.261112+0000","last_clean":"2023-10-10T20:40:38.261112+0000","last_became_active":"2023-10-10T20:40:37.263072+0000","last_became_peered":"2023-10-10T20:40:37.263072+0000","last_unstale":"2023-10-10T20:40:38.261112+0000","last_undegraded":"2023-10-10T20:40:38.261112+0000","last_fullsized":"2023-10-10T20:40:38.261112+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:31:56.472091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.274617+0000","last_change":"2023-10-10T20:40:38.266541+0000","last_active":"2023-10-10T20:40:38.274617+0000","last_peered":"2023-10-10T20:40:38.274617+0000","last_clean":"2023-10-10T20:40:38.274617+0000","last_became_active":"2023-10-10T20:40:38.265649+0000","last_became_peered":"2023-10-10T20:40:38.265649+0000","last_unstale":"2023-10-10T20:40:38.274617+0000","last_undegraded":"2023-10-10T20:40:38.274617+0000","last_fullsized":"2023-10-10T20:40:38.274617+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:39:38.169839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.252063+0000","last_change":"2023-10-10T20:40:37.262448+0000","last_active":"2023-10-10T20:40:38.252063+0000","last_peered":"2023-10-10T20:40:38.252063+0000","last_clean":"2023-10-10T20:40:38.252063+0000","last_became_active":"2023-10-10T20:40:37.262193+0000","last_became_peered":"2023-10-10T20:40:37.262193+0000","last_unstale":"2023-10-10T20:40:38.252063+0000","last_undegraded":"2023-10-10T20:40:38.252063+0000","last_fullsized":"2023-10-10T20:40:38.252063+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:05.415898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.268281+0000","last_change":"2023-10-10T20:40:38.262437+0000","last_active":"2023-10-10T20:40:38.268281+0000","last_peered":"2023-10-10T20:40:38.268281+0000","last_clean":"2023-10-10T20:40:38.268281+0000","last_became_active":"2023-10-10T20:40:38.262199+0000","last_became_peered":"2023-10-10T20:40:38.262199+0000","last_unstale":"2023-10-10T20:40:38.268281+0000","last_undegraded":"2023-10-10T20:40:38.268281+0000","last_fullsized":"2023-10-10T20:40:38.268281+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:38:00.681114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.254475+0000","last_change":"2023-10-10T20:40:37.265730+0000","last_active":"2023-10-10T20:40:38.254475+0000","last_peered":"2023-10-10T20:40:38.254475+0000","last_clean":"2023-10-10T20:40:38.254475+0000","last_became_active":"2023-10-10T20:40:37.264840+0000","last_became_peered":"2023-10-10T20:40:37.264840+0000","last_unstale":"2023-10-10T20:40:38.254475+0000","last_undegraded":"2023-10-10T20:40:38.254475+0000","last_fullsized":"2023-10-10T20:40:38.254475+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:39:11.321117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.280966+0000","last_change":"2023-10-10T20:40:38.275186+0000","last_active":"2023-10-10T20:40:38.280966+0000","last_peered":"2023-10-10T20:40:38.280966+0000","last_clean":"2023-10-10T20:40:38.280966+0000","last_became_active":"2023-10-10T20:40:38.274889+0000","last_became_peered":"2023-10-10T20:40:38.274889+0000","last_unstale":"2023-10-10T20:40:38.280966+0000","last_undegraded":"2023-10-10T20:40:38.280966+0000","last_fullsized":"2023-10-10T20:40:38.280966+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:14.934213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"99'2","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.288541+0000","last_change":"2023-10-10T20:40:37.261516+0000","last_active":"2023-10-10T20:40:38.288541+0000","last_peered":"2023-10-10T20:40:38.288541+0000","last_clean":"2023-10-10T20:40:38.288541+0000","last_became_active":"2023-10-10T20:40:37.261250+0000","last_became_peered":"2023-10-10T20:40:37.261250+0000","last_unstale":"2023-10-10T20:40:38.288541+0000","last_undegraded":"2023-10-10T20:40:38.288541+0000","last_fullsized":"2023-10-10T20:40:38.288541+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:37:03.617984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.288133+0000","last_change":"2023-10-10T20:40:38.274044+0000","last_active":"2023-10-10T20:40:38.288133+0000","last_peered":"2023-10-10T20:40:38.288133+0000","last_clean":"2023-10-10T20:40:38.288133+0000","last_became_active":"2023-10-10T20:40:38.273660+0000","last_became_peered":"2023-10-10T20:40:38.273660+0000","last_unstale":"2023-10-10T20:40:38.288133+0000","last_undegraded":"2023-10-10T20:40:38.288133+0000","last_fullsized":"2023-10-10T20:40:38.288133+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:31:46.616333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.267998+0000","last_change":"2023-10-10T20:40:37.263365+0000","last_active":"2023-10-10T20:40:38.267998+0000","last_peered":"2023-10-10T20:40:38.267998+0000","last_clean":"2023-10-10T20:40:38.267998+0000","last_became_active":"2023-10-10T20:40:37.263032+0000","last_became_peered":"2023-10-10T20:40:37.263032+0000","last_unstale":"2023-10-10T20:40:38.267998+0000","last_undegraded":"2023-10-10T20:40:38.267998+0000","last_fullsized":"2023-10-10T20:40:38.267998+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:06.817315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.276619+0000","last_change":"2023-10-10T20:40:38.261942+0000","last_active":"2023-10-10T20:40:38.276619+0000","last_peered":"2023-10-10T20:40:38.276619+0000","last_clean":"2023-10-10T20:40:38.276619+0000","last_became_active":"2023-10-10T20:40:38.261559+0000","last_became_peered":"2023-10-10T20:40:38.261559+0000","last_unstale":"2023-10-10T20:40:38.276619+0000","last_undegraded":"2023-10-10T20:40:38.276619+0000","last_fullsized":"2023-10-10T20:40:38.276619+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:07:17.016723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.251550+0000","last_change":"2023-10-10T20:40:37.264758+0000","last_active":"2023-10-10T20:40:38.251550+0000","last_peered":"2023-10-10T20:40:38.251550+0000","last_clean":"2023-10-10T20:40:38.251550+0000","last_became_active":"2023-10-10T20:40:37.264496+0000","last_became_peered":"2023-10-10T20:40:37.264496+0000","last_unstale":"2023-10-10T20:40:38.251550+0000","last_undegraded":"2023-10-10T20:40:38.251550+0000","last_fullsized":"2023-10-10T20:40:38.251550+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:43:18.734440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.280906+0000","last_change":"2023-10-10T20:40:38.275294+0000","last_active":"2023-10-10T20:40:38.280906+0000","last_peered":"2023-10-10T20:40:38.280906+0000","last_clean":"2023-10-10T20:40:38.280906+0000","last_became_active":"2023-10-10T20:40:38.274994+0000","last_became_peered":"2023-10-10T20:40:38.274994+0000","last_unstale":"2023-10-10T20:40:38.280906+0000","last_undegraded":"2023-10-10T20:40:38.280906+0000","last_fullsized":"2023-10-10T20:40:38.280906+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:43:19.739277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.252766+0000","last_change":"2023-10-10T20:40:37.261011+0000","last_active":"2023-10-10T20:40:38.252766+0000","last_peered":"2023-10-10T20:40:38.252766+0000","last_clean":"2023-10-10T20:40:38.252766+0000","last_became_active":"2023-10-10T20:40:37.260760+0000","last_became_peered":"2023-10-10T20:40:37.260760+0000","last_unstale":"2023-10-10T20:40:38.252766+0000","last_undegraded":"2023-10-10T20:40:38.252766+0000","last_fullsized":"2023-10-10T20:40:38.252766+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:42.139421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.272997+0000","last_change":"2023-10-10T20:40:38.270660+0000","last_active":"2023-10-10T20:40:38.272997+0000","last_peered":"2023-10-10T20:40:38.272997+0000","last_clean":"2023-10-10T20:40:38.272997+0000","last_became_active":"2023-10-10T20:40:38.270361+0000","last_became_peered":"2023-10-10T20:40:38.270361+0000","last_unstale":"2023-10-10T20:40:38.272997+0000","last_undegraded":"2023-10-10T20:40:38.272997+0000","last_fullsized":"2023-10-10T20:40:38.272997+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:23.463451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.253332+0000","last_change":"2023-10-10T20:40:37.267680+0000","last_active":"2023-10-10T20:40:38.253332+0000","last_peered":"2023-10-10T20:40:38.253332+0000","last_clean":"2023-10-10T20:40:38.253332+0000","last_became_active":"2023-10-10T20:40:37.267373+0000","last_became_peered":"2023-10-10T20:40:37.267373+0000","last_unstale":"2023-10-10T20:40:38.253332+0000","last_undegraded":"2023-10-10T20:40:38.253332+0000","last_fullsized":"2023-10-10T20:40:38.253332+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:30:44.468056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.271902+0000","last_change":"2023-10-10T20:40:38.267103+0000","last_active":"2023-10-10T20:40:38.271902+0000","last_peered":"2023-10-10T20:40:38.271902+0000","last_clean":"2023-10-10T20:40:38.271902+0000","last_became_active":"2023-10-10T20:40:38.266777+0000","last_became_peered":"2023-10-10T20:40:38.266777+0000","last_unstale":"2023-10-10T20:40:38.271902+0000","last_undegraded":"2023-10-10T20:40:38.271902+0000","last_fullsized":"2023-10-10T20:40:38.271902+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:59:28.728877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.264347+0000","last_change":"2023-10-10T20:40:37.265941+0000","last_active":"2023-10-10T20:40:38.264347+0000","last_peered":"2023-10-10T20:40:38.264347+0000","last_clean":"2023-10-10T20:40:38.264347+0000","last_became_active":"2023-10-10T20:40:37.265532+0000","last_became_peered":"2023-10-10T20:40:37.265532+0000","last_unstale":"2023-10-10T20:40:38.264347+0000","last_undegraded":"2023-10-10T20:40:38.264347+0000","last_fullsized":"2023-10-10T20:40:38.264347+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:02.961417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.274638+0000","last_change":"2023-10-10T20:40:38.269843+0000","last_active":"2023-10-10T20:40:38.274638+0000","last_peered":"2023-10-10T20:40:38.274638+0000","last_clean":"2023-10-10T20:40:38.274638+0000","last_became_active":"2023-10-10T20:40:38.269420+0000","last_became_peered":"2023-10-10T20:40:38.269420+0000","last_unstale":"2023-10-10T20:40:38.274638+0000","last_undegraded":"2023-10-10T20:40:38.274638+0000","last_fullsized":"2023-10-10T20:40:38.274638+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:28:37.819892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.261005+0000","last_change":"2023-10-10T20:40:37.264545+0000","last_active":"2023-10-10T20:40:38.261005+0000","last_peered":"2023-10-10T20:40:38.261005+0000","last_clean":"2023-10-10T20:40:38.261005+0000","last_became_active":"2023-10-10T20:40:37.264300+0000","last_became_peered":"2023-10-10T20:40:37.264300+0000","last_unstale":"2023-10-10T20:40:38.261005+0000","last_undegraded":"2023-10-10T20:40:38.261005+0000","last_fullsized":"2023-10-10T20:40:38.261005+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:03:53.037877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.261716+0000","last_change":"2023-10-10T20:40:38.260105+0000","last_active":"2023-10-10T20:40:38.261716+0000","last_peered":"2023-10-10T20:40:38.261716+0000","last_clean":"2023-10-10T20:40:38.261716+0000","last_became_active":"2023-10-10T20:40:38.259845+0000","last_became_peered":"2023-10-10T20:40:38.259845+0000","last_unstale":"2023-10-10T20:40:38.261716+0000","last_undegraded":"2023-10-10T20:40:38.261716+0000","last_fullsized":"2023-10-10T20:40:38.261716+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:45:33.016220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.254514+0000","last_change":"2023-10-10T20:40:37.265540+0000","last_active":"2023-10-10T20:40:38.254514+0000","last_peered":"2023-10-10T20:40:38.254514+0000","last_clean":"2023-10-10T20:40:38.254514+0000","last_became_active":"2023-10-10T20:40:37.264555+0000","last_became_peered":"2023-10-10T20:40:37.264555+0000","last_unstale":"2023-10-10T20:40:38.254514+0000","last_undegraded":"2023-10-10T20:40:38.254514+0000","last_fullsized":"2023-10-10T20:40:38.254514+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:37:03.617984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.267696+0000","last_change":"2023-10-10T20:40:38.259678+0000","last_active":"2023-10-10T20:40:38.267696+0000","last_peered":"2023-10-10T20:40:38.267696+0000","last_clean":"2023-10-10T20:40:38.267696+0000","last_became_active":"2023-10-10T20:40:38.259049+0000","last_became_peered":"2023-10-10T20:40:38.259049+0000","last_unstale":"2023-10-10T20:40:38.267696+0000","last_undegraded":"2023-10-10T20:40:38.267696+0000","last_fullsized":"2023-10-10T20:40:38.267696+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:37.239039+0000","last_clean_scrub_stamp":"2023-10-10T20:40:37.239039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:37:30.151565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.254577+0000","last_change":"2023-10-10T20:40:37.265098+0000","last_active":"2023-10-10T20:40:38.254577+0000","last_peered":"2023-10-10T20:40:38.254577+0000","last_clean":"2023-10-10T20:40:38.254577+0000","last_became_active":"2023-10-10T20:40:37.264742+0000","last_became_peered":"2023-10-10T20:40:37.264742+0000","last_unstale":"2023-10-10T20:40:38.254577+0000","last_undegraded":"2023-10-10T20:40:38.254577+0000","last_fullsized":"2023-10-10T20:40:38.254577+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:40:36.234202+0000","last_clean_scrub_stamp":"2023-10-10T20:40:36.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:42:00.401594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":243,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-10T20:40:38.252487+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:40:38.252487+0000","last_peered":"2023-10-10T20:40:38.252487+0000","last_clean":"2023-10-10T20:40:38.252487+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:40:38.252487+0000","last_undegraded":"2023-10-10T20:40:38.252487+0000","last_fullsized":"2023-10-10T20:40:38.252487+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303188,"kb_used_data":1804,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068652,"statfs":{"total":96636764160,"available":96326299648,"internally_reserved":0,"allocated":1847296,"data_stored":1128615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51500000000000001,"15min":0.81999999999999995},"min":{"1min":0.31,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53000000000000003,"15min":0.627},"min":{"1min":0.32200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71999999999999997,"5min":0.95399999999999996,"15min":4.1959999999999997},"last":0.52700000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.504,"15min":1},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.67800000000000005,"5min":0.78800000000000003,"15min":10.1},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55800000000000005,"15min":0.72999999999999998},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.28999999999999998},"max":{"1min":0.81799999999999995,"5min":1.006,"15min":6.806},"last":0.67900000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57899999999999996,"15min":0.78100000000000003},"min":{"1min":0.43099999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89000000000000001,"5min":1.089,"15min":6.9340000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53200000000000003,"15min":0.73799999999999999},"min":{"1min":0.39000000000000001,"5min":0.36399999999999999,"15min":0.30099999999999999},"max":{"1min":0.77100000000000002,"5min":0.85299999999999998,"15min":6.9870000000000001},"last":0.625}]},{"osd":3,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.75800000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.754,"5min":0.80200000000000005,"15min":6.6719999999999997},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.67000000000000004},"min":{"1min":0.38700000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.70599999999999996,"5min":1.2789999999999999,"15min":4.1609999999999996},"last":0.59099999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47399999999999998,"15min":0.46500000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.73199999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52400000000000002,"15min":0.52300000000000002},"min":{"1min":0.309,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.48899999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.32500000000000001,"5min":0.32100000000000001,"15min":0.314},"max":{"1min":0.86099999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75800000000000001,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.45000000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72999999999999998,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84099999999999997,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.69799999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302604,"kb_used_data":1220,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069236,"statfs":{"total":96636764160,"available":96326897664,"internally_reserved":0,"allocated":1249280,"data_stored":537729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50700000000000001,"15min":0.67200000000000004},"min":{"1min":0.33500000000000002,"5min":0.32400000000000001,"15min":0.312},"max":{"1min":0.68600000000000005,"5min":0.73099999999999998,"15min":7.7590000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52100000000000002,"15min":0.71299999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":7.8090000000000002},"last":0.42899999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.70999999999999996},"min":{"1min":0.38400000000000001,"5min":0.35599999999999998,"15min":0.27800000000000002},"max":{"1min":0.628,"5min":0.80000000000000004,"15min":7.867},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53300000000000003,"15min":0.88500000000000001},"min":{"1min":0.36599999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":9.7089999999999996},"last":0.57299999999999995}]},{"osd":2,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51400000000000001,"15min":0.59599999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.72499999999999998,"5min":0.747,"15min":4.7999999999999998},"last":0.498},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54600000000000004,"15min":0.84499999999999997},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.71499999999999997,"5min":0.80000000000000004,"15min":6.7629999999999999},"last":0.46000000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.62},"min":{"1min":0.40999999999999998,"5min":0.36299999999999999,"15min":0.35299999999999998},"max":{"1min":0.79100000000000004,"5min":0.83999999999999997,"15min":4.8620000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54800000000000004,"15min":0.71699999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.35299999999999998},"max":{"1min":0.68600000000000005,"5min":1.1850000000000001,"15min":7.9009999999999998},"last":0.40500000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51500000000000001,"15min":0.46800000000000003},"min":{"1min":0.32400000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71099999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59199999999999997,"15min":0.56799999999999995},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.37},"max":{"1min":0.73499999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52800000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52000000000000002,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.314,"15min":0.312},"max":{"1min":0.69199999999999995,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.443},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.747,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.39000000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55100000000000005,"15min":0.52000000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.72599999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.374},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57799999999999996,"15min":0.54400000000000004},"min":{"1min":0.42899999999999999,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":0.69399999999999995,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.54800000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302608,"kb_used_data":1224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069232,"statfs":{"total":96636764160,"available":96326893568,"internally_reserved":0,"allocated":1253376,"data_stored":537783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.31900000000000001,"15min":0.47599999999999998},"min":{"1min":0.20999999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.441,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.223},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.32300000000000001,"15min":0.36299999999999999},"min":{"1min":0.23599999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46000000000000002,"5min":0.68700000000000006,"15min":1.194},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.32400000000000001,"15min":0.36399999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.379,"5min":0.67400000000000004,"15min":1.2869999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.33800000000000002,"15min":0.505},"min":{"1min":0.26000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.54200000000000004,"5min":0.57199999999999995,"15min":5.0919999999999996},"last":0.26000000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36299999999999999,"15min":0.52800000000000002},"min":{"1min":0.26800000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.51000000000000001,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36499999999999999,"15min":0.40699999999999997},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.48299999999999998,"5min":0.72099999999999997,"15min":1.2170000000000001},"last":0.247}]},{"osd":4,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57899999999999996,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.81000000000000005,"5min":1.133,"15min":1.254},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56599999999999995,"15min":0.70599999999999996},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69899999999999995,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.53700000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57899999999999996,"15min":0.59499999999999997},"min":{"1min":0.42299999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.751,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.60799999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71899999999999997,"5min":1.0720000000000001,"15min":1.3109999999999999},"last":0.69299999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60699999999999998,"15min":0.84399999999999997},"min":{"1min":0.435,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86199999999999999,"5min":0.995,"15min":5.0659999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61699999999999999,"15min":0.63200000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84599999999999997,"5min":1.1439999999999999,"15min":1.171},"last":0.68100000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59299999999999997,"15min":0.61699999999999999},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.753,"5min":1.214,"15min":1.214},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60099999999999998,"15min":0.72499999999999998},"min":{"1min":0.45000000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76900000000000002,"5min":1.115,"15min":4.0499999999999998},"last":0.56200000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303196,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068644,"statfs":{"total":96636764160,"available":96326291456,"internally_reserved":0,"allocated":1855488,"data_stored":1128714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.309,"15min":0.52800000000000002},"min":{"1min":0.20999999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.38300000000000001,"5min":0.51600000000000001,"15min":7.9249999999999998},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.318,"15min":0.505},"min":{"1min":0.23699999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.41899999999999998,"5min":0.54900000000000004,"15min":6.6299999999999999},"last":0.29699999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.32600000000000001,"15min":0.55200000000000005},"min":{"1min":0.22,"5min":0.216,"15min":0.216},"max":{"1min":0.48799999999999999,"5min":0.55400000000000005,"15min":8.2509999999999994},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.34000000000000002,"15min":0.52800000000000002},"min":{"1min":0.23100000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.45700000000000002,"5min":0.54100000000000004,"15min":6.7370000000000001},"last":0.26200000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.34399999999999997,"15min":0.55700000000000005},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.48299999999999998,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.61199999999999999},"min":{"1min":0.32000000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":7.8929999999999998},"last":0.38500000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55600000000000005,"15min":0.71399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.72499999999999998,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56299999999999994,"15min":0.77200000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.67600000000000005,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.55900000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.55200000000000005,"15min":0.751},"min":{"1min":0.38800000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69299999999999995,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58099999999999996,"15min":0.77800000000000002},"min":{"1min":0.42299999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77000000000000002,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.54500000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60199999999999998,"15min":0.81799999999999995},"min":{"1min":0.39800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72099999999999997,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61099999999999999,"15min":0.76100000000000001},"min":{"1min":0.41399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74299999999999999,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.48599999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58799999999999997,"15min":0.752},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78300000000000003,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59099999999999997,"15min":0.78600000000000003},"min":{"1min":0.45200000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.69299999999999995,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.497}]}]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302600,"kb_used_data":1216,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069240,"statfs":{"total":96636764160,"available":96326901760,"internally_reserved":0,"allocated":1245184,"data_stored":537659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.28299999999999997,"5min":0.309,"15min":0.32500000000000001},"min":{"1min":0.21099999999999999,"5min":0.189,"15min":0.189},"max":{"1min":0.38300000000000001,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.29399999999999998,"5min":0.32100000000000001,"15min":0.35299999999999998},"min":{"1min":0.215,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.39500000000000002,"5min":0.5,"15min":0.746},"last":0.30499999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.32900000000000001,"15min":0.72899999999999998},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.35699999999999998,"5min":0.63,"15min":11.967000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.34499999999999997,"15min":0.48299999999999998},"min":{"1min":0.26800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.443,"5min":0.53400000000000003,"15min":4.1100000000000003},"last":0.34100000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.34499999999999997,"15min":0.55700000000000005},"min":{"1min":0.23699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.39400000000000002,"5min":0.55400000000000005,"15min":6.3040000000000003},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38700000000000001,"15min":0.78100000000000003},"min":{"1min":0.27900000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.46899999999999997,"5min":0.65900000000000003,"15min":11.935},"last":0.36299999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.54300000000000004,"15min":0.55500000000000005},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.58699999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.378},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71699999999999997,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.439}]},{"osd":5,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.378},"max":{"1min":0.66300000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54400000000000004,"15min":0.58399999999999996},"min":{"1min":0.39700000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74199999999999999,"5min":0.874,"15min":0.96099999999999997},"last":0.48399999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57699999999999996,"15min":0.60099999999999998},"min":{"1min":0.42199999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.745,"5min":0.91600000000000004,"15min":0.91800000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73799999999999999,"5min":0.93100000000000005,"15min":0.93600000000000005},"last":0.49199999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56499999999999995,"15min":0.58499999999999996},"min":{"1min":0.41799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71099999999999997,"5min":0.89100000000000001,"15min":1.052},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57699999999999996,"15min":0.78000000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69799999999999995,"5min":0.85999999999999999,"15min":6.2649999999999997},"last":0.52500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302616,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069224,"statfs":{"total":96636764160,"available":96326885376,"internally_reserved":0,"allocated":1261568,"data_stored":539265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.31,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.253},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.31900000000000001},"min":{"1min":0.25,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.27800000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.23499999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38600000000000001,"15min":0.377},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.57099999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.29699999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.34999999999999998},"min":{"1min":0.23699999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.308},{"interface":"front","average":{"1min":0.374,"5min":0.35699999999999998,"15min":0.36199999999999999},"min":{"1min":0.26300000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.33000000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69999999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.435},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.82299999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.47999999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.35899999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55900000000000005,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.76900000000000002,"5min":1.01,"15min":1.01},"last":0.48799999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.42099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85699999999999998,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.497}]},{"osd":7,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.441,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.496,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.88400000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.52500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574926,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302600,"kb_used_data":1216,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069240,"statfs":{"total":96636764160,"available":96326901760,"internally_reserved":0,"allocated":1245184,"data_stored":537659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68300000000000005,"5min":0.74199999999999999,"15min":0.747},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.76600000000000001},"min":{"1min":0.374,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67500000000000004,"5min":0.78400000000000003,"15min":7.6719999999999997},"last":0.39400000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52900000000000003,"15min":0.77400000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.755,"5min":0.755,"15min":7.7160000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65300000000000002,"5min":0.76900000000000002,"15min":0.86799999999999999},"last":0.49399999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.65300000000000002,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56299999999999994,"15min":0.79300000000000004},"min":{"1min":0.39700000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.72799999999999998,"5min":0.79800000000000004,"15min":7.7839999999999998},"last":0.60099999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55200000000000005,"15min":0.54900000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65500000000000003,"5min":0.73399999999999999,"15min":0.96999999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.41199999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.66100000000000003,"5min":0.78100000000000003,"15min":0.79400000000000004},"last":0.57699999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51000000000000001,"15min":0.48699999999999999},"min":{"1min":0.32500000000000001,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.77500000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54900000000000004,"15min":0.54300000000000004},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.53100000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.39200000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68799999999999994,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.35899999999999999},"max":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.63}]},{"osd":7,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.378,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.84599999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.46100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.748,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574928,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298512,"kb_used_data":1224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073328,"statfs":{"total":96636764160,"available":96331087872,"internally_reserved":0,"allocated":1253376,"data_stored":538235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.307,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63600000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.34300000000000003,"5min":0.215,"15min":0.215},"max":{"1min":0.70399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.47399999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.56200000000000006,"15min":0.65300000000000002},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55300000000000005,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68200000000000005,"5min":5.3109999999999999,"15min":5.3109999999999999},"last":0.42299999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56899999999999995,"15min":0.66100000000000003},"min":{"1min":0.32900000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64700000000000002,"5min":5.2960000000000003,"15min":5.2960000000000003},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72399999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.379}]},{"osd":3,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59499999999999997,"15min":0.69099999999999995},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70399999999999996,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59699999999999998,"15min":0.70399999999999996},"min":{"1min":0.40300000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70599999999999996,"5min":5.2850000000000001,"15min":5.2850000000000001},"last":0.44400000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48299999999999998,"15min":0.497},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.63800000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65000000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.63700000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.65100000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71299999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.53900000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.33000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72699999999999998,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.58499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:40:47.192 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:40:47.193 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:40:47.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.346+0000 7f04ac3cd700 1 Processor -- start 2023-10-10T20:40:47.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.346+0000 7f04ac3cd700 1 -- start start 2023-10-10T20:40:47.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.347+0000 7f04ac3cd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04a40cbb90 0x7f04a40af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:47.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.347+0000 7f04ac3cd700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a40af860 con 0x7f04a4159950 2023-10-10T20:40:47.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.347+0000 7f04ac3cd700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a40cbff0 con 0x7f04a4069c70 2023-10-10T20:40:47.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.347+0000 7f04ac3cd700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a40ce380 con 0x7f04a40cbb90 2023-10-10T20:40:47.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.347+0000 7f04aa169700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f04a4159950 0x7f04a4069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49208/0 (socket says 172.21.15.89:49208) 2023-10-10T20:40:47.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.347+0000 7f04aa169700 1 -- 172.21.15.89:0/2217103836 learned_addr learned my addr 172.21.15.89:0/2217103836 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:47.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.348+0000 7f04aa96a700 1 --2- 172.21.15.89:0/2217103836 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04a40cbb90 0x7f04a40af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:47.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.348+0000 7f04a9167700 1 -- 172.21.15.89:0/2217103836 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961167581 0 0) 0x7f04a40af860 con 0x7f04a4159950 2023-10-10T20:40:47.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.348+0000 7f04a9167700 1 -- 172.21.15.89:0/2217103836 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0490003570 con 0x7f04a4159950 2023-10-10T20:40:47.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.348+0000 7f04a9167700 1 -- 172.21.15.89:0/2217103836 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108418333 0 0) 0x7f0494001110 con 0x7f04a4069c70 2023-10-10T20:40:47.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.348+0000 7f04a9167700 1 -- 172.21.15.89:0/2217103836 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04a40af860 con 0x7f04a4069c70 2023-10-10T20:40:47.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.348+0000 7f04a9167700 1 -- 172.21.15.89:0/2217103836 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3445160059 0 0) 0x7f0490003570 con 0x7f04a4159950 2023-10-10T20:40:47.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.348+0000 7f04aa96a700 1 -- 172.21.15.89:0/2217103836 >> v1:172.21.15.111:6789/0 conn(0x7f04a4069c70 legacy=0x7f04a40cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:47.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.348+0000 7f04aa96a700 1 -- 172.21.15.89:0/2217103836 >> v1:172.21.15.89:6789/0 conn(0x7f04a4159950 legacy=0x7f04a4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:47.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.348+0000 7f04aa96a700 1 -- 172.21.15.89:0/2217103836 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a40cec20 con 0x7f04a40cbb90 2023-10-10T20:40:47.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.348+0000 7f04aa96a700 1 --2- 172.21.15.89:0/2217103836 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04a40cbb90 0x7f04a40af320 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f049c009e90 tx=0x7f049c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f432d2bc989ea9f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:47.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.348+0000 7f04a9167700 1 -- 172.21.15.89:0/2217103836 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c00e070 con 0x7f04a40cbb90 2023-10-10T20:40:47.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.348+0000 7f04a9167700 1 -- 172.21.15.89:0/2217103836 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049c00b870 con 0x7f04a40cbb90 2023-10-10T20:40:47.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.348+0000 7f04a9167700 1 -- 172.21.15.89:0/2217103836 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a0001470 con 0x7f04a40cbb90 2023-10-10T20:40:47.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.349+0000 7f04ac3cd700 1 -- 172.21.15.89:0/2217103836 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04a40cbb90 msgr2=0x7f04a40af320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:47.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.349+0000 7f04ac3cd700 1 --2- 172.21.15.89:0/2217103836 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04a40cbb90 0x7f04a40af320 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f049c009e90 tx=0x7f049c009780 comp rx=0 tx=0).stop 2023-10-10T20:40:47.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.349+0000 7f04ac3cd700 1 -- 172.21.15.89:0/2217103836 shutdown_connections 2023-10-10T20:40:47.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.349+0000 7f04ac3cd700 1 --2- 172.21.15.89:0/2217103836 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04a40cbb90 0x7f04a40af320 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.349+0000 7f04ac3cd700 1 -- 172.21.15.89:0/2217103836 >> 172.21.15.89:0/2217103836 conn(0x7f04a4068e00 msgr2=0x7f04a40d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:47.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.349+0000 7f04ac3cd700 1 -- 172.21.15.89:0/2217103836 shutdown_connections 2023-10-10T20:40:47.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.349+0000 7f04ac3cd700 1 -- 172.21.15.89:0/2217103836 wait complete. 2023-10-10T20:40:47.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.350+0000 7f04ac3cd700 1 Processor -- start 2023-10-10T20:40:47.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.350+0000 7f04ac3cd700 1 -- start start 2023-10-10T20:40:47.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f04ac3cd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04a4069c70 0x7f04a40ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:47.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f04ac3cd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04a40cbb90 0x7f04a4154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:47.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f04ac3cd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04a4159950 0x7f04a416a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:47.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f04ac3cd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a40b50e0 con 0x7f04a40cbb90 2023-10-10T20:40:47.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f04ac3cd700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a40b44d0 con 0x7f04a4159950 2023-10-10T20:40:47.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f04ac3cd700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a40b47b0 con 0x7f04a4069c70 2023-10-10T20:40:47.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f04a9968700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04a40cbb90 0x7f04a4154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:47.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f04a9968700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04a40cbb90 0x7f04a4154380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39062/0 (socket says 172.21.15.89:39062) 2023-10-10T20:40:47.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f04a9968700 1 -- 172.21.15.89:0/158501791 learned_addr learned my addr 172.21.15.89:0/158501791 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:47.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f04aa169700 1 --2- 172.21.15.89:0/158501791 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04a4069c70 0x7f04a40ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:47.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f04aa96a700 1 --2- 172.21.15.89:0/158501791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04a4159950 0x7f04a416a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:47.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f04a9968700 1 -- 172.21.15.89:0/158501791 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04a4069c70 msgr2=0x7f04a40ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:47.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f04a9968700 1 --2- 172.21.15.89:0/158501791 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04a4069c70 0x7f04a40ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f04a9968700 1 -- 172.21.15.89:0/158501791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04a4159950 msgr2=0x7f04a416a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:47.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f04a9968700 1 --2- 172.21.15.89:0/158501791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04a4159950 0x7f04a416a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f04a9968700 1 -- 172.21.15.89:0/158501791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049c009a20 con 0x7f04a40cbb90 2023-10-10T20:40:47.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f04aa96a700 1 --2- 172.21.15.89:0/158501791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04a4159950 0x7f04a416a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:47.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f04aa169700 1 --2- 172.21.15.89:0/158501791 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04a4069c70 0x7f04a40ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:47.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f04a9968700 1 --2- 172.21.15.89:0/158501791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04a40cbb90 0x7f04a4154380 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f049400c710 tx=0x7f0494007c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:47.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.351+0000 7f049a7fc700 1 -- 172.21.15.89:0/158501791 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494011590 con 0x7f04a40cbb90 2023-10-10T20:40:47.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.352+0000 7f049a7fc700 1 -- 172.21.15.89:0/158501791 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0494011710 con 0x7f04a40cbb90 2023-10-10T20:40:47.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.352+0000 7f04ac3cd700 1 -- 172.21.15.89:0/158501791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a40b4a90 con 0x7f04a40cbb90 2023-10-10T20:40:47.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.352+0000 7f049a7fc700 1 -- 172.21.15.89:0/158501791 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494011ad0 con 0x7f04a40cbb90 2023-10-10T20:40:47.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.352+0000 7f04ac3cd700 1 -- 172.21.15.89:0/158501791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a40ca5a0 con 0x7f04a40cbb90 2023-10-10T20:40:47.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.353+0000 7f04ac3cd700 1 -- 172.21.15.89:0/158501791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a40b5290 con 0x7f04a40cbb90 2023-10-10T20:40:47.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.353+0000 7f049a7fc700 1 -- 172.21.15.89:0/158501791 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f049401c460 con 0x7f04a40cbb90 2023-10-10T20:40:47.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.353+0000 7f049a7fc700 1 --2- 172.21.15.89:0/158501791 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0488067290 0x7f0488069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:47.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.353+0000 7f049a7fc700 1 -- 172.21.15.89:0/158501791 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0494084610 con 0x7f04a40cbb90 2023-10-10T20:40:47.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.353+0000 7f04aa169700 1 --2- 172.21.15.89:0/158501791 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0488067290 0x7f0488069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:47.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.354+0000 7f04aa169700 1 --2- 172.21.15.89:0/158501791 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0488067290 0x7f0488069740 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f04a00015a0 tx=0x7f04a0000bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:47.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.357+0000 7f049a7fc700 1 -- 172.21.15.89:0/158501791 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04a40b5290 con 0x7f04a40cbb90 2023-10-10T20:40:47.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.613+0000 7f04ac3cd700 1 -- 172.21.15.89:0/158501791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f04a40cbff0 con 0x7f04a40cbb90 2023-10-10T20:40:47.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.619+0000 7f049a7fc700 1 -- 172.21.15.89:0/158501791 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7f04a40cbff0 con 0x7f04a40cbb90 2023-10-10T20:40:47.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.621+0000 7f04ac3cd700 1 -- 172.21.15.89:0/158501791 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0488067290 msgr2=0x7f0488069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:47.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.621+0000 7f04ac3cd700 1 --2- 172.21.15.89:0/158501791 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0488067290 0x7f0488069740 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f04a00015a0 tx=0x7f04a0000bc0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.621+0000 7f04ac3cd700 1 -- 172.21.15.89:0/158501791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04a40cbb90 msgr2=0x7f04a4154380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:47.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.621+0000 7f04ac3cd700 1 --2- 172.21.15.89:0/158501791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04a40cbb90 0x7f04a4154380 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f049400c710 tx=0x7f0494007c40 comp rx=0 tx=0).stop 2023-10-10T20:40:47.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.621+0000 7f04ac3cd700 1 -- 172.21.15.89:0/158501791 shutdown_connections 2023-10-10T20:40:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.621+0000 7f04ac3cd700 1 --2- 172.21.15.89:0/158501791 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0488067290 0x7f0488069740 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.621+0000 7f04ac3cd700 1 --2- 172.21.15.89:0/158501791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04a4159950 0x7f04a416a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.621+0000 7f04ac3cd700 1 --2- 172.21.15.89:0/158501791 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f04a4069c70 0x7f04a40ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.621+0000 7f04ac3cd700 1 --2- 172.21.15.89:0/158501791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04a40cbb90 0x7f04a4154380 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.621+0000 7f04ac3cd700 1 -- 172.21.15.89:0/158501791 >> 172.21.15.89:0/158501791 conn(0x7f04a4068e00 msgr2=0x7f04a41586d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:47.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.621+0000 7f04ac3cd700 1 -- 172.21.15.89:0/158501791 shutdown_connections 2023-10-10T20:40:47.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.621+0000 7f04ac3cd700 1 -- 172.21.15.89:0/158501791 wait complete. 2023-10-10T20:40:47.627 INFO:teuthology.orchestra.run.smithi089.stderr:updated caps for client.0 2023-10-10T20:40:47.636 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:40:47.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.747+0000 7f64b349c700 1 Processor -- start 2023-10-10T20:40:47.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.747+0000 7f64b349c700 1 -- start start 2023-10-10T20:40:47.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.748+0000 7f64b349c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f64ac159890 0x7f64ac15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:47.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.748+0000 7f64b349c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64ac159c60 0x7f64ac15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:47.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.748+0000 7f64b349c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f64ac15db10 0x7f64ac069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:47.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.748+0000 7f64b349c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ac15a160 con 0x7f64ac159890 2023-10-10T20:40:47.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.748+0000 7f64b349c700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ac0d13a0 con 0x7f64ac15db10 2023-10-10T20:40:47.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.748+0000 7f64b349c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac15e140 con 0x7f64ac159c60 2023-10-10T20:40:47.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.748+0000 7f64b1238700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f64ac159890 0x7f64ac15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:47.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.748+0000 7f64b1238700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f64ac159890 0x7f64ac15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39074/0 (socket says 172.21.15.89:39074) 2023-10-10T20:40:47.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.748+0000 7f64b1238700 1 -- 172.21.15.89:0/3001790245 learned_addr learned my addr 172.21.15.89:0/3001790245 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:47.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.749+0000 7f64b0a37700 1 --2- 172.21.15.89:0/3001790245 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64ac159c60 0x7f64ac15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:47.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.749+0000 7f64b1a39700 1 --2- 172.21.15.89:0/3001790245 >> v2:172.21.15.111:3300/0 conn(0x7f64ac15db10 0x7f64ac069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:47.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.749+0000 7f64b1238700 1 -- 172.21.15.89:0/3001790245 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64ac159c60 msgr2=0x7f64ac15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:47.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.749+0000 7f64b1238700 1 --2- 172.21.15.89:0/3001790245 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64ac159c60 0x7f64ac15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.749+0000 7f64b1238700 1 -- 172.21.15.89:0/3001790245 >> v2:172.21.15.111:3300/0 conn(0x7f64ac15db10 msgr2=0x7f64ac069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:47.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.749+0000 7f64b1238700 1 --2- 172.21.15.89:0/3001790245 >> v2:172.21.15.111:3300/0 conn(0x7f64ac15db10 0x7f64ac069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.749+0000 7f64b1238700 1 -- 172.21.15.89:0/3001790245 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac15a3d0 con 0x7f64ac159890 2023-10-10T20:40:47.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.749+0000 7f64b1238700 1 --2- 172.21.15.89:0/3001790245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64ac159890 0x7f64ac15abd0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f64ac089b50 tx=0x7f64a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef05b875054306d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:47.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.749+0000 7f64a3fff700 1 -- 172.21.15.89:0/3001790245 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a800e070 con 0x7f64ac159890 2023-10-10T20:40:47.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.749+0000 7f64a3fff700 1 -- 172.21.15.89:0/3001790245 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a800b970 con 0x7f64ac159890 2023-10-10T20:40:47.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.749+0000 7f64a3fff700 1 -- 172.21.15.89:0/3001790245 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a80092e0 con 0x7f64ac159890 2023-10-10T20:40:47.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.749+0000 7f64b349c700 1 -- 172.21.15.89:0/3001790245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64ac159890 msgr2=0x7f64ac15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:47.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.750+0000 7f64b349c700 1 --2- 172.21.15.89:0/3001790245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64ac159890 0x7f64ac15abd0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f64ac089b50 tx=0x7f64a8009840 comp rx=0 tx=0).stop 2023-10-10T20:40:47.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.750+0000 7f64b349c700 1 -- 172.21.15.89:0/3001790245 shutdown_connections 2023-10-10T20:40:47.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.750+0000 7f64b349c700 1 --2- 172.21.15.89:0/3001790245 >> v2:172.21.15.111:3300/0 conn(0x7f64ac15db10 0x7f64ac069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.750+0000 7f64b349c700 1 --2- 172.21.15.89:0/3001790245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64ac159890 0x7f64ac15abd0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.750+0000 7f64b349c700 1 --2- 172.21.15.89:0/3001790245 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64ac159c60 0x7f64ac15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.750+0000 7f64b349c700 1 -- 172.21.15.89:0/3001790245 >> 172.21.15.89:0/3001790245 conn(0x7f64ac068e20 msgr2=0x7f64ac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:47.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.750+0000 7f64b349c700 1 -- 172.21.15.89:0/3001790245 shutdown_connections 2023-10-10T20:40:47.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.750+0000 7f64b349c700 1 -- 172.21.15.89:0/3001790245 wait complete. 2023-10-10T20:40:47.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.751+0000 7f64b349c700 1 Processor -- start 2023-10-10T20:40:47.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.751+0000 7f64b349c700 1 -- start start 2023-10-10T20:40:47.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b349c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64ac159890 0x7f64ac147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:47.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b349c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64ac159c60 0x7f64ac15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:47.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b349c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64ac15db10 0x7f64ac15bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:47.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b349c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac0cb6f0 con 0x7f64ac159890 2023-10-10T20:40:47.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b349c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac0c78a0 con 0x7f64ac15db10 2023-10-10T20:40:47.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b349c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac0c8140 con 0x7f64ac159c60 2023-10-10T20:40:47.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b1238700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64ac159890 0x7f64ac147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:47.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b1238700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64ac159890 0x7f64ac147280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39078/0 (socket says 172.21.15.89:39078) 2023-10-10T20:40:47.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b1238700 1 -- 172.21.15.89:0/1119154447 learned_addr learned my addr 172.21.15.89:0/1119154447 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:47.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b0a37700 1 --2- 172.21.15.89:0/1119154447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64ac159c60 0x7f64ac15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:47.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b1a39700 1 --2- 172.21.15.89:0/1119154447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64ac15db10 0x7f64ac15bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:47.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b1238700 1 -- 172.21.15.89:0/1119154447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64ac159c60 msgr2=0x7f64ac15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:47.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b1238700 1 --2- 172.21.15.89:0/1119154447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64ac159c60 0x7f64ac15b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b1238700 1 -- 172.21.15.89:0/1119154447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64ac15db10 msgr2=0x7f64ac15bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:47.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b1238700 1 --2- 172.21.15.89:0/1119154447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64ac15db10 0x7f64ac15bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b1238700 1 -- 172.21.15.89:0/1119154447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a8009ae0 con 0x7f64ac159890 2023-10-10T20:40:47.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b0a37700 1 --2- 172.21.15.89:0/1119154447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64ac159c60 0x7f64ac15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:47.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b1238700 1 --2- 172.21.15.89:0/1119154447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64ac159890 0x7f64ac147280 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f64ac070e20 tx=0x7f64a8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:47.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64a17fa700 1 -- 172.21.15.89:0/1119154447 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8008950 con 0x7f64ac159890 2023-10-10T20:40:47.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64a17fa700 1 -- 172.21.15.89:0/1119154447 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a8008ab0 con 0x7f64ac159890 2023-10-10T20:40:47.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b349c700 1 -- 172.21.15.89:0/1119154447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ac1477c0 con 0x7f64ac159890 2023-10-10T20:40:47.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64a17fa700 1 -- 172.21.15.89:0/1119154447 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8008e70 con 0x7f64ac159890 2023-10-10T20:40:47.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.752+0000 7f64b349c700 1 -- 172.21.15.89:0/1119154447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ac0c7af0 con 0x7f64ac159890 2023-10-10T20:40:47.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.753+0000 7f64b349c700 1 -- 172.21.15.89:0/1119154447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ac04c3c0 con 0x7f64ac159890 2023-10-10T20:40:47.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.754+0000 7f64a17fa700 1 -- 172.21.15.89:0/1119154447 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f64a8003040 con 0x7f64ac159890 2023-10-10T20:40:47.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.754+0000 7f64a17fa700 1 --2- 172.21.15.89:0/1119154447 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6490067290 0x7f6490069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:47.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.754+0000 7f64a17fa700 1 -- 172.21.15.89:0/1119154447 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f64a8086840 con 0x7f64ac159890 2023-10-10T20:40:47.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.754+0000 7f64b0a37700 1 --2- 172.21.15.89:0/1119154447 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6490067290 0x7f6490069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:47.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.755+0000 7f64b0a37700 1 --2- 172.21.15.89:0/1119154447 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6490067290 0x7f6490069740 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f64ac15c360 tx=0x7f649c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:47.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.758+0000 7f64a17fa700 1 -- 172.21.15.89:0/1119154447 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64a8053590 con 0x7f64ac159890 2023-10-10T20:40:47.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.986+0000 7f64b349c700 1 -- 172.21.15.89:0/1119154447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64ac06a5b0 con 0x7f64ac159890 2023-10-10T20:40:47.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.987+0000 7f64a17fa700 1 -- 172.21.15.89:0/1119154447 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f64a8056bb0 con 0x7f64ac159890 2023-10-10T20:40:47.991 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:47.992 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1635581143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1635581143},{"type":"v1","addr":"172.21.15.89:6837","nonce":1635581143}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5732,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/321638734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":321638734},{"type":"v1","addr":"172.21.15.111:6835","nonce":321638734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1070384555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1070384555},{"type":"v1","addr":"172.21.15.111:6834","nonce":1070384555}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:40:37.630844+0000","modified":"2023-10-10T20:40:41.269543+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6822},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6822":{"gid":6822,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6836/2274240619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2274240619},{"type":"v1","addr":"172.21.15.89:6836","nonce":2274240619}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-10T20:40:47.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.989+0000 7f64b349c700 1 -- 172.21.15.89:0/1119154447 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6490067290 msgr2=0x7f6490069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:47.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.989+0000 7f64b349c700 1 --2- 172.21.15.89:0/1119154447 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6490067290 0x7f6490069740 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f64ac15c360 tx=0x7f649c008040 comp rx=0 tx=0).stop 2023-10-10T20:40:47.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.989+0000 7f64b349c700 1 -- 172.21.15.89:0/1119154447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64ac159890 msgr2=0x7f64ac147280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:47.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.989+0000 7f64b349c700 1 --2- 172.21.15.89:0/1119154447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64ac159890 0x7f64ac147280 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f64ac070e20 tx=0x7f64a8018000 comp rx=0 tx=0).stop 2023-10-10T20:40:47.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.989+0000 7f64b349c700 1 -- 172.21.15.89:0/1119154447 shutdown_connections 2023-10-10T20:40:47.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.989+0000 7f64b349c700 1 --2- 172.21.15.89:0/1119154447 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6490067290 0x7f6490069740 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.989+0000 7f64b349c700 1 --2- 172.21.15.89:0/1119154447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64ac15db10 0x7f64ac15bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.989+0000 7f64b349c700 1 --2- 172.21.15.89:0/1119154447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64ac159890 0x7f64ac147280 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.989+0000 7f64b349c700 1 --2- 172.21.15.89:0/1119154447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64ac159c60 0x7f64ac15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:47.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.989+0000 7f64b349c700 1 -- 172.21.15.89:0/1119154447 >> 172.21.15.89:0/1119154447 conn(0x7f64ac068e20 msgr2=0x7f64ac069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:47.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.989+0000 7f64b349c700 1 -- 172.21.15.89:0/1119154447 shutdown_connections 2023-10-10T20:40:47.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:47.989+0000 7f64b349c700 1 -- 172.21.15.89:0/1119154447 wait complete. 2023-10-10T20:40:47.998 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 64 2023-10-10T20:40:48.407 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:40:37.630844+0000', 'modified': '2023-10-10T20:40:41.269543+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6822}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6822': {'gid': 6822, 'name': 'a', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6836/2274240619', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6834', 'nonce': 2274240619}, {'type': 'v1', 'addr': '172.21.15.89:6836', 'nonce': 2274240619}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:40:48.407 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:40:48.408 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-10T20:40:48.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.520+0000 7f092ba60700 1 Processor -- start 2023-10-10T20:40:48.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.520+0000 7f092ba60700 1 -- start start 2023-10-10T20:40:48.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f092ba60700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0924159c60 0x7f092415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f092ba60700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f092415db10 0x7f0924069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f092ba60700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092415a160 con 0x7f0924159890 2023-10-10T20:40:48.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f092ba60700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09240d13a0 con 0x7f092415db10 2023-10-10T20:40:48.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f092ba60700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092415e140 con 0x7f0924159c60 2023-10-10T20:40:48.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f0929ffd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f092415db10 0x7f0924069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f0928ffb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0924159c60 0x7f092415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f09297fc700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f0924159890 0x7f092415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:59844/0 (socket says 172.21.15.89:59844) 2023-10-10T20:40:48.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f09297fc700 1 -- 172.21.15.89:0/314936488 learned_addr learned my addr 172.21.15.89:0/314936488 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:48.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f091bfff700 1 -- 172.21.15.89:0/314936488 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3028836376 0 0) 0x7f092415a160 con 0x7f0924159890 2023-10-10T20:40:48.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f091bfff700 1 -- 172.21.15.89:0/314936488 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0910003570 con 0x7f0924159890 2023-10-10T20:40:48.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f0928ffb700 1 -- 172.21.15.89:0/314936488 >> v2:172.21.15.111:3300/0 conn(0x7f092415db10 msgr2=0x7f0924069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:48.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f0928ffb700 1 --2- 172.21.15.89:0/314936488 >> v2:172.21.15.111:3300/0 conn(0x7f092415db10 0x7f0924069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f0928ffb700 1 -- 172.21.15.89:0/314936488 >> v1:172.21.15.111:6789/0 conn(0x7f0924159890 legacy=0x7f092415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:48.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f0928ffb700 1 -- 172.21.15.89:0/314936488 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092415a3d0 con 0x7f0924159c60 2023-10-10T20:40:48.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f0929ffd700 1 --2- 172.21.15.89:0/314936488 >> v2:172.21.15.111:3300/0 conn(0x7f092415db10 0x7f0924069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:40:48.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f0928ffb700 1 --2- 172.21.15.89:0/314936488 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0924159c60 0x7f092415d5d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f09140098e0 tx=0x7f09140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72088fd4d8154fc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:48.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f091bfff700 1 -- 172.21.15.89:0/314936488 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914011630 con 0x7f0924159c60 2023-10-10T20:40:48.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f091bfff700 1 -- 172.21.15.89:0/314936488 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0914011790 con 0x7f0924159c60 2023-10-10T20:40:48.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.522+0000 7f091bfff700 1 -- 172.21.15.89:0/314936488 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914011b10 con 0x7f0924159c60 2023-10-10T20:40:48.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.523+0000 7f092ba60700 1 -- 172.21.15.89:0/314936488 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0924159c60 msgr2=0x7f092415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:48.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.523+0000 7f092ba60700 1 --2- 172.21.15.89:0/314936488 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0924159c60 0x7f092415d5d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f09140098e0 tx=0x7f09140091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.523+0000 7f092ba60700 1 -- 172.21.15.89:0/314936488 shutdown_connections 2023-10-10T20:40:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.523+0000 7f092ba60700 1 --2- 172.21.15.89:0/314936488 >> v2:172.21.15.111:3300/0 conn(0x7f092415db10 0x7f0924069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.523+0000 7f092ba60700 1 --2- 172.21.15.89:0/314936488 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0924159c60 0x7f092415d5d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.523+0000 7f092ba60700 1 -- 172.21.15.89:0/314936488 >> 172.21.15.89:0/314936488 conn(0x7f0924068e20 msgr2=0x7f09240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.523+0000 7f092ba60700 1 -- 172.21.15.89:0/314936488 shutdown_connections 2023-10-10T20:40:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.523+0000 7f092ba60700 1 -- 172.21.15.89:0/314936488 wait complete. 2023-10-10T20:40:48.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.524+0000 7f092ba60700 1 Processor -- start 2023-10-10T20:40:48.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.524+0000 7f092ba60700 1 -- start start 2023-10-10T20:40:48.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.525+0000 7f092ba60700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0924159890 0x7f0924152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.525+0000 7f092ba60700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0924159c60 0x7f092414ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.525+0000 7f092ba60700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f092415db10 0x7f092414d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.525+0000 7f092ba60700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092414d880 con 0x7f0924159890 2023-10-10T20:40:48.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.525+0000 7f092ba60700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09240cee80 con 0x7f0924159c60 2023-10-10T20:40:48.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.525+0000 7f092ba60700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09240cf160 con 0x7f092415db10 2023-10-10T20:40:48.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.525+0000 7f09297fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0924159890 0x7f0924152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.525+0000 7f09297fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0924159890 0x7f0924152b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59686/0 (socket says 172.21.15.89:59686) 2023-10-10T20:40:48.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.525+0000 7f09297fc700 1 -- 172.21.15.89:0/1568966365 learned_addr learned my addr 172.21.15.89:0/1568966365 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:48.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.525+0000 7f0929ffd700 1 --2- 172.21.15.89:0/1568966365 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f092415db10 0x7f092414d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.525+0000 7f0928ffb700 1 --2- 172.21.15.89:0/1568966365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0924159c60 0x7f092414ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.525+0000 7f09297fc700 1 -- 172.21.15.89:0/1568966365 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f092415db10 msgr2=0x7f092414d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:48.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.525+0000 7f09297fc700 1 --2- 172.21.15.89:0/1568966365 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f092415db10 0x7f092414d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.525+0000 7f09297fc700 1 -- 172.21.15.89:0/1568966365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0924159c60 msgr2=0x7f092414ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:48.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.525+0000 7f09297fc700 1 --2- 172.21.15.89:0/1568966365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0924159c60 0x7f092414ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.525+0000 7f09297fc700 1 -- 172.21.15.89:0/1568966365 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c009a20 con 0x7f0924159890 2023-10-10T20:40:48.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.526+0000 7f09297fc700 1 --2- 172.21.15.89:0/1568966365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0924159890 0x7f0924152b10 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f09200011e0 tx=0x7f0920007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:48.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.526+0000 7f09197fa700 1 -- 172.21.15.89:0/1568966365 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0920005b30 con 0x7f0924159890 2023-10-10T20:40:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.526+0000 7f09197fa700 1 -- 172.21.15.89:0/1568966365 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0920005c90 con 0x7f0924159890 2023-10-10T20:40:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.526+0000 7f092ba60700 1 -- 172.21.15.89:0/1568966365 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0914009420 con 0x7f0924159890 2023-10-10T20:40:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.526+0000 7f09197fa700 1 -- 172.21.15.89:0/1568966365 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09200072e0 con 0x7f0924159890 2023-10-10T20:40:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.526+0000 7f092ba60700 1 -- 172.21.15.89:0/1568966365 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09240cf720 con 0x7f0924159890 2023-10-10T20:40:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.527+0000 7f09197fa700 1 -- 172.21.15.89:0/1568966365 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f0920007650 con 0x7f0924159890 2023-10-10T20:40:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.527+0000 7f09197fa700 1 --2- 172.21.15.89:0/1568966365 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0908067290 0x7f0908069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.527+0000 7f09197fa700 1 -- 172.21.15.89:0/1568966365 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0920083ff0 con 0x7f0924159890 2023-10-10T20:40:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.528+0000 7f0928ffb700 1 --2- 172.21.15.89:0/1568966365 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0908067290 0x7f0908069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.528+0000 7f0928ffb700 1 --2- 172.21.15.89:0/1568966365 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0908067290 0x7f0908069740 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f092415a090 tx=0x7f0914017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:48.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.529+0000 7f0934d59b80 1 Processor -- start 2023-10-10T20:40:48.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.529+0000 7f0934d59b80 1 -- start start 2023-10-10T20:40:48.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.530+0000 7f0934d59b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ea8a50 0x560837ea8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.530+0000 7f0934d59b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 0x560837ed1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.530+0000 7f0934d59b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ed1f60 0x560837ea4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.530+0000 7f0934d59b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560837dffa70 con 0x560837ef2d00 2023-10-10T20:40:48.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.530+0000 7f0934d59b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x560837dfd270 con 0x560837ed1f60 2023-10-10T20:40:48.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.530+0000 7f0934d59b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x560837df9060 con 0x560837ea8a50 2023-10-10T20:40:48.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.530+0000 7f0928ffb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 0x560837ed1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.530+0000 7f0928ffb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 0x560837ed1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59698/0 (socket says 172.21.15.89:59698) 2023-10-10T20:40:48.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.530+0000 7f0928ffb700 1 -- 172.21.15.89:0/4192565164 learned_addr learned my addr 172.21.15.89:0/4192565164 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:48.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.530+0000 7f09297fc700 1 --2- 172.21.15.89:0/4192565164 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ea8a50 0x560837ea8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.530+0000 7f0929ffd700 1 --2- 172.21.15.89:0/4192565164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ed1f60 0x560837ea4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.530+0000 7f0928ffb700 1 -- 172.21.15.89:0/4192565164 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ea8a50 msgr2=0x560837ea8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:48.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.530+0000 7f0928ffb700 1 --2- 172.21.15.89:0/4192565164 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ea8a50 0x560837ea8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.530+0000 7f0928ffb700 1 -- 172.21.15.89:0/4192565164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ed1f60 msgr2=0x560837ea4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:48.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.530+0000 7f0928ffb700 1 --2- 172.21.15.89:0/4192565164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ed1f60 0x560837ea4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.530+0000 7f0928ffb700 1 -- 172.21.15.89:0/4192565164 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560837c93270 con 0x560837ef2d00 2023-10-10T20:40:48.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.530+0000 7f0929ffd700 1 --2- 172.21.15.89:0/4192565164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ed1f60 0x560837ea4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:48.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.530+0000 7f09297fc700 1 --2- 172.21.15.89:0/4192565164 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ea8a50 0x560837ea8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:48.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.531+0000 7f0928ffb700 1 --2- 172.21.15.89:0/4192565164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 0x560837ed1a20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0914011fd0 tx=0x7f091401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b32de21846009cb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:48.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.531+0000 7f090e7fc700 1 -- 172.21.15.89:0/4192565164 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914011ab0 con 0x560837ef2d00 2023-10-10T20:40:48.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.531+0000 7f090e7fc700 1 -- 172.21.15.89:0/4192565164 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091401cc60 con 0x560837ef2d00 2023-10-10T20:40:48.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.531+0000 7f090e7fc700 1 -- 172.21.15.89:0/4192565164 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091401a430 con 0x560837ef2d00 2023-10-10T20:40:48.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.531+0000 7f0934d59b80 1 -- 172.21.15.89:0/4192565164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 msgr2=0x560837ed1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:48.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.531+0000 7f0934d59b80 1 --2- 172.21.15.89:0/4192565164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 0x560837ed1a20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0914011fd0 tx=0x7f091401e040 comp rx=0 tx=0).stop 2023-10-10T20:40:48.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.531+0000 7f0934d59b80 1 -- 172.21.15.89:0/4192565164 shutdown_connections 2023-10-10T20:40:48.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.531+0000 7f0934d59b80 1 --2- 172.21.15.89:0/4192565164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ed1f60 0x560837ea4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.531+0000 7f0934d59b80 1 --2- 172.21.15.89:0/4192565164 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ea8a50 0x560837ea8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.531+0000 7f0934d59b80 1 --2- 172.21.15.89:0/4192565164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 0x560837ed1a20 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.531+0000 7f0934d59b80 1 -- 172.21.15.89:0/4192565164 >> 172.21.15.89:0/4192565164 conn(0x560837eaa370 msgr2=0x560837eaa740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.531+0000 7f0934d59b80 1 -- 172.21.15.89:0/4192565164 shutdown_connections 2023-10-10T20:40:48.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.531+0000 7f0934d59b80 1 -- 172.21.15.89:0/4192565164 wait complete. 2023-10-10T20:40:48.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.532+0000 7f0934d59b80 1 -- start start 2023-10-10T20:40:48.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.532+0000 7f0934d59b80 1 Processor -- start 2023-10-10T20:40:48.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.533+0000 7f0934d59b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ea8a50 0x560837ed8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.533+0000 7f0934d59b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ed1f60 0x560837ef26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.533+0000 7f0934d59b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ef2d00 0x560837eebdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.533+0000 7f0934d59b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560837da4890 con 0x560837ea8a50 2023-10-10T20:40:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.533+0000 7f0934d59b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x560837dbfea0 con 0x560837ef2d00 2023-10-10T20:40:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.533+0000 7f0934d59b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x560837c7be30 con 0x560837ed1f60 2023-10-10T20:40:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.533+0000 7f0934d59b80 10 client.0 _ll_register_callbacks cb 0x560837e6c110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:40:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.533+0000 7f09297fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ea8a50 0x560837ed8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.533+0000 7f09297fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ea8a50 0x560837ed8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59712/0 (socket says 172.21.15.89:59712) 2023-10-10T20:40:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.533+0000 7f09297fc700 1 -- 172.21.15.89:0/550257858 learned_addr learned my addr 172.21.15.89:0/550257858 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.533+0000 7f0928ffb700 1 --2- 172.21.15.89:0/550257858 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ed1f60 0x560837ef26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.533+0000 7f0929ffd700 1 --2- 172.21.15.89:0/550257858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ef2d00 0x560837eebdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.533+0000 7f09297fc700 1 -- 172.21.15.89:0/550257858 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ed1f60 msgr2=0x560837ef26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.533+0000 7f09297fc700 1 --2- 172.21.15.89:0/550257858 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ed1f60 0x560837ef26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.533+0000 7f09297fc700 1 -- 172.21.15.89:0/550257858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ef2d00 msgr2=0x560837eebdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.533+0000 7f09297fc700 1 --2- 172.21.15.89:0/550257858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ef2d00 0x560837eebdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.533+0000 7f09297fc700 1 -- 172.21.15.89:0/550257858 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0914022070 con 0x560837ea8a50 2023-10-10T20:40:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.533+0000 7f09297fc700 1 --2- 172.21.15.89:0/550257858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ea8a50 0x560837ed8a20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0920050fc0 tx=0x7f0920054420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38dc55372f89099 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.533+0000 7f09037fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:40:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.534+0000 7f09037fe700 1 -- 172.21.15.89:0/550257858 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092005b730 con 0x560837ea8a50 2023-10-10T20:40:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.534+0000 7f09037fe700 1 -- 172.21.15.89:0/550257858 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f09200508b0 con 0x560837ea8a50 2023-10-10T20:40:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.534+0000 7f0934d59b80 1 -- 172.21.15.89:0/550257858 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560837e9db70 con 0x560837ea8a50 2023-10-10T20:40:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.534+0000 7f09037fe700 1 -- 172.21.15.89:0/550257858 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0920053a20 con 0x560837ea8a50 2023-10-10T20:40:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.534+0000 7f0934d59b80 10 client.6993 fetch_fsmap learned FSMap version 64 2023-10-10T20:40:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.534+0000 7f0934d59b80 1 -- 172.21.15.89:0/550257858 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x560837c35fe0 con 0x560837ea8a50 2023-10-10T20:40:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.534+0000 7f09037fe700 1 -- 172.21.15.89:0/550257858 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560837e9db70 con 0x560837ea8a50 2023-10-10T20:40:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.534+0000 7f09037fe700 1 -- 172.21.15.89:0/550257858 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 64) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0920053b80 con 0x560837ea8a50 2023-10-10T20:40:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.534+0000 7f0934d59b80 10 client.6993 fetch_fsmap finished waiting for FSMap version 64 2023-10-10T20:40:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.534+0000 7f0934d59b80 10 client.6993 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-10T20:40:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.534+0000 7f0934d59b80 20 client.6993 populate_metadata read hostname 'smithi089' 2023-10-10T20:40:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.534+0000 7f0934d59b80 1 --2- 172.21.15.89:0/550257858 >> [v2:172.21.15.89:6834/2274240619,v1:172.21.15.89:6836/2274240619] conn(0x560837eea2b0 0x560837ee8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.534+0000 7f0934d59b80 4 client.6993 mds_command: new command op to 6822 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:40:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.534+0000 7f0934d59b80 1 -- 172.21.15.89:0/550257858 --> [v2:172.21.15.89:6834/2274240619,v1:172.21.15.89:6836/2274240619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560837e7aaa0 con 0x560837eea2b0 2023-10-10T20:40:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.534+0000 7f0928ffb700 1 --2- 172.21.15.89:0/550257858 >> [v2:172.21.15.89:6834/2274240619,v1:172.21.15.89:6836/2274240619] conn(0x560837eea2b0 0x560837ee8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.534+0000 7f0928ffb700 1 --2- 172.21.15.89:0/550257858 >> [v2:172.21.15.89:6834/2274240619,v1:172.21.15.89:6836/2274240619] conn(0x560837eea2b0 0x560837ee8f40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66ab1ce4bd40f91b server_cookie=8b95c55a7d4ab662 in_seq=0 out_seq=0 2023-10-10T20:40:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.534+0000 7f09037fe700 10 client.6993 ms_handle_connect on v2:172.21.15.89:6834/2274240619 2023-10-10T20:40:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.535+0000 7f09037fe700 1 -- 172.21.15.89:0/550257858 <== mds.0 v2:172.21.15.89:6834/2274240619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x560837e7aaa0 con 0x560837eea2b0 2023-10-10T20:40:48.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.535+0000 7f09037fe700 10 client.6993 handle_command_reply: tid=0 2023-10-10T20:40:48.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.535+0000 7f0934d59b80 1 client.6993 shutdown 2023-10-10T20:40:48.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.535+0000 7f0934d59b80 1 -- 172.21.15.89:0/550257858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ea8a50 msgr2=0x560837ed8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:48.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.535+0000 7f0934d59b80 1 --2- 172.21.15.89:0/550257858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ea8a50 0x560837ed8a20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0920050fc0 tx=0x7f0920054420 comp rx=0 tx=0).stop 2023-10-10T20:40:48.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.535+0000 7f0934d59b80 1 -- 172.21.15.89:0/550257858 shutdown_connections 2023-10-10T20:40:48.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.535+0000 7f0934d59b80 1 --2- 172.21.15.89:0/550257858 >> [v2:172.21.15.89:6834/2274240619,v1:172.21.15.89:6836/2274240619] conn(0x560837eea2b0 0x560837ee8f40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.535+0000 7f0934d59b80 1 --2- 172.21.15.89:0/550257858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ef2d00 0x560837eebdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.535+0000 7f09037fe700 0 client.6993 ms_handle_reset on v2:172.21.15.89:6834/2274240619 2023-10-10T20:40:48.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.535+0000 7f0934d59b80 1 --2- 172.21.15.89:0/550257858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ea8a50 0x560837ed8a20 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.535+0000 7f0934d59b80 1 --2- 172.21.15.89:0/550257858 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ed1f60 0x560837ef26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.535+0000 7f0934d59b80 1 -- 172.21.15.89:0/550257858 >> 172.21.15.89:0/550257858 conn(0x560837eaa370 msgr2=0x560837ea5780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:48.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.535+0000 7f0934d59b80 1 -- 172.21.15.89:0/550257858 shutdown_connections 2023-10-10T20:40:48.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.535+0000 7f0934d59b80 1 -- 172.21.15.89:0/550257858 wait complete. 2023-10-10T20:40:48.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.535+0000 7f0934d59b80 20 client.6993 trim_cache size 0 max 16384 2023-10-10T20:40:48.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.552+0000 7f0934d59b80 1 Processor -- start 2023-10-10T20:40:48.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.552+0000 7f0934d59b80 1 -- start start 2023-10-10T20:40:48.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.552+0000 7f0934d59b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ea8a50 0x560837ea5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.552+0000 7f0934d59b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ed1f60 0x560837eaa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.552+0000 7f0934d59b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 0x560837ee5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.552+0000 7f0934d59b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560837eea750 con 0x560837ef2d00 2023-10-10T20:40:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.552+0000 7f0934d59b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x560837ea3dc0 con 0x560837ea8a50 2023-10-10T20:40:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.552+0000 7f0934d59b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x560837ef17c0 con 0x560837ed1f60 2023-10-10T20:40:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.553+0000 7f0929ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 0x560837ee5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.553+0000 7f0929ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 0x560837ee5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59718/0 (socket says 172.21.15.89:59718) 2023-10-10T20:40:48.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.553+0000 7f0929ffd700 1 -- 172.21.15.89:0/1134699480 learned_addr learned my addr 172.21.15.89:0/1134699480 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:48.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.553+0000 7f0928ffb700 1 --2- 172.21.15.89:0/1134699480 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ed1f60 0x560837eaa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.553+0000 7f0929ffd700 1 -- 172.21.15.89:0/1134699480 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ed1f60 msgr2=0x560837eaa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:48.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.553+0000 7f09297fc700 1 --2- 172.21.15.89:0/1134699480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ea8a50 0x560837ea5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.553+0000 7f0929ffd700 1 --2- 172.21.15.89:0/1134699480 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ed1f60 0x560837eaa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.553+0000 7f0929ffd700 1 -- 172.21.15.89:0/1134699480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ea8a50 msgr2=0x560837ea5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:48.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.553+0000 7f0929ffd700 1 --2- 172.21.15.89:0/1134699480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ea8a50 0x560837ea5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.553+0000 7f0929ffd700 1 -- 172.21.15.89:0/1134699480 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560837dc7780 con 0x560837ef2d00 2023-10-10T20:40:48.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.553+0000 7f0928ffb700 1 --2- 172.21.15.89:0/1134699480 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ed1f60 0x560837eaa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:48.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.553+0000 7f0929ffd700 1 --2- 172.21.15.89:0/1134699480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 0x560837ee5560 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f091c0077d0 tx=0x7f091c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f5beb3c1a09ee8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:48.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.553+0000 7f090effd700 1 -- 172.21.15.89:0/1134699480 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c017680 con 0x560837ef2d00 2023-10-10T20:40:48.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.553+0000 7f090effd700 1 -- 172.21.15.89:0/1134699480 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c00f040 con 0x560837ef2d00 2023-10-10T20:40:48.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.553+0000 7f090effd700 1 -- 172.21.15.89:0/1134699480 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c017a50 con 0x560837ef2d00 2023-10-10T20:40:48.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.554+0000 7f0934d59b80 1 -- 172.21.15.89:0/1134699480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 msgr2=0x560837ee5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.554+0000 7f0934d59b80 1 --2- 172.21.15.89:0/1134699480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 0x560837ee5560 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f091c0077d0 tx=0x7f091c00b040 comp rx=0 tx=0).stop 2023-10-10T20:40:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.554+0000 7f0934d59b80 1 -- 172.21.15.89:0/1134699480 shutdown_connections 2023-10-10T20:40:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.554+0000 7f0934d59b80 1 --2- 172.21.15.89:0/1134699480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 0x560837ee5560 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.554+0000 7f0934d59b80 1 --2- 172.21.15.89:0/1134699480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ea8a50 0x560837ea5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.554+0000 7f0934d59b80 1 --2- 172.21.15.89:0/1134699480 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ed1f60 0x560837eaa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.554+0000 7f0934d59b80 1 -- 172.21.15.89:0/1134699480 >> 172.21.15.89:0/1134699480 conn(0x560837eaa370 msgr2=0x560837ea14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.554+0000 7f0934d59b80 1 -- 172.21.15.89:0/1134699480 shutdown_connections 2023-10-10T20:40:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.554+0000 7f0934d59b80 1 -- 172.21.15.89:0/1134699480 wait complete. 2023-10-10T20:40:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.554+0000 7f0934d59b80 1 -- start start 2023-10-10T20:40:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.554+0000 7f0934d59b80 1 Processor -- start 2023-10-10T20:40:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.555+0000 7f0934d59b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ea8a50 0x560837ef30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.555+0000 7f0934d59b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ed1f60 0x560837ef3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.555+0000 7f0934d59b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 0x560837f01110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.555+0000 7f0934d59b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560837f10c50 con 0x560837ef2d00 2023-10-10T20:40:48.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.555+0000 7f0934d59b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x560837ea49c0 con 0x560837ea8a50 2023-10-10T20:40:48.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.555+0000 7f0934d59b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x560837f0f480 con 0x560837ed1f60 2023-10-10T20:40:48.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.555+0000 7f0934d59b80 10 client.0 _ll_register_callbacks cb 0x560837e6c110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:40:48.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.555+0000 7f0929ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 0x560837f01110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.555+0000 7f0929ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 0x560837f01110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59726/0 (socket says 172.21.15.89:59726) 2023-10-10T20:40:48.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.555+0000 7f0929ffd700 1 -- 172.21.15.89:0/1436078189 learned_addr learned my addr 172.21.15.89:0/1436078189 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:48.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.555+0000 7f09297fc700 1 --2- 172.21.15.89:0/1436078189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ea8a50 0x560837ef30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.555+0000 7f0928ffb700 1 --2- 172.21.15.89:0/1436078189 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ed1f60 0x560837ef3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f0929ffd700 1 -- 172.21.15.89:0/1436078189 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ed1f60 msgr2=0x560837ef3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:48.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f0929ffd700 1 --2- 172.21.15.89:0/1436078189 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ed1f60 0x560837ef3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f0929ffd700 1 -- 172.21.15.89:0/1436078189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ea8a50 msgr2=0x560837ef30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:48.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f0929ffd700 1 --2- 172.21.15.89:0/1436078189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ea8a50 0x560837ef30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f0929ffd700 1 -- 172.21.15.89:0/1436078189 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091c007530 con 0x560837ef2d00 2023-10-10T20:40:48.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f0929ffd700 1 --2- 172.21.15.89:0/1436078189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 0x560837f01110 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f091c009f10 tx=0x7f091c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4092f1ebdc46b968 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:48.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f09037fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:40:48.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f09037fe700 1 -- 172.21.15.89:0/1436078189 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c001060 con 0x560837ef2d00 2023-10-10T20:40:48.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f09037fe700 1 -- 172.21.15.89:0/1436078189 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f091c021640 con 0x560837ef2d00 2023-10-10T20:40:48.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f0934d59b80 1 -- 172.21.15.89:0/1436078189 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f091401cc60 con 0x560837ef2d00 2023-10-10T20:40:48.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f09037fe700 1 -- 172.21.15.89:0/1436078189 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f091c021a10 con 0x560837ef2d00 2023-10-10T20:40:48.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f0934d59b80 10 client.6999 fetch_fsmap learned FSMap version 64 2023-10-10T20:40:48.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f0934d59b80 1 -- 172.21.15.89:0/1436078189 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x560837dae470 con 0x560837ef2d00 2023-10-10T20:40:48.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f09037fe700 1 -- 172.21.15.89:0/1436078189 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091401cc60 con 0x560837ef2d00 2023-10-10T20:40:48.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f09037fe700 1 -- 172.21.15.89:0/1436078189 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 64) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f091c021b70 con 0x560837ef2d00 2023-10-10T20:40:48.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f0934d59b80 10 client.6999 fetch_fsmap finished waiting for FSMap version 64 2023-10-10T20:40:48.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f0934d59b80 10 client.6999 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-10T20:40:48.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f0934d59b80 20 client.6999 populate_metadata read hostname 'smithi089' 2023-10-10T20:40:48.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f0934d59b80 1 --2- 172.21.15.89:0/1436078189 >> [v2:172.21.15.89:6834/2274240619,v1:172.21.15.89:6836/2274240619] conn(0x560837f01710 0x560837f01ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f0934d59b80 4 client.6999 mds_command: new command op to 6822 tid=0[{"prefix": "status"}] 2023-10-10T20:40:48.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f0934d59b80 1 -- 172.21.15.89:0/1436078189 --> [v2:172.21.15.89:6834/2274240619,v1:172.21.15.89:6836/2274240619] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560837d9be00 con 0x560837f01710 2023-10-10T20:40:48.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.556+0000 7f09297fc700 1 --2- 172.21.15.89:0/1436078189 >> [v2:172.21.15.89:6834/2274240619,v1:172.21.15.89:6836/2274240619] conn(0x560837f01710 0x560837f01ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.557+0000 7f09037fe700 10 client.6999 ms_handle_connect on v2:172.21.15.89:6834/2274240619 2023-10-10T20:40:48.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.557+0000 7f09297fc700 1 --2- 172.21.15.89:0/1436078189 >> [v2:172.21.15.89:6834/2274240619,v1:172.21.15.89:6836/2274240619] conn(0x560837f01710 0x560837f01ae0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d23a059c75a81cde server_cookie=323a1551bdb3d686 in_seq=0 out_seq=0 2023-10-10T20:40:48.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.557+0000 7f09037fe700 1 -- 172.21.15.89:0/1436078189 <== mds.0 v2:172.21.15.89:6834/2274240619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x560837d9be00 con 0x560837f01710 2023-10-10T20:40:48.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.557+0000 7f09037fe700 10 client.6999 handle_command_reply: tid=0 2023-10-10T20:40:48.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.557+0000 7f0934d59b80 1 client.6999 shutdown 2023-10-10T20:40:48.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.557+0000 7f0934d59b80 1 -- 172.21.15.89:0/1436078189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 msgr2=0x560837f01110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:48.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.557+0000 7f0934d59b80 1 --2- 172.21.15.89:0/1436078189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 0x560837f01110 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f091c009f10 tx=0x7f091c01a000 comp rx=0 tx=0).stop 2023-10-10T20:40:48.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.557+0000 7f0934d59b80 1 -- 172.21.15.89:0/1436078189 shutdown_connections 2023-10-10T20:40:48.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.557+0000 7f0934d59b80 1 --2- 172.21.15.89:0/1436078189 >> [v2:172.21.15.89:6834/2274240619,v1:172.21.15.89:6836/2274240619] conn(0x560837f01710 0x560837f01ae0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.557+0000 7f0934d59b80 1 --2- 172.21.15.89:0/1436078189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560837ef2d00 0x560837f01110 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.557+0000 7f0934d59b80 1 --2- 172.21.15.89:0/1436078189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560837ea8a50 0x560837ef30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.557+0000 7f0934d59b80 1 --2- 172.21.15.89:0/1436078189 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x560837ed1f60 0x560837ef3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.557+0000 7f09037fe700 0 client.6999 ms_handle_reset on v2:172.21.15.89:6834/2274240619 2023-10-10T20:40:48.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.557+0000 7f0934d59b80 1 -- 172.21.15.89:0/1436078189 >> 172.21.15.89:0/1436078189 conn(0x560837eaa370 msgr2=0x560837ea55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:48.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.557+0000 7f0934d59b80 1 -- 172.21.15.89:0/1436078189 shutdown_connections 2023-10-10T20:40:48.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.557+0000 7f0934d59b80 1 -- 172.21.15.89:0/1436078189 wait complete. 2023-10-10T20:40:48.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.557+0000 7f0934d59b80 20 client.6999 trim_cache size 0 max 16384 2023-10-10T20:40:48.595 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:40:48.595 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "5dd675b8-9026-4d8a-a1bf-62fdc5e7de67", 2023-10-10T20:40:48.595 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-10-10T20:40:48.595 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 6822, 2023-10-10T20:40:48.595 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-10-10T20:40:48.596 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-10-10T20:40:48.596 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-10-10T20:40:48.596 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 10.293661793, 2023-10-10T20:40:48.596 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 64, 2023-10-10T20:40:48.596 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 100, 2023-10-10T20:40:48.597 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 100, 2023-10-10T20:40:48.597 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 15.414867593 2023-10-10T20:40:48.597 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:40:48.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.558+0000 7f092ba60700 1 -- 172.21.15.89:0/1568966365 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0908067290 msgr2=0x7f0908069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:48.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.558+0000 7f092ba60700 1 --2- 172.21.15.89:0/1568966365 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0908067290 0x7f0908069740 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f092415a090 tx=0x7f0914017000 comp rx=0 tx=0).stop 2023-10-10T20:40:48.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.558+0000 7f092ba60700 1 -- 172.21.15.89:0/1568966365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0924159890 msgr2=0x7f0924152b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:48.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.558+0000 7f092ba60700 1 --2- 172.21.15.89:0/1568966365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0924159890 0x7f0924152b10 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f09200011e0 tx=0x7f0920007df0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.558+0000 7f092ba60700 1 -- 172.21.15.89:0/1568966365 shutdown_connections 2023-10-10T20:40:48.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.558+0000 7f092ba60700 1 --2- 172.21.15.89:0/1568966365 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f092415db10 0x7f092414d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.558+0000 7f092ba60700 1 --2- 172.21.15.89:0/1568966365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0924159890 0x7f0924152b10 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.558+0000 7f092ba60700 1 --2- 172.21.15.89:0/1568966365 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0908067290 0x7f0908069740 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.558+0000 7f092ba60700 1 --2- 172.21.15.89:0/1568966365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0924159c60 0x7f092414ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.558+0000 7f092ba60700 1 -- 172.21.15.89:0/1568966365 >> 172.21.15.89:0/1568966365 conn(0x7f0924068e20 msgr2=0x7f092415cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:48.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.558+0000 7f092ba60700 1 -- 172.21.15.89:0/1568966365 shutdown_connections 2023-10-10T20:40:48.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.558+0000 7f092ba60700 1 -- 172.21.15.89:0/1568966365 wait complete. 2023-10-10T20:40:48.601 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:40:48.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.713+0000 7f61bb165700 1 Processor -- start 2023-10-10T20:40:48.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.713+0000 7f61bb165700 1 -- start start 2023-10-10T20:40:48.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.714+0000 7f61bb165700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61b4159890 0x7f61b415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.714+0000 7f61bb165700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b415a160 con 0x7f61b4159c60 2023-10-10T20:40:48.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.714+0000 7f61bb165700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b40d13a0 con 0x7f61b4069560 2023-10-10T20:40:48.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.714+0000 7f61bb165700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b415e140 con 0x7f61b4159890 2023-10-10T20:40:48.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.714+0000 7f61b3fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f61b4159c60 0x7f61b415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34060/0 (socket says 172.21.15.89:34060) 2023-10-10T20:40:48.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.714+0000 7f61b3fff700 1 -- 172.21.15.89:0/3185829784 learned_addr learned my addr 172.21.15.89:0/3185829784 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:48.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.714+0000 7f61b8f01700 1 --2- 172.21.15.89:0/3185829784 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61b4159890 0x7f61b415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.714+0000 7f61b37fe700 1 -- 172.21.15.89:0/3185829784 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916283661 0 0) 0x7f61b415a160 con 0x7f61b4159c60 2023-10-10T20:40:48.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b37fe700 1 -- 172.21.15.89:0/3185829784 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61a0003570 con 0x7f61b4159c60 2023-10-10T20:40:48.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b37fe700 1 -- 172.21.15.89:0/3185829784 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2089426629 0 0) 0x7f61a0003570 con 0x7f61b4159c60 2023-10-10T20:40:48.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b37fe700 1 -- 172.21.15.89:0/3185829784 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61b4159890 msgr2=0x7f61b415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:48.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b37fe700 1 --2- 172.21.15.89:0/3185829784 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61b4159890 0x7f61b415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b37fe700 1 -- 172.21.15.89:0/3185829784 >> v1:172.21.15.111:6789/0 conn(0x7f61b4069560 legacy=0x7f61b40ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:48.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b37fe700 1 -- 172.21.15.89:0/3185829784 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b415a3d0 con 0x7f61b4159c60 2023-10-10T20:40:48.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b37fe700 1 -- 172.21.15.89:0/3185829784 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f619c001470 con 0x7f61b4159c60 2023-10-10T20:40:48.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b8f01700 1 --2- 172.21.15.89:0/3185829784 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61b4159890 0x7f61b415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:40:48.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b37fe700 1 -- 172.21.15.89:0/3185829784 >> v1:172.21.15.89:6789/0 conn(0x7f61b4159c60 legacy=0x7f61b415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:48.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b37fe700 1 --2- 172.21.15.89:0/3185829784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61a0003a90 0x7f61a0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b37fe700 1 --2- 172.21.15.89:0/3185829784 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61b4159890 0x7f61a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b37fe700 1 --2- 172.21.15.89:0/3185829784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a0004d50 0x7f61a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b37fe700 1 -- 172.21.15.89:0/3185829784 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b415e140 con 0x7f61a0004d50 2023-10-10T20:40:48.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b37fe700 1 -- 172.21.15.89:0/3185829784 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a000bb10 con 0x7f61a0003a90 2023-10-10T20:40:48.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b37fe700 1 -- 172.21.15.89:0/3185829784 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a000bc80 con 0x7f61b4159890 2023-10-10T20:40:48.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b9702700 1 --2- 172.21.15.89:0/3185829784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a0004d50 0x7f61a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b8f01700 1 --2- 172.21.15.89:0/3185829784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61a0003a90 0x7f61a0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b3fff700 1 --2- 172.21.15.89:0/3185829784 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61b4159890 0x7f61a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b9702700 1 -- 172.21.15.89:0/3185829784 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61b4159890 msgr2=0x7f61a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:48.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b9702700 1 --2- 172.21.15.89:0/3185829784 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61b4159890 0x7f61a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b9702700 1 -- 172.21.15.89:0/3185829784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61a0003a90 msgr2=0x7f61a0003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:48.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b9702700 1 --2- 172.21.15.89:0/3185829784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61a0003a90 0x7f61a0003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b9702700 1 -- 172.21.15.89:0/3185829784 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a4009a20 con 0x7f61a0004d50 2023-10-10T20:40:48.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.715+0000 7f61b9702700 1 --2- 172.21.15.89:0/3185829784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a0004d50 0x7f61a00051c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0019a0 tx=0x7f61ac007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af72f37289d17cc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:48.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.716+0000 7f61b37fe700 1 -- 172.21.15.89:0/3185829784 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac00c5d0 con 0x7f61a0004d50 2023-10-10T20:40:48.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.716+0000 7f61b37fe700 1 -- 172.21.15.89:0/3185829784 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ac00c750 con 0x7f61a0004d50 2023-10-10T20:40:48.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.716+0000 7f61b37fe700 1 -- 172.21.15.89:0/3185829784 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac00cb10 con 0x7f61a0004d50 2023-10-10T20:40:48.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.716+0000 7f61bb165700 1 -- 172.21.15.89:0/3185829784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a0004d50 msgr2=0x7f61a00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:48.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.716+0000 7f61bb165700 1 --2- 172.21.15.89:0/3185829784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a0004d50 0x7f61a00051c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0019a0 tx=0x7f61ac007d10 comp rx=0 tx=0).stop 2023-10-10T20:40:48.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.716+0000 7f61b9702700 1 -- 172.21.15.89:0/3185829784 reap_dead start 2023-10-10T20:40:48.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.716+0000 7f61bb165700 1 -- 172.21.15.89:0/3185829784 shutdown_connections 2023-10-10T20:40:48.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.716+0000 7f61bb165700 1 -- 172.21.15.89:0/3185829784 >> 172.21.15.89:0/3185829784 conn(0x7f61b4068e20 msgr2=0x7f61b40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:48.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.716+0000 7f61bb165700 1 -- 172.21.15.89:0/3185829784 shutdown_connections 2023-10-10T20:40:48.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.716+0000 7f61bb165700 1 -- 172.21.15.89:0/3185829784 wait complete. 2023-10-10T20:40:48.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.717+0000 7f61bb165700 1 Processor -- start 2023-10-10T20:40:48.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61bb165700 1 -- start start 2023-10-10T20:40:48.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61bb165700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61b4149c20 0x7f61b415ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61bb165700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61b4149ff0 0x7f61b415cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61bb165700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61b40cd750 0x7f61b40cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61bb165700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b40ce4a0 con 0x7f61b4149c20 2023-10-10T20:40:48.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61bb165700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b40ce780 con 0x7f61b40cd750 2023-10-10T20:40:48.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61bb165700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b4156e80 con 0x7f61b4149ff0 2023-10-10T20:40:48.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61b8f01700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61b4149c20 0x7f61b415ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61b8f01700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61b4149c20 0x7f61b415ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59732/0 (socket says 172.21.15.89:59732) 2023-10-10T20:40:48.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61b8f01700 1 -- 172.21.15.89:0/281400133 learned_addr learned my addr 172.21.15.89:0/281400133 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:48.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61b8f01700 1 -- 172.21.15.89:0/281400133 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61b4149ff0 msgr2=0x7f61b415cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:48.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61b8f01700 1 --2- 172.21.15.89:0/281400133 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61b4149ff0 0x7f61b415cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61b8f01700 1 -- 172.21.15.89:0/281400133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61b40cd750 msgr2=0x7f61b40cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:48.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61b8f01700 1 --2- 172.21.15.89:0/281400133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61b40cd750 0x7f61b40cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61b8f01700 1 -- 172.21.15.89:0/281400133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b4157160 con 0x7f61b4149c20 2023-10-10T20:40:48.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61b8f01700 1 --2- 172.21.15.89:0/281400133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61b4149c20 0x7f61b415ca40 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f61a400f230 tx=0x7f61a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:48.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61b0ff9700 1 -- 172.21.15.89:0/281400133 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a400e040 con 0x7f61b4149c20 2023-10-10T20:40:48.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61b0ff9700 1 -- 172.21.15.89:0/281400133 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a400f040 con 0x7f61b4149c20 2023-10-10T20:40:48.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61bb165700 1 -- 172.21.15.89:0/281400133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b4157440 con 0x7f61b4149c20 2023-10-10T20:40:48.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61b0ff9700 1 -- 172.21.15.89:0/281400133 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4015660 con 0x7f61b4149c20 2023-10-10T20:40:48.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.719+0000 7f61bb165700 1 -- 172.21.15.89:0/281400133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b40ce0e0 con 0x7f61b4149c20 2023-10-10T20:40:48.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.720+0000 7f61bb165700 1 -- 172.21.15.89:0/281400133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b4157660 con 0x7f61b4149c20 2023-10-10T20:40:48.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.720+0000 7f61b0ff9700 1 -- 172.21.15.89:0/281400133 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f61a4015a10 con 0x7f61b4149c20 2023-10-10T20:40:48.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.721+0000 7f61b0ff9700 1 --2- 172.21.15.89:0/281400133 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f61940672c0 0x7f6194069770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:48.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.721+0000 7f61b0ff9700 1 -- 172.21.15.89:0/281400133 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f61a4085c20 con 0x7f61b4149c20 2023-10-10T20:40:48.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.721+0000 7f61b3fff700 1 --2- 172.21.15.89:0/281400133 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f61940672c0 0x7f6194069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:48.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.721+0000 7f61b3fff700 1 --2- 172.21.15.89:0/281400133 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f61940672c0 0x7f6194069770 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f61b40b29f0 tx=0x7f619c001e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:48.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.725+0000 7f61b0ff9700 1 -- 172.21.15.89:0/281400133 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61a4052870 con 0x7f61b4149c20 2023-10-10T20:40:48.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.952+0000 7f61bb165700 1 -- 172.21.15.89:0/281400133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f61b404c3c0 con 0x7f61b4149c20 2023-10-10T20:40:48.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.957+0000 7f61b0ff9700 1 -- 172.21.15.89:0/281400133 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) v1 ==== 53+0+73 (secure 0 0 0) 0x7f61a4019060 con 0x7f61b4149c20 2023-10-10T20:40:48.959 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:40:48.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.958+0000 7f61bb165700 1 -- 172.21.15.89:0/281400133 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f61940672c0 msgr2=0x7f6194069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:48.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.958+0000 7f61bb165700 1 --2- 172.21.15.89:0/281400133 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f61940672c0 0x7f6194069770 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f61b40b29f0 tx=0x7f619c001e70 comp rx=0 tx=0).stop 2023-10-10T20:40:48.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.958+0000 7f61bb165700 1 -- 172.21.15.89:0/281400133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61b4149c20 msgr2=0x7f61b415ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:48.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.958+0000 7f61bb165700 1 --2- 172.21.15.89:0/281400133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61b4149c20 0x7f61b415ca40 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f61a400f230 tx=0x7f61a400b040 comp rx=0 tx=0).stop 2023-10-10T20:40:48.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.958+0000 7f61bb165700 1 -- 172.21.15.89:0/281400133 shutdown_connections 2023-10-10T20:40:48.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.958+0000 7f61bb165700 1 --2- 172.21.15.89:0/281400133 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f61940672c0 0x7f6194069770 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.958+0000 7f61bb165700 1 --2- 172.21.15.89:0/281400133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61b40cd750 0x7f61b40cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.958+0000 7f61bb165700 1 --2- 172.21.15.89:0/281400133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61b4149c20 0x7f61b415ca40 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.958+0000 7f61bb165700 1 --2- 172.21.15.89:0/281400133 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61b4149ff0 0x7f61b415cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:48.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.958+0000 7f61bb165700 1 -- 172.21.15.89:0/281400133 >> 172.21.15.89:0/281400133 conn(0x7f61b4068e20 msgr2=0x7f61b40c6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:48.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.958+0000 7f61bb165700 1 -- 172.21.15.89:0/281400133 shutdown_connections 2023-10-10T20:40:48.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:48.958+0000 7f61bb165700 1 -- 172.21.15.89:0/281400133 wait complete. 2023-10-10T20:40:48.974 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:40:48.974 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:40:48.975 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-10T20:40:48.975 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:40:48.976 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:40:48.976 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:40:48.977 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:40:48.977 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:40:48.978 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:40:48.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:48.979 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-10T20:40:48.995 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:40:48.996 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:40:48.996 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:40:48.996 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:48.997 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-10T20:40:48.997 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:48.998 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:40:48.998 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:48.999 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-10T20:40:48.999 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:40:48.999 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:48.999 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-10T20:40:49.000 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:40:49.000 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:49.000 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:40:49.000 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:49.000 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:40:49.001 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42117sec preferred_lft 42117sec 2023-10-10T20:40:49.001 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2023-10-10T20:40:49.001 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:49.001 INFO:teuthology.orchestra.run.smithi089.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:40:49.001 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:49.002 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:40:49.002 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:49.002 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::98a3:d4ff:fec1:83bb/64 scope link 2023-10-10T20:40:49.002 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:49.003 INFO:teuthology.orchestra.run.smithi089.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:40:49.003 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether ee:16:22:10:ca:9a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:40:49.003 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec16:22ff:fe10:ca9a/64 scope link 2023-10-10T20:40:49.003 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:49.003 INFO:teuthology.orchestra.run.smithi089.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:40:49.004 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:40:49.004 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::6864:2bff:fe5c:71d4/64 scope link 2023-10-10T20:40:49.004 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:49.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:49.005 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:40:49.058 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-10T20:40:49.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:49.060 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-10T20:40:49.115 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-10T20:40:49.116 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:40:49.116 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:40:49.116 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:40:49.116 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:40:49.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:49.117 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:40:49.117 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:40:49.118 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:40:49.118 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:40:49.119 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:40:49.119 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:40:49.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:49.216 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-10T20:40:49.231 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:40:49.231 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:40:49.231 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:40:49.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:49.233 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-10-10T20:40:49.295 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:40:49.295 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:40:49.350 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:40:49.351 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:40:49.351 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:40:49.405 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:40:49.405 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:40:49.406 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:40:49.484 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:40:49.484 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:40:49.485 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:40:49.492 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:40:49.492 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:40:49.493 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:40:49.493 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:40:49.493 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:40:49.514 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:40:49.515 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:40:49.515 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:40:49.515 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:40:49.515 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:40:49.516 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:40:49.516 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:40:49.516 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:40:49.517 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:40:49.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:49.518 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:40:49.582 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:40:49.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.735+0000 7f37cc5a7700 1 Processor -- start 2023-10-10T20:40:49.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.735+0000 7f37cc5a7700 1 -- start start 2023-10-10T20:40:49.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.736+0000 7f37cc5a7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37c41578b0 0x7f37c4155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:49.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.736+0000 7f37cc5a7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c40ad930 con 0x7f37c40af2f0 2023-10-10T20:40:49.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.736+0000 7f37cc5a7700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c40acd20 con 0x7f37c4155c10 2023-10-10T20:40:49.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.736+0000 7f37cc5a7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c40ad000 con 0x7f37c41578b0 2023-10-10T20:40:49.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.736+0000 7f37cab44700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f37c40af2f0 0x7f37c40af710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34076/0 (socket says 172.21.15.89:34076) 2023-10-10T20:40:49.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.736+0000 7f37cab44700 1 -- 172.21.15.89:0/2887643804 learned_addr learned my addr 172.21.15.89:0/2887643804 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:49.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37c9341700 1 -- 172.21.15.89:0/2887643804 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732687364 0 0) 0x7f37c40ad930 con 0x7f37c40af2f0 2023-10-10T20:40:49.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37c9b42700 1 --2- 172.21.15.89:0/2887643804 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37c41578b0 0x7f37c4155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:49.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37c9341700 1 -- 172.21.15.89:0/2887643804 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37b0003570 con 0x7f37c40af2f0 2023-10-10T20:40:49.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37c9341700 1 -- 172.21.15.89:0/2887643804 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1683494053 0 0) 0x7f37b0003570 con 0x7f37c40af2f0 2023-10-10T20:40:49.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37c9341700 1 -- 172.21.15.89:0/2887643804 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37c41578b0 msgr2=0x7f37c4155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:49.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37c9341700 1 --2- 172.21.15.89:0/2887643804 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37c41578b0 0x7f37c4155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:49.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37c9341700 1 -- 172.21.15.89:0/2887643804 >> v1:172.21.15.111:6789/0 conn(0x7f37c4155c10 legacy=0x7f37c40ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:49.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37c9341700 1 -- 172.21.15.89:0/2887643804 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c40ad2e0 con 0x7f37c40af2f0 2023-10-10T20:40:49.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37c9341700 1 -- 172.21.15.89:0/2887643804 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f37bc001470 con 0x7f37c40af2f0 2023-10-10T20:40:49.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37c9341700 1 -- 172.21.15.89:0/2887643804 >> v1:172.21.15.89:6789/0 conn(0x7f37c40af2f0 legacy=0x7f37c40af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:49.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37c9341700 1 --2- 172.21.15.89:0/2887643804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37b0003a90 0x7f37b0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:49.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37c9341700 1 --2- 172.21.15.89:0/2887643804 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37c41578b0 0x7f37b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:49.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37c9341700 1 --2- 172.21.15.89:0/2887643804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37b0004d50 0x7f37b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:49.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37c9341700 1 -- 172.21.15.89:0/2887643804 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c40ad000 con 0x7f37b0004d50 2023-10-10T20:40:49.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37c9341700 1 -- 172.21.15.89:0/2887643804 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b000bba0 con 0x7f37b0003a90 2023-10-10T20:40:49.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37c9341700 1 -- 172.21.15.89:0/2887643804 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b000bd60 con 0x7f37c41578b0 2023-10-10T20:40:49.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37cab44700 1 --2- 172.21.15.89:0/2887643804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37b0004d50 0x7f37b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:49.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37c9b42700 1 --2- 172.21.15.89:0/2887643804 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37c41578b0 0x7f37b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:49.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37ca343700 1 --2- 172.21.15.89:0/2887643804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37b0003a90 0x7f37b0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:49.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37cab44700 1 -- 172.21.15.89:0/2887643804 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37c41578b0 msgr2=0x7f37b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:49.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37cab44700 1 --2- 172.21.15.89:0/2887643804 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37c41578b0 0x7f37b0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:49.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37cab44700 1 -- 172.21.15.89:0/2887643804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37b0003a90 msgr2=0x7f37b0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:49.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37cab44700 1 --2- 172.21.15.89:0/2887643804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37b0003a90 0x7f37b0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:49.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.737+0000 7f37cab44700 1 -- 172.21.15.89:0/2887643804 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b0003570 con 0x7f37b0004d50 2023-10-10T20:40:49.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.738+0000 7f37cab44700 1 --2- 172.21.15.89:0/2887643804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37b0004d50 0x7f37b00051c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f37bc007f30 tx=0x7f37c40ad8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5f17b532a1364b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:49.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.738+0000 7f37c9341700 1 -- 172.21.15.89:0/2887643804 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc00cc30 con 0x7f37b0004d50 2023-10-10T20:40:49.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.738+0000 7f37c9341700 1 -- 172.21.15.89:0/2887643804 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37bc00cd90 con 0x7f37b0004d50 2023-10-10T20:40:49.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.738+0000 7f37c9341700 1 -- 172.21.15.89:0/2887643804 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc005b50 con 0x7f37b0004d50 2023-10-10T20:40:49.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.738+0000 7f37cc5a7700 1 -- 172.21.15.89:0/2887643804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37b0004d50 msgr2=0x7f37b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:49.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.738+0000 7f37cc5a7700 1 --2- 172.21.15.89:0/2887643804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37b0004d50 0x7f37b00051c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f37bc007f30 tx=0x7f37c40ad8a0 comp rx=0 tx=0).stop 2023-10-10T20:40:49.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.738+0000 7f37cab44700 1 -- 172.21.15.89:0/2887643804 reap_dead start 2023-10-10T20:40:49.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.738+0000 7f37cc5a7700 1 -- 172.21.15.89:0/2887643804 shutdown_connections 2023-10-10T20:40:49.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.738+0000 7f37cc5a7700 1 -- 172.21.15.89:0/2887643804 >> 172.21.15.89:0/2887643804 conn(0x7f37c406a320 msgr2=0x7f37c40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:49.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.738+0000 7f37cc5a7700 1 -- 172.21.15.89:0/2887643804 shutdown_connections 2023-10-10T20:40:49.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.738+0000 7f37cc5a7700 1 -- 172.21.15.89:0/2887643804 wait complete. 2023-10-10T20:40:49.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.739+0000 7f37cc5a7700 1 Processor -- start 2023-10-10T20:40:49.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.739+0000 7f37cc5a7700 1 -- start start 2023-10-10T20:40:49.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.740+0000 7f37cc5a7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37b0004d50 0x7f37c40ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:49.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.740+0000 7f37cc5a7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37c4154af0 0x7f37c414a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:49.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.740+0000 7f37cc5a7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37c4168fa0 0x7f37c4169410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:49.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.740+0000 7f37cc5a7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c41522e0 con 0x7f37c4168fa0 2023-10-10T20:40:49.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.740+0000 7f37cc5a7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c4169950 con 0x7f37b0004d50 2023-10-10T20:40:49.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.740+0000 7f37cc5a7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c4152c40 con 0x7f37c4154af0 2023-10-10T20:40:49.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.740+0000 7f37cab44700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37c4168fa0 0x7f37c4169410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:49.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.740+0000 7f37cab44700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37c4168fa0 0x7f37c4169410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59748/0 (socket says 172.21.15.89:59748) 2023-10-10T20:40:49.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.740+0000 7f37cab44700 1 -- 172.21.15.89:0/1324297605 learned_addr learned my addr 172.21.15.89:0/1324297605 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:49.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.740+0000 7f37ca343700 1 --2- 172.21.15.89:0/1324297605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37b0004d50 0x7f37c40ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:49.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.740+0000 7f37c9b42700 1 --2- 172.21.15.89:0/1324297605 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37c4154af0 0x7f37c414a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:49.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.741+0000 7f37cab44700 1 -- 172.21.15.89:0/1324297605 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37c4154af0 msgr2=0x7f37c414a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:49.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.741+0000 7f37cab44700 1 --2- 172.21.15.89:0/1324297605 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37c4154af0 0x7f37c414a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:49.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.741+0000 7f37cab44700 1 -- 172.21.15.89:0/1324297605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37b0004d50 msgr2=0x7f37c40ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:49.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.741+0000 7f37cab44700 1 --2- 172.21.15.89:0/1324297605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37b0004d50 0x7f37c40ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:49.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.741+0000 7f37cab44700 1 -- 172.21.15.89:0/1324297605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c4151e50 con 0x7f37c4168fa0 2023-10-10T20:40:49.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.741+0000 7f37cab44700 1 --2- 172.21.15.89:0/1324297605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37c4168fa0 0x7f37c4169410 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00c950 tx=0x7f37bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:49.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.741+0000 7f37ba7fc700 1 -- 172.21.15.89:0/1324297605 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc021070 con 0x7f37c4168fa0 2023-10-10T20:40:49.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.741+0000 7f37ba7fc700 1 -- 172.21.15.89:0/1324297605 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37bc004b10 con 0x7f37c4168fa0 2023-10-10T20:40:49.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.741+0000 7f37cc5a7700 1 -- 172.21.15.89:0/1324297605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c414a7a0 con 0x7f37c4168fa0 2023-10-10T20:40:49.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.741+0000 7f37ba7fc700 1 -- 172.21.15.89:0/1324297605 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0070d0 con 0x7f37c4168fa0 2023-10-10T20:40:49.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.741+0000 7f37cc5a7700 1 -- 172.21.15.89:0/1324297605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c41550f0 con 0x7f37c4168fa0 2023-10-10T20:40:49.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.742+0000 7f37cc5a7700 1 -- 172.21.15.89:0/1324297605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c4154ec0 con 0x7f37c4168fa0 2023-10-10T20:40:49.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.742+0000 7f37ba7fc700 1 -- 172.21.15.89:0/1324297605 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f37bc007480 con 0x7f37c4168fa0 2023-10-10T20:40:49.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.743+0000 7f37ba7fc700 1 --2- 172.21.15.89:0/1324297605 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f37a8067290 0x7f37a8069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:49.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.743+0000 7f37ba7fc700 1 -- 172.21.15.89:0/1324297605 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f37bc086ab0 con 0x7f37c4168fa0 2023-10-10T20:40:49.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.743+0000 7f37ca343700 1 --2- 172.21.15.89:0/1324297605 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f37a8067290 0x7f37a8069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:49.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.743+0000 7f37ca343700 1 --2- 172.21.15.89:0/1324297605 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f37a8067290 0x7f37a8069740 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f37c0001750 tx=0x7f37c0001640 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:49.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.747+0000 7f37ba7fc700 1 -- 172.21.15.89:0/1324297605 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37bc053910 con 0x7f37c4168fa0 2023-10-10T20:40:49.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.974+0000 7f37cc5a7700 1 -- 172.21.15.89:0/1324297605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f37c4151fe0 con 0x7f37c4168fa0 2023-10-10T20:40:49.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.975+0000 7f37ba7fc700 1 -- 172.21.15.89:0/1324297605 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) v1 ==== 53+0+73 (secure 0 0 0) 0x7f37bc056f30 con 0x7f37c4168fa0 2023-10-10T20:40:49.977 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:40:49.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.977+0000 7f37cc5a7700 1 -- 172.21.15.89:0/1324297605 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f37a8067290 msgr2=0x7f37a8069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:49.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.977+0000 7f37cc5a7700 1 --2- 172.21.15.89:0/1324297605 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f37a8067290 0x7f37a8069740 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f37c0001750 tx=0x7f37c0001640 comp rx=0 tx=0).stop 2023-10-10T20:40:49.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.977+0000 7f37cc5a7700 1 -- 172.21.15.89:0/1324297605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37c4168fa0 msgr2=0x7f37c4169410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:49.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.977+0000 7f37cc5a7700 1 --2- 172.21.15.89:0/1324297605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37c4168fa0 0x7f37c4169410 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00c950 tx=0x7f37bc017000 comp rx=0 tx=0).stop 2023-10-10T20:40:49.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.977+0000 7f37cc5a7700 1 -- 172.21.15.89:0/1324297605 shutdown_connections 2023-10-10T20:40:49.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.977+0000 7f37cc5a7700 1 --2- 172.21.15.89:0/1324297605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37c4168fa0 0x7f37c4169410 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:49.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.977+0000 7f37cc5a7700 1 --2- 172.21.15.89:0/1324297605 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f37a8067290 0x7f37a8069740 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:49.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.977+0000 7f37cc5a7700 1 --2- 172.21.15.89:0/1324297605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37b0004d50 0x7f37c40ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:49.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.977+0000 7f37cc5a7700 1 --2- 172.21.15.89:0/1324297605 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37c4154af0 0x7f37c414a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:49.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.977+0000 7f37cc5a7700 1 -- 172.21.15.89:0/1324297605 >> 172.21.15.89:0/1324297605 conn(0x7f37c406a320 msgr2=0x7f37c4159f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:49.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.977+0000 7f37cc5a7700 1 -- 172.21.15.89:0/1324297605 shutdown_connections 2023-10-10T20:40:49.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:49.977+0000 7f37cc5a7700 1 -- 172.21.15.89:0/1324297605 wait complete. 2023-10-10T20:40:49.991 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:40:49.991 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:40:49.992 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-10-10T20:40:49.992 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:40:49.992 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:40:49.992 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:40:49.993 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:40:49.993 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:40:49.993 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:40:49.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:49.994 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-10-10T20:40:50.013 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:40:50.013 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:40:50.013 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:40:50.013 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:50.013 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-10-10T20:40:50.014 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:50.014 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:40:50.014 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:50.014 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-10-10T20:40:50.014 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:40:50.014 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:50.015 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-10-10T20:40:50.015 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:40:50.015 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:50.015 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:40:50.015 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:50.015 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:40:50.016 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42132sec preferred_lft 42132sec 2023-10-10T20:40:50.016 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-10-10T20:40:50.016 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:50.016 INFO:teuthology.orchestra.run.smithi111.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:40:50.016 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:40:50.017 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:40:50.017 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:50.017 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::a486:a6ff:fe5f:6021/64 scope link 2023-10-10T20:40:50.017 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:50.018 INFO:teuthology.orchestra.run.smithi111.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:40:50.018 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 1e:d0:81:1b:e2:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:40:50.018 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::1cd0:81ff:fe1b:e26b/64 scope link 2023-10-10T20:40:50.018 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:50.018 INFO:teuthology.orchestra.run.smithi111.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:40:50.019 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:40:50.019 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::88d:cdff:fe75:455e/64 scope link 2023-10-10T20:40:50.019 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:40:50.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:50.020 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:40:50.081 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-10-10T20:40:50.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:50.083 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-10-10T20:40:50.142 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-10-10T20:40:50.142 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:40:50.142 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:40:50.142 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:40:50.143 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:40:50.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:50.143 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:40:50.144 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:40:50.144 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:40:50.145 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:40:50.145 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:40:50.145 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:40:50.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:50.247 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-10-10T20:40:50.263 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:40:50.263 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:40:50.263 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:40:50.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:50.264 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-10-10T20:40:50.328 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:40:50.328 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:40:50.383 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:40:50.383 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:40:50.384 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:40:50.437 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:40:50.437 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:40:50.438 DEBUG:teuthology.orchestra.run.smithi111:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:40:50.516 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:40:50.517 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:40:50.517 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:40:50.524 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:40:50.525 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:40:50.525 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:40:50.525 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:40:50.525 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:40:50.547 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:40:50.547 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:40:50.547 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:40:50.547 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:40:50.548 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:40:50.548 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:40:50.548 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:40:50.548 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:40:50.549 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:40:50.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:50.550 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:40:50.614 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-10T20:40:50.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.735+0000 7f78c8bbb700 1 Processor -- start 2023-10-10T20:40:50.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.735+0000 7f78c8bbb700 1 -- start start 2023-10-10T20:40:50.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.736+0000 7f78c8bbb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f78c40cd910 0x7f78c40ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:50.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.736+0000 7f78c8bbb700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f78c40b55a0 0x7f78c40ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:50.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.736+0000 7f78c8bbb700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c40b5b00 con 0x7f78c4159ec0 2023-10-10T20:40:50.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.736+0000 7f78c8bbb700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c40b5da0 con 0x7f78c40b55a0 2023-10-10T20:40:50.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.736+0000 7f78c8bbb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c40b6080 con 0x7f78c40cd910 2023-10-10T20:40:50.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.736+0000 7f78bbfff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f78c40cd910 0x7f78c40ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:50.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.736+0000 7f78c2d9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f78c40b55a0 0x7f78c40ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:50.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.736+0000 7f78bbfff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f78c40cd910 0x7f78c40ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:37326/0 (socket says 172.21.15.89:37326) 2023-10-10T20:40:50.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.736+0000 7f78bbfff700 1 -- 172.21.15.89:0/23491430 learned_addr learned my addr 172.21.15.89:0/23491430 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:50.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.736+0000 7f78bbfff700 1 -- 172.21.15.89:0/23491430 >> v2:172.21.15.111:3300/0 conn(0x7f78c40b55a0 msgr2=0x7f78c40ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:50.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.736+0000 7f78bbfff700 1 --2- 172.21.15.89:0/23491430 >> v2:172.21.15.111:3300/0 conn(0x7f78c40b55a0 0x7f78c40ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:50.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.736+0000 7f78bbfff700 1 -- 172.21.15.89:0/23491430 >> v1:172.21.15.111:6789/0 conn(0x7f78c4159ec0 legacy=0x7f78c40b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:50.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.736+0000 7f78bbfff700 1 -- 172.21.15.89:0/23491430 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c40cc100 con 0x7f78c40cd910 2023-10-10T20:40:50.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.736+0000 7f78c2d9d700 1 --2- 172.21.15.89:0/23491430 >> v2:172.21.15.111:3300/0 conn(0x7f78c40b55a0 0x7f78c40ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:40:50.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.737+0000 7f78bbfff700 1 --2- 172.21.15.89:0/23491430 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f78c40cd910 0x7f78c40ce340 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0099b0 tx=0x7f78ac009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3808deadbc842513 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:50.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.737+0000 7f78c1d9b700 1 -- 172.21.15.89:0/23491430 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac009ea0 con 0x7f78c40cd910 2023-10-10T20:40:50.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.737+0000 7f78c1d9b700 1 -- 172.21.15.89:0/23491430 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ac00b860 con 0x7f78c40cd910 2023-10-10T20:40:50.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.737+0000 7f78c1d9b700 1 -- 172.21.15.89:0/23491430 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac00bc00 con 0x7f78c40cd910 2023-10-10T20:40:50.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.737+0000 7f78c8bbb700 1 -- 172.21.15.89:0/23491430 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f78c40cd910 msgr2=0x7f78c40ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:50.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.737+0000 7f78c8bbb700 1 --2- 172.21.15.89:0/23491430 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f78c40cd910 0x7f78c40ce340 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0099b0 tx=0x7f78ac009250 comp rx=0 tx=0).stop 2023-10-10T20:40:50.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.737+0000 7f78c8bbb700 1 -- 172.21.15.89:0/23491430 shutdown_connections 2023-10-10T20:40:50.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.737+0000 7f78c8bbb700 1 --2- 172.21.15.89:0/23491430 >> v2:172.21.15.111:3300/0 conn(0x7f78c40b55a0 0x7f78c40ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:50.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.737+0000 7f78c8bbb700 1 --2- 172.21.15.89:0/23491430 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f78c40cd910 0x7f78c40ce340 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:50.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.737+0000 7f78c8bbb700 1 -- 172.21.15.89:0/23491430 >> 172.21.15.89:0/23491430 conn(0x7f78c40999f0 msgr2=0x7f78c40b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:50.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.737+0000 7f78c8bbb700 1 -- 172.21.15.89:0/23491430 shutdown_connections 2023-10-10T20:40:50.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.737+0000 7f78c8bbb700 1 -- 172.21.15.89:0/23491430 wait complete. 2023-10-10T20:40:50.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.738+0000 7f78c8bbb700 1 Processor -- start 2023-10-10T20:40:50.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.738+0000 7f78c8bbb700 1 -- start start 2023-10-10T20:40:50.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.739+0000 7f78c8bbb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78c40b55a0 0x7f78c4157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:50.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.739+0000 7f78c8bbb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78c4159ec0 0x7f78c4157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:50.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.739+0000 7f78c8bbb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f78c4068810 0x7f78c4068c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:50.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.739+0000 7f78c8bbb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c4159420 con 0x7f78c40b55a0 2023-10-10T20:40:50.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.739+0000 7f78c8bbb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c416a510 con 0x7f78c4159ec0 2023-10-10T20:40:50.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.739+0000 7f78c8bbb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c4158810 con 0x7f78c4068810 2023-10-10T20:40:50.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.739+0000 7f78c259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78c40b55a0 0x7f78c4157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:50.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.739+0000 7f78c259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78c40b55a0 0x7f78c4157a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59758/0 (socket says 172.21.15.89:59758) 2023-10-10T20:40:50.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.739+0000 7f78c259c700 1 -- 172.21.15.89:0/2100064743 learned_addr learned my addr 172.21.15.89:0/2100064743 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:50.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.739+0000 7f78bbfff700 1 --2- 172.21.15.89:0/2100064743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78c4159ec0 0x7f78c4157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:50.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.739+0000 7f78c2d9d700 1 --2- 172.21.15.89:0/2100064743 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f78c4068810 0x7f78c4068c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:50.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.739+0000 7f78c259c700 1 -- 172.21.15.89:0/2100064743 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f78c4068810 msgr2=0x7f78c4068c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:50.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.739+0000 7f78c259c700 1 --2- 172.21.15.89:0/2100064743 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f78c4068810 0x7f78c4068c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:50.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.739+0000 7f78c259c700 1 -- 172.21.15.89:0/2100064743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78c4159ec0 msgr2=0x7f78c4157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:50.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.740+0000 7f78c259c700 1 --2- 172.21.15.89:0/2100064743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78c4159ec0 0x7f78c4157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:50.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.740+0000 7f78c259c700 1 -- 172.21.15.89:0/2100064743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78bc009a20 con 0x7f78c40b55a0 2023-10-10T20:40:50.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.740+0000 7f78bbfff700 1 --2- 172.21.15.89:0/2100064743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78c4159ec0 0x7f78c4157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:40:50.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.740+0000 7f78c259c700 1 --2- 172.21.15.89:0/2100064743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78c40b55a0 0x7f78c4157a90 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f78b400c540 tx=0x7f78b4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:50.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.740+0000 7f78ba7fc700 1 -- 172.21.15.89:0/2100064743 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b4011590 con 0x7f78c40b55a0 2023-10-10T20:40:50.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.740+0000 7f78ba7fc700 1 -- 172.21.15.89:0/2100064743 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b40116f0 con 0x7f78c40b55a0 2023-10-10T20:40:50.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.740+0000 7f78c8bbb700 1 -- 172.21.15.89:0/2100064743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78ac0094f0 con 0x7f78c40b55a0 2023-10-10T20:40:50.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.740+0000 7f78ba7fc700 1 -- 172.21.15.89:0/2100064743 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b4011ab0 con 0x7f78c40b55a0 2023-10-10T20:40:50.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.740+0000 7f78c8bbb700 1 -- 172.21.15.89:0/2100064743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c4158dd0 con 0x7f78c40b55a0 2023-10-10T20:40:50.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.741+0000 7f78c8bbb700 1 -- 172.21.15.89:0/2100064743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c4063c90 con 0x7f78c40b55a0 2023-10-10T20:40:50.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.741+0000 7f78ba7fc700 1 -- 172.21.15.89:0/2100064743 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f78b401b3b0 con 0x7f78c40b55a0 2023-10-10T20:40:50.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.741+0000 7f78ba7fc700 1 --2- 172.21.15.89:0/2100064743 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f78a4067290 0x7f78a4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:50.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.741+0000 7f78ba7fc700 1 -- 172.21.15.89:0/2100064743 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f78b4083d30 con 0x7f78c40b55a0 2023-10-10T20:40:50.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.742+0000 7f78bbfff700 1 --2- 172.21.15.89:0/2100064743 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f78a4067290 0x7f78a4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:50.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.742+0000 7f78bbfff700 1 --2- 172.21.15.89:0/2100064743 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f78a4067290 0x7f78a4069740 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0099b0 tx=0x7f78ac017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:50.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.746+0000 7f78ba7fc700 1 -- 172.21.15.89:0/2100064743 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78b4050d00 con 0x7f78c40b55a0 2023-10-10T20:40:50.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:50.974+0000 7f78c8bbb700 1 -- 172.21.15.89:0/2100064743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f78c40718b0 con 0x7f78c40b55a0 2023-10-10T20:40:51.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:51.964+0000 7f78ba7fc700 1 -- 172.21.15.89:0/2100064743 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v65) v1 ==== 136+0+0 (secure 0 0 0) 0x7f78b4017020 con 0x7f78c40b55a0 2023-10-10T20:40:51.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:51.967+0000 7f78c8bbb700 1 -- 172.21.15.89:0/2100064743 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f78a4067290 msgr2=0x7f78a4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:51.967+0000 7f78c8bbb700 1 --2- 172.21.15.89:0/2100064743 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f78a4067290 0x7f78a4069740 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0099b0 tx=0x7f78ac017000 comp rx=0 tx=0).stop 2023-10-10T20:40:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:51.967+0000 7f78c8bbb700 1 -- 172.21.15.89:0/2100064743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78c40b55a0 msgr2=0x7f78c4157a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:51.967+0000 7f78c8bbb700 1 --2- 172.21.15.89:0/2100064743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78c40b55a0 0x7f78c4157a90 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f78b400c540 tx=0x7f78b4001070 comp rx=0 tx=0).stop 2023-10-10T20:40:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:51.967+0000 7f78c8bbb700 1 -- 172.21.15.89:0/2100064743 shutdown_connections 2023-10-10T20:40:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:51.967+0000 7f78c8bbb700 1 --2- 172.21.15.89:0/2100064743 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f78c4068810 0x7f78c4068c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:51.967+0000 7f78c8bbb700 1 --2- 172.21.15.89:0/2100064743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78c40b55a0 0x7f78c4157a90 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:51.967+0000 7f78c8bbb700 1 --2- 172.21.15.89:0/2100064743 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f78a4067290 0x7f78a4069740 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:51.967+0000 7f78c8bbb700 1 --2- 172.21.15.89:0/2100064743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78c4159ec0 0x7f78c4157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:51.967+0000 7f78c8bbb700 1 -- 172.21.15.89:0/2100064743 >> 172.21.15.89:0/2100064743 conn(0x7f78c40999f0 msgr2=0x7f78c40c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:51.967+0000 7f78c8bbb700 1 -- 172.21.15.89:0/2100064743 shutdown_connections 2023-10-10T20:40:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:51.967+0000 7f78c8bbb700 1 -- 172.21.15.89:0/2100064743 wait complete. 2023-10-10T20:40:51.984 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-10T20:40:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.103+0000 7ff74a910700 1 Processor -- start 2023-10-10T20:40:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.104+0000 7ff74a910700 1 -- start start 2023-10-10T20:40:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.104+0000 7ff74a910700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff74406c0a0 0x7ff7440caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.104+0000 7ff74a910700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff74415df50 0x7ff74415e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.104+0000 7ff74a910700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff74415d7e0 con 0x7ff74406bcd0 2023-10-10T20:40:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.104+0000 7ff74a910700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff74415dac0 con 0x7ff74406c0a0 2023-10-10T20:40:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.104+0000 7ff74a910700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74415aa30 con 0x7ff74415df50 2023-10-10T20:40:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.104+0000 7ff743fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff74406bcd0 0x7ff74415a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34092/0 (socket says 172.21.15.89:34092) 2023-10-10T20:40:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff743fff700 1 -- 172.21.15.89:0/2463872442 learned_addr learned my addr 172.21.15.89:0/2463872442 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff7437fe700 1 --2- 172.21.15.89:0/2463872442 >> v2:172.21.15.111:3300/0 conn(0x7ff74406c0a0 0x7ff7440caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff748ead700 1 --2- 172.21.15.89:0/2463872442 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff74415df50 0x7ff74415e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff742ffd700 1 -- 172.21.15.89:0/2463872442 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 728075505 0 0) 0x7ff74415d7e0 con 0x7ff74406bcd0 2023-10-10T20:40:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff742ffd700 1 -- 172.21.15.89:0/2463872442 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff72c003570 con 0x7ff74406bcd0 2023-10-10T20:40:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff742ffd700 1 -- 172.21.15.89:0/2463872442 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1739574072 0 0) 0x7ff72c003570 con 0x7ff74406bcd0 2023-10-10T20:40:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff742ffd700 1 -- 172.21.15.89:0/2463872442 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff74415df50 msgr2=0x7ff74415e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff742ffd700 1 --2- 172.21.15.89:0/2463872442 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff74415df50 0x7ff74415e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff742ffd700 1 -- 172.21.15.89:0/2463872442 >> v2:172.21.15.111:3300/0 conn(0x7ff74406c0a0 msgr2=0x7ff7440caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff742ffd700 1 --2- 172.21.15.89:0/2463872442 >> v2:172.21.15.111:3300/0 conn(0x7ff74406c0a0 0x7ff7440caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff742ffd700 1 -- 172.21.15.89:0/2463872442 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74415ad10 con 0x7ff74406bcd0 2023-10-10T20:40:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff742ffd700 1 -- 172.21.15.89:0/2463872442 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7ff734001470 con 0x7ff74406bcd0 2023-10-10T20:40:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff742ffd700 1 -- 172.21.15.89:0/2463872442 >> v1:172.21.15.89:6789/0 conn(0x7ff74406bcd0 legacy=0x7ff74415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff742ffd700 1 --2- 172.21.15.89:0/2463872442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff72c003a60 0x7ff72c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff742ffd700 1 --2- 172.21.15.89:0/2463872442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c004810 0x7ff72c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff742ffd700 1 --2- 172.21.15.89:0/2463872442 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff74415df50 0x7ff72c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff742ffd700 1 -- 172.21.15.89:0/2463872442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74415dac0 con 0x7ff72c004810 2023-10-10T20:40:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff742ffd700 1 -- 172.21.15.89:0/2463872442 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74415aa30 con 0x7ff72c003a60 2023-10-10T20:40:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff742ffd700 1 -- 172.21.15.89:0/2463872442 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c00bae0 con 0x7ff74415df50 2023-10-10T20:40:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff7437fe700 1 --2- 172.21.15.89:0/2463872442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c004810 0x7ff72c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff748ead700 1 --2- 172.21.15.89:0/2463872442 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff74415df50 0x7ff72c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff743fff700 1 --2- 172.21.15.89:0/2463872442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff72c003a60 0x7ff72c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff7437fe700 1 -- 172.21.15.89:0/2463872442 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff74415df50 msgr2=0x7ff72c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff7437fe700 1 --2- 172.21.15.89:0/2463872442 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff74415df50 0x7ff72c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff7437fe700 1 -- 172.21.15.89:0/2463872442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff72c003a60 msgr2=0x7ff72c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff7437fe700 1 --2- 172.21.15.89:0/2463872442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff72c003a60 0x7ff72c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.105+0000 7ff7437fe700 1 -- 172.21.15.89:0/2463872442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c003570 con 0x7ff72c004810 2023-10-10T20:40:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.106+0000 7ff7437fe700 1 --2- 172.21.15.89:0/2463872442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c004810 0x7ff72c004c80 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7ff738009c00 tx=0x7ff7380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462133d4431fc5d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.106+0000 7ff742ffd700 1 -- 172.21.15.89:0/2463872442 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73800e710 con 0x7ff72c004810 2023-10-10T20:40:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.106+0000 7ff742ffd700 1 -- 172.21.15.89:0/2463872442 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73800e870 con 0x7ff72c004810 2023-10-10T20:40:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.106+0000 7ff742ffd700 1 -- 172.21.15.89:0/2463872442 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73800ec30 con 0x7ff72c004810 2023-10-10T20:40:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.106+0000 7ff74a910700 1 -- 172.21.15.89:0/2463872442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c004810 msgr2=0x7ff72c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.106+0000 7ff74a910700 1 --2- 172.21.15.89:0/2463872442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c004810 0x7ff72c004c80 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7ff738009c00 tx=0x7ff7380091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.106+0000 7ff748ead700 1 -- 172.21.15.89:0/2463872442 reap_dead start 2023-10-10T20:40:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.106+0000 7ff74a910700 1 -- 172.21.15.89:0/2463872442 shutdown_connections 2023-10-10T20:40:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.106+0000 7ff74a910700 1 -- 172.21.15.89:0/2463872442 >> 172.21.15.89:0/2463872442 conn(0x7ff7440ce600 msgr2=0x7ff7440cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.106+0000 7ff74a910700 1 -- 172.21.15.89:0/2463872442 shutdown_connections 2023-10-10T20:40:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.106+0000 7ff74a910700 1 -- 172.21.15.89:0/2463872442 wait complete. 2023-10-10T20:40:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.107+0000 7ff74a910700 1 Processor -- start 2023-10-10T20:40:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.108+0000 7ff74a910700 1 -- start start 2023-10-10T20:40:52.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.108+0000 7ff74a910700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff72c004810 0x7ff7440b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.108+0000 7ff74a910700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7441548d0 0x7ff744152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.108+0000 7ff74a910700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7440c4370 0x7ff7440c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.108+0000 7ff74a910700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74416b010 con 0x7ff7441548d0 2023-10-10T20:40:52.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.108+0000 7ff74a910700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7440c4d00 con 0x7ff7440c4370 2023-10-10T20:40:52.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.108+0000 7ff74a910700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74416a6e0 con 0x7ff72c004810 2023-10-10T20:40:52.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.108+0000 7ff7437fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7441548d0 0x7ff744152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.108+0000 7ff7437fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7441548d0 0x7ff744152510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59780/0 (socket says 172.21.15.89:59780) 2023-10-10T20:40:52.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.108+0000 7ff7437fe700 1 -- 172.21.15.89:0/1779877611 learned_addr learned my addr 172.21.15.89:0/1779877611 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:52.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.108+0000 7ff743fff700 1 --2- 172.21.15.89:0/1779877611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff72c004810 0x7ff7440b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.108+0000 7ff748ead700 1 --2- 172.21.15.89:0/1779877611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7440c4370 0x7ff7440c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.109+0000 7ff7437fe700 1 -- 172.21.15.89:0/1779877611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff72c004810 msgr2=0x7ff7440b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:52.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.109+0000 7ff7437fe700 1 --2- 172.21.15.89:0/1779877611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff72c004810 0x7ff7440b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.109+0000 7ff7437fe700 1 -- 172.21.15.89:0/1779877611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7440c4370 msgr2=0x7ff7440c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:52.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.109+0000 7ff7437fe700 1 --2- 172.21.15.89:0/1779877611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7440c4370 0x7ff7440c47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.109+0000 7ff7437fe700 1 -- 172.21.15.89:0/1779877611 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74416a9c0 con 0x7ff7441548d0 2023-10-10T20:40:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.109+0000 7ff7437fe700 1 --2- 172.21.15.89:0/1779877611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7441548d0 0x7ff744152510 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7ff73800e4b0 tx=0x7ff73801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.109+0000 7ff72bfff700 1 -- 172.21.15.89:0/1779877611 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738022070 con 0x7ff7441548d0 2023-10-10T20:40:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.109+0000 7ff72bfff700 1 -- 172.21.15.89:0/1779877611 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73800e9e0 con 0x7ff7441548d0 2023-10-10T20:40:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.109+0000 7ff74a910700 1 -- 172.21.15.89:0/1779877611 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff74416aca0 con 0x7ff7441548d0 2023-10-10T20:40:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.109+0000 7ff72bfff700 1 -- 172.21.15.89:0/1779877611 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73800e710 con 0x7ff7441548d0 2023-10-10T20:40:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.109+0000 7ff74a910700 1 -- 172.21.15.89:0/1779877611 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74416a380 con 0x7ff7441548d0 2023-10-10T20:40:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.110+0000 7ff74a910700 1 -- 172.21.15.89:0/1779877611 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff74416b180 con 0x7ff7441548d0 2023-10-10T20:40:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.110+0000 7ff72bfff700 1 -- 172.21.15.89:0/1779877611 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7ff73801c660 con 0x7ff7441548d0 2023-10-10T20:40:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.111+0000 7ff72bfff700 1 --2- 172.21.15.89:0/1779877611 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff7300672c0 0x7ff730069770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.111+0000 7ff72bfff700 1 -- 172.21.15.89:0/1779877611 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff738086210 con 0x7ff7441548d0 2023-10-10T20:40:52.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.111+0000 7ff743fff700 1 --2- 172.21.15.89:0/1779877611 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff7300672c0 0x7ff730069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.111+0000 7ff743fff700 1 --2- 172.21.15.89:0/1779877611 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff7300672c0 0x7ff730069770 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff734001620 tx=0x7ff734001e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:52.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.115+0000 7ff72bfff700 1 -- 172.21.15.89:0/1779877611 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff738053070 con 0x7ff7441548d0 2023-10-10T20:40:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.327+0000 7ff74a910700 1 -- 172.21.15.89:0/1779877611 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff74416b010 con 0x7ff7300672c0 2023-10-10T20:40:52.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.328+0000 7ff72bfff700 1 -- 172.21.15.89:0/1779877611 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff74416b010 con 0x7ff7300672c0 2023-10-10T20:40:52.330 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:40:52.331 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:52.331 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs" 2023-10-10T20:40:52.331 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:52.332 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:40:52.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.330+0000 7ff74a910700 1 -- 172.21.15.89:0/1779877611 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff7300672c0 msgr2=0x7ff730069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:52.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.330+0000 7ff74a910700 1 --2- 172.21.15.89:0/1779877611 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff7300672c0 0x7ff730069770 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff734001620 tx=0x7ff734001e90 comp rx=0 tx=0).stop 2023-10-10T20:40:52.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.330+0000 7ff74a910700 1 -- 172.21.15.89:0/1779877611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7441548d0 msgr2=0x7ff744152510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:52.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.330+0000 7ff74a910700 1 --2- 172.21.15.89:0/1779877611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7441548d0 0x7ff744152510 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7ff73800e4b0 tx=0x7ff73801a000 comp rx=0 tx=0).stop 2023-10-10T20:40:52.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.330+0000 7ff74a910700 1 -- 172.21.15.89:0/1779877611 shutdown_connections 2023-10-10T20:40:52.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.330+0000 7ff74a910700 1 --2- 172.21.15.89:0/1779877611 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff7300672c0 0x7ff730069770 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.330+0000 7ff74a910700 1 --2- 172.21.15.89:0/1779877611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7440c4370 0x7ff7440c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.330+0000 7ff74a910700 1 --2- 172.21.15.89:0/1779877611 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff72c004810 0x7ff7440b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.330+0000 7ff74a910700 1 --2- 172.21.15.89:0/1779877611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7441548d0 0x7ff744152510 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.330+0000 7ff74a910700 1 -- 172.21.15.89:0/1779877611 >> 172.21.15.89:0/1779877611 conn(0x7ff7440ce600 msgr2=0x7ff74406d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.330+0000 7ff74a910700 1 -- 172.21.15.89:0/1779877611 shutdown_connections 2023-10-10T20:40:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.330+0000 7ff74a910700 1 -- 172.21.15.89:0/1779877611 wait complete. 2023-10-10T20:40:52.344 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:40:52.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.459+0000 7f72298e2700 1 Processor -- start 2023-10-10T20:40:52.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.459+0000 7f72298e2700 1 -- start start 2023-10-10T20:40:52.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.460+0000 7f72298e2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f72240aca80 0x7f722406aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.460+0000 7f72298e2700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72240c83c0 0x7f72240cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.460+0000 7f72298e2700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f722406bdb0 con 0x7f7224069d60 2023-10-10T20:40:52.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.460+0000 7f72298e2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72240cb370 con 0x7f72240aca80 2023-10-10T20:40:52.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.460+0000 7f72298e2700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7224159f60 con 0x7f72240c83c0 2023-10-10T20:40:52.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.460+0000 7f7222ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f72240aca80 0x7f722406aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.460+0000 7f7222ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f72240aca80 0x7f722406aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59792/0 (socket says 172.21.15.89:59792) 2023-10-10T20:40:52.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.460+0000 7f7222ffd700 1 -- 172.21.15.89:0/392603059 learned_addr learned my addr 172.21.15.89:0/392603059 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:52.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.460+0000 7f72237fe700 1 --2- 172.21.15.89:0/392603059 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72240c83c0 0x7f72240cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.461+0000 7f7222ffd700 1 -- 172.21.15.89:0/392603059 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72240c83c0 msgr2=0x7f72240cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:52.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.461+0000 7f7222ffd700 1 --2- 172.21.15.89:0/392603059 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72240c83c0 0x7f72240cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.461+0000 7f7222ffd700 1 -- 172.21.15.89:0/392603059 >> v1:172.21.15.111:6789/0 conn(0x7f7224069d60 legacy=0x7f72240c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:52.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.461+0000 7f7222ffd700 1 -- 172.21.15.89:0/392603059 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722406b510 con 0x7f72240aca80 2023-10-10T20:40:52.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.461+0000 7f7222ffd700 1 --2- 172.21.15.89:0/392603059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72240aca80 0x7f722406aba0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f7214009f70 tx=0x7f7214009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f1e33dc9367e3d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:52.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.461+0000 7f7221ffb700 1 -- 172.21.15.89:0/392603059 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721400e070 con 0x7f72240aca80 2023-10-10T20:40:52.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.461+0000 7f7221ffb700 1 -- 172.21.15.89:0/392603059 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721400ba20 con 0x7f72240aca80 2023-10-10T20:40:52.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.461+0000 7f7221ffb700 1 -- 172.21.15.89:0/392603059 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72140092e0 con 0x7f72240aca80 2023-10-10T20:40:52.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.461+0000 7f72298e2700 1 -- 172.21.15.89:0/392603059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72240aca80 msgr2=0x7f722406aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.461+0000 7f72298e2700 1 --2- 172.21.15.89:0/392603059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72240aca80 0x7f722406aba0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f7214009f70 tx=0x7f7214009840 comp rx=0 tx=0).stop 2023-10-10T20:40:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.461+0000 7f72298e2700 1 -- 172.21.15.89:0/392603059 shutdown_connections 2023-10-10T20:40:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.461+0000 7f72298e2700 1 --2- 172.21.15.89:0/392603059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72240aca80 0x7f722406aba0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.461+0000 7f72298e2700 1 --2- 172.21.15.89:0/392603059 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72240c83c0 0x7f72240cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.461+0000 7f72298e2700 1 -- 172.21.15.89:0/392603059 >> 172.21.15.89:0/392603059 conn(0x7f72240cd670 msgr2=0x7f72240cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:52.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.462+0000 7f72298e2700 1 -- 172.21.15.89:0/392603059 shutdown_connections 2023-10-10T20:40:52.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.462+0000 7f72298e2700 1 -- 172.21.15.89:0/392603059 wait complete. 2023-10-10T20:40:52.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.463+0000 7f72298e2700 1 Processor -- start 2023-10-10T20:40:52.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.463+0000 7f72298e2700 1 -- start start 2023-10-10T20:40:52.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f72298e2700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7224069d60 0x7f72240ab620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f72298e2700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72240aca80 0x7f722415c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f72298e2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72240c83c0 0x7f722415c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f72298e2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72240c7c20 con 0x7f72240c83c0 2023-10-10T20:40:52.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f72298e2700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722415cc00 con 0x7f7224069d60 2023-10-10T20:40:52.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f72298e2700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722416d780 con 0x7f72240aca80 2023-10-10T20:40:52.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f72237fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72240c83c0 0x7f722415c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f72237fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72240c83c0 0x7f722415c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59794/0 (socket says 172.21.15.89:59794) 2023-10-10T20:40:52.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f72237fe700 1 -- 172.21.15.89:0/289578124 learned_addr learned my addr 172.21.15.89:0/289578124 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:52.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f7222ffd700 1 --2- 172.21.15.89:0/289578124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7224069d60 0x7f72240ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f72227fc700 1 --2- 172.21.15.89:0/289578124 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72240aca80 0x7f722415c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f72237fe700 1 -- 172.21.15.89:0/289578124 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72240aca80 msgr2=0x7f722415c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:52.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f72237fe700 1 --2- 172.21.15.89:0/289578124 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72240aca80 0x7f722415c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f72237fe700 1 -- 172.21.15.89:0/289578124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7224069d60 msgr2=0x7f72240ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f72237fe700 1 --2- 172.21.15.89:0/289578124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7224069d60 0x7f72240ab620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f72237fe700 1 -- 172.21.15.89:0/289578124 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7214009ae0 con 0x7f72240c83c0 2023-10-10T20:40:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f72237fe700 1 --2- 172.21.15.89:0/289578124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72240c83c0 0x7f722415c5d0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f721c00e8d0 tx=0x7f721c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f720affd700 1 -- 172.21.15.89:0/289578124 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c013680 con 0x7f72240c83c0 2023-10-10T20:40:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f720affd700 1 -- 172.21.15.89:0/289578124 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c013800 con 0x7f72240c83c0 2023-10-10T20:40:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f72298e2700 1 -- 172.21.15.89:0/289578124 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f722416da60 con 0x7f72240c83c0 2023-10-10T20:40:52.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f720affd700 1 -- 172.21.15.89:0/289578124 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c013bc0 con 0x7f72240c83c0 2023-10-10T20:40:52.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.464+0000 7f72298e2700 1 -- 172.21.15.89:0/289578124 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72240abb60 con 0x7f72240c83c0 2023-10-10T20:40:52.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.465+0000 7f72298e2700 1 -- 172.21.15.89:0/289578124 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f722406eba0 con 0x7f72240c83c0 2023-10-10T20:40:52.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.466+0000 7f720affd700 1 -- 172.21.15.89:0/289578124 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f721c0123d0 con 0x7f72240c83c0 2023-10-10T20:40:52.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.466+0000 7f720affd700 1 --2- 172.21.15.89:0/289578124 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f72100672c0 0x7f7210069770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.466+0000 7f7222ffd700 1 --2- 172.21.15.89:0/289578124 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f72100672c0 0x7f7210069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.466+0000 7f720affd700 1 -- 172.21.15.89:0/289578124 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f721c084a30 con 0x7f72240c83c0 2023-10-10T20:40:52.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.467+0000 7f7222ffd700 1 --2- 172.21.15.89:0/289578124 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f72100672c0 0x7f7210069770 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f72240afec0 tx=0x7f7214018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:52.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.470+0000 7f720affd700 1 -- 172.21.15.89:0/289578124 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f721c051890 con 0x7f72240c83c0 2023-10-10T20:40:52.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.652+0000 7f72298e2700 1 -- 172.21.15.89:0/289578124 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f722404c3c0 con 0x7f72240c83c0 2023-10-10T20:40:52.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.652+0000 7f720affd700 1 -- 172.21.15.89:0/289578124 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f721c0170c0 con 0x7f72240c83c0 2023-10-10T20:40:52.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.654+0000 7f72298e2700 1 -- 172.21.15.89:0/289578124 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f72100672c0 msgr2=0x7f7210069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:52.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.654+0000 7f72298e2700 1 --2- 172.21.15.89:0/289578124 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f72100672c0 0x7f7210069770 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f72240afec0 tx=0x7f7214018000 comp rx=0 tx=0).stop 2023-10-10T20:40:52.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.654+0000 7f72298e2700 1 -- 172.21.15.89:0/289578124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72240c83c0 msgr2=0x7f722415c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:52.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.654+0000 7f72298e2700 1 --2- 172.21.15.89:0/289578124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72240c83c0 0x7f722415c5d0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f721c00e8d0 tx=0x7f721c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.654+0000 7f72298e2700 1 -- 172.21.15.89:0/289578124 shutdown_connections 2023-10-10T20:40:52.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.654+0000 7f72298e2700 1 --2- 172.21.15.89:0/289578124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72240c83c0 0x7f722415c5d0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.654+0000 7f72298e2700 1 --2- 172.21.15.89:0/289578124 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f72100672c0 0x7f7210069770 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.654+0000 7f72298e2700 1 --2- 172.21.15.89:0/289578124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7224069d60 0x7f72240ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.654+0000 7f72298e2700 1 --2- 172.21.15.89:0/289578124 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72240aca80 0x7f722415c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.654+0000 7f72298e2700 1 -- 172.21.15.89:0/289578124 >> 172.21.15.89:0/289578124 conn(0x7f72240cd670 msgr2=0x7f72240c2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:52.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.654+0000 7f72298e2700 1 -- 172.21.15.89:0/289578124 shutdown_connections 2023-10-10T20:40:52.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.654+0000 7f72298e2700 1 -- 172.21.15.89:0/289578124 wait complete. 2023-10-10T20:40:52.668 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs 2023-10-10T20:40:52.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.783+0000 7fd4c7364700 1 Processor -- start 2023-10-10T20:40:52.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.783+0000 7fd4c7364700 1 -- start start 2023-10-10T20:40:52.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.784+0000 7fd4c7364700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd4c0159890 0x7fd4c015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.784+0000 7fd4c7364700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd4c0069560 0x7fd4c00699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.784+0000 7fd4c7364700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c015a160 con 0x7fd4c0159c60 2023-10-10T20:40:52.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.784+0000 7fd4c7364700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4c0069f20 con 0x7fd4c0069560 2023-10-10T20:40:52.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.784+0000 7fd4c7364700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c00d13a0 con 0x7fd4c0159890 2023-10-10T20:40:52.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.784+0000 7fd4c48ff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd4c0159c60 0x7fd4c015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34106/0 (socket says 172.21.15.89:34106) 2023-10-10T20:40:52.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.784+0000 7fd4c48ff700 1 -- 172.21.15.89:0/61912537 learned_addr learned my addr 172.21.15.89:0/61912537 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:52.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.784+0000 7fd4c5100700 1 --2- 172.21.15.89:0/61912537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd4c0159890 0x7fd4c015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.784+0000 7fd4c5901700 1 --2- 172.21.15.89:0/61912537 >> v2:172.21.15.111:3300/0 conn(0x7fd4c0069560 0x7fd4c00699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.784+0000 7fd4bffff700 1 -- 172.21.15.89:0/61912537 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715266348 0 0) 0x7fd4c015a160 con 0x7fd4c0159c60 2023-10-10T20:40:52.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.784+0000 7fd4bffff700 1 -- 172.21.15.89:0/61912537 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4a8003570 con 0x7fd4c0159c60 2023-10-10T20:40:52.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.785+0000 7fd4c5901700 1 -- 172.21.15.89:0/61912537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd4c0159890 msgr2=0x7fd4c015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:52.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.785+0000 7fd4c5901700 1 --2- 172.21.15.89:0/61912537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd4c0159890 0x7fd4c015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.785+0000 7fd4c5901700 1 -- 172.21.15.89:0/61912537 >> v1:172.21.15.89:6789/0 conn(0x7fd4c0159c60 legacy=0x7fd4c015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:52.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.785+0000 7fd4c5901700 1 -- 172.21.15.89:0/61912537 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c015e140 con 0x7fd4c0069560 2023-10-10T20:40:52.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.785+0000 7fd4c5901700 1 --2- 172.21.15.89:0/61912537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0069560 0x7fd4c00699e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80098e0 tx=0x7fd4b80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4204d6860f2ddf0d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:52.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.785+0000 7fd4bffff700 1 -- 172.21.15.89:0/61912537 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8011630 con 0x7fd4c0069560 2023-10-10T20:40:52.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.785+0000 7fd4bffff700 1 -- 172.21.15.89:0/61912537 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b8011790 con 0x7fd4c0069560 2023-10-10T20:40:52.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.785+0000 7fd4bffff700 1 -- 172.21.15.89:0/61912537 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8011b30 con 0x7fd4c0069560 2023-10-10T20:40:52.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.785+0000 7fd4c7364700 1 -- 172.21.15.89:0/61912537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0069560 msgr2=0x7fd4c00699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:52.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.785+0000 7fd4c7364700 1 --2- 172.21.15.89:0/61912537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0069560 0x7fd4c00699e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80098e0 tx=0x7fd4b80091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.785+0000 7fd4c7364700 1 -- 172.21.15.89:0/61912537 shutdown_connections 2023-10-10T20:40:52.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.785+0000 7fd4c7364700 1 --2- 172.21.15.89:0/61912537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0069560 0x7fd4c00699e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.785+0000 7fd4c7364700 1 --2- 172.21.15.89:0/61912537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd4c0159890 0x7fd4c015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.785+0000 7fd4c7364700 1 -- 172.21.15.89:0/61912537 >> 172.21.15.89:0/61912537 conn(0x7fd4c0068e20 msgr2=0x7fd4c00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:52.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.785+0000 7fd4c7364700 1 -- 172.21.15.89:0/61912537 shutdown_connections 2023-10-10T20:40:52.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.786+0000 7fd4c7364700 1 -- 172.21.15.89:0/61912537 wait complete. 2023-10-10T20:40:52.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.786+0000 7fd4c7364700 1 Processor -- start 2023-10-10T20:40:52.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.787+0000 7fd4c7364700 1 -- start start 2023-10-10T20:40:52.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.787+0000 7fd4c7364700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd4c0069560 0x7fd4c016d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.787+0000 7fd4c7364700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4c0159890 0x7fd4c00b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.787+0000 7fd4c7364700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0159c60 0x7fd4c0155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.787+0000 7fd4c7364700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c016df90 con 0x7fd4c0159890 2023-10-10T20:40:52.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.787+0000 7fd4c7364700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c00b6500 con 0x7fd4c0159c60 2023-10-10T20:40:52.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.787+0000 7fd4c7364700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c00c78a0 con 0x7fd4c0069560 2023-10-10T20:40:52.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.787+0000 7fd4c48ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4c0159890 0x7fd4c00b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.788+0000 7fd4c48ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4c0159890 0x7fd4c00b5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59806/0 (socket says 172.21.15.89:59806) 2023-10-10T20:40:52.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.788+0000 7fd4c48ff700 1 -- 172.21.15.89:0/4153587682 learned_addr learned my addr 172.21.15.89:0/4153587682 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:52.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.788+0000 7fd4c5100700 1 --2- 172.21.15.89:0/4153587682 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd4c0069560 0x7fd4c016d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.788+0000 7fd4c5901700 1 --2- 172.21.15.89:0/4153587682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0159c60 0x7fd4c0155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.788+0000 7fd4c48ff700 1 -- 172.21.15.89:0/4153587682 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd4c0069560 msgr2=0x7fd4c016d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:52.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.788+0000 7fd4c48ff700 1 --2- 172.21.15.89:0/4153587682 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd4c0069560 0x7fd4c016d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.788+0000 7fd4c48ff700 1 -- 172.21.15.89:0/4153587682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0159c60 msgr2=0x7fd4c0155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:52.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.788+0000 7fd4c48ff700 1 --2- 172.21.15.89:0/4153587682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0159c60 0x7fd4c0155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:52.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.788+0000 7fd4c48ff700 1 -- 172.21.15.89:0/4153587682 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b8009420 con 0x7fd4c0159890 2023-10-10T20:40:52.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.788+0000 7fd4c5901700 1 --2- 172.21.15.89:0/4153587682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0159c60 0x7fd4c0155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:52.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.788+0000 7fd4c48ff700 1 --2- 172.21.15.89:0/4153587682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4c0159890 0x7fd4c00b5fc0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400c7c0 tx=0x7fd4b4007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:52.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.788+0000 7fd4bd7fa700 1 -- 172.21.15.89:0/4153587682 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4005b30 con 0x7fd4c0159890 2023-10-10T20:40:52.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.788+0000 7fd4bd7fa700 1 -- 172.21.15.89:0/4153587682 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b4005cb0 con 0x7fd4c0159890 2023-10-10T20:40:52.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.788+0000 7fd4c7364700 1 -- 172.21.15.89:0/4153587682 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c015c300 con 0x7fd4c0159890 2023-10-10T20:40:52.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.788+0000 7fd4bd7fa700 1 -- 172.21.15.89:0/4153587682 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4007320 con 0x7fd4c0159890 2023-10-10T20:40:52.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.788+0000 7fd4c7364700 1 -- 172.21.15.89:0/4153587682 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c016d9d0 con 0x7fd4c0159890 2023-10-10T20:40:52.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.789+0000 7fd4c7364700 1 -- 172.21.15.89:0/4153587682 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4c015c490 con 0x7fd4c0159890 2023-10-10T20:40:52.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.789+0000 7fd4bd7fa700 1 -- 172.21.15.89:0/4153587682 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fd4b401b3b0 con 0x7fd4c0159890 2023-10-10T20:40:52.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.790+0000 7fd4bd7fa700 1 --2- 172.21.15.89:0/4153587682 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd4ac067290 0x7fd4ac069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:52.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.790+0000 7fd4bd7fa700 1 -- 172.21.15.89:0/4153587682 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd4b4083f60 con 0x7fd4c0159890 2023-10-10T20:40:52.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.790+0000 7fd4c5100700 1 --2- 172.21.15.89:0/4153587682 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd4ac067290 0x7fd4ac069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:52.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.790+0000 7fd4c5100700 1 --2- 172.21.15.89:0/4153587682 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd4ac067290 0x7fd4ac069740 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000f570 tx=0x7fd4b000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:52.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:52.794+0000 7fd4bd7fa700 1 -- 172.21.15.89:0/4153587682 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4b4050dc0 con 0x7fd4c0159890 2023-10-10T20:40:53.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.007+0000 7fd4c7364700 1 -- 172.21.15.89:0/4153587682 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4c004c3c0 con 0x7fd4ac067290 2023-10-10T20:40:53.014 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:40:53.012+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:40:53.014 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:40:53.012+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:40:53.014 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:40:53.012+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:40:53.015 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:40:53.012+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:40:53.015 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:40:53.012+0000 7f023b8aa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-10T20:40:53.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.017+0000 7fd4bd7fa700 1 -- 172.21.15.89:0/4153587682 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+472 (secure 0 0 0) 0x7fd4c004c3c0 con 0x7fd4ac067290 2023-10-10T20:40:53.019 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:40:53.019 INFO:teuthology.orchestra.run.smithi089.stdout: "mon_addrs": [ 2023-10-10T20:40:53.019 INFO:teuthology.orchestra.run.smithi089.stdout: "172.21.15.89:6789", 2023-10-10T20:40:53.020 INFO:teuthology.orchestra.run.smithi089.stdout: "172.21.15.111:6789", 2023-10-10T20:40:53.020 INFO:teuthology.orchestra.run.smithi089.stdout: "172.21.15.111:6790" 2023-10-10T20:40:53.020 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-10T20:40:53.020 INFO:teuthology.orchestra.run.smithi089.stdout: "pools": { 2023-10-10T20:40:53.021 INFO:teuthology.orchestra.run.smithi089.stdout: "data": [ 2023-10-10T20:40:53.021 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:53.021 INFO:teuthology.orchestra.run.smithi089.stdout: "avail": 365977796608, 2023-10-10T20:40:53.021 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs_data", 2023-10-10T20:40:53.021 INFO:teuthology.orchestra.run.smithi089.stdout: "used": 0 2023-10-10T20:40:53.022 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:53.022 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-10T20:40:53.022 INFO:teuthology.orchestra.run.smithi089.stdout: "metadata": [ 2023-10-10T20:40:53.022 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:40:53.022 INFO:teuthology.orchestra.run.smithi089.stdout: "avail": 365977796608, 2023-10-10T20:40:53.023 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs_metadata", 2023-10-10T20:40:53.023 INFO:teuthology.orchestra.run.smithi089.stdout: "used": 65536 2023-10-10T20:40:53.023 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:53.023 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-10T20:40:53.023 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:40:53.024 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:40:53.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.019+0000 7fd4c7364700 1 -- 172.21.15.89:0/4153587682 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd4ac067290 msgr2=0x7fd4ac069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:53.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.019+0000 7fd4c7364700 1 --2- 172.21.15.89:0/4153587682 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd4ac067290 0x7fd4ac069740 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000f570 tx=0x7fd4b000b040 comp rx=0 tx=0).stop 2023-10-10T20:40:53.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.019+0000 7fd4c7364700 1 -- 172.21.15.89:0/4153587682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4c0159890 msgr2=0x7fd4c00b5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:53.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.019+0000 7fd4c7364700 1 --2- 172.21.15.89:0/4153587682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4c0159890 0x7fd4c00b5fc0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400c7c0 tx=0x7fd4b4007dd0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.019+0000 7fd4c7364700 1 -- 172.21.15.89:0/4153587682 shutdown_connections 2023-10-10T20:40:53.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.019+0000 7fd4c7364700 1 --2- 172.21.15.89:0/4153587682 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd4ac067290 0x7fd4ac069740 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.019+0000 7fd4c7364700 1 --2- 172.21.15.89:0/4153587682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0159c60 0x7fd4c0155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.019+0000 7fd4c7364700 1 --2- 172.21.15.89:0/4153587682 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd4c0069560 0x7fd4c016d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.019+0000 7fd4c7364700 1 --2- 172.21.15.89:0/4153587682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4c0159890 0x7fd4c00b5fc0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.019+0000 7fd4c7364700 1 -- 172.21.15.89:0/4153587682 >> 172.21.15.89:0/4153587682 conn(0x7fd4c0068e20 msgr2=0x7fd4c00d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:53.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.019+0000 7fd4c7364700 1 -- 172.21.15.89:0/4153587682 shutdown_connections 2023-10-10T20:40:53.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.019+0000 7fd4c7364700 1 -- 172.21.15.89:0/4153587682 wait complete. 2023-10-10T20:40:53.033 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:40:53.033 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:40:53.076 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:40:53.121 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:40:53.121 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:40:53.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:40:53.122 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:40:53.166 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:40:53.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:53.167 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:40:53.221 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:40:53.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:40:53.222 DEBUG:teuthology.orchestra.run.smithi111:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:40:53.262 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:40:53.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:40:53.263 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:40:53.320 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:40:53.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.434+0000 7ff2aeb8d700 1 Processor -- start 2023-10-10T20:40:53.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.434+0000 7ff2aeb8d700 1 -- start start 2023-10-10T20:40:53.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.435+0000 7ff2aeb8d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff2a8143050 0x7ff2a8159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:53.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.435+0000 7ff2aeb8d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a8143420 con 0x7ff2a80b3f20 2023-10-10T20:40:53.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.435+0000 7ff2aeb8d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a806a1e0 con 0x7ff2a806b9e0 2023-10-10T20:40:53.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.435+0000 7ff2aeb8d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a80695d0 con 0x7ff2a8143050 2023-10-10T20:40:53.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.435+0000 7ff2ae38c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff2a80b3f20 0x7ff2a806a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34112/0 (socket says 172.21.15.89:34112) 2023-10-10T20:40:53.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.435+0000 7ff2ae38c700 1 -- 172.21.15.89:0/1972030213 learned_addr learned my addr 172.21.15.89:0/1972030213 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:53.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.435+0000 7ff2adb8b700 1 --2- 172.21.15.89:0/1972030213 >> v2:172.21.15.111:3300/0 conn(0x7ff2a8143050 0x7ff2a8159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:53.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.435+0000 7ff2acb89700 1 -- 172.21.15.89:0/1972030213 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3248042367 0 0) 0x7ff2a8143420 con 0x7ff2a80b3f20 2023-10-10T20:40:53.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.435+0000 7ff2acb89700 1 -- 172.21.15.89:0/1972030213 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff290003570 con 0x7ff2a80b3f20 2023-10-10T20:40:53.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2acb89700 1 -- 172.21.15.89:0/1972030213 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3490775202 0 0) 0x7ff290003570 con 0x7ff2a80b3f20 2023-10-10T20:40:53.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2acb89700 1 -- 172.21.15.89:0/1972030213 >> v2:172.21.15.111:3300/0 conn(0x7ff2a8143050 msgr2=0x7ff2a8159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:53.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2acb89700 1 --2- 172.21.15.89:0/1972030213 >> v2:172.21.15.111:3300/0 conn(0x7ff2a8143050 0x7ff2a8159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2acb89700 1 -- 172.21.15.89:0/1972030213 >> v1:172.21.15.111:6789/0 conn(0x7ff2a806b9e0 legacy=0x7ff2a80b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:53.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2acb89700 1 -- 172.21.15.89:0/1972030213 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a80698b0 con 0x7ff2a80b3f20 2023-10-10T20:40:53.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2acb89700 1 -- 172.21.15.89:0/1972030213 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7ff2a0001470 con 0x7ff2a80b3f20 2023-10-10T20:40:53.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2acb89700 1 -- 172.21.15.89:0/1972030213 >> v1:172.21.15.89:6789/0 conn(0x7ff2a80b3f20 legacy=0x7ff2a806a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:53.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2acb89700 1 --2- 172.21.15.89:0/1972030213 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff290003a60 0x7ff290003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:53.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2acb89700 1 --2- 172.21.15.89:0/1972030213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff290004810 0x7ff290004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:53.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2acb89700 1 --2- 172.21.15.89:0/1972030213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2900052e0 0x7ff29000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:53.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2acb89700 1 -- 172.21.15.89:0/1972030213 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80695d0 con 0x7ff2900052e0 2023-10-10T20:40:53.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2acb89700 1 -- 172.21.15.89:0/1972030213 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29000c2a0 con 0x7ff290004810 2023-10-10T20:40:53.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2acb89700 1 -- 172.21.15.89:0/1972030213 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29000c460 con 0x7ff290003a60 2023-10-10T20:40:53.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2ae38c700 1 --2- 172.21.15.89:0/1972030213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2900052e0 0x7ff29000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:53.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2adb8b700 1 --2- 172.21.15.89:0/1972030213 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff290003a60 0x7ff290003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:53.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2ad38a700 1 --2- 172.21.15.89:0/1972030213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff290004810 0x7ff290004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:53.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2ae38c700 1 -- 172.21.15.89:0/1972030213 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff290003a60 msgr2=0x7ff290003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:53.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2ae38c700 1 --2- 172.21.15.89:0/1972030213 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff290003a60 0x7ff290003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2ae38c700 1 -- 172.21.15.89:0/1972030213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff290004810 msgr2=0x7ff290004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2ae38c700 1 --2- 172.21.15.89:0/1972030213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff290004810 0x7ff290004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2ae38c700 1 -- 172.21.15.89:0/1972030213 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff290003570 con 0x7ff2900052e0 2023-10-10T20:40:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2ae38c700 1 -- 172.21.15.89:0/1972030213 reap_dead start 2023-10-10T20:40:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2ae38c700 1 --2- 172.21.15.89:0/1972030213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2900052e0 0x7ff29000ba60 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000c3c0 tx=0x7ff2a0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ba8e9156e46e0e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2acb89700 1 -- 172.21.15.89:0/1972030213 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a000cc90 con 0x7ff2900052e0 2023-10-10T20:40:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2acb89700 1 -- 172.21.15.89:0/1972030213 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a000cdf0 con 0x7ff2900052e0 2023-10-10T20:40:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.436+0000 7ff2acb89700 1 -- 172.21.15.89:0/1972030213 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0005b50 con 0x7ff2900052e0 2023-10-10T20:40:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.437+0000 7ff2aeb8d700 1 -- 172.21.15.89:0/1972030213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2900052e0 msgr2=0x7ff29000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.437+0000 7ff2aeb8d700 1 --2- 172.21.15.89:0/1972030213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2900052e0 0x7ff29000ba60 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000c3c0 tx=0x7ff2a0001e50 comp rx=0 tx=0).stop 2023-10-10T20:40:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.437+0000 7ff2aeb8d700 1 -- 172.21.15.89:0/1972030213 shutdown_connections 2023-10-10T20:40:53.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.437+0000 7ff2aeb8d700 1 --2- 172.21.15.89:0/1972030213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2900052e0 0x7ff29000ba60 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.437+0000 7ff2aeb8d700 1 -- 172.21.15.89:0/1972030213 >> 172.21.15.89:0/1972030213 conn(0x7ff2a80ce430 msgr2=0x7ff2a80ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:53.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.437+0000 7ff2aeb8d700 1 -- 172.21.15.89:0/1972030213 shutdown_connections 2023-10-10T20:40:53.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.437+0000 7ff2aeb8d700 1 -- 172.21.15.89:0/1972030213 wait complete. 2023-10-10T20:40:53.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.438+0000 7ff2aeb8d700 1 Processor -- start 2023-10-10T20:40:53.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.438+0000 7ff2aeb8d700 1 -- start start 2023-10-10T20:40:53.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.439+0000 7ff2aeb8d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2900052e0 0x7ff2a80cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:53.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.439+0000 7ff2aeb8d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2a80c8890 0x7ff2a815d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.439+0000 7ff2aeb8d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2a80ca0f0 0x7ff2a8169d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.439+0000 7ff2aeb8d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80c9c60 con 0x7ff2a80ca0f0 2023-10-10T20:40:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.439+0000 7ff2aeb8d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80c9050 con 0x7ff2900052e0 2023-10-10T20:40:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.439+0000 7ff2aeb8d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80c9330 con 0x7ff2a80c8890 2023-10-10T20:40:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.439+0000 7ff2ae38c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2a80ca0f0 0x7ff2a8169d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:53.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.439+0000 7ff2ae38c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2a80ca0f0 0x7ff2a8169d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59830/0 (socket says 172.21.15.89:59830) 2023-10-10T20:40:53.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.439+0000 7ff2ae38c700 1 -- 172.21.15.89:0/1442587407 learned_addr learned my addr 172.21.15.89:0/1442587407 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:53.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.439+0000 7ff2adb8b700 1 --2- 172.21.15.89:0/1442587407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2900052e0 0x7ff2a80cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:53.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.439+0000 7ff2ad38a700 1 --2- 172.21.15.89:0/1442587407 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2a80c8890 0x7ff2a815d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:53.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.439+0000 7ff2ae38c700 1 -- 172.21.15.89:0/1442587407 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2a80c8890 msgr2=0x7ff2a815d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:53.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.439+0000 7ff2ae38c700 1 --2- 172.21.15.89:0/1442587407 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2a80c8890 0x7ff2a815d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.439+0000 7ff2ae38c700 1 -- 172.21.15.89:0/1442587407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2900052e0 msgr2=0x7ff2a80cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:53.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.439+0000 7ff2ae38c700 1 --2- 172.21.15.89:0/1442587407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2900052e0 0x7ff2a80cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.439+0000 7ff2ae38c700 1 -- 172.21.15.89:0/1442587407 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a80c9610 con 0x7ff2a80ca0f0 2023-10-10T20:40:53.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.439+0000 7ff2ae38c700 1 --2- 172.21.15.89:0/1442587407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2a80ca0f0 0x7ff2a8169d80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0000c00 tx=0x7ff2a0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:53.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.439+0000 7ff2a5ffb700 1 -- 172.21.15.89:0/1442587407 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a000c9e0 con 0x7ff2a80ca0f0 2023-10-10T20:40:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.440+0000 7ff2a5ffb700 1 -- 172.21.15.89:0/1442587407 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a0000ed0 con 0x7ff2a80ca0f0 2023-10-10T20:40:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.440+0000 7ff2a5ffb700 1 -- 172.21.15.89:0/1442587407 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00074f0 con 0x7ff2a80ca0f0 2023-10-10T20:40:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.440+0000 7ff2aeb8d700 1 -- 172.21.15.89:0/1442587407 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a80c98f0 con 0x7ff2a80ca0f0 2023-10-10T20:40:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.440+0000 7ff2aeb8d700 1 -- 172.21.15.89:0/1442587407 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a815d920 con 0x7ff2a80ca0f0 2023-10-10T20:40:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.441+0000 7ff2aeb8d700 1 -- 172.21.15.89:0/1442587407 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a80c9e10 con 0x7ff2a80ca0f0 2023-10-10T20:40:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.441+0000 7ff2a5ffb700 1 -- 172.21.15.89:0/1442587407 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7ff2a001d420 con 0x7ff2a80ca0f0 2023-10-10T20:40:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.441+0000 7ff2a5ffb700 1 --2- 172.21.15.89:0/1442587407 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff294067290 0x7ff294069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.441+0000 7ff2a5ffb700 1 -- 172.21.15.89:0/1442587407 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff2a0085590 con 0x7ff2a80ca0f0 2023-10-10T20:40:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.442+0000 7ff2adb8b700 1 --2- 172.21.15.89:0/1442587407 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff294067290 0x7ff294069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.442+0000 7ff2adb8b700 1 --2- 172.21.15.89:0/1442587407 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff294067290 0x7ff294069740 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff2980094d0 tx=0x7ff29800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:53.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.446+0000 7ff2a5ffb700 1 -- 172.21.15.89:0/1442587407 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2a80c9e10 con 0x7ff2a80ca0f0 2023-10-10T20:40:53.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.672+0000 7ff2aeb8d700 1 -- 172.21.15.89:0/1442587407 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2a804c3c0 con 0x7ff2a80ca0f0 2023-10-10T20:40:53.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.673+0000 7ff2a5ffb700 1 -- 172.21.15.89:0/1442587407 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4602 (secure 0 0 0) 0x7ff2a00523f0 con 0x7ff2a80ca0f0 2023-10-10T20:40:53.675 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:53.676 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1635581143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1635581143},{"type":"v1","addr":"172.21.15.89:6837","nonce":1635581143}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5732,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/321638734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":321638734},{"type":"v1","addr":"172.21.15.111:6835","nonce":321638734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1070384555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1070384555},{"type":"v1","addr":"172.21.15.111:6834","nonce":1070384555}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:40:37.630844+0000","modified":"2023-10-10T20:40:41.269543+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6822},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6822":{"gid":6822,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6836/2274240619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2274240619},{"type":"v1","addr":"172.21.15.89:6836","nonce":2274240619}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-10T20:40:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.675+0000 7ff2aeb8d700 1 -- 172.21.15.89:0/1442587407 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff294067290 msgr2=0x7ff294069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.675+0000 7ff2aeb8d700 1 --2- 172.21.15.89:0/1442587407 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff294067290 0x7ff294069740 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff2980094d0 tx=0x7ff29800b040 comp rx=0 tx=0).stop 2023-10-10T20:40:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.675+0000 7ff2aeb8d700 1 -- 172.21.15.89:0/1442587407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2a80ca0f0 msgr2=0x7ff2a8169d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.676+0000 7ff2aeb8d700 1 --2- 172.21.15.89:0/1442587407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2a80ca0f0 0x7ff2a8169d80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0000c00 tx=0x7ff2a0013000 comp rx=0 tx=0).stop 2023-10-10T20:40:53.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.676+0000 7ff2aeb8d700 1 -- 172.21.15.89:0/1442587407 shutdown_connections 2023-10-10T20:40:53.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.676+0000 7ff2aeb8d700 1 --2- 172.21.15.89:0/1442587407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2a80ca0f0 0x7ff2a8169d80 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.676+0000 7ff2aeb8d700 1 --2- 172.21.15.89:0/1442587407 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff294067290 0x7ff294069740 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.676+0000 7ff2aeb8d700 1 --2- 172.21.15.89:0/1442587407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2900052e0 0x7ff2a80cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.676+0000 7ff2aeb8d700 1 --2- 172.21.15.89:0/1442587407 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff2a80c8890 0x7ff2a815d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.676+0000 7ff2aeb8d700 1 -- 172.21.15.89:0/1442587407 >> 172.21.15.89:0/1442587407 conn(0x7ff2a80ce430 msgr2=0x7ff2a81419b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:53.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.676+0000 7ff2aeb8d700 1 -- 172.21.15.89:0/1442587407 shutdown_connections 2023-10-10T20:40:53.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.676+0000 7ff2aeb8d700 1 -- 172.21.15.89:0/1442587407 wait complete. 2023-10-10T20:40:53.680 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 65 2023-10-10T20:40:53.690 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:40:53.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.803+0000 7f159ea53700 1 Processor -- start 2023-10-10T20:40:53.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.803+0000 7f159ea53700 1 -- start start 2023-10-10T20:40:53.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.804+0000 7f159ea53700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1598159890 0x7f159815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:53.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.804+0000 7f159ea53700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1598069560 0x7f15980699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:53.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.804+0000 7f159ea53700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f159815a160 con 0x7f1598159c60 2023-10-10T20:40:53.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.804+0000 7f159ea53700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1598069f20 con 0x7f1598069560 2023-10-10T20:40:53.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.804+0000 7f159ea53700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15980d13a0 con 0x7f1598159890 2023-10-10T20:40:53.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.804+0000 7f159cff0700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1598069560 0x7f15980699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:53.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.804+0000 7f1597fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1598159890 0x7f159815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:53.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.804+0000 7f15977fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1598159c60 0x7f159815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:59910/0 (socket says 172.21.15.89:59910) 2023-10-10T20:40:53.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.804+0000 7f159cff0700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1598069560 0x7f15980699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:46732/0 (socket says 172.21.15.89:46732) 2023-10-10T20:40:53.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.804+0000 7f1597fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1598159890 0x7f159815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:37430/0 (socket says 172.21.15.89:37430) 2023-10-10T20:40:53.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.804+0000 7f15977fe700 1 -- 172.21.15.89:0/2281877783 learned_addr learned my addr 172.21.15.89:0/2281877783 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:53.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.805+0000 7f159cff0700 1 -- 172.21.15.89:0/2281877783 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1598159890 msgr2=0x7f159815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:53.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.805+0000 7f159cff0700 1 --2- 172.21.15.89:0/2281877783 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1598159890 0x7f159815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.805+0000 7f159cff0700 1 -- 172.21.15.89:0/2281877783 >> v1:172.21.15.111:6789/0 conn(0x7f1598159c60 legacy=0x7f159815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:53.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.805+0000 7f1596ffd700 1 -- 172.21.15.89:0/2281877783 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970927996 0 0) 0x7f159815a160 con 0x7f1598159c60 2023-10-10T20:40:53.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.805+0000 7f159cff0700 1 -- 172.21.15.89:0/2281877783 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159815e140 con 0x7f1598069560 2023-10-10T20:40:53.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.805+0000 7f159cff0700 1 --2- 172.21.15.89:0/2281877783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1598069560 0x7f15980699e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1590009e60 tx=0x7f1590009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90ff4acc0d14633e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:53.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.805+0000 7f1596ffd700 1 -- 172.21.15.89:0/2281877783 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159000e070 con 0x7f1598069560 2023-10-10T20:40:53.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.805+0000 7f1596ffd700 1 -- 172.21.15.89:0/2281877783 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159000bb60 con 0x7f1598069560 2023-10-10T20:40:53.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.805+0000 7f1596ffd700 1 -- 172.21.15.89:0/2281877783 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590012430 con 0x7f1598069560 2023-10-10T20:40:53.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.805+0000 7f159ea53700 1 -- 172.21.15.89:0/2281877783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1598069560 msgr2=0x7f15980699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:53.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.805+0000 7f159ea53700 1 --2- 172.21.15.89:0/2281877783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1598069560 0x7f15980699e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1590009e60 tx=0x7f1590009780 comp rx=0 tx=0).stop 2023-10-10T20:40:53.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.806+0000 7f159ea53700 1 -- 172.21.15.89:0/2281877783 shutdown_connections 2023-10-10T20:40:53.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.806+0000 7f159ea53700 1 --2- 172.21.15.89:0/2281877783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1598069560 0x7f15980699e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.806+0000 7f159ea53700 1 --2- 172.21.15.89:0/2281877783 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1598159890 0x7f159815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.806+0000 7f159ea53700 1 -- 172.21.15.89:0/2281877783 >> 172.21.15.89:0/2281877783 conn(0x7f1598068e20 msgr2=0x7f15980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:53.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.806+0000 7f159ea53700 1 -- 172.21.15.89:0/2281877783 shutdown_connections 2023-10-10T20:40:53.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.806+0000 7f159ea53700 1 -- 172.21.15.89:0/2281877783 wait complete. 2023-10-10T20:40:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.807+0000 7f159ea53700 1 Processor -- start 2023-10-10T20:40:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.807+0000 7f159ea53700 1 -- start start 2023-10-10T20:40:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f159ea53700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1598069560 0x7f159815d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f159ea53700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1598159890 0x7f15980b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f159ea53700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1598159c60 0x7f1598154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f159ea53700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1598155770 con 0x7f1598159890 2023-10-10T20:40:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f159ea53700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159815efc0 con 0x7f1598069560 2023-10-10T20:40:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f159ea53700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1598154cc0 con 0x7f1598159c60 2023-10-10T20:40:53.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f15977fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1598159890 0x7f15980b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:53.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f15977fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1598159890 0x7f15980b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59846/0 (socket says 172.21.15.89:59846) 2023-10-10T20:40:53.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f15977fe700 1 -- 172.21.15.89:0/261444474 learned_addr learned my addr 172.21.15.89:0/261444474 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:53.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f159cff0700 1 --2- 172.21.15.89:0/261444474 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1598159c60 0x7f1598154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:53.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f1597fff700 1 --2- 172.21.15.89:0/261444474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1598069560 0x7f159815d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:53.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f15977fe700 1 -- 172.21.15.89:0/261444474 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1598159c60 msgr2=0x7f1598154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:53.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f15977fe700 1 --2- 172.21.15.89:0/261444474 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1598159c60 0x7f1598154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f15977fe700 1 -- 172.21.15.89:0/261444474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1598069560 msgr2=0x7f159815d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:53.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f15977fe700 1 --2- 172.21.15.89:0/261444474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1598069560 0x7f159815d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:53.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f15977fe700 1 -- 172.21.15.89:0/261444474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1590009a20 con 0x7f1598159890 2023-10-10T20:40:53.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f15977fe700 1 --2- 172.21.15.89:0/261444474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1598159890 0x7f15980b3590 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c840 tx=0x7f158c007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:53.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f157ffff700 1 -- 172.21.15.89:0/261444474 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c005c00 con 0x7f1598159890 2023-10-10T20:40:53.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f157ffff700 1 -- 172.21.15.89:0/261444474 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158c005d60 con 0x7f1598159890 2023-10-10T20:40:53.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f159ea53700 1 -- 172.21.15.89:0/261444474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15980b3fe0 con 0x7f1598159890 2023-10-10T20:40:53.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f157ffff700 1 -- 172.21.15.89:0/261444474 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c007300 con 0x7f1598159890 2023-10-10T20:40:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.808+0000 7f159ea53700 1 -- 172.21.15.89:0/261444474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159815b8a0 con 0x7f1598159890 2023-10-10T20:40:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.809+0000 7f159ea53700 1 -- 172.21.15.89:0/261444474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15980b4500 con 0x7f1598159890 2023-10-10T20:40:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.810+0000 7f157ffff700 1 -- 172.21.15.89:0/261444474 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f158c01b3b0 con 0x7f1598159890 2023-10-10T20:40:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.810+0000 7f157ffff700 1 --2- 172.21.15.89:0/261444474 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1584067290 0x7f1584069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.810+0000 7f1597fff700 1 --2- 172.21.15.89:0/261444474 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1584067290 0x7f1584069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.810+0000 7f157ffff700 1 -- 172.21.15.89:0/261444474 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f158c083ec0 con 0x7f1598159890 2023-10-10T20:40:53.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.811+0000 7f1597fff700 1 --2- 172.21.15.89:0/261444474 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1584067290 0x7f1584069740 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1588009db0 tx=0x7f15880091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:53.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:53.814+0000 7f157ffff700 1 -- 172.21.15.89:0/261444474 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15980b4500 con 0x7f1598159890 2023-10-10T20:40:54.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.000+0000 7f159ea53700 1 -- 172.21.15.89:0/261444474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f159815ed90 con 0x7f1598159890 2023-10-10T20:40:54.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.000+0000 7f157ffff700 1 -- 172.21.15.89:0/261444474 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+15355 (secure 0 0 0) 0x7f159815ed90 con 0x7f1598159890 2023-10-10T20:40:54.003 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:54.003 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":100,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:40:39.247030+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:40:36.002449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-10T20:40:36.591960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:40:54.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.002+0000 7f159ea53700 1 -- 172.21.15.89:0/261444474 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1584067290 msgr2=0x7f1584069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.002+0000 7f159ea53700 1 --2- 172.21.15.89:0/261444474 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1584067290 0x7f1584069740 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1588009db0 tx=0x7f15880091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.002+0000 7f159ea53700 1 -- 172.21.15.89:0/261444474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1598159890 msgr2=0x7f15980b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.002+0000 7f159ea53700 1 --2- 172.21.15.89:0/261444474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1598159890 0x7f15980b3590 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c840 tx=0x7f158c007db0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.002+0000 7f159ea53700 1 -- 172.21.15.89:0/261444474 shutdown_connections 2023-10-10T20:40:54.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.002+0000 7f159ea53700 1 --2- 172.21.15.89:0/261444474 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1598159c60 0x7f1598154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.002+0000 7f159ea53700 1 --2- 172.21.15.89:0/261444474 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1584067290 0x7f1584069740 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.002+0000 7f159ea53700 1 --2- 172.21.15.89:0/261444474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1598069560 0x7f159815d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.002+0000 7f159ea53700 1 --2- 172.21.15.89:0/261444474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1598159890 0x7f15980b3590 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.002+0000 7f159ea53700 1 -- 172.21.15.89:0/261444474 >> 172.21.15.89:0/261444474 conn(0x7f1598068e20 msgr2=0x7f15981580a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:54.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.002+0000 7f159ea53700 1 -- 172.21.15.89:0/261444474 shutdown_connections 2023-10-10T20:40:54.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.002+0000 7f159ea53700 1 -- 172.21.15.89:0/261444474 wait complete. 2023-10-10T20:40:54.017 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:40:36.002449+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:40:36.591960+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:40:54.017 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:40:54.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.132+0000 7f1d206a7700 1 Processor -- start 2023-10-10T20:40:54.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.132+0000 7f1d206a7700 1 -- start start 2023-10-10T20:40:54.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.133+0000 7f1d206a7700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1d180b5ea0 0x7f1d18068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.133+0000 7f1d206a7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d18068bb0 0x7f1d1815c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.133+0000 7f1d206a7700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d1815c480 con 0x7f1d18159c10 2023-10-10T20:40:54.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.133+0000 7f1d206a7700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d180b2a40 con 0x7f1d180b5ea0 2023-10-10T20:40:54.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.133+0000 7f1d206a7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1815b870 con 0x7f1d18068bb0 2023-10-10T20:40:54.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.133+0000 7f1d1ec44700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d18068bb0 0x7f1d1815c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.133+0000 7f1d1dc42700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1d180b5ea0 0x7f1d18068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.133+0000 7f1d1dc42700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1d180b5ea0 0x7f1d18068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:46748/0 (socket says 172.21.15.89:46748) 2023-10-10T20:40:54.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.133+0000 7f1d1ec44700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d18068bb0 0x7f1d1815c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:37458/0 (socket says 172.21.15.89:37458) 2023-10-10T20:40:54.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.133+0000 7f1d1dc42700 1 -- 172.21.15.89:0/2074001380 learned_addr learned my addr 172.21.15.89:0/2074001380 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:54.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.133+0000 7f1d1e443700 1 --1- 172.21.15.89:0/2074001380 >> v1:172.21.15.111:6789/0 conn(0x7f1d18159c10 0x7f1d1815b090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:59914/0 (socket says 172.21.15.89:59914) 2023-10-10T20:40:54.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.134+0000 7f1d1dc42700 1 -- 172.21.15.89:0/2074001380 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d18068bb0 msgr2=0x7f1d1815c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:54.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.134+0000 7f1d1dc42700 1 --2- 172.21.15.89:0/2074001380 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d18068bb0 0x7f1d1815c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.134+0000 7f1d1dc42700 1 -- 172.21.15.89:0/2074001380 >> v1:172.21.15.111:6789/0 conn(0x7f1d18159c10 legacy=0x7f1d1815b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.134+0000 7f1d1dc42700 1 -- 172.21.15.89:0/2074001380 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1815bb50 con 0x7f1d180b5ea0 2023-10-10T20:40:54.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.134+0000 7f1d1dc42700 1 --2- 172.21.15.89:0/2074001380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d180b5ea0 0x7f1d18068670 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1d080099b0 tx=0x7f1d08009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d35b19b6b684910 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:54.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.134+0000 7f1d1d441700 1 -- 172.21.15.89:0/2074001380 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d08009ea0 con 0x7f1d180b5ea0 2023-10-10T20:40:54.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.134+0000 7f1d1d441700 1 -- 172.21.15.89:0/2074001380 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0800b860 con 0x7f1d180b5ea0 2023-10-10T20:40:54.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.134+0000 7f1d1d441700 1 -- 172.21.15.89:0/2074001380 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0800bc00 con 0x7f1d180b5ea0 2023-10-10T20:40:54.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.135+0000 7f1d206a7700 1 -- 172.21.15.89:0/2074001380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d180b5ea0 msgr2=0x7f1d18068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.135+0000 7f1d206a7700 1 --2- 172.21.15.89:0/2074001380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d180b5ea0 0x7f1d18068670 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1d080099b0 tx=0x7f1d08009250 comp rx=0 tx=0).stop 2023-10-10T20:40:54.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.135+0000 7f1d206a7700 1 -- 172.21.15.89:0/2074001380 shutdown_connections 2023-10-10T20:40:54.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.135+0000 7f1d206a7700 1 --2- 172.21.15.89:0/2074001380 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d18068bb0 0x7f1d1815c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.135+0000 7f1d206a7700 1 --2- 172.21.15.89:0/2074001380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d180b5ea0 0x7f1d18068670 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.135+0000 7f1d206a7700 1 -- 172.21.15.89:0/2074001380 >> 172.21.15.89:0/2074001380 conn(0x7f1d18099a20 msgr2=0x7f1d180af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:54.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.135+0000 7f1d206a7700 1 -- 172.21.15.89:0/2074001380 shutdown_connections 2023-10-10T20:40:54.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.135+0000 7f1d206a7700 1 -- 172.21.15.89:0/2074001380 wait complete. 2023-10-10T20:40:54.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.136+0000 7f1d206a7700 1 Processor -- start 2023-10-10T20:40:54.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.136+0000 7f1d206a7700 1 -- start start 2023-10-10T20:40:54.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d206a7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d18068bb0 0x7f1d18143150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d206a7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d180b5ea0 0x7f1d18069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d206a7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d18159c10 0x7f1d1815f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d206a7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1815d2a0 con 0x7f1d18159c10 2023-10-10T20:40:54.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d206a7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1815c9b0 con 0x7f1d180b5ea0 2023-10-10T20:40:54.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d206a7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1815cc50 con 0x7f1d18068bb0 2023-10-10T20:40:54.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d1ec44700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d18159c10 0x7f1d1815f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d1e443700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d18068bb0 0x7f1d18143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d1ec44700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d18159c10 0x7f1d1815f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59852/0 (socket says 172.21.15.89:59852) 2023-10-10T20:40:54.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d1ec44700 1 -- 172.21.15.89:0/3031796630 learned_addr learned my addr 172.21.15.89:0/3031796630 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:54.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d1dc42700 1 --2- 172.21.15.89:0/3031796630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d180b5ea0 0x7f1d18069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d1ec44700 1 -- 172.21.15.89:0/3031796630 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d18068bb0 msgr2=0x7f1d18143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d1ec44700 1 --2- 172.21.15.89:0/3031796630 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d18068bb0 0x7f1d18143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d1ec44700 1 -- 172.21.15.89:0/3031796630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d180b5ea0 msgr2=0x7f1d18069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d1ec44700 1 --2- 172.21.15.89:0/3031796630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d180b5ea0 0x7f1d18069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d1ec44700 1 -- 172.21.15.89:0/3031796630 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d080094f0 con 0x7f1d18159c10 2023-10-10T20:40:54.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d1ec44700 1 --2- 172.21.15.89:0/3031796630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d18159c10 0x7f1d1815f0d0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f1d10009e90 tx=0x7f1d1000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:54.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d0e7fc700 1 -- 172.21.15.89:0/3031796630 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1000cb70 con 0x7f1d18159c10 2023-10-10T20:40:54.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d0e7fc700 1 -- 172.21.15.89:0/3031796630 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1000f040 con 0x7f1d18159c10 2023-10-10T20:40:54.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d0e7fc700 1 -- 172.21.15.89:0/3031796630 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d10014480 con 0x7f1d18159c10 2023-10-10T20:40:54.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d206a7700 1 -- 172.21.15.89:0/3031796630 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1815cf30 con 0x7f1d18159c10 2023-10-10T20:40:54.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.137+0000 7f1d206a7700 1 -- 172.21.15.89:0/3031796630 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1806a570 con 0x7f1d18159c10 2023-10-10T20:40:54.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.138+0000 7f1d206a7700 1 -- 172.21.15.89:0/3031796630 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d1815d410 con 0x7f1d18159c10 2023-10-10T20:40:54.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.139+0000 7f1d0e7fc700 1 -- 172.21.15.89:0/3031796630 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f1d10014830 con 0x7f1d18159c10 2023-10-10T20:40:54.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.139+0000 7f1d0e7fc700 1 --2- 172.21.15.89:0/3031796630 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1cfc067290 0x7f1cfc069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.139+0000 7f1d0e7fc700 1 -- 172.21.15.89:0/3031796630 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1d10085550 con 0x7f1d18159c10 2023-10-10T20:40:54.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.139+0000 7f1d1e443700 1 --2- 172.21.15.89:0/3031796630 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1cfc067290 0x7f1cfc069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.140+0000 7f1d1e443700 1 --2- 172.21.15.89:0/3031796630 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1cfc067290 0x7f1cfc069740 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1d14004f40 tx=0x7f1d14007eb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:54.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.143+0000 7f1d0e7fc700 1 -- 172.21.15.89:0/3031796630 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d100522a0 con 0x7f1d18159c10 2023-10-10T20:40:54.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.329+0000 7f1d206a7700 1 -- 172.21.15.89:0/3031796630 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d1804c3c0 con 0x7f1d18159c10 2023-10-10T20:40:54.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.330+0000 7f1d0e7fc700 1 -- 172.21.15.89:0/3031796630 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d100558c0 con 0x7f1d18159c10 2023-10-10T20:40:54.332 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:40:54.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.332+0000 7f1d206a7700 1 -- 172.21.15.89:0/3031796630 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1cfc067290 msgr2=0x7f1cfc069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.332+0000 7f1d206a7700 1 --2- 172.21.15.89:0/3031796630 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1cfc067290 0x7f1cfc069740 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1d14004f40 tx=0x7f1d14007eb0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.332+0000 7f1d206a7700 1 -- 172.21.15.89:0/3031796630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d18159c10 msgr2=0x7f1d1815f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.332+0000 7f1d206a7700 1 --2- 172.21.15.89:0/3031796630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d18159c10 0x7f1d1815f0d0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f1d10009e90 tx=0x7f1d1000b040 comp rx=0 tx=0).stop 2023-10-10T20:40:54.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.332+0000 7f1d206a7700 1 -- 172.21.15.89:0/3031796630 shutdown_connections 2023-10-10T20:40:54.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.332+0000 7f1d206a7700 1 --2- 172.21.15.89:0/3031796630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d18159c10 0x7f1d1815f0d0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.332+0000 7f1d206a7700 1 --2- 172.21.15.89:0/3031796630 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d18068bb0 0x7f1d18143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.332+0000 7f1d206a7700 1 --2- 172.21.15.89:0/3031796630 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1cfc067290 0x7f1cfc069740 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.332+0000 7f1d206a7700 1 --2- 172.21.15.89:0/3031796630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d180b5ea0 0x7f1d18069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.332+0000 7f1d206a7700 1 -- 172.21.15.89:0/3031796630 >> 172.21.15.89:0/3031796630 conn(0x7f1d18099a20 msgr2=0x7f1d180b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:54.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.332+0000 7f1d206a7700 1 -- 172.21.15.89:0/3031796630 shutdown_connections 2023-10-10T20:40:54.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.332+0000 7f1d206a7700 1 -- 172.21.15.89:0/3031796630 wait complete. 2023-10-10T20:40:54.346 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:40:54.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.462+0000 7f1d83027700 1 Processor -- start 2023-10-10T20:40:54.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.462+0000 7f1d83027700 1 -- start start 2023-10-10T20:40:54.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.463+0000 7f1d83027700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d7c157680 0x7f1d7c0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.463+0000 7f1d83027700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1d7c0b1450 0x7f1d7c0c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.463+0000 7f1d83027700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d7c0cd240 con 0x7f1d7c157a50 2023-10-10T20:40:54.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.463+0000 7f1d83027700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7c0cc700 con 0x7f1d7c0b1450 2023-10-10T20:40:54.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.463+0000 7f1d83027700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0c9a20 con 0x7f1d7c157680 2023-10-10T20:40:54.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.463+0000 7f1d81824700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1d7c157a50 0x7f1d7c0c8cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34122/0 (socket says 172.21.15.89:34122) 2023-10-10T20:40:54.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.463+0000 7f1d81824700 1 -- 172.21.15.89:0/1506241616 learned_addr learned my addr 172.21.15.89:0/1506241616 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:54.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.463+0000 7f1d82826700 1 --2- 172.21.15.89:0/1506241616 >> v2:172.21.15.111:3300/0 conn(0x7f1d7c0b1450 0x7f1d7c0c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.463+0000 7f1d82025700 1 --2- 172.21.15.89:0/1506241616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d7c157680 0x7f1d7c0cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.463+0000 7f1d81023700 1 -- 172.21.15.89:0/1506241616 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3855328126 0 0) 0x7f1d7c0cd240 con 0x7f1d7c157a50 2023-10-10T20:40:54.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.463+0000 7f1d81023700 1 -- 172.21.15.89:0/1506241616 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d68003570 con 0x7f1d7c157a50 2023-10-10T20:40:54.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.463+0000 7f1d81023700 1 -- 172.21.15.89:0/1506241616 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4189889906 0 0) 0x7f1d68003570 con 0x7f1d7c157a50 2023-10-10T20:40:54.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.463+0000 7f1d81023700 1 -- 172.21.15.89:0/1506241616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d7c157680 msgr2=0x7f1d7c0cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:54.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.463+0000 7f1d81023700 1 --2- 172.21.15.89:0/1506241616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d7c157680 0x7f1d7c0cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.463+0000 7f1d81023700 1 -- 172.21.15.89:0/1506241616 >> v2:172.21.15.111:3300/0 conn(0x7f1d7c0b1450 msgr2=0x7f1d7c0c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:54.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.463+0000 7f1d81023700 1 --2- 172.21.15.89:0/1506241616 >> v2:172.21.15.111:3300/0 conn(0x7f1d7c0b1450 0x7f1d7c0c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.463+0000 7f1d81023700 1 -- 172.21.15.89:0/1506241616 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c0cc910 con 0x7f1d7c157a50 2023-10-10T20:40:54.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.463+0000 7f1d81023700 1 -- 172.21.15.89:0/1506241616 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f1d6c001470 con 0x7f1d7c157a50 2023-10-10T20:40:54.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.464+0000 7f1d81023700 1 -- 172.21.15.89:0/1506241616 >> v1:172.21.15.89:6789/0 conn(0x7f1d7c157a50 legacy=0x7f1d7c0c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.464+0000 7f1d81023700 1 --2- 172.21.15.89:0/1506241616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d68003a60 0x7f1d68003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.464+0000 7f1d81023700 1 --2- 172.21.15.89:0/1506241616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d68004810 0x7f1d68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.464+0000 7f1d81023700 1 --2- 172.21.15.89:0/1506241616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d7c157680 0x7f1d680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.464+0000 7f1d81023700 1 -- 172.21.15.89:0/1506241616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0cc700 con 0x7f1d68004810 2023-10-10T20:40:54.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.464+0000 7f1d81023700 1 -- 172.21.15.89:0/1506241616 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0c9a20 con 0x7f1d68003a60 2023-10-10T20:40:54.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.464+0000 7f1d81023700 1 -- 172.21.15.89:0/1506241616 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6800ba60 con 0x7f1d7c157680 2023-10-10T20:40:54.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.464+0000 7f1d81824700 1 --2- 172.21.15.89:0/1506241616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d68004810 0x7f1d68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.464+0000 7f1d82826700 1 --2- 172.21.15.89:0/1506241616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d7c157680 0x7f1d680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.464+0000 7f1d82025700 1 --2- 172.21.15.89:0/1506241616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d68003a60 0x7f1d68003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.464+0000 7f1d81824700 1 -- 172.21.15.89:0/1506241616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d7c157680 msgr2=0x7f1d680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:54.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.464+0000 7f1d81824700 1 --2- 172.21.15.89:0/1506241616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d7c157680 0x7f1d680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.464+0000 7f1d81824700 1 -- 172.21.15.89:0/1506241616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d68003a60 msgr2=0x7f1d68003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:54.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.464+0000 7f1d81824700 1 --2- 172.21.15.89:0/1506241616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d68003a60 0x7f1d68003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.464+0000 7f1d81824700 1 -- 172.21.15.89:0/1506241616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d68003570 con 0x7f1d68004810 2023-10-10T20:40:54.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.464+0000 7f1d81824700 1 --2- 172.21.15.89:0/1506241616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d68004810 0x7f1d68004c80 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00c3c0 tx=0x7f1d6c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa05a3190819abd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:54.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.464+0000 7f1d81023700 1 -- 172.21.15.89:0/1506241616 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c00cc50 con 0x7f1d68004810 2023-10-10T20:40:54.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.464+0000 7f1d81023700 1 -- 172.21.15.89:0/1506241616 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c00cdd0 con 0x7f1d68004810 2023-10-10T20:40:54.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.464+0000 7f1d81023700 1 -- 172.21.15.89:0/1506241616 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c005b70 con 0x7f1d68004810 2023-10-10T20:40:54.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.465+0000 7f1d83027700 1 -- 172.21.15.89:0/1506241616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d68004810 msgr2=0x7f1d68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.465+0000 7f1d83027700 1 --2- 172.21.15.89:0/1506241616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d68004810 0x7f1d68004c80 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00c3c0 tx=0x7f1d6c001e90 comp rx=0 tx=0).stop 2023-10-10T20:40:54.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.465+0000 7f1d82826700 1 -- 172.21.15.89:0/1506241616 reap_dead start 2023-10-10T20:40:54.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.465+0000 7f1d83027700 1 -- 172.21.15.89:0/1506241616 shutdown_connections 2023-10-10T20:40:54.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.465+0000 7f1d83027700 1 -- 172.21.15.89:0/1506241616 >> 172.21.15.89:0/1506241616 conn(0x7f1d7c06baa0 msgr2=0x7f1d7c06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:54.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.465+0000 7f1d83027700 1 -- 172.21.15.89:0/1506241616 shutdown_connections 2023-10-10T20:40:54.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.465+0000 7f1d83027700 1 -- 172.21.15.89:0/1506241616 wait complete. 2023-10-10T20:40:54.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.466+0000 7f1d83027700 1 Processor -- start 2023-10-10T20:40:54.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.466+0000 7f1d83027700 1 -- start start 2023-10-10T20:40:54.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d83027700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d68004810 0x7f1d7c145120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d83027700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d7c0c34e0 0x7f1d7c0c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d83027700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d7c159740 0x7f1d7c159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d83027700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c151810 con 0x7f1d7c159740 2023-10-10T20:40:54.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d83027700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0c3e90 con 0x7f1d7c0c34e0 2023-10-10T20:40:54.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d83027700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c15a0f0 con 0x7f1d68004810 2023-10-10T20:40:54.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d82826700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d7c159740 0x7f1d7c159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d82826700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d7c159740 0x7f1d7c159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59882/0 (socket says 172.21.15.89:59882) 2023-10-10T20:40:54.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d82826700 1 -- 172.21.15.89:0/487838893 learned_addr learned my addr 172.21.15.89:0/487838893 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:54.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d81824700 1 --2- 172.21.15.89:0/487838893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d7c0c34e0 0x7f1d7c0c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d82025700 1 --2- 172.21.15.89:0/487838893 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d68004810 0x7f1d7c145120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d82826700 1 -- 172.21.15.89:0/487838893 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d68004810 msgr2=0x7f1d7c145120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d82826700 1 --2- 172.21.15.89:0/487838893 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d68004810 0x7f1d7c145120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d82826700 1 -- 172.21.15.89:0/487838893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d7c0c34e0 msgr2=0x7f1d7c0c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d82826700 1 --2- 172.21.15.89:0/487838893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d7c0c34e0 0x7f1d7c0c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d82826700 1 -- 172.21.15.89:0/487838893 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c0c4cd0 con 0x7f1d7c159740 2023-10-10T20:40:54.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d82826700 1 --2- 172.21.15.89:0/487838893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d7c159740 0x7f1d7c159bb0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0b4670 tx=0x7f1d780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:54.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d727fc700 1 -- 172.21.15.89:0/487838893 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78013750 con 0x7f1d7c159740 2023-10-10T20:40:54.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d727fc700 1 -- 172.21.15.89:0/487838893 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d780138b0 con 0x7f1d7c159740 2023-10-10T20:40:54.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d83027700 1 -- 172.21.15.89:0/487838893 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7c145660 con 0x7f1d7c159740 2023-10-10T20:40:54.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d727fc700 1 -- 172.21.15.89:0/487838893 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78013c50 con 0x7f1d7c159740 2023-10-10T20:40:54.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.467+0000 7f1d83027700 1 -- 172.21.15.89:0/487838893 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7c0c43a0 con 0x7f1d7c159740 2023-10-10T20:40:54.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.468+0000 7f1d83027700 1 -- 172.21.15.89:0/487838893 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7c0c4e60 con 0x7f1d7c159740 2023-10-10T20:40:54.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.469+0000 7f1d727fc700 1 -- 172.21.15.89:0/487838893 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f1d78014650 con 0x7f1d7c159740 2023-10-10T20:40:54.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.470+0000 7f1d727fc700 1 --2- 172.21.15.89:0/487838893 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1d60067290 0x7f1d60069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.470+0000 7f1d727fc700 1 -- 172.21.15.89:0/487838893 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1d78085340 con 0x7f1d7c159740 2023-10-10T20:40:54.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.470+0000 7f1d82025700 1 --2- 172.21.15.89:0/487838893 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1d60067290 0x7f1d60069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.470+0000 7f1d82025700 1 --2- 172.21.15.89:0/487838893 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1d60067290 0x7f1d60069740 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1d740094d0 tx=0x7f1d7400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:54.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.473+0000 7f1d727fc700 1 -- 172.21.15.89:0/487838893 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d78052320 con 0x7f1d7c159740 2023-10-10T20:40:54.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.656+0000 7f1d83027700 1 -- 172.21.15.89:0/487838893 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d7c04bb80 con 0x7f1d7c159740 2023-10-10T20:40:54.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.656+0000 7f1d727fc700 1 -- 172.21.15.89:0/487838893 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1d78000e60 con 0x7f1d7c159740 2023-10-10T20:40:54.659 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:40:54.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.658+0000 7f1d83027700 1 -- 172.21.15.89:0/487838893 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1d60067290 msgr2=0x7f1d60069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.658+0000 7f1d83027700 1 --2- 172.21.15.89:0/487838893 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1d60067290 0x7f1d60069740 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1d740094d0 tx=0x7f1d7400b040 comp rx=0 tx=0).stop 2023-10-10T20:40:54.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.658+0000 7f1d83027700 1 -- 172.21.15.89:0/487838893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d7c159740 msgr2=0x7f1d7c159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.658+0000 7f1d83027700 1 --2- 172.21.15.89:0/487838893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d7c159740 0x7f1d7c159bb0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0b4670 tx=0x7f1d780091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.658+0000 7f1d83027700 1 -- 172.21.15.89:0/487838893 shutdown_connections 2023-10-10T20:40:54.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.658+0000 7f1d83027700 1 --2- 172.21.15.89:0/487838893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d7c159740 0x7f1d7c159bb0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.658+0000 7f1d83027700 1 --2- 172.21.15.89:0/487838893 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1d68004810 0x7f1d7c145120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.658+0000 7f1d83027700 1 --2- 172.21.15.89:0/487838893 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1d60067290 0x7f1d60069740 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.658+0000 7f1d83027700 1 --2- 172.21.15.89:0/487838893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d7c0c34e0 0x7f1d7c0c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.658+0000 7f1d83027700 1 -- 172.21.15.89:0/487838893 >> 172.21.15.89:0/487838893 conn(0x7f1d7c06baa0 msgr2=0x7f1d7c153150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:54.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.658+0000 7f1d83027700 1 -- 172.21.15.89:0/487838893 shutdown_connections 2023-10-10T20:40:54.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.658+0000 7f1d83027700 1 -- 172.21.15.89:0/487838893 wait complete. 2023-10-10T20:40:54.672 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:40:54.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.787+0000 7f1a4c91d700 1 Processor -- start 2023-10-10T20:40:54.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.787+0000 7f1a4c91d700 1 -- start start 2023-10-10T20:40:54.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a4c91d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1a480b1b40 0x7f1a480b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a4c91d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a480cfb00 con 0x7f1a480b16d0 2023-10-10T20:40:54.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a4c91d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a480b5c40 con 0x7f1a480b4e70 2023-10-10T20:40:54.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a4c91d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a480b5f20 con 0x7f1a480b1b40 2023-10-10T20:40:54.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a3effd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1a480b16d0 0x7f1a480b33b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34130/0 (socket says 172.21.15.89:34130) 2023-10-10T20:40:54.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a3effd700 1 -- 172.21.15.89:0/1104272397 learned_addr learned my addr 172.21.15.89:0/1104272397 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:54.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a47fff700 1 --2- 172.21.15.89:0/1104272397 >> v2:172.21.15.111:3300/0 conn(0x7f1a480b1b40 0x7f1a480b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a46ffd700 1 -- 172.21.15.89:0/1104272397 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431795622 0 0) 0x7f1a480cfb00 con 0x7f1a480b16d0 2023-10-10T20:40:54.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a46ffd700 1 -- 172.21.15.89:0/1104272397 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a34003570 con 0x7f1a480b16d0 2023-10-10T20:40:54.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a46ffd700 1 -- 172.21.15.89:0/1104272397 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3145221583 0 0) 0x7f1a34003570 con 0x7f1a480b16d0 2023-10-10T20:40:54.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a46ffd700 1 -- 172.21.15.89:0/1104272397 >> v2:172.21.15.111:3300/0 conn(0x7f1a480b1b40 msgr2=0x7f1a480b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:54.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a46ffd700 1 --2- 172.21.15.89:0/1104272397 >> v2:172.21.15.111:3300/0 conn(0x7f1a480b1b40 0x7f1a480b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a46ffd700 1 -- 172.21.15.89:0/1104272397 >> v1:172.21.15.111:6789/0 conn(0x7f1a480b4e70 legacy=0x7f1a480ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a46ffd700 1 -- 172.21.15.89:0/1104272397 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a480b6200 con 0x7f1a480b16d0 2023-10-10T20:40:54.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a46ffd700 1 -- 172.21.15.89:0/1104272397 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f1a30001470 con 0x7f1a480b16d0 2023-10-10T20:40:54.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a46ffd700 1 -- 172.21.15.89:0/1104272397 >> v1:172.21.15.89:6789/0 conn(0x7f1a480b16d0 legacy=0x7f1a480b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a46ffd700 1 --2- 172.21.15.89:0/1104272397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a34003a60 0x7f1a34003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a46ffd700 1 --2- 172.21.15.89:0/1104272397 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a34004810 0x7f1a34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a46ffd700 1 --2- 172.21.15.89:0/1104272397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a340052e0 0x7f1a3400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a46ffd700 1 -- 172.21.15.89:0/1104272397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a480b5f20 con 0x7f1a340052e0 2023-10-10T20:40:54.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a46ffd700 1 -- 172.21.15.89:0/1104272397 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3400c2a0 con 0x7f1a34003a60 2023-10-10T20:40:54.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a46ffd700 1 -- 172.21.15.89:0/1104272397 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3400c460 con 0x7f1a34004810 2023-10-10T20:40:54.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.788+0000 7f1a47fff700 1 --2- 172.21.15.89:0/1104272397 >> v2:172.21.15.111:3300/0 conn(0x7f1a480b1b40 0x7f1a480b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:40:54.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.789+0000 7f1a47fff700 1 --2- 172.21.15.89:0/1104272397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a340052e0 0x7f1a3400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.789+0000 7f1a477fe700 1 --2- 172.21.15.89:0/1104272397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a34003a60 0x7f1a34003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.789+0000 7f1a3effd700 1 --2- 172.21.15.89:0/1104272397 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a34004810 0x7f1a34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.789+0000 7f1a47fff700 1 -- 172.21.15.89:0/1104272397 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a34004810 msgr2=0x7f1a34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:54.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.789+0000 7f1a47fff700 1 --2- 172.21.15.89:0/1104272397 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a34004810 0x7f1a34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.789+0000 7f1a47fff700 1 -- 172.21.15.89:0/1104272397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a34003a60 msgr2=0x7f1a34003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:54.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.789+0000 7f1a47fff700 1 --2- 172.21.15.89:0/1104272397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a34003a60 0x7f1a34003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.789+0000 7f1a47fff700 1 -- 172.21.15.89:0/1104272397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a34003570 con 0x7f1a340052e0 2023-10-10T20:40:54.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.789+0000 7f1a47fff700 1 -- 172.21.15.89:0/1104272397 reap_dead start 2023-10-10T20:40:54.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.789+0000 7f1a47fff700 1 --2- 172.21.15.89:0/1104272397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a340052e0 0x7f1a3400ba60 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000e3c0 tx=0x7f1a4000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc6b508801513b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:54.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.789+0000 7f1a46ffd700 1 -- 172.21.15.89:0/1104272397 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a40009c50 con 0x7f1a340052e0 2023-10-10T20:40:54.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.789+0000 7f1a46ffd700 1 -- 172.21.15.89:0/1104272397 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4000f040 con 0x7f1a340052e0 2023-10-10T20:40:54.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.789+0000 7f1a46ffd700 1 -- 172.21.15.89:0/1104272397 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4000eea0 con 0x7f1a340052e0 2023-10-10T20:40:54.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.790+0000 7f1a4c91d700 1 -- 172.21.15.89:0/1104272397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a340052e0 msgr2=0x7f1a3400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.790+0000 7f1a4c91d700 1 --2- 172.21.15.89:0/1104272397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a340052e0 0x7f1a3400ba60 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000e3c0 tx=0x7f1a4000b040 comp rx=0 tx=0).stop 2023-10-10T20:40:54.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.790+0000 7f1a4c91d700 1 -- 172.21.15.89:0/1104272397 shutdown_connections 2023-10-10T20:40:54.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.790+0000 7f1a4c91d700 1 --2- 172.21.15.89:0/1104272397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a340052e0 0x7f1a3400ba60 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.790+0000 7f1a4c91d700 1 -- 172.21.15.89:0/1104272397 >> 172.21.15.89:0/1104272397 conn(0x7f1a480d10c0 msgr2=0x7f1a480affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:54.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.790+0000 7f1a4c91d700 1 -- 172.21.15.89:0/1104272397 shutdown_connections 2023-10-10T20:40:54.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.790+0000 7f1a4c91d700 1 -- 172.21.15.89:0/1104272397 wait complete. 2023-10-10T20:40:54.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.791+0000 7f1a4c91d700 1 Processor -- start 2023-10-10T20:40:54.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.791+0000 7f1a4c91d700 1 -- start start 2023-10-10T20:40:54.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a4c91d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a340052e0 0x7f1a48157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a4c91d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a48169c10 0x7f1a480c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a4c91d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a4816ae20 0x7f1a480b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a4c91d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a480a72b0 con 0x7f1a4816ae20 2023-10-10T20:40:54.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a4c91d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4816a080 con 0x7f1a340052e0 2023-10-10T20:40:54.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a4c91d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4816a2a0 con 0x7f1a48169c10 2023-10-10T20:40:54.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a47fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a4816ae20 0x7f1a480b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a47fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a4816ae20 0x7f1a480b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59888/0 (socket says 172.21.15.89:59888) 2023-10-10T20:40:54.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a47fff700 1 -- 172.21.15.89:0/2275153005 learned_addr learned my addr 172.21.15.89:0/2275153005 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:54.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a3effd700 1 --2- 172.21.15.89:0/2275153005 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a48169c10 0x7f1a480c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a477fe700 1 --2- 172.21.15.89:0/2275153005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a340052e0 0x7f1a48157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a47fff700 1 -- 172.21.15.89:0/2275153005 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a48169c10 msgr2=0x7f1a480c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a47fff700 1 --2- 172.21.15.89:0/2275153005 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a48169c10 0x7f1a480c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a47fff700 1 -- 172.21.15.89:0/2275153005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a340052e0 msgr2=0x7f1a48157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a47fff700 1 --2- 172.21.15.89:0/2275153005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a340052e0 0x7f1a48157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a47fff700 1 -- 172.21.15.89:0/2275153005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4816a580 con 0x7f1a4816ae20 2023-10-10T20:40:54.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a477fe700 1 --2- 172.21.15.89:0/2275153005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a340052e0 0x7f1a48157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:54.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a47fff700 1 --2- 172.21.15.89:0/2275153005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a4816ae20 0x7f1a480b3390 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000afd0 tx=0x7f1a40019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:54.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a3ffff700 1 -- 172.21.15.89:0/2275153005 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4001e720 con 0x7f1a4816ae20 2023-10-10T20:40:54.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a3ffff700 1 -- 172.21.15.89:0/2275153005 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4001e8a0 con 0x7f1a4816ae20 2023-10-10T20:40:54.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a4c91d700 1 -- 172.21.15.89:0/2275153005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a4816a860 con 0x7f1a4816ae20 2023-10-10T20:40:54.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a3ffff700 1 -- 172.21.15.89:0/2275153005 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4001ec60 con 0x7f1a4816ae20 2023-10-10T20:40:54.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.792+0000 7f1a4c91d700 1 -- 172.21.15.89:0/2275153005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a480c9f10 con 0x7f1a4816ae20 2023-10-10T20:40:54.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.793+0000 7f1a4c91d700 1 -- 172.21.15.89:0/2275153005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a480c9d60 con 0x7f1a4816ae20 2023-10-10T20:40:54.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.794+0000 7f1a3ffff700 1 -- 172.21.15.89:0/2275153005 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f1a4001d440 con 0x7f1a4816ae20 2023-10-10T20:40:54.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.794+0000 7f1a3ffff700 1 --2- 172.21.15.89:0/2275153005 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1a28067290 0x7f1a28069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:54.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.794+0000 7f1a477fe700 1 --2- 172.21.15.89:0/2275153005 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1a28067290 0x7f1a28069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:54.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.794+0000 7f1a3ffff700 1 -- 172.21.15.89:0/2275153005 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1a400875e0 con 0x7f1a4816ae20 2023-10-10T20:40:54.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.795+0000 7f1a477fe700 1 --2- 172.21.15.89:0/2275153005 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1a28067290 0x7f1a28069740 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1a480b4a70 tx=0x7f1a38009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:54.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.798+0000 7f1a3ffff700 1 -- 172.21.15.89:0/2275153005 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a40054230 con 0x7f1a4816ae20 2023-10-10T20:40:54.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.982+0000 7f1a3ffff700 1 -- 172.21.15.89:0/2275153005 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1a40054600 con 0x7f1a4816ae20 2023-10-10T20:40:54.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.983+0000 7f1a4c91d700 1 -- 172.21.15.89:0/2275153005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1a480ca100 con 0x7f1a4816ae20 2023-10-10T20:40:54.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.983+0000 7f1a3ffff700 1 -- 172.21.15.89:0/2275153005 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1a40057cc0 con 0x7f1a4816ae20 2023-10-10T20:40:54.986 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-10T20:40:54.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.985+0000 7f1a4c91d700 1 -- 172.21.15.89:0/2275153005 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1a28067290 msgr2=0x7f1a28069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.985+0000 7f1a4c91d700 1 --2- 172.21.15.89:0/2275153005 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1a28067290 0x7f1a28069740 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1a480b4a70 tx=0x7f1a38009040 comp rx=0 tx=0).stop 2023-10-10T20:40:54.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.985+0000 7f1a4c91d700 1 -- 172.21.15.89:0/2275153005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a4816ae20 msgr2=0x7f1a480b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:54.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.985+0000 7f1a4c91d700 1 --2- 172.21.15.89:0/2275153005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a4816ae20 0x7f1a480b3390 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000afd0 tx=0x7f1a40019000 comp rx=0 tx=0).stop 2023-10-10T20:40:54.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.985+0000 7f1a4c91d700 1 -- 172.21.15.89:0/2275153005 shutdown_connections 2023-10-10T20:40:54.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.985+0000 7f1a4c91d700 1 --2- 172.21.15.89:0/2275153005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a4816ae20 0x7f1a480b3390 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.985+0000 7f1a4c91d700 1 --2- 172.21.15.89:0/2275153005 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1a28067290 0x7f1a28069740 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.985+0000 7f1a4c91d700 1 --2- 172.21.15.89:0/2275153005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a340052e0 0x7f1a48157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.985+0000 7f1a4c91d700 1 --2- 172.21.15.89:0/2275153005 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a48169c10 0x7f1a480c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:54.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.985+0000 7f1a4c91d700 1 -- 172.21.15.89:0/2275153005 >> 172.21.15.89:0/2275153005 conn(0x7f1a480d10c0 msgr2=0x7f1a480ccdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:54.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.985+0000 7f1a4c91d700 1 -- 172.21.15.89:0/2275153005 shutdown_connections 2023-10-10T20:40:54.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:54.986+0000 7f1a4c91d700 1 -- 172.21.15.89:0/2275153005 wait complete. 2023-10-10T20:40:54.999 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:40:55.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.111+0000 7f7890b8c700 1 Processor -- start 2023-10-10T20:40:55.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.111+0000 7f7890b8c700 1 -- start start 2023-10-10T20:40:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.112+0000 7f7890b8c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f788c06c0f0 0x7f788c1642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.112+0000 7f7890b8c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f788c0b40f0 con 0x7f788c142ea0 2023-10-10T20:40:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.112+0000 7f7890b8c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f788c069640 con 0x7f788c0ad870 2023-10-10T20:40:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.112+0000 7f7890b8c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788c069920 con 0x7f788c06c0f0 2023-10-10T20:40:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.112+0000 7f788bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f788c06c0f0 0x7f788c1642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.112+0000 7f788b7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f788c142ea0 0x7f788c0b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34144/0 (socket says 172.21.15.89:34144) 2023-10-10T20:40:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.112+0000 7f788b7fe700 1 -- 172.21.15.89:0/3173563030 learned_addr learned my addr 172.21.15.89:0/3173563030 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.112+0000 7f788a7fc700 1 -- 172.21.15.89:0/3173563030 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044609638 0 0) 0x7f788c0b40f0 con 0x7f788c142ea0 2023-10-10T20:40:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.112+0000 7f788a7fc700 1 -- 172.21.15.89:0/3173563030 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7874003570 con 0x7f788c142ea0 2023-10-10T20:40:55.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.112+0000 7f788bfff700 1 -- 172.21.15.89:0/3173563030 >> v1:172.21.15.111:6789/0 conn(0x7f788c0ad870 legacy=0x7f788c06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:55.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.112+0000 7f788bfff700 1 -- 172.21.15.89:0/3173563030 >> v1:172.21.15.89:6789/0 conn(0x7f788c142ea0 legacy=0x7f788c0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:55.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.112+0000 7f788bfff700 1 -- 172.21.15.89:0/3173563030 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c069c00 con 0x7f788c06c0f0 2023-10-10T20:40:55.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.113+0000 7f788bfff700 1 --2- 172.21.15.89:0/3173563030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f788c06c0f0 0x7f788c1642e0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f78840098e0 tx=0x7f78840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74d46cd47950d3d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:55.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.113+0000 7f788a7fc700 1 -- 172.21.15.89:0/3173563030 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884011630 con 0x7f788c06c0f0 2023-10-10T20:40:55.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.113+0000 7f788a7fc700 1 -- 172.21.15.89:0/3173563030 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7884011790 con 0x7f788c06c0f0 2023-10-10T20:40:55.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.113+0000 7f788a7fc700 1 -- 172.21.15.89:0/3173563030 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884011b30 con 0x7f788c06c0f0 2023-10-10T20:40:55.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.113+0000 7f7890b8c700 1 -- 172.21.15.89:0/3173563030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f788c06c0f0 msgr2=0x7f788c1642e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:55.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.113+0000 7f7890b8c700 1 --2- 172.21.15.89:0/3173563030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f788c06c0f0 0x7f788c1642e0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f78840098e0 tx=0x7f78840091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.113+0000 7f7890b8c700 1 -- 172.21.15.89:0/3173563030 shutdown_connections 2023-10-10T20:40:55.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.113+0000 7f7890b8c700 1 --2- 172.21.15.89:0/3173563030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f788c06c0f0 0x7f788c1642e0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.113+0000 7f7890b8c700 1 -- 172.21.15.89:0/3173563030 >> 172.21.15.89:0/3173563030 conn(0x7f788c0ce430 msgr2=0x7f788c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:55.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.113+0000 7f7890b8c700 1 -- 172.21.15.89:0/3173563030 shutdown_connections 2023-10-10T20:40:55.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.113+0000 7f7890b8c700 1 -- 172.21.15.89:0/3173563030 wait complete. 2023-10-10T20:40:55.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.114+0000 7f7890b8c700 1 Processor -- start 2023-10-10T20:40:55.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.114+0000 7f7890b8c700 1 -- start start 2023-10-10T20:40:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.115+0000 7f7890b8c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f788c06c0f0 0x7f788c169e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.115+0000 7f7890b8c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f788c142ea0 0x7f788c159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.115+0000 7f7890b8c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f788c0aca10 0x7f788c0c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.115+0000 7f7890b8c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c15b100 con 0x7f788c142ea0 2023-10-10T20:40:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.115+0000 7f7890b8c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c15a250 con 0x7f788c06c0f0 2023-10-10T20:40:55.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.115+0000 7f7890b8c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788c15a4f0 con 0x7f788c0aca10 2023-10-10T20:40:55.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.115+0000 7f788affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f788c142ea0 0x7f788c159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:55.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.115+0000 7f788affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f788c142ea0 0x7f788c159d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59908/0 (socket says 172.21.15.89:59908) 2023-10-10T20:40:55.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.115+0000 7f788affd700 1 -- 172.21.15.89:0/2695058800 learned_addr learned my addr 172.21.15.89:0/2695058800 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:55.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.115+0000 7f788affd700 1 -- 172.21.15.89:0/2695058800 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f788c0aca10 msgr2=0x7f788c0c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:55.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.115+0000 7f788affd700 1 --2- 172.21.15.89:0/2695058800 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f788c0aca10 0x7f788c0c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.115+0000 7f788affd700 1 -- 172.21.15.89:0/2695058800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f788c06c0f0 msgr2=0x7f788c169e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:55.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.115+0000 7f788b7fe700 1 --2- 172.21.15.89:0/2695058800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f788c06c0f0 0x7f788c169e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:55.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.116+0000 7f788affd700 1 --2- 172.21.15.89:0/2695058800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f788c06c0f0 0x7f788c169e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.116+0000 7f788affd700 1 -- 172.21.15.89:0/2695058800 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7884009420 con 0x7f788c142ea0 2023-10-10T20:40:55.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.116+0000 7f788b7fe700 1 --2- 172.21.15.89:0/2695058800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f788c06c0f0 0x7f788c169e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:55.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.116+0000 7f788affd700 1 --2- 172.21.15.89:0/2695058800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f788c142ea0 0x7f788c159d10 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f788000c590 tx=0x7f7880007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:55.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.116+0000 7f78737fe700 1 -- 172.21.15.89:0/2695058800 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880011590 con 0x7f788c142ea0 2023-10-10T20:40:55.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.116+0000 7f78737fe700 1 -- 172.21.15.89:0/2695058800 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7880011730 con 0x7f788c142ea0 2023-10-10T20:40:55.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.116+0000 7f7890b8c700 1 -- 172.21.15.89:0/2695058800 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788c15a7d0 con 0x7f788c142ea0 2023-10-10T20:40:55.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.116+0000 7f78737fe700 1 -- 172.21.15.89:0/2695058800 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880011af0 con 0x7f788c142ea0 2023-10-10T20:40:55.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.116+0000 7f7890b8c700 1 -- 172.21.15.89:0/2695058800 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788c15ad90 con 0x7f788c142ea0 2023-10-10T20:40:55.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.117+0000 7f7890b8c700 1 -- 172.21.15.89:0/2695058800 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788c04c3c0 con 0x7f788c142ea0 2023-10-10T20:40:55.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.117+0000 7f78737fe700 1 -- 172.21.15.89:0/2695058800 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f788001c460 con 0x7f788c142ea0 2023-10-10T20:40:55.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.117+0000 7f78737fe700 1 --2- 172.21.15.89:0/2695058800 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7878067320 0x7f78780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:55.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.118+0000 7f78737fe700 1 -- 172.21.15.89:0/2695058800 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7880084a20 con 0x7f788c142ea0 2023-10-10T20:40:55.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.118+0000 7f788b7fe700 1 --2- 172.21.15.89:0/2695058800 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7878067320 0x7f78780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:55.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.118+0000 7f788b7fe700 1 --2- 172.21.15.89:0/2695058800 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7878067320 0x7f78780697d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f787c005c50 tx=0x7f787c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:55.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.122+0000 7f78737fe700 1 -- 172.21.15.89:0/2695058800 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78800516b0 con 0x7f788c142ea0 2023-10-10T20:40:55.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.347+0000 7f7890b8c700 1 -- 172.21.15.89:0/2695058800 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f788c15b240 con 0x7f788c142ea0 2023-10-10T20:40:55.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.348+0000 7f78737fe700 1 -- 172.21.15.89:0/2695058800 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f7880054cd0 con 0x7f788c142ea0 2023-10-10T20:40:55.350 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:55.351 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1635581143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1635581143},{"type":"v1","addr":"172.21.15.89:6837","nonce":1635581143}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5732,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/321638734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":321638734},{"type":"v1","addr":"172.21.15.111:6835","nonce":321638734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1070384555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1070384555},{"type":"v1","addr":"172.21.15.111:6834","nonce":1070384555}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:40:37.630844+0000","modified":"2023-10-10T20:40:41.269543+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6822},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6822":{"gid":6822,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6836/2274240619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2274240619},{"type":"v1","addr":"172.21.15.89:6836","nonce":2274240619}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-10T20:40:55.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.350+0000 7f7890b8c700 1 -- 172.21.15.89:0/2695058800 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7878067320 msgr2=0x7f78780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:55.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.350+0000 7f7890b8c700 1 --2- 172.21.15.89:0/2695058800 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7878067320 0x7f78780697d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f787c005c50 tx=0x7f787c009040 comp rx=0 tx=0).stop 2023-10-10T20:40:55.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.350+0000 7f7890b8c700 1 -- 172.21.15.89:0/2695058800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f788c142ea0 msgr2=0x7f788c159d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:55.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.350+0000 7f7890b8c700 1 --2- 172.21.15.89:0/2695058800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f788c142ea0 0x7f788c159d10 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f788000c590 tx=0x7f7880007f30 comp rx=0 tx=0).stop 2023-10-10T20:40:55.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.350+0000 7f7890b8c700 1 -- 172.21.15.89:0/2695058800 shutdown_connections 2023-10-10T20:40:55.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.350+0000 7f7890b8c700 1 --2- 172.21.15.89:0/2695058800 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f788c0aca10 0x7f788c0c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.350+0000 7f7890b8c700 1 --2- 172.21.15.89:0/2695058800 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7878067320 0x7f78780697d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.350+0000 7f7890b8c700 1 --2- 172.21.15.89:0/2695058800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f788c06c0f0 0x7f788c169e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.350+0000 7f7890b8c700 1 --2- 172.21.15.89:0/2695058800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f788c142ea0 0x7f788c159d10 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.350+0000 7f7890b8c700 1 -- 172.21.15.89:0/2695058800 >> 172.21.15.89:0/2695058800 conn(0x7f788c0ce430 msgr2=0x7f788c0ccb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:55.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.350+0000 7f7890b8c700 1 -- 172.21.15.89:0/2695058800 shutdown_connections 2023-10-10T20:40:55.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.350+0000 7f7890b8c700 1 -- 172.21.15.89:0/2695058800 wait complete. 2023-10-10T20:40:55.356 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 65 2023-10-10T20:40:55.365 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:40:55.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.477+0000 7fb003fff700 1 Processor -- start 2023-10-10T20:40:55.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.477+0000 7fb003fff700 1 -- start start 2023-10-10T20:40:55.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.478+0000 7fb003fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb004159420 0x7fb0040b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:55.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.478+0000 7fb003fff700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0040b2980 con 0x7fb0041597f0 2023-10-10T20:40:55.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.478+0000 7fb003fff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0040b11d0 con 0x7fb00415d230 2023-10-10T20:40:55.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.478+0000 7fb003fff700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0040b14b0 con 0x7fb004159420 2023-10-10T20:40:55.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.478+0000 7fb002ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb004159420 0x7fb0040b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:55.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.478+0000 7faffa7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb0041597f0 0x7fb0040b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34150/0 (socket says 172.21.15.89:34150) 2023-10-10T20:40:55.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.478+0000 7faffa7fc700 1 -- 172.21.15.89:0/3016042999 learned_addr learned my addr 172.21.15.89:0/3016042999 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:55.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.478+0000 7fb002ffd700 1 -- 172.21.15.89:0/3016042999 >> v1:172.21.15.111:6789/0 conn(0x7fb00415d230 legacy=0x7fb00415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:55.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.478+0000 7fb002ffd700 1 -- 172.21.15.89:0/3016042999 >> v1:172.21.15.89:6789/0 conn(0x7fb0041597f0 legacy=0x7fb0040b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:55.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.478+0000 7fb002ffd700 1 -- 172.21.15.89:0/3016042999 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0040b1790 con 0x7fb004159420 2023-10-10T20:40:55.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.478+0000 7fb0027fc700 1 -- 172.21.15.89:0/3016042999 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4256475778 0 0) 0x7fb0040b2980 con 0x7fb0041597f0 2023-10-10T20:40:55.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.479+0000 7fb002ffd700 1 --2- 172.21.15.89:0/3016042999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb004159420 0x7fb0040b5f40 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7faff40098e0 tx=0x7faff40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5706be059b792e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:55.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.479+0000 7fb0027fc700 1 -- 172.21.15.89:0/3016042999 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff4009e80 con 0x7fb004159420 2023-10-10T20:40:55.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.479+0000 7fb0027fc700 1 -- 172.21.15.89:0/3016042999 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff400b860 con 0x7fb004159420 2023-10-10T20:40:55.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.479+0000 7fb0027fc700 1 -- 172.21.15.89:0/3016042999 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff400bbe0 con 0x7fb004159420 2023-10-10T20:40:55.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.479+0000 7fb003fff700 1 -- 172.21.15.89:0/3016042999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb004159420 msgr2=0x7fb0040b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:55.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.479+0000 7fb003fff700 1 --2- 172.21.15.89:0/3016042999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb004159420 0x7fb0040b5f40 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7faff40098e0 tx=0x7faff40091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.479+0000 7fb003fff700 1 -- 172.21.15.89:0/3016042999 shutdown_connections 2023-10-10T20:40:55.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.479+0000 7fb003fff700 1 --2- 172.21.15.89:0/3016042999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb004159420 0x7fb0040b5f40 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.479+0000 7fb003fff700 1 -- 172.21.15.89:0/3016042999 >> 172.21.15.89:0/3016042999 conn(0x7fb0040d0660 msgr2=0x7fb0040d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:55.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.479+0000 7fb003fff700 1 -- 172.21.15.89:0/3016042999 shutdown_connections 2023-10-10T20:40:55.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.479+0000 7fb003fff700 1 -- 172.21.15.89:0/3016042999 wait complete. 2023-10-10T20:40:55.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.481+0000 7fb003fff700 1 Processor -- start 2023-10-10T20:40:55.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.481+0000 7fb003fff700 1 -- start start 2023-10-10T20:40:55.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.481+0000 7fb003fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb004159420 0x7fb0041552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:55.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.481+0000 7fb003fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0041597f0 0x7fb0040ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:55.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.481+0000 7fb003fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb00415d230 0x7fb0040cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:55.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.481+0000 7fb003fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb004155810 con 0x7fb0041597f0 2023-10-10T20:40:55.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.481+0000 7fb003fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0040cae80 con 0x7fb00415d230 2023-10-10T20:40:55.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.481+0000 7fb003fff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0040cb160 con 0x7fb004159420 2023-10-10T20:40:55.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.481+0000 7faffa7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0041597f0 0x7fb0040ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:55.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.481+0000 7faffa7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0041597f0 0x7fb0040ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59932/0 (socket says 172.21.15.89:59932) 2023-10-10T20:40:55.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.481+0000 7faffa7fc700 1 -- 172.21.15.89:0/2080110012 learned_addr learned my addr 172.21.15.89:0/2080110012 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:55.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.482+0000 7fb002ffd700 1 --2- 172.21.15.89:0/2080110012 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb004159420 0x7fb0041552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:55.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.482+0000 7fb0037fe700 1 --2- 172.21.15.89:0/2080110012 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb00415d230 0x7fb0040cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:55.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.482+0000 7faffa7fc700 1 -- 172.21.15.89:0/2080110012 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb004159420 msgr2=0x7fb0041552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:55.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.482+0000 7faffa7fc700 1 --2- 172.21.15.89:0/2080110012 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb004159420 0x7fb0041552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.482+0000 7faffa7fc700 1 -- 172.21.15.89:0/2080110012 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb00415d230 msgr2=0x7fb0040cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:55.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.482+0000 7faffa7fc700 1 --2- 172.21.15.89:0/2080110012 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb00415d230 0x7fb0040cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.482+0000 7faffa7fc700 1 -- 172.21.15.89:0/2080110012 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff4009420 con 0x7fb0041597f0 2023-10-10T20:40:55.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.482+0000 7faffa7fc700 1 --2- 172.21.15.89:0/2080110012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0041597f0 0x7fb0040ca3c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fafec00c7e0 tx=0x7fafec007d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:55.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.482+0000 7faffb7fe700 1 -- 172.21.15.89:0/2080110012 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec005c00 con 0x7fb0041597f0 2023-10-10T20:40:55.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.482+0000 7faffb7fe700 1 -- 172.21.15.89:0/2080110012 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafec005d80 con 0x7fb0041597f0 2023-10-10T20:40:55.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.482+0000 7fb003fff700 1 -- 172.21.15.89:0/2080110012 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0040cb440 con 0x7fb0041597f0 2023-10-10T20:40:55.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.482+0000 7faffb7fe700 1 -- 172.21.15.89:0/2080110012 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec007320 con 0x7fb0041597f0 2023-10-10T20:40:55.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.482+0000 7fb003fff700 1 -- 172.21.15.89:0/2080110012 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0040b4dc0 con 0x7fb0041597f0 2023-10-10T20:40:55.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.483+0000 7fb003fff700 1 -- 172.21.15.89:0/2080110012 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0040c6610 con 0x7fb0041597f0 2023-10-10T20:40:55.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.483+0000 7faffb7fe700 1 -- 172.21.15.89:0/2080110012 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fafec01b3b0 con 0x7fb0041597f0 2023-10-10T20:40:55.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.484+0000 7faffb7fe700 1 --2- 172.21.15.89:0/2080110012 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fafe4067320 0x7fafe40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:55.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.484+0000 7faffb7fe700 1 -- 172.21.15.89:0/2080110012 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fafec084240 con 0x7fb0041597f0 2023-10-10T20:40:55.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.484+0000 7fb002ffd700 1 --2- 172.21.15.89:0/2080110012 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fafe4067320 0x7fafe40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:55.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.484+0000 7fb002ffd700 1 --2- 172.21.15.89:0/2080110012 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fafe4067320 0x7fafe40697d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7faff40093f0 tx=0x7faff4017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:55.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.488+0000 7faffb7fe700 1 -- 172.21.15.89:0/2080110012 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0040c6610 con 0x7fb0041597f0 2023-10-10T20:40:55.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.670+0000 7fb003fff700 1 -- 172.21.15.89:0/2080110012 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0040c6610 con 0x7fb0041597f0 2023-10-10T20:40:55.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.671+0000 7faffb7fe700 1 -- 172.21.15.89:0/2080110012 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+15355 (secure 0 0 0) 0x7fb0040c6610 con 0x7fb0041597f0 2023-10-10T20:40:55.673 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:55.673 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":100,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:40:39.247030+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:40:36.002449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-10T20:40:36.591960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:40:55.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.672+0000 7fb003fff700 1 -- 172.21.15.89:0/2080110012 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fafe4067320 msgr2=0x7fafe40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:55.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.672+0000 7fb003fff700 1 --2- 172.21.15.89:0/2080110012 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fafe4067320 0x7fafe40697d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7faff40093f0 tx=0x7faff4017000 comp rx=0 tx=0).stop 2023-10-10T20:40:55.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.672+0000 7fb003fff700 1 -- 172.21.15.89:0/2080110012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0041597f0 msgr2=0x7fb0040ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:55.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.672+0000 7fb003fff700 1 --2- 172.21.15.89:0/2080110012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0041597f0 0x7fb0040ca3c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fafec00c7e0 tx=0x7fafec007d70 comp rx=0 tx=0).stop 2023-10-10T20:40:55.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.673+0000 7fb003fff700 1 -- 172.21.15.89:0/2080110012 shutdown_connections 2023-10-10T20:40:55.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.673+0000 7fb003fff700 1 --2- 172.21.15.89:0/2080110012 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fafe4067320 0x7fafe40697d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.673+0000 7fb003fff700 1 --2- 172.21.15.89:0/2080110012 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb00415d230 0x7fb0040cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.673+0000 7fb003fff700 1 --2- 172.21.15.89:0/2080110012 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb004159420 0x7fb0041552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.673+0000 7fb003fff700 1 --2- 172.21.15.89:0/2080110012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0041597f0 0x7fb0040ca3c0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.673+0000 7fb003fff700 1 -- 172.21.15.89:0/2080110012 >> 172.21.15.89:0/2080110012 conn(0x7fb0040d0660 msgr2=0x7fb00415d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:55.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.673+0000 7fb003fff700 1 -- 172.21.15.89:0/2080110012 shutdown_connections 2023-10-10T20:40:55.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.673+0000 7fb003fff700 1 -- 172.21.15.89:0/2080110012 wait complete. 2023-10-10T20:40:55.687 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:40:55.687 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:40:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.800+0000 7fb8dffff700 1 Processor -- start 2023-10-10T20:40:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.800+0000 7fb8dffff700 1 -- start start 2023-10-10T20:40:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.801+0000 7fb8dffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb8e0159420 0x7fb8e00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.801+0000 7fb8dffff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb8e015d230 0x7fb8e015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.801+0000 7fb8dffff700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8e00b2980 con 0x7fb8e01597f0 2023-10-10T20:40:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.801+0000 7fb8dffff700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e015db70 con 0x7fb8e0159420 2023-10-10T20:40:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.801+0000 7fb8dffff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e00b11d0 con 0x7fb8e015d230 2023-10-10T20:40:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.801+0000 7fb8deffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb8e0159420 0x7fb8e00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.801+0000 7fb8de7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb8e01597f0 0x7fb8e00b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34166/0 (socket says 172.21.15.89:34166) 2023-10-10T20:40:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.801+0000 7fb8de7fc700 1 -- 172.21.15.89:0/55380342 learned_addr learned my addr 172.21.15.89:0/55380342 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.801+0000 7fb8df7fe700 1 --2- 172.21.15.89:0/55380342 >> v2:172.21.15.111:3300/0 conn(0x7fb8e015d230 0x7fb8e015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.801+0000 7fb8deffd700 1 -- 172.21.15.89:0/55380342 >> v2:172.21.15.111:3300/0 conn(0x7fb8e015d230 msgr2=0x7fb8e015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.801+0000 7fb8deffd700 1 --2- 172.21.15.89:0/55380342 >> v2:172.21.15.111:3300/0 conn(0x7fb8e015d230 0x7fb8e015d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.801+0000 7fb8deffd700 1 -- 172.21.15.89:0/55380342 >> v1:172.21.15.89:6789/0 conn(0x7fb8e01597f0 legacy=0x7fb8e00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.801+0000 7fb8deffd700 1 -- 172.21.15.89:0/55380342 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e00b14b0 con 0x7fb8e0159420 2023-10-10T20:40:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.801+0000 7fb8ddffb700 1 -- 172.21.15.89:0/55380342 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455952316 0 0) 0x7fb8e00b2980 con 0x7fb8e01597f0 2023-10-10T20:40:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.801+0000 7fb8deffd700 1 --2- 172.21.15.89:0/55380342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8e0159420 0x7fb8e00b5f40 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fb8e015d170 tx=0x7fb8d0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75ebd62bc4fd506 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.802+0000 7fb8ddffb700 1 -- 172.21.15.89:0/55380342 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d000e070 con 0x7fb8e0159420 2023-10-10T20:40:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.802+0000 7fb8ddffb700 1 -- 172.21.15.89:0/55380342 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d000bc30 con 0x7fb8e0159420 2023-10-10T20:40:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.802+0000 7fb8ddffb700 1 -- 172.21.15.89:0/55380342 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0012430 con 0x7fb8e0159420 2023-10-10T20:40:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.802+0000 7fb8dffff700 1 -- 172.21.15.89:0/55380342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8e0159420 msgr2=0x7fb8e00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.802+0000 7fb8dffff700 1 --2- 172.21.15.89:0/55380342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8e0159420 0x7fb8e00b5f40 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fb8e015d170 tx=0x7fb8d0009780 comp rx=0 tx=0).stop 2023-10-10T20:40:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.802+0000 7fb8dffff700 1 -- 172.21.15.89:0/55380342 shutdown_connections 2023-10-10T20:40:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.802+0000 7fb8dffff700 1 --2- 172.21.15.89:0/55380342 >> v2:172.21.15.111:3300/0 conn(0x7fb8e015d230 0x7fb8e015d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.802+0000 7fb8dffff700 1 --2- 172.21.15.89:0/55380342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8e0159420 0x7fb8e00b5f40 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.802+0000 7fb8dffff700 1 -- 172.21.15.89:0/55380342 >> 172.21.15.89:0/55380342 conn(0x7fb8e00d0660 msgr2=0x7fb8e00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.802+0000 7fb8dffff700 1 -- 172.21.15.89:0/55380342 shutdown_connections 2023-10-10T20:40:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.802+0000 7fb8dffff700 1 -- 172.21.15.89:0/55380342 wait complete. 2023-10-10T20:40:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.803+0000 7fb8dffff700 1 Processor -- start 2023-10-10T20:40:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.803+0000 7fb8dffff700 1 -- start start 2023-10-10T20:40:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.804+0000 7fb8dffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8e0159420 0x7fb8e015b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.804+0000 7fb8dffff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8e01597f0 0x7fb8e00c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.804+0000 7fb8dffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8e015d230 0x7fb8e00c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.804+0000 7fb8dffff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e015a710 con 0x7fb8e015d230 2023-10-10T20:40:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.804+0000 7fb8dffff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e015a9f0 con 0x7fb8e0159420 2023-10-10T20:40:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.804+0000 7fb8dffff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e015acd0 con 0x7fb8e01597f0 2023-10-10T20:40:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.804+0000 7fb8df7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8e015d230 0x7fb8e00c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.804+0000 7fb8df7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8e015d230 0x7fb8e00c7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59958/0 (socket says 172.21.15.89:59958) 2023-10-10T20:40:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.804+0000 7fb8df7fe700 1 -- 172.21.15.89:0/3444838463 learned_addr learned my addr 172.21.15.89:0/3444838463 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.804+0000 7fb8df7fe700 1 -- 172.21.15.89:0/3444838463 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8e01597f0 msgr2=0x7fb8e00c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.804+0000 7fb8df7fe700 1 --2- 172.21.15.89:0/3444838463 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8e01597f0 0x7fb8e00c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.804+0000 7fb8df7fe700 1 -- 172.21.15.89:0/3444838463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8e0159420 msgr2=0x7fb8e015b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.804+0000 7fb8df7fe700 1 --2- 172.21.15.89:0/3444838463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8e0159420 0x7fb8e015b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.804+0000 7fb8df7fe700 1 -- 172.21.15.89:0/3444838463 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d0009a20 con 0x7fb8e015d230 2023-10-10T20:40:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.805+0000 7fb8df7fe700 1 --2- 172.21.15.89:0/3444838463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8e015d230 0x7fb8e00c7fa0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800c880 tx=0x7fb8d8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.805+0000 7fb8c6ffd700 1 -- 172.21.15.89:0/3444838463 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8011600 con 0x7fb8e015d230 2023-10-10T20:40:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.805+0000 7fb8c6ffd700 1 -- 172.21.15.89:0/3444838463 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d8011760 con 0x7fb8e015d230 2023-10-10T20:40:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.805+0000 7fb8dffff700 1 -- 172.21.15.89:0/3444838463 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e015afb0 con 0x7fb8e015d230 2023-10-10T20:40:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.805+0000 7fb8c6ffd700 1 -- 172.21.15.89:0/3444838463 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8011b00 con 0x7fb8e015d230 2023-10-10T20:40:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.805+0000 7fb8dffff700 1 -- 172.21.15.89:0/3444838463 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e015a360 con 0x7fb8e015d230 2023-10-10T20:40:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.806+0000 7fb8dffff700 1 -- 172.21.15.89:0/3444838463 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e015b4d0 con 0x7fb8e015d230 2023-10-10T20:40:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.806+0000 7fb8c6ffd700 1 -- 172.21.15.89:0/3444838463 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb8d8018360 con 0x7fb8e015d230 2023-10-10T20:40:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.806+0000 7fb8c6ffd700 1 --2- 172.21.15.89:0/3444838463 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb8cc067320 0x7fb8cc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.807+0000 7fb8c6ffd700 1 -- 172.21.15.89:0/3444838463 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb8d8084af0 con 0x7fb8e015d230 2023-10-10T20:40:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.807+0000 7fb8deffd700 1 --2- 172.21.15.89:0/3444838463 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb8cc067320 0x7fb8cc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.807+0000 7fb8deffd700 1 --2- 172.21.15.89:0/3444838463 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb8cc067320 0x7fb8cc0697d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb8e015d170 tx=0x7fb8d0019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:55.811+0000 7fb8c6ffd700 1 -- 172.21.15.89:0/3444838463 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8e015b4d0 con 0x7fb8e015d230 2023-10-10T20:40:56.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.039+0000 7fb8dffff700 1 -- 172.21.15.89:0/3444838463 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8e015b4d0 con 0x7fb8e015d230 2023-10-10T20:40:56.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.040+0000 7fb8c6ffd700 1 -- 172.21.15.89:0/3444838463 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fb8e015b4d0 con 0x7fb8e015d230 2023-10-10T20:40:56.042 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:56.042 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1635581143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1635581143},{"type":"v1","addr":"172.21.15.89:6837","nonce":1635581143}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5732,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/321638734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":321638734},{"type":"v1","addr":"172.21.15.111:6835","nonce":321638734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1070384555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1070384555},{"type":"v1","addr":"172.21.15.111:6834","nonce":1070384555}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:40:37.630844+0000","modified":"2023-10-10T20:40:41.269543+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6822},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6822":{"gid":6822,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6836/2274240619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2274240619},{"type":"v1","addr":"172.21.15.89:6836","nonce":2274240619}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-10T20:40:56.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.042+0000 7fb8dffff700 1 -- 172.21.15.89:0/3444838463 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb8cc067320 msgr2=0x7fb8cc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:56.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.042+0000 7fb8dffff700 1 --2- 172.21.15.89:0/3444838463 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb8cc067320 0x7fb8cc0697d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb8e015d170 tx=0x7fb8d0019000 comp rx=0 tx=0).stop 2023-10-10T20:40:56.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.042+0000 7fb8dffff700 1 -- 172.21.15.89:0/3444838463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8e015d230 msgr2=0x7fb8e00c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:56.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.042+0000 7fb8dffff700 1 --2- 172.21.15.89:0/3444838463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8e015d230 0x7fb8e00c7fa0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800c880 tx=0x7fb8d8008040 comp rx=0 tx=0).stop 2023-10-10T20:40:56.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.042+0000 7fb8dffff700 1 -- 172.21.15.89:0/3444838463 shutdown_connections 2023-10-10T20:40:56.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.042+0000 7fb8dffff700 1 --2- 172.21.15.89:0/3444838463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8e015d230 0x7fb8e00c7fa0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.042+0000 7fb8dffff700 1 --2- 172.21.15.89:0/3444838463 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb8cc067320 0x7fb8cc0697d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.042+0000 7fb8dffff700 1 --2- 172.21.15.89:0/3444838463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8e0159420 0x7fb8e015b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.042+0000 7fb8dffff700 1 --2- 172.21.15.89:0/3444838463 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8e01597f0 0x7fb8e00c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.042+0000 7fb8dffff700 1 -- 172.21.15.89:0/3444838463 >> 172.21.15.89:0/3444838463 conn(0x7fb8e00d0660 msgr2=0x7fb8e00698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:56.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.042+0000 7fb8dffff700 1 -- 172.21.15.89:0/3444838463 shutdown_connections 2023-10-10T20:40:56.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.042+0000 7fb8dffff700 1 -- 172.21.15.89:0/3444838463 wait complete. 2023-10-10T20:40:56.047 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 65 2023-10-10T20:40:56.056 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:40:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.169+0000 7fabcd5dc700 1 Processor -- start 2023-10-10T20:40:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.169+0000 7fabcd5dc700 1 -- start start 2023-10-10T20:40:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.170+0000 7fabcd5dc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fabc8159890 0x7fabc815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.170+0000 7fabcd5dc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabc8069560 0x7fabc80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.170+0000 7fabcd5dc700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc8069ee0 con 0x7fabc8159c60 2023-10-10T20:40:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.170+0000 7fabcd5dc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc815a160 con 0x7fabc8159890 2023-10-10T20:40:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.170+0000 7fabcd5dc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80d13a0 con 0x7fabc8069560 2023-10-10T20:40:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.170+0000 7fabc6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fabc8159890 0x7fabc815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.170+0000 7fabc6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fabc8159890 0x7fabc815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59962/0 (socket says 172.21.15.89:59962) 2023-10-10T20:40:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.170+0000 7fabc6ffd700 1 -- 172.21.15.89:0/44101758 learned_addr learned my addr 172.21.15.89:0/44101758 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.170+0000 7fabc77fe700 1 --2- 172.21.15.89:0/44101758 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabc8069560 0x7fabc80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.170+0000 7fabc6ffd700 1 -- 172.21.15.89:0/44101758 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabc8069560 msgr2=0x7fabc80699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.170+0000 7fabc6ffd700 1 --2- 172.21.15.89:0/44101758 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabc8069560 0x7fabc80699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.170+0000 7fabc6ffd700 1 -- 172.21.15.89:0/44101758 >> v1:172.21.15.111:6789/0 conn(0x7fabc8159c60 legacy=0x7fabc815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.170+0000 7fabc6ffd700 1 -- 172.21.15.89:0/44101758 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc815e140 con 0x7fabc8159890 2023-10-10T20:40:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.170+0000 7fabc77fe700 1 --2- 172.21.15.89:0/44101758 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabc8069560 0x7fabc80699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.171+0000 7fabc6ffd700 1 --2- 172.21.15.89:0/44101758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc8159890 0x7fabc815abd0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fabb80099b0 tx=0x7fabb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1542ec3f662563 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.171+0000 7fabc67fc700 1 -- 172.21.15.89:0/44101758 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb800b860 con 0x7fabc8159890 2023-10-10T20:40:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.171+0000 7fabc67fc700 1 -- 172.21.15.89:0/44101758 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb800b9c0 con 0x7fabc8159890 2023-10-10T20:40:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.171+0000 7fabc67fc700 1 -- 172.21.15.89:0/44101758 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb800bd60 con 0x7fabc8159890 2023-10-10T20:40:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.171+0000 7fabcd5dc700 1 -- 172.21.15.89:0/44101758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc8159890 msgr2=0x7fabc815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.171+0000 7fabcd5dc700 1 --2- 172.21.15.89:0/44101758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc8159890 0x7fabc815abd0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fabb80099b0 tx=0x7fabb8009250 comp rx=0 tx=0).stop 2023-10-10T20:40:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.171+0000 7fabcd5dc700 1 -- 172.21.15.89:0/44101758 shutdown_connections 2023-10-10T20:40:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.171+0000 7fabcd5dc700 1 --2- 172.21.15.89:0/44101758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc8159890 0x7fabc815abd0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.171+0000 7fabcd5dc700 1 --2- 172.21.15.89:0/44101758 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabc8069560 0x7fabc80699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.171+0000 7fabcd5dc700 1 -- 172.21.15.89:0/44101758 >> 172.21.15.89:0/44101758 conn(0x7fabc8068e20 msgr2=0x7fabc80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:56.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.171+0000 7fabcd5dc700 1 -- 172.21.15.89:0/44101758 shutdown_connections 2023-10-10T20:40:56.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.171+0000 7fabcd5dc700 1 -- 172.21.15.89:0/44101758 wait complete. 2023-10-10T20:40:56.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.172+0000 7fabcd5dc700 1 Processor -- start 2023-10-10T20:40:56.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.173+0000 7fabcd5dc700 1 -- start start 2023-10-10T20:40:56.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.173+0000 7fabcd5dc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabc8069560 0x7fabc81524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:56.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.173+0000 7fabcd5dc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabc8159890 0x7fabc80cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:56.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.173+0000 7fabcd5dc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc8159c60 0x7fabc80b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:56.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.173+0000 7fabcd5dc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80b3a20 con 0x7fabc8159c60 2023-10-10T20:40:56.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.173+0000 7fabcd5dc700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80b3d00 con 0x7fabc8069560 2023-10-10T20:40:56.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.173+0000 7fabcd5dc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80b3fe0 con 0x7fabc8159890 2023-10-10T20:40:56.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.173+0000 7fabc77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc8159c60 0x7fabc80b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:56.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.173+0000 7fabc77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc8159c60 0x7fabc80b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59974/0 (socket says 172.21.15.89:59974) 2023-10-10T20:40:56.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.173+0000 7fabc77fe700 1 -- 172.21.15.89:0/515585475 learned_addr learned my addr 172.21.15.89:0/515585475 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:56.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.173+0000 7fabc6ffd700 1 --2- 172.21.15.89:0/515585475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabc8069560 0x7fabc81524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:56.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.173+0000 7fabbe7fc700 1 --2- 172.21.15.89:0/515585475 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabc8159890 0x7fabc80cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:56.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.174+0000 7fabc77fe700 1 -- 172.21.15.89:0/515585475 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabc8159890 msgr2=0x7fabc80cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.174+0000 7fabc77fe700 1 --2- 172.21.15.89:0/515585475 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabc8159890 0x7fabc80cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.174+0000 7fabc77fe700 1 -- 172.21.15.89:0/515585475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabc8069560 msgr2=0x7fabc81524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.174+0000 7fabc77fe700 1 --2- 172.21.15.89:0/515585475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabc8069560 0x7fabc81524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.174+0000 7fabc77fe700 1 -- 172.21.15.89:0/515585475 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb80094f0 con 0x7fabc8159c60 2023-10-10T20:40:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.174+0000 7fabc77fe700 1 --2- 172.21.15.89:0/515585475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc8159c60 0x7fabc80b14e0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009fc0 tx=0x7fabc00099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.174+0000 7fabbf7fe700 1 -- 172.21.15.89:0/515585475 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc000dbe0 con 0x7fabc8159c60 2023-10-10T20:40:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.174+0000 7fabbf7fe700 1 -- 172.21.15.89:0/515585475 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc000f040 con 0x7fabc8159c60 2023-10-10T20:40:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.174+0000 7fabcd5dc700 1 -- 172.21.15.89:0/515585475 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc80b42c0 con 0x7fabc8159c60 2023-10-10T20:40:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.174+0000 7fabbf7fe700 1 -- 172.21.15.89:0/515585475 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc0015440 con 0x7fabc8159c60 2023-10-10T20:40:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.174+0000 7fabcd5dc700 1 -- 172.21.15.89:0/515585475 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc81529f0 con 0x7fabc8159c60 2023-10-10T20:40:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.175+0000 7fabcd5dc700 1 -- 172.21.15.89:0/515585475 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc8169b90 con 0x7fabc8159c60 2023-10-10T20:40:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.175+0000 7fabbf7fe700 1 -- 172.21.15.89:0/515585475 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fabc001e3b0 con 0x7fabc8159c60 2023-10-10T20:40:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.176+0000 7fabbf7fe700 1 --2- 172.21.15.89:0/515585475 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faba8067320 0x7faba80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.176+0000 7fabbf7fe700 1 -- 172.21.15.89:0/515585475 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fabc0085160 con 0x7fabc8159c60 2023-10-10T20:40:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.176+0000 7fabc6ffd700 1 --2- 172.21.15.89:0/515585475 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faba8067320 0x7faba80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.176+0000 7fabc6ffd700 1 --2- 172.21.15.89:0/515585475 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faba8067320 0x7faba80697d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fabb80099b0 tx=0x7fabb8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.180+0000 7fabbf7fe700 1 -- 172.21.15.89:0/515585475 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fabc0051ce0 con 0x7fabc8159c60 2023-10-10T20:40:56.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.405+0000 7fabcd5dc700 1 -- 172.21.15.89:0/515585475 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabc804c3c0 con 0x7fabc8159c60 2023-10-10T20:40:56.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.405+0000 7fabbf7fe700 1 -- 172.21.15.89:0/515585475 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fabc0055300 con 0x7fabc8159c60 2023-10-10T20:40:56.408 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:56.408 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1635581143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1635581143},{"type":"v1","addr":"172.21.15.89:6837","nonce":1635581143}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5732,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/321638734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":321638734},{"type":"v1","addr":"172.21.15.111:6835","nonce":321638734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1070384555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1070384555},{"type":"v1","addr":"172.21.15.111:6834","nonce":1070384555}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:40:37.630844+0000","modified":"2023-10-10T20:40:41.269543+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6822},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6822":{"gid":6822,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6836/2274240619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2274240619},{"type":"v1","addr":"172.21.15.89:6836","nonce":2274240619}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-10T20:40:56.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.407+0000 7fabcd5dc700 1 -- 172.21.15.89:0/515585475 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faba8067320 msgr2=0x7faba80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:56.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.407+0000 7fabcd5dc700 1 --2- 172.21.15.89:0/515585475 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faba8067320 0x7faba80697d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fabb80099b0 tx=0x7fabb8017000 comp rx=0 tx=0).stop 2023-10-10T20:40:56.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.407+0000 7fabcd5dc700 1 -- 172.21.15.89:0/515585475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc8159c60 msgr2=0x7fabc80b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:56.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.407+0000 7fabcd5dc700 1 --2- 172.21.15.89:0/515585475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc8159c60 0x7fabc80b14e0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009fc0 tx=0x7fabc00099b0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.407+0000 7fabcd5dc700 1 -- 172.21.15.89:0/515585475 shutdown_connections 2023-10-10T20:40:56.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.407+0000 7fabcd5dc700 1 --2- 172.21.15.89:0/515585475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc8159c60 0x7fabc80b14e0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.407+0000 7fabcd5dc700 1 --2- 172.21.15.89:0/515585475 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faba8067320 0x7faba80697d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.407+0000 7fabcd5dc700 1 --2- 172.21.15.89:0/515585475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabc8069560 0x7fabc81524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.407+0000 7fabcd5dc700 1 --2- 172.21.15.89:0/515585475 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fabc8159890 0x7fabc80cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.407+0000 7fabcd5dc700 1 -- 172.21.15.89:0/515585475 >> 172.21.15.89:0/515585475 conn(0x7fabc8068e20 msgr2=0x7fabc80b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:56.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.407+0000 7fabcd5dc700 1 -- 172.21.15.89:0/515585475 shutdown_connections 2023-10-10T20:40:56.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.407+0000 7fabcd5dc700 1 -- 172.21.15.89:0/515585475 wait complete. 2023-10-10T20:40:56.412 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 65 2023-10-10T20:40:56.421 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:40:56.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.536+0000 7fd151136700 1 Processor -- start 2023-10-10T20:40:56.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.536+0000 7fd151136700 1 -- start start 2023-10-10T20:40:56.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.537+0000 7fd151136700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd14c159420 0x7fd14c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:56.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.537+0000 7fd151136700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd14c1597f0 0x7fd14c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:56.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.537+0000 7fd151136700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c0b11d0 con 0x7fd14c15d230 2023-10-10T20:40:56.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.537+0000 7fd151136700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14c0b14b0 con 0x7fd14c159420 2023-10-10T20:40:56.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.537+0000 7fd151136700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14c0b1790 con 0x7fd14c1597f0 2023-10-10T20:40:56.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.537+0000 7fd14bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd14c159420 0x7fd14c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:56.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.537+0000 7fd14bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd14c159420 0x7fd14c0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59990/0 (socket says 172.21.15.89:59990) 2023-10-10T20:40:56.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.537+0000 7fd14bfff700 1 -- 172.21.15.89:0/516492977 learned_addr learned my addr 172.21.15.89:0/516492977 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:56.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.537+0000 7fd14b7fe700 1 --2- 172.21.15.89:0/516492977 >> v2:172.21.15.111:3300/0 conn(0x7fd14c1597f0 0x7fd14c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:56.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.537+0000 7fd14bfff700 1 -- 172.21.15.89:0/516492977 >> v2:172.21.15.111:3300/0 conn(0x7fd14c1597f0 msgr2=0x7fd14c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:56.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.537+0000 7fd14bfff700 1 --2- 172.21.15.89:0/516492977 >> v2:172.21.15.111:3300/0 conn(0x7fd14c1597f0 0x7fd14c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.537+0000 7fd14bfff700 1 -- 172.21.15.89:0/516492977 >> v1:172.21.15.111:6789/0 conn(0x7fd14c15d230 legacy=0x7fd14c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:56.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.538+0000 7fd14bfff700 1 -- 172.21.15.89:0/516492977 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c0b1a70 con 0x7fd14c159420 2023-10-10T20:40:56.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.538+0000 7fd14bfff700 1 --2- 172.21.15.89:0/516492977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd14c159420 0x7fd14c0b5f40 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fd14c089750 tx=0x7fd13c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450e324539d52460 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:56.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.538+0000 7fd14affd700 1 -- 172.21.15.89:0/516492977 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c009e70 con 0x7fd14c159420 2023-10-10T20:40:56.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.538+0000 7fd14affd700 1 -- 172.21.15.89:0/516492977 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13c00b860 con 0x7fd14c159420 2023-10-10T20:40:56.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.538+0000 7fd14affd700 1 -- 172.21.15.89:0/516492977 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c00bc00 con 0x7fd14c159420 2023-10-10T20:40:56.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.538+0000 7fd151136700 1 -- 172.21.15.89:0/516492977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd14c159420 msgr2=0x7fd14c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:56.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.538+0000 7fd151136700 1 --2- 172.21.15.89:0/516492977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd14c159420 0x7fd14c0b5f40 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fd14c089750 tx=0x7fd13c009250 comp rx=0 tx=0).stop 2023-10-10T20:40:56.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.538+0000 7fd151136700 1 -- 172.21.15.89:0/516492977 shutdown_connections 2023-10-10T20:40:56.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.538+0000 7fd151136700 1 --2- 172.21.15.89:0/516492977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd14c159420 0x7fd14c0b5f40 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.538+0000 7fd151136700 1 --2- 172.21.15.89:0/516492977 >> v2:172.21.15.111:3300/0 conn(0x7fd14c1597f0 0x7fd14c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.538+0000 7fd151136700 1 -- 172.21.15.89:0/516492977 >> 172.21.15.89:0/516492977 conn(0x7fd14c0d0660 msgr2=0x7fd14c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:56.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.538+0000 7fd151136700 1 -- 172.21.15.89:0/516492977 shutdown_connections 2023-10-10T20:40:56.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.538+0000 7fd151136700 1 -- 172.21.15.89:0/516492977 wait complete. 2023-10-10T20:40:56.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.539+0000 7fd151136700 1 Processor -- start 2023-10-10T20:40:56.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.540+0000 7fd151136700 1 -- start start 2023-10-10T20:40:56.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.540+0000 7fd151136700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd14c159420 0x7fd14c0c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:56.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.540+0000 7fd151136700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd14c1597f0 0x7fd14c0c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:56.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.540+0000 7fd151136700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd14c15d230 0x7fd14c146b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:56.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.540+0000 7fd151136700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0cb410 con 0x7fd14c1597f0 2023-10-10T20:40:56.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.540+0000 7fd151136700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0d03a0 con 0x7fd14c159420 2023-10-10T20:40:56.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.540+0000 7fd151136700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0c6360 con 0x7fd14c15d230 2023-10-10T20:40:56.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.540+0000 7fd14b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd14c1597f0 0x7fd14c0c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:56.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.540+0000 7fd14b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd14c1597f0 0x7fd14c0c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60000/0 (socket says 172.21.15.89:60000) 2023-10-10T20:40:56.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.540+0000 7fd14b7fe700 1 -- 172.21.15.89:0/3836149346 learned_addr learned my addr 172.21.15.89:0/3836149346 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:56.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.541+0000 7fd14bfff700 1 --2- 172.21.15.89:0/3836149346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd14c159420 0x7fd14c0c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:56.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.541+0000 7fd150935700 1 --2- 172.21.15.89:0/3836149346 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd14c15d230 0x7fd14c146b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:56.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.541+0000 7fd14b7fe700 1 -- 172.21.15.89:0/3836149346 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd14c15d230 msgr2=0x7fd14c146b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:56.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.541+0000 7fd14b7fe700 1 --2- 172.21.15.89:0/3836149346 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd14c15d230 0x7fd14c146b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.541+0000 7fd14b7fe700 1 -- 172.21.15.89:0/3836149346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd14c159420 msgr2=0x7fd14c0c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.541+0000 7fd14b7fe700 1 --2- 172.21.15.89:0/3836149346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd14c159420 0x7fd14c0c5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.541+0000 7fd14b7fe700 1 -- 172.21.15.89:0/3836149346 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c0094f0 con 0x7fd14c1597f0 2023-10-10T20:40:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.541+0000 7fd14bfff700 1 --2- 172.21.15.89:0/3836149346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd14c159420 0x7fd14c0c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.541+0000 7fd150935700 1 --2- 172.21.15.89:0/3836149346 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd14c15d230 0x7fd14c146b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:56.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.541+0000 7fd14b7fe700 1 --2- 172.21.15.89:0/3836149346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd14c1597f0 0x7fd14c0c8a90 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fd14000ec80 tx=0x7fd1400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:56.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.541+0000 7fd133fff700 1 -- 172.21.15.89:0/3836149346 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14000c9d0 con 0x7fd14c1597f0 2023-10-10T20:40:56.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.541+0000 7fd133fff700 1 -- 172.21.15.89:0/3836149346 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14000cb30 con 0x7fd14c1597f0 2023-10-10T20:40:56.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.541+0000 7fd133fff700 1 -- 172.21.15.89:0/3836149346 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd140013470 con 0x7fd14c1597f0 2023-10-10T20:40:56.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.541+0000 7fd151136700 1 -- 172.21.15.89:0/3836149346 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14c0caab0 con 0x7fd14c1597f0 2023-10-10T20:40:56.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.541+0000 7fd151136700 1 -- 172.21.15.89:0/3836149346 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14c0cae00 con 0x7fd14c1597f0 2023-10-10T20:40:56.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.542+0000 7fd151136700 1 -- 172.21.15.89:0/3836149346 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14c04c3c0 con 0x7fd14c1597f0 2023-10-10T20:40:56.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.542+0000 7fd133fff700 1 -- 172.21.15.89:0/3836149346 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd140013860 con 0x7fd14c1597f0 2023-10-10T20:40:56.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.543+0000 7fd133fff700 1 --2- 172.21.15.89:0/3836149346 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd138067320 0x7fd1380697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:56.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.543+0000 7fd133fff700 1 -- 172.21.15.89:0/3836149346 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd140084fc0 con 0x7fd14c1597f0 2023-10-10T20:40:56.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.543+0000 7fd14bfff700 1 --2- 172.21.15.89:0/3836149346 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd138067320 0x7fd1380697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:56.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.543+0000 7fd14bfff700 1 --2- 172.21.15.89:0/3836149346 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd138067320 0x7fd1380697d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd14c089750 tx=0x7fd13c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:56.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.547+0000 7fd133fff700 1 -- 172.21.15.89:0/3836149346 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd140051c50 con 0x7fd14c1597f0 2023-10-10T20:40:56.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.731+0000 7fd151136700 1 -- 172.21.15.89:0/3836149346 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd14c0ca860 con 0x7fd14c1597f0 2023-10-10T20:40:56.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.732+0000 7fd133fff700 1 -- 172.21.15.89:0/3836149346 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+15355 (secure 0 0 0) 0x7fd140055270 con 0x7fd14c1597f0 2023-10-10T20:40:56.734 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:40:56.734 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":100,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:40:39.247030+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:40:36.002449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-10T20:40:36.591960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:40:56.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.734+0000 7fd151136700 1 -- 172.21.15.89:0/3836149346 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd138067320 msgr2=0x7fd1380697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:56.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.734+0000 7fd151136700 1 --2- 172.21.15.89:0/3836149346 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd138067320 0x7fd1380697d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd14c089750 tx=0x7fd13c017000 comp rx=0 tx=0).stop 2023-10-10T20:40:56.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.734+0000 7fd151136700 1 -- 172.21.15.89:0/3836149346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd14c1597f0 msgr2=0x7fd14c0c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:56.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.734+0000 7fd151136700 1 --2- 172.21.15.89:0/3836149346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd14c1597f0 0x7fd14c0c8a90 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fd14000ec80 tx=0x7fd1400091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.734+0000 7fd151136700 1 -- 172.21.15.89:0/3836149346 shutdown_connections 2023-10-10T20:40:56.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.734+0000 7fd151136700 1 --2- 172.21.15.89:0/3836149346 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd14c15d230 0x7fd14c146b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.734+0000 7fd151136700 1 --2- 172.21.15.89:0/3836149346 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd138067320 0x7fd1380697d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.734+0000 7fd151136700 1 --2- 172.21.15.89:0/3836149346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd14c159420 0x7fd14c0c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.734+0000 7fd151136700 1 --2- 172.21.15.89:0/3836149346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd14c1597f0 0x7fd14c0c8a90 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.734+0000 7fd151136700 1 -- 172.21.15.89:0/3836149346 >> 172.21.15.89:0/3836149346 conn(0x7fd14c0d0660 msgr2=0x7fd14c069940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:56.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.734+0000 7fd151136700 1 -- 172.21.15.89:0/3836149346 shutdown_connections 2023-10-10T20:40:56.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.734+0000 7fd151136700 1 -- 172.21.15.89:0/3836149346 wait complete. 2023-10-10T20:40:56.748 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:40:56.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.862+0000 7fb394c91700 1 Processor -- start 2023-10-10T20:40:56.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.862+0000 7fb394c91700 1 -- start start 2023-10-10T20:40:56.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.863+0000 7fb394c91700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb390159890 0x7fb39015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:56.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.863+0000 7fb394c91700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb390159c60 0x7fb39015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:56.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.863+0000 7fb394c91700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb39015db10 0x7fb390069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:56.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.863+0000 7fb394c91700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb39015a160 con 0x7fb390159c60 2023-10-10T20:40:56.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.863+0000 7fb394c91700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3900d13a0 con 0x7fb39015db10 2023-10-10T20:40:56.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.863+0000 7fb394c91700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39015e140 con 0x7fb390159890 2023-10-10T20:40:56.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.863+0000 7fb38dd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb390159c60 0x7fb39015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:56.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.863+0000 7fb38dd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb390159c60 0x7fb39015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60014/0 (socket says 172.21.15.89:60014) 2023-10-10T20:40:56.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.863+0000 7fb38dd9b700 1 -- 172.21.15.89:0/1609598320 learned_addr learned my addr 172.21.15.89:0/1609598320 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:56.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.863+0000 7fb38e59c700 1 --2- 172.21.15.89:0/1609598320 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb390159890 0x7fb39015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:56.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.863+0000 7fb38ed9d700 1 --2- 172.21.15.89:0/1609598320 >> v2:172.21.15.111:3300/0 conn(0x7fb39015db10 0x7fb390069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:56.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.863+0000 7fb38dd9b700 1 -- 172.21.15.89:0/1609598320 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb390159890 msgr2=0x7fb39015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:56.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.863+0000 7fb38dd9b700 1 --2- 172.21.15.89:0/1609598320 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb390159890 0x7fb39015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.863+0000 7fb38dd9b700 1 -- 172.21.15.89:0/1609598320 >> v2:172.21.15.111:3300/0 conn(0x7fb39015db10 msgr2=0x7fb390069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:56.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.863+0000 7fb38dd9b700 1 --2- 172.21.15.89:0/1609598320 >> v2:172.21.15.111:3300/0 conn(0x7fb39015db10 0x7fb390069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.863+0000 7fb38dd9b700 1 -- 172.21.15.89:0/1609598320 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39015a3d0 con 0x7fb390159c60 2023-10-10T20:40:56.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.863+0000 7fb38e59c700 1 --2- 172.21.15.89:0/1609598320 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb390159890 0x7fb39015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:40:56.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.863+0000 7fb38dd9b700 1 --2- 172.21.15.89:0/1609598320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb390159c60 0x7fb39015d5d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb390089b50 tx=0x7fb378009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b180b2d22de244 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:56.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.863+0000 7fb38d59a700 1 -- 172.21.15.89:0/1609598320 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37800b860 con 0x7fb390159c60 2023-10-10T20:40:56.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.864+0000 7fb38d59a700 1 -- 172.21.15.89:0/1609598320 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37800b9c0 con 0x7fb390159c60 2023-10-10T20:40:56.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.864+0000 7fb38d59a700 1 -- 172.21.15.89:0/1609598320 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37800bd60 con 0x7fb390159c60 2023-10-10T20:40:56.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.864+0000 7fb394c91700 1 -- 172.21.15.89:0/1609598320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb390159c60 msgr2=0x7fb39015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:56.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.864+0000 7fb394c91700 1 --2- 172.21.15.89:0/1609598320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb390159c60 0x7fb39015d5d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb390089b50 tx=0x7fb378009250 comp rx=0 tx=0).stop 2023-10-10T20:40:56.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.864+0000 7fb394c91700 1 -- 172.21.15.89:0/1609598320 shutdown_connections 2023-10-10T20:40:56.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.864+0000 7fb394c91700 1 --2- 172.21.15.89:0/1609598320 >> v2:172.21.15.111:3300/0 conn(0x7fb39015db10 0x7fb390069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.864+0000 7fb394c91700 1 --2- 172.21.15.89:0/1609598320 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb390159890 0x7fb39015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.864+0000 7fb394c91700 1 --2- 172.21.15.89:0/1609598320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb390159c60 0x7fb39015d5d0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.864+0000 7fb394c91700 1 -- 172.21.15.89:0/1609598320 >> 172.21.15.89:0/1609598320 conn(0x7fb390068e20 msgr2=0x7fb3900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:56.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.864+0000 7fb394c91700 1 -- 172.21.15.89:0/1609598320 shutdown_connections 2023-10-10T20:40:56.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.864+0000 7fb394c91700 1 -- 172.21.15.89:0/1609598320 wait complete. 2023-10-10T20:40:56.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.865+0000 7fb394c91700 1 Processor -- start 2023-10-10T20:40:56.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.865+0000 7fb394c91700 1 -- start start 2023-10-10T20:40:56.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.866+0000 7fb394c91700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb390159890 0x7fb3900b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:56.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.866+0000 7fb394c91700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb390159c60 0x7fb3901585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:56.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.866+0000 7fb394c91700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb39015db10 0x7fb390158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:56.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.866+0000 7fb394c91700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39014dc40 con 0x7fb390159890 2023-10-10T20:40:56.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.866+0000 7fb394c91700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900b60d0 con 0x7fb39015db10 2023-10-10T20:40:56.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.866+0000 7fb394c91700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39016a8b0 con 0x7fb390159c60 2023-10-10T20:40:56.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.866+0000 7fb38e59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb390159890 0x7fb3900b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:56.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.866+0000 7fb38e59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb390159890 0x7fb3900b5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60026/0 (socket says 172.21.15.89:60026) 2023-10-10T20:40:56.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.866+0000 7fb38e59c700 1 -- 172.21.15.89:0/4029808155 learned_addr learned my addr 172.21.15.89:0/4029808155 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:56.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.866+0000 7fb38ed9d700 1 --2- 172.21.15.89:0/4029808155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb39015db10 0x7fb390158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:56.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.866+0000 7fb38dd9b700 1 --2- 172.21.15.89:0/4029808155 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb390159c60 0x7fb3901585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:56.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.866+0000 7fb38e59c700 1 -- 172.21.15.89:0/4029808155 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb390159c60 msgr2=0x7fb3901585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:56.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.866+0000 7fb38e59c700 1 --2- 172.21.15.89:0/4029808155 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb390159c60 0x7fb3901585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.866+0000 7fb38e59c700 1 -- 172.21.15.89:0/4029808155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb39015db10 msgr2=0x7fb390158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:56.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.866+0000 7fb38e59c700 1 --2- 172.21.15.89:0/4029808155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb39015db10 0x7fb390158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:56.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.866+0000 7fb38e59c700 1 -- 172.21.15.89:0/4029808155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3780094f0 con 0x7fb390159890 2023-10-10T20:40:56.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.866+0000 7fb38e59c700 1 --2- 172.21.15.89:0/4029808155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb390159890 0x7fb3900b5b90 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb38800d1d0 tx=0x7fb388009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:56.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.866+0000 7fb3867fc700 1 -- 172.21.15.89:0/4029808155 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38800dcb0 con 0x7fb390159890 2023-10-10T20:40:56.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.867+0000 7fb3867fc700 1 -- 172.21.15.89:0/4029808155 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38800f040 con 0x7fb390159890 2023-10-10T20:40:56.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.867+0000 7fb394c91700 1 -- 172.21.15.89:0/4029808155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb390169f80 con 0x7fb390159890 2023-10-10T20:40:56.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.867+0000 7fb3867fc700 1 -- 172.21.15.89:0/4029808155 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388015670 con 0x7fb390159890 2023-10-10T20:40:56.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.867+0000 7fb394c91700 1 -- 172.21.15.89:0/4029808155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39016a540 con 0x7fb390159890 2023-10-10T20:40:56.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.867+0000 7fb394c91700 1 -- 172.21.15.89:0/4029808155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb39016ab90 con 0x7fb390159890 2023-10-10T20:40:56.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.868+0000 7fb3867fc700 1 -- 172.21.15.89:0/4029808155 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb388015a60 con 0x7fb390159890 2023-10-10T20:40:56.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.868+0000 7fb3867fc700 1 --2- 172.21.15.89:0/4029808155 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb370067320 0x7fb3700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:56.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.868+0000 7fb3867fc700 1 -- 172.21.15.89:0/4029808155 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb3880850c0 con 0x7fb390159890 2023-10-10T20:40:56.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.868+0000 7fb38dd9b700 1 --2- 172.21.15.89:0/4029808155 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb370067320 0x7fb3700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:56.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.869+0000 7fb38dd9b700 1 --2- 172.21.15.89:0/4029808155 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb370067320 0x7fb3700697d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb390089b50 tx=0x7fb378017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:56.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:56.873+0000 7fb3867fc700 1 -- 172.21.15.89:0/4029808155 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb38804e5a0 con 0x7fb390159890 2023-10-10T20:40:57.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.099+0000 7fb394c91700 1 -- 172.21.15.89:0/4029808155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb39004c3c0 con 0x7fb390159890 2023-10-10T20:40:57.102 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-10-10T20:40:57.100+0000 7fd0184ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:40:57.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.104+0000 7fb3867fc700 1 -- 172.21.15.89:0/4029808155 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb388055780 con 0x7fb390159890 2023-10-10T20:40:57.107 INFO:tasks.ceph.mds.a.smithi089.stderr: -889> 2023-10-10T20:40:43.555+0000 7fb1d1f81700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:40:57.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.107+0000 7fb394c91700 1 -- 172.21.15.89:0/4029808155 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb370067320 msgr2=0x7fb3700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:57.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.107+0000 7fb394c91700 1 --2- 172.21.15.89:0/4029808155 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb370067320 0x7fb3700697d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb390089b50 tx=0x7fb378017000 comp rx=0 tx=0).stop 2023-10-10T20:40:57.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.107+0000 7fb394c91700 1 -- 172.21.15.89:0/4029808155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb390159890 msgr2=0x7fb3900b5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:57.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.107+0000 7fb394c91700 1 --2- 172.21.15.89:0/4029808155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb390159890 0x7fb3900b5b90 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb38800d1d0 tx=0x7fb388009a90 comp rx=0 tx=0).stop 2023-10-10T20:40:57.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.107+0000 7fb394c91700 1 -- 172.21.15.89:0/4029808155 shutdown_connections 2023-10-10T20:40:57.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.107+0000 7fb394c91700 1 --2- 172.21.15.89:0/4029808155 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb370067320 0x7fb3700697d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:57.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.107+0000 7fb394c91700 1 --2- 172.21.15.89:0/4029808155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb39015db10 0x7fb390158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:57.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.107+0000 7fb394c91700 1 --2- 172.21.15.89:0/4029808155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb390159890 0x7fb3900b5b90 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:57.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.107+0000 7fb394c91700 1 --2- 172.21.15.89:0/4029808155 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb390159c60 0x7fb3901585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:57.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.107+0000 7fb394c91700 1 -- 172.21.15.89:0/4029808155 >> 172.21.15.89:0/4029808155 conn(0x7fb390068e20 msgr2=0x7fb390154920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:57.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.107+0000 7fb394c91700 1 -- 172.21.15.89:0/4029808155 shutdown_connections 2023-10-10T20:40:57.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.107+0000 7fb394c91700 1 -- 172.21.15.89:0/4029808155 wait complete. 2023-10-10T20:40:57.112 INFO:teuthology.orchestra.run.smithi089.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:40:57.127 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:40:57.128 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:40:57.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.243+0000 7fb4691c3700 1 Processor -- start 2023-10-10T20:40:57.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.243+0000 7fb4691c3700 1 -- start start 2023-10-10T20:40:57.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.244+0000 7fb4691c3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb464159c60 0x7fb46415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:57.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.244+0000 7fb4691c3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb46415a160 con 0x7fb464159890 2023-10-10T20:40:57.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.244+0000 7fb4691c3700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4640d13a0 con 0x7fb46415db10 2023-10-10T20:40:57.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.244+0000 7fb4691c3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46415e140 con 0x7fb464159c60 2023-10-10T20:40:57.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.244+0000 7fb462d9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb464159890 0x7fb46415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34188/0 (socket says 172.21.15.89:34188) 2023-10-10T20:40:57.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.244+0000 7fb462d9d700 1 -- 172.21.15.89:0/655591441 learned_addr learned my addr 172.21.15.89:0/655591441 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:57.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.244+0000 7fb45a59c700 1 --2- 172.21.15.89:0/655591441 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb464159c60 0x7fb46415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:57.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.244+0000 7fb46259c700 1 -- 172.21.15.89:0/655591441 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843217353 0 0) 0x7fb46415a160 con 0x7fb464159890 2023-10-10T20:40:57.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.244+0000 7fb46259c700 1 -- 172.21.15.89:0/655591441 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb450003570 con 0x7fb464159890 2023-10-10T20:40:57.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.244+0000 7fb46259c700 1 -- 172.21.15.89:0/655591441 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3312274810 0 0) 0x7fb450003570 con 0x7fb464159890 2023-10-10T20:40:57.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.244+0000 7fb46259c700 1 -- 172.21.15.89:0/655591441 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb464159c60 msgr2=0x7fb46415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:57.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.244+0000 7fb46259c700 1 --2- 172.21.15.89:0/655591441 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb464159c60 0x7fb46415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:57.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.244+0000 7fb46259c700 1 -- 172.21.15.89:0/655591441 >> v1:172.21.15.111:6789/0 conn(0x7fb46415db10 legacy=0x7fb464069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:57.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.244+0000 7fb46259c700 1 -- 172.21.15.89:0/655591441 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46415a3d0 con 0x7fb464159890 2023-10-10T20:40:57.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.244+0000 7fb46259c700 1 -- 172.21.15.89:0/655591441 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fb454001470 con 0x7fb464159890 2023-10-10T20:40:57.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.244+0000 7fb46259c700 1 -- 172.21.15.89:0/655591441 >> v1:172.21.15.89:6789/0 conn(0x7fb464159890 legacy=0x7fb46415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:57.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.244+0000 7fb46259c700 1 --2- 172.21.15.89:0/655591441 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb464159c60 0x7fb450003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:57.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.244+0000 7fb46259c700 1 --2- 172.21.15.89:0/655591441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb450003fd0 0x7fb450004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:57.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.245+0000 7fb46259c700 1 --2- 172.21.15.89:0/655591441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb450004d50 0x7fb4500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:57.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.245+0000 7fb46259c700 1 -- 172.21.15.89:0/655591441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46415e140 con 0x7fb450004d50 2023-10-10T20:40:57.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.245+0000 7fb46259c700 1 -- 172.21.15.89:0/655591441 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45000bb40 con 0x7fb450003fd0 2023-10-10T20:40:57.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.245+0000 7fb46259c700 1 -- 172.21.15.89:0/655591441 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45000bd00 con 0x7fb464159c60 2023-10-10T20:40:57.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.245+0000 7fb46359e700 1 --2- 172.21.15.89:0/655591441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb450004d50 0x7fb4500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:57.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.245+0000 7fb45a59c700 1 --2- 172.21.15.89:0/655591441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb450003fd0 0x7fb450004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:57.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.245+0000 7fb462d9d700 1 --2- 172.21.15.89:0/655591441 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb464159c60 0x7fb450003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:57.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.245+0000 7fb46359e700 1 -- 172.21.15.89:0/655591441 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb464159c60 msgr2=0x7fb450003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:57.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.245+0000 7fb46359e700 1 --2- 172.21.15.89:0/655591441 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb464159c60 0x7fb450003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:57.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.245+0000 7fb46359e700 1 -- 172.21.15.89:0/655591441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb450003fd0 msgr2=0x7fb450004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:57.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.245+0000 7fb46359e700 1 --2- 172.21.15.89:0/655591441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb450003fd0 0x7fb450004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:57.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.245+0000 7fb46359e700 1 -- 172.21.15.89:0/655591441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb450003570 con 0x7fb450004d50 2023-10-10T20:40:57.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.245+0000 7fb46359e700 1 --2- 172.21.15.89:0/655591441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb450004d50 0x7fb4500051c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0020a0 tx=0x7fb45c0016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c9252b382d8058 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:57.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.245+0000 7fb46259c700 1 -- 172.21.15.89:0/655591441 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0019e0 con 0x7fb450004d50 2023-10-10T20:40:57.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.245+0000 7fb46259c700 1 -- 172.21.15.89:0/655591441 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c00c8b0 con 0x7fb450004d50 2023-10-10T20:40:57.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.245+0000 7fb46259c700 1 -- 172.21.15.89:0/655591441 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c00cc50 con 0x7fb450004d50 2023-10-10T20:40:57.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.246+0000 7fb4691c3700 1 -- 172.21.15.89:0/655591441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb450004d50 msgr2=0x7fb4500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:57.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.246+0000 7fb4691c3700 1 --2- 172.21.15.89:0/655591441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb450004d50 0x7fb4500051c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0020a0 tx=0x7fb45c0016d0 comp rx=0 tx=0).stop 2023-10-10T20:40:57.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.246+0000 7fb46359e700 1 -- 172.21.15.89:0/655591441 reap_dead start 2023-10-10T20:40:57.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.246+0000 7fb4691c3700 1 -- 172.21.15.89:0/655591441 shutdown_connections 2023-10-10T20:40:57.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.246+0000 7fb4691c3700 1 -- 172.21.15.89:0/655591441 >> 172.21.15.89:0/655591441 conn(0x7fb464068e20 msgr2=0x7fb4640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:57.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.246+0000 7fb4691c3700 1 -- 172.21.15.89:0/655591441 shutdown_connections 2023-10-10T20:40:57.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.246+0000 7fb4691c3700 1 -- 172.21.15.89:0/655591441 wait complete. 2023-10-10T20:40:57.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.247+0000 7fb4691c3700 1 Processor -- start 2023-10-10T20:40:57.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.247+0000 7fb4691c3700 1 -- start start 2023-10-10T20:40:57.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb4691c3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb464142230 0x7fb464151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:57.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb4691c3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb464154f40 0x7fb46415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:57.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb4691c3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb46415fae0 0x7fb46414ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:57.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb4691c3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46415ff50 con 0x7fb464154f40 2023-10-10T20:40:57.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb4691c3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4640b1680 con 0x7fb464142230 2023-10-10T20:40:57.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb4691c3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4640b1070 con 0x7fb46415fae0 2023-10-10T20:40:57.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb45a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb464154f40 0x7fb46415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:57.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb45a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb464154f40 0x7fb46415f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60058/0 (socket says 172.21.15.89:60058) 2023-10-10T20:40:57.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb45a59c700 1 -- 172.21.15.89:0/3063678791 learned_addr learned my addr 172.21.15.89:0/3063678791 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:57.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb462d9d700 1 --2- 172.21.15.89:0/3063678791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb464142230 0x7fb464151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:57.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb46359e700 1 --2- 172.21.15.89:0/3063678791 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb46415fae0 0x7fb46414ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:57.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb45a59c700 1 -- 172.21.15.89:0/3063678791 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb46415fae0 msgr2=0x7fb46414ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:57.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb45a59c700 1 --2- 172.21.15.89:0/3063678791 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb46415fae0 0x7fb46414ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:57.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb45a59c700 1 -- 172.21.15.89:0/3063678791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb464142230 msgr2=0x7fb464151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:57.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb45a59c700 1 --2- 172.21.15.89:0/3063678791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb464142230 0x7fb464151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:57.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb45a59c700 1 -- 172.21.15.89:0/3063678791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4641554d0 con 0x7fb464154f40 2023-10-10T20:40:57.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb45a59c700 1 --2- 172.21.15.89:0/3063678791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb464154f40 0x7fb46415f5a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fb44c011b40 tx=0x7fb44c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:57.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb45b7fe700 1 -- 172.21.15.89:0/3063678791 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c00f940 con 0x7fb464154f40 2023-10-10T20:40:57.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb45b7fe700 1 -- 172.21.15.89:0/3063678791 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44c00faa0 con 0x7fb464154f40 2023-10-10T20:40:57.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb4691c3700 1 -- 172.21.15.89:0/3063678791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4640b1310 con 0x7fb464154f40 2023-10-10T20:40:57.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb45b7fe700 1 -- 172.21.15.89:0/3063678791 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c00fe20 con 0x7fb464154f40 2023-10-10T20:40:57.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.248+0000 7fb4691c3700 1 -- 172.21.15.89:0/3063678791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb464151d70 con 0x7fb464154f40 2023-10-10T20:40:57.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.249+0000 7fb4691c3700 1 -- 172.21.15.89:0/3063678791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46415d950 con 0x7fb464154f40 2023-10-10T20:40:57.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.250+0000 7fb45b7fe700 1 -- 172.21.15.89:0/3063678791 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb44c014650 con 0x7fb464154f40 2023-10-10T20:40:57.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.250+0000 7fb45b7fe700 1 --2- 172.21.15.89:0/3063678791 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb444067320 0x7fb4440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:57.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.250+0000 7fb462d9d700 1 --2- 172.21.15.89:0/3063678791 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb444067320 0x7fb4440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:57.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.250+0000 7fb45b7fe700 1 -- 172.21.15.89:0/3063678791 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fb44c018030 con 0x7fb464154f40 2023-10-10T20:40:57.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.251+0000 7fb462d9d700 1 --2- 172.21.15.89:0/3063678791 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb444067320 0x7fb4440697d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb454007f10 tx=0x7fb454001e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:57.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.255+0000 7fb45b7fe700 1 -- 172.21.15.89:0/3063678791 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb44c052280 con 0x7fb464154f40 2023-10-10T20:40:57.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:57.478+0000 7fb4691c3700 1 -- 172.21.15.89:0/3063678791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb46404c3c0 con 0x7fb464154f40 2023-10-10T20:40:58.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.116+0000 7fb45b7fe700 1 -- 172.21.15.89:0/3063678791 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb44c0558a0 con 0x7fb464154f40 2023-10-10T20:40:58.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.118+0000 7fb4691c3700 1 -- 172.21.15.89:0/3063678791 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb444067320 msgr2=0x7fb4440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:58.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.118+0000 7fb4691c3700 1 --2- 172.21.15.89:0/3063678791 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb444067320 0x7fb4440697d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb454007f10 tx=0x7fb454001e70 comp rx=0 tx=0).stop 2023-10-10T20:40:58.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.118+0000 7fb4691c3700 1 -- 172.21.15.89:0/3063678791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb464154f40 msgr2=0x7fb46415f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:58.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.118+0000 7fb4691c3700 1 --2- 172.21.15.89:0/3063678791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb464154f40 0x7fb46415f5a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fb44c011b40 tx=0x7fb44c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:58.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.118+0000 7fb4691c3700 1 -- 172.21.15.89:0/3063678791 shutdown_connections 2023-10-10T20:40:58.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.118+0000 7fb4691c3700 1 --2- 172.21.15.89:0/3063678791 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb46415fae0 0x7fb46414ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:58.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.118+0000 7fb4691c3700 1 --2- 172.21.15.89:0/3063678791 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb444067320 0x7fb4440697d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:58.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.118+0000 7fb4691c3700 1 --2- 172.21.15.89:0/3063678791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb464142230 0x7fb464151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:58.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.118+0000 7fb4691c3700 1 --2- 172.21.15.89:0/3063678791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb464154f40 0x7fb46415f5a0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:58.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.118+0000 7fb4691c3700 1 -- 172.21.15.89:0/3063678791 >> 172.21.15.89:0/3063678791 conn(0x7fb464068e20 msgr2=0x7fb464157320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:58.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.118+0000 7fb4691c3700 1 -- 172.21.15.89:0/3063678791 shutdown_connections 2023-10-10T20:40:58.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.118+0000 7fb4691c3700 1 -- 172.21.15.89:0/3063678791 wait complete. 2023-10-10T20:40:58.134 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:40:58.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.253+0000 7f3ecc133700 1 Processor -- start 2023-10-10T20:40:58.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.254+0000 7f3ecc133700 1 -- start start 2023-10-10T20:40:58.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.254+0000 7f3ecc133700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3ec40ceea0 0x7f3ec40b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:58.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.254+0000 7f3ecc133700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ec40b5f80 con 0x7f3ec40cd0c0 2023-10-10T20:40:58.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.254+0000 7f3ecc133700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ec40b4490 con 0x7f3ec40cbeb0 2023-10-10T20:40:58.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.254+0000 7f3ecc133700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ec40cba90 con 0x7f3ec40ceea0 2023-10-10T20:40:58.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.254+0000 7f3ec96ce700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3ec40ceea0 0x7f3ec40b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:58.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.254+0000 7f3ec9ecf700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3ec40cd0c0 0x7f3ec40cd490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45112/0 (socket says 172.21.15.89:45112) 2023-10-10T20:40:58.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.254+0000 7f3ec9ecf700 1 -- 172.21.15.89:0/2641384714 learned_addr learned my addr 172.21.15.89:0/2641384714 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:58.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.255+0000 7f3ec96ce700 1 -- 172.21.15.89:0/2641384714 >> v1:172.21.15.111:6789/0 conn(0x7f3ec40cbeb0 legacy=0x7f3ec40cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:58.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.255+0000 7f3ec96ce700 1 -- 172.21.15.89:0/2641384714 >> v1:172.21.15.89:6789/0 conn(0x7f3ec40cd0c0 legacy=0x7f3ec40cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:58.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.255+0000 7f3ec96ce700 1 -- 172.21.15.89:0/2641384714 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec40cb6f0 con 0x7f3ec40ceea0 2023-10-10T20:40:58.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.255+0000 7f3ec8ecd700 1 -- 172.21.15.89:0/2641384714 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2996449099 0 0) 0x7f3ec40b5f80 con 0x7f3ec40cd0c0 2023-10-10T20:40:58.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.255+0000 7f3ec96ce700 1 --2- 172.21.15.89:0/2641384714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec40ceea0 0x7f3ec40b3f50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40098e0 tx=0x7f3eb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d42e20b8f028fac server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:58.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.255+0000 7f3ec8ecd700 1 -- 172.21.15.89:0/2641384714 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4009e80 con 0x7f3ec40ceea0 2023-10-10T20:40:58.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.255+0000 7f3ec8ecd700 1 -- 172.21.15.89:0/2641384714 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb400b820 con 0x7f3ec40ceea0 2023-10-10T20:40:58.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.255+0000 7f3ec8ecd700 1 -- 172.21.15.89:0/2641384714 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb400bba0 con 0x7f3ec40ceea0 2023-10-10T20:40:58.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.256+0000 7f3ecc133700 1 -- 172.21.15.89:0/2641384714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec40ceea0 msgr2=0x7f3ec40b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:58.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.256+0000 7f3ecc133700 1 --2- 172.21.15.89:0/2641384714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec40ceea0 0x7f3ec40b3f50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40098e0 tx=0x7f3eb40091a0 comp rx=0 tx=0).stop 2023-10-10T20:40:58.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.256+0000 7f3ecc133700 1 -- 172.21.15.89:0/2641384714 shutdown_connections 2023-10-10T20:40:58.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.256+0000 7f3ecc133700 1 --2- 172.21.15.89:0/2641384714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec40ceea0 0x7f3ec40b3f50 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:58.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.256+0000 7f3ecc133700 1 -- 172.21.15.89:0/2641384714 >> 172.21.15.89:0/2641384714 conn(0x7f3ec40999f0 msgr2=0x7f3ec40b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:58.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.256+0000 7f3ecc133700 1 -- 172.21.15.89:0/2641384714 shutdown_connections 2023-10-10T20:40:58.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.256+0000 7f3ecc133700 1 -- 172.21.15.89:0/2641384714 wait complete. 2023-10-10T20:40:58.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.257+0000 7f3ecc133700 1 Processor -- start 2023-10-10T20:40:58.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.257+0000 7f3ecc133700 1 -- start start 2023-10-10T20:40:58.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.257+0000 7f3ecc133700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec40cbeb0 0x7f3ec414f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:58.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.257+0000 7f3ecc133700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ec40cd0c0 0x7f3ec406df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:58.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.257+0000 7f3ecc133700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ec40ceea0 0x7f3ec406e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:58.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.257+0000 7f3ecc133700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec406c180 con 0x7f3ec40cbeb0 2023-10-10T20:40:58.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.258+0000 7f3ecc133700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec406b850 con 0x7f3ec40ceea0 2023-10-10T20:40:58.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.258+0000 7f3ecc133700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec406bb30 con 0x7f3ec40cd0c0 2023-10-10T20:40:58.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.258+0000 7f3ec9ecf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec40cbeb0 0x7f3ec414f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:58.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.258+0000 7f3ec9ecf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec40cbeb0 0x7f3ec414f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54344/0 (socket says 172.21.15.89:54344) 2023-10-10T20:40:58.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.258+0000 7f3ec9ecf700 1 -- 172.21.15.89:0/2854808425 learned_addr learned my addr 172.21.15.89:0/2854808425 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:58.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.258+0000 7f3ec96ce700 1 --2- 172.21.15.89:0/2854808425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ec40cd0c0 0x7f3ec406df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:58.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.258+0000 7f3eca6d0700 1 --2- 172.21.15.89:0/2854808425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ec40ceea0 0x7f3ec406e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:58.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.258+0000 7f3ec9ecf700 1 -- 172.21.15.89:0/2854808425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ec40cd0c0 msgr2=0x7f3ec406df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:58.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.258+0000 7f3ec9ecf700 1 --2- 172.21.15.89:0/2854808425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ec40cd0c0 0x7f3ec406df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:58.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.258+0000 7f3ec9ecf700 1 -- 172.21.15.89:0/2854808425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ec40ceea0 msgr2=0x7f3ec406e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:58.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.258+0000 7f3ec9ecf700 1 --2- 172.21.15.89:0/2854808425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ec40ceea0 0x7f3ec406e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:58.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.258+0000 7f3ec9ecf700 1 -- 172.21.15.89:0/2854808425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb4009420 con 0x7f3ec40cbeb0 2023-10-10T20:40:58.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.258+0000 7f3eca6d0700 1 --2- 172.21.15.89:0/2854808425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ec40ceea0 0x7f3ec406e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:58.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.258+0000 7f3ec9ecf700 1 --2- 172.21.15.89:0/2854808425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec40cbeb0 0x7f3ec414f970 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000c7e0 tx=0x7f3ec0007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:58.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.258+0000 7f3eb9ffb700 1 -- 172.21.15.89:0/2854808425 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0007300 con 0x7f3ec40cbeb0 2023-10-10T20:40:58.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.258+0000 7f3eb9ffb700 1 -- 172.21.15.89:0/2854808425 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec0007460 con 0x7f3ec40cbeb0 2023-10-10T20:40:58.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.258+0000 7f3ecc133700 1 -- 172.21.15.89:0/2854808425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec406be10 con 0x7f3ec40cbeb0 2023-10-10T20:40:58.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.258+0000 7f3eb9ffb700 1 -- 172.21.15.89:0/2854808425 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec00057f0 con 0x7f3ec40cbeb0 2023-10-10T20:40:58.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.258+0000 7f3ecc133700 1 -- 172.21.15.89:0/2854808425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec414feb0 con 0x7f3ec40cbeb0 2023-10-10T20:40:58.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.259+0000 7f3ecc133700 1 -- 172.21.15.89:0/2854808425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec4063c90 con 0x7f3ec40cbeb0 2023-10-10T20:40:58.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.260+0000 7f3eb9ffb700 1 -- 172.21.15.89:0/2854808425 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3ec0005c00 con 0x7f3ec40cbeb0 2023-10-10T20:40:58.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.260+0000 7f3eb9ffb700 1 --2- 172.21.15.89:0/2854808425 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3ea8067320 0x7f3ea80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:58.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.260+0000 7f3ec96ce700 1 --2- 172.21.15.89:0/2854808425 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3ea8067320 0x7f3ea80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:58.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.260+0000 7f3eb9ffb700 1 -- 172.21.15.89:0/2854808425 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f3ec0084780 con 0x7f3ec40cbeb0 2023-10-10T20:40:58.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.260+0000 7f3ec96ce700 1 --2- 172.21.15.89:0/2854808425 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3ea8067320 0x7f3ea80697d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40030c0 tx=0x7f3eb4016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:58.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.264+0000 7f3eb9ffb700 1 -- 172.21.15.89:0/2854808425 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ec00514c0 con 0x7f3ec40cbeb0 2023-10-10T20:40:58.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:58.451+0000 7f3ecc133700 1 -- 172.21.15.89:0/2854808425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3ec40718b0 con 0x7f3ec40cbeb0 2023-10-10T20:40:59.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.119+0000 7f3eb9ffb700 1 -- 172.21.15.89:0/2854808425 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3ec0054ae0 con 0x7f3ec40cbeb0 2023-10-10T20:40:59.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.123+0000 7f3ecc133700 1 -- 172.21.15.89:0/2854808425 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3ea8067320 msgr2=0x7f3ea80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:59.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.123+0000 7f3ecc133700 1 --2- 172.21.15.89:0/2854808425 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3ea8067320 0x7f3ea80697d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40030c0 tx=0x7f3eb4016000 comp rx=0 tx=0).stop 2023-10-10T20:40:59.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.123+0000 7f3ecc133700 1 -- 172.21.15.89:0/2854808425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec40cbeb0 msgr2=0x7f3ec414f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:59.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.123+0000 7f3ecc133700 1 --2- 172.21.15.89:0/2854808425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec40cbeb0 0x7f3ec414f970 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000c7e0 tx=0x7f3ec0007c80 comp rx=0 tx=0).stop 2023-10-10T20:40:59.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.123+0000 7f3ecc133700 1 -- 172.21.15.89:0/2854808425 shutdown_connections 2023-10-10T20:40:59.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.123+0000 7f3ecc133700 1 --2- 172.21.15.89:0/2854808425 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3ea8067320 0x7f3ea80697d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:59.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.123+0000 7f3ecc133700 1 --2- 172.21.15.89:0/2854808425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ec40ceea0 0x7f3ec406e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:59.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.123+0000 7f3ecc133700 1 --2- 172.21.15.89:0/2854808425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec40cbeb0 0x7f3ec414f970 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:59.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.123+0000 7f3ecc133700 1 --2- 172.21.15.89:0/2854808425 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ec40cd0c0 0x7f3ec406df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:59.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.123+0000 7f3ecc133700 1 -- 172.21.15.89:0/2854808425 >> 172.21.15.89:0/2854808425 conn(0x7f3ec40999f0 msgr2=0x7f3ec406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:59.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.123+0000 7f3ecc133700 1 -- 172.21.15.89:0/2854808425 shutdown_connections 2023-10-10T20:40:59.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.123+0000 7f3ecc133700 1 -- 172.21.15.89:0/2854808425 wait complete. 2023-10-10T20:40:59.128 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:40:59.138 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:40:59.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.260+0000 7f3ea578a700 1 Processor -- start 2023-10-10T20:40:59.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.260+0000 7f3ea578a700 1 -- start start 2023-10-10T20:40:59.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.261+0000 7f3ea578a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ea00c8490 0x7f3ea00c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:59.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.261+0000 7f3ea578a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3ea0069e20 0x7f3ea01569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:59.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.261+0000 7f3ea578a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea00ac030 con 0x7f3ea0069a50 2023-10-10T20:40:59.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.261+0000 7f3ea578a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea00ab820 con 0x7f3ea0069e20 2023-10-10T20:40:59.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.261+0000 7f3ea578a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea006dbb0 con 0x7f3ea00c8490 2023-10-10T20:40:59.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.261+0000 7f3e9effd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3ea0069a50 0x7f3ea015a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45126/0 (socket says 172.21.15.89:45126) 2023-10-10T20:40:59.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.261+0000 7f3e9effd700 1 -- 172.21.15.89:0/3313437535 learned_addr learned my addr 172.21.15.89:0/3313437535 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:40:59.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.261+0000 7f3e967fc700 1 --2- 172.21.15.89:0/3313437535 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ea00c8490 0x7f3ea00c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:59.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.261+0000 7f3e9f7fe700 1 --2- 172.21.15.89:0/3313437535 >> v2:172.21.15.111:3300/0 conn(0x7f3ea0069e20 0x7f3ea01569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:59.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9e7fc700 1 -- 172.21.15.89:0/3313437535 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2390183400 0 0) 0x7f3e90001110 con 0x7f3ea0069a50 2023-10-10T20:40:59.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9e7fc700 1 -- 172.21.15.89:0/3313437535 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e8c003570 con 0x7f3ea0069a50 2023-10-10T20:40:59.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9e7fc700 1 -- 172.21.15.89:0/3313437535 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 508189067 0 0) 0x7f3e8c003570 con 0x7f3ea0069a50 2023-10-10T20:40:59.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9e7fc700 1 -- 172.21.15.89:0/3313437535 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ea00c8490 msgr2=0x7f3ea00c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:59.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9e7fc700 1 --2- 172.21.15.89:0/3313437535 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ea00c8490 0x7f3ea00c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:59.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9e7fc700 1 -- 172.21.15.89:0/3313437535 >> v2:172.21.15.111:3300/0 conn(0x7f3ea0069e20 msgr2=0x7f3ea01569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:59.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9e7fc700 1 --2- 172.21.15.89:0/3313437535 >> v2:172.21.15.111:3300/0 conn(0x7f3ea0069e20 0x7f3ea01569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:59.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9e7fc700 1 -- 172.21.15.89:0/3313437535 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea00aba70 con 0x7f3ea0069a50 2023-10-10T20:40:59.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9e7fc700 1 -- 172.21.15.89:0/3313437535 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f3e90001600 con 0x7f3ea0069a50 2023-10-10T20:40:59.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9e7fc700 1 -- 172.21.15.89:0/3313437535 >> v1:172.21.15.89:6789/0 conn(0x7f3ea0069a50 legacy=0x7f3ea015a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:59.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9e7fc700 1 --2- 172.21.15.89:0/3313437535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e8c003a60 0x7f3e8c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:59.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9e7fc700 1 --2- 172.21.15.89:0/3313437535 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ea00c8490 0x7f3e8c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:59.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9e7fc700 1 --2- 172.21.15.89:0/3313437535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e8c004d50 0x7f3e8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:59.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9e7fc700 1 -- 172.21.15.89:0/3313437535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00ab820 con 0x7f3e8c003a60 2023-10-10T20:40:59.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9e7fc700 1 -- 172.21.15.89:0/3313437535 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea006dbb0 con 0x7f3e8c004d50 2023-10-10T20:40:59.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9e7fc700 1 -- 172.21.15.89:0/3313437535 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c00cb70 con 0x7f3ea00c8490 2023-10-10T20:40:59.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9effd700 1 --2- 172.21.15.89:0/3313437535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e8c003a60 0x7f3e8c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:59.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9f7fe700 1 --2- 172.21.15.89:0/3313437535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e8c004d50 0x7f3e8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:59.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e967fc700 1 --2- 172.21.15.89:0/3313437535 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ea00c8490 0x7f3e8c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:59.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9effd700 1 -- 172.21.15.89:0/3313437535 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ea00c8490 msgr2=0x7f3e8c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:59.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9effd700 1 --2- 172.21.15.89:0/3313437535 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ea00c8490 0x7f3e8c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:59.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9effd700 1 -- 172.21.15.89:0/3313437535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e8c004d50 msgr2=0x7f3e8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:40:59.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9effd700 1 --2- 172.21.15.89:0/3313437535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e8c004d50 0x7f3e8c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:59.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9effd700 1 -- 172.21.15.89:0/3313437535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c003570 con 0x7f3e8c003a60 2023-10-10T20:40:59.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.262+0000 7f3e9effd700 1 --2- 172.21.15.89:0/3313437535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e8c003a60 0x7f3e8c003e70 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000c5f0 tx=0x7f3ea00ab960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593175489751628b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:59.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.263+0000 7f3e9e7fc700 1 -- 172.21.15.89:0/3313437535 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9000cd90 con 0x7f3e8c003a60 2023-10-10T20:40:59.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.263+0000 7f3e9e7fc700 1 -- 172.21.15.89:0/3313437535 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e90005920 con 0x7f3e8c003a60 2023-10-10T20:40:59.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.263+0000 7f3e9e7fc700 1 -- 172.21.15.89:0/3313437535 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90005ce0 con 0x7f3e8c003a60 2023-10-10T20:40:59.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.263+0000 7f3ea578a700 1 -- 172.21.15.89:0/3313437535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e8c003a60 msgr2=0x7f3e8c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:59.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.263+0000 7f3ea578a700 1 --2- 172.21.15.89:0/3313437535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e8c003a60 0x7f3e8c003e70 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000c5f0 tx=0x7f3ea00ab960 comp rx=0 tx=0).stop 2023-10-10T20:40:59.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.263+0000 7f3e9f7fe700 1 -- 172.21.15.89:0/3313437535 reap_dead start 2023-10-10T20:40:59.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.263+0000 7f3ea578a700 1 -- 172.21.15.89:0/3313437535 shutdown_connections 2023-10-10T20:40:59.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.263+0000 7f3ea578a700 1 -- 172.21.15.89:0/3313437535 >> 172.21.15.89:0/3313437535 conn(0x7f3ea00c8f90 msgr2=0x7f3ea00c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:40:59.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.263+0000 7f3ea578a700 1 -- 172.21.15.89:0/3313437535 shutdown_connections 2023-10-10T20:40:59.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.263+0000 7f3ea578a700 1 -- 172.21.15.89:0/3313437535 wait complete. 2023-10-10T20:40:59.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.264+0000 7f3ea578a700 1 Processor -- start 2023-10-10T20:40:59.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.264+0000 7f3ea578a700 1 -- start start 2023-10-10T20:40:59.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.265+0000 7f3ea578a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e8c003a60 0x7f3ea00bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:59.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.265+0000 7f3ea578a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ea00ad440 0x7f3ea00c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:59.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.265+0000 7f3ea578a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ea0146810 0x7f3ea0146c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:59.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.265+0000 7f3ea578a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00adc10 con 0x7f3e8c003a60 2023-10-10T20:40:59.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.265+0000 7f3ea578a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00c14a0 con 0x7f3ea0146810 2023-10-10T20:40:59.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.265+0000 7f3ea578a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00c0010 con 0x7f3ea00ad440 2023-10-10T20:40:59.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.265+0000 7f3e9effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e8c003a60 0x7f3ea00bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:59.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.265+0000 7f3e9effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e8c003a60 0x7f3ea00bfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54362/0 (socket says 172.21.15.89:54362) 2023-10-10T20:40:59.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.265+0000 7f3e9effd700 1 -- 172.21.15.89:0/1445599571 learned_addr learned my addr 172.21.15.89:0/1445599571 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:40:59.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.265+0000 7f3e9effd700 1 -- 172.21.15.89:0/1445599571 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ea00ad440 msgr2=0x7f3ea00c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:59.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.265+0000 7f3e967fc700 1 --2- 172.21.15.89:0/1445599571 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ea00ad440 0x7f3ea00c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:59.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.265+0000 7f3e9f7fe700 1 --2- 172.21.15.89:0/1445599571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ea0146810 0x7f3ea0146c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:59.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.265+0000 7f3e9effd700 1 --2- 172.21.15.89:0/1445599571 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ea00ad440 0x7f3ea00c5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:59.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.265+0000 7f3e9effd700 1 -- 172.21.15.89:0/1445599571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ea0146810 msgr2=0x7f3ea0146c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:40:59.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.265+0000 7f3e9effd700 1 --2- 172.21.15.89:0/1445599571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ea0146810 0x7f3ea0146c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:40:59.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.265+0000 7f3e9effd700 1 -- 172.21.15.89:0/1445599571 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea01471b0 con 0x7f3e8c003a60 2023-10-10T20:40:59.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.265+0000 7f3e9f7fe700 1 --2- 172.21.15.89:0/1445599571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ea0146810 0x7f3ea0146c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:59.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.265+0000 7f3e967fc700 1 --2- 172.21.15.89:0/1445599571 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ea00ad440 0x7f3ea00c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:40:59.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.266+0000 7f3e9effd700 1 --2- 172.21.15.89:0/1445599571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e8c003a60 0x7f3ea00bfad0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f3e90011000 tx=0x7f3e90015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:59.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.266+0000 7f3e977fe700 1 -- 172.21.15.89:0/1445599571 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90005d70 con 0x7f3e8c003a60 2023-10-10T20:40:59.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.266+0000 7f3e977fe700 1 -- 172.21.15.89:0/1445599571 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e900072e0 con 0x7f3e8c003a60 2023-10-10T20:40:59.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.266+0000 7f3ea578a700 1 -- 172.21.15.89:0/1445599571 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea0153bf0 con 0x7f3e8c003a60 2023-10-10T20:40:59.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.266+0000 7f3e977fe700 1 -- 172.21.15.89:0/1445599571 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e900076a0 con 0x7f3e8c003a60 2023-10-10T20:40:59.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.266+0000 7f3ea578a700 1 -- 172.21.15.89:0/1445599571 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea00c16b0 con 0x7f3e8c003a60 2023-10-10T20:40:59.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.267+0000 7f3ea578a700 1 -- 172.21.15.89:0/1445599571 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea00718b0 con 0x7f3e8c003a60 2023-10-10T20:40:59.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.267+0000 7f3e977fe700 1 -- 172.21.15.89:0/1445599571 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3e90019070 con 0x7f3e8c003a60 2023-10-10T20:40:59.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.267+0000 7f3e977fe700 1 --2- 172.21.15.89:0/1445599571 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3e80067320 0x7f3e800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:40:59.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.268+0000 7f3e977fe700 1 -- 172.21.15.89:0/1445599571 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f3e90085ab0 con 0x7f3e8c003a60 2023-10-10T20:40:59.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.268+0000 7f3e967fc700 1 --2- 172.21.15.89:0/1445599571 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3e80067320 0x7f3e800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:40:59.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.268+0000 7f3e967fc700 1 --2- 172.21.15.89:0/1445599571 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3e80067320 0x7f3e800697d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3e880093f0 tx=0x7f3e88008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:40:59.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.272+0000 7f3e977fe700 1 -- 172.21.15.89:0/1445599571 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e900528a0 con 0x7f3e8c003a60 2023-10-10T20:40:59.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:40:59.454+0000 7f3ea578a700 1 -- 172.21.15.89:0/1445599571 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3ea00ac030 con 0x7f3e8c003a60 2023-10-10T20:41:00.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.124+0000 7f3e977fe700 1 -- 172.21.15.89:0/1445599571 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3ea00ac030 con 0x7f3e8c003a60 2023-10-10T20:41:00.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.129+0000 7f3ea578a700 1 -- 172.21.15.89:0/1445599571 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3e80067320 msgr2=0x7f3e800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:00.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.129+0000 7f3ea578a700 1 --2- 172.21.15.89:0/1445599571 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3e80067320 0x7f3e800697d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3e880093f0 tx=0x7f3e88008040 comp rx=0 tx=0).stop 2023-10-10T20:41:00.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.129+0000 7f3ea578a700 1 -- 172.21.15.89:0/1445599571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e8c003a60 msgr2=0x7f3ea00bfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:00.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.129+0000 7f3ea578a700 1 --2- 172.21.15.89:0/1445599571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e8c003a60 0x7f3ea00bfad0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f3e90011000 tx=0x7f3e90015000 comp rx=0 tx=0).stop 2023-10-10T20:41:00.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.129+0000 7f3ea578a700 1 -- 172.21.15.89:0/1445599571 shutdown_connections 2023-10-10T20:41:00.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.129+0000 7f3ea578a700 1 --2- 172.21.15.89:0/1445599571 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3e80067320 0x7f3e800697d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:00.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.129+0000 7f3ea578a700 1 --2- 172.21.15.89:0/1445599571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ea0146810 0x7f3ea0146c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:00.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.129+0000 7f3ea578a700 1 --2- 172.21.15.89:0/1445599571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e8c003a60 0x7f3ea00bfad0 secure :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f3e90011000 tx=0x7f3e90015000 comp rx=0 tx=0).stop 2023-10-10T20:41:00.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.129+0000 7f3ea578a700 1 --2- 172.21.15.89:0/1445599571 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3ea00ad440 0x7f3ea00c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:00.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.129+0000 7f3ea578a700 1 -- 172.21.15.89:0/1445599571 >> 172.21.15.89:0/1445599571 conn(0x7f3ea00c8f90 msgr2=0x7f3ea0156c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:00.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.129+0000 7f3ea578a700 1 -- 172.21.15.89:0/1445599571 shutdown_connections 2023-10-10T20:41:00.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.130+0000 7f3ea578a700 1 -- 172.21.15.89:0/1445599571 wait complete. 2023-10-10T20:41:00.134 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' removed 2023-10-10T20:41:00.146 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:41:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.262+0000 7fa5373a8700 1 Processor -- start 2023-10-10T20:41:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.262+0000 7fa5373a8700 1 -- start start 2023-10-10T20:41:00.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.263+0000 7fa5373a8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa530159890 0x7fa53015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:00.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.263+0000 7fa5373a8700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa530159c60 0x7fa53015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:00.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.263+0000 7fa5373a8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa53015a160 con 0x7fa53015db10 2023-10-10T20:41:00.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.263+0000 7fa5373a8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa53015df50 con 0x7fa530159890 2023-10-10T20:41:00.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.263+0000 7fa5373a8700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5300d13a0 con 0x7fa530159c60 2023-10-10T20:41:00.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.263+0000 7fa535144700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa530159890 0x7fa53015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:00.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.263+0000 7fa535945700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa53015db10 0x7fa530069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45138/0 (socket says 172.21.15.89:45138) 2023-10-10T20:41:00.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.263+0000 7fa535945700 1 -- 172.21.15.89:0/504260984 learned_addr learned my addr 172.21.15.89:0/504260984 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:00.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.263+0000 7fa535144700 1 -- 172.21.15.89:0/504260984 >> v2:172.21.15.111:3300/0 conn(0x7fa530159c60 msgr2=0x7fa53015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:00.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.263+0000 7fa535144700 1 --2- 172.21.15.89:0/504260984 >> v2:172.21.15.111:3300/0 conn(0x7fa530159c60 0x7fa53015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:00.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.263+0000 7fa535144700 1 -- 172.21.15.89:0/504260984 >> v1:172.21.15.89:6789/0 conn(0x7fa53015db10 legacy=0x7fa530069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:00.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.263+0000 7fa535144700 1 -- 172.21.15.89:0/504260984 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53015e140 con 0x7fa530159890 2023-10-10T20:41:00.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.263+0000 7fa535144700 1 --2- 172.21.15.89:0/504260984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa530159890 0x7fa53015abd0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fa5300b67f0 tx=0x7fa52c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9dfc1e1baff568 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:00.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.263+0000 7fa527fff700 1 -- 172.21.15.89:0/504260984 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c00b6b0 con 0x7fa530159890 2023-10-10T20:41:00.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.264+0000 7fa527fff700 1 -- 172.21.15.89:0/504260984 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c00b810 con 0x7fa530159890 2023-10-10T20:41:00.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.264+0000 7fa527fff700 1 -- 172.21.15.89:0/504260984 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c00bbb0 con 0x7fa530159890 2023-10-10T20:41:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.264+0000 7fa5373a8700 1 -- 172.21.15.89:0/504260984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa530159890 msgr2=0x7fa53015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.264+0000 7fa5373a8700 1 --2- 172.21.15.89:0/504260984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa530159890 0x7fa53015abd0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fa5300b67f0 tx=0x7fa52c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.264+0000 7fa5373a8700 1 -- 172.21.15.89:0/504260984 shutdown_connections 2023-10-10T20:41:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.264+0000 7fa5373a8700 1 --2- 172.21.15.89:0/504260984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa530159890 0x7fa53015abd0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.264+0000 7fa5373a8700 1 --2- 172.21.15.89:0/504260984 >> v2:172.21.15.111:3300/0 conn(0x7fa530159c60 0x7fa53015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.264+0000 7fa5373a8700 1 -- 172.21.15.89:0/504260984 >> 172.21.15.89:0/504260984 conn(0x7fa530068e20 msgr2=0x7fa5300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.264+0000 7fa5373a8700 1 -- 172.21.15.89:0/504260984 shutdown_connections 2023-10-10T20:41:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.264+0000 7fa5373a8700 1 -- 172.21.15.89:0/504260984 wait complete. 2023-10-10T20:41:00.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.265+0000 7fa5373a8700 1 Processor -- start 2023-10-10T20:41:00.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.265+0000 7fa5373a8700 1 -- start start 2023-10-10T20:41:00.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.266+0000 7fa5373a8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa530159890 0x7fa5300c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:00.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.266+0000 7fa5373a8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa530159c60 0x7fa5300cf470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:00.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.266+0000 7fa5373a8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa53015db10 0x7fa5300cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:00.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.266+0000 7fa5373a8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5300ca360 con 0x7fa530159c60 2023-10-10T20:41:00.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.266+0000 7fa5373a8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5300c8ae0 con 0x7fa53015db10 2023-10-10T20:41:00.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.266+0000 7fa5373a8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5300c8d80 con 0x7fa530159890 2023-10-10T20:41:00.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.266+0000 7fa534943700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa530159c60 0x7fa5300cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:00.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.266+0000 7fa534943700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa530159c60 0x7fa5300cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54376/0 (socket says 172.21.15.89:54376) 2023-10-10T20:41:00.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.266+0000 7fa534943700 1 -- 172.21.15.89:0/3762547191 learned_addr learned my addr 172.21.15.89:0/3762547191 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:00.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.266+0000 7fa535144700 1 --2- 172.21.15.89:0/3762547191 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa530159890 0x7fa5300c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:00.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.266+0000 7fa535945700 1 --2- 172.21.15.89:0/3762547191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa53015db10 0x7fa5300cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:00.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.266+0000 7fa534943700 1 -- 172.21.15.89:0/3762547191 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa530159890 msgr2=0x7fa5300c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:00.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.266+0000 7fa534943700 1 --2- 172.21.15.89:0/3762547191 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa530159890 0x7fa5300c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:00.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.267+0000 7fa534943700 1 -- 172.21.15.89:0/3762547191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa53015db10 msgr2=0x7fa5300cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:00.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.267+0000 7fa534943700 1 --2- 172.21.15.89:0/3762547191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa53015db10 0x7fa5300cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:00.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.267+0000 7fa534943700 1 -- 172.21.15.89:0/3762547191 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c009420 con 0x7fa530159c60 2023-10-10T20:41:00.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.267+0000 7fa535144700 1 --2- 172.21.15.89:0/3762547191 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa530159890 0x7fa5300c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:00.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.267+0000 7fa535945700 1 --2- 172.21.15.89:0/3762547191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa53015db10 0x7fa5300cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:00.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.267+0000 7fa534943700 1 --2- 172.21.15.89:0/3762547191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa530159c60 0x7fa5300cf470 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fa52000b810 tx=0x7fa520009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:00.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.267+0000 7fa5257fa700 1 -- 172.21.15.89:0/3762547191 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52000f5f0 con 0x7fa530159c60 2023-10-10T20:41:00.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.267+0000 7fa5257fa700 1 -- 172.21.15.89:0/3762547191 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52000f770 con 0x7fa530159c60 2023-10-10T20:41:00.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.267+0000 7fa5257fa700 1 -- 172.21.15.89:0/3762547191 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52000fb30 con 0x7fa530159c60 2023-10-10T20:41:00.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.267+0000 7fa5373a8700 1 -- 172.21.15.89:0/3762547191 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5300c9060 con 0x7fa530159c60 2023-10-10T20:41:00.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.267+0000 7fa5373a8700 1 -- 172.21.15.89:0/3762547191 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5300c9620 con 0x7fa530159c60 2023-10-10T20:41:00.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.268+0000 7fa5373a8700 1 -- 172.21.15.89:0/3762547191 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa53004c3c0 con 0x7fa530159c60 2023-10-10T20:41:00.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.268+0000 7fa5257fa700 1 -- 172.21.15.89:0/3762547191 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa520010480 con 0x7fa530159c60 2023-10-10T20:41:00.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.269+0000 7fa5257fa700 1 --2- 172.21.15.89:0/3762547191 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa514067320 0x7fa5140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:00.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.269+0000 7fa5257fa700 1 -- 172.21.15.89:0/3762547191 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa5200848d0 con 0x7fa530159c60 2023-10-10T20:41:00.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.269+0000 7fa535144700 1 --2- 172.21.15.89:0/3762547191 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa514067320 0x7fa5140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:00.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.269+0000 7fa535144700 1 --2- 172.21.15.89:0/3762547191 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa514067320 0x7fa5140697d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0093f0 tx=0x7fa52c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:00.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.273+0000 7fa5257fa700 1 -- 172.21.15.89:0/3762547191 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5200518e0 con 0x7fa530159c60 2023-10-10T20:41:00.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.456+0000 7fa5373a8700 1 -- 172.21.15.89:0/3762547191 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa53006a5b0 con 0x7fa530159c60 2023-10-10T20:41:00.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.460+0000 7fa5257fa700 1 -- 172.21.15.89:0/3762547191 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa520054f00 con 0x7fa530159c60 2023-10-10T20:41:00.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.462+0000 7fa5373a8700 1 -- 172.21.15.89:0/3762547191 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa514067320 msgr2=0x7fa5140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:00.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.462+0000 7fa5373a8700 1 --2- 172.21.15.89:0/3762547191 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa514067320 0x7fa5140697d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0093f0 tx=0x7fa52c017000 comp rx=0 tx=0).stop 2023-10-10T20:41:00.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.462+0000 7fa5373a8700 1 -- 172.21.15.89:0/3762547191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa530159c60 msgr2=0x7fa5300cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:00.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.462+0000 7fa5373a8700 1 --2- 172.21.15.89:0/3762547191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa530159c60 0x7fa5300cf470 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fa52000b810 tx=0x7fa520009250 comp rx=0 tx=0).stop 2023-10-10T20:41:00.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.462+0000 7fa5373a8700 1 -- 172.21.15.89:0/3762547191 shutdown_connections 2023-10-10T20:41:00.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.462+0000 7fa5373a8700 1 --2- 172.21.15.89:0/3762547191 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa514067320 0x7fa5140697d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:00.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.462+0000 7fa5373a8700 1 --2- 172.21.15.89:0/3762547191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa53015db10 0x7fa5300cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:00.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.462+0000 7fa5373a8700 1 --2- 172.21.15.89:0/3762547191 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa530159890 0x7fa5300c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:00.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.462+0000 7fa5373a8700 1 --2- 172.21.15.89:0/3762547191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa530159c60 0x7fa5300cf470 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:00.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.462+0000 7fa5373a8700 1 -- 172.21.15.89:0/3762547191 >> 172.21.15.89:0/3762547191 conn(0x7fa530068e20 msgr2=0x7fa5300c7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:00.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.462+0000 7fa5373a8700 1 -- 172.21.15.89:0/3762547191 shutdown_connections 2023-10-10T20:41:00.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.462+0000 7fa5373a8700 1 -- 172.21.15.89:0/3762547191 wait complete. 2023-10-10T20:41:00.476 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup' 2023-10-10T20:41:00.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.589+0000 7fd33be34700 1 Processor -- start 2023-10-10T20:41:00.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.589+0000 7fd33be34700 1 -- start start 2023-10-10T20:41:00.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.590+0000 7fd33be34700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd3340b2830 0x7fd334069f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:00.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.590+0000 7fd33be34700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd33406a4d0 con 0x7fd3340cb540 2023-10-10T20:41:00.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.590+0000 7fd33be34700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3340cebb0 con 0x7fd33415dc70 2023-10-10T20:41:00.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.590+0000 7fd33be34700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3340b31f0 con 0x7fd3340b2830 2023-10-10T20:41:00.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.590+0000 7fd33a3d1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd3340b2830 0x7fd334069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:00.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.590+0000 7fd3393cf700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd3340cb540 0x7fd3340cf040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45148/0 (socket says 172.21.15.89:45148) 2023-10-10T20:41:00.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.590+0000 7fd3393cf700 1 -- 172.21.15.89:0/1670246647 learned_addr learned my addr 172.21.15.89:0/1670246647 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:00.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.591+0000 7fd338bce700 1 -- 172.21.15.89:0/1670246647 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2501194532 0 0) 0x7fd328001110 con 0x7fd3340cb540 2023-10-10T20:41:00.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.591+0000 7fd338bce700 1 -- 172.21.15.89:0/1670246647 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd31c003570 con 0x7fd3340cb540 2023-10-10T20:41:00.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.591+0000 7fd33a3d1700 1 -- 172.21.15.89:0/1670246647 >> v1:172.21.15.111:6789/0 conn(0x7fd33415dc70 legacy=0x7fd3340cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:00.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.591+0000 7fd33a3d1700 1 -- 172.21.15.89:0/1670246647 >> v1:172.21.15.89:6789/0 conn(0x7fd3340cb540 legacy=0x7fd3340cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:00.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.591+0000 7fd33a3d1700 1 -- 172.21.15.89:0/1670246647 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3340b2c70 con 0x7fd3340b2830 2023-10-10T20:41:00.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.591+0000 7fd33a3d1700 1 --2- 172.21.15.89:0/1670246647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3340b2830 0x7fd334069f90 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd324009e60 tx=0x7fd324009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297778849f3551a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:00.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.591+0000 7fd338bce700 1 -- 172.21.15.89:0/1670246647 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32400e070 con 0x7fd3340b2830 2023-10-10T20:41:00.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.591+0000 7fd338bce700 1 -- 172.21.15.89:0/1670246647 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32400bb60 con 0x7fd3340b2830 2023-10-10T20:41:00.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.591+0000 7fd338bce700 1 -- 172.21.15.89:0/1670246647 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324012430 con 0x7fd3340b2830 2023-10-10T20:41:00.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.591+0000 7fd33be34700 1 -- 172.21.15.89:0/1670246647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3340b2830 msgr2=0x7fd334069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:00.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.591+0000 7fd33be34700 1 --2- 172.21.15.89:0/1670246647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3340b2830 0x7fd334069f90 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd324009e60 tx=0x7fd324009780 comp rx=0 tx=0).stop 2023-10-10T20:41:00.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.592+0000 7fd33be34700 1 -- 172.21.15.89:0/1670246647 shutdown_connections 2023-10-10T20:41:00.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.592+0000 7fd33be34700 1 --2- 172.21.15.89:0/1670246647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3340b2830 0x7fd334069f90 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:00.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.592+0000 7fd33be34700 1 -- 172.21.15.89:0/1670246647 >> 172.21.15.89:0/1670246647 conn(0x7fd334068e20 msgr2=0x7fd3340d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:00.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.592+0000 7fd33be34700 1 -- 172.21.15.89:0/1670246647 shutdown_connections 2023-10-10T20:41:00.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.592+0000 7fd33be34700 1 -- 172.21.15.89:0/1670246647 wait complete. 2023-10-10T20:41:00.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.593+0000 7fd33be34700 1 Processor -- start 2023-10-10T20:41:00.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.593+0000 7fd33be34700 1 -- start start 2023-10-10T20:41:00.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.593+0000 7fd33be34700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd3340b2830 0x7fd3340b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:00.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.593+0000 7fd33be34700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3340cb540 0x7fd33415cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:00.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.593+0000 7fd33be34700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd33415dc70 0x7fd334157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:00.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.594+0000 7fd33be34700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3340b4570 con 0x7fd3340cb540 2023-10-10T20:41:00.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.594+0000 7fd33be34700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3340c8760 con 0x7fd33415dc70 2023-10-10T20:41:00.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.594+0000 7fd33be34700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3340c8a40 con 0x7fd3340b2830 2023-10-10T20:41:00.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.594+0000 7fd3393cf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3340cb540 0x7fd33415cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:00.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.594+0000 7fd3393cf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3340cb540 0x7fd33415cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54382/0 (socket says 172.21.15.89:54382) 2023-10-10T20:41:00.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.594+0000 7fd3393cf700 1 -- 172.21.15.89:0/2257351133 learned_addr learned my addr 172.21.15.89:0/2257351133 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:00.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.594+0000 7fd339bd0700 1 --2- 172.21.15.89:0/2257351133 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd3340b2830 0x7fd3340b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:00.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.594+0000 7fd33a3d1700 1 --2- 172.21.15.89:0/2257351133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd33415dc70 0x7fd334157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:00.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.594+0000 7fd3393cf700 1 -- 172.21.15.89:0/2257351133 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd3340b2830 msgr2=0x7fd3340b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:00.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.594+0000 7fd3393cf700 1 --2- 172.21.15.89:0/2257351133 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd3340b2830 0x7fd3340b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:00.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.594+0000 7fd3393cf700 1 -- 172.21.15.89:0/2257351133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd33415dc70 msgr2=0x7fd334157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:00.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.594+0000 7fd3393cf700 1 --2- 172.21.15.89:0/2257351133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd33415dc70 0x7fd334157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:00.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.594+0000 7fd3393cf700 1 -- 172.21.15.89:0/2257351133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd324009a20 con 0x7fd3340cb540 2023-10-10T20:41:00.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.594+0000 7fd33a3d1700 1 --2- 172.21.15.89:0/2257351133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd33415dc70 0x7fd334157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:00.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.594+0000 7fd3393cf700 1 --2- 172.21.15.89:0/2257351133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3340cb540 0x7fd33415cf10 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fd32800c7c0 tx=0x7fd328007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:00.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.594+0000 7fd331ffb700 1 -- 172.21.15.89:0/2257351133 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328005b30 con 0x7fd3340cb540 2023-10-10T20:41:00.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.594+0000 7fd331ffb700 1 -- 172.21.15.89:0/2257351133 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd328005cb0 con 0x7fd3340cb540 2023-10-10T20:41:00.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.594+0000 7fd33be34700 1 -- 172.21.15.89:0/2257351133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3340c8d20 con 0x7fd3340cb540 2023-10-10T20:41:00.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.594+0000 7fd331ffb700 1 -- 172.21.15.89:0/2257351133 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328007320 con 0x7fd3340cb540 2023-10-10T20:41:00.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.595+0000 7fd33be34700 1 -- 172.21.15.89:0/2257351133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33415d450 con 0x7fd3340cb540 2023-10-10T20:41:00.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.596+0000 7fd33be34700 1 -- 172.21.15.89:0/2257351133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3340c9240 con 0x7fd3340cb540 2023-10-10T20:41:00.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.599+0000 7fd331ffb700 1 -- 172.21.15.89:0/2257351133 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd32801b3b0 con 0x7fd3340cb540 2023-10-10T20:41:00.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.599+0000 7fd331ffb700 1 --2- 172.21.15.89:0/2257351133 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd3200673a0 0x7fd320069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:00.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.600+0000 7fd331ffb700 1 -- 172.21.15.89:0/2257351133 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd328083d40 con 0x7fd3340cb540 2023-10-10T20:41:00.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.600+0000 7fd339bd0700 1 --2- 172.21.15.89:0/2257351133 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd3200673a0 0x7fd320069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:00.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.601+0000 7fd339bd0700 1 --2- 172.21.15.89:0/2257351133 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd3200673a0 0x7fd320069850 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd32c003a10 tx=0x7fd32c007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:00.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.605+0000 7fd331ffb700 1 -- 172.21.15.89:0/2257351133 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3340c9240 con 0x7fd3340cb540 2023-10-10T20:41:00.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:00.863+0000 7fd33be34700 1 -- 172.21.15.89:0/2257351133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]} v 0) v1 -- 0x7fd33406a4d0 con 0x7fd3340cb540 2023-10-10T20:41:01.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.126+0000 7fd331ffb700 1 -- 172.21.15.89:0/2257351133 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]}]=0 v312) v1 ==== 156+0+0 (secure 0 0 0) 0x7fd33406a4d0 con 0x7fd3340cb540 2023-10-10T20:41:01.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.129+0000 7fd33be34700 1 -- 172.21.15.89:0/2257351133 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd3200673a0 msgr2=0x7fd320069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:01.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.129+0000 7fd33be34700 1 --2- 172.21.15.89:0/2257351133 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd3200673a0 0x7fd320069850 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd32c003a10 tx=0x7fd32c007ec0 comp rx=0 tx=0).stop 2023-10-10T20:41:01.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.129+0000 7fd33be34700 1 -- 172.21.15.89:0/2257351133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3340cb540 msgr2=0x7fd33415cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:01.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.129+0000 7fd33be34700 1 --2- 172.21.15.89:0/2257351133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3340cb540 0x7fd33415cf10 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fd32800c7c0 tx=0x7fd328007c70 comp rx=0 tx=0).stop 2023-10-10T20:41:01.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.129+0000 7fd33be34700 1 -- 172.21.15.89:0/2257351133 shutdown_connections 2023-10-10T20:41:01.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.129+0000 7fd33be34700 1 --2- 172.21.15.89:0/2257351133 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd3200673a0 0x7fd320069850 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:01.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.129+0000 7fd33be34700 1 --2- 172.21.15.89:0/2257351133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd33415dc70 0x7fd334157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:01.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.129+0000 7fd33be34700 1 --2- 172.21.15.89:0/2257351133 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd3340b2830 0x7fd3340b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:01.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.129+0000 7fd33be34700 1 --2- 172.21.15.89:0/2257351133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3340cb540 0x7fd33415cf10 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:01.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.129+0000 7fd33be34700 1 -- 172.21.15.89:0/2257351133 >> 172.21.15.89:0/2257351133 conn(0x7fd334068e20 msgr2=0x7fd3340c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:01.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.129+0000 7fd33be34700 1 -- 172.21.15.89:0/2257351133 shutdown_connections 2023-10-10T20:41:01.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.129+0000 7fd33be34700 1 -- 172.21.15.89:0/2257351133 wait complete. 2023-10-10T20:41:01.143 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info' command without subvolume group ... ok 2023-10-10T20:41:01.143 INFO:tasks.cephfs_test_runner:Starting test: test_volume_ls (tasks.cephfs.test_volumes.TestVolumes) That the existing and the newly created volumes can be listed and 2023-10-10T20:41:01.144 INFO:tasks.cephfs_test_runner:test_volume_ls (tasks.cephfs.test_volumes.TestVolumes) 2023-10-10T20:41:01.144 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls' 2023-10-10T20:41:01.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.257+0000 7f335412b700 1 Processor -- start 2023-10-10T20:41:01.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.258+0000 7f335412b700 1 -- start start 2023-10-10T20:41:01.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.258+0000 7f335412b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f334c141500 0x7f334c0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:01.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.258+0000 7f335412b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f334c06c0a0 0x7f334c0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:01.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.258+0000 7f335412b700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334c15c2a0 con 0x7f334c06bcd0 2023-10-10T20:41:01.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.258+0000 7f335412b700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334c0c9880 con 0x7f334c141500 2023-10-10T20:41:01.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.258+0000 7f335412b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334c15b690 con 0x7f334c06c0a0 2023-10-10T20:41:01.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.259+0000 7f33516c6700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f334c141500 0x7f334c0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:01.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.259+0000 7f33526c8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f334c06c0a0 0x7f334c0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:01.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.259+0000 7f33516c6700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f334c141500 0x7f334c0c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:43464/0 (socket says 172.21.15.89:43464) 2023-10-10T20:41:01.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.259+0000 7f33516c6700 1 -- 172.21.15.89:0/1671839384 learned_addr learned my addr 172.21.15.89:0/1671839384 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:01.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.259+0000 7f33516c6700 1 -- 172.21.15.89:0/1671839384 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f334c06c0a0 msgr2=0x7f334c0c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:01.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.259+0000 7f33516c6700 1 --2- 172.21.15.89:0/1671839384 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f334c06c0a0 0x7f334c0c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:01.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.259+0000 7f33516c6700 1 -- 172.21.15.89:0/1671839384 >> v1:172.21.15.111:6789/0 conn(0x7f334c06bcd0 legacy=0x7f334c140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:01.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.259+0000 7f33516c6700 1 -- 172.21.15.89:0/1671839384 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c15b970 con 0x7f334c141500 2023-10-10T20:41:01.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.259+0000 7f3350ec5700 1 -- 172.21.15.89:0/1671839384 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3944700680 0 0) 0x7f334c15c2a0 con 0x7f334c06bcd0 2023-10-10T20:41:01.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.259+0000 7f33516c6700 1 --2- 172.21.15.89:0/1671839384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f334c141500 0x7f334c0c8e00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f333c0099b0 tx=0x7f333c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b58e4f104db3705 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:01.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.260+0000 7f3350ec5700 1 -- 172.21.15.89:0/1671839384 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c009ea0 con 0x7f334c141500 2023-10-10T20:41:01.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.260+0000 7f3350ec5700 1 -- 172.21.15.89:0/1671839384 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333c00b860 con 0x7f334c141500 2023-10-10T20:41:01.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.260+0000 7f3350ec5700 1 -- 172.21.15.89:0/1671839384 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c00bc00 con 0x7f334c141500 2023-10-10T20:41:01.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.260+0000 7f335412b700 1 -- 172.21.15.89:0/1671839384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f334c141500 msgr2=0x7f334c0c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:01.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.260+0000 7f335412b700 1 --2- 172.21.15.89:0/1671839384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f334c141500 0x7f334c0c8e00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f333c0099b0 tx=0x7f333c009250 comp rx=0 tx=0).stop 2023-10-10T20:41:01.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.260+0000 7f335412b700 1 -- 172.21.15.89:0/1671839384 shutdown_connections 2023-10-10T20:41:01.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.260+0000 7f335412b700 1 --2- 172.21.15.89:0/1671839384 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f334c06c0a0 0x7f334c0c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:01.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.260+0000 7f335412b700 1 --2- 172.21.15.89:0/1671839384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f334c141500 0x7f334c0c8e00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:01.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.260+0000 7f335412b700 1 -- 172.21.15.89:0/1671839384 >> 172.21.15.89:0/1671839384 conn(0x7f334c0ce600 msgr2=0x7f334c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:01.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.260+0000 7f335412b700 1 -- 172.21.15.89:0/1671839384 shutdown_connections 2023-10-10T20:41:01.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.260+0000 7f335412b700 1 -- 172.21.15.89:0/1671839384 wait complete. 2023-10-10T20:41:01.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.261+0000 7f335412b700 1 Processor -- start 2023-10-10T20:41:01.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.261+0000 7f335412b700 1 -- start start 2023-10-10T20:41:01.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.262+0000 7f335412b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f334c06bcd0 0x7f334c157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:01.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.262+0000 7f335412b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f334c06c0a0 0x7f334c13ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:01.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.262+0000 7f335412b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f334c141500 0x7f334c15dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:01.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.262+0000 7f335412b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c0c5890 con 0x7f334c06c0a0 2023-10-10T20:41:01.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.262+0000 7f335412b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c157aa0 con 0x7f334c06bcd0 2023-10-10T20:41:01.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.262+0000 7f335412b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334c13e6e0 con 0x7f334c141500 2023-10-10T20:41:01.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.262+0000 7f33516c6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f334c06c0a0 0x7f334c13ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:01.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.262+0000 7f33516c6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f334c06c0a0 0x7f334c13ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54396/0 (socket says 172.21.15.89:54396) 2023-10-10T20:41:01.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.262+0000 7f33516c6700 1 -- 172.21.15.89:0/2301213511 learned_addr learned my addr 172.21.15.89:0/2301213511 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:01.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.262+0000 7f33526c8700 1 --2- 172.21.15.89:0/2301213511 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f334c141500 0x7f334c15dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:01.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.262+0000 7f3351ec7700 1 --2- 172.21.15.89:0/2301213511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f334c06bcd0 0x7f334c157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:01.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.263+0000 7f33516c6700 1 -- 172.21.15.89:0/2301213511 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f334c141500 msgr2=0x7f334c15dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:01.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.263+0000 7f33516c6700 1 --2- 172.21.15.89:0/2301213511 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f334c141500 0x7f334c15dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:01.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.263+0000 7f33516c6700 1 -- 172.21.15.89:0/2301213511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f334c06bcd0 msgr2=0x7f334c157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:01.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.263+0000 7f33516c6700 1 --2- 172.21.15.89:0/2301213511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f334c06bcd0 0x7f334c157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:01.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.263+0000 7f33516c6700 1 -- 172.21.15.89:0/2301213511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c0094f0 con 0x7f334c06c0a0 2023-10-10T20:41:01.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.263+0000 7f3351ec7700 1 --2- 172.21.15.89:0/2301213511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f334c06bcd0 0x7f334c157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:01.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.263+0000 7f33516c6700 1 --2- 172.21.15.89:0/2301213511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f334c06c0a0 0x7f334c13ece0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f334c069220 tx=0x7f333c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:01.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.263+0000 7f3341ffb700 1 -- 172.21.15.89:0/2301213511 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c00bbe0 con 0x7f334c06c0a0 2023-10-10T20:41:01.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.263+0000 7f3341ffb700 1 -- 172.21.15.89:0/2301213511 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333c01a450 con 0x7f334c06c0a0 2023-10-10T20:41:01.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.263+0000 7f335412b700 1 -- 172.21.15.89:0/2301213511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334c13d640 con 0x7f334c06c0a0 2023-10-10T20:41:01.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.263+0000 7f3341ffb700 1 -- 172.21.15.89:0/2301213511 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c01a810 con 0x7f334c06c0a0 2023-10-10T20:41:01.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.263+0000 7f335412b700 1 -- 172.21.15.89:0/2301213511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c13dc00 con 0x7f334c06c0a0 2023-10-10T20:41:01.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.264+0000 7f335412b700 1 -- 172.21.15.89:0/2301213511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334c04c3c0 con 0x7f334c06c0a0 2023-10-10T20:41:01.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.264+0000 7f3341ffb700 1 -- 172.21.15.89:0/2301213511 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f333c01ac00 con 0x7f334c06c0a0 2023-10-10T20:41:01.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.265+0000 7f3341ffb700 1 --2- 172.21.15.89:0/2301213511 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3330067320 0x7f33300697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:01.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.265+0000 7f3341ffb700 1 -- 172.21.15.89:0/2301213511 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f333c086350 con 0x7f334c06c0a0 2023-10-10T20:41:01.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.265+0000 7f3351ec7700 1 --2- 172.21.15.89:0/2301213511 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3330067320 0x7f33300697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:01.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.265+0000 7f3351ec7700 1 --2- 172.21.15.89:0/2301213511 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3330067320 0x7f33300697d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3348001560 tx=0x7f3348009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:01.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.269+0000 7f3341ffb700 1 -- 172.21.15.89:0/2301213511 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f333c0533e0 con 0x7f334c06c0a0 2023-10-10T20:41:01.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:01.527+0000 7f335412b700 1 -- 172.21.15.89:0/2301213511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]} v 0) v1 -- 0x7f334c15bc50 con 0x7f334c06c0a0 2023-10-10T20:41:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.129+0000 7f3341ffb700 1 -- 172.21.15.89:0/2301213511 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]}]=0 v313) v1 ==== 134+0+0 (secure 0 0 0) 0x7f333c056a00 con 0x7f334c06c0a0 2023-10-10T20:41:02.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.131+0000 7f335412b700 1 -- 172.21.15.89:0/2301213511 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3330067320 msgr2=0x7f33300697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:02.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.131+0000 7f335412b700 1 --2- 172.21.15.89:0/2301213511 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3330067320 0x7f33300697d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3348001560 tx=0x7f3348009040 comp rx=0 tx=0).stop 2023-10-10T20:41:02.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.131+0000 7f335412b700 1 -- 172.21.15.89:0/2301213511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f334c06c0a0 msgr2=0x7f334c13ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:02.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.131+0000 7f335412b700 1 --2- 172.21.15.89:0/2301213511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f334c06c0a0 0x7f334c13ece0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f334c069220 tx=0x7f333c017000 comp rx=0 tx=0).stop 2023-10-10T20:41:02.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.131+0000 7f335412b700 1 -- 172.21.15.89:0/2301213511 shutdown_connections 2023-10-10T20:41:02.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.131+0000 7f335412b700 1 --2- 172.21.15.89:0/2301213511 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f334c141500 0x7f334c15dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.131+0000 7f335412b700 1 --2- 172.21.15.89:0/2301213511 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3330067320 0x7f33300697d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.131+0000 7f335412b700 1 --2- 172.21.15.89:0/2301213511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f334c06bcd0 0x7f334c157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.131+0000 7f335412b700 1 --2- 172.21.15.89:0/2301213511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f334c06c0a0 0x7f334c13ece0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.131+0000 7f335412b700 1 -- 172.21.15.89:0/2301213511 >> 172.21.15.89:0/2301213511 conn(0x7f334c0ce600 msgr2=0x7f334c0cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:02.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.131+0000 7f335412b700 1 -- 172.21.15.89:0/2301213511 shutdown_connections 2023-10-10T20:41:02.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.131+0000 7f335412b700 1 -- 172.21.15.89:0/2301213511 wait complete. 2023-10-10T20:41:02.146 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:41:02.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.263+0000 7fdf1450d700 1 Processor -- start 2023-10-10T20:41:02.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.263+0000 7fdf1450d700 1 -- start start 2023-10-10T20:41:02.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.264+0000 7fdf1450d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdf0c06b160 0x7fdf0c06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:02.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.264+0000 7fdf1450d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdf0c0637d0 0x7fdf0c068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:02.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.264+0000 7fdf1450d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0c0ac450 con 0x7fdf0c06a2b0 2023-10-10T20:41:02.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.264+0000 7fdf1450d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0c069d00 con 0x7fdf0c06b160 2023-10-10T20:41:02.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.264+0000 7fdf1450d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c068830 con 0x7fdf0c0637d0 2023-10-10T20:41:02.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.264+0000 7fdf122a9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdf0c06b160 0x7fdf0c06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:02.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.264+0000 7fdf12aaa700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdf0c06a2b0 0x7fdf0c06a710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45154/0 (socket says 172.21.15.89:45154) 2023-10-10T20:41:02.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.264+0000 7fdf12aaa700 1 -- 172.21.15.89:0/2959259263 learned_addr learned my addr 172.21.15.89:0/2959259263 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:02.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.265+0000 7fdf122a9700 1 -- 172.21.15.89:0/2959259263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdf0c0637d0 msgr2=0x7fdf0c068cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:02.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.265+0000 7fdf122a9700 1 --2- 172.21.15.89:0/2959259263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdf0c0637d0 0x7fdf0c068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.265+0000 7fdf122a9700 1 -- 172.21.15.89:0/2959259263 >> v1:172.21.15.89:6789/0 conn(0x7fdf0c06a2b0 legacy=0x7fdf0c06a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:02.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.265+0000 7fdf122a9700 1 -- 172.21.15.89:0/2959259263 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c069740 con 0x7fdf0c06b160 2023-10-10T20:41:02.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.265+0000 7fdf112a7700 1 -- 172.21.15.89:0/2959259263 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1567465249 0 0) 0x7fdf0c0ac450 con 0x7fdf0c06a2b0 2023-10-10T20:41:02.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.265+0000 7fdf122a9700 1 --2- 172.21.15.89:0/2959259263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c06b160 0x7fdf0c06b530 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fdf080098e0 tx=0x7fdf080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8366a325815f85e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:02.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.265+0000 7fdf112a7700 1 -- 172.21.15.89:0/2959259263 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08009e80 con 0x7fdf0c06b160 2023-10-10T20:41:02.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.265+0000 7fdf112a7700 1 -- 172.21.15.89:0/2959259263 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0800b860 con 0x7fdf0c06b160 2023-10-10T20:41:02.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.265+0000 7fdf112a7700 1 -- 172.21.15.89:0/2959259263 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0800bbe0 con 0x7fdf0c06b160 2023-10-10T20:41:02.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.266+0000 7fdf1450d700 1 -- 172.21.15.89:0/2959259263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c06b160 msgr2=0x7fdf0c06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:02.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.266+0000 7fdf1450d700 1 --2- 172.21.15.89:0/2959259263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c06b160 0x7fdf0c06b530 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fdf080098e0 tx=0x7fdf080091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.266+0000 7fdf1450d700 1 -- 172.21.15.89:0/2959259263 shutdown_connections 2023-10-10T20:41:02.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.266+0000 7fdf1450d700 1 --2- 172.21.15.89:0/2959259263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c06b160 0x7fdf0c06b530 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.266+0000 7fdf1450d700 1 --2- 172.21.15.89:0/2959259263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdf0c0637d0 0x7fdf0c068cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.266+0000 7fdf1450d700 1 -- 172.21.15.89:0/2959259263 >> 172.21.15.89:0/2959259263 conn(0x7fdf0c06ded0 msgr2=0x7fdf0c06e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:02.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.266+0000 7fdf1450d700 1 -- 172.21.15.89:0/2959259263 shutdown_connections 2023-10-10T20:41:02.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.266+0000 7fdf1450d700 1 -- 172.21.15.89:0/2959259263 wait complete. 2023-10-10T20:41:02.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.267+0000 7fdf1450d700 1 Processor -- start 2023-10-10T20:41:02.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.267+0000 7fdf1450d700 1 -- start start 2023-10-10T20:41:02.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf1450d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c0637d0 0x7fdf0c14eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:02.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf1450d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdf0c06a2b0 0x7fdf0c169f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:02.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf1450d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf0c06b160 0x7fdf0c1561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:02.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf1450d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c157ae0 con 0x7fdf0c0637d0 2023-10-10T20:41:02.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf1450d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c1580a0 con 0x7fdf0c06b160 2023-10-10T20:41:02.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf1450d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c14f0c0 con 0x7fdf0c06a2b0 2023-10-10T20:41:02.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf122a9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c0637d0 0x7fdf0c14eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:02.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf122a9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c0637d0 0x7fdf0c14eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54410/0 (socket says 172.21.15.89:54410) 2023-10-10T20:41:02.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf122a9700 1 -- 172.21.15.89:0/2838325595 learned_addr learned my addr 172.21.15.89:0/2838325595 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:02.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf12aaa700 1 --2- 172.21.15.89:0/2838325595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf0c06b160 0x7fdf0c1561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:02.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf11aa8700 1 --2- 172.21.15.89:0/2838325595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdf0c06a2b0 0x7fdf0c169f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:02.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf122a9700 1 -- 172.21.15.89:0/2838325595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdf0c06a2b0 msgr2=0x7fdf0c169f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:02.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf122a9700 1 --2- 172.21.15.89:0/2838325595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdf0c06a2b0 0x7fdf0c169f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf122a9700 1 -- 172.21.15.89:0/2838325595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf0c06b160 msgr2=0x7fdf0c1561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:02.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf122a9700 1 --2- 172.21.15.89:0/2838325595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf0c06b160 0x7fdf0c1561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf122a9700 1 -- 172.21.15.89:0/2838325595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf08009420 con 0x7fdf0c0637d0 2023-10-10T20:41:02.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf12aaa700 1 --2- 172.21.15.89:0/2838325595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf0c06b160 0x7fdf0c1561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:02.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf11aa8700 1 --2- 172.21.15.89:0/2838325595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdf0c06a2b0 0x7fdf0c169f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:02.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf122a9700 1 --2- 172.21.15.89:0/2838325595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c0637d0 0x7fdf0c14eb80 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fdf08006f70 tx=0x7fdf08017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:02.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf027fc700 1 -- 172.21.15.89:0/2838325595 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0800bbe0 con 0x7fdf0c0637d0 2023-10-10T20:41:02.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf027fc700 1 -- 172.21.15.89:0/2838325595 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf08006af0 con 0x7fdf0c0637d0 2023-10-10T20:41:02.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf1450d700 1 -- 172.21.15.89:0/2838325595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0c16a9c0 con 0x7fdf0c0637d0 2023-10-10T20:41:02.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf027fc700 1 -- 172.21.15.89:0/2838325595 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08011670 con 0x7fdf0c0637d0 2023-10-10T20:41:02.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.268+0000 7fdf1450d700 1 -- 172.21.15.89:0/2838325595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c157d30 con 0x7fdf0c0637d0 2023-10-10T20:41:02.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.269+0000 7fdf1450d700 1 -- 172.21.15.89:0/2838325595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0c04c3c0 con 0x7fdf0c0637d0 2023-10-10T20:41:02.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.270+0000 7fdf027fc700 1 -- 172.21.15.89:0/2838325595 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdf08011a60 con 0x7fdf0c0637d0 2023-10-10T20:41:02.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.270+0000 7fdf027fc700 1 --2- 172.21.15.89:0/2838325595 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdef0067320 0x7fdef00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:02.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.270+0000 7fdf027fc700 1 -- 172.21.15.89:0/2838325595 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fdf08086320 con 0x7fdf0c0637d0 2023-10-10T20:41:02.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.270+0000 7fdf11aa8700 1 --2- 172.21.15.89:0/2838325595 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdef0067320 0x7fdef00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:02.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.270+0000 7fdf11aa8700 1 --2- 172.21.15.89:0/2838325595 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdef0067320 0x7fdef00697d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c159fe0 tx=0x7fdefc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:02.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.274+0000 7fdf027fc700 1 -- 172.21.15.89:0/2838325595 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf080532b0 con 0x7fdf0c0637d0 2023-10-10T20:41:02.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.457+0000 7fdf1450d700 1 -- 172.21.15.89:0/2838325595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdf0c0635b0 con 0x7fdf0c0637d0 2023-10-10T20:41:02.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.463+0000 7fdf027fc700 1 -- 172.21.15.89:0/2838325595 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdf080568d0 con 0x7fdf0c0637d0 2023-10-10T20:41:02.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.465+0000 7fdf1450d700 1 -- 172.21.15.89:0/2838325595 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdef0067320 msgr2=0x7fdef00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:02.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.465+0000 7fdf1450d700 1 --2- 172.21.15.89:0/2838325595 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdef0067320 0x7fdef00697d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c159fe0 tx=0x7fdefc008040 comp rx=0 tx=0).stop 2023-10-10T20:41:02.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.465+0000 7fdf1450d700 1 -- 172.21.15.89:0/2838325595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c0637d0 msgr2=0x7fdf0c14eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:02.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.465+0000 7fdf1450d700 1 --2- 172.21.15.89:0/2838325595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c0637d0 0x7fdf0c14eb80 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fdf08006f70 tx=0x7fdf08017000 comp rx=0 tx=0).stop 2023-10-10T20:41:02.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.465+0000 7fdf1450d700 1 -- 172.21.15.89:0/2838325595 shutdown_connections 2023-10-10T20:41:02.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.465+0000 7fdf1450d700 1 --2- 172.21.15.89:0/2838325595 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdef0067320 0x7fdef00697d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.465+0000 7fdf1450d700 1 --2- 172.21.15.89:0/2838325595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf0c06b160 0x7fdf0c1561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.465+0000 7fdf1450d700 1 --2- 172.21.15.89:0/2838325595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c0637d0 0x7fdf0c14eb80 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.465+0000 7fdf1450d700 1 --2- 172.21.15.89:0/2838325595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdf0c06a2b0 0x7fdf0c169f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.465+0000 7fdf1450d700 1 -- 172.21.15.89:0/2838325595 >> 172.21.15.89:0/2838325595 conn(0x7fdf0c06ded0 msgr2=0x7fdf0c14f940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:02.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.465+0000 7fdf1450d700 1 -- 172.21.15.89:0/2838325595 shutdown_connections 2023-10-10T20:41:02.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.465+0000 7fdf1450d700 1 -- 172.21.15.89:0/2838325595 wait complete. 2023-10-10T20:41:02.479 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:41:02.480 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:41:02.517 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:41:02.562 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:41:02.562 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:41:02.563 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:41:02.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.675+0000 7f51f3147700 1 Processor -- start 2023-10-10T20:41:02.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.675+0000 7f51f3147700 1 -- start start 2023-10-10T20:41:02.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.676+0000 7f51f3147700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f51ec069560 0x7f51ec0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:02.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.676+0000 7f51f3147700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ec069ee0 con 0x7f51ec159c60 2023-10-10T20:41:02.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.676+0000 7f51f3147700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ec15a160 con 0x7f51ec159890 2023-10-10T20:41:02.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.676+0000 7f51f3147700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ec0d13a0 con 0x7f51ec069560 2023-10-10T20:41:02.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.676+0000 7f51e3fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f51ec159c60 0x7f51ec15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45164/0 (socket says 172.21.15.89:45164) 2023-10-10T20:41:02.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.676+0000 7f51f16e4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f51ec069560 0x7f51ec0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:02.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.676+0000 7f51e3fff700 1 -- 172.21.15.89:0/3230396585 learned_addr learned my addr 172.21.15.89:0/3230396585 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:02.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.676+0000 7f51f16e4700 1 -- 172.21.15.89:0/3230396585 >> v1:172.21.15.111:6789/0 conn(0x7f51ec159890 legacy=0x7f51ec15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:02.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.676+0000 7f51f16e4700 1 -- 172.21.15.89:0/3230396585 >> v1:172.21.15.89:6789/0 conn(0x7f51ec159c60 legacy=0x7f51ec15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:02.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.676+0000 7f51f16e4700 1 -- 172.21.15.89:0/3230396585 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec15e140 con 0x7f51ec069560 2023-10-10T20:41:02.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.676+0000 7f51e37fe700 1 -- 172.21.15.89:0/3230396585 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332167845 0 0) 0x7f51ec069ee0 con 0x7f51ec159c60 2023-10-10T20:41:02.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.677+0000 7f51f16e4700 1 --2- 172.21.15.89:0/3230396585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51ec069560 0x7f51ec0699a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f51e40098e0 tx=0x7f51e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3028e680d9c758fd server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:02.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.677+0000 7f51e37fe700 1 -- 172.21.15.89:0/3230396585 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4009e80 con 0x7f51ec069560 2023-10-10T20:41:02.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.677+0000 7f51e37fe700 1 -- 172.21.15.89:0/3230396585 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e400b820 con 0x7f51ec069560 2023-10-10T20:41:02.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.677+0000 7f51e37fe700 1 -- 172.21.15.89:0/3230396585 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e400bba0 con 0x7f51ec069560 2023-10-10T20:41:02.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.677+0000 7f51f3147700 1 -- 172.21.15.89:0/3230396585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51ec069560 msgr2=0x7f51ec0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:02.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.677+0000 7f51f3147700 1 --2- 172.21.15.89:0/3230396585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51ec069560 0x7f51ec0699a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f51e40098e0 tx=0x7f51e40091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.677+0000 7f51f3147700 1 -- 172.21.15.89:0/3230396585 shutdown_connections 2023-10-10T20:41:02.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.677+0000 7f51f3147700 1 --2- 172.21.15.89:0/3230396585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51ec069560 0x7f51ec0699a0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.677+0000 7f51f3147700 1 -- 172.21.15.89:0/3230396585 >> 172.21.15.89:0/3230396585 conn(0x7f51ec068e20 msgr2=0x7f51ec0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:02.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.677+0000 7f51f3147700 1 -- 172.21.15.89:0/3230396585 shutdown_connections 2023-10-10T20:41:02.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.677+0000 7f51f3147700 1 -- 172.21.15.89:0/3230396585 wait complete. 2023-10-10T20:41:02.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.678+0000 7f51f3147700 1 Processor -- start 2023-10-10T20:41:02.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.679+0000 7f51f3147700 1 -- start start 2023-10-10T20:41:02.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.679+0000 7f51f3147700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51ec069560 0x7f51ec14d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:02.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.679+0000 7f51f3147700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f51ec159890 0x7f51ec153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:02.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.679+0000 7f51f3147700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51ec159c60 0x7f51ec153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:02.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.679+0000 7f51f3147700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec15c4e0 con 0x7f51ec159c60 2023-10-10T20:41:02.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.679+0000 7f51f3147700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0ce170 con 0x7f51ec069560 2023-10-10T20:41:02.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.679+0000 7f51f3147700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0ce450 con 0x7f51ec159890 2023-10-10T20:41:02.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.679+0000 7f51f16e4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51ec159c60 0x7f51ec153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:02.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.679+0000 7f51f16e4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51ec159c60 0x7f51ec153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54440/0 (socket says 172.21.15.89:54440) 2023-10-10T20:41:02.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.679+0000 7f51f16e4700 1 -- 172.21.15.89:0/1821464557 learned_addr learned my addr 172.21.15.89:0/1821464557 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:02.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.680+0000 7f51e3fff700 1 --2- 172.21.15.89:0/1821464557 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f51ec159890 0x7f51ec153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:02.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.680+0000 7f51f0ee3700 1 --2- 172.21.15.89:0/1821464557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51ec069560 0x7f51ec14d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:02.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.680+0000 7f51f16e4700 1 -- 172.21.15.89:0/1821464557 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f51ec159890 msgr2=0x7f51ec153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:02.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.680+0000 7f51f16e4700 1 --2- 172.21.15.89:0/1821464557 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f51ec159890 0x7f51ec153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.680+0000 7f51f16e4700 1 -- 172.21.15.89:0/1821464557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51ec069560 msgr2=0x7f51ec14d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:02.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.680+0000 7f51f16e4700 1 --2- 172.21.15.89:0/1821464557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51ec069560 0x7f51ec14d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.680+0000 7f51f16e4700 1 -- 172.21.15.89:0/1821464557 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e4009420 con 0x7f51ec159c60 2023-10-10T20:41:02.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.680+0000 7f51e3fff700 1 --2- 172.21.15.89:0/1821464557 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f51ec159890 0x7f51ec153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:02.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.680+0000 7f51f0ee3700 1 --2- 172.21.15.89:0/1821464557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51ec069560 0x7f51ec14d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:02.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.680+0000 7f51f16e4700 1 --2- 172.21.15.89:0/1821464557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51ec159c60 0x7f51ec153d70 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f51e40093f0 tx=0x7f51e4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:02.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.680+0000 7f51e0ff9700 1 -- 172.21.15.89:0/1821464557 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e401b680 con 0x7f51ec159c60 2023-10-10T20:41:02.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.680+0000 7f51e0ff9700 1 -- 172.21.15.89:0/1821464557 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e401b800 con 0x7f51ec159c60 2023-10-10T20:41:02.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.680+0000 7f51f3147700 1 -- 172.21.15.89:0/1821464557 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ec0ce730 con 0x7f51ec159c60 2023-10-10T20:41:02.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.680+0000 7f51e0ff9700 1 -- 172.21.15.89:0/1821464557 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e401bbc0 con 0x7f51ec159c60 2023-10-10T20:41:02.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.680+0000 7f51f3147700 1 -- 172.21.15.89:0/1821464557 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ec0cd5c0 con 0x7f51ec159c60 2023-10-10T20:41:02.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.681+0000 7f51f3147700 1 -- 172.21.15.89:0/1821464557 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ec15c650 con 0x7f51ec159c60 2023-10-10T20:41:02.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.681+0000 7f51e0ff9700 1 -- 172.21.15.89:0/1821464557 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f51e401c450 con 0x7f51ec159c60 2023-10-10T20:41:02.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.682+0000 7f51e0ff9700 1 --2- 172.21.15.89:0/1821464557 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f51d8067320 0x7f51d80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:02.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.682+0000 7f51f0ee3700 1 --2- 172.21.15.89:0/1821464557 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f51d8067320 0x7f51d80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:02.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.682+0000 7f51e0ff9700 1 -- 172.21.15.89:0/1821464557 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f51e40868e0 con 0x7f51ec159c60 2023-10-10T20:41:02.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.682+0000 7f51f0ee3700 1 --2- 172.21.15.89:0/1821464557 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f51d8067320 0x7f51d80697d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f51e8007fd0 tx=0x7f51e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:02.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.686+0000 7f51e0ff9700 1 -- 172.21.15.89:0/1821464557 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51e4053870 con 0x7f51ec159c60 2023-10-10T20:41:02.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.916+0000 7f51f3147700 1 -- 172.21.15.89:0/1821464557 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51ec04c3c0 con 0x7f51ec159c60 2023-10-10T20:41:02.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.917+0000 7f51e0ff9700 1 -- 172.21.15.89:0/1821464557 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f51e402a020 con 0x7f51ec159c60 2023-10-10T20:41:02.919 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:02.919 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":67,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1635581143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1635581143},{"type":"v1","addr":"172.21.15.89:6837","nonce":1635581143}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5732,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/321638734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":321638734},{"type":"v1","addr":"172.21.15.111:6835","nonce":321638734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1070384555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1070384555},{"type":"v1","addr":"172.21.15.111:6834","nonce":1070384555}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":7116,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/3591239640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3591239640},{"type":"v1","addr":"172.21.15.89:6836","nonce":3591239640}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[]} 2023-10-10T20:41:02.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.918+0000 7f51f3147700 1 -- 172.21.15.89:0/1821464557 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f51d8067320 msgr2=0x7f51d80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:02.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.918+0000 7f51f3147700 1 --2- 172.21.15.89:0/1821464557 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f51d8067320 0x7f51d80697d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f51e8007fd0 tx=0x7f51e8009040 comp rx=0 tx=0).stop 2023-10-10T20:41:02.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.918+0000 7f51f3147700 1 -- 172.21.15.89:0/1821464557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51ec159c60 msgr2=0x7f51ec153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:02.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.918+0000 7f51f3147700 1 --2- 172.21.15.89:0/1821464557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51ec159c60 0x7f51ec153d70 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f51e40093f0 tx=0x7f51e4016000 comp rx=0 tx=0).stop 2023-10-10T20:41:02.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.919+0000 7f51f3147700 1 -- 172.21.15.89:0/1821464557 shutdown_connections 2023-10-10T20:41:02.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.919+0000 7f51f3147700 1 --2- 172.21.15.89:0/1821464557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51ec159c60 0x7f51ec153d70 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.919+0000 7f51f3147700 1 --2- 172.21.15.89:0/1821464557 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f51d8067320 0x7f51d80697d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.919+0000 7f51f3147700 1 --2- 172.21.15.89:0/1821464557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51ec069560 0x7f51ec14d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.919+0000 7f51f3147700 1 --2- 172.21.15.89:0/1821464557 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f51ec159890 0x7f51ec153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:02.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.919+0000 7f51f3147700 1 -- 172.21.15.89:0/1821464557 >> 172.21.15.89:0/1821464557 conn(0x7f51ec068e20 msgr2=0x7f51ec157cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:02.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.919+0000 7f51f3147700 1 -- 172.21.15.89:0/1821464557 shutdown_connections 2023-10-10T20:41:02.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:02.919+0000 7f51f3147700 1 -- 172.21.15.89:0/1821464557 wait complete. 2023-10-10T20:41:02.923 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 67 2023-10-10T20:41:02.933 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:41:02.933 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:41:02.933 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:41:02.934 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:41:02.934 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:41:02.934 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:41:02.935 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:41:02.935 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:41:02.935 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:41:02.935 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:41:02.936 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:41:02.936 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:41:02.936 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:41:02.936 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:41:02.937 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:41:02.937 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:41:02.942 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:41:02.942 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:41:02.943 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:41:02.944 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:41:02.946 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:41:02.946 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:41:02.947 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:41:02.947 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:41:02.986 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:41:02.989 DEBUG:teuthology.parallel:result is None 2023-10-10T20:41:02.991 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:41:02.991 DEBUG:teuthology.parallel:result is None 2023-10-10T20:41:02.992 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:41:02.992 DEBUG:teuthology.parallel:result is None 2023-10-10T20:41:02.993 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:41:02.993 DEBUG:teuthology.parallel:result is None 2023-10-10T20:41:02.994 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:41:03.059 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:41:03.061 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-10T20:41:03.072 INFO:tasks.ceph.mds.d.smithi111.stdout:starting mds.d at 2023-10-10T20:41:03.072 INFO:tasks.ceph.mds.b.smithi111.stdout:starting mds.b at 2023-10-10T20:41:03.073 INFO:tasks.ceph.mds.d.smithi111.stderr:2023-10-10T20:41:03.070+0000 7f7241b48700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-10T20:41:03.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.113+0000 7fd618858700 1 Processor -- start 2023-10-10T20:41:03.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.113+0000 7fd618858700 1 -- start start 2023-10-10T20:41:03.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.114+0000 7fd618858700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd614159890 0x7fd61415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:03.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.114+0000 7fd618858700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61415a160 con 0x7fd614159c60 2023-10-10T20:41:03.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.114+0000 7fd618858700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6140d13a0 con 0x7fd614069560 2023-10-10T20:41:03.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.114+0000 7fd618858700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61415e140 con 0x7fd614159890 2023-10-10T20:41:03.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.114+0000 7fd611d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd614159c60 0x7fd61415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45166/0 (socket says 172.21.15.89:45166) 2023-10-10T20:41:03.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.114+0000 7fd611d9b700 1 -- 172.21.15.89:0/9591280 learned_addr learned my addr 172.21.15.89:0/9591280 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:03.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.115+0000 7fd61259c700 1 --2- 172.21.15.89:0/9591280 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd614159890 0x7fd61415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:03.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.115+0000 7fd61159a700 1 -- 172.21.15.89:0/9591280 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853212354 0 0) 0x7fd61415a160 con 0x7fd614159c60 2023-10-10T20:41:03.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.115+0000 7fd61159a700 1 -- 172.21.15.89:0/9591280 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5fc003570 con 0x7fd614159c60 2023-10-10T20:41:03.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.115+0000 7fd61159a700 1 -- 172.21.15.89:0/9591280 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393733654 0 0) 0x7fd6140d13a0 con 0x7fd614069560 2023-10-10T20:41:03.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.115+0000 7fd61159a700 1 -- 172.21.15.89:0/9591280 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd61415a160 con 0x7fd614069560 2023-10-10T20:41:03.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.115+0000 7fd61259c700 1 -- 172.21.15.89:0/9591280 >> v1:172.21.15.111:6789/0 conn(0x7fd614069560 legacy=0x7fd6140ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:03.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.115+0000 7fd61159a700 1 -- 172.21.15.89:0/9591280 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 688835113 0 0) 0x7fd5fc003570 con 0x7fd614159c60 2023-10-10T20:41:03.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.115+0000 7fd61259c700 1 -- 172.21.15.89:0/9591280 >> v1:172.21.15.89:6789/0 conn(0x7fd614159c60 legacy=0x7fd61415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:03.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.115+0000 7fd61259c700 1 -- 172.21.15.89:0/9591280 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61415a3d0 con 0x7fd614159890 2023-10-10T20:41:03.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.115+0000 7fd61259c700 1 --2- 172.21.15.89:0/9591280 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd614159890 0x7fd61415abd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd604009910 tx=0x7fd6040091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de7efa454ac95d4e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:03.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.115+0000 7fd61159a700 1 -- 172.21.15.89:0/9591280 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608001470 con 0x7fd614159890 2023-10-10T20:41:03.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.115+0000 7fd61159a700 1 -- 172.21.15.89:0/9591280 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd604011420 con 0x7fd614159890 2023-10-10T20:41:03.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.115+0000 7fd61159a700 1 -- 172.21.15.89:0/9591280 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6040117c0 con 0x7fd614159890 2023-10-10T20:41:03.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.116+0000 7fd618858700 1 -- 172.21.15.89:0/9591280 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd614159890 msgr2=0x7fd61415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:03.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.116+0000 7fd618858700 1 --2- 172.21.15.89:0/9591280 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd614159890 0x7fd61415abd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd604009910 tx=0x7fd6040091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:03.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.116+0000 7fd618858700 1 -- 172.21.15.89:0/9591280 shutdown_connections 2023-10-10T20:41:03.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.116+0000 7fd618858700 1 --2- 172.21.15.89:0/9591280 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd614159890 0x7fd61415abd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:03.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.116+0000 7fd618858700 1 -- 172.21.15.89:0/9591280 >> 172.21.15.89:0/9591280 conn(0x7fd614068e20 msgr2=0x7fd6140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:03.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.116+0000 7fd618858700 1 -- 172.21.15.89:0/9591280 shutdown_connections 2023-10-10T20:41:03.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.116+0000 7fd618858700 1 -- 172.21.15.89:0/9591280 wait complete. 2023-10-10T20:41:03.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.117+0000 7fd618858700 1 Processor -- start 2023-10-10T20:41:03.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.117+0000 7fd618858700 1 -- start start 2023-10-10T20:41:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.118+0000 7fd618858700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd614069560 0x7fd614147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.118+0000 7fd618858700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd614159890 0x7fd6141565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.118+0000 7fd618858700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd614159c60 0x7fd6140b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.118+0000 7fd618858700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6140b44f0 con 0x7fd614159890 2023-10-10T20:41:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.118+0000 7fd618858700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6140b38e0 con 0x7fd614159c60 2023-10-10T20:41:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.118+0000 7fd618858700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6140b3bc0 con 0x7fd614069560 2023-10-10T20:41:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.118+0000 7fd611d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd614159890 0x7fd6141565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.118+0000 7fd611d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd614159890 0x7fd6141565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54482/0 (socket says 172.21.15.89:54482) 2023-10-10T20:41:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.118+0000 7fd611d9b700 1 -- 172.21.15.89:0/841370328 learned_addr learned my addr 172.21.15.89:0/841370328 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:03.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.118+0000 7fd61259c700 1 --2- 172.21.15.89:0/841370328 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd614069560 0x7fd614147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:03.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.118+0000 7fd612d9d700 1 --2- 172.21.15.89:0/841370328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd614159c60 0x7fd6140b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:03.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.118+0000 7fd611d9b700 1 -- 172.21.15.89:0/841370328 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd614069560 msgr2=0x7fd614147150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:03.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.118+0000 7fd611d9b700 1 --2- 172.21.15.89:0/841370328 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd614069560 0x7fd614147150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:03.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.118+0000 7fd611d9b700 1 -- 172.21.15.89:0/841370328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd614159c60 msgr2=0x7fd6140b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:03.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.118+0000 7fd611d9b700 1 --2- 172.21.15.89:0/841370328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd614159c60 0x7fd6140b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:03.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.118+0000 7fd611d9b700 1 -- 172.21.15.89:0/841370328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd604009420 con 0x7fd614159890 2023-10-10T20:41:03.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.118+0000 7fd612d9d700 1 --2- 172.21.15.89:0/841370328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd614159c60 0x7fd6140b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:03.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.119+0000 7fd611d9b700 1 --2- 172.21.15.89:0/841370328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd614159890 0x7fd6141565e0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fd60800c960 tx=0x7fd6080019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:03.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.119+0000 7fd6027fc700 1 -- 172.21.15.89:0/841370328 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60800cea0 con 0x7fd614159890 2023-10-10T20:41:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.119+0000 7fd6027fc700 1 -- 172.21.15.89:0/841370328 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd608005b30 con 0x7fd614159890 2023-10-10T20:41:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.119+0000 7fd618858700 1 -- 172.21.15.89:0/841370328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6140b3ea0 con 0x7fd614159890 2023-10-10T20:41:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.119+0000 7fd6027fc700 1 -- 172.21.15.89:0/841370328 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6080070f0 con 0x7fd614159890 2023-10-10T20:41:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.119+0000 7fd618858700 1 -- 172.21.15.89:0/841370328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6140b3590 con 0x7fd614159890 2023-10-10T20:41:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.120+0000 7fd618858700 1 -- 172.21.15.89:0/841370328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61404c3c0 con 0x7fd614159890 2023-10-10T20:41:03.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.120+0000 7fd6027fc700 1 -- 172.21.15.89:0/841370328 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd6080074a0 con 0x7fd614159890 2023-10-10T20:41:03.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.120+0000 7fd6027fc700 1 --2- 172.21.15.89:0/841370328 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd5f4067320 0x7fd5f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:03.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.120+0000 7fd6027fc700 1 -- 172.21.15.89:0/841370328 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd6080846d0 con 0x7fd614159890 2023-10-10T20:41:03.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.120+0000 7fd61259c700 1 --2- 172.21.15.89:0/841370328 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd5f4067320 0x7fd5f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:03.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.121+0000 7fd61259c700 1 --2- 172.21.15.89:0/841370328 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd5f4067320 0x7fd5f40697d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd6040030c0 tx=0x7fd604016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:03.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.125+0000 7fd6027fc700 1 -- 172.21.15.89:0/841370328 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6080516e0 con 0x7fd614159890 2023-10-10T20:41:03.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:03.331+0000 7fd618858700 1 -- 172.21.15.89:0/841370328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd6140cf1f0 con 0x7fd614159890 2023-10-10T20:41:04.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.161+0000 7fd6027fc700 1 -- 172.21.15.89:0/841370328 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v108) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd608054d00 con 0x7fd614159890 2023-10-10T20:41:04.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.163+0000 7fd618858700 1 -- 172.21.15.89:0/841370328 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd5f4067320 msgr2=0x7fd5f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:04.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.163+0000 7fd618858700 1 --2- 172.21.15.89:0/841370328 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd5f4067320 0x7fd5f40697d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd6040030c0 tx=0x7fd604016000 comp rx=0 tx=0).stop 2023-10-10T20:41:04.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.163+0000 7fd618858700 1 -- 172.21.15.89:0/841370328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd614159890 msgr2=0x7fd6141565e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:04.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.163+0000 7fd618858700 1 --2- 172.21.15.89:0/841370328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd614159890 0x7fd6141565e0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fd60800c960 tx=0x7fd6080019b0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.163+0000 7fd618858700 1 -- 172.21.15.89:0/841370328 shutdown_connections 2023-10-10T20:41:04.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.163+0000 7fd618858700 1 --2- 172.21.15.89:0/841370328 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd5f4067320 0x7fd5f40697d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.163+0000 7fd618858700 1 --2- 172.21.15.89:0/841370328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd614159c60 0x7fd6140b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.163+0000 7fd618858700 1 --2- 172.21.15.89:0/841370328 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd614069560 0x7fd614147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.163+0000 7fd618858700 1 --2- 172.21.15.89:0/841370328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd614159890 0x7fd6141565e0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.163+0000 7fd618858700 1 -- 172.21.15.89:0/841370328 >> 172.21.15.89:0/841370328 conn(0x7fd614068e20 msgr2=0x7fd614152b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:04.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.163+0000 7fd618858700 1 -- 172.21.15.89:0/841370328 shutdown_connections 2023-10-10T20:41:04.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.163+0000 7fd618858700 1 -- 172.21.15.89:0/841370328 wait complete. 2023-10-10T20:41:04.170 INFO:teuthology.orchestra.run.smithi089.stderr: removed all blocklist entries 2023-10-10T20:41:04.179 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:41:04.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.294+0000 7ff9e0a02700 1 Processor -- start 2023-10-10T20:41:04.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.294+0000 7ff9e0a02700 1 -- start start 2023-10-10T20:41:04.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9e0a02700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff9dc0c7bd0 0x7ff9dc15aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:04.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9e0a02700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9dc06bbe0 0x7ff9dc06c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:04.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9e0a02700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9dc15af50 con 0x7ff9dc1598d0 2023-10-10T20:41:04.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9e0a02700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9dc06e980 con 0x7ff9dc0c7bd0 2023-10-10T20:41:04.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9e0a02700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc0ce180 con 0x7ff9dc06bbe0 2023-10-10T20:41:04.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9dad9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff9dc1598d0 0x7ff9dc06ee10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45172/0 (socket says 172.21.15.89:45172) 2023-10-10T20:41:04.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9dad9d700 1 -- 172.21.15.89:0/1417195781 learned_addr learned my addr 172.21.15.89:0/1417195781 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:04.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9d9d9b700 1 -- 172.21.15.89:0/1417195781 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2076103274 0 0) 0x7ff9d4001490 con 0x7ff9dc1598d0 2023-10-10T20:41:04.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9db59e700 1 --2- 172.21.15.89:0/1417195781 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9dc06bbe0 0x7ff9dc06c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:04.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9da59c700 1 --2- 172.21.15.89:0/1417195781 >> v2:172.21.15.111:3300/0 conn(0x7ff9dc0c7bd0 0x7ff9dc15aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:04.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9d9d9b700 1 -- 172.21.15.89:0/1417195781 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9c8003570 con 0x7ff9dc1598d0 2023-10-10T20:41:04.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9d9d9b700 1 -- 172.21.15.89:0/1417195781 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 484589035 0 0) 0x7ff9c8003570 con 0x7ff9dc1598d0 2023-10-10T20:41:04.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9d9d9b700 1 -- 172.21.15.89:0/1417195781 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9dc06bbe0 msgr2=0x7ff9dc06c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:04.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9d9d9b700 1 --2- 172.21.15.89:0/1417195781 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9dc06bbe0 0x7ff9dc06c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9d9d9b700 1 -- 172.21.15.89:0/1417195781 >> v2:172.21.15.111:3300/0 conn(0x7ff9dc0c7bd0 msgr2=0x7ff9dc15aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:04.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9d9d9b700 1 --2- 172.21.15.89:0/1417195781 >> v2:172.21.15.111:3300/0 conn(0x7ff9dc0c7bd0 0x7ff9dc15aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9d9d9b700 1 -- 172.21.15.89:0/1417195781 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9dc0c80f0 con 0x7ff9dc1598d0 2023-10-10T20:41:04.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9d9d9b700 1 -- 172.21.15.89:0/1417195781 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7ff9d4001980 con 0x7ff9dc1598d0 2023-10-10T20:41:04.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9d9d9b700 1 -- 172.21.15.89:0/1417195781 >> v1:172.21.15.89:6789/0 conn(0x7ff9dc1598d0 legacy=0x7ff9dc06ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:04.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9d9d9b700 1 --2- 172.21.15.89:0/1417195781 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9dc06bbe0 0x7ff9c8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:04.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9d9d9b700 1 --2- 172.21.15.89:0/1417195781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9c8003fe0 0x7ff9c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:04.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9d9d9b700 1 --2- 172.21.15.89:0/1417195781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9c8004d50 0x7ff9c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:04.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9d9d9b700 1 -- 172.21.15.89:0/1417195781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc06e980 con 0x7ff9c8004d50 2023-10-10T20:41:04.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9d9d9b700 1 -- 172.21.15.89:0/1417195781 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc0ce180 con 0x7ff9c8003fe0 2023-10-10T20:41:04.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9d9d9b700 1 -- 172.21.15.89:0/1417195781 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c800ba90 con 0x7ff9dc06bbe0 2023-10-10T20:41:04.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.295+0000 7ff9db59e700 1 --2- 172.21.15.89:0/1417195781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9c8004d50 0x7ff9c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:04.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.296+0000 7ff9dad9d700 1 --2- 172.21.15.89:0/1417195781 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9dc06bbe0 0x7ff9c8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:04.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.296+0000 7ff9da59c700 1 --2- 172.21.15.89:0/1417195781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9c8003fe0 0x7ff9c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:04.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.296+0000 7ff9db59e700 1 -- 172.21.15.89:0/1417195781 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9dc06bbe0 msgr2=0x7ff9c8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:04.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.296+0000 7ff9db59e700 1 --2- 172.21.15.89:0/1417195781 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9dc06bbe0 0x7ff9c8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.296+0000 7ff9db59e700 1 -- 172.21.15.89:0/1417195781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9c8003fe0 msgr2=0x7ff9c8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:04.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.296+0000 7ff9db59e700 1 --2- 172.21.15.89:0/1417195781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9c8003fe0 0x7ff9c8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.296+0000 7ff9db59e700 1 -- 172.21.15.89:0/1417195781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c8003570 con 0x7ff9c8004d50 2023-10-10T20:41:04.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.296+0000 7ff9db59e700 1 --2- 172.21.15.89:0/1417195781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9c8004d50 0x7ff9c80051c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009c00 tx=0x7ff9cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9423bde71d3eec server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:04.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.296+0000 7ff9d9d9b700 1 -- 172.21.15.89:0/1417195781 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc00e6b0 con 0x7ff9c8004d50 2023-10-10T20:41:04.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.296+0000 7ff9d9d9b700 1 -- 172.21.15.89:0/1417195781 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc00e810 con 0x7ff9c8004d50 2023-10-10T20:41:04.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.296+0000 7ff9d9d9b700 1 -- 172.21.15.89:0/1417195781 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc00ebd0 con 0x7ff9c8004d50 2023-10-10T20:41:04.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.296+0000 7ff9e0a02700 1 -- 172.21.15.89:0/1417195781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9c8004d50 msgr2=0x7ff9c80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:04.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.296+0000 7ff9e0a02700 1 --2- 172.21.15.89:0/1417195781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9c8004d50 0x7ff9c80051c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009c00 tx=0x7ff9cc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.297+0000 7ff9db59e700 1 -- 172.21.15.89:0/1417195781 reap_dead start 2023-10-10T20:41:04.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.297+0000 7ff9e0a02700 1 -- 172.21.15.89:0/1417195781 shutdown_connections 2023-10-10T20:41:04.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.297+0000 7ff9e0a02700 1 -- 172.21.15.89:0/1417195781 >> 172.21.15.89:0/1417195781 conn(0x7ff9dc06b490 msgr2=0x7ff9dc0cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:04.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.297+0000 7ff9e0a02700 1 -- 172.21.15.89:0/1417195781 shutdown_connections 2023-10-10T20:41:04.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.297+0000 7ff9e0a02700 1 -- 172.21.15.89:0/1417195781 wait complete. 2023-10-10T20:41:04.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.298+0000 7ff9e0a02700 1 Processor -- start 2023-10-10T20:41:04.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.298+0000 7ff9e0a02700 1 -- start start 2023-10-10T20:41:04.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9e0a02700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9c8004d50 0x7ff9dc0c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:04.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9e0a02700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9dc15bf40 0x7ff9dc150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:04.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9e0a02700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9dc16a5f0 0x7ff9dc0b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:04.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9e0a02700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc150da0 con 0x7ff9dc16a5f0 2023-10-10T20:41:04.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9e0a02700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc169bf0 con 0x7ff9dc15bf40 2023-10-10T20:41:04.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9e0a02700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc0c7320 con 0x7ff9c8004d50 2023-10-10T20:41:04.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9db59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9dc16a5f0 0x7ff9dc0b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:04.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9db59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9dc16a5f0 0x7ff9dc0b1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54498/0 (socket says 172.21.15.89:54498) 2023-10-10T20:41:04.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9db59e700 1 -- 172.21.15.89:0/4065353110 learned_addr learned my addr 172.21.15.89:0/4065353110 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:04.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9dad9d700 1 --2- 172.21.15.89:0/4065353110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9c8004d50 0x7ff9dc0c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:04.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9da59c700 1 --2- 172.21.15.89:0/4065353110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9dc15bf40 0x7ff9dc150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:04.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9db59e700 1 -- 172.21.15.89:0/4065353110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9c8004d50 msgr2=0x7ff9dc0c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:04.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9db59e700 1 --2- 172.21.15.89:0/4065353110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9c8004d50 0x7ff9dc0c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9db59e700 1 -- 172.21.15.89:0/4065353110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9dc15bf40 msgr2=0x7ff9dc150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:04.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9db59e700 1 --2- 172.21.15.89:0/4065353110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9dc15bf40 0x7ff9dc150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9db59e700 1 -- 172.21.15.89:0/4065353110 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9dc16a160 con 0x7ff9dc16a5f0 2023-10-10T20:41:04.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9db59e700 1 --2- 172.21.15.89:0/4065353110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9dc16a5f0 0x7ff9dc0b1140 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8004730 tx=0x7ff9cc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:04.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9d2ffd700 1 -- 172.21.15.89:0/4065353110 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc022070 con 0x7ff9dc16a5f0 2023-10-10T20:41:04.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9d2ffd700 1 -- 172.21.15.89:0/4065353110 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc00eb90 con 0x7ff9dc16a5f0 2023-10-10T20:41:04.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9e0a02700 1 -- 172.21.15.89:0/4065353110 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9dc0c6d60 con 0x7ff9dc16a5f0 2023-10-10T20:41:04.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9d2ffd700 1 -- 172.21.15.89:0/4065353110 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc00e6b0 con 0x7ff9dc16a5f0 2023-10-10T20:41:04.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.299+0000 7ff9e0a02700 1 -- 172.21.15.89:0/4065353110 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9dc169df0 con 0x7ff9dc16a5f0 2023-10-10T20:41:04.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.300+0000 7ff9e0a02700 1 -- 172.21.15.89:0/4065353110 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9dc16a2f0 con 0x7ff9dc16a5f0 2023-10-10T20:41:04.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.301+0000 7ff9d2ffd700 1 -- 172.21.15.89:0/4065353110 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff9cc027900 con 0x7ff9dc16a5f0 2023-10-10T20:41:04.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.301+0000 7ff9d2ffd700 1 --2- 172.21.15.89:0/4065353110 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff9bc067320 0x7ff9bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:04.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.301+0000 7ff9d2ffd700 1 -- 172.21.15.89:0/4065353110 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff9cc000e40 con 0x7ff9dc16a5f0 2023-10-10T20:41:04.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.301+0000 7ff9dad9d700 1 --2- 172.21.15.89:0/4065353110 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff9bc067320 0x7ff9bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:04.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.302+0000 7ff9dad9d700 1 --2- 172.21.15.89:0/4065353110 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff9bc067320 0x7ff9bc0697d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400a290 tx=0x7ff9d4006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:04.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.305+0000 7ff9d2ffd700 1 -- 172.21.15.89:0/4065353110 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9cc053460 con 0x7ff9dc16a5f0 2023-10-10T20:41:04.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.565+0000 7ff9e0a02700 1 -- 172.21.15.89:0/4065353110 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff9dc04c3c0 con 0x7ff9dc16a5f0 2023-10-10T20:41:04.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.565+0000 7ff9d2ffd700 1 -- 172.21.15.89:0/4065353110 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff9cc056a00 con 0x7ff9dc16a5f0 2023-10-10T20:41:04.568 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:04.568 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:41:04.568 INFO:teuthology.orchestra.run.smithi089.stdout: "auth_dump": [ 2023-10-10T20:41:04.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:04.569 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.a", 2023-10-10T20:41:04.569 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlZLSNLhAA8WZTrsht6tAOP+gxaTXKMg==", 2023-10-10T20:41:04.569 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:04.569 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:41:04.570 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:41:04.570 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:41:04.570 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:04.570 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:04.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:04.571 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.b", 2023-10-10T20:41:04.571 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgyDEORAAMMoSxPFZc0bCIwWy8UsXgA==", 2023-10-10T20:41:04.571 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:04.571 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:41:04.572 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:41:04.572 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:41:04.572 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:04.572 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:04.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:04.573 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.c", 2023-10-10T20:41:04.573 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlMFxSNRAASuQEJFghLIOkTQWa4ckTvg==", 2023-10-10T20:41:04.573 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:04.574 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:41:04.574 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:41:04.574 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:41:04.574 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:04.574 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:04.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:04.575 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.d", 2023-10-10T20:41:04.575 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlPkvrBBAA2VLJmv2t8szTEGtCVwalJw==", 2023-10-10T20:41:04.575 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:04.576 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:41:04.576 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:41:04.576 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:41:04.576 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:04.576 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:04.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:04.577 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.0", 2023-10-10T20:41:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQArtSVltOcKHRAAk8+zUus04oTVI9KvfhjlXg==", 2023-10-10T20:41:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:41:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:41:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:04.579 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:04.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:04.579 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.1", 2023-10-10T20:41:04.579 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAutSVly8P0GhAAe1bvR63+ptJg84lizMNdQw==", 2023-10-10T20:41:04.579 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:04.580 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:41:04.580 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:41:04.580 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:04.580 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:04.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:04.581 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.2", 2023-10-10T20:41:04.581 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAxtSVl0x9fGRAAEKi7+6EW5mzPbebmz0+sgQ==", 2023-10-10T20:41:04.581 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:04.582 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:41:04.582 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:41:04.582 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:04.582 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:04.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:04.583 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.3", 2023-10-10T20:41:04.583 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA0tSVlfJyaGBAAa+8w54U6v7+npbdufKJruA==", 2023-10-10T20:41:04.583 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:04.583 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:41:04.584 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:41:04.584 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:04.584 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:04.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:04.585 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.4", 2023-10-10T20:41:04.585 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA6tSVlEQ/QFxAA/PRB3c90BuAlF6xekGfX9A==", 2023-10-10T20:41:04.585 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:04.585 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:41:04.585 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:41:04.586 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:04.586 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:04.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:04.586 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.5", 2023-10-10T20:41:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA9tSVlMOKrFRAAuBnIsEz2VmeSZi9XWzjlzA==", 2023-10-10T20:41:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:41:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:41:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:04.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:04.588 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.6", 2023-10-10T20:41:04.588 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBAtSVlH7PQFBAAHr6c7d4TYJEu/KSFEf+EOA==", 2023-10-10T20:41:04.588 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:04.589 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:41:04.590 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:41:04.590 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:04.590 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:04.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:04.591 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.7", 2023-10-10T20:41:04.591 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBDtSVlAhC1FBAAuE2BwAA/f7VOkZfVdqAG7A==", 2023-10-10T20:41:04.591 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:04.592 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:41:04.592 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:41:04.592 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:04.592 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:04.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:04.593 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.0", 2023-10-10T20:41:04.593 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlRd3HCBAAb0QP2VfTS7aelno1mFpGAA==", 2023-10-10T20:41:04.593 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:04.594 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:41:04.594 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow r", 2023-10-10T20:41:04.594 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:41:04.594 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:04.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:04.596 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.1", 2023-10-10T20:41:04.597 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlgBouERAAqPdE0KvJU3M53B6Y6AH5GQ==", 2023-10-10T20:41:04.598 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:04.599 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:41:04.599 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:41:04.599 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:41:04.599 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:41:04.599 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:04.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:04.600 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.2", 2023-10-10T20:41:04.600 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlXA4tDhAAoPVYQjECpzD+C67mnsY5IA==", 2023-10-10T20:41:04.600 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:04.601 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:41:04.601 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:41:04.601 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:41:04.601 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:41:04.601 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:04.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:04.602 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.3", 2023-10-10T20:41:04.602 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlZ5KGFhAAuKmZwbOfmmw1nAj5ry6tHw==", 2023-10-10T20:41:04.602 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:04.603 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:41:04.603 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:41:04.603 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:41:04.603 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:41:04.603 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:04.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:04.605 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.admin", 2023-10-10T20:41:04.605 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgpL2EhAAoDOki002rfBrauTgfu481Q==", 2023-10-10T20:41:04.605 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:04.605 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:41:04.605 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:41:04.606 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:41:04.606 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:04.607 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:04.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:04.608 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.x", 2023-10-10T20:41:04.608 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlBhaJKxAArraBxT5xWg515LfpJ7MlbA==", 2023-10-10T20:41:04.608 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:04.608 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:41:04.608 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:41:04.609 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:04.609 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:04.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:04.610 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.y", 2023-10-10T20:41:04.610 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlTZIyJxAAWAOsitHwsIxJWwp3b1Z98Q==", 2023-10-10T20:41:04.610 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:04.610 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:41:04.610 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:41:04.611 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:04.611 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.611 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:04.612 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-10T20:41:04.612 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:41:04.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.568+0000 7ff9e0a02700 1 -- 172.21.15.89:0/4065353110 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff9bc067320 msgr2=0x7ff9bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:04.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.568+0000 7ff9e0a02700 1 --2- 172.21.15.89:0/4065353110 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff9bc067320 0x7ff9bc0697d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400a290 tx=0x7ff9d4006040 comp rx=0 tx=0).stop 2023-10-10T20:41:04.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.568+0000 7ff9e0a02700 1 -- 172.21.15.89:0/4065353110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9dc16a5f0 msgr2=0x7ff9dc0b1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:04.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.568+0000 7ff9e0a02700 1 --2- 172.21.15.89:0/4065353110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9dc16a5f0 0x7ff9dc0b1140 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8004730 tx=0x7ff9cc01a000 comp rx=0 tx=0).stop 2023-10-10T20:41:04.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.568+0000 7ff9e0a02700 1 -- 172.21.15.89:0/4065353110 shutdown_connections 2023-10-10T20:41:04.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.568+0000 7ff9e0a02700 1 --2- 172.21.15.89:0/4065353110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9dc16a5f0 0x7ff9dc0b1140 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.568+0000 7ff9e0a02700 1 --2- 172.21.15.89:0/4065353110 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9c8004d50 0x7ff9dc0c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.568+0000 7ff9e0a02700 1 --2- 172.21.15.89:0/4065353110 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff9bc067320 0x7ff9bc0697d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.568+0000 7ff9e0a02700 1 --2- 172.21.15.89:0/4065353110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9dc15bf40 0x7ff9dc150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.568+0000 7ff9e0a02700 1 -- 172.21.15.89:0/4065353110 >> 172.21.15.89:0/4065353110 conn(0x7ff9dc06b490 msgr2=0x7ff9dc0b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:04.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.568+0000 7ff9e0a02700 1 -- 172.21.15.89:0/4065353110 shutdown_connections 2023-10-10T20:41:04.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.568+0000 7ff9e0a02700 1 -- 172.21.15.89:0/4065353110 wait complete. 2023-10-10T20:41:04.618 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:41:04.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.731+0000 7fb3bec7e700 1 Processor -- start 2023-10-10T20:41:04.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.731+0000 7fb3bec7e700 1 -- start start 2023-10-10T20:41:04.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.732+0000 7fb3bec7e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3b8159c60 0x7fb3b815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:04.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.732+0000 7fb3bec7e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb3b815db10 0x7fb3b8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:04.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.732+0000 7fb3bec7e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b815a160 con 0x7fb3b8159890 2023-10-10T20:41:04.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.732+0000 7fb3bec7e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b80d13a0 con 0x7fb3b815db10 2023-10-10T20:41:04.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.732+0000 7fb3bec7e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b815e140 con 0x7fb3b8159c60 2023-10-10T20:41:04.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.732+0000 7fb3bca1a700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb3b8159890 0x7fb3b815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45186/0 (socket says 172.21.15.89:45186) 2023-10-10T20:41:04.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.732+0000 7fb3bca1a700 1 -- 172.21.15.89:0/3305784170 learned_addr learned my addr 172.21.15.89:0/3305784170 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:04.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3af7fe700 1 -- 172.21.15.89:0/3305784170 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486135789 0 0) 0x7fb3b815a160 con 0x7fb3b8159890 2023-10-10T20:41:04.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3affff700 1 --2- 172.21.15.89:0/3305784170 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3b8159c60 0x7fb3b815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:04.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3af7fe700 1 -- 172.21.15.89:0/3305784170 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3a0003570 con 0x7fb3b8159890 2023-10-10T20:41:04.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3bd21b700 1 --2- 172.21.15.89:0/3305784170 >> v2:172.21.15.111:3300/0 conn(0x7fb3b815db10 0x7fb3b8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:04.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3af7fe700 1 -- 172.21.15.89:0/3305784170 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2234199005 0 0) 0x7fb3a0003570 con 0x7fb3b8159890 2023-10-10T20:41:04.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3af7fe700 1 -- 172.21.15.89:0/3305784170 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3b8159c60 msgr2=0x7fb3b815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:04.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3af7fe700 1 --2- 172.21.15.89:0/3305784170 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3b8159c60 0x7fb3b815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3af7fe700 1 -- 172.21.15.89:0/3305784170 >> v2:172.21.15.111:3300/0 conn(0x7fb3b815db10 msgr2=0x7fb3b8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:04.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3af7fe700 1 --2- 172.21.15.89:0/3305784170 >> v2:172.21.15.111:3300/0 conn(0x7fb3b815db10 0x7fb3b8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3af7fe700 1 -- 172.21.15.89:0/3305784170 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b815a3d0 con 0x7fb3b8159890 2023-10-10T20:41:04.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3af7fe700 1 -- 172.21.15.89:0/3305784170 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fb3b40017f0 con 0x7fb3b8159890 2023-10-10T20:41:04.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3af7fe700 1 -- 172.21.15.89:0/3305784170 >> v1:172.21.15.89:6789/0 conn(0x7fb3b8159890 legacy=0x7fb3b815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:04.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3af7fe700 1 --2- 172.21.15.89:0/3305784170 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3b8159c60 0x7fb3a0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:04.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3af7fe700 1 --2- 172.21.15.89:0/3305784170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3a0004090 0x7fb3a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:04.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3af7fe700 1 --2- 172.21.15.89:0/3305784170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3a0004d50 0x7fb3a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:04.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3af7fe700 1 -- 172.21.15.89:0/3305784170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b80d13a0 con 0x7fb3a0004d50 2023-10-10T20:41:04.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3af7fe700 1 -- 172.21.15.89:0/3305784170 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b815e140 con 0x7fb3a0004090 2023-10-10T20:41:04.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3af7fe700 1 -- 172.21.15.89:0/3305784170 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a000bbf0 con 0x7fb3b8159c60 2023-10-10T20:41:04.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3bd21b700 1 --2- 172.21.15.89:0/3305784170 >> v2:172.21.15.111:3300/0 conn(0x7fb3b815db10 0x7fb3b8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:04.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3affff700 1 --2- 172.21.15.89:0/3305784170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3a0004090 0x7fb3a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:04.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3bd21b700 1 --2- 172.21.15.89:0/3305784170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3a0004d50 0x7fb3a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:04.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3bca1a700 1 --2- 172.21.15.89:0/3305784170 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3b8159c60 0x7fb3a0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:04.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3bd21b700 1 -- 172.21.15.89:0/3305784170 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3b8159c60 msgr2=0x7fb3a0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:04.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3bd21b700 1 --2- 172.21.15.89:0/3305784170 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3b8159c60 0x7fb3a0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3bd21b700 1 -- 172.21.15.89:0/3305784170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3a0004090 msgr2=0x7fb3a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:04.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3bd21b700 1 --2- 172.21.15.89:0/3305784170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3a0004090 0x7fb3a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.733+0000 7fb3bd21b700 1 -- 172.21.15.89:0/3305784170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a0003570 con 0x7fb3a0004d50 2023-10-10T20:41:04.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.734+0000 7fb3affff700 1 --2- 172.21.15.89:0/3305784170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3a0004090 0x7fb3a0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:41:04.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.734+0000 7fb3bd21b700 1 --2- 172.21.15.89:0/3305784170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3a0004d50 0x7fb3a00051c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009ba0 tx=0x7fb3b0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd0304ffce79864 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:04.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.734+0000 7fb3af7fe700 1 -- 172.21.15.89:0/3305784170 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b000c070 con 0x7fb3a0004d50 2023-10-10T20:41:04.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.734+0000 7fb3af7fe700 1 -- 172.21.15.89:0/3305784170 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b000e5e0 con 0x7fb3a0004d50 2023-10-10T20:41:04.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.734+0000 7fb3af7fe700 1 -- 172.21.15.89:0/3305784170 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b000e790 con 0x7fb3a0004d50 2023-10-10T20:41:04.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.734+0000 7fb3bec7e700 1 -- 172.21.15.89:0/3305784170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3a0004d50 msgr2=0x7fb3a00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:04.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.734+0000 7fb3bec7e700 1 --2- 172.21.15.89:0/3305784170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3a0004d50 0x7fb3a00051c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009ba0 tx=0x7fb3b0008040 comp rx=0 tx=0).stop 2023-10-10T20:41:04.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.734+0000 7fb3bd21b700 1 -- 172.21.15.89:0/3305784170 reap_dead start 2023-10-10T20:41:04.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.734+0000 7fb3bec7e700 1 -- 172.21.15.89:0/3305784170 shutdown_connections 2023-10-10T20:41:04.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.734+0000 7fb3bec7e700 1 -- 172.21.15.89:0/3305784170 >> 172.21.15.89:0/3305784170 conn(0x7fb3b8068e20 msgr2=0x7fb3b80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:04.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.734+0000 7fb3bec7e700 1 -- 172.21.15.89:0/3305784170 shutdown_connections 2023-10-10T20:41:04.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.734+0000 7fb3bec7e700 1 -- 172.21.15.89:0/3305784170 wait complete. 2023-10-10T20:41:04.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.735+0000 7fb3bec7e700 1 Processor -- start 2023-10-10T20:41:04.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.736+0000 7fb3bec7e700 1 -- start start 2023-10-10T20:41:04.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.736+0000 7fb3bec7e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3b8152660 0x7fb3b816a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:04.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.736+0000 7fb3bec7e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3b80b42b0 0x7fb3b80b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:04.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.736+0000 7fb3bec7e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3b816d590 0x7fb3b816da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:04.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.736+0000 7fb3bec7e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b80b4c60 con 0x7fb3b8152660 2023-10-10T20:41:04.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.736+0000 7fb3bec7e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b816af00 con 0x7fb3b80b42b0 2023-10-10T20:41:04.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.736+0000 7fb3bec7e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b80b3930 con 0x7fb3b816d590 2023-10-10T20:41:04.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.736+0000 7fb3bca1a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3b8152660 0x7fb3b816a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:04.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.736+0000 7fb3bca1a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3b8152660 0x7fb3b816a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54508/0 (socket says 172.21.15.89:54508) 2023-10-10T20:41:04.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.736+0000 7fb3bca1a700 1 -- 172.21.15.89:0/1841830602 learned_addr learned my addr 172.21.15.89:0/1841830602 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:04.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.737+0000 7fb3affff700 1 --2- 172.21.15.89:0/1841830602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3b80b42b0 0x7fb3b80b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:04.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.737+0000 7fb3bd21b700 1 --2- 172.21.15.89:0/1841830602 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3b816d590 0x7fb3b816da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:04.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.737+0000 7fb3bca1a700 1 -- 172.21.15.89:0/1841830602 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3b816d590 msgr2=0x7fb3b816da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:04.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.737+0000 7fb3bca1a700 1 --2- 172.21.15.89:0/1841830602 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3b816d590 0x7fb3b816da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.737+0000 7fb3bca1a700 1 -- 172.21.15.89:0/1841830602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3b80b42b0 msgr2=0x7fb3b80b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:04.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.737+0000 7fb3bca1a700 1 --2- 172.21.15.89:0/1841830602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3b80b42b0 0x7fb3b80b4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.737+0000 7fb3bca1a700 1 -- 172.21.15.89:0/1841830602 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b80b3c10 con 0x7fb3b8152660 2023-10-10T20:41:04.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.737+0000 7fb3affff700 1 --2- 172.21.15.89:0/1841830602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3b80b42b0 0x7fb3b80b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:04.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.737+0000 7fb3bca1a700 1 --2- 172.21.15.89:0/1841830602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3b8152660 0x7fb3b816a9c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400a100 tx=0x7fb3b4002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:04.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.737+0000 7fb3acff9700 1 -- 172.21.15.89:0/1841830602 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4013600 con 0x7fb3b8152660 2023-10-10T20:41:04.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.737+0000 7fb3acff9700 1 -- 172.21.15.89:0/1841830602 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b4013760 con 0x7fb3b8152660 2023-10-10T20:41:04.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.737+0000 7fb3bec7e700 1 -- 172.21.15.89:0/1841830602 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b815bc20 con 0x7fb3b8152660 2023-10-10T20:41:04.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.737+0000 7fb3acff9700 1 -- 172.21.15.89:0/1841830602 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4013b60 con 0x7fb3b8152660 2023-10-10T20:41:04.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.737+0000 7fb3bec7e700 1 -- 172.21.15.89:0/1841830602 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b815c1e0 con 0x7fb3b8152660 2023-10-10T20:41:04.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.738+0000 7fb3bec7e700 1 -- 172.21.15.89:0/1841830602 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b815c400 con 0x7fb3b8152660 2023-10-10T20:41:04.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.738+0000 7fb3acff9700 1 -- 172.21.15.89:0/1841830602 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb3b401c400 con 0x7fb3b8152660 2023-10-10T20:41:04.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.739+0000 7fb3acff9700 1 --2- 172.21.15.89:0/1841830602 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb3a4067320 0x7fb3a40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:04.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.739+0000 7fb3acff9700 1 -- 172.21.15.89:0/1841830602 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb3b4085ba0 con 0x7fb3b8152660 2023-10-10T20:41:04.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.739+0000 7fb3affff700 1 --2- 172.21.15.89:0/1841830602 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb3a4067320 0x7fb3a40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:04.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.739+0000 7fb3affff700 1 --2- 172.21.15.89:0/1841830602 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb3a4067320 0x7fb3a40697d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8156c70 tx=0x7fb3a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:04.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.743+0000 7fb3acff9700 1 -- 172.21.15.89:0/1841830602 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3b4052b80 con 0x7fb3b8152660 2023-10-10T20:41:04.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.928+0000 7fb3bec7e700 1 -- 172.21.15.89:0/1841830602 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3b804c3c0 con 0x7fb3b8152660 2023-10-10T20:41:04.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.929+0000 7fb3acff9700 1 -- 172.21.15.89:0/1841830602 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+12136 (secure 0 0 0) 0x7fb3b406ce60 con 0x7fb3b8152660 2023-10-10T20:41:04.931 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:04.932 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":108,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:41:04.156663+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:41:04.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.931+0000 7fb3bec7e700 1 -- 172.21.15.89:0/1841830602 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb3a4067320 msgr2=0x7fb3a40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:04.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.931+0000 7fb3bec7e700 1 --2- 172.21.15.89:0/1841830602 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb3a4067320 0x7fb3a40697d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8156c70 tx=0x7fb3a8008040 comp rx=0 tx=0).stop 2023-10-10T20:41:04.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.931+0000 7fb3bec7e700 1 -- 172.21.15.89:0/1841830602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3b8152660 msgr2=0x7fb3b816a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:04.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.931+0000 7fb3bec7e700 1 --2- 172.21.15.89:0/1841830602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3b8152660 0x7fb3b816a9c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400a100 tx=0x7fb3b4002210 comp rx=0 tx=0).stop 2023-10-10T20:41:04.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.931+0000 7fb3bec7e700 1 -- 172.21.15.89:0/1841830602 shutdown_connections 2023-10-10T20:41:04.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.931+0000 7fb3bec7e700 1 --2- 172.21.15.89:0/1841830602 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb3b816d590 0x7fb3b816da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.931+0000 7fb3bec7e700 1 --2- 172.21.15.89:0/1841830602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3b8152660 0x7fb3b816a9c0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.931+0000 7fb3bec7e700 1 --2- 172.21.15.89:0/1841830602 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb3a4067320 0x7fb3a40697d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.931+0000 7fb3bec7e700 1 --2- 172.21.15.89:0/1841830602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3b80b42b0 0x7fb3b80b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:04.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.931+0000 7fb3bec7e700 1 -- 172.21.15.89:0/1841830602 >> 172.21.15.89:0/1841830602 conn(0x7fb3b8068e20 msgr2=0x7fb3b8158b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:04.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.931+0000 7fb3bec7e700 1 -- 172.21.15.89:0/1841830602 shutdown_connections 2023-10-10T20:41:04.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:04.931+0000 7fb3bec7e700 1 -- 172.21.15.89:0/1841830602 wait complete. 2023-10-10T20:41:04.945 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:41:04.946 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:41:05.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.061+0000 7fe6696fa700 1 Processor -- start 2023-10-10T20:41:05.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.061+0000 7fe6696fa700 1 -- start start 2023-10-10T20:41:05.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.062+0000 7fe6696fa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe664159c60 0x7fe66415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:05.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.062+0000 7fe6696fa700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe66415db10 0x7fe664069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:05.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.062+0000 7fe6696fa700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe66415a160 con 0x7fe664159890 2023-10-10T20:41:05.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.062+0000 7fe6696fa700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe66415df50 con 0x7fe664159c60 2023-10-10T20:41:05.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.062+0000 7fe6696fa700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6640d13a0 con 0x7fe66415db10 2023-10-10T20:41:05.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.062+0000 7fe65a7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe664159c60 0x7fe66415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:05.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.062+0000 7fe65a7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe664159c60 0x7fe66415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54518/0 (socket says 172.21.15.89:54518) 2023-10-10T20:41:05.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.062+0000 7fe65a7fc700 1 -- 172.21.15.89:0/1844730574 learned_addr learned my addr 172.21.15.89:0/1844730574 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:05.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.063+0000 7fe6637fe700 1 --2- 172.21.15.89:0/1844730574 >> v2:172.21.15.111:3300/0 conn(0x7fe66415db10 0x7fe664069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:05.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.063+0000 7fe65a7fc700 1 -- 172.21.15.89:0/1844730574 >> v2:172.21.15.111:3300/0 conn(0x7fe66415db10 msgr2=0x7fe664069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:05.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.063+0000 7fe65a7fc700 1 --2- 172.21.15.89:0/1844730574 >> v2:172.21.15.111:3300/0 conn(0x7fe66415db10 0x7fe664069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.063+0000 7fe65a7fc700 1 -- 172.21.15.89:0/1844730574 >> v1:172.21.15.111:6789/0 conn(0x7fe664159890 legacy=0x7fe66415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:05.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.063+0000 7fe65a7fc700 1 -- 172.21.15.89:0/1844730574 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66415e140 con 0x7fe664159c60 2023-10-10T20:41:05.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.063+0000 7fe6637fe700 1 --2- 172.21.15.89:0/1844730574 >> v2:172.21.15.111:3300/0 conn(0x7fe66415db10 0x7fe664069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:05.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.063+0000 7fe65a7fc700 1 --2- 172.21.15.89:0/1844730574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe664159c60 0x7fe66415d5d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fe66406a890 tx=0x7fe64c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4496d5c202fd5d08 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:05.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.063+0000 7fe6627fc700 1 -- 172.21.15.89:0/1844730574 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c00b820 con 0x7fe664159c60 2023-10-10T20:41:05.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.063+0000 7fe6627fc700 1 -- 172.21.15.89:0/1844730574 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64c00b980 con 0x7fe664159c60 2023-10-10T20:41:05.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.063+0000 7fe6627fc700 1 -- 172.21.15.89:0/1844730574 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c00bd20 con 0x7fe664159c60 2023-10-10T20:41:05.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.063+0000 7fe6696fa700 1 -- 172.21.15.89:0/1844730574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe664159c60 msgr2=0x7fe66415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:05.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.063+0000 7fe6696fa700 1 --2- 172.21.15.89:0/1844730574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe664159c60 0x7fe66415d5d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fe66406a890 tx=0x7fe64c009250 comp rx=0 tx=0).stop 2023-10-10T20:41:05.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.063+0000 7fe6696fa700 1 -- 172.21.15.89:0/1844730574 shutdown_connections 2023-10-10T20:41:05.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.063+0000 7fe6696fa700 1 --2- 172.21.15.89:0/1844730574 >> v2:172.21.15.111:3300/0 conn(0x7fe66415db10 0x7fe664069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.063+0000 7fe6696fa700 1 --2- 172.21.15.89:0/1844730574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe664159c60 0x7fe66415d5d0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.064+0000 7fe6696fa700 1 -- 172.21.15.89:0/1844730574 >> 172.21.15.89:0/1844730574 conn(0x7fe664068e20 msgr2=0x7fe6640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:05.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.064+0000 7fe6696fa700 1 -- 172.21.15.89:0/1844730574 shutdown_connections 2023-10-10T20:41:05.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.064+0000 7fe6696fa700 1 -- 172.21.15.89:0/1844730574 wait complete. 2023-10-10T20:41:05.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.065+0000 7fe6696fa700 1 Processor -- start 2023-10-10T20:41:05.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.065+0000 7fe6696fa700 1 -- start start 2023-10-10T20:41:05.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.065+0000 7fe6696fa700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe664159890 0x7fe664147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:05.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.065+0000 7fe6696fa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe664159c60 0x7fe66416a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:05.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.065+0000 7fe6696fa700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe66415db10 0x7fe6640c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:05.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.065+0000 7fe6696fa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66416afd0 con 0x7fe664159c60 2023-10-10T20:41:05.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.065+0000 7fe6696fa700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6640b5cc0 con 0x7fe664159890 2023-10-10T20:41:05.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.065+0000 7fe6696fa700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6640b50f0 con 0x7fe66415db10 2023-10-10T20:41:05.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.066+0000 7fe65a7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe664159c60 0x7fe66416a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:05.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.066+0000 7fe65a7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe664159c60 0x7fe66416a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54520/0 (socket says 172.21.15.89:54520) 2023-10-10T20:41:05.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.066+0000 7fe65a7fc700 1 -- 172.21.15.89:0/1672609455 learned_addr learned my addr 172.21.15.89:0/1672609455 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:05.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.066+0000 7fe662ffd700 1 --2- 172.21.15.89:0/1672609455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe664159890 0x7fe664147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:05.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.066+0000 7fe6637fe700 1 --2- 172.21.15.89:0/1672609455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe66415db10 0x7fe6640c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:05.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.066+0000 7fe65a7fc700 1 -- 172.21.15.89:0/1672609455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe66415db10 msgr2=0x7fe6640c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:05.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.066+0000 7fe65a7fc700 1 --2- 172.21.15.89:0/1672609455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe66415db10 0x7fe6640c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.066+0000 7fe65a7fc700 1 -- 172.21.15.89:0/1672609455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe664159890 msgr2=0x7fe664147520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:05.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.066+0000 7fe65a7fc700 1 --2- 172.21.15.89:0/1672609455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe664159890 0x7fe664147520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.066+0000 7fe65a7fc700 1 -- 172.21.15.89:0/1672609455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64c0094f0 con 0x7fe664159c60 2023-10-10T20:41:05.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.066+0000 7fe662ffd700 1 --2- 172.21.15.89:0/1672609455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe664159890 0x7fe664147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:05.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.066+0000 7fe65a7fc700 1 --2- 172.21.15.89:0/1672609455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe664159c60 0x7fe66416a7a0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fe64c0030c0 tx=0x7fe64c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:05.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.066+0000 7fe65b7fe700 1 -- 172.21.15.89:0/1672609455 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c011e60 con 0x7fe664159c60 2023-10-10T20:41:05.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.066+0000 7fe65b7fe700 1 -- 172.21.15.89:0/1672609455 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64c020070 con 0x7fe664159c60 2023-10-10T20:41:05.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.066+0000 7fe65b7fe700 1 -- 172.21.15.89:0/1672609455 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c01b6a0 con 0x7fe664159c60 2023-10-10T20:41:05.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.066+0000 7fe6696fa700 1 -- 172.21.15.89:0/1672609455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6640b5390 con 0x7fe664159c60 2023-10-10T20:41:05.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.066+0000 7fe6696fa700 1 -- 172.21.15.89:0/1672609455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6640b5950 con 0x7fe664159c60 2023-10-10T20:41:05.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.067+0000 7fe6696fa700 1 -- 172.21.15.89:0/1672609455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6640b5e50 con 0x7fe664159c60 2023-10-10T20:41:05.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.068+0000 7fe65b7fe700 1 -- 172.21.15.89:0/1672609455 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe64c022030 con 0x7fe664159c60 2023-10-10T20:41:05.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.068+0000 7fe65b7fe700 1 --2- 172.21.15.89:0/1672609455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe644067320 0x7fe6440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:05.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.068+0000 7fe65b7fe700 1 -- 172.21.15.89:0/1672609455 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe64c086830 con 0x7fe664159c60 2023-10-10T20:41:05.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.068+0000 7fe662ffd700 1 --2- 172.21.15.89:0/1672609455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe644067320 0x7fe6440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:05.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.068+0000 7fe662ffd700 1 --2- 172.21.15.89:0/1672609455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe644067320 0x7fe6440697d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe654007fd0 tx=0x7fe654009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:05.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.072+0000 7fe65b7fe700 1 -- 172.21.15.89:0/1672609455 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe64c053920 con 0x7fe664159c60 2023-10-10T20:41:05.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.256+0000 7fe6696fa700 1 -- 172.21.15.89:0/1672609455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6640b5e00 con 0x7fe664159c60 2023-10-10T20:41:05.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.256+0000 7fe65b7fe700 1 -- 172.21.15.89:0/1672609455 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe64c056f40 con 0x7fe664159c60 2023-10-10T20:41:05.259 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:41:05.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.258+0000 7fe6696fa700 1 -- 172.21.15.89:0/1672609455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe644067320 msgr2=0x7fe6440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:05.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.258+0000 7fe6696fa700 1 --2- 172.21.15.89:0/1672609455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe644067320 0x7fe6440697d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe654007fd0 tx=0x7fe654009040 comp rx=0 tx=0).stop 2023-10-10T20:41:05.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.258+0000 7fe6696fa700 1 -- 172.21.15.89:0/1672609455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe664159c60 msgr2=0x7fe66416a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:05.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.258+0000 7fe6696fa700 1 --2- 172.21.15.89:0/1672609455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe664159c60 0x7fe66416a7a0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fe64c0030c0 tx=0x7fe64c016000 comp rx=0 tx=0).stop 2023-10-10T20:41:05.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.258+0000 7fe6696fa700 1 -- 172.21.15.89:0/1672609455 shutdown_connections 2023-10-10T20:41:05.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.258+0000 7fe6696fa700 1 --2- 172.21.15.89:0/1672609455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe66415db10 0x7fe6640c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.258+0000 7fe6696fa700 1 --2- 172.21.15.89:0/1672609455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe644067320 0x7fe6440697d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.258+0000 7fe6696fa700 1 --2- 172.21.15.89:0/1672609455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe664159890 0x7fe664147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.258+0000 7fe6696fa700 1 --2- 172.21.15.89:0/1672609455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe664159c60 0x7fe66416a7a0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.258+0000 7fe6696fa700 1 -- 172.21.15.89:0/1672609455 >> 172.21.15.89:0/1672609455 conn(0x7fe664068e20 msgr2=0x7fe6641530e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:05.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.258+0000 7fe6696fa700 1 -- 172.21.15.89:0/1672609455 shutdown_connections 2023-10-10T20:41:05.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.258+0000 7fe6696fa700 1 -- 172.21.15.89:0/1672609455 wait complete. 2023-10-10T20:41:05.273 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:41:05.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.385+0000 7f684c6ca700 1 Processor -- start 2023-10-10T20:41:05.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.386+0000 7f684c6ca700 1 -- start start 2023-10-10T20:41:05.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.386+0000 7f684c6ca700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6844159c60 0x7f684415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:05.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.386+0000 7f684c6ca700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f684415db10 0x7f6844069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:05.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.386+0000 7f684c6ca700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684415a160 con 0x7f6844159890 2023-10-10T20:41:05.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.386+0000 7f684c6ca700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684415df50 con 0x7f684415db10 2023-10-10T20:41:05.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.386+0000 7f684c6ca700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68440d13a0 con 0x7f6844159c60 2023-10-10T20:41:05.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.387+0000 7f684ac67700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f684415db10 0x7f6844069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:05.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.387+0000 7f684ac67700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f684415db10 0x7f6844069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54530/0 (socket says 172.21.15.89:54530) 2023-10-10T20:41:05.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.387+0000 7f684ac67700 1 -- 172.21.15.89:0/3849815852 learned_addr learned my addr 172.21.15.89:0/3849815852 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:05.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.387+0000 7f6849c65700 1 --2- 172.21.15.89:0/3849815852 >> v2:172.21.15.111:3300/0 conn(0x7f6844159c60 0x7f684415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:05.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.387+0000 7f684ac67700 1 -- 172.21.15.89:0/3849815852 >> v2:172.21.15.111:3300/0 conn(0x7f6844159c60 msgr2=0x7f684415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:05.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.387+0000 7f684ac67700 1 --2- 172.21.15.89:0/3849815852 >> v2:172.21.15.111:3300/0 conn(0x7f6844159c60 0x7f684415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.387+0000 7f684ac67700 1 -- 172.21.15.89:0/3849815852 >> v1:172.21.15.111:6789/0 conn(0x7f6844159890 legacy=0x7f684415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:05.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.387+0000 7f684ac67700 1 -- 172.21.15.89:0/3849815852 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684415e140 con 0x7f684415db10 2023-10-10T20:41:05.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.387+0000 7f684ac67700 1 --2- 172.21.15.89:0/3849815852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f684415db10 0x7f6844069560 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f68440b67f0 tx=0x7f683c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114d48bf8c91595 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:05.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.387+0000 7f6849464700 1 -- 172.21.15.89:0/3849815852 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c00e070 con 0x7f684415db10 2023-10-10T20:41:05.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.387+0000 7f6849464700 1 -- 172.21.15.89:0/3849815852 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683c00ba50 con 0x7f684415db10 2023-10-10T20:41:05.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.387+0000 7f6849464700 1 -- 172.21.15.89:0/3849815852 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c0092e0 con 0x7f684415db10 2023-10-10T20:41:05.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.388+0000 7f684c6ca700 1 -- 172.21.15.89:0/3849815852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f684415db10 msgr2=0x7f6844069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:05.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.388+0000 7f684c6ca700 1 --2- 172.21.15.89:0/3849815852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f684415db10 0x7f6844069560 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f68440b67f0 tx=0x7f683c009840 comp rx=0 tx=0).stop 2023-10-10T20:41:05.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.388+0000 7f684c6ca700 1 -- 172.21.15.89:0/3849815852 shutdown_connections 2023-10-10T20:41:05.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.388+0000 7f684c6ca700 1 --2- 172.21.15.89:0/3849815852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f684415db10 0x7f6844069560 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.388+0000 7f684c6ca700 1 --2- 172.21.15.89:0/3849815852 >> v2:172.21.15.111:3300/0 conn(0x7f6844159c60 0x7f684415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.388+0000 7f684c6ca700 1 -- 172.21.15.89:0/3849815852 >> 172.21.15.89:0/3849815852 conn(0x7f6844068e20 msgr2=0x7f68440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:05.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.388+0000 7f684c6ca700 1 -- 172.21.15.89:0/3849815852 shutdown_connections 2023-10-10T20:41:05.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.388+0000 7f684c6ca700 1 -- 172.21.15.89:0/3849815852 wait complete. 2023-10-10T20:41:05.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.389+0000 7f684c6ca700 1 Processor -- start 2023-10-10T20:41:05.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.389+0000 7f684c6ca700 1 -- start start 2023-10-10T20:41:05.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f684c6ca700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6844159890 0x7f684416cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:05.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f684c6ca700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6844159c60 0x7f6844140d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:05.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f684c6ca700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f684415db10 0x7f68441412c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:05.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f684c6ca700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6844141800 con 0x7f6844159890 2023-10-10T20:41:05.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f684c6ca700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6844152100 con 0x7f6844159c60 2023-10-10T20:41:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f684c6ca700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68441523e0 con 0x7f684415db10 2023-10-10T20:41:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f684a466700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6844159890 0x7f684416cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f684a466700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6844159890 0x7f684416cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54536/0 (socket says 172.21.15.89:54536) 2023-10-10T20:41:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f684a466700 1 -- 172.21.15.89:0/2619038999 learned_addr learned my addr 172.21.15.89:0/2619038999 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f6849c65700 1 --2- 172.21.15.89:0/2619038999 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6844159c60 0x7f6844140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f684ac67700 1 --2- 172.21.15.89:0/2619038999 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f684415db10 0x7f68441412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f684a466700 1 -- 172.21.15.89:0/2619038999 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f684415db10 msgr2=0x7f68441412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f684a466700 1 --2- 172.21.15.89:0/2619038999 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f684415db10 0x7f68441412c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f684a466700 1 -- 172.21.15.89:0/2619038999 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6844159c60 msgr2=0x7f6844140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f684a466700 1 --2- 172.21.15.89:0/2619038999 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6844159c60 0x7f6844140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f684a466700 1 -- 172.21.15.89:0/2619038999 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c009ae0 con 0x7f6844159890 2023-10-10T20:41:05.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f6849c65700 1 --2- 172.21.15.89:0/2619038999 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6844159c60 0x7f6844140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:05.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f684a466700 1 --2- 172.21.15.89:0/2619038999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6844159890 0x7f684416cf70 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f683400c590 tx=0x7f683400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:05.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f68427fc700 1 -- 172.21.15.89:0/2619038999 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6834011590 con 0x7f6844159890 2023-10-10T20:41:05.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f68427fc700 1 -- 172.21.15.89:0/2619038999 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6834011710 con 0x7f6844159890 2023-10-10T20:41:05.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f684c6ca700 1 -- 172.21.15.89:0/2619038999 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68441526c0 con 0x7f6844159890 2023-10-10T20:41:05.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f68427fc700 1 -- 172.21.15.89:0/2619038999 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6834011ad0 con 0x7f6844159890 2023-10-10T20:41:05.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.390+0000 7f684c6ca700 1 -- 172.21.15.89:0/2619038999 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6844152c80 con 0x7f6844159890 2023-10-10T20:41:05.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.391+0000 7f684c6ca700 1 -- 172.21.15.89:0/2619038999 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684404c3c0 con 0x7f6844159890 2023-10-10T20:41:05.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.392+0000 7f68427fc700 1 -- 172.21.15.89:0/2619038999 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f683401b3b0 con 0x7f6844159890 2023-10-10T20:41:05.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.392+0000 7f68427fc700 1 --2- 172.21.15.89:0/2619038999 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6830067320 0x7f68300697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:05.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.392+0000 7f68427fc700 1 -- 172.21.15.89:0/2619038999 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6834083ea0 con 0x7f6844159890 2023-10-10T20:41:05.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.392+0000 7f6849c65700 1 --2- 172.21.15.89:0/2619038999 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6830067320 0x7f68300697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:05.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.393+0000 7f6849c65700 1 --2- 172.21.15.89:0/2619038999 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6830067320 0x7f68300697d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6838006fd0 tx=0x7f6838008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:05.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.396+0000 7f68427fc700 1 -- 172.21.15.89:0/2619038999 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6834050e40 con 0x7f6844159890 2023-10-10T20:41:05.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.582+0000 7f684c6ca700 1 -- 172.21.15.89:0/2619038999 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f684406a5b0 con 0x7f6844159890 2023-10-10T20:41:05.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.583+0000 7f68427fc700 1 -- 172.21.15.89:0/2619038999 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v108) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6834017070 con 0x7f6844159890 2023-10-10T20:41:05.585 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:05.585 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:41:05.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:05.586 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2023-10-10T20:41:05.586 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2023-10-10T20:41:05.586 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:05.586 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:41:05.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.584+0000 7f684c6ca700 1 -- 172.21.15.89:0/2619038999 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6830067320 msgr2=0x7f68300697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:05.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.584+0000 7f684c6ca700 1 --2- 172.21.15.89:0/2619038999 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6830067320 0x7f68300697d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6838006fd0 tx=0x7f6838008040 comp rx=0 tx=0).stop 2023-10-10T20:41:05.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.584+0000 7f684c6ca700 1 -- 172.21.15.89:0/2619038999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6844159890 msgr2=0x7f684416cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:05.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.584+0000 7f684c6ca700 1 --2- 172.21.15.89:0/2619038999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6844159890 0x7f684416cf70 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f683400c590 tx=0x7f683400c8a0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.584+0000 7f684c6ca700 1 -- 172.21.15.89:0/2619038999 shutdown_connections 2023-10-10T20:41:05.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.584+0000 7f684c6ca700 1 --2- 172.21.15.89:0/2619038999 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f684415db10 0x7f68441412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.584+0000 7f684c6ca700 1 --2- 172.21.15.89:0/2619038999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6844159890 0x7f684416cf70 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.584+0000 7f684c6ca700 1 --2- 172.21.15.89:0/2619038999 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6830067320 0x7f68300697d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.584+0000 7f684c6ca700 1 --2- 172.21.15.89:0/2619038999 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6844159c60 0x7f6844140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.585+0000 7f684c6ca700 1 -- 172.21.15.89:0/2619038999 >> 172.21.15.89:0/2619038999 conn(0x7f6844068e20 msgr2=0x7f684414d430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:05.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.585+0000 7f684c6ca700 1 -- 172.21.15.89:0/2619038999 shutdown_connections 2023-10-10T20:41:05.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.585+0000 7f684c6ca700 1 -- 172.21.15.89:0/2619038999 wait complete. 2023-10-10T20:41:05.598 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:41:05.599 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:41:05.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.714+0000 7f0a59826700 1 Processor -- start 2023-10-10T20:41:05.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.714+0000 7f0a59826700 1 -- start start 2023-10-10T20:41:05.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.715+0000 7f0a59826700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0a54155360 0x7f0a540b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:05.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.715+0000 7f0a59826700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0a540b40f0 0x7f0a540ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:05.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.715+0000 7f0a59826700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a5415c320 con 0x7f0a540ceb60 2023-10-10T20:41:05.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.715+0000 7f0a59826700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a540b4590 con 0x7f0a540b40f0 2023-10-10T20:41:05.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.715+0000 7f0a59826700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a540cefc0 con 0x7f0a54155360 2023-10-10T20:41:05.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.715+0000 7f0a4a7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0a540b40f0 0x7f0a540ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:05.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.715+0000 7f0a537fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0a540ceb60 0x7f0a5406d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45194/0 (socket says 172.21.15.89:45194) 2023-10-10T20:41:05.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.715+0000 7f0a537fe700 1 -- 172.21.15.89:0/1093980813 learned_addr learned my addr 172.21.15.89:0/1093980813 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:05.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.715+0000 7f0a4a7fc700 1 -- 172.21.15.89:0/1093980813 >> v2:172.21.15.111:3300/0 conn(0x7f0a54155360 msgr2=0x7f0a540b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.715+0000 7f0a52ffd700 1 --2- 172.21.15.89:0/1093980813 >> v2:172.21.15.111:3300/0 conn(0x7f0a54155360 0x7f0a540b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.715+0000 7f0a4a7fc700 1 --2- 172.21.15.89:0/1093980813 >> v2:172.21.15.111:3300/0 conn(0x7f0a54155360 0x7f0a540b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.715+0000 7f0a4a7fc700 1 -- 172.21.15.89:0/1093980813 >> v1:172.21.15.89:6789/0 conn(0x7f0a540ceb60 legacy=0x7f0a5406d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.715+0000 7f0a527fc700 1 -- 172.21.15.89:0/1093980813 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386500604 0 0) 0x7f0a5415c320 con 0x7f0a540ceb60 2023-10-10T20:41:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.715+0000 7f0a4a7fc700 1 -- 172.21.15.89:0/1093980813 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5415b710 con 0x7f0a540b40f0 2023-10-10T20:41:05.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.716+0000 7f0a52ffd700 1 --2- 172.21.15.89:0/1093980813 >> v2:172.21.15.111:3300/0 conn(0x7f0a54155360 0x7f0a540b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:05.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.716+0000 7f0a4a7fc700 1 --2- 172.21.15.89:0/1093980813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a540b40f0 0x7f0a540ce620 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f0a54070e00 tx=0x7f0a3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103440656c0ee01d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:05.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.716+0000 7f0a527fc700 1 -- 172.21.15.89:0/1093980813 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c00b6b0 con 0x7f0a540b40f0 2023-10-10T20:41:05.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.716+0000 7f0a527fc700 1 -- 172.21.15.89:0/1093980813 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3c00b810 con 0x7f0a540b40f0 2023-10-10T20:41:05.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.716+0000 7f0a527fc700 1 -- 172.21.15.89:0/1093980813 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c00bbb0 con 0x7f0a540b40f0 2023-10-10T20:41:05.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.716+0000 7f0a59826700 1 -- 172.21.15.89:0/1093980813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a540b40f0 msgr2=0x7f0a540ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:05.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.716+0000 7f0a59826700 1 --2- 172.21.15.89:0/1093980813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a540b40f0 0x7f0a540ce620 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f0a54070e00 tx=0x7f0a3c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.716+0000 7f0a59826700 1 -- 172.21.15.89:0/1093980813 shutdown_connections 2023-10-10T20:41:05.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.716+0000 7f0a59826700 1 --2- 172.21.15.89:0/1093980813 >> v2:172.21.15.111:3300/0 conn(0x7f0a54155360 0x7f0a540b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.716+0000 7f0a59826700 1 --2- 172.21.15.89:0/1093980813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a540b40f0 0x7f0a540ce620 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.716+0000 7f0a59826700 1 -- 172.21.15.89:0/1093980813 >> 172.21.15.89:0/1093980813 conn(0x7f0a5406baf0 msgr2=0x7f0a5406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.716+0000 7f0a59826700 1 -- 172.21.15.89:0/1093980813 shutdown_connections 2023-10-10T20:41:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.716+0000 7f0a59826700 1 -- 172.21.15.89:0/1093980813 wait complete. 2023-10-10T20:41:05.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.718+0000 7f0a59826700 1 Processor -- start 2023-10-10T20:41:05.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.718+0000 7f0a59826700 1 -- start start 2023-10-10T20:41:05.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.718+0000 7f0a59826700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a540b40f0 0x7f0a54157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:05.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.718+0000 7f0a59826700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a540ceb60 0x7f0a5414b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:05.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.718+0000 7f0a59826700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a54155360 0x7f0a5406d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:05.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.718+0000 7f0a59826700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5414b7d0 con 0x7f0a54155360 2023-10-10T20:41:05.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.718+0000 7f0a59826700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a54156740 con 0x7f0a540b40f0 2023-10-10T20:41:05.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.718+0000 7f0a59826700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a54156a20 con 0x7f0a540ceb60 2023-10-10T20:41:05.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.718+0000 7f0a537fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a54155360 0x7f0a5406d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:05.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.719+0000 7f0a537fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a54155360 0x7f0a5406d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54562/0 (socket says 172.21.15.89:54562) 2023-10-10T20:41:05.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.719+0000 7f0a537fe700 1 -- 172.21.15.89:0/2514357076 learned_addr learned my addr 172.21.15.89:0/2514357076 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:05.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.719+0000 7f0a52ffd700 1 --2- 172.21.15.89:0/2514357076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a540b40f0 0x7f0a54157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:05.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.719+0000 7f0a4a7fc700 1 --2- 172.21.15.89:0/2514357076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a540ceb60 0x7f0a5414b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:05.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.719+0000 7f0a537fe700 1 -- 172.21.15.89:0/2514357076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a540ceb60 msgr2=0x7f0a5414b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:05.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.719+0000 7f0a537fe700 1 --2- 172.21.15.89:0/2514357076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a540ceb60 0x7f0a5414b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.719+0000 7f0a537fe700 1 -- 172.21.15.89:0/2514357076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a540b40f0 msgr2=0x7f0a54157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:05.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.719+0000 7f0a537fe700 1 --2- 172.21.15.89:0/2514357076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a540b40f0 0x7f0a54157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:05.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.719+0000 7f0a537fe700 1 -- 172.21.15.89:0/2514357076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c009420 con 0x7f0a54155360 2023-10-10T20:41:05.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.719+0000 7f0a537fe700 1 --2- 172.21.15.89:0/2514357076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a54155360 0x7f0a5406d130 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00cc90 tx=0x7f0a4c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:05.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.719+0000 7f0a4b7fe700 1 -- 172.21.15.89:0/2514357076 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c00a8b0 con 0x7f0a54155360 2023-10-10T20:41:05.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.719+0000 7f0a4b7fe700 1 -- 172.21.15.89:0/2514357076 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c00aa10 con 0x7f0a54155360 2023-10-10T20:41:05.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.719+0000 7f0a59826700 1 -- 172.21.15.89:0/2514357076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a54156d00 con 0x7f0a54155360 2023-10-10T20:41:05.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.719+0000 7f0a4b7fe700 1 -- 172.21.15.89:0/2514357076 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c00add0 con 0x7f0a54155360 2023-10-10T20:41:05.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.719+0000 7f0a59826700 1 -- 172.21.15.89:0/2514357076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5406d670 con 0x7f0a54155360 2023-10-10T20:41:05.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.720+0000 7f0a59826700 1 -- 172.21.15.89:0/2514357076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a540718b0 con 0x7f0a54155360 2023-10-10T20:41:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.720+0000 7f0a4b7fe700 1 -- 172.21.15.89:0/2514357076 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0a4c007300 con 0x7f0a54155360 2023-10-10T20:41:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.721+0000 7f0a4b7fe700 1 --2- 172.21.15.89:0/2514357076 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0a34067320 0x7f0a340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.721+0000 7f0a4b7fe700 1 -- 172.21.15.89:0/2514357076 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f0a4c084c40 con 0x7f0a54155360 2023-10-10T20:41:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.721+0000 7f0a52ffd700 1 --2- 172.21.15.89:0/2514357076 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0a34067320 0x7f0a340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.721+0000 7f0a52ffd700 1 --2- 172.21.15.89:0/2514357076 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0a34067320 0x7f0a340697d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0a44006fd0 tx=0x7f0a44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.725+0000 7f0a4b7fe700 1 -- 172.21.15.89:0/2514357076 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a4c051ca0 con 0x7f0a54155360 2023-10-10T20:41:05.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:05.909+0000 7f0a59826700 1 -- 172.21.15.89:0/2514357076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0a5415b9f0 con 0x7f0a54155360 2023-10-10T20:41:06.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.171+0000 7f0a4b7fe700 1 -- 172.21.15.89:0/2514357076 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v109) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0a4c0552c0 con 0x7f0a54155360 2023-10-10T20:41:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.174+0000 7f0a59826700 1 -- 172.21.15.89:0/2514357076 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0a34067320 msgr2=0x7f0a340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.174+0000 7f0a59826700 1 --2- 172.21.15.89:0/2514357076 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0a34067320 0x7f0a340697d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0a44006fd0 tx=0x7f0a44008040 comp rx=0 tx=0).stop 2023-10-10T20:41:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.174+0000 7f0a59826700 1 -- 172.21.15.89:0/2514357076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a54155360 msgr2=0x7f0a5406d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.174+0000 7f0a59826700 1 --2- 172.21.15.89:0/2514357076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a54155360 0x7f0a5406d130 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00cc90 tx=0x7f0a4c001a80 comp rx=0 tx=0).stop 2023-10-10T20:41:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.174+0000 7f0a59826700 1 -- 172.21.15.89:0/2514357076 shutdown_connections 2023-10-10T20:41:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.174+0000 7f0a59826700 1 --2- 172.21.15.89:0/2514357076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a54155360 0x7f0a5406d130 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.174+0000 7f0a59826700 1 --2- 172.21.15.89:0/2514357076 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0a34067320 0x7f0a340697d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.174+0000 7f0a59826700 1 --2- 172.21.15.89:0/2514357076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a540b40f0 0x7f0a54157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.174+0000 7f0a59826700 1 --2- 172.21.15.89:0/2514357076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0a540ceb60 0x7f0a5414b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.174+0000 7f0a59826700 1 -- 172.21.15.89:0/2514357076 >> 172.21.15.89:0/2514357076 conn(0x7f0a5406baf0 msgr2=0x7f0a5414c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.174+0000 7f0a59826700 1 -- 172.21.15.89:0/2514357076 shutdown_connections 2023-10-10T20:41:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.174+0000 7f0a59826700 1 -- 172.21.15.89:0/2514357076 wait complete. 2023-10-10T20:41:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2023-10-10T20:41:06.193 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:41:06.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.313+0000 7fef6b59e700 1 Processor -- start 2023-10-10T20:41:06.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.313+0000 7fef6b59e700 1 -- start start 2023-10-10T20:41:06.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.314+0000 7fef6b59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fef6c1595d0 0x7fef6c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:06.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.314+0000 7fef6b59e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fef6c0cf480 0x7fef6c15d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:06.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.314+0000 7fef6b59e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6c15dbc0 con 0x7fef6c15d760 2023-10-10T20:41:06.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.314+0000 7fef6b59e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6c0b10f0 con 0x7fef6c0cf480 2023-10-10T20:41:06.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.314+0000 7fef6b59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c0cd530 con 0x7fef6c1595d0 2023-10-10T20:41:06.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.314+0000 7fef69d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fef6c0cf480 0x7fef6c15d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:06.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.314+0000 7fef69d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fef6c0cf480 0x7fef6c15d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54578/0 (socket says 172.21.15.89:54578) 2023-10-10T20:41:06.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.314+0000 7fef69d9b700 1 -- 172.21.15.89:0/319133528 learned_addr learned my addr 172.21.15.89:0/319133528 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:06.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.315+0000 7fef6a59c700 1 --2- 172.21.15.89:0/319133528 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fef6c1595d0 0x7fef6c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.315+0000 7fef69d9b700 1 -- 172.21.15.89:0/319133528 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fef6c1595d0 msgr2=0x7fef6c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.315+0000 7fef69d9b700 1 --2- 172.21.15.89:0/319133528 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fef6c1595d0 0x7fef6c0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.315+0000 7fef69d9b700 1 -- 172.21.15.89:0/319133528 >> v1:172.21.15.111:6789/0 conn(0x7fef6c15d760 legacy=0x7fef6c1599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.315+0000 7fef69d9b700 1 -- 172.21.15.89:0/319133528 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c15e000 con 0x7fef6c0cf480 2023-10-10T20:41:06.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.315+0000 7fef69d9b700 1 --2- 172.21.15.89:0/319133528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef6c0cf480 0x7fef6c15d220 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fef600099b0 tx=0x7fef60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f4581a49b8cc80 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:06.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.315+0000 7fef6959a700 1 -- 172.21.15.89:0/319133528 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6000b820 con 0x7fef6c0cf480 2023-10-10T20:41:06.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.315+0000 7fef6959a700 1 -- 172.21.15.89:0/319133528 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6000b980 con 0x7fef6c0cf480 2023-10-10T20:41:06.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.315+0000 7fef6959a700 1 -- 172.21.15.89:0/319133528 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6000bd20 con 0x7fef6c0cf480 2023-10-10T20:41:06.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.316+0000 7fef6b59e700 1 -- 172.21.15.89:0/319133528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef6c0cf480 msgr2=0x7fef6c15d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:06.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.316+0000 7fef6b59e700 1 --2- 172.21.15.89:0/319133528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef6c0cf480 0x7fef6c15d220 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fef600099b0 tx=0x7fef60009250 comp rx=0 tx=0).stop 2023-10-10T20:41:06.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.316+0000 7fef6b59e700 1 -- 172.21.15.89:0/319133528 shutdown_connections 2023-10-10T20:41:06.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.316+0000 7fef6b59e700 1 --2- 172.21.15.89:0/319133528 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fef6c1595d0 0x7fef6c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:06.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.316+0000 7fef6b59e700 1 --2- 172.21.15.89:0/319133528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef6c0cf480 0x7fef6c15d220 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:06.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.316+0000 7fef6b59e700 1 -- 172.21.15.89:0/319133528 >> 172.21.15.89:0/319133528 conn(0x7fef6c0d05f0 msgr2=0x7fef6c0d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:06.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.316+0000 7fef6b59e700 1 -- 172.21.15.89:0/319133528 shutdown_connections 2023-10-10T20:41:06.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.316+0000 7fef6b59e700 1 -- 172.21.15.89:0/319133528 wait complete. 2023-10-10T20:41:06.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.317+0000 7fef6b59e700 1 Processor -- start 2023-10-10T20:41:06.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.317+0000 7fef6b59e700 1 -- start start 2023-10-10T20:41:06.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.317+0000 7fef6b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef6c0cf480 0x7fef6c0c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:06.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.317+0000 7fef6b59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fef6c1595d0 0x7fef6c169ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:06.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.317+0000 7fef6b59e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef6c15d760 0x7fef6c1578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:06.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.317+0000 7fef6b59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c16aa60 con 0x7fef6c0cf480 2023-10-10T20:41:06.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.317+0000 7fef6b59e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c16a1c0 con 0x7fef6c15d760 2023-10-10T20:41:06.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.317+0000 7fef6b59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c16a4a0 con 0x7fef6c1595d0 2023-10-10T20:41:06.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.318+0000 7fef6a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef6c0cf480 0x7fef6c0c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:06.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.318+0000 7fef6a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef6c0cf480 0x7fef6c0c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54586/0 (socket says 172.21.15.89:54586) 2023-10-10T20:41:06.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.318+0000 7fef6a59c700 1 -- 172.21.15.89:0/3729034526 learned_addr learned my addr 172.21.15.89:0/3729034526 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:06.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.318+0000 7fef69d9b700 1 --2- 172.21.15.89:0/3729034526 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fef6c1595d0 0x7fef6c169ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:06.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.318+0000 7fef6ad9d700 1 --2- 172.21.15.89:0/3729034526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef6c15d760 0x7fef6c1578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:06.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.318+0000 7fef6a59c700 1 -- 172.21.15.89:0/3729034526 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fef6c1595d0 msgr2=0x7fef6c169ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:06.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.318+0000 7fef6a59c700 1 --2- 172.21.15.89:0/3729034526 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fef6c1595d0 0x7fef6c169ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:06.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.318+0000 7fef6a59c700 1 -- 172.21.15.89:0/3729034526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef6c15d760 msgr2=0x7fef6c1578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:06.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.318+0000 7fef6a59c700 1 --2- 172.21.15.89:0/3729034526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef6c15d760 0x7fef6c1578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:06.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.318+0000 7fef6a59c700 1 -- 172.21.15.89:0/3729034526 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef600094f0 con 0x7fef6c0cf480 2023-10-10T20:41:06.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.318+0000 7fef6a59c700 1 --2- 172.21.15.89:0/3729034526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef6c0cf480 0x7fef6c0c5390 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00a320 tx=0x7fef5c007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:06.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.318+0000 7fef5a7fc700 1 -- 172.21.15.89:0/3729034526 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c00e450 con 0x7fef6c0cf480 2023-10-10T20:41:06.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.318+0000 7fef5a7fc700 1 -- 172.21.15.89:0/3729034526 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c00e5b0 con 0x7fef6c0cf480 2023-10-10T20:41:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.318+0000 7fef6b59e700 1 -- 172.21.15.89:0/3729034526 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6c16a780 con 0x7fef6c0cf480 2023-10-10T20:41:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.318+0000 7fef5a7fc700 1 -- 172.21.15.89:0/3729034526 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c00e950 con 0x7fef6c0cf480 2023-10-10T20:41:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.318+0000 7fef6b59e700 1 -- 172.21.15.89:0/3729034526 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6c157e30 con 0x7fef6c0cf480 2023-10-10T20:41:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.319+0000 7fef6b59e700 1 -- 172.21.15.89:0/3729034526 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6c16d0c0 con 0x7fef6c0cf480 2023-10-10T20:41:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.320+0000 7fef5a7fc700 1 -- 172.21.15.89:0/3729034526 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fef5c003020 con 0x7fef6c0cf480 2023-10-10T20:41:06.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.320+0000 7fef5a7fc700 1 --2- 172.21.15.89:0/3729034526 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fef4c067320 0x7fef4c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:06.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.320+0000 7fef5a7fc700 1 -- 172.21.15.89:0/3729034526 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fef5c012070 con 0x7fef6c0cf480 2023-10-10T20:41:06.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.320+0000 7fef69d9b700 1 --2- 172.21.15.89:0/3729034526 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fef4c067320 0x7fef4c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:06.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.320+0000 7fef69d9b700 1 --2- 172.21.15.89:0/3729034526 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fef4c067320 0x7fef4c0697d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fef600099b0 tx=0x7fef60016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:06.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.324+0000 7fef5a7fc700 1 -- 172.21.15.89:0/3729034526 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef6c16d0c0 con 0x7fef6c0cf480 2023-10-10T20:41:06.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:06.509+0000 7fef6b59e700 1 -- 172.21.15.89:0/3729034526 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fef6c16d0c0 con 0x7fef6c0cf480 2023-10-10T20:41:07.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.173+0000 7fef5a7fc700 1 -- 172.21.15.89:0/3729034526 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v110) v1 ==== 170+0+0 (secure 0 0 0) 0x7fef6c16d0c0 con 0x7fef6c0cf480 2023-10-10T20:41:07.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.177+0000 7fef6b59e700 1 -- 172.21.15.89:0/3729034526 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fef4c067320 msgr2=0x7fef4c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:07.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.177+0000 7fef6b59e700 1 --2- 172.21.15.89:0/3729034526 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fef4c067320 0x7fef4c0697d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fef600099b0 tx=0x7fef60016000 comp rx=0 tx=0).stop 2023-10-10T20:41:07.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.177+0000 7fef6b59e700 1 -- 172.21.15.89:0/3729034526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef6c0cf480 msgr2=0x7fef6c0c5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:07.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.177+0000 7fef6b59e700 1 --2- 172.21.15.89:0/3729034526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef6c0cf480 0x7fef6c0c5390 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00a320 tx=0x7fef5c007900 comp rx=0 tx=0).stop 2023-10-10T20:41:07.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.178+0000 7fef6b59e700 1 -- 172.21.15.89:0/3729034526 shutdown_connections 2023-10-10T20:41:07.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.178+0000 7fef6b59e700 1 --2- 172.21.15.89:0/3729034526 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fef4c067320 0x7fef4c0697d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:07.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.178+0000 7fef6b59e700 1 --2- 172.21.15.89:0/3729034526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef6c15d760 0x7fef6c1578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:07.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.178+0000 7fef6b59e700 1 --2- 172.21.15.89:0/3729034526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef6c0cf480 0x7fef6c0c5390 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:07.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.178+0000 7fef6b59e700 1 --2- 172.21.15.89:0/3729034526 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fef6c1595d0 0x7fef6c169ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:07.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.178+0000 7fef6b59e700 1 -- 172.21.15.89:0/3729034526 >> 172.21.15.89:0/3729034526 conn(0x7fef6c0d05f0 msgr2=0x7fef6c0b5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:07.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.178+0000 7fef6b59e700 1 -- 172.21.15.89:0/3729034526 shutdown_connections 2023-10-10T20:41:07.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.178+0000 7fef6b59e700 1 -- 172.21.15.89:0/3729034526 wait complete. 2023-10-10T20:41:07.190 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2023-10-10T20:41:07.205 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:41:07.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.325+0000 7fc34a4f1700 1 Processor -- start 2023-10-10T20:41:07.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.325+0000 7fc34a4f1700 1 -- start start 2023-10-10T20:41:07.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.326+0000 7fc34a4f1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc344159c60 0x7fc34415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:07.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.326+0000 7fc34a4f1700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc34415db10 0x7fc344069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:07.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.326+0000 7fc34a4f1700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34415a160 con 0x7fc344159890 2023-10-10T20:41:07.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.326+0000 7fc34a4f1700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3440d13a0 con 0x7fc34415db10 2023-10-10T20:41:07.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.326+0000 7fc34a4f1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34415e140 con 0x7fc344159c60 2023-10-10T20:41:07.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.326+0000 7fc348a8e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc34415db10 0x7fc344069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:07.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.326+0000 7fc3437fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc344159c60 0x7fc34415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:07.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.326+0000 7fc3437fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc344159c60 0x7fc34415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:60778/0 (socket says 172.21.15.89:60778) 2023-10-10T20:41:07.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.326+0000 7fc3437fe700 1 -- 172.21.15.89:0/3602356211 learned_addr learned my addr 172.21.15.89:0/3602356211 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:07.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.326+0000 7fc343fff700 1 --1- 172.21.15.89:0/3602356211 >> v1:172.21.15.111:6789/0 conn(0x7fc344159890 0x7fc34415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:36130/0 (socket says 172.21.15.89:36130) 2023-10-10T20:41:07.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.326+0000 7fc3437fe700 1 -- 172.21.15.89:0/3602356211 >> v2:172.21.15.111:3300/0 conn(0x7fc34415db10 msgr2=0x7fc344069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:07.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.326+0000 7fc3437fe700 1 --2- 172.21.15.89:0/3602356211 >> v2:172.21.15.111:3300/0 conn(0x7fc34415db10 0x7fc344069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:07.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.326+0000 7fc3437fe700 1 -- 172.21.15.89:0/3602356211 >> v1:172.21.15.111:6789/0 conn(0x7fc344159890 legacy=0x7fc34415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:07.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.326+0000 7fc3437fe700 1 -- 172.21.15.89:0/3602356211 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34415a3d0 con 0x7fc344159c60 2023-10-10T20:41:07.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.327+0000 7fc3437fe700 1 --2- 172.21.15.89:0/3602356211 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc344159c60 0x7fc34415d5d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc3380099b0 tx=0x7fc338009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96f5dcc5f30d89a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:07.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.327+0000 7fc342ffd700 1 -- 172.21.15.89:0/3602356211 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc338009ea0 con 0x7fc344159c60 2023-10-10T20:41:07.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.327+0000 7fc342ffd700 1 -- 172.21.15.89:0/3602356211 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33800b860 con 0x7fc344159c60 2023-10-10T20:41:07.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.327+0000 7fc342ffd700 1 -- 172.21.15.89:0/3602356211 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33800bc00 con 0x7fc344159c60 2023-10-10T20:41:07.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.327+0000 7fc34a4f1700 1 -- 172.21.15.89:0/3602356211 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc344159c60 msgr2=0x7fc34415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:07.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.327+0000 7fc34a4f1700 1 --2- 172.21.15.89:0/3602356211 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc344159c60 0x7fc34415d5d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc3380099b0 tx=0x7fc338009250 comp rx=0 tx=0).stop 2023-10-10T20:41:07.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.327+0000 7fc34a4f1700 1 -- 172.21.15.89:0/3602356211 shutdown_connections 2023-10-10T20:41:07.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.327+0000 7fc34a4f1700 1 --2- 172.21.15.89:0/3602356211 >> v2:172.21.15.111:3300/0 conn(0x7fc34415db10 0x7fc344069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:07.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.327+0000 7fc34a4f1700 1 --2- 172.21.15.89:0/3602356211 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc344159c60 0x7fc34415d5d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:07.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.327+0000 7fc34a4f1700 1 -- 172.21.15.89:0/3602356211 >> 172.21.15.89:0/3602356211 conn(0x7fc344068e20 msgr2=0x7fc3440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:07.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.327+0000 7fc34a4f1700 1 -- 172.21.15.89:0/3602356211 shutdown_connections 2023-10-10T20:41:07.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.327+0000 7fc34a4f1700 1 -- 172.21.15.89:0/3602356211 wait complete. 2023-10-10T20:41:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.328+0000 7fc34a4f1700 1 Processor -- start 2023-10-10T20:41:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.329+0000 7fc34a4f1700 1 -- start start 2023-10-10T20:41:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.329+0000 7fc34a4f1700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc344159890 0x7fc3441526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.329+0000 7fc34a4f1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc344159c60 0x7fc3440c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.329+0000 7fc34a4f1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc34415db10 0x7fc3440cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.329+0000 7fc34a4f1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3440cf0d0 con 0x7fc34415db10 2023-10-10T20:41:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.329+0000 7fc34a4f1700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3440ce4c0 con 0x7fc344159890 2023-10-10T20:41:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.329+0000 7fc34a4f1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3440ce7a0 con 0x7fc344159c60 2023-10-10T20:41:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.330+0000 7fc348a8e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc34415db10 0x7fc3440cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.330+0000 7fc348a8e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc34415db10 0x7fc3440cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54602/0 (socket says 172.21.15.89:54602) 2023-10-10T20:41:07.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.330+0000 7fc348a8e700 1 -- 172.21.15.89:0/2592266453 learned_addr learned my addr 172.21.15.89:0/2592266453 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:07.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.330+0000 7fc3437fe700 1 --2- 172.21.15.89:0/2592266453 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc344159c60 0x7fc3440c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:07.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.330+0000 7fc343fff700 1 --2- 172.21.15.89:0/2592266453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc344159890 0x7fc3441526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:07.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.330+0000 7fc348a8e700 1 -- 172.21.15.89:0/2592266453 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc344159c60 msgr2=0x7fc3440c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:07.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.330+0000 7fc348a8e700 1 --2- 172.21.15.89:0/2592266453 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc344159c60 0x7fc3440c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:07.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.330+0000 7fc348a8e700 1 -- 172.21.15.89:0/2592266453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc344159890 msgr2=0x7fc3441526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:07.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.330+0000 7fc348a8e700 1 --2- 172.21.15.89:0/2592266453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc344159890 0x7fc3441526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:07.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.330+0000 7fc348a8e700 1 -- 172.21.15.89:0/2592266453 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3380094f0 con 0x7fc34415db10 2023-10-10T20:41:07.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.330+0000 7fc3437fe700 1 --2- 172.21.15.89:0/2592266453 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc344159c60 0x7fc3440c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:41:07.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.330+0000 7fc348a8e700 1 --2- 172.21.15.89:0/2592266453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc34415db10 0x7fc3440cafa0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009df0 tx=0x7fc33c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:07.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.330+0000 7fc32bfff700 1 -- 172.21.15.89:0/2592266453 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c00caa0 con 0x7fc34415db10 2023-10-10T20:41:07.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.330+0000 7fc32bfff700 1 -- 172.21.15.89:0/2592266453 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33c00f040 con 0x7fc34415db10 2023-10-10T20:41:07.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.330+0000 7fc34a4f1700 1 -- 172.21.15.89:0/2592266453 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3440cea80 con 0x7fc34415db10 2023-10-10T20:41:07.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.330+0000 7fc32bfff700 1 -- 172.21.15.89:0/2592266453 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c00ce70 con 0x7fc34415db10 2023-10-10T20:41:07.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.330+0000 7fc34a4f1700 1 -- 172.21.15.89:0/2592266453 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc344152c10 con 0x7fc34415db10 2023-10-10T20:41:07.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.331+0000 7fc34a4f1700 1 -- 172.21.15.89:0/2592266453 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3440cf240 con 0x7fc34415db10 2023-10-10T20:41:07.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.332+0000 7fc32bfff700 1 -- 172.21.15.89:0/2592266453 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc33c014690 con 0x7fc34415db10 2023-10-10T20:41:07.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.332+0000 7fc32bfff700 1 --2- 172.21.15.89:0/2592266453 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc330067320 0x7fc3300697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:07.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.332+0000 7fc32bfff700 1 -- 172.21.15.89:0/2592266453 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fc33c085530 con 0x7fc34415db10 2023-10-10T20:41:07.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.332+0000 7fc343fff700 1 --2- 172.21.15.89:0/2592266453 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc330067320 0x7fc3300697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:07.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.332+0000 7fc343fff700 1 --2- 172.21.15.89:0/2592266453 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc330067320 0x7fc3300697d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc334004fd0 tx=0x7fc334007eb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:07.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.336+0000 7fc32bfff700 1 -- 172.21.15.89:0/2592266453 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc33c055ba0 con 0x7fc34415db10 2023-10-10T20:41:07.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:07.564+0000 7fc34a4f1700 1 -- 172.21.15.89:0/2592266453 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc34404c3c0 con 0x7fc34415db10 2023-10-10T20:41:08.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.205+0000 7fc32bfff700 1 -- 172.21.15.89:0/2592266453 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v69) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc33c055730 con 0x7fc34415db10 2023-10-10T20:41:08.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.209+0000 7fc34a4f1700 1 -- 172.21.15.89:0/2592266453 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc330067320 msgr2=0x7fc3300697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:08.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.209+0000 7fc34a4f1700 1 --2- 172.21.15.89:0/2592266453 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc330067320 0x7fc3300697d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc334004fd0 tx=0x7fc334007eb0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.209+0000 7fc34a4f1700 1 -- 172.21.15.89:0/2592266453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc34415db10 msgr2=0x7fc3440cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:08.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.209+0000 7fc34a4f1700 1 --2- 172.21.15.89:0/2592266453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc34415db10 0x7fc3440cafa0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009df0 tx=0x7fc33c00b040 comp rx=0 tx=0).stop 2023-10-10T20:41:08.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.215+0000 7fc34a4f1700 1 -- 172.21.15.89:0/2592266453 shutdown_connections 2023-10-10T20:41:08.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.215+0000 7fc34a4f1700 1 --2- 172.21.15.89:0/2592266453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc34415db10 0x7fc3440cafa0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.215+0000 7fc34a4f1700 1 --2- 172.21.15.89:0/2592266453 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc330067320 0x7fc3300697d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.215+0000 7fc34a4f1700 1 --2- 172.21.15.89:0/2592266453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc344159890 0x7fc3441526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.215+0000 7fc34a4f1700 1 --2- 172.21.15.89:0/2592266453 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc344159c60 0x7fc3440c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.215+0000 7fc34a4f1700 1 -- 172.21.15.89:0/2592266453 >> 172.21.15.89:0/2592266453 conn(0x7fc344068e20 msgr2=0x7fc344069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:08.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.216+0000 7fc34a4f1700 1 -- 172.21.15.89:0/2592266453 shutdown_connections 2023-10-10T20:41:08.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.217+0000 7fc34a4f1700 1 -- 172.21.15.89:0/2592266453 wait complete. 2023-10-10T20:41:08.225 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 18 and data pool 19 2023-10-10T20:41:08.239 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:41:08.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.351+0000 7fc037fff700 1 Processor -- start 2023-10-10T20:41:08.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.351+0000 7fc037fff700 1 -- start start 2023-10-10T20:41:08.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.352+0000 7fc037fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0381597f0 0x7fc0380b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:08.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.352+0000 7fc037fff700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0380b11d0 con 0x7fc03815d230 2023-10-10T20:41:08.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.352+0000 7fc037fff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0380b14b0 con 0x7fc038159420 2023-10-10T20:41:08.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.352+0000 7fc037fff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380b1790 con 0x7fc0381597f0 2023-10-10T20:41:08.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.352+0000 7fc0377fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc03815d230 0x7fc0380b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41304/0 (socket says 172.21.15.89:41304) 2023-10-10T20:41:08.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.352+0000 7fc0377fe700 1 -- 172.21.15.89:0/1803454788 learned_addr learned my addr 172.21.15.89:0/1803454788 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:08.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.352+0000 7fc0367fc700 1 --2- 172.21.15.89:0/1803454788 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0381597f0 0x7fc0380b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:08.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.352+0000 7fc035ffb700 1 -- 172.21.15.89:0/1803454788 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157956087 0 0) 0x7fc0380b11d0 con 0x7fc03815d230 2023-10-10T20:41:08.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.352+0000 7fc035ffb700 1 -- 172.21.15.89:0/1803454788 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc020003570 con 0x7fc03815d230 2023-10-10T20:41:08.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.352+0000 7fc035ffb700 1 -- 172.21.15.89:0/1803454788 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2970887895 0 0) 0x7fc020003570 con 0x7fc03815d230 2023-10-10T20:41:08.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.352+0000 7fc035ffb700 1 -- 172.21.15.89:0/1803454788 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0381597f0 msgr2=0x7fc0380b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:08.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.352+0000 7fc035ffb700 1 --2- 172.21.15.89:0/1803454788 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0381597f0 0x7fc0380b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.352+0000 7fc035ffb700 1 -- 172.21.15.89:0/1803454788 >> v1:172.21.15.111:6789/0 conn(0x7fc038159420 legacy=0x7fc0380b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:08.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc035ffb700 1 -- 172.21.15.89:0/1803454788 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0380b1a70 con 0x7fc03815d230 2023-10-10T20:41:08.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc035ffb700 1 -- 172.21.15.89:0/1803454788 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fc030001470 con 0x7fc03815d230 2023-10-10T20:41:08.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc035ffb700 1 -- 172.21.15.89:0/1803454788 >> v1:172.21.15.89:6789/0 conn(0x7fc03815d230 legacy=0x7fc0380b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:08.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc035ffb700 1 --2- 172.21.15.89:0/1803454788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc020003a90 0x7fc020003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:08.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc035ffb700 1 --2- 172.21.15.89:0/1803454788 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0381597f0 0x7fc020004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:08.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc035ffb700 1 --2- 172.21.15.89:0/1803454788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc020004d50 0x7fc0200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:08.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc035ffb700 1 -- 172.21.15.89:0/1803454788 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380b1790 con 0x7fc020004d50 2023-10-10T20:41:08.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc035ffb700 1 -- 172.21.15.89:0/1803454788 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02000bb10 con 0x7fc020003a90 2023-10-10T20:41:08.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc035ffb700 1 -- 172.21.15.89:0/1803454788 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02000bcd0 con 0x7fc0381597f0 2023-10-10T20:41:08.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc0377fe700 1 --2- 172.21.15.89:0/1803454788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc020004d50 0x7fc0200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:08.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc0367fc700 1 --2- 172.21.15.89:0/1803454788 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0381597f0 0x7fc020004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:08.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc036ffd700 1 --2- 172.21.15.89:0/1803454788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc020003a90 0x7fc020003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:08.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc0377fe700 1 -- 172.21.15.89:0/1803454788 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0381597f0 msgr2=0x7fc020004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:08.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc0377fe700 1 --2- 172.21.15.89:0/1803454788 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0381597f0 0x7fc020004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc0377fe700 1 -- 172.21.15.89:0/1803454788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc020003a90 msgr2=0x7fc020003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:08.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc0377fe700 1 --2- 172.21.15.89:0/1803454788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc020003a90 0x7fc020003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc0377fe700 1 -- 172.21.15.89:0/1803454788 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc020003570 con 0x7fc020004d50 2023-10-10T20:41:08.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc0377fe700 1 --2- 172.21.15.89:0/1803454788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc020004d50 0x7fc0200051c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc03000c3c0 tx=0x7fc030001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66914a38fefc50ee server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:08.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc035ffb700 1 -- 172.21.15.89:0/1803454788 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03000cc90 con 0x7fc020004d50 2023-10-10T20:41:08.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc035ffb700 1 -- 172.21.15.89:0/1803454788 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03000cdf0 con 0x7fc020004d50 2023-10-10T20:41:08.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.353+0000 7fc035ffb700 1 -- 172.21.15.89:0/1803454788 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc030005b70 con 0x7fc020004d50 2023-10-10T20:41:08.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.354+0000 7fc037fff700 1 -- 172.21.15.89:0/1803454788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc020004d50 msgr2=0x7fc0200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:08.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.354+0000 7fc037fff700 1 --2- 172.21.15.89:0/1803454788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc020004d50 0x7fc0200051c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc03000c3c0 tx=0x7fc030001e70 comp rx=0 tx=0).stop 2023-10-10T20:41:08.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.354+0000 7fc0377fe700 1 -- 172.21.15.89:0/1803454788 reap_dead start 2023-10-10T20:41:08.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.354+0000 7fc037fff700 1 -- 172.21.15.89:0/1803454788 shutdown_connections 2023-10-10T20:41:08.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.354+0000 7fc037fff700 1 -- 172.21.15.89:0/1803454788 >> 172.21.15.89:0/1803454788 conn(0x7fc0380d0660 msgr2=0x7fc0380d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:08.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.354+0000 7fc037fff700 1 -- 172.21.15.89:0/1803454788 shutdown_connections 2023-10-10T20:41:08.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.354+0000 7fc037fff700 1 -- 172.21.15.89:0/1803454788 wait complete. 2023-10-10T20:41:08.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.355+0000 7fc037fff700 1 Processor -- start 2023-10-10T20:41:08.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.355+0000 7fc037fff700 1 -- start start 2023-10-10T20:41:08.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.356+0000 7fc037fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc020004d50 0x7fc0380c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:08.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.356+0000 7fc037fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc03815a590 0x7fc03815aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:08.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.356+0000 7fc037fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc03815aa00 0x7fc03815b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:08.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.356+0000 7fc037fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03815f160 con 0x7fc03815aa00 2023-10-10T20:41:08.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.356+0000 7fc037fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03815f440 con 0x7fc03815a590 2023-10-10T20:41:08.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.356+0000 7fc037fff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03815f720 con 0x7fc020004d50 2023-10-10T20:41:08.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.356+0000 7fc0377fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc03815aa00 0x7fc03815b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:08.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.356+0000 7fc0377fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc03815aa00 0x7fc03815b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49444/0 (socket says 172.21.15.89:49444) 2023-10-10T20:41:08.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.356+0000 7fc0377fe700 1 -- 172.21.15.89:0/1898588969 learned_addr learned my addr 172.21.15.89:0/1898588969 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:08.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.356+0000 7fc0367fc700 1 --2- 172.21.15.89:0/1898588969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc03815a590 0x7fc03815aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:08.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.356+0000 7fc036ffd700 1 --2- 172.21.15.89:0/1898588969 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc020004d50 0x7fc0380c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:08.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.356+0000 7fc0377fe700 1 -- 172.21.15.89:0/1898588969 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc020004d50 msgr2=0x7fc0380c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:08.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.356+0000 7fc0377fe700 1 --2- 172.21.15.89:0/1898588969 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc020004d50 0x7fc0380c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.356+0000 7fc0377fe700 1 -- 172.21.15.89:0/1898588969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc03815a590 msgr2=0x7fc03815aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:08.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.356+0000 7fc0377fe700 1 --2- 172.21.15.89:0/1898588969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc03815a590 0x7fc03815aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.356+0000 7fc0377fe700 1 -- 172.21.15.89:0/1898588969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03815fa00 con 0x7fc03815aa00 2023-10-10T20:41:08.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.357+0000 7fc0377fe700 1 --2- 172.21.15.89:0/1898588969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc03815aa00 0x7fc03815b430 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fc030007cf0 tx=0x7fc030015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:08.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.357+0000 7fc01effd700 1 -- 172.21.15.89:0/1898588969 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03000cc90 con 0x7fc03815aa00 2023-10-10T20:41:08.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.357+0000 7fc01effd700 1 -- 172.21.15.89:0/1898588969 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0300072e0 con 0x7fc03815aa00 2023-10-10T20:41:08.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.357+0000 7fc037fff700 1 -- 172.21.15.89:0/1898588969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc03815ed80 con 0x7fc03815aa00 2023-10-10T20:41:08.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.357+0000 7fc01effd700 1 -- 172.21.15.89:0/1898588969 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0300076a0 con 0x7fc03815aa00 2023-10-10T20:41:08.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.357+0000 7fc037fff700 1 -- 172.21.15.89:0/1898588969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0380c6080 con 0x7fc03815aa00 2023-10-10T20:41:08.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.358+0000 7fc037fff700 1 -- 172.21.15.89:0/1898588969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03804c3c0 con 0x7fc03815aa00 2023-10-10T20:41:08.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.358+0000 7fc01effd700 1 -- 172.21.15.89:0/1898588969 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc030019070 con 0x7fc03815aa00 2023-10-10T20:41:08.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.359+0000 7fc01effd700 1 --2- 172.21.15.89:0/1898588969 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc024067320 0x7fc0240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:08.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.359+0000 7fc01effd700 1 -- 172.21.15.89:0/1898588969 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc030085430 con 0x7fc03815aa00 2023-10-10T20:41:08.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.359+0000 7fc036ffd700 1 --2- 172.21.15.89:0/1898588969 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc024067320 0x7fc0240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:08.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.359+0000 7fc036ffd700 1 --2- 172.21.15.89:0/1898588969 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc024067320 0x7fc0240697d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc028000f80 tx=0x7fc0280016d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:08.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.363+0000 7fc01effd700 1 -- 172.21.15.89:0/1898588969 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0300520c0 con 0x7fc03815aa00 2023-10-10T20:41:08.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.546+0000 7fc037fff700 1 -- 172.21.15.89:0/1898588969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0380b1d50 con 0x7fc03815aa00 2023-10-10T20:41:08.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.547+0000 7fc01effd700 1 -- 172.21.15.89:0/1898588969 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+15377 (secure 0 0 0) 0x7fc0300556e0 con 0x7fc03815aa00 2023-10-10T20:41:08.550 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:08.550 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":111,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:41:07.565654+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:41:05.928269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-10T20:41:06.527456+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:41:08.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.549+0000 7fc037fff700 1 -- 172.21.15.89:0/1898588969 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc024067320 msgr2=0x7fc0240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:08.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.549+0000 7fc037fff700 1 --2- 172.21.15.89:0/1898588969 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc024067320 0x7fc0240697d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc028000f80 tx=0x7fc0280016d0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.549+0000 7fc037fff700 1 -- 172.21.15.89:0/1898588969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc03815aa00 msgr2=0x7fc03815b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:08.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.549+0000 7fc037fff700 1 --2- 172.21.15.89:0/1898588969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc03815aa00 0x7fc03815b430 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fc030007cf0 tx=0x7fc030015000 comp rx=0 tx=0).stop 2023-10-10T20:41:08.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.549+0000 7fc037fff700 1 -- 172.21.15.89:0/1898588969 shutdown_connections 2023-10-10T20:41:08.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.549+0000 7fc037fff700 1 --2- 172.21.15.89:0/1898588969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc03815aa00 0x7fc03815b430 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.549+0000 7fc037fff700 1 --2- 172.21.15.89:0/1898588969 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc020004d50 0x7fc0380c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.549+0000 7fc037fff700 1 --2- 172.21.15.89:0/1898588969 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc024067320 0x7fc0240697d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.549+0000 7fc037fff700 1 --2- 172.21.15.89:0/1898588969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc03815a590 0x7fc03815aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.549+0000 7fc037fff700 1 -- 172.21.15.89:0/1898588969 >> 172.21.15.89:0/1898588969 conn(0x7fc0380d0660 msgr2=0x7fc0380b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:08.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.549+0000 7fc037fff700 1 -- 172.21.15.89:0/1898588969 shutdown_connections 2023-10-10T20:41:08.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.549+0000 7fc037fff700 1 -- 172.21.15.89:0/1898588969 wait complete. 2023-10-10T20:41:08.564 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:41:08.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.676+0000 7fccf2701700 1 Processor -- start 2023-10-10T20:41:08.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.676+0000 7fccf2701700 1 -- start start 2023-10-10T20:41:08.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.677+0000 7fccf2701700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccec06c0f0 0x7fccec159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.677+0000 7fccf2701700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccec143420 con 0x7fccec0b3810 2023-10-10T20:41:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.677+0000 7fccf2701700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccec06a1e0 con 0x7fccec143050 2023-10-10T20:41:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.677+0000 7fccf2701700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccec0695d0 con 0x7fccec06c0f0 2023-10-10T20:41:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.677+0000 7fccf1f00700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fccec0b3810 0x7fccec0b3c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41316/0 (socket says 172.21.15.89:41316) 2023-10-10T20:41:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.677+0000 7fccf1f00700 1 -- 172.21.15.89:0/1078354496 learned_addr learned my addr 172.21.15.89:0/1078354496 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.677+0000 7fccf0efe700 1 --2- 172.21.15.89:0/1078354496 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccec06c0f0 0x7fccec159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.677+0000 7fcce3fff700 1 -- 172.21.15.89:0/1078354496 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2640527612 0 0) 0x7fccec143420 con 0x7fccec0b3810 2023-10-10T20:41:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.677+0000 7fcce3fff700 1 -- 172.21.15.89:0/1078354496 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccd8003570 con 0x7fccec0b3810 2023-10-10T20:41:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.677+0000 7fccf0efe700 1 -- 172.21.15.89:0/1078354496 >> v1:172.21.15.111:6789/0 conn(0x7fccec143050 legacy=0x7fccec06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.677+0000 7fcce3fff700 1 -- 172.21.15.89:0/1078354496 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 9780098 0 0) 0x7fccec06a1e0 con 0x7fccec143050 2023-10-10T20:41:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.677+0000 7fccf0efe700 1 -- 172.21.15.89:0/1078354496 >> v1:172.21.15.89:6789/0 conn(0x7fccec0b3810 legacy=0x7fccec0b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.677+0000 7fccf0efe700 1 -- 172.21.15.89:0/1078354496 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec0698b0 con 0x7fccec06c0f0 2023-10-10T20:41:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.678+0000 7fccf0efe700 1 --2- 172.21.15.89:0/1078354496 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccec06c0f0 0x7fccec159c90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009910 tx=0x7fccdc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d06636a08b849a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.678+0000 7fcce3fff700 1 -- 172.21.15.89:0/1078354496 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00b680 con 0x7fccec06c0f0 2023-10-10T20:41:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.678+0000 7fcce3fff700 1 -- 172.21.15.89:0/1078354496 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccdc00b7e0 con 0x7fccec06c0f0 2023-10-10T20:41:08.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.678+0000 7fcce3fff700 1 -- 172.21.15.89:0/1078354496 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00bb60 con 0x7fccec06c0f0 2023-10-10T20:41:08.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.678+0000 7fccf2701700 1 -- 172.21.15.89:0/1078354496 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccec06c0f0 msgr2=0x7fccec159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:08.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.678+0000 7fccf2701700 1 --2- 172.21.15.89:0/1078354496 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccec06c0f0 0x7fccec159c90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009910 tx=0x7fccdc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.678+0000 7fccf2701700 1 -- 172.21.15.89:0/1078354496 shutdown_connections 2023-10-10T20:41:08.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.678+0000 7fccf2701700 1 --2- 172.21.15.89:0/1078354496 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccec06c0f0 0x7fccec159c90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.678+0000 7fccf2701700 1 -- 172.21.15.89:0/1078354496 >> 172.21.15.89:0/1078354496 conn(0x7fccec0ce430 msgr2=0x7fccec0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:08.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.679+0000 7fccf2701700 1 -- 172.21.15.89:0/1078354496 shutdown_connections 2023-10-10T20:41:08.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.679+0000 7fccf2701700 1 -- 172.21.15.89:0/1078354496 wait complete. 2023-10-10T20:41:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.680+0000 7fccf2701700 1 Processor -- start 2023-10-10T20:41:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.680+0000 7fccf2701700 1 -- start start 2023-10-10T20:41:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.680+0000 7fccf2701700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccec06c0f0 0x7fccec13f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.680+0000 7fccf2701700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccec0b3810 0x7fccec16d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.680+0000 7fccf2701700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccec143050 0x7fccec0ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.680+0000 7fccf2701700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec15a160 con 0x7fccec143050 2023-10-10T20:41:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.680+0000 7fccf2701700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec159e10 con 0x7fccec0b3810 2023-10-10T20:41:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.680+0000 7fccf2701700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccec16daf0 con 0x7fccec06c0f0 2023-10-10T20:41:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.681+0000 7fccf1f00700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccec143050 0x7fccec0ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.681+0000 7fccf1f00700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccec143050 0x7fccec0ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49458/0 (socket says 172.21.15.89:49458) 2023-10-10T20:41:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.681+0000 7fccf1f00700 1 -- 172.21.15.89:0/3414702522 learned_addr learned my addr 172.21.15.89:0/3414702522 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:08.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.681+0000 7fccf16ff700 1 --2- 172.21.15.89:0/3414702522 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccec06c0f0 0x7fccec13f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:08.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.681+0000 7fccf0efe700 1 --2- 172.21.15.89:0/3414702522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccec0b3810 0x7fccec16d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:08.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.681+0000 7fccf1f00700 1 -- 172.21.15.89:0/3414702522 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccec06c0f0 msgr2=0x7fccec13f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:08.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.681+0000 7fccf1f00700 1 --2- 172.21.15.89:0/3414702522 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccec06c0f0 0x7fccec13f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.681+0000 7fccf1f00700 1 -- 172.21.15.89:0/3414702522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccec0b3810 msgr2=0x7fccec16d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.681+0000 7fccf1f00700 1 --2- 172.21.15.89:0/3414702522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccec0b3810 0x7fccec16d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.681+0000 7fccf1f00700 1 -- 172.21.15.89:0/3414702522 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccdc009420 con 0x7fccec143050 2023-10-10T20:41:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.681+0000 7fccf0efe700 1 --2- 172.21.15.89:0/3414702522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccec0b3810 0x7fccec16d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.681+0000 7fccf1f00700 1 --2- 172.21.15.89:0/3414702522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccec143050 0x7fccec0ccc80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fcce400cc20 tx=0x7fcce4001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.681+0000 7fcce17fa700 1 -- 172.21.15.89:0/3414702522 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce400a910 con 0x7fccec143050 2023-10-10T20:41:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.681+0000 7fcce17fa700 1 -- 172.21.15.89:0/3414702522 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce400aa90 con 0x7fccec143050 2023-10-10T20:41:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.681+0000 7fccf2701700 1 -- 172.21.15.89:0/3414702522 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccec16ddd0 con 0x7fccec143050 2023-10-10T20:41:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.681+0000 7fcce17fa700 1 -- 172.21.15.89:0/3414702522 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce400ae50 con 0x7fccec143050 2023-10-10T20:41:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.682+0000 7fccf2701700 1 -- 172.21.15.89:0/3414702522 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccec0c82e0 con 0x7fccec143050 2023-10-10T20:41:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.682+0000 7fccf2701700 1 -- 172.21.15.89:0/3414702522 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccec04c3c0 con 0x7fccec143050 2023-10-10T20:41:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.683+0000 7fcce17fa700 1 -- 172.21.15.89:0/3414702522 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcce4007300 con 0x7fccec143050 2023-10-10T20:41:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.683+0000 7fcce17fa700 1 --2- 172.21.15.89:0/3414702522 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fccd0067320 0x7fccd00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.683+0000 7fcce17fa700 1 -- 172.21.15.89:0/3414702522 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcce4084ff0 con 0x7fccec143050 2023-10-10T20:41:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.683+0000 7fccf16ff700 1 --2- 172.21.15.89:0/3414702522 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fccd0067320 0x7fccd00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.684+0000 7fccf16ff700 1 --2- 172.21.15.89:0/3414702522 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fccd0067320 0x7fccd00697d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fcce8003a30 tx=0x7fcce8007ef0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.687+0000 7fcce17fa700 1 -- 172.21.15.89:0/3414702522 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcce4051d00 con 0x7fccec143050 2023-10-10T20:41:08.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.874+0000 7fccf2701700 1 -- 172.21.15.89:0/3414702522 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccec069b90 con 0x7fccec143050 2023-10-10T20:41:08.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.875+0000 7fcce17fa700 1 -- 172.21.15.89:0/3414702522 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+15377 (secure 0 0 0) 0x7fcce4055320 con 0x7fccec143050 2023-10-10T20:41:08.877 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:08.878 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":111,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:41:07.565654+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:41:05.928269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-10T20:41:06.527456+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:41:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.877+0000 7fccf2701700 1 -- 172.21.15.89:0/3414702522 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fccd0067320 msgr2=0x7fccd00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.877+0000 7fccf2701700 1 --2- 172.21.15.89:0/3414702522 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fccd0067320 0x7fccd00697d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fcce8003a30 tx=0x7fcce8007ef0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.877+0000 7fccf2701700 1 -- 172.21.15.89:0/3414702522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccec143050 msgr2=0x7fccec0ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:08.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.877+0000 7fccf2701700 1 --2- 172.21.15.89:0/3414702522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccec143050 0x7fccec0ccc80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fcce400cc20 tx=0x7fcce4001a60 comp rx=0 tx=0).stop 2023-10-10T20:41:08.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.877+0000 7fccf2701700 1 -- 172.21.15.89:0/3414702522 shutdown_connections 2023-10-10T20:41:08.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.877+0000 7fccf2701700 1 --2- 172.21.15.89:0/3414702522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccec143050 0x7fccec0ccc80 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.877+0000 7fccf2701700 1 --2- 172.21.15.89:0/3414702522 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccec06c0f0 0x7fccec13f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.877+0000 7fccf2701700 1 --2- 172.21.15.89:0/3414702522 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fccd0067320 0x7fccd00697d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.877+0000 7fccf2701700 1 --2- 172.21.15.89:0/3414702522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccec0b3810 0x7fccec16d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:08.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.877+0000 7fccf2701700 1 -- 172.21.15.89:0/3414702522 >> 172.21.15.89:0/3414702522 conn(0x7fccec0ce430 msgr2=0x7fccec0adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:08.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.877+0000 7fccf2701700 1 -- 172.21.15.89:0/3414702522 shutdown_connections 2023-10-10T20:41:08.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:08.877+0000 7fccf2701700 1 -- 172.21.15.89:0/3414702522 wait complete. 2023-10-10T20:41:08.891 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:41:09.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.007+0000 7f090709a700 1 Processor -- start 2023-10-10T20:41:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.008+0000 7f090709a700 1 -- start start 2023-10-10T20:41:09.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.008+0000 7f090709a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0900159890 0x7f090015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:09.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.008+0000 7f090709a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0900069560 0x7f09000699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:09.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.008+0000 7f090709a700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0900069ee0 con 0x7f0900159c60 2023-10-10T20:41:09.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.008+0000 7f090709a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f090015a160 con 0x7f0900159890 2023-10-10T20:41:09.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.008+0000 7f090709a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09000d13a0 con 0x7f0900069560 2023-10-10T20:41:09.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.009+0000 7f0904e36700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0900159890 0x7f090015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:09.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.009+0000 7f0905637700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0900069560 0x7f09000699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:09.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.009+0000 7f0904e36700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0900159890 0x7f090015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:41518/0 (socket says 172.21.15.89:41518) 2023-10-10T20:41:09.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.009+0000 7f0904e36700 1 -- 172.21.15.89:0/2231141094 learned_addr learned my addr 172.21.15.89:0/2231141094 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:09.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.009+0000 7f0905637700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0900069560 0x7f09000699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:55840/0 (socket says 172.21.15.89:55840) 2023-10-10T20:41:09.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.009+0000 7f08f7fff700 1 --1- 172.21.15.89:0/2231141094 >> v1:172.21.15.111:6789/0 conn(0x7f0900159c60 0x7f090015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:46086/0 (socket says 172.21.15.89:46086) 2023-10-10T20:41:09.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.009+0000 7f0904e36700 1 -- 172.21.15.89:0/2231141094 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0900069560 msgr2=0x7f09000699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:09.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.009+0000 7f0904e36700 1 --2- 172.21.15.89:0/2231141094 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0900069560 0x7f09000699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:09.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.009+0000 7f0904e36700 1 -- 172.21.15.89:0/2231141094 >> v1:172.21.15.111:6789/0 conn(0x7f0900159c60 legacy=0x7f090015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:09.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.009+0000 7f0904e36700 1 -- 172.21.15.89:0/2231141094 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090015e140 con 0x7f0900159890 2023-10-10T20:41:09.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.009+0000 7f0905637700 1 --2- 172.21.15.89:0/2231141094 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0900069560 0x7f09000699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:41:09.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.009+0000 7f0904e36700 1 --2- 172.21.15.89:0/2231141094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0900159890 0x7f090015abd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009f20 tx=0x7f08fc009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15921c8780d5ed72 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:09.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.009+0000 7f08f77fe700 1 -- 172.21.15.89:0/2231141094 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc00e070 con 0x7f0900159890 2023-10-10T20:41:09.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.009+0000 7f08f77fe700 1 -- 172.21.15.89:0/2231141094 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc00b9d0 con 0x7f0900159890 2023-10-10T20:41:09.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.009+0000 7f08f77fe700 1 -- 172.21.15.89:0/2231141094 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc0092e0 con 0x7f0900159890 2023-10-10T20:41:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.010+0000 7f090709a700 1 -- 172.21.15.89:0/2231141094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0900159890 msgr2=0x7f090015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.010+0000 7f090709a700 1 --2- 172.21.15.89:0/2231141094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0900159890 0x7f090015abd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009f20 tx=0x7f08fc009840 comp rx=0 tx=0).stop 2023-10-10T20:41:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.010+0000 7f090709a700 1 -- 172.21.15.89:0/2231141094 shutdown_connections 2023-10-10T20:41:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.010+0000 7f090709a700 1 --2- 172.21.15.89:0/2231141094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0900159890 0x7f090015abd0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.010+0000 7f090709a700 1 --2- 172.21.15.89:0/2231141094 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0900069560 0x7f09000699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.010+0000 7f090709a700 1 -- 172.21.15.89:0/2231141094 >> 172.21.15.89:0/2231141094 conn(0x7f0900068e20 msgr2=0x7f09000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.010+0000 7f090709a700 1 -- 172.21.15.89:0/2231141094 shutdown_connections 2023-10-10T20:41:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.010+0000 7f090709a700 1 -- 172.21.15.89:0/2231141094 wait complete. 2023-10-10T20:41:09.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.011+0000 7f090709a700 1 Processor -- start 2023-10-10T20:41:09.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.011+0000 7f090709a700 1 -- start start 2023-10-10T20:41:09.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.012+0000 7f090709a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0900069560 0x7f09000c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:09.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.012+0000 7f090709a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0900159890 0x7f0900153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:09.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.012+0000 7f090709a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0900159c60 0x7f0900158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:09.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.012+0000 7f090709a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0900158c50 con 0x7f0900159c60 2023-10-10T20:41:09.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.012+0000 7f090709a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09001589b0 con 0x7f0900069560 2023-10-10T20:41:09.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.012+0000 7f090709a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09000b40d0 con 0x7f0900159890 2023-10-10T20:41:09.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.012+0000 7f0905637700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0900159c60 0x7f0900158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:09.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.012+0000 7f0905637700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0900159c60 0x7f0900158470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49466/0 (socket says 172.21.15.89:49466) 2023-10-10T20:41:09.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.012+0000 7f0905637700 1 -- 172.21.15.89:0/3470278616 learned_addr learned my addr 172.21.15.89:0/3470278616 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.012+0000 7f08f7fff700 1 --2- 172.21.15.89:0/3470278616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0900159890 0x7f0900153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.012+0000 7f0904e36700 1 --2- 172.21.15.89:0/3470278616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0900069560 0x7f09000c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.012+0000 7f0905637700 1 -- 172.21.15.89:0/3470278616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0900159890 msgr2=0x7f0900153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.012+0000 7f0905637700 1 --2- 172.21.15.89:0/3470278616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0900159890 0x7f0900153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.012+0000 7f0905637700 1 -- 172.21.15.89:0/3470278616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0900069560 msgr2=0x7f09000c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:09.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.012+0000 7f0905637700 1 --2- 172.21.15.89:0/3470278616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0900069560 0x7f09000c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:09.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.012+0000 7f0905637700 1 -- 172.21.15.89:0/3470278616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f8009420 con 0x7f0900159c60 2023-10-10T20:41:09.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.013+0000 7f0905637700 1 --2- 172.21.15.89:0/3470278616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0900159c60 0x7f0900158470 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f08f800ea20 tx=0x7f08f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:09.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.013+0000 7f08f4ff9700 1 -- 172.21.15.89:0/3470278616 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f800ee60 con 0x7f0900159c60 2023-10-10T20:41:09.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.013+0000 7f08f4ff9700 1 -- 172.21.15.89:0/3470278616 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f800c850 con 0x7f0900159c60 2023-10-10T20:41:09.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.013+0000 7f090709a700 1 -- 172.21.15.89:0/3470278616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08fc009ae0 con 0x7f0900159c60 2023-10-10T20:41:09.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.013+0000 7f08f4ff9700 1 -- 172.21.15.89:0/3470278616 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f800cc10 con 0x7f0900159c60 2023-10-10T20:41:09.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.013+0000 7f090709a700 1 -- 172.21.15.89:0/3470278616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09000b4690 con 0x7f0900159c60 2023-10-10T20:41:09.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.014+0000 7f090709a700 1 -- 172.21.15.89:0/3470278616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090004c3c0 con 0x7f0900159c60 2023-10-10T20:41:09.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.014+0000 7f08f4ff9700 1 -- 172.21.15.89:0/3470278616 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f08f8013470 con 0x7f0900159c60 2023-10-10T20:41:09.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.015+0000 7f08f4ff9700 1 --2- 172.21.15.89:0/3470278616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f08ec067320 0x7f08ec0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:09.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.015+0000 7f0904e36700 1 --2- 172.21.15.89:0/3470278616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f08ec067320 0x7f08ec0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:09.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.015+0000 7f08f4ff9700 1 -- 172.21.15.89:0/3470278616 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f08f8084f60 con 0x7f0900159c60 2023-10-10T20:41:09.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.015+0000 7f0904e36700 1 --2- 172.21.15.89:0/3470278616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f08ec067320 0x7f08ec0697d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009880 tx=0x7f08fc018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:09.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.019+0000 7f08f4ff9700 1 -- 172.21.15.89:0/3470278616 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08f8051bf0 con 0x7f0900159c60 2023-10-10T20:41:09.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:09.247+0000 7f090709a700 1 -- 172.21.15.89:0/3470278616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f090006a5b0 con 0x7f0900159c60 2023-10-10T20:41:10.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.226+0000 7f08f4ff9700 1 -- 172.21.15.89:0/3470278616 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v72) v1 ==== 118+0+0 (secure 0 0 0) 0x7f08f8055210 con 0x7f0900159c60 2023-10-10T20:41:10.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.228+0000 7f090709a700 1 -- 172.21.15.89:0/3470278616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f08ec067320 msgr2=0x7f08ec0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:10.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.228+0000 7f090709a700 1 --2- 172.21.15.89:0/3470278616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f08ec067320 0x7f08ec0697d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009880 tx=0x7f08fc018000 comp rx=0 tx=0).stop 2023-10-10T20:41:10.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.228+0000 7f090709a700 1 -- 172.21.15.89:0/3470278616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0900159c60 msgr2=0x7f0900158470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:10.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.228+0000 7f090709a700 1 --2- 172.21.15.89:0/3470278616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0900159c60 0x7f0900158470 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f08f800ea20 tx=0x7f08f80091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:10.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.228+0000 7f090709a700 1 -- 172.21.15.89:0/3470278616 shutdown_connections 2023-10-10T20:41:10.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.228+0000 7f090709a700 1 --2- 172.21.15.89:0/3470278616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0900159c60 0x7f0900158470 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:10.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.228+0000 7f090709a700 1 --2- 172.21.15.89:0/3470278616 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f08ec067320 0x7f08ec0697d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:10.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.228+0000 7f090709a700 1 --2- 172.21.15.89:0/3470278616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0900069560 0x7f09000c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:10.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.228+0000 7f090709a700 1 --2- 172.21.15.89:0/3470278616 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0900159890 0x7f0900153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:10.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.228+0000 7f090709a700 1 -- 172.21.15.89:0/3470278616 >> 172.21.15.89:0/3470278616 conn(0x7f0900068e20 msgr2=0x7f09000d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:10.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.228+0000 7f090709a700 1 -- 172.21.15.89:0/3470278616 shutdown_connections 2023-10-10T20:41:10.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.228+0000 7f090709a700 1 -- 172.21.15.89:0/3470278616 wait complete. 2023-10-10T20:41:10.244 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:41:10.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.363+0000 7fd6011f9700 1 Processor -- start 2023-10-10T20:41:10.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.363+0000 7fd6011f9700 1 -- start start 2023-10-10T20:41:10.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.364+0000 7fd6011f9700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd5fc0b4e70 0x7fd5fc0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:10.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.364+0000 7fd6011f9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd5fc0b16d0 0x7fd5fc0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:10.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.364+0000 7fd6011f9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5fc15d240 0x7fd5fc15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:10.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.364+0000 7fd6011f9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5fc0b2080 con 0x7fd5fc0b16d0 2023-10-10T20:41:10.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.364+0000 7fd6011f9700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5fc0cfb00 con 0x7fd5fc0b4e70 2023-10-10T20:41:10.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.364+0000 7fd6011f9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0b5c40 con 0x7fd5fc15d240 2023-10-10T20:41:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.364+0000 7fd5fb7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd5fc0b16d0 0x7fd5fc0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.364+0000 7fd5fb7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd5fc0b16d0 0x7fd5fc0b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49468/0 (socket says 172.21.15.89:49468) 2023-10-10T20:41:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.364+0000 7fd5fb7fe700 1 -- 172.21.15.89:0/2906141766 learned_addr learned my addr 172.21.15.89:0/2906141766 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.365+0000 7fd5fb7fe700 1 -- 172.21.15.89:0/2906141766 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5fc15d240 msgr2=0x7fd5fc15d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.365+0000 7fd5fbfff700 1 --2- 172.21.15.89:0/2906141766 >> v2:172.21.15.111:3300/0 conn(0x7fd5fc0b4e70 0x7fd5fc0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:10.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.365+0000 7fd6009f8700 1 --2- 172.21.15.89:0/2906141766 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5fc15d240 0x7fd5fc15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:10.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.365+0000 7fd5fb7fe700 1 --2- 172.21.15.89:0/2906141766 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5fc15d240 0x7fd5fc15d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:10.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.365+0000 7fd5fb7fe700 1 -- 172.21.15.89:0/2906141766 >> v2:172.21.15.111:3300/0 conn(0x7fd5fc0b4e70 msgr2=0x7fd5fc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:10.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.365+0000 7fd5fb7fe700 1 --2- 172.21.15.89:0/2906141766 >> v2:172.21.15.111:3300/0 conn(0x7fd5fc0b4e70 0x7fd5fc0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:10.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.365+0000 7fd5fb7fe700 1 -- 172.21.15.89:0/2906141766 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc0b5f20 con 0x7fd5fc0b16d0 2023-10-10T20:41:10.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.365+0000 7fd5fb7fe700 1 --2- 172.21.15.89:0/2906141766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5fc0b16d0 0x7fd5fc0b1b40 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc064910 tx=0x7fd5f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec85e2c2e1cc210 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:10.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.365+0000 7fd5faffd700 1 -- 172.21.15.89:0/2906141766 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f000b860 con 0x7fd5fc0b16d0 2023-10-10T20:41:10.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.365+0000 7fd5faffd700 1 -- 172.21.15.89:0/2906141766 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f000b9c0 con 0x7fd5fc0b16d0 2023-10-10T20:41:10.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.365+0000 7fd5faffd700 1 -- 172.21.15.89:0/2906141766 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f000bd60 con 0x7fd5fc0b16d0 2023-10-10T20:41:10.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.366+0000 7fd6011f9700 1 -- 172.21.15.89:0/2906141766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5fc0b16d0 msgr2=0x7fd5fc0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.366+0000 7fd6011f9700 1 --2- 172.21.15.89:0/2906141766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5fc0b16d0 0x7fd5fc0b1b40 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc064910 tx=0x7fd5f0009250 comp rx=0 tx=0).stop 2023-10-10T20:41:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.366+0000 7fd6011f9700 1 -- 172.21.15.89:0/2906141766 shutdown_connections 2023-10-10T20:41:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.366+0000 7fd6011f9700 1 --2- 172.21.15.89:0/2906141766 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5fc15d240 0x7fd5fc15d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.366+0000 7fd6011f9700 1 --2- 172.21.15.89:0/2906141766 >> v2:172.21.15.111:3300/0 conn(0x7fd5fc0b4e70 0x7fd5fc0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.366+0000 7fd6011f9700 1 --2- 172.21.15.89:0/2906141766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5fc0b16d0 0x7fd5fc0b1b40 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.366+0000 7fd6011f9700 1 -- 172.21.15.89:0/2906141766 >> 172.21.15.89:0/2906141766 conn(0x7fd5fc0d10c0 msgr2=0x7fd5fc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.366+0000 7fd6011f9700 1 -- 172.21.15.89:0/2906141766 shutdown_connections 2023-10-10T20:41:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.366+0000 7fd6011f9700 1 -- 172.21.15.89:0/2906141766 wait complete. 2023-10-10T20:41:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.367+0000 7fd6011f9700 1 Processor -- start 2023-10-10T20:41:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.367+0000 7fd6011f9700 1 -- start start 2023-10-10T20:41:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd6011f9700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5fc0b16d0 0x7fd5fc1477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd6011f9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5fc0b4e70 0x7fd5fc169c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd6011f9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5fc15d240 0x7fd5fc16a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd6011f9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0c8fc0 con 0x7fd5fc0b4e70 2023-10-10T20:41:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd6011f9700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0c8440 con 0x7fd5fc0b16d0 2023-10-10T20:41:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd6011f9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0c8720 con 0x7fd5fc15d240 2023-10-10T20:41:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd5fb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5fc0b4e70 0x7fd5fc169c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd5fb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5fc0b4e70 0x7fd5fc169c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49470/0 (socket says 172.21.15.89:49470) 2023-10-10T20:41:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd5fb7fe700 1 -- 172.21.15.89:0/233631250 learned_addr learned my addr 172.21.15.89:0/233631250 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd6009f8700 1 --2- 172.21.15.89:0/233631250 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5fc15d240 0x7fd5fc16a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd5fbfff700 1 --2- 172.21.15.89:0/233631250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5fc0b16d0 0x7fd5fc1477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:10.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd5fb7fe700 1 -- 172.21.15.89:0/233631250 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5fc15d240 msgr2=0x7fd5fc16a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:10.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd5fb7fe700 1 --2- 172.21.15.89:0/233631250 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5fc15d240 0x7fd5fc16a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:10.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd5fb7fe700 1 -- 172.21.15.89:0/233631250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5fc0b16d0 msgr2=0x7fd5fc1477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:10.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd5fb7fe700 1 --2- 172.21.15.89:0/233631250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5fc0b16d0 0x7fd5fc1477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:10.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd5fb7fe700 1 -- 172.21.15.89:0/233631250 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f00094f0 con 0x7fd5fc0b4e70 2023-10-10T20:41:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd5fbfff700 1 --2- 172.21.15.89:0/233631250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5fc0b16d0 0x7fd5fc1477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd5fb7fe700 1 --2- 172.21.15.89:0/233631250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5fc0b4e70 0x7fd5fc169c40 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc070dd0 tx=0x7fd5f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd5e3fff700 1 -- 172.21.15.89:0/233631250 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0011e70 con 0x7fd5fc0b4e70 2023-10-10T20:41:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd5e3fff700 1 -- 172.21.15.89:0/233631250 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f0024440 con 0x7fd5fc0b4e70 2023-10-10T20:41:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd6011f9700 1 -- 172.21.15.89:0/233631250 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5fc0c8a00 con 0x7fd5fc0b4e70 2023-10-10T20:41:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd5e3fff700 1 -- 172.21.15.89:0/233631250 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0024800 con 0x7fd5fc0b4e70 2023-10-10T20:41:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.368+0000 7fd6011f9700 1 -- 172.21.15.89:0/233631250 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5fc0cc0e0 con 0x7fd5fc0b4e70 2023-10-10T20:41:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.369+0000 7fd6011f9700 1 -- 172.21.15.89:0/233631250 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5fc088990 con 0x7fd5fc0b4e70 2023-10-10T20:41:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.370+0000 7fd5e3fff700 1 -- 172.21.15.89:0/233631250 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd5f001a420 con 0x7fd5fc0b4e70 2023-10-10T20:41:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.370+0000 7fd5e3fff700 1 --2- 172.21.15.89:0/233631250 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd5e8067320 0x7fd5e80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:10.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.370+0000 7fd5e3fff700 1 -- 172.21.15.89:0/233631250 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd5f0086bd0 con 0x7fd5fc0b4e70 2023-10-10T20:41:10.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.370+0000 7fd5fbfff700 1 --2- 172.21.15.89:0/233631250 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd5e8067320 0x7fd5e80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:10.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.370+0000 7fd5fbfff700 1 --2- 172.21.15.89:0/233631250 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd5e8067320 0x7fd5e80697d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc16d160 tx=0x7fd5ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:10.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.374+0000 7fd5e3fff700 1 -- 172.21.15.89:0/233631250 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5f0053860 con 0x7fd5fc0b4e70 2023-10-10T20:41:10.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:10.598+0000 7fd6011f9700 1 -- 172.21.15.89:0/233631250 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd5fc04bb80 con 0x7fd5fc0b4e70 2023-10-10T20:41:11.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.233+0000 7fd5e3fff700 1 -- 172.21.15.89:0/233631250 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v73) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd5f00150c0 con 0x7fd5fc0b4e70 2023-10-10T20:41:11.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.236+0000 7fd6011f9700 1 -- 172.21.15.89:0/233631250 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd5e8067320 msgr2=0x7fd5e80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:11.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.236+0000 7fd6011f9700 1 --2- 172.21.15.89:0/233631250 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd5e8067320 0x7fd5e80697d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc16d160 tx=0x7fd5ec008040 comp rx=0 tx=0).stop 2023-10-10T20:41:11.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.236+0000 7fd6011f9700 1 -- 172.21.15.89:0/233631250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5fc0b4e70 msgr2=0x7fd5fc169c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:11.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.236+0000 7fd6011f9700 1 --2- 172.21.15.89:0/233631250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5fc0b4e70 0x7fd5fc169c40 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc070dd0 tx=0x7fd5f0017000 comp rx=0 tx=0).stop 2023-10-10T20:41:11.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.236+0000 7fd6011f9700 1 -- 172.21.15.89:0/233631250 shutdown_connections 2023-10-10T20:41:11.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.236+0000 7fd6011f9700 1 --2- 172.21.15.89:0/233631250 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5fc15d240 0x7fd5fc16a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.236+0000 7fd6011f9700 1 --2- 172.21.15.89:0/233631250 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd5e8067320 0x7fd5e80697d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.236+0000 7fd6011f9700 1 --2- 172.21.15.89:0/233631250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5fc0b16d0 0x7fd5fc1477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.236+0000 7fd6011f9700 1 --2- 172.21.15.89:0/233631250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5fc0b4e70 0x7fd5fc169c40 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.236+0000 7fd6011f9700 1 -- 172.21.15.89:0/233631250 >> 172.21.15.89:0/233631250 conn(0x7fd5fc0d10c0 msgr2=0x7fd5fc0b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:11.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.236+0000 7fd6011f9700 1 -- 172.21.15.89:0/233631250 shutdown_connections 2023-10-10T20:41:11.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.236+0000 7fd6011f9700 1 -- 172.21.15.89:0/233631250 wait complete. 2023-10-10T20:41:11.250 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:41:11.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.363+0000 7f4adffc0700 1 Processor -- start 2023-10-10T20:41:11.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.363+0000 7f4adffc0700 1 -- start start 2023-10-10T20:41:11.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.364+0000 7f4adffc0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4ad8159890 0x7f4ad815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:11.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.364+0000 7f4adffc0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad815a160 con 0x7f4ad8159c60 2023-10-10T20:41:11.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.364+0000 7f4adffc0700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad80d13a0 con 0x7f4ad8069560 2023-10-10T20:41:11.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.364+0000 7f4adffc0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad815e140 con 0x7f4ad8159890 2023-10-10T20:41:11.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.364+0000 7f4addd5c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4ad8159890 0x7f4ad815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:11.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.364+0000 7f4add55b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4ad8159c60 0x7f4ad815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41328/0 (socket says 172.21.15.89:41328) 2023-10-10T20:41:11.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.364+0000 7f4add55b700 1 -- 172.21.15.89:0/1352704608 learned_addr learned my addr 172.21.15.89:0/1352704608 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:11.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.364+0000 7f4addd5c700 1 -- 172.21.15.89:0/1352704608 >> v1:172.21.15.111:6789/0 conn(0x7f4ad8069560 legacy=0x7f4ad80ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.364+0000 7f4addd5c700 1 -- 172.21.15.89:0/1352704608 >> v1:172.21.15.89:6789/0 conn(0x7f4ad8159c60 legacy=0x7f4ad815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.364+0000 7f4addd5c700 1 -- 172.21.15.89:0/1352704608 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad815a3d0 con 0x7f4ad8159890 2023-10-10T20:41:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.364+0000 7f4adcd5a700 1 -- 172.21.15.89:0/1352704608 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559783193 0 0) 0x7f4ad815a160 con 0x7f4ad8159c60 2023-10-10T20:41:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.364+0000 7f4addd5c700 1 --2- 172.21.15.89:0/1352704608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ad8159890 0x7f4ad815abd0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4ad40098e0 tx=0x7f4ad40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839e84c029c540f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.365+0000 7f4adcd5a700 1 -- 172.21.15.89:0/1352704608 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4009e80 con 0x7f4ad8159890 2023-10-10T20:41:11.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.365+0000 7f4adcd5a700 1 -- 172.21.15.89:0/1352704608 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad400b860 con 0x7f4ad8159890 2023-10-10T20:41:11.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.365+0000 7f4adcd5a700 1 -- 172.21.15.89:0/1352704608 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad400bbe0 con 0x7f4ad8159890 2023-10-10T20:41:11.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.365+0000 7f4adffc0700 1 -- 172.21.15.89:0/1352704608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ad8159890 msgr2=0x7f4ad815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:11.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.365+0000 7f4adffc0700 1 --2- 172.21.15.89:0/1352704608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ad8159890 0x7f4ad815abd0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4ad40098e0 tx=0x7f4ad40091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.365+0000 7f4adffc0700 1 -- 172.21.15.89:0/1352704608 shutdown_connections 2023-10-10T20:41:11.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.365+0000 7f4adffc0700 1 --2- 172.21.15.89:0/1352704608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ad8159890 0x7f4ad815abd0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.365+0000 7f4adffc0700 1 -- 172.21.15.89:0/1352704608 >> 172.21.15.89:0/1352704608 conn(0x7f4ad8068e20 msgr2=0x7f4ad80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:11.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.365+0000 7f4adffc0700 1 -- 172.21.15.89:0/1352704608 shutdown_connections 2023-10-10T20:41:11.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.365+0000 7f4adffc0700 1 -- 172.21.15.89:0/1352704608 wait complete. 2023-10-10T20:41:11.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.366+0000 7f4adffc0700 1 Processor -- start 2023-10-10T20:41:11.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.366+0000 7f4adffc0700 1 -- start start 2023-10-10T20:41:11.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.367+0000 7f4adffc0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ad8069560 0x7f4ad814d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:11.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.367+0000 7f4adffc0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4ad8159890 0x7f4ad815d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:11.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.367+0000 7f4adffc0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ad8159c60 0x7f4ad815d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:11.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.367+0000 7f4adffc0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad814da20 con 0x7f4ad8069560 2023-10-10T20:41:11.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.367+0000 7f4adffc0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad80b3ec0 con 0x7f4ad8159c60 2023-10-10T20:41:11.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.367+0000 7f4adffc0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad80cef50 con 0x7f4ad8159890 2023-10-10T20:41:11.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.367+0000 7f4addd5c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ad8069560 0x7f4ad814d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:11.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.367+0000 7f4addd5c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ad8069560 0x7f4ad814d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49486/0 (socket says 172.21.15.89:49486) 2023-10-10T20:41:11.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.367+0000 7f4addd5c700 1 -- 172.21.15.89:0/101710717 learned_addr learned my addr 172.21.15.89:0/101710717 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:11.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.367+0000 7f4addd5c700 1 -- 172.21.15.89:0/101710717 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4ad8159890 msgr2=0x7f4ad815d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:11.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.367+0000 7f4addd5c700 1 --2- 172.21.15.89:0/101710717 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4ad8159890 0x7f4ad815d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.367+0000 7f4addd5c700 1 -- 172.21.15.89:0/101710717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ad8159c60 msgr2=0x7f4ad815d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:11.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.367+0000 7f4addd5c700 1 --2- 172.21.15.89:0/101710717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ad8159c60 0x7f4ad815d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.367+0000 7f4addd5c700 1 -- 172.21.15.89:0/101710717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad4009420 con 0x7f4ad8069560 2023-10-10T20:41:11.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.368+0000 7f4addd5c700 1 --2- 172.21.15.89:0/101710717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ad8069560 0x7f4ad814d4e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8070e20 tx=0x7f4ad4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:11.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.368+0000 7f4acdffb700 1 -- 172.21.15.89:0/101710717 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4011420 con 0x7f4ad8069560 2023-10-10T20:41:11.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.368+0000 7f4acdffb700 1 -- 172.21.15.89:0/101710717 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad4011580 con 0x7f4ad8069560 2023-10-10T20:41:11.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.368+0000 7f4adffc0700 1 -- 172.21.15.89:0/101710717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad80b3590 con 0x7f4ad8069560 2023-10-10T20:41:11.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.368+0000 7f4acdffb700 1 -- 172.21.15.89:0/101710717 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4011920 con 0x7f4ad8069560 2023-10-10T20:41:11.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.368+0000 7f4adffc0700 1 -- 172.21.15.89:0/101710717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad80b3b50 con 0x7f4ad8069560 2023-10-10T20:41:11.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.369+0000 7f4adffc0700 1 -- 172.21.15.89:0/101710717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad804c3c0 con 0x7f4ad8069560 2023-10-10T20:41:11.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.369+0000 7f4acdffb700 1 -- 172.21.15.89:0/101710717 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4ad4011c90 con 0x7f4ad8069560 2023-10-10T20:41:11.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.370+0000 7f4acdffb700 1 --2- 172.21.15.89:0/101710717 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4abc067320 0x7f4abc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:11.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.370+0000 7f4acdffb700 1 -- 172.21.15.89:0/101710717 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4ad40867d0 con 0x7f4ad8069560 2023-10-10T20:41:11.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.370+0000 7f4add55b700 1 --2- 172.21.15.89:0/101710717 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4abc067320 0x7f4abc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:11.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.370+0000 7f4add55b700 1 --2- 172.21.15.89:0/101710717 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4abc067320 0x7f4abc0697d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8007fd0 tx=0x7f4ac8007bb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:11.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.374+0000 7f4acdffb700 1 -- 172.21.15.89:0/101710717 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ad40534e0 con 0x7f4ad8069560 2023-10-10T20:41:11.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.597+0000 7f4adffc0700 1 -- 172.21.15.89:0/101710717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ad80b1720 con 0x7f4ad8069560 2023-10-10T20:41:11.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.598+0000 7f4acdffb700 1 -- 172.21.15.89:0/101710717 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f4ad4056b00 con 0x7f4ad8069560 2023-10-10T20:41:11.601 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:11.601 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":73,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6517,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/2658508419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2658508419},{"type":"v1","addr":"172.21.15.111:6834","nonce":2658508419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7185,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3349522977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3349522977},{"type":"v1","addr":"172.21.15.89:6835","nonce":3349522977}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7188,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3182024657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3182024657},{"type":"v1","addr":"172.21.15.89:6837","nonce":3182024657}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:41:07.565995+0000","modified":"2023-10-10T20:41:11.225905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"d","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/2582059153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2582059153},{"type":"v1","addr":"172.21.15.111:6835","nonce":2582059153}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-10T20:41:11.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.600+0000 7f4adffc0700 1 -- 172.21.15.89:0/101710717 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4abc067320 msgr2=0x7f4abc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:11.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.600+0000 7f4adffc0700 1 --2- 172.21.15.89:0/101710717 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4abc067320 0x7f4abc0697d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8007fd0 tx=0x7f4ac8007bb0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.600+0000 7f4adffc0700 1 -- 172.21.15.89:0/101710717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ad8069560 msgr2=0x7f4ad814d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:11.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.600+0000 7f4adffc0700 1 --2- 172.21.15.89:0/101710717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ad8069560 0x7f4ad814d4e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8070e20 tx=0x7f4ad4017000 comp rx=0 tx=0).stop 2023-10-10T20:41:11.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.600+0000 7f4adffc0700 1 -- 172.21.15.89:0/101710717 shutdown_connections 2023-10-10T20:41:11.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.600+0000 7f4adffc0700 1 --2- 172.21.15.89:0/101710717 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4abc067320 0x7f4abc0697d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.600+0000 7f4adffc0700 1 --2- 172.21.15.89:0/101710717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ad8159c60 0x7f4ad815d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.600+0000 7f4adffc0700 1 --2- 172.21.15.89:0/101710717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ad8069560 0x7f4ad814d4e0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.600+0000 7f4adffc0700 1 --2- 172.21.15.89:0/101710717 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4ad8159890 0x7f4ad815d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.600+0000 7f4adffc0700 1 -- 172.21.15.89:0/101710717 >> 172.21.15.89:0/101710717 conn(0x7f4ad8068e20 msgr2=0x7f4ad80b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:11.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.600+0000 7f4adffc0700 1 -- 172.21.15.89:0/101710717 shutdown_connections 2023-10-10T20:41:11.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.600+0000 7f4adffc0700 1 -- 172.21.15.89:0/101710717 wait complete. 2023-10-10T20:41:11.607 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 73 2023-10-10T20:41:11.614 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:41:11.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.727+0000 7f4c905d9700 1 Processor -- start 2023-10-10T20:41:11.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.727+0000 7f4c905d9700 1 -- start start 2023-10-10T20:41:11.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.728+0000 7f4c905d9700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4c88159890 0x7f4c8815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:11.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.728+0000 7f4c905d9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c88159c60 0x7f4c8815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:11.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.728+0000 7f4c905d9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4c8815db10 0x7f4c88069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:11.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.728+0000 7f4c905d9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c8815a160 con 0x7f4c8815db10 2023-10-10T20:41:11.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.728+0000 7f4c905d9700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c880d13a0 con 0x7f4c88159890 2023-10-10T20:41:11.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.728+0000 7f4c905d9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8815e140 con 0x7f4c88159c60 2023-10-10T20:41:11.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.728+0000 7f4c8eb76700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4c8815db10 0x7f4c88069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:11.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.728+0000 7f4c8eb76700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4c8815db10 0x7f4c88069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49490/0 (socket says 172.21.15.89:49490) 2023-10-10T20:41:11.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.728+0000 7f4c8eb76700 1 -- 172.21.15.89:0/536473981 learned_addr learned my addr 172.21.15.89:0/536473981 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:11.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.729+0000 7f4c8db74700 1 --2- 172.21.15.89:0/536473981 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c88159c60 0x7f4c8815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:11.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.729+0000 7f4c8e375700 1 --2- 172.21.15.89:0/536473981 >> v2:172.21.15.111:3300/0 conn(0x7f4c88159890 0x7f4c8815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:11.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.729+0000 7f4c8eb76700 1 -- 172.21.15.89:0/536473981 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c88159c60 msgr2=0x7f4c8815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:11.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.729+0000 7f4c8eb76700 1 --2- 172.21.15.89:0/536473981 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c88159c60 0x7f4c8815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.729+0000 7f4c8eb76700 1 -- 172.21.15.89:0/536473981 >> v2:172.21.15.111:3300/0 conn(0x7f4c88159890 msgr2=0x7f4c8815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:11.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.729+0000 7f4c8eb76700 1 --2- 172.21.15.89:0/536473981 >> v2:172.21.15.111:3300/0 conn(0x7f4c88159890 0x7f4c8815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.729+0000 7f4c8eb76700 1 -- 172.21.15.89:0/536473981 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8815a3d0 con 0x7f4c8815db10 2023-10-10T20:41:11.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.729+0000 7f4c8e375700 1 --2- 172.21.15.89:0/536473981 >> v2:172.21.15.111:3300/0 conn(0x7f4c88159890 0x7f4c8815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:11.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.729+0000 7f4c8eb76700 1 --2- 172.21.15.89:0/536473981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c8815db10 0x7f4c88069560 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f4c880d1090 tx=0x7f4c80009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8493b2aa6720f6ac server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:11.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.729+0000 7f4c8d373700 1 -- 172.21.15.89:0/536473981 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8000e070 con 0x7f4c8815db10 2023-10-10T20:41:11.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.729+0000 7f4c8d373700 1 -- 172.21.15.89:0/536473981 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8000ba50 con 0x7f4c8815db10 2023-10-10T20:41:11.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.729+0000 7f4c8d373700 1 -- 172.21.15.89:0/536473981 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c800092e0 con 0x7f4c8815db10 2023-10-10T20:41:11.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.729+0000 7f4c905d9700 1 -- 172.21.15.89:0/536473981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c8815db10 msgr2=0x7f4c88069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:11.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.729+0000 7f4c905d9700 1 --2- 172.21.15.89:0/536473981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c8815db10 0x7f4c88069560 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f4c880d1090 tx=0x7f4c80009840 comp rx=0 tx=0).stop 2023-10-10T20:41:11.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.729+0000 7f4c905d9700 1 -- 172.21.15.89:0/536473981 shutdown_connections 2023-10-10T20:41:11.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.729+0000 7f4c905d9700 1 --2- 172.21.15.89:0/536473981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c8815db10 0x7f4c88069560 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.729+0000 7f4c905d9700 1 --2- 172.21.15.89:0/536473981 >> v2:172.21.15.111:3300/0 conn(0x7f4c88159890 0x7f4c8815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.729+0000 7f4c905d9700 1 --2- 172.21.15.89:0/536473981 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c88159c60 0x7f4c8815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.730+0000 7f4c905d9700 1 -- 172.21.15.89:0/536473981 >> 172.21.15.89:0/536473981 conn(0x7f4c88068e20 msgr2=0x7f4c880d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:11.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.730+0000 7f4c905d9700 1 -- 172.21.15.89:0/536473981 shutdown_connections 2023-10-10T20:41:11.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.730+0000 7f4c905d9700 1 -- 172.21.15.89:0/536473981 wait complete. 2023-10-10T20:41:11.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.731+0000 7f4c905d9700 1 Processor -- start 2023-10-10T20:41:11.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.731+0000 7f4c905d9700 1 -- start start 2023-10-10T20:41:11.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.731+0000 7f4c905d9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c88159890 0x7f4c8814feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:11.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.731+0000 7f4c905d9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c88159c60 0x7f4c880cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:11.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.731+0000 7f4c905d9700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c8815db10 0x7f4c880cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:11.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.731+0000 7f4c905d9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c880ca5b0 con 0x7f4c88159890 2023-10-10T20:41:11.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.731+0000 7f4c905d9700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c88154460 con 0x7f4c8815db10 2023-10-10T20:41:11.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.731+0000 7f4c905d9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c88154740 con 0x7f4c88159c60 2023-10-10T20:41:11.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.731+0000 7f4c8e375700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c88159890 0x7f4c8814feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:11.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.731+0000 7f4c8e375700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c88159890 0x7f4c8814feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49496/0 (socket says 172.21.15.89:49496) 2023-10-10T20:41:11.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.732+0000 7f4c8e375700 1 -- 172.21.15.89:0/3761417449 learned_addr learned my addr 172.21.15.89:0/3761417449 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:11.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.732+0000 7f4c8e375700 1 -- 172.21.15.89:0/3761417449 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c88159c60 msgr2=0x7f4c880cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:11.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.732+0000 7f4c8e375700 1 --2- 172.21.15.89:0/3761417449 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c88159c60 0x7f4c880cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.732+0000 7f4c8e375700 1 -- 172.21.15.89:0/3761417449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c8815db10 msgr2=0x7f4c880cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:11.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.732+0000 7f4c8e375700 1 --2- 172.21.15.89:0/3761417449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c8815db10 0x7f4c880cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.732+0000 7f4c8e375700 1 -- 172.21.15.89:0/3761417449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c80009ae0 con 0x7f4c88159890 2023-10-10T20:41:11.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.732+0000 7f4c8e375700 1 --2- 172.21.15.89:0/3761417449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c88159890 0x7f4c8814feb0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400c830 tx=0x7f4c84008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:11.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.732+0000 7f4c7e7fc700 1 -- 172.21.15.89:0/3761417449 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84011600 con 0x7f4c88159890 2023-10-10T20:41:11.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.732+0000 7f4c7e7fc700 1 -- 172.21.15.89:0/3761417449 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c84011760 con 0x7f4c88159890 2023-10-10T20:41:11.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.732+0000 7f4c905d9700 1 -- 172.21.15.89:0/3761417449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c88154a20 con 0x7f4c88159890 2023-10-10T20:41:11.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.732+0000 7f4c7e7fc700 1 -- 172.21.15.89:0/3761417449 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84011b20 con 0x7f4c88159890 2023-10-10T20:41:11.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.732+0000 7f4c905d9700 1 -- 172.21.15.89:0/3761417449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c881540e0 con 0x7f4c88159890 2023-10-10T20:41:11.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.733+0000 7f4c905d9700 1 -- 172.21.15.89:0/3761417449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c8804c3c0 con 0x7f4c88159890 2023-10-10T20:41:11.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.733+0000 7f4c7e7fc700 1 -- 172.21.15.89:0/3761417449 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4c8400a3e0 con 0x7f4c88159890 2023-10-10T20:41:11.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.734+0000 7f4c7e7fc700 1 --2- 172.21.15.89:0/3761417449 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c6c067320 0x7f4c6c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:11.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.734+0000 7f4c7e7fc700 1 -- 172.21.15.89:0/3761417449 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4c84085250 con 0x7f4c88159890 2023-10-10T20:41:11.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.734+0000 7f4c8db74700 1 --2- 172.21.15.89:0/3761417449 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c6c067320 0x7f4c6c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:11.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.734+0000 7f4c8db74700 1 --2- 172.21.15.89:0/3761417449 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c6c067320 0x7f4c6c0697d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4c881592b0 tx=0x7f4c78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:11.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.738+0000 7f4c7e7fc700 1 -- 172.21.15.89:0/3761417449 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c84051dd0 con 0x7f4c88159890 2023-10-10T20:41:11.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.922+0000 7f4c905d9700 1 -- 172.21.15.89:0/3761417449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c8806a5b0 con 0x7f4c88159890 2023-10-10T20:41:11.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.923+0000 7f4c7e7fc700 1 -- 172.21.15.89:0/3761417449 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+15364 (secure 0 0 0) 0x7f4c840553f0 con 0x7f4c88159890 2023-10-10T20:41:11.926 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:11.926 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":112,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:41:09.202940+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:41:05.928269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-10T20:41:06.527456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:41:11.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.925+0000 7f4c905d9700 1 -- 172.21.15.89:0/3761417449 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c6c067320 msgr2=0x7f4c6c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:11.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.925+0000 7f4c905d9700 1 --2- 172.21.15.89:0/3761417449 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c6c067320 0x7f4c6c0697d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4c881592b0 tx=0x7f4c78008040 comp rx=0 tx=0).stop 2023-10-10T20:41:11.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.925+0000 7f4c905d9700 1 -- 172.21.15.89:0/3761417449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c88159890 msgr2=0x7f4c8814feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:11.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.925+0000 7f4c905d9700 1 --2- 172.21.15.89:0/3761417449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c88159890 0x7f4c8814feb0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400c830 tx=0x7f4c84008040 comp rx=0 tx=0).stop 2023-10-10T20:41:11.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.925+0000 7f4c905d9700 1 -- 172.21.15.89:0/3761417449 shutdown_connections 2023-10-10T20:41:11.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.926+0000 7f4c905d9700 1 --2- 172.21.15.89:0/3761417449 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4c6c067320 0x7f4c6c0697d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.926+0000 7f4c905d9700 1 --2- 172.21.15.89:0/3761417449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c8815db10 0x7f4c880cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.926+0000 7f4c905d9700 1 --2- 172.21.15.89:0/3761417449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c88159890 0x7f4c8814feb0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.926+0000 7f4c905d9700 1 --2- 172.21.15.89:0/3761417449 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c88159c60 0x7f4c880cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:11.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.926+0000 7f4c905d9700 1 -- 172.21.15.89:0/3761417449 >> 172.21.15.89:0/3761417449 conn(0x7f4c88068e20 msgr2=0x7f4c88156f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:11.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.926+0000 7f4c905d9700 1 -- 172.21.15.89:0/3761417449 shutdown_connections 2023-10-10T20:41:11.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:11.926+0000 7f4c905d9700 1 -- 172.21.15.89:0/3761417449 wait complete. 2023-10-10T20:41:11.940 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:41:11.940 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:41:12.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.053+0000 7fe3fb9a9700 1 Processor -- start 2023-10-10T20:41:12.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.053+0000 7fe3fb9a9700 1 -- start start 2023-10-10T20:41:12.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.054+0000 7fe3fb9a9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe3f41579f0 0x7fe3f4162530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.054+0000 7fe3fb9a9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f40cb300 con 0x7fe3f40b1c20 2023-10-10T20:41:12.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.054+0000 7fe3fb9a9700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f40cb5e0 con 0x7fe3f40b1850 2023-10-10T20:41:12.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.054+0000 7fe3fb9a9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f406fc30 con 0x7fe3f41579f0 2023-10-10T20:41:12.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.054+0000 7fe3f8f44700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe3f40b1c20 0x7fe3f41572e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41340/0 (socket says 172.21.15.89:41340) 2023-10-10T20:41:12.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.054+0000 7fe3f8f44700 1 -- 172.21.15.89:0/582637119 learned_addr learned my addr 172.21.15.89:0/582637119 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:12.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3f9f46700 1 --2- 172.21.15.89:0/582637119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe3f41579f0 0x7fe3f4162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3ebfff700 1 -- 172.21.15.89:0/582637119 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865446022 0 0) 0x7fe3f40cb300 con 0x7fe3f40b1c20 2023-10-10T20:41:12.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3ebfff700 1 -- 172.21.15.89:0/582637119 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3e0003570 con 0x7fe3f40b1c20 2023-10-10T20:41:12.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3ebfff700 1 -- 172.21.15.89:0/582637119 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3384828293 0 0) 0x7fe3f40cb5e0 con 0x7fe3f40b1850 2023-10-10T20:41:12.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3ebfff700 1 -- 172.21.15.89:0/582637119 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3f40cb300 con 0x7fe3f40b1850 2023-10-10T20:41:12.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3ebfff700 1 -- 172.21.15.89:0/582637119 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3428734969 0 0) 0x7fe3e0003570 con 0x7fe3f40b1c20 2023-10-10T20:41:12.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3ebfff700 1 -- 172.21.15.89:0/582637119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe3f41579f0 msgr2=0x7fe3f4162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:12.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3ebfff700 1 --2- 172.21.15.89:0/582637119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe3f41579f0 0x7fe3f4162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3ebfff700 1 -- 172.21.15.89:0/582637119 >> v1:172.21.15.111:6789/0 conn(0x7fe3f40b1850 legacy=0x7fe3f40ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:12.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3ebfff700 1 -- 172.21.15.89:0/582637119 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f40cb8c0 con 0x7fe3f40b1c20 2023-10-10T20:41:12.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3ebfff700 1 -- 172.21.15.89:0/582637119 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fe3e4001470 con 0x7fe3f40b1c20 2023-10-10T20:41:12.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3ebfff700 1 -- 172.21.15.89:0/582637119 >> v1:172.21.15.89:6789/0 conn(0x7fe3f40b1c20 legacy=0x7fe3f41572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:12.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3ebfff700 1 --2- 172.21.15.89:0/582637119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3e0003c20 0x7fe3e0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3ebfff700 1 --2- 172.21.15.89:0/582637119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3e0004810 0x7fe3e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3ebfff700 1 --2- 172.21.15.89:0/582637119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe3f41579f0 0x7fe3e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3ebfff700 1 -- 172.21.15.89:0/582637119 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f406fc30 con 0x7fe3e0003c20 2023-10-10T20:41:12.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3ebfff700 1 -- 172.21.15.89:0/582637119 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e000bb40 con 0x7fe3e0004810 2023-10-10T20:41:12.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3ebfff700 1 -- 172.21.15.89:0/582637119 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e000bd00 con 0x7fe3f41579f0 2023-10-10T20:41:12.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3f9745700 1 --2- 172.21.15.89:0/582637119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3e0003c20 0x7fe3e0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3f8f44700 1 --2- 172.21.15.89:0/582637119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3e0004810 0x7fe3e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3f9f46700 1 --2- 172.21.15.89:0/582637119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe3f41579f0 0x7fe3e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3f9745700 1 -- 172.21.15.89:0/582637119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe3f41579f0 msgr2=0x7fe3e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:12.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3f9745700 1 --2- 172.21.15.89:0/582637119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe3f41579f0 0x7fe3e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3f9745700 1 -- 172.21.15.89:0/582637119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3e0004810 msgr2=0x7fe3e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:12.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3f9745700 1 --2- 172.21.15.89:0/582637119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3e0004810 0x7fe3e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3f9745700 1 -- 172.21.15.89:0/582637119 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e0003570 con 0x7fe3e0003c20 2023-10-10T20:41:12.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.055+0000 7fe3f9f46700 1 --2- 172.21.15.89:0/582637119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe3f41579f0 0x7fe3e00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:12.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.056+0000 7fe3f9745700 1 --2- 172.21.15.89:0/582637119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3e0003c20 0x7fe3e0004010 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0002120 tx=0x7fe3f00017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867b1e8960394416 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:12.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.056+0000 7fe3ebfff700 1 -- 172.21.15.89:0/582637119 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0001ab0 con 0x7fe3e0003c20 2023-10-10T20:41:12.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.056+0000 7fe3ebfff700 1 -- 172.21.15.89:0/582637119 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f000c980 con 0x7fe3e0003c20 2023-10-10T20:41:12.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.056+0000 7fe3ebfff700 1 -- 172.21.15.89:0/582637119 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f000cd20 con 0x7fe3e0003c20 2023-10-10T20:41:12.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.056+0000 7fe3fb9a9700 1 -- 172.21.15.89:0/582637119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3e0003c20 msgr2=0x7fe3e0004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:12.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.056+0000 7fe3fb9a9700 1 --2- 172.21.15.89:0/582637119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3e0003c20 0x7fe3e0004010 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0002120 tx=0x7fe3f00017a0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.056+0000 7fe3f9f46700 1 -- 172.21.15.89:0/582637119 reap_dead start 2023-10-10T20:41:12.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.056+0000 7fe3fb9a9700 1 -- 172.21.15.89:0/582637119 shutdown_connections 2023-10-10T20:41:12.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.056+0000 7fe3fb9a9700 1 -- 172.21.15.89:0/582637119 >> 172.21.15.89:0/582637119 conn(0x7fe3f406a320 msgr2=0x7fe3f40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:12.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.056+0000 7fe3fb9a9700 1 -- 172.21.15.89:0/582637119 shutdown_connections 2023-10-10T20:41:12.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.056+0000 7fe3fb9a9700 1 -- 172.21.15.89:0/582637119 wait complete. 2023-10-10T20:41:12.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.057+0000 7fe3fb9a9700 1 Processor -- start 2023-10-10T20:41:12.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.057+0000 7fe3fb9a9700 1 -- start start 2023-10-10T20:41:12.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.058+0000 7fe3fb9a9700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3e0003c20 0x7fe3f414b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.058+0000 7fe3fb9a9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3f4154c00 0x7fe3f4155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.058+0000 7fe3fb9a9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe3f414cd70 0x7fe3f414d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.058+0000 7fe3fb9a9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f40cc260 con 0x7fe3f4154c00 2023-10-10T20:41:12.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.058+0000 7fe3fb9a9700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f41555b0 con 0x7fe3e0003c20 2023-10-10T20:41:12.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.058+0000 7fe3fb9a9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f414dac0 con 0x7fe3f414cd70 2023-10-10T20:41:12.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.058+0000 7fe3f8f44700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3f4154c00 0x7fe3f4155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.058+0000 7fe3f8f44700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3f4154c00 0x7fe3f4155070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49520/0 (socket says 172.21.15.89:49520) 2023-10-10T20:41:12.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.058+0000 7fe3f8f44700 1 -- 172.21.15.89:0/1054734417 learned_addr learned my addr 172.21.15.89:0/1054734417 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:12.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.058+0000 7fe3f9745700 1 --2- 172.21.15.89:0/1054734417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3e0003c20 0x7fe3f414b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.058+0000 7fe3f9f46700 1 --2- 172.21.15.89:0/1054734417 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe3f414cd70 0x7fe3f414d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.059+0000 7fe3f8f44700 1 -- 172.21.15.89:0/1054734417 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe3f414cd70 msgr2=0x7fe3f414d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:12.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.059+0000 7fe3f8f44700 1 --2- 172.21.15.89:0/1054734417 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe3f414cd70 0x7fe3f414d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.059+0000 7fe3f8f44700 1 -- 172.21.15.89:0/1054734417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3e0003c20 msgr2=0x7fe3f414b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:12.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.059+0000 7fe3f8f44700 1 --2- 172.21.15.89:0/1054734417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3e0003c20 0x7fe3f414b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.059+0000 7fe3f8f44700 1 -- 172.21.15.89:0/1054734417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f414dda0 con 0x7fe3f4154c00 2023-10-10T20:41:12.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.059+0000 7fe3f8f44700 1 --2- 172.21.15.89:0/1054734417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3f4154c00 0x7fe3f4155070 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40073f0 tx=0x7fe3e4001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:12.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.059+0000 7fe3e97fa700 1 -- 172.21.15.89:0/1054734417 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4018030 con 0x7fe3f4154c00 2023-10-10T20:41:12.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.059+0000 7fe3e97fa700 1 -- 172.21.15.89:0/1054734417 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e400f8d0 con 0x7fe3f4154c00 2023-10-10T20:41:12.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.059+0000 7fe3fb9a9700 1 -- 172.21.15.89:0/1054734417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f414e080 con 0x7fe3f4154c00 2023-10-10T20:41:12.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.059+0000 7fe3e97fa700 1 -- 172.21.15.89:0/1054734417 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e400fc70 con 0x7fe3f4154c00 2023-10-10T20:41:12.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.059+0000 7fe3fb9a9700 1 -- 172.21.15.89:0/1054734417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f414e640 con 0x7fe3f4154c00 2023-10-10T20:41:12.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.060+0000 7fe3fb9a9700 1 -- 172.21.15.89:0/1054734417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f414e860 con 0x7fe3f4154c00 2023-10-10T20:41:12.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.060+0000 7fe3e97fa700 1 -- 172.21.15.89:0/1054734417 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe3e401b070 con 0x7fe3f4154c00 2023-10-10T20:41:12.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.060+0000 7fe3e97fa700 1 --2- 172.21.15.89:0/1054734417 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe3d8067320 0x7fe3d80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.061+0000 7fe3e97fa700 1 -- 172.21.15.89:0/1054734417 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe3e4019070 con 0x7fe3f4154c00 2023-10-10T20:41:12.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.061+0000 7fe3f9745700 1 --2- 172.21.15.89:0/1054734417 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe3d8067320 0x7fe3d80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.061+0000 7fe3f9745700 1 --2- 172.21.15.89:0/1054734417 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe3d8067320 0x7fe3d80697d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0000f80 tx=0x7fe3f0016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:12.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.065+0000 7fe3e97fa700 1 -- 172.21.15.89:0/1054734417 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3e40517f0 con 0x7fe3f4154c00 2023-10-10T20:41:12.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.243+0000 7fe3fb9a9700 1 -- 172.21.15.89:0/1054734417 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3f404c3c0 con 0x7fe3d8067320 2023-10-10T20:41:12.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.251+0000 7fe3e97fa700 1 -- 172.21.15.89:0/1054734417 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287184 (secure 0 0 0) 0x7fe3f404c3c0 con 0x7fe3d8067320 2023-10-10T20:41:12.257 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:12.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.254+0000 7fe3fb9a9700 1 -- 172.21.15.89:0/1054734417 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe3d8067320 msgr2=0x7fe3d80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:12.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.254+0000 7fe3fb9a9700 1 --2- 172.21.15.89:0/1054734417 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe3d8067320 0x7fe3d80697d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0000f80 tx=0x7fe3f0016000 comp rx=0 tx=0).stop 2023-10-10T20:41:12.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.254+0000 7fe3fb9a9700 1 -- 172.21.15.89:0/1054734417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3f4154c00 msgr2=0x7fe3f4155070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:12.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.254+0000 7fe3fb9a9700 1 --2- 172.21.15.89:0/1054734417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3f4154c00 0x7fe3f4155070 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40073f0 tx=0x7fe3e4001df0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.254+0000 7fe3fb9a9700 1 -- 172.21.15.89:0/1054734417 shutdown_connections 2023-10-10T20:41:12.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.254+0000 7fe3fb9a9700 1 --2- 172.21.15.89:0/1054734417 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe3f414cd70 0x7fe3f414d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.254+0000 7fe3fb9a9700 1 --2- 172.21.15.89:0/1054734417 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe3d8067320 0x7fe3d80697d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.254+0000 7fe3fb9a9700 1 --2- 172.21.15.89:0/1054734417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3e0003c20 0x7fe3f414b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.254+0000 7fe3fb9a9700 1 --2- 172.21.15.89:0/1054734417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3f4154c00 0x7fe3f4155070 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.254+0000 7fe3fb9a9700 1 -- 172.21.15.89:0/1054734417 >> 172.21.15.89:0/1054734417 conn(0x7fe3f406a320 msgr2=0x7fe3f4152c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:12.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.254+0000 7fe3fb9a9700 1 -- 172.21.15.89:0/1054734417 shutdown_connections 2023-10-10T20:41:12.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.254+0000 7fe3fb9a9700 1 -- 172.21.15.89:0/1054734417 wait complete. 2023-10-10T20:41:12.262 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:41:12.268 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2023-10-10T20:41:10.731060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422956,"kb_used_data":11884,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752551764,"statfs":{"total":773094113280,"available":770613006336,"internally_reserved":0,"allocated":12169216,"data_stored":5911081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.561490"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.185044+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:43:38.336996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179076+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:42:10.326290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183633+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:31:04.342810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184156+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:29:07.750059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183700+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:06:14.543497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180424+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:15:09.117651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180699+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:43:54.668141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.185584+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:51.630860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182871+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:00.815725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178630+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:45:17.413763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.176188+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:28:49.571035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182500+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:27:02.185210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184448+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:23.971915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.177799+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:35.431888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183897+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:08.100006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182045+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:51.136287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181388+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:51:14.573623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182606+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:21:00.592871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.175771+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:29:15.059073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182998+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:35:58.896847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182721+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:57.837359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.175675+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:54:47.622590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184138+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:56.592906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.175733+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:04.315349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178752+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:39:12.818520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181110+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:36:04.128348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183202+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:24.718214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179120+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:57:20.319921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.185431+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:38:00.081732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183275+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:09.031752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.186844+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:54.943577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179704+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:51.630860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180587+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:06:14.543497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194387+0000","last_change":"2023-10-10T20:41:07.194387+0000","last_active":"2023-10-10T20:41:07.194387+0000","last_peered":"2023-10-10T20:41:07.194387+0000","last_clean":"2023-10-10T20:41:07.194387+0000","last_became_active":"2023-10-10T20:41:07.194097+0000","last_became_peered":"2023-10-10T20:41:07.194097+0000","last_unstale":"2023-10-10T20:41:07.194387+0000","last_undegraded":"2023-10-10T20:41:07.194387+0000","last_fullsized":"2023-10-10T20:41:07.194387+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:26.546168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182362+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:37:12.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192372+0000","last_change":"2023-10-10T20:41:07.192372+0000","last_active":"2023-10-10T20:41:07.192372+0000","last_peered":"2023-10-10T20:41:07.192372+0000","last_clean":"2023-10-10T20:41:07.192372+0000","last_became_active":"2023-10-10T20:41:07.192088+0000","last_became_peered":"2023-10-10T20:41:07.192088+0000","last_unstale":"2023-10-10T20:41:07.192372+0000","last_undegraded":"2023-10-10T20:41:07.192372+0000","last_fullsized":"2023-10-10T20:41:07.192372+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:59.684638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181983+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:01.392782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.187579+0000","last_change":"2023-10-10T20:41:07.187579+0000","last_active":"2023-10-10T20:41:07.187579+0000","last_peered":"2023-10-10T20:41:07.187579+0000","last_clean":"2023-10-10T20:41:07.187579+0000","last_became_active":"2023-10-10T20:41:07.187296+0000","last_became_peered":"2023-10-10T20:41:07.187296+0000","last_unstale":"2023-10-10T20:41:07.187579+0000","last_undegraded":"2023-10-10T20:41:07.187579+0000","last_fullsized":"2023-10-10T20:41:07.187579+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:18.852267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182717+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:56:26.100041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194195+0000","last_change":"2023-10-10T20:41:07.194195+0000","last_active":"2023-10-10T20:41:07.194195+0000","last_peered":"2023-10-10T20:41:07.194195+0000","last_clean":"2023-10-10T20:41:07.194195+0000","last_became_active":"2023-10-10T20:41:07.193920+0000","last_became_peered":"2023-10-10T20:41:07.193920+0000","last_unstale":"2023-10-10T20:41:07.194195+0000","last_undegraded":"2023-10-10T20:41:07.194195+0000","last_fullsized":"2023-10-10T20:41:07.194195+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:36.588822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182202+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:40:14.372905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.193266+0000","last_change":"2023-10-10T20:41:07.193266+0000","last_active":"2023-10-10T20:41:07.193266+0000","last_peered":"2023-10-10T20:41:07.193266+0000","last_clean":"2023-10-10T20:41:07.193266+0000","last_became_active":"2023-10-10T20:41:07.192991+0000","last_became_peered":"2023-10-10T20:41:07.192991+0000","last_unstale":"2023-10-10T20:41:07.193266+0000","last_undegraded":"2023-10-10T20:41:07.193266+0000","last_fullsized":"2023-10-10T20:41:07.193266+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:15.545053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182281+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:51:25.089205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.193695+0000","last_change":"2023-10-10T20:41:07.193695+0000","last_active":"2023-10-10T20:41:07.193695+0000","last_peered":"2023-10-10T20:41:07.193695+0000","last_clean":"2023-10-10T20:41:07.193695+0000","last_became_active":"2023-10-10T20:41:07.193394+0000","last_became_peered":"2023-10-10T20:41:07.193394+0000","last_unstale":"2023-10-10T20:41:07.193695+0000","last_undegraded":"2023-10-10T20:41:07.193695+0000","last_fullsized":"2023-10-10T20:41:07.193695+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:56.835912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.186003+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:44.120514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.193880+0000","last_change":"2023-10-10T20:41:07.193880+0000","last_active":"2023-10-10T20:41:07.193880+0000","last_peered":"2023-10-10T20:41:07.193880+0000","last_clean":"2023-10-10T20:41:07.193880+0000","last_became_active":"2023-10-10T20:41:07.193586+0000","last_became_peered":"2023-10-10T20:41:07.193586+0000","last_unstale":"2023-10-10T20:41:07.193880+0000","last_undegraded":"2023-10-10T20:41:07.193880+0000","last_fullsized":"2023-10-10T20:41:07.193880+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:15:36.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178163+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:13:00.611054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194069+0000","last_change":"2023-10-10T20:41:07.194069+0000","last_active":"2023-10-10T20:41:07.194069+0000","last_peered":"2023-10-10T20:41:07.194069+0000","last_clean":"2023-10-10T20:41:07.194069+0000","last_became_active":"2023-10-10T20:41:07.193800+0000","last_became_peered":"2023-10-10T20:41:07.193800+0000","last_unstale":"2023-10-10T20:41:07.194069+0000","last_undegraded":"2023-10-10T20:41:07.194069+0000","last_fullsized":"2023-10-10T20:41:07.194069+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:00:42.940622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183477+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:59:58.659044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.190099+0000","last_change":"2023-10-10T20:41:07.190099+0000","last_active":"2023-10-10T20:41:07.190099+0000","last_peered":"2023-10-10T20:41:07.190099+0000","last_clean":"2023-10-10T20:41:07.190099+0000","last_became_active":"2023-10-10T20:41:07.189840+0000","last_became_peered":"2023-10-10T20:41:07.189840+0000","last_unstale":"2023-10-10T20:41:07.190099+0000","last_undegraded":"2023-10-10T20:41:07.190099+0000","last_fullsized":"2023-10-10T20:41:07.190099+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:48:33.094041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178595+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:35.431888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.187415+0000","last_change":"2023-10-10T20:41:07.187415+0000","last_active":"2023-10-10T20:41:07.187415+0000","last_peered":"2023-10-10T20:41:07.187415+0000","last_clean":"2023-10-10T20:41:07.187415+0000","last_became_active":"2023-10-10T20:41:07.187114+0000","last_became_peered":"2023-10-10T20:41:07.187114+0000","last_unstale":"2023-10-10T20:41:07.187415+0000","last_undegraded":"2023-10-10T20:41:07.187415+0000","last_fullsized":"2023-10-10T20:41:07.187415+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:58:20.063048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181497+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:42:34.466499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.188746+0000","last_change":"2023-10-10T20:41:07.188746+0000","last_active":"2023-10-10T20:41:07.188746+0000","last_peered":"2023-10-10T20:41:07.188746+0000","last_clean":"2023-10-10T20:41:07.188746+0000","last_became_active":"2023-10-10T20:41:07.188486+0000","last_became_peered":"2023-10-10T20:41:07.188486+0000","last_unstale":"2023-10-10T20:41:07.188746+0000","last_undegraded":"2023-10-10T20:41:07.188746+0000","last_fullsized":"2023-10-10T20:41:07.188746+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:36.588822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182276+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:04:10.686499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.191180+0000","last_change":"2023-10-10T20:41:07.191180+0000","last_active":"2023-10-10T20:41:07.191180+0000","last_peered":"2023-10-10T20:41:07.191180+0000","last_clean":"2023-10-10T20:41:07.191180+0000","last_became_active":"2023-10-10T20:41:07.190910+0000","last_became_peered":"2023-10-10T20:41:07.190910+0000","last_unstale":"2023-10-10T20:41:07.191180+0000","last_undegraded":"2023-10-10T20:41:07.191180+0000","last_fullsized":"2023-10-10T20:41:07.191180+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:13.928318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184946+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:43.974985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.189943+0000","last_change":"2023-10-10T20:41:07.189943+0000","last_active":"2023-10-10T20:41:07.189943+0000","last_peered":"2023-10-10T20:41:07.189943+0000","last_clean":"2023-10-10T20:41:07.189943+0000","last_became_active":"2023-10-10T20:41:07.189639+0000","last_became_peered":"2023-10-10T20:41:07.189639+0000","last_unstale":"2023-10-10T20:41:07.189943+0000","last_undegraded":"2023-10-10T20:41:07.189943+0000","last_fullsized":"2023-10-10T20:41:07.189943+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:24:45.124009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178350+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:07:36.012741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.193331+0000","last_change":"2023-10-10T20:41:07.193331+0000","last_active":"2023-10-10T20:41:07.193331+0000","last_peered":"2023-10-10T20:41:07.193331+0000","last_clean":"2023-10-10T20:41:07.193331+0000","last_became_active":"2023-10-10T20:41:07.193077+0000","last_became_peered":"2023-10-10T20:41:07.193077+0000","last_unstale":"2023-10-10T20:41:07.193331+0000","last_undegraded":"2023-10-10T20:41:07.193331+0000","last_fullsized":"2023-10-10T20:41:07.193331+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:56.816260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182401+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:47:38.801771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.190581+0000","last_change":"2023-10-10T20:41:07.190581+0000","last_active":"2023-10-10T20:41:07.190581+0000","last_peered":"2023-10-10T20:41:07.190581+0000","last_clean":"2023-10-10T20:41:07.190581+0000","last_became_active":"2023-10-10T20:41:07.190319+0000","last_became_peered":"2023-10-10T20:41:07.190319+0000","last_unstale":"2023-10-10T20:41:07.190581+0000","last_undegraded":"2023-10-10T20:41:07.190581+0000","last_fullsized":"2023-10-10T20:41:07.190581+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:42:09.324842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183200+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:15:37.860366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.191241+0000","last_change":"2023-10-10T20:41:07.191241+0000","last_active":"2023-10-10T20:41:07.191241+0000","last_peered":"2023-10-10T20:41:07.191241+0000","last_clean":"2023-10-10T20:41:07.191241+0000","last_became_active":"2023-10-10T20:41:07.191020+0000","last_became_peered":"2023-10-10T20:41:07.191020+0000","last_unstale":"2023-10-10T20:41:07.191241+0000","last_undegraded":"2023-10-10T20:41:07.191241+0000","last_fullsized":"2023-10-10T20:41:07.191241+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:56.816260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180046+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:07:36.012741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.187395+0000","last_change":"2023-10-10T20:41:07.187395+0000","last_active":"2023-10-10T20:41:07.187395+0000","last_peered":"2023-10-10T20:41:07.187395+0000","last_clean":"2023-10-10T20:41:07.187395+0000","last_became_active":"2023-10-10T20:41:07.187130+0000","last_became_peered":"2023-10-10T20:41:07.187130+0000","last_unstale":"2023-10-10T20:41:07.187395+0000","last_undegraded":"2023-10-10T20:41:07.187395+0000","last_fullsized":"2023-10-10T20:41:07.187395+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:32:03.550923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181825+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:54.943577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.190127+0000","last_change":"2023-10-10T20:41:07.190127+0000","last_active":"2023-10-10T20:41:07.190127+0000","last_peered":"2023-10-10T20:41:07.190127+0000","last_clean":"2023-10-10T20:41:07.190127+0000","last_became_active":"2023-10-10T20:41:07.189866+0000","last_became_peered":"2023-10-10T20:41:07.189866+0000","last_unstale":"2023-10-10T20:41:07.190127+0000","last_undegraded":"2023-10-10T20:41:07.190127+0000","last_fullsized":"2023-10-10T20:41:07.190127+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:31:03.341362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.177885+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:57:20.319921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192468+0000","last_change":"2023-10-10T20:41:07.192468+0000","last_active":"2023-10-10T20:41:07.192468+0000","last_peered":"2023-10-10T20:41:07.192468+0000","last_clean":"2023-10-10T20:41:07.192468+0000","last_became_active":"2023-10-10T20:41:07.192177+0000","last_became_peered":"2023-10-10T20:41:07.192177+0000","last_unstale":"2023-10-10T20:41:07.192468+0000","last_undegraded":"2023-10-10T20:41:07.192468+0000","last_fullsized":"2023-10-10T20:41:07.192468+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:15.545053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183818+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:53.393618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.188588+0000","last_change":"2023-10-10T20:41:07.188588+0000","last_active":"2023-10-10T20:41:07.188588+0000","last_peered":"2023-10-10T20:41:07.188588+0000","last_clean":"2023-10-10T20:41:07.188588+0000","last_became_active":"2023-10-10T20:41:07.188209+0000","last_became_peered":"2023-10-10T20:41:07.188209+0000","last_unstale":"2023-10-10T20:41:07.188588+0000","last_undegraded":"2023-10-10T20:41:07.188588+0000","last_fullsized":"2023-10-10T20:41:07.188588+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:00:42.940622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180348+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:44.120514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194274+0000","last_change":"2023-10-10T20:41:07.194274+0000","last_active":"2023-10-10T20:41:07.194274+0000","last_peered":"2023-10-10T20:41:07.194274+0000","last_clean":"2023-10-10T20:41:07.194274+0000","last_became_active":"2023-10-10T20:41:07.193987+0000","last_became_peered":"2023-10-10T20:41:07.193987+0000","last_unstale":"2023-10-10T20:41:07.194274+0000","last_undegraded":"2023-10-10T20:41:07.194274+0000","last_fullsized":"2023-10-10T20:41:07.194274+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:17.174004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179361+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:14.929765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194646+0000","last_change":"2023-10-10T20:41:07.194646+0000","last_active":"2023-10-10T20:41:07.194646+0000","last_peered":"2023-10-10T20:41:07.194646+0000","last_clean":"2023-10-10T20:41:07.194646+0000","last_became_active":"2023-10-10T20:41:07.194426+0000","last_became_peered":"2023-10-10T20:41:07.194426+0000","last_unstale":"2023-10-10T20:41:07.194646+0000","last_undegraded":"2023-10-10T20:41:07.194646+0000","last_fullsized":"2023-10-10T20:41:07.194646+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:00.391334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181308+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:47:27.526950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192740+0000","last_change":"2023-10-10T20:41:07.192740+0000","last_active":"2023-10-10T20:41:07.192740+0000","last_peered":"2023-10-10T20:41:07.192740+0000","last_clean":"2023-10-10T20:41:07.192740+0000","last_became_active":"2023-10-10T20:41:07.192480+0000","last_became_peered":"2023-10-10T20:41:07.192480+0000","last_unstale":"2023-10-10T20:41:07.192740+0000","last_undegraded":"2023-10-10T20:41:07.192740+0000","last_fullsized":"2023-10-10T20:41:07.192740+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:48.161107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181783+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:27.547616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194467+0000","last_change":"2023-10-10T20:41:07.194467+0000","last_active":"2023-10-10T20:41:07.194467+0000","last_peered":"2023-10-10T20:41:07.194467+0000","last_clean":"2023-10-10T20:41:07.194467+0000","last_became_active":"2023-10-10T20:41:07.193987+0000","last_became_peered":"2023-10-10T20:41:07.193987+0000","last_unstale":"2023-10-10T20:41:07.194467+0000","last_undegraded":"2023-10-10T20:41:07.194467+0000","last_fullsized":"2023-10-10T20:41:07.194467+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:43.862933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184078+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:38:30.611281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.196552+0000","last_change":"2023-10-10T20:41:07.196552+0000","last_active":"2023-10-10T20:41:07.196552+0000","last_peered":"2023-10-10T20:41:07.196552+0000","last_clean":"2023-10-10T20:41:07.196552+0000","last_became_active":"2023-10-10T20:41:07.196285+0000","last_became_peered":"2023-10-10T20:41:07.196285+0000","last_unstale":"2023-10-10T20:41:07.196552+0000","last_undegraded":"2023-10-10T20:41:07.196552+0000","last_fullsized":"2023-10-10T20:41:07.196552+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:43:53.666694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178501+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:43.761996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192768+0000","last_change":"2023-10-10T20:41:07.192768+0000","last_active":"2023-10-10T20:41:07.192768+0000","last_peered":"2023-10-10T20:41:07.192768+0000","last_clean":"2023-10-10T20:41:07.192768+0000","last_became_active":"2023-10-10T20:41:07.192467+0000","last_became_peered":"2023-10-10T20:41:07.192467+0000","last_unstale":"2023-10-10T20:41:07.192768+0000","last_undegraded":"2023-10-10T20:41:07.192768+0000","last_fullsized":"2023-10-10T20:41:07.192768+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:59.684638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182731+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:47:27.526950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.186703+0000","last_change":"2023-10-10T20:41:07.186703+0000","last_active":"2023-10-10T20:41:07.186703+0000","last_peered":"2023-10-10T20:41:07.186703+0000","last_clean":"2023-10-10T20:41:07.186703+0000","last_became_active":"2023-10-10T20:41:07.186447+0000","last_became_peered":"2023-10-10T20:41:07.186447+0000","last_unstale":"2023-10-10T20:41:07.186703+0000","last_undegraded":"2023-10-10T20:41:07.186703+0000","last_fullsized":"2023-10-10T20:41:07.186703+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:57.197462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181353+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:58:21.064495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192864+0000","last_change":"2023-10-10T20:41:07.192864+0000","last_active":"2023-10-10T20:41:07.192864+0000","last_peered":"2023-10-10T20:41:07.192864+0000","last_clean":"2023-10-10T20:41:07.192864+0000","last_became_active":"2023-10-10T20:41:07.192602+0000","last_became_peered":"2023-10-10T20:41:07.192602+0000","last_unstale":"2023-10-10T20:41:07.192864+0000","last_undegraded":"2023-10-10T20:41:07.192864+0000","last_fullsized":"2023-10-10T20:41:07.192864+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:37:19.121105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179266+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:13:00.611054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192698+0000","last_change":"2023-10-10T20:41:07.192698+0000","last_active":"2023-10-10T20:41:07.192698+0000","last_peered":"2023-10-10T20:41:07.192698+0000","last_clean":"2023-10-10T20:41:07.192698+0000","last_became_active":"2023-10-10T20:41:07.192370+0000","last_became_peered":"2023-10-10T20:41:07.192370+0000","last_unstale":"2023-10-10T20:41:07.192698+0000","last_undegraded":"2023-10-10T20:41:07.192698+0000","last_fullsized":"2023-10-10T20:41:07.192698+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:15:08.116204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180520+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:56:26.100041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194285+0000","last_change":"2023-10-10T20:41:07.194285+0000","last_active":"2023-10-10T20:41:07.194285+0000","last_peered":"2023-10-10T20:41:07.194285+0000","last_clean":"2023-10-10T20:41:07.194285+0000","last_became_active":"2023-10-10T20:41:07.194002+0000","last_became_peered":"2023-10-10T20:41:07.194002+0000","last_unstale":"2023-10-10T20:41:07.194285+0000","last_undegraded":"2023-10-10T20:41:07.194285+0000","last_fullsized":"2023-10-10T20:41:07.194285+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:42:33.465052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179142+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:56:52.494512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.186005+0000","last_change":"2023-10-10T20:41:07.186005+0000","last_active":"2023-10-10T20:41:07.186005+0000","last_peered":"2023-10-10T20:41:07.186005+0000","last_clean":"2023-10-10T20:41:07.186005+0000","last_became_active":"2023-10-10T20:41:07.185723+0000","last_became_peered":"2023-10-10T20:41:07.185723+0000","last_unstale":"2023-10-10T20:41:07.186005+0000","last_undegraded":"2023-10-10T20:41:07.186005+0000","last_fullsized":"2023-10-10T20:41:07.186005+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:39.932733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181885+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:18.175451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.195197+0000","last_change":"2023-10-10T20:41:07.195197+0000","last_active":"2023-10-10T20:41:07.195197+0000","last_peered":"2023-10-10T20:41:07.195197+0000","last_clean":"2023-10-10T20:41:07.195197+0000","last_became_active":"2023-10-10T20:41:07.194893+0000","last_became_peered":"2023-10-10T20:41:07.194893+0000","last_unstale":"2023-10-10T20:41:07.195197+0000","last_undegraded":"2023-10-10T20:41:07.195197+0000","last_fullsized":"2023-10-10T20:41:07.195197+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:51:13.572175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":253,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-10-10T20:41:06.176359+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:41:06.176359+0000","last_peered":"2023-10-10T20:41:06.176359+0000","last_clean":"2023-10-10T20:41:06.176359+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:41:06.176359+0000","last_undegraded":"2023-10-10T20:41:06.176359+0000","last_fullsized":"2023-10-10T20:41:06.176359+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303308,"kb_used_data":1924,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068532,"statfs":{"total":96636764160,"available":96326176768,"internally_reserved":0,"allocated":1970176,"data_stored":1183347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51500000000000001,"15min":0.81999999999999995},"min":{"1min":0.31,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53000000000000003,"15min":0.627},"min":{"1min":0.32200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71999999999999997,"5min":0.95399999999999996,"15min":4.1959999999999997},"last":0.54000000000000004}]},{"osd":1,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.504,"15min":1},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.67800000000000005,"5min":0.78800000000000003,"15min":10.1},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55800000000000005,"15min":0.72999999999999998},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.28999999999999998},"max":{"1min":0.81799999999999995,"5min":1.006,"15min":6.806},"last":0.65300000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57899999999999996,"15min":0.78100000000000003},"min":{"1min":0.43099999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89000000000000001,"5min":1.089,"15min":6.9340000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53200000000000003,"15min":0.73799999999999999},"min":{"1min":0.39000000000000001,"5min":0.36399999999999999,"15min":0.30099999999999999},"max":{"1min":0.77100000000000002,"5min":0.85299999999999998,"15min":6.9870000000000001},"last":0.46899999999999997}]},{"osd":3,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.75800000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.754,"5min":0.80200000000000005,"15min":6.6719999999999997},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.67000000000000004},"min":{"1min":0.38700000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.70599999999999996,"5min":1.2789999999999999,"15min":4.1609999999999996},"last":0.60099999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47399999999999998,"15min":0.46500000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.73199999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.621},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52400000000000002,"15min":0.52300000000000002},"min":{"1min":0.309,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.55200000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.32500000000000001,"5min":0.32100000000000001,"15min":0.314},"max":{"1min":0.86099999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75800000000000001,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.68899999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72999999999999998,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84099999999999997,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.66500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302728,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069112,"statfs":{"total":96636764160,"available":96326770688,"internally_reserved":0,"allocated":1376256,"data_stored":592979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50700000000000001,"15min":0.67200000000000004},"min":{"1min":0.33500000000000002,"5min":0.32400000000000001,"15min":0.312},"max":{"1min":0.68600000000000005,"5min":0.73099999999999998,"15min":7.7590000000000003},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52100000000000002,"15min":0.71299999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":7.8090000000000002},"last":0.52400000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.70999999999999996},"min":{"1min":0.38400000000000001,"5min":0.35599999999999998,"15min":0.27800000000000002},"max":{"1min":0.628,"5min":0.80000000000000004,"15min":7.867},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53300000000000003,"15min":0.88500000000000001},"min":{"1min":0.36599999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":9.7089999999999996},"last":0.54400000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51400000000000001,"15min":0.59599999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.72499999999999998,"5min":0.747,"15min":4.7999999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54600000000000004,"15min":0.84499999999999997},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.71499999999999997,"5min":0.80000000000000004,"15min":6.7629999999999999},"last":0.63}]},{"osd":3,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.62},"min":{"1min":0.40999999999999998,"5min":0.36299999999999999,"15min":0.35299999999999998},"max":{"1min":0.79100000000000004,"5min":0.83999999999999997,"15min":4.8620000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54800000000000004,"15min":0.71699999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.35299999999999998},"max":{"1min":0.68600000000000005,"5min":1.1850000000000001,"15min":7.9009999999999998},"last":0.64900000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51500000000000001,"15min":0.46800000000000003},"min":{"1min":0.32400000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71099999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59199999999999997,"15min":0.56799999999999995},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.37},"max":{"1min":0.73499999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.49099999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52000000000000002,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.314,"15min":0.312},"max":{"1min":0.69199999999999995,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.747,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.433}]},{"osd":7,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55100000000000005,"15min":0.52000000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.72599999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57799999999999996,"15min":0.54400000000000004},"min":{"1min":0.42899999999999999,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":0.69399999999999995,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.47499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302716,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069124,"statfs":{"total":96636764160,"available":96326782976,"internally_reserved":0,"allocated":1363968,"data_stored":586716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.31900000000000001,"15min":0.47599999999999998},"min":{"1min":0.20999999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.441,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.32300000000000001,"15min":0.36299999999999999},"min":{"1min":0.23599999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46000000000000002,"5min":0.68700000000000006,"15min":1.194},"last":0.308}]},{"osd":2,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.32400000000000001,"15min":0.36399999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.379,"5min":0.67400000000000004,"15min":1.2869999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.33800000000000002,"15min":0.505},"min":{"1min":0.26000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.54200000000000004,"5min":0.57199999999999995,"15min":5.0919999999999996},"last":0.51300000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36299999999999999,"15min":0.52800000000000002},"min":{"1min":0.26800000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.51000000000000001,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36499999999999999,"15min":0.40699999999999997},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.48299999999999998,"5min":0.72099999999999997,"15min":1.2170000000000001},"last":0.41399999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57899999999999996,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.81000000000000005,"5min":1.133,"15min":1.254},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56599999999999995,"15min":0.70599999999999996},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69899999999999995,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.67100000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57899999999999996,"15min":0.59499999999999997},"min":{"1min":0.42299999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.751,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.499},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.60799999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71899999999999997,"5min":1.0720000000000001,"15min":1.3109999999999999},"last":0.55700000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60699999999999998,"15min":0.84399999999999997},"min":{"1min":0.435,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86199999999999999,"5min":0.995,"15min":5.0659999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61699999999999999,"15min":0.63200000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84599999999999997,"5min":1.1439999999999999,"15min":1.171},"last":0.58899999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59299999999999997,"15min":0.61699999999999999},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.753,"5min":1.214,"15min":1.214},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60099999999999998,"15min":0.72499999999999998},"min":{"1min":0.45000000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76900000000000002,"5min":1.115,"15min":4.0499999999999998},"last":0.625}]}]},{"osd":0,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303304,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068536,"statfs":{"total":96636764160,"available":96326180864,"internally_reserved":0,"allocated":1966080,"data_stored":1182386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.309,"15min":0.52800000000000002},"min":{"1min":0.20999999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.38300000000000001,"5min":0.51600000000000001,"15min":7.9249999999999998},"last":0.371},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.318,"15min":0.505},"min":{"1min":0.23699999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.41899999999999998,"5min":0.54900000000000004,"15min":6.6299999999999999},"last":0.255}]},{"osd":2,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.32600000000000001,"15min":0.55200000000000005},"min":{"1min":0.22,"5min":0.216,"15min":0.216},"max":{"1min":0.48799999999999999,"5min":0.55400000000000005,"15min":8.2509999999999994},"last":0.313},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.34000000000000002,"15min":0.52800000000000002},"min":{"1min":0.23100000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.45700000000000002,"5min":0.54100000000000004,"15min":6.7370000000000001},"last":0.32800000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.34399999999999997,"15min":0.55700000000000005},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.48299999999999998,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.61199999999999999},"min":{"1min":0.32000000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":7.8929999999999998},"last":0.41599999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55600000000000005,"15min":0.71399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.72499999999999998,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.748},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56299999999999994,"15min":0.77200000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.67600000000000005,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.71099999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.55200000000000005,"15min":0.751},"min":{"1min":0.38800000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69299999999999995,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58099999999999996,"15min":0.77800000000000002},"min":{"1min":0.42299999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77000000000000002,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.75700000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60199999999999998,"15min":0.81799999999999995},"min":{"1min":0.39800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72099999999999997,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61099999999999999,"15min":0.76100000000000001},"min":{"1min":0.41399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74299999999999999,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58799999999999997,"15min":0.752},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78300000000000003,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59099999999999997,"15min":0.78600000000000003},"min":{"1min":0.45200000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.69299999999999995,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.60299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302728,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069112,"statfs":{"total":96636764160,"available":96326770688,"internally_reserved":0,"allocated":1376256,"data_stored":592979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.28299999999999997,"5min":0.309,"15min":0.32500000000000001},"min":{"1min":0.21099999999999999,"5min":0.189,"15min":0.189},"max":{"1min":0.38300000000000001,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.29399999999999998,"5min":0.32100000000000001,"15min":0.35299999999999998},"min":{"1min":0.215,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.39500000000000002,"5min":0.5,"15min":0.746},"last":0.29799999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.32900000000000001,"15min":0.72899999999999998},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.35699999999999998,"5min":0.63,"15min":11.967000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.34499999999999997,"15min":0.48299999999999998},"min":{"1min":0.26800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.443,"5min":0.53400000000000003,"15min":4.1100000000000003},"last":0.36499999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.34499999999999997,"15min":0.55700000000000005},"min":{"1min":0.23699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.39400000000000002,"5min":0.55400000000000005,"15min":6.3040000000000003},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38700000000000001,"15min":0.78100000000000003},"min":{"1min":0.27900000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.46899999999999997,"5min":0.65900000000000003,"15min":11.935},"last":0.35199999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.54300000000000004,"15min":0.55500000000000005},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.58699999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71699999999999997,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.58599999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.378},"max":{"1min":0.66300000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54400000000000004,"15min":0.58399999999999996},"min":{"1min":0.39700000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74199999999999999,"5min":0.874,"15min":0.96099999999999997},"last":0.54100000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57699999999999996,"15min":0.60099999999999998},"min":{"1min":0.42199999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.745,"5min":0.91600000000000004,"15min":0.91800000000000004},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73799999999999999,"5min":0.93100000000000005,"15min":0.93600000000000005},"last":0.55500000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56499999999999995,"15min":0.58499999999999996},"min":{"1min":0.41799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71099999999999997,"5min":0.89100000000000001,"15min":1.052},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57699999999999996,"15min":0.78000000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69799999999999995,"5min":0.85999999999999999,"15min":6.2649999999999997},"last":0.45900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302728,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069112,"statfs":{"total":96636764160,"available":96326770688,"internally_reserved":0,"allocated":1376256,"data_stored":592979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.31,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.31900000000000001},"min":{"1min":0.25,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.33600000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.23499999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.316},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38600000000000001,"15min":0.377},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.57099999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.40500000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.34999999999999998},"min":{"1min":0.23699999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.35699999999999998,"15min":0.36199999999999999},"min":{"1min":0.26300000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.35899999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69999999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.82299999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55300000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.35899999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55900000000000005,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.76900000000000002,"5min":1.01,"15min":1.01},"last":0.626}]},{"osd":6,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.42099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85699999999999998,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.60999999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.441,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.496,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.88400000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302716,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069124,"statfs":{"total":96636764160,"available":96326782976,"internally_reserved":0,"allocated":1363968,"data_stored":586716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68300000000000005,"5min":0.74199999999999999,"15min":0.747},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.76600000000000001},"min":{"1min":0.374,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67500000000000004,"5min":0.78400000000000003,"15min":7.6719999999999997},"last":0.439}]},{"osd":1,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52900000000000003,"15min":0.77400000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.755,"5min":0.755,"15min":7.7160000000000002},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65300000000000002,"5min":0.76900000000000002,"15min":0.86799999999999999},"last":0.66100000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.65300000000000002,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56299999999999994,"15min":0.79300000000000004},"min":{"1min":0.39700000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.72799999999999998,"5min":0.79800000000000004,"15min":7.7839999999999998},"last":0.68500000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55200000000000005,"15min":0.54900000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65500000000000003,"5min":0.73399999999999999,"15min":0.96999999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.41199999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.66100000000000003,"5min":0.78100000000000003,"15min":0.79400000000000004},"last":0.47099999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51000000000000001,"15min":0.48699999999999999},"min":{"1min":0.32500000000000001,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.77500000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54900000000000004,"15min":0.54300000000000004},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.55300000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.39200000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68799999999999994,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.35899999999999999},"max":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.378,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.84599999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.46100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.748,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.60899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302728,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069112,"statfs":{"total":96636764160,"available":96326770688,"internally_reserved":0,"allocated":1376256,"data_stored":592979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.28599999999999998,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.79300000000000004},"last":0.52400000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52700000000000002,"15min":0.64100000000000001},"min":{"1min":0.33000000000000002,"5min":0.32200000000000001,"15min":0.316},"max":{"1min":0.70399999999999996,"5min":1.2330000000000001,"15min":5.2679999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51800000000000002,"15min":0.629},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.47099999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52800000000000002,"15min":0.64800000000000002},"min":{"1min":0.35999999999999999,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.76600000000000001,"5min":0.86199999999999999,"15min":5.2960000000000003},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72899999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.45000000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55300000000000005,"15min":0.67700000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.73899999999999999,"5min":0.93000000000000005,"15min":5.3330000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.69099999999999995},"min":{"1min":0.39300000000000002,"5min":0.36899999999999999,"15min":0.27700000000000002},"max":{"1min":0.76000000000000001,"5min":0.94399999999999995,"15min":5.2850000000000001},"last":0.53900000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47799999999999998,"15min":0.496},"min":{"1min":0.311,"5min":0.29599999999999999,"15min":0.28699999999999998},"max":{"1min":0.72499999999999998,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34300000000000003,"15min":0.32200000000000001},"max":{"1min":0.70799999999999996,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.63800000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54100000000000004,"15min":0.54700000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.80100000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82499999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.624}]},{"osd":7,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.378,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68899999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60199999999999998,"15min":0.60099999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83399999999999996,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.61399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:41:12.274 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:41:12.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.420+0000 7fecc7018700 1 Processor -- start 2023-10-10T20:41:12.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.420+0000 7fecc7018700 1 -- start start 2023-10-10T20:41:12.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.421+0000 7fecc7018700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fecc0159c60 0x7fecc015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.421+0000 7fecc7018700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc015a160 con 0x7fecc0159890 2023-10-10T20:41:12.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.421+0000 7fecc7018700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc00d13a0 con 0x7fecc015db10 2023-10-10T20:41:12.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.421+0000 7fecc7018700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc015df50 con 0x7fecc0159c60 2023-10-10T20:41:12.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.421+0000 7fecc4db4700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fecc0159890 0x7fecc015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41348/0 (socket says 172.21.15.89:41348) 2023-10-10T20:41:12.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.421+0000 7fecc4db4700 1 -- 172.21.15.89:0/2144091629 learned_addr learned my addr 172.21.15.89:0/2144091629 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:12.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.421+0000 7fecb77fe700 1 -- 172.21.15.89:0/2144091629 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1305931279 0 0) 0x7fecc015a160 con 0x7fecc0159890 2023-10-10T20:41:12.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.421+0000 7fecb77fe700 1 -- 172.21.15.89:0/2144091629 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feca8003570 con 0x7fecc0159890 2023-10-10T20:41:12.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.421+0000 7fecb7fff700 1 --2- 172.21.15.89:0/2144091629 >> v2:172.21.15.111:3300/0 conn(0x7fecc0159c60 0x7fecc015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.421+0000 7fecb77fe700 1 -- 172.21.15.89:0/2144091629 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 642168460 0 0) 0x7feca8003570 con 0x7fecc0159890 2023-10-10T20:41:12.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.421+0000 7fecb77fe700 1 -- 172.21.15.89:0/2144091629 >> v2:172.21.15.111:3300/0 conn(0x7fecc0159c60 msgr2=0x7fecc015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:12.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.421+0000 7fecb77fe700 1 --2- 172.21.15.89:0/2144091629 >> v2:172.21.15.111:3300/0 conn(0x7fecc0159c60 0x7fecc015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.421+0000 7fecb77fe700 1 -- 172.21.15.89:0/2144091629 >> v1:172.21.15.111:6789/0 conn(0x7fecc015db10 legacy=0x7fecc0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:12.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.421+0000 7fecb77fe700 1 -- 172.21.15.89:0/2144091629 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc015e140 con 0x7fecc0159890 2023-10-10T20:41:12.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.421+0000 7fecb77fe700 1 -- 172.21.15.89:0/2144091629 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fecbc0017f0 con 0x7fecc0159890 2023-10-10T20:41:12.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecb77fe700 1 -- 172.21.15.89:0/2144091629 >> v1:172.21.15.89:6789/0 conn(0x7fecc0159890 legacy=0x7fecc015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:12.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecb77fe700 1 --2- 172.21.15.89:0/2144091629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feca8003a60 0x7feca8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecb77fe700 1 --2- 172.21.15.89:0/2144091629 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feca8004810 0x7feca8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecb77fe700 1 --2- 172.21.15.89:0/2144091629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feca80051c0 0x7feca800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecb77fe700 1 -- 172.21.15.89:0/2144091629 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca800c0a0 con 0x7feca80051c0 2023-10-10T20:41:12.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecb77fe700 1 -- 172.21.15.89:0/2144091629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca800c210 con 0x7feca8003a60 2023-10-10T20:41:12.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecb77fe700 1 -- 172.21.15.89:0/2144091629 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca800c3d0 con 0x7feca8004810 2023-10-10T20:41:12.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecc55b5700 1 --2- 172.21.15.89:0/2144091629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feca80051c0 0x7feca800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecc4db4700 1 --2- 172.21.15.89:0/2144091629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feca8003a60 0x7feca8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecb7fff700 1 --2- 172.21.15.89:0/2144091629 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feca8004810 0x7feca8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecc55b5700 1 -- 172.21.15.89:0/2144091629 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feca8004810 msgr2=0x7feca8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:12.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecc55b5700 1 --2- 172.21.15.89:0/2144091629 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feca8004810 0x7feca8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecc55b5700 1 -- 172.21.15.89:0/2144091629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feca8003a60 msgr2=0x7feca8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:12.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecc55b5700 1 --2- 172.21.15.89:0/2144091629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feca8003a60 0x7feca8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecc55b5700 1 -- 172.21.15.89:0/2144091629 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca8003570 con 0x7feca80051c0 2023-10-10T20:41:12.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecc55b5700 1 -- 172.21.15.89:0/2144091629 reap_dead start 2023-10-10T20:41:12.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecc55b5700 1 --2- 172.21.15.89:0/2144091629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feca80051c0 0x7feca800ba60 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fecb80018b0 tx=0x7fecb8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74be96aa46328cc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:12.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecb77fe700 1 -- 172.21.15.89:0/2144091629 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb8012420 con 0x7feca80051c0 2023-10-10T20:41:12.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecb77fe700 1 -- 172.21.15.89:0/2144091629 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb8012580 con 0x7feca80051c0 2023-10-10T20:41:12.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.422+0000 7fecb77fe700 1 -- 172.21.15.89:0/2144091629 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb8012920 con 0x7feca80051c0 2023-10-10T20:41:12.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.423+0000 7fecc7018700 1 -- 172.21.15.89:0/2144091629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feca80051c0 msgr2=0x7feca800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:12.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.423+0000 7fecc7018700 1 --2- 172.21.15.89:0/2144091629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feca80051c0 0x7feca800ba60 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fecb80018b0 tx=0x7fecb8007d30 comp rx=0 tx=0).stop 2023-10-10T20:41:12.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.423+0000 7fecc7018700 1 -- 172.21.15.89:0/2144091629 shutdown_connections 2023-10-10T20:41:12.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.423+0000 7fecc7018700 1 --2- 172.21.15.89:0/2144091629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feca80051c0 0x7feca800ba60 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.423+0000 7fecc7018700 1 -- 172.21.15.89:0/2144091629 >> 172.21.15.89:0/2144091629 conn(0x7fecc0068e20 msgr2=0x7fecc00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:12.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.423+0000 7fecc7018700 1 -- 172.21.15.89:0/2144091629 shutdown_connections 2023-10-10T20:41:12.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.423+0000 7fecc7018700 1 -- 172.21.15.89:0/2144091629 wait complete. 2023-10-10T20:41:12.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.424+0000 7fecc7018700 1 Processor -- start 2023-10-10T20:41:12.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.424+0000 7fecc7018700 1 -- start start 2023-10-10T20:41:12.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecc7018700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feca80051c0 0x7fecc015b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecc7018700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fecc01554b0 0x7fecc00cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecc7018700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fecc00cd810 0x7fecc016d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecc7018700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc014cba0 con 0x7feca80051c0 2023-10-10T20:41:12.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecc7018700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc00cdc90 con 0x7fecc00cd810 2023-10-10T20:41:12.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecc7018700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc015cf50 con 0x7fecc01554b0 2023-10-10T20:41:12.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecc4db4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feca80051c0 0x7fecc015b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecc4db4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feca80051c0 0x7fecc015b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49534/0 (socket says 172.21.15.89:49534) 2023-10-10T20:41:12.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecc4db4700 1 -- 172.21.15.89:0/1219221253 learned_addr learned my addr 172.21.15.89:0/1219221253 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:12.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecc55b5700 1 --2- 172.21.15.89:0/1219221253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fecc00cd810 0x7fecc016d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecb7fff700 1 --2- 172.21.15.89:0/1219221253 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fecc01554b0 0x7fecc00cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecc4db4700 1 -- 172.21.15.89:0/1219221253 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fecc01554b0 msgr2=0x7fecc00cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:12.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecc4db4700 1 --2- 172.21.15.89:0/1219221253 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fecc01554b0 0x7fecc00cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecc4db4700 1 -- 172.21.15.89:0/1219221253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fecc00cd810 msgr2=0x7fecc016d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:12.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecc4db4700 1 --2- 172.21.15.89:0/1219221253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fecc00cd810 0x7fecc016d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecc4db4700 1 -- 172.21.15.89:0/1219221253 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc015bc50 con 0x7feca80051c0 2023-10-10T20:41:12.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecc55b5700 1 --2- 172.21.15.89:0/1219221253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fecc00cd810 0x7fecc016d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:12.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecc4db4700 1 --2- 172.21.15.89:0/1219221253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feca80051c0 0x7fecc015b710 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fecbc007a90 tx=0x7fecbc0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:12.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecb4ff9700 1 -- 172.21.15.89:0/1219221253 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc00ad70 con 0x7feca80051c0 2023-10-10T20:41:12.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecb4ff9700 1 -- 172.21.15.89:0/1219221253 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecbc0133f0 con 0x7feca80051c0 2023-10-10T20:41:12.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.425+0000 7fecc7018700 1 -- 172.21.15.89:0/1219221253 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc015c340 con 0x7feca80051c0 2023-10-10T20:41:12.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.426+0000 7fecb4ff9700 1 -- 172.21.15.89:0/1219221253 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc0137b0 con 0x7feca80051c0 2023-10-10T20:41:12.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.426+0000 7fecc7018700 1 -- 172.21.15.89:0/1219221253 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc015c900 con 0x7feca80051c0 2023-10-10T20:41:12.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.427+0000 7fecc7018700 1 -- 172.21.15.89:0/1219221253 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc004c3c0 con 0x7feca80051c0 2023-10-10T20:41:12.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.427+0000 7fecb4ff9700 1 -- 172.21.15.89:0/1219221253 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fecbc013b60 con 0x7feca80051c0 2023-10-10T20:41:12.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.427+0000 7fecb4ff9700 1 --2- 172.21.15.89:0/1219221253 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fecac067320 0x7fecac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.427+0000 7fecb4ff9700 1 -- 172.21.15.89:0/1219221253 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fecbc085670 con 0x7feca80051c0 2023-10-10T20:41:12.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.427+0000 7fecb7fff700 1 --2- 172.21.15.89:0/1219221253 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fecac067320 0x7fecac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.428+0000 7fecb7fff700 1 --2- 172.21.15.89:0/1219221253 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fecac067320 0x7fecac0697d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fecc00c6f40 tx=0x7fecb0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:12.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.431+0000 7fecb4ff9700 1 -- 172.21.15.89:0/1219221253 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fecbc052400 con 0x7feca80051c0 2023-10-10T20:41:12.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.608+0000 7fecc7018700 1 -- 172.21.15.89:0/1219221253 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fecc006a5b0 con 0x7fecac067320 2023-10-10T20:41:12.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.616+0000 7fecb4ff9700 1 -- 172.21.15.89:0/1219221253 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287184 (secure 0 0 0) 0x7fecc006a5b0 con 0x7fecac067320 2023-10-10T20:41:12.621 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:12.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.619+0000 7fecc7018700 1 -- 172.21.15.89:0/1219221253 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fecac067320 msgr2=0x7fecac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:12.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.619+0000 7fecc7018700 1 --2- 172.21.15.89:0/1219221253 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fecac067320 0x7fecac0697d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fecc00c6f40 tx=0x7fecb0008040 comp rx=0 tx=0).stop 2023-10-10T20:41:12.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.619+0000 7fecc7018700 1 -- 172.21.15.89:0/1219221253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feca80051c0 msgr2=0x7fecc015b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:12.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.619+0000 7fecc7018700 1 --2- 172.21.15.89:0/1219221253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feca80051c0 0x7fecc015b710 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fecbc007a90 tx=0x7fecbc0021d0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.619+0000 7fecc7018700 1 -- 172.21.15.89:0/1219221253 shutdown_connections 2023-10-10T20:41:12.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.619+0000 7fecc7018700 1 --2- 172.21.15.89:0/1219221253 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fecac067320 0x7fecac0697d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.619+0000 7fecc7018700 1 --2- 172.21.15.89:0/1219221253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fecc00cd810 0x7fecc016d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.619+0000 7fecc7018700 1 --2- 172.21.15.89:0/1219221253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feca80051c0 0x7fecc015b710 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.619+0000 7fecc7018700 1 --2- 172.21.15.89:0/1219221253 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fecc01554b0 0x7fecc00cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.619+0000 7fecc7018700 1 -- 172.21.15.89:0/1219221253 >> 172.21.15.89:0/1219221253 conn(0x7fecc0068e20 msgr2=0x7fecc00c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:12.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.619+0000 7fecc7018700 1 -- 172.21.15.89:0/1219221253 shutdown_connections 2023-10-10T20:41:12.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.619+0000 7fecc7018700 1 -- 172.21.15.89:0/1219221253 wait complete. 2023-10-10T20:41:12.626 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:41:12.632 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:41:12.633 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2023-10-10T20:41:10.731060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422956,"kb_used_data":11884,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752551764,"statfs":{"total":773094113280,"available":770613006336,"internally_reserved":0,"allocated":12169216,"data_stored":5911081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.561490"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.185044+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:43:38.336996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179076+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:42:10.326290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183633+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:31:04.342810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184156+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:29:07.750059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183700+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:06:14.543497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180424+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:15:09.117651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180699+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:43:54.668141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.185584+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:51.630860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182871+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:00.815725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178630+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:45:17.413763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.176188+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:28:49.571035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182500+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:27:02.185210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184448+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:23.971915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.177799+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:35.431888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183897+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:08.100006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182045+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:51.136287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181388+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:51:14.573623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182606+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:21:00.592871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.175771+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:29:15.059073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182998+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:35:58.896847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182721+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:57.837359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.175675+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:54:47.622590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184138+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:56.592906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.175733+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:04.315349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178752+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:39:12.818520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181110+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:36:04.128348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183202+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:24.718214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179120+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:57:20.319921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.185431+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:38:00.081732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183275+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:09.031752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.186844+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:54.943577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179704+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:51.630860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180587+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:06:14.543497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194387+0000","last_change":"2023-10-10T20:41:07.194387+0000","last_active":"2023-10-10T20:41:07.194387+0000","last_peered":"2023-10-10T20:41:07.194387+0000","last_clean":"2023-10-10T20:41:07.194387+0000","last_became_active":"2023-10-10T20:41:07.194097+0000","last_became_peered":"2023-10-10T20:41:07.194097+0000","last_unstale":"2023-10-10T20:41:07.194387+0000","last_undegraded":"2023-10-10T20:41:07.194387+0000","last_fullsized":"2023-10-10T20:41:07.194387+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:26.546168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182362+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:37:12.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192372+0000","last_change":"2023-10-10T20:41:07.192372+0000","last_active":"2023-10-10T20:41:07.192372+0000","last_peered":"2023-10-10T20:41:07.192372+0000","last_clean":"2023-10-10T20:41:07.192372+0000","last_became_active":"2023-10-10T20:41:07.192088+0000","last_became_peered":"2023-10-10T20:41:07.192088+0000","last_unstale":"2023-10-10T20:41:07.192372+0000","last_undegraded":"2023-10-10T20:41:07.192372+0000","last_fullsized":"2023-10-10T20:41:07.192372+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:59.684638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181983+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:01.392782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.187579+0000","last_change":"2023-10-10T20:41:07.187579+0000","last_active":"2023-10-10T20:41:07.187579+0000","last_peered":"2023-10-10T20:41:07.187579+0000","last_clean":"2023-10-10T20:41:07.187579+0000","last_became_active":"2023-10-10T20:41:07.187296+0000","last_became_peered":"2023-10-10T20:41:07.187296+0000","last_unstale":"2023-10-10T20:41:07.187579+0000","last_undegraded":"2023-10-10T20:41:07.187579+0000","last_fullsized":"2023-10-10T20:41:07.187579+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:18.852267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182717+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:56:26.100041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194195+0000","last_change":"2023-10-10T20:41:07.194195+0000","last_active":"2023-10-10T20:41:07.194195+0000","last_peered":"2023-10-10T20:41:07.194195+0000","last_clean":"2023-10-10T20:41:07.194195+0000","last_became_active":"2023-10-10T20:41:07.193920+0000","last_became_peered":"2023-10-10T20:41:07.193920+0000","last_unstale":"2023-10-10T20:41:07.194195+0000","last_undegraded":"2023-10-10T20:41:07.194195+0000","last_fullsized":"2023-10-10T20:41:07.194195+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:36.588822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182202+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:40:14.372905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.193266+0000","last_change":"2023-10-10T20:41:07.193266+0000","last_active":"2023-10-10T20:41:07.193266+0000","last_peered":"2023-10-10T20:41:07.193266+0000","last_clean":"2023-10-10T20:41:07.193266+0000","last_became_active":"2023-10-10T20:41:07.192991+0000","last_became_peered":"2023-10-10T20:41:07.192991+0000","last_unstale":"2023-10-10T20:41:07.193266+0000","last_undegraded":"2023-10-10T20:41:07.193266+0000","last_fullsized":"2023-10-10T20:41:07.193266+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:15.545053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182281+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:51:25.089205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.193695+0000","last_change":"2023-10-10T20:41:07.193695+0000","last_active":"2023-10-10T20:41:07.193695+0000","last_peered":"2023-10-10T20:41:07.193695+0000","last_clean":"2023-10-10T20:41:07.193695+0000","last_became_active":"2023-10-10T20:41:07.193394+0000","last_became_peered":"2023-10-10T20:41:07.193394+0000","last_unstale":"2023-10-10T20:41:07.193695+0000","last_undegraded":"2023-10-10T20:41:07.193695+0000","last_fullsized":"2023-10-10T20:41:07.193695+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:56.835912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.186003+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:44.120514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.193880+0000","last_change":"2023-10-10T20:41:07.193880+0000","last_active":"2023-10-10T20:41:07.193880+0000","last_peered":"2023-10-10T20:41:07.193880+0000","last_clean":"2023-10-10T20:41:07.193880+0000","last_became_active":"2023-10-10T20:41:07.193586+0000","last_became_peered":"2023-10-10T20:41:07.193586+0000","last_unstale":"2023-10-10T20:41:07.193880+0000","last_undegraded":"2023-10-10T20:41:07.193880+0000","last_fullsized":"2023-10-10T20:41:07.193880+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:15:36.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178163+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:13:00.611054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194069+0000","last_change":"2023-10-10T20:41:07.194069+0000","last_active":"2023-10-10T20:41:07.194069+0000","last_peered":"2023-10-10T20:41:07.194069+0000","last_clean":"2023-10-10T20:41:07.194069+0000","last_became_active":"2023-10-10T20:41:07.193800+0000","last_became_peered":"2023-10-10T20:41:07.193800+0000","last_unstale":"2023-10-10T20:41:07.194069+0000","last_undegraded":"2023-10-10T20:41:07.194069+0000","last_fullsized":"2023-10-10T20:41:07.194069+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:00:42.940622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183477+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:59:58.659044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.190099+0000","last_change":"2023-10-10T20:41:07.190099+0000","last_active":"2023-10-10T20:41:07.190099+0000","last_peered":"2023-10-10T20:41:07.190099+0000","last_clean":"2023-10-10T20:41:07.190099+0000","last_became_active":"2023-10-10T20:41:07.189840+0000","last_became_peered":"2023-10-10T20:41:07.189840+0000","last_unstale":"2023-10-10T20:41:07.190099+0000","last_undegraded":"2023-10-10T20:41:07.190099+0000","last_fullsized":"2023-10-10T20:41:07.190099+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:48:33.094041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178595+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:35.431888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.187415+0000","last_change":"2023-10-10T20:41:07.187415+0000","last_active":"2023-10-10T20:41:07.187415+0000","last_peered":"2023-10-10T20:41:07.187415+0000","last_clean":"2023-10-10T20:41:07.187415+0000","last_became_active":"2023-10-10T20:41:07.187114+0000","last_became_peered":"2023-10-10T20:41:07.187114+0000","last_unstale":"2023-10-10T20:41:07.187415+0000","last_undegraded":"2023-10-10T20:41:07.187415+0000","last_fullsized":"2023-10-10T20:41:07.187415+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:58:20.063048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181497+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:42:34.466499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.188746+0000","last_change":"2023-10-10T20:41:07.188746+0000","last_active":"2023-10-10T20:41:07.188746+0000","last_peered":"2023-10-10T20:41:07.188746+0000","last_clean":"2023-10-10T20:41:07.188746+0000","last_became_active":"2023-10-10T20:41:07.188486+0000","last_became_peered":"2023-10-10T20:41:07.188486+0000","last_unstale":"2023-10-10T20:41:07.188746+0000","last_undegraded":"2023-10-10T20:41:07.188746+0000","last_fullsized":"2023-10-10T20:41:07.188746+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:36.588822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182276+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:04:10.686499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.191180+0000","last_change":"2023-10-10T20:41:07.191180+0000","last_active":"2023-10-10T20:41:07.191180+0000","last_peered":"2023-10-10T20:41:07.191180+0000","last_clean":"2023-10-10T20:41:07.191180+0000","last_became_active":"2023-10-10T20:41:07.190910+0000","last_became_peered":"2023-10-10T20:41:07.190910+0000","last_unstale":"2023-10-10T20:41:07.191180+0000","last_undegraded":"2023-10-10T20:41:07.191180+0000","last_fullsized":"2023-10-10T20:41:07.191180+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:13.928318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184946+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:43.974985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.189943+0000","last_change":"2023-10-10T20:41:07.189943+0000","last_active":"2023-10-10T20:41:07.189943+0000","last_peered":"2023-10-10T20:41:07.189943+0000","last_clean":"2023-10-10T20:41:07.189943+0000","last_became_active":"2023-10-10T20:41:07.189639+0000","last_became_peered":"2023-10-10T20:41:07.189639+0000","last_unstale":"2023-10-10T20:41:07.189943+0000","last_undegraded":"2023-10-10T20:41:07.189943+0000","last_fullsized":"2023-10-10T20:41:07.189943+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:24:45.124009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178350+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:07:36.012741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.193331+0000","last_change":"2023-10-10T20:41:07.193331+0000","last_active":"2023-10-10T20:41:07.193331+0000","last_peered":"2023-10-10T20:41:07.193331+0000","last_clean":"2023-10-10T20:41:07.193331+0000","last_became_active":"2023-10-10T20:41:07.193077+0000","last_became_peered":"2023-10-10T20:41:07.193077+0000","last_unstale":"2023-10-10T20:41:07.193331+0000","last_undegraded":"2023-10-10T20:41:07.193331+0000","last_fullsized":"2023-10-10T20:41:07.193331+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:56.816260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182401+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:47:38.801771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.190581+0000","last_change":"2023-10-10T20:41:07.190581+0000","last_active":"2023-10-10T20:41:07.190581+0000","last_peered":"2023-10-10T20:41:07.190581+0000","last_clean":"2023-10-10T20:41:07.190581+0000","last_became_active":"2023-10-10T20:41:07.190319+0000","last_became_peered":"2023-10-10T20:41:07.190319+0000","last_unstale":"2023-10-10T20:41:07.190581+0000","last_undegraded":"2023-10-10T20:41:07.190581+0000","last_fullsized":"2023-10-10T20:41:07.190581+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:42:09.324842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183200+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:15:37.860366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.191241+0000","last_change":"2023-10-10T20:41:07.191241+0000","last_active":"2023-10-10T20:41:07.191241+0000","last_peered":"2023-10-10T20:41:07.191241+0000","last_clean":"2023-10-10T20:41:07.191241+0000","last_became_active":"2023-10-10T20:41:07.191020+0000","last_became_peered":"2023-10-10T20:41:07.191020+0000","last_unstale":"2023-10-10T20:41:07.191241+0000","last_undegraded":"2023-10-10T20:41:07.191241+0000","last_fullsized":"2023-10-10T20:41:07.191241+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:56.816260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180046+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:07:36.012741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.187395+0000","last_change":"2023-10-10T20:41:07.187395+0000","last_active":"2023-10-10T20:41:07.187395+0000","last_peered":"2023-10-10T20:41:07.187395+0000","last_clean":"2023-10-10T20:41:07.187395+0000","last_became_active":"2023-10-10T20:41:07.187130+0000","last_became_peered":"2023-10-10T20:41:07.187130+0000","last_unstale":"2023-10-10T20:41:07.187395+0000","last_undegraded":"2023-10-10T20:41:07.187395+0000","last_fullsized":"2023-10-10T20:41:07.187395+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:32:03.550923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181825+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:54.943577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.190127+0000","last_change":"2023-10-10T20:41:07.190127+0000","last_active":"2023-10-10T20:41:07.190127+0000","last_peered":"2023-10-10T20:41:07.190127+0000","last_clean":"2023-10-10T20:41:07.190127+0000","last_became_active":"2023-10-10T20:41:07.189866+0000","last_became_peered":"2023-10-10T20:41:07.189866+0000","last_unstale":"2023-10-10T20:41:07.190127+0000","last_undegraded":"2023-10-10T20:41:07.190127+0000","last_fullsized":"2023-10-10T20:41:07.190127+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:31:03.341362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.177885+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:57:20.319921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192468+0000","last_change":"2023-10-10T20:41:07.192468+0000","last_active":"2023-10-10T20:41:07.192468+0000","last_peered":"2023-10-10T20:41:07.192468+0000","last_clean":"2023-10-10T20:41:07.192468+0000","last_became_active":"2023-10-10T20:41:07.192177+0000","last_became_peered":"2023-10-10T20:41:07.192177+0000","last_unstale":"2023-10-10T20:41:07.192468+0000","last_undegraded":"2023-10-10T20:41:07.192468+0000","last_fullsized":"2023-10-10T20:41:07.192468+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:15.545053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183818+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:53.393618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.188588+0000","last_change":"2023-10-10T20:41:07.188588+0000","last_active":"2023-10-10T20:41:07.188588+0000","last_peered":"2023-10-10T20:41:07.188588+0000","last_clean":"2023-10-10T20:41:07.188588+0000","last_became_active":"2023-10-10T20:41:07.188209+0000","last_became_peered":"2023-10-10T20:41:07.188209+0000","last_unstale":"2023-10-10T20:41:07.188588+0000","last_undegraded":"2023-10-10T20:41:07.188588+0000","last_fullsized":"2023-10-10T20:41:07.188588+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:00:42.940622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180348+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:44.120514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194274+0000","last_change":"2023-10-10T20:41:07.194274+0000","last_active":"2023-10-10T20:41:07.194274+0000","last_peered":"2023-10-10T20:41:07.194274+0000","last_clean":"2023-10-10T20:41:07.194274+0000","last_became_active":"2023-10-10T20:41:07.193987+0000","last_became_peered":"2023-10-10T20:41:07.193987+0000","last_unstale":"2023-10-10T20:41:07.194274+0000","last_undegraded":"2023-10-10T20:41:07.194274+0000","last_fullsized":"2023-10-10T20:41:07.194274+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:17.174004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179361+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:14.929765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194646+0000","last_change":"2023-10-10T20:41:07.194646+0000","last_active":"2023-10-10T20:41:07.194646+0000","last_peered":"2023-10-10T20:41:07.194646+0000","last_clean":"2023-10-10T20:41:07.194646+0000","last_became_active":"2023-10-10T20:41:07.194426+0000","last_became_peered":"2023-10-10T20:41:07.194426+0000","last_unstale":"2023-10-10T20:41:07.194646+0000","last_undegraded":"2023-10-10T20:41:07.194646+0000","last_fullsized":"2023-10-10T20:41:07.194646+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:00.391334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181308+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:47:27.526950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192740+0000","last_change":"2023-10-10T20:41:07.192740+0000","last_active":"2023-10-10T20:41:07.192740+0000","last_peered":"2023-10-10T20:41:07.192740+0000","last_clean":"2023-10-10T20:41:07.192740+0000","last_became_active":"2023-10-10T20:41:07.192480+0000","last_became_peered":"2023-10-10T20:41:07.192480+0000","last_unstale":"2023-10-10T20:41:07.192740+0000","last_undegraded":"2023-10-10T20:41:07.192740+0000","last_fullsized":"2023-10-10T20:41:07.192740+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:48.161107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181783+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:27.547616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194467+0000","last_change":"2023-10-10T20:41:07.194467+0000","last_active":"2023-10-10T20:41:07.194467+0000","last_peered":"2023-10-10T20:41:07.194467+0000","last_clean":"2023-10-10T20:41:07.194467+0000","last_became_active":"2023-10-10T20:41:07.193987+0000","last_became_peered":"2023-10-10T20:41:07.193987+0000","last_unstale":"2023-10-10T20:41:07.194467+0000","last_undegraded":"2023-10-10T20:41:07.194467+0000","last_fullsized":"2023-10-10T20:41:07.194467+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:43.862933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184078+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:38:30.611281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.196552+0000","last_change":"2023-10-10T20:41:07.196552+0000","last_active":"2023-10-10T20:41:07.196552+0000","last_peered":"2023-10-10T20:41:07.196552+0000","last_clean":"2023-10-10T20:41:07.196552+0000","last_became_active":"2023-10-10T20:41:07.196285+0000","last_became_peered":"2023-10-10T20:41:07.196285+0000","last_unstale":"2023-10-10T20:41:07.196552+0000","last_undegraded":"2023-10-10T20:41:07.196552+0000","last_fullsized":"2023-10-10T20:41:07.196552+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:43:53.666694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178501+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:43.761996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192768+0000","last_change":"2023-10-10T20:41:07.192768+0000","last_active":"2023-10-10T20:41:07.192768+0000","last_peered":"2023-10-10T20:41:07.192768+0000","last_clean":"2023-10-10T20:41:07.192768+0000","last_became_active":"2023-10-10T20:41:07.192467+0000","last_became_peered":"2023-10-10T20:41:07.192467+0000","last_unstale":"2023-10-10T20:41:07.192768+0000","last_undegraded":"2023-10-10T20:41:07.192768+0000","last_fullsized":"2023-10-10T20:41:07.192768+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:59.684638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182731+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:47:27.526950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.186703+0000","last_change":"2023-10-10T20:41:07.186703+0000","last_active":"2023-10-10T20:41:07.186703+0000","last_peered":"2023-10-10T20:41:07.186703+0000","last_clean":"2023-10-10T20:41:07.186703+0000","last_became_active":"2023-10-10T20:41:07.186447+0000","last_became_peered":"2023-10-10T20:41:07.186447+0000","last_unstale":"2023-10-10T20:41:07.186703+0000","last_undegraded":"2023-10-10T20:41:07.186703+0000","last_fullsized":"2023-10-10T20:41:07.186703+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:57.197462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181353+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:58:21.064495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192864+0000","last_change":"2023-10-10T20:41:07.192864+0000","last_active":"2023-10-10T20:41:07.192864+0000","last_peered":"2023-10-10T20:41:07.192864+0000","last_clean":"2023-10-10T20:41:07.192864+0000","last_became_active":"2023-10-10T20:41:07.192602+0000","last_became_peered":"2023-10-10T20:41:07.192602+0000","last_unstale":"2023-10-10T20:41:07.192864+0000","last_undegraded":"2023-10-10T20:41:07.192864+0000","last_fullsized":"2023-10-10T20:41:07.192864+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:37:19.121105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179266+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:13:00.611054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192698+0000","last_change":"2023-10-10T20:41:07.192698+0000","last_active":"2023-10-10T20:41:07.192698+0000","last_peered":"2023-10-10T20:41:07.192698+0000","last_clean":"2023-10-10T20:41:07.192698+0000","last_became_active":"2023-10-10T20:41:07.192370+0000","last_became_peered":"2023-10-10T20:41:07.192370+0000","last_unstale":"2023-10-10T20:41:07.192698+0000","last_undegraded":"2023-10-10T20:41:07.192698+0000","last_fullsized":"2023-10-10T20:41:07.192698+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:15:08.116204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180520+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:56:26.100041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194285+0000","last_change":"2023-10-10T20:41:07.194285+0000","last_active":"2023-10-10T20:41:07.194285+0000","last_peered":"2023-10-10T20:41:07.194285+0000","last_clean":"2023-10-10T20:41:07.194285+0000","last_became_active":"2023-10-10T20:41:07.194002+0000","last_became_peered":"2023-10-10T20:41:07.194002+0000","last_unstale":"2023-10-10T20:41:07.194285+0000","last_undegraded":"2023-10-10T20:41:07.194285+0000","last_fullsized":"2023-10-10T20:41:07.194285+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:42:33.465052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179142+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:56:52.494512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.186005+0000","last_change":"2023-10-10T20:41:07.186005+0000","last_active":"2023-10-10T20:41:07.186005+0000","last_peered":"2023-10-10T20:41:07.186005+0000","last_clean":"2023-10-10T20:41:07.186005+0000","last_became_active":"2023-10-10T20:41:07.185723+0000","last_became_peered":"2023-10-10T20:41:07.185723+0000","last_unstale":"2023-10-10T20:41:07.186005+0000","last_undegraded":"2023-10-10T20:41:07.186005+0000","last_fullsized":"2023-10-10T20:41:07.186005+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:39.932733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181885+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:18.175451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.195197+0000","last_change":"2023-10-10T20:41:07.195197+0000","last_active":"2023-10-10T20:41:07.195197+0000","last_peered":"2023-10-10T20:41:07.195197+0000","last_clean":"2023-10-10T20:41:07.195197+0000","last_became_active":"2023-10-10T20:41:07.194893+0000","last_became_peered":"2023-10-10T20:41:07.194893+0000","last_unstale":"2023-10-10T20:41:07.195197+0000","last_undegraded":"2023-10-10T20:41:07.195197+0000","last_fullsized":"2023-10-10T20:41:07.195197+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:51:13.572175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":253,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-10-10T20:41:06.176359+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:41:06.176359+0000","last_peered":"2023-10-10T20:41:06.176359+0000","last_clean":"2023-10-10T20:41:06.176359+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:41:06.176359+0000","last_undegraded":"2023-10-10T20:41:06.176359+0000","last_fullsized":"2023-10-10T20:41:06.176359+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303308,"kb_used_data":1924,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068532,"statfs":{"total":96636764160,"available":96326176768,"internally_reserved":0,"allocated":1970176,"data_stored":1183347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51500000000000001,"15min":0.81999999999999995},"min":{"1min":0.31,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53000000000000003,"15min":0.627},"min":{"1min":0.32200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71999999999999997,"5min":0.95399999999999996,"15min":4.1959999999999997},"last":0.54000000000000004}]},{"osd":1,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.504,"15min":1},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.67800000000000005,"5min":0.78800000000000003,"15min":10.1},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55800000000000005,"15min":0.72999999999999998},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.28999999999999998},"max":{"1min":0.81799999999999995,"5min":1.006,"15min":6.806},"last":0.65300000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57899999999999996,"15min":0.78100000000000003},"min":{"1min":0.43099999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89000000000000001,"5min":1.089,"15min":6.9340000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53200000000000003,"15min":0.73799999999999999},"min":{"1min":0.39000000000000001,"5min":0.36399999999999999,"15min":0.30099999999999999},"max":{"1min":0.77100000000000002,"5min":0.85299999999999998,"15min":6.9870000000000001},"last":0.46899999999999997}]},{"osd":3,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.75800000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.754,"5min":0.80200000000000005,"15min":6.6719999999999997},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.67000000000000004},"min":{"1min":0.38700000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.70599999999999996,"5min":1.2789999999999999,"15min":4.1609999999999996},"last":0.60099999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47399999999999998,"15min":0.46500000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.73199999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.621},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52400000000000002,"15min":0.52300000000000002},"min":{"1min":0.309,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.55200000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.32500000000000001,"5min":0.32100000000000001,"15min":0.314},"max":{"1min":0.86099999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75800000000000001,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.68899999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72999999999999998,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84099999999999997,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.66500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302728,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069112,"statfs":{"total":96636764160,"available":96326770688,"internally_reserved":0,"allocated":1376256,"data_stored":592979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50700000000000001,"15min":0.67200000000000004},"min":{"1min":0.33500000000000002,"5min":0.32400000000000001,"15min":0.312},"max":{"1min":0.68600000000000005,"5min":0.73099999999999998,"15min":7.7590000000000003},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52100000000000002,"15min":0.71299999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":7.8090000000000002},"last":0.52400000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.70999999999999996},"min":{"1min":0.38400000000000001,"5min":0.35599999999999998,"15min":0.27800000000000002},"max":{"1min":0.628,"5min":0.80000000000000004,"15min":7.867},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53300000000000003,"15min":0.88500000000000001},"min":{"1min":0.36599999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":9.7089999999999996},"last":0.54400000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51400000000000001,"15min":0.59599999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.72499999999999998,"5min":0.747,"15min":4.7999999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54600000000000004,"15min":0.84499999999999997},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.71499999999999997,"5min":0.80000000000000004,"15min":6.7629999999999999},"last":0.63}]},{"osd":3,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.62},"min":{"1min":0.40999999999999998,"5min":0.36299999999999999,"15min":0.35299999999999998},"max":{"1min":0.79100000000000004,"5min":0.83999999999999997,"15min":4.8620000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54800000000000004,"15min":0.71699999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.35299999999999998},"max":{"1min":0.68600000000000005,"5min":1.1850000000000001,"15min":7.9009999999999998},"last":0.64900000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51500000000000001,"15min":0.46800000000000003},"min":{"1min":0.32400000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71099999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59199999999999997,"15min":0.56799999999999995},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.37},"max":{"1min":0.73499999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.49099999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52000000000000002,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.314,"15min":0.312},"max":{"1min":0.69199999999999995,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.747,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.433}]},{"osd":7,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55100000000000005,"15min":0.52000000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.72599999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57799999999999996,"15min":0.54400000000000004},"min":{"1min":0.42899999999999999,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":0.69399999999999995,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.47499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302716,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069124,"statfs":{"total":96636764160,"available":96326782976,"internally_reserved":0,"allocated":1363968,"data_stored":586716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.31900000000000001,"15min":0.47599999999999998},"min":{"1min":0.20999999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.441,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.32300000000000001,"15min":0.36299999999999999},"min":{"1min":0.23599999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46000000000000002,"5min":0.68700000000000006,"15min":1.194},"last":0.308}]},{"osd":2,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.32400000000000001,"15min":0.36399999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.379,"5min":0.67400000000000004,"15min":1.2869999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.33800000000000002,"15min":0.505},"min":{"1min":0.26000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.54200000000000004,"5min":0.57199999999999995,"15min":5.0919999999999996},"last":0.51300000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36299999999999999,"15min":0.52800000000000002},"min":{"1min":0.26800000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.51000000000000001,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36499999999999999,"15min":0.40699999999999997},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.48299999999999998,"5min":0.72099999999999997,"15min":1.2170000000000001},"last":0.41399999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57899999999999996,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.81000000000000005,"5min":1.133,"15min":1.254},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56599999999999995,"15min":0.70599999999999996},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69899999999999995,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.67100000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57899999999999996,"15min":0.59499999999999997},"min":{"1min":0.42299999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.751,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.499},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.60799999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71899999999999997,"5min":1.0720000000000001,"15min":1.3109999999999999},"last":0.55700000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60699999999999998,"15min":0.84399999999999997},"min":{"1min":0.435,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86199999999999999,"5min":0.995,"15min":5.0659999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61699999999999999,"15min":0.63200000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84599999999999997,"5min":1.1439999999999999,"15min":1.171},"last":0.58899999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59299999999999997,"15min":0.61699999999999999},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.753,"5min":1.214,"15min":1.214},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60099999999999998,"15min":0.72499999999999998},"min":{"1min":0.45000000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76900000000000002,"5min":1.115,"15min":4.0499999999999998},"last":0.625}]}]},{"osd":0,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303304,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068536,"statfs":{"total":96636764160,"available":96326180864,"internally_reserved":0,"allocated":1966080,"data_stored":1182386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.309,"15min":0.52800000000000002},"min":{"1min":0.20999999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.38300000000000001,"5min":0.51600000000000001,"15min":7.9249999999999998},"last":0.371},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.318,"15min":0.505},"min":{"1min":0.23699999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.41899999999999998,"5min":0.54900000000000004,"15min":6.6299999999999999},"last":0.255}]},{"osd":2,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.32600000000000001,"15min":0.55200000000000005},"min":{"1min":0.22,"5min":0.216,"15min":0.216},"max":{"1min":0.48799999999999999,"5min":0.55400000000000005,"15min":8.2509999999999994},"last":0.313},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.34000000000000002,"15min":0.52800000000000002},"min":{"1min":0.23100000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.45700000000000002,"5min":0.54100000000000004,"15min":6.7370000000000001},"last":0.32800000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.34399999999999997,"15min":0.55700000000000005},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.48299999999999998,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.61199999999999999},"min":{"1min":0.32000000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":7.8929999999999998},"last":0.41599999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55600000000000005,"15min":0.71399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.72499999999999998,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.748},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56299999999999994,"15min":0.77200000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.67600000000000005,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.71099999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.55200000000000005,"15min":0.751},"min":{"1min":0.38800000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69299999999999995,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58099999999999996,"15min":0.77800000000000002},"min":{"1min":0.42299999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77000000000000002,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.75700000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60199999999999998,"15min":0.81799999999999995},"min":{"1min":0.39800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72099999999999997,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61099999999999999,"15min":0.76100000000000001},"min":{"1min":0.41399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74299999999999999,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58799999999999997,"15min":0.752},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78300000000000003,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59099999999999997,"15min":0.78600000000000003},"min":{"1min":0.45200000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.69299999999999995,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.60299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302728,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069112,"statfs":{"total":96636764160,"available":96326770688,"internally_reserved":0,"allocated":1376256,"data_stored":592979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.28299999999999997,"5min":0.309,"15min":0.32500000000000001},"min":{"1min":0.21099999999999999,"5min":0.189,"15min":0.189},"max":{"1min":0.38300000000000001,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.29399999999999998,"5min":0.32100000000000001,"15min":0.35299999999999998},"min":{"1min":0.215,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.39500000000000002,"5min":0.5,"15min":0.746},"last":0.29799999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.32900000000000001,"15min":0.72899999999999998},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.35699999999999998,"5min":0.63,"15min":11.967000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.34499999999999997,"15min":0.48299999999999998},"min":{"1min":0.26800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.443,"5min":0.53400000000000003,"15min":4.1100000000000003},"last":0.36499999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.34499999999999997,"15min":0.55700000000000005},"min":{"1min":0.23699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.39400000000000002,"5min":0.55400000000000005,"15min":6.3040000000000003},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38700000000000001,"15min":0.78100000000000003},"min":{"1min":0.27900000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.46899999999999997,"5min":0.65900000000000003,"15min":11.935},"last":0.35199999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.54300000000000004,"15min":0.55500000000000005},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.58699999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71699999999999997,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.58599999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.378},"max":{"1min":0.66300000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54400000000000004,"15min":0.58399999999999996},"min":{"1min":0.39700000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74199999999999999,"5min":0.874,"15min":0.96099999999999997},"last":0.54100000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57699999999999996,"15min":0.60099999999999998},"min":{"1min":0.42199999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.745,"5min":0.91600000000000004,"15min":0.91800000000000004},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73799999999999999,"5min":0.93100000000000005,"15min":0.93600000000000005},"last":0.55500000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56499999999999995,"15min":0.58499999999999996},"min":{"1min":0.41799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71099999999999997,"5min":0.89100000000000001,"15min":1.052},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57699999999999996,"15min":0.78000000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69799999999999995,"5min":0.85999999999999999,"15min":6.2649999999999997},"last":0.45900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302728,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069112,"statfs":{"total":96636764160,"available":96326770688,"internally_reserved":0,"allocated":1376256,"data_stored":592979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.31,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.31900000000000001},"min":{"1min":0.25,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.33600000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.23499999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.316},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38600000000000001,"15min":0.377},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.57099999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.40500000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.34999999999999998},"min":{"1min":0.23699999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.35699999999999998,"15min":0.36199999999999999},"min":{"1min":0.26300000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.35899999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69999999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.82299999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55300000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.35899999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55900000000000005,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.76900000000000002,"5min":1.01,"15min":1.01},"last":0.626}]},{"osd":6,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.42099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85699999999999998,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.60999999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.441,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.496,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.88400000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302716,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069124,"statfs":{"total":96636764160,"available":96326782976,"internally_reserved":0,"allocated":1363968,"data_stored":586716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68300000000000005,"5min":0.74199999999999999,"15min":0.747},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.76600000000000001},"min":{"1min":0.374,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67500000000000004,"5min":0.78400000000000003,"15min":7.6719999999999997},"last":0.439}]},{"osd":1,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52900000000000003,"15min":0.77400000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.755,"5min":0.755,"15min":7.7160000000000002},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65300000000000002,"5min":0.76900000000000002,"15min":0.86799999999999999},"last":0.66100000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.65300000000000002,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56299999999999994,"15min":0.79300000000000004},"min":{"1min":0.39700000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.72799999999999998,"5min":0.79800000000000004,"15min":7.7839999999999998},"last":0.68500000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55200000000000005,"15min":0.54900000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65500000000000003,"5min":0.73399999999999999,"15min":0.96999999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.41199999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.66100000000000003,"5min":0.78100000000000003,"15min":0.79400000000000004},"last":0.47099999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51000000000000001,"15min":0.48699999999999999},"min":{"1min":0.32500000000000001,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.77500000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54900000000000004,"15min":0.54300000000000004},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.55300000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.39200000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68799999999999994,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.35899999999999999},"max":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.378,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.84599999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.46100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.748,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.60899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302728,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069112,"statfs":{"total":96636764160,"available":96326770688,"internally_reserved":0,"allocated":1376256,"data_stored":592979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.28599999999999998,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.79300000000000004},"last":0.52400000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52700000000000002,"15min":0.64100000000000001},"min":{"1min":0.33000000000000002,"5min":0.32200000000000001,"15min":0.316},"max":{"1min":0.70399999999999996,"5min":1.2330000000000001,"15min":5.2679999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51800000000000002,"15min":0.629},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.47099999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52800000000000002,"15min":0.64800000000000002},"min":{"1min":0.35999999999999999,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.76600000000000001,"5min":0.86199999999999999,"15min":5.2960000000000003},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72899999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.45000000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55300000000000005,"15min":0.67700000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.73899999999999999,"5min":0.93000000000000005,"15min":5.3330000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.69099999999999995},"min":{"1min":0.39300000000000002,"5min":0.36899999999999999,"15min":0.27700000000000002},"max":{"1min":0.76000000000000001,"5min":0.94399999999999995,"15min":5.2850000000000001},"last":0.53900000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47799999999999998,"15min":0.496},"min":{"1min":0.311,"5min":0.29599999999999999,"15min":0.28699999999999998},"max":{"1min":0.72499999999999998,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34300000000000003,"15min":0.32200000000000001},"max":{"1min":0.70799999999999996,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.63800000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54100000000000004,"15min":0.54700000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.80100000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82499999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.624}]},{"osd":7,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.378,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68899999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60199999999999998,"15min":0.60099999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83399999999999996,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.61399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:41:12.643 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:41:12.676 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:41:12.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.787+0000 7f9f1f574700 1 Processor -- start 2023-10-10T20:41:12.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.788+0000 7f9f1f574700 1 -- start start 2023-10-10T20:41:12.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.788+0000 7f9f1f574700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f18159c60 0x7f9f1815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.788+0000 7f9f1f574700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9f1815db10 0x7f9f18069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.788+0000 7f9f1f574700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f1815a160 con 0x7f9f18159890 2023-10-10T20:41:12.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.788+0000 7f9f1f574700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f180d13a0 con 0x7f9f1815db10 2023-10-10T20:41:12.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.788+0000 7f9f1f574700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1815e140 con 0x7f9f18159c60 2023-10-10T20:41:12.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.789+0000 7f9f1db11700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9f1815db10 0x7f9f18069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.789+0000 7f9f1db11700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9f1815db10 0x7f9f18069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49546/0 (socket says 172.21.15.89:49546) 2023-10-10T20:41:12.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.789+0000 7f9f1db11700 1 -- 172.21.15.89:0/2025574889 learned_addr learned my addr 172.21.15.89:0/2025574889 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:12.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.789+0000 7f9f1cb0f700 1 --2- 172.21.15.89:0/2025574889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f18159c60 0x7f9f1815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.789+0000 7f9f1db11700 1 -- 172.21.15.89:0/2025574889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f18159c60 msgr2=0x7f9f1815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:12.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.789+0000 7f9f1db11700 1 --2- 172.21.15.89:0/2025574889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f18159c60 0x7f9f1815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.789+0000 7f9f1db11700 1 -- 172.21.15.89:0/2025574889 >> v1:172.21.15.111:6789/0 conn(0x7f9f18159890 legacy=0x7f9f1815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:12.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.789+0000 7f9f1db11700 1 -- 172.21.15.89:0/2025574889 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1815a3d0 con 0x7f9f1815db10 2023-10-10T20:41:12.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.789+0000 7f9f1cb0f700 1 --2- 172.21.15.89:0/2025574889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f18159c60 0x7f9f1815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:12.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.789+0000 7f9f1db11700 1 --2- 172.21.15.89:0/2025574889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f1815db10 0x7f9f18069560 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f9f100099b0 tx=0x7f9f10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a597f35c9f97f6b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:12.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.789+0000 7f9f0ffff700 1 -- 172.21.15.89:0/2025574889 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1000b820 con 0x7f9f1815db10 2023-10-10T20:41:12.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.789+0000 7f9f0ffff700 1 -- 172.21.15.89:0/2025574889 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1000b980 con 0x7f9f1815db10 2023-10-10T20:41:12.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.789+0000 7f9f0ffff700 1 -- 172.21.15.89:0/2025574889 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1000bd20 con 0x7f9f1815db10 2023-10-10T20:41:12.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.790+0000 7f9f1f574700 1 -- 172.21.15.89:0/2025574889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f1815db10 msgr2=0x7f9f18069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:12.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.790+0000 7f9f1f574700 1 --2- 172.21.15.89:0/2025574889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f1815db10 0x7f9f18069560 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f9f100099b0 tx=0x7f9f10009250 comp rx=0 tx=0).stop 2023-10-10T20:41:12.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.790+0000 7f9f1f574700 1 -- 172.21.15.89:0/2025574889 shutdown_connections 2023-10-10T20:41:12.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.790+0000 7f9f1f574700 1 --2- 172.21.15.89:0/2025574889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f1815db10 0x7f9f18069560 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.790+0000 7f9f1f574700 1 --2- 172.21.15.89:0/2025574889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f18159c60 0x7f9f1815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.790+0000 7f9f1f574700 1 -- 172.21.15.89:0/2025574889 >> 172.21.15.89:0/2025574889 conn(0x7f9f18068e20 msgr2=0x7f9f180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:12.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.790+0000 7f9f1f574700 1 -- 172.21.15.89:0/2025574889 shutdown_connections 2023-10-10T20:41:12.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.790+0000 7f9f1f574700 1 -- 172.21.15.89:0/2025574889 wait complete. 2023-10-10T20:41:12.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.791+0000 7f9f1f574700 1 Processor -- start 2023-10-10T20:41:12.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.791+0000 7f9f1f574700 1 -- start start 2023-10-10T20:41:12.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.792+0000 7f9f1f574700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f18159890 0x7f9f180ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.792+0000 7f9f1f574700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f18159c60 0x7f9f180c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.792+0000 7f9f1f574700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f1815db10 0x7f9f180b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.792+0000 7f9f1f574700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f180cdde0 con 0x7f9f18159890 2023-10-10T20:41:12.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.792+0000 7f9f1f574700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f180ce080 con 0x7f9f1815db10 2023-10-10T20:41:12.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.792+0000 7f9f1f574700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f180ce360 con 0x7f9f18159c60 2023-10-10T20:41:12.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.792+0000 7f9f1d310700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f18159890 0x7f9f180ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.792+0000 7f9f1d310700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f18159890 0x7f9f180ceb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49556/0 (socket says 172.21.15.89:49556) 2023-10-10T20:41:12.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.792+0000 7f9f1d310700 1 -- 172.21.15.89:0/4261547379 learned_addr learned my addr 172.21.15.89:0/4261547379 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:12.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.792+0000 7f9f1db11700 1 --2- 172.21.15.89:0/4261547379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f1815db10 0x7f9f180b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.792+0000 7f9f1cb0f700 1 --2- 172.21.15.89:0/4261547379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f18159c60 0x7f9f180c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.792+0000 7f9f1d310700 1 -- 172.21.15.89:0/4261547379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f18159c60 msgr2=0x7f9f180c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:12.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.792+0000 7f9f1d310700 1 --2- 172.21.15.89:0/4261547379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f18159c60 0x7f9f180c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.792+0000 7f9f1d310700 1 -- 172.21.15.89:0/4261547379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f1815db10 msgr2=0x7f9f180b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:12.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.792+0000 7f9f1d310700 1 --2- 172.21.15.89:0/4261547379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f1815db10 0x7f9f180b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.792+0000 7f9f1d310700 1 -- 172.21.15.89:0/4261547379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f100094f0 con 0x7f9f18159890 2023-10-10T20:41:12.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.792+0000 7f9f1db11700 1 --2- 172.21.15.89:0/4261547379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f1815db10 0x7f9f180b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:12.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.792+0000 7f9f1d310700 1 --2- 172.21.15.89:0/4261547379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f18159890 0x7f9f180ceb40 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400c950 tx=0x7f9f14002760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:12.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.793+0000 7f9f0d7fa700 1 -- 172.21.15.89:0/4261547379 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1400a7e0 con 0x7f9f18159890 2023-10-10T20:41:12.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.793+0000 7f9f0d7fa700 1 -- 172.21.15.89:0/4261547379 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1400a940 con 0x7f9f18159890 2023-10-10T20:41:12.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.793+0000 7f9f1f574700 1 -- 172.21.15.89:0/4261547379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f180ce640 con 0x7f9f18159890 2023-10-10T20:41:12.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.793+0000 7f9f0d7fa700 1 -- 172.21.15.89:0/4261547379 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1400ad00 con 0x7f9f18159890 2023-10-10T20:41:12.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.793+0000 7f9f1f574700 1 -- 172.21.15.89:0/4261547379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f180b2050 con 0x7f9f18159890 2023-10-10T20:41:12.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.793+0000 7f9f1f574700 1 -- 172.21.15.89:0/4261547379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f180ce9b0 con 0x7f9f18159890 2023-10-10T20:41:12.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.794+0000 7f9f0d7fa700 1 -- 172.21.15.89:0/4261547379 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9f14012630 con 0x7f9f18159890 2023-10-10T20:41:12.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.794+0000 7f9f0d7fa700 1 --2- 172.21.15.89:0/4261547379 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9efc067320 0x7f9efc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:12.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.794+0000 7f9f0d7fa700 1 -- 172.21.15.89:0/4261547379 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9f140851d0 con 0x7f9f18159890 2023-10-10T20:41:12.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.794+0000 7f9f1cb0f700 1 --2- 172.21.15.89:0/4261547379 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9efc067320 0x7f9efc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:12.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.795+0000 7f9f1cb0f700 1 --2- 172.21.15.89:0/4261547379 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9efc067320 0x7f9efc0697d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9f1815a090 tx=0x7f9f08006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:12.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.799+0000 7f9f0d7fa700 1 -- 172.21.15.89:0/4261547379 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f14051f10 con 0x7f9f18159890 2023-10-10T20:41:12.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.977+0000 7f9f1f574700 1 -- 172.21.15.89:0/4261547379 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9f1815aec0 con 0x7f9efc067320 2023-10-10T20:41:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.984+0000 7f9f0d7fa700 1 -- 172.21.15.89:0/4261547379 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287184 (secure 0 0 0) 0x7f9f1815aec0 con 0x7f9efc067320 2023-10-10T20:41:12.989 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:12.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.988+0000 7f9f1f574700 1 -- 172.21.15.89:0/4261547379 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9efc067320 msgr2=0x7f9efc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:12.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.988+0000 7f9f1f574700 1 --2- 172.21.15.89:0/4261547379 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9efc067320 0x7f9efc0697d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9f1815a090 tx=0x7f9f08006f90 comp rx=0 tx=0).stop 2023-10-10T20:41:12.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.988+0000 7f9f1f574700 1 -- 172.21.15.89:0/4261547379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f18159890 msgr2=0x7f9f180ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:12.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.988+0000 7f9f1f574700 1 --2- 172.21.15.89:0/4261547379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f18159890 0x7f9f180ceb40 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400c950 tx=0x7f9f14002760 comp rx=0 tx=0).stop 2023-10-10T20:41:12.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.988+0000 7f9f1f574700 1 -- 172.21.15.89:0/4261547379 shutdown_connections 2023-10-10T20:41:12.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.988+0000 7f9f1f574700 1 --2- 172.21.15.89:0/4261547379 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9efc067320 0x7f9efc0697d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.988+0000 7f9f1f574700 1 --2- 172.21.15.89:0/4261547379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f1815db10 0x7f9f180b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.988+0000 7f9f1f574700 1 --2- 172.21.15.89:0/4261547379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f18159890 0x7f9f180ceb40 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.988+0000 7f9f1f574700 1 --2- 172.21.15.89:0/4261547379 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9f18159c60 0x7f9f180c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:12.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.988+0000 7f9f1f574700 1 -- 172.21.15.89:0/4261547379 >> 172.21.15.89:0/4261547379 conn(0x7f9f18068e20 msgr2=0x7f9f180cb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:12.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.988+0000 7f9f1f574700 1 -- 172.21.15.89:0/4261547379 shutdown_connections 2023-10-10T20:41:12.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:12.988+0000 7f9f1f574700 1 -- 172.21.15.89:0/4261547379 wait complete. 2023-10-10T20:41:12.994 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:41:13.002 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2023-10-10T20:41:12.731370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422956,"kb_used_data":11884,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752551764,"statfs":{"total":773094113280,"available":770613006336,"internally_reserved":0,"allocated":12169216,"data_stored":5911081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001097"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.185044+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:43:38.336996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179076+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:42:10.326290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183633+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:31:04.342810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184156+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:29:07.750059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183700+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:06:14.543497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180424+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:15:09.117651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180699+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:43:54.668141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.185584+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:51.630860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182871+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:00.815725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178630+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:45:17.413763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.176188+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:28:49.571035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182500+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:27:02.185210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184448+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:23.971915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.177799+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:35.431888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183897+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:08.100006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182045+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:51.136287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181388+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:51:14.573623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182606+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:21:00.592871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.175771+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:29:15.059073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182998+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:35:58.896847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182721+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:57.837359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.175675+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:54:47.622590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184138+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:56.592906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.175733+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:04.315349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178752+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:39:12.818520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181110+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:36:04.128348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183202+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:24.718214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179120+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:57:20.319921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.185431+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:38:00.081732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183275+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:09.031752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.186844+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:54.943577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179704+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:51.630860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180587+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:06:14.543497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194387+0000","last_change":"2023-10-10T20:41:07.194387+0000","last_active":"2023-10-10T20:41:07.194387+0000","last_peered":"2023-10-10T20:41:07.194387+0000","last_clean":"2023-10-10T20:41:07.194387+0000","last_became_active":"2023-10-10T20:41:07.194097+0000","last_became_peered":"2023-10-10T20:41:07.194097+0000","last_unstale":"2023-10-10T20:41:07.194387+0000","last_undegraded":"2023-10-10T20:41:07.194387+0000","last_fullsized":"2023-10-10T20:41:07.194387+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:26.546168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182362+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:37:12.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192372+0000","last_change":"2023-10-10T20:41:07.192372+0000","last_active":"2023-10-10T20:41:07.192372+0000","last_peered":"2023-10-10T20:41:07.192372+0000","last_clean":"2023-10-10T20:41:07.192372+0000","last_became_active":"2023-10-10T20:41:07.192088+0000","last_became_peered":"2023-10-10T20:41:07.192088+0000","last_unstale":"2023-10-10T20:41:07.192372+0000","last_undegraded":"2023-10-10T20:41:07.192372+0000","last_fullsized":"2023-10-10T20:41:07.192372+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:59.684638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181983+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:01.392782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.187579+0000","last_change":"2023-10-10T20:41:07.187579+0000","last_active":"2023-10-10T20:41:07.187579+0000","last_peered":"2023-10-10T20:41:07.187579+0000","last_clean":"2023-10-10T20:41:07.187579+0000","last_became_active":"2023-10-10T20:41:07.187296+0000","last_became_peered":"2023-10-10T20:41:07.187296+0000","last_unstale":"2023-10-10T20:41:07.187579+0000","last_undegraded":"2023-10-10T20:41:07.187579+0000","last_fullsized":"2023-10-10T20:41:07.187579+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:18.852267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182717+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:56:26.100041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194195+0000","last_change":"2023-10-10T20:41:07.194195+0000","last_active":"2023-10-10T20:41:07.194195+0000","last_peered":"2023-10-10T20:41:07.194195+0000","last_clean":"2023-10-10T20:41:07.194195+0000","last_became_active":"2023-10-10T20:41:07.193920+0000","last_became_peered":"2023-10-10T20:41:07.193920+0000","last_unstale":"2023-10-10T20:41:07.194195+0000","last_undegraded":"2023-10-10T20:41:07.194195+0000","last_fullsized":"2023-10-10T20:41:07.194195+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:36.588822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182202+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:40:14.372905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.193266+0000","last_change":"2023-10-10T20:41:07.193266+0000","last_active":"2023-10-10T20:41:07.193266+0000","last_peered":"2023-10-10T20:41:07.193266+0000","last_clean":"2023-10-10T20:41:07.193266+0000","last_became_active":"2023-10-10T20:41:07.192991+0000","last_became_peered":"2023-10-10T20:41:07.192991+0000","last_unstale":"2023-10-10T20:41:07.193266+0000","last_undegraded":"2023-10-10T20:41:07.193266+0000","last_fullsized":"2023-10-10T20:41:07.193266+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:15.545053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182281+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:51:25.089205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.193695+0000","last_change":"2023-10-10T20:41:07.193695+0000","last_active":"2023-10-10T20:41:07.193695+0000","last_peered":"2023-10-10T20:41:07.193695+0000","last_clean":"2023-10-10T20:41:07.193695+0000","last_became_active":"2023-10-10T20:41:07.193394+0000","last_became_peered":"2023-10-10T20:41:07.193394+0000","last_unstale":"2023-10-10T20:41:07.193695+0000","last_undegraded":"2023-10-10T20:41:07.193695+0000","last_fullsized":"2023-10-10T20:41:07.193695+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:56.835912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.186003+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:44.120514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.193880+0000","last_change":"2023-10-10T20:41:07.193880+0000","last_active":"2023-10-10T20:41:07.193880+0000","last_peered":"2023-10-10T20:41:07.193880+0000","last_clean":"2023-10-10T20:41:07.193880+0000","last_became_active":"2023-10-10T20:41:07.193586+0000","last_became_peered":"2023-10-10T20:41:07.193586+0000","last_unstale":"2023-10-10T20:41:07.193880+0000","last_undegraded":"2023-10-10T20:41:07.193880+0000","last_fullsized":"2023-10-10T20:41:07.193880+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:15:36.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178163+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:13:00.611054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194069+0000","last_change":"2023-10-10T20:41:07.194069+0000","last_active":"2023-10-10T20:41:07.194069+0000","last_peered":"2023-10-10T20:41:07.194069+0000","last_clean":"2023-10-10T20:41:07.194069+0000","last_became_active":"2023-10-10T20:41:07.193800+0000","last_became_peered":"2023-10-10T20:41:07.193800+0000","last_unstale":"2023-10-10T20:41:07.194069+0000","last_undegraded":"2023-10-10T20:41:07.194069+0000","last_fullsized":"2023-10-10T20:41:07.194069+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:00:42.940622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183477+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:59:58.659044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.190099+0000","last_change":"2023-10-10T20:41:07.190099+0000","last_active":"2023-10-10T20:41:07.190099+0000","last_peered":"2023-10-10T20:41:07.190099+0000","last_clean":"2023-10-10T20:41:07.190099+0000","last_became_active":"2023-10-10T20:41:07.189840+0000","last_became_peered":"2023-10-10T20:41:07.189840+0000","last_unstale":"2023-10-10T20:41:07.190099+0000","last_undegraded":"2023-10-10T20:41:07.190099+0000","last_fullsized":"2023-10-10T20:41:07.190099+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:48:33.094041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178595+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:35.431888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.187415+0000","last_change":"2023-10-10T20:41:07.187415+0000","last_active":"2023-10-10T20:41:07.187415+0000","last_peered":"2023-10-10T20:41:07.187415+0000","last_clean":"2023-10-10T20:41:07.187415+0000","last_became_active":"2023-10-10T20:41:07.187114+0000","last_became_peered":"2023-10-10T20:41:07.187114+0000","last_unstale":"2023-10-10T20:41:07.187415+0000","last_undegraded":"2023-10-10T20:41:07.187415+0000","last_fullsized":"2023-10-10T20:41:07.187415+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:58:20.063048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181497+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:42:34.466499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.188746+0000","last_change":"2023-10-10T20:41:07.188746+0000","last_active":"2023-10-10T20:41:07.188746+0000","last_peered":"2023-10-10T20:41:07.188746+0000","last_clean":"2023-10-10T20:41:07.188746+0000","last_became_active":"2023-10-10T20:41:07.188486+0000","last_became_peered":"2023-10-10T20:41:07.188486+0000","last_unstale":"2023-10-10T20:41:07.188746+0000","last_undegraded":"2023-10-10T20:41:07.188746+0000","last_fullsized":"2023-10-10T20:41:07.188746+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:36.588822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182276+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:04:10.686499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.191180+0000","last_change":"2023-10-10T20:41:07.191180+0000","last_active":"2023-10-10T20:41:07.191180+0000","last_peered":"2023-10-10T20:41:07.191180+0000","last_clean":"2023-10-10T20:41:07.191180+0000","last_became_active":"2023-10-10T20:41:07.190910+0000","last_became_peered":"2023-10-10T20:41:07.190910+0000","last_unstale":"2023-10-10T20:41:07.191180+0000","last_undegraded":"2023-10-10T20:41:07.191180+0000","last_fullsized":"2023-10-10T20:41:07.191180+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:13.928318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184946+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:43.974985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.189943+0000","last_change":"2023-10-10T20:41:07.189943+0000","last_active":"2023-10-10T20:41:07.189943+0000","last_peered":"2023-10-10T20:41:07.189943+0000","last_clean":"2023-10-10T20:41:07.189943+0000","last_became_active":"2023-10-10T20:41:07.189639+0000","last_became_peered":"2023-10-10T20:41:07.189639+0000","last_unstale":"2023-10-10T20:41:07.189943+0000","last_undegraded":"2023-10-10T20:41:07.189943+0000","last_fullsized":"2023-10-10T20:41:07.189943+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:24:45.124009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178350+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:07:36.012741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.193331+0000","last_change":"2023-10-10T20:41:07.193331+0000","last_active":"2023-10-10T20:41:07.193331+0000","last_peered":"2023-10-10T20:41:07.193331+0000","last_clean":"2023-10-10T20:41:07.193331+0000","last_became_active":"2023-10-10T20:41:07.193077+0000","last_became_peered":"2023-10-10T20:41:07.193077+0000","last_unstale":"2023-10-10T20:41:07.193331+0000","last_undegraded":"2023-10-10T20:41:07.193331+0000","last_fullsized":"2023-10-10T20:41:07.193331+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:56.816260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182401+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:47:38.801771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.190581+0000","last_change":"2023-10-10T20:41:07.190581+0000","last_active":"2023-10-10T20:41:07.190581+0000","last_peered":"2023-10-10T20:41:07.190581+0000","last_clean":"2023-10-10T20:41:07.190581+0000","last_became_active":"2023-10-10T20:41:07.190319+0000","last_became_peered":"2023-10-10T20:41:07.190319+0000","last_unstale":"2023-10-10T20:41:07.190581+0000","last_undegraded":"2023-10-10T20:41:07.190581+0000","last_fullsized":"2023-10-10T20:41:07.190581+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:42:09.324842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183200+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:15:37.860366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.191241+0000","last_change":"2023-10-10T20:41:07.191241+0000","last_active":"2023-10-10T20:41:07.191241+0000","last_peered":"2023-10-10T20:41:07.191241+0000","last_clean":"2023-10-10T20:41:07.191241+0000","last_became_active":"2023-10-10T20:41:07.191020+0000","last_became_peered":"2023-10-10T20:41:07.191020+0000","last_unstale":"2023-10-10T20:41:07.191241+0000","last_undegraded":"2023-10-10T20:41:07.191241+0000","last_fullsized":"2023-10-10T20:41:07.191241+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:56.816260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180046+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:07:36.012741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.187395+0000","last_change":"2023-10-10T20:41:07.187395+0000","last_active":"2023-10-10T20:41:07.187395+0000","last_peered":"2023-10-10T20:41:07.187395+0000","last_clean":"2023-10-10T20:41:07.187395+0000","last_became_active":"2023-10-10T20:41:07.187130+0000","last_became_peered":"2023-10-10T20:41:07.187130+0000","last_unstale":"2023-10-10T20:41:07.187395+0000","last_undegraded":"2023-10-10T20:41:07.187395+0000","last_fullsized":"2023-10-10T20:41:07.187395+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:32:03.550923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181825+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:54.943577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.190127+0000","last_change":"2023-10-10T20:41:07.190127+0000","last_active":"2023-10-10T20:41:07.190127+0000","last_peered":"2023-10-10T20:41:07.190127+0000","last_clean":"2023-10-10T20:41:07.190127+0000","last_became_active":"2023-10-10T20:41:07.189866+0000","last_became_peered":"2023-10-10T20:41:07.189866+0000","last_unstale":"2023-10-10T20:41:07.190127+0000","last_undegraded":"2023-10-10T20:41:07.190127+0000","last_fullsized":"2023-10-10T20:41:07.190127+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:31:03.341362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.177885+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:57:20.319921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192468+0000","last_change":"2023-10-10T20:41:07.192468+0000","last_active":"2023-10-10T20:41:07.192468+0000","last_peered":"2023-10-10T20:41:07.192468+0000","last_clean":"2023-10-10T20:41:07.192468+0000","last_became_active":"2023-10-10T20:41:07.192177+0000","last_became_peered":"2023-10-10T20:41:07.192177+0000","last_unstale":"2023-10-10T20:41:07.192468+0000","last_undegraded":"2023-10-10T20:41:07.192468+0000","last_fullsized":"2023-10-10T20:41:07.192468+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:15.545053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183818+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:53.393618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.188588+0000","last_change":"2023-10-10T20:41:07.188588+0000","last_active":"2023-10-10T20:41:07.188588+0000","last_peered":"2023-10-10T20:41:07.188588+0000","last_clean":"2023-10-10T20:41:07.188588+0000","last_became_active":"2023-10-10T20:41:07.188209+0000","last_became_peered":"2023-10-10T20:41:07.188209+0000","last_unstale":"2023-10-10T20:41:07.188588+0000","last_undegraded":"2023-10-10T20:41:07.188588+0000","last_fullsized":"2023-10-10T20:41:07.188588+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:00:42.940622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180348+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:44.120514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194274+0000","last_change":"2023-10-10T20:41:07.194274+0000","last_active":"2023-10-10T20:41:07.194274+0000","last_peered":"2023-10-10T20:41:07.194274+0000","last_clean":"2023-10-10T20:41:07.194274+0000","last_became_active":"2023-10-10T20:41:07.193987+0000","last_became_peered":"2023-10-10T20:41:07.193987+0000","last_unstale":"2023-10-10T20:41:07.194274+0000","last_undegraded":"2023-10-10T20:41:07.194274+0000","last_fullsized":"2023-10-10T20:41:07.194274+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:17.174004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179361+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:14.929765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194646+0000","last_change":"2023-10-10T20:41:07.194646+0000","last_active":"2023-10-10T20:41:07.194646+0000","last_peered":"2023-10-10T20:41:07.194646+0000","last_clean":"2023-10-10T20:41:07.194646+0000","last_became_active":"2023-10-10T20:41:07.194426+0000","last_became_peered":"2023-10-10T20:41:07.194426+0000","last_unstale":"2023-10-10T20:41:07.194646+0000","last_undegraded":"2023-10-10T20:41:07.194646+0000","last_fullsized":"2023-10-10T20:41:07.194646+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:00.391334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181308+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:47:27.526950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192740+0000","last_change":"2023-10-10T20:41:07.192740+0000","last_active":"2023-10-10T20:41:07.192740+0000","last_peered":"2023-10-10T20:41:07.192740+0000","last_clean":"2023-10-10T20:41:07.192740+0000","last_became_active":"2023-10-10T20:41:07.192480+0000","last_became_peered":"2023-10-10T20:41:07.192480+0000","last_unstale":"2023-10-10T20:41:07.192740+0000","last_undegraded":"2023-10-10T20:41:07.192740+0000","last_fullsized":"2023-10-10T20:41:07.192740+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:48.161107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181783+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:27.547616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194467+0000","last_change":"2023-10-10T20:41:07.194467+0000","last_active":"2023-10-10T20:41:07.194467+0000","last_peered":"2023-10-10T20:41:07.194467+0000","last_clean":"2023-10-10T20:41:07.194467+0000","last_became_active":"2023-10-10T20:41:07.193987+0000","last_became_peered":"2023-10-10T20:41:07.193987+0000","last_unstale":"2023-10-10T20:41:07.194467+0000","last_undegraded":"2023-10-10T20:41:07.194467+0000","last_fullsized":"2023-10-10T20:41:07.194467+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:43.862933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184078+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:38:30.611281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.196552+0000","last_change":"2023-10-10T20:41:07.196552+0000","last_active":"2023-10-10T20:41:07.196552+0000","last_peered":"2023-10-10T20:41:07.196552+0000","last_clean":"2023-10-10T20:41:07.196552+0000","last_became_active":"2023-10-10T20:41:07.196285+0000","last_became_peered":"2023-10-10T20:41:07.196285+0000","last_unstale":"2023-10-10T20:41:07.196552+0000","last_undegraded":"2023-10-10T20:41:07.196552+0000","last_fullsized":"2023-10-10T20:41:07.196552+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:43:53.666694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178501+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:43.761996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192768+0000","last_change":"2023-10-10T20:41:07.192768+0000","last_active":"2023-10-10T20:41:07.192768+0000","last_peered":"2023-10-10T20:41:07.192768+0000","last_clean":"2023-10-10T20:41:07.192768+0000","last_became_active":"2023-10-10T20:41:07.192467+0000","last_became_peered":"2023-10-10T20:41:07.192467+0000","last_unstale":"2023-10-10T20:41:07.192768+0000","last_undegraded":"2023-10-10T20:41:07.192768+0000","last_fullsized":"2023-10-10T20:41:07.192768+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:59.684638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182731+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:47:27.526950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.186703+0000","last_change":"2023-10-10T20:41:07.186703+0000","last_active":"2023-10-10T20:41:07.186703+0000","last_peered":"2023-10-10T20:41:07.186703+0000","last_clean":"2023-10-10T20:41:07.186703+0000","last_became_active":"2023-10-10T20:41:07.186447+0000","last_became_peered":"2023-10-10T20:41:07.186447+0000","last_unstale":"2023-10-10T20:41:07.186703+0000","last_undegraded":"2023-10-10T20:41:07.186703+0000","last_fullsized":"2023-10-10T20:41:07.186703+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:57.197462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181353+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:58:21.064495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192864+0000","last_change":"2023-10-10T20:41:07.192864+0000","last_active":"2023-10-10T20:41:07.192864+0000","last_peered":"2023-10-10T20:41:07.192864+0000","last_clean":"2023-10-10T20:41:07.192864+0000","last_became_active":"2023-10-10T20:41:07.192602+0000","last_became_peered":"2023-10-10T20:41:07.192602+0000","last_unstale":"2023-10-10T20:41:07.192864+0000","last_undegraded":"2023-10-10T20:41:07.192864+0000","last_fullsized":"2023-10-10T20:41:07.192864+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:37:19.121105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179266+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:13:00.611054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192698+0000","last_change":"2023-10-10T20:41:07.192698+0000","last_active":"2023-10-10T20:41:07.192698+0000","last_peered":"2023-10-10T20:41:07.192698+0000","last_clean":"2023-10-10T20:41:07.192698+0000","last_became_active":"2023-10-10T20:41:07.192370+0000","last_became_peered":"2023-10-10T20:41:07.192370+0000","last_unstale":"2023-10-10T20:41:07.192698+0000","last_undegraded":"2023-10-10T20:41:07.192698+0000","last_fullsized":"2023-10-10T20:41:07.192698+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:15:08.116204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180520+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:56:26.100041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194285+0000","last_change":"2023-10-10T20:41:07.194285+0000","last_active":"2023-10-10T20:41:07.194285+0000","last_peered":"2023-10-10T20:41:07.194285+0000","last_clean":"2023-10-10T20:41:07.194285+0000","last_became_active":"2023-10-10T20:41:07.194002+0000","last_became_peered":"2023-10-10T20:41:07.194002+0000","last_unstale":"2023-10-10T20:41:07.194285+0000","last_undegraded":"2023-10-10T20:41:07.194285+0000","last_fullsized":"2023-10-10T20:41:07.194285+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:42:33.465052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179142+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:56:52.494512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.186005+0000","last_change":"2023-10-10T20:41:07.186005+0000","last_active":"2023-10-10T20:41:07.186005+0000","last_peered":"2023-10-10T20:41:07.186005+0000","last_clean":"2023-10-10T20:41:07.186005+0000","last_became_active":"2023-10-10T20:41:07.185723+0000","last_became_peered":"2023-10-10T20:41:07.185723+0000","last_unstale":"2023-10-10T20:41:07.186005+0000","last_undegraded":"2023-10-10T20:41:07.186005+0000","last_fullsized":"2023-10-10T20:41:07.186005+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:39.932733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181885+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:18.175451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.195197+0000","last_change":"2023-10-10T20:41:07.195197+0000","last_active":"2023-10-10T20:41:07.195197+0000","last_peered":"2023-10-10T20:41:07.195197+0000","last_clean":"2023-10-10T20:41:07.195197+0000","last_became_active":"2023-10-10T20:41:07.194893+0000","last_became_peered":"2023-10-10T20:41:07.194893+0000","last_unstale":"2023-10-10T20:41:07.195197+0000","last_undegraded":"2023-10-10T20:41:07.195197+0000","last_fullsized":"2023-10-10T20:41:07.195197+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:51:13.572175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":253,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-10-10T20:41:06.176359+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:41:06.176359+0000","last_peered":"2023-10-10T20:41:06.176359+0000","last_clean":"2023-10-10T20:41:06.176359+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:41:06.176359+0000","last_undegraded":"2023-10-10T20:41:06.176359+0000","last_fullsized":"2023-10-10T20:41:06.176359+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303308,"kb_used_data":1924,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068532,"statfs":{"total":96636764160,"available":96326176768,"internally_reserved":0,"allocated":1970176,"data_stored":1183347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51500000000000001,"15min":0.81999999999999995},"min":{"1min":0.31,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53000000000000003,"15min":0.627},"min":{"1min":0.32200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71999999999999997,"5min":0.95399999999999996,"15min":4.1959999999999997},"last":0.54000000000000004}]},{"osd":1,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.504,"15min":1},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.67800000000000005,"5min":0.78800000000000003,"15min":10.1},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55800000000000005,"15min":0.72999999999999998},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.28999999999999998},"max":{"1min":0.81799999999999995,"5min":1.006,"15min":6.806},"last":0.65300000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57899999999999996,"15min":0.78100000000000003},"min":{"1min":0.43099999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89000000000000001,"5min":1.089,"15min":6.9340000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53200000000000003,"15min":0.73799999999999999},"min":{"1min":0.39000000000000001,"5min":0.36399999999999999,"15min":0.30099999999999999},"max":{"1min":0.77100000000000002,"5min":0.85299999999999998,"15min":6.9870000000000001},"last":0.46899999999999997}]},{"osd":3,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.75800000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.754,"5min":0.80200000000000005,"15min":6.6719999999999997},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.67000000000000004},"min":{"1min":0.38700000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.70599999999999996,"5min":1.2789999999999999,"15min":4.1609999999999996},"last":0.60099999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47399999999999998,"15min":0.46500000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.73199999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.621},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52400000000000002,"15min":0.52300000000000002},"min":{"1min":0.309,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.55200000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.32500000000000001,"5min":0.32100000000000001,"15min":0.314},"max":{"1min":0.86099999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75800000000000001,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.68899999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72999999999999998,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84099999999999997,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.66500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302728,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069112,"statfs":{"total":96636764160,"available":96326770688,"internally_reserved":0,"allocated":1376256,"data_stored":592979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50700000000000001,"15min":0.67200000000000004},"min":{"1min":0.33500000000000002,"5min":0.32400000000000001,"15min":0.312},"max":{"1min":0.68600000000000005,"5min":0.73099999999999998,"15min":7.7590000000000003},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52100000000000002,"15min":0.71299999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":7.8090000000000002},"last":0.52400000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.70999999999999996},"min":{"1min":0.38400000000000001,"5min":0.35599999999999998,"15min":0.27800000000000002},"max":{"1min":0.628,"5min":0.80000000000000004,"15min":7.867},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53300000000000003,"15min":0.88500000000000001},"min":{"1min":0.36599999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":9.7089999999999996},"last":0.54400000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51400000000000001,"15min":0.59599999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.72499999999999998,"5min":0.747,"15min":4.7999999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54600000000000004,"15min":0.84499999999999997},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.71499999999999997,"5min":0.80000000000000004,"15min":6.7629999999999999},"last":0.63}]},{"osd":3,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.62},"min":{"1min":0.40999999999999998,"5min":0.36299999999999999,"15min":0.35299999999999998},"max":{"1min":0.79100000000000004,"5min":0.83999999999999997,"15min":4.8620000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54800000000000004,"15min":0.71699999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.35299999999999998},"max":{"1min":0.68600000000000005,"5min":1.1850000000000001,"15min":7.9009999999999998},"last":0.64900000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51500000000000001,"15min":0.46800000000000003},"min":{"1min":0.32400000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71099999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59199999999999997,"15min":0.56799999999999995},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.37},"max":{"1min":0.73499999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.49099999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52000000000000002,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.314,"15min":0.312},"max":{"1min":0.69199999999999995,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.747,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.433}]},{"osd":7,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55100000000000005,"15min":0.52000000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.72599999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57799999999999996,"15min":0.54400000000000004},"min":{"1min":0.42899999999999999,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":0.69399999999999995,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.47499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302716,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069124,"statfs":{"total":96636764160,"available":96326782976,"internally_reserved":0,"allocated":1363968,"data_stored":586716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.31900000000000001,"15min":0.47599999999999998},"min":{"1min":0.20999999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.441,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.32300000000000001,"15min":0.36299999999999999},"min":{"1min":0.23599999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46000000000000002,"5min":0.68700000000000006,"15min":1.194},"last":0.308}]},{"osd":2,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.32400000000000001,"15min":0.36399999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.379,"5min":0.67400000000000004,"15min":1.2869999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.33800000000000002,"15min":0.505},"min":{"1min":0.26000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.54200000000000004,"5min":0.57199999999999995,"15min":5.0919999999999996},"last":0.51300000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36299999999999999,"15min":0.52800000000000002},"min":{"1min":0.26800000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.51000000000000001,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36499999999999999,"15min":0.40699999999999997},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.48299999999999998,"5min":0.72099999999999997,"15min":1.2170000000000001},"last":0.41399999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57899999999999996,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.81000000000000005,"5min":1.133,"15min":1.254},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56599999999999995,"15min":0.70599999999999996},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69899999999999995,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.67100000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57899999999999996,"15min":0.59499999999999997},"min":{"1min":0.42299999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.751,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.499},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.60799999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71899999999999997,"5min":1.0720000000000001,"15min":1.3109999999999999},"last":0.55700000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60699999999999998,"15min":0.84399999999999997},"min":{"1min":0.435,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86199999999999999,"5min":0.995,"15min":5.0659999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61699999999999999,"15min":0.63200000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84599999999999997,"5min":1.1439999999999999,"15min":1.171},"last":0.58899999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59299999999999997,"15min":0.61699999999999999},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.753,"5min":1.214,"15min":1.214},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60099999999999998,"15min":0.72499999999999998},"min":{"1min":0.45000000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76900000000000002,"5min":1.115,"15min":4.0499999999999998},"last":0.625}]}]},{"osd":0,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303304,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068536,"statfs":{"total":96636764160,"available":96326180864,"internally_reserved":0,"allocated":1966080,"data_stored":1182386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.309,"15min":0.52800000000000002},"min":{"1min":0.20999999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.38300000000000001,"5min":0.51600000000000001,"15min":7.9249999999999998},"last":0.371},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.318,"15min":0.505},"min":{"1min":0.23699999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.41899999999999998,"5min":0.54900000000000004,"15min":6.6299999999999999},"last":0.255}]},{"osd":2,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.32600000000000001,"15min":0.55200000000000005},"min":{"1min":0.22,"5min":0.216,"15min":0.216},"max":{"1min":0.48799999999999999,"5min":0.55400000000000005,"15min":8.2509999999999994},"last":0.313},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.34000000000000002,"15min":0.52800000000000002},"min":{"1min":0.23100000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.45700000000000002,"5min":0.54100000000000004,"15min":6.7370000000000001},"last":0.32800000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.34399999999999997,"15min":0.55700000000000005},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.48299999999999998,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.61199999999999999},"min":{"1min":0.32000000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":7.8929999999999998},"last":0.41599999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55600000000000005,"15min":0.71399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.72499999999999998,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.748},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56299999999999994,"15min":0.77200000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.67600000000000005,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.71099999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.55200000000000005,"15min":0.751},"min":{"1min":0.38800000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69299999999999995,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58099999999999996,"15min":0.77800000000000002},"min":{"1min":0.42299999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77000000000000002,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.75700000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60199999999999998,"15min":0.81799999999999995},"min":{"1min":0.39800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72099999999999997,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61099999999999999,"15min":0.76100000000000001},"min":{"1min":0.41399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74299999999999999,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58799999999999997,"15min":0.752},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78300000000000003,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59099999999999997,"15min":0.78600000000000003},"min":{"1min":0.45200000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.69299999999999995,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.60299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302728,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069112,"statfs":{"total":96636764160,"available":96326770688,"internally_reserved":0,"allocated":1376256,"data_stored":592979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.28299999999999997,"5min":0.309,"15min":0.32500000000000001},"min":{"1min":0.21099999999999999,"5min":0.189,"15min":0.189},"max":{"1min":0.38300000000000001,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.29399999999999998,"5min":0.32100000000000001,"15min":0.35299999999999998},"min":{"1min":0.215,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.39500000000000002,"5min":0.5,"15min":0.746},"last":0.29799999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.32900000000000001,"15min":0.72899999999999998},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.35699999999999998,"5min":0.63,"15min":11.967000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.34499999999999997,"15min":0.48299999999999998},"min":{"1min":0.26800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.443,"5min":0.53400000000000003,"15min":4.1100000000000003},"last":0.36499999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.34499999999999997,"15min":0.55700000000000005},"min":{"1min":0.23699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.39400000000000002,"5min":0.55400000000000005,"15min":6.3040000000000003},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38700000000000001,"15min":0.78100000000000003},"min":{"1min":0.27900000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.46899999999999997,"5min":0.65900000000000003,"15min":11.935},"last":0.35199999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.54300000000000004,"15min":0.55500000000000005},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.58699999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71699999999999997,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.58599999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.378},"max":{"1min":0.66300000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54400000000000004,"15min":0.58399999999999996},"min":{"1min":0.39700000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74199999999999999,"5min":0.874,"15min":0.96099999999999997},"last":0.54100000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57699999999999996,"15min":0.60099999999999998},"min":{"1min":0.42199999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.745,"5min":0.91600000000000004,"15min":0.91800000000000004},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73799999999999999,"5min":0.93100000000000005,"15min":0.93600000000000005},"last":0.55500000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56499999999999995,"15min":0.58499999999999996},"min":{"1min":0.41799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71099999999999997,"5min":0.89100000000000001,"15min":1.052},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57699999999999996,"15min":0.78000000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69799999999999995,"5min":0.85999999999999999,"15min":6.2649999999999997},"last":0.45900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302728,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069112,"statfs":{"total":96636764160,"available":96326770688,"internally_reserved":0,"allocated":1376256,"data_stored":592979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.31,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.31900000000000001},"min":{"1min":0.25,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.33600000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.23499999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.316},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38600000000000001,"15min":0.377},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.57099999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.40500000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.34999999999999998},"min":{"1min":0.23699999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.35699999999999998,"15min":0.36199999999999999},"min":{"1min":0.26300000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.35899999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69999999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.82299999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55300000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.35899999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55900000000000005,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.76900000000000002,"5min":1.01,"15min":1.01},"last":0.626}]},{"osd":6,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.42099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85699999999999998,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.60999999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.441,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.496,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.88400000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302716,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069124,"statfs":{"total":96636764160,"available":96326782976,"internally_reserved":0,"allocated":1363968,"data_stored":586716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68300000000000005,"5min":0.74199999999999999,"15min":0.747},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.76600000000000001},"min":{"1min":0.374,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67500000000000004,"5min":0.78400000000000003,"15min":7.6719999999999997},"last":0.439}]},{"osd":1,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52900000000000003,"15min":0.77400000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.755,"5min":0.755,"15min":7.7160000000000002},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65300000000000002,"5min":0.76900000000000002,"15min":0.86799999999999999},"last":0.66100000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.65300000000000002,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56299999999999994,"15min":0.79300000000000004},"min":{"1min":0.39700000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.72799999999999998,"5min":0.79800000000000004,"15min":7.7839999999999998},"last":0.68500000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55200000000000005,"15min":0.54900000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65500000000000003,"5min":0.73399999999999999,"15min":0.96999999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.41199999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.66100000000000003,"5min":0.78100000000000003,"15min":0.79400000000000004},"last":0.47099999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51000000000000001,"15min":0.48699999999999999},"min":{"1min":0.32500000000000001,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.77500000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54900000000000004,"15min":0.54300000000000004},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.55300000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.39200000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68799999999999994,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.35899999999999999},"max":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.378,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.84599999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.46100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.748,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.60899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302728,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069112,"statfs":{"total":96636764160,"available":96326770688,"internally_reserved":0,"allocated":1376256,"data_stored":592979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.28599999999999998,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.79300000000000004},"last":0.52400000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52700000000000002,"15min":0.64100000000000001},"min":{"1min":0.33000000000000002,"5min":0.32200000000000001,"15min":0.316},"max":{"1min":0.70399999999999996,"5min":1.2330000000000001,"15min":5.2679999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51800000000000002,"15min":0.629},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.47099999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52800000000000002,"15min":0.64800000000000002},"min":{"1min":0.35999999999999999,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.76600000000000001,"5min":0.86199999999999999,"15min":5.2960000000000003},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72899999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.45000000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55300000000000005,"15min":0.67700000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.73899999999999999,"5min":0.93000000000000005,"15min":5.3330000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.69099999999999995},"min":{"1min":0.39300000000000002,"5min":0.36899999999999999,"15min":0.27700000000000002},"max":{"1min":0.76000000000000001,"5min":0.94399999999999995,"15min":5.2850000000000001},"last":0.53900000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47799999999999998,"15min":0.496},"min":{"1min":0.311,"5min":0.29599999999999999,"15min":0.28699999999999998},"max":{"1min":0.72499999999999998,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34300000000000003,"15min":0.32200000000000001},"max":{"1min":0.70799999999999996,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.63800000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54100000000000004,"15min":0.54700000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.80100000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82499999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.624}]},{"osd":7,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.378,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68899999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60199999999999998,"15min":0.60099999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83399999999999996,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.61399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:41:13.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3f is not active+clean 2023-10-10T20:41:13.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.185044+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:43:38.336996+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:41:13.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3e is not active+clean 2023-10-10T20:41:13.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.179076+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:42:10.326290+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:41:13.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3d is not active+clean 2023-10-10T20:41:13.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.183633+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:31:04.342810+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:41:13.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3c is not active+clean 2023-10-10T20:41:13.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.184156+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:29:07.750059+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:41:13.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3b is not active+clean 2023-10-10T20:41:13.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.183700+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:06:14.543497+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:41:13.012 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3a is not active+clean 2023-10-10T20:41:13.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.180424+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:15:09.117651+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:41:13.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.39 is not active+clean 2023-10-10T20:41:13.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.180699+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:43:54.668141+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:41:13.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.38 is not active+clean 2023-10-10T20:41:13.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.185584+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:32:51.630860+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:41:13.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.37 is not active+clean 2023-10-10T20:41:13.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.182871+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:50:00.815725+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:41:13.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.36 is not active+clean 2023-10-10T20:41:13.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.178630+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:45:17.413763+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:41:13.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.35 is not active+clean 2023-10-10T20:41:13.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.176188+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:28:49.571035+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:41:13.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.34 is not active+clean 2023-10-10T20:41:13.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.182500+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:27:02.185210+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:41:13.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.33 is not active+clean 2023-10-10T20:41:13.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.184448+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:13:23.971915+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:41:13.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.32 is not active+clean 2023-10-10T20:41:13.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.177799+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:19:35.431888+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:41:13.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.31 is not active+clean 2023-10-10T20:41:13.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.183897+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:40:08.100006+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:41:13.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.30 is not active+clean 2023-10-10T20:41:13.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.182045+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:19:51.136287+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:41:13.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2f is not active+clean 2023-10-10T20:41:13.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.181388+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:51:14.573623+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:41:13.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2e is not active+clean 2023-10-10T20:41:13.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.182606+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:21:00.592871+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:41:13.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2d is not active+clean 2023-10-10T20:41:13.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.175771+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:29:15.059073+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:41:13.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2c is not active+clean 2023-10-10T20:41:13.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.182998+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:35:58.896847+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:41:13.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2b is not active+clean 2023-10-10T20:41:13.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.182721+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:17:57.837359+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:41:13.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2a is not active+clean 2023-10-10T20:41:13.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.175675+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:54:47.622590+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:41:13.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.29 is not active+clean 2023-10-10T20:41:13.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.184138+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:54:56.592906+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:41:13.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.28 is not active+clean 2023-10-10T20:41:13.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.175733+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:14:04.315349+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:41:13.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.27 is not active+clean 2023-10-10T20:41:13.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.178752+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:39:12.818520+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:41:13.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.26 is not active+clean 2023-10-10T20:41:13.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.181110+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:36:04.128348+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:41:13.030 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.25 is not active+clean 2023-10-10T20:41:13.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.183202+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:57:24.718214+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:41:13.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.24 is not active+clean 2023-10-10T20:41:13.031 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.179120+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:57:20.319921+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:41:13.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.23 is not active+clean 2023-10-10T20:41:13.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.185431+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:38:00.081732+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:41:13.032 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.22 is not active+clean 2023-10-10T20:41:13.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.183275+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:30:09.031752+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:41:13.033 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.21 is not active+clean 2023-10-10T20:41:13.033 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.186844+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:22:54.943577+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:41:13.034 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.20 is not active+clean 2023-10-10T20:41:13.034 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.179704+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:32:51.630860+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:41:13.035 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1e is not active+clean 2023-10-10T20:41:13.035 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.180587+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:06:14.543497+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:41:13.035 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1f is not active+clean 2023-10-10T20:41:13.036 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.182362+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:37:12.824750+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:41:13.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1c is not active+clean 2023-10-10T20:41:13.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.181983+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T20:50:01.392782+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:41:13.037 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1d is not active+clean 2023-10-10T20:41:13.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.182717+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:56:26.100041+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:41:13.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1a is not active+clean 2023-10-10T20:41:13.038 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.182202+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:40:14.372905+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:41:13.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1b is not active+clean 2023-10-10T20:41:13.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.182281+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:51:25.089205+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:41:13.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.18 is not active+clean 2023-10-10T20:41:13.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.186003+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:57:44.120514+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:41:13.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.19 is not active+clean 2023-10-10T20:41:13.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.178163+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:13:00.611054+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:41:13.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.16 is not active+clean 2023-10-10T20:41:13.041 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.183477+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:59:58.659044+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:41:13.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.17 is not active+clean 2023-10-10T20:41:13.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.178595+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:19:35.431888+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:41:13.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.14 is not active+clean 2023-10-10T20:41:13.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.181497+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T20:42:34.466499+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:41:13.043 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.15 is not active+clean 2023-10-10T20:41:13.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.182276+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:04:10.686499+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:41:13.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.13 is not active+clean 2023-10-10T20:41:13.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.184946+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:12:43.974985+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:41:13.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.10 is not active+clean 2023-10-10T20:41:13.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.178350+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:07:36.012741+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:41:13.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.11 is not active+clean 2023-10-10T20:41:13.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.182401+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:47:38.801771+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:41:13.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.e is not active+clean 2023-10-10T20:41:13.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.183200+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:15:37.860366+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:41:13.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.f is not active+clean 2023-10-10T20:41:13.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.180046+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:07:36.012741+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:41:13.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.c is not active+clean 2023-10-10T20:41:13.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.181825+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:22:54.943577+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:41:13.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.d is not active+clean 2023-10-10T20:41:13.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.177885+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:57:20.319921+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:41:13.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.a is not active+clean 2023-10-10T20:41:13.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.183818+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:49:53.393618+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:41:13.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.b is not active+clean 2023-10-10T20:41:13.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.180348+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:57:44.120514+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:41:13.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.8 is not active+clean 2023-10-10T20:41:13.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.179361+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:57:14.929765+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:41:13.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.9 is not active+clean 2023-10-10T20:41:13.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.181308+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T20:47:27.526950+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:41:13.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.6 is not active+clean 2023-10-10T20:41:13.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.181783+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:07:27.547616+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:41:13.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.7 is not active+clean 2023-10-10T20:41:13.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.184078+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:38:30.611281+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:41:13.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.4 is not active+clean 2023-10-10T20:41:13.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.178501+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:54:43.761996+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:41:13.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.5 is not active+clean 2023-10-10T20:41:13.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.182731+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T20:47:27.526950+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:41:13.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2 is not active+clean 2023-10-10T20:41:13.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.181353+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:58:21.064495+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:41:13.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3 is not active+clean 2023-10-10T20:41:13.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.179266+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:13:00.611054+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:41:13.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.0 is not active+clean 2023-10-10T20:41:13.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.180520+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:56:26.100041+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:41:13.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1 is not active+clean 2023-10-10T20:41:13.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.179142+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:56:52.494512+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:41:13.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.12 is not active+clean 2023-10-10T20:41:13.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:41:07.169206+0000', 'last_change': '2023-10-10T20:41:07.181885+0000', 'last_active': '2023-10-10T20:41:07.169206+0000', 'last_peered': '2023-10-10T20:41:07.169206+0000', 'last_clean': '2023-10-10T20:41:07.169206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:41:07.169206+0000', 'last_undegraded': '2023-10-10T20:41:07.169206+0000', 'last_fullsized': '2023-10-10T20:41:07.169206+0000', 'mapping_epoch': 110, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 110, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'last_clean_scrub_stamp': '2023-10-10T20:41:07.169206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:31:18.175451+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:41:13.060 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-10T20:41:13.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.173+0000 7f8d640b4700 1 Processor -- start 2023-10-10T20:41:13.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.173+0000 7f8d640b4700 1 -- start start 2023-10-10T20:41:13.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.174+0000 7f8d640b4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d5c0cd910 0x7f8d5c0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:13.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.174+0000 7f8d640b4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8d5c0b55a0 0x7f8d5c0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:13.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.174+0000 7f8d640b4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5c0b5b00 con 0x7f8d5c159ec0 2023-10-10T20:41:13.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.174+0000 7f8d640b4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d5c0b5da0 con 0x7f8d5c0b55a0 2023-10-10T20:41:13.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.174+0000 7f8d640b4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c0b6080 con 0x7f8d5c0cd910 2023-10-10T20:41:13.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.174+0000 7f8d62651700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8d5c0b55a0 0x7f8d5c0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:13.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.174+0000 7f8d61e50700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8d5c159ec0 0x7f8d5c0b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41364/0 (socket says 172.21.15.89:41364) 2023-10-10T20:41:13.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.174+0000 7f8d61e50700 1 -- 172.21.15.89:0/729071956 learned_addr learned my addr 172.21.15.89:0/729071956 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:13.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.174+0000 7f8d6164f700 1 --2- 172.21.15.89:0/729071956 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d5c0cd910 0x7f8d5c0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:13.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.174+0000 7f8d62651700 1 -- 172.21.15.89:0/729071956 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d5c0cd910 msgr2=0x7f8d5c0ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:13.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.174+0000 7f8d62651700 1 --2- 172.21.15.89:0/729071956 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d5c0cd910 0x7f8d5c0ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.174+0000 7f8d62651700 1 -- 172.21.15.89:0/729071956 >> v1:172.21.15.89:6789/0 conn(0x7f8d5c159ec0 legacy=0x7f8d5c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:13.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.174+0000 7f8d60e4e700 1 -- 172.21.15.89:0/729071956 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2566455137 0 0) 0x7f8d5c0b5b00 con 0x7f8d5c159ec0 2023-10-10T20:41:13.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.174+0000 7f8d62651700 1 -- 172.21.15.89:0/729071956 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5c0cc100 con 0x7f8d5c0b55a0 2023-10-10T20:41:13.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.174+0000 7f8d6164f700 1 --2- 172.21.15.89:0/729071956 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d5c0cd910 0x7f8d5c0ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:13.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.175+0000 7f8d62651700 1 --2- 172.21.15.89:0/729071956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d5c0b55a0 0x7f8d5c0ce880 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009e60 tx=0x7f8d54009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecdf2c80ac89ef6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:13.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.175+0000 7f8d60e4e700 1 -- 172.21.15.89:0/729071956 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5400e070 con 0x7f8d5c0b55a0 2023-10-10T20:41:13.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.175+0000 7f8d60e4e700 1 -- 172.21.15.89:0/729071956 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5400bc30 con 0x7f8d5c0b55a0 2023-10-10T20:41:13.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.175+0000 7f8d60e4e700 1 -- 172.21.15.89:0/729071956 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d54012430 con 0x7f8d5c0b55a0 2023-10-10T20:41:13.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.175+0000 7f8d640b4700 1 -- 172.21.15.89:0/729071956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d5c0b55a0 msgr2=0x7f8d5c0ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:13.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.175+0000 7f8d640b4700 1 --2- 172.21.15.89:0/729071956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d5c0b55a0 0x7f8d5c0ce880 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009e60 tx=0x7f8d54009780 comp rx=0 tx=0).stop 2023-10-10T20:41:13.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.175+0000 7f8d640b4700 1 -- 172.21.15.89:0/729071956 shutdown_connections 2023-10-10T20:41:13.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.175+0000 7f8d640b4700 1 --2- 172.21.15.89:0/729071956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d5c0b55a0 0x7f8d5c0ce880 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.175+0000 7f8d640b4700 1 --2- 172.21.15.89:0/729071956 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d5c0cd910 0x7f8d5c0ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.175+0000 7f8d640b4700 1 -- 172.21.15.89:0/729071956 >> 172.21.15.89:0/729071956 conn(0x7f8d5c0999f0 msgr2=0x7f8d5c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:13.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.175+0000 7f8d640b4700 1 -- 172.21.15.89:0/729071956 shutdown_connections 2023-10-10T20:41:13.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.175+0000 7f8d640b4700 1 -- 172.21.15.89:0/729071956 wait complete. 2023-10-10T20:41:13.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.176+0000 7f8d640b4700 1 Processor -- start 2023-10-10T20:41:13.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.176+0000 7f8d640b4700 1 -- start start 2023-10-10T20:41:13.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.177+0000 7f8d640b4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d5c0b55a0 0x7f8d5c09d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:13.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.177+0000 7f8d640b4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d5c159ec0 0x7f8d5c0cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:13.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.177+0000 7f8d640b4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d5c0ce230 0x7f8d5c16d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:13.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.177+0000 7f8d640b4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c09d910 con 0x7f8d5c159ec0 2023-10-10T20:41:13.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.177+0000 7f8d640b4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c0ce6b0 con 0x7f8d5c0ce230 2023-10-10T20:41:13.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.177+0000 7f8d640b4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c09af30 con 0x7f8d5c0b55a0 2023-10-10T20:41:13.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.177+0000 7f8d6164f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d5c159ec0 0x7f8d5c0cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:13.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.177+0000 7f8d6164f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d5c159ec0 0x7f8d5c0cdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49572/0 (socket says 172.21.15.89:49572) 2023-10-10T20:41:13.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.177+0000 7f8d6164f700 1 -- 172.21.15.89:0/4139287263 learned_addr learned my addr 172.21.15.89:0/4139287263 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:13.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.177+0000 7f8d61e50700 1 --2- 172.21.15.89:0/4139287263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d5c0b55a0 0x7f8d5c09d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:13.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.177+0000 7f8d62651700 1 --2- 172.21.15.89:0/4139287263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d5c0ce230 0x7f8d5c16d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:13.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.177+0000 7f8d6164f700 1 -- 172.21.15.89:0/4139287263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d5c0b55a0 msgr2=0x7f8d5c09d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:13.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.178+0000 7f8d6164f700 1 --2- 172.21.15.89:0/4139287263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d5c0b55a0 0x7f8d5c09d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.178+0000 7f8d6164f700 1 -- 172.21.15.89:0/4139287263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d5c0ce230 msgr2=0x7f8d5c16d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:13.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.178+0000 7f8d6164f700 1 --2- 172.21.15.89:0/4139287263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d5c0ce230 0x7f8d5c16d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.178+0000 7f8d6164f700 1 -- 172.21.15.89:0/4139287263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d54009a20 con 0x7f8d5c159ec0 2023-10-10T20:41:13.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.178+0000 7f8d6164f700 1 --2- 172.21.15.89:0/4139287263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d5c159ec0 0x7f8d5c0cdcf0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00e930 tx=0x7f8d4c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:13.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.178+0000 7f8d51ffb700 1 -- 172.21.15.89:0/4139287263 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c00eed0 con 0x7f8d5c159ec0 2023-10-10T20:41:13.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.178+0000 7f8d51ffb700 1 -- 172.21.15.89:0/4139287263 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4c00c5e0 con 0x7f8d5c159ec0 2023-10-10T20:41:13.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.178+0000 7f8d640b4700 1 -- 172.21.15.89:0/4139287263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5c09a600 con 0x7f8d5c159ec0 2023-10-10T20:41:13.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.178+0000 7f8d51ffb700 1 -- 172.21.15.89:0/4139287263 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c00c980 con 0x7f8d5c159ec0 2023-10-10T20:41:13.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.178+0000 7f8d640b4700 1 -- 172.21.15.89:0/4139287263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5c09abc0 con 0x7f8d5c159ec0 2023-10-10T20:41:13.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.179+0000 7f8d640b4700 1 -- 172.21.15.89:0/4139287263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d5c063c90 con 0x7f8d5c159ec0 2023-10-10T20:41:13.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.179+0000 7f8d51ffb700 1 -- 172.21.15.89:0/4139287263 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8d4c00cd70 con 0x7f8d5c159ec0 2023-10-10T20:41:13.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.180+0000 7f8d51ffb700 1 --2- 172.21.15.89:0/4139287263 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8d40067320 0x7f8d400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:13.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.180+0000 7f8d61e50700 1 --2- 172.21.15.89:0/4139287263 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8d40067320 0x7f8d400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:13.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.180+0000 7f8d51ffb700 1 -- 172.21.15.89:0/4139287263 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8d4c084c00 con 0x7f8d5c159ec0 2023-10-10T20:41:13.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.180+0000 7f8d61e50700 1 --2- 172.21.15.89:0/4139287263 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8d40067320 0x7f8d400697d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8d58002800 tx=0x7f8d58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:13.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.184+0000 7f8d51ffb700 1 -- 172.21.15.89:0/4139287263 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d4c051780 con 0x7f8d5c159ec0 2023-10-10T20:41:13.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.447+0000 7f8d640b4700 1 -- 172.21.15.89:0/4139287263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8d5c04c3c0 con 0x7f8d5c159ec0 2023-10-10T20:41:13.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.447+0000 7f8d51ffb700 1 -- 172.21.15.89:0/4139287263 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1037 (secure 0 0 0) 0x7f8d4c054da0 con 0x7f8d5c159ec0 2023-10-10T20:41:13.450 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:13.450 INFO:teuthology.orchestra.run.smithi089.stdout:{"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":416,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":112,"num_osds":8,"num_up_osds":8,"osd_up_since":1696970067,"num_in_osds":8,"osd_in_since":1696970059,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2481106944,"bytes_avail":770613006336,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":73,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"d","status":"up:active","gid":7200}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-10-10T20:37:28.685504+0000","services":{}},"progress_events":{}} 2023-10-10T20:41:13.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.449+0000 7f8d640b4700 1 -- 172.21.15.89:0/4139287263 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8d40067320 msgr2=0x7f8d400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:13.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.449+0000 7f8d640b4700 1 --2- 172.21.15.89:0/4139287263 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8d40067320 0x7f8d400697d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8d58002800 tx=0x7f8d58009040 comp rx=0 tx=0).stop 2023-10-10T20:41:13.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.449+0000 7f8d640b4700 1 -- 172.21.15.89:0/4139287263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d5c159ec0 msgr2=0x7f8d5c0cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:13.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.449+0000 7f8d640b4700 1 --2- 172.21.15.89:0/4139287263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d5c159ec0 0x7f8d5c0cdcf0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00e930 tx=0x7f8d4c008040 comp rx=0 tx=0).stop 2023-10-10T20:41:13.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.449+0000 7f8d640b4700 1 -- 172.21.15.89:0/4139287263 shutdown_connections 2023-10-10T20:41:13.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.449+0000 7f8d640b4700 1 --2- 172.21.15.89:0/4139287263 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8d40067320 0x7f8d400697d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.449+0000 7f8d640b4700 1 --2- 172.21.15.89:0/4139287263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d5c0ce230 0x7f8d5c16d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.449+0000 7f8d640b4700 1 --2- 172.21.15.89:0/4139287263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8d5c0b55a0 0x7f8d5c09d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.449+0000 7f8d640b4700 1 --2- 172.21.15.89:0/4139287263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d5c159ec0 0x7f8d5c0cdcf0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.449+0000 7f8d640b4700 1 -- 172.21.15.89:0/4139287263 >> 172.21.15.89:0/4139287263 conn(0x7f8d5c0999f0 msgr2=0x7f8d5c0c5830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.449+0000 7f8d640b4700 1 -- 172.21.15.89:0/4139287263 shutdown_connections 2023-10-10T20:41:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.449+0000 7f8d640b4700 1 -- 172.21.15.89:0/4139287263 wait complete. 2023-10-10T20:41:13.463 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-10T20:41:13.464 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:41:13.473 INFO:tasks.ceph.mds.d.smithi111.stderr:2023-10-10T20:41:13.471+0000 7f723bb3c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:41:13.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.576+0000 7f4d4e341700 1 Processor -- start 2023-10-10T20:41:13.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.576+0000 7f4d4e341700 1 -- start start 2023-10-10T20:41:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.577+0000 7f4d4e341700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4d48159890 0x7f4d4815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.577+0000 7f4d4e341700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4d48159c60 0x7f4d4815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:13.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.577+0000 7f4d4e341700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d4815db10 0x7f4d48069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:13.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.577+0000 7f4d4e341700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4815df50 con 0x7f4d48159c60 2023-10-10T20:41:13.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.577+0000 7f4d4e341700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4815a160 con 0x7f4d48159890 2023-10-10T20:41:13.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.577+0000 7f4d4e341700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480d13a0 con 0x7f4d4815db10 2023-10-10T20:41:13.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.577+0000 7f4d477fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4d48159c60 0x7f4d4815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:13.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.577+0000 7f4d477fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4d48159c60 0x7f4d4815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49584/0 (socket says 172.21.15.89:49584) 2023-10-10T20:41:13.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.577+0000 7f4d477fe700 1 -- 172.21.15.89:0/1491215132 learned_addr learned my addr 172.21.15.89:0/1491215132 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:13.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.577+0000 7f4d4c8de700 1 --2- 172.21.15.89:0/1491215132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d4815db10 0x7f4d48069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:13.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.577+0000 7f4d477fe700 1 -- 172.21.15.89:0/1491215132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d4815db10 msgr2=0x7f4d48069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:13.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.577+0000 7f4d47fff700 1 --2- 172.21.15.89:0/1491215132 >> v2:172.21.15.111:3300/0 conn(0x7f4d48159890 0x7f4d4815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:13.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.577+0000 7f4d477fe700 1 --2- 172.21.15.89:0/1491215132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d4815db10 0x7f4d48069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.577+0000 7f4d477fe700 1 -- 172.21.15.89:0/1491215132 >> v2:172.21.15.111:3300/0 conn(0x7f4d48159890 msgr2=0x7f4d4815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:13.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.577+0000 7f4d477fe700 1 --2- 172.21.15.89:0/1491215132 >> v2:172.21.15.111:3300/0 conn(0x7f4d48159890 0x7f4d4815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.577+0000 7f4d477fe700 1 -- 172.21.15.89:0/1491215132 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4815e140 con 0x7f4d48159c60 2023-10-10T20:41:13.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.577+0000 7f4d4c8de700 1 --2- 172.21.15.89:0/1491215132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d4815db10 0x7f4d48069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:13.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.577+0000 7f4d477fe700 1 --2- 172.21.15.89:0/1491215132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d48159c60 0x7f4d4815d5d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f4d480d1090 tx=0x7f4d3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93035097316a084 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:13.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.578+0000 7f4d46ffd700 1 -- 172.21.15.89:0/1491215132 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c00b860 con 0x7f4d48159c60 2023-10-10T20:41:13.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.578+0000 7f4d46ffd700 1 -- 172.21.15.89:0/1491215132 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3c00b9c0 con 0x7f4d48159c60 2023-10-10T20:41:13.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.578+0000 7f4d46ffd700 1 -- 172.21.15.89:0/1491215132 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c00bd60 con 0x7f4d48159c60 2023-10-10T20:41:13.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.578+0000 7f4d4e341700 1 -- 172.21.15.89:0/1491215132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d48159c60 msgr2=0x7f4d4815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:13.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.578+0000 7f4d4e341700 1 --2- 172.21.15.89:0/1491215132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d48159c60 0x7f4d4815d5d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f4d480d1090 tx=0x7f4d3c009250 comp rx=0 tx=0).stop 2023-10-10T20:41:13.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.578+0000 7f4d4e341700 1 -- 172.21.15.89:0/1491215132 shutdown_connections 2023-10-10T20:41:13.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.578+0000 7f4d4e341700 1 --2- 172.21.15.89:0/1491215132 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d4815db10 0x7f4d48069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.578+0000 7f4d4e341700 1 --2- 172.21.15.89:0/1491215132 >> v2:172.21.15.111:3300/0 conn(0x7f4d48159890 0x7f4d4815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.578+0000 7f4d4e341700 1 --2- 172.21.15.89:0/1491215132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d48159c60 0x7f4d4815d5d0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.578+0000 7f4d4e341700 1 -- 172.21.15.89:0/1491215132 >> 172.21.15.89:0/1491215132 conn(0x7f4d48068e20 msgr2=0x7f4d480d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:13.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.578+0000 7f4d4e341700 1 -- 172.21.15.89:0/1491215132 shutdown_connections 2023-10-10T20:41:13.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.578+0000 7f4d4e341700 1 -- 172.21.15.89:0/1491215132 wait complete. 2023-10-10T20:41:13.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.579+0000 7f4d4e341700 1 Processor -- start 2023-10-10T20:41:13.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.579+0000 7f4d4e341700 1 -- start start 2023-10-10T20:41:13.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.580+0000 7f4d4e341700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d48159890 0x7f4d480b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:13.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.580+0000 7f4d4e341700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d48159c60 0x7f4d481473d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:13.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.580+0000 7f4d4e341700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d4815db10 0x7f4d480b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:13.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.580+0000 7f4d4e341700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d48144b90 con 0x7f4d48159890 2023-10-10T20:41:13.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.580+0000 7f4d4e341700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480b4e60 con 0x7f4d48159c60 2023-10-10T20:41:13.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.580+0000 7f4d4e341700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480b48a0 con 0x7f4d4815db10 2023-10-10T20:41:13.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.580+0000 7f4d47fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d48159890 0x7f4d480b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:13.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.580+0000 7f4d47fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d48159890 0x7f4d480b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49598/0 (socket says 172.21.15.89:49598) 2023-10-10T20:41:13.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.580+0000 7f4d47fff700 1 -- 172.21.15.89:0/3876857692 learned_addr learned my addr 172.21.15.89:0/3876857692 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:13.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.581+0000 7f4d477fe700 1 --2- 172.21.15.89:0/3876857692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d48159c60 0x7f4d481473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:13.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.581+0000 7f4d47fff700 1 -- 172.21.15.89:0/3876857692 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d4815db10 msgr2=0x7f4d480b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:13.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.581+0000 7f4d4c8de700 1 --2- 172.21.15.89:0/3876857692 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d4815db10 0x7f4d480b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:13.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.581+0000 7f4d47fff700 1 --2- 172.21.15.89:0/3876857692 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d4815db10 0x7f4d480b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.581+0000 7f4d47fff700 1 -- 172.21.15.89:0/3876857692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d48159c60 msgr2=0x7f4d481473d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.581+0000 7f4d47fff700 1 --2- 172.21.15.89:0/3876857692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d48159c60 0x7f4d481473d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.581+0000 7f4d47fff700 1 -- 172.21.15.89:0/3876857692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c0094f0 con 0x7f4d48159890 2023-10-10T20:41:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.581+0000 7f4d47fff700 1 --2- 172.21.15.89:0/3876857692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d48159890 0x7f4d480b3f50 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c880 tx=0x7f4d38008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.581+0000 7f4d2ffff700 1 -- 172.21.15.89:0/3876857692 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d38011600 con 0x7f4d48159890 2023-10-10T20:41:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.581+0000 7f4d2ffff700 1 -- 172.21.15.89:0/3876857692 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d38011780 con 0x7f4d48159890 2023-10-10T20:41:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.581+0000 7f4d4e341700 1 -- 172.21.15.89:0/3876857692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d480b4b80 con 0x7f4d48159890 2023-10-10T20:41:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.581+0000 7f4d2ffff700 1 -- 172.21.15.89:0/3876857692 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d38011b40 con 0x7f4d48159890 2023-10-10T20:41:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.581+0000 7f4d4e341700 1 -- 172.21.15.89:0/3876857692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d480c7c20 con 0x7f4d48159890 2023-10-10T20:41:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.582+0000 7f4d4e341700 1 -- 172.21.15.89:0/3876857692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d480c7e40 con 0x7f4d48159890 2023-10-10T20:41:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.582+0000 7f4d2ffff700 1 -- 172.21.15.89:0/3876857692 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4d38018360 con 0x7f4d48159890 2023-10-10T20:41:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.583+0000 7f4d2ffff700 1 --2- 172.21.15.89:0/3876857692 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4d34067320 0x7f4d340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.583+0000 7f4d477fe700 1 --2- 172.21.15.89:0/3876857692 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4d34067320 0x7f4d340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.583+0000 7f4d2ffff700 1 -- 172.21.15.89:0/3876857692 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4d38084a90 con 0x7f4d48159890 2023-10-10T20:41:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.583+0000 7f4d477fe700 1 --2- 172.21.15.89:0/3876857692 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4d34067320 0x7f4d340697d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4d480d1090 tx=0x7f4d3c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.587+0000 7f4d2ffff700 1 -- 172.21.15.89:0/3876857692 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d38051610 con 0x7f4d48159890 2023-10-10T20:41:13.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.770+0000 7f4d4e341700 1 -- 172.21.15.89:0/3876857692 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4d4804c3c0 con 0x7f4d34067320 2023-10-10T20:41:13.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.778+0000 7f4d2ffff700 1 -- 172.21.15.89:0/3876857692 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287184 (secure 0 0 0) 0x7f4d4804c3c0 con 0x7f4d34067320 2023-10-10T20:41:13.783 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:13.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.781+0000 7f4d4e341700 1 -- 172.21.15.89:0/3876857692 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4d34067320 msgr2=0x7f4d340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:13.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.781+0000 7f4d4e341700 1 --2- 172.21.15.89:0/3876857692 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4d34067320 0x7f4d340697d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4d480d1090 tx=0x7f4d3c017000 comp rx=0 tx=0).stop 2023-10-10T20:41:13.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.781+0000 7f4d4e341700 1 -- 172.21.15.89:0/3876857692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d48159890 msgr2=0x7f4d480b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:13.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.781+0000 7f4d4e341700 1 --2- 172.21.15.89:0/3876857692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d48159890 0x7f4d480b3f50 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c880 tx=0x7f4d38008040 comp rx=0 tx=0).stop 2023-10-10T20:41:13.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.781+0000 7f4d4e341700 1 -- 172.21.15.89:0/3876857692 shutdown_connections 2023-10-10T20:41:13.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.781+0000 7f4d4e341700 1 --2- 172.21.15.89:0/3876857692 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d4815db10 0x7f4d480b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.781+0000 7f4d4e341700 1 --2- 172.21.15.89:0/3876857692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d48159890 0x7f4d480b3f50 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.781+0000 7f4d4e341700 1 --2- 172.21.15.89:0/3876857692 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4d34067320 0x7f4d340697d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.781+0000 7f4d4e341700 1 --2- 172.21.15.89:0/3876857692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d48159c60 0x7f4d481473d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:13.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.781+0000 7f4d4e341700 1 -- 172.21.15.89:0/3876857692 >> 172.21.15.89:0/3876857692 conn(0x7f4d48068e20 msgr2=0x7f4d4815d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:13.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.781+0000 7f4d4e341700 1 -- 172.21.15.89:0/3876857692 shutdown_connections 2023-10-10T20:41:13.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:13.781+0000 7f4d4e341700 1 -- 172.21.15.89:0/3876857692 wait complete. 2023-10-10T20:41:13.788 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:41:13.795 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2023-10-10T20:41:12.731370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422956,"kb_used_data":11884,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752551764,"statfs":{"total":773094113280,"available":770613006336,"internally_reserved":0,"allocated":12169216,"data_stored":5911081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001097"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.185044+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:43:38.336996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179076+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:42:10.326290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183633+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:31:04.342810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184156+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:29:07.750059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183700+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:06:14.543497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180424+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:15:09.117651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180699+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:43:54.668141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.185584+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:51.630860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182871+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:00.815725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178630+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:45:17.413763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.176188+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:28:49.571035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182500+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:27:02.185210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184448+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:23.971915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.177799+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:35.431888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183897+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:08.100006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182045+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:51.136287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181388+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:51:14.573623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182606+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:21:00.592871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.175771+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:29:15.059073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182998+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:35:58.896847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182721+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:57.837359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.175675+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:54:47.622590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184138+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:56.592906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.175733+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:04.315349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178752+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:39:12.818520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181110+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:36:04.128348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183202+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:24.718214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179120+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:57:20.319921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.185431+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:38:00.081732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183275+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:30:09.031752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.186844+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:54.943577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179704+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:51.630860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180587+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:06:14.543497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194387+0000","last_change":"2023-10-10T20:41:07.194387+0000","last_active":"2023-10-10T20:41:07.194387+0000","last_peered":"2023-10-10T20:41:07.194387+0000","last_clean":"2023-10-10T20:41:07.194387+0000","last_became_active":"2023-10-10T20:41:07.194097+0000","last_became_peered":"2023-10-10T20:41:07.194097+0000","last_unstale":"2023-10-10T20:41:07.194387+0000","last_undegraded":"2023-10-10T20:41:07.194387+0000","last_fullsized":"2023-10-10T20:41:07.194387+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:26.546168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182362+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:37:12.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192372+0000","last_change":"2023-10-10T20:41:07.192372+0000","last_active":"2023-10-10T20:41:07.192372+0000","last_peered":"2023-10-10T20:41:07.192372+0000","last_clean":"2023-10-10T20:41:07.192372+0000","last_became_active":"2023-10-10T20:41:07.192088+0000","last_became_peered":"2023-10-10T20:41:07.192088+0000","last_unstale":"2023-10-10T20:41:07.192372+0000","last_undegraded":"2023-10-10T20:41:07.192372+0000","last_fullsized":"2023-10-10T20:41:07.192372+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:59.684638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181983+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:01.392782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.187579+0000","last_change":"2023-10-10T20:41:07.187579+0000","last_active":"2023-10-10T20:41:07.187579+0000","last_peered":"2023-10-10T20:41:07.187579+0000","last_clean":"2023-10-10T20:41:07.187579+0000","last_became_active":"2023-10-10T20:41:07.187296+0000","last_became_peered":"2023-10-10T20:41:07.187296+0000","last_unstale":"2023-10-10T20:41:07.187579+0000","last_undegraded":"2023-10-10T20:41:07.187579+0000","last_fullsized":"2023-10-10T20:41:07.187579+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:18.852267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182717+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:56:26.100041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194195+0000","last_change":"2023-10-10T20:41:07.194195+0000","last_active":"2023-10-10T20:41:07.194195+0000","last_peered":"2023-10-10T20:41:07.194195+0000","last_clean":"2023-10-10T20:41:07.194195+0000","last_became_active":"2023-10-10T20:41:07.193920+0000","last_became_peered":"2023-10-10T20:41:07.193920+0000","last_unstale":"2023-10-10T20:41:07.194195+0000","last_undegraded":"2023-10-10T20:41:07.194195+0000","last_fullsized":"2023-10-10T20:41:07.194195+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:36.588822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182202+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:40:14.372905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.193266+0000","last_change":"2023-10-10T20:41:07.193266+0000","last_active":"2023-10-10T20:41:07.193266+0000","last_peered":"2023-10-10T20:41:07.193266+0000","last_clean":"2023-10-10T20:41:07.193266+0000","last_became_active":"2023-10-10T20:41:07.192991+0000","last_became_peered":"2023-10-10T20:41:07.192991+0000","last_unstale":"2023-10-10T20:41:07.193266+0000","last_undegraded":"2023-10-10T20:41:07.193266+0000","last_fullsized":"2023-10-10T20:41:07.193266+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:15.545053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182281+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:51:25.089205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.193695+0000","last_change":"2023-10-10T20:41:07.193695+0000","last_active":"2023-10-10T20:41:07.193695+0000","last_peered":"2023-10-10T20:41:07.193695+0000","last_clean":"2023-10-10T20:41:07.193695+0000","last_became_active":"2023-10-10T20:41:07.193394+0000","last_became_peered":"2023-10-10T20:41:07.193394+0000","last_unstale":"2023-10-10T20:41:07.193695+0000","last_undegraded":"2023-10-10T20:41:07.193695+0000","last_fullsized":"2023-10-10T20:41:07.193695+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:56.835912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.186003+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:44.120514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.193880+0000","last_change":"2023-10-10T20:41:07.193880+0000","last_active":"2023-10-10T20:41:07.193880+0000","last_peered":"2023-10-10T20:41:07.193880+0000","last_clean":"2023-10-10T20:41:07.193880+0000","last_became_active":"2023-10-10T20:41:07.193586+0000","last_became_peered":"2023-10-10T20:41:07.193586+0000","last_unstale":"2023-10-10T20:41:07.193880+0000","last_undegraded":"2023-10-10T20:41:07.193880+0000","last_fullsized":"2023-10-10T20:41:07.193880+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:15:36.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178163+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:13:00.611054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194069+0000","last_change":"2023-10-10T20:41:07.194069+0000","last_active":"2023-10-10T20:41:07.194069+0000","last_peered":"2023-10-10T20:41:07.194069+0000","last_clean":"2023-10-10T20:41:07.194069+0000","last_became_active":"2023-10-10T20:41:07.193800+0000","last_became_peered":"2023-10-10T20:41:07.193800+0000","last_unstale":"2023-10-10T20:41:07.194069+0000","last_undegraded":"2023-10-10T20:41:07.194069+0000","last_fullsized":"2023-10-10T20:41:07.194069+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:00:42.940622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183477+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:59:58.659044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.190099+0000","last_change":"2023-10-10T20:41:07.190099+0000","last_active":"2023-10-10T20:41:07.190099+0000","last_peered":"2023-10-10T20:41:07.190099+0000","last_clean":"2023-10-10T20:41:07.190099+0000","last_became_active":"2023-10-10T20:41:07.189840+0000","last_became_peered":"2023-10-10T20:41:07.189840+0000","last_unstale":"2023-10-10T20:41:07.190099+0000","last_undegraded":"2023-10-10T20:41:07.190099+0000","last_fullsized":"2023-10-10T20:41:07.190099+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:48:33.094041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178595+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:35.431888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.187415+0000","last_change":"2023-10-10T20:41:07.187415+0000","last_active":"2023-10-10T20:41:07.187415+0000","last_peered":"2023-10-10T20:41:07.187415+0000","last_clean":"2023-10-10T20:41:07.187415+0000","last_became_active":"2023-10-10T20:41:07.187114+0000","last_became_peered":"2023-10-10T20:41:07.187114+0000","last_unstale":"2023-10-10T20:41:07.187415+0000","last_undegraded":"2023-10-10T20:41:07.187415+0000","last_fullsized":"2023-10-10T20:41:07.187415+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:58:20.063048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181497+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:42:34.466499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.188746+0000","last_change":"2023-10-10T20:41:07.188746+0000","last_active":"2023-10-10T20:41:07.188746+0000","last_peered":"2023-10-10T20:41:07.188746+0000","last_clean":"2023-10-10T20:41:07.188746+0000","last_became_active":"2023-10-10T20:41:07.188486+0000","last_became_peered":"2023-10-10T20:41:07.188486+0000","last_unstale":"2023-10-10T20:41:07.188746+0000","last_undegraded":"2023-10-10T20:41:07.188746+0000","last_fullsized":"2023-10-10T20:41:07.188746+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:36.588822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182276+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:04:10.686499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.191180+0000","last_change":"2023-10-10T20:41:07.191180+0000","last_active":"2023-10-10T20:41:07.191180+0000","last_peered":"2023-10-10T20:41:07.191180+0000","last_clean":"2023-10-10T20:41:07.191180+0000","last_became_active":"2023-10-10T20:41:07.190910+0000","last_became_peered":"2023-10-10T20:41:07.190910+0000","last_unstale":"2023-10-10T20:41:07.191180+0000","last_undegraded":"2023-10-10T20:41:07.191180+0000","last_fullsized":"2023-10-10T20:41:07.191180+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:13.928318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184946+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:12:43.974985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.189943+0000","last_change":"2023-10-10T20:41:07.189943+0000","last_active":"2023-10-10T20:41:07.189943+0000","last_peered":"2023-10-10T20:41:07.189943+0000","last_clean":"2023-10-10T20:41:07.189943+0000","last_became_active":"2023-10-10T20:41:07.189639+0000","last_became_peered":"2023-10-10T20:41:07.189639+0000","last_unstale":"2023-10-10T20:41:07.189943+0000","last_undegraded":"2023-10-10T20:41:07.189943+0000","last_fullsized":"2023-10-10T20:41:07.189943+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:24:45.124009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178350+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:07:36.012741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.193331+0000","last_change":"2023-10-10T20:41:07.193331+0000","last_active":"2023-10-10T20:41:07.193331+0000","last_peered":"2023-10-10T20:41:07.193331+0000","last_clean":"2023-10-10T20:41:07.193331+0000","last_became_active":"2023-10-10T20:41:07.193077+0000","last_became_peered":"2023-10-10T20:41:07.193077+0000","last_unstale":"2023-10-10T20:41:07.193331+0000","last_undegraded":"2023-10-10T20:41:07.193331+0000","last_fullsized":"2023-10-10T20:41:07.193331+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:56.816260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182401+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:47:38.801771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.190581+0000","last_change":"2023-10-10T20:41:07.190581+0000","last_active":"2023-10-10T20:41:07.190581+0000","last_peered":"2023-10-10T20:41:07.190581+0000","last_clean":"2023-10-10T20:41:07.190581+0000","last_became_active":"2023-10-10T20:41:07.190319+0000","last_became_peered":"2023-10-10T20:41:07.190319+0000","last_unstale":"2023-10-10T20:41:07.190581+0000","last_undegraded":"2023-10-10T20:41:07.190581+0000","last_fullsized":"2023-10-10T20:41:07.190581+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:42:09.324842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183200+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:15:37.860366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.191241+0000","last_change":"2023-10-10T20:41:07.191241+0000","last_active":"2023-10-10T20:41:07.191241+0000","last_peered":"2023-10-10T20:41:07.191241+0000","last_clean":"2023-10-10T20:41:07.191241+0000","last_became_active":"2023-10-10T20:41:07.191020+0000","last_became_peered":"2023-10-10T20:41:07.191020+0000","last_unstale":"2023-10-10T20:41:07.191241+0000","last_undegraded":"2023-10-10T20:41:07.191241+0000","last_fullsized":"2023-10-10T20:41:07.191241+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:56.816260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180046+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:07:36.012741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.187395+0000","last_change":"2023-10-10T20:41:07.187395+0000","last_active":"2023-10-10T20:41:07.187395+0000","last_peered":"2023-10-10T20:41:07.187395+0000","last_clean":"2023-10-10T20:41:07.187395+0000","last_became_active":"2023-10-10T20:41:07.187130+0000","last_became_peered":"2023-10-10T20:41:07.187130+0000","last_unstale":"2023-10-10T20:41:07.187395+0000","last_undegraded":"2023-10-10T20:41:07.187395+0000","last_fullsized":"2023-10-10T20:41:07.187395+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:32:03.550923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181825+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:54.943577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.190127+0000","last_change":"2023-10-10T20:41:07.190127+0000","last_active":"2023-10-10T20:41:07.190127+0000","last_peered":"2023-10-10T20:41:07.190127+0000","last_clean":"2023-10-10T20:41:07.190127+0000","last_became_active":"2023-10-10T20:41:07.189866+0000","last_became_peered":"2023-10-10T20:41:07.189866+0000","last_unstale":"2023-10-10T20:41:07.190127+0000","last_undegraded":"2023-10-10T20:41:07.190127+0000","last_fullsized":"2023-10-10T20:41:07.190127+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:31:03.341362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.177885+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:57:20.319921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192468+0000","last_change":"2023-10-10T20:41:07.192468+0000","last_active":"2023-10-10T20:41:07.192468+0000","last_peered":"2023-10-10T20:41:07.192468+0000","last_clean":"2023-10-10T20:41:07.192468+0000","last_became_active":"2023-10-10T20:41:07.192177+0000","last_became_peered":"2023-10-10T20:41:07.192177+0000","last_unstale":"2023-10-10T20:41:07.192468+0000","last_undegraded":"2023-10-10T20:41:07.192468+0000","last_fullsized":"2023-10-10T20:41:07.192468+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:15.545053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.183818+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:53.393618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.188588+0000","last_change":"2023-10-10T20:41:07.188588+0000","last_active":"2023-10-10T20:41:07.188588+0000","last_peered":"2023-10-10T20:41:07.188588+0000","last_clean":"2023-10-10T20:41:07.188588+0000","last_became_active":"2023-10-10T20:41:07.188209+0000","last_became_peered":"2023-10-10T20:41:07.188209+0000","last_unstale":"2023-10-10T20:41:07.188588+0000","last_undegraded":"2023-10-10T20:41:07.188588+0000","last_fullsized":"2023-10-10T20:41:07.188588+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:00:42.940622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180348+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:44.120514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194274+0000","last_change":"2023-10-10T20:41:07.194274+0000","last_active":"2023-10-10T20:41:07.194274+0000","last_peered":"2023-10-10T20:41:07.194274+0000","last_clean":"2023-10-10T20:41:07.194274+0000","last_became_active":"2023-10-10T20:41:07.193987+0000","last_became_peered":"2023-10-10T20:41:07.193987+0000","last_unstale":"2023-10-10T20:41:07.194274+0000","last_undegraded":"2023-10-10T20:41:07.194274+0000","last_fullsized":"2023-10-10T20:41:07.194274+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:17.174004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179361+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:14.929765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194646+0000","last_change":"2023-10-10T20:41:07.194646+0000","last_active":"2023-10-10T20:41:07.194646+0000","last_peered":"2023-10-10T20:41:07.194646+0000","last_clean":"2023-10-10T20:41:07.194646+0000","last_became_active":"2023-10-10T20:41:07.194426+0000","last_became_peered":"2023-10-10T20:41:07.194426+0000","last_unstale":"2023-10-10T20:41:07.194646+0000","last_undegraded":"2023-10-10T20:41:07.194646+0000","last_fullsized":"2023-10-10T20:41:07.194646+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:00.391334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181308+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:47:27.526950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192740+0000","last_change":"2023-10-10T20:41:07.192740+0000","last_active":"2023-10-10T20:41:07.192740+0000","last_peered":"2023-10-10T20:41:07.192740+0000","last_clean":"2023-10-10T20:41:07.192740+0000","last_became_active":"2023-10-10T20:41:07.192480+0000","last_became_peered":"2023-10-10T20:41:07.192480+0000","last_unstale":"2023-10-10T20:41:07.192740+0000","last_undegraded":"2023-10-10T20:41:07.192740+0000","last_fullsized":"2023-10-10T20:41:07.192740+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:48.161107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181783+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:27.547616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194467+0000","last_change":"2023-10-10T20:41:07.194467+0000","last_active":"2023-10-10T20:41:07.194467+0000","last_peered":"2023-10-10T20:41:07.194467+0000","last_clean":"2023-10-10T20:41:07.194467+0000","last_became_active":"2023-10-10T20:41:07.193987+0000","last_became_peered":"2023-10-10T20:41:07.193987+0000","last_unstale":"2023-10-10T20:41:07.194467+0000","last_undegraded":"2023-10-10T20:41:07.194467+0000","last_fullsized":"2023-10-10T20:41:07.194467+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:43.862933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.184078+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:38:30.611281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.196552+0000","last_change":"2023-10-10T20:41:07.196552+0000","last_active":"2023-10-10T20:41:07.196552+0000","last_peered":"2023-10-10T20:41:07.196552+0000","last_clean":"2023-10-10T20:41:07.196552+0000","last_became_active":"2023-10-10T20:41:07.196285+0000","last_became_peered":"2023-10-10T20:41:07.196285+0000","last_unstale":"2023-10-10T20:41:07.196552+0000","last_undegraded":"2023-10-10T20:41:07.196552+0000","last_fullsized":"2023-10-10T20:41:07.196552+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:43:53.666694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.178501+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:43.761996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192768+0000","last_change":"2023-10-10T20:41:07.192768+0000","last_active":"2023-10-10T20:41:07.192768+0000","last_peered":"2023-10-10T20:41:07.192768+0000","last_clean":"2023-10-10T20:41:07.192768+0000","last_became_active":"2023-10-10T20:41:07.192467+0000","last_became_peered":"2023-10-10T20:41:07.192467+0000","last_unstale":"2023-10-10T20:41:07.192768+0000","last_undegraded":"2023-10-10T20:41:07.192768+0000","last_fullsized":"2023-10-10T20:41:07.192768+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:59.684638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.182731+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:47:27.526950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.186703+0000","last_change":"2023-10-10T20:41:07.186703+0000","last_active":"2023-10-10T20:41:07.186703+0000","last_peered":"2023-10-10T20:41:07.186703+0000","last_clean":"2023-10-10T20:41:07.186703+0000","last_became_active":"2023-10-10T20:41:07.186447+0000","last_became_peered":"2023-10-10T20:41:07.186447+0000","last_unstale":"2023-10-10T20:41:07.186703+0000","last_undegraded":"2023-10-10T20:41:07.186703+0000","last_fullsized":"2023-10-10T20:41:07.186703+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:57.197462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181353+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:58:21.064495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192864+0000","last_change":"2023-10-10T20:41:07.192864+0000","last_active":"2023-10-10T20:41:07.192864+0000","last_peered":"2023-10-10T20:41:07.192864+0000","last_clean":"2023-10-10T20:41:07.192864+0000","last_became_active":"2023-10-10T20:41:07.192602+0000","last_became_peered":"2023-10-10T20:41:07.192602+0000","last_unstale":"2023-10-10T20:41:07.192864+0000","last_undegraded":"2023-10-10T20:41:07.192864+0000","last_fullsized":"2023-10-10T20:41:07.192864+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:37:19.121105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179266+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:13:00.611054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.192698+0000","last_change":"2023-10-10T20:41:07.192698+0000","last_active":"2023-10-10T20:41:07.192698+0000","last_peered":"2023-10-10T20:41:07.192698+0000","last_clean":"2023-10-10T20:41:07.192698+0000","last_became_active":"2023-10-10T20:41:07.192370+0000","last_became_peered":"2023-10-10T20:41:07.192370+0000","last_unstale":"2023-10-10T20:41:07.192698+0000","last_undegraded":"2023-10-10T20:41:07.192698+0000","last_fullsized":"2023-10-10T20:41:07.192698+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:15:08.116204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.180520+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:56:26.100041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.194285+0000","last_change":"2023-10-10T20:41:07.194285+0000","last_active":"2023-10-10T20:41:07.194285+0000","last_peered":"2023-10-10T20:41:07.194285+0000","last_clean":"2023-10-10T20:41:07.194285+0000","last_became_active":"2023-10-10T20:41:07.194002+0000","last_became_peered":"2023-10-10T20:41:07.194002+0000","last_unstale":"2023-10-10T20:41:07.194285+0000","last_undegraded":"2023-10-10T20:41:07.194285+0000","last_fullsized":"2023-10-10T20:41:07.194285+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:42:33.465052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.179142+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:56:52.494512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.186005+0000","last_change":"2023-10-10T20:41:07.186005+0000","last_active":"2023-10-10T20:41:07.186005+0000","last_peered":"2023-10-10T20:41:07.186005+0000","last_clean":"2023-10-10T20:41:07.186005+0000","last_became_active":"2023-10-10T20:41:07.185723+0000","last_became_peered":"2023-10-10T20:41:07.185723+0000","last_unstale":"2023-10-10T20:41:07.186005+0000","last_undegraded":"2023-10-10T20:41:07.186005+0000","last_fullsized":"2023-10-10T20:41:07.186005+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:39.932733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:41:07.169206+0000","last_change":"2023-10-10T20:41:07.181885+0000","last_active":"2023-10-10T20:41:07.169206+0000","last_peered":"2023-10-10T20:41:07.169206+0000","last_clean":"2023-10-10T20:41:07.169206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:07.169206+0000","last_undegraded":"2023-10-10T20:41:07.169206+0000","last_fullsized":"2023-10-10T20:41:07.169206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:18.175451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":8,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-10T20:41:07.195197+0000","last_change":"2023-10-10T20:41:07.195197+0000","last_active":"2023-10-10T20:41:07.195197+0000","last_peered":"2023-10-10T20:41:07.195197+0000","last_clean":"2023-10-10T20:41:07.195197+0000","last_became_active":"2023-10-10T20:41:07.194893+0000","last_became_peered":"2023-10-10T20:41:07.194893+0000","last_unstale":"2023-10-10T20:41:07.195197+0000","last_undegraded":"2023-10-10T20:41:07.195197+0000","last_fullsized":"2023-10-10T20:41:07.195197+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:51:13.572175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":253,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-10-10T20:41:06.176359+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:41:06.176359+0000","last_peered":"2023-10-10T20:41:06.176359+0000","last_clean":"2023-10-10T20:41:06.176359+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:41:06.176359+0000","last_undegraded":"2023-10-10T20:41:06.176359+0000","last_fullsized":"2023-10-10T20:41:06.176359+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303308,"kb_used_data":1924,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068532,"statfs":{"total":96636764160,"available":96326176768,"internally_reserved":0,"allocated":1970176,"data_stored":1183347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51500000000000001,"15min":0.81999999999999995},"min":{"1min":0.31,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53000000000000003,"15min":0.627},"min":{"1min":0.32200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71999999999999997,"5min":0.95399999999999996,"15min":4.1959999999999997},"last":0.54000000000000004}]},{"osd":1,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.504,"15min":1},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.67800000000000005,"5min":0.78800000000000003,"15min":10.1},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55800000000000005,"15min":0.72999999999999998},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.28999999999999998},"max":{"1min":0.81799999999999995,"5min":1.006,"15min":6.806},"last":0.65300000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57899999999999996,"15min":0.78100000000000003},"min":{"1min":0.43099999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89000000000000001,"5min":1.089,"15min":6.9340000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53200000000000003,"15min":0.73799999999999999},"min":{"1min":0.39000000000000001,"5min":0.36399999999999999,"15min":0.30099999999999999},"max":{"1min":0.77100000000000002,"5min":0.85299999999999998,"15min":6.9870000000000001},"last":0.46899999999999997}]},{"osd":3,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.75800000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.754,"5min":0.80200000000000005,"15min":6.6719999999999997},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.67000000000000004},"min":{"1min":0.38700000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.70599999999999996,"5min":1.2789999999999999,"15min":4.1609999999999996},"last":0.60099999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47399999999999998,"15min":0.46500000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.73199999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.621},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52400000000000002,"15min":0.52300000000000002},"min":{"1min":0.309,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.55200000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.32500000000000001,"5min":0.32100000000000001,"15min":0.314},"max":{"1min":0.86099999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75800000000000001,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.68899999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72999999999999998,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84099999999999997,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.66500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302728,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069112,"statfs":{"total":96636764160,"available":96326770688,"internally_reserved":0,"allocated":1376256,"data_stored":592979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50700000000000001,"15min":0.67200000000000004},"min":{"1min":0.33500000000000002,"5min":0.32400000000000001,"15min":0.312},"max":{"1min":0.68600000000000005,"5min":0.73099999999999998,"15min":7.7590000000000003},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52100000000000002,"15min":0.71299999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":7.8090000000000002},"last":0.52400000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.70999999999999996},"min":{"1min":0.38400000000000001,"5min":0.35599999999999998,"15min":0.27800000000000002},"max":{"1min":0.628,"5min":0.80000000000000004,"15min":7.867},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53300000000000003,"15min":0.88500000000000001},"min":{"1min":0.36599999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":9.7089999999999996},"last":0.54400000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51400000000000001,"15min":0.59599999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.72499999999999998,"5min":0.747,"15min":4.7999999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54600000000000004,"15min":0.84499999999999997},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.71499999999999997,"5min":0.80000000000000004,"15min":6.7629999999999999},"last":0.63}]},{"osd":3,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.62},"min":{"1min":0.40999999999999998,"5min":0.36299999999999999,"15min":0.35299999999999998},"max":{"1min":0.79100000000000004,"5min":0.83999999999999997,"15min":4.8620000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54800000000000004,"15min":0.71699999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.35299999999999998},"max":{"1min":0.68600000000000005,"5min":1.1850000000000001,"15min":7.9009999999999998},"last":0.64900000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51500000000000001,"15min":0.46800000000000003},"min":{"1min":0.32400000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71099999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59199999999999997,"15min":0.56799999999999995},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.37},"max":{"1min":0.73499999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.49099999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52000000000000002,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.314,"15min":0.312},"max":{"1min":0.69199999999999995,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.747,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.433}]},{"osd":7,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55100000000000005,"15min":0.52000000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.72599999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57799999999999996,"15min":0.54400000000000004},"min":{"1min":0.42899999999999999,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":0.69399999999999995,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.47499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302716,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069124,"statfs":{"total":96636764160,"available":96326782976,"internally_reserved":0,"allocated":1363968,"data_stored":586716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.31900000000000001,"15min":0.47599999999999998},"min":{"1min":0.20999999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.441,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.32300000000000001,"15min":0.36299999999999999},"min":{"1min":0.23599999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46000000000000002,"5min":0.68700000000000006,"15min":1.194},"last":0.308}]},{"osd":2,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.32400000000000001,"15min":0.36399999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.379,"5min":0.67400000000000004,"15min":1.2869999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.33800000000000002,"15min":0.505},"min":{"1min":0.26000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.54200000000000004,"5min":0.57199999999999995,"15min":5.0919999999999996},"last":0.51300000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36299999999999999,"15min":0.52800000000000002},"min":{"1min":0.26800000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.51000000000000001,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36499999999999999,"15min":0.40699999999999997},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.48299999999999998,"5min":0.72099999999999997,"15min":1.2170000000000001},"last":0.41399999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57899999999999996,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.81000000000000005,"5min":1.133,"15min":1.254},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56599999999999995,"15min":0.70599999999999996},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69899999999999995,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.67100000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57899999999999996,"15min":0.59499999999999997},"min":{"1min":0.42299999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.751,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.499},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.60799999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71899999999999997,"5min":1.0720000000000001,"15min":1.3109999999999999},"last":0.55700000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60699999999999998,"15min":0.84399999999999997},"min":{"1min":0.435,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86199999999999999,"5min":0.995,"15min":5.0659999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61699999999999999,"15min":0.63200000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84599999999999997,"5min":1.1439999999999999,"15min":1.171},"last":0.58899999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59299999999999997,"15min":0.61699999999999999},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.753,"5min":1.214,"15min":1.214},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60099999999999998,"15min":0.72499999999999998},"min":{"1min":0.45000000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76900000000000002,"5min":1.115,"15min":4.0499999999999998},"last":0.625}]}]},{"osd":0,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303304,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068536,"statfs":{"total":96636764160,"available":96326180864,"internally_reserved":0,"allocated":1966080,"data_stored":1182386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.309,"15min":0.52800000000000002},"min":{"1min":0.20999999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.38300000000000001,"5min":0.51600000000000001,"15min":7.9249999999999998},"last":0.371},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.318,"15min":0.505},"min":{"1min":0.23699999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.41899999999999998,"5min":0.54900000000000004,"15min":6.6299999999999999},"last":0.255}]},{"osd":2,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.32600000000000001,"15min":0.55200000000000005},"min":{"1min":0.22,"5min":0.216,"15min":0.216},"max":{"1min":0.48799999999999999,"5min":0.55400000000000005,"15min":8.2509999999999994},"last":0.313},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.34000000000000002,"15min":0.52800000000000002},"min":{"1min":0.23100000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.45700000000000002,"5min":0.54100000000000004,"15min":6.7370000000000001},"last":0.32800000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.34399999999999997,"15min":0.55700000000000005},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.48299999999999998,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.61199999999999999},"min":{"1min":0.32000000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":7.8929999999999998},"last":0.41599999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55600000000000005,"15min":0.71399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.72499999999999998,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.748},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56299999999999994,"15min":0.77200000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.67600000000000005,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.71099999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.55200000000000005,"15min":0.751},"min":{"1min":0.38800000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69299999999999995,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58099999999999996,"15min":0.77800000000000002},"min":{"1min":0.42299999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77000000000000002,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.75700000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60199999999999998,"15min":0.81799999999999995},"min":{"1min":0.39800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72099999999999997,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61099999999999999,"15min":0.76100000000000001},"min":{"1min":0.41399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74299999999999999,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58799999999999997,"15min":0.752},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78300000000000003,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59099999999999997,"15min":0.78600000000000003},"min":{"1min":0.45200000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.69299999999999995,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.60299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302728,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069112,"statfs":{"total":96636764160,"available":96326770688,"internally_reserved":0,"allocated":1376256,"data_stored":592979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.28299999999999997,"5min":0.309,"15min":0.32500000000000001},"min":{"1min":0.21099999999999999,"5min":0.189,"15min":0.189},"max":{"1min":0.38300000000000001,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.29399999999999998,"5min":0.32100000000000001,"15min":0.35299999999999998},"min":{"1min":0.215,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.39500000000000002,"5min":0.5,"15min":0.746},"last":0.29799999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.32900000000000001,"15min":0.72899999999999998},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.35699999999999998,"5min":0.63,"15min":11.967000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.34499999999999997,"15min":0.48299999999999998},"min":{"1min":0.26800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.443,"5min":0.53400000000000003,"15min":4.1100000000000003},"last":0.36499999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.34499999999999997,"15min":0.55700000000000005},"min":{"1min":0.23699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.39400000000000002,"5min":0.55400000000000005,"15min":6.3040000000000003},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38700000000000001,"15min":0.78100000000000003},"min":{"1min":0.27900000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.46899999999999997,"5min":0.65900000000000003,"15min":11.935},"last":0.35199999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.54300000000000004,"15min":0.55500000000000005},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.58699999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71699999999999997,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.58599999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.378},"max":{"1min":0.66300000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54400000000000004,"15min":0.58399999999999996},"min":{"1min":0.39700000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74199999999999999,"5min":0.874,"15min":0.96099999999999997},"last":0.54100000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57699999999999996,"15min":0.60099999999999998},"min":{"1min":0.42199999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.745,"5min":0.91600000000000004,"15min":0.91800000000000004},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73799999999999999,"5min":0.93100000000000005,"15min":0.93600000000000005},"last":0.55500000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56499999999999995,"15min":0.58499999999999996},"min":{"1min":0.41799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71099999999999997,"5min":0.89100000000000001,"15min":1.052},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57699999999999996,"15min":0.78000000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69799999999999995,"5min":0.85999999999999999,"15min":6.2649999999999997},"last":0.45900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302728,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069112,"statfs":{"total":96636764160,"available":96326770688,"internally_reserved":0,"allocated":1376256,"data_stored":592979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.31,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.31900000000000001},"min":{"1min":0.25,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.33600000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.23499999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.316},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38600000000000001,"15min":0.377},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.57099999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.40500000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.34999999999999998},"min":{"1min":0.23699999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.35699999999999998,"15min":0.36199999999999999},"min":{"1min":0.26300000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.35899999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69999999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.82299999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55300000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.35899999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55900000000000005,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.76900000000000002,"5min":1.01,"15min":1.01},"last":0.626}]},{"osd":6,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.42099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85699999999999998,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.60999999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.441,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.496,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.88400000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302716,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069124,"statfs":{"total":96636764160,"available":96326782976,"internally_reserved":0,"allocated":1363968,"data_stored":586716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68300000000000005,"5min":0.74199999999999999,"15min":0.747},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.76600000000000001},"min":{"1min":0.374,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67500000000000004,"5min":0.78400000000000003,"15min":7.6719999999999997},"last":0.439}]},{"osd":1,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52900000000000003,"15min":0.77400000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.755,"5min":0.755,"15min":7.7160000000000002},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65300000000000002,"5min":0.76900000000000002,"15min":0.86799999999999999},"last":0.66100000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.65300000000000002,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56299999999999994,"15min":0.79300000000000004},"min":{"1min":0.39700000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.72799999999999998,"5min":0.79800000000000004,"15min":7.7839999999999998},"last":0.68500000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55200000000000005,"15min":0.54900000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65500000000000003,"5min":0.73399999999999999,"15min":0.96999999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.41199999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.66100000000000003,"5min":0.78100000000000003,"15min":0.79400000000000004},"last":0.47099999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51000000000000001,"15min":0.48699999999999999},"min":{"1min":0.32500000000000001,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.77500000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54900000000000004,"15min":0.54300000000000004},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.55300000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.39200000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68799999999999994,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.35899999999999999},"max":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.378,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.84599999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.46100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.748,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.60899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302728,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069112,"statfs":{"total":96636764160,"available":96326770688,"internally_reserved":0,"allocated":1376256,"data_stored":592979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.28599999999999998,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.79300000000000004},"last":0.52400000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52700000000000002,"15min":0.64100000000000001},"min":{"1min":0.33000000000000002,"5min":0.32200000000000001,"15min":0.316},"max":{"1min":0.70399999999999996,"5min":1.2330000000000001,"15min":5.2679999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51800000000000002,"15min":0.629},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.47099999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52800000000000002,"15min":0.64800000000000002},"min":{"1min":0.35999999999999999,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.76600000000000001,"5min":0.86199999999999999,"15min":5.2960000000000003},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72899999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.45000000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55300000000000005,"15min":0.67700000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.73899999999999999,"5min":0.93000000000000005,"15min":5.3330000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.69099999999999995},"min":{"1min":0.39300000000000002,"5min":0.36899999999999999,"15min":0.27700000000000002},"max":{"1min":0.76000000000000001,"5min":0.94399999999999995,"15min":5.2850000000000001},"last":0.53900000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47799999999999998,"15min":0.496},"min":{"1min":0.311,"5min":0.29599999999999999,"15min":0.28699999999999998},"max":{"1min":0.72499999999999998,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34300000000000003,"15min":0.32200000000000001},"max":{"1min":0.70799999999999996,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.63800000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54100000000000004,"15min":0.54700000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.80100000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82499999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.624}]},{"osd":7,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.378,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68899999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60199999999999998,"15min":0.60099999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83399999999999996,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.61399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:41:16.802 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:41:16.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.927+0000 7f2ecbc9e700 1 Processor -- start 2023-10-10T20:41:16.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.927+0000 7f2ecbc9e700 1 -- start start 2023-10-10T20:41:16.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.928+0000 7f2ecbc9e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2ec4159890 0x7f2ec415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:16.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.928+0000 7f2ecbc9e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2ec4069560 0x7f2ec40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:16.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.928+0000 7f2ecbc9e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec4069ee0 con 0x7f2ec4159c60 2023-10-10T20:41:16.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.928+0000 7f2ecbc9e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec415a160 con 0x7f2ec4069560 2023-10-10T20:41:16.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.928+0000 7f2ecbc9e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec40d13a0 con 0x7f2ec4159890 2023-10-10T20:41:16.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.928+0000 7f2eca23b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2ec4069560 0x7f2ec40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:16.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.928+0000 7f2ec9239700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2ec4159c60 0x7f2ec415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41376/0 (socket says 172.21.15.89:41376) 2023-10-10T20:41:16.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.928+0000 7f2ec9239700 1 -- 172.21.15.89:0/3207716990 learned_addr learned my addr 172.21.15.89:0/3207716990 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:16.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.928+0000 7f2ec9a3a700 1 --2- 172.21.15.89:0/3207716990 >> v2:172.21.15.111:3300/0 conn(0x7f2ec4159890 0x7f2ec415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:16.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.928+0000 7f2eca23b700 1 -- 172.21.15.89:0/3207716990 >> v2:172.21.15.111:3300/0 conn(0x7f2ec4159890 msgr2=0x7f2ec415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:16.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.928+0000 7f2eca23b700 1 --2- 172.21.15.89:0/3207716990 >> v2:172.21.15.111:3300/0 conn(0x7f2ec4159890 0x7f2ec415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:16.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.928+0000 7f2eca23b700 1 -- 172.21.15.89:0/3207716990 >> v1:172.21.15.89:6789/0 conn(0x7f2ec4159c60 legacy=0x7f2ec415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.928+0000 7f2eca23b700 1 -- 172.21.15.89:0/3207716990 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec415e140 con 0x7f2ec4069560 2023-10-10T20:41:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.928+0000 7f2ec8a38700 1 -- 172.21.15.89:0/3207716990 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147471378 0 0) 0x7f2ec4069ee0 con 0x7f2ec4159c60 2023-10-10T20:41:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.928+0000 7f2ec9a3a700 1 --2- 172.21.15.89:0/3207716990 >> v2:172.21.15.111:3300/0 conn(0x7f2ec4159890 0x7f2ec415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.928+0000 7f2eca23b700 1 --2- 172.21.15.89:0/3207716990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ec4069560 0x7f2ec40699a0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40d1090 tx=0x7f2eb4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f635ba98acabc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:16.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.928+0000 7f2ec8a38700 1 -- 172.21.15.89:0/3207716990 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb400e070 con 0x7f2ec4069560 2023-10-10T20:41:16.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.929+0000 7f2ec8a38700 1 -- 172.21.15.89:0/3207716990 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb400bc30 con 0x7f2ec4069560 2023-10-10T20:41:16.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.929+0000 7f2ec8a38700 1 -- 172.21.15.89:0/3207716990 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4012430 con 0x7f2ec4069560 2023-10-10T20:41:16.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.929+0000 7f2ecbc9e700 1 -- 172.21.15.89:0/3207716990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ec4069560 msgr2=0x7f2ec40699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:16.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.929+0000 7f2ecbc9e700 1 --2- 172.21.15.89:0/3207716990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ec4069560 0x7f2ec40699a0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40d1090 tx=0x7f2eb4009780 comp rx=0 tx=0).stop 2023-10-10T20:41:16.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.929+0000 7f2ecbc9e700 1 -- 172.21.15.89:0/3207716990 shutdown_connections 2023-10-10T20:41:16.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.929+0000 7f2ecbc9e700 1 --2- 172.21.15.89:0/3207716990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ec4069560 0x7f2ec40699a0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:16.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.929+0000 7f2ecbc9e700 1 --2- 172.21.15.89:0/3207716990 >> v2:172.21.15.111:3300/0 conn(0x7f2ec4159890 0x7f2ec415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:16.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.929+0000 7f2ecbc9e700 1 -- 172.21.15.89:0/3207716990 >> 172.21.15.89:0/3207716990 conn(0x7f2ec4068e20 msgr2=0x7f2ec40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:16.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.929+0000 7f2ecbc9e700 1 -- 172.21.15.89:0/3207716990 shutdown_connections 2023-10-10T20:41:16.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.929+0000 7f2ecbc9e700 1 -- 172.21.15.89:0/3207716990 wait complete. 2023-10-10T20:41:16.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.930+0000 7f2ecbc9e700 1 Processor -- start 2023-10-10T20:41:16.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.930+0000 7f2ecbc9e700 1 -- start start 2023-10-10T20:41:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.931+0000 7f2ecbc9e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ec4069560 0x7f2ec40cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.931+0000 7f2ecbc9e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ec4159890 0x7f2ec415c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.931+0000 7f2ecbc9e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ec4159c60 0x7f2ec4144d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.931+0000 7f2ecbc9e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec40cdff0 con 0x7f2ec4159890 2023-10-10T20:41:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.931+0000 7f2ecbc9e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec40ce860 con 0x7f2ec4159c60 2023-10-10T20:41:16.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.931+0000 7f2ecbc9e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec40cda30 con 0x7f2ec4069560 2023-10-10T20:41:16.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.931+0000 7f2ec9239700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ec4159890 0x7f2ec415c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:16.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.931+0000 7f2ec9239700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ec4159890 0x7f2ec415c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49626/0 (socket says 172.21.15.89:49626) 2023-10-10T20:41:16.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.931+0000 7f2ec9239700 1 -- 172.21.15.89:0/3066120530 learned_addr learned my addr 172.21.15.89:0/3066120530 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.931+0000 7f2ec9a3a700 1 --2- 172.21.15.89:0/3066120530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ec4069560 0x7f2ec40cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.931+0000 7f2eca23b700 1 --2- 172.21.15.89:0/3066120530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ec4159c60 0x7f2ec4144d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.931+0000 7f2ec9239700 1 -- 172.21.15.89:0/3066120530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ec4069560 msgr2=0x7f2ec40cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.931+0000 7f2ec9239700 1 --2- 172.21.15.89:0/3066120530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ec4069560 0x7f2ec40cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.931+0000 7f2ec9239700 1 -- 172.21.15.89:0/3066120530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ec4159c60 msgr2=0x7f2ec4144d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.931+0000 7f2ec9239700 1 --2- 172.21.15.89:0/3066120530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ec4159c60 0x7f2ec4144d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.931+0000 7f2ec9239700 1 -- 172.21.15.89:0/3066120530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb4009a20 con 0x7f2ec4159890 2023-10-10T20:41:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.931+0000 7f2ec9239700 1 --2- 172.21.15.89:0/3066120530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ec4159890 0x7f2ec415c2b0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800c5e0 tx=0x7f2eb8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.932+0000 7f2ec1ffb700 1 -- 172.21.15.89:0/3066120530 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8005b30 con 0x7f2ec4159890 2023-10-10T20:41:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.932+0000 7f2ec1ffb700 1 -- 172.21.15.89:0/3066120530 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb8005c90 con 0x7f2ec4159890 2023-10-10T20:41:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.932+0000 7f2ec1ffb700 1 -- 172.21.15.89:0/3066120530 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb80072e0 con 0x7f2ec4159890 2023-10-10T20:41:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.932+0000 7f2ecbc9e700 1 -- 172.21.15.89:0/3066120530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec40cdd10 con 0x7f2ec4159890 2023-10-10T20:41:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.932+0000 7f2ecbc9e700 1 -- 172.21.15.89:0/3066120530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec40cd600 con 0x7f2ec4159890 2023-10-10T20:41:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.933+0000 7f2ecbc9e700 1 -- 172.21.15.89:0/3066120530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec404c3c0 con 0x7f2ec4159890 2023-10-10T20:41:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.933+0000 7f2ec1ffb700 1 -- 172.21.15.89:0/3066120530 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2eb801b3b0 con 0x7f2ec4159890 2023-10-10T20:41:16.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.933+0000 7f2ec1ffb700 1 --2- 172.21.15.89:0/3066120530 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2eb0067320 0x7f2eb00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:16.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.933+0000 7f2ec1ffb700 1 -- 172.21.15.89:0/3066120530 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2eb80840a0 con 0x7f2ec4159890 2023-10-10T20:41:16.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.933+0000 7f2ec9a3a700 1 --2- 172.21.15.89:0/3066120530 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2eb0067320 0x7f2eb00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:16.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.934+0000 7f2ec9a3a700 1 --2- 172.21.15.89:0/3066120530 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2eb0067320 0x7f2eb00697d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc006f00 tx=0x7f2ebc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:16.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:16.938+0000 7f2ec1ffb700 1 -- 172.21.15.89:0/3066120530 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2eb804b5a0 con 0x7f2ec4159890 2023-10-10T20:41:17.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.118+0000 7f2ecbc9e700 1 -- 172.21.15.89:0/3066120530 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ec40cd820 con 0x7f2eb0067320 2023-10-10T20:41:17.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.126+0000 7f2ec1ffb700 1 -- 172.21.15.89:0/3066120530 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290248 (secure 0 0 0) 0x7f2ec40cd820 con 0x7f2eb0067320 2023-10-10T20:41:17.132 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:17.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.129+0000 7f2ecbc9e700 1 -- 172.21.15.89:0/3066120530 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2eb0067320 msgr2=0x7f2eb00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:17.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.129+0000 7f2ecbc9e700 1 --2- 172.21.15.89:0/3066120530 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2eb0067320 0x7f2eb00697d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc006f00 tx=0x7f2ebc008040 comp rx=0 tx=0).stop 2023-10-10T20:41:17.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.129+0000 7f2ecbc9e700 1 -- 172.21.15.89:0/3066120530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ec4159890 msgr2=0x7f2ec415c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:17.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.129+0000 7f2ecbc9e700 1 --2- 172.21.15.89:0/3066120530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ec4159890 0x7f2ec415c2b0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800c5e0 tx=0x7f2eb8001070 comp rx=0 tx=0).stop 2023-10-10T20:41:17.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.129+0000 7f2ecbc9e700 1 -- 172.21.15.89:0/3066120530 shutdown_connections 2023-10-10T20:41:17.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.129+0000 7f2ecbc9e700 1 --2- 172.21.15.89:0/3066120530 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2eb0067320 0x7f2eb00697d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.129+0000 7f2ecbc9e700 1 --2- 172.21.15.89:0/3066120530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ec4159c60 0x7f2ec4144d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.129+0000 7f2ecbc9e700 1 --2- 172.21.15.89:0/3066120530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2ec4069560 0x7f2ec40cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.129+0000 7f2ecbc9e700 1 --2- 172.21.15.89:0/3066120530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ec4159890 0x7f2ec415c2b0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.129+0000 7f2ecbc9e700 1 -- 172.21.15.89:0/3066120530 >> 172.21.15.89:0/3066120530 conn(0x7f2ec4068e20 msgr2=0x7f2ec40d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:17.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.130+0000 7f2ecbc9e700 1 -- 172.21.15.89:0/3066120530 shutdown_connections 2023-10-10T20:41:17.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.130+0000 7f2ecbc9e700 1 -- 172.21.15.89:0/3066120530 wait complete. 2023-10-10T20:41:17.140 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:41:17.144 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2023-10-10T20:41:16.732266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2423236,"kb_used_data":12164,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752551484,"statfs":{"total":773094113280,"available":770612719616,"internally_reserved":0,"allocated":12455936,"data_stored":6032627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.528395"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.229684+0000","last_change":"2023-10-10T20:41:08.224135+0000","last_active":"2023-10-10T20:41:08.229684+0000","last_peered":"2023-10-10T20:41:08.229684+0000","last_clean":"2023-10-10T20:41:08.229684+0000","last_became_active":"2023-10-10T20:41:08.223854+0000","last_became_peered":"2023-10-10T20:41:08.223854+0000","last_unstale":"2023-10-10T20:41:08.229684+0000","last_undegraded":"2023-10-10T20:41:08.229684+0000","last_fullsized":"2023-10-10T20:41:08.229684+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:11:16.859904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.227060+0000","last_change":"2023-10-10T20:41:08.221827+0000","last_active":"2023-10-10T20:41:08.227060+0000","last_peered":"2023-10-10T20:41:08.227060+0000","last_clean":"2023-10-10T20:41:08.227060+0000","last_became_active":"2023-10-10T20:41:08.221445+0000","last_became_peered":"2023-10-10T20:41:08.221445+0000","last_unstale":"2023-10-10T20:41:08.227060+0000","last_undegraded":"2023-10-10T20:41:08.227060+0000","last_fullsized":"2023-10-10T20:41:08.227060+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:50:00.815725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.230990+0000","last_change":"2023-10-10T20:41:08.226218+0000","last_active":"2023-10-10T20:41:08.230990+0000","last_peered":"2023-10-10T20:41:08.230990+0000","last_clean":"2023-10-10T20:41:08.230990+0000","last_became_active":"2023-10-10T20:41:08.225943+0000","last_became_peered":"2023-10-10T20:41:08.225943+0000","last_unstale":"2023-10-10T20:41:08.230990+0000","last_undegraded":"2023-10-10T20:41:08.230990+0000","last_fullsized":"2023-10-10T20:41:08.230990+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:51:25.089205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.226324+0000","last_change":"2023-10-10T20:41:08.217010+0000","last_active":"2023-10-10T20:41:08.226324+0000","last_peered":"2023-10-10T20:41:08.226324+0000","last_clean":"2023-10-10T20:41:08.226324+0000","last_became_active":"2023-10-10T20:41:08.216657+0000","last_became_peered":"2023-10-10T20:41:08.216657+0000","last_unstale":"2023-10-10T20:41:08.226324+0000","last_undegraded":"2023-10-10T20:41:08.226324+0000","last_fullsized":"2023-10-10T20:41:08.226324+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:09.326826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.221978+0000","last_change":"2023-10-10T20:41:08.216586+0000","last_active":"2023-10-10T20:41:08.221978+0000","last_peered":"2023-10-10T20:41:08.221978+0000","last_clean":"2023-10-10T20:41:08.221978+0000","last_became_active":"2023-10-10T20:41:08.215968+0000","last_became_peered":"2023-10-10T20:41:08.215968+0000","last_unstale":"2023-10-10T20:41:08.221978+0000","last_undegraded":"2023-10-10T20:41:08.221978+0000","last_fullsized":"2023-10-10T20:41:08.221978+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:32:04.552371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.232696+0000","last_change":"2023-10-10T20:41:08.229971+0000","last_active":"2023-10-10T20:41:08.232696+0000","last_peered":"2023-10-10T20:41:08.232696+0000","last_clean":"2023-10-10T20:41:08.232696+0000","last_became_active":"2023-10-10T20:41:08.229703+0000","last_became_peered":"2023-10-10T20:41:08.229703+0000","last_unstale":"2023-10-10T20:41:08.232696+0000","last_undegraded":"2023-10-10T20:41:08.232696+0000","last_fullsized":"2023-10-10T20:41:08.232696+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:04:10.686499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.234293+0000","last_change":"2023-10-10T20:41:08.232170+0000","last_active":"2023-10-10T20:41:08.234293+0000","last_peered":"2023-10-10T20:41:08.234293+0000","last_clean":"2023-10-10T20:41:08.234293+0000","last_became_active":"2023-10-10T20:41:08.231900+0000","last_became_peered":"2023-10-10T20:41:08.231900+0000","last_unstale":"2023-10-10T20:41:08.234293+0000","last_undegraded":"2023-10-10T20:41:08.234293+0000","last_fullsized":"2023-10-10T20:41:08.234293+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:56:26.100041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.236212+0000","last_change":"2023-10-10T20:41:08.235767+0000","last_active":"2023-10-10T20:41:08.236212+0000","last_peered":"2023-10-10T20:41:08.236212+0000","last_clean":"2023-10-10T20:41:08.236212+0000","last_became_active":"2023-10-10T20:41:08.234718+0000","last_became_peered":"2023-10-10T20:41:08.234718+0000","last_unstale":"2023-10-10T20:41:08.236212+0000","last_undegraded":"2023-10-10T20:41:08.236212+0000","last_fullsized":"2023-10-10T20:41:08.236212+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:47:27.526950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.230946+0000","last_change":"2023-10-10T20:41:08.226568+0000","last_active":"2023-10-10T20:41:08.230946+0000","last_peered":"2023-10-10T20:41:08.230946+0000","last_clean":"2023-10-10T20:41:08.230946+0000","last_became_active":"2023-10-10T20:41:08.226301+0000","last_became_peered":"2023-10-10T20:41:08.226301+0000","last_unstale":"2023-10-10T20:41:08.230946+0000","last_undegraded":"2023-10-10T20:41:08.230946+0000","last_fullsized":"2023-10-10T20:41:08.230946+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:43:54.668141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.227040+0000","last_change":"2023-10-10T20:41:08.221672+0000","last_active":"2023-10-10T20:41:08.227040+0000","last_peered":"2023-10-10T20:41:08.227040+0000","last_clean":"2023-10-10T20:41:08.227040+0000","last_became_active":"2023-10-10T20:41:08.221176+0000","last_became_peered":"2023-10-10T20:41:08.221176+0000","last_unstale":"2023-10-10T20:41:08.227040+0000","last_undegraded":"2023-10-10T20:41:08.227040+0000","last_fullsized":"2023-10-10T20:41:08.227040+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:27:02.185210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.228622+0000","last_change":"2023-10-10T20:41:08.222642+0000","last_active":"2023-10-10T20:41:08.228622+0000","last_peered":"2023-10-10T20:41:08.228622+0000","last_clean":"2023-10-10T20:41:08.228622+0000","last_became_active":"2023-10-10T20:41:08.222231+0000","last_became_peered":"2023-10-10T20:41:08.222231+0000","last_unstale":"2023-10-10T20:41:08.228622+0000","last_undegraded":"2023-10-10T20:41:08.228622+0000","last_fullsized":"2023-10-10T20:41:08.228622+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:39:22.463470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.232601+0000","last_change":"2023-10-10T20:41:08.230299+0000","last_active":"2023-10-10T20:41:08.232601+0000","last_peered":"2023-10-10T20:41:08.232601+0000","last_clean":"2023-10-10T20:41:08.232601+0000","last_became_active":"2023-10-10T20:41:08.230020+0000","last_became_peered":"2023-10-10T20:41:08.230020+0000","last_unstale":"2023-10-10T20:41:08.232601+0000","last_undegraded":"2023-10-10T20:41:08.232601+0000","last_fullsized":"2023-10-10T20:41:08.232601+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:51:14.573623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.231594+0000","last_change":"2023-10-10T20:41:08.227663+0000","last_active":"2023-10-10T20:41:08.231594+0000","last_peered":"2023-10-10T20:41:08.231594+0000","last_clean":"2023-10-10T20:41:08.231594+0000","last_became_active":"2023-10-10T20:41:08.224716+0000","last_became_peered":"2023-10-10T20:41:08.224716+0000","last_unstale":"2023-10-10T20:41:08.231594+0000","last_undegraded":"2023-10-10T20:41:08.231594+0000","last_fullsized":"2023-10-10T20:41:08.231594+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:21:05.064210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.224307+0000","last_change":"2023-10-10T20:41:08.218899+0000","last_active":"2023-10-10T20:41:08.224307+0000","last_peered":"2023-10-10T20:41:08.224307+0000","last_clean":"2023-10-10T20:41:08.224307+0000","last_became_active":"2023-10-10T20:41:08.218641+0000","last_became_peered":"2023-10-10T20:41:08.218641+0000","last_unstale":"2023-10-10T20:41:08.224307+0000","last_undegraded":"2023-10-10T20:41:08.224307+0000","last_fullsized":"2023-10-10T20:41:08.224307+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:47:38.801771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.231561+0000","last_change":"2023-10-10T20:41:08.227271+0000","last_active":"2023-10-10T20:41:08.231561+0000","last_peered":"2023-10-10T20:41:08.231561+0000","last_clean":"2023-10-10T20:41:08.231561+0000","last_became_active":"2023-10-10T20:41:08.226882+0000","last_became_peered":"2023-10-10T20:41:08.226882+0000","last_unstale":"2023-10-10T20:41:08.231561+0000","last_undegraded":"2023-10-10T20:41:08.231561+0000","last_fullsized":"2023-10-10T20:41:08.231561+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:58:20.939700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.229613+0000","last_change":"2023-10-10T20:41:08.223301+0000","last_active":"2023-10-10T20:41:08.229613+0000","last_peered":"2023-10-10T20:41:08.229613+0000","last_clean":"2023-10-10T20:41:08.229613+0000","last_became_active":"2023-10-10T20:41:08.223061+0000","last_became_peered":"2023-10-10T20:41:08.223061+0000","last_unstale":"2023-10-10T20:41:08.229613+0000","last_undegraded":"2023-10-10T20:41:08.229613+0000","last_fullsized":"2023-10-10T20:41:08.229613+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:41:17.529989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.234184+0000","last_change":"2023-10-10T20:41:08.231185+0000","last_active":"2023-10-10T20:41:08.234184+0000","last_peered":"2023-10-10T20:41:08.234184+0000","last_clean":"2023-10-10T20:41:08.234184+0000","last_became_active":"2023-10-10T20:41:08.227724+0000","last_became_peered":"2023-10-10T20:41:08.227724+0000","last_unstale":"2023-10-10T20:41:08.234184+0000","last_undegraded":"2023-10-10T20:41:08.234184+0000","last_fullsized":"2023-10-10T20:41:08.234184+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:01.392782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.228861+0000","last_change":"2023-10-10T20:41:08.224091+0000","last_active":"2023-10-10T20:41:08.228861+0000","last_peered":"2023-10-10T20:41:08.228861+0000","last_clean":"2023-10-10T20:41:08.228861+0000","last_became_active":"2023-10-10T20:41:08.222946+0000","last_became_peered":"2023-10-10T20:41:08.222946+0000","last_unstale":"2023-10-10T20:41:08.228861+0000","last_undegraded":"2023-10-10T20:41:08.228861+0000","last_fullsized":"2023-10-10T20:41:08.228861+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:49.162555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.228607+0000","last_change":"2023-10-10T20:41:08.222471+0000","last_active":"2023-10-10T20:41:08.228607+0000","last_peered":"2023-10-10T20:41:08.228607+0000","last_clean":"2023-10-10T20:41:08.228607+0000","last_became_active":"2023-10-10T20:41:08.221847+0000","last_became_peered":"2023-10-10T20:41:08.221847+0000","last_unstale":"2023-10-10T20:41:08.228607+0000","last_undegraded":"2023-10-10T20:41:08.228607+0000","last_fullsized":"2023-10-10T20:41:08.228607+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:09.734284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.226358+0000","last_change":"2023-10-10T20:41:08.222612+0000","last_active":"2023-10-10T20:41:08.226358+0000","last_peered":"2023-10-10T20:41:08.226358+0000","last_clean":"2023-10-10T20:41:08.226358+0000","last_became_active":"2023-10-10T20:41:08.222172+0000","last_became_peered":"2023-10-10T20:41:08.222172+0000","last_unstale":"2023-10-10T20:41:08.226358+0000","last_undegraded":"2023-10-10T20:41:08.226358+0000","last_fullsized":"2023-10-10T20:41:08.226358+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:12:38.107785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.224894+0000","last_change":"2023-10-10T20:41:08.216773+0000","last_active":"2023-10-10T20:41:08.224894+0000","last_peered":"2023-10-10T20:41:08.224894+0000","last_clean":"2023-10-10T20:41:08.224894+0000","last_became_active":"2023-10-10T20:41:08.216336+0000","last_became_peered":"2023-10-10T20:41:08.216336+0000","last_unstale":"2023-10-10T20:41:08.224894+0000","last_undegraded":"2023-10-10T20:41:08.224894+0000","last_fullsized":"2023-10-10T20:41:08.224894+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:41:17.529989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.226463+0000","last_change":"2023-10-10T20:41:08.218960+0000","last_active":"2023-10-10T20:41:08.226463+0000","last_peered":"2023-10-10T20:41:08.226463+0000","last_clean":"2023-10-10T20:41:08.226463+0000","last_became_active":"2023-10-10T20:41:08.218762+0000","last_became_peered":"2023-10-10T20:41:08.218762+0000","last_unstale":"2023-10-10T20:41:08.226463+0000","last_undegraded":"2023-10-10T20:41:08.226463+0000","last_fullsized":"2023-10-10T20:41:08.226463+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:55:09.949445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.232596+0000","last_change":"2023-10-10T20:41:08.230561+0000","last_active":"2023-10-10T20:41:08.232596+0000","last_peered":"2023-10-10T20:41:08.232596+0000","last_clean":"2023-10-10T20:41:08.232596+0000","last_became_active":"2023-10-10T20:41:08.230096+0000","last_became_peered":"2023-10-10T20:41:08.230096+0000","last_unstale":"2023-10-10T20:41:08.232596+0000","last_undegraded":"2023-10-10T20:41:08.232596+0000","last_fullsized":"2023-10-10T20:41:08.232596+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:42:34.466499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.226450+0000","last_change":"2023-10-10T20:41:08.216087+0000","last_active":"2023-10-10T20:41:08.226450+0000","last_peered":"2023-10-10T20:41:08.226450+0000","last_clean":"2023-10-10T20:41:08.226450+0000","last_became_active":"2023-10-10T20:41:08.215832+0000","last_became_peered":"2023-10-10T20:41:08.215832+0000","last_unstale":"2023-10-10T20:41:08.226450+0000","last_undegraded":"2023-10-10T20:41:08.226450+0000","last_fullsized":"2023-10-10T20:41:08.226450+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:16:51.814305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.226877+0000","last_change":"2023-10-10T20:41:08.222309+0000","last_active":"2023-10-10T20:41:08.226877+0000","last_peered":"2023-10-10T20:41:08.226877+0000","last_clean":"2023-10-10T20:41:08.226877+0000","last_became_active":"2023-10-10T20:41:08.222020+0000","last_became_peered":"2023-10-10T20:41:08.222020+0000","last_unstale":"2023-10-10T20:41:08.226877+0000","last_undegraded":"2023-10-10T20:41:08.226877+0000","last_fullsized":"2023-10-10T20:41:08.226877+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:51.630860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.228835+0000","last_change":"2023-10-10T20:41:08.223433+0000","last_active":"2023-10-10T20:41:08.228835+0000","last_peered":"2023-10-10T20:41:08.228835+0000","last_clean":"2023-10-10T20:41:08.228835+0000","last_became_active":"2023-10-10T20:41:08.223156+0000","last_became_peered":"2023-10-10T20:41:08.223156+0000","last_unstale":"2023-10-10T20:41:08.228835+0000","last_undegraded":"2023-10-10T20:41:08.228835+0000","last_fullsized":"2023-10-10T20:41:08.228835+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:36:45.918177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.229161+0000","last_change":"2023-10-10T20:41:08.224149+0000","last_active":"2023-10-10T20:41:08.229161+0000","last_peered":"2023-10-10T20:41:08.229161+0000","last_clean":"2023-10-10T20:41:08.229161+0000","last_became_active":"2023-10-10T20:41:08.223861+0000","last_became_peered":"2023-10-10T20:41:08.223861+0000","last_unstale":"2023-10-10T20:41:08.229161+0000","last_undegraded":"2023-10-10T20:41:08.229161+0000","last_fullsized":"2023-10-10T20:41:08.229161+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:58:46.845198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.225455+0000","last_change":"2023-10-10T20:41:08.223162+0000","last_active":"2023-10-10T20:41:08.225455+0000","last_peered":"2023-10-10T20:41:08.225455+0000","last_clean":"2023-10-10T20:41:08.225455+0000","last_became_active":"2023-10-10T20:41:08.222856+0000","last_became_peered":"2023-10-10T20:41:08.222856+0000","last_unstale":"2023-10-10T20:41:08.225455+0000","last_undegraded":"2023-10-10T20:41:08.225455+0000","last_fullsized":"2023-10-10T20:41:08.225455+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:14.929765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.231575+0000","last_change":"2023-10-10T20:41:08.227383+0000","last_active":"2023-10-10T20:41:08.231575+0000","last_peered":"2023-10-10T20:41:08.231575+0000","last_clean":"2023-10-10T20:41:08.231575+0000","last_became_active":"2023-10-10T20:41:08.226906+0000","last_became_peered":"2023-10-10T20:41:08.226906+0000","last_unstale":"2023-10-10T20:41:08.231575+0000","last_undegraded":"2023-10-10T20:41:08.231575+0000","last_fullsized":"2023-10-10T20:41:08.231575+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:01:19.693951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.231636+0000","last_change":"2023-10-10T20:41:08.228366+0000","last_active":"2023-10-10T20:41:08.231636+0000","last_peered":"2023-10-10T20:41:08.231636+0000","last_clean":"2023-10-10T20:41:08.231636+0000","last_became_active":"2023-10-10T20:41:08.228111+0000","last_became_peered":"2023-10-10T20:41:08.228111+0000","last_unstale":"2023-10-10T20:41:08.231636+0000","last_undegraded":"2023-10-10T20:41:08.231636+0000","last_fullsized":"2023-10-10T20:41:08.231636+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:07:46.946890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.225201+0000","last_change":"2023-10-10T20:41:08.219865+0000","last_active":"2023-10-10T20:41:08.225201+0000","last_peered":"2023-10-10T20:41:08.225201+0000","last_clean":"2023-10-10T20:41:08.225201+0000","last_became_active":"2023-10-10T20:41:08.218821+0000","last_became_peered":"2023-10-10T20:41:08.218821+0000","last_unstale":"2023-10-10T20:41:08.225201+0000","last_undegraded":"2023-10-10T20:41:08.225201+0000","last_fullsized":"2023-10-10T20:41:08.225201+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:49.162555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.234212+0000","last_change":"2023-10-10T20:41:08.231386+0000","last_active":"2023-10-10T20:41:08.234212+0000","last_peered":"2023-10-10T20:41:08.234212+0000","last_clean":"2023-10-10T20:41:08.234212+0000","last_became_active":"2023-10-10T20:41:08.227972+0000","last_became_peered":"2023-10-10T20:41:08.227972+0000","last_unstale":"2023-10-10T20:41:08.234212+0000","last_undegraded":"2023-10-10T20:41:08.234212+0000","last_fullsized":"2023-10-10T20:41:08.234212+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:57.837359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.225961+0000","last_change":"2023-10-10T20:41:08.221611+0000","last_active":"2023-10-10T20:41:08.225961+0000","last_peered":"2023-10-10T20:41:08.225961+0000","last_clean":"2023-10-10T20:41:08.225961+0000","last_became_active":"2023-10-10T20:41:08.221252+0000","last_became_peered":"2023-10-10T20:41:08.221252+0000","last_unstale":"2023-10-10T20:41:08.225961+0000","last_undegraded":"2023-10-10T20:41:08.225961+0000","last_fullsized":"2023-10-10T20:41:08.225961+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:19.853715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"111'4","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.222022+0000","last_change":"2023-10-10T20:41:07.194387+0000","last_active":"2023-10-10T20:41:08.222022+0000","last_peered":"2023-10-10T20:41:08.222022+0000","last_clean":"2023-10-10T20:41:08.222022+0000","last_became_active":"2023-10-10T20:41:07.194097+0000","last_became_peered":"2023-10-10T20:41:07.194097+0000","last_unstale":"2023-10-10T20:41:08.222022+0000","last_undegraded":"2023-10-10T20:41:08.222022+0000","last_fullsized":"2023-10-10T20:41:08.222022+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:26.546168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.230965+0000","last_change":"2023-10-10T20:41:08.227080+0000","last_active":"2023-10-10T20:41:08.230965+0000","last_peered":"2023-10-10T20:41:08.230965+0000","last_clean":"2023-10-10T20:41:08.230965+0000","last_became_active":"2023-10-10T20:41:08.226825+0000","last_became_peered":"2023-10-10T20:41:08.226825+0000","last_unstale":"2023-10-10T20:41:08.230965+0000","last_undegraded":"2023-10-10T20:41:08.230965+0000","last_fullsized":"2023-10-10T20:41:08.230965+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:27.547616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"111'1","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.228003+0000","last_change":"2023-10-10T20:41:07.192372+0000","last_active":"2023-10-10T20:41:08.228003+0000","last_peered":"2023-10-10T20:41:08.228003+0000","last_clean":"2023-10-10T20:41:08.228003+0000","last_became_active":"2023-10-10T20:41:07.192088+0000","last_became_peered":"2023-10-10T20:41:07.192088+0000","last_unstale":"2023-10-10T20:41:08.228003+0000","last_undegraded":"2023-10-10T20:41:08.228003+0000","last_fullsized":"2023-10-10T20:41:08.228003+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:59.684638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.227351+0000","last_change":"2023-10-10T20:41:08.224960+0000","last_active":"2023-10-10T20:41:08.227351+0000","last_peered":"2023-10-10T20:41:08.227351+0000","last_clean":"2023-10-10T20:41:08.227351+0000","last_became_active":"2023-10-10T20:41:08.224497+0000","last_became_peered":"2023-10-10T20:41:08.224497+0000","last_unstale":"2023-10-10T20:41:08.227351+0000","last_undegraded":"2023-10-10T20:41:08.227351+0000","last_fullsized":"2023-10-10T20:41:08.227351+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:35:58.896847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"111'2","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.225326+0000","last_change":"2023-10-10T20:41:07.187579+0000","last_active":"2023-10-10T20:41:08.225326+0000","last_peered":"2023-10-10T20:41:08.225326+0000","last_clean":"2023-10-10T20:41:08.225326+0000","last_became_active":"2023-10-10T20:41:07.187296+0000","last_became_peered":"2023-10-10T20:41:07.187296+0000","last_unstale":"2023-10-10T20:41:08.225326+0000","last_undegraded":"2023-10-10T20:41:08.225326+0000","last_fullsized":"2023-10-10T20:41:08.225326+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:18.852267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.220052+0000","last_change":"2023-10-10T20:41:08.215326+0000","last_active":"2023-10-10T20:41:08.220052+0000","last_peered":"2023-10-10T20:41:08.220052+0000","last_clean":"2023-10-10T20:41:08.220052+0000","last_became_active":"2023-10-10T20:41:08.215020+0000","last_became_peered":"2023-10-10T20:41:08.215020+0000","last_unstale":"2023-10-10T20:41:08.220052+0000","last_undegraded":"2023-10-10T20:41:08.220052+0000","last_fullsized":"2023-10-10T20:41:08.220052+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:40.934181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"111'2","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.226948+0000","last_change":"2023-10-10T20:41:07.194195+0000","last_active":"2023-10-10T20:41:08.226948+0000","last_peered":"2023-10-10T20:41:08.226948+0000","last_clean":"2023-10-10T20:41:08.226948+0000","last_became_active":"2023-10-10T20:41:07.193920+0000","last_became_peered":"2023-10-10T20:41:07.193920+0000","last_unstale":"2023-10-10T20:41:08.226948+0000","last_undegraded":"2023-10-10T20:41:08.226948+0000","last_fullsized":"2023-10-10T20:41:08.226948+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:36.588822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.228888+0000","last_change":"2023-10-10T20:41:08.223592+0000","last_active":"2023-10-10T20:41:08.228888+0000","last_peered":"2023-10-10T20:41:08.228888+0000","last_clean":"2023-10-10T20:41:08.228888+0000","last_became_active":"2023-10-10T20:41:08.223373+0000","last_became_peered":"2023-10-10T20:41:08.223373+0000","last_unstale":"2023-10-10T20:41:08.228888+0000","last_undegraded":"2023-10-10T20:41:08.228888+0000","last_fullsized":"2023-10-10T20:41:08.228888+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:58.198910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"111'1","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.224459+0000","last_change":"2023-10-10T20:41:07.193266+0000","last_active":"2023-10-10T20:41:08.224459+0000","last_peered":"2023-10-10T20:41:08.224459+0000","last_clean":"2023-10-10T20:41:08.224459+0000","last_became_active":"2023-10-10T20:41:07.192991+0000","last_became_peered":"2023-10-10T20:41:07.192991+0000","last_unstale":"2023-10-10T20:41:08.224459+0000","last_undegraded":"2023-10-10T20:41:08.224459+0000","last_fullsized":"2023-10-10T20:41:08.224459+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:15.545053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.234309+0000","last_change":"2023-10-10T20:41:08.232285+0000","last_active":"2023-10-10T20:41:08.234309+0000","last_peered":"2023-10-10T20:41:08.234309+0000","last_clean":"2023-10-10T20:41:08.234309+0000","last_became_active":"2023-10-10T20:41:08.227619+0000","last_became_peered":"2023-10-10T20:41:08.227619+0000","last_unstale":"2023-10-10T20:41:08.234309+0000","last_undegraded":"2023-10-10T20:41:08.234309+0000","last_fullsized":"2023-10-10T20:41:08.234309+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:18.175451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"111'1","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.228918+0000","last_change":"2023-10-10T20:41:07.193695+0000","last_active":"2023-10-10T20:41:08.228918+0000","last_peered":"2023-10-10T20:41:08.228918+0000","last_clean":"2023-10-10T20:41:08.228918+0000","last_became_active":"2023-10-10T20:41:07.193394+0000","last_became_peered":"2023-10-10T20:41:07.193394+0000","last_unstale":"2023-10-10T20:41:08.228918+0000","last_undegraded":"2023-10-10T20:41:08.228918+0000","last_fullsized":"2023-10-10T20:41:08.228918+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:56.835912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.236505+0000","last_change":"2023-10-10T20:41:08.235842+0000","last_active":"2023-10-10T20:41:08.236505+0000","last_peered":"2023-10-10T20:41:08.236505+0000","last_clean":"2023-10-10T20:41:08.236505+0000","last_became_active":"2023-10-10T20:41:08.234858+0000","last_became_peered":"2023-10-10T20:41:08.234858+0000","last_unstale":"2023-10-10T20:41:08.236505+0000","last_undegraded":"2023-10-10T20:41:08.236505+0000","last_fullsized":"2023-10-10T20:41:08.236505+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:58:21.064495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.219666+0000","last_change":"2023-10-10T20:41:07.193880+0000","last_active":"2023-10-10T20:41:08.219666+0000","last_peered":"2023-10-10T20:41:08.219666+0000","last_clean":"2023-10-10T20:41:08.219666+0000","last_became_active":"2023-10-10T20:41:07.193586+0000","last_became_peered":"2023-10-10T20:41:07.193586+0000","last_unstale":"2023-10-10T20:41:08.219666+0000","last_undegraded":"2023-10-10T20:41:08.219666+0000","last_fullsized":"2023-10-10T20:41:08.219666+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:15:36.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.224402+0000","last_change":"2023-10-10T20:41:08.218239+0000","last_active":"2023-10-10T20:41:08.224402+0000","last_peered":"2023-10-10T20:41:08.224402+0000","last_clean":"2023-10-10T20:41:08.224402+0000","last_became_active":"2023-10-10T20:41:08.217981+0000","last_became_peered":"2023-10-10T20:41:08.217981+0000","last_unstale":"2023-10-10T20:41:08.224402+0000","last_undegraded":"2023-10-10T20:41:08.224402+0000","last_fullsized":"2023-10-10T20:41:08.224402+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:31:04.342810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.211484+0000","last_change":"2023-10-10T20:41:07.194069+0000","last_active":"2023-10-10T20:41:08.211484+0000","last_peered":"2023-10-10T20:41:08.211484+0000","last_clean":"2023-10-10T20:41:08.211484+0000","last_became_active":"2023-10-10T20:41:07.193800+0000","last_became_peered":"2023-10-10T20:41:07.193800+0000","last_unstale":"2023-10-10T20:41:08.211484+0000","last_undegraded":"2023-10-10T20:41:08.211484+0000","last_fullsized":"2023-10-10T20:41:08.211484+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:00:42.940622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.228902+0000","last_change":"2023-10-10T20:41:08.221110+0000","last_active":"2023-10-10T20:41:08.228902+0000","last_peered":"2023-10-10T20:41:08.228902+0000","last_clean":"2023-10-10T20:41:08.228902+0000","last_became_active":"2023-10-10T20:41:08.220223+0000","last_became_peered":"2023-10-10T20:41:08.220223+0000","last_unstale":"2023-10-10T20:41:08.228902+0000","last_undegraded":"2023-10-10T20:41:08.228902+0000","last_fullsized":"2023-10-10T20:41:08.228902+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:46:35.644708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.207333+0000","last_change":"2023-10-10T20:41:07.190099+0000","last_active":"2023-10-10T20:41:08.207333+0000","last_peered":"2023-10-10T20:41:08.207333+0000","last_clean":"2023-10-10T20:41:08.207333+0000","last_became_active":"2023-10-10T20:41:07.189840+0000","last_became_peered":"2023-10-10T20:41:07.189840+0000","last_unstale":"2023-10-10T20:41:08.207333+0000","last_undegraded":"2023-10-10T20:41:08.207333+0000","last_fullsized":"2023-10-10T20:41:08.207333+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:48:33.094041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.227453+0000","last_change":"2023-10-10T20:41:08.223845+0000","last_active":"2023-10-10T20:41:08.227453+0000","last_peered":"2023-10-10T20:41:08.227453+0000","last_clean":"2023-10-10T20:41:08.227453+0000","last_became_active":"2023-10-10T20:41:08.223588+0000","last_became_peered":"2023-10-10T20:41:08.223588+0000","last_unstale":"2023-10-10T20:41:08.227453+0000","last_undegraded":"2023-10-10T20:41:08.227453+0000","last_fullsized":"2023-10-10T20:41:08.227453+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:45:17.413763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.214793+0000","last_change":"2023-10-10T20:41:07.187415+0000","last_active":"2023-10-10T20:41:08.214793+0000","last_peered":"2023-10-10T20:41:08.214793+0000","last_clean":"2023-10-10T20:41:08.214793+0000","last_became_active":"2023-10-10T20:41:07.187114+0000","last_became_peered":"2023-10-10T20:41:07.187114+0000","last_unstale":"2023-10-10T20:41:08.214793+0000","last_undegraded":"2023-10-10T20:41:08.214793+0000","last_fullsized":"2023-10-10T20:41:08.214793+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:58:20.063048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.227332+0000","last_change":"2023-10-10T20:41:08.224772+0000","last_active":"2023-10-10T20:41:08.227332+0000","last_peered":"2023-10-10T20:41:08.227332+0000","last_clean":"2023-10-10T20:41:08.227332+0000","last_became_active":"2023-10-10T20:41:08.224156+0000","last_became_peered":"2023-10-10T20:41:08.224156+0000","last_unstale":"2023-10-10T20:41:08.227332+0000","last_undegraded":"2023-10-10T20:41:08.227332+0000","last_fullsized":"2023-10-10T20:41:08.227332+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:36:04.128348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"111'3","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.225467+0000","last_change":"2023-10-10T20:41:07.188746+0000","last_active":"2023-10-10T20:41:08.225467+0000","last_peered":"2023-10-10T20:41:08.225467+0000","last_clean":"2023-10-10T20:41:08.225467+0000","last_became_active":"2023-10-10T20:41:07.188486+0000","last_became_peered":"2023-10-10T20:41:07.188486+0000","last_unstale":"2023-10-10T20:41:08.225467+0000","last_undegraded":"2023-10-10T20:41:08.225467+0000","last_fullsized":"2023-10-10T20:41:08.225467+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:36.588822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.232231+0000","last_change":"2023-10-10T20:41:08.230772+0000","last_active":"2023-10-10T20:41:08.232231+0000","last_peered":"2023-10-10T20:41:08.232231+0000","last_clean":"2023-10-10T20:41:08.232231+0000","last_became_active":"2023-10-10T20:41:08.230470+0000","last_became_peered":"2023-10-10T20:41:08.230470+0000","last_unstale":"2023-10-10T20:41:08.232231+0000","last_undegraded":"2023-10-10T20:41:08.232231+0000","last_fullsized":"2023-10-10T20:41:08.232231+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:21:00.592871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"111'6","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.233435+0000","last_change":"2023-10-10T20:41:07.191180+0000","last_active":"2023-10-10T20:41:08.233435+0000","last_peered":"2023-10-10T20:41:08.233435+0000","last_clean":"2023-10-10T20:41:08.233435+0000","last_became_active":"2023-10-10T20:41:07.190910+0000","last_became_peered":"2023-10-10T20:41:07.190910+0000","last_unstale":"2023-10-10T20:41:08.233435+0000","last_undegraded":"2023-10-10T20:41:08.233435+0000","last_fullsized":"2023-10-10T20:41:08.233435+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:13.928318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.227457+0000","last_change":"2023-10-10T20:41:08.217758+0000","last_active":"2023-10-10T20:41:08.227457+0000","last_peered":"2023-10-10T20:41:08.227457+0000","last_clean":"2023-10-10T20:41:08.227457+0000","last_became_active":"2023-10-10T20:41:08.217510+0000","last_became_peered":"2023-10-10T20:41:08.217510+0000","last_unstale":"2023-10-10T20:41:08.227457+0000","last_undegraded":"2023-10-10T20:41:08.227457+0000","last_fullsized":"2023-10-10T20:41:08.227457+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:09:06.227800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.206187+0000","last_change":"2023-10-10T20:41:07.189943+0000","last_active":"2023-10-10T20:41:08.206187+0000","last_peered":"2023-10-10T20:41:08.206187+0000","last_clean":"2023-10-10T20:41:08.206187+0000","last_became_active":"2023-10-10T20:41:07.189639+0000","last_became_peered":"2023-10-10T20:41:07.189639+0000","last_unstale":"2023-10-10T20:41:08.206187+0000","last_undegraded":"2023-10-10T20:41:08.206187+0000","last_fullsized":"2023-10-10T20:41:08.206187+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:24:45.124009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.229435+0000","last_change":"2023-10-10T20:41:08.227784+0000","last_active":"2023-10-10T20:41:08.229435+0000","last_peered":"2023-10-10T20:41:08.229435+0000","last_clean":"2023-10-10T20:41:08.229435+0000","last_became_active":"2023-10-10T20:41:08.227525+0000","last_became_peered":"2023-10-10T20:41:08.227525+0000","last_unstale":"2023-10-10T20:41:08.229435+0000","last_undegraded":"2023-10-10T20:41:08.229435+0000","last_fullsized":"2023-10-10T20:41:08.229435+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:15:37.860366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.211774+0000","last_change":"2023-10-10T20:41:07.193331+0000","last_active":"2023-10-10T20:41:08.211774+0000","last_peered":"2023-10-10T20:41:08.211774+0000","last_clean":"2023-10-10T20:41:08.211774+0000","last_became_active":"2023-10-10T20:41:07.193077+0000","last_became_peered":"2023-10-10T20:41:07.193077+0000","last_unstale":"2023-10-10T20:41:08.211774+0000","last_undegraded":"2023-10-10T20:41:08.211774+0000","last_fullsized":"2023-10-10T20:41:08.211774+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:56.816260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.231030+0000","last_change":"2023-10-10T20:41:08.227968+0000","last_active":"2023-10-10T20:41:08.231030+0000","last_peered":"2023-10-10T20:41:08.231030+0000","last_clean":"2023-10-10T20:41:08.231030+0000","last_became_active":"2023-10-10T20:41:08.226731+0000","last_became_peered":"2023-10-10T20:41:08.226731+0000","last_unstale":"2023-10-10T20:41:08.231030+0000","last_undegraded":"2023-10-10T20:41:08.231030+0000","last_fullsized":"2023-10-10T20:41:08.231030+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:15:09.117651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"111'8","reported_seq":18,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.235703+0000","last_change":"2023-10-10T20:41:07.190581+0000","last_active":"2023-10-10T20:41:08.235703+0000","last_peered":"2023-10-10T20:41:08.235703+0000","last_clean":"2023-10-10T20:41:08.235703+0000","last_became_active":"2023-10-10T20:41:07.190319+0000","last_became_peered":"2023-10-10T20:41:07.190319+0000","last_unstale":"2023-10-10T20:41:08.235703+0000","last_undegraded":"2023-10-10T20:41:08.235703+0000","last_fullsized":"2023-10-10T20:41:08.235703+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:42:09.324842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.230910+0000","last_change":"2023-10-10T20:41:08.227496+0000","last_active":"2023-10-10T20:41:08.230910+0000","last_peered":"2023-10-10T20:41:08.230910+0000","last_clean":"2023-10-10T20:41:08.230910+0000","last_became_active":"2023-10-10T20:41:08.225982+0000","last_became_peered":"2023-10-10T20:41:08.225982+0000","last_unstale":"2023-10-10T20:41:08.230910+0000","last_undegraded":"2023-10-10T20:41:08.230910+0000","last_fullsized":"2023-10-10T20:41:08.230910+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:18.175451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"111'1","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.225502+0000","last_change":"2023-10-10T20:41:07.191241+0000","last_active":"2023-10-10T20:41:08.225502+0000","last_peered":"2023-10-10T20:41:08.225502+0000","last_clean":"2023-10-10T20:41:08.225502+0000","last_became_active":"2023-10-10T20:41:07.191020+0000","last_became_peered":"2023-10-10T20:41:07.191020+0000","last_unstale":"2023-10-10T20:41:08.225502+0000","last_undegraded":"2023-10-10T20:41:08.225502+0000","last_fullsized":"2023-10-10T20:41:08.225502+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:56.816260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.221271+0000","last_change":"2023-10-10T20:41:08.217674+0000","last_active":"2023-10-10T20:41:08.221271+0000","last_peered":"2023-10-10T20:41:08.221271+0000","last_clean":"2023-10-10T20:41:08.221271+0000","last_became_active":"2023-10-10T20:41:08.215165+0000","last_became_peered":"2023-10-10T20:41:08.215165+0000","last_unstale":"2023-10-10T20:41:08.221271+0000","last_undegraded":"2023-10-10T20:41:08.221271+0000","last_fullsized":"2023-10-10T20:41:08.221271+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:43.761996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"111'2","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.231724+0000","last_change":"2023-10-10T20:41:07.187395+0000","last_active":"2023-10-10T20:41:08.231724+0000","last_peered":"2023-10-10T20:41:08.231724+0000","last_clean":"2023-10-10T20:41:08.231724+0000","last_became_active":"2023-10-10T20:41:07.187130+0000","last_became_peered":"2023-10-10T20:41:07.187130+0000","last_unstale":"2023-10-10T20:41:08.231724+0000","last_undegraded":"2023-10-10T20:41:08.231724+0000","last_fullsized":"2023-10-10T20:41:08.231724+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:32:03.550923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.226342+0000","last_change":"2023-10-10T20:41:08.222427+0000","last_active":"2023-10-10T20:41:08.226342+0000","last_peered":"2023-10-10T20:41:08.226342+0000","last_clean":"2023-10-10T20:41:08.226342+0000","last_became_active":"2023-10-10T20:41:08.221815+0000","last_became_peered":"2023-10-10T20:41:08.221815+0000","last_unstale":"2023-10-10T20:41:08.226342+0000","last_undegraded":"2023-10-10T20:41:08.226342+0000","last_fullsized":"2023-10-10T20:41:08.226342+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:32:04.552371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"111'1","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.227285+0000","last_change":"2023-10-10T20:41:07.190127+0000","last_active":"2023-10-10T20:41:08.227285+0000","last_peered":"2023-10-10T20:41:08.227285+0000","last_clean":"2023-10-10T20:41:08.227285+0000","last_became_active":"2023-10-10T20:41:07.189866+0000","last_became_peered":"2023-10-10T20:41:07.189866+0000","last_unstale":"2023-10-10T20:41:08.227285+0000","last_undegraded":"2023-10-10T20:41:08.227285+0000","last_fullsized":"2023-10-10T20:41:08.227285+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:31:03.341362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.227382+0000","last_change":"2023-10-10T20:41:08.224840+0000","last_active":"2023-10-10T20:41:08.227382+0000","last_peered":"2023-10-10T20:41:08.227382+0000","last_clean":"2023-10-10T20:41:08.227382+0000","last_became_active":"2023-10-10T20:41:08.224555+0000","last_became_peered":"2023-10-10T20:41:08.224555+0000","last_unstale":"2023-10-10T20:41:08.227382+0000","last_undegraded":"2023-10-10T20:41:08.227382+0000","last_fullsized":"2023-10-10T20:41:08.227382+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:56.592906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.206241+0000","last_change":"2023-10-10T20:41:07.192468+0000","last_active":"2023-10-10T20:41:08.206241+0000","last_peered":"2023-10-10T20:41:08.206241+0000","last_clean":"2023-10-10T20:41:08.206241+0000","last_became_active":"2023-10-10T20:41:07.192177+0000","last_became_peered":"2023-10-10T20:41:07.192177+0000","last_unstale":"2023-10-10T20:41:08.206241+0000","last_undegraded":"2023-10-10T20:41:08.206241+0000","last_fullsized":"2023-10-10T20:41:08.206241+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:32:15.545053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.231587+0000","last_change":"2023-10-10T20:41:08.227836+0000","last_active":"2023-10-10T20:41:08.231587+0000","last_peered":"2023-10-10T20:41:08.231587+0000","last_clean":"2023-10-10T20:41:08.231587+0000","last_became_active":"2023-10-10T20:41:08.224561+0000","last_became_peered":"2023-10-10T20:41:08.224561+0000","last_unstale":"2023-10-10T20:41:08.231587+0000","last_undegraded":"2023-10-10T20:41:08.231587+0000","last_fullsized":"2023-10-10T20:41:08.231587+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:55.160181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"111'2","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.224892+0000","last_change":"2023-10-10T20:41:07.188588+0000","last_active":"2023-10-10T20:41:08.224892+0000","last_peered":"2023-10-10T20:41:08.224892+0000","last_clean":"2023-10-10T20:41:08.224892+0000","last_became_active":"2023-10-10T20:41:07.188209+0000","last_became_peered":"2023-10-10T20:41:07.188209+0000","last_unstale":"2023-10-10T20:41:08.224892+0000","last_undegraded":"2023-10-10T20:41:08.224892+0000","last_fullsized":"2023-10-10T20:41:08.224892+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:00:42.940622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.232594+0000","last_change":"2023-10-10T20:41:08.227325+0000","last_active":"2023-10-10T20:41:08.232594+0000","last_peered":"2023-10-10T20:41:08.232594+0000","last_clean":"2023-10-10T20:41:08.232594+0000","last_became_active":"2023-10-10T20:41:08.226959+0000","last_became_peered":"2023-10-10T20:41:08.226959+0000","last_unstale":"2023-10-10T20:41:08.232594+0000","last_undegraded":"2023-10-10T20:41:08.232594+0000","last_fullsized":"2023-10-10T20:41:08.232594+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:58:21.064495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.214000+0000","last_change":"2023-10-10T20:41:07.194274+0000","last_active":"2023-10-10T20:41:08.214000+0000","last_peered":"2023-10-10T20:41:08.214000+0000","last_clean":"2023-10-10T20:41:08.214000+0000","last_became_active":"2023-10-10T20:41:07.193987+0000","last_became_peered":"2023-10-10T20:41:07.193987+0000","last_unstale":"2023-10-10T20:41:08.214000+0000","last_undegraded":"2023-10-10T20:41:08.214000+0000","last_fullsized":"2023-10-10T20:41:08.214000+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:17.174004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.220016+0000","last_change":"2023-10-10T20:41:08.216380+0000","last_active":"2023-10-10T20:41:08.220016+0000","last_peered":"2023-10-10T20:41:08.220016+0000","last_clean":"2023-10-10T20:41:08.220016+0000","last_became_active":"2023-10-10T20:41:08.216135+0000","last_became_peered":"2023-10-10T20:41:08.216135+0000","last_unstale":"2023-10-10T20:41:08.220016+0000","last_undegraded":"2023-10-10T20:41:08.220016+0000","last_fullsized":"2023-10-10T20:41:08.220016+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:37:12.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.215987+0000","last_change":"2023-10-10T20:41:07.194646+0000","last_active":"2023-10-10T20:41:08.215987+0000","last_peered":"2023-10-10T20:41:08.215987+0000","last_clean":"2023-10-10T20:41:08.215987+0000","last_became_active":"2023-10-10T20:41:07.194426+0000","last_became_peered":"2023-10-10T20:41:07.194426+0000","last_unstale":"2023-10-10T20:41:08.215987+0000","last_undegraded":"2023-10-10T20:41:08.215987+0000","last_fullsized":"2023-10-10T20:41:08.215987+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:50:00.391334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.225180+0000","last_change":"2023-10-10T20:41:08.219296+0000","last_active":"2023-10-10T20:41:08.225180+0000","last_peered":"2023-10-10T20:41:08.225180+0000","last_clean":"2023-10-10T20:41:08.225180+0000","last_became_active":"2023-10-10T20:41:08.219066+0000","last_became_peered":"2023-10-10T20:41:08.219066+0000","last_unstale":"2023-10-10T20:41:08.225180+0000","last_undegraded":"2023-10-10T20:41:08.225180+0000","last_fullsized":"2023-10-10T20:41:08.225180+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:36:45.918177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.209837+0000","last_change":"2023-10-10T20:41:07.192740+0000","last_active":"2023-10-10T20:41:08.209837+0000","last_peered":"2023-10-10T20:41:08.209837+0000","last_clean":"2023-10-10T20:41:08.209837+0000","last_became_active":"2023-10-10T20:41:07.192480+0000","last_became_peered":"2023-10-10T20:41:07.192480+0000","last_unstale":"2023-10-10T20:41:08.209837+0000","last_undegraded":"2023-10-10T20:41:08.209837+0000","last_fullsized":"2023-10-10T20:41:08.209837+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:48.161107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.234205+0000","last_change":"2023-10-10T20:41:08.231186+0000","last_active":"2023-10-10T20:41:08.234205+0000","last_peered":"2023-10-10T20:41:08.234205+0000","last_clean":"2023-10-10T20:41:08.234205+0000","last_became_active":"2023-10-10T20:41:08.227698+0000","last_became_peered":"2023-10-10T20:41:08.227698+0000","last_unstale":"2023-10-10T20:41:08.234205+0000","last_undegraded":"2023-10-10T20:41:08.234205+0000","last_fullsized":"2023-10-10T20:41:08.234205+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:06:14.543497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"111'2","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.223687+0000","last_change":"2023-10-10T20:41:07.194467+0000","last_active":"2023-10-10T20:41:08.223687+0000","last_peered":"2023-10-10T20:41:08.223687+0000","last_clean":"2023-10-10T20:41:08.223687+0000","last_became_active":"2023-10-10T20:41:07.193987+0000","last_became_peered":"2023-10-10T20:41:07.193987+0000","last_unstale":"2023-10-10T20:41:08.223687+0000","last_undegraded":"2023-10-10T20:41:08.223687+0000","last_fullsized":"2023-10-10T20:41:08.223687+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:28:43.862933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.234036+0000","last_change":"2023-10-10T20:41:08.229557+0000","last_active":"2023-10-10T20:41:08.234036+0000","last_peered":"2023-10-10T20:41:08.234036+0000","last_clean":"2023-10-10T20:41:08.234036+0000","last_became_active":"2023-10-10T20:41:08.229299+0000","last_became_peered":"2023-10-10T20:41:08.229299+0000","last_unstale":"2023-10-10T20:41:08.234036+0000","last_undegraded":"2023-10-10T20:41:08.234036+0000","last_fullsized":"2023-10-10T20:41:08.234036+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:46:02.946387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.206821+0000","last_change":"2023-10-10T20:41:07.196552+0000","last_active":"2023-10-10T20:41:08.206821+0000","last_peered":"2023-10-10T20:41:08.206821+0000","last_clean":"2023-10-10T20:41:08.206821+0000","last_became_active":"2023-10-10T20:41:07.196285+0000","last_became_peered":"2023-10-10T20:41:07.196285+0000","last_unstale":"2023-10-10T20:41:08.206821+0000","last_undegraded":"2023-10-10T20:41:08.206821+0000","last_fullsized":"2023-10-10T20:41:08.206821+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:43:53.666694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.220484+0000","last_change":"2023-10-10T20:41:08.217190+0000","last_active":"2023-10-10T20:41:08.220484+0000","last_peered":"2023-10-10T20:41:08.220484+0000","last_clean":"2023-10-10T20:41:08.220484+0000","last_became_active":"2023-10-10T20:41:08.216613+0000","last_became_peered":"2023-10-10T20:41:08.216613+0000","last_unstale":"2023-10-10T20:41:08.220484+0000","last_undegraded":"2023-10-10T20:41:08.220484+0000","last_fullsized":"2023-10-10T20:41:08.220484+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:44.120514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.212887+0000","last_change":"2023-10-10T20:41:07.192768+0000","last_active":"2023-10-10T20:41:08.212887+0000","last_peered":"2023-10-10T20:41:08.212887+0000","last_clean":"2023-10-10T20:41:08.212887+0000","last_became_active":"2023-10-10T20:41:07.192467+0000","last_became_peered":"2023-10-10T20:41:07.192467+0000","last_unstale":"2023-10-10T20:41:08.212887+0000","last_undegraded":"2023-10-10T20:41:08.212887+0000","last_fullsized":"2023-10-10T20:41:08.212887+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:59.684638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.227086+0000","last_change":"2023-10-10T20:41:08.223957+0000","last_active":"2023-10-10T20:41:08.227086+0000","last_peered":"2023-10-10T20:41:08.227086+0000","last_clean":"2023-10-10T20:41:08.227086+0000","last_became_active":"2023-10-10T20:41:08.223685+0000","last_became_peered":"2023-10-10T20:41:08.223685+0000","last_unstale":"2023-10-10T20:41:08.227086+0000","last_undegraded":"2023-10-10T20:41:08.227086+0000","last_fullsized":"2023-10-10T20:41:08.227086+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:42:34.466499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.209711+0000","last_change":"2023-10-10T20:41:07.186703+0000","last_active":"2023-10-10T20:41:08.209711+0000","last_peered":"2023-10-10T20:41:08.209711+0000","last_clean":"2023-10-10T20:41:08.209711+0000","last_became_active":"2023-10-10T20:41:07.186447+0000","last_became_peered":"2023-10-10T20:41:07.186447+0000","last_unstale":"2023-10-10T20:41:08.209711+0000","last_undegraded":"2023-10-10T20:41:08.209711+0000","last_fullsized":"2023-10-10T20:41:08.209711+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:57.197462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.225252+0000","last_change":"2023-10-10T20:41:08.224205+0000","last_active":"2023-10-10T20:41:08.225252+0000","last_peered":"2023-10-10T20:41:08.225252+0000","last_clean":"2023-10-10T20:41:08.225252+0000","last_became_active":"2023-10-10T20:41:08.223920+0000","last_became_peered":"2023-10-10T20:41:08.223920+0000","last_unstale":"2023-10-10T20:41:08.225252+0000","last_undegraded":"2023-10-10T20:41:08.225252+0000","last_fullsized":"2023-10-10T20:41:08.225252+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:40:14.372905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.212063+0000","last_change":"2023-10-10T20:41:07.192864+0000","last_active":"2023-10-10T20:41:08.212063+0000","last_peered":"2023-10-10T20:41:08.212063+0000","last_clean":"2023-10-10T20:41:08.212063+0000","last_became_active":"2023-10-10T20:41:07.192602+0000","last_became_peered":"2023-10-10T20:41:07.192602+0000","last_unstale":"2023-10-10T20:41:08.212063+0000","last_undegraded":"2023-10-10T20:41:08.212063+0000","last_fullsized":"2023-10-10T20:41:08.212063+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:37:19.121105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.229051+0000","last_change":"2023-10-10T20:41:08.225511+0000","last_active":"2023-10-10T20:41:08.229051+0000","last_peered":"2023-10-10T20:41:08.229051+0000","last_clean":"2023-10-10T20:41:08.229051+0000","last_became_active":"2023-10-10T20:41:08.225147+0000","last_became_peered":"2023-10-10T20:41:08.225147+0000","last_unstale":"2023-10-10T20:41:08.229051+0000","last_undegraded":"2023-10-10T20:41:08.229051+0000","last_fullsized":"2023-10-10T20:41:08.229051+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:39:12.818520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.210997+0000","last_change":"2023-10-10T20:41:07.192698+0000","last_active":"2023-10-10T20:41:08.210997+0000","last_peered":"2023-10-10T20:41:08.210997+0000","last_clean":"2023-10-10T20:41:08.210997+0000","last_became_active":"2023-10-10T20:41:07.192370+0000","last_became_peered":"2023-10-10T20:41:07.192370+0000","last_unstale":"2023-10-10T20:41:08.210997+0000","last_undegraded":"2023-10-10T20:41:08.210997+0000","last_fullsized":"2023-10-10T20:41:08.210997+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:15:08.116204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.234482+0000","last_change":"2023-10-10T20:41:08.232831+0000","last_active":"2023-10-10T20:41:08.234482+0000","last_peered":"2023-10-10T20:41:08.234482+0000","last_clean":"2023-10-10T20:41:08.234482+0000","last_became_active":"2023-10-10T20:41:08.232614+0000","last_became_peered":"2023-10-10T20:41:08.232614+0000","last_unstale":"2023-10-10T20:41:08.234482+0000","last_undegraded":"2023-10-10T20:41:08.234482+0000","last_fullsized":"2023-10-10T20:41:08.234482+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:40.934181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.216085+0000","last_change":"2023-10-10T20:41:07.194285+0000","last_active":"2023-10-10T20:41:08.216085+0000","last_peered":"2023-10-10T20:41:08.216085+0000","last_clean":"2023-10-10T20:41:08.216085+0000","last_became_active":"2023-10-10T20:41:07.194002+0000","last_became_peered":"2023-10-10T20:41:07.194002+0000","last_unstale":"2023-10-10T20:41:08.216085+0000","last_undegraded":"2023-10-10T20:41:08.216085+0000","last_fullsized":"2023-10-10T20:41:08.216085+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:42:33.465052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.226862+0000","last_change":"2023-10-10T20:41:08.220919+0000","last_active":"2023-10-10T20:41:08.226862+0000","last_peered":"2023-10-10T20:41:08.226862+0000","last_clean":"2023-10-10T20:41:08.226862+0000","last_became_active":"2023-10-10T20:41:08.220639+0000","last_became_peered":"2023-10-10T20:41:08.220639+0000","last_unstale":"2023-10-10T20:41:08.226862+0000","last_undegraded":"2023-10-10T20:41:08.226862+0000","last_fullsized":"2023-10-10T20:41:08.226862+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:15:09.117651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.209923+0000","last_change":"2023-10-10T20:41:07.186005+0000","last_active":"2023-10-10T20:41:08.209923+0000","last_peered":"2023-10-10T20:41:08.209923+0000","last_clean":"2023-10-10T20:41:08.209923+0000","last_became_active":"2023-10-10T20:41:07.185723+0000","last_became_peered":"2023-10-10T20:41:07.185723+0000","last_unstale":"2023-10-10T20:41:08.209923+0000","last_undegraded":"2023-10-10T20:41:08.209923+0000","last_fullsized":"2023-10-10T20:41:08.209923+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:39.932733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.224816+0000","last_change":"2023-10-10T20:41:08.219395+0000","last_active":"2023-10-10T20:41:08.224816+0000","last_peered":"2023-10-10T20:41:08.224816+0000","last_clean":"2023-10-10T20:41:08.224816+0000","last_became_active":"2023-10-10T20:41:08.218578+0000","last_became_peered":"2023-10-10T20:41:08.218578+0000","last_unstale":"2023-10-10T20:41:08.224816+0000","last_undegraded":"2023-10-10T20:41:08.224816+0000","last_fullsized":"2023-10-10T20:41:08.224816+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:07.169206+0000","last_clean_scrub_stamp":"2023-10-10T20:41:07.169206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:51.136287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"111'2","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.222388+0000","last_change":"2023-10-10T20:41:07.195197+0000","last_active":"2023-10-10T20:41:08.222388+0000","last_peered":"2023-10-10T20:41:08.222388+0000","last_clean":"2023-10-10T20:41:08.222388+0000","last_became_active":"2023-10-10T20:41:07.194893+0000","last_became_peered":"2023-10-10T20:41:07.194893+0000","last_unstale":"2023-10-10T20:41:08.222388+0000","last_undegraded":"2023-10-10T20:41:08.222388+0000","last_fullsized":"2023-10-10T20:41:08.222388+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:06.167758+0000","last_clean_scrub_stamp":"2023-10-10T20:41:06.167758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:51:13.572175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":255,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-10T20:41:08.206182+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:41:08.206182+0000","last_peered":"2023-10-10T20:41:08.206182+0000","last_clean":"2023-10-10T20:41:08.206182+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:41:08.206182+0000","last_undegraded":"2023-10-10T20:41:08.206182+0000","last_fullsized":"2023-10-10T20:41:08.206182+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303356,"kb_used_data":1972,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068484,"statfs":{"total":96636764160,"available":96326127616,"internally_reserved":0,"allocated":2019328,"data_stored":1198544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51500000000000001,"15min":0.81999999999999995},"min":{"1min":0.31,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.496},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53000000000000003,"15min":0.627},"min":{"1min":0.32200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71999999999999997,"5min":0.95399999999999996,"15min":4.1959999999999997},"last":0.46300000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.504,"15min":1},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.67800000000000005,"5min":0.78800000000000003,"15min":10.1},"last":0.621},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55800000000000005,"15min":0.72999999999999998},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.28999999999999998},"max":{"1min":0.81799999999999995,"5min":1.006,"15min":6.806},"last":0.39700000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57899999999999996,"15min":0.78100000000000003},"min":{"1min":0.43099999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89000000000000001,"5min":1.089,"15min":6.9340000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53200000000000003,"15min":0.73799999999999999},"min":{"1min":0.39000000000000001,"5min":0.36399999999999999,"15min":0.30099999999999999},"max":{"1min":0.77100000000000002,"5min":0.85299999999999998,"15min":6.9870000000000001},"last":0.65900000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.75800000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.754,"5min":0.80200000000000005,"15min":6.6719999999999997},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.67000000000000004},"min":{"1min":0.38700000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.70599999999999996,"5min":1.2789999999999999,"15min":4.1609999999999996},"last":0.64400000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47399999999999998,"15min":0.46500000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.73199999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52400000000000002,"15min":0.52300000000000002},"min":{"1min":0.309,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.378}]},{"osd":5,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.32500000000000001,"5min":0.32100000000000001,"15min":0.314},"max":{"1min":0.86099999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75800000000000001,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60599999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72999999999999998,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84099999999999997,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.67400000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574934,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302764,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069076,"statfs":{"total":96636764160,"available":96326733824,"internally_reserved":0,"allocated":1413120,"data_stored":607566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50700000000000001,"15min":0.67200000000000004},"min":{"1min":0.33500000000000002,"5min":0.32400000000000001,"15min":0.312},"max":{"1min":0.68600000000000005,"5min":0.73099999999999998,"15min":7.7590000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52100000000000002,"15min":0.71299999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":7.8090000000000002},"last":0.33300000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.70999999999999996},"min":{"1min":0.38400000000000001,"5min":0.35599999999999998,"15min":0.27800000000000002},"max":{"1min":0.628,"5min":0.80000000000000004,"15min":7.867},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53300000000000003,"15min":0.88500000000000001},"min":{"1min":0.36599999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":9.7089999999999996},"last":0.46600000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51400000000000001,"15min":0.59599999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.72499999999999998,"5min":0.747,"15min":4.7999999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54600000000000004,"15min":0.84499999999999997},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.71499999999999997,"5min":0.80000000000000004,"15min":6.7629999999999999},"last":0.39300000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.62},"min":{"1min":0.40999999999999998,"5min":0.36299999999999999,"15min":0.35299999999999998},"max":{"1min":0.79100000000000004,"5min":0.83999999999999997,"15min":4.8620000000000001},"last":0.374},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54800000000000004,"15min":0.71699999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.35299999999999998},"max":{"1min":0.68600000000000005,"5min":1.1850000000000001,"15min":7.9009999999999998},"last":0.41899999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51500000000000001,"15min":0.46800000000000003},"min":{"1min":0.32400000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71099999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59199999999999997,"15min":0.56799999999999995},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.37},"max":{"1min":0.73499999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.40999999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52000000000000002,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.314,"15min":0.312},"max":{"1min":0.69199999999999995,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.747,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.50900000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55100000000000005,"15min":0.52000000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.72599999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57799999999999996,"15min":0.54400000000000004},"min":{"1min":0.42899999999999999,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":0.69399999999999995,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.54000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302752,"kb_used_data":1368,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069088,"statfs":{"total":96636764160,"available":96326746112,"internally_reserved":0,"allocated":1400832,"data_stored":606030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.31900000000000001,"15min":0.47599999999999998},"min":{"1min":0.20999999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.441,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.32300000000000001,"15min":0.36299999999999999},"min":{"1min":0.23599999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46000000000000002,"5min":0.68700000000000006,"15min":1.194},"last":0.38900000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.32400000000000001,"15min":0.36399999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.379,"5min":0.67400000000000004,"15min":1.2869999999999999},"last":0.314},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.33800000000000002,"15min":0.505},"min":{"1min":0.26000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.54200000000000004,"5min":0.57199999999999995,"15min":5.0919999999999996},"last":0.27200000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36299999999999999,"15min":0.52800000000000002},"min":{"1min":0.26800000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.51000000000000001,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.37},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36499999999999999,"15min":0.40699999999999997},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.48299999999999998,"5min":0.72099999999999997,"15min":1.2170000000000001},"last":0.33100000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57899999999999996,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.81000000000000005,"5min":1.133,"15min":1.254},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56599999999999995,"15min":0.70599999999999996},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69899999999999995,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.49299999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57899999999999996,"15min":0.59499999999999997},"min":{"1min":0.42299999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.751,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.60799999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71899999999999997,"5min":1.0720000000000001,"15min":1.3109999999999999},"last":0.46999999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60699999999999998,"15min":0.84399999999999997},"min":{"1min":0.435,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86199999999999999,"5min":0.995,"15min":5.0659999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61699999999999999,"15min":0.63200000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84599999999999997,"5min":1.1439999999999999,"15min":1.171},"last":0.51800000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59299999999999997,"15min":0.61699999999999999},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.753,"5min":1.214,"15min":1.214},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60099999999999998,"15min":0.72499999999999998},"min":{"1min":0.45000000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76900000000000002,"5min":1.115,"15min":4.0499999999999998},"last":0.54500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303328,"kb_used_data":1944,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068512,"statfs":{"total":96636764160,"available":96326156288,"internally_reserved":0,"allocated":1990656,"data_stored":1195437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.309,"15min":0.52800000000000002},"min":{"1min":0.20999999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.38300000000000001,"5min":0.51600000000000001,"15min":7.9249999999999998},"last":0.17799999999999999},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.318,"15min":0.505},"min":{"1min":0.23699999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.41899999999999998,"5min":0.54900000000000004,"15min":6.6299999999999999},"last":0.25600000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.32600000000000001,"15min":0.55200000000000005},"min":{"1min":0.22,"5min":0.216,"15min":0.216},"max":{"1min":0.48799999999999999,"5min":0.55400000000000005,"15min":8.2509999999999994},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.34000000000000002,"15min":0.52800000000000002},"min":{"1min":0.23100000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.45700000000000002,"5min":0.54100000000000004,"15min":6.7370000000000001},"last":0.30499999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.34399999999999997,"15min":0.55700000000000005},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.48299999999999998,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.61199999999999999},"min":{"1min":0.32000000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":7.8929999999999998},"last":0.27700000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55600000000000005,"15min":0.71399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.72499999999999998,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56299999999999994,"15min":0.77200000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.67600000000000005,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.55200000000000005,"15min":0.751},"min":{"1min":0.38800000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69299999999999995,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58099999999999996,"15min":0.77800000000000002},"min":{"1min":0.42299999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77000000000000002,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.53700000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60199999999999998,"15min":0.81799999999999995},"min":{"1min":0.39800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72099999999999997,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61099999999999999,"15min":0.76100000000000001},"min":{"1min":0.41399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74299999999999999,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.68400000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58799999999999997,"15min":0.752},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78300000000000003,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59099999999999997,"15min":0.78600000000000003},"min":{"1min":0.45200000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.69299999999999995,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.51300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302760,"kb_used_data":1376,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069080,"statfs":{"total":96636764160,"available":96326737920,"internally_reserved":0,"allocated":1409024,"data_stored":606618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.28299999999999997,"5min":0.309,"15min":0.32500000000000001},"min":{"1min":0.21099999999999999,"5min":0.189,"15min":0.189},"max":{"1min":0.38300000000000001,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.378},{"interface":"front","average":{"1min":0.29399999999999998,"5min":0.32100000000000001,"15min":0.35299999999999998},"min":{"1min":0.215,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.39500000000000002,"5min":0.5,"15min":0.746},"last":0.23400000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.32900000000000001,"15min":0.72899999999999998},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.35699999999999998,"5min":0.63,"15min":11.967000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.34499999999999997,"15min":0.48299999999999998},"min":{"1min":0.26800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.443,"5min":0.53400000000000003,"15min":4.1100000000000003},"last":0.32500000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.34499999999999997,"15min":0.55700000000000005},"min":{"1min":0.23699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.39400000000000002,"5min":0.55400000000000005,"15min":6.3040000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38700000000000001,"15min":0.78100000000000003},"min":{"1min":0.27900000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.46899999999999997,"5min":0.65900000000000003,"15min":11.935},"last":0.59299999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.54300000000000004,"15min":0.55500000000000005},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.58699999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71699999999999997,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.47899999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.378},"max":{"1min":0.66300000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54400000000000004,"15min":0.58399999999999996},"min":{"1min":0.39700000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74199999999999999,"5min":0.874,"15min":0.96099999999999997},"last":0.35999999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57699999999999996,"15min":0.60099999999999998},"min":{"1min":0.42199999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.745,"5min":0.91600000000000004,"15min":0.91800000000000004},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73799999999999999,"5min":0.93100000000000005,"15min":0.93600000000000005},"last":0.629}]},{"osd":7,"last update":"Tue Oct 10 20:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56499999999999995,"15min":0.58499999999999996},"min":{"1min":0.41799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71099999999999997,"5min":0.89100000000000001,"15min":1.052},"last":0.435},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57699999999999996,"15min":0.78000000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69799999999999995,"5min":0.85999999999999999,"15min":6.2649999999999997},"last":0.54400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574934,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302760,"kb_used_data":1376,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069080,"statfs":{"total":96636764160,"available":96326737920,"internally_reserved":0,"allocated":1409024,"data_stored":606190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.31,"15min":0.314},"min":{"1min":0.21299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.31900000000000001},"min":{"1min":0.25,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.48799999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.23499999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38600000000000001,"15min":0.377},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.57099999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.47199999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.34999999999999998},"min":{"1min":0.23699999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.35699999999999998,"15min":0.36199999999999999},"min":{"1min":0.26300000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.50900000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69999999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.82299999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.71099999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.35899999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55900000000000005,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.76900000000000002,"5min":1.01,"15min":1.01},"last":0.74099999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.42099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85699999999999998,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.78900000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.441,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.496,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.88400000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.83699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302756,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069084,"statfs":{"total":96636764160,"available":96326742016,"internally_reserved":0,"allocated":1404928,"data_stored":606100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68300000000000005,"5min":0.74199999999999999,"15min":0.747},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.76600000000000001},"min":{"1min":0.374,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67500000000000004,"5min":0.78400000000000003,"15min":7.6719999999999997},"last":0.45600000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52900000000000003,"15min":0.77400000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.755,"5min":0.755,"15min":7.7160000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.497,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65300000000000002,"5min":0.76900000000000002,"15min":0.86799999999999999},"last":0.5}]},{"osd":2,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.65300000000000002,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56299999999999994,"15min":0.79300000000000004},"min":{"1min":0.39700000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.72799999999999998,"5min":0.79800000000000004,"15min":7.7839999999999998},"last":0.56699999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55200000000000005,"15min":0.54900000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65500000000000003,"5min":0.73399999999999999,"15min":0.96999999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.41199999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.66100000000000003,"5min":0.78100000000000003,"15min":0.79400000000000004},"last":0.54800000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51000000000000001,"15min":0.48699999999999999},"min":{"1min":0.32500000000000001,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.77500000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54900000000000004,"15min":0.54300000000000004},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.68200000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.39200000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68799999999999994,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.35899999999999999},"max":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.628}]},{"osd":7,"last update":"Tue Oct 10 20:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.378,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.84599999999999997},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.46100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.748,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61299999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302760,"kb_used_data":1376,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069080,"statfs":{"total":96636764160,"available":96326737920,"internally_reserved":0,"allocated":1409024,"data_stored":606142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.28599999999999998,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.79300000000000004},"last":0.50900000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52700000000000002,"15min":0.64100000000000001},"min":{"1min":0.33000000000000002,"5min":0.32200000000000001,"15min":0.316},"max":{"1min":0.70399999999999996,"5min":1.2330000000000001,"15min":5.2679999999999998},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51800000000000002,"15min":0.629},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.55200000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52800000000000002,"15min":0.64800000000000002},"min":{"1min":0.35999999999999999,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.76600000000000001,"5min":0.86199999999999999,"15min":5.2960000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72899999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.53400000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55300000000000005,"15min":0.67700000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.73899999999999999,"5min":0.93000000000000005,"15min":5.3330000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.69099999999999995},"min":{"1min":0.39300000000000002,"5min":0.36899999999999999,"15min":0.27700000000000002},"max":{"1min":0.76000000000000001,"5min":0.94399999999999995,"15min":5.2850000000000001},"last":0.59699999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47799999999999998,"15min":0.496},"min":{"1min":0.311,"5min":0.29599999999999999,"15min":0.28699999999999998},"max":{"1min":0.72499999999999998,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34300000000000003,"15min":0.32200000000000001},"max":{"1min":0.70799999999999996,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.66100000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54100000000000004,"15min":0.54700000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.80100000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82499999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.64600000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.378,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68899999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60199999999999998,"15min":0.60099999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83399999999999996,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:41:17.150 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:41:17.151 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:41:17.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.303+0000 7f0cce8cb700 1 Processor -- start 2023-10-10T20:41:17.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.303+0000 7f0cce8cb700 1 -- start start 2023-10-10T20:41:17.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.304+0000 7f0cce8cb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0cc806bcd0 0x7f0cc80c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:17.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.304+0000 7f0cce8cb700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0cc8158d40 0x7f0cc8140db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:17.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.304+0000 7f0cce8cb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc815b790 con 0x7f0cc806c0a0 2023-10-10T20:41:17.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.304+0000 7f0cce8cb700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc815ba70 con 0x7f0cc8158d40 2023-10-10T20:41:17.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.304+0000 7f0cce8cb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc815bd50 con 0x7f0cc806bcd0 2023-10-10T20:41:17.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.304+0000 7f0ccce68700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0cc806c0a0 0x7f0cc806cfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41386/0 (socket says 172.21.15.89:41386) 2023-10-10T20:41:17.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.304+0000 7f0ccce68700 1 -- 172.21.15.89:0/3555027705 learned_addr learned my addr 172.21.15.89:0/3555027705 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:17.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc77fe700 1 --2- 172.21.15.89:0/3555027705 >> v2:172.21.15.111:3300/0 conn(0x7f0cc8158d40 0x7f0cc8140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:17.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3555027705 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762080768 0 0) 0x7f0cc815b790 con 0x7f0cc806c0a0 2023-10-10T20:41:17.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc7fff700 1 --2- 172.21.15.89:0/3555027705 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0cc806bcd0 0x7f0cc80c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:17.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3555027705 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cb0003570 con 0x7f0cc806c0a0 2023-10-10T20:41:17.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3555027705 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2120631198 0 0) 0x7f0cb0003570 con 0x7f0cc806c0a0 2023-10-10T20:41:17.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3555027705 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0cc806bcd0 msgr2=0x7f0cc80c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:17.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc6ffd700 1 --2- 172.21.15.89:0/3555027705 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0cc806bcd0 0x7f0cc80c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3555027705 >> v2:172.21.15.111:3300/0 conn(0x7f0cc8158d40 msgr2=0x7f0cc8140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:17.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc6ffd700 1 --2- 172.21.15.89:0/3555027705 >> v2:172.21.15.111:3300/0 conn(0x7f0cc8158d40 0x7f0cc8140db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3555027705 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc815c030 con 0x7f0cc806c0a0 2023-10-10T20:41:17.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3555027705 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f0cc00017f0 con 0x7f0cc806c0a0 2023-10-10T20:41:17.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3555027705 >> v1:172.21.15.89:6789/0 conn(0x7f0cc806c0a0 legacy=0x7f0cc806cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:17.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc6ffd700 1 --2- 172.21.15.89:0/3555027705 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0cc806bcd0 0x7f0cb0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:17.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc6ffd700 1 --2- 172.21.15.89:0/3555027705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cb0003fe0 0x7f0cb0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:17.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc6ffd700 1 --2- 172.21.15.89:0/3555027705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cb0004d50 0x7f0cb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:17.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3555027705 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc815ba70 con 0x7f0cb0004d50 2023-10-10T20:41:17.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3555027705 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc815bd50 con 0x7f0cb0003fe0 2023-10-10T20:41:17.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3555027705 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb000ba90 con 0x7f0cc806bcd0 2023-10-10T20:41:17.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0ccce68700 1 --2- 172.21.15.89:0/3555027705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cb0004d50 0x7f0cb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:17.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc7fff700 1 --2- 172.21.15.89:0/3555027705 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0cc806bcd0 0x7f0cb0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:17.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0cc77fe700 1 --2- 172.21.15.89:0/3555027705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cb0003fe0 0x7f0cb0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:17.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0ccce68700 1 -- 172.21.15.89:0/3555027705 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0cc806bcd0 msgr2=0x7f0cb0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:17.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0ccce68700 1 --2- 172.21.15.89:0/3555027705 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0cc806bcd0 0x7f0cb0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0ccce68700 1 -- 172.21.15.89:0/3555027705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cb0003fe0 msgr2=0x7f0cb0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:17.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0ccce68700 1 --2- 172.21.15.89:0/3555027705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cb0003fe0 0x7f0cb0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.305+0000 7f0ccce68700 1 -- 172.21.15.89:0/3555027705 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb0003570 con 0x7f0cb0004d50 2023-10-10T20:41:17.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.306+0000 7f0ccce68700 1 --2- 172.21.15.89:0/3555027705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cb0004d50 0x7f0cb00051c0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000a970 tx=0x7f0cc80ca050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4e382255589f68 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:17.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.306+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3555027705 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00070d0 con 0x7f0cb0004d50 2023-10-10T20:41:17.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.306+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3555027705 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc0007230 con 0x7f0cb0004d50 2023-10-10T20:41:17.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.306+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3555027705 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00075f0 con 0x7f0cb0004d50 2023-10-10T20:41:17.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.306+0000 7f0cce8cb700 1 -- 172.21.15.89:0/3555027705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cb0004d50 msgr2=0x7f0cb00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:17.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.306+0000 7f0cce8cb700 1 --2- 172.21.15.89:0/3555027705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cb0004d50 0x7f0cb00051c0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000a970 tx=0x7f0cc80ca050 comp rx=0 tx=0).stop 2023-10-10T20:41:17.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.306+0000 7f0ccce68700 1 -- 172.21.15.89:0/3555027705 reap_dead start 2023-10-10T20:41:17.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.306+0000 7f0cce8cb700 1 -- 172.21.15.89:0/3555027705 shutdown_connections 2023-10-10T20:41:17.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.306+0000 7f0cce8cb700 1 -- 172.21.15.89:0/3555027705 >> 172.21.15.89:0/3555027705 conn(0x7f0cc80ce5c0 msgr2=0x7f0cc80ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:17.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.306+0000 7f0cce8cb700 1 -- 172.21.15.89:0/3555027705 shutdown_connections 2023-10-10T20:41:17.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.306+0000 7f0cce8cb700 1 -- 172.21.15.89:0/3555027705 wait complete. 2023-10-10T20:41:17.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.307+0000 7f0cce8cb700 1 Processor -- start 2023-10-10T20:41:17.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.308+0000 7f0cce8cb700 1 -- start start 2023-10-10T20:41:17.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.308+0000 7f0cce8cb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0cc813f4a0 0x7f0cc80c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:17.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.308+0000 7f0cce8cb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cc813f870 0x7f0cc80c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:17.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.308+0000 7f0cce8cb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cc80c6020 0x7f0cc80c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:17.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.308+0000 7f0cce8cb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc8168dc0 con 0x7f0cc813f870 2023-10-10T20:41:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.308+0000 7f0cce8cb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc80c8c20 con 0x7f0cc80c6020 2023-10-10T20:41:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.308+0000 7f0cce8cb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc80c8f00 con 0x7f0cc813f4a0 2023-10-10T20:41:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.308+0000 7f0cc77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cc813f870 0x7f0cc80c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.308+0000 7f0cc77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cc813f870 0x7f0cc80c6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49650/0 (socket says 172.21.15.89:49650) 2023-10-10T20:41:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.308+0000 7f0cc77fe700 1 -- 172.21.15.89:0/539103455 learned_addr learned my addr 172.21.15.89:0/539103455 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.308+0000 7f0cc7fff700 1 --2- 172.21.15.89:0/539103455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0cc813f4a0 0x7f0cc80c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.308+0000 7f0ccce68700 1 --2- 172.21.15.89:0/539103455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cc80c6020 0x7f0cc80c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.309+0000 7f0cc77fe700 1 -- 172.21.15.89:0/539103455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0cc813f4a0 msgr2=0x7f0cc80c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.309+0000 7f0cc77fe700 1 --2- 172.21.15.89:0/539103455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0cc813f4a0 0x7f0cc80c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.309+0000 7f0cc77fe700 1 -- 172.21.15.89:0/539103455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cc80c6020 msgr2=0x7f0cc80c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:17.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.309+0000 7f0cc77fe700 1 --2- 172.21.15.89:0/539103455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cc80c6020 0x7f0cc80c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.309+0000 7f0cc77fe700 1 -- 172.21.15.89:0/539103455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc80c91e0 con 0x7f0cc813f870 2023-10-10T20:41:17.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.309+0000 7f0cc77fe700 1 --2- 172.21.15.89:0/539103455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cc813f870 0x7f0cc80c6780 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009fd0 tx=0x7f0cbc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:17.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.309+0000 7f0caffff700 1 -- 172.21.15.89:0/539103455 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc00f940 con 0x7f0cc813f870 2023-10-10T20:41:17.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.309+0000 7f0caffff700 1 -- 172.21.15.89:0/539103455 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cbc00faa0 con 0x7f0cc813f870 2023-10-10T20:41:17.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.309+0000 7f0cce8cb700 1 -- 172.21.15.89:0/539103455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc80b18a0 con 0x7f0cc813f870 2023-10-10T20:41:17.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.309+0000 7f0caffff700 1 -- 172.21.15.89:0/539103455 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc00fe20 con 0x7f0cc813f870 2023-10-10T20:41:17.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.309+0000 7f0cce8cb700 1 -- 172.21.15.89:0/539103455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc80c88e0 con 0x7f0cc813f870 2023-10-10T20:41:17.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.310+0000 7f0cce8cb700 1 -- 172.21.15.89:0/539103455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc80718b0 con 0x7f0cc813f870 2023-10-10T20:41:17.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.310+0000 7f0caffff700 1 -- 172.21.15.89:0/539103455 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0cbc014650 con 0x7f0cc813f870 2023-10-10T20:41:17.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.311+0000 7f0caffff700 1 --2- 172.21.15.89:0/539103455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0cb4067320 0x7f0cb40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:17.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.311+0000 7f0caffff700 1 -- 172.21.15.89:0/539103455 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0cbc018030 con 0x7f0cc813f870 2023-10-10T20:41:17.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.311+0000 7f0cc7fff700 1 --2- 172.21.15.89:0/539103455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0cb4067320 0x7f0cb40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:17.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.311+0000 7f0cc7fff700 1 --2- 172.21.15.89:0/539103455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0cb4067320 0x7f0cb40697d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8006f00 tx=0x7f0cb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:17.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.315+0000 7f0caffff700 1 -- 172.21.15.89:0/539103455 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0cbc052320 con 0x7f0cc813f870 2023-10-10T20:41:17.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.571+0000 7f0cce8cb700 1 -- 172.21.15.89:0/539103455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0cc80cf1f0 con 0x7f0cc813f870 2023-10-10T20:41:17.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.575+0000 7f0caffff700 1 -- 172.21.15.89:0/539103455 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+0 (secure 0 0 0) 0x7f0cbc055940 con 0x7f0cc813f870 2023-10-10T20:41:17.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.577+0000 7f0cce8cb700 1 -- 172.21.15.89:0/539103455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0cb4067320 msgr2=0x7f0cb40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:17.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.577+0000 7f0cce8cb700 1 --2- 172.21.15.89:0/539103455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0cb4067320 0x7f0cb40697d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8006f00 tx=0x7f0cb8008040 comp rx=0 tx=0).stop 2023-10-10T20:41:17.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.577+0000 7f0cce8cb700 1 -- 172.21.15.89:0/539103455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cc813f870 msgr2=0x7f0cc80c6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:17.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.577+0000 7f0cce8cb700 1 --2- 172.21.15.89:0/539103455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cc813f870 0x7f0cc80c6780 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009fd0 tx=0x7f0cbc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.577+0000 7f0cce8cb700 1 -- 172.21.15.89:0/539103455 shutdown_connections 2023-10-10T20:41:17.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.577+0000 7f0cce8cb700 1 --2- 172.21.15.89:0/539103455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0cb4067320 0x7f0cb40697d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.577+0000 7f0cce8cb700 1 --2- 172.21.15.89:0/539103455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cc80c6020 0x7f0cc80c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.577+0000 7f0cce8cb700 1 --2- 172.21.15.89:0/539103455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0cc813f4a0 0x7f0cc80c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.577+0000 7f0cce8cb700 1 --2- 172.21.15.89:0/539103455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cc813f870 0x7f0cc80c6780 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.577+0000 7f0cce8cb700 1 -- 172.21.15.89:0/539103455 >> 172.21.15.89:0/539103455 conn(0x7f0cc80ce5c0 msgr2=0x7f0cc815aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:17.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.578+0000 7f0cce8cb700 1 -- 172.21.15.89:0/539103455 shutdown_connections 2023-10-10T20:41:17.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.578+0000 7f0cce8cb700 1 -- 172.21.15.89:0/539103455 wait complete. 2023-10-10T20:41:17.583 INFO:teuthology.orchestra.run.smithi089.stderr:updated caps for client.0 2023-10-10T20:41:17.591 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:41:17.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.704+0000 7f83263d3700 1 Processor -- start 2023-10-10T20:41:17.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.704+0000 7f83263d3700 1 -- start start 2023-10-10T20:41:17.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.705+0000 7f83263d3700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f83200b1420 0x7f83200ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:17.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.705+0000 7f83263d3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f83200ccf90 0x7f83200c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:17.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.705+0000 7f83263d3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83200cc300 con 0x7f83200cbf30 2023-10-10T20:41:17.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.705+0000 7f83263d3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8320156bf0 con 0x7f83200ccf90 2023-10-10T20:41:17.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.705+0000 7f83263d3700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83200ae950 con 0x7f83200b1420 2023-10-10T20:41:17.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.705+0000 7f831ffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f83200cbf30 0x7f83200b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41388/0 (socket says 172.21.15.89:41388) 2023-10-10T20:41:17.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.705+0000 7f831ffff700 1 -- 172.21.15.89:0/4058673102 learned_addr learned my addr 172.21.15.89:0/4058673102 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:17.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.705+0000 7f8324970700 1 --2- 172.21.15.89:0/4058673102 >> v2:172.21.15.89:3300/0 conn(0x7f83200ccf90 0x7f83200c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:17.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.705+0000 7f831f7fe700 1 --2- 172.21.15.89:0/4058673102 >> v2:172.21.15.111:3300/0 conn(0x7f83200b1420 0x7f83200ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:17.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.706+0000 7f831effd700 1 -- 172.21.15.89:0/4058673102 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2756877547 0 0) 0x7f83200cc300 con 0x7f83200cbf30 2023-10-10T20:41:17.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.706+0000 7f831effd700 1 -- 172.21.15.89:0/4058673102 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8308003570 con 0x7f83200cbf30 2023-10-10T20:41:17.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.706+0000 7f8324970700 1 -- 172.21.15.89:0/4058673102 >> v2:172.21.15.111:3300/0 conn(0x7f83200b1420 msgr2=0x7f83200ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:17.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.706+0000 7f8324970700 1 --2- 172.21.15.89:0/4058673102 >> v2:172.21.15.111:3300/0 conn(0x7f83200b1420 0x7f83200ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.706+0000 7f8324970700 1 -- 172.21.15.89:0/4058673102 >> v1:172.21.15.89:6789/0 conn(0x7f83200cbf30 legacy=0x7f83200b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:17.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.706+0000 7f8324970700 1 -- 172.21.15.89:0/4058673102 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83200ad370 con 0x7f83200ccf90 2023-10-10T20:41:17.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.706+0000 7f8324970700 1 --2- 172.21.15.89:0/4058673102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83200ccf90 0x7f83200c9740 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f83200c9280 tx=0x7f8318009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50f5b13920f0b3b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:17.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.706+0000 7f831effd700 1 -- 172.21.15.89:0/4058673102 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831800e070 con 0x7f83200ccf90 2023-10-10T20:41:17.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.706+0000 7f831effd700 1 -- 172.21.15.89:0/4058673102 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831800bb20 con 0x7f83200ccf90 2023-10-10T20:41:17.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.706+0000 7f831effd700 1 -- 172.21.15.89:0/4058673102 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831800bea0 con 0x7f83200ccf90 2023-10-10T20:41:17.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.706+0000 7f83263d3700 1 -- 172.21.15.89:0/4058673102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83200ccf90 msgr2=0x7f83200c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:17.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.706+0000 7f83263d3700 1 --2- 172.21.15.89:0/4058673102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83200ccf90 0x7f83200c9740 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f83200c9280 tx=0x7f8318009780 comp rx=0 tx=0).stop 2023-10-10T20:41:17.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.706+0000 7f83263d3700 1 -- 172.21.15.89:0/4058673102 shutdown_connections 2023-10-10T20:41:17.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.706+0000 7f83263d3700 1 --2- 172.21.15.89:0/4058673102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83200ccf90 0x7f83200c9740 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.706+0000 7f83263d3700 1 --2- 172.21.15.89:0/4058673102 >> v2:172.21.15.111:3300/0 conn(0x7f83200b1420 0x7f83200ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.706+0000 7f83263d3700 1 -- 172.21.15.89:0/4058673102 >> 172.21.15.89:0/4058673102 conn(0x7f832006a320 msgr2=0x7f83200cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:17.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.707+0000 7f83263d3700 1 -- 172.21.15.89:0/4058673102 shutdown_connections 2023-10-10T20:41:17.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.707+0000 7f83263d3700 1 -- 172.21.15.89:0/4058673102 wait complete. 2023-10-10T20:41:17.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.708+0000 7f83263d3700 1 Processor -- start 2023-10-10T20:41:17.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.708+0000 7f83263d3700 1 -- start start 2023-10-10T20:41:17.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.708+0000 7f83263d3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83200b1420 0x7f83201698c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:17.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.708+0000 7f83263d3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83200cbf30 0x7f8320152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:17.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.708+0000 7f83263d3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83200ccf90 0x7f83200ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:17.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.708+0000 7f83263d3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8320153430 con 0x7f83200cbf30 2023-10-10T20:41:17.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.708+0000 7f83263d3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8320169f00 con 0x7f83200b1420 2023-10-10T20:41:17.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.708+0000 7f83263d3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832016a1e0 con 0x7f83200ccf90 2023-10-10T20:41:17.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.709+0000 7f831f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83200cbf30 0x7f8320152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:17.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.709+0000 7f831f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83200cbf30 0x7f8320152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49674/0 (socket says 172.21.15.89:49674) 2023-10-10T20:41:17.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.709+0000 7f831f7fe700 1 -- 172.21.15.89:0/1881587335 learned_addr learned my addr 172.21.15.89:0/1881587335 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:17.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.709+0000 7f831f7fe700 1 -- 172.21.15.89:0/1881587335 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83200ccf90 msgr2=0x7f83200ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:17.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.709+0000 7f831f7fe700 1 --2- 172.21.15.89:0/1881587335 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83200ccf90 0x7f83200ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.709+0000 7f831f7fe700 1 -- 172.21.15.89:0/1881587335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83200b1420 msgr2=0x7f83201698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:17.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.709+0000 7f831f7fe700 1 --2- 172.21.15.89:0/1881587335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83200b1420 0x7f83201698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.709+0000 7f831f7fe700 1 -- 172.21.15.89:0/1881587335 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8318009a20 con 0x7f83200cbf30 2023-10-10T20:41:17.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.709+0000 7f831f7fe700 1 --2- 172.21.15.89:0/1881587335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83200cbf30 0x7f8320152ef0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f831400d890 tx=0x7f8314008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:17.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.709+0000 7f8307fff700 1 -- 172.21.15.89:0/1881587335 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83140125f0 con 0x7f83200cbf30 2023-10-10T20:41:17.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.709+0000 7f8307fff700 1 -- 172.21.15.89:0/1881587335 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8314012750 con 0x7f83200cbf30 2023-10-10T20:41:17.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.709+0000 7f83263d3700 1 -- 172.21.15.89:0/1881587335 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832016a4c0 con 0x7f83200cbf30 2023-10-10T20:41:17.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.709+0000 7f8307fff700 1 -- 172.21.15.89:0/1881587335 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8314012b10 con 0x7f83200cbf30 2023-10-10T20:41:17.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.709+0000 7f83263d3700 1 -- 172.21.15.89:0/1881587335 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832016aaf0 con 0x7f83200cbf30 2023-10-10T20:41:17.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.710+0000 7f83263d3700 1 -- 172.21.15.89:0/1881587335 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832016ad10 con 0x7f83200cbf30 2023-10-10T20:41:17.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.710+0000 7f8307fff700 1 -- 172.21.15.89:0/1881587335 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f831401e460 con 0x7f83200cbf30 2023-10-10T20:41:17.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.711+0000 7f8307fff700 1 --2- 172.21.15.89:0/1881587335 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f830c067320 0x7f830c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:17.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.711+0000 7f8307fff700 1 -- 172.21.15.89:0/1881587335 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8314085350 con 0x7f83200cbf30 2023-10-10T20:41:17.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.711+0000 7f831ffff700 1 --2- 172.21.15.89:0/1881587335 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f830c067320 0x7f830c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:17.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.711+0000 7f831ffff700 1 --2- 172.21.15.89:0/1881587335 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f830c067320 0x7f830c0697d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f83100058d0 tx=0x7f83100079d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:17.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.715+0000 7f8307fff700 1 -- 172.21.15.89:0/1881587335 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8314051fe0 con 0x7f83200cbf30 2023-10-10T20:41:17.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.940+0000 7f83263d3700 1 -- 172.21.15.89:0/1881587335 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f832004c3c0 con 0x7f83200cbf30 2023-10-10T20:41:17.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.941+0000 7f8307fff700 1 -- 172.21.15.89:0/1881587335 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f8314055600 con 0x7f83200cbf30 2023-10-10T20:41:17.946 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:17.946 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":73,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6517,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/2658508419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2658508419},{"type":"v1","addr":"172.21.15.111:6834","nonce":2658508419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7185,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3349522977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3349522977},{"type":"v1","addr":"172.21.15.89:6835","nonce":3349522977}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7188,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3182024657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3182024657},{"type":"v1","addr":"172.21.15.89:6837","nonce":3182024657}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:41:07.565995+0000","modified":"2023-10-10T20:41:11.225905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"d","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/2582059153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2582059153},{"type":"v1","addr":"172.21.15.111:6835","nonce":2582059153}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-10T20:41:17.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.943+0000 7f83263d3700 1 -- 172.21.15.89:0/1881587335 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f830c067320 msgr2=0x7f830c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:17.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.943+0000 7f83263d3700 1 --2- 172.21.15.89:0/1881587335 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f830c067320 0x7f830c0697d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f83100058d0 tx=0x7f83100079d0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.943+0000 7f83263d3700 1 -- 172.21.15.89:0/1881587335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83200cbf30 msgr2=0x7f8320152ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:17.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.943+0000 7f83263d3700 1 --2- 172.21.15.89:0/1881587335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83200cbf30 0x7f8320152ef0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f831400d890 tx=0x7f8314008040 comp rx=0 tx=0).stop 2023-10-10T20:41:17.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.943+0000 7f83263d3700 1 -- 172.21.15.89:0/1881587335 shutdown_connections 2023-10-10T20:41:17.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.943+0000 7f83263d3700 1 --2- 172.21.15.89:0/1881587335 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83200ccf90 0x7f83200ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.943+0000 7f83263d3700 1 --2- 172.21.15.89:0/1881587335 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f830c067320 0x7f830c0697d0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.943+0000 7f83263d3700 1 --2- 172.21.15.89:0/1881587335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83200b1420 0x7f83201698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.943+0000 7f83263d3700 1 --2- 172.21.15.89:0/1881587335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83200cbf30 0x7f8320152ef0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:17.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.943+0000 7f83263d3700 1 -- 172.21.15.89:0/1881587335 >> 172.21.15.89:0/1881587335 conn(0x7f832006a320 msgr2=0x7f83200b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:17.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.943+0000 7f83263d3700 1 -- 172.21.15.89:0/1881587335 shutdown_connections 2023-10-10T20:41:17.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:17.943+0000 7f83263d3700 1 -- 172.21.15.89:0/1881587335 wait complete. 2023-10-10T20:41:17.951 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 73 2023-10-10T20:41:18.360 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:41:07.565995+0000', 'modified': '2023-10-10T20:41:11.225905+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7200}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7200': {'gid': 7200, 'name': 'd', 'rank': 0, 'incarnation': 69, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6835/2582059153', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6833', 'nonce': 2582059153}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 2582059153}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:41:18.361 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:41:18.361 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-10T20:41:18.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.476+0000 7f5e5dafd700 1 Processor -- start 2023-10-10T20:41:18.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.476+0000 7f5e5dafd700 1 -- start start 2023-10-10T20:41:18.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.477+0000 7f5e5dafd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e5815d230 0x7f5e5815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.477+0000 7f5e5dafd700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e580b2980 con 0x7f5e58159420 2023-10-10T20:41:18.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.477+0000 7f5e5dafd700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e5815dbb0 con 0x7f5e581597f0 2023-10-10T20:41:18.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.477+0000 7f5e5dafd700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e580b11d0 con 0x7f5e5815d230 2023-10-10T20:41:18.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.477+0000 7f5e5cafb700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5e58159420 0x7f5e580b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45344/0 (socket says 172.21.15.89:45344) 2023-10-10T20:41:18.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.477+0000 7f5e5cafb700 1 -- 172.21.15.89:0/3020249829 learned_addr learned my addr 172.21.15.89:0/3020249829 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:18.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.477+0000 7f5e5d2fc700 1 --2- 172.21.15.89:0/3020249829 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e5815d230 0x7f5e5815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.477+0000 7f5e577fe700 1 -- 172.21.15.89:0/3020249829 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887050010 0 0) 0x7f5e580b2980 con 0x7f5e58159420 2023-10-10T20:41:18.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.477+0000 7f5e577fe700 1 -- 172.21.15.89:0/3020249829 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e44003570 con 0x7f5e58159420 2023-10-10T20:41:18.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.477+0000 7f5e577fe700 1 -- 172.21.15.89:0/3020249829 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2695234824 0 0) 0x7f5e44003570 con 0x7f5e58159420 2023-10-10T20:41:18.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.477+0000 7f5e577fe700 1 -- 172.21.15.89:0/3020249829 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e5815d230 msgr2=0x7f5e5815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:18.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.477+0000 7f5e577fe700 1 --2- 172.21.15.89:0/3020249829 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e5815d230 0x7f5e5815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.477+0000 7f5e577fe700 1 -- 172.21.15.89:0/3020249829 >> v1:172.21.15.111:6789/0 conn(0x7f5e581597f0 legacy=0x7f5e580b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:18.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.477+0000 7f5e577fe700 1 -- 172.21.15.89:0/3020249829 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e580b14b0 con 0x7f5e58159420 2023-10-10T20:41:18.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.477+0000 7f5e577fe700 1 -- 172.21.15.89:0/3020249829 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f5e50001470 con 0x7f5e58159420 2023-10-10T20:41:18.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.477+0000 7f5e577fe700 1 -- 172.21.15.89:0/3020249829 >> v1:172.21.15.89:6789/0 conn(0x7f5e58159420 legacy=0x7f5e580b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:18.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.477+0000 7f5e577fe700 1 --2- 172.21.15.89:0/3020249829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e44003a90 0x7f5e44003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.478+0000 7f5e577fe700 1 --2- 172.21.15.89:0/3020249829 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e5815d230 0x7f5e44004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.478+0000 7f5e577fe700 1 --2- 172.21.15.89:0/3020249829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e44004d50 0x7f5e440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.478+0000 7f5e577fe700 1 -- 172.21.15.89:0/3020249829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e580b11d0 con 0x7f5e44003a90 2023-10-10T20:41:18.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.478+0000 7f5e577fe700 1 -- 172.21.15.89:0/3020249829 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4400bb70 con 0x7f5e44004d50 2023-10-10T20:41:18.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.478+0000 7f5e577fe700 1 -- 172.21.15.89:0/3020249829 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4400bd30 con 0x7f5e5815d230 2023-10-10T20:41:18.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.478+0000 7f5e5cafb700 1 --2- 172.21.15.89:0/3020249829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e44003a90 0x7f5e44003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.478+0000 7f5e57fff700 1 --2- 172.21.15.89:0/3020249829 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e5815d230 0x7f5e44004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.478+0000 7f5e5d2fc700 1 --2- 172.21.15.89:0/3020249829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e44004d50 0x7f5e440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.478+0000 7f5e5cafb700 1 -- 172.21.15.89:0/3020249829 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e5815d230 msgr2=0x7f5e44004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:18.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.478+0000 7f5e5cafb700 1 --2- 172.21.15.89:0/3020249829 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e5815d230 0x7f5e44004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.478+0000 7f5e5cafb700 1 -- 172.21.15.89:0/3020249829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e44004d50 msgr2=0x7f5e440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:18.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.478+0000 7f5e5cafb700 1 --2- 172.21.15.89:0/3020249829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e44004d50 0x7f5e440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.478+0000 7f5e5cafb700 1 -- 172.21.15.89:0/3020249829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e44003570 con 0x7f5e44003a90 2023-10-10T20:41:18.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.478+0000 7f5e5cafb700 1 --2- 172.21.15.89:0/3020249829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e44003a90 0x7f5e44003ea0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000c3c0 tx=0x7f5e50001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fb486744b0a124 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:18.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.478+0000 7f5e577fe700 1 -- 172.21.15.89:0/3020249829 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5000cc90 con 0x7f5e44003a90 2023-10-10T20:41:18.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.478+0000 7f5e577fe700 1 -- 172.21.15.89:0/3020249829 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5000cdf0 con 0x7f5e44003a90 2023-10-10T20:41:18.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.478+0000 7f5e577fe700 1 -- 172.21.15.89:0/3020249829 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e50005b50 con 0x7f5e44003a90 2023-10-10T20:41:18.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.479+0000 7f5e5dafd700 1 -- 172.21.15.89:0/3020249829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e44003a90 msgr2=0x7f5e44003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:18.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.479+0000 7f5e5dafd700 1 --2- 172.21.15.89:0/3020249829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e44003a90 0x7f5e44003ea0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000c3c0 tx=0x7f5e50001e50 comp rx=0 tx=0).stop 2023-10-10T20:41:18.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.479+0000 7f5e5d2fc700 1 -- 172.21.15.89:0/3020249829 reap_dead start 2023-10-10T20:41:18.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.479+0000 7f5e5dafd700 1 -- 172.21.15.89:0/3020249829 shutdown_connections 2023-10-10T20:41:18.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.479+0000 7f5e5dafd700 1 -- 172.21.15.89:0/3020249829 >> 172.21.15.89:0/3020249829 conn(0x7f5e580d0660 msgr2=0x7f5e580d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:18.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.479+0000 7f5e5dafd700 1 -- 172.21.15.89:0/3020249829 shutdown_connections 2023-10-10T20:41:18.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.479+0000 7f5e5dafd700 1 -- 172.21.15.89:0/3020249829 wait complete. 2023-10-10T20:41:18.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.480+0000 7f5e5dafd700 1 Processor -- start 2023-10-10T20:41:18.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.480+0000 7f5e5dafd700 1 -- start start 2023-10-10T20:41:18.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e5dafd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e44003a90 0x7f5e581550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e5dafd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e580cf950 0x7f5e580cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e5dafd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e580c5470 0x7f5e580c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e5dafd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e580d02e0 con 0x7f5e580cf950 2023-10-10T20:41:18.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e5dafd700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e580c6d20 con 0x7f5e580c5470 2023-10-10T20:41:18.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e5dafd700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e580c61a0 con 0x7f5e44003a90 2023-10-10T20:41:18.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e57fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e580cf950 0x7f5e580cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e57fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e580cf950 0x7f5e580cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49026/0 (socket says 172.21.15.89:49026) 2023-10-10T20:41:18.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e57fff700 1 -- 172.21.15.89:0/228055106 learned_addr learned my addr 172.21.15.89:0/228055106 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:18.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e5cafb700 1 --2- 172.21.15.89:0/228055106 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e44003a90 0x7f5e581550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e5d2fc700 1 --2- 172.21.15.89:0/228055106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e580c5470 0x7f5e580c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e57fff700 1 -- 172.21.15.89:0/228055106 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e44003a90 msgr2=0x7f5e581550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:18.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e57fff700 1 --2- 172.21.15.89:0/228055106 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e44003a90 0x7f5e581550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e57fff700 1 -- 172.21.15.89:0/228055106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e580c5470 msgr2=0x7f5e580c58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:18.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e57fff700 1 --2- 172.21.15.89:0/228055106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e580c5470 0x7f5e580c58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e57fff700 1 -- 172.21.15.89:0/228055106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e580c6480 con 0x7f5e580cf950 2023-10-10T20:41:18.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e57fff700 1 --2- 172.21.15.89:0/228055106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e580cf950 0x7f5e580cfda0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f5e48007fd0 tx=0x7f5e48009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:18.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e54ff9700 1 -- 172.21.15.89:0/228055106 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4800ee70 con 0x7f5e580cf950 2023-10-10T20:41:18.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e54ff9700 1 -- 172.21.15.89:0/228055106 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e480032e0 con 0x7f5e580cf950 2023-10-10T20:41:18.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e5dafd700 1 -- 172.21.15.89:0/228055106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e580c6760 con 0x7f5e580cf950 2023-10-10T20:41:18.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e54ff9700 1 -- 172.21.15.89:0/228055106 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4800c5d0 con 0x7f5e580cf950 2023-10-10T20:41:18.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.481+0000 7f5e5dafd700 1 -- 172.21.15.89:0/228055106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e580c5e30 con 0x7f5e580cf950 2023-10-10T20:41:18.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.483+0000 7f5e54ff9700 1 -- 172.21.15.89:0/228055106 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5e4800c9a0 con 0x7f5e580cf950 2023-10-10T20:41:18.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.484+0000 7f5e54ff9700 1 --2- 172.21.15.89:0/228055106 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5e38067320 0x7f5e380697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.484+0000 7f5e54ff9700 1 -- 172.21.15.89:0/228055106 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e48084dc0 con 0x7f5e580cf950 2023-10-10T20:41:18.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.484+0000 7f5e5cafb700 1 --2- 172.21.15.89:0/228055106 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5e38067320 0x7f5e380697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.484+0000 7f5e5cafb700 1 --2- 172.21.15.89:0/228055106 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5e38067320 0x7f5e380697d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5e50007cd0 tx=0x7f5e50015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:18.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.485+0000 7f5e68058b80 1 Processor -- start 2023-10-10T20:41:18.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.485+0000 7f5e68058b80 1 -- start start 2023-10-10T20:41:18.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.486+0000 7f5e68058b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a9576480 0x55a1a95557d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.486+0000 7f5e68058b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a95295f0 0x55a1a9576850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.486+0000 7f5e68058b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a1a952bb80 0x55a1a952bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.486+0000 7f5e68058b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a9482a70 con 0x55a1a95295f0 2023-10-10T20:41:18.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.486+0000 7f5e68058b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a9480270 con 0x55a1a952bb80 2023-10-10T20:41:18.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.486+0000 7f5e68058b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1a947c060 con 0x55a1a9576480 2023-10-10T20:41:18.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.486+0000 7f5e57fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a95295f0 0x55a1a9576850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.486+0000 7f5e57fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a95295f0 0x55a1a9576850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49038/0 (socket says 172.21.15.89:49038) 2023-10-10T20:41:18.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.486+0000 7f5e57fff700 1 -- 172.21.15.89:0/779536044 learned_addr learned my addr 172.21.15.89:0/779536044 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:18.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.486+0000 7f5e5cafb700 1 --2- 172.21.15.89:0/779536044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a9576480 0x55a1a95557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.486+0000 7f5e57fff700 1 -- 172.21.15.89:0/779536044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a9576480 msgr2=0x55a1a95557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:18.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.486+0000 7f5e57fff700 1 --2- 172.21.15.89:0/779536044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a9576480 0x55a1a95557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.486+0000 7f5e57fff700 1 -- 172.21.15.89:0/779536044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a1a952bb80 msgr2=0x55a1a952bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:18.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.486+0000 7f5e57fff700 1 --2- 172.21.15.89:0/779536044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a1a952bb80 0x55a1a952bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.486+0000 7f5e57fff700 1 -- 172.21.15.89:0/779536044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1a9316270 con 0x55a1a95295f0 2023-10-10T20:41:18.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.487+0000 7f5e57fff700 1 --2- 172.21.15.89:0/779536044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a95295f0 0x55a1a9576850 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f5e48003660 tx=0x7f5e48055550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad1b0bd4e10dc19 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:18.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.487+0000 7f5e4dffb700 1 -- 172.21.15.89:0/779536044 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e480556d0 con 0x55a1a95295f0 2023-10-10T20:41:18.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.487+0000 7f5e4dffb700 1 -- 172.21.15.89:0/779536044 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e48058920 con 0x55a1a95295f0 2023-10-10T20:41:18.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.487+0000 7f5e4dffb700 1 -- 172.21.15.89:0/779536044 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4805c670 con 0x55a1a95295f0 2023-10-10T20:41:18.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.487+0000 7f5e68058b80 1 -- 172.21.15.89:0/779536044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a95295f0 msgr2=0x55a1a9576850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:18.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.487+0000 7f5e68058b80 1 --2- 172.21.15.89:0/779536044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a95295f0 0x55a1a9576850 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f5e48003660 tx=0x7f5e48055550 comp rx=0 tx=0).stop 2023-10-10T20:41:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.487+0000 7f5e68058b80 1 -- 172.21.15.89:0/779536044 shutdown_connections 2023-10-10T20:41:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.487+0000 7f5e68058b80 1 --2- 172.21.15.89:0/779536044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a1a952bb80 0x55a1a952bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.487+0000 7f5e68058b80 1 --2- 172.21.15.89:0/779536044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a9576480 0x55a1a95557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.487+0000 7f5e68058b80 1 --2- 172.21.15.89:0/779536044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a95295f0 0x55a1a9576850 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.487+0000 7f5e68058b80 1 -- 172.21.15.89:0/779536044 >> 172.21.15.89:0/779536044 conn(0x55a1a952d0c0 msgr2=0x55a1a952d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:18.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.487+0000 7f5e68058b80 1 -- 172.21.15.89:0/779536044 shutdown_connections 2023-10-10T20:41:18.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.487+0000 7f5e68058b80 1 -- 172.21.15.89:0/779536044 wait complete. 2023-10-10T20:41:18.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.488+0000 7f5e68058b80 1 -- start start 2023-10-10T20:41:18.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.488+0000 7f5e68058b80 1 Processor -- start 2023-10-10T20:41:18.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e68058b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a95295f0 0x55a1a95500c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e68058b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a952bb80 0x55a1a9572ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e68058b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a1a9576480 0x55a1a95574e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e68058b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a9427890 con 0x55a1a952bb80 2023-10-10T20:41:18.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e68058b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a9442ea0 con 0x55a1a9576480 2023-10-10T20:41:18.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e68058b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1a92fee30 con 0x55a1a95295f0 2023-10-10T20:41:18.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e68058b80 10 client.0 _ll_register_callbacks cb 0x55a1a94ef2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:41:18.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e57fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a952bb80 0x55a1a9572ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e57fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a952bb80 0x55a1a9572ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49054/0 (socket says 172.21.15.89:49054) 2023-10-10T20:41:18.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e57fff700 1 -- 172.21.15.89:0/4008047187 learned_addr learned my addr 172.21.15.89:0/4008047187 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:18.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e5cafb700 1 --2- 172.21.15.89:0/4008047187 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a95295f0 0x55a1a95500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e5d2fc700 1 --2- 172.21.15.89:0/4008047187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a1a9576480 0x55a1a95574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e57fff700 1 -- 172.21.15.89:0/4008047187 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a95295f0 msgr2=0x55a1a95500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:18.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e57fff700 1 --2- 172.21.15.89:0/4008047187 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a95295f0 0x55a1a95500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e57fff700 1 -- 172.21.15.89:0/4008047187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a1a9576480 msgr2=0x55a1a95574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:18.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e57fff700 1 --2- 172.21.15.89:0/4008047187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a1a9576480 0x55a1a95574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e57fff700 1 -- 172.21.15.89:0/4008047187 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e580c5e30 con 0x55a1a952bb80 2023-10-10T20:41:18.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e57fff700 1 --2- 172.21.15.89:0/4008047187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a952bb80 0x55a1a9572ff0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f5e48053a20 tx=0x7f5e4806a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc7bdf67df4da8f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:18.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:41:18.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e36ffd700 1 -- 172.21.15.89:0/4008047187 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e48081030 con 0x55a1a952bb80 2023-10-10T20:41:18.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e36ffd700 1 -- 172.21.15.89:0/4008047187 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e4805c670 con 0x55a1a952bb80 2023-10-10T20:41:18.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.489+0000 7f5e68058b80 1 -- 172.21.15.89:0/4008047187 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1a9520cd0 con 0x55a1a952bb80 2023-10-10T20:41:18.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.490+0000 7f5e36ffd700 1 -- 172.21.15.89:0/4008047187 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e48071240 con 0x55a1a952bb80 2023-10-10T20:41:18.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.490+0000 7f5e68058b80 10 client.7356 fetch_fsmap learned FSMap version 73 2023-10-10T20:41:18.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.490+0000 7f5e68058b80 1 -- 172.21.15.89:0/4008047187 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55a1a92b8fe0 con 0x55a1a952bb80 2023-10-10T20:41:18.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.490+0000 7f5e36ffd700 1 -- 172.21.15.89:0/4008047187 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a1a9520cd0 con 0x55a1a952bb80 2023-10-10T20:41:18.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.490+0000 7f5e36ffd700 1 -- 172.21.15.89:0/4008047187 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 73) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5e48054bc0 con 0x55a1a952bb80 2023-10-10T20:41:18.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.490+0000 7f5e68058b80 10 client.7356 fetch_fsmap finished waiting for FSMap version 73 2023-10-10T20:41:18.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.490+0000 7f5e68058b80 10 client.7356 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-10T20:41:18.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.490+0000 7f5e68058b80 20 client.7356 populate_metadata read hostname 'smithi089' 2023-10-10T20:41:18.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.490+0000 7f5e68058b80 1 --2- 172.21.15.89:0/4008047187 >> [v2:172.21.15.111:6833/2582059153,v1:172.21.15.111:6835/2582059153] conn(0x55a1a9557a20 0x55a1a955b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.490+0000 7f5e68058b80 4 client.7356 mds_command: new command op to 7200 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:41:18.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.490+0000 7f5e68058b80 1 -- 172.21.15.89:0/4008047187 --> [v2:172.21.15.111:6833/2582059153,v1:172.21.15.111:6835/2582059153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1a94fdaa0 con 0x55a1a9557a20 2023-10-10T20:41:18.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.490+0000 7f5e5cafb700 1 --2- 172.21.15.89:0/4008047187 >> [v2:172.21.15.111:6833/2582059153,v1:172.21.15.111:6835/2582059153] conn(0x55a1a9557a20 0x55a1a955b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.490+0000 7f5e36ffd700 10 client.7356 ms_handle_connect on v2:172.21.15.111:6833/2582059153 2023-10-10T20:41:18.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.490+0000 7f5e5cafb700 1 --2- 172.21.15.89:0/4008047187 >> [v2:172.21.15.111:6833/2582059153,v1:172.21.15.111:6835/2582059153] conn(0x55a1a9557a20 0x55a1a955b490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89fe3ec77da09542 server_cookie=3100147bb481d3c2 in_seq=0 out_seq=0 2023-10-10T20:41:18.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.491+0000 7f5e36ffd700 1 -- 172.21.15.89:0/4008047187 <== mds.0 v2:172.21.15.111:6833/2582059153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a1a94fdaa0 con 0x55a1a9557a20 2023-10-10T20:41:18.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.491+0000 7f5e36ffd700 10 client.7356 handle_command_reply: tid=0 2023-10-10T20:41:18.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.491+0000 7f5e68058b80 1 client.7356 shutdown 2023-10-10T20:41:18.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.491+0000 7f5e68058b80 1 -- 172.21.15.89:0/4008047187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a952bb80 msgr2=0x55a1a9572ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:18.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.491+0000 7f5e68058b80 1 --2- 172.21.15.89:0/4008047187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a952bb80 0x55a1a9572ff0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f5e48053a20 tx=0x7f5e4806a000 comp rx=0 tx=0).stop 2023-10-10T20:41:18.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.491+0000 7f5e68058b80 1 -- 172.21.15.89:0/4008047187 shutdown_connections 2023-10-10T20:41:18.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.491+0000 7f5e68058b80 1 --2- 172.21.15.89:0/4008047187 >> [v2:172.21.15.111:6833/2582059153,v1:172.21.15.111:6835/2582059153] conn(0x55a1a9557a20 0x55a1a955b490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.491+0000 7f5e68058b80 1 --2- 172.21.15.89:0/4008047187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a1a9576480 0x55a1a95574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.491+0000 7f5e36ffd700 0 client.7356 ms_handle_reset on v2:172.21.15.111:6833/2582059153 2023-10-10T20:41:18.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.491+0000 7f5e68058b80 1 --2- 172.21.15.89:0/4008047187 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a95295f0 0x55a1a95500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.491+0000 7f5e68058b80 1 --2- 172.21.15.89:0/4008047187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a952bb80 0x55a1a9572ff0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.491+0000 7f5e68058b80 1 -- 172.21.15.89:0/4008047187 >> 172.21.15.89:0/4008047187 conn(0x55a1a952d0c0 msgr2=0x55a1a952d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:18.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.491+0000 7f5e68058b80 1 -- 172.21.15.89:0/4008047187 shutdown_connections 2023-10-10T20:41:18.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.491+0000 7f5e68058b80 1 -- 172.21.15.89:0/4008047187 wait complete. 2023-10-10T20:41:18.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.492+0000 7f5e68058b80 20 client.7356 trim_cache size 0 max 16384 2023-10-10T20:41:18.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.508+0000 7f5e68058b80 1 Processor -- start 2023-10-10T20:41:18.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.508+0000 7f5e68058b80 1 -- start start 2023-10-10T20:41:18.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.509+0000 7f5e68058b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a95295f0 0x55a1a9528c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.509+0000 7f5e68058b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a1a952bb80 0x55a1a95584f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.509+0000 7f5e68058b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a9576480 0x55a1a9558a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.509+0000 7f5e68058b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a9594cb0 con 0x55a1a95295f0 2023-10-10T20:41:18.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.509+0000 7f5e68058b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a952fb30 con 0x55a1a952bb80 2023-10-10T20:41:18.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.509+0000 7f5e68058b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1a9504410 con 0x55a1a9576480 2023-10-10T20:41:18.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.509+0000 7f5e5cafb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a95295f0 0x55a1a9528c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.509+0000 7f5e5cafb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a95295f0 0x55a1a9528c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49068/0 (socket says 172.21.15.89:49068) 2023-10-10T20:41:18.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.509+0000 7f5e5cafb700 1 -- 172.21.15.89:0/2090506846 learned_addr learned my addr 172.21.15.89:0/2090506846 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:18.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.509+0000 7f5e57fff700 1 --2- 172.21.15.89:0/2090506846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a1a952bb80 0x55a1a95584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.509+0000 7f5e5d2fc700 1 --2- 172.21.15.89:0/2090506846 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a9576480 0x55a1a9558a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.510+0000 7f5e5cafb700 1 -- 172.21.15.89:0/2090506846 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a9576480 msgr2=0x55a1a9558a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:18.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.510+0000 7f5e5cafb700 1 --2- 172.21.15.89:0/2090506846 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a9576480 0x55a1a9558a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.510+0000 7f5e5cafb700 1 -- 172.21.15.89:0/2090506846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a1a952bb80 msgr2=0x55a1a95584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:18.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.510+0000 7f5e5cafb700 1 --2- 172.21.15.89:0/2090506846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a1a952bb80 0x55a1a95584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.510+0000 7f5e5cafb700 1 -- 172.21.15.89:0/2090506846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1a944a780 con 0x55a1a95295f0 2023-10-10T20:41:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.510+0000 7f5e5cafb700 1 --2- 172.21.15.89:0/2090506846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a95295f0 0x55a1a9528c10 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000d010 tx=0x7f5e50012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b12ee1b835143df server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.510+0000 7f5e4e7fc700 1 -- 172.21.15.89:0/2090506846 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e500017c0 con 0x55a1a95295f0 2023-10-10T20:41:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.510+0000 7f5e4e7fc700 1 -- 172.21.15.89:0/2090506846 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e50004c10 con 0x55a1a95295f0 2023-10-10T20:41:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.510+0000 7f5e4e7fc700 1 -- 172.21.15.89:0/2090506846 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e50004d80 con 0x55a1a95295f0 2023-10-10T20:41:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.510+0000 7f5e68058b80 1 -- 172.21.15.89:0/2090506846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a95295f0 msgr2=0x55a1a9528c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.510+0000 7f5e68058b80 1 --2- 172.21.15.89:0/2090506846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a95295f0 0x55a1a9528c10 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000d010 tx=0x7f5e50012000 comp rx=0 tx=0).stop 2023-10-10T20:41:18.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.510+0000 7f5e68058b80 1 -- 172.21.15.89:0/2090506846 shutdown_connections 2023-10-10T20:41:18.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.510+0000 7f5e68058b80 1 --2- 172.21.15.89:0/2090506846 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a9576480 0x55a1a9558a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.510+0000 7f5e68058b80 1 --2- 172.21.15.89:0/2090506846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a95295f0 0x55a1a9528c10 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.510+0000 7f5e68058b80 1 --2- 172.21.15.89:0/2090506846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a1a952bb80 0x55a1a95584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.510+0000 7f5e68058b80 1 -- 172.21.15.89:0/2090506846 >> 172.21.15.89:0/2090506846 conn(0x55a1a952d0c0 msgr2=0x55a1a9529d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:18.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.510+0000 7f5e68058b80 1 -- 172.21.15.89:0/2090506846 shutdown_connections 2023-10-10T20:41:18.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.510+0000 7f5e68058b80 1 -- 172.21.15.89:0/2090506846 wait complete. 2023-10-10T20:41:18.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.511+0000 7f5e68058b80 1 -- start start 2023-10-10T20:41:18.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.511+0000 7f5e68058b80 1 Processor -- start 2023-10-10T20:41:18.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e68058b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a95295f0 0x55a1a95581f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e68058b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a1a952bb80 0x55a1a9558730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e68058b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a9576480 0x55a1a95d8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e68058b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a95d7f10 con 0x55a1a9576480 2023-10-10T20:41:18.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e68058b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a95d9720 con 0x55a1a952bb80 2023-10-10T20:41:18.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e68058b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1a9528e80 con 0x55a1a95295f0 2023-10-10T20:41:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e68058b80 10 client.0 _ll_register_callbacks cb 0x55a1a94ef2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:41:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e5d2fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a9576480 0x55a1a95d8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e5d2fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a9576480 0x55a1a95d8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49072/0 (socket says 172.21.15.89:49072) 2023-10-10T20:41:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e5d2fc700 1 -- 172.21.15.89:0/2934964801 learned_addr learned my addr 172.21.15.89:0/2934964801 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e5cafb700 1 --2- 172.21.15.89:0/2934964801 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a95295f0 0x55a1a95581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e57fff700 1 --2- 172.21.15.89:0/2934964801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a1a952bb80 0x55a1a9558730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e5d2fc700 1 -- 172.21.15.89:0/2934964801 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a95295f0 msgr2=0x55a1a95581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:18.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e5d2fc700 1 --2- 172.21.15.89:0/2934964801 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a95295f0 0x55a1a95581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e5d2fc700 1 -- 172.21.15.89:0/2934964801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a1a952bb80 msgr2=0x55a1a9558730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:18.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e5d2fc700 1 --2- 172.21.15.89:0/2934964801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a1a952bb80 0x55a1a9558730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e5d2fc700 1 -- 172.21.15.89:0/2934964801 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e44003570 con 0x55a1a9576480 2023-10-10T20:41:18.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e5d2fc700 1 --2- 172.21.15.89:0/2934964801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a9576480 0x55a1a95d8360 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f5e5815e340 tx=0x7f5e40008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ce9a8fb2d21ca9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:18.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:41:18.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e36ffd700 1 -- 172.21.15.89:0/2934964801 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4001c070 con 0x55a1a9576480 2023-10-10T20:41:18.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e36ffd700 1 -- 172.21.15.89:0/2934964801 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e400155d0 con 0x55a1a9576480 2023-10-10T20:41:18.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.512+0000 7f5e68058b80 1 -- 172.21.15.89:0/2934964801 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e48058920 con 0x55a1a9576480 2023-10-10T20:41:18.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.513+0000 7f5e36ffd700 1 -- 172.21.15.89:0/2934964801 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e400159a0 con 0x55a1a9576480 2023-10-10T20:41:18.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.513+0000 7f5e68058b80 10 client.7362 fetch_fsmap learned FSMap version 73 2023-10-10T20:41:18.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.513+0000 7f5e68058b80 1 -- 172.21.15.89:0/2934964801 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55a1a9431470 con 0x55a1a9576480 2023-10-10T20:41:18.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.513+0000 7f5e36ffd700 1 -- 172.21.15.89:0/2934964801 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e48058920 con 0x55a1a9576480 2023-10-10T20:41:18.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.513+0000 7f5e36ffd700 1 -- 172.21.15.89:0/2934964801 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 73) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5e40015b00 con 0x55a1a9576480 2023-10-10T20:41:18.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.513+0000 7f5e68058b80 10 client.7362 fetch_fsmap finished waiting for FSMap version 73 2023-10-10T20:41:18.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.513+0000 7f5e68058b80 10 client.7362 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-10T20:41:18.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.513+0000 7f5e68058b80 20 client.7362 populate_metadata read hostname 'smithi089' 2023-10-10T20:41:18.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.513+0000 7f5e68058b80 1 --2- 172.21.15.89:0/2934964801 >> [v2:172.21.15.111:6833/2582059153,v1:172.21.15.111:6835/2582059153] conn(0x55a1a95d88a0 0x55a1a95727a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.513+0000 7f5e68058b80 4 client.7362 mds_command: new command op to 7200 tid=0[{"prefix": "status"}] 2023-10-10T20:41:18.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.513+0000 7f5e68058b80 1 -- 172.21.15.89:0/2934964801 --> [v2:172.21.15.111:6833/2582059153,v1:172.21.15.111:6835/2582059153] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a1a941ee00 con 0x55a1a95d88a0 2023-10-10T20:41:18.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.513+0000 7f5e5cafb700 1 --2- 172.21.15.89:0/2934964801 >> [v2:172.21.15.111:6833/2582059153,v1:172.21.15.111:6835/2582059153] conn(0x55a1a95d88a0 0x55a1a95727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.513+0000 7f5e36ffd700 10 client.7362 ms_handle_connect on v2:172.21.15.111:6833/2582059153 2023-10-10T20:41:18.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.513+0000 7f5e5cafb700 1 --2- 172.21.15.89:0/2934964801 >> [v2:172.21.15.111:6833/2582059153,v1:172.21.15.111:6835/2582059153] conn(0x55a1a95d88a0 0x55a1a95727a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b023751a8f2f2b06 server_cookie=50dae32be1868841 in_seq=0 out_seq=0 2023-10-10T20:41:18.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.514+0000 7f5e36ffd700 1 -- 172.21.15.89:0/2934964801 <== mds.0 v2:172.21.15.111:6833/2582059153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55a1a941ee00 con 0x55a1a95d88a0 2023-10-10T20:41:18.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.514+0000 7f5e36ffd700 10 client.7362 handle_command_reply: tid=0 2023-10-10T20:41:18.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.514+0000 7f5e68058b80 1 client.7362 shutdown 2023-10-10T20:41:18.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.514+0000 7f5e68058b80 1 -- 172.21.15.89:0/2934964801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a9576480 msgr2=0x55a1a95d8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:18.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.514+0000 7f5e68058b80 1 --2- 172.21.15.89:0/2934964801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a9576480 0x55a1a95d8360 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f5e5815e340 tx=0x7f5e40008040 comp rx=0 tx=0).stop 2023-10-10T20:41:18.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.514+0000 7f5e68058b80 1 -- 172.21.15.89:0/2934964801 shutdown_connections 2023-10-10T20:41:18.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.514+0000 7f5e68058b80 1 --2- 172.21.15.89:0/2934964801 >> [v2:172.21.15.111:6833/2582059153,v1:172.21.15.111:6835/2582059153] conn(0x55a1a95d88a0 0x55a1a95727a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.514+0000 7f5e68058b80 1 --2- 172.21.15.89:0/2934964801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1a9576480 0x55a1a95d8360 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.514+0000 7f5e36ffd700 0 client.7362 ms_handle_reset on v2:172.21.15.111:6833/2582059153 2023-10-10T20:41:18.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.514+0000 7f5e68058b80 1 --2- 172.21.15.89:0/2934964801 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55a1a95295f0 0x55a1a95581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.514+0000 7f5e68058b80 1 --2- 172.21.15.89:0/2934964801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a1a952bb80 0x55a1a9558730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.514+0000 7f5e68058b80 1 -- 172.21.15.89:0/2934964801 >> 172.21.15.89:0/2934964801 conn(0x55a1a952d0c0 msgr2=0x55a1a952d490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:18.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.514+0000 7f5e68058b80 1 -- 172.21.15.89:0/2934964801 shutdown_connections 2023-10-10T20:41:18.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.514+0000 7f5e68058b80 1 -- 172.21.15.89:0/2934964801 wait complete. 2023-10-10T20:41:18.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.514+0000 7f5e68058b80 20 client.7362 trim_cache size 0 max 16384 2023-10-10T20:41:18.552 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:41:18.553 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "5dd675b8-9026-4d8a-a1bf-62fdc5e7de67", 2023-10-10T20:41:18.553 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-10-10T20:41:18.553 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 7200, 2023-10-10T20:41:18.553 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-10-10T20:41:18.553 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-10-10T20:41:18.554 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-10-10T20:41:18.554 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 10.311365461999999, 2023-10-10T20:41:18.554 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 73, 2023-10-10T20:41:18.554 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 112, 2023-10-10T20:41:18.554 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 112, 2023-10-10T20:41:18.555 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 15.442048867 2023-10-10T20:41:18.555 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:41:18.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.514+0000 7f5e5dafd700 1 -- 172.21.15.89:0/228055106 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5e38067320 msgr2=0x7f5e380697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:18.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.514+0000 7f5e5dafd700 1 --2- 172.21.15.89:0/228055106 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5e38067320 0x7f5e380697d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5e50007cd0 tx=0x7f5e50015000 comp rx=0 tx=0).stop 2023-10-10T20:41:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.514+0000 7f5e5dafd700 1 -- 172.21.15.89:0/228055106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e580cf950 msgr2=0x7f5e580cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.514+0000 7f5e5dafd700 1 --2- 172.21.15.89:0/228055106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e580cf950 0x7f5e580cfda0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f5e48007fd0 tx=0x7f5e48009040 comp rx=0 tx=0).stop 2023-10-10T20:41:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.515+0000 7f5e5dafd700 1 -- 172.21.15.89:0/228055106 shutdown_connections 2023-10-10T20:41:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.515+0000 7f5e5dafd700 1 --2- 172.21.15.89:0/228055106 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5e38067320 0x7f5e380697d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.515+0000 7f5e5dafd700 1 --2- 172.21.15.89:0/228055106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e580c5470 0x7f5e580c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.515+0000 7f5e5dafd700 1 --2- 172.21.15.89:0/228055106 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5e44003a90 0x7f5e581550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.515+0000 7f5e5dafd700 1 --2- 172.21.15.89:0/228055106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e580cf950 0x7f5e580cfda0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.515+0000 7f5e5dafd700 1 -- 172.21.15.89:0/228055106 >> 172.21.15.89:0/228055106 conn(0x7f5e580d0660 msgr2=0x7f5e58068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.515+0000 7f5e5dafd700 1 -- 172.21.15.89:0/228055106 shutdown_connections 2023-10-10T20:41:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.515+0000 7f5e5dafd700 1 -- 172.21.15.89:0/228055106 wait complete. 2023-10-10T20:41:18.562 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:41:18.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.674+0000 7f9b5d77f700 1 Processor -- start 2023-10-10T20:41:18.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.674+0000 7f9b5d77f700 1 -- start start 2023-10-10T20:41:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.675+0000 7f9b5d77f700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9b5806aa40 0x7f9b580cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.675+0000 7f9b5d77f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9b580c91e0 0x7f9b58160390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.675+0000 7f9b5d77f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b580cc210 con 0x7f9b5806bac0 2023-10-10T20:41:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.675+0000 7f9b5d77f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b580cc4f0 con 0x7f9b580c91e0 2023-10-10T20:41:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.675+0000 7f9b5d77f700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b580cc7d0 con 0x7f9b5806aa40 2023-10-10T20:41:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.675+0000 7f9b577fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9b580c91e0 0x7f9b58160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.675+0000 7f9b567fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9b5806bac0 0x7f9b5806ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45346/0 (socket says 172.21.15.89:45346) 2023-10-10T20:41:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.675+0000 7f9b567fc700 1 -- 172.21.15.89:0/2715745839 learned_addr learned my addr 172.21.15.89:0/2715745839 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.675+0000 7f9b56ffd700 1 --2- 172.21.15.89:0/2715745839 >> v2:172.21.15.111:3300/0 conn(0x7f9b5806aa40 0x7f9b580cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.676+0000 7f9b577fe700 1 -- 172.21.15.89:0/2715745839 >> v2:172.21.15.111:3300/0 conn(0x7f9b5806aa40 msgr2=0x7f9b580cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.676+0000 7f9b577fe700 1 --2- 172.21.15.89:0/2715745839 >> v2:172.21.15.111:3300/0 conn(0x7f9b5806aa40 0x7f9b580cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.676+0000 7f9b577fe700 1 -- 172.21.15.89:0/2715745839 >> v1:172.21.15.89:6789/0 conn(0x7f9b5806bac0 legacy=0x7f9b5806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.676+0000 7f9b55ffb700 1 -- 172.21.15.89:0/2715745839 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3223530859 0 0) 0x7f9b580cc210 con 0x7f9b5806bac0 2023-10-10T20:41:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.676+0000 7f9b577fe700 1 -- 172.21.15.89:0/2715745839 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b580ccab0 con 0x7f9b580c91e0 2023-10-10T20:41:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.676+0000 7f9b577fe700 1 --2- 172.21.15.89:0/2715745839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b580c91e0 0x7f9b58160390 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f9b5806e570 tx=0x7f9b50009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871047df5f7e398b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.676+0000 7f9b55ffb700 1 -- 172.21.15.89:0/2715745839 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5000e070 con 0x7f9b580c91e0 2023-10-10T20:41:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.676+0000 7f9b55ffb700 1 -- 172.21.15.89:0/2715745839 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5000bc30 con 0x7f9b580c91e0 2023-10-10T20:41:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.676+0000 7f9b55ffb700 1 -- 172.21.15.89:0/2715745839 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50012430 con 0x7f9b580c91e0 2023-10-10T20:41:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.676+0000 7f9b5d77f700 1 -- 172.21.15.89:0/2715745839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b580c91e0 msgr2=0x7f9b58160390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:18.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.676+0000 7f9b5d77f700 1 --2- 172.21.15.89:0/2715745839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b580c91e0 0x7f9b58160390 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f9b5806e570 tx=0x7f9b50009780 comp rx=0 tx=0).stop 2023-10-10T20:41:18.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.677+0000 7f9b5d77f700 1 -- 172.21.15.89:0/2715745839 shutdown_connections 2023-10-10T20:41:18.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.677+0000 7f9b5d77f700 1 --2- 172.21.15.89:0/2715745839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b580c91e0 0x7f9b58160390 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.677+0000 7f9b5d77f700 1 --2- 172.21.15.89:0/2715745839 >> v2:172.21.15.111:3300/0 conn(0x7f9b5806aa40 0x7f9b580cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.677+0000 7f9b5d77f700 1 -- 172.21.15.89:0/2715745839 >> 172.21.15.89:0/2715745839 conn(0x7f9b580cd670 msgr2=0x7f9b580cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:18.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.677+0000 7f9b5d77f700 1 -- 172.21.15.89:0/2715745839 shutdown_connections 2023-10-10T20:41:18.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.677+0000 7f9b5d77f700 1 -- 172.21.15.89:0/2715745839 wait complete. 2023-10-10T20:41:18.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.678+0000 7f9b5d77f700 1 Processor -- start 2023-10-10T20:41:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.678+0000 7f9b5d77f700 1 -- start start 2023-10-10T20:41:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b5d77f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b5806aa40 0x7f9b5814dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b5d77f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b5806bac0 0x7f9b58158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b5d77f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b580c91e0 0x7f9b580c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b5d77f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5814e710 con 0x7f9b5806aa40 2023-10-10T20:41:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b5d77f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580b0880 con 0x7f9b580c91e0 2023-10-10T20:41:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b5d77f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5815be30 con 0x7f9b5806bac0 2023-10-10T20:41:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b56ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b5806aa40 0x7f9b5814dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b56ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b5806aa40 0x7f9b5814dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49096/0 (socket says 172.21.15.89:49096) 2023-10-10T20:41:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b56ffd700 1 -- 172.21.15.89:0/785457811 learned_addr learned my addr 172.21.15.89:0/785457811 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b577fe700 1 --2- 172.21.15.89:0/785457811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b580c91e0 0x7f9b580c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b567fc700 1 --2- 172.21.15.89:0/785457811 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b5806bac0 0x7f9b58158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b56ffd700 1 -- 172.21.15.89:0/785457811 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b5806bac0 msgr2=0x7f9b58158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:18.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b56ffd700 1 --2- 172.21.15.89:0/785457811 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b5806bac0 0x7f9b58158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b56ffd700 1 -- 172.21.15.89:0/785457811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b580c91e0 msgr2=0x7f9b580c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:18.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b56ffd700 1 --2- 172.21.15.89:0/785457811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b580c91e0 0x7f9b580c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b56ffd700 1 -- 172.21.15.89:0/785457811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b50009a20 con 0x7f9b5806aa40 2023-10-10T20:41:18.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b56ffd700 1 --2- 172.21.15.89:0/785457811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b5806aa40 0x7f9b5814dd40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800d810 tx=0x7f9b48008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:18.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b3effd700 1 -- 172.21.15.89:0/785457811 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b480125f0 con 0x7f9b5806aa40 2023-10-10T20:41:18.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b3effd700 1 -- 172.21.15.89:0/785457811 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b48012750 con 0x7f9b5806aa40 2023-10-10T20:41:18.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b3effd700 1 -- 172.21.15.89:0/785457811 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b48012b10 con 0x7f9b5806aa40 2023-10-10T20:41:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b5d77f700 1 -- 172.21.15.89:0/785457811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5815b590 con 0x7f9b5806aa40 2023-10-10T20:41:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.679+0000 7f9b5d77f700 1 -- 172.21.15.89:0/785457811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5815bb50 con 0x7f9b5806aa40 2023-10-10T20:41:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.680+0000 7f9b5d77f700 1 -- 172.21.15.89:0/785457811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5804c3c0 con 0x7f9b5806aa40 2023-10-10T20:41:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.681+0000 7f9b3effd700 1 -- 172.21.15.89:0/785457811 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9b4801e460 con 0x7f9b5806aa40 2023-10-10T20:41:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.681+0000 7f9b3effd700 1 --2- 172.21.15.89:0/785457811 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9b44067320 0x7f9b440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.681+0000 7f9b567fc700 1 --2- 172.21.15.89:0/785457811 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9b44067320 0x7f9b440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.681+0000 7f9b3effd700 1 -- 172.21.15.89:0/785457811 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9b48085230 con 0x7f9b5806aa40 2023-10-10T20:41:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.682+0000 7f9b567fc700 1 --2- 172.21.15.89:0/785457811 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9b44067320 0x7f9b440697d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9b5814ff80 tx=0x7f9b4c007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.685+0000 7f9b3effd700 1 -- 172.21.15.89:0/785457811 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9b48051ec0 con 0x7f9b5806aa40 2023-10-10T20:41:18.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.914+0000 7f9b5d77f700 1 -- 172.21.15.89:0/785457811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9b580b0b60 con 0x7f9b5806aa40 2023-10-10T20:41:18.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.914+0000 7f9b3effd700 1 -- 172.21.15.89:0/785457811 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v73) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9b480554e0 con 0x7f9b5806aa40 2023-10-10T20:41:18.917 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:41:18.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.916+0000 7f9b5d77f700 1 -- 172.21.15.89:0/785457811 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9b44067320 msgr2=0x7f9b440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:18.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.916+0000 7f9b5d77f700 1 --2- 172.21.15.89:0/785457811 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9b44067320 0x7f9b440697d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9b5814ff80 tx=0x7f9b4c007d30 comp rx=0 tx=0).stop 2023-10-10T20:41:18.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.916+0000 7f9b5d77f700 1 -- 172.21.15.89:0/785457811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b5806aa40 msgr2=0x7f9b5814dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:18.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.916+0000 7f9b5d77f700 1 --2- 172.21.15.89:0/785457811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b5806aa40 0x7f9b5814dd40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800d810 tx=0x7f9b48008040 comp rx=0 tx=0).stop 2023-10-10T20:41:18.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.916+0000 7f9b5d77f700 1 -- 172.21.15.89:0/785457811 shutdown_connections 2023-10-10T20:41:18.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.916+0000 7f9b5d77f700 1 --2- 172.21.15.89:0/785457811 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9b44067320 0x7f9b440697d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.916+0000 7f9b5d77f700 1 --2- 172.21.15.89:0/785457811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b580c91e0 0x7f9b580c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.916+0000 7f9b5d77f700 1 --2- 172.21.15.89:0/785457811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b5806aa40 0x7f9b5814dd40 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.916+0000 7f9b5d77f700 1 --2- 172.21.15.89:0/785457811 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9b5806bac0 0x7f9b58158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:18.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.916+0000 7f9b5d77f700 1 -- 172.21.15.89:0/785457811 >> 172.21.15.89:0/785457811 conn(0x7f9b580cd670 msgr2=0x7f9b581500e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:18.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.916+0000 7f9b5d77f700 1 -- 172.21.15.89:0/785457811 shutdown_connections 2023-10-10T20:41:18.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:18.916+0000 7f9b5d77f700 1 -- 172.21.15.89:0/785457811 wait complete. 2023-10-10T20:41:18.930 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:41:18.930 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:41:18.931 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-10T20:41:18.931 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:41:18.931 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:41:18.932 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:41:18.932 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:41:18.932 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:41:18.933 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:41:18.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:41:18.933 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-10T20:41:18.949 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:41:18.949 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:41:18.949 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:41:18.950 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:41:18.950 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-10T20:41:18.950 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:41:18.950 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:41:18.951 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:41:18.951 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-10T20:41:18.951 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:41:18.951 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:41:18.951 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-10T20:41:18.952 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:41:18.952 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:41:18.952 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:41:18.952 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:41:18.953 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:41:18.953 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42087sec preferred_lft 42087sec 2023-10-10T20:41:18.953 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2023-10-10T20:41:18.953 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:41:18.953 INFO:teuthology.orchestra.run.smithi089.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:41:18.953 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:41:18.954 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:41:18.954 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:41:18.954 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::98a3:d4ff:fec1:83bb/64 scope link 2023-10-10T20:41:18.954 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:41:18.955 INFO:teuthology.orchestra.run.smithi089.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:41:18.955 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether ee:16:22:10:ca:9a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:41:18.955 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec16:22ff:fe10:ca9a/64 scope link 2023-10-10T20:41:18.956 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:41:18.956 INFO:teuthology.orchestra.run.smithi089.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:41:18.956 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:41:18.957 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::6864:2bff:fe5c:71d4/64 scope link 2023-10-10T20:41:18.957 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:41:18.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:41:18.959 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:41:19.012 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-10T20:41:19.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:41:19.015 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-10T20:41:19.069 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-10T20:41:19.070 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:41:19.070 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:41:19.070 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:41:19.071 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:41:19.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:41:19.071 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:41:19.072 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:41:19.072 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:41:19.072 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:41:19.073 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:41:19.073 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:41:19.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:41:19.170 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-10T20:41:19.184 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:41:19.185 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:41:19.185 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:41:19.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:41:19.186 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-10-10T20:41:19.250 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:41:19.250 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:41:19.305 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:41:19.306 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:41:19.306 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:41:19.360 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:41:19.360 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:41:19.361 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:41:19.439 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:41:19.439 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:41:19.439 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:41:19.447 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:41:19.447 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:41:19.448 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:41:19.448 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:41:19.448 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:41:19.469 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:41:19.470 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:41:19.470 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:41:19.470 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:41:19.471 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:41:19.471 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:41:19.471 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:41:19.471 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:41:19.473 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:41:19.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:41:19.474 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:41:19.538 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:41:19.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.691+0000 7f8a62ff9700 1 Processor -- start 2023-10-10T20:41:19.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.691+0000 7f8a62ff9700 1 -- start start 2023-10-10T20:41:19.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.691+0000 7f8a62ff9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8a5c159890 0x7f8a5c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:19.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.691+0000 7f8a62ff9700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8a5c069560 0x7f8a5c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:19.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.691+0000 7f8a62ff9700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a5c069ee0 con 0x7f8a5c159c60 2023-10-10T20:41:19.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.691+0000 7f8a62ff9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a5c15a160 con 0x7f8a5c159890 2023-10-10T20:41:19.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.691+0000 7f8a62ff9700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a5c0d13a0 con 0x7f8a5c069560 2023-10-10T20:41:19.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.692+0000 7f8a60d95700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8a5c159890 0x7f8a5c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:19.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.692+0000 7f8a60d95700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8a5c159890 0x7f8a5c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49108/0 (socket says 172.21.15.89:49108) 2023-10-10T20:41:19.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.692+0000 7f8a60d95700 1 -- 172.21.15.89:0/3931917378 learned_addr learned my addr 172.21.15.89:0/3931917378 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:19.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.692+0000 7f8a61596700 1 --2- 172.21.15.89:0/3931917378 >> v2:172.21.15.111:3300/0 conn(0x7f8a5c069560 0x7f8a5c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:19.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.692+0000 7f8a60d95700 1 -- 172.21.15.89:0/3931917378 >> v2:172.21.15.111:3300/0 conn(0x7f8a5c069560 msgr2=0x7f8a5c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:19.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.692+0000 7f8a60d95700 1 --2- 172.21.15.89:0/3931917378 >> v2:172.21.15.111:3300/0 conn(0x7f8a5c069560 0x7f8a5c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:19.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.692+0000 7f8a60d95700 1 -- 172.21.15.89:0/3931917378 >> v1:172.21.15.111:6789/0 conn(0x7f8a5c159c60 legacy=0x7f8a5c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:19.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.692+0000 7f8a60d95700 1 -- 172.21.15.89:0/3931917378 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5c15e140 con 0x7f8a5c159890 2023-10-10T20:41:19.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.692+0000 7f8a60d95700 1 --2- 172.21.15.89:0/3931917378 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5c159890 0x7f8a5c15abd0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c06a890 tx=0x7f8a58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3076214e756926f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:19.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.692+0000 7f8a537fe700 1 -- 172.21.15.89:0/3931917378 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5800e070 con 0x7f8a5c159890 2023-10-10T20:41:19.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.692+0000 7f8a537fe700 1 -- 172.21.15.89:0/3931917378 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5800ba50 con 0x7f8a5c159890 2023-10-10T20:41:19.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.692+0000 7f8a537fe700 1 -- 172.21.15.89:0/3931917378 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a580092e0 con 0x7f8a5c159890 2023-10-10T20:41:19.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.693+0000 7f8a62ff9700 1 -- 172.21.15.89:0/3931917378 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5c159890 msgr2=0x7f8a5c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:19.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.693+0000 7f8a62ff9700 1 --2- 172.21.15.89:0/3931917378 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5c159890 0x7f8a5c15abd0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c06a890 tx=0x7f8a58009840 comp rx=0 tx=0).stop 2023-10-10T20:41:19.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.693+0000 7f8a62ff9700 1 -- 172.21.15.89:0/3931917378 shutdown_connections 2023-10-10T20:41:19.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.693+0000 7f8a62ff9700 1 --2- 172.21.15.89:0/3931917378 >> v2:172.21.15.111:3300/0 conn(0x7f8a5c069560 0x7f8a5c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:19.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.693+0000 7f8a62ff9700 1 --2- 172.21.15.89:0/3931917378 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5c159890 0x7f8a5c15abd0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:19.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.693+0000 7f8a62ff9700 1 -- 172.21.15.89:0/3931917378 >> 172.21.15.89:0/3931917378 conn(0x7f8a5c068e20 msgr2=0x7f8a5c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:19.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.693+0000 7f8a62ff9700 1 -- 172.21.15.89:0/3931917378 shutdown_connections 2023-10-10T20:41:19.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.693+0000 7f8a62ff9700 1 -- 172.21.15.89:0/3931917378 wait complete. 2023-10-10T20:41:19.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.694+0000 7f8a62ff9700 1 Processor -- start 2023-10-10T20:41:19.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.694+0000 7f8a62ff9700 1 -- start start 2023-10-10T20:41:19.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.695+0000 7f8a62ff9700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c069560 0x7f8a5c0c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:19.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.695+0000 7f8a62ff9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5c159890 0x7f8a5c0b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:19.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.695+0000 7f8a62ff9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a5c159c60 0x7f8a5c16d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:19.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.695+0000 7f8a62ff9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c0b64c0 con 0x7f8a5c159890 2023-10-10T20:41:19.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.695+0000 7f8a62ff9700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c0b4430 con 0x7f8a5c069560 2023-10-10T20:41:19.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.695+0000 7f8a62ff9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c0b4710 con 0x7f8a5c159c60 2023-10-10T20:41:19.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.695+0000 7f8a53fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5c159890 0x7f8a5c0b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:19.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.695+0000 7f8a53fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5c159890 0x7f8a5c0b5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49122/0 (socket says 172.21.15.89:49122) 2023-10-10T20:41:19.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.695+0000 7f8a53fff700 1 -- 172.21.15.89:0/387999595 learned_addr learned my addr 172.21.15.89:0/387999595 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:19.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.695+0000 7f8a60d95700 1 --2- 172.21.15.89:0/387999595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c069560 0x7f8a5c0c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:19.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.695+0000 7f8a61596700 1 --2- 172.21.15.89:0/387999595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a5c159c60 0x7f8a5c16d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:19.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.695+0000 7f8a53fff700 1 -- 172.21.15.89:0/387999595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a5c159c60 msgr2=0x7f8a5c16d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:19.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.695+0000 7f8a53fff700 1 --2- 172.21.15.89:0/387999595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a5c159c60 0x7f8a5c16d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:19.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.695+0000 7f8a53fff700 1 -- 172.21.15.89:0/387999595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c069560 msgr2=0x7f8a5c0c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:19.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.695+0000 7f8a53fff700 1 --2- 172.21.15.89:0/387999595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c069560 0x7f8a5c0c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:19.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.695+0000 7f8a53fff700 1 -- 172.21.15.89:0/387999595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a58009ae0 con 0x7f8a5c159890 2023-10-10T20:41:19.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.695+0000 7f8a53fff700 1 --2- 172.21.15.89:0/387999595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5c159890 0x7f8a5c0b5c70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00c4e0 tx=0x7f8a4c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:19.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.696+0000 7f8a50ff9700 1 -- 172.21.15.89:0/387999595 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c011590 con 0x7f8a5c159890 2023-10-10T20:41:19.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.696+0000 7f8a50ff9700 1 -- 172.21.15.89:0/387999595 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4c011710 con 0x7f8a5c159890 2023-10-10T20:41:19.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.696+0000 7f8a62ff9700 1 -- 172.21.15.89:0/387999595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a5c0b49f0 con 0x7f8a5c159890 2023-10-10T20:41:19.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.696+0000 7f8a50ff9700 1 -- 172.21.15.89:0/387999595 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c011ad0 con 0x7f8a5c159890 2023-10-10T20:41:19.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.696+0000 7f8a62ff9700 1 -- 172.21.15.89:0/387999595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5c0b61b0 con 0x7f8a5c159890 2023-10-10T20:41:19.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.697+0000 7f8a62ff9700 1 -- 172.21.15.89:0/387999595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a5c0b4f10 con 0x7f8a5c159890 2023-10-10T20:41:19.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.697+0000 7f8a50ff9700 1 -- 172.21.15.89:0/387999595 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8a4c01b3b0 con 0x7f8a5c159890 2023-10-10T20:41:19.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.697+0000 7f8a50ff9700 1 --2- 172.21.15.89:0/387999595 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8a48067320 0x7f8a480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:19.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.697+0000 7f8a50ff9700 1 -- 172.21.15.89:0/387999595 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8a4c084000 con 0x7f8a5c159890 2023-10-10T20:41:19.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.697+0000 7f8a60d95700 1 --2- 172.21.15.89:0/387999595 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8a48067320 0x7f8a480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:19.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.698+0000 7f8a60d95700 1 --2- 172.21.15.89:0/387999595 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8a48067320 0x7f8a480697d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009880 tx=0x7f8a58018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:19.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.701+0000 7f8a50ff9700 1 -- 172.21.15.89:0/387999595 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a5c0b4f10 con 0x7f8a5c159890 2023-10-10T20:41:19.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.926+0000 7f8a62ff9700 1 -- 172.21.15.89:0/387999595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8a5c0b4f10 con 0x7f8a5c159890 2023-10-10T20:41:19.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.927+0000 7f8a50ff9700 1 -- 172.21.15.89:0/387999595 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v73) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8a5c0b4f10 con 0x7f8a5c159890 2023-10-10T20:41:19.929 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:41:19.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.929+0000 7f8a62ff9700 1 -- 172.21.15.89:0/387999595 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8a48067320 msgr2=0x7f8a480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:19.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.929+0000 7f8a62ff9700 1 --2- 172.21.15.89:0/387999595 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8a48067320 0x7f8a480697d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009880 tx=0x7f8a58018000 comp rx=0 tx=0).stop 2023-10-10T20:41:19.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.929+0000 7f8a62ff9700 1 -- 172.21.15.89:0/387999595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5c159890 msgr2=0x7f8a5c0b5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.929+0000 7f8a62ff9700 1 --2- 172.21.15.89:0/387999595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5c159890 0x7f8a5c0b5c70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00c4e0 tx=0x7f8a4c00c7f0 comp rx=0 tx=0).stop 2023-10-10T20:41:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.929+0000 7f8a62ff9700 1 -- 172.21.15.89:0/387999595 shutdown_connections 2023-10-10T20:41:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.929+0000 7f8a62ff9700 1 --2- 172.21.15.89:0/387999595 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a5c159c60 0x7f8a5c16d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.929+0000 7f8a62ff9700 1 --2- 172.21.15.89:0/387999595 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8a48067320 0x7f8a480697d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:19.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.929+0000 7f8a62ff9700 1 --2- 172.21.15.89:0/387999595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c069560 0x7f8a5c0c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:19.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.929+0000 7f8a62ff9700 1 --2- 172.21.15.89:0/387999595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5c159890 0x7f8a5c0b5c70 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:19.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.929+0000 7f8a62ff9700 1 -- 172.21.15.89:0/387999595 >> 172.21.15.89:0/387999595 conn(0x7f8a5c068e20 msgr2=0x7f8a5c1534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:19.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.929+0000 7f8a62ff9700 1 -- 172.21.15.89:0/387999595 shutdown_connections 2023-10-10T20:41:19.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:19.929+0000 7f8a62ff9700 1 -- 172.21.15.89:0/387999595 wait complete. 2023-10-10T20:41:19.943 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:41:19.943 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:41:19.944 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-10-10T20:41:19.944 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:41:19.945 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:41:19.945 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:41:19.946 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:41:19.946 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:41:19.947 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:41:19.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:41:19.949 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-10-10T20:41:19.967 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:41:19.968 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:41:19.968 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:41:19.968 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:41:19.968 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-10-10T20:41:19.968 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:41:19.969 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:41:19.969 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:41:19.969 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-10-10T20:41:19.969 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:41:19.969 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:41:19.970 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-10-10T20:41:19.970 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:41:19.970 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:41:19.970 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:41:19.970 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-10-10T20:41:19.971 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:41:19.971 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42102sec preferred_lft 42102sec 2023-10-10T20:41:19.971 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-10-10T20:41:19.971 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:41:19.971 INFO:teuthology.orchestra.run.smithi111.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:41:19.972 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:41:19.972 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:41:19.972 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:41:19.972 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::a486:a6ff:fe5f:6021/64 scope link 2023-10-10T20:41:19.972 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:41:19.973 INFO:teuthology.orchestra.run.smithi111.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:41:19.973 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 1e:d0:81:1b:e2:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:41:19.973 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::1cd0:81ff:fe1b:e26b/64 scope link 2023-10-10T20:41:19.973 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:41:19.973 INFO:teuthology.orchestra.run.smithi111.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:41:19.973 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:41:19.974 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::88d:cdff:fe75:455e/64 scope link 2023-10-10T20:41:19.974 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:41:19.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:41:19.975 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:41:20.035 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-10-10T20:41:20.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:41:20.037 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-10-10T20:41:20.095 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-10-10T20:41:20.095 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:41:20.095 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:41:20.095 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:41:20.096 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:41:20.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:41:20.097 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:41:20.097 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:41:20.097 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:41:20.098 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:41:20.098 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:41:20.099 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:41:20.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:41:20.201 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-10-10T20:41:20.216 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:41:20.216 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:41:20.217 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:41:20.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:41:20.218 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-10-10T20:41:20.282 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:41:20.283 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:41:20.338 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:41:20.339 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:41:20.339 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:41:20.394 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:41:20.394 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:41:20.395 DEBUG:teuthology.orchestra.run.smithi111:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:41:20.473 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:41:20.474 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:41:20.474 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:41:20.481 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:41:20.482 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:41:20.482 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:41:20.482 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:41:20.482 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:41:20.503 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:41:20.503 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:41:20.504 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:41:20.504 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:41:20.504 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:41:20.504 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:41:20.504 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:41:20.505 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:41:20.506 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:41:20.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:41:20.506 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:41:20.570 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-10T20:41:20.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.687+0000 7f35627a7700 1 Processor -- start 2023-10-10T20:41:20.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.687+0000 7f35627a7700 1 -- start start 2023-10-10T20:41:20.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.688+0000 7f35627a7700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f355c159ec0 0x7f355c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:20.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.688+0000 7f35627a7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f355c0b53d0 0x7f355c0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:20.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.688+0000 7f35627a7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f355c0cce40 0x7f355c0cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:20.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.688+0000 7f35627a7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f355c0b5b00 con 0x7f355c0cce40 2023-10-10T20:41:20.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.688+0000 7f35627a7700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f355c0b5850 con 0x7f355c159ec0 2023-10-10T20:41:20.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.688+0000 7f35627a7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0b5da0 con 0x7f355c0b53d0 2023-10-10T20:41:20.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.688+0000 7f3560d44700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f355c0cce40 0x7f355c0cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:20.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.688+0000 7f3560d44700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f355c0cce40 0x7f355c0cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49128/0 (socket says 172.21.15.89:49128) 2023-10-10T20:41:20.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.688+0000 7f3560d44700 1 -- 172.21.15.89:0/3846575352 learned_addr learned my addr 172.21.15.89:0/3846575352 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:20.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.688+0000 7f355b7fe700 1 --2- 172.21.15.89:0/3846575352 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f355c0b53d0 0x7f355c0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:20.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.688+0000 7f355bfff700 1 --2- 172.21.15.89:0/3846575352 >> v2:172.21.15.111:3300/0 conn(0x7f355c159ec0 0x7f355c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:20.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.688+0000 7f3560d44700 1 -- 172.21.15.89:0/3846575352 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f355c0b53d0 msgr2=0x7f355c0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.688+0000 7f3560d44700 1 --2- 172.21.15.89:0/3846575352 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f355c0b53d0 0x7f355c0cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.689+0000 7f3560d44700 1 -- 172.21.15.89:0/3846575352 >> v2:172.21.15.111:3300/0 conn(0x7f355c159ec0 msgr2=0x7f355c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.689+0000 7f3560d44700 1 --2- 172.21.15.89:0/3846575352 >> v2:172.21.15.111:3300/0 conn(0x7f355c159ec0 0x7f355c0b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.689+0000 7f3560d44700 1 -- 172.21.15.89:0/3846575352 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c0b6080 con 0x7f355c0cce40 2023-10-10T20:41:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.689+0000 7f355bfff700 1 --2- 172.21.15.89:0/3846575352 >> v2:172.21.15.111:3300/0 conn(0x7f355c159ec0 0x7f355c0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:20.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.689+0000 7f3560d44700 1 --2- 172.21.15.89:0/3846575352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c0cce40 0x7f355c0cd2a0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f355c089120 tx=0x7f3554009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec6b64394b6e379 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:20.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.689+0000 7f355affd700 1 -- 172.21.15.89:0/3846575352 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355400e070 con 0x7f355c0cce40 2023-10-10T20:41:20.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.689+0000 7f355affd700 1 -- 172.21.15.89:0/3846575352 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355400ba50 con 0x7f355c0cce40 2023-10-10T20:41:20.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.689+0000 7f355affd700 1 -- 172.21.15.89:0/3846575352 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35540092e0 con 0x7f355c0cce40 2023-10-10T20:41:20.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.689+0000 7f35627a7700 1 -- 172.21.15.89:0/3846575352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c0cce40 msgr2=0x7f355c0cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:20.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.689+0000 7f35627a7700 1 --2- 172.21.15.89:0/3846575352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c0cce40 0x7f355c0cd2a0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f355c089120 tx=0x7f3554009840 comp rx=0 tx=0).stop 2023-10-10T20:41:20.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.689+0000 7f35627a7700 1 -- 172.21.15.89:0/3846575352 shutdown_connections 2023-10-10T20:41:20.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.689+0000 7f35627a7700 1 --2- 172.21.15.89:0/3846575352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c0cce40 0x7f355c0cd2a0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:20.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.689+0000 7f35627a7700 1 --2- 172.21.15.89:0/3846575352 >> v2:172.21.15.111:3300/0 conn(0x7f355c159ec0 0x7f355c0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:20.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.689+0000 7f35627a7700 1 --2- 172.21.15.89:0/3846575352 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f355c0b53d0 0x7f355c0cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:20.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.689+0000 7f35627a7700 1 -- 172.21.15.89:0/3846575352 >> 172.21.15.89:0/3846575352 conn(0x7f355c0999f0 msgr2=0x7f355c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:20.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.689+0000 7f35627a7700 1 -- 172.21.15.89:0/3846575352 shutdown_connections 2023-10-10T20:41:20.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.689+0000 7f35627a7700 1 -- 172.21.15.89:0/3846575352 wait complete. 2023-10-10T20:41:20.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.690+0000 7f35627a7700 1 Processor -- start 2023-10-10T20:41:20.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.691+0000 7f35627a7700 1 -- start start 2023-10-10T20:41:20.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.691+0000 7f35627a7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f355c0b53d0 0x7f355c13eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:20.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.691+0000 7f35627a7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c0cce40 0x7f355c069200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:20.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.691+0000 7f35627a7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f355c159ec0 0x7f355c0ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:20.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.691+0000 7f35627a7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c068d20 con 0x7f355c0cce40 2023-10-10T20:41:20.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.691+0000 7f35627a7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0cd2b0 con 0x7f355c159ec0 2023-10-10T20:41:20.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.691+0000 7f35627a7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0cd590 con 0x7f355c0b53d0 2023-10-10T20:41:20.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.691+0000 7f355b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c0cce40 0x7f355c069200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:20.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.691+0000 7f355b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c0cce40 0x7f355c069200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49132/0 (socket says 172.21.15.89:49132) 2023-10-10T20:41:20.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.691+0000 7f355b7fe700 1 -- 172.21.15.89:0/371540570 learned_addr learned my addr 172.21.15.89:0/371540570 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:20.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.692+0000 7f355bfff700 1 --2- 172.21.15.89:0/371540570 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f355c0b53d0 0x7f355c13eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:20.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.692+0000 7f3560d44700 1 --2- 172.21.15.89:0/371540570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f355c159ec0 0x7f355c0ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:20.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.692+0000 7f355b7fe700 1 -- 172.21.15.89:0/371540570 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f355c0b53d0 msgr2=0x7f355c13eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:20.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.692+0000 7f355b7fe700 1 --2- 172.21.15.89:0/371540570 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f355c0b53d0 0x7f355c13eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:20.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.692+0000 7f355b7fe700 1 -- 172.21.15.89:0/371540570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f355c159ec0 msgr2=0x7f355c0ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:20.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.692+0000 7f355b7fe700 1 --2- 172.21.15.89:0/371540570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f355c159ec0 0x7f355c0ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:20.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.692+0000 7f355b7fe700 1 -- 172.21.15.89:0/371540570 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3554009ae0 con 0x7f355c0cce40 2023-10-10T20:41:20.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.692+0000 7f355b7fe700 1 --2- 172.21.15.89:0/371540570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c0cce40 0x7f355c069200 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f355000d860 tx=0x7f3550008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:20.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.692+0000 7f3543fff700 1 -- 172.21.15.89:0/371540570 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35500125f0 con 0x7f355c0cce40 2023-10-10T20:41:20.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.692+0000 7f3543fff700 1 -- 172.21.15.89:0/371540570 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3550012750 con 0x7f355c0cce40 2023-10-10T20:41:20.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.692+0000 7f3543fff700 1 -- 172.21.15.89:0/371540570 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3550012b10 con 0x7f355c0cce40 2023-10-10T20:41:20.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.692+0000 7f35627a7700 1 -- 172.21.15.89:0/371540570 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355c0cd870 con 0x7f355c0cce40 2023-10-10T20:41:20.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.692+0000 7f35627a7700 1 -- 172.21.15.89:0/371540570 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355c0cde30 con 0x7f355c0cce40 2023-10-10T20:41:20.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.693+0000 7f35627a7700 1 -- 172.21.15.89:0/371540570 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355c063c90 con 0x7f355c0cce40 2023-10-10T20:41:20.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.693+0000 7f3543fff700 1 -- 172.21.15.89:0/371540570 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f355001e460 con 0x7f355c0cce40 2023-10-10T20:41:20.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.694+0000 7f3543fff700 1 --2- 172.21.15.89:0/371540570 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3548067320 0x7f35480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.694+0000 7f3543fff700 1 -- 172.21.15.89:0/371540570 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3550085160 con 0x7f355c0cce40 2023-10-10T20:41:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.694+0000 7f355bfff700 1 --2- 172.21.15.89:0/371540570 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3548067320 0x7f35480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.694+0000 7f355bfff700 1 --2- 172.21.15.89:0/371540570 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3548067320 0x7f35480697d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f354c006ea0 tx=0x7f354c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.698+0000 7f3543fff700 1 -- 172.21.15.89:0/371540570 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3550055880 con 0x7f355c0cce40 2023-10-10T20:41:20.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.922+0000 7f35627a7700 1 -- 172.21.15.89:0/371540570 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f355c0ce020 con 0x7f355c0cce40 2023-10-10T20:41:20.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.960+0000 7f3543fff700 1 -- 172.21.15.89:0/371540570 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v74) v1 ==== 136+0+0 (secure 0 0 0) 0x7f3550055410 con 0x7f355c0cce40 2023-10-10T20:41:20.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.962+0000 7f35627a7700 1 -- 172.21.15.89:0/371540570 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3548067320 msgr2=0x7f35480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:20.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.962+0000 7f35627a7700 1 --2- 172.21.15.89:0/371540570 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3548067320 0x7f35480697d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f354c006ea0 tx=0x7f354c008040 comp rx=0 tx=0).stop 2023-10-10T20:41:20.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.962+0000 7f35627a7700 1 -- 172.21.15.89:0/371540570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c0cce40 msgr2=0x7f355c069200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:20.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.962+0000 7f35627a7700 1 --2- 172.21.15.89:0/371540570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c0cce40 0x7f355c069200 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f355000d860 tx=0x7f3550008040 comp rx=0 tx=0).stop 2023-10-10T20:41:20.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.962+0000 7f35627a7700 1 -- 172.21.15.89:0/371540570 shutdown_connections 2023-10-10T20:41:20.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.962+0000 7f35627a7700 1 --2- 172.21.15.89:0/371540570 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3548067320 0x7f35480697d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:20.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.962+0000 7f35627a7700 1 --2- 172.21.15.89:0/371540570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f355c159ec0 0x7f355c0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:20.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.962+0000 7f35627a7700 1 --2- 172.21.15.89:0/371540570 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f355c0b53d0 0x7f355c13eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:20.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.962+0000 7f35627a7700 1 --2- 172.21.15.89:0/371540570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c0cce40 0x7f355c069200 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:20.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.962+0000 7f35627a7700 1 -- 172.21.15.89:0/371540570 >> 172.21.15.89:0/371540570 conn(0x7f355c0999f0 msgr2=0x7f355c158f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:20.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.962+0000 7f35627a7700 1 -- 172.21.15.89:0/371540570 shutdown_connections 2023-10-10T20:41:20.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:20.962+0000 7f35627a7700 1 -- 172.21.15.89:0/371540570 wait complete. 2023-10-10T20:41:20.976 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-10T20:41:21.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.091+0000 7fa177431700 1 Processor -- start 2023-10-10T20:41:21.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.091+0000 7fa177431700 1 -- start start 2023-10-10T20:41:21.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.092+0000 7fa177431700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa170159890 0x7fa17015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:21.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.092+0000 7fa177431700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17015a160 con 0x7fa170069560 2023-10-10T20:41:21.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.092+0000 7fa177431700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1700d13a0 con 0x7fa170159c60 2023-10-10T20:41:21.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.092+0000 7fa177431700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17015e140 con 0x7fa170159890 2023-10-10T20:41:21.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.092+0000 7fa1751cd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa170159890 0x7fa17015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:21.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.092+0000 7fa1759ce700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa170069560 0x7fa1700ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45360/0 (socket says 172.21.15.89:45360) 2023-10-10T20:41:21.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.092+0000 7fa1759ce700 1 -- 172.21.15.89:0/2498419963 learned_addr learned my addr 172.21.15.89:0/2498419963 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:21.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.092+0000 7fa1751cd700 1 -- 172.21.15.89:0/2498419963 >> v1:172.21.15.111:6789/0 conn(0x7fa170159c60 legacy=0x7fa17015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.092+0000 7fa1751cd700 1 -- 172.21.15.89:0/2498419963 >> v1:172.21.15.89:6789/0 conn(0x7fa170069560 legacy=0x7fa1700ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.092+0000 7fa1751cd700 1 -- 172.21.15.89:0/2498419963 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17015a3d0 con 0x7fa170159890 2023-10-10T20:41:21.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.092+0000 7fa167fff700 1 -- 172.21.15.89:0/2498419963 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672600222 0 0) 0x7fa17015a160 con 0x7fa170069560 2023-10-10T20:41:21.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.092+0000 7fa1751cd700 1 --2- 172.21.15.89:0/2498419963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa170159890 0x7fa17015abd0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0098e0 tx=0x7fa16c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48293081effb76 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:21.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.092+0000 7fa167fff700 1 -- 172.21.15.89:0/2498419963 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c009e80 con 0x7fa170159890 2023-10-10T20:41:21.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.092+0000 7fa167fff700 1 -- 172.21.15.89:0/2498419963 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16c00b860 con 0x7fa170159890 2023-10-10T20:41:21.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.092+0000 7fa167fff700 1 -- 172.21.15.89:0/2498419963 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c00bbe0 con 0x7fa170159890 2023-10-10T20:41:21.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.093+0000 7fa177431700 1 -- 172.21.15.89:0/2498419963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa170159890 msgr2=0x7fa17015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.093+0000 7fa177431700 1 --2- 172.21.15.89:0/2498419963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa170159890 0x7fa17015abd0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0098e0 tx=0x7fa16c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.093+0000 7fa177431700 1 -- 172.21.15.89:0/2498419963 shutdown_connections 2023-10-10T20:41:21.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.093+0000 7fa177431700 1 --2- 172.21.15.89:0/2498419963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa170159890 0x7fa17015abd0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.093+0000 7fa177431700 1 -- 172.21.15.89:0/2498419963 >> 172.21.15.89:0/2498419963 conn(0x7fa170068e20 msgr2=0x7fa1700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:21.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.093+0000 7fa177431700 1 -- 172.21.15.89:0/2498419963 shutdown_connections 2023-10-10T20:41:21.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.093+0000 7fa177431700 1 -- 172.21.15.89:0/2498419963 wait complete. 2023-10-10T20:41:21.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.094+0000 7fa177431700 1 Processor -- start 2023-10-10T20:41:21.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.094+0000 7fa177431700 1 -- start start 2023-10-10T20:41:21.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.095+0000 7fa177431700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa170069560 0x7fa170154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:21.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.095+0000 7fa177431700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa170159890 0x7fa17015d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:21.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.095+0000 7fa177431700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa170159c60 0x7fa17015c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:21.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.095+0000 7fa177431700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17014cba0 con 0x7fa170159c60 2023-10-10T20:41:21.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.095+0000 7fa177431700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17016b440 con 0x7fa170159890 2023-10-10T20:41:21.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.095+0000 7fa177431700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17015ca30 con 0x7fa170069560 2023-10-10T20:41:21.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.095+0000 7fa1759ce700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa170159c60 0x7fa17015c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:21.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.095+0000 7fa1759ce700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa170159c60 0x7fa17015c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49142/0 (socket says 172.21.15.89:49142) 2023-10-10T20:41:21.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.095+0000 7fa1759ce700 1 -- 172.21.15.89:0/3982083297 learned_addr learned my addr 172.21.15.89:0/3982083297 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:21.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.095+0000 7fa1749cc700 1 --2- 172.21.15.89:0/3982083297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa170159890 0x7fa17015d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:21.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.095+0000 7fa1751cd700 1 --2- 172.21.15.89:0/3982083297 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa170069560 0x7fa170154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:21.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.096+0000 7fa1759ce700 1 -- 172.21.15.89:0/3982083297 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa170069560 msgr2=0x7fa170154b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.096+0000 7fa1759ce700 1 --2- 172.21.15.89:0/3982083297 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa170069560 0x7fa170154b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.096+0000 7fa1759ce700 1 -- 172.21.15.89:0/3982083297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa170159890 msgr2=0x7fa17015d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.096+0000 7fa1759ce700 1 --2- 172.21.15.89:0/3982083297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa170159890 0x7fa17015d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.096+0000 7fa1759ce700 1 -- 172.21.15.89:0/3982083297 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c009420 con 0x7fa170159c60 2023-10-10T20:41:21.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.096+0000 7fa1749cc700 1 --2- 172.21.15.89:0/3982083297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa170159890 0x7fa17015d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:21.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.096+0000 7fa1759ce700 1 --2- 172.21.15.89:0/3982083297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa170159c60 0x7fa17015c040 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa16800cc90 tx=0x7fa168001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:21.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.096+0000 7fa1657fa700 1 -- 172.21.15.89:0/3982083297 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16800cea0 con 0x7fa170159c60 2023-10-10T20:41:21.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.096+0000 7fa1657fa700 1 -- 172.21.15.89:0/3982083297 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16800ab70 con 0x7fa170159c60 2023-10-10T20:41:21.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.096+0000 7fa177431700 1 -- 172.21.15.89:0/3982083297 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17016b160 con 0x7fa170159c60 2023-10-10T20:41:21.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.096+0000 7fa1657fa700 1 -- 172.21.15.89:0/3982083297 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1680070f0 con 0x7fa170159c60 2023-10-10T20:41:21.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.096+0000 7fa177431700 1 -- 172.21.15.89:0/3982083297 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17015cfb0 con 0x7fa170159c60 2023-10-10T20:41:21.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.097+0000 7fa177431700 1 -- 172.21.15.89:0/3982083297 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17015d1d0 con 0x7fa170159c60 2023-10-10T20:41:21.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.097+0000 7fa1657fa700 1 -- 172.21.15.89:0/3982083297 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa1680074a0 con 0x7fa170159c60 2023-10-10T20:41:21.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.098+0000 7fa1657fa700 1 --2- 172.21.15.89:0/3982083297 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa154067320 0x7fa1540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:21.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.098+0000 7fa1751cd700 1 --2- 172.21.15.89:0/3982083297 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa154067320 0x7fa1540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:21.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.098+0000 7fa1657fa700 1 -- 172.21.15.89:0/3982083297 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa168085120 con 0x7fa170159c60 2023-10-10T20:41:21.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.098+0000 7fa1751cd700 1 --2- 172.21.15.89:0/3982083297 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa154067320 0x7fa1540697d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0098e0 tx=0x7fa16c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:21.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.102+0000 7fa1657fa700 1 -- 172.21.15.89:0/3982083297 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa168051e30 con 0x7fa170159c60 2023-10-10T20:41:21.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.314+0000 7fa177431700 1 -- 172.21.15.89:0/3982083297 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa17004c3c0 con 0x7fa154067320 2023-10-10T20:41:21.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.315+0000 7fa1657fa700 1 -- 172.21.15.89:0/3982083297 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa17004c3c0 con 0x7fa154067320 2023-10-10T20:41:21.317 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:41:21.318 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:21.318 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs" 2023-10-10T20:41:21.318 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:21.318 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:41:21.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.317+0000 7fa177431700 1 -- 172.21.15.89:0/3982083297 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa154067320 msgr2=0x7fa1540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.317+0000 7fa177431700 1 --2- 172.21.15.89:0/3982083297 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa154067320 0x7fa1540697d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0098e0 tx=0x7fa16c017000 comp rx=0 tx=0).stop 2023-10-10T20:41:21.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.317+0000 7fa177431700 1 -- 172.21.15.89:0/3982083297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa170159c60 msgr2=0x7fa17015c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.317+0000 7fa177431700 1 --2- 172.21.15.89:0/3982083297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa170159c60 0x7fa17015c040 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa16800cc90 tx=0x7fa168001a80 comp rx=0 tx=0).stop 2023-10-10T20:41:21.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.317+0000 7fa177431700 1 -- 172.21.15.89:0/3982083297 shutdown_connections 2023-10-10T20:41:21.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.317+0000 7fa177431700 1 --2- 172.21.15.89:0/3982083297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa170159c60 0x7fa17015c040 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.317+0000 7fa177431700 1 --2- 172.21.15.89:0/3982083297 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa170069560 0x7fa170154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.317+0000 7fa177431700 1 --2- 172.21.15.89:0/3982083297 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa154067320 0x7fa1540697d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.317+0000 7fa177431700 1 --2- 172.21.15.89:0/3982083297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa170159890 0x7fa17015d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.317+0000 7fa177431700 1 -- 172.21.15.89:0/3982083297 >> 172.21.15.89:0/3982083297 conn(0x7fa170068e20 msgr2=0x7fa170152da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:21.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.317+0000 7fa177431700 1 -- 172.21.15.89:0/3982083297 shutdown_connections 2023-10-10T20:41:21.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.317+0000 7fa177431700 1 -- 172.21.15.89:0/3982083297 wait complete. 2023-10-10T20:41:21.331 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:41:21.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.447+0000 7f4507390700 1 Processor -- start 2023-10-10T20:41:21.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.447+0000 7f4507390700 1 -- start start 2023-10-10T20:41:21.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.448+0000 7f4507390700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4500069560 0x7f45000699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:21.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.448+0000 7f4507390700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4500069ee0 con 0x7f4500159890 2023-10-10T20:41:21.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.448+0000 7f4507390700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f450015a160 con 0x7f4500159c60 2023-10-10T20:41:21.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.448+0000 7f4507390700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45000d13a0 con 0x7f4500069560 2023-10-10T20:41:21.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.448+0000 7f450512c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4500159890 0x7f450015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45364/0 (socket says 172.21.15.89:45364) 2023-10-10T20:41:21.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.448+0000 7f450512c700 1 -- 172.21.15.89:0/44453134 learned_addr learned my addr 172.21.15.89:0/44453134 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:21.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.448+0000 7f450592d700 1 --2- 172.21.15.89:0/44453134 >> v2:172.21.15.111:3300/0 conn(0x7f4500069560 0x7f45000699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:21.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.448+0000 7f44f7fff700 1 -- 172.21.15.89:0/44453134 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645552660 0 0) 0x7f4500069ee0 con 0x7f4500159890 2023-10-10T20:41:21.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.448+0000 7f44f7fff700 1 -- 172.21.15.89:0/44453134 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44ec003570 con 0x7f4500159890 2023-10-10T20:41:21.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.449+0000 7f44f7fff700 1 -- 172.21.15.89:0/44453134 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 702056245 0 0) 0x7f44ec003570 con 0x7f4500159890 2023-10-10T20:41:21.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.449+0000 7f450592d700 1 -- 172.21.15.89:0/44453134 >> v1:172.21.15.111:6789/0 conn(0x7f4500159c60 legacy=0x7f450015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.449+0000 7f450592d700 1 -- 172.21.15.89:0/44453134 >> v1:172.21.15.89:6789/0 conn(0x7f4500159890 legacy=0x7f450015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.449+0000 7f450592d700 1 -- 172.21.15.89:0/44453134 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450015e140 con 0x7f4500069560 2023-10-10T20:41:21.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.449+0000 7f450592d700 1 --2- 172.21.15.89:0/44453134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4500069560 0x7f45000699a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f44f8009e60 tx=0x7f44f8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4a2247b0f8d3936 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:21.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.449+0000 7f44f7fff700 1 -- 172.21.15.89:0/44453134 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f800e070 con 0x7f4500069560 2023-10-10T20:41:21.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.449+0000 7f44f7fff700 1 -- 172.21.15.89:0/44453134 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f800b850 con 0x7f4500069560 2023-10-10T20:41:21.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.449+0000 7f44f7fff700 1 -- 172.21.15.89:0/44453134 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc001470 con 0x7f4500069560 2023-10-10T20:41:21.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.450+0000 7f4507390700 1 -- 172.21.15.89:0/44453134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4500069560 msgr2=0x7f45000699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.450+0000 7f4507390700 1 --2- 172.21.15.89:0/44453134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4500069560 0x7f45000699a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f44f8009e60 tx=0x7f44f8009780 comp rx=0 tx=0).stop 2023-10-10T20:41:21.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.450+0000 7f4507390700 1 -- 172.21.15.89:0/44453134 shutdown_connections 2023-10-10T20:41:21.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.450+0000 7f4507390700 1 --2- 172.21.15.89:0/44453134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4500069560 0x7f45000699a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.450+0000 7f4507390700 1 -- 172.21.15.89:0/44453134 >> 172.21.15.89:0/44453134 conn(0x7f4500068e20 msgr2=0x7f45000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:21.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.450+0000 7f4507390700 1 -- 172.21.15.89:0/44453134 shutdown_connections 2023-10-10T20:41:21.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.450+0000 7f4507390700 1 -- 172.21.15.89:0/44453134 wait complete. 2023-10-10T20:41:21.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.451+0000 7f4507390700 1 Processor -- start 2023-10-10T20:41:21.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.451+0000 7f4507390700 1 -- start start 2023-10-10T20:41:21.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f4507390700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4500069560 0x7f4500169320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:21.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f4507390700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4500159890 0x7f45000b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:21.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f4507390700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4500159c60 0x7f45000ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:21.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f4507390700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45000b3900 con 0x7f4500069560 2023-10-10T20:41:21.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f4507390700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4500169be0 con 0x7f4500159890 2023-10-10T20:41:21.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f4507390700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4500169ec0 con 0x7f4500159c60 2023-10-10T20:41:21.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f450512c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4500069560 0x7f4500169320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:21.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f450512c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4500069560 0x7f4500169320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49144/0 (socket says 172.21.15.89:49144) 2023-10-10T20:41:21.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f450512c700 1 -- 172.21.15.89:0/1852689413 learned_addr learned my addr 172.21.15.89:0/1852689413 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:21.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f450492b700 1 --2- 172.21.15.89:0/1852689413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4500159890 0x7f45000b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:21.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f450592d700 1 --2- 172.21.15.89:0/1852689413 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4500159c60 0x7f45000ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:21.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f450512c700 1 -- 172.21.15.89:0/1852689413 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4500159c60 msgr2=0x7f45000ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f450512c700 1 --2- 172.21.15.89:0/1852689413 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4500159c60 0x7f45000ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f450512c700 1 -- 172.21.15.89:0/1852689413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4500159890 msgr2=0x7f45000b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f450512c700 1 --2- 172.21.15.89:0/1852689413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4500159890 0x7f45000b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f450512c700 1 -- 172.21.15.89:0/1852689413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f8009a20 con 0x7f4500069560 2023-10-10T20:41:21.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f450512c700 1 --2- 172.21.15.89:0/1852689413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4500069560 0x7f4500169320 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00cb60 tx=0x7f44fc001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:21.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f44f57fa700 1 -- 172.21.15.89:0/1852689413 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc00ce70 con 0x7f4500069560 2023-10-10T20:41:21.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f44f57fa700 1 -- 172.21.15.89:0/1852689413 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44fc007360 con 0x7f4500069560 2023-10-10T20:41:21.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f44f57fa700 1 -- 172.21.15.89:0/1852689413 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc004830 con 0x7f4500069560 2023-10-10T20:41:21.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f4507390700 1 -- 172.21.15.89:0/1852689413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450016a1a0 con 0x7f4500069560 2023-10-10T20:41:21.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.452+0000 7f4507390700 1 -- 172.21.15.89:0/1852689413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4500169860 con 0x7f4500069560 2023-10-10T20:41:21.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.453+0000 7f4507390700 1 -- 172.21.15.89:0/1852689413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45000b3be0 con 0x7f4500069560 2023-10-10T20:41:21.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.454+0000 7f44f57fa700 1 -- 172.21.15.89:0/1852689413 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f44fc004be0 con 0x7f4500069560 2023-10-10T20:41:21.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.454+0000 7f44f57fa700 1 --2- 172.21.15.89:0/1852689413 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f44e4067350 0x7f44e4069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:21.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.454+0000 7f450492b700 1 --2- 172.21.15.89:0/1852689413 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f44e4067350 0x7f44e4069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:21.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.454+0000 7f44f57fa700 1 -- 172.21.15.89:0/1852689413 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f44fc083e80 con 0x7f4500069560 2023-10-10T20:41:21.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.454+0000 7f450492b700 1 --2- 172.21.15.89:0/1852689413 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f44e4067350 0x7f44e4069800 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f450016c880 tx=0x7f44f0007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:21.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.458+0000 7f44f57fa700 1 -- 172.21.15.89:0/1852689413 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44fc050b90 con 0x7f4500069560 2023-10-10T20:41:21.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.640+0000 7f4507390700 1 -- 172.21.15.89:0/1852689413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f450004c3c0 con 0x7f4500069560 2023-10-10T20:41:21.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.640+0000 7f44f57fa700 1 -- 172.21.15.89:0/1852689413 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f44fc018020 con 0x7f4500069560 2023-10-10T20:41:21.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.642+0000 7f4507390700 1 -- 172.21.15.89:0/1852689413 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f44e4067350 msgr2=0x7f44e4069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.642+0000 7f4507390700 1 --2- 172.21.15.89:0/1852689413 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f44e4067350 0x7f44e4069800 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f450016c880 tx=0x7f44f0007ec0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.642+0000 7f4507390700 1 -- 172.21.15.89:0/1852689413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4500069560 msgr2=0x7f4500169320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.642+0000 7f4507390700 1 --2- 172.21.15.89:0/1852689413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4500069560 0x7f4500169320 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00cb60 tx=0x7f44fc001b50 comp rx=0 tx=0).stop 2023-10-10T20:41:21.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.642+0000 7f4507390700 1 -- 172.21.15.89:0/1852689413 shutdown_connections 2023-10-10T20:41:21.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.642+0000 7f4507390700 1 --2- 172.21.15.89:0/1852689413 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4500159c60 0x7f45000ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.642+0000 7f4507390700 1 --2- 172.21.15.89:0/1852689413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4500069560 0x7f4500169320 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.642+0000 7f4507390700 1 --2- 172.21.15.89:0/1852689413 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f44e4067350 0x7f44e4069800 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.642+0000 7f4507390700 1 --2- 172.21.15.89:0/1852689413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4500159890 0x7f45000b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.642+0000 7f4507390700 1 -- 172.21.15.89:0/1852689413 >> 172.21.15.89:0/1852689413 conn(0x7f4500068e20 msgr2=0x7f45000d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:21.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.642+0000 7f4507390700 1 -- 172.21.15.89:0/1852689413 shutdown_connections 2023-10-10T20:41:21.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.642+0000 7f4507390700 1 -- 172.21.15.89:0/1852689413 wait complete. 2023-10-10T20:41:21.657 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-10T20:41:21.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.771+0000 7f6164994700 1 Processor -- start 2023-10-10T20:41:21.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.771+0000 7f6164994700 1 -- start start 2023-10-10T20:41:21.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.772+0000 7f6164994700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6160159890 0x7f616015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:21.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.772+0000 7f6164994700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6160069560 0x7f61600699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:21.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.772+0000 7f6164994700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6160069ee0 con 0x7f6160159c60 2023-10-10T20:41:21.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.772+0000 7f6164994700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f616015a160 con 0x7f6160159890 2023-10-10T20:41:21.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.772+0000 7f6164994700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61600d13a0 con 0x7f6160069560 2023-10-10T20:41:21.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.772+0000 7f615ed9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6160069560 0x7f61600699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:21.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.772+0000 7f615dd9b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f6160159c60 0x7f616015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:57216/0 (socket says 172.21.15.89:57216) 2023-10-10T20:41:21.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.772+0000 7f615e59c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6160159890 0x7f616015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:21.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.772+0000 7f615dd9b700 1 -- 172.21.15.89:0/3593961737 learned_addr learned my addr 172.21.15.89:0/3593961737 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:21.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.772+0000 7f615e59c700 1 -- 172.21.15.89:0/3593961737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6160069560 msgr2=0x7f61600699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:21.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.772+0000 7f615e59c700 1 --2- 172.21.15.89:0/3593961737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6160069560 0x7f61600699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.772+0000 7f615e59c700 1 -- 172.21.15.89:0/3593961737 >> v1:172.21.15.111:6789/0 conn(0x7f6160159c60 legacy=0x7f616015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.772+0000 7f615e59c700 1 -- 172.21.15.89:0/3593961737 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616015e140 con 0x7f6160159890 2023-10-10T20:41:21.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.772+0000 7f615d59a700 1 -- 172.21.15.89:0/3593961737 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550453573 0 0) 0x7f6160069ee0 con 0x7f6160159c60 2023-10-10T20:41:21.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.772+0000 7f615e59c700 1 --2- 172.21.15.89:0/3593961737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6160159890 0x7f616015abd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6150009e60 tx=0x7f6150009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83f0b4a35005f2ff server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:21.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.773+0000 7f615d59a700 1 -- 172.21.15.89:0/3593961737 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615000e070 con 0x7f6160159890 2023-10-10T20:41:21.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.773+0000 7f615d59a700 1 -- 172.21.15.89:0/3593961737 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615000bb60 con 0x7f6160159890 2023-10-10T20:41:21.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.773+0000 7f615d59a700 1 -- 172.21.15.89:0/3593961737 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6150012430 con 0x7f6160159890 2023-10-10T20:41:21.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.773+0000 7f6164994700 1 -- 172.21.15.89:0/3593961737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6160159890 msgr2=0x7f616015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.773+0000 7f6164994700 1 --2- 172.21.15.89:0/3593961737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6160159890 0x7f616015abd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6150009e60 tx=0x7f6150009780 comp rx=0 tx=0).stop 2023-10-10T20:41:21.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.773+0000 7f6164994700 1 -- 172.21.15.89:0/3593961737 shutdown_connections 2023-10-10T20:41:21.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.773+0000 7f6164994700 1 --2- 172.21.15.89:0/3593961737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6160159890 0x7f616015abd0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.773+0000 7f6164994700 1 --2- 172.21.15.89:0/3593961737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6160069560 0x7f61600699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.773+0000 7f6164994700 1 -- 172.21.15.89:0/3593961737 >> 172.21.15.89:0/3593961737 conn(0x7f6160068e20 msgr2=0x7f61600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:21.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.773+0000 7f6164994700 1 -- 172.21.15.89:0/3593961737 shutdown_connections 2023-10-10T20:41:21.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.773+0000 7f6164994700 1 -- 172.21.15.89:0/3593961737 wait complete. 2023-10-10T20:41:21.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.774+0000 7f6164994700 1 Processor -- start 2023-10-10T20:41:21.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.774+0000 7f6164994700 1 -- start start 2023-10-10T20:41:21.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.775+0000 7f6164994700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6160069560 0x7f616016d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:21.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.775+0000 7f6164994700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6160159890 0x7f61600c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:21.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.775+0000 7f6164994700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6160159c60 0x7f61600cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:21.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.775+0000 7f6164994700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61601537e0 con 0x7f6160159890 2023-10-10T20:41:21.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.775+0000 7f6164994700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616016ddd0 con 0x7f6160159c60 2023-10-10T20:41:21.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.775+0000 7f6164994700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6160153a50 con 0x7f6160069560 2023-10-10T20:41:21.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.775+0000 7f615dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6160159890 0x7f61600c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:21.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.775+0000 7f615dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6160159890 0x7f61600c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49152/0 (socket says 172.21.15.89:49152) 2023-10-10T20:41:21.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.775+0000 7f615dd9b700 1 -- 172.21.15.89:0/2523813850 learned_addr learned my addr 172.21.15.89:0/2523813850 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:21.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.775+0000 7f615e59c700 1 --2- 172.21.15.89:0/2523813850 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6160069560 0x7f616016d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:21.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.775+0000 7f615ed9d700 1 --2- 172.21.15.89:0/2523813850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6160159c60 0x7f61600cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:21.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.775+0000 7f615dd9b700 1 -- 172.21.15.89:0/2523813850 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6160069560 msgr2=0x7f616016d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.775+0000 7f615dd9b700 1 --2- 172.21.15.89:0/2523813850 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6160069560 0x7f616016d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.776+0000 7f615dd9b700 1 -- 172.21.15.89:0/2523813850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6160159c60 msgr2=0x7f61600cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.776+0000 7f615dd9b700 1 --2- 172.21.15.89:0/2523813850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6160159c60 0x7f61600cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.776+0000 7f615dd9b700 1 -- 172.21.15.89:0/2523813850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6150009a20 con 0x7f6160159890 2023-10-10T20:41:21.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.776+0000 7f615dd9b700 1 --2- 172.21.15.89:0/2523813850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6160159890 0x7f61600c9740 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f615400c810 tx=0x7f6154007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:21.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.776+0000 7f614e7fc700 1 -- 172.21.15.89:0/2523813850 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154005b30 con 0x7f6160159890 2023-10-10T20:41:21.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.776+0000 7f614e7fc700 1 -- 172.21.15.89:0/2523813850 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6154005c90 con 0x7f6160159890 2023-10-10T20:41:21.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.776+0000 7f6164994700 1 -- 172.21.15.89:0/2523813850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616016e000 con 0x7f6160159890 2023-10-10T20:41:21.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.776+0000 7f614e7fc700 1 -- 172.21.15.89:0/2523813850 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154007300 con 0x7f6160159890 2023-10-10T20:41:21.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.776+0000 7f6164994700 1 -- 172.21.15.89:0/2523813850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616016e5c0 con 0x7f6160159890 2023-10-10T20:41:21.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.777+0000 7f6164994700 1 -- 172.21.15.89:0/2523813850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f616004c3c0 con 0x7f6160159890 2023-10-10T20:41:21.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.777+0000 7f614e7fc700 1 -- 172.21.15.89:0/2523813850 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f615401b400 con 0x7f6160159890 2023-10-10T20:41:21.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.777+0000 7f614e7fc700 1 --2- 172.21.15.89:0/2523813850 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6140067320 0x7f61400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:21.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.778+0000 7f614e7fc700 1 -- 172.21.15.89:0/2523813850 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6154084640 con 0x7f6160159890 2023-10-10T20:41:21.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.778+0000 7f615e59c700 1 --2- 172.21.15.89:0/2523813850 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6140067320 0x7f61400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:21.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.778+0000 7f615e59c700 1 --2- 172.21.15.89:0/2523813850 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6140067320 0x7f61400697d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f61500097c0 tx=0x7f6150019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:21.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.782+0000 7f614e7fc700 1 -- 172.21.15.89:0/2523813850 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6154051450 con 0x7f6160159890 2023-10-10T20:41:21.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.991+0000 7f6164994700 1 -- 172.21.15.89:0/2523813850 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6160153b90 con 0x7f6140067320 2023-10-10T20:41:21.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.993+0000 7f614e7fc700 1 -- 172.21.15.89:0/2523813850 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6160153b90 con 0x7f6140067320 2023-10-10T20:41:21.995 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:41:21.995 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:21.995 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs" 2023-10-10T20:41:21.996 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:21.996 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:41:21.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.995+0000 7f6164994700 1 -- 172.21.15.89:0/2523813850 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6140067320 msgr2=0x7f61400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.995+0000 7f6164994700 1 --2- 172.21.15.89:0/2523813850 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6140067320 0x7f61400697d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f61500097c0 tx=0x7f6150019000 comp rx=0 tx=0).stop 2023-10-10T20:41:21.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.995+0000 7f6164994700 1 -- 172.21.15.89:0/2523813850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6160159890 msgr2=0x7f61600c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:21.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.995+0000 7f6164994700 1 --2- 172.21.15.89:0/2523813850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6160159890 0x7f61600c9740 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f615400c810 tx=0x7f6154007df0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.995+0000 7f6164994700 1 -- 172.21.15.89:0/2523813850 shutdown_connections 2023-10-10T20:41:21.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.995+0000 7f6164994700 1 --2- 172.21.15.89:0/2523813850 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6140067320 0x7f61400697d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.995+0000 7f6164994700 1 --2- 172.21.15.89:0/2523813850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6160159c60 0x7f61600cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.995+0000 7f6164994700 1 --2- 172.21.15.89:0/2523813850 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6160069560 0x7f616016d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.995+0000 7f6164994700 1 --2- 172.21.15.89:0/2523813850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6160159890 0x7f61600c9740 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:21.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.995+0000 7f6164994700 1 -- 172.21.15.89:0/2523813850 >> 172.21.15.89:0/2523813850 conn(0x7f6160068e20 msgr2=0x7f6160152f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:21.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.995+0000 7f6164994700 1 -- 172.21.15.89:0/2523813850 shutdown_connections 2023-10-10T20:41:22.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:21.995+0000 7f6164994700 1 -- 172.21.15.89:0/2523813850 wait complete. 2023-10-10T20:41:22.009 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000000990214 2023-10-10T20:41:22.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.123+0000 7fae20481700 1 Processor -- start 2023-10-10T20:41:22.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.123+0000 7fae20481700 1 -- start start 2023-10-10T20:41:22.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae20481700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fae18159890 0x7fae1815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:22.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae20481700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae18159c60 0x7fae1815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:22.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae20481700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fae1815db10 0x7fae18069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:22.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae20481700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1815a160 con 0x7fae18159890 2023-10-10T20:41:22.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae20481700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae180d13a0 con 0x7fae1815db10 2023-10-10T20:41:22.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae20481700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1815e140 con 0x7fae18159c60 2023-10-10T20:41:22.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae1e21d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fae18159890 0x7fae1815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:22.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae1e21d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fae18159890 0x7fae1815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49158/0 (socket says 172.21.15.89:49158) 2023-10-10T20:41:22.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae1e21d700 1 -- 172.21.15.89:0/826879699 learned_addr learned my addr 172.21.15.89:0/826879699 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:22.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae1ea1e700 1 --2- 172.21.15.89:0/826879699 >> v2:172.21.15.111:3300/0 conn(0x7fae1815db10 0x7fae18069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:22.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae1e21d700 1 -- 172.21.15.89:0/826879699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae18159c60 msgr2=0x7fae1815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:22.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae1da1c700 1 --2- 172.21.15.89:0/826879699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae18159c60 0x7fae1815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:22.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae1e21d700 1 --2- 172.21.15.89:0/826879699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae18159c60 0x7fae1815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:22.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae1e21d700 1 -- 172.21.15.89:0/826879699 >> v2:172.21.15.111:3300/0 conn(0x7fae1815db10 msgr2=0x7fae18069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:22.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae1e21d700 1 --2- 172.21.15.89:0/826879699 >> v2:172.21.15.111:3300/0 conn(0x7fae1815db10 0x7fae18069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:22.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae1e21d700 1 -- 172.21.15.89:0/826879699 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1815a3d0 con 0x7fae18159890 2023-10-10T20:41:22.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae1ea1e700 1 --2- 172.21.15.89:0/826879699 >> v2:172.21.15.111:3300/0 conn(0x7fae1815db10 0x7fae18069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:22.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae1da1c700 1 --2- 172.21.15.89:0/826879699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae18159c60 0x7fae1815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:22.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae1e21d700 1 --2- 172.21.15.89:0/826879699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae18159890 0x7fae1815abd0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fae18089b50 tx=0x7fae14009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9249a23cf7507e3b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:22.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae1d21b700 1 -- 172.21.15.89:0/826879699 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1400e070 con 0x7fae18159890 2023-10-10T20:41:22.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae1d21b700 1 -- 172.21.15.89:0/826879699 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1400ba50 con 0x7fae18159890 2023-10-10T20:41:22.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.124+0000 7fae1d21b700 1 -- 172.21.15.89:0/826879699 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae140092e0 con 0x7fae18159890 2023-10-10T20:41:22.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.125+0000 7fae20481700 1 -- 172.21.15.89:0/826879699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae18159890 msgr2=0x7fae1815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:22.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.125+0000 7fae20481700 1 --2- 172.21.15.89:0/826879699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae18159890 0x7fae1815abd0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fae18089b50 tx=0x7fae14009840 comp rx=0 tx=0).stop 2023-10-10T20:41:22.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.125+0000 7fae20481700 1 -- 172.21.15.89:0/826879699 shutdown_connections 2023-10-10T20:41:22.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.125+0000 7fae20481700 1 --2- 172.21.15.89:0/826879699 >> v2:172.21.15.111:3300/0 conn(0x7fae1815db10 0x7fae18069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:22.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.125+0000 7fae20481700 1 --2- 172.21.15.89:0/826879699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae18159890 0x7fae1815abd0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:22.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.125+0000 7fae20481700 1 --2- 172.21.15.89:0/826879699 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae18159c60 0x7fae1815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:22.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.125+0000 7fae20481700 1 -- 172.21.15.89:0/826879699 >> 172.21.15.89:0/826879699 conn(0x7fae18068e20 msgr2=0x7fae180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:22.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.125+0000 7fae20481700 1 -- 172.21.15.89:0/826879699 shutdown_connections 2023-10-10T20:41:22.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.125+0000 7fae20481700 1 -- 172.21.15.89:0/826879699 wait complete. 2023-10-10T20:41:22.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.126+0000 7fae20481700 1 Processor -- start 2023-10-10T20:41:22.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.126+0000 7fae20481700 1 -- start start 2023-10-10T20:41:22.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.127+0000 7fae20481700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae18159890 0x7fae1814c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:22.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.127+0000 7fae20481700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae18159c60 0x7fae180cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:22.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.127+0000 7fae20481700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1815db10 0x7fae18151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:22.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.127+0000 7fae20481700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1814caf0 con 0x7fae18159890 2023-10-10T20:41:22.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.127+0000 7fae20481700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1814b2c0 con 0x7fae1815db10 2023-10-10T20:41:22.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.127+0000 7fae20481700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1814c120 con 0x7fae18159c60 2023-10-10T20:41:22.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.127+0000 7fae1e21d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae18159890 0x7fae1814c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:22.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.127+0000 7fae1e21d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae18159890 0x7fae1814c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49160/0 (socket says 172.21.15.89:49160) 2023-10-10T20:41:22.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.127+0000 7fae1e21d700 1 -- 172.21.15.89:0/3342682652 learned_addr learned my addr 172.21.15.89:0/3342682652 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:22.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.127+0000 7fae1ea1e700 1 --2- 172.21.15.89:0/3342682652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1815db10 0x7fae18151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:22.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.127+0000 7fae1da1c700 1 --2- 172.21.15.89:0/3342682652 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae18159c60 0x7fae180cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:22.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.127+0000 7fae1e21d700 1 -- 172.21.15.89:0/3342682652 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae18159c60 msgr2=0x7fae180cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:22.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.127+0000 7fae1e21d700 1 --2- 172.21.15.89:0/3342682652 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae18159c60 0x7fae180cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:22.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.127+0000 7fae1e21d700 1 -- 172.21.15.89:0/3342682652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1815db10 msgr2=0x7fae18151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:22.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.127+0000 7fae1e21d700 1 --2- 172.21.15.89:0/3342682652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1815db10 0x7fae18151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:22.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.127+0000 7fae1e21d700 1 -- 172.21.15.89:0/3342682652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae14009ae0 con 0x7fae18159890 2023-10-10T20:41:22.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.127+0000 7fae1e21d700 1 --2- 172.21.15.89:0/3342682652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae18159890 0x7fae1814c5b0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fae18070e20 tx=0x7fae14018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:22.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.128+0000 7fae0e7fc700 1 -- 172.21.15.89:0/3342682652 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14008980 con 0x7fae18159890 2023-10-10T20:41:22.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.128+0000 7fae0e7fc700 1 -- 172.21.15.89:0/3342682652 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae14008ae0 con 0x7fae18159890 2023-10-10T20:41:22.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.128+0000 7fae20481700 1 -- 172.21.15.89:0/3342682652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1814b510 con 0x7fae18159890 2023-10-10T20:41:22.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.128+0000 7fae0e7fc700 1 -- 172.21.15.89:0/3342682652 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14008ea0 con 0x7fae18159890 2023-10-10T20:41:22.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.128+0000 7fae20481700 1 -- 172.21.15.89:0/3342682652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1814bad0 con 0x7fae18159890 2023-10-10T20:41:22.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.128+0000 7fae20481700 1 -- 172.21.15.89:0/3342682652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae1814c400 con 0x7fae18159890 2023-10-10T20:41:22.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.129+0000 7fae0e7fc700 1 -- 172.21.15.89:0/3342682652 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fae14003040 con 0x7fae18159890 2023-10-10T20:41:22.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.129+0000 7fae0e7fc700 1 --2- 172.21.15.89:0/3342682652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fadfc067320 0x7fadfc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:22.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.129+0000 7fae0e7fc700 1 -- 172.21.15.89:0/3342682652 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fae14086c40 con 0x7fae18159890 2023-10-10T20:41:22.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.129+0000 7fae1da1c700 1 --2- 172.21.15.89:0/3342682652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fadfc067320 0x7fadfc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:22.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.130+0000 7fae1da1c700 1 --2- 172.21.15.89:0/3342682652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fadfc067320 0x7fadfc0697d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fae180b0c90 tx=0x7fae08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:22.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.134+0000 7fae0e7fc700 1 -- 172.21.15.89:0/3342682652 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae14053870 con 0x7fae18159890 2023-10-10T20:41:22.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:22.341+0000 7fae20481700 1 -- 172.21.15.89:0/3342682652 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000000990214", "target": ["mon-mgr", ""]}) v1 -- 0x7fae1804c3c0 con 0x7fadfc067320 2023-10-10T20:41:24.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:24.973+0000 7fae0e7fc700 1 -- 172.21.15.89:0/3342682652 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7fae1804c3c0 con 0x7fadfc067320 2023-10-10T20:41:24.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:24.977+0000 7fae20481700 1 -- 172.21.15.89:0/3342682652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fadfc067320 msgr2=0x7fadfc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:24.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:24.977+0000 7fae20481700 1 --2- 172.21.15.89:0/3342682652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fadfc067320 0x7fadfc0697d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fae180b0c90 tx=0x7fae08008040 comp rx=0 tx=0).stop 2023-10-10T20:41:24.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:24.977+0000 7fae20481700 1 -- 172.21.15.89:0/3342682652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae18159890 msgr2=0x7fae1814c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:24.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:24.977+0000 7fae20481700 1 --2- 172.21.15.89:0/3342682652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae18159890 0x7fae1814c5b0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fae18070e20 tx=0x7fae14018000 comp rx=0 tx=0).stop 2023-10-10T20:41:24.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:24.978+0000 7fae20481700 1 -- 172.21.15.89:0/3342682652 shutdown_connections 2023-10-10T20:41:24.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:24.978+0000 7fae20481700 1 --2- 172.21.15.89:0/3342682652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fadfc067320 0x7fadfc0697d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:24.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:24.978+0000 7fae20481700 1 --2- 172.21.15.89:0/3342682652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae1815db10 0x7fae18151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:24.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:24.978+0000 7fae20481700 1 --2- 172.21.15.89:0/3342682652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae18159890 0x7fae1814c5b0 secure :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fae18070e20 tx=0x7fae14018000 comp rx=0 tx=0).stop 2023-10-10T20:41:24.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:24.978+0000 7fae20481700 1 --2- 172.21.15.89:0/3342682652 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae18159c60 0x7fae180cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:24.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:24.978+0000 7fae20481700 1 -- 172.21.15.89:0/3342682652 >> 172.21.15.89:0/3342682652 conn(0x7fae18068e20 msgr2=0x7fae1814d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:24.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:24.979+0000 7fae20481700 1 -- 172.21.15.89:0/3342682652 shutdown_connections 2023-10-10T20:41:24.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:24.979+0000 7fae20481700 1 -- 172.21.15.89:0/3342682652 wait complete. 2023-10-10T20:41:24.982 INFO:teuthology.orchestra.run.smithi089.stderr:Volume created successfully (no MDS daemons created) 2023-10-10T20:41:24.997 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000000990215 2023-10-10T20:41:25.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.122+0000 7f72b5974700 1 Processor -- start 2023-10-10T20:41:25.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.122+0000 7f72b5974700 1 -- start start 2023-10-10T20:41:25.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.123+0000 7f72b5974700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f72b00c91e0 0x7f72b00cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:25.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.123+0000 7f72b5974700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f72b006ae10 0x7f72b006b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:25.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.123+0000 7f72b5974700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b00cc210 con 0x7f72b006aa40 2023-10-10T20:41:25.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.123+0000 7f72b5974700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b00cc4f0 con 0x7f72b00c91e0 2023-10-10T20:41:25.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.123+0000 7f72b5974700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b00cc7d0 con 0x7f72b006ae10 2023-10-10T20:41:25.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.123+0000 7f72a67fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f72b00c91e0 0x7f72b00cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:25.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.123+0000 7f72a67fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f72b00c91e0 0x7f72b00cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49168/0 (socket says 172.21.15.89:49168) 2023-10-10T20:41:25.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.123+0000 7f72a67fc700 1 -- 172.21.15.89:0/2417973881 learned_addr learned my addr 172.21.15.89:0/2417973881 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:25.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.123+0000 7f72af7fe700 1 --2- 172.21.15.89:0/2417973881 >> v2:172.21.15.111:3300/0 conn(0x7f72b006ae10 0x7f72b006b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:25.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.123+0000 7f72a67fc700 1 -- 172.21.15.89:0/2417973881 >> v2:172.21.15.111:3300/0 conn(0x7f72b006ae10 msgr2=0x7f72b006b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:25.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.123+0000 7f72a67fc700 1 --2- 172.21.15.89:0/2417973881 >> v2:172.21.15.111:3300/0 conn(0x7f72b006ae10 0x7f72b006b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:25.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.123+0000 7f72a67fc700 1 -- 172.21.15.89:0/2417973881 >> v1:172.21.15.111:6789/0 conn(0x7f72b006aa40 legacy=0x7f72b006bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:25.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.123+0000 7f72a67fc700 1 -- 172.21.15.89:0/2417973881 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b00ccab0 con 0x7f72b00c91e0 2023-10-10T20:41:25.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.123+0000 7f72a67fc700 1 --2- 172.21.15.89:0/2417973881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72b00c91e0 0x7f72b00cbc00 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f729800bc50 tx=0x7f729800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142c6ba50fece30a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:25.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.124+0000 7f72ae7fc700 1 -- 172.21.15.89:0/2417973881 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729800dbb0 con 0x7f72b00c91e0 2023-10-10T20:41:25.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.124+0000 7f72ae7fc700 1 -- 172.21.15.89:0/2417973881 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729800dd10 con 0x7f72b00c91e0 2023-10-10T20:41:25.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.124+0000 7f72ae7fc700 1 -- 172.21.15.89:0/2417973881 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980196b0 con 0x7f72b00c91e0 2023-10-10T20:41:25.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.124+0000 7f72b5974700 1 -- 172.21.15.89:0/2417973881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72b00c91e0 msgr2=0x7f72b00cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:25.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.124+0000 7f72b5974700 1 --2- 172.21.15.89:0/2417973881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72b00c91e0 0x7f72b00cbc00 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f729800bc50 tx=0x7f729800b4f0 comp rx=0 tx=0).stop 2023-10-10T20:41:25.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.124+0000 7f72b5974700 1 -- 172.21.15.89:0/2417973881 shutdown_connections 2023-10-10T20:41:25.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.124+0000 7f72b5974700 1 --2- 172.21.15.89:0/2417973881 >> v2:172.21.15.111:3300/0 conn(0x7f72b006ae10 0x7f72b006b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:25.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.124+0000 7f72b5974700 1 --2- 172.21.15.89:0/2417973881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72b00c91e0 0x7f72b00cbc00 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:25.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.124+0000 7f72b5974700 1 -- 172.21.15.89:0/2417973881 >> 172.21.15.89:0/2417973881 conn(0x7f72b00cd670 msgr2=0x7f72b00cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:25.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.124+0000 7f72b5974700 1 -- 172.21.15.89:0/2417973881 shutdown_connections 2023-10-10T20:41:25.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.124+0000 7f72b5974700 1 -- 172.21.15.89:0/2417973881 wait complete. 2023-10-10T20:41:25.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.125+0000 7f72b5974700 1 Processor -- start 2023-10-10T20:41:25.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.125+0000 7f72b5974700 1 -- start start 2023-10-10T20:41:25.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.126+0000 7f72b5974700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72b006aa40 0x7f72b0150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:25.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.126+0000 7f72b5974700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72b006ae10 0x7f72b0168ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:25.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.126+0000 7f72b5974700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72b00c91e0 0x7f72b00b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:25.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.126+0000 7f72b5974700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b01658d0 con 0x7f72b006ae10 2023-10-10T20:41:25.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.126+0000 7f72b5974700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b0169410 con 0x7f72b00c91e0 2023-10-10T20:41:25.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.126+0000 7f72b5974700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b0169620 con 0x7f72b006aa40 2023-10-10T20:41:25.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.126+0000 7f72a67fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72b006ae10 0x7f72b0168ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:25.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.126+0000 7f72a67fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72b006ae10 0x7f72b0168ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49182/0 (socket says 172.21.15.89:49182) 2023-10-10T20:41:25.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.126+0000 7f72a67fc700 1 -- 172.21.15.89:0/312166750 learned_addr learned my addr 172.21.15.89:0/312166750 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:25.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.126+0000 7f72aeffd700 1 --2- 172.21.15.89:0/312166750 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72b006aa40 0x7f72b0150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:25.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.126+0000 7f72af7fe700 1 --2- 172.21.15.89:0/312166750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72b00c91e0 0x7f72b00b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:25.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.127+0000 7f72a67fc700 1 -- 172.21.15.89:0/312166750 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72b006aa40 msgr2=0x7f72b0150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:25.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.127+0000 7f72a67fc700 1 --2- 172.21.15.89:0/312166750 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72b006aa40 0x7f72b0150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:25.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.127+0000 7f72a67fc700 1 -- 172.21.15.89:0/312166750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72b00c91e0 msgr2=0x7f72b00b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:25.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.127+0000 7f72a67fc700 1 --2- 172.21.15.89:0/312166750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72b00c91e0 0x7f72b00b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:25.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.127+0000 7f72a67fc700 1 -- 172.21.15.89:0/312166750 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729800b790 con 0x7f72b006ae10 2023-10-10T20:41:25.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.127+0000 7f72af7fe700 1 --2- 172.21.15.89:0/312166750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72b00c91e0 0x7f72b00b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:25.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.127+0000 7f72a67fc700 1 --2- 172.21.15.89:0/312166750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72b006ae10 0x7f72b0168ed0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f72b006c6a0 tx=0x7f729801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:25.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.127+0000 7f72a77fe700 1 -- 172.21.15.89:0/312166750 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298010540 con 0x7f72b006ae10 2023-10-10T20:41:25.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.127+0000 7f72a77fe700 1 -- 172.21.15.89:0/312166750 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72980106a0 con 0x7f72b006ae10 2023-10-10T20:41:25.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.127+0000 7f72b5974700 1 -- 172.21.15.89:0/312166750 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b0169900 con 0x7f72b006ae10 2023-10-10T20:41:25.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.127+0000 7f72a77fe700 1 -- 172.21.15.89:0/312166750 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298010a60 con 0x7f72b006ae10 2023-10-10T20:41:25.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.127+0000 7f72b5974700 1 -- 172.21.15.89:0/312166750 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b0165ae0 con 0x7f72b006ae10 2023-10-10T20:41:25.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.128+0000 7f72b5974700 1 -- 172.21.15.89:0/312166750 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b0165cd0 con 0x7f72b006ae10 2023-10-10T20:41:25.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.128+0000 7f72a77fe700 1 -- 172.21.15.89:0/312166750 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7298022030 con 0x7f72b006ae10 2023-10-10T20:41:25.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.129+0000 7f72a77fe700 1 --2- 172.21.15.89:0/312166750 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7290067320 0x7f72900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:25.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.129+0000 7f72a77fe700 1 -- 172.21.15.89:0/312166750 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f7298014070 con 0x7f72b006ae10 2023-10-10T20:41:25.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.129+0000 7f72aeffd700 1 --2- 172.21.15.89:0/312166750 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7290067320 0x7f72900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:25.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.129+0000 7f72aeffd700 1 --2- 172.21.15.89:0/312166750 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7290067320 0x7f72900697d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f72a0003890 tx=0x7f72a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:25.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.133+0000 7f72a77fe700 1 -- 172.21.15.89:0/312166750 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7298055aa0 con 0x7f72b006ae10 2023-10-10T20:41:25.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:25.343+0000 7f72b5974700 1 -- 172.21.15.89:0/312166750 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000000990215", "target": ["mon-mgr", ""]}) v1 -- 0x7f72b004c3c0 con 0x7f7290067320 2023-10-10T20:41:28.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.010+0000 7f72a77fe700 1 -- 172.21.15.89:0/312166750 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f72b004c3c0 con 0x7f7290067320 2023-10-10T20:41:28.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.014+0000 7f72b5974700 1 -- 172.21.15.89:0/312166750 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7290067320 msgr2=0x7f72900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:28.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.014+0000 7f72b5974700 1 --2- 172.21.15.89:0/312166750 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7290067320 0x7f72900697d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f72a0003890 tx=0x7f72a0009040 comp rx=0 tx=0).stop 2023-10-10T20:41:28.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.014+0000 7f72b5974700 1 -- 172.21.15.89:0/312166750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72b006ae10 msgr2=0x7f72b0168ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:28.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.014+0000 7f72b5974700 1 --2- 172.21.15.89:0/312166750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72b006ae10 0x7f72b0168ed0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f72b006c6a0 tx=0x7f729801b000 comp rx=0 tx=0).stop 2023-10-10T20:41:28.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.014+0000 7f72b5974700 1 -- 172.21.15.89:0/312166750 shutdown_connections 2023-10-10T20:41:28.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.014+0000 7f72b5974700 1 --2- 172.21.15.89:0/312166750 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7290067320 0x7f72900697d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.014+0000 7f72b5974700 1 --2- 172.21.15.89:0/312166750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72b00c91e0 0x7f72b00b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.014+0000 7f72b5974700 1 --2- 172.21.15.89:0/312166750 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f72b006aa40 0x7f72b0150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.014+0000 7f72b5974700 1 --2- 172.21.15.89:0/312166750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72b006ae10 0x7f72b0168ed0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.014+0000 7f72b5974700 1 -- 172.21.15.89:0/312166750 >> 172.21.15.89:0/312166750 conn(0x7f72b00cd670 msgr2=0x7f72b00c2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:28.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.014+0000 7f72b5974700 1 -- 172.21.15.89:0/312166750 shutdown_connections 2023-10-10T20:41:28.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.014+0000 7f72b5974700 1 -- 172.21.15.89:0/312166750 wait complete. 2023-10-10T20:41:28.019 INFO:teuthology.orchestra.run.smithi089.stderr:Volume created successfully (no MDS daemons created) 2023-10-10T20:41:28.034 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-10T20:41:28.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.153+0000 7f730d71d700 1 Processor -- start 2023-10-10T20:41:28.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.153+0000 7f730d71d700 1 -- start start 2023-10-10T20:41:28.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.154+0000 7f730d71d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7308157670 0x7f73080c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:28.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.154+0000 7f730d71d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7308157a40 0x7f73080cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:28.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.154+0000 7f730d71d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73080ce3d0 con 0x7f73080cae80 2023-10-10T20:41:28.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.154+0000 7f730d71d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73080cd560 con 0x7f7308157a40 2023-10-10T20:41:28.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.154+0000 7f730d71d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73080cd7c0 con 0x7f7308157670 2023-10-10T20:41:28.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.154+0000 7f7306ffd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7308157670 0x7f73080c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:28.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.154+0000 7f73067fc700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7308157a40 0x7f73080cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:28.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.155+0000 7f73077fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f73080cae80 0x7f73080cb2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:34948/0 (socket says 172.21.15.89:34948) 2023-10-10T20:41:28.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.155+0000 7f7306ffd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7308157670 0x7f73080c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:54622/0 (socket says 172.21.15.89:54622) 2023-10-10T20:41:28.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.155+0000 7f73067fc700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7308157a40 0x7f73080cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:43488/0 (socket says 172.21.15.89:43488) 2023-10-10T20:41:28.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.155+0000 7f7306ffd700 1 -- 172.21.15.89:0/2801102567 learned_addr learned my addr 172.21.15.89:0/2801102567 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:28.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.155+0000 7f7306ffd700 1 -- 172.21.15.89:0/2801102567 >> v2:172.21.15.111:3300/0 conn(0x7f7308157a40 msgr2=0x7f73080cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:28.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.155+0000 7f7306ffd700 1 --2- 172.21.15.89:0/2801102567 >> v2:172.21.15.111:3300/0 conn(0x7f7308157a40 0x7f73080cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.155+0000 7f7306ffd700 1 -- 172.21.15.89:0/2801102567 >> v1:172.21.15.111:6789/0 conn(0x7f73080cae80 legacy=0x7f73080cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:28.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.155+0000 7f7306ffd700 1 -- 172.21.15.89:0/2801102567 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73080cdaa0 con 0x7f7308157670 2023-10-10T20:41:28.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.155+0000 7f73067fc700 1 --2- 172.21.15.89:0/2801102567 >> v2:172.21.15.111:3300/0 conn(0x7f7308157a40 0x7f73080cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:41:28.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.155+0000 7f7306ffd700 1 --2- 172.21.15.89:0/2801102567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7308157670 0x7f73080c95d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009f70 tx=0x7f72f8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e96badfd932cc558 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:28.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.155+0000 7f7305ffb700 1 -- 172.21.15.89:0/2801102567 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f800e070 con 0x7f7308157670 2023-10-10T20:41:28.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.155+0000 7f7305ffb700 1 -- 172.21.15.89:0/2801102567 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f800ba20 con 0x7f7308157670 2023-10-10T20:41:28.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.155+0000 7f7305ffb700 1 -- 172.21.15.89:0/2801102567 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f80092e0 con 0x7f7308157670 2023-10-10T20:41:28.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.156+0000 7f730d71d700 1 -- 172.21.15.89:0/2801102567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7308157670 msgr2=0x7f73080c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:28.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.156+0000 7f730d71d700 1 --2- 172.21.15.89:0/2801102567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7308157670 0x7f73080c95d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009f70 tx=0x7f72f8009840 comp rx=0 tx=0).stop 2023-10-10T20:41:28.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.156+0000 7f730d71d700 1 -- 172.21.15.89:0/2801102567 shutdown_connections 2023-10-10T20:41:28.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.156+0000 7f730d71d700 1 --2- 172.21.15.89:0/2801102567 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7308157670 0x7f73080c95d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.156+0000 7f730d71d700 1 --2- 172.21.15.89:0/2801102567 >> v2:172.21.15.111:3300/0 conn(0x7f7308157a40 0x7f73080cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.156+0000 7f730d71d700 1 -- 172.21.15.89:0/2801102567 >> 172.21.15.89:0/2801102567 conn(0x7f730806baa0 msgr2=0x7f730806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:28.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.156+0000 7f730d71d700 1 -- 172.21.15.89:0/2801102567 shutdown_connections 2023-10-10T20:41:28.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.156+0000 7f730d71d700 1 -- 172.21.15.89:0/2801102567 wait complete. 2023-10-10T20:41:28.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.157+0000 7f730d71d700 1 Processor -- start 2023-10-10T20:41:28.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.157+0000 7f730d71d700 1 -- start start 2023-10-10T20:41:28.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f730d71d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73080cae80 0x7f73081520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:28.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f730d71d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308157670 0x7f730815c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:28.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f730d71d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7308157a40 0x7f73080694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:28.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f730d71d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730816a6f0 con 0x7f7308157670 2023-10-10T20:41:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f730d71d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730815bfa0 con 0x7f7308157a40 2023-10-10T20:41:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f730d71d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730815b3d0 con 0x7f73080cae80 2023-10-10T20:41:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f73067fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308157670 0x7f730815c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f73067fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308157670 0x7f730815c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52278/0 (socket says 172.21.15.89:52278) 2023-10-10T20:41:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f73067fc700 1 -- 172.21.15.89:0/800052947 learned_addr learned my addr 172.21.15.89:0/800052947 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f7306ffd700 1 --2- 172.21.15.89:0/800052947 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73080cae80 0x7f73081520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:28.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f73077fe700 1 --2- 172.21.15.89:0/800052947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7308157a40 0x7f73080694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:28.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f73067fc700 1 -- 172.21.15.89:0/800052947 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73080cae80 msgr2=0x7f73081520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:28.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f73067fc700 1 --2- 172.21.15.89:0/800052947 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73080cae80 0x7f73081520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f73067fc700 1 -- 172.21.15.89:0/800052947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7308157a40 msgr2=0x7f73080694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f73067fc700 1 --2- 172.21.15.89:0/800052947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7308157a40 0x7f73080694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f73067fc700 1 -- 172.21.15.89:0/800052947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72fc009420 con 0x7f7308157670 2023-10-10T20:41:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f7306ffd700 1 --2- 172.21.15.89:0/800052947 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73080cae80 0x7f73081520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:41:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f73067fc700 1 --2- 172.21.15.89:0/800052947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308157670 0x7f730815c430 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00ec80 tx=0x7f72fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f72eeffd700 1 -- 172.21.15.89:0/800052947 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc00c850 con 0x7f7308157670 2023-10-10T20:41:28.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.158+0000 7f72eeffd700 1 -- 172.21.15.89:0/800052947 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72fc00c9b0 con 0x7f7308157670 2023-10-10T20:41:28.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.159+0000 7f72eeffd700 1 -- 172.21.15.89:0/800052947 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc00cd30 con 0x7f7308157670 2023-10-10T20:41:28.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.159+0000 7f730d71d700 1 -- 172.21.15.89:0/800052947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f8009ae0 con 0x7f7308157670 2023-10-10T20:41:28.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.159+0000 7f730d71d700 1 -- 172.21.15.89:0/800052947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730815b950 con 0x7f7308157670 2023-10-10T20:41:28.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.159+0000 7f730d71d700 1 -- 172.21.15.89:0/800052947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730815c110 con 0x7f7308157670 2023-10-10T20:41:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.160+0000 7f72eeffd700 1 -- 172.21.15.89:0/800052947 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f72fc013680 con 0x7f7308157670 2023-10-10T20:41:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.160+0000 7f72eeffd700 1 --2- 172.21.15.89:0/800052947 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f72f4067320 0x7f72f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.160+0000 7f72eeffd700 1 -- 172.21.15.89:0/800052947 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 7501+0+0 (secure 0 0 0) 0x7f72fc0855b0 con 0x7f7308157670 2023-10-10T20:41:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.160+0000 7f7306ffd700 1 --2- 172.21.15.89:0/800052947 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f72f4067320 0x7f72f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.161+0000 7f7306ffd700 1 --2- 172.21.15.89:0/800052947 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f72f4067320 0x7f72f40697d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f7308156990 tx=0x7f72f8018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.165+0000 7f72eeffd700 1 -- 172.21.15.89:0/800052947 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72fc0555d0 con 0x7f7308157670 2023-10-10T20:41:28.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.374+0000 7f730d71d700 1 -- 172.21.15.89:0/800052947 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f730804c460 con 0x7f72f4067320 2023-10-10T20:41:28.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.375+0000 7f72eeffd700 1 -- 172.21.15.89:0/800052947 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+151 (secure 0 0 0) 0x7f730804c460 con 0x7f72f4067320 2023-10-10T20:41:28.377 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:41:28.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:28.378 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs" 2023-10-10T20:41:28.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:28.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:28.379 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "volume_0000000000990214" 2023-10-10T20:41:28.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:28.379 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:28.379 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "volume_0000000000990215" 2023-10-10T20:41:28.379 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:28.380 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:41:28.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.377+0000 7f730d71d700 1 -- 172.21.15.89:0/800052947 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f72f4067320 msgr2=0x7f72f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:28.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.377+0000 7f730d71d700 1 --2- 172.21.15.89:0/800052947 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f72f4067320 0x7f72f40697d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f7308156990 tx=0x7f72f8018000 comp rx=0 tx=0).stop 2023-10-10T20:41:28.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.377+0000 7f730d71d700 1 -- 172.21.15.89:0/800052947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308157670 msgr2=0x7f730815c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:28.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.377+0000 7f730d71d700 1 --2- 172.21.15.89:0/800052947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308157670 0x7f730815c430 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00ec80 tx=0x7f72fc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.377+0000 7f730d71d700 1 -- 172.21.15.89:0/800052947 shutdown_connections 2023-10-10T20:41:28.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.377+0000 7f730d71d700 1 --2- 172.21.15.89:0/800052947 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f72f4067320 0x7f72f40697d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.377+0000 7f730d71d700 1 --2- 172.21.15.89:0/800052947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7308157a40 0x7f73080694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.377+0000 7f730d71d700 1 --2- 172.21.15.89:0/800052947 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73080cae80 0x7f73081520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.377+0000 7f730d71d700 1 --2- 172.21.15.89:0/800052947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308157670 0x7f730815c430 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.377+0000 7f730d71d700 1 -- 172.21.15.89:0/800052947 >> 172.21.15.89:0/800052947 conn(0x7f730806baa0 msgr2=0x7f73080c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:28.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.377+0000 7f730d71d700 1 -- 172.21.15.89:0/800052947 shutdown_connections 2023-10-10T20:41:28.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.377+0000 7f730d71d700 1 -- 172.21.15.89:0/800052947 wait complete. 2023-10-10T20:41:28.391 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000990214 --yes-i-really-mean-it 2023-10-10T20:41:28.473 INFO:tasks.ceph.mds.c.smithi089.stderr:2023-10-10T20:41:28.471+0000 7f5f4a732700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:41:28.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.507+0000 7ff5ae1d7700 1 Processor -- start 2023-10-10T20:41:28.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.507+0000 7ff5ae1d7700 1 -- start start 2023-10-10T20:41:28.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.508+0000 7ff5ae1d7700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff5a8159890 0x7ff5a815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:28.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.508+0000 7ff5ae1d7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff5a8159c60 0x7ff5a815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:28.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.508+0000 7ff5ae1d7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff5a815db10 0x7ff5a8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:28.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.508+0000 7ff5ae1d7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a815df50 con 0x7ff5a8159c60 2023-10-10T20:41:28.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.508+0000 7ff5ae1d7700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a815a160 con 0x7ff5a8159890 2023-10-10T20:41:28.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.508+0000 7ff5ae1d7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80d13a0 con 0x7ff5a815db10 2023-10-10T20:41:28.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.508+0000 7ff5a6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff5a8159c60 0x7ff5a815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:28.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.508+0000 7ff5a6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff5a8159c60 0x7ff5a815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52294/0 (socket says 172.21.15.89:52294) 2023-10-10T20:41:28.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.508+0000 7ff5a6ffd700 1 -- 172.21.15.89:0/2871621222 learned_addr learned my addr 172.21.15.89:0/2871621222 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:28.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.509+0000 7ff5a7fff700 1 --2- 172.21.15.89:0/2871621222 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff5a815db10 0x7ff5a8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:28.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.509+0000 7ff5a77fe700 1 --2- 172.21.15.89:0/2871621222 >> v2:172.21.15.111:3300/0 conn(0x7ff5a8159890 0x7ff5a815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:28.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.509+0000 7ff5a6ffd700 1 -- 172.21.15.89:0/2871621222 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff5a815db10 msgr2=0x7ff5a8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:28.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.509+0000 7ff5a6ffd700 1 --2- 172.21.15.89:0/2871621222 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff5a815db10 0x7ff5a8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.509+0000 7ff5a6ffd700 1 -- 172.21.15.89:0/2871621222 >> v2:172.21.15.111:3300/0 conn(0x7ff5a8159890 msgr2=0x7ff5a815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:28.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.509+0000 7ff5a6ffd700 1 --2- 172.21.15.89:0/2871621222 >> v2:172.21.15.111:3300/0 conn(0x7ff5a8159890 0x7ff5a815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.509+0000 7ff5a6ffd700 1 -- 172.21.15.89:0/2871621222 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a815e140 con 0x7ff5a8159c60 2023-10-10T20:41:28.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.509+0000 7ff5a6ffd700 1 --2- 172.21.15.89:0/2871621222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5a8159c60 0x7ff5a815d5d0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80d1090 tx=0x7ff59c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f2a868425544ac server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:28.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.509+0000 7ff5a67fc700 1 -- 172.21.15.89:0/2871621222 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c00b860 con 0x7ff5a8159c60 2023-10-10T20:41:28.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.509+0000 7ff5a67fc700 1 -- 172.21.15.89:0/2871621222 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c00b9c0 con 0x7ff5a8159c60 2023-10-10T20:41:28.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.509+0000 7ff5a67fc700 1 -- 172.21.15.89:0/2871621222 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c00bd60 con 0x7ff5a8159c60 2023-10-10T20:41:28.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.509+0000 7ff5ae1d7700 1 -- 172.21.15.89:0/2871621222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5a8159c60 msgr2=0x7ff5a815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:28.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.509+0000 7ff5ae1d7700 1 --2- 172.21.15.89:0/2871621222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5a8159c60 0x7ff5a815d5d0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80d1090 tx=0x7ff59c009250 comp rx=0 tx=0).stop 2023-10-10T20:41:28.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.510+0000 7ff5ae1d7700 1 -- 172.21.15.89:0/2871621222 shutdown_connections 2023-10-10T20:41:28.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.510+0000 7ff5ae1d7700 1 --2- 172.21.15.89:0/2871621222 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff5a815db10 0x7ff5a8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.510+0000 7ff5ae1d7700 1 --2- 172.21.15.89:0/2871621222 >> v2:172.21.15.111:3300/0 conn(0x7ff5a8159890 0x7ff5a815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.510+0000 7ff5ae1d7700 1 --2- 172.21.15.89:0/2871621222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5a8159c60 0x7ff5a815d5d0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.510+0000 7ff5ae1d7700 1 -- 172.21.15.89:0/2871621222 >> 172.21.15.89:0/2871621222 conn(0x7ff5a8068e20 msgr2=0x7ff5a80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:28.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.510+0000 7ff5ae1d7700 1 -- 172.21.15.89:0/2871621222 shutdown_connections 2023-10-10T20:41:28.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.510+0000 7ff5ae1d7700 1 -- 172.21.15.89:0/2871621222 wait complete. 2023-10-10T20:41:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.511+0000 7ff5ae1d7700 1 Processor -- start 2023-10-10T20:41:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.511+0000 7ff5ae1d7700 1 -- start start 2023-10-10T20:41:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.511+0000 7ff5ae1d7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5a8159890 0x7ff5a8147260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.511+0000 7ff5ae1d7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff5a8159c60 0x7ff5a80ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.511+0000 7ff5ae1d7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5a815db10 0x7ff5a80b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.511+0000 7ff5ae1d7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a8152710 con 0x7ff5a8159890 2023-10-10T20:41:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.511+0000 7ff5ae1d7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80b45a0 con 0x7ff5a815db10 2023-10-10T20:41:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.511+0000 7ff5ae1d7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a81477a0 con 0x7ff5a8159c60 2023-10-10T20:41:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.512+0000 7ff5a77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5a8159890 0x7ff5a8147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.512+0000 7ff5a77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5a8159890 0x7ff5a8147260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52304/0 (socket says 172.21.15.89:52304) 2023-10-10T20:41:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.512+0000 7ff5a77fe700 1 -- 172.21.15.89:0/2871884020 learned_addr learned my addr 172.21.15.89:0/2871884020 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.512+0000 7ff5a7fff700 1 --2- 172.21.15.89:0/2871884020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5a815db10 0x7ff5a80b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.512+0000 7ff5a6ffd700 1 --2- 172.21.15.89:0/2871884020 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff5a8159c60 0x7ff5a80ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.512+0000 7ff5a77fe700 1 -- 172.21.15.89:0/2871884020 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff5a8159c60 msgr2=0x7ff5a80ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:28.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.512+0000 7ff5a77fe700 1 --2- 172.21.15.89:0/2871884020 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff5a8159c60 0x7ff5a80ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.512+0000 7ff5a77fe700 1 -- 172.21.15.89:0/2871884020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5a815db10 msgr2=0x7ff5a80b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:28.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.512+0000 7ff5a77fe700 1 --2- 172.21.15.89:0/2871884020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5a815db10 0x7ff5a80b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:28.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.512+0000 7ff5a77fe700 1 -- 172.21.15.89:0/2871884020 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c0094f0 con 0x7ff5a8159890 2023-10-10T20:41:28.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.512+0000 7ff5a77fe700 1 --2- 172.21.15.89:0/2871884020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5a8159890 0x7ff5a8147260 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff598009fa0 tx=0x7ff598009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:28.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.512+0000 7ff58f7fe700 1 -- 172.21.15.89:0/2871884020 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59800ebc0 con 0x7ff5a8159890 2023-10-10T20:41:28.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.512+0000 7ff58f7fe700 1 -- 172.21.15.89:0/2871884020 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59800ed20 con 0x7ff5a8159890 2023-10-10T20:41:28.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.512+0000 7ff5ae1d7700 1 -- 172.21.15.89:0/2871884020 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a80ca6b0 con 0x7ff5a8159890 2023-10-10T20:41:28.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.512+0000 7ff58f7fe700 1 -- 172.21.15.89:0/2871884020 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598014670 con 0x7ff5a8159890 2023-10-10T20:41:28.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.512+0000 7ff5ae1d7700 1 -- 172.21.15.89:0/2871884020 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a80b47f0 con 0x7ff5a8159890 2023-10-10T20:41:28.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.513+0000 7ff5ae1d7700 1 -- 172.21.15.89:0/2871884020 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a804c3c0 con 0x7ff5a8159890 2023-10-10T20:41:28.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.513+0000 7ff58f7fe700 1 -- 172.21.15.89:0/2871884020 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff5980149e0 con 0x7ff5a8159890 2023-10-10T20:41:28.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.514+0000 7ff58f7fe700 1 --2- 172.21.15.89:0/2871884020 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff594067320 0x7ff5940697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:28.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.514+0000 7ff58f7fe700 1 -- 172.21.15.89:0/2871884020 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 7501+0+0 (secure 0 0 0) 0x7ff5980859b0 con 0x7ff5a8159890 2023-10-10T20:41:28.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.514+0000 7ff5a6ffd700 1 --2- 172.21.15.89:0/2871884020 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff594067320 0x7ff5940697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:28.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.514+0000 7ff5a6ffd700 1 --2- 172.21.15.89:0/2871884020 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff594067320 0x7ff5940697d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80d1090 tx=0x7ff59c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:28.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.518+0000 7ff58f7fe700 1 -- 172.21.15.89:0/2871884020 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff598051f40 con 0x7ff5a8159890 2023-10-10T20:41:28.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:28.727+0000 7ff5ae1d7700 1 -- 172.21.15.89:0/2871884020 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000000990214", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5a806a5b0 con 0x7ff594067320 2023-10-10T20:41:28.732 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-10-10T20:41:28.731+0000 7fd0184ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:41:28.736 INFO:tasks.ceph.mds.c.smithi089.stderr: -10> 2023-10-10T20:41:28.471+0000 7f5f4a732700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:41:28.761 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-10T20:41:31.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.755+0000 7ff58f7fe700 1 -- 172.21.15.89:0/2871884020 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+110 (secure 0 0 0) 0x7ff5a806a5b0 con 0x7ff594067320 2023-10-10T20:41:31.758 INFO:teuthology.orchestra.run.smithi089.stdout:metadata pool: cephfs.volume_0000000000990214.meta data pool: ['cephfs.volume_0000000000990214.data'] removed 2023-10-10T20:41:31.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.758+0000 7ff5ae1d7700 1 -- 172.21.15.89:0/2871884020 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff594067320 msgr2=0x7ff5940697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:31.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.758+0000 7ff5ae1d7700 1 --2- 172.21.15.89:0/2871884020 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff594067320 0x7ff5940697d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80d1090 tx=0x7ff59c017000 comp rx=0 tx=0).stop 2023-10-10T20:41:31.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.758+0000 7ff5ae1d7700 1 -- 172.21.15.89:0/2871884020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5a8159890 msgr2=0x7ff5a8147260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:31.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.758+0000 7ff5ae1d7700 1 --2- 172.21.15.89:0/2871884020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5a8159890 0x7ff5a8147260 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff598009fa0 tx=0x7ff598009880 comp rx=0 tx=0).stop 2023-10-10T20:41:31.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.758+0000 7ff5ae1d7700 1 -- 172.21.15.89:0/2871884020 shutdown_connections 2023-10-10T20:41:31.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.758+0000 7ff5ae1d7700 1 --2- 172.21.15.89:0/2871884020 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff594067320 0x7ff5940697d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:31.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.758+0000 7ff5ae1d7700 1 --2- 172.21.15.89:0/2871884020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5a815db10 0x7ff5a80b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:31.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.758+0000 7ff5ae1d7700 1 --2- 172.21.15.89:0/2871884020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5a8159890 0x7ff5a8147260 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:31.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.758+0000 7ff5ae1d7700 1 --2- 172.21.15.89:0/2871884020 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff5a8159c60 0x7ff5a80ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:31.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.758+0000 7ff5ae1d7700 1 -- 172.21.15.89:0/2871884020 >> 172.21.15.89:0/2871884020 conn(0x7ff5a8068e20 msgr2=0x7ff5a8153060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:31.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.759+0000 7ff5ae1d7700 1 -- 172.21.15.89:0/2871884020 shutdown_connections 2023-10-10T20:41:31.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.759+0000 7ff5ae1d7700 1 -- 172.21.15.89:0/2871884020 wait complete. 2023-10-10T20:41:31.777 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000990215 --yes-i-really-mean-it 2023-10-10T20:41:31.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.897+0000 7f436a8ed700 1 Processor -- start 2023-10-10T20:41:31.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.897+0000 7f436a8ed700 1 -- start start 2023-10-10T20:41:31.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.898+0000 7f436a8ed700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4364159c60 0x7f436415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:31.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.898+0000 7f436a8ed700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f436415a160 con 0x7f436415db10 2023-10-10T20:41:31.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.898+0000 7f436a8ed700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43640d13a0 con 0x7f4364159890 2023-10-10T20:41:31.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.898+0000 7f436a8ed700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436415e140 con 0x7f4364159c60 2023-10-10T20:41:31.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.898+0000 7f4368e8a700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f436415db10 0x7f4364069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44806/0 (socket says 172.21.15.89:44806) 2023-10-10T20:41:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.898+0000 7f4368e8a700 1 -- 172.21.15.89:0/423480849 learned_addr learned my addr 172.21.15.89:0/423480849 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.898+0000 7f4362ffd700 1 -- 172.21.15.89:0/423480849 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418110962 0 0) 0x7f436415a160 con 0x7f436415db10 2023-10-10T20:41:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.898+0000 7f43637fe700 1 --2- 172.21.15.89:0/423480849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4364159c60 0x7f436415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.898+0000 7f4362ffd700 1 -- 172.21.15.89:0/423480849 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f434c003570 con 0x7f436415db10 2023-10-10T20:41:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4362ffd700 1 -- 172.21.15.89:0/423480849 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1997791208 0 0) 0x7f434c003570 con 0x7f436415db10 2023-10-10T20:41:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4362ffd700 1 -- 172.21.15.89:0/423480849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4364159c60 msgr2=0x7f436415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4362ffd700 1 --2- 172.21.15.89:0/423480849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4364159c60 0x7f436415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4362ffd700 1 -- 172.21.15.89:0/423480849 >> v1:172.21.15.111:6789/0 conn(0x7f4364159890 legacy=0x7f436415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4362ffd700 1 -- 172.21.15.89:0/423480849 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436415a3d0 con 0x7f436415db10 2023-10-10T20:41:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4362ffd700 1 -- 172.21.15.89:0/423480849 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f435c0017f0 con 0x7f436415db10 2023-10-10T20:41:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4362ffd700 1 -- 172.21.15.89:0/423480849 >> v1:172.21.15.89:6789/0 conn(0x7f436415db10 legacy=0x7f4364069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4362ffd700 1 --2- 172.21.15.89:0/423480849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f434c003a90 0x7f434c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4362ffd700 1 --2- 172.21.15.89:0/423480849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4364159c60 0x7f434c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4362ffd700 1 --2- 172.21.15.89:0/423480849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f434c004d50 0x7f434c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4362ffd700 1 -- 172.21.15.89:0/423480849 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436415e140 con 0x7f434c003a90 2023-10-10T20:41:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4362ffd700 1 -- 172.21.15.89:0/423480849 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c00bb70 con 0x7f434c004d50 2023-10-10T20:41:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4362ffd700 1 -- 172.21.15.89:0/423480849 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434c00bd30 con 0x7f4364159c60 2023-10-10T20:41:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4363fff700 1 --2- 172.21.15.89:0/423480849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f434c003a90 0x7f434c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:31.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f43637fe700 1 --2- 172.21.15.89:0/423480849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4364159c60 0x7f434c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:31.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4368e8a700 1 --2- 172.21.15.89:0/423480849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f434c004d50 0x7f434c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:31.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4363fff700 1 -- 172.21.15.89:0/423480849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4364159c60 msgr2=0x7f434c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:31.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4363fff700 1 --2- 172.21.15.89:0/423480849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4364159c60 0x7f434c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:31.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4363fff700 1 -- 172.21.15.89:0/423480849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f434c004d50 msgr2=0x7f434c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:31.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4363fff700 1 --2- 172.21.15.89:0/423480849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f434c004d50 0x7f434c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:31.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4363fff700 1 -- 172.21.15.89:0/423480849 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434c003570 con 0x7f434c003a90 2023-10-10T20:41:31.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4368e8a700 1 --2- 172.21.15.89:0/423480849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f434c004d50 0x7f434c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:31.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4363fff700 1 --2- 172.21.15.89:0/423480849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f434c003a90 0x7f434c003ea0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f43540018b0 tx=0x7f4354007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441c14c68ec0e157 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:31.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4362ffd700 1 -- 172.21.15.89:0/423480849 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354012420 con 0x7f434c003a90 2023-10-10T20:41:31.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4362ffd700 1 -- 172.21.15.89:0/423480849 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4354012580 con 0x7f434c003a90 2023-10-10T20:41:31.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.899+0000 7f4362ffd700 1 -- 172.21.15.89:0/423480849 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354012920 con 0x7f434c003a90 2023-10-10T20:41:31.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.900+0000 7f436a8ed700 1 -- 172.21.15.89:0/423480849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f434c003a90 msgr2=0x7f434c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:31.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.900+0000 7f436a8ed700 1 --2- 172.21.15.89:0/423480849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f434c003a90 0x7f434c003ea0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f43540018b0 tx=0x7f4354007d30 comp rx=0 tx=0).stop 2023-10-10T20:41:31.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.900+0000 7f4368e8a700 1 -- 172.21.15.89:0/423480849 reap_dead start 2023-10-10T20:41:31.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.900+0000 7f436a8ed700 1 -- 172.21.15.89:0/423480849 shutdown_connections 2023-10-10T20:41:31.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.900+0000 7f436a8ed700 1 -- 172.21.15.89:0/423480849 >> 172.21.15.89:0/423480849 conn(0x7f4364068e20 msgr2=0x7f43640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:31.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.900+0000 7f436a8ed700 1 -- 172.21.15.89:0/423480849 shutdown_connections 2023-10-10T20:41:31.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.900+0000 7f436a8ed700 1 -- 172.21.15.89:0/423480849 wait complete. 2023-10-10T20:41:31.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.901+0000 7f436a8ed700 1 Processor -- start 2023-10-10T20:41:31.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.901+0000 7f436a8ed700 1 -- start start 2023-10-10T20:41:31.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.902+0000 7f436a8ed700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f434c003a90 0x7f43641480f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:31.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.902+0000 7f436a8ed700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43640c78a0 0x7f43640ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:31.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.902+0000 7f436a8ed700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4364155120 0x7f43640cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:31.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.902+0000 7f436a8ed700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43640b4b20 con 0x7f4364155120 2023-10-10T20:41:31.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.902+0000 7f436a8ed700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43640cab90 con 0x7f43640c78a0 2023-10-10T20:41:31.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.902+0000 7f436a8ed700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4364169d60 con 0x7f434c003a90 2023-10-10T20:41:31.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.902+0000 7f4368e8a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4364155120 0x7f43640cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:31.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.902+0000 7f4368e8a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4364155120 0x7f43640cf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52332/0 (socket says 172.21.15.89:52332) 2023-10-10T20:41:31.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.902+0000 7f4368e8a700 1 -- 172.21.15.89:0/3283936355 learned_addr learned my addr 172.21.15.89:0/3283936355 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:31.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.902+0000 7f43637fe700 1 --2- 172.21.15.89:0/3283936355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43640c78a0 0x7f43640ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:31.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.902+0000 7f4363fff700 1 --2- 172.21.15.89:0/3283936355 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f434c003a90 0x7f43641480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:31.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.902+0000 7f4368e8a700 1 -- 172.21.15.89:0/3283936355 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f434c003a90 msgr2=0x7f43641480f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:31.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.902+0000 7f4368e8a700 1 --2- 172.21.15.89:0/3283936355 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f434c003a90 0x7f43641480f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:31.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.902+0000 7f4368e8a700 1 -- 172.21.15.89:0/3283936355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43640c78a0 msgr2=0x7f43640ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:31.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.902+0000 7f4368e8a700 1 --2- 172.21.15.89:0/3283936355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43640c78a0 0x7f43640ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:31.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.902+0000 7f4368e8a700 1 -- 172.21.15.89:0/3283936355 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436416a040 con 0x7f4364155120 2023-10-10T20:41:31.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.902+0000 7f4363fff700 1 --2- 172.21.15.89:0/3283936355 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f434c003a90 0x7f43641480f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:31.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.902+0000 7f4368e8a700 1 --2- 172.21.15.89:0/3283936355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4364155120 0x7f43640cf0c0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f435c002250 tx=0x7f435c002110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:31.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.903+0000 7f434bfff700 1 -- 172.21.15.89:0/3283936355 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c00ebc0 con 0x7f4364155120 2023-10-10T20:41:31.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.903+0000 7f434bfff700 1 -- 172.21.15.89:0/3283936355 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435c00ed20 con 0x7f4364155120 2023-10-10T20:41:31.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.903+0000 7f436a8ed700 1 -- 172.21.15.89:0/3283936355 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436416a320 con 0x7f4364155120 2023-10-10T20:41:31.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.903+0000 7f434bfff700 1 -- 172.21.15.89:0/3283936355 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c013630 con 0x7f4364155120 2023-10-10T20:41:31.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.903+0000 7f436a8ed700 1 -- 172.21.15.89:0/3283936355 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436416ab20 con 0x7f4364155120 2023-10-10T20:41:31.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.904+0000 7f436a8ed700 1 -- 172.21.15.89:0/3283936355 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436404c3c0 con 0x7f4364155120 2023-10-10T20:41:31.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.904+0000 7f434bfff700 1 -- 172.21.15.89:0/3283936355 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f435c0139a0 con 0x7f4364155120 2023-10-10T20:41:31.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.904+0000 7f434bfff700 1 --2- 172.21.15.89:0/3283936355 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4350067320 0x7f43500697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:31.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.904+0000 7f434bfff700 1 -- 172.21.15.89:0/3283936355 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6689+0+0 (secure 0 0 0) 0x7f435c085600 con 0x7f4364155120 2023-10-10T20:41:31.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.905+0000 7f4363fff700 1 --2- 172.21.15.89:0/3283936355 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4350067320 0x7f43500697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:31.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.905+0000 7f4363fff700 1 --2- 172.21.15.89:0/3283936355 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4350067320 0x7f43500697d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f43540018b0 tx=0x7f4354015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:31.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:31.909+0000 7f434bfff700 1 -- 172.21.15.89:0/3283936355 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f435c051eb0 con 0x7f4364155120 2023-10-10T20:41:32.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:32.128+0000 7f436a8ed700 1 -- 172.21.15.89:0/3283936355 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000000990215", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f4364169ed0 con 0x7f4350067320 2023-10-10T20:41:32.134 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-10-10T20:41:32.131+0000 7fd0184ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:41:32.158 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:41:35.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.145+0000 7f434bfff700 1 -- 172.21.15.89:0/3283936355 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+110 (secure 0 0 0) 0x7f4364169ed0 con 0x7f4350067320 2023-10-10T20:41:35.148 INFO:teuthology.orchestra.run.smithi089.stdout:metadata pool: cephfs.volume_0000000000990215.meta data pool: ['cephfs.volume_0000000000990215.data'] removed 2023-10-10T20:41:35.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.148+0000 7f436a8ed700 1 -- 172.21.15.89:0/3283936355 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4350067320 msgr2=0x7f43500697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:35.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.148+0000 7f436a8ed700 1 --2- 172.21.15.89:0/3283936355 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4350067320 0x7f43500697d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f43540018b0 tx=0x7f4354015000 comp rx=0 tx=0).stop 2023-10-10T20:41:35.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.148+0000 7f436a8ed700 1 -- 172.21.15.89:0/3283936355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4364155120 msgr2=0x7f43640cf0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:35.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.148+0000 7f436a8ed700 1 --2- 172.21.15.89:0/3283936355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4364155120 0x7f43640cf0c0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f435c002250 tx=0x7f435c002110 comp rx=0 tx=0).stop 2023-10-10T20:41:35.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.149+0000 7f436a8ed700 1 -- 172.21.15.89:0/3283936355 shutdown_connections 2023-10-10T20:41:35.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.149+0000 7f436a8ed700 1 --2- 172.21.15.89:0/3283936355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4364155120 0x7f43640cf0c0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:35.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.149+0000 7f436a8ed700 1 --2- 172.21.15.89:0/3283936355 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f434c003a90 0x7f43641480f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.149+0000 7f436a8ed700 1 --2- 172.21.15.89:0/3283936355 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4350067320 0x7f43500697d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.149+0000 7f436a8ed700 1 --2- 172.21.15.89:0/3283936355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43640c78a0 0x7f43640ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.149+0000 7f436a8ed700 1 -- 172.21.15.89:0/3283936355 >> 172.21.15.89:0/3283936355 conn(0x7f4364068e20 msgr2=0x7f43640d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.149+0000 7f436a8ed700 1 -- 172.21.15.89:0/3283936355 shutdown_connections 2023-10-10T20:41:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.149+0000 7f436a8ed700 1 -- 172.21.15.89:0/3283936355 wait complete. 2023-10-10T20:41:35.167 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:41:35.167 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:41:35.212 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:41:35.263 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:41:35.263 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:41:35.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:41:35.264 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:41:35.306 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:41:35.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:41:35.307 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:41:35.362 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:41:35.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:41:35.363 DEBUG:teuthology.orchestra.run.smithi111:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:41:35.404 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:41:35.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:41:35.405 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:41:35.462 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:41:35.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.577+0000 7f19d8cc0700 1 Processor -- start 2023-10-10T20:41:35.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.577+0000 7f19d8cc0700 1 -- start start 2023-10-10T20:41:35.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.578+0000 7f19d8cc0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19d4141500 0x7f19d40c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:35.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.578+0000 7f19d8cc0700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f19d406c0a0 0x7f19d40c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:35.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.578+0000 7f19d8cc0700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d415c2a0 con 0x7f19d406bcd0 2023-10-10T20:41:35.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.578+0000 7f19d8cc0700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d40c9880 con 0x7f19d406c0a0 2023-10-10T20:41:35.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.578+0000 7f19d8cc0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d415b690 con 0x7f19d4141500 2023-10-10T20:41:35.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.578+0000 7f19c9d9b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19d4141500 0x7f19d40c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:35.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.578+0000 7f19d2d9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f19d406c0a0 0x7f19d40c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:35.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.578+0000 7f19c9d9b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19d4141500 0x7f19d40c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:54710/0 (socket says 172.21.15.89:54710) 2023-10-10T20:41:35.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.579+0000 7f19c9d9b700 1 -- 172.21.15.89:0/2871164065 learned_addr learned my addr 172.21.15.89:0/2871164065 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:35.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.578+0000 7f19d259c700 1 --1- 172.21.15.89:0/2871164065 >> v1:172.21.15.111:6789/0 conn(0x7f19d406bcd0 0x7f19d4140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:34972/0 (socket says 172.21.15.89:34972) 2023-10-10T20:41:35.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.579+0000 7f19c9d9b700 1 -- 172.21.15.89:0/2871164065 >> v2:172.21.15.111:3300/0 conn(0x7f19d406c0a0 msgr2=0x7f19d40c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:35.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.579+0000 7f19c9d9b700 1 --2- 172.21.15.89:0/2871164065 >> v2:172.21.15.111:3300/0 conn(0x7f19d406c0a0 0x7f19d40c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:35.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.579+0000 7f19c9d9b700 1 -- 172.21.15.89:0/2871164065 >> v1:172.21.15.111:6789/0 conn(0x7f19d406bcd0 legacy=0x7f19d4140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:35.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.579+0000 7f19c9d9b700 1 -- 172.21.15.89:0/2871164065 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d415b970 con 0x7f19d4141500 2023-10-10T20:41:35.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.579+0000 7f19d2d9d700 1 --2- 172.21.15.89:0/2871164065 >> v2:172.21.15.111:3300/0 conn(0x7f19d406c0a0 0x7f19d40c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:41:35.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.579+0000 7f19d1d9b700 1 -- 172.21.15.89:0/2871164065 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115580542 0 0) 0x7f19d415c2a0 con 0x7f19d406bcd0 2023-10-10T20:41:35.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.579+0000 7f19c9d9b700 1 --2- 172.21.15.89:0/2871164065 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19d4141500 0x7f19d40c8e00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0099b0 tx=0x7f19bc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=275e84a83f34fc3e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:35.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.579+0000 7f19d1d9b700 1 -- 172.21.15.89:0/2871164065 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc009ea0 con 0x7f19d4141500 2023-10-10T20:41:35.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.579+0000 7f19d1d9b700 1 -- 172.21.15.89:0/2871164065 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19bc00b860 con 0x7f19d4141500 2023-10-10T20:41:35.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.579+0000 7f19d1d9b700 1 -- 172.21.15.89:0/2871164065 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc00bc00 con 0x7f19d4141500 2023-10-10T20:41:35.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.580+0000 7f19d8cc0700 1 -- 172.21.15.89:0/2871164065 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19d4141500 msgr2=0x7f19d40c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:35.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.580+0000 7f19d8cc0700 1 --2- 172.21.15.89:0/2871164065 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19d4141500 0x7f19d40c8e00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0099b0 tx=0x7f19bc009250 comp rx=0 tx=0).stop 2023-10-10T20:41:35.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.580+0000 7f19d8cc0700 1 -- 172.21.15.89:0/2871164065 shutdown_connections 2023-10-10T20:41:35.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.580+0000 7f19d8cc0700 1 --2- 172.21.15.89:0/2871164065 >> v2:172.21.15.111:3300/0 conn(0x7f19d406c0a0 0x7f19d40c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:35.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.580+0000 7f19d8cc0700 1 --2- 172.21.15.89:0/2871164065 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19d4141500 0x7f19d40c8e00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:35.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.580+0000 7f19d8cc0700 1 -- 172.21.15.89:0/2871164065 >> 172.21.15.89:0/2871164065 conn(0x7f19d40ce600 msgr2=0x7f19d40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:35.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.580+0000 7f19d8cc0700 1 -- 172.21.15.89:0/2871164065 shutdown_connections 2023-10-10T20:41:35.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.580+0000 7f19d8cc0700 1 -- 172.21.15.89:0/2871164065 wait complete. 2023-10-10T20:41:35.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.581+0000 7f19d8cc0700 1 Processor -- start 2023-10-10T20:41:35.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.581+0000 7f19d8cc0700 1 -- start start 2023-10-10T20:41:35.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19d8cc0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19d406bcd0 0x7f19d4156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:35.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19d8cc0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19d406c0a0 0x7f19d413d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:35.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19d8cc0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19d4141500 0x7f19d413d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:35.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19d8cc0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4157300 con 0x7f19d406bcd0 2023-10-10T20:41:35.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19d8cc0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d40b26e0 con 0x7f19d4141500 2023-10-10T20:41:35.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19d8cc0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d40b3dc0 con 0x7f19d406c0a0 2023-10-10T20:41:35.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19d259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19d406bcd0 0x7f19d4156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:35.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19d259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19d406bcd0 0x7f19d4156dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52370/0 (socket says 172.21.15.89:52370) 2023-10-10T20:41:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19d259c700 1 -- 172.21.15.89:0/830442739 learned_addr learned my addr 172.21.15.89:0/830442739 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19c9d9b700 1 --2- 172.21.15.89:0/830442739 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19d406c0a0 0x7f19d413d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19d2d9d700 1 --2- 172.21.15.89:0/830442739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19d4141500 0x7f19d413d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19d259c700 1 -- 172.21.15.89:0/830442739 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19d406c0a0 msgr2=0x7f19d413d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19d259c700 1 --2- 172.21.15.89:0/830442739 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19d406c0a0 0x7f19d413d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:35.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19d259c700 1 -- 172.21.15.89:0/830442739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19d4141500 msgr2=0x7f19d413d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:35.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19d259c700 1 --2- 172.21.15.89:0/830442739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19d4141500 0x7f19d413d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:35.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19d259c700 1 -- 172.21.15.89:0/830442739 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19cc009a20 con 0x7f19d406bcd0 2023-10-10T20:41:35.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19d259c700 1 --2- 172.21.15.89:0/830442739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19d406bcd0 0x7f19d4156dc0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f19c40011e0 tx=0x7f19c4007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:35.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19caffd700 1 -- 172.21.15.89:0/830442739 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c4005a70 con 0x7f19d406bcd0 2023-10-10T20:41:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19caffd700 1 -- 172.21.15.89:0/830442739 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c4005bd0 con 0x7f19d406bcd0 2023-10-10T20:41:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19caffd700 1 -- 172.21.15.89:0/830442739 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c40070d0 con 0x7f19d406bcd0 2023-10-10T20:41:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19d8cc0700 1 -- 172.21.15.89:0/830442739 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19bc0094f0 con 0x7f19d406bcd0 2023-10-10T20:41:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.582+0000 7f19d8cc0700 1 -- 172.21.15.89:0/830442739 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d40b1e60 con 0x7f19d406bcd0 2023-10-10T20:41:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.583+0000 7f19d8cc0700 1 -- 172.21.15.89:0/830442739 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d404c3c0 con 0x7f19d406bcd0 2023-10-10T20:41:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.584+0000 7f19caffd700 1 -- 172.21.15.89:0/830442739 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f19c4007440 con 0x7f19d406bcd0 2023-10-10T20:41:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.584+0000 7f19caffd700 1 --2- 172.21.15.89:0/830442739 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f19b4067320 0x7f19b40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.584+0000 7f19caffd700 1 -- 172.21.15.89:0/830442739 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f19c4084060 con 0x7f19d406bcd0 2023-10-10T20:41:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.584+0000 7f19c9d9b700 1 --2- 172.21.15.89:0/830442739 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f19b4067320 0x7f19b40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.585+0000 7f19c9d9b700 1 --2- 172.21.15.89:0/830442739 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f19b4067320 0x7f19b40697d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0099b0 tx=0x7f19bc017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:35.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.589+0000 7f19caffd700 1 -- 172.21.15.89:0/830442739 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19c404b5a0 con 0x7f19d406bcd0 2023-10-10T20:41:35.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.813+0000 7f19d8cc0700 1 -- 172.21.15.89:0/830442739 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19d415bc50 con 0x7f19d406bcd0 2023-10-10T20:41:35.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.813+0000 7f19caffd700 1 -- 172.21.15.89:0/830442739 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f19c4017070 con 0x7f19d406bcd0 2023-10-10T20:41:35.816 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:35.816 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6517,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/2658508419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2658508419},{"type":"v1","addr":"172.21.15.111:6834","nonce":2658508419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7431,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1872461145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1872461145},{"type":"v1","addr":"172.21.15.89:6837","nonce":1872461145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3243330447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3243330447},{"type":"v1","addr":"172.21.15.89:6835","nonce":3243330447}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:41:07.565995+0000","modified":"2023-10-10T20:41:11.225905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"d","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/2582059153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2582059153},{"type":"v1","addr":"172.21.15.111:6835","nonce":2582059153}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-10T20:41:35.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.815+0000 7f19d8cc0700 1 -- 172.21.15.89:0/830442739 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f19b4067320 msgr2=0x7f19b40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:35.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.815+0000 7f19d8cc0700 1 --2- 172.21.15.89:0/830442739 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f19b4067320 0x7f19b40697d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0099b0 tx=0x7f19bc017000 comp rx=0 tx=0).stop 2023-10-10T20:41:35.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.815+0000 7f19d8cc0700 1 -- 172.21.15.89:0/830442739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19d406bcd0 msgr2=0x7f19d4156dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:35.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.815+0000 7f19d8cc0700 1 --2- 172.21.15.89:0/830442739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19d406bcd0 0x7f19d4156dc0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f19c40011e0 tx=0x7f19c4007e10 comp rx=0 tx=0).stop 2023-10-10T20:41:35.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.815+0000 7f19d8cc0700 1 -- 172.21.15.89:0/830442739 shutdown_connections 2023-10-10T20:41:35.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.815+0000 7f19d8cc0700 1 --2- 172.21.15.89:0/830442739 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f19b4067320 0x7f19b40697d0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:35.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.815+0000 7f19d8cc0700 1 --2- 172.21.15.89:0/830442739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19d4141500 0x7f19d413d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:35.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.815+0000 7f19d8cc0700 1 --2- 172.21.15.89:0/830442739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19d406bcd0 0x7f19d4156dc0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:35.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.815+0000 7f19d8cc0700 1 --2- 172.21.15.89:0/830442739 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19d406c0a0 0x7f19d413d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:35.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.815+0000 7f19d8cc0700 1 -- 172.21.15.89:0/830442739 >> 172.21.15.89:0/830442739 conn(0x7f19d40ce600 msgr2=0x7f19d40c4500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:35.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.815+0000 7f19d8cc0700 1 -- 172.21.15.89:0/830442739 shutdown_connections 2023-10-10T20:41:35.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.815+0000 7f19d8cc0700 1 -- 172.21.15.89:0/830442739 wait complete. 2023-10-10T20:41:35.820 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 83 2023-10-10T20:41:35.829 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:41:35.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.942+0000 7f869ace7700 1 Processor -- start 2023-10-10T20:41:35.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.942+0000 7f869ace7700 1 -- start start 2023-10-10T20:41:35.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.942+0000 7f869ace7700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8694159c60 0x7f869415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:35.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.942+0000 7f869ace7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f869415db10 0x7f8694069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:35.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.942+0000 7f869ace7700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f869415a160 con 0x7f8694159890 2023-10-10T20:41:35.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.942+0000 7f869ace7700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f869415df50 con 0x7f8694159c60 2023-10-10T20:41:35.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.942+0000 7f869ace7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86940d13a0 con 0x7f869415db10 2023-10-10T20:41:35.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.943+0000 7f8699284700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f869415db10 0x7f8694069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:35.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.943+0000 7f868bfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8694159c60 0x7f869415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:35.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.943+0000 7f8699284700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f869415db10 0x7f8694069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:54736/0 (socket says 172.21.15.89:54736) 2023-10-10T20:41:35.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.943+0000 7f8699284700 1 -- 172.21.15.89:0/1253130942 learned_addr learned my addr 172.21.15.89:0/1253130942 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:35.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.943+0000 7f8699284700 1 -- 172.21.15.89:0/1253130942 >> v2:172.21.15.111:3300/0 conn(0x7f8694159c60 msgr2=0x7f869415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:35.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.943+0000 7f8699284700 1 --2- 172.21.15.89:0/1253130942 >> v2:172.21.15.111:3300/0 conn(0x7f8694159c60 0x7f869415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:35.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.943+0000 7f8699284700 1 -- 172.21.15.89:0/1253130942 >> v1:172.21.15.111:6789/0 conn(0x7f8694159890 legacy=0x7f869415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:35.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.943+0000 7f8699284700 1 -- 172.21.15.89:0/1253130942 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869415e140 con 0x7f869415db10 2023-10-10T20:41:35.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.943+0000 7f8699284700 1 --2- 172.21.15.89:0/1253130942 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f869415db10 0x7f8694069560 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f868c0099b0 tx=0x7f868c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea8eabb8cf20cf90 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:35.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.944+0000 7f868b7fe700 1 -- 172.21.15.89:0/1253130942 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c009ea0 con 0x7f869415db10 2023-10-10T20:41:35.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.944+0000 7f868b7fe700 1 -- 172.21.15.89:0/1253130942 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c00b860 con 0x7f869415db10 2023-10-10T20:41:35.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.944+0000 7f868b7fe700 1 -- 172.21.15.89:0/1253130942 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c00bc00 con 0x7f869415db10 2023-10-10T20:41:35.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.944+0000 7f869ace7700 1 -- 172.21.15.89:0/1253130942 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f869415db10 msgr2=0x7f8694069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:35.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.944+0000 7f869ace7700 1 --2- 172.21.15.89:0/1253130942 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f869415db10 0x7f8694069560 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f868c0099b0 tx=0x7f868c009250 comp rx=0 tx=0).stop 2023-10-10T20:41:35.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.944+0000 7f869ace7700 1 -- 172.21.15.89:0/1253130942 shutdown_connections 2023-10-10T20:41:35.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.944+0000 7f869ace7700 1 --2- 172.21.15.89:0/1253130942 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f869415db10 0x7f8694069560 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:35.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.944+0000 7f869ace7700 1 --2- 172.21.15.89:0/1253130942 >> v2:172.21.15.111:3300/0 conn(0x7f8694159c60 0x7f869415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:35.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.944+0000 7f869ace7700 1 -- 172.21.15.89:0/1253130942 >> 172.21.15.89:0/1253130942 conn(0x7f8694068e20 msgr2=0x7f86940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:35.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.944+0000 7f869ace7700 1 -- 172.21.15.89:0/1253130942 shutdown_connections 2023-10-10T20:41:35.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.944+0000 7f869ace7700 1 -- 172.21.15.89:0/1253130942 wait complete. 2023-10-10T20:41:35.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.945+0000 7f869ace7700 1 Processor -- start 2023-10-10T20:41:35.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.945+0000 7f869ace7700 1 -- start start 2023-10-10T20:41:35.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.946+0000 7f869ace7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8694159890 0x7f869414c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:35.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.946+0000 7f869ace7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8694159c60 0x7f8694151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:35.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.946+0000 7f869ace7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f869415db10 0x7f8694154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:35.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.946+0000 7f869ace7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8694151b80 con 0x7f8694159c60 2023-10-10T20:41:35.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.946+0000 7f869ace7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8694151e60 con 0x7f869415db10 2023-10-10T20:41:35.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.946+0000 7f869ace7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8694152140 con 0x7f8694159890 2023-10-10T20:41:35.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.946+0000 7f868bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8694159c60 0x7f8694151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:35.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.946+0000 7f868bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8694159c60 0x7f8694151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52378/0 (socket says 172.21.15.89:52378) 2023-10-10T20:41:35.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.946+0000 7f868bfff700 1 -- 172.21.15.89:0/820398024 learned_addr learned my addr 172.21.15.89:0/820398024 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:35.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.946+0000 7f8698a83700 1 --2- 172.21.15.89:0/820398024 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8694159890 0x7f869414c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:35.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.946+0000 7f8699284700 1 --2- 172.21.15.89:0/820398024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f869415db10 0x7f8694154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:35.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.946+0000 7f868bfff700 1 -- 172.21.15.89:0/820398024 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8694159890 msgr2=0x7f869414c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:35.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.946+0000 7f868bfff700 1 --2- 172.21.15.89:0/820398024 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8694159890 0x7f869414c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:35.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.946+0000 7f868bfff700 1 -- 172.21.15.89:0/820398024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f869415db10 msgr2=0x7f8694154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:35.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.946+0000 7f868bfff700 1 --2- 172.21.15.89:0/820398024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f869415db10 0x7f8694154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:35.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.946+0000 7f868bfff700 1 -- 172.21.15.89:0/820398024 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868c0094f0 con 0x7f8694159c60 2023-10-10T20:41:35.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.946+0000 7f8699284700 1 --2- 172.21.15.89:0/820398024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f869415db10 0x7f8694154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:35.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.947+0000 7f868bfff700 1 --2- 172.21.15.89:0/820398024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8694159c60 0x7f8694151280 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f868400ec80 tx=0x7f86840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:35.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.947+0000 7f8688ff9700 1 -- 172.21.15.89:0/820398024 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868400c850 con 0x7f8694159c60 2023-10-10T20:41:35.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.947+0000 7f8688ff9700 1 -- 172.21.15.89:0/820398024 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868400c9b0 con 0x7f8694159c60 2023-10-10T20:41:35.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.947+0000 7f869ace7700 1 -- 172.21.15.89:0/820398024 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8694152420 con 0x7f8694159c60 2023-10-10T20:41:35.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.947+0000 7f8688ff9700 1 -- 172.21.15.89:0/820398024 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868400cd50 con 0x7f8694159c60 2023-10-10T20:41:35.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.947+0000 7f869ace7700 1 -- 172.21.15.89:0/820398024 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86941517c0 con 0x7f8694159c60 2023-10-10T20:41:35.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.948+0000 7f869ace7700 1 -- 172.21.15.89:0/820398024 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86941525b0 con 0x7f8694159c60 2023-10-10T20:41:35.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.948+0000 7f8688ff9700 1 -- 172.21.15.89:0/820398024 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8684013680 con 0x7f8694159c60 2023-10-10T20:41:35.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.948+0000 7f8688ff9700 1 --2- 172.21.15.89:0/820398024 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8680067320 0x7f86800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:35.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.949+0000 7f8688ff9700 1 -- 172.21.15.89:0/820398024 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f8684084fe0 con 0x7f8694159c60 2023-10-10T20:41:35.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.949+0000 7f8698a83700 1 --2- 172.21.15.89:0/820398024 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8680067320 0x7f86800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:35.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.949+0000 7f8698a83700 1 --2- 172.21.15.89:0/820398024 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8680067320 0x7f86800697d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8690007fd0 tx=0x7f86900016f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:35.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:35.953+0000 7f8688ff9700 1 -- 172.21.15.89:0/820398024 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8684051bc0 con 0x7f8694159c60 2023-10-10T20:41:36.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.139+0000 7f869ace7700 1 -- 172.21.15.89:0/820398024 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86940cd150 con 0x7f8694159c60 2023-10-10T20:41:36.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.140+0000 7f8688ff9700 1 -- 172.21.15.89:0/820398024 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+15623 (secure 0 0 0) 0x7f86940cd150 con 0x7f8694159c60 2023-10-10T20:41:36.142 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:36.143 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":126,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:41:35.137778+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:41:05.928269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-10T20:41:06.527456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6835/3349522977":"2023-10-11T20:41:32.133265+0000","172.21.15.89:6834/3349522977":"2023-10-11T20:41:32.133265+0000","172.21.15.89:6837/3182024657":"2023-10-11T20:41:28.731530+0000","172.21.15.89:6836/3182024657":"2023-10-11T20:41:28.731530+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:41:36.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.142+0000 7f869ace7700 1 -- 172.21.15.89:0/820398024 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8680067320 msgr2=0x7f86800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.142+0000 7f869ace7700 1 --2- 172.21.15.89:0/820398024 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8680067320 0x7f86800697d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8690007fd0 tx=0x7f86900016f0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.142+0000 7f869ace7700 1 -- 172.21.15.89:0/820398024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8694159c60 msgr2=0x7f8694151280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.142+0000 7f869ace7700 1 --2- 172.21.15.89:0/820398024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8694159c60 0x7f8694151280 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f868400ec80 tx=0x7f86840091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.143+0000 7f869ace7700 1 -- 172.21.15.89:0/820398024 shutdown_connections 2023-10-10T20:41:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.143+0000 7f869ace7700 1 --2- 172.21.15.89:0/820398024 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8680067320 0x7f86800697d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.143+0000 7f869ace7700 1 --2- 172.21.15.89:0/820398024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f869415db10 0x7f8694154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.143+0000 7f869ace7700 1 --2- 172.21.15.89:0/820398024 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8694159890 0x7f869414c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.143+0000 7f869ace7700 1 --2- 172.21.15.89:0/820398024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8694159c60 0x7f8694151280 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.143+0000 7f869ace7700 1 -- 172.21.15.89:0/820398024 >> 172.21.15.89:0/820398024 conn(0x7f8694068e20 msgr2=0x7f8694157300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:36.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.143+0000 7f869ace7700 1 -- 172.21.15.89:0/820398024 shutdown_connections 2023-10-10T20:41:36.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.143+0000 7f869ace7700 1 -- 172.21.15.89:0/820398024 wait complete. 2023-10-10T20:41:36.158 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:41:05.928269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:41:06.527456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:41:36.159 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:41:36.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.275+0000 7f6098d3f700 1 Processor -- start 2023-10-10T20:41:36.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.275+0000 7f6098d3f700 1 -- start start 2023-10-10T20:41:36.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f6098d3f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6094156690 0x7f60940ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:36.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f6098d3f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6094157e30 con 0x7f6094156a60 2023-10-10T20:41:36.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f6098d3f700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60941578b0 con 0x7f60940af480 2023-10-10T20:41:36.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f6098d3f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609406aa00 con 0x7f6094156690 2023-10-10T20:41:36.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f6092d9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6094156a60 0x7f609406ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44828/0 (socket says 172.21.15.89:44828) 2023-10-10T20:41:36.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f6092d9d700 1 -- 172.21.15.89:0/3404950623 learned_addr learned my addr 172.21.15.89:0/3404950623 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:36.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f609159a700 1 -- 172.21.15.89:0/3404950623 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060787739 0 0) 0x7f6094157e30 con 0x7f6094156a60 2023-10-10T20:41:36.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f609259c700 1 --2- 172.21.15.89:0/3404950623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6094156690 0x7f60940ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:36.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f609159a700 1 -- 172.21.15.89:0/3404950623 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f607c003570 con 0x7f6094156a60 2023-10-10T20:41:36.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f609159a700 1 -- 172.21.15.89:0/3404950623 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1105425801 0 0) 0x7f607c003570 con 0x7f6094156a60 2023-10-10T20:41:36.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f609159a700 1 -- 172.21.15.89:0/3404950623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6094156690 msgr2=0x7f60940ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:36.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f609159a700 1 --2- 172.21.15.89:0/3404950623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6094156690 0x7f60940ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f609159a700 1 -- 172.21.15.89:0/3404950623 >> v1:172.21.15.111:6789/0 conn(0x7f60940af480 legacy=0x7f60940c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f609159a700 1 -- 172.21.15.89:0/3404950623 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6094157b50 con 0x7f6094156a60 2023-10-10T20:41:36.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f609159a700 1 -- 172.21.15.89:0/3404950623 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f608c0017f0 con 0x7f6094156a60 2023-10-10T20:41:36.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f609159a700 1 -- 172.21.15.89:0/3404950623 >> v1:172.21.15.89:6789/0 conn(0x7f6094156a60 legacy=0x7f609406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f609159a700 1 --2- 172.21.15.89:0/3404950623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f607c003a90 0x7f607c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:36.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f609159a700 1 --2- 172.21.15.89:0/3404950623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6094156690 0x7f607c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:36.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f609159a700 1 --2- 172.21.15.89:0/3404950623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f607c004d50 0x7f607c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:36.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f609159a700 1 -- 172.21.15.89:0/3404950623 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609406aa00 con 0x7f607c003a90 2023-10-10T20:41:36.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f609159a700 1 -- 172.21.15.89:0/3404950623 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c00bc20 con 0x7f607c004d50 2023-10-10T20:41:36.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.276+0000 7f609159a700 1 -- 172.21.15.89:0/3404950623 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607c00bde0 con 0x7f6094156690 2023-10-10T20:41:36.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.277+0000 7f609259c700 1 --2- 172.21.15.89:0/3404950623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f607c003a90 0x7f607c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:36.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.277+0000 7f6091d9b700 1 --2- 172.21.15.89:0/3404950623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6094156690 0x7f607c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:36.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.277+0000 7f6092d9d700 1 --2- 172.21.15.89:0/3404950623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f607c004d50 0x7f607c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:36.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.277+0000 7f609259c700 1 -- 172.21.15.89:0/3404950623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6094156690 msgr2=0x7f607c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:36.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.277+0000 7f609259c700 1 --2- 172.21.15.89:0/3404950623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6094156690 0x7f607c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.277+0000 7f609259c700 1 -- 172.21.15.89:0/3404950623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f607c004d50 msgr2=0x7f607c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:36.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.277+0000 7f609259c700 1 --2- 172.21.15.89:0/3404950623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f607c004d50 0x7f607c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.277+0000 7f609259c700 1 -- 172.21.15.89:0/3404950623 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c003570 con 0x7f607c003a90 2023-10-10T20:41:36.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.277+0000 7f6091d9b700 1 --2- 172.21.15.89:0/3404950623 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6094156690 0x7f607c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:36.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.277+0000 7f609259c700 1 --2- 172.21.15.89:0/3404950623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f607c003a90 0x7f607c003ea0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f6084006f00 tx=0x7f60840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776bb00fa03a95b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:36.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.277+0000 7f609159a700 1 -- 172.21.15.89:0/3404950623 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608400e710 con 0x7f607c003a90 2023-10-10T20:41:36.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.277+0000 7f609159a700 1 -- 172.21.15.89:0/3404950623 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608400e870 con 0x7f607c003a90 2023-10-10T20:41:36.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.277+0000 7f609159a700 1 -- 172.21.15.89:0/3404950623 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608400ec30 con 0x7f607c003a90 2023-10-10T20:41:36.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.278+0000 7f6098d3f700 1 -- 172.21.15.89:0/3404950623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f607c003a90 msgr2=0x7f607c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.278+0000 7f6098d3f700 1 --2- 172.21.15.89:0/3404950623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f607c003a90 0x7f607c003ea0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f6084006f00 tx=0x7f60840091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.278+0000 7f6092d9d700 1 -- 172.21.15.89:0/3404950623 reap_dead start 2023-10-10T20:41:36.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.278+0000 7f6098d3f700 1 -- 172.21.15.89:0/3404950623 shutdown_connections 2023-10-10T20:41:36.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.278+0000 7f6098d3f700 1 -- 172.21.15.89:0/3404950623 >> 172.21.15.89:0/3404950623 conn(0x7f60940c9870 msgr2=0x7f60940c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:36.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.278+0000 7f6098d3f700 1 -- 172.21.15.89:0/3404950623 shutdown_connections 2023-10-10T20:41:36.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.278+0000 7f6098d3f700 1 -- 172.21.15.89:0/3404950623 wait complete. 2023-10-10T20:41:36.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.279+0000 7f6098d3f700 1 Processor -- start 2023-10-10T20:41:36.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.279+0000 7f6098d3f700 1 -- start start 2023-10-10T20:41:36.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.279+0000 7f6098d3f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f607c003a90 0x7f60940b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:36.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.279+0000 7f6098d3f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6094147960 0x7f60940c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:36.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.279+0000 7f6098d3f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6094068390 0x7f6094068830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:36.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.280+0000 7f6098d3f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6094068d70 con 0x7f607c003a90 2023-10-10T20:41:36.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.280+0000 7f6098d3f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60940c5210 con 0x7f6094068390 2023-10-10T20:41:36.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.280+0000 7f6098d3f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60940c43f0 con 0x7f6094147960 2023-10-10T20:41:36.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.280+0000 7f609259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f607c003a90 0x7f60940b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:36.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.280+0000 7f609259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f607c003a90 0x7f60940b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52396/0 (socket says 172.21.15.89:52396) 2023-10-10T20:41:36.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.280+0000 7f609259c700 1 -- 172.21.15.89:0/2564570516 learned_addr learned my addr 172.21.15.89:0/2564570516 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:36.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.280+0000 7f6092d9d700 1 --2- 172.21.15.89:0/2564570516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6094068390 0x7f6094068830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:36.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.280+0000 7f6091d9b700 1 --2- 172.21.15.89:0/2564570516 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6094147960 0x7f60940c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:36.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.280+0000 7f609259c700 1 -- 172.21.15.89:0/2564570516 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6094147960 msgr2=0x7f60940c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.280+0000 7f609259c700 1 --2- 172.21.15.89:0/2564570516 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6094147960 0x7f60940c3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.280+0000 7f609259c700 1 -- 172.21.15.89:0/2564570516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6094068390 msgr2=0x7f6094068830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.280+0000 7f609259c700 1 --2- 172.21.15.89:0/2564570516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6094068390 0x7f6094068830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.280+0000 7f609259c700 1 -- 172.21.15.89:0/2564570516 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6094147dd0 con 0x7f607c003a90 2023-10-10T20:41:36.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.280+0000 7f609259c700 1 --2- 172.21.15.89:0/2564570516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f607c003a90 0x7f60940b2760 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f6084013ef0 tx=0x7f608401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:36.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.280+0000 7f60827fc700 1 -- 172.21.15.89:0/2564570516 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6084022070 con 0x7f607c003a90 2023-10-10T20:41:36.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.280+0000 7f60827fc700 1 -- 172.21.15.89:0/2564570516 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608400e9e0 con 0x7f607c003a90 2023-10-10T20:41:36.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.280+0000 7f60827fc700 1 -- 172.21.15.89:0/2564570516 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6084027470 con 0x7f607c003a90 2023-10-10T20:41:36.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.280+0000 7f6098d3f700 1 -- 172.21.15.89:0/2564570516 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60940c4690 con 0x7f607c003a90 2023-10-10T20:41:36.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.280+0000 7f6098d3f700 1 -- 172.21.15.89:0/2564570516 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60940c4c50 con 0x7f607c003a90 2023-10-10T20:41:36.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.281+0000 7f6098d3f700 1 -- 172.21.15.89:0/2564570516 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f609404c3c0 con 0x7f607c003a90 2023-10-10T20:41:36.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.282+0000 7f60827fc700 1 -- 172.21.15.89:0/2564570516 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f608401c450 con 0x7f607c003a90 2023-10-10T20:41:36.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.282+0000 7f60827fc700 1 --2- 172.21.15.89:0/2564570516 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6074067320 0x7f60740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:36.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.282+0000 7f60827fc700 1 -- 172.21.15.89:0/2564570516 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f6084086870 con 0x7f607c003a90 2023-10-10T20:41:36.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.282+0000 7f6091d9b700 1 --2- 172.21.15.89:0/2564570516 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6074067320 0x7f60740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:36.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.282+0000 7f6091d9b700 1 --2- 172.21.15.89:0/2564570516 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6074067320 0x7f60740697d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6088000c00 tx=0x7f6088009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:36.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.286+0000 7f60827fc700 1 -- 172.21.15.89:0/2564570516 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6084053450 con 0x7f607c003a90 2023-10-10T20:41:36.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.467+0000 7f6098d3f700 1 -- 172.21.15.89:0/2564570516 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f609406ef30 con 0x7f607c003a90 2023-10-10T20:41:36.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.467+0000 7f60827fc700 1 -- 172.21.15.89:0/2564570516 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) v1 ==== 93+0+10 (secure 0 0 0) 0x7f60840170a0 con 0x7f607c003a90 2023-10-10T20:41:36.470 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:41:36.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.469+0000 7f6098d3f700 1 -- 172.21.15.89:0/2564570516 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6074067320 msgr2=0x7f60740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.469+0000 7f6098d3f700 1 --2- 172.21.15.89:0/2564570516 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6074067320 0x7f60740697d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6088000c00 tx=0x7f6088009040 comp rx=0 tx=0).stop 2023-10-10T20:41:36.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.469+0000 7f6098d3f700 1 -- 172.21.15.89:0/2564570516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f607c003a90 msgr2=0x7f60940b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.469+0000 7f6098d3f700 1 --2- 172.21.15.89:0/2564570516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f607c003a90 0x7f60940b2760 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f6084013ef0 tx=0x7f608401a000 comp rx=0 tx=0).stop 2023-10-10T20:41:36.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.469+0000 7f6098d3f700 1 -- 172.21.15.89:0/2564570516 shutdown_connections 2023-10-10T20:41:36.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.469+0000 7f6098d3f700 1 --2- 172.21.15.89:0/2564570516 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6074067320 0x7f60740697d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.469+0000 7f6098d3f700 1 --2- 172.21.15.89:0/2564570516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6094068390 0x7f6094068830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.469+0000 7f6098d3f700 1 --2- 172.21.15.89:0/2564570516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f607c003a90 0x7f60940b2760 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.469+0000 7f6098d3f700 1 --2- 172.21.15.89:0/2564570516 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6094147960 0x7f60940c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.469+0000 7f6098d3f700 1 -- 172.21.15.89:0/2564570516 >> 172.21.15.89:0/2564570516 conn(0x7f60940c9870 msgr2=0x7f60940c26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:36.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.469+0000 7f6098d3f700 1 -- 172.21.15.89:0/2564570516 shutdown_connections 2023-10-10T20:41:36.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.469+0000 7f6098d3f700 1 -- 172.21.15.89:0/2564570516 wait complete. 2023-10-10T20:41:36.485 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:41:36.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.601+0000 7f97f08a8700 1 Processor -- start 2023-10-10T20:41:36.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.601+0000 7f97f08a8700 1 -- start start 2023-10-10T20:41:36.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.602+0000 7f97f08a8700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f97ec0b4e70 0x7f97ec0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:36.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.602+0000 7f97f08a8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f97ec0b16d0 0x7f97ec0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:36.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.602+0000 7f97f08a8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ec0b2080 con 0x7f97ec15d240 2023-10-10T20:41:36.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.602+0000 7f97f08a8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ec0cfb00 con 0x7f97ec0b16d0 2023-10-10T20:41:36.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.602+0000 7f97f08a8700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ec0b5c40 con 0x7f97ec0b4e70 2023-10-10T20:41:36.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.602+0000 7f97eaffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f97ec0b16d0 0x7f97ec0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:36.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.602+0000 7f97ebfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f97ec15d240 0x7f97ec15d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44834/0 (socket says 172.21.15.89:44834) 2023-10-10T20:41:36.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.602+0000 7f97ebfff700 1 -- 172.21.15.89:0/2827728483 learned_addr learned my addr 172.21.15.89:0/2827728483 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:36.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.602+0000 7f97eb7fe700 1 --2- 172.21.15.89:0/2827728483 >> v2:172.21.15.111:3300/0 conn(0x7f97ec0b4e70 0x7f97ec0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:36.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.602+0000 7f97eaffd700 1 -- 172.21.15.89:0/2827728483 >> v2:172.21.15.111:3300/0 conn(0x7f97ec0b4e70 msgr2=0x7f97ec0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:36.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.602+0000 7f97eaffd700 1 --2- 172.21.15.89:0/2827728483 >> v2:172.21.15.111:3300/0 conn(0x7f97ec0b4e70 0x7f97ec0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.602+0000 7f97eaffd700 1 -- 172.21.15.89:0/2827728483 >> v1:172.21.15.89:6789/0 conn(0x7f97ec15d240 legacy=0x7f97ec15d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.602+0000 7f97eaffd700 1 -- 172.21.15.89:0/2827728483 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec0b5f20 con 0x7f97ec0b16d0 2023-10-10T20:41:36.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.602+0000 7f97ea7fc700 1 -- 172.21.15.89:0/2827728483 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3751395923 0 0) 0x7f97ec0b2080 con 0x7f97ec15d240 2023-10-10T20:41:36.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.603+0000 7f97eaffd700 1 --2- 172.21.15.89:0/2827728483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97ec0b16d0 0x7f97ec0b1b40 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f97ec064910 tx=0x7f97e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e87e31afcb9a1b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:36.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.603+0000 7f97ea7fc700 1 -- 172.21.15.89:0/2827728483 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0011630 con 0x7f97ec0b16d0 2023-10-10T20:41:36.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.603+0000 7f97ea7fc700 1 -- 172.21.15.89:0/2827728483 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e0011790 con 0x7f97ec0b16d0 2023-10-10T20:41:36.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.603+0000 7f97ea7fc700 1 -- 172.21.15.89:0/2827728483 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0011b30 con 0x7f97ec0b16d0 2023-10-10T20:41:36.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.603+0000 7f97f08a8700 1 -- 172.21.15.89:0/2827728483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97ec0b16d0 msgr2=0x7f97ec0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.603+0000 7f97f08a8700 1 --2- 172.21.15.89:0/2827728483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97ec0b16d0 0x7f97ec0b1b40 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f97ec064910 tx=0x7f97e00091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.603+0000 7f97f08a8700 1 -- 172.21.15.89:0/2827728483 shutdown_connections 2023-10-10T20:41:36.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.603+0000 7f97f08a8700 1 --2- 172.21.15.89:0/2827728483 >> v2:172.21.15.111:3300/0 conn(0x7f97ec0b4e70 0x7f97ec0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.603+0000 7f97f08a8700 1 --2- 172.21.15.89:0/2827728483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97ec0b16d0 0x7f97ec0b1b40 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.603+0000 7f97f08a8700 1 -- 172.21.15.89:0/2827728483 >> 172.21.15.89:0/2827728483 conn(0x7f97ec0d10c0 msgr2=0x7f97ec0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:36.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.603+0000 7f97f08a8700 1 -- 172.21.15.89:0/2827728483 shutdown_connections 2023-10-10T20:41:36.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.603+0000 7f97f08a8700 1 -- 172.21.15.89:0/2827728483 wait complete. 2023-10-10T20:41:36.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.605+0000 7f97f08a8700 1 Processor -- start 2023-10-10T20:41:36.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.605+0000 7f97f08a8700 1 -- start start 2023-10-10T20:41:36.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.605+0000 7f97f08a8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ec0b16d0 0x7f97ec0c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:36.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.605+0000 7f97f08a8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f97ec0b4e70 0x7f97ec169ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:36.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.605+0000 7f97f08a8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97ec15d240 0x7f97ec15e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:36.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.605+0000 7f97f08a8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec16ac80 con 0x7f97ec15d240 2023-10-10T20:41:36.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.605+0000 7f97f08a8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec16d750 con 0x7f97ec0b16d0 2023-10-10T20:41:36.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.605+0000 7f97f08a8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec16a6c0 con 0x7f97ec0b4e70 2023-10-10T20:41:36.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.605+0000 7f97ebfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97ec15d240 0x7f97ec15e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:36.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.605+0000 7f97ebfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97ec15d240 0x7f97ec15e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52422/0 (socket says 172.21.15.89:52422) 2023-10-10T20:41:36.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.605+0000 7f97ebfff700 1 -- 172.21.15.89:0/2074634341 learned_addr learned my addr 172.21.15.89:0/2074634341 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:36.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.606+0000 7f97eb7fe700 1 --2- 172.21.15.89:0/2074634341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ec0b16d0 0x7f97ec0c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:36.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.606+0000 7f97eaffd700 1 --2- 172.21.15.89:0/2074634341 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f97ec0b4e70 0x7f97ec169ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:36.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.606+0000 7f97ebfff700 1 -- 172.21.15.89:0/2074634341 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f97ec0b4e70 msgr2=0x7f97ec169ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.606+0000 7f97ebfff700 1 --2- 172.21.15.89:0/2074634341 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f97ec0b4e70 0x7f97ec169ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.606+0000 7f97ebfff700 1 -- 172.21.15.89:0/2074634341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ec0b16d0 msgr2=0x7f97ec0c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.606+0000 7f97ebfff700 1 --2- 172.21.15.89:0/2074634341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ec0b16d0 0x7f97ec0c81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.606+0000 7f97ebfff700 1 -- 172.21.15.89:0/2074634341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e0009420 con 0x7f97ec15d240 2023-10-10T20:41:36.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.606+0000 7f97eb7fe700 1 --2- 172.21.15.89:0/2074634341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ec0b16d0 0x7f97ec0c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:36.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.606+0000 7f97ebfff700 1 --2- 172.21.15.89:0/2074634341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97ec15d240 0x7f97ec15e630 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f97e400cbf0 tx=0x7f97e40019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:36.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.606+0000 7f97d37fe700 1 -- 172.21.15.89:0/2074634341 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e400a9c0 con 0x7f97ec15d240 2023-10-10T20:41:36.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.606+0000 7f97d37fe700 1 -- 172.21.15.89:0/2074634341 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e400ab20 con 0x7f97ec15d240 2023-10-10T20:41:36.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.606+0000 7f97f08a8700 1 -- 172.21.15.89:0/2074634341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ec16a9a0 con 0x7f97ec15d240 2023-10-10T20:41:36.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.606+0000 7f97d37fe700 1 -- 172.21.15.89:0/2074634341 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e40070f0 con 0x7f97ec15d240 2023-10-10T20:41:36.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.606+0000 7f97f08a8700 1 -- 172.21.15.89:0/2074634341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ec0c86f0 con 0x7f97ec15d240 2023-10-10T20:41:36.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.607+0000 7f97f08a8700 1 -- 172.21.15.89:0/2074634341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ec16d240 con 0x7f97ec15d240 2023-10-10T20:41:36.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.607+0000 7f97d37fe700 1 -- 172.21.15.89:0/2074634341 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f97e4007520 con 0x7f97ec15d240 2023-10-10T20:41:36.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.608+0000 7f97d37fe700 1 --2- 172.21.15.89:0/2074634341 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f97d8067320 0x7f97d80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:36.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.608+0000 7f97d37fe700 1 -- 172.21.15.89:0/2074634341 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f97e4085100 con 0x7f97ec15d240 2023-10-10T20:41:36.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.608+0000 7f97eb7fe700 1 --2- 172.21.15.89:0/2074634341 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f97d8067320 0x7f97d80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:36.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.608+0000 7f97eb7fe700 1 --2- 172.21.15.89:0/2074634341 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f97d8067320 0x7f97d80697d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f97dc006f00 tx=0x7f97dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:36.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.612+0000 7f97d37fe700 1 -- 172.21.15.89:0/2074634341 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97ec16d240 con 0x7f97ec15d240 2023-10-10T20:41:36.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.799+0000 7f97f08a8700 1 -- 172.21.15.89:0/2074634341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f97ec16d240 con 0x7f97ec15d240 2023-10-10T20:41:36.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.799+0000 7f97d37fe700 1 -- 172.21.15.89:0/2074634341 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v126) v1 ==== 104+0+11 (secure 0 0 0) 0x7f97ec16d240 con 0x7f97ec15d240 2023-10-10T20:41:36.802 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:41:36.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.801+0000 7f97f08a8700 1 -- 172.21.15.89:0/2074634341 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f97d8067320 msgr2=0x7f97d80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.801+0000 7f97f08a8700 1 --2- 172.21.15.89:0/2074634341 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f97d8067320 0x7f97d80697d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f97dc006f00 tx=0x7f97dc008040 comp rx=0 tx=0).stop 2023-10-10T20:41:36.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.801+0000 7f97f08a8700 1 -- 172.21.15.89:0/2074634341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97ec15d240 msgr2=0x7f97ec15e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.801+0000 7f97f08a8700 1 --2- 172.21.15.89:0/2074634341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97ec15d240 0x7f97ec15e630 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f97e400cbf0 tx=0x7f97e40019b0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.801+0000 7f97f08a8700 1 -- 172.21.15.89:0/2074634341 shutdown_connections 2023-10-10T20:41:36.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.801+0000 7f97f08a8700 1 --2- 172.21.15.89:0/2074634341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97ec15d240 0x7f97ec15e630 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.801+0000 7f97f08a8700 1 --2- 172.21.15.89:0/2074634341 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f97d8067320 0x7f97d80697d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.801+0000 7f97f08a8700 1 --2- 172.21.15.89:0/2074634341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ec0b16d0 0x7f97ec0c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.801+0000 7f97f08a8700 1 --2- 172.21.15.89:0/2074634341 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f97ec0b4e70 0x7f97ec169ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.801+0000 7f97f08a8700 1 -- 172.21.15.89:0/2074634341 >> 172.21.15.89:0/2074634341 conn(0x7f97ec0d10c0 msgr2=0x7f97ec0b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:36.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.801+0000 7f97f08a8700 1 -- 172.21.15.89:0/2074634341 shutdown_connections 2023-10-10T20:41:36.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.801+0000 7f97f08a8700 1 -- 172.21.15.89:0/2074634341 wait complete. 2023-10-10T20:41:36.815 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:41:36.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.931+0000 7f63c0c34700 1 Processor -- start 2023-10-10T20:41:36.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.931+0000 7f63c0c34700 1 -- start start 2023-10-10T20:41:36.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.932+0000 7f63c0c34700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f63bc159c60 0x7f63bc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:36.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.932+0000 7f63c0c34700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63bc15a160 con 0x7f63bc159890 2023-10-10T20:41:36.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.932+0000 7f63c0c34700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63bc0d13a0 con 0x7f63bc15db10 2023-10-10T20:41:36.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.932+0000 7f63c0c34700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63bc15df50 con 0x7f63bc159c60 2023-10-10T20:41:36.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.932+0000 7f63b9d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f63bc159c60 0x7f63bc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:36.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.932+0000 7f63ba59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f63bc159890 0x7f63bc15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44846/0 (socket says 172.21.15.89:44846) 2023-10-10T20:41:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.932+0000 7f63ba59c700 1 -- 172.21.15.89:0/1360234068 learned_addr learned my addr 172.21.15.89:0/1360234068 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.932+0000 7f63b9d9b700 1 -- 172.21.15.89:0/1360234068 >> v1:172.21.15.111:6789/0 conn(0x7f63bc15db10 legacy=0x7f63bc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.932+0000 7f63b9d9b700 1 -- 172.21.15.89:0/1360234068 >> v1:172.21.15.89:6789/0 conn(0x7f63bc159890 legacy=0x7f63bc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.932+0000 7f63b9d9b700 1 -- 172.21.15.89:0/1360234068 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc15e140 con 0x7f63bc159c60 2023-10-10T20:41:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.932+0000 7f63b959a700 1 -- 172.21.15.89:0/1360234068 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393238900 0 0) 0x7f63bc15a160 con 0x7f63bc159890 2023-10-10T20:41:36.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.932+0000 7f63b9d9b700 1 --2- 172.21.15.89:0/1360234068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63bc159c60 0x7f63bc15d5d0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f63a40098e0 tx=0x7f63a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfca7fa54ff1481 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:36.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.933+0000 7f63b959a700 1 -- 172.21.15.89:0/1360234068 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a4009e80 con 0x7f63bc159c60 2023-10-10T20:41:36.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.933+0000 7f63b959a700 1 -- 172.21.15.89:0/1360234068 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a400b860 con 0x7f63bc159c60 2023-10-10T20:41:36.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.933+0000 7f63b959a700 1 -- 172.21.15.89:0/1360234068 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a400bbe0 con 0x7f63bc159c60 2023-10-10T20:41:36.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.933+0000 7f63c0c34700 1 -- 172.21.15.89:0/1360234068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63bc159c60 msgr2=0x7f63bc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.933+0000 7f63c0c34700 1 --2- 172.21.15.89:0/1360234068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63bc159c60 0x7f63bc15d5d0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f63a40098e0 tx=0x7f63a40091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.933+0000 7f63c0c34700 1 -- 172.21.15.89:0/1360234068 shutdown_connections 2023-10-10T20:41:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.933+0000 7f63c0c34700 1 --2- 172.21.15.89:0/1360234068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63bc159c60 0x7f63bc15d5d0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.933+0000 7f63c0c34700 1 -- 172.21.15.89:0/1360234068 >> 172.21.15.89:0/1360234068 conn(0x7f63bc068e20 msgr2=0x7f63bc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.933+0000 7f63c0c34700 1 -- 172.21.15.89:0/1360234068 shutdown_connections 2023-10-10T20:41:36.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.933+0000 7f63c0c34700 1 -- 172.21.15.89:0/1360234068 wait complete. 2023-10-10T20:41:36.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.934+0000 7f63c0c34700 1 Processor -- start 2023-10-10T20:41:36.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.934+0000 7f63c0c34700 1 -- start start 2023-10-10T20:41:36.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.935+0000 7f63c0c34700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63bc159890 0x7f63bc0b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:36.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.935+0000 7f63c0c34700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63bc159c60 0x7f63bc0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:36.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.935+0000 7f63c0c34700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63bc15db10 0x7f63bc0b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.935+0000 7f63c0c34700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0b2090 con 0x7f63bc159c60 2023-10-10T20:41:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.935+0000 7f63c0c34700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0b64b0 con 0x7f63bc159890 2023-10-10T20:41:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.935+0000 7f63c0c34700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0b1480 con 0x7f63bc15db10 2023-10-10T20:41:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.935+0000 7f63b9d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63bc159c60 0x7f63bc0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.935+0000 7f63b9d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63bc159c60 0x7f63bc0b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52438/0 (socket says 172.21.15.89:52438) 2023-10-10T20:41:36.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.935+0000 7f63b9d9b700 1 -- 172.21.15.89:0/1933040652 learned_addr learned my addr 172.21.15.89:0/1933040652 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:36.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.935+0000 7f63ba59c700 1 --2- 172.21.15.89:0/1933040652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63bc159890 0x7f63bc0b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:36.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.935+0000 7f63bad9d700 1 --2- 172.21.15.89:0/1933040652 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63bc15db10 0x7f63bc0b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:36.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.935+0000 7f63b9d9b700 1 -- 172.21.15.89:0/1933040652 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63bc15db10 msgr2=0x7f63bc0b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.935+0000 7f63b9d9b700 1 --2- 172.21.15.89:0/1933040652 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63bc15db10 0x7f63bc0b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.935+0000 7f63b9d9b700 1 -- 172.21.15.89:0/1933040652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63bc159890 msgr2=0x7f63bc0b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.935+0000 7f63b9d9b700 1 --2- 172.21.15.89:0/1933040652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63bc159890 0x7f63bc0b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.935+0000 7f63b9d9b700 1 -- 172.21.15.89:0/1933040652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a4009420 con 0x7f63bc159c60 2023-10-10T20:41:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.935+0000 7f63b9d9b700 1 --2- 172.21.15.89:0/1933040652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63bc159c60 0x7f63bc0b3590 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f63bc070e20 tx=0x7f63a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.936+0000 7f63b27fc700 1 -- 172.21.15.89:0/1933040652 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a401f070 con 0x7f63bc159c60 2023-10-10T20:41:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.936+0000 7f63b27fc700 1 -- 172.21.15.89:0/1933040652 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a400b9d0 con 0x7f63bc159c60 2023-10-10T20:41:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.936+0000 7f63b27fc700 1 -- 172.21.15.89:0/1933040652 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a4025670 con 0x7f63bc159c60 2023-10-10T20:41:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.936+0000 7f63c0c34700 1 -- 172.21.15.89:0/1933040652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63bc0b1760 con 0x7f63bc159c60 2023-10-10T20:41:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.936+0000 7f63c0c34700 1 -- 172.21.15.89:0/1933040652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63bc0b1d20 con 0x7f63bc159c60 2023-10-10T20:41:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.936+0000 7f63c0c34700 1 -- 172.21.15.89:0/1933040652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63bc0cd100 con 0x7f63bc159c60 2023-10-10T20:41:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.937+0000 7f63b27fc700 1 -- 172.21.15.89:0/1933040652 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f63a4023070 con 0x7f63bc159c60 2023-10-10T20:41:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.937+0000 7f63b27fc700 1 --2- 172.21.15.89:0/1933040652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f639c067320 0x7f639c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.937+0000 7f63b27fc700 1 -- 172.21.15.89:0/1933040652 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f63a40863d0 con 0x7f63bc159c60 2023-10-10T20:41:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.937+0000 7f63ba59c700 1 --2- 172.21.15.89:0/1933040652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f639c067320 0x7f639c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.938+0000 7f63ba59c700 1 --2- 172.21.15.89:0/1933040652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f639c067320 0x7f639c0697d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f63b4001560 tx=0x7f63b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:36.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:36.941+0000 7f63b27fc700 1 -- 172.21.15.89:0/1933040652 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63bc0cd100 con 0x7f63bc159c60 2023-10-10T20:41:37.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.125+0000 7f63c0c34700 1 -- 172.21.15.89:0/1933040652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f63bc0cd100 con 0x7f63bc159c60 2023-10-10T20:41:37.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.125+0000 7f63b27fc700 1 -- 172.21.15.89:0/1933040652 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v126) v1 ==== 100+0+11 (secure 0 0 0) 0x7f63bc0cd100 con 0x7f63bc159c60 2023-10-10T20:41:37.128 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-10T20:41:37.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.127+0000 7f63c0c34700 1 -- 172.21.15.89:0/1933040652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f639c067320 msgr2=0x7f639c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:37.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.127+0000 7f63c0c34700 1 --2- 172.21.15.89:0/1933040652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f639c067320 0x7f639c0697d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f63b4001560 tx=0x7f63b4009040 comp rx=0 tx=0).stop 2023-10-10T20:41:37.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.127+0000 7f63c0c34700 1 -- 172.21.15.89:0/1933040652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63bc159c60 msgr2=0x7f63bc0b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:37.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.127+0000 7f63c0c34700 1 --2- 172.21.15.89:0/1933040652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63bc159c60 0x7f63bc0b3590 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f63bc070e20 tx=0x7f63a4017000 comp rx=0 tx=0).stop 2023-10-10T20:41:37.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.127+0000 7f63c0c34700 1 -- 172.21.15.89:0/1933040652 shutdown_connections 2023-10-10T20:41:37.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.127+0000 7f63c0c34700 1 --2- 172.21.15.89:0/1933040652 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63bc15db10 0x7f63bc0b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.127+0000 7f63c0c34700 1 --2- 172.21.15.89:0/1933040652 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f639c067320 0x7f639c0697d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.127+0000 7f63c0c34700 1 --2- 172.21.15.89:0/1933040652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63bc159890 0x7f63bc0b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.127+0000 7f63c0c34700 1 --2- 172.21.15.89:0/1933040652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63bc159c60 0x7f63bc0b3590 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.127+0000 7f63c0c34700 1 -- 172.21.15.89:0/1933040652 >> 172.21.15.89:0/1933040652 conn(0x7f63bc068e20 msgr2=0x7f63bc153fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:37.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.127+0000 7f63c0c34700 1 -- 172.21.15.89:0/1933040652 shutdown_connections 2023-10-10T20:41:37.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.127+0000 7f63c0c34700 1 -- 172.21.15.89:0/1933040652 wait complete. 2023-10-10T20:41:37.141 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:41:37.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.254+0000 7fc98d38c700 1 Processor -- start 2023-10-10T20:41:37.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.255+0000 7fc98d38c700 1 -- start start 2023-10-10T20:41:37.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.255+0000 7fc98d38c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc988069560 0x7fc9880699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.255+0000 7fc98d38c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc988069ee0 con 0x7fc988159c60 2023-10-10T20:41:37.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.255+0000 7fc98d38c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc98815a160 con 0x7fc988159890 2023-10-10T20:41:37.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.255+0000 7fc98d38c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9880d13a0 con 0x7fc988069560 2023-10-10T20:41:37.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.255+0000 7fc97e7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc988159c60 0x7fc98815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44850/0 (socket says 172.21.15.89:44850) 2023-10-10T20:41:37.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.256+0000 7fc97e7fc700 1 -- 172.21.15.89:0/3704015412 learned_addr learned my addr 172.21.15.89:0/3704015412 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:37.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.256+0000 7fc9877fe700 1 --2- 172.21.15.89:0/3704015412 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc988069560 0x7fc9880699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:37.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.256+0000 7fc9867fc700 1 -- 172.21.15.89:0/3704015412 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598657181 0 0) 0x7fc988069ee0 con 0x7fc988159c60 2023-10-10T20:41:37.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.256+0000 7fc9867fc700 1 -- 172.21.15.89:0/3704015412 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc974003570 con 0x7fc988159c60 2023-10-10T20:41:37.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.256+0000 7fc9867fc700 1 -- 172.21.15.89:0/3704015412 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19043537 0 0) 0x7fc98815a160 con 0x7fc988159890 2023-10-10T20:41:37.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.256+0000 7fc9867fc700 1 -- 172.21.15.89:0/3704015412 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc988069ee0 con 0x7fc988159890 2023-10-10T20:41:37.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.256+0000 7fc9877fe700 1 -- 172.21.15.89:0/3704015412 >> v1:172.21.15.111:6789/0 conn(0x7fc988159890 legacy=0x7fc98815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:37.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.256+0000 7fc9877fe700 1 -- 172.21.15.89:0/3704015412 >> v1:172.21.15.89:6789/0 conn(0x7fc988159c60 legacy=0x7fc98815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:37.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.256+0000 7fc9867fc700 1 -- 172.21.15.89:0/3704015412 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1130226633 0 0) 0x7fc974003570 con 0x7fc988159c60 2023-10-10T20:41:37.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.256+0000 7fc9877fe700 1 -- 172.21.15.89:0/3704015412 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98815e140 con 0x7fc988069560 2023-10-10T20:41:37.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.256+0000 7fc9877fe700 1 --2- 172.21.15.89:0/3704015412 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc988069560 0x7fc9880699a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc980009e90 tx=0x7fc980009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caaea378f4258c5e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:37.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.256+0000 7fc9867fc700 1 -- 172.21.15.89:0/3704015412 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98000e070 con 0x7fc988069560 2023-10-10T20:41:37.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.256+0000 7fc9867fc700 1 -- 172.21.15.89:0/3704015412 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98000b870 con 0x7fc988069560 2023-10-10T20:41:37.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.256+0000 7fc9867fc700 1 -- 172.21.15.89:0/3704015412 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc970001470 con 0x7fc988069560 2023-10-10T20:41:37.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.257+0000 7fc98d38c700 1 -- 172.21.15.89:0/3704015412 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc988069560 msgr2=0x7fc9880699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:37.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.257+0000 7fc98d38c700 1 --2- 172.21.15.89:0/3704015412 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc988069560 0x7fc9880699a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc980009e90 tx=0x7fc980009780 comp rx=0 tx=0).stop 2023-10-10T20:41:37.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.257+0000 7fc98d38c700 1 -- 172.21.15.89:0/3704015412 shutdown_connections 2023-10-10T20:41:37.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.257+0000 7fc98d38c700 1 --2- 172.21.15.89:0/3704015412 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc988069560 0x7fc9880699a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.257+0000 7fc98d38c700 1 -- 172.21.15.89:0/3704015412 >> 172.21.15.89:0/3704015412 conn(0x7fc988068e20 msgr2=0x7fc9880d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:37.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.257+0000 7fc98d38c700 1 -- 172.21.15.89:0/3704015412 shutdown_connections 2023-10-10T20:41:37.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.257+0000 7fc98d38c700 1 -- 172.21.15.89:0/3704015412 wait complete. 2023-10-10T20:41:37.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.258+0000 7fc98d38c700 1 Processor -- start 2023-10-10T20:41:37.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.258+0000 7fc98d38c700 1 -- start start 2023-10-10T20:41:37.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.259+0000 7fc98d38c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc988069560 0x7fc9880c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.259+0000 7fc98d38c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc988159890 0x7fc9880cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.259+0000 7fc98d38c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc988159c60 0x7fc988157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.259+0000 7fc98d38c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9880c88b0 con 0x7fc988159890 2023-10-10T20:41:37.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.259+0000 7fc98d38c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98816cfd0 con 0x7fc988159c60 2023-10-10T20:41:37.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.259+0000 7fc98d38c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98816d2b0 con 0x7fc988069560 2023-10-10T20:41:37.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.259+0000 7fc97e7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc988159890 0x7fc9880cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:37.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.259+0000 7fc97e7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc988159890 0x7fc9880cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52448/0 (socket says 172.21.15.89:52448) 2023-10-10T20:41:37.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.259+0000 7fc97e7fc700 1 -- 172.21.15.89:0/2766856813 learned_addr learned my addr 172.21.15.89:0/2766856813 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:37.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.259+0000 7fc9877fe700 1 --2- 172.21.15.89:0/2766856813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc988159c60 0x7fc988157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:37.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.259+0000 7fc986ffd700 1 --2- 172.21.15.89:0/2766856813 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc988069560 0x7fc9880c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:37.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.259+0000 7fc97e7fc700 1 -- 172.21.15.89:0/2766856813 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc988069560 msgr2=0x7fc9880c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:37.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.259+0000 7fc97e7fc700 1 --2- 172.21.15.89:0/2766856813 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc988069560 0x7fc9880c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.259+0000 7fc97e7fc700 1 -- 172.21.15.89:0/2766856813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc988159c60 msgr2=0x7fc988157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:37.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.259+0000 7fc97e7fc700 1 --2- 172.21.15.89:0/2766856813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc988159c60 0x7fc988157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.259+0000 7fc97e7fc700 1 -- 172.21.15.89:0/2766856813 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc980009a20 con 0x7fc988159890 2023-10-10T20:41:37.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.259+0000 7fc9877fe700 1 --2- 172.21.15.89:0/2766856813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc988159c60 0x7fc988157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:37.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.259+0000 7fc986ffd700 1 --2- 172.21.15.89:0/2766856813 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc988069560 0x7fc9880c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:37.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.259+0000 7fc97e7fc700 1 --2- 172.21.15.89:0/2766856813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc988159890 0x7fc9880cb040 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc97000cb60 tx=0x7fc970001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:37.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.260+0000 7fc97f7fe700 1 -- 172.21.15.89:0/2766856813 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc970001d60 con 0x7fc988159890 2023-10-10T20:41:37.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.260+0000 7fc97f7fe700 1 -- 172.21.15.89:0/2766856813 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc970007360 con 0x7fc988159890 2023-10-10T20:41:37.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.260+0000 7fc98d38c700 1 -- 172.21.15.89:0/2766856813 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98816d590 con 0x7fc988159890 2023-10-10T20:41:37.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.260+0000 7fc97f7fe700 1 -- 172.21.15.89:0/2766856813 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc970007700 con 0x7fc988159890 2023-10-10T20:41:37.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.260+0000 7fc98d38c700 1 -- 172.21.15.89:0/2766856813 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98816db50 con 0x7fc988159890 2023-10-10T20:41:37.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.261+0000 7fc98d38c700 1 -- 172.21.15.89:0/2766856813 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc98804c3c0 con 0x7fc988159890 2023-10-10T20:41:37.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.261+0000 7fc97f7fe700 1 -- 172.21.15.89:0/2766856813 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc970004a40 con 0x7fc988159890 2023-10-10T20:41:37.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.261+0000 7fc97f7fe700 1 --2- 172.21.15.89:0/2766856813 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc968067320 0x7fc9680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.262+0000 7fc97f7fe700 1 -- 172.21.15.89:0/2766856813 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5877+0+0 (secure 0 0 0) 0x7fc970084140 con 0x7fc988159890 2023-10-10T20:41:37.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.262+0000 7fc986ffd700 1 --2- 172.21.15.89:0/2766856813 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc968067320 0x7fc9680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:37.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.262+0000 7fc986ffd700 1 --2- 172.21.15.89:0/2766856813 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc968067320 0x7fc9680697d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc98806a070 tx=0x7fc978009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:37.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.266+0000 7fc97f7fe700 1 -- 172.21.15.89:0/2766856813 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc970050da0 con 0x7fc988159890 2023-10-10T20:41:37.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.491+0000 7fc98d38c700 1 -- 172.21.15.89:0/2766856813 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc98806a5b0 con 0x7fc988159890 2023-10-10T20:41:37.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.491+0000 7fc97f7fe700 1 -- 172.21.15.89:0/2766856813 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fc9700543c0 con 0x7fc988159890 2023-10-10T20:41:37.494 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:37.494 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6517,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/2658508419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2658508419},{"type":"v1","addr":"172.21.15.111:6834","nonce":2658508419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7431,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1872461145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1872461145},{"type":"v1","addr":"172.21.15.89:6837","nonce":1872461145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3243330447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3243330447},{"type":"v1","addr":"172.21.15.89:6835","nonce":3243330447}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:41:07.565995+0000","modified":"2023-10-10T20:41:11.225905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"d","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/2582059153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2582059153},{"type":"v1","addr":"172.21.15.111:6835","nonce":2582059153}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-10T20:41:37.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.494+0000 7fc98d38c700 1 -- 172.21.15.89:0/2766856813 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc968067320 msgr2=0x7fc9680697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:37.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.494+0000 7fc98d38c700 1 --2- 172.21.15.89:0/2766856813 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc968067320 0x7fc9680697d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc98806a070 tx=0x7fc978009040 comp rx=0 tx=0).stop 2023-10-10T20:41:37.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.494+0000 7fc98d38c700 1 -- 172.21.15.89:0/2766856813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc988159890 msgr2=0x7fc9880cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:37.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.494+0000 7fc98d38c700 1 --2- 172.21.15.89:0/2766856813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc988159890 0x7fc9880cb040 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc97000cb60 tx=0x7fc970001ad0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.494+0000 7fc98d38c700 1 -- 172.21.15.89:0/2766856813 shutdown_connections 2023-10-10T20:41:37.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.494+0000 7fc98d38c700 1 --2- 172.21.15.89:0/2766856813 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc968067320 0x7fc9680697d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.494+0000 7fc98d38c700 1 --2- 172.21.15.89:0/2766856813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc988159c60 0x7fc988157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.494+0000 7fc98d38c700 1 --2- 172.21.15.89:0/2766856813 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc988069560 0x7fc9880c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.494+0000 7fc98d38c700 1 --2- 172.21.15.89:0/2766856813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc988159890 0x7fc9880cb040 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.494+0000 7fc98d38c700 1 -- 172.21.15.89:0/2766856813 >> 172.21.15.89:0/2766856813 conn(0x7fc988068e20 msgr2=0x7fc9880cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:37.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.494+0000 7fc98d38c700 1 -- 172.21.15.89:0/2766856813 shutdown_connections 2023-10-10T20:41:37.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.494+0000 7fc98d38c700 1 -- 172.21.15.89:0/2766856813 wait complete. 2023-10-10T20:41:37.499 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 83 2023-10-10T20:41:37.508 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:41:37.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.620+0000 7ff7db37c700 1 Processor -- start 2023-10-10T20:41:37.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.620+0000 7ff7db37c700 1 -- start start 2023-10-10T20:41:37.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.621+0000 7ff7db37c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff7d4159420 0x7ff7d40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.621+0000 7ff7db37c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7d41597f0 0x7ff7d40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.621+0000 7ff7db37c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff7d415d230 0x7ff7d415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.621+0000 7ff7db37c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d415dbd0 con 0x7ff7d4159420 2023-10-10T20:41:37.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.621+0000 7ff7db37c700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d40b11d0 con 0x7ff7d415d230 2023-10-10T20:41:37.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.621+0000 7ff7db37c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d40b14b0 con 0x7ff7d41597f0 2023-10-10T20:41:37.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.621+0000 7ff7da37a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff7d4159420 0x7ff7d40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:37.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.621+0000 7ff7da37a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff7d4159420 0x7ff7d40b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52464/0 (socket says 172.21.15.89:52464) 2023-10-10T20:41:37.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.621+0000 7ff7da37a700 1 -- 172.21.15.89:0/3486674666 learned_addr learned my addr 172.21.15.89:0/3486674666 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:37.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.621+0000 7ff7d9b79700 1 --2- 172.21.15.89:0/3486674666 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7d41597f0 0x7ff7d40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:37.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.621+0000 7ff7dab7b700 1 --2- 172.21.15.89:0/3486674666 >> v2:172.21.15.111:3300/0 conn(0x7ff7d415d230 0x7ff7d415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:37.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.621+0000 7ff7da37a700 1 -- 172.21.15.89:0/3486674666 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7d41597f0 msgr2=0x7ff7d40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:37.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.621+0000 7ff7da37a700 1 --2- 172.21.15.89:0/3486674666 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7d41597f0 0x7ff7d40b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.621+0000 7ff7da37a700 1 -- 172.21.15.89:0/3486674666 >> v2:172.21.15.111:3300/0 conn(0x7ff7d415d230 msgr2=0x7ff7d415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:37.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.621+0000 7ff7da37a700 1 --2- 172.21.15.89:0/3486674666 >> v2:172.21.15.111:3300/0 conn(0x7ff7d415d230 0x7ff7d415d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.621+0000 7ff7da37a700 1 -- 172.21.15.89:0/3486674666 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d40b1790 con 0x7ff7d4159420 2023-10-10T20:41:37.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.621+0000 7ff7dab7b700 1 --2- 172.21.15.89:0/3486674666 >> v2:172.21.15.111:3300/0 conn(0x7ff7d415d230 0x7ff7d415d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:37.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.621+0000 7ff7da37a700 1 --2- 172.21.15.89:0/3486674666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d4159420 0x7ff7d40b5f40 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40d1720 tx=0x7ff7cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f587a46de25d9fc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:37.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.622+0000 7ff7d9378700 1 -- 172.21.15.89:0/3486674666 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc00b860 con 0x7ff7d4159420 2023-10-10T20:41:37.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.622+0000 7ff7d9378700 1 -- 172.21.15.89:0/3486674666 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7cc00b9c0 con 0x7ff7d4159420 2023-10-10T20:41:37.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.622+0000 7ff7d9378700 1 -- 172.21.15.89:0/3486674666 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc00bd60 con 0x7ff7d4159420 2023-10-10T20:41:37.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.622+0000 7ff7db37c700 1 -- 172.21.15.89:0/3486674666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d4159420 msgr2=0x7ff7d40b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:37.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.622+0000 7ff7db37c700 1 --2- 172.21.15.89:0/3486674666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d4159420 0x7ff7d40b5f40 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40d1720 tx=0x7ff7cc009250 comp rx=0 tx=0).stop 2023-10-10T20:41:37.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.622+0000 7ff7db37c700 1 -- 172.21.15.89:0/3486674666 shutdown_connections 2023-10-10T20:41:37.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.622+0000 7ff7db37c700 1 --2- 172.21.15.89:0/3486674666 >> v2:172.21.15.111:3300/0 conn(0x7ff7d415d230 0x7ff7d415d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.622+0000 7ff7db37c700 1 --2- 172.21.15.89:0/3486674666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d4159420 0x7ff7d40b5f40 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.622+0000 7ff7db37c700 1 --2- 172.21.15.89:0/3486674666 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7d41597f0 0x7ff7d40b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.622+0000 7ff7db37c700 1 -- 172.21.15.89:0/3486674666 >> 172.21.15.89:0/3486674666 conn(0x7ff7d40d0660 msgr2=0x7ff7d40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:37.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.622+0000 7ff7db37c700 1 -- 172.21.15.89:0/3486674666 shutdown_connections 2023-10-10T20:41:37.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.622+0000 7ff7db37c700 1 -- 172.21.15.89:0/3486674666 wait complete. 2023-10-10T20:41:37.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.623+0000 7ff7db37c700 1 Processor -- start 2023-10-10T20:41:37.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.623+0000 7ff7db37c700 1 -- start start 2023-10-10T20:41:37.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.624+0000 7ff7db37c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7d4159420 0x7ff7d40c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.624+0000 7ff7db37c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7d41597f0 0x7ff7d40ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.624+0000 7ff7db37c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d415d230 0x7ff7d4145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.624+0000 7ff7db37c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d40c6da0 con 0x7ff7d415d230 2023-10-10T20:41:37.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.624+0000 7ff7db37c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d40c6910 con 0x7ff7d41597f0 2023-10-10T20:41:37.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.624+0000 7ff7db37c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d40c5d00 con 0x7ff7d4159420 2023-10-10T20:41:37.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.624+0000 7ff7dab7b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d415d230 0x7ff7d4145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:37.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.624+0000 7ff7dab7b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d415d230 0x7ff7d4145800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52476/0 (socket says 172.21.15.89:52476) 2023-10-10T20:41:37.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.624+0000 7ff7dab7b700 1 -- 172.21.15.89:0/1891888478 learned_addr learned my addr 172.21.15.89:0/1891888478 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:37.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.624+0000 7ff7dab7b700 1 -- 172.21.15.89:0/1891888478 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7d4159420 msgr2=0x7ff7d40c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:37.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.624+0000 7ff7dab7b700 1 --2- 172.21.15.89:0/1891888478 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7d4159420 0x7ff7d40c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.624+0000 7ff7dab7b700 1 -- 172.21.15.89:0/1891888478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7d41597f0 msgr2=0x7ff7d40ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:37.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.624+0000 7ff7dab7b700 1 --2- 172.21.15.89:0/1891888478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7d41597f0 0x7ff7d40ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.624+0000 7ff7dab7b700 1 -- 172.21.15.89:0/1891888478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc0094f0 con 0x7ff7d415d230 2023-10-10T20:41:37.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.624+0000 7ff7dab7b700 1 --2- 172.21.15.89:0/1891888478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d415d230 0x7ff7d4145800 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000cef0 tx=0x7ff7d0007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:37.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.625+0000 7ff7ca7fc700 1 -- 172.21.15.89:0/1891888478 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d000ae00 con 0x7ff7d415d230 2023-10-10T20:41:37.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.625+0000 7ff7ca7fc700 1 -- 172.21.15.89:0/1891888478 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d0016420 con 0x7ff7d415d230 2023-10-10T20:41:37.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.625+0000 7ff7db37c700 1 -- 172.21.15.89:0/1891888478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d40c5fe0 con 0x7ff7d415d230 2023-10-10T20:41:37.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.625+0000 7ff7ca7fc700 1 -- 172.21.15.89:0/1891888478 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d00167e0 con 0x7ff7d415d230 2023-10-10T20:41:37.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.625+0000 7ff7db37c700 1 -- 172.21.15.89:0/1891888478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d40c65a0 con 0x7ff7d415d230 2023-10-10T20:41:37.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.626+0000 7ff7db37c700 1 -- 172.21.15.89:0/1891888478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d40c6bf0 con 0x7ff7d415d230 2023-10-10T20:41:37.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.626+0000 7ff7ca7fc700 1 -- 172.21.15.89:0/1891888478 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff7d0016b90 con 0x7ff7d415d230 2023-10-10T20:41:37.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.626+0000 7ff7ca7fc700 1 --2- 172.21.15.89:0/1891888478 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff7b8067320 0x7ff7b80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.626+0000 7ff7ca7fc700 1 -- 172.21.15.89:0/1891888478 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5877+0+0 (secure 0 0 0) 0x7ff7d0012070 con 0x7ff7d415d230 2023-10-10T20:41:37.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.626+0000 7ff7da37a700 1 --2- 172.21.15.89:0/1891888478 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff7b8067320 0x7ff7b80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:37.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.627+0000 7ff7da37a700 1 --2- 172.21.15.89:0/1891888478 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff7b8067320 0x7ff7b80697d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40d1720 tx=0x7ff7cc017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:37.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.630+0000 7ff7ca7fc700 1 -- 172.21.15.89:0/1891888478 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7d004e5a0 con 0x7ff7d415d230 2023-10-10T20:41:37.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.815+0000 7ff7db37c700 1 -- 172.21.15.89:0/1891888478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7d404c3c0 con 0x7ff7d415d230 2023-10-10T20:41:37.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.816+0000 7ff7ca7fc700 1 -- 172.21.15.89:0/1891888478 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+15623 (secure 0 0 0) 0x7ff7d0055d10 con 0x7ff7d415d230 2023-10-10T20:41:37.818 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:37.818 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":126,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:41:35.137778+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:41:05.928269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-10T20:41:06.527456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6835/3349522977":"2023-10-11T20:41:32.133265+0000","172.21.15.89:6834/3349522977":"2023-10-11T20:41:32.133265+0000","172.21.15.89:6837/3182024657":"2023-10-11T20:41:28.731530+0000","172.21.15.89:6836/3182024657":"2023-10-11T20:41:28.731530+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:41:37.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.818+0000 7ff7db37c700 1 -- 172.21.15.89:0/1891888478 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff7b8067320 msgr2=0x7ff7b80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:37.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.818+0000 7ff7db37c700 1 --2- 172.21.15.89:0/1891888478 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff7b8067320 0x7ff7b80697d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40d1720 tx=0x7ff7cc017000 comp rx=0 tx=0).stop 2023-10-10T20:41:37.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.818+0000 7ff7db37c700 1 -- 172.21.15.89:0/1891888478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d415d230 msgr2=0x7ff7d4145800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:37.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.818+0000 7ff7db37c700 1 --2- 172.21.15.89:0/1891888478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d415d230 0x7ff7d4145800 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000cef0 tx=0x7ff7d0007860 comp rx=0 tx=0).stop 2023-10-10T20:41:37.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.818+0000 7ff7db37c700 1 -- 172.21.15.89:0/1891888478 shutdown_connections 2023-10-10T20:41:37.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.818+0000 7ff7db37c700 1 --2- 172.21.15.89:0/1891888478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d415d230 0x7ff7d4145800 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.818+0000 7ff7db37c700 1 --2- 172.21.15.89:0/1891888478 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7d4159420 0x7ff7d40c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.818+0000 7ff7db37c700 1 --2- 172.21.15.89:0/1891888478 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff7b8067320 0x7ff7b80697d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.818+0000 7ff7db37c700 1 --2- 172.21.15.89:0/1891888478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7d41597f0 0x7ff7d40ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.818+0000 7ff7db37c700 1 -- 172.21.15.89:0/1891888478 >> 172.21.15.89:0/1891888478 conn(0x7ff7d40d0660 msgr2=0x7ff7d41583f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:37.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.818+0000 7ff7db37c700 1 -- 172.21.15.89:0/1891888478 shutdown_connections 2023-10-10T20:41:37.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.818+0000 7ff7db37c700 1 -- 172.21.15.89:0/1891888478 wait complete. 2023-10-10T20:41:37.832 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:41:37.833 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:41:37.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.944+0000 7f83336a4700 1 Processor -- start 2023-10-10T20:41:37.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.945+0000 7f83336a4700 1 -- start start 2023-10-10T20:41:37.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.945+0000 7f83336a4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f832c0c91e0 0x7f832c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.945+0000 7f83336a4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832c0cc210 con 0x7f832c06ae10 2023-10-10T20:41:37.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.945+0000 7f83336a4700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832c0cc4f0 con 0x7f832c06aa40 2023-10-10T20:41:37.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.945+0000 7f83336a4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0cc7d0 con 0x7f832c0c91e0 2023-10-10T20:41:37.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8331c41700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f832c06ae10 0x7f832c06b2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44856/0 (socket says 172.21.15.89:44856) 2023-10-10T20:41:37.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8331c41700 1 -- 172.21.15.89:0/2299866691 learned_addr learned my addr 172.21.15.89:0/2299866691 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:37.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8330c3f700 1 --2- 172.21.15.89:0/2299866691 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f832c0c91e0 0x7f832c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:37.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8323fff700 1 -- 172.21.15.89:0/2299866691 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1031484901 0 0) 0x7f832c0cc210 con 0x7f832c06ae10 2023-10-10T20:41:37.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8323fff700 1 -- 172.21.15.89:0/2299866691 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8318003570 con 0x7f832c06ae10 2023-10-10T20:41:37.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8323fff700 1 -- 172.21.15.89:0/2299866691 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3046523302 0 0) 0x7f832c0cc4f0 con 0x7f832c06aa40 2023-10-10T20:41:37.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8323fff700 1 -- 172.21.15.89:0/2299866691 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f832c0cc210 con 0x7f832c06aa40 2023-10-10T20:41:37.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8323fff700 1 -- 172.21.15.89:0/2299866691 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3147533030 0 0) 0x7f8318003570 con 0x7f832c06ae10 2023-10-10T20:41:37.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8323fff700 1 -- 172.21.15.89:0/2299866691 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f832c0c91e0 msgr2=0x7f832c0cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:37.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8323fff700 1 --2- 172.21.15.89:0/2299866691 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f832c0c91e0 0x7f832c0cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8323fff700 1 -- 172.21.15.89:0/2299866691 >> v1:172.21.15.111:6789/0 conn(0x7f832c06aa40 legacy=0x7f832c06bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:37.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8323fff700 1 -- 172.21.15.89:0/2299866691 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c0ccab0 con 0x7f832c06ae10 2023-10-10T20:41:37.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8323fff700 1 -- 172.21.15.89:0/2299866691 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f83240017f0 con 0x7f832c06ae10 2023-10-10T20:41:37.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8323fff700 1 -- 172.21.15.89:0/2299866691 >> v1:172.21.15.89:6789/0 conn(0x7f832c06ae10 legacy=0x7f832c06b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:37.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8323fff700 1 --2- 172.21.15.89:0/2299866691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8318003c20 0x7f8318004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8323fff700 1 --2- 172.21.15.89:0/2299866691 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f832c0c91e0 0x7f8318004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8323fff700 1 --2- 172.21.15.89:0/2299866691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8318004d50 0x7f83180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8323fff700 1 -- 172.21.15.89:0/2299866691 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0cc7d0 con 0x7f8318003c20 2023-10-10T20:41:37.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8323fff700 1 -- 172.21.15.89:0/2299866691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831800bbf0 con 0x7f8318004d50 2023-10-10T20:41:37.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8323fff700 1 -- 172.21.15.89:0/2299866691 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831800bdb0 con 0x7f832c0c91e0 2023-10-10T20:41:37.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8331440700 1 --2- 172.21.15.89:0/2299866691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8318003c20 0x7f8318004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:37.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8330c3f700 1 --2- 172.21.15.89:0/2299866691 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f832c0c91e0 0x7f8318004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:37.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.946+0000 7f8331c41700 1 --2- 172.21.15.89:0/2299866691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8318004d50 0x7f83180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:37.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.947+0000 7f8331440700 1 -- 172.21.15.89:0/2299866691 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f832c0c91e0 msgr2=0x7f8318004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:37.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.947+0000 7f8331440700 1 --2- 172.21.15.89:0/2299866691 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f832c0c91e0 0x7f8318004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.947+0000 7f8331440700 1 -- 172.21.15.89:0/2299866691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8318004d50 msgr2=0x7f83180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:37.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.947+0000 7f8331440700 1 --2- 172.21.15.89:0/2299866691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8318004d50 0x7f83180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.947+0000 7f8331440700 1 -- 172.21.15.89:0/2299866691 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8318003570 con 0x7f8318003c20 2023-10-10T20:41:37.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.947+0000 7f8331440700 1 --2- 172.21.15.89:0/2299866691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8318003c20 0x7f8318004010 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f8328007fd0 tx=0x7f8328007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce796e165272ba49 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:37.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.947+0000 7f8323fff700 1 -- 172.21.15.89:0/2299866691 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83280013f0 con 0x7f8318003c20 2023-10-10T20:41:37.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.947+0000 7f8323fff700 1 -- 172.21.15.89:0/2299866691 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832800c5d0 con 0x7f8318003c20 2023-10-10T20:41:37.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.947+0000 7f8323fff700 1 -- 172.21.15.89:0/2299866691 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832800c780 con 0x7f8318003c20 2023-10-10T20:41:37.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.947+0000 7f83336a4700 1 -- 172.21.15.89:0/2299866691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8318003c20 msgr2=0x7f8318004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:37.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.947+0000 7f83336a4700 1 --2- 172.21.15.89:0/2299866691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8318003c20 0x7f8318004010 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f8328007fd0 tx=0x7f8328007d80 comp rx=0 tx=0).stop 2023-10-10T20:41:37.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.947+0000 7f8331c41700 1 -- 172.21.15.89:0/2299866691 reap_dead start 2023-10-10T20:41:37.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.947+0000 7f83336a4700 1 -- 172.21.15.89:0/2299866691 shutdown_connections 2023-10-10T20:41:37.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.947+0000 7f83336a4700 1 -- 172.21.15.89:0/2299866691 >> 172.21.15.89:0/2299866691 conn(0x7f832c0cd670 msgr2=0x7f832c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:37.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.947+0000 7f83336a4700 1 -- 172.21.15.89:0/2299866691 shutdown_connections 2023-10-10T20:41:37.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.947+0000 7f83336a4700 1 -- 172.21.15.89:0/2299866691 wait complete. 2023-10-10T20:41:37.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.948+0000 7f83336a4700 1 Processor -- start 2023-10-10T20:41:37.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.949+0000 7f83336a4700 1 -- start start 2023-10-10T20:41:37.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.949+0000 7f83336a4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8318003c20 0x7f832c14f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.949+0000 7f83336a4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f832c0b1170 0x7f832c146230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.949+0000 7f83336a4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f832c14fd10 0x7f832c1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.949+0000 7f83336a4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0b3290 con 0x7f832c0b1170 2023-10-10T20:41:37.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.949+0000 7f83336a4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c1506f0 con 0x7f8318003c20 2023-10-10T20:41:37.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.949+0000 7f83336a4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c14d830 con 0x7f832c14fd10 2023-10-10T20:41:37.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.949+0000 7f8330c3f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f832c0b1170 0x7f832c146230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:37.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.949+0000 7f8330c3f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f832c0b1170 0x7f832c146230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52486/0 (socket says 172.21.15.89:52486) 2023-10-10T20:41:37.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.949+0000 7f8330c3f700 1 -- 172.21.15.89:0/3226866275 learned_addr learned my addr 172.21.15.89:0/3226866275 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:37.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.950+0000 7f8330c3f700 1 -- 172.21.15.89:0/3226866275 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f832c14fd10 msgr2=0x7f832c1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:37.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.950+0000 7f8330c3f700 1 --2- 172.21.15.89:0/3226866275 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f832c14fd10 0x7f832c1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.950+0000 7f8330c3f700 1 -- 172.21.15.89:0/3226866275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8318003c20 msgr2=0x7f832c14f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:37.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.950+0000 7f8330c3f700 1 --2- 172.21.15.89:0/3226866275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8318003c20 0x7f832c14f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:37.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.950+0000 7f8330c3f700 1 -- 172.21.15.89:0/3226866275 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c0b2cd0 con 0x7f832c0b1170 2023-10-10T20:41:37.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.950+0000 7f8330c3f700 1 --2- 172.21.15.89:0/3226866275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f832c0b1170 0x7f832c146230 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f831c011b40 tx=0x7f831c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:37.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.950+0000 7f83217fa700 1 -- 172.21.15.89:0/3226866275 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c00fa70 con 0x7f832c0b1170 2023-10-10T20:41:37.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.950+0000 7f83217fa700 1 -- 172.21.15.89:0/3226866275 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c00fbd0 con 0x7f832c0b1170 2023-10-10T20:41:37.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.950+0000 7f83336a4700 1 -- 172.21.15.89:0/3226866275 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832c0b2fb0 con 0x7f832c0b1170 2023-10-10T20:41:37.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.950+0000 7f83217fa700 1 -- 172.21.15.89:0/3226866275 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c01d480 con 0x7f832c0b1170 2023-10-10T20:41:37.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.950+0000 7f83336a4700 1 -- 172.21.15.89:0/3226866275 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c0b2910 con 0x7f832c0b1170 2023-10-10T20:41:37.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.951+0000 7f83336a4700 1 -- 172.21.15.89:0/3226866275 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832c14d600 con 0x7f832c0b1170 2023-10-10T20:41:37.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.951+0000 7f83217fa700 1 -- 172.21.15.89:0/3226866275 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f831c01d870 con 0x7f832c0b1170 2023-10-10T20:41:37.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.952+0000 7f83217fa700 1 --2- 172.21.15.89:0/3226866275 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8310067320 0x7f83100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:37.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.952+0000 7f83217fa700 1 -- 172.21.15.89:0/3226866275 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f831c018070 con 0x7f832c0b1170 2023-10-10T20:41:37.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.952+0000 7f8331440700 1 --2- 172.21.15.89:0/3226866275 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8310067320 0x7f83100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:37.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.952+0000 7f8331440700 1 --2- 172.21.15.89:0/3226866275 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8310067320 0x7f83100697d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8328000c00 tx=0x7f8328017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:37.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:37.956+0000 7f83217fa700 1 -- 172.21.15.89:0/3226866275 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f832c14d600 con 0x7f832c0b1170 2023-10-10T20:41:38.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.180+0000 7f83336a4700 1 -- 172.21.15.89:0/3226866275 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f832c14d600 con 0x7f832c0b1170 2023-10-10T20:41:38.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.180+0000 7f83217fa700 1 -- 172.21.15.89:0/3226866275 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f832c14d600 con 0x7f832c0b1170 2023-10-10T20:41:38.182 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:38.183 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6517,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/2658508419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2658508419},{"type":"v1","addr":"172.21.15.111:6834","nonce":2658508419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7431,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1872461145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1872461145},{"type":"v1","addr":"172.21.15.89:6837","nonce":1872461145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3243330447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3243330447},{"type":"v1","addr":"172.21.15.89:6835","nonce":3243330447}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:41:07.565995+0000","modified":"2023-10-10T20:41:11.225905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"d","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/2582059153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2582059153},{"type":"v1","addr":"172.21.15.111:6835","nonce":2582059153}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-10T20:41:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.182+0000 7f83336a4700 1 -- 172.21.15.89:0/3226866275 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8310067320 msgr2=0x7f83100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:38.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.182+0000 7f83336a4700 1 --2- 172.21.15.89:0/3226866275 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8310067320 0x7f83100697d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8328000c00 tx=0x7f8328017000 comp rx=0 tx=0).stop 2023-10-10T20:41:38.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.182+0000 7f83336a4700 1 -- 172.21.15.89:0/3226866275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f832c0b1170 msgr2=0x7f832c146230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:38.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.182+0000 7f83336a4700 1 --2- 172.21.15.89:0/3226866275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f832c0b1170 0x7f832c146230 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f831c011b40 tx=0x7f831c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.182+0000 7f83336a4700 1 -- 172.21.15.89:0/3226866275 shutdown_connections 2023-10-10T20:41:38.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.182+0000 7f83336a4700 1 --2- 172.21.15.89:0/3226866275 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f832c14fd10 0x7f832c1501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.182+0000 7f83336a4700 1 --2- 172.21.15.89:0/3226866275 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8310067320 0x7f83100697d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.182+0000 7f83336a4700 1 --2- 172.21.15.89:0/3226866275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8318003c20 0x7f832c14f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.182+0000 7f83336a4700 1 --2- 172.21.15.89:0/3226866275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f832c0b1170 0x7f832c146230 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.182+0000 7f83336a4700 1 -- 172.21.15.89:0/3226866275 >> 172.21.15.89:0/3226866275 conn(0x7f832c0cd670 msgr2=0x7f832c15b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:38.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.182+0000 7f83336a4700 1 -- 172.21.15.89:0/3226866275 shutdown_connections 2023-10-10T20:41:38.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.182+0000 7f83336a4700 1 -- 172.21.15.89:0/3226866275 wait complete. 2023-10-10T20:41:38.187 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 83 2023-10-10T20:41:38.197 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:41:38.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.309+0000 7f8a07384700 1 Processor -- start 2023-10-10T20:41:38.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.309+0000 7f8a07384700 1 -- start start 2023-10-10T20:41:38.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f8a07384700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a00159c60 0x7f8a0015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:38.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f8a07384700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0015a160 con 0x7f8a00159890 2023-10-10T20:41:38.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f8a07384700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a000d13a0 con 0x7f8a0015db10 2023-10-10T20:41:38.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f8a07384700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0015e140 con 0x7f8a00159c60 2023-10-10T20:41:38.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f8a05120700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8a00159890 0x7f8a0015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53950/0 (socket says 172.21.15.89:53950) 2023-10-10T20:41:38.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f8a05120700 1 -- 172.21.15.89:0/385644049 learned_addr learned my addr 172.21.15.89:0/385644049 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:38.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f89f7fff700 1 -- 172.21.15.89:0/385644049 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395884581 0 0) 0x7f8a0015a160 con 0x7f8a00159890 2023-10-10T20:41:38.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f89f7fff700 1 -- 172.21.15.89:0/385644049 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89ec003570 con 0x7f8a00159890 2023-10-10T20:41:38.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f8a0491f700 1 --2- 172.21.15.89:0/385644049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a00159c60 0x7f8a0015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:38.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f89f7fff700 1 -- 172.21.15.89:0/385644049 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4249909402 0 0) 0x7f89ec003570 con 0x7f8a00159890 2023-10-10T20:41:38.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f89f7fff700 1 -- 172.21.15.89:0/385644049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a00159c60 msgr2=0x7f8a0015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:38.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f89f7fff700 1 --2- 172.21.15.89:0/385644049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a00159c60 0x7f8a0015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f89f7fff700 1 -- 172.21.15.89:0/385644049 >> v1:172.21.15.111:6789/0 conn(0x7f8a0015db10 legacy=0x7f8a00069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:38.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f89f7fff700 1 -- 172.21.15.89:0/385644049 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0015a3d0 con 0x7f8a00159890 2023-10-10T20:41:38.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f89f7fff700 1 -- 172.21.15.89:0/385644049 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f89fc001470 con 0x7f8a00159890 2023-10-10T20:41:38.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f89f7fff700 1 -- 172.21.15.89:0/385644049 >> v1:172.21.15.89:6789/0 conn(0x7f8a00159890 legacy=0x7f8a0015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:38.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f89f7fff700 1 --2- 172.21.15.89:0/385644049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a00159c60 0x7f89ec003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:38.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f89f7fff700 1 --2- 172.21.15.89:0/385644049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89ec004010 0x7f89ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:38.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f89f7fff700 1 --2- 172.21.15.89:0/385644049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89ec004d50 0x7f89ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:38.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f89f7fff700 1 -- 172.21.15.89:0/385644049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0015e140 con 0x7f89ec004010 2023-10-10T20:41:38.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f89f7fff700 1 -- 172.21.15.89:0/385644049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec00bbf0 con 0x7f89ec004d50 2023-10-10T20:41:38.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.310+0000 7f89f7fff700 1 -- 172.21.15.89:0/385644049 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec00bdb0 con 0x7f8a00159c60 2023-10-10T20:41:38.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.311+0000 7f8a0491f700 1 --2- 172.21.15.89:0/385644049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89ec004010 0x7f89ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:38.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.311+0000 7f8a05120700 1 --2- 172.21.15.89:0/385644049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a00159c60 0x7f89ec003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:38.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.311+0000 7f8a05921700 1 --2- 172.21.15.89:0/385644049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89ec004d50 0x7f89ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:38.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.311+0000 7f8a0491f700 1 -- 172.21.15.89:0/385644049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a00159c60 msgr2=0x7f89ec003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:38.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.311+0000 7f8a0491f700 1 --2- 172.21.15.89:0/385644049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a00159c60 0x7f89ec003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.311+0000 7f8a0491f700 1 -- 172.21.15.89:0/385644049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89ec004d50 msgr2=0x7f89ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:38.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.311+0000 7f8a0491f700 1 --2- 172.21.15.89:0/385644049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89ec004d50 0x7f89ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.311+0000 7f8a0491f700 1 -- 172.21.15.89:0/385644049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ec003570 con 0x7f89ec004010 2023-10-10T20:41:38.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.311+0000 7f8a0491f700 1 --2- 172.21.15.89:0/385644049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89ec004010 0x7f89ec004810 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f89f0006f00 tx=0x7f89f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a44362adfa9254 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:38.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.311+0000 7f89f7fff700 1 -- 172.21.15.89:0/385644049 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f000e6e0 con 0x7f89ec004010 2023-10-10T20:41:38.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.311+0000 7f89f7fff700 1 -- 172.21.15.89:0/385644049 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f000e840 con 0x7f89ec004010 2023-10-10T20:41:38.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.311+0000 7f89f7fff700 1 -- 172.21.15.89:0/385644049 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f000ec00 con 0x7f89ec004010 2023-10-10T20:41:38.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.312+0000 7f8a07384700 1 -- 172.21.15.89:0/385644049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89ec004010 msgr2=0x7f89ec004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:38.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.312+0000 7f8a07384700 1 --2- 172.21.15.89:0/385644049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89ec004010 0x7f89ec004810 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f89f0006f00 tx=0x7f89f00091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.312+0000 7f8a05921700 1 -- 172.21.15.89:0/385644049 reap_dead start 2023-10-10T20:41:38.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.312+0000 7f8a07384700 1 -- 172.21.15.89:0/385644049 shutdown_connections 2023-10-10T20:41:38.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.312+0000 7f8a07384700 1 -- 172.21.15.89:0/385644049 >> 172.21.15.89:0/385644049 conn(0x7f8a00068e20 msgr2=0x7f8a000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:38.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.312+0000 7f8a07384700 1 -- 172.21.15.89:0/385644049 shutdown_connections 2023-10-10T20:41:38.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.312+0000 7f8a07384700 1 -- 172.21.15.89:0/385644049 wait complete. 2023-10-10T20:41:38.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.313+0000 7f8a07384700 1 Processor -- start 2023-10-10T20:41:38.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.313+0000 7f8a07384700 1 -- start start 2023-10-10T20:41:38.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f8a07384700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89ec004010 0x7f8a000cea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:38.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f8a07384700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a0015b4d0 0x7f8a0015b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:38.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f8a07384700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a000b3d30 0x7f8a000b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:38.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f8a07384700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0015c600 con 0x7f8a000b3d30 2023-10-10T20:41:38.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f8a07384700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0015d780 con 0x7f89ec004010 2023-10-10T20:41:38.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f8a07384700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0015c040 con 0x7f8a0015b4d0 2023-10-10T20:41:38.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f8a05921700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a000b3d30 0x7f8a000b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:38.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f8a05921700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a000b3d30 0x7f8a000b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38260/0 (socket says 172.21.15.89:38260) 2023-10-10T20:41:38.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f8a05921700 1 -- 172.21.15.89:0/3453164291 learned_addr learned my addr 172.21.15.89:0/3453164291 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:38.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f8a05120700 1 --2- 172.21.15.89:0/3453164291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89ec004010 0x7f8a000cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:38.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f8a0491f700 1 --2- 172.21.15.89:0/3453164291 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a0015b4d0 0x7f8a0015b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:38.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f8a05921700 1 -- 172.21.15.89:0/3453164291 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a0015b4d0 msgr2=0x7f8a0015b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:38.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f8a05921700 1 --2- 172.21.15.89:0/3453164291 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a0015b4d0 0x7f8a0015b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f8a05921700 1 -- 172.21.15.89:0/3453164291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89ec004010 msgr2=0x7f8a000cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:38.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f8a05921700 1 --2- 172.21.15.89:0/3453164291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89ec004010 0x7f8a000cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f8a05921700 1 -- 172.21.15.89:0/3453164291 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0015c320 con 0x7f8a000b3d30 2023-10-10T20:41:38.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f8a05921700 1 --2- 172.21.15.89:0/3453164291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a000b3d30 0x7f8a000b41d0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f89f8007fd0 tx=0x7f89f8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:38.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f89f57fa700 1 -- 172.21.15.89:0/3453164291 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f80032e0 con 0x7f8a000b3d30 2023-10-10T20:41:38.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f89f57fa700 1 -- 172.21.15.89:0/3453164291 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f8001580 con 0x7f8a000b3d30 2023-10-10T20:41:38.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f8a07384700 1 -- 172.21.15.89:0/3453164291 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a000cd260 con 0x7f8a000b3d30 2023-10-10T20:41:38.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f89f57fa700 1 -- 172.21.15.89:0/3453164291 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8001940 con 0x7f8a000b3d30 2023-10-10T20:41:38.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.314+0000 7f8a07384700 1 -- 172.21.15.89:0/3453164291 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a000cef60 con 0x7f8a000b3d30 2023-10-10T20:41:38.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.315+0000 7f8a07384700 1 -- 172.21.15.89:0/3453164291 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a000cd0c0 con 0x7f8a000b3d30 2023-10-10T20:41:38.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.316+0000 7f89f57fa700 1 -- 172.21.15.89:0/3453164291 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f89f8001cb0 con 0x7f8a000b3d30 2023-10-10T20:41:38.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.316+0000 7f89f57fa700 1 --2- 172.21.15.89:0/3453164291 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f89e4067320 0x7f89e40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:38.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.316+0000 7f8a05120700 1 --2- 172.21.15.89:0/3453164291 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f89e4067320 0x7f89e40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:38.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.316+0000 7f89f57fa700 1 -- 172.21.15.89:0/3453164291 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f89f8084370 con 0x7f8a000b3d30 2023-10-10T20:41:38.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.317+0000 7f8a05120700 1 --2- 172.21.15.89:0/3453164291 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f89e4067320 0x7f89e40697d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f89fc000e50 tx=0x7f89fc001e50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:38.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.321+0000 7f89f57fa700 1 -- 172.21.15.89:0/3453164291 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a000cd0c0 con 0x7f8a000b3d30 2023-10-10T20:41:38.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.545+0000 7f8a07384700 1 -- 172.21.15.89:0/3453164291 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a000cd0c0 con 0x7f8a000b3d30 2023-10-10T20:41:38.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.546+0000 7f89f57fa700 1 -- 172.21.15.89:0/3453164291 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f8a000cd0c0 con 0x7f8a000b3d30 2023-10-10T20:41:38.549 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:38.549 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6517,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/2658508419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2658508419},{"type":"v1","addr":"172.21.15.111:6834","nonce":2658508419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7431,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1872461145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1872461145},{"type":"v1","addr":"172.21.15.89:6837","nonce":1872461145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3243330447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3243330447},{"type":"v1","addr":"172.21.15.89:6835","nonce":3243330447}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:41:07.565995+0000","modified":"2023-10-10T20:41:11.225905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"d","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/2582059153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2582059153},{"type":"v1","addr":"172.21.15.111:6835","nonce":2582059153}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-10T20:41:38.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.548+0000 7f8a07384700 1 -- 172.21.15.89:0/3453164291 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f89e4067320 msgr2=0x7f89e40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.548+0000 7f8a07384700 1 --2- 172.21.15.89:0/3453164291 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f89e4067320 0x7f89e40697d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f89fc000e50 tx=0x7f89fc001e50 comp rx=0 tx=0).stop 2023-10-10T20:41:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.548+0000 7f8a07384700 1 -- 172.21.15.89:0/3453164291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a000b3d30 msgr2=0x7f8a000b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.548+0000 7f8a07384700 1 --2- 172.21.15.89:0/3453164291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a000b3d30 0x7f8a000b41d0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f89f8007fd0 tx=0x7f89f8009040 comp rx=0 tx=0).stop 2023-10-10T20:41:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.548+0000 7f8a07384700 1 -- 172.21.15.89:0/3453164291 shutdown_connections 2023-10-10T20:41:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.548+0000 7f8a07384700 1 --2- 172.21.15.89:0/3453164291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a000b3d30 0x7f8a000b41d0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.548+0000 7f8a07384700 1 --2- 172.21.15.89:0/3453164291 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f89e4067320 0x7f89e40697d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.548+0000 7f8a07384700 1 --2- 172.21.15.89:0/3453164291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89ec004010 0x7f8a000cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.548+0000 7f8a07384700 1 --2- 172.21.15.89:0/3453164291 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a0015b4d0 0x7f8a0015b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.548+0000 7f8a07384700 1 -- 172.21.15.89:0/3453164291 >> 172.21.15.89:0/3453164291 conn(0x7f8a00068e20 msgr2=0x7f8a00154280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.548+0000 7f8a07384700 1 -- 172.21.15.89:0/3453164291 shutdown_connections 2023-10-10T20:41:38.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.548+0000 7f8a07384700 1 -- 172.21.15.89:0/3453164291 wait complete. 2023-10-10T20:41:38.553 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 83 2023-10-10T20:41:38.562 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:41:38.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.674+0000 7fccfeb45700 1 Processor -- start 2023-10-10T20:41:38.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.674+0000 7fccfeb45700 1 -- start start 2023-10-10T20:41:38.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.675+0000 7fccfeb45700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccf8159c60 0x7fccf815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:38.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.675+0000 7fccfeb45700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf815a160 con 0x7fccf8159890 2023-10-10T20:41:38.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.675+0000 7fccfeb45700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf80d13a0 con 0x7fccf815db10 2023-10-10T20:41:38.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.675+0000 7fccfeb45700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf815e140 con 0x7fccf8159c60 2023-10-10T20:41:38.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.675+0000 7fccfc8e1700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fccf8159890 0x7fccf815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53952/0 (socket says 172.21.15.89:53952) 2023-10-10T20:41:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.675+0000 7fccfc8e1700 1 -- 172.21.15.89:0/596847667 learned_addr learned my addr 172.21.15.89:0/596847667 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.675+0000 7fcceffff700 1 --2- 172.21.15.89:0/596847667 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccf8159c60 0x7fccf815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccef7fe700 1 -- 172.21.15.89:0/596847667 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961703814 0 0) 0x7fccf815a160 con 0x7fccf8159890 2023-10-10T20:41:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccef7fe700 1 -- 172.21.15.89:0/596847667 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcce0003570 con 0x7fccf8159890 2023-10-10T20:41:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccef7fe700 1 -- 172.21.15.89:0/596847667 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3100268581 0 0) 0x7fcce0003570 con 0x7fccf8159890 2023-10-10T20:41:38.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccef7fe700 1 -- 172.21.15.89:0/596847667 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccf8159c60 msgr2=0x7fccf815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:38.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccef7fe700 1 --2- 172.21.15.89:0/596847667 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccf8159c60 0x7fccf815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccef7fe700 1 -- 172.21.15.89:0/596847667 >> v1:172.21.15.111:6789/0 conn(0x7fccf815db10 legacy=0x7fccf8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:38.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccef7fe700 1 -- 172.21.15.89:0/596847667 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf815a3d0 con 0x7fccf8159890 2023-10-10T20:41:38.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccef7fe700 1 -- 172.21.15.89:0/596847667 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fccf4001470 con 0x7fccf8159890 2023-10-10T20:41:38.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccef7fe700 1 -- 172.21.15.89:0/596847667 >> v1:172.21.15.89:6789/0 conn(0x7fccf8159890 legacy=0x7fccf815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:38.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccef7fe700 1 --2- 172.21.15.89:0/596847667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcce0003a90 0x7fcce0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:38.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccef7fe700 1 --2- 172.21.15.89:0/596847667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcce0004810 0x7fcce0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:38.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccef7fe700 1 --2- 172.21.15.89:0/596847667 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcce00051c0 0x7fcce000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:38.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccef7fe700 1 -- 172.21.15.89:0/596847667 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf815e140 con 0x7fcce0003a90 2023-10-10T20:41:38.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccef7fe700 1 -- 172.21.15.89:0/596847667 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce000bfa0 con 0x7fcce0004810 2023-10-10T20:41:38.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccef7fe700 1 -- 172.21.15.89:0/596847667 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce000c160 con 0x7fcce00051c0 2023-10-10T20:41:38.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fcceffff700 1 --2- 172.21.15.89:0/596847667 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccf8159c60 0x7fccf815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:41:38.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccfc8e1700 1 --2- 172.21.15.89:0/596847667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcce0003a90 0x7fcce0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:38.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccfd0e2700 1 --2- 172.21.15.89:0/596847667 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcce00051c0 0x7fcce000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:38.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fcceffff700 1 --2- 172.21.15.89:0/596847667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcce0004810 0x7fcce0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:38.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccfc8e1700 1 -- 172.21.15.89:0/596847667 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcce00051c0 msgr2=0x7fcce000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:38.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccfc8e1700 1 --2- 172.21.15.89:0/596847667 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcce00051c0 0x7fcce000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccfc8e1700 1 -- 172.21.15.89:0/596847667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcce0004810 msgr2=0x7fcce0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:38.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccfc8e1700 1 --2- 172.21.15.89:0/596847667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcce0004810 0x7fcce0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccfc8e1700 1 -- 172.21.15.89:0/596847667 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce0003570 con 0x7fcce0003a90 2023-10-10T20:41:38.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fcceffff700 1 --2- 172.21.15.89:0/596847667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcce0004810 0x7fcce0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:38.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.676+0000 7fccfc8e1700 1 --2- 172.21.15.89:0/596847667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcce0003a90 0x7fcce0003e60 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fccf400c3c0 tx=0x7fccf4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b480df9b9faffbca server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:38.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.677+0000 7fccef7fe700 1 -- 172.21.15.89:0/596847667 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf400cc90 con 0x7fcce0003a90 2023-10-10T20:41:38.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.677+0000 7fccef7fe700 1 -- 172.21.15.89:0/596847667 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf400cdf0 con 0x7fcce0003a90 2023-10-10T20:41:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.677+0000 7fccef7fe700 1 -- 172.21.15.89:0/596847667 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf4005b70 con 0x7fcce0003a90 2023-10-10T20:41:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.677+0000 7fccfeb45700 1 -- 172.21.15.89:0/596847667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcce0003a90 msgr2=0x7fcce0003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.677+0000 7fccfeb45700 1 --2- 172.21.15.89:0/596847667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcce0003a90 0x7fcce0003e60 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fccf400c3c0 tx=0x7fccf4001e70 comp rx=0 tx=0).stop 2023-10-10T20:41:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.677+0000 7fccfd0e2700 1 -- 172.21.15.89:0/596847667 reap_dead start 2023-10-10T20:41:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.677+0000 7fccfeb45700 1 -- 172.21.15.89:0/596847667 shutdown_connections 2023-10-10T20:41:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.677+0000 7fccfeb45700 1 -- 172.21.15.89:0/596847667 >> 172.21.15.89:0/596847667 conn(0x7fccf8068e20 msgr2=0x7fccf80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.677+0000 7fccfeb45700 1 -- 172.21.15.89:0/596847667 shutdown_connections 2023-10-10T20:41:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.677+0000 7fccfeb45700 1 -- 172.21.15.89:0/596847667 wait complete. 2023-10-10T20:41:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.678+0000 7fccfeb45700 1 Processor -- start 2023-10-10T20:41:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.678+0000 7fccfeb45700 1 -- start start 2023-10-10T20:41:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.679+0000 7fccfeb45700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcce0003a90 0x7fccf80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:38.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.679+0000 7fccfeb45700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf80b3f70 0x7fccf814be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:38.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.679+0000 7fccfeb45700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccf80cebd0 0x7fccf80cf050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:38.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.679+0000 7fccfeb45700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf80b6480 con 0x7fccf80b3f70 2023-10-10T20:41:38.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.679+0000 7fccfeb45700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf80b4750 con 0x7fccf80cebd0 2023-10-10T20:41:38.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.679+0000 7fccfeb45700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf80b3d80 con 0x7fcce0003a90 2023-10-10T20:41:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.679+0000 7fcceffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf80b3f70 0x7fccf814be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.679+0000 7fcceffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf80b3f70 0x7fccf814be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38272/0 (socket says 172.21.15.89:38272) 2023-10-10T20:41:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.679+0000 7fcceffff700 1 -- 172.21.15.89:0/458870563 learned_addr learned my addr 172.21.15.89:0/458870563 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.679+0000 7fccfc8e1700 1 --2- 172.21.15.89:0/458870563 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcce0003a90 0x7fccf80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.679+0000 7fccfd0e2700 1 --2- 172.21.15.89:0/458870563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccf80cebd0 0x7fccf80cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:38.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.679+0000 7fcceffff700 1 -- 172.21.15.89:0/458870563 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcce0003a90 msgr2=0x7fccf80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:38.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.679+0000 7fcceffff700 1 --2- 172.21.15.89:0/458870563 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcce0003a90 0x7fccf80b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.679+0000 7fcceffff700 1 -- 172.21.15.89:0/458870563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccf80cebd0 msgr2=0x7fccf80cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:38.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.679+0000 7fcceffff700 1 --2- 172.21.15.89:0/458870563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccf80cebd0 0x7fccf80cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.679+0000 7fcceffff700 1 -- 172.21.15.89:0/458870563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf814c3b0 con 0x7fccf80b3f70 2023-10-10T20:41:38.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.680+0000 7fcceffff700 1 --2- 172.21.15.89:0/458870563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf80b3f70 0x7fccf814be70 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fcce8010cb0 tx=0x7fcce8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:38.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.680+0000 7fccecff9700 1 -- 172.21.15.89:0/458870563 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce800e870 con 0x7fccf80b3f70 2023-10-10T20:41:38.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.680+0000 7fccecff9700 1 -- 172.21.15.89:0/458870563 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce800e9d0 con 0x7fccf80b3f70 2023-10-10T20:41:38.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.680+0000 7fccfeb45700 1 -- 172.21.15.89:0/458870563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf80cf590 con 0x7fccf80b3f70 2023-10-10T20:41:38.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.680+0000 7fccecff9700 1 -- 172.21.15.89:0/458870563 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce800ed50 con 0x7fccf80b3f70 2023-10-10T20:41:38.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.680+0000 7fccfeb45700 1 -- 172.21.15.89:0/458870563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf80b3730 con 0x7fccf80b3f70 2023-10-10T20:41:38.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.681+0000 7fccfeb45700 1 -- 172.21.15.89:0/458870563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf80b3590 con 0x7fccf80b3f70 2023-10-10T20:41:38.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.681+0000 7fccecff9700 1 -- 172.21.15.89:0/458870563 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcce80143f0 con 0x7fccf80b3f70 2023-10-10T20:41:38.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.682+0000 7fccecff9700 1 --2- 172.21.15.89:0/458870563 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcce4067320 0x7fcce40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:38.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.682+0000 7fccecff9700 1 -- 172.21.15.89:0/458870563 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5877+0+0 (secure 0 0 0) 0x7fcce8085010 con 0x7fccf80b3f70 2023-10-10T20:41:38.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.682+0000 7fccfc8e1700 1 --2- 172.21.15.89:0/458870563 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcce4067320 0x7fcce40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:38.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.682+0000 7fccfc8e1700 1 --2- 172.21.15.89:0/458870563 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcce4067320 0x7fcce40697d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fccf81532e0 tx=0x7fccf4015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:38.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.686+0000 7fccecff9700 1 -- 172.21.15.89:0/458870563 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fccf80b3590 con 0x7fccf80b3f70 2023-10-10T20:41:38.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.872+0000 7fccfeb45700 1 -- 172.21.15.89:0/458870563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccf80b3590 con 0x7fccf80b3f70 2023-10-10T20:41:38.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.872+0000 7fccecff9700 1 -- 172.21.15.89:0/458870563 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+15623 (secure 0 0 0) 0x7fccf80b3590 con 0x7fccf80b3f70 2023-10-10T20:41:38.875 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:38.875 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":126,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:41:35.137778+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:41:05.928269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-10T20:41:06.527456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6835/3349522977":"2023-10-11T20:41:32.133265+0000","172.21.15.89:6834/3349522977":"2023-10-11T20:41:32.133265+0000","172.21.15.89:6837/3182024657":"2023-10-11T20:41:28.731530+0000","172.21.15.89:6836/3182024657":"2023-10-11T20:41:28.731530+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:41:38.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.874+0000 7fccfeb45700 1 -- 172.21.15.89:0/458870563 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcce4067320 msgr2=0x7fcce40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:38.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.874+0000 7fccfeb45700 1 --2- 172.21.15.89:0/458870563 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcce4067320 0x7fcce40697d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fccf81532e0 tx=0x7fccf4015000 comp rx=0 tx=0).stop 2023-10-10T20:41:38.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.874+0000 7fccfeb45700 1 -- 172.21.15.89:0/458870563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf80b3f70 msgr2=0x7fccf814be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:38.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.874+0000 7fccfeb45700 1 --2- 172.21.15.89:0/458870563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf80b3f70 0x7fccf814be70 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fcce8010cb0 tx=0x7fcce8008040 comp rx=0 tx=0).stop 2023-10-10T20:41:38.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.874+0000 7fccfeb45700 1 -- 172.21.15.89:0/458870563 shutdown_connections 2023-10-10T20:41:38.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.874+0000 7fccfeb45700 1 --2- 172.21.15.89:0/458870563 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcce4067320 0x7fcce40697d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.874+0000 7fccfeb45700 1 --2- 172.21.15.89:0/458870563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccf80cebd0 0x7fccf80cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.874+0000 7fccfeb45700 1 --2- 172.21.15.89:0/458870563 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcce0003a90 0x7fccf80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.874+0000 7fccfeb45700 1 --2- 172.21.15.89:0/458870563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf80b3f70 0x7fccf814be70 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:38.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.874+0000 7fccfeb45700 1 -- 172.21.15.89:0/458870563 >> 172.21.15.89:0/458870563 conn(0x7fccf8068e20 msgr2=0x7fccf80d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:38.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.875+0000 7fccfeb45700 1 -- 172.21.15.89:0/458870563 shutdown_connections 2023-10-10T20:41:38.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:38.875+0000 7fccfeb45700 1 -- 172.21.15.89:0/458870563 wait complete. 2023-10-10T20:41:38.889 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:41:39.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.002+0000 7faf9759e700 1 Processor -- start 2023-10-10T20:41:39.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.002+0000 7faf9759e700 1 -- start start 2023-10-10T20:41:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9759e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faf9815d230 0x7faf9815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9759e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf980b2980 con 0x7faf98159420 2023-10-10T20:41:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9759e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9815dbb0 con 0x7faf981597f0 2023-10-10T20:41:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9759e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf980b11d0 con 0x7faf9815d230 2023-10-10T20:41:39.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9659c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7faf98159420 0x7faf980b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53966/0 (socket says 172.21.15.89:53966) 2023-10-10T20:41:39.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9659c700 1 -- 172.21.15.89:0/1219117165 learned_addr learned my addr 172.21.15.89:0/1219117165 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:39.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf96d9d700 1 --2- 172.21.15.89:0/1219117165 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faf9815d230 0x7faf9815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:39.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9559a700 1 -- 172.21.15.89:0/1219117165 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745950544 0 0) 0x7faf980b2980 con 0x7faf98159420 2023-10-10T20:41:39.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9559a700 1 -- 172.21.15.89:0/1219117165 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf84003570 con 0x7faf98159420 2023-10-10T20:41:39.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9559a700 1 -- 172.21.15.89:0/1219117165 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 717970673 0 0) 0x7faf84003570 con 0x7faf98159420 2023-10-10T20:41:39.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9559a700 1 -- 172.21.15.89:0/1219117165 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faf9815d230 msgr2=0x7faf9815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:39.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9559a700 1 --2- 172.21.15.89:0/1219117165 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faf9815d230 0x7faf9815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:39.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9559a700 1 -- 172.21.15.89:0/1219117165 >> v1:172.21.15.111:6789/0 conn(0x7faf981597f0 legacy=0x7faf980b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9559a700 1 -- 172.21.15.89:0/1219117165 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf980b14b0 con 0x7faf98159420 2023-10-10T20:41:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9559a700 1 -- 172.21.15.89:0/1219117165 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7faf900017f0 con 0x7faf98159420 2023-10-10T20:41:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9559a700 1 -- 172.21.15.89:0/1219117165 >> v1:172.21.15.89:6789/0 conn(0x7faf98159420 legacy=0x7faf980b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9559a700 1 --2- 172.21.15.89:0/1219117165 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faf9815d230 0x7faf84003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9559a700 1 --2- 172.21.15.89:0/1219117165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf84004010 0x7faf84004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9559a700 1 --2- 172.21.15.89:0/1219117165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf84004d50 0x7faf840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9559a700 1 -- 172.21.15.89:0/1219117165 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf980b11d0 con 0x7faf84004010 2023-10-10T20:41:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9559a700 1 -- 172.21.15.89:0/1219117165 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8400bbf0 con 0x7faf84004d50 2023-10-10T20:41:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.003+0000 7faf9559a700 1 -- 172.21.15.89:0/1219117165 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8400bdb0 con 0x7faf9815d230 2023-10-10T20:41:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.004+0000 7faf95d9b700 1 --2- 172.21.15.89:0/1219117165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf84004010 0x7faf84004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.004+0000 7faf96d9d700 1 --2- 172.21.15.89:0/1219117165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf84004d50 0x7faf840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.004+0000 7faf9659c700 1 --2- 172.21.15.89:0/1219117165 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faf9815d230 0x7faf84003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.004+0000 7faf95d9b700 1 -- 172.21.15.89:0/1219117165 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faf9815d230 msgr2=0x7faf84003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.004+0000 7faf95d9b700 1 --2- 172.21.15.89:0/1219117165 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faf9815d230 0x7faf84003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.004+0000 7faf95d9b700 1 -- 172.21.15.89:0/1219117165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf84004d50 msgr2=0x7faf840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.004+0000 7faf95d9b700 1 --2- 172.21.15.89:0/1219117165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf84004d50 0x7faf840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:39.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.004+0000 7faf95d9b700 1 -- 172.21.15.89:0/1219117165 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf84003570 con 0x7faf84004010 2023-10-10T20:41:39.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.004+0000 7faf95d9b700 1 --2- 172.21.15.89:0/1219117165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf84004010 0x7faf84004810 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7faf98159c70 tx=0x7faf80007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14cdfa5d6adff2a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:39.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.004+0000 7faf9559a700 1 -- 172.21.15.89:0/1219117165 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf80012420 con 0x7faf84004010 2023-10-10T20:41:39.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.004+0000 7faf9559a700 1 -- 172.21.15.89:0/1219117165 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf80012580 con 0x7faf84004010 2023-10-10T20:41:39.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.004+0000 7faf9559a700 1 -- 172.21.15.89:0/1219117165 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf80012920 con 0x7faf84004010 2023-10-10T20:41:39.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.005+0000 7faf9759e700 1 -- 172.21.15.89:0/1219117165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf84004010 msgr2=0x7faf84004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:39.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.005+0000 7faf9759e700 1 --2- 172.21.15.89:0/1219117165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf84004010 0x7faf84004810 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7faf98159c70 tx=0x7faf80007d30 comp rx=0 tx=0).stop 2023-10-10T20:41:39.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.005+0000 7faf96d9d700 1 -- 172.21.15.89:0/1219117165 reap_dead start 2023-10-10T20:41:39.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.005+0000 7faf9759e700 1 -- 172.21.15.89:0/1219117165 shutdown_connections 2023-10-10T20:41:39.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.005+0000 7faf9759e700 1 -- 172.21.15.89:0/1219117165 >> 172.21.15.89:0/1219117165 conn(0x7faf980d0660 msgr2=0x7faf980d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:39.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.005+0000 7faf9759e700 1 -- 172.21.15.89:0/1219117165 shutdown_connections 2023-10-10T20:41:39.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.005+0000 7faf9759e700 1 -- 172.21.15.89:0/1219117165 wait complete. 2023-10-10T20:41:39.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.006+0000 7faf9759e700 1 Processor -- start 2023-10-10T20:41:39.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.006+0000 7faf9759e700 1 -- start start 2023-10-10T20:41:39.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf9759e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faf84004010 0x7faf98155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:39.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf9759e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf98153f70 0x7faf981543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:39.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf9759e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf980ca1a0 0x7faf980ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:39.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf9759e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf980c5490 con 0x7faf98153f70 2023-10-10T20:41:39.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf9759e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf98154920 con 0x7faf980ca1a0 2023-10-10T20:41:39.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf9759e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf980caed0 con 0x7faf84004010 2023-10-10T20:41:39.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf95d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf98153f70 0x7faf981543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:39.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf95d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf98153f70 0x7faf981543e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38302/0 (socket says 172.21.15.89:38302) 2023-10-10T20:41:39.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf95d9b700 1 -- 172.21.15.89:0/4217836625 learned_addr learned my addr 172.21.15.89:0/4217836625 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:39.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf9659c700 1 --2- 172.21.15.89:0/4217836625 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faf84004010 0x7faf98155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:39.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf96d9d700 1 --2- 172.21.15.89:0/4217836625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf980ca1a0 0x7faf980ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:39.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf95d9b700 1 -- 172.21.15.89:0/4217836625 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faf84004010 msgr2=0x7faf98155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:39.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf95d9b700 1 --2- 172.21.15.89:0/4217836625 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faf84004010 0x7faf98155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:39.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf95d9b700 1 -- 172.21.15.89:0/4217836625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf980ca1a0 msgr2=0x7faf980ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:39.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf95d9b700 1 --2- 172.21.15.89:0/4217836625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf980ca1a0 0x7faf980ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:39.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf95d9b700 1 -- 172.21.15.89:0/4217836625 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf980cb1b0 con 0x7faf98153f70 2023-10-10T20:41:39.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf96d9d700 1 --2- 172.21.15.89:0/4217836625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf980ca1a0 0x7faf980ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:39.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf95d9b700 1 --2- 172.21.15.89:0/4217836625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf98153f70 0x7faf981543e0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7faf840057d0 tx=0x7faf80015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:39.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf8e7fc700 1 -- 172.21.15.89:0/4217836625 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8001e6e0 con 0x7faf98153f70 2023-10-10T20:41:39.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf8e7fc700 1 -- 172.21.15.89:0/4217836625 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8001e860 con 0x7faf98153f70 2023-10-10T20:41:39.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf8e7fc700 1 -- 172.21.15.89:0/4217836625 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8001ec20 con 0x7faf98153f70 2023-10-10T20:41:39.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf9759e700 1 -- 172.21.15.89:0/4217836625 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf980cb490 con 0x7faf98153f70 2023-10-10T20:41:39.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.007+0000 7faf9759e700 1 -- 172.21.15.89:0/4217836625 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf980cab30 con 0x7faf98153f70 2023-10-10T20:41:39.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.008+0000 7faf9759e700 1 -- 172.21.15.89:0/4217836625 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9804c3c0 con 0x7faf98153f70 2023-10-10T20:41:39.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.009+0000 7faf8e7fc700 1 -- 172.21.15.89:0/4217836625 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7faf80019030 con 0x7faf98153f70 2023-10-10T20:41:39.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.009+0000 7faf8e7fc700 1 --2- 172.21.15.89:0/4217836625 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faf78067320 0x7faf780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:39.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.009+0000 7faf8e7fc700 1 -- 172.21.15.89:0/4217836625 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5877+0+0 (secure 0 0 0) 0x7faf80086170 con 0x7faf98153f70 2023-10-10T20:41:39.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.009+0000 7faf9659c700 1 --2- 172.21.15.89:0/4217836625 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faf78067320 0x7faf780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:39.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.009+0000 7faf9659c700 1 --2- 172.21.15.89:0/4217836625 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faf78067320 0x7faf780697d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7faf90004ea0 tx=0x7faf90006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:39.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.013+0000 7faf8e7fc700 1 -- 172.21.15.89:0/4217836625 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf80052d50 con 0x7faf98153f70 2023-10-10T20:41:39.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.243+0000 7faf9759e700 1 -- 172.21.15.89:0/4217836625 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7faf980b1790 con 0x7faf98153f70 2023-10-10T20:41:39.246 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-10-10T20:41:39.244+0000 7fd0184ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:41:39.251 INFO:tasks.ceph.mds.d.smithi111.stderr: -2664> 2023-10-10T20:41:03.070+0000 7f7241b48700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-10T20:41:39.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.252+0000 7faf8e7fc700 1 -- 172.21.15.89:0/4217836625 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7faf80056370 con 0x7faf98153f70 2023-10-10T20:41:39.254 INFO:tasks.ceph.mds.d.smithi111.stderr: -1469> 2023-10-10T20:41:13.471+0000 7f723bb3c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:41:39.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.254+0000 7faf9759e700 1 -- 172.21.15.89:0/4217836625 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faf78067320 msgr2=0x7faf780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:39.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.254+0000 7faf9759e700 1 --2- 172.21.15.89:0/4217836625 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faf78067320 0x7faf780697d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7faf90004ea0 tx=0x7faf90006040 comp rx=0 tx=0).stop 2023-10-10T20:41:39.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.254+0000 7faf9759e700 1 -- 172.21.15.89:0/4217836625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf98153f70 msgr2=0x7faf981543e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:39.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.254+0000 7faf9759e700 1 --2- 172.21.15.89:0/4217836625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf98153f70 0x7faf981543e0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7faf840057d0 tx=0x7faf80015000 comp rx=0 tx=0).stop 2023-10-10T20:41:39.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.254+0000 7faf9759e700 1 -- 172.21.15.89:0/4217836625 shutdown_connections 2023-10-10T20:41:39.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.254+0000 7faf9759e700 1 --2- 172.21.15.89:0/4217836625 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faf78067320 0x7faf780697d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:39.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.254+0000 7faf9759e700 1 --2- 172.21.15.89:0/4217836625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf980ca1a0 0x7faf980ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:39.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.254+0000 7faf9759e700 1 --2- 172.21.15.89:0/4217836625 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faf84004010 0x7faf98155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:39.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.254+0000 7faf9759e700 1 --2- 172.21.15.89:0/4217836625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf98153f70 0x7faf981543e0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:39.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.254+0000 7faf9759e700 1 -- 172.21.15.89:0/4217836625 >> 172.21.15.89:0/4217836625 conn(0x7faf980d0660 msgr2=0x7faf980696e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:39.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.254+0000 7faf9759e700 1 -- 172.21.15.89:0/4217836625 shutdown_connections 2023-10-10T20:41:39.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.254+0000 7faf9759e700 1 -- 172.21.15.89:0/4217836625 wait complete. 2023-10-10T20:41:39.259 INFO:teuthology.orchestra.run.smithi089.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:41:39.268 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:41:39.276 INFO:tasks.ceph.mds.d.smithi111.stdout:starting mds.d at 2023-10-10T20:41:39.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.383+0000 7fd7ede29700 1 Processor -- start 2023-10-10T20:41:39.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.383+0000 7fd7ede29700 1 -- start start 2023-10-10T20:41:39.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.384+0000 7fd7ede29700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd7e80b16d0 0x7fd7e80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.384+0000 7fd7ede29700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd7e80b33b0 0x7fd7e80b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.384+0000 7fd7ede29700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e80b2080 con 0x7fd7e80b4e70 2023-10-10T20:41:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.384+0000 7fd7ede29700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e80cfb00 con 0x7fd7e80b33b0 2023-10-10T20:41:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.384+0000 7fd7ede29700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e80b5c40 con 0x7fd7e80b16d0 2023-10-10T20:41:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.384+0000 7fd7ece27700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fd7e80b4e70 0x7fd7e80ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:43092/0 (socket says 172.21.15.89:43092) 2023-10-10T20:41:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.384+0000 7fd7ece27700 1 -- 172.21.15.89:0/3488118908 learned_addr learned my addr 172.21.15.89:0/3488118908 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.384+0000 7fd7ed628700 1 --2- 172.21.15.89:0/3488118908 >> v2:172.21.15.111:3300/0 conn(0x7fd7e80b33b0 0x7fd7e80b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.384+0000 7fd7e7fff700 1 --2- 172.21.15.89:0/3488118908 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd7e80b16d0 0x7fd7e80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.384+0000 7fd7e77fe700 1 -- 172.21.15.89:0/3488118908 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2196320372 0 0) 0x7fd7e80b2080 con 0x7fd7e80b4e70 2023-10-10T20:41:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.384+0000 7fd7e77fe700 1 -- 172.21.15.89:0/3488118908 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7d8003570 con 0x7fd7e80b4e70 2023-10-10T20:41:39.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.385+0000 7fd7e7fff700 1 -- 172.21.15.89:0/3488118908 >> v2:172.21.15.111:3300/0 conn(0x7fd7e80b33b0 msgr2=0x7fd7e80b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:39.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.385+0000 7fd7e7fff700 1 --2- 172.21.15.89:0/3488118908 >> v2:172.21.15.111:3300/0 conn(0x7fd7e80b33b0 0x7fd7e80b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:39.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.385+0000 7fd7e7fff700 1 -- 172.21.15.89:0/3488118908 >> v1:172.21.15.111:6789/0 conn(0x7fd7e80b4e70 legacy=0x7fd7e80ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:39.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.385+0000 7fd7e7fff700 1 -- 172.21.15.89:0/3488118908 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e80b5f20 con 0x7fd7e80b16d0 2023-10-10T20:41:39.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.385+0000 7fd7e7fff700 1 --2- 172.21.15.89:0/3488118908 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd7e80b16d0 0x7fd7e80b1b40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40098e0 tx=0x7fd7d40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d08c7747a9a77793 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:39.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.385+0000 7fd7e77fe700 1 -- 172.21.15.89:0/3488118908 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4011630 con 0x7fd7e80b16d0 2023-10-10T20:41:39.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.385+0000 7fd7e77fe700 1 -- 172.21.15.89:0/3488118908 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d4011790 con 0x7fd7e80b16d0 2023-10-10T20:41:39.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.385+0000 7fd7e77fe700 1 -- 172.21.15.89:0/3488118908 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4011b30 con 0x7fd7e80b16d0 2023-10-10T20:41:39.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.385+0000 7fd7ede29700 1 -- 172.21.15.89:0/3488118908 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd7e80b16d0 msgr2=0x7fd7e80b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:39.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.385+0000 7fd7ede29700 1 --2- 172.21.15.89:0/3488118908 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd7e80b16d0 0x7fd7e80b1b40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40098e0 tx=0x7fd7d40091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.385+0000 7fd7ede29700 1 -- 172.21.15.89:0/3488118908 shutdown_connections 2023-10-10T20:41:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.385+0000 7fd7ede29700 1 --2- 172.21.15.89:0/3488118908 >> v2:172.21.15.111:3300/0 conn(0x7fd7e80b33b0 0x7fd7e80b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.385+0000 7fd7ede29700 1 --2- 172.21.15.89:0/3488118908 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd7e80b16d0 0x7fd7e80b1b40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.385+0000 7fd7ede29700 1 -- 172.21.15.89:0/3488118908 >> 172.21.15.89:0/3488118908 conn(0x7fd7e80d10c0 msgr2=0x7fd7e80affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.385+0000 7fd7ede29700 1 -- 172.21.15.89:0/3488118908 shutdown_connections 2023-10-10T20:41:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.385+0000 7fd7ede29700 1 -- 172.21.15.89:0/3488118908 wait complete. 2023-10-10T20:41:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.387+0000 7fd7ede29700 1 Processor -- start 2023-10-10T20:41:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.387+0000 7fd7ede29700 1 -- start start 2023-10-10T20:41:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.387+0000 7fd7ede29700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7e80b16d0 0x7fd7e8158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.387+0000 7fd7ede29700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd7e80b33b0 0x7fd7e8169b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.387+0000 7fd7ede29700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7e80b4e70 0x7fd7e815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.387+0000 7fd7ede29700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e81586d0 con 0x7fd7e80b4e70 2023-10-10T20:41:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.387+0000 7fd7ede29700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e80cab10 con 0x7fd7e80b16d0 2023-10-10T20:41:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.387+0000 7fd7ede29700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e80cadf0 con 0x7fd7e80b33b0 2023-10-10T20:41:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.388+0000 7fd7ed628700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7e80b4e70 0x7fd7e815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.388+0000 7fd7ed628700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7e80b4e70 0x7fd7e815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38316/0 (socket says 172.21.15.89:38316) 2023-10-10T20:41:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.388+0000 7fd7ed628700 1 -- 172.21.15.89:0/2291604504 learned_addr learned my addr 172.21.15.89:0/2291604504 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.388+0000 7fd7ece27700 1 --2- 172.21.15.89:0/2291604504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7e80b16d0 0x7fd7e8158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.388+0000 7fd7e7fff700 1 --2- 172.21.15.89:0/2291604504 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd7e80b33b0 0x7fd7e8169b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.388+0000 7fd7ed628700 1 -- 172.21.15.89:0/2291604504 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd7e80b33b0 msgr2=0x7fd7e8169b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.388+0000 7fd7ed628700 1 --2- 172.21.15.89:0/2291604504 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd7e80b33b0 0x7fd7e8169b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:39.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.388+0000 7fd7ed628700 1 -- 172.21.15.89:0/2291604504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7e80b16d0 msgr2=0x7fd7e8158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:39.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.388+0000 7fd7ed628700 1 --2- 172.21.15.89:0/2291604504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7e80b16d0 0x7fd7e8158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:39.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.388+0000 7fd7ed628700 1 -- 172.21.15.89:0/2291604504 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d4009420 con 0x7fd7e80b4e70 2023-10-10T20:41:39.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.388+0000 7fd7ed628700 1 --2- 172.21.15.89:0/2291604504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7e80b4e70 0x7fd7e815d4c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0009e90 tx=0x7fd7e000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:39.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.388+0000 7fd7e4ff9700 1 -- 172.21.15.89:0/2291604504 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e000cb70 con 0x7fd7e80b4e70 2023-10-10T20:41:39.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.388+0000 7fd7e4ff9700 1 -- 172.21.15.89:0/2291604504 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e000f040 con 0x7fd7e80b4e70 2023-10-10T20:41:39.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.388+0000 7fd7e4ff9700 1 -- 172.21.15.89:0/2291604504 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0014480 con 0x7fd7e80b4e70 2023-10-10T20:41:39.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.388+0000 7fd7ede29700 1 -- 172.21.15.89:0/2291604504 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e80cb0d0 con 0x7fd7e80b4e70 2023-10-10T20:41:39.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.388+0000 7fd7ede29700 1 -- 172.21.15.89:0/2291604504 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e80cb690 con 0x7fd7e80b4e70 2023-10-10T20:41:39.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.389+0000 7fd7ede29700 1 -- 172.21.15.89:0/2291604504 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e80cb880 con 0x7fd7e80b4e70 2023-10-10T20:41:39.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.389+0000 7fd7e4ff9700 1 -- 172.21.15.89:0/2291604504 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd7e0014830 con 0x7fd7e80b4e70 2023-10-10T20:41:39.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.390+0000 7fd7e4ff9700 1 --2- 172.21.15.89:0/2291604504 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd7cc067320 0x7fd7cc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:39.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.390+0000 7fd7e4ff9700 1 -- 172.21.15.89:0/2291604504 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5963+0+0 (secure 0 0 0) 0x7fd7e0085900 con 0x7fd7e80b4e70 2023-10-10T20:41:39.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.390+0000 7fd7ece27700 1 --2- 172.21.15.89:0/2291604504 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd7cc067320 0x7fd7cc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:39.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.390+0000 7fd7ece27700 1 --2- 172.21.15.89:0/2291604504 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd7cc067320 0x7fd7cc0697d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc007fd0 tx=0x7fd7dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.394+0000 7fd7e4ff9700 1 -- 172.21.15.89:0/2291604504 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd7e0052380 con 0x7fd7e80b4e70 2023-10-10T20:41:39.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:39.622+0000 7fd7ede29700 1 -- 172.21.15.89:0/2291604504 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd7e804bb80 con 0x7fd7e80b4e70 2023-10-10T20:41:40.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.252+0000 7fd7e4ff9700 1 -- 172.21.15.89:0/2291604504 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd7e00559a0 con 0x7fd7e80b4e70 2023-10-10T20:41:40.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.255+0000 7fd7ede29700 1 -- 172.21.15.89:0/2291604504 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd7cc067320 msgr2=0x7fd7cc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:40.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.255+0000 7fd7ede29700 1 --2- 172.21.15.89:0/2291604504 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd7cc067320 0x7fd7cc0697d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc007fd0 tx=0x7fd7dc009040 comp rx=0 tx=0).stop 2023-10-10T20:41:40.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.255+0000 7fd7ede29700 1 -- 172.21.15.89:0/2291604504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7e80b4e70 msgr2=0x7fd7e815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:40.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.255+0000 7fd7ede29700 1 --2- 172.21.15.89:0/2291604504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7e80b4e70 0x7fd7e815d4c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0009e90 tx=0x7fd7e000b040 comp rx=0 tx=0).stop 2023-10-10T20:41:40.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.255+0000 7fd7ede29700 1 -- 172.21.15.89:0/2291604504 shutdown_connections 2023-10-10T20:41:40.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.255+0000 7fd7ede29700 1 --2- 172.21.15.89:0/2291604504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7e80b4e70 0x7fd7e815d4c0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:40.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.255+0000 7fd7ede29700 1 --2- 172.21.15.89:0/2291604504 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd7cc067320 0x7fd7cc0697d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:40.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.255+0000 7fd7ede29700 1 --2- 172.21.15.89:0/2291604504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7e80b16d0 0x7fd7e8158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:40.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.255+0000 7fd7ede29700 1 --2- 172.21.15.89:0/2291604504 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd7e80b33b0 0x7fd7e8169b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:40.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.255+0000 7fd7ede29700 1 -- 172.21.15.89:0/2291604504 >> 172.21.15.89:0/2291604504 conn(0x7fd7e80d10c0 msgr2=0x7fd7e8069bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:40.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.255+0000 7fd7ede29700 1 -- 172.21.15.89:0/2291604504 shutdown_connections 2023-10-10T20:41:40.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.255+0000 7fd7ede29700 1 -- 172.21.15.89:0/2291604504 wait complete. 2023-10-10T20:41:40.270 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:41:40.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.388+0000 7f25a7fff700 1 Processor -- start 2023-10-10T20:41:40.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.388+0000 7f25a7fff700 1 -- start start 2023-10-10T20:41:40.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.389+0000 7f25a7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f25a80b03c0 0x7f25a80cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:40.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.389+0000 7f25a7fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25a80b0790 0x7f25a80b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:40.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.389+0000 7f25a7fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f25a81591e0 0x7f25a8159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:40.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.389+0000 7f25a7fff700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a80cfa00 con 0x7f25a80b03c0 2023-10-10T20:41:40.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.389+0000 7f25a7fff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a80cb580 con 0x7f25a81591e0 2023-10-10T20:41:40.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.389+0000 7f25a7fff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a80b2de0 con 0x7f25a80b0790 2023-10-10T20:41:40.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.389+0000 7f25a6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f25a80b03c0 0x7f25a80cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:40.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.389+0000 7f25a6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f25a80b03c0 0x7f25a80cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38326/0 (socket says 172.21.15.89:38326) 2023-10-10T20:41:40.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.389+0000 7f25a6ffd700 1 -- 172.21.15.89:0/623892509 learned_addr learned my addr 172.21.15.89:0/623892509 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:40.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.389+0000 7f259e7fc700 1 --2- 172.21.15.89:0/623892509 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25a80b0790 0x7f25a80b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:40.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.389+0000 7f25a77fe700 1 --2- 172.21.15.89:0/623892509 >> v2:172.21.15.111:3300/0 conn(0x7f25a81591e0 0x7f25a8159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:40.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.389+0000 7f25a6ffd700 1 -- 172.21.15.89:0/623892509 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25a80b0790 msgr2=0x7f25a80b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:40.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.389+0000 7f25a6ffd700 1 --2- 172.21.15.89:0/623892509 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25a80b0790 0x7f25a80b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:40.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.389+0000 7f25a6ffd700 1 -- 172.21.15.89:0/623892509 >> v2:172.21.15.111:3300/0 conn(0x7f25a81591e0 msgr2=0x7f25a8159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:40.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.389+0000 7f25a6ffd700 1 --2- 172.21.15.89:0/623892509 >> v2:172.21.15.111:3300/0 conn(0x7f25a81591e0 0x7f25a8159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:40.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.389+0000 7f25a6ffd700 1 -- 172.21.15.89:0/623892509 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a80cf160 con 0x7f25a80b03c0 2023-10-10T20:41:40.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.389+0000 7f25a77fe700 1 --2- 172.21.15.89:0/623892509 >> v2:172.21.15.111:3300/0 conn(0x7f25a81591e0 0x7f25a8159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:40.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.390+0000 7f25a6ffd700 1 --2- 172.21.15.89:0/623892509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25a80b03c0 0x7f25a80cea70 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f25a80cb4f0 tx=0x7f2598009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14be9ba408be4056 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:40.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.390+0000 7f25a67fc700 1 -- 172.21.15.89:0/623892509 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598009e70 con 0x7f25a80b03c0 2023-10-10T20:41:40.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.390+0000 7f25a67fc700 1 -- 172.21.15.89:0/623892509 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259800b860 con 0x7f25a80b03c0 2023-10-10T20:41:40.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.390+0000 7f25a67fc700 1 -- 172.21.15.89:0/623892509 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259800bc00 con 0x7f25a80b03c0 2023-10-10T20:41:40.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.390+0000 7f25a7fff700 1 -- 172.21.15.89:0/623892509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25a80b03c0 msgr2=0x7f25a80cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:40.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.390+0000 7f25a7fff700 1 --2- 172.21.15.89:0/623892509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25a80b03c0 0x7f25a80cea70 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f25a80cb4f0 tx=0x7f2598009250 comp rx=0 tx=0).stop 2023-10-10T20:41:40.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.390+0000 7f25a7fff700 1 -- 172.21.15.89:0/623892509 shutdown_connections 2023-10-10T20:41:40.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.390+0000 7f25a7fff700 1 --2- 172.21.15.89:0/623892509 >> v2:172.21.15.111:3300/0 conn(0x7f25a81591e0 0x7f25a8159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:40.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.390+0000 7f25a7fff700 1 --2- 172.21.15.89:0/623892509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25a80b03c0 0x7f25a80cea70 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:40.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.390+0000 7f25a7fff700 1 --2- 172.21.15.89:0/623892509 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25a80b0790 0x7f25a80b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:40.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.390+0000 7f25a7fff700 1 -- 172.21.15.89:0/623892509 >> 172.21.15.89:0/623892509 conn(0x7f25a8063230 msgr2=0x7f25a8063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:40.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.390+0000 7f25a7fff700 1 -- 172.21.15.89:0/623892509 shutdown_connections 2023-10-10T20:41:40.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.390+0000 7f25a7fff700 1 -- 172.21.15.89:0/623892509 wait complete. 2023-10-10T20:41:40.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.392+0000 7f25a7fff700 1 Processor -- start 2023-10-10T20:41:40.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.392+0000 7f25a7fff700 1 -- start start 2023-10-10T20:41:40.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.392+0000 7f25a7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25a80b03c0 0x7f25a8159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:40.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.392+0000 7f25a7fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25a80b0790 0x7f25a8145190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:40.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.392+0000 7f25a7fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25a81591e0 0x7f25a80c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:40.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.392+0000 7f25a7fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a815db40 con 0x7f25a80b03c0 2023-10-10T20:41:40.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.392+0000 7f25a7fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a80c5fe0 con 0x7f25a81591e0 2023-10-10T20:41:40.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.392+0000 7f25a7fff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a81456d0 con 0x7f25a80b0790 2023-10-10T20:41:40.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.392+0000 7f25a6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25a80b03c0 0x7f25a8159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:40.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.392+0000 7f25a6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25a80b03c0 0x7f25a8159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38328/0 (socket says 172.21.15.89:38328) 2023-10-10T20:41:40.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.392+0000 7f25a6ffd700 1 -- 172.21.15.89:0/3649730414 learned_addr learned my addr 172.21.15.89:0/3649730414 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:40.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.392+0000 7f259e7fc700 1 --2- 172.21.15.89:0/3649730414 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25a80b0790 0x7f25a8145190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:40.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.393+0000 7f25a77fe700 1 --2- 172.21.15.89:0/3649730414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25a81591e0 0x7f25a80c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:40.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.393+0000 7f25a6ffd700 1 -- 172.21.15.89:0/3649730414 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25a80b0790 msgr2=0x7f25a8145190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:40.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.393+0000 7f25a6ffd700 1 --2- 172.21.15.89:0/3649730414 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25a80b0790 0x7f25a8145190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:40.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.393+0000 7f25a6ffd700 1 -- 172.21.15.89:0/3649730414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25a81591e0 msgr2=0x7f25a80c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:40.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.393+0000 7f25a6ffd700 1 --2- 172.21.15.89:0/3649730414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25a81591e0 0x7f25a80c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:40.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.393+0000 7f25a6ffd700 1 -- 172.21.15.89:0/3649730414 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25980094f0 con 0x7f25a80b03c0 2023-10-10T20:41:40.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.393+0000 7f25a77fe700 1 --2- 172.21.15.89:0/3649730414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25a81591e0 0x7f25a80c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:40.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.393+0000 7f25a6ffd700 1 --2- 172.21.15.89:0/3649730414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25a80b03c0 0x7f25a8159620 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f25980090b0 tx=0x7f2598017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:40.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.393+0000 7f259f7fe700 1 -- 172.21.15.89:0/3649730414 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598000d10 con 0x7f25a80b03c0 2023-10-10T20:41:40.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.393+0000 7f259f7fe700 1 -- 172.21.15.89:0/3649730414 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259800bbe0 con 0x7f25a80b03c0 2023-10-10T20:41:40.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.393+0000 7f25a7fff700 1 -- 172.21.15.89:0/3649730414 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a815d580 con 0x7f25a80b03c0 2023-10-10T20:41:40.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.393+0000 7f259f7fe700 1 -- 172.21.15.89:0/3649730414 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259801a680 con 0x7f25a80b03c0 2023-10-10T20:41:40.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.393+0000 7f25a7fff700 1 -- 172.21.15.89:0/3649730414 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a80c5c70 con 0x7f25a80b03c0 2023-10-10T20:41:40.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.394+0000 7f25a7fff700 1 -- 172.21.15.89:0/3649730414 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a804c3c0 con 0x7f25a80b03c0 2023-10-10T20:41:40.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.394+0000 7f259f7fe700 1 -- 172.21.15.89:0/3649730414 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f259801aa30 con 0x7f25a80b03c0 2023-10-10T20:41:40.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.394+0000 7f259f7fe700 1 --2- 172.21.15.89:0/3649730414 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2588067320 0x7f25880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:40.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.395+0000 7f259f7fe700 1 -- 172.21.15.89:0/3649730414 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f2598086230 con 0x7f25a80b03c0 2023-10-10T20:41:40.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.395+0000 7f259e7fc700 1 --2- 172.21.15.89:0/3649730414 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2588067320 0x7f25880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:40.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.395+0000 7f259e7fc700 1 --2- 172.21.15.89:0/3649730414 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2588067320 0x7f25880697d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f25a80c51d0 tx=0x7f25900091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:40.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.399+0000 7f259f7fe700 1 -- 172.21.15.89:0/3649730414 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2598052dc0 con 0x7f25a80b03c0 2023-10-10T20:41:40.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:40.583+0000 7f25a7fff700 1 -- 172.21.15.89:0/3649730414 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f25a80b2770 con 0x7f25a80b03c0 2023-10-10T20:41:41.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.261+0000 7f259f7fe700 1 -- 172.21.15.89:0/3649730414 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v128) v1 ==== 181+0+0 (secure 0 0 0) 0x7f25980563e0 con 0x7f25a80b03c0 2023-10-10T20:41:41.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.264+0000 7f25a7fff700 1 -- 172.21.15.89:0/3649730414 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2588067320 msgr2=0x7f25880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:41.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.264+0000 7f25a7fff700 1 --2- 172.21.15.89:0/3649730414 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2588067320 0x7f25880697d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f25a80c51d0 tx=0x7f25900091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:41.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.264+0000 7f25a7fff700 1 -- 172.21.15.89:0/3649730414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25a80b03c0 msgr2=0x7f25a8159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:41.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.264+0000 7f25a7fff700 1 --2- 172.21.15.89:0/3649730414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25a80b03c0 0x7f25a8159620 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f25980090b0 tx=0x7f2598017000 comp rx=0 tx=0).stop 2023-10-10T20:41:41.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.264+0000 7f25a7fff700 1 -- 172.21.15.89:0/3649730414 shutdown_connections 2023-10-10T20:41:41.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.264+0000 7f25a7fff700 1 --2- 172.21.15.89:0/3649730414 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2588067320 0x7f25880697d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:41.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.264+0000 7f25a7fff700 1 --2- 172.21.15.89:0/3649730414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25a81591e0 0x7f25a80c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:41.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.264+0000 7f25a7fff700 1 --2- 172.21.15.89:0/3649730414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25a80b03c0 0x7f25a8159620 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:41.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.264+0000 7f25a7fff700 1 --2- 172.21.15.89:0/3649730414 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f25a80b0790 0x7f25a8145190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:41.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.264+0000 7f25a7fff700 1 -- 172.21.15.89:0/3649730414 >> 172.21.15.89:0/3649730414 conn(0x7f25a8063230 msgr2=0x7f25a80ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:41.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.264+0000 7f25a7fff700 1 -- 172.21.15.89:0/3649730414 shutdown_connections 2023-10-10T20:41:41.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.264+0000 7f25a7fff700 1 -- 172.21.15.89:0/3649730414 wait complete. 2023-10-10T20:41:41.268 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:41:41.281 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:41:41.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.399+0000 7fac98b48700 1 Processor -- start 2023-10-10T20:41:41.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.399+0000 7fac98b48700 1 -- start start 2023-10-10T20:41:41.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.400+0000 7fac98b48700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac94159ec0 0x7fac940b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:41.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.400+0000 7fac98b48700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fac940b53d0 0x7fac940cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:41.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.400+0000 7fac98b48700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fac940cce40 0x7fac940cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:41.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.400+0000 7fac98b48700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac940b5b00 con 0x7fac940b53d0 2023-10-10T20:41:41.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.400+0000 7fac98b48700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac940b5da0 con 0x7fac940cce40 2023-10-10T20:41:41.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.400+0000 7fac98b48700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac940b6080 con 0x7fac94159ec0 2023-10-10T20:41:41.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.400+0000 7fac89d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fac940b53d0 0x7fac940cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:41.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.400+0000 7fac89d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fac940b53d0 0x7fac940cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38336/0 (socket says 172.21.15.89:38336) 2023-10-10T20:41:41.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.400+0000 7fac89d9b700 1 -- 172.21.15.89:0/3489796628 learned_addr learned my addr 172.21.15.89:0/3489796628 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:41.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.400+0000 7fac92d9d700 1 --2- 172.21.15.89:0/3489796628 >> v2:172.21.15.111:3300/0 conn(0x7fac940cce40 0x7fac940cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:41.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.400+0000 7fac9259c700 1 --2- 172.21.15.89:0/3489796628 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac94159ec0 0x7fac940b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:41.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.400+0000 7fac89d9b700 1 -- 172.21.15.89:0/3489796628 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac94159ec0 msgr2=0x7fac940b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:41.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.400+0000 7fac89d9b700 1 --2- 172.21.15.89:0/3489796628 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac94159ec0 0x7fac940b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:41.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.400+0000 7fac89d9b700 1 -- 172.21.15.89:0/3489796628 >> v2:172.21.15.111:3300/0 conn(0x7fac940cce40 msgr2=0x7fac940cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:41.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.400+0000 7fac89d9b700 1 --2- 172.21.15.89:0/3489796628 >> v2:172.21.15.111:3300/0 conn(0x7fac940cce40 0x7fac940cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:41.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.400+0000 7fac89d9b700 1 -- 172.21.15.89:0/3489796628 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac940cc100 con 0x7fac940b53d0 2023-10-10T20:41:41.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.400+0000 7fac89d9b700 1 --2- 172.21.15.89:0/3489796628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac940b53d0 0x7fac940cc900 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fac9409e530 tx=0x7fac7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc1be4cde631b7e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:41.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.400+0000 7fac91d9b700 1 -- 172.21.15.89:0/3489796628 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c00b860 con 0x7fac940b53d0 2023-10-10T20:41:41.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.401+0000 7fac91d9b700 1 -- 172.21.15.89:0/3489796628 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7c00b9c0 con 0x7fac940b53d0 2023-10-10T20:41:41.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.401+0000 7fac91d9b700 1 -- 172.21.15.89:0/3489796628 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c00bd60 con 0x7fac940b53d0 2023-10-10T20:41:41.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.401+0000 7fac98b48700 1 -- 172.21.15.89:0/3489796628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac940b53d0 msgr2=0x7fac940cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:41.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.401+0000 7fac98b48700 1 --2- 172.21.15.89:0/3489796628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac940b53d0 0x7fac940cc900 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fac9409e530 tx=0x7fac7c009250 comp rx=0 tx=0).stop 2023-10-10T20:41:41.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.401+0000 7fac98b48700 1 -- 172.21.15.89:0/3489796628 shutdown_connections 2023-10-10T20:41:41.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.401+0000 7fac98b48700 1 --2- 172.21.15.89:0/3489796628 >> v2:172.21.15.111:3300/0 conn(0x7fac940cce40 0x7fac940cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:41.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.401+0000 7fac98b48700 1 --2- 172.21.15.89:0/3489796628 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac94159ec0 0x7fac940b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:41.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.401+0000 7fac98b48700 1 --2- 172.21.15.89:0/3489796628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac940b53d0 0x7fac940cc900 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:41.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.401+0000 7fac98b48700 1 -- 172.21.15.89:0/3489796628 >> 172.21.15.89:0/3489796628 conn(0x7fac940999f0 msgr2=0x7fac940b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:41.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.401+0000 7fac98b48700 1 -- 172.21.15.89:0/3489796628 shutdown_connections 2023-10-10T20:41:41.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.401+0000 7fac98b48700 1 -- 172.21.15.89:0/3489796628 wait complete. 2023-10-10T20:41:41.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.402+0000 7fac98b48700 1 Processor -- start 2023-10-10T20:41:41.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.402+0000 7fac98b48700 1 -- start start 2023-10-10T20:41:41.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.403+0000 7fac98b48700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac940b53d0 0x7fac9406af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:41.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.403+0000 7fac98b48700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac940cce40 0x7fac94069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:41.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.403+0000 7fac98b48700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac94159ec0 0x7fac9413d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:41.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.403+0000 7fac98b48700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9416b5b0 con 0x7fac94159ec0 2023-10-10T20:41:41.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.403+0000 7fac98b48700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9416ac80 con 0x7fac940b53d0 2023-10-10T20:41:41.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.403+0000 7fac98b48700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9416af60 con 0x7fac940cce40 2023-10-10T20:41:41.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.403+0000 7fac92d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac94159ec0 0x7fac9413d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:41.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.403+0000 7fac92d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac94159ec0 0x7fac9413d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38350/0 (socket says 172.21.15.89:38350) 2023-10-10T20:41:41.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.403+0000 7fac92d9d700 1 -- 172.21.15.89:0/3114362269 learned_addr learned my addr 172.21.15.89:0/3114362269 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:41.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.403+0000 7fac9259c700 1 --2- 172.21.15.89:0/3114362269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac940b53d0 0x7fac9406af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:41.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.403+0000 7fac89d9b700 1 --2- 172.21.15.89:0/3114362269 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac940cce40 0x7fac94069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:41.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.403+0000 7fac92d9d700 1 -- 172.21.15.89:0/3114362269 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac940cce40 msgr2=0x7fac94069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:41.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.404+0000 7fac92d9d700 1 --2- 172.21.15.89:0/3114362269 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac940cce40 0x7fac94069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:41.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.404+0000 7fac92d9d700 1 -- 172.21.15.89:0/3114362269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac940b53d0 msgr2=0x7fac9406af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:41.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.404+0000 7fac92d9d700 1 --2- 172.21.15.89:0/3114362269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac940b53d0 0x7fac9406af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:41.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.404+0000 7fac92d9d700 1 -- 172.21.15.89:0/3114362269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c0094f0 con 0x7fac94159ec0 2023-10-10T20:41:41.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.404+0000 7fac92d9d700 1 --2- 172.21.15.89:0/3114362269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac94159ec0 0x7fac9413d880 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fac8c009fa0 tx=0x7fac8c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:41.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.404+0000 7fac8affd700 1 -- 172.21.15.89:0/3114362269 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c00ebc0 con 0x7fac94159ec0 2023-10-10T20:41:41.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.404+0000 7fac8affd700 1 -- 172.21.15.89:0/3114362269 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8c00ed20 con 0x7fac94159ec0 2023-10-10T20:41:41.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.404+0000 7fac98b48700 1 -- 172.21.15.89:0/3114362269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9416b240 con 0x7fac94159ec0 2023-10-10T20:41:41.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.404+0000 7fac8affd700 1 -- 172.21.15.89:0/3114362269 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c014670 con 0x7fac94159ec0 2023-10-10T20:41:41.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.404+0000 7fac98b48700 1 -- 172.21.15.89:0/3114362269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9406a410 con 0x7fac94159ec0 2023-10-10T20:41:41.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.405+0000 7fac98b48700 1 -- 172.21.15.89:0/3114362269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac940718b0 con 0x7fac94159ec0 2023-10-10T20:41:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.405+0000 7fac8affd700 1 -- 172.21.15.89:0/3114362269 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fac8c0149e0 con 0x7fac94159ec0 2023-10-10T20:41:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.405+0000 7fac8affd700 1 --2- 172.21.15.89:0/3114362269 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fac74067320 0x7fac740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.406+0000 7fac8affd700 1 -- 172.21.15.89:0/3114362269 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5525+0+0 (secure 0 0 0) 0x7fac8c085260 con 0x7fac94159ec0 2023-10-10T20:41:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.406+0000 7fac9259c700 1 --2- 172.21.15.89:0/3114362269 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fac74067320 0x7fac740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:41.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.406+0000 7fac9259c700 1 --2- 172.21.15.89:0/3114362269 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fac74067320 0x7fac740697d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fac9406aad0 tx=0x7fac84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:41.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.410+0000 7fac8affd700 1 -- 172.21.15.89:0/3114362269 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac8c051fa0 con 0x7fac94159ec0 2023-10-10T20:41:41.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:41.591+0000 7fac98b48700 1 -- 172.21.15.89:0/3114362269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fac94063c40 con 0x7fac94159ec0 2023-10-10T20:41:42.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.261+0000 7fac8affd700 1 -- 172.21.15.89:0/3114362269 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v129) v1 ==== 169+0+0 (secure 0 0 0) 0x7fac8c0555c0 con 0x7fac94159ec0 2023-10-10T20:41:42.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.263+0000 7fac98b48700 1 -- 172.21.15.89:0/3114362269 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fac74067320 msgr2=0x7fac740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:42.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.263+0000 7fac98b48700 1 --2- 172.21.15.89:0/3114362269 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fac74067320 0x7fac740697d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fac9406aad0 tx=0x7fac84008040 comp rx=0 tx=0).stop 2023-10-10T20:41:42.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.263+0000 7fac98b48700 1 -- 172.21.15.89:0/3114362269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac94159ec0 msgr2=0x7fac9413d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:42.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.263+0000 7fac98b48700 1 --2- 172.21.15.89:0/3114362269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac94159ec0 0x7fac9413d880 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fac8c009fa0 tx=0x7fac8c009900 comp rx=0 tx=0).stop 2023-10-10T20:41:42.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.263+0000 7fac98b48700 1 -- 172.21.15.89:0/3114362269 shutdown_connections 2023-10-10T20:41:42.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.263+0000 7fac98b48700 1 --2- 172.21.15.89:0/3114362269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac94159ec0 0x7fac9413d880 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.263+0000 7fac98b48700 1 --2- 172.21.15.89:0/3114362269 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fac74067320 0x7fac740697d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.263+0000 7fac98b48700 1 --2- 172.21.15.89:0/3114362269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac940b53d0 0x7fac9406af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.263+0000 7fac98b48700 1 --2- 172.21.15.89:0/3114362269 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac940cce40 0x7fac94069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.263+0000 7fac98b48700 1 -- 172.21.15.89:0/3114362269 >> 172.21.15.89:0/3114362269 conn(0x7fac940999f0 msgr2=0x7fac940c6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:42.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.264+0000 7fac98b48700 1 -- 172.21.15.89:0/3114362269 shutdown_connections 2023-10-10T20:41:42.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.264+0000 7fac98b48700 1 -- 172.21.15.89:0/3114362269 wait complete. 2023-10-10T20:41:42.269 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' removed 2023-10-10T20:41:42.279 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:41:42.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.396+0000 7f3da3d04700 1 Processor -- start 2023-10-10T20:41:42.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.396+0000 7f3da3d04700 1 -- start start 2023-10-10T20:41:42.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.397+0000 7f3da3d04700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3d9c0aca80 0x7f3d9c06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:42.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.397+0000 7f3da3d04700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d9c069d60 0x7f3d9c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:42.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.397+0000 7f3da3d04700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3d9c0c81f0 0x7f3d9c068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:42.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.397+0000 7f3da3d04700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d9c0c8670 con 0x7f3d9c0c81f0 2023-10-10T20:41:42.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.397+0000 7f3da3d04700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d9c06bdb0 con 0x7f3d9c0aca80 2023-10-10T20:41:42.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.397+0000 7f3da3d04700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c0cb370 con 0x7f3d9c069d60 2023-10-10T20:41:42.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.397+0000 7f3da22a1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3d9c0c81f0 0x7f3d9c068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:42.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.397+0000 7f3da22a1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3d9c0c81f0 0x7f3d9c068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38364/0 (socket says 172.21.15.89:38364) 2023-10-10T20:41:42.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.397+0000 7f3da22a1700 1 -- 172.21.15.89:0/568139825 learned_addr learned my addr 172.21.15.89:0/568139825 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:42.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.397+0000 7f3da129f700 1 --2- 172.21.15.89:0/568139825 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d9c069d60 0x7f3d9c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:42.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.397+0000 7f3da1aa0700 1 --2- 172.21.15.89:0/568139825 >> v2:172.21.15.111:3300/0 conn(0x7f3d9c0aca80 0x7f3d9c06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:42.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.397+0000 7f3da22a1700 1 -- 172.21.15.89:0/568139825 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d9c069d60 msgr2=0x7f3d9c0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:42.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.397+0000 7f3da22a1700 1 --2- 172.21.15.89:0/568139825 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d9c069d60 0x7f3d9c0c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.397+0000 7f3da22a1700 1 -- 172.21.15.89:0/568139825 >> v2:172.21.15.111:3300/0 conn(0x7f3d9c0aca80 msgr2=0x7f3d9c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:42.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.397+0000 7f3da22a1700 1 --2- 172.21.15.89:0/568139825 >> v2:172.21.15.111:3300/0 conn(0x7f3d9c0aca80 0x7f3d9c06aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.397+0000 7f3da22a1700 1 -- 172.21.15.89:0/568139825 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c159f60 con 0x7f3d9c0c81f0 2023-10-10T20:41:42.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.397+0000 7f3da22a1700 1 --2- 172.21.15.89:0/568139825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9c0c81f0 0x7f3d9c068390 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c06e570 tx=0x7f3d8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e362fc19e1a1a73f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:42.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.398+0000 7f3da0a9e700 1 -- 172.21.15.89:0/568139825 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c00b640 con 0x7f3d9c0c81f0 2023-10-10T20:41:42.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.398+0000 7f3da0a9e700 1 -- 172.21.15.89:0/568139825 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8c00b7a0 con 0x7f3d9c0c81f0 2023-10-10T20:41:42.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.398+0000 7f3da0a9e700 1 -- 172.21.15.89:0/568139825 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c00bb60 con 0x7f3d9c0c81f0 2023-10-10T20:41:42.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.398+0000 7f3da3d04700 1 -- 172.21.15.89:0/568139825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9c0c81f0 msgr2=0x7f3d9c068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:42.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.398+0000 7f3da3d04700 1 --2- 172.21.15.89:0/568139825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9c0c81f0 0x7f3d9c068390 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c06e570 tx=0x7f3d8c009250 comp rx=0 tx=0).stop 2023-10-10T20:41:42.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.398+0000 7f3da3d04700 1 -- 172.21.15.89:0/568139825 shutdown_connections 2023-10-10T20:41:42.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.398+0000 7f3da3d04700 1 --2- 172.21.15.89:0/568139825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9c0c81f0 0x7f3d9c068390 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.398+0000 7f3da3d04700 1 --2- 172.21.15.89:0/568139825 >> v2:172.21.15.111:3300/0 conn(0x7f3d9c0aca80 0x7f3d9c06aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.398+0000 7f3da3d04700 1 --2- 172.21.15.89:0/568139825 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d9c069d60 0x7f3d9c0c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.398+0000 7f3da3d04700 1 -- 172.21.15.89:0/568139825 >> 172.21.15.89:0/568139825 conn(0x7f3d9c0cd670 msgr2=0x7f3d9c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:42.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.398+0000 7f3da3d04700 1 -- 172.21.15.89:0/568139825 shutdown_connections 2023-10-10T20:41:42.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.398+0000 7f3da3d04700 1 -- 172.21.15.89:0/568139825 wait complete. 2023-10-10T20:41:42.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.399+0000 7f3da3d04700 1 Processor -- start 2023-10-10T20:41:42.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.399+0000 7f3da3d04700 1 -- start start 2023-10-10T20:41:42.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.400+0000 7f3da3d04700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d9c069d60 0x7f3d9c0b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:42.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.400+0000 7f3da3d04700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9c0aca80 0x7f3d9c15f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:42.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.400+0000 7f3da3d04700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d9c0c81f0 0x7f3d9c15f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:42.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.400+0000 7f3da3d04700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c16dab0 con 0x7f3d9c0aca80 2023-10-10T20:41:42.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.400+0000 7f3da3d04700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c16cea0 con 0x7f3d9c0c81f0 2023-10-10T20:41:42.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.400+0000 7f3da3d04700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c16d180 con 0x7f3d9c069d60 2023-10-10T20:41:42.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.400+0000 7f3da129f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9c0aca80 0x7f3d9c15f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:42.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.400+0000 7f3da129f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9c0aca80 0x7f3d9c15f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38374/0 (socket says 172.21.15.89:38374) 2023-10-10T20:41:42.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.400+0000 7f3da129f700 1 -- 172.21.15.89:0/1595426130 learned_addr learned my addr 172.21.15.89:0/1595426130 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:42.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.400+0000 7f3da1aa0700 1 --2- 172.21.15.89:0/1595426130 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d9c069d60 0x7f3d9c0b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:42.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.400+0000 7f3da22a1700 1 --2- 172.21.15.89:0/1595426130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d9c0c81f0 0x7f3d9c15f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:42.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.400+0000 7f3da129f700 1 -- 172.21.15.89:0/1595426130 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d9c069d60 msgr2=0x7f3d9c0b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:42.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.400+0000 7f3da129f700 1 --2- 172.21.15.89:0/1595426130 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d9c069d60 0x7f3d9c0b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.400+0000 7f3da129f700 1 -- 172.21.15.89:0/1595426130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d9c0c81f0 msgr2=0x7f3d9c15f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:42.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.400+0000 7f3da129f700 1 --2- 172.21.15.89:0/1595426130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d9c0c81f0 0x7f3d9c15f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.400+0000 7f3da129f700 1 -- 172.21.15.89:0/1595426130 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c0094f0 con 0x7f3d9c0aca80 2023-10-10T20:41:42.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.400+0000 7f3da1aa0700 1 --2- 172.21.15.89:0/1595426130 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d9c069d60 0x7f3d9c0b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:42.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.400+0000 7f3da129f700 1 --2- 172.21.15.89:0/1595426130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9c0aca80 0x7f3d9c15f460 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009eb0 tx=0x7f3d9000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:42.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.401+0000 7f3d99ffb700 1 -- 172.21.15.89:0/1595426130 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9000cc20 con 0x7f3d9c0aca80 2023-10-10T20:41:42.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.401+0000 7f3d99ffb700 1 -- 172.21.15.89:0/1595426130 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9000f040 con 0x7f3d9c0aca80 2023-10-10T20:41:42.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.401+0000 7f3da3d04700 1 -- 172.21.15.89:0/1595426130 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d9c16d460 con 0x7f3d9c0aca80 2023-10-10T20:41:42.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.401+0000 7f3d99ffb700 1 -- 172.21.15.89:0/1595426130 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d90014480 con 0x7f3d9c0aca80 2023-10-10T20:41:42.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.401+0000 7f3da3d04700 1 -- 172.21.15.89:0/1595426130 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9c0b3130 con 0x7f3d9c0aca80 2023-10-10T20:41:42.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.402+0000 7f3da3d04700 1 -- 172.21.15.89:0/1595426130 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d9c0ac340 con 0x7f3d9c0aca80 2023-10-10T20:41:42.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.402+0000 7f3d99ffb700 1 -- 172.21.15.89:0/1595426130 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3d900147f0 con 0x7f3d9c0aca80 2023-10-10T20:41:42.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.402+0000 7f3d99ffb700 1 --2- 172.21.15.89:0/1595426130 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3d88067320 0x7f3d880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:42.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.403+0000 7f3d99ffb700 1 -- 172.21.15.89:0/1595426130 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5111+0+0 (secure 0 0 0) 0x7f3d90085850 con 0x7f3d9c0aca80 2023-10-10T20:41:42.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.403+0000 7f3da1aa0700 1 --2- 172.21.15.89:0/1595426130 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3d88067320 0x7f3d880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:42.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.403+0000 7f3da1aa0700 1 --2- 172.21.15.89:0/1595426130 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3d88067320 0x7f3d880697d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009d70 tx=0x7f3d94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:42.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.407+0000 7f3d99ffb700 1 -- 172.21.15.89:0/1595426130 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d900560b0 con 0x7f3d9c0aca80 2023-10-10T20:41:42.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.590+0000 7f3da3d04700 1 -- 172.21.15.89:0/1595426130 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3d9c04c3c0 con 0x7f3d9c0aca80 2023-10-10T20:41:42.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.597+0000 7f3d99ffb700 1 -- 172.21.15.89:0/1595426130 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3d90055c40 con 0x7f3d9c0aca80 2023-10-10T20:41:42.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.599+0000 7f3da3d04700 1 -- 172.21.15.89:0/1595426130 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3d88067320 msgr2=0x7f3d880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:42.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.599+0000 7f3da3d04700 1 --2- 172.21.15.89:0/1595426130 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3d88067320 0x7f3d880697d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009d70 tx=0x7f3d94008040 comp rx=0 tx=0).stop 2023-10-10T20:41:42.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.599+0000 7f3da3d04700 1 -- 172.21.15.89:0/1595426130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9c0aca80 msgr2=0x7f3d9c15f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:42.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.599+0000 7f3da3d04700 1 --2- 172.21.15.89:0/1595426130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9c0aca80 0x7f3d9c15f460 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009eb0 tx=0x7f3d9000b040 comp rx=0 tx=0).stop 2023-10-10T20:41:42.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.599+0000 7f3da3d04700 1 -- 172.21.15.89:0/1595426130 shutdown_connections 2023-10-10T20:41:42.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.599+0000 7f3da3d04700 1 --2- 172.21.15.89:0/1595426130 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3d88067320 0x7f3d880697d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.599+0000 7f3da3d04700 1 --2- 172.21.15.89:0/1595426130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d9c0c81f0 0x7f3d9c15f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.599+0000 7f3da3d04700 1 --2- 172.21.15.89:0/1595426130 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3d9c069d60 0x7f3d9c0b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.599+0000 7f3da3d04700 1 --2- 172.21.15.89:0/1595426130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9c0aca80 0x7f3d9c15f460 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.599+0000 7f3da3d04700 1 -- 172.21.15.89:0/1595426130 >> 172.21.15.89:0/1595426130 conn(0x7f3d9c0cd670 msgr2=0x7f3d9c0c3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:42.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.599+0000 7f3da3d04700 1 -- 172.21.15.89:0/1595426130 shutdown_connections 2023-10-10T20:41:42.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.599+0000 7f3da3d04700 1 -- 172.21.15.89:0/1595426130 wait complete. 2023-10-10T20:41:42.613 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls' 2023-10-10T20:41:42.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.726+0000 7f86249f5700 1 Processor -- start 2023-10-10T20:41:42.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.726+0000 7f86249f5700 1 -- start start 2023-10-10T20:41:42.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.727+0000 7f86249f5700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8620159890 0x7f862015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:42.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.727+0000 7f86249f5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8620159c60 0x7f862015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:42.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.727+0000 7f86249f5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f862015db10 0x7f8620069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:42.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.727+0000 7f86249f5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f862015df50 con 0x7f8620159c60 2023-10-10T20:41:42.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.727+0000 7f86249f5700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f862015a160 con 0x7f8620159890 2023-10-10T20:41:42.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.727+0000 7f86249f5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86200d13a0 con 0x7f862015db10 2023-10-10T20:41:42.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.727+0000 7f8615d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8620159c60 0x7f862015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:42.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.727+0000 7f8615d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8620159c60 0x7f862015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38384/0 (socket says 172.21.15.89:38384) 2023-10-10T20:41:42.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.727+0000 7f8615d9b700 1 -- 172.21.15.89:0/3673188355 learned_addr learned my addr 172.21.15.89:0/3673188355 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:42.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.727+0000 7f861ed9d700 1 --2- 172.21.15.89:0/3673188355 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f862015db10 0x7f8620069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:42.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.727+0000 7f861e59c700 1 --2- 172.21.15.89:0/3673188355 >> v2:172.21.15.111:3300/0 conn(0x7f8620159890 0x7f862015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:42.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.727+0000 7f8615d9b700 1 -- 172.21.15.89:0/3673188355 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f862015db10 msgr2=0x7f8620069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:42.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.727+0000 7f8615d9b700 1 --2- 172.21.15.89:0/3673188355 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f862015db10 0x7f8620069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.727+0000 7f8615d9b700 1 -- 172.21.15.89:0/3673188355 >> v2:172.21.15.111:3300/0 conn(0x7f8620159890 msgr2=0x7f862015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:42.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.727+0000 7f8615d9b700 1 --2- 172.21.15.89:0/3673188355 >> v2:172.21.15.111:3300/0 conn(0x7f8620159890 0x7f862015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.727+0000 7f8615d9b700 1 -- 172.21.15.89:0/3673188355 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862015e140 con 0x7f8620159c60 2023-10-10T20:41:42.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.728+0000 7f8615d9b700 1 --2- 172.21.15.89:0/3673188355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8620159c60 0x7f862015d5d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f86200d1090 tx=0x7f8608009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5075be658553f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:42.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.728+0000 7f861dd9b700 1 -- 172.21.15.89:0/3673188355 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860800b860 con 0x7f8620159c60 2023-10-10T20:41:42.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.728+0000 7f861dd9b700 1 -- 172.21.15.89:0/3673188355 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860800b9c0 con 0x7f8620159c60 2023-10-10T20:41:42.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.728+0000 7f861dd9b700 1 -- 172.21.15.89:0/3673188355 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860800bd60 con 0x7f8620159c60 2023-10-10T20:41:42.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.728+0000 7f86249f5700 1 -- 172.21.15.89:0/3673188355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8620159c60 msgr2=0x7f862015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:42.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.728+0000 7f86249f5700 1 --2- 172.21.15.89:0/3673188355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8620159c60 0x7f862015d5d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f86200d1090 tx=0x7f8608009250 comp rx=0 tx=0).stop 2023-10-10T20:41:42.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.728+0000 7f86249f5700 1 -- 172.21.15.89:0/3673188355 shutdown_connections 2023-10-10T20:41:42.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.728+0000 7f86249f5700 1 --2- 172.21.15.89:0/3673188355 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f862015db10 0x7f8620069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.728+0000 7f86249f5700 1 --2- 172.21.15.89:0/3673188355 >> v2:172.21.15.111:3300/0 conn(0x7f8620159890 0x7f862015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.728+0000 7f86249f5700 1 --2- 172.21.15.89:0/3673188355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8620159c60 0x7f862015d5d0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.728+0000 7f86249f5700 1 -- 172.21.15.89:0/3673188355 >> 172.21.15.89:0/3673188355 conn(0x7f8620068e20 msgr2=0x7f86200d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:42.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.728+0000 7f86249f5700 1 -- 172.21.15.89:0/3673188355 shutdown_connections 2023-10-10T20:41:42.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.728+0000 7f86249f5700 1 -- 172.21.15.89:0/3673188355 wait complete. 2023-10-10T20:41:42.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.729+0000 7f86249f5700 1 Processor -- start 2023-10-10T20:41:42.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.729+0000 7f86249f5700 1 -- start start 2023-10-10T20:41:42.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.730+0000 7f86249f5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8620159890 0x7f86200b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:42.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.730+0000 7f86249f5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8620159c60 0x7f86200b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:42.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.730+0000 7f86249f5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862015db10 0x7f86200b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:42.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.730+0000 7f86249f5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86200b05d0 con 0x7f862015db10 2023-10-10T20:41:42.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.730+0000 7f86249f5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86200b46a0 con 0x7f8620159c60 2023-10-10T20:41:42.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.730+0000 7f86249f5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86200b4980 con 0x7f8620159890 2023-10-10T20:41:42.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.730+0000 7f861ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862015db10 0x7f86200b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:42.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.730+0000 7f861ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862015db10 0x7f86200b5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38398/0 (socket says 172.21.15.89:38398) 2023-10-10T20:41:42.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.730+0000 7f861ed9d700 1 -- 172.21.15.89:0/276465457 learned_addr learned my addr 172.21.15.89:0/276465457 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:42.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.730+0000 7f8615d9b700 1 --2- 172.21.15.89:0/276465457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8620159c60 0x7f86200b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:42.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.730+0000 7f861e59c700 1 --2- 172.21.15.89:0/276465457 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8620159890 0x7f86200b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:42.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.730+0000 7f861ed9d700 1 -- 172.21.15.89:0/276465457 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8620159890 msgr2=0x7f86200b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:42.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.731+0000 7f861ed9d700 1 --2- 172.21.15.89:0/276465457 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8620159890 0x7f86200b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.731+0000 7f861ed9d700 1 -- 172.21.15.89:0/276465457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8620159c60 msgr2=0x7f86200b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:42.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.731+0000 7f861ed9d700 1 --2- 172.21.15.89:0/276465457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8620159c60 0x7f86200b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:42.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.731+0000 7f861ed9d700 1 -- 172.21.15.89:0/276465457 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86080094f0 con 0x7f862015db10 2023-10-10T20:41:42.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.731+0000 7f8615d9b700 1 --2- 172.21.15.89:0/276465457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8620159c60 0x7f86200b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:42.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.731+0000 7f861ed9d700 1 --2- 172.21.15.89:0/276465457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862015db10 0x7f86200b5550 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f8618009fa0 tx=0x7f8618009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:42.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.731+0000 7f8616ffd700 1 -- 172.21.15.89:0/276465457 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861800ed20 con 0x7f862015db10 2023-10-10T20:41:42.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.731+0000 7f8616ffd700 1 -- 172.21.15.89:0/276465457 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861800ee80 con 0x7f862015db10 2023-10-10T20:41:42.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.731+0000 7f86249f5700 1 -- 172.21.15.89:0/276465457 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86200b07e0 con 0x7f862015db10 2023-10-10T20:41:42.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.731+0000 7f8616ffd700 1 -- 172.21.15.89:0/276465457 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618014670 con 0x7f862015db10 2023-10-10T20:41:42.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.731+0000 7f86249f5700 1 -- 172.21.15.89:0/276465457 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86200c8560 con 0x7f862015db10 2023-10-10T20:41:42.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.732+0000 7f86249f5700 1 -- 172.21.15.89:0/276465457 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86200b4ba0 con 0x7f862015db10 2023-10-10T20:41:42.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.732+0000 7f8616ffd700 1 -- 172.21.15.89:0/276465457 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f86180149e0 con 0x7f862015db10 2023-10-10T20:41:42.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.732+0000 7f8616ffd700 1 --2- 172.21.15.89:0/276465457 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8600067320 0x7f86000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:42.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.733+0000 7f8616ffd700 1 -- 172.21.15.89:0/276465457 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5111+0+0 (secure 0 0 0) 0x7f8618085210 con 0x7f862015db10 2023-10-10T20:41:42.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.733+0000 7f861e59c700 1 --2- 172.21.15.89:0/276465457 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8600067320 0x7f86000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:42.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.733+0000 7f861e59c700 1 --2- 172.21.15.89:0/276465457 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8600067320 0x7f86000697d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8610006fd0 tx=0x7f8610008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:42.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:42.737+0000 7f8616ffd700 1 -- 172.21.15.89:0/276465457 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86180520f0 con 0x7f862015db10 2023-10-10T20:41:42.770 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:41:42.773 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:41:43.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.001+0000 7f86249f5700 1 -- 172.21.15.89:0/276465457 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]} v 0) v1 -- 0x7f86200b0d00 con 0x7f862015db10 2023-10-10T20:41:43.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.262+0000 7f8616ffd700 1 -- 172.21.15.89:0/276465457 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]}]=0 v351) v1 ==== 131+0+0 (secure 0 0 0) 0x7f86200b0d00 con 0x7f862015db10 2023-10-10T20:41:43.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.264+0000 7f86249f5700 1 -- 172.21.15.89:0/276465457 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8600067320 msgr2=0x7f86000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:43.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.264+0000 7f86249f5700 1 --2- 172.21.15.89:0/276465457 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8600067320 0x7f86000697d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8610006fd0 tx=0x7f8610008040 comp rx=0 tx=0).stop 2023-10-10T20:41:43.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.264+0000 7f86249f5700 1 -- 172.21.15.89:0/276465457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862015db10 msgr2=0x7f86200b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:43.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.264+0000 7f86249f5700 1 --2- 172.21.15.89:0/276465457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862015db10 0x7f86200b5550 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f8618009fa0 tx=0x7f8618009900 comp rx=0 tx=0).stop 2023-10-10T20:41:43.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.264+0000 7f86249f5700 1 -- 172.21.15.89:0/276465457 shutdown_connections 2023-10-10T20:41:43.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.264+0000 7f86249f5700 1 --2- 172.21.15.89:0/276465457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862015db10 0x7f86200b5550 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:43.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.264+0000 7f86249f5700 1 --2- 172.21.15.89:0/276465457 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8620159890 0x7f86200b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:43.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.264+0000 7f86249f5700 1 --2- 172.21.15.89:0/276465457 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8600067320 0x7f86000697d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:43.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.264+0000 7f86249f5700 1 --2- 172.21.15.89:0/276465457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8620159c60 0x7f86200b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:43.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.264+0000 7f86249f5700 1 -- 172.21.15.89:0/276465457 >> 172.21.15.89:0/276465457 conn(0x7f8620068e20 msgr2=0x7f86200cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:43.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.264+0000 7f86249f5700 1 -- 172.21.15.89:0/276465457 shutdown_connections 2023-10-10T20:41:43.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.264+0000 7f86249f5700 1 -- 172.21.15.89:0/276465457 wait complete. 2023-10-10T20:41:43.278 INFO:tasks.cephfs_test_runner:That the existing and the newly created volumes can be listed and ... ok 2023-10-10T20:41:43.279 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename (tasks.cephfs.test_volumes.TestVolumes) That volume, its file system and pools, can be renamed. 2023-10-10T20:41:43.279 INFO:tasks.cephfs_test_runner:test_volume_rename (tasks.cephfs.test_volumes.TestVolumes) 2023-10-10T20:41:43.280 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename' 2023-10-10T20:41:43.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.392+0000 7f47d9552700 1 Processor -- start 2023-10-10T20:41:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.392+0000 7f47d9552700 1 -- start start 2023-10-10T20:41:43.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.393+0000 7f47d9552700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f47d40cbf30 0x7f47d40ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:43.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.393+0000 7f47d9552700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f47d40ccf90 0x7f47d40b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:43.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.393+0000 7f47d9552700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f47d415a270 0x7f47d40b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:43.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.393+0000 7f47d9552700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d4156bf0 con 0x7f47d40cbf30 2023-10-10T20:41:43.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.393+0000 7f47d9552700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d40ae950 con 0x7f47d40ccf90 2023-10-10T20:41:43.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.393+0000 7f47d9552700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40cc300 con 0x7f47d415a270 2023-10-10T20:41:43.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.393+0000 7f47d2ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f47d40cbf30 0x7f47d40ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:43.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.393+0000 7f47d2ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f47d40cbf30 0x7f47d40ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38408/0 (socket says 172.21.15.89:38408) 2023-10-10T20:41:43.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.393+0000 7f47d2ffd700 1 -- 172.21.15.89:0/2995863380 learned_addr learned my addr 172.21.15.89:0/2995863380 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:43.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.394+0000 7f47d37fe700 1 --2- 172.21.15.89:0/2995863380 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f47d415a270 0x7f47d40b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:43.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.394+0000 7f47ca7fc700 1 --2- 172.21.15.89:0/2995863380 >> v2:172.21.15.111:3300/0 conn(0x7f47d40ccf90 0x7f47d40b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:43.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.394+0000 7f47d2ffd700 1 -- 172.21.15.89:0/2995863380 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f47d415a270 msgr2=0x7f47d40b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:43.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.394+0000 7f47d2ffd700 1 --2- 172.21.15.89:0/2995863380 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f47d415a270 0x7f47d40b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:43.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.394+0000 7f47d2ffd700 1 -- 172.21.15.89:0/2995863380 >> v2:172.21.15.111:3300/0 conn(0x7f47d40ccf90 msgr2=0x7f47d40b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:43.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.394+0000 7f47d2ffd700 1 --2- 172.21.15.89:0/2995863380 >> v2:172.21.15.111:3300/0 conn(0x7f47d40ccf90 0x7f47d40b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:43.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.394+0000 7f47d2ffd700 1 -- 172.21.15.89:0/2995863380 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d40ad370 con 0x7f47d40cbf30 2023-10-10T20:41:43.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.394+0000 7f47d2ffd700 1 --2- 172.21.15.89:0/2995863380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47d40cbf30 0x7f47d40ae410 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f47d406d670 tx=0x7f47c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8694453ec64d5527 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:43.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.394+0000 7f47d27fc700 1 -- 172.21.15.89:0/2995863380 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c400b640 con 0x7f47d40cbf30 2023-10-10T20:41:43.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.394+0000 7f47d27fc700 1 -- 172.21.15.89:0/2995863380 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c400b7a0 con 0x7f47d40cbf30 2023-10-10T20:41:43.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.394+0000 7f47d27fc700 1 -- 172.21.15.89:0/2995863380 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c400bb40 con 0x7f47d40cbf30 2023-10-10T20:41:43.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.395+0000 7f47d9552700 1 -- 172.21.15.89:0/2995863380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47d40cbf30 msgr2=0x7f47d40ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:43.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.395+0000 7f47d9552700 1 --2- 172.21.15.89:0/2995863380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47d40cbf30 0x7f47d40ae410 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f47d406d670 tx=0x7f47c4009250 comp rx=0 tx=0).stop 2023-10-10T20:41:43.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.395+0000 7f47d9552700 1 -- 172.21.15.89:0/2995863380 shutdown_connections 2023-10-10T20:41:43.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.395+0000 7f47d9552700 1 --2- 172.21.15.89:0/2995863380 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f47d415a270 0x7f47d40b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:43.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.395+0000 7f47d9552700 1 --2- 172.21.15.89:0/2995863380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47d40cbf30 0x7f47d40ae410 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:43.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.395+0000 7f47d9552700 1 --2- 172.21.15.89:0/2995863380 >> v2:172.21.15.111:3300/0 conn(0x7f47d40ccf90 0x7f47d40b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:43.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.395+0000 7f47d9552700 1 -- 172.21.15.89:0/2995863380 >> 172.21.15.89:0/2995863380 conn(0x7f47d406a320 msgr2=0x7f47d40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:43.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.395+0000 7f47d9552700 1 -- 172.21.15.89:0/2995863380 shutdown_connections 2023-10-10T20:41:43.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.395+0000 7f47d9552700 1 -- 172.21.15.89:0/2995863380 wait complete. 2023-10-10T20:41:43.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.396+0000 7f47d9552700 1 Processor -- start 2023-10-10T20:41:43.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.396+0000 7f47d9552700 1 -- start start 2023-10-10T20:41:43.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47d9552700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47d40cbf30 0x7f47d41686a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:43.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47d9552700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47d40ccf90 0x7f47d414e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:43.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47d9552700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f47d415a270 0x7f47d40abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:43.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47d9552700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d41679b0 con 0x7f47d40ccf90 2023-10-10T20:41:43.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47d9552700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d4150200 con 0x7f47d40cbf30 2023-10-10T20:41:43.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47d9552700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d4168210 con 0x7f47d415a270 2023-10-10T20:41:43.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47ca7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47d40ccf90 0x7f47d414e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:43.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47ca7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47d40ccf90 0x7f47d414e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38420/0 (socket says 172.21.15.89:38420) 2023-10-10T20:41:43.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47ca7fc700 1 -- 172.21.15.89:0/2069249381 learned_addr learned my addr 172.21.15.89:0/2069249381 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:43.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47d2ffd700 1 --2- 172.21.15.89:0/2069249381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47d40cbf30 0x7f47d41686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:43.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47d37fe700 1 --2- 172.21.15.89:0/2069249381 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f47d415a270 0x7f47d40abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:43.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47ca7fc700 1 -- 172.21.15.89:0/2069249381 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f47d415a270 msgr2=0x7f47d40abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:43.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47ca7fc700 1 --2- 172.21.15.89:0/2069249381 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f47d415a270 0x7f47d40abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:43.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47ca7fc700 1 -- 172.21.15.89:0/2069249381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47d40cbf30 msgr2=0x7f47d41686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:43.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47ca7fc700 1 --2- 172.21.15.89:0/2069249381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47d40cbf30 0x7f47d41686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:43.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47ca7fc700 1 -- 172.21.15.89:0/2069249381 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c40094f0 con 0x7f47d40ccf90 2023-10-10T20:41:43.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47ca7fc700 1 --2- 172.21.15.89:0/2069249381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47d40ccf90 0x7f47d414e180 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00ec80 tx=0x7f47bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:43.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47cb7fe700 1 -- 172.21.15.89:0/2069249381 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc00c850 con 0x7f47d40ccf90 2023-10-10T20:41:43.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47cb7fe700 1 -- 172.21.15.89:0/2069249381 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47bc00c9b0 con 0x7f47d40ccf90 2023-10-10T20:41:43.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47d9552700 1 -- 172.21.15.89:0/2069249381 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d414e6c0 con 0x7f47d40ccf90 2023-10-10T20:41:43.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47cb7fe700 1 -- 172.21.15.89:0/2069249381 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc00cd50 con 0x7f47d40ccf90 2023-10-10T20:41:43.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.397+0000 7f47d9552700 1 -- 172.21.15.89:0/2069249381 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d414ec20 con 0x7f47d40ccf90 2023-10-10T20:41:43.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.398+0000 7f47d9552700 1 -- 172.21.15.89:0/2069249381 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d404c3c0 con 0x7f47d40ccf90 2023-10-10T20:41:43.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.399+0000 7f47cb7fe700 1 -- 172.21.15.89:0/2069249381 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f47bc013680 con 0x7f47d40ccf90 2023-10-10T20:41:43.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.399+0000 7f47cb7fe700 1 --2- 172.21.15.89:0/2069249381 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f47b4067320 0x7f47b40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:43.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.399+0000 7f47cb7fe700 1 -- 172.21.15.89:0/2069249381 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5111+0+0 (secure 0 0 0) 0x7f47bc084ca0 con 0x7f47d40ccf90 2023-10-10T20:41:43.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.399+0000 7f47d2ffd700 1 --2- 172.21.15.89:0/2069249381 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f47b4067320 0x7f47b40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:43.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.399+0000 7f47d2ffd700 1 --2- 172.21.15.89:0/2069249381 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f47b4067320 0x7f47b40697d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f47d406d670 tx=0x7f47c4017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:43.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.403+0000 7f47cb7fe700 1 -- 172.21.15.89:0/2069249381 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47bc051b80 con 0x7f47d40ccf90 2023-10-10T20:41:43.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:43.664+0000 7f47d9552700 1 -- 172.21.15.89:0/2069249381 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]} v 0) v1 -- 0x7f47d4168350 con 0x7f47d40ccf90 2023-10-10T20:41:44.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.265+0000 7f47cb7fe700 1 -- 172.21.15.89:0/2069249381 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]}]=0 v352) v1 ==== 138+0+0 (secure 0 0 0) 0x7f47bc0551a0 con 0x7f47d40ccf90 2023-10-10T20:41:44.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.267+0000 7f47d9552700 1 -- 172.21.15.89:0/2069249381 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f47b4067320 msgr2=0x7f47b40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:44.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.267+0000 7f47d9552700 1 --2- 172.21.15.89:0/2069249381 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f47b4067320 0x7f47b40697d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f47d406d670 tx=0x7f47c4017000 comp rx=0 tx=0).stop 2023-10-10T20:41:44.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.267+0000 7f47d9552700 1 -- 172.21.15.89:0/2069249381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47d40ccf90 msgr2=0x7f47d414e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:44.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.267+0000 7f47d9552700 1 --2- 172.21.15.89:0/2069249381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47d40ccf90 0x7f47d414e180 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00ec80 tx=0x7f47bc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.267+0000 7f47d9552700 1 -- 172.21.15.89:0/2069249381 shutdown_connections 2023-10-10T20:41:44.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.267+0000 7f47d9552700 1 --2- 172.21.15.89:0/2069249381 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f47d415a270 0x7f47d40abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.267+0000 7f47d9552700 1 --2- 172.21.15.89:0/2069249381 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f47b4067320 0x7f47b40697d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.267+0000 7f47d9552700 1 --2- 172.21.15.89:0/2069249381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47d40cbf30 0x7f47d41686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.267+0000 7f47d9552700 1 --2- 172.21.15.89:0/2069249381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47d40ccf90 0x7f47d414e180 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.268+0000 7f47d9552700 1 -- 172.21.15.89:0/2069249381 >> 172.21.15.89:0/2069249381 conn(0x7f47d406a320 msgr2=0x7f47d406f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:44.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.268+0000 7f47d9552700 1 -- 172.21.15.89:0/2069249381 shutdown_connections 2023-10-10T20:41:44.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.268+0000 7f47d9552700 1 -- 172.21.15.89:0/2069249381 wait complete. 2023-10-10T20:41:44.282 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:41:44.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.397+0000 7f034edd1700 1 Processor -- start 2023-10-10T20:41:44.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.397+0000 7f034edd1700 1 -- start start 2023-10-10T20:41:44.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.398+0000 7f034edd1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0348141500 0x7f03480c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:44.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.398+0000 7f034edd1700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f034806c0a0 0x7f03480c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:44.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.398+0000 7f034edd1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034815c2a0 con 0x7f034806bcd0 2023-10-10T20:41:44.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.398+0000 7f034edd1700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03480c9880 con 0x7f034806c0a0 2023-10-10T20:41:44.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.398+0000 7f034edd1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034815b690 con 0x7f0348141500 2023-10-10T20:41:44.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.398+0000 7f034cb6d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f034806bcd0 0x7f0348140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53970/0 (socket says 172.21.15.89:53970) 2023-10-10T20:41:44.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.398+0000 7f034cb6d700 1 -- 172.21.15.89:0/1460972843 learned_addr learned my addr 172.21.15.89:0/1460972843 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:44.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f03477fe700 1 -- 172.21.15.89:0/1460972843 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628525598 0 0) 0x7f034815c2a0 con 0x7f034806bcd0 2023-10-10T20:41:44.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f03477fe700 1 -- 172.21.15.89:0/1460972843 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0330003570 con 0x7f034806bcd0 2023-10-10T20:41:44.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f0347fff700 1 --2- 172.21.15.89:0/1460972843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0348141500 0x7f03480c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:44.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f034d36e700 1 --2- 172.21.15.89:0/1460972843 >> v2:172.21.15.111:3300/0 conn(0x7f034806c0a0 0x7f03480c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:44.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f03477fe700 1 -- 172.21.15.89:0/1460972843 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3227401556 0 0) 0x7f0330003570 con 0x7f034806bcd0 2023-10-10T20:41:44.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f03477fe700 1 -- 172.21.15.89:0/1460972843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0348141500 msgr2=0x7f03480c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:44.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f03477fe700 1 --2- 172.21.15.89:0/1460972843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0348141500 0x7f03480c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f03477fe700 1 -- 172.21.15.89:0/1460972843 >> v2:172.21.15.111:3300/0 conn(0x7f034806c0a0 msgr2=0x7f03480c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:44.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f03477fe700 1 --2- 172.21.15.89:0/1460972843 >> v2:172.21.15.111:3300/0 conn(0x7f034806c0a0 0x7f03480c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f03477fe700 1 -- 172.21.15.89:0/1460972843 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034815b970 con 0x7f034806bcd0 2023-10-10T20:41:44.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f03477fe700 1 -- 172.21.15.89:0/1460972843 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f03380017f0 con 0x7f034806bcd0 2023-10-10T20:41:44.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f03477fe700 1 -- 172.21.15.89:0/1460972843 >> v1:172.21.15.89:6789/0 conn(0x7f034806bcd0 legacy=0x7f0348140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:44.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f03477fe700 1 --2- 172.21.15.89:0/1460972843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0330003a60 0x7f0330003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:44.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f03477fe700 1 --2- 172.21.15.89:0/1460972843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0330004810 0x7f0330004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:44.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f03477fe700 1 --2- 172.21.15.89:0/1460972843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0348141500 0x7f03300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:44.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f03477fe700 1 -- 172.21.15.89:0/1460972843 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03480c9880 con 0x7f0330003a60 2023-10-10T20:41:44.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f03477fe700 1 -- 172.21.15.89:0/1460972843 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034815b690 con 0x7f0330004810 2023-10-10T20:41:44.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f03477fe700 1 -- 172.21.15.89:0/1460972843 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033000ba60 con 0x7f0348141500 2023-10-10T20:41:44.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f034cb6d700 1 --2- 172.21.15.89:0/1460972843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0330003a60 0x7f0330003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:44.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f0347fff700 1 --2- 172.21.15.89:0/1460972843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0330004810 0x7f0330004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:44.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f034d36e700 1 --2- 172.21.15.89:0/1460972843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0348141500 0x7f03300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:44.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f034cb6d700 1 -- 172.21.15.89:0/1460972843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0348141500 msgr2=0x7f03300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:44.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f034cb6d700 1 --2- 172.21.15.89:0/1460972843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0348141500 0x7f03300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f034cb6d700 1 -- 172.21.15.89:0/1460972843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0330004810 msgr2=0x7f0330004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:44.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f034cb6d700 1 --2- 172.21.15.89:0/1460972843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0330004810 0x7f0330004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f034cb6d700 1 -- 172.21.15.89:0/1460972843 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0330003570 con 0x7f0330003a60 2023-10-10T20:41:44.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.399+0000 7f034cb6d700 1 --2- 172.21.15.89:0/1460972843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0330003a60 0x7f0330003e70 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f033800a9a0 tx=0x7f034806e470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff0a8ed4204bc58 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:44.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.400+0000 7f03477fe700 1 -- 172.21.15.89:0/1460972843 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03380070d0 con 0x7f0330003a60 2023-10-10T20:41:44.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.400+0000 7f03477fe700 1 -- 172.21.15.89:0/1460972843 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0338007230 con 0x7f0330003a60 2023-10-10T20:41:44.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.400+0000 7f03477fe700 1 -- 172.21.15.89:0/1460972843 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03380075f0 con 0x7f0330003a60 2023-10-10T20:41:44.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.400+0000 7f034edd1700 1 -- 172.21.15.89:0/1460972843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0330003a60 msgr2=0x7f0330003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:44.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.400+0000 7f034edd1700 1 --2- 172.21.15.89:0/1460972843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0330003a60 0x7f0330003e70 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f033800a9a0 tx=0x7f034806e470 comp rx=0 tx=0).stop 2023-10-10T20:41:44.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.400+0000 7f034d36e700 1 -- 172.21.15.89:0/1460972843 reap_dead start 2023-10-10T20:41:44.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.400+0000 7f034edd1700 1 -- 172.21.15.89:0/1460972843 shutdown_connections 2023-10-10T20:41:44.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.400+0000 7f034edd1700 1 -- 172.21.15.89:0/1460972843 >> 172.21.15.89:0/1460972843 conn(0x7f03480ce600 msgr2=0x7f03480cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:44.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.400+0000 7f034edd1700 1 -- 172.21.15.89:0/1460972843 shutdown_connections 2023-10-10T20:41:44.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.400+0000 7f034edd1700 1 -- 172.21.15.89:0/1460972843 wait complete. 2023-10-10T20:41:44.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.401+0000 7f034edd1700 1 Processor -- start 2023-10-10T20:41:44.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.401+0000 7f034edd1700 1 -- start start 2023-10-10T20:41:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.402+0000 7f034edd1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0330003a60 0x7f03480c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:44.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.402+0000 7f034edd1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03480c60c0 0x7f034813f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:44.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.402+0000 7f034edd1700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0348167de0 0x7f03480c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:44.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.402+0000 7f034edd1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0348159560 con 0x7f03480c60c0 2023-10-10T20:41:44.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.402+0000 7f034edd1700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0348159e00 con 0x7f0348167de0 2023-10-10T20:41:44.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.402+0000 7f034edd1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03480c6830 con 0x7f0330003a60 2023-10-10T20:41:44.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.402+0000 7f0347fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03480c60c0 0x7f034813f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:44.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.402+0000 7f0347fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03480c60c0 0x7f034813f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38448/0 (socket says 172.21.15.89:38448) 2023-10-10T20:41:44.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.402+0000 7f0347fff700 1 -- 172.21.15.89:0/423388166 learned_addr learned my addr 172.21.15.89:0/423388166 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:44.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.402+0000 7f034cb6d700 1 --2- 172.21.15.89:0/423388166 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0330003a60 0x7f03480c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:44.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.402+0000 7f034d36e700 1 --2- 172.21.15.89:0/423388166 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0348167de0 0x7f03480c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:44.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.402+0000 7f0347fff700 1 -- 172.21.15.89:0/423388166 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0330003a60 msgr2=0x7f03480c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:44.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.402+0000 7f0347fff700 1 --2- 172.21.15.89:0/423388166 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0330003a60 0x7f03480c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.402+0000 7f0347fff700 1 -- 172.21.15.89:0/423388166 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0348167de0 msgr2=0x7f03480c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:44.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.402+0000 7f0347fff700 1 --2- 172.21.15.89:0/423388166 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0348167de0 0x7f03480c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.402+0000 7f0347fff700 1 -- 172.21.15.89:0/423388166 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03481597b0 con 0x7f03480c60c0 2023-10-10T20:41:44.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.402+0000 7f034cb6d700 1 --2- 172.21.15.89:0/423388166 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0330003a60 0x7f03480c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:44.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.402+0000 7f034d36e700 1 --2- 172.21.15.89:0/423388166 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0348167de0 0x7f03480c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:44.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.403+0000 7f0347fff700 1 --2- 172.21.15.89:0/423388166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03480c60c0 0x7f034813f3c0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f033c011d70 tx=0x7f033c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:44.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.403+0000 7f0344ff9700 1 -- 172.21.15.89:0/423388166 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c009e20 con 0x7f03480c60c0 2023-10-10T20:41:44.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.403+0000 7f0344ff9700 1 -- 172.21.15.89:0/423388166 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c00fcd0 con 0x7f03480c60c0 2023-10-10T20:41:44.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.403+0000 7f034edd1700 1 -- 172.21.15.89:0/423388166 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0348159a90 con 0x7f03480c60c0 2023-10-10T20:41:44.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.403+0000 7f0344ff9700 1 -- 172.21.15.89:0/423388166 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c014650 con 0x7f03480c60c0 2023-10-10T20:41:44.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.403+0000 7f034edd1700 1 -- 172.21.15.89:0/423388166 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034813f900 con 0x7f03480c60c0 2023-10-10T20:41:44.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.404+0000 7f034edd1700 1 -- 172.21.15.89:0/423388166 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034804c3c0 con 0x7f03480c60c0 2023-10-10T20:41:44.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.404+0000 7f0344ff9700 1 -- 172.21.15.89:0/423388166 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f033c014a80 con 0x7f03480c60c0 2023-10-10T20:41:44.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.404+0000 7f0344ff9700 1 --2- 172.21.15.89:0/423388166 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0334067320 0x7f03340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:44.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.404+0000 7f0344ff9700 1 -- 172.21.15.89:0/423388166 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5111+0+0 (secure 0 0 0) 0x7f033c018030 con 0x7f03480c60c0 2023-10-10T20:41:44.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.404+0000 7f034cb6d700 1 --2- 172.21.15.89:0/423388166 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0334067320 0x7f03340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:44.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.405+0000 7f034cb6d700 1 --2- 172.21.15.89:0/423388166 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0334067320 0x7f03340697d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f033800a9a0 tx=0x7f0338016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:44.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.408+0000 7f0344ff9700 1 -- 172.21.15.89:0/423388166 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f033c052480 con 0x7f03480c60c0 2023-10-10T20:41:44.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.591+0000 7f034edd1700 1 -- 172.21.15.89:0/423388166 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f034815bc50 con 0x7f03480c60c0 2023-10-10T20:41:44.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.596+0000 7f0344ff9700 1 -- 172.21.15.89:0/423388166 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f033c055aa0 con 0x7f03480c60c0 2023-10-10T20:41:44.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.599+0000 7f034edd1700 1 -- 172.21.15.89:0/423388166 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0334067320 msgr2=0x7f03340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:44.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.599+0000 7f034edd1700 1 --2- 172.21.15.89:0/423388166 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0334067320 0x7f03340697d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f033800a9a0 tx=0x7f0338016000 comp rx=0 tx=0).stop 2023-10-10T20:41:44.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.599+0000 7f034edd1700 1 -- 172.21.15.89:0/423388166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03480c60c0 msgr2=0x7f034813f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:44.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.599+0000 7f034edd1700 1 --2- 172.21.15.89:0/423388166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03480c60c0 0x7f034813f3c0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f033c011d70 tx=0x7f033c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.599+0000 7f034edd1700 1 -- 172.21.15.89:0/423388166 shutdown_connections 2023-10-10T20:41:44.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.599+0000 7f034edd1700 1 --2- 172.21.15.89:0/423388166 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0334067320 0x7f03340697d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.599+0000 7f034edd1700 1 --2- 172.21.15.89:0/423388166 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0348167de0 0x7f03480c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.599+0000 7f034edd1700 1 --2- 172.21.15.89:0/423388166 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0330003a60 0x7f03480c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.599+0000 7f034edd1700 1 --2- 172.21.15.89:0/423388166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03480c60c0 0x7f034813f3c0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.599+0000 7f034edd1700 1 -- 172.21.15.89:0/423388166 >> 172.21.15.89:0/423388166 conn(0x7f03480ce600 msgr2=0x7f034813fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:44.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.599+0000 7f034edd1700 1 -- 172.21.15.89:0/423388166 shutdown_connections 2023-10-10T20:41:44.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.599+0000 7f034edd1700 1 -- 172.21.15.89:0/423388166 wait complete. 2023-10-10T20:41:44.613 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:41:44.613 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:41:44.661 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:41:44.709 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:41:44.709 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:41:44.710 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:41:44.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.822+0000 7f48ea0b6700 1 Processor -- start 2023-10-10T20:41:44.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.822+0000 7f48ea0b6700 1 -- start start 2023-10-10T20:41:44.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.823+0000 7f48ea0b6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f48e40cbf30 0x7f48e40ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:44.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.823+0000 7f48ea0b6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e40ccf90 0x7f48e40b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:44.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.823+0000 7f48ea0b6700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e40cc300 con 0x7f48e415a270 2023-10-10T20:41:44.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.823+0000 7f48ea0b6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e4156bf0 con 0x7f48e40cbf30 2023-10-10T20:41:44.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.823+0000 7f48ea0b6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e40ae950 con 0x7f48e40ccf90 2023-10-10T20:41:44.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.823+0000 7f48e37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f48e40cbf30 0x7f48e40ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:44.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.824+0000 7f48e37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f48e40cbf30 0x7f48e40ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38456/0 (socket says 172.21.15.89:38456) 2023-10-10T20:41:44.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.824+0000 7f48e37fe700 1 -- 172.21.15.89:0/2158010501 learned_addr learned my addr 172.21.15.89:0/2158010501 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:44.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.824+0000 7f48e2ffd700 1 --2- 172.21.15.89:0/2158010501 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e40ccf90 0x7f48e40b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:44.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.824+0000 7f48e37fe700 1 -- 172.21.15.89:0/2158010501 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e40ccf90 msgr2=0x7f48e40b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:44.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.824+0000 7f48e37fe700 1 --2- 172.21.15.89:0/2158010501 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e40ccf90 0x7f48e40b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.824+0000 7f48e37fe700 1 -- 172.21.15.89:0/2158010501 >> v1:172.21.15.111:6789/0 conn(0x7f48e415a270 legacy=0x7f48e40c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:44.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.824+0000 7f48e37fe700 1 -- 172.21.15.89:0/2158010501 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e40ad370 con 0x7f48e40cbf30 2023-10-10T20:41:44.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.824+0000 7f48e37fe700 1 --2- 172.21.15.89:0/2158010501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e40cbf30 0x7f48e40ae410 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f48d40099e0 tx=0x7f48d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5878fb837be70c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:44.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.824+0000 7f48e27fc700 1 -- 172.21.15.89:0/2158010501 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d400b7e0 con 0x7f48e40cbf30 2023-10-10T20:41:44.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.824+0000 7f48e27fc700 1 -- 172.21.15.89:0/2158010501 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d400b940 con 0x7f48e40cbf30 2023-10-10T20:41:44.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.824+0000 7f48e27fc700 1 -- 172.21.15.89:0/2158010501 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d400bce0 con 0x7f48e40cbf30 2023-10-10T20:41:44.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.825+0000 7f48ea0b6700 1 -- 172.21.15.89:0/2158010501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e40cbf30 msgr2=0x7f48e40ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:44.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.825+0000 7f48ea0b6700 1 --2- 172.21.15.89:0/2158010501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e40cbf30 0x7f48e40ae410 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f48d40099e0 tx=0x7f48d4009250 comp rx=0 tx=0).stop 2023-10-10T20:41:44.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.825+0000 7f48ea0b6700 1 -- 172.21.15.89:0/2158010501 shutdown_connections 2023-10-10T20:41:44.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.825+0000 7f48ea0b6700 1 --2- 172.21.15.89:0/2158010501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e40cbf30 0x7f48e40ae410 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.825+0000 7f48ea0b6700 1 --2- 172.21.15.89:0/2158010501 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e40ccf90 0x7f48e40b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.825+0000 7f48ea0b6700 1 -- 172.21.15.89:0/2158010501 >> 172.21.15.89:0/2158010501 conn(0x7f48e406a320 msgr2=0x7f48e40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:44.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.825+0000 7f48ea0b6700 1 -- 172.21.15.89:0/2158010501 shutdown_connections 2023-10-10T20:41:44.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.825+0000 7f48ea0b6700 1 -- 172.21.15.89:0/2158010501 wait complete. 2023-10-10T20:41:44.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.826+0000 7f48ea0b6700 1 Processor -- start 2023-10-10T20:41:44.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.826+0000 7f48ea0b6700 1 -- start start 2023-10-10T20:41:44.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48ea0b6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e40cbf30 0x7f48e4155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:44.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48ea0b6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e40ccf90 0x7f48e4167810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:44.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48ea0b6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48e415a270 0x7f48e40aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:44.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48ea0b6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e40aa9d0 con 0x7f48e40ccf90 2023-10-10T20:41:44.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48ea0b6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e40a9d50 con 0x7f48e415a270 2023-10-10T20:41:44.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48ea0b6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e40aa030 con 0x7f48e40cbf30 2023-10-10T20:41:44.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48e2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e40ccf90 0x7f48e4167810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:44.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48e2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e40ccf90 0x7f48e4167810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38460/0 (socket says 172.21.15.89:38460) 2023-10-10T20:41:44.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48e2ffd700 1 -- 172.21.15.89:0/2855928985 learned_addr learned my addr 172.21.15.89:0/2855928985 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:44.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48e37fe700 1 --2- 172.21.15.89:0/2855928985 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e40cbf30 0x7f48e4155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:44.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48e3fff700 1 --2- 172.21.15.89:0/2855928985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48e415a270 0x7f48e40aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:44.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48e2ffd700 1 -- 172.21.15.89:0/2855928985 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e40cbf30 msgr2=0x7f48e4155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:44.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48e2ffd700 1 --2- 172.21.15.89:0/2855928985 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e40cbf30 0x7f48e4155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48e2ffd700 1 -- 172.21.15.89:0/2855928985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48e415a270 msgr2=0x7f48e40aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:44.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48e2ffd700 1 --2- 172.21.15.89:0/2855928985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48e415a270 0x7f48e40aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:44.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48e2ffd700 1 -- 172.21.15.89:0/2855928985 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d40094f0 con 0x7f48e40ccf90 2023-10-10T20:41:44.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48e2ffd700 1 --2- 172.21.15.89:0/2855928985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e40ccf90 0x7f48e4167810 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f48d800d890 tx=0x7f48d8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:44.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48cb7fe700 1 -- 172.21.15.89:0/2855928985 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d80125f0 con 0x7f48e40ccf90 2023-10-10T20:41:44.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48cb7fe700 1 -- 172.21.15.89:0/2855928985 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d8012750 con 0x7f48e40ccf90 2023-10-10T20:41:44.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48ea0b6700 1 -- 172.21.15.89:0/2855928985 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e40aa310 con 0x7f48e40ccf90 2023-10-10T20:41:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48cb7fe700 1 -- 172.21.15.89:0/2855928985 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d8012af0 con 0x7f48e40ccf90 2023-10-10T20:41:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.827+0000 7f48ea0b6700 1 -- 172.21.15.89:0/2855928985 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e41562e0 con 0x7f48e40ccf90 2023-10-10T20:41:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.828+0000 7f48ea0b6700 1 -- 172.21.15.89:0/2855928985 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e4156500 con 0x7f48e40ccf90 2023-10-10T20:41:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.829+0000 7f48cb7fe700 1 -- 172.21.15.89:0/2855928985 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f48d801e460 con 0x7f48e40ccf90 2023-10-10T20:41:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.829+0000 7f48cb7fe700 1 --2- 172.21.15.89:0/2855928985 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f48d0067320 0x7f48d00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.829+0000 7f48cb7fe700 1 -- 172.21.15.89:0/2855928985 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5111+0+0 (secure 0 0 0) 0x7f48d8084f30 con 0x7f48e40ccf90 2023-10-10T20:41:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.829+0000 7f48e37fe700 1 --2- 172.21.15.89:0/2855928985 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f48d0067320 0x7f48d00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.830+0000 7f48e37fe700 1 --2- 172.21.15.89:0/2855928985 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f48d0067320 0x7f48d00697d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f48d40099e0 tx=0x7f48d4016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:44.833+0000 7f48cb7fe700 1 -- 172.21.15.89:0/2855928985 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48d8051e10 con 0x7f48e40ccf90 2023-10-10T20:41:45.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.059+0000 7f48ea0b6700 1 -- 172.21.15.89:0/2855928985 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48e404c3c0 con 0x7f48e40ccf90 2023-10-10T20:41:45.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.060+0000 7f48cb7fe700 1 -- 172.21.15.89:0/2855928985 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f48d8055430 con 0x7f48e40ccf90 2023-10-10T20:41:45.062 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:45.063 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":85,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6140,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/4001997216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":4001997216},{"type":"v1","addr":"172.21.15.111:6835","nonce":4001997216}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":6517,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/2658508419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2658508419},{"type":"v1","addr":"172.21.15.111:6834","nonce":2658508419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7431,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1872461145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1872461145},{"type":"v1","addr":"172.21.15.89:6837","nonce":1872461145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3243330447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3243330447},{"type":"v1","addr":"172.21.15.89:6835","nonce":3243330447}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[]} 2023-10-10T20:41:45.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.062+0000 7f48ea0b6700 1 -- 172.21.15.89:0/2855928985 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f48d0067320 msgr2=0x7f48d00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:45.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.062+0000 7f48ea0b6700 1 --2- 172.21.15.89:0/2855928985 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f48d0067320 0x7f48d00697d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f48d40099e0 tx=0x7f48d4016000 comp rx=0 tx=0).stop 2023-10-10T20:41:45.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.062+0000 7f48ea0b6700 1 -- 172.21.15.89:0/2855928985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e40ccf90 msgr2=0x7f48e4167810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:45.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.062+0000 7f48ea0b6700 1 --2- 172.21.15.89:0/2855928985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e40ccf90 0x7f48e4167810 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f48d800d890 tx=0x7f48d8008040 comp rx=0 tx=0).stop 2023-10-10T20:41:45.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.062+0000 7f48ea0b6700 1 -- 172.21.15.89:0/2855928985 shutdown_connections 2023-10-10T20:41:45.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.062+0000 7f48ea0b6700 1 --2- 172.21.15.89:0/2855928985 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f48d0067320 0x7f48d00697d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:45.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.062+0000 7f48ea0b6700 1 --2- 172.21.15.89:0/2855928985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48e415a270 0x7f48e40aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:45.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.062+0000 7f48ea0b6700 1 --2- 172.21.15.89:0/2855928985 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f48e40cbf30 0x7f48e4155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:45.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.062+0000 7f48ea0b6700 1 --2- 172.21.15.89:0/2855928985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e40ccf90 0x7f48e4167810 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:45.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.062+0000 7f48ea0b6700 1 -- 172.21.15.89:0/2855928985 >> 172.21.15.89:0/2855928985 conn(0x7f48e406a320 msgr2=0x7f48e40c9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:45.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.062+0000 7f48ea0b6700 1 -- 172.21.15.89:0/2855928985 shutdown_connections 2023-10-10T20:41:45.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.062+0000 7f48ea0b6700 1 -- 172.21.15.89:0/2855928985 wait complete. 2023-10-10T20:41:45.067 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 85 2023-10-10T20:41:45.076 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:41:45.076 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:41:45.077 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:41:45.077 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:41:45.077 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:41:45.077 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:41:45.078 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:41:45.078 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:41:45.078 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:41:45.079 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:41:45.079 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:41:45.079 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:41:45.079 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:41:45.080 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:41:45.080 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:41:45.080 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:41:45.085 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:41:45.086 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:41:45.087 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:41:45.087 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:41:45.090 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:41:45.090 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:41:45.091 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:41:45.091 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:41:45.128 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:41:45.129 DEBUG:teuthology.parallel:result is None 2023-10-10T20:41:45.129 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:41:45.130 DEBUG:teuthology.parallel:result is None 2023-10-10T20:41:45.132 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:41:45.133 DEBUG:teuthology.parallel:result is None 2023-10-10T20:41:45.133 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:41:45.134 DEBUG:teuthology.parallel:result is None 2023-10-10T20:41:45.134 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:41:45.201 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:41:45.201 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-10T20:41:45.212 INFO:tasks.ceph.mds.b.smithi111.stdout:starting mds.b at 2023-10-10T20:41:45.213 INFO:tasks.ceph.mds.d.smithi111.stdout:starting mds.d at 2023-10-10T20:41:45.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.253+0000 7f4d01637700 1 Processor -- start 2023-10-10T20:41:45.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.253+0000 7f4d01637700 1 -- start start 2023-10-10T20:41:45.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.254+0000 7f4d01637700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4cfc1597c0 0x7f4cfc159b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:45.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.254+0000 7f4d01637700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4cfc0af210 0x7f4cfc06e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:45.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.254+0000 7f4d01637700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cfc0ae390 con 0x7f4cfc0aedc0 2023-10-10T20:41:45.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.254+0000 7f4d01637700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cfc15e390 con 0x7f4cfc0af210 2023-10-10T20:41:45.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.254+0000 7f4d01637700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cfc0ae930 con 0x7f4cfc1597c0 2023-10-10T20:41:45.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.254+0000 7f4cfb7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4cfc0af210 0x7f4cfc06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:45.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.254+0000 7f4cfb7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4cfc0af210 0x7f4cfc06e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38514/0 (socket says 172.21.15.89:38514) 2023-10-10T20:41:45.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.254+0000 7f4cfb7fe700 1 -- 172.21.15.89:0/627395763 learned_addr learned my addr 172.21.15.89:0/627395763 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:45.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.255+0000 7f4cfaffd700 1 --2- 172.21.15.89:0/627395763 >> v2:172.21.15.111:3300/0 conn(0x7f4cfc1597c0 0x7f4cfc159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:45.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.255+0000 7f4cfb7fe700 1 -- 172.21.15.89:0/627395763 >> v2:172.21.15.111:3300/0 conn(0x7f4cfc1597c0 msgr2=0x7f4cfc159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:45.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.255+0000 7f4cfb7fe700 1 --2- 172.21.15.89:0/627395763 >> v2:172.21.15.111:3300/0 conn(0x7f4cfc1597c0 0x7f4cfc159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:45.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.255+0000 7f4cfb7fe700 1 -- 172.21.15.89:0/627395763 >> v1:172.21.15.111:6789/0 conn(0x7f4cfc0aedc0 legacy=0x7f4cfc15d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:45.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.255+0000 7f4cfb7fe700 1 -- 172.21.15.89:0/627395763 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc15db00 con 0x7f4cfc0af210 2023-10-10T20:41:45.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.255+0000 7f4cfb7fe700 1 --2- 172.21.15.89:0/627395763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cfc0af210 0x7f4cfc06e470 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc06db60 tx=0x7f4cf4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac68e0a68148e1ab server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:45.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.255+0000 7f4cf9ffb700 1 -- 172.21.15.89:0/627395763 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf400e070 con 0x7f4cfc0af210 2023-10-10T20:41:45.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.255+0000 7f4cf9ffb700 1 -- 172.21.15.89:0/627395763 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf400ba20 con 0x7f4cfc0af210 2023-10-10T20:41:45.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.255+0000 7f4cf9ffb700 1 -- 172.21.15.89:0/627395763 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf40092e0 con 0x7f4cfc0af210 2023-10-10T20:41:45.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.256+0000 7f4d01637700 1 -- 172.21.15.89:0/627395763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cfc0af210 msgr2=0x7f4cfc06e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:45.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.256+0000 7f4d01637700 1 --2- 172.21.15.89:0/627395763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cfc0af210 0x7f4cfc06e470 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc06db60 tx=0x7f4cf4009840 comp rx=0 tx=0).stop 2023-10-10T20:41:45.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.256+0000 7f4d01637700 1 -- 172.21.15.89:0/627395763 shutdown_connections 2023-10-10T20:41:45.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.256+0000 7f4d01637700 1 --2- 172.21.15.89:0/627395763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cfc0af210 0x7f4cfc06e470 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:45.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.256+0000 7f4d01637700 1 --2- 172.21.15.89:0/627395763 >> v2:172.21.15.111:3300/0 conn(0x7f4cfc1597c0 0x7f4cfc159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:45.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.256+0000 7f4d01637700 1 -- 172.21.15.89:0/627395763 >> 172.21.15.89:0/627395763 conn(0x7f4cfc0c9f30 msgr2=0x7f4cfc06dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:45.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.256+0000 7f4d01637700 1 -- 172.21.15.89:0/627395763 shutdown_connections 2023-10-10T20:41:45.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.256+0000 7f4d01637700 1 -- 172.21.15.89:0/627395763 wait complete. 2023-10-10T20:41:45.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.257+0000 7f4d01637700 1 Processor -- start 2023-10-10T20:41:45.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.257+0000 7f4d01637700 1 -- start start 2023-10-10T20:41:45.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4d01637700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cfc0aedc0 0x7f4cfc16d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:45.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4d01637700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cfc0af210 0x7f4cfc0c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:45.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4d01637700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cfc1597c0 0x7f4cfc1585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:45.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4d01637700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc16e2b0 con 0x7f4cfc1597c0 2023-10-10T20:41:45.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4d01637700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc169d10 con 0x7f4cfc0aedc0 2023-10-10T20:41:45.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4d01637700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc169fb0 con 0x7f4cfc0af210 2023-10-10T20:41:45.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4cfb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cfc1597c0 0x7f4cfc1585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:45.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4cfb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cfc1597c0 0x7f4cfc1585e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38526/0 (socket says 172.21.15.89:38526) 2023-10-10T20:41:45.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4cfb7fe700 1 -- 172.21.15.89:0/1862699618 learned_addr learned my addr 172.21.15.89:0/1862699618 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:45.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4cfaffd700 1 --2- 172.21.15.89:0/1862699618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cfc0aedc0 0x7f4cfc16d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:45.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4cfa7fc700 1 --2- 172.21.15.89:0/1862699618 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cfc0af210 0x7f4cfc0c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:45.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4cfb7fe700 1 -- 172.21.15.89:0/1862699618 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cfc0af210 msgr2=0x7f4cfc0c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:45.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4cfb7fe700 1 --2- 172.21.15.89:0/1862699618 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cfc0af210 0x7f4cfc0c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:45.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4cfb7fe700 1 -- 172.21.15.89:0/1862699618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cfc0aedc0 msgr2=0x7f4cfc16d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:45.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4cfb7fe700 1 --2- 172.21.15.89:0/1862699618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cfc0aedc0 0x7f4cfc16d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:45.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4cfb7fe700 1 -- 172.21.15.89:0/1862699618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf4009ae0 con 0x7f4cfc1597c0 2023-10-10T20:41:45.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4cfaffd700 1 --2- 172.21.15.89:0/1862699618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cfc0aedc0 0x7f4cfc16d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:45.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4cfb7fe700 1 --2- 172.21.15.89:0/1862699618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cfc1597c0 0x7f4cfc1585e0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40072f0 tx=0x7f4cf4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:45.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4ce2ffd700 1 -- 172.21.15.89:0/1862699618 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4008c60 con 0x7f4cfc1597c0 2023-10-10T20:41:45.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4ce2ffd700 1 -- 172.21.15.89:0/1862699618 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf4008de0 con 0x7f4cfc1597c0 2023-10-10T20:41:45.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4d01637700 1 -- 172.21.15.89:0/1862699618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cfc0c4c60 con 0x7f4cfc1597c0 2023-10-10T20:41:45.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.258+0000 7f4ce2ffd700 1 -- 172.21.15.89:0/1862699618 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf40266b0 con 0x7f4cfc1597c0 2023-10-10T20:41:45.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.259+0000 7f4d01637700 1 -- 172.21.15.89:0/1862699618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cfc16dcf0 con 0x7f4cfc1597c0 2023-10-10T20:41:45.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.260+0000 7f4d01637700 1 -- 172.21.15.89:0/1862699618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cfc06d720 con 0x7f4cfc1597c0 2023-10-10T20:41:45.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.260+0000 7f4ce2ffd700 1 -- 172.21.15.89:0/1862699618 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4cf4003040 con 0x7f4cfc1597c0 2023-10-10T20:41:45.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.260+0000 7f4ce2ffd700 1 --2- 172.21.15.89:0/1862699618 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4ce8067320 0x7f4ce80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:45.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.260+0000 7f4ce2ffd700 1 -- 172.21.15.89:0/1862699618 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5111+0+0 (secure 0 0 0) 0x7f4cf4086420 con 0x7f4cfc1597c0 2023-10-10T20:41:45.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.260+0000 7f4cfaffd700 1 --2- 172.21.15.89:0/1862699618 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4ce8067320 0x7f4ce80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:45.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.261+0000 7f4cfaffd700 1 --2- 172.21.15.89:0/1862699618 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4ce8067320 0x7f4ce80697d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0c5ee0 tx=0x7f4cec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:45.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.264+0000 7f4ce2ffd700 1 -- 172.21.15.89:0/1862699618 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4cf40532f0 con 0x7f4cfc1597c0 2023-10-10T20:41:45.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:45.463+0000 7f4d01637700 1 -- 172.21.15.89:0/1862699618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4cfc15ddd0 con 0x7f4cfc1597c0 2023-10-10T20:41:46.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.283+0000 7f4ce2ffd700 1 -- 172.21.15.89:0/1862699618 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v134) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4cf4056910 con 0x7f4cfc1597c0 2023-10-10T20:41:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.286+0000 7f4d01637700 1 -- 172.21.15.89:0/1862699618 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4ce8067320 msgr2=0x7f4ce80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.286+0000 7f4d01637700 1 --2- 172.21.15.89:0/1862699618 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4ce8067320 0x7f4ce80697d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0c5ee0 tx=0x7f4cec008040 comp rx=0 tx=0).stop 2023-10-10T20:41:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.286+0000 7f4d01637700 1 -- 172.21.15.89:0/1862699618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cfc1597c0 msgr2=0x7f4cfc1585e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.286+0000 7f4d01637700 1 --2- 172.21.15.89:0/1862699618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cfc1597c0 0x7f4cfc1585e0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40072f0 tx=0x7f4cf4018000 comp rx=0 tx=0).stop 2023-10-10T20:41:46.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.286+0000 7f4d01637700 1 -- 172.21.15.89:0/1862699618 shutdown_connections 2023-10-10T20:41:46.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.286+0000 7f4d01637700 1 --2- 172.21.15.89:0/1862699618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cfc1597c0 0x7f4cfc1585e0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.286+0000 7f4d01637700 1 --2- 172.21.15.89:0/1862699618 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4ce8067320 0x7f4ce80697d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.286+0000 7f4d01637700 1 --2- 172.21.15.89:0/1862699618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cfc0aedc0 0x7f4cfc16d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.286+0000 7f4d01637700 1 --2- 172.21.15.89:0/1862699618 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4cfc0af210 0x7f4cfc0c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.286+0000 7f4d01637700 1 -- 172.21.15.89:0/1862699618 >> 172.21.15.89:0/1862699618 conn(0x7f4cfc0c9f30 msgr2=0x7f4cfc06ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:46.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.286+0000 7f4d01637700 1 -- 172.21.15.89:0/1862699618 shutdown_connections 2023-10-10T20:41:46.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.286+0000 7f4d01637700 1 -- 172.21.15.89:0/1862699618 wait complete. 2023-10-10T20:41:46.290 INFO:teuthology.orchestra.run.smithi089.stderr: removed all blocklist entries 2023-10-10T20:41:46.301 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:41:46.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.417+0000 7fa93e52d700 1 Processor -- start 2023-10-10T20:41:46.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.417+0000 7fa93e52d700 1 -- start start 2023-10-10T20:41:46.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.418+0000 7fa93e52d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa9380b1420 0x7fa9380ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:46.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.418+0000 7fa93e52d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa9380ccf90 0x7fa9380c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:46.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.418+0000 7fa93e52d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9380cc300 con 0x7fa9380cbf30 2023-10-10T20:41:46.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.418+0000 7fa93e52d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa938156bf0 con 0x7fa9380ccf90 2023-10-10T20:41:46.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.418+0000 7fa93e52d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9380ae950 con 0x7fa9380b1420 2023-10-10T20:41:46.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.418+0000 7fa93caca700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa9380ccf90 0x7fa9380c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:46.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.418+0000 7fa937fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa9380cbf30 0x7fa9380b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53994/0 (socket says 172.21.15.89:53994) 2023-10-10T20:41:46.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.418+0000 7fa937fff700 1 -- 172.21.15.89:0/1315290200 learned_addr learned my addr 172.21.15.89:0/1315290200 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:46.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.419+0000 7fa93caca700 1 -- 172.21.15.89:0/1315290200 >> v2:172.21.15.111:3300/0 conn(0x7fa9380b1420 msgr2=0x7fa9380ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:46.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.419+0000 7fa93caca700 1 --2- 172.21.15.89:0/1315290200 >> v2:172.21.15.111:3300/0 conn(0x7fa9380b1420 0x7fa9380ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.419+0000 7fa93caca700 1 -- 172.21.15.89:0/1315290200 >> v1:172.21.15.89:6789/0 conn(0x7fa9380cbf30 legacy=0x7fa9380b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:46.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.419+0000 7fa93caca700 1 -- 172.21.15.89:0/1315290200 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9380ad370 con 0x7fa9380ccf90 2023-10-10T20:41:46.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.419+0000 7fa936ffd700 1 -- 172.21.15.89:0/1315290200 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148643966 0 0) 0x7fa9380cc300 con 0x7fa9380cbf30 2023-10-10T20:41:46.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.419+0000 7fa93caca700 1 --2- 172.21.15.89:0/1315290200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9380ccf90 0x7fa9380c9740 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa9380c9280 tx=0x7fa930009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36dfd83568589b8d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:46.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.419+0000 7fa936ffd700 1 -- 172.21.15.89:0/1315290200 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93000e070 con 0x7fa9380ccf90 2023-10-10T20:41:46.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.419+0000 7fa936ffd700 1 -- 172.21.15.89:0/1315290200 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93000bbf0 con 0x7fa9380ccf90 2023-10-10T20:41:46.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.419+0000 7fa936ffd700 1 -- 172.21.15.89:0/1315290200 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa930012430 con 0x7fa9380ccf90 2023-10-10T20:41:46.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.419+0000 7fa93e52d700 1 -- 172.21.15.89:0/1315290200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9380ccf90 msgr2=0x7fa9380c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:46.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.419+0000 7fa93e52d700 1 --2- 172.21.15.89:0/1315290200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9380ccf90 0x7fa9380c9740 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa9380c9280 tx=0x7fa930009780 comp rx=0 tx=0).stop 2023-10-10T20:41:46.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.419+0000 7fa93e52d700 1 -- 172.21.15.89:0/1315290200 shutdown_connections 2023-10-10T20:41:46.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.419+0000 7fa93e52d700 1 --2- 172.21.15.89:0/1315290200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9380ccf90 0x7fa9380c9740 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.419+0000 7fa93e52d700 1 --2- 172.21.15.89:0/1315290200 >> v2:172.21.15.111:3300/0 conn(0x7fa9380b1420 0x7fa9380ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.419+0000 7fa93e52d700 1 -- 172.21.15.89:0/1315290200 >> 172.21.15.89:0/1315290200 conn(0x7fa93806a320 msgr2=0x7fa9380cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:46.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.419+0000 7fa93e52d700 1 -- 172.21.15.89:0/1315290200 shutdown_connections 2023-10-10T20:41:46.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.419+0000 7fa93e52d700 1 -- 172.21.15.89:0/1315290200 wait complete. 2023-10-10T20:41:46.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.420+0000 7fa93e52d700 1 Processor -- start 2023-10-10T20:41:46.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.421+0000 7fa93e52d700 1 -- start start 2023-10-10T20:41:46.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.421+0000 7fa93e52d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9380b1420 0x7fa938168b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:46.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.421+0000 7fa93e52d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9380cbf30 0x7fa9381690d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:46.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.421+0000 7fa93e52d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9380ccf90 0x7fa9381416d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:46.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.421+0000 7fa93e52d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9380a6f10 con 0x7fa9380cbf30 2023-10-10T20:41:46.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.421+0000 7fa93e52d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa938167230 con 0x7fa9380b1420 2023-10-10T20:41:46.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.421+0000 7fa93e52d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa938166900 con 0x7fa9380ccf90 2023-10-10T20:41:46.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.422+0000 7fa9377fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9380cbf30 0x7fa9381690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:46.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.422+0000 7fa9377fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9380cbf30 0x7fa9381690d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38546/0 (socket says 172.21.15.89:38546) 2023-10-10T20:41:46.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.422+0000 7fa9377fe700 1 -- 172.21.15.89:0/3854553779 learned_addr learned my addr 172.21.15.89:0/3854553779 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:46.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.422+0000 7fa9377fe700 1 -- 172.21.15.89:0/3854553779 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9380ccf90 msgr2=0x7fa9381416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:46.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.422+0000 7fa9377fe700 1 --2- 172.21.15.89:0/3854553779 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9380ccf90 0x7fa9381416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.422+0000 7fa9377fe700 1 -- 172.21.15.89:0/3854553779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9380b1420 msgr2=0x7fa938168b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:46.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.422+0000 7fa9377fe700 1 --2- 172.21.15.89:0/3854553779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9380b1420 0x7fa938168b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.422+0000 7fa9377fe700 1 -- 172.21.15.89:0/3854553779 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa930009a20 con 0x7fa9380cbf30 2023-10-10T20:41:46.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.422+0000 7fa9377fe700 1 --2- 172.21.15.89:0/3854553779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9380cbf30 0x7fa9381690d0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00b7e0 tx=0x7fa92c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:46.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.422+0000 7fa91ffff700 1 -- 172.21.15.89:0/3854553779 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c00f570 con 0x7fa9380cbf30 2023-10-10T20:41:46.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.422+0000 7fa91ffff700 1 -- 172.21.15.89:0/3854553779 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92c00f6d0 con 0x7fa9380cbf30 2023-10-10T20:41:46.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.422+0000 7fa91ffff700 1 -- 172.21.15.89:0/3854553779 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c00fa70 con 0x7fa9380cbf30 2023-10-10T20:41:46.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.422+0000 7fa93e52d700 1 -- 172.21.15.89:0/3854553779 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa938166be0 con 0x7fa9380cbf30 2023-10-10T20:41:46.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.422+0000 7fa93e52d700 1 -- 172.21.15.89:0/3854553779 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa938141c10 con 0x7fa9380cbf30 2023-10-10T20:41:46.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.423+0000 7fa93e52d700 1 -- 172.21.15.89:0/3854553779 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa93804c3c0 con 0x7fa9380cbf30 2023-10-10T20:41:46.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.424+0000 7fa91ffff700 1 -- 172.21.15.89:0/3854553779 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa92c01a460 con 0x7fa9380cbf30 2023-10-10T20:41:46.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.424+0000 7fa91ffff700 1 --2- 172.21.15.89:0/3854553779 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa924067320 0x7fa9240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:46.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.424+0000 7fa91ffff700 1 -- 172.21.15.89:0/3854553779 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa92c0846c0 con 0x7fa9380cbf30 2023-10-10T20:41:46.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.424+0000 7fa937fff700 1 --2- 172.21.15.89:0/3854553779 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa924067320 0x7fa9240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:46.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.424+0000 7fa937fff700 1 --2- 172.21.15.89:0/3854553779 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa924067320 0x7fa9240697d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa9280011e0 tx=0x7fa9280079d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:46.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.428+0000 7fa91ffff700 1 -- 172.21.15.89:0/3854553779 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa92c0516a0 con 0x7fa9380cbf30 2023-10-10T20:41:46.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.683+0000 7fa93e52d700 1 -- 172.21.15.89:0/3854553779 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa938167370 con 0x7fa9380cbf30 2023-10-10T20:41:46.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.683+0000 7fa91ffff700 1 -- 172.21.15.89:0/3854553779 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa92c054cc0 con 0x7fa9380cbf30 2023-10-10T20:41:46.686 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:46.686 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:41:46.686 INFO:teuthology.orchestra.run.smithi089.stdout: "auth_dump": [ 2023-10-10T20:41:46.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:46.687 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.a", 2023-10-10T20:41:46.687 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlZLSNLhAA8WZTrsht6tAOP+gxaTXKMg==", 2023-10-10T20:41:46.687 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:46.687 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:41:46.687 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:41:46.688 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:41:46.688 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:46.688 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:46.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:46.689 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.b", 2023-10-10T20:41:46.689 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgyDEORAAMMoSxPFZc0bCIwWy8UsXgA==", 2023-10-10T20:41:46.689 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:46.689 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:41:46.689 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:41:46.690 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:41:46.690 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:46.690 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:46.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:46.691 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.c", 2023-10-10T20:41:46.691 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlMFxSNRAASuQEJFghLIOkTQWa4ckTvg==", 2023-10-10T20:41:46.691 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:46.691 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:41:46.691 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:41:46.692 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:41:46.692 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:46.692 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:46.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:46.692 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.d", 2023-10-10T20:41:46.693 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlPkvrBBAA2VLJmv2t8szTEGtCVwalJw==", 2023-10-10T20:41:46.693 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:46.693 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:41:46.693 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:41:46.693 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:41:46.694 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:46.694 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:46.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:46.694 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.0", 2023-10-10T20:41:46.695 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQArtSVltOcKHRAAk8+zUus04oTVI9KvfhjlXg==", 2023-10-10T20:41:46.695 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:46.695 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:41:46.695 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:41:46.695 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:46.696 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:46.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:46.696 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.1", 2023-10-10T20:41:46.696 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAutSVly8P0GhAAe1bvR63+ptJg84lizMNdQw==", 2023-10-10T20:41:46.696 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:46.697 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:41:46.697 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:41:46.697 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:46.698 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:46.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:46.698 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.2", 2023-10-10T20:41:46.698 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAxtSVl0x9fGRAAEKi7+6EW5mzPbebmz0+sgQ==", 2023-10-10T20:41:46.699 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:46.699 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:41:46.699 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:41:46.699 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:46.699 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:46.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:46.700 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.3", 2023-10-10T20:41:46.700 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA0tSVlfJyaGBAAa+8w54U6v7+npbdufKJruA==", 2023-10-10T20:41:46.700 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:46.700 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:41:46.701 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:41:46.701 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:46.701 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:46.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:46.702 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.4", 2023-10-10T20:41:46.702 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA6tSVlEQ/QFxAA/PRB3c90BuAlF6xekGfX9A==", 2023-10-10T20:41:46.702 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:46.702 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:41:46.702 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:41:46.703 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:46.703 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:46.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:46.703 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.5", 2023-10-10T20:41:46.704 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA9tSVlMOKrFRAAuBnIsEz2VmeSZi9XWzjlzA==", 2023-10-10T20:41:46.704 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:46.704 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:41:46.704 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:41:46.704 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:46.705 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:46.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:46.705 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.6", 2023-10-10T20:41:46.705 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBAtSVlH7PQFBAAHr6c7d4TYJEu/KSFEf+EOA==", 2023-10-10T20:41:46.705 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:46.706 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:41:46.706 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:41:46.706 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:46.706 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:46.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:46.707 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.7", 2023-10-10T20:41:46.707 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBDtSVlAhC1FBAAuE2BwAA/f7VOkZfVdqAG7A==", 2023-10-10T20:41:46.707 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:46.707 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:41:46.708 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:41:46.708 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:46.708 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:46.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:46.709 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.0", 2023-10-10T20:41:46.709 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlRd3HCBAAb0QP2VfTS7aelno1mFpGAA==", 2023-10-10T20:41:46.709 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:46.709 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:41:46.709 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow r", 2023-10-10T20:41:46.710 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:41:46.710 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:46.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:46.710 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.1", 2023-10-10T20:41:46.711 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlgBouERAAqPdE0KvJU3M53B6Y6AH5GQ==", 2023-10-10T20:41:46.711 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:46.711 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:41:46.711 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:41:46.711 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:41:46.712 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:41:46.712 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:46.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:46.712 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.2", 2023-10-10T20:41:46.713 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlXA4tDhAAoPVYQjECpzD+C67mnsY5IA==", 2023-10-10T20:41:46.713 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:46.713 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:41:46.713 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:41:46.713 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:41:46.714 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:41:46.714 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:46.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:46.714 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.3", 2023-10-10T20:41:46.715 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlZ5KGFhAAuKmZwbOfmmw1nAj5ry6tHw==", 2023-10-10T20:41:46.715 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:46.715 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:41:46.715 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:41:46.715 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:41:46.716 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:41:46.716 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:46.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:46.716 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.admin", 2023-10-10T20:41:46.717 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgpL2EhAAoDOki002rfBrauTgfu481Q==", 2023-10-10T20:41:46.717 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:46.717 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:41:46.717 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:41:46.717 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:41:46.717 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:46.718 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:46.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:46.718 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.x", 2023-10-10T20:41:46.718 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlBhaJKxAArraBxT5xWg515LfpJ7MlbA==", 2023-10-10T20:41:46.719 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:46.719 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:41:46.719 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:41:46.719 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:46.719 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:41:46.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:46.720 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.y", 2023-10-10T20:41:46.720 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlTZIyJxAAWAOsitHwsIxJWwp3b1Z98Q==", 2023-10-10T20:41:46.720 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:41:46.720 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:41:46.721 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:41:46.721 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:41:46.721 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.721 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:46.721 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-10T20:41:46.722 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:41:46.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.685+0000 7fa93e52d700 1 -- 172.21.15.89:0/3854553779 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa924067320 msgr2=0x7fa9240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:46.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.685+0000 7fa93e52d700 1 --2- 172.21.15.89:0/3854553779 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa924067320 0x7fa9240697d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa9280011e0 tx=0x7fa9280079d0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.685+0000 7fa93e52d700 1 -- 172.21.15.89:0/3854553779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9380cbf30 msgr2=0x7fa9381690d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.685+0000 7fa93e52d700 1 --2- 172.21.15.89:0/3854553779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9380cbf30 0x7fa9381690d0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00b7e0 tx=0x7fa92c009220 comp rx=0 tx=0).stop 2023-10-10T20:41:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.685+0000 7fa93e52d700 1 -- 172.21.15.89:0/3854553779 shutdown_connections 2023-10-10T20:41:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.685+0000 7fa93e52d700 1 --2- 172.21.15.89:0/3854553779 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9380ccf90 0x7fa9381416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.685+0000 7fa93e52d700 1 --2- 172.21.15.89:0/3854553779 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa924067320 0x7fa9240697d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.685+0000 7fa93e52d700 1 --2- 172.21.15.89:0/3854553779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9380b1420 0x7fa938168b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.685+0000 7fa93e52d700 1 --2- 172.21.15.89:0/3854553779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9380cbf30 0x7fa9381690d0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.685+0000 7fa93e52d700 1 -- 172.21.15.89:0/3854553779 >> 172.21.15.89:0/3854553779 conn(0x7fa93806a320 msgr2=0x7fa93815a260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:46.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.685+0000 7fa93e52d700 1 -- 172.21.15.89:0/3854553779 shutdown_connections 2023-10-10T20:41:46.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.685+0000 7fa93e52d700 1 -- 172.21.15.89:0/3854553779 wait complete. 2023-10-10T20:41:46.726 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:41:46.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.839+0000 7f6bdd1e3700 1 Processor -- start 2023-10-10T20:41:46.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.839+0000 7f6bdd1e3700 1 -- start start 2023-10-10T20:41:46.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.840+0000 7f6bdd1e3700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6bd8159420 0x7f6bd80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:46.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.840+0000 7f6bdd1e3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bd81597f0 0x7f6bd80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:46.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.840+0000 7f6bdd1e3700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd80b11d0 con 0x7f6bd815d230 2023-10-10T20:41:46.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.840+0000 7f6bdd1e3700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd80b14b0 con 0x7f6bd8159420 2023-10-10T20:41:46.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.840+0000 7f6bdd1e3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd80b1790 con 0x7f6bd81597f0 2023-10-10T20:41:46.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.840+0000 7f6bd7fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6bd8159420 0x7f6bd80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:46.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.840+0000 7f6bd77fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bd81597f0 0x7f6bd80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:46.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.840+0000 7f6bdc9e2700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f6bd815d230 0x7f6bd80b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:43138/0 (socket says 172.21.15.89:43138) 2023-10-10T20:41:46.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.840+0000 7f6bdc9e2700 1 -- 172.21.15.89:0/1485630211 learned_addr learned my addr 172.21.15.89:0/1485630211 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:46.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.841+0000 7f6bd77fe700 1 -- 172.21.15.89:0/1485630211 >> v2:172.21.15.111:3300/0 conn(0x7f6bd8159420 msgr2=0x7f6bd80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:46.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.841+0000 7f6bd77fe700 1 --2- 172.21.15.89:0/1485630211 >> v2:172.21.15.111:3300/0 conn(0x7f6bd8159420 0x7f6bd80b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.841+0000 7f6bd77fe700 1 -- 172.21.15.89:0/1485630211 >> v1:172.21.15.111:6789/0 conn(0x7f6bd815d230 legacy=0x7f6bd80b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:46.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.841+0000 7f6bd77fe700 1 -- 172.21.15.89:0/1485630211 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd80b1a70 con 0x7f6bd81597f0 2023-10-10T20:41:46.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.841+0000 7f6bd6ffd700 1 -- 172.21.15.89:0/1485630211 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1768770377 0 0) 0x7f6bd80b11d0 con 0x7f6bd815d230 2023-10-10T20:41:46.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.841+0000 7f6bd7fff700 1 --2- 172.21.15.89:0/1485630211 >> v2:172.21.15.111:3300/0 conn(0x7f6bd8159420 0x7f6bd80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:41:46.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.841+0000 7f6bd77fe700 1 --2- 172.21.15.89:0/1485630211 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bd81597f0 0x7f6bd80b2270 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0098e0 tx=0x7f6bcc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cbecc34d1a77ba4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:46.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.841+0000 7f6bd6ffd700 1 -- 172.21.15.89:0/1485630211 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc00b640 con 0x7f6bd81597f0 2023-10-10T20:41:46.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.841+0000 7f6bd6ffd700 1 -- 172.21.15.89:0/1485630211 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc00b7a0 con 0x7f6bd81597f0 2023-10-10T20:41:46.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.841+0000 7f6bd6ffd700 1 -- 172.21.15.89:0/1485630211 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc00bb40 con 0x7f6bd81597f0 2023-10-10T20:41:46.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.842+0000 7f6bdd1e3700 1 -- 172.21.15.89:0/1485630211 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bd81597f0 msgr2=0x7f6bd80b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:46.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.842+0000 7f6bdd1e3700 1 --2- 172.21.15.89:0/1485630211 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bd81597f0 0x7f6bd80b2270 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0098e0 tx=0x7f6bcc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.842+0000 7f6bdd1e3700 1 -- 172.21.15.89:0/1485630211 shutdown_connections 2023-10-10T20:41:46.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.842+0000 7f6bdd1e3700 1 --2- 172.21.15.89:0/1485630211 >> v2:172.21.15.111:3300/0 conn(0x7f6bd8159420 0x7f6bd80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.842+0000 7f6bdd1e3700 1 --2- 172.21.15.89:0/1485630211 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bd81597f0 0x7f6bd80b2270 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.842+0000 7f6bdd1e3700 1 -- 172.21.15.89:0/1485630211 >> 172.21.15.89:0/1485630211 conn(0x7f6bd80d0660 msgr2=0x7f6bd80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:46.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.842+0000 7f6bdd1e3700 1 -- 172.21.15.89:0/1485630211 shutdown_connections 2023-10-10T20:41:46.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.842+0000 7f6bdd1e3700 1 -- 172.21.15.89:0/1485630211 wait complete. 2023-10-10T20:41:46.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.843+0000 7f6bdd1e3700 1 Processor -- start 2023-10-10T20:41:46.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.843+0000 7f6bdd1e3700 1 -- start start 2023-10-10T20:41:46.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.844+0000 7f6bdd1e3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bd8159420 0x7f6bd814e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:46.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.844+0000 7f6bdd1e3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bd81597f0 0x7f6bd816ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:46.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.844+0000 7f6bdd1e3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bd815d230 0x7f6bd8068d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:46.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.844+0000 7f6bdd1e3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd814e6a0 con 0x7f6bd8159420 2023-10-10T20:41:46.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.844+0000 7f6bdd1e3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd806b750 con 0x7f6bd81597f0 2023-10-10T20:41:46.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.844+0000 7f6bdd1e3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd806b9f0 con 0x7f6bd815d230 2023-10-10T20:41:46.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.844+0000 7f6bd7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bd8159420 0x7f6bd814e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:46.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.844+0000 7f6bd7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bd8159420 0x7f6bd814e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38562/0 (socket says 172.21.15.89:38562) 2023-10-10T20:41:46.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.844+0000 7f6bd7fff700 1 -- 172.21.15.89:0/4152528263 learned_addr learned my addr 172.21.15.89:0/4152528263 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:46.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.844+0000 7f6bd77fe700 1 --2- 172.21.15.89:0/4152528263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bd81597f0 0x7f6bd816ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:46.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.844+0000 7f6bdc9e2700 1 --2- 172.21.15.89:0/4152528263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bd815d230 0x7f6bd8068d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:46.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.844+0000 7f6bd7fff700 1 -- 172.21.15.89:0/4152528263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bd815d230 msgr2=0x7f6bd8068d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:46.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.844+0000 7f6bd7fff700 1 --2- 172.21.15.89:0/4152528263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bd815d230 0x7f6bd8068d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.844+0000 7f6bd7fff700 1 -- 172.21.15.89:0/4152528263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bd81597f0 msgr2=0x7f6bd816ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:46.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.844+0000 7f6bd7fff700 1 --2- 172.21.15.89:0/4152528263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bd81597f0 0x7f6bd816ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:46.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.844+0000 7f6bd7fff700 1 -- 172.21.15.89:0/4152528263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc8009420 con 0x7f6bd8159420 2023-10-10T20:41:46.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.844+0000 7f6bd7fff700 1 --2- 172.21.15.89:0/4152528263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bd8159420 0x7f6bd814e160 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800ec80 tx=0x7f6bc80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:46.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.844+0000 7f6bbffff700 1 -- 172.21.15.89:0/4152528263 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc800c920 con 0x7f6bd8159420 2023-10-10T20:41:46.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.845+0000 7f6bbffff700 1 -- 172.21.15.89:0/4152528263 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc800ca80 con 0x7f6bd8159420 2023-10-10T20:41:46.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.845+0000 7f6bdd1e3700 1 -- 172.21.15.89:0/4152528263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bcc009420 con 0x7f6bd8159420 2023-10-10T20:41:46.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.845+0000 7f6bbffff700 1 -- 172.21.15.89:0/4152528263 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc800ce00 con 0x7f6bd8159420 2023-10-10T20:41:46.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.845+0000 7f6bdd1e3700 1 -- 172.21.15.89:0/4152528263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd806bfb0 con 0x7f6bd8159420 2023-10-10T20:41:46.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.845+0000 7f6bdd1e3700 1 -- 172.21.15.89:0/4152528263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd804c3c0 con 0x7f6bd8159420 2023-10-10T20:41:46.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.846+0000 7f6bbffff700 1 -- 172.21.15.89:0/4152528263 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6bc8013680 con 0x7f6bd8159420 2023-10-10T20:41:46.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.846+0000 7f6bbffff700 1 --2- 172.21.15.89:0/4152528263 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6bc4067320 0x7f6bc40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:46.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.846+0000 7f6bd77fe700 1 --2- 172.21.15.89:0/4152528263 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6bc4067320 0x7f6bc40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:46.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.846+0000 7f6bbffff700 1 -- 172.21.15.89:0/4152528263 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6bc8084d00 con 0x7f6bd8159420 2023-10-10T20:41:46.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.847+0000 7f6bd77fe700 1 --2- 172.21.15.89:0/4152528263 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6bc4067320 0x7f6bc40697d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0091e0 tx=0x7f6bcc017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:46.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:46.851+0000 7f6bbffff700 1 -- 172.21.15.89:0/4152528263 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6bc8051ce0 con 0x7f6bd8159420 2023-10-10T20:41:47.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.036+0000 7f6bdd1e3700 1 -- 172.21.15.89:0/4152528263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6bd80b1d50 con 0x7f6bd8159420 2023-10-10T20:41:47.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.036+0000 7f6bbffff700 1 -- 172.21.15.89:0/4152528263 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+12144 (secure 0 0 0) 0x7f6bc8055300 con 0x7f6bd8159420 2023-10-10T20:41:47.039 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:47.039 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":134,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:41:46.280696+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:41:47.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.039+0000 7f6bdd1e3700 1 -- 172.21.15.89:0/4152528263 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6bc4067320 msgr2=0x7f6bc40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:47.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.039+0000 7f6bdd1e3700 1 --2- 172.21.15.89:0/4152528263 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6bc4067320 0x7f6bc40697d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0091e0 tx=0x7f6bcc017000 comp rx=0 tx=0).stop 2023-10-10T20:41:47.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.039+0000 7f6bdd1e3700 1 -- 172.21.15.89:0/4152528263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bd8159420 msgr2=0x7f6bd814e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:47.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.039+0000 7f6bdd1e3700 1 --2- 172.21.15.89:0/4152528263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bd8159420 0x7f6bd814e160 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800ec80 tx=0x7f6bc80091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.039+0000 7f6bdd1e3700 1 -- 172.21.15.89:0/4152528263 shutdown_connections 2023-10-10T20:41:47.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.039+0000 7f6bdd1e3700 1 --2- 172.21.15.89:0/4152528263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6bd815d230 0x7f6bd8068d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.039+0000 7f6bdd1e3700 1 --2- 172.21.15.89:0/4152528263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bd8159420 0x7f6bd814e160 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.039+0000 7f6bdd1e3700 1 --2- 172.21.15.89:0/4152528263 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6bc4067320 0x7f6bc40697d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.039+0000 7f6bdd1e3700 1 --2- 172.21.15.89:0/4152528263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bd81597f0 0x7f6bd816ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.039+0000 7f6bdd1e3700 1 -- 172.21.15.89:0/4152528263 >> 172.21.15.89:0/4152528263 conn(0x7f6bd80d0660 msgr2=0x7f6bd8069a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:47.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.039+0000 7f6bdd1e3700 1 -- 172.21.15.89:0/4152528263 shutdown_connections 2023-10-10T20:41:47.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.039+0000 7f6bdd1e3700 1 -- 172.21.15.89:0/4152528263 wait complete. 2023-10-10T20:41:47.053 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:41:47.054 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:41:47.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.169+0000 7f46a1938700 1 Processor -- start 2023-10-10T20:41:47.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.169+0000 7f46a1938700 1 -- start start 2023-10-10T20:41:47.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.170+0000 7f46a1938700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f469c156490 0x7f469c0c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.170+0000 7f46a1938700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f469c0c88b0 0x7f469c0ca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.170+0000 7f46a1938700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469c0c7a70 con 0x7f469c0ccee0 2023-10-10T20:41:47.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.170+0000 7f46a1938700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469c15a450 con 0x7f469c0c88b0 2023-10-10T20:41:47.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.170+0000 7f46a1938700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469c15a730 con 0x7f469c156490 2023-10-10T20:41:47.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.170+0000 7f46927fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f469c0ccee0 0x7f469c0c81a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54000/0 (socket says 172.21.15.89:54000) 2023-10-10T20:41:47.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.170+0000 7f46927fc700 1 -- 172.21.15.89:0/3857909449 learned_addr learned my addr 172.21.15.89:0/3857909449 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:47.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469b7fe700 1 --2- 172.21.15.89:0/3857909449 >> v2:172.21.15.111:3300/0 conn(0x7f469c0c88b0 0x7f469c0ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:47.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469affd700 1 --2- 172.21.15.89:0/3857909449 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f469c156490 0x7f469c0c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:47.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469a7fc700 1 -- 172.21.15.89:0/3857909449 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 579917608 0 0) 0x7f469c0c7a70 con 0x7f469c0ccee0 2023-10-10T20:41:47.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469a7fc700 1 -- 172.21.15.89:0/3857909449 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4688003570 con 0x7f469c0ccee0 2023-10-10T20:41:47.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469a7fc700 1 -- 172.21.15.89:0/3857909449 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 339135926 0 0) 0x7f4688003570 con 0x7f469c0ccee0 2023-10-10T20:41:47.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469a7fc700 1 -- 172.21.15.89:0/3857909449 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f469c156490 msgr2=0x7f469c0c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:47.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469a7fc700 1 --2- 172.21.15.89:0/3857909449 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f469c156490 0x7f469c0c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469a7fc700 1 -- 172.21.15.89:0/3857909449 >> v2:172.21.15.111:3300/0 conn(0x7f469c0c88b0 msgr2=0x7f469c0ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:47.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469a7fc700 1 --2- 172.21.15.89:0/3857909449 >> v2:172.21.15.111:3300/0 conn(0x7f469c0c88b0 0x7f469c0ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469a7fc700 1 -- 172.21.15.89:0/3857909449 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c15aa10 con 0x7f469c0ccee0 2023-10-10T20:41:47.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469a7fc700 1 -- 172.21.15.89:0/3857909449 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f4684001470 con 0x7f469c0ccee0 2023-10-10T20:41:47.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469a7fc700 1 -- 172.21.15.89:0/3857909449 >> v1:172.21.15.89:6789/0 conn(0x7f469c0ccee0 legacy=0x7f469c0c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:47.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469a7fc700 1 --2- 172.21.15.89:0/3857909449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4688003a60 0x7f4688003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469a7fc700 1 --2- 172.21.15.89:0/3857909449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4688004810 0x7f4688004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469a7fc700 1 --2- 172.21.15.89:0/3857909449 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f469c156490 0x7f46880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469a7fc700 1 -- 172.21.15.89:0/3857909449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c15a450 con 0x7f4688003a60 2023-10-10T20:41:47.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469a7fc700 1 -- 172.21.15.89:0/3857909449 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c15a730 con 0x7f4688004810 2023-10-10T20:41:47.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469a7fc700 1 -- 172.21.15.89:0/3857909449 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468800ba60 con 0x7f469c156490 2023-10-10T20:41:47.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469affd700 1 --2- 172.21.15.89:0/3857909449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4688003a60 0x7f4688003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:47.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f46927fc700 1 --2- 172.21.15.89:0/3857909449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4688004810 0x7f4688004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:47.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469b7fe700 1 --2- 172.21.15.89:0/3857909449 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f469c156490 0x7f46880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:47.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469affd700 1 -- 172.21.15.89:0/3857909449 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f469c156490 msgr2=0x7f46880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:47.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469affd700 1 --2- 172.21.15.89:0/3857909449 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f469c156490 0x7f46880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469affd700 1 -- 172.21.15.89:0/3857909449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4688004810 msgr2=0x7f4688004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:47.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469affd700 1 --2- 172.21.15.89:0/3857909449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4688004810 0x7f4688004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469affd700 1 -- 172.21.15.89:0/3857909449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4688003570 con 0x7f4688003a60 2023-10-10T20:41:47.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.171+0000 7f469affd700 1 --2- 172.21.15.89:0/3857909449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4688003a60 0x7f4688003e70 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f468c00e240 tx=0x7f468c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313d35f60e4f30ea server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:47.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.172+0000 7f469a7fc700 1 -- 172.21.15.89:0/3857909449 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c00eaf0 con 0x7f4688003a60 2023-10-10T20:41:47.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.172+0000 7f469a7fc700 1 -- 172.21.15.89:0/3857909449 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468c00f040 con 0x7f4688003a60 2023-10-10T20:41:47.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.172+0000 7f469a7fc700 1 -- 172.21.15.89:0/3857909449 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c00eea0 con 0x7f4688003a60 2023-10-10T20:41:47.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.172+0000 7f46a1938700 1 -- 172.21.15.89:0/3857909449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4688003a60 msgr2=0x7f4688003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:47.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.172+0000 7f46a1938700 1 --2- 172.21.15.89:0/3857909449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4688003a60 0x7f4688003e70 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f468c00e240 tx=0x7f468c00b040 comp rx=0 tx=0).stop 2023-10-10T20:41:47.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.172+0000 7f469b7fe700 1 -- 172.21.15.89:0/3857909449 reap_dead start 2023-10-10T20:41:47.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.172+0000 7f46a1938700 1 -- 172.21.15.89:0/3857909449 shutdown_connections 2023-10-10T20:41:47.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.172+0000 7f46a1938700 1 -- 172.21.15.89:0/3857909449 >> 172.21.15.89:0/3857909449 conn(0x7f469c068e10 msgr2=0x7f469c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:47.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.172+0000 7f46a1938700 1 -- 172.21.15.89:0/3857909449 shutdown_connections 2023-10-10T20:41:47.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.172+0000 7f46a1938700 1 -- 172.21.15.89:0/3857909449 wait complete. 2023-10-10T20:41:47.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.173+0000 7f46a1938700 1 Processor -- start 2023-10-10T20:41:47.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.173+0000 7f46a1938700 1 -- start start 2023-10-10T20:41:47.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.174+0000 7f46a1938700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4688003a60 0x7f469c153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.174+0000 7f46a1938700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f469c157270 0x7f469c159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.174+0000 7f46a1938700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f469c1576e0 0x7f469c159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.174+0000 7f46a1938700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c157b70 con 0x7f4688003a60 2023-10-10T20:41:47.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.174+0000 7f46a1938700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c157e50 con 0x7f469c157270 2023-10-10T20:41:47.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.174+0000 7f46a1938700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469c158130 con 0x7f469c1576e0 2023-10-10T20:41:47.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.174+0000 7f469affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4688003a60 0x7f469c153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:47.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.174+0000 7f469affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4688003a60 0x7f469c153110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38580/0 (socket says 172.21.15.89:38580) 2023-10-10T20:41:47.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.174+0000 7f469affd700 1 -- 172.21.15.89:0/2455957314 learned_addr learned my addr 172.21.15.89:0/2455957314 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:47.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.174+0000 7f469affd700 1 -- 172.21.15.89:0/2455957314 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f469c1576e0 msgr2=0x7f469c159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:47.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.174+0000 7f469affd700 1 --2- 172.21.15.89:0/2455957314 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f469c1576e0 0x7f469c159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.174+0000 7f469affd700 1 -- 172.21.15.89:0/2455957314 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f469c157270 msgr2=0x7f469c159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:47.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.174+0000 7f469affd700 1 --2- 172.21.15.89:0/2455957314 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f469c157270 0x7f469c159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.174+0000 7f469affd700 1 -- 172.21.15.89:0/2455957314 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c158410 con 0x7f4688003a60 2023-10-10T20:41:47.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.175+0000 7f469affd700 1 --2- 172.21.15.89:0/2455957314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4688003a60 0x7f469c153110 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f468c00e830 tx=0x7f468c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:47.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.175+0000 7f46937fe700 1 -- 172.21.15.89:0/2455957314 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c014e70 con 0x7f4688003a60 2023-10-10T20:41:47.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.175+0000 7f46937fe700 1 -- 172.21.15.89:0/2455957314 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468c01d630 con 0x7f4688003a60 2023-10-10T20:41:47.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.175+0000 7f46937fe700 1 -- 172.21.15.89:0/2455957314 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c01da10 con 0x7f4688003a60 2023-10-10T20:41:47.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.175+0000 7f46a1938700 1 -- 172.21.15.89:0/2455957314 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469c1586f0 con 0x7f4688003a60 2023-10-10T20:41:47.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.175+0000 7f46a1938700 1 -- 172.21.15.89:0/2455957314 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469c0ad650 con 0x7f4688003a60 2023-10-10T20:41:47.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.176+0000 7f46a1938700 1 -- 172.21.15.89:0/2455957314 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f469c04c3c0 con 0x7f4688003a60 2023-10-10T20:41:47.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.176+0000 7f46937fe700 1 -- 172.21.15.89:0/2455957314 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f468c003040 con 0x7f4688003a60 2023-10-10T20:41:47.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.177+0000 7f46937fe700 1 --2- 172.21.15.89:0/2455957314 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f467c067390 0x7f467c069840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.177+0000 7f46927fc700 1 --2- 172.21.15.89:0/2455957314 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f467c067390 0x7f467c069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:47.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.177+0000 7f46937fe700 1 -- 172.21.15.89:0/2455957314 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f468c086b80 con 0x7f4688003a60 2023-10-10T20:41:47.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.177+0000 7f46927fc700 1 --2- 172.21.15.89:0/2455957314 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f467c067390 0x7f467c069840 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f469c15a2f0 tx=0x7f469c0cd350 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:47.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.181+0000 7f46937fe700 1 -- 172.21.15.89:0/2455957314 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f468c053a10 con 0x7f4688003a60 2023-10-10T20:41:47.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.367+0000 7f46a1938700 1 -- 172.21.15.89:0/2455957314 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f469c0aee80 con 0x7f4688003a60 2023-10-10T20:41:47.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.367+0000 7f46937fe700 1 -- 172.21.15.89:0/2455957314 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) v1 ==== 93+0+10 (secure 0 0 0) 0x7f468c018020 con 0x7f4688003a60 2023-10-10T20:41:47.369 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:41:47.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.369+0000 7f46a1938700 1 -- 172.21.15.89:0/2455957314 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f467c067390 msgr2=0x7f467c069840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:47.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.369+0000 7f46a1938700 1 --2- 172.21.15.89:0/2455957314 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f467c067390 0x7f467c069840 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f469c15a2f0 tx=0x7f469c0cd350 comp rx=0 tx=0).stop 2023-10-10T20:41:47.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.369+0000 7f46a1938700 1 -- 172.21.15.89:0/2455957314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4688003a60 msgr2=0x7f469c153110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:47.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.369+0000 7f46a1938700 1 --2- 172.21.15.89:0/2455957314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4688003a60 0x7f469c153110 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f468c00e830 tx=0x7f468c01a000 comp rx=0 tx=0).stop 2023-10-10T20:41:47.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.369+0000 7f46a1938700 1 -- 172.21.15.89:0/2455957314 shutdown_connections 2023-10-10T20:41:47.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.369+0000 7f46a1938700 1 --2- 172.21.15.89:0/2455957314 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f469c1576e0 0x7f469c159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.369+0000 7f46a1938700 1 --2- 172.21.15.89:0/2455957314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4688003a60 0x7f469c153110 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.369+0000 7f46a1938700 1 --2- 172.21.15.89:0/2455957314 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f467c067390 0x7f467c069840 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.369+0000 7f46a1938700 1 --2- 172.21.15.89:0/2455957314 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f469c157270 0x7f469c159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.369+0000 7f46a1938700 1 -- 172.21.15.89:0/2455957314 >> 172.21.15.89:0/2455957314 conn(0x7f469c068e10 msgr2=0x7f469c1551c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:47.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.369+0000 7f46a1938700 1 -- 172.21.15.89:0/2455957314 shutdown_connections 2023-10-10T20:41:47.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.369+0000 7f46a1938700 1 -- 172.21.15.89:0/2455957314 wait complete. 2023-10-10T20:41:47.383 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:41:47.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.496+0000 7f8b76bb3700 1 Processor -- start 2023-10-10T20:41:47.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.496+0000 7f8b76bb3700 1 -- start start 2023-10-10T20:41:47.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.496+0000 7f8b76bb3700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8b70159c60 0x7f8b7015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.496+0000 7f8b76bb3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7015db10 0x7f8b70069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.496+0000 7f8b76bb3700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7015a160 con 0x7f8b70159890 2023-10-10T20:41:47.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.496+0000 7f8b76bb3700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b7015df50 con 0x7f8b70159c60 2023-10-10T20:41:47.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.496+0000 7f8b76bb3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700d13a0 con 0x7f8b7015db10 2023-10-10T20:41:47.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.497+0000 7f8b75150700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7015db10 0x7f8b70069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:47.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.497+0000 7f8b67fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8b70159c60 0x7f8b7015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:47.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.497+0000 7f8b7494f700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f8b70159890 0x7f8b7015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:43140/0 (socket says 172.21.15.89:43140) 2023-10-10T20:41:47.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.497+0000 7f8b7494f700 1 -- 172.21.15.89:0/3080244949 learned_addr learned my addr 172.21.15.89:0/3080244949 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:47.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.497+0000 7f8b75150700 1 -- 172.21.15.89:0/3080244949 >> v2:172.21.15.111:3300/0 conn(0x7f8b70159c60 msgr2=0x7f8b7015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:47.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.497+0000 7f8b75150700 1 --2- 172.21.15.89:0/3080244949 >> v2:172.21.15.111:3300/0 conn(0x7f8b70159c60 0x7f8b7015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.497+0000 7f8b75150700 1 -- 172.21.15.89:0/3080244949 >> v1:172.21.15.111:6789/0 conn(0x7f8b70159890 legacy=0x7f8b7015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:47.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.497+0000 7f8b75150700 1 -- 172.21.15.89:0/3080244949 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7015e140 con 0x7f8b7015db10 2023-10-10T20:41:47.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.497+0000 7f8b67fff700 1 --2- 172.21.15.89:0/3080244949 >> v2:172.21.15.111:3300/0 conn(0x7f8b70159c60 0x7f8b7015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:41:47.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.497+0000 7f8b75150700 1 --2- 172.21.15.89:0/3080244949 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7015db10 0x7f8b70069560 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009e60 tx=0x7f8b68009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f96b61de0c22b626 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:47.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.497+0000 7f8b677fe700 1 -- 172.21.15.89:0/3080244949 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6800e070 con 0x7f8b7015db10 2023-10-10T20:41:47.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.498+0000 7f8b677fe700 1 -- 172.21.15.89:0/3080244949 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6800bb60 con 0x7f8b7015db10 2023-10-10T20:41:47.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.498+0000 7f8b677fe700 1 -- 172.21.15.89:0/3080244949 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b68012430 con 0x7f8b7015db10 2023-10-10T20:41:47.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.498+0000 7f8b76bb3700 1 -- 172.21.15.89:0/3080244949 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7015db10 msgr2=0x7f8b70069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:47.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.498+0000 7f8b76bb3700 1 --2- 172.21.15.89:0/3080244949 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7015db10 0x7f8b70069560 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009e60 tx=0x7f8b68009780 comp rx=0 tx=0).stop 2023-10-10T20:41:47.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.498+0000 7f8b76bb3700 1 -- 172.21.15.89:0/3080244949 shutdown_connections 2023-10-10T20:41:47.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.498+0000 7f8b76bb3700 1 --2- 172.21.15.89:0/3080244949 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7015db10 0x7f8b70069560 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.498+0000 7f8b76bb3700 1 --2- 172.21.15.89:0/3080244949 >> v2:172.21.15.111:3300/0 conn(0x7f8b70159c60 0x7f8b7015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.498+0000 7f8b76bb3700 1 -- 172.21.15.89:0/3080244949 >> 172.21.15.89:0/3080244949 conn(0x7f8b70068e20 msgr2=0x7f8b700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:47.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.498+0000 7f8b76bb3700 1 -- 172.21.15.89:0/3080244949 shutdown_connections 2023-10-10T20:41:47.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.498+0000 7f8b76bb3700 1 -- 172.21.15.89:0/3080244949 wait complete. 2023-10-10T20:41:47.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.499+0000 7f8b76bb3700 1 Processor -- start 2023-10-10T20:41:47.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.499+0000 7f8b76bb3700 1 -- start start 2023-10-10T20:41:47.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.500+0000 7f8b76bb3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b70159890 0x7f8b70151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.500+0000 7f8b76bb3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b70159c60 0x7f8b700ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.500+0000 7f8b76bb3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7015db10 0x7f8b7015cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.500+0000 7f8b76bb3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b70152410 con 0x7f8b70159c60 2023-10-10T20:41:47.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.500+0000 7f8b76bb3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700c9200 con 0x7f8b70159890 2023-10-10T20:41:47.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.500+0000 7f8b76bb3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700ca730 con 0x7f8b7015db10 2023-10-10T20:41:47.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.500+0000 7f8b67fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b70159c60 0x7f8b700ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:47.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.500+0000 7f8b67fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b70159c60 0x7f8b700ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38596/0 (socket says 172.21.15.89:38596) 2023-10-10T20:41:47.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.500+0000 7f8b67fff700 1 -- 172.21.15.89:0/1181747758 learned_addr learned my addr 172.21.15.89:0/1181747758 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:47.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.500+0000 7f8b7494f700 1 --2- 172.21.15.89:0/1181747758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b70159890 0x7f8b70151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:47.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.500+0000 7f8b75150700 1 --2- 172.21.15.89:0/1181747758 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7015db10 0x7f8b7015cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:47.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.501+0000 7f8b67fff700 1 -- 172.21.15.89:0/1181747758 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7015db10 msgr2=0x7f8b7015cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:47.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.501+0000 7f8b67fff700 1 --2- 172.21.15.89:0/1181747758 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7015db10 0x7f8b7015cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.501+0000 7f8b67fff700 1 -- 172.21.15.89:0/1181747758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b70159890 msgr2=0x7f8b70151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:47.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.501+0000 7f8b67fff700 1 --2- 172.21.15.89:0/1181747758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b70159890 0x7f8b70151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.501+0000 7f8b67fff700 1 -- 172.21.15.89:0/1181747758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b60009420 con 0x7f8b70159c60 2023-10-10T20:41:47.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.501+0000 7f8b67fff700 1 --2- 172.21.15.89:0/1181747758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b70159c60 0x7f8b700ca1f0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000ecb0 tx=0x7f8b600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:47.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.501+0000 7f8b64ff9700 1 -- 172.21.15.89:0/1181747758 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6000c8c0 con 0x7f8b70159c60 2023-10-10T20:41:47.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.501+0000 7f8b76bb3700 1 -- 172.21.15.89:0/1181747758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b68009a20 con 0x7f8b70159c60 2023-10-10T20:41:47.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.501+0000 7f8b64ff9700 1 -- 172.21.15.89:0/1181747758 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6000ca20 con 0x7f8b70159c60 2023-10-10T20:41:47.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.501+0000 7f8b64ff9700 1 -- 172.21.15.89:0/1181747758 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6000cda0 con 0x7f8b70159c60 2023-10-10T20:41:47.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.501+0000 7f8b76bb3700 1 -- 172.21.15.89:0/1181747758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b700c99f0 con 0x7f8b70159c60 2023-10-10T20:41:47.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.502+0000 7f8b76bb3700 1 -- 172.21.15.89:0/1181747758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b700ca040 con 0x7f8b70159c60 2023-10-10T20:41:47.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.502+0000 7f8b64ff9700 1 -- 172.21.15.89:0/1181747758 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8b60013680 con 0x7f8b70159c60 2023-10-10T20:41:47.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.502+0000 7f8b64ff9700 1 --2- 172.21.15.89:0/1181747758 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b5c067320 0x7f8b5c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.503+0000 7f8b64ff9700 1 -- 172.21.15.89:0/1181747758 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8b60084cc0 con 0x7f8b70159c60 2023-10-10T20:41:47.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.503+0000 7f8b7494f700 1 --2- 172.21.15.89:0/1181747758 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b5c067320 0x7f8b5c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:47.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.503+0000 7f8b7494f700 1 --2- 172.21.15.89:0/1181747758 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b5c067320 0x7f8b5c0697d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c007130 tx=0x7f8b6c007eb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:47.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.507+0000 7f8b64ff9700 1 -- 172.21.15.89:0/1181747758 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b6008b080 con 0x7f8b70159c60 2023-10-10T20:41:47.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.690+0000 7f8b76bb3700 1 -- 172.21.15.89:0/1181747758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8b7004c3c0 con 0x7f8b70159c60 2023-10-10T20:41:47.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.690+0000 7f8b64ff9700 1 -- 172.21.15.89:0/1181747758 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v134) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8b60021090 con 0x7f8b70159c60 2023-10-10T20:41:47.693 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:47.693 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:41:47.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:41:47.694 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2023-10-10T20:41:47.694 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2023-10-10T20:41:47.694 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:41:47.694 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:41:47.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.692+0000 7f8b76bb3700 1 -- 172.21.15.89:0/1181747758 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b5c067320 msgr2=0x7f8b5c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:47.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.692+0000 7f8b76bb3700 1 --2- 172.21.15.89:0/1181747758 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b5c067320 0x7f8b5c0697d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c007130 tx=0x7f8b6c007eb0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.692+0000 7f8b76bb3700 1 -- 172.21.15.89:0/1181747758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b70159c60 msgr2=0x7f8b700ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:47.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.692+0000 7f8b76bb3700 1 --2- 172.21.15.89:0/1181747758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b70159c60 0x7f8b700ca1f0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000ecb0 tx=0x7f8b600091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.692+0000 7f8b76bb3700 1 -- 172.21.15.89:0/1181747758 shutdown_connections 2023-10-10T20:41:47.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.692+0000 7f8b76bb3700 1 --2- 172.21.15.89:0/1181747758 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b7015db10 0x7f8b7015cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.692+0000 7f8b76bb3700 1 --2- 172.21.15.89:0/1181747758 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b5c067320 0x7f8b5c0697d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.692+0000 7f8b76bb3700 1 --2- 172.21.15.89:0/1181747758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b70159890 0x7f8b70151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.692+0000 7f8b76bb3700 1 --2- 172.21.15.89:0/1181747758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b70159c60 0x7f8b700ca1f0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.692+0000 7f8b76bb3700 1 -- 172.21.15.89:0/1181747758 >> 172.21.15.89:0/1181747758 conn(0x7f8b70068e20 msgr2=0x7f8b70152da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:47.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.692+0000 7f8b76bb3700 1 -- 172.21.15.89:0/1181747758 shutdown_connections 2023-10-10T20:41:47.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.692+0000 7f8b76bb3700 1 -- 172.21.15.89:0/1181747758 wait complete. 2023-10-10T20:41:47.706 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:41:47.707 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:41:47.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.824+0000 7f84214e7700 1 Processor -- start 2023-10-10T20:41:47.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.824+0000 7f84214e7700 1 -- start start 2023-10-10T20:41:47.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.824+0000 7f84214e7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f841c159ec0 0x7f841c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.824+0000 7f84214e7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f841c0cf350 0x7f841c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.824+0000 7f84214e7700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f841c0b5b00 con 0x7f841c0b53d0 2023-10-10T20:41:47.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.824+0000 7f84214e7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f841c0b5850 con 0x7f841c159ec0 2023-10-10T20:41:47.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.824+0000 7f84214e7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841c0b5da0 con 0x7f841c0cf350 2023-10-10T20:41:47.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.825+0000 7f841affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f841c159ec0 0x7f841c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:47.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.825+0000 7f841affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f841c159ec0 0x7f841c0b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38604/0 (socket says 172.21.15.89:38604) 2023-10-10T20:41:47.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.825+0000 7f841affd700 1 -- 172.21.15.89:0/2405361632 learned_addr learned my addr 172.21.15.89:0/2405361632 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:47.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.825+0000 7f841b7fe700 1 --2- 172.21.15.89:0/2405361632 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f841c0cf350 0x7f841c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:47.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.825+0000 7f841affd700 1 -- 172.21.15.89:0/2405361632 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f841c0cf350 msgr2=0x7f841c0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:47.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.825+0000 7f841affd700 1 --2- 172.21.15.89:0/2405361632 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f841c0cf350 0x7f841c0cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.825+0000 7f841affd700 1 -- 172.21.15.89:0/2405361632 >> v1:172.21.15.111:6789/0 conn(0x7f841c0b53d0 legacy=0x7f841c0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:47.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.825+0000 7f841affd700 1 -- 172.21.15.89:0/2405361632 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841c0b6080 con 0x7f841c159ec0 2023-10-10T20:41:47.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.825+0000 7f841affd700 1 --2- 172.21.15.89:0/2405361632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841c159ec0 0x7f841c0b4e90 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f840c009990 tx=0x7f840c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ee2a08945deef6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:47.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.825+0000 7f841a7fc700 1 -- 172.21.15.89:0/2405361632 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c005c10 con 0x7f841c159ec0 2023-10-10T20:41:47.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.825+0000 7f841a7fc700 1 -- 172.21.15.89:0/2405361632 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c005d70 con 0x7f841c159ec0 2023-10-10T20:41:47.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.825+0000 7f841a7fc700 1 -- 172.21.15.89:0/2405361632 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c00b880 con 0x7f841c159ec0 2023-10-10T20:41:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.826+0000 7f84214e7700 1 -- 172.21.15.89:0/2405361632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841c159ec0 msgr2=0x7f841c0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.826+0000 7f84214e7700 1 --2- 172.21.15.89:0/2405361632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841c159ec0 0x7f841c0b4e90 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f840c009990 tx=0x7f840c009220 comp rx=0 tx=0).stop 2023-10-10T20:41:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.826+0000 7f84214e7700 1 -- 172.21.15.89:0/2405361632 shutdown_connections 2023-10-10T20:41:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.826+0000 7f84214e7700 1 --2- 172.21.15.89:0/2405361632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841c159ec0 0x7f841c0b4e90 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.826+0000 7f84214e7700 1 --2- 172.21.15.89:0/2405361632 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f841c0cf350 0x7f841c0cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.826+0000 7f84214e7700 1 -- 172.21.15.89:0/2405361632 >> 172.21.15.89:0/2405361632 conn(0x7f841c0999f0 msgr2=0x7f841c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:47.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.826+0000 7f84214e7700 1 -- 172.21.15.89:0/2405361632 shutdown_connections 2023-10-10T20:41:47.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.826+0000 7f84214e7700 1 -- 172.21.15.89:0/2405361632 wait complete. 2023-10-10T20:41:47.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.827+0000 7f84214e7700 1 Processor -- start 2023-10-10T20:41:47.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.827+0000 7f84214e7700 1 -- start start 2023-10-10T20:41:47.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f84214e7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841c0b53d0 0x7f841c152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f84214e7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f841c0cf350 0x7f841c0c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f84214e7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f841c159ec0 0x7f841c0c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f84214e7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c154c20 con 0x7f841c0b53d0 2023-10-10T20:41:47.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f84214e7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c154f00 con 0x7f841c0cf350 2023-10-10T20:41:47.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f84214e7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841c1551e0 con 0x7f841c159ec0 2023-10-10T20:41:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f841affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841c0b53d0 0x7f841c152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f841affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841c0b53d0 0x7f841c152e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38610/0 (socket says 172.21.15.89:38610) 2023-10-10T20:41:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f841affd700 1 -- 172.21.15.89:0/2978297248 learned_addr learned my addr 172.21.15.89:0/2978297248 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f841b7fe700 1 --2- 172.21.15.89:0/2978297248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f841c159ec0 0x7f841c0c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f84127fc700 1 --2- 172.21.15.89:0/2978297248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f841c0cf350 0x7f841c0c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f841affd700 1 -- 172.21.15.89:0/2978297248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f841c159ec0 msgr2=0x7f841c0c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f841affd700 1 --2- 172.21.15.89:0/2978297248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f841c159ec0 0x7f841c0c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f841affd700 1 -- 172.21.15.89:0/2978297248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f841c0cf350 msgr2=0x7f841c0c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f841affd700 1 --2- 172.21.15.89:0/2978297248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f841c0cf350 0x7f841c0c98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f841affd700 1 -- 172.21.15.89:0/2978297248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c009480 con 0x7f841c0b53d0 2023-10-10T20:41:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f84127fc700 1 --2- 172.21.15.89:0/2978297248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f841c0cf350 0x7f841c0c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:47.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f841affd700 1 --2- 172.21.15.89:0/2978297248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841c0b53d0 0x7f841c152e10 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f840c005d40 tx=0x7f840c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:47.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f84137fe700 1 -- 172.21.15.89:0/2978297248 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c0119b0 con 0x7f841c0b53d0 2023-10-10T20:41:47.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f84137fe700 1 -- 172.21.15.89:0/2978297248 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c011b10 con 0x7f841c0b53d0 2023-10-10T20:41:47.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f84214e7700 1 -- 172.21.15.89:0/2978297248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841c1554c0 con 0x7f841c0b53d0 2023-10-10T20:41:47.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f84137fe700 1 -- 172.21.15.89:0/2978297248 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c011e90 con 0x7f841c0b53d0 2023-10-10T20:41:47.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.828+0000 7f84214e7700 1 -- 172.21.15.89:0/2978297248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841c153350 con 0x7f841c0b53d0 2023-10-10T20:41:47.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.829+0000 7f84214e7700 1 -- 172.21.15.89:0/2978297248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f841c152960 con 0x7f841c0b53d0 2023-10-10T20:41:47.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.830+0000 7f84137fe700 1 -- 172.21.15.89:0/2978297248 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f840c01b640 con 0x7f841c0b53d0 2023-10-10T20:41:47.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.830+0000 7f84137fe700 1 --2- 172.21.15.89:0/2978297248 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83fc067320 0x7f83fc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:47.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.830+0000 7f84137fe700 1 -- 172.21.15.89:0/2978297248 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f840c086340 con 0x7f841c0b53d0 2023-10-10T20:41:47.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.830+0000 7f84127fc700 1 --2- 172.21.15.89:0/2978297248 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83fc067320 0x7f83fc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:47.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.831+0000 7f84127fc700 1 --2- 172.21.15.89:0/2978297248 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83fc067320 0x7f83fc0697d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8404007fd0 tx=0x7f8404009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:47.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:47.835+0000 7f84137fe700 1 -- 172.21.15.89:0/2978297248 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f840c0532e0 con 0x7f841c0b53d0 2023-10-10T20:41:48.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.022+0000 7f84214e7700 1 -- 172.21.15.89:0/2978297248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f841c063c90 con 0x7f841c0b53d0 2023-10-10T20:41:48.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.292+0000 7f84137fe700 1 -- 172.21.15.89:0/2978297248 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v135) v1 ==== 138+0+0 (secure 0 0 0) 0x7f840c056900 con 0x7f841c0b53d0 2023-10-10T20:41:48.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.296+0000 7f84214e7700 1 -- 172.21.15.89:0/2978297248 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83fc067320 msgr2=0x7f83fc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:48.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.296+0000 7f84214e7700 1 --2- 172.21.15.89:0/2978297248 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83fc067320 0x7f83fc0697d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8404007fd0 tx=0x7f8404009040 comp rx=0 tx=0).stop 2023-10-10T20:41:48.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.296+0000 7f84214e7700 1 -- 172.21.15.89:0/2978297248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841c0b53d0 msgr2=0x7f841c152e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:48.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.296+0000 7f84214e7700 1 --2- 172.21.15.89:0/2978297248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841c0b53d0 0x7f841c152e10 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f840c005d40 tx=0x7f840c016000 comp rx=0 tx=0).stop 2023-10-10T20:41:48.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.296+0000 7f84214e7700 1 -- 172.21.15.89:0/2978297248 shutdown_connections 2023-10-10T20:41:48.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.296+0000 7f84214e7700 1 --2- 172.21.15.89:0/2978297248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f841c159ec0 0x7f841c0c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:48.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.296+0000 7f84214e7700 1 --2- 172.21.15.89:0/2978297248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841c0b53d0 0x7f841c152e10 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:48.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.296+0000 7f84214e7700 1 --2- 172.21.15.89:0/2978297248 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83fc067320 0x7f83fc0697d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:48.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.296+0000 7f84214e7700 1 --2- 172.21.15.89:0/2978297248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f841c0cf350 0x7f841c0c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:48.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.296+0000 7f84214e7700 1 -- 172.21.15.89:0/2978297248 >> 172.21.15.89:0/2978297248 conn(0x7f841c0999f0 msgr2=0x7f841c152250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:48.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.297+0000 7f84214e7700 1 -- 172.21.15.89:0/2978297248 shutdown_connections 2023-10-10T20:41:48.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.298+0000 7f84214e7700 1 -- 172.21.15.89:0/2978297248 wait complete. 2023-10-10T20:41:48.301 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2023-10-10T20:41:48.312 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:41:48.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.433+0000 7f949c69f700 1 Processor -- start 2023-10-10T20:41:48.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.433+0000 7f949c69f700 1 -- start start 2023-10-10T20:41:48.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.433+0000 7f949c69f700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9494159950 0x7f94940cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:48.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.433+0000 7f949c69f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f94940af320 0x7f9494069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:48.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.434+0000 7f949c69f700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94940ce380 con 0x7f9494069aa0 2023-10-10T20:41:48.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.434+0000 7f949c69f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94940cec20 con 0x7f94940af320 2023-10-10T20:41:48.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.434+0000 7f949c69f700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9494069ee0 con 0x7f9494159950 2023-10-10T20:41:48.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.434+0000 7f9499c3a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f94940af320 0x7f9494069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:48.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.434+0000 7f9499c3a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f94940af320 0x7f9494069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56424/0 (socket says 172.21.15.89:56424) 2023-10-10T20:41:48.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.434+0000 7f9499c3a700 1 -- 172.21.15.89:0/1140106376 learned_addr learned my addr 172.21.15.89:0/1140106376 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:48.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.434+0000 7f949a43b700 1 --2- 172.21.15.89:0/1140106376 >> v2:172.21.15.111:3300/0 conn(0x7f9494159950 0x7f94940cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:48.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.434+0000 7f9499c3a700 1 -- 172.21.15.89:0/1140106376 >> v2:172.21.15.111:3300/0 conn(0x7f9494159950 msgr2=0x7f94940cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:48.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.434+0000 7f9499c3a700 1 --2- 172.21.15.89:0/1140106376 >> v2:172.21.15.111:3300/0 conn(0x7f9494159950 0x7f94940cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:48.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.434+0000 7f9499c3a700 1 -- 172.21.15.89:0/1140106376 >> v1:172.21.15.111:6789/0 conn(0x7f9494069aa0 legacy=0x7f949415d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:48.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.434+0000 7f9499c3a700 1 -- 172.21.15.89:0/1140106376 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94940afa90 con 0x7f94940af320 2023-10-10T20:41:48.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.434+0000 7f9499c3a700 1 --2- 172.21.15.89:0/1140106376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94940af320 0x7f9494069560 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f94940896e0 tx=0x7f9484009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34d5348c5fbe7bf server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:48.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.434+0000 7f9499439700 1 -- 172.21.15.89:0/1140106376 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948400b820 con 0x7f94940af320 2023-10-10T20:41:48.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.434+0000 7f9499439700 1 -- 172.21.15.89:0/1140106376 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948400b980 con 0x7f94940af320 2023-10-10T20:41:48.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.434+0000 7f9499439700 1 -- 172.21.15.89:0/1140106376 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948400bd20 con 0x7f94940af320 2023-10-10T20:41:48.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.435+0000 7f949c69f700 1 -- 172.21.15.89:0/1140106376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94940af320 msgr2=0x7f9494069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:48.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.435+0000 7f949c69f700 1 --2- 172.21.15.89:0/1140106376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94940af320 0x7f9494069560 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f94940896e0 tx=0x7f9484009250 comp rx=0 tx=0).stop 2023-10-10T20:41:48.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.435+0000 7f949c69f700 1 -- 172.21.15.89:0/1140106376 shutdown_connections 2023-10-10T20:41:48.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.435+0000 7f949c69f700 1 --2- 172.21.15.89:0/1140106376 >> v2:172.21.15.111:3300/0 conn(0x7f9494159950 0x7f94940cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:48.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.435+0000 7f949c69f700 1 --2- 172.21.15.89:0/1140106376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94940af320 0x7f9494069560 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:48.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.435+0000 7f949c69f700 1 -- 172.21.15.89:0/1140106376 >> 172.21.15.89:0/1140106376 conn(0x7f9494068e00 msgr2=0x7f94940d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:48.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.435+0000 7f949c69f700 1 -- 172.21.15.89:0/1140106376 shutdown_connections 2023-10-10T20:41:48.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.435+0000 7f949c69f700 1 -- 172.21.15.89:0/1140106376 wait complete. 2023-10-10T20:41:48.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.436+0000 7f949c69f700 1 Processor -- start 2023-10-10T20:41:48.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.436+0000 7f949c69f700 1 -- start start 2023-10-10T20:41:48.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f949c69f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9494069aa0 0x7f9494157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:48.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f949c69f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94940af320 0x7f9494155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:48.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f949c69f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9494159950 0x7f94940b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:48.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f949c69f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9494158530 con 0x7f9494159950 2023-10-10T20:41:48.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f949c69f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94940b39d0 con 0x7f94940af320 2023-10-10T20:41:48.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f949c69f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9494158d20 con 0x7f9494069aa0 2023-10-10T20:41:48.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f949ac3c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9494159950 0x7f94940b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:48.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f949ac3c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9494159950 0x7f94940b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56438/0 (socket says 172.21.15.89:56438) 2023-10-10T20:41:48.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f949ac3c700 1 -- 172.21.15.89:0/3905169769 learned_addr learned my addr 172.21.15.89:0/3905169769 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:48.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f9499c3a700 1 --2- 172.21.15.89:0/3905169769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94940af320 0x7f9494155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:48.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f949a43b700 1 --2- 172.21.15.89:0/3905169769 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9494069aa0 0x7f9494157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:48.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f949ac3c700 1 -- 172.21.15.89:0/3905169769 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9494069aa0 msgr2=0x7f9494157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:48.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f949ac3c700 1 --2- 172.21.15.89:0/3905169769 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9494069aa0 0x7f9494157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:48.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f949ac3c700 1 -- 172.21.15.89:0/3905169769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94940af320 msgr2=0x7f9494155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:48.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f949ac3c700 1 --2- 172.21.15.89:0/3905169769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94940af320 0x7f9494155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:48.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f949ac3c700 1 -- 172.21.15.89:0/3905169769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94840094f0 con 0x7f9494159950 2023-10-10T20:41:48.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f949ac3c700 1 --2- 172.21.15.89:0/3905169769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9494159950 0x7f94940b3e60 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f948c00c960 tx=0x7f948c0026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:48.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f948a7fc700 1 -- 172.21.15.89:0/3905169769 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c012630 con 0x7f9494159950 2023-10-10T20:41:48.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f948a7fc700 1 -- 172.21.15.89:0/3905169769 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c012790 con 0x7f9494159950 2023-10-10T20:41:48.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f948a7fc700 1 -- 172.21.15.89:0/3905169769 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c012b50 con 0x7f9494159950 2023-10-10T20:41:48.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.437+0000 7f949c69f700 1 -- 172.21.15.89:0/3905169769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9494155a60 con 0x7f9494159950 2023-10-10T20:41:48.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.438+0000 7f949c69f700 1 -- 172.21.15.89:0/3905169769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9494158a40 con 0x7f9494159950 2023-10-10T20:41:48.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.439+0000 7f949c69f700 1 -- 172.21.15.89:0/3905169769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949406a1f0 con 0x7f9494159950 2023-10-10T20:41:48.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.439+0000 7f948a7fc700 1 -- 172.21.15.89:0/3905169769 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f948c01b400 con 0x7f9494159950 2023-10-10T20:41:48.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.440+0000 7f948a7fc700 1 --2- 172.21.15.89:0/3905169769 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9478067320 0x7f94780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:48.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.440+0000 7f949a43b700 1 --2- 172.21.15.89:0/3905169769 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9478067320 0x7f94780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:48.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.440+0000 7f948a7fc700 1 -- 172.21.15.89:0/3905169769 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f948c084d00 con 0x7f9494159950 2023-10-10T20:41:48.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.440+0000 7f949a43b700 1 --2- 172.21.15.89:0/3905169769 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9478067320 0x7f94780697d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9490006fd0 tx=0x7f9490008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:48.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.444+0000 7f948a7fc700 1 -- 172.21.15.89:0/3905169769 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f948c055600 con 0x7f9494159950 2023-10-10T20:41:48.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:48.627+0000 7f949c69f700 1 -- 172.21.15.89:0/3905169769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f94940718b0 con 0x7f9494159950 2023-10-10T20:41:49.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.299+0000 7f948a7fc700 1 -- 172.21.15.89:0/3905169769 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v136) v1 ==== 170+0+0 (secure 0 0 0) 0x7f948c055190 con 0x7f9494159950 2023-10-10T20:41:49.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.302+0000 7f949c69f700 1 -- 172.21.15.89:0/3905169769 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9478067320 msgr2=0x7f94780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:49.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.302+0000 7f949c69f700 1 --2- 172.21.15.89:0/3905169769 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9478067320 0x7f94780697d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9490006fd0 tx=0x7f9490008040 comp rx=0 tx=0).stop 2023-10-10T20:41:49.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.302+0000 7f949c69f700 1 -- 172.21.15.89:0/3905169769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9494159950 msgr2=0x7f94940b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:49.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.302+0000 7f949c69f700 1 --2- 172.21.15.89:0/3905169769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9494159950 0x7f94940b3e60 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f948c00c960 tx=0x7f948c0026e0 comp rx=0 tx=0).stop 2023-10-10T20:41:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.305+0000 7f949c69f700 1 -- 172.21.15.89:0/3905169769 shutdown_connections 2023-10-10T20:41:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.305+0000 7f949c69f700 1 --2- 172.21.15.89:0/3905169769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9494159950 0x7f94940b3e60 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.305+0000 7f949c69f700 1 --2- 172.21.15.89:0/3905169769 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9494069aa0 0x7f9494157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.305+0000 7f949c69f700 1 --2- 172.21.15.89:0/3905169769 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9478067320 0x7f94780697d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.305+0000 7f949c69f700 1 --2- 172.21.15.89:0/3905169769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94940af320 0x7f9494155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:49.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.305+0000 7f949c69f700 1 -- 172.21.15.89:0/3905169769 >> 172.21.15.89:0/3905169769 conn(0x7f9494068e00 msgr2=0x7f94940b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:49.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.306+0000 7f949c69f700 1 -- 172.21.15.89:0/3905169769 shutdown_connections 2023-10-10T20:41:49.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.307+0000 7f949c69f700 1 -- 172.21.15.89:0/3905169769 wait complete. 2023-10-10T20:41:49.312 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2023-10-10T20:41:49.329 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:41:49.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.447+0000 7f946af97700 1 Processor -- start 2023-10-10T20:41:49.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.447+0000 7f946af97700 1 -- start start 2023-10-10T20:41:49.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.448+0000 7f946af97700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94640b4e70 0x7f94640ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:49.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.448+0000 7f946af97700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f94640b1b40 0x7f946415d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:49.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.448+0000 7f946af97700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94640cfb00 con 0x7f94640b16d0 2023-10-10T20:41:49.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.448+0000 7f946af97700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94640b5c40 con 0x7f94640b1b40 2023-10-10T20:41:49.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.448+0000 7f946af97700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94640b5f20 con 0x7f94640b4e70 2023-10-10T20:41:49.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.448+0000 7f9469794700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f94640b16d0 0x7f946415d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55786/0 (socket says 172.21.15.89:55786) 2023-10-10T20:41:49.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.448+0000 7f9469794700 1 -- 172.21.15.89:0/1237819028 learned_addr learned my addr 172.21.15.89:0/1237819028 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:49.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.448+0000 7f9468f93700 1 -- 172.21.15.89:0/1237819028 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3316494625 0 0) 0x7f94640cfb00 con 0x7f94640b16d0 2023-10-10T20:41:49.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.448+0000 7f9468f93700 1 -- 172.21.15.89:0/1237819028 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f944c003570 con 0x7f94640b16d0 2023-10-10T20:41:49.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.448+0000 7f9469f95700 1 --2- 172.21.15.89:0/1237819028 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94640b4e70 0x7f94640ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:49.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.448+0000 7f946a796700 1 --2- 172.21.15.89:0/1237819028 >> v2:172.21.15.111:3300/0 conn(0x7f94640b1b40 0x7f946415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:49.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.448+0000 7f9468f93700 1 -- 172.21.15.89:0/1237819028 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3779070325 0 0) 0x7f944c003570 con 0x7f94640b16d0 2023-10-10T20:41:49.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.448+0000 7f9468f93700 1 -- 172.21.15.89:0/1237819028 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94640b4e70 msgr2=0x7f94640ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:49.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.448+0000 7f9468f93700 1 --2- 172.21.15.89:0/1237819028 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94640b4e70 0x7f94640ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:49.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.448+0000 7f9468f93700 1 -- 172.21.15.89:0/1237819028 >> v2:172.21.15.111:3300/0 conn(0x7f94640b1b40 msgr2=0x7f946415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:49.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.448+0000 7f9468f93700 1 --2- 172.21.15.89:0/1237819028 >> v2:172.21.15.111:3300/0 conn(0x7f94640b1b40 0x7f946415d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:49.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.448+0000 7f9468f93700 1 -- 172.21.15.89:0/1237819028 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94640b6200 con 0x7f94640b16d0 2023-10-10T20:41:49.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.448+0000 7f9468f93700 1 -- 172.21.15.89:0/1237819028 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f9458001470 con 0x7f94640b16d0 2023-10-10T20:41:49.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f9468f93700 1 -- 172.21.15.89:0/1237819028 >> v1:172.21.15.89:6789/0 conn(0x7f94640b16d0 legacy=0x7f946415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:49.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f9468f93700 1 --2- 172.21.15.89:0/1237819028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f944c003a60 0x7f944c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:49.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f9468f93700 1 --2- 172.21.15.89:0/1237819028 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94640b4e70 0x7f944c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:49.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f9468f93700 1 --2- 172.21.15.89:0/1237819028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f944c004d50 0x7f944c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:49.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f9468f93700 1 -- 172.21.15.89:0/1237819028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94640b5c40 con 0x7f944c003a60 2023-10-10T20:41:49.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f9468f93700 1 -- 172.21.15.89:0/1237819028 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94640b5f20 con 0x7f944c004d50 2023-10-10T20:41:49.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f9468f93700 1 -- 172.21.15.89:0/1237819028 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944c00ba60 con 0x7f94640b4e70 2023-10-10T20:41:49.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f9469f95700 1 --2- 172.21.15.89:0/1237819028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f944c003a60 0x7f944c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:49.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f9469794700 1 --2- 172.21.15.89:0/1237819028 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94640b4e70 0x7f944c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:49.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f946a796700 1 --2- 172.21.15.89:0/1237819028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f944c004d50 0x7f944c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:49.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f9469f95700 1 -- 172.21.15.89:0/1237819028 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94640b4e70 msgr2=0x7f944c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:49.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f9469f95700 1 --2- 172.21.15.89:0/1237819028 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94640b4e70 0x7f944c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f9469f95700 1 -- 172.21.15.89:0/1237819028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f944c004d50 msgr2=0x7f944c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f9469f95700 1 --2- 172.21.15.89:0/1237819028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f944c004d50 0x7f944c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f9469f95700 1 -- 172.21.15.89:0/1237819028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944c003570 con 0x7f944c003a60 2023-10-10T20:41:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f9469794700 1 --2- 172.21.15.89:0/1237819028 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94640b4e70 0x7f944c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f946a796700 1 --2- 172.21.15.89:0/1237819028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f944c004d50 0x7f944c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f9469f95700 1 --2- 172.21.15.89:0/1237819028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f944c003a60 0x7f944c003e70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f945c009c00 tx=0x7f945c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330795e7a1e7830f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f9468f93700 1 -- 172.21.15.89:0/1237819028 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c00e6b0 con 0x7f944c003a60 2023-10-10T20:41:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f9468f93700 1 -- 172.21.15.89:0/1237819028 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c00e810 con 0x7f944c003a60 2023-10-10T20:41:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.449+0000 7f9468f93700 1 -- 172.21.15.89:0/1237819028 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c00ebd0 con 0x7f944c003a60 2023-10-10T20:41:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.450+0000 7f946af97700 1 -- 172.21.15.89:0/1237819028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f944c003a60 msgr2=0x7f944c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.450+0000 7f946af97700 1 --2- 172.21.15.89:0/1237819028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f944c003a60 0x7f944c003e70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f945c009c00 tx=0x7f945c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.450+0000 7f946a796700 1 -- 172.21.15.89:0/1237819028 reap_dead start 2023-10-10T20:41:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.450+0000 7f946af97700 1 -- 172.21.15.89:0/1237819028 shutdown_connections 2023-10-10T20:41:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.450+0000 7f946af97700 1 -- 172.21.15.89:0/1237819028 >> 172.21.15.89:0/1237819028 conn(0x7f94640d10c0 msgr2=0x7f94640affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.450+0000 7f946af97700 1 -- 172.21.15.89:0/1237819028 shutdown_connections 2023-10-10T20:41:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.450+0000 7f946af97700 1 -- 172.21.15.89:0/1237819028 wait complete. 2023-10-10T20:41:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.451+0000 7f946af97700 1 Processor -- start 2023-10-10T20:41:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.451+0000 7f946af97700 1 -- start start 2023-10-10T20:41:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f946af97700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f944c003a60 0x7f94640cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f946af97700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94640ca770 0x7f946416a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f946af97700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f946416a780 0x7f94640c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f946af97700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94640ccbb0 con 0x7f94640ca770 2023-10-10T20:41:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f946af97700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94640cd380 con 0x7f944c003a60 2023-10-10T20:41:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f946af97700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94640cbff0 con 0x7f946416a780 2023-10-10T20:41:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f9469794700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94640ca770 0x7f946416a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f9469794700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94640ca770 0x7f946416a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56446/0 (socket says 172.21.15.89:56446) 2023-10-10T20:41:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f9469794700 1 -- 172.21.15.89:0/4208793531 learned_addr learned my addr 172.21.15.89:0/4208793531 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f9469f95700 1 --2- 172.21.15.89:0/4208793531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f944c003a60 0x7f94640cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f946a796700 1 --2- 172.21.15.89:0/4208793531 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f946416a780 0x7f94640c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f9469794700 1 -- 172.21.15.89:0/4208793531 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f946416a780 msgr2=0x7f94640c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f9469794700 1 --2- 172.21.15.89:0/4208793531 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f946416a780 0x7f94640c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f9469794700 1 -- 172.21.15.89:0/4208793531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f944c003a60 msgr2=0x7f94640cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f9469794700 1 --2- 172.21.15.89:0/4208793531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f944c003a60 0x7f94640cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f9469794700 1 -- 172.21.15.89:0/4208793531 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94640ccdc0 con 0x7f94640ca770 2023-10-10T20:41:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f9469794700 1 --2- 172.21.15.89:0/4208793531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94640ca770 0x7f946416a240 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f9458001ed0 tx=0x7f94580014c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f9461ffb700 1 -- 172.21.15.89:0/4208793531 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945800ee70 con 0x7f94640ca770 2023-10-10T20:41:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f9461ffb700 1 -- 172.21.15.89:0/4208793531 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945800c760 con 0x7f94640ca770 2023-10-10T20:41:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f946af97700 1 -- 172.21.15.89:0/4208793531 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94640cd0a0 con 0x7f94640ca770 2023-10-10T20:41:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.452+0000 7f9461ffb700 1 -- 172.21.15.89:0/4208793531 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945800cae0 con 0x7f94640ca770 2023-10-10T20:41:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.453+0000 7f946af97700 1 -- 172.21.15.89:0/4208793531 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946416abf0 con 0x7f94640ca770 2023-10-10T20:41:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.453+0000 7f946af97700 1 -- 172.21.15.89:0/4208793531 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94640c8400 con 0x7f94640ca770 2023-10-10T20:41:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.454+0000 7f9461ffb700 1 -- 172.21.15.89:0/4208793531 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f945801d460 con 0x7f94640ca770 2023-10-10T20:41:49.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.454+0000 7f9461ffb700 1 --2- 172.21.15.89:0/4208793531 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9450067320 0x7f94500697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:49.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.454+0000 7f9461ffb700 1 -- 172.21.15.89:0/4208793531 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f9458084b10 con 0x7f94640ca770 2023-10-10T20:41:49.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.454+0000 7f9469f95700 1 --2- 172.21.15.89:0/4208793531 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9450067320 0x7f94500697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:49.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.455+0000 7f9469f95700 1 --2- 172.21.15.89:0/4208793531 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9450067320 0x7f94500697d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f945c0091e0 tx=0x7f945c01a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:49.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.459+0000 7f9461ffb700 1 -- 172.21.15.89:0/4208793531 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94640c8400 con 0x7f94640ca770 2023-10-10T20:41:49.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:49.686+0000 7f946af97700 1 -- 172.21.15.89:0/4208793531 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f94640c8400 con 0x7f94640ca770 2023-10-10T20:41:50.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.300+0000 7f9461ffb700 1 -- 172.21.15.89:0/4208793531 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v87) v1 ==== 174+0+0 (secure 0 0 0) 0x7f94640c8400 con 0x7f94640ca770 2023-10-10T20:41:50.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.305+0000 7f946af97700 1 -- 172.21.15.89:0/4208793531 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9450067320 msgr2=0x7f94500697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:50.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.305+0000 7f946af97700 1 --2- 172.21.15.89:0/4208793531 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9450067320 0x7f94500697d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f945c0091e0 tx=0x7f945c01a000 comp rx=0 tx=0).stop 2023-10-10T20:41:50.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.305+0000 7f946af97700 1 -- 172.21.15.89:0/4208793531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94640ca770 msgr2=0x7f946416a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:50.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.305+0000 7f946af97700 1 --2- 172.21.15.89:0/4208793531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94640ca770 0x7f946416a240 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f9458001ed0 tx=0x7f94580014c0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.305+0000 7f946af97700 1 -- 172.21.15.89:0/4208793531 shutdown_connections 2023-10-10T20:41:50.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.305+0000 7f946af97700 1 --2- 172.21.15.89:0/4208793531 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f946416a780 0x7f94640c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.305+0000 7f946af97700 1 --2- 172.21.15.89:0/4208793531 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9450067320 0x7f94500697d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.305+0000 7f946af97700 1 --2- 172.21.15.89:0/4208793531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f944c003a60 0x7f94640cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.305+0000 7f946af97700 1 --2- 172.21.15.89:0/4208793531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94640ca770 0x7f946416a240 secure :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f9458001ed0 tx=0x7f94580014c0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.305+0000 7f946af97700 1 -- 172.21.15.89:0/4208793531 >> 172.21.15.89:0/4208793531 conn(0x7f94640d10c0 msgr2=0x7f94640b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:50.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.305+0000 7f946af97700 1 -- 172.21.15.89:0/4208793531 shutdown_connections 2023-10-10T20:41:50.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.305+0000 7f946af97700 1 -- 172.21.15.89:0/4208793531 wait complete. 2023-10-10T20:41:50.310 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 24 and data pool 25 2023-10-10T20:41:50.335 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:41:50.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.447+0000 7fcb6664d700 1 Processor -- start 2023-10-10T20:41:50.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.447+0000 7fcb6664d700 1 -- start start 2023-10-10T20:41:50.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.448+0000 7fcb6664d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb60159890 0x7fcb6015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:50.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.448+0000 7fcb6664d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb60159c60 0x7fcb6015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:50.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.448+0000 7fcb6664d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6015a160 con 0x7fcb6015db10 2023-10-10T20:41:50.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.448+0000 7fcb6664d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb600d13a0 con 0x7fcb60159c60 2023-10-10T20:41:50.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.448+0000 7fcb6664d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6015e140 con 0x7fcb60159890 2023-10-10T20:41:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.448+0000 7fcb5f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb60159c60 0x7fcb6015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.448+0000 7fcb5f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb60159c60 0x7fcb6015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56452/0 (socket says 172.21.15.89:56452) 2023-10-10T20:41:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.448+0000 7fcb5f7fe700 1 -- 172.21.15.89:0/3287777657 learned_addr learned my addr 172.21.15.89:0/3287777657 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.448+0000 7fcb5ffff700 1 --2- 172.21.15.89:0/3287777657 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb60159890 0x7fcb6015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.449+0000 7fcb5f7fe700 1 -- 172.21.15.89:0/3287777657 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb60159890 msgr2=0x7fcb6015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.449+0000 7fcb5f7fe700 1 --2- 172.21.15.89:0/3287777657 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb60159890 0x7fcb6015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.449+0000 7fcb5f7fe700 1 -- 172.21.15.89:0/3287777657 >> v1:172.21.15.111:6789/0 conn(0x7fcb6015db10 legacy=0x7fcb60069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.449+0000 7fcb5f7fe700 1 -- 172.21.15.89:0/3287777657 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6015a3d0 con 0x7fcb60159c60 2023-10-10T20:41:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.449+0000 7fcb5ffff700 1 --2- 172.21.15.89:0/3287777657 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb60159890 0x7fcb6015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.449+0000 7fcb5f7fe700 1 --2- 172.21.15.89:0/3287777657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb60159c60 0x7fcb6015d5d0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fcb540099b0 tx=0x7fcb54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22646b2ad3de605c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.449+0000 7fcb5effd700 1 -- 172.21.15.89:0/3287777657 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5400b860 con 0x7fcb60159c60 2023-10-10T20:41:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.449+0000 7fcb5effd700 1 -- 172.21.15.89:0/3287777657 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5400b9c0 con 0x7fcb60159c60 2023-10-10T20:41:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.449+0000 7fcb5effd700 1 -- 172.21.15.89:0/3287777657 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5400bd60 con 0x7fcb60159c60 2023-10-10T20:41:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.449+0000 7fcb6664d700 1 -- 172.21.15.89:0/3287777657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb60159c60 msgr2=0x7fcb6015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.449+0000 7fcb6664d700 1 --2- 172.21.15.89:0/3287777657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb60159c60 0x7fcb6015d5d0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fcb540099b0 tx=0x7fcb54009250 comp rx=0 tx=0).stop 2023-10-10T20:41:50.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.449+0000 7fcb6664d700 1 -- 172.21.15.89:0/3287777657 shutdown_connections 2023-10-10T20:41:50.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.449+0000 7fcb6664d700 1 --2- 172.21.15.89:0/3287777657 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb60159890 0x7fcb6015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.449+0000 7fcb6664d700 1 --2- 172.21.15.89:0/3287777657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb60159c60 0x7fcb6015d5d0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.449+0000 7fcb6664d700 1 -- 172.21.15.89:0/3287777657 >> 172.21.15.89:0/3287777657 conn(0x7fcb60068e20 msgr2=0x7fcb600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:50.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.450+0000 7fcb6664d700 1 -- 172.21.15.89:0/3287777657 shutdown_connections 2023-10-10T20:41:50.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.450+0000 7fcb6664d700 1 -- 172.21.15.89:0/3287777657 wait complete. 2023-10-10T20:41:50.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.451+0000 7fcb6664d700 1 Processor -- start 2023-10-10T20:41:50.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.451+0000 7fcb6664d700 1 -- start start 2023-10-10T20:41:50.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.451+0000 7fcb6664d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb60159890 0x7fcb600c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:50.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb6664d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb60159c60 0x7fcb600cf880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:50.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb6664d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb6015db10 0x7fcb600cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:50.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb6664d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb600c5a90 con 0x7fcb60159890 2023-10-10T20:41:50.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb6664d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb600c5d70 con 0x7fcb6015db10 2023-10-10T20:41:50.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb6664d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb600c6050 con 0x7fcb60159c60 2023-10-10T20:41:50.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb5ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb60159890 0x7fcb600c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:50.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb5ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb60159890 0x7fcb600c6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56466/0 (socket says 172.21.15.89:56466) 2023-10-10T20:41:50.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb5ffff700 1 -- 172.21.15.89:0/1098347479 learned_addr learned my addr 172.21.15.89:0/1098347479 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:50.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb64bea700 1 --2- 172.21.15.89:0/1098347479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb6015db10 0x7fcb600cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:50.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb5f7fe700 1 --2- 172.21.15.89:0/1098347479 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb60159c60 0x7fcb600cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:50.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb5ffff700 1 -- 172.21.15.89:0/1098347479 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb60159c60 msgr2=0x7fcb600cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:50.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb5ffff700 1 --2- 172.21.15.89:0/1098347479 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb60159c60 0x7fcb600cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb5ffff700 1 -- 172.21.15.89:0/1098347479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb6015db10 msgr2=0x7fcb600cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:50.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb5ffff700 1 --2- 172.21.15.89:0/1098347479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb6015db10 0x7fcb600cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb5ffff700 1 -- 172.21.15.89:0/1098347479 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb540094f0 con 0x7fcb60159890 2023-10-10T20:41:50.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb5f7fe700 1 --2- 172.21.15.89:0/1098347479 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb60159c60 0x7fcb600cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:50.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb5ffff700 1 --2- 172.21.15.89:0/1098347479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb60159890 0x7fcb600c6b30 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000e8d0 tx=0x7fcb50008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:50.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb47fff700 1 -- 172.21.15.89:0/1098347479 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50013410 con 0x7fcb60159890 2023-10-10T20:41:50.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb47fff700 1 -- 172.21.15.89:0/1098347479 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb50013570 con 0x7fcb60159890 2023-10-10T20:41:50.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb6664d700 1 -- 172.21.15.89:0/1098347479 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb600c6330 con 0x7fcb60159890 2023-10-10T20:41:50.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb47fff700 1 -- 172.21.15.89:0/1098347479 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50013910 con 0x7fcb60159890 2023-10-10T20:41:50.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.452+0000 7fcb6664d700 1 -- 172.21.15.89:0/1098347479 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb600cfdc0 con 0x7fcb60159890 2023-10-10T20:41:50.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.453+0000 7fcb6664d700 1 -- 172.21.15.89:0/1098347479 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb600c67d0 con 0x7fcb60159890 2023-10-10T20:41:50.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.454+0000 7fcb47fff700 1 -- 172.21.15.89:0/1098347479 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcb50013d00 con 0x7fcb60159890 2023-10-10T20:41:50.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.454+0000 7fcb47fff700 1 --2- 172.21.15.89:0/1098347479 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb4c067320 0x7fcb4c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:50.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.454+0000 7fcb47fff700 1 -- 172.21.15.89:0/1098347479 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcb50084ef0 con 0x7fcb60159890 2023-10-10T20:41:50.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.455+0000 7fcb5f7fe700 1 --2- 172.21.15.89:0/1098347479 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb4c067320 0x7fcb4c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:50.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.455+0000 7fcb5f7fe700 1 --2- 172.21.15.89:0/1098347479 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb4c067320 0x7fcb4c0697d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcb540099b0 tx=0x7fcb54017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:50.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.459+0000 7fcb47fff700 1 -- 172.21.15.89:0/1098347479 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb50051af0 con 0x7fcb60159890 2023-10-10T20:41:50.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.643+0000 7fcb6664d700 1 -- 172.21.15.89:0/1098347479 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb6004c3c0 con 0x7fcb60159890 2023-10-10T20:41:50.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.644+0000 7fcb47fff700 1 -- 172.21.15.89:0/1098347479 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+15377 (secure 0 0 0) 0x7fcb50020090 con 0x7fcb60159890 2023-10-10T20:41:50.646 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:50.646 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":137,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:41:49.687524+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:41:48.040814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-10T20:41:48.646159+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:41:50.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.646+0000 7fcb6664d700 1 -- 172.21.15.89:0/1098347479 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb4c067320 msgr2=0x7fcb4c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:50.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.646+0000 7fcb6664d700 1 --2- 172.21.15.89:0/1098347479 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb4c067320 0x7fcb4c0697d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcb540099b0 tx=0x7fcb54017000 comp rx=0 tx=0).stop 2023-10-10T20:41:50.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.646+0000 7fcb6664d700 1 -- 172.21.15.89:0/1098347479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb60159890 msgr2=0x7fcb600c6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:50.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.646+0000 7fcb6664d700 1 --2- 172.21.15.89:0/1098347479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb60159890 0x7fcb600c6b30 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000e8d0 tx=0x7fcb50008040 comp rx=0 tx=0).stop 2023-10-10T20:41:50.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.646+0000 7fcb6664d700 1 -- 172.21.15.89:0/1098347479 shutdown_connections 2023-10-10T20:41:50.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.646+0000 7fcb6664d700 1 --2- 172.21.15.89:0/1098347479 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb4c067320 0x7fcb4c0697d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.646+0000 7fcb6664d700 1 --2- 172.21.15.89:0/1098347479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb6015db10 0x7fcb600cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.646+0000 7fcb6664d700 1 --2- 172.21.15.89:0/1098347479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb60159890 0x7fcb600c6b30 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.646+0000 7fcb6664d700 1 --2- 172.21.15.89:0/1098347479 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb60159c60 0x7fcb600cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.646+0000 7fcb6664d700 1 -- 172.21.15.89:0/1098347479 >> 172.21.15.89:0/1098347479 conn(0x7fcb60068e20 msgr2=0x7fcb60154320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:50.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.646+0000 7fcb6664d700 1 -- 172.21.15.89:0/1098347479 shutdown_connections 2023-10-10T20:41:50.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.646+0000 7fcb6664d700 1 -- 172.21.15.89:0/1098347479 wait complete. 2023-10-10T20:41:50.660 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:41:50.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.774+0000 7fc5917ee700 1 Processor -- start 2023-10-10T20:41:50.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.774+0000 7fc5917ee700 1 -- start start 2023-10-10T20:41:50.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.775+0000 7fc5917ee700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc58c159890 0x7fc58c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:50.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.775+0000 7fc5917ee700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc58c069560 0x7fc58c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:50.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.775+0000 7fc5917ee700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc58c15a160 con 0x7fc58c159c60 2023-10-10T20:41:50.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.775+0000 7fc5917ee700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc58c069f20 con 0x7fc58c069560 2023-10-10T20:41:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.775+0000 7fc5917ee700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c0d13a0 con 0x7fc58c159890 2023-10-10T20:41:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.775+0000 7fc58b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc58c069560 0x7fc58c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.775+0000 7fc583fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc58c159c60 0x7fc58c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55800/0 (socket says 172.21.15.89:55800) 2023-10-10T20:41:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.775+0000 7fc583fff700 1 -- 172.21.15.89:0/909484119 learned_addr learned my addr 172.21.15.89:0/909484119 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.775+0000 7fc58affd700 1 --2- 172.21.15.89:0/909484119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc58c159890 0x7fc58c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:50.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.775+0000 7fc58b7fe700 1 -- 172.21.15.89:0/909484119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc58c159890 msgr2=0x7fc58c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:50.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.775+0000 7fc58b7fe700 1 --2- 172.21.15.89:0/909484119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc58c159890 0x7fc58c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.775+0000 7fc58b7fe700 1 -- 172.21.15.89:0/909484119 >> v1:172.21.15.89:6789/0 conn(0x7fc58c159c60 legacy=0x7fc58c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:50.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.775+0000 7fc58b7fe700 1 -- 172.21.15.89:0/909484119 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58c15e140 con 0x7fc58c069560 2023-10-10T20:41:50.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.775+0000 7fc58b7fe700 1 --2- 172.21.15.89:0/909484119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc58c069560 0x7fc58c0699e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fc584009e60 tx=0x7fc584009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e58d6779b6f8299 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:50.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.775+0000 7fc58a7fc700 1 -- 172.21.15.89:0/909484119 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58400e070 con 0x7fc58c069560 2023-10-10T20:41:50.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.775+0000 7fc58a7fc700 1 -- 172.21.15.89:0/909484119 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58400bb30 con 0x7fc58c069560 2023-10-10T20:41:50.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.775+0000 7fc58a7fc700 1 -- 172.21.15.89:0/909484119 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584012430 con 0x7fc58c069560 2023-10-10T20:41:50.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.776+0000 7fc5917ee700 1 -- 172.21.15.89:0/909484119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc58c069560 msgr2=0x7fc58c0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:50.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.776+0000 7fc5917ee700 1 --2- 172.21.15.89:0/909484119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc58c069560 0x7fc58c0699e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fc584009e60 tx=0x7fc584009780 comp rx=0 tx=0).stop 2023-10-10T20:41:50.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.776+0000 7fc5917ee700 1 -- 172.21.15.89:0/909484119 shutdown_connections 2023-10-10T20:41:50.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.776+0000 7fc5917ee700 1 --2- 172.21.15.89:0/909484119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc58c069560 0x7fc58c0699e0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.776+0000 7fc5917ee700 1 --2- 172.21.15.89:0/909484119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc58c159890 0x7fc58c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.776+0000 7fc5917ee700 1 -- 172.21.15.89:0/909484119 >> 172.21.15.89:0/909484119 conn(0x7fc58c068e20 msgr2=0x7fc58c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:50.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.776+0000 7fc5917ee700 1 -- 172.21.15.89:0/909484119 shutdown_connections 2023-10-10T20:41:50.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.776+0000 7fc5917ee700 1 -- 172.21.15.89:0/909484119 wait complete. 2023-10-10T20:41:50.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.777+0000 7fc5917ee700 1 Processor -- start 2023-10-10T20:41:50.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.777+0000 7fc5917ee700 1 -- start start 2023-10-10T20:41:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.778+0000 7fc5917ee700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc58c069560 0x7fc58c152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.778+0000 7fc5917ee700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc58c159890 0x7fc58c14d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.778+0000 7fc5917ee700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc58c159c60 0x7fc58c0b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.778+0000 7fc5917ee700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c1524f0 con 0x7fc58c159890 2023-10-10T20:41:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.778+0000 7fc5917ee700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c0b0d00 con 0x7fc58c159c60 2023-10-10T20:41:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.778+0000 7fc5917ee700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c14db50 con 0x7fc58c069560 2023-10-10T20:41:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.778+0000 7fc583fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc58c159890 0x7fc58c14d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.778+0000 7fc583fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc58c159890 0x7fc58c14d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56482/0 (socket says 172.21.15.89:56482) 2023-10-10T20:41:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.778+0000 7fc583fff700 1 -- 172.21.15.89:0/407563260 learned_addr learned my addr 172.21.15.89:0/407563260 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.778+0000 7fc58affd700 1 --2- 172.21.15.89:0/407563260 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc58c069560 0x7fc58c152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.778+0000 7fc58b7fe700 1 --2- 172.21.15.89:0/407563260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc58c159c60 0x7fc58c0b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.778+0000 7fc583fff700 1 -- 172.21.15.89:0/407563260 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc58c069560 msgr2=0x7fc58c152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.778+0000 7fc583fff700 1 --2- 172.21.15.89:0/407563260 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc58c069560 0x7fc58c152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.778+0000 7fc583fff700 1 -- 172.21.15.89:0/407563260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc58c159c60 msgr2=0x7fc58c0b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.778+0000 7fc583fff700 1 --2- 172.21.15.89:0/407563260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc58c159c60 0x7fc58c0b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.778+0000 7fc583fff700 1 -- 172.21.15.89:0/407563260 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc584009a20 con 0x7fc58c159890 2023-10-10T20:41:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.778+0000 7fc583fff700 1 --2- 172.21.15.89:0/407563260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc58c159890 0x7fc58c14d610 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fc57400c840 tx=0x7fc574007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.779+0000 7fc582ffd700 1 -- 172.21.15.89:0/407563260 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc574005b30 con 0x7fc58c159890 2023-10-10T20:41:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.779+0000 7fc582ffd700 1 -- 172.21.15.89:0/407563260 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc574005c90 con 0x7fc58c159890 2023-10-10T20:41:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.779+0000 7fc5917ee700 1 -- 172.21.15.89:0/407563260 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58c0b0a20 con 0x7fc58c159890 2023-10-10T20:41:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.779+0000 7fc582ffd700 1 -- 172.21.15.89:0/407563260 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc574007320 con 0x7fc58c159890 2023-10-10T20:41:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.779+0000 7fc5917ee700 1 -- 172.21.15.89:0/407563260 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58c152180 con 0x7fc58c159890 2023-10-10T20:41:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.780+0000 7fc5917ee700 1 -- 172.21.15.89:0/407563260 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc58c04c3c0 con 0x7fc58c159890 2023-10-10T20:41:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.780+0000 7fc582ffd700 1 -- 172.21.15.89:0/407563260 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc57401b400 con 0x7fc58c159890 2023-10-10T20:41:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.780+0000 7fc582ffd700 1 --2- 172.21.15.89:0/407563260 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc56c067320 0x7fc56c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.780+0000 7fc58affd700 1 --2- 172.21.15.89:0/407563260 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc56c067320 0x7fc56c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.781+0000 7fc582ffd700 1 -- 172.21.15.89:0/407563260 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc5740847f0 con 0x7fc58c159890 2023-10-10T20:41:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.781+0000 7fc58affd700 1 --2- 172.21.15.89:0/407563260 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc56c067320 0x7fc56c0697d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc57c006fd0 tx=0x7fc57c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.785+0000 7fc582ffd700 1 -- 172.21.15.89:0/407563260 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc574051500 con 0x7fc58c159890 2023-10-10T20:41:50.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.970+0000 7fc5917ee700 1 -- 172.21.15.89:0/407563260 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc58c152630 con 0x7fc58c159890 2023-10-10T20:41:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.971+0000 7fc582ffd700 1 -- 172.21.15.89:0/407563260 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+15377 (secure 0 0 0) 0x7fc574054b20 con 0x7fc58c159890 2023-10-10T20:41:50.973 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:50.974 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":137,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:41:49.687524+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:41:48.040814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-10T20:41:48.646159+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:41:50.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.973+0000 7fc5917ee700 1 -- 172.21.15.89:0/407563260 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc56c067320 msgr2=0x7fc56c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:50.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.973+0000 7fc5917ee700 1 --2- 172.21.15.89:0/407563260 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc56c067320 0x7fc56c0697d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc57c006fd0 tx=0x7fc57c008040 comp rx=0 tx=0).stop 2023-10-10T20:41:50.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.973+0000 7fc5917ee700 1 -- 172.21.15.89:0/407563260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc58c159890 msgr2=0x7fc58c14d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:50.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.973+0000 7fc5917ee700 1 --2- 172.21.15.89:0/407563260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc58c159890 0x7fc58c14d610 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fc57400c840 tx=0x7fc574007f00 comp rx=0 tx=0).stop 2023-10-10T20:41:50.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.973+0000 7fc5917ee700 1 -- 172.21.15.89:0/407563260 shutdown_connections 2023-10-10T20:41:50.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.973+0000 7fc5917ee700 1 --2- 172.21.15.89:0/407563260 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc56c067320 0x7fc56c0697d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.973+0000 7fc5917ee700 1 --2- 172.21.15.89:0/407563260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc58c159c60 0x7fc58c0b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.973+0000 7fc5917ee700 1 --2- 172.21.15.89:0/407563260 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc58c069560 0x7fc58c152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.973+0000 7fc5917ee700 1 --2- 172.21.15.89:0/407563260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc58c159890 0x7fc58c14d610 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:50.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.973+0000 7fc5917ee700 1 -- 172.21.15.89:0/407563260 >> 172.21.15.89:0/407563260 conn(0x7fc58c068e20 msgr2=0x7fc58c1540b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:50.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.973+0000 7fc5917ee700 1 -- 172.21.15.89:0/407563260 shutdown_connections 2023-10-10T20:41:50.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:50.973+0000 7fc5917ee700 1 -- 172.21.15.89:0/407563260 wait complete. 2023-10-10T20:41:50.988 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:41:51.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.103+0000 7f781479d700 1 Processor -- start 2023-10-10T20:41:51.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.103+0000 7f781479d700 1 -- start start 2023-10-10T20:41:51.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.104+0000 7f781479d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f780c159890 0x7f780c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:51.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.104+0000 7f781479d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f780c15a160 con 0x7f780c159c60 2023-10-10T20:41:51.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.104+0000 7f781479d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f780c0d13a0 con 0x7f780c069560 2023-10-10T20:41:51.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.104+0000 7f781479d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f780c15e140 con 0x7f780c159890 2023-10-10T20:41:51.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.104+0000 7f7811d38700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f780c159c60 0x7f780c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55806/0 (socket says 172.21.15.89:55806) 2023-10-10T20:41:51.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.104+0000 7f7812539700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f780c159890 0x7f780c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:51.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.104+0000 7f7811d38700 1 -- 172.21.15.89:0/1718358151 learned_addr learned my addr 172.21.15.89:0/1718358151 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:51.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.104+0000 7f7812539700 1 -- 172.21.15.89:0/1718358151 >> v1:172.21.15.111:6789/0 conn(0x7f780c069560 legacy=0x7f780c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:51.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.105+0000 7f7812539700 1 -- 172.21.15.89:0/1718358151 >> v1:172.21.15.89:6789/0 conn(0x7f780c159c60 legacy=0x7f780c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:51.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.105+0000 7f7812539700 1 -- 172.21.15.89:0/1718358151 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c15a3d0 con 0x7f780c159890 2023-10-10T20:41:51.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.105+0000 7f7811537700 1 -- 172.21.15.89:0/1718358151 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1634338919 0 0) 0x7f780c15a160 con 0x7f780c159c60 2023-10-10T20:41:51.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.105+0000 7f7812539700 1 --2- 172.21.15.89:0/1718358151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f780c159890 0x7f780c15abd0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f78080098e0 tx=0x7f78080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee60c72f6deaa8e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:51.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.105+0000 7f7811537700 1 -- 172.21.15.89:0/1718358151 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7808009e80 con 0x7f780c159890 2023-10-10T20:41:51.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.105+0000 7f7811537700 1 -- 172.21.15.89:0/1718358151 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780800b860 con 0x7f780c159890 2023-10-10T20:41:51.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.105+0000 7f7811537700 1 -- 172.21.15.89:0/1718358151 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780800bbe0 con 0x7f780c159890 2023-10-10T20:41:51.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.105+0000 7f781479d700 1 -- 172.21.15.89:0/1718358151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f780c159890 msgr2=0x7f780c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:51.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.105+0000 7f781479d700 1 --2- 172.21.15.89:0/1718358151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f780c159890 0x7f780c15abd0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f78080098e0 tx=0x7f78080091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:51.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.105+0000 7f781479d700 1 -- 172.21.15.89:0/1718358151 shutdown_connections 2023-10-10T20:41:51.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.105+0000 7f781479d700 1 --2- 172.21.15.89:0/1718358151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f780c159890 0x7f780c15abd0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:51.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.105+0000 7f781479d700 1 -- 172.21.15.89:0/1718358151 >> 172.21.15.89:0/1718358151 conn(0x7f780c068e20 msgr2=0x7f780c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:51.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.105+0000 7f781479d700 1 -- 172.21.15.89:0/1718358151 shutdown_connections 2023-10-10T20:41:51.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.105+0000 7f781479d700 1 -- 172.21.15.89:0/1718358151 wait complete. 2023-10-10T20:41:51.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.106+0000 7f781479d700 1 Processor -- start 2023-10-10T20:41:51.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.107+0000 7f781479d700 1 -- start start 2023-10-10T20:41:51.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.107+0000 7f781479d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f780c069560 0x7f780c151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:51.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.107+0000 7f781479d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f780c159890 0x7f780c152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:51.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.107+0000 7f781479d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f780c159c60 0x7f780c15d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:51.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.107+0000 7f781479d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c15de90 con 0x7f780c069560 2023-10-10T20:41:51.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.107+0000 7f781479d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c152730 con 0x7f780c159c60 2023-10-10T20:41:51.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.107+0000 7f781479d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780c151b20 con 0x7f780c159890 2023-10-10T20:41:51.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.107+0000 7f7812539700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f780c069560 0x7f780c151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:51.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.107+0000 7f7812539700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f780c069560 0x7f780c151540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56504/0 (socket says 172.21.15.89:56504) 2023-10-10T20:41:51.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.107+0000 7f7812539700 1 -- 172.21.15.89:0/3863719573 learned_addr learned my addr 172.21.15.89:0/3863719573 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:51.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.107+0000 7f7812539700 1 -- 172.21.15.89:0/3863719573 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f780c159890 msgr2=0x7f780c152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:51.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.107+0000 7f7812539700 1 --2- 172.21.15.89:0/3863719573 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f780c159890 0x7f780c152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:51.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.107+0000 7f7812539700 1 -- 172.21.15.89:0/3863719573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f780c159c60 msgr2=0x7f780c15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:51.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.107+0000 7f7812539700 1 --2- 172.21.15.89:0/3863719573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f780c159c60 0x7f780c15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:51.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.108+0000 7f7812539700 1 -- 172.21.15.89:0/3863719573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7808009420 con 0x7f780c069560 2023-10-10T20:41:51.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.108+0000 7f7812539700 1 --2- 172.21.15.89:0/3863719573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f780c069560 0x7f780c151540 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f780c070e20 tx=0x7f7808017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:51.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.108+0000 7f78027fc700 1 -- 172.21.15.89:0/3863719573 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780800baa0 con 0x7f780c069560 2023-10-10T20:41:51.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.108+0000 7f78027fc700 1 -- 172.21.15.89:0/3863719573 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7808025420 con 0x7f780c069560 2023-10-10T20:41:51.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.108+0000 7f78027fc700 1 -- 172.21.15.89:0/3863719573 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78080257e0 con 0x7f780c069560 2023-10-10T20:41:51.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.108+0000 7f781479d700 1 -- 172.21.15.89:0/3863719573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780c151e00 con 0x7f780c069560 2023-10-10T20:41:51.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.108+0000 7f781479d700 1 -- 172.21.15.89:0/3863719573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780c1523c0 con 0x7f780c069560 2023-10-10T20:41:51.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.109+0000 7f781479d700 1 -- 172.21.15.89:0/3863719573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780c152a10 con 0x7f780c069560 2023-10-10T20:41:51.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.109+0000 7f78027fc700 1 -- 172.21.15.89:0/3863719573 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7808023070 con 0x7f780c069560 2023-10-10T20:41:51.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.110+0000 7f78027fc700 1 --2- 172.21.15.89:0/3863719573 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f77f0067320 0x7f77f00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:51.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.110+0000 7f7811d38700 1 --2- 172.21.15.89:0/3863719573 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f77f0067320 0x7f77f00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:51.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.110+0000 7f78027fc700 1 -- 172.21.15.89:0/3863719573 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7808086440 con 0x7f780c069560 2023-10-10T20:41:51.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.110+0000 7f7811d38700 1 --2- 172.21.15.89:0/3863719573 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f77f0067320 0x7f77f00697d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f780c0cc720 tx=0x7f77fc007bd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:51.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.114+0000 7f78027fc700 1 -- 172.21.15.89:0/3863719573 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78080530d0 con 0x7f780c069560 2023-10-10T20:41:51.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:51.342+0000 7f781479d700 1 -- 172.21.15.89:0/3863719573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f780c04c3c0 con 0x7f780c069560 2023-10-10T20:41:52.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.324+0000 7f78027fc700 1 -- 172.21.15.89:0/3863719573 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v90) v1 ==== 118+0+0 (secure 0 0 0) 0x7f78080566f0 con 0x7f780c069560 2023-10-10T20:41:52.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.327+0000 7f781479d700 1 -- 172.21.15.89:0/3863719573 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f77f0067320 msgr2=0x7f77f00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:52.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.327+0000 7f781479d700 1 --2- 172.21.15.89:0/3863719573 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f77f0067320 0x7f77f00697d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f780c0cc720 tx=0x7f77fc007bd0 comp rx=0 tx=0).stop 2023-10-10T20:41:52.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.327+0000 7f781479d700 1 -- 172.21.15.89:0/3863719573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f780c069560 msgr2=0x7f780c151540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:52.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.327+0000 7f781479d700 1 --2- 172.21.15.89:0/3863719573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f780c069560 0x7f780c151540 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f780c070e20 tx=0x7f7808017000 comp rx=0 tx=0).stop 2023-10-10T20:41:52.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.327+0000 7f781479d700 1 -- 172.21.15.89:0/3863719573 shutdown_connections 2023-10-10T20:41:52.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.327+0000 7f781479d700 1 --2- 172.21.15.89:0/3863719573 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f77f0067320 0x7f77f00697d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:52.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.327+0000 7f781479d700 1 --2- 172.21.15.89:0/3863719573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f780c159c60 0x7f780c15d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:52.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.327+0000 7f781479d700 1 --2- 172.21.15.89:0/3863719573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f780c069560 0x7f780c151540 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:52.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.327+0000 7f781479d700 1 --2- 172.21.15.89:0/3863719573 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f780c159890 0x7f780c152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:52.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.327+0000 7f781479d700 1 -- 172.21.15.89:0/3863719573 >> 172.21.15.89:0/3863719573 conn(0x7f780c068e20 msgr2=0x7f780c0c7130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:52.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.327+0000 7f781479d700 1 -- 172.21.15.89:0/3863719573 shutdown_connections 2023-10-10T20:41:52.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.327+0000 7f781479d700 1 -- 172.21.15.89:0/3863719573 wait complete. 2023-10-10T20:41:52.344 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:41:52.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.465+0000 7f2d4172e700 1 Processor -- start 2023-10-10T20:41:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.465+0000 7f2d4172e700 1 -- start start 2023-10-10T20:41:52.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.466+0000 7f2d4172e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d3c069560 0x7f2d3c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:52.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.466+0000 7f2d4172e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d3c069ee0 con 0x7f2d3c159890 2023-10-10T20:41:52.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.466+0000 7f2d4172e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d3c15a160 con 0x7f2d3c159c60 2023-10-10T20:41:52.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.466+0000 7f2d4172e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c0d13a0 con 0x7f2d3c069560 2023-10-10T20:41:52.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.466+0000 7f2d3affd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2d3c159890 0x7f2d3c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55820/0 (socket says 172.21.15.89:55820) 2023-10-10T20:41:52.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.466+0000 7f2d3affd700 1 -- 172.21.15.89:0/32698968 learned_addr learned my addr 172.21.15.89:0/32698968 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:52.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d3b7fe700 1 --2- 172.21.15.89:0/32698968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d3c069560 0x7f2d3c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:52.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d3a7fc700 1 -- 172.21.15.89:0/32698968 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4126076516 0 0) 0x7f2d3c069ee0 con 0x7f2d3c159890 2023-10-10T20:41:52.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d3a7fc700 1 -- 172.21.15.89:0/32698968 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d28003570 con 0x7f2d3c159890 2023-10-10T20:41:52.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d3a7fc700 1 -- 172.21.15.89:0/32698968 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 679223561 0 0) 0x7f2d28003570 con 0x7f2d3c159890 2023-10-10T20:41:52.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d3a7fc700 1 -- 172.21.15.89:0/32698968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d3c069560 msgr2=0x7f2d3c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:52.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d3a7fc700 1 --2- 172.21.15.89:0/32698968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d3c069560 0x7f2d3c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:52.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d3a7fc700 1 -- 172.21.15.89:0/32698968 >> v1:172.21.15.111:6789/0 conn(0x7f2d3c159c60 legacy=0x7f2d3c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:52.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d3a7fc700 1 -- 172.21.15.89:0/32698968 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3c15e140 con 0x7f2d3c159890 2023-10-10T20:41:52.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d3a7fc700 1 -- 172.21.15.89:0/32698968 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f2d2c001470 con 0x7f2d3c159890 2023-10-10T20:41:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d3a7fc700 1 -- 172.21.15.89:0/32698968 >> v1:172.21.15.89:6789/0 conn(0x7f2d3c159890 legacy=0x7f2d3c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d3a7fc700 1 --2- 172.21.15.89:0/32698968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d28003a90 0x7f2d28003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d3a7fc700 1 --2- 172.21.15.89:0/32698968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d28004810 0x7f2d28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d3a7fc700 1 --2- 172.21.15.89:0/32698968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d3c069560 0x7f2d280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d3a7fc700 1 -- 172.21.15.89:0/32698968 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c0d13a0 con 0x7f2d28004810 2023-10-10T20:41:52.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d3a7fc700 1 -- 172.21.15.89:0/32698968 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2800bb40 con 0x7f2d28003a90 2023-10-10T20:41:52.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d3a7fc700 1 -- 172.21.15.89:0/32698968 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2800bd00 con 0x7f2d3c069560 2023-10-10T20:41:52.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d327fc700 1 --2- 172.21.15.89:0/32698968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d28004810 0x7f2d28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:52.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d3affd700 1 --2- 172.21.15.89:0/32698968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d28003a90 0x7f2d28003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:52.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d3b7fe700 1 --2- 172.21.15.89:0/32698968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d3c069560 0x7f2d280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:52.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d327fc700 1 -- 172.21.15.89:0/32698968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d3c069560 msgr2=0x7f2d280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:52.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d327fc700 1 --2- 172.21.15.89:0/32698968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d3c069560 0x7f2d280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:52.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d327fc700 1 -- 172.21.15.89:0/32698968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d28003a90 msgr2=0x7f2d28003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:52.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d327fc700 1 --2- 172.21.15.89:0/32698968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d28003a90 0x7f2d28003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:52.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.467+0000 7f2d327fc700 1 -- 172.21.15.89:0/32698968 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d28003570 con 0x7f2d28004810 2023-10-10T20:41:52.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.468+0000 7f2d327fc700 1 --2- 172.21.15.89:0/32698968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d28004810 0x7f2d28004c80 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f2d24001970 tx=0x7f2d24007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30c0d876c6b0484 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:52.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.468+0000 7f2d3a7fc700 1 -- 172.21.15.89:0/32698968 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2400c5d0 con 0x7f2d28004810 2023-10-10T20:41:52.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.468+0000 7f2d3a7fc700 1 -- 172.21.15.89:0/32698968 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2400c730 con 0x7f2d28004810 2023-10-10T20:41:52.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.468+0000 7f2d3a7fc700 1 -- 172.21.15.89:0/32698968 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2400cad0 con 0x7f2d28004810 2023-10-10T20:41:52.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.468+0000 7f2d4172e700 1 -- 172.21.15.89:0/32698968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d28004810 msgr2=0x7f2d28004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:52.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.468+0000 7f2d4172e700 1 --2- 172.21.15.89:0/32698968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d28004810 0x7f2d28004c80 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f2d24001970 tx=0x7f2d24007d30 comp rx=0 tx=0).stop 2023-10-10T20:41:52.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.468+0000 7f2d3b7fe700 1 -- 172.21.15.89:0/32698968 reap_dead start 2023-10-10T20:41:52.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.468+0000 7f2d4172e700 1 -- 172.21.15.89:0/32698968 shutdown_connections 2023-10-10T20:41:52.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.468+0000 7f2d4172e700 1 -- 172.21.15.89:0/32698968 >> 172.21.15.89:0/32698968 conn(0x7f2d3c068e20 msgr2=0x7f2d3c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:52.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.468+0000 7f2d4172e700 1 -- 172.21.15.89:0/32698968 shutdown_connections 2023-10-10T20:41:52.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.468+0000 7f2d4172e700 1 -- 172.21.15.89:0/32698968 wait complete. 2023-10-10T20:41:52.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.470+0000 7f2d4172e700 1 Processor -- start 2023-10-10T20:41:52.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.470+0000 7f2d4172e700 1 -- start start 2023-10-10T20:41:52.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.470+0000 7f2d4172e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d28004810 0x7f2d3c14d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:52.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.470+0000 7f2d4172e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d3c0cae60 0x7f2d3c0cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:52.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.470+0000 7f2d4172e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d3c15c540 0x7f2d3c15c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:52.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.470+0000 7f2d4172e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c0cb810 con 0x7f2d3c0cae60 2023-10-10T20:41:52.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.470+0000 7f2d4172e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c15de60 con 0x7f2d28004810 2023-10-10T20:41:52.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.470+0000 7f2d4172e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c14d890 con 0x7f2d3c15c540 2023-10-10T20:41:52.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.470+0000 7f2d327fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d3c0cae60 0x7f2d3c0cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:52.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.470+0000 7f2d327fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d3c0cae60 0x7f2d3c0cb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56514/0 (socket says 172.21.15.89:56514) 2023-10-10T20:41:52.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.470+0000 7f2d327fc700 1 -- 172.21.15.89:0/1015885331 learned_addr learned my addr 172.21.15.89:0/1015885331 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:52.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.471+0000 7f2d3b7fe700 1 --2- 172.21.15.89:0/1015885331 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d3c15c540 0x7f2d3c15c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:52.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.471+0000 7f2d327fc700 1 -- 172.21.15.89:0/1015885331 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d3c15c540 msgr2=0x7f2d3c15c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:52.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.471+0000 7f2d3affd700 1 --2- 172.21.15.89:0/1015885331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d28004810 0x7f2d3c14d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:52.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.471+0000 7f2d327fc700 1 --2- 172.21.15.89:0/1015885331 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d3c15c540 0x7f2d3c15c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:52.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.471+0000 7f2d327fc700 1 -- 172.21.15.89:0/1015885331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d28004810 msgr2=0x7f2d3c14d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:52.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.471+0000 7f2d327fc700 1 --2- 172.21.15.89:0/1015885331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d28004810 0x7f2d3c14d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:52.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.471+0000 7f2d327fc700 1 -- 172.21.15.89:0/1015885331 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3c15d2e0 con 0x7f2d3c0cae60 2023-10-10T20:41:52.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.471+0000 7f2d3affd700 1 --2- 172.21.15.89:0/1015885331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d28004810 0x7f2d3c14d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:52.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.471+0000 7f2d327fc700 1 --2- 172.21.15.89:0/1015885331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d3c0cae60 0x7f2d3c0cb2d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400c3c0 tx=0x7f2d24015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:52.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.471+0000 7f2d337fe700 1 -- 172.21.15.89:0/1015885331 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2400c5d0 con 0x7f2d3c0cae60 2023-10-10T20:41:52.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.471+0000 7f2d337fe700 1 -- 172.21.15.89:0/1015885331 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2400cab0 con 0x7f2d3c0cae60 2023-10-10T20:41:52.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.471+0000 7f2d4172e700 1 -- 172.21.15.89:0/1015885331 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d3c15d5c0 con 0x7f2d3c0cae60 2023-10-10T20:41:52.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.471+0000 7f2d337fe700 1 -- 172.21.15.89:0/1015885331 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24006a70 con 0x7f2d3c0cae60 2023-10-10T20:41:52.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.471+0000 7f2d4172e700 1 -- 172.21.15.89:0/1015885331 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d3c15db80 con 0x7f2d3c0cae60 2023-10-10T20:41:52.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.472+0000 7f2d4172e700 1 -- 172.21.15.89:0/1015885331 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d3c0b3590 con 0x7f2d3c0cae60 2023-10-10T20:41:52.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.472+0000 7f2d337fe700 1 -- 172.21.15.89:0/1015885331 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2d24019070 con 0x7f2d3c0cae60 2023-10-10T20:41:52.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.473+0000 7f2d337fe700 1 --2- 172.21.15.89:0/1015885331 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2d1c0673a0 0x7f2d1c069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:52.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.473+0000 7f2d3affd700 1 --2- 172.21.15.89:0/1015885331 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2d1c0673a0 0x7f2d1c069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:52.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.473+0000 7f2d337fe700 1 -- 172.21.15.89:0/1015885331 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2d24085860 con 0x7f2d3c0cae60 2023-10-10T20:41:52.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.473+0000 7f2d3affd700 1 --2- 172.21.15.89:0/1015885331 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2d1c0673a0 0x7f2d1c069850 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c007e70 tx=0x7f2d2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:52.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.477+0000 7f2d337fe700 1 -- 172.21.15.89:0/1015885331 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d3c0b3590 con 0x7f2d3c0cae60 2023-10-10T20:41:52.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:52.702+0000 7f2d4172e700 1 -- 172.21.15.89:0/1015885331 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2d3c0b3590 con 0x7f2d3c0cae60 2023-10-10T20:41:53.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.329+0000 7f2d337fe700 1 -- 172.21.15.89:0/1015885331 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v91) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2d3c0b3590 con 0x7f2d3c0cae60 2023-10-10T20:41:53.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.332+0000 7f2d4172e700 1 -- 172.21.15.89:0/1015885331 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2d1c0673a0 msgr2=0x7f2d1c069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:53.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.332+0000 7f2d4172e700 1 --2- 172.21.15.89:0/1015885331 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2d1c0673a0 0x7f2d1c069850 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c007e70 tx=0x7f2d2c009040 comp rx=0 tx=0).stop 2023-10-10T20:41:53.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.332+0000 7f2d4172e700 1 -- 172.21.15.89:0/1015885331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d3c0cae60 msgr2=0x7f2d3c0cb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:53.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.332+0000 7f2d4172e700 1 --2- 172.21.15.89:0/1015885331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d3c0cae60 0x7f2d3c0cb2d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400c3c0 tx=0x7f2d24015000 comp rx=0 tx=0).stop 2023-10-10T20:41:53.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.332+0000 7f2d4172e700 1 -- 172.21.15.89:0/1015885331 shutdown_connections 2023-10-10T20:41:53.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.332+0000 7f2d4172e700 1 --2- 172.21.15.89:0/1015885331 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d3c15c540 0x7f2d3c15c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:53.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.332+0000 7f2d4172e700 1 --2- 172.21.15.89:0/1015885331 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2d1c0673a0 0x7f2d1c069850 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:53.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.332+0000 7f2d4172e700 1 --2- 172.21.15.89:0/1015885331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d28004810 0x7f2d3c14d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:53.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.332+0000 7f2d4172e700 1 --2- 172.21.15.89:0/1015885331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d3c0cae60 0x7f2d3c0cb2d0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:53.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.332+0000 7f2d4172e700 1 -- 172.21.15.89:0/1015885331 >> 172.21.15.89:0/1015885331 conn(0x7f2d3c068e20 msgr2=0x7f2d3c0b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:53.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.332+0000 7f2d4172e700 1 -- 172.21.15.89:0/1015885331 shutdown_connections 2023-10-10T20:41:53.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.332+0000 7f2d4172e700 1 -- 172.21.15.89:0/1015885331 wait complete. 2023-10-10T20:41:53.346 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:41:53.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.459+0000 7ff85a746700 1 Processor -- start 2023-10-10T20:41:53.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.459+0000 7ff85a746700 1 -- start start 2023-10-10T20:41:53.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.460+0000 7ff85a746700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff854159c60 0x7ff85415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:53.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.460+0000 7ff85a746700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff85415db10 0x7ff854069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:53.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.460+0000 7ff85a746700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff85415a160 con 0x7ff854159890 2023-10-10T20:41:53.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.460+0000 7ff85a746700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8540d13a0 con 0x7ff85415db10 2023-10-10T20:41:53.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.460+0000 7ff85a746700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85415e140 con 0x7ff854159c60 2023-10-10T20:41:53.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.460+0000 7ff858ce3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff85415db10 0x7ff854069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:53.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.460+0000 7ff858ce3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff85415db10 0x7ff854069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56526/0 (socket says 172.21.15.89:56526) 2023-10-10T20:41:53.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.460+0000 7ff858ce3700 1 -- 172.21.15.89:0/1742749781 learned_addr learned my addr 172.21.15.89:0/1742749781 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:53.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.460+0000 7ff858ce3700 1 -- 172.21.15.89:0/1742749781 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff854159c60 msgr2=0x7ff85415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:53.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.460+0000 7ff858ce3700 1 --2- 172.21.15.89:0/1742749781 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff854159c60 0x7ff85415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:53.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.460+0000 7ff858ce3700 1 -- 172.21.15.89:0/1742749781 >> v1:172.21.15.111:6789/0 conn(0x7ff854159890 legacy=0x7ff85415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:53.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.461+0000 7ff858ce3700 1 -- 172.21.15.89:0/1742749781 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85415a3d0 con 0x7ff85415db10 2023-10-10T20:41:53.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.461+0000 7ff858ce3700 1 --2- 172.21.15.89:0/1742749781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff85415db10 0x7ff854069560 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009f20 tx=0x7ff84c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd1e02d78636e20 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:53.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.461+0000 7ff852ffd700 1 -- 172.21.15.89:0/1742749781 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c00e070 con 0x7ff85415db10 2023-10-10T20:41:53.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.461+0000 7ff852ffd700 1 -- 172.21.15.89:0/1742749781 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84c00ba40 con 0x7ff85415db10 2023-10-10T20:41:53.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.461+0000 7ff852ffd700 1 -- 172.21.15.89:0/1742749781 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c0092e0 con 0x7ff85415db10 2023-10-10T20:41:53.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.461+0000 7ff85a746700 1 -- 172.21.15.89:0/1742749781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff85415db10 msgr2=0x7ff854069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:53.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.461+0000 7ff85a746700 1 --2- 172.21.15.89:0/1742749781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff85415db10 0x7ff854069560 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009f20 tx=0x7ff84c009840 comp rx=0 tx=0).stop 2023-10-10T20:41:53.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.461+0000 7ff85a746700 1 -- 172.21.15.89:0/1742749781 shutdown_connections 2023-10-10T20:41:53.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.461+0000 7ff85a746700 1 --2- 172.21.15.89:0/1742749781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff85415db10 0x7ff854069560 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:53.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.461+0000 7ff85a746700 1 --2- 172.21.15.89:0/1742749781 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff854159c60 0x7ff85415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:53.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.461+0000 7ff85a746700 1 -- 172.21.15.89:0/1742749781 >> 172.21.15.89:0/1742749781 conn(0x7ff854068e20 msgr2=0x7ff8540d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:53.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.461+0000 7ff85a746700 1 -- 172.21.15.89:0/1742749781 shutdown_connections 2023-10-10T20:41:53.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.461+0000 7ff85a746700 1 -- 172.21.15.89:0/1742749781 wait complete. 2023-10-10T20:41:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.462+0000 7ff85a746700 1 Processor -- start 2023-10-10T20:41:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.463+0000 7ff85a746700 1 -- start start 2023-10-10T20:41:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.463+0000 7ff85a746700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff854159890 0x7ff854150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.463+0000 7ff85a746700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff854159c60 0x7ff8540cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.463+0000 7ff85a746700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff85415db10 0x7ff8540cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.463+0000 7ff85a746700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8540cff20 con 0x7ff854159890 2023-10-10T20:41:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.463+0000 7ff85a746700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8540b1490 con 0x7ff85415db10 2023-10-10T20:41:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.463+0000 7ff85a746700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8540b1770 con 0x7ff854159c60 2023-10-10T20:41:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.463+0000 7ff853fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff854159890 0x7ff854150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.463+0000 7ff853fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff854159890 0x7ff854150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56536/0 (socket says 172.21.15.89:56536) 2023-10-10T20:41:53.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.463+0000 7ff853fff700 1 -- 172.21.15.89:0/4287151621 learned_addr learned my addr 172.21.15.89:0/4287151621 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:53.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.464+0000 7ff853fff700 1 -- 172.21.15.89:0/4287151621 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff854159c60 msgr2=0x7ff8540cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:53.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.464+0000 7ff853fff700 1 --2- 172.21.15.89:0/4287151621 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff854159c60 0x7ff8540cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:53.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.464+0000 7ff853fff700 1 -- 172.21.15.89:0/4287151621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff85415db10 msgr2=0x7ff8540cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:53.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.464+0000 7ff853fff700 1 --2- 172.21.15.89:0/4287151621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff85415db10 0x7ff8540cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.464+0000 7ff853fff700 1 -- 172.21.15.89:0/4287151621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff84c009ae0 con 0x7ff854159890 2023-10-10T20:41:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.464+0000 7ff853fff700 1 --2- 172.21.15.89:0/4287151621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff854159890 0x7ff854150ec0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7ff84400c5d0 tx=0x7ff844007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.464+0000 7ff83bfff700 1 -- 172.21.15.89:0/4287151621 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff844011590 con 0x7ff854159890 2023-10-10T20:41:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.464+0000 7ff83bfff700 1 -- 172.21.15.89:0/4287151621 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff844011710 con 0x7ff854159890 2023-10-10T20:41:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.464+0000 7ff83bfff700 1 -- 172.21.15.89:0/4287151621 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff844011ad0 con 0x7ff854159890 2023-10-10T20:41:53.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.464+0000 7ff85a746700 1 -- 172.21.15.89:0/4287151621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8540b1a50 con 0x7ff854159890 2023-10-10T20:41:53.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.464+0000 7ff85a746700 1 -- 172.21.15.89:0/4287151621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8540b0720 con 0x7ff854159890 2023-10-10T20:41:53.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.465+0000 7ff85a746700 1 -- 172.21.15.89:0/4287151621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8541537c0 con 0x7ff854159890 2023-10-10T20:41:53.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.465+0000 7ff83bfff700 1 -- 172.21.15.89:0/4287151621 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff84401c460 con 0x7ff854159890 2023-10-10T20:41:53.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.466+0000 7ff83bfff700 1 --2- 172.21.15.89:0/4287151621 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff840067320 0x7ff8400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:53.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.466+0000 7ff83bfff700 1 -- 172.21.15.89:0/4287151621 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff8440849d0 con 0x7ff854159890 2023-10-10T20:41:53.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.466+0000 7ff8537fe700 1 --2- 172.21.15.89:0/4287151621 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff840067320 0x7ff8400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:53.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.466+0000 7ff8537fe700 1 --2- 172.21.15.89:0/4287151621 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff840067320 0x7ff8400697d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff848006fd0 tx=0x7ff848006db0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:53.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.470+0000 7ff83bfff700 1 -- 172.21.15.89:0/4287151621 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff844051660 con 0x7ff854159890 2023-10-10T20:41:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.697+0000 7ff85a746700 1 -- 172.21.15.89:0/4287151621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff85404c3c0 con 0x7ff854159890 2023-10-10T20:41:53.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.698+0000 7ff83bfff700 1 -- 172.21.15.89:0/4287151621 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4601 (secure 0 0 0) 0x7ff844054c80 con 0x7ff854159890 2023-10-10T20:41:53.700 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:53.700 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":91,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6197,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/4123310016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4123310016},{"type":"v1","addr":"172.21.15.111:6834","nonce":4123310016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":6200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/141017824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":141017824},{"type":"v1","addr":"172.21.15.111:6835","nonce":141017824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7575,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/728309352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":728309352},{"type":"v1","addr":"172.21.15.89:6836","nonce":728309352}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:41:49.687845+0000","modified":"2023-10-10T20:41:53.324518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7578":{"gid":7578,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/2102263806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2102263806},{"type":"v1","addr":"172.21.15.89:6837","nonce":2102263806}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-10T20:41:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.699+0000 7ff85a746700 1 -- 172.21.15.89:0/4287151621 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff840067320 msgr2=0x7ff8400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.699+0000 7ff85a746700 1 --2- 172.21.15.89:0/4287151621 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff840067320 0x7ff8400697d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff848006fd0 tx=0x7ff848006db0 comp rx=0 tx=0).stop 2023-10-10T20:41:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.700+0000 7ff85a746700 1 -- 172.21.15.89:0/4287151621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff854159890 msgr2=0x7ff854150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.700+0000 7ff85a746700 1 --2- 172.21.15.89:0/4287151621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff854159890 0x7ff854150ec0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7ff84400c5d0 tx=0x7ff844007f60 comp rx=0 tx=0).stop 2023-10-10T20:41:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.700+0000 7ff85a746700 1 -- 172.21.15.89:0/4287151621 shutdown_connections 2023-10-10T20:41:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.700+0000 7ff85a746700 1 --2- 172.21.15.89:0/4287151621 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff840067320 0x7ff8400697d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.700+0000 7ff85a746700 1 --2- 172.21.15.89:0/4287151621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff85415db10 0x7ff8540cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.700+0000 7ff85a746700 1 --2- 172.21.15.89:0/4287151621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff854159890 0x7ff854150ec0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.700+0000 7ff85a746700 1 --2- 172.21.15.89:0/4287151621 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff854159c60 0x7ff8540cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:53.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.700+0000 7ff85a746700 1 -- 172.21.15.89:0/4287151621 >> 172.21.15.89:0/4287151621 conn(0x7ff854068e20 msgr2=0x7ff854157550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:53.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.700+0000 7ff85a746700 1 -- 172.21.15.89:0/4287151621 shutdown_connections 2023-10-10T20:41:53.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.700+0000 7ff85a746700 1 -- 172.21.15.89:0/4287151621 wait complete. 2023-10-10T20:41:53.704 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 91 2023-10-10T20:41:53.714 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:41:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.826+0000 7ff8eb47a700 1 Processor -- start 2023-10-10T20:41:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.826+0000 7ff8eb47a700 1 -- start start 2023-10-10T20:41:53.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.827+0000 7ff8eb47a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff8e41597f0 0x7ff8e40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:53.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.827+0000 7ff8eb47a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e40b11d0 con 0x7ff8e415d230 2023-10-10T20:41:53.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.827+0000 7ff8eb47a700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e40b14b0 con 0x7ff8e4159420 2023-10-10T20:41:53.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.827+0000 7ff8eb47a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e40b1790 con 0x7ff8e41597f0 2023-10-10T20:41:53.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.827+0000 7ff8eac79700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff8e415d230 0x7ff8e415d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55826/0 (socket says 172.21.15.89:55826) 2023-10-10T20:41:53.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.827+0000 7ff8e9c77700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff8e41597f0 0x7ff8e40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:53.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.827+0000 7ff8eac79700 1 -- 172.21.15.89:0/1732510732 learned_addr learned my addr 172.21.15.89:0/1732510732 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:53.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.827+0000 7ff8e9476700 1 -- 172.21.15.89:0/1732510732 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399778097 0 0) 0x7ff8e40b11d0 con 0x7ff8e415d230 2023-10-10T20:41:53.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.827+0000 7ff8e9476700 1 -- 172.21.15.89:0/1732510732 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8d0003570 con 0x7ff8e415d230 2023-10-10T20:41:53.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.827+0000 7ff8e9c77700 1 -- 172.21.15.89:0/1732510732 >> v1:172.21.15.111:6789/0 conn(0x7ff8e4159420 legacy=0x7ff8e40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:53.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.828+0000 7ff8e9c77700 1 -- 172.21.15.89:0/1732510732 >> v1:172.21.15.89:6789/0 conn(0x7ff8e415d230 legacy=0x7ff8e415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:53.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.828+0000 7ff8e9c77700 1 -- 172.21.15.89:0/1732510732 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e40b1a70 con 0x7ff8e41597f0 2023-10-10T20:41:53.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.828+0000 7ff8e9c77700 1 --2- 172.21.15.89:0/1732510732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8e41597f0 0x7ff8e40b2270 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40098e0 tx=0x7ff8d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b03ef38b2c2721 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:53.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.828+0000 7ff8e9476700 1 -- 172.21.15.89:0/1732510732 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d400b640 con 0x7ff8e41597f0 2023-10-10T20:41:53.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.828+0000 7ff8e9476700 1 -- 172.21.15.89:0/1732510732 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d400b7a0 con 0x7ff8e41597f0 2023-10-10T20:41:53.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.828+0000 7ff8e9476700 1 -- 172.21.15.89:0/1732510732 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d400bb40 con 0x7ff8e41597f0 2023-10-10T20:41:53.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.828+0000 7ff8eb47a700 1 -- 172.21.15.89:0/1732510732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8e41597f0 msgr2=0x7ff8e40b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:53.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.828+0000 7ff8eb47a700 1 --2- 172.21.15.89:0/1732510732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8e41597f0 0x7ff8e40b2270 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40098e0 tx=0x7ff8d40091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:53.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.828+0000 7ff8eb47a700 1 -- 172.21.15.89:0/1732510732 shutdown_connections 2023-10-10T20:41:53.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.828+0000 7ff8eb47a700 1 --2- 172.21.15.89:0/1732510732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8e41597f0 0x7ff8e40b2270 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:53.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.828+0000 7ff8eb47a700 1 -- 172.21.15.89:0/1732510732 >> 172.21.15.89:0/1732510732 conn(0x7ff8e40d0660 msgr2=0x7ff8e40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:53.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.828+0000 7ff8eb47a700 1 -- 172.21.15.89:0/1732510732 shutdown_connections 2023-10-10T20:41:53.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.828+0000 7ff8eb47a700 1 -- 172.21.15.89:0/1732510732 wait complete. 2023-10-10T20:41:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.830+0000 7ff8eb47a700 1 Processor -- start 2023-10-10T20:41:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.830+0000 7ff8eb47a700 1 -- start start 2023-10-10T20:41:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.830+0000 7ff8eb47a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff8e4159420 0x7ff8e4154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.830+0000 7ff8eb47a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8e41597f0 0x7ff8e40c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:53.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.830+0000 7ff8eb47a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8e415d230 0x7ff8e40c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:53.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.830+0000 7ff8eb47a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e415bf80 con 0x7ff8e415d230 2023-10-10T20:41:53.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.830+0000 7ff8eb47a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e41695d0 con 0x7ff8e41597f0 2023-10-10T20:41:53.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.830+0000 7ff8eb47a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e41698b0 con 0x7ff8e4159420 2023-10-10T20:41:53.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.830+0000 7ff8eac79700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8e415d230 0x7ff8e40c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:53.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.830+0000 7ff8eac79700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8e415d230 0x7ff8e40c6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56560/0 (socket says 172.21.15.89:56560) 2023-10-10T20:41:53.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.830+0000 7ff8eac79700 1 -- 172.21.15.89:0/1127714597 learned_addr learned my addr 172.21.15.89:0/1127714597 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:53.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.831+0000 7ff8e9c77700 1 --2- 172.21.15.89:0/1127714597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8e41597f0 0x7ff8e40c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:53.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.831+0000 7ff8eac79700 1 -- 172.21.15.89:0/1127714597 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff8e4159420 msgr2=0x7ff8e4154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:53.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.831+0000 7ff8eac79700 1 --2- 172.21.15.89:0/1127714597 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff8e4159420 0x7ff8e4154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:53.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.831+0000 7ff8eac79700 1 -- 172.21.15.89:0/1127714597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8e41597f0 msgr2=0x7ff8e40c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:53.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.831+0000 7ff8eac79700 1 --2- 172.21.15.89:0/1127714597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8e41597f0 0x7ff8e40c5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:53.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.831+0000 7ff8eac79700 1 -- 172.21.15.89:0/1127714597 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d4009420 con 0x7ff8e415d230 2023-10-10T20:41:53.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.831+0000 7ff8eac79700 1 --2- 172.21.15.89:0/1127714597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8e415d230 0x7ff8e40c6300 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00cc70 tx=0x7ff8dc001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:53.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.831+0000 7ff8da7fc700 1 -- 172.21.15.89:0/1127714597 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc00ce20 con 0x7ff8e415d230 2023-10-10T20:41:53.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.831+0000 7ff8da7fc700 1 -- 172.21.15.89:0/1127714597 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8dc00ab30 con 0x7ff8e415d230 2023-10-10T20:41:53.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.831+0000 7ff8eb47a700 1 -- 172.21.15.89:0/1127714597 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e40c7220 con 0x7ff8e415d230 2023-10-10T20:41:53.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.831+0000 7ff8da7fc700 1 -- 172.21.15.89:0/1127714597 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0070f0 con 0x7ff8e415d230 2023-10-10T20:41:53.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.831+0000 7ff8eb47a700 1 -- 172.21.15.89:0/1127714597 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e40c6c60 con 0x7ff8e415d230 2023-10-10T20:41:53.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.832+0000 7ff8eb47a700 1 -- 172.21.15.89:0/1127714597 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e404c3c0 con 0x7ff8e415d230 2023-10-10T20:41:53.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.833+0000 7ff8da7fc700 1 -- 172.21.15.89:0/1127714597 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff8dc0074e0 con 0x7ff8e415d230 2023-10-10T20:41:53.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.833+0000 7ff8da7fc700 1 --2- 172.21.15.89:0/1127714597 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff8c8067320 0x7ff8c80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:53.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.833+0000 7ff8da7fc700 1 -- 172.21.15.89:0/1127714597 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff8dc0850e0 con 0x7ff8e415d230 2023-10-10T20:41:53.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.833+0000 7ff8ea478700 1 --2- 172.21.15.89:0/1127714597 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff8c8067320 0x7ff8c80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:53.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.833+0000 7ff8ea478700 1 --2- 172.21.15.89:0/1127714597 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff8c8067320 0x7ff8c80697d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0001730 tx=0x7ff8e0007b40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:53.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:53.837+0000 7ff8da7fc700 1 -- 172.21.15.89:0/1127714597 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff8dc051d60 con 0x7ff8e415d230 2023-10-10T20:41:54.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.022+0000 7ff8eb47a700 1 -- 172.21.15.89:0/1127714597 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8e40b1d50 con 0x7ff8e415d230 2023-10-10T20:41:54.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.022+0000 7ff8da7fc700 1 -- 172.21.15.89:0/1127714597 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+15364 (secure 0 0 0) 0x7ff8dc055380 con 0x7ff8e415d230 2023-10-10T20:41:54.025 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:54.025 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":138,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:41:51.309245+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:41:48.040814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-10T20:41:48.646159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:41:54.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.024+0000 7ff8eb47a700 1 -- 172.21.15.89:0/1127714597 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff8c8067320 msgr2=0x7ff8c80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.024+0000 7ff8eb47a700 1 --2- 172.21.15.89:0/1127714597 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff8c8067320 0x7ff8c80697d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0001730 tx=0x7ff8e0007b40 comp rx=0 tx=0).stop 2023-10-10T20:41:54.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.024+0000 7ff8eb47a700 1 -- 172.21.15.89:0/1127714597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8e415d230 msgr2=0x7ff8e40c6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.024+0000 7ff8eb47a700 1 --2- 172.21.15.89:0/1127714597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8e415d230 0x7ff8e40c6300 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00cc70 tx=0x7ff8dc001a00 comp rx=0 tx=0).stop 2023-10-10T20:41:54.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.024+0000 7ff8eb47a700 1 -- 172.21.15.89:0/1127714597 shutdown_connections 2023-10-10T20:41:54.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.024+0000 7ff8eb47a700 1 --2- 172.21.15.89:0/1127714597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8e415d230 0x7ff8e40c6300 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.024+0000 7ff8eb47a700 1 --2- 172.21.15.89:0/1127714597 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff8e4159420 0x7ff8e4154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.024+0000 7ff8eb47a700 1 --2- 172.21.15.89:0/1127714597 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff8c8067320 0x7ff8c80697d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.024+0000 7ff8eb47a700 1 --2- 172.21.15.89:0/1127714597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8e41597f0 0x7ff8e40c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.024+0000 7ff8eb47a700 1 -- 172.21.15.89:0/1127714597 >> 172.21.15.89:0/1127714597 conn(0x7ff8e40d0660 msgr2=0x7ff8e4155240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:54.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.025+0000 7ff8eb47a700 1 -- 172.21.15.89:0/1127714597 shutdown_connections 2023-10-10T20:41:54.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.025+0000 7ff8eb47a700 1 -- 172.21.15.89:0/1127714597 wait complete. 2023-10-10T20:41:54.039 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:41:54.039 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:41:54.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.152+0000 7f9ae5024700 1 Processor -- start 2023-10-10T20:41:54.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.152+0000 7f9ae5024700 1 -- start start 2023-10-10T20:41:54.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.153+0000 7f9ae5024700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9ae0143050 0x7f9ae0159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:54.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.153+0000 7f9ae5024700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ae006b9e0 0x7f9ae006be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:54.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.153+0000 7f9ae5024700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae006c390 con 0x7f9ae00b3810 2023-10-10T20:41:54.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.153+0000 7f9ae5024700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae0143420 con 0x7f9ae0143050 2023-10-10T20:41:54.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.153+0000 7f9ae5024700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae006a1e0 con 0x7f9ae006b9e0 2023-10-10T20:41:54.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.153+0000 7f9adffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9ae0143050 0x7f9ae0159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:54.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.153+0000 7f9ae4823700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9ae00b3810 0x7f9ae00b3c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55834/0 (socket says 172.21.15.89:55834) 2023-10-10T20:41:54.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.153+0000 7f9ae4823700 1 -- 172.21.15.89:0/3901579966 learned_addr learned my addr 172.21.15.89:0/3901579966 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:54.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.153+0000 7f9adf7fe700 1 --2- 172.21.15.89:0/3901579966 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ae006b9e0 0x7f9ae006be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:54.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.153+0000 7f9adffff700 1 -- 172.21.15.89:0/3901579966 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ae006b9e0 msgr2=0x7f9ae006be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:54.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.153+0000 7f9adffff700 1 --2- 172.21.15.89:0/3901579966 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ae006b9e0 0x7f9ae006be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.153+0000 7f9adffff700 1 -- 172.21.15.89:0/3901579966 >> v1:172.21.15.89:6789/0 conn(0x7f9ae00b3810 legacy=0x7f9ae00b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.153+0000 7f9adffff700 1 -- 172.21.15.89:0/3901579966 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae00695d0 con 0x7f9ae0143050 2023-10-10T20:41:54.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.153+0000 7f9adeffd700 1 -- 172.21.15.89:0/3901579966 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372054406 0 0) 0x7f9ae006c390 con 0x7f9ae00b3810 2023-10-10T20:41:54.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.154+0000 7f9adffff700 1 --2- 172.21.15.89:0/3901579966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae0143050 0x7f9ae0159c90 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00098d0 tx=0x7f9ad0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a9c93aef0ebb8c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:54.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.154+0000 7f9adeffd700 1 -- 172.21.15.89:0/3901579966 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0009e70 con 0x7f9ae0143050 2023-10-10T20:41:54.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.154+0000 7f9adeffd700 1 -- 172.21.15.89:0/3901579966 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad0005c10 con 0x7f9ae0143050 2023-10-10T20:41:54.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.154+0000 7f9adeffd700 1 -- 172.21.15.89:0/3901579966 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad000b650 con 0x7f9ae0143050 2023-10-10T20:41:54.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.154+0000 7f9ae5024700 1 -- 172.21.15.89:0/3901579966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae0143050 msgr2=0x7f9ae0159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.154+0000 7f9ae5024700 1 --2- 172.21.15.89:0/3901579966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae0143050 0x7f9ae0159c90 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00098d0 tx=0x7f9ad0009120 comp rx=0 tx=0).stop 2023-10-10T20:41:54.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.154+0000 7f9ae5024700 1 -- 172.21.15.89:0/3901579966 shutdown_connections 2023-10-10T20:41:54.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.154+0000 7f9ae5024700 1 --2- 172.21.15.89:0/3901579966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae0143050 0x7f9ae0159c90 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.154+0000 7f9ae5024700 1 --2- 172.21.15.89:0/3901579966 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ae006b9e0 0x7f9ae006be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.154+0000 7f9ae5024700 1 -- 172.21.15.89:0/3901579966 >> 172.21.15.89:0/3901579966 conn(0x7f9ae00ce430 msgr2=0x7f9ae00ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:54.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.154+0000 7f9ae5024700 1 -- 172.21.15.89:0/3901579966 shutdown_connections 2023-10-10T20:41:54.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.154+0000 7f9ae5024700 1 -- 172.21.15.89:0/3901579966 wait complete. 2023-10-10T20:41:54.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.155+0000 7f9ae5024700 1 Processor -- start 2023-10-10T20:41:54.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.156+0000 7f9ae5024700 1 -- start start 2023-10-10T20:41:54.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.156+0000 7f9ae5024700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae006b9e0 0x7f9ae016a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:54.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.156+0000 7f9ae5024700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ae00b3810 0x7f9ae0138520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:54.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.156+0000 7f9ae5024700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ae0143050 0x7f9ae016cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:54.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.156+0000 7f9ae5024700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae013a210 con 0x7f9ae006b9e0 2023-10-10T20:41:54.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.156+0000 7f9ae5024700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae0169930 con 0x7f9ae0143050 2023-10-10T20:41:54.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.156+0000 7f9ae5024700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae016df00 con 0x7f9ae00b3810 2023-10-10T20:41:54.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.156+0000 7f9adffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae006b9e0 0x7f9ae016a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:54.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.156+0000 7f9adffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae006b9e0 0x7f9ae016a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56570/0 (socket says 172.21.15.89:56570) 2023-10-10T20:41:54.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.156+0000 7f9adffff700 1 -- 172.21.15.89:0/4273233763 learned_addr learned my addr 172.21.15.89:0/4273233763 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:54.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.157+0000 7f9adffff700 1 -- 172.21.15.89:0/4273233763 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ae00b3810 msgr2=0x7f9ae0138520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.157+0000 7f9adffff700 1 --2- 172.21.15.89:0/4273233763 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ae00b3810 0x7f9ae0138520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.157+0000 7f9adffff700 1 -- 172.21.15.89:0/4273233763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ae0143050 msgr2=0x7f9ae016cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.157+0000 7f9adffff700 1 --2- 172.21.15.89:0/4273233763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ae0143050 0x7f9ae016cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.157+0000 7f9adffff700 1 -- 172.21.15.89:0/4273233763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad00093c0 con 0x7f9ae006b9e0 2023-10-10T20:41:54.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.157+0000 7f9adffff700 1 --2- 172.21.15.89:0/4273233763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae006b9e0 0x7f9ae016a340 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000b3c0 tx=0x7f9ad0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:54.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.157+0000 7f9ac7fff700 1 -- 172.21.15.89:0/4273233763 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0009e70 con 0x7f9ae006b9e0 2023-10-10T20:41:54.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.157+0000 7f9ac7fff700 1 -- 172.21.15.89:0/4273233763 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad001dcf0 con 0x7f9ae006b9e0 2023-10-10T20:41:54.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.157+0000 7f9ae5024700 1 -- 172.21.15.89:0/4273233763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae0169eb0 con 0x7f9ae006b9e0 2023-10-10T20:41:54.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.157+0000 7f9ac7fff700 1 -- 172.21.15.89:0/4273233763 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0026680 con 0x7f9ae006b9e0 2023-10-10T20:41:54.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.157+0000 7f9ae5024700 1 -- 172.21.15.89:0/4273233763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae0169b40 con 0x7f9ae006b9e0 2023-10-10T20:41:54.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.158+0000 7f9ae5024700 1 -- 172.21.15.89:0/4273233763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae016a190 con 0x7f9ae006b9e0 2023-10-10T20:41:54.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.158+0000 7f9ac7fff700 1 -- 172.21.15.89:0/4273233763 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9ad001a420 con 0x7f9ae006b9e0 2023-10-10T20:41:54.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.159+0000 7f9ac7fff700 1 --2- 172.21.15.89:0/4273233763 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9acc067320 0x7f9acc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:54.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.159+0000 7f9ac7fff700 1 -- 172.21.15.89:0/4273233763 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9ad0086590 con 0x7f9ae006b9e0 2023-10-10T20:41:54.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.159+0000 7f9adf7fe700 1 --2- 172.21.15.89:0/4273233763 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9acc067320 0x7f9acc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:54.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.159+0000 7f9adf7fe700 1 --2- 172.21.15.89:0/4273233763 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9acc067320 0x7f9acc0697d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00b0b00 tx=0x7f9ad4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:54.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.163+0000 7f9ac7fff700 1 -- 172.21.15.89:0/4273233763 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ad00532a0 con 0x7f9ae006b9e0 2023-10-10T20:41:54.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.345+0000 7f9ae5024700 1 -- 172.21.15.89:0/4273233763 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ae004c3c0 con 0x7f9acc067320 2023-10-10T20:41:54.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.352+0000 7f9ac7fff700 1 -- 172.21.15.89:0/4273233763 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275063 (secure 0 0 0) 0x7f9ae004c3c0 con 0x7f9acc067320 2023-10-10T20:41:54.358 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:54.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.355+0000 7f9ae5024700 1 -- 172.21.15.89:0/4273233763 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9acc067320 msgr2=0x7f9acc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.355+0000 7f9ae5024700 1 --2- 172.21.15.89:0/4273233763 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9acc067320 0x7f9acc0697d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00b0b00 tx=0x7f9ad4008040 comp rx=0 tx=0).stop 2023-10-10T20:41:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.355+0000 7f9ae5024700 1 -- 172.21.15.89:0/4273233763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae006b9e0 msgr2=0x7f9ae016a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.355+0000 7f9ae5024700 1 --2- 172.21.15.89:0/4273233763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae006b9e0 0x7f9ae016a340 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000b3c0 tx=0x7f9ad0016000 comp rx=0 tx=0).stop 2023-10-10T20:41:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.355+0000 7f9ae5024700 1 -- 172.21.15.89:0/4273233763 shutdown_connections 2023-10-10T20:41:54.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.355+0000 7f9ae5024700 1 --2- 172.21.15.89:0/4273233763 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9acc067320 0x7f9acc0697d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.355+0000 7f9ae5024700 1 --2- 172.21.15.89:0/4273233763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ae0143050 0x7f9ae016cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.355+0000 7f9ae5024700 1 --2- 172.21.15.89:0/4273233763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ae006b9e0 0x7f9ae016a340 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.355+0000 7f9ae5024700 1 --2- 172.21.15.89:0/4273233763 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9ae00b3810 0x7f9ae0138520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.355+0000 7f9ae5024700 1 -- 172.21.15.89:0/4273233763 >> 172.21.15.89:0/4273233763 conn(0x7f9ae00ce430 msgr2=0x7f9ae01404b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:54.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.355+0000 7f9ae5024700 1 -- 172.21.15.89:0/4273233763 shutdown_connections 2023-10-10T20:41:54.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.355+0000 7f9ae5024700 1 -- 172.21.15.89:0/4273233763 wait complete. 2023-10-10T20:41:54.364 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:41:54.369 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2023-10-10T20:41:52.738975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2425440,"kb_used_data":14368,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752549280,"statfs":{"total":773094113280,"available":770610462720,"internally_reserved":0,"allocated":14712832,"data_stored":7210794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.447275"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":277,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-10T20:41:45.287323+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:41:45.287323+0000","last_peered":"2023-10-10T20:41:45.287323+0000","last_clean":"2023-10-10T20:41:45.287323+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:41:45.287323+0000","last_undegraded":"2023-10-10T20:41:45.287323+0000","last_fullsized":"2023-10-10T20:41:45.287323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303620,"kb_used_data":2236,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068220,"statfs":{"total":96636764160,"available":96325857280,"internally_reserved":0,"allocated":2289664,"data_stored":1346216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.79000000000000004},"min":{"1min":0.309,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71799999999999997,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52400000000000002,"15min":0.61599999999999999},"min":{"1min":0.32300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.69999999999999996,"5min":0.82099999999999995,"15min":4.1959999999999997},"last":0.54100000000000004}]},{"osd":1,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.95099999999999996},"min":{"1min":0.379,"5min":0.317,"15min":0.317},"max":{"1min":0.78300000000000003,"5min":0.78800000000000003,"15min":10.1},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55200000000000005,"15min":0.71399999999999997},"min":{"1min":0.36699999999999999,"5min":0.32300000000000001,"15min":0.28999999999999998},"max":{"1min":0.73199999999999998,"5min":0.90400000000000003,"15min":6.806},"last":0.50800000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.76400000000000001},"min":{"1min":0.40100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79200000000000004,"5min":0.89000000000000001,"15min":6.9340000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.71699999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.73299999999999998,"5min":0.83899999999999997,"15min":6.9870000000000001},"last":0.52700000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54900000000000004,"15min":0.73599999999999999},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.753,"5min":0.80200000000000005,"15min":6.6719999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56699999999999995,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.71699999999999997,"5min":0.87,"15min":4.1609999999999996},"last":0.56699999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.36599999999999999,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.65100000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36299999999999999,"5min":0.309,"15min":0.30399999999999999},"max":{"1min":0.72299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.42799999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.314},"max":{"1min":0.77900000000000003,"5min":0.86099999999999999,"15min":1.3180000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.41799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71499999999999997,"5min":0.75800000000000001,"15min":0.84099999999999997},"last":0.44400000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":0.78700000000000003,"15min":1.7509999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.40899999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.85699999999999998,"15min":1.7090000000000001},"last":0.48299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":750593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51300000000000001,"15min":0.65500000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":7.7590000000000003},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.52100000000000002,"15min":0.69199999999999995},"min":{"1min":0.32200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.075,"5min":1.075,"15min":7.8090000000000002},"last":0.33700000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.69099999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.70199999999999996,"5min":0.80000000000000004,"15min":7.867},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.85599999999999998},"min":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.38400000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.58799999999999997},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.56100000000000005,"15min":0.81999999999999995},"min":{"1min":0.34200000000000003,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":2.641,"5min":2.641,"15min":6.7629999999999999},"last":0.51700000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54300000000000004,"15min":0.61099999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.107,"5min":1.107,"15min":4.8620000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.70099999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.718,"5min":1.718,"15min":7.9009999999999998},"last":0.53000000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.47299999999999998},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.498}]},{"osd":5,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52700000000000002,"15min":0.495},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.53400000000000003},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56000000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55800000000000005,"15min":0.52500000000000002},"min":{"1min":0.36099999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.372,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.60599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":750593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.312,"15min":0.45900000000000002},"min":{"1min":0.22800000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.437,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":1.194},"last":0.22500000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.24199999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51000000000000001,"5min":0.51400000000000001,"15min":1.2869999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.48999999999999999},"min":{"1min":0.27200000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.52800000000000002,"5min":0.54200000000000004,"15min":5.0919999999999996},"last":0.31900000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36399999999999999,"15min":0.51200000000000001},"min":{"1min":0.29399999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35699999999999998,"15min":0.40200000000000002},"min":{"1min":0.23799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":1.2170000000000001},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.73999999999999999,"5min":0.92400000000000004,"15min":1.254},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.68999999999999995},"min":{"1min":0.39300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68799999999999994,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.625}]},{"osd":5,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.36099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":0.97599999999999998,"15min":1.1799999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56899999999999995,"15min":0.60299999999999998},"min":{"1min":0.41799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71399999999999997,"5min":0.878,"15min":1.3109999999999999},"last":0.44700000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.81799999999999995},"min":{"1min":0.46000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78900000000000003,"5min":0.995,"15min":5.0659999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80700000000000005,"5min":0.89800000000000002,"15min":1.171},"last":0.56599999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57999999999999996,"15min":0.61299999999999999},"min":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77300000000000002,"5min":1.004,"15min":1.214},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.70999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78200000000000003,"5min":0.90500000000000003,"15min":4.0499999999999998},"last":0.52800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574939,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303616,"kb_used_data":2232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068224,"statfs":{"total":96636764160,"available":96325861376,"internally_reserved":0,"allocated":2285568,"data_stored":1345255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.316,"15min":0.50800000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":7.9249999999999998},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.48699999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":6.6299999999999999},"last":0.28599999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.33100000000000002,"15min":0.53100000000000003},"min":{"1min":0.23100000000000001,"5min":0.22,"15min":0.216},"max":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":8.2509999999999994},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.51100000000000001},"min":{"1min":0.253,"5min":0.222,"15min":0.222},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":6.7370000000000001},"last":0.253}]},{"osd":3,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.35399999999999998,"15min":0.53900000000000003},"min":{"1min":0.247,"5min":0.23100000000000001,"15min":0.21099999999999999},"max":{"1min":0.53800000000000003,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.59099999999999997},"min":{"1min":0.27700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":7.8929999999999998},"last":0.46999999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57099999999999995,"15min":0.70299999999999996},"min":{"1min":0.45500000000000002,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.82499999999999996,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57299999999999995,"15min":0.754},"min":{"1min":0.441,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.80900000000000005,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.76700000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56100000000000005,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71299999999999997,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.76100000000000001},"min":{"1min":0.48399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.751}]},{"osd":6,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61499999999999999,"15min":0.80000000000000004},"min":{"1min":0.52200000000000002,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.83299999999999996,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.62,"15min":0.749},"min":{"1min":0.47399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82599999999999996,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.63400000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.60099999999999998,"15min":0.73999999999999999},"min":{"1min":0.48399999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85399999999999998,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.76800000000000002},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.75700000000000001,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.73599999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":750593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.311,"15min":0.32500000000000001},"min":{"1min":0.214,"5min":0.189,"15min":0.189},"max":{"1min":0.52500000000000002,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.35099999999999998},"min":{"1min":0.219,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.746},"last":0.316}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.69099999999999995},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52100000000000002,"5min":0.63,"15min":11.967000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.34899999999999998,"15min":0.46999999999999997},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48899999999999999,"5min":0.53400000000000003,"15min":4.1100000000000003},"last":0.375}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34699999999999998,"15min":0.53700000000000003},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.50800000000000001,"5min":0.51600000000000001,"15min":6.3040000000000003},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.74299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59299999999999997,"5min":0.65900000000000003,"15min":11.935},"last":0.505}]},{"osd":4,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.55700000000000005,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.67200000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.38,"15min":0.378},"max":{"1min":0.83699999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54200000000000004,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.96099999999999997},"last":0.76100000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.71799999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.58599999999999997},"min":{"1min":0.435,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.052},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58099999999999996,"15min":0.76300000000000001},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":6.2649999999999997},"last":0.65300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303040,"kb_used_data":1656,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068800,"statfs":{"total":96636764160,"available":96326451200,"internally_reserved":0,"allocated":1695744,"data_stored":755848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.315,"15min":0.315},"min":{"1min":0.223,"5min":0.21199999999999999,"15min":0.21099999999999999},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.249},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32100000000000001},"min":{"1min":0.23999999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48799999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.371}]},{"osd":1,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.34599999999999997,"15min":0.33300000000000002},"min":{"1min":0.25700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.379},"min":{"1min":0.29199999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.72099999999999997},"last":0.35699999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.36199999999999999,"15min":0.35299999999999998},"min":{"1min":0.25800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57599999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.28999999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.50900000000000001,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.28799999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.53400000000000003},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77200000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.745,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.69699999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57299999999999995,"15min":0.55200000000000005},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.35899999999999999},"max":{"1min":0.82399999999999995,"5min":1.347,"15min":1.347},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.42199999999999999,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.83499999999999996,"5min":1.01,"15min":1.01},"last":0.59999999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.48399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86299999999999999,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.81200000000000006,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78700000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.628},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.83699999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65000000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574939,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303040,"kb_used_data":1656,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068800,"statfs":{"total":96636764160,"available":96326451200,"internally_reserved":0,"allocated":1695744,"data_stored":755848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.32400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66600000000000004,"5min":0.74199999999999999,"15min":0.747},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51700000000000002,"15min":0.74099999999999999},"min":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77600000000000002,"5min":0.78400000000000003,"15min":7.6719999999999997},"last":0.77600000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52000000000000002,"15min":0.748},"min":{"1min":0.32000000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.755,"15min":7.7160000000000002},"last":0.501},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66100000000000003,"5min":0.76900000000000002,"15min":0.86799999999999999},"last":0.56799999999999995}]},{"osd":2,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.73899999999999999,"5min":0.79800000000000004,"15min":7.7839999999999998},"last":0.73899999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.39200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.79900000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.49099999999999999},"min":{"1min":0.35199999999999998,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.88400000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.85199999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58399999999999996,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.91400000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.41399999999999998,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.90100000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303040,"kb_used_data":1656,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068800,"statfs":{"total":96636764160,"available":96326451200,"internally_reserved":0,"allocated":1695744,"data_stored":755848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.28599999999999998,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.79300000000000004},"last":0.629}]},{"osd":1,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52700000000000002,"15min":0.64100000000000001},"min":{"1min":0.33000000000000002,"5min":0.32200000000000001,"15min":0.316},"max":{"1min":0.70399999999999996,"5min":1.2330000000000001,"15min":5.2679999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51800000000000002,"15min":0.629},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.44700000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52800000000000002,"15min":0.64800000000000002},"min":{"1min":0.35999999999999999,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.76600000000000001,"5min":0.86199999999999999,"15min":5.2960000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72899999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.57099999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55300000000000005,"15min":0.67700000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.73899999999999999,"5min":0.93000000000000005,"15min":5.3330000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.69099999999999995},"min":{"1min":0.39300000000000002,"5min":0.36899999999999999,"15min":0.27700000000000002},"max":{"1min":0.76000000000000001,"5min":0.94399999999999995,"15min":5.2850000000000001},"last":0.66000000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47799999999999998,"15min":0.496},"min":{"1min":0.311,"5min":0.29599999999999999,"15min":0.28699999999999998},"max":{"1min":0.72499999999999998,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34300000000000003,"15min":0.32200000000000001},"max":{"1min":0.70799999999999996,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.77200000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54100000000000004,"15min":0.54700000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.80100000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.746},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82499999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.78400000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.378,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68899999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60199999999999998,"15min":0.60099999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83399999999999996,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.82899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:41:54.375 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:41:54.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.523+0000 7f7203799700 1 Processor -- start 2023-10-10T20:41:54.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.523+0000 7f7203799700 1 -- start start 2023-10-10T20:41:54.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.524+0000 7f7203799700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f71fc159c60 0x7f71fc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:54.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.524+0000 7f7203799700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71fc15a160 con 0x7f71fc15db10 2023-10-10T20:41:54.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.524+0000 7f7203799700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71fc0d13a0 con 0x7f71fc159890 2023-10-10T20:41:54.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.524+0000 7f7203799700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71fc15df50 con 0x7f71fc159c60 2023-10-10T20:41:54.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.524+0000 7f7201d36700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f71fc15db10 0x7f71fc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55838/0 (socket says 172.21.15.89:55838) 2023-10-10T20:41:54.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.524+0000 7f7201d36700 1 -- 172.21.15.89:0/1592179537 learned_addr learned my addr 172.21.15.89:0/1592179537 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:54.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.524+0000 7f71f3fff700 1 -- 172.21.15.89:0/1592179537 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1444903031 0 0) 0x7f71fc15a160 con 0x7f71fc15db10 2023-10-10T20:41:54.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f71f3fff700 1 -- 172.21.15.89:0/1592179537 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71e8003570 con 0x7f71fc15db10 2023-10-10T20:41:54.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.524+0000 7f7200d34700 1 --2- 172.21.15.89:0/1592179537 >> v2:172.21.15.111:3300/0 conn(0x7f71fc159c60 0x7f71fc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:54.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f71f3fff700 1 -- 172.21.15.89:0/1592179537 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 33883537 0 0) 0x7f71e8003570 con 0x7f71fc15db10 2023-10-10T20:41:54.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f71f3fff700 1 -- 172.21.15.89:0/1592179537 >> v2:172.21.15.111:3300/0 conn(0x7f71fc159c60 msgr2=0x7f71fc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:54.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f71f3fff700 1 --2- 172.21.15.89:0/1592179537 >> v2:172.21.15.111:3300/0 conn(0x7f71fc159c60 0x7f71fc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f71f3fff700 1 -- 172.21.15.89:0/1592179537 >> v1:172.21.15.111:6789/0 conn(0x7f71fc159890 legacy=0x7f71fc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f71f3fff700 1 -- 172.21.15.89:0/1592179537 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc15e140 con 0x7f71fc15db10 2023-10-10T20:41:54.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f71f3fff700 1 -- 172.21.15.89:0/1592179537 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f71f4001470 con 0x7f71fc15db10 2023-10-10T20:41:54.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f71f3fff700 1 -- 172.21.15.89:0/1592179537 >> v1:172.21.15.89:6789/0 conn(0x7f71fc15db10 legacy=0x7f71fc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f71f3fff700 1 --2- 172.21.15.89:0/1592179537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f71e8003a60 0x7f71e8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:54.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f71f3fff700 1 --2- 172.21.15.89:0/1592179537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71e8004810 0x7f71e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:54.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f71f3fff700 1 --2- 172.21.15.89:0/1592179537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71e80052e0 0x7f71e800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:54.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f71f3fff700 1 -- 172.21.15.89:0/1592179537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e800c160 con 0x7f71e8004810 2023-10-10T20:41:54.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f71f3fff700 1 -- 172.21.15.89:0/1592179537 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e800c2d0 con 0x7f71e80052e0 2023-10-10T20:41:54.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f71f3fff700 1 -- 172.21.15.89:0/1592179537 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e800c490 con 0x7f71e8003a60 2023-10-10T20:41:54.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f7200d34700 1 --2- 172.21.15.89:0/1592179537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71e8004810 0x7f71e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:54.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f7201535700 1 --2- 172.21.15.89:0/1592179537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f71e8003a60 0x7f71e8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:54.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f7201d36700 1 --2- 172.21.15.89:0/1592179537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71e80052e0 0x7f71e800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:54.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f7200d34700 1 -- 172.21.15.89:0/1592179537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f71e8003a60 msgr2=0x7f71e8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:54.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f7200d34700 1 --2- 172.21.15.89:0/1592179537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f71e8003a60 0x7f71e8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f7200d34700 1 -- 172.21.15.89:0/1592179537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71e80052e0 msgr2=0x7f71e800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:54.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f7200d34700 1 --2- 172.21.15.89:0/1592179537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71e80052e0 0x7f71e800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f7200d34700 1 -- 172.21.15.89:0/1592179537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e8003570 con 0x7f71e8004810 2023-10-10T20:41:54.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f7201d36700 1 --2- 172.21.15.89:0/1592179537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71e80052e0 0x7f71e800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:54.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f7201535700 1 --2- 172.21.15.89:0/1592179537 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f71e8003a60 0x7f71e8003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:54.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f7201d36700 1 -- 172.21.15.89:0/1592179537 reap_dead start 2023-10-10T20:41:54.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f7200d34700 1 --2- 172.21.15.89:0/1592179537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71e8004810 0x7f71e8004c80 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00bb90 tx=0x7f71ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d355ef4210495858 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:54.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f71f3fff700 1 -- 172.21.15.89:0/1592179537 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec00d5d0 con 0x7f71e8004810 2023-10-10T20:41:54.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f71f3fff700 1 -- 172.21.15.89:0/1592179537 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ec00d730 con 0x7f71e8004810 2023-10-10T20:41:54.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.525+0000 7f71f3fff700 1 -- 172.21.15.89:0/1592179537 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec00daf0 con 0x7f71e8004810 2023-10-10T20:41:54.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.526+0000 7f7203799700 1 -- 172.21.15.89:0/1592179537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71e8004810 msgr2=0x7f71e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.526+0000 7f7203799700 1 --2- 172.21.15.89:0/1592179537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71e8004810 0x7f71e8004c80 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00bb90 tx=0x7f71ec008040 comp rx=0 tx=0).stop 2023-10-10T20:41:54.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.526+0000 7f7203799700 1 -- 172.21.15.89:0/1592179537 shutdown_connections 2023-10-10T20:41:54.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.526+0000 7f7203799700 1 --2- 172.21.15.89:0/1592179537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71e8004810 0x7f71e8004c80 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.526+0000 7f7203799700 1 -- 172.21.15.89:0/1592179537 >> 172.21.15.89:0/1592179537 conn(0x7f71fc068e20 msgr2=0x7f71fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:54.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.526+0000 7f7203799700 1 -- 172.21.15.89:0/1592179537 shutdown_connections 2023-10-10T20:41:54.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.526+0000 7f7203799700 1 -- 172.21.15.89:0/1592179537 wait complete. 2023-10-10T20:41:54.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.527+0000 7f7203799700 1 Processor -- start 2023-10-10T20:41:54.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.527+0000 7f7203799700 1 -- start start 2023-10-10T20:41:54.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.528+0000 7f7203799700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f71e8004810 0x7f71fc0b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:54.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.528+0000 7f7203799700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71fc0b5e80 0x7f71fc151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:54.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.528+0000 7f7203799700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71fc15c510 0x7f71fc15c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:54.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.528+0000 7f7203799700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0b4860 con 0x7f71fc15c510 2023-10-10T20:41:54.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.528+0000 7f7203799700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc15ced0 con 0x7f71fc0b5e80 2023-10-10T20:41:54.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.528+0000 7f7203799700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0b4580 con 0x7f71e8004810 2023-10-10T20:41:54.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.528+0000 7f7201d36700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71fc15c510 0x7f71fc15c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:54.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.528+0000 7f7201d36700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71fc15c510 0x7f71fc15c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56582/0 (socket says 172.21.15.89:56582) 2023-10-10T20:41:54.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.528+0000 7f7201d36700 1 -- 172.21.15.89:0/1627763774 learned_addr learned my addr 172.21.15.89:0/1627763774 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:54.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.528+0000 7f7201535700 1 --2- 172.21.15.89:0/1627763774 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f71e8004810 0x7f71fc0b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:54.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.528+0000 7f7200d34700 1 --2- 172.21.15.89:0/1627763774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71fc0b5e80 0x7f71fc151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:54.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.528+0000 7f7201d36700 1 -- 172.21.15.89:0/1627763774 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f71e8004810 msgr2=0x7f71fc0b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.528+0000 7f7201d36700 1 --2- 172.21.15.89:0/1627763774 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f71e8004810 0x7f71fc0b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.528+0000 7f7201d36700 1 -- 172.21.15.89:0/1627763774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71fc0b5e80 msgr2=0x7f71fc151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.528+0000 7f7201d36700 1 --2- 172.21.15.89:0/1627763774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71fc0b5e80 0x7f71fc151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.528+0000 7f7201d36700 1 -- 172.21.15.89:0/1627763774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc0cd260 con 0x7f71fc15c510 2023-10-10T20:41:54.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.528+0000 7f7201535700 1 --2- 172.21.15.89:0/1627763774 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f71e8004810 0x7f71fc0b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:54.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.528+0000 7f7200d34700 1 --2- 172.21.15.89:0/1627763774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71fc0b5e80 0x7f71fc151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:54.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.529+0000 7f7201d36700 1 --2- 172.21.15.89:0/1627763774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71fc15c510 0x7f71fc15c990 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f71f4000c00 tx=0x7f71f4001d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:54.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.529+0000 7f71f17fa700 1 -- 172.21.15.89:0/1627763774 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f400eeb0 con 0x7f71fc15c510 2023-10-10T20:41:54.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.529+0000 7f71f17fa700 1 -- 172.21.15.89:0/1627763774 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f400c750 con 0x7f71fc15c510 2023-10-10T20:41:54.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.529+0000 7f7203799700 1 -- 172.21.15.89:0/1627763774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71fc0cd540 con 0x7f71fc15c510 2023-10-10T20:41:54.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.529+0000 7f71f17fa700 1 -- 172.21.15.89:0/1627763774 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f400cb10 con 0x7f71fc15c510 2023-10-10T20:41:54.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.529+0000 7f7203799700 1 -- 172.21.15.89:0/1627763774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71fc1519a0 con 0x7f71fc15c510 2023-10-10T20:41:54.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.530+0000 7f7203799700 1 -- 172.21.15.89:0/1627763774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71fc0cd0c0 con 0x7f71fc15c510 2023-10-10T20:41:54.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.530+0000 7f71f17fa700 1 -- 172.21.15.89:0/1627763774 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f71f40133e0 con 0x7f71fc15c510 2023-10-10T20:41:54.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.530+0000 7f71f17fa700 1 --2- 172.21.15.89:0/1627763774 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f71e0067320 0x7f71e00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:54.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.531+0000 7f71f17fa700 1 -- 172.21.15.89:0/1627763774 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f71f40844b0 con 0x7f71fc15c510 2023-10-10T20:41:54.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.531+0000 7f7201535700 1 --2- 172.21.15.89:0/1627763774 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f71e0067320 0x7f71e00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:54.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.531+0000 7f7201535700 1 --2- 172.21.15.89:0/1627763774 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f71e0067320 0x7f71e00697d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f71f8000f80 tx=0x7f71f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:54.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.535+0000 7f71f17fa700 1 -- 172.21.15.89:0/1627763774 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71fc0cd0c0 con 0x7f71fc15c510 2023-10-10T20:41:54.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.714+0000 7f7203799700 1 -- 172.21.15.89:0/1627763774 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f71fc0cd0c0 con 0x7f71e0067320 2023-10-10T20:41:54.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.722+0000 7f71f17fa700 1 -- 172.21.15.89:0/1627763774 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275063 (secure 0 0 0) 0x7f71fc0cd0c0 con 0x7f71e0067320 2023-10-10T20:41:54.727 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:54.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.725+0000 7f7203799700 1 -- 172.21.15.89:0/1627763774 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f71e0067320 msgr2=0x7f71e00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.725+0000 7f7203799700 1 --2- 172.21.15.89:0/1627763774 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f71e0067320 0x7f71e00697d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f71f8000f80 tx=0x7f71f8009040 comp rx=0 tx=0).stop 2023-10-10T20:41:54.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.725+0000 7f7203799700 1 -- 172.21.15.89:0/1627763774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71fc15c510 msgr2=0x7f71fc15c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.725+0000 7f7203799700 1 --2- 172.21.15.89:0/1627763774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71fc15c510 0x7f71fc15c990 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f71f4000c00 tx=0x7f71f4001d90 comp rx=0 tx=0).stop 2023-10-10T20:41:54.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.725+0000 7f7203799700 1 -- 172.21.15.89:0/1627763774 shutdown_connections 2023-10-10T20:41:54.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.725+0000 7f7203799700 1 --2- 172.21.15.89:0/1627763774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71fc15c510 0x7f71fc15c990 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.725+0000 7f7203799700 1 --2- 172.21.15.89:0/1627763774 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f71e8004810 0x7f71fc0b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.725+0000 7f7203799700 1 --2- 172.21.15.89:0/1627763774 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f71e0067320 0x7f71e00697d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.725+0000 7f7203799700 1 --2- 172.21.15.89:0/1627763774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71fc0b5e80 0x7f71fc151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.725+0000 7f7203799700 1 -- 172.21.15.89:0/1627763774 >> 172.21.15.89:0/1627763774 conn(0x7f71fc068e20 msgr2=0x7f71fc1545f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:54.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.725+0000 7f7203799700 1 -- 172.21.15.89:0/1627763774 shutdown_connections 2023-10-10T20:41:54.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.725+0000 7f7203799700 1 -- 172.21.15.89:0/1627763774 wait complete. 2023-10-10T20:41:54.731 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:41:54.739 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2023-10-10T20:41:52.738975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2425440,"kb_used_data":14368,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752549280,"statfs":{"total":773094113280,"available":770610462720,"internally_reserved":0,"allocated":14712832,"data_stored":7210794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.447275"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:49.291991+0000","last_change":"2023-10-10T20:41:49.291991+0000","last_active":"2023-10-10T20:41:49.291991+0000","last_peered":"2023-10-10T20:41:49.291991+0000","last_clean":"2023-10-10T20:41:49.291991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:49.291991+0000","last_undegraded":"2023-10-10T20:41:49.291991+0000","last_fullsized":"2023-10-10T20:41:49.291991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:41:48.286224+0000","last_change":"2023-10-10T20:41:48.286224+0000","last_active":"2023-10-10T20:41:48.286224+0000","last_peered":"2023-10-10T20:41:48.286224+0000","last_clean":"2023-10-10T20:41:48.286224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:41:48.286224+0000","last_undegraded":"2023-10-10T20:41:48.286224+0000","last_fullsized":"2023-10-10T20:41:48.286224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":277,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-10T20:41:45.287323+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:41:45.287323+0000","last_peered":"2023-10-10T20:41:45.287323+0000","last_clean":"2023-10-10T20:41:45.287323+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:41:45.287323+0000","last_undegraded":"2023-10-10T20:41:45.287323+0000","last_fullsized":"2023-10-10T20:41:45.287323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303620,"kb_used_data":2236,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068220,"statfs":{"total":96636764160,"available":96325857280,"internally_reserved":0,"allocated":2289664,"data_stored":1346216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.79000000000000004},"min":{"1min":0.309,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71799999999999997,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52400000000000002,"15min":0.61599999999999999},"min":{"1min":0.32300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.69999999999999996,"5min":0.82099999999999995,"15min":4.1959999999999997},"last":0.54100000000000004}]},{"osd":1,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.95099999999999996},"min":{"1min":0.379,"5min":0.317,"15min":0.317},"max":{"1min":0.78300000000000003,"5min":0.78800000000000003,"15min":10.1},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55200000000000005,"15min":0.71399999999999997},"min":{"1min":0.36699999999999999,"5min":0.32300000000000001,"15min":0.28999999999999998},"max":{"1min":0.73199999999999998,"5min":0.90400000000000003,"15min":6.806},"last":0.50800000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.76400000000000001},"min":{"1min":0.40100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79200000000000004,"5min":0.89000000000000001,"15min":6.9340000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.71699999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.73299999999999998,"5min":0.83899999999999997,"15min":6.9870000000000001},"last":0.52700000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54900000000000004,"15min":0.73599999999999999},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.753,"5min":0.80200000000000005,"15min":6.6719999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56699999999999995,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.71699999999999997,"5min":0.87,"15min":4.1609999999999996},"last":0.56699999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.36599999999999999,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.65100000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36299999999999999,"5min":0.309,"15min":0.30399999999999999},"max":{"1min":0.72299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.42799999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.314},"max":{"1min":0.77900000000000003,"5min":0.86099999999999999,"15min":1.3180000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.41799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71499999999999997,"5min":0.75800000000000001,"15min":0.84099999999999997},"last":0.44400000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":0.78700000000000003,"15min":1.7509999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.40899999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.85699999999999998,"15min":1.7090000000000001},"last":0.48299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":750593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51300000000000001,"15min":0.65500000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":7.7590000000000003},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.52100000000000002,"15min":0.69199999999999995},"min":{"1min":0.32200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.075,"5min":1.075,"15min":7.8090000000000002},"last":0.33700000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.69099999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.70199999999999996,"5min":0.80000000000000004,"15min":7.867},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.85599999999999998},"min":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.38400000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.58799999999999997},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.56100000000000005,"15min":0.81999999999999995},"min":{"1min":0.34200000000000003,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":2.641,"5min":2.641,"15min":6.7629999999999999},"last":0.51700000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54300000000000004,"15min":0.61099999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.107,"5min":1.107,"15min":4.8620000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.70099999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.718,"5min":1.718,"15min":7.9009999999999998},"last":0.53000000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.47299999999999998},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.498}]},{"osd":5,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52700000000000002,"15min":0.495},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.53400000000000003},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56000000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55800000000000005,"15min":0.52500000000000002},"min":{"1min":0.36099999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.372,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.60599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":750593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.312,"15min":0.45900000000000002},"min":{"1min":0.22800000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.437,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":1.194},"last":0.22500000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.24199999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51000000000000001,"5min":0.51400000000000001,"15min":1.2869999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.48999999999999999},"min":{"1min":0.27200000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.52800000000000002,"5min":0.54200000000000004,"15min":5.0919999999999996},"last":0.31900000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36399999999999999,"15min":0.51200000000000001},"min":{"1min":0.29399999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35699999999999998,"15min":0.40200000000000002},"min":{"1min":0.23799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":1.2170000000000001},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.73999999999999999,"5min":0.92400000000000004,"15min":1.254},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.68999999999999995},"min":{"1min":0.39300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68799999999999994,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.625}]},{"osd":5,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.36099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":0.97599999999999998,"15min":1.1799999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56899999999999995,"15min":0.60299999999999998},"min":{"1min":0.41799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71399999999999997,"5min":0.878,"15min":1.3109999999999999},"last":0.44700000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.81799999999999995},"min":{"1min":0.46000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78900000000000003,"5min":0.995,"15min":5.0659999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80700000000000005,"5min":0.89800000000000002,"15min":1.171},"last":0.56599999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57999999999999996,"15min":0.61299999999999999},"min":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77300000000000002,"5min":1.004,"15min":1.214},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.70999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78200000000000003,"5min":0.90500000000000003,"15min":4.0499999999999998},"last":0.52800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574939,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303616,"kb_used_data":2232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068224,"statfs":{"total":96636764160,"available":96325861376,"internally_reserved":0,"allocated":2285568,"data_stored":1345255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.316,"15min":0.50800000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":7.9249999999999998},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.48699999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":6.6299999999999999},"last":0.28599999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.33100000000000002,"15min":0.53100000000000003},"min":{"1min":0.23100000000000001,"5min":0.22,"15min":0.216},"max":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":8.2509999999999994},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.51100000000000001},"min":{"1min":0.253,"5min":0.222,"15min":0.222},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":6.7370000000000001},"last":0.253}]},{"osd":3,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.35399999999999998,"15min":0.53900000000000003},"min":{"1min":0.247,"5min":0.23100000000000001,"15min":0.21099999999999999},"max":{"1min":0.53800000000000003,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.59099999999999997},"min":{"1min":0.27700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":7.8929999999999998},"last":0.46999999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57099999999999995,"15min":0.70299999999999996},"min":{"1min":0.45500000000000002,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.82499999999999996,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57299999999999995,"15min":0.754},"min":{"1min":0.441,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.80900000000000005,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.76700000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56100000000000005,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71299999999999997,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.76100000000000001},"min":{"1min":0.48399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.751}]},{"osd":6,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61499999999999999,"15min":0.80000000000000004},"min":{"1min":0.52200000000000002,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.83299999999999996,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.62,"15min":0.749},"min":{"1min":0.47399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82599999999999996,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.63400000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.60099999999999998,"15min":0.73999999999999999},"min":{"1min":0.48399999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85399999999999998,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.76800000000000002},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.75700000000000001,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.73599999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":750593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.311,"15min":0.32500000000000001},"min":{"1min":0.214,"5min":0.189,"15min":0.189},"max":{"1min":0.52500000000000002,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.35099999999999998},"min":{"1min":0.219,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.746},"last":0.316}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.69099999999999995},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52100000000000002,"5min":0.63,"15min":11.967000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.34899999999999998,"15min":0.46999999999999997},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48899999999999999,"5min":0.53400000000000003,"15min":4.1100000000000003},"last":0.375}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34699999999999998,"15min":0.53700000000000003},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.50800000000000001,"5min":0.51600000000000001,"15min":6.3040000000000003},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.74299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59299999999999997,"5min":0.65900000000000003,"15min":11.935},"last":0.505}]},{"osd":4,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.55700000000000005,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.67200000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.38,"15min":0.378},"max":{"1min":0.83699999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54200000000000004,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.96099999999999997},"last":0.76100000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.71799999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.58599999999999997},"min":{"1min":0.435,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.052},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58099999999999996,"15min":0.76300000000000001},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":6.2649999999999997},"last":0.65300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303040,"kb_used_data":1656,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068800,"statfs":{"total":96636764160,"available":96326451200,"internally_reserved":0,"allocated":1695744,"data_stored":755848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.315,"15min":0.315},"min":{"1min":0.223,"5min":0.21199999999999999,"15min":0.21099999999999999},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.249},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32100000000000001},"min":{"1min":0.23999999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48799999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.371}]},{"osd":1,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.34599999999999997,"15min":0.33300000000000002},"min":{"1min":0.25700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.379},"min":{"1min":0.29199999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.72099999999999997},"last":0.35699999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.36199999999999999,"15min":0.35299999999999998},"min":{"1min":0.25800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57599999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.28999999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.50900000000000001,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.28799999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.53400000000000003},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77200000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.745,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.69699999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57299999999999995,"15min":0.55200000000000005},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.35899999999999999},"max":{"1min":0.82399999999999995,"5min":1.347,"15min":1.347},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.42199999999999999,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.83499999999999996,"5min":1.01,"15min":1.01},"last":0.59999999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.48399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86299999999999999,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.81200000000000006,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78700000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.628},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.83699999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65000000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574939,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303040,"kb_used_data":1656,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068800,"statfs":{"total":96636764160,"available":96326451200,"internally_reserved":0,"allocated":1695744,"data_stored":755848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.32400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66600000000000004,"5min":0.74199999999999999,"15min":0.747},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51700000000000002,"15min":0.74099999999999999},"min":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77600000000000002,"5min":0.78400000000000003,"15min":7.6719999999999997},"last":0.77600000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52000000000000002,"15min":0.748},"min":{"1min":0.32000000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.755,"15min":7.7160000000000002},"last":0.501},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66100000000000003,"5min":0.76900000000000002,"15min":0.86799999999999999},"last":0.56799999999999995}]},{"osd":2,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.73899999999999999,"5min":0.79800000000000004,"15min":7.7839999999999998},"last":0.73899999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.39200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.79900000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.49099999999999999},"min":{"1min":0.35199999999999998,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.88400000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.85199999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58399999999999996,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.91400000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.41399999999999998,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.90100000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303040,"kb_used_data":1656,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068800,"statfs":{"total":96636764160,"available":96326451200,"internally_reserved":0,"allocated":1695744,"data_stored":755848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.28599999999999998,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.79300000000000004},"last":0.629}]},{"osd":1,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52700000000000002,"15min":0.64100000000000001},"min":{"1min":0.33000000000000002,"5min":0.32200000000000001,"15min":0.316},"max":{"1min":0.70399999999999996,"5min":1.2330000000000001,"15min":5.2679999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51800000000000002,"15min":0.629},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.44700000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52800000000000002,"15min":0.64800000000000002},"min":{"1min":0.35999999999999999,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.76600000000000001,"5min":0.86199999999999999,"15min":5.2960000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72899999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.57099999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55300000000000005,"15min":0.67700000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.73899999999999999,"5min":0.93000000000000005,"15min":5.3330000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.69099999999999995},"min":{"1min":0.39300000000000002,"5min":0.36899999999999999,"15min":0.27700000000000002},"max":{"1min":0.76000000000000001,"5min":0.94399999999999995,"15min":5.2850000000000001},"last":0.66000000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47799999999999998,"15min":0.496},"min":{"1min":0.311,"5min":0.29599999999999999,"15min":0.28699999999999998},"max":{"1min":0.72499999999999998,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34300000000000003,"15min":0.32200000000000001},"max":{"1min":0.70799999999999996,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.77200000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54100000000000004,"15min":0.54700000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.80100000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.746},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82499999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.78400000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.378,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68899999999999995,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60199999999999998,"15min":0.60099999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83399999999999996,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.82899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:41:54.745 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:41:54.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.893+0000 7f37f1a37700 1 Processor -- start 2023-10-10T20:41:54.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.893+0000 7f37f1a37700 1 -- start start 2023-10-10T20:41:54.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.893+0000 7f37f1a37700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37ec069560 0x7f37ec0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:54.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.893+0000 7f37f1a37700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ec069ee0 con 0x7f37ec159c60 2023-10-10T20:41:54.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.893+0000 7f37f1a37700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ec15a160 con 0x7f37ec159890 2023-10-10T20:41:54.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.893+0000 7f37f1a37700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ec0d13a0 con 0x7f37ec069560 2023-10-10T20:41:54.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.894+0000 7f37eb7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37ec069560 0x7f37ec0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:54.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.894+0000 7f37e27fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f37ec159c60 0x7f37ec15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55846/0 (socket says 172.21.15.89:55846) 2023-10-10T20:41:54.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.894+0000 7f37e27fc700 1 -- 172.21.15.89:0/1310626973 learned_addr learned my addr 172.21.15.89:0/1310626973 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:54.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.894+0000 7f37eb7fe700 1 -- 172.21.15.89:0/1310626973 >> v1:172.21.15.111:6789/0 conn(0x7f37ec159890 legacy=0x7f37ec15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.894+0000 7f37eb7fe700 1 -- 172.21.15.89:0/1310626973 >> v1:172.21.15.89:6789/0 conn(0x7f37ec159c60 legacy=0x7f37ec15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.894+0000 7f37ea7fc700 1 -- 172.21.15.89:0/1310626973 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568521450 0 0) 0x7f37ec069ee0 con 0x7f37ec159c60 2023-10-10T20:41:54.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.894+0000 7f37eb7fe700 1 -- 172.21.15.89:0/1310626973 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec15e140 con 0x7f37ec069560 2023-10-10T20:41:54.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.894+0000 7f37eb7fe700 1 --2- 172.21.15.89:0/1310626973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ec069560 0x7f37ec0699a0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009e60 tx=0x7f37e4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d16b67667f1be05 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:54.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.894+0000 7f37ea7fc700 1 -- 172.21.15.89:0/1310626973 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e400e070 con 0x7f37ec069560 2023-10-10T20:41:54.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.894+0000 7f37ea7fc700 1 -- 172.21.15.89:0/1310626973 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e400bc30 con 0x7f37ec069560 2023-10-10T20:41:54.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.894+0000 7f37ea7fc700 1 -- 172.21.15.89:0/1310626973 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e4012430 con 0x7f37ec069560 2023-10-10T20:41:54.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.895+0000 7f37f1a37700 1 -- 172.21.15.89:0/1310626973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ec069560 msgr2=0x7f37ec0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.895+0000 7f37f1a37700 1 --2- 172.21.15.89:0/1310626973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ec069560 0x7f37ec0699a0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009e60 tx=0x7f37e4009780 comp rx=0 tx=0).stop 2023-10-10T20:41:54.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.895+0000 7f37f1a37700 1 -- 172.21.15.89:0/1310626973 shutdown_connections 2023-10-10T20:41:54.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.895+0000 7f37f1a37700 1 --2- 172.21.15.89:0/1310626973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ec069560 0x7f37ec0699a0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.895+0000 7f37f1a37700 1 -- 172.21.15.89:0/1310626973 >> 172.21.15.89:0/1310626973 conn(0x7f37ec068e20 msgr2=0x7f37ec0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:54.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.895+0000 7f37f1a37700 1 -- 172.21.15.89:0/1310626973 shutdown_connections 2023-10-10T20:41:54.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.895+0000 7f37f1a37700 1 -- 172.21.15.89:0/1310626973 wait complete. 2023-10-10T20:41:54.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.896+0000 7f37f1a37700 1 Processor -- start 2023-10-10T20:41:54.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.896+0000 7f37f1a37700 1 -- start start 2023-10-10T20:41:54.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.897+0000 7f37f1a37700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ec069560 0x7f37ec1542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:54.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.897+0000 7f37f1a37700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37ec159890 0x7f37ec0c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:54.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.897+0000 7f37f1a37700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37ec159c60 0x7f37ec0b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:54.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.897+0000 7f37f1a37700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec0b3800 con 0x7f37ec069560 2023-10-10T20:41:54.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.897+0000 7f37f1a37700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec0cd5c0 con 0x7f37ec159890 2023-10-10T20:41:54.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.897+0000 7f37f1a37700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec0cd8a0 con 0x7f37ec159c60 2023-10-10T20:41:54.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.897+0000 7f37eaffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ec069560 0x7f37ec1542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:54.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.897+0000 7f37eaffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ec069560 0x7f37ec1542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56606/0 (socket says 172.21.15.89:56606) 2023-10-10T20:41:54.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.897+0000 7f37eaffd700 1 -- 172.21.15.89:0/1503142928 learned_addr learned my addr 172.21.15.89:0/1503142928 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:54.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.897+0000 7f37e27fc700 1 --2- 172.21.15.89:0/1503142928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37ec159890 0x7f37ec0c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:54.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.897+0000 7f37eb7fe700 1 --2- 172.21.15.89:0/1503142928 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37ec159c60 0x7f37ec0b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:54.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.898+0000 7f37eaffd700 1 -- 172.21.15.89:0/1503142928 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37ec159c60 msgr2=0x7f37ec0b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.898+0000 7f37eaffd700 1 --2- 172.21.15.89:0/1503142928 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37ec159c60 0x7f37ec0b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.898+0000 7f37eaffd700 1 -- 172.21.15.89:0/1503142928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37ec159890 msgr2=0x7f37ec0c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:54.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.898+0000 7f37eaffd700 1 --2- 172.21.15.89:0/1503142928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37ec159890 0x7f37ec0c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:54.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.898+0000 7f37eaffd700 1 -- 172.21.15.89:0/1503142928 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e4009a20 con 0x7f37ec069560 2023-10-10T20:41:54.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.898+0000 7f37eb7fe700 1 --2- 172.21.15.89:0/1503142928 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37ec159c60 0x7f37ec0b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:54.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.898+0000 7f37e27fc700 1 --2- 172.21.15.89:0/1503142928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37ec159890 0x7f37ec0c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:54.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.898+0000 7f37eaffd700 1 --2- 172.21.15.89:0/1503142928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ec069560 0x7f37ec1542d0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00c590 tx=0x7f37dc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:54.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.898+0000 7f37e37fe700 1 -- 172.21.15.89:0/1503142928 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc011590 con 0x7f37ec069560 2023-10-10T20:41:54.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.898+0000 7f37e37fe700 1 -- 172.21.15.89:0/1503142928 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37dc0116f0 con 0x7f37ec069560 2023-10-10T20:41:54.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.898+0000 7f37f1a37700 1 -- 172.21.15.89:0/1503142928 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ec0cdb80 con 0x7f37ec069560 2023-10-10T20:41:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.898+0000 7f37e37fe700 1 -- 172.21.15.89:0/1503142928 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc011ab0 con 0x7f37ec069560 2023-10-10T20:41:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.898+0000 7f37f1a37700 1 -- 172.21.15.89:0/1503142928 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ec154810 con 0x7f37ec069560 2023-10-10T20:41:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.899+0000 7f37f1a37700 1 -- 172.21.15.89:0/1503142928 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ec04c3c0 con 0x7f37ec069560 2023-10-10T20:41:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.899+0000 7f37e37fe700 1 -- 172.21.15.89:0/1503142928 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f37dc01b3b0 con 0x7f37ec069560 2023-10-10T20:41:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.900+0000 7f37e37fe700 1 --2- 172.21.15.89:0/1503142928 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f37cc067320 0x7f37cc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.900+0000 7f37e37fe700 1 -- 172.21.15.89:0/1503142928 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f37dc084220 con 0x7f37ec069560 2023-10-10T20:41:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.900+0000 7f37e27fc700 1 --2- 172.21.15.89:0/1503142928 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f37cc067320 0x7f37cc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.900+0000 7f37e27fc700 1 --2- 172.21.15.89:0/1503142928 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f37cc067320 0x7f37cc0697d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0b2a70 tx=0x7f37d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:54.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:54.904+0000 7f37e37fe700 1 -- 172.21.15.89:0/1503142928 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37dc050eb0 con 0x7f37ec069560 2023-10-10T20:41:55.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.084+0000 7f37f1a37700 1 -- 172.21.15.89:0/1503142928 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f37ec0b3940 con 0x7f37cc067320 2023-10-10T20:41:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.092+0000 7f37e37fe700 1 -- 172.21.15.89:0/1503142928 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290156 (secure 0 0 0) 0x7f37ec0b3940 con 0x7f37cc067320 2023-10-10T20:41:55.097 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.095+0000 7f37f1a37700 1 -- 172.21.15.89:0/1503142928 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f37cc067320 msgr2=0x7f37cc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.095+0000 7f37f1a37700 1 --2- 172.21.15.89:0/1503142928 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f37cc067320 0x7f37cc0697d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0b2a70 tx=0x7f37d4009040 comp rx=0 tx=0).stop 2023-10-10T20:41:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.095+0000 7f37f1a37700 1 -- 172.21.15.89:0/1503142928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ec069560 msgr2=0x7f37ec1542d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.095+0000 7f37f1a37700 1 --2- 172.21.15.89:0/1503142928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ec069560 0x7f37ec1542d0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00c590 tx=0x7f37dc00c950 comp rx=0 tx=0).stop 2023-10-10T20:41:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.095+0000 7f37f1a37700 1 -- 172.21.15.89:0/1503142928 shutdown_connections 2023-10-10T20:41:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.095+0000 7f37f1a37700 1 --2- 172.21.15.89:0/1503142928 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37ec159c60 0x7f37ec0b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.095+0000 7f37f1a37700 1 --2- 172.21.15.89:0/1503142928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ec069560 0x7f37ec1542d0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.095+0000 7f37f1a37700 1 --2- 172.21.15.89:0/1503142928 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f37cc067320 0x7f37cc0697d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.095+0000 7f37f1a37700 1 --2- 172.21.15.89:0/1503142928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37ec159890 0x7f37ec0c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.095+0000 7f37f1a37700 1 -- 172.21.15.89:0/1503142928 >> 172.21.15.89:0/1503142928 conn(0x7f37ec068e20 msgr2=0x7f37ec15aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:55.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.095+0000 7f37f1a37700 1 -- 172.21.15.89:0/1503142928 shutdown_connections 2023-10-10T20:41:55.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.096+0000 7f37f1a37700 1 -- 172.21.15.89:0/1503142928 wait complete. 2023-10-10T20:41:55.102 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:41:55.109 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2023-10-10T20:41:54.739427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425924,"kb_used_data":14852,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752548796,"statfs":{"total":773094113280,"available":770609967104,"internally_reserved":0,"allocated":15208448,"data_stored":7432123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001128"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328850+0000","last_change":"2023-10-10T20:41:50.324854+0000","last_active":"2023-10-10T20:41:50.328850+0000","last_peered":"2023-10-10T20:41:50.328850+0000","last_clean":"2023-10-10T20:41:50.328850+0000","last_became_active":"2023-10-10T20:41:50.324573+0000","last_became_peered":"2023-10-10T20:41:50.324573+0000","last_unstale":"2023-10-10T20:41:50.328850+0000","last_undegraded":"2023-10-10T20:41:50.328850+0000","last_fullsized":"2023-10-10T20:41:50.328850+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:07:32.681938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329631+0000","last_change":"2023-10-10T20:41:50.328756+0000","last_active":"2023-10-10T20:41:50.329631+0000","last_peered":"2023-10-10T20:41:50.329631+0000","last_clean":"2023-10-10T20:41:50.329631+0000","last_became_active":"2023-10-10T20:41:50.328488+0000","last_became_peered":"2023-10-10T20:41:50.328488+0000","last_unstale":"2023-10-10T20:41:50.329631+0000","last_undegraded":"2023-10-10T20:41:50.329631+0000","last_fullsized":"2023-10-10T20:41:50.329631+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:32:34.541780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.322750+0000","last_change":"2023-10-10T20:41:50.317128+0000","last_active":"2023-10-10T20:41:50.322750+0000","last_peered":"2023-10-10T20:41:50.322750+0000","last_clean":"2023-10-10T20:41:50.322750+0000","last_became_active":"2023-10-10T20:41:50.316862+0000","last_became_peered":"2023-10-10T20:41:50.316862+0000","last_unstale":"2023-10-10T20:41:50.322750+0000","last_undegraded":"2023-10-10T20:41:50.322750+0000","last_fullsized":"2023-10-10T20:41:50.322750+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:59.167375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329233+0000","last_change":"2023-10-10T20:41:50.325779+0000","last_active":"2023-10-10T20:41:50.329233+0000","last_peered":"2023-10-10T20:41:50.329233+0000","last_clean":"2023-10-10T20:41:50.329233+0000","last_became_active":"2023-10-10T20:41:50.325527+0000","last_became_peered":"2023-10-10T20:41:50.325527+0000","last_unstale":"2023-10-10T20:41:50.329233+0000","last_undegraded":"2023-10-10T20:41:50.329233+0000","last_fullsized":"2023-10-10T20:41:50.329233+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:09:42.528347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329492+0000","last_change":"2023-10-10T20:41:50.322671+0000","last_active":"2023-10-10T20:41:50.329492+0000","last_peered":"2023-10-10T20:41:50.329492+0000","last_clean":"2023-10-10T20:41:50.329492+0000","last_became_active":"2023-10-10T20:41:50.322277+0000","last_became_peered":"2023-10-10T20:41:50.322277+0000","last_unstale":"2023-10-10T20:41:50.329492+0000","last_undegraded":"2023-10-10T20:41:50.329492+0000","last_fullsized":"2023-10-10T20:41:50.329492+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:03.021663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.320162+0000","last_change":"2023-10-10T20:41:50.314512+0000","last_active":"2023-10-10T20:41:50.320162+0000","last_peered":"2023-10-10T20:41:50.320162+0000","last_clean":"2023-10-10T20:41:50.320162+0000","last_became_active":"2023-10-10T20:41:50.314257+0000","last_became_peered":"2023-10-10T20:41:50.314257+0000","last_unstale":"2023-10-10T20:41:50.320162+0000","last_undegraded":"2023-10-10T20:41:50.320162+0000","last_fullsized":"2023-10-10T20:41:50.320162+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:52:08.397497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329082+0000","last_change":"2023-10-10T20:41:50.322740+0000","last_active":"2023-10-10T20:41:50.329082+0000","last_peered":"2023-10-10T20:41:50.329082+0000","last_clean":"2023-10-10T20:41:50.329082+0000","last_became_active":"2023-10-10T20:41:50.322415+0000","last_became_peered":"2023-10-10T20:41:50.322415+0000","last_unstale":"2023-10-10T20:41:50.329082+0000","last_undegraded":"2023-10-10T20:41:50.329082+0000","last_fullsized":"2023-10-10T20:41:50.329082+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:59.167375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327463+0000","last_change":"2023-10-10T20:41:50.320942+0000","last_active":"2023-10-10T20:41:50.327463+0000","last_peered":"2023-10-10T20:41:50.327463+0000","last_clean":"2023-10-10T20:41:50.327463+0000","last_became_active":"2023-10-10T20:41:50.320312+0000","last_became_peered":"2023-10-10T20:41:50.320312+0000","last_unstale":"2023-10-10T20:41:50.327463+0000","last_undegraded":"2023-10-10T20:41:50.327463+0000","last_fullsized":"2023-10-10T20:41:50.327463+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:08:58.453242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.324745+0000","last_change":"2023-10-10T20:41:50.318424+0000","last_active":"2023-10-10T20:41:50.324745+0000","last_peered":"2023-10-10T20:41:50.324745+0000","last_clean":"2023-10-10T20:41:50.324745+0000","last_became_active":"2023-10-10T20:41:50.318177+0000","last_became_peered":"2023-10-10T20:41:50.318177+0000","last_unstale":"2023-10-10T20:41:50.324745+0000","last_undegraded":"2023-10-10T20:41:50.324745+0000","last_fullsized":"2023-10-10T20:41:50.324745+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:25.820041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.325823+0000","last_change":"2023-10-10T20:41:50.320526+0000","last_active":"2023-10-10T20:41:50.325823+0000","last_peered":"2023-10-10T20:41:50.325823+0000","last_clean":"2023-10-10T20:41:50.325823+0000","last_became_active":"2023-10-10T20:41:50.320284+0000","last_became_peered":"2023-10-10T20:41:50.320284+0000","last_unstale":"2023-10-10T20:41:50.325823+0000","last_undegraded":"2023-10-10T20:41:50.325823+0000","last_fullsized":"2023-10-10T20:41:50.325823+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:56.229435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.316595+0000","last_change":"2023-10-10T20:41:50.312426+0000","last_active":"2023-10-10T20:41:50.316595+0000","last_peered":"2023-10-10T20:41:50.316595+0000","last_clean":"2023-10-10T20:41:50.316595+0000","last_became_active":"2023-10-10T20:41:50.312160+0000","last_became_peered":"2023-10-10T20:41:50.312160+0000","last_unstale":"2023-10-10T20:41:50.316595+0000","last_undegraded":"2023-10-10T20:41:50.316595+0000","last_fullsized":"2023-10-10T20:41:50.316595+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:49:46.940449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.324082+0000","last_change":"2023-10-10T20:41:50.319250+0000","last_active":"2023-10-10T20:41:50.324082+0000","last_peered":"2023-10-10T20:41:50.324082+0000","last_clean":"2023-10-10T20:41:50.324082+0000","last_became_active":"2023-10-10T20:41:50.318945+0000","last_became_peered":"2023-10-10T20:41:50.318945+0000","last_unstale":"2023-10-10T20:41:50.324082+0000","last_undegraded":"2023-10-10T20:41:50.324082+0000","last_fullsized":"2023-10-10T20:41:50.324082+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:29.171361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.324092+0000","last_change":"2023-10-10T20:41:50.320071+0000","last_active":"2023-10-10T20:41:50.324092+0000","last_peered":"2023-10-10T20:41:50.324092+0000","last_clean":"2023-10-10T20:41:50.324092+0000","last_became_active":"2023-10-10T20:41:50.319800+0000","last_became_peered":"2023-10-10T20:41:50.319800+0000","last_unstale":"2023-10-10T20:41:50.324092+0000","last_undegraded":"2023-10-10T20:41:50.324092+0000","last_fullsized":"2023-10-10T20:41:50.324092+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:59.707661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.324909+0000","last_change":"2023-10-10T20:41:50.320391+0000","last_active":"2023-10-10T20:41:50.324909+0000","last_peered":"2023-10-10T20:41:50.324909+0000","last_clean":"2023-10-10T20:41:50.324909+0000","last_became_active":"2023-10-10T20:41:50.320087+0000","last_became_peered":"2023-10-10T20:41:50.320087+0000","last_unstale":"2023-10-10T20:41:50.324909+0000","last_undegraded":"2023-10-10T20:41:50.324909+0000","last_fullsized":"2023-10-10T20:41:50.324909+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:56:15.566395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.323944+0000","last_change":"2023-10-10T20:41:50.319419+0000","last_active":"2023-10-10T20:41:50.323944+0000","last_peered":"2023-10-10T20:41:50.323944+0000","last_clean":"2023-10-10T20:41:50.323944+0000","last_became_active":"2023-10-10T20:41:50.318904+0000","last_became_peered":"2023-10-10T20:41:50.318904+0000","last_unstale":"2023-10-10T20:41:50.323944+0000","last_undegraded":"2023-10-10T20:41:50.323944+0000","last_fullsized":"2023-10-10T20:41:50.323944+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:59.167375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327445+0000","last_change":"2023-10-10T20:41:50.314811+0000","last_active":"2023-10-10T20:41:50.327445+0000","last_peered":"2023-10-10T20:41:50.327445+0000","last_clean":"2023-10-10T20:41:50.327445+0000","last_became_active":"2023-10-10T20:41:50.313773+0000","last_became_peered":"2023-10-10T20:41:50.313773+0000","last_unstale":"2023-10-10T20:41:50.327445+0000","last_undegraded":"2023-10-10T20:41:50.327445+0000","last_fullsized":"2023-10-10T20:41:50.327445+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:06.177167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329995+0000","last_change":"2023-10-10T20:41:50.326631+0000","last_active":"2023-10-10T20:41:50.329995+0000","last_peered":"2023-10-10T20:41:50.329995+0000","last_clean":"2023-10-10T20:41:50.329995+0000","last_became_active":"2023-10-10T20:41:50.323639+0000","last_became_peered":"2023-10-10T20:41:50.323639+0000","last_unstale":"2023-10-10T20:41:50.329995+0000","last_undegraded":"2023-10-10T20:41:50.329995+0000","last_fullsized":"2023-10-10T20:41:50.329995+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:37:21.016202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.315357+0000","last_change":"2023-10-10T20:41:50.311398+0000","last_active":"2023-10-10T20:41:50.315357+0000","last_peered":"2023-10-10T20:41:50.315357+0000","last_clean":"2023-10-10T20:41:50.315357+0000","last_became_active":"2023-10-10T20:41:50.311136+0000","last_became_peered":"2023-10-10T20:41:50.311136+0000","last_unstale":"2023-10-10T20:41:50.315357+0000","last_undegraded":"2023-10-10T20:41:50.315357+0000","last_fullsized":"2023-10-10T20:41:50.315357+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:51:24.010245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327174+0000","last_change":"2023-10-10T20:41:50.322755+0000","last_active":"2023-10-10T20:41:50.327174+0000","last_peered":"2023-10-10T20:41:50.327174+0000","last_clean":"2023-10-10T20:41:50.327174+0000","last_became_active":"2023-10-10T20:41:50.322514+0000","last_became_peered":"2023-10-10T20:41:50.322514+0000","last_unstale":"2023-10-10T20:41:50.327174+0000","last_undegraded":"2023-10-10T20:41:50.327174+0000","last_fullsized":"2023-10-10T20:41:50.327174+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:06.177167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329970+0000","last_change":"2023-10-10T20:41:50.326356+0000","last_active":"2023-10-10T20:41:50.329970+0000","last_peered":"2023-10-10T20:41:50.329970+0000","last_clean":"2023-10-10T20:41:50.329970+0000","last_became_active":"2023-10-10T20:41:50.326042+0000","last_became_peered":"2023-10-10T20:41:50.326042+0000","last_unstale":"2023-10-10T20:41:50.329970+0000","last_undegraded":"2023-10-10T20:41:50.329970+0000","last_fullsized":"2023-10-10T20:41:50.329970+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:36:22.310224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329474+0000","last_change":"2023-10-10T20:41:50.322516+0000","last_active":"2023-10-10T20:41:50.329474+0000","last_peered":"2023-10-10T20:41:50.329474+0000","last_clean":"2023-10-10T20:41:50.329474+0000","last_became_active":"2023-10-10T20:41:50.321967+0000","last_became_peered":"2023-10-10T20:41:50.321967+0000","last_unstale":"2023-10-10T20:41:50.329474+0000","last_undegraded":"2023-10-10T20:41:50.329474+0000","last_fullsized":"2023-10-10T20:41:50.329474+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:58.433900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.334853+0000","last_change":"2023-10-10T20:41:50.332271+0000","last_active":"2023-10-10T20:41:50.334853+0000","last_peered":"2023-10-10T20:41:50.334853+0000","last_clean":"2023-10-10T20:41:50.334853+0000","last_became_active":"2023-10-10T20:41:50.331819+0000","last_became_peered":"2023-10-10T20:41:50.331819+0000","last_unstale":"2023-10-10T20:41:50.334853+0000","last_undegraded":"2023-10-10T20:41:50.334853+0000","last_fullsized":"2023-10-10T20:41:50.334853+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:08:58.453242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328951+0000","last_change":"2023-10-10T20:41:50.325040+0000","last_active":"2023-10-10T20:41:50.328951+0000","last_peered":"2023-10-10T20:41:50.328951+0000","last_clean":"2023-10-10T20:41:50.328951+0000","last_became_active":"2023-10-10T20:41:50.324726+0000","last_became_peered":"2023-10-10T20:41:50.324726+0000","last_unstale":"2023-10-10T20:41:50.328951+0000","last_undegraded":"2023-10-10T20:41:50.328951+0000","last_fullsized":"2023-10-10T20:41:50.328951+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:29.171361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.330924+0000","last_change":"2023-10-10T20:41:50.321301+0000","last_active":"2023-10-10T20:41:50.330924+0000","last_peered":"2023-10-10T20:41:50.330924+0000","last_clean":"2023-10-10T20:41:50.330924+0000","last_became_active":"2023-10-10T20:41:50.320658+0000","last_became_peered":"2023-10-10T20:41:50.320658+0000","last_unstale":"2023-10-10T20:41:50.330924+0000","last_undegraded":"2023-10-10T20:41:50.330924+0000","last_fullsized":"2023-10-10T20:41:50.330924+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:36:51.689597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.325951+0000","last_change":"2023-10-10T20:41:50.322737+0000","last_active":"2023-10-10T20:41:50.325951+0000","last_peered":"2023-10-10T20:41:50.325951+0000","last_clean":"2023-10-10T20:41:50.325951+0000","last_became_active":"2023-10-10T20:41:50.322509+0000","last_became_peered":"2023-10-10T20:41:50.322509+0000","last_unstale":"2023-10-10T20:41:50.325951+0000","last_undegraded":"2023-10-10T20:41:50.325951+0000","last_fullsized":"2023-10-10T20:41:50.325951+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:49:46.940449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328624+0000","last_change":"2023-10-10T20:41:50.321194+0000","last_active":"2023-10-10T20:41:50.328624+0000","last_peered":"2023-10-10T20:41:50.328624+0000","last_clean":"2023-10-10T20:41:50.328624+0000","last_became_active":"2023-10-10T20:41:50.320546+0000","last_became_peered":"2023-10-10T20:41:50.320546+0000","last_unstale":"2023-10-10T20:41:50.328624+0000","last_undegraded":"2023-10-10T20:41:50.328624+0000","last_fullsized":"2023-10-10T20:41:50.328624+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:29.171361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.319678+0000","last_change":"2023-10-10T20:41:50.314660+0000","last_active":"2023-10-10T20:41:50.319678+0000","last_peered":"2023-10-10T20:41:50.319678+0000","last_clean":"2023-10-10T20:41:50.319678+0000","last_became_active":"2023-10-10T20:41:50.314414+0000","last_became_peered":"2023-10-10T20:41:50.314414+0000","last_unstale":"2023-10-10T20:41:50.319678+0000","last_undegraded":"2023-10-10T20:41:50.319678+0000","last_fullsized":"2023-10-10T20:41:50.319678+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:09:42.528347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329592+0000","last_change":"2023-10-10T20:41:50.325931+0000","last_active":"2023-10-10T20:41:50.329592+0000","last_peered":"2023-10-10T20:41:50.329592+0000","last_clean":"2023-10-10T20:41:50.329592+0000","last_became_active":"2023-10-10T20:41:50.325457+0000","last_became_peered":"2023-10-10T20:41:50.325457+0000","last_unstale":"2023-10-10T20:41:50.329592+0000","last_undegraded":"2023-10-10T20:41:50.329592+0000","last_fullsized":"2023-10-10T20:41:50.329592+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:57.360052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.324074+0000","last_change":"2023-10-10T20:41:50.319188+0000","last_active":"2023-10-10T20:41:50.324074+0000","last_peered":"2023-10-10T20:41:50.324074+0000","last_clean":"2023-10-10T20:41:50.324074+0000","last_became_active":"2023-10-10T20:41:50.318887+0000","last_became_peered":"2023-10-10T20:41:50.318887+0000","last_unstale":"2023-10-10T20:41:50.324074+0000","last_undegraded":"2023-10-10T20:41:50.324074+0000","last_fullsized":"2023-10-10T20:41:50.324074+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:38.872388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.315523+0000","last_change":"2023-10-10T20:41:50.310296+0000","last_active":"2023-10-10T20:41:50.315523+0000","last_peered":"2023-10-10T20:41:50.315523+0000","last_clean":"2023-10-10T20:41:50.315523+0000","last_became_active":"2023-10-10T20:41:50.309872+0000","last_became_peered":"2023-10-10T20:41:50.309872+0000","last_unstale":"2023-10-10T20:41:50.315523+0000","last_undegraded":"2023-10-10T20:41:50.315523+0000","last_fullsized":"2023-10-10T20:41:50.315523+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:08:57.143352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.325071+0000","last_change":"2023-10-10T20:41:50.320021+0000","last_active":"2023-10-10T20:41:50.325071+0000","last_peered":"2023-10-10T20:41:50.325071+0000","last_clean":"2023-10-10T20:41:50.325071+0000","last_became_active":"2023-10-10T20:41:50.319667+0000","last_became_peered":"2023-10-10T20:41:50.319667+0000","last_unstale":"2023-10-10T20:41:50.325071+0000","last_undegraded":"2023-10-10T20:41:50.325071+0000","last_fullsized":"2023-10-10T20:41:50.325071+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:09:58.004059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329518+0000","last_change":"2023-10-10T20:41:50.320084+0000","last_active":"2023-10-10T20:41:50.329518+0000","last_peered":"2023-10-10T20:41:50.329518+0000","last_clean":"2023-10-10T20:41:50.329518+0000","last_became_active":"2023-10-10T20:41:50.319816+0000","last_became_peered":"2023-10-10T20:41:50.319816+0000","last_unstale":"2023-10-10T20:41:50.329518+0000","last_undegraded":"2023-10-10T20:41:50.329518+0000","last_fullsized":"2023-10-10T20:41:50.329518+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:25.711425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327656+0000","last_change":"2023-10-10T20:41:50.318101+0000","last_active":"2023-10-10T20:41:50.327656+0000","last_peered":"2023-10-10T20:41:50.327656+0000","last_clean":"2023-10-10T20:41:50.327656+0000","last_became_active":"2023-10-10T20:41:50.317813+0000","last_became_peered":"2023-10-10T20:41:50.317813+0000","last_unstale":"2023-10-10T20:41:50.327656+0000","last_undegraded":"2023-10-10T20:41:50.327656+0000","last_fullsized":"2023-10-10T20:41:50.327656+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:45:42.233949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"137'4","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.326399+0000","last_change":"2023-10-10T20:41:49.316818+0000","last_active":"2023-10-10T20:41:50.326399+0000","last_peered":"2023-10-10T20:41:50.326399+0000","last_clean":"2023-10-10T20:41:50.326399+0000","last_became_active":"2023-10-10T20:41:49.315998+0000","last_became_peered":"2023-10-10T20:41:49.315998+0000","last_unstale":"2023-10-10T20:41:50.326399+0000","last_undegraded":"2023-10-10T20:41:50.326399+0000","last_fullsized":"2023-10-10T20:41:50.326399+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:23:47.101110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329947+0000","last_change":"2023-10-10T20:41:50.328491+0000","last_active":"2023-10-10T20:41:50.329947+0000","last_peered":"2023-10-10T20:41:50.329947+0000","last_clean":"2023-10-10T20:41:50.329947+0000","last_became_active":"2023-10-10T20:41:50.328230+0000","last_became_peered":"2023-10-10T20:41:50.328230+0000","last_unstale":"2023-10-10T20:41:50.329947+0000","last_undegraded":"2023-10-10T20:41:50.329947+0000","last_fullsized":"2023-10-10T20:41:50.329947+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:45:42.233949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"137'1","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.314284+0000","last_change":"2023-10-10T20:41:49.310393+0000","last_active":"2023-10-10T20:41:50.314284+0000","last_peered":"2023-10-10T20:41:50.314284+0000","last_clean":"2023-10-10T20:41:50.314284+0000","last_became_active":"2023-10-10T20:41:49.309899+0000","last_became_peered":"2023-10-10T20:41:49.309899+0000","last_unstale":"2023-10-10T20:41:50.314284+0000","last_undegraded":"2023-10-10T20:41:50.314284+0000","last_fullsized":"2023-10-10T20:41:50.314284+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:56.354284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329152+0000","last_change":"2023-10-10T20:41:50.323854+0000","last_active":"2023-10-10T20:41:50.329152+0000","last_peered":"2023-10-10T20:41:50.329152+0000","last_clean":"2023-10-10T20:41:50.329152+0000","last_became_active":"2023-10-10T20:41:50.323559+0000","last_became_peered":"2023-10-10T20:41:50.323559+0000","last_unstale":"2023-10-10T20:41:50.329152+0000","last_undegraded":"2023-10-10T20:41:50.329152+0000","last_fullsized":"2023-10-10T20:41:50.329152+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:59.707661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"137'2","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.325205+0000","last_change":"2023-10-10T20:41:49.316507+0000","last_active":"2023-10-10T20:41:50.325205+0000","last_peered":"2023-10-10T20:41:50.325205+0000","last_clean":"2023-10-10T20:41:50.325205+0000","last_became_active":"2023-10-10T20:41:49.316246+0000","last_became_peered":"2023-10-10T20:41:49.316246+0000","last_unstale":"2023-10-10T20:41:50.325205+0000","last_undegraded":"2023-10-10T20:41:50.325205+0000","last_fullsized":"2023-10-10T20:41:50.325205+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:59.824875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.321151+0000","last_change":"2023-10-10T20:41:50.316576+0000","last_active":"2023-10-10T20:41:50.321151+0000","last_peered":"2023-10-10T20:41:50.321151+0000","last_clean":"2023-10-10T20:41:50.321151+0000","last_became_active":"2023-10-10T20:41:50.316279+0000","last_became_peered":"2023-10-10T20:41:50.316279+0000","last_unstale":"2023-10-10T20:41:50.321151+0000","last_undegraded":"2023-10-10T20:41:50.321151+0000","last_fullsized":"2023-10-10T20:41:50.321151+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:33:19.443146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"137'2","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.319554+0000","last_change":"2023-10-10T20:41:49.314721+0000","last_active":"2023-10-10T20:41:50.319554+0000","last_peered":"2023-10-10T20:41:50.319554+0000","last_clean":"2023-10-10T20:41:50.319554+0000","last_became_active":"2023-10-10T20:41:49.312771+0000","last_became_peered":"2023-10-10T20:41:49.312771+0000","last_unstale":"2023-10-10T20:41:50.319554+0000","last_undegraded":"2023-10-10T20:41:50.319554+0000","last_fullsized":"2023-10-10T20:41:50.319554+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:56:14.560628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.334834+0000","last_change":"2023-10-10T20:41:50.332094+0000","last_active":"2023-10-10T20:41:50.334834+0000","last_peered":"2023-10-10T20:41:50.334834+0000","last_clean":"2023-10-10T20:41:50.334834+0000","last_became_active":"2023-10-10T20:41:50.331523+0000","last_became_peered":"2023-10-10T20:41:50.331523+0000","last_unstale":"2023-10-10T20:41:50.334834+0000","last_undegraded":"2023-10-10T20:41:50.334834+0000","last_fullsized":"2023-10-10T20:41:50.334834+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:36:55.761497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"137'1","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327568+0000","last_change":"2023-10-10T20:41:49.315617+0000","last_active":"2023-10-10T20:41:50.327568+0000","last_peered":"2023-10-10T20:41:50.327568+0000","last_clean":"2023-10-10T20:41:50.327568+0000","last_became_active":"2023-10-10T20:41:49.315233+0000","last_became_peered":"2023-10-10T20:41:49.315233+0000","last_unstale":"2023-10-10T20:41:50.327568+0000","last_undegraded":"2023-10-10T20:41:50.327568+0000","last_fullsized":"2023-10-10T20:41:50.327568+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:41:47.663214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328784+0000","last_change":"2023-10-10T20:41:50.322989+0000","last_active":"2023-10-10T20:41:50.328784+0000","last_peered":"2023-10-10T20:41:50.328784+0000","last_clean":"2023-10-10T20:41:50.328784+0000","last_became_active":"2023-10-10T20:41:50.320266+0000","last_became_peered":"2023-10-10T20:41:50.320266+0000","last_unstale":"2023-10-10T20:41:50.328784+0000","last_undegraded":"2023-10-10T20:41:50.328784+0000","last_fullsized":"2023-10-10T20:41:50.328784+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:22:41.479243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"137'1","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.321165+0000","last_change":"2023-10-10T20:41:49.308749+0000","last_active":"2023-10-10T20:41:50.321165+0000","last_peered":"2023-10-10T20:41:50.321165+0000","last_clean":"2023-10-10T20:41:50.321165+0000","last_became_active":"2023-10-10T20:41:49.307933+0000","last_became_peered":"2023-10-10T20:41:49.307933+0000","last_unstale":"2023-10-10T20:41:50.321165+0000","last_undegraded":"2023-10-10T20:41:50.321165+0000","last_fullsized":"2023-10-10T20:41:50.321165+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:30:49.584095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.316607+0000","last_change":"2023-10-10T20:41:50.311319+0000","last_active":"2023-10-10T20:41:50.316607+0000","last_peered":"2023-10-10T20:41:50.316607+0000","last_clean":"2023-10-10T20:41:50.316607+0000","last_became_active":"2023-10-10T20:41:50.310519+0000","last_became_peered":"2023-10-10T20:41:50.310519+0000","last_unstale":"2023-10-10T20:41:50.316607+0000","last_undegraded":"2023-10-10T20:41:50.316607+0000","last_fullsized":"2023-10-10T20:41:50.316607+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:25.142717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.301856+0000","last_change":"2023-10-10T20:41:49.315721+0000","last_active":"2023-10-10T20:41:50.301856+0000","last_peered":"2023-10-10T20:41:50.301856+0000","last_clean":"2023-10-10T20:41:50.301856+0000","last_became_active":"2023-10-10T20:41:49.315438+0000","last_became_peered":"2023-10-10T20:41:49.315438+0000","last_unstale":"2023-10-10T20:41:50.301856+0000","last_undegraded":"2023-10-10T20:41:50.301856+0000","last_fullsized":"2023-10-10T20:41:50.301856+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:17:06.508904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.332154+0000","last_change":"2023-10-10T20:41:50.329023+0000","last_active":"2023-10-10T20:41:50.332154+0000","last_peered":"2023-10-10T20:41:50.332154+0000","last_clean":"2023-10-10T20:41:50.332154+0000","last_became_active":"2023-10-10T20:41:50.328773+0000","last_became_peered":"2023-10-10T20:41:50.328773+0000","last_unstale":"2023-10-10T20:41:50.332154+0000","last_undegraded":"2023-10-10T20:41:50.332154+0000","last_fullsized":"2023-10-10T20:41:50.332154+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:37:14.358493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.302232+0000","last_change":"2023-10-10T20:41:49.310044+0000","last_active":"2023-10-10T20:41:50.302232+0000","last_peered":"2023-10-10T20:41:50.302232+0000","last_clean":"2023-10-10T20:41:50.302232+0000","last_became_active":"2023-10-10T20:41:49.309759+0000","last_became_peered":"2023-10-10T20:41:49.309759+0000","last_unstale":"2023-10-10T20:41:50.302232+0000","last_undegraded":"2023-10-10T20:41:50.302232+0000","last_fullsized":"2023-10-10T20:41:50.302232+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:30:13.642642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328695+0000","last_change":"2023-10-10T20:41:50.323785+0000","last_active":"2023-10-10T20:41:50.328695+0000","last_peered":"2023-10-10T20:41:50.328695+0000","last_clean":"2023-10-10T20:41:50.328695+0000","last_became_active":"2023-10-10T20:41:50.323571+0000","last_became_peered":"2023-10-10T20:41:50.323571+0000","last_unstale":"2023-10-10T20:41:50.328695+0000","last_undegraded":"2023-10-10T20:41:50.328695+0000","last_fullsized":"2023-10-10T20:41:50.328695+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:58:22.113298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.310448+0000","last_change":"2023-10-10T20:41:49.317302+0000","last_active":"2023-10-10T20:41:50.310448+0000","last_peered":"2023-10-10T20:41:50.310448+0000","last_clean":"2023-10-10T20:41:50.310448+0000","last_became_active":"2023-10-10T20:41:49.317066+0000","last_became_peered":"2023-10-10T20:41:49.317066+0000","last_unstale":"2023-10-10T20:41:50.310448+0000","last_undegraded":"2023-10-10T20:41:50.310448+0000","last_fullsized":"2023-10-10T20:41:50.310448+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:41:24.307438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.317448+0000","last_change":"2023-10-10T20:41:50.311080+0000","last_active":"2023-10-10T20:41:50.317448+0000","last_peered":"2023-10-10T20:41:50.317448+0000","last_clean":"2023-10-10T20:41:50.317448+0000","last_became_active":"2023-10-10T20:41:50.310822+0000","last_became_peered":"2023-10-10T20:41:50.310822+0000","last_unstale":"2023-10-10T20:41:50.317448+0000","last_undegraded":"2023-10-10T20:41:50.317448+0000","last_fullsized":"2023-10-10T20:41:50.317448+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:09.931645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"137'3","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328903+0000","last_change":"2023-10-10T20:41:49.315940+0000","last_active":"2023-10-10T20:41:50.328903+0000","last_peered":"2023-10-10T20:41:50.328903+0000","last_clean":"2023-10-10T20:41:50.328903+0000","last_became_active":"2023-10-10T20:41:49.315620+0000","last_became_peered":"2023-10-10T20:41:49.315620+0000","last_unstale":"2023-10-10T20:41:50.328903+0000","last_undegraded":"2023-10-10T20:41:50.328903+0000","last_fullsized":"2023-10-10T20:41:50.328903+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:29:06.709268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.332378+0000","last_change":"2023-10-10T20:41:50.331754+0000","last_active":"2023-10-10T20:41:50.332378+0000","last_peered":"2023-10-10T20:41:50.332378+0000","last_clean":"2023-10-10T20:41:50.332378+0000","last_became_active":"2023-10-10T20:41:50.331492+0000","last_became_peered":"2023-10-10T20:41:50.331492+0000","last_unstale":"2023-10-10T20:41:50.332378+0000","last_undegraded":"2023-10-10T20:41:50.332378+0000","last_fullsized":"2023-10-10T20:41:50.332378+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:58:22.113298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"137'6","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329762+0000","last_change":"2023-10-10T20:41:49.307503+0000","last_active":"2023-10-10T20:41:50.329762+0000","last_peered":"2023-10-10T20:41:50.329762+0000","last_clean":"2023-10-10T20:41:50.329762+0000","last_became_active":"2023-10-10T20:41:49.307188+0000","last_became_peered":"2023-10-10T20:41:49.307188+0000","last_unstale":"2023-10-10T20:41:50.329762+0000","last_undegraded":"2023-10-10T20:41:50.329762+0000","last_fullsized":"2023-10-10T20:41:50.329762+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:22.143245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.325048+0000","last_change":"2023-10-10T20:41:50.320264+0000","last_active":"2023-10-10T20:41:50.325048+0000","last_peered":"2023-10-10T20:41:50.325048+0000","last_clean":"2023-10-10T20:41:50.325048+0000","last_became_active":"2023-10-10T20:41:50.320001+0000","last_became_peered":"2023-10-10T20:41:50.320001+0000","last_unstale":"2023-10-10T20:41:50.325048+0000","last_undegraded":"2023-10-10T20:41:50.325048+0000","last_fullsized":"2023-10-10T20:41:50.325048+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:59.707661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"137'2","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327672+0000","last_change":"2023-10-10T20:41:49.314907+0000","last_active":"2023-10-10T20:41:50.327672+0000","last_peered":"2023-10-10T20:41:50.327672+0000","last_clean":"2023-10-10T20:41:50.327672+0000","last_became_active":"2023-10-10T20:41:49.314607+0000","last_became_peered":"2023-10-10T20:41:49.314607+0000","last_unstale":"2023-10-10T20:41:50.327672+0000","last_undegraded":"2023-10-10T20:41:50.327672+0000","last_fullsized":"2023-10-10T20:41:50.327672+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:04:21.204106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.325850+0000","last_change":"2023-10-10T20:41:50.320973+0000","last_active":"2023-10-10T20:41:50.325850+0000","last_peered":"2023-10-10T20:41:50.325850+0000","last_clean":"2023-10-10T20:41:50.325850+0000","last_became_active":"2023-10-10T20:41:50.320708+0000","last_became_peered":"2023-10-10T20:41:50.320708+0000","last_unstale":"2023-10-10T20:41:50.325850+0000","last_undegraded":"2023-10-10T20:41:50.325850+0000","last_fullsized":"2023-10-10T20:41:50.325850+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:09:42.528347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.312504+0000","last_change":"2023-10-10T20:41:49.313210+0000","last_active":"2023-10-10T20:41:50.312504+0000","last_peered":"2023-10-10T20:41:50.312504+0000","last_clean":"2023-10-10T20:41:50.312504+0000","last_became_active":"2023-10-10T20:41:49.312963+0000","last_became_peered":"2023-10-10T20:41:49.312963+0000","last_unstale":"2023-10-10T20:41:50.312504+0000","last_undegraded":"2023-10-10T20:41:50.312504+0000","last_fullsized":"2023-10-10T20:41:50.312504+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:52.358088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.326056+0000","last_change":"2023-10-10T20:41:50.322767+0000","last_active":"2023-10-10T20:41:50.326056+0000","last_peered":"2023-10-10T20:41:50.326056+0000","last_clean":"2023-10-10T20:41:50.326056+0000","last_became_active":"2023-10-10T20:41:50.322502+0000","last_became_peered":"2023-10-10T20:41:50.322502+0000","last_unstale":"2023-10-10T20:41:50.326056+0000","last_undegraded":"2023-10-10T20:41:50.326056+0000","last_fullsized":"2023-10-10T20:41:50.326056+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:15:18.258032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.306854+0000","last_change":"2023-10-10T20:41:49.317998+0000","last_active":"2023-10-10T20:41:50.306854+0000","last_peered":"2023-10-10T20:41:50.306854+0000","last_clean":"2023-10-10T20:41:50.306854+0000","last_became_active":"2023-10-10T20:41:49.317703+0000","last_became_peered":"2023-10-10T20:41:49.317703+0000","last_unstale":"2023-10-10T20:41:50.306854+0000","last_undegraded":"2023-10-10T20:41:50.306854+0000","last_fullsized":"2023-10-10T20:41:50.306854+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:39:49.147457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.320938+0000","last_change":"2023-10-10T20:41:50.317758+0000","last_active":"2023-10-10T20:41:50.320938+0000","last_peered":"2023-10-10T20:41:50.320938+0000","last_clean":"2023-10-10T20:41:50.320938+0000","last_became_active":"2023-10-10T20:41:50.317500+0000","last_became_peered":"2023-10-10T20:41:50.317500+0000","last_unstale":"2023-10-10T20:41:50.320938+0000","last_undegraded":"2023-10-10T20:41:50.320938+0000","last_fullsized":"2023-10-10T20:41:50.320938+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:22:41.479243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"137'8","reported_seq":18,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328528+0000","last_change":"2023-10-10T20:41:49.316006+0000","last_active":"2023-10-10T20:41:50.328528+0000","last_peered":"2023-10-10T20:41:50.328528+0000","last_clean":"2023-10-10T20:41:50.328528+0000","last_became_active":"2023-10-10T20:41:49.315687+0000","last_became_peered":"2023-10-10T20:41:49.315687+0000","last_unstale":"2023-10-10T20:41:50.328528+0000","last_undegraded":"2023-10-10T20:41:50.328528+0000","last_fullsized":"2023-10-10T20:41:50.328528+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:09:56.998292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327185+0000","last_change":"2023-10-10T20:41:50.322343+0000","last_active":"2023-10-10T20:41:50.327185+0000","last_peered":"2023-10-10T20:41:50.327185+0000","last_clean":"2023-10-10T20:41:50.327185+0000","last_became_active":"2023-10-10T20:41:50.322089+0000","last_became_peered":"2023-10-10T20:41:50.322089+0000","last_unstale":"2023-10-10T20:41:50.327185+0000","last_undegraded":"2023-10-10T20:41:50.327185+0000","last_fullsized":"2023-10-10T20:41:50.327185+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:28:58.113040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"137'1","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329391+0000","last_change":"2023-10-10T20:41:49.316983+0000","last_active":"2023-10-10T20:41:50.329391+0000","last_peered":"2023-10-10T20:41:50.329391+0000","last_clean":"2023-10-10T20:41:50.329391+0000","last_became_active":"2023-10-10T20:41:49.316318+0000","last_became_peered":"2023-10-10T20:41:49.316318+0000","last_unstale":"2023-10-10T20:41:50.329391+0000","last_undegraded":"2023-10-10T20:41:50.329391+0000","last_fullsized":"2023-10-10T20:41:50.329391+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:35:39.211243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.326095+0000","last_change":"2023-10-10T20:41:50.323324+0000","last_active":"2023-10-10T20:41:50.326095+0000","last_peered":"2023-10-10T20:41:50.326095+0000","last_clean":"2023-10-10T20:41:50.326095+0000","last_became_active":"2023-10-10T20:41:50.323065+0000","last_became_peered":"2023-10-10T20:41:50.323065+0000","last_unstale":"2023-10-10T20:41:50.326095+0000","last_undegraded":"2023-10-10T20:41:50.326095+0000","last_fullsized":"2023-10-10T20:41:50.326095+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:44.651808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"137'2","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.326104+0000","last_change":"2023-10-10T20:41:49.308012+0000","last_active":"2023-10-10T20:41:50.326104+0000","last_peered":"2023-10-10T20:41:50.326104+0000","last_clean":"2023-10-10T20:41:50.326104+0000","last_became_active":"2023-10-10T20:41:49.307712+0000","last_became_peered":"2023-10-10T20:41:49.307712+0000","last_unstale":"2023-10-10T20:41:50.326104+0000","last_undegraded":"2023-10-10T20:41:50.326104+0000","last_fullsized":"2023-10-10T20:41:50.326104+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:34:34.956156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.315166+0000","last_change":"2023-10-10T20:41:50.309435+0000","last_active":"2023-10-10T20:41:50.315166+0000","last_peered":"2023-10-10T20:41:50.315166+0000","last_clean":"2023-10-10T20:41:50.315166+0000","last_became_active":"2023-10-10T20:41:50.309084+0000","last_became_peered":"2023-10-10T20:41:50.309084+0000","last_unstale":"2023-10-10T20:41:50.315166+0000","last_undegraded":"2023-10-10T20:41:50.315166+0000","last_fullsized":"2023-10-10T20:41:50.315166+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:09:21.122158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"137'1","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327050+0000","last_change":"2023-10-10T20:41:49.314806+0000","last_active":"2023-10-10T20:41:50.327050+0000","last_peered":"2023-10-10T20:41:50.327050+0000","last_clean":"2023-10-10T20:41:50.327050+0000","last_became_active":"2023-10-10T20:41:49.314532+0000","last_became_peered":"2023-10-10T20:41:49.314532+0000","last_unstale":"2023-10-10T20:41:50.327050+0000","last_undegraded":"2023-10-10T20:41:50.327050+0000","last_fullsized":"2023-10-10T20:41:50.327050+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:43:37.542864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.332248+0000","last_change":"2023-10-10T20:41:50.329246+0000","last_active":"2023-10-10T20:41:50.332248+0000","last_peered":"2023-10-10T20:41:50.332248+0000","last_clean":"2023-10-10T20:41:50.332248+0000","last_became_active":"2023-10-10T20:41:50.328957+0000","last_became_peered":"2023-10-10T20:41:50.328957+0000","last_unstale":"2023-10-10T20:41:50.332248+0000","last_undegraded":"2023-10-10T20:41:50.332248+0000","last_fullsized":"2023-10-10T20:41:50.332248+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:24:36.913433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.305148+0000","last_change":"2023-10-10T20:41:49.312016+0000","last_active":"2023-10-10T20:41:50.305148+0000","last_peered":"2023-10-10T20:41:50.305148+0000","last_clean":"2023-10-10T20:41:50.305148+0000","last_became_active":"2023-10-10T20:41:49.311737+0000","last_became_peered":"2023-10-10T20:41:49.311737+0000","last_unstale":"2023-10-10T20:41:50.305148+0000","last_undegraded":"2023-10-10T20:41:50.305148+0000","last_fullsized":"2023-10-10T20:41:50.305148+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:40:21.122555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327289+0000","last_change":"2023-10-10T20:41:50.324679+0000","last_active":"2023-10-10T20:41:50.327289+0000","last_peered":"2023-10-10T20:41:50.327289+0000","last_clean":"2023-10-10T20:41:50.327289+0000","last_became_active":"2023-10-10T20:41:50.324434+0000","last_became_peered":"2023-10-10T20:41:50.324434+0000","last_unstale":"2023-10-10T20:41:50.327289+0000","last_undegraded":"2023-10-10T20:41:50.327289+0000","last_fullsized":"2023-10-10T20:41:50.327289+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:21.594296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"137'2","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329850+0000","last_change":"2023-10-10T20:41:49.318176+0000","last_active":"2023-10-10T20:41:50.329850+0000","last_peered":"2023-10-10T20:41:50.329850+0000","last_clean":"2023-10-10T20:41:50.329850+0000","last_became_active":"2023-10-10T20:41:49.317925+0000","last_became_peered":"2023-10-10T20:41:49.317925+0000","last_unstale":"2023-10-10T20:41:50.329850+0000","last_undegraded":"2023-10-10T20:41:50.329850+0000","last_fullsized":"2023-10-10T20:41:50.329850+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:24.136950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329156+0000","last_change":"2023-10-10T20:41:50.324209+0000","last_active":"2023-10-10T20:41:50.329156+0000","last_peered":"2023-10-10T20:41:50.329156+0000","last_clean":"2023-10-10T20:41:50.329156+0000","last_became_active":"2023-10-10T20:41:50.323891+0000","last_became_peered":"2023-10-10T20:41:50.323891+0000","last_unstale":"2023-10-10T20:41:50.329156+0000","last_undegraded":"2023-10-10T20:41:50.329156+0000","last_fullsized":"2023-10-10T20:41:50.329156+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:28:06.266010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.308876+0000","last_change":"2023-10-10T20:41:49.317374+0000","last_active":"2023-10-10T20:41:50.308876+0000","last_peered":"2023-10-10T20:41:50.308876+0000","last_clean":"2023-10-10T20:41:50.308876+0000","last_became_active":"2023-10-10T20:41:49.317022+0000","last_became_peered":"2023-10-10T20:41:49.317022+0000","last_unstale":"2023-10-10T20:41:50.308876+0000","last_undegraded":"2023-10-10T20:41:50.308876+0000","last_fullsized":"2023-10-10T20:41:50.308876+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:44.809357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.324040+0000","last_change":"2023-10-10T20:41:50.320161+0000","last_active":"2023-10-10T20:41:50.324040+0000","last_peered":"2023-10-10T20:41:50.324040+0000","last_clean":"2023-10-10T20:41:50.324040+0000","last_became_active":"2023-10-10T20:41:50.319896+0000","last_became_peered":"2023-10-10T20:41:50.319896+0000","last_unstale":"2023-10-10T20:41:50.324040+0000","last_undegraded":"2023-10-10T20:41:50.324040+0000","last_fullsized":"2023-10-10T20:41:50.324040+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:28:06.266010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.310033+0000","last_change":"2023-10-10T20:41:49.318718+0000","last_active":"2023-10-10T20:41:50.310033+0000","last_peered":"2023-10-10T20:41:50.310033+0000","last_clean":"2023-10-10T20:41:50.310033+0000","last_became_active":"2023-10-10T20:41:49.318480+0000","last_became_peered":"2023-10-10T20:41:49.318480+0000","last_unstale":"2023-10-10T20:41:50.310033+0000","last_undegraded":"2023-10-10T20:41:50.310033+0000","last_fullsized":"2023-10-10T20:41:50.310033+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:57.541968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327533+0000","last_change":"2023-10-10T20:41:50.321470+0000","last_active":"2023-10-10T20:41:50.327533+0000","last_peered":"2023-10-10T20:41:50.327533+0000","last_clean":"2023-10-10T20:41:50.327533+0000","last_became_active":"2023-10-10T20:41:50.315093+0000","last_became_peered":"2023-10-10T20:41:50.315093+0000","last_unstale":"2023-10-10T20:41:50.327533+0000","last_undegraded":"2023-10-10T20:41:50.327533+0000","last_fullsized":"2023-10-10T20:41:50.327533+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:42:13.861221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.302813+0000","last_change":"2023-10-10T20:41:49.314130+0000","last_active":"2023-10-10T20:41:50.302813+0000","last_peered":"2023-10-10T20:41:50.302813+0000","last_clean":"2023-10-10T20:41:50.302813+0000","last_became_active":"2023-10-10T20:41:49.313882+0000","last_became_peered":"2023-10-10T20:41:49.313882+0000","last_unstale":"2023-10-10T20:41:50.302813+0000","last_undegraded":"2023-10-10T20:41:50.302813+0000","last_fullsized":"2023-10-10T20:41:50.302813+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:51:13.831963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.323070+0000","last_change":"2023-10-10T20:41:50.313028+0000","last_active":"2023-10-10T20:41:50.323070+0000","last_peered":"2023-10-10T20:41:50.323070+0000","last_clean":"2023-10-10T20:41:50.323070+0000","last_became_active":"2023-10-10T20:41:50.312757+0000","last_became_peered":"2023-10-10T20:41:50.312757+0000","last_unstale":"2023-10-10T20:41:50.323070+0000","last_undegraded":"2023-10-10T20:41:50.323070+0000","last_fullsized":"2023-10-10T20:41:50.323070+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:51:14.837731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"137'2","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.319856+0000","last_change":"2023-10-10T20:41:49.318208+0000","last_active":"2023-10-10T20:41:50.319856+0000","last_peered":"2023-10-10T20:41:50.319856+0000","last_clean":"2023-10-10T20:41:50.319856+0000","last_became_active":"2023-10-10T20:41:49.317937+0000","last_became_peered":"2023-10-10T20:41:49.317937+0000","last_unstale":"2023-10-10T20:41:50.319856+0000","last_undegraded":"2023-10-10T20:41:50.319856+0000","last_fullsized":"2023-10-10T20:41:50.319856+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:37:13.352726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.326844+0000","last_change":"2023-10-10T20:41:50.321622+0000","last_active":"2023-10-10T20:41:50.326844+0000","last_peered":"2023-10-10T20:41:50.326844+0000","last_clean":"2023-10-10T20:41:50.326844+0000","last_became_active":"2023-10-10T20:41:50.321200+0000","last_became_peered":"2023-10-10T20:41:50.321200+0000","last_unstale":"2023-10-10T20:41:50.326844+0000","last_undegraded":"2023-10-10T20:41:50.326844+0000","last_fullsized":"2023-10-10T20:41:50.326844+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:33.972875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.314099+0000","last_change":"2023-10-10T20:41:49.315072+0000","last_active":"2023-10-10T20:41:50.314099+0000","last_peered":"2023-10-10T20:41:50.314099+0000","last_clean":"2023-10-10T20:41:50.314099+0000","last_became_active":"2023-10-10T20:41:49.314618+0000","last_became_peered":"2023-10-10T20:41:49.314618+0000","last_unstale":"2023-10-10T20:41:50.314099+0000","last_undegraded":"2023-10-10T20:41:50.314099+0000","last_fullsized":"2023-10-10T20:41:50.314099+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:04:21.204106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328741+0000","last_change":"2023-10-10T20:41:50.321375+0000","last_active":"2023-10-10T20:41:50.328741+0000","last_peered":"2023-10-10T20:41:50.328741+0000","last_clean":"2023-10-10T20:41:50.328741+0000","last_became_active":"2023-10-10T20:41:50.321099+0000","last_became_peered":"2023-10-10T20:41:50.321099+0000","last_unstale":"2023-10-10T20:41:50.328741+0000","last_undegraded":"2023-10-10T20:41:50.328741+0000","last_fullsized":"2023-10-10T20:41:50.328741+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:21.594296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.315641+0000","last_change":"2023-10-10T20:41:49.310539+0000","last_active":"2023-10-10T20:41:50.315641+0000","last_peered":"2023-10-10T20:41:50.315641+0000","last_clean":"2023-10-10T20:41:50.315641+0000","last_became_active":"2023-10-10T20:41:49.310219+0000","last_became_peered":"2023-10-10T20:41:49.310219+0000","last_unstale":"2023-10-10T20:41:50.315641+0000","last_undegraded":"2023-10-10T20:41:50.315641+0000","last_fullsized":"2023-10-10T20:41:50.315641+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:43:37.542864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327153+0000","last_change":"2023-10-10T20:41:50.318523+0000","last_active":"2023-10-10T20:41:50.327153+0000","last_peered":"2023-10-10T20:41:50.327153+0000","last_clean":"2023-10-10T20:41:50.327153+0000","last_became_active":"2023-10-10T20:41:50.318341+0000","last_became_peered":"2023-10-10T20:41:50.318341+0000","last_unstale":"2023-10-10T20:41:50.327153+0000","last_undegraded":"2023-10-10T20:41:50.327153+0000","last_fullsized":"2023-10-10T20:41:50.327153+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:36:55.761497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.308130+0000","last_change":"2023-10-10T20:41:49.308240+0000","last_active":"2023-10-10T20:41:50.308130+0000","last_peered":"2023-10-10T20:41:50.308130+0000","last_clean":"2023-10-10T20:41:50.308130+0000","last_became_active":"2023-10-10T20:41:49.307687+0000","last_became_peered":"2023-10-10T20:41:49.307687+0000","last_unstale":"2023-10-10T20:41:50.308130+0000","last_undegraded":"2023-10-10T20:41:50.308130+0000","last_fullsized":"2023-10-10T20:41:50.308130+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:22.420406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.313751+0000","last_change":"2023-10-10T20:41:50.307863+0000","last_active":"2023-10-10T20:41:50.313751+0000","last_peered":"2023-10-10T20:41:50.313751+0000","last_clean":"2023-10-10T20:41:50.313751+0000","last_became_active":"2023-10-10T20:41:50.307587+0000","last_became_peered":"2023-10-10T20:41:50.307587+0000","last_unstale":"2023-10-10T20:41:50.313751+0000","last_undegraded":"2023-10-10T20:41:50.313751+0000","last_fullsized":"2023-10-10T20:41:50.313751+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:23:01.318341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.302594+0000","last_change":"2023-10-10T20:41:49.313676+0000","last_active":"2023-10-10T20:41:50.302594+0000","last_peered":"2023-10-10T20:41:50.302594+0000","last_clean":"2023-10-10T20:41:50.302594+0000","last_became_active":"2023-10-10T20:41:49.313425+0000","last_became_peered":"2023-10-10T20:41:49.313425+0000","last_unstale":"2023-10-10T20:41:50.302594+0000","last_undegraded":"2023-10-10T20:41:50.302594+0000","last_fullsized":"2023-10-10T20:41:50.302594+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:02.015895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.318356+0000","last_change":"2023-10-10T20:41:50.315515+0000","last_active":"2023-10-10T20:41:50.318356+0000","last_peered":"2023-10-10T20:41:50.318356+0000","last_clean":"2023-10-10T20:41:50.318356+0000","last_became_active":"2023-10-10T20:41:50.314937+0000","last_became_peered":"2023-10-10T20:41:50.314937+0000","last_unstale":"2023-10-10T20:41:50.318356+0000","last_undegraded":"2023-10-10T20:41:50.318356+0000","last_fullsized":"2023-10-10T20:41:50.318356+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:23.426173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.302835+0000","last_change":"2023-10-10T20:41:49.313069+0000","last_active":"2023-10-10T20:41:50.302835+0000","last_peered":"2023-10-10T20:41:50.302835+0000","last_clean":"2023-10-10T20:41:50.302835+0000","last_became_active":"2023-10-10T20:41:49.312729+0000","last_became_peered":"2023-10-10T20:41:49.312729+0000","last_unstale":"2023-10-10T20:41:50.302835+0000","last_undegraded":"2023-10-10T20:41:50.302835+0000","last_fullsized":"2023-10-10T20:41:50.302835+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:42:12.855453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.323987+0000","last_change":"2023-10-10T20:41:50.319309+0000","last_active":"2023-10-10T20:41:50.323987+0000","last_peered":"2023-10-10T20:41:50.323987+0000","last_clean":"2023-10-10T20:41:50.323987+0000","last_became_active":"2023-10-10T20:41:50.318090+0000","last_became_peered":"2023-10-10T20:41:50.318090+0000","last_unstale":"2023-10-10T20:41:50.323987+0000","last_undegraded":"2023-10-10T20:41:50.323987+0000","last_fullsized":"2023-10-10T20:41:50.323987+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:09.931645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.310889+0000","last_change":"2023-10-10T20:41:49.317299+0000","last_active":"2023-10-10T20:41:50.310889+0000","last_peered":"2023-10-10T20:41:50.310889+0000","last_clean":"2023-10-10T20:41:50.310889+0000","last_became_active":"2023-10-10T20:41:49.316536+0000","last_became_peered":"2023-10-10T20:41:49.316536+0000","last_unstale":"2023-10-10T20:41:50.310889+0000","last_undegraded":"2023-10-10T20:41:50.310889+0000","last_fullsized":"2023-10-10T20:41:50.310889+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:24.814273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329532+0000","last_change":"2023-10-10T20:41:50.324946+0000","last_active":"2023-10-10T20:41:50.329532+0000","last_peered":"2023-10-10T20:41:50.329532+0000","last_clean":"2023-10-10T20:41:50.329532+0000","last_became_active":"2023-10-10T20:41:50.324655+0000","last_became_peered":"2023-10-10T20:41:50.324655+0000","last_unstale":"2023-10-10T20:41:50.329532+0000","last_undegraded":"2023-10-10T20:41:50.329532+0000","last_fullsized":"2023-10-10T20:41:50.329532+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:02.059224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.309870+0000","last_change":"2023-10-10T20:41:49.312704+0000","last_active":"2023-10-10T20:41:50.309870+0000","last_peered":"2023-10-10T20:41:50.309870+0000","last_clean":"2023-10-10T20:41:50.309870+0000","last_became_active":"2023-10-10T20:41:49.312386+0000","last_became_peered":"2023-10-10T20:41:49.312386+0000","last_unstale":"2023-10-10T20:41:50.309870+0000","last_undegraded":"2023-10-10T20:41:50.309870+0000","last_fullsized":"2023-10-10T20:41:50.309870+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:10:58.865055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327147+0000","last_change":"2023-10-10T20:41:50.325233+0000","last_active":"2023-10-10T20:41:50.327147+0000","last_peered":"2023-10-10T20:41:50.327147+0000","last_clean":"2023-10-10T20:41:50.327147+0000","last_became_active":"2023-10-10T20:41:50.324824+0000","last_became_peered":"2023-10-10T20:41:50.324824+0000","last_unstale":"2023-10-10T20:41:50.327147+0000","last_undegraded":"2023-10-10T20:41:50.327147+0000","last_fullsized":"2023-10-10T20:41:50.327147+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:36:51.689597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.303274+0000","last_change":"2023-10-10T20:41:49.317251+0000","last_active":"2023-10-10T20:41:50.303274+0000","last_peered":"2023-10-10T20:41:50.303274+0000","last_clean":"2023-10-10T20:41:50.303274+0000","last_became_active":"2023-10-10T20:41:49.316927+0000","last_became_peered":"2023-10-10T20:41:49.316927+0000","last_unstale":"2023-10-10T20:41:50.303274+0000","last_undegraded":"2023-10-10T20:41:50.303274+0000","last_fullsized":"2023-10-10T20:41:50.303274+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:52.358088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":281,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.303736+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:41:50.303736+0000","last_peered":"2023-10-10T20:41:50.303736+0000","last_clean":"2023-10-10T20:41:50.303736+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:41:50.303736+0000","last_undegraded":"2023-10-10T20:41:50.303736+0000","last_fullsized":"2023-10-10T20:41:50.303736+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303676,"kb_used_data":2292,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068164,"statfs":{"total":96636764160,"available":96325799936,"internally_reserved":0,"allocated":2347008,"data_stored":1371439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.79000000000000004},"min":{"1min":0.309,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71799999999999997,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52400000000000002,"15min":0.61599999999999999},"min":{"1min":0.32300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.69999999999999996,"5min":0.82099999999999995,"15min":4.1959999999999997},"last":0.56399999999999995}]},{"osd":1,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.95099999999999996},"min":{"1min":0.379,"5min":0.317,"15min":0.317},"max":{"1min":0.78300000000000003,"5min":0.78800000000000003,"15min":10.1},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55200000000000005,"15min":0.71399999999999997},"min":{"1min":0.36699999999999999,"5min":0.32300000000000001,"15min":0.28999999999999998},"max":{"1min":0.73199999999999998,"5min":0.90400000000000003,"15min":6.806},"last":0.46500000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.76400000000000001},"min":{"1min":0.40100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79200000000000004,"5min":0.89000000000000001,"15min":6.9340000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.71699999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.73299999999999998,"5min":0.83899999999999997,"15min":6.9870000000000001},"last":0.433}]},{"osd":3,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54900000000000004,"15min":0.73599999999999999},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.753,"5min":0.80200000000000005,"15min":6.6719999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56699999999999995,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.71699999999999997,"5min":0.87,"15min":4.1609999999999996},"last":0.498}]},{"osd":4,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.36599999999999999,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.65100000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36299999999999999,"5min":0.309,"15min":0.30399999999999999},"max":{"1min":0.72299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38500000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.314},"max":{"1min":0.77900000000000003,"5min":0.86099999999999999,"15min":1.3180000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.41799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71499999999999997,"5min":0.75800000000000001,"15min":0.84099999999999997},"last":0.40500000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":0.78700000000000003,"15min":1.7509999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.40899999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.85699999999999998,"15min":1.7090000000000001},"last":0.52200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":780967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51300000000000001,"15min":0.65500000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":7.7590000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.52100000000000002,"15min":0.69199999999999995},"min":{"1min":0.32200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.075,"5min":1.075,"15min":7.8090000000000002},"last":0.50800000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.69099999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.70199999999999996,"5min":0.80000000000000004,"15min":7.867},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.85599999999999998},"min":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.55000000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.58799999999999997},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.56100000000000005,"15min":0.81999999999999995},"min":{"1min":0.34200000000000003,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":2.641,"5min":2.641,"15min":6.7629999999999999},"last":0.56000000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54300000000000004,"15min":0.61099999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.107,"5min":1.107,"15min":4.8620000000000001},"last":0.62},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.70099999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.718,"5min":1.718,"15min":7.9009999999999998},"last":0.71799999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.47299999999999998},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.65200000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52700000000000002,"15min":0.495},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.53400000000000003},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52600000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55800000000000005,"15min":0.52500000000000002},"min":{"1min":0.36099999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.372,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.73799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303100,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068740,"statfs":{"total":96636764160,"available":96326389760,"internally_reserved":0,"allocated":1757184,"data_stored":782503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.312,"15min":0.45900000000000002},"min":{"1min":0.22800000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.437,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.247},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":1.194},"last":0.28299999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.24199999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51000000000000001,"5min":0.51400000000000001,"15min":1.2869999999999999},"last":0.309},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.48999999999999999},"min":{"1min":0.27200000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.52800000000000002,"5min":0.54200000000000004,"15min":5.0919999999999996},"last":0.29799999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36399999999999999,"15min":0.51200000000000001},"min":{"1min":0.29399999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35699999999999998,"15min":0.40200000000000002},"min":{"1min":0.23799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":1.2170000000000001},"last":0.32200000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.73999999999999999,"5min":0.92400000000000004,"15min":1.254},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.68999999999999995},"min":{"1min":0.39300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68799999999999994,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.45300000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.36099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":0.97599999999999998,"15min":1.1799999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56899999999999995,"15min":0.60299999999999998},"min":{"1min":0.41799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71399999999999997,"5min":0.878,"15min":1.3109999999999999},"last":0.46500000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.81799999999999995},"min":{"1min":0.46000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78900000000000003,"5min":0.995,"15min":5.0659999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80700000000000005,"5min":0.89800000000000002,"15min":1.171},"last":0.55800000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57999999999999996,"15min":0.61299999999999999},"min":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77300000000000002,"5min":1.004,"15min":1.214},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.70999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78200000000000003,"5min":0.90500000000000003,"15min":4.0499999999999998},"last":0.54000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303668,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068172,"statfs":{"total":96636764160,"available":96325808128,"internally_reserved":0,"allocated":2338816,"data_stored":1370396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.316,"15min":0.50800000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":7.9249999999999998},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.48699999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":6.6299999999999999},"last":0.30599999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.33100000000000002,"15min":0.53100000000000003},"min":{"1min":0.23100000000000001,"5min":0.22,"15min":0.216},"max":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":8.2509999999999994},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.51100000000000001},"min":{"1min":0.253,"5min":0.222,"15min":0.222},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":6.7370000000000001},"last":0.35899999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.35399999999999998,"15min":0.53900000000000003},"min":{"1min":0.247,"5min":0.23100000000000001,"15min":0.21099999999999999},"max":{"1min":0.53800000000000003,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.373},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.59099999999999997},"min":{"1min":0.27700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":7.8929999999999998},"last":0.42199999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57099999999999995,"15min":0.70299999999999996},"min":{"1min":0.45500000000000002,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.82499999999999996,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57299999999999995,"15min":0.754},"min":{"1min":0.441,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.80900000000000005,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.51600000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56100000000000005,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71299999999999997,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.76100000000000001},"min":{"1min":0.48399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.52300000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61499999999999999,"15min":0.80000000000000004},"min":{"1min":0.52200000000000002,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.83299999999999996,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.62,"15min":0.749},"min":{"1min":0.47399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82599999999999996,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.627}]},{"osd":7,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.60099999999999998,"15min":0.73999999999999999},"min":{"1min":0.48399999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85399999999999998,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.76800000000000002},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.75700000000000001,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.498}]}]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303100,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068740,"statfs":{"total":96636764160,"available":96326389760,"internally_reserved":0,"allocated":1757184,"data_stored":781633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.311,"15min":0.32500000000000001},"min":{"1min":0.214,"5min":0.189,"15min":0.189},"max":{"1min":0.52500000000000002,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.20200000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.35099999999999998},"min":{"1min":0.219,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.746},"last":0.224}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.69099999999999995},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52100000000000002,"5min":0.63,"15min":11.967000000000001},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.371,"5min":0.34899999999999998,"15min":0.46999999999999997},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48899999999999999,"5min":0.53400000000000003,"15min":4.1100000000000003},"last":0.33500000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34699999999999998,"15min":0.53700000000000003},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.50800000000000001,"5min":0.51600000000000001,"15min":6.3040000000000003},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.74299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59299999999999997,"5min":0.65900000000000003,"15min":11.935},"last":0.32200000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.55700000000000005,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.47399999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.38,"15min":0.378},"max":{"1min":0.83699999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54200000000000004,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.96099999999999997},"last":0.46100000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.55200000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.58599999999999997},"min":{"1min":0.435,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.052},"last":0.5},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58099999999999996,"15min":0.76300000000000001},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":6.2649999999999997},"last":0.45500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574942,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303100,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068740,"statfs":{"total":96636764160,"available":96326389760,"internally_reserved":0,"allocated":1757184,"data_stored":781161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.315,"15min":0.315},"min":{"1min":0.223,"5min":0.21199999999999999,"15min":0.21099999999999999},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32100000000000001},"min":{"1min":0.23999999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48799999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.23799999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.34599999999999997,"15min":0.33300000000000002},"min":{"1min":0.25700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.379},"min":{"1min":0.29199999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.72099999999999997},"last":0.30299999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.36199999999999999,"15min":0.35299999999999998},"min":{"1min":0.25800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57599999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.28999999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.50900000000000001,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.374}]},{"osd":4,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.53400000000000003},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77200000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.745,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.57199999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57299999999999995,"15min":0.55200000000000005},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.35899999999999999},"max":{"1min":0.82399999999999995,"5min":1.347,"15min":1.347},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.42199999999999999,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.83499999999999996,"5min":1.01,"15min":1.01},"last":0.52700000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.48399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86299999999999999,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.81200000000000006,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.59099999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78700000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.624},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.83699999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.54700000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303104,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068736,"statfs":{"total":96636764160,"available":96326385664,"internally_reserved":0,"allocated":1761280,"data_stored":783057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.32400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66600000000000004,"5min":0.74199999999999999,"15min":0.747},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51700000000000002,"15min":0.74099999999999999},"min":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77600000000000002,"5min":0.78400000000000003,"15min":7.6719999999999997},"last":0.46000000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52000000000000002,"15min":0.748},"min":{"1min":0.32000000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.755,"15min":7.7160000000000002},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66100000000000003,"5min":0.76900000000000002,"15min":0.86799999999999999},"last":0.67100000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.73899999999999999,"5min":0.79800000000000004,"15min":7.7839999999999998},"last":0.74099999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.39200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.71499999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.49099999999999999},"min":{"1min":0.35199999999999998,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.874},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.82999999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.85199999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58399999999999996,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.81599999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.41399999999999998,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.85299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":780967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.47599999999999998,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.66100000000000003,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.51800000000000002},"min":{"1min":0.33000000000000002,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.68899999999999995,"5min":0.76700000000000002,"15min":0.79300000000000004},"last":0.61799999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51300000000000001,"15min":0.628},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63400000000000001,"5min":0.75700000000000001,"15min":5.2679999999999998},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.62},"min":{"1min":0.40500000000000003,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79200000000000004,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.58099999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52400000000000002,"15min":0.63700000000000001},"min":{"1min":0.39100000000000001,"5min":0.32900000000000001,"15min":0.28299999999999997},"max":{"1min":0.73299999999999998,"5min":0.76600000000000001,"15min":5.2960000000000003},"last":0.505},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.38500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.76400000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55900000000000005,"15min":0.66900000000000004},"min":{"1min":0.41999999999999998,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":5.3330000000000002},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56399999999999995,"15min":0.68000000000000005},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.27700000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.81499999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.48899999999999999,"15min":0.498},"min":{"1min":0.33400000000000002,"5min":0.29599999999999999,"15min":0.28699999999999998},"max":{"1min":0.67000000000000004,"5min":0.72499999999999998,"15min":0.82999999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.34499999999999997,"15min":0.32200000000000001},"max":{"1min":0.77200000000000002,"5min":0.88,"15min":1.0800000000000001},"last":0.56000000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.39900000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78900000000000003,"5min":0.82499999999999996,"15min":0.89800000000000002},"last":0.78900000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.32300000000000001,"15min":0.30099999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98399999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.92600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:41:55.116 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-10T20:41:55.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.263+0000 7f3156e39700 1 Processor -- start 2023-10-10T20:41:55.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.263+0000 7f3156e39700 1 -- start start 2023-10-10T20:41:55.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.263+0000 7f3156e39700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f31500cd910 0x7f31500ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:55.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.263+0000 7f3156e39700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f31500b55a0 0x7f31500ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:55.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.263+0000 7f3156e39700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31500b5b00 con 0x7f3150159ec0 2023-10-10T20:41:55.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.263+0000 7f3156e39700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31500b5da0 con 0x7f31500cd910 2023-10-10T20:41:55.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.263+0000 7f3156e39700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31500b6080 con 0x7f31500b55a0 2023-10-10T20:41:55.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.264+0000 7f3147fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f31500cd910 0x7f31500ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:55.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.264+0000 7f3154bd5700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3150159ec0 0x7f31500b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55854/0 (socket says 172.21.15.89:55854) 2023-10-10T20:41:55.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.264+0000 7f3154bd5700 1 -- 172.21.15.89:0/2395071342 learned_addr learned my addr 172.21.15.89:0/2395071342 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:55.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.264+0000 7f31553d6700 1 --2- 172.21.15.89:0/2395071342 >> v2:172.21.15.111:3300/0 conn(0x7f31500b55a0 0x7f31500ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:55.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.264+0000 7f3147fff700 1 -- 172.21.15.89:0/2395071342 >> v2:172.21.15.111:3300/0 conn(0x7f31500b55a0 msgr2=0x7f31500ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:55.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.264+0000 7f3147fff700 1 --2- 172.21.15.89:0/2395071342 >> v2:172.21.15.111:3300/0 conn(0x7f31500b55a0 0x7f31500ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.264+0000 7f3147fff700 1 -- 172.21.15.89:0/2395071342 >> v1:172.21.15.89:6789/0 conn(0x7f3150159ec0 legacy=0x7f31500b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:55.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.264+0000 7f31477fe700 1 -- 172.21.15.89:0/2395071342 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4187488632 0 0) 0x7f31500b5b00 con 0x7f3150159ec0 2023-10-10T20:41:55.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.264+0000 7f3147fff700 1 -- 172.21.15.89:0/2395071342 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31500cc100 con 0x7f31500cd910 2023-10-10T20:41:55.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.264+0000 7f3147fff700 1 --2- 172.21.15.89:0/2395071342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31500cd910 0x7f31500ce340 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f31400098e0 tx=0x7f31400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e769727dda5d62 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:55.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.264+0000 7f31477fe700 1 -- 172.21.15.89:0/2395071342 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140009e80 con 0x7f31500cd910 2023-10-10T20:41:55.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.264+0000 7f31477fe700 1 -- 172.21.15.89:0/2395071342 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314000b820 con 0x7f31500cd910 2023-10-10T20:41:55.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.264+0000 7f31477fe700 1 -- 172.21.15.89:0/2395071342 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314000bba0 con 0x7f31500cd910 2023-10-10T20:41:55.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.265+0000 7f3156e39700 1 -- 172.21.15.89:0/2395071342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31500cd910 msgr2=0x7f31500ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:55.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.265+0000 7f3156e39700 1 --2- 172.21.15.89:0/2395071342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31500cd910 0x7f31500ce340 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f31400098e0 tx=0x7f31400091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.265+0000 7f3156e39700 1 -- 172.21.15.89:0/2395071342 shutdown_connections 2023-10-10T20:41:55.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.265+0000 7f3156e39700 1 --2- 172.21.15.89:0/2395071342 >> v2:172.21.15.111:3300/0 conn(0x7f31500b55a0 0x7f31500ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.265+0000 7f3156e39700 1 --2- 172.21.15.89:0/2395071342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31500cd910 0x7f31500ce340 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.265+0000 7f3156e39700 1 -- 172.21.15.89:0/2395071342 >> 172.21.15.89:0/2395071342 conn(0x7f31500999f0 msgr2=0x7f31500b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:55.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.265+0000 7f3156e39700 1 -- 172.21.15.89:0/2395071342 shutdown_connections 2023-10-10T20:41:55.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.265+0000 7f3156e39700 1 -- 172.21.15.89:0/2395071342 wait complete. 2023-10-10T20:41:55.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.266+0000 7f3156e39700 1 Processor -- start 2023-10-10T20:41:55.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.266+0000 7f3156e39700 1 -- start start 2023-10-10T20:41:55.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3156e39700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31500b55a0 0x7f315016b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:55.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3156e39700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3150159ec0 0x7f315014b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:55.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3156e39700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f315006a440 0x7f315006a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3156e39700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315009a6e0 con 0x7f31500b55a0 2023-10-10T20:41:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3156e39700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315014b770 con 0x7f315006a440 2023-10-10T20:41:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3156e39700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f315009a930 con 0x7f3150159ec0 2023-10-10T20:41:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3154bd5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31500b55a0 0x7f315016b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3154bd5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31500b55a0 0x7f315016b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56624/0 (socket says 172.21.15.89:56624) 2023-10-10T20:41:55.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3154bd5700 1 -- 172.21.15.89:0/3785505497 learned_addr learned my addr 172.21.15.89:0/3785505497 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:55.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3147fff700 1 --2- 172.21.15.89:0/3785505497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3150159ec0 0x7f315014b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:55.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f31553d6700 1 --2- 172.21.15.89:0/3785505497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f315006a440 0x7f315006a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:55.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3154bd5700 1 -- 172.21.15.89:0/3785505497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3150159ec0 msgr2=0x7f315014b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:55.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3154bd5700 1 --2- 172.21.15.89:0/3785505497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3150159ec0 0x7f315014b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3154bd5700 1 -- 172.21.15.89:0/3785505497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f315006a440 msgr2=0x7f315006a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:55.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3154bd5700 1 --2- 172.21.15.89:0/3785505497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f315006a440 0x7f315006a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3154bd5700 1 -- 172.21.15.89:0/3785505497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3140009420 con 0x7f31500b55a0 2023-10-10T20:41:55.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f31553d6700 1 --2- 172.21.15.89:0/3785505497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f315006a440 0x7f315006a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:55.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3154bd5700 1 --2- 172.21.15.89:0/3785505497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31500b55a0 0x7f315016b030 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f314c00cb60 tx=0x7f314c0017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:55.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3144ff9700 1 -- 172.21.15.89:0/3785505497 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c005d80 con 0x7f31500b55a0 2023-10-10T20:41:55.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3144ff9700 1 -- 172.21.15.89:0/3785505497 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314c00a5a0 con 0x7f31500b55a0 2023-10-10T20:41:55.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3144ff9700 1 -- 172.21.15.89:0/3785505497 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c00a960 con 0x7f31500b55a0 2023-10-10T20:41:55.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3156e39700 1 -- 172.21.15.89:0/3785505497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f315009ac10 con 0x7f31500b55a0 2023-10-10T20:41:55.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.267+0000 7f3156e39700 1 -- 172.21.15.89:0/3785505497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f315009b1d0 con 0x7f31500b55a0 2023-10-10T20:41:55.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.268+0000 7f3156e39700 1 -- 172.21.15.89:0/3785505497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3150063c90 con 0x7f31500b55a0 2023-10-10T20:41:55.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.269+0000 7f3144ff9700 1 -- 172.21.15.89:0/3785505497 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f314c00acd0 con 0x7f31500b55a0 2023-10-10T20:41:55.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.269+0000 7f3144ff9700 1 --2- 172.21.15.89:0/3785505497 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f313c067320 0x7f313c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:55.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.269+0000 7f3144ff9700 1 -- 172.21.15.89:0/3785505497 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f314c084630 con 0x7f31500b55a0 2023-10-10T20:41:55.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.269+0000 7f3147fff700 1 --2- 172.21.15.89:0/3785505497 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f313c067320 0x7f313c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:55.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.269+0000 7f3147fff700 1 --2- 172.21.15.89:0/3785505497 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f313c067320 0x7f313c0697d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f31500c75d0 tx=0x7f3140017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:55.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.273+0000 7f3144ff9700 1 -- 172.21.15.89:0/3785505497 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f314c051440 con 0x7f31500b55a0 2023-10-10T20:41:55.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.536+0000 7f3156e39700 1 -- 172.21.15.89:0/3785505497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f315004c3c0 con 0x7f31500b55a0 2023-10-10T20:41:55.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.537+0000 7f3144ff9700 1 -- 172.21.15.89:0/3785505497 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1437 (secure 0 0 0) 0x7f314c0185b0 con 0x7f31500b55a0 2023-10-10T20:41:55.539 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:55.540 INFO:teuthology.orchestra.run.smithi089.stdout:{"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":458,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":138,"num_osds":8,"num_up_osds":8,"osd_up_since":1696970067,"num_in_osds":8,"osd_in_since":1696970059,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592674,"bytes_used":2484146176,"bytes_avail":770609967104,"bytes_total":773094113280,"write_bytes_sec":2218,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":91,"id":12,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":12,"rank":0,"name":"a","status":"up:active","gid":7578}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2023-10-10T20:41:32.735512+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-10T20:41:55.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.539+0000 7f3156e39700 1 -- 172.21.15.89:0/3785505497 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f313c067320 msgr2=0x7f313c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:55.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.539+0000 7f3156e39700 1 --2- 172.21.15.89:0/3785505497 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f313c067320 0x7f313c0697d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f31500c75d0 tx=0x7f3140017000 comp rx=0 tx=0).stop 2023-10-10T20:41:55.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.539+0000 7f3156e39700 1 -- 172.21.15.89:0/3785505497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31500b55a0 msgr2=0x7f315016b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:55.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.539+0000 7f3156e39700 1 --2- 172.21.15.89:0/3785505497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31500b55a0 0x7f315016b030 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f314c00cb60 tx=0x7f314c0017b0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.539+0000 7f3156e39700 1 -- 172.21.15.89:0/3785505497 shutdown_connections 2023-10-10T20:41:55.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.539+0000 7f3156e39700 1 --2- 172.21.15.89:0/3785505497 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f313c067320 0x7f313c0697d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.539+0000 7f3156e39700 1 --2- 172.21.15.89:0/3785505497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f315006a440 0x7f315006a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.539+0000 7f3156e39700 1 --2- 172.21.15.89:0/3785505497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31500b55a0 0x7f315016b030 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.539+0000 7f3156e39700 1 --2- 172.21.15.89:0/3785505497 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3150159ec0 0x7f315014b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.539+0000 7f3156e39700 1 -- 172.21.15.89:0/3785505497 >> 172.21.15.89:0/3785505497 conn(0x7f31500999f0 msgr2=0x7f31500c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:55.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.539+0000 7f3156e39700 1 -- 172.21.15.89:0/3785505497 shutdown_connections 2023-10-10T20:41:55.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.539+0000 7f3156e39700 1 -- 172.21.15.89:0/3785505497 wait complete. 2023-10-10T20:41:55.553 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-10T20:41:55.554 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:41:55.608 INFO:tasks.ceph.mds.a.smithi089.stderr:2023-10-10T20:41:55.606+0000 7f125766d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:41:55.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.666+0000 7fcb8f59e700 1 Processor -- start 2023-10-10T20:41:55.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.666+0000 7fcb8f59e700 1 -- start start 2023-10-10T20:41:55.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.667+0000 7fcb8f59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb90159420 0x7fcb900b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:55.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.667+0000 7fcb8f59e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb900b2980 con 0x7fcb9015d230 2023-10-10T20:41:55.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.667+0000 7fcb8f59e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb900b11d0 con 0x7fcb901597f0 2023-10-10T20:41:55.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.667+0000 7fcb8f59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb900b14b0 con 0x7fcb90159420 2023-10-10T20:41:55.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.667+0000 7fcb8ed9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcb9015d230 0x7fcb900b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55864/0 (socket says 172.21.15.89:55864) 2023-10-10T20:41:55.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.667+0000 7fcb8ed9d700 1 -- 172.21.15.89:0/627148581 learned_addr learned my addr 172.21.15.89:0/627148581 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:55.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.667+0000 7fcb8e59c700 1 --2- 172.21.15.89:0/627148581 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb90159420 0x7fcb900b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:55.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.667+0000 7fcb8dd9b700 1 -- 172.21.15.89:0/627148581 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724218551 0 0) 0x7fcb900b2980 con 0x7fcb9015d230 2023-10-10T20:41:55.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.667+0000 7fcb8dd9b700 1 -- 172.21.15.89:0/627148581 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb7c003570 con 0x7fcb9015d230 2023-10-10T20:41:55.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.667+0000 7fcb8dd9b700 1 -- 172.21.15.89:0/627148581 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098519154 0 0) 0x7fcb900b11d0 con 0x7fcb901597f0 2023-10-10T20:41:55.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.667+0000 7fcb8dd9b700 1 -- 172.21.15.89:0/627148581 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb900b2980 con 0x7fcb901597f0 2023-10-10T20:41:55.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.667+0000 7fcb8dd9b700 1 -- 172.21.15.89:0/627148581 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2706943098 0 0) 0x7fcb7c003570 con 0x7fcb9015d230 2023-10-10T20:41:55.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.667+0000 7fcb8e59c700 1 -- 172.21.15.89:0/627148581 >> v1:172.21.15.111:6789/0 conn(0x7fcb901597f0 legacy=0x7fcb900b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:55.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.667+0000 7fcb8e59c700 1 -- 172.21.15.89:0/627148581 >> v1:172.21.15.89:6789/0 conn(0x7fcb9015d230 legacy=0x7fcb900b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:55.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.667+0000 7fcb8e59c700 1 -- 172.21.15.89:0/627148581 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb900b1790 con 0x7fcb90159420 2023-10-10T20:41:55.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.668+0000 7fcb8e59c700 1 --2- 172.21.15.89:0/627148581 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb90159420 0x7fcb900b5f40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009910 tx=0x7fcb800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82d19f65952b5592 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:55.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.668+0000 7fcb8dd9b700 1 -- 172.21.15.89:0/627148581 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb880017f0 con 0x7fcb90159420 2023-10-10T20:41:55.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.668+0000 7fcb8dd9b700 1 -- 172.21.15.89:0/627148581 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb80011420 con 0x7fcb90159420 2023-10-10T20:41:55.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.668+0000 7fcb8dd9b700 1 -- 172.21.15.89:0/627148581 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb800117c0 con 0x7fcb90159420 2023-10-10T20:41:55.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.668+0000 7fcb8f59e700 1 -- 172.21.15.89:0/627148581 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb90159420 msgr2=0x7fcb900b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:55.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.668+0000 7fcb8f59e700 1 --2- 172.21.15.89:0/627148581 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb90159420 0x7fcb900b5f40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009910 tx=0x7fcb800091a0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.668+0000 7fcb8f59e700 1 -- 172.21.15.89:0/627148581 shutdown_connections 2023-10-10T20:41:55.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.668+0000 7fcb8f59e700 1 --2- 172.21.15.89:0/627148581 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb90159420 0x7fcb900b5f40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.668+0000 7fcb8f59e700 1 -- 172.21.15.89:0/627148581 >> 172.21.15.89:0/627148581 conn(0x7fcb900d0660 msgr2=0x7fcb900d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:55.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.668+0000 7fcb8f59e700 1 -- 172.21.15.89:0/627148581 shutdown_connections 2023-10-10T20:41:55.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.668+0000 7fcb8f59e700 1 -- 172.21.15.89:0/627148581 wait complete. 2023-10-10T20:41:55.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.670+0000 7fcb8f59e700 1 Processor -- start 2023-10-10T20:41:55.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.670+0000 7fcb8f59e700 1 -- start start 2023-10-10T20:41:55.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.670+0000 7fcb8f59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb90159420 0x7fcb9014ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:55.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.670+0000 7fcb8f59e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb901597f0 0x7fcb90150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:55.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.670+0000 7fcb8f59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb9015d230 0x7fcb90151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:55.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.670+0000 7fcb8f59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9015aa00 con 0x7fcb90159420 2023-10-10T20:41:55.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.670+0000 7fcb8f59e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9015ace0 con 0x7fcb901597f0 2023-10-10T20:41:55.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.670+0000 7fcb8f59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9015afc0 con 0x7fcb9015d230 2023-10-10T20:41:55.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.670+0000 7fcb8e59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb90159420 0x7fcb9014ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:55.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.670+0000 7fcb8e59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb90159420 0x7fcb9014ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56634/0 (socket says 172.21.15.89:56634) 2023-10-10T20:41:55.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.671+0000 7fcb8e59c700 1 -- 172.21.15.89:0/3757513590 learned_addr learned my addr 172.21.15.89:0/3757513590 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:55.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.671+0000 7fcb85d9b700 1 --2- 172.21.15.89:0/3757513590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb901597f0 0x7fcb90150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:55.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.671+0000 7fcb8ed9d700 1 --2- 172.21.15.89:0/3757513590 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb9015d230 0x7fcb90151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:55.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.671+0000 7fcb8e59c700 1 -- 172.21.15.89:0/3757513590 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb9015d230 msgr2=0x7fcb90151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:55.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.671+0000 7fcb8e59c700 1 --2- 172.21.15.89:0/3757513590 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb9015d230 0x7fcb90151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.671+0000 7fcb8e59c700 1 -- 172.21.15.89:0/3757513590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb901597f0 msgr2=0x7fcb90150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:55.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.671+0000 7fcb8e59c700 1 --2- 172.21.15.89:0/3757513590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb901597f0 0x7fcb90150bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.671+0000 7fcb8e59c700 1 -- 172.21.15.89:0/3757513590 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb80009420 con 0x7fcb90159420 2023-10-10T20:41:55.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.671+0000 7fcb8e59c700 1 --2- 172.21.15.89:0/3757513590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb90159420 0x7fcb9014ccf0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009860 tx=0x7fcb80016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:55.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.671+0000 7fcb86ffd700 1 -- 172.21.15.89:0/3757513590 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8001cb30 con 0x7fcb90159420 2023-10-10T20:41:55.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.671+0000 7fcb86ffd700 1 -- 172.21.15.89:0/3757513590 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8001cc90 con 0x7fcb90159420 2023-10-10T20:41:55.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.671+0000 7fcb8f59e700 1 -- 172.21.15.89:0/3757513590 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb9015b2a0 con 0x7fcb90159420 2023-10-10T20:41:55.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.671+0000 7fcb86ffd700 1 -- 172.21.15.89:0/3757513590 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8001a6a0 con 0x7fcb90159420 2023-10-10T20:41:55.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.671+0000 7fcb8f59e700 1 -- 172.21.15.89:0/3757513590 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb9014d230 con 0x7fcb90159420 2023-10-10T20:41:55.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.672+0000 7fcb8f59e700 1 -- 172.21.15.89:0/3757513590 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9004c3c0 con 0x7fcb90159420 2023-10-10T20:41:55.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.673+0000 7fcb86ffd700 1 -- 172.21.15.89:0/3757513590 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcb8001aad0 con 0x7fcb90159420 2023-10-10T20:41:55.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.673+0000 7fcb86ffd700 1 --2- 172.21.15.89:0/3757513590 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb70067320 0x7fcb700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:55.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.673+0000 7fcb86ffd700 1 -- 172.21.15.89:0/3757513590 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcb80086900 con 0x7fcb90159420 2023-10-10T20:41:55.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.673+0000 7fcb85d9b700 1 --2- 172.21.15.89:0/3757513590 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb70067320 0x7fcb700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:55.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.673+0000 7fcb85d9b700 1 --2- 172.21.15.89:0/3757513590 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb70067320 0x7fcb700697d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcb90154da0 tx=0x7fcb78007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:55.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.677+0000 7fcb86ffd700 1 -- 172.21.15.89:0/3757513590 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb800536a0 con 0x7fcb90159420 2023-10-10T20:41:55.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.856+0000 7fcb8f59e700 1 -- 172.21.15.89:0/3757513590 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcb900b1a70 con 0x7fcb70067320 2023-10-10T20:41:55.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.863+0000 7fcb86ffd700 1 -- 172.21.15.89:0/3757513590 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290156 (secure 0 0 0) 0x7fcb900b1a70 con 0x7fcb70067320 2023-10-10T20:41:55.869 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:55.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.866+0000 7fcb8f59e700 1 -- 172.21.15.89:0/3757513590 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb70067320 msgr2=0x7fcb700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:55.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.866+0000 7fcb8f59e700 1 --2- 172.21.15.89:0/3757513590 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb70067320 0x7fcb700697d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcb90154da0 tx=0x7fcb78007d50 comp rx=0 tx=0).stop 2023-10-10T20:41:55.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.866+0000 7fcb8f59e700 1 -- 172.21.15.89:0/3757513590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb90159420 msgr2=0x7fcb9014ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:55.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.866+0000 7fcb8f59e700 1 --2- 172.21.15.89:0/3757513590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb90159420 0x7fcb9014ccf0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009860 tx=0x7fcb80016000 comp rx=0 tx=0).stop 2023-10-10T20:41:55.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.866+0000 7fcb8f59e700 1 -- 172.21.15.89:0/3757513590 shutdown_connections 2023-10-10T20:41:55.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.866+0000 7fcb8f59e700 1 --2- 172.21.15.89:0/3757513590 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb9015d230 0x7fcb90151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.866+0000 7fcb8f59e700 1 --2- 172.21.15.89:0/3757513590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb90159420 0x7fcb9014ccf0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.866+0000 7fcb8f59e700 1 --2- 172.21.15.89:0/3757513590 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb70067320 0x7fcb700697d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.866+0000 7fcb8f59e700 1 --2- 172.21.15.89:0/3757513590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb901597f0 0x7fcb90150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:55.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.866+0000 7fcb8f59e700 1 -- 172.21.15.89:0/3757513590 >> 172.21.15.89:0/3757513590 conn(0x7fcb900d0660 msgr2=0x7fcb90069bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:55.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.866+0000 7fcb8f59e700 1 -- 172.21.15.89:0/3757513590 shutdown_connections 2023-10-10T20:41:55.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:55.866+0000 7fcb8f59e700 1 -- 172.21.15.89:0/3757513590 wait complete. 2023-10-10T20:41:55.874 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:41:55.880 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2023-10-10T20:41:54.739427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425924,"kb_used_data":14852,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752548796,"statfs":{"total":773094113280,"available":770609967104,"internally_reserved":0,"allocated":15208448,"data_stored":7432123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001128"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328850+0000","last_change":"2023-10-10T20:41:50.324854+0000","last_active":"2023-10-10T20:41:50.328850+0000","last_peered":"2023-10-10T20:41:50.328850+0000","last_clean":"2023-10-10T20:41:50.328850+0000","last_became_active":"2023-10-10T20:41:50.324573+0000","last_became_peered":"2023-10-10T20:41:50.324573+0000","last_unstale":"2023-10-10T20:41:50.328850+0000","last_undegraded":"2023-10-10T20:41:50.328850+0000","last_fullsized":"2023-10-10T20:41:50.328850+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:07:32.681938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329631+0000","last_change":"2023-10-10T20:41:50.328756+0000","last_active":"2023-10-10T20:41:50.329631+0000","last_peered":"2023-10-10T20:41:50.329631+0000","last_clean":"2023-10-10T20:41:50.329631+0000","last_became_active":"2023-10-10T20:41:50.328488+0000","last_became_peered":"2023-10-10T20:41:50.328488+0000","last_unstale":"2023-10-10T20:41:50.329631+0000","last_undegraded":"2023-10-10T20:41:50.329631+0000","last_fullsized":"2023-10-10T20:41:50.329631+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:32:34.541780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.322750+0000","last_change":"2023-10-10T20:41:50.317128+0000","last_active":"2023-10-10T20:41:50.322750+0000","last_peered":"2023-10-10T20:41:50.322750+0000","last_clean":"2023-10-10T20:41:50.322750+0000","last_became_active":"2023-10-10T20:41:50.316862+0000","last_became_peered":"2023-10-10T20:41:50.316862+0000","last_unstale":"2023-10-10T20:41:50.322750+0000","last_undegraded":"2023-10-10T20:41:50.322750+0000","last_fullsized":"2023-10-10T20:41:50.322750+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:59.167375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329233+0000","last_change":"2023-10-10T20:41:50.325779+0000","last_active":"2023-10-10T20:41:50.329233+0000","last_peered":"2023-10-10T20:41:50.329233+0000","last_clean":"2023-10-10T20:41:50.329233+0000","last_became_active":"2023-10-10T20:41:50.325527+0000","last_became_peered":"2023-10-10T20:41:50.325527+0000","last_unstale":"2023-10-10T20:41:50.329233+0000","last_undegraded":"2023-10-10T20:41:50.329233+0000","last_fullsized":"2023-10-10T20:41:50.329233+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:09:42.528347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329492+0000","last_change":"2023-10-10T20:41:50.322671+0000","last_active":"2023-10-10T20:41:50.329492+0000","last_peered":"2023-10-10T20:41:50.329492+0000","last_clean":"2023-10-10T20:41:50.329492+0000","last_became_active":"2023-10-10T20:41:50.322277+0000","last_became_peered":"2023-10-10T20:41:50.322277+0000","last_unstale":"2023-10-10T20:41:50.329492+0000","last_undegraded":"2023-10-10T20:41:50.329492+0000","last_fullsized":"2023-10-10T20:41:50.329492+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:03.021663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.320162+0000","last_change":"2023-10-10T20:41:50.314512+0000","last_active":"2023-10-10T20:41:50.320162+0000","last_peered":"2023-10-10T20:41:50.320162+0000","last_clean":"2023-10-10T20:41:50.320162+0000","last_became_active":"2023-10-10T20:41:50.314257+0000","last_became_peered":"2023-10-10T20:41:50.314257+0000","last_unstale":"2023-10-10T20:41:50.320162+0000","last_undegraded":"2023-10-10T20:41:50.320162+0000","last_fullsized":"2023-10-10T20:41:50.320162+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:52:08.397497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329082+0000","last_change":"2023-10-10T20:41:50.322740+0000","last_active":"2023-10-10T20:41:50.329082+0000","last_peered":"2023-10-10T20:41:50.329082+0000","last_clean":"2023-10-10T20:41:50.329082+0000","last_became_active":"2023-10-10T20:41:50.322415+0000","last_became_peered":"2023-10-10T20:41:50.322415+0000","last_unstale":"2023-10-10T20:41:50.329082+0000","last_undegraded":"2023-10-10T20:41:50.329082+0000","last_fullsized":"2023-10-10T20:41:50.329082+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:59.167375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327463+0000","last_change":"2023-10-10T20:41:50.320942+0000","last_active":"2023-10-10T20:41:50.327463+0000","last_peered":"2023-10-10T20:41:50.327463+0000","last_clean":"2023-10-10T20:41:50.327463+0000","last_became_active":"2023-10-10T20:41:50.320312+0000","last_became_peered":"2023-10-10T20:41:50.320312+0000","last_unstale":"2023-10-10T20:41:50.327463+0000","last_undegraded":"2023-10-10T20:41:50.327463+0000","last_fullsized":"2023-10-10T20:41:50.327463+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:08:58.453242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.324745+0000","last_change":"2023-10-10T20:41:50.318424+0000","last_active":"2023-10-10T20:41:50.324745+0000","last_peered":"2023-10-10T20:41:50.324745+0000","last_clean":"2023-10-10T20:41:50.324745+0000","last_became_active":"2023-10-10T20:41:50.318177+0000","last_became_peered":"2023-10-10T20:41:50.318177+0000","last_unstale":"2023-10-10T20:41:50.324745+0000","last_undegraded":"2023-10-10T20:41:50.324745+0000","last_fullsized":"2023-10-10T20:41:50.324745+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:25.820041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.325823+0000","last_change":"2023-10-10T20:41:50.320526+0000","last_active":"2023-10-10T20:41:50.325823+0000","last_peered":"2023-10-10T20:41:50.325823+0000","last_clean":"2023-10-10T20:41:50.325823+0000","last_became_active":"2023-10-10T20:41:50.320284+0000","last_became_peered":"2023-10-10T20:41:50.320284+0000","last_unstale":"2023-10-10T20:41:50.325823+0000","last_undegraded":"2023-10-10T20:41:50.325823+0000","last_fullsized":"2023-10-10T20:41:50.325823+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:56.229435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.316595+0000","last_change":"2023-10-10T20:41:50.312426+0000","last_active":"2023-10-10T20:41:50.316595+0000","last_peered":"2023-10-10T20:41:50.316595+0000","last_clean":"2023-10-10T20:41:50.316595+0000","last_became_active":"2023-10-10T20:41:50.312160+0000","last_became_peered":"2023-10-10T20:41:50.312160+0000","last_unstale":"2023-10-10T20:41:50.316595+0000","last_undegraded":"2023-10-10T20:41:50.316595+0000","last_fullsized":"2023-10-10T20:41:50.316595+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:49:46.940449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.324082+0000","last_change":"2023-10-10T20:41:50.319250+0000","last_active":"2023-10-10T20:41:50.324082+0000","last_peered":"2023-10-10T20:41:50.324082+0000","last_clean":"2023-10-10T20:41:50.324082+0000","last_became_active":"2023-10-10T20:41:50.318945+0000","last_became_peered":"2023-10-10T20:41:50.318945+0000","last_unstale":"2023-10-10T20:41:50.324082+0000","last_undegraded":"2023-10-10T20:41:50.324082+0000","last_fullsized":"2023-10-10T20:41:50.324082+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:29.171361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.324092+0000","last_change":"2023-10-10T20:41:50.320071+0000","last_active":"2023-10-10T20:41:50.324092+0000","last_peered":"2023-10-10T20:41:50.324092+0000","last_clean":"2023-10-10T20:41:50.324092+0000","last_became_active":"2023-10-10T20:41:50.319800+0000","last_became_peered":"2023-10-10T20:41:50.319800+0000","last_unstale":"2023-10-10T20:41:50.324092+0000","last_undegraded":"2023-10-10T20:41:50.324092+0000","last_fullsized":"2023-10-10T20:41:50.324092+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:59.707661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.324909+0000","last_change":"2023-10-10T20:41:50.320391+0000","last_active":"2023-10-10T20:41:50.324909+0000","last_peered":"2023-10-10T20:41:50.324909+0000","last_clean":"2023-10-10T20:41:50.324909+0000","last_became_active":"2023-10-10T20:41:50.320087+0000","last_became_peered":"2023-10-10T20:41:50.320087+0000","last_unstale":"2023-10-10T20:41:50.324909+0000","last_undegraded":"2023-10-10T20:41:50.324909+0000","last_fullsized":"2023-10-10T20:41:50.324909+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:56:15.566395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.323944+0000","last_change":"2023-10-10T20:41:50.319419+0000","last_active":"2023-10-10T20:41:50.323944+0000","last_peered":"2023-10-10T20:41:50.323944+0000","last_clean":"2023-10-10T20:41:50.323944+0000","last_became_active":"2023-10-10T20:41:50.318904+0000","last_became_peered":"2023-10-10T20:41:50.318904+0000","last_unstale":"2023-10-10T20:41:50.323944+0000","last_undegraded":"2023-10-10T20:41:50.323944+0000","last_fullsized":"2023-10-10T20:41:50.323944+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:59.167375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327445+0000","last_change":"2023-10-10T20:41:50.314811+0000","last_active":"2023-10-10T20:41:50.327445+0000","last_peered":"2023-10-10T20:41:50.327445+0000","last_clean":"2023-10-10T20:41:50.327445+0000","last_became_active":"2023-10-10T20:41:50.313773+0000","last_became_peered":"2023-10-10T20:41:50.313773+0000","last_unstale":"2023-10-10T20:41:50.327445+0000","last_undegraded":"2023-10-10T20:41:50.327445+0000","last_fullsized":"2023-10-10T20:41:50.327445+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:06.177167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329995+0000","last_change":"2023-10-10T20:41:50.326631+0000","last_active":"2023-10-10T20:41:50.329995+0000","last_peered":"2023-10-10T20:41:50.329995+0000","last_clean":"2023-10-10T20:41:50.329995+0000","last_became_active":"2023-10-10T20:41:50.323639+0000","last_became_peered":"2023-10-10T20:41:50.323639+0000","last_unstale":"2023-10-10T20:41:50.329995+0000","last_undegraded":"2023-10-10T20:41:50.329995+0000","last_fullsized":"2023-10-10T20:41:50.329995+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:37:21.016202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.315357+0000","last_change":"2023-10-10T20:41:50.311398+0000","last_active":"2023-10-10T20:41:50.315357+0000","last_peered":"2023-10-10T20:41:50.315357+0000","last_clean":"2023-10-10T20:41:50.315357+0000","last_became_active":"2023-10-10T20:41:50.311136+0000","last_became_peered":"2023-10-10T20:41:50.311136+0000","last_unstale":"2023-10-10T20:41:50.315357+0000","last_undegraded":"2023-10-10T20:41:50.315357+0000","last_fullsized":"2023-10-10T20:41:50.315357+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:51:24.010245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327174+0000","last_change":"2023-10-10T20:41:50.322755+0000","last_active":"2023-10-10T20:41:50.327174+0000","last_peered":"2023-10-10T20:41:50.327174+0000","last_clean":"2023-10-10T20:41:50.327174+0000","last_became_active":"2023-10-10T20:41:50.322514+0000","last_became_peered":"2023-10-10T20:41:50.322514+0000","last_unstale":"2023-10-10T20:41:50.327174+0000","last_undegraded":"2023-10-10T20:41:50.327174+0000","last_fullsized":"2023-10-10T20:41:50.327174+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:06.177167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329970+0000","last_change":"2023-10-10T20:41:50.326356+0000","last_active":"2023-10-10T20:41:50.329970+0000","last_peered":"2023-10-10T20:41:50.329970+0000","last_clean":"2023-10-10T20:41:50.329970+0000","last_became_active":"2023-10-10T20:41:50.326042+0000","last_became_peered":"2023-10-10T20:41:50.326042+0000","last_unstale":"2023-10-10T20:41:50.329970+0000","last_undegraded":"2023-10-10T20:41:50.329970+0000","last_fullsized":"2023-10-10T20:41:50.329970+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:36:22.310224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329474+0000","last_change":"2023-10-10T20:41:50.322516+0000","last_active":"2023-10-10T20:41:50.329474+0000","last_peered":"2023-10-10T20:41:50.329474+0000","last_clean":"2023-10-10T20:41:50.329474+0000","last_became_active":"2023-10-10T20:41:50.321967+0000","last_became_peered":"2023-10-10T20:41:50.321967+0000","last_unstale":"2023-10-10T20:41:50.329474+0000","last_undegraded":"2023-10-10T20:41:50.329474+0000","last_fullsized":"2023-10-10T20:41:50.329474+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:58.433900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.334853+0000","last_change":"2023-10-10T20:41:50.332271+0000","last_active":"2023-10-10T20:41:50.334853+0000","last_peered":"2023-10-10T20:41:50.334853+0000","last_clean":"2023-10-10T20:41:50.334853+0000","last_became_active":"2023-10-10T20:41:50.331819+0000","last_became_peered":"2023-10-10T20:41:50.331819+0000","last_unstale":"2023-10-10T20:41:50.334853+0000","last_undegraded":"2023-10-10T20:41:50.334853+0000","last_fullsized":"2023-10-10T20:41:50.334853+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:08:58.453242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328951+0000","last_change":"2023-10-10T20:41:50.325040+0000","last_active":"2023-10-10T20:41:50.328951+0000","last_peered":"2023-10-10T20:41:50.328951+0000","last_clean":"2023-10-10T20:41:50.328951+0000","last_became_active":"2023-10-10T20:41:50.324726+0000","last_became_peered":"2023-10-10T20:41:50.324726+0000","last_unstale":"2023-10-10T20:41:50.328951+0000","last_undegraded":"2023-10-10T20:41:50.328951+0000","last_fullsized":"2023-10-10T20:41:50.328951+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:29.171361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.330924+0000","last_change":"2023-10-10T20:41:50.321301+0000","last_active":"2023-10-10T20:41:50.330924+0000","last_peered":"2023-10-10T20:41:50.330924+0000","last_clean":"2023-10-10T20:41:50.330924+0000","last_became_active":"2023-10-10T20:41:50.320658+0000","last_became_peered":"2023-10-10T20:41:50.320658+0000","last_unstale":"2023-10-10T20:41:50.330924+0000","last_undegraded":"2023-10-10T20:41:50.330924+0000","last_fullsized":"2023-10-10T20:41:50.330924+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:36:51.689597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.325951+0000","last_change":"2023-10-10T20:41:50.322737+0000","last_active":"2023-10-10T20:41:50.325951+0000","last_peered":"2023-10-10T20:41:50.325951+0000","last_clean":"2023-10-10T20:41:50.325951+0000","last_became_active":"2023-10-10T20:41:50.322509+0000","last_became_peered":"2023-10-10T20:41:50.322509+0000","last_unstale":"2023-10-10T20:41:50.325951+0000","last_undegraded":"2023-10-10T20:41:50.325951+0000","last_fullsized":"2023-10-10T20:41:50.325951+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:49:46.940449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328624+0000","last_change":"2023-10-10T20:41:50.321194+0000","last_active":"2023-10-10T20:41:50.328624+0000","last_peered":"2023-10-10T20:41:50.328624+0000","last_clean":"2023-10-10T20:41:50.328624+0000","last_became_active":"2023-10-10T20:41:50.320546+0000","last_became_peered":"2023-10-10T20:41:50.320546+0000","last_unstale":"2023-10-10T20:41:50.328624+0000","last_undegraded":"2023-10-10T20:41:50.328624+0000","last_fullsized":"2023-10-10T20:41:50.328624+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:29.171361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.319678+0000","last_change":"2023-10-10T20:41:50.314660+0000","last_active":"2023-10-10T20:41:50.319678+0000","last_peered":"2023-10-10T20:41:50.319678+0000","last_clean":"2023-10-10T20:41:50.319678+0000","last_became_active":"2023-10-10T20:41:50.314414+0000","last_became_peered":"2023-10-10T20:41:50.314414+0000","last_unstale":"2023-10-10T20:41:50.319678+0000","last_undegraded":"2023-10-10T20:41:50.319678+0000","last_fullsized":"2023-10-10T20:41:50.319678+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:09:42.528347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329592+0000","last_change":"2023-10-10T20:41:50.325931+0000","last_active":"2023-10-10T20:41:50.329592+0000","last_peered":"2023-10-10T20:41:50.329592+0000","last_clean":"2023-10-10T20:41:50.329592+0000","last_became_active":"2023-10-10T20:41:50.325457+0000","last_became_peered":"2023-10-10T20:41:50.325457+0000","last_unstale":"2023-10-10T20:41:50.329592+0000","last_undegraded":"2023-10-10T20:41:50.329592+0000","last_fullsized":"2023-10-10T20:41:50.329592+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:57.360052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.324074+0000","last_change":"2023-10-10T20:41:50.319188+0000","last_active":"2023-10-10T20:41:50.324074+0000","last_peered":"2023-10-10T20:41:50.324074+0000","last_clean":"2023-10-10T20:41:50.324074+0000","last_became_active":"2023-10-10T20:41:50.318887+0000","last_became_peered":"2023-10-10T20:41:50.318887+0000","last_unstale":"2023-10-10T20:41:50.324074+0000","last_undegraded":"2023-10-10T20:41:50.324074+0000","last_fullsized":"2023-10-10T20:41:50.324074+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:38.872388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.315523+0000","last_change":"2023-10-10T20:41:50.310296+0000","last_active":"2023-10-10T20:41:50.315523+0000","last_peered":"2023-10-10T20:41:50.315523+0000","last_clean":"2023-10-10T20:41:50.315523+0000","last_became_active":"2023-10-10T20:41:50.309872+0000","last_became_peered":"2023-10-10T20:41:50.309872+0000","last_unstale":"2023-10-10T20:41:50.315523+0000","last_undegraded":"2023-10-10T20:41:50.315523+0000","last_fullsized":"2023-10-10T20:41:50.315523+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:08:57.143352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.325071+0000","last_change":"2023-10-10T20:41:50.320021+0000","last_active":"2023-10-10T20:41:50.325071+0000","last_peered":"2023-10-10T20:41:50.325071+0000","last_clean":"2023-10-10T20:41:50.325071+0000","last_became_active":"2023-10-10T20:41:50.319667+0000","last_became_peered":"2023-10-10T20:41:50.319667+0000","last_unstale":"2023-10-10T20:41:50.325071+0000","last_undegraded":"2023-10-10T20:41:50.325071+0000","last_fullsized":"2023-10-10T20:41:50.325071+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:09:58.004059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329518+0000","last_change":"2023-10-10T20:41:50.320084+0000","last_active":"2023-10-10T20:41:50.329518+0000","last_peered":"2023-10-10T20:41:50.329518+0000","last_clean":"2023-10-10T20:41:50.329518+0000","last_became_active":"2023-10-10T20:41:50.319816+0000","last_became_peered":"2023-10-10T20:41:50.319816+0000","last_unstale":"2023-10-10T20:41:50.329518+0000","last_undegraded":"2023-10-10T20:41:50.329518+0000","last_fullsized":"2023-10-10T20:41:50.329518+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:25.711425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327656+0000","last_change":"2023-10-10T20:41:50.318101+0000","last_active":"2023-10-10T20:41:50.327656+0000","last_peered":"2023-10-10T20:41:50.327656+0000","last_clean":"2023-10-10T20:41:50.327656+0000","last_became_active":"2023-10-10T20:41:50.317813+0000","last_became_peered":"2023-10-10T20:41:50.317813+0000","last_unstale":"2023-10-10T20:41:50.327656+0000","last_undegraded":"2023-10-10T20:41:50.327656+0000","last_fullsized":"2023-10-10T20:41:50.327656+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:45:42.233949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"137'4","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.326399+0000","last_change":"2023-10-10T20:41:49.316818+0000","last_active":"2023-10-10T20:41:50.326399+0000","last_peered":"2023-10-10T20:41:50.326399+0000","last_clean":"2023-10-10T20:41:50.326399+0000","last_became_active":"2023-10-10T20:41:49.315998+0000","last_became_peered":"2023-10-10T20:41:49.315998+0000","last_unstale":"2023-10-10T20:41:50.326399+0000","last_undegraded":"2023-10-10T20:41:50.326399+0000","last_fullsized":"2023-10-10T20:41:50.326399+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:23:47.101110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329947+0000","last_change":"2023-10-10T20:41:50.328491+0000","last_active":"2023-10-10T20:41:50.329947+0000","last_peered":"2023-10-10T20:41:50.329947+0000","last_clean":"2023-10-10T20:41:50.329947+0000","last_became_active":"2023-10-10T20:41:50.328230+0000","last_became_peered":"2023-10-10T20:41:50.328230+0000","last_unstale":"2023-10-10T20:41:50.329947+0000","last_undegraded":"2023-10-10T20:41:50.329947+0000","last_fullsized":"2023-10-10T20:41:50.329947+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:45:42.233949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"137'1","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.314284+0000","last_change":"2023-10-10T20:41:49.310393+0000","last_active":"2023-10-10T20:41:50.314284+0000","last_peered":"2023-10-10T20:41:50.314284+0000","last_clean":"2023-10-10T20:41:50.314284+0000","last_became_active":"2023-10-10T20:41:49.309899+0000","last_became_peered":"2023-10-10T20:41:49.309899+0000","last_unstale":"2023-10-10T20:41:50.314284+0000","last_undegraded":"2023-10-10T20:41:50.314284+0000","last_fullsized":"2023-10-10T20:41:50.314284+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:56.354284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329152+0000","last_change":"2023-10-10T20:41:50.323854+0000","last_active":"2023-10-10T20:41:50.329152+0000","last_peered":"2023-10-10T20:41:50.329152+0000","last_clean":"2023-10-10T20:41:50.329152+0000","last_became_active":"2023-10-10T20:41:50.323559+0000","last_became_peered":"2023-10-10T20:41:50.323559+0000","last_unstale":"2023-10-10T20:41:50.329152+0000","last_undegraded":"2023-10-10T20:41:50.329152+0000","last_fullsized":"2023-10-10T20:41:50.329152+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:59.707661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"137'2","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.325205+0000","last_change":"2023-10-10T20:41:49.316507+0000","last_active":"2023-10-10T20:41:50.325205+0000","last_peered":"2023-10-10T20:41:50.325205+0000","last_clean":"2023-10-10T20:41:50.325205+0000","last_became_active":"2023-10-10T20:41:49.316246+0000","last_became_peered":"2023-10-10T20:41:49.316246+0000","last_unstale":"2023-10-10T20:41:50.325205+0000","last_undegraded":"2023-10-10T20:41:50.325205+0000","last_fullsized":"2023-10-10T20:41:50.325205+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:59.824875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.321151+0000","last_change":"2023-10-10T20:41:50.316576+0000","last_active":"2023-10-10T20:41:50.321151+0000","last_peered":"2023-10-10T20:41:50.321151+0000","last_clean":"2023-10-10T20:41:50.321151+0000","last_became_active":"2023-10-10T20:41:50.316279+0000","last_became_peered":"2023-10-10T20:41:50.316279+0000","last_unstale":"2023-10-10T20:41:50.321151+0000","last_undegraded":"2023-10-10T20:41:50.321151+0000","last_fullsized":"2023-10-10T20:41:50.321151+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:33:19.443146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"137'2","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.319554+0000","last_change":"2023-10-10T20:41:49.314721+0000","last_active":"2023-10-10T20:41:50.319554+0000","last_peered":"2023-10-10T20:41:50.319554+0000","last_clean":"2023-10-10T20:41:50.319554+0000","last_became_active":"2023-10-10T20:41:49.312771+0000","last_became_peered":"2023-10-10T20:41:49.312771+0000","last_unstale":"2023-10-10T20:41:50.319554+0000","last_undegraded":"2023-10-10T20:41:50.319554+0000","last_fullsized":"2023-10-10T20:41:50.319554+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:56:14.560628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.334834+0000","last_change":"2023-10-10T20:41:50.332094+0000","last_active":"2023-10-10T20:41:50.334834+0000","last_peered":"2023-10-10T20:41:50.334834+0000","last_clean":"2023-10-10T20:41:50.334834+0000","last_became_active":"2023-10-10T20:41:50.331523+0000","last_became_peered":"2023-10-10T20:41:50.331523+0000","last_unstale":"2023-10-10T20:41:50.334834+0000","last_undegraded":"2023-10-10T20:41:50.334834+0000","last_fullsized":"2023-10-10T20:41:50.334834+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:36:55.761497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"137'1","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327568+0000","last_change":"2023-10-10T20:41:49.315617+0000","last_active":"2023-10-10T20:41:50.327568+0000","last_peered":"2023-10-10T20:41:50.327568+0000","last_clean":"2023-10-10T20:41:50.327568+0000","last_became_active":"2023-10-10T20:41:49.315233+0000","last_became_peered":"2023-10-10T20:41:49.315233+0000","last_unstale":"2023-10-10T20:41:50.327568+0000","last_undegraded":"2023-10-10T20:41:50.327568+0000","last_fullsized":"2023-10-10T20:41:50.327568+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:41:47.663214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328784+0000","last_change":"2023-10-10T20:41:50.322989+0000","last_active":"2023-10-10T20:41:50.328784+0000","last_peered":"2023-10-10T20:41:50.328784+0000","last_clean":"2023-10-10T20:41:50.328784+0000","last_became_active":"2023-10-10T20:41:50.320266+0000","last_became_peered":"2023-10-10T20:41:50.320266+0000","last_unstale":"2023-10-10T20:41:50.328784+0000","last_undegraded":"2023-10-10T20:41:50.328784+0000","last_fullsized":"2023-10-10T20:41:50.328784+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:22:41.479243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"137'1","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.321165+0000","last_change":"2023-10-10T20:41:49.308749+0000","last_active":"2023-10-10T20:41:50.321165+0000","last_peered":"2023-10-10T20:41:50.321165+0000","last_clean":"2023-10-10T20:41:50.321165+0000","last_became_active":"2023-10-10T20:41:49.307933+0000","last_became_peered":"2023-10-10T20:41:49.307933+0000","last_unstale":"2023-10-10T20:41:50.321165+0000","last_undegraded":"2023-10-10T20:41:50.321165+0000","last_fullsized":"2023-10-10T20:41:50.321165+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:30:49.584095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.316607+0000","last_change":"2023-10-10T20:41:50.311319+0000","last_active":"2023-10-10T20:41:50.316607+0000","last_peered":"2023-10-10T20:41:50.316607+0000","last_clean":"2023-10-10T20:41:50.316607+0000","last_became_active":"2023-10-10T20:41:50.310519+0000","last_became_peered":"2023-10-10T20:41:50.310519+0000","last_unstale":"2023-10-10T20:41:50.316607+0000","last_undegraded":"2023-10-10T20:41:50.316607+0000","last_fullsized":"2023-10-10T20:41:50.316607+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:25.142717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.301856+0000","last_change":"2023-10-10T20:41:49.315721+0000","last_active":"2023-10-10T20:41:50.301856+0000","last_peered":"2023-10-10T20:41:50.301856+0000","last_clean":"2023-10-10T20:41:50.301856+0000","last_became_active":"2023-10-10T20:41:49.315438+0000","last_became_peered":"2023-10-10T20:41:49.315438+0000","last_unstale":"2023-10-10T20:41:50.301856+0000","last_undegraded":"2023-10-10T20:41:50.301856+0000","last_fullsized":"2023-10-10T20:41:50.301856+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:17:06.508904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.332154+0000","last_change":"2023-10-10T20:41:50.329023+0000","last_active":"2023-10-10T20:41:50.332154+0000","last_peered":"2023-10-10T20:41:50.332154+0000","last_clean":"2023-10-10T20:41:50.332154+0000","last_became_active":"2023-10-10T20:41:50.328773+0000","last_became_peered":"2023-10-10T20:41:50.328773+0000","last_unstale":"2023-10-10T20:41:50.332154+0000","last_undegraded":"2023-10-10T20:41:50.332154+0000","last_fullsized":"2023-10-10T20:41:50.332154+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:37:14.358493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.302232+0000","last_change":"2023-10-10T20:41:49.310044+0000","last_active":"2023-10-10T20:41:50.302232+0000","last_peered":"2023-10-10T20:41:50.302232+0000","last_clean":"2023-10-10T20:41:50.302232+0000","last_became_active":"2023-10-10T20:41:49.309759+0000","last_became_peered":"2023-10-10T20:41:49.309759+0000","last_unstale":"2023-10-10T20:41:50.302232+0000","last_undegraded":"2023-10-10T20:41:50.302232+0000","last_fullsized":"2023-10-10T20:41:50.302232+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:30:13.642642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328695+0000","last_change":"2023-10-10T20:41:50.323785+0000","last_active":"2023-10-10T20:41:50.328695+0000","last_peered":"2023-10-10T20:41:50.328695+0000","last_clean":"2023-10-10T20:41:50.328695+0000","last_became_active":"2023-10-10T20:41:50.323571+0000","last_became_peered":"2023-10-10T20:41:50.323571+0000","last_unstale":"2023-10-10T20:41:50.328695+0000","last_undegraded":"2023-10-10T20:41:50.328695+0000","last_fullsized":"2023-10-10T20:41:50.328695+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:58:22.113298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.310448+0000","last_change":"2023-10-10T20:41:49.317302+0000","last_active":"2023-10-10T20:41:50.310448+0000","last_peered":"2023-10-10T20:41:50.310448+0000","last_clean":"2023-10-10T20:41:50.310448+0000","last_became_active":"2023-10-10T20:41:49.317066+0000","last_became_peered":"2023-10-10T20:41:49.317066+0000","last_unstale":"2023-10-10T20:41:50.310448+0000","last_undegraded":"2023-10-10T20:41:50.310448+0000","last_fullsized":"2023-10-10T20:41:50.310448+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:41:24.307438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.317448+0000","last_change":"2023-10-10T20:41:50.311080+0000","last_active":"2023-10-10T20:41:50.317448+0000","last_peered":"2023-10-10T20:41:50.317448+0000","last_clean":"2023-10-10T20:41:50.317448+0000","last_became_active":"2023-10-10T20:41:50.310822+0000","last_became_peered":"2023-10-10T20:41:50.310822+0000","last_unstale":"2023-10-10T20:41:50.317448+0000","last_undegraded":"2023-10-10T20:41:50.317448+0000","last_fullsized":"2023-10-10T20:41:50.317448+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:09.931645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"137'3","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328903+0000","last_change":"2023-10-10T20:41:49.315940+0000","last_active":"2023-10-10T20:41:50.328903+0000","last_peered":"2023-10-10T20:41:50.328903+0000","last_clean":"2023-10-10T20:41:50.328903+0000","last_became_active":"2023-10-10T20:41:49.315620+0000","last_became_peered":"2023-10-10T20:41:49.315620+0000","last_unstale":"2023-10-10T20:41:50.328903+0000","last_undegraded":"2023-10-10T20:41:50.328903+0000","last_fullsized":"2023-10-10T20:41:50.328903+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:29:06.709268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.332378+0000","last_change":"2023-10-10T20:41:50.331754+0000","last_active":"2023-10-10T20:41:50.332378+0000","last_peered":"2023-10-10T20:41:50.332378+0000","last_clean":"2023-10-10T20:41:50.332378+0000","last_became_active":"2023-10-10T20:41:50.331492+0000","last_became_peered":"2023-10-10T20:41:50.331492+0000","last_unstale":"2023-10-10T20:41:50.332378+0000","last_undegraded":"2023-10-10T20:41:50.332378+0000","last_fullsized":"2023-10-10T20:41:50.332378+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:58:22.113298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"137'6","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329762+0000","last_change":"2023-10-10T20:41:49.307503+0000","last_active":"2023-10-10T20:41:50.329762+0000","last_peered":"2023-10-10T20:41:50.329762+0000","last_clean":"2023-10-10T20:41:50.329762+0000","last_became_active":"2023-10-10T20:41:49.307188+0000","last_became_peered":"2023-10-10T20:41:49.307188+0000","last_unstale":"2023-10-10T20:41:50.329762+0000","last_undegraded":"2023-10-10T20:41:50.329762+0000","last_fullsized":"2023-10-10T20:41:50.329762+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:22.143245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.325048+0000","last_change":"2023-10-10T20:41:50.320264+0000","last_active":"2023-10-10T20:41:50.325048+0000","last_peered":"2023-10-10T20:41:50.325048+0000","last_clean":"2023-10-10T20:41:50.325048+0000","last_became_active":"2023-10-10T20:41:50.320001+0000","last_became_peered":"2023-10-10T20:41:50.320001+0000","last_unstale":"2023-10-10T20:41:50.325048+0000","last_undegraded":"2023-10-10T20:41:50.325048+0000","last_fullsized":"2023-10-10T20:41:50.325048+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:59.707661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"137'2","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327672+0000","last_change":"2023-10-10T20:41:49.314907+0000","last_active":"2023-10-10T20:41:50.327672+0000","last_peered":"2023-10-10T20:41:50.327672+0000","last_clean":"2023-10-10T20:41:50.327672+0000","last_became_active":"2023-10-10T20:41:49.314607+0000","last_became_peered":"2023-10-10T20:41:49.314607+0000","last_unstale":"2023-10-10T20:41:50.327672+0000","last_undegraded":"2023-10-10T20:41:50.327672+0000","last_fullsized":"2023-10-10T20:41:50.327672+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:04:21.204106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.325850+0000","last_change":"2023-10-10T20:41:50.320973+0000","last_active":"2023-10-10T20:41:50.325850+0000","last_peered":"2023-10-10T20:41:50.325850+0000","last_clean":"2023-10-10T20:41:50.325850+0000","last_became_active":"2023-10-10T20:41:50.320708+0000","last_became_peered":"2023-10-10T20:41:50.320708+0000","last_unstale":"2023-10-10T20:41:50.325850+0000","last_undegraded":"2023-10-10T20:41:50.325850+0000","last_fullsized":"2023-10-10T20:41:50.325850+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:09:42.528347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.312504+0000","last_change":"2023-10-10T20:41:49.313210+0000","last_active":"2023-10-10T20:41:50.312504+0000","last_peered":"2023-10-10T20:41:50.312504+0000","last_clean":"2023-10-10T20:41:50.312504+0000","last_became_active":"2023-10-10T20:41:49.312963+0000","last_became_peered":"2023-10-10T20:41:49.312963+0000","last_unstale":"2023-10-10T20:41:50.312504+0000","last_undegraded":"2023-10-10T20:41:50.312504+0000","last_fullsized":"2023-10-10T20:41:50.312504+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:52.358088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.326056+0000","last_change":"2023-10-10T20:41:50.322767+0000","last_active":"2023-10-10T20:41:50.326056+0000","last_peered":"2023-10-10T20:41:50.326056+0000","last_clean":"2023-10-10T20:41:50.326056+0000","last_became_active":"2023-10-10T20:41:50.322502+0000","last_became_peered":"2023-10-10T20:41:50.322502+0000","last_unstale":"2023-10-10T20:41:50.326056+0000","last_undegraded":"2023-10-10T20:41:50.326056+0000","last_fullsized":"2023-10-10T20:41:50.326056+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:15:18.258032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.306854+0000","last_change":"2023-10-10T20:41:49.317998+0000","last_active":"2023-10-10T20:41:50.306854+0000","last_peered":"2023-10-10T20:41:50.306854+0000","last_clean":"2023-10-10T20:41:50.306854+0000","last_became_active":"2023-10-10T20:41:49.317703+0000","last_became_peered":"2023-10-10T20:41:49.317703+0000","last_unstale":"2023-10-10T20:41:50.306854+0000","last_undegraded":"2023-10-10T20:41:50.306854+0000","last_fullsized":"2023-10-10T20:41:50.306854+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:39:49.147457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.320938+0000","last_change":"2023-10-10T20:41:50.317758+0000","last_active":"2023-10-10T20:41:50.320938+0000","last_peered":"2023-10-10T20:41:50.320938+0000","last_clean":"2023-10-10T20:41:50.320938+0000","last_became_active":"2023-10-10T20:41:50.317500+0000","last_became_peered":"2023-10-10T20:41:50.317500+0000","last_unstale":"2023-10-10T20:41:50.320938+0000","last_undegraded":"2023-10-10T20:41:50.320938+0000","last_fullsized":"2023-10-10T20:41:50.320938+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:22:41.479243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"137'8","reported_seq":18,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328528+0000","last_change":"2023-10-10T20:41:49.316006+0000","last_active":"2023-10-10T20:41:50.328528+0000","last_peered":"2023-10-10T20:41:50.328528+0000","last_clean":"2023-10-10T20:41:50.328528+0000","last_became_active":"2023-10-10T20:41:49.315687+0000","last_became_peered":"2023-10-10T20:41:49.315687+0000","last_unstale":"2023-10-10T20:41:50.328528+0000","last_undegraded":"2023-10-10T20:41:50.328528+0000","last_fullsized":"2023-10-10T20:41:50.328528+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:09:56.998292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327185+0000","last_change":"2023-10-10T20:41:50.322343+0000","last_active":"2023-10-10T20:41:50.327185+0000","last_peered":"2023-10-10T20:41:50.327185+0000","last_clean":"2023-10-10T20:41:50.327185+0000","last_became_active":"2023-10-10T20:41:50.322089+0000","last_became_peered":"2023-10-10T20:41:50.322089+0000","last_unstale":"2023-10-10T20:41:50.327185+0000","last_undegraded":"2023-10-10T20:41:50.327185+0000","last_fullsized":"2023-10-10T20:41:50.327185+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:28:58.113040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"137'1","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329391+0000","last_change":"2023-10-10T20:41:49.316983+0000","last_active":"2023-10-10T20:41:50.329391+0000","last_peered":"2023-10-10T20:41:50.329391+0000","last_clean":"2023-10-10T20:41:50.329391+0000","last_became_active":"2023-10-10T20:41:49.316318+0000","last_became_peered":"2023-10-10T20:41:49.316318+0000","last_unstale":"2023-10-10T20:41:50.329391+0000","last_undegraded":"2023-10-10T20:41:50.329391+0000","last_fullsized":"2023-10-10T20:41:50.329391+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:35:39.211243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.326095+0000","last_change":"2023-10-10T20:41:50.323324+0000","last_active":"2023-10-10T20:41:50.326095+0000","last_peered":"2023-10-10T20:41:50.326095+0000","last_clean":"2023-10-10T20:41:50.326095+0000","last_became_active":"2023-10-10T20:41:50.323065+0000","last_became_peered":"2023-10-10T20:41:50.323065+0000","last_unstale":"2023-10-10T20:41:50.326095+0000","last_undegraded":"2023-10-10T20:41:50.326095+0000","last_fullsized":"2023-10-10T20:41:50.326095+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:44.651808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"137'2","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.326104+0000","last_change":"2023-10-10T20:41:49.308012+0000","last_active":"2023-10-10T20:41:50.326104+0000","last_peered":"2023-10-10T20:41:50.326104+0000","last_clean":"2023-10-10T20:41:50.326104+0000","last_became_active":"2023-10-10T20:41:49.307712+0000","last_became_peered":"2023-10-10T20:41:49.307712+0000","last_unstale":"2023-10-10T20:41:50.326104+0000","last_undegraded":"2023-10-10T20:41:50.326104+0000","last_fullsized":"2023-10-10T20:41:50.326104+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:34:34.956156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.315166+0000","last_change":"2023-10-10T20:41:50.309435+0000","last_active":"2023-10-10T20:41:50.315166+0000","last_peered":"2023-10-10T20:41:50.315166+0000","last_clean":"2023-10-10T20:41:50.315166+0000","last_became_active":"2023-10-10T20:41:50.309084+0000","last_became_peered":"2023-10-10T20:41:50.309084+0000","last_unstale":"2023-10-10T20:41:50.315166+0000","last_undegraded":"2023-10-10T20:41:50.315166+0000","last_fullsized":"2023-10-10T20:41:50.315166+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:09:21.122158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"137'1","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327050+0000","last_change":"2023-10-10T20:41:49.314806+0000","last_active":"2023-10-10T20:41:50.327050+0000","last_peered":"2023-10-10T20:41:50.327050+0000","last_clean":"2023-10-10T20:41:50.327050+0000","last_became_active":"2023-10-10T20:41:49.314532+0000","last_became_peered":"2023-10-10T20:41:49.314532+0000","last_unstale":"2023-10-10T20:41:50.327050+0000","last_undegraded":"2023-10-10T20:41:50.327050+0000","last_fullsized":"2023-10-10T20:41:50.327050+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:43:37.542864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.332248+0000","last_change":"2023-10-10T20:41:50.329246+0000","last_active":"2023-10-10T20:41:50.332248+0000","last_peered":"2023-10-10T20:41:50.332248+0000","last_clean":"2023-10-10T20:41:50.332248+0000","last_became_active":"2023-10-10T20:41:50.328957+0000","last_became_peered":"2023-10-10T20:41:50.328957+0000","last_unstale":"2023-10-10T20:41:50.332248+0000","last_undegraded":"2023-10-10T20:41:50.332248+0000","last_fullsized":"2023-10-10T20:41:50.332248+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:24:36.913433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.305148+0000","last_change":"2023-10-10T20:41:49.312016+0000","last_active":"2023-10-10T20:41:50.305148+0000","last_peered":"2023-10-10T20:41:50.305148+0000","last_clean":"2023-10-10T20:41:50.305148+0000","last_became_active":"2023-10-10T20:41:49.311737+0000","last_became_peered":"2023-10-10T20:41:49.311737+0000","last_unstale":"2023-10-10T20:41:50.305148+0000","last_undegraded":"2023-10-10T20:41:50.305148+0000","last_fullsized":"2023-10-10T20:41:50.305148+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:40:21.122555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327289+0000","last_change":"2023-10-10T20:41:50.324679+0000","last_active":"2023-10-10T20:41:50.327289+0000","last_peered":"2023-10-10T20:41:50.327289+0000","last_clean":"2023-10-10T20:41:50.327289+0000","last_became_active":"2023-10-10T20:41:50.324434+0000","last_became_peered":"2023-10-10T20:41:50.324434+0000","last_unstale":"2023-10-10T20:41:50.327289+0000","last_undegraded":"2023-10-10T20:41:50.327289+0000","last_fullsized":"2023-10-10T20:41:50.327289+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:21.594296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"137'2","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329850+0000","last_change":"2023-10-10T20:41:49.318176+0000","last_active":"2023-10-10T20:41:50.329850+0000","last_peered":"2023-10-10T20:41:50.329850+0000","last_clean":"2023-10-10T20:41:50.329850+0000","last_became_active":"2023-10-10T20:41:49.317925+0000","last_became_peered":"2023-10-10T20:41:49.317925+0000","last_unstale":"2023-10-10T20:41:50.329850+0000","last_undegraded":"2023-10-10T20:41:50.329850+0000","last_fullsized":"2023-10-10T20:41:50.329850+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:24.136950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329156+0000","last_change":"2023-10-10T20:41:50.324209+0000","last_active":"2023-10-10T20:41:50.329156+0000","last_peered":"2023-10-10T20:41:50.329156+0000","last_clean":"2023-10-10T20:41:50.329156+0000","last_became_active":"2023-10-10T20:41:50.323891+0000","last_became_peered":"2023-10-10T20:41:50.323891+0000","last_unstale":"2023-10-10T20:41:50.329156+0000","last_undegraded":"2023-10-10T20:41:50.329156+0000","last_fullsized":"2023-10-10T20:41:50.329156+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:28:06.266010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.308876+0000","last_change":"2023-10-10T20:41:49.317374+0000","last_active":"2023-10-10T20:41:50.308876+0000","last_peered":"2023-10-10T20:41:50.308876+0000","last_clean":"2023-10-10T20:41:50.308876+0000","last_became_active":"2023-10-10T20:41:49.317022+0000","last_became_peered":"2023-10-10T20:41:49.317022+0000","last_unstale":"2023-10-10T20:41:50.308876+0000","last_undegraded":"2023-10-10T20:41:50.308876+0000","last_fullsized":"2023-10-10T20:41:50.308876+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:44.809357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.324040+0000","last_change":"2023-10-10T20:41:50.320161+0000","last_active":"2023-10-10T20:41:50.324040+0000","last_peered":"2023-10-10T20:41:50.324040+0000","last_clean":"2023-10-10T20:41:50.324040+0000","last_became_active":"2023-10-10T20:41:50.319896+0000","last_became_peered":"2023-10-10T20:41:50.319896+0000","last_unstale":"2023-10-10T20:41:50.324040+0000","last_undegraded":"2023-10-10T20:41:50.324040+0000","last_fullsized":"2023-10-10T20:41:50.324040+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:28:06.266010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.310033+0000","last_change":"2023-10-10T20:41:49.318718+0000","last_active":"2023-10-10T20:41:50.310033+0000","last_peered":"2023-10-10T20:41:50.310033+0000","last_clean":"2023-10-10T20:41:50.310033+0000","last_became_active":"2023-10-10T20:41:49.318480+0000","last_became_peered":"2023-10-10T20:41:49.318480+0000","last_unstale":"2023-10-10T20:41:50.310033+0000","last_undegraded":"2023-10-10T20:41:50.310033+0000","last_fullsized":"2023-10-10T20:41:50.310033+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:57.541968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327533+0000","last_change":"2023-10-10T20:41:50.321470+0000","last_active":"2023-10-10T20:41:50.327533+0000","last_peered":"2023-10-10T20:41:50.327533+0000","last_clean":"2023-10-10T20:41:50.327533+0000","last_became_active":"2023-10-10T20:41:50.315093+0000","last_became_peered":"2023-10-10T20:41:50.315093+0000","last_unstale":"2023-10-10T20:41:50.327533+0000","last_undegraded":"2023-10-10T20:41:50.327533+0000","last_fullsized":"2023-10-10T20:41:50.327533+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:42:13.861221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.302813+0000","last_change":"2023-10-10T20:41:49.314130+0000","last_active":"2023-10-10T20:41:50.302813+0000","last_peered":"2023-10-10T20:41:50.302813+0000","last_clean":"2023-10-10T20:41:50.302813+0000","last_became_active":"2023-10-10T20:41:49.313882+0000","last_became_peered":"2023-10-10T20:41:49.313882+0000","last_unstale":"2023-10-10T20:41:50.302813+0000","last_undegraded":"2023-10-10T20:41:50.302813+0000","last_fullsized":"2023-10-10T20:41:50.302813+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:51:13.831963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.323070+0000","last_change":"2023-10-10T20:41:50.313028+0000","last_active":"2023-10-10T20:41:50.323070+0000","last_peered":"2023-10-10T20:41:50.323070+0000","last_clean":"2023-10-10T20:41:50.323070+0000","last_became_active":"2023-10-10T20:41:50.312757+0000","last_became_peered":"2023-10-10T20:41:50.312757+0000","last_unstale":"2023-10-10T20:41:50.323070+0000","last_undegraded":"2023-10-10T20:41:50.323070+0000","last_fullsized":"2023-10-10T20:41:50.323070+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:51:14.837731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"137'2","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.319856+0000","last_change":"2023-10-10T20:41:49.318208+0000","last_active":"2023-10-10T20:41:50.319856+0000","last_peered":"2023-10-10T20:41:50.319856+0000","last_clean":"2023-10-10T20:41:50.319856+0000","last_became_active":"2023-10-10T20:41:49.317937+0000","last_became_peered":"2023-10-10T20:41:49.317937+0000","last_unstale":"2023-10-10T20:41:50.319856+0000","last_undegraded":"2023-10-10T20:41:50.319856+0000","last_fullsized":"2023-10-10T20:41:50.319856+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:37:13.352726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.326844+0000","last_change":"2023-10-10T20:41:50.321622+0000","last_active":"2023-10-10T20:41:50.326844+0000","last_peered":"2023-10-10T20:41:50.326844+0000","last_clean":"2023-10-10T20:41:50.326844+0000","last_became_active":"2023-10-10T20:41:50.321200+0000","last_became_peered":"2023-10-10T20:41:50.321200+0000","last_unstale":"2023-10-10T20:41:50.326844+0000","last_undegraded":"2023-10-10T20:41:50.326844+0000","last_fullsized":"2023-10-10T20:41:50.326844+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:33.972875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.314099+0000","last_change":"2023-10-10T20:41:49.315072+0000","last_active":"2023-10-10T20:41:50.314099+0000","last_peered":"2023-10-10T20:41:50.314099+0000","last_clean":"2023-10-10T20:41:50.314099+0000","last_became_active":"2023-10-10T20:41:49.314618+0000","last_became_peered":"2023-10-10T20:41:49.314618+0000","last_unstale":"2023-10-10T20:41:50.314099+0000","last_undegraded":"2023-10-10T20:41:50.314099+0000","last_fullsized":"2023-10-10T20:41:50.314099+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:04:21.204106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328741+0000","last_change":"2023-10-10T20:41:50.321375+0000","last_active":"2023-10-10T20:41:50.328741+0000","last_peered":"2023-10-10T20:41:50.328741+0000","last_clean":"2023-10-10T20:41:50.328741+0000","last_became_active":"2023-10-10T20:41:50.321099+0000","last_became_peered":"2023-10-10T20:41:50.321099+0000","last_unstale":"2023-10-10T20:41:50.328741+0000","last_undegraded":"2023-10-10T20:41:50.328741+0000","last_fullsized":"2023-10-10T20:41:50.328741+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:21.594296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.315641+0000","last_change":"2023-10-10T20:41:49.310539+0000","last_active":"2023-10-10T20:41:50.315641+0000","last_peered":"2023-10-10T20:41:50.315641+0000","last_clean":"2023-10-10T20:41:50.315641+0000","last_became_active":"2023-10-10T20:41:49.310219+0000","last_became_peered":"2023-10-10T20:41:49.310219+0000","last_unstale":"2023-10-10T20:41:50.315641+0000","last_undegraded":"2023-10-10T20:41:50.315641+0000","last_fullsized":"2023-10-10T20:41:50.315641+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:43:37.542864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327153+0000","last_change":"2023-10-10T20:41:50.318523+0000","last_active":"2023-10-10T20:41:50.327153+0000","last_peered":"2023-10-10T20:41:50.327153+0000","last_clean":"2023-10-10T20:41:50.327153+0000","last_became_active":"2023-10-10T20:41:50.318341+0000","last_became_peered":"2023-10-10T20:41:50.318341+0000","last_unstale":"2023-10-10T20:41:50.327153+0000","last_undegraded":"2023-10-10T20:41:50.327153+0000","last_fullsized":"2023-10-10T20:41:50.327153+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:36:55.761497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.308130+0000","last_change":"2023-10-10T20:41:49.308240+0000","last_active":"2023-10-10T20:41:50.308130+0000","last_peered":"2023-10-10T20:41:50.308130+0000","last_clean":"2023-10-10T20:41:50.308130+0000","last_became_active":"2023-10-10T20:41:49.307687+0000","last_became_peered":"2023-10-10T20:41:49.307687+0000","last_unstale":"2023-10-10T20:41:50.308130+0000","last_undegraded":"2023-10-10T20:41:50.308130+0000","last_fullsized":"2023-10-10T20:41:50.308130+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:22.420406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.313751+0000","last_change":"2023-10-10T20:41:50.307863+0000","last_active":"2023-10-10T20:41:50.313751+0000","last_peered":"2023-10-10T20:41:50.313751+0000","last_clean":"2023-10-10T20:41:50.313751+0000","last_became_active":"2023-10-10T20:41:50.307587+0000","last_became_peered":"2023-10-10T20:41:50.307587+0000","last_unstale":"2023-10-10T20:41:50.313751+0000","last_undegraded":"2023-10-10T20:41:50.313751+0000","last_fullsized":"2023-10-10T20:41:50.313751+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:23:01.318341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.302594+0000","last_change":"2023-10-10T20:41:49.313676+0000","last_active":"2023-10-10T20:41:50.302594+0000","last_peered":"2023-10-10T20:41:50.302594+0000","last_clean":"2023-10-10T20:41:50.302594+0000","last_became_active":"2023-10-10T20:41:49.313425+0000","last_became_peered":"2023-10-10T20:41:49.313425+0000","last_unstale":"2023-10-10T20:41:50.302594+0000","last_undegraded":"2023-10-10T20:41:50.302594+0000","last_fullsized":"2023-10-10T20:41:50.302594+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:02.015895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.318356+0000","last_change":"2023-10-10T20:41:50.315515+0000","last_active":"2023-10-10T20:41:50.318356+0000","last_peered":"2023-10-10T20:41:50.318356+0000","last_clean":"2023-10-10T20:41:50.318356+0000","last_became_active":"2023-10-10T20:41:50.314937+0000","last_became_peered":"2023-10-10T20:41:50.314937+0000","last_unstale":"2023-10-10T20:41:50.318356+0000","last_undegraded":"2023-10-10T20:41:50.318356+0000","last_fullsized":"2023-10-10T20:41:50.318356+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:23.426173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.302835+0000","last_change":"2023-10-10T20:41:49.313069+0000","last_active":"2023-10-10T20:41:50.302835+0000","last_peered":"2023-10-10T20:41:50.302835+0000","last_clean":"2023-10-10T20:41:50.302835+0000","last_became_active":"2023-10-10T20:41:49.312729+0000","last_became_peered":"2023-10-10T20:41:49.312729+0000","last_unstale":"2023-10-10T20:41:50.302835+0000","last_undegraded":"2023-10-10T20:41:50.302835+0000","last_fullsized":"2023-10-10T20:41:50.302835+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:42:12.855453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.323987+0000","last_change":"2023-10-10T20:41:50.319309+0000","last_active":"2023-10-10T20:41:50.323987+0000","last_peered":"2023-10-10T20:41:50.323987+0000","last_clean":"2023-10-10T20:41:50.323987+0000","last_became_active":"2023-10-10T20:41:50.318090+0000","last_became_peered":"2023-10-10T20:41:50.318090+0000","last_unstale":"2023-10-10T20:41:50.323987+0000","last_undegraded":"2023-10-10T20:41:50.323987+0000","last_fullsized":"2023-10-10T20:41:50.323987+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:09.931645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.310889+0000","last_change":"2023-10-10T20:41:49.317299+0000","last_active":"2023-10-10T20:41:50.310889+0000","last_peered":"2023-10-10T20:41:50.310889+0000","last_clean":"2023-10-10T20:41:50.310889+0000","last_became_active":"2023-10-10T20:41:49.316536+0000","last_became_peered":"2023-10-10T20:41:49.316536+0000","last_unstale":"2023-10-10T20:41:50.310889+0000","last_undegraded":"2023-10-10T20:41:50.310889+0000","last_fullsized":"2023-10-10T20:41:50.310889+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:24.814273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329532+0000","last_change":"2023-10-10T20:41:50.324946+0000","last_active":"2023-10-10T20:41:50.329532+0000","last_peered":"2023-10-10T20:41:50.329532+0000","last_clean":"2023-10-10T20:41:50.329532+0000","last_became_active":"2023-10-10T20:41:50.324655+0000","last_became_peered":"2023-10-10T20:41:50.324655+0000","last_unstale":"2023-10-10T20:41:50.329532+0000","last_undegraded":"2023-10-10T20:41:50.329532+0000","last_fullsized":"2023-10-10T20:41:50.329532+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:02.059224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.309870+0000","last_change":"2023-10-10T20:41:49.312704+0000","last_active":"2023-10-10T20:41:50.309870+0000","last_peered":"2023-10-10T20:41:50.309870+0000","last_clean":"2023-10-10T20:41:50.309870+0000","last_became_active":"2023-10-10T20:41:49.312386+0000","last_became_peered":"2023-10-10T20:41:49.312386+0000","last_unstale":"2023-10-10T20:41:50.309870+0000","last_undegraded":"2023-10-10T20:41:50.309870+0000","last_fullsized":"2023-10-10T20:41:50.309870+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:10:58.865055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327147+0000","last_change":"2023-10-10T20:41:50.325233+0000","last_active":"2023-10-10T20:41:50.327147+0000","last_peered":"2023-10-10T20:41:50.327147+0000","last_clean":"2023-10-10T20:41:50.327147+0000","last_became_active":"2023-10-10T20:41:50.324824+0000","last_became_peered":"2023-10-10T20:41:50.324824+0000","last_unstale":"2023-10-10T20:41:50.327147+0000","last_undegraded":"2023-10-10T20:41:50.327147+0000","last_fullsized":"2023-10-10T20:41:50.327147+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:36:51.689597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.303274+0000","last_change":"2023-10-10T20:41:49.317251+0000","last_active":"2023-10-10T20:41:50.303274+0000","last_peered":"2023-10-10T20:41:50.303274+0000","last_clean":"2023-10-10T20:41:50.303274+0000","last_became_active":"2023-10-10T20:41:49.316927+0000","last_became_peered":"2023-10-10T20:41:49.316927+0000","last_unstale":"2023-10-10T20:41:50.303274+0000","last_undegraded":"2023-10-10T20:41:50.303274+0000","last_fullsized":"2023-10-10T20:41:50.303274+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:52.358088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":281,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.303736+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:41:50.303736+0000","last_peered":"2023-10-10T20:41:50.303736+0000","last_clean":"2023-10-10T20:41:50.303736+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:41:50.303736+0000","last_undegraded":"2023-10-10T20:41:50.303736+0000","last_fullsized":"2023-10-10T20:41:50.303736+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303676,"kb_used_data":2292,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068164,"statfs":{"total":96636764160,"available":96325799936,"internally_reserved":0,"allocated":2347008,"data_stored":1371439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.79000000000000004},"min":{"1min":0.309,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71799999999999997,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52400000000000002,"15min":0.61599999999999999},"min":{"1min":0.32300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.69999999999999996,"5min":0.82099999999999995,"15min":4.1959999999999997},"last":0.56399999999999995}]},{"osd":1,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.95099999999999996},"min":{"1min":0.379,"5min":0.317,"15min":0.317},"max":{"1min":0.78300000000000003,"5min":0.78800000000000003,"15min":10.1},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55200000000000005,"15min":0.71399999999999997},"min":{"1min":0.36699999999999999,"5min":0.32300000000000001,"15min":0.28999999999999998},"max":{"1min":0.73199999999999998,"5min":0.90400000000000003,"15min":6.806},"last":0.46500000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.76400000000000001},"min":{"1min":0.40100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79200000000000004,"5min":0.89000000000000001,"15min":6.9340000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.71699999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.73299999999999998,"5min":0.83899999999999997,"15min":6.9870000000000001},"last":0.433}]},{"osd":3,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54900000000000004,"15min":0.73599999999999999},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.753,"5min":0.80200000000000005,"15min":6.6719999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56699999999999995,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.71699999999999997,"5min":0.87,"15min":4.1609999999999996},"last":0.498}]},{"osd":4,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.36599999999999999,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.65100000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36299999999999999,"5min":0.309,"15min":0.30399999999999999},"max":{"1min":0.72299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38500000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.314},"max":{"1min":0.77900000000000003,"5min":0.86099999999999999,"15min":1.3180000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.41799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71499999999999997,"5min":0.75800000000000001,"15min":0.84099999999999997},"last":0.40500000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":0.78700000000000003,"15min":1.7509999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.40899999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.85699999999999998,"15min":1.7090000000000001},"last":0.52200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":780967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51300000000000001,"15min":0.65500000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":7.7590000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.52100000000000002,"15min":0.69199999999999995},"min":{"1min":0.32200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.075,"5min":1.075,"15min":7.8090000000000002},"last":0.50800000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.69099999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.70199999999999996,"5min":0.80000000000000004,"15min":7.867},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.85599999999999998},"min":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.55000000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.58799999999999997},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.56100000000000005,"15min":0.81999999999999995},"min":{"1min":0.34200000000000003,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":2.641,"5min":2.641,"15min":6.7629999999999999},"last":0.56000000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54300000000000004,"15min":0.61099999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.107,"5min":1.107,"15min":4.8620000000000001},"last":0.62},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.70099999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.718,"5min":1.718,"15min":7.9009999999999998},"last":0.71799999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.47299999999999998},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.65200000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52700000000000002,"15min":0.495},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.53400000000000003},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52600000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55800000000000005,"15min":0.52500000000000002},"min":{"1min":0.36099999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.372,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.73799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303100,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068740,"statfs":{"total":96636764160,"available":96326389760,"internally_reserved":0,"allocated":1757184,"data_stored":782503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.312,"15min":0.45900000000000002},"min":{"1min":0.22800000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.437,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.247},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":1.194},"last":0.28299999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.24199999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51000000000000001,"5min":0.51400000000000001,"15min":1.2869999999999999},"last":0.309},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.48999999999999999},"min":{"1min":0.27200000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.52800000000000002,"5min":0.54200000000000004,"15min":5.0919999999999996},"last":0.29799999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36399999999999999,"15min":0.51200000000000001},"min":{"1min":0.29399999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35699999999999998,"15min":0.40200000000000002},"min":{"1min":0.23799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":1.2170000000000001},"last":0.32200000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.73999999999999999,"5min":0.92400000000000004,"15min":1.254},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.68999999999999995},"min":{"1min":0.39300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68799999999999994,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.45300000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.36099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":0.97599999999999998,"15min":1.1799999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56899999999999995,"15min":0.60299999999999998},"min":{"1min":0.41799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71399999999999997,"5min":0.878,"15min":1.3109999999999999},"last":0.46500000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.81799999999999995},"min":{"1min":0.46000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78900000000000003,"5min":0.995,"15min":5.0659999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80700000000000005,"5min":0.89800000000000002,"15min":1.171},"last":0.55800000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57999999999999996,"15min":0.61299999999999999},"min":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77300000000000002,"5min":1.004,"15min":1.214},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.70999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78200000000000003,"5min":0.90500000000000003,"15min":4.0499999999999998},"last":0.54000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303668,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068172,"statfs":{"total":96636764160,"available":96325808128,"internally_reserved":0,"allocated":2338816,"data_stored":1370396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.316,"15min":0.50800000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":7.9249999999999998},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.48699999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":6.6299999999999999},"last":0.30599999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.33100000000000002,"15min":0.53100000000000003},"min":{"1min":0.23100000000000001,"5min":0.22,"15min":0.216},"max":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":8.2509999999999994},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.51100000000000001},"min":{"1min":0.253,"5min":0.222,"15min":0.222},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":6.7370000000000001},"last":0.35899999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.35399999999999998,"15min":0.53900000000000003},"min":{"1min":0.247,"5min":0.23100000000000001,"15min":0.21099999999999999},"max":{"1min":0.53800000000000003,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.373},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.59099999999999997},"min":{"1min":0.27700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":7.8929999999999998},"last":0.42199999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57099999999999995,"15min":0.70299999999999996},"min":{"1min":0.45500000000000002,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.82499999999999996,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57299999999999995,"15min":0.754},"min":{"1min":0.441,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.80900000000000005,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.51600000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56100000000000005,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71299999999999997,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.76100000000000001},"min":{"1min":0.48399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.52300000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61499999999999999,"15min":0.80000000000000004},"min":{"1min":0.52200000000000002,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.83299999999999996,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.62,"15min":0.749},"min":{"1min":0.47399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82599999999999996,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.627}]},{"osd":7,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.60099999999999998,"15min":0.73999999999999999},"min":{"1min":0.48399999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85399999999999998,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.76800000000000002},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.75700000000000001,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.498}]}]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303100,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068740,"statfs":{"total":96636764160,"available":96326389760,"internally_reserved":0,"allocated":1757184,"data_stored":781633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.311,"15min":0.32500000000000001},"min":{"1min":0.214,"5min":0.189,"15min":0.189},"max":{"1min":0.52500000000000002,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.20200000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.35099999999999998},"min":{"1min":0.219,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.746},"last":0.224}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.69099999999999995},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52100000000000002,"5min":0.63,"15min":11.967000000000001},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.371,"5min":0.34899999999999998,"15min":0.46999999999999997},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48899999999999999,"5min":0.53400000000000003,"15min":4.1100000000000003},"last":0.33500000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34699999999999998,"15min":0.53700000000000003},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.50800000000000001,"5min":0.51600000000000001,"15min":6.3040000000000003},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.74299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59299999999999997,"5min":0.65900000000000003,"15min":11.935},"last":0.32200000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.55700000000000005,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.47399999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.38,"15min":0.378},"max":{"1min":0.83699999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54200000000000004,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.96099999999999997},"last":0.46100000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.55200000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.58599999999999997},"min":{"1min":0.435,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.052},"last":0.5},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58099999999999996,"15min":0.76300000000000001},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":6.2649999999999997},"last":0.45500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574942,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303100,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068740,"statfs":{"total":96636764160,"available":96326389760,"internally_reserved":0,"allocated":1757184,"data_stored":781161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.315,"15min":0.315},"min":{"1min":0.223,"5min":0.21199999999999999,"15min":0.21099999999999999},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32100000000000001},"min":{"1min":0.23999999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48799999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.23799999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.34599999999999997,"15min":0.33300000000000002},"min":{"1min":0.25700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.379},"min":{"1min":0.29199999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.72099999999999997},"last":0.30299999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.36199999999999999,"15min":0.35299999999999998},"min":{"1min":0.25800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57599999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.28999999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.50900000000000001,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.374}]},{"osd":4,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.53400000000000003},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77200000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.745,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.57199999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57299999999999995,"15min":0.55200000000000005},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.35899999999999999},"max":{"1min":0.82399999999999995,"5min":1.347,"15min":1.347},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.42199999999999999,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.83499999999999996,"5min":1.01,"15min":1.01},"last":0.52700000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.48399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86299999999999999,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.81200000000000006,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.59099999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78700000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.624},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.83699999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.54700000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303104,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068736,"statfs":{"total":96636764160,"available":96326385664,"internally_reserved":0,"allocated":1761280,"data_stored":783057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.32400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66600000000000004,"5min":0.74199999999999999,"15min":0.747},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51700000000000002,"15min":0.74099999999999999},"min":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77600000000000002,"5min":0.78400000000000003,"15min":7.6719999999999997},"last":0.46000000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52000000000000002,"15min":0.748},"min":{"1min":0.32000000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.755,"15min":7.7160000000000002},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66100000000000003,"5min":0.76900000000000002,"15min":0.86799999999999999},"last":0.67100000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.73899999999999999,"5min":0.79800000000000004,"15min":7.7839999999999998},"last":0.74099999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.39200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.71499999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.49099999999999999},"min":{"1min":0.35199999999999998,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.874},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.82999999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.85199999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58399999999999996,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.81599999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.41399999999999998,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.85299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":780967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.47599999999999998,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.66100000000000003,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.51800000000000002},"min":{"1min":0.33000000000000002,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.68899999999999995,"5min":0.76700000000000002,"15min":0.79300000000000004},"last":0.61799999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51300000000000001,"15min":0.628},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63400000000000001,"5min":0.75700000000000001,"15min":5.2679999999999998},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.62},"min":{"1min":0.40500000000000003,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79200000000000004,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.58099999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52400000000000002,"15min":0.63700000000000001},"min":{"1min":0.39100000000000001,"5min":0.32900000000000001,"15min":0.28299999999999997},"max":{"1min":0.73299999999999998,"5min":0.76600000000000001,"15min":5.2960000000000003},"last":0.505},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.38500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.76400000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55900000000000005,"15min":0.66900000000000004},"min":{"1min":0.41999999999999998,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":5.3330000000000002},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56399999999999995,"15min":0.68000000000000005},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.27700000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.81499999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.48899999999999999,"15min":0.498},"min":{"1min":0.33400000000000002,"5min":0.29599999999999999,"15min":0.28699999999999998},"max":{"1min":0.67000000000000004,"5min":0.72499999999999998,"15min":0.82999999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.34499999999999997,"15min":0.32200000000000001},"max":{"1min":0.77200000000000002,"5min":0.88,"15min":1.0800000000000001},"last":0.56000000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.39900000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78900000000000003,"5min":0.82499999999999996,"15min":0.89800000000000002},"last":0.78900000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.32300000000000001,"15min":0.30099999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98399999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.92600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:41:58.888 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:41:59.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.014+0000 7f9a98eea700 1 Processor -- start 2023-10-10T20:41:59.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.014+0000 7f9a98eea700 1 -- start start 2023-10-10T20:41:59.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.015+0000 7f9a98eea700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9a94159420 0x7f9a940b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:59.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.015+0000 7f9a98eea700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a940b2980 con 0x7f9a9415d230 2023-10-10T20:41:59.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.015+0000 7f9a98eea700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a940b11d0 con 0x7f9a941597f0 2023-10-10T20:41:59.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.015+0000 7f9a98eea700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a940b14b0 con 0x7f9a94159420 2023-10-10T20:41:59.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.015+0000 7f9a93fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9a9415d230 0x7f9a940b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35636/0 (socket says 172.21.15.89:35636) 2023-10-10T20:41:59.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.015+0000 7f9a93fff700 1 -- 172.21.15.89:0/2659378450 learned_addr learned my addr 172.21.15.89:0/2659378450 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:59.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.015+0000 7f9a927fc700 1 -- 172.21.15.89:0/2659378450 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810095850 0 0) 0x7f9a940b2980 con 0x7f9a9415d230 2023-10-10T20:41:59.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.015+0000 7f9a937fe700 1 --2- 172.21.15.89:0/2659378450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9a94159420 0x7f9a940b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:59.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a927fc700 1 -- 172.21.15.89:0/2659378450 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a7c003570 con 0x7f9a9415d230 2023-10-10T20:41:59.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a927fc700 1 -- 172.21.15.89:0/2659378450 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 132001014 0 0) 0x7f9a7c003570 con 0x7f9a9415d230 2023-10-10T20:41:59.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a927fc700 1 -- 172.21.15.89:0/2659378450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9a94159420 msgr2=0x7f9a940b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:59.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a927fc700 1 --2- 172.21.15.89:0/2659378450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9a94159420 0x7f9a940b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a927fc700 1 -- 172.21.15.89:0/2659378450 >> v1:172.21.15.111:6789/0 conn(0x7f9a941597f0 legacy=0x7f9a940b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:59.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a927fc700 1 -- 172.21.15.89:0/2659378450 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a940b1790 con 0x7f9a9415d230 2023-10-10T20:41:59.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a927fc700 1 -- 172.21.15.89:0/2659378450 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f9a8c001470 con 0x7f9a9415d230 2023-10-10T20:41:59.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a927fc700 1 -- 172.21.15.89:0/2659378450 >> v1:172.21.15.89:6789/0 conn(0x7f9a9415d230 legacy=0x7f9a940b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:59.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a927fc700 1 --2- 172.21.15.89:0/2659378450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a7c003a90 0x7f9a7c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:59.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a927fc700 1 --2- 172.21.15.89:0/2659378450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a7c004810 0x7f9a7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:59.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a927fc700 1 --2- 172.21.15.89:0/2659378450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9a94159420 0x7f9a7c0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:59.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a927fc700 1 -- 172.21.15.89:0/2659378450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a940b14b0 con 0x7f9a7c003a90 2023-10-10T20:41:59.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a927fc700 1 -- 172.21.15.89:0/2659378450 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c00bb60 con 0x7f9a7c004810 2023-10-10T20:41:59.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a927fc700 1 -- 172.21.15.89:0/2659378450 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c00bd20 con 0x7f9a94159420 2023-10-10T20:41:59.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a937fe700 1 --2- 172.21.15.89:0/2659378450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a7c003a90 0x7f9a7c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:59.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a92ffd700 1 --2- 172.21.15.89:0/2659378450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a7c004810 0x7f9a7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:59.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a93fff700 1 --2- 172.21.15.89:0/2659378450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9a94159420 0x7f9a7c0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:59.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a937fe700 1 -- 172.21.15.89:0/2659378450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9a94159420 msgr2=0x7f9a7c0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:59.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a937fe700 1 --2- 172.21.15.89:0/2659378450 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9a94159420 0x7f9a7c0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a937fe700 1 -- 172.21.15.89:0/2659378450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a7c004810 msgr2=0x7f9a7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:59.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a937fe700 1 --2- 172.21.15.89:0/2659378450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a7c004810 0x7f9a7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a937fe700 1 -- 172.21.15.89:0/2659378450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7c003570 con 0x7f9a7c003a90 2023-10-10T20:41:59.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a92ffd700 1 --2- 172.21.15.89:0/2659378450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a7c004810 0x7f9a7c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:59.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a937fe700 1 --2- 172.21.15.89:0/2659378450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a7c003a90 0x7f9a7c003ea0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400e210 tx=0x7f9a8400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4713b517a188fed server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:59.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.016+0000 7f9a927fc700 1 -- 172.21.15.89:0/2659378450 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8400eac0 con 0x7f9a7c003a90 2023-10-10T20:41:59.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.017+0000 7f9a927fc700 1 -- 172.21.15.89:0/2659378450 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8400f040 con 0x7f9a7c003a90 2023-10-10T20:41:59.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.017+0000 7f9a927fc700 1 -- 172.21.15.89:0/2659378450 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8400ee70 con 0x7f9a7c003a90 2023-10-10T20:41:59.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.017+0000 7f9a98eea700 1 -- 172.21.15.89:0/2659378450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a7c003a90 msgr2=0x7f9a7c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:59.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.017+0000 7f9a98eea700 1 --2- 172.21.15.89:0/2659378450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a7c003a90 0x7f9a7c003ea0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400e210 tx=0x7f9a8400b040 comp rx=0 tx=0).stop 2023-10-10T20:41:59.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.017+0000 7f9a93fff700 1 -- 172.21.15.89:0/2659378450 reap_dead start 2023-10-10T20:41:59.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.017+0000 7f9a98eea700 1 -- 172.21.15.89:0/2659378450 shutdown_connections 2023-10-10T20:41:59.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.017+0000 7f9a98eea700 1 -- 172.21.15.89:0/2659378450 >> 172.21.15.89:0/2659378450 conn(0x7f9a940d0660 msgr2=0x7f9a940d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:59.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.017+0000 7f9a98eea700 1 -- 172.21.15.89:0/2659378450 shutdown_connections 2023-10-10T20:41:59.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.017+0000 7f9a98eea700 1 -- 172.21.15.89:0/2659378450 wait complete. 2023-10-10T20:41:59.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.018+0000 7f9a98eea700 1 Processor -- start 2023-10-10T20:41:59.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.018+0000 7f9a98eea700 1 -- start start 2023-10-10T20:41:59.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.019+0000 7f9a98eea700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a7c003a90 0x7f9a94142290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:59.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.019+0000 7f9a98eea700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a9415ed30 0x7f9a9415f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:59.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.019+0000 7f9a98eea700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9a9415a8b0 0x7f9a9415ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:59.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.019+0000 7f9a98eea700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9415b290 con 0x7f9a7c003a90 2023-10-10T20:41:59.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.019+0000 7f9a98eea700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a94153e40 con 0x7f9a9415ed30 2023-10-10T20:41:59.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.019+0000 7f9a98eea700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a941427d0 con 0x7f9a9415a8b0 2023-10-10T20:41:59.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.019+0000 7f9a937fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a7c003a90 0x7f9a94142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:59.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.019+0000 7f9a937fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a7c003a90 0x7f9a94142290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43954/0 (socket says 172.21.15.89:43954) 2023-10-10T20:41:59.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.019+0000 7f9a937fe700 1 -- 172.21.15.89:0/3677102370 learned_addr learned my addr 172.21.15.89:0/3677102370 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:59.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.019+0000 7f9a92ffd700 1 --2- 172.21.15.89:0/3677102370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a9415ed30 0x7f9a9415f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:59.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.019+0000 7f9a93fff700 1 --2- 172.21.15.89:0/3677102370 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9a9415a8b0 0x7f9a9415ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:59.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.020+0000 7f9a937fe700 1 -- 172.21.15.89:0/3677102370 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9a9415a8b0 msgr2=0x7f9a9415ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:59.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.020+0000 7f9a937fe700 1 --2- 172.21.15.89:0/3677102370 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9a9415a8b0 0x7f9a9415ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.020+0000 7f9a937fe700 1 -- 172.21.15.89:0/3677102370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a9415ed30 msgr2=0x7f9a9415f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:59.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.020+0000 7f9a937fe700 1 --2- 172.21.15.89:0/3677102370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a9415ed30 0x7f9a9415f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.020+0000 7f9a937fe700 1 -- 172.21.15.89:0/3677102370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9413f8a0 con 0x7f9a7c003a90 2023-10-10T20:41:59.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.020+0000 7f9a92ffd700 1 --2- 172.21.15.89:0/3677102370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a9415ed30 0x7f9a9415f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:41:59.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.020+0000 7f9a937fe700 1 --2- 172.21.15.89:0/3677102370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a7c003a90 0x7f9a94142290 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400afd0 tx=0x7f9a8401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:59.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.020+0000 7f9a7b7fe700 1 -- 172.21.15.89:0/3677102370 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84029450 con 0x7f9a7c003a90 2023-10-10T20:41:59.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.020+0000 7f9a7b7fe700 1 -- 172.21.15.89:0/3677102370 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a840295d0 con 0x7f9a7c003a90 2023-10-10T20:41:59.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.020+0000 7f9a98eea700 1 -- 172.21.15.89:0/3677102370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9415f7d0 con 0x7f9a7c003a90 2023-10-10T20:41:59.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.020+0000 7f9a7b7fe700 1 -- 172.21.15.89:0/3677102370 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84029990 con 0x7f9a7c003a90 2023-10-10T20:41:59.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.020+0000 7f9a98eea700 1 -- 172.21.15.89:0/3677102370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a94153ad0 con 0x7f9a7c003a90 2023-10-10T20:41:59.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.021+0000 7f9a98eea700 1 -- 172.21.15.89:0/3677102370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a94153930 con 0x7f9a7c003a90 2023-10-10T20:41:59.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.021+0000 7f9a7b7fe700 1 -- 172.21.15.89:0/3677102370 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9a84003040 con 0x7f9a7c003a90 2023-10-10T20:41:59.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.022+0000 7f9a7b7fe700 1 --2- 172.21.15.89:0/3677102370 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9a80067320 0x7f9a800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:59.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.022+0000 7f9a92ffd700 1 --2- 172.21.15.89:0/3677102370 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9a80067320 0x7f9a800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:59.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.022+0000 7f9a7b7fe700 1 -- 172.21.15.89:0/3677102370 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9a840870f0 con 0x7f9a7c003a90 2023-10-10T20:41:59.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.022+0000 7f9a92ffd700 1 --2- 172.21.15.89:0/3677102370 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9a80067320 0x7f9a800697d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9a88000c00 tx=0x7f9a88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:59.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.026+0000 7f9a7b7fe700 1 -- 172.21.15.89:0/3677102370 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a94153930 con 0x7f9a7c003a90 2023-10-10T20:41:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.206+0000 7f9a98eea700 1 -- 172.21.15.89:0/3677102370 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a94153f80 con 0x7f9a80067320 2023-10-10T20:41:59.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.214+0000 7f9a7b7fe700 1 -- 172.21.15.89:0/3677102370 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289963 (secure 0 0 0) 0x7f9a880016d0 con 0x7f9a80067320 2023-10-10T20:41:59.219 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:41:59.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.217+0000 7f9a98eea700 1 -- 172.21.15.89:0/3677102370 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9a80067320 msgr2=0x7f9a800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:59.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.217+0000 7f9a98eea700 1 --2- 172.21.15.89:0/3677102370 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9a80067320 0x7f9a800697d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9a88000c00 tx=0x7f9a88009040 comp rx=0 tx=0).stop 2023-10-10T20:41:59.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.217+0000 7f9a98eea700 1 -- 172.21.15.89:0/3677102370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a7c003a90 msgr2=0x7f9a94142290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:59.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.217+0000 7f9a98eea700 1 --2- 172.21.15.89:0/3677102370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a7c003a90 0x7f9a94142290 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400afd0 tx=0x7f9a8401a000 comp rx=0 tx=0).stop 2023-10-10T20:41:59.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.217+0000 7f9a98eea700 1 -- 172.21.15.89:0/3677102370 shutdown_connections 2023-10-10T20:41:59.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.217+0000 7f9a98eea700 1 --2- 172.21.15.89:0/3677102370 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9a9415a8b0 0x7f9a9415ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.217+0000 7f9a98eea700 1 --2- 172.21.15.89:0/3677102370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a7c003a90 0x7f9a94142290 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.217+0000 7f9a98eea700 1 --2- 172.21.15.89:0/3677102370 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9a80067320 0x7f9a800697d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.217+0000 7f9a98eea700 1 --2- 172.21.15.89:0/3677102370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a9415ed30 0x7f9a9415f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.217+0000 7f9a98eea700 1 -- 172.21.15.89:0/3677102370 >> 172.21.15.89:0/3677102370 conn(0x7f9a940d0660 msgr2=0x7f9a94068c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:59.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.217+0000 7f9a98eea700 1 -- 172.21.15.89:0/3677102370 shutdown_connections 2023-10-10T20:41:59.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.217+0000 7f9a98eea700 1 -- 172.21.15.89:0/3677102370 wait complete. 2023-10-10T20:41:59.224 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:41:59.231 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2023-10-10T20:41:58.740247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425924,"kb_used_data":14852,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752548796,"statfs":{"total":773094113280,"available":770609967104,"internally_reserved":0,"allocated":15208448,"data_stored":7432123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.440970"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328850+0000","last_change":"2023-10-10T20:41:50.324854+0000","last_active":"2023-10-10T20:41:50.328850+0000","last_peered":"2023-10-10T20:41:50.328850+0000","last_clean":"2023-10-10T20:41:50.328850+0000","last_became_active":"2023-10-10T20:41:50.324573+0000","last_became_peered":"2023-10-10T20:41:50.324573+0000","last_unstale":"2023-10-10T20:41:50.328850+0000","last_undegraded":"2023-10-10T20:41:50.328850+0000","last_fullsized":"2023-10-10T20:41:50.328850+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:07:32.681938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329631+0000","last_change":"2023-10-10T20:41:50.328756+0000","last_active":"2023-10-10T20:41:50.329631+0000","last_peered":"2023-10-10T20:41:50.329631+0000","last_clean":"2023-10-10T20:41:50.329631+0000","last_became_active":"2023-10-10T20:41:50.328488+0000","last_became_peered":"2023-10-10T20:41:50.328488+0000","last_unstale":"2023-10-10T20:41:50.329631+0000","last_undegraded":"2023-10-10T20:41:50.329631+0000","last_fullsized":"2023-10-10T20:41:50.329631+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:32:34.541780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.322750+0000","last_change":"2023-10-10T20:41:50.317128+0000","last_active":"2023-10-10T20:41:50.322750+0000","last_peered":"2023-10-10T20:41:50.322750+0000","last_clean":"2023-10-10T20:41:50.322750+0000","last_became_active":"2023-10-10T20:41:50.316862+0000","last_became_peered":"2023-10-10T20:41:50.316862+0000","last_unstale":"2023-10-10T20:41:50.322750+0000","last_undegraded":"2023-10-10T20:41:50.322750+0000","last_fullsized":"2023-10-10T20:41:50.322750+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:59.167375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329233+0000","last_change":"2023-10-10T20:41:50.325779+0000","last_active":"2023-10-10T20:41:50.329233+0000","last_peered":"2023-10-10T20:41:50.329233+0000","last_clean":"2023-10-10T20:41:50.329233+0000","last_became_active":"2023-10-10T20:41:50.325527+0000","last_became_peered":"2023-10-10T20:41:50.325527+0000","last_unstale":"2023-10-10T20:41:50.329233+0000","last_undegraded":"2023-10-10T20:41:50.329233+0000","last_fullsized":"2023-10-10T20:41:50.329233+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:09:42.528347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329492+0000","last_change":"2023-10-10T20:41:50.322671+0000","last_active":"2023-10-10T20:41:50.329492+0000","last_peered":"2023-10-10T20:41:50.329492+0000","last_clean":"2023-10-10T20:41:50.329492+0000","last_became_active":"2023-10-10T20:41:50.322277+0000","last_became_peered":"2023-10-10T20:41:50.322277+0000","last_unstale":"2023-10-10T20:41:50.329492+0000","last_undegraded":"2023-10-10T20:41:50.329492+0000","last_fullsized":"2023-10-10T20:41:50.329492+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:03.021663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.320162+0000","last_change":"2023-10-10T20:41:50.314512+0000","last_active":"2023-10-10T20:41:50.320162+0000","last_peered":"2023-10-10T20:41:50.320162+0000","last_clean":"2023-10-10T20:41:50.320162+0000","last_became_active":"2023-10-10T20:41:50.314257+0000","last_became_peered":"2023-10-10T20:41:50.314257+0000","last_unstale":"2023-10-10T20:41:50.320162+0000","last_undegraded":"2023-10-10T20:41:50.320162+0000","last_fullsized":"2023-10-10T20:41:50.320162+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:52:08.397497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329082+0000","last_change":"2023-10-10T20:41:50.322740+0000","last_active":"2023-10-10T20:41:50.329082+0000","last_peered":"2023-10-10T20:41:50.329082+0000","last_clean":"2023-10-10T20:41:50.329082+0000","last_became_active":"2023-10-10T20:41:50.322415+0000","last_became_peered":"2023-10-10T20:41:50.322415+0000","last_unstale":"2023-10-10T20:41:50.329082+0000","last_undegraded":"2023-10-10T20:41:50.329082+0000","last_fullsized":"2023-10-10T20:41:50.329082+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:59.167375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327463+0000","last_change":"2023-10-10T20:41:50.320942+0000","last_active":"2023-10-10T20:41:50.327463+0000","last_peered":"2023-10-10T20:41:50.327463+0000","last_clean":"2023-10-10T20:41:50.327463+0000","last_became_active":"2023-10-10T20:41:50.320312+0000","last_became_peered":"2023-10-10T20:41:50.320312+0000","last_unstale":"2023-10-10T20:41:50.327463+0000","last_undegraded":"2023-10-10T20:41:50.327463+0000","last_fullsized":"2023-10-10T20:41:50.327463+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:08:58.453242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.324745+0000","last_change":"2023-10-10T20:41:50.318424+0000","last_active":"2023-10-10T20:41:50.324745+0000","last_peered":"2023-10-10T20:41:50.324745+0000","last_clean":"2023-10-10T20:41:50.324745+0000","last_became_active":"2023-10-10T20:41:50.318177+0000","last_became_peered":"2023-10-10T20:41:50.318177+0000","last_unstale":"2023-10-10T20:41:50.324745+0000","last_undegraded":"2023-10-10T20:41:50.324745+0000","last_fullsized":"2023-10-10T20:41:50.324745+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:25.820041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.325823+0000","last_change":"2023-10-10T20:41:50.320526+0000","last_active":"2023-10-10T20:41:50.325823+0000","last_peered":"2023-10-10T20:41:50.325823+0000","last_clean":"2023-10-10T20:41:50.325823+0000","last_became_active":"2023-10-10T20:41:50.320284+0000","last_became_peered":"2023-10-10T20:41:50.320284+0000","last_unstale":"2023-10-10T20:41:50.325823+0000","last_undegraded":"2023-10-10T20:41:50.325823+0000","last_fullsized":"2023-10-10T20:41:50.325823+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:56.229435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.316595+0000","last_change":"2023-10-10T20:41:50.312426+0000","last_active":"2023-10-10T20:41:50.316595+0000","last_peered":"2023-10-10T20:41:50.316595+0000","last_clean":"2023-10-10T20:41:50.316595+0000","last_became_active":"2023-10-10T20:41:50.312160+0000","last_became_peered":"2023-10-10T20:41:50.312160+0000","last_unstale":"2023-10-10T20:41:50.316595+0000","last_undegraded":"2023-10-10T20:41:50.316595+0000","last_fullsized":"2023-10-10T20:41:50.316595+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:49:46.940449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.324082+0000","last_change":"2023-10-10T20:41:50.319250+0000","last_active":"2023-10-10T20:41:50.324082+0000","last_peered":"2023-10-10T20:41:50.324082+0000","last_clean":"2023-10-10T20:41:50.324082+0000","last_became_active":"2023-10-10T20:41:50.318945+0000","last_became_peered":"2023-10-10T20:41:50.318945+0000","last_unstale":"2023-10-10T20:41:50.324082+0000","last_undegraded":"2023-10-10T20:41:50.324082+0000","last_fullsized":"2023-10-10T20:41:50.324082+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:29.171361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.324092+0000","last_change":"2023-10-10T20:41:50.320071+0000","last_active":"2023-10-10T20:41:50.324092+0000","last_peered":"2023-10-10T20:41:50.324092+0000","last_clean":"2023-10-10T20:41:50.324092+0000","last_became_active":"2023-10-10T20:41:50.319800+0000","last_became_peered":"2023-10-10T20:41:50.319800+0000","last_unstale":"2023-10-10T20:41:50.324092+0000","last_undegraded":"2023-10-10T20:41:50.324092+0000","last_fullsized":"2023-10-10T20:41:50.324092+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:59.707661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.324909+0000","last_change":"2023-10-10T20:41:50.320391+0000","last_active":"2023-10-10T20:41:50.324909+0000","last_peered":"2023-10-10T20:41:50.324909+0000","last_clean":"2023-10-10T20:41:50.324909+0000","last_became_active":"2023-10-10T20:41:50.320087+0000","last_became_peered":"2023-10-10T20:41:50.320087+0000","last_unstale":"2023-10-10T20:41:50.324909+0000","last_undegraded":"2023-10-10T20:41:50.324909+0000","last_fullsized":"2023-10-10T20:41:50.324909+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:56:15.566395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.323944+0000","last_change":"2023-10-10T20:41:50.319419+0000","last_active":"2023-10-10T20:41:50.323944+0000","last_peered":"2023-10-10T20:41:50.323944+0000","last_clean":"2023-10-10T20:41:50.323944+0000","last_became_active":"2023-10-10T20:41:50.318904+0000","last_became_peered":"2023-10-10T20:41:50.318904+0000","last_unstale":"2023-10-10T20:41:50.323944+0000","last_undegraded":"2023-10-10T20:41:50.323944+0000","last_fullsized":"2023-10-10T20:41:50.323944+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:59.167375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327445+0000","last_change":"2023-10-10T20:41:50.314811+0000","last_active":"2023-10-10T20:41:50.327445+0000","last_peered":"2023-10-10T20:41:50.327445+0000","last_clean":"2023-10-10T20:41:50.327445+0000","last_became_active":"2023-10-10T20:41:50.313773+0000","last_became_peered":"2023-10-10T20:41:50.313773+0000","last_unstale":"2023-10-10T20:41:50.327445+0000","last_undegraded":"2023-10-10T20:41:50.327445+0000","last_fullsized":"2023-10-10T20:41:50.327445+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:06.177167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329995+0000","last_change":"2023-10-10T20:41:50.326631+0000","last_active":"2023-10-10T20:41:50.329995+0000","last_peered":"2023-10-10T20:41:50.329995+0000","last_clean":"2023-10-10T20:41:50.329995+0000","last_became_active":"2023-10-10T20:41:50.323639+0000","last_became_peered":"2023-10-10T20:41:50.323639+0000","last_unstale":"2023-10-10T20:41:50.329995+0000","last_undegraded":"2023-10-10T20:41:50.329995+0000","last_fullsized":"2023-10-10T20:41:50.329995+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:37:21.016202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.315357+0000","last_change":"2023-10-10T20:41:50.311398+0000","last_active":"2023-10-10T20:41:50.315357+0000","last_peered":"2023-10-10T20:41:50.315357+0000","last_clean":"2023-10-10T20:41:50.315357+0000","last_became_active":"2023-10-10T20:41:50.311136+0000","last_became_peered":"2023-10-10T20:41:50.311136+0000","last_unstale":"2023-10-10T20:41:50.315357+0000","last_undegraded":"2023-10-10T20:41:50.315357+0000","last_fullsized":"2023-10-10T20:41:50.315357+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:51:24.010245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327174+0000","last_change":"2023-10-10T20:41:50.322755+0000","last_active":"2023-10-10T20:41:50.327174+0000","last_peered":"2023-10-10T20:41:50.327174+0000","last_clean":"2023-10-10T20:41:50.327174+0000","last_became_active":"2023-10-10T20:41:50.322514+0000","last_became_peered":"2023-10-10T20:41:50.322514+0000","last_unstale":"2023-10-10T20:41:50.327174+0000","last_undegraded":"2023-10-10T20:41:50.327174+0000","last_fullsized":"2023-10-10T20:41:50.327174+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:06.177167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329970+0000","last_change":"2023-10-10T20:41:50.326356+0000","last_active":"2023-10-10T20:41:50.329970+0000","last_peered":"2023-10-10T20:41:50.329970+0000","last_clean":"2023-10-10T20:41:50.329970+0000","last_became_active":"2023-10-10T20:41:50.326042+0000","last_became_peered":"2023-10-10T20:41:50.326042+0000","last_unstale":"2023-10-10T20:41:50.329970+0000","last_undegraded":"2023-10-10T20:41:50.329970+0000","last_fullsized":"2023-10-10T20:41:50.329970+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:36:22.310224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329474+0000","last_change":"2023-10-10T20:41:50.322516+0000","last_active":"2023-10-10T20:41:50.329474+0000","last_peered":"2023-10-10T20:41:50.329474+0000","last_clean":"2023-10-10T20:41:50.329474+0000","last_became_active":"2023-10-10T20:41:50.321967+0000","last_became_peered":"2023-10-10T20:41:50.321967+0000","last_unstale":"2023-10-10T20:41:50.329474+0000","last_undegraded":"2023-10-10T20:41:50.329474+0000","last_fullsized":"2023-10-10T20:41:50.329474+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:58.433900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.334853+0000","last_change":"2023-10-10T20:41:50.332271+0000","last_active":"2023-10-10T20:41:50.334853+0000","last_peered":"2023-10-10T20:41:50.334853+0000","last_clean":"2023-10-10T20:41:50.334853+0000","last_became_active":"2023-10-10T20:41:50.331819+0000","last_became_peered":"2023-10-10T20:41:50.331819+0000","last_unstale":"2023-10-10T20:41:50.334853+0000","last_undegraded":"2023-10-10T20:41:50.334853+0000","last_fullsized":"2023-10-10T20:41:50.334853+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:08:58.453242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328951+0000","last_change":"2023-10-10T20:41:50.325040+0000","last_active":"2023-10-10T20:41:50.328951+0000","last_peered":"2023-10-10T20:41:50.328951+0000","last_clean":"2023-10-10T20:41:50.328951+0000","last_became_active":"2023-10-10T20:41:50.324726+0000","last_became_peered":"2023-10-10T20:41:50.324726+0000","last_unstale":"2023-10-10T20:41:50.328951+0000","last_undegraded":"2023-10-10T20:41:50.328951+0000","last_fullsized":"2023-10-10T20:41:50.328951+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:29.171361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.330924+0000","last_change":"2023-10-10T20:41:50.321301+0000","last_active":"2023-10-10T20:41:50.330924+0000","last_peered":"2023-10-10T20:41:50.330924+0000","last_clean":"2023-10-10T20:41:50.330924+0000","last_became_active":"2023-10-10T20:41:50.320658+0000","last_became_peered":"2023-10-10T20:41:50.320658+0000","last_unstale":"2023-10-10T20:41:50.330924+0000","last_undegraded":"2023-10-10T20:41:50.330924+0000","last_fullsized":"2023-10-10T20:41:50.330924+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:36:51.689597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.325951+0000","last_change":"2023-10-10T20:41:50.322737+0000","last_active":"2023-10-10T20:41:50.325951+0000","last_peered":"2023-10-10T20:41:50.325951+0000","last_clean":"2023-10-10T20:41:50.325951+0000","last_became_active":"2023-10-10T20:41:50.322509+0000","last_became_peered":"2023-10-10T20:41:50.322509+0000","last_unstale":"2023-10-10T20:41:50.325951+0000","last_undegraded":"2023-10-10T20:41:50.325951+0000","last_fullsized":"2023-10-10T20:41:50.325951+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:49:46.940449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328624+0000","last_change":"2023-10-10T20:41:50.321194+0000","last_active":"2023-10-10T20:41:50.328624+0000","last_peered":"2023-10-10T20:41:50.328624+0000","last_clean":"2023-10-10T20:41:50.328624+0000","last_became_active":"2023-10-10T20:41:50.320546+0000","last_became_peered":"2023-10-10T20:41:50.320546+0000","last_unstale":"2023-10-10T20:41:50.328624+0000","last_undegraded":"2023-10-10T20:41:50.328624+0000","last_fullsized":"2023-10-10T20:41:50.328624+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:29.171361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.319678+0000","last_change":"2023-10-10T20:41:50.314660+0000","last_active":"2023-10-10T20:41:50.319678+0000","last_peered":"2023-10-10T20:41:50.319678+0000","last_clean":"2023-10-10T20:41:50.319678+0000","last_became_active":"2023-10-10T20:41:50.314414+0000","last_became_peered":"2023-10-10T20:41:50.314414+0000","last_unstale":"2023-10-10T20:41:50.319678+0000","last_undegraded":"2023-10-10T20:41:50.319678+0000","last_fullsized":"2023-10-10T20:41:50.319678+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:09:42.528347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329592+0000","last_change":"2023-10-10T20:41:50.325931+0000","last_active":"2023-10-10T20:41:50.329592+0000","last_peered":"2023-10-10T20:41:50.329592+0000","last_clean":"2023-10-10T20:41:50.329592+0000","last_became_active":"2023-10-10T20:41:50.325457+0000","last_became_peered":"2023-10-10T20:41:50.325457+0000","last_unstale":"2023-10-10T20:41:50.329592+0000","last_undegraded":"2023-10-10T20:41:50.329592+0000","last_fullsized":"2023-10-10T20:41:50.329592+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:57.360052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.324074+0000","last_change":"2023-10-10T20:41:50.319188+0000","last_active":"2023-10-10T20:41:50.324074+0000","last_peered":"2023-10-10T20:41:50.324074+0000","last_clean":"2023-10-10T20:41:50.324074+0000","last_became_active":"2023-10-10T20:41:50.318887+0000","last_became_peered":"2023-10-10T20:41:50.318887+0000","last_unstale":"2023-10-10T20:41:50.324074+0000","last_undegraded":"2023-10-10T20:41:50.324074+0000","last_fullsized":"2023-10-10T20:41:50.324074+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:38.872388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.315523+0000","last_change":"2023-10-10T20:41:50.310296+0000","last_active":"2023-10-10T20:41:50.315523+0000","last_peered":"2023-10-10T20:41:50.315523+0000","last_clean":"2023-10-10T20:41:50.315523+0000","last_became_active":"2023-10-10T20:41:50.309872+0000","last_became_peered":"2023-10-10T20:41:50.309872+0000","last_unstale":"2023-10-10T20:41:50.315523+0000","last_undegraded":"2023-10-10T20:41:50.315523+0000","last_fullsized":"2023-10-10T20:41:50.315523+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:08:57.143352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.325071+0000","last_change":"2023-10-10T20:41:50.320021+0000","last_active":"2023-10-10T20:41:50.325071+0000","last_peered":"2023-10-10T20:41:50.325071+0000","last_clean":"2023-10-10T20:41:50.325071+0000","last_became_active":"2023-10-10T20:41:50.319667+0000","last_became_peered":"2023-10-10T20:41:50.319667+0000","last_unstale":"2023-10-10T20:41:50.325071+0000","last_undegraded":"2023-10-10T20:41:50.325071+0000","last_fullsized":"2023-10-10T20:41:50.325071+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:09:58.004059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329518+0000","last_change":"2023-10-10T20:41:50.320084+0000","last_active":"2023-10-10T20:41:50.329518+0000","last_peered":"2023-10-10T20:41:50.329518+0000","last_clean":"2023-10-10T20:41:50.329518+0000","last_became_active":"2023-10-10T20:41:50.319816+0000","last_became_peered":"2023-10-10T20:41:50.319816+0000","last_unstale":"2023-10-10T20:41:50.329518+0000","last_undegraded":"2023-10-10T20:41:50.329518+0000","last_fullsized":"2023-10-10T20:41:50.329518+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:25.711425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327656+0000","last_change":"2023-10-10T20:41:50.318101+0000","last_active":"2023-10-10T20:41:50.327656+0000","last_peered":"2023-10-10T20:41:50.327656+0000","last_clean":"2023-10-10T20:41:50.327656+0000","last_became_active":"2023-10-10T20:41:50.317813+0000","last_became_peered":"2023-10-10T20:41:50.317813+0000","last_unstale":"2023-10-10T20:41:50.327656+0000","last_undegraded":"2023-10-10T20:41:50.327656+0000","last_fullsized":"2023-10-10T20:41:50.327656+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:45:42.233949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"137'4","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.326399+0000","last_change":"2023-10-10T20:41:49.316818+0000","last_active":"2023-10-10T20:41:50.326399+0000","last_peered":"2023-10-10T20:41:50.326399+0000","last_clean":"2023-10-10T20:41:50.326399+0000","last_became_active":"2023-10-10T20:41:49.315998+0000","last_became_peered":"2023-10-10T20:41:49.315998+0000","last_unstale":"2023-10-10T20:41:50.326399+0000","last_undegraded":"2023-10-10T20:41:50.326399+0000","last_fullsized":"2023-10-10T20:41:50.326399+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:23:47.101110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329947+0000","last_change":"2023-10-10T20:41:50.328491+0000","last_active":"2023-10-10T20:41:50.329947+0000","last_peered":"2023-10-10T20:41:50.329947+0000","last_clean":"2023-10-10T20:41:50.329947+0000","last_became_active":"2023-10-10T20:41:50.328230+0000","last_became_peered":"2023-10-10T20:41:50.328230+0000","last_unstale":"2023-10-10T20:41:50.329947+0000","last_undegraded":"2023-10-10T20:41:50.329947+0000","last_fullsized":"2023-10-10T20:41:50.329947+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:45:42.233949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"137'1","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.314284+0000","last_change":"2023-10-10T20:41:49.310393+0000","last_active":"2023-10-10T20:41:50.314284+0000","last_peered":"2023-10-10T20:41:50.314284+0000","last_clean":"2023-10-10T20:41:50.314284+0000","last_became_active":"2023-10-10T20:41:49.309899+0000","last_became_peered":"2023-10-10T20:41:49.309899+0000","last_unstale":"2023-10-10T20:41:50.314284+0000","last_undegraded":"2023-10-10T20:41:50.314284+0000","last_fullsized":"2023-10-10T20:41:50.314284+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:56.354284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329152+0000","last_change":"2023-10-10T20:41:50.323854+0000","last_active":"2023-10-10T20:41:50.329152+0000","last_peered":"2023-10-10T20:41:50.329152+0000","last_clean":"2023-10-10T20:41:50.329152+0000","last_became_active":"2023-10-10T20:41:50.323559+0000","last_became_peered":"2023-10-10T20:41:50.323559+0000","last_unstale":"2023-10-10T20:41:50.329152+0000","last_undegraded":"2023-10-10T20:41:50.329152+0000","last_fullsized":"2023-10-10T20:41:50.329152+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:59.707661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"137'2","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.325205+0000","last_change":"2023-10-10T20:41:49.316507+0000","last_active":"2023-10-10T20:41:50.325205+0000","last_peered":"2023-10-10T20:41:50.325205+0000","last_clean":"2023-10-10T20:41:50.325205+0000","last_became_active":"2023-10-10T20:41:49.316246+0000","last_became_peered":"2023-10-10T20:41:49.316246+0000","last_unstale":"2023-10-10T20:41:50.325205+0000","last_undegraded":"2023-10-10T20:41:50.325205+0000","last_fullsized":"2023-10-10T20:41:50.325205+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:59.824875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.321151+0000","last_change":"2023-10-10T20:41:50.316576+0000","last_active":"2023-10-10T20:41:50.321151+0000","last_peered":"2023-10-10T20:41:50.321151+0000","last_clean":"2023-10-10T20:41:50.321151+0000","last_became_active":"2023-10-10T20:41:50.316279+0000","last_became_peered":"2023-10-10T20:41:50.316279+0000","last_unstale":"2023-10-10T20:41:50.321151+0000","last_undegraded":"2023-10-10T20:41:50.321151+0000","last_fullsized":"2023-10-10T20:41:50.321151+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:33:19.443146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"137'2","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.319554+0000","last_change":"2023-10-10T20:41:49.314721+0000","last_active":"2023-10-10T20:41:50.319554+0000","last_peered":"2023-10-10T20:41:50.319554+0000","last_clean":"2023-10-10T20:41:50.319554+0000","last_became_active":"2023-10-10T20:41:49.312771+0000","last_became_peered":"2023-10-10T20:41:49.312771+0000","last_unstale":"2023-10-10T20:41:50.319554+0000","last_undegraded":"2023-10-10T20:41:50.319554+0000","last_fullsized":"2023-10-10T20:41:50.319554+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:56:14.560628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.334834+0000","last_change":"2023-10-10T20:41:50.332094+0000","last_active":"2023-10-10T20:41:50.334834+0000","last_peered":"2023-10-10T20:41:50.334834+0000","last_clean":"2023-10-10T20:41:50.334834+0000","last_became_active":"2023-10-10T20:41:50.331523+0000","last_became_peered":"2023-10-10T20:41:50.331523+0000","last_unstale":"2023-10-10T20:41:50.334834+0000","last_undegraded":"2023-10-10T20:41:50.334834+0000","last_fullsized":"2023-10-10T20:41:50.334834+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:36:55.761497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"137'1","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327568+0000","last_change":"2023-10-10T20:41:49.315617+0000","last_active":"2023-10-10T20:41:50.327568+0000","last_peered":"2023-10-10T20:41:50.327568+0000","last_clean":"2023-10-10T20:41:50.327568+0000","last_became_active":"2023-10-10T20:41:49.315233+0000","last_became_peered":"2023-10-10T20:41:49.315233+0000","last_unstale":"2023-10-10T20:41:50.327568+0000","last_undegraded":"2023-10-10T20:41:50.327568+0000","last_fullsized":"2023-10-10T20:41:50.327568+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:41:47.663214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328784+0000","last_change":"2023-10-10T20:41:50.322989+0000","last_active":"2023-10-10T20:41:50.328784+0000","last_peered":"2023-10-10T20:41:50.328784+0000","last_clean":"2023-10-10T20:41:50.328784+0000","last_became_active":"2023-10-10T20:41:50.320266+0000","last_became_peered":"2023-10-10T20:41:50.320266+0000","last_unstale":"2023-10-10T20:41:50.328784+0000","last_undegraded":"2023-10-10T20:41:50.328784+0000","last_fullsized":"2023-10-10T20:41:50.328784+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:22:41.479243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"137'1","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.321165+0000","last_change":"2023-10-10T20:41:49.308749+0000","last_active":"2023-10-10T20:41:50.321165+0000","last_peered":"2023-10-10T20:41:50.321165+0000","last_clean":"2023-10-10T20:41:50.321165+0000","last_became_active":"2023-10-10T20:41:49.307933+0000","last_became_peered":"2023-10-10T20:41:49.307933+0000","last_unstale":"2023-10-10T20:41:50.321165+0000","last_undegraded":"2023-10-10T20:41:50.321165+0000","last_fullsized":"2023-10-10T20:41:50.321165+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:30:49.584095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.316607+0000","last_change":"2023-10-10T20:41:50.311319+0000","last_active":"2023-10-10T20:41:50.316607+0000","last_peered":"2023-10-10T20:41:50.316607+0000","last_clean":"2023-10-10T20:41:50.316607+0000","last_became_active":"2023-10-10T20:41:50.310519+0000","last_became_peered":"2023-10-10T20:41:50.310519+0000","last_unstale":"2023-10-10T20:41:50.316607+0000","last_undegraded":"2023-10-10T20:41:50.316607+0000","last_fullsized":"2023-10-10T20:41:50.316607+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:25.142717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.301856+0000","last_change":"2023-10-10T20:41:49.315721+0000","last_active":"2023-10-10T20:41:50.301856+0000","last_peered":"2023-10-10T20:41:50.301856+0000","last_clean":"2023-10-10T20:41:50.301856+0000","last_became_active":"2023-10-10T20:41:49.315438+0000","last_became_peered":"2023-10-10T20:41:49.315438+0000","last_unstale":"2023-10-10T20:41:50.301856+0000","last_undegraded":"2023-10-10T20:41:50.301856+0000","last_fullsized":"2023-10-10T20:41:50.301856+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:17:06.508904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.332154+0000","last_change":"2023-10-10T20:41:50.329023+0000","last_active":"2023-10-10T20:41:50.332154+0000","last_peered":"2023-10-10T20:41:50.332154+0000","last_clean":"2023-10-10T20:41:50.332154+0000","last_became_active":"2023-10-10T20:41:50.328773+0000","last_became_peered":"2023-10-10T20:41:50.328773+0000","last_unstale":"2023-10-10T20:41:50.332154+0000","last_undegraded":"2023-10-10T20:41:50.332154+0000","last_fullsized":"2023-10-10T20:41:50.332154+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:37:14.358493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.302232+0000","last_change":"2023-10-10T20:41:49.310044+0000","last_active":"2023-10-10T20:41:50.302232+0000","last_peered":"2023-10-10T20:41:50.302232+0000","last_clean":"2023-10-10T20:41:50.302232+0000","last_became_active":"2023-10-10T20:41:49.309759+0000","last_became_peered":"2023-10-10T20:41:49.309759+0000","last_unstale":"2023-10-10T20:41:50.302232+0000","last_undegraded":"2023-10-10T20:41:50.302232+0000","last_fullsized":"2023-10-10T20:41:50.302232+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:30:13.642642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328695+0000","last_change":"2023-10-10T20:41:50.323785+0000","last_active":"2023-10-10T20:41:50.328695+0000","last_peered":"2023-10-10T20:41:50.328695+0000","last_clean":"2023-10-10T20:41:50.328695+0000","last_became_active":"2023-10-10T20:41:50.323571+0000","last_became_peered":"2023-10-10T20:41:50.323571+0000","last_unstale":"2023-10-10T20:41:50.328695+0000","last_undegraded":"2023-10-10T20:41:50.328695+0000","last_fullsized":"2023-10-10T20:41:50.328695+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:58:22.113298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.310448+0000","last_change":"2023-10-10T20:41:49.317302+0000","last_active":"2023-10-10T20:41:50.310448+0000","last_peered":"2023-10-10T20:41:50.310448+0000","last_clean":"2023-10-10T20:41:50.310448+0000","last_became_active":"2023-10-10T20:41:49.317066+0000","last_became_peered":"2023-10-10T20:41:49.317066+0000","last_unstale":"2023-10-10T20:41:50.310448+0000","last_undegraded":"2023-10-10T20:41:50.310448+0000","last_fullsized":"2023-10-10T20:41:50.310448+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:41:24.307438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.317448+0000","last_change":"2023-10-10T20:41:50.311080+0000","last_active":"2023-10-10T20:41:50.317448+0000","last_peered":"2023-10-10T20:41:50.317448+0000","last_clean":"2023-10-10T20:41:50.317448+0000","last_became_active":"2023-10-10T20:41:50.310822+0000","last_became_peered":"2023-10-10T20:41:50.310822+0000","last_unstale":"2023-10-10T20:41:50.317448+0000","last_undegraded":"2023-10-10T20:41:50.317448+0000","last_fullsized":"2023-10-10T20:41:50.317448+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:09.931645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"137'3","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328903+0000","last_change":"2023-10-10T20:41:49.315940+0000","last_active":"2023-10-10T20:41:50.328903+0000","last_peered":"2023-10-10T20:41:50.328903+0000","last_clean":"2023-10-10T20:41:50.328903+0000","last_became_active":"2023-10-10T20:41:49.315620+0000","last_became_peered":"2023-10-10T20:41:49.315620+0000","last_unstale":"2023-10-10T20:41:50.328903+0000","last_undegraded":"2023-10-10T20:41:50.328903+0000","last_fullsized":"2023-10-10T20:41:50.328903+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:29:06.709268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.332378+0000","last_change":"2023-10-10T20:41:50.331754+0000","last_active":"2023-10-10T20:41:50.332378+0000","last_peered":"2023-10-10T20:41:50.332378+0000","last_clean":"2023-10-10T20:41:50.332378+0000","last_became_active":"2023-10-10T20:41:50.331492+0000","last_became_peered":"2023-10-10T20:41:50.331492+0000","last_unstale":"2023-10-10T20:41:50.332378+0000","last_undegraded":"2023-10-10T20:41:50.332378+0000","last_fullsized":"2023-10-10T20:41:50.332378+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:58:22.113298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"137'6","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329762+0000","last_change":"2023-10-10T20:41:49.307503+0000","last_active":"2023-10-10T20:41:50.329762+0000","last_peered":"2023-10-10T20:41:50.329762+0000","last_clean":"2023-10-10T20:41:50.329762+0000","last_became_active":"2023-10-10T20:41:49.307188+0000","last_became_peered":"2023-10-10T20:41:49.307188+0000","last_unstale":"2023-10-10T20:41:50.329762+0000","last_undegraded":"2023-10-10T20:41:50.329762+0000","last_fullsized":"2023-10-10T20:41:50.329762+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:22.143245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.325048+0000","last_change":"2023-10-10T20:41:50.320264+0000","last_active":"2023-10-10T20:41:50.325048+0000","last_peered":"2023-10-10T20:41:50.325048+0000","last_clean":"2023-10-10T20:41:50.325048+0000","last_became_active":"2023-10-10T20:41:50.320001+0000","last_became_peered":"2023-10-10T20:41:50.320001+0000","last_unstale":"2023-10-10T20:41:50.325048+0000","last_undegraded":"2023-10-10T20:41:50.325048+0000","last_fullsized":"2023-10-10T20:41:50.325048+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:59.707661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"137'2","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327672+0000","last_change":"2023-10-10T20:41:49.314907+0000","last_active":"2023-10-10T20:41:50.327672+0000","last_peered":"2023-10-10T20:41:50.327672+0000","last_clean":"2023-10-10T20:41:50.327672+0000","last_became_active":"2023-10-10T20:41:49.314607+0000","last_became_peered":"2023-10-10T20:41:49.314607+0000","last_unstale":"2023-10-10T20:41:50.327672+0000","last_undegraded":"2023-10-10T20:41:50.327672+0000","last_fullsized":"2023-10-10T20:41:50.327672+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:04:21.204106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.325850+0000","last_change":"2023-10-10T20:41:50.320973+0000","last_active":"2023-10-10T20:41:50.325850+0000","last_peered":"2023-10-10T20:41:50.325850+0000","last_clean":"2023-10-10T20:41:50.325850+0000","last_became_active":"2023-10-10T20:41:50.320708+0000","last_became_peered":"2023-10-10T20:41:50.320708+0000","last_unstale":"2023-10-10T20:41:50.325850+0000","last_undegraded":"2023-10-10T20:41:50.325850+0000","last_fullsized":"2023-10-10T20:41:50.325850+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:09:42.528347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.312504+0000","last_change":"2023-10-10T20:41:49.313210+0000","last_active":"2023-10-10T20:41:50.312504+0000","last_peered":"2023-10-10T20:41:50.312504+0000","last_clean":"2023-10-10T20:41:50.312504+0000","last_became_active":"2023-10-10T20:41:49.312963+0000","last_became_peered":"2023-10-10T20:41:49.312963+0000","last_unstale":"2023-10-10T20:41:50.312504+0000","last_undegraded":"2023-10-10T20:41:50.312504+0000","last_fullsized":"2023-10-10T20:41:50.312504+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:52.358088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.326056+0000","last_change":"2023-10-10T20:41:50.322767+0000","last_active":"2023-10-10T20:41:50.326056+0000","last_peered":"2023-10-10T20:41:50.326056+0000","last_clean":"2023-10-10T20:41:50.326056+0000","last_became_active":"2023-10-10T20:41:50.322502+0000","last_became_peered":"2023-10-10T20:41:50.322502+0000","last_unstale":"2023-10-10T20:41:50.326056+0000","last_undegraded":"2023-10-10T20:41:50.326056+0000","last_fullsized":"2023-10-10T20:41:50.326056+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:15:18.258032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.306854+0000","last_change":"2023-10-10T20:41:49.317998+0000","last_active":"2023-10-10T20:41:50.306854+0000","last_peered":"2023-10-10T20:41:50.306854+0000","last_clean":"2023-10-10T20:41:50.306854+0000","last_became_active":"2023-10-10T20:41:49.317703+0000","last_became_peered":"2023-10-10T20:41:49.317703+0000","last_unstale":"2023-10-10T20:41:50.306854+0000","last_undegraded":"2023-10-10T20:41:50.306854+0000","last_fullsized":"2023-10-10T20:41:50.306854+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:39:49.147457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.320938+0000","last_change":"2023-10-10T20:41:50.317758+0000","last_active":"2023-10-10T20:41:50.320938+0000","last_peered":"2023-10-10T20:41:50.320938+0000","last_clean":"2023-10-10T20:41:50.320938+0000","last_became_active":"2023-10-10T20:41:50.317500+0000","last_became_peered":"2023-10-10T20:41:50.317500+0000","last_unstale":"2023-10-10T20:41:50.320938+0000","last_undegraded":"2023-10-10T20:41:50.320938+0000","last_fullsized":"2023-10-10T20:41:50.320938+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:22:41.479243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"137'8","reported_seq":18,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328528+0000","last_change":"2023-10-10T20:41:49.316006+0000","last_active":"2023-10-10T20:41:50.328528+0000","last_peered":"2023-10-10T20:41:50.328528+0000","last_clean":"2023-10-10T20:41:50.328528+0000","last_became_active":"2023-10-10T20:41:49.315687+0000","last_became_peered":"2023-10-10T20:41:49.315687+0000","last_unstale":"2023-10-10T20:41:50.328528+0000","last_undegraded":"2023-10-10T20:41:50.328528+0000","last_fullsized":"2023-10-10T20:41:50.328528+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:09:56.998292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327185+0000","last_change":"2023-10-10T20:41:50.322343+0000","last_active":"2023-10-10T20:41:50.327185+0000","last_peered":"2023-10-10T20:41:50.327185+0000","last_clean":"2023-10-10T20:41:50.327185+0000","last_became_active":"2023-10-10T20:41:50.322089+0000","last_became_peered":"2023-10-10T20:41:50.322089+0000","last_unstale":"2023-10-10T20:41:50.327185+0000","last_undegraded":"2023-10-10T20:41:50.327185+0000","last_fullsized":"2023-10-10T20:41:50.327185+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:28:58.113040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"137'1","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329391+0000","last_change":"2023-10-10T20:41:49.316983+0000","last_active":"2023-10-10T20:41:50.329391+0000","last_peered":"2023-10-10T20:41:50.329391+0000","last_clean":"2023-10-10T20:41:50.329391+0000","last_became_active":"2023-10-10T20:41:49.316318+0000","last_became_peered":"2023-10-10T20:41:49.316318+0000","last_unstale":"2023-10-10T20:41:50.329391+0000","last_undegraded":"2023-10-10T20:41:50.329391+0000","last_fullsized":"2023-10-10T20:41:50.329391+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:35:39.211243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.326095+0000","last_change":"2023-10-10T20:41:50.323324+0000","last_active":"2023-10-10T20:41:50.326095+0000","last_peered":"2023-10-10T20:41:50.326095+0000","last_clean":"2023-10-10T20:41:50.326095+0000","last_became_active":"2023-10-10T20:41:50.323065+0000","last_became_peered":"2023-10-10T20:41:50.323065+0000","last_unstale":"2023-10-10T20:41:50.326095+0000","last_undegraded":"2023-10-10T20:41:50.326095+0000","last_fullsized":"2023-10-10T20:41:50.326095+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:44.651808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"137'2","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.326104+0000","last_change":"2023-10-10T20:41:49.308012+0000","last_active":"2023-10-10T20:41:50.326104+0000","last_peered":"2023-10-10T20:41:50.326104+0000","last_clean":"2023-10-10T20:41:50.326104+0000","last_became_active":"2023-10-10T20:41:49.307712+0000","last_became_peered":"2023-10-10T20:41:49.307712+0000","last_unstale":"2023-10-10T20:41:50.326104+0000","last_undegraded":"2023-10-10T20:41:50.326104+0000","last_fullsized":"2023-10-10T20:41:50.326104+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:34:34.956156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.315166+0000","last_change":"2023-10-10T20:41:50.309435+0000","last_active":"2023-10-10T20:41:50.315166+0000","last_peered":"2023-10-10T20:41:50.315166+0000","last_clean":"2023-10-10T20:41:50.315166+0000","last_became_active":"2023-10-10T20:41:50.309084+0000","last_became_peered":"2023-10-10T20:41:50.309084+0000","last_unstale":"2023-10-10T20:41:50.315166+0000","last_undegraded":"2023-10-10T20:41:50.315166+0000","last_fullsized":"2023-10-10T20:41:50.315166+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:09:21.122158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"137'1","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327050+0000","last_change":"2023-10-10T20:41:49.314806+0000","last_active":"2023-10-10T20:41:50.327050+0000","last_peered":"2023-10-10T20:41:50.327050+0000","last_clean":"2023-10-10T20:41:50.327050+0000","last_became_active":"2023-10-10T20:41:49.314532+0000","last_became_peered":"2023-10-10T20:41:49.314532+0000","last_unstale":"2023-10-10T20:41:50.327050+0000","last_undegraded":"2023-10-10T20:41:50.327050+0000","last_fullsized":"2023-10-10T20:41:50.327050+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:43:37.542864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.332248+0000","last_change":"2023-10-10T20:41:50.329246+0000","last_active":"2023-10-10T20:41:50.332248+0000","last_peered":"2023-10-10T20:41:50.332248+0000","last_clean":"2023-10-10T20:41:50.332248+0000","last_became_active":"2023-10-10T20:41:50.328957+0000","last_became_peered":"2023-10-10T20:41:50.328957+0000","last_unstale":"2023-10-10T20:41:50.332248+0000","last_undegraded":"2023-10-10T20:41:50.332248+0000","last_fullsized":"2023-10-10T20:41:50.332248+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:24:36.913433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.305148+0000","last_change":"2023-10-10T20:41:49.312016+0000","last_active":"2023-10-10T20:41:50.305148+0000","last_peered":"2023-10-10T20:41:50.305148+0000","last_clean":"2023-10-10T20:41:50.305148+0000","last_became_active":"2023-10-10T20:41:49.311737+0000","last_became_peered":"2023-10-10T20:41:49.311737+0000","last_unstale":"2023-10-10T20:41:50.305148+0000","last_undegraded":"2023-10-10T20:41:50.305148+0000","last_fullsized":"2023-10-10T20:41:50.305148+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:40:21.122555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327289+0000","last_change":"2023-10-10T20:41:50.324679+0000","last_active":"2023-10-10T20:41:50.327289+0000","last_peered":"2023-10-10T20:41:50.327289+0000","last_clean":"2023-10-10T20:41:50.327289+0000","last_became_active":"2023-10-10T20:41:50.324434+0000","last_became_peered":"2023-10-10T20:41:50.324434+0000","last_unstale":"2023-10-10T20:41:50.327289+0000","last_undegraded":"2023-10-10T20:41:50.327289+0000","last_fullsized":"2023-10-10T20:41:50.327289+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:21.594296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"137'2","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329850+0000","last_change":"2023-10-10T20:41:49.318176+0000","last_active":"2023-10-10T20:41:50.329850+0000","last_peered":"2023-10-10T20:41:50.329850+0000","last_clean":"2023-10-10T20:41:50.329850+0000","last_became_active":"2023-10-10T20:41:49.317925+0000","last_became_peered":"2023-10-10T20:41:49.317925+0000","last_unstale":"2023-10-10T20:41:50.329850+0000","last_undegraded":"2023-10-10T20:41:50.329850+0000","last_fullsized":"2023-10-10T20:41:50.329850+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:24.136950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329156+0000","last_change":"2023-10-10T20:41:50.324209+0000","last_active":"2023-10-10T20:41:50.329156+0000","last_peered":"2023-10-10T20:41:50.329156+0000","last_clean":"2023-10-10T20:41:50.329156+0000","last_became_active":"2023-10-10T20:41:50.323891+0000","last_became_peered":"2023-10-10T20:41:50.323891+0000","last_unstale":"2023-10-10T20:41:50.329156+0000","last_undegraded":"2023-10-10T20:41:50.329156+0000","last_fullsized":"2023-10-10T20:41:50.329156+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:28:06.266010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.308876+0000","last_change":"2023-10-10T20:41:49.317374+0000","last_active":"2023-10-10T20:41:50.308876+0000","last_peered":"2023-10-10T20:41:50.308876+0000","last_clean":"2023-10-10T20:41:50.308876+0000","last_became_active":"2023-10-10T20:41:49.317022+0000","last_became_peered":"2023-10-10T20:41:49.317022+0000","last_unstale":"2023-10-10T20:41:50.308876+0000","last_undegraded":"2023-10-10T20:41:50.308876+0000","last_fullsized":"2023-10-10T20:41:50.308876+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:44.809357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.324040+0000","last_change":"2023-10-10T20:41:50.320161+0000","last_active":"2023-10-10T20:41:50.324040+0000","last_peered":"2023-10-10T20:41:50.324040+0000","last_clean":"2023-10-10T20:41:50.324040+0000","last_became_active":"2023-10-10T20:41:50.319896+0000","last_became_peered":"2023-10-10T20:41:50.319896+0000","last_unstale":"2023-10-10T20:41:50.324040+0000","last_undegraded":"2023-10-10T20:41:50.324040+0000","last_fullsized":"2023-10-10T20:41:50.324040+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:28:06.266010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.310033+0000","last_change":"2023-10-10T20:41:49.318718+0000","last_active":"2023-10-10T20:41:50.310033+0000","last_peered":"2023-10-10T20:41:50.310033+0000","last_clean":"2023-10-10T20:41:50.310033+0000","last_became_active":"2023-10-10T20:41:49.318480+0000","last_became_peered":"2023-10-10T20:41:49.318480+0000","last_unstale":"2023-10-10T20:41:50.310033+0000","last_undegraded":"2023-10-10T20:41:50.310033+0000","last_fullsized":"2023-10-10T20:41:50.310033+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:57.541968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327533+0000","last_change":"2023-10-10T20:41:50.321470+0000","last_active":"2023-10-10T20:41:50.327533+0000","last_peered":"2023-10-10T20:41:50.327533+0000","last_clean":"2023-10-10T20:41:50.327533+0000","last_became_active":"2023-10-10T20:41:50.315093+0000","last_became_peered":"2023-10-10T20:41:50.315093+0000","last_unstale":"2023-10-10T20:41:50.327533+0000","last_undegraded":"2023-10-10T20:41:50.327533+0000","last_fullsized":"2023-10-10T20:41:50.327533+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:42:13.861221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.302813+0000","last_change":"2023-10-10T20:41:49.314130+0000","last_active":"2023-10-10T20:41:50.302813+0000","last_peered":"2023-10-10T20:41:50.302813+0000","last_clean":"2023-10-10T20:41:50.302813+0000","last_became_active":"2023-10-10T20:41:49.313882+0000","last_became_peered":"2023-10-10T20:41:49.313882+0000","last_unstale":"2023-10-10T20:41:50.302813+0000","last_undegraded":"2023-10-10T20:41:50.302813+0000","last_fullsized":"2023-10-10T20:41:50.302813+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:51:13.831963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.323070+0000","last_change":"2023-10-10T20:41:50.313028+0000","last_active":"2023-10-10T20:41:50.323070+0000","last_peered":"2023-10-10T20:41:50.323070+0000","last_clean":"2023-10-10T20:41:50.323070+0000","last_became_active":"2023-10-10T20:41:50.312757+0000","last_became_peered":"2023-10-10T20:41:50.312757+0000","last_unstale":"2023-10-10T20:41:50.323070+0000","last_undegraded":"2023-10-10T20:41:50.323070+0000","last_fullsized":"2023-10-10T20:41:50.323070+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:51:14.837731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"137'2","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.319856+0000","last_change":"2023-10-10T20:41:49.318208+0000","last_active":"2023-10-10T20:41:50.319856+0000","last_peered":"2023-10-10T20:41:50.319856+0000","last_clean":"2023-10-10T20:41:50.319856+0000","last_became_active":"2023-10-10T20:41:49.317937+0000","last_became_peered":"2023-10-10T20:41:49.317937+0000","last_unstale":"2023-10-10T20:41:50.319856+0000","last_undegraded":"2023-10-10T20:41:50.319856+0000","last_fullsized":"2023-10-10T20:41:50.319856+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:37:13.352726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.326844+0000","last_change":"2023-10-10T20:41:50.321622+0000","last_active":"2023-10-10T20:41:50.326844+0000","last_peered":"2023-10-10T20:41:50.326844+0000","last_clean":"2023-10-10T20:41:50.326844+0000","last_became_active":"2023-10-10T20:41:50.321200+0000","last_became_peered":"2023-10-10T20:41:50.321200+0000","last_unstale":"2023-10-10T20:41:50.326844+0000","last_undegraded":"2023-10-10T20:41:50.326844+0000","last_fullsized":"2023-10-10T20:41:50.326844+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:33.972875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-10T20:41:51.500500+0000","last_change":"2023-10-10T20:41:49.315072+0000","last_active":"2023-10-10T20:41:51.500500+0000","last_peered":"2023-10-10T20:41:51.500500+0000","last_clean":"2023-10-10T20:41:51.500500+0000","last_became_active":"2023-10-10T20:41:49.314618+0000","last_became_peered":"2023-10-10T20:41:49.314618+0000","last_unstale":"2023-10-10T20:41:51.500500+0000","last_undegraded":"2023-10-10T20:41:51.500500+0000","last_fullsized":"2023-10-10T20:41:51.500500+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:04:21.204106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.328741+0000","last_change":"2023-10-10T20:41:50.321375+0000","last_active":"2023-10-10T20:41:50.328741+0000","last_peered":"2023-10-10T20:41:50.328741+0000","last_clean":"2023-10-10T20:41:50.328741+0000","last_became_active":"2023-10-10T20:41:50.321099+0000","last_became_peered":"2023-10-10T20:41:50.321099+0000","last_unstale":"2023-10-10T20:41:50.328741+0000","last_undegraded":"2023-10-10T20:41:50.328741+0000","last_fullsized":"2023-10-10T20:41:50.328741+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:21.594296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.315641+0000","last_change":"2023-10-10T20:41:49.310539+0000","last_active":"2023-10-10T20:41:50.315641+0000","last_peered":"2023-10-10T20:41:50.315641+0000","last_clean":"2023-10-10T20:41:50.315641+0000","last_became_active":"2023-10-10T20:41:49.310219+0000","last_became_peered":"2023-10-10T20:41:49.310219+0000","last_unstale":"2023-10-10T20:41:50.315641+0000","last_undegraded":"2023-10-10T20:41:50.315641+0000","last_fullsized":"2023-10-10T20:41:50.315641+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:43:37.542864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327153+0000","last_change":"2023-10-10T20:41:50.318523+0000","last_active":"2023-10-10T20:41:50.327153+0000","last_peered":"2023-10-10T20:41:50.327153+0000","last_clean":"2023-10-10T20:41:50.327153+0000","last_became_active":"2023-10-10T20:41:50.318341+0000","last_became_peered":"2023-10-10T20:41:50.318341+0000","last_unstale":"2023-10-10T20:41:50.327153+0000","last_undegraded":"2023-10-10T20:41:50.327153+0000","last_fullsized":"2023-10-10T20:41:50.327153+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:36:55.761497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.308130+0000","last_change":"2023-10-10T20:41:49.308240+0000","last_active":"2023-10-10T20:41:50.308130+0000","last_peered":"2023-10-10T20:41:50.308130+0000","last_clean":"2023-10-10T20:41:50.308130+0000","last_became_active":"2023-10-10T20:41:49.307687+0000","last_became_peered":"2023-10-10T20:41:49.307687+0000","last_unstale":"2023-10-10T20:41:50.308130+0000","last_undegraded":"2023-10-10T20:41:50.308130+0000","last_fullsized":"2023-10-10T20:41:50.308130+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:22.420406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.313751+0000","last_change":"2023-10-10T20:41:50.307863+0000","last_active":"2023-10-10T20:41:50.313751+0000","last_peered":"2023-10-10T20:41:50.313751+0000","last_clean":"2023-10-10T20:41:50.313751+0000","last_became_active":"2023-10-10T20:41:50.307587+0000","last_became_peered":"2023-10-10T20:41:50.307587+0000","last_unstale":"2023-10-10T20:41:50.313751+0000","last_undegraded":"2023-10-10T20:41:50.313751+0000","last_fullsized":"2023-10-10T20:41:50.313751+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:23:01.318341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.302594+0000","last_change":"2023-10-10T20:41:49.313676+0000","last_active":"2023-10-10T20:41:50.302594+0000","last_peered":"2023-10-10T20:41:50.302594+0000","last_clean":"2023-10-10T20:41:50.302594+0000","last_became_active":"2023-10-10T20:41:49.313425+0000","last_became_peered":"2023-10-10T20:41:49.313425+0000","last_unstale":"2023-10-10T20:41:50.302594+0000","last_undegraded":"2023-10-10T20:41:50.302594+0000","last_fullsized":"2023-10-10T20:41:50.302594+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:02.015895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.318356+0000","last_change":"2023-10-10T20:41:50.315515+0000","last_active":"2023-10-10T20:41:50.318356+0000","last_peered":"2023-10-10T20:41:50.318356+0000","last_clean":"2023-10-10T20:41:50.318356+0000","last_became_active":"2023-10-10T20:41:50.314937+0000","last_became_peered":"2023-10-10T20:41:50.314937+0000","last_unstale":"2023-10-10T20:41:50.318356+0000","last_undegraded":"2023-10-10T20:41:50.318356+0000","last_fullsized":"2023-10-10T20:41:50.318356+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:23.426173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.302835+0000","last_change":"2023-10-10T20:41:49.313069+0000","last_active":"2023-10-10T20:41:50.302835+0000","last_peered":"2023-10-10T20:41:50.302835+0000","last_clean":"2023-10-10T20:41:50.302835+0000","last_became_active":"2023-10-10T20:41:49.312729+0000","last_became_peered":"2023-10-10T20:41:49.312729+0000","last_unstale":"2023-10-10T20:41:50.302835+0000","last_undegraded":"2023-10-10T20:41:50.302835+0000","last_fullsized":"2023-10-10T20:41:50.302835+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:42:12.855453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.323987+0000","last_change":"2023-10-10T20:41:50.319309+0000","last_active":"2023-10-10T20:41:50.323987+0000","last_peered":"2023-10-10T20:41:50.323987+0000","last_clean":"2023-10-10T20:41:50.323987+0000","last_became_active":"2023-10-10T20:41:50.318090+0000","last_became_peered":"2023-10-10T20:41:50.318090+0000","last_unstale":"2023-10-10T20:41:50.323987+0000","last_undegraded":"2023-10-10T20:41:50.323987+0000","last_fullsized":"2023-10-10T20:41:50.323987+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:09.931645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.310889+0000","last_change":"2023-10-10T20:41:49.317299+0000","last_active":"2023-10-10T20:41:50.310889+0000","last_peered":"2023-10-10T20:41:50.310889+0000","last_clean":"2023-10-10T20:41:50.310889+0000","last_became_active":"2023-10-10T20:41:49.316536+0000","last_became_peered":"2023-10-10T20:41:49.316536+0000","last_unstale":"2023-10-10T20:41:50.310889+0000","last_undegraded":"2023-10-10T20:41:50.310889+0000","last_fullsized":"2023-10-10T20:41:50.310889+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:16:24.814273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.329532+0000","last_change":"2023-10-10T20:41:50.324946+0000","last_active":"2023-10-10T20:41:50.329532+0000","last_peered":"2023-10-10T20:41:50.329532+0000","last_clean":"2023-10-10T20:41:50.329532+0000","last_became_active":"2023-10-10T20:41:50.324655+0000","last_became_peered":"2023-10-10T20:41:50.324655+0000","last_unstale":"2023-10-10T20:41:50.329532+0000","last_undegraded":"2023-10-10T20:41:50.329532+0000","last_fullsized":"2023-10-10T20:41:50.329532+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:02.059224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.309870+0000","last_change":"2023-10-10T20:41:49.312704+0000","last_active":"2023-10-10T20:41:50.309870+0000","last_peered":"2023-10-10T20:41:50.309870+0000","last_clean":"2023-10-10T20:41:50.309870+0000","last_became_active":"2023-10-10T20:41:49.312386+0000","last_became_peered":"2023-10-10T20:41:49.312386+0000","last_unstale":"2023-10-10T20:41:50.309870+0000","last_undegraded":"2023-10-10T20:41:50.309870+0000","last_fullsized":"2023-10-10T20:41:50.309870+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:10:58.865055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.327147+0000","last_change":"2023-10-10T20:41:50.325233+0000","last_active":"2023-10-10T20:41:50.327147+0000","last_peered":"2023-10-10T20:41:50.327147+0000","last_clean":"2023-10-10T20:41:50.327147+0000","last_became_active":"2023-10-10T20:41:50.324824+0000","last_became_peered":"2023-10-10T20:41:50.324824+0000","last_unstale":"2023-10-10T20:41:50.327147+0000","last_undegraded":"2023-10-10T20:41:50.327147+0000","last_fullsized":"2023-10-10T20:41:50.327147+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:49.291991+0000","last_clean_scrub_stamp":"2023-10-10T20:41:49.291991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:36:51.689597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.303274+0000","last_change":"2023-10-10T20:41:49.317251+0000","last_active":"2023-10-10T20:41:50.303274+0000","last_peered":"2023-10-10T20:41:50.303274+0000","last_clean":"2023-10-10T20:41:50.303274+0000","last_became_active":"2023-10-10T20:41:49.316927+0000","last_became_peered":"2023-10-10T20:41:49.316927+0000","last_unstale":"2023-10-10T20:41:50.303274+0000","last_undegraded":"2023-10-10T20:41:50.303274+0000","last_fullsized":"2023-10-10T20:41:50.303274+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:41:48.286224+0000","last_clean_scrub_stamp":"2023-10-10T20:41:48.286224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:35:52.358088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":281,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-10T20:41:50.303736+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:41:50.303736+0000","last_peered":"2023-10-10T20:41:50.303736+0000","last_clean":"2023-10-10T20:41:50.303736+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:41:50.303736+0000","last_undegraded":"2023-10-10T20:41:50.303736+0000","last_fullsized":"2023-10-10T20:41:50.303736+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303676,"kb_used_data":2292,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068164,"statfs":{"total":96636764160,"available":96325799936,"internally_reserved":0,"allocated":2347008,"data_stored":1371439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.79000000000000004},"min":{"1min":0.309,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71799999999999997,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52400000000000002,"15min":0.61599999999999999},"min":{"1min":0.32300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.69999999999999996,"5min":0.82099999999999995,"15min":4.1959999999999997},"last":0.39100000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.95099999999999996},"min":{"1min":0.379,"5min":0.317,"15min":0.317},"max":{"1min":0.78300000000000003,"5min":0.78800000000000003,"15min":10.1},"last":0.443},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55200000000000005,"15min":0.71399999999999997},"min":{"1min":0.36699999999999999,"5min":0.32300000000000001,"15min":0.28999999999999998},"max":{"1min":0.73199999999999998,"5min":0.90400000000000003,"15min":6.806},"last":0.46999999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.76400000000000001},"min":{"1min":0.40100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79200000000000004,"5min":0.89000000000000001,"15min":6.9340000000000002},"last":0.499},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.71699999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.73299999999999998,"5min":0.83899999999999997,"15min":6.9870000000000001},"last":0.41999999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54900000000000004,"15min":0.73599999999999999},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.753,"5min":0.80200000000000005,"15min":6.6719999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56699999999999995,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.71699999999999997,"5min":0.87,"15min":4.1609999999999996},"last":0.433}]},{"osd":4,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.36599999999999999,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.65100000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36299999999999999,"5min":0.309,"15min":0.30399999999999999},"max":{"1min":0.72299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.32300000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.314},"max":{"1min":0.77900000000000003,"5min":0.86099999999999999,"15min":1.3180000000000001},"last":0.307},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.41799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71499999999999997,"5min":0.75800000000000001,"15min":0.84099999999999997},"last":0.35999999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":0.78700000000000003,"15min":1.7509999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.40899999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.85699999999999998,"15min":1.7090000000000001},"last":0.40300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574943,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":780967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51300000000000001,"15min":0.65500000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":7.7590000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.52100000000000002,"15min":0.69199999999999995},"min":{"1min":0.32200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.075,"5min":1.075,"15min":7.8090000000000002},"last":0.66500000000000004}]},{"osd":1,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.69099999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.70199999999999996,"5min":0.80000000000000004,"15min":7.867},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.85599999999999998},"min":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.624}]},{"osd":2,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.58799999999999997},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.56100000000000005,"15min":0.81999999999999995},"min":{"1min":0.34200000000000003,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":2.641,"5min":2.641,"15min":6.7629999999999999},"last":0.495}]},{"osd":3,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54300000000000004,"15min":0.61099999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.107,"5min":1.107,"15min":4.8620000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.70099999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.718,"5min":1.718,"15min":7.9009999999999998},"last":0.51000000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.47299999999999998},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.51700000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52700000000000002,"15min":0.495},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.432},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.53400000000000003},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.41499999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55800000000000005,"15min":0.52500000000000002},"min":{"1min":0.36099999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.372,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.48299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303100,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068740,"statfs":{"total":96636764160,"available":96326389760,"internally_reserved":0,"allocated":1757184,"data_stored":782503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.312,"15min":0.45900000000000002},"min":{"1min":0.22800000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.437,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":1.194},"last":0.35699999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.24199999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51000000000000001,"5min":0.51400000000000001,"15min":1.2869999999999999},"last":0.254},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.48999999999999999},"min":{"1min":0.27200000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.52800000000000002,"5min":0.54200000000000004,"15min":5.0919999999999996},"last":0.379}]},{"osd":3,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36399999999999999,"15min":0.51200000000000001},"min":{"1min":0.29399999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35699999999999998,"15min":0.40200000000000002},"min":{"1min":0.23799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":1.2170000000000001},"last":0.30199999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.73999999999999999,"5min":0.92400000000000004,"15min":1.254},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.68999999999999995},"min":{"1min":0.39300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68799999999999994,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.54400000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.36099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":0.97599999999999998,"15min":1.1799999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56899999999999995,"15min":0.60299999999999998},"min":{"1min":0.41799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71399999999999997,"5min":0.878,"15min":1.3109999999999999},"last":0.73299999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.81799999999999995},"min":{"1min":0.46000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78900000000000003,"5min":0.995,"15min":5.0659999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80700000000000005,"5min":0.89800000000000002,"15min":1.171},"last":0.68700000000000006}]},{"osd":7,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57999999999999996,"15min":0.61299999999999999},"min":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77300000000000002,"5min":1.004,"15min":1.214},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.70999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78200000000000003,"5min":0.90500000000000003,"15min":4.0499999999999998},"last":0.67400000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303668,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068172,"statfs":{"total":96636764160,"available":96325808128,"internally_reserved":0,"allocated":2338816,"data_stored":1370396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.316,"15min":0.50800000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":7.9249999999999998},"last":0.246},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.48699999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":6.6299999999999999},"last":0.33600000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.33100000000000002,"15min":0.53100000000000003},"min":{"1min":0.23100000000000001,"5min":0.22,"15min":0.216},"max":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":8.2509999999999994},"last":0.314},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.51100000000000001},"min":{"1min":0.253,"5min":0.222,"15min":0.222},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":6.7370000000000001},"last":0.29099999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.35399999999999998,"15min":0.53900000000000003},"min":{"1min":0.247,"5min":0.23100000000000001,"15min":0.21099999999999999},"max":{"1min":0.53800000000000003,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.59099999999999997},"min":{"1min":0.27700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":7.8929999999999998},"last":0.35899999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57099999999999995,"15min":0.70299999999999996},"min":{"1min":0.45500000000000002,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.82499999999999996,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57299999999999995,"15min":0.754},"min":{"1min":0.441,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.80900000000000005,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.58699999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56100000000000005,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71299999999999997,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.76100000000000001},"min":{"1min":0.48399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.502}]},{"osd":6,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61499999999999999,"15min":0.80000000000000004},"min":{"1min":0.52200000000000002,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.83299999999999996,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.62,"15min":0.749},"min":{"1min":0.47399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82599999999999996,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.57799999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.60099999999999998,"15min":0.73999999999999999},"min":{"1min":0.48399999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85399999999999998,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.76800000000000002},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.75700000000000001,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.51800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574943,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303100,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068740,"statfs":{"total":96636764160,"available":96326389760,"internally_reserved":0,"allocated":1757184,"data_stored":781633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.311,"15min":0.32500000000000001},"min":{"1min":0.214,"5min":0.189,"15min":0.189},"max":{"1min":0.52500000000000002,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.35099999999999998},"min":{"1min":0.219,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.746},"last":0.312}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.69099999999999995},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52100000000000002,"5min":0.63,"15min":11.967000000000001},"last":0.44},{"interface":"front","average":{"1min":0.371,"5min":0.34899999999999998,"15min":0.46999999999999997},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48899999999999999,"5min":0.53400000000000003,"15min":4.1100000000000003},"last":0.27400000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34699999999999998,"15min":0.53700000000000003},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.50800000000000001,"5min":0.51600000000000001,"15min":6.3040000000000003},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.74299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59299999999999997,"5min":0.65900000000000003,"15min":11.935},"last":0.504}]},{"osd":4,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.55700000000000005,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.38400000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.38,"15min":0.378},"max":{"1min":0.83699999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54200000000000004,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.96099999999999997},"last":0.39700000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57899999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.58599999999999997},"min":{"1min":0.435,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.052},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58099999999999996,"15min":0.76300000000000001},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":6.2649999999999997},"last":0.40999999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574943,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303100,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068740,"statfs":{"total":96636764160,"available":96326389760,"internally_reserved":0,"allocated":1757184,"data_stored":781161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.315,"15min":0.315},"min":{"1min":0.223,"5min":0.21199999999999999,"15min":0.21099999999999999},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32100000000000001},"min":{"1min":0.23999999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48799999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.29499999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.34599999999999997,"15min":0.33300000000000002},"min":{"1min":0.25700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.379},"min":{"1min":0.29199999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.72099999999999997},"last":0.33500000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.36199999999999999,"15min":0.35299999999999998},"min":{"1min":0.25800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57599999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.28999999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.50900000000000001,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.315}]},{"osd":4,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.53400000000000003},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77200000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.745,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.503}]},{"osd":5,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57299999999999995,"15min":0.55200000000000005},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.35899999999999999},"max":{"1min":0.82399999999999995,"5min":1.347,"15min":1.347},"last":0.62},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.42199999999999999,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.83499999999999996,"5min":1.01,"15min":1.01},"last":0.53400000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.48399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86299999999999999,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.81200000000000006,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.56699999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78700000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.83699999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.54800000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303104,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068736,"statfs":{"total":96636764160,"available":96326385664,"internally_reserved":0,"allocated":1761280,"data_stored":783057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.32400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66600000000000004,"5min":0.74199999999999999,"15min":0.747},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51700000000000002,"15min":0.74099999999999999},"min":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77600000000000002,"5min":0.78400000000000003,"15min":7.6719999999999997},"last":0.68000000000000005}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52000000000000002,"15min":0.748},"min":{"1min":0.32000000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.755,"15min":7.7160000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66100000000000003,"5min":0.76900000000000002,"15min":0.86799999999999999},"last":0.55400000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.73899999999999999,"5min":0.79800000000000004,"15min":7.7839999999999998},"last":0.625}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.39200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48599999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.49099999999999999},"min":{"1min":0.35199999999999998,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58699999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.85199999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58399999999999996,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.69999999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.41399999999999998,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.65600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574943,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":780967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.47599999999999998,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.66100000000000003,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.51800000000000002},"min":{"1min":0.33000000000000002,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.68899999999999995,"5min":0.76700000000000002,"15min":0.79300000000000004},"last":0.72399999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51300000000000001,"15min":0.628},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63400000000000001,"5min":0.75700000000000001,"15min":5.2679999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.62},"min":{"1min":0.40500000000000003,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79200000000000004,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52400000000000002,"15min":0.63700000000000001},"min":{"1min":0.39100000000000001,"5min":0.32900000000000001,"15min":0.28299999999999997},"max":{"1min":0.73299999999999998,"5min":0.76600000000000001,"15min":5.2960000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.38500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.61599999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55900000000000005,"15min":0.66900000000000004},"min":{"1min":0.41999999999999998,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":5.3330000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56399999999999995,"15min":0.68000000000000005},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.27700000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.67100000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.48899999999999999,"15min":0.498},"min":{"1min":0.33400000000000002,"5min":0.29599999999999999,"15min":0.28699999999999998},"max":{"1min":0.67000000000000004,"5min":0.72499999999999998,"15min":0.82999999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.34499999999999997,"15min":0.32200000000000001},"max":{"1min":0.77200000000000002,"5min":0.88,"15min":1.0800000000000001},"last":0.63900000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.39900000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78900000000000003,"5min":0.82499999999999996,"15min":0.89800000000000002},"last":0.59599999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.32300000000000001,"15min":0.30099999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98399999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.74199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:41:59.278 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:41:59.279 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:41:59.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.400+0000 7f2f54a37700 1 Processor -- start 2023-10-10T20:41:59.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.400+0000 7f2f54a37700 1 -- start start 2023-10-10T20:41:59.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.400+0000 7f2f54a37700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f500b0360 0x7f2f500c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:59.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.400+0000 7f2f54a37700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f500adf00 con 0x7f2f500cd5f0 2023-10-10T20:41:59.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.400+0000 7f2f54a37700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f500cda80 con 0x7f2f500cd180 2023-10-10T20:41:59.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.400+0000 7f2f54a37700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500cdd60 con 0x7f2f500b0360 2023-10-10T20:41:59.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.401+0000 7f2f4ffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2f500cd5f0 0x7f2f500ae390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35644/0 (socket says 172.21.15.89:35644) 2023-10-10T20:41:59.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.401+0000 7f2f4ffff700 1 -- 172.21.15.89:0/4051606827 learned_addr learned my addr 172.21.15.89:0/4051606827 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:41:59.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.401+0000 7f2f4f7fe700 1 --2- 172.21.15.89:0/4051606827 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f500b0360 0x7f2f500c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:59.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.401+0000 7f2f4e7fc700 1 -- 172.21.15.89:0/4051606827 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2151735782 0 0) 0x7f2f500adf00 con 0x7f2f500cd5f0 2023-10-10T20:41:59.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.401+0000 7f2f4e7fc700 1 -- 172.21.15.89:0/4051606827 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f38003570 con 0x7f2f500cd5f0 2023-10-10T20:41:59.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.401+0000 7f2f4e7fc700 1 -- 172.21.15.89:0/4051606827 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442324116 0 0) 0x7f2f500cda80 con 0x7f2f500cd180 2023-10-10T20:41:59.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.401+0000 7f2f4e7fc700 1 -- 172.21.15.89:0/4051606827 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f500adf00 con 0x7f2f500cd180 2023-10-10T20:41:59.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.401+0000 7f2f4f7fe700 1 -- 172.21.15.89:0/4051606827 >> v1:172.21.15.111:6789/0 conn(0x7f2f500cd180 legacy=0x7f2f500caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:59.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.401+0000 7f2f4f7fe700 1 -- 172.21.15.89:0/4051606827 >> v1:172.21.15.89:6789/0 conn(0x7f2f500cd5f0 legacy=0x7f2f500ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:59.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.401+0000 7f2f4f7fe700 1 -- 172.21.15.89:0/4051606827 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f500ce040 con 0x7f2f500b0360 2023-10-10T20:41:59.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.401+0000 7f2f4f7fe700 1 --2- 172.21.15.89:0/4051606827 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f500b0360 0x7f2f500c9700 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009e90 tx=0x7f2f40009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccd3d23e19d3e283 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:59.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.401+0000 7f2f4e7fc700 1 -- 172.21.15.89:0/4051606827 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4000e070 con 0x7f2f500b0360 2023-10-10T20:41:59.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.401+0000 7f2f4e7fc700 1 -- 172.21.15.89:0/4051606827 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4000bb80 con 0x7f2f500b0360 2023-10-10T20:41:59.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.401+0000 7f2f4e7fc700 1 -- 172.21.15.89:0/4051606827 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40012430 con 0x7f2f500b0360 2023-10-10T20:41:59.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.402+0000 7f2f54a37700 1 -- 172.21.15.89:0/4051606827 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f500b0360 msgr2=0x7f2f500c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:59.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.402+0000 7f2f54a37700 1 --2- 172.21.15.89:0/4051606827 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f500b0360 0x7f2f500c9700 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009e90 tx=0x7f2f40009780 comp rx=0 tx=0).stop 2023-10-10T20:41:59.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.402+0000 7f2f54a37700 1 -- 172.21.15.89:0/4051606827 shutdown_connections 2023-10-10T20:41:59.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.402+0000 7f2f54a37700 1 --2- 172.21.15.89:0/4051606827 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f500b0360 0x7f2f500c9700 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.402+0000 7f2f54a37700 1 -- 172.21.15.89:0/4051606827 >> 172.21.15.89:0/4051606827 conn(0x7f2f5006baa0 msgr2=0x7f2f5006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:59.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.402+0000 7f2f54a37700 1 -- 172.21.15.89:0/4051606827 shutdown_connections 2023-10-10T20:41:59.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.402+0000 7f2f54a37700 1 -- 172.21.15.89:0/4051606827 wait complete. 2023-10-10T20:41:59.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.403+0000 7f2f54a37700 1 Processor -- start 2023-10-10T20:41:59.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.403+0000 7f2f54a37700 1 -- start start 2023-10-10T20:41:59.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.404+0000 7f2f54a37700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f500b0360 0x7f2f501682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:59.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.404+0000 7f2f54a37700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f500cd180 0x7f2f500b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:59.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.404+0000 7f2f54a37700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f500cd5f0 0x7f2f50155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:59.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.404+0000 7f2f54a37700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f50168810 con 0x7f2f500b0360 2023-10-10T20:41:59.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.404+0000 7f2f54a37700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f50154950 con 0x7f2f500cd180 2023-10-10T20:41:59.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.404+0000 7f2f54a37700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500b3560 con 0x7f2f500cd5f0 2023-10-10T20:41:59.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.404+0000 7f2f4f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f500b0360 0x7f2f501682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:59.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.404+0000 7f2f4f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f500b0360 0x7f2f501682d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43956/0 (socket says 172.21.15.89:43956) 2023-10-10T20:41:59.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.404+0000 7f2f4f7fe700 1 -- 172.21.15.89:0/3460181794 learned_addr learned my addr 172.21.15.89:0/3460181794 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:59.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.404+0000 7f2f4effd700 1 --2- 172.21.15.89:0/3460181794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f500cd180 0x7f2f500b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:59.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.404+0000 7f2f4ffff700 1 --2- 172.21.15.89:0/3460181794 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f500cd5f0 0x7f2f50155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:59.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.404+0000 7f2f4f7fe700 1 -- 172.21.15.89:0/3460181794 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f500cd5f0 msgr2=0x7f2f50155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:59.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.404+0000 7f2f4f7fe700 1 --2- 172.21.15.89:0/3460181794 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f500cd5f0 0x7f2f50155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.404+0000 7f2f4f7fe700 1 -- 172.21.15.89:0/3460181794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f500cd180 msgr2=0x7f2f500b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:59.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.404+0000 7f2f4f7fe700 1 --2- 172.21.15.89:0/3460181794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f500cd180 0x7f2f500b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.404+0000 7f2f4f7fe700 1 -- 172.21.15.89:0/3460181794 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f40009a20 con 0x7f2f500b0360 2023-10-10T20:41:59.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.405+0000 7f2f4f7fe700 1 --2- 172.21.15.89:0/3460181794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f500b0360 0x7f2f501682d0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f2f400097c0 tx=0x7f2f40019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:59.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.405+0000 7f2f377fe700 1 -- 172.21.15.89:0/3460181794 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40012e30 con 0x7f2f500b0360 2023-10-10T20:41:59.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.405+0000 7f2f377fe700 1 -- 172.21.15.89:0/3460181794 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4001b420 con 0x7f2f500b0360 2023-10-10T20:41:59.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.405+0000 7f2f54a37700 1 -- 172.21.15.89:0/3460181794 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f500b3f30 con 0x7f2f500b0360 2023-10-10T20:41:59.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.405+0000 7f2f377fe700 1 -- 172.21.15.89:0/3460181794 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4001b7e0 con 0x7f2f500b0360 2023-10-10T20:41:59.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.405+0000 7f2f54a37700 1 -- 172.21.15.89:0/3460181794 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f500b2c30 con 0x7f2f500b0360 2023-10-10T20:41:59.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.406+0000 7f2f54a37700 1 -- 172.21.15.89:0/3460181794 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f5004c3c0 con 0x7f2f500b0360 2023-10-10T20:41:59.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.406+0000 7f2f377fe700 1 -- 172.21.15.89:0/3460181794 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2f4001bbd0 con 0x7f2f500b0360 2023-10-10T20:41:59.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.406+0000 7f2f377fe700 1 --2- 172.21.15.89:0/3460181794 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2f3c067320 0x7f2f3c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:59.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.406+0000 7f2f377fe700 1 -- 172.21.15.89:0/3460181794 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2f400869a0 con 0x7f2f500b0360 2023-10-10T20:41:59.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.406+0000 7f2f4effd700 1 --2- 172.21.15.89:0/3460181794 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2f3c067320 0x7f2f3c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:59.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.407+0000 7f2f4effd700 1 --2- 172.21.15.89:0/3460181794 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2f3c067320 0x7f2f3c0697d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2f50156710 tx=0x7f2f44009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:59.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.411+0000 7f2f377fe700 1 -- 172.21.15.89:0/3460181794 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2f400534e0 con 0x7f2f500b0360 2023-10-10T20:41:59.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.666+0000 7f2f54a37700 1 -- 172.21.15.89:0/3460181794 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2f50065910 con 0x7f2f500b0360 2023-10-10T20:41:59.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.670+0000 7f2f377fe700 1 -- 172.21.15.89:0/3460181794 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+0 (secure 0 0 0) 0x7f2f40056b00 con 0x7f2f500b0360 2023-10-10T20:41:59.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.672+0000 7f2f54a37700 1 -- 172.21.15.89:0/3460181794 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2f3c067320 msgr2=0x7f2f3c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:59.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.672+0000 7f2f54a37700 1 --2- 172.21.15.89:0/3460181794 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2f3c067320 0x7f2f3c0697d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2f50156710 tx=0x7f2f44009040 comp rx=0 tx=0).stop 2023-10-10T20:41:59.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.672+0000 7f2f54a37700 1 -- 172.21.15.89:0/3460181794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f500b0360 msgr2=0x7f2f501682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:59.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.672+0000 7f2f54a37700 1 --2- 172.21.15.89:0/3460181794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f500b0360 0x7f2f501682d0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f2f400097c0 tx=0x7f2f40019000 comp rx=0 tx=0).stop 2023-10-10T20:41:59.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.672+0000 7f2f54a37700 1 -- 172.21.15.89:0/3460181794 shutdown_connections 2023-10-10T20:41:59.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.672+0000 7f2f54a37700 1 --2- 172.21.15.89:0/3460181794 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f500cd5f0 0x7f2f50155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.672+0000 7f2f54a37700 1 --2- 172.21.15.89:0/3460181794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f500b0360 0x7f2f501682d0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.672+0000 7f2f54a37700 1 --2- 172.21.15.89:0/3460181794 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2f3c067320 0x7f2f3c0697d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.672+0000 7f2f54a37700 1 --2- 172.21.15.89:0/3460181794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f500cd180 0x7f2f500b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.672+0000 7f2f54a37700 1 -- 172.21.15.89:0/3460181794 >> 172.21.15.89:0/3460181794 conn(0x7f2f5006baa0 msgr2=0x7f2f500c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:59.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.672+0000 7f2f54a37700 1 -- 172.21.15.89:0/3460181794 shutdown_connections 2023-10-10T20:41:59.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.672+0000 7f2f54a37700 1 -- 172.21.15.89:0/3460181794 wait complete. 2023-10-10T20:41:59.677 INFO:teuthology.orchestra.run.smithi089.stderr:updated caps for client.0 2023-10-10T20:41:59.686 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:41:59.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.799+0000 7fb6f97c5700 1 Processor -- start 2023-10-10T20:41:59.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.800+0000 7fb6f97c5700 1 -- start start 2023-10-10T20:41:59.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.800+0000 7fb6f97c5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb6f4159890 0x7fb6f415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:59.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.800+0000 7fb6f97c5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb6f4069560 0x7fb6f40699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:59.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.800+0000 7fb6f97c5700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f415a160 con 0x7fb6f4159c60 2023-10-10T20:41:59.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.800+0000 7fb6f97c5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f4069f20 con 0x7fb6f4069560 2023-10-10T20:41:59.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.800+0000 7fb6f97c5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f40d13a0 con 0x7fb6f4159890 2023-10-10T20:41:59.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.800+0000 7fb6f37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb6f4069560 0x7fb6f40699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:59.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.801+0000 7fb6f37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb6f4069560 0x7fb6f40699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43964/0 (socket says 172.21.15.89:43964) 2023-10-10T20:41:59.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.801+0000 7fb6f37fe700 1 -- 172.21.15.89:0/2284780593 learned_addr learned my addr 172.21.15.89:0/2284780593 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:59.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.801+0000 7fb6f2ffd700 1 --2- 172.21.15.89:0/2284780593 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb6f4159890 0x7fb6f415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:59.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.801+0000 7fb6f37fe700 1 -- 172.21.15.89:0/2284780593 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb6f4159890 msgr2=0x7fb6f415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:59.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.801+0000 7fb6f37fe700 1 --2- 172.21.15.89:0/2284780593 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb6f4159890 0x7fb6f415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.801+0000 7fb6f37fe700 1 -- 172.21.15.89:0/2284780593 >> v1:172.21.15.111:6789/0 conn(0x7fb6f4159c60 legacy=0x7fb6f415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:41:59.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.801+0000 7fb6f37fe700 1 -- 172.21.15.89:0/2284780593 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f415e140 con 0x7fb6f4069560 2023-10-10T20:41:59.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.801+0000 7fb6f2ffd700 1 --2- 172.21.15.89:0/2284780593 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb6f4159890 0x7fb6f415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:41:59.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.801+0000 7fb6f37fe700 1 --2- 172.21.15.89:0/2284780593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6f4069560 0x7fb6f40699e0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0099b0 tx=0x7fb6ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1845be92b4b0f8f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.801+0000 7fb6f27fc700 1 -- 172.21.15.89:0/2284780593 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec00b820 con 0x7fb6f4069560 2023-10-10T20:41:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.801+0000 7fb6f27fc700 1 -- 172.21.15.89:0/2284780593 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ec00b980 con 0x7fb6f4069560 2023-10-10T20:41:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.801+0000 7fb6f27fc700 1 -- 172.21.15.89:0/2284780593 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec00bd20 con 0x7fb6f4069560 2023-10-10T20:41:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.802+0000 7fb6f97c5700 1 -- 172.21.15.89:0/2284780593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6f4069560 msgr2=0x7fb6f40699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:59.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.802+0000 7fb6f97c5700 1 --2- 172.21.15.89:0/2284780593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6f4069560 0x7fb6f40699e0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0099b0 tx=0x7fb6ec009250 comp rx=0 tx=0).stop 2023-10-10T20:41:59.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.802+0000 7fb6f97c5700 1 -- 172.21.15.89:0/2284780593 shutdown_connections 2023-10-10T20:41:59.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.802+0000 7fb6f97c5700 1 --2- 172.21.15.89:0/2284780593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6f4069560 0x7fb6f40699e0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.802+0000 7fb6f97c5700 1 --2- 172.21.15.89:0/2284780593 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb6f4159890 0x7fb6f415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.802+0000 7fb6f97c5700 1 -- 172.21.15.89:0/2284780593 >> 172.21.15.89:0/2284780593 conn(0x7fb6f4068e20 msgr2=0x7fb6f40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:41:59.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.802+0000 7fb6f97c5700 1 -- 172.21.15.89:0/2284780593 shutdown_connections 2023-10-10T20:41:59.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.802+0000 7fb6f97c5700 1 -- 172.21.15.89:0/2284780593 wait complete. 2023-10-10T20:41:59.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.803+0000 7fb6f97c5700 1 Processor -- start 2023-10-10T20:41:59.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.803+0000 7fb6f97c5700 1 -- start start 2023-10-10T20:41:59.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6f97c5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6f4069560 0x7fb6f4157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:59.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6f97c5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb6f4159890 0x7fb6f41512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:59.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6f97c5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6f4159c60 0x7fb6f4151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:59.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6f97c5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f4151eb0 con 0x7fb6f4069560 2023-10-10T20:41:59.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6f97c5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f4152190 con 0x7fb6f4159c60 2023-10-10T20:41:59.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6f97c5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f4152470 con 0x7fb6f4159890 2023-10-10T20:41:59.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6f2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6f4069560 0x7fb6f4157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:59.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6f2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6f4069560 0x7fb6f4157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43966/0 (socket says 172.21.15.89:43966) 2023-10-10T20:41:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6f2ffd700 1 -- 172.21.15.89:0/3928793042 learned_addr learned my addr 172.21.15.89:0/3928793042 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:41:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6f37fe700 1 --2- 172.21.15.89:0/3928793042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6f4159c60 0x7fb6f4151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6ea7fc700 1 --2- 172.21.15.89:0/3928793042 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb6f4159890 0x7fb6f41512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6f2ffd700 1 -- 172.21.15.89:0/3928793042 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb6f4159890 msgr2=0x7fb6f41512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6f2ffd700 1 --2- 172.21.15.89:0/3928793042 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb6f4159890 0x7fb6f41512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6f2ffd700 1 -- 172.21.15.89:0/3928793042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6f4159c60 msgr2=0x7fb6f4151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:41:59.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6f2ffd700 1 --2- 172.21.15.89:0/3928793042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6f4159c60 0x7fb6f4151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:41:59.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6f2ffd700 1 -- 172.21.15.89:0/3928793042 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ec0094f0 con 0x7fb6f4069560 2023-10-10T20:41:59.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6f2ffd700 1 --2- 172.21.15.89:0/3928793042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6f4069560 0x7fb6f4157c40 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400a400 tx=0x7fb6e4007910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:59.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6eb7fe700 1 -- 172.21.15.89:0/3928793042 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e400d650 con 0x7fb6f4069560 2023-10-10T20:41:59.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6eb7fe700 1 -- 172.21.15.89:0/3928793042 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e400d7b0 con 0x7fb6f4069560 2023-10-10T20:41:59.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6eb7fe700 1 -- 172.21.15.89:0/3928793042 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e400db50 con 0x7fb6f4069560 2023-10-10T20:41:59.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6f97c5700 1 -- 172.21.15.89:0/3928793042 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f40c5590 con 0x7fb6f4069560 2023-10-10T20:41:59.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.804+0000 7fb6f97c5700 1 -- 172.21.15.89:0/3928793042 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f4158180 con 0x7fb6f4069560 2023-10-10T20:41:59.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.805+0000 7fb6f97c5700 1 -- 172.21.15.89:0/3928793042 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f404c3c0 con 0x7fb6f4069560 2023-10-10T20:41:59.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.806+0000 7fb6eb7fe700 1 -- 172.21.15.89:0/3928793042 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb6e4015440 con 0x7fb6f4069560 2023-10-10T20:41:59.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.806+0000 7fb6eb7fe700 1 --2- 172.21.15.89:0/3928793042 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb6d4067320 0x7fb6d40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:41:59.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.806+0000 7fb6eb7fe700 1 -- 172.21.15.89:0/3928793042 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb6e4085580 con 0x7fb6f4069560 2023-10-10T20:41:59.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.806+0000 7fb6ea7fc700 1 --2- 172.21.15.89:0/3928793042 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb6d4067320 0x7fb6d40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:41:59.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.807+0000 7fb6ea7fc700 1 --2- 172.21.15.89:0/3928793042 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb6d4067320 0x7fb6d40697d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40cad90 tx=0x7fb6dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:41:59.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:41:59.810+0000 7fb6eb7fe700 1 -- 172.21.15.89:0/3928793042 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6e404e5a0 con 0x7fb6f4069560 2023-10-10T20:42:00.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.037+0000 7fb6f97c5700 1 -- 172.21.15.89:0/3928793042 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6f406a5b0 con 0x7fb6f4069560 2023-10-10T20:42:00.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.037+0000 7fb6eb7fe700 1 -- 172.21.15.89:0/3928793042 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fb6e4059e60 con 0x7fb6f4069560 2023-10-10T20:42:00.040 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:00.040 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":91,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6197,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/4123310016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4123310016},{"type":"v1","addr":"172.21.15.111:6834","nonce":4123310016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":6200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/141017824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":141017824},{"type":"v1","addr":"172.21.15.111:6835","nonce":141017824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7575,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/728309352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":728309352},{"type":"v1","addr":"172.21.15.89:6836","nonce":728309352}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:41:49.687845+0000","modified":"2023-10-10T20:41:53.324518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7578":{"gid":7578,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/2102263806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2102263806},{"type":"v1","addr":"172.21.15.89:6837","nonce":2102263806}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-10T20:42:00.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.040+0000 7fb6f97c5700 1 -- 172.21.15.89:0/3928793042 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb6d4067320 msgr2=0x7fb6d40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:00.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.040+0000 7fb6f97c5700 1 --2- 172.21.15.89:0/3928793042 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb6d4067320 0x7fb6d40697d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40cad90 tx=0x7fb6dc009040 comp rx=0 tx=0).stop 2023-10-10T20:42:00.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.040+0000 7fb6f97c5700 1 -- 172.21.15.89:0/3928793042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6f4069560 msgr2=0x7fb6f4157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:00.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.040+0000 7fb6f97c5700 1 --2- 172.21.15.89:0/3928793042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6f4069560 0x7fb6f4157c40 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400a400 tx=0x7fb6e4007910 comp rx=0 tx=0).stop 2023-10-10T20:42:00.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.040+0000 7fb6f97c5700 1 -- 172.21.15.89:0/3928793042 shutdown_connections 2023-10-10T20:42:00.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.040+0000 7fb6f97c5700 1 --2- 172.21.15.89:0/3928793042 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb6d4067320 0x7fb6d40697d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.040+0000 7fb6f97c5700 1 --2- 172.21.15.89:0/3928793042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6f4159c60 0x7fb6f4151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.040+0000 7fb6f97c5700 1 --2- 172.21.15.89:0/3928793042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6f4069560 0x7fb6f4157c40 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.040+0000 7fb6f97c5700 1 --2- 172.21.15.89:0/3928793042 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb6f4159890 0x7fb6f41512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.040+0000 7fb6f97c5700 1 -- 172.21.15.89:0/3928793042 >> 172.21.15.89:0/3928793042 conn(0x7fb6f4068e20 msgr2=0x7fb6f40cf3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:00.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.040+0000 7fb6f97c5700 1 -- 172.21.15.89:0/3928793042 shutdown_connections 2023-10-10T20:42:00.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.040+0000 7fb6f97c5700 1 -- 172.21.15.89:0/3928793042 wait complete. 2023-10-10T20:42:00.045 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 91 2023-10-10T20:42:00.456 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:41:49.687845+0000', 'modified': '2023-10-10T20:41:53.324518+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7578}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7578': {'gid': 7578, 'name': 'a', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6837/2102263806', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6835', 'nonce': 2102263806}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 2102263806}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:42:00.457 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:42:00.457 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-10T20:42:00.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.571+0000 7f2f73f5f700 1 Processor -- start 2023-10-10T20:42:00.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.571+0000 7f2f73f5f700 1 -- start start 2023-10-10T20:42:00.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.572+0000 7f2f73f5f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f6c159890 0x7f2f6c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.572+0000 7f2f73f5f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2f6c069560 0x7f2f6c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.572+0000 7f2f73f5f700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6c15a160 con 0x7f2f6c159c60 2023-10-10T20:42:00.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.572+0000 7f2f73f5f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f6c069f20 con 0x7f2f6c069560 2023-10-10T20:42:00.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.572+0000 7f2f73f5f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0d13a0 con 0x7f2f6c159890 2023-10-10T20:42:00.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.572+0000 7f2f724fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2f6c069560 0x7f2f6c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.572+0000 7f2f724fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2f6c069560 0x7f2f6c0699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43974/0 (socket says 172.21.15.89:43974) 2023-10-10T20:42:00.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.572+0000 7f2f724fc700 1 -- 172.21.15.89:0/1012127087 learned_addr learned my addr 172.21.15.89:0/1012127087 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:00.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.572+0000 7f2f71cfb700 1 --2- 172.21.15.89:0/1012127087 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f6c159890 0x7f2f6c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.572+0000 7f2f724fc700 1 -- 172.21.15.89:0/1012127087 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f6c159890 msgr2=0x7f2f6c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:00.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.572+0000 7f2f724fc700 1 --2- 172.21.15.89:0/1012127087 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f6c159890 0x7f2f6c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.572+0000 7f2f724fc700 1 -- 172.21.15.89:0/1012127087 >> v1:172.21.15.111:6789/0 conn(0x7f2f6c159c60 legacy=0x7f2f6c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:00.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.572+0000 7f2f724fc700 1 -- 172.21.15.89:0/1012127087 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c15e140 con 0x7f2f6c069560 2023-10-10T20:42:00.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.572+0000 7f2f71cfb700 1 --2- 172.21.15.89:0/1012127087 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f6c159890 0x7f2f6c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:00.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.572+0000 7f2f724fc700 1 --2- 172.21.15.89:0/1012127087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f6c069560 0x7f2f6c0699e0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0099b0 tx=0x7f2f5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0beaf3d0ace974d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:00.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.572+0000 7f2f70cf9700 1 -- 172.21.15.89:0/1012127087 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c00b820 con 0x7f2f6c069560 2023-10-10T20:42:00.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.572+0000 7f2f70cf9700 1 -- 172.21.15.89:0/1012127087 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5c00b980 con 0x7f2f6c069560 2023-10-10T20:42:00.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.572+0000 7f2f70cf9700 1 -- 172.21.15.89:0/1012127087 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c00bd20 con 0x7f2f6c069560 2023-10-10T20:42:00.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.573+0000 7f2f73f5f700 1 -- 172.21.15.89:0/1012127087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f6c069560 msgr2=0x7f2f6c0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:00.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.573+0000 7f2f73f5f700 1 --2- 172.21.15.89:0/1012127087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f6c069560 0x7f2f6c0699e0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0099b0 tx=0x7f2f5c009250 comp rx=0 tx=0).stop 2023-10-10T20:42:00.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.573+0000 7f2f73f5f700 1 -- 172.21.15.89:0/1012127087 shutdown_connections 2023-10-10T20:42:00.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.573+0000 7f2f73f5f700 1 --2- 172.21.15.89:0/1012127087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f6c069560 0x7f2f6c0699e0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.573+0000 7f2f73f5f700 1 --2- 172.21.15.89:0/1012127087 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f6c159890 0x7f2f6c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.573+0000 7f2f73f5f700 1 -- 172.21.15.89:0/1012127087 >> 172.21.15.89:0/1012127087 conn(0x7f2f6c068e20 msgr2=0x7f2f6c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:00.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.573+0000 7f2f73f5f700 1 -- 172.21.15.89:0/1012127087 shutdown_connections 2023-10-10T20:42:00.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.573+0000 7f2f73f5f700 1 -- 172.21.15.89:0/1012127087 wait complete. 2023-10-10T20:42:00.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.574+0000 7f2f73f5f700 1 Processor -- start 2023-10-10T20:42:00.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.574+0000 7f2f73f5f700 1 -- start start 2023-10-10T20:42:00.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.575+0000 7f2f73f5f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f6c069560 0x7f2f6c157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.575+0000 7f2f73f5f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f6c159890 0x7f2f6c1512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.575+0000 7f2f73f5f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f6c159c60 0x7f2f6c151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.575+0000 7f2f73f5f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c151eb0 con 0x7f2f6c069560 2023-10-10T20:42:00.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.575+0000 7f2f73f5f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c152190 con 0x7f2f6c159890 2023-10-10T20:42:00.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.575+0000 7f2f73f5f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c152470 con 0x7f2f6c159c60 2023-10-10T20:42:00.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.575+0000 7f2f71cfb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f6c069560 0x7f2f6c157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.575+0000 7f2f71cfb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f6c069560 0x7f2f6c157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43984/0 (socket says 172.21.15.89:43984) 2023-10-10T20:42:00.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.575+0000 7f2f71cfb700 1 -- 172.21.15.89:0/3982321237 learned_addr learned my addr 172.21.15.89:0/3982321237 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:00.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.575+0000 7f2f714fa700 1 --2- 172.21.15.89:0/3982321237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f6c159890 0x7f2f6c1512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.575+0000 7f2f724fc700 1 --2- 172.21.15.89:0/3982321237 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f6c159c60 0x7f2f6c151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.575+0000 7f2f71cfb700 1 -- 172.21.15.89:0/3982321237 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f6c159c60 msgr2=0x7f2f6c151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:00.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.575+0000 7f2f71cfb700 1 --2- 172.21.15.89:0/3982321237 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f6c159c60 0x7f2f6c151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.575+0000 7f2f71cfb700 1 -- 172.21.15.89:0/3982321237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f6c159890 msgr2=0x7f2f6c1512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:00.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.575+0000 7f2f71cfb700 1 --2- 172.21.15.89:0/3982321237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f6c159890 0x7f2f6c1512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.575+0000 7f2f71cfb700 1 -- 172.21.15.89:0/3982321237 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5c0094f0 con 0x7f2f6c069560 2023-10-10T20:42:00.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.575+0000 7f2f714fa700 1 --2- 172.21.15.89:0/3982321237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f6c159890 0x7f2f6c1512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:00.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.575+0000 7f2f724fc700 1 --2- 172.21.15.89:0/3982321237 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f6c159c60 0x7f2f6c151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:00.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.575+0000 7f2f71cfb700 1 --2- 172.21.15.89:0/3982321237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f6c069560 0x7f2f6c157c40 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400c830 tx=0x7f2f64008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:00.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.576+0000 7f2f69ffb700 1 -- 172.21.15.89:0/3982321237 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64011600 con 0x7f2f6c069560 2023-10-10T20:42:00.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.576+0000 7f2f69ffb700 1 -- 172.21.15.89:0/3982321237 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f64011760 con 0x7f2f6c069560 2023-10-10T20:42:00.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.576+0000 7f2f73f5f700 1 -- 172.21.15.89:0/3982321237 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f6c0c5590 con 0x7f2f6c069560 2023-10-10T20:42:00.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.576+0000 7f2f69ffb700 1 -- 172.21.15.89:0/3982321237 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64011b20 con 0x7f2f6c069560 2023-10-10T20:42:00.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.576+0000 7f2f73f5f700 1 -- 172.21.15.89:0/3982321237 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f6c158180 con 0x7f2f6c069560 2023-10-10T20:42:00.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.577+0000 7f2f69ffb700 1 -- 172.21.15.89:0/3982321237 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2f6400a3e0 con 0x7f2f6c069560 2023-10-10T20:42:00.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.578+0000 7f2f69ffb700 1 --2- 172.21.15.89:0/3982321237 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2f58067320 0x7f2f580697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.578+0000 7f2f714fa700 1 --2- 172.21.15.89:0/3982321237 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2f58067320 0x7f2f580697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.578+0000 7f2f69ffb700 1 -- 172.21.15.89:0/3982321237 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2f64085290 con 0x7f2f6c069560 2023-10-10T20:42:00.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.578+0000 7f2f714fa700 1 --2- 172.21.15.89:0/3982321237 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2f58067320 0x7f2f580697d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2f60007fd0 tx=0x7f2f60009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:00.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.580+0000 7f2f7d258b80 1 Processor -- start 2023-10-10T20:42:00.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.580+0000 7f2f7d258b80 1 -- start start 2023-10-10T20:42:00.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.580+0000 7f2f7d258b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae642a50 0x5630ae642e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.580+0000 7f2f7d258b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae68cd00 0x5630ae66ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.580+0000 7f2f7d258b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ae66bf60 0x5630ae63ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.580+0000 7f2f7d258b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ae599a70 con 0x5630ae642a50 2023-10-10T20:42:00.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.580+0000 7f2f7d258b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ae597270 con 0x5630ae66bf60 2023-10-10T20:42:00.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.580+0000 7f2f7d258b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x5630ae593060 con 0x5630ae68cd00 2023-10-10T20:42:00.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.580+0000 7f2f71cfb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae642a50 0x5630ae642e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.580+0000 7f2f71cfb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae642a50 0x5630ae642e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43990/0 (socket says 172.21.15.89:43990) 2023-10-10T20:42:00.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.580+0000 7f2f71cfb700 1 -- 172.21.15.89:0/827762062 learned_addr learned my addr 172.21.15.89:0/827762062 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:00.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.581+0000 7f2f724fc700 1 --2- 172.21.15.89:0/827762062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ae66bf60 0x5630ae63ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.581+0000 7f2f714fa700 1 --2- 172.21.15.89:0/827762062 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae68cd00 0x5630ae66ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.581+0000 7f2f71cfb700 1 -- 172.21.15.89:0/827762062 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae68cd00 msgr2=0x5630ae66ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:00.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.581+0000 7f2f71cfb700 1 --2- 172.21.15.89:0/827762062 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae68cd00 0x5630ae66ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.581+0000 7f2f71cfb700 1 -- 172.21.15.89:0/827762062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ae66bf60 msgr2=0x5630ae63ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:00.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.581+0000 7f2f71cfb700 1 --2- 172.21.15.89:0/827762062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ae66bf60 0x5630ae63ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.581+0000 7f2f71cfb700 1 -- 172.21.15.89:0/827762062 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630ae42d270 con 0x5630ae642a50 2023-10-10T20:42:00.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.581+0000 7f2f714fa700 1 --2- 172.21.15.89:0/827762062 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae68cd00 0x5630ae66ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:00.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.581+0000 7f2f71cfb700 1 --2- 172.21.15.89:0/827762062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae642a50 0x5630ae642e20 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f2f64083160 tx=0x7f2f64051ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2743356abdf7274 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:00.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.581+0000 7f2f52ffd700 1 -- 172.21.15.89:0/827762062 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f640850c0 con 0x5630ae642a50 2023-10-10T20:42:00.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.581+0000 7f2f52ffd700 1 -- 172.21.15.89:0/827762062 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f64058d20 con 0x5630ae642a50 2023-10-10T20:42:00.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.581+0000 7f2f52ffd700 1 -- 172.21.15.89:0/827762062 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6405ca70 con 0x5630ae642a50 2023-10-10T20:42:00.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.581+0000 7f2f7d258b80 1 -- 172.21.15.89:0/827762062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae642a50 msgr2=0x5630ae642e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:00.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.581+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/827762062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae642a50 0x5630ae642e20 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f2f64083160 tx=0x7f2f64051ec0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.582+0000 7f2f7d258b80 1 -- 172.21.15.89:0/827762062 shutdown_connections 2023-10-10T20:42:00.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.582+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/827762062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ae66bf60 0x5630ae63ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.582+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/827762062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae642a50 0x5630ae642e20 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.582+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/827762062 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae68cd00 0x5630ae66ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.582+0000 7f2f7d258b80 1 -- 172.21.15.89:0/827762062 >> 172.21.15.89:0/827762062 conn(0x5630ae644370 msgr2=0x5630ae644740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:00.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.582+0000 7f2f7d258b80 1 -- 172.21.15.89:0/827762062 shutdown_connections 2023-10-10T20:42:00.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.582+0000 7f2f7d258b80 1 -- 172.21.15.89:0/827762062 wait complete. 2023-10-10T20:42:00.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.582+0000 7f2f7d258b80 1 -- start start 2023-10-10T20:42:00.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.582+0000 7f2f7d258b80 1 Processor -- start 2023-10-10T20:42:00.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.583+0000 7f2f7d258b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae642a50 0x5630ae68c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.583+0000 7f2f7d258b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae66bf60 0x5630ae63dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.583+0000 7f2f7d258b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ae68cd00 0x5630ae63e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.583+0000 7f2f7d258b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ae53e890 con 0x5630ae66bf60 2023-10-10T20:42:00.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.583+0000 7f2f7d258b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ae559ea0 con 0x5630ae68cd00 2023-10-10T20:42:00.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.583+0000 7f2f7d258b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x5630ae415e30 con 0x5630ae642a50 2023-10-10T20:42:00.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.583+0000 7f2f7d258b80 10 client.0 _ll_register_callbacks cb 0x5630ae606110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:42:00.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.583+0000 7f2f714fa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae66bf60 0x5630ae63dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.583+0000 7f2f714fa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae66bf60 0x5630ae63dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44000/0 (socket says 172.21.15.89:44000) 2023-10-10T20:42:00.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.583+0000 7f2f714fa700 1 -- 172.21.15.89:0/3726884613 learned_addr learned my addr 172.21.15.89:0/3726884613 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:00.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.583+0000 7f2f71cfb700 1 --2- 172.21.15.89:0/3726884613 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae642a50 0x5630ae68c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.583+0000 7f2f724fc700 1 --2- 172.21.15.89:0/3726884613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ae68cd00 0x5630ae63e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.583+0000 7f2f714fa700 1 -- 172.21.15.89:0/3726884613 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae642a50 msgr2=0x5630ae68c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:00.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.583+0000 7f2f714fa700 1 --2- 172.21.15.89:0/3726884613 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae642a50 0x5630ae68c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.583+0000 7f2f714fa700 1 -- 172.21.15.89:0/3726884613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ae68cd00 msgr2=0x5630ae63e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:00.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.583+0000 7f2f714fa700 1 --2- 172.21.15.89:0/3726884613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ae68cd00 0x5630ae63e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.583+0000 7f2f714fa700 1 -- 172.21.15.89:0/3726884613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f6c158180 con 0x5630ae66bf60 2023-10-10T20:42:00.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.584+0000 7f2f714fa700 1 --2- 172.21.15.89:0/3726884613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae66bf60 0x5630ae63dc20 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f2f60002bd0 tx=0x7f2f60016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b6fb0053563902 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:00.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.584+0000 7f2f4bfff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:42:00.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.584+0000 7f2f4bfff700 1 -- 172.21.15.89:0/3726884613 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f60023070 con 0x5630ae66bf60 2023-10-10T20:42:00.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.584+0000 7f2f4bfff700 1 -- 172.21.15.89:0/3726884613 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2f60006ba0 con 0x5630ae66bf60 2023-10-10T20:42:00.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.584+0000 7f2f7d258b80 1 -- 172.21.15.89:0/3726884613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5630ae637b70 con 0x5630ae66bf60 2023-10-10T20:42:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.584+0000 7f2f4bfff700 1 -- 172.21.15.89:0/3726884613 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f60004130 con 0x5630ae66bf60 2023-10-10T20:42:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.584+0000 7f2f7d258b80 10 client.7746 fetch_fsmap learned FSMap version 91 2023-10-10T20:42:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.584+0000 7f2f7d258b80 1 -- 172.21.15.89:0/3726884613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x5630ae3cffe0 con 0x5630ae66bf60 2023-10-10T20:42:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.584+0000 7f2f4bfff700 1 -- 172.21.15.89:0/3726884613 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5630ae637b70 con 0x5630ae66bf60 2023-10-10T20:42:00.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.584+0000 7f2f4bfff700 1 -- 172.21.15.89:0/3726884613 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 91) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f2f60026020 con 0x5630ae66bf60 2023-10-10T20:42:00.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.584+0000 7f2f7d258b80 10 client.7746 fetch_fsmap finished waiting for FSMap version 91 2023-10-10T20:42:00.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.584+0000 7f2f7d258b80 10 client.7746 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-10T20:42:00.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.584+0000 7f2f7d258b80 20 client.7746 populate_metadata read hostname 'smithi089' 2023-10-10T20:42:00.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.584+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/3726884613 >> [v2:172.21.15.89:6835/2102263806,v1:172.21.15.89:6837/2102263806] conn(0x5630ae667eb0 0x5630ae668280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.584+0000 7f2f7d258b80 4 client.7746 mds_command: new command op to 7578 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:42:00.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.584+0000 7f2f7d258b80 1 -- 172.21.15.89:0/3726884613 --> [v2:172.21.15.89:6835/2102263806,v1:172.21.15.89:6837/2102263806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630ae614aa0 con 0x5630ae667eb0 2023-10-10T20:42:00.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.584+0000 7f2f71cfb700 1 --2- 172.21.15.89:0/3726884613 >> [v2:172.21.15.89:6835/2102263806,v1:172.21.15.89:6837/2102263806] conn(0x5630ae667eb0 0x5630ae668280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.585+0000 7f2f4bfff700 10 client.7746 ms_handle_connect on v2:172.21.15.89:6835/2102263806 2023-10-10T20:42:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.585+0000 7f2f71cfb700 1 --2- 172.21.15.89:0/3726884613 >> [v2:172.21.15.89:6835/2102263806,v1:172.21.15.89:6837/2102263806] conn(0x5630ae667eb0 0x5630ae668280 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf3cdf82edfb7638 server_cookie=72a3645b3f9792c9 in_seq=0 out_seq=0 2023-10-10T20:42:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.585+0000 7f2f4bfff700 1 -- 172.21.15.89:0/3726884613 <== mds.0 v2:172.21.15.89:6835/2102263806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5630ae614aa0 con 0x5630ae667eb0 2023-10-10T20:42:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.585+0000 7f2f4bfff700 10 client.7746 handle_command_reply: tid=0 2023-10-10T20:42:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.585+0000 7f2f7d258b80 1 client.7746 shutdown 2023-10-10T20:42:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.585+0000 7f2f7d258b80 1 -- 172.21.15.89:0/3726884613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae66bf60 msgr2=0x5630ae63dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:00.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.585+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/3726884613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae66bf60 0x5630ae63dc20 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f2f60002bd0 tx=0x7f2f60016040 comp rx=0 tx=0).stop 2023-10-10T20:42:00.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.585+0000 7f2f7d258b80 1 -- 172.21.15.89:0/3726884613 shutdown_connections 2023-10-10T20:42:00.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.585+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/3726884613 >> [v2:172.21.15.89:6835/2102263806,v1:172.21.15.89:6837/2102263806] conn(0x5630ae667eb0 0x5630ae668280 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.585+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/3726884613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ae68cd00 0x5630ae63e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.585+0000 7f2f4bfff700 0 client.7746 ms_handle_reset on v2:172.21.15.89:6835/2102263806 2023-10-10T20:42:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.585+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/3726884613 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae642a50 0x5630ae68c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.585+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/3726884613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae66bf60 0x5630ae63dc20 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.585+0000 7f2f7d258b80 1 -- 172.21.15.89:0/3726884613 >> 172.21.15.89:0/3726884613 conn(0x5630ae644370 msgr2=0x5630ae63f880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.585+0000 7f2f7d258b80 1 -- 172.21.15.89:0/3726884613 shutdown_connections 2023-10-10T20:42:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.585+0000 7f2f7d258b80 1 -- 172.21.15.89:0/3726884613 wait complete. 2023-10-10T20:42:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.586+0000 7f2f7d258b80 20 client.7746 trim_cache size 0 max 16384 2023-10-10T20:42:00.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.602+0000 7f2f7d258b80 1 Processor -- start 2023-10-10T20:42:00.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.602+0000 7f2f7d258b80 1 -- start start 2023-10-10T20:42:00.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.603+0000 7f2f7d258b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae642a50 0x5630ae68b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.603+0000 7f2f7d258b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae66bf60 0x5630ae6a3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.603+0000 7f2f7d258b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ae68cd00 0x5630ae6a3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.603+0000 7f2f7d258b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ae644a10 con 0x5630ae642a50 2023-10-10T20:42:00.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.603+0000 7f2f7d258b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ae63d5a0 con 0x5630ae68cd00 2023-10-10T20:42:00.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.603+0000 7f2f7d258b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x5630ae640470 con 0x5630ae66bf60 2023-10-10T20:42:00.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.603+0000 7f2f71cfb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae642a50 0x5630ae68b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.603+0000 7f2f71cfb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae642a50 0x5630ae68b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44010/0 (socket says 172.21.15.89:44010) 2023-10-10T20:42:00.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.603+0000 7f2f71cfb700 1 -- 172.21.15.89:0/61685532 learned_addr learned my addr 172.21.15.89:0/61685532 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:00.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.603+0000 7f2f71cfb700 1 -- 172.21.15.89:0/61685532 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae66bf60 msgr2=0x5630ae6a3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:00.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.603+0000 7f2f71cfb700 1 --2- 172.21.15.89:0/61685532 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae66bf60 0x5630ae6a3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.603+0000 7f2f71cfb700 1 -- 172.21.15.89:0/61685532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ae68cd00 msgr2=0x5630ae6a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:00.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.603+0000 7f2f71cfb700 1 --2- 172.21.15.89:0/61685532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ae68cd00 0x5630ae6a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.603+0000 7f2f71cfb700 1 -- 172.21.15.89:0/61685532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630ae561780 con 0x5630ae642a50 2023-10-10T20:42:00.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.603+0000 7f2f71cfb700 1 --2- 172.21.15.89:0/61685532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae642a50 0x5630ae68b8f0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400a0a0 tx=0x7f2f6406a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d79eefbeac8b95 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:00.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.603+0000 7f2f537fe700 1 -- 172.21.15.89:0/61685532 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64055d30 con 0x5630ae642a50 2023-10-10T20:42:00.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.604+0000 7f2f537fe700 1 -- 172.21.15.89:0/61685532 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f64055e90 con 0x5630ae642a50 2023-10-10T20:42:00.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.604+0000 7f2f537fe700 1 -- 172.21.15.89:0/61685532 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64056000 con 0x5630ae642a50 2023-10-10T20:42:00.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.604+0000 7f2f7d258b80 1 -- 172.21.15.89:0/61685532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae642a50 msgr2=0x5630ae68b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:00.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.604+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/61685532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae642a50 0x5630ae68b8f0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400a0a0 tx=0x7f2f6406a000 comp rx=0 tx=0).stop 2023-10-10T20:42:00.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.604+0000 7f2f7d258b80 1 -- 172.21.15.89:0/61685532 shutdown_connections 2023-10-10T20:42:00.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.604+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/61685532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ae68cd00 0x5630ae6a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.604+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/61685532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae642a50 0x5630ae68b8f0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.604+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/61685532 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae66bf60 0x5630ae6a3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.604+0000 7f2f7d258b80 1 -- 172.21.15.89:0/61685532 >> 172.21.15.89:0/61685532 conn(0x5630ae644370 msgr2=0x5630ae643710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:00.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.604+0000 7f2f7d258b80 1 -- 172.21.15.89:0/61685532 shutdown_connections 2023-10-10T20:42:00.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.604+0000 7f2f7d258b80 1 -- 172.21.15.89:0/61685532 wait complete. 2023-10-10T20:42:00.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.605+0000 7f2f7d258b80 1 -- start start 2023-10-10T20:42:00.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.605+0000 7f2f7d258b80 1 Processor -- start 2023-10-10T20:42:00.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f7d258b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae642a50 0x5630ae6ef180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f7d258b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae66bf60 0x5630ae6867d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f7d258b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ae68cd00 0x5630ae672050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f7d258b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ae6e5620 con 0x5630ae66bf60 2023-10-10T20:42:00.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f7d258b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ae667160 con 0x5630ae68cd00 2023-10-10T20:42:00.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f7d258b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x5630ae6ef6c0 con 0x5630ae642a50 2023-10-10T20:42:00.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f7d258b80 10 client.0 _ll_register_callbacks cb 0x5630ae606110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:42:00.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f714fa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae66bf60 0x5630ae6867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f714fa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae66bf60 0x5630ae6867d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44018/0 (socket says 172.21.15.89:44018) 2023-10-10T20:42:00.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f714fa700 1 -- 172.21.15.89:0/985185715 learned_addr learned my addr 172.21.15.89:0/985185715 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:00.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f71cfb700 1 --2- 172.21.15.89:0/985185715 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae642a50 0x5630ae6ef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f724fc700 1 --2- 172.21.15.89:0/985185715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ae68cd00 0x5630ae672050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f714fa700 1 -- 172.21.15.89:0/985185715 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae642a50 msgr2=0x5630ae6ef180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:00.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f714fa700 1 --2- 172.21.15.89:0/985185715 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae642a50 0x5630ae6ef180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f714fa700 1 -- 172.21.15.89:0/985185715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ae68cd00 msgr2=0x5630ae672050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:00.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f714fa700 1 --2- 172.21.15.89:0/985185715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ae68cd00 0x5630ae672050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f714fa700 1 -- 172.21.15.89:0/985185715 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5630ae42d270 con 0x5630ae66bf60 2023-10-10T20:42:00.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f71cfb700 1 --2- 172.21.15.89:0/985185715 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae642a50 0x5630ae6ef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:00.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f714fa700 1 --2- 172.21.15.89:0/985185715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae66bf60 0x5630ae6867d0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009010 tx=0x7f2f60018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97330c6b541f0036 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:00.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f4bfff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:42:00.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f4bfff700 1 -- 172.21.15.89:0/985185715 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f60023070 con 0x5630ae66bf60 2023-10-10T20:42:00.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f4bfff700 1 -- 172.21.15.89:0/985185715 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2f6000de40 con 0x5630ae66bf60 2023-10-10T20:42:00.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f7d258b80 1 -- 172.21.15.89:0/985185715 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f64058d20 con 0x5630ae66bf60 2023-10-10T20:42:00.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f4bfff700 1 -- 172.21.15.89:0/985185715 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f60004110 con 0x5630ae66bf60 2023-10-10T20:42:00.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f7d258b80 10 client.7752 fetch_fsmap learned FSMap version 91 2023-10-10T20:42:00.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.606+0000 7f2f7d258b80 1 -- 172.21.15.89:0/985185715 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x5630ae548470 con 0x5630ae66bf60 2023-10-10T20:42:00.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f4bfff700 1 -- 172.21.15.89:0/985185715 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f64058d20 con 0x5630ae66bf60 2023-10-10T20:42:00.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f4bfff700 1 -- 172.21.15.89:0/985185715 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 91) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f2f600147a0 con 0x5630ae66bf60 2023-10-10T20:42:00.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f7d258b80 10 client.7752 fetch_fsmap finished waiting for FSMap version 91 2023-10-10T20:42:00.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f7d258b80 10 client.7752 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-10T20:42:00.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f7d258b80 20 client.7752 populate_metadata read hostname 'smithi089' 2023-10-10T20:42:00.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/985185715 >> [v2:172.21.15.89:6835/2102263806,v1:172.21.15.89:6837/2102263806] conn(0x5630ae686d10 0x5630ae672590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f7d258b80 4 client.7752 mds_command: new command op to 7578 tid=0[{"prefix": "status"}] 2023-10-10T20:42:00.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f7d258b80 1 -- 172.21.15.89:0/985185715 --> [v2:172.21.15.89:6835/2102263806,v1:172.21.15.89:6837/2102263806] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5630ae535e00 con 0x5630ae686d10 2023-10-10T20:42:00.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f71cfb700 1 --2- 172.21.15.89:0/985185715 >> [v2:172.21.15.89:6835/2102263806,v1:172.21.15.89:6837/2102263806] conn(0x5630ae686d10 0x5630ae672590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f4bfff700 10 client.7752 ms_handle_connect on v2:172.21.15.89:6835/2102263806 2023-10-10T20:42:00.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f71cfb700 1 --2- 172.21.15.89:0/985185715 >> [v2:172.21.15.89:6835/2102263806,v1:172.21.15.89:6837/2102263806] conn(0x5630ae686d10 0x5630ae672590 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86cdf027d96df7bc server_cookie=9c0475c3619003f in_seq=0 out_seq=0 2023-10-10T20:42:00.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f4bfff700 1 -- 172.21.15.89:0/985185715 <== mds.0 v2:172.21.15.89:6835/2102263806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x5630ae535e00 con 0x5630ae686d10 2023-10-10T20:42:00.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f4bfff700 10 client.7752 handle_command_reply: tid=0 2023-10-10T20:42:00.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f7d258b80 1 client.7752 shutdown 2023-10-10T20:42:00.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f7d258b80 1 -- 172.21.15.89:0/985185715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae66bf60 msgr2=0x5630ae6867d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:00.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/985185715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae66bf60 0x5630ae6867d0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009010 tx=0x7f2f60018000 comp rx=0 tx=0).stop 2023-10-10T20:42:00.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f7d258b80 1 -- 172.21.15.89:0/985185715 shutdown_connections 2023-10-10T20:42:00.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/985185715 >> [v2:172.21.15.89:6835/2102263806,v1:172.21.15.89:6837/2102263806] conn(0x5630ae686d10 0x5630ae672590 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/985185715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ae68cd00 0x5630ae672050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f4bfff700 0 client.7752 ms_handle_reset on v2:172.21.15.89:6835/2102263806 2023-10-10T20:42:00.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/985185715 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5630ae642a50 0x5630ae6ef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f7d258b80 1 --2- 172.21.15.89:0/985185715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ae66bf60 0x5630ae6867d0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.607+0000 7f2f7d258b80 1 -- 172.21.15.89:0/985185715 >> 172.21.15.89:0/985185715 conn(0x5630ae644370 msgr2=0x5630ae6a2630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:00.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.608+0000 7f2f7d258b80 1 -- 172.21.15.89:0/985185715 shutdown_connections 2023-10-10T20:42:00.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.608+0000 7f2f7d258b80 1 -- 172.21.15.89:0/985185715 wait complete. 2023-10-10T20:42:00.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.608+0000 7f2f7d258b80 20 client.7752 trim_cache size 0 max 16384 2023-10-10T20:42:00.646 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:42:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "5dd675b8-9026-4d8a-a1bf-62fdc5e7de67", 2023-10-10T20:42:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-10-10T20:42:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 7578, 2023-10-10T20:42:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-10-10T20:42:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-10-10T20:42:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-10-10T20:42:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 10.309579715, 2023-10-10T20:42:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 91, 2023-10-10T20:42:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 138, 2023-10-10T20:42:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 138, 2023-10-10T20:42:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 15.407150413 2023-10-10T20:42:00.649 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:42:00.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.608+0000 7f2f73f5f700 1 -- 172.21.15.89:0/3982321237 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2f58067320 msgr2=0x7f2f580697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:00.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.608+0000 7f2f73f5f700 1 --2- 172.21.15.89:0/3982321237 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2f58067320 0x7f2f580697d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2f60007fd0 tx=0x7f2f60009040 comp rx=0 tx=0).stop 2023-10-10T20:42:00.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.608+0000 7f2f73f5f700 1 -- 172.21.15.89:0/3982321237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f6c069560 msgr2=0x7f2f6c157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:00.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.608+0000 7f2f73f5f700 1 --2- 172.21.15.89:0/3982321237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f6c069560 0x7f2f6c157c40 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400c830 tx=0x7f2f64008040 comp rx=0 tx=0).stop 2023-10-10T20:42:00.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.608+0000 7f2f73f5f700 1 -- 172.21.15.89:0/3982321237 shutdown_connections 2023-10-10T20:42:00.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.608+0000 7f2f73f5f700 1 --2- 172.21.15.89:0/3982321237 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2f6c159c60 0x7f2f6c151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.608+0000 7f2f73f5f700 1 --2- 172.21.15.89:0/3982321237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f6c069560 0x7f2f6c157c40 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.608+0000 7f2f73f5f700 1 --2- 172.21.15.89:0/3982321237 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2f58067320 0x7f2f580697d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.608+0000 7f2f73f5f700 1 --2- 172.21.15.89:0/3982321237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f6c159890 0x7f2f6c1512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.608+0000 7f2f73f5f700 1 -- 172.21.15.89:0/3982321237 >> 172.21.15.89:0/3982321237 conn(0x7f2f6c068e20 msgr2=0x7f2f6c0cf3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:00.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.608+0000 7f2f73f5f700 1 -- 172.21.15.89:0/3982321237 shutdown_connections 2023-10-10T20:42:00.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.608+0000 7f2f73f5f700 1 -- 172.21.15.89:0/3982321237 wait complete. 2023-10-10T20:42:00.654 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:42:00.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.768+0000 7fa26dcbf700 1 Processor -- start 2023-10-10T20:42:00.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.768+0000 7fa26dcbf700 1 -- start start 2023-10-10T20:42:00.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa26dcbf700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa268140350 0x7fa268069c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa26dcbf700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa26806a1c0 0x7fa268068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa26dcbf700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa268140720 0x7fa2680688d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa26dcbf700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2680c7de0 con 0x7fa268140350 2023-10-10T20:42:00.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa26dcbf700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa268140c10 con 0x7fa26806a1c0 2023-10-10T20:42:00.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa26dcbf700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa268141490 con 0x7fa268140720 2023-10-10T20:42:00.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa2677fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa268140350 0x7fa268069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa2677fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa268140350 0x7fa268069c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44034/0 (socket says 172.21.15.89:44034) 2023-10-10T20:42:00.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa2677fe700 1 -- 172.21.15.89:0/652867929 learned_addr learned my addr 172.21.15.89:0/652867929 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:00.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa266ffd700 1 --2- 172.21.15.89:0/652867929 >> v2:172.21.15.111:3300/0 conn(0x7fa26806a1c0 0x7fa268068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa2677fe700 1 -- 172.21.15.89:0/652867929 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa268140720 msgr2=0x7fa2680688d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:00.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa2677fe700 1 --2- 172.21.15.89:0/652867929 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa268140720 0x7fa2680688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa2677fe700 1 -- 172.21.15.89:0/652867929 >> v2:172.21.15.111:3300/0 conn(0x7fa26806a1c0 msgr2=0x7fa268068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:00.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa2677fe700 1 --2- 172.21.15.89:0/652867929 >> v2:172.21.15.111:3300/0 conn(0x7fa26806a1c0 0x7fa268068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa2677fe700 1 -- 172.21.15.89:0/652867929 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa268140ed0 con 0x7fa268140350 2023-10-10T20:42:00.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa266ffd700 1 --2- 172.21.15.89:0/652867929 >> v2:172.21.15.111:3300/0 conn(0x7fa26806a1c0 0x7fa268068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:00.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa2677fe700 1 --2- 172.21.15.89:0/652867929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa268140350 0x7fa268069c80 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fa26806f770 tx=0x7fa258009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5140b26497fe587e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:00.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa2667fc700 1 -- 172.21.15.89:0/652867929 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa258005c10 con 0x7fa268140350 2023-10-10T20:42:00.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa2667fc700 1 -- 172.21.15.89:0/652867929 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa258005d70 con 0x7fa268140350 2023-10-10T20:42:00.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.769+0000 7fa2667fc700 1 -- 172.21.15.89:0/652867929 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25800b880 con 0x7fa268140350 2023-10-10T20:42:00.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.770+0000 7fa26dcbf700 1 -- 172.21.15.89:0/652867929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa268140350 msgr2=0x7fa268069c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:00.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.770+0000 7fa26dcbf700 1 --2- 172.21.15.89:0/652867929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa268140350 0x7fa268069c80 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fa26806f770 tx=0x7fa258009220 comp rx=0 tx=0).stop 2023-10-10T20:42:00.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.770+0000 7fa26dcbf700 1 -- 172.21.15.89:0/652867929 shutdown_connections 2023-10-10T20:42:00.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.770+0000 7fa26dcbf700 1 --2- 172.21.15.89:0/652867929 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa268140720 0x7fa2680688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.770+0000 7fa26dcbf700 1 --2- 172.21.15.89:0/652867929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa268140350 0x7fa268069c80 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.770+0000 7fa26dcbf700 1 --2- 172.21.15.89:0/652867929 >> v2:172.21.15.111:3300/0 conn(0x7fa26806a1c0 0x7fa268068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.770+0000 7fa26dcbf700 1 -- 172.21.15.89:0/652867929 >> 172.21.15.89:0/652867929 conn(0x7fa2680cd670 msgr2=0x7fa2680cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:00.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.770+0000 7fa26dcbf700 1 -- 172.21.15.89:0/652867929 shutdown_connections 2023-10-10T20:42:00.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.770+0000 7fa26dcbf700 1 -- 172.21.15.89:0/652867929 wait complete. 2023-10-10T20:42:00.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.771+0000 7fa26dcbf700 1 Processor -- start 2023-10-10T20:42:00.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.771+0000 7fa26dcbf700 1 -- start start 2023-10-10T20:42:00.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.772+0000 7fa26dcbf700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa26806a1c0 0x7fa2681390b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.772+0000 7fa26dcbf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa268140350 0x7fa268139ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.772+0000 7fa26dcbf700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa268140720 0x7fa26813a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.772+0000 7fa26dcbf700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2681567a0 con 0x7fa268140350 2023-10-10T20:42:00.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.772+0000 7fa26dcbf700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa268155b90 con 0x7fa268140720 2023-10-10T20:42:00.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.772+0000 7fa26dcbf700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa268155e70 con 0x7fa26806a1c0 2023-10-10T20:42:00.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.772+0000 7fa266ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa268140350 0x7fa268139ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.772+0000 7fa266ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa268140350 0x7fa268139ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44044/0 (socket says 172.21.15.89:44044) 2023-10-10T20:42:00.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.772+0000 7fa266ffd700 1 -- 172.21.15.89:0/91055844 learned_addr learned my addr 172.21.15.89:0/91055844 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:00.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.772+0000 7fa267fff700 1 --2- 172.21.15.89:0/91055844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa268140720 0x7fa26813a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.772+0000 7fa2677fe700 1 --2- 172.21.15.89:0/91055844 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa26806a1c0 0x7fa2681390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.772+0000 7fa266ffd700 1 -- 172.21.15.89:0/91055844 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa26806a1c0 msgr2=0x7fa2681390b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:00.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.772+0000 7fa266ffd700 1 --2- 172.21.15.89:0/91055844 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa26806a1c0 0x7fa2681390b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.772+0000 7fa266ffd700 1 -- 172.21.15.89:0/91055844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa268140720 msgr2=0x7fa26813a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:00.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.772+0000 7fa266ffd700 1 --2- 172.21.15.89:0/91055844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa268140720 0x7fa26813a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:00.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.772+0000 7fa266ffd700 1 -- 172.21.15.89:0/91055844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa258009480 con 0x7fa268140350 2023-10-10T20:42:00.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.772+0000 7fa2677fe700 1 --2- 172.21.15.89:0/91055844 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa26806a1c0 0x7fa2681390b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:00.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.773+0000 7fa266ffd700 1 --2- 172.21.15.89:0/91055844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa268140350 0x7fa268139ec0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00c830 tx=0x7fa25c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:00.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.773+0000 7fa24f7fe700 1 -- 172.21.15.89:0/91055844 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c011600 con 0x7fa268140350 2023-10-10T20:42:00.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.773+0000 7fa24f7fe700 1 -- 172.21.15.89:0/91055844 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25c011760 con 0x7fa268140350 2023-10-10T20:42:00.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.773+0000 7fa26dcbf700 1 -- 172.21.15.89:0/91055844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa268156150 con 0x7fa268140350 2023-10-10T20:42:00.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.773+0000 7fa24f7fe700 1 -- 172.21.15.89:0/91055844 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c011b20 con 0x7fa268140350 2023-10-10T20:42:00.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.773+0000 7fa26dcbf700 1 -- 172.21.15.89:0/91055844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2681395f0 con 0x7fa268140350 2023-10-10T20:42:00.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.774+0000 7fa26dcbf700 1 -- 172.21.15.89:0/91055844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2680635a0 con 0x7fa268140350 2023-10-10T20:42:00.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.774+0000 7fa24f7fe700 1 -- 172.21.15.89:0/91055844 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa25c0193b0 con 0x7fa268140350 2023-10-10T20:42:00.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.774+0000 7fa24f7fe700 1 --2- 172.21.15.89:0/91055844 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa254067320 0x7fa2540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:00.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.775+0000 7fa24f7fe700 1 -- 172.21.15.89:0/91055844 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa25c084dd0 con 0x7fa268140350 2023-10-10T20:42:00.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.775+0000 7fa2677fe700 1 --2- 172.21.15.89:0/91055844 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa254067320 0x7fa2540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:00.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.775+0000 7fa2677fe700 1 --2- 172.21.15.89:0/91055844 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa254067320 0x7fa2540697d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa258007fd0 tx=0x7fa258016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:00.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:00.779+0000 7fa24f7fe700 1 -- 172.21.15.89:0/91055844 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa25c051950 con 0x7fa268140350 2023-10-10T20:42:01.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.007+0000 7fa26dcbf700 1 -- 172.21.15.89:0/91055844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa268064d80 con 0x7fa268140350 2023-10-10T20:42:01.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.007+0000 7fa24f7fe700 1 -- 172.21.15.89:0/91055844 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa25c015070 con 0x7fa268140350 2023-10-10T20:42:01.010 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:42:01.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.009+0000 7fa26dcbf700 1 -- 172.21.15.89:0/91055844 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa254067320 msgr2=0x7fa2540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:01.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.009+0000 7fa26dcbf700 1 --2- 172.21.15.89:0/91055844 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa254067320 0x7fa2540697d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa258007fd0 tx=0x7fa258016000 comp rx=0 tx=0).stop 2023-10-10T20:42:01.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.009+0000 7fa26dcbf700 1 -- 172.21.15.89:0/91055844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa268140350 msgr2=0x7fa268139ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:01.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.009+0000 7fa26dcbf700 1 --2- 172.21.15.89:0/91055844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa268140350 0x7fa268139ec0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00c830 tx=0x7fa25c008040 comp rx=0 tx=0).stop 2023-10-10T20:42:01.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.009+0000 7fa26dcbf700 1 -- 172.21.15.89:0/91055844 shutdown_connections 2023-10-10T20:42:01.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.009+0000 7fa26dcbf700 1 --2- 172.21.15.89:0/91055844 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa254067320 0x7fa2540697d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:01.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.009+0000 7fa26dcbf700 1 --2- 172.21.15.89:0/91055844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa268140720 0x7fa26813a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:01.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.009+0000 7fa26dcbf700 1 --2- 172.21.15.89:0/91055844 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa26806a1c0 0x7fa2681390b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:01.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.009+0000 7fa26dcbf700 1 --2- 172.21.15.89:0/91055844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa268140350 0x7fa268139ec0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:01.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.009+0000 7fa26dcbf700 1 -- 172.21.15.89:0/91055844 >> 172.21.15.89:0/91055844 conn(0x7fa2680cd670 msgr2=0x7fa26813c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:01.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.009+0000 7fa26dcbf700 1 -- 172.21.15.89:0/91055844 shutdown_connections 2023-10-10T20:42:01.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.009+0000 7fa26dcbf700 1 -- 172.21.15.89:0/91055844 wait complete. 2023-10-10T20:42:01.023 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:42:01.024 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:42:01.024 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-10T20:42:01.024 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:42:01.025 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:42:01.025 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:42:01.025 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:42:01.026 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:42:01.026 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:42:01.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:01.027 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-10T20:42:01.043 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:42:01.043 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:42:01.043 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:42:01.043 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:01.044 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-10T20:42:01.044 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:01.044 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:42:01.044 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:01.044 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-10T20:42:01.044 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:42:01.045 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:01.045 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-10T20:42:01.045 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:42:01.045 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:01.046 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:42:01.046 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:01.046 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:42:01.046 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42044sec preferred_lft 42044sec 2023-10-10T20:42:01.047 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2023-10-10T20:42:01.047 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:01.047 INFO:teuthology.orchestra.run.smithi089.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:42:01.047 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:01.047 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:42:01.047 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:01.048 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::98a3:d4ff:fec1:83bb/64 scope link 2023-10-10T20:42:01.048 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:01.048 INFO:teuthology.orchestra.run.smithi089.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:42:01.048 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether ee:16:22:10:ca:9a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:42:01.048 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec16:22ff:fe10:ca9a/64 scope link 2023-10-10T20:42:01.049 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:01.049 INFO:teuthology.orchestra.run.smithi089.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:42:01.049 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:42:01.049 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::6864:2bff:fe5c:71d4/64 scope link 2023-10-10T20:42:01.049 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:01.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:01.050 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:42:01.106 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-10T20:42:01.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:01.107 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-10T20:42:01.164 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-10T20:42:01.164 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:42:01.164 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:42:01.165 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:42:01.165 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:42:01.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:01.166 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:42:01.166 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:42:01.166 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:42:01.167 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:42:01.167 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:42:01.167 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:42:01.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:01.264 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-10T20:42:01.279 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:42:01.279 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:42:01.279 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:42:01.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:01.280 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-10-10T20:42:01.343 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:42:01.344 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:42:01.398 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:42:01.399 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:42:01.399 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:42:01.453 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:42:01.453 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:42:01.454 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:42:01.592 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:42:01.592 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:42:01.593 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:42:01.600 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:42:01.601 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:42:01.601 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:42:01.601 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:42:01.601 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:42:01.624 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:42:01.625 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:42:01.625 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:42:01.625 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:42:01.625 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:42:01.625 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:42:01.626 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:42:01.626 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:42:01.627 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:42:01.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:01.628 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:42:01.652 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:42:01.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.805+0000 7f0dd3006700 1 Processor -- start 2023-10-10T20:42:01.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.805+0000 7f0dd3006700 1 -- start start 2023-10-10T20:42:01.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.806+0000 7f0dd3006700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0dcc159420 0x7f0dcc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:01.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.806+0000 7f0dd3006700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0dcc1597f0 0x7f0dcc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:01.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.806+0000 7f0dd3006700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dcc15d230 0x7f0dcc15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:01.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.806+0000 7f0dd3006700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dcc15dbb0 con 0x7f0dcc159420 2023-10-10T20:42:01.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.806+0000 7f0dd3006700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dcc0b11d0 con 0x7f0dcc1597f0 2023-10-10T20:42:01.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.806+0000 7f0dd3006700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc0b14b0 con 0x7f0dcc15d230 2023-10-10T20:42:01.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.806+0000 7f0dd2004700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0dcc159420 0x7f0dcc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:01.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.806+0000 7f0dd2805700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dcc15d230 0x7f0dcc15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:01.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.806+0000 7f0dd2004700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0dcc159420 0x7f0dcc0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44052/0 (socket says 172.21.15.89:44052) 2023-10-10T20:42:01.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.806+0000 7f0dd2004700 1 -- 172.21.15.89:0/210161251 learned_addr learned my addr 172.21.15.89:0/210161251 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:01.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.806+0000 7f0dd1803700 1 --2- 172.21.15.89:0/210161251 >> v2:172.21.15.111:3300/0 conn(0x7f0dcc1597f0 0x7f0dcc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:01.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.807+0000 7f0dd2004700 1 -- 172.21.15.89:0/210161251 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dcc15d230 msgr2=0x7f0dcc15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:01.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.807+0000 7f0dd2004700 1 --2- 172.21.15.89:0/210161251 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dcc15d230 0x7f0dcc15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:01.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.807+0000 7f0dd2004700 1 -- 172.21.15.89:0/210161251 >> v2:172.21.15.111:3300/0 conn(0x7f0dcc1597f0 msgr2=0x7f0dcc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:01.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.807+0000 7f0dd2004700 1 --2- 172.21.15.89:0/210161251 >> v2:172.21.15.111:3300/0 conn(0x7f0dcc1597f0 0x7f0dcc0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:01.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.807+0000 7f0dd2004700 1 -- 172.21.15.89:0/210161251 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc0b1790 con 0x7f0dcc159420 2023-10-10T20:42:01.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.807+0000 7f0dd2004700 1 --2- 172.21.15.89:0/210161251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dcc159420 0x7f0dcc0b5f40 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc089750 tx=0x7f0dc8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc742655e957cc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:01.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.807+0000 7f0dd1002700 1 -- 172.21.15.89:0/210161251 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc800e070 con 0x7f0dcc159420 2023-10-10T20:42:01.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.807+0000 7f0dd1002700 1 -- 172.21.15.89:0/210161251 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc800b970 con 0x7f0dcc159420 2023-10-10T20:42:01.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.807+0000 7f0dd1002700 1 -- 172.21.15.89:0/210161251 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc80092e0 con 0x7f0dcc159420 2023-10-10T20:42:01.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.807+0000 7f0dd3006700 1 -- 172.21.15.89:0/210161251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dcc159420 msgr2=0x7f0dcc0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:01.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.807+0000 7f0dd3006700 1 --2- 172.21.15.89:0/210161251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dcc159420 0x7f0dcc0b5f40 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc089750 tx=0x7f0dc8009840 comp rx=0 tx=0).stop 2023-10-10T20:42:01.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.808+0000 7f0dd3006700 1 -- 172.21.15.89:0/210161251 shutdown_connections 2023-10-10T20:42:01.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.808+0000 7f0dd3006700 1 --2- 172.21.15.89:0/210161251 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dcc15d230 0x7f0dcc15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:01.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.808+0000 7f0dd3006700 1 --2- 172.21.15.89:0/210161251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dcc159420 0x7f0dcc0b5f40 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:01.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.808+0000 7f0dd3006700 1 --2- 172.21.15.89:0/210161251 >> v2:172.21.15.111:3300/0 conn(0x7f0dcc1597f0 0x7f0dcc0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:01.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.808+0000 7f0dd3006700 1 -- 172.21.15.89:0/210161251 >> 172.21.15.89:0/210161251 conn(0x7f0dcc0d0660 msgr2=0x7f0dcc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:01.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.808+0000 7f0dd3006700 1 -- 172.21.15.89:0/210161251 shutdown_connections 2023-10-10T20:42:01.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.808+0000 7f0dd3006700 1 -- 172.21.15.89:0/210161251 wait complete. 2023-10-10T20:42:01.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.809+0000 7f0dd3006700 1 Processor -- start 2023-10-10T20:42:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.809+0000 7f0dd3006700 1 -- start start 2023-10-10T20:42:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.809+0000 7f0dd3006700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dcc159420 0x7f0dcc16aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.809+0000 7f0dd3006700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dcc1597f0 0x7f0dcc141af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.809+0000 7f0dd3006700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0dcc15d230 0x7f0dcc16c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.809+0000 7f0dd3006700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc16be90 con 0x7f0dcc1597f0 2023-10-10T20:42:01.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.809+0000 7f0dd3006700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc16b280 con 0x7f0dcc15d230 2023-10-10T20:42:01.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.809+0000 7f0dd3006700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc16b560 con 0x7f0dcc159420 2023-10-10T20:42:01.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.810+0000 7f0dd1803700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dcc1597f0 0x7f0dcc141af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:01.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.810+0000 7f0dd1803700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dcc1597f0 0x7f0dcc141af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44054/0 (socket says 172.21.15.89:44054) 2023-10-10T20:42:01.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.810+0000 7f0dd1803700 1 -- 172.21.15.89:0/565981362 learned_addr learned my addr 172.21.15.89:0/565981362 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:01.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.810+0000 7f0dd2004700 1 --2- 172.21.15.89:0/565981362 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dcc159420 0x7f0dcc16aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:01.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.810+0000 7f0dd2805700 1 --2- 172.21.15.89:0/565981362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0dcc15d230 0x7f0dcc16c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:01.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.810+0000 7f0dd1803700 1 -- 172.21.15.89:0/565981362 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dcc159420 msgr2=0x7f0dcc16aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:01.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.810+0000 7f0dd1803700 1 --2- 172.21.15.89:0/565981362 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dcc159420 0x7f0dcc16aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:01.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.810+0000 7f0dd1803700 1 -- 172.21.15.89:0/565981362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0dcc15d230 msgr2=0x7f0dcc16c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:01.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.810+0000 7f0dd1803700 1 --2- 172.21.15.89:0/565981362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0dcc15d230 0x7f0dcc16c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:01.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.810+0000 7f0dd1803700 1 -- 172.21.15.89:0/565981362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc8009ae0 con 0x7f0dcc1597f0 2023-10-10T20:42:01.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.810+0000 7f0dd1803700 1 --2- 172.21.15.89:0/565981362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dcc1597f0 0x7f0dcc141af0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00ec50 tx=0x7f0dbc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:01.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.810+0000 7f0dc27fc700 1 -- 172.21.15.89:0/565981362 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc00c850 con 0x7f0dcc1597f0 2023-10-10T20:42:01.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.810+0000 7f0dc27fc700 1 -- 172.21.15.89:0/565981362 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dbc00c9b0 con 0x7f0dcc1597f0 2023-10-10T20:42:01.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.810+0000 7f0dd3006700 1 -- 172.21.15.89:0/565981362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dcc16b840 con 0x7f0dcc1597f0 2023-10-10T20:42:01.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.810+0000 7f0dc27fc700 1 -- 172.21.15.89:0/565981362 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc00cd70 con 0x7f0dcc1597f0 2023-10-10T20:42:01.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.810+0000 7f0dd3006700 1 -- 172.21.15.89:0/565981362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dcc16af70 con 0x7f0dcc1597f0 2023-10-10T20:42:01.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.811+0000 7f0dd3006700 1 -- 172.21.15.89:0/565981362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dcc16c040 con 0x7f0dcc1597f0 2023-10-10T20:42:01.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.811+0000 7f0dc27fc700 1 -- 172.21.15.89:0/565981362 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0dbc013680 con 0x7f0dcc1597f0 2023-10-10T20:42:01.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.812+0000 7f0dc27fc700 1 --2- 172.21.15.89:0/565981362 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0db0067320 0x7f0db00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:01.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.812+0000 7f0dc27fc700 1 -- 172.21.15.89:0/565981362 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0dbc084e60 con 0x7f0dcc1597f0 2023-10-10T20:42:01.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.812+0000 7f0dd2004700 1 --2- 172.21.15.89:0/565981362 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0db0067320 0x7f0db00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:01.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.812+0000 7f0dd2004700 1 --2- 172.21.15.89:0/565981362 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0db0067320 0x7f0db00697d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009880 tx=0x7f0dc8018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:01.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:01.816+0000 7f0dc27fc700 1 -- 172.21.15.89:0/565981362 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0dcc16c040 con 0x7f0dcc1597f0 2023-10-10T20:42:02.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.044+0000 7f0dd3006700 1 -- 172.21.15.89:0/565981362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0dcc088350 con 0x7f0dcc1597f0 2023-10-10T20:42:02.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.044+0000 7f0dc27fc700 1 -- 172.21.15.89:0/565981362 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0dbc051af0 con 0x7f0dcc1597f0 2023-10-10T20:42:02.047 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:42:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.047+0000 7f0dd3006700 1 -- 172.21.15.89:0/565981362 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0db0067320 msgr2=0x7f0db00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.047+0000 7f0dd3006700 1 --2- 172.21.15.89:0/565981362 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0db0067320 0x7f0db00697d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009880 tx=0x7f0dc8018000 comp rx=0 tx=0).stop 2023-10-10T20:42:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.047+0000 7f0dd3006700 1 -- 172.21.15.89:0/565981362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dcc1597f0 msgr2=0x7f0dcc141af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:02.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.047+0000 7f0dd3006700 1 --2- 172.21.15.89:0/565981362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dcc1597f0 0x7f0dcc141af0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00ec50 tx=0x7f0dbc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:02.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.047+0000 7f0dd3006700 1 -- 172.21.15.89:0/565981362 shutdown_connections 2023-10-10T20:42:02.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.047+0000 7f0dd3006700 1 --2- 172.21.15.89:0/565981362 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0db0067320 0x7f0db00697d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:02.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.047+0000 7f0dd3006700 1 --2- 172.21.15.89:0/565981362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0dcc15d230 0x7f0dcc16c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:02.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.047+0000 7f0dd3006700 1 --2- 172.21.15.89:0/565981362 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0dcc159420 0x7f0dcc16aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:02.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.047+0000 7f0dd3006700 1 --2- 172.21.15.89:0/565981362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dcc1597f0 0x7f0dcc141af0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:02.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.047+0000 7f0dd3006700 1 -- 172.21.15.89:0/565981362 >> 172.21.15.89:0/565981362 conn(0x7f0dcc0d0660 msgr2=0x7f0dcc0cfcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:02.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.047+0000 7f0dd3006700 1 -- 172.21.15.89:0/565981362 shutdown_connections 2023-10-10T20:42:02.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.047+0000 7f0dd3006700 1 -- 172.21.15.89:0/565981362 wait complete. 2023-10-10T20:42:02.061 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:42:02.062 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:42:02.062 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-10-10T20:42:02.063 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:42:02.063 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:42:02.064 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:42:02.064 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:42:02.064 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:42:02.065 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:42:02.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:02.066 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-10-10T20:42:02.083 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:42:02.084 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:42:02.084 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:42:02.084 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:02.084 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-10-10T20:42:02.085 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:02.085 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:42:02.085 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:02.085 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-10-10T20:42:02.085 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:42:02.086 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:02.086 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-10-10T20:42:02.086 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:42:02.086 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:02.086 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:42:02.087 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:02.087 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:42:02.087 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42060sec preferred_lft 42060sec 2023-10-10T20:42:02.087 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-10-10T20:42:02.087 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:02.088 INFO:teuthology.orchestra.run.smithi111.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:42:02.088 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:02.088 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:42:02.088 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:02.088 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::a486:a6ff:fe5f:6021/64 scope link 2023-10-10T20:42:02.089 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:02.089 INFO:teuthology.orchestra.run.smithi111.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:42:02.089 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 1e:d0:81:1b:e2:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:42:02.089 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::1cd0:81ff:fe1b:e26b/64 scope link 2023-10-10T20:42:02.089 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:02.090 INFO:teuthology.orchestra.run.smithi111.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:42:02.090 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:42:02.090 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::88d:cdff:fe75:455e/64 scope link 2023-10-10T20:42:02.090 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:02.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:02.091 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:42:02.149 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-10-10T20:42:02.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:02.150 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-10-10T20:42:02.207 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-10-10T20:42:02.207 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:42:02.207 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:42:02.208 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:42:02.208 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:42:02.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:02.209 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:42:02.209 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:42:02.209 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:42:02.210 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:42:02.210 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:42:02.210 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:42:02.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:02.308 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-10-10T20:42:02.323 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:42:02.324 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:42:02.324 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:42:02.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:02.325 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-10-10T20:42:02.388 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:42:02.389 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:42:02.444 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:42:02.444 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:42:02.445 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:42:02.498 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:42:02.499 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:42:02.499 DEBUG:teuthology.orchestra.run.smithi111:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:42:02.578 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:42:02.579 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:42:02.579 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:42:02.586 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:42:02.587 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:42:02.587 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:42:02.587 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:42:02.587 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:42:02.609 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:42:02.609 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:42:02.609 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:42:02.610 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:42:02.610 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:42:02.610 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:42:02.610 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:42:02.610 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:42:02.611 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:42:02.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:02.612 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:42:02.677 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-10T20:42:02.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.795+0000 7f99d77c5700 1 Processor -- start 2023-10-10T20:42:02.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.795+0000 7f99d77c5700 1 -- start start 2023-10-10T20:42:02.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.796+0000 7f99d77c5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99d0159ec0 0x7f99d00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:02.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.796+0000 7f99d77c5700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f99d00b53d0 0x7f99d00cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:02.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.796+0000 7f99d77c5700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d00b5b00 con 0x7f99d00cce40 2023-10-10T20:42:02.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.796+0000 7f99d77c5700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d00b5da0 con 0x7f99d00b53d0 2023-10-10T20:42:02.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.796+0000 7f99d77c5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d00b6080 con 0x7f99d0159ec0 2023-10-10T20:42:02.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.797+0000 7f99d5561700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99d0159ec0 0x7f99d00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:02.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.797+0000 7f99d4d60700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f99d00b53d0 0x7f99d00cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:02.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.797+0000 7f99d5561700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99d0159ec0 0x7f99d00b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:57146/0 (socket says 172.21.15.89:57146) 2023-10-10T20:42:02.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.797+0000 7f99d5d62700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f99d00cce40 0x7f99d00cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:53286/0 (socket says 172.21.15.89:53286) 2023-10-10T20:42:02.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.797+0000 7f99d5561700 1 -- 172.21.15.89:0/3566150162 learned_addr learned my addr 172.21.15.89:0/3566150162 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:02.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.797+0000 7f99d4d60700 1 -- 172.21.15.89:0/3566150162 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99d0159ec0 msgr2=0x7f99d00b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:02.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.797+0000 7f99d4d60700 1 --2- 172.21.15.89:0/3566150162 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99d0159ec0 0x7f99d00b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:02.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.797+0000 7f99d4d60700 1 -- 172.21.15.89:0/3566150162 >> v1:172.21.15.111:6789/0 conn(0x7f99d00cce40 legacy=0x7f99d00cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:02.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.797+0000 7f99c7fff700 1 -- 172.21.15.89:0/3566150162 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4150968893 0 0) 0x7f99d00b5b00 con 0x7f99d00cce40 2023-10-10T20:42:02.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.797+0000 7f99d4d60700 1 -- 172.21.15.89:0/3566150162 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d00cc100 con 0x7f99d00b53d0 2023-10-10T20:42:02.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.797+0000 7f99d5561700 1 --2- 172.21.15.89:0/3566150162 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99d0159ec0 0x7f99d00b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:42:02.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.797+0000 7f99d4d60700 1 --2- 172.21.15.89:0/3566150162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99d00b53d0 0x7f99d00cc900 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f99c00098e0 tx=0x7f99c00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ca966ae18836968 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:02.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.797+0000 7f99c7fff700 1 -- 172.21.15.89:0/3566150162 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c000b640 con 0x7f99d00b53d0 2023-10-10T20:42:02.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.797+0000 7f99c7fff700 1 -- 172.21.15.89:0/3566150162 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c000b7a0 con 0x7f99d00b53d0 2023-10-10T20:42:02.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.797+0000 7f99c7fff700 1 -- 172.21.15.89:0/3566150162 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c000bb40 con 0x7f99d00b53d0 2023-10-10T20:42:02.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.798+0000 7f99d77c5700 1 -- 172.21.15.89:0/3566150162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99d00b53d0 msgr2=0x7f99d00cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:02.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.798+0000 7f99d77c5700 1 --2- 172.21.15.89:0/3566150162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99d00b53d0 0x7f99d00cc900 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f99c00098e0 tx=0x7f99c00091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:02.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.798+0000 7f99d77c5700 1 -- 172.21.15.89:0/3566150162 shutdown_connections 2023-10-10T20:42:02.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.798+0000 7f99d77c5700 1 --2- 172.21.15.89:0/3566150162 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99d0159ec0 0x7f99d00b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:02.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.798+0000 7f99d77c5700 1 --2- 172.21.15.89:0/3566150162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99d00b53d0 0x7f99d00cc900 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:02.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.798+0000 7f99d77c5700 1 -- 172.21.15.89:0/3566150162 >> 172.21.15.89:0/3566150162 conn(0x7f99d00999f0 msgr2=0x7f99d00b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:02.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.798+0000 7f99d77c5700 1 -- 172.21.15.89:0/3566150162 shutdown_connections 2023-10-10T20:42:02.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.798+0000 7f99d77c5700 1 -- 172.21.15.89:0/3566150162 wait complete. 2023-10-10T20:42:02.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.799+0000 7f99d77c5700 1 Processor -- start 2023-10-10T20:42:02.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.799+0000 7f99d77c5700 1 -- start start 2023-10-10T20:42:02.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.800+0000 7f99d77c5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d00b53d0 0x7f99d0169d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:02.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.800+0000 7f99d77c5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99d00cce40 0x7f99d016a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:02.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.800+0000 7f99d77c5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99d0159ec0 0x7f99d016a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:02.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.800+0000 7f99d77c5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d0069d20 con 0x7f99d00b53d0 2023-10-10T20:42:02.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.800+0000 7f99d77c5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d0159220 con 0x7f99d0159ec0 2023-10-10T20:42:02.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.800+0000 7f99d77c5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d016acd0 con 0x7f99d00cce40 2023-10-10T20:42:02.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.800+0000 7f99d5561700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d00b53d0 0x7f99d0169d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:02.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.800+0000 7f99d5561700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d00b53d0 0x7f99d0169d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44066/0 (socket says 172.21.15.89:44066) 2023-10-10T20:42:02.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.800+0000 7f99d5561700 1 -- 172.21.15.89:0/1856806034 learned_addr learned my addr 172.21.15.89:0/1856806034 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:02.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.800+0000 7f99d4d60700 1 --2- 172.21.15.89:0/1856806034 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99d00cce40 0x7f99d016a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:02.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.800+0000 7f99d5d62700 1 --2- 172.21.15.89:0/1856806034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99d0159ec0 0x7f99d016a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:02.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.800+0000 7f99d5561700 1 -- 172.21.15.89:0/1856806034 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99d00cce40 msgr2=0x7f99d016a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:02.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.800+0000 7f99d5561700 1 --2- 172.21.15.89:0/1856806034 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99d00cce40 0x7f99d016a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:02.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.800+0000 7f99d5561700 1 -- 172.21.15.89:0/1856806034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99d0159ec0 msgr2=0x7f99d016a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:02.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.800+0000 7f99d5561700 1 --2- 172.21.15.89:0/1856806034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99d0159ec0 0x7f99d016a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:02.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.800+0000 7f99d5561700 1 -- 172.21.15.89:0/1856806034 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99cc009480 con 0x7f99d00b53d0 2023-10-10T20:42:02.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.800+0000 7f99d4d60700 1 --2- 172.21.15.89:0/1856806034 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99d00cce40 0x7f99d016a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:02.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.801+0000 7f99d5561700 1 --2- 172.21.15.89:0/1856806034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d00b53d0 0x7f99d0169d10 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00ec80 tx=0x7f99cc009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:02.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.801+0000 7f99c57fa700 1 -- 172.21.15.89:0/1856806034 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc005a00 con 0x7f99d00b53d0 2023-10-10T20:42:02.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.801+0000 7f99c57fa700 1 -- 172.21.15.89:0/1856806034 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99cc005b60 con 0x7f99d00b53d0 2023-10-10T20:42:02.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.801+0000 7f99d77c5700 1 -- 172.21.15.89:0/1856806034 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c0009420 con 0x7f99d00b53d0 2023-10-10T20:42:02.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.801+0000 7f99c57fa700 1 -- 172.21.15.89:0/1856806034 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc00c970 con 0x7f99d00b53d0 2023-10-10T20:42:02.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.801+0000 7f99d77c5700 1 -- 172.21.15.89:0/1856806034 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d006a560 con 0x7f99d00b53d0 2023-10-10T20:42:02.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.802+0000 7f99d77c5700 1 -- 172.21.15.89:0/1856806034 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99d0063c90 con 0x7f99d00b53d0 2023-10-10T20:42:02.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.802+0000 7f99c57fa700 1 -- 172.21.15.89:0/1856806034 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f99cc006030 con 0x7f99d00b53d0 2023-10-10T20:42:02.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.802+0000 7f99c57fa700 1 --2- 172.21.15.89:0/1856806034 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f99b4067320 0x7f99b40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:02.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.803+0000 7f99c57fa700 1 -- 172.21.15.89:0/1856806034 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f99cc0851c0 con 0x7f99d00b53d0 2023-10-10T20:42:02.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.803+0000 7f99d4d60700 1 --2- 172.21.15.89:0/1856806034 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f99b4067320 0x7f99b40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:02.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.803+0000 7f99d4d60700 1 --2- 172.21.15.89:0/1856806034 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f99b4067320 0x7f99b40697d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f99c00093f0 tx=0x7f99c0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:02.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:02.807+0000 7f99c57fa700 1 -- 172.21.15.89:0/1856806034 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99cc051e50 con 0x7f99d00b53d0 2023-10-10T20:42:03.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:03.040+0000 7f99d77c5700 1 -- 172.21.15.89:0/1856806034 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f99d00718b0 con 0x7f99d00b53d0 2023-10-10T20:42:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:03.982+0000 7f99c57fa700 1 -- 172.21.15.89:0/1856806034 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v92) v1 ==== 136+0+0 (secure 0 0 0) 0x7f99cc055470 con 0x7f99d00b53d0 2023-10-10T20:42:03.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:03.984+0000 7f99d77c5700 1 -- 172.21.15.89:0/1856806034 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f99b4067320 msgr2=0x7f99b40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:03.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:03.984+0000 7f99d77c5700 1 --2- 172.21.15.89:0/1856806034 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f99b4067320 0x7f99b40697d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f99c00093f0 tx=0x7f99c0017000 comp rx=0 tx=0).stop 2023-10-10T20:42:03.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:03.984+0000 7f99d77c5700 1 -- 172.21.15.89:0/1856806034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d00b53d0 msgr2=0x7f99d0169d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:03.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:03.984+0000 7f99d77c5700 1 --2- 172.21.15.89:0/1856806034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d00b53d0 0x7f99d0169d10 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00ec80 tx=0x7f99cc009220 comp rx=0 tx=0).stop 2023-10-10T20:42:03.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:03.985+0000 7f99d77c5700 1 -- 172.21.15.89:0/1856806034 shutdown_connections 2023-10-10T20:42:03.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:03.985+0000 7f99d77c5700 1 --2- 172.21.15.89:0/1856806034 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f99b4067320 0x7f99b40697d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:03.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:03.985+0000 7f99d77c5700 1 --2- 172.21.15.89:0/1856806034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99d0159ec0 0x7f99d016a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:03.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:03.985+0000 7f99d77c5700 1 --2- 172.21.15.89:0/1856806034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d00b53d0 0x7f99d0169d10 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:03.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:03.985+0000 7f99d77c5700 1 --2- 172.21.15.89:0/1856806034 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f99d00cce40 0x7f99d016a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:03.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:03.985+0000 7f99d77c5700 1 -- 172.21.15.89:0/1856806034 >> 172.21.15.89:0/1856806034 conn(0x7f99d00999f0 msgr2=0x7f99d01556e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:03.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:03.985+0000 7f99d77c5700 1 -- 172.21.15.89:0/1856806034 shutdown_connections 2023-10-10T20:42:03.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:03.985+0000 7f99d77c5700 1 -- 172.21.15.89:0/1856806034 wait complete. 2023-10-10T20:42:04.001 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-10T20:42:04.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.120+0000 7f560759e700 1 Processor -- start 2023-10-10T20:42:04.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.120+0000 7f560759e700 1 -- start start 2023-10-10T20:42:04.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.121+0000 7f560759e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5608159420 0x7f56080b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:04.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.121+0000 7f560759e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f560815d230 0x7f560815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:04.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.121+0000 7f560759e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56080b2980 con 0x7f56081597f0 2023-10-10T20:42:04.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.121+0000 7f560759e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560815dbb0 con 0x7f5608159420 2023-10-10T20:42:04.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.121+0000 7f560759e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56080b11d0 con 0x7f560815d230 2023-10-10T20:42:04.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.121+0000 7f560659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5608159420 0x7f56080b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:04.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.121+0000 7f5605d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f56081597f0 0x7f56080b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35648/0 (socket says 172.21.15.89:35648) 2023-10-10T20:42:04.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.121+0000 7f560659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5608159420 0x7f56080b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44076/0 (socket says 172.21.15.89:44076) 2023-10-10T20:42:04.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.121+0000 7f5605d9b700 1 -- 172.21.15.89:0/4008154531 learned_addr learned my addr 172.21.15.89:0/4008154531 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:04.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.121+0000 7f560559a700 1 -- 172.21.15.89:0/4008154531 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646286315 0 0) 0x7f56080b2980 con 0x7f56081597f0 2023-10-10T20:42:04.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.121+0000 7f560559a700 1 -- 172.21.15.89:0/4008154531 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55f0003570 con 0x7f56081597f0 2023-10-10T20:42:04.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.121+0000 7f5606d9d700 1 --2- 172.21.15.89:0/4008154531 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f560815d230 0x7f560815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:04.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.121+0000 7f560659c700 1 -- 172.21.15.89:0/4008154531 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f560815d230 msgr2=0x7f560815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:04.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.121+0000 7f560659c700 1 --2- 172.21.15.89:0/4008154531 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f560815d230 0x7f560815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:04.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.121+0000 7f560659c700 1 -- 172.21.15.89:0/4008154531 >> v1:172.21.15.89:6789/0 conn(0x7f56081597f0 legacy=0x7f56080b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:04.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.121+0000 7f560659c700 1 -- 172.21.15.89:0/4008154531 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56080b14b0 con 0x7f5608159420 2023-10-10T20:42:04.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.121+0000 7f5606d9d700 1 --2- 172.21.15.89:0/4008154531 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f560815d230 0x7f560815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:04.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.122+0000 7f560659c700 1 --2- 172.21.15.89:0/4008154531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5608159420 0x7f56080b5f40 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f56000098e0 tx=0x7f56000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62fb7715caa21a7d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:04.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.122+0000 7f560559a700 1 -- 172.21.15.89:0/4008154531 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560000b640 con 0x7f5608159420 2023-10-10T20:42:04.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.122+0000 7f560559a700 1 -- 172.21.15.89:0/4008154531 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560000b7a0 con 0x7f5608159420 2023-10-10T20:42:04.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.122+0000 7f560559a700 1 -- 172.21.15.89:0/4008154531 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560000bb40 con 0x7f5608159420 2023-10-10T20:42:04.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.122+0000 7f560759e700 1 -- 172.21.15.89:0/4008154531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5608159420 msgr2=0x7f56080b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:04.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.122+0000 7f560759e700 1 --2- 172.21.15.89:0/4008154531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5608159420 0x7f56080b5f40 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f56000098e0 tx=0x7f56000091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:04.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.122+0000 7f560759e700 1 -- 172.21.15.89:0/4008154531 shutdown_connections 2023-10-10T20:42:04.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.122+0000 7f560759e700 1 --2- 172.21.15.89:0/4008154531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5608159420 0x7f56080b5f40 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:04.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.122+0000 7f560759e700 1 --2- 172.21.15.89:0/4008154531 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f560815d230 0x7f560815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:04.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.122+0000 7f560759e700 1 -- 172.21.15.89:0/4008154531 >> 172.21.15.89:0/4008154531 conn(0x7f56080d0660 msgr2=0x7f56080d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:04.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.122+0000 7f560759e700 1 -- 172.21.15.89:0/4008154531 shutdown_connections 2023-10-10T20:42:04.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.122+0000 7f560759e700 1 -- 172.21.15.89:0/4008154531 wait complete. 2023-10-10T20:42:04.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.123+0000 7f560759e700 1 Processor -- start 2023-10-10T20:42:04.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.123+0000 7f560759e700 1 -- start start 2023-10-10T20:42:04.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.124+0000 7f560759e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5608159420 0x7f5608159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:04.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.124+0000 7f560759e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f56081597f0 0x7f5608148530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:04.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.124+0000 7f560759e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f560815d230 0x7f56080b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:04.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.124+0000 7f560759e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560815b070 con 0x7f560815d230 2023-10-10T20:42:04.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.124+0000 7f560759e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560815a1e0 con 0x7f5608159420 2023-10-10T20:42:04.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.124+0000 7f560759e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560815a460 con 0x7f56081597f0 2023-10-10T20:42:04.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.124+0000 7f5606d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f560815d230 0x7f56080b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:04.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.124+0000 7f5606d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f560815d230 0x7f56080b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44086/0 (socket says 172.21.15.89:44086) 2023-10-10T20:42:04.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.124+0000 7f5606d9d700 1 -- 172.21.15.89:0/3156065383 learned_addr learned my addr 172.21.15.89:0/3156065383 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:04.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.124+0000 7f5605d9b700 1 --2- 172.21.15.89:0/3156065383 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f56081597f0 0x7f5608148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:04.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.124+0000 7f560659c700 1 --2- 172.21.15.89:0/3156065383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5608159420 0x7f5608159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:04.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.124+0000 7f5606d9d700 1 -- 172.21.15.89:0/3156065383 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f56081597f0 msgr2=0x7f5608148530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:04.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.124+0000 7f5606d9d700 1 --2- 172.21.15.89:0/3156065383 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f56081597f0 0x7f5608148530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:04.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.124+0000 7f5606d9d700 1 -- 172.21.15.89:0/3156065383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5608159420 msgr2=0x7f5608159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:04.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.124+0000 7f5606d9d700 1 --2- 172.21.15.89:0/3156065383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5608159420 0x7f5608159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:04.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.124+0000 7f5606d9d700 1 -- 172.21.15.89:0/3156065383 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5600009420 con 0x7f560815d230 2023-10-10T20:42:04.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.125+0000 7f5605d9b700 1 --2- 172.21.15.89:0/3156065383 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f56081597f0 0x7f5608148530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:04.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.125+0000 7f560659c700 1 --2- 172.21.15.89:0/3156065383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5608159420 0x7f5608159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:04.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.125+0000 7f5606d9d700 1 --2- 172.21.15.89:0/3156065383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f560815d230 0x7f56080b5ab0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f55f800cef0 tx=0x7f55f8007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:04.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.125+0000 7f55f67fc700 1 -- 172.21.15.89:0/3156065383 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f800ae00 con 0x7f560815d230 2023-10-10T20:42:04.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.125+0000 7f55f67fc700 1 -- 172.21.15.89:0/3156065383 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f8016420 con 0x7f560815d230 2023-10-10T20:42:04.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.125+0000 7f55f67fc700 1 -- 172.21.15.89:0/3156065383 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f80167e0 con 0x7f560815d230 2023-10-10T20:42:04.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.125+0000 7f560759e700 1 -- 172.21.15.89:0/3156065383 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560815a740 con 0x7f560815d230 2023-10-10T20:42:04.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.125+0000 7f560759e700 1 -- 172.21.15.89:0/3156065383 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560815ad00 con 0x7f560815d230 2023-10-10T20:42:04.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.126+0000 7f560759e700 1 -- 172.21.15.89:0/3156065383 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560815b220 con 0x7f560815d230 2023-10-10T20:42:04.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.126+0000 7f55f67fc700 1 -- 172.21.15.89:0/3156065383 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f55f8016bd0 con 0x7f560815d230 2023-10-10T20:42:04.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.127+0000 7f55f67fc700 1 --2- 172.21.15.89:0/3156065383 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f55e8067350 0x7f55e8069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:04.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.127+0000 7f560659c700 1 --2- 172.21.15.89:0/3156065383 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f55e8067350 0x7f55e8069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:04.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.127+0000 7f55f67fc700 1 -- 172.21.15.89:0/3156065383 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f55f8012070 con 0x7f560815d230 2023-10-10T20:42:04.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.127+0000 7f560659c700 1 --2- 172.21.15.89:0/3156065383 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f55e8067350 0x7f55e8069800 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5600003080 tx=0x7f5600017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:04.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.131+0000 7f55f67fc700 1 -- 172.21.15.89:0/3156065383 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55f804e5a0 con 0x7f560815d230 2023-10-10T20:42:04.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.342+0000 7f560759e700 1 -- 172.21.15.89:0/3156065383 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f560804c3c0 con 0x7f55e8067350 2023-10-10T20:42:04.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.343+0000 7f55f67fc700 1 -- 172.21.15.89:0/3156065383 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f560804c3c0 con 0x7f55e8067350 2023-10-10T20:42:04.345 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:42:04.345 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:04.345 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs" 2023-10-10T20:42:04.346 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:04.346 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:42:04.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.345+0000 7f560759e700 1 -- 172.21.15.89:0/3156065383 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f55e8067350 msgr2=0x7f55e8069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:04.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.345+0000 7f560759e700 1 --2- 172.21.15.89:0/3156065383 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f55e8067350 0x7f55e8069800 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5600003080 tx=0x7f5600017000 comp rx=0 tx=0).stop 2023-10-10T20:42:04.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.345+0000 7f560759e700 1 -- 172.21.15.89:0/3156065383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f560815d230 msgr2=0x7f56080b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:04.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.345+0000 7f560759e700 1 --2- 172.21.15.89:0/3156065383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f560815d230 0x7f56080b5ab0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f55f800cef0 tx=0x7f55f8007860 comp rx=0 tx=0).stop 2023-10-10T20:42:04.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.345+0000 7f560759e700 1 -- 172.21.15.89:0/3156065383 shutdown_connections 2023-10-10T20:42:04.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.345+0000 7f560759e700 1 --2- 172.21.15.89:0/3156065383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f560815d230 0x7f56080b5ab0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:04.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.345+0000 7f560759e700 1 --2- 172.21.15.89:0/3156065383 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f55e8067350 0x7f55e8069800 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:04.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.345+0000 7f560759e700 1 --2- 172.21.15.89:0/3156065383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5608159420 0x7f5608159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:04.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.345+0000 7f560759e700 1 --2- 172.21.15.89:0/3156065383 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f56081597f0 0x7f5608148530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:04.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.345+0000 7f560759e700 1 -- 172.21.15.89:0/3156065383 >> 172.21.15.89:0/3156065383 conn(0x7f56080d0660 msgr2=0x7f56080b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:04.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.345+0000 7f560759e700 1 -- 172.21.15.89:0/3156065383 shutdown_connections 2023-10-10T20:42:04.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.345+0000 7f560759e700 1 -- 172.21.15.89:0/3156065383 wait complete. 2023-10-10T20:42:04.359 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:42:04.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.474+0000 7fedcb59e700 1 Processor -- start 2023-10-10T20:42:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.474+0000 7fedcb59e700 1 -- start start 2023-10-10T20:42:04.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.475+0000 7fedcb59e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fedcc15a3b0 0x7fedcc15a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:04.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.475+0000 7fedcb59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fedcc0aef00 0x7fedcc15d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:04.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.475+0000 7fedcb59e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc0aea70 con 0x7fedcc06e470 2023-10-10T20:42:04.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.475+0000 7fedcb59e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedcc0ae480 con 0x7fedcc15a3b0 2023-10-10T20:42:04.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.475+0000 7fedcb59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc15acc0 con 0x7fedcc0aef00 2023-10-10T20:42:04.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.476+0000 7fedc1d9b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fedcc0aef00 0x7fedcc15d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.476+0000 7fedc1d9b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fedcc0aef00 0x7fedcc15d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:57182/0 (socket says 172.21.15.89:57182) 2023-10-10T20:42:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.476+0000 7fedc1d9b700 1 -- 172.21.15.89:0/1118586457 learned_addr learned my addr 172.21.15.89:0/1118586457 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.476+0000 7fedca59c700 1 --2- 172.21.15.89:0/1118586457 >> v2:172.21.15.111:3300/0 conn(0x7fedcc15a3b0 0x7fedcc15a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.476+0000 7fedc1d9b700 1 -- 172.21.15.89:0/1118586457 >> v2:172.21.15.111:3300/0 conn(0x7fedcc15a3b0 msgr2=0x7fedcc15a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.476+0000 7fedc1d9b700 1 --2- 172.21.15.89:0/1118586457 >> v2:172.21.15.111:3300/0 conn(0x7fedcc15a3b0 0x7fedcc15a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.476+0000 7fedc1d9b700 1 -- 172.21.15.89:0/1118586457 >> v1:172.21.15.111:6789/0 conn(0x7fedcc06e470 legacy=0x7fedcc0c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.476+0000 7fedc1d9b700 1 -- 172.21.15.89:0/1118586457 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc15dd30 con 0x7fedcc0aef00 2023-10-10T20:42:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.476+0000 7fedc1d9b700 1 --2- 172.21.15.89:0/1118586457 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fedcc0aef00 0x7fedcc15d3c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fedb40099b0 tx=0x7fedb4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8102aa42a7e78e99 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.476+0000 7fedc9d9b700 1 -- 172.21.15.89:0/1118586457 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4009ea0 con 0x7fedcc0aef00 2023-10-10T20:42:04.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.476+0000 7fedc9d9b700 1 -- 172.21.15.89:0/1118586457 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb400b860 con 0x7fedcc0aef00 2023-10-10T20:42:04.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.476+0000 7fedc9d9b700 1 -- 172.21.15.89:0/1118586457 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb400bc00 con 0x7fedcc0aef00 2023-10-10T20:42:04.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.477+0000 7fedcb59e700 1 -- 172.21.15.89:0/1118586457 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fedcc0aef00 msgr2=0x7fedcc15d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:04.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.477+0000 7fedcb59e700 1 --2- 172.21.15.89:0/1118586457 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fedcc0aef00 0x7fedcc15d3c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fedb40099b0 tx=0x7fedb4009250 comp rx=0 tx=0).stop 2023-10-10T20:42:04.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.477+0000 7fedcb59e700 1 -- 172.21.15.89:0/1118586457 shutdown_connections 2023-10-10T20:42:04.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.477+0000 7fedcb59e700 1 --2- 172.21.15.89:0/1118586457 >> v2:172.21.15.111:3300/0 conn(0x7fedcc15a3b0 0x7fedcc15a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:04.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.477+0000 7fedcb59e700 1 --2- 172.21.15.89:0/1118586457 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fedcc0aef00 0x7fedcc15d3c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:04.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.477+0000 7fedcb59e700 1 -- 172.21.15.89:0/1118586457 >> 172.21.15.89:0/1118586457 conn(0x7fedcc0c9f70 msgr2=0x7fedcc06dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:04.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.477+0000 7fedcb59e700 1 -- 172.21.15.89:0/1118586457 shutdown_connections 2023-10-10T20:42:04.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.477+0000 7fedcb59e700 1 -- 172.21.15.89:0/1118586457 wait complete. 2023-10-10T20:42:04.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.478+0000 7fedcb59e700 1 Processor -- start 2023-10-10T20:42:04.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.478+0000 7fedcb59e700 1 -- start start 2023-10-10T20:42:04.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.479+0000 7fedcb59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fedcc06e470 0x7fedcc14f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:04.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.479+0000 7fedcb59e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedcc0aef00 0x7fedcc0c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:04.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.479+0000 7fedcb59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fedcc15a3b0 0x7fedcc0c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:04.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.479+0000 7fedcb59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0c51b0 con 0x7fedcc15a3b0 2023-10-10T20:42:04.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.479+0000 7fedcb59e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc14fbf0 con 0x7fedcc0aef00 2023-10-10T20:42:04.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.479+0000 7fedcb59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0c45a0 con 0x7fedcc06e470 2023-10-10T20:42:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.479+0000 7fedcad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fedcc15a3b0 0x7fedcc0c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.479+0000 7fedcad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fedcc15a3b0 0x7fedcc0c5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44094/0 (socket says 172.21.15.89:44094) 2023-10-10T20:42:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.479+0000 7fedcad9d700 1 -- 172.21.15.89:0/2523043843 learned_addr learned my addr 172.21.15.89:0/2523043843 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.479+0000 7fedc1d9b700 1 --2- 172.21.15.89:0/2523043843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedcc0aef00 0x7fedcc0c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.479+0000 7fedca59c700 1 --2- 172.21.15.89:0/2523043843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fedcc06e470 0x7fedcc14f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.479+0000 7fedcad9d700 1 -- 172.21.15.89:0/2523043843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fedcc06e470 msgr2=0x7fedcc14f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.479+0000 7fedcad9d700 1 --2- 172.21.15.89:0/2523043843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fedcc06e470 0x7fedcc14f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.479+0000 7fedcad9d700 1 -- 172.21.15.89:0/2523043843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedcc0aef00 msgr2=0x7fedcc0c3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.479+0000 7fedcad9d700 1 --2- 172.21.15.89:0/2523043843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedcc0aef00 0x7fedcc0c3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.479+0000 7fedcad9d700 1 -- 172.21.15.89:0/2523043843 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb40094f0 con 0x7fedcc15a3b0 2023-10-10T20:42:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.480+0000 7fedcad9d700 1 --2- 172.21.15.89:0/2523043843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fedcc15a3b0 0x7fedcc0c5df0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fedc400c8e0 tx=0x7fedc4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.480+0000 7fedc2ffd700 1 -- 172.21.15.89:0/2523043843 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc400ce80 con 0x7fedcc15a3b0 2023-10-10T20:42:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.480+0000 7fedc2ffd700 1 -- 172.21.15.89:0/2523043843 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc400a7e0 con 0x7fedcc15a3b0 2023-10-10T20:42:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.480+0000 7fedcb59e700 1 -- 172.21.15.89:0/2523043843 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedcc0c4880 con 0x7fedcc15a3b0 2023-10-10T20:42:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.480+0000 7fedc2ffd700 1 -- 172.21.15.89:0/2523043843 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc400ab80 con 0x7fedcc15a3b0 2023-10-10T20:42:04.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.480+0000 7fedcb59e700 1 -- 172.21.15.89:0/2523043843 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc0c4e40 con 0x7fedcc15a3b0 2023-10-10T20:42:04.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.481+0000 7fedcb59e700 1 -- 172.21.15.89:0/2523043843 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedcc0c5320 con 0x7fedcc15a3b0 2023-10-10T20:42:04.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.481+0000 7fedc2ffd700 1 -- 172.21.15.89:0/2523043843 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fedc4012420 con 0x7fedcc15a3b0 2023-10-10T20:42:04.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.481+0000 7fedc2ffd700 1 --2- 172.21.15.89:0/2523043843 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fedac067320 0x7fedac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:04.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.482+0000 7fedc2ffd700 1 -- 172.21.15.89:0/2523043843 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fedc4084f60 con 0x7fedcc15a3b0 2023-10-10T20:42:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.482+0000 7fedca59c700 1 --2- 172.21.15.89:0/2523043843 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fedac067320 0x7fedac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.482+0000 7fedca59c700 1 --2- 172.21.15.89:0/2523043843 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fedac067320 0x7fedac0697d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009ca0 tx=0x7fedbc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.486+0000 7fedc2ffd700 1 -- 172.21.15.89:0/2523043843 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fedc4051bf0 con 0x7fedcc15a3b0 2023-10-10T20:42:04.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.668+0000 7fedcb59e700 1 -- 172.21.15.89:0/2523043843 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fedcc0c5540 con 0x7fedcc15a3b0 2023-10-10T20:42:04.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.669+0000 7fedc2ffd700 1 -- 172.21.15.89:0/2523043843 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fedc4055210 con 0x7fedcc15a3b0 2023-10-10T20:42:04.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.670+0000 7fedcb59e700 1 -- 172.21.15.89:0/2523043843 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fedac067320 msgr2=0x7fedac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:04.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.670+0000 7fedcb59e700 1 --2- 172.21.15.89:0/2523043843 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fedac067320 0x7fedac0697d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009ca0 tx=0x7fedbc008040 comp rx=0 tx=0).stop 2023-10-10T20:42:04.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.670+0000 7fedcb59e700 1 -- 172.21.15.89:0/2523043843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fedcc15a3b0 msgr2=0x7fedcc0c5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:04.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.670+0000 7fedcb59e700 1 --2- 172.21.15.89:0/2523043843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fedcc15a3b0 0x7fedcc0c5df0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fedc400c8e0 tx=0x7fedc4001710 comp rx=0 tx=0).stop 2023-10-10T20:42:04.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.670+0000 7fedcb59e700 1 -- 172.21.15.89:0/2523043843 shutdown_connections 2023-10-10T20:42:04.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.670+0000 7fedcb59e700 1 --2- 172.21.15.89:0/2523043843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fedcc15a3b0 0x7fedcc0c5df0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:04.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.670+0000 7fedcb59e700 1 --2- 172.21.15.89:0/2523043843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fedcc06e470 0x7fedcc14f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:04.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.670+0000 7fedcb59e700 1 --2- 172.21.15.89:0/2523043843 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fedac067320 0x7fedac0697d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:04.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.670+0000 7fedcb59e700 1 --2- 172.21.15.89:0/2523043843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedcc0aef00 0x7fedcc0c3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:04.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.670+0000 7fedcb59e700 1 -- 172.21.15.89:0/2523043843 >> 172.21.15.89:0/2523043843 conn(0x7fedcc0c9f70 msgr2=0x7fedcc15b550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:04.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.671+0000 7fedcb59e700 1 -- 172.21.15.89:0/2523043843 shutdown_connections 2023-10-10T20:42:04.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:04.671+0000 7fedcb59e700 1 -- 172.21.15.89:0/2523043843 wait complete. 2023-10-10T20:42:04.684 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:42:04.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:42:04.685 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:42:04.720 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:42:04.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:04.721 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:42:04.775 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:42:04.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:42:04.777 DEBUG:teuthology.orchestra.run.smithi111:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:42:04.825 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:42:04.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:04.826 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:42:04.843 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:42:04.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:04.844 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-10T20:42:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-10T20:42:04.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:42:04.859 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:42:04.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:04.860 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-10T20:42:04.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:42:04.901 INFO:teuthology.orchestra.run.smithi111.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-10T20:42:04.902 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000887918 --yes-i-really-mean-it 2023-10-10T20:42:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.020+0000 7f85a7aa4700 1 Processor -- start 2023-10-10T20:42:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.020+0000 7f85a7aa4700 1 -- start start 2023-10-10T20:42:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.021+0000 7f85a7aa4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85a0159ec0 0x7f85a00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.021+0000 7f85a7aa4700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f85a00cf350 0x7f85a00cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.021+0000 7f85a7aa4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a00b5b00 con 0x7f85a00b53d0 2023-10-10T20:42:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.021+0000 7f85a7aa4700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a00b5da0 con 0x7f85a00cf350 2023-10-10T20:42:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.021+0000 7f85a7aa4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00b6080 con 0x7f85a0159ec0 2023-10-10T20:42:05.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.021+0000 7f85a503f700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f85a00b53d0 0x7f85a00cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35656/0 (socket says 172.21.15.89:35656) 2023-10-10T20:42:05.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.021+0000 7f85a503f700 1 -- 172.21.15.89:0/2852349486 learned_addr learned my addr 172.21.15.89:0/2852349486 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:05.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.021+0000 7f85a6041700 1 --2- 172.21.15.89:0/2852349486 >> v2:172.21.15.111:3300/0 conn(0x7f85a00cf350 0x7f85a00cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:05.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.021+0000 7f85a5840700 1 --2- 172.21.15.89:0/2852349486 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85a0159ec0 0x7f85a00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:05.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.021+0000 7f85a483e700 1 -- 172.21.15.89:0/2852349486 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2728845159 0 0) 0x7f85a00b5b00 con 0x7f85a00b53d0 2023-10-10T20:42:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.021+0000 7f85a483e700 1 -- 172.21.15.89:0/2852349486 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8588003570 con 0x7f85a00b53d0 2023-10-10T20:42:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a483e700 1 -- 172.21.15.89:0/2852349486 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1527526447 0 0) 0x7f8588003570 con 0x7f85a00b53d0 2023-10-10T20:42:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a483e700 1 -- 172.21.15.89:0/2852349486 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85a0159ec0 msgr2=0x7f85a00b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a483e700 1 --2- 172.21.15.89:0/2852349486 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85a0159ec0 0x7f85a00b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:05.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a483e700 1 -- 172.21.15.89:0/2852349486 >> v2:172.21.15.111:3300/0 conn(0x7f85a00cf350 msgr2=0x7f85a00cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:05.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a483e700 1 --2- 172.21.15.89:0/2852349486 >> v2:172.21.15.111:3300/0 conn(0x7f85a00cf350 0x7f85a00cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:05.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a483e700 1 -- 172.21.15.89:0/2852349486 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a00cc100 con 0x7f85a00b53d0 2023-10-10T20:42:05.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a483e700 1 -- 172.21.15.89:0/2852349486 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f8594001470 con 0x7f85a00b53d0 2023-10-10T20:42:05.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a483e700 1 -- 172.21.15.89:0/2852349486 >> v1:172.21.15.89:6789/0 conn(0x7f85a00b53d0 legacy=0x7f85a00cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:05.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a483e700 1 --2- 172.21.15.89:0/2852349486 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85a0159ec0 0x7f8588003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:05.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a483e700 1 --2- 172.21.15.89:0/2852349486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8588003fe0 0x7f8588004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:05.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a483e700 1 --2- 172.21.15.89:0/2852349486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8588004d50 0x7f85880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:05.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a483e700 1 -- 172.21.15.89:0/2852349486 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00b5da0 con 0x7f8588003fe0 2023-10-10T20:42:05.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a483e700 1 -- 172.21.15.89:0/2852349486 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00b6080 con 0x7f8588004d50 2023-10-10T20:42:05.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a483e700 1 -- 172.21.15.89:0/2852349486 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858800ba90 con 0x7f85a0159ec0 2023-10-10T20:42:05.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a503f700 1 --2- 172.21.15.89:0/2852349486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8588003fe0 0x7f8588004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:05.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a6041700 1 --2- 172.21.15.89:0/2852349486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8588004d50 0x7f85880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:05.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a5840700 1 --2- 172.21.15.89:0/2852349486 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85a0159ec0 0x7f8588003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:05.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a503f700 1 -- 172.21.15.89:0/2852349486 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85a0159ec0 msgr2=0x7f8588003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:05.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a503f700 1 --2- 172.21.15.89:0/2852349486 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85a0159ec0 0x7f8588003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:05.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a503f700 1 -- 172.21.15.89:0/2852349486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8588004d50 msgr2=0x7f85880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:05.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a503f700 1 --2- 172.21.15.89:0/2852349486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8588004d50 0x7f85880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:05.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a503f700 1 -- 172.21.15.89:0/2852349486 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8588003570 con 0x7f8588003fe0 2023-10-10T20:42:05.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a5840700 1 --2- 172.21.15.89:0/2852349486 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85a0159ec0 0x7f8588003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:05.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a503f700 1 --2- 172.21.15.89:0/2852349486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8588003fe0 0x7f8588004810 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f859400c3c0 tx=0x7f8594001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0828d45f40abc8b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:05.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a483e700 1 -- 172.21.15.89:0/2852349486 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859400cc30 con 0x7f8588003fe0 2023-10-10T20:42:05.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a483e700 1 -- 172.21.15.89:0/2852349486 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859400cdb0 con 0x7f8588003fe0 2023-10-10T20:42:05.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.022+0000 7f85a483e700 1 -- 172.21.15.89:0/2852349486 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8594005b70 con 0x7f8588003fe0 2023-10-10T20:42:05.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.023+0000 7f85a7aa4700 1 -- 172.21.15.89:0/2852349486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8588003fe0 msgr2=0x7f8588004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:05.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.023+0000 7f85a7aa4700 1 --2- 172.21.15.89:0/2852349486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8588003fe0 0x7f8588004810 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f859400c3c0 tx=0x7f8594001e90 comp rx=0 tx=0).stop 2023-10-10T20:42:05.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.023+0000 7f85a6041700 1 -- 172.21.15.89:0/2852349486 reap_dead start 2023-10-10T20:42:05.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.023+0000 7f85a7aa4700 1 -- 172.21.15.89:0/2852349486 shutdown_connections 2023-10-10T20:42:05.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.023+0000 7f85a7aa4700 1 -- 172.21.15.89:0/2852349486 >> 172.21.15.89:0/2852349486 conn(0x7f85a00999f0 msgr2=0x7f85a00b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:05.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.023+0000 7f85a7aa4700 1 -- 172.21.15.89:0/2852349486 shutdown_connections 2023-10-10T20:42:05.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.023+0000 7f85a7aa4700 1 -- 172.21.15.89:0/2852349486 wait complete. 2023-10-10T20:42:05.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.024+0000 7f85a7aa4700 1 Processor -- start 2023-10-10T20:42:05.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.024+0000 7f85a7aa4700 1 -- start start 2023-10-10T20:42:05.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f85a7aa4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8588003fe0 0x7f85a009d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:05.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f85a7aa4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85a0151500 0x7f85a016a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:05.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f85a7aa4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85a0151eb0 0x7f85a0152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:05.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f85a7aa4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a016aa00 con 0x7f85a0151eb0 2023-10-10T20:42:05.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f85a7aa4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a0152860 con 0x7f8588003fe0 2023-10-10T20:42:05.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f85a7aa4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a01540a0 con 0x7f85a0151500 2023-10-10T20:42:05.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f85a6041700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85a0151eb0 0x7f85a0152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:05.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f85a6041700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85a0151eb0 0x7f85a0152320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44108/0 (socket says 172.21.15.89:44108) 2023-10-10T20:42:05.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f85a6041700 1 -- 172.21.15.89:0/1821571841 learned_addr learned my addr 172.21.15.89:0/1821571841 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:05.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f85a5840700 1 --2- 172.21.15.89:0/1821571841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8588003fe0 0x7f85a009d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:05.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f85a503f700 1 --2- 172.21.15.89:0/1821571841 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85a0151500 0x7f85a016a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:05.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f85a6041700 1 -- 172.21.15.89:0/1821571841 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85a0151500 msgr2=0x7f85a016a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:05.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f85a6041700 1 --2- 172.21.15.89:0/1821571841 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85a0151500 0x7f85a016a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:05.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f85a6041700 1 -- 172.21.15.89:0/1821571841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8588003fe0 msgr2=0x7f85a009d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:05.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f85a6041700 1 --2- 172.21.15.89:0/1821571841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8588003fe0 0x7f85a009d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:05.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f85a6041700 1 -- 172.21.15.89:0/1821571841 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a0154320 con 0x7f85a0151eb0 2023-10-10T20:42:05.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f85a6041700 1 --2- 172.21.15.89:0/1821571841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85a0151eb0 0x7f85a0152320 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f85a00b61c0 tx=0x7f859000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:05.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f859dffb700 1 -- 172.21.15.89:0/1821571841 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859000e040 con 0x7f85a0151eb0 2023-10-10T20:42:05.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f859dffb700 1 -- 172.21.15.89:0/1821571841 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859000f040 con 0x7f85a0151eb0 2023-10-10T20:42:05.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f85a7aa4700 1 -- 172.21.15.89:0/1821571841 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a0154600 con 0x7f85a0151eb0 2023-10-10T20:42:05.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.025+0000 7f859dffb700 1 -- 172.21.15.89:0/1821571841 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859000fe20 con 0x7f85a0151eb0 2023-10-10T20:42:05.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.026+0000 7f85a7aa4700 1 -- 172.21.15.89:0/1821571841 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a0154bc0 con 0x7f85a0151eb0 2023-10-10T20:42:05.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.026+0000 7f85a7aa4700 1 -- 172.21.15.89:0/1821571841 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a0063c90 con 0x7f85a0151eb0 2023-10-10T20:42:05.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.027+0000 7f859dffb700 1 -- 172.21.15.89:0/1821571841 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8590015640 con 0x7f85a0151eb0 2023-10-10T20:42:05.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.027+0000 7f859dffb700 1 --2- 172.21.15.89:0/1821571841 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f858c067350 0x7f858c069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:05.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.028+0000 7f85a5840700 1 --2- 172.21.15.89:0/1821571841 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f858c067350 0x7f858c069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:05.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.028+0000 7f859dffb700 1 -- 172.21.15.89:0/1821571841 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8590085610 con 0x7f85a0151eb0 2023-10-10T20:42:05.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.028+0000 7f85a5840700 1 --2- 172.21.15.89:0/1821571841 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f858c067350 0x7f858c069800 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8598009390 tx=0x7f8598008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:05.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.032+0000 7f859dffb700 1 -- 172.21.15.89:0/1821571841 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8590052150 con 0x7f85a0151eb0 2023-10-10T20:42:05.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:05.242+0000 7f85a7aa4700 1 -- 172.21.15.89:0/1821571841 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000887918", "yes_i_really_mean_it": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f85a00718b0 con 0x7f858c067350 2023-10-10T20:42:07.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:07.993+0000 7f859dffb700 1 -- 172.21.15.89:0/1821571841 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+56 (secure 0 0 0) 0x7f85a00718b0 con 0x7f858c067350 2023-10-10T20:42:07.995 INFO:teuthology.orchestra.run.smithi089.stdout:FS volume 'cephfs' renamed to 'volume_0000000000887918' 2023-10-10T20:42:07.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:07.995+0000 7f85a7aa4700 1 -- 172.21.15.89:0/1821571841 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f858c067350 msgr2=0x7f858c069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:07.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:07.995+0000 7f85a7aa4700 1 --2- 172.21.15.89:0/1821571841 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f858c067350 0x7f858c069800 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8598009390 tx=0x7f8598008040 comp rx=0 tx=0).stop 2023-10-10T20:42:07.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:07.995+0000 7f85a7aa4700 1 -- 172.21.15.89:0/1821571841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85a0151eb0 msgr2=0x7f85a0152320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:07.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:07.995+0000 7f85a7aa4700 1 --2- 172.21.15.89:0/1821571841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85a0151eb0 0x7f85a0152320 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f85a00b61c0 tx=0x7f859000b040 comp rx=0 tx=0).stop 2023-10-10T20:42:07.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:07.995+0000 7f85a7aa4700 1 -- 172.21.15.89:0/1821571841 shutdown_connections 2023-10-10T20:42:07.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:07.995+0000 7f85a7aa4700 1 --2- 172.21.15.89:0/1821571841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85a0151eb0 0x7f85a0152320 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:07.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:07.995+0000 7f85a7aa4700 1 --2- 172.21.15.89:0/1821571841 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f858c067350 0x7f858c069800 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:07.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:07.995+0000 7f85a7aa4700 1 --2- 172.21.15.89:0/1821571841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8588003fe0 0x7f85a009d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:07.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:07.995+0000 7f85a7aa4700 1 --2- 172.21.15.89:0/1821571841 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85a0151500 0x7f85a016a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:07.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:07.995+0000 7f85a7aa4700 1 -- 172.21.15.89:0/1821571841 >> 172.21.15.89:0/1821571841 conn(0x7f85a00999f0 msgr2=0x7f85a00b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:08.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:07.995+0000 7f85a7aa4700 1 -- 172.21.15.89:0/1821571841 shutdown_connections 2023-10-10T20:42:08.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:07.996+0000 7f85a7aa4700 1 -- 172.21.15.89:0/1821571841 wait complete. 2023-10-10T20:42:08.013 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-10T20:42:08.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.132+0000 7ff2351a6700 1 Processor -- start 2023-10-10T20:42:08.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.132+0000 7ff2351a6700 1 -- start start 2023-10-10T20:42:08.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.133+0000 7ff2351a6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff230159c60 0x7ff23015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:08.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.133+0000 7ff2351a6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23015a160 con 0x7ff23015db10 2023-10-10T20:42:08.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.133+0000 7ff2351a6700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2300d13a0 con 0x7ff230159890 2023-10-10T20:42:08.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.133+0000 7ff2351a6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff23015df50 con 0x7ff230159c60 2023-10-10T20:42:08.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.133+0000 7ff22659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff230159c60 0x7ff23015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:08.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.133+0000 7ff22f59e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff23015db10 0x7ff230069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35658/0 (socket says 172.21.15.89:35658) 2023-10-10T20:42:08.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.133+0000 7ff22f59e700 1 -- 172.21.15.89:0/2378735778 learned_addr learned my addr 172.21.15.89:0/2378735778 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:08.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.133+0000 7ff22659c700 1 -- 172.21.15.89:0/2378735778 >> v1:172.21.15.111:6789/0 conn(0x7ff230159890 legacy=0x7ff23015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:08.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.133+0000 7ff22659c700 1 -- 172.21.15.89:0/2378735778 >> v1:172.21.15.89:6789/0 conn(0x7ff23015db10 legacy=0x7ff230069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:08.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.133+0000 7ff22659c700 1 -- 172.21.15.89:0/2378735778 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23015e140 con 0x7ff230159c60 2023-10-10T20:42:08.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.133+0000 7ff22e59c700 1 -- 172.21.15.89:0/2378735778 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3161451539 0 0) 0x7ff23015a160 con 0x7ff23015db10 2023-10-10T20:42:08.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.134+0000 7ff22659c700 1 --2- 172.21.15.89:0/2378735778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff230159c60 0x7ff23015d5d0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7ff2180098e0 tx=0x7ff2180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119583a097938195 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:08.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.134+0000 7ff22e59c700 1 -- 172.21.15.89:0/2378735778 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff218009e80 con 0x7ff230159c60 2023-10-10T20:42:08.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.134+0000 7ff22e59c700 1 -- 172.21.15.89:0/2378735778 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21800b820 con 0x7ff230159c60 2023-10-10T20:42:08.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.134+0000 7ff22e59c700 1 -- 172.21.15.89:0/2378735778 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21800bba0 con 0x7ff230159c60 2023-10-10T20:42:08.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.134+0000 7ff2351a6700 1 -- 172.21.15.89:0/2378735778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff230159c60 msgr2=0x7ff23015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:08.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.134+0000 7ff2351a6700 1 --2- 172.21.15.89:0/2378735778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff230159c60 0x7ff23015d5d0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7ff2180098e0 tx=0x7ff2180091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.134+0000 7ff2351a6700 1 -- 172.21.15.89:0/2378735778 shutdown_connections 2023-10-10T20:42:08.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.134+0000 7ff2351a6700 1 --2- 172.21.15.89:0/2378735778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff230159c60 0x7ff23015d5d0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.134+0000 7ff2351a6700 1 -- 172.21.15.89:0/2378735778 >> 172.21.15.89:0/2378735778 conn(0x7ff230068e20 msgr2=0x7ff2300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:08.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.134+0000 7ff2351a6700 1 -- 172.21.15.89:0/2378735778 shutdown_connections 2023-10-10T20:42:08.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.134+0000 7ff2351a6700 1 -- 172.21.15.89:0/2378735778 wait complete. 2023-10-10T20:42:08.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.136+0000 7ff2351a6700 1 Processor -- start 2023-10-10T20:42:08.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.136+0000 7ff2351a6700 1 -- start start 2023-10-10T20:42:08.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.136+0000 7ff2351a6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff230159890 0x7ff2301474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:08.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.136+0000 7ff2351a6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff230159c60 0x7ff2300cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:08.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.136+0000 7ff2351a6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff23015db10 0x7ff2300b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:08.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.136+0000 7ff2351a6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23016db40 con 0x7ff230159c60 2023-10-10T20:42:08.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.136+0000 7ff2351a6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23016cf70 con 0x7ff23015db10 2023-10-10T20:42:08.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.136+0000 7ff2351a6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23016d210 con 0x7ff230159890 2023-10-10T20:42:08.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.136+0000 7ff22659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff230159c60 0x7ff2300cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:08.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.136+0000 7ff22659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff230159c60 0x7ff2300cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44128/0 (socket says 172.21.15.89:44128) 2023-10-10T20:42:08.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.136+0000 7ff22659c700 1 -- 172.21.15.89:0/2950789685 learned_addr learned my addr 172.21.15.89:0/2950789685 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:08.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.137+0000 7ff22659c700 1 -- 172.21.15.89:0/2950789685 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff230159890 msgr2=0x7ff2301474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:08.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.137+0000 7ff22f59e700 1 --2- 172.21.15.89:0/2950789685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff23015db10 0x7ff2300b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:08.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.137+0000 7ff22659c700 1 --2- 172.21.15.89:0/2950789685 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff230159890 0x7ff2301474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.137+0000 7ff22659c700 1 -- 172.21.15.89:0/2950789685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff23015db10 msgr2=0x7ff2300b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:08.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.137+0000 7ff22659c700 1 --2- 172.21.15.89:0/2950789685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff23015db10 0x7ff2300b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.137+0000 7ff22659c700 1 -- 172.21.15.89:0/2950789685 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff218009420 con 0x7ff230159c60 2023-10-10T20:42:08.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.137+0000 7ff22f59e700 1 --2- 172.21.15.89:0/2950789685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff23015db10 0x7ff2300b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:08.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.137+0000 7ff22659c700 1 --2- 172.21.15.89:0/2950789685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff230159c60 0x7ff2300cebf0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7ff2180030c0 tx=0x7ff218016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:08.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.137+0000 7ff2277fe700 1 -- 172.21.15.89:0/2950789685 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21800bae0 con 0x7ff230159c60 2023-10-10T20:42:08.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.137+0000 7ff2351a6700 1 -- 172.21.15.89:0/2950789685 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23016d4f0 con 0x7ff230159c60 2023-10-10T20:42:08.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.137+0000 7ff2277fe700 1 -- 172.21.15.89:0/2950789685 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff218020070 con 0x7ff230159c60 2023-10-10T20:42:08.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.137+0000 7ff2277fe700 1 -- 172.21.15.89:0/2950789685 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21801b6a0 con 0x7ff230159c60 2023-10-10T20:42:08.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.137+0000 7ff2351a6700 1 -- 172.21.15.89:0/2950789685 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2300cf130 con 0x7ff230159c60 2023-10-10T20:42:08.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.138+0000 7ff2351a6700 1 -- 172.21.15.89:0/2950789685 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23004c3c0 con 0x7ff230159c60 2023-10-10T20:42:08.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.138+0000 7ff2277fe700 1 -- 172.21.15.89:0/2950789685 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff218022030 con 0x7ff230159c60 2023-10-10T20:42:08.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.139+0000 7ff2277fe700 1 --2- 172.21.15.89:0/2950789685 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff210067320 0x7ff2100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:08.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.139+0000 7ff2277fe700 1 -- 172.21.15.89:0/2950789685 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5783+0+0 (secure 0 0 0) 0x7ff218086ca0 con 0x7ff230159c60 2023-10-10T20:42:08.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.139+0000 7ff22ed9d700 1 --2- 172.21.15.89:0/2950789685 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff210067320 0x7ff2100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:08.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.139+0000 7ff22ed9d700 1 --2- 172.21.15.89:0/2950789685 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff210067320 0x7ff2100697d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff220009740 tx=0x7ff220006c60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:08.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.143+0000 7ff2277fe700 1 -- 172.21.15.89:0/2950789685 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2180539f0 con 0x7ff230159c60 2023-10-10T20:42:08.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.356+0000 7ff2351a6700 1 -- 172.21.15.89:0/2950789685 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff23016dc80 con 0x7ff210067320 2023-10-10T20:42:08.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.357+0000 7ff2277fe700 1 -- 172.21.15.89:0/2950789685 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+58 (secure 0 0 0) 0x7ff2200113f0 con 0x7ff210067320 2023-10-10T20:42:08.359 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:42:08.360 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:08.360 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "volume_0000000000887918" 2023-10-10T20:42:08.360 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:08.361 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:42:08.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.359+0000 7ff2351a6700 1 -- 172.21.15.89:0/2950789685 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff210067320 msgr2=0x7ff2100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:08.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.359+0000 7ff2351a6700 1 --2- 172.21.15.89:0/2950789685 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff210067320 0x7ff2100697d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff220009740 tx=0x7ff220006c60 comp rx=0 tx=0).stop 2023-10-10T20:42:08.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.359+0000 7ff2351a6700 1 -- 172.21.15.89:0/2950789685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff230159c60 msgr2=0x7ff2300cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:08.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.359+0000 7ff2351a6700 1 --2- 172.21.15.89:0/2950789685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff230159c60 0x7ff2300cebf0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7ff2180030c0 tx=0x7ff218016000 comp rx=0 tx=0).stop 2023-10-10T20:42:08.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.359+0000 7ff2351a6700 1 -- 172.21.15.89:0/2950789685 shutdown_connections 2023-10-10T20:42:08.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.359+0000 7ff2351a6700 1 --2- 172.21.15.89:0/2950789685 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff210067320 0x7ff2100697d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.359+0000 7ff2351a6700 1 --2- 172.21.15.89:0/2950789685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff23015db10 0x7ff2300b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.359+0000 7ff2351a6700 1 --2- 172.21.15.89:0/2950789685 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff230159890 0x7ff2301474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.359+0000 7ff2351a6700 1 --2- 172.21.15.89:0/2950789685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff230159c60 0x7ff2300cebf0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.359+0000 7ff2351a6700 1 -- 172.21.15.89:0/2950789685 >> 172.21.15.89:0/2950789685 conn(0x7ff230068e20 msgr2=0x7ff2301574d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:08.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.359+0000 7ff2351a6700 1 -- 172.21.15.89:0/2950789685 shutdown_connections 2023-10-10T20:42:08.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.359+0000 7ff2351a6700 1 -- 172.21.15.89:0/2950789685 wait complete. 2023-10-10T20:42:08.373 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:42:08.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.486+0000 7f3a23453700 1 Processor -- start 2023-10-10T20:42:08.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.486+0000 7f3a23453700 1 -- start start 2023-10-10T20:42:08.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.487+0000 7f3a23453700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3a1c159c60 0x7f3a1c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:08.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.487+0000 7f3a23453700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3a1c15db10 0x7f3a1c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:08.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.487+0000 7f3a23453700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a1c15a160 con 0x7f3a1c159890 2023-10-10T20:42:08.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.487+0000 7f3a23453700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a1c15df50 con 0x7f3a1c159c60 2023-10-10T20:42:08.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.487+0000 7f3a23453700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a1c0d13a0 con 0x7f3a1c15db10 2023-10-10T20:42:08.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.487+0000 7f3a211ef700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3a1c159890 0x7f3a1c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58716/0 (socket says 172.21.15.89:58716) 2023-10-10T20:42:08.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.487+0000 7f3a211ef700 1 -- 172.21.15.89:0/3615514853 learned_addr learned my addr 172.21.15.89:0/3615514853 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:08.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.487+0000 7f3a209ee700 1 --2- 172.21.15.89:0/3615514853 >> v2:172.21.15.89:3300/0 conn(0x7f3a1c159c60 0x7f3a1c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:08.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.488+0000 7f3a13fff700 1 -- 172.21.15.89:0/3615514853 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3579934862 0 0) 0x7f3a1c15a160 con 0x7f3a1c159890 2023-10-10T20:42:08.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.488+0000 7f3a219f0700 1 --2- 172.21.15.89:0/3615514853 >> v2:172.21.15.111:3300/0 conn(0x7f3a1c15db10 0x7f3a1c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:08.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.488+0000 7f3a13fff700 1 -- 172.21.15.89:0/3615514853 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a08003570 con 0x7f3a1c159890 2023-10-10T20:42:08.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.488+0000 7f3a209ee700 1 -- 172.21.15.89:0/3615514853 >> v2:172.21.15.111:3300/0 conn(0x7f3a1c15db10 msgr2=0x7f3a1c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:08.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.488+0000 7f3a209ee700 1 --2- 172.21.15.89:0/3615514853 >> v2:172.21.15.111:3300/0 conn(0x7f3a1c15db10 0x7f3a1c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.488+0000 7f3a209ee700 1 -- 172.21.15.89:0/3615514853 >> v1:172.21.15.89:6789/0 conn(0x7f3a1c159890 legacy=0x7f3a1c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:08.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.488+0000 7f3a209ee700 1 -- 172.21.15.89:0/3615514853 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c15e140 con 0x7f3a1c159c60 2023-10-10T20:42:08.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.488+0000 7f3a209ee700 1 --2- 172.21.15.89:0/3615514853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a1c159c60 0x7f3a1c15d5d0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c06a890 tx=0x7f3a0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b76b5c12f54f972 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:08.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.488+0000 7f3a13fff700 1 -- 172.21.15.89:0/3615514853 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c011630 con 0x7f3a1c159c60 2023-10-10T20:42:08.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.488+0000 7f3a13fff700 1 -- 172.21.15.89:0/3615514853 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c011790 con 0x7f3a1c159c60 2023-10-10T20:42:08.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.488+0000 7f3a13fff700 1 -- 172.21.15.89:0/3615514853 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c011b30 con 0x7f3a1c159c60 2023-10-10T20:42:08.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.488+0000 7f3a23453700 1 -- 172.21.15.89:0/3615514853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a1c159c60 msgr2=0x7f3a1c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:08.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.488+0000 7f3a23453700 1 --2- 172.21.15.89:0/3615514853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a1c159c60 0x7f3a1c15d5d0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c06a890 tx=0x7f3a0c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.489+0000 7f3a23453700 1 -- 172.21.15.89:0/3615514853 shutdown_connections 2023-10-10T20:42:08.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.489+0000 7f3a23453700 1 --2- 172.21.15.89:0/3615514853 >> v2:172.21.15.111:3300/0 conn(0x7f3a1c15db10 0x7f3a1c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.489+0000 7f3a23453700 1 --2- 172.21.15.89:0/3615514853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a1c159c60 0x7f3a1c15d5d0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.489+0000 7f3a23453700 1 -- 172.21.15.89:0/3615514853 >> 172.21.15.89:0/3615514853 conn(0x7f3a1c068e20 msgr2=0x7f3a1c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:08.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.489+0000 7f3a23453700 1 -- 172.21.15.89:0/3615514853 shutdown_connections 2023-10-10T20:42:08.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.489+0000 7f3a23453700 1 -- 172.21.15.89:0/3615514853 wait complete. 2023-10-10T20:42:08.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.490+0000 7f3a23453700 1 Processor -- start 2023-10-10T20:42:08.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.490+0000 7f3a23453700 1 -- start start 2023-10-10T20:42:08.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a23453700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a1c159890 0x7f3a1c0cf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:08.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a23453700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a1c159c60 0x7f3a1c147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:08.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a23453700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a1c15db10 0x7f3a1c169cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:08.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a23453700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c0b0540 con 0x7f3a1c159890 2023-10-10T20:42:08.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a23453700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c0b5a60 con 0x7f3a1c15db10 2023-10-10T20:42:08.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a23453700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c0b5d00 con 0x7f3a1c159c60 2023-10-10T20:42:08.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a211ef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a1c159890 0x7f3a1c0cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:08.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a211ef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a1c159890 0x7f3a1c0cf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54538/0 (socket says 172.21.15.89:54538) 2023-10-10T20:42:08.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a211ef700 1 -- 172.21.15.89:0/2866309419 learned_addr learned my addr 172.21.15.89:0/2866309419 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:08.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a219f0700 1 --2- 172.21.15.89:0/2866309419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a1c15db10 0x7f3a1c169cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:08.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a209ee700 1 --2- 172.21.15.89:0/2866309419 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a1c159c60 0x7f3a1c147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:08.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a211ef700 1 -- 172.21.15.89:0/2866309419 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a1c159c60 msgr2=0x7f3a1c147150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:08.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a211ef700 1 --2- 172.21.15.89:0/2866309419 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a1c159c60 0x7f3a1c147150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a211ef700 1 -- 172.21.15.89:0/2866309419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a1c15db10 msgr2=0x7f3a1c169cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:08.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a211ef700 1 --2- 172.21.15.89:0/2866309419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a1c15db10 0x7f3a1c169cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a211ef700 1 -- 172.21.15.89:0/2866309419 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0c009420 con 0x7f3a1c159890 2023-10-10T20:42:08.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a209ee700 1 --2- 172.21.15.89:0/2866309419 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a1c159c60 0x7f3a1c147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:08.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a211ef700 1 --2- 172.21.15.89:0/2866309419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a1c159890 0x7f3a1c0cf910 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800cd30 tx=0x7f3a18001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:08.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a117fa700 1 -- 172.21.15.89:0/2866309419 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1800ce90 con 0x7f3a1c159890 2023-10-10T20:42:08.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a117fa700 1 -- 172.21.15.89:0/2866309419 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1800ab70 con 0x7f3a1c159890 2023-10-10T20:42:08.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a23453700 1 -- 172.21.15.89:0/2866309419 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1c0b5fe0 con 0x7f3a1c159890 2023-10-10T20:42:08.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a117fa700 1 -- 172.21.15.89:0/2866309419 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a180070f0 con 0x7f3a1c159890 2023-10-10T20:42:08.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.491+0000 7f3a23453700 1 -- 172.21.15.89:0/2866309419 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1c0cfe50 con 0x7f3a1c159890 2023-10-10T20:42:08.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.492+0000 7f3a23453700 1 -- 172.21.15.89:0/2866309419 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1c04c3c0 con 0x7f3a1c159890 2023-10-10T20:42:08.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.493+0000 7f3a117fa700 1 -- 172.21.15.89:0/2866309419 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3a18007520 con 0x7f3a1c159890 2023-10-10T20:42:08.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.493+0000 7f3a117fa700 1 --2- 172.21.15.89:0/2866309419 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3a00067320 0x7f3a000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:08.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.493+0000 7f3a117fa700 1 -- 172.21.15.89:0/2866309419 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f3a18085140 con 0x7f3a1c159890 2023-10-10T20:42:08.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.493+0000 7f3a209ee700 1 --2- 172.21.15.89:0/2866309419 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3a00067320 0x7f3a000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:08.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.494+0000 7f3a209ee700 1 --2- 172.21.15.89:0/2866309419 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3a00067320 0x7f3a000697d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0093f0 tx=0x7f3a0c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:08.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.497+0000 7f3a117fa700 1 -- 172.21.15.89:0/2866309419 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a18051e00 con 0x7f3a1c159890 2023-10-10T20:42:08.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.721+0000 7f3a23453700 1 -- 172.21.15.89:0/2866309419 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a1c156e90 con 0x7f3a1c159890 2023-10-10T20:42:08.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.722+0000 7f3a117fa700 1 -- 172.21.15.89:0/2866309419 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4618 (secure 0 0 0) 0x7f3a18055420 con 0x7f3a1c159890 2023-10-10T20:42:08.724 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:08.724 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":93,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6197,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/4123310016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4123310016},{"type":"v1","addr":"172.21.15.111:6834","nonce":4123310016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":6200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/141017824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":141017824},{"type":"v1","addr":"172.21.15.111:6835","nonce":141017824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7575,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/728309352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":728309352},{"type":"v1","addr":"172.21.15.89:6836","nonce":728309352}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:41:49.687845+0000","modified":"2023-10-10T20:42:05.985626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7578":{"gid":7578,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/2102263806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2102263806},{"type":"v1","addr":"172.21.15.89:6837","nonce":2102263806}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"volume_0000000000887918","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-10T20:42:08.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.724+0000 7f3a23453700 1 -- 172.21.15.89:0/2866309419 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3a00067320 msgr2=0x7f3a000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:08.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.724+0000 7f3a23453700 1 --2- 172.21.15.89:0/2866309419 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3a00067320 0x7f3a000697d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0093f0 tx=0x7f3a0c017000 comp rx=0 tx=0).stop 2023-10-10T20:42:08.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.724+0000 7f3a23453700 1 -- 172.21.15.89:0/2866309419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a1c159890 msgr2=0x7f3a1c0cf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:08.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.724+0000 7f3a23453700 1 --2- 172.21.15.89:0/2866309419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a1c159890 0x7f3a1c0cf910 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800cd30 tx=0x7f3a18001b10 comp rx=0 tx=0).stop 2023-10-10T20:42:08.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.724+0000 7f3a23453700 1 -- 172.21.15.89:0/2866309419 shutdown_connections 2023-10-10T20:42:08.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.724+0000 7f3a23453700 1 --2- 172.21.15.89:0/2866309419 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3a00067320 0x7f3a000697d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.724+0000 7f3a23453700 1 --2- 172.21.15.89:0/2866309419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a1c15db10 0x7f3a1c169cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.724+0000 7f3a23453700 1 --2- 172.21.15.89:0/2866309419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a1c159890 0x7f3a1c0cf910 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.724+0000 7f3a23453700 1 --2- 172.21.15.89:0/2866309419 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a1c159c60 0x7f3a1c147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.724+0000 7f3a23453700 1 -- 172.21.15.89:0/2866309419 >> 172.21.15.89:0/2866309419 conn(0x7f3a1c068e20 msgr2=0x7f3a1c153100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:08.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.724+0000 7f3a23453700 1 -- 172.21.15.89:0/2866309419 shutdown_connections 2023-10-10T20:42:08.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.724+0000 7f3a23453700 1 -- 172.21.15.89:0/2866309419 wait complete. 2023-10-10T20:42:08.729 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 93 2023-10-10T20:42:08.738 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:42:08.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.850+0000 7fc8cc860700 1 Processor -- start 2023-10-10T20:42:08.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.850+0000 7fc8cc860700 1 -- start start 2023-10-10T20:42:08.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.851+0000 7fc8cc860700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8c806bcd0 0x7fc8c8140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:08.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.851+0000 7fc8cc860700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc8c806c0a0 0x7fc8c8141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:08.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.851+0000 7fc8cc860700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc8c80c8e00 0x7fc8c80c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:08.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.851+0000 7fc8cc860700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c815c2a0 con 0x7fc8c80c8e00 2023-10-10T20:42:08.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.851+0000 7fc8cc860700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c815b690 con 0x7fc8c806c0a0 2023-10-10T20:42:08.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.851+0000 7fc8cc860700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c815b970 con 0x7fc8c806bcd0 2023-10-10T20:42:08.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.851+0000 7fc8c6d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc8c80c8e00 0x7fc8c80c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:08.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.851+0000 7fc8c6d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc8c80c8e00 0x7fc8c80c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54540/0 (socket says 172.21.15.89:54540) 2023-10-10T20:42:08.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.851+0000 7fc8c6d9d700 1 -- 172.21.15.89:0/3679487814 learned_addr learned my addr 172.21.15.89:0/3679487814 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:08.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.852+0000 7fc8c5d9b700 1 --2- 172.21.15.89:0/3679487814 >> v2:172.21.15.111:3300/0 conn(0x7fc8c806c0a0 0x7fc8c8141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:08.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.852+0000 7fc8c659c700 1 --2- 172.21.15.89:0/3679487814 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8c806bcd0 0x7fc8c8140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:08.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.852+0000 7fc8c6d9d700 1 -- 172.21.15.89:0/3679487814 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8c806bcd0 msgr2=0x7fc8c8140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:08.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.852+0000 7fc8c6d9d700 1 --2- 172.21.15.89:0/3679487814 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8c806bcd0 0x7fc8c8140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.852+0000 7fc8c6d9d700 1 -- 172.21.15.89:0/3679487814 >> v2:172.21.15.111:3300/0 conn(0x7fc8c806c0a0 msgr2=0x7fc8c8141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:08.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.852+0000 7fc8c6d9d700 1 --2- 172.21.15.89:0/3679487814 >> v2:172.21.15.111:3300/0 conn(0x7fc8c806c0a0 0x7fc8c8141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.852+0000 7fc8c6d9d700 1 -- 172.21.15.89:0/3679487814 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c815bc50 con 0x7fc8c80c8e00 2023-10-10T20:42:08.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.852+0000 7fc8c6d9d700 1 --2- 172.21.15.89:0/3679487814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8c80c8e00 0x7fc8c80c9260 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80c5660 tx=0x7fc8b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fc2dd77e5d4a8d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:08.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.852+0000 7fc8c559a700 1 -- 172.21.15.89:0/3679487814 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b800b640 con 0x7fc8c80c8e00 2023-10-10T20:42:08.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.852+0000 7fc8c559a700 1 -- 172.21.15.89:0/3679487814 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b800b7a0 con 0x7fc8c80c8e00 2023-10-10T20:42:08.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.852+0000 7fc8c559a700 1 -- 172.21.15.89:0/3679487814 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b800bb60 con 0x7fc8c80c8e00 2023-10-10T20:42:08.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.853+0000 7fc8cc860700 1 -- 172.21.15.89:0/3679487814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8c80c8e00 msgr2=0x7fc8c80c9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:08.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.853+0000 7fc8cc860700 1 --2- 172.21.15.89:0/3679487814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8c80c8e00 0x7fc8c80c9260 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80c5660 tx=0x7fc8b8009250 comp rx=0 tx=0).stop 2023-10-10T20:42:08.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.853+0000 7fc8cc860700 1 -- 172.21.15.89:0/3679487814 shutdown_connections 2023-10-10T20:42:08.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.853+0000 7fc8cc860700 1 --2- 172.21.15.89:0/3679487814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8c80c8e00 0x7fc8c80c9260 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.853+0000 7fc8cc860700 1 --2- 172.21.15.89:0/3679487814 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8c806bcd0 0x7fc8c8140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.853+0000 7fc8cc860700 1 --2- 172.21.15.89:0/3679487814 >> v2:172.21.15.111:3300/0 conn(0x7fc8c806c0a0 0x7fc8c8141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.853+0000 7fc8cc860700 1 -- 172.21.15.89:0/3679487814 >> 172.21.15.89:0/3679487814 conn(0x7fc8c80ce600 msgr2=0x7fc8c80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:08.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.853+0000 7fc8cc860700 1 -- 172.21.15.89:0/3679487814 shutdown_connections 2023-10-10T20:42:08.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.853+0000 7fc8cc860700 1 -- 172.21.15.89:0/3679487814 wait complete. 2023-10-10T20:42:08.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.854+0000 7fc8cc860700 1 Processor -- start 2023-10-10T20:42:08.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.854+0000 7fc8cc860700 1 -- start start 2023-10-10T20:42:08.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8cc860700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8c806bcd0 0x7fc8c813c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:08.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8cc860700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8c806c0a0 0x7fc8c80cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:08.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8cc860700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8c80c8e00 0x7fc8c80cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:08.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8cc860700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c813c850 con 0x7fc8c806bcd0 2023-10-10T20:42:08.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8cc860700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c80c6360 con 0x7fc8c806c0a0 2023-10-10T20:42:08.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8cc860700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c80c5750 con 0x7fc8c80c8e00 2023-10-10T20:42:08.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8c659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8c806bcd0 0x7fc8c813c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:08.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8c659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8c806bcd0 0x7fc8c813c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54554/0 (socket says 172.21.15.89:54554) 2023-10-10T20:42:08.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8c659c700 1 -- 172.21.15.89:0/920946591 learned_addr learned my addr 172.21.15.89:0/920946591 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:08.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8c6d9d700 1 --2- 172.21.15.89:0/920946591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8c80c8e00 0x7fc8c80cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:08.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8c5d9b700 1 --2- 172.21.15.89:0/920946591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8c806c0a0 0x7fc8c80cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:08.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8c659c700 1 -- 172.21.15.89:0/920946591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8c80c8e00 msgr2=0x7fc8c80cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:08.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8c659c700 1 --2- 172.21.15.89:0/920946591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8c80c8e00 0x7fc8c80cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8c659c700 1 -- 172.21.15.89:0/920946591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8c806c0a0 msgr2=0x7fc8c80cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8c659c700 1 --2- 172.21.15.89:0/920946591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8c806c0a0 0x7fc8c80cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8c659c700 1 -- 172.21.15.89:0/920946591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b80094f0 con 0x7fc8c806bcd0 2023-10-10T20:42:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8c6d9d700 1 --2- 172.21.15.89:0/920946591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8c80c8e00 0x7fc8c80cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8c5d9b700 1 --2- 172.21.15.89:0/920946591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8c806c0a0 0x7fc8c80cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:08.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8c659c700 1 --2- 172.21.15.89:0/920946591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8c806bcd0 0x7fc8c813c310 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009ef0 tx=0x7fc8c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:08.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8be7fc700 1 -- 172.21.15.89:0/920946591 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c000cc50 con 0x7fc8c806bcd0 2023-10-10T20:42:08.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8be7fc700 1 -- 172.21.15.89:0/920946591 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c000f040 con 0x7fc8c806bcd0 2023-10-10T20:42:08.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8cc860700 1 -- 172.21.15.89:0/920946591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c80c5a30 con 0x7fc8c806bcd0 2023-10-10T20:42:08.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8be7fc700 1 -- 172.21.15.89:0/920946591 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0014480 con 0x7fc8c806bcd0 2023-10-10T20:42:08.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.855+0000 7fc8cc860700 1 -- 172.21.15.89:0/920946591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c80c5ff0 con 0x7fc8c806bcd0 2023-10-10T20:42:08.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.856+0000 7fc8cc860700 1 -- 172.21.15.89:0/920946591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c804c3c0 con 0x7fc8c806bcd0 2023-10-10T20:42:08.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.857+0000 7fc8be7fc700 1 -- 172.21.15.89:0/920946591 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc8c00147f0 con 0x7fc8c806bcd0 2023-10-10T20:42:08.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.857+0000 7fc8be7fc700 1 --2- 172.21.15.89:0/920946591 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc8a8067320 0x7fc8a80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:08.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.857+0000 7fc8be7fc700 1 -- 172.21.15.89:0/920946591 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fc8c0085b90 con 0x7fc8c806bcd0 2023-10-10T20:42:08.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.857+0000 7fc8c5d9b700 1 --2- 172.21.15.89:0/920946591 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc8a8067320 0x7fc8a80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.857+0000 7fc8c5d9b700 1 --2- 172.21.15.89:0/920946591 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc8a8067320 0x7fc8a80697d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8151200 tx=0x7fc8b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:08.861+0000 7fc8be7fc700 1 -- 172.21.15.89:0/920946591 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8c00526c0 con 0x7fc8c806bcd0 2023-10-10T20:42:09.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.047+0000 7fc8cc860700 1 -- 172.21.15.89:0/920946591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8c806c9d0 con 0x7fc8c806bcd0 2023-10-10T20:42:09.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.047+0000 7fc8be7fc700 1 -- 172.21.15.89:0/920946591 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+15442 (secure 0 0 0) 0x7fc8c0055ce0 con 0x7fc8c806bcd0 2023-10-10T20:42:09.050 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:09.050 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":141,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:42:07.988420+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs.volume_0000000000887918.meta","create_time":"2023-10-10T20:41:48.040814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000887918"}}},{"pool":25,"pool_name":"cephfs.volume_0000000000887918.data","create_time":"2023-10-10T20:41:48.646159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000887918"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:42:09.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.050+0000 7fc8cc860700 1 -- 172.21.15.89:0/920946591 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc8a8067320 msgr2=0x7fc8a80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:09.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.050+0000 7fc8cc860700 1 --2- 172.21.15.89:0/920946591 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc8a8067320 0x7fc8a80697d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8151200 tx=0x7fc8b0008040 comp rx=0 tx=0).stop 2023-10-10T20:42:09.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.050+0000 7fc8cc860700 1 -- 172.21.15.89:0/920946591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8c806bcd0 msgr2=0x7fc8c813c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:09.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.050+0000 7fc8cc860700 1 --2- 172.21.15.89:0/920946591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8c806bcd0 0x7fc8c813c310 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009ef0 tx=0x7fc8c000b040 comp rx=0 tx=0).stop 2023-10-10T20:42:09.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.050+0000 7fc8cc860700 1 -- 172.21.15.89:0/920946591 shutdown_connections 2023-10-10T20:42:09.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.050+0000 7fc8cc860700 1 --2- 172.21.15.89:0/920946591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc8c80c8e00 0x7fc8c80cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.050+0000 7fc8cc860700 1 --2- 172.21.15.89:0/920946591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8c806bcd0 0x7fc8c813c310 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.050+0000 7fc8cc860700 1 --2- 172.21.15.89:0/920946591 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc8a8067320 0x7fc8a80697d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.050+0000 7fc8cc860700 1 --2- 172.21.15.89:0/920946591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8c806c0a0 0x7fc8c80cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.050+0000 7fc8cc860700 1 -- 172.21.15.89:0/920946591 >> 172.21.15.89:0/920946591 conn(0x7fc8c80ce600 msgr2=0x7fc8c80c33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:09.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.050+0000 7fc8cc860700 1 -- 172.21.15.89:0/920946591 shutdown_connections 2023-10-10T20:42:09.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.050+0000 7fc8cc860700 1 -- 172.21.15.89:0/920946591 wait complete. 2023-10-10T20:42:09.065 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:42:09.066 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:42:09.107 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:42:09.151 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:42:09.151 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:42:09.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.264+0000 7f2d2cfd7700 1 Processor -- start 2023-10-10T20:42:09.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.264+0000 7f2d2cfd7700 1 -- start start 2023-10-10T20:42:09.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.265+0000 7f2d2cfd7700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2d281597f0 0x7f2d280b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.265+0000 7f2d2cfd7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2d2815d230 0x7f2d2815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.265+0000 7f2d2cfd7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2815db70 con 0x7f2d28159420 2023-10-10T20:42:09.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.265+0000 7f2d2cfd7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d280b11d0 con 0x7f2d2815d230 2023-10-10T20:42:09.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.265+0000 7f2d2cfd7700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d280b14b0 con 0x7f2d281597f0 2023-10-10T20:42:09.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.265+0000 7f2d27fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2d2815d230 0x7f2d2815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:09.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.265+0000 7f2d277fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2d28159420 0x7f2d280b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58722/0 (socket says 172.21.15.89:58722) 2023-10-10T20:42:09.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.265+0000 7f2d277fe700 1 -- 172.21.15.89:0/831458260 learned_addr learned my addr 172.21.15.89:0/831458260 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:09.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.265+0000 7f2d26ffd700 1 --2- 172.21.15.89:0/831458260 >> v2:172.21.15.111:3300/0 conn(0x7f2d281597f0 0x7f2d280b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:09.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.265+0000 7f2d267fc700 1 -- 172.21.15.89:0/831458260 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503953123 0 0) 0x7f2d2815db70 con 0x7f2d28159420 2023-10-10T20:42:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.265+0000 7f2d267fc700 1 -- 172.21.15.89:0/831458260 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d10003570 con 0x7f2d28159420 2023-10-10T20:42:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.265+0000 7f2d27fff700 1 -- 172.21.15.89:0/831458260 >> v2:172.21.15.111:3300/0 conn(0x7f2d281597f0 msgr2=0x7f2d280b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.265+0000 7f2d27fff700 1 --2- 172.21.15.89:0/831458260 >> v2:172.21.15.111:3300/0 conn(0x7f2d281597f0 0x7f2d280b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.265+0000 7f2d27fff700 1 -- 172.21.15.89:0/831458260 >> v1:172.21.15.89:6789/0 conn(0x7f2d28159420 legacy=0x7f2d280b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.265+0000 7f2d27fff700 1 -- 172.21.15.89:0/831458260 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d280b1790 con 0x7f2d2815d230 2023-10-10T20:42:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.266+0000 7f2d27fff700 1 --2- 172.21.15.89:0/831458260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d2815d230 0x7f2d2815d630 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f2d28089750 tx=0x7f2d200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df12f572af18418d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.266+0000 7f2d267fc700 1 -- 172.21.15.89:0/831458260 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20011630 con 0x7f2d2815d230 2023-10-10T20:42:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.266+0000 7f2d267fc700 1 -- 172.21.15.89:0/831458260 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d20011790 con 0x7f2d2815d230 2023-10-10T20:42:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.266+0000 7f2d267fc700 1 -- 172.21.15.89:0/831458260 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20011b30 con 0x7f2d2815d230 2023-10-10T20:42:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.266+0000 7f2d2cfd7700 1 -- 172.21.15.89:0/831458260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d2815d230 msgr2=0x7f2d2815d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:09.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.266+0000 7f2d2cfd7700 1 --2- 172.21.15.89:0/831458260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d2815d230 0x7f2d2815d630 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f2d28089750 tx=0x7f2d200091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.266+0000 7f2d2cfd7700 1 -- 172.21.15.89:0/831458260 shutdown_connections 2023-10-10T20:42:09.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.266+0000 7f2d2cfd7700 1 --2- 172.21.15.89:0/831458260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d2815d230 0x7f2d2815d630 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.266+0000 7f2d2cfd7700 1 --2- 172.21.15.89:0/831458260 >> v2:172.21.15.111:3300/0 conn(0x7f2d281597f0 0x7f2d280b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.266+0000 7f2d2cfd7700 1 -- 172.21.15.89:0/831458260 >> 172.21.15.89:0/831458260 conn(0x7f2d280d0660 msgr2=0x7f2d280d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:09.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.266+0000 7f2d2cfd7700 1 -- 172.21.15.89:0/831458260 shutdown_connections 2023-10-10T20:42:09.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.266+0000 7f2d2cfd7700 1 -- 172.21.15.89:0/831458260 wait complete. 2023-10-10T20:42:09.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.268+0000 7f2d2cfd7700 1 Processor -- start 2023-10-10T20:42:09.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.268+0000 7f2d2cfd7700 1 -- start start 2023-10-10T20:42:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.268+0000 7f2d2cfd7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d28159420 0x7f2d28154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.268+0000 7f2d2cfd7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d281597f0 0x7f2d2815ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.268+0000 7f2d2cfd7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d2815d230 0x7f2d2815c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.268+0000 7f2d2cfd7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d28155480 con 0x7f2d28159420 2023-10-10T20:42:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.268+0000 7f2d2cfd7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2815a9b0 con 0x7f2d281597f0 2023-10-10T20:42:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.268+0000 7f2d2cfd7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2815ac50 con 0x7f2d2815d230 2023-10-10T20:42:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.268+0000 7f2d277fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d28159420 0x7f2d28154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.268+0000 7f2d277fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d28159420 0x7f2d28154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54574/0 (socket says 172.21.15.89:54574) 2023-10-10T20:42:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.268+0000 7f2d277fe700 1 -- 172.21.15.89:0/1323177465 learned_addr learned my addr 172.21.15.89:0/1323177465 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.269+0000 7f2d26ffd700 1 --2- 172.21.15.89:0/1323177465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d281597f0 0x7f2d2815ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.269+0000 7f2d277fe700 1 -- 172.21.15.89:0/1323177465 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d2815d230 msgr2=0x7f2d2815c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.269+0000 7f2d277fe700 1 --2- 172.21.15.89:0/1323177465 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d2815d230 0x7f2d2815c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.269+0000 7f2d277fe700 1 -- 172.21.15.89:0/1323177465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d281597f0 msgr2=0x7f2d2815ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.269+0000 7f2d277fe700 1 --2- 172.21.15.89:0/1323177465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d281597f0 0x7f2d2815ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.269+0000 7f2d277fe700 1 -- 172.21.15.89:0/1323177465 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d20009420 con 0x7f2d28159420 2023-10-10T20:42:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.269+0000 7f2d277fe700 1 --2- 172.21.15.89:0/1323177465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d28159420 0x7f2d28154f40 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800cd30 tx=0x7f2d18001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.269+0000 7f2d0f7fe700 1 -- 172.21.15.89:0/1323177465 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1800ab70 con 0x7f2d28159420 2023-10-10T20:42:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.269+0000 7f2d0f7fe700 1 -- 172.21.15.89:0/1323177465 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1800acf0 con 0x7f2d28159420 2023-10-10T20:42:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.269+0000 7f2d2cfd7700 1 -- 172.21.15.89:0/1323177465 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2815af30 con 0x7f2d28159420 2023-10-10T20:42:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.269+0000 7f2d0f7fe700 1 -- 172.21.15.89:0/1323177465 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18007340 con 0x7f2d28159420 2023-10-10T20:42:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.269+0000 7f2d2cfd7700 1 -- 172.21.15.89:0/1323177465 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2815b4f0 con 0x7f2d28159420 2023-10-10T20:42:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.270+0000 7f2d2cfd7700 1 -- 172.21.15.89:0/1323177465 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2815b710 con 0x7f2d28159420 2023-10-10T20:42:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.270+0000 7f2d0f7fe700 1 -- 172.21.15.89:0/1323177465 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2d18007770 con 0x7f2d28159420 2023-10-10T20:42:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.271+0000 7f2d0f7fe700 1 --2- 172.21.15.89:0/1323177465 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2d14067320 0x7f2d140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.271+0000 7f2d0f7fe700 1 -- 172.21.15.89:0/1323177465 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f2d180851c0 con 0x7f2d28159420 2023-10-10T20:42:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.271+0000 7f2d26ffd700 1 --2- 172.21.15.89:0/1323177465 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2d14067320 0x7f2d140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.271+0000 7f2d26ffd700 1 --2- 172.21.15.89:0/1323177465 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2d14067320 0x7f2d140697d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c006fd0 tx=0x7f2d1c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.275+0000 7f2d0f7fe700 1 -- 172.21.15.89:0/1323177465 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d18051e80 con 0x7f2d28159420 2023-10-10T20:42:09.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.499+0000 7f2d2cfd7700 1 -- 172.21.15.89:0/1323177465 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d2804c3c0 con 0x7f2d28159420 2023-10-10T20:42:09.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.500+0000 7f2d0f7fe700 1 -- 172.21.15.89:0/1323177465 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4618 (secure 0 0 0) 0x7f2d1800ae60 con 0x7f2d28159420 2023-10-10T20:42:09.502 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:09.503 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":93,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6197,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/4123310016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4123310016},{"type":"v1","addr":"172.21.15.111:6834","nonce":4123310016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":6200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/141017824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":141017824},{"type":"v1","addr":"172.21.15.111:6835","nonce":141017824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7575,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/728309352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":728309352},{"type":"v1","addr":"172.21.15.89:6836","nonce":728309352}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:41:49.687845+0000","modified":"2023-10-10T20:42:05.985626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7578":{"gid":7578,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/2102263806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2102263806},{"type":"v1","addr":"172.21.15.89:6837","nonce":2102263806}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"volume_0000000000887918","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-10T20:42:09.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.502+0000 7f2d2cfd7700 1 -- 172.21.15.89:0/1323177465 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2d14067320 msgr2=0x7f2d140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:09.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.502+0000 7f2d2cfd7700 1 --2- 172.21.15.89:0/1323177465 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2d14067320 0x7f2d140697d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c006fd0 tx=0x7f2d1c008040 comp rx=0 tx=0).stop 2023-10-10T20:42:09.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.502+0000 7f2d2cfd7700 1 -- 172.21.15.89:0/1323177465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d28159420 msgr2=0x7f2d28154f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:09.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.502+0000 7f2d2cfd7700 1 --2- 172.21.15.89:0/1323177465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d28159420 0x7f2d28154f40 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800cd30 tx=0x7f2d18001b10 comp rx=0 tx=0).stop 2023-10-10T20:42:09.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.502+0000 7f2d2cfd7700 1 -- 172.21.15.89:0/1323177465 shutdown_connections 2023-10-10T20:42:09.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.502+0000 7f2d2cfd7700 1 --2- 172.21.15.89:0/1323177465 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d2815d230 0x7f2d2815c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.502+0000 7f2d2cfd7700 1 --2- 172.21.15.89:0/1323177465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d28159420 0x7f2d28154f40 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.502+0000 7f2d2cfd7700 1 --2- 172.21.15.89:0/1323177465 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2d14067320 0x7f2d140697d0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.502+0000 7f2d2cfd7700 1 --2- 172.21.15.89:0/1323177465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d281597f0 0x7f2d2815ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.502+0000 7f2d2cfd7700 1 -- 172.21.15.89:0/1323177465 >> 172.21.15.89:0/1323177465 conn(0x7f2d280d0660 msgr2=0x7f2d28069890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:09.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.502+0000 7f2d2cfd7700 1 -- 172.21.15.89:0/1323177465 shutdown_connections 2023-10-10T20:42:09.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.502+0000 7f2d2cfd7700 1 -- 172.21.15.89:0/1323177465 wait complete. 2023-10-10T20:42:09.507 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 93 2023-10-10T20:42:09.516 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:42:09.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.629+0000 7f2813fff700 1 Processor -- start 2023-10-10T20:42:09.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.629+0000 7f2813fff700 1 -- start start 2023-10-10T20:42:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2813fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f28141597f0 0x7f28140b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2813fff700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28140b11d0 con 0x7f2814159420 2023-10-10T20:42:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2813fff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28140b14b0 con 0x7f281415d230 2023-10-10T20:42:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2813fff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28140b1790 con 0x7f28141597f0 2023-10-10T20:42:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2812ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2814159420 0x7f28140b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58736/0 (socket says 172.21.15.89:58736) 2023-10-10T20:42:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2812ffd700 1 -- 172.21.15.89:0/2788795979 learned_addr learned my addr 172.21.15.89:0/2788795979 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2811ffb700 1 -- 172.21.15.89:0/2788795979 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4188768819 0 0) 0x7f28140b11d0 con 0x7f2814159420 2023-10-10T20:42:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2811ffb700 1 -- 172.21.15.89:0/2788795979 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2800003570 con 0x7f2814159420 2023-10-10T20:42:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f28127fc700 1 --2- 172.21.15.89:0/2788795979 >> v2:172.21.15.111:3300/0 conn(0x7f28141597f0 0x7f28140b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2811ffb700 1 -- 172.21.15.89:0/2788795979 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2797111055 0 0) 0x7f2800003570 con 0x7f2814159420 2023-10-10T20:42:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2811ffb700 1 -- 172.21.15.89:0/2788795979 >> v2:172.21.15.111:3300/0 conn(0x7f28141597f0 msgr2=0x7f28140b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2811ffb700 1 --2- 172.21.15.89:0/2788795979 >> v2:172.21.15.111:3300/0 conn(0x7f28141597f0 0x7f28140b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2811ffb700 1 -- 172.21.15.89:0/2788795979 >> v1:172.21.15.111:6789/0 conn(0x7f281415d230 legacy=0x7f281415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:09.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2811ffb700 1 -- 172.21.15.89:0/2788795979 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28140b1a70 con 0x7f2814159420 2023-10-10T20:42:09.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2811ffb700 1 -- 172.21.15.89:0/2788795979 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f280c0017f0 con 0x7f2814159420 2023-10-10T20:42:09.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2811ffb700 1 -- 172.21.15.89:0/2788795979 >> v1:172.21.15.89:6789/0 conn(0x7f2814159420 legacy=0x7f28140b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:09.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2811ffb700 1 --2- 172.21.15.89:0/2788795979 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2800003a60 0x7f2800003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2811ffb700 1 --2- 172.21.15.89:0/2788795979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2800004810 0x7f2800004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2811ffb700 1 --2- 172.21.15.89:0/2788795979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28000051c0 0x7f280000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2811ffb700 1 -- 172.21.15.89:0/2788795979 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28140b1790 con 0x7f28000051c0 2023-10-10T20:42:09.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2811ffb700 1 -- 172.21.15.89:0/2788795979 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280000c0d0 con 0x7f2800004810 2023-10-10T20:42:09.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.630+0000 7f2811ffb700 1 -- 172.21.15.89:0/2788795979 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280000c290 con 0x7f2800003a60 2023-10-10T20:42:09.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.631+0000 7f28137fe700 1 --2- 172.21.15.89:0/2788795979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28000051c0 0x7f280000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:09.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.631+0000 7f28127fc700 1 --2- 172.21.15.89:0/2788795979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2800004810 0x7f2800004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:09.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.631+0000 7f2812ffd700 1 --2- 172.21.15.89:0/2788795979 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2800003a60 0x7f2800003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:09.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.631+0000 7f28137fe700 1 -- 172.21.15.89:0/2788795979 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2800003a60 msgr2=0x7f2800003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:09.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.631+0000 7f28137fe700 1 --2- 172.21.15.89:0/2788795979 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2800003a60 0x7f2800003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.631+0000 7f28137fe700 1 -- 172.21.15.89:0/2788795979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2800004810 msgr2=0x7f2800004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:09.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.631+0000 7f28137fe700 1 --2- 172.21.15.89:0/2788795979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2800004810 0x7f2800004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.631+0000 7f28137fe700 1 -- 172.21.15.89:0/2788795979 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2800003570 con 0x7f28000051c0 2023-10-10T20:42:09.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.631+0000 7f28137fe700 1 -- 172.21.15.89:0/2788795979 reap_dead start 2023-10-10T20:42:09.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.631+0000 7f28137fe700 1 --2- 172.21.15.89:0/2788795979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28000051c0 0x7f280000ba60 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f2804001880 tx=0x7f2804007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f34f4597e5acccf server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:09.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.631+0000 7f2811ffb700 1 -- 172.21.15.89:0/2788795979 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804012420 con 0x7f28000051c0 2023-10-10T20:42:09.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.631+0000 7f2811ffb700 1 -- 172.21.15.89:0/2788795979 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2804012580 con 0x7f28000051c0 2023-10-10T20:42:09.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.631+0000 7f2811ffb700 1 -- 172.21.15.89:0/2788795979 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804012920 con 0x7f28000051c0 2023-10-10T20:42:09.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.632+0000 7f2813fff700 1 -- 172.21.15.89:0/2788795979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28000051c0 msgr2=0x7f280000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:09.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.632+0000 7f2813fff700 1 --2- 172.21.15.89:0/2788795979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28000051c0 0x7f280000ba60 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f2804001880 tx=0x7f2804007d30 comp rx=0 tx=0).stop 2023-10-10T20:42:09.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.632+0000 7f2813fff700 1 -- 172.21.15.89:0/2788795979 shutdown_connections 2023-10-10T20:42:09.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.632+0000 7f2813fff700 1 --2- 172.21.15.89:0/2788795979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28000051c0 0x7f280000ba60 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.632+0000 7f2813fff700 1 -- 172.21.15.89:0/2788795979 >> 172.21.15.89:0/2788795979 conn(0x7f28140d0660 msgr2=0x7f28140d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:09.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.632+0000 7f2813fff700 1 -- 172.21.15.89:0/2788795979 shutdown_connections 2023-10-10T20:42:09.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.632+0000 7f2813fff700 1 -- 172.21.15.89:0/2788795979 wait complete. 2023-10-10T20:42:09.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.633+0000 7f2813fff700 1 Processor -- start 2023-10-10T20:42:09.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.633+0000 7f2813fff700 1 -- start start 2023-10-10T20:42:09.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.633+0000 7f2813fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28000051c0 0x7f2814153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f2813fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28141538b0 0x7f2814155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f2813fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f281416a220 0x7f2814156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f2813fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281416cd80 con 0x7f28000051c0 2023-10-10T20:42:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f2813fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281416d8c0 con 0x7f281416a220 2023-10-10T20:42:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f2813fff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28141540d0 con 0x7f28141538b0 2023-10-10T20:42:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f2812ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28000051c0 0x7f2814153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f2812ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28000051c0 0x7f2814153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54600/0 (socket says 172.21.15.89:54600) 2023-10-10T20:42:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f2812ffd700 1 -- 172.21.15.89:0/4111203439 learned_addr learned my addr 172.21.15.89:0/4111203439 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f28137fe700 1 --2- 172.21.15.89:0/4111203439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f281416a220 0x7f2814156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f28127fc700 1 --2- 172.21.15.89:0/4111203439 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28141538b0 0x7f2814155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f2812ffd700 1 -- 172.21.15.89:0/4111203439 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28141538b0 msgr2=0x7f2814155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f2812ffd700 1 --2- 172.21.15.89:0/4111203439 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28141538b0 0x7f2814155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f2812ffd700 1 -- 172.21.15.89:0/4111203439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f281416a220 msgr2=0x7f2814156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f2812ffd700 1 --2- 172.21.15.89:0/4111203439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f281416a220 0x7f2814156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f2812ffd700 1 -- 172.21.15.89:0/4111203439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2814153690 con 0x7f28000051c0 2023-10-10T20:42:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f2812ffd700 1 --2- 172.21.15.89:0/4111203439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28000051c0 0x7f2814153150 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f280c001920 tx=0x7f280c0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:09.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f280affd700 1 -- 172.21.15.89:0/4111203439 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c007b60 con 0x7f28000051c0 2023-10-10T20:42:09.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f280affd700 1 -- 172.21.15.89:0/4111203439 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280c00ad70 con 0x7f28000051c0 2023-10-10T20:42:09.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f2813fff700 1 -- 172.21.15.89:0/4111203439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281416cf90 con 0x7f28000051c0 2023-10-10T20:42:09.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f280affd700 1 -- 172.21.15.89:0/4111203439 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c013640 con 0x7f28000051c0 2023-10-10T20:42:09.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.634+0000 7f2813fff700 1 -- 172.21.15.89:0/4111203439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281416d550 con 0x7f28000051c0 2023-10-10T20:42:09.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.635+0000 7f2813fff700 1 -- 172.21.15.89:0/4111203439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f281416dba0 con 0x7f28000051c0 2023-10-10T20:42:09.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.636+0000 7f280affd700 1 -- 172.21.15.89:0/4111203439 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f280c0139f0 con 0x7f28000051c0 2023-10-10T20:42:09.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.636+0000 7f280affd700 1 --2- 172.21.15.89:0/4111203439 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f27f4067320 0x7f27f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.636+0000 7f280affd700 1 -- 172.21.15.89:0/4111203439 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f280c085a70 con 0x7f28000051c0 2023-10-10T20:42:09.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.636+0000 7f28127fc700 1 --2- 172.21.15.89:0/4111203439 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f27f4067320 0x7f27f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:09.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.637+0000 7f28127fc700 1 --2- 172.21.15.89:0/4111203439 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f27f4067320 0x7f27f40697d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f28140687e0 tx=0x7f27fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:09.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.640+0000 7f280affd700 1 -- 172.21.15.89:0/4111203439 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f280c0526b0 con 0x7f28000051c0 2023-10-10T20:42:09.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.825+0000 7f2813fff700 1 -- 172.21.15.89:0/4111203439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f281404c3c0 con 0x7f28000051c0 2023-10-10T20:42:09.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.826+0000 7f280affd700 1 -- 172.21.15.89:0/4111203439 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+15442 (secure 0 0 0) 0x7f280c017090 con 0x7f28000051c0 2023-10-10T20:42:09.828 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:09.828 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":141,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:42:07.988420+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs.volume_0000000000887918.meta","create_time":"2023-10-10T20:41:48.040814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000887918"}}},{"pool":25,"pool_name":"cephfs.volume_0000000000887918.data","create_time":"2023-10-10T20:41:48.646159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000887918"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:42:09.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.828+0000 7f2813fff700 1 -- 172.21.15.89:0/4111203439 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f27f4067320 msgr2=0x7f27f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:09.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.828+0000 7f2813fff700 1 --2- 172.21.15.89:0/4111203439 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f27f4067320 0x7f27f40697d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f28140687e0 tx=0x7f27fc008040 comp rx=0 tx=0).stop 2023-10-10T20:42:09.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.828+0000 7f2813fff700 1 -- 172.21.15.89:0/4111203439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28000051c0 msgr2=0x7f2814153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:09.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.828+0000 7f2813fff700 1 --2- 172.21.15.89:0/4111203439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28000051c0 0x7f2814153150 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f280c001920 tx=0x7f280c0021d0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.828+0000 7f2813fff700 1 -- 172.21.15.89:0/4111203439 shutdown_connections 2023-10-10T20:42:09.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.828+0000 7f2813fff700 1 --2- 172.21.15.89:0/4111203439 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f27f4067320 0x7f27f40697d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.828+0000 7f2813fff700 1 --2- 172.21.15.89:0/4111203439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f281416a220 0x7f2814156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.828+0000 7f2813fff700 1 --2- 172.21.15.89:0/4111203439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28000051c0 0x7f2814153150 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.828+0000 7f2813fff700 1 --2- 172.21.15.89:0/4111203439 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28141538b0 0x7f2814155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.828+0000 7f2813fff700 1 -- 172.21.15.89:0/4111203439 >> 172.21.15.89:0/4111203439 conn(0x7f28140d0660 msgr2=0x7f28140d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.828+0000 7f2813fff700 1 -- 172.21.15.89:0/4111203439 shutdown_connections 2023-10-10T20:42:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.828+0000 7f2813fff700 1 -- 172.21.15.89:0/4111203439 wait complete. 2023-10-10T20:42:09.843 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 24, 'pool_name': 'cephfs.volume_0000000000887918.meta', 'create_time': '2023-10-10T20:41:48.040814+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000887918'}}}, {'pool': 25, 'pool_name': 'cephfs.volume_0000000000887918.data', 'create_time': '2023-10-10T20:41:48.646159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000887918'}}}] 2023-10-10T20:42:09.844 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:42:09.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.959+0000 7fbbbabf6700 1 Processor -- start 2023-10-10T20:42:09.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.959+0000 7fbbbabf6700 1 -- start start 2023-10-10T20:42:09.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.960+0000 7fbbbabf6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbb406bcd0 0x7fbbb4140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.960+0000 7fbbbabf6700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbbb406c0a0 0x7fbbb4141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.960+0000 7fbbbabf6700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb415c2a0 con 0x7fbbb40c8e00 2023-10-10T20:42:09.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.960+0000 7fbbbabf6700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb415b690 con 0x7fbbb406c0a0 2023-10-10T20:42:09.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.960+0000 7fbbbabf6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb415b970 con 0x7fbbb406bcd0 2023-10-10T20:42:09.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.961+0000 7fbbb8992700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbb406bcd0 0x7fbbb4140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:09.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.961+0000 7fbbabfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbbb406c0a0 0x7fbbb4141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:09.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.961+0000 7fbbb9193700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fbbb40c8e00 0x7fbbb40c9260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:53778/0 (socket says 172.21.15.89:53778) 2023-10-10T20:42:09.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.961+0000 7fbbb9193700 1 -- 172.21.15.89:0/2775901618 learned_addr learned my addr 172.21.15.89:0/2775901618 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:09.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.961+0000 7fbbabfff700 1 -- 172.21.15.89:0/2775901618 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbb406bcd0 msgr2=0x7fbbb4140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:09.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.961+0000 7fbbabfff700 1 --2- 172.21.15.89:0/2775901618 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbb406bcd0 0x7fbbb4140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.961+0000 7fbbabfff700 1 -- 172.21.15.89:0/2775901618 >> v1:172.21.15.111:6789/0 conn(0x7fbbb40c8e00 legacy=0x7fbbb40c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:09.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.961+0000 7fbbab7fe700 1 -- 172.21.15.89:0/2775901618 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2632108189 0 0) 0x7fbbb415c2a0 con 0x7fbbb40c8e00 2023-10-10T20:42:09.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.961+0000 7fbbabfff700 1 -- 172.21.15.89:0/2775901618 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb415bc50 con 0x7fbbb406c0a0 2023-10-10T20:42:09.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.961+0000 7fbbb8992700 1 --2- 172.21.15.89:0/2775901618 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbb406bcd0 0x7fbbb4140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:42:09.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.961+0000 7fbbabfff700 1 --2- 172.21.15.89:0/2775901618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbb406c0a0 0x7fbbb4141330 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbba40098e0 tx=0x7fbba40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65044afee59f98ed server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:09.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.961+0000 7fbbab7fe700 1 -- 172.21.15.89:0/2775901618 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba400b640 con 0x7fbbb406c0a0 2023-10-10T20:42:09.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.961+0000 7fbbab7fe700 1 -- 172.21.15.89:0/2775901618 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba400b7a0 con 0x7fbbb406c0a0 2023-10-10T20:42:09.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.961+0000 7fbbab7fe700 1 -- 172.21.15.89:0/2775901618 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba400bb40 con 0x7fbbb406c0a0 2023-10-10T20:42:09.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.962+0000 7fbbbabf6700 1 -- 172.21.15.89:0/2775901618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbb406c0a0 msgr2=0x7fbbb4141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:09.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.962+0000 7fbbbabf6700 1 --2- 172.21.15.89:0/2775901618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbb406c0a0 0x7fbbb4141330 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbba40098e0 tx=0x7fbba40091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.962+0000 7fbbbabf6700 1 -- 172.21.15.89:0/2775901618 shutdown_connections 2023-10-10T20:42:09.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.962+0000 7fbbbabf6700 1 --2- 172.21.15.89:0/2775901618 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbb406bcd0 0x7fbbb4140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.962+0000 7fbbbabf6700 1 --2- 172.21.15.89:0/2775901618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbb406c0a0 0x7fbbb4141330 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.962+0000 7fbbbabf6700 1 -- 172.21.15.89:0/2775901618 >> 172.21.15.89:0/2775901618 conn(0x7fbbb40ce600 msgr2=0x7fbbb40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:09.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.962+0000 7fbbbabf6700 1 -- 172.21.15.89:0/2775901618 shutdown_connections 2023-10-10T20:42:09.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.962+0000 7fbbbabf6700 1 -- 172.21.15.89:0/2775901618 wait complete. 2023-10-10T20:42:09.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.963+0000 7fbbbabf6700 1 Processor -- start 2023-10-10T20:42:09.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.963+0000 7fbbbabf6700 1 -- start start 2023-10-10T20:42:09.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.964+0000 7fbbbabf6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbb406bcd0 0x7fbbb41584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.964+0000 7fbbbabf6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbb406c0a0 0x7fbbb416aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.964+0000 7fbbbabf6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbb40c8e00 0x7fbbb40c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.964+0000 7fbbbabf6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb4069640 con 0x7fbbb406bcd0 2023-10-10T20:42:09.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.964+0000 7fbbbabf6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb416b9d0 con 0x7fbbb406c0a0 2023-10-10T20:42:09.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.964+0000 7fbbbabf6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb416bcb0 con 0x7fbbb40c8e00 2023-10-10T20:42:09.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.964+0000 7fbbb8992700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbb406bcd0 0x7fbbb41584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:09.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.964+0000 7fbbb8992700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbb406bcd0 0x7fbbb41584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54604/0 (socket says 172.21.15.89:54604) 2023-10-10T20:42:09.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.964+0000 7fbbb8992700 1 -- 172.21.15.89:0/2922551079 learned_addr learned my addr 172.21.15.89:0/2922551079 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:09.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.964+0000 7fbbabfff700 1 --2- 172.21.15.89:0/2922551079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbb406c0a0 0x7fbbb416aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:09.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.964+0000 7fbbb9193700 1 --2- 172.21.15.89:0/2922551079 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbb40c8e00 0x7fbbb40c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:09.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.964+0000 7fbbb8992700 1 -- 172.21.15.89:0/2922551079 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbb40c8e00 msgr2=0x7fbbb40c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:09.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.964+0000 7fbbb8992700 1 --2- 172.21.15.89:0/2922551079 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbb40c8e00 0x7fbbb40c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.964+0000 7fbbb8992700 1 -- 172.21.15.89:0/2922551079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbb406c0a0 msgr2=0x7fbbb416aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:09.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.964+0000 7fbbb8992700 1 --2- 172.21.15.89:0/2922551079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbb406c0a0 0x7fbbb416aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:09.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.964+0000 7fbbb8992700 1 -- 172.21.15.89:0/2922551079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb0009420 con 0x7fbbb406bcd0 2023-10-10T20:42:09.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.964+0000 7fbbabfff700 1 --2- 172.21.15.89:0/2922551079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbb406c0a0 0x7fbbb416aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:09.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.964+0000 7fbbb8992700 1 --2- 172.21.15.89:0/2922551079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbb406bcd0 0x7fbbb41584f0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000ec80 tx=0x7fbbb00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:09.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.965+0000 7fbba8ff9700 1 -- 172.21.15.89:0/2922551079 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb000c9d0 con 0x7fbbb406bcd0 2023-10-10T20:42:09.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.965+0000 7fbba8ff9700 1 -- 172.21.15.89:0/2922551079 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb000cb30 con 0x7fbbb406bcd0 2023-10-10T20:42:09.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.965+0000 7fbbbabf6700 1 -- 172.21.15.89:0/2922551079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba4009420 con 0x7fbbb406bcd0 2023-10-10T20:42:09.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.965+0000 7fbba8ff9700 1 -- 172.21.15.89:0/2922551079 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb000ceb0 con 0x7fbbb406bcd0 2023-10-10T20:42:09.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.965+0000 7fbbbabf6700 1 -- 172.21.15.89:0/2922551079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb4158a30 con 0x7fbbb406bcd0 2023-10-10T20:42:09.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.966+0000 7fbbbabf6700 1 -- 172.21.15.89:0/2922551079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb4069920 con 0x7fbbb406bcd0 2023-10-10T20:42:09.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.966+0000 7fbba8ff9700 1 -- 172.21.15.89:0/2922551079 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbbb0013680 con 0x7fbbb406bcd0 2023-10-10T20:42:09.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.966+0000 7fbba8ff9700 1 --2- 172.21.15.89:0/2922551079 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbba0067320 0x7fbba00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:09.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.967+0000 7fbbabfff700 1 --2- 172.21.15.89:0/2922551079 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbba0067320 0x7fbba00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:09.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.967+0000 7fbba8ff9700 1 -- 172.21.15.89:0/2922551079 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fbbb0085060 con 0x7fbbb406bcd0 2023-10-10T20:42:09.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.967+0000 7fbbabfff700 1 --2- 172.21.15.89:0/2922551079 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbba0067320 0x7fbba00697d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbba40093f0 tx=0x7fbba4017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:09.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:09.971+0000 7fbba8ff9700 1 -- 172.21.15.89:0/2922551079 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbbb0051ca0 con 0x7fbbb406bcd0 2023-10-10T20:42:10.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.155+0000 7fbbbabf6700 1 -- 172.21.15.89:0/2922551079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbbb404c3c0 con 0x7fbbb406bcd0 2023-10-10T20:42:10.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.156+0000 7fbba8ff9700 1 -- 172.21.15.89:0/2922551079 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v141) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbbb00552c0 con 0x7fbbb406bcd0 2023-10-10T20:42:10.158 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:42:10.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.158+0000 7fbbbabf6700 1 -- 172.21.15.89:0/2922551079 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbba0067320 msgr2=0x7fbba00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:10.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.158+0000 7fbbbabf6700 1 --2- 172.21.15.89:0/2922551079 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbba0067320 0x7fbba00697d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbba40093f0 tx=0x7fbba4017000 comp rx=0 tx=0).stop 2023-10-10T20:42:10.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.158+0000 7fbbbabf6700 1 -- 172.21.15.89:0/2922551079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbb406bcd0 msgr2=0x7fbbb41584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:10.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.158+0000 7fbbbabf6700 1 --2- 172.21.15.89:0/2922551079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbb406bcd0 0x7fbbb41584f0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000ec80 tx=0x7fbbb00091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.158+0000 7fbbbabf6700 1 -- 172.21.15.89:0/2922551079 shutdown_connections 2023-10-10T20:42:10.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.158+0000 7fbbbabf6700 1 --2- 172.21.15.89:0/2922551079 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbbb40c8e00 0x7fbbb40c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.158+0000 7fbbbabf6700 1 --2- 172.21.15.89:0/2922551079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbb406bcd0 0x7fbbb41584f0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.158+0000 7fbbbabf6700 1 --2- 172.21.15.89:0/2922551079 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbba0067320 0x7fbba00697d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.158+0000 7fbbbabf6700 1 --2- 172.21.15.89:0/2922551079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbb406c0a0 0x7fbbb416aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.158+0000 7fbbbabf6700 1 -- 172.21.15.89:0/2922551079 >> 172.21.15.89:0/2922551079 conn(0x7fbbb40ce600 msgr2=0x7fbbb40c9260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:10.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.158+0000 7fbbbabf6700 1 -- 172.21.15.89:0/2922551079 shutdown_connections 2023-10-10T20:42:10.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.158+0000 7fbbbabf6700 1 -- 172.21.15.89:0/2922551079 wait complete. 2023-10-10T20:42:10.172 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000887918.meta pg_num 2023-10-10T20:42:10.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.288+0000 7f3bf0a10700 1 Processor -- start 2023-10-10T20:42:10.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.288+0000 7f3bf0a10700 1 -- start start 2023-10-10T20:42:10.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.289+0000 7f3bf0a10700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bec069d60 0x7f3bec0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:10.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.289+0000 7f3bf0a10700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3bec0c81f0 0x7f3bec0cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:10.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.289+0000 7f3bf0a10700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bec06bdb0 con 0x7f3bec0aca80 2023-10-10T20:42:10.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.289+0000 7f3bf0a10700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bec0c8670 con 0x7f3bec0c81f0 2023-10-10T20:42:10.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.289+0000 7f3bf0a10700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0cb370 con 0x7f3bec069d60 2023-10-10T20:42:10.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.290+0000 7f3be1d9b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bec069d60 0x7f3bec0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:10.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.290+0000 7f3bead9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3bec0c81f0 0x7f3bec0cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:10.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.290+0000 7f3bea59c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f3bec0aca80 0x7f3bec06aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:53782/0 (socket says 172.21.15.89:53782) 2023-10-10T20:42:10.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.290+0000 7f3bea59c700 1 -- 172.21.15.89:0/2343796476 learned_addr learned my addr 172.21.15.89:0/2343796476 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:10.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.290+0000 7f3bead9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3bec0c81f0 0x7f3bec0cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:43806/0 (socket says 172.21.15.89:43806) 2023-10-10T20:42:10.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.290+0000 7f3bead9d700 1 -- 172.21.15.89:0/2343796476 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bec069d60 msgr2=0x7f3bec0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:10.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.290+0000 7f3bead9d700 1 --2- 172.21.15.89:0/2343796476 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bec069d60 0x7f3bec0c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.290+0000 7f3bead9d700 1 -- 172.21.15.89:0/2343796476 >> v1:172.21.15.111:6789/0 conn(0x7f3bec0aca80 legacy=0x7f3bec06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:10.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.290+0000 7f3bead9d700 1 -- 172.21.15.89:0/2343796476 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec159f60 con 0x7f3bec0c81f0 2023-10-10T20:42:10.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.290+0000 7f3be1d9b700 1 --2- 172.21.15.89:0/2343796476 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bec069d60 0x7f3bec0c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:42:10.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.290+0000 7f3bead9d700 1 --2- 172.21.15.89:0/2343796476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bec0c81f0 0x7f3bec0cb800 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009e60 tx=0x7f3be4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef4294f43b7aed3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:10.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.290+0000 7f3be9d9b700 1 -- 172.21.15.89:0/2343796476 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be400e070 con 0x7f3bec0c81f0 2023-10-10T20:42:10.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.290+0000 7f3be9d9b700 1 -- 172.21.15.89:0/2343796476 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be400bb60 con 0x7f3bec0c81f0 2023-10-10T20:42:10.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.290+0000 7f3be9d9b700 1 -- 172.21.15.89:0/2343796476 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be4012430 con 0x7f3bec0c81f0 2023-10-10T20:42:10.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.291+0000 7f3bf0a10700 1 -- 172.21.15.89:0/2343796476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bec0c81f0 msgr2=0x7f3bec0cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:10.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.291+0000 7f3bf0a10700 1 --2- 172.21.15.89:0/2343796476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bec0c81f0 0x7f3bec0cb800 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009e60 tx=0x7f3be4009780 comp rx=0 tx=0).stop 2023-10-10T20:42:10.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.291+0000 7f3bf0a10700 1 -- 172.21.15.89:0/2343796476 shutdown_connections 2023-10-10T20:42:10.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.291+0000 7f3bf0a10700 1 --2- 172.21.15.89:0/2343796476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bec0c81f0 0x7f3bec0cb800 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.291+0000 7f3bf0a10700 1 --2- 172.21.15.89:0/2343796476 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bec069d60 0x7f3bec0c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.291+0000 7f3bf0a10700 1 -- 172.21.15.89:0/2343796476 >> 172.21.15.89:0/2343796476 conn(0x7f3bec0cd670 msgr2=0x7f3bec0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:10.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.291+0000 7f3bf0a10700 1 -- 172.21.15.89:0/2343796476 shutdown_connections 2023-10-10T20:42:10.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.291+0000 7f3bf0a10700 1 -- 172.21.15.89:0/2343796476 wait complete. 2023-10-10T20:42:10.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.292+0000 7f3bf0a10700 1 Processor -- start 2023-10-10T20:42:10.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.292+0000 7f3bf0a10700 1 -- start start 2023-10-10T20:42:10.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.293+0000 7f3bf0a10700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bec069d60 0x7f3bec0ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:10.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.293+0000 7f3bf0a10700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bec0aca80 0x7f3bec0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:10.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.293+0000 7f3bf0a10700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bec0c81f0 0x7f3bec15c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:10.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.293+0000 7f3bf0a10700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec15b0d0 con 0x7f3bec069d60 2023-10-10T20:42:10.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.293+0000 7f3bf0a10700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec15b3b0 con 0x7f3bec0aca80 2023-10-10T20:42:10.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.293+0000 7f3bf0a10700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec15b690 con 0x7f3bec0c81f0 2023-10-10T20:42:10.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.293+0000 7f3bea59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bec069d60 0x7f3bec0ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:10.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.293+0000 7f3bea59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bec069d60 0x7f3bec0ad750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54612/0 (socket says 172.21.15.89:54612) 2023-10-10T20:42:10.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.293+0000 7f3bea59c700 1 -- 172.21.15.89:0/254103597 learned_addr learned my addr 172.21.15.89:0/254103597 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:10.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.293+0000 7f3be1d9b700 1 --2- 172.21.15.89:0/254103597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bec0aca80 0x7f3bec0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:10.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.293+0000 7f3bead9d700 1 --2- 172.21.15.89:0/254103597 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bec0c81f0 0x7f3bec15c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:10.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.293+0000 7f3bea59c700 1 -- 172.21.15.89:0/254103597 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bec0c81f0 msgr2=0x7f3bec15c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:10.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.293+0000 7f3bea59c700 1 --2- 172.21.15.89:0/254103597 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bec0c81f0 0x7f3bec15c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.293+0000 7f3bea59c700 1 -- 172.21.15.89:0/254103597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bec0aca80 msgr2=0x7f3bec0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:10.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.293+0000 7f3bea59c700 1 --2- 172.21.15.89:0/254103597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bec0aca80 0x7f3bec0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.293+0000 7f3bea59c700 1 -- 172.21.15.89:0/254103597 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd4009420 con 0x7f3bec069d60 2023-10-10T20:42:10.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.294+0000 7f3bea59c700 1 --2- 172.21.15.89:0/254103597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bec069d60 0x7f3bec0ad750 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00c790 tx=0x7f3bdc007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:10.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.294+0000 7f3be2ffd700 1 -- 172.21.15.89:0/254103597 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc005c50 con 0x7f3bec069d60 2023-10-10T20:42:10.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.294+0000 7f3be2ffd700 1 -- 172.21.15.89:0/254103597 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bdc005db0 con 0x7f3bec069d60 2023-10-10T20:42:10.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.294+0000 7f3bf0a10700 1 -- 172.21.15.89:0/254103597 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be4009a20 con 0x7f3bec069d60 2023-10-10T20:42:10.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.294+0000 7f3be2ffd700 1 -- 172.21.15.89:0/254103597 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc007320 con 0x7f3bec069d60 2023-10-10T20:42:10.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.294+0000 7f3bf0a10700 1 -- 172.21.15.89:0/254103597 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bec15bc50 con 0x7f3bec069d60 2023-10-10T20:42:10.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.295+0000 7f3bf0a10700 1 -- 172.21.15.89:0/254103597 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bec15be70 con 0x7f3bec069d60 2023-10-10T20:42:10.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.295+0000 7f3be2ffd700 1 -- 172.21.15.89:0/254103597 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3bdc01b3b0 con 0x7f3bec069d60 2023-10-10T20:42:10.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.295+0000 7f3be2ffd700 1 --2- 172.21.15.89:0/254103597 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3bcc067320 0x7f3bcc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:10.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.295+0000 7f3be2ffd700 1 -- 172.21.15.89:0/254103597 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f3bdc0842a0 con 0x7f3bec069d60 2023-10-10T20:42:10.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.296+0000 7f3be1d9b700 1 --2- 172.21.15.89:0/254103597 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3bcc067320 0x7f3bcc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:10.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.296+0000 7f3be1d9b700 1 --2- 172.21.15.89:0/254103597 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3bcc067320 0x7f3bcc0697d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3bec06a130 tx=0x7f3bd40091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:10.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.300+0000 7f3be2ffd700 1 -- 172.21.15.89:0/254103597 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3bdc050ee0 con 0x7f3bec069d60 2023-10-10T20:42:10.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.482+0000 7f3bf0a10700 1 -- 172.21.15.89:0/254103597 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000887918.meta", "var": "pg_num"} v 0) v1 -- 0x7f3bec04c3c0 con 0x7f3bec069d60 2023-10-10T20:42:10.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.482+0000 7f3be2ffd700 1 -- 172.21.15.89:0/254103597 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000887918.meta", "var": "pg_num"}]=0 v141) v1 ==== 124+0+11 (secure 0 0 0) 0x7f3bdc054500 con 0x7f3bec069d60 2023-10-10T20:42:10.485 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:42:10.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.484+0000 7f3bf0a10700 1 -- 172.21.15.89:0/254103597 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3bcc067320 msgr2=0x7f3bcc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:10.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.484+0000 7f3bf0a10700 1 --2- 172.21.15.89:0/254103597 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3bcc067320 0x7f3bcc0697d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3bec06a130 tx=0x7f3bd40091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.484+0000 7f3bf0a10700 1 -- 172.21.15.89:0/254103597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bec069d60 msgr2=0x7f3bec0ad750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:10.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.484+0000 7f3bf0a10700 1 --2- 172.21.15.89:0/254103597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bec069d60 0x7f3bec0ad750 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00c790 tx=0x7f3bdc007f60 comp rx=0 tx=0).stop 2023-10-10T20:42:10.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.484+0000 7f3bf0a10700 1 -- 172.21.15.89:0/254103597 shutdown_connections 2023-10-10T20:42:10.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.484+0000 7f3bf0a10700 1 --2- 172.21.15.89:0/254103597 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bec0c81f0 0x7f3bec15c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.484+0000 7f3bf0a10700 1 --2- 172.21.15.89:0/254103597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bec069d60 0x7f3bec0ad750 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.484+0000 7f3bf0a10700 1 --2- 172.21.15.89:0/254103597 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3bcc067320 0x7f3bcc0697d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.484+0000 7f3bf0a10700 1 --2- 172.21.15.89:0/254103597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bec0aca80 0x7f3bec0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.484+0000 7f3bf0a10700 1 -- 172.21.15.89:0/254103597 >> 172.21.15.89:0/254103597 conn(0x7f3bec0cd670 msgr2=0x7f3bec0c3510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:10.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.484+0000 7f3bf0a10700 1 -- 172.21.15.89:0/254103597 shutdown_connections 2023-10-10T20:42:10.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.484+0000 7f3bf0a10700 1 -- 172.21.15.89:0/254103597 wait complete. 2023-10-10T20:42:10.498 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000887918.data pg_num 2023-10-10T20:42:10.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.614+0000 7fbd59ef5700 1 Processor -- start 2023-10-10T20:42:10.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.614+0000 7fbd59ef5700 1 -- start start 2023-10-10T20:42:10.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd59ef5700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbd54159c60 0x7fbd5415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:10.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd59ef5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5415a160 con 0x7fbd54159890 2023-10-10T20:42:10.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd59ef5700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd540d13a0 con 0x7fbd5415db10 2023-10-10T20:42:10.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd59ef5700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5415df50 con 0x7fbd54159c60 2023-10-10T20:42:10.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd537fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbd54159890 0x7fbd5415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58742/0 (socket says 172.21.15.89:58742) 2023-10-10T20:42:10.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd537fe700 1 -- 172.21.15.89:0/491564263 learned_addr learned my addr 172.21.15.89:0/491564263 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:10.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd527fc700 1 -- 172.21.15.89:0/491564263 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573515317 0 0) 0x7fbd5415a160 con 0x7fbd54159890 2023-10-10T20:42:10.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd527fc700 1 -- 172.21.15.89:0/491564263 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd3c003570 con 0x7fbd54159890 2023-10-10T20:42:10.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd52ffd700 1 --2- 172.21.15.89:0/491564263 >> v2:172.21.15.111:3300/0 conn(0x7fbd54159c60 0x7fbd5415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:10.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd527fc700 1 -- 172.21.15.89:0/491564263 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2152210123 0 0) 0x7fbd3c003570 con 0x7fbd54159890 2023-10-10T20:42:10.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd527fc700 1 -- 172.21.15.89:0/491564263 >> v2:172.21.15.111:3300/0 conn(0x7fbd54159c60 msgr2=0x7fbd5415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:10.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd527fc700 1 --2- 172.21.15.89:0/491564263 >> v2:172.21.15.111:3300/0 conn(0x7fbd54159c60 0x7fbd5415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd527fc700 1 -- 172.21.15.89:0/491564263 >> v1:172.21.15.111:6789/0 conn(0x7fbd5415db10 legacy=0x7fbd54069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:10.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd527fc700 1 -- 172.21.15.89:0/491564263 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5415e140 con 0x7fbd54159890 2023-10-10T20:42:10.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd527fc700 1 -- 172.21.15.89:0/491564263 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fbd440017f0 con 0x7fbd54159890 2023-10-10T20:42:10.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd527fc700 1 -- 172.21.15.89:0/491564263 >> v1:172.21.15.89:6789/0 conn(0x7fbd54159890 legacy=0x7fbd5415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:10.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd527fc700 1 --2- 172.21.15.89:0/491564263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3c003a60 0x7fbd3c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:10.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd527fc700 1 --2- 172.21.15.89:0/491564263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd3c004810 0x7fbd3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:10.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd527fc700 1 --2- 172.21.15.89:0/491564263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbd3c0051c0 0x7fbd3c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:10.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd527fc700 1 -- 172.21.15.89:0/491564263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c00c0a0 con 0x7fbd3c003a60 2023-10-10T20:42:10.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd527fc700 1 -- 172.21.15.89:0/491564263 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c00c210 con 0x7fbd3c004810 2023-10-10T20:42:10.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd527fc700 1 -- 172.21.15.89:0/491564263 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c00c3d0 con 0x7fbd3c0051c0 2023-10-10T20:42:10.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.615+0000 7fbd537fe700 1 --2- 172.21.15.89:0/491564263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3c003a60 0x7fbd3c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:10.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.616+0000 7fbd52ffd700 1 --2- 172.21.15.89:0/491564263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd3c004810 0x7fbd3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:10.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.616+0000 7fbd53fff700 1 --2- 172.21.15.89:0/491564263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbd3c0051c0 0x7fbd3c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:10.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.616+0000 7fbd537fe700 1 -- 172.21.15.89:0/491564263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbd3c0051c0 msgr2=0x7fbd3c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:10.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.616+0000 7fbd537fe700 1 --2- 172.21.15.89:0/491564263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbd3c0051c0 0x7fbd3c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.616+0000 7fbd537fe700 1 -- 172.21.15.89:0/491564263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd3c004810 msgr2=0x7fbd3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:10.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.616+0000 7fbd537fe700 1 --2- 172.21.15.89:0/491564263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd3c004810 0x7fbd3c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.616+0000 7fbd537fe700 1 -- 172.21.15.89:0/491564263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3c003570 con 0x7fbd3c003a60 2023-10-10T20:42:10.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.616+0000 7fbd53fff700 1 -- 172.21.15.89:0/491564263 reap_dead start 2023-10-10T20:42:10.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.616+0000 7fbd537fe700 1 --2- 172.21.15.89:0/491564263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3c003a60 0x7fbd3c003e70 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400a930 tx=0x7fbd440021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b61fe522fac383 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:10.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.616+0000 7fbd527fc700 1 -- 172.21.15.89:0/491564263 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd440070d0 con 0x7fbd3c003a60 2023-10-10T20:42:10.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.616+0000 7fbd527fc700 1 -- 172.21.15.89:0/491564263 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd44007230 con 0x7fbd3c003a60 2023-10-10T20:42:10.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.616+0000 7fbd527fc700 1 -- 172.21.15.89:0/491564263 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd440075d0 con 0x7fbd3c003a60 2023-10-10T20:42:10.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.616+0000 7fbd59ef5700 1 -- 172.21.15.89:0/491564263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3c003a60 msgr2=0x7fbd3c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:10.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.616+0000 7fbd59ef5700 1 --2- 172.21.15.89:0/491564263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3c003a60 0x7fbd3c003e70 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400a930 tx=0x7fbd440021d0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.617+0000 7fbd59ef5700 1 -- 172.21.15.89:0/491564263 shutdown_connections 2023-10-10T20:42:10.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.617+0000 7fbd59ef5700 1 --2- 172.21.15.89:0/491564263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3c003a60 0x7fbd3c003e70 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.617+0000 7fbd59ef5700 1 -- 172.21.15.89:0/491564263 >> 172.21.15.89:0/491564263 conn(0x7fbd54068e20 msgr2=0x7fbd540d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:10.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.617+0000 7fbd59ef5700 1 -- 172.21.15.89:0/491564263 shutdown_connections 2023-10-10T20:42:10.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.617+0000 7fbd59ef5700 1 -- 172.21.15.89:0/491564263 wait complete. 2023-10-10T20:42:10.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.618+0000 7fbd59ef5700 1 Processor -- start 2023-10-10T20:42:10.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.618+0000 7fbd59ef5700 1 -- start start 2023-10-10T20:42:10.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.618+0000 7fbd59ef5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3c003a60 0x7fbd5414bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:10.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.618+0000 7fbd59ef5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbd5415d2b0 0x7fbd540b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:10.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.618+0000 7fbd59ef5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd5415f7f0 0x7fbd5415fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:10.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.618+0000 7fbd59ef5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5414c9c0 con 0x7fbd3c003a60 2023-10-10T20:42:10.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.618+0000 7fbd59ef5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd540b46d0 con 0x7fbd5415f7f0 2023-10-10T20:42:10.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.618+0000 7fbd59ef5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd54160190 con 0x7fbd5415d2b0 2023-10-10T20:42:10.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.619+0000 7fbd537fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3c003a60 0x7fbd5414bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:10.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.619+0000 7fbd537fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3c003a60 0x7fbd5414bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54624/0 (socket says 172.21.15.89:54624) 2023-10-10T20:42:10.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.619+0000 7fbd537fe700 1 -- 172.21.15.89:0/1600314577 learned_addr learned my addr 172.21.15.89:0/1600314577 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:10.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.619+0000 7fbd53fff700 1 --2- 172.21.15.89:0/1600314577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd5415f7f0 0x7fbd5415fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:10.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.619+0000 7fbd537fe700 1 -- 172.21.15.89:0/1600314577 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbd5415d2b0 msgr2=0x7fbd540b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:10.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.619+0000 7fbd537fe700 1 --2- 172.21.15.89:0/1600314577 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbd5415d2b0 0x7fbd540b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.619+0000 7fbd537fe700 1 -- 172.21.15.89:0/1600314577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd5415f7f0 msgr2=0x7fbd5415fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:10.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.619+0000 7fbd537fe700 1 --2- 172.21.15.89:0/1600314577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd5415f7f0 0x7fbd5415fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.619+0000 7fbd537fe700 1 -- 172.21.15.89:0/1600314577 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5415c750 con 0x7fbd3c003a60 2023-10-10T20:42:10.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.619+0000 7fbd537fe700 1 --2- 172.21.15.89:0/1600314577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3c003a60 0x7fbd5414bf10 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fbd44002250 tx=0x7fbd44016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:10.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.619+0000 7fbd3b7fe700 1 -- 172.21.15.89:0/1600314577 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4401d430 con 0x7fbd3c003a60 2023-10-10T20:42:10.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.619+0000 7fbd3b7fe700 1 -- 172.21.15.89:0/1600314577 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4401d5b0 con 0x7fbd3c003a60 2023-10-10T20:42:10.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.619+0000 7fbd3b7fe700 1 -- 172.21.15.89:0/1600314577 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4401d970 con 0x7fbd3c003a60 2023-10-10T20:42:10.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.619+0000 7fbd59ef5700 1 -- 172.21.15.89:0/1600314577 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd540b4f50 con 0x7fbd3c003a60 2023-10-10T20:42:10.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.619+0000 7fbd59ef5700 1 -- 172.21.15.89:0/1600314577 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd540b4900 con 0x7fbd3c003a60 2023-10-10T20:42:10.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.620+0000 7fbd59ef5700 1 -- 172.21.15.89:0/1600314577 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5404c3c0 con 0x7fbd3c003a60 2023-10-10T20:42:10.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.620+0000 7fbd3b7fe700 1 -- 172.21.15.89:0/1600314577 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbd4401e430 con 0x7fbd3c003a60 2023-10-10T20:42:10.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.621+0000 7fbd3b7fe700 1 --2- 172.21.15.89:0/1600314577 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbd40067320 0x7fbd400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:10.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.621+0000 7fbd3b7fe700 1 -- 172.21.15.89:0/1600314577 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fbd44087130 con 0x7fbd3c003a60 2023-10-10T20:42:10.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.621+0000 7fbd52ffd700 1 --2- 172.21.15.89:0/1600314577 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbd40067320 0x7fbd400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:10.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.621+0000 7fbd52ffd700 1 --2- 172.21.15.89:0/1600314577 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbd40067320 0x7fbd400697d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbd48006f00 tx=0x7fbd480091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:10.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.625+0000 7fbd3b7fe700 1 -- 172.21.15.89:0/1600314577 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd44053e80 con 0x7fbd3c003a60 2023-10-10T20:42:10.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.804+0000 7fbd59ef5700 1 -- 172.21.15.89:0/1600314577 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000887918.data", "var": "pg_num"} v 0) v1 -- 0x7fbd5406a5b0 con 0x7fbd3c003a60 2023-10-10T20:42:10.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.805+0000 7fbd3b7fe700 1 -- 172.21.15.89:0/1600314577 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000887918.data", "var": "pg_num"}]=0 v141) v1 ==== 124+0+11 (secure 0 0 0) 0x7fbd440574a0 con 0x7fbd3c003a60 2023-10-10T20:42:10.807 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-10T20:42:10.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.807+0000 7fbd59ef5700 1 -- 172.21.15.89:0/1600314577 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbd40067320 msgr2=0x7fbd400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:10.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.807+0000 7fbd59ef5700 1 --2- 172.21.15.89:0/1600314577 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbd40067320 0x7fbd400697d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbd48006f00 tx=0x7fbd480091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.807+0000 7fbd59ef5700 1 -- 172.21.15.89:0/1600314577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3c003a60 msgr2=0x7fbd5414bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:10.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.807+0000 7fbd59ef5700 1 --2- 172.21.15.89:0/1600314577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3c003a60 0x7fbd5414bf10 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fbd44002250 tx=0x7fbd44016000 comp rx=0 tx=0).stop 2023-10-10T20:42:10.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.807+0000 7fbd59ef5700 1 -- 172.21.15.89:0/1600314577 shutdown_connections 2023-10-10T20:42:10.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.807+0000 7fbd59ef5700 1 --2- 172.21.15.89:0/1600314577 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbd40067320 0x7fbd400697d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.807+0000 7fbd59ef5700 1 --2- 172.21.15.89:0/1600314577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd5415f7f0 0x7fbd5415fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.807+0000 7fbd59ef5700 1 --2- 172.21.15.89:0/1600314577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3c003a60 0x7fbd5414bf10 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.807+0000 7fbd59ef5700 1 --2- 172.21.15.89:0/1600314577 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbd5415d2b0 0x7fbd540b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.807+0000 7fbd59ef5700 1 -- 172.21.15.89:0/1600314577 >> 172.21.15.89:0/1600314577 conn(0x7fbd54068e20 msgr2=0x7fbd540c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:10.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.807+0000 7fbd59ef5700 1 -- 172.21.15.89:0/1600314577 shutdown_connections 2023-10-10T20:42:10.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.807+0000 7fbd59ef5700 1 -- 172.21.15.89:0/1600314577 wait complete. 2023-10-10T20:42:10.821 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:42:10.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.934+0000 7f725eaa7700 1 Processor -- start 2023-10-10T20:42:10.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.934+0000 7f725eaa7700 1 -- start start 2023-10-10T20:42:10.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.935+0000 7f725eaa7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7258069560 0x7f72580699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:10.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.935+0000 7f725eaa7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7258069ee0 con 0x7f7258159c60 2023-10-10T20:42:10.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.935+0000 7f725eaa7700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725815a160 con 0x7f7258159890 2023-10-10T20:42:10.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.935+0000 7f725eaa7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72580d13a0 con 0x7f7258069560 2023-10-10T20:42:10.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.935+0000 7f725d044700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7258069560 0x7f72580699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:10.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.935+0000 7f7257fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7258159c60 0x7f725815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58746/0 (socket says 172.21.15.89:58746) 2023-10-10T20:42:10.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.935+0000 7f7257fff700 1 -- 172.21.15.89:0/755621607 learned_addr learned my addr 172.21.15.89:0/755621607 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:10.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.935+0000 7f725d044700 1 -- 172.21.15.89:0/755621607 >> v1:172.21.15.111:6789/0 conn(0x7f7258159890 legacy=0x7f725815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:10.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.935+0000 7f725d044700 1 -- 172.21.15.89:0/755621607 >> v1:172.21.15.89:6789/0 conn(0x7f7258159c60 legacy=0x7f725815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:10.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.935+0000 7f725d044700 1 -- 172.21.15.89:0/755621607 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725815e140 con 0x7f7258069560 2023-10-10T20:42:10.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.935+0000 7f725d044700 1 --2- 172.21.15.89:0/755621607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258069560 0x7f72580699a0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f7250009e60 tx=0x7f7250009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0cd2c501927897 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:10.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.935+0000 7f72577fe700 1 -- 172.21.15.89:0/755621607 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725000e070 con 0x7f7258069560 2023-10-10T20:42:10.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.935+0000 7f72577fe700 1 -- 172.21.15.89:0/755621607 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725000bbf0 con 0x7f7258069560 2023-10-10T20:42:10.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.935+0000 7f72577fe700 1 -- 172.21.15.89:0/755621607 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7250012430 con 0x7f7258069560 2023-10-10T20:42:10.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.936+0000 7f725eaa7700 1 -- 172.21.15.89:0/755621607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258069560 msgr2=0x7f72580699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:10.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.936+0000 7f725eaa7700 1 --2- 172.21.15.89:0/755621607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258069560 0x7f72580699a0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f7250009e60 tx=0x7f7250009780 comp rx=0 tx=0).stop 2023-10-10T20:42:10.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.936+0000 7f725eaa7700 1 -- 172.21.15.89:0/755621607 shutdown_connections 2023-10-10T20:42:10.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.936+0000 7f725eaa7700 1 --2- 172.21.15.89:0/755621607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258069560 0x7f72580699a0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.936+0000 7f725eaa7700 1 -- 172.21.15.89:0/755621607 >> 172.21.15.89:0/755621607 conn(0x7f7258068e20 msgr2=0x7f72580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:10.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.936+0000 7f725eaa7700 1 -- 172.21.15.89:0/755621607 shutdown_connections 2023-10-10T20:42:10.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.936+0000 7f725eaa7700 1 -- 172.21.15.89:0/755621607 wait complete. 2023-10-10T20:42:10.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.937+0000 7f725eaa7700 1 Processor -- start 2023-10-10T20:42:10.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.937+0000 7f725eaa7700 1 -- start start 2023-10-10T20:42:10.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.938+0000 7f725eaa7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258069560 0x7f725816dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:10.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.938+0000 7f725eaa7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7258159890 0x7f7258151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:10.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.938+0000 7f725eaa7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7258159c60 0x7f725814c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:10.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.938+0000 7f725eaa7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725816e270 con 0x7f7258069560 2023-10-10T20:42:10.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.938+0000 7f725eaa7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72580cd2d0 con 0x7f7258159890 2023-10-10T20:42:10.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.938+0000 7f725eaa7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7258151870 con 0x7f7258159c60 2023-10-10T20:42:10.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.938+0000 7f725c843700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258069560 0x7f725816dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:10.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.938+0000 7f725c843700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258069560 0x7f725816dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54632/0 (socket says 172.21.15.89:54632) 2023-10-10T20:42:10.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.938+0000 7f725c843700 1 -- 172.21.15.89:0/2359709302 learned_addr learned my addr 172.21.15.89:0/2359709302 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:10.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.938+0000 7f7257fff700 1 --2- 172.21.15.89:0/2359709302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7258159890 0x7f7258151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:10.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.938+0000 7f725d044700 1 --2- 172.21.15.89:0/2359709302 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7258159c60 0x7f725814c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:10.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.938+0000 7f725c843700 1 -- 172.21.15.89:0/2359709302 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7258159c60 msgr2=0x7f725814c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:10.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.938+0000 7f725c843700 1 --2- 172.21.15.89:0/2359709302 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7258159c60 0x7f725814c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.938+0000 7f725c843700 1 -- 172.21.15.89:0/2359709302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7258159890 msgr2=0x7f7258151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:10.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.938+0000 7f725c843700 1 --2- 172.21.15.89:0/2359709302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7258159890 0x7f7258151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:10.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.938+0000 7f725c843700 1 -- 172.21.15.89:0/2359709302 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7250009a20 con 0x7f7258069560 2023-10-10T20:42:10.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.939+0000 7f725c843700 1 --2- 172.21.15.89:0/2359709302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258069560 0x7f725816dd30 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f724c00c600 tx=0x7f724c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:10.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.939+0000 7f7254ff9700 1 -- 172.21.15.89:0/2359709302 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c011590 con 0x7f7258069560 2023-10-10T20:42:10.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.939+0000 7f7254ff9700 1 -- 172.21.15.89:0/2359709302 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c011710 con 0x7f7258069560 2023-10-10T20:42:10.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.939+0000 7f7254ff9700 1 -- 172.21.15.89:0/2359709302 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c011ad0 con 0x7f7258069560 2023-10-10T20:42:10.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.939+0000 7f725eaa7700 1 -- 172.21.15.89:0/2359709302 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725816a270 con 0x7f7258069560 2023-10-10T20:42:10.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.939+0000 7f725eaa7700 1 -- 172.21.15.89:0/2359709302 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725816a830 con 0x7f7258069560 2023-10-10T20:42:10.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.940+0000 7f725eaa7700 1 -- 172.21.15.89:0/2359709302 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f725804c3c0 con 0x7f7258069560 2023-10-10T20:42:10.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.940+0000 7f7254ff9700 1 -- 172.21.15.89:0/2359709302 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f724c01c460 con 0x7f7258069560 2023-10-10T20:42:10.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.940+0000 7f7254ff9700 1 --2- 172.21.15.89:0/2359709302 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f723c067320 0x7f723c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:10.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.940+0000 7f7254ff9700 1 -- 172.21.15.89:0/2359709302 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f724c084aa0 con 0x7f7258069560 2023-10-10T20:42:10.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.941+0000 7f7257fff700 1 --2- 172.21.15.89:0/2359709302 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f723c067320 0x7f723c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:10.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.941+0000 7f7257fff700 1 --2- 172.21.15.89:0/2359709302 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f723c067320 0x7f723c0697d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7244001780 tx=0x7f7244009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:10.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:10.945+0000 7f7254ff9700 1 -- 172.21.15.89:0/2359709302 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f724c0516e0 con 0x7f7258069560 2023-10-10T20:42:11.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.171+0000 7f725eaa7700 1 -- 172.21.15.89:0/2359709302 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f725806a5e0 con 0x7f7258069560 2023-10-10T20:42:11.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.171+0000 7f7254ff9700 1 -- 172.21.15.89:0/2359709302 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4618 (secure 0 0 0) 0x7f724c054d00 con 0x7f7258069560 2023-10-10T20:42:11.174 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:11.174 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":93,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6197,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/4123310016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4123310016},{"type":"v1","addr":"172.21.15.111:6834","nonce":4123310016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":6200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/141017824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":141017824},{"type":"v1","addr":"172.21.15.111:6835","nonce":141017824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7575,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/728309352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":728309352},{"type":"v1","addr":"172.21.15.89:6836","nonce":728309352}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:41:49.687845+0000","modified":"2023-10-10T20:42:05.985626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7578":{"gid":7578,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/2102263806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2102263806},{"type":"v1","addr":"172.21.15.89:6837","nonce":2102263806}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"volume_0000000000887918","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-10T20:42:11.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.173+0000 7f725eaa7700 1 -- 172.21.15.89:0/2359709302 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f723c067320 msgr2=0x7f723c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:11.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.173+0000 7f725eaa7700 1 --2- 172.21.15.89:0/2359709302 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f723c067320 0x7f723c0697d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7244001780 tx=0x7f7244009040 comp rx=0 tx=0).stop 2023-10-10T20:42:11.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.173+0000 7f725eaa7700 1 -- 172.21.15.89:0/2359709302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258069560 msgr2=0x7f725816dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:11.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.173+0000 7f725eaa7700 1 --2- 172.21.15.89:0/2359709302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258069560 0x7f725816dd30 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f724c00c600 tx=0x7f724c007f60 comp rx=0 tx=0).stop 2023-10-10T20:42:11.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.173+0000 7f725eaa7700 1 -- 172.21.15.89:0/2359709302 shutdown_connections 2023-10-10T20:42:11.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.173+0000 7f725eaa7700 1 --2- 172.21.15.89:0/2359709302 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7258159c60 0x7f725814c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.173+0000 7f725eaa7700 1 --2- 172.21.15.89:0/2359709302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7258069560 0x7f725816dd30 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.173+0000 7f725eaa7700 1 --2- 172.21.15.89:0/2359709302 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f723c067320 0x7f723c0697d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.173+0000 7f725eaa7700 1 --2- 172.21.15.89:0/2359709302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7258159890 0x7f7258151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.173+0000 7f725eaa7700 1 -- 172.21.15.89:0/2359709302 >> 172.21.15.89:0/2359709302 conn(0x7f7258068e20 msgr2=0x7f725814d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:11.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.173+0000 7f725eaa7700 1 -- 172.21.15.89:0/2359709302 shutdown_connections 2023-10-10T20:42:11.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.173+0000 7f725eaa7700 1 -- 172.21.15.89:0/2359709302 wait complete. 2023-10-10T20:42:11.180 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 93 2023-10-10T20:42:11.187 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:42:11.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.301+0000 7fc1cbf6c700 1 Processor -- start 2023-10-10T20:42:11.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.301+0000 7fc1cbf6c700 1 -- start start 2023-10-10T20:42:11.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.302+0000 7fc1cbf6c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc1c406bcd0 0x7fc1c40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:11.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.302+0000 7fc1cbf6c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc1c406c0a0 0x7fc1c415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:11.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.302+0000 7fc1cbf6c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc1c415df50 0x7fc1c415e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:11.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.302+0000 7fc1cbf6c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c415d7e0 con 0x7fc1c406bcd0 2023-10-10T20:42:11.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.302+0000 7fc1cbf6c700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c415dac0 con 0x7fc1c406c0a0 2023-10-10T20:42:11.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.302+0000 7fc1cbf6c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c415aa30 con 0x7fc1c415df50 2023-10-10T20:42:11.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.302+0000 7fc1c9d08700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc1c406bcd0 0x7fc1c40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:11.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.302+0000 7fc1c9d08700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc1c406bcd0 0x7fc1c40caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54642/0 (socket says 172.21.15.89:54642) 2023-10-10T20:42:11.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.302+0000 7fc1c9d08700 1 -- 172.21.15.89:0/1051505240 learned_addr learned my addr 172.21.15.89:0/1051505240 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:11.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.302+0000 7fc1c9507700 1 --2- 172.21.15.89:0/1051505240 >> v2:172.21.15.111:3300/0 conn(0x7fc1c406c0a0 0x7fc1c415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:11.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.302+0000 7fc1ca509700 1 --2- 172.21.15.89:0/1051505240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc1c415df50 0x7fc1c415e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:11.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.302+0000 7fc1c9d08700 1 -- 172.21.15.89:0/1051505240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc1c415df50 msgr2=0x7fc1c415e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:11.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.302+0000 7fc1c9d08700 1 --2- 172.21.15.89:0/1051505240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc1c415df50 0x7fc1c415e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.302+0000 7fc1c9d08700 1 -- 172.21.15.89:0/1051505240 >> v2:172.21.15.111:3300/0 conn(0x7fc1c406c0a0 msgr2=0x7fc1c415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:11.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.302+0000 7fc1c9d08700 1 --2- 172.21.15.89:0/1051505240 >> v2:172.21.15.111:3300/0 conn(0x7fc1c406c0a0 0x7fc1c415a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.302+0000 7fc1c9d08700 1 -- 172.21.15.89:0/1051505240 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c415ad10 con 0x7fc1c406bcd0 2023-10-10T20:42:11.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.302+0000 7fc1c9d08700 1 --2- 172.21.15.89:0/1051505240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1c406bcd0 0x7fc1c40caa90 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fc1c406b4e0 tx=0x7fc1c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ccff7659fb8cfd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:11.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.302+0000 7fc1c8d06700 1 -- 172.21.15.89:0/1051505240 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c000b860 con 0x7fc1c406bcd0 2023-10-10T20:42:11.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.303+0000 7fc1c8d06700 1 -- 172.21.15.89:0/1051505240 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c000b9c0 con 0x7fc1c406bcd0 2023-10-10T20:42:11.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.303+0000 7fc1c8d06700 1 -- 172.21.15.89:0/1051505240 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c000bd80 con 0x7fc1c406bcd0 2023-10-10T20:42:11.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.303+0000 7fc1cbf6c700 1 -- 172.21.15.89:0/1051505240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1c406bcd0 msgr2=0x7fc1c40caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:11.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.303+0000 7fc1cbf6c700 1 --2- 172.21.15.89:0/1051505240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1c406bcd0 0x7fc1c40caa90 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fc1c406b4e0 tx=0x7fc1c0009250 comp rx=0 tx=0).stop 2023-10-10T20:42:11.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.303+0000 7fc1cbf6c700 1 -- 172.21.15.89:0/1051505240 shutdown_connections 2023-10-10T20:42:11.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.303+0000 7fc1cbf6c700 1 --2- 172.21.15.89:0/1051505240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc1c415df50 0x7fc1c415e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.303+0000 7fc1cbf6c700 1 --2- 172.21.15.89:0/1051505240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1c406bcd0 0x7fc1c40caa90 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.303+0000 7fc1cbf6c700 1 --2- 172.21.15.89:0/1051505240 >> v2:172.21.15.111:3300/0 conn(0x7fc1c406c0a0 0x7fc1c415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.303+0000 7fc1cbf6c700 1 -- 172.21.15.89:0/1051505240 >> 172.21.15.89:0/1051505240 conn(0x7fc1c40ce600 msgr2=0x7fc1c40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:11.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.303+0000 7fc1cbf6c700 1 -- 172.21.15.89:0/1051505240 shutdown_connections 2023-10-10T20:42:11.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.303+0000 7fc1cbf6c700 1 -- 172.21.15.89:0/1051505240 wait complete. 2023-10-10T20:42:11.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.304+0000 7fc1cbf6c700 1 Processor -- start 2023-10-10T20:42:11.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.304+0000 7fc1cbf6c700 1 -- start start 2023-10-10T20:42:11.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.305+0000 7fc1cbf6c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1c406bcd0 0x7fc1c415b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:11.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.305+0000 7fc1cbf6c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc1c406c0a0 0x7fc1c40c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:11.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.305+0000 7fc1cbf6c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1c415df50 0x7fc1c4146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:11.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.305+0000 7fc1cbf6c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40c8070 con 0x7fc1c415df50 2023-10-10T20:42:11.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.305+0000 7fc1cbf6c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40c8bd0 con 0x7fc1c406bcd0 2023-10-10T20:42:11.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.305+0000 7fc1cbf6c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c416e380 con 0x7fc1c406c0a0 2023-10-10T20:42:11.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.305+0000 7fc1ca509700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1c415df50 0x7fc1c4146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:11.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.305+0000 7fc1ca509700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1c415df50 0x7fc1c4146eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54648/0 (socket says 172.21.15.89:54648) 2023-10-10T20:42:11.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.305+0000 7fc1ca509700 1 -- 172.21.15.89:0/2964555343 learned_addr learned my addr 172.21.15.89:0/2964555343 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:11.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.305+0000 7fc1c9d08700 1 --2- 172.21.15.89:0/2964555343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1c406bcd0 0x7fc1c415b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:11.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.305+0000 7fc1ca509700 1 -- 172.21.15.89:0/2964555343 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc1c406c0a0 msgr2=0x7fc1c40c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:11.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.305+0000 7fc1ca509700 1 --2- 172.21.15.89:0/2964555343 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc1c406c0a0 0x7fc1c40c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.305+0000 7fc1ca509700 1 -- 172.21.15.89:0/2964555343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1c406bcd0 msgr2=0x7fc1c415b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:11.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.305+0000 7fc1ca509700 1 --2- 172.21.15.89:0/2964555343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1c406bcd0 0x7fc1c415b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.305+0000 7fc1ca509700 1 -- 172.21.15.89:0/2964555343 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c00094f0 con 0x7fc1c415df50 2023-10-10T20:42:11.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.305+0000 7fc1ca509700 1 --2- 172.21.15.89:0/2964555343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1c415df50 0x7fc1c4146eb0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc007f50 tx=0x7fc1bc009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:11.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.306+0000 7fc1b9ffb700 1 -- 172.21.15.89:0/2964555343 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc00ecd0 con 0x7fc1c415df50 2023-10-10T20:42:11.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.306+0000 7fc1b9ffb700 1 -- 172.21.15.89:0/2964555343 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc00ee50 con 0x7fc1c415df50 2023-10-10T20:42:11.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.306+0000 7fc1cbf6c700 1 -- 172.21.15.89:0/2964555343 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c40c82a0 con 0x7fc1c415df50 2023-10-10T20:42:11.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.306+0000 7fc1b9ffb700 1 -- 172.21.15.89:0/2964555343 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc014670 con 0x7fc1c415df50 2023-10-10T20:42:11.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.306+0000 7fc1cbf6c700 1 -- 172.21.15.89:0/2964555343 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c40c8860 con 0x7fc1c415df50 2023-10-10T20:42:11.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.307+0000 7fc1cbf6c700 1 -- 172.21.15.89:0/2964555343 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c40c8d80 con 0x7fc1c415df50 2023-10-10T20:42:11.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.307+0000 7fc1b9ffb700 1 -- 172.21.15.89:0/2964555343 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc1bc0149e0 con 0x7fc1c415df50 2023-10-10T20:42:11.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.307+0000 7fc1b9ffb700 1 --2- 172.21.15.89:0/2964555343 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc1a8067320 0x7fc1a80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:11.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.308+0000 7fc1b9ffb700 1 -- 172.21.15.89:0/2964555343 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fc1bc0853b0 con 0x7fc1c415df50 2023-10-10T20:42:11.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.308+0000 7fc1c9d08700 1 --2- 172.21.15.89:0/2964555343 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc1a8067320 0x7fc1a80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:11.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.308+0000 7fc1c9d08700 1 --2- 172.21.15.89:0/2964555343 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc1a8067320 0x7fc1a80697d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc1c406b4e0 tx=0x7fc1c0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:11.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.312+0000 7fc1b9ffb700 1 -- 172.21.15.89:0/2964555343 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1c40c8d80 con 0x7fc1c415df50 2023-10-10T20:42:11.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.497+0000 7fc1cbf6c700 1 -- 172.21.15.89:0/2964555343 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1c40c8d80 con 0x7fc1c415df50 2023-10-10T20:42:11.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.498+0000 7fc1b9ffb700 1 -- 172.21.15.89:0/2964555343 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+15442 (secure 0 0 0) 0x7fc1c40c8d80 con 0x7fc1c415df50 2023-10-10T20:42:11.500 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:11.501 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":141,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:42:07.988420+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs.volume_0000000000887918.meta","create_time":"2023-10-10T20:41:48.040814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000887918"}}},{"pool":25,"pool_name":"cephfs.volume_0000000000887918.data","create_time":"2023-10-10T20:41:48.646159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000887918"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:42:11.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.500+0000 7fc1cbf6c700 1 -- 172.21.15.89:0/2964555343 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc1a8067320 msgr2=0x7fc1a80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:11.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.500+0000 7fc1cbf6c700 1 --2- 172.21.15.89:0/2964555343 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc1a8067320 0x7fc1a80697d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc1c406b4e0 tx=0x7fc1c0017000 comp rx=0 tx=0).stop 2023-10-10T20:42:11.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.500+0000 7fc1cbf6c700 1 -- 172.21.15.89:0/2964555343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1c415df50 msgr2=0x7fc1c4146eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:11.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.500+0000 7fc1cbf6c700 1 --2- 172.21.15.89:0/2964555343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1c415df50 0x7fc1c4146eb0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc007f50 tx=0x7fc1bc009860 comp rx=0 tx=0).stop 2023-10-10T20:42:11.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.500+0000 7fc1cbf6c700 1 -- 172.21.15.89:0/2964555343 shutdown_connections 2023-10-10T20:42:11.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.500+0000 7fc1cbf6c700 1 --2- 172.21.15.89:0/2964555343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1c415df50 0x7fc1c4146eb0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.500+0000 7fc1cbf6c700 1 --2- 172.21.15.89:0/2964555343 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc1a8067320 0x7fc1a80697d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.500+0000 7fc1cbf6c700 1 --2- 172.21.15.89:0/2964555343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1c406bcd0 0x7fc1c415b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.500+0000 7fc1cbf6c700 1 --2- 172.21.15.89:0/2964555343 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc1c406c0a0 0x7fc1c40c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.500+0000 7fc1cbf6c700 1 -- 172.21.15.89:0/2964555343 >> 172.21.15.89:0/2964555343 conn(0x7fc1c40ce600 msgr2=0x7fc1c4159e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:11.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.500+0000 7fc1cbf6c700 1 -- 172.21.15.89:0/2964555343 shutdown_connections 2023-10-10T20:42:11.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.500+0000 7fc1cbf6c700 1 -- 172.21.15.89:0/2964555343 wait complete. 2023-10-10T20:42:11.514 INFO:tasks.cephfs.filesystem:Destroying file system volume_0000000000887918 and related pools 2023-10-10T20:42:11.515 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:42:11.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.626+0000 7f108ae55700 1 Processor -- start 2023-10-10T20:42:11.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.626+0000 7f108ae55700 1 -- start start 2023-10-10T20:42:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.627+0000 7f108ae55700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f10841597f0 0x7f10840b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.627+0000 7f108ae55700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f108415d230 0x7f108415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.627+0000 7f108ae55700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108415db70 con 0x7f1084159420 2023-10-10T20:42:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.627+0000 7f108ae55700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10840b11d0 con 0x7f108415d230 2023-10-10T20:42:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.627+0000 7f108ae55700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10840b14b0 con 0x7f10841597f0 2023-10-10T20:42:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.627+0000 7f108a654700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f108415d230 0x7f108415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.627+0000 7f108a654700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f108415d230 0x7f108415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54650/0 (socket says 172.21.15.89:54650) 2023-10-10T20:42:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.627+0000 7f108a654700 1 -- 172.21.15.89:0/4097720439 learned_addr learned my addr 172.21.15.89:0/4097720439 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.627+0000 7f1089652700 1 --2- 172.21.15.89:0/4097720439 >> v2:172.21.15.111:3300/0 conn(0x7f10841597f0 0x7f10840b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:11.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.627+0000 7f108a654700 1 -- 172.21.15.89:0/4097720439 >> v2:172.21.15.111:3300/0 conn(0x7f10841597f0 msgr2=0x7f10840b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:11.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.627+0000 7f108a654700 1 --2- 172.21.15.89:0/4097720439 >> v2:172.21.15.111:3300/0 conn(0x7f10841597f0 0x7f10840b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.627+0000 7f108a654700 1 -- 172.21.15.89:0/4097720439 >> v1:172.21.15.111:6789/0 conn(0x7f1084159420 legacy=0x7f10840b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:11.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.627+0000 7f108a654700 1 -- 172.21.15.89:0/4097720439 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10840b1790 con 0x7f108415d230 2023-10-10T20:42:11.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.628+0000 7f108a654700 1 --2- 172.21.15.89:0/4097720439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f108415d230 0x7f108415d630 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f1084089750 tx=0x7f107c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e66cbc4040ed90 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:11.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.628+0000 7f1088e51700 1 -- 172.21.15.89:0/4097720439 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c00b860 con 0x7f108415d230 2023-10-10T20:42:11.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.628+0000 7f1088e51700 1 -- 172.21.15.89:0/4097720439 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c00b9c0 con 0x7f108415d230 2023-10-10T20:42:11.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.628+0000 7f1088e51700 1 -- 172.21.15.89:0/4097720439 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c00bd60 con 0x7f108415d230 2023-10-10T20:42:11.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.628+0000 7f108ae55700 1 -- 172.21.15.89:0/4097720439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f108415d230 msgr2=0x7f108415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:11.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.628+0000 7f108ae55700 1 --2- 172.21.15.89:0/4097720439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f108415d230 0x7f108415d630 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f1084089750 tx=0x7f107c009250 comp rx=0 tx=0).stop 2023-10-10T20:42:11.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.628+0000 7f108ae55700 1 -- 172.21.15.89:0/4097720439 shutdown_connections 2023-10-10T20:42:11.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.628+0000 7f108ae55700 1 --2- 172.21.15.89:0/4097720439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f108415d230 0x7f108415d630 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.628+0000 7f108ae55700 1 --2- 172.21.15.89:0/4097720439 >> v2:172.21.15.111:3300/0 conn(0x7f10841597f0 0x7f10840b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.628+0000 7f108ae55700 1 -- 172.21.15.89:0/4097720439 >> 172.21.15.89:0/4097720439 conn(0x7f10840d0660 msgr2=0x7f10840d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:11.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.628+0000 7f108ae55700 1 -- 172.21.15.89:0/4097720439 shutdown_connections 2023-10-10T20:42:11.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.628+0000 7f108ae55700 1 -- 172.21.15.89:0/4097720439 wait complete. 2023-10-10T20:42:11.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.629+0000 7f108ae55700 1 Processor -- start 2023-10-10T20:42:11.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.630+0000 7f108ae55700 1 -- start start 2023-10-10T20:42:11.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.630+0000 7f108ae55700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1084159420 0x7f108415c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:11.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.630+0000 7f108ae55700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10841597f0 0x7f108415a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:11.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.630+0000 7f108ae55700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f108415d230 0x7f1084153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:11.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.630+0000 7f108ae55700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108415b650 con 0x7f10841597f0 2023-10-10T20:42:11.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.630+0000 7f108ae55700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108415b090 con 0x7f1084159420 2023-10-10T20:42:11.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.630+0000 7f108ae55700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108415b370 con 0x7f108415d230 2023-10-10T20:42:11.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.630+0000 7f1089652700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10841597f0 0x7f108415a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:11.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.630+0000 7f1089652700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10841597f0 0x7f108415a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54654/0 (socket says 172.21.15.89:54654) 2023-10-10T20:42:11.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.630+0000 7f1089652700 1 -- 172.21.15.89:0/2666505039 learned_addr learned my addr 172.21.15.89:0/2666505039 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:11.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.631+0000 7f1089652700 1 -- 172.21.15.89:0/2666505039 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f108415d230 msgr2=0x7f1084153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:11.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.631+0000 7f1089e53700 1 --2- 172.21.15.89:0/2666505039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1084159420 0x7f108415c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:11.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.631+0000 7f1089652700 1 --2- 172.21.15.89:0/2666505039 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f108415d230 0x7f1084153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.631+0000 7f1089652700 1 -- 172.21.15.89:0/2666505039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1084159420 msgr2=0x7f108415c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:11.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.631+0000 7f1089652700 1 --2- 172.21.15.89:0/2666505039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1084159420 0x7f108415c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.631+0000 7f1089652700 1 -- 172.21.15.89:0/2666505039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c0094f0 con 0x7f10841597f0 2023-10-10T20:42:11.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.631+0000 7f1089652700 1 --2- 172.21.15.89:0/2666505039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10841597f0 0x7f108415a660 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f107800d860 tx=0x7f1078008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:11.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.631+0000 7f1081ffb700 1 -- 172.21.15.89:0/2666505039 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10780125f0 con 0x7f10841597f0 2023-10-10T20:42:11.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.631+0000 7f1081ffb700 1 -- 172.21.15.89:0/2666505039 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1078012750 con 0x7f10841597f0 2023-10-10T20:42:11.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.631+0000 7f108ae55700 1 -- 172.21.15.89:0/2666505039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10840c55a0 con 0x7f10841597f0 2023-10-10T20:42:11.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.631+0000 7f1081ffb700 1 -- 172.21.15.89:0/2666505039 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1078012b10 con 0x7f10841597f0 2023-10-10T20:42:11.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.631+0000 7f108ae55700 1 -- 172.21.15.89:0/2666505039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10840c7430 con 0x7f10841597f0 2023-10-10T20:42:11.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.632+0000 7f108ae55700 1 -- 172.21.15.89:0/2666505039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10840c5400 con 0x7f10841597f0 2023-10-10T20:42:11.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.632+0000 7f1081ffb700 1 -- 172.21.15.89:0/2666505039 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f107801e460 con 0x7f10841597f0 2023-10-10T20:42:11.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.633+0000 7f1081ffb700 1 --2- 172.21.15.89:0/2666505039 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1070067320 0x7f10700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:11.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.633+0000 7f1081ffb700 1 -- 172.21.15.89:0/2666505039 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f10780852d0 con 0x7f10841597f0 2023-10-10T20:42:11.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.633+0000 7f1089e53700 1 --2- 172.21.15.89:0/2666505039 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1070067320 0x7f10700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:11.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.633+0000 7f1089e53700 1 --2- 172.21.15.89:0/2666505039 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1070067320 0x7f10700697d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1074007fd0 tx=0x7f1074009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:11.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.637+0000 7f1081ffb700 1 -- 172.21.15.89:0/2666505039 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f10840c5400 con 0x7f10841597f0 2023-10-10T20:42:11.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.862+0000 7f108ae55700 1 -- 172.21.15.89:0/2666505039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10840c5ac0 con 0x7f10841597f0 2023-10-10T20:42:11.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.863+0000 7f1081ffb700 1 -- 172.21.15.89:0/2666505039 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4618 (secure 0 0 0) 0x7f10840c5ac0 con 0x7f10841597f0 2023-10-10T20:42:11.865 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:11.865 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":93,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6197,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/4123310016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4123310016},{"type":"v1","addr":"172.21.15.111:6834","nonce":4123310016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":6200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/141017824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":141017824},{"type":"v1","addr":"172.21.15.111:6835","nonce":141017824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7575,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/728309352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":728309352},{"type":"v1","addr":"172.21.15.89:6836","nonce":728309352}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:41:49.687845+0000","modified":"2023-10-10T20:42:05.985626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7578":{"gid":7578,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/2102263806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2102263806},{"type":"v1","addr":"172.21.15.89:6837","nonce":2102263806}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"volume_0000000000887918","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-10T20:42:11.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.865+0000 7f108ae55700 1 -- 172.21.15.89:0/2666505039 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1070067320 msgr2=0x7f10700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:11.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.865+0000 7f108ae55700 1 --2- 172.21.15.89:0/2666505039 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1070067320 0x7f10700697d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1074007fd0 tx=0x7f1074009040 comp rx=0 tx=0).stop 2023-10-10T20:42:11.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.865+0000 7f108ae55700 1 -- 172.21.15.89:0/2666505039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10841597f0 msgr2=0x7f108415a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:11.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.865+0000 7f108ae55700 1 --2- 172.21.15.89:0/2666505039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10841597f0 0x7f108415a660 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f107800d860 tx=0x7f1078008040 comp rx=0 tx=0).stop 2023-10-10T20:42:11.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.865+0000 7f108ae55700 1 -- 172.21.15.89:0/2666505039 shutdown_connections 2023-10-10T20:42:11.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.865+0000 7f108ae55700 1 --2- 172.21.15.89:0/2666505039 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f108415d230 0x7f1084153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.865+0000 7f108ae55700 1 --2- 172.21.15.89:0/2666505039 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1070067320 0x7f10700697d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.865+0000 7f108ae55700 1 --2- 172.21.15.89:0/2666505039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1084159420 0x7f108415c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.865+0000 7f108ae55700 1 --2- 172.21.15.89:0/2666505039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10841597f0 0x7f108415a660 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.865+0000 7f108ae55700 1 -- 172.21.15.89:0/2666505039 >> 172.21.15.89:0/2666505039 conn(0x7f10840d0660 msgr2=0x7f10840b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:11.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.865+0000 7f108ae55700 1 -- 172.21.15.89:0/2666505039 shutdown_connections 2023-10-10T20:42:11.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.865+0000 7f108ae55700 1 -- 172.21.15.89:0/2666505039 wait complete. 2023-10-10T20:42:11.870 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 93 2023-10-10T20:42:11.879 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:42:11.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.992+0000 7fb7335dc700 1 Processor -- start 2023-10-10T20:42:11.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.993+0000 7fb7335dc700 1 -- start start 2023-10-10T20:42:11.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.993+0000 7fb7335dc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb72c06bcd0 0x7fb72c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:11.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.993+0000 7fb7335dc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb72c15df50 0x7fb72c15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:11.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.993+0000 7fb7335dc700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb72c15d7e0 con 0x7fb72c06c0a0 2023-10-10T20:42:11.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.993+0000 7fb7335dc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb72c15dac0 con 0x7fb72c15df50 2023-10-10T20:42:11.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.993+0000 7fb7335dc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c15aa30 con 0x7fb72c06bcd0 2023-10-10T20:42:11.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.994+0000 7fb731b79700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb72c15df50 0x7fb72c15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:11.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.994+0000 7fb731b79700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb72c15df50 0x7fb72c15e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54656/0 (socket says 172.21.15.89:54656) 2023-10-10T20:42:11.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.994+0000 7fb731b79700 1 -- 172.21.15.89:0/2505116772 learned_addr learned my addr 172.21.15.89:0/2505116772 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:11.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.994+0000 7fb731378700 1 --2- 172.21.15.89:0/2505116772 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb72c06bcd0 0x7fb72c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:11.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.994+0000 7fb731b79700 1 -- 172.21.15.89:0/2505116772 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb72c06bcd0 msgr2=0x7fb72c0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:11.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.994+0000 7fb731b79700 1 --2- 172.21.15.89:0/2505116772 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb72c06bcd0 0x7fb72c0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:11.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.994+0000 7fb731b79700 1 -- 172.21.15.89:0/2505116772 >> v1:172.21.15.111:6789/0 conn(0x7fb72c06c0a0 legacy=0x7fb72c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:11.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.994+0000 7fb731b79700 1 -- 172.21.15.89:0/2505116772 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c15ad10 con 0x7fb72c15df50 2023-10-10T20:42:12.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.994+0000 7fb731378700 1 --2- 172.21.15.89:0/2505116772 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb72c06bcd0 0x7fb72c0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:12.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.994+0000 7fb731b79700 1 --2- 172.21.15.89:0/2505116772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb72c15df50 0x7fb72c15e3b0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fb724009f20 tx=0x7fb724009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab50fb0bf613aef server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:12.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.994+0000 7fb723fff700 1 -- 172.21.15.89:0/2505116772 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72400e070 con 0x7fb72c15df50 2023-10-10T20:42:12.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.994+0000 7fb723fff700 1 -- 172.21.15.89:0/2505116772 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72400ba80 con 0x7fb72c15df50 2023-10-10T20:42:12.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.994+0000 7fb723fff700 1 -- 172.21.15.89:0/2505116772 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7240092e0 con 0x7fb72c15df50 2023-10-10T20:42:12.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.995+0000 7fb7335dc700 1 -- 172.21.15.89:0/2505116772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb72c15df50 msgr2=0x7fb72c15e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:12.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.995+0000 7fb7335dc700 1 --2- 172.21.15.89:0/2505116772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb72c15df50 0x7fb72c15e3b0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fb724009f20 tx=0x7fb724009840 comp rx=0 tx=0).stop 2023-10-10T20:42:12.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.995+0000 7fb7335dc700 1 -- 172.21.15.89:0/2505116772 shutdown_connections 2023-10-10T20:42:12.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.995+0000 7fb7335dc700 1 --2- 172.21.15.89:0/2505116772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb72c15df50 0x7fb72c15e3b0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.995+0000 7fb7335dc700 1 --2- 172.21.15.89:0/2505116772 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb72c06bcd0 0x7fb72c0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.995+0000 7fb7335dc700 1 -- 172.21.15.89:0/2505116772 >> 172.21.15.89:0/2505116772 conn(0x7fb72c0ce600 msgr2=0x7fb72c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:12.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.995+0000 7fb7335dc700 1 -- 172.21.15.89:0/2505116772 shutdown_connections 2023-10-10T20:42:12.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.995+0000 7fb7335dc700 1 -- 172.21.15.89:0/2505116772 wait complete. 2023-10-10T20:42:12.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.996+0000 7fb7335dc700 1 Processor -- start 2023-10-10T20:42:12.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.996+0000 7fb7335dc700 1 -- start start 2023-10-10T20:42:12.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb7335dc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb72c06bcd0 0x7fb72c16aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:12.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb7335dc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb72c06c0a0 0x7fb72c156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:12.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb7335dc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb72c15df50 0x7fb72c1574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:12.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb7335dc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0b3c80 con 0x7fb72c15df50 2023-10-10T20:42:12.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb7335dc700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0b3070 con 0x7fb72c06bcd0 2023-10-10T20:42:12.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb7335dc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0b3350 con 0x7fb72c06c0a0 2023-10-10T20:42:12.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb731b79700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb72c15df50 0x7fb72c1574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:12.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb731b79700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb72c15df50 0x7fb72c1574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54664/0 (socket says 172.21.15.89:54664) 2023-10-10T20:42:12.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb731b79700 1 -- 172.21.15.89:0/1188753398 learned_addr learned my addr 172.21.15.89:0/1188753398 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:12.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb731378700 1 --2- 172.21.15.89:0/1188753398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb72c06bcd0 0x7fb72c16aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:12.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb730b77700 1 --2- 172.21.15.89:0/1188753398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb72c06c0a0 0x7fb72c156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:12.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb731b79700 1 -- 172.21.15.89:0/1188753398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb72c06c0a0 msgr2=0x7fb72c156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:12.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb731b79700 1 --2- 172.21.15.89:0/1188753398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb72c06c0a0 0x7fb72c156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb731b79700 1 -- 172.21.15.89:0/1188753398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb72c06bcd0 msgr2=0x7fb72c16aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:12.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb731b79700 1 --2- 172.21.15.89:0/1188753398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb72c06bcd0 0x7fb72c16aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb731b79700 1 -- 172.21.15.89:0/1188753398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb724009ae0 con 0x7fb72c15df50 2023-10-10T20:42:12.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb731378700 1 --2- 172.21.15.89:0/1188753398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb72c06bcd0 0x7fb72c16aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:12.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb731b79700 1 --2- 172.21.15.89:0/1188753398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb72c15df50 0x7fb72c1574f0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fb72400b6c0 tx=0x7fb724018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:12.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb7217fa700 1 -- 172.21.15.89:0/1188753398 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724007000 con 0x7fb72c15df50 2023-10-10T20:42:12.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.997+0000 7fb7217fa700 1 -- 172.21.15.89:0/1188753398 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72401d420 con 0x7fb72c15df50 2023-10-10T20:42:12.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.998+0000 7fb7217fa700 1 -- 172.21.15.89:0/1188753398 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72401d820 con 0x7fb72c15df50 2023-10-10T20:42:12.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.998+0000 7fb7335dc700 1 -- 172.21.15.89:0/1188753398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb72c0b3630 con 0x7fb72c15df50 2023-10-10T20:42:12.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.998+0000 7fb7335dc700 1 -- 172.21.15.89:0/1188753398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72c16af90 con 0x7fb72c15df50 2023-10-10T20:42:12.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.998+0000 7fb7335dc700 1 -- 172.21.15.89:0/1188753398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72c0b3e30 con 0x7fb72c15df50 2023-10-10T20:42:12.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.999+0000 7fb7217fa700 1 -- 172.21.15.89:0/1188753398 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb724003050 con 0x7fb72c15df50 2023-10-10T20:42:12.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.999+0000 7fb7217fa700 1 --2- 172.21.15.89:0/1188753398 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb710067320 0x7fb7100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:12.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.999+0000 7fb7217fa700 1 -- 172.21.15.89:0/1188753398 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fb724086ca0 con 0x7fb72c15df50 2023-10-10T20:42:12.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:11.999+0000 7fb731378700 1 --2- 172.21.15.89:0/1188753398 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb710067320 0x7fb7100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:12.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.000+0000 7fb731378700 1 --2- 172.21.15.89:0/1188753398 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb710067320 0x7fb7100697d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb728006fd0 tx=0x7fb728008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:12.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.003+0000 7fb7217fa700 1 -- 172.21.15.89:0/1188753398 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb72c0b3e30 con 0x7fb72c15df50 2023-10-10T20:42:12.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.229+0000 7fb7335dc700 1 -- 172.21.15.89:0/1188753398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb72c06b040 con 0x7fb72c15df50 2023-10-10T20:42:12.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.230+0000 7fb7217fa700 1 -- 172.21.15.89:0/1188753398 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4618 (secure 0 0 0) 0x7fb724053790 con 0x7fb72c15df50 2023-10-10T20:42:12.232 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:12.233 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":93,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6197,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/4123310016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4123310016},{"type":"v1","addr":"172.21.15.111:6834","nonce":4123310016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":6200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/141017824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":141017824},{"type":"v1","addr":"172.21.15.111:6835","nonce":141017824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7575,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/728309352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":728309352},{"type":"v1","addr":"172.21.15.89:6836","nonce":728309352}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:41:49.687845+0000","modified":"2023-10-10T20:42:05.985626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7578":{"gid":7578,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/2102263806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2102263806},{"type":"v1","addr":"172.21.15.89:6837","nonce":2102263806}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"volume_0000000000887918","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-10T20:42:12.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.232+0000 7fb7335dc700 1 -- 172.21.15.89:0/1188753398 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb710067320 msgr2=0x7fb7100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:12.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.232+0000 7fb7335dc700 1 --2- 172.21.15.89:0/1188753398 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb710067320 0x7fb7100697d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb728006fd0 tx=0x7fb728008040 comp rx=0 tx=0).stop 2023-10-10T20:42:12.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.232+0000 7fb7335dc700 1 -- 172.21.15.89:0/1188753398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb72c15df50 msgr2=0x7fb72c1574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:12.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.232+0000 7fb7335dc700 1 --2- 172.21.15.89:0/1188753398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb72c15df50 0x7fb72c1574f0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fb72400b6c0 tx=0x7fb724018000 comp rx=0 tx=0).stop 2023-10-10T20:42:12.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.232+0000 7fb7335dc700 1 -- 172.21.15.89:0/1188753398 shutdown_connections 2023-10-10T20:42:12.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.232+0000 7fb7335dc700 1 --2- 172.21.15.89:0/1188753398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb72c15df50 0x7fb72c1574f0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.232+0000 7fb7335dc700 1 --2- 172.21.15.89:0/1188753398 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb710067320 0x7fb7100697d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.232+0000 7fb7335dc700 1 --2- 172.21.15.89:0/1188753398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb72c06bcd0 0x7fb72c16aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.232+0000 7fb7335dc700 1 --2- 172.21.15.89:0/1188753398 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb72c06c0a0 0x7fb72c156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.232+0000 7fb7335dc700 1 -- 172.21.15.89:0/1188753398 >> 172.21.15.89:0/1188753398 conn(0x7fb72c0ce600 msgr2=0x7fb72c06a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:12.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.232+0000 7fb7335dc700 1 -- 172.21.15.89:0/1188753398 shutdown_connections 2023-10-10T20:42:12.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.232+0000 7fb7335dc700 1 -- 172.21.15.89:0/1188753398 wait complete. 2023-10-10T20:42:12.236 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 93 2023-10-10T20:42:12.246 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:42:12.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.358+0000 7f4659e1c700 1 Processor -- start 2023-10-10T20:42:12.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.358+0000 7f4659e1c700 1 -- start start 2023-10-10T20:42:12.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.359+0000 7f4659e1c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46540b0a10 0x7f4654155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:12.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.359+0000 7f4659e1c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46540ad930 con 0x7f4654155c10 2023-10-10T20:42:12.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.359+0000 7f4659e1c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46540acd20 con 0x7f46541578b0 2023-10-10T20:42:12.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.359+0000 7f4659e1c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46540ad000 con 0x7f46540b0a10 2023-10-10T20:42:12.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.359+0000 7f46537fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4654155c10 0x7f46540ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58760/0 (socket says 172.21.15.89:58760) 2023-10-10T20:42:12.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.359+0000 7f46537fe700 1 -- 172.21.15.89:0/3389127561 learned_addr learned my addr 172.21.15.89:0/3389127561 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:12.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.359+0000 7f46527fc700 1 -- 172.21.15.89:0/3389127561 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4145547103 0 0) 0x7f46540ad930 con 0x7f4654155c10 2023-10-10T20:42:12.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.359+0000 7f46527fc700 1 -- 172.21.15.89:0/3389127561 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f463c003570 con 0x7f4654155c10 2023-10-10T20:42:12.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.359+0000 7f4653fff700 1 --2- 172.21.15.89:0/3389127561 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46540b0a10 0x7f4654155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:12.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46527fc700 1 -- 172.21.15.89:0/3389127561 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3952952948 0 0) 0x7f463c003570 con 0x7f4654155c10 2023-10-10T20:42:12.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46527fc700 1 -- 172.21.15.89:0/3389127561 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46540b0a10 msgr2=0x7f4654155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:12.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46527fc700 1 --2- 172.21.15.89:0/3389127561 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46540b0a10 0x7f4654155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46527fc700 1 -- 172.21.15.89:0/3389127561 >> v1:172.21.15.111:6789/0 conn(0x7f46541578b0 legacy=0x7f46540af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:12.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46527fc700 1 -- 172.21.15.89:0/3389127561 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46540ad2e0 con 0x7f4654155c10 2023-10-10T20:42:12.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46527fc700 1 -- 172.21.15.89:0/3389127561 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f4644001470 con 0x7f4654155c10 2023-10-10T20:42:12.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46527fc700 1 -- 172.21.15.89:0/3389127561 >> v1:172.21.15.89:6789/0 conn(0x7f4654155c10 legacy=0x7f46540ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:12.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46527fc700 1 --2- 172.21.15.89:0/3389127561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f463c003a90 0x7f463c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:12.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46527fc700 1 --2- 172.21.15.89:0/3389127561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f463c004810 0x7f463c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:12.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46527fc700 1 --2- 172.21.15.89:0/3389127561 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46540b0a10 0x7f463c005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:12.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46527fc700 1 -- 172.21.15.89:0/3389127561 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46540ad000 con 0x7f463c003a90 2023-10-10T20:42:12.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46527fc700 1 -- 172.21.15.89:0/3389127561 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c00bc20 con 0x7f463c004810 2023-10-10T20:42:12.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46527fc700 1 -- 172.21.15.89:0/3389127561 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463c00bde0 con 0x7f46540b0a10 2023-10-10T20:42:12.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46537fe700 1 --2- 172.21.15.89:0/3389127561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f463c003a90 0x7f463c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:12.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f4653fff700 1 --2- 172.21.15.89:0/3389127561 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46540b0a10 0x7f463c005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:12.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f4652ffd700 1 --2- 172.21.15.89:0/3389127561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f463c004810 0x7f463c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:12.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46537fe700 1 -- 172.21.15.89:0/3389127561 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46540b0a10 msgr2=0x7f463c005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:12.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46537fe700 1 --2- 172.21.15.89:0/3389127561 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46540b0a10 0x7f463c005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46537fe700 1 -- 172.21.15.89:0/3389127561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f463c004810 msgr2=0x7f463c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:12.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46537fe700 1 --2- 172.21.15.89:0/3389127561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f463c004810 0x7f463c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46537fe700 1 -- 172.21.15.89:0/3389127561 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c003570 con 0x7f463c003a90 2023-10-10T20:42:12.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f4652ffd700 1 --2- 172.21.15.89:0/3389127561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f463c004810 0x7f463c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:12.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46537fe700 1 --2- 172.21.15.89:0/3389127561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f463c003a90 0x7f463c003ea0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4644001ed0 tx=0x7f46540ad8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67d77a9d4830b76 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:12.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46527fc700 1 -- 172.21.15.89:0/3389127561 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464400cc30 con 0x7f463c003a90 2023-10-10T20:42:12.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46527fc700 1 -- 172.21.15.89:0/3389127561 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464400cd90 con 0x7f463c003a90 2023-10-10T20:42:12.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.360+0000 7f46527fc700 1 -- 172.21.15.89:0/3389127561 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4644005b70 con 0x7f463c003a90 2023-10-10T20:42:12.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.361+0000 7f4659e1c700 1 -- 172.21.15.89:0/3389127561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f463c003a90 msgr2=0x7f463c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:12.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.361+0000 7f4659e1c700 1 --2- 172.21.15.89:0/3389127561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f463c003a90 0x7f463c003ea0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4644001ed0 tx=0x7f46540ad8a0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.361+0000 7f4653fff700 1 -- 172.21.15.89:0/3389127561 reap_dead start 2023-10-10T20:42:12.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.361+0000 7f4659e1c700 1 -- 172.21.15.89:0/3389127561 shutdown_connections 2023-10-10T20:42:12.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.361+0000 7f4659e1c700 1 -- 172.21.15.89:0/3389127561 >> 172.21.15.89:0/3389127561 conn(0x7f465406a320 msgr2=0x7f46540cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:12.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.361+0000 7f4659e1c700 1 -- 172.21.15.89:0/3389127561 shutdown_connections 2023-10-10T20:42:12.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.361+0000 7f4659e1c700 1 -- 172.21.15.89:0/3389127561 wait complete. 2023-10-10T20:42:12.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.362+0000 7f4659e1c700 1 Processor -- start 2023-10-10T20:42:12.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.362+0000 7f4659e1c700 1 -- start start 2023-10-10T20:42:12.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.363+0000 7f4659e1c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f463c003a90 0x7f465416b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:12.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.363+0000 7f4659e1c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f465416b9f0 0x7f46540cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:12.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.363+0000 7f4659e1c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46540cb9b0 0x7f4654147200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:12.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.363+0000 7f4659e1c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465416be60 con 0x7f463c003a90 2023-10-10T20:42:12.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.363+0000 7f4659e1c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4654169080 con 0x7f465416b9f0 2023-10-10T20:42:12.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.363+0000 7f4659e1c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46540cc1e0 con 0x7f46540cb9b0 2023-10-10T20:42:12.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.363+0000 7f46537fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f463c003a90 0x7f465416b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:12.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.363+0000 7f46537fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f463c003a90 0x7f465416b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54674/0 (socket says 172.21.15.89:54674) 2023-10-10T20:42:12.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.363+0000 7f46537fe700 1 -- 172.21.15.89:0/3055130848 learned_addr learned my addr 172.21.15.89:0/3055130848 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:12.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.363+0000 7f4652ffd700 1 --2- 172.21.15.89:0/3055130848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f465416b9f0 0x7f46540cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:12.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.363+0000 7f4653fff700 1 --2- 172.21.15.89:0/3055130848 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46540cb9b0 0x7f4654147200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:12.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.363+0000 7f46537fe700 1 -- 172.21.15.89:0/3055130848 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46540cb9b0 msgr2=0x7f4654147200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:12.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.363+0000 7f46537fe700 1 --2- 172.21.15.89:0/3055130848 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46540cb9b0 0x7f4654147200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.363+0000 7f46537fe700 1 -- 172.21.15.89:0/3055130848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f465416b9f0 msgr2=0x7f46540cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:12.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.363+0000 7f46537fe700 1 --2- 172.21.15.89:0/3055130848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f465416b9f0 0x7f46540cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.364+0000 7f46537fe700 1 -- 172.21.15.89:0/3055130848 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46540cc4c0 con 0x7f463c003a90 2023-10-10T20:42:12.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.364+0000 7f4653fff700 1 --2- 172.21.15.89:0/3055130848 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46540cb9b0 0x7f4654147200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:12.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.364+0000 7f46537fe700 1 --2- 172.21.15.89:0/3055130848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f463c003a90 0x7f465416b4b0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f4644007cd0 tx=0x7f4644015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:12.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.364+0000 7f463b7fe700 1 -- 172.21.15.89:0/3055130848 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464400cc30 con 0x7f463c003a90 2023-10-10T20:42:12.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.364+0000 7f463b7fe700 1 -- 172.21.15.89:0/3055130848 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46440072e0 con 0x7f463c003a90 2023-10-10T20:42:12.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.364+0000 7f463b7fe700 1 -- 172.21.15.89:0/3055130848 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46440076a0 con 0x7f463c003a90 2023-10-10T20:42:12.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.364+0000 7f4659e1c700 1 -- 172.21.15.89:0/3055130848 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46540cc7a0 con 0x7f463c003a90 2023-10-10T20:42:12.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.364+0000 7f4659e1c700 1 -- 172.21.15.89:0/3055130848 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46540cbe30 con 0x7f463c003a90 2023-10-10T20:42:12.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.365+0000 7f4659e1c700 1 -- 172.21.15.89:0/3055130848 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465416bdc0 con 0x7f463c003a90 2023-10-10T20:42:12.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.365+0000 7f463b7fe700 1 -- 172.21.15.89:0/3055130848 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4644019030 con 0x7f463c003a90 2023-10-10T20:42:12.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.366+0000 7f463b7fe700 1 --2- 172.21.15.89:0/3055130848 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4640067320 0x7f46400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:12.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.366+0000 7f463b7fe700 1 -- 172.21.15.89:0/3055130848 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f4644085d50 con 0x7f463c003a90 2023-10-10T20:42:12.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.366+0000 7f4652ffd700 1 --2- 172.21.15.89:0/3055130848 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4640067320 0x7f46400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:12.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.366+0000 7f4652ffd700 1 --2- 172.21.15.89:0/3055130848 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4640067320 0x7f46400697d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4648007fd0 tx=0x7f4648009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:12.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.370+0000 7f463b7fe700 1 -- 172.21.15.89:0/3055130848 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46440b75a0 con 0x7f463c003a90 2023-10-10T20:42:12.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.553+0000 7f4659e1c700 1 -- 172.21.15.89:0/3055130848 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4654169360 con 0x7f463c003a90 2023-10-10T20:42:12.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.554+0000 7f463b7fe700 1 -- 172.21.15.89:0/3055130848 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+15442 (secure 0 0 0) 0x7f4644052990 con 0x7f463c003a90 2023-10-10T20:42:12.556 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:12.557 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":141,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:42:07.988420+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs.volume_0000000000887918.meta","create_time":"2023-10-10T20:41:48.040814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000887918"}}},{"pool":25,"pool_name":"cephfs.volume_0000000000887918.data","create_time":"2023-10-10T20:41:48.646159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000887918"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:42:12.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.556+0000 7f4659e1c700 1 -- 172.21.15.89:0/3055130848 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4640067320 msgr2=0x7f46400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:12.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.556+0000 7f4659e1c700 1 --2- 172.21.15.89:0/3055130848 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4640067320 0x7f46400697d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4648007fd0 tx=0x7f4648009040 comp rx=0 tx=0).stop 2023-10-10T20:42:12.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.556+0000 7f4659e1c700 1 -- 172.21.15.89:0/3055130848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f463c003a90 msgr2=0x7f465416b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:12.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.556+0000 7f4659e1c700 1 --2- 172.21.15.89:0/3055130848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f463c003a90 0x7f465416b4b0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f4644007cd0 tx=0x7f4644015000 comp rx=0 tx=0).stop 2023-10-10T20:42:12.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.556+0000 7f4659e1c700 1 -- 172.21.15.89:0/3055130848 shutdown_connections 2023-10-10T20:42:12.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.556+0000 7f4659e1c700 1 --2- 172.21.15.89:0/3055130848 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f46540cb9b0 0x7f4654147200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.556+0000 7f4659e1c700 1 --2- 172.21.15.89:0/3055130848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f463c003a90 0x7f465416b4b0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.556+0000 7f4659e1c700 1 --2- 172.21.15.89:0/3055130848 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4640067320 0x7f46400697d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.556+0000 7f4659e1c700 1 --2- 172.21.15.89:0/3055130848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f465416b9f0 0x7f46540cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.556+0000 7f4659e1c700 1 -- 172.21.15.89:0/3055130848 >> 172.21.15.89:0/3055130848 conn(0x7f465406a320 msgr2=0x7f465406e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.556+0000 7f4659e1c700 1 -- 172.21.15.89:0/3055130848 shutdown_connections 2023-10-10T20:42:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.556+0000 7f4659e1c700 1 -- 172.21.15.89:0/3055130848 wait complete. 2023-10-10T20:42:12.571 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail volume_0000000000887918 2023-10-10T20:42:12.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.686+0000 7fa31e25a700 1 Processor -- start 2023-10-10T20:42:12.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.686+0000 7fa31e25a700 1 -- start start 2023-10-10T20:42:12.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.687+0000 7fa31e25a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa318068550 0x7fa318068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:12.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.687+0000 7fa31e25a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa31806f810 0x7fa318159510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:12.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.687+0000 7fa31e25a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31806ec70 con 0x7fa31815e010 2023-10-10T20:42:12.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.687+0000 7fa31e25a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3180b3430 con 0x7fa31806f810 2023-10-10T20:42:12.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.687+0000 7fa31e25a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31806fc90 con 0x7fa318068550 2023-10-10T20:42:12.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.687+0000 7fa317fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa31806f810 0x7fa318159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:12.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.687+0000 7fa316ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa31815e010 0x7fa31806f100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58770/0 (socket says 172.21.15.89:58770) 2023-10-10T20:42:12.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.687+0000 7fa316ffd700 1 -- 172.21.15.89:0/3641499241 learned_addr learned my addr 172.21.15.89:0/3641499241 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:12.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.687+0000 7fa3177fe700 1 --2- 172.21.15.89:0/3641499241 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa318068550 0x7fa318068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:12.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.687+0000 7fa317fff700 1 -- 172.21.15.89:0/3641499241 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa318068550 msgr2=0x7fa318068920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:12.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.687+0000 7fa317fff700 1 --2- 172.21.15.89:0/3641499241 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa318068550 0x7fa318068920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.687+0000 7fa317fff700 1 -- 172.21.15.89:0/3641499241 >> v1:172.21.15.89:6789/0 conn(0x7fa31815e010 legacy=0x7fa31806f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:12.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.687+0000 7fa317fff700 1 -- 172.21.15.89:0/3641499241 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3180b2e70 con 0x7fa31806f810 2023-10-10T20:42:12.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.687+0000 7fa3167fc700 1 -- 172.21.15.89:0/3641499241 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 681964780 0 0) 0x7fa31806ec70 con 0x7fa31815e010 2023-10-10T20:42:12.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.687+0000 7fa317fff700 1 --2- 172.21.15.89:0/3641499241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa31806f810 0x7fa318159510 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fa310009e60 tx=0x7fa310009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45310c93999e6ab5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:12.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.687+0000 7fa3167fc700 1 -- 172.21.15.89:0/3641499241 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31000e070 con 0x7fa31806f810 2023-10-10T20:42:12.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.687+0000 7fa3167fc700 1 -- 172.21.15.89:0/3641499241 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31000bc30 con 0x7fa31806f810 2023-10-10T20:42:12.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.687+0000 7fa3167fc700 1 -- 172.21.15.89:0/3641499241 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310012430 con 0x7fa31806f810 2023-10-10T20:42:12.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.688+0000 7fa31e25a700 1 -- 172.21.15.89:0/3641499241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa31806f810 msgr2=0x7fa318159510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.688+0000 7fa31e25a700 1 --2- 172.21.15.89:0/3641499241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa31806f810 0x7fa318159510 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fa310009e60 tx=0x7fa310009780 comp rx=0 tx=0).stop 2023-10-10T20:42:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.688+0000 7fa31e25a700 1 -- 172.21.15.89:0/3641499241 shutdown_connections 2023-10-10T20:42:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.688+0000 7fa31e25a700 1 --2- 172.21.15.89:0/3641499241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa31806f810 0x7fa318159510 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.688+0000 7fa31e25a700 1 --2- 172.21.15.89:0/3641499241 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa318068550 0x7fa318068920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.688+0000 7fa31e25a700 1 -- 172.21.15.89:0/3641499241 >> 172.21.15.89:0/3641499241 conn(0x7fa3180cd480 msgr2=0x7fa3180cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:12.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.688+0000 7fa31e25a700 1 -- 172.21.15.89:0/3641499241 shutdown_connections 2023-10-10T20:42:12.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.688+0000 7fa31e25a700 1 -- 172.21.15.89:0/3641499241 wait complete. 2023-10-10T20:42:12.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.689+0000 7fa31e25a700 1 Processor -- start 2023-10-10T20:42:12.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.689+0000 7fa31e25a700 1 -- start start 2023-10-10T20:42:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.690+0000 7fa31e25a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa318068550 0x7fa31815c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.690+0000 7fa31e25a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa31806f810 0x7fa31815af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.690+0000 7fa31e25a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa31815e010 0x7fa3180aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.690+0000 7fa31e25a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31815c500 con 0x7fa31815e010 2023-10-10T20:42:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.690+0000 7fa31e25a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31815b8f0 con 0x7fa318068550 2023-10-10T20:42:12.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.690+0000 7fa31e25a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31815bbd0 con 0x7fa31806f810 2023-10-10T20:42:12.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.690+0000 7fa317fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa31815e010 0x7fa3180aa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:12.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.690+0000 7fa317fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa31815e010 0x7fa3180aa1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54702/0 (socket says 172.21.15.89:54702) 2023-10-10T20:42:12.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.690+0000 7fa317fff700 1 -- 172.21.15.89:0/2532073921 learned_addr learned my addr 172.21.15.89:0/2532073921 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:12.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.690+0000 7fa316ffd700 1 --2- 172.21.15.89:0/2532073921 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa31806f810 0x7fa31815af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:12.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.690+0000 7fa3177fe700 1 --2- 172.21.15.89:0/2532073921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa318068550 0x7fa31815c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:12.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.690+0000 7fa317fff700 1 -- 172.21.15.89:0/2532073921 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa31806f810 msgr2=0x7fa31815af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:12.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.690+0000 7fa317fff700 1 --2- 172.21.15.89:0/2532073921 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa31806f810 0x7fa31815af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.690+0000 7fa317fff700 1 -- 172.21.15.89:0/2532073921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa318068550 msgr2=0x7fa31815c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:12.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.690+0000 7fa317fff700 1 --2- 172.21.15.89:0/2532073921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa318068550 0x7fa31815c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.690+0000 7fa317fff700 1 -- 172.21.15.89:0/2532073921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa310009a20 con 0x7fa31815e010 2023-10-10T20:42:12.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.690+0000 7fa316ffd700 1 --2- 172.21.15.89:0/2532073921 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa31806f810 0x7fa31815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:12.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.690+0000 7fa3177fe700 1 --2- 172.21.15.89:0/2532073921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa318068550 0x7fa31815c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:12.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.691+0000 7fa317fff700 1 --2- 172.21.15.89:0/2532073921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa31815e010 0x7fa3180aa1a0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fa310013000 tx=0x7fa310019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:12.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.691+0000 7fa2ff7fe700 1 -- 172.21.15.89:0/2532073921 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31001e630 con 0x7fa31815e010 2023-10-10T20:42:12.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.691+0000 7fa2ff7fe700 1 -- 172.21.15.89:0/2532073921 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31001e790 con 0x7fa31815e010 2023-10-10T20:42:12.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.691+0000 7fa31e25a700 1 -- 172.21.15.89:0/2532073921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31815beb0 con 0x7fa31815e010 2023-10-10T20:42:12.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.691+0000 7fa2ff7fe700 1 -- 172.21.15.89:0/2532073921 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31001eb50 con 0x7fa31815e010 2023-10-10T20:42:12.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.691+0000 7fa31e25a700 1 -- 172.21.15.89:0/2532073921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31815ced0 con 0x7fa31815e010 2023-10-10T20:42:12.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.692+0000 7fa31e25a700 1 -- 172.21.15.89:0/2532073921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31804c3c0 con 0x7fa31815e010 2023-10-10T20:42:12.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.692+0000 7fa2ff7fe700 1 -- 172.21.15.89:0/2532073921 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa310024070 con 0x7fa31815e010 2023-10-10T20:42:12.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.692+0000 7fa2ff7fe700 1 --2- 172.21.15.89:0/2532073921 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa304067320 0x7fa3040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:12.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.692+0000 7fa2ff7fe700 1 -- 172.21.15.89:0/2532073921 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fa3100876e0 con 0x7fa31815e010 2023-10-10T20:42:12.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.692+0000 7fa3177fe700 1 --2- 172.21.15.89:0/2532073921 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa304067320 0x7fa3040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:12.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.693+0000 7fa3177fe700 1 --2- 172.21.15.89:0/2532073921 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa304067320 0x7fa3040697d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa30c008a80 tx=0x7fa30c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:12.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.697+0000 7fa2ff7fe700 1 -- 172.21.15.89:0/2532073921 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa310054320 con 0x7fa31815e010 2023-10-10T20:42:12.830 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:42:12.833 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:42:12.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.927+0000 7fa31e25a700 1 -- 172.21.15.89:0/2532073921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "volume_0000000000887918"} v 0) v1 -- 0x7fa31815c640 con 0x7fa31815e010 2023-10-10T20:42:12.929 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-10-10T20:42:12.927+0000 7fd0184ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:42:12.933 INFO:tasks.ceph.mds.a.smithi089.stderr: -834> 2023-10-10T20:41:55.606+0000 7f125766d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:42:12.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.933+0000 7fa2ff7fe700 1 -- 172.21.15.89:0/2532073921 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "volume_0000000000887918"}]=0 volume_0000000000887918 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v94) v1 ==== 195+0+0 (secure 0 0 0) 0x7fa310057940 con 0x7fa31815e010 2023-10-10T20:42:12.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.936+0000 7fa31e25a700 1 -- 172.21.15.89:0/2532073921 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa304067320 msgr2=0x7fa3040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:12.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.936+0000 7fa31e25a700 1 --2- 172.21.15.89:0/2532073921 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa304067320 0x7fa3040697d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa30c008a80 tx=0x7fa30c008040 comp rx=0 tx=0).stop 2023-10-10T20:42:12.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.936+0000 7fa31e25a700 1 -- 172.21.15.89:0/2532073921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa31815e010 msgr2=0x7fa3180aa1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:12.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.936+0000 7fa31e25a700 1 --2- 172.21.15.89:0/2532073921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa31815e010 0x7fa3180aa1a0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fa310013000 tx=0x7fa310019000 comp rx=0 tx=0).stop 2023-10-10T20:42:12.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.936+0000 7fa31e25a700 1 -- 172.21.15.89:0/2532073921 shutdown_connections 2023-10-10T20:42:12.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.936+0000 7fa31e25a700 1 --2- 172.21.15.89:0/2532073921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa31815e010 0x7fa3180aa1a0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.936+0000 7fa31e25a700 1 --2- 172.21.15.89:0/2532073921 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa304067320 0x7fa3040697d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.936+0000 7fa31e25a700 1 --2- 172.21.15.89:0/2532073921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa318068550 0x7fa31815c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.936+0000 7fa31e25a700 1 --2- 172.21.15.89:0/2532073921 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa31806f810 0x7fa31815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:12.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.936+0000 7fa31e25a700 1 -- 172.21.15.89:0/2532073921 >> 172.21.15.89:0/2532073921 conn(0x7fa3180cd480 msgr2=0x7fa3180af830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:12.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.936+0000 7fa31e25a700 1 -- 172.21.15.89:0/2532073921 shutdown_connections 2023-10-10T20:42:12.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:12.936+0000 7fa31e25a700 1 -- 172.21.15.89:0/2532073921 wait complete. 2023-10-10T20:42:12.940 INFO:teuthology.orchestra.run.smithi089.stderr:volume_0000000000887918 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:42:12.951 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm volume_0000000000887918 --yes-i-really-mean-it 2023-10-10T20:42:12.954 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:42:13.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.072+0000 7f7f52ad9700 1 Processor -- start 2023-10-10T20:42:13.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.073+0000 7f7f52ad9700 1 -- start start 2023-10-10T20:42:13.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.073+0000 7f7f52ad9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7f4c0aca80 0x7f7f4c06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:13.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.073+0000 7f7f52ad9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7f4c069d60 0x7f7f4c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:13.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.073+0000 7f7f52ad9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f4c06bdb0 con 0x7f7f4c0c81f0 2023-10-10T20:42:13.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.073+0000 7f7f52ad9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f4c0c8670 con 0x7f7f4c069d60 2023-10-10T20:42:13.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.073+0000 7f7f52ad9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c0cb370 con 0x7f7f4c0aca80 2023-10-10T20:42:13.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.073+0000 7f7f43fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7f4c069d60 0x7f7f4c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:13.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.073+0000 7f7f51076700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7f4c0c81f0 0x7f7f4c068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58794/0 (socket says 172.21.15.89:58794) 2023-10-10T20:42:13.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.073+0000 7f7f51076700 1 -- 172.21.15.89:0/2091461313 learned_addr learned my addr 172.21.15.89:0/2091461313 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:13.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.074+0000 7f7f50875700 1 --2- 172.21.15.89:0/2091461313 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7f4c0aca80 0x7f7f4c06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:13.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.074+0000 7f7f43fff700 1 -- 172.21.15.89:0/2091461313 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7f4c0aca80 msgr2=0x7f7f4c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:13.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.074+0000 7f7f43fff700 1 --2- 172.21.15.89:0/2091461313 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7f4c0aca80 0x7f7f4c06aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:13.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.074+0000 7f7f43fff700 1 -- 172.21.15.89:0/2091461313 >> v1:172.21.15.89:6789/0 conn(0x7f7f4c0c81f0 legacy=0x7f7f4c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:13.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.074+0000 7f7f43fff700 1 -- 172.21.15.89:0/2091461313 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4c159f60 con 0x7f7f4c069d60 2023-10-10T20:42:13.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.074+0000 7f7f437fe700 1 -- 172.21.15.89:0/2091461313 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475417668 0 0) 0x7f7f4c06bdb0 con 0x7f7f4c0c81f0 2023-10-10T20:42:13.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.074+0000 7f7f43fff700 1 --2- 172.21.15.89:0/2091461313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f4c069d60 0x7f7f4c0c7cb0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0098e0 tx=0x7f7f3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1527f3d80f545bca server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:13.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.074+0000 7f7f437fe700 1 -- 172.21.15.89:0/2091461313 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c009e80 con 0x7f7f4c069d60 2023-10-10T20:42:13.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.074+0000 7f7f437fe700 1 -- 172.21.15.89:0/2091461313 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3c00b860 con 0x7f7f4c069d60 2023-10-10T20:42:13.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.074+0000 7f7f437fe700 1 -- 172.21.15.89:0/2091461313 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c00bbe0 con 0x7f7f4c069d60 2023-10-10T20:42:13.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.075+0000 7f7f52ad9700 1 -- 172.21.15.89:0/2091461313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f4c069d60 msgr2=0x7f7f4c0c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:13.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.075+0000 7f7f52ad9700 1 --2- 172.21.15.89:0/2091461313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f4c069d60 0x7f7f4c0c7cb0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0098e0 tx=0x7f7f3c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:13.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.075+0000 7f7f52ad9700 1 -- 172.21.15.89:0/2091461313 shutdown_connections 2023-10-10T20:42:13.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.075+0000 7f7f52ad9700 1 --2- 172.21.15.89:0/2091461313 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7f4c0aca80 0x7f7f4c06aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:13.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.075+0000 7f7f52ad9700 1 --2- 172.21.15.89:0/2091461313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f4c069d60 0x7f7f4c0c7cb0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:13.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.075+0000 7f7f52ad9700 1 -- 172.21.15.89:0/2091461313 >> 172.21.15.89:0/2091461313 conn(0x7f7f4c0cd670 msgr2=0x7f7f4c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:13.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.075+0000 7f7f52ad9700 1 -- 172.21.15.89:0/2091461313 shutdown_connections 2023-10-10T20:42:13.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.075+0000 7f7f52ad9700 1 -- 172.21.15.89:0/2091461313 wait complete. 2023-10-10T20:42:13.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.076+0000 7f7f52ad9700 1 Processor -- start 2023-10-10T20:42:13.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.076+0000 7f7f52ad9700 1 -- start start 2023-10-10T20:42:13.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f52ad9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f4c069d60 0x7f7f4c0b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:13.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f52ad9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7f4c0aca80 0x7f7f4c0c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:13.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f52ad9700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f4c0c81f0 0x7f7f4c0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:13.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f52ad9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c1557e0 con 0x7f7f4c069d60 2023-10-10T20:42:13.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f52ad9700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c0c6b20 con 0x7f7f4c0c81f0 2023-10-10T20:42:13.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f52ad9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c0c75b0 con 0x7f7f4c0aca80 2023-10-10T20:42:13.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f50875700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f4c069d60 0x7f7f4c0b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:13.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f50875700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f4c069d60 0x7f7f4c0b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54724/0 (socket says 172.21.15.89:54724) 2023-10-10T20:42:13.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f50875700 1 -- 172.21.15.89:0/2101473782 learned_addr learned my addr 172.21.15.89:0/2101473782 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:13.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f50875700 1 -- 172.21.15.89:0/2101473782 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7f4c0aca80 msgr2=0x7f7f4c0c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:13.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f50875700 1 --2- 172.21.15.89:0/2101473782 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7f4c0aca80 0x7f7f4c0c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:13.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f50875700 1 -- 172.21.15.89:0/2101473782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f4c0c81f0 msgr2=0x7f7f4c0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:13.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f50875700 1 --2- 172.21.15.89:0/2101473782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f4c0c81f0 0x7f7f4c0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:13.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f50875700 1 -- 172.21.15.89:0/2101473782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c009420 con 0x7f7f4c069d60 2023-10-10T20:42:13.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f50875700 1 --2- 172.21.15.89:0/2101473782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f4c069d60 0x7f7f4c0b2fb0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009fd0 tx=0x7f7f48009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:13.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f40ff9700 1 -- 172.21.15.89:0/2101473782 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4800ed70 con 0x7f7f4c069d60 2023-10-10T20:42:13.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f40ff9700 1 -- 172.21.15.89:0/2101473782 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f48014420 con 0x7f7f4c069d60 2023-10-10T20:42:13.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f52ad9700 1 -- 172.21.15.89:0/2101473782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4c0adef0 con 0x7f7f4c069d60 2023-10-10T20:42:13.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f40ff9700 1 -- 172.21.15.89:0/2101473782 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f480147e0 con 0x7f7f4c069d60 2023-10-10T20:42:13.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.077+0000 7f7f52ad9700 1 -- 172.21.15.89:0/2101473782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4c0c6ff0 con 0x7f7f4c069d60 2023-10-10T20:42:13.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.078+0000 7f7f52ad9700 1 -- 172.21.15.89:0/2101473782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f4c0adcc0 con 0x7f7f4c069d60 2023-10-10T20:42:13.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.079+0000 7f7f40ff9700 1 -- 172.21.15.89:0/2101473782 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7f48014b90 con 0x7f7f4c069d60 2023-10-10T20:42:13.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.079+0000 7f7f40ff9700 1 --2- 172.21.15.89:0/2101473782 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7f38067320 0x7f7f380697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:13.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.079+0000 7f7f40ff9700 1 -- 172.21.15.89:0/2101473782 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5869+0+0 (secure 0 0 0) 0x7f7f48085510 con 0x7f7f4c069d60 2023-10-10T20:42:13.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.079+0000 7f7f43fff700 1 --2- 172.21.15.89:0/2101473782 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7f38067320 0x7f7f380697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:13.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.080+0000 7f7f43fff700 1 --2- 172.21.15.89:0/2101473782 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7f38067320 0x7f7f380697d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c06a130 tx=0x7f7f3c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:13.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.084+0000 7f7f40ff9700 1 -- 172.21.15.89:0/2101473782 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f4c0adcc0 con 0x7f7f4c069d60 2023-10-10T20:42:13.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.313+0000 7f7f52ad9700 1 -- 172.21.15.89:0/2101473782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "volume_0000000000887918", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7f4c0adcc0 con 0x7f7f4c069d60 2023-10-10T20:42:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.932+0000 7f7f40ff9700 1 -- 172.21.15.89:0/2101473782 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "volume_0000000000887918", "yes_i_really_mean_it": true}]=0 v95) v1 ==== 121+0+0 (secure 0 0 0) 0x7f7f4c0adcc0 con 0x7f7f4c069d60 2023-10-10T20:42:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.935+0000 7f7f52ad9700 1 -- 172.21.15.89:0/2101473782 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7f38067320 msgr2=0x7f7f380697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.935+0000 7f7f52ad9700 1 --2- 172.21.15.89:0/2101473782 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7f38067320 0x7f7f380697d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c06a130 tx=0x7f7f3c017000 comp rx=0 tx=0).stop 2023-10-10T20:42:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.935+0000 7f7f52ad9700 1 -- 172.21.15.89:0/2101473782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f4c069d60 msgr2=0x7f7f4c0b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.935+0000 7f7f52ad9700 1 --2- 172.21.15.89:0/2101473782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f4c069d60 0x7f7f4c0b2fb0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009fd0 tx=0x7f7f48009970 comp rx=0 tx=0).stop 2023-10-10T20:42:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.935+0000 7f7f52ad9700 1 -- 172.21.15.89:0/2101473782 shutdown_connections 2023-10-10T20:42:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.935+0000 7f7f52ad9700 1 --2- 172.21.15.89:0/2101473782 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7f38067320 0x7f7f380697d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.935+0000 7f7f52ad9700 1 --2- 172.21.15.89:0/2101473782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f4c0c81f0 0x7f7f4c0b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.935+0000 7f7f52ad9700 1 --2- 172.21.15.89:0/2101473782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f4c069d60 0x7f7f4c0b2fb0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.935+0000 7f7f52ad9700 1 --2- 172.21.15.89:0/2101473782 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7f4c0aca80 0x7f7f4c0c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.935+0000 7f7f52ad9700 1 -- 172.21.15.89:0/2101473782 >> 172.21.15.89:0/2101473782 conn(0x7f7f4c0cd670 msgr2=0x7f7f4c152350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.935+0000 7f7f52ad9700 1 -- 172.21.15.89:0/2101473782 shutdown_connections 2023-10-10T20:42:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:13.935+0000 7f7f52ad9700 1 -- 172.21.15.89:0/2101473782 wait complete. 2023-10-10T20:42:13.950 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000887918.meta cephfs.volume_0000000000887918.meta --yes-i-really-really-mean-it 2023-10-10T20:42:14.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.070+0000 7fd88f5dd700 1 Processor -- start 2023-10-10T20:42:14.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.070+0000 7fd88f5dd700 1 -- start start 2023-10-10T20:42:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.071+0000 7fd88f5dd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd8880cd910 0x7fd8880ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.071+0000 7fd88f5dd700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8880b5b00 con 0x7fd888159ec0 2023-10-10T20:42:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.071+0000 7fd88f5dd700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8880b5da0 con 0x7fd8880b55a0 2023-10-10T20:42:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.071+0000 7fd88f5dd700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8880b6080 con 0x7fd8880cd910 2023-10-10T20:42:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.071+0000 7fd88cb78700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd8880cd910 0x7fd8880ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.071+0000 7fd88d379700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd888159ec0 0x7fd8880b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58798/0 (socket says 172.21.15.89:58798) 2023-10-10T20:42:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.071+0000 7fd88d379700 1 -- 172.21.15.89:0/1498802622 learned_addr learned my addr 172.21.15.89:0/1498802622 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.071+0000 7fd88cb78700 1 -- 172.21.15.89:0/1498802622 >> v1:172.21.15.111:6789/0 conn(0x7fd8880b55a0 legacy=0x7fd8880ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.071+0000 7fd88cb78700 1 -- 172.21.15.89:0/1498802622 >> v1:172.21.15.89:6789/0 conn(0x7fd888159ec0 legacy=0x7fd8880b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.071+0000 7fd88cb78700 1 -- 172.21.15.89:0/1498802622 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8880cc100 con 0x7fd8880cd910 2023-10-10T20:42:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.072+0000 7fd88cb78700 1 --2- 172.21.15.89:0/1498802622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8880cd910 0x7fd8880ce340 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fd878009910 tx=0x7fd8780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1d3f9e0c049afd server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.072+0000 7fd87ffff700 1 -- 172.21.15.89:0/1498802622 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878009eb0 con 0x7fd8880cd910 2023-10-10T20:42:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.072+0000 7fd87ffff700 1 -- 172.21.15.89:0/1498802622 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87800b760 con 0x7fd8880cd910 2023-10-10T20:42:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.072+0000 7fd87ffff700 1 -- 172.21.15.89:0/1498802622 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87800bb00 con 0x7fd8880cd910 2023-10-10T20:42:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.072+0000 7fd88f5dd700 1 -- 172.21.15.89:0/1498802622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8880cd910 msgr2=0x7fd8880ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.072+0000 7fd88f5dd700 1 --2- 172.21.15.89:0/1498802622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8880cd910 0x7fd8880ce340 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fd878009910 tx=0x7fd8780091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.072+0000 7fd88f5dd700 1 -- 172.21.15.89:0/1498802622 shutdown_connections 2023-10-10T20:42:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.072+0000 7fd88f5dd700 1 --2- 172.21.15.89:0/1498802622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8880cd910 0x7fd8880ce340 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.072+0000 7fd88f5dd700 1 -- 172.21.15.89:0/1498802622 >> 172.21.15.89:0/1498802622 conn(0x7fd8880999f0 msgr2=0x7fd8880b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.072+0000 7fd88f5dd700 1 -- 172.21.15.89:0/1498802622 shutdown_connections 2023-10-10T20:42:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.072+0000 7fd88f5dd700 1 -- 172.21.15.89:0/1498802622 wait complete. 2023-10-10T20:42:14.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.073+0000 7fd88f5dd700 1 Processor -- start 2023-10-10T20:42:14.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.073+0000 7fd88f5dd700 1 -- start start 2023-10-10T20:42:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.074+0000 7fd88f5dd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8880b55a0 0x7fd8881506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.074+0000 7fd88f5dd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd888159ec0 0x7fd888158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.074+0000 7fd88f5dd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8881589a0 0x7fd8880695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.074+0000 7fd88f5dd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd888158e40 con 0x7fd8881589a0 2023-10-10T20:42:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.074+0000 7fd88f5dd700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8881592a0 con 0x7fd8880b55a0 2023-10-10T20:42:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.074+0000 7fd88f5dd700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd888155c10 con 0x7fd888159ec0 2023-10-10T20:42:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.074+0000 7fd88db7a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8881589a0 0x7fd8880695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.074+0000 7fd88db7a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8881589a0 0x7fd8880695d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54740/0 (socket says 172.21.15.89:54740) 2023-10-10T20:42:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.074+0000 7fd88db7a700 1 -- 172.21.15.89:0/1126561521 learned_addr learned my addr 172.21.15.89:0/1126561521 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.075+0000 7fd88cb78700 1 --2- 172.21.15.89:0/1126561521 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd888159ec0 0x7fd888158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.075+0000 7fd88d379700 1 --2- 172.21.15.89:0/1126561521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8880b55a0 0x7fd8881506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.075+0000 7fd88db7a700 1 -- 172.21.15.89:0/1126561521 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd888159ec0 msgr2=0x7fd888158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.075+0000 7fd88db7a700 1 --2- 172.21.15.89:0/1126561521 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd888159ec0 0x7fd888158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.075+0000 7fd88db7a700 1 -- 172.21.15.89:0/1126561521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8880b55a0 msgr2=0x7fd8881506b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.075+0000 7fd88db7a700 1 --2- 172.21.15.89:0/1126561521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8880b55a0 0x7fd8881506b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.075+0000 7fd88db7a700 1 -- 172.21.15.89:0/1126561521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd878009420 con 0x7fd8881589a0 2023-10-10T20:42:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.075+0000 7fd88db7a700 1 --2- 172.21.15.89:0/1126561521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8881589a0 0x7fd8880695d0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fd88000bc60 tx=0x7fd880009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.075+0000 7fd87d7fa700 1 -- 172.21.15.89:0/1126561521 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88000c9c0 con 0x7fd8881589a0 2023-10-10T20:42:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.075+0000 7fd87d7fa700 1 -- 172.21.15.89:0/1126561521 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88000cb60 con 0x7fd8881589a0 2023-10-10T20:42:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.075+0000 7fd88f5dd700 1 -- 172.21.15.89:0/1126561521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd888155000 con 0x7fd8881589a0 2023-10-10T20:42:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.075+0000 7fd87d7fa700 1 -- 172.21.15.89:0/1126561521 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880014410 con 0x7fd8881589a0 2023-10-10T20:42:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.075+0000 7fd88f5dd700 1 -- 172.21.15.89:0/1126561521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8881555c0 con 0x7fd8881589a0 2023-10-10T20:42:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.076+0000 7fd88f5dd700 1 -- 172.21.15.89:0/1126561521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd888063c90 con 0x7fd8881589a0 2023-10-10T20:42:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.076+0000 7fd87d7fa700 1 -- 172.21.15.89:0/1126561521 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd880014800 con 0x7fd8881589a0 2023-10-10T20:42:14.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.077+0000 7fd87d7fa700 1 --2- 172.21.15.89:0/1126561521 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd86c067320 0x7fd86c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:14.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.077+0000 7fd87d7fa700 1 -- 172.21.15.89:0/1126561521 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5869+0+0 (secure 0 0 0) 0x7fd880084dd0 con 0x7fd8881589a0 2023-10-10T20:42:14.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.077+0000 7fd88d379700 1 --2- 172.21.15.89:0/1126561521 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd86c067320 0x7fd86c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:14.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.077+0000 7fd88d379700 1 --2- 172.21.15.89:0/1126561521 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd86c067320 0x7fd86c0697d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd884000ef0 tx=0x7fd884009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:14.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.081+0000 7fd87d7fa700 1 -- 172.21.15.89:0/1126561521 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd88004c5a0 con 0x7fd8881589a0 2023-10-10T20:42:14.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:14.266+0000 7fd88f5dd700 1 -- 172.21.15.89:0/1126561521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000887918.meta", "pool2": "cephfs.volume_0000000000887918.meta", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd8880718b0 con 0x7fd8881589a0 2023-10-10T20:42:15.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.031+0000 7fd87d7fa700 1 -- 172.21.15.89:0/1126561521 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000887918.meta", "pool2": "cephfs.volume_0000000000887918.meta", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000887918.meta' removed v143) v1 ==== 241+0+0 (secure 0 0 0) 0x7fd880051980 con 0x7fd8881589a0 2023-10-10T20:42:15.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.033+0000 7fd88f5dd700 1 -- 172.21.15.89:0/1126561521 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd86c067320 msgr2=0x7fd86c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:15.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.033+0000 7fd88f5dd700 1 --2- 172.21.15.89:0/1126561521 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd86c067320 0x7fd86c0697d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd884000ef0 tx=0x7fd884009040 comp rx=0 tx=0).stop 2023-10-10T20:42:15.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.033+0000 7fd88f5dd700 1 -- 172.21.15.89:0/1126561521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8881589a0 msgr2=0x7fd8880695d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:15.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.033+0000 7fd88f5dd700 1 --2- 172.21.15.89:0/1126561521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8881589a0 0x7fd8880695d0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fd88000bc60 tx=0x7fd880009820 comp rx=0 tx=0).stop 2023-10-10T20:42:15.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.033+0000 7fd88f5dd700 1 -- 172.21.15.89:0/1126561521 shutdown_connections 2023-10-10T20:42:15.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.033+0000 7fd88f5dd700 1 --2- 172.21.15.89:0/1126561521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8881589a0 0x7fd8880695d0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:15.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.033+0000 7fd88f5dd700 1 --2- 172.21.15.89:0/1126561521 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd86c067320 0x7fd86c0697d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:15.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.033+0000 7fd88f5dd700 1 --2- 172.21.15.89:0/1126561521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8880b55a0 0x7fd8881506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:15.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.033+0000 7fd88f5dd700 1 --2- 172.21.15.89:0/1126561521 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd888159ec0 0x7fd888158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:15.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.033+0000 7fd88f5dd700 1 -- 172.21.15.89:0/1126561521 >> 172.21.15.89:0/1126561521 conn(0x7fd8880999f0 msgr2=0x7fd8881525b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:15.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.033+0000 7fd88f5dd700 1 -- 172.21.15.89:0/1126561521 shutdown_connections 2023-10-10T20:42:15.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.033+0000 7fd88f5dd700 1 -- 172.21.15.89:0/1126561521 wait complete. 2023-10-10T20:42:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs.volume_0000000000887918.meta' removed 2023-10-10T20:42:15.047 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000887918.data cephfs.volume_0000000000887918.data --yes-i-really-really-mean-it 2023-10-10T20:42:15.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.164+0000 7f8329bde700 1 Processor -- start 2023-10-10T20:42:15.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.164+0000 7f8329bde700 1 -- start start 2023-10-10T20:42:15.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f8329bde700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83240b55a0 0x7f8324164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:15.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f8329bde700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83240cfa60 con 0x7f83240cd910 2023-10-10T20:42:15.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f8329bde700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83240b5b00 con 0x7f8324159ec0 2023-10-10T20:42:15.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f8329bde700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83240b5da0 con 0x7f83240b55a0 2023-10-10T20:42:15.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f8322ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f83240cd910 0x7f83240ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58812/0 (socket says 172.21.15.89:58812) 2023-10-10T20:42:15.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f8322ffd700 1 -- 172.21.15.89:0/4171784447 learned_addr learned my addr 172.21.15.89:0/4171784447 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:15.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f8323fff700 1 --2- 172.21.15.89:0/4171784447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83240b55a0 0x7f8324164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:15.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f83227fc700 1 -- 172.21.15.89:0/4171784447 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406067586 0 0) 0x7f83240cfa60 con 0x7f83240cd910 2023-10-10T20:42:15.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f83227fc700 1 -- 172.21.15.89:0/4171784447 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f830c003570 con 0x7f83240cd910 2023-10-10T20:42:15.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f83227fc700 1 -- 172.21.15.89:0/4171784447 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2471993459 0 0) 0x7f830c003570 con 0x7f83240cd910 2023-10-10T20:42:15.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f83227fc700 1 -- 172.21.15.89:0/4171784447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83240b55a0 msgr2=0x7f8324164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:15.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f83227fc700 1 --2- 172.21.15.89:0/4171784447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83240b55a0 0x7f8324164400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:15.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f83227fc700 1 -- 172.21.15.89:0/4171784447 >> v1:172.21.15.111:6789/0 conn(0x7f8324159ec0 legacy=0x7f83240b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:15.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f83227fc700 1 -- 172.21.15.89:0/4171784447 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83240b6080 con 0x7f83240cd910 2023-10-10T20:42:15.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f83227fc700 1 -- 172.21.15.89:0/4171784447 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f8318001470 con 0x7f83240cd910 2023-10-10T20:42:15.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f83227fc700 1 -- 172.21.15.89:0/4171784447 >> v1:172.21.15.89:6789/0 conn(0x7f83240cd910 legacy=0x7f83240ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:15.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f83227fc700 1 --2- 172.21.15.89:0/4171784447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f830c003a90 0x7f830c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:15.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f83227fc700 1 --2- 172.21.15.89:0/4171784447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f830c004810 0x7f830c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:15.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f83227fc700 1 --2- 172.21.15.89:0/4171784447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83240b55a0 0x7f830c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:15.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f83227fc700 1 -- 172.21.15.89:0/4171784447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83240b5da0 con 0x7f830c003a90 2023-10-10T20:42:15.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f83227fc700 1 -- 172.21.15.89:0/4171784447 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c00bc20 con 0x7f830c004810 2023-10-10T20:42:15.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.165+0000 7f83227fc700 1 -- 172.21.15.89:0/4171784447 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830c00bde0 con 0x7f83240b55a0 2023-10-10T20:42:15.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.166+0000 7f83237fe700 1 --2- 172.21.15.89:0/4171784447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f830c003a90 0x7f830c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:15.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.166+0000 7f8322ffd700 1 --2- 172.21.15.89:0/4171784447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f830c004810 0x7f830c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:15.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.166+0000 7f8323fff700 1 --2- 172.21.15.89:0/4171784447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83240b55a0 0x7f830c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:15.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.166+0000 7f83237fe700 1 -- 172.21.15.89:0/4171784447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83240b55a0 msgr2=0x7f830c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:15.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.166+0000 7f83237fe700 1 --2- 172.21.15.89:0/4171784447 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83240b55a0 0x7f830c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:15.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.166+0000 7f83237fe700 1 -- 172.21.15.89:0/4171784447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f830c004810 msgr2=0x7f830c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:15.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.166+0000 7f83237fe700 1 --2- 172.21.15.89:0/4171784447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f830c004810 0x7f830c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:15.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.166+0000 7f83237fe700 1 -- 172.21.15.89:0/4171784447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830c003570 con 0x7f830c003a90 2023-10-10T20:42:15.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.166+0000 7f83237fe700 1 --2- 172.21.15.89:0/4171784447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f830c003a90 0x7f830c003ea0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f831400c2d0 tx=0x7f8314001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e7c4f0ccc69bdd server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:15.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.166+0000 7f83227fc700 1 -- 172.21.15.89:0/4171784447 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8314001d80 con 0x7f830c003a90 2023-10-10T20:42:15.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.166+0000 7f83227fc700 1 -- 172.21.15.89:0/4171784447 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831400cba0 con 0x7f830c003a90 2023-10-10T20:42:15.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.166+0000 7f83227fc700 1 -- 172.21.15.89:0/4171784447 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8314012400 con 0x7f830c003a90 2023-10-10T20:42:15.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.167+0000 7f8329bde700 1 -- 172.21.15.89:0/4171784447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f830c003a90 msgr2=0x7f830c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:15.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.167+0000 7f8329bde700 1 --2- 172.21.15.89:0/4171784447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f830c003a90 0x7f830c003ea0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f831400c2d0 tx=0x7f8314001a70 comp rx=0 tx=0).stop 2023-10-10T20:42:15.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.167+0000 7f8323fff700 1 -- 172.21.15.89:0/4171784447 reap_dead start 2023-10-10T20:42:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.167+0000 7f8329bde700 1 -- 172.21.15.89:0/4171784447 shutdown_connections 2023-10-10T20:42:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.167+0000 7f8329bde700 1 -- 172.21.15.89:0/4171784447 >> 172.21.15.89:0/4171784447 conn(0x7f83240999f0 msgr2=0x7f83240b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.167+0000 7f8329bde700 1 -- 172.21.15.89:0/4171784447 shutdown_connections 2023-10-10T20:42:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.167+0000 7f8329bde700 1 -- 172.21.15.89:0/4171784447 wait complete. 2023-10-10T20:42:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.168+0000 7f8329bde700 1 Processor -- start 2023-10-10T20:42:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.168+0000 7f8329bde700 1 -- start start 2023-10-10T20:42:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f8329bde700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f830c003a90 0x7f83241570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f8329bde700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f832409a460 0x7f832409a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f8329bde700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83240b3530 0x7f83240b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f8329bde700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83240b3ee0 con 0x7f830c003a90 2023-10-10T20:42:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f8329bde700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8324169ac0 con 0x7f832409a460 2023-10-10T20:42:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f8329bde700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8324169d90 con 0x7f83240b3530 2023-10-10T20:42:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f83237fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f830c003a90 0x7f83241570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f83237fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f830c003a90 0x7f83241570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54760/0 (socket says 172.21.15.89:54760) 2023-10-10T20:42:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f83237fe700 1 -- 172.21.15.89:0/244766673 learned_addr learned my addr 172.21.15.89:0/244766673 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f8323fff700 1 --2- 172.21.15.89:0/244766673 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83240b3530 0x7f83240b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f8322ffd700 1 --2- 172.21.15.89:0/244766673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f832409a460 0x7f832409a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f83237fe700 1 -- 172.21.15.89:0/244766673 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83240b3530 msgr2=0x7f83240b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f83237fe700 1 --2- 172.21.15.89:0/244766673 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83240b3530 0x7f83240b39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f83237fe700 1 -- 172.21.15.89:0/244766673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f832409a460 msgr2=0x7f832409a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f83237fe700 1 --2- 172.21.15.89:0/244766673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f832409a460 0x7f832409a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f83237fe700 1 -- 172.21.15.89:0/244766673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832406b1e0 con 0x7f830c003a90 2023-10-10T20:42:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f83237fe700 1 --2- 172.21.15.89:0/244766673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f830c003a90 0x7f83241570d0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f8314012fd0 tx=0x7f8314016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f830b7fe700 1 -- 172.21.15.89:0/244766673 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8314027470 con 0x7f830c003a90 2023-10-10T20:42:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f830b7fe700 1 -- 172.21.15.89:0/244766673 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83140275d0 con 0x7f830c003a90 2023-10-10T20:42:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f8329bde700 1 -- 172.21.15.89:0/244766673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832409afd0 con 0x7f830c003a90 2023-10-10T20:42:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f830b7fe700 1 -- 172.21.15.89:0/244766673 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8314027990 con 0x7f830c003a90 2023-10-10T20:42:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.169+0000 7f8329bde700 1 -- 172.21.15.89:0/244766673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832409b590 con 0x7f830c003a90 2023-10-10T20:42:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.170+0000 7f8329bde700 1 -- 172.21.15.89:0/244766673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8324063c90 con 0x7f830c003a90 2023-10-10T20:42:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.171+0000 7f830b7fe700 1 -- 172.21.15.89:0/244766673 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8314024070 con 0x7f830c003a90 2023-10-10T20:42:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.171+0000 7f830b7fe700 1 --2- 172.21.15.89:0/244766673 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8310067320 0x7f83100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.171+0000 7f830b7fe700 1 -- 172.21.15.89:0/244766673 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5394+0+0 (secure 0 0 0) 0x7f8314087090 con 0x7f830c003a90 2023-10-10T20:42:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.171+0000 7f8322ffd700 1 --2- 172.21.15.89:0/244766673 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8310067320 0x7f83100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.172+0000 7f8322ffd700 1 --2- 172.21.15.89:0/244766673 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8310067320 0x7f83100697d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f83180010e0 tx=0x7f8318001e50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.175+0000 7f830b7fe700 1 -- 172.21.15.89:0/244766673 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8314053f60 con 0x7f830c003a90 2023-10-10T20:42:15.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:15.357+0000 7f8329bde700 1 -- 172.21.15.89:0/244766673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000887918.data", "pool2": "cephfs.volume_0000000000887918.data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f83240718b0 con 0x7f830c003a90 2023-10-10T20:42:16.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.032+0000 7f830b7fe700 1 -- 172.21.15.89:0/244766673 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000887918.data", "pool2": "cephfs.volume_0000000000887918.data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000887918.data' removed v144) v1 ==== 241+0+0 (secure 0 0 0) 0x7f8314057580 con 0x7f830c003a90 2023-10-10T20:42:16.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.035+0000 7f8329bde700 1 -- 172.21.15.89:0/244766673 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8310067320 msgr2=0x7f83100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:16.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.035+0000 7f8329bde700 1 --2- 172.21.15.89:0/244766673 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8310067320 0x7f83100697d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f83180010e0 tx=0x7f8318001e50 comp rx=0 tx=0).stop 2023-10-10T20:42:16.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.035+0000 7f8329bde700 1 -- 172.21.15.89:0/244766673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f830c003a90 msgr2=0x7f83241570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:16.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.035+0000 7f8329bde700 1 --2- 172.21.15.89:0/244766673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f830c003a90 0x7f83241570d0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f8314012fd0 tx=0x7f8314016000 comp rx=0 tx=0).stop 2023-10-10T20:42:16.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.035+0000 7f8329bde700 1 -- 172.21.15.89:0/244766673 shutdown_connections 2023-10-10T20:42:16.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.035+0000 7f8329bde700 1 --2- 172.21.15.89:0/244766673 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f83240b3530 0x7f83240b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.035+0000 7f8329bde700 1 --2- 172.21.15.89:0/244766673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f830c003a90 0x7f83241570d0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.035+0000 7f8329bde700 1 --2- 172.21.15.89:0/244766673 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8310067320 0x7f83100697d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.035+0000 7f8329bde700 1 --2- 172.21.15.89:0/244766673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f832409a460 0x7f832409a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.035+0000 7f8329bde700 1 -- 172.21.15.89:0/244766673 >> 172.21.15.89:0/244766673 conn(0x7f83240999f0 msgr2=0x7f8324158fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.035+0000 7f8329bde700 1 -- 172.21.15.89:0/244766673 shutdown_connections 2023-10-10T20:42:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.035+0000 7f8329bde700 1 -- 172.21.15.89:0/244766673 wait complete. 2023-10-10T20:42:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs.volume_0000000000887918.data' removed 2023-10-10T20:42:16.051 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:42:16.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.165+0000 7f3bf5a42700 1 Processor -- start 2023-10-10T20:42:16.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.165+0000 7f3bf5a42700 1 -- start start 2023-10-10T20:42:16.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.166+0000 7f3bf5a42700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bf0159890 0x7f3bf015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:16.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.166+0000 7f3bf5a42700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf015a160 con 0x7f3bf0159c60 2023-10-10T20:42:16.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.166+0000 7f3bf5a42700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf00d13a0 con 0x7f3bf0069560 2023-10-10T20:42:16.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.166+0000 7f3bf5a42700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf015e140 con 0x7f3bf0159890 2023-10-10T20:42:16.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.166+0000 7f3be67fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3bf0159c60 0x7f3bf015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58814/0 (socket says 172.21.15.89:58814) 2023-10-10T20:42:16.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.166+0000 7f3be67fc700 1 -- 172.21.15.89:0/2657307700 learned_addr learned my addr 172.21.15.89:0/2657307700 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:16.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.166+0000 7f3beeffd700 1 --2- 172.21.15.89:0/2657307700 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bf0159890 0x7f3bf015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:16.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.166+0000 7f3bee7fc700 1 -- 172.21.15.89:0/2657307700 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 187419832 0 0) 0x7f3bf015a160 con 0x7f3bf0159c60 2023-10-10T20:42:16.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bee7fc700 1 -- 172.21.15.89:0/2657307700 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bdc003570 con 0x7f3bf0159c60 2023-10-10T20:42:16.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bee7fc700 1 -- 172.21.15.89:0/2657307700 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3994930023 0 0) 0x7f3bf00d13a0 con 0x7f3bf0069560 2023-10-10T20:42:16.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bee7fc700 1 -- 172.21.15.89:0/2657307700 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bf015a160 con 0x7f3bf0069560 2023-10-10T20:42:16.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bee7fc700 1 -- 172.21.15.89:0/2657307700 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2211594005 0 0) 0x7f3bdc003570 con 0x7f3bf0159c60 2023-10-10T20:42:16.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bee7fc700 1 -- 172.21.15.89:0/2657307700 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bf0159890 msgr2=0x7f3bf015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:16.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bee7fc700 1 --2- 172.21.15.89:0/2657307700 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bf0159890 0x7f3bf015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bee7fc700 1 -- 172.21.15.89:0/2657307700 >> v1:172.21.15.111:6789/0 conn(0x7f3bf0069560 legacy=0x7f3bf00ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:16.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bee7fc700 1 -- 172.21.15.89:0/2657307700 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf015a3d0 con 0x7f3bf0159c60 2023-10-10T20:42:16.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bee7fc700 1 -- 172.21.15.89:0/2657307700 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f3bd8001470 con 0x7f3bf0159c60 2023-10-10T20:42:16.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bee7fc700 1 -- 172.21.15.89:0/2657307700 >> v1:172.21.15.89:6789/0 conn(0x7f3bf0159c60 legacy=0x7f3bf015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:16.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bee7fc700 1 --2- 172.21.15.89:0/2657307700 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bf0159890 0x7f3bdc003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:16.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bee7fc700 1 --2- 172.21.15.89:0/2657307700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bdc0041d0 0x7f3bdc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:16.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bee7fc700 1 --2- 172.21.15.89:0/2657307700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bdc004d50 0x7f3bdc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:16.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bee7fc700 1 -- 172.21.15.89:0/2657307700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf015e140 con 0x7f3bdc004d50 2023-10-10T20:42:16.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bee7fc700 1 -- 172.21.15.89:0/2657307700 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc014400 con 0x7f3bdc0041d0 2023-10-10T20:42:16.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bee7fc700 1 -- 172.21.15.89:0/2657307700 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc0145c0 con 0x7f3bf0159890 2023-10-10T20:42:16.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bef7fe700 1 --2- 172.21.15.89:0/2657307700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bdc004d50 0x7f3bdc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:16.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3beeffd700 1 --2- 172.21.15.89:0/2657307700 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bf0159890 0x7f3bdc003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:16.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3be67fc700 1 --2- 172.21.15.89:0/2657307700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bdc0041d0 0x7f3bdc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:16.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bef7fe700 1 -- 172.21.15.89:0/2657307700 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bf0159890 msgr2=0x7f3bdc003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:16.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bef7fe700 1 --2- 172.21.15.89:0/2657307700 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bf0159890 0x7f3bdc003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bef7fe700 1 -- 172.21.15.89:0/2657307700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bdc0041d0 msgr2=0x7f3bdc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:16.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bef7fe700 1 --2- 172.21.15.89:0/2657307700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bdc0041d0 0x7f3bdc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bef7fe700 1 -- 172.21.15.89:0/2657307700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc003570 con 0x7f3bdc004d50 2023-10-10T20:42:16.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.167+0000 7f3bef7fe700 1 --2- 172.21.15.89:0/2657307700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bdc004d50 0x7f3bdc0051c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f3be8002120 tx=0x7f3be80017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a14a0c2983a9dc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:16.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.168+0000 7f3bee7fc700 1 -- 172.21.15.89:0/2657307700 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be8001ab0 con 0x7f3bdc004d50 2023-10-10T20:42:16.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.168+0000 7f3bee7fc700 1 -- 172.21.15.89:0/2657307700 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be800c980 con 0x7f3bdc004d50 2023-10-10T20:42:16.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.168+0000 7f3bee7fc700 1 -- 172.21.15.89:0/2657307700 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be800cd20 con 0x7f3bdc004d50 2023-10-10T20:42:16.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.168+0000 7f3bf5a42700 1 -- 172.21.15.89:0/2657307700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bdc004d50 msgr2=0x7f3bdc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:16.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.168+0000 7f3bf5a42700 1 --2- 172.21.15.89:0/2657307700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bdc004d50 0x7f3bdc0051c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f3be8002120 tx=0x7f3be80017a0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.168+0000 7f3bef7fe700 1 -- 172.21.15.89:0/2657307700 reap_dead start 2023-10-10T20:42:16.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.168+0000 7f3bf5a42700 1 -- 172.21.15.89:0/2657307700 shutdown_connections 2023-10-10T20:42:16.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.168+0000 7f3bf5a42700 1 -- 172.21.15.89:0/2657307700 >> 172.21.15.89:0/2657307700 conn(0x7f3bf0068e20 msgr2=0x7f3bf00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:16.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.168+0000 7f3bf5a42700 1 -- 172.21.15.89:0/2657307700 shutdown_connections 2023-10-10T20:42:16.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.168+0000 7f3bf5a42700 1 -- 172.21.15.89:0/2657307700 wait complete. 2023-10-10T20:42:16.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.169+0000 7f3bf5a42700 1 Processor -- start 2023-10-10T20:42:16.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.169+0000 7f3bf5a42700 1 -- start start 2023-10-10T20:42:16.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.170+0000 7f3bf5a42700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bdc004d50 0x7f3bf014dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:16.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.170+0000 7f3bf5a42700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf00c78a0 0x7f3bf014c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:16.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.170+0000 7f3bf5a42700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bf00c7d70 0x7f3bf014cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:16.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.170+0000 7f3bf5a42700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf014d420 con 0x7f3bdc004d50 2023-10-10T20:42:16.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.170+0000 7f3bf5a42700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf0169e60 con 0x7f3bf00c78a0 2023-10-10T20:42:16.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.170+0000 7f3bf5a42700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf01692e0 con 0x7f3bf00c7d70 2023-10-10T20:42:16.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.170+0000 7f3beeffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bdc004d50 0x7f3bf014dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:16.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.170+0000 7f3beeffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bdc004d50 0x7f3bf014dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54774/0 (socket says 172.21.15.89:54774) 2023-10-10T20:42:16.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.170+0000 7f3beeffd700 1 -- 172.21.15.89:0/3496170257 learned_addr learned my addr 172.21.15.89:0/3496170257 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:16.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.171+0000 7f3be67fc700 1 --2- 172.21.15.89:0/3496170257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf00c78a0 0x7f3bf014c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:16.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.171+0000 7f3bef7fe700 1 --2- 172.21.15.89:0/3496170257 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bf00c7d70 0x7f3bf014cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:16.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.171+0000 7f3beeffd700 1 -- 172.21.15.89:0/3496170257 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bf00c7d70 msgr2=0x7f3bf014cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:16.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.171+0000 7f3beeffd700 1 --2- 172.21.15.89:0/3496170257 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bf00c7d70 0x7f3bf014cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.171+0000 7f3beeffd700 1 -- 172.21.15.89:0/3496170257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf00c78a0 msgr2=0x7f3bf014c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:16.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.171+0000 7f3beeffd700 1 --2- 172.21.15.89:0/3496170257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf00c78a0 0x7f3bf014c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.171+0000 7f3beeffd700 1 -- 172.21.15.89:0/3496170257 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf01695c0 con 0x7f3bdc004d50 2023-10-10T20:42:16.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.171+0000 7f3be67fc700 1 --2- 172.21.15.89:0/3496170257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf00c78a0 0x7f3bf014c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:16.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.171+0000 7f3beeffd700 1 --2- 172.21.15.89:0/3496170257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bdc004d50 0x7f3bf014dbd0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f3be0006ac0 tx=0x7f3be00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:16.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.171+0000 7f3be77fe700 1 -- 172.21.15.89:0/3496170257 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be000fa10 con 0x7f3bdc004d50 2023-10-10T20:42:16.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.171+0000 7f3be77fe700 1 -- 172.21.15.89:0/3496170257 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be000fb70 con 0x7f3bdc004d50 2023-10-10T20:42:16.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.171+0000 7f3bf5a42700 1 -- 172.21.15.89:0/3496170257 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf01698a0 con 0x7f3bdc004d50 2023-10-10T20:42:16.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.171+0000 7f3be77fe700 1 -- 172.21.15.89:0/3496170257 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0014440 con 0x7f3bdc004d50 2023-10-10T20:42:16.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.171+0000 7f3bf5a42700 1 -- 172.21.15.89:0/3496170257 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf0168f90 con 0x7f3bdc004d50 2023-10-10T20:42:16.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.172+0000 7f3bf5a42700 1 -- 172.21.15.89:0/3496170257 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf00c9380 con 0x7f3bdc004d50 2023-10-10T20:42:16.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.172+0000 7f3be77fe700 1 -- 172.21.15.89:0/3496170257 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3be0014870 con 0x7f3bdc004d50 2023-10-10T20:42:16.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.173+0000 7f3be77fe700 1 --2- 172.21.15.89:0/3496170257 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3bd0067320 0x7f3bd00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:16.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.173+0000 7f3be77fe700 1 -- 172.21.15.89:0/3496170257 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3be0018030 con 0x7f3bdc004d50 2023-10-10T20:42:16.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.173+0000 7f3be67fc700 1 --2- 172.21.15.89:0/3496170257 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3bd0067320 0x7f3bd00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:16.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.173+0000 7f3be67fc700 1 --2- 172.21.15.89:0/3496170257 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3bd0067320 0x7f3bd00697d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8001bb0 tx=0x7f3bd8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:16.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.177+0000 7f3be77fe700 1 -- 172.21.15.89:0/3496170257 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3be0052400 con 0x7f3bdc004d50 2023-10-10T20:42:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.361+0000 7f3bf5a42700 1 -- 172.21.15.89:0/3496170257 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3bf004c3c0 con 0x7f3bdc004d50 2023-10-10T20:42:16.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.365+0000 7f3be77fe700 1 -- 172.21.15.89:0/3496170257 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3be0055a20 con 0x7f3bdc004d50 2023-10-10T20:42:16.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.367+0000 7f3bf5a42700 1 -- 172.21.15.89:0/3496170257 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3bd0067320 msgr2=0x7f3bd00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:16.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.367+0000 7f3bf5a42700 1 --2- 172.21.15.89:0/3496170257 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3bd0067320 0x7f3bd00697d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8001bb0 tx=0x7f3bd8009040 comp rx=0 tx=0).stop 2023-10-10T20:42:16.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.367+0000 7f3bf5a42700 1 -- 172.21.15.89:0/3496170257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bdc004d50 msgr2=0x7f3bf014dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:16.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.367+0000 7f3bf5a42700 1 --2- 172.21.15.89:0/3496170257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bdc004d50 0x7f3bf014dbd0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f3be0006ac0 tx=0x7f3be00091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.367+0000 7f3bf5a42700 1 -- 172.21.15.89:0/3496170257 shutdown_connections 2023-10-10T20:42:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.367+0000 7f3bf5a42700 1 --2- 172.21.15.89:0/3496170257 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3bf00c7d70 0x7f3bf014cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.367+0000 7f3bf5a42700 1 --2- 172.21.15.89:0/3496170257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bdc004d50 0x7f3bf014dbd0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.367+0000 7f3bf5a42700 1 --2- 172.21.15.89:0/3496170257 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3bd0067320 0x7f3bd00697d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.367+0000 7f3bf5a42700 1 --2- 172.21.15.89:0/3496170257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf00c78a0 0x7f3bf014c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.367+0000 7f3bf5a42700 1 -- 172.21.15.89:0/3496170257 >> 172.21.15.89:0/3496170257 conn(0x7f3bf0068e20 msgr2=0x7f3bf014e800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:16.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.367+0000 7f3bf5a42700 1 -- 172.21.15.89:0/3496170257 shutdown_connections 2023-10-10T20:42:16.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.367+0000 7f3bf5a42700 1 -- 172.21.15.89:0/3496170257 wait complete. 2023-10-10T20:42:16.381 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename' 2023-10-10T20:42:16.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.494+0000 7f4fb38a3700 1 Processor -- start 2023-10-10T20:42:16.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.494+0000 7f4fb38a3700 1 -- start start 2023-10-10T20:42:16.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.494+0000 7f4fb38a3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4fac159c60 0x7f4fac15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:16.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.494+0000 7f4fb38a3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fac15db10 0x7f4fac069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:16.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.494+0000 7f4fb38a3700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fac15a160 con 0x7f4fac159890 2023-10-10T20:42:16.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.494+0000 7f4fb38a3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fac15df50 con 0x7f4fac159c60 2023-10-10T20:42:16.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.494+0000 7f4fb38a3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0d13a0 con 0x7f4fac15db10 2023-10-10T20:42:16.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.494+0000 7f4fb0e3e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4fac159c60 0x7f4fac15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:16.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.495+0000 7f4fb0e3e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4fac159c60 0x7f4fac15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54776/0 (socket says 172.21.15.89:54776) 2023-10-10T20:42:16.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.495+0000 7f4fb0e3e700 1 -- 172.21.15.89:0/4213179915 learned_addr learned my addr 172.21.15.89:0/4213179915 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:16.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.495+0000 7f4fb1e40700 1 --2- 172.21.15.89:0/4213179915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fac15db10 0x7f4fac069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:16.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.495+0000 7f4fb0e3e700 1 -- 172.21.15.89:0/4213179915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fac15db10 msgr2=0x7f4fac069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:16.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.495+0000 7f4fb0e3e700 1 --2- 172.21.15.89:0/4213179915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fac15db10 0x7f4fac069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.495+0000 7f4fb0e3e700 1 -- 172.21.15.89:0/4213179915 >> v1:172.21.15.111:6789/0 conn(0x7f4fac159890 legacy=0x7f4fac15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:16.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.495+0000 7f4fb0e3e700 1 -- 172.21.15.89:0/4213179915 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fac15e140 con 0x7f4fac159c60 2023-10-10T20:42:16.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.495+0000 7f4fb1e40700 1 --2- 172.21.15.89:0/4213179915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fac15db10 0x7f4fac069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:16.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.495+0000 7f4fb0e3e700 1 --2- 172.21.15.89:0/4213179915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fac159c60 0x7f4fac15d5d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c0099b0 tx=0x7f4f9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9125c7ca4a2ac077 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:16.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.495+0000 7f4fa3fff700 1 -- 172.21.15.89:0/4213179915 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c00b820 con 0x7f4fac159c60 2023-10-10T20:42:16.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.495+0000 7f4fa3fff700 1 -- 172.21.15.89:0/4213179915 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9c00b980 con 0x7f4fac159c60 2023-10-10T20:42:16.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.495+0000 7f4fa3fff700 1 -- 172.21.15.89:0/4213179915 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c00bd20 con 0x7f4fac159c60 2023-10-10T20:42:16.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.495+0000 7f4fb38a3700 1 -- 172.21.15.89:0/4213179915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fac159c60 msgr2=0x7f4fac15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:16.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.495+0000 7f4fb38a3700 1 --2- 172.21.15.89:0/4213179915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fac159c60 0x7f4fac15d5d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c0099b0 tx=0x7f4f9c009250 comp rx=0 tx=0).stop 2023-10-10T20:42:16.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.496+0000 7f4fb38a3700 1 -- 172.21.15.89:0/4213179915 shutdown_connections 2023-10-10T20:42:16.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.496+0000 7f4fb38a3700 1 --2- 172.21.15.89:0/4213179915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fac15db10 0x7f4fac069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.496+0000 7f4fb38a3700 1 --2- 172.21.15.89:0/4213179915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fac159c60 0x7f4fac15d5d0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.496+0000 7f4fb38a3700 1 -- 172.21.15.89:0/4213179915 >> 172.21.15.89:0/4213179915 conn(0x7f4fac068e20 msgr2=0x7f4fac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:16.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.496+0000 7f4fb38a3700 1 -- 172.21.15.89:0/4213179915 shutdown_connections 2023-10-10T20:42:16.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.496+0000 7f4fb38a3700 1 -- 172.21.15.89:0/4213179915 wait complete. 2023-10-10T20:42:16.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.497+0000 7f4fb38a3700 1 Processor -- start 2023-10-10T20:42:16.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.497+0000 7f4fb38a3700 1 -- start start 2023-10-10T20:42:16.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fb38a3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fac159890 0x7f4fac0b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:16.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fb38a3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fac159c60 0x7f4fac0cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:16.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fb38a3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fac15db10 0x7f4fac0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:16.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fb38a3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac152440 con 0x7f4fac159c60 2023-10-10T20:42:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fb38a3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac151fb0 con 0x7f4fac159890 2023-10-10T20:42:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fb38a3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac1513a0 con 0x7f4fac15db10 2023-10-10T20:42:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fb0e3e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fac159c60 0x7f4fac0cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fb0e3e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fac159c60 0x7f4fac0cd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54782/0 (socket says 172.21.15.89:54782) 2023-10-10T20:42:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fb0e3e700 1 -- 172.21.15.89:0/4020952122 learned_addr learned my addr 172.21.15.89:0/4020952122 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fb1e40700 1 --2- 172.21.15.89:0/4020952122 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fac15db10 0x7f4fac0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fb163f700 1 --2- 172.21.15.89:0/4020952122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fac159890 0x7f4fac0b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fb0e3e700 1 -- 172.21.15.89:0/4020952122 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fac15db10 msgr2=0x7f4fac0cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fb0e3e700 1 --2- 172.21.15.89:0/4020952122 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fac15db10 0x7f4fac0cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fb0e3e700 1 -- 172.21.15.89:0/4020952122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fac159890 msgr2=0x7f4fac0b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:16.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fb0e3e700 1 --2- 172.21.15.89:0/4020952122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fac159890 0x7f4fac0b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:16.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fb0e3e700 1 -- 172.21.15.89:0/4020952122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9c0094f0 con 0x7f4fac159c60 2023-10-10T20:42:16.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fb0e3e700 1 --2- 172.21.15.89:0/4020952122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fac159c60 0x7f4fac0cd370 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009290 tx=0x7f4f9c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:16.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fa17fa700 1 -- 172.21.15.89:0/4020952122 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c011c10 con 0x7f4fac159c60 2023-10-10T20:42:16.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fa17fa700 1 -- 172.21.15.89:0/4020952122 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9c020070 con 0x7f4fac159c60 2023-10-10T20:42:16.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fb38a3700 1 -- 172.21.15.89:0/4020952122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fac151680 con 0x7f4fac159c60 2023-10-10T20:42:16.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fa17fa700 1 -- 172.21.15.89:0/4020952122 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c01b450 con 0x7f4fac159c60 2023-10-10T20:42:16.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.498+0000 7f4fb38a3700 1 -- 172.21.15.89:0/4020952122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fac151c40 con 0x7f4fac159c60 2023-10-10T20:42:16.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.499+0000 7f4fb38a3700 1 -- 172.21.15.89:0/4020952122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fac152120 con 0x7f4fac159c60 2023-10-10T20:42:16.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.500+0000 7f4fa17fa700 1 -- 172.21.15.89:0/4020952122 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4f9c022030 con 0x7f4fac159c60 2023-10-10T20:42:16.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.500+0000 7f4fa17fa700 1 --2- 172.21.15.89:0/4020952122 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4f90067320 0x7f4f900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:16.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.500+0000 7f4fa17fa700 1 -- 172.21.15.89:0/4020952122 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4f9c086680 con 0x7f4fac159c60 2023-10-10T20:42:16.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.500+0000 7f4fb163f700 1 --2- 172.21.15.89:0/4020952122 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4f90067320 0x7f4f900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:16.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.501+0000 7f4fb163f700 1 --2- 172.21.15.89:0/4020952122 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4f90067320 0x7f4f900697d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8007fd0 tx=0x7f4fa80026a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:16.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.504+0000 7f4fa17fa700 1 -- 172.21.15.89:0/4020952122 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f9c053720 con 0x7f4fac159c60 2023-10-10T20:42:16.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:16.762+0000 7f4fb38a3700 1 -- 172.21.15.89:0/4020952122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]} v 0) v1 -- 0x7f4fac152120 con 0x7f4fac159c60 2023-10-10T20:42:17.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.041+0000 7f4fa17fa700 1 -- 172.21.15.89:0/4020952122 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]}]=0 v383) v1 ==== 135+0+0 (secure 0 0 0) 0x7f4f9c056d40 con 0x7f4fac159c60 2023-10-10T20:42:17.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.043+0000 7f4fb38a3700 1 -- 172.21.15.89:0/4020952122 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4f90067320 msgr2=0x7f4f900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:17.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.043+0000 7f4fb38a3700 1 --2- 172.21.15.89:0/4020952122 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4f90067320 0x7f4f900697d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8007fd0 tx=0x7f4fa80026a0 comp rx=0 tx=0).stop 2023-10-10T20:42:17.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.043+0000 7f4fb38a3700 1 -- 172.21.15.89:0/4020952122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fac159c60 msgr2=0x7f4fac0cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:17.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.043+0000 7f4fb38a3700 1 --2- 172.21.15.89:0/4020952122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fac159c60 0x7f4fac0cd370 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009290 tx=0x7f4f9c016000 comp rx=0 tx=0).stop 2023-10-10T20:42:17.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.043+0000 7f4fb38a3700 1 -- 172.21.15.89:0/4020952122 shutdown_connections 2023-10-10T20:42:17.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.043+0000 7f4fb38a3700 1 --2- 172.21.15.89:0/4020952122 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4fac15db10 0x7f4fac0cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:17.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.043+0000 7f4fb38a3700 1 --2- 172.21.15.89:0/4020952122 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4f90067320 0x7f4f900697d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:17.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.043+0000 7f4fb38a3700 1 --2- 172.21.15.89:0/4020952122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fac159890 0x7f4fac0b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:17.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.043+0000 7f4fb38a3700 1 --2- 172.21.15.89:0/4020952122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fac159c60 0x7f4fac0cd370 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:17.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.043+0000 7f4fb38a3700 1 -- 172.21.15.89:0/4020952122 >> 172.21.15.89:0/4020952122 conn(0x7f4fac068e20 msgr2=0x7f4fac0cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:17.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.043+0000 7f4fb38a3700 1 -- 172.21.15.89:0/4020952122 shutdown_connections 2023-10-10T20:42:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.043+0000 7f4fb38a3700 1 -- 172.21.15.89:0/4020952122 wait complete. 2023-10-10T20:42:17.059 INFO:tasks.cephfs_test_runner:That volume, its file system and pools, can be renamed. ... ok 2023-10-10T20:42:17.059 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename_fails_without_confirmation_flag (tasks.cephfs.test_volumes.TestVolumes) That renaming volume fails without --yes-i-really-mean-it flag. 2023-10-10T20:42:17.059 INFO:tasks.cephfs_test_runner:test_volume_rename_fails_without_confirmation_flag (tasks.cephfs.test_volumes.TestVolumes) 2023-10-10T20:42:17.060 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag' 2023-10-10T20:42:17.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.183+0000 7fb0e83b0700 1 Processor -- start 2023-10-10T20:42:17.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.183+0000 7fb0e83b0700 1 -- start start 2023-10-10T20:42:17.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.184+0000 7fb0e83b0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0e015dba0 0x7fb0e006ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:17.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.184+0000 7fb0e83b0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e00c74b0 con 0x7fb0e015d730 2023-10-10T20:42:17.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.184+0000 7fb0e83b0700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e00c8700 con 0x7fb0e0159b10 2023-10-10T20:42:17.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.184+0000 7fb0e83b0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e00c9070 con 0x7fb0e015dba0 2023-10-10T20:42:17.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.184+0000 7fb0e594b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb0e015d730 0x7fb0e006a410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58830/0 (socket says 172.21.15.89:58830) 2023-10-10T20:42:17.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.184+0000 7fb0e594b700 1 -- 172.21.15.89:0/3596402357 learned_addr learned my addr 172.21.15.89:0/3596402357 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:17.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.184+0000 7fb0e694d700 1 --2- 172.21.15.89:0/3596402357 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0e015dba0 0x7fb0e006ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:17.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.184+0000 7fb0e514a700 1 -- 172.21.15.89:0/3596402357 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2732501796 0 0) 0x7fb0e00c74b0 con 0x7fb0e015d730 2023-10-10T20:42:17.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.184+0000 7fb0e514a700 1 -- 172.21.15.89:0/3596402357 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0c8003570 con 0x7fb0e015d730 2023-10-10T20:42:17.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.184+0000 7fb0e514a700 1 -- 172.21.15.89:0/3596402357 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507885617 0 0) 0x7fb0e00c8700 con 0x7fb0e0159b10 2023-10-10T20:42:17.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.184+0000 7fb0e514a700 1 -- 172.21.15.89:0/3596402357 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0e00c74b0 con 0x7fb0e0159b10 2023-10-10T20:42:17.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.184+0000 7fb0e514a700 1 -- 172.21.15.89:0/3596402357 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 71752588 0 0) 0x7fb0c8003570 con 0x7fb0e015d730 2023-10-10T20:42:17.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.184+0000 7fb0e514a700 1 -- 172.21.15.89:0/3596402357 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0e015dba0 msgr2=0x7fb0e006ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:17.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.184+0000 7fb0e514a700 1 --2- 172.21.15.89:0/3596402357 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0e015dba0 0x7fb0e006ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:17.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.184+0000 7fb0e514a700 1 -- 172.21.15.89:0/3596402357 >> v1:172.21.15.111:6789/0 conn(0x7fb0e0159b10 legacy=0x7fb0e00cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:17.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.184+0000 7fb0e514a700 1 -- 172.21.15.89:0/3596402357 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e00c80f0 con 0x7fb0e015d730 2023-10-10T20:42:17.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.184+0000 7fb0e514a700 1 -- 172.21.15.89:0/3596402357 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fb0d4001470 con 0x7fb0e015d730 2023-10-10T20:42:17.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.184+0000 7fb0e514a700 1 -- 172.21.15.89:0/3596402357 >> v1:172.21.15.89:6789/0 conn(0x7fb0e015d730 legacy=0x7fb0e006a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:17.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.185+0000 7fb0e514a700 1 --2- 172.21.15.89:0/3596402357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0c8003c20 0x7fb0c8004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:17.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.185+0000 7fb0e514a700 1 --2- 172.21.15.89:0/3596402357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c8004810 0x7fb0c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:17.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.185+0000 7fb0e514a700 1 --2- 172.21.15.89:0/3596402357 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0e015dba0 0x7fb0c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:17.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.185+0000 7fb0e514a700 1 -- 172.21.15.89:0/3596402357 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e00c9070 con 0x7fb0c8003c20 2023-10-10T20:42:17.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.185+0000 7fb0e514a700 1 -- 172.21.15.89:0/3596402357 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c800bb40 con 0x7fb0c8004810 2023-10-10T20:42:17.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.185+0000 7fb0e514a700 1 -- 172.21.15.89:0/3596402357 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c800bd00 con 0x7fb0e015dba0 2023-10-10T20:42:17.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.185+0000 7fb0e614c700 1 --2- 172.21.15.89:0/3596402357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0c8003c20 0x7fb0c8004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:17.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.185+0000 7fb0e594b700 1 --2- 172.21.15.89:0/3596402357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c8004810 0x7fb0c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:17.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.185+0000 7fb0e694d700 1 --2- 172.21.15.89:0/3596402357 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0e015dba0 0x7fb0c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:17.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.185+0000 7fb0e614c700 1 -- 172.21.15.89:0/3596402357 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0e015dba0 msgr2=0x7fb0c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:17.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.185+0000 7fb0e614c700 1 --2- 172.21.15.89:0/3596402357 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0e015dba0 0x7fb0c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:17.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.185+0000 7fb0e614c700 1 -- 172.21.15.89:0/3596402357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c8004810 msgr2=0x7fb0c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:17.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.185+0000 7fb0e614c700 1 --2- 172.21.15.89:0/3596402357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c8004810 0x7fb0c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:17.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.185+0000 7fb0e614c700 1 -- 172.21.15.89:0/3596402357 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c8003570 con 0x7fb0c8003c20 2023-10-10T20:42:17.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.185+0000 7fb0e614c700 1 --2- 172.21.15.89:0/3596402357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0c8003c20 0x7fb0c8004010 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8002120 tx=0x7fb0d80017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf7ddb3cc1c5218 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:17.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.185+0000 7fb0e514a700 1 -- 172.21.15.89:0/3596402357 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8001ab0 con 0x7fb0c8003c20 2023-10-10T20:42:17.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.185+0000 7fb0e514a700 1 -- 172.21.15.89:0/3596402357 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d800c980 con 0x7fb0c8003c20 2023-10-10T20:42:17.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.185+0000 7fb0e514a700 1 -- 172.21.15.89:0/3596402357 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d800cd20 con 0x7fb0c8003c20 2023-10-10T20:42:17.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.186+0000 7fb0e83b0700 1 -- 172.21.15.89:0/3596402357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0c8003c20 msgr2=0x7fb0c8004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:17.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.186+0000 7fb0e83b0700 1 --2- 172.21.15.89:0/3596402357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0c8003c20 0x7fb0c8004010 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8002120 tx=0x7fb0d80017a0 comp rx=0 tx=0).stop 2023-10-10T20:42:17.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.186+0000 7fb0e694d700 1 -- 172.21.15.89:0/3596402357 reap_dead start 2023-10-10T20:42:17.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.186+0000 7fb0e83b0700 1 -- 172.21.15.89:0/3596402357 shutdown_connections 2023-10-10T20:42:17.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.186+0000 7fb0e83b0700 1 -- 172.21.15.89:0/3596402357 >> 172.21.15.89:0/3596402357 conn(0x7fb0e00c9870 msgr2=0x7fb0e00c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:17.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.186+0000 7fb0e83b0700 1 -- 172.21.15.89:0/3596402357 shutdown_connections 2023-10-10T20:42:17.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.186+0000 7fb0e83b0700 1 -- 172.21.15.89:0/3596402357 wait complete. 2023-10-10T20:42:17.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.187+0000 7fb0e83b0700 1 Processor -- start 2023-10-10T20:42:17.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.187+0000 7fb0e83b0700 1 -- start start 2023-10-10T20:42:17.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0e83b0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0c8003c20 0x7fb0e014a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:17.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0e83b0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0e0155cd0 0x7fb0e00c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:17.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0e83b0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0e0068390 0x7fb0e00687e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:17.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0e83b0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e016e290 con 0x7fb0e0155cd0 2023-10-10T20:42:17.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0e83b0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e0068d20 con 0x7fb0e0068390 2023-10-10T20:42:17.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0e83b0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e016d680 con 0x7fb0c8003c20 2023-10-10T20:42:17.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0e594b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0e0155cd0 0x7fb0e00c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:17.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0e594b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0e0155cd0 0x7fb0e00c6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54804/0 (socket says 172.21.15.89:54804) 2023-10-10T20:42:17.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0e594b700 1 -- 172.21.15.89:0/3514782295 learned_addr learned my addr 172.21.15.89:0/3514782295 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:17.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0e614c700 1 --2- 172.21.15.89:0/3514782295 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0c8003c20 0x7fb0e014a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:17.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0e694d700 1 --2- 172.21.15.89:0/3514782295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0e0068390 0x7fb0e00687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:17.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0e594b700 1 -- 172.21.15.89:0/3514782295 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0c8003c20 msgr2=0x7fb0e014a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:17.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0e594b700 1 --2- 172.21.15.89:0/3514782295 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0c8003c20 0x7fb0e014a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:17.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0e594b700 1 -- 172.21.15.89:0/3514782295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0e0068390 msgr2=0x7fb0e00687e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:17.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0e594b700 1 --2- 172.21.15.89:0/3514782295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0e0068390 0x7fb0e00687e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:17.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0e594b700 1 -- 172.21.15.89:0/3514782295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e016d960 con 0x7fb0e0155cd0 2023-10-10T20:42:17.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0e594b700 1 --2- 172.21.15.89:0/3514782295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0e0155cd0 0x7fb0e00c6710 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4007fd0 tx=0x7fb0e00c8670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:17.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0de7fc700 1 -- 172.21.15.89:0/3514782295 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4018030 con 0x7fb0e0155cd0 2023-10-10T20:42:17.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0de7fc700 1 -- 172.21.15.89:0/3514782295 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d400f5f0 con 0x7fb0e0155cd0 2023-10-10T20:42:17.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0e83b0700 1 -- 172.21.15.89:0/3514782295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e016dc40 con 0x7fb0e0155cd0 2023-10-10T20:42:17.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.188+0000 7fb0de7fc700 1 -- 172.21.15.89:0/3514782295 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d400f9b0 con 0x7fb0e0155cd0 2023-10-10T20:42:17.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.189+0000 7fb0e83b0700 1 -- 172.21.15.89:0/3514782295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e014aeb0 con 0x7fb0e0155cd0 2023-10-10T20:42:17.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.189+0000 7fb0e83b0700 1 -- 172.21.15.89:0/3514782295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e004c3c0 con 0x7fb0e0155cd0 2023-10-10T20:42:17.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.190+0000 7fb0de7fc700 1 -- 172.21.15.89:0/3514782295 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb0d401b070 con 0x7fb0e0155cd0 2023-10-10T20:42:17.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.190+0000 7fb0de7fc700 1 --2- 172.21.15.89:0/3514782295 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb0cc067320 0x7fb0cc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:17.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.190+0000 7fb0de7fc700 1 -- 172.21.15.89:0/3514782295 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb0d4019070 con 0x7fb0e0155cd0 2023-10-10T20:42:17.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.190+0000 7fb0e614c700 1 --2- 172.21.15.89:0/3514782295 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb0cc067320 0x7fb0cc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:17.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.191+0000 7fb0e614c700 1 --2- 172.21.15.89:0/3514782295 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb0cc067320 0x7fb0cc0697d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8000f80 tx=0x7fb0d8016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:17.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.194+0000 7fb0de7fc700 1 -- 172.21.15.89:0/3514782295 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0d4051730 con 0x7fb0e0155cd0 2023-10-10T20:42:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:17.455+0000 7fb0e83b0700 1 -- 172.21.15.89:0/3514782295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]} v 0) v1 -- 0x7fb0e016e3d0 con 0x7fb0e0155cd0 2023-10-10T20:42:18.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.043+0000 7fb0de7fc700 1 -- 172.21.15.89:0/3514782295 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]}]=0 v384) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb0e016e3d0 con 0x7fb0e0155cd0 2023-10-10T20:42:18.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.045+0000 7fb0e83b0700 1 -- 172.21.15.89:0/3514782295 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb0cc067320 msgr2=0x7fb0cc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:18.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.045+0000 7fb0e83b0700 1 --2- 172.21.15.89:0/3514782295 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb0cc067320 0x7fb0cc0697d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8000f80 tx=0x7fb0d8016000 comp rx=0 tx=0).stop 2023-10-10T20:42:18.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.045+0000 7fb0e83b0700 1 -- 172.21.15.89:0/3514782295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0e0155cd0 msgr2=0x7fb0e00c6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:18.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.045+0000 7fb0e83b0700 1 --2- 172.21.15.89:0/3514782295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0e0155cd0 0x7fb0e00c6710 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4007fd0 tx=0x7fb0e00c8670 comp rx=0 tx=0).stop 2023-10-10T20:42:18.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.045+0000 7fb0e83b0700 1 -- 172.21.15.89:0/3514782295 shutdown_connections 2023-10-10T20:42:18.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.045+0000 7fb0e83b0700 1 --2- 172.21.15.89:0/3514782295 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb0cc067320 0x7fb0cc0697d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.045+0000 7fb0e83b0700 1 --2- 172.21.15.89:0/3514782295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0e0068390 0x7fb0e00687e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.045+0000 7fb0e83b0700 1 --2- 172.21.15.89:0/3514782295 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0c8003c20 0x7fb0e014a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.045+0000 7fb0e83b0700 1 --2- 172.21.15.89:0/3514782295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0e0155cd0 0x7fb0e00c6710 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.045+0000 7fb0e83b0700 1 -- 172.21.15.89:0/3514782295 >> 172.21.15.89:0/3514782295 conn(0x7fb0e00c9870 msgr2=0x7fb0e0156b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:18.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.045+0000 7fb0e83b0700 1 -- 172.21.15.89:0/3514782295 shutdown_connections 2023-10-10T20:42:18.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.045+0000 7fb0e83b0700 1 -- 172.21.15.89:0/3514782295 wait complete. 2023-10-10T20:42:18.061 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:42:18.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.175+0000 7f1a5fbf4700 1 Processor -- start 2023-10-10T20:42:18.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.175+0000 7f1a5fbf4700 1 -- start start 2023-10-10T20:42:18.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.176+0000 7f1a5fbf4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a58159890 0x7f1a5815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.176+0000 7f1a5fbf4700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1a58159c60 0x7f1a5815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.176+0000 7f1a5fbf4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1a5815db10 0x7f1a58069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.176+0000 7f1a5fbf4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a5815a160 con 0x7f1a5815db10 2023-10-10T20:42:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.176+0000 7f1a5fbf4700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a580d13a0 con 0x7f1a58159c60 2023-10-10T20:42:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.176+0000 7f1a5fbf4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5815e140 con 0x7f1a58159890 2023-10-10T20:42:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.176+0000 7f1a5e191700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1a5815db10 0x7f1a58069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.176+0000 7f1a5e191700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1a5815db10 0x7f1a58069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44460/0 (socket says 172.21.15.89:44460) 2023-10-10T20:42:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.176+0000 7f1a5e191700 1 -- 172.21.15.89:0/3832910222 learned_addr learned my addr 172.21.15.89:0/3832910222 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.176+0000 7f1a5d18f700 1 --2- 172.21.15.89:0/3832910222 >> v2:172.21.15.111:3300/0 conn(0x7f1a58159c60 0x7f1a5815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.176+0000 7f1a5d990700 1 --2- 172.21.15.89:0/3832910222 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a58159890 0x7f1a5815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:18.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.176+0000 7f1a5e191700 1 -- 172.21.15.89:0/3832910222 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a58159890 msgr2=0x7f1a5815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:18.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.176+0000 7f1a5e191700 1 --2- 172.21.15.89:0/3832910222 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a58159890 0x7f1a5815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.176+0000 7f1a5e191700 1 -- 172.21.15.89:0/3832910222 >> v2:172.21.15.111:3300/0 conn(0x7f1a58159c60 msgr2=0x7f1a5815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:18.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.176+0000 7f1a5e191700 1 --2- 172.21.15.89:0/3832910222 >> v2:172.21.15.111:3300/0 conn(0x7f1a58159c60 0x7f1a5815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.176+0000 7f1a5e191700 1 -- 172.21.15.89:0/3832910222 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5815a3d0 con 0x7f1a5815db10 2023-10-10T20:42:18.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.176+0000 7f1a5d18f700 1 --2- 172.21.15.89:0/3832910222 >> v2:172.21.15.111:3300/0 conn(0x7f1a58159c60 0x7f1a5815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:18.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.176+0000 7f1a5e191700 1 --2- 172.21.15.89:0/3832910222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a5815db10 0x7f1a58069560 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f1a580650f0 tx=0x7f1a50009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511d35b4e4ee7f1f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:18.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.176+0000 7f1a5c98e700 1 -- 172.21.15.89:0/3832910222 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5000e070 con 0x7f1a5815db10 2023-10-10T20:42:18.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.177+0000 7f1a5c98e700 1 -- 172.21.15.89:0/3832910222 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5000ba50 con 0x7f1a5815db10 2023-10-10T20:42:18.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.177+0000 7f1a5c98e700 1 -- 172.21.15.89:0/3832910222 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a500092e0 con 0x7f1a5815db10 2023-10-10T20:42:18.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.177+0000 7f1a5fbf4700 1 -- 172.21.15.89:0/3832910222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a5815db10 msgr2=0x7f1a58069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:18.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.177+0000 7f1a5fbf4700 1 --2- 172.21.15.89:0/3832910222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a5815db10 0x7f1a58069560 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f1a580650f0 tx=0x7f1a50009840 comp rx=0 tx=0).stop 2023-10-10T20:42:18.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.177+0000 7f1a5fbf4700 1 -- 172.21.15.89:0/3832910222 shutdown_connections 2023-10-10T20:42:18.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.177+0000 7f1a5fbf4700 1 --2- 172.21.15.89:0/3832910222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a5815db10 0x7f1a58069560 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.177+0000 7f1a5fbf4700 1 --2- 172.21.15.89:0/3832910222 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a58159890 0x7f1a5815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.177+0000 7f1a5fbf4700 1 --2- 172.21.15.89:0/3832910222 >> v2:172.21.15.111:3300/0 conn(0x7f1a58159c60 0x7f1a5815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.177+0000 7f1a5fbf4700 1 -- 172.21.15.89:0/3832910222 >> 172.21.15.89:0/3832910222 conn(0x7f1a58068e20 msgr2=0x7f1a580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:18.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.177+0000 7f1a5fbf4700 1 -- 172.21.15.89:0/3832910222 shutdown_connections 2023-10-10T20:42:18.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.177+0000 7f1a5fbf4700 1 -- 172.21.15.89:0/3832910222 wait complete. 2023-10-10T20:42:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.178+0000 7f1a5fbf4700 1 Processor -- start 2023-10-10T20:42:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.178+0000 7f1a5fbf4700 1 -- start start 2023-10-10T20:42:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5fbf4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a58159890 0x7f1a5816daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5fbf4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a58159c60 0x7f1a580b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5fbf4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a5815db10 0x7f1a580cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5fbf4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5816d610 con 0x7f1a58159890 2023-10-10T20:42:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5fbf4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a580b5860 con 0x7f1a58159c60 2023-10-10T20:42:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5fbf4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a580cdb10 con 0x7f1a5815db10 2023-10-10T20:42:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5d990700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a58159890 0x7f1a5816daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5d990700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a58159890 0x7f1a5816daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44476/0 (socket says 172.21.15.89:44476) 2023-10-10T20:42:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5d990700 1 -- 172.21.15.89:0/2039484727 learned_addr learned my addr 172.21.15.89:0/2039484727 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5d18f700 1 --2- 172.21.15.89:0/2039484727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a58159c60 0x7f1a580b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5e191700 1 --2- 172.21.15.89:0/2039484727 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a5815db10 0x7f1a580cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5d990700 1 -- 172.21.15.89:0/2039484727 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a5815db10 msgr2=0x7f1a580cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5d990700 1 --2- 172.21.15.89:0/2039484727 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a5815db10 0x7f1a580cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5d990700 1 -- 172.21.15.89:0/2039484727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a58159c60 msgr2=0x7f1a580b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:18.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5d990700 1 --2- 172.21.15.89:0/2039484727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a58159c60 0x7f1a580b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5d990700 1 -- 172.21.15.89:0/2039484727 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a50009ae0 con 0x7f1a58159890 2023-10-10T20:42:18.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5e191700 1 --2- 172.21.15.89:0/2039484727 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a5815db10 0x7f1a580cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:18.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5d18f700 1 --2- 172.21.15.89:0/2039484727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a58159c60 0x7f1a580b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5d990700 1 --2- 172.21.15.89:0/2039484727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a58159890 0x7f1a5816daa0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400d8c0 tx=0x7f1a54008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a4dffb700 1 -- 172.21.15.89:0/2039484727 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5400de60 con 0x7f1a58159890 2023-10-10T20:42:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a4dffb700 1 -- 172.21.15.89:0/2039484727 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5400b780 con 0x7f1a58159890 2023-10-10T20:42:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a4dffb700 1 -- 172.21.15.89:0/2039484727 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5400bb20 con 0x7f1a58159890 2023-10-10T20:42:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.179+0000 7f1a5fbf4700 1 -- 172.21.15.89:0/2039484727 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a580cddf0 con 0x7f1a58159890 2023-10-10T20:42:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.180+0000 7f1a5fbf4700 1 -- 172.21.15.89:0/2039484727 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5816d2a0 con 0x7f1a58159890 2023-10-10T20:42:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.181+0000 7f1a5fbf4700 1 -- 172.21.15.89:0/2039484727 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a5804c3c0 con 0x7f1a58159890 2023-10-10T20:42:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.181+0000 7f1a4dffb700 1 -- 172.21.15.89:0/2039484727 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1a5401e460 con 0x7f1a58159890 2023-10-10T20:42:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.181+0000 7f1a4dffb700 1 --2- 172.21.15.89:0/2039484727 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1a3c067320 0x7f1a3c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:18.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.181+0000 7f1a4dffb700 1 -- 172.21.15.89:0/2039484727 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1a54085050 con 0x7f1a58159890 2023-10-10T20:42:18.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.181+0000 7f1a5d18f700 1 --2- 172.21.15.89:0/2039484727 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1a3c067320 0x7f1a3c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:18.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.182+0000 7f1a5d18f700 1 --2- 172.21.15.89:0/2039484727 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1a3c067320 0x7f1a3c0697d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1a48006f00 tx=0x7f1a48008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:18.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.186+0000 7f1a4dffb700 1 -- 172.21.15.89:0/2039484727 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a54051fa0 con 0x7f1a58159890 2023-10-10T20:42:18.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.367+0000 7f1a5fbf4700 1 -- 172.21.15.89:0/2039484727 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1a5806a5b0 con 0x7f1a58159890 2023-10-10T20:42:18.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.374+0000 7f1a4dffb700 1 -- 172.21.15.89:0/2039484727 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1a540555c0 con 0x7f1a58159890 2023-10-10T20:42:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.376+0000 7f1a5fbf4700 1 -- 172.21.15.89:0/2039484727 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1a3c067320 msgr2=0x7f1a3c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.376+0000 7f1a5fbf4700 1 --2- 172.21.15.89:0/2039484727 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1a3c067320 0x7f1a3c0697d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1a48006f00 tx=0x7f1a48008040 comp rx=0 tx=0).stop 2023-10-10T20:42:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.376+0000 7f1a5fbf4700 1 -- 172.21.15.89:0/2039484727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a58159890 msgr2=0x7f1a5816daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.376+0000 7f1a5fbf4700 1 --2- 172.21.15.89:0/2039484727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a58159890 0x7f1a5816daa0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400d8c0 tx=0x7f1a54008040 comp rx=0 tx=0).stop 2023-10-10T20:42:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.376+0000 7f1a5fbf4700 1 -- 172.21.15.89:0/2039484727 shutdown_connections 2023-10-10T20:42:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.376+0000 7f1a5fbf4700 1 --2- 172.21.15.89:0/2039484727 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1a5815db10 0x7f1a580cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.377+0000 7f1a5fbf4700 1 --2- 172.21.15.89:0/2039484727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a58159890 0x7f1a5816daa0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.377+0000 7f1a5fbf4700 1 --2- 172.21.15.89:0/2039484727 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1a3c067320 0x7f1a3c0697d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.377+0000 7f1a5fbf4700 1 --2- 172.21.15.89:0/2039484727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a58159c60 0x7f1a580b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.377+0000 7f1a5fbf4700 1 -- 172.21.15.89:0/2039484727 >> 172.21.15.89:0/2039484727 conn(0x7f1a58068e20 msgr2=0x7f1a5815be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:18.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.377+0000 7f1a5fbf4700 1 -- 172.21.15.89:0/2039484727 shutdown_connections 2023-10-10T20:42:18.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.377+0000 7f1a5fbf4700 1 -- 172.21.15.89:0/2039484727 wait complete. 2023-10-10T20:42:18.391 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:42:18.392 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:42:18.434 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:42:18.479 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:42:18.479 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:42:18.480 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:42:18.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.591+0000 7fcd8193f700 1 Processor -- start 2023-10-10T20:42:18.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.591+0000 7fcd8193f700 1 -- start start 2023-10-10T20:42:18.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.592+0000 7fcd8193f700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcd7c1574d0 0x7fcd7c0cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:18.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.592+0000 7fcd8193f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd7c0cc3c0 0x7fcd7c06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:18.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.592+0000 7fcd8193f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7c06c380 con 0x7fcd7c06bf20 2023-10-10T20:42:18.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.592+0000 7fcd8193f700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7c0b16a0 con 0x7fcd7c1574d0 2023-10-10T20:42:18.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.592+0000 7fcd8193f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c0b1c60 con 0x7fcd7c0cc3c0 2023-10-10T20:42:18.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.592+0000 7fcd8113e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcd7c06bf20 0x7fcd7c06cf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48884/0 (socket says 172.21.15.89:48884) 2023-10-10T20:42:18.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.592+0000 7fcd8113e700 1 -- 172.21.15.89:0/4090555958 learned_addr learned my addr 172.21.15.89:0/4090555958 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:18.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7b7fe700 1 -- 172.21.15.89:0/4090555958 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3804613415 0 0) 0x7fcd74001490 con 0x7fcd7c06bf20 2023-10-10T20:42:18.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd8093d700 1 --2- 172.21.15.89:0/4090555958 >> v2:172.21.15.111:3300/0 conn(0x7fcd7c1574d0 0x7fcd7c0cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:18.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7b7fe700 1 -- 172.21.15.89:0/4090555958 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd6c003570 con 0x7fcd7c06bf20 2023-10-10T20:42:18.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7bfff700 1 --2- 172.21.15.89:0/4090555958 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd7c0cc3c0 0x7fcd7c06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:18.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7b7fe700 1 -- 172.21.15.89:0/4090555958 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 976955424 0 0) 0x7fcd6c003570 con 0x7fcd7c06bf20 2023-10-10T20:42:18.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7b7fe700 1 -- 172.21.15.89:0/4090555958 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd7c0cc3c0 msgr2=0x7fcd7c06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:18.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7b7fe700 1 --2- 172.21.15.89:0/4090555958 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd7c0cc3c0 0x7fcd7c06b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7b7fe700 1 -- 172.21.15.89:0/4090555958 >> v2:172.21.15.111:3300/0 conn(0x7fcd7c1574d0 msgr2=0x7fcd7c0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:18.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7b7fe700 1 --2- 172.21.15.89:0/4090555958 >> v2:172.21.15.111:3300/0 conn(0x7fcd7c1574d0 0x7fcd7c0cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7b7fe700 1 -- 172.21.15.89:0/4090555958 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c1578a0 con 0x7fcd7c06bf20 2023-10-10T20:42:18.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7b7fe700 1 -- 172.21.15.89:0/4090555958 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fcd74001980 con 0x7fcd7c06bf20 2023-10-10T20:42:18.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7b7fe700 1 -- 172.21.15.89:0/4090555958 >> v1:172.21.15.89:6789/0 conn(0x7fcd7c06bf20 legacy=0x7fcd7c06cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:18.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7b7fe700 1 --2- 172.21.15.89:0/4090555958 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd7c0cc3c0 0x7fcd6c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:18.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7b7fe700 1 --2- 172.21.15.89:0/4090555958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd6c003fe0 0x7fcd6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:18.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7b7fe700 1 --2- 172.21.15.89:0/4090555958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd6c004d50 0x7fcd6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:18.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7b7fe700 1 -- 172.21.15.89:0/4090555958 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c0b16a0 con 0x7fcd6c003fe0 2023-10-10T20:42:18.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7b7fe700 1 -- 172.21.15.89:0/4090555958 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c0b1c60 con 0x7fcd6c004d50 2023-10-10T20:42:18.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7b7fe700 1 -- 172.21.15.89:0/4090555958 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c00ba90 con 0x7fcd7c0cc3c0 2023-10-10T20:42:18.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7bfff700 1 --2- 172.21.15.89:0/4090555958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd6c003fe0 0x7fcd6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:18.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd8093d700 1 --2- 172.21.15.89:0/4090555958 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd7c0cc3c0 0x7fcd6c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:18.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd8113e700 1 --2- 172.21.15.89:0/4090555958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd6c004d50 0x7fcd6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:18.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7bfff700 1 -- 172.21.15.89:0/4090555958 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd7c0cc3c0 msgr2=0x7fcd6c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:18.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7bfff700 1 --2- 172.21.15.89:0/4090555958 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd7c0cc3c0 0x7fcd6c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7bfff700 1 -- 172.21.15.89:0/4090555958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd6c004d50 msgr2=0x7fcd6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:18.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7bfff700 1 --2- 172.21.15.89:0/4090555958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd6c004d50 0x7fcd6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.593+0000 7fcd7bfff700 1 -- 172.21.15.89:0/4090555958 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c003570 con 0x7fcd6c003fe0 2023-10-10T20:42:18.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.594+0000 7fcd7bfff700 1 --2- 172.21.15.89:0/4090555958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd6c003fe0 0x7fcd6c004810 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009c00 tx=0x7fcd680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314d8ab2ee64669d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:18.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.594+0000 7fcd7b7fe700 1 -- 172.21.15.89:0/4090555958 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6800e6b0 con 0x7fcd6c003fe0 2023-10-10T20:42:18.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.594+0000 7fcd7b7fe700 1 -- 172.21.15.89:0/4090555958 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6800e810 con 0x7fcd6c003fe0 2023-10-10T20:42:18.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.594+0000 7fcd7b7fe700 1 -- 172.21.15.89:0/4090555958 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6800ebd0 con 0x7fcd6c003fe0 2023-10-10T20:42:18.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.594+0000 7fcd8193f700 1 -- 172.21.15.89:0/4090555958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd6c003fe0 msgr2=0x7fcd6c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:18.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.594+0000 7fcd8193f700 1 --2- 172.21.15.89:0/4090555958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd6c003fe0 0x7fcd6c004810 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009c00 tx=0x7fcd680091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.594+0000 7fcd8113e700 1 -- 172.21.15.89:0/4090555958 reap_dead start 2023-10-10T20:42:18.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.594+0000 7fcd8193f700 1 -- 172.21.15.89:0/4090555958 shutdown_connections 2023-10-10T20:42:18.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.594+0000 7fcd8193f700 1 -- 172.21.15.89:0/4090555958 >> 172.21.15.89:0/4090555958 conn(0x7fcd7c0ce430 msgr2=0x7fcd7c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:18.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.594+0000 7fcd8193f700 1 -- 172.21.15.89:0/4090555958 shutdown_connections 2023-10-10T20:42:18.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.594+0000 7fcd8193f700 1 -- 172.21.15.89:0/4090555958 wait complete. 2023-10-10T20:42:18.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.595+0000 7fcd8193f700 1 Processor -- start 2023-10-10T20:42:18.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.596+0000 7fcd8193f700 1 -- start start 2023-10-10T20:42:18.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.596+0000 7fcd8193f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd6c003fe0 0x7fcd7c0af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:18.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.596+0000 7fcd8193f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd7c151120 0x7fcd7c15a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:18.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.596+0000 7fcd8193f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd7c154880 0x7fcd7c154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:18.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.596+0000 7fcd8193f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c167700 con 0x7fcd7c151120 2023-10-10T20:42:18.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.596+0000 7fcd8193f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c155230 con 0x7fcd7c154880 2023-10-10T20:42:18.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.596+0000 7fcd8193f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c15ada0 con 0x7fcd6c003fe0 2023-10-10T20:42:18.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.596+0000 7fcd7bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd7c151120 0x7fcd7c15a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:18.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.596+0000 7fcd7bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd7c151120 0x7fcd7c15a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44494/0 (socket says 172.21.15.89:44494) 2023-10-10T20:42:18.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.596+0000 7fcd7bfff700 1 -- 172.21.15.89:0/574071721 learned_addr learned my addr 172.21.15.89:0/574071721 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:18.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.597+0000 7fcd7bfff700 1 -- 172.21.15.89:0/574071721 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd6c003fe0 msgr2=0x7fcd7c0af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:18.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.597+0000 7fcd7bfff700 1 --2- 172.21.15.89:0/574071721 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd6c003fe0 0x7fcd7c0af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.597+0000 7fcd8113e700 1 --2- 172.21.15.89:0/574071721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd7c154880 0x7fcd7c154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:18.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.597+0000 7fcd7bfff700 1 -- 172.21.15.89:0/574071721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd7c154880 msgr2=0x7fcd7c154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:18.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.597+0000 7fcd7bfff700 1 --2- 172.21.15.89:0/574071721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd7c154880 0x7fcd7c154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.597+0000 7fcd7bfff700 1 -- 172.21.15.89:0/574071721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c15b070 con 0x7fcd7c151120 2023-10-10T20:42:18.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.597+0000 7fcd7bfff700 1 --2- 172.21.15.89:0/574071721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd7c151120 0x7fcd7c15a860 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800e4b0 tx=0x7fcd6801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:18.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.597+0000 7fcd78ff9700 1 -- 172.21.15.89:0/574071721 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68022070 con 0x7fcd7c151120 2023-10-10T20:42:18.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.597+0000 7fcd78ff9700 1 -- 172.21.15.89:0/574071721 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6800e980 con 0x7fcd7c151120 2023-10-10T20:42:18.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.597+0000 7fcd78ff9700 1 -- 172.21.15.89:0/574071721 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68027470 con 0x7fcd7c151120 2023-10-10T20:42:18.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.597+0000 7fcd8193f700 1 -- 172.21.15.89:0/574071721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7c15b350 con 0x7fcd7c151120 2023-10-10T20:42:18.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.597+0000 7fcd8193f700 1 -- 172.21.15.89:0/574071721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7c15b910 con 0x7fcd7c151120 2023-10-10T20:42:18.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.598+0000 7fcd8193f700 1 -- 172.21.15.89:0/574071721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7c04c3c0 con 0x7fcd7c151120 2023-10-10T20:42:18.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.598+0000 7fcd78ff9700 1 -- 172.21.15.89:0/574071721 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcd68027820 con 0x7fcd7c151120 2023-10-10T20:42:18.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.598+0000 7fcd78ff9700 1 --2- 172.21.15.89:0/574071721 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcd60067350 0x7fcd60069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:18.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.599+0000 7fcd78ff9700 1 -- 172.21.15.89:0/574071721 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fcd680862e0 con 0x7fcd7c151120 2023-10-10T20:42:18.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.599+0000 7fcd8093d700 1 --2- 172.21.15.89:0/574071721 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcd60067350 0x7fcd60069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:18.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.599+0000 7fcd8093d700 1 --2- 172.21.15.89:0/574071721 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcd60067350 0x7fcd60069800 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcd70006ea0 tx=0x7fcd70009120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:18.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.603+0000 7fcd78ff9700 1 -- 172.21.15.89:0/574071721 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd68053420 con 0x7fcd7c151120 2023-10-10T20:42:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.826+0000 7fcd8193f700 1 -- 172.21.15.89:0/574071721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd7c0cdb70 con 0x7fcd7c151120 2023-10-10T20:42:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.826+0000 7fcd78ff9700 1 -- 172.21.15.89:0/574071721 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fcd68017de0 con 0x7fcd7c151120 2023-10-10T20:42:18.830 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:18.830 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":95,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6197,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/4123310016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4123310016},{"type":"v1","addr":"172.21.15.111:6834","nonce":4123310016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":6200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/141017824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":141017824},{"type":"v1","addr":"172.21.15.111:6835","nonce":141017824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7575,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/728309352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":728309352},{"type":"v1","addr":"172.21.15.89:6836","nonce":728309352}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7890,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1838686233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":1838686233},{"type":"v1","addr":"172.21.15.89:6837","nonce":1838686233}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[]} 2023-10-10T20:42:18.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.828+0000 7fcd8193f700 1 -- 172.21.15.89:0/574071721 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcd60067350 msgr2=0x7fcd60069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:18.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.828+0000 7fcd8193f700 1 --2- 172.21.15.89:0/574071721 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcd60067350 0x7fcd60069800 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcd70006ea0 tx=0x7fcd70009120 comp rx=0 tx=0).stop 2023-10-10T20:42:18.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.828+0000 7fcd8193f700 1 -- 172.21.15.89:0/574071721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd7c151120 msgr2=0x7fcd7c15a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:18.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.828+0000 7fcd8193f700 1 --2- 172.21.15.89:0/574071721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd7c151120 0x7fcd7c15a860 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800e4b0 tx=0x7fcd6801a000 comp rx=0 tx=0).stop 2023-10-10T20:42:18.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.828+0000 7fcd8193f700 1 -- 172.21.15.89:0/574071721 shutdown_connections 2023-10-10T20:42:18.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.828+0000 7fcd8193f700 1 --2- 172.21.15.89:0/574071721 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcd60067350 0x7fcd60069800 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.828+0000 7fcd8193f700 1 --2- 172.21.15.89:0/574071721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd7c154880 0x7fcd7c154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.828+0000 7fcd8193f700 1 --2- 172.21.15.89:0/574071721 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcd6c003fe0 0x7fcd7c0af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.828+0000 7fcd8193f700 1 --2- 172.21.15.89:0/574071721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd7c151120 0x7fcd7c15a860 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:18.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.828+0000 7fcd8193f700 1 -- 172.21.15.89:0/574071721 >> 172.21.15.89:0/574071721 conn(0x7fcd7c0ce430 msgr2=0x7fcd7c155800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:18.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.828+0000 7fcd8193f700 1 -- 172.21.15.89:0/574071721 shutdown_connections 2023-10-10T20:42:18.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:18.828+0000 7fcd8193f700 1 -- 172.21.15.89:0/574071721 wait complete. 2023-10-10T20:42:18.835 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 95 2023-10-10T20:42:18.844 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:42:18.844 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:42:18.844 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:42:18.845 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:42:18.845 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:42:18.845 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:42:18.846 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:42:18.846 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:42:18.846 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:42:18.847 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:42:18.848 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:42:18.848 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:42:18.848 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:42:18.849 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:42:18.849 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:42:18.849 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:42:18.853 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:42:18.853 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:42:18.854 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:42:18.855 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:42:18.867 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:42:18.868 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:42:18.869 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:42:18.870 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:42:18.897 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:42:18.898 DEBUG:teuthology.parallel:result is None 2023-10-10T20:42:18.900 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:42:18.901 DEBUG:teuthology.parallel:result is None 2023-10-10T20:42:18.910 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:42:18.911 DEBUG:teuthology.parallel:result is None 2023-10-10T20:42:18.912 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:42:18.913 DEBUG:teuthology.parallel:result is None 2023-10-10T20:42:18.913 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:42:18.970 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:42:18.971 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-10T20:42:18.991 INFO:tasks.ceph.mds.b.smithi111.stdout:starting mds.b at 2023-10-10T20:42:18.992 INFO:tasks.ceph.mds.d.smithi111.stdout:starting mds.d at 2023-10-10T20:42:19.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.032+0000 7f8a594d0700 1 Processor -- start 2023-10-10T20:42:19.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.032+0000 7f8a594d0700 1 -- start start 2023-10-10T20:42:19.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.033+0000 7f8a594d0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8a5406e6c0 0x7f8a540abec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:19.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.033+0000 7f8a594d0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a5415bee0 0x7f8a540aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:19.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.033+0000 7f8a594d0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a540b0ab0 con 0x7f8a5406ea90 2023-10-10T20:42:19.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.033+0000 7f8a594d0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a540b18f0 con 0x7f8a5406e6c0 2023-10-10T20:42:19.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.033+0000 7f8a594d0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a540b0ce0 con 0x7f8a5415bee0 2023-10-10T20:42:19.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.033+0000 7f8a52ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8a5406e6c0 0x7f8a540abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:19.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.033+0000 7f8a537fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8a5406ea90 0x7f8a54156ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48904/0 (socket says 172.21.15.89:48904) 2023-10-10T20:42:19.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.033+0000 7f8a537fe700 1 -- 172.21.15.89:0/3604863220 learned_addr learned my addr 172.21.15.89:0/3604863220 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:19.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.033+0000 7f8a4a7fc700 1 --2- 172.21.15.89:0/3604863220 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a5415bee0 0x7f8a540aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:19.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.033+0000 7f8a52ffd700 1 -- 172.21.15.89:0/3604863220 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a5415bee0 msgr2=0x7f8a540aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:19.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.033+0000 7f8a52ffd700 1 --2- 172.21.15.89:0/3604863220 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a5415bee0 0x7f8a540aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:19.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.033+0000 7f8a52ffd700 1 -- 172.21.15.89:0/3604863220 >> v1:172.21.15.89:6789/0 conn(0x7f8a5406ea90 legacy=0x7f8a54156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:19.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.033+0000 7f8a527fc700 1 -- 172.21.15.89:0/3604863220 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405498885 0 0) 0x7f8a540b0ab0 con 0x7f8a5406ea90 2023-10-10T20:42:19.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.033+0000 7f8a52ffd700 1 -- 172.21.15.89:0/3604863220 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a540b0fc0 con 0x7f8a5406e6c0 2023-10-10T20:42:19.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.034+0000 7f8a52ffd700 1 --2- 172.21.15.89:0/3604863220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5406e6c0 0x7f8a540abec0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009910 tx=0x7f8a440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fcd8d5dc0c41493 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:19.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.034+0000 7f8a527fc700 1 -- 172.21.15.89:0/3604863220 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4400b820 con 0x7f8a5406e6c0 2023-10-10T20:42:19.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.034+0000 7f8a527fc700 1 -- 172.21.15.89:0/3604863220 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4400b980 con 0x7f8a5406e6c0 2023-10-10T20:42:19.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.034+0000 7f8a527fc700 1 -- 172.21.15.89:0/3604863220 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4400bd20 con 0x7f8a5406e6c0 2023-10-10T20:42:19.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.034+0000 7f8a594d0700 1 -- 172.21.15.89:0/3604863220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5406e6c0 msgr2=0x7f8a540abec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:19.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.034+0000 7f8a594d0700 1 --2- 172.21.15.89:0/3604863220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5406e6c0 0x7f8a540abec0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009910 tx=0x7f8a440091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:19.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.034+0000 7f8a594d0700 1 -- 172.21.15.89:0/3604863220 shutdown_connections 2023-10-10T20:42:19.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.034+0000 7f8a594d0700 1 --2- 172.21.15.89:0/3604863220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5406e6c0 0x7f8a540abec0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:19.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.034+0000 7f8a594d0700 1 --2- 172.21.15.89:0/3604863220 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a5415bee0 0x7f8a540aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:19.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.034+0000 7f8a594d0700 1 -- 172.21.15.89:0/3604863220 >> 172.21.15.89:0/3604863220 conn(0x7f8a540c9f30 msgr2=0x7f8a5406dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:19.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.034+0000 7f8a594d0700 1 -- 172.21.15.89:0/3604863220 shutdown_connections 2023-10-10T20:42:19.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.034+0000 7f8a594d0700 1 -- 172.21.15.89:0/3604863220 wait complete. 2023-10-10T20:42:19.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.036+0000 7f8a594d0700 1 Processor -- start 2023-10-10T20:42:19.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.036+0000 7f8a594d0700 1 -- start start 2023-10-10T20:42:19.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.036+0000 7f8a594d0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5406e6c0 0x7f8a54148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:19.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.036+0000 7f8a594d0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5406ea90 0x7f8a5415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:19.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.036+0000 7f8a594d0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a5415bee0 0x7f8a54156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:19.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.036+0000 7f8a594d0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a54148ac0 con 0x7f8a5406ea90 2023-10-10T20:42:19.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.036+0000 7f8a594d0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5406a820 con 0x7f8a5406e6c0 2023-10-10T20:42:19.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.036+0000 7f8a594d0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5406ab00 con 0x7f8a5415bee0 2023-10-10T20:42:19.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.036+0000 7f8a4a7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5406ea90 0x7f8a5415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:19.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.036+0000 7f8a4a7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5406ea90 0x7f8a5415a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44528/0 (socket says 172.21.15.89:44528) 2023-10-10T20:42:19.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.036+0000 7f8a4a7fc700 1 -- 172.21.15.89:0/3854096769 learned_addr learned my addr 172.21.15.89:0/3854096769 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:19.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.037+0000 7f8a537fe700 1 --2- 172.21.15.89:0/3854096769 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a5415bee0 0x7f8a54156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:19.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.037+0000 7f8a52ffd700 1 --2- 172.21.15.89:0/3854096769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5406e6c0 0x7f8a54148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:19.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.037+0000 7f8a4a7fc700 1 -- 172.21.15.89:0/3854096769 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a5415bee0 msgr2=0x7f8a54156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:19.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.037+0000 7f8a4a7fc700 1 --2- 172.21.15.89:0/3854096769 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a5415bee0 0x7f8a54156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:19.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.037+0000 7f8a4a7fc700 1 -- 172.21.15.89:0/3854096769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5406e6c0 msgr2=0x7f8a54148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:19.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.037+0000 7f8a4a7fc700 1 --2- 172.21.15.89:0/3854096769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5406e6c0 0x7f8a54148580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:19.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.037+0000 7f8a4a7fc700 1 -- 172.21.15.89:0/3854096769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a44009420 con 0x7f8a5406ea90 2023-10-10T20:42:19.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.037+0000 7f8a4a7fc700 1 --2- 172.21.15.89:0/3854096769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5406ea90 0x7f8a5415a1a0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00d860 tx=0x7f8a3c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:19.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.037+0000 7f8a4b7fe700 1 -- 172.21.15.89:0/3854096769 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0125f0 con 0x7f8a5406ea90 2023-10-10T20:42:19.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.037+0000 7f8a4b7fe700 1 -- 172.21.15.89:0/3854096769 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3c012750 con 0x7f8a5406ea90 2023-10-10T20:42:19.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.037+0000 7f8a594d0700 1 -- 172.21.15.89:0/3854096769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a5406ade0 con 0x7f8a5406ea90 2023-10-10T20:42:19.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.037+0000 7f8a4b7fe700 1 -- 172.21.15.89:0/3854096769 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c012b10 con 0x7f8a5406ea90 2023-10-10T20:42:19.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.037+0000 7f8a594d0700 1 -- 172.21.15.89:0/3854096769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5406b3a0 con 0x7f8a5406ea90 2023-10-10T20:42:19.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.038+0000 7f8a594d0700 1 -- 172.21.15.89:0/3854096769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a5404c3c0 con 0x7f8a5406ea90 2023-10-10T20:42:19.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.038+0000 7f8a4b7fe700 1 -- 172.21.15.89:0/3854096769 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8a3c01e460 con 0x7f8a5406ea90 2023-10-10T20:42:19.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.039+0000 7f8a4b7fe700 1 --2- 172.21.15.89:0/3854096769 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8a34067320 0x7f8a340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:19.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.039+0000 7f8a4b7fe700 1 -- 172.21.15.89:0/3854096769 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8a3c085030 con 0x7f8a5406ea90 2023-10-10T20:42:19.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.039+0000 7f8a52ffd700 1 --2- 172.21.15.89:0/3854096769 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8a34067320 0x7f8a340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:19.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.039+0000 7f8a52ffd700 1 --2- 172.21.15.89:0/3854096769 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8a34067320 0x7f8a340697d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8a440091e0 tx=0x7f8a44017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:19.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.043+0000 7f8a4b7fe700 1 -- 172.21.15.89:0/3854096769 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a3c055a50 con 0x7f8a5406ea90 2023-10-10T20:42:19.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:19.248+0000 7f8a594d0700 1 -- 172.21.15.89:0/3854096769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8a540b12a0 con 0x7f8a5406ea90 2023-10-10T20:42:20.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.072+0000 7f8a4b7fe700 1 -- 172.21.15.89:0/3854096769 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v149) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8a3c0555e0 con 0x7f8a5406ea90 2023-10-10T20:42:20.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.074+0000 7f8a594d0700 1 -- 172.21.15.89:0/3854096769 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8a34067320 msgr2=0x7f8a340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:20.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.074+0000 7f8a594d0700 1 --2- 172.21.15.89:0/3854096769 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8a34067320 0x7f8a340697d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8a440091e0 tx=0x7f8a44017000 comp rx=0 tx=0).stop 2023-10-10T20:42:20.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.074+0000 7f8a594d0700 1 -- 172.21.15.89:0/3854096769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5406ea90 msgr2=0x7f8a5415a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:20.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.074+0000 7f8a594d0700 1 --2- 172.21.15.89:0/3854096769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5406ea90 0x7f8a5415a1a0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00d860 tx=0x7f8a3c008040 comp rx=0 tx=0).stop 2023-10-10T20:42:20.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.074+0000 7f8a594d0700 1 -- 172.21.15.89:0/3854096769 shutdown_connections 2023-10-10T20:42:20.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.074+0000 7f8a594d0700 1 --2- 172.21.15.89:0/3854096769 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8a5415bee0 0x7f8a54156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.074+0000 7f8a594d0700 1 --2- 172.21.15.89:0/3854096769 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8a34067320 0x7f8a340697d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.074+0000 7f8a594d0700 1 --2- 172.21.15.89:0/3854096769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5406e6c0 0x7f8a54148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.074+0000 7f8a594d0700 1 --2- 172.21.15.89:0/3854096769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a5406ea90 0x7f8a5415a1a0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.074+0000 7f8a594d0700 1 -- 172.21.15.89:0/3854096769 >> 172.21.15.89:0/3854096769 conn(0x7f8a540c9f30 msgr2=0x7f8a540cb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:20.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.074+0000 7f8a594d0700 1 -- 172.21.15.89:0/3854096769 shutdown_connections 2023-10-10T20:42:20.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.074+0000 7f8a594d0700 1 -- 172.21.15.89:0/3854096769 wait complete. 2023-10-10T20:42:20.080 INFO:teuthology.orchestra.run.smithi089.stderr: removed all blocklist entries 2023-10-10T20:42:20.089 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:42:20.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.206+0000 7f6a1c7ae700 1 Processor -- start 2023-10-10T20:42:20.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.206+0000 7f6a1c7ae700 1 -- start start 2023-10-10T20:42:20.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.207+0000 7f6a1c7ae700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a14159890 0x7f6a1415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.207+0000 7f6a1c7ae700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a1415a160 con 0x7f6a14069560 2023-10-10T20:42:20.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.207+0000 7f6a1c7ae700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a140d13a0 con 0x7f6a14159c60 2023-10-10T20:42:20.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.207+0000 7f6a1c7ae700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1415e140 con 0x7f6a14159890 2023-10-10T20:42:20.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.207+0000 7f6a1ad4b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6a14069560 0x7f6a140ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48912/0 (socket says 172.21.15.89:48912) 2023-10-10T20:42:20.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.207+0000 7f6a1ad4b700 1 -- 172.21.15.89:0/299194049 learned_addr learned my addr 172.21.15.89:0/299194049 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:20.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.207+0000 7f6a19548700 1 -- 172.21.15.89:0/299194049 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3689560594 0 0) 0x7f6a1415a160 con 0x7f6a14069560 2023-10-10T20:42:20.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.207+0000 7f6a1a54a700 1 --2- 172.21.15.89:0/299194049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a14159890 0x7f6a1415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.207+0000 7f6a19548700 1 -- 172.21.15.89:0/299194049 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a00003570 con 0x7f6a14069560 2023-10-10T20:42:20.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.207+0000 7f6a19548700 1 -- 172.21.15.89:0/299194049 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 323835591 0 0) 0x7f6a00003570 con 0x7f6a14069560 2023-10-10T20:42:20.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.207+0000 7f6a19548700 1 -- 172.21.15.89:0/299194049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a14159890 msgr2=0x7f6a1415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:20.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.207+0000 7f6a19548700 1 --2- 172.21.15.89:0/299194049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a14159890 0x7f6a1415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.207+0000 7f6a19548700 1 -- 172.21.15.89:0/299194049 >> v1:172.21.15.111:6789/0 conn(0x7f6a14159c60 legacy=0x7f6a1415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:20.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.207+0000 7f6a19548700 1 -- 172.21.15.89:0/299194049 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1415a3d0 con 0x7f6a14069560 2023-10-10T20:42:20.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.207+0000 7f6a19548700 1 -- 172.21.15.89:0/299194049 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f6a0c0017f0 con 0x7f6a14069560 2023-10-10T20:42:20.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.207+0000 7f6a19548700 1 -- 172.21.15.89:0/299194049 >> v1:172.21.15.89:6789/0 conn(0x7f6a14069560 legacy=0x7f6a140ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:20.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.207+0000 7f6a19548700 1 --2- 172.21.15.89:0/299194049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a00003a90 0x7f6a00003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.207+0000 7f6a19548700 1 --2- 172.21.15.89:0/299194049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a00004810 0x7f6a00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.208+0000 7f6a19548700 1 --2- 172.21.15.89:0/299194049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a14159890 0x7f6a000051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.208+0000 7f6a19548700 1 -- 172.21.15.89:0/299194049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1415e140 con 0x7f6a00003a90 2023-10-10T20:42:20.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.208+0000 7f6a19548700 1 -- 172.21.15.89:0/299194049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0000bb60 con 0x7f6a00004810 2023-10-10T20:42:20.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.208+0000 7f6a19548700 1 -- 172.21.15.89:0/299194049 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0000bd20 con 0x7f6a14159890 2023-10-10T20:42:20.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.208+0000 7f6a1a54a700 1 --2- 172.21.15.89:0/299194049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a00003a90 0x7f6a00003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.208+0000 7f6a1ad4b700 1 --2- 172.21.15.89:0/299194049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a14159890 0x7f6a000051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.208+0000 7f6a19d49700 1 --2- 172.21.15.89:0/299194049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a00004810 0x7f6a00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.208+0000 7f6a1a54a700 1 -- 172.21.15.89:0/299194049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a14159890 msgr2=0x7f6a000051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:20.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.208+0000 7f6a1a54a700 1 --2- 172.21.15.89:0/299194049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a14159890 0x7f6a000051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.208+0000 7f6a1a54a700 1 -- 172.21.15.89:0/299194049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a00004810 msgr2=0x7f6a00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:20.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.208+0000 7f6a1a54a700 1 --2- 172.21.15.89:0/299194049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a00004810 0x7f6a00004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.208+0000 7f6a1a54a700 1 -- 172.21.15.89:0/299194049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a00003570 con 0x7f6a00003a90 2023-10-10T20:42:20.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.208+0000 7f6a1ad4b700 1 --2- 172.21.15.89:0/299194049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a14159890 0x7f6a000051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:20.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.208+0000 7f6a1a54a700 1 --2- 172.21.15.89:0/299194049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a00003a90 0x7f6a00003ea0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009c00 tx=0x7f6a100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab6a4486edf632b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:20.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.208+0000 7f6a19548700 1 -- 172.21.15.89:0/299194049 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1000e710 con 0x7f6a00003a90 2023-10-10T20:42:20.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.208+0000 7f6a19548700 1 -- 172.21.15.89:0/299194049 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1000e870 con 0x7f6a00003a90 2023-10-10T20:42:20.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.208+0000 7f6a19548700 1 -- 172.21.15.89:0/299194049 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1000ec30 con 0x7f6a00003a90 2023-10-10T20:42:20.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.209+0000 7f6a1c7ae700 1 -- 172.21.15.89:0/299194049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a00003a90 msgr2=0x7f6a00003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:20.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.209+0000 7f6a1c7ae700 1 --2- 172.21.15.89:0/299194049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a00003a90 0x7f6a00003ea0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009c00 tx=0x7f6a100091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.209+0000 7f6a1ad4b700 1 -- 172.21.15.89:0/299194049 reap_dead start 2023-10-10T20:42:20.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.209+0000 7f6a1c7ae700 1 -- 172.21.15.89:0/299194049 shutdown_connections 2023-10-10T20:42:20.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.209+0000 7f6a1c7ae700 1 -- 172.21.15.89:0/299194049 >> 172.21.15.89:0/299194049 conn(0x7f6a14068e20 msgr2=0x7f6a140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:20.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.209+0000 7f6a1c7ae700 1 -- 172.21.15.89:0/299194049 shutdown_connections 2023-10-10T20:42:20.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.209+0000 7f6a1c7ae700 1 -- 172.21.15.89:0/299194049 wait complete. 2023-10-10T20:42:20.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.210+0000 7f6a1c7ae700 1 Processor -- start 2023-10-10T20:42:20.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.210+0000 7f6a1c7ae700 1 -- start start 2023-10-10T20:42:20.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.210+0000 7f6a1c7ae700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a00003a90 0x7f6a140b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.210+0000 7f6a1c7ae700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a140c78a0 0x7f6a140c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.210+0000 7f6a1c7ae700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a140c9860 0x7f6a140cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.210+0000 7f6a1c7ae700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140b3a50 con 0x7f6a140c78a0 2023-10-10T20:42:20.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.210+0000 7f6a1c7ae700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140ce750 con 0x7f6a00003a90 2023-10-10T20:42:20.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.210+0000 7f6a1c7ae700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140cea30 con 0x7f6a140c9860 2023-10-10T20:42:20.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.211+0000 7f6a19d49700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a140c78a0 0x7f6a140c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.211+0000 7f6a19d49700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a140c78a0 0x7f6a140c9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44544/0 (socket says 172.21.15.89:44544) 2023-10-10T20:42:20.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.211+0000 7f6a19d49700 1 -- 172.21.15.89:0/4280007199 learned_addr learned my addr 172.21.15.89:0/4280007199 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:20.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.211+0000 7f6a1a54a700 1 --2- 172.21.15.89:0/4280007199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a00003a90 0x7f6a140b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.211+0000 7f6a1ad4b700 1 --2- 172.21.15.89:0/4280007199 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a140c9860 0x7f6a140cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.211+0000 7f6a19d49700 1 -- 172.21.15.89:0/4280007199 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a140c9860 msgr2=0x7f6a140cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:20.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.211+0000 7f6a19d49700 1 --2- 172.21.15.89:0/4280007199 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a140c9860 0x7f6a140cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.211+0000 7f6a19d49700 1 -- 172.21.15.89:0/4280007199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a00003a90 msgr2=0x7f6a140b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:20.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.211+0000 7f6a19d49700 1 --2- 172.21.15.89:0/4280007199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a00003a90 0x7f6a140b5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.211+0000 7f6a19d49700 1 -- 172.21.15.89:0/4280007199 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a140ced10 con 0x7f6a140c78a0 2023-10-10T20:42:20.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.211+0000 7f6a19d49700 1 --2- 172.21.15.89:0/4280007199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a140c78a0 0x7f6a140c9320 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400e610 tx=0x7f6a04009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:20.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.211+0000 7f6a0a7fc700 1 -- 172.21.15.89:0/4280007199 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a040115c0 con 0x7f6a140c78a0 2023-10-10T20:42:20.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.211+0000 7f6a0a7fc700 1 -- 172.21.15.89:0/4280007199 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a04011720 con 0x7f6a140c78a0 2023-10-10T20:42:20.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.211+0000 7f6a0a7fc700 1 -- 172.21.15.89:0/4280007199 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04011ac0 con 0x7f6a140c78a0 2023-10-10T20:42:20.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.211+0000 7f6a1c7ae700 1 -- 172.21.15.89:0/4280007199 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a140b36e0 con 0x7f6a140c78a0 2023-10-10T20:42:20.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.211+0000 7f6a1c7ae700 1 -- 172.21.15.89:0/4280007199 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a14151390 con 0x7f6a140c78a0 2023-10-10T20:42:20.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.212+0000 7f6a1c7ae700 1 -- 172.21.15.89:0/4280007199 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a1404c3c0 con 0x7f6a140c78a0 2023-10-10T20:42:20.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.212+0000 7f6a0a7fc700 1 -- 172.21.15.89:0/4280007199 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6a04014480 con 0x7f6a140c78a0 2023-10-10T20:42:20.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.213+0000 7f6a0a7fc700 1 --2- 172.21.15.89:0/4280007199 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f69f8067320 0x7f69f80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.213+0000 7f6a0a7fc700 1 -- 172.21.15.89:0/4280007199 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6a040844d0 con 0x7f6a140c78a0 2023-10-10T20:42:20.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.213+0000 7f6a1a54a700 1 --2- 172.21.15.89:0/4280007199 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f69f8067320 0x7f69f80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.213+0000 7f6a1a54a700 1 --2- 172.21.15.89:0/4280007199 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f69f8067320 0x7f69f80697d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6a100091e0 tx=0x7f6a1001a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:20.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.217+0000 7f6a0a7fc700 1 -- 172.21.15.89:0/4280007199 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a0404d5a0 con 0x7f6a140c78a0 2023-10-10T20:42:20.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.472+0000 7f6a1c7ae700 1 -- 172.21.15.89:0/4280007199 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6a1406a5b0 con 0x7f6a140c78a0 2023-10-10T20:42:20.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.472+0000 7f6a0a7fc700 1 -- 172.21.15.89:0/4280007199 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6a04051620 con 0x7f6a140c78a0 2023-10-10T20:42:20.476 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:20.476 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:42:20.476 INFO:teuthology.orchestra.run.smithi089.stdout: "auth_dump": [ 2023-10-10T20:42:20.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:20.477 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.a", 2023-10-10T20:42:20.477 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlZLSNLhAA8WZTrsht6tAOP+gxaTXKMg==", 2023-10-10T20:42:20.477 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:20.478 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:42:20.478 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:42:20.478 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:42:20.478 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:20.478 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:20.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:20.479 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.b", 2023-10-10T20:42:20.479 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgyDEORAAMMoSxPFZc0bCIwWy8UsXgA==", 2023-10-10T20:42:20.480 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:20.480 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:42:20.480 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:42:20.480 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:42:20.480 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:20.481 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:20.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:20.481 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.c", 2023-10-10T20:42:20.482 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlMFxSNRAASuQEJFghLIOkTQWa4ckTvg==", 2023-10-10T20:42:20.482 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:20.482 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:42:20.482 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:42:20.482 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:42:20.483 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:20.483 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:20.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:20.483 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.d", 2023-10-10T20:42:20.484 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlPkvrBBAA2VLJmv2t8szTEGtCVwalJw==", 2023-10-10T20:42:20.484 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:20.484 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:42:20.484 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:42:20.485 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:42:20.485 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:20.485 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:20.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:20.486 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.0", 2023-10-10T20:42:20.486 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQArtSVltOcKHRAAk8+zUus04oTVI9KvfhjlXg==", 2023-10-10T20:42:20.486 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:20.486 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:42:20.487 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:42:20.487 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:20.489 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:20.490 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:20.490 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.1", 2023-10-10T20:42:20.490 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAutSVly8P0GhAAe1bvR63+ptJg84lizMNdQw==", 2023-10-10T20:42:20.490 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:20.490 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:42:20.491 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:42:20.491 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:20.491 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.491 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:20.491 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:20.491 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.2", 2023-10-10T20:42:20.492 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAxtSVl0x9fGRAAEKi7+6EW5mzPbebmz0+sgQ==", 2023-10-10T20:42:20.492 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:20.492 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:42:20.492 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:42:20.492 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:20.493 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:20.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:20.493 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.3", 2023-10-10T20:42:20.493 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA0tSVlfJyaGBAAa+8w54U6v7+npbdufKJruA==", 2023-10-10T20:42:20.494 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:20.494 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:42:20.494 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:42:20.494 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:20.494 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:20.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:20.495 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.4", 2023-10-10T20:42:20.495 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA6tSVlEQ/QFxAA/PRB3c90BuAlF6xekGfX9A==", 2023-10-10T20:42:20.495 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:20.495 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:42:20.496 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:42:20.496 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:20.496 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:20.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:20.497 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.5", 2023-10-10T20:42:20.497 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA9tSVlMOKrFRAAuBnIsEz2VmeSZi9XWzjlzA==", 2023-10-10T20:42:20.497 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:20.497 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:42:20.497 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:42:20.498 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:20.498 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:20.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:20.498 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.6", 2023-10-10T20:42:20.499 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBAtSVlH7PQFBAAHr6c7d4TYJEu/KSFEf+EOA==", 2023-10-10T20:42:20.499 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:20.499 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:42:20.500 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:42:20.500 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:20.500 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:20.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:20.501 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.7", 2023-10-10T20:42:20.501 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBDtSVlAhC1FBAAuE2BwAA/f7VOkZfVdqAG7A==", 2023-10-10T20:42:20.501 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:20.501 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:42:20.501 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:42:20.502 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:20.502 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:20.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:20.502 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.0", 2023-10-10T20:42:20.505 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlRd3HCBAAb0QP2VfTS7aelno1mFpGAA==", 2023-10-10T20:42:20.505 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:20.505 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:42:20.506 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow r", 2023-10-10T20:42:20.506 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:42:20.506 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:20.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:20.507 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.1", 2023-10-10T20:42:20.507 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlgBouERAAqPdE0KvJU3M53B6Y6AH5GQ==", 2023-10-10T20:42:20.507 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:20.507 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:42:20.507 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:42:20.508 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:42:20.509 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:42:20.509 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:20.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:20.510 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.2", 2023-10-10T20:42:20.510 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlXA4tDhAAoPVYQjECpzD+C67mnsY5IA==", 2023-10-10T20:42:20.510 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:20.510 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:42:20.510 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:42:20.511 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:42:20.511 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:42:20.511 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:20.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:20.512 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.3", 2023-10-10T20:42:20.512 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlZ5KGFhAAuKmZwbOfmmw1nAj5ry6tHw==", 2023-10-10T20:42:20.512 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:20.512 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:42:20.512 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:42:20.513 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:42:20.513 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:42:20.513 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:20.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:20.513 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.admin", 2023-10-10T20:42:20.514 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgpL2EhAAoDOki002rfBrauTgfu481Q==", 2023-10-10T20:42:20.514 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:20.514 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:42:20.514 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:42:20.514 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:42:20.515 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:20.515 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:20.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:20.515 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.x", 2023-10-10T20:42:20.516 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlBhaJKxAArraBxT5xWg515LfpJ7MlbA==", 2023-10-10T20:42:20.516 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:20.516 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:42:20.516 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:42:20.516 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:20.517 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:20.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:20.517 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.y", 2023-10-10T20:42:20.517 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlTZIyJxAAWAOsitHwsIxJWwp3b1Z98Q==", 2023-10-10T20:42:20.518 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:20.518 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:42:20.518 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:42:20.518 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:20.518 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.518 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:20.519 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-10T20:42:20.519 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:42:20.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.474+0000 7f6a1c7ae700 1 -- 172.21.15.89:0/4280007199 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f69f8067320 msgr2=0x7f69f80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:20.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.474+0000 7f6a1c7ae700 1 --2- 172.21.15.89:0/4280007199 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f69f8067320 0x7f69f80697d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6a100091e0 tx=0x7f6a1001a000 comp rx=0 tx=0).stop 2023-10-10T20:42:20.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.474+0000 7f6a1c7ae700 1 -- 172.21.15.89:0/4280007199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a140c78a0 msgr2=0x7f6a140c9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:20.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.474+0000 7f6a1c7ae700 1 --2- 172.21.15.89:0/4280007199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a140c78a0 0x7f6a140c9320 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400e610 tx=0x7f6a04009040 comp rx=0 tx=0).stop 2023-10-10T20:42:20.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.474+0000 7f6a1c7ae700 1 -- 172.21.15.89:0/4280007199 shutdown_connections 2023-10-10T20:42:20.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.474+0000 7f6a1c7ae700 1 --2- 172.21.15.89:0/4280007199 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a140c9860 0x7f6a140cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.474+0000 7f6a1c7ae700 1 --2- 172.21.15.89:0/4280007199 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f69f8067320 0x7f69f80697d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.474+0000 7f6a1c7ae700 1 --2- 172.21.15.89:0/4280007199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a00003a90 0x7f6a140b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.474+0000 7f6a1c7ae700 1 --2- 172.21.15.89:0/4280007199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a140c78a0 0x7f6a140c9320 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.474+0000 7f6a1c7ae700 1 -- 172.21.15.89:0/4280007199 >> 172.21.15.89:0/4280007199 conn(0x7f6a14068e20 msgr2=0x7f6a1415c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:20.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.474+0000 7f6a1c7ae700 1 -- 172.21.15.89:0/4280007199 shutdown_connections 2023-10-10T20:42:20.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.474+0000 7f6a1c7ae700 1 -- 172.21.15.89:0/4280007199 wait complete. 2023-10-10T20:42:20.525 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:42:20.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.635+0000 7fcb088b4700 1 Processor -- start 2023-10-10T20:42:20.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.635+0000 7fcb088b4700 1 -- start start 2023-10-10T20:42:20.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.636+0000 7fcb088b4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb04159890 0x7fcb0415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.636+0000 7fcb088b4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb04069560 0x7fcb040699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.636+0000 7fcb088b4700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0415a160 con 0x7fcb04159c60 2023-10-10T20:42:20.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.636+0000 7fcb088b4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb04069f20 con 0x7fcb04069560 2023-10-10T20:42:20.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.636+0000 7fcb088b4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb040d13a0 con 0x7fcb04159890 2023-10-10T20:42:20.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.636+0000 7fcb02d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb04069560 0x7fcb040699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.636+0000 7fcb02d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb04069560 0x7fcb040699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44550/0 (socket says 172.21.15.89:44550) 2023-10-10T20:42:20.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.636+0000 7fcb02d9d700 1 -- 172.21.15.89:0/332241051 learned_addr learned my addr 172.21.15.89:0/332241051 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:20.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.637+0000 7fcb0259c700 1 --2- 172.21.15.89:0/332241051 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb04159890 0x7fcb0415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.637+0000 7fcb02d9d700 1 -- 172.21.15.89:0/332241051 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb04159890 msgr2=0x7fcb0415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:20.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.637+0000 7fcb02d9d700 1 --2- 172.21.15.89:0/332241051 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb04159890 0x7fcb0415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.637+0000 7fcb02d9d700 1 -- 172.21.15.89:0/332241051 >> v1:172.21.15.111:6789/0 conn(0x7fcb04159c60 legacy=0x7fcb0415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:20.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.637+0000 7fcb02d9d700 1 -- 172.21.15.89:0/332241051 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0415e140 con 0x7fcb04069560 2023-10-10T20:42:20.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.637+0000 7fcb02d9d700 1 --2- 172.21.15.89:0/332241051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb04069560 0x7fcb040699e0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009f20 tx=0x7fcafc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e1e081292de481 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:20.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.637+0000 7fcb01d9b700 1 -- 172.21.15.89:0/332241051 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc00e070 con 0x7fcb04069560 2023-10-10T20:42:20.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.637+0000 7fcb01d9b700 1 -- 172.21.15.89:0/332241051 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc00ba40 con 0x7fcb04069560 2023-10-10T20:42:20.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.637+0000 7fcb01d9b700 1 -- 172.21.15.89:0/332241051 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc0092e0 con 0x7fcb04069560 2023-10-10T20:42:20.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.637+0000 7fcb088b4700 1 -- 172.21.15.89:0/332241051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb04069560 msgr2=0x7fcb040699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:20.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.637+0000 7fcb088b4700 1 --2- 172.21.15.89:0/332241051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb04069560 0x7fcb040699e0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009f20 tx=0x7fcafc009840 comp rx=0 tx=0).stop 2023-10-10T20:42:20.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.637+0000 7fcb088b4700 1 -- 172.21.15.89:0/332241051 shutdown_connections 2023-10-10T20:42:20.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.638+0000 7fcb088b4700 1 --2- 172.21.15.89:0/332241051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb04069560 0x7fcb040699e0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.638+0000 7fcb088b4700 1 --2- 172.21.15.89:0/332241051 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb04159890 0x7fcb0415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.638+0000 7fcb088b4700 1 -- 172.21.15.89:0/332241051 >> 172.21.15.89:0/332241051 conn(0x7fcb04068e20 msgr2=0x7fcb040d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:20.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.638+0000 7fcb088b4700 1 -- 172.21.15.89:0/332241051 shutdown_connections 2023-10-10T20:42:20.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.638+0000 7fcb088b4700 1 -- 172.21.15.89:0/332241051 wait complete. 2023-10-10T20:42:20.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.639+0000 7fcb088b4700 1 Processor -- start 2023-10-10T20:42:20.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.639+0000 7fcb088b4700 1 -- start start 2023-10-10T20:42:20.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.639+0000 7fcb088b4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb04069560 0x7fcb04153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.639+0000 7fcb088b4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb04159890 0x7fcb0416d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.639+0000 7fcb088b4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb04159c60 0x7fcb0416d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.639+0000 7fcb088b4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb04154030 con 0x7fcb04159c60 2023-10-10T20:42:20.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.639+0000 7fcb088b4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb04154310 con 0x7fcb04159890 2023-10-10T20:42:20.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.639+0000 7fcb088b4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb041545f0 con 0x7fcb04069560 2023-10-10T20:42:20.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.640+0000 7fcb02d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb04159c60 0x7fcb0416d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.640+0000 7fcb02d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb04159c60 0x7fcb0416d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44566/0 (socket says 172.21.15.89:44566) 2023-10-10T20:42:20.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.640+0000 7fcb02d9d700 1 -- 172.21.15.89:0/1212044672 learned_addr learned my addr 172.21.15.89:0/1212044672 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:20.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.640+0000 7fcaf9d9b700 1 --2- 172.21.15.89:0/1212044672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb04159890 0x7fcb0416d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.640+0000 7fcb0259c700 1 --2- 172.21.15.89:0/1212044672 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb04069560 0x7fcb04153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.640+0000 7fcb02d9d700 1 -- 172.21.15.89:0/1212044672 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb04069560 msgr2=0x7fcb04153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:20.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.640+0000 7fcb02d9d700 1 --2- 172.21.15.89:0/1212044672 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb04069560 0x7fcb04153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.640+0000 7fcb02d9d700 1 -- 172.21.15.89:0/1212044672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb04159890 msgr2=0x7fcb0416d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:20.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.640+0000 7fcb02d9d700 1 --2- 172.21.15.89:0/1212044672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb04159890 0x7fcb0416d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.640+0000 7fcb02d9d700 1 -- 172.21.15.89:0/1212044672 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc009ae0 con 0x7fcb04159c60 2023-10-10T20:42:20.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.640+0000 7fcb02d9d700 1 --2- 172.21.15.89:0/1212044672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb04159c60 0x7fcb0416d930 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fcb040899c0 tx=0x7fcafc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:20.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.640+0000 7fcafaffd700 1 -- 172.21.15.89:0/1212044672 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc012e60 con 0x7fcb04159c60 2023-10-10T20:42:20.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.640+0000 7fcafaffd700 1 -- 172.21.15.89:0/1212044672 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc0092e0 con 0x7fcb04159c60 2023-10-10T20:42:20.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.640+0000 7fcb088b4700 1 -- 172.21.15.89:0/1212044672 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb041548d0 con 0x7fcb04159c60 2023-10-10T20:42:20.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.640+0000 7fcafaffd700 1 -- 172.21.15.89:0/1212044672 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc01d670 con 0x7fcb04159c60 2023-10-10T20:42:20.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.640+0000 7fcb088b4700 1 -- 172.21.15.89:0/1212044672 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb04153c70 con 0x7fcb04159c60 2023-10-10T20:42:20.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.641+0000 7fcb088b4700 1 -- 172.21.15.89:0/1212044672 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb0404c3c0 con 0x7fcb04159c60 2023-10-10T20:42:20.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.642+0000 7fcafaffd700 1 -- 172.21.15.89:0/1212044672 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcafc01d9e0 con 0x7fcb04159c60 2023-10-10T20:42:20.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.642+0000 7fcafaffd700 1 --2- 172.21.15.89:0/1212044672 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcae4067320 0x7fcae40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.642+0000 7fcafaffd700 1 -- 172.21.15.89:0/1212044672 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fcafc0862c0 con 0x7fcb04159c60 2023-10-10T20:42:20.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.642+0000 7fcb0259c700 1 --2- 172.21.15.89:0/1212044672 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcae4067320 0x7fcae40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.642+0000 7fcb0259c700 1 --2- 172.21.15.89:0/1212044672 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcae4067320 0x7fcae40697d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4006fd0 tx=0x7fcaf4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:20.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.647+0000 7fcafaffd700 1 -- 172.21.15.89:0/1212044672 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcafc053190 con 0x7fcb04159c60 2023-10-10T20:42:20.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.832+0000 7fcb088b4700 1 -- 172.21.15.89:0/1212044672 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb04069d00 con 0x7fcb04159c60 2023-10-10T20:42:20.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.832+0000 7fcafaffd700 1 -- 172.21.15.89:0/1212044672 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+12144 (secure 0 0 0) 0x7fcafc0567b0 con 0x7fcb04159c60 2023-10-10T20:42:20.836 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:20.836 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":149,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:42:20.067306+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:42:20.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.835+0000 7fcb088b4700 1 -- 172.21.15.89:0/1212044672 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcae4067320 msgr2=0x7fcae40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:20.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.835+0000 7fcb088b4700 1 --2- 172.21.15.89:0/1212044672 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcae4067320 0x7fcae40697d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4006fd0 tx=0x7fcaf4008040 comp rx=0 tx=0).stop 2023-10-10T20:42:20.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.835+0000 7fcb088b4700 1 -- 172.21.15.89:0/1212044672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb04159c60 msgr2=0x7fcb0416d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:20.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.835+0000 7fcb088b4700 1 --2- 172.21.15.89:0/1212044672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb04159c60 0x7fcb0416d930 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fcb040899c0 tx=0x7fcafc017000 comp rx=0 tx=0).stop 2023-10-10T20:42:20.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.835+0000 7fcb088b4700 1 -- 172.21.15.89:0/1212044672 shutdown_connections 2023-10-10T20:42:20.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.835+0000 7fcb088b4700 1 --2- 172.21.15.89:0/1212044672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb04159c60 0x7fcb0416d930 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.835+0000 7fcb088b4700 1 --2- 172.21.15.89:0/1212044672 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb04069560 0x7fcb04153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.835+0000 7fcb088b4700 1 --2- 172.21.15.89:0/1212044672 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcae4067320 0x7fcae40697d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.835+0000 7fcb088b4700 1 --2- 172.21.15.89:0/1212044672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb04159890 0x7fcb0416d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.835+0000 7fcb088b4700 1 -- 172.21.15.89:0/1212044672 >> 172.21.15.89:0/1212044672 conn(0x7fcb04068e20 msgr2=0x7fcb04156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:20.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.835+0000 7fcb088b4700 1 -- 172.21.15.89:0/1212044672 shutdown_connections 2023-10-10T20:42:20.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.835+0000 7fcb088b4700 1 -- 172.21.15.89:0/1212044672 wait complete. 2023-10-10T20:42:20.850 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:42:20.851 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:42:20.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.965+0000 7ff9f5470700 1 Processor -- start 2023-10-10T20:42:20.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.965+0000 7ff9f5470700 1 -- start start 2023-10-10T20:42:20.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.966+0000 7ff9f5470700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff9f0159890 0x7ff9f015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.966+0000 7ff9f5470700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9f0069560 0x7ff9f00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.966+0000 7ff9f5470700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f0069ee0 con 0x7ff9f0159c60 2023-10-10T20:42:20.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.966+0000 7ff9f5470700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f015a160 con 0x7ff9f0159890 2023-10-10T20:42:20.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.966+0000 7ff9f5470700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f00d13a0 con 0x7ff9f0069560 2023-10-10T20:42:20.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.966+0000 7ff9e67fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff9f0159c60 0x7ff9f015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48926/0 (socket says 172.21.15.89:48926) 2023-10-10T20:42:20.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.966+0000 7ff9e67fc700 1 -- 172.21.15.89:0/236343762 learned_addr learned my addr 172.21.15.89:0/236343762 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:20.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.966+0000 7ff9ee7fc700 1 -- 172.21.15.89:0/236343762 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041284615 0 0) 0x7ff9f0069ee0 con 0x7ff9f0159c60 2023-10-10T20:42:20.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.966+0000 7ff9ef7fe700 1 --2- 172.21.15.89:0/236343762 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9f0069560 0x7ff9f00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.966+0000 7ff9ee7fc700 1 -- 172.21.15.89:0/236343762 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9dc003570 con 0x7ff9f0159c60 2023-10-10T20:42:20.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.966+0000 7ff9eeffd700 1 --2- 172.21.15.89:0/236343762 >> v2:172.21.15.111:3300/0 conn(0x7ff9f0159890 0x7ff9f015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.966+0000 7ff9ee7fc700 1 -- 172.21.15.89:0/236343762 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3695485050 0 0) 0x7ff9dc003570 con 0x7ff9f0159c60 2023-10-10T20:42:20.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.966+0000 7ff9ee7fc700 1 -- 172.21.15.89:0/236343762 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9f0069560 msgr2=0x7ff9f00699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:20.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.966+0000 7ff9ee7fc700 1 --2- 172.21.15.89:0/236343762 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9f0069560 0x7ff9f00699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.966+0000 7ff9ee7fc700 1 -- 172.21.15.89:0/236343762 >> v2:172.21.15.111:3300/0 conn(0x7ff9f0159890 msgr2=0x7ff9f015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:20.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.966+0000 7ff9ee7fc700 1 --2- 172.21.15.89:0/236343762 >> v2:172.21.15.111:3300/0 conn(0x7ff9f0159890 0x7ff9f015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.966+0000 7ff9ee7fc700 1 -- 172.21.15.89:0/236343762 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f015e140 con 0x7ff9f0159c60 2023-10-10T20:42:20.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.966+0000 7ff9ee7fc700 1 -- 172.21.15.89:0/236343762 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7ff9d8001470 con 0x7ff9f0159c60 2023-10-10T20:42:20.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9ee7fc700 1 -- 172.21.15.89:0/236343762 >> v1:172.21.15.89:6789/0 conn(0x7ff9f0159c60 legacy=0x7ff9f015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:20.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9ee7fc700 1 --2- 172.21.15.89:0/236343762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9dc003a60 0x7ff9dc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9ee7fc700 1 --2- 172.21.15.89:0/236343762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9dc004810 0x7ff9dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9ee7fc700 1 --2- 172.21.15.89:0/236343762 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9f0069560 0x7ff9dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9ee7fc700 1 -- 172.21.15.89:0/236343762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f015a160 con 0x7ff9dc003a60 2023-10-10T20:42:20.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9ee7fc700 1 -- 172.21.15.89:0/236343762 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f00d13a0 con 0x7ff9dc004810 2023-10-10T20:42:20.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9ee7fc700 1 -- 172.21.15.89:0/236343762 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc00bb10 con 0x7ff9f0069560 2023-10-10T20:42:20.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9eeffd700 1 --2- 172.21.15.89:0/236343762 >> v2:172.21.15.111:3300/0 conn(0x7ff9f0159890 0x7ff9f015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:20.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9eeffd700 1 --2- 172.21.15.89:0/236343762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9dc003a60 0x7ff9dc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9ef7fe700 1 --2- 172.21.15.89:0/236343762 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9f0069560 0x7ff9dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9e67fc700 1 --2- 172.21.15.89:0/236343762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9dc004810 0x7ff9dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9eeffd700 1 -- 172.21.15.89:0/236343762 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9f0069560 msgr2=0x7ff9dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:20.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9eeffd700 1 --2- 172.21.15.89:0/236343762 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9f0069560 0x7ff9dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9eeffd700 1 -- 172.21.15.89:0/236343762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9dc004810 msgr2=0x7ff9dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:20.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9eeffd700 1 --2- 172.21.15.89:0/236343762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9dc004810 0x7ff9dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9eeffd700 1 -- 172.21.15.89:0/236343762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9dc003570 con 0x7ff9dc003a60 2023-10-10T20:42:20.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9eeffd700 1 --2- 172.21.15.89:0/236343762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9dc003a60 0x7ff9dc003e70 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009ba0 tx=0x7ff9e0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b10088d8709a81 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:20.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9ee7fc700 1 -- 172.21.15.89:0/236343762 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e000c070 con 0x7ff9dc003a60 2023-10-10T20:42:20.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9ee7fc700 1 -- 172.21.15.89:0/236343762 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e000e5e0 con 0x7ff9dc003a60 2023-10-10T20:42:20.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.967+0000 7ff9ee7fc700 1 -- 172.21.15.89:0/236343762 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e000e790 con 0x7ff9dc003a60 2023-10-10T20:42:20.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.968+0000 7ff9f5470700 1 -- 172.21.15.89:0/236343762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9dc003a60 msgr2=0x7ff9dc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:20.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.968+0000 7ff9f5470700 1 --2- 172.21.15.89:0/236343762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9dc003a60 0x7ff9dc003e70 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009ba0 tx=0x7ff9e0008040 comp rx=0 tx=0).stop 2023-10-10T20:42:20.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.968+0000 7ff9ef7fe700 1 -- 172.21.15.89:0/236343762 reap_dead start 2023-10-10T20:42:20.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.968+0000 7ff9f5470700 1 -- 172.21.15.89:0/236343762 shutdown_connections 2023-10-10T20:42:20.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.968+0000 7ff9f5470700 1 -- 172.21.15.89:0/236343762 >> 172.21.15.89:0/236343762 conn(0x7ff9f0068e20 msgr2=0x7ff9f00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:20.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.968+0000 7ff9f5470700 1 -- 172.21.15.89:0/236343762 shutdown_connections 2023-10-10T20:42:20.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.968+0000 7ff9f5470700 1 -- 172.21.15.89:0/236343762 wait complete. 2023-10-10T20:42:20.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.969+0000 7ff9f5470700 1 Processor -- start 2023-10-10T20:42:20.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.969+0000 7ff9f5470700 1 -- start start 2023-10-10T20:42:20.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.970+0000 7ff9f5470700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9f0152340 0x7ff9f00cf160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.970+0000 7ff9f5470700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9f0155c20 0x7ff9f013c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.970+0000 7ff9f5470700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9f00cf6a0 0x7ff9f013c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.970+0000 7ff9f5470700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f00b3c80 con 0x7ff9f00cf6a0 2023-10-10T20:42:20.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.970+0000 7ff9f5470700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f00cfee0 con 0x7ff9f0155c20 2023-10-10T20:42:20.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.970+0000 7ff9f5470700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f00cfc00 con 0x7ff9f0152340 2023-10-10T20:42:20.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.970+0000 7ff9ef7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9f00cf6a0 0x7ff9f013c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.970+0000 7ff9ef7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9f00cf6a0 0x7ff9f013c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44586/0 (socket says 172.21.15.89:44586) 2023-10-10T20:42:20.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.970+0000 7ff9ef7fe700 1 -- 172.21.15.89:0/4130868649 learned_addr learned my addr 172.21.15.89:0/4130868649 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:20.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.970+0000 7ff9eeffd700 1 --2- 172.21.15.89:0/4130868649 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9f0152340 0x7ff9f00cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.970+0000 7ff9e67fc700 1 --2- 172.21.15.89:0/4130868649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9f0155c20 0x7ff9f013c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.970+0000 7ff9ef7fe700 1 -- 172.21.15.89:0/4130868649 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9f0152340 msgr2=0x7ff9f00cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:20.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.970+0000 7ff9ef7fe700 1 --2- 172.21.15.89:0/4130868649 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9f0152340 0x7ff9f00cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.970+0000 7ff9ef7fe700 1 -- 172.21.15.89:0/4130868649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9f0155c20 msgr2=0x7ff9f013c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:20.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.970+0000 7ff9ef7fe700 1 --2- 172.21.15.89:0/4130868649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9f0155c20 0x7ff9f013c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:20.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.970+0000 7ff9ef7fe700 1 -- 172.21.15.89:0/4130868649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f00b3630 con 0x7ff9f00cf6a0 2023-10-10T20:42:20.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.970+0000 7ff9e67fc700 1 --2- 172.21.15.89:0/4130868649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9f0155c20 0x7ff9f013c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:20.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.970+0000 7ff9ef7fe700 1 --2- 172.21.15.89:0/4130868649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9f00cf6a0 0x7ff9f013c5c0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00d03b0 tx=0x7ff9e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:20.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.970+0000 7ff9e77fe700 1 -- 172.21.15.89:0/4130868649 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e800e040 con 0x7ff9f00cf6a0 2023-10-10T20:42:20.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.971+0000 7ff9e77fe700 1 -- 172.21.15.89:0/4130868649 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e800f040 con 0x7ff9f00cf6a0 2023-10-10T20:42:20.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.971+0000 7ff9e77fe700 1 -- 172.21.15.89:0/4130868649 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e800fcb0 con 0x7ff9f00cf6a0 2023-10-10T20:42:20.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.971+0000 7ff9f5470700 1 -- 172.21.15.89:0/4130868649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f00b3910 con 0x7ff9f00cf6a0 2023-10-10T20:42:20.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.971+0000 7ff9f5470700 1 -- 172.21.15.89:0/4130868649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f0152710 con 0x7ff9f00cf6a0 2023-10-10T20:42:20.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.971+0000 7ff9f5470700 1 -- 172.21.15.89:0/4130868649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f00b3df0 con 0x7ff9f00cf6a0 2023-10-10T20:42:20.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.972+0000 7ff9e77fe700 1 -- 172.21.15.89:0/4130868649 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff9e8015640 con 0x7ff9f00cf6a0 2023-10-10T20:42:20.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.972+0000 7ff9e77fe700 1 --2- 172.21.15.89:0/4130868649 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff9d0067350 0x7ff9d0069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:20.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.972+0000 7ff9e77fe700 1 -- 172.21.15.89:0/4130868649 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff9e80851c0 con 0x7ff9f00cf6a0 2023-10-10T20:42:20.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.972+0000 7ff9eeffd700 1 --2- 172.21.15.89:0/4130868649 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff9d0067350 0x7ff9d0069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:20.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.973+0000 7ff9eeffd700 1 --2- 172.21.15.89:0/4130868649 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff9d0067350 0x7ff9d0069800 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0003080 tx=0x7ff9e001a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:20.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:20.977+0000 7ff9e77fe700 1 -- 172.21.15.89:0/4130868649 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9e8052050 con 0x7ff9f00cf6a0 2023-10-10T20:42:21.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.159+0000 7ff9f5470700 1 -- 172.21.15.89:0/4130868649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9f004c3c0 con 0x7ff9f00cf6a0 2023-10-10T20:42:21.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.159+0000 7ff9e77fe700 1 -- 172.21.15.89:0/4130868649 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v149) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9e8055670 con 0x7ff9f00cf6a0 2023-10-10T20:42:21.163 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:42:21.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.161+0000 7ff9f5470700 1 -- 172.21.15.89:0/4130868649 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff9d0067350 msgr2=0x7ff9d0069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:21.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.161+0000 7ff9f5470700 1 --2- 172.21.15.89:0/4130868649 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff9d0067350 0x7ff9d0069800 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0003080 tx=0x7ff9e001a000 comp rx=0 tx=0).stop 2023-10-10T20:42:21.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.161+0000 7ff9f5470700 1 -- 172.21.15.89:0/4130868649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9f00cf6a0 msgr2=0x7ff9f013c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:21.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.161+0000 7ff9f5470700 1 --2- 172.21.15.89:0/4130868649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9f00cf6a0 0x7ff9f013c5c0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00d03b0 tx=0x7ff9e800b040 comp rx=0 tx=0).stop 2023-10-10T20:42:21.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.161+0000 7ff9f5470700 1 -- 172.21.15.89:0/4130868649 shutdown_connections 2023-10-10T20:42:21.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.161+0000 7ff9f5470700 1 --2- 172.21.15.89:0/4130868649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9f00cf6a0 0x7ff9f013c5c0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.161+0000 7ff9f5470700 1 --2- 172.21.15.89:0/4130868649 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff9f0152340 0x7ff9f00cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.161+0000 7ff9f5470700 1 --2- 172.21.15.89:0/4130868649 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff9d0067350 0x7ff9d0069800 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.161+0000 7ff9f5470700 1 --2- 172.21.15.89:0/4130868649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9f0155c20 0x7ff9f013c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.161+0000 7ff9f5470700 1 -- 172.21.15.89:0/4130868649 >> 172.21.15.89:0/4130868649 conn(0x7ff9f0068e20 msgr2=0x7ff9f00c7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:21.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.161+0000 7ff9f5470700 1 -- 172.21.15.89:0/4130868649 shutdown_connections 2023-10-10T20:42:21.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.161+0000 7ff9f5470700 1 -- 172.21.15.89:0/4130868649 wait complete. 2023-10-10T20:42:21.177 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:42:21.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.289+0000 7fb48bd48700 1 Processor -- start 2023-10-10T20:42:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.289+0000 7fb48bd48700 1 -- start start 2023-10-10T20:42:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.289+0000 7fb48bd48700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb484159890 0x7fb48415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:21.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.289+0000 7fb48bd48700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb484069560 0x7fb4840699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:21.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.289+0000 7fb48bd48700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb484069ee0 con 0x7fb484159c60 2023-10-10T20:42:21.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.289+0000 7fb48bd48700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48415a160 con 0x7fb484159890 2023-10-10T20:42:21.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.289+0000 7fb48bd48700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4840d13a0 con 0x7fb484069560 2023-10-10T20:42:21.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.289+0000 7fb489ae4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb484159890 0x7fb48415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.289+0000 7fb4892e3700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb484159c60 0x7fb48415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48938/0 (socket says 172.21.15.89:48938) 2023-10-10T20:42:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.289+0000 7fb489ae4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb484159890 0x7fb48415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44594/0 (socket says 172.21.15.89:44594) 2023-10-10T20:42:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.289+0000 7fb4892e3700 1 -- 172.21.15.89:0/4029080736 learned_addr learned my addr 172.21.15.89:0/4029080736 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.290+0000 7fb48a2e5700 1 --2- 172.21.15.89:0/4029080736 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb484069560 0x7fb4840699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.290+0000 7fb489ae4700 1 -- 172.21.15.89:0/4029080736 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb484069560 msgr2=0x7fb4840699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:21.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.290+0000 7fb489ae4700 1 --2- 172.21.15.89:0/4029080736 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb484069560 0x7fb4840699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.290+0000 7fb489ae4700 1 -- 172.21.15.89:0/4029080736 >> v1:172.21.15.89:6789/0 conn(0x7fb484159c60 legacy=0x7fb48415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:21.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.290+0000 7fb488ae2700 1 -- 172.21.15.89:0/4029080736 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026239287 0 0) 0x7fb484069ee0 con 0x7fb484159c60 2023-10-10T20:42:21.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.290+0000 7fb489ae4700 1 -- 172.21.15.89:0/4029080736 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48415e140 con 0x7fb484159890 2023-10-10T20:42:21.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.290+0000 7fb489ae4700 1 --2- 172.21.15.89:0/4029080736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb484159890 0x7fb48415abd0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fb4800098e0 tx=0x7fb4800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9caf0533da43894b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:21.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.290+0000 7fb488ae2700 1 -- 172.21.15.89:0/4029080736 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48000b640 con 0x7fb484159890 2023-10-10T20:42:21.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.290+0000 7fb488ae2700 1 -- 172.21.15.89:0/4029080736 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48000b7a0 con 0x7fb484159890 2023-10-10T20:42:21.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.290+0000 7fb488ae2700 1 -- 172.21.15.89:0/4029080736 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48000bb40 con 0x7fb484159890 2023-10-10T20:42:21.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.290+0000 7fb48bd48700 1 -- 172.21.15.89:0/4029080736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb484159890 msgr2=0x7fb48415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:21.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.290+0000 7fb48bd48700 1 --2- 172.21.15.89:0/4029080736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb484159890 0x7fb48415abd0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fb4800098e0 tx=0x7fb4800091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.290+0000 7fb48bd48700 1 -- 172.21.15.89:0/4029080736 shutdown_connections 2023-10-10T20:42:21.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.290+0000 7fb48bd48700 1 --2- 172.21.15.89:0/4029080736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb484159890 0x7fb48415abd0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.290+0000 7fb48bd48700 1 --2- 172.21.15.89:0/4029080736 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb484069560 0x7fb4840699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.290+0000 7fb48bd48700 1 -- 172.21.15.89:0/4029080736 >> 172.21.15.89:0/4029080736 conn(0x7fb484068e20 msgr2=0x7fb4840d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:21.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.291+0000 7fb48bd48700 1 -- 172.21.15.89:0/4029080736 shutdown_connections 2023-10-10T20:42:21.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.291+0000 7fb48bd48700 1 -- 172.21.15.89:0/4029080736 wait complete. 2023-10-10T20:42:21.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.292+0000 7fb48bd48700 1 Processor -- start 2023-10-10T20:42:21.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.292+0000 7fb48bd48700 1 -- start start 2023-10-10T20:42:21.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.292+0000 7fb48bd48700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb484069560 0x7fb4840b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:21.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.292+0000 7fb48bd48700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb484159890 0x7fb4840cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:21.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.292+0000 7fb48bd48700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb484159c60 0x7fb484149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:21.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.292+0000 7fb48bd48700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4840b6520 con 0x7fb484159890 2023-10-10T20:42:21.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.292+0000 7fb48bd48700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4840c7ec0 con 0x7fb484069560 2023-10-10T20:42:21.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.292+0000 7fb48bd48700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4840cd880 con 0x7fb484159c60 2023-10-10T20:42:21.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.292+0000 7fb4892e3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb484159890 0x7fb4840cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:21.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.293+0000 7fb4892e3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb484159890 0x7fb4840cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44606/0 (socket says 172.21.15.89:44606) 2023-10-10T20:42:21.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.293+0000 7fb4892e3700 1 -- 172.21.15.89:0/1353428031 learned_addr learned my addr 172.21.15.89:0/1353428031 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:21.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.293+0000 7fb48a2e5700 1 --2- 172.21.15.89:0/1353428031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb484159c60 0x7fb484149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:21.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.293+0000 7fb489ae4700 1 --2- 172.21.15.89:0/1353428031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb484069560 0x7fb4840b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:21.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.293+0000 7fb4892e3700 1 -- 172.21.15.89:0/1353428031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb484159c60 msgr2=0x7fb484149390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:21.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.293+0000 7fb4892e3700 1 --2- 172.21.15.89:0/1353428031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb484159c60 0x7fb484149390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.293+0000 7fb4892e3700 1 -- 172.21.15.89:0/1353428031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb484069560 msgr2=0x7fb4840b5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:21.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.293+0000 7fb4892e3700 1 --2- 172.21.15.89:0/1353428031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb484069560 0x7fb4840b5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.293+0000 7fb4892e3700 1 -- 172.21.15.89:0/1353428031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb480009420 con 0x7fb484159890 2023-10-10T20:42:21.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.293+0000 7fb4892e3700 1 --2- 172.21.15.89:0/1353428031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb484159890 0x7fb4840cd340 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fb47400c810 tx=0x7fb474007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:21.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.293+0000 7fb479ffb700 1 -- 172.21.15.89:0/1353428031 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb474005b30 con 0x7fb484159890 2023-10-10T20:42:21.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.293+0000 7fb479ffb700 1 -- 172.21.15.89:0/1353428031 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb474005c90 con 0x7fb484159890 2023-10-10T20:42:21.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.293+0000 7fb479ffb700 1 -- 172.21.15.89:0/1353428031 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb474007300 con 0x7fb484159890 2023-10-10T20:42:21.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.293+0000 7fb48bd48700 1 -- 172.21.15.89:0/1353428031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4840c7940 con 0x7fb484159890 2023-10-10T20:42:21.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.293+0000 7fb48bd48700 1 -- 172.21.15.89:0/1353428031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4840c9860 con 0x7fb484159890 2023-10-10T20:42:21.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.294+0000 7fb48bd48700 1 -- 172.21.15.89:0/1353428031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4840c9a80 con 0x7fb484159890 2023-10-10T20:42:21.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.295+0000 7fb479ffb700 1 -- 172.21.15.89:0/1353428031 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb47401b3b0 con 0x7fb484159890 2023-10-10T20:42:21.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.295+0000 7fb479ffb700 1 --2- 172.21.15.89:0/1353428031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb468067320 0x7fb4680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:21.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.295+0000 7fb479ffb700 1 -- 172.21.15.89:0/1353428031 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb474083ec0 con 0x7fb484159890 2023-10-10T20:42:21.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.295+0000 7fb489ae4700 1 --2- 172.21.15.89:0/1353428031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb468067320 0x7fb4680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:21.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.295+0000 7fb489ae4700 1 --2- 172.21.15.89:0/1353428031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb468067320 0x7fb4680697d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb4800093f0 tx=0x7fb480017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:21.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.299+0000 7fb479ffb700 1 -- 172.21.15.89:0/1353428031 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb474050ea0 con 0x7fb484159890 2023-10-10T20:42:21.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.485+0000 7fb48bd48700 1 -- 172.21.15.89:0/1353428031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb48404c3c0 con 0x7fb484159890 2023-10-10T20:42:21.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.485+0000 7fb479ffb700 1 -- 172.21.15.89:0/1353428031 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v149) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb474017020 con 0x7fb484159890 2023-10-10T20:42:21.488 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:21.488 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:42:21.488 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:21.488 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2023-10-10T20:42:21.489 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2023-10-10T20:42:21.489 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:21.489 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:42:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.488+0000 7fb48bd48700 1 -- 172.21.15.89:0/1353428031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb468067320 msgr2=0x7fb4680697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.488+0000 7fb48bd48700 1 --2- 172.21.15.89:0/1353428031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb468067320 0x7fb4680697d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb4800093f0 tx=0x7fb480017000 comp rx=0 tx=0).stop 2023-10-10T20:42:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.488+0000 7fb48bd48700 1 -- 172.21.15.89:0/1353428031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb484159890 msgr2=0x7fb4840cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.488+0000 7fb48bd48700 1 --2- 172.21.15.89:0/1353428031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb484159890 0x7fb4840cd340 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fb47400c810 tx=0x7fb474007df0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.488+0000 7fb48bd48700 1 -- 172.21.15.89:0/1353428031 shutdown_connections 2023-10-10T20:42:21.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.488+0000 7fb48bd48700 1 --2- 172.21.15.89:0/1353428031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb484159c60 0x7fb484149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.488+0000 7fb48bd48700 1 --2- 172.21.15.89:0/1353428031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb468067320 0x7fb4680697d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.488+0000 7fb48bd48700 1 --2- 172.21.15.89:0/1353428031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb484069560 0x7fb4840b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.488+0000 7fb48bd48700 1 --2- 172.21.15.89:0/1353428031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb484159890 0x7fb4840cd340 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.488+0000 7fb48bd48700 1 -- 172.21.15.89:0/1353428031 >> 172.21.15.89:0/1353428031 conn(0x7fb484068e20 msgr2=0x7fb484153300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.488+0000 7fb48bd48700 1 -- 172.21.15.89:0/1353428031 shutdown_connections 2023-10-10T20:42:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.488+0000 7fb48bd48700 1 -- 172.21.15.89:0/1353428031 wait complete. 2023-10-10T20:42:21.502 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:42:21.502 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:42:21.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.618+0000 7ff512d9d700 1 Processor -- start 2023-10-10T20:42:21.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.619+0000 7ff512d9d700 1 -- start start 2023-10-10T20:42:21.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.619+0000 7ff512d9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff50c0685e0 0x7ff50c15a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:21.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.619+0000 7ff512d9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff50c15d3f0 0x7ff50c1538f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:21.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.619+0000 7ff512d9d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff50c0c9480 con 0x7ff50c0689b0 2023-10-10T20:42:21.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.619+0000 7ff512d9d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff50c15d880 con 0x7ff50c0685e0 2023-10-10T20:42:21.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.619+0000 7ff512d9d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c15db60 con 0x7ff50c15d3f0 2023-10-10T20:42:21.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.620+0000 7ff51159a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff50c15d3f0 0x7ff50c1538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.620+0000 7ff511d9b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff50c0685e0 0x7ff50c15a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.620+0000 7ff51159a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff50c15d3f0 0x7ff50c1538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:60978/0 (socket says 172.21.15.89:60978) 2023-10-10T20:42:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.620+0000 7ff51159a700 1 -- 172.21.15.89:0/2573677286 learned_addr learned my addr 172.21.15.89:0/2573677286 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.620+0000 7ff51159a700 1 -- 172.21.15.89:0/2573677286 >> v2:172.21.15.111:3300/0 conn(0x7ff50c0685e0 msgr2=0x7ff50c15a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.620+0000 7ff51159a700 1 --2- 172.21.15.89:0/2573677286 >> v2:172.21.15.111:3300/0 conn(0x7ff50c0685e0 0x7ff50c15a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.620+0000 7ff51159a700 1 -- 172.21.15.89:0/2573677286 >> v1:172.21.15.111:6789/0 conn(0x7ff50c0689b0 legacy=0x7ff50c06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:21.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.620+0000 7ff51159a700 1 -- 172.21.15.89:0/2573677286 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c15de40 con 0x7ff50c15d3f0 2023-10-10T20:42:21.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.620+0000 7ff511d9b700 1 --2- 172.21.15.89:0/2573677286 >> v2:172.21.15.111:3300/0 conn(0x7ff50c0685e0 0x7ff50c15a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:42:21.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.620+0000 7ff51159a700 1 --2- 172.21.15.89:0/2573677286 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff50c15d3f0 0x7ff50c1538f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009990 tx=0x7ff4fc009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fe33d2d0028ab server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:21.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.621+0000 7ff510d99700 1 -- 172.21.15.89:0/2573677286 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc009e80 con 0x7ff50c15d3f0 2023-10-10T20:42:21.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.621+0000 7ff510d99700 1 -- 172.21.15.89:0/2573677286 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc005c10 con 0x7ff50c15d3f0 2023-10-10T20:42:21.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.621+0000 7ff510d99700 1 -- 172.21.15.89:0/2573677286 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00b6e0 con 0x7ff50c15d3f0 2023-10-10T20:42:21.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.621+0000 7ff512d9d700 1 -- 172.21.15.89:0/2573677286 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff50c15d3f0 msgr2=0x7ff50c1538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:21.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.621+0000 7ff512d9d700 1 --2- 172.21.15.89:0/2573677286 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff50c15d3f0 0x7ff50c1538f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009990 tx=0x7ff4fc009220 comp rx=0 tx=0).stop 2023-10-10T20:42:21.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.621+0000 7ff512d9d700 1 -- 172.21.15.89:0/2573677286 shutdown_connections 2023-10-10T20:42:21.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.621+0000 7ff512d9d700 1 --2- 172.21.15.89:0/2573677286 >> v2:172.21.15.111:3300/0 conn(0x7ff50c0685e0 0x7ff50c15a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.621+0000 7ff512d9d700 1 --2- 172.21.15.89:0/2573677286 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff50c15d3f0 0x7ff50c1538f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.621+0000 7ff512d9d700 1 -- 172.21.15.89:0/2573677286 >> 172.21.15.89:0/2573677286 conn(0x7ff50c06b420 msgr2=0x7ff50c0cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:21.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.621+0000 7ff512d9d700 1 -- 172.21.15.89:0/2573677286 shutdown_connections 2023-10-10T20:42:21.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.621+0000 7ff512d9d700 1 -- 172.21.15.89:0/2573677286 wait complete. 2023-10-10T20:42:21.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.622+0000 7ff512d9d700 1 Processor -- start 2023-10-10T20:42:21.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.622+0000 7ff512d9d700 1 -- start start 2023-10-10T20:42:21.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.623+0000 7ff512d9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff50c0685e0 0x7ff50c149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:21.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.623+0000 7ff512d9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff50c0689b0 0x7ff50c0c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:21.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.623+0000 7ff512d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff50c15d3f0 0x7ff50c0c3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:21.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.623+0000 7ff512d9d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c14a4b0 con 0x7ff50c15d3f0 2023-10-10T20:42:21.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.623+0000 7ff512d9d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c0c5dc0 con 0x7ff50c0689b0 2023-10-10T20:42:21.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.623+0000 7ff512d9d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c0b1760 con 0x7ff50c0685e0 2023-10-10T20:42:21.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.623+0000 7ff51259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff50c15d3f0 0x7ff50c0c3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:21.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.623+0000 7ff51259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff50c15d3f0 0x7ff50c0c3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44618/0 (socket says 172.21.15.89:44618) 2023-10-10T20:42:21.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.623+0000 7ff51259c700 1 -- 172.21.15.89:0/768832348 learned_addr learned my addr 172.21.15.89:0/768832348 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:21.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.623+0000 7ff511d9b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff50c0685e0 0x7ff50c149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:21.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.623+0000 7ff51159a700 1 --2- 172.21.15.89:0/768832348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff50c0689b0 0x7ff50c0c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:21.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.623+0000 7ff51259c700 1 -- 172.21.15.89:0/768832348 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff50c0685e0 msgr2=0x7ff50c149f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:21.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.623+0000 7ff51259c700 1 --2- 172.21.15.89:0/768832348 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff50c0685e0 0x7ff50c149f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.623+0000 7ff51259c700 1 -- 172.21.15.89:0/768832348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff50c0689b0 msgr2=0x7ff50c0c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:21.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.623+0000 7ff51259c700 1 --2- 172.21.15.89:0/768832348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff50c0689b0 0x7ff50c0c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:21.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.623+0000 7ff51259c700 1 -- 172.21.15.89:0/768832348 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff508009a20 con 0x7ff50c15d3f0 2023-10-10T20:42:21.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.623+0000 7ff51259c700 1 --2- 172.21.15.89:0/768832348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff50c15d3f0 0x7ff50c0c3390 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7ff50400c560 tx=0x7ff504007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:21.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.624+0000 7ff501ffb700 1 -- 172.21.15.89:0/768832348 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504011590 con 0x7ff50c15d3f0 2023-10-10T20:42:21.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.624+0000 7ff501ffb700 1 -- 172.21.15.89:0/768832348 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5040116f0 con 0x7ff50c15d3f0 2023-10-10T20:42:21.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.624+0000 7ff512d9d700 1 -- 172.21.15.89:0/768832348 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4fc009480 con 0x7ff50c15d3f0 2023-10-10T20:42:21.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.624+0000 7ff501ffb700 1 -- 172.21.15.89:0/768832348 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504011ab0 con 0x7ff50c15d3f0 2023-10-10T20:42:21.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.624+0000 7ff512d9d700 1 -- 172.21.15.89:0/768832348 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50c0c5ae0 con 0x7ff50c15d3f0 2023-10-10T20:42:21.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.625+0000 7ff512d9d700 1 -- 172.21.15.89:0/768832348 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50c04c3c0 con 0x7ff50c15d3f0 2023-10-10T20:42:21.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.625+0000 7ff501ffb700 1 -- 172.21.15.89:0/768832348 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff50401b3b0 con 0x7ff50c15d3f0 2023-10-10T20:42:21.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.626+0000 7ff501ffb700 1 --2- 172.21.15.89:0/768832348 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff4f0067320 0x7ff4f00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:21.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.626+0000 7ff511d9b700 1 --2- 172.21.15.89:0/768832348 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff4f0067320 0x7ff4f00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:21.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.626+0000 7ff501ffb700 1 -- 172.21.15.89:0/768832348 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff504083c90 con 0x7ff50c15d3f0 2023-10-10T20:42:21.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.626+0000 7ff511d9b700 1 --2- 172.21.15.89:0/768832348 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff4f0067320 0x7ff4f00697d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff508009fd0 tx=0x7ff50800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:21.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.630+0000 7ff501ffb700 1 -- 172.21.15.89:0/768832348 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff504050c70 con 0x7ff50c15d3f0 2023-10-10T20:42:21.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:21.813+0000 7ff512d9d700 1 -- 172.21.15.89:0/768832348 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff50c06f8e0 con 0x7ff50c15d3f0 2023-10-10T20:42:22.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.086+0000 7ff501ffb700 1 -- 172.21.15.89:0/768832348 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v150) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff504017070 con 0x7ff50c15d3f0 2023-10-10T20:42:22.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.089+0000 7ff512d9d700 1 -- 172.21.15.89:0/768832348 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff4f0067320 msgr2=0x7ff4f00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:22.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.089+0000 7ff512d9d700 1 --2- 172.21.15.89:0/768832348 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff4f0067320 0x7ff4f00697d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff508009fd0 tx=0x7ff50800b040 comp rx=0 tx=0).stop 2023-10-10T20:42:22.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.089+0000 7ff512d9d700 1 -- 172.21.15.89:0/768832348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff50c15d3f0 msgr2=0x7ff50c0c3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:22.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.089+0000 7ff512d9d700 1 --2- 172.21.15.89:0/768832348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff50c15d3f0 0x7ff50c0c3390 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7ff50400c560 tx=0x7ff504007ef0 comp rx=0 tx=0).stop 2023-10-10T20:42:22.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.089+0000 7ff512d9d700 1 -- 172.21.15.89:0/768832348 shutdown_connections 2023-10-10T20:42:22.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.089+0000 7ff512d9d700 1 --2- 172.21.15.89:0/768832348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff50c15d3f0 0x7ff50c0c3390 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:22.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.089+0000 7ff512d9d700 1 --2- 172.21.15.89:0/768832348 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff50c0685e0 0x7ff50c149f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:22.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.089+0000 7ff512d9d700 1 --2- 172.21.15.89:0/768832348 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff4f0067320 0x7ff4f00697d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:22.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.089+0000 7ff512d9d700 1 --2- 172.21.15.89:0/768832348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff50c0689b0 0x7ff50c0c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:22.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.089+0000 7ff512d9d700 1 -- 172.21.15.89:0/768832348 >> 172.21.15.89:0/768832348 conn(0x7ff50c06b420 msgr2=0x7ff50c0c8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:22.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.089+0000 7ff512d9d700 1 -- 172.21.15.89:0/768832348 shutdown_connections 2023-10-10T20:42:22.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.089+0000 7ff512d9d700 1 -- 172.21.15.89:0/768832348 wait complete. 2023-10-10T20:42:22.093 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2023-10-10T20:42:22.107 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:42:22.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.225+0000 7f824359e700 1 Processor -- start 2023-10-10T20:42:22.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.225+0000 7f824359e700 1 -- start start 2023-10-10T20:42:22.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.226+0000 7f824359e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82441595d0 0x7f82440ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:22.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.226+0000 7f824359e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82440b10f0 con 0x7f82440cf480 2023-10-10T20:42:22.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.226+0000 7f824359e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82440cd530 con 0x7f824415d930 2023-10-10T20:42:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.226+0000 7f824359e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824415e000 con 0x7f82441595d0 2023-10-10T20:42:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.226+0000 7f8239d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f82440cf480 0x7f824415d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48948/0 (socket says 172.21.15.89:48948) 2023-10-10T20:42:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.226+0000 7f8239d9b700 1 -- 172.21.15.89:0/4074364830 learned_addr learned my addr 172.21.15.89:0/4074364830 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.226+0000 7f8241d9b700 1 -- 172.21.15.89:0/4074364830 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926709961 0 0) 0x7f82440b10f0 con 0x7f82440cf480 2023-10-10T20:42:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8241d9b700 1 -- 172.21.15.89:0/4074364830 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8230003570 con 0x7f82440cf480 2023-10-10T20:42:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f824259c700 1 --2- 172.21.15.89:0/4074364830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82441595d0 0x7f82440ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8241d9b700 1 -- 172.21.15.89:0/4074364830 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3096484538 0 0) 0x7f8230003570 con 0x7f82440cf480 2023-10-10T20:42:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8241d9b700 1 -- 172.21.15.89:0/4074364830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82441595d0 msgr2=0x7f82440ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8241d9b700 1 --2- 172.21.15.89:0/4074364830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82441595d0 0x7f82440ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8241d9b700 1 -- 172.21.15.89:0/4074364830 >> v1:172.21.15.111:6789/0 conn(0x7f824415d930 legacy=0x7f82440b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:22.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8241d9b700 1 -- 172.21.15.89:0/4074364830 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82440cf990 con 0x7f82440cf480 2023-10-10T20:42:22.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8241d9b700 1 -- 172.21.15.89:0/4074364830 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f822c001470 con 0x7f82440cf480 2023-10-10T20:42:22.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8241d9b700 1 -- 172.21.15.89:0/4074364830 >> v1:172.21.15.89:6789/0 conn(0x7f82440cf480 legacy=0x7f824415d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:22.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8241d9b700 1 --2- 172.21.15.89:0/4074364830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8230003a90 0x7f8230003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:22.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8241d9b700 1 --2- 172.21.15.89:0/4074364830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8230004810 0x7f8230004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:22.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8241d9b700 1 --2- 172.21.15.89:0/4074364830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82441595d0 0x7f8230005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:22.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8241d9b700 1 -- 172.21.15.89:0/4074364830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824415e000 con 0x7f8230004810 2023-10-10T20:42:22.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8241d9b700 1 -- 172.21.15.89:0/4074364830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823000bc20 con 0x7f8230003a90 2023-10-10T20:42:22.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8241d9b700 1 -- 172.21.15.89:0/4074364830 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823000bde0 con 0x7f82441595d0 2023-10-10T20:42:22.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8239d9b700 1 --2- 172.21.15.89:0/4074364830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8230004810 0x7f8230004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:22.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f824259c700 1 --2- 172.21.15.89:0/4074364830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8230003a90 0x7f8230003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:22.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8242d9d700 1 --2- 172.21.15.89:0/4074364830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82441595d0 0x7f8230005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:22.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8239d9b700 1 -- 172.21.15.89:0/4074364830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82441595d0 msgr2=0x7f8230005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:22.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8239d9b700 1 --2- 172.21.15.89:0/4074364830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82441595d0 0x7f8230005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:22.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8239d9b700 1 -- 172.21.15.89:0/4074364830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8230003a90 msgr2=0x7f8230003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:22.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8239d9b700 1 --2- 172.21.15.89:0/4074364830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8230003a90 0x7f8230003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:22.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8239d9b700 1 -- 172.21.15.89:0/4074364830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8230003570 con 0x7f8230004810 2023-10-10T20:42:22.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8239d9b700 1 --2- 172.21.15.89:0/4074364830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8230004810 0x7f8230004c80 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f822c00c390 tx=0x7f822c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc21090de96d251 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:22.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.227+0000 7f8241d9b700 1 -- 172.21.15.89:0/4074364830 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c00cc60 con 0x7f8230004810 2023-10-10T20:42:22.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.228+0000 7f8241d9b700 1 -- 172.21.15.89:0/4074364830 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c00cdc0 con 0x7f8230004810 2023-10-10T20:42:22.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.228+0000 7f8241d9b700 1 -- 172.21.15.89:0/4074364830 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c005b70 con 0x7f8230004810 2023-10-10T20:42:22.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.228+0000 7f824359e700 1 -- 172.21.15.89:0/4074364830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8230004810 msgr2=0x7f8230004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:22.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.228+0000 7f824359e700 1 --2- 172.21.15.89:0/4074364830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8230004810 0x7f8230004c80 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f822c00c390 tx=0x7f822c001e50 comp rx=0 tx=0).stop 2023-10-10T20:42:22.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.228+0000 7f8242d9d700 1 -- 172.21.15.89:0/4074364830 reap_dead start 2023-10-10T20:42:22.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.228+0000 7f824359e700 1 -- 172.21.15.89:0/4074364830 shutdown_connections 2023-10-10T20:42:22.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.228+0000 7f824359e700 1 -- 172.21.15.89:0/4074364830 >> 172.21.15.89:0/4074364830 conn(0x7f82440d05f0 msgr2=0x7f82440d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:22.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.228+0000 7f824359e700 1 -- 172.21.15.89:0/4074364830 shutdown_connections 2023-10-10T20:42:22.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.228+0000 7f824359e700 1 -- 172.21.15.89:0/4074364830 wait complete. 2023-10-10T20:42:22.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.229+0000 7f824359e700 1 Processor -- start 2023-10-10T20:42:22.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.229+0000 7f824359e700 1 -- start start 2023-10-10T20:42:22.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.230+0000 7f824359e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8230004810 0x7f8244140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:22.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.230+0000 7f824359e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8244141430 0x7f82440c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:22.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.230+0000 7f824359e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8244140990 0x7f82440c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:22.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.230+0000 7f824359e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82440b4a90 con 0x7f8230004810 2023-10-10T20:42:22.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.230+0000 7f824359e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8244157c70 con 0x7f8244141430 2023-10-10T20:42:22.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.230+0000 7f824359e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8244157060 con 0x7f8244140990 2023-10-10T20:42:22.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.230+0000 7f824259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8230004810 0x7f8244140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:22.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.230+0000 7f824259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8230004810 0x7f8244140ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44632/0 (socket says 172.21.15.89:44632) 2023-10-10T20:42:22.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.230+0000 7f824259c700 1 -- 172.21.15.89:0/633927811 learned_addr learned my addr 172.21.15.89:0/633927811 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:22.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.230+0000 7f824259c700 1 -- 172.21.15.89:0/633927811 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8244140990 msgr2=0x7f82440c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:22.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.230+0000 7f824259c700 1 --2- 172.21.15.89:0/633927811 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8244140990 0x7f82440c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:22.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.230+0000 7f8239d9b700 1 --2- 172.21.15.89:0/633927811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8244141430 0x7f82440c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:22.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.230+0000 7f824259c700 1 -- 172.21.15.89:0/633927811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8244141430 msgr2=0x7f82440c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:22.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.230+0000 7f824259c700 1 --2- 172.21.15.89:0/633927811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8244141430 0x7f82440c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:22.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.230+0000 7f824259c700 1 -- 172.21.15.89:0/633927811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8244157340 con 0x7f8230004810 2023-10-10T20:42:22.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.231+0000 7f824259c700 1 --2- 172.21.15.89:0/633927811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8230004810 0x7f8244140ef0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f823400bbd0 tx=0x7f8234008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:22.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.231+0000 7f823affd700 1 -- 172.21.15.89:0/633927811 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82340108f0 con 0x7f8230004810 2023-10-10T20:42:22.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.231+0000 7f823affd700 1 -- 172.21.15.89:0/633927811 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8234010a50 con 0x7f8230004810 2023-10-10T20:42:22.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.231+0000 7f824359e700 1 -- 172.21.15.89:0/633927811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8244157620 con 0x7f8230004810 2023-10-10T20:42:22.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.231+0000 7f823affd700 1 -- 172.21.15.89:0/633927811 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234010e10 con 0x7f8230004810 2023-10-10T20:42:22.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.231+0000 7f824359e700 1 -- 172.21.15.89:0/633927811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8244158100 con 0x7f8230004810 2023-10-10T20:42:22.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.232+0000 7f824359e700 1 -- 172.21.15.89:0/633927811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f824404c3c0 con 0x7f8230004810 2023-10-10T20:42:22.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.232+0000 7f823affd700 1 -- 172.21.15.89:0/633927811 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8234014640 con 0x7f8230004810 2023-10-10T20:42:22.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.233+0000 7f823affd700 1 --2- 172.21.15.89:0/633927811 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8224067320 0x7f82240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:22.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.233+0000 7f823affd700 1 -- 172.21.15.89:0/633927811 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f82340856b0 con 0x7f8230004810 2023-10-10T20:42:22.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.233+0000 7f8239d9b700 1 --2- 172.21.15.89:0/633927811 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8224067320 0x7f82240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:22.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.233+0000 7f8239d9b700 1 --2- 172.21.15.89:0/633927811 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8224067320 0x7f82240697d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f82440b55d0 tx=0x7f822c015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:22.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.237+0000 7f823affd700 1 -- 172.21.15.89:0/633927811 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82340525a0 con 0x7f8230004810 2023-10-10T20:42:22.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:22.422+0000 7f824359e700 1 -- 172.21.15.89:0/633927811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f82440b3860 con 0x7f8230004810 2023-10-10T20:42:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.092+0000 7f823affd700 1 -- 172.21.15.89:0/633927811 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v151) v1 ==== 170+0+0 (secure 0 0 0) 0x7f82340220d0 con 0x7f8230004810 2023-10-10T20:42:23.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.103+0000 7f824359e700 1 -- 172.21.15.89:0/633927811 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8224067320 msgr2=0x7f82240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:23.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.103+0000 7f824359e700 1 --2- 172.21.15.89:0/633927811 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8224067320 0x7f82240697d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f82440b55d0 tx=0x7f822c015000 comp rx=0 tx=0).stop 2023-10-10T20:42:23.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.103+0000 7f824359e700 1 -- 172.21.15.89:0/633927811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8230004810 msgr2=0x7f8244140ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:23.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.103+0000 7f824359e700 1 --2- 172.21.15.89:0/633927811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8230004810 0x7f8244140ef0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f823400bbd0 tx=0x7f8234008040 comp rx=0 tx=0).stop 2023-10-10T20:42:23.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.106+0000 7f824359e700 1 -- 172.21.15.89:0/633927811 shutdown_connections 2023-10-10T20:42:23.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.106+0000 7f824359e700 1 --2- 172.21.15.89:0/633927811 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8244140990 0x7f82440c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:23.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.106+0000 7f824359e700 1 --2- 172.21.15.89:0/633927811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8230004810 0x7f8244140ef0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:23.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.106+0000 7f824359e700 1 --2- 172.21.15.89:0/633927811 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8224067320 0x7f82240697d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:23.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.106+0000 7f824359e700 1 --2- 172.21.15.89:0/633927811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8244141430 0x7f82440c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:23.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.106+0000 7f824359e700 1 -- 172.21.15.89:0/633927811 >> 172.21.15.89:0/633927811 conn(0x7f82440d05f0 msgr2=0x7f8244152a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:23.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.107+0000 7f824359e700 1 -- 172.21.15.89:0/633927811 shutdown_connections 2023-10-10T20:42:23.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.108+0000 7f824359e700 1 -- 172.21.15.89:0/633927811 wait complete. 2023-10-10T20:42:23.112 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2023-10-10T20:42:23.124 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:42:23.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.240+0000 7fa53f464700 1 Processor -- start 2023-10-10T20:42:23.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.240+0000 7fa53f464700 1 -- start start 2023-10-10T20:42:23.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.241+0000 7fa53f464700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa538156490 0x7fa5380c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:23.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.241+0000 7fa53f464700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa5380ccee0 0x7fa5380c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:23.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.241+0000 7fa53f464700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5380c7a70 con 0x7fa538156860 2023-10-10T20:42:23.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.241+0000 7fa53f464700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa53815a450 con 0x7fa538156490 2023-10-10T20:42:23.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.241+0000 7fa53f464700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53815a730 con 0x7fa5380ccee0 2023-10-10T20:42:23.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.241+0000 7fa53d200700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa538156490 0x7fa5380c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:23.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.241+0000 7fa53d200700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa538156490 0x7fa5380c7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44640/0 (socket says 172.21.15.89:44640) 2023-10-10T20:42:23.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.241+0000 7fa53d200700 1 -- 172.21.15.89:0/347564394 learned_addr learned my addr 172.21.15.89:0/347564394 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:23.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.242+0000 7fa53c9ff700 1 --2- 172.21.15.89:0/347564394 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa5380ccee0 0x7fa5380c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:23.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.242+0000 7fa53d200700 1 -- 172.21.15.89:0/347564394 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa5380ccee0 msgr2=0x7fa5380c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:23.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.242+0000 7fa53d200700 1 --2- 172.21.15.89:0/347564394 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa5380ccee0 0x7fa5380c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:23.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.242+0000 7fa53d200700 1 -- 172.21.15.89:0/347564394 >> v1:172.21.15.89:6789/0 conn(0x7fa538156860 legacy=0x7fa53806eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:23.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.242+0000 7fa53d200700 1 -- 172.21.15.89:0/347564394 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53815aa10 con 0x7fa538156490 2023-10-10T20:42:23.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.242+0000 7fa52ffff700 1 -- 172.21.15.89:0/347564394 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2179943583 0 0) 0x7fa5380c7a70 con 0x7fa538156860 2023-10-10T20:42:23.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.242+0000 7fa53c9ff700 1 --2- 172.21.15.89:0/347564394 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa5380ccee0 0x7fa5380c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:23.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.242+0000 7fa53d200700 1 --2- 172.21.15.89:0/347564394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa538156490 0x7fa5380c7530 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fa5380c7c00 tx=0x7fa534009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f931828c5e2173dc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:23.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.242+0000 7fa52ffff700 1 -- 172.21.15.89:0/347564394 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53400b860 con 0x7fa538156490 2023-10-10T20:42:23.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.242+0000 7fa52ffff700 1 -- 172.21.15.89:0/347564394 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53400b9c0 con 0x7fa538156490 2023-10-10T20:42:23.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.242+0000 7fa52ffff700 1 -- 172.21.15.89:0/347564394 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53400bd80 con 0x7fa538156490 2023-10-10T20:42:23.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.243+0000 7fa53f464700 1 -- 172.21.15.89:0/347564394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa538156490 msgr2=0x7fa5380c7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:23.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.243+0000 7fa53f464700 1 --2- 172.21.15.89:0/347564394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa538156490 0x7fa5380c7530 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fa5380c7c00 tx=0x7fa534009250 comp rx=0 tx=0).stop 2023-10-10T20:42:23.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.243+0000 7fa53f464700 1 -- 172.21.15.89:0/347564394 shutdown_connections 2023-10-10T20:42:23.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.243+0000 7fa53f464700 1 --2- 172.21.15.89:0/347564394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa538156490 0x7fa5380c7530 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:23.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.243+0000 7fa53f464700 1 --2- 172.21.15.89:0/347564394 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa5380ccee0 0x7fa5380c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:23.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.243+0000 7fa53f464700 1 -- 172.21.15.89:0/347564394 >> 172.21.15.89:0/347564394 conn(0x7fa538068e10 msgr2=0x7fa5380cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:23.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.243+0000 7fa53f464700 1 -- 172.21.15.89:0/347564394 shutdown_connections 2023-10-10T20:42:23.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.243+0000 7fa53f464700 1 -- 172.21.15.89:0/347564394 wait complete. 2023-10-10T20:42:23.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.244+0000 7fa53f464700 1 Processor -- start 2023-10-10T20:42:23.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.244+0000 7fa53f464700 1 -- start start 2023-10-10T20:42:23.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa53f464700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5380ccee0 0x7fa538145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:23.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa53f464700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa538156490 0x7fa5380b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:23.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa53f464700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa538156860 0x7fa5380b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:23.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa53f464700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5380c4810 con 0x7fa538156860 2023-10-10T20:42:23.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa53f464700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5381477a0 con 0x7fa5380ccee0 2023-10-10T20:42:23.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa53f464700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5380c3c00 con 0x7fa538156490 2023-10-10T20:42:23.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa53da01700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa538156860 0x7fa5380b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:23.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa53da01700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa538156860 0x7fa5380b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44648/0 (socket says 172.21.15.89:44648) 2023-10-10T20:42:23.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa53da01700 1 -- 172.21.15.89:0/191173912 learned_addr learned my addr 172.21.15.89:0/191173912 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:23.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa53c9ff700 1 --2- 172.21.15.89:0/191173912 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa538156490 0x7fa5380b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:23.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa53d200700 1 --2- 172.21.15.89:0/191173912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5380ccee0 0x7fa538145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:23.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa53da01700 1 -- 172.21.15.89:0/191173912 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa538156490 msgr2=0x7fa5380b15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:23.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa53da01700 1 --2- 172.21.15.89:0/191173912 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa538156490 0x7fa5380b15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:23.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa53da01700 1 -- 172.21.15.89:0/191173912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5380ccee0 msgr2=0x7fa538145410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:23.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa53da01700 1 --2- 172.21.15.89:0/191173912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5380ccee0 0x7fa538145410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:23.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa53da01700 1 -- 172.21.15.89:0/191173912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5340094f0 con 0x7fa538156860 2023-10-10T20:42:23.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa53c9ff700 1 --2- 172.21.15.89:0/191173912 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa538156490 0x7fa5380b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa53da01700 1 --2- 172.21.15.89:0/191173912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa538156860 0x7fa5380b05f0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fa53000c890 tx=0x7fa530001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa52d7fa700 1 -- 172.21.15.89:0/191173912 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa530017070 con 0x7fa538156860 2023-10-10T20:42:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa52d7fa700 1 -- 172.21.15.89:0/191173912 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa530006ba0 con 0x7fa538156860 2023-10-10T20:42:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa52d7fa700 1 -- 172.21.15.89:0/191173912 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53000a6c0 con 0x7fa538156860 2023-10-10T20:42:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.245+0000 7fa53f464700 1 -- 172.21.15.89:0/191173912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5380c3ee0 con 0x7fa538156860 2023-10-10T20:42:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.246+0000 7fa53f464700 1 -- 172.21.15.89:0/191173912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5380c44a0 con 0x7fa538156860 2023-10-10T20:42:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.247+0000 7fa53f464700 1 -- 172.21.15.89:0/191173912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5380c4980 con 0x7fa538156860 2023-10-10T20:42:23.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.247+0000 7fa52d7fa700 1 -- 172.21.15.89:0/191173912 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa53000aaf0 con 0x7fa538156860 2023-10-10T20:42:23.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.248+0000 7fa52d7fa700 1 --2- 172.21.15.89:0/191173912 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa51c067320 0x7fa51c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:23.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.248+0000 7fa52d7fa700 1 -- 172.21.15.89:0/191173912 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fa530084b00 con 0x7fa538156860 2023-10-10T20:42:23.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.248+0000 7fa53d200700 1 --2- 172.21.15.89:0/191173912 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa51c067320 0x7fa51c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:23.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.248+0000 7fa53d200700 1 --2- 172.21.15.89:0/191173912 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa51c067320 0x7fa51c0697d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa5380c7c00 tx=0x7fa534016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:23.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.252+0000 7fa52d7fa700 1 -- 172.21.15.89:0/191173912 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5300517b0 con 0x7fa538156860 2023-10-10T20:42:23.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:23.477+0000 7fa53f464700 1 -- 172.21.15.89:0/191173912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa53804c3c0 con 0x7fa538156860 2023-10-10T20:42:24.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.108+0000 7fa52d7fa700 1 -- 172.21.15.89:0/191173912 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v97) v1 ==== 174+0+0 (secure 0 0 0) 0x7fa530054dd0 con 0x7fa538156860 2023-10-10T20:42:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.122+0000 7fa53f464700 1 -- 172.21.15.89:0/191173912 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa51c067320 msgr2=0x7fa51c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.122+0000 7fa53f464700 1 --2- 172.21.15.89:0/191173912 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa51c067320 0x7fa51c0697d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa5380c7c00 tx=0x7fa534016000 comp rx=0 tx=0).stop 2023-10-10T20:42:24.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.122+0000 7fa53f464700 1 -- 172.21.15.89:0/191173912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa538156860 msgr2=0x7fa5380b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:24.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.122+0000 7fa53f464700 1 --2- 172.21.15.89:0/191173912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa538156860 0x7fa5380b05f0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fa53000c890 tx=0x7fa530001670 comp rx=0 tx=0).stop 2023-10-10T20:42:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.123+0000 7fa53f464700 1 -- 172.21.15.89:0/191173912 shutdown_connections 2023-10-10T20:42:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.123+0000 7fa53f464700 1 --2- 172.21.15.89:0/191173912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa538156860 0x7fa5380b05f0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.123+0000 7fa53f464700 1 --2- 172.21.15.89:0/191173912 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa51c067320 0x7fa51c0697d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.123+0000 7fa53f464700 1 --2- 172.21.15.89:0/191173912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5380ccee0 0x7fa538145410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.123+0000 7fa53f464700 1 --2- 172.21.15.89:0/191173912 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa538156490 0x7fa5380b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.123+0000 7fa53f464700 1 -- 172.21.15.89:0/191173912 >> 172.21.15.89:0/191173912 conn(0x7fa538068e10 msgr2=0x7fa5380cb600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:24.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.123+0000 7fa53f464700 1 -- 172.21.15.89:0/191173912 shutdown_connections 2023-10-10T20:42:24.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.123+0000 7fa53f464700 1 -- 172.21.15.89:0/191173912 wait complete. 2023-10-10T20:42:24.128 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 26 and data pool 27 2023-10-10T20:42:24.138 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:42:24.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.251+0000 7f984e8d0700 1 Processor -- start 2023-10-10T20:42:24.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.251+0000 7f984e8d0700 1 -- start start 2023-10-10T20:42:24.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984e8d0700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9848159420 0x7f98480b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984e8d0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98480b2980 con 0x7f98481597f0 2023-10-10T20:42:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984e8d0700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98480b11d0 con 0x7f984815d230 2023-10-10T20:42:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984e8d0700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98480b14b0 con 0x7f9848159420 2023-10-10T20:42:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984d0cd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f98481597f0 0x7f98480b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48956/0 (socket says 172.21.15.89:48956) 2023-10-10T20:42:24.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984d0cd700 1 -- 172.21.15.89:0/1018423501 learned_addr learned my addr 172.21.15.89:0/1018423501 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:24.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984c8cc700 1 -- 172.21.15.89:0/1018423501 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720775636 0 0) 0x7f98480b2980 con 0x7f98481597f0 2023-10-10T20:42:24.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984c8cc700 1 -- 172.21.15.89:0/1018423501 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9830003570 con 0x7f98481597f0 2023-10-10T20:42:24.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984d8ce700 1 --2- 172.21.15.89:0/1018423501 >> v2:172.21.15.111:3300/0 conn(0x7f9848159420 0x7f98480b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984c8cc700 1 -- 172.21.15.89:0/1018423501 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1672237076 0 0) 0x7f9830003570 con 0x7f98481597f0 2023-10-10T20:42:24.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984c8cc700 1 -- 172.21.15.89:0/1018423501 >> v2:172.21.15.111:3300/0 conn(0x7f9848159420 msgr2=0x7f98480b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:24.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984c8cc700 1 --2- 172.21.15.89:0/1018423501 >> v2:172.21.15.111:3300/0 conn(0x7f9848159420 0x7f98480b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984c8cc700 1 -- 172.21.15.89:0/1018423501 >> v1:172.21.15.111:6789/0 conn(0x7f984815d230 legacy=0x7f984815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:24.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984c8cc700 1 -- 172.21.15.89:0/1018423501 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98480b1790 con 0x7f98481597f0 2023-10-10T20:42:24.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984c8cc700 1 -- 172.21.15.89:0/1018423501 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f983c001470 con 0x7f98481597f0 2023-10-10T20:42:24.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984c8cc700 1 -- 172.21.15.89:0/1018423501 >> v1:172.21.15.89:6789/0 conn(0x7f98481597f0 legacy=0x7f98480b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:24.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984c8cc700 1 --2- 172.21.15.89:0/1018423501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9830003a60 0x7f9830003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984c8cc700 1 --2- 172.21.15.89:0/1018423501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9830004810 0x7f9830004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984c8cc700 1 --2- 172.21.15.89:0/1018423501 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f98300051c0 0x7f983000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984c8cc700 1 -- 172.21.15.89:0/1018423501 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98480b14b0 con 0x7f9830004810 2023-10-10T20:42:24.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984c8cc700 1 -- 172.21.15.89:0/1018423501 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983000c0d0 con 0x7f9830003a60 2023-10-10T20:42:24.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.252+0000 7f984c8cc700 1 -- 172.21.15.89:0/1018423501 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983000c290 con 0x7f98300051c0 2023-10-10T20:42:24.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.253+0000 7f984d0cd700 1 --2- 172.21.15.89:0/1018423501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9830004810 0x7f9830004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.253+0000 7f984d8ce700 1 --2- 172.21.15.89:0/1018423501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9830003a60 0x7f9830003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.253+0000 7f984e0cf700 1 --2- 172.21.15.89:0/1018423501 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f98300051c0 0x7f983000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.253+0000 7f984d0cd700 1 -- 172.21.15.89:0/1018423501 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f98300051c0 msgr2=0x7f983000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:24.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.253+0000 7f984d0cd700 1 --2- 172.21.15.89:0/1018423501 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f98300051c0 0x7f983000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.253+0000 7f984d0cd700 1 -- 172.21.15.89:0/1018423501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9830003a60 msgr2=0x7f9830003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:24.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.253+0000 7f984d0cd700 1 --2- 172.21.15.89:0/1018423501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9830003a60 0x7f9830003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.253+0000 7f984d0cd700 1 -- 172.21.15.89:0/1018423501 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9830003570 con 0x7f9830004810 2023-10-10T20:42:24.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.253+0000 7f984e0cf700 1 --2- 172.21.15.89:0/1018423501 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f98300051c0 0x7f983000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:24.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.253+0000 7f984e0cf700 1 -- 172.21.15.89:0/1018423501 reap_dead start 2023-10-10T20:42:24.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.253+0000 7f984d0cd700 1 --2- 172.21.15.89:0/1018423501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9830004810 0x7f9830004c80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f983c00c390 tx=0x7f983c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9359c9958a449fb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:24.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.253+0000 7f984c8cc700 1 -- 172.21.15.89:0/1018423501 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c00cc60 con 0x7f9830004810 2023-10-10T20:42:24.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.253+0000 7f984c8cc700 1 -- 172.21.15.89:0/1018423501 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983c00cdc0 con 0x7f9830004810 2023-10-10T20:42:24.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.253+0000 7f984c8cc700 1 -- 172.21.15.89:0/1018423501 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c005b50 con 0x7f9830004810 2023-10-10T20:42:24.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.254+0000 7f984e8d0700 1 -- 172.21.15.89:0/1018423501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9830004810 msgr2=0x7f9830004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:24.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.254+0000 7f984e8d0700 1 --2- 172.21.15.89:0/1018423501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9830004810 0x7f9830004c80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f983c00c390 tx=0x7f983c001e50 comp rx=0 tx=0).stop 2023-10-10T20:42:24.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.254+0000 7f984e8d0700 1 -- 172.21.15.89:0/1018423501 shutdown_connections 2023-10-10T20:42:24.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.254+0000 7f984e8d0700 1 --2- 172.21.15.89:0/1018423501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9830004810 0x7f9830004c80 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.254+0000 7f984e8d0700 1 -- 172.21.15.89:0/1018423501 >> 172.21.15.89:0/1018423501 conn(0x7f98480d0660 msgr2=0x7f98480d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:24.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.254+0000 7f984e8d0700 1 -- 172.21.15.89:0/1018423501 shutdown_connections 2023-10-10T20:42:24.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.254+0000 7f984e8d0700 1 -- 172.21.15.89:0/1018423501 wait complete. 2023-10-10T20:42:24.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.255+0000 7f984e8d0700 1 Processor -- start 2023-10-10T20:42:24.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.255+0000 7f984e8d0700 1 -- start start 2023-10-10T20:42:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.255+0000 7f984e8d0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9830004810 0x7f9848155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.255+0000 7f984e8d0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98481505b0 0x7f984815c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.255+0000 7f984e8d0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9848151ee0 0x7f9848152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.255+0000 7f984e8d0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98481543a0 con 0x7f9848151ee0 2023-10-10T20:42:24.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.255+0000 7f984e8d0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984815b9f0 con 0x7f98481505b0 2023-10-10T20:42:24.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.255+0000 7f984e8d0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98481545f0 con 0x7f9830004810 2023-10-10T20:42:24.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.256+0000 7f984e0cf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9848151ee0 0x7f9848152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.256+0000 7f984e0cf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9848151ee0 0x7f9848152360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44662/0 (socket says 172.21.15.89:44662) 2023-10-10T20:42:24.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.256+0000 7f984e0cf700 1 -- 172.21.15.89:0/2346959332 learned_addr learned my addr 172.21.15.89:0/2346959332 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:24.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.256+0000 7f984d8ce700 1 --2- 172.21.15.89:0/2346959332 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9830004810 0x7f9848155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.256+0000 7f984d0cd700 1 --2- 172.21.15.89:0/2346959332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98481505b0 0x7f984815c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.256+0000 7f984e0cf700 1 -- 172.21.15.89:0/2346959332 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9830004810 msgr2=0x7f9848155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.256+0000 7f984e0cf700 1 --2- 172.21.15.89:0/2346959332 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9830004810 0x7f9848155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.256+0000 7f984e0cf700 1 -- 172.21.15.89:0/2346959332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98481505b0 msgr2=0x7f984815c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.256+0000 7f984e0cf700 1 --2- 172.21.15.89:0/2346959332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98481505b0 0x7f984815c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.256+0000 7f984e0cf700 1 -- 172.21.15.89:0/2346959332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98481548d0 con 0x7f9848151ee0 2023-10-10T20:42:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.256+0000 7f984e0cf700 1 --2- 172.21.15.89:0/2346959332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9848151ee0 0x7f9848152360 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f98480af930 tx=0x7f9838009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:24.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.256+0000 7f9845ffb700 1 -- 172.21.15.89:0/2346959332 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9838012590 con 0x7f9848151ee0 2023-10-10T20:42:24.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.256+0000 7f9845ffb700 1 -- 172.21.15.89:0/2346959332 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98380126f0 con 0x7f9848151ee0 2023-10-10T20:42:24.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.256+0000 7f984e8d0700 1 -- 172.21.15.89:0/2346959332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9848154bb0 con 0x7f9848151ee0 2023-10-10T20:42:24.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.256+0000 7f9845ffb700 1 -- 172.21.15.89:0/2346959332 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9838012ab0 con 0x7f9848151ee0 2023-10-10T20:42:24.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.256+0000 7f984e8d0700 1 -- 172.21.15.89:0/2346959332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9848155170 con 0x7f9848151ee0 2023-10-10T20:42:24.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.257+0000 7f984e8d0700 1 -- 172.21.15.89:0/2346959332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9848155360 con 0x7f9848151ee0 2023-10-10T20:42:24.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.257+0000 7f9845ffb700 1 -- 172.21.15.89:0/2346959332 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9838003030 con 0x7f9848151ee0 2023-10-10T20:42:24.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.258+0000 7f9845ffb700 1 --2- 172.21.15.89:0/2346959332 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9834067320 0x7f98340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.258+0000 7f9845ffb700 1 -- 172.21.15.89:0/2346959332 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9838084ae0 con 0x7f9848151ee0 2023-10-10T20:42:24.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.258+0000 7f984d8ce700 1 --2- 172.21.15.89:0/2346959332 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9834067320 0x7f98340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.258+0000 7f984d8ce700 1 --2- 172.21.15.89:0/2346959332 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9834067320 0x7f98340697d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9840006f00 tx=0x7f98400091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:24.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.262+0000 7f9845ffb700 1 -- 172.21.15.89:0/2346959332 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9838051770 con 0x7f9848151ee0 2023-10-10T20:42:24.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.448+0000 7f984e8d0700 1 -- 172.21.15.89:0/2346959332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f984804c3c0 con 0x7f9848151ee0 2023-10-10T20:42:24.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.449+0000 7f9845ffb700 1 -- 172.21.15.89:0/2346959332 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+15377 (secure 0 0 0) 0x7f9838054d90 con 0x7f9848151ee0 2023-10-10T20:42:24.452 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:24.452 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":152,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:42:23.478922+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:42:21.831397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-10T20:42:22.440492+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:42:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.451+0000 7f984e8d0700 1 -- 172.21.15.89:0/2346959332 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9834067320 msgr2=0x7f98340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.451+0000 7f984e8d0700 1 --2- 172.21.15.89:0/2346959332 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9834067320 0x7f98340697d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9840006f00 tx=0x7f98400091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.451+0000 7f984e8d0700 1 -- 172.21.15.89:0/2346959332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9848151ee0 msgr2=0x7f9848152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.451+0000 7f984e8d0700 1 --2- 172.21.15.89:0/2346959332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9848151ee0 0x7f9848152360 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f98480af930 tx=0x7f9838009040 comp rx=0 tx=0).stop 2023-10-10T20:42:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.452+0000 7f984e8d0700 1 -- 172.21.15.89:0/2346959332 shutdown_connections 2023-10-10T20:42:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.452+0000 7f984e8d0700 1 --2- 172.21.15.89:0/2346959332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9848151ee0 0x7f9848152360 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.452+0000 7f984e8d0700 1 --2- 172.21.15.89:0/2346959332 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9830004810 0x7f9848155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.452+0000 7f984e8d0700 1 --2- 172.21.15.89:0/2346959332 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9834067320 0x7f98340697d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.452+0000 7f984e8d0700 1 --2- 172.21.15.89:0/2346959332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98481505b0 0x7f984815c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.452+0000 7f984e8d0700 1 -- 172.21.15.89:0/2346959332 >> 172.21.15.89:0/2346959332 conn(0x7f98480d0660 msgr2=0x7f98480b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.452+0000 7f984e8d0700 1 -- 172.21.15.89:0/2346959332 shutdown_connections 2023-10-10T20:42:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.452+0000 7f984e8d0700 1 -- 172.21.15.89:0/2346959332 wait complete. 2023-10-10T20:42:24.466 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:42:24.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.578+0000 7f31dc1ec700 1 Processor -- start 2023-10-10T20:42:24.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.578+0000 7f31dc1ec700 1 -- start start 2023-10-10T20:42:24.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.579+0000 7f31dc1ec700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f31d4159890 0x7f31d415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.579+0000 7f31dc1ec700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f31d4159c60 0x7f31d415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.579+0000 7f31dc1ec700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31d415db10 0x7f31d4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.579+0000 7f31dc1ec700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d415df50 con 0x7f31d4159890 2023-10-10T20:42:24.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.579+0000 7f31dc1ec700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d415a160 con 0x7f31d4159c60 2023-10-10T20:42:24.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.579+0000 7f31dc1ec700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d40d13a0 con 0x7f31d415db10 2023-10-10T20:42:24.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.579+0000 7f31d9f88700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f31d4159890 0x7f31d415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.579+0000 7f31d9f88700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f31d4159890 0x7f31d415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44678/0 (socket says 172.21.15.89:44678) 2023-10-10T20:42:24.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.579+0000 7f31d9f88700 1 -- 172.21.15.89:0/3145940105 learned_addr learned my addr 172.21.15.89:0/3145940105 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:24.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.579+0000 7f31da789700 1 --2- 172.21.15.89:0/3145940105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31d415db10 0x7f31d4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.580+0000 7f31d9787700 1 --2- 172.21.15.89:0/3145940105 >> v2:172.21.15.111:3300/0 conn(0x7f31d4159c60 0x7f31d415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.580+0000 7f31d9f88700 1 -- 172.21.15.89:0/3145940105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31d415db10 msgr2=0x7f31d4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:24.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.580+0000 7f31d9f88700 1 --2- 172.21.15.89:0/3145940105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31d415db10 0x7f31d4069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.580+0000 7f31d9f88700 1 -- 172.21.15.89:0/3145940105 >> v2:172.21.15.111:3300/0 conn(0x7f31d4159c60 msgr2=0x7f31d415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:24.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.580+0000 7f31d9f88700 1 --2- 172.21.15.89:0/3145940105 >> v2:172.21.15.111:3300/0 conn(0x7f31d4159c60 0x7f31d415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.580+0000 7f31d9f88700 1 -- 172.21.15.89:0/3145940105 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d415e140 con 0x7f31d4159890 2023-10-10T20:42:24.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.580+0000 7f31da789700 1 --2- 172.21.15.89:0/3145940105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31d415db10 0x7f31d4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:24.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.580+0000 7f31d9787700 1 --2- 172.21.15.89:0/3145940105 >> v2:172.21.15.111:3300/0 conn(0x7f31d4159c60 0x7f31d415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:24.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.580+0000 7f31d9f88700 1 --2- 172.21.15.89:0/3145940105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31d4159890 0x7f31d415abd0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f31d40b67f0 tx=0x7f31d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ef1e8fa9c4042b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:24.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.580+0000 7f31d8f86700 1 -- 172.21.15.89:0/3145940105 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d000e070 con 0x7f31d4159890 2023-10-10T20:42:24.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.580+0000 7f31d8f86700 1 -- 172.21.15.89:0/3145940105 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d000ba50 con 0x7f31d4159890 2023-10-10T20:42:24.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.580+0000 7f31d8f86700 1 -- 172.21.15.89:0/3145940105 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d00092e0 con 0x7f31d4159890 2023-10-10T20:42:24.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.580+0000 7f31dc1ec700 1 -- 172.21.15.89:0/3145940105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31d4159890 msgr2=0x7f31d415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:24.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.580+0000 7f31dc1ec700 1 --2- 172.21.15.89:0/3145940105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31d4159890 0x7f31d415abd0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f31d40b67f0 tx=0x7f31d0009840 comp rx=0 tx=0).stop 2023-10-10T20:42:24.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.581+0000 7f31dc1ec700 1 -- 172.21.15.89:0/3145940105 shutdown_connections 2023-10-10T20:42:24.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.581+0000 7f31dc1ec700 1 --2- 172.21.15.89:0/3145940105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31d415db10 0x7f31d4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.581+0000 7f31dc1ec700 1 --2- 172.21.15.89:0/3145940105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31d4159890 0x7f31d415abd0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.581+0000 7f31dc1ec700 1 --2- 172.21.15.89:0/3145940105 >> v2:172.21.15.111:3300/0 conn(0x7f31d4159c60 0x7f31d415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.581+0000 7f31dc1ec700 1 -- 172.21.15.89:0/3145940105 >> 172.21.15.89:0/3145940105 conn(0x7f31d4068e20 msgr2=0x7f31d40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:24.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.581+0000 7f31dc1ec700 1 -- 172.21.15.89:0/3145940105 shutdown_connections 2023-10-10T20:42:24.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.581+0000 7f31dc1ec700 1 -- 172.21.15.89:0/3145940105 wait complete. 2023-10-10T20:42:24.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.582+0000 7f31dc1ec700 1 Processor -- start 2023-10-10T20:42:24.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.582+0000 7f31dc1ec700 1 -- start start 2023-10-10T20:42:24.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.582+0000 7f31dc1ec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31d4159890 0x7f31d40cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.582+0000 7f31dc1ec700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31d4159c60 0x7f31d4147420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.582+0000 7f31dc1ec700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31d415db10 0x7f31d40b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.582+0000 7f31dc1ec700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d40cff00 con 0x7f31d4159890 2023-10-10T20:42:24.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.582+0000 7f31dc1ec700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d40cf530 con 0x7f31d415db10 2023-10-10T20:42:24.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.582+0000 7f31dc1ec700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d4147960 con 0x7f31d4159c60 2023-10-10T20:42:24.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.583+0000 7f31d9f88700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31d4159890 0x7f31d40cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.583+0000 7f31d9f88700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31d4159890 0x7f31d40cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44688/0 (socket says 172.21.15.89:44688) 2023-10-10T20:42:24.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.583+0000 7f31d9f88700 1 -- 172.21.15.89:0/2434368174 learned_addr learned my addr 172.21.15.89:0/2434368174 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:24.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.583+0000 7f31d9787700 1 --2- 172.21.15.89:0/2434368174 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31d4159c60 0x7f31d4147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.583+0000 7f31da789700 1 --2- 172.21.15.89:0/2434368174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31d415db10 0x7f31d40b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.583+0000 7f31d9f88700 1 -- 172.21.15.89:0/2434368174 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31d4159c60 msgr2=0x7f31d4147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:24.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.583+0000 7f31d9f88700 1 --2- 172.21.15.89:0/2434368174 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31d4159c60 0x7f31d4147420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.583+0000 7f31d9f88700 1 -- 172.21.15.89:0/2434368174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31d415db10 msgr2=0x7f31d40b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:24.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.583+0000 7f31d9f88700 1 --2- 172.21.15.89:0/2434368174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31d415db10 0x7f31d40b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.583+0000 7f31d9f88700 1 -- 172.21.15.89:0/2434368174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d0009ae0 con 0x7f31d4159890 2023-10-10T20:42:24.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.583+0000 7f31d9f88700 1 --2- 172.21.15.89:0/2434368174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31d4159890 0x7f31d40cf9c0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f31d000bef0 tx=0x7f31d0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:24.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.583+0000 7f31c9ffb700 1 -- 172.21.15.89:0/2434368174 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d0008a80 con 0x7f31d4159890 2023-10-10T20:42:24.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.583+0000 7f31c9ffb700 1 -- 172.21.15.89:0/2434368174 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d0008c00 con 0x7f31d4159890 2023-10-10T20:42:24.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.583+0000 7f31dc1ec700 1 -- 172.21.15.89:0/2434368174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d40ce960 con 0x7f31d4159890 2023-10-10T20:42:24.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.583+0000 7f31c9ffb700 1 -- 172.21.15.89:0/2434368174 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d0026460 con 0x7f31d4159890 2023-10-10T20:42:24.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.583+0000 7f31dc1ec700 1 -- 172.21.15.89:0/2434368174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d40ceee0 con 0x7f31d4159890 2023-10-10T20:42:24.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.584+0000 7f31dc1ec700 1 -- 172.21.15.89:0/2434368174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d40cf810 con 0x7f31d4159890 2023-10-10T20:42:24.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.585+0000 7f31c9ffb700 1 -- 172.21.15.89:0/2434368174 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f31d0003040 con 0x7f31d4159890 2023-10-10T20:42:24.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.585+0000 7f31c9ffb700 1 --2- 172.21.15.89:0/2434368174 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31b8067320 0x7f31b80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.585+0000 7f31c9ffb700 1 -- 172.21.15.89:0/2434368174 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f31d0086b90 con 0x7f31d4159890 2023-10-10T20:42:24.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.585+0000 7f31d9787700 1 --2- 172.21.15.89:0/2434368174 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31b8067320 0x7f31b80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.585+0000 7f31d9787700 1 --2- 172.21.15.89:0/2434368174 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31b8067320 0x7f31b80697d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f31c4006f00 tx=0x7f31c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:24.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.589+0000 7f31c9ffb700 1 -- 172.21.15.89:0/2434368174 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31d0053710 con 0x7f31d4159890 2023-10-10T20:42:24.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.775+0000 7f31dc1ec700 1 -- 172.21.15.89:0/2434368174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31d404c3c0 con 0x7f31d4159890 2023-10-10T20:42:24.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.776+0000 7f31c9ffb700 1 -- 172.21.15.89:0/2434368174 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+15377 (secure 0 0 0) 0x7f31d0056d30 con 0x7f31d4159890 2023-10-10T20:42:24.778 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:24.779 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":152,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:42:23.478922+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:42:21.831397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-10T20:42:22.440492+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:42:24.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.778+0000 7f31dc1ec700 1 -- 172.21.15.89:0/2434368174 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31b8067320 msgr2=0x7f31b80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:24.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.778+0000 7f31dc1ec700 1 --2- 172.21.15.89:0/2434368174 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31b8067320 0x7f31b80697d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f31c4006f00 tx=0x7f31c4008040 comp rx=0 tx=0).stop 2023-10-10T20:42:24.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.778+0000 7f31dc1ec700 1 -- 172.21.15.89:0/2434368174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31d4159890 msgr2=0x7f31d40cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:24.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.778+0000 7f31dc1ec700 1 --2- 172.21.15.89:0/2434368174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31d4159890 0x7f31d40cf9c0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f31d000bef0 tx=0x7f31d0018000 comp rx=0 tx=0).stop 2023-10-10T20:42:24.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.778+0000 7f31dc1ec700 1 -- 172.21.15.89:0/2434368174 shutdown_connections 2023-10-10T20:42:24.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.778+0000 7f31dc1ec700 1 --2- 172.21.15.89:0/2434368174 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31b8067320 0x7f31b80697d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.778+0000 7f31dc1ec700 1 --2- 172.21.15.89:0/2434368174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31d415db10 0x7f31d40b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.778+0000 7f31dc1ec700 1 --2- 172.21.15.89:0/2434368174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31d4159890 0x7f31d40cf9c0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.778+0000 7f31dc1ec700 1 --2- 172.21.15.89:0/2434368174 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f31d4159c60 0x7f31d4147420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.778+0000 7f31dc1ec700 1 -- 172.21.15.89:0/2434368174 >> 172.21.15.89:0/2434368174 conn(0x7f31d4068e20 msgr2=0x7f31d4152fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:24.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.778+0000 7f31dc1ec700 1 -- 172.21.15.89:0/2434368174 shutdown_connections 2023-10-10T20:42:24.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.778+0000 7f31dc1ec700 1 -- 172.21.15.89:0/2434368174 wait complete. 2023-10-10T20:42:24.794 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:42:24.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.910+0000 7f068649d700 1 Processor -- start 2023-10-10T20:42:24.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.910+0000 7f068649d700 1 -- start start 2023-10-10T20:42:24.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.911+0000 7f068649d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06800b1b40 0x7f06800b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.911+0000 7f068649d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06800cfb00 con 0x7f06800b16d0 2023-10-10T20:42:24.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.911+0000 7f068649d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06800b5c40 con 0x7f06800b4e70 2023-10-10T20:42:24.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.911+0000 7f068649d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06800b5f20 con 0x7f06800b1b40 2023-10-10T20:42:24.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.911+0000 7f0684c9a700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f06800b16d0 0x7f06800b33b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48972/0 (socket says 172.21.15.89:48972) 2023-10-10T20:42:24.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.911+0000 7f0684c9a700 1 -- 172.21.15.89:0/4166042234 learned_addr learned my addr 172.21.15.89:0/4166042234 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:24.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.911+0000 7f0685c9c700 1 --2- 172.21.15.89:0/4166042234 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06800b1b40 0x7f06800b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.911+0000 7f0677fff700 1 -- 172.21.15.89:0/4166042234 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889848122 0 0) 0x7f06800cfb00 con 0x7f06800b16d0 2023-10-10T20:42:24.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.911+0000 7f0677fff700 1 -- 172.21.15.89:0/4166042234 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f066c003570 con 0x7f06800b16d0 2023-10-10T20:42:24.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.911+0000 7f0685c9c700 1 -- 172.21.15.89:0/4166042234 >> v1:172.21.15.111:6789/0 conn(0x7f06800b4e70 legacy=0x7f06800ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:24.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.911+0000 7f0677fff700 1 -- 172.21.15.89:0/4166042234 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4028883180 0 0) 0x7f06800b5c40 con 0x7f06800b4e70 2023-10-10T20:42:24.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.911+0000 7f0685c9c700 1 -- 172.21.15.89:0/4166042234 >> v1:172.21.15.89:6789/0 conn(0x7f06800b16d0 legacy=0x7f06800b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:24.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.911+0000 7f0685c9c700 1 -- 172.21.15.89:0/4166042234 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06800b6200 con 0x7f06800b1b40 2023-10-10T20:42:24.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.912+0000 7f0685c9c700 1 --2- 172.21.15.89:0/4166042234 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06800b1b40 0x7f06800b3ac0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0678009e90 tx=0x7f0678009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=993d8814c86e7dcd server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:24.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.912+0000 7f0677fff700 1 -- 172.21.15.89:0/4166042234 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067800e070 con 0x7f06800b1b40 2023-10-10T20:42:24.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.912+0000 7f0677fff700 1 -- 172.21.15.89:0/4166042234 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067800bc30 con 0x7f06800b1b40 2023-10-10T20:42:24.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.912+0000 7f0677fff700 1 -- 172.21.15.89:0/4166042234 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0678012430 con 0x7f06800b1b40 2023-10-10T20:42:24.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.912+0000 7f068649d700 1 -- 172.21.15.89:0/4166042234 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06800b1b40 msgr2=0x7f06800b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:24.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.912+0000 7f068649d700 1 --2- 172.21.15.89:0/4166042234 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06800b1b40 0x7f06800b3ac0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0678009e90 tx=0x7f0678009780 comp rx=0 tx=0).stop 2023-10-10T20:42:24.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.912+0000 7f068649d700 1 -- 172.21.15.89:0/4166042234 shutdown_connections 2023-10-10T20:42:24.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.912+0000 7f068649d700 1 --2- 172.21.15.89:0/4166042234 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06800b1b40 0x7f06800b3ac0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.912+0000 7f068649d700 1 -- 172.21.15.89:0/4166042234 >> 172.21.15.89:0/4166042234 conn(0x7f06800d10c0 msgr2=0x7f06800affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:24.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.912+0000 7f068649d700 1 -- 172.21.15.89:0/4166042234 shutdown_connections 2023-10-10T20:42:24.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.912+0000 7f068649d700 1 -- 172.21.15.89:0/4166042234 wait complete. 2023-10-10T20:42:24.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.913+0000 7f068649d700 1 Processor -- start 2023-10-10T20:42:24.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.913+0000 7f068649d700 1 -- start start 2023-10-10T20:42:24.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.914+0000 7f068649d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06800b16d0 0x7f06801699a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.914+0000 7f068649d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06800b1b40 0x7f06800b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.914+0000 7f068649d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06800b4e70 0x7f06800c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.914+0000 7f068649d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06800c7b00 con 0x7f06800b4e70 2023-10-10T20:42:24.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.914+0000 7f068649d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0680169ee0 con 0x7f06800b1b40 2023-10-10T20:42:24.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.914+0000 7f068649d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06800c8030 con 0x7f06800b16d0 2023-10-10T20:42:24.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.914+0000 7f0685c9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06800b4e70 0x7f06800c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.914+0000 7f0685c9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06800b4e70 0x7f06800c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44696/0 (socket says 172.21.15.89:44696) 2023-10-10T20:42:24.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.914+0000 7f0685c9c700 1 -- 172.21.15.89:0/1024918185 learned_addr learned my addr 172.21.15.89:0/1024918185 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:24.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.915+0000 7f068549b700 1 --2- 172.21.15.89:0/1024918185 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06800b16d0 0x7f06801699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.915+0000 7f0684c9a700 1 --2- 172.21.15.89:0/1024918185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06800b1b40 0x7f06800b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.915+0000 7f0685c9c700 1 -- 172.21.15.89:0/1024918185 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06800b16d0 msgr2=0x7f06801699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:24.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.915+0000 7f0685c9c700 1 --2- 172.21.15.89:0/1024918185 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06800b16d0 0x7f06801699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.915+0000 7f0685c9c700 1 -- 172.21.15.89:0/1024918185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06800b1b40 msgr2=0x7f06800b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:24.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.915+0000 7f0685c9c700 1 --2- 172.21.15.89:0/1024918185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06800b1b40 0x7f06800b5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:24.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.915+0000 7f0685c9c700 1 -- 172.21.15.89:0/1024918185 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0678009a20 con 0x7f06800b4e70 2023-10-10T20:42:24.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.915+0000 7f068549b700 1 --2- 172.21.15.89:0/1024918185 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06800b16d0 0x7f06801699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:24.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.915+0000 7f0685c9c700 1 --2- 172.21.15.89:0/1024918185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06800b4e70 0x7f06800c6360 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f0680070dd0 tx=0x7f0678019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:24.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.915+0000 7f06757fa700 1 -- 172.21.15.89:0/1024918185 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0678012e40 con 0x7f06800b4e70 2023-10-10T20:42:24.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.915+0000 7f06757fa700 1 -- 172.21.15.89:0/1024918185 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0678007000 con 0x7f06800b4e70 2023-10-10T20:42:24.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.915+0000 7f068649d700 1 -- 172.21.15.89:0/1024918185 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06800c6f40 con 0x7f06800b4e70 2023-10-10T20:42:24.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.915+0000 7f06757fa700 1 -- 172.21.15.89:0/1024918185 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067801b830 con 0x7f06800b4e70 2023-10-10T20:42:24.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.915+0000 7f068649d700 1 -- 172.21.15.89:0/1024918185 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06800c82f0 con 0x7f06800b4e70 2023-10-10T20:42:24.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.916+0000 7f068649d700 1 -- 172.21.15.89:0/1024918185 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0680088990 con 0x7f06800b4e70 2023-10-10T20:42:24.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.916+0000 7f06757fa700 1 -- 172.21.15.89:0/1024918185 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f067801bc60 con 0x7f06800b4e70 2023-10-10T20:42:24.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.917+0000 7f06757fa700 1 --2- 172.21.15.89:0/1024918185 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0664067350 0x7f0664069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:24.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.917+0000 7f06757fa700 1 -- 172.21.15.89:0/1024918185 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0678086a20 con 0x7f06800b4e70 2023-10-10T20:42:24.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.917+0000 7f068549b700 1 --2- 172.21.15.89:0/1024918185 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0664067350 0x7f0664069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:24.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.917+0000 7f068549b700 1 --2- 172.21.15.89:0/1024918185 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0664067350 0x7f0664069800 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f067c007fd0 tx=0x7f067c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:24.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:24.921+0000 7f06757fa700 1 -- 172.21.15.89:0/1024918185 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06780535a0 con 0x7f06800b4e70 2023-10-10T20:42:25.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:25.152+0000 7f068649d700 1 -- 172.21.15.89:0/1024918185 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f068004bb80 con 0x7f06800b4e70 2023-10-10T20:42:26.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.115+0000 7f06757fa700 1 -- 172.21.15.89:0/1024918185 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v100) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0678056bc0 con 0x7f06800b4e70 2023-10-10T20:42:26.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.118+0000 7f068649d700 1 -- 172.21.15.89:0/1024918185 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0664067350 msgr2=0x7f0664069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:26.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.118+0000 7f068649d700 1 --2- 172.21.15.89:0/1024918185 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0664067350 0x7f0664069800 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f067c007fd0 tx=0x7f067c009040 comp rx=0 tx=0).stop 2023-10-10T20:42:26.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.118+0000 7f068649d700 1 -- 172.21.15.89:0/1024918185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06800b4e70 msgr2=0x7f06800c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:26.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.118+0000 7f068649d700 1 --2- 172.21.15.89:0/1024918185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06800b4e70 0x7f06800c6360 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f0680070dd0 tx=0x7f0678019000 comp rx=0 tx=0).stop 2023-10-10T20:42:26.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.118+0000 7f068649d700 1 -- 172.21.15.89:0/1024918185 shutdown_connections 2023-10-10T20:42:26.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.118+0000 7f068649d700 1 --2- 172.21.15.89:0/1024918185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06800b4e70 0x7f06800c6360 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:26.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.118+0000 7f068649d700 1 --2- 172.21.15.89:0/1024918185 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06800b16d0 0x7f06801699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:26.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.118+0000 7f068649d700 1 --2- 172.21.15.89:0/1024918185 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0664067350 0x7f0664069800 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:26.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.118+0000 7f068649d700 1 --2- 172.21.15.89:0/1024918185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06800b1b40 0x7f06800b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:26.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.118+0000 7f068649d700 1 -- 172.21.15.89:0/1024918185 >> 172.21.15.89:0/1024918185 conn(0x7f06800d10c0 msgr2=0x7f068015c730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:26.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.118+0000 7f068649d700 1 -- 172.21.15.89:0/1024918185 shutdown_connections 2023-10-10T20:42:26.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.118+0000 7f068649d700 1 -- 172.21.15.89:0/1024918185 wait complete. 2023-10-10T20:42:26.133 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:42:26.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.250+0000 7fd020a78700 1 Processor -- start 2023-10-10T20:42:26.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.250+0000 7fd020a78700 1 -- start start 2023-10-10T20:42:26.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.251+0000 7fd020a78700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd01c0b1b40 0x7fd01c0b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:26.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.251+0000 7fd020a78700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd01c0cfb00 con 0x7fd01c0b4e70 2023-10-10T20:42:26.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.251+0000 7fd020a78700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd01c0b5c40 con 0x7fd01c0b16d0 2023-10-10T20:42:26.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.251+0000 7fd020a78700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd01c0b5f20 con 0x7fd01c0b1b40 2023-10-10T20:42:26.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.251+0000 7fd01b7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd01c0b4e70 0x7fd01c0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48986/0 (socket says 172.21.15.89:48986) 2023-10-10T20:42:26.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.251+0000 7fd01b7fe700 1 -- 172.21.15.89:0/2924982222 learned_addr learned my addr 172.21.15.89:0/2924982222 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:26.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01a7fc700 1 -- 172.21.15.89:0/2924982222 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3219023966 0 0) 0x7fd01c0cfb00 con 0x7fd01c0b4e70 2023-10-10T20:42:26.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01a7fc700 1 -- 172.21.15.89:0/2924982222 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd004003570 con 0x7fd01c0b4e70 2023-10-10T20:42:26.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01bfff700 1 --2- 172.21.15.89:0/2924982222 >> v2:172.21.15.111:3300/0 conn(0x7fd01c0b1b40 0x7fd01c0b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:26.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01a7fc700 1 -- 172.21.15.89:0/2924982222 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1907670868 0 0) 0x7fd004003570 con 0x7fd01c0b4e70 2023-10-10T20:42:26.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01a7fc700 1 -- 172.21.15.89:0/2924982222 >> v2:172.21.15.111:3300/0 conn(0x7fd01c0b1b40 msgr2=0x7fd01c0b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:26.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01a7fc700 1 --2- 172.21.15.89:0/2924982222 >> v2:172.21.15.111:3300/0 conn(0x7fd01c0b1b40 0x7fd01c0b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:26.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01a7fc700 1 -- 172.21.15.89:0/2924982222 >> v1:172.21.15.111:6789/0 conn(0x7fd01c0b16d0 legacy=0x7fd01c0b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:26.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01a7fc700 1 -- 172.21.15.89:0/2924982222 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01c0b6200 con 0x7fd01c0b4e70 2023-10-10T20:42:26.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01a7fc700 1 -- 172.21.15.89:0/2924982222 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fd00c0017f0 con 0x7fd01c0b4e70 2023-10-10T20:42:26.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01a7fc700 1 -- 172.21.15.89:0/2924982222 >> v1:172.21.15.89:6789/0 conn(0x7fd01c0b4e70 legacy=0x7fd01c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:26.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01a7fc700 1 --2- 172.21.15.89:0/2924982222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd004003a60 0x7fd004003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:26.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01a7fc700 1 --2- 172.21.15.89:0/2924982222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd004004810 0x7fd004004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:26.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01a7fc700 1 --2- 172.21.15.89:0/2924982222 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd0040053c0 0x7fd00400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:26.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01a7fc700 1 -- 172.21.15.89:0/2924982222 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c0b5f20 con 0x7fd004004810 2023-10-10T20:42:26.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01a7fc700 1 -- 172.21.15.89:0/2924982222 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00400c2d0 con 0x7fd004003a60 2023-10-10T20:42:26.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01a7fc700 1 -- 172.21.15.89:0/2924982222 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00400c490 con 0x7fd0040053c0 2023-10-10T20:42:26.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01affd700 1 --2- 172.21.15.89:0/2924982222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd004004810 0x7fd004004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:26.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01b7fe700 1 --2- 172.21.15.89:0/2924982222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd004003a60 0x7fd004003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:26.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01bfff700 1 --2- 172.21.15.89:0/2924982222 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd0040053c0 0x7fd00400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:26.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01affd700 1 -- 172.21.15.89:0/2924982222 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd0040053c0 msgr2=0x7fd00400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:26.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01affd700 1 --2- 172.21.15.89:0/2924982222 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd0040053c0 0x7fd00400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:26.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01affd700 1 -- 172.21.15.89:0/2924982222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd004003a60 msgr2=0x7fd004003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:26.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01affd700 1 --2- 172.21.15.89:0/2924982222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd004003a60 0x7fd004003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:26.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01bfff700 1 -- 172.21.15.89:0/2924982222 reap_dead start 2023-10-10T20:42:26.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.252+0000 7fd01affd700 1 -- 172.21.15.89:0/2924982222 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd004003570 con 0x7fd004004810 2023-10-10T20:42:26.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.253+0000 7fd01affd700 1 --2- 172.21.15.89:0/2924982222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd004004810 0x7fd004004c80 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd0100018b0 tx=0x7fd010007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aee3c8611d76431 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:26.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.253+0000 7fd01a7fc700 1 -- 172.21.15.89:0/2924982222 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd010012420 con 0x7fd004004810 2023-10-10T20:42:26.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.253+0000 7fd01a7fc700 1 -- 172.21.15.89:0/2924982222 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0100125a0 con 0x7fd004004810 2023-10-10T20:42:26.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.253+0000 7fd01a7fc700 1 -- 172.21.15.89:0/2924982222 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd010012960 con 0x7fd004004810 2023-10-10T20:42:26.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.253+0000 7fd020a78700 1 -- 172.21.15.89:0/2924982222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd004004810 msgr2=0x7fd004004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:26.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.253+0000 7fd020a78700 1 --2- 172.21.15.89:0/2924982222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd004004810 0x7fd004004c80 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd0100018b0 tx=0x7fd010007d30 comp rx=0 tx=0).stop 2023-10-10T20:42:26.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.253+0000 7fd020a78700 1 -- 172.21.15.89:0/2924982222 shutdown_connections 2023-10-10T20:42:26.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.253+0000 7fd020a78700 1 --2- 172.21.15.89:0/2924982222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd004004810 0x7fd004004c80 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:26.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.253+0000 7fd020a78700 1 -- 172.21.15.89:0/2924982222 >> 172.21.15.89:0/2924982222 conn(0x7fd01c0d10c0 msgr2=0x7fd01c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:26.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.253+0000 7fd020a78700 1 -- 172.21.15.89:0/2924982222 shutdown_connections 2023-10-10T20:42:26.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.253+0000 7fd020a78700 1 -- 172.21.15.89:0/2924982222 wait complete. 2023-10-10T20:42:26.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.254+0000 7fd020a78700 1 Processor -- start 2023-10-10T20:42:26.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.254+0000 7fd020a78700 1 -- start start 2023-10-10T20:42:26.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.255+0000 7fd020a78700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd004004810 0x7fd01c0c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:26.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.255+0000 7fd020a78700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd01c16aaa0 0x7fd01c0a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:26.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.255+0000 7fd020a78700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd01c0a7440 0x7fd01c0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:26.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.255+0000 7fd020a78700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c16b200 con 0x7fd01c16aaa0 2023-10-10T20:42:26.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.255+0000 7fd020a78700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c0c8760 con 0x7fd004004810 2023-10-10T20:42:26.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.255+0000 7fd020a78700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c0a7870 con 0x7fd01c0a7440 2023-10-10T20:42:26.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.255+0000 7fd01affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd01c16aaa0 0x7fd01c0a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:26.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.255+0000 7fd01affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd01c16aaa0 0x7fd01c0a6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44712/0 (socket says 172.21.15.89:44712) 2023-10-10T20:42:26.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.255+0000 7fd01affd700 1 -- 172.21.15.89:0/2580825113 learned_addr learned my addr 172.21.15.89:0/2580825113 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:26.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.256+0000 7fd01b7fe700 1 --2- 172.21.15.89:0/2580825113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd004004810 0x7fd01c0c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:26.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.256+0000 7fd01bfff700 1 --2- 172.21.15.89:0/2580825113 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd01c0a7440 0x7fd01c0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:26.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.256+0000 7fd01affd700 1 -- 172.21.15.89:0/2580825113 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd01c0a7440 msgr2=0x7fd01c0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:26.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.256+0000 7fd01affd700 1 --2- 172.21.15.89:0/2580825113 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd01c0a7440 0x7fd01c0cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:26.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.256+0000 7fd01affd700 1 -- 172.21.15.89:0/2580825113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd004004810 msgr2=0x7fd01c0c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:26.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.256+0000 7fd01affd700 1 --2- 172.21.15.89:0/2580825113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd004004810 0x7fd01c0c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:26.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.256+0000 7fd01affd700 1 -- 172.21.15.89:0/2580825113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01c0c7ec0 con 0x7fd01c16aaa0 2023-10-10T20:42:26.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.256+0000 7fd01bfff700 1 --2- 172.21.15.89:0/2580825113 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd01c0a7440 0x7fd01c0cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:26.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.256+0000 7fd01b7fe700 1 --2- 172.21.15.89:0/2580825113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd004004810 0x7fd01c0c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:26.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.256+0000 7fd01affd700 1 --2- 172.21.15.89:0/2580825113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd01c16aaa0 0x7fd01c0a6f00 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd010001010 tx=0x7fd010015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:26.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.256+0000 7fd0037fe700 1 -- 172.21.15.89:0/2580825113 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01000c800 con 0x7fd01c16aaa0 2023-10-10T20:42:26.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.256+0000 7fd0037fe700 1 -- 172.21.15.89:0/2580825113 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01000c960 con 0x7fd01c16aaa0 2023-10-10T20:42:26.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.256+0000 7fd020a78700 1 -- 172.21.15.89:0/2580825113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd01c0c81a0 con 0x7fd01c16aaa0 2023-10-10T20:42:26.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.256+0000 7fd0037fe700 1 -- 172.21.15.89:0/2580825113 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01000cd00 con 0x7fd01c16aaa0 2023-10-10T20:42:26.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.256+0000 7fd020a78700 1 -- 172.21.15.89:0/2580825113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd01c0c7bc0 con 0x7fd01c16aaa0 2023-10-10T20:42:26.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.257+0000 7fd020a78700 1 -- 172.21.15.89:0/2580825113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd01c16b370 con 0x7fd01c16aaa0 2023-10-10T20:42:26.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.257+0000 7fd0037fe700 1 -- 172.21.15.89:0/2580825113 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd01001f620 con 0x7fd01c16aaa0 2023-10-10T20:42:26.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.258+0000 7fd0037fe700 1 --2- 172.21.15.89:0/2580825113 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd008067320 0x7fd0080697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:26.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.258+0000 7fd0037fe700 1 -- 172.21.15.89:0/2580825113 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd010086480 con 0x7fd01c16aaa0 2023-10-10T20:42:26.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.258+0000 7fd01b7fe700 1 --2- 172.21.15.89:0/2580825113 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd008067320 0x7fd0080697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:26.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.258+0000 7fd01b7fe700 1 --2- 172.21.15.89:0/2580825113 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd008067320 0x7fd0080697d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd00c001190 tx=0x7fd00c006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:26.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.262+0000 7fd0037fe700 1 -- 172.21.15.89:0/2580825113 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd010053110 con 0x7fd01c16aaa0 2023-10-10T20:42:26.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:26.490+0000 7fd020a78700 1 -- 172.21.15.89:0/2580825113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd01c04bb80 con 0x7fd01c16aaa0 2023-10-10T20:42:27.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.123+0000 7fd0037fe700 1 -- 172.21.15.89:0/2580825113 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v101) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd010056730 con 0x7fd01c16aaa0 2023-10-10T20:42:27.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.125+0000 7fd020a78700 1 -- 172.21.15.89:0/2580825113 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd008067320 msgr2=0x7fd0080697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:27.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.125+0000 7fd020a78700 1 --2- 172.21.15.89:0/2580825113 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd008067320 0x7fd0080697d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd00c001190 tx=0x7fd00c006040 comp rx=0 tx=0).stop 2023-10-10T20:42:27.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.125+0000 7fd020a78700 1 -- 172.21.15.89:0/2580825113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd01c16aaa0 msgr2=0x7fd01c0a6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:27.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.125+0000 7fd020a78700 1 --2- 172.21.15.89:0/2580825113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd01c16aaa0 0x7fd01c0a6f00 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd010001010 tx=0x7fd010015000 comp rx=0 tx=0).stop 2023-10-10T20:42:27.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.125+0000 7fd020a78700 1 -- 172.21.15.89:0/2580825113 shutdown_connections 2023-10-10T20:42:27.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.125+0000 7fd020a78700 1 --2- 172.21.15.89:0/2580825113 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd01c0a7440 0x7fd01c0cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.125+0000 7fd020a78700 1 --2- 172.21.15.89:0/2580825113 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd008067320 0x7fd0080697d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.125+0000 7fd020a78700 1 --2- 172.21.15.89:0/2580825113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd004004810 0x7fd01c0c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.125+0000 7fd020a78700 1 --2- 172.21.15.89:0/2580825113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd01c16aaa0 0x7fd01c0a6f00 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.125+0000 7fd020a78700 1 -- 172.21.15.89:0/2580825113 >> 172.21.15.89:0/2580825113 conn(0x7fd01c0d10c0 msgr2=0x7fd01c159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:27.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.125+0000 7fd020a78700 1 -- 172.21.15.89:0/2580825113 shutdown_connections 2023-10-10T20:42:27.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.125+0000 7fd020a78700 1 -- 172.21.15.89:0/2580825113 wait complete. 2023-10-10T20:42:27.140 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:42:27.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.253+0000 7ffb3cc90700 1 Processor -- start 2023-10-10T20:42:27.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.253+0000 7ffb3cc90700 1 -- start start 2023-10-10T20:42:27.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.254+0000 7ffb3cc90700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ffb38159b30 0x7ffb380697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.254+0000 7ffb3cc90700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffb38159f00 0x7ffb380ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.254+0000 7ffb3cc90700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb380c8e20 con 0x7ffb3806ba80 2023-10-10T20:42:27.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.254+0000 7ffb3cc90700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb380c8c90 con 0x7ffb38159b30 2023-10-10T20:42:27.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.254+0000 7ffb3cc90700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3815a7c0 con 0x7ffb38159f00 2023-10-10T20:42:27.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.254+0000 7ffb37fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ffb3806ba80 0x7ffb380c92b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48988/0 (socket says 172.21.15.89:48988) 2023-10-10T20:42:27.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.254+0000 7ffb37fff700 1 -- 172.21.15.89:0/2629460604 learned_addr learned my addr 172.21.15.89:0/2629460604 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:27.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.254+0000 7ffb367fc700 1 -- 172.21.15.89:0/2629460604 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807110786 0 0) 0x7ffb380c8e20 con 0x7ffb3806ba80 2023-10-10T20:42:27.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.254+0000 7ffb367fc700 1 -- 172.21.15.89:0/2629460604 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb20003570 con 0x7ffb3806ba80 2023-10-10T20:42:27.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.254+0000 7ffb36ffd700 1 --2- 172.21.15.89:0/2629460604 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffb38159f00 0x7ffb380ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:27.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.254+0000 7ffb377fe700 1 --2- 172.21.15.89:0/2629460604 >> v2:172.21.15.111:3300/0 conn(0x7ffb38159b30 0x7ffb380697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:27.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb367fc700 1 -- 172.21.15.89:0/2629460604 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 140214969 0 0) 0x7ffb20003570 con 0x7ffb3806ba80 2023-10-10T20:42:27.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb367fc700 1 -- 172.21.15.89:0/2629460604 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffb38159f00 msgr2=0x7ffb380ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb367fc700 1 --2- 172.21.15.89:0/2629460604 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffb38159f00 0x7ffb380ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb367fc700 1 -- 172.21.15.89:0/2629460604 >> v2:172.21.15.111:3300/0 conn(0x7ffb38159b30 msgr2=0x7ffb380697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb367fc700 1 --2- 172.21.15.89:0/2629460604 >> v2:172.21.15.111:3300/0 conn(0x7ffb38159b30 0x7ffb380697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb367fc700 1 -- 172.21.15.89:0/2629460604 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3815aaa0 con 0x7ffb3806ba80 2023-10-10T20:42:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb367fc700 1 -- 172.21.15.89:0/2629460604 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7ffb30001470 con 0x7ffb3806ba80 2023-10-10T20:42:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb367fc700 1 -- 172.21.15.89:0/2629460604 >> v1:172.21.15.89:6789/0 conn(0x7ffb3806ba80 legacy=0x7ffb380c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb367fc700 1 --2- 172.21.15.89:0/2629460604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb20003a60 0x7ffb20003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb367fc700 1 --2- 172.21.15.89:0/2629460604 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffb38159f00 0x7ffb20004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb367fc700 1 --2- 172.21.15.89:0/2629460604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb20004d50 0x7ffb200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb367fc700 1 -- 172.21.15.89:0/2629460604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb380c8c90 con 0x7ffb20004d50 2023-10-10T20:42:27.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb367fc700 1 -- 172.21.15.89:0/2629460604 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3815a7c0 con 0x7ffb20003a60 2023-10-10T20:42:27.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb367fc700 1 -- 172.21.15.89:0/2629460604 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2000bae0 con 0x7ffb38159f00 2023-10-10T20:42:27.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb377fe700 1 --2- 172.21.15.89:0/2629460604 >> v2:172.21.15.111:3300/0 conn(0x7ffb38159b30 0x7ffb380697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:27.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb37fff700 1 --2- 172.21.15.89:0/2629460604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb20004d50 0x7ffb200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:27.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb36ffd700 1 --2- 172.21.15.89:0/2629460604 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffb38159f00 0x7ffb20004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:27.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb377fe700 1 --2- 172.21.15.89:0/2629460604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb20003a60 0x7ffb20003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:27.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb37fff700 1 -- 172.21.15.89:0/2629460604 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffb38159f00 msgr2=0x7ffb20004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:27.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb37fff700 1 --2- 172.21.15.89:0/2629460604 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffb38159f00 0x7ffb20004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb37fff700 1 -- 172.21.15.89:0/2629460604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb20003a60 msgr2=0x7ffb20003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:27.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb37fff700 1 --2- 172.21.15.89:0/2629460604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb20003a60 0x7ffb20003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb37fff700 1 -- 172.21.15.89:0/2629460604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb20003570 con 0x7ffb20004d50 2023-10-10T20:42:27.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb37fff700 1 --2- 172.21.15.89:0/2629460604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb20004d50 0x7ffb200051c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000c3c0 tx=0x7ffb30001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3238d6dc9a61a80d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:27.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb367fc700 1 -- 172.21.15.89:0/2629460604 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3000cc90 con 0x7ffb20004d50 2023-10-10T20:42:27.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb367fc700 1 -- 172.21.15.89:0/2629460604 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3000ce10 con 0x7ffb20004d50 2023-10-10T20:42:27.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.255+0000 7ffb367fc700 1 -- 172.21.15.89:0/2629460604 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30005b70 con 0x7ffb20004d50 2023-10-10T20:42:27.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.256+0000 7ffb3cc90700 1 -- 172.21.15.89:0/2629460604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb20004d50 msgr2=0x7ffb200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:27.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.256+0000 7ffb3cc90700 1 --2- 172.21.15.89:0/2629460604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb20004d50 0x7ffb200051c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000c3c0 tx=0x7ffb30001e90 comp rx=0 tx=0).stop 2023-10-10T20:42:27.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.256+0000 7ffb37fff700 1 -- 172.21.15.89:0/2629460604 reap_dead start 2023-10-10T20:42:27.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.256+0000 7ffb3cc90700 1 -- 172.21.15.89:0/2629460604 shutdown_connections 2023-10-10T20:42:27.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.256+0000 7ffb3cc90700 1 -- 172.21.15.89:0/2629460604 >> 172.21.15.89:0/2629460604 conn(0x7ffb380ce650 msgr2=0x7ffb380cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:27.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.256+0000 7ffb3cc90700 1 -- 172.21.15.89:0/2629460604 shutdown_connections 2023-10-10T20:42:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.256+0000 7ffb3cc90700 1 -- 172.21.15.89:0/2629460604 wait complete. 2023-10-10T20:42:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.257+0000 7ffb3cc90700 1 Processor -- start 2023-10-10T20:42:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.257+0000 7ffb3cc90700 1 -- start start 2023-10-10T20:42:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.258+0000 7ffb3cc90700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffb38169e30 0x7ffb3816b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.258+0000 7ffb3cc90700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb3816d8f0 0x7ffb3816dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.258+0000 7ffb3cc90700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb380c78c0 0x7ffb380c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.258+0000 7ffb3cc90700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3816e2a0 con 0x7ffb380c78c0 2023-10-10T20:42:27.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.258+0000 7ffb3cc90700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb38157760 con 0x7ffb3816d8f0 2023-10-10T20:42:27.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.258+0000 7ffb3cc90700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb380c8660 con 0x7ffb38169e30 2023-10-10T20:42:27.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.258+0000 7ffb37fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb380c78c0 0x7ffb380c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:27.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.258+0000 7ffb37fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb380c78c0 0x7ffb380c7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44718/0 (socket says 172.21.15.89:44718) 2023-10-10T20:42:27.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.258+0000 7ffb37fff700 1 -- 172.21.15.89:0/3474093199 learned_addr learned my addr 172.21.15.89:0/3474093199 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:27.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.258+0000 7ffb37fff700 1 -- 172.21.15.89:0/3474093199 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffb38169e30 msgr2=0x7ffb3816b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:27.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.258+0000 7ffb37fff700 1 --2- 172.21.15.89:0/3474093199 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffb38169e30 0x7ffb3816b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.258+0000 7ffb37fff700 1 -- 172.21.15.89:0/3474093199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb3816d8f0 msgr2=0x7ffb3816dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:27.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.258+0000 7ffb36ffd700 1 --2- 172.21.15.89:0/3474093199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb3816d8f0 0x7ffb3816dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:27.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.258+0000 7ffb37fff700 1 --2- 172.21.15.89:0/3474093199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb3816d8f0 0x7ffb3816dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.258+0000 7ffb37fff700 1 -- 172.21.15.89:0/3474093199 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb38157110 con 0x7ffb380c78c0 2023-10-10T20:42:27.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.259+0000 7ffb37fff700 1 --2- 172.21.15.89:0/3474093199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb380c78c0 0x7ffb380c7d30 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7ffb30001ed0 tx=0x7ffb30015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:27.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.259+0000 7ffb1f7fe700 1 -- 172.21.15.89:0/3474093199 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3000cc90 con 0x7ffb380c78c0 2023-10-10T20:42:27.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.259+0000 7ffb1f7fe700 1 -- 172.21.15.89:0/3474093199 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb30001430 con 0x7ffb380c78c0 2023-10-10T20:42:27.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.259+0000 7ffb3cc90700 1 -- 172.21.15.89:0/3474093199 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb381573f0 con 0x7ffb380c78c0 2023-10-10T20:42:27.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.259+0000 7ffb1f7fe700 1 -- 172.21.15.89:0/3474093199 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30007530 con 0x7ffb380c78c0 2023-10-10T20:42:27.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.259+0000 7ffb3cc90700 1 -- 172.21.15.89:0/3474093199 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb380c8430 con 0x7ffb380c78c0 2023-10-10T20:42:27.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.260+0000 7ffb3cc90700 1 -- 172.21.15.89:0/3474093199 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3804c3c0 con 0x7ffb380c78c0 2023-10-10T20:42:27.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.260+0000 7ffb1f7fe700 1 -- 172.21.15.89:0/3474093199 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ffb30012400 con 0x7ffb380c78c0 2023-10-10T20:42:27.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.260+0000 7ffb1f7fe700 1 --2- 172.21.15.89:0/3474093199 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ffb24067350 0x7ffb24069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.260+0000 7ffb1f7fe700 1 -- 172.21.15.89:0/3474093199 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffb30085310 con 0x7ffb380c78c0 2023-10-10T20:42:27.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.260+0000 7ffb377fe700 1 --2- 172.21.15.89:0/3474093199 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ffb24067350 0x7ffb24069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:27.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.261+0000 7ffb377fe700 1 --2- 172.21.15.89:0/3474093199 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ffb24067350 0x7ffb24069800 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ffb28007460 tx=0x7ffb2800e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:27.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.265+0000 7ffb1f7fe700 1 -- 172.21.15.89:0/3474093199 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb30051fa0 con 0x7ffb380c78c0 2023-10-10T20:42:27.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.490+0000 7ffb3cc90700 1 -- 172.21.15.89:0/3474093199 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb3815ad80 con 0x7ffb380c78c0 2023-10-10T20:42:27.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.490+0000 7ffb1f7fe700 1 -- 172.21.15.89:0/3474093199 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4606 (secure 0 0 0) 0x7ffb300555c0 con 0x7ffb380c78c0 2023-10-10T20:42:27.492 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:27.493 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":101,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/830099189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":830099189},{"type":"v1","addr":"172.21.15.111:6834","nonce":830099189}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2380789310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2380789310},{"type":"v1","addr":"172.21.15.111:6835","nonce":2380789310}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7956,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2087922155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2087922155},{"type":"v1","addr":"172.21.15.89:6835","nonce":2087922155}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:42:23.479232+0000","modified":"2023-10-10T20:42:27.115668+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1243871428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1243871428},{"type":"v1","addr":"172.21.15.89:6837","nonce":1243871428}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-10T20:42:27.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.492+0000 7ffb3cc90700 1 -- 172.21.15.89:0/3474093199 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ffb24067350 msgr2=0x7ffb24069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:27.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.492+0000 7ffb3cc90700 1 --2- 172.21.15.89:0/3474093199 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ffb24067350 0x7ffb24069800 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ffb28007460 tx=0x7ffb2800e040 comp rx=0 tx=0).stop 2023-10-10T20:42:27.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.492+0000 7ffb3cc90700 1 -- 172.21.15.89:0/3474093199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb380c78c0 msgr2=0x7ffb380c7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:27.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.492+0000 7ffb3cc90700 1 --2- 172.21.15.89:0/3474093199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb380c78c0 0x7ffb380c7d30 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7ffb30001ed0 tx=0x7ffb30015000 comp rx=0 tx=0).stop 2023-10-10T20:42:27.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.492+0000 7ffb3cc90700 1 -- 172.21.15.89:0/3474093199 shutdown_connections 2023-10-10T20:42:27.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.492+0000 7ffb3cc90700 1 --2- 172.21.15.89:0/3474093199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb380c78c0 0x7ffb380c7d30 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.492+0000 7ffb3cc90700 1 --2- 172.21.15.89:0/3474093199 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ffb38169e30 0x7ffb3816b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.492+0000 7ffb3cc90700 1 --2- 172.21.15.89:0/3474093199 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ffb24067350 0x7ffb24069800 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.492+0000 7ffb3cc90700 1 --2- 172.21.15.89:0/3474093199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb3816d8f0 0x7ffb3816dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.492+0000 7ffb3cc90700 1 -- 172.21.15.89:0/3474093199 >> 172.21.15.89:0/3474093199 conn(0x7ffb380ce650 msgr2=0x7ffb38156330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:27.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.492+0000 7ffb3cc90700 1 -- 172.21.15.89:0/3474093199 shutdown_connections 2023-10-10T20:42:27.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.493+0000 7ffb3cc90700 1 -- 172.21.15.89:0/3474093199 wait complete. 2023-10-10T20:42:27.498 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 101 2023-10-10T20:42:27.507 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:42:27.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.621+0000 7ff47f249700 1 Processor -- start 2023-10-10T20:42:27.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.621+0000 7ff47f249700 1 -- start start 2023-10-10T20:42:27.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.622+0000 7ff47f249700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff478159890 0x7ff47815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.622+0000 7ff47f249700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff47815a160 con 0x7ff478069560 2023-10-10T20:42:27.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.622+0000 7ff47f249700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4780d13a0 con 0x7ff478159c60 2023-10-10T20:42:27.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.622+0000 7ff47f249700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff47815e140 con 0x7ff478159890 2023-10-10T20:42:27.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.622+0000 7ff47cfe5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff478159890 0x7ff47815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:27.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.622+0000 7ff47cfe5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff478159890 0x7ff47815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44722/0 (socket says 172.21.15.89:44722) 2023-10-10T20:42:27.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.622+0000 7ff47cfe5700 1 -- 172.21.15.89:0/318354975 learned_addr learned my addr 172.21.15.89:0/318354975 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:27.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.622+0000 7ff47d7e6700 1 --1- 172.21.15.89:0/318354975 >> v1:172.21.15.89:6789/0 conn(0x7ff478069560 0x7ff4780ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48998/0 (socket says 172.21.15.89:48998) 2023-10-10T20:42:27.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.622+0000 7ff47cfe5700 1 -- 172.21.15.89:0/318354975 >> v1:172.21.15.111:6789/0 conn(0x7ff478159c60 legacy=0x7ff47815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:27.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.622+0000 7ff47cfe5700 1 -- 172.21.15.89:0/318354975 >> v1:172.21.15.89:6789/0 conn(0x7ff478069560 legacy=0x7ff4780ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:27.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.622+0000 7ff47cfe5700 1 -- 172.21.15.89:0/318354975 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47815a3d0 con 0x7ff478159890 2023-10-10T20:42:27.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.622+0000 7ff4777fe700 1 -- 172.21.15.89:0/318354975 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301605160 0 0) 0x7ff47815a160 con 0x7ff478069560 2023-10-10T20:42:27.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.622+0000 7ff47cfe5700 1 --2- 172.21.15.89:0/318354975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff478159890 0x7ff47815abd0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7ff470009f20 tx=0x7ff470009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c9753259367823 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:27.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.622+0000 7ff4777fe700 1 -- 172.21.15.89:0/318354975 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47000e070 con 0x7ff478159890 2023-10-10T20:42:27.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.622+0000 7ff4777fe700 1 -- 172.21.15.89:0/318354975 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47000ba80 con 0x7ff478159890 2023-10-10T20:42:27.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.622+0000 7ff4777fe700 1 -- 172.21.15.89:0/318354975 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4700092e0 con 0x7ff478159890 2023-10-10T20:42:27.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.623+0000 7ff47f249700 1 -- 172.21.15.89:0/318354975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff478159890 msgr2=0x7ff47815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:27.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.623+0000 7ff47f249700 1 --2- 172.21.15.89:0/318354975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff478159890 0x7ff47815abd0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7ff470009f20 tx=0x7ff470009840 comp rx=0 tx=0).stop 2023-10-10T20:42:27.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.623+0000 7ff47f249700 1 -- 172.21.15.89:0/318354975 shutdown_connections 2023-10-10T20:42:27.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.623+0000 7ff47f249700 1 --2- 172.21.15.89:0/318354975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff478159890 0x7ff47815abd0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.623+0000 7ff47f249700 1 -- 172.21.15.89:0/318354975 >> 172.21.15.89:0/318354975 conn(0x7ff478068e20 msgr2=0x7ff4780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:27.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.623+0000 7ff47f249700 1 -- 172.21.15.89:0/318354975 shutdown_connections 2023-10-10T20:42:27.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.623+0000 7ff47f249700 1 -- 172.21.15.89:0/318354975 wait complete. 2023-10-10T20:42:27.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.624+0000 7ff47f249700 1 Processor -- start 2023-10-10T20:42:27.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.624+0000 7ff47f249700 1 -- start start 2023-10-10T20:42:27.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.625+0000 7ff47f249700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff478069560 0x7ff478154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.625+0000 7ff47f249700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff478159890 0x7ff47815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.625+0000 7ff47f249700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff478159c60 0x7ff47815c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.625+0000 7ff47f249700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47814cba0 con 0x7ff478069560 2023-10-10T20:42:27.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.625+0000 7ff47f249700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47816b440 con 0x7ff478159890 2023-10-10T20:42:27.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.625+0000 7ff47f249700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47815ca30 con 0x7ff478159c60 2023-10-10T20:42:27.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.625+0000 7ff47cfe5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff478069560 0x7ff478154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:27.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.625+0000 7ff47cfe5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff478069560 0x7ff478154b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44728/0 (socket says 172.21.15.89:44728) 2023-10-10T20:42:27.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.625+0000 7ff47cfe5700 1 -- 172.21.15.89:0/4130974324 learned_addr learned my addr 172.21.15.89:0/4130974324 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:27.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.625+0000 7ff47d7e6700 1 --2- 172.21.15.89:0/4130974324 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff478159c60 0x7ff47815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:27.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.625+0000 7ff477fff700 1 --2- 172.21.15.89:0/4130974324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff478159890 0x7ff47815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:27.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.625+0000 7ff47cfe5700 1 -- 172.21.15.89:0/4130974324 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff478159c60 msgr2=0x7ff47815c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:27.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.625+0000 7ff47cfe5700 1 --2- 172.21.15.89:0/4130974324 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff478159c60 0x7ff47815c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.625+0000 7ff47cfe5700 1 -- 172.21.15.89:0/4130974324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff478159890 msgr2=0x7ff47815d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:27.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.625+0000 7ff47cfe5700 1 --2- 172.21.15.89:0/4130974324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff478159890 0x7ff47815d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.625+0000 7ff47cfe5700 1 -- 172.21.15.89:0/4130974324 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff470009ae0 con 0x7ff478069560 2023-10-10T20:42:27.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.625+0000 7ff477fff700 1 --2- 172.21.15.89:0/4130974324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff478159890 0x7ff47815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:27.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.626+0000 7ff47cfe5700 1 --2- 172.21.15.89:0/4130974324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff478069560 0x7ff478154b80 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7ff470009ab0 tx=0x7ff470018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:27.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.626+0000 7ff474ff9700 1 -- 172.21.15.89:0/4130974324 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff470007ab0 con 0x7ff478069560 2023-10-10T20:42:27.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.626+0000 7ff474ff9700 1 -- 172.21.15.89:0/4130974324 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff470007c30 con 0x7ff478069560 2023-10-10T20:42:27.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.626+0000 7ff47f249700 1 -- 172.21.15.89:0/4130974324 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff47816b160 con 0x7ff478069560 2023-10-10T20:42:27.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.626+0000 7ff474ff9700 1 -- 172.21.15.89:0/4130974324 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff470026460 con 0x7ff478069560 2023-10-10T20:42:27.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.626+0000 7ff47f249700 1 -- 172.21.15.89:0/4130974324 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47815cfb0 con 0x7ff478069560 2023-10-10T20:42:27.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.627+0000 7ff47f249700 1 -- 172.21.15.89:0/4130974324 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47804c3c0 con 0x7ff478069560 2023-10-10T20:42:27.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.627+0000 7ff474ff9700 1 -- 172.21.15.89:0/4130974324 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff470026870 con 0x7ff478069560 2023-10-10T20:42:27.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.628+0000 7ff474ff9700 1 --2- 172.21.15.89:0/4130974324 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff458067320 0x7ff4580697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.628+0000 7ff477fff700 1 --2- 172.21.15.89:0/4130974324 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff458067320 0x7ff4580697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:27.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.628+0000 7ff474ff9700 1 -- 172.21.15.89:0/4130974324 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff470086bd0 con 0x7ff478069560 2023-10-10T20:42:27.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.628+0000 7ff477fff700 1 --2- 172.21.15.89:0/4130974324 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff458067320 0x7ff4580697d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff468007fd0 tx=0x7ff468009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:27.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.632+0000 7ff474ff9700 1 -- 172.21.15.89:0/4130974324 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff470053750 con 0x7ff478069560 2023-10-10T20:42:27.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.815+0000 7ff47f249700 1 -- 172.21.15.89:0/4130974324 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff478158ef0 con 0x7ff478069560 2023-10-10T20:42:27.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.816+0000 7ff474ff9700 1 -- 172.21.15.89:0/4130974324 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+15364 (secure 0 0 0) 0x7ff470056d70 con 0x7ff478069560 2023-10-10T20:42:27.818 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:27.819 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":153,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:42:25.092451+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:42:21.831397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-10T20:42:22.440492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:42:27.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.818+0000 7ff47f249700 1 -- 172.21.15.89:0/4130974324 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff458067320 msgr2=0x7ff4580697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:27.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.818+0000 7ff47f249700 1 --2- 172.21.15.89:0/4130974324 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff458067320 0x7ff4580697d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff468007fd0 tx=0x7ff468009040 comp rx=0 tx=0).stop 2023-10-10T20:42:27.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.818+0000 7ff47f249700 1 -- 172.21.15.89:0/4130974324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff478069560 msgr2=0x7ff478154b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:27.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.818+0000 7ff47f249700 1 --2- 172.21.15.89:0/4130974324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff478069560 0x7ff478154b80 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7ff470009ab0 tx=0x7ff470018000 comp rx=0 tx=0).stop 2023-10-10T20:42:27.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.818+0000 7ff47f249700 1 -- 172.21.15.89:0/4130974324 shutdown_connections 2023-10-10T20:42:27.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.818+0000 7ff47f249700 1 --2- 172.21.15.89:0/4130974324 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff478159c60 0x7ff47815c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.818+0000 7ff47f249700 1 --2- 172.21.15.89:0/4130974324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff478069560 0x7ff478154b80 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.818+0000 7ff47f249700 1 --2- 172.21.15.89:0/4130974324 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff458067320 0x7ff4580697d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.818+0000 7ff47f249700 1 --2- 172.21.15.89:0/4130974324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff478159890 0x7ff47815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.818+0000 7ff47f249700 1 -- 172.21.15.89:0/4130974324 >> 172.21.15.89:0/4130974324 conn(0x7ff478068e20 msgr2=0x7ff478152da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:27.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.818+0000 7ff47f249700 1 -- 172.21.15.89:0/4130974324 shutdown_connections 2023-10-10T20:42:27.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.818+0000 7ff47f249700 1 -- 172.21.15.89:0/4130974324 wait complete. 2023-10-10T20:42:27.832 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:42:27.833 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:42:27.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.945+0000 7f840c751700 1 Processor -- start 2023-10-10T20:42:27.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.945+0000 7f840c751700 1 -- start start 2023-10-10T20:42:27.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.946+0000 7f840c751700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8404159c60 0x7f840415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.946+0000 7f840c751700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f840415db10 0x7f8404069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.946+0000 7f840c751700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840415a160 con 0x7f8404159890 2023-10-10T20:42:27.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.946+0000 7f840c751700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84040d13a0 con 0x7f840415db10 2023-10-10T20:42:27.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.946+0000 7f840c751700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840415e140 con 0x7f8404159c60 2023-10-10T20:42:27.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.946+0000 7f840a4ed700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8404159890 0x7f840415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49012/0 (socket says 172.21.15.89:49012) 2023-10-10T20:42:27.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.946+0000 7f840a4ed700 1 -- 172.21.15.89:0/3322237105 learned_addr learned my addr 172.21.15.89:0/3322237105 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:27.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.946+0000 7f8409cec700 1 --2- 172.21.15.89:0/3322237105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8404159c60 0x7f840415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:27.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.946+0000 7f840acee700 1 --2- 172.21.15.89:0/3322237105 >> v2:172.21.15.111:3300/0 conn(0x7f840415db10 0x7f8404069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:27.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.947+0000 7f84094eb700 1 -- 172.21.15.89:0/3322237105 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2868460713 0 0) 0x7f840415a160 con 0x7f8404159890 2023-10-10T20:42:27.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.947+0000 7f84094eb700 1 -- 172.21.15.89:0/3322237105 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83f0003570 con 0x7f8404159890 2023-10-10T20:42:27.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.947+0000 7f84094eb700 1 -- 172.21.15.89:0/3322237105 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3515066427 0 0) 0x7f83f0003570 con 0x7f8404159890 2023-10-10T20:42:27.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.947+0000 7f8409cec700 1 -- 172.21.15.89:0/3322237105 >> v2:172.21.15.111:3300/0 conn(0x7f840415db10 msgr2=0x7f8404069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:27.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.947+0000 7f8409cec700 1 --2- 172.21.15.89:0/3322237105 >> v2:172.21.15.111:3300/0 conn(0x7f840415db10 0x7f8404069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.947+0000 7f8409cec700 1 -- 172.21.15.89:0/3322237105 >> v1:172.21.15.89:6789/0 conn(0x7f8404159890 legacy=0x7f840415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:27.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.947+0000 7f8409cec700 1 -- 172.21.15.89:0/3322237105 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840415a3d0 con 0x7f8404159c60 2023-10-10T20:42:27.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.947+0000 7f8409cec700 1 --2- 172.21.15.89:0/3322237105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8404159c60 0x7f840415d5d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f83f40098e0 tx=0x7f83f40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6745f328a5a7b562 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:27.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.947+0000 7f84094eb700 1 -- 172.21.15.89:0/3322237105 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400001470 con 0x7f8404159c60 2023-10-10T20:42:27.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.947+0000 7f84094eb700 1 -- 172.21.15.89:0/3322237105 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f4011420 con 0x7f8404159c60 2023-10-10T20:42:27.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.947+0000 7f84094eb700 1 -- 172.21.15.89:0/3322237105 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f40117a0 con 0x7f8404159c60 2023-10-10T20:42:27.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.948+0000 7f840c751700 1 -- 172.21.15.89:0/3322237105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8404159c60 msgr2=0x7f840415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:27.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.948+0000 7f840c751700 1 --2- 172.21.15.89:0/3322237105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8404159c60 0x7f840415d5d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f83f40098e0 tx=0x7f83f40091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.948+0000 7f840c751700 1 -- 172.21.15.89:0/3322237105 shutdown_connections 2023-10-10T20:42:27.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.948+0000 7f840c751700 1 --2- 172.21.15.89:0/3322237105 >> v2:172.21.15.111:3300/0 conn(0x7f840415db10 0x7f8404069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.948+0000 7f840c751700 1 --2- 172.21.15.89:0/3322237105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8404159c60 0x7f840415d5d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.948+0000 7f840c751700 1 -- 172.21.15.89:0/3322237105 >> 172.21.15.89:0/3322237105 conn(0x7f8404068e20 msgr2=0x7f84040d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:27.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.948+0000 7f840c751700 1 -- 172.21.15.89:0/3322237105 shutdown_connections 2023-10-10T20:42:27.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.948+0000 7f840c751700 1 -- 172.21.15.89:0/3322237105 wait complete. 2023-10-10T20:42:27.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.949+0000 7f840c751700 1 Processor -- start 2023-10-10T20:42:27.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.949+0000 7f840c751700 1 -- start start 2023-10-10T20:42:27.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f840c751700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8404159890 0x7f84041526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f840c751700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8404159c60 0x7f840416a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f840c751700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f840415db10 0x7f8404150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f840c751700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8404150e10 con 0x7f8404159890 2023-10-10T20:42:27.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f840c751700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840416a8f0 con 0x7f840415db10 2023-10-10T20:42:27.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f840c751700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840416ab90 con 0x7f8404159c60 2023-10-10T20:42:27.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f840a4ed700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8404159890 0x7f84041526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:27.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f840a4ed700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8404159890 0x7f84041526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44734/0 (socket says 172.21.15.89:44734) 2023-10-10T20:42:27.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f840a4ed700 1 -- 172.21.15.89:0/3432081530 learned_addr learned my addr 172.21.15.89:0/3432081530 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:27.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f8409cec700 1 --2- 172.21.15.89:0/3432081530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8404159c60 0x7f840416a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:27.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f840acee700 1 --2- 172.21.15.89:0/3432081530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f840415db10 0x7f8404150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:27.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f840a4ed700 1 -- 172.21.15.89:0/3432081530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8404159c60 msgr2=0x7f840416a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:27.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f840a4ed700 1 --2- 172.21.15.89:0/3432081530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8404159c60 0x7f840416a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f840a4ed700 1 -- 172.21.15.89:0/3432081530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f840415db10 msgr2=0x7f8404150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:27.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f840a4ed700 1 --2- 172.21.15.89:0/3432081530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f840415db10 0x7f8404150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:27.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f840a4ed700 1 -- 172.21.15.89:0/3432081530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f4009420 con 0x7f8404159890 2023-10-10T20:42:27.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f840a4ed700 1 --2- 172.21.15.89:0/3432081530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8404159890 0x7f84041526e0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f840000cb90 tx=0x7f8400001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:27.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f83fa7fc700 1 -- 172.21.15.89:0/3432081530 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400001d60 con 0x7f8404159890 2023-10-10T20:42:27.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f83fa7fc700 1 -- 172.21.15.89:0/3432081530 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8400007360 con 0x7f8404159890 2023-10-10T20:42:27.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f840c751700 1 -- 172.21.15.89:0/3432081530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840416ae70 con 0x7f8404159890 2023-10-10T20:42:27.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f83fa7fc700 1 -- 172.21.15.89:0/3432081530 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400007700 con 0x7f8404159890 2023-10-10T20:42:27.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.950+0000 7f840c751700 1 -- 172.21.15.89:0/3432081530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8404150950 con 0x7f8404159890 2023-10-10T20:42:27.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.951+0000 7f840c751700 1 -- 172.21.15.89:0/3432081530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840416b390 con 0x7f8404159890 2023-10-10T20:42:27.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.952+0000 7f83fa7fc700 1 -- 172.21.15.89:0/3432081530 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8400004a40 con 0x7f8404159890 2023-10-10T20:42:27.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.952+0000 7f83fa7fc700 1 --2- 172.21.15.89:0/3432081530 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83e8067320 0x7f83e80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:27.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.952+0000 7f83fa7fc700 1 -- 172.21.15.89:0/3432081530 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8400084500 con 0x7f8404159890 2023-10-10T20:42:27.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.952+0000 7f8409cec700 1 --2- 172.21.15.89:0/3432081530 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83e8067320 0x7f83e80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:27.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.952+0000 7f8409cec700 1 --2- 172.21.15.89:0/3432081530 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83e8067320 0x7f83e80697d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f840415a090 tx=0x7f83f4016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:27.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:27.956+0000 7f83fa7fc700 1 -- 172.21.15.89:0/3432081530 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f840416b390 con 0x7f8404159890 2023-10-10T20:42:28.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.136+0000 7f840c751700 1 -- 172.21.15.89:0/3432081530 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8404150f50 con 0x7f83e8067320 2023-10-10T20:42:28.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.144+0000 7f83fa7fc700 1 -- 172.21.15.89:0/3432081530 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279133 (secure 0 0 0) 0x7f83f4011790 con 0x7f83e8067320 2023-10-10T20:42:28.150 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:28.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.147+0000 7f840c751700 1 -- 172.21.15.89:0/3432081530 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83e8067320 msgr2=0x7f83e80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:28.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.147+0000 7f840c751700 1 --2- 172.21.15.89:0/3432081530 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83e8067320 0x7f83e80697d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f840415a090 tx=0x7f83f4016000 comp rx=0 tx=0).stop 2023-10-10T20:42:28.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.147+0000 7f840c751700 1 -- 172.21.15.89:0/3432081530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8404159890 msgr2=0x7f84041526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:28.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.147+0000 7f840c751700 1 --2- 172.21.15.89:0/3432081530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8404159890 0x7f84041526e0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f840000cb90 tx=0x7f8400001ad0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.147+0000 7f840c751700 1 -- 172.21.15.89:0/3432081530 shutdown_connections 2023-10-10T20:42:28.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.147+0000 7f840c751700 1 --2- 172.21.15.89:0/3432081530 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83e8067320 0x7f83e80697d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.147+0000 7f840c751700 1 --2- 172.21.15.89:0/3432081530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f840415db10 0x7f8404150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.147+0000 7f840c751700 1 --2- 172.21.15.89:0/3432081530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8404159890 0x7f84041526e0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.147+0000 7f840c751700 1 --2- 172.21.15.89:0/3432081530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8404159c60 0x7f840416a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.147+0000 7f840c751700 1 -- 172.21.15.89:0/3432081530 >> 172.21.15.89:0/3432081530 conn(0x7f8404068e20 msgr2=0x7f8404069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:28.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.147+0000 7f840c751700 1 -- 172.21.15.89:0/3432081530 shutdown_connections 2023-10-10T20:42:28.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.147+0000 7f840c751700 1 -- 172.21.15.89:0/3432081530 wait complete. 2023-10-10T20:42:28.155 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:42:28.161 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2023-10-10T20:42:26.745422+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427076,"kb_used_data":16004,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547644,"statfs":{"total":773094113280,"available":770608787456,"internally_reserved":0,"allocated":16388096,"data_stored":7971839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.658883"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.095350+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:06:20.346560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.091710+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:29:30.897091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.098436+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:40:22.937275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.092780+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:36:42.204376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.095195+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:59.544805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.093851+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:17.799833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.091829+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:06:20.346560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.095866+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:37:08.365854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.091984+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:17.799833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.094377+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:52:50.371745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.091289+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:21:58.481035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.093229+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:48:40.304153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.091790+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:56.914227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.093591+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:41:54.647934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.098056+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:31:23.373914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.094346+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:04.056737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.091172+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:32:29.013486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.093986+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:20:33.756714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.099072+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:08:05.465989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.094095+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:46:26.675244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.095136+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:33.642377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.095221+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:39:58.918529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.095508+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:00:27.632584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.098081+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:35:08.745975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.091540+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:14:17.624964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.093876+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:48:40.304153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.098788+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:51:56.794296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.091222+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:46:14.740269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.093991+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:42.951721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.091754+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:43:43.081600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.094296+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:56.914227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.095382+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:17.799833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":293,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-10T20:42:20.075255+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:42:20.075255+0000","last_peered":"2023-10-10T20:42:20.075255+0000","last_clean":"2023-10-10T20:42:20.075255+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:42:20.075255+0000","last_undegraded":"2023-10-10T20:42:20.075255+0000","last_fullsized":"2023-10-10T20:42:20.075255+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574947,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303820,"kb_used_data":2436,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068020,"statfs":{"total":96636764160,"available":96325652480,"internally_reserved":0,"allocated":2494464,"data_stored":1439376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.79000000000000004},"min":{"1min":0.309,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71799999999999997,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52400000000000002,"15min":0.61599999999999999},"min":{"1min":0.32300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.69999999999999996,"5min":0.82099999999999995,"15min":4.1959999999999997},"last":0.53600000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.95099999999999996},"min":{"1min":0.379,"5min":0.317,"15min":0.317},"max":{"1min":0.78300000000000003,"5min":0.78800000000000003,"15min":10.1},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55200000000000005,"15min":0.71399999999999997},"min":{"1min":0.36699999999999999,"5min":0.32300000000000001,"15min":0.28999999999999998},"max":{"1min":0.73199999999999998,"5min":0.90400000000000003,"15min":6.806},"last":0.501}]},{"osd":2,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.76400000000000001},"min":{"1min":0.40100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79200000000000004,"5min":0.89000000000000001,"15min":6.9340000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.71699999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.73299999999999998,"5min":0.83899999999999997,"15min":6.9870000000000001},"last":0.47999999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54900000000000004,"15min":0.73599999999999999},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.753,"5min":0.80200000000000005,"15min":6.6719999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56699999999999995,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.71699999999999997,"5min":0.87,"15min":4.1609999999999996},"last":0.46500000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.36599999999999999,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.65100000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36299999999999999,"5min":0.309,"15min":0.30399999999999999},"max":{"1min":0.72299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.314}]},{"osd":5,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.314},"max":{"1min":0.77900000000000003,"5min":0.86099999999999999,"15min":1.3180000000000001},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.41799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71499999999999997,"5min":0.75800000000000001,"15min":0.84099999999999997},"last":0.36199999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":0.78700000000000003,"15min":1.7509999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.40899999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.85699999999999998,"15min":1.7090000000000001},"last":0.374}]}]},{"osd":6,"up_from":13,"seq":55834574948,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":849008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51300000000000001,"15min":0.65500000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":7.7590000000000003},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.52100000000000002,"15min":0.69199999999999995},"min":{"1min":0.32200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.075,"5min":1.075,"15min":7.8090000000000002},"last":0.436}]},{"osd":1,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.69099999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.70199999999999996,"5min":0.80000000000000004,"15min":7.867},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.85599999999999998},"min":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.63200000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.58799999999999997},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.56100000000000005,"15min":0.81999999999999995},"min":{"1min":0.34200000000000003,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":2.641,"5min":2.641,"15min":6.7629999999999999},"last":0.81299999999999994}]},{"osd":3,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54300000000000004,"15min":0.61099999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.107,"5min":1.107,"15min":4.8620000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.70099999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.718,"5min":1.718,"15min":7.9009999999999998},"last":0.67000000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.47299999999999998},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.82899999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52700000000000002,"15min":0.495},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.53400000000000003},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.85399999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55800000000000005,"15min":0.52500000000000002},"min":{"1min":0.36099999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.372,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.78900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":849008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.312,"15min":0.45900000000000002},"min":{"1min":0.22800000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.437,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":1.194},"last":0.318}]},{"osd":2,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.24199999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51000000000000001,"5min":0.51400000000000001,"15min":1.2869999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.48999999999999999},"min":{"1min":0.27200000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.52800000000000002,"5min":0.54200000000000004,"15min":5.0919999999999996},"last":0.33500000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36399999999999999,"15min":0.51200000000000001},"min":{"1min":0.29399999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35699999999999998,"15min":0.40200000000000002},"min":{"1min":0.23799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":1.2170000000000001},"last":0.38}]},{"osd":4,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.73999999999999999,"5min":0.92400000000000004,"15min":1.254},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.68999999999999995},"min":{"1min":0.39300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68799999999999994,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.41199999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.36099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":0.97599999999999998,"15min":1.1799999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56899999999999995,"15min":0.60299999999999998},"min":{"1min":0.41799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71399999999999997,"5min":0.878,"15min":1.3109999999999999},"last":0.45900000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.81799999999999995},"min":{"1min":0.46000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78900000000000003,"5min":0.995,"15min":5.0659999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80700000000000005,"5min":0.89800000000000002,"15min":1.171},"last":0.54900000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57999999999999996,"15min":0.61299999999999999},"min":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77300000000000002,"5min":1.004,"15min":1.214},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.70999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78200000000000003,"5min":0.90500000000000003,"15min":4.0499999999999998},"last":0.48799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574946,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303816,"kb_used_data":2432,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068024,"statfs":{"total":96636764160,"available":96325656576,"internally_reserved":0,"allocated":2490368,"data_stored":1438415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.316,"15min":0.50800000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":7.9249999999999998},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.48699999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":6.6299999999999999},"last":0.46100000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.33100000000000002,"15min":0.53100000000000003},"min":{"1min":0.23100000000000001,"5min":0.22,"15min":0.216},"max":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":8.2509999999999994},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.51100000000000001},"min":{"1min":0.253,"5min":0.222,"15min":0.222},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":6.7370000000000001},"last":0.41599999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.35399999999999998,"15min":0.53900000000000003},"min":{"1min":0.247,"5min":0.23100000000000001,"15min":0.21099999999999999},"max":{"1min":0.53800000000000003,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.37},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.59099999999999997},"min":{"1min":0.27700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":7.8929999999999998},"last":0.38700000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57099999999999995,"15min":0.70299999999999996},"min":{"1min":0.45500000000000002,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.82499999999999996,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57299999999999995,"15min":0.754},"min":{"1min":0.441,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.80900000000000005,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.56200000000000006}]},{"osd":5,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56100000000000005,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71299999999999997,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.76100000000000001},"min":{"1min":0.48399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.71499999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61499999999999999,"15min":0.80000000000000004},"min":{"1min":0.52200000000000002,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.83299999999999996,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.62,"15min":0.749},"min":{"1min":0.47399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82599999999999996,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.80400000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.60099999999999998,"15min":0.73999999999999999},"min":{"1min":0.48399999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85399999999999998,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.76800000000000002},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.75700000000000001,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.69599999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574948,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":849008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.311,"15min":0.32500000000000001},"min":{"1min":0.214,"5min":0.189,"15min":0.189},"max":{"1min":0.52500000000000002,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.35099999999999998},"min":{"1min":0.219,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.746},"last":0.26100000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.69099999999999995},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52100000000000002,"5min":0.63,"15min":11.967000000000001},"last":0.248},{"interface":"front","average":{"1min":0.371,"5min":0.34899999999999998,"15min":0.46999999999999997},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48899999999999999,"5min":0.53400000000000003,"15min":4.1100000000000003},"last":0.33100000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34699999999999998,"15min":0.53700000000000003},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.50800000000000001,"5min":0.51600000000000001,"15min":6.3040000000000003},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.74299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59299999999999997,"5min":0.65900000000000003,"15min":11.935},"last":0.313}]},{"osd":4,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.55700000000000005,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.61599999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.38,"15min":0.378},"max":{"1min":0.83699999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54200000000000004,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.96099999999999997},"last":0.5}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.63500000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.58599999999999997},"min":{"1min":0.435,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.052},"last":0.627},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58099999999999996,"15min":0.76300000000000001},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":6.2649999999999997},"last":0.56799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574948,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":849008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.315,"15min":0.315},"min":{"1min":0.223,"5min":0.21199999999999999,"15min":0.21099999999999999},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32100000000000001},"min":{"1min":0.23999999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48799999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.34599999999999997,"15min":0.33300000000000002},"min":{"1min":0.25700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.379},"min":{"1min":0.29199999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.72099999999999997},"last":0.48599999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.36199999999999999,"15min":0.35299999999999998},"min":{"1min":0.25800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57599999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.28999999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.50900000000000001,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.38200000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.53400000000000003},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77200000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.745,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.56100000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57299999999999995,"15min":0.55200000000000005},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.35899999999999999},"max":{"1min":0.82399999999999995,"5min":1.347,"15min":1.347},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.42199999999999999,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.83499999999999996,"5min":1.01,"15min":1.01},"last":0.67100000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.48399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86299999999999999,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.81200000000000006,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.65100000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78700000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.83699999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.66200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574946,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":849008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.32400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66600000000000004,"5min":0.74199999999999999,"15min":0.747},"last":0.624},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51700000000000002,"15min":0.74099999999999999},"min":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77600000000000002,"5min":0.78400000000000003,"15min":7.6719999999999997},"last":0.58899999999999997}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52000000000000002,"15min":0.748},"min":{"1min":0.32000000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.755,"15min":7.7160000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66100000000000003,"5min":0.76900000000000002,"15min":0.86799999999999999},"last":0.55700000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.73899999999999999,"5min":0.79800000000000004,"15min":7.7839999999999998},"last":0.66200000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.39200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60999999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.49099999999999999},"min":{"1min":0.35199999999999998,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.63400000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.85199999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58399999999999996,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59899999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.41399999999999998,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.435},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574948,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":849008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.47599999999999998,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.66100000000000003,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.51800000000000002},"min":{"1min":0.33000000000000002,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.68899999999999995,"5min":0.76700000000000002,"15min":0.79300000000000004},"last":0.53200000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51300000000000001,"15min":0.628},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63400000000000001,"5min":0.75700000000000001,"15min":5.2679999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.62},"min":{"1min":0.40500000000000003,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79200000000000004,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.628}]},{"osd":2,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52400000000000002,"15min":0.63700000000000001},"min":{"1min":0.39100000000000001,"5min":0.32900000000000001,"15min":0.28299999999999997},"max":{"1min":0.73299999999999998,"5min":0.76600000000000001,"15min":5.2960000000000003},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.38500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.60399999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55900000000000005,"15min":0.66900000000000004},"min":{"1min":0.41999999999999998,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":5.3330000000000002},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56399999999999995,"15min":0.68000000000000005},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.27700000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.65900000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.48899999999999999,"15min":0.498},"min":{"1min":0.33400000000000002,"5min":0.29599999999999999,"15min":0.28699999999999998},"max":{"1min":0.67000000000000004,"5min":0.72499999999999998,"15min":0.82999999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.34499999999999997,"15min":0.32200000000000001},"max":{"1min":0.77200000000000002,"5min":0.88,"15min":1.0800000000000001},"last":0.83099999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.496},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.39900000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78900000000000003,"5min":0.82499999999999996,"15min":0.89800000000000002},"last":0.68600000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.32300000000000001,"15min":0.30099999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98399999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.76900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:42:28.168 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:42:28.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.313+0000 7fa1abc4f700 1 Processor -- start 2023-10-10T20:42:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.314+0000 7fa1abc4f700 1 -- start start 2023-10-10T20:42:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.314+0000 7fa1abc4f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa1a4068550 0x7fa1a4068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.314+0000 7fa1abc4f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa1a406f810 0x7fa1a4159510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.314+0000 7fa1abc4f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a406ec70 con 0x7fa1a415e010 2023-10-10T20:42:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.315+0000 7fa1abc4f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a40b3430 con 0x7fa1a4068550 2023-10-10T20:42:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.315+0000 7fa1abc4f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a406fc70 con 0x7fa1a406f810 2023-10-10T20:42:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.315+0000 7fa1a99eb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa1a4068550 0x7fa1a4068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.315+0000 7fa1a91ea700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa1a415e010 0x7fa1a406f100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54034/0 (socket says 172.21.15.89:54034) 2023-10-10T20:42:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.315+0000 7fa1a99eb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa1a4068550 0x7fa1a4068920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59098/0 (socket says 172.21.15.89:59098) 2023-10-10T20:42:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.315+0000 7fa1a91ea700 1 -- 172.21.15.89:0/321020160 learned_addr learned my addr 172.21.15.89:0/321020160 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.315+0000 7fa1aa1ec700 1 --2- 172.21.15.89:0/321020160 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa1a406f810 0x7fa1a4159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.315+0000 7fa1a99eb700 1 -- 172.21.15.89:0/321020160 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa1a406f810 msgr2=0x7fa1a4159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.315+0000 7fa1a99eb700 1 --2- 172.21.15.89:0/321020160 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa1a406f810 0x7fa1a4159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.315+0000 7fa1a99eb700 1 -- 172.21.15.89:0/321020160 >> v1:172.21.15.89:6789/0 conn(0x7fa1a415e010 legacy=0x7fa1a406f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.315+0000 7fa1a99eb700 1 -- 172.21.15.89:0/321020160 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a40b2e70 con 0x7fa1a4068550 2023-10-10T20:42:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.315+0000 7fa1a89e9700 1 -- 172.21.15.89:0/321020160 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837233864 0 0) 0x7fa1a406ec70 con 0x7fa1a415e010 2023-10-10T20:42:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.315+0000 7fa1aa1ec700 1 --2- 172.21.15.89:0/321020160 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa1a406f810 0x7fa1a4159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.315+0000 7fa1a99eb700 1 --2- 172.21.15.89:0/321020160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1a4068550 0x7fa1a4068920 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00098e0 tx=0x7fa1a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9feac3775c0ea1b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.315+0000 7fa1a89e9700 1 -- 172.21.15.89:0/321020160 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0009e80 con 0x7fa1a4068550 2023-10-10T20:42:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.315+0000 7fa1a89e9700 1 -- 172.21.15.89:0/321020160 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a000b860 con 0x7fa1a4068550 2023-10-10T20:42:28.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.315+0000 7fa1a89e9700 1 -- 172.21.15.89:0/321020160 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a000bbe0 con 0x7fa1a4068550 2023-10-10T20:42:28.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.316+0000 7fa1abc4f700 1 -- 172.21.15.89:0/321020160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1a4068550 msgr2=0x7fa1a4068920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:28.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.316+0000 7fa1abc4f700 1 --2- 172.21.15.89:0/321020160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1a4068550 0x7fa1a4068920 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00098e0 tx=0x7fa1a00091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.316+0000 7fa1abc4f700 1 -- 172.21.15.89:0/321020160 shutdown_connections 2023-10-10T20:42:28.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.316+0000 7fa1abc4f700 1 --2- 172.21.15.89:0/321020160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1a4068550 0x7fa1a4068920 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.316+0000 7fa1abc4f700 1 --2- 172.21.15.89:0/321020160 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa1a406f810 0x7fa1a4159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.316+0000 7fa1abc4f700 1 -- 172.21.15.89:0/321020160 >> 172.21.15.89:0/321020160 conn(0x7fa1a40cd480 msgr2=0x7fa1a40cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:28.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.316+0000 7fa1abc4f700 1 -- 172.21.15.89:0/321020160 shutdown_connections 2023-10-10T20:42:28.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.316+0000 7fa1abc4f700 1 -- 172.21.15.89:0/321020160 wait complete. 2023-10-10T20:42:28.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.317+0000 7fa1abc4f700 1 Processor -- start 2023-10-10T20:42:28.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.317+0000 7fa1abc4f700 1 -- start start 2023-10-10T20:42:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.318+0000 7fa1abc4f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1a4068550 0x7fa1a414d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.318+0000 7fa1abc4f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1a406f810 0x7fa1a40af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.318+0000 7fa1abc4f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa1a415e010 0x7fa1a4149200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.318+0000 7fa1abc4f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40ad590 con 0x7fa1a4068550 2023-10-10T20:42:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.318+0000 7fa1abc4f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40afb50 con 0x7fa1a406f810 2023-10-10T20:42:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.318+0000 7fa1abc4f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40afe30 con 0x7fa1a415e010 2023-10-10T20:42:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.318+0000 7fa1a99eb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1a4068550 0x7fa1a414d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.318+0000 7fa1a99eb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1a4068550 0x7fa1a414d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59114/0 (socket says 172.21.15.89:59114) 2023-10-10T20:42:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.318+0000 7fa1a99eb700 1 -- 172.21.15.89:0/3446779824 learned_addr learned my addr 172.21.15.89:0/3446779824 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:28.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.318+0000 7fa1a91ea700 1 --2- 172.21.15.89:0/3446779824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1a406f810 0x7fa1a40af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:28.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.318+0000 7fa1a99eb700 1 -- 172.21.15.89:0/3446779824 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa1a415e010 msgr2=0x7fa1a4149200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:28.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.318+0000 7fa1a99eb700 1 --2- 172.21.15.89:0/3446779824 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa1a415e010 0x7fa1a4149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.318+0000 7fa1a99eb700 1 -- 172.21.15.89:0/3446779824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1a406f810 msgr2=0x7fa1a40af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:28.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.318+0000 7fa1a99eb700 1 --2- 172.21.15.89:0/3446779824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1a406f810 0x7fa1a40af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.318+0000 7fa1a99eb700 1 -- 172.21.15.89:0/3446779824 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a0009420 con 0x7fa1a4068550 2023-10-10T20:42:28.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.318+0000 7fa1a99eb700 1 --2- 172.21.15.89:0/3446779824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1a4068550 0x7fa1a414d8b0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40ce220 tx=0x7fa1a0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:28.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.318+0000 7fa199ffb700 1 -- 172.21.15.89:0/3446779824 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a000bbe0 con 0x7fa1a4068550 2023-10-10T20:42:28.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.318+0000 7fa199ffb700 1 -- 172.21.15.89:0/3446779824 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a0011420 con 0x7fa1a4068550 2023-10-10T20:42:28.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.319+0000 7fa1abc4f700 1 -- 172.21.15.89:0/3446779824 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a40acf40 con 0x7fa1a4068550 2023-10-10T20:42:28.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.319+0000 7fa199ffb700 1 -- 172.21.15.89:0/3446779824 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a00117e0 con 0x7fa1a4068550 2023-10-10T20:42:28.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.319+0000 7fa1abc4f700 1 -- 172.21.15.89:0/3446779824 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a40af800 con 0x7fa1a4068550 2023-10-10T20:42:28.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.320+0000 7fa1abc4f700 1 -- 172.21.15.89:0/3446779824 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a404c3c0 con 0x7fa1a4068550 2023-10-10T20:42:28.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.320+0000 7fa199ffb700 1 -- 172.21.15.89:0/3446779824 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa1a0011bd0 con 0x7fa1a4068550 2023-10-10T20:42:28.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.320+0000 7fa199ffb700 1 --2- 172.21.15.89:0/3446779824 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa188067320 0x7fa1880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:28.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.320+0000 7fa199ffb700 1 -- 172.21.15.89:0/3446779824 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa1a0086700 con 0x7fa1a4068550 2023-10-10T20:42:28.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.320+0000 7fa1a91ea700 1 --2- 172.21.15.89:0/3446779824 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa188067320 0x7fa1880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:28.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.321+0000 7fa1a91ea700 1 --2- 172.21.15.89:0/3446779824 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa188067320 0x7fa1880697d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa1a416d6e0 tx=0x7fa194009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:28.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.324+0000 7fa199ffb700 1 -- 172.21.15.89:0/3446779824 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1a0053410 con 0x7fa1a4068550 2023-10-10T20:42:28.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.505+0000 7fa1abc4f700 1 -- 172.21.15.89:0/3446779824 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa1a40ad6d0 con 0x7fa188067320 2023-10-10T20:42:28.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.513+0000 7fa199ffb700 1 -- 172.21.15.89:0/3446779824 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279133 (secure 0 0 0) 0x7fa1940058e0 con 0x7fa188067320 2023-10-10T20:42:28.518 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:28.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.516+0000 7fa1abc4f700 1 -- 172.21.15.89:0/3446779824 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa188067320 msgr2=0x7fa1880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:28.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.516+0000 7fa1abc4f700 1 --2- 172.21.15.89:0/3446779824 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa188067320 0x7fa1880697d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa1a416d6e0 tx=0x7fa194009040 comp rx=0 tx=0).stop 2023-10-10T20:42:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.516+0000 7fa1abc4f700 1 -- 172.21.15.89:0/3446779824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1a4068550 msgr2=0x7fa1a414d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.516+0000 7fa1abc4f700 1 --2- 172.21.15.89:0/3446779824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1a4068550 0x7fa1a414d8b0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40ce220 tx=0x7fa1a0017000 comp rx=0 tx=0).stop 2023-10-10T20:42:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.516+0000 7fa1abc4f700 1 -- 172.21.15.89:0/3446779824 shutdown_connections 2023-10-10T20:42:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.516+0000 7fa1abc4f700 1 --2- 172.21.15.89:0/3446779824 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa1a415e010 0x7fa1a4149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.516+0000 7fa1abc4f700 1 --2- 172.21.15.89:0/3446779824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1a4068550 0x7fa1a414d8b0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.516+0000 7fa1abc4f700 1 --2- 172.21.15.89:0/3446779824 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa188067320 0x7fa1880697d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.516+0000 7fa1abc4f700 1 --2- 172.21.15.89:0/3446779824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1a406f810 0x7fa1a40af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.516+0000 7fa1abc4f700 1 -- 172.21.15.89:0/3446779824 >> 172.21.15.89:0/3446779824 conn(0x7fa1a40cd480 msgr2=0x7fa1a4158940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.516+0000 7fa1abc4f700 1 -- 172.21.15.89:0/3446779824 shutdown_connections 2023-10-10T20:42:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.516+0000 7fa1abc4f700 1 -- 172.21.15.89:0/3446779824 wait complete. 2023-10-10T20:42:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:42:28.530 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2023-10-10T20:42:26.745422+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427076,"kb_used_data":16004,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547644,"statfs":{"total":773094113280,"available":770608787456,"internally_reserved":0,"allocated":16388096,"data_stored":7971839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.658883"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.095350+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:06:20.346560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.091710+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:29:30.897091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.098436+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:40:22.937275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.092780+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:36:42.204376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.095195+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:59.544805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.093851+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:17.799833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.091829+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:06:20.346560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.095866+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:37:08.365854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.091984+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:17.799833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.094377+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:52:50.371745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.091289+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:21:58.481035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.093229+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:48:40.304153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.091790+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:56.914227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.093591+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:41:54.647934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.098056+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:31:23.373914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.094346+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:04.056737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.091172+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:32:29.013486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.093986+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:20:33.756714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.099072+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:08:05.465989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.094095+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:46:26.675244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.095136+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:33.642377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.095221+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:39:58.918529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.095508+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:00:27.632584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.098081+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:35:08.745975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.091540+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:14:17.624964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.093876+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:48:40.304153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.098788+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:51:56.794296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.091222+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:46:14.740269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.093991+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:42.951721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.091754+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:43:43.081600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.094296+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:56.914227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:42:23.086777+0000","last_change":"2023-10-10T20:42:23.086777+0000","last_active":"2023-10-10T20:42:23.086777+0000","last_peered":"2023-10-10T20:42:23.086777+0000","last_clean":"2023-10-10T20:42:23.086777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:23.086777+0000","last_undegraded":"2023-10-10T20:42:23.086777+0000","last_fullsized":"2023-10-10T20:42:23.086777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:22.076043+0000","last_change":"2023-10-10T20:42:22.095382+0000","last_active":"2023-10-10T20:42:22.076043+0000","last_peered":"2023-10-10T20:42:22.076043+0000","last_clean":"2023-10-10T20:42:22.076043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:22.076043+0000","last_undegraded":"2023-10-10T20:42:22.076043+0000","last_fullsized":"2023-10-10T20:42:22.076043+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:17.799833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":293,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-10T20:42:20.075255+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:42:20.075255+0000","last_peered":"2023-10-10T20:42:20.075255+0000","last_clean":"2023-10-10T20:42:20.075255+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:42:20.075255+0000","last_undegraded":"2023-10-10T20:42:20.075255+0000","last_fullsized":"2023-10-10T20:42:20.075255+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574947,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303820,"kb_used_data":2436,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068020,"statfs":{"total":96636764160,"available":96325652480,"internally_reserved":0,"allocated":2494464,"data_stored":1439376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.79000000000000004},"min":{"1min":0.309,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71799999999999997,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52400000000000002,"15min":0.61599999999999999},"min":{"1min":0.32300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.69999999999999996,"5min":0.82099999999999995,"15min":4.1959999999999997},"last":0.53600000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.95099999999999996},"min":{"1min":0.379,"5min":0.317,"15min":0.317},"max":{"1min":0.78300000000000003,"5min":0.78800000000000003,"15min":10.1},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55200000000000005,"15min":0.71399999999999997},"min":{"1min":0.36699999999999999,"5min":0.32300000000000001,"15min":0.28999999999999998},"max":{"1min":0.73199999999999998,"5min":0.90400000000000003,"15min":6.806},"last":0.501}]},{"osd":2,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.76400000000000001},"min":{"1min":0.40100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79200000000000004,"5min":0.89000000000000001,"15min":6.9340000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.71699999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.73299999999999998,"5min":0.83899999999999997,"15min":6.9870000000000001},"last":0.47999999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54900000000000004,"15min":0.73599999999999999},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.753,"5min":0.80200000000000005,"15min":6.6719999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56699999999999995,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.71699999999999997,"5min":0.87,"15min":4.1609999999999996},"last":0.46500000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.36599999999999999,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.65100000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36299999999999999,"5min":0.309,"15min":0.30399999999999999},"max":{"1min":0.72299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.314}]},{"osd":5,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.314},"max":{"1min":0.77900000000000003,"5min":0.86099999999999999,"15min":1.3180000000000001},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.41799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71499999999999997,"5min":0.75800000000000001,"15min":0.84099999999999997},"last":0.36199999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":0.78700000000000003,"15min":1.7509999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.40899999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.85699999999999998,"15min":1.7090000000000001},"last":0.374}]}]},{"osd":6,"up_from":13,"seq":55834574948,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":849008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51300000000000001,"15min":0.65500000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":7.7590000000000003},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.52100000000000002,"15min":0.69199999999999995},"min":{"1min":0.32200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.075,"5min":1.075,"15min":7.8090000000000002},"last":0.436}]},{"osd":1,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.69099999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.70199999999999996,"5min":0.80000000000000004,"15min":7.867},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.85599999999999998},"min":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.63200000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.58799999999999997},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.56100000000000005,"15min":0.81999999999999995},"min":{"1min":0.34200000000000003,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":2.641,"5min":2.641,"15min":6.7629999999999999},"last":0.81299999999999994}]},{"osd":3,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54300000000000004,"15min":0.61099999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.107,"5min":1.107,"15min":4.8620000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.70099999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.718,"5min":1.718,"15min":7.9009999999999998},"last":0.67000000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.47299999999999998},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.82899999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52700000000000002,"15min":0.495},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.53400000000000003},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.85399999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55800000000000005,"15min":0.52500000000000002},"min":{"1min":0.36099999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.372,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.78900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":849008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.312,"15min":0.45900000000000002},"min":{"1min":0.22800000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.437,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":1.194},"last":0.318}]},{"osd":2,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.24199999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51000000000000001,"5min":0.51400000000000001,"15min":1.2869999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.48999999999999999},"min":{"1min":0.27200000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.52800000000000002,"5min":0.54200000000000004,"15min":5.0919999999999996},"last":0.33500000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36399999999999999,"15min":0.51200000000000001},"min":{"1min":0.29399999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35699999999999998,"15min":0.40200000000000002},"min":{"1min":0.23799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":1.2170000000000001},"last":0.38}]},{"osd":4,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.73999999999999999,"5min":0.92400000000000004,"15min":1.254},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.68999999999999995},"min":{"1min":0.39300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68799999999999994,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.41199999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.36099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":0.97599999999999998,"15min":1.1799999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56899999999999995,"15min":0.60299999999999998},"min":{"1min":0.41799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71399999999999997,"5min":0.878,"15min":1.3109999999999999},"last":0.45900000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.81799999999999995},"min":{"1min":0.46000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78900000000000003,"5min":0.995,"15min":5.0659999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80700000000000005,"5min":0.89800000000000002,"15min":1.171},"last":0.54900000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57999999999999996,"15min":0.61299999999999999},"min":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77300000000000002,"5min":1.004,"15min":1.214},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.70999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78200000000000003,"5min":0.90500000000000003,"15min":4.0499999999999998},"last":0.48799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574946,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303816,"kb_used_data":2432,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068024,"statfs":{"total":96636764160,"available":96325656576,"internally_reserved":0,"allocated":2490368,"data_stored":1438415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.316,"15min":0.50800000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":7.9249999999999998},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.48699999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":6.6299999999999999},"last":0.46100000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.33100000000000002,"15min":0.53100000000000003},"min":{"1min":0.23100000000000001,"5min":0.22,"15min":0.216},"max":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":8.2509999999999994},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.51100000000000001},"min":{"1min":0.253,"5min":0.222,"15min":0.222},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":6.7370000000000001},"last":0.41599999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.35399999999999998,"15min":0.53900000000000003},"min":{"1min":0.247,"5min":0.23100000000000001,"15min":0.21099999999999999},"max":{"1min":0.53800000000000003,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.37},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.59099999999999997},"min":{"1min":0.27700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":7.8929999999999998},"last":0.38700000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57099999999999995,"15min":0.70299999999999996},"min":{"1min":0.45500000000000002,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.82499999999999996,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57299999999999995,"15min":0.754},"min":{"1min":0.441,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.80900000000000005,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.56200000000000006}]},{"osd":5,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56100000000000005,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71299999999999997,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.76100000000000001},"min":{"1min":0.48399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.71499999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61499999999999999,"15min":0.80000000000000004},"min":{"1min":0.52200000000000002,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.83299999999999996,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.62,"15min":0.749},"min":{"1min":0.47399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82599999999999996,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.80400000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.60099999999999998,"15min":0.73999999999999999},"min":{"1min":0.48399999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85399999999999998,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.76800000000000002},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.75700000000000001,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.69599999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574948,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":849008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.311,"15min":0.32500000000000001},"min":{"1min":0.214,"5min":0.189,"15min":0.189},"max":{"1min":0.52500000000000002,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.35099999999999998},"min":{"1min":0.219,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.746},"last":0.26100000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.69099999999999995},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52100000000000002,"5min":0.63,"15min":11.967000000000001},"last":0.248},{"interface":"front","average":{"1min":0.371,"5min":0.34899999999999998,"15min":0.46999999999999997},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48899999999999999,"5min":0.53400000000000003,"15min":4.1100000000000003},"last":0.33100000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34699999999999998,"15min":0.53700000000000003},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.50800000000000001,"5min":0.51600000000000001,"15min":6.3040000000000003},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.74299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59299999999999997,"5min":0.65900000000000003,"15min":11.935},"last":0.313}]},{"osd":4,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.55700000000000005,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.61599999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.38,"15min":0.378},"max":{"1min":0.83699999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54200000000000004,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.96099999999999997},"last":0.5}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.63500000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.58599999999999997},"min":{"1min":0.435,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.052},"last":0.627},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58099999999999996,"15min":0.76300000000000001},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":6.2649999999999997},"last":0.56799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574948,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":849008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.315,"15min":0.315},"min":{"1min":0.223,"5min":0.21199999999999999,"15min":0.21099999999999999},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32100000000000001},"min":{"1min":0.23999999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48799999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.34599999999999997,"15min":0.33300000000000002},"min":{"1min":0.25700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.379},"min":{"1min":0.29199999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.72099999999999997},"last":0.48599999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.36199999999999999,"15min":0.35299999999999998},"min":{"1min":0.25800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57599999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.28999999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.50900000000000001,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.38200000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.53400000000000003},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77200000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.745,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.56100000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57299999999999995,"15min":0.55200000000000005},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.35899999999999999},"max":{"1min":0.82399999999999995,"5min":1.347,"15min":1.347},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.42199999999999999,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.83499999999999996,"5min":1.01,"15min":1.01},"last":0.67100000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.48399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86299999999999999,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.81200000000000006,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.65100000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78700000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.83699999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.66200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574946,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":849008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.32400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66600000000000004,"5min":0.74199999999999999,"15min":0.747},"last":0.624},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51700000000000002,"15min":0.74099999999999999},"min":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77600000000000002,"5min":0.78400000000000003,"15min":7.6719999999999997},"last":0.58899999999999997}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52000000000000002,"15min":0.748},"min":{"1min":0.32000000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.755,"15min":7.7160000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66100000000000003,"5min":0.76900000000000002,"15min":0.86799999999999999},"last":0.55700000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.73899999999999999,"5min":0.79800000000000004,"15min":7.7839999999999998},"last":0.66200000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.39200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60999999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.49099999999999999},"min":{"1min":0.35199999999999998,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.63400000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.85199999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58399999999999996,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59899999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.41399999999999998,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.435},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574948,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":849008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.47599999999999998,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.66100000000000003,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.51800000000000002},"min":{"1min":0.33000000000000002,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.68899999999999995,"5min":0.76700000000000002,"15min":0.79300000000000004},"last":0.53200000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51300000000000001,"15min":0.628},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63400000000000001,"5min":0.75700000000000001,"15min":5.2679999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.62},"min":{"1min":0.40500000000000003,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79200000000000004,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.628}]},{"osd":2,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52400000000000002,"15min":0.63700000000000001},"min":{"1min":0.39100000000000001,"5min":0.32900000000000001,"15min":0.28299999999999997},"max":{"1min":0.73299999999999998,"5min":0.76600000000000001,"15min":5.2960000000000003},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.38500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.60399999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55900000000000005,"15min":0.66900000000000004},"min":{"1min":0.41999999999999998,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":5.3330000000000002},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56399999999999995,"15min":0.68000000000000005},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.27700000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.65900000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.48899999999999999,"15min":0.498},"min":{"1min":0.33400000000000002,"5min":0.29599999999999999,"15min":0.28699999999999998},"max":{"1min":0.67000000000000004,"5min":0.72499999999999998,"15min":0.82999999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.34499999999999997,"15min":0.32200000000000001},"max":{"1min":0.77200000000000002,"5min":0.88,"15min":1.0800000000000001},"last":0.83099999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.496},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.39900000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78900000000000003,"5min":0.82499999999999996,"15min":0.89800000000000002},"last":0.68600000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.32300000000000001,"15min":0.30099999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98399999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.76900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:42:28.536 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:42:28.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.683+0000 7f22d493e700 1 Processor -- start 2023-10-10T20:42:28.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.683+0000 7f22d493e700 1 -- start start 2023-10-10T20:42:28.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.684+0000 7f22d493e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f22d0159c60 0x7f22d015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:28.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.684+0000 7f22d493e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d015a160 con 0x7f22d0159890 2023-10-10T20:42:28.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.684+0000 7f22d493e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d00d13a0 con 0x7f22d015db10 2023-10-10T20:42:28.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.684+0000 7f22d493e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d015df50 con 0x7f22d0159c60 2023-10-10T20:42:28.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.684+0000 7f22ce59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f22d0159890 0x7f22d015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54048/0 (socket says 172.21.15.89:54048) 2023-10-10T20:42:28.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.684+0000 7f22c5d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f22d0159c60 0x7f22d015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:28.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.684+0000 7f22ce59c700 1 -- 172.21.15.89:0/27777808 learned_addr learned my addr 172.21.15.89:0/27777808 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:28.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.685+0000 7f22c5d9b700 1 -- 172.21.15.89:0/27777808 >> v1:172.21.15.111:6789/0 conn(0x7f22d015db10 legacy=0x7f22d0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:28.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.685+0000 7f22c5d9b700 1 -- 172.21.15.89:0/27777808 >> v1:172.21.15.89:6789/0 conn(0x7f22d0159890 legacy=0x7f22d015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:28.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.685+0000 7f22c5d9b700 1 -- 172.21.15.89:0/27777808 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d015e140 con 0x7f22d0159c60 2023-10-10T20:42:28.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.685+0000 7f22c5d9b700 1 --2- 172.21.15.89:0/27777808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d0159c60 0x7f22d015d5d0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f22b80098e0 tx=0x7f22b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702f660baeed1d24 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:28.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.685+0000 7f22cdd9b700 1 -- 172.21.15.89:0/27777808 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b8009e80 con 0x7f22d0159c60 2023-10-10T20:42:28.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.685+0000 7f22cdd9b700 1 -- 172.21.15.89:0/27777808 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b800b820 con 0x7f22d0159c60 2023-10-10T20:42:28.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.685+0000 7f22cdd9b700 1 -- 172.21.15.89:0/27777808 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b800bba0 con 0x7f22d0159c60 2023-10-10T20:42:28.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.686+0000 7f22d493e700 1 -- 172.21.15.89:0/27777808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d0159c60 msgr2=0x7f22d015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:28.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.686+0000 7f22d493e700 1 --2- 172.21.15.89:0/27777808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d0159c60 0x7f22d015d5d0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f22b80098e0 tx=0x7f22b80091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.686+0000 7f22d493e700 1 -- 172.21.15.89:0/27777808 shutdown_connections 2023-10-10T20:42:28.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.686+0000 7f22d493e700 1 --2- 172.21.15.89:0/27777808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d0159c60 0x7f22d015d5d0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.686+0000 7f22d493e700 1 -- 172.21.15.89:0/27777808 >> 172.21.15.89:0/27777808 conn(0x7f22d0068e20 msgr2=0x7f22d00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:28.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.686+0000 7f22d493e700 1 -- 172.21.15.89:0/27777808 shutdown_connections 2023-10-10T20:42:28.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.686+0000 7f22d493e700 1 -- 172.21.15.89:0/27777808 wait complete. 2023-10-10T20:42:28.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.687+0000 7f22d493e700 1 Processor -- start 2023-10-10T20:42:28.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.687+0000 7f22d493e700 1 -- start start 2023-10-10T20:42:28.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22d493e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f22d0159890 0x7f22d00ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:28.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22d493e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d0159c60 0x7f22d01526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:28.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22d493e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d015db10 0x7f22d00b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:28.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22d493e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d00ce0f0 con 0x7f22d015db10 2023-10-10T20:42:28.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22d493e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d00ce6b0 con 0x7f22d0159c60 2023-10-10T20:42:28.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22d493e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d00b5fb0 con 0x7f22d0159890 2023-10-10T20:42:28.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22ced9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d015db10 0x7f22d00b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:28.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22ced9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d015db10 0x7f22d00b5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59126/0 (socket says 172.21.15.89:59126) 2023-10-10T20:42:28.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22ced9d700 1 -- 172.21.15.89:0/1639096690 learned_addr learned my addr 172.21.15.89:0/1639096690 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:28.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22ce59c700 1 --2- 172.21.15.89:0/1639096690 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f22d0159890 0x7f22d00ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:28.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22c5d9b700 1 --2- 172.21.15.89:0/1639096690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d0159c60 0x7f22d01526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:28.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22ced9d700 1 -- 172.21.15.89:0/1639096690 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f22d0159890 msgr2=0x7f22d00ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:28.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22ced9d700 1 --2- 172.21.15.89:0/1639096690 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f22d0159890 0x7f22d00ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22ced9d700 1 -- 172.21.15.89:0/1639096690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d0159c60 msgr2=0x7f22d01526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:28.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22ced9d700 1 --2- 172.21.15.89:0/1639096690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d0159c60 0x7f22d01526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22ced9d700 1 -- 172.21.15.89:0/1639096690 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b8009420 con 0x7f22d015db10 2023-10-10T20:42:28.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22c5d9b700 1 --2- 172.21.15.89:0/1639096690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d0159c60 0x7f22d01526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:28.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22ced9d700 1 --2- 172.21.15.89:0/1639096690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d015db10 0x7f22d00b5660 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f22c800c930 tx=0x7f22c80026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:28.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22c6ffd700 1 -- 172.21.15.89:0/1639096690 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8012630 con 0x7f22d015db10 2023-10-10T20:42:28.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22c6ffd700 1 -- 172.21.15.89:0/1639096690 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c80127b0 con 0x7f22d015db10 2023-10-10T20:42:28.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22d493e700 1 -- 172.21.15.89:0/1639096690 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d00b6290 con 0x7f22d015db10 2023-10-10T20:42:28.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22c6ffd700 1 -- 172.21.15.89:0/1639096690 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8012b70 con 0x7f22d015db10 2023-10-10T20:42:28.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.688+0000 7f22d493e700 1 -- 172.21.15.89:0/1639096690 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d00cf080 con 0x7f22d015db10 2023-10-10T20:42:28.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.689+0000 7f22d493e700 1 -- 172.21.15.89:0/1639096690 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d00ce820 con 0x7f22d015db10 2023-10-10T20:42:28.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.690+0000 7f22c6ffd700 1 -- 172.21.15.89:0/1639096690 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f22c800a3b0 con 0x7f22d015db10 2023-10-10T20:42:28.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.690+0000 7f22c6ffd700 1 --2- 172.21.15.89:0/1639096690 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f22b0067320 0x7f22b00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:28.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.690+0000 7f22c6ffd700 1 -- 172.21.15.89:0/1639096690 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f22c80848a0 con 0x7f22d015db10 2023-10-10T20:42:28.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.690+0000 7f22ce59c700 1 --2- 172.21.15.89:0/1639096690 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f22b0067320 0x7f22b00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:28.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.691+0000 7f22ce59c700 1 --2- 172.21.15.89:0/1639096690 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f22b0067320 0x7f22b00697d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f22c0007130 tx=0x7f22c0000bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:28.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.694+0000 7f22c6ffd700 1 -- 172.21.15.89:0/1639096690 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22c8051530 con 0x7f22d015db10 2023-10-10T20:42:28.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.873+0000 7f22d493e700 1 -- 172.21.15.89:0/1639096690 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22d004c3c0 con 0x7f22b0067320 2023-10-10T20:42:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.881+0000 7f22c6ffd700 1 -- 172.21.15.89:0/1639096690 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290160 (secure 0 0 0) 0x7f22d004c3c0 con 0x7f22b0067320 2023-10-10T20:42:28.886 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:28.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.884+0000 7f22d493e700 1 -- 172.21.15.89:0/1639096690 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f22b0067320 msgr2=0x7f22b00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:28.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.884+0000 7f22d493e700 1 --2- 172.21.15.89:0/1639096690 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f22b0067320 0x7f22b00697d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f22c0007130 tx=0x7f22c0000bc0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.884+0000 7f22d493e700 1 -- 172.21.15.89:0/1639096690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d015db10 msgr2=0x7f22d00b5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:28.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.884+0000 7f22d493e700 1 --2- 172.21.15.89:0/1639096690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d015db10 0x7f22d00b5660 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f22c800c930 tx=0x7f22c80026e0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.884+0000 7f22d493e700 1 -- 172.21.15.89:0/1639096690 shutdown_connections 2023-10-10T20:42:28.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.884+0000 7f22d493e700 1 --2- 172.21.15.89:0/1639096690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d015db10 0x7f22d00b5660 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.884+0000 7f22d493e700 1 --2- 172.21.15.89:0/1639096690 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f22d0159890 0x7f22d00ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.884+0000 7f22d493e700 1 --2- 172.21.15.89:0/1639096690 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f22b0067320 0x7f22b00697d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.884+0000 7f22d493e700 1 --2- 172.21.15.89:0/1639096690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d0159c60 0x7f22d01526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:28.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.884+0000 7f22d493e700 1 -- 172.21.15.89:0/1639096690 >> 172.21.15.89:0/1639096690 conn(0x7f22d0068e20 msgr2=0x7f22d015aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:28.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.884+0000 7f22d493e700 1 -- 172.21.15.89:0/1639096690 shutdown_connections 2023-10-10T20:42:28.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:28.884+0000 7f22d493e700 1 -- 172.21.15.89:0/1639096690 wait complete. 2023-10-10T20:42:28.891 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:42:28.898 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2023-10-10T20:42:28.745808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427428,"kb_used_data":16356,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547292,"statfs":{"total":773094113280,"available":770608427008,"internally_reserved":0,"allocated":16748544,"data_stored":8130963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001060"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121111+0000","last_change":"2023-10-10T20:42:24.113315+0000","last_active":"2023-10-10T20:42:24.121111+0000","last_peered":"2023-10-10T20:42:24.121111+0000","last_clean":"2023-10-10T20:42:24.121111+0000","last_became_active":"2023-10-10T20:42:24.113091+0000","last_became_peered":"2023-10-10T20:42:24.113091+0000","last_unstale":"2023-10-10T20:42:24.121111+0000","last_undegraded":"2023-10-10T20:42:24.121111+0000","last_fullsized":"2023-10-10T20:42:24.121111+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:25:55.562871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114620+0000","last_change":"2023-10-10T20:42:24.111977+0000","last_active":"2023-10-10T20:42:24.114620+0000","last_peered":"2023-10-10T20:42:24.114620+0000","last_clean":"2023-10-10T20:42:24.114620+0000","last_became_active":"2023-10-10T20:42:24.111740+0000","last_became_peered":"2023-10-10T20:42:24.111740+0000","last_unstale":"2023-10-10T20:42:24.114620+0000","last_undegraded":"2023-10-10T20:42:24.114620+0000","last_fullsized":"2023-10-10T20:42:24.114620+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:43:44.092334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.107328+0000","last_change":"2023-10-10T20:42:24.101396+0000","last_active":"2023-10-10T20:42:24.107328+0000","last_peered":"2023-10-10T20:42:24.107328+0000","last_clean":"2023-10-10T20:42:24.107328+0000","last_became_active":"2023-10-10T20:42:24.101140+0000","last_became_peered":"2023-10-10T20:42:24.101140+0000","last_unstale":"2023-10-10T20:42:24.107328+0000","last_undegraded":"2023-10-10T20:42:24.107328+0000","last_fullsized":"2023-10-10T20:42:24.107328+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:22:58.559364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113453+0000","last_change":"2023-10-10T20:42:24.107229+0000","last_active":"2023-10-10T20:42:24.113453+0000","last_peered":"2023-10-10T20:42:24.113453+0000","last_clean":"2023-10-10T20:42:24.113453+0000","last_became_active":"2023-10-10T20:42:24.106160+0000","last_became_peered":"2023-10-10T20:42:24.106160+0000","last_unstale":"2023-10-10T20:42:24.113453+0000","last_undegraded":"2023-10-10T20:42:24.113453+0000","last_fullsized":"2023-10-10T20:42:24.113453+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:04:22.281292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114347+0000","last_change":"2023-10-10T20:42:24.112159+0000","last_active":"2023-10-10T20:42:24.114347+0000","last_peered":"2023-10-10T20:42:24.114347+0000","last_clean":"2023-10-10T20:42:24.114347+0000","last_became_active":"2023-10-10T20:42:24.111823+0000","last_became_peered":"2023-10-10T20:42:24.111823+0000","last_unstale":"2023-10-10T20:42:24.114347+0000","last_undegraded":"2023-10-10T20:42:24.114347+0000","last_fullsized":"2023-10-10T20:42:24.114347+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:37:14.508787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.117428+0000","last_change":"2023-10-10T20:42:24.114360+0000","last_active":"2023-10-10T20:42:24.117428+0000","last_peered":"2023-10-10T20:42:24.117428+0000","last_clean":"2023-10-10T20:42:24.117428+0000","last_became_active":"2023-10-10T20:42:24.114175+0000","last_became_peered":"2023-10-10T20:42:24.114175+0000","last_unstale":"2023-10-10T20:42:24.117428+0000","last_undegraded":"2023-10-10T20:42:24.117428+0000","last_fullsized":"2023-10-10T20:42:24.117428+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:39:43.171315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121416+0000","last_change":"2023-10-10T20:42:24.116618+0000","last_active":"2023-10-10T20:42:24.121416+0000","last_peered":"2023-10-10T20:42:24.121416+0000","last_clean":"2023-10-10T20:42:24.121416+0000","last_became_active":"2023-10-10T20:42:24.115742+0000","last_became_peered":"2023-10-10T20:42:24.115742+0000","last_unstale":"2023-10-10T20:42:24.121416+0000","last_undegraded":"2023-10-10T20:42:24.121416+0000","last_fullsized":"2023-10-10T20:42:24.121416+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:49:38.681149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114418+0000","last_change":"2023-10-10T20:42:24.112723+0000","last_active":"2023-10-10T20:42:24.114418+0000","last_peered":"2023-10-10T20:42:24.114418+0000","last_clean":"2023-10-10T20:42:24.114418+0000","last_became_active":"2023-10-10T20:42:24.112506+0000","last_became_peered":"2023-10-10T20:42:24.112506+0000","last_unstale":"2023-10-10T20:42:24.114418+0000","last_undegraded":"2023-10-10T20:42:24.114418+0000","last_fullsized":"2023-10-10T20:42:24.114418+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:46:59.225281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.107890+0000","last_change":"2023-10-10T20:42:24.101774+0000","last_active":"2023-10-10T20:42:24.107890+0000","last_peered":"2023-10-10T20:42:24.107890+0000","last_clean":"2023-10-10T20:42:24.107890+0000","last_became_active":"2023-10-10T20:42:24.101500+0000","last_became_peered":"2023-10-10T20:42:24.101500+0000","last_unstale":"2023-10-10T20:42:24.107890+0000","last_undegraded":"2023-10-10T20:42:24.107890+0000","last_fullsized":"2023-10-10T20:42:24.107890+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:32:22.661032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121119+0000","last_change":"2023-10-10T20:42:24.117698+0000","last_active":"2023-10-10T20:42:24.121119+0000","last_peered":"2023-10-10T20:42:24.121119+0000","last_clean":"2023-10-10T20:42:24.121119+0000","last_became_active":"2023-10-10T20:42:24.117453+0000","last_became_peered":"2023-10-10T20:42:24.117453+0000","last_unstale":"2023-10-10T20:42:24.121119+0000","last_undegraded":"2023-10-10T20:42:24.121119+0000","last_fullsized":"2023-10-10T20:42:24.121119+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:14.819673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.112709+0000","last_change":"2023-10-10T20:42:24.108830+0000","last_active":"2023-10-10T20:42:24.112709+0000","last_peered":"2023-10-10T20:42:24.112709+0000","last_clean":"2023-10-10T20:42:24.112709+0000","last_became_active":"2023-10-10T20:42:24.108592+0000","last_became_peered":"2023-10-10T20:42:24.108592+0000","last_unstale":"2023-10-10T20:42:24.112709+0000","last_undegraded":"2023-10-10T20:42:24.112709+0000","last_fullsized":"2023-10-10T20:42:24.112709+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:32:50.550864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.128843+0000","last_change":"2023-10-10T20:42:24.124278+0000","last_active":"2023-10-10T20:42:24.128843+0000","last_peered":"2023-10-10T20:42:24.128843+0000","last_clean":"2023-10-10T20:42:24.128843+0000","last_became_active":"2023-10-10T20:42:24.123491+0000","last_became_peered":"2023-10-10T20:42:24.123491+0000","last_unstale":"2023-10-10T20:42:24.128843+0000","last_undegraded":"2023-10-10T20:42:24.128843+0000","last_fullsized":"2023-10-10T20:42:24.128843+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:47:26.708886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.128950+0000","last_change":"2023-10-10T20:42:24.124820+0000","last_active":"2023-10-10T20:42:24.128950+0000","last_peered":"2023-10-10T20:42:24.128950+0000","last_clean":"2023-10-10T20:42:24.128950+0000","last_became_active":"2023-10-10T20:42:24.124570+0000","last_became_peered":"2023-10-10T20:42:24.124570+0000","last_unstale":"2023-10-10T20:42:24.128950+0000","last_undegraded":"2023-10-10T20:42:24.128950+0000","last_fullsized":"2023-10-10T20:42:24.128950+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:21:59.491769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.112675+0000","last_change":"2023-10-10T20:42:24.110144+0000","last_active":"2023-10-10T20:42:24.112675+0000","last_peered":"2023-10-10T20:42:24.112675+0000","last_clean":"2023-10-10T20:42:24.112675+0000","last_became_active":"2023-10-10T20:42:24.109876+0000","last_became_peered":"2023-10-10T20:42:24.109876+0000","last_unstale":"2023-10-10T20:42:24.112675+0000","last_undegraded":"2023-10-10T20:42:24.112675+0000","last_fullsized":"2023-10-10T20:42:24.112675+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:40:58.862440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.115631+0000","last_change":"2023-10-10T20:42:24.112507+0000","last_active":"2023-10-10T20:42:24.115631+0000","last_peered":"2023-10-10T20:42:24.115631+0000","last_clean":"2023-10-10T20:42:24.115631+0000","last_became_active":"2023-10-10T20:42:24.112275+0000","last_became_peered":"2023-10-10T20:42:24.112275+0000","last_unstale":"2023-10-10T20:42:24.115631+0000","last_undegraded":"2023-10-10T20:42:24.115631+0000","last_fullsized":"2023-10-10T20:42:24.115631+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:06.445923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126524+0000","last_change":"2023-10-10T20:42:24.125813+0000","last_active":"2023-10-10T20:42:24.126524+0000","last_peered":"2023-10-10T20:42:24.126524+0000","last_clean":"2023-10-10T20:42:24.126524+0000","last_became_active":"2023-10-10T20:42:24.125554+0000","last_became_peered":"2023-10-10T20:42:24.125554+0000","last_unstale":"2023-10-10T20:42:24.126524+0000","last_undegraded":"2023-10-10T20:42:24.126524+0000","last_fullsized":"2023-10-10T20:42:24.126524+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:40:58.862440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.117282+0000","last_change":"2023-10-10T20:42:24.113162+0000","last_active":"2023-10-10T20:42:24.117282+0000","last_peered":"2023-10-10T20:42:24.117282+0000","last_clean":"2023-10-10T20:42:24.117282+0000","last_became_active":"2023-10-10T20:42:24.112919+0000","last_became_peered":"2023-10-10T20:42:24.112919+0000","last_unstale":"2023-10-10T20:42:24.117282+0000","last_undegraded":"2023-10-10T20:42:24.117282+0000","last_fullsized":"2023-10-10T20:42:24.117282+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:58:46.317589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.118301+0000","last_change":"2023-10-10T20:42:24.111610+0000","last_active":"2023-10-10T20:42:24.118301+0000","last_peered":"2023-10-10T20:42:24.118301+0000","last_clean":"2023-10-10T20:42:24.118301+0000","last_became_active":"2023-10-10T20:42:24.111317+0000","last_became_peered":"2023-10-10T20:42:24.111317+0000","last_unstale":"2023-10-10T20:42:24.118301+0000","last_undegraded":"2023-10-10T20:42:24.118301+0000","last_fullsized":"2023-10-10T20:42:24.118301+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:49:38.681149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.118411+0000","last_change":"2023-10-10T20:42:24.111047+0000","last_active":"2023-10-10T20:42:24.118411+0000","last_peered":"2023-10-10T20:42:24.118411+0000","last_clean":"2023-10-10T20:42:24.118411+0000","last_became_active":"2023-10-10T20:42:24.110810+0000","last_became_peered":"2023-10-10T20:42:24.110810+0000","last_unstale":"2023-10-10T20:42:24.118411+0000","last_undegraded":"2023-10-10T20:42:24.118411+0000","last_fullsized":"2023-10-10T20:42:24.118411+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:47.040763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113486+0000","last_change":"2023-10-10T20:42:24.109452+0000","last_active":"2023-10-10T20:42:24.113486+0000","last_peered":"2023-10-10T20:42:24.113486+0000","last_clean":"2023-10-10T20:42:24.113486+0000","last_became_active":"2023-10-10T20:42:24.109179+0000","last_became_peered":"2023-10-10T20:42:24.109179+0000","last_unstale":"2023-10-10T20:42:24.113486+0000","last_undegraded":"2023-10-10T20:42:24.113486+0000","last_fullsized":"2023-10-10T20:42:24.113486+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:59.819150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.112215+0000","last_change":"2023-10-10T20:42:24.105755+0000","last_active":"2023-10-10T20:42:24.112215+0000","last_peered":"2023-10-10T20:42:24.112215+0000","last_clean":"2023-10-10T20:42:24.112215+0000","last_became_active":"2023-10-10T20:42:24.104420+0000","last_became_peered":"2023-10-10T20:42:24.104420+0000","last_unstale":"2023-10-10T20:42:24.112215+0000","last_undegraded":"2023-10-10T20:42:24.112215+0000","last_fullsized":"2023-10-10T20:42:24.112215+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:03:33.363174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113655+0000","last_change":"2023-10-10T20:42:24.110180+0000","last_active":"2023-10-10T20:42:24.113655+0000","last_peered":"2023-10-10T20:42:24.113655+0000","last_clean":"2023-10-10T20:42:24.113655+0000","last_became_active":"2023-10-10T20:42:24.109893+0000","last_became_peered":"2023-10-10T20:42:24.109893+0000","last_unstale":"2023-10-10T20:42:24.113655+0000","last_undegraded":"2023-10-10T20:42:24.113655+0000","last_fullsized":"2023-10-10T20:42:24.113655+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:09:32.731249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.117221+0000","last_change":"2023-10-10T20:42:24.116069+0000","last_active":"2023-10-10T20:42:24.117221+0000","last_peered":"2023-10-10T20:42:24.117221+0000","last_clean":"2023-10-10T20:42:24.117221+0000","last_became_active":"2023-10-10T20:42:24.115782+0000","last_became_peered":"2023-10-10T20:42:24.115782+0000","last_unstale":"2023-10-10T20:42:24.117221+0000","last_undegraded":"2023-10-10T20:42:24.117221+0000","last_fullsized":"2023-10-10T20:42:24.117221+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:14:18.635698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.107687+0000","last_change":"2023-10-10T20:42:24.102224+0000","last_active":"2023-10-10T20:42:24.107687+0000","last_peered":"2023-10-10T20:42:24.107687+0000","last_clean":"2023-10-10T20:42:24.107687+0000","last_became_active":"2023-10-10T20:42:24.101963+0000","last_became_peered":"2023-10-10T20:42:24.101963+0000","last_unstale":"2023-10-10T20:42:24.107687+0000","last_undegraded":"2023-10-10T20:42:24.107687+0000","last_fullsized":"2023-10-10T20:42:24.107687+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:38:54.118178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113541+0000","last_change":"2023-10-10T20:42:24.109079+0000","last_active":"2023-10-10T20:42:24.113541+0000","last_peered":"2023-10-10T20:42:24.113541+0000","last_clean":"2023-10-10T20:42:24.113541+0000","last_became_active":"2023-10-10T20:42:24.108731+0000","last_became_peered":"2023-10-10T20:42:24.108731+0000","last_unstale":"2023-10-10T20:42:24.113541+0000","last_undegraded":"2023-10-10T20:42:24.113541+0000","last_fullsized":"2023-10-10T20:42:24.113541+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:06:21.357294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121390+0000","last_change":"2023-10-10T20:42:24.115936+0000","last_active":"2023-10-10T20:42:24.121390+0000","last_peered":"2023-10-10T20:42:24.121390+0000","last_clean":"2023-10-10T20:42:24.121390+0000","last_became_active":"2023-10-10T20:42:24.115659+0000","last_became_peered":"2023-10-10T20:42:24.115659+0000","last_unstale":"2023-10-10T20:42:24.121390+0000","last_undegraded":"2023-10-10T20:42:24.121390+0000","last_fullsized":"2023-10-10T20:42:24.121390+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:47.040763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126170+0000","last_change":"2023-10-10T20:42:24.123600+0000","last_active":"2023-10-10T20:42:24.126170+0000","last_peered":"2023-10-10T20:42:24.126170+0000","last_clean":"2023-10-10T20:42:24.126170+0000","last_became_active":"2023-10-10T20:42:24.123226+0000","last_became_peered":"2023-10-10T20:42:24.123226+0000","last_unstale":"2023-10-10T20:42:24.126170+0000","last_undegraded":"2023-10-10T20:42:24.126170+0000","last_fullsized":"2023-10-10T20:42:24.126170+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:46:15.751003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.112765+0000","last_change":"2023-10-10T20:42:24.110429+0000","last_active":"2023-10-10T20:42:24.112765+0000","last_peered":"2023-10-10T20:42:24.112765+0000","last_clean":"2023-10-10T20:42:24.112765+0000","last_became_active":"2023-10-10T20:42:24.110171+0000","last_became_peered":"2023-10-10T20:42:24.110171+0000","last_unstale":"2023-10-10T20:42:24.112765+0000","last_undegraded":"2023-10-10T20:42:24.112765+0000","last_fullsized":"2023-10-10T20:42:24.112765+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:02.631228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.128984+0000","last_change":"2023-10-10T20:42:24.125254+0000","last_active":"2023-10-10T20:42:24.128984+0000","last_peered":"2023-10-10T20:42:24.128984+0000","last_clean":"2023-10-10T20:42:24.128984+0000","last_became_active":"2023-10-10T20:42:24.124838+0000","last_became_peered":"2023-10-10T20:42:24.124838+0000","last_unstale":"2023-10-10T20:42:24.128984+0000","last_undegraded":"2023-10-10T20:42:24.128984+0000","last_fullsized":"2023-10-10T20:42:24.128984+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:13.353418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114550+0000","last_change":"2023-10-10T20:42:24.110653+0000","last_active":"2023-10-10T20:42:24.114550+0000","last_peered":"2023-10-10T20:42:24.114550+0000","last_clean":"2023-10-10T20:42:24.114550+0000","last_became_active":"2023-10-10T20:42:24.110226+0000","last_became_peered":"2023-10-10T20:42:24.110226+0000","last_unstale":"2023-10-10T20:42:24.114550+0000","last_undegraded":"2023-10-10T20:42:24.114550+0000","last_fullsized":"2023-10-10T20:42:24.114550+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:02.631228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.120869+0000","last_change":"2023-10-10T20:42:24.112486+0000","last_active":"2023-10-10T20:42:24.120869+0000","last_peered":"2023-10-10T20:42:24.120869+0000","last_clean":"2023-10-10T20:42:24.120869+0000","last_became_active":"2023-10-10T20:42:24.112258+0000","last_became_peered":"2023-10-10T20:42:24.112258+0000","last_unstale":"2023-10-10T20:42:24.120869+0000","last_undegraded":"2023-10-10T20:42:24.120869+0000","last_fullsized":"2023-10-10T20:42:24.120869+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:46:07.440450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121137+0000","last_change":"2023-10-10T20:42:24.115300+0000","last_active":"2023-10-10T20:42:24.121137+0000","last_peered":"2023-10-10T20:42:24.121137+0000","last_clean":"2023-10-10T20:42:24.121137+0000","last_became_active":"2023-10-10T20:42:24.115040+0000","last_became_peered":"2023-10-10T20:42:24.115040+0000","last_unstale":"2023-10-10T20:42:24.121137+0000","last_undegraded":"2023-10-10T20:42:24.121137+0000","last_fullsized":"2023-10-10T20:42:24.121137+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:46:19.157900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.111054+0000","last_change":"2023-10-10T20:42:24.104338+0000","last_active":"2023-10-10T20:42:24.111054+0000","last_peered":"2023-10-10T20:42:24.111054+0000","last_clean":"2023-10-10T20:42:24.111054+0000","last_became_active":"2023-10-10T20:42:24.104037+0000","last_became_peered":"2023-10-10T20:42:24.104037+0000","last_unstale":"2023-10-10T20:42:24.111054+0000","last_undegraded":"2023-10-10T20:42:24.111054+0000","last_fullsized":"2023-10-10T20:42:24.111054+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:25:09.330944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"152'4","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.131700+0000","last_change":"2023-10-10T20:42:23.105959+0000","last_active":"2023-10-10T20:42:24.131700+0000","last_peered":"2023-10-10T20:42:24.131700+0000","last_clean":"2023-10-10T20:42:24.131700+0000","last_became_active":"2023-10-10T20:42:23.105709+0000","last_became_peered":"2023-10-10T20:42:23.105709+0000","last_unstale":"2023-10-10T20:42:24.131700+0000","last_undegraded":"2023-10-10T20:42:24.131700+0000","last_fullsized":"2023-10-10T20:42:24.131700+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:42.951721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.128823+0000","last_change":"2023-10-10T20:42:24.124965+0000","last_active":"2023-10-10T20:42:24.128823+0000","last_peered":"2023-10-10T20:42:24.128823+0000","last_clean":"2023-10-10T20:42:24.128823+0000","last_became_active":"2023-10-10T20:42:24.124706+0000","last_became_peered":"2023-10-10T20:42:24.124706+0000","last_unstale":"2023-10-10T20:42:24.128823+0000","last_undegraded":"2023-10-10T20:42:24.128823+0000","last_fullsized":"2023-10-10T20:42:24.128823+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:47:26.708886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.120664+0000","last_change":"2023-10-10T20:42:23.105858+0000","last_active":"2023-10-10T20:42:24.120664+0000","last_peered":"2023-10-10T20:42:24.120664+0000","last_clean":"2023-10-10T20:42:24.120664+0000","last_became_active":"2023-10-10T20:42:23.105592+0000","last_became_peered":"2023-10-10T20:42:23.105592+0000","last_unstale":"2023-10-10T20:42:24.120664+0000","last_undegraded":"2023-10-10T20:42:24.120664+0000","last_fullsized":"2023-10-10T20:42:24.120664+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:54.378347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121344+0000","last_change":"2023-10-10T20:42:24.115929+0000","last_active":"2023-10-10T20:42:24.121344+0000","last_peered":"2023-10-10T20:42:24.121344+0000","last_clean":"2023-10-10T20:42:24.121344+0000","last_became_active":"2023-10-10T20:42:24.115637+0000","last_became_peered":"2023-10-10T20:42:24.115637+0000","last_unstale":"2023-10-10T20:42:24.121344+0000","last_undegraded":"2023-10-10T20:42:24.121344+0000","last_fullsized":"2023-10-10T20:42:24.121344+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:39:10.428774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.124476+0000","last_change":"2023-10-10T20:42:23.103827+0000","last_active":"2023-10-10T20:42:24.124476+0000","last_peered":"2023-10-10T20:42:24.124476+0000","last_clean":"2023-10-10T20:42:24.124476+0000","last_became_active":"2023-10-10T20:42:23.103329+0000","last_became_peered":"2023-10-10T20:42:23.103329+0000","last_unstale":"2023-10-10T20:42:24.124476+0000","last_undegraded":"2023-10-10T20:42:24.124476+0000","last_fullsized":"2023-10-10T20:42:24.124476+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:42:08.332947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121209+0000","last_change":"2023-10-10T20:42:24.114480+0000","last_active":"2023-10-10T20:42:24.121209+0000","last_peered":"2023-10-10T20:42:24.121209+0000","last_clean":"2023-10-10T20:42:24.121209+0000","last_became_active":"2023-10-10T20:42:24.114244+0000","last_became_peered":"2023-10-10T20:42:24.114244+0000","last_unstale":"2023-10-10T20:42:24.121209+0000","last_undegraded":"2023-10-10T20:42:24.121209+0000","last_fullsized":"2023-10-10T20:42:24.121209+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:00:26.956205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.124758+0000","last_change":"2023-10-10T20:42:23.104651+0000","last_active":"2023-10-10T20:42:24.124758+0000","last_peered":"2023-10-10T20:42:24.124758+0000","last_clean":"2023-10-10T20:42:24.124758+0000","last_became_active":"2023-10-10T20:42:23.104297+0000","last_became_peered":"2023-10-10T20:42:23.104297+0000","last_unstale":"2023-10-10T20:42:24.124758+0000","last_undegraded":"2023-10-10T20:42:24.124758+0000","last_fullsized":"2023-10-10T20:42:24.124758+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:49:42.586346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114531+0000","last_change":"2023-10-10T20:42:24.112093+0000","last_active":"2023-10-10T20:42:24.114531+0000","last_peered":"2023-10-10T20:42:24.114531+0000","last_clean":"2023-10-10T20:42:24.114531+0000","last_became_active":"2023-10-10T20:42:24.111868+0000","last_became_peered":"2023-10-10T20:42:24.111868+0000","last_unstale":"2023-10-10T20:42:24.114531+0000","last_undegraded":"2023-10-10T20:42:24.114531+0000","last_fullsized":"2023-10-10T20:42:24.114531+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:30.464320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.125138+0000","last_change":"2023-10-10T20:42:23.105709+0000","last_active":"2023-10-10T20:42:24.125138+0000","last_peered":"2023-10-10T20:42:24.125138+0000","last_clean":"2023-10-10T20:42:24.125138+0000","last_became_active":"2023-10-10T20:42:23.105538+0000","last_became_peered":"2023-10-10T20:42:23.105538+0000","last_unstale":"2023-10-10T20:42:24.125138+0000","last_undegraded":"2023-10-10T20:42:24.125138+0000","last_fullsized":"2023-10-10T20:42:24.125138+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:26:59.934888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113606+0000","last_change":"2023-10-10T20:42:24.109974+0000","last_active":"2023-10-10T20:42:24.113606+0000","last_peered":"2023-10-10T20:42:24.113606+0000","last_clean":"2023-10-10T20:42:24.113606+0000","last_became_active":"2023-10-10T20:42:24.109707+0000","last_became_peered":"2023-10-10T20:42:24.109707+0000","last_unstale":"2023-10-10T20:42:24.113606+0000","last_undegraded":"2023-10-10T20:42:24.113606+0000","last_fullsized":"2023-10-10T20:42:24.113606+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:34.653111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.101968+0000","last_change":"2023-10-10T20:42:23.109627+0000","last_active":"2023-10-10T20:42:24.101968+0000","last_peered":"2023-10-10T20:42:24.101968+0000","last_clean":"2023-10-10T20:42:24.101968+0000","last_became_active":"2023-10-10T20:42:23.109387+0000","last_became_peered":"2023-10-10T20:42:23.109387+0000","last_unstale":"2023-10-10T20:42:24.101968+0000","last_undegraded":"2023-10-10T20:42:24.101968+0000","last_fullsized":"2023-10-10T20:42:24.101968+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:12.342683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114505+0000","last_change":"2023-10-10T20:42:24.111171+0000","last_active":"2023-10-10T20:42:24.114505+0000","last_peered":"2023-10-10T20:42:24.114505+0000","last_clean":"2023-10-10T20:42:24.114505+0000","last_became_active":"2023-10-10T20:42:24.110894+0000","last_became_peered":"2023-10-10T20:42:24.110894+0000","last_unstale":"2023-10-10T20:42:24.114505+0000","last_undegraded":"2023-10-10T20:42:24.114505+0000","last_fullsized":"2023-10-10T20:42:24.114505+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:29:16.754131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.102665+0000","last_change":"2023-10-10T20:42:23.105273+0000","last_active":"2023-10-10T20:42:24.102665+0000","last_peered":"2023-10-10T20:42:24.102665+0000","last_clean":"2023-10-10T20:42:24.102665+0000","last_became_active":"2023-10-10T20:42:23.104999+0000","last_became_peered":"2023-10-10T20:42:23.104999+0000","last_unstale":"2023-10-10T20:42:24.102665+0000","last_undegraded":"2023-10-10T20:42:24.102665+0000","last_fullsized":"2023-10-10T20:42:24.102665+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:04.056737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.103733+0000","last_change":"2023-10-10T20:42:24.101017+0000","last_active":"2023-10-10T20:42:24.103733+0000","last_peered":"2023-10-10T20:42:24.103733+0000","last_clean":"2023-10-10T20:42:24.103733+0000","last_became_active":"2023-10-10T20:42:24.100716+0000","last_became_peered":"2023-10-10T20:42:24.100716+0000","last_unstale":"2023-10-10T20:42:24.103733+0000","last_undegraded":"2023-10-10T20:42:24.103733+0000","last_fullsized":"2023-10-10T20:42:24.103733+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:20:48.561636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.101835+0000","last_change":"2023-10-10T20:42:23.108782+0000","last_active":"2023-10-10T20:42:24.101835+0000","last_peered":"2023-10-10T20:42:24.101835+0000","last_clean":"2023-10-10T20:42:24.101835+0000","last_became_active":"2023-10-10T20:42:23.108480+0000","last_became_peered":"2023-10-10T20:42:23.108480+0000","last_unstale":"2023-10-10T20:42:24.101835+0000","last_undegraded":"2023-10-10T20:42:24.101835+0000","last_fullsized":"2023-10-10T20:42:24.101835+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:32:14.835359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.112616+0000","last_change":"2023-10-10T20:42:24.108473+0000","last_active":"2023-10-10T20:42:24.112616+0000","last_peered":"2023-10-10T20:42:24.112616+0000","last_clean":"2023-10-10T20:42:24.112616+0000","last_became_active":"2023-10-10T20:42:24.108220+0000","last_became_peered":"2023-10-10T20:42:24.108220+0000","last_unstale":"2023-10-10T20:42:24.112616+0000","last_undegraded":"2023-10-10T20:42:24.112616+0000","last_fullsized":"2023-10-10T20:42:24.112616+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:49.847316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.103586+0000","last_change":"2023-10-10T20:42:23.109104+0000","last_active":"2023-10-10T20:42:24.103586+0000","last_peered":"2023-10-10T20:42:24.103586+0000","last_clean":"2023-10-10T20:42:24.103586+0000","last_became_active":"2023-10-10T20:42:23.108836+0000","last_became_peered":"2023-10-10T20:42:23.108836+0000","last_unstale":"2023-10-10T20:42:24.103586+0000","last_undegraded":"2023-10-10T20:42:24.103586+0000","last_fullsized":"2023-10-10T20:42:24.103586+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:42.951721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113476+0000","last_change":"2023-10-10T20:42:24.108573+0000","last_active":"2023-10-10T20:42:24.113476+0000","last_peered":"2023-10-10T20:42:24.113476+0000","last_clean":"2023-10-10T20:42:24.113476+0000","last_became_active":"2023-10-10T20:42:24.108300+0000","last_became_peered":"2023-10-10T20:42:24.108300+0000","last_unstale":"2023-10-10T20:42:24.113476+0000","last_undegraded":"2023-10-10T20:42:24.113476+0000","last_fullsized":"2023-10-10T20:42:24.113476+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:52:51.382479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"152'3","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.123020+0000","last_change":"2023-10-10T20:42:23.107169+0000","last_active":"2023-10-10T20:42:24.123020+0000","last_peered":"2023-10-10T20:42:24.123020+0000","last_clean":"2023-10-10T20:42:24.123020+0000","last_became_active":"2023-10-10T20:42:23.106721+0000","last_became_peered":"2023-10-10T20:42:23.106721+0000","last_unstale":"2023-10-10T20:42:24.123020+0000","last_undegraded":"2023-10-10T20:42:24.123020+0000","last_fullsized":"2023-10-10T20:42:24.123020+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:03:32.352440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121029+0000","last_change":"2023-10-10T20:42:24.115401+0000","last_active":"2023-10-10T20:42:24.121029+0000","last_peered":"2023-10-10T20:42:24.121029+0000","last_clean":"2023-10-10T20:42:24.121029+0000","last_became_active":"2023-10-10T20:42:24.113812+0000","last_became_peered":"2023-10-10T20:42:24.113812+0000","last_unstale":"2023-10-10T20:42:24.121029+0000","last_undegraded":"2023-10-10T20:42:24.121029+0000","last_fullsized":"2023-10-10T20:42:24.121029+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:01.948796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"152'6","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.132627+0000","last_change":"2023-10-10T20:42:23.108302+0000","last_active":"2023-10-10T20:42:24.132627+0000","last_peered":"2023-10-10T20:42:24.132627+0000","last_clean":"2023-10-10T20:42:24.132627+0000","last_became_active":"2023-10-10T20:42:23.108085+0000","last_became_peered":"2023-10-10T20:42:23.108085+0000","last_unstale":"2023-10-10T20:42:24.132627+0000","last_undegraded":"2023-10-10T20:42:24.132627+0000","last_fullsized":"2023-10-10T20:42:24.132627+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:02:33.953621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113456+0000","last_change":"2023-10-10T20:42:24.109419+0000","last_active":"2023-10-10T20:42:24.113456+0000","last_peered":"2023-10-10T20:42:24.113456+0000","last_clean":"2023-10-10T20:42:24.113456+0000","last_became_active":"2023-10-10T20:42:24.109068+0000","last_became_peered":"2023-10-10T20:42:24.109068+0000","last_unstale":"2023-10-10T20:42:24.113456+0000","last_undegraded":"2023-10-10T20:42:24.113456+0000","last_fullsized":"2023-10-10T20:42:24.113456+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:39:59.929264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126772+0000","last_change":"2023-10-10T20:42:23.105344+0000","last_active":"2023-10-10T20:42:24.126772+0000","last_peered":"2023-10-10T20:42:24.126772+0000","last_clean":"2023-10-10T20:42:24.126772+0000","last_became_active":"2023-10-10T20:42:23.105013+0000","last_became_peered":"2023-10-10T20:42:23.105013+0000","last_unstale":"2023-10-10T20:42:24.126772+0000","last_undegraded":"2023-10-10T20:42:24.126772+0000","last_fullsized":"2023-10-10T20:42:24.126772+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:59.544805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121309+0000","last_change":"2023-10-10T20:42:24.115013+0000","last_active":"2023-10-10T20:42:24.121309+0000","last_peered":"2023-10-10T20:42:24.121309+0000","last_clean":"2023-10-10T20:42:24.121309+0000","last_became_active":"2023-10-10T20:42:24.114763+0000","last_became_peered":"2023-10-10T20:42:24.114763+0000","last_unstale":"2023-10-10T20:42:24.121309+0000","last_undegraded":"2023-10-10T20:42:24.121309+0000","last_fullsized":"2023-10-10T20:42:24.121309+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:06.445923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.103163+0000","last_change":"2023-10-10T20:42:23.105949+0000","last_active":"2023-10-10T20:42:24.103163+0000","last_peered":"2023-10-10T20:42:24.103163+0000","last_clean":"2023-10-10T20:42:24.103163+0000","last_became_active":"2023-10-10T20:42:23.105676+0000","last_became_peered":"2023-10-10T20:42:23.105676+0000","last_unstale":"2023-10-10T20:42:24.103163+0000","last_undegraded":"2023-10-10T20:42:24.103163+0000","last_fullsized":"2023-10-10T20:42:24.103163+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:43:10.994181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126457+0000","last_change":"2023-10-10T20:42:24.126007+0000","last_active":"2023-10-10T20:42:24.126457+0000","last_peered":"2023-10-10T20:42:24.126457+0000","last_clean":"2023-10-10T20:42:24.126457+0000","last_became_active":"2023-10-10T20:42:24.125699+0000","last_became_peered":"2023-10-10T20:42:24.125699+0000","last_unstale":"2023-10-10T20:42:24.126457+0000","last_undegraded":"2023-10-10T20:42:24.126457+0000","last_fullsized":"2023-10-10T20:42:24.126457+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:09:15.013589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.101618+0000","last_change":"2023-10-10T20:42:23.103097+0000","last_active":"2023-10-10T20:42:24.101618+0000","last_peered":"2023-10-10T20:42:24.101618+0000","last_clean":"2023-10-10T20:42:24.101618+0000","last_became_active":"2023-10-10T20:42:23.102819+0000","last_became_peered":"2023-10-10T20:42:23.102819+0000","last_unstale":"2023-10-10T20:42:24.101618+0000","last_undegraded":"2023-10-10T20:42:24.101618+0000","last_fullsized":"2023-10-10T20:42:24.101618+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:20:47.550902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121175+0000","last_change":"2023-10-10T20:42:24.115553+0000","last_active":"2023-10-10T20:42:24.121175+0000","last_peered":"2023-10-10T20:42:24.121175+0000","last_clean":"2023-10-10T20:42:24.121175+0000","last_became_active":"2023-10-10T20:42:24.115239+0000","last_became_peered":"2023-10-10T20:42:24.115239+0000","last_unstale":"2023-10-10T20:42:24.121175+0000","last_undegraded":"2023-10-10T20:42:24.121175+0000","last_fullsized":"2023-10-10T20:42:24.121175+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:14.819673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"152'8","reported_seq":18,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.125165+0000","last_change":"2023-10-10T20:42:23.103610+0000","last_active":"2023-10-10T20:42:24.125165+0000","last_peered":"2023-10-10T20:42:24.125165+0000","last_clean":"2023-10-10T20:42:24.125165+0000","last_became_active":"2023-10-10T20:42:23.103353+0000","last_became_peered":"2023-10-10T20:42:23.103353+0000","last_unstale":"2023-10-10T20:42:24.125165+0000","last_undegraded":"2023-10-10T20:42:24.125165+0000","last_fullsized":"2023-10-10T20:42:24.125165+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:23:34.102393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.111073+0000","last_change":"2023-10-10T20:42:24.104841+0000","last_active":"2023-10-10T20:42:24.111073+0000","last_peered":"2023-10-10T20:42:24.111073+0000","last_clean":"2023-10-10T20:42:24.111073+0000","last_became_active":"2023-10-10T20:42:24.104561+0000","last_became_peered":"2023-10-10T20:42:24.104561+0000","last_unstale":"2023-10-10T20:42:24.111073+0000","last_undegraded":"2023-10-10T20:42:24.111073+0000","last_fullsized":"2023-10-10T20:42:24.111073+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:45:43.231122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.123211+0000","last_change":"2023-10-10T20:42:23.105236+0000","last_active":"2023-10-10T20:42:24.123211+0000","last_peered":"2023-10-10T20:42:24.123211+0000","last_clean":"2023-10-10T20:42:24.123211+0000","last_became_active":"2023-10-10T20:42:23.104983+0000","last_became_peered":"2023-10-10T20:42:23.104983+0000","last_unstale":"2023-10-10T20:42:24.123211+0000","last_undegraded":"2023-10-10T20:42:24.123211+0000","last_fullsized":"2023-10-10T20:42:24.123211+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:03.181433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113521+0000","last_change":"2023-10-10T20:42:24.107994+0000","last_active":"2023-10-10T20:42:24.113521+0000","last_peered":"2023-10-10T20:42:24.113521+0000","last_clean":"2023-10-10T20:42:24.113521+0000","last_became_active":"2023-10-10T20:42:24.107722+0000","last_became_peered":"2023-10-10T20:42:24.107722+0000","last_unstale":"2023-10-10T20:42:24.113521+0000","last_undegraded":"2023-10-10T20:42:24.113521+0000","last_fullsized":"2023-10-10T20:42:24.113521+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:57.924961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.123838+0000","last_change":"2023-10-10T20:42:23.111459+0000","last_active":"2023-10-10T20:42:24.123838+0000","last_peered":"2023-10-10T20:42:24.123838+0000","last_clean":"2023-10-10T20:42:24.123838+0000","last_became_active":"2023-10-10T20:42:23.111197+0000","last_became_peered":"2023-10-10T20:42:23.111197+0000","last_unstale":"2023-10-10T20:42:24.123838+0000","last_undegraded":"2023-10-10T20:42:24.123838+0000","last_fullsized":"2023-10-10T20:42:24.123838+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:23:14.263295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.106150+0000","last_change":"2023-10-10T20:42:24.101341+0000","last_active":"2023-10-10T20:42:24.106150+0000","last_peered":"2023-10-10T20:42:24.106150+0000","last_clean":"2023-10-10T20:42:24.106150+0000","last_became_active":"2023-10-10T20:42:24.101086+0000","last_became_peered":"2023-10-10T20:42:24.101086+0000","last_unstale":"2023-10-10T20:42:24.106150+0000","last_undegraded":"2023-10-10T20:42:24.106150+0000","last_fullsized":"2023-10-10T20:42:24.106150+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:11:04.270228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126614+0000","last_change":"2023-10-10T20:42:23.107193+0000","last_active":"2023-10-10T20:42:24.126614+0000","last_peered":"2023-10-10T20:42:24.126614+0000","last_clean":"2023-10-10T20:42:24.126614+0000","last_became_active":"2023-10-10T20:42:23.106136+0000","last_became_peered":"2023-10-10T20:42:23.106136+0000","last_unstale":"2023-10-10T20:42:24.126614+0000","last_undegraded":"2023-10-10T20:42:24.126614+0000","last_fullsized":"2023-10-10T20:42:24.126614+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:26:51.984824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.120858+0000","last_change":"2023-10-10T20:42:24.112738+0000","last_active":"2023-10-10T20:42:24.120858+0000","last_peered":"2023-10-10T20:42:24.120858+0000","last_clean":"2023-10-10T20:42:24.120858+0000","last_became_active":"2023-10-10T20:42:24.112515+0000","last_became_peered":"2023-10-10T20:42:24.112515+0000","last_unstale":"2023-10-10T20:42:24.120858+0000","last_undegraded":"2023-10-10T20:42:24.120858+0000","last_fullsized":"2023-10-10T20:42:24.120858+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:01.948796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.096078+0000","last_change":"2023-10-10T20:42:23.107823+0000","last_active":"2023-10-10T20:42:24.096078+0000","last_peered":"2023-10-10T20:42:24.096078+0000","last_clean":"2023-10-10T20:42:24.096078+0000","last_became_active":"2023-10-10T20:42:23.107643+0000","last_became_peered":"2023-10-10T20:42:23.107643+0000","last_unstale":"2023-10-10T20:42:24.096078+0000","last_undegraded":"2023-10-10T20:42:24.096078+0000","last_fullsized":"2023-10-10T20:42:24.096078+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:43:41.316715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.120251+0000","last_change":"2023-10-10T20:42:24.110967+0000","last_active":"2023-10-10T20:42:24.120251+0000","last_peered":"2023-10-10T20:42:24.120251+0000","last_clean":"2023-10-10T20:42:24.120251+0000","last_became_active":"2023-10-10T20:42:24.110687+0000","last_became_peered":"2023-10-10T20:42:24.110687+0000","last_unstale":"2023-10-10T20:42:24.120251+0000","last_undegraded":"2023-10-10T20:42:24.120251+0000","last_fullsized":"2023-10-10T20:42:24.120251+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:53.653343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121843+0000","last_change":"2023-10-10T20:42:23.105742+0000","last_active":"2023-10-10T20:42:24.121843+0000","last_peered":"2023-10-10T20:42:24.121843+0000","last_clean":"2023-10-10T20:42:24.121843+0000","last_became_active":"2023-10-10T20:42:23.105472+0000","last_became_peered":"2023-10-10T20:42:23.105472+0000","last_unstale":"2023-10-10T20:42:24.121843+0000","last_undegraded":"2023-10-10T20:42:24.121843+0000","last_fullsized":"2023-10-10T20:42:24.121843+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:29.453586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113971+0000","last_change":"2023-10-10T20:42:24.112508+0000","last_active":"2023-10-10T20:42:24.113971+0000","last_peered":"2023-10-10T20:42:24.113971+0000","last_clean":"2023-10-10T20:42:24.113971+0000","last_became_active":"2023-10-10T20:42:24.112276+0000","last_became_peered":"2023-10-10T20:42:24.112276+0000","last_unstale":"2023-10-10T20:42:24.113971+0000","last_undegraded":"2023-10-10T20:42:24.113971+0000","last_fullsized":"2023-10-10T20:42:24.113971+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:55:23.468746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.104357+0000","last_change":"2023-10-10T20:42:23.110977+0000","last_active":"2023-10-10T20:42:24.104357+0000","last_peered":"2023-10-10T20:42:24.104357+0000","last_clean":"2023-10-10T20:42:24.104357+0000","last_became_active":"2023-10-10T20:42:23.110704+0000","last_became_peered":"2023-10-10T20:42:23.110704+0000","last_unstale":"2023-10-10T20:42:24.104357+0000","last_undegraded":"2023-10-10T20:42:24.104357+0000","last_fullsized":"2023-10-10T20:42:24.104357+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:22:28.404747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114385+0000","last_change":"2023-10-10T20:42:24.109982+0000","last_active":"2023-10-10T20:42:24.114385+0000","last_peered":"2023-10-10T20:42:24.114385+0000","last_clean":"2023-10-10T20:42:24.114385+0000","last_became_active":"2023-10-10T20:42:24.109703+0000","last_became_peered":"2023-10-10T20:42:24.109703+0000","last_unstale":"2023-10-10T20:42:24.114385+0000","last_undegraded":"2023-10-10T20:42:24.114385+0000","last_fullsized":"2023-10-10T20:42:24.114385+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:46:22.492928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.103008+0000","last_change":"2023-10-10T20:42:23.109984+0000","last_active":"2023-10-10T20:42:24.103008+0000","last_peered":"2023-10-10T20:42:24.103008+0000","last_clean":"2023-10-10T20:42:24.103008+0000","last_became_active":"2023-10-10T20:42:23.109710+0000","last_became_peered":"2023-10-10T20:42:23.109710+0000","last_unstale":"2023-10-10T20:42:24.103008+0000","last_undegraded":"2023-10-10T20:42:24.103008+0000","last_fullsized":"2023-10-10T20:42:24.103008+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:41:54.647934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113784+0000","last_change":"2023-10-10T20:42:24.110731+0000","last_active":"2023-10-10T20:42:24.113784+0000","last_peered":"2023-10-10T20:42:24.113784+0000","last_clean":"2023-10-10T20:42:24.113784+0000","last_became_active":"2023-10-10T20:42:24.110499+0000","last_became_peered":"2023-10-10T20:42:24.110499+0000","last_unstale":"2023-10-10T20:42:24.113784+0000","last_undegraded":"2023-10-10T20:42:24.113784+0000","last_fullsized":"2023-10-10T20:42:24.113784+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:46:15.751003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.102364+0000","last_change":"2023-10-10T20:42:23.104363+0000","last_active":"2023-10-10T20:42:24.102364+0000","last_peered":"2023-10-10T20:42:24.102364+0000","last_clean":"2023-10-10T20:42:24.102364+0000","last_became_active":"2023-10-10T20:42:23.104108+0000","last_became_peered":"2023-10-10T20:42:23.104108+0000","last_unstale":"2023-10-10T20:42:24.102364+0000","last_undegraded":"2023-10-10T20:42:24.102364+0000","last_fullsized":"2023-10-10T20:42:24.102364+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:52.876825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.124500+0000","last_change":"2023-10-10T20:42:24.122887+0000","last_active":"2023-10-10T20:42:24.124500+0000","last_peered":"2023-10-10T20:42:24.124500+0000","last_clean":"2023-10-10T20:42:24.124500+0000","last_became_active":"2023-10-10T20:42:24.122521+0000","last_became_peered":"2023-10-10T20:42:24.122521+0000","last_unstale":"2023-10-10T20:42:24.124500+0000","last_undegraded":"2023-10-10T20:42:24.124500+0000","last_fullsized":"2023-10-10T20:42:24.124500+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:09:32.731249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.125650+0000","last_change":"2023-10-10T20:42:23.109167+0000","last_active":"2023-10-10T20:42:24.125650+0000","last_peered":"2023-10-10T20:42:24.125650+0000","last_clean":"2023-10-10T20:42:24.125650+0000","last_became_active":"2023-10-10T20:42:23.108692+0000","last_became_peered":"2023-10-10T20:42:23.108692+0000","last_unstale":"2023-10-10T20:42:24.125650+0000","last_undegraded":"2023-10-10T20:42:24.125650+0000","last_fullsized":"2023-10-10T20:42:24.125650+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:56.210224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126436+0000","last_change":"2023-10-10T20:42:24.125413+0000","last_active":"2023-10-10T20:42:24.126436+0000","last_peered":"2023-10-10T20:42:24.126436+0000","last_clean":"2023-10-10T20:42:24.126436+0000","last_became_active":"2023-10-10T20:42:24.125149+0000","last_became_peered":"2023-10-10T20:42:24.125149+0000","last_unstale":"2023-10-10T20:42:24.126436+0000","last_undegraded":"2023-10-10T20:42:24.126436+0000","last_fullsized":"2023-10-10T20:42:24.126436+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:00:26.956205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.096131+0000","last_change":"2023-10-10T20:42:23.098780+0000","last_active":"2023-10-10T20:42:24.096131+0000","last_peered":"2023-10-10T20:42:24.096131+0000","last_clean":"2023-10-10T20:42:24.096131+0000","last_became_active":"2023-10-10T20:42:23.098462+0000","last_became_peered":"2023-10-10T20:42:23.098462+0000","last_unstale":"2023-10-10T20:42:24.096131+0000","last_undegraded":"2023-10-10T20:42:24.096131+0000","last_fullsized":"2023-10-10T20:42:24.096131+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:37:13.498053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114471+0000","last_change":"2023-10-10T20:42:24.110340+0000","last_active":"2023-10-10T20:42:24.114471+0000","last_peered":"2023-10-10T20:42:24.114471+0000","last_clean":"2023-10-10T20:42:24.114471+0000","last_became_active":"2023-10-10T20:42:24.110079+0000","last_became_peered":"2023-10-10T20:42:24.110079+0000","last_unstale":"2023-10-10T20:42:24.114471+0000","last_undegraded":"2023-10-10T20:42:24.114471+0000","last_fullsized":"2023-10-10T20:42:24.114471+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:35:23.790779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.102018+0000","last_change":"2023-10-10T20:42:23.108778+0000","last_active":"2023-10-10T20:42:24.102018+0000","last_peered":"2023-10-10T20:42:24.102018+0000","last_clean":"2023-10-10T20:42:24.102018+0000","last_became_active":"2023-10-10T20:42:23.108454+0000","last_became_peered":"2023-10-10T20:42:23.108454+0000","last_unstale":"2023-10-10T20:42:24.102018+0000","last_undegraded":"2023-10-10T20:42:24.102018+0000","last_fullsized":"2023-10-10T20:42:24.102018+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:09:31.720515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114247+0000","last_change":"2023-10-10T20:42:24.111895+0000","last_active":"2023-10-10T20:42:24.114247+0000","last_peered":"2023-10-10T20:42:24.114247+0000","last_clean":"2023-10-10T20:42:24.114247+0000","last_became_active":"2023-10-10T20:42:24.111432+0000","last_became_peered":"2023-10-10T20:42:24.111432+0000","last_unstale":"2023-10-10T20:42:24.114247+0000","last_undegraded":"2023-10-10T20:42:24.114247+0000","last_fullsized":"2023-10-10T20:42:24.114247+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:21:59.491769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.096099+0000","last_change":"2023-10-10T20:42:23.108978+0000","last_active":"2023-10-10T20:42:24.096099+0000","last_peered":"2023-10-10T20:42:24.096099+0000","last_clean":"2023-10-10T20:42:24.096099+0000","last_became_active":"2023-10-10T20:42:23.108707+0000","last_became_peered":"2023-10-10T20:42:23.108707+0000","last_unstale":"2023-10-10T20:42:24.096099+0000","last_undegraded":"2023-10-10T20:42:24.096099+0000","last_fullsized":"2023-10-10T20:42:24.096099+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:44.681204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121726+0000","last_change":"2023-10-10T20:42:24.116434+0000","last_active":"2023-10-10T20:42:24.121726+0000","last_peered":"2023-10-10T20:42:24.121726+0000","last_clean":"2023-10-10T20:42:24.121726+0000","last_became_active":"2023-10-10T20:42:24.115786+0000","last_became_peered":"2023-10-10T20:42:24.115786+0000","last_unstale":"2023-10-10T20:42:24.121726+0000","last_undegraded":"2023-10-10T20:42:24.121726+0000","last_fullsized":"2023-10-10T20:42:24.121726+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:46:07.440450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.098460+0000","last_change":"2023-10-10T20:42:23.107133+0000","last_active":"2023-10-10T20:42:24.098460+0000","last_peered":"2023-10-10T20:42:24.098460+0000","last_clean":"2023-10-10T20:42:24.098460+0000","last_became_active":"2023-10-10T20:42:23.106863+0000","last_became_peered":"2023-10-10T20:42:23.106863+0000","last_unstale":"2023-10-10T20:42:24.098460+0000","last_undegraded":"2023-10-10T20:42:24.098460+0000","last_fullsized":"2023-10-10T20:42:24.098460+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:58.808416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126598+0000","last_change":"2023-10-10T20:42:24.123718+0000","last_active":"2023-10-10T20:42:24.126598+0000","last_peered":"2023-10-10T20:42:24.126598+0000","last_clean":"2023-10-10T20:42:24.126598+0000","last_became_active":"2023-10-10T20:42:24.123096+0000","last_became_peered":"2023-10-10T20:42:24.123096+0000","last_unstale":"2023-10-10T20:42:24.126598+0000","last_undegraded":"2023-10-10T20:42:24.126598+0000","last_fullsized":"2023-10-10T20:42:24.126598+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:14:18.635698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.104118+0000","last_change":"2023-10-10T20:42:23.104363+0000","last_active":"2023-10-10T20:42:24.104118+0000","last_peered":"2023-10-10T20:42:24.104118+0000","last_clean":"2023-10-10T20:42:24.104118+0000","last_became_active":"2023-10-10T20:42:23.104109+0000","last_became_peered":"2023-10-10T20:42:23.104109+0000","last_unstale":"2023-10-10T20:42:24.104118+0000","last_undegraded":"2023-10-10T20:42:24.104118+0000","last_fullsized":"2023-10-10T20:42:24.104118+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:43:10.994181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.112743+0000","last_change":"2023-10-10T20:42:24.109363+0000","last_active":"2023-10-10T20:42:24.112743+0000","last_peered":"2023-10-10T20:42:24.112743+0000","last_clean":"2023-10-10T20:42:24.112743+0000","last_became_active":"2023-10-10T20:42:24.109127+0000","last_became_peered":"2023-10-10T20:42:24.109127+0000","last_unstale":"2023-10-10T20:42:24.112743+0000","last_undegraded":"2023-10-10T20:42:24.112743+0000","last_fullsized":"2023-10-10T20:42:24.112743+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:58:55.908084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.103440+0000","last_change":"2023-10-10T20:42:23.108038+0000","last_active":"2023-10-10T20:42:24.103440+0000","last_peered":"2023-10-10T20:42:24.103440+0000","last_clean":"2023-10-10T20:42:24.103440+0000","last_became_active":"2023-10-10T20:42:23.107750+0000","last_became_peered":"2023-10-10T20:42:23.107750+0000","last_unstale":"2023-10-10T20:42:24.103440+0000","last_undegraded":"2023-10-10T20:42:24.103440+0000","last_fullsized":"2023-10-10T20:42:24.103440+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:00:27.632584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114575+0000","last_change":"2023-10-10T20:42:24.110672+0000","last_active":"2023-10-10T20:42:24.114575+0000","last_peered":"2023-10-10T20:42:24.114575+0000","last_clean":"2023-10-10T20:42:24.114575+0000","last_became_active":"2023-10-10T20:42:24.110384+0000","last_became_peered":"2023-10-10T20:42:24.110384+0000","last_unstale":"2023-10-10T20:42:24.114575+0000","last_undegraded":"2023-10-10T20:42:24.114575+0000","last_fullsized":"2023-10-10T20:42:24.114575+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:49.847316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.108826+0000","last_change":"2023-10-10T20:42:23.103184+0000","last_active":"2023-10-10T20:42:24.108826+0000","last_peered":"2023-10-10T20:42:24.108826+0000","last_clean":"2023-10-10T20:42:24.108826+0000","last_became_active":"2023-10-10T20:42:23.102891+0000","last_became_peered":"2023-10-10T20:42:23.102891+0000","last_unstale":"2023-10-10T20:42:24.108826+0000","last_undegraded":"2023-10-10T20:42:24.108826+0000","last_fullsized":"2023-10-10T20:42:24.108826+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:48:40.304153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.117438+0000","last_change":"2023-10-10T20:42:24.113117+0000","last_active":"2023-10-10T20:42:24.117438+0000","last_peered":"2023-10-10T20:42:24.117438+0000","last_clean":"2023-10-10T20:42:24.117438+0000","last_became_active":"2023-10-10T20:42:24.112829+0000","last_became_peered":"2023-10-10T20:42:24.112829+0000","last_unstale":"2023-10-10T20:42:24.117438+0000","last_undegraded":"2023-10-10T20:42:24.117438+0000","last_fullsized":"2023-10-10T20:42:24.117438+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:17:06.399895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.131253+0000","last_change":"2023-10-10T20:42:23.110595+0000","last_active":"2023-10-10T20:42:24.131253+0000","last_peered":"2023-10-10T20:42:24.131253+0000","last_clean":"2023-10-10T20:42:24.131253+0000","last_became_active":"2023-10-10T20:42:23.110274+0000","last_became_peered":"2023-10-10T20:42:23.110274+0000","last_unstale":"2023-10-10T20:42:24.131253+0000","last_undegraded":"2023-10-10T20:42:24.131253+0000","last_fullsized":"2023-10-10T20:42:24.131253+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:04.056737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":296,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.096203+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:42:24.096203+0000","last_peered":"2023-10-10T20:42:24.096203+0000","last_clean":"2023-10-10T20:42:24.096203+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:42:24.096203+0000","last_undegraded":"2023-10-10T20:42:24.096203+0000","last_fullsized":"2023-10-10T20:42:24.096203+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574948,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303868,"kb_used_data":2484,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067972,"statfs":{"total":96636764160,"available":96325603328,"internally_reserved":0,"allocated":2543616,"data_stored":1460226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.79000000000000004},"min":{"1min":0.309,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71799999999999997,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52400000000000002,"15min":0.61599999999999999},"min":{"1min":0.32300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.69999999999999996,"5min":0.82099999999999995,"15min":4.1959999999999997},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.95099999999999996},"min":{"1min":0.379,"5min":0.317,"15min":0.317},"max":{"1min":0.78300000000000003,"5min":0.78800000000000003,"15min":10.1},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55200000000000005,"15min":0.71399999999999997},"min":{"1min":0.36699999999999999,"5min":0.32300000000000001,"15min":0.28999999999999998},"max":{"1min":0.73199999999999998,"5min":0.90400000000000003,"15min":6.806},"last":0.46999999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.76400000000000001},"min":{"1min":0.40100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79200000000000004,"5min":0.89000000000000001,"15min":6.9340000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.71699999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.73299999999999998,"5min":0.83899999999999997,"15min":6.9870000000000001},"last":0.54000000000000004}]},{"osd":3,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54900000000000004,"15min":0.73599999999999999},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.753,"5min":0.80200000000000005,"15min":6.6719999999999997},"last":0.504},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56699999999999995,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.71699999999999997,"5min":0.87,"15min":4.1609999999999996},"last":0.52200000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.36599999999999999,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.65100000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36299999999999999,"5min":0.309,"15min":0.30399999999999999},"max":{"1min":0.72299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.40999999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.314},"max":{"1min":0.77900000000000003,"5min":0.86099999999999999,"15min":1.3180000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.41799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71499999999999997,"5min":0.75800000000000001,"15min":0.84099999999999997},"last":0.38700000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":0.78700000000000003,"15min":1.7509999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.40899999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.85699999999999998,"15min":1.7090000000000001},"last":0.59399999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574949,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303280,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068560,"statfs":{"total":96636764160,"available":96326205440,"internally_reserved":0,"allocated":1941504,"data_stored":868876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51300000000000001,"15min":0.65500000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":7.7590000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.52100000000000002,"15min":0.69199999999999995},"min":{"1min":0.32200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.075,"5min":1.075,"15min":7.8090000000000002},"last":0.62}]},{"osd":1,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.69099999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.70199999999999996,"5min":0.80000000000000004,"15min":7.867},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.85599999999999998},"min":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.65600000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.58799999999999997},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.56100000000000005,"15min":0.81999999999999995},"min":{"1min":0.34200000000000003,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":2.641,"5min":2.641,"15min":6.7629999999999999},"last":0.58799999999999997}]},{"osd":3,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54300000000000004,"15min":0.61099999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.107,"5min":1.107,"15min":4.8620000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.70099999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.718,"5min":1.718,"15min":7.9009999999999998},"last":0.54100000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.47299999999999998},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.45500000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52700000000000002,"15min":0.495},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.376},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.53400000000000003},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.35299999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55800000000000005,"15min":0.52500000000000002},"min":{"1min":0.36099999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.372,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.433}]}]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303284,"kb_used_data":1900,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068556,"statfs":{"total":96636764160,"available":96326201344,"internally_reserved":0,"allocated":1945600,"data_stored":868378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.312,"15min":0.45900000000000002},"min":{"1min":0.22800000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.437,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":1.194},"last":0.29999999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.24199999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51000000000000001,"5min":0.51400000000000001,"15min":1.2869999999999999},"last":0.246},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.48999999999999999},"min":{"1min":0.27200000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.52800000000000002,"5min":0.54200000000000004,"15min":5.0919999999999996},"last":0.35599999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36399999999999999,"15min":0.51200000000000001},"min":{"1min":0.29399999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35699999999999998,"15min":0.40200000000000002},"min":{"1min":0.23799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":1.2170000000000001},"last":0.32600000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.73999999999999999,"5min":0.92400000000000004,"15min":1.254},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.68999999999999995},"min":{"1min":0.39300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68799999999999994,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.53300000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.36099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":0.97599999999999998,"15min":1.1799999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56899999999999995,"15min":0.60299999999999998},"min":{"1min":0.41799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71399999999999997,"5min":0.878,"15min":1.3109999999999999},"last":0.57899999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.81799999999999995},"min":{"1min":0.46000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78900000000000003,"5min":0.995,"15min":5.0659999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80700000000000005,"5min":0.89800000000000002,"15min":1.171},"last":0.46400000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57999999999999996,"15min":0.61299999999999999},"min":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77300000000000002,"5min":1.004,"15min":1.214},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.70999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78200000000000003,"5min":0.90500000000000003,"15min":4.0499999999999998},"last":0.54500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574947,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303856,"kb_used_data":2472,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067984,"statfs":{"total":96636764160,"available":96325615616,"internally_reserved":0,"allocated":2531328,"data_stored":1457819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.316,"15min":0.50800000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":7.9249999999999998},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.48699999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":6.6299999999999999},"last":0.311}]},{"osd":2,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.33100000000000002,"15min":0.53100000000000003},"min":{"1min":0.23100000000000001,"5min":0.22,"15min":0.216},"max":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":8.2509999999999994},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.51100000000000001},"min":{"1min":0.253,"5min":0.222,"15min":0.222},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":6.7370000000000001},"last":0.34899999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.35399999999999998,"15min":0.53900000000000003},"min":{"1min":0.247,"5min":0.23100000000000001,"15min":0.21099999999999999},"max":{"1min":0.53800000000000003,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.59099999999999997},"min":{"1min":0.27700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":7.8929999999999998},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57099999999999995,"15min":0.70299999999999996},"min":{"1min":0.45500000000000002,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.82499999999999996,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57299999999999995,"15min":0.754},"min":{"1min":0.441,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.80900000000000005,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.51300000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56100000000000005,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71299999999999997,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.76100000000000001},"min":{"1min":0.48399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.48699999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61499999999999999,"15min":0.80000000000000004},"min":{"1min":0.52200000000000002,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.83299999999999996,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.62,"15min":0.749},"min":{"1min":0.47399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82599999999999996,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.502}]},{"osd":7,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.60099999999999998,"15min":0.73999999999999999},"min":{"1min":0.48399999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85399999999999998,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.76800000000000002},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.75700000000000001,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.60399999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574949,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":868322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.311,"15min":0.32500000000000001},"min":{"1min":0.214,"5min":0.189,"15min":0.189},"max":{"1min":0.52500000000000002,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.35099999999999998},"min":{"1min":0.219,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.746},"last":0.23000000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.69099999999999995},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52100000000000002,"5min":0.63,"15min":11.967000000000001},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.34899999999999998,"15min":0.46999999999999997},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48899999999999999,"5min":0.53400000000000003,"15min":4.1100000000000003},"last":0.309}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34699999999999998,"15min":0.53700000000000003},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.50800000000000001,"5min":0.51600000000000001,"15min":6.3040000000000003},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.74299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59299999999999997,"5min":0.65900000000000003,"15min":11.935},"last":0.29499999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.55700000000000005,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.53700000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.38,"15min":0.378},"max":{"1min":0.83699999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54200000000000004,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.96099999999999997},"last":0.53000000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69599999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.58599999999999997},"min":{"1min":0.435,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.052},"last":0.623},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58099999999999996,"15min":0.76300000000000001},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":6.2649999999999997},"last":0.55800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574949,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303288,"kb_used_data":1904,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068552,"statfs":{"total":96636764160,"available":96326197248,"internally_reserved":0,"allocated":1949696,"data_stored":868504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.315,"15min":0.315},"min":{"1min":0.223,"5min":0.21199999999999999,"15min":0.21099999999999999},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32100000000000001},"min":{"1min":0.23999999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48799999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.313}]},{"osd":1,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.34599999999999997,"15min":0.33300000000000002},"min":{"1min":0.25700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.379},"min":{"1min":0.29199999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.72099999999999997},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.36199999999999999,"15min":0.35299999999999998},"min":{"1min":0.25800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57599999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.37},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.28999999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.50900000000000001,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.32600000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.53400000000000003},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77200000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.745,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.79400000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57299999999999995,"15min":0.55200000000000005},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.35899999999999999},"max":{"1min":0.82399999999999995,"5min":1.347,"15min":1.347},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.42199999999999999,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.83499999999999996,"5min":1.01,"15min":1.01},"last":0.65700000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.48399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86299999999999999,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.81200000000000006,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.71799999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78700000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.83699999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.56699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574947,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303296,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068544,"statfs":{"total":96636764160,"available":96326189056,"internally_reserved":0,"allocated":1957888,"data_stored":869962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.32400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66600000000000004,"5min":0.74199999999999999,"15min":0.747},"last":0.376},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51700000000000002,"15min":0.74099999999999999},"min":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77600000000000002,"5min":0.78400000000000003,"15min":7.6719999999999997},"last":0.53100000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52000000000000002,"15min":0.748},"min":{"1min":0.32000000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.755,"15min":7.7160000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66100000000000003,"5min":0.76900000000000002,"15min":0.86799999999999999},"last":0.56899999999999995}]},{"osd":2,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.73899999999999999,"5min":0.79800000000000004,"15min":7.7839999999999998},"last":0.50900000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.39200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.49099999999999999},"min":{"1min":0.35199999999999998,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58699999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.85199999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58399999999999996,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.61199999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.41399999999999998,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.626}]}]},{"osd":5,"up_from":13,"seq":55834574949,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303280,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068560,"statfs":{"total":96636764160,"available":96326205440,"internally_reserved":0,"allocated":1941504,"data_stored":868876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.47599999999999998,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.66100000000000003,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.51800000000000002},"min":{"1min":0.33000000000000002,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.68899999999999995,"5min":0.76700000000000002,"15min":0.79300000000000004},"last":0.60599999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51300000000000001,"15min":0.628},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63400000000000001,"5min":0.75700000000000001,"15min":5.2679999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.62},"min":{"1min":0.40500000000000003,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79200000000000004,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.65200000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52400000000000002,"15min":0.63700000000000001},"min":{"1min":0.39100000000000001,"5min":0.32900000000000001,"15min":0.28299999999999997},"max":{"1min":0.73299999999999998,"5min":0.76600000000000001,"15min":5.2960000000000003},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.38500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.58299999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55900000000000005,"15min":0.66900000000000004},"min":{"1min":0.41999999999999998,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":5.3330000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56399999999999995,"15min":0.68000000000000005},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.27700000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.67000000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.48899999999999999,"15min":0.498},"min":{"1min":0.33400000000000002,"5min":0.29599999999999999,"15min":0.28699999999999998},"max":{"1min":0.67000000000000004,"5min":0.72499999999999998,"15min":0.82999999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.34499999999999997,"15min":0.32200000000000001},"max":{"1min":0.77200000000000002,"5min":0.88,"15min":1.0800000000000001},"last":0.38900000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.39900000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78900000000000003,"5min":0.82499999999999996,"15min":0.89800000000000002},"last":0.51500000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.32300000000000001,"15min":0.30099999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98399999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.44}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:42:28.905 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-10T20:42:29.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.050+0000 7f1b07fff700 1 Processor -- start 2023-10-10T20:42:29.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.050+0000 7f1b07fff700 1 -- start start 2023-10-10T20:42:29.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.051+0000 7f1b07fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b081593c0 0x7f1b080cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:29.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.051+0000 7f1b07fff700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b080cff70 con 0x7f1b080b1700 2023-10-10T20:42:29.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.051+0000 7f1b07fff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b080cf360 con 0x7f1b08159790 2023-10-10T20:42:29.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.051+0000 7f1b07fff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b080cf640 con 0x7f1b081593c0 2023-10-10T20:42:29.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.051+0000 7f1b077fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1b080b1700 0x7f1b0815aeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54050/0 (socket says 172.21.15.89:54050) 2023-10-10T20:42:29.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.051+0000 7f1b077fe700 1 -- 172.21.15.89:0/3532779119 learned_addr learned my addr 172.21.15.89:0/3532779119 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:29.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.051+0000 7f1b06ffd700 1 --2- 172.21.15.89:0/3532779119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b081593c0 0x7f1b080cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:29.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.051+0000 7f1b067fc700 1 -- 172.21.15.89:0/3532779119 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884796095 0 0) 0x7f1b080cff70 con 0x7f1b080b1700 2023-10-10T20:42:29.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.051+0000 7f1b067fc700 1 -- 172.21.15.89:0/3532779119 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1af4003570 con 0x7f1b080b1700 2023-10-10T20:42:29.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.051+0000 7f1b067fc700 1 -- 172.21.15.89:0/3532779119 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784898165 0 0) 0x7f1b080cf360 con 0x7f1b08159790 2023-10-10T20:42:29.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.051+0000 7f1b067fc700 1 -- 172.21.15.89:0/3532779119 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b080cff70 con 0x7f1b08159790 2023-10-10T20:42:29.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.051+0000 7f1b06ffd700 1 -- 172.21.15.89:0/3532779119 >> v1:172.21.15.111:6789/0 conn(0x7f1b08159790 legacy=0x7f1b0815a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:29.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.051+0000 7f1b06ffd700 1 -- 172.21.15.89:0/3532779119 >> v1:172.21.15.89:6789/0 conn(0x7f1b080b1700 legacy=0x7f1b0815aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:29.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.052+0000 7f1b06ffd700 1 -- 172.21.15.89:0/3532779119 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b080cf920 con 0x7f1b081593c0 2023-10-10T20:42:29.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.052+0000 7f1b06ffd700 1 --2- 172.21.15.89:0/3532779119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b081593c0 0x7f1b080cecc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1af8009910 tx=0x7f1af80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fec56334a6f0070 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.052+0000 7f1b067fc700 1 -- 172.21.15.89:0/3532779119 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af8011630 con 0x7f1b081593c0 2023-10-10T20:42:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.052+0000 7f1b067fc700 1 -- 172.21.15.89:0/3532779119 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af8011790 con 0x7f1b081593c0 2023-10-10T20:42:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.052+0000 7f1b067fc700 1 -- 172.21.15.89:0/3532779119 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af8011b10 con 0x7f1b081593c0 2023-10-10T20:42:29.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.052+0000 7f1b07fff700 1 -- 172.21.15.89:0/3532779119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b081593c0 msgr2=0x7f1b080cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:29.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.052+0000 7f1b07fff700 1 --2- 172.21.15.89:0/3532779119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b081593c0 0x7f1b080cecc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1af8009910 tx=0x7f1af80091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.052+0000 7f1b07fff700 1 -- 172.21.15.89:0/3532779119 shutdown_connections 2023-10-10T20:42:29.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.052+0000 7f1b07fff700 1 --2- 172.21.15.89:0/3532779119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b081593c0 0x7f1b080cecc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.052+0000 7f1b07fff700 1 -- 172.21.15.89:0/3532779119 >> 172.21.15.89:0/3532779119 conn(0x7f1b080d1080 msgr2=0x7f1b080affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:29.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.052+0000 7f1b07fff700 1 -- 172.21.15.89:0/3532779119 shutdown_connections 2023-10-10T20:42:29.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.053+0000 7f1b07fff700 1 -- 172.21.15.89:0/3532779119 wait complete. 2023-10-10T20:42:29.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.054+0000 7f1b07fff700 1 Processor -- start 2023-10-10T20:42:29.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.054+0000 7f1b07fff700 1 -- start start 2023-10-10T20:42:29.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.054+0000 7f1b07fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b080b1700 0x7f1b080c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:29.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.054+0000 7f1b07fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b081593c0 0x7f1b0815af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:29.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.054+0000 7f1b07fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b08159790 0x7f1b0816ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:29.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.054+0000 7f1b07fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0815b950 con 0x7f1b08159790 2023-10-10T20:42:29.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.054+0000 7f1b07fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0815bc30 con 0x7f1b080b1700 2023-10-10T20:42:29.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.054+0000 7f1b07fff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0815bf10 con 0x7f1b081593c0 2023-10-10T20:42:29.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.055+0000 7f1b077fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b08159790 0x7f1b0816ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:29.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.055+0000 7f1b077fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b08159790 0x7f1b0816ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59128/0 (socket says 172.21.15.89:59128) 2023-10-10T20:42:29.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.055+0000 7f1b077fe700 1 -- 172.21.15.89:0/834548151 learned_addr learned my addr 172.21.15.89:0/834548151 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:29.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.055+0000 7f1afe7fc700 1 --2- 172.21.15.89:0/834548151 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b081593c0 0x7f1b0815af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:29.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.055+0000 7f1b06ffd700 1 --2- 172.21.15.89:0/834548151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b080b1700 0x7f1b080c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:29.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.055+0000 7f1b077fe700 1 -- 172.21.15.89:0/834548151 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b081593c0 msgr2=0x7f1b0815af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:29.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.055+0000 7f1b077fe700 1 --2- 172.21.15.89:0/834548151 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b081593c0 0x7f1b0815af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.055+0000 7f1b077fe700 1 -- 172.21.15.89:0/834548151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b080b1700 msgr2=0x7f1b080c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:29.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.055+0000 7f1b077fe700 1 --2- 172.21.15.89:0/834548151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b080b1700 0x7f1b080c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.055+0000 7f1b077fe700 1 -- 172.21.15.89:0/834548151 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af8009420 con 0x7f1b08159790 2023-10-10T20:42:29.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.055+0000 7f1b077fe700 1 --2- 172.21.15.89:0/834548151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b08159790 0x7f1b0816ca20 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f1b0000cd00 tx=0x7f1b00001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.055+0000 7f1aff7fe700 1 -- 172.21.15.89:0/834548151 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0000ce00 con 0x7f1b08159790 2023-10-10T20:42:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.055+0000 7f1aff7fe700 1 -- 172.21.15.89:0/834548151 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0000aac0 con 0x7f1b08159790 2023-10-10T20:42:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.055+0000 7f1b07fff700 1 -- 172.21.15.89:0/834548151 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b0815c1f0 con 0x7f1b08159790 2023-10-10T20:42:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.055+0000 7f1aff7fe700 1 -- 172.21.15.89:0/834548151 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0000ae80 con 0x7f1b08159790 2023-10-10T20:42:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.055+0000 7f1b07fff700 1 -- 172.21.15.89:0/834548151 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b080c5f60 con 0x7f1b08159790 2023-10-10T20:42:29.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.056+0000 7f1b07fff700 1 -- 172.21.15.89:0/834548151 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b08088c00 con 0x7f1b08159790 2023-10-10T20:42:29.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.057+0000 7f1aff7fe700 1 -- 172.21.15.89:0/834548151 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1b00007300 con 0x7f1b08159790 2023-10-10T20:42:29.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.057+0000 7f1aff7fe700 1 --2- 172.21.15.89:0/834548151 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1ae8067320 0x7f1ae80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:29.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.057+0000 7f1aff7fe700 1 -- 172.21.15.89:0/834548151 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1b00084f70 con 0x7f1b08159790 2023-10-10T20:42:29.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.057+0000 7f1b06ffd700 1 --2- 172.21.15.89:0/834548151 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1ae8067320 0x7f1ae80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.057+0000 7f1b06ffd700 1 --2- 172.21.15.89:0/834548151 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1ae8067320 0x7f1ae80697d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1af80091e0 tx=0x7f1af8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.061+0000 7f1aff7fe700 1 -- 172.21.15.89:0/834548151 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b00051c80 con 0x7f1b08159790 2023-10-10T20:42:29.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.321+0000 7f1b07fff700 1 -- 172.21.15.89:0/834548151 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1b080cfc00 con 0x7f1b08159790 2023-10-10T20:42:29.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.322+0000 7f1aff7fe700 1 -- 172.21.15.89:0/834548151 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1438 (secure 0 0 0) 0x7f1b000552a0 con 0x7f1b08159790 2023-10-10T20:42:29.324 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:29.325 INFO:teuthology.orchestra.run.smithi089.stdout:{"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":492,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":153,"num_osds":8,"num_up_osds":8,"osd_up_since":1696970067,"num_in_osds":8,"osd_in_since":1696970059,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592674,"bytes_used":2485686272,"bytes_avail":770608427008,"bytes_total":773094113280,"write_bytes_sec":2218,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":101,"id":13,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":13,"rank":0,"name":"c","status":"up:active","gid":7959}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2023-10-10T20:41:32.735512+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-10T20:42:29.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.324+0000 7f1b07fff700 1 -- 172.21.15.89:0/834548151 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1ae8067320 msgr2=0x7f1ae80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:29.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.324+0000 7f1b07fff700 1 --2- 172.21.15.89:0/834548151 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1ae8067320 0x7f1ae80697d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1af80091e0 tx=0x7f1af8017000 comp rx=0 tx=0).stop 2023-10-10T20:42:29.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.324+0000 7f1b07fff700 1 -- 172.21.15.89:0/834548151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b08159790 msgr2=0x7f1b0816ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:29.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.324+0000 7f1b07fff700 1 --2- 172.21.15.89:0/834548151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b08159790 0x7f1b0816ca20 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f1b0000cd00 tx=0x7f1b00001ac0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.324+0000 7f1b07fff700 1 -- 172.21.15.89:0/834548151 shutdown_connections 2023-10-10T20:42:29.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.324+0000 7f1b07fff700 1 --2- 172.21.15.89:0/834548151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b08159790 0x7f1b0816ca20 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.324+0000 7f1b07fff700 1 --2- 172.21.15.89:0/834548151 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1ae8067320 0x7f1ae80697d0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.324+0000 7f1b07fff700 1 --2- 172.21.15.89:0/834548151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b080b1700 0x7f1b080c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.324+0000 7f1b07fff700 1 --2- 172.21.15.89:0/834548151 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b081593c0 0x7f1b0815af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.324+0000 7f1b07fff700 1 -- 172.21.15.89:0/834548151 >> 172.21.15.89:0/834548151 conn(0x7f1b080d1080 msgr2=0x7f1b080b0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:29.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.324+0000 7f1b07fff700 1 -- 172.21.15.89:0/834548151 shutdown_connections 2023-10-10T20:42:29.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.324+0000 7f1b07fff700 1 -- 172.21.15.89:0/834548151 wait complete. 2023-10-10T20:42:29.338 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-10T20:42:29.338 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:42:29.387 INFO:tasks.ceph.mds.c.smithi089.stderr:2023-10-10T20:42:29.385+0000 7f0e85cdc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:42:29.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.450+0000 7fa776f99700 1 Processor -- start 2023-10-10T20:42:29.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.450+0000 7fa776f99700 1 -- start start 2023-10-10T20:42:29.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.451+0000 7fa776f99700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa770159890 0x7fa77015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:29.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.451+0000 7fa776f99700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa770069560 0x7fa7700699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:29.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.451+0000 7fa776f99700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa770069ee0 con 0x7fa770159c60 2023-10-10T20:42:29.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.451+0000 7fa776f99700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77015a160 con 0x7fa770159890 2023-10-10T20:42:29.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.451+0000 7fa776f99700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700d13a0 con 0x7fa770069560 2023-10-10T20:42:29.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.451+0000 7fa774d35700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa770159890 0x7fa77015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:29.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.451+0000 7fa767fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa770159c60 0x7fa77015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54058/0 (socket says 172.21.15.89:54058) 2023-10-10T20:42:29.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.451+0000 7fa774d35700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa770159890 0x7fa77015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59136/0 (socket says 172.21.15.89:59136) 2023-10-10T20:42:29.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.451+0000 7fa767fff700 1 -- 172.21.15.89:0/477045262 learned_addr learned my addr 172.21.15.89:0/477045262 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:29.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.452+0000 7fa775536700 1 --2- 172.21.15.89:0/477045262 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa770069560 0x7fa7700699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:29.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.452+0000 7fa774d35700 1 -- 172.21.15.89:0/477045262 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa770069560 msgr2=0x7fa7700699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:29.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.452+0000 7fa774d35700 1 --2- 172.21.15.89:0/477045262 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa770069560 0x7fa7700699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.452+0000 7fa774d35700 1 -- 172.21.15.89:0/477045262 >> v1:172.21.15.89:6789/0 conn(0x7fa770159c60 legacy=0x7fa77015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:29.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.452+0000 7fa7677fe700 1 -- 172.21.15.89:0/477045262 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179401639 0 0) 0x7fa770069ee0 con 0x7fa770159c60 2023-10-10T20:42:29.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.452+0000 7fa774d35700 1 -- 172.21.15.89:0/477045262 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77015e140 con 0x7fa770159890 2023-10-10T20:42:29.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.452+0000 7fa774d35700 1 --2- 172.21.15.89:0/477045262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa770159890 0x7fa77015abd0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0098e0 tx=0x7fa76c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d6d6e33612485b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:29.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.452+0000 7fa7677fe700 1 -- 172.21.15.89:0/477045262 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c009e80 con 0x7fa770159890 2023-10-10T20:42:29.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.452+0000 7fa7677fe700 1 -- 172.21.15.89:0/477045262 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76c00b860 con 0x7fa770159890 2023-10-10T20:42:29.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.452+0000 7fa7677fe700 1 -- 172.21.15.89:0/477045262 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c00bbe0 con 0x7fa770159890 2023-10-10T20:42:29.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.452+0000 7fa776f99700 1 -- 172.21.15.89:0/477045262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa770159890 msgr2=0x7fa77015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:29.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.452+0000 7fa776f99700 1 --2- 172.21.15.89:0/477045262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa770159890 0x7fa77015abd0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0098e0 tx=0x7fa76c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.452+0000 7fa776f99700 1 -- 172.21.15.89:0/477045262 shutdown_connections 2023-10-10T20:42:29.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.452+0000 7fa776f99700 1 --2- 172.21.15.89:0/477045262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa770159890 0x7fa77015abd0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.452+0000 7fa776f99700 1 --2- 172.21.15.89:0/477045262 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa770069560 0x7fa7700699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.452+0000 7fa776f99700 1 -- 172.21.15.89:0/477045262 >> 172.21.15.89:0/477045262 conn(0x7fa770068e20 msgr2=0x7fa7700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:29.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.453+0000 7fa776f99700 1 -- 172.21.15.89:0/477045262 shutdown_connections 2023-10-10T20:42:29.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.453+0000 7fa776f99700 1 -- 172.21.15.89:0/477045262 wait complete. 2023-10-10T20:42:29.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.454+0000 7fa776f99700 1 Processor -- start 2023-10-10T20:42:29.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.454+0000 7fa776f99700 1 -- start start 2023-10-10T20:42:29.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.454+0000 7fa776f99700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa770069560 0x7fa77014ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:29.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.454+0000 7fa776f99700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa770159890 0x7fa77016d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:29.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.454+0000 7fa776f99700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa770159c60 0x7fa7700cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:29.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.454+0000 7fa776f99700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77016dee0 con 0x7fa770159890 2023-10-10T20:42:29.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.454+0000 7fa776f99700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700b3730 con 0x7fa770069560 2023-10-10T20:42:29.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.454+0000 7fa776f99700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700b3a10 con 0x7fa770159c60 2023-10-10T20:42:29.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.455+0000 7fa767fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa770159890 0x7fa77016d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:29.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.455+0000 7fa767fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa770159890 0x7fa77016d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59152/0 (socket says 172.21.15.89:59152) 2023-10-10T20:42:29.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.455+0000 7fa767fff700 1 -- 172.21.15.89:0/2971931403 learned_addr learned my addr 172.21.15.89:0/2971931403 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:29.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.455+0000 7fa775536700 1 --2- 172.21.15.89:0/2971931403 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa770159c60 0x7fa7700cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:29.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.455+0000 7fa774d35700 1 --2- 172.21.15.89:0/2971931403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa770069560 0x7fa77014ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:29.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.455+0000 7fa767fff700 1 -- 172.21.15.89:0/2971931403 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa770159c60 msgr2=0x7fa7700cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:29.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.455+0000 7fa767fff700 1 --2- 172.21.15.89:0/2971931403 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa770159c60 0x7fa7700cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.455+0000 7fa767fff700 1 -- 172.21.15.89:0/2971931403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa770069560 msgr2=0x7fa77014ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:29.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.455+0000 7fa767fff700 1 --2- 172.21.15.89:0/2971931403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa770069560 0x7fa77014ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.455+0000 7fa767fff700 1 -- 172.21.15.89:0/2971931403 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76c009420 con 0x7fa770159890 2023-10-10T20:42:29.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.455+0000 7fa775536700 1 --2- 172.21.15.89:0/2971931403 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa770159c60 0x7fa7700cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:29.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.455+0000 7fa767fff700 1 --2- 172.21.15.89:0/2971931403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa770159890 0x7fa77016d9a0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fa76000c7e0 tx=0x7fa760007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:29.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.455+0000 7fa764ff9700 1 -- 172.21.15.89:0/2971931403 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760005b30 con 0x7fa770159890 2023-10-10T20:42:29.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.455+0000 7fa764ff9700 1 -- 172.21.15.89:0/2971931403 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa760005c90 con 0x7fa770159890 2023-10-10T20:42:29.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.455+0000 7fa776f99700 1 -- 172.21.15.89:0/2971931403 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7700cd1e0 con 0x7fa770159890 2023-10-10T20:42:29.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.455+0000 7fa764ff9700 1 -- 172.21.15.89:0/2971931403 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760007300 con 0x7fa770159890 2023-10-10T20:42:29.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.455+0000 7fa776f99700 1 -- 172.21.15.89:0/2971931403 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7700cfb90 con 0x7fa770159890 2023-10-10T20:42:29.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.456+0000 7fa776f99700 1 -- 172.21.15.89:0/2971931403 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7700cd680 con 0x7fa770159890 2023-10-10T20:42:29.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.456+0000 7fa764ff9700 1 -- 172.21.15.89:0/2971931403 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa76001b400 con 0x7fa770159890 2023-10-10T20:42:29.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.457+0000 7fa764ff9700 1 --2- 172.21.15.89:0/2971931403 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa75c067320 0x7fa75c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:29.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.457+0000 7fa764ff9700 1 -- 172.21.15.89:0/2971931403 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa7600847e0 con 0x7fa770159890 2023-10-10T20:42:29.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.457+0000 7fa774d35700 1 --2- 172.21.15.89:0/2971931403 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa75c067320 0x7fa75c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:29.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.457+0000 7fa774d35700 1 --2- 172.21.15.89:0/2971931403 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa75c067320 0x7fa75c0697d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0091e0 tx=0x7fa76c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:29.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.461+0000 7fa764ff9700 1 -- 172.21.15.89:0/2971931403 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7600514f0 con 0x7fa770159890 2023-10-10T20:42:29.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.640+0000 7fa776f99700 1 -- 172.21.15.89:0/2971931403 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa77004c3c0 con 0x7fa75c067320 2023-10-10T20:42:29.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.648+0000 7fa764ff9700 1 -- 172.21.15.89:0/2971931403 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290160 (secure 0 0 0) 0x7fa77004c3c0 con 0x7fa75c067320 2023-10-10T20:42:29.653 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:29.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.651+0000 7fa776f99700 1 -- 172.21.15.89:0/2971931403 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa75c067320 msgr2=0x7fa75c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:29.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.651+0000 7fa776f99700 1 --2- 172.21.15.89:0/2971931403 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa75c067320 0x7fa75c0697d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0091e0 tx=0x7fa76c017000 comp rx=0 tx=0).stop 2023-10-10T20:42:29.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.651+0000 7fa776f99700 1 -- 172.21.15.89:0/2971931403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa770159890 msgr2=0x7fa77016d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:29.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.651+0000 7fa776f99700 1 --2- 172.21.15.89:0/2971931403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa770159890 0x7fa77016d9a0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fa76000c7e0 tx=0x7fa760007df0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.651+0000 7fa776f99700 1 -- 172.21.15.89:0/2971931403 shutdown_connections 2023-10-10T20:42:29.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.651+0000 7fa776f99700 1 --2- 172.21.15.89:0/2971931403 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa770159c60 0x7fa7700cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.651+0000 7fa776f99700 1 --2- 172.21.15.89:0/2971931403 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa75c067320 0x7fa75c0697d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.651+0000 7fa776f99700 1 --2- 172.21.15.89:0/2971931403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa770069560 0x7fa77014ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.651+0000 7fa776f99700 1 --2- 172.21.15.89:0/2971931403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa770159890 0x7fa77016d9a0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:29.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.651+0000 7fa776f99700 1 -- 172.21.15.89:0/2971931403 >> 172.21.15.89:0/2971931403 conn(0x7fa770068e20 msgr2=0x7fa7700b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:29.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.651+0000 7fa776f99700 1 -- 172.21.15.89:0/2971931403 shutdown_connections 2023-10-10T20:42:29.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:29.651+0000 7fa776f99700 1 -- 172.21.15.89:0/2971931403 wait complete. 2023-10-10T20:42:29.658 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:42:29.665 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2023-10-10T20:42:28.745808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427428,"kb_used_data":16356,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547292,"statfs":{"total":773094113280,"available":770608427008,"internally_reserved":0,"allocated":16748544,"data_stored":8130963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001060"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121111+0000","last_change":"2023-10-10T20:42:24.113315+0000","last_active":"2023-10-10T20:42:24.121111+0000","last_peered":"2023-10-10T20:42:24.121111+0000","last_clean":"2023-10-10T20:42:24.121111+0000","last_became_active":"2023-10-10T20:42:24.113091+0000","last_became_peered":"2023-10-10T20:42:24.113091+0000","last_unstale":"2023-10-10T20:42:24.121111+0000","last_undegraded":"2023-10-10T20:42:24.121111+0000","last_fullsized":"2023-10-10T20:42:24.121111+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:25:55.562871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114620+0000","last_change":"2023-10-10T20:42:24.111977+0000","last_active":"2023-10-10T20:42:24.114620+0000","last_peered":"2023-10-10T20:42:24.114620+0000","last_clean":"2023-10-10T20:42:24.114620+0000","last_became_active":"2023-10-10T20:42:24.111740+0000","last_became_peered":"2023-10-10T20:42:24.111740+0000","last_unstale":"2023-10-10T20:42:24.114620+0000","last_undegraded":"2023-10-10T20:42:24.114620+0000","last_fullsized":"2023-10-10T20:42:24.114620+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:43:44.092334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.107328+0000","last_change":"2023-10-10T20:42:24.101396+0000","last_active":"2023-10-10T20:42:24.107328+0000","last_peered":"2023-10-10T20:42:24.107328+0000","last_clean":"2023-10-10T20:42:24.107328+0000","last_became_active":"2023-10-10T20:42:24.101140+0000","last_became_peered":"2023-10-10T20:42:24.101140+0000","last_unstale":"2023-10-10T20:42:24.107328+0000","last_undegraded":"2023-10-10T20:42:24.107328+0000","last_fullsized":"2023-10-10T20:42:24.107328+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:22:58.559364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113453+0000","last_change":"2023-10-10T20:42:24.107229+0000","last_active":"2023-10-10T20:42:24.113453+0000","last_peered":"2023-10-10T20:42:24.113453+0000","last_clean":"2023-10-10T20:42:24.113453+0000","last_became_active":"2023-10-10T20:42:24.106160+0000","last_became_peered":"2023-10-10T20:42:24.106160+0000","last_unstale":"2023-10-10T20:42:24.113453+0000","last_undegraded":"2023-10-10T20:42:24.113453+0000","last_fullsized":"2023-10-10T20:42:24.113453+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:04:22.281292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114347+0000","last_change":"2023-10-10T20:42:24.112159+0000","last_active":"2023-10-10T20:42:24.114347+0000","last_peered":"2023-10-10T20:42:24.114347+0000","last_clean":"2023-10-10T20:42:24.114347+0000","last_became_active":"2023-10-10T20:42:24.111823+0000","last_became_peered":"2023-10-10T20:42:24.111823+0000","last_unstale":"2023-10-10T20:42:24.114347+0000","last_undegraded":"2023-10-10T20:42:24.114347+0000","last_fullsized":"2023-10-10T20:42:24.114347+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:37:14.508787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.117428+0000","last_change":"2023-10-10T20:42:24.114360+0000","last_active":"2023-10-10T20:42:24.117428+0000","last_peered":"2023-10-10T20:42:24.117428+0000","last_clean":"2023-10-10T20:42:24.117428+0000","last_became_active":"2023-10-10T20:42:24.114175+0000","last_became_peered":"2023-10-10T20:42:24.114175+0000","last_unstale":"2023-10-10T20:42:24.117428+0000","last_undegraded":"2023-10-10T20:42:24.117428+0000","last_fullsized":"2023-10-10T20:42:24.117428+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:39:43.171315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121416+0000","last_change":"2023-10-10T20:42:24.116618+0000","last_active":"2023-10-10T20:42:24.121416+0000","last_peered":"2023-10-10T20:42:24.121416+0000","last_clean":"2023-10-10T20:42:24.121416+0000","last_became_active":"2023-10-10T20:42:24.115742+0000","last_became_peered":"2023-10-10T20:42:24.115742+0000","last_unstale":"2023-10-10T20:42:24.121416+0000","last_undegraded":"2023-10-10T20:42:24.121416+0000","last_fullsized":"2023-10-10T20:42:24.121416+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:49:38.681149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114418+0000","last_change":"2023-10-10T20:42:24.112723+0000","last_active":"2023-10-10T20:42:24.114418+0000","last_peered":"2023-10-10T20:42:24.114418+0000","last_clean":"2023-10-10T20:42:24.114418+0000","last_became_active":"2023-10-10T20:42:24.112506+0000","last_became_peered":"2023-10-10T20:42:24.112506+0000","last_unstale":"2023-10-10T20:42:24.114418+0000","last_undegraded":"2023-10-10T20:42:24.114418+0000","last_fullsized":"2023-10-10T20:42:24.114418+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:46:59.225281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.107890+0000","last_change":"2023-10-10T20:42:24.101774+0000","last_active":"2023-10-10T20:42:24.107890+0000","last_peered":"2023-10-10T20:42:24.107890+0000","last_clean":"2023-10-10T20:42:24.107890+0000","last_became_active":"2023-10-10T20:42:24.101500+0000","last_became_peered":"2023-10-10T20:42:24.101500+0000","last_unstale":"2023-10-10T20:42:24.107890+0000","last_undegraded":"2023-10-10T20:42:24.107890+0000","last_fullsized":"2023-10-10T20:42:24.107890+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:32:22.661032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121119+0000","last_change":"2023-10-10T20:42:24.117698+0000","last_active":"2023-10-10T20:42:24.121119+0000","last_peered":"2023-10-10T20:42:24.121119+0000","last_clean":"2023-10-10T20:42:24.121119+0000","last_became_active":"2023-10-10T20:42:24.117453+0000","last_became_peered":"2023-10-10T20:42:24.117453+0000","last_unstale":"2023-10-10T20:42:24.121119+0000","last_undegraded":"2023-10-10T20:42:24.121119+0000","last_fullsized":"2023-10-10T20:42:24.121119+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:14.819673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.112709+0000","last_change":"2023-10-10T20:42:24.108830+0000","last_active":"2023-10-10T20:42:24.112709+0000","last_peered":"2023-10-10T20:42:24.112709+0000","last_clean":"2023-10-10T20:42:24.112709+0000","last_became_active":"2023-10-10T20:42:24.108592+0000","last_became_peered":"2023-10-10T20:42:24.108592+0000","last_unstale":"2023-10-10T20:42:24.112709+0000","last_undegraded":"2023-10-10T20:42:24.112709+0000","last_fullsized":"2023-10-10T20:42:24.112709+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:32:50.550864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.128843+0000","last_change":"2023-10-10T20:42:24.124278+0000","last_active":"2023-10-10T20:42:24.128843+0000","last_peered":"2023-10-10T20:42:24.128843+0000","last_clean":"2023-10-10T20:42:24.128843+0000","last_became_active":"2023-10-10T20:42:24.123491+0000","last_became_peered":"2023-10-10T20:42:24.123491+0000","last_unstale":"2023-10-10T20:42:24.128843+0000","last_undegraded":"2023-10-10T20:42:24.128843+0000","last_fullsized":"2023-10-10T20:42:24.128843+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:47:26.708886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.128950+0000","last_change":"2023-10-10T20:42:24.124820+0000","last_active":"2023-10-10T20:42:24.128950+0000","last_peered":"2023-10-10T20:42:24.128950+0000","last_clean":"2023-10-10T20:42:24.128950+0000","last_became_active":"2023-10-10T20:42:24.124570+0000","last_became_peered":"2023-10-10T20:42:24.124570+0000","last_unstale":"2023-10-10T20:42:24.128950+0000","last_undegraded":"2023-10-10T20:42:24.128950+0000","last_fullsized":"2023-10-10T20:42:24.128950+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:21:59.491769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.112675+0000","last_change":"2023-10-10T20:42:24.110144+0000","last_active":"2023-10-10T20:42:24.112675+0000","last_peered":"2023-10-10T20:42:24.112675+0000","last_clean":"2023-10-10T20:42:24.112675+0000","last_became_active":"2023-10-10T20:42:24.109876+0000","last_became_peered":"2023-10-10T20:42:24.109876+0000","last_unstale":"2023-10-10T20:42:24.112675+0000","last_undegraded":"2023-10-10T20:42:24.112675+0000","last_fullsized":"2023-10-10T20:42:24.112675+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:40:58.862440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.115631+0000","last_change":"2023-10-10T20:42:24.112507+0000","last_active":"2023-10-10T20:42:24.115631+0000","last_peered":"2023-10-10T20:42:24.115631+0000","last_clean":"2023-10-10T20:42:24.115631+0000","last_became_active":"2023-10-10T20:42:24.112275+0000","last_became_peered":"2023-10-10T20:42:24.112275+0000","last_unstale":"2023-10-10T20:42:24.115631+0000","last_undegraded":"2023-10-10T20:42:24.115631+0000","last_fullsized":"2023-10-10T20:42:24.115631+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:06.445923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126524+0000","last_change":"2023-10-10T20:42:24.125813+0000","last_active":"2023-10-10T20:42:24.126524+0000","last_peered":"2023-10-10T20:42:24.126524+0000","last_clean":"2023-10-10T20:42:24.126524+0000","last_became_active":"2023-10-10T20:42:24.125554+0000","last_became_peered":"2023-10-10T20:42:24.125554+0000","last_unstale":"2023-10-10T20:42:24.126524+0000","last_undegraded":"2023-10-10T20:42:24.126524+0000","last_fullsized":"2023-10-10T20:42:24.126524+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:40:58.862440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.117282+0000","last_change":"2023-10-10T20:42:24.113162+0000","last_active":"2023-10-10T20:42:24.117282+0000","last_peered":"2023-10-10T20:42:24.117282+0000","last_clean":"2023-10-10T20:42:24.117282+0000","last_became_active":"2023-10-10T20:42:24.112919+0000","last_became_peered":"2023-10-10T20:42:24.112919+0000","last_unstale":"2023-10-10T20:42:24.117282+0000","last_undegraded":"2023-10-10T20:42:24.117282+0000","last_fullsized":"2023-10-10T20:42:24.117282+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:58:46.317589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.118301+0000","last_change":"2023-10-10T20:42:24.111610+0000","last_active":"2023-10-10T20:42:24.118301+0000","last_peered":"2023-10-10T20:42:24.118301+0000","last_clean":"2023-10-10T20:42:24.118301+0000","last_became_active":"2023-10-10T20:42:24.111317+0000","last_became_peered":"2023-10-10T20:42:24.111317+0000","last_unstale":"2023-10-10T20:42:24.118301+0000","last_undegraded":"2023-10-10T20:42:24.118301+0000","last_fullsized":"2023-10-10T20:42:24.118301+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:49:38.681149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.118411+0000","last_change":"2023-10-10T20:42:24.111047+0000","last_active":"2023-10-10T20:42:24.118411+0000","last_peered":"2023-10-10T20:42:24.118411+0000","last_clean":"2023-10-10T20:42:24.118411+0000","last_became_active":"2023-10-10T20:42:24.110810+0000","last_became_peered":"2023-10-10T20:42:24.110810+0000","last_unstale":"2023-10-10T20:42:24.118411+0000","last_undegraded":"2023-10-10T20:42:24.118411+0000","last_fullsized":"2023-10-10T20:42:24.118411+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:47.040763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113486+0000","last_change":"2023-10-10T20:42:24.109452+0000","last_active":"2023-10-10T20:42:24.113486+0000","last_peered":"2023-10-10T20:42:24.113486+0000","last_clean":"2023-10-10T20:42:24.113486+0000","last_became_active":"2023-10-10T20:42:24.109179+0000","last_became_peered":"2023-10-10T20:42:24.109179+0000","last_unstale":"2023-10-10T20:42:24.113486+0000","last_undegraded":"2023-10-10T20:42:24.113486+0000","last_fullsized":"2023-10-10T20:42:24.113486+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:59.819150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.112215+0000","last_change":"2023-10-10T20:42:24.105755+0000","last_active":"2023-10-10T20:42:24.112215+0000","last_peered":"2023-10-10T20:42:24.112215+0000","last_clean":"2023-10-10T20:42:24.112215+0000","last_became_active":"2023-10-10T20:42:24.104420+0000","last_became_peered":"2023-10-10T20:42:24.104420+0000","last_unstale":"2023-10-10T20:42:24.112215+0000","last_undegraded":"2023-10-10T20:42:24.112215+0000","last_fullsized":"2023-10-10T20:42:24.112215+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:03:33.363174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113655+0000","last_change":"2023-10-10T20:42:24.110180+0000","last_active":"2023-10-10T20:42:24.113655+0000","last_peered":"2023-10-10T20:42:24.113655+0000","last_clean":"2023-10-10T20:42:24.113655+0000","last_became_active":"2023-10-10T20:42:24.109893+0000","last_became_peered":"2023-10-10T20:42:24.109893+0000","last_unstale":"2023-10-10T20:42:24.113655+0000","last_undegraded":"2023-10-10T20:42:24.113655+0000","last_fullsized":"2023-10-10T20:42:24.113655+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:09:32.731249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.117221+0000","last_change":"2023-10-10T20:42:24.116069+0000","last_active":"2023-10-10T20:42:24.117221+0000","last_peered":"2023-10-10T20:42:24.117221+0000","last_clean":"2023-10-10T20:42:24.117221+0000","last_became_active":"2023-10-10T20:42:24.115782+0000","last_became_peered":"2023-10-10T20:42:24.115782+0000","last_unstale":"2023-10-10T20:42:24.117221+0000","last_undegraded":"2023-10-10T20:42:24.117221+0000","last_fullsized":"2023-10-10T20:42:24.117221+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:14:18.635698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.107687+0000","last_change":"2023-10-10T20:42:24.102224+0000","last_active":"2023-10-10T20:42:24.107687+0000","last_peered":"2023-10-10T20:42:24.107687+0000","last_clean":"2023-10-10T20:42:24.107687+0000","last_became_active":"2023-10-10T20:42:24.101963+0000","last_became_peered":"2023-10-10T20:42:24.101963+0000","last_unstale":"2023-10-10T20:42:24.107687+0000","last_undegraded":"2023-10-10T20:42:24.107687+0000","last_fullsized":"2023-10-10T20:42:24.107687+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:38:54.118178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113541+0000","last_change":"2023-10-10T20:42:24.109079+0000","last_active":"2023-10-10T20:42:24.113541+0000","last_peered":"2023-10-10T20:42:24.113541+0000","last_clean":"2023-10-10T20:42:24.113541+0000","last_became_active":"2023-10-10T20:42:24.108731+0000","last_became_peered":"2023-10-10T20:42:24.108731+0000","last_unstale":"2023-10-10T20:42:24.113541+0000","last_undegraded":"2023-10-10T20:42:24.113541+0000","last_fullsized":"2023-10-10T20:42:24.113541+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:06:21.357294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121390+0000","last_change":"2023-10-10T20:42:24.115936+0000","last_active":"2023-10-10T20:42:24.121390+0000","last_peered":"2023-10-10T20:42:24.121390+0000","last_clean":"2023-10-10T20:42:24.121390+0000","last_became_active":"2023-10-10T20:42:24.115659+0000","last_became_peered":"2023-10-10T20:42:24.115659+0000","last_unstale":"2023-10-10T20:42:24.121390+0000","last_undegraded":"2023-10-10T20:42:24.121390+0000","last_fullsized":"2023-10-10T20:42:24.121390+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:47.040763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126170+0000","last_change":"2023-10-10T20:42:24.123600+0000","last_active":"2023-10-10T20:42:24.126170+0000","last_peered":"2023-10-10T20:42:24.126170+0000","last_clean":"2023-10-10T20:42:24.126170+0000","last_became_active":"2023-10-10T20:42:24.123226+0000","last_became_peered":"2023-10-10T20:42:24.123226+0000","last_unstale":"2023-10-10T20:42:24.126170+0000","last_undegraded":"2023-10-10T20:42:24.126170+0000","last_fullsized":"2023-10-10T20:42:24.126170+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:46:15.751003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.112765+0000","last_change":"2023-10-10T20:42:24.110429+0000","last_active":"2023-10-10T20:42:24.112765+0000","last_peered":"2023-10-10T20:42:24.112765+0000","last_clean":"2023-10-10T20:42:24.112765+0000","last_became_active":"2023-10-10T20:42:24.110171+0000","last_became_peered":"2023-10-10T20:42:24.110171+0000","last_unstale":"2023-10-10T20:42:24.112765+0000","last_undegraded":"2023-10-10T20:42:24.112765+0000","last_fullsized":"2023-10-10T20:42:24.112765+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:02.631228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.128984+0000","last_change":"2023-10-10T20:42:24.125254+0000","last_active":"2023-10-10T20:42:24.128984+0000","last_peered":"2023-10-10T20:42:24.128984+0000","last_clean":"2023-10-10T20:42:24.128984+0000","last_became_active":"2023-10-10T20:42:24.124838+0000","last_became_peered":"2023-10-10T20:42:24.124838+0000","last_unstale":"2023-10-10T20:42:24.128984+0000","last_undegraded":"2023-10-10T20:42:24.128984+0000","last_fullsized":"2023-10-10T20:42:24.128984+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:13.353418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114550+0000","last_change":"2023-10-10T20:42:24.110653+0000","last_active":"2023-10-10T20:42:24.114550+0000","last_peered":"2023-10-10T20:42:24.114550+0000","last_clean":"2023-10-10T20:42:24.114550+0000","last_became_active":"2023-10-10T20:42:24.110226+0000","last_became_peered":"2023-10-10T20:42:24.110226+0000","last_unstale":"2023-10-10T20:42:24.114550+0000","last_undegraded":"2023-10-10T20:42:24.114550+0000","last_fullsized":"2023-10-10T20:42:24.114550+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:02.631228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.120869+0000","last_change":"2023-10-10T20:42:24.112486+0000","last_active":"2023-10-10T20:42:24.120869+0000","last_peered":"2023-10-10T20:42:24.120869+0000","last_clean":"2023-10-10T20:42:24.120869+0000","last_became_active":"2023-10-10T20:42:24.112258+0000","last_became_peered":"2023-10-10T20:42:24.112258+0000","last_unstale":"2023-10-10T20:42:24.120869+0000","last_undegraded":"2023-10-10T20:42:24.120869+0000","last_fullsized":"2023-10-10T20:42:24.120869+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:46:07.440450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121137+0000","last_change":"2023-10-10T20:42:24.115300+0000","last_active":"2023-10-10T20:42:24.121137+0000","last_peered":"2023-10-10T20:42:24.121137+0000","last_clean":"2023-10-10T20:42:24.121137+0000","last_became_active":"2023-10-10T20:42:24.115040+0000","last_became_peered":"2023-10-10T20:42:24.115040+0000","last_unstale":"2023-10-10T20:42:24.121137+0000","last_undegraded":"2023-10-10T20:42:24.121137+0000","last_fullsized":"2023-10-10T20:42:24.121137+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:46:19.157900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.111054+0000","last_change":"2023-10-10T20:42:24.104338+0000","last_active":"2023-10-10T20:42:24.111054+0000","last_peered":"2023-10-10T20:42:24.111054+0000","last_clean":"2023-10-10T20:42:24.111054+0000","last_became_active":"2023-10-10T20:42:24.104037+0000","last_became_peered":"2023-10-10T20:42:24.104037+0000","last_unstale":"2023-10-10T20:42:24.111054+0000","last_undegraded":"2023-10-10T20:42:24.111054+0000","last_fullsized":"2023-10-10T20:42:24.111054+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:25:09.330944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"152'4","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.131700+0000","last_change":"2023-10-10T20:42:23.105959+0000","last_active":"2023-10-10T20:42:24.131700+0000","last_peered":"2023-10-10T20:42:24.131700+0000","last_clean":"2023-10-10T20:42:24.131700+0000","last_became_active":"2023-10-10T20:42:23.105709+0000","last_became_peered":"2023-10-10T20:42:23.105709+0000","last_unstale":"2023-10-10T20:42:24.131700+0000","last_undegraded":"2023-10-10T20:42:24.131700+0000","last_fullsized":"2023-10-10T20:42:24.131700+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:42.951721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.128823+0000","last_change":"2023-10-10T20:42:24.124965+0000","last_active":"2023-10-10T20:42:24.128823+0000","last_peered":"2023-10-10T20:42:24.128823+0000","last_clean":"2023-10-10T20:42:24.128823+0000","last_became_active":"2023-10-10T20:42:24.124706+0000","last_became_peered":"2023-10-10T20:42:24.124706+0000","last_unstale":"2023-10-10T20:42:24.128823+0000","last_undegraded":"2023-10-10T20:42:24.128823+0000","last_fullsized":"2023-10-10T20:42:24.128823+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:47:26.708886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.120664+0000","last_change":"2023-10-10T20:42:23.105858+0000","last_active":"2023-10-10T20:42:24.120664+0000","last_peered":"2023-10-10T20:42:24.120664+0000","last_clean":"2023-10-10T20:42:24.120664+0000","last_became_active":"2023-10-10T20:42:23.105592+0000","last_became_peered":"2023-10-10T20:42:23.105592+0000","last_unstale":"2023-10-10T20:42:24.120664+0000","last_undegraded":"2023-10-10T20:42:24.120664+0000","last_fullsized":"2023-10-10T20:42:24.120664+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:54.378347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121344+0000","last_change":"2023-10-10T20:42:24.115929+0000","last_active":"2023-10-10T20:42:24.121344+0000","last_peered":"2023-10-10T20:42:24.121344+0000","last_clean":"2023-10-10T20:42:24.121344+0000","last_became_active":"2023-10-10T20:42:24.115637+0000","last_became_peered":"2023-10-10T20:42:24.115637+0000","last_unstale":"2023-10-10T20:42:24.121344+0000","last_undegraded":"2023-10-10T20:42:24.121344+0000","last_fullsized":"2023-10-10T20:42:24.121344+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:39:10.428774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.124476+0000","last_change":"2023-10-10T20:42:23.103827+0000","last_active":"2023-10-10T20:42:24.124476+0000","last_peered":"2023-10-10T20:42:24.124476+0000","last_clean":"2023-10-10T20:42:24.124476+0000","last_became_active":"2023-10-10T20:42:23.103329+0000","last_became_peered":"2023-10-10T20:42:23.103329+0000","last_unstale":"2023-10-10T20:42:24.124476+0000","last_undegraded":"2023-10-10T20:42:24.124476+0000","last_fullsized":"2023-10-10T20:42:24.124476+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:42:08.332947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121209+0000","last_change":"2023-10-10T20:42:24.114480+0000","last_active":"2023-10-10T20:42:24.121209+0000","last_peered":"2023-10-10T20:42:24.121209+0000","last_clean":"2023-10-10T20:42:24.121209+0000","last_became_active":"2023-10-10T20:42:24.114244+0000","last_became_peered":"2023-10-10T20:42:24.114244+0000","last_unstale":"2023-10-10T20:42:24.121209+0000","last_undegraded":"2023-10-10T20:42:24.121209+0000","last_fullsized":"2023-10-10T20:42:24.121209+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:00:26.956205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.124758+0000","last_change":"2023-10-10T20:42:23.104651+0000","last_active":"2023-10-10T20:42:24.124758+0000","last_peered":"2023-10-10T20:42:24.124758+0000","last_clean":"2023-10-10T20:42:24.124758+0000","last_became_active":"2023-10-10T20:42:23.104297+0000","last_became_peered":"2023-10-10T20:42:23.104297+0000","last_unstale":"2023-10-10T20:42:24.124758+0000","last_undegraded":"2023-10-10T20:42:24.124758+0000","last_fullsized":"2023-10-10T20:42:24.124758+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:49:42.586346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114531+0000","last_change":"2023-10-10T20:42:24.112093+0000","last_active":"2023-10-10T20:42:24.114531+0000","last_peered":"2023-10-10T20:42:24.114531+0000","last_clean":"2023-10-10T20:42:24.114531+0000","last_became_active":"2023-10-10T20:42:24.111868+0000","last_became_peered":"2023-10-10T20:42:24.111868+0000","last_unstale":"2023-10-10T20:42:24.114531+0000","last_undegraded":"2023-10-10T20:42:24.114531+0000","last_fullsized":"2023-10-10T20:42:24.114531+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:30.464320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.125138+0000","last_change":"2023-10-10T20:42:23.105709+0000","last_active":"2023-10-10T20:42:24.125138+0000","last_peered":"2023-10-10T20:42:24.125138+0000","last_clean":"2023-10-10T20:42:24.125138+0000","last_became_active":"2023-10-10T20:42:23.105538+0000","last_became_peered":"2023-10-10T20:42:23.105538+0000","last_unstale":"2023-10-10T20:42:24.125138+0000","last_undegraded":"2023-10-10T20:42:24.125138+0000","last_fullsized":"2023-10-10T20:42:24.125138+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:26:59.934888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113606+0000","last_change":"2023-10-10T20:42:24.109974+0000","last_active":"2023-10-10T20:42:24.113606+0000","last_peered":"2023-10-10T20:42:24.113606+0000","last_clean":"2023-10-10T20:42:24.113606+0000","last_became_active":"2023-10-10T20:42:24.109707+0000","last_became_peered":"2023-10-10T20:42:24.109707+0000","last_unstale":"2023-10-10T20:42:24.113606+0000","last_undegraded":"2023-10-10T20:42:24.113606+0000","last_fullsized":"2023-10-10T20:42:24.113606+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:34.653111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.101968+0000","last_change":"2023-10-10T20:42:23.109627+0000","last_active":"2023-10-10T20:42:24.101968+0000","last_peered":"2023-10-10T20:42:24.101968+0000","last_clean":"2023-10-10T20:42:24.101968+0000","last_became_active":"2023-10-10T20:42:23.109387+0000","last_became_peered":"2023-10-10T20:42:23.109387+0000","last_unstale":"2023-10-10T20:42:24.101968+0000","last_undegraded":"2023-10-10T20:42:24.101968+0000","last_fullsized":"2023-10-10T20:42:24.101968+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:12.342683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114505+0000","last_change":"2023-10-10T20:42:24.111171+0000","last_active":"2023-10-10T20:42:24.114505+0000","last_peered":"2023-10-10T20:42:24.114505+0000","last_clean":"2023-10-10T20:42:24.114505+0000","last_became_active":"2023-10-10T20:42:24.110894+0000","last_became_peered":"2023-10-10T20:42:24.110894+0000","last_unstale":"2023-10-10T20:42:24.114505+0000","last_undegraded":"2023-10-10T20:42:24.114505+0000","last_fullsized":"2023-10-10T20:42:24.114505+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:29:16.754131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.102665+0000","last_change":"2023-10-10T20:42:23.105273+0000","last_active":"2023-10-10T20:42:24.102665+0000","last_peered":"2023-10-10T20:42:24.102665+0000","last_clean":"2023-10-10T20:42:24.102665+0000","last_became_active":"2023-10-10T20:42:23.104999+0000","last_became_peered":"2023-10-10T20:42:23.104999+0000","last_unstale":"2023-10-10T20:42:24.102665+0000","last_undegraded":"2023-10-10T20:42:24.102665+0000","last_fullsized":"2023-10-10T20:42:24.102665+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:04.056737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.103733+0000","last_change":"2023-10-10T20:42:24.101017+0000","last_active":"2023-10-10T20:42:24.103733+0000","last_peered":"2023-10-10T20:42:24.103733+0000","last_clean":"2023-10-10T20:42:24.103733+0000","last_became_active":"2023-10-10T20:42:24.100716+0000","last_became_peered":"2023-10-10T20:42:24.100716+0000","last_unstale":"2023-10-10T20:42:24.103733+0000","last_undegraded":"2023-10-10T20:42:24.103733+0000","last_fullsized":"2023-10-10T20:42:24.103733+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:20:48.561636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.101835+0000","last_change":"2023-10-10T20:42:23.108782+0000","last_active":"2023-10-10T20:42:24.101835+0000","last_peered":"2023-10-10T20:42:24.101835+0000","last_clean":"2023-10-10T20:42:24.101835+0000","last_became_active":"2023-10-10T20:42:23.108480+0000","last_became_peered":"2023-10-10T20:42:23.108480+0000","last_unstale":"2023-10-10T20:42:24.101835+0000","last_undegraded":"2023-10-10T20:42:24.101835+0000","last_fullsized":"2023-10-10T20:42:24.101835+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:32:14.835359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.112616+0000","last_change":"2023-10-10T20:42:24.108473+0000","last_active":"2023-10-10T20:42:24.112616+0000","last_peered":"2023-10-10T20:42:24.112616+0000","last_clean":"2023-10-10T20:42:24.112616+0000","last_became_active":"2023-10-10T20:42:24.108220+0000","last_became_peered":"2023-10-10T20:42:24.108220+0000","last_unstale":"2023-10-10T20:42:24.112616+0000","last_undegraded":"2023-10-10T20:42:24.112616+0000","last_fullsized":"2023-10-10T20:42:24.112616+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:49.847316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.103586+0000","last_change":"2023-10-10T20:42:23.109104+0000","last_active":"2023-10-10T20:42:24.103586+0000","last_peered":"2023-10-10T20:42:24.103586+0000","last_clean":"2023-10-10T20:42:24.103586+0000","last_became_active":"2023-10-10T20:42:23.108836+0000","last_became_peered":"2023-10-10T20:42:23.108836+0000","last_unstale":"2023-10-10T20:42:24.103586+0000","last_undegraded":"2023-10-10T20:42:24.103586+0000","last_fullsized":"2023-10-10T20:42:24.103586+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:42.951721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113476+0000","last_change":"2023-10-10T20:42:24.108573+0000","last_active":"2023-10-10T20:42:24.113476+0000","last_peered":"2023-10-10T20:42:24.113476+0000","last_clean":"2023-10-10T20:42:24.113476+0000","last_became_active":"2023-10-10T20:42:24.108300+0000","last_became_peered":"2023-10-10T20:42:24.108300+0000","last_unstale":"2023-10-10T20:42:24.113476+0000","last_undegraded":"2023-10-10T20:42:24.113476+0000","last_fullsized":"2023-10-10T20:42:24.113476+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:52:51.382479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"152'3","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.123020+0000","last_change":"2023-10-10T20:42:23.107169+0000","last_active":"2023-10-10T20:42:24.123020+0000","last_peered":"2023-10-10T20:42:24.123020+0000","last_clean":"2023-10-10T20:42:24.123020+0000","last_became_active":"2023-10-10T20:42:23.106721+0000","last_became_peered":"2023-10-10T20:42:23.106721+0000","last_unstale":"2023-10-10T20:42:24.123020+0000","last_undegraded":"2023-10-10T20:42:24.123020+0000","last_fullsized":"2023-10-10T20:42:24.123020+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:03:32.352440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121029+0000","last_change":"2023-10-10T20:42:24.115401+0000","last_active":"2023-10-10T20:42:24.121029+0000","last_peered":"2023-10-10T20:42:24.121029+0000","last_clean":"2023-10-10T20:42:24.121029+0000","last_became_active":"2023-10-10T20:42:24.113812+0000","last_became_peered":"2023-10-10T20:42:24.113812+0000","last_unstale":"2023-10-10T20:42:24.121029+0000","last_undegraded":"2023-10-10T20:42:24.121029+0000","last_fullsized":"2023-10-10T20:42:24.121029+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:01.948796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"152'6","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.132627+0000","last_change":"2023-10-10T20:42:23.108302+0000","last_active":"2023-10-10T20:42:24.132627+0000","last_peered":"2023-10-10T20:42:24.132627+0000","last_clean":"2023-10-10T20:42:24.132627+0000","last_became_active":"2023-10-10T20:42:23.108085+0000","last_became_peered":"2023-10-10T20:42:23.108085+0000","last_unstale":"2023-10-10T20:42:24.132627+0000","last_undegraded":"2023-10-10T20:42:24.132627+0000","last_fullsized":"2023-10-10T20:42:24.132627+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:02:33.953621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113456+0000","last_change":"2023-10-10T20:42:24.109419+0000","last_active":"2023-10-10T20:42:24.113456+0000","last_peered":"2023-10-10T20:42:24.113456+0000","last_clean":"2023-10-10T20:42:24.113456+0000","last_became_active":"2023-10-10T20:42:24.109068+0000","last_became_peered":"2023-10-10T20:42:24.109068+0000","last_unstale":"2023-10-10T20:42:24.113456+0000","last_undegraded":"2023-10-10T20:42:24.113456+0000","last_fullsized":"2023-10-10T20:42:24.113456+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:39:59.929264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126772+0000","last_change":"2023-10-10T20:42:23.105344+0000","last_active":"2023-10-10T20:42:24.126772+0000","last_peered":"2023-10-10T20:42:24.126772+0000","last_clean":"2023-10-10T20:42:24.126772+0000","last_became_active":"2023-10-10T20:42:23.105013+0000","last_became_peered":"2023-10-10T20:42:23.105013+0000","last_unstale":"2023-10-10T20:42:24.126772+0000","last_undegraded":"2023-10-10T20:42:24.126772+0000","last_fullsized":"2023-10-10T20:42:24.126772+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:59.544805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121309+0000","last_change":"2023-10-10T20:42:24.115013+0000","last_active":"2023-10-10T20:42:24.121309+0000","last_peered":"2023-10-10T20:42:24.121309+0000","last_clean":"2023-10-10T20:42:24.121309+0000","last_became_active":"2023-10-10T20:42:24.114763+0000","last_became_peered":"2023-10-10T20:42:24.114763+0000","last_unstale":"2023-10-10T20:42:24.121309+0000","last_undegraded":"2023-10-10T20:42:24.121309+0000","last_fullsized":"2023-10-10T20:42:24.121309+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:06.445923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.103163+0000","last_change":"2023-10-10T20:42:23.105949+0000","last_active":"2023-10-10T20:42:24.103163+0000","last_peered":"2023-10-10T20:42:24.103163+0000","last_clean":"2023-10-10T20:42:24.103163+0000","last_became_active":"2023-10-10T20:42:23.105676+0000","last_became_peered":"2023-10-10T20:42:23.105676+0000","last_unstale":"2023-10-10T20:42:24.103163+0000","last_undegraded":"2023-10-10T20:42:24.103163+0000","last_fullsized":"2023-10-10T20:42:24.103163+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:43:10.994181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126457+0000","last_change":"2023-10-10T20:42:24.126007+0000","last_active":"2023-10-10T20:42:24.126457+0000","last_peered":"2023-10-10T20:42:24.126457+0000","last_clean":"2023-10-10T20:42:24.126457+0000","last_became_active":"2023-10-10T20:42:24.125699+0000","last_became_peered":"2023-10-10T20:42:24.125699+0000","last_unstale":"2023-10-10T20:42:24.126457+0000","last_undegraded":"2023-10-10T20:42:24.126457+0000","last_fullsized":"2023-10-10T20:42:24.126457+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:09:15.013589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.101618+0000","last_change":"2023-10-10T20:42:23.103097+0000","last_active":"2023-10-10T20:42:24.101618+0000","last_peered":"2023-10-10T20:42:24.101618+0000","last_clean":"2023-10-10T20:42:24.101618+0000","last_became_active":"2023-10-10T20:42:23.102819+0000","last_became_peered":"2023-10-10T20:42:23.102819+0000","last_unstale":"2023-10-10T20:42:24.101618+0000","last_undegraded":"2023-10-10T20:42:24.101618+0000","last_fullsized":"2023-10-10T20:42:24.101618+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:20:47.550902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121175+0000","last_change":"2023-10-10T20:42:24.115553+0000","last_active":"2023-10-10T20:42:24.121175+0000","last_peered":"2023-10-10T20:42:24.121175+0000","last_clean":"2023-10-10T20:42:24.121175+0000","last_became_active":"2023-10-10T20:42:24.115239+0000","last_became_peered":"2023-10-10T20:42:24.115239+0000","last_unstale":"2023-10-10T20:42:24.121175+0000","last_undegraded":"2023-10-10T20:42:24.121175+0000","last_fullsized":"2023-10-10T20:42:24.121175+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:14.819673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"152'8","reported_seq":18,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.125165+0000","last_change":"2023-10-10T20:42:23.103610+0000","last_active":"2023-10-10T20:42:24.125165+0000","last_peered":"2023-10-10T20:42:24.125165+0000","last_clean":"2023-10-10T20:42:24.125165+0000","last_became_active":"2023-10-10T20:42:23.103353+0000","last_became_peered":"2023-10-10T20:42:23.103353+0000","last_unstale":"2023-10-10T20:42:24.125165+0000","last_undegraded":"2023-10-10T20:42:24.125165+0000","last_fullsized":"2023-10-10T20:42:24.125165+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:23:34.102393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.111073+0000","last_change":"2023-10-10T20:42:24.104841+0000","last_active":"2023-10-10T20:42:24.111073+0000","last_peered":"2023-10-10T20:42:24.111073+0000","last_clean":"2023-10-10T20:42:24.111073+0000","last_became_active":"2023-10-10T20:42:24.104561+0000","last_became_peered":"2023-10-10T20:42:24.104561+0000","last_unstale":"2023-10-10T20:42:24.111073+0000","last_undegraded":"2023-10-10T20:42:24.111073+0000","last_fullsized":"2023-10-10T20:42:24.111073+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:45:43.231122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.123211+0000","last_change":"2023-10-10T20:42:23.105236+0000","last_active":"2023-10-10T20:42:24.123211+0000","last_peered":"2023-10-10T20:42:24.123211+0000","last_clean":"2023-10-10T20:42:24.123211+0000","last_became_active":"2023-10-10T20:42:23.104983+0000","last_became_peered":"2023-10-10T20:42:23.104983+0000","last_unstale":"2023-10-10T20:42:24.123211+0000","last_undegraded":"2023-10-10T20:42:24.123211+0000","last_fullsized":"2023-10-10T20:42:24.123211+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:03.181433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113521+0000","last_change":"2023-10-10T20:42:24.107994+0000","last_active":"2023-10-10T20:42:24.113521+0000","last_peered":"2023-10-10T20:42:24.113521+0000","last_clean":"2023-10-10T20:42:24.113521+0000","last_became_active":"2023-10-10T20:42:24.107722+0000","last_became_peered":"2023-10-10T20:42:24.107722+0000","last_unstale":"2023-10-10T20:42:24.113521+0000","last_undegraded":"2023-10-10T20:42:24.113521+0000","last_fullsized":"2023-10-10T20:42:24.113521+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:57.924961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.123838+0000","last_change":"2023-10-10T20:42:23.111459+0000","last_active":"2023-10-10T20:42:24.123838+0000","last_peered":"2023-10-10T20:42:24.123838+0000","last_clean":"2023-10-10T20:42:24.123838+0000","last_became_active":"2023-10-10T20:42:23.111197+0000","last_became_peered":"2023-10-10T20:42:23.111197+0000","last_unstale":"2023-10-10T20:42:24.123838+0000","last_undegraded":"2023-10-10T20:42:24.123838+0000","last_fullsized":"2023-10-10T20:42:24.123838+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:23:14.263295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.106150+0000","last_change":"2023-10-10T20:42:24.101341+0000","last_active":"2023-10-10T20:42:24.106150+0000","last_peered":"2023-10-10T20:42:24.106150+0000","last_clean":"2023-10-10T20:42:24.106150+0000","last_became_active":"2023-10-10T20:42:24.101086+0000","last_became_peered":"2023-10-10T20:42:24.101086+0000","last_unstale":"2023-10-10T20:42:24.106150+0000","last_undegraded":"2023-10-10T20:42:24.106150+0000","last_fullsized":"2023-10-10T20:42:24.106150+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:11:04.270228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126614+0000","last_change":"2023-10-10T20:42:23.107193+0000","last_active":"2023-10-10T20:42:24.126614+0000","last_peered":"2023-10-10T20:42:24.126614+0000","last_clean":"2023-10-10T20:42:24.126614+0000","last_became_active":"2023-10-10T20:42:23.106136+0000","last_became_peered":"2023-10-10T20:42:23.106136+0000","last_unstale":"2023-10-10T20:42:24.126614+0000","last_undegraded":"2023-10-10T20:42:24.126614+0000","last_fullsized":"2023-10-10T20:42:24.126614+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:26:51.984824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.120858+0000","last_change":"2023-10-10T20:42:24.112738+0000","last_active":"2023-10-10T20:42:24.120858+0000","last_peered":"2023-10-10T20:42:24.120858+0000","last_clean":"2023-10-10T20:42:24.120858+0000","last_became_active":"2023-10-10T20:42:24.112515+0000","last_became_peered":"2023-10-10T20:42:24.112515+0000","last_unstale":"2023-10-10T20:42:24.120858+0000","last_undegraded":"2023-10-10T20:42:24.120858+0000","last_fullsized":"2023-10-10T20:42:24.120858+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:01.948796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.096078+0000","last_change":"2023-10-10T20:42:23.107823+0000","last_active":"2023-10-10T20:42:24.096078+0000","last_peered":"2023-10-10T20:42:24.096078+0000","last_clean":"2023-10-10T20:42:24.096078+0000","last_became_active":"2023-10-10T20:42:23.107643+0000","last_became_peered":"2023-10-10T20:42:23.107643+0000","last_unstale":"2023-10-10T20:42:24.096078+0000","last_undegraded":"2023-10-10T20:42:24.096078+0000","last_fullsized":"2023-10-10T20:42:24.096078+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:43:41.316715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.120251+0000","last_change":"2023-10-10T20:42:24.110967+0000","last_active":"2023-10-10T20:42:24.120251+0000","last_peered":"2023-10-10T20:42:24.120251+0000","last_clean":"2023-10-10T20:42:24.120251+0000","last_became_active":"2023-10-10T20:42:24.110687+0000","last_became_peered":"2023-10-10T20:42:24.110687+0000","last_unstale":"2023-10-10T20:42:24.120251+0000","last_undegraded":"2023-10-10T20:42:24.120251+0000","last_fullsized":"2023-10-10T20:42:24.120251+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:53.653343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121843+0000","last_change":"2023-10-10T20:42:23.105742+0000","last_active":"2023-10-10T20:42:24.121843+0000","last_peered":"2023-10-10T20:42:24.121843+0000","last_clean":"2023-10-10T20:42:24.121843+0000","last_became_active":"2023-10-10T20:42:23.105472+0000","last_became_peered":"2023-10-10T20:42:23.105472+0000","last_unstale":"2023-10-10T20:42:24.121843+0000","last_undegraded":"2023-10-10T20:42:24.121843+0000","last_fullsized":"2023-10-10T20:42:24.121843+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:29.453586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113971+0000","last_change":"2023-10-10T20:42:24.112508+0000","last_active":"2023-10-10T20:42:24.113971+0000","last_peered":"2023-10-10T20:42:24.113971+0000","last_clean":"2023-10-10T20:42:24.113971+0000","last_became_active":"2023-10-10T20:42:24.112276+0000","last_became_peered":"2023-10-10T20:42:24.112276+0000","last_unstale":"2023-10-10T20:42:24.113971+0000","last_undegraded":"2023-10-10T20:42:24.113971+0000","last_fullsized":"2023-10-10T20:42:24.113971+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:55:23.468746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.104357+0000","last_change":"2023-10-10T20:42:23.110977+0000","last_active":"2023-10-10T20:42:24.104357+0000","last_peered":"2023-10-10T20:42:24.104357+0000","last_clean":"2023-10-10T20:42:24.104357+0000","last_became_active":"2023-10-10T20:42:23.110704+0000","last_became_peered":"2023-10-10T20:42:23.110704+0000","last_unstale":"2023-10-10T20:42:24.104357+0000","last_undegraded":"2023-10-10T20:42:24.104357+0000","last_fullsized":"2023-10-10T20:42:24.104357+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:22:28.404747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114385+0000","last_change":"2023-10-10T20:42:24.109982+0000","last_active":"2023-10-10T20:42:24.114385+0000","last_peered":"2023-10-10T20:42:24.114385+0000","last_clean":"2023-10-10T20:42:24.114385+0000","last_became_active":"2023-10-10T20:42:24.109703+0000","last_became_peered":"2023-10-10T20:42:24.109703+0000","last_unstale":"2023-10-10T20:42:24.114385+0000","last_undegraded":"2023-10-10T20:42:24.114385+0000","last_fullsized":"2023-10-10T20:42:24.114385+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:46:22.492928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.103008+0000","last_change":"2023-10-10T20:42:23.109984+0000","last_active":"2023-10-10T20:42:24.103008+0000","last_peered":"2023-10-10T20:42:24.103008+0000","last_clean":"2023-10-10T20:42:24.103008+0000","last_became_active":"2023-10-10T20:42:23.109710+0000","last_became_peered":"2023-10-10T20:42:23.109710+0000","last_unstale":"2023-10-10T20:42:24.103008+0000","last_undegraded":"2023-10-10T20:42:24.103008+0000","last_fullsized":"2023-10-10T20:42:24.103008+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:41:54.647934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113784+0000","last_change":"2023-10-10T20:42:24.110731+0000","last_active":"2023-10-10T20:42:24.113784+0000","last_peered":"2023-10-10T20:42:24.113784+0000","last_clean":"2023-10-10T20:42:24.113784+0000","last_became_active":"2023-10-10T20:42:24.110499+0000","last_became_peered":"2023-10-10T20:42:24.110499+0000","last_unstale":"2023-10-10T20:42:24.113784+0000","last_undegraded":"2023-10-10T20:42:24.113784+0000","last_fullsized":"2023-10-10T20:42:24.113784+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:46:15.751003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.102364+0000","last_change":"2023-10-10T20:42:23.104363+0000","last_active":"2023-10-10T20:42:24.102364+0000","last_peered":"2023-10-10T20:42:24.102364+0000","last_clean":"2023-10-10T20:42:24.102364+0000","last_became_active":"2023-10-10T20:42:23.104108+0000","last_became_peered":"2023-10-10T20:42:23.104108+0000","last_unstale":"2023-10-10T20:42:24.102364+0000","last_undegraded":"2023-10-10T20:42:24.102364+0000","last_fullsized":"2023-10-10T20:42:24.102364+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:52.876825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.124500+0000","last_change":"2023-10-10T20:42:24.122887+0000","last_active":"2023-10-10T20:42:24.124500+0000","last_peered":"2023-10-10T20:42:24.124500+0000","last_clean":"2023-10-10T20:42:24.124500+0000","last_became_active":"2023-10-10T20:42:24.122521+0000","last_became_peered":"2023-10-10T20:42:24.122521+0000","last_unstale":"2023-10-10T20:42:24.124500+0000","last_undegraded":"2023-10-10T20:42:24.124500+0000","last_fullsized":"2023-10-10T20:42:24.124500+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:09:32.731249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.125650+0000","last_change":"2023-10-10T20:42:23.109167+0000","last_active":"2023-10-10T20:42:24.125650+0000","last_peered":"2023-10-10T20:42:24.125650+0000","last_clean":"2023-10-10T20:42:24.125650+0000","last_became_active":"2023-10-10T20:42:23.108692+0000","last_became_peered":"2023-10-10T20:42:23.108692+0000","last_unstale":"2023-10-10T20:42:24.125650+0000","last_undegraded":"2023-10-10T20:42:24.125650+0000","last_fullsized":"2023-10-10T20:42:24.125650+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:56.210224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126436+0000","last_change":"2023-10-10T20:42:24.125413+0000","last_active":"2023-10-10T20:42:24.126436+0000","last_peered":"2023-10-10T20:42:24.126436+0000","last_clean":"2023-10-10T20:42:24.126436+0000","last_became_active":"2023-10-10T20:42:24.125149+0000","last_became_peered":"2023-10-10T20:42:24.125149+0000","last_unstale":"2023-10-10T20:42:24.126436+0000","last_undegraded":"2023-10-10T20:42:24.126436+0000","last_fullsized":"2023-10-10T20:42:24.126436+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:00:26.956205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.096131+0000","last_change":"2023-10-10T20:42:23.098780+0000","last_active":"2023-10-10T20:42:24.096131+0000","last_peered":"2023-10-10T20:42:24.096131+0000","last_clean":"2023-10-10T20:42:24.096131+0000","last_became_active":"2023-10-10T20:42:23.098462+0000","last_became_peered":"2023-10-10T20:42:23.098462+0000","last_unstale":"2023-10-10T20:42:24.096131+0000","last_undegraded":"2023-10-10T20:42:24.096131+0000","last_fullsized":"2023-10-10T20:42:24.096131+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:37:13.498053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114471+0000","last_change":"2023-10-10T20:42:24.110340+0000","last_active":"2023-10-10T20:42:24.114471+0000","last_peered":"2023-10-10T20:42:24.114471+0000","last_clean":"2023-10-10T20:42:24.114471+0000","last_became_active":"2023-10-10T20:42:24.110079+0000","last_became_peered":"2023-10-10T20:42:24.110079+0000","last_unstale":"2023-10-10T20:42:24.114471+0000","last_undegraded":"2023-10-10T20:42:24.114471+0000","last_fullsized":"2023-10-10T20:42:24.114471+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:35:23.790779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.102018+0000","last_change":"2023-10-10T20:42:23.108778+0000","last_active":"2023-10-10T20:42:24.102018+0000","last_peered":"2023-10-10T20:42:24.102018+0000","last_clean":"2023-10-10T20:42:24.102018+0000","last_became_active":"2023-10-10T20:42:23.108454+0000","last_became_peered":"2023-10-10T20:42:23.108454+0000","last_unstale":"2023-10-10T20:42:24.102018+0000","last_undegraded":"2023-10-10T20:42:24.102018+0000","last_fullsized":"2023-10-10T20:42:24.102018+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:09:31.720515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114247+0000","last_change":"2023-10-10T20:42:24.111895+0000","last_active":"2023-10-10T20:42:24.114247+0000","last_peered":"2023-10-10T20:42:24.114247+0000","last_clean":"2023-10-10T20:42:24.114247+0000","last_became_active":"2023-10-10T20:42:24.111432+0000","last_became_peered":"2023-10-10T20:42:24.111432+0000","last_unstale":"2023-10-10T20:42:24.114247+0000","last_undegraded":"2023-10-10T20:42:24.114247+0000","last_fullsized":"2023-10-10T20:42:24.114247+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:21:59.491769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.096099+0000","last_change":"2023-10-10T20:42:23.108978+0000","last_active":"2023-10-10T20:42:24.096099+0000","last_peered":"2023-10-10T20:42:24.096099+0000","last_clean":"2023-10-10T20:42:24.096099+0000","last_became_active":"2023-10-10T20:42:23.108707+0000","last_became_peered":"2023-10-10T20:42:23.108707+0000","last_unstale":"2023-10-10T20:42:24.096099+0000","last_undegraded":"2023-10-10T20:42:24.096099+0000","last_fullsized":"2023-10-10T20:42:24.096099+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:44.681204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121726+0000","last_change":"2023-10-10T20:42:24.116434+0000","last_active":"2023-10-10T20:42:24.121726+0000","last_peered":"2023-10-10T20:42:24.121726+0000","last_clean":"2023-10-10T20:42:24.121726+0000","last_became_active":"2023-10-10T20:42:24.115786+0000","last_became_peered":"2023-10-10T20:42:24.115786+0000","last_unstale":"2023-10-10T20:42:24.121726+0000","last_undegraded":"2023-10-10T20:42:24.121726+0000","last_fullsized":"2023-10-10T20:42:24.121726+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:46:07.440450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.098460+0000","last_change":"2023-10-10T20:42:23.107133+0000","last_active":"2023-10-10T20:42:24.098460+0000","last_peered":"2023-10-10T20:42:24.098460+0000","last_clean":"2023-10-10T20:42:24.098460+0000","last_became_active":"2023-10-10T20:42:23.106863+0000","last_became_peered":"2023-10-10T20:42:23.106863+0000","last_unstale":"2023-10-10T20:42:24.098460+0000","last_undegraded":"2023-10-10T20:42:24.098460+0000","last_fullsized":"2023-10-10T20:42:24.098460+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:58.808416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126598+0000","last_change":"2023-10-10T20:42:24.123718+0000","last_active":"2023-10-10T20:42:24.126598+0000","last_peered":"2023-10-10T20:42:24.126598+0000","last_clean":"2023-10-10T20:42:24.126598+0000","last_became_active":"2023-10-10T20:42:24.123096+0000","last_became_peered":"2023-10-10T20:42:24.123096+0000","last_unstale":"2023-10-10T20:42:24.126598+0000","last_undegraded":"2023-10-10T20:42:24.126598+0000","last_fullsized":"2023-10-10T20:42:24.126598+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:14:18.635698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.104118+0000","last_change":"2023-10-10T20:42:23.104363+0000","last_active":"2023-10-10T20:42:24.104118+0000","last_peered":"2023-10-10T20:42:24.104118+0000","last_clean":"2023-10-10T20:42:24.104118+0000","last_became_active":"2023-10-10T20:42:23.104109+0000","last_became_peered":"2023-10-10T20:42:23.104109+0000","last_unstale":"2023-10-10T20:42:24.104118+0000","last_undegraded":"2023-10-10T20:42:24.104118+0000","last_fullsized":"2023-10-10T20:42:24.104118+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:43:10.994181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.112743+0000","last_change":"2023-10-10T20:42:24.109363+0000","last_active":"2023-10-10T20:42:24.112743+0000","last_peered":"2023-10-10T20:42:24.112743+0000","last_clean":"2023-10-10T20:42:24.112743+0000","last_became_active":"2023-10-10T20:42:24.109127+0000","last_became_peered":"2023-10-10T20:42:24.109127+0000","last_unstale":"2023-10-10T20:42:24.112743+0000","last_undegraded":"2023-10-10T20:42:24.112743+0000","last_fullsized":"2023-10-10T20:42:24.112743+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:58:55.908084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.103440+0000","last_change":"2023-10-10T20:42:23.108038+0000","last_active":"2023-10-10T20:42:24.103440+0000","last_peered":"2023-10-10T20:42:24.103440+0000","last_clean":"2023-10-10T20:42:24.103440+0000","last_became_active":"2023-10-10T20:42:23.107750+0000","last_became_peered":"2023-10-10T20:42:23.107750+0000","last_unstale":"2023-10-10T20:42:24.103440+0000","last_undegraded":"2023-10-10T20:42:24.103440+0000","last_fullsized":"2023-10-10T20:42:24.103440+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:00:27.632584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114575+0000","last_change":"2023-10-10T20:42:24.110672+0000","last_active":"2023-10-10T20:42:24.114575+0000","last_peered":"2023-10-10T20:42:24.114575+0000","last_clean":"2023-10-10T20:42:24.114575+0000","last_became_active":"2023-10-10T20:42:24.110384+0000","last_became_peered":"2023-10-10T20:42:24.110384+0000","last_unstale":"2023-10-10T20:42:24.114575+0000","last_undegraded":"2023-10-10T20:42:24.114575+0000","last_fullsized":"2023-10-10T20:42:24.114575+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:49.847316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.108826+0000","last_change":"2023-10-10T20:42:23.103184+0000","last_active":"2023-10-10T20:42:24.108826+0000","last_peered":"2023-10-10T20:42:24.108826+0000","last_clean":"2023-10-10T20:42:24.108826+0000","last_became_active":"2023-10-10T20:42:23.102891+0000","last_became_peered":"2023-10-10T20:42:23.102891+0000","last_unstale":"2023-10-10T20:42:24.108826+0000","last_undegraded":"2023-10-10T20:42:24.108826+0000","last_fullsized":"2023-10-10T20:42:24.108826+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:48:40.304153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.117438+0000","last_change":"2023-10-10T20:42:24.113117+0000","last_active":"2023-10-10T20:42:24.117438+0000","last_peered":"2023-10-10T20:42:24.117438+0000","last_clean":"2023-10-10T20:42:24.117438+0000","last_became_active":"2023-10-10T20:42:24.112829+0000","last_became_peered":"2023-10-10T20:42:24.112829+0000","last_unstale":"2023-10-10T20:42:24.117438+0000","last_undegraded":"2023-10-10T20:42:24.117438+0000","last_fullsized":"2023-10-10T20:42:24.117438+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:17:06.399895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.131253+0000","last_change":"2023-10-10T20:42:23.110595+0000","last_active":"2023-10-10T20:42:24.131253+0000","last_peered":"2023-10-10T20:42:24.131253+0000","last_clean":"2023-10-10T20:42:24.131253+0000","last_became_active":"2023-10-10T20:42:23.110274+0000","last_became_peered":"2023-10-10T20:42:23.110274+0000","last_unstale":"2023-10-10T20:42:24.131253+0000","last_undegraded":"2023-10-10T20:42:24.131253+0000","last_fullsized":"2023-10-10T20:42:24.131253+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:04.056737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":296,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.096203+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:42:24.096203+0000","last_peered":"2023-10-10T20:42:24.096203+0000","last_clean":"2023-10-10T20:42:24.096203+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:42:24.096203+0000","last_undegraded":"2023-10-10T20:42:24.096203+0000","last_fullsized":"2023-10-10T20:42:24.096203+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574948,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303868,"kb_used_data":2484,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067972,"statfs":{"total":96636764160,"available":96325603328,"internally_reserved":0,"allocated":2543616,"data_stored":1460226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.79000000000000004},"min":{"1min":0.309,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71799999999999997,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52400000000000002,"15min":0.61599999999999999},"min":{"1min":0.32300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.69999999999999996,"5min":0.82099999999999995,"15min":4.1959999999999997},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.95099999999999996},"min":{"1min":0.379,"5min":0.317,"15min":0.317},"max":{"1min":0.78300000000000003,"5min":0.78800000000000003,"15min":10.1},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55200000000000005,"15min":0.71399999999999997},"min":{"1min":0.36699999999999999,"5min":0.32300000000000001,"15min":0.28999999999999998},"max":{"1min":0.73199999999999998,"5min":0.90400000000000003,"15min":6.806},"last":0.46999999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.76400000000000001},"min":{"1min":0.40100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79200000000000004,"5min":0.89000000000000001,"15min":6.9340000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.71699999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.73299999999999998,"5min":0.83899999999999997,"15min":6.9870000000000001},"last":0.54000000000000004}]},{"osd":3,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54900000000000004,"15min":0.73599999999999999},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.753,"5min":0.80200000000000005,"15min":6.6719999999999997},"last":0.504},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56699999999999995,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.71699999999999997,"5min":0.87,"15min":4.1609999999999996},"last":0.52200000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.36599999999999999,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.65100000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36299999999999999,"5min":0.309,"15min":0.30399999999999999},"max":{"1min":0.72299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.40999999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.314},"max":{"1min":0.77900000000000003,"5min":0.86099999999999999,"15min":1.3180000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.41799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71499999999999997,"5min":0.75800000000000001,"15min":0.84099999999999997},"last":0.38700000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":0.78700000000000003,"15min":1.7509999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.40899999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.85699999999999998,"15min":1.7090000000000001},"last":0.59399999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574949,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303280,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068560,"statfs":{"total":96636764160,"available":96326205440,"internally_reserved":0,"allocated":1941504,"data_stored":868876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51300000000000001,"15min":0.65500000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":7.7590000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.52100000000000002,"15min":0.69199999999999995},"min":{"1min":0.32200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.075,"5min":1.075,"15min":7.8090000000000002},"last":0.62}]},{"osd":1,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.69099999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.70199999999999996,"5min":0.80000000000000004,"15min":7.867},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.85599999999999998},"min":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.65600000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.58799999999999997},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.56100000000000005,"15min":0.81999999999999995},"min":{"1min":0.34200000000000003,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":2.641,"5min":2.641,"15min":6.7629999999999999},"last":0.58799999999999997}]},{"osd":3,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54300000000000004,"15min":0.61099999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.107,"5min":1.107,"15min":4.8620000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.70099999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.718,"5min":1.718,"15min":7.9009999999999998},"last":0.54100000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.47299999999999998},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.45500000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52700000000000002,"15min":0.495},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.376},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.53400000000000003},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.35299999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55800000000000005,"15min":0.52500000000000002},"min":{"1min":0.36099999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.372,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.433}]}]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303284,"kb_used_data":1900,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068556,"statfs":{"total":96636764160,"available":96326201344,"internally_reserved":0,"allocated":1945600,"data_stored":868378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.312,"15min":0.45900000000000002},"min":{"1min":0.22800000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.437,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":1.194},"last":0.29999999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.24199999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51000000000000001,"5min":0.51400000000000001,"15min":1.2869999999999999},"last":0.246},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.48999999999999999},"min":{"1min":0.27200000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.52800000000000002,"5min":0.54200000000000004,"15min":5.0919999999999996},"last":0.35599999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36399999999999999,"15min":0.51200000000000001},"min":{"1min":0.29399999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35699999999999998,"15min":0.40200000000000002},"min":{"1min":0.23799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":1.2170000000000001},"last":0.32600000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.73999999999999999,"5min":0.92400000000000004,"15min":1.254},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.68999999999999995},"min":{"1min":0.39300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68799999999999994,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.53300000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.36099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":0.97599999999999998,"15min":1.1799999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56899999999999995,"15min":0.60299999999999998},"min":{"1min":0.41799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71399999999999997,"5min":0.878,"15min":1.3109999999999999},"last":0.57899999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.81799999999999995},"min":{"1min":0.46000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78900000000000003,"5min":0.995,"15min":5.0659999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80700000000000005,"5min":0.89800000000000002,"15min":1.171},"last":0.46400000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57999999999999996,"15min":0.61299999999999999},"min":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77300000000000002,"5min":1.004,"15min":1.214},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.70999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78200000000000003,"5min":0.90500000000000003,"15min":4.0499999999999998},"last":0.54500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574947,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303856,"kb_used_data":2472,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067984,"statfs":{"total":96636764160,"available":96325615616,"internally_reserved":0,"allocated":2531328,"data_stored":1457819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.316,"15min":0.50800000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":7.9249999999999998},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.48699999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":6.6299999999999999},"last":0.311}]},{"osd":2,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.33100000000000002,"15min":0.53100000000000003},"min":{"1min":0.23100000000000001,"5min":0.22,"15min":0.216},"max":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":8.2509999999999994},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.51100000000000001},"min":{"1min":0.253,"5min":0.222,"15min":0.222},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":6.7370000000000001},"last":0.34899999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.35399999999999998,"15min":0.53900000000000003},"min":{"1min":0.247,"5min":0.23100000000000001,"15min":0.21099999999999999},"max":{"1min":0.53800000000000003,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.59099999999999997},"min":{"1min":0.27700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":7.8929999999999998},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57099999999999995,"15min":0.70299999999999996},"min":{"1min":0.45500000000000002,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.82499999999999996,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57299999999999995,"15min":0.754},"min":{"1min":0.441,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.80900000000000005,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.51300000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56100000000000005,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71299999999999997,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.76100000000000001},"min":{"1min":0.48399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.48699999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61499999999999999,"15min":0.80000000000000004},"min":{"1min":0.52200000000000002,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.83299999999999996,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.62,"15min":0.749},"min":{"1min":0.47399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82599999999999996,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.502}]},{"osd":7,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.60099999999999998,"15min":0.73999999999999999},"min":{"1min":0.48399999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85399999999999998,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.76800000000000002},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.75700000000000001,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.60399999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574949,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":868322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.311,"15min":0.32500000000000001},"min":{"1min":0.214,"5min":0.189,"15min":0.189},"max":{"1min":0.52500000000000002,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.35099999999999998},"min":{"1min":0.219,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.746},"last":0.23000000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.69099999999999995},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52100000000000002,"5min":0.63,"15min":11.967000000000001},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.34899999999999998,"15min":0.46999999999999997},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48899999999999999,"5min":0.53400000000000003,"15min":4.1100000000000003},"last":0.309}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34699999999999998,"15min":0.53700000000000003},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.50800000000000001,"5min":0.51600000000000001,"15min":6.3040000000000003},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.74299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59299999999999997,"5min":0.65900000000000003,"15min":11.935},"last":0.29499999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.55700000000000005,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.53700000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.38,"15min":0.378},"max":{"1min":0.83699999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54200000000000004,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.96099999999999997},"last":0.53000000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69599999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.58599999999999997},"min":{"1min":0.435,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.052},"last":0.623},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58099999999999996,"15min":0.76300000000000001},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":6.2649999999999997},"last":0.55800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574949,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303288,"kb_used_data":1904,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068552,"statfs":{"total":96636764160,"available":96326197248,"internally_reserved":0,"allocated":1949696,"data_stored":868504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.315,"15min":0.315},"min":{"1min":0.223,"5min":0.21199999999999999,"15min":0.21099999999999999},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32100000000000001},"min":{"1min":0.23999999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48799999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.313}]},{"osd":1,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.34599999999999997,"15min":0.33300000000000002},"min":{"1min":0.25700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.379},"min":{"1min":0.29199999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.72099999999999997},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.36199999999999999,"15min":0.35299999999999998},"min":{"1min":0.25800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57599999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.37},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.28999999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.50900000000000001,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.32600000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.53400000000000003},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77200000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.745,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.79400000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57299999999999995,"15min":0.55200000000000005},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.35899999999999999},"max":{"1min":0.82399999999999995,"5min":1.347,"15min":1.347},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.42199999999999999,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.83499999999999996,"5min":1.01,"15min":1.01},"last":0.65700000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.48399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86299999999999999,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.81200000000000006,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.71799999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78700000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.83699999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.56699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574947,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303296,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068544,"statfs":{"total":96636764160,"available":96326189056,"internally_reserved":0,"allocated":1957888,"data_stored":869962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.32400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66600000000000004,"5min":0.74199999999999999,"15min":0.747},"last":0.376},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51700000000000002,"15min":0.74099999999999999},"min":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77600000000000002,"5min":0.78400000000000003,"15min":7.6719999999999997},"last":0.53100000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52000000000000002,"15min":0.748},"min":{"1min":0.32000000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.755,"15min":7.7160000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66100000000000003,"5min":0.76900000000000002,"15min":0.86799999999999999},"last":0.56899999999999995}]},{"osd":2,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.73899999999999999,"5min":0.79800000000000004,"15min":7.7839999999999998},"last":0.50900000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.39200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.49099999999999999},"min":{"1min":0.35199999999999998,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58699999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.85199999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58399999999999996,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.61199999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.41399999999999998,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.626}]}]},{"osd":5,"up_from":13,"seq":55834574949,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303280,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068560,"statfs":{"total":96636764160,"available":96326205440,"internally_reserved":0,"allocated":1941504,"data_stored":868876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.47599999999999998,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.66100000000000003,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.51800000000000002},"min":{"1min":0.33000000000000002,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.68899999999999995,"5min":0.76700000000000002,"15min":0.79300000000000004},"last":0.60599999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51300000000000001,"15min":0.628},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63400000000000001,"5min":0.75700000000000001,"15min":5.2679999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.62},"min":{"1min":0.40500000000000003,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79200000000000004,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.65200000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52400000000000002,"15min":0.63700000000000001},"min":{"1min":0.39100000000000001,"5min":0.32900000000000001,"15min":0.28299999999999997},"max":{"1min":0.73299999999999998,"5min":0.76600000000000001,"15min":5.2960000000000003},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.38500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.58299999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55900000000000005,"15min":0.66900000000000004},"min":{"1min":0.41999999999999998,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":5.3330000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56399999999999995,"15min":0.68000000000000005},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.27700000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.67000000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.48899999999999999,"15min":0.498},"min":{"1min":0.33400000000000002,"5min":0.29599999999999999,"15min":0.28699999999999998},"max":{"1min":0.67000000000000004,"5min":0.72499999999999998,"15min":0.82999999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.34499999999999997,"15min":0.32200000000000001},"max":{"1min":0.77200000000000002,"5min":0.88,"15min":1.0800000000000001},"last":0.38900000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.39900000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78900000000000003,"5min":0.82499999999999996,"15min":0.89800000000000002},"last":0.51500000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.32300000000000001,"15min":0.30099999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98399999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.44}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:42:32.672 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:42:32.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.798+0000 7f69104a3700 1 Processor -- start 2023-10-10T20:42:32.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.798+0000 7f69104a3700 1 -- start start 2023-10-10T20:42:32.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.799+0000 7f69104a3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f690806c1e0 0x7f69080cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:32.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.799+0000 7f69104a3700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f69080cbc10 0x7f690815baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:32.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.799+0000 7f69104a3700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690806c750 con 0x7f6908157940 2023-10-10T20:42:32.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.799+0000 7f69104a3700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690806ca30 con 0x7f69080cbc10 2023-10-10T20:42:32.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.799+0000 7f69104a3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690806cd10 con 0x7f690806c1e0 2023-10-10T20:42:32.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.800+0000 7f690da3e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f690806c1e0 0x7f69080cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:32.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.800+0000 7f690ea40700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f69080cbc10 0x7f690815baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:32.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.800+0000 7f690da3e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f690806c1e0 0x7f69080cb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:57822/0 (socket says 172.21.15.89:57822) 2023-10-10T20:42:32.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.800+0000 7f690ea40700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f69080cbc10 0x7f690815baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:34076/0 (socket says 172.21.15.89:34076) 2023-10-10T20:42:32.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.800+0000 7f690da3e700 1 -- 172.21.15.89:0/3602058824 learned_addr learned my addr 172.21.15.89:0/3602058824 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:32.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.800+0000 7f690da3e700 1 -- 172.21.15.89:0/3602058824 >> v2:172.21.15.111:3300/0 conn(0x7f69080cbc10 msgr2=0x7f690815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:32.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.800+0000 7f690da3e700 1 --2- 172.21.15.89:0/3602058824 >> v2:172.21.15.111:3300/0 conn(0x7f69080cbc10 0x7f690815baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:32.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.800+0000 7f690da3e700 1 -- 172.21.15.89:0/3602058824 >> v1:172.21.15.111:6789/0 conn(0x7f6908157940 legacy=0x7f690806f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:32.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.800+0000 7f690da3e700 1 -- 172.21.15.89:0/3602058824 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690806cff0 con 0x7f690806c1e0 2023-10-10T20:42:32.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.800+0000 7f690da3e700 1 --2- 172.21.15.89:0/3602058824 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f690806c1e0 0x7f69080cb6d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f68f80099b0 tx=0x7f68f8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7788a7a57b7d25c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:32.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.801+0000 7f690d23d700 1 -- 172.21.15.89:0/3602058824 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8009ea0 con 0x7f690806c1e0 2023-10-10T20:42:32.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.801+0000 7f690d23d700 1 -- 172.21.15.89:0/3602058824 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f800b860 con 0x7f690806c1e0 2023-10-10T20:42:32.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.801+0000 7f690d23d700 1 -- 172.21.15.89:0/3602058824 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f800bc00 con 0x7f690806c1e0 2023-10-10T20:42:32.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.801+0000 7f69104a3700 1 -- 172.21.15.89:0/3602058824 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f690806c1e0 msgr2=0x7f69080cb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:32.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.801+0000 7f69104a3700 1 --2- 172.21.15.89:0/3602058824 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f690806c1e0 0x7f69080cb6d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f68f80099b0 tx=0x7f68f8009250 comp rx=0 tx=0).stop 2023-10-10T20:42:32.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.801+0000 7f69104a3700 1 -- 172.21.15.89:0/3602058824 shutdown_connections 2023-10-10T20:42:32.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.801+0000 7f69104a3700 1 --2- 172.21.15.89:0/3602058824 >> v2:172.21.15.111:3300/0 conn(0x7f69080cbc10 0x7f690815baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:32.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.801+0000 7f69104a3700 1 --2- 172.21.15.89:0/3602058824 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f690806c1e0 0x7f69080cb6d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:32.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.801+0000 7f69104a3700 1 -- 172.21.15.89:0/3602058824 >> 172.21.15.89:0/3602058824 conn(0x7f69080cd480 msgr2=0x7f69080cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:32.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.801+0000 7f69104a3700 1 -- 172.21.15.89:0/3602058824 shutdown_connections 2023-10-10T20:42:32.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.801+0000 7f69104a3700 1 -- 172.21.15.89:0/3602058824 wait complete. 2023-10-10T20:42:32.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.802+0000 7f69104a3700 1 Processor -- start 2023-10-10T20:42:32.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.803+0000 7f69104a3700 1 -- start start 2023-10-10T20:42:32.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.803+0000 7f69104a3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f690806c1e0 0x7f6908139d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:32.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.803+0000 7f69104a3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69080cbc10 0x7f690815dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:32.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.803+0000 7f69104a3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6908157940 0x7f690815e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:32.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.803+0000 7f69104a3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69081507a0 con 0x7f690806c1e0 2023-10-10T20:42:32.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.803+0000 7f69104a3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690814fe70 con 0x7f69080cbc10 2023-10-10T20:42:32.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.803+0000 7f69104a3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6908150150 con 0x7f6908157940 2023-10-10T20:42:32.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.803+0000 7f690e23f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f690806c1e0 0x7f6908139d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:32.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.803+0000 7f690e23f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f690806c1e0 0x7f6908139d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59162/0 (socket says 172.21.15.89:59162) 2023-10-10T20:42:32.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.803+0000 7f690e23f700 1 -- 172.21.15.89:0/288474214 learned_addr learned my addr 172.21.15.89:0/288474214 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:32.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.804+0000 7f690ea40700 1 --2- 172.21.15.89:0/288474214 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6908157940 0x7f690815e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:32.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.804+0000 7f690da3e700 1 --2- 172.21.15.89:0/288474214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69080cbc10 0x7f690815dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:32.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.804+0000 7f690e23f700 1 -- 172.21.15.89:0/288474214 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6908157940 msgr2=0x7f690815e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:32.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.804+0000 7f690e23f700 1 --2- 172.21.15.89:0/288474214 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6908157940 0x7f690815e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:32.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.804+0000 7f690e23f700 1 -- 172.21.15.89:0/288474214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69080cbc10 msgr2=0x7f690815dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:32.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.804+0000 7f690e23f700 1 --2- 172.21.15.89:0/288474214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69080cbc10 0x7f690815dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:32.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.804+0000 7f690e23f700 1 -- 172.21.15.89:0/288474214 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f80094f0 con 0x7f690806c1e0 2023-10-10T20:42:32.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.804+0000 7f690e23f700 1 --2- 172.21.15.89:0/288474214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f690806c1e0 0x7f6908139d40 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f690400c4e0 tx=0x7f690400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:32.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.804+0000 7f68fe7fc700 1 -- 172.21.15.89:0/288474214 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6904011590 con 0x7f690806c1e0 2023-10-10T20:42:32.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.804+0000 7f68fe7fc700 1 -- 172.21.15.89:0/288474214 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6904011710 con 0x7f690806c1e0 2023-10-10T20:42:32.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.804+0000 7f69104a3700 1 -- 172.21.15.89:0/288474214 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6908150430 con 0x7f690806c1e0 2023-10-10T20:42:32.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.804+0000 7f68fe7fc700 1 -- 172.21.15.89:0/288474214 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6904011ad0 con 0x7f690806c1e0 2023-10-10T20:42:32.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.804+0000 7f69104a3700 1 -- 172.21.15.89:0/288474214 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690813a280 con 0x7f690806c1e0 2023-10-10T20:42:32.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.805+0000 7f69104a3700 1 -- 172.21.15.89:0/288474214 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690804c3c0 con 0x7f690806c1e0 2023-10-10T20:42:32.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.805+0000 7f68fe7fc700 1 -- 172.21.15.89:0/288474214 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f690401b3b0 con 0x7f690806c1e0 2023-10-10T20:42:32.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.806+0000 7f68fe7fc700 1 --2- 172.21.15.89:0/288474214 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f68ec067320 0x7f68ec0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:32.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.806+0000 7f68fe7fc700 1 -- 172.21.15.89:0/288474214 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f69040841b0 con 0x7f690806c1e0 2023-10-10T20:42:32.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.806+0000 7f690da3e700 1 --2- 172.21.15.89:0/288474214 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f68ec067320 0x7f68ec0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:32.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.806+0000 7f690da3e700 1 --2- 172.21.15.89:0/288474214 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f68ec067320 0x7f68ec0697d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f68f80099b0 tx=0x7f68f8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:32.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.810+0000 7f68fe7fc700 1 -- 172.21.15.89:0/288474214 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6904050e40 con 0x7f690806c1e0 2023-10-10T20:42:32.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.990+0000 7f69104a3700 1 -- 172.21.15.89:0/288474214 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69081508e0 con 0x7f68ec067320 2023-10-10T20:42:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:32.998+0000 7f68fe7fc700 1 -- 172.21.15.89:0/288474214 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290160 (secure 0 0 0) 0x7f68f800b9d0 con 0x7f68ec067320 2023-10-10T20:42:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:33.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.001+0000 7f69104a3700 1 -- 172.21.15.89:0/288474214 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f68ec067320 msgr2=0x7f68ec0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:33.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.001+0000 7f69104a3700 1 --2- 172.21.15.89:0/288474214 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f68ec067320 0x7f68ec0697d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f68f80099b0 tx=0x7f68f8017000 comp rx=0 tx=0).stop 2023-10-10T20:42:33.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.001+0000 7f69104a3700 1 -- 172.21.15.89:0/288474214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f690806c1e0 msgr2=0x7f6908139d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:33.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.001+0000 7f69104a3700 1 --2- 172.21.15.89:0/288474214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f690806c1e0 0x7f6908139d40 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f690400c4e0 tx=0x7f690400c8a0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.001+0000 7f69104a3700 1 -- 172.21.15.89:0/288474214 shutdown_connections 2023-10-10T20:42:33.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.001+0000 7f69104a3700 1 --2- 172.21.15.89:0/288474214 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6908157940 0x7f690815e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.001+0000 7f69104a3700 1 --2- 172.21.15.89:0/288474214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f690806c1e0 0x7f6908139d40 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.001+0000 7f69104a3700 1 --2- 172.21.15.89:0/288474214 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f68ec067320 0x7f68ec0697d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.001+0000 7f69104a3700 1 --2- 172.21.15.89:0/288474214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69080cbc10 0x7f690815dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.001+0000 7f69104a3700 1 -- 172.21.15.89:0/288474214 >> 172.21.15.89:0/288474214 conn(0x7f69080cd480 msgr2=0x7f6908159be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:33.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.001+0000 7f69104a3700 1 -- 172.21.15.89:0/288474214 shutdown_connections 2023-10-10T20:42:33.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.001+0000 7f69104a3700 1 -- 172.21.15.89:0/288474214 wait complete. 2023-10-10T20:42:33.010 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:42:33.019 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2023-10-10T20:42:32.746753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427428,"kb_used_data":16356,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547292,"statfs":{"total":773094113280,"available":770608427008,"internally_reserved":0,"allocated":16748544,"data_stored":8130963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.651502"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121111+0000","last_change":"2023-10-10T20:42:24.113315+0000","last_active":"2023-10-10T20:42:24.121111+0000","last_peered":"2023-10-10T20:42:24.121111+0000","last_clean":"2023-10-10T20:42:24.121111+0000","last_became_active":"2023-10-10T20:42:24.113091+0000","last_became_peered":"2023-10-10T20:42:24.113091+0000","last_unstale":"2023-10-10T20:42:24.121111+0000","last_undegraded":"2023-10-10T20:42:24.121111+0000","last_fullsized":"2023-10-10T20:42:24.121111+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:25:55.562871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114620+0000","last_change":"2023-10-10T20:42:24.111977+0000","last_active":"2023-10-10T20:42:24.114620+0000","last_peered":"2023-10-10T20:42:24.114620+0000","last_clean":"2023-10-10T20:42:24.114620+0000","last_became_active":"2023-10-10T20:42:24.111740+0000","last_became_peered":"2023-10-10T20:42:24.111740+0000","last_unstale":"2023-10-10T20:42:24.114620+0000","last_undegraded":"2023-10-10T20:42:24.114620+0000","last_fullsized":"2023-10-10T20:42:24.114620+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:43:44.092334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.107328+0000","last_change":"2023-10-10T20:42:24.101396+0000","last_active":"2023-10-10T20:42:24.107328+0000","last_peered":"2023-10-10T20:42:24.107328+0000","last_clean":"2023-10-10T20:42:24.107328+0000","last_became_active":"2023-10-10T20:42:24.101140+0000","last_became_peered":"2023-10-10T20:42:24.101140+0000","last_unstale":"2023-10-10T20:42:24.107328+0000","last_undegraded":"2023-10-10T20:42:24.107328+0000","last_fullsized":"2023-10-10T20:42:24.107328+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:22:58.559364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113453+0000","last_change":"2023-10-10T20:42:24.107229+0000","last_active":"2023-10-10T20:42:24.113453+0000","last_peered":"2023-10-10T20:42:24.113453+0000","last_clean":"2023-10-10T20:42:24.113453+0000","last_became_active":"2023-10-10T20:42:24.106160+0000","last_became_peered":"2023-10-10T20:42:24.106160+0000","last_unstale":"2023-10-10T20:42:24.113453+0000","last_undegraded":"2023-10-10T20:42:24.113453+0000","last_fullsized":"2023-10-10T20:42:24.113453+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:04:22.281292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114347+0000","last_change":"2023-10-10T20:42:24.112159+0000","last_active":"2023-10-10T20:42:24.114347+0000","last_peered":"2023-10-10T20:42:24.114347+0000","last_clean":"2023-10-10T20:42:24.114347+0000","last_became_active":"2023-10-10T20:42:24.111823+0000","last_became_peered":"2023-10-10T20:42:24.111823+0000","last_unstale":"2023-10-10T20:42:24.114347+0000","last_undegraded":"2023-10-10T20:42:24.114347+0000","last_fullsized":"2023-10-10T20:42:24.114347+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:37:14.508787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.117428+0000","last_change":"2023-10-10T20:42:24.114360+0000","last_active":"2023-10-10T20:42:24.117428+0000","last_peered":"2023-10-10T20:42:24.117428+0000","last_clean":"2023-10-10T20:42:24.117428+0000","last_became_active":"2023-10-10T20:42:24.114175+0000","last_became_peered":"2023-10-10T20:42:24.114175+0000","last_unstale":"2023-10-10T20:42:24.117428+0000","last_undegraded":"2023-10-10T20:42:24.117428+0000","last_fullsized":"2023-10-10T20:42:24.117428+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:39:43.171315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121416+0000","last_change":"2023-10-10T20:42:24.116618+0000","last_active":"2023-10-10T20:42:24.121416+0000","last_peered":"2023-10-10T20:42:24.121416+0000","last_clean":"2023-10-10T20:42:24.121416+0000","last_became_active":"2023-10-10T20:42:24.115742+0000","last_became_peered":"2023-10-10T20:42:24.115742+0000","last_unstale":"2023-10-10T20:42:24.121416+0000","last_undegraded":"2023-10-10T20:42:24.121416+0000","last_fullsized":"2023-10-10T20:42:24.121416+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:49:38.681149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114418+0000","last_change":"2023-10-10T20:42:24.112723+0000","last_active":"2023-10-10T20:42:24.114418+0000","last_peered":"2023-10-10T20:42:24.114418+0000","last_clean":"2023-10-10T20:42:24.114418+0000","last_became_active":"2023-10-10T20:42:24.112506+0000","last_became_peered":"2023-10-10T20:42:24.112506+0000","last_unstale":"2023-10-10T20:42:24.114418+0000","last_undegraded":"2023-10-10T20:42:24.114418+0000","last_fullsized":"2023-10-10T20:42:24.114418+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:46:59.225281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.107890+0000","last_change":"2023-10-10T20:42:24.101774+0000","last_active":"2023-10-10T20:42:24.107890+0000","last_peered":"2023-10-10T20:42:24.107890+0000","last_clean":"2023-10-10T20:42:24.107890+0000","last_became_active":"2023-10-10T20:42:24.101500+0000","last_became_peered":"2023-10-10T20:42:24.101500+0000","last_unstale":"2023-10-10T20:42:24.107890+0000","last_undegraded":"2023-10-10T20:42:24.107890+0000","last_fullsized":"2023-10-10T20:42:24.107890+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:32:22.661032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121119+0000","last_change":"2023-10-10T20:42:24.117698+0000","last_active":"2023-10-10T20:42:24.121119+0000","last_peered":"2023-10-10T20:42:24.121119+0000","last_clean":"2023-10-10T20:42:24.121119+0000","last_became_active":"2023-10-10T20:42:24.117453+0000","last_became_peered":"2023-10-10T20:42:24.117453+0000","last_unstale":"2023-10-10T20:42:24.121119+0000","last_undegraded":"2023-10-10T20:42:24.121119+0000","last_fullsized":"2023-10-10T20:42:24.121119+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:14.819673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.112709+0000","last_change":"2023-10-10T20:42:24.108830+0000","last_active":"2023-10-10T20:42:24.112709+0000","last_peered":"2023-10-10T20:42:24.112709+0000","last_clean":"2023-10-10T20:42:24.112709+0000","last_became_active":"2023-10-10T20:42:24.108592+0000","last_became_peered":"2023-10-10T20:42:24.108592+0000","last_unstale":"2023-10-10T20:42:24.112709+0000","last_undegraded":"2023-10-10T20:42:24.112709+0000","last_fullsized":"2023-10-10T20:42:24.112709+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:32:50.550864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.128843+0000","last_change":"2023-10-10T20:42:24.124278+0000","last_active":"2023-10-10T20:42:24.128843+0000","last_peered":"2023-10-10T20:42:24.128843+0000","last_clean":"2023-10-10T20:42:24.128843+0000","last_became_active":"2023-10-10T20:42:24.123491+0000","last_became_peered":"2023-10-10T20:42:24.123491+0000","last_unstale":"2023-10-10T20:42:24.128843+0000","last_undegraded":"2023-10-10T20:42:24.128843+0000","last_fullsized":"2023-10-10T20:42:24.128843+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:47:26.708886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.128950+0000","last_change":"2023-10-10T20:42:24.124820+0000","last_active":"2023-10-10T20:42:24.128950+0000","last_peered":"2023-10-10T20:42:24.128950+0000","last_clean":"2023-10-10T20:42:24.128950+0000","last_became_active":"2023-10-10T20:42:24.124570+0000","last_became_peered":"2023-10-10T20:42:24.124570+0000","last_unstale":"2023-10-10T20:42:24.128950+0000","last_undegraded":"2023-10-10T20:42:24.128950+0000","last_fullsized":"2023-10-10T20:42:24.128950+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:21:59.491769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.112675+0000","last_change":"2023-10-10T20:42:24.110144+0000","last_active":"2023-10-10T20:42:24.112675+0000","last_peered":"2023-10-10T20:42:24.112675+0000","last_clean":"2023-10-10T20:42:24.112675+0000","last_became_active":"2023-10-10T20:42:24.109876+0000","last_became_peered":"2023-10-10T20:42:24.109876+0000","last_unstale":"2023-10-10T20:42:24.112675+0000","last_undegraded":"2023-10-10T20:42:24.112675+0000","last_fullsized":"2023-10-10T20:42:24.112675+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:40:58.862440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.115631+0000","last_change":"2023-10-10T20:42:24.112507+0000","last_active":"2023-10-10T20:42:24.115631+0000","last_peered":"2023-10-10T20:42:24.115631+0000","last_clean":"2023-10-10T20:42:24.115631+0000","last_became_active":"2023-10-10T20:42:24.112275+0000","last_became_peered":"2023-10-10T20:42:24.112275+0000","last_unstale":"2023-10-10T20:42:24.115631+0000","last_undegraded":"2023-10-10T20:42:24.115631+0000","last_fullsized":"2023-10-10T20:42:24.115631+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:06.445923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126524+0000","last_change":"2023-10-10T20:42:24.125813+0000","last_active":"2023-10-10T20:42:24.126524+0000","last_peered":"2023-10-10T20:42:24.126524+0000","last_clean":"2023-10-10T20:42:24.126524+0000","last_became_active":"2023-10-10T20:42:24.125554+0000","last_became_peered":"2023-10-10T20:42:24.125554+0000","last_unstale":"2023-10-10T20:42:24.126524+0000","last_undegraded":"2023-10-10T20:42:24.126524+0000","last_fullsized":"2023-10-10T20:42:24.126524+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:40:58.862440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.117282+0000","last_change":"2023-10-10T20:42:24.113162+0000","last_active":"2023-10-10T20:42:24.117282+0000","last_peered":"2023-10-10T20:42:24.117282+0000","last_clean":"2023-10-10T20:42:24.117282+0000","last_became_active":"2023-10-10T20:42:24.112919+0000","last_became_peered":"2023-10-10T20:42:24.112919+0000","last_unstale":"2023-10-10T20:42:24.117282+0000","last_undegraded":"2023-10-10T20:42:24.117282+0000","last_fullsized":"2023-10-10T20:42:24.117282+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:58:46.317589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.118301+0000","last_change":"2023-10-10T20:42:24.111610+0000","last_active":"2023-10-10T20:42:24.118301+0000","last_peered":"2023-10-10T20:42:24.118301+0000","last_clean":"2023-10-10T20:42:24.118301+0000","last_became_active":"2023-10-10T20:42:24.111317+0000","last_became_peered":"2023-10-10T20:42:24.111317+0000","last_unstale":"2023-10-10T20:42:24.118301+0000","last_undegraded":"2023-10-10T20:42:24.118301+0000","last_fullsized":"2023-10-10T20:42:24.118301+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:49:38.681149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.118411+0000","last_change":"2023-10-10T20:42:24.111047+0000","last_active":"2023-10-10T20:42:24.118411+0000","last_peered":"2023-10-10T20:42:24.118411+0000","last_clean":"2023-10-10T20:42:24.118411+0000","last_became_active":"2023-10-10T20:42:24.110810+0000","last_became_peered":"2023-10-10T20:42:24.110810+0000","last_unstale":"2023-10-10T20:42:24.118411+0000","last_undegraded":"2023-10-10T20:42:24.118411+0000","last_fullsized":"2023-10-10T20:42:24.118411+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:47.040763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113486+0000","last_change":"2023-10-10T20:42:24.109452+0000","last_active":"2023-10-10T20:42:24.113486+0000","last_peered":"2023-10-10T20:42:24.113486+0000","last_clean":"2023-10-10T20:42:24.113486+0000","last_became_active":"2023-10-10T20:42:24.109179+0000","last_became_peered":"2023-10-10T20:42:24.109179+0000","last_unstale":"2023-10-10T20:42:24.113486+0000","last_undegraded":"2023-10-10T20:42:24.113486+0000","last_fullsized":"2023-10-10T20:42:24.113486+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:59.819150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.112215+0000","last_change":"2023-10-10T20:42:24.105755+0000","last_active":"2023-10-10T20:42:24.112215+0000","last_peered":"2023-10-10T20:42:24.112215+0000","last_clean":"2023-10-10T20:42:24.112215+0000","last_became_active":"2023-10-10T20:42:24.104420+0000","last_became_peered":"2023-10-10T20:42:24.104420+0000","last_unstale":"2023-10-10T20:42:24.112215+0000","last_undegraded":"2023-10-10T20:42:24.112215+0000","last_fullsized":"2023-10-10T20:42:24.112215+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:03:33.363174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113655+0000","last_change":"2023-10-10T20:42:24.110180+0000","last_active":"2023-10-10T20:42:24.113655+0000","last_peered":"2023-10-10T20:42:24.113655+0000","last_clean":"2023-10-10T20:42:24.113655+0000","last_became_active":"2023-10-10T20:42:24.109893+0000","last_became_peered":"2023-10-10T20:42:24.109893+0000","last_unstale":"2023-10-10T20:42:24.113655+0000","last_undegraded":"2023-10-10T20:42:24.113655+0000","last_fullsized":"2023-10-10T20:42:24.113655+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:09:32.731249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.117221+0000","last_change":"2023-10-10T20:42:24.116069+0000","last_active":"2023-10-10T20:42:24.117221+0000","last_peered":"2023-10-10T20:42:24.117221+0000","last_clean":"2023-10-10T20:42:24.117221+0000","last_became_active":"2023-10-10T20:42:24.115782+0000","last_became_peered":"2023-10-10T20:42:24.115782+0000","last_unstale":"2023-10-10T20:42:24.117221+0000","last_undegraded":"2023-10-10T20:42:24.117221+0000","last_fullsized":"2023-10-10T20:42:24.117221+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:14:18.635698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.107687+0000","last_change":"2023-10-10T20:42:24.102224+0000","last_active":"2023-10-10T20:42:24.107687+0000","last_peered":"2023-10-10T20:42:24.107687+0000","last_clean":"2023-10-10T20:42:24.107687+0000","last_became_active":"2023-10-10T20:42:24.101963+0000","last_became_peered":"2023-10-10T20:42:24.101963+0000","last_unstale":"2023-10-10T20:42:24.107687+0000","last_undegraded":"2023-10-10T20:42:24.107687+0000","last_fullsized":"2023-10-10T20:42:24.107687+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:38:54.118178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113541+0000","last_change":"2023-10-10T20:42:24.109079+0000","last_active":"2023-10-10T20:42:24.113541+0000","last_peered":"2023-10-10T20:42:24.113541+0000","last_clean":"2023-10-10T20:42:24.113541+0000","last_became_active":"2023-10-10T20:42:24.108731+0000","last_became_peered":"2023-10-10T20:42:24.108731+0000","last_unstale":"2023-10-10T20:42:24.113541+0000","last_undegraded":"2023-10-10T20:42:24.113541+0000","last_fullsized":"2023-10-10T20:42:24.113541+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:06:21.357294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121390+0000","last_change":"2023-10-10T20:42:24.115936+0000","last_active":"2023-10-10T20:42:24.121390+0000","last_peered":"2023-10-10T20:42:24.121390+0000","last_clean":"2023-10-10T20:42:24.121390+0000","last_became_active":"2023-10-10T20:42:24.115659+0000","last_became_peered":"2023-10-10T20:42:24.115659+0000","last_unstale":"2023-10-10T20:42:24.121390+0000","last_undegraded":"2023-10-10T20:42:24.121390+0000","last_fullsized":"2023-10-10T20:42:24.121390+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:47.040763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126170+0000","last_change":"2023-10-10T20:42:24.123600+0000","last_active":"2023-10-10T20:42:24.126170+0000","last_peered":"2023-10-10T20:42:24.126170+0000","last_clean":"2023-10-10T20:42:24.126170+0000","last_became_active":"2023-10-10T20:42:24.123226+0000","last_became_peered":"2023-10-10T20:42:24.123226+0000","last_unstale":"2023-10-10T20:42:24.126170+0000","last_undegraded":"2023-10-10T20:42:24.126170+0000","last_fullsized":"2023-10-10T20:42:24.126170+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:46:15.751003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.112765+0000","last_change":"2023-10-10T20:42:24.110429+0000","last_active":"2023-10-10T20:42:24.112765+0000","last_peered":"2023-10-10T20:42:24.112765+0000","last_clean":"2023-10-10T20:42:24.112765+0000","last_became_active":"2023-10-10T20:42:24.110171+0000","last_became_peered":"2023-10-10T20:42:24.110171+0000","last_unstale":"2023-10-10T20:42:24.112765+0000","last_undegraded":"2023-10-10T20:42:24.112765+0000","last_fullsized":"2023-10-10T20:42:24.112765+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:02.631228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.128984+0000","last_change":"2023-10-10T20:42:24.125254+0000","last_active":"2023-10-10T20:42:24.128984+0000","last_peered":"2023-10-10T20:42:24.128984+0000","last_clean":"2023-10-10T20:42:24.128984+0000","last_became_active":"2023-10-10T20:42:24.124838+0000","last_became_peered":"2023-10-10T20:42:24.124838+0000","last_unstale":"2023-10-10T20:42:24.128984+0000","last_undegraded":"2023-10-10T20:42:24.128984+0000","last_fullsized":"2023-10-10T20:42:24.128984+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:13.353418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114550+0000","last_change":"2023-10-10T20:42:24.110653+0000","last_active":"2023-10-10T20:42:24.114550+0000","last_peered":"2023-10-10T20:42:24.114550+0000","last_clean":"2023-10-10T20:42:24.114550+0000","last_became_active":"2023-10-10T20:42:24.110226+0000","last_became_peered":"2023-10-10T20:42:24.110226+0000","last_unstale":"2023-10-10T20:42:24.114550+0000","last_undegraded":"2023-10-10T20:42:24.114550+0000","last_fullsized":"2023-10-10T20:42:24.114550+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:17:02.631228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.120869+0000","last_change":"2023-10-10T20:42:24.112486+0000","last_active":"2023-10-10T20:42:24.120869+0000","last_peered":"2023-10-10T20:42:24.120869+0000","last_clean":"2023-10-10T20:42:24.120869+0000","last_became_active":"2023-10-10T20:42:24.112258+0000","last_became_peered":"2023-10-10T20:42:24.112258+0000","last_unstale":"2023-10-10T20:42:24.120869+0000","last_undegraded":"2023-10-10T20:42:24.120869+0000","last_fullsized":"2023-10-10T20:42:24.120869+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:46:07.440450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121137+0000","last_change":"2023-10-10T20:42:24.115300+0000","last_active":"2023-10-10T20:42:24.121137+0000","last_peered":"2023-10-10T20:42:24.121137+0000","last_clean":"2023-10-10T20:42:24.121137+0000","last_became_active":"2023-10-10T20:42:24.115040+0000","last_became_peered":"2023-10-10T20:42:24.115040+0000","last_unstale":"2023-10-10T20:42:24.121137+0000","last_undegraded":"2023-10-10T20:42:24.121137+0000","last_fullsized":"2023-10-10T20:42:24.121137+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:46:19.157900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.111054+0000","last_change":"2023-10-10T20:42:24.104338+0000","last_active":"2023-10-10T20:42:24.111054+0000","last_peered":"2023-10-10T20:42:24.111054+0000","last_clean":"2023-10-10T20:42:24.111054+0000","last_became_active":"2023-10-10T20:42:24.104037+0000","last_became_peered":"2023-10-10T20:42:24.104037+0000","last_unstale":"2023-10-10T20:42:24.111054+0000","last_undegraded":"2023-10-10T20:42:24.111054+0000","last_fullsized":"2023-10-10T20:42:24.111054+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:25:09.330944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"152'4","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.131700+0000","last_change":"2023-10-10T20:42:23.105959+0000","last_active":"2023-10-10T20:42:24.131700+0000","last_peered":"2023-10-10T20:42:24.131700+0000","last_clean":"2023-10-10T20:42:24.131700+0000","last_became_active":"2023-10-10T20:42:23.105709+0000","last_became_peered":"2023-10-10T20:42:23.105709+0000","last_unstale":"2023-10-10T20:42:24.131700+0000","last_undegraded":"2023-10-10T20:42:24.131700+0000","last_fullsized":"2023-10-10T20:42:24.131700+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:42.951721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.128823+0000","last_change":"2023-10-10T20:42:24.124965+0000","last_active":"2023-10-10T20:42:24.128823+0000","last_peered":"2023-10-10T20:42:24.128823+0000","last_clean":"2023-10-10T20:42:24.128823+0000","last_became_active":"2023-10-10T20:42:24.124706+0000","last_became_peered":"2023-10-10T20:42:24.124706+0000","last_unstale":"2023-10-10T20:42:24.128823+0000","last_undegraded":"2023-10-10T20:42:24.128823+0000","last_fullsized":"2023-10-10T20:42:24.128823+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:47:26.708886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.120664+0000","last_change":"2023-10-10T20:42:23.105858+0000","last_active":"2023-10-10T20:42:24.120664+0000","last_peered":"2023-10-10T20:42:24.120664+0000","last_clean":"2023-10-10T20:42:24.120664+0000","last_became_active":"2023-10-10T20:42:23.105592+0000","last_became_peered":"2023-10-10T20:42:23.105592+0000","last_unstale":"2023-10-10T20:42:24.120664+0000","last_undegraded":"2023-10-10T20:42:24.120664+0000","last_fullsized":"2023-10-10T20:42:24.120664+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:54.378347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121344+0000","last_change":"2023-10-10T20:42:24.115929+0000","last_active":"2023-10-10T20:42:24.121344+0000","last_peered":"2023-10-10T20:42:24.121344+0000","last_clean":"2023-10-10T20:42:24.121344+0000","last_became_active":"2023-10-10T20:42:24.115637+0000","last_became_peered":"2023-10-10T20:42:24.115637+0000","last_unstale":"2023-10-10T20:42:24.121344+0000","last_undegraded":"2023-10-10T20:42:24.121344+0000","last_fullsized":"2023-10-10T20:42:24.121344+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:39:10.428774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.124476+0000","last_change":"2023-10-10T20:42:23.103827+0000","last_active":"2023-10-10T20:42:24.124476+0000","last_peered":"2023-10-10T20:42:24.124476+0000","last_clean":"2023-10-10T20:42:24.124476+0000","last_became_active":"2023-10-10T20:42:23.103329+0000","last_became_peered":"2023-10-10T20:42:23.103329+0000","last_unstale":"2023-10-10T20:42:24.124476+0000","last_undegraded":"2023-10-10T20:42:24.124476+0000","last_fullsized":"2023-10-10T20:42:24.124476+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:42:08.332947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121209+0000","last_change":"2023-10-10T20:42:24.114480+0000","last_active":"2023-10-10T20:42:24.121209+0000","last_peered":"2023-10-10T20:42:24.121209+0000","last_clean":"2023-10-10T20:42:24.121209+0000","last_became_active":"2023-10-10T20:42:24.114244+0000","last_became_peered":"2023-10-10T20:42:24.114244+0000","last_unstale":"2023-10-10T20:42:24.121209+0000","last_undegraded":"2023-10-10T20:42:24.121209+0000","last_fullsized":"2023-10-10T20:42:24.121209+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:00:26.956205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.124758+0000","last_change":"2023-10-10T20:42:23.104651+0000","last_active":"2023-10-10T20:42:24.124758+0000","last_peered":"2023-10-10T20:42:24.124758+0000","last_clean":"2023-10-10T20:42:24.124758+0000","last_became_active":"2023-10-10T20:42:23.104297+0000","last_became_peered":"2023-10-10T20:42:23.104297+0000","last_unstale":"2023-10-10T20:42:24.124758+0000","last_undegraded":"2023-10-10T20:42:24.124758+0000","last_fullsized":"2023-10-10T20:42:24.124758+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:49:42.586346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114531+0000","last_change":"2023-10-10T20:42:24.112093+0000","last_active":"2023-10-10T20:42:24.114531+0000","last_peered":"2023-10-10T20:42:24.114531+0000","last_clean":"2023-10-10T20:42:24.114531+0000","last_became_active":"2023-10-10T20:42:24.111868+0000","last_became_peered":"2023-10-10T20:42:24.111868+0000","last_unstale":"2023-10-10T20:42:24.114531+0000","last_undegraded":"2023-10-10T20:42:24.114531+0000","last_fullsized":"2023-10-10T20:42:24.114531+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:30.464320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.125138+0000","last_change":"2023-10-10T20:42:23.105709+0000","last_active":"2023-10-10T20:42:24.125138+0000","last_peered":"2023-10-10T20:42:24.125138+0000","last_clean":"2023-10-10T20:42:24.125138+0000","last_became_active":"2023-10-10T20:42:23.105538+0000","last_became_peered":"2023-10-10T20:42:23.105538+0000","last_unstale":"2023-10-10T20:42:24.125138+0000","last_undegraded":"2023-10-10T20:42:24.125138+0000","last_fullsized":"2023-10-10T20:42:24.125138+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:26:59.934888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113606+0000","last_change":"2023-10-10T20:42:24.109974+0000","last_active":"2023-10-10T20:42:24.113606+0000","last_peered":"2023-10-10T20:42:24.113606+0000","last_clean":"2023-10-10T20:42:24.113606+0000","last_became_active":"2023-10-10T20:42:24.109707+0000","last_became_peered":"2023-10-10T20:42:24.109707+0000","last_unstale":"2023-10-10T20:42:24.113606+0000","last_undegraded":"2023-10-10T20:42:24.113606+0000","last_fullsized":"2023-10-10T20:42:24.113606+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:34.653111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.101968+0000","last_change":"2023-10-10T20:42:23.109627+0000","last_active":"2023-10-10T20:42:24.101968+0000","last_peered":"2023-10-10T20:42:24.101968+0000","last_clean":"2023-10-10T20:42:24.101968+0000","last_became_active":"2023-10-10T20:42:23.109387+0000","last_became_peered":"2023-10-10T20:42:23.109387+0000","last_unstale":"2023-10-10T20:42:24.101968+0000","last_undegraded":"2023-10-10T20:42:24.101968+0000","last_fullsized":"2023-10-10T20:42:24.101968+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:12.342683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114505+0000","last_change":"2023-10-10T20:42:24.111171+0000","last_active":"2023-10-10T20:42:24.114505+0000","last_peered":"2023-10-10T20:42:24.114505+0000","last_clean":"2023-10-10T20:42:24.114505+0000","last_became_active":"2023-10-10T20:42:24.110894+0000","last_became_peered":"2023-10-10T20:42:24.110894+0000","last_unstale":"2023-10-10T20:42:24.114505+0000","last_undegraded":"2023-10-10T20:42:24.114505+0000","last_fullsized":"2023-10-10T20:42:24.114505+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:29:16.754131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.102665+0000","last_change":"2023-10-10T20:42:23.105273+0000","last_active":"2023-10-10T20:42:24.102665+0000","last_peered":"2023-10-10T20:42:24.102665+0000","last_clean":"2023-10-10T20:42:24.102665+0000","last_became_active":"2023-10-10T20:42:23.104999+0000","last_became_peered":"2023-10-10T20:42:23.104999+0000","last_unstale":"2023-10-10T20:42:24.102665+0000","last_undegraded":"2023-10-10T20:42:24.102665+0000","last_fullsized":"2023-10-10T20:42:24.102665+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:04.056737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.103733+0000","last_change":"2023-10-10T20:42:24.101017+0000","last_active":"2023-10-10T20:42:24.103733+0000","last_peered":"2023-10-10T20:42:24.103733+0000","last_clean":"2023-10-10T20:42:24.103733+0000","last_became_active":"2023-10-10T20:42:24.100716+0000","last_became_peered":"2023-10-10T20:42:24.100716+0000","last_unstale":"2023-10-10T20:42:24.103733+0000","last_undegraded":"2023-10-10T20:42:24.103733+0000","last_fullsized":"2023-10-10T20:42:24.103733+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:20:48.561636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.101835+0000","last_change":"2023-10-10T20:42:23.108782+0000","last_active":"2023-10-10T20:42:24.101835+0000","last_peered":"2023-10-10T20:42:24.101835+0000","last_clean":"2023-10-10T20:42:24.101835+0000","last_became_active":"2023-10-10T20:42:23.108480+0000","last_became_peered":"2023-10-10T20:42:23.108480+0000","last_unstale":"2023-10-10T20:42:24.101835+0000","last_undegraded":"2023-10-10T20:42:24.101835+0000","last_fullsized":"2023-10-10T20:42:24.101835+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:32:14.835359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.112616+0000","last_change":"2023-10-10T20:42:24.108473+0000","last_active":"2023-10-10T20:42:24.112616+0000","last_peered":"2023-10-10T20:42:24.112616+0000","last_clean":"2023-10-10T20:42:24.112616+0000","last_became_active":"2023-10-10T20:42:24.108220+0000","last_became_peered":"2023-10-10T20:42:24.108220+0000","last_unstale":"2023-10-10T20:42:24.112616+0000","last_undegraded":"2023-10-10T20:42:24.112616+0000","last_fullsized":"2023-10-10T20:42:24.112616+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:49.847316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.103586+0000","last_change":"2023-10-10T20:42:23.109104+0000","last_active":"2023-10-10T20:42:24.103586+0000","last_peered":"2023-10-10T20:42:24.103586+0000","last_clean":"2023-10-10T20:42:24.103586+0000","last_became_active":"2023-10-10T20:42:23.108836+0000","last_became_peered":"2023-10-10T20:42:23.108836+0000","last_unstale":"2023-10-10T20:42:24.103586+0000","last_undegraded":"2023-10-10T20:42:24.103586+0000","last_fullsized":"2023-10-10T20:42:24.103586+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:42.951721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113476+0000","last_change":"2023-10-10T20:42:24.108573+0000","last_active":"2023-10-10T20:42:24.113476+0000","last_peered":"2023-10-10T20:42:24.113476+0000","last_clean":"2023-10-10T20:42:24.113476+0000","last_became_active":"2023-10-10T20:42:24.108300+0000","last_became_peered":"2023-10-10T20:42:24.108300+0000","last_unstale":"2023-10-10T20:42:24.113476+0000","last_undegraded":"2023-10-10T20:42:24.113476+0000","last_fullsized":"2023-10-10T20:42:24.113476+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:52:51.382479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"152'3","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.123020+0000","last_change":"2023-10-10T20:42:23.107169+0000","last_active":"2023-10-10T20:42:24.123020+0000","last_peered":"2023-10-10T20:42:24.123020+0000","last_clean":"2023-10-10T20:42:24.123020+0000","last_became_active":"2023-10-10T20:42:23.106721+0000","last_became_peered":"2023-10-10T20:42:23.106721+0000","last_unstale":"2023-10-10T20:42:24.123020+0000","last_undegraded":"2023-10-10T20:42:24.123020+0000","last_fullsized":"2023-10-10T20:42:24.123020+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:03:32.352440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121029+0000","last_change":"2023-10-10T20:42:24.115401+0000","last_active":"2023-10-10T20:42:24.121029+0000","last_peered":"2023-10-10T20:42:24.121029+0000","last_clean":"2023-10-10T20:42:24.121029+0000","last_became_active":"2023-10-10T20:42:24.113812+0000","last_became_peered":"2023-10-10T20:42:24.113812+0000","last_unstale":"2023-10-10T20:42:24.121029+0000","last_undegraded":"2023-10-10T20:42:24.121029+0000","last_fullsized":"2023-10-10T20:42:24.121029+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:01.948796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"152'6","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.132627+0000","last_change":"2023-10-10T20:42:23.108302+0000","last_active":"2023-10-10T20:42:24.132627+0000","last_peered":"2023-10-10T20:42:24.132627+0000","last_clean":"2023-10-10T20:42:24.132627+0000","last_became_active":"2023-10-10T20:42:23.108085+0000","last_became_peered":"2023-10-10T20:42:23.108085+0000","last_unstale":"2023-10-10T20:42:24.132627+0000","last_undegraded":"2023-10-10T20:42:24.132627+0000","last_fullsized":"2023-10-10T20:42:24.132627+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:02:33.953621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113456+0000","last_change":"2023-10-10T20:42:24.109419+0000","last_active":"2023-10-10T20:42:24.113456+0000","last_peered":"2023-10-10T20:42:24.113456+0000","last_clean":"2023-10-10T20:42:24.113456+0000","last_became_active":"2023-10-10T20:42:24.109068+0000","last_became_peered":"2023-10-10T20:42:24.109068+0000","last_unstale":"2023-10-10T20:42:24.113456+0000","last_undegraded":"2023-10-10T20:42:24.113456+0000","last_fullsized":"2023-10-10T20:42:24.113456+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:39:59.929264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126772+0000","last_change":"2023-10-10T20:42:23.105344+0000","last_active":"2023-10-10T20:42:24.126772+0000","last_peered":"2023-10-10T20:42:24.126772+0000","last_clean":"2023-10-10T20:42:24.126772+0000","last_became_active":"2023-10-10T20:42:23.105013+0000","last_became_peered":"2023-10-10T20:42:23.105013+0000","last_unstale":"2023-10-10T20:42:24.126772+0000","last_undegraded":"2023-10-10T20:42:24.126772+0000","last_fullsized":"2023-10-10T20:42:24.126772+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:59.544805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121309+0000","last_change":"2023-10-10T20:42:24.115013+0000","last_active":"2023-10-10T20:42:24.121309+0000","last_peered":"2023-10-10T20:42:24.121309+0000","last_clean":"2023-10-10T20:42:24.121309+0000","last_became_active":"2023-10-10T20:42:24.114763+0000","last_became_peered":"2023-10-10T20:42:24.114763+0000","last_unstale":"2023-10-10T20:42:24.121309+0000","last_undegraded":"2023-10-10T20:42:24.121309+0000","last_fullsized":"2023-10-10T20:42:24.121309+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:06.445923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.103163+0000","last_change":"2023-10-10T20:42:23.105949+0000","last_active":"2023-10-10T20:42:24.103163+0000","last_peered":"2023-10-10T20:42:24.103163+0000","last_clean":"2023-10-10T20:42:24.103163+0000","last_became_active":"2023-10-10T20:42:23.105676+0000","last_became_peered":"2023-10-10T20:42:23.105676+0000","last_unstale":"2023-10-10T20:42:24.103163+0000","last_undegraded":"2023-10-10T20:42:24.103163+0000","last_fullsized":"2023-10-10T20:42:24.103163+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:43:10.994181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126457+0000","last_change":"2023-10-10T20:42:24.126007+0000","last_active":"2023-10-10T20:42:24.126457+0000","last_peered":"2023-10-10T20:42:24.126457+0000","last_clean":"2023-10-10T20:42:24.126457+0000","last_became_active":"2023-10-10T20:42:24.125699+0000","last_became_peered":"2023-10-10T20:42:24.125699+0000","last_unstale":"2023-10-10T20:42:24.126457+0000","last_undegraded":"2023-10-10T20:42:24.126457+0000","last_fullsized":"2023-10-10T20:42:24.126457+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:09:15.013589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.101618+0000","last_change":"2023-10-10T20:42:23.103097+0000","last_active":"2023-10-10T20:42:24.101618+0000","last_peered":"2023-10-10T20:42:24.101618+0000","last_clean":"2023-10-10T20:42:24.101618+0000","last_became_active":"2023-10-10T20:42:23.102819+0000","last_became_peered":"2023-10-10T20:42:23.102819+0000","last_unstale":"2023-10-10T20:42:24.101618+0000","last_undegraded":"2023-10-10T20:42:24.101618+0000","last_fullsized":"2023-10-10T20:42:24.101618+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:20:47.550902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121175+0000","last_change":"2023-10-10T20:42:24.115553+0000","last_active":"2023-10-10T20:42:24.121175+0000","last_peered":"2023-10-10T20:42:24.121175+0000","last_clean":"2023-10-10T20:42:24.121175+0000","last_became_active":"2023-10-10T20:42:24.115239+0000","last_became_peered":"2023-10-10T20:42:24.115239+0000","last_unstale":"2023-10-10T20:42:24.121175+0000","last_undegraded":"2023-10-10T20:42:24.121175+0000","last_fullsized":"2023-10-10T20:42:24.121175+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:14.819673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"152'8","reported_seq":18,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.125165+0000","last_change":"2023-10-10T20:42:23.103610+0000","last_active":"2023-10-10T20:42:24.125165+0000","last_peered":"2023-10-10T20:42:24.125165+0000","last_clean":"2023-10-10T20:42:24.125165+0000","last_became_active":"2023-10-10T20:42:23.103353+0000","last_became_peered":"2023-10-10T20:42:23.103353+0000","last_unstale":"2023-10-10T20:42:24.125165+0000","last_undegraded":"2023-10-10T20:42:24.125165+0000","last_fullsized":"2023-10-10T20:42:24.125165+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:23:34.102393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.111073+0000","last_change":"2023-10-10T20:42:24.104841+0000","last_active":"2023-10-10T20:42:24.111073+0000","last_peered":"2023-10-10T20:42:24.111073+0000","last_clean":"2023-10-10T20:42:24.111073+0000","last_became_active":"2023-10-10T20:42:24.104561+0000","last_became_peered":"2023-10-10T20:42:24.104561+0000","last_unstale":"2023-10-10T20:42:24.111073+0000","last_undegraded":"2023-10-10T20:42:24.111073+0000","last_fullsized":"2023-10-10T20:42:24.111073+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:45:43.231122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.123211+0000","last_change":"2023-10-10T20:42:23.105236+0000","last_active":"2023-10-10T20:42:24.123211+0000","last_peered":"2023-10-10T20:42:24.123211+0000","last_clean":"2023-10-10T20:42:24.123211+0000","last_became_active":"2023-10-10T20:42:23.104983+0000","last_became_peered":"2023-10-10T20:42:23.104983+0000","last_unstale":"2023-10-10T20:42:24.123211+0000","last_undegraded":"2023-10-10T20:42:24.123211+0000","last_fullsized":"2023-10-10T20:42:24.123211+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:24:03.181433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113521+0000","last_change":"2023-10-10T20:42:24.107994+0000","last_active":"2023-10-10T20:42:24.113521+0000","last_peered":"2023-10-10T20:42:24.113521+0000","last_clean":"2023-10-10T20:42:24.113521+0000","last_became_active":"2023-10-10T20:42:24.107722+0000","last_became_peered":"2023-10-10T20:42:24.107722+0000","last_unstale":"2023-10-10T20:42:24.113521+0000","last_undegraded":"2023-10-10T20:42:24.113521+0000","last_fullsized":"2023-10-10T20:42:24.113521+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:57.924961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.123838+0000","last_change":"2023-10-10T20:42:23.111459+0000","last_active":"2023-10-10T20:42:24.123838+0000","last_peered":"2023-10-10T20:42:24.123838+0000","last_clean":"2023-10-10T20:42:24.123838+0000","last_became_active":"2023-10-10T20:42:23.111197+0000","last_became_peered":"2023-10-10T20:42:23.111197+0000","last_unstale":"2023-10-10T20:42:24.123838+0000","last_undegraded":"2023-10-10T20:42:24.123838+0000","last_fullsized":"2023-10-10T20:42:24.123838+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:23:14.263295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.106150+0000","last_change":"2023-10-10T20:42:24.101341+0000","last_active":"2023-10-10T20:42:24.106150+0000","last_peered":"2023-10-10T20:42:24.106150+0000","last_clean":"2023-10-10T20:42:24.106150+0000","last_became_active":"2023-10-10T20:42:24.101086+0000","last_became_peered":"2023-10-10T20:42:24.101086+0000","last_unstale":"2023-10-10T20:42:24.106150+0000","last_undegraded":"2023-10-10T20:42:24.106150+0000","last_fullsized":"2023-10-10T20:42:24.106150+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:11:04.270228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126614+0000","last_change":"2023-10-10T20:42:23.107193+0000","last_active":"2023-10-10T20:42:24.126614+0000","last_peered":"2023-10-10T20:42:24.126614+0000","last_clean":"2023-10-10T20:42:24.126614+0000","last_became_active":"2023-10-10T20:42:23.106136+0000","last_became_peered":"2023-10-10T20:42:23.106136+0000","last_unstale":"2023-10-10T20:42:24.126614+0000","last_undegraded":"2023-10-10T20:42:24.126614+0000","last_fullsized":"2023-10-10T20:42:24.126614+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:26:51.984824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.120858+0000","last_change":"2023-10-10T20:42:24.112738+0000","last_active":"2023-10-10T20:42:24.120858+0000","last_peered":"2023-10-10T20:42:24.120858+0000","last_clean":"2023-10-10T20:42:24.120858+0000","last_became_active":"2023-10-10T20:42:24.112515+0000","last_became_peered":"2023-10-10T20:42:24.112515+0000","last_unstale":"2023-10-10T20:42:24.120858+0000","last_undegraded":"2023-10-10T20:42:24.120858+0000","last_fullsized":"2023-10-10T20:42:24.120858+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:01.948796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.096078+0000","last_change":"2023-10-10T20:42:23.107823+0000","last_active":"2023-10-10T20:42:24.096078+0000","last_peered":"2023-10-10T20:42:24.096078+0000","last_clean":"2023-10-10T20:42:24.096078+0000","last_became_active":"2023-10-10T20:42:23.107643+0000","last_became_peered":"2023-10-10T20:42:23.107643+0000","last_unstale":"2023-10-10T20:42:24.096078+0000","last_undegraded":"2023-10-10T20:42:24.096078+0000","last_fullsized":"2023-10-10T20:42:24.096078+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:43:41.316715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.120251+0000","last_change":"2023-10-10T20:42:24.110967+0000","last_active":"2023-10-10T20:42:24.120251+0000","last_peered":"2023-10-10T20:42:24.120251+0000","last_clean":"2023-10-10T20:42:24.120251+0000","last_became_active":"2023-10-10T20:42:24.110687+0000","last_became_peered":"2023-10-10T20:42:24.110687+0000","last_unstale":"2023-10-10T20:42:24.120251+0000","last_undegraded":"2023-10-10T20:42:24.120251+0000","last_fullsized":"2023-10-10T20:42:24.120251+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:53.653343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121843+0000","last_change":"2023-10-10T20:42:23.105742+0000","last_active":"2023-10-10T20:42:24.121843+0000","last_peered":"2023-10-10T20:42:24.121843+0000","last_clean":"2023-10-10T20:42:24.121843+0000","last_became_active":"2023-10-10T20:42:23.105472+0000","last_became_peered":"2023-10-10T20:42:23.105472+0000","last_unstale":"2023-10-10T20:42:24.121843+0000","last_undegraded":"2023-10-10T20:42:24.121843+0000","last_fullsized":"2023-10-10T20:42:24.121843+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:29.453586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113971+0000","last_change":"2023-10-10T20:42:24.112508+0000","last_active":"2023-10-10T20:42:24.113971+0000","last_peered":"2023-10-10T20:42:24.113971+0000","last_clean":"2023-10-10T20:42:24.113971+0000","last_became_active":"2023-10-10T20:42:24.112276+0000","last_became_peered":"2023-10-10T20:42:24.112276+0000","last_unstale":"2023-10-10T20:42:24.113971+0000","last_undegraded":"2023-10-10T20:42:24.113971+0000","last_fullsized":"2023-10-10T20:42:24.113971+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:55:23.468746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.104357+0000","last_change":"2023-10-10T20:42:23.110977+0000","last_active":"2023-10-10T20:42:24.104357+0000","last_peered":"2023-10-10T20:42:24.104357+0000","last_clean":"2023-10-10T20:42:24.104357+0000","last_became_active":"2023-10-10T20:42:23.110704+0000","last_became_peered":"2023-10-10T20:42:23.110704+0000","last_unstale":"2023-10-10T20:42:24.104357+0000","last_undegraded":"2023-10-10T20:42:24.104357+0000","last_fullsized":"2023-10-10T20:42:24.104357+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:22:28.404747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114385+0000","last_change":"2023-10-10T20:42:24.109982+0000","last_active":"2023-10-10T20:42:24.114385+0000","last_peered":"2023-10-10T20:42:24.114385+0000","last_clean":"2023-10-10T20:42:24.114385+0000","last_became_active":"2023-10-10T20:42:24.109703+0000","last_became_peered":"2023-10-10T20:42:24.109703+0000","last_unstale":"2023-10-10T20:42:24.114385+0000","last_undegraded":"2023-10-10T20:42:24.114385+0000","last_fullsized":"2023-10-10T20:42:24.114385+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:46:22.492928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.103008+0000","last_change":"2023-10-10T20:42:23.109984+0000","last_active":"2023-10-10T20:42:24.103008+0000","last_peered":"2023-10-10T20:42:24.103008+0000","last_clean":"2023-10-10T20:42:24.103008+0000","last_became_active":"2023-10-10T20:42:23.109710+0000","last_became_peered":"2023-10-10T20:42:23.109710+0000","last_unstale":"2023-10-10T20:42:24.103008+0000","last_undegraded":"2023-10-10T20:42:24.103008+0000","last_fullsized":"2023-10-10T20:42:24.103008+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:41:54.647934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.113784+0000","last_change":"2023-10-10T20:42:24.110731+0000","last_active":"2023-10-10T20:42:24.113784+0000","last_peered":"2023-10-10T20:42:24.113784+0000","last_clean":"2023-10-10T20:42:24.113784+0000","last_became_active":"2023-10-10T20:42:24.110499+0000","last_became_peered":"2023-10-10T20:42:24.110499+0000","last_unstale":"2023-10-10T20:42:24.113784+0000","last_undegraded":"2023-10-10T20:42:24.113784+0000","last_fullsized":"2023-10-10T20:42:24.113784+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:46:15.751003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.102364+0000","last_change":"2023-10-10T20:42:23.104363+0000","last_active":"2023-10-10T20:42:24.102364+0000","last_peered":"2023-10-10T20:42:24.102364+0000","last_clean":"2023-10-10T20:42:24.102364+0000","last_became_active":"2023-10-10T20:42:23.104108+0000","last_became_peered":"2023-10-10T20:42:23.104108+0000","last_unstale":"2023-10-10T20:42:24.102364+0000","last_undegraded":"2023-10-10T20:42:24.102364+0000","last_fullsized":"2023-10-10T20:42:24.102364+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:52.876825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.124500+0000","last_change":"2023-10-10T20:42:24.122887+0000","last_active":"2023-10-10T20:42:24.124500+0000","last_peered":"2023-10-10T20:42:24.124500+0000","last_clean":"2023-10-10T20:42:24.124500+0000","last_became_active":"2023-10-10T20:42:24.122521+0000","last_became_peered":"2023-10-10T20:42:24.122521+0000","last_unstale":"2023-10-10T20:42:24.124500+0000","last_undegraded":"2023-10-10T20:42:24.124500+0000","last_fullsized":"2023-10-10T20:42:24.124500+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:09:32.731249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.125650+0000","last_change":"2023-10-10T20:42:23.109167+0000","last_active":"2023-10-10T20:42:24.125650+0000","last_peered":"2023-10-10T20:42:24.125650+0000","last_clean":"2023-10-10T20:42:24.125650+0000","last_became_active":"2023-10-10T20:42:23.108692+0000","last_became_peered":"2023-10-10T20:42:23.108692+0000","last_unstale":"2023-10-10T20:42:24.125650+0000","last_undegraded":"2023-10-10T20:42:24.125650+0000","last_fullsized":"2023-10-10T20:42:24.125650+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:56.210224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126436+0000","last_change":"2023-10-10T20:42:24.125413+0000","last_active":"2023-10-10T20:42:24.126436+0000","last_peered":"2023-10-10T20:42:24.126436+0000","last_clean":"2023-10-10T20:42:24.126436+0000","last_became_active":"2023-10-10T20:42:24.125149+0000","last_became_peered":"2023-10-10T20:42:24.125149+0000","last_unstale":"2023-10-10T20:42:24.126436+0000","last_undegraded":"2023-10-10T20:42:24.126436+0000","last_fullsized":"2023-10-10T20:42:24.126436+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:00:26.956205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.096131+0000","last_change":"2023-10-10T20:42:23.098780+0000","last_active":"2023-10-10T20:42:24.096131+0000","last_peered":"2023-10-10T20:42:24.096131+0000","last_clean":"2023-10-10T20:42:24.096131+0000","last_became_active":"2023-10-10T20:42:23.098462+0000","last_became_peered":"2023-10-10T20:42:23.098462+0000","last_unstale":"2023-10-10T20:42:24.096131+0000","last_undegraded":"2023-10-10T20:42:24.096131+0000","last_fullsized":"2023-10-10T20:42:24.096131+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:37:13.498053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114471+0000","last_change":"2023-10-10T20:42:24.110340+0000","last_active":"2023-10-10T20:42:24.114471+0000","last_peered":"2023-10-10T20:42:24.114471+0000","last_clean":"2023-10-10T20:42:24.114471+0000","last_became_active":"2023-10-10T20:42:24.110079+0000","last_became_peered":"2023-10-10T20:42:24.110079+0000","last_unstale":"2023-10-10T20:42:24.114471+0000","last_undegraded":"2023-10-10T20:42:24.114471+0000","last_fullsized":"2023-10-10T20:42:24.114471+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:35:23.790779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.102018+0000","last_change":"2023-10-10T20:42:23.108778+0000","last_active":"2023-10-10T20:42:24.102018+0000","last_peered":"2023-10-10T20:42:24.102018+0000","last_clean":"2023-10-10T20:42:24.102018+0000","last_became_active":"2023-10-10T20:42:23.108454+0000","last_became_peered":"2023-10-10T20:42:23.108454+0000","last_unstale":"2023-10-10T20:42:24.102018+0000","last_undegraded":"2023-10-10T20:42:24.102018+0000","last_fullsized":"2023-10-10T20:42:24.102018+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:09:31.720515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114247+0000","last_change":"2023-10-10T20:42:24.111895+0000","last_active":"2023-10-10T20:42:24.114247+0000","last_peered":"2023-10-10T20:42:24.114247+0000","last_clean":"2023-10-10T20:42:24.114247+0000","last_became_active":"2023-10-10T20:42:24.111432+0000","last_became_peered":"2023-10-10T20:42:24.111432+0000","last_unstale":"2023-10-10T20:42:24.114247+0000","last_undegraded":"2023-10-10T20:42:24.114247+0000","last_fullsized":"2023-10-10T20:42:24.114247+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:21:59.491769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.096099+0000","last_change":"2023-10-10T20:42:23.108978+0000","last_active":"2023-10-10T20:42:24.096099+0000","last_peered":"2023-10-10T20:42:24.096099+0000","last_clean":"2023-10-10T20:42:24.096099+0000","last_became_active":"2023-10-10T20:42:23.108707+0000","last_became_peered":"2023-10-10T20:42:23.108707+0000","last_unstale":"2023-10-10T20:42:24.096099+0000","last_undegraded":"2023-10-10T20:42:24.096099+0000","last_fullsized":"2023-10-10T20:42:24.096099+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:44.681204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.121726+0000","last_change":"2023-10-10T20:42:24.116434+0000","last_active":"2023-10-10T20:42:24.121726+0000","last_peered":"2023-10-10T20:42:24.121726+0000","last_clean":"2023-10-10T20:42:24.121726+0000","last_became_active":"2023-10-10T20:42:24.115786+0000","last_became_peered":"2023-10-10T20:42:24.115786+0000","last_unstale":"2023-10-10T20:42:24.121726+0000","last_undegraded":"2023-10-10T20:42:24.121726+0000","last_fullsized":"2023-10-10T20:42:24.121726+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:46:07.440450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.098460+0000","last_change":"2023-10-10T20:42:23.107133+0000","last_active":"2023-10-10T20:42:24.098460+0000","last_peered":"2023-10-10T20:42:24.098460+0000","last_clean":"2023-10-10T20:42:24.098460+0000","last_became_active":"2023-10-10T20:42:23.106863+0000","last_became_peered":"2023-10-10T20:42:23.106863+0000","last_unstale":"2023-10-10T20:42:24.098460+0000","last_undegraded":"2023-10-10T20:42:24.098460+0000","last_fullsized":"2023-10-10T20:42:24.098460+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:58.808416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.126598+0000","last_change":"2023-10-10T20:42:24.123718+0000","last_active":"2023-10-10T20:42:24.126598+0000","last_peered":"2023-10-10T20:42:24.126598+0000","last_clean":"2023-10-10T20:42:24.126598+0000","last_became_active":"2023-10-10T20:42:24.123096+0000","last_became_peered":"2023-10-10T20:42:24.123096+0000","last_unstale":"2023-10-10T20:42:24.126598+0000","last_undegraded":"2023-10-10T20:42:24.126598+0000","last_fullsized":"2023-10-10T20:42:24.126598+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:14:18.635698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.104118+0000","last_change":"2023-10-10T20:42:23.104363+0000","last_active":"2023-10-10T20:42:24.104118+0000","last_peered":"2023-10-10T20:42:24.104118+0000","last_clean":"2023-10-10T20:42:24.104118+0000","last_became_active":"2023-10-10T20:42:23.104109+0000","last_became_peered":"2023-10-10T20:42:23.104109+0000","last_unstale":"2023-10-10T20:42:24.104118+0000","last_undegraded":"2023-10-10T20:42:24.104118+0000","last_fullsized":"2023-10-10T20:42:24.104118+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:43:10.994181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.112743+0000","last_change":"2023-10-10T20:42:24.109363+0000","last_active":"2023-10-10T20:42:24.112743+0000","last_peered":"2023-10-10T20:42:24.112743+0000","last_clean":"2023-10-10T20:42:24.112743+0000","last_became_active":"2023-10-10T20:42:24.109127+0000","last_became_peered":"2023-10-10T20:42:24.109127+0000","last_unstale":"2023-10-10T20:42:24.112743+0000","last_undegraded":"2023-10-10T20:42:24.112743+0000","last_fullsized":"2023-10-10T20:42:24.112743+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:58:55.908084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.103440+0000","last_change":"2023-10-10T20:42:23.108038+0000","last_active":"2023-10-10T20:42:24.103440+0000","last_peered":"2023-10-10T20:42:24.103440+0000","last_clean":"2023-10-10T20:42:24.103440+0000","last_became_active":"2023-10-10T20:42:23.107750+0000","last_became_peered":"2023-10-10T20:42:23.107750+0000","last_unstale":"2023-10-10T20:42:24.103440+0000","last_undegraded":"2023-10-10T20:42:24.103440+0000","last_fullsized":"2023-10-10T20:42:24.103440+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:00:27.632584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.114575+0000","last_change":"2023-10-10T20:42:24.110672+0000","last_active":"2023-10-10T20:42:24.114575+0000","last_peered":"2023-10-10T20:42:24.114575+0000","last_clean":"2023-10-10T20:42:24.114575+0000","last_became_active":"2023-10-10T20:42:24.110384+0000","last_became_peered":"2023-10-10T20:42:24.110384+0000","last_unstale":"2023-10-10T20:42:24.114575+0000","last_undegraded":"2023-10-10T20:42:24.114575+0000","last_fullsized":"2023-10-10T20:42:24.114575+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:49.847316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.108826+0000","last_change":"2023-10-10T20:42:23.103184+0000","last_active":"2023-10-10T20:42:24.108826+0000","last_peered":"2023-10-10T20:42:24.108826+0000","last_clean":"2023-10-10T20:42:24.108826+0000","last_became_active":"2023-10-10T20:42:23.102891+0000","last_became_peered":"2023-10-10T20:42:23.102891+0000","last_unstale":"2023-10-10T20:42:24.108826+0000","last_undegraded":"2023-10-10T20:42:24.108826+0000","last_fullsized":"2023-10-10T20:42:24.108826+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:48:40.304153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.117438+0000","last_change":"2023-10-10T20:42:24.113117+0000","last_active":"2023-10-10T20:42:24.117438+0000","last_peered":"2023-10-10T20:42:24.117438+0000","last_clean":"2023-10-10T20:42:24.117438+0000","last_became_active":"2023-10-10T20:42:24.112829+0000","last_became_peered":"2023-10-10T20:42:24.112829+0000","last_unstale":"2023-10-10T20:42:24.117438+0000","last_undegraded":"2023-10-10T20:42:24.117438+0000","last_fullsized":"2023-10-10T20:42:24.117438+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:23.086777+0000","last_clean_scrub_stamp":"2023-10-10T20:42:23.086777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:17:06.399895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.131253+0000","last_change":"2023-10-10T20:42:23.110595+0000","last_active":"2023-10-10T20:42:24.131253+0000","last_peered":"2023-10-10T20:42:24.131253+0000","last_clean":"2023-10-10T20:42:24.131253+0000","last_became_active":"2023-10-10T20:42:23.110274+0000","last_became_peered":"2023-10-10T20:42:23.110274+0000","last_unstale":"2023-10-10T20:42:24.131253+0000","last_undegraded":"2023-10-10T20:42:24.131253+0000","last_fullsized":"2023-10-10T20:42:24.131253+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:22.076043+0000","last_clean_scrub_stamp":"2023-10-10T20:42:22.076043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:04.056737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":296,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-10T20:42:24.096203+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:42:24.096203+0000","last_peered":"2023-10-10T20:42:24.096203+0000","last_clean":"2023-10-10T20:42:24.096203+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:42:24.096203+0000","last_undegraded":"2023-10-10T20:42:24.096203+0000","last_fullsized":"2023-10-10T20:42:24.096203+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574948,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303868,"kb_used_data":2484,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067972,"statfs":{"total":96636764160,"available":96325603328,"internally_reserved":0,"allocated":2543616,"data_stored":1460226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.79000000000000004},"min":{"1min":0.309,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71799999999999997,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52400000000000002,"15min":0.61599999999999999},"min":{"1min":0.32300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.69999999999999996,"5min":0.82099999999999995,"15min":4.1959999999999997},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.95099999999999996},"min":{"1min":0.379,"5min":0.317,"15min":0.317},"max":{"1min":0.78300000000000003,"5min":0.78800000000000003,"15min":10.1},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55200000000000005,"15min":0.71399999999999997},"min":{"1min":0.36699999999999999,"5min":0.32300000000000001,"15min":0.28999999999999998},"max":{"1min":0.73199999999999998,"5min":0.90400000000000003,"15min":6.806},"last":0.46999999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.76400000000000001},"min":{"1min":0.40100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79200000000000004,"5min":0.89000000000000001,"15min":6.9340000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.71699999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.73299999999999998,"5min":0.83899999999999997,"15min":6.9870000000000001},"last":0.54000000000000004}]},{"osd":3,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54900000000000004,"15min":0.73599999999999999},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.753,"5min":0.80200000000000005,"15min":6.6719999999999997},"last":0.504},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56699999999999995,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"max":{"1min":0.71699999999999997,"5min":0.87,"15min":4.1609999999999996},"last":0.52200000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.36599999999999999,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.65100000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36299999999999999,"5min":0.309,"15min":0.30399999999999999},"max":{"1min":0.72299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.40999999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.314},"max":{"1min":0.77900000000000003,"5min":0.86099999999999999,"15min":1.3180000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.41799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71499999999999997,"5min":0.75800000000000001,"15min":0.84099999999999997},"last":0.38700000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":0.78700000000000003,"15min":1.7509999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.40899999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.85699999999999998,"15min":1.7090000000000001},"last":0.59399999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574949,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303280,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068560,"statfs":{"total":96636764160,"available":96326205440,"internally_reserved":0,"allocated":1941504,"data_stored":868876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51300000000000001,"15min":0.65500000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":7.7590000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.52100000000000002,"15min":0.69199999999999995},"min":{"1min":0.32200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.075,"5min":1.075,"15min":7.8090000000000002},"last":0.62}]},{"osd":1,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.69099999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.70199999999999996,"5min":0.80000000000000004,"15min":7.867},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.85599999999999998},"min":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.65600000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.58799999999999997},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.56100000000000005,"15min":0.81999999999999995},"min":{"1min":0.34200000000000003,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":2.641,"5min":2.641,"15min":6.7629999999999999},"last":0.58799999999999997}]},{"osd":3,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54300000000000004,"15min":0.61099999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.107,"5min":1.107,"15min":4.8620000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.70099999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.718,"5min":1.718,"15min":7.9009999999999998},"last":0.54100000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.47299999999999998},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.45500000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52700000000000002,"15min":0.495},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.376},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.53400000000000003},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.35299999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55800000000000005,"15min":0.52500000000000002},"min":{"1min":0.36099999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.372,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.433}]}]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303284,"kb_used_data":1900,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068556,"statfs":{"total":96636764160,"available":96326201344,"internally_reserved":0,"allocated":1945600,"data_stored":868378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.312,"15min":0.45900000000000002},"min":{"1min":0.22800000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.437,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.23300000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":1.194},"last":0.29999999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32300000000000001,"15min":0.35999999999999999},"min":{"1min":0.24199999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51000000000000001,"5min":0.51400000000000001,"15min":1.2869999999999999},"last":0.246},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.48999999999999999},"min":{"1min":0.27200000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.52800000000000002,"5min":0.54200000000000004,"15min":5.0919999999999996},"last":0.35599999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36399999999999999,"15min":0.51200000000000001},"min":{"1min":0.29399999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35699999999999998,"15min":0.40200000000000002},"min":{"1min":0.23799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":1.2170000000000001},"last":0.32600000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.73999999999999999,"5min":0.92400000000000004,"15min":1.254},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.68999999999999995},"min":{"1min":0.39300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68799999999999994,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.53300000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.36099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":0.97599999999999998,"15min":1.1799999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56899999999999995,"15min":0.60299999999999998},"min":{"1min":0.41799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71399999999999997,"5min":0.878,"15min":1.3109999999999999},"last":0.57899999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.81799999999999995},"min":{"1min":0.46000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78900000000000003,"5min":0.995,"15min":5.0659999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80700000000000005,"5min":0.89800000000000002,"15min":1.171},"last":0.46400000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57999999999999996,"15min":0.61299999999999999},"min":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77300000000000002,"5min":1.004,"15min":1.214},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.70999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78200000000000003,"5min":0.90500000000000003,"15min":4.0499999999999998},"last":0.54500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574947,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303856,"kb_used_data":2472,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067984,"statfs":{"total":96636764160,"available":96325615616,"internally_reserved":0,"allocated":2531328,"data_stored":1457819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.316,"15min":0.50800000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":7.9249999999999998},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.48699999999999999},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":6.6299999999999999},"last":0.311}]},{"osd":2,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.33100000000000002,"15min":0.53100000000000003},"min":{"1min":0.23100000000000001,"5min":0.22,"15min":0.216},"max":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":8.2509999999999994},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.51100000000000001},"min":{"1min":0.253,"5min":0.222,"15min":0.222},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":6.7370000000000001},"last":0.34899999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.35399999999999998,"15min":0.53900000000000003},"min":{"1min":0.247,"5min":0.23100000000000001,"15min":0.21099999999999999},"max":{"1min":0.53800000000000003,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.59099999999999997},"min":{"1min":0.27700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":7.8929999999999998},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57099999999999995,"15min":0.70299999999999996},"min":{"1min":0.45500000000000002,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.82499999999999996,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57299999999999995,"15min":0.754},"min":{"1min":0.441,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.80900000000000005,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.51300000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56100000000000005,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71299999999999997,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.76100000000000001},"min":{"1min":0.48399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.48699999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61499999999999999,"15min":0.80000000000000004},"min":{"1min":0.52200000000000002,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.83299999999999996,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.62,"15min":0.749},"min":{"1min":0.47399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82599999999999996,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.502}]},{"osd":7,"last update":"Tue Oct 10 20:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.60099999999999998,"15min":0.73999999999999999},"min":{"1min":0.48399999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85399999999999998,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.76800000000000002},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.75700000000000001,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.60399999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574949,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":868322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.311,"15min":0.32500000000000001},"min":{"1min":0.214,"5min":0.189,"15min":0.189},"max":{"1min":0.52500000000000002,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.35099999999999998},"min":{"1min":0.219,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.746},"last":0.23000000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.69099999999999995},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52100000000000002,"5min":0.63,"15min":11.967000000000001},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.34899999999999998,"15min":0.46999999999999997},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48899999999999999,"5min":0.53400000000000003,"15min":4.1100000000000003},"last":0.309}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34699999999999998,"15min":0.53700000000000003},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.50800000000000001,"5min":0.51600000000000001,"15min":6.3040000000000003},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.74299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59299999999999997,"5min":0.65900000000000003,"15min":11.935},"last":0.29499999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.55700000000000005,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.53700000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.38,"15min":0.378},"max":{"1min":0.83699999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54200000000000004,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.96099999999999997},"last":0.53000000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69599999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.58599999999999997},"min":{"1min":0.435,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.052},"last":0.623},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58099999999999996,"15min":0.76300000000000001},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":6.2649999999999997},"last":0.55800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574949,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303288,"kb_used_data":1904,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068552,"statfs":{"total":96636764160,"available":96326197248,"internally_reserved":0,"allocated":1949696,"data_stored":868504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.315,"15min":0.315},"min":{"1min":0.223,"5min":0.21199999999999999,"15min":0.21099999999999999},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32100000000000001},"min":{"1min":0.23999999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48799999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.313}]},{"osd":1,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.34599999999999997,"15min":0.33300000000000002},"min":{"1min":0.25700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.379},"min":{"1min":0.29199999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.72099999999999997},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.36199999999999999,"15min":0.35299999999999998},"min":{"1min":0.25800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57599999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.37},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.28999999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.50900000000000001,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.32600000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.53400000000000003},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77200000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.745,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.79400000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57299999999999995,"15min":0.55200000000000005},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.35899999999999999},"max":{"1min":0.82399999999999995,"5min":1.347,"15min":1.347},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.42199999999999999,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.83499999999999996,"5min":1.01,"15min":1.01},"last":0.65700000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.48399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86299999999999999,"5min":0.92000000000000004,"15min":1.0189999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.81200000000000006,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.71799999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:41:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78700000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":0.83699999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.56699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574947,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303296,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068544,"statfs":{"total":96636764160,"available":96326189056,"internally_reserved":0,"allocated":1957888,"data_stored":869962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.32400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66600000000000004,"5min":0.74199999999999999,"15min":0.747},"last":0.376},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51700000000000002,"15min":0.74099999999999999},"min":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77600000000000002,"5min":0.78400000000000003,"15min":7.6719999999999997},"last":0.53100000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52000000000000002,"15min":0.748},"min":{"1min":0.32000000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.755,"15min":7.7160000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66100000000000003,"5min":0.76900000000000002,"15min":0.86799999999999999},"last":0.56899999999999995}]},{"osd":2,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.73899999999999999,"5min":0.79800000000000004,"15min":7.7839999999999998},"last":0.50900000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.39200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.49099999999999999},"min":{"1min":0.35199999999999998,"5min":0.314,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58699999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.85199999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58399999999999996,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.61199999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.41399999999999998,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.626}]}]},{"osd":5,"up_from":13,"seq":55834574949,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303280,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068560,"statfs":{"total":96636764160,"available":96326205440,"internally_reserved":0,"allocated":1941504,"data_stored":868876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.47599999999999998,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.66100000000000003,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.51800000000000002},"min":{"1min":0.33000000000000002,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.68899999999999995,"5min":0.76700000000000002,"15min":0.79300000000000004},"last":0.60599999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51300000000000001,"15min":0.628},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63400000000000001,"5min":0.75700000000000001,"15min":5.2679999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.62},"min":{"1min":0.40500000000000003,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79200000000000004,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.65200000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52400000000000002,"15min":0.63700000000000001},"min":{"1min":0.39100000000000001,"5min":0.32900000000000001,"15min":0.28299999999999997},"max":{"1min":0.73299999999999998,"5min":0.76600000000000001,"15min":5.2960000000000003},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.38500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.58299999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55900000000000005,"15min":0.66900000000000004},"min":{"1min":0.41999999999999998,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":5.3330000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56399999999999995,"15min":0.68000000000000005},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.27700000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.67000000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.48899999999999999,"15min":0.498},"min":{"1min":0.33400000000000002,"5min":0.29599999999999999,"15min":0.28699999999999998},"max":{"1min":0.67000000000000004,"5min":0.72499999999999998,"15min":0.82999999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.34499999999999997,"15min":0.32200000000000001},"max":{"1min":0.77200000000000002,"5min":0.88,"15min":1.0800000000000001},"last":0.38900000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.39900000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78900000000000003,"5min":0.82499999999999996,"15min":0.89800000000000002},"last":0.51500000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.32300000000000001,"15min":0.30099999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98399999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.44}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:42:33.025 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:42:33.025 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:42:33.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.179+0000 7f9e014f4700 1 Processor -- start 2023-10-10T20:42:33.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.179+0000 7f9e014f4700 1 -- start start 2023-10-10T20:42:33.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.180+0000 7f9e014f4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9dfc0cca50 0x7f9dfc0caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:33.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.180+0000 7f9e014f4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9dfc15df40 0x7f9dfc15e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:33.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.180+0000 7f9e014f4700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dfc157c50 con 0x7f9dfc0cce20 2023-10-10T20:42:33.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.180+0000 7f9e014f4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dfc15d7d0 con 0x7f9dfc15df40 2023-10-10T20:42:33.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.180+0000 7f9e014f4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc15aa20 con 0x7f9dfc0cca50 2023-10-10T20:42:33.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.180+0000 7f9dfb7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9dfc15df40 0x7f9dfc15e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:33.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.180+0000 7f9dfb7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9dfc15df40 0x7f9dfc15e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59176/0 (socket says 172.21.15.89:59176) 2023-10-10T20:42:33.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.180+0000 7f9dfb7fe700 1 -- 172.21.15.89:0/3466784018 learned_addr learned my addr 172.21.15.89:0/3466784018 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:33.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.181+0000 7f9dfaffd700 1 --2- 172.21.15.89:0/3466784018 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9dfc0cca50 0x7f9dfc0caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:33.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.181+0000 7f9dfb7fe700 1 -- 172.21.15.89:0/3466784018 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9dfc0cca50 msgr2=0x7f9dfc0caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:33.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.181+0000 7f9dfb7fe700 1 --2- 172.21.15.89:0/3466784018 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9dfc0cca50 0x7f9dfc0caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.181+0000 7f9dfb7fe700 1 -- 172.21.15.89:0/3466784018 >> v1:172.21.15.111:6789/0 conn(0x7f9dfc0cce20 legacy=0x7f9dfc15a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:33.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.181+0000 7f9dfb7fe700 1 -- 172.21.15.89:0/3466784018 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc15ad00 con 0x7f9dfc15df40 2023-10-10T20:42:33.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.181+0000 7f9dfb7fe700 1 --2- 172.21.15.89:0/3466784018 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dfc15df40 0x7f9dfc15e380 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009f20 tx=0x7f9df4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae96dee12bac5050 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:33.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.181+0000 7f9dfa7fc700 1 -- 172.21.15.89:0/3466784018 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df400e070 con 0x7f9dfc15df40 2023-10-10T20:42:33.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.181+0000 7f9dfa7fc700 1 -- 172.21.15.89:0/3466784018 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df400ba40 con 0x7f9dfc15df40 2023-10-10T20:42:33.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.181+0000 7f9dfa7fc700 1 -- 172.21.15.89:0/3466784018 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df40092e0 con 0x7f9dfc15df40 2023-10-10T20:42:33.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.181+0000 7f9e014f4700 1 -- 172.21.15.89:0/3466784018 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dfc15df40 msgr2=0x7f9dfc15e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:33.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.181+0000 7f9e014f4700 1 --2- 172.21.15.89:0/3466784018 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dfc15df40 0x7f9dfc15e380 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009f20 tx=0x7f9df4009840 comp rx=0 tx=0).stop 2023-10-10T20:42:33.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.182+0000 7f9e014f4700 1 -- 172.21.15.89:0/3466784018 shutdown_connections 2023-10-10T20:42:33.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.182+0000 7f9e014f4700 1 --2- 172.21.15.89:0/3466784018 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dfc15df40 0x7f9dfc15e380 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.182+0000 7f9e014f4700 1 --2- 172.21.15.89:0/3466784018 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9dfc0cca50 0x7f9dfc0caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.182+0000 7f9e014f4700 1 -- 172.21.15.89:0/3466784018 >> 172.21.15.89:0/3466784018 conn(0x7f9dfc0ce5c0 msgr2=0x7f9dfc0ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:33.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.182+0000 7f9e014f4700 1 -- 172.21.15.89:0/3466784018 shutdown_connections 2023-10-10T20:42:33.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.182+0000 7f9e014f4700 1 -- 172.21.15.89:0/3466784018 wait complete. 2023-10-10T20:42:33.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.183+0000 7f9e014f4700 1 Processor -- start 2023-10-10T20:42:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.183+0000 7f9e014f4700 1 -- start start 2023-10-10T20:42:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9e014f4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9dfc0cca50 0x7f9dfc0c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9e014f4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dfc0cce20 0x7f9dfc06bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9e014f4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dfc15df40 0x7f9dfc16d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9e014f4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0c7710 con 0x7f9dfc15df40 2023-10-10T20:42:33.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9e014f4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0b1be0 con 0x7f9dfc0cce20 2023-10-10T20:42:33.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9e014f4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0c5940 con 0x7f9dfc0cca50 2023-10-10T20:42:33.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9dfb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dfc15df40 0x7f9dfc16d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:33.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9dfb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dfc15df40 0x7f9dfc16d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59180/0 (socket says 172.21.15.89:59180) 2023-10-10T20:42:33.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9dfb7fe700 1 -- 172.21.15.89:0/244985633 learned_addr learned my addr 172.21.15.89:0/244985633 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:33.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9dfaffd700 1 --2- 172.21.15.89:0/244985633 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9dfc0cca50 0x7f9dfc0c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:33.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9df27fc700 1 --2- 172.21.15.89:0/244985633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dfc0cce20 0x7f9dfc06bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:33.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9dfb7fe700 1 -- 172.21.15.89:0/244985633 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9dfc0cca50 msgr2=0x7f9dfc0c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:33.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9dfb7fe700 1 --2- 172.21.15.89:0/244985633 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9dfc0cca50 0x7f9dfc0c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9dfb7fe700 1 -- 172.21.15.89:0/244985633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dfc0cce20 msgr2=0x7f9dfc06bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9dfb7fe700 1 --2- 172.21.15.89:0/244985633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dfc0cce20 0x7f9dfc06bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9dfb7fe700 1 -- 172.21.15.89:0/244985633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df4009ae0 con 0x7f9dfc15df40 2023-10-10T20:42:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9df27fc700 1 --2- 172.21.15.89:0/244985633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dfc0cce20 0x7f9dfc06bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9dfb7fe700 1 --2- 172.21.15.89:0/244985633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dfc15df40 0x7f9dfc16d110 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009880 tx=0x7f9df4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9df37fe700 1 -- 172.21.15.89:0/244985633 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df400beb0 con 0x7f9dfc15df40 2023-10-10T20:42:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9df37fe700 1 -- 172.21.15.89:0/244985633 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df4012e60 con 0x7f9dfc15df40 2023-10-10T20:42:33.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9e014f4700 1 -- 172.21.15.89:0/244985633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dfc16dc80 con 0x7f9dfc15df40 2023-10-10T20:42:33.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9df37fe700 1 -- 172.21.15.89:0/244985633 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df401d670 con 0x7f9dfc15df40 2023-10-10T20:42:33.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.184+0000 7f9e014f4700 1 -- 172.21.15.89:0/244985633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc16e010 con 0x7f9dfc15df40 2023-10-10T20:42:33.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.185+0000 7f9e014f4700 1 -- 172.21.15.89:0/244985633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dfc0b1d90 con 0x7f9dfc15df40 2023-10-10T20:42:33.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.186+0000 7f9df37fe700 1 -- 172.21.15.89:0/244985633 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9df401daa0 con 0x7f9dfc15df40 2023-10-10T20:42:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.186+0000 7f9df37fe700 1 --2- 172.21.15.89:0/244985633 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ddc067320 0x7f9ddc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.186+0000 7f9df37fe700 1 -- 172.21.15.89:0/244985633 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9df4086820 con 0x7f9dfc15df40 2023-10-10T20:42:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.186+0000 7f9dfaffd700 1 --2- 172.21.15.89:0/244985633 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ddc067320 0x7f9ddc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.187+0000 7f9dfaffd700 1 --2- 172.21.15.89:0/244985633 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ddc067320 0x7f9ddc0697d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9dec006fd0 tx=0x7f9dec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:33.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.190+0000 7f9df37fe700 1 -- 172.21.15.89:0/244985633 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9dfc0b1d90 con 0x7f9dfc15df40 2023-10-10T20:42:33.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.447+0000 7f9e014f4700 1 -- 172.21.15.89:0/244985633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9dfc157c50 con 0x7f9dfc15df40 2023-10-10T20:42:33.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.453+0000 7f9df37fe700 1 -- 172.21.15.89:0/244985633 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+0 (secure 0 0 0) 0x7f9dfc157c50 con 0x7f9dfc15df40 2023-10-10T20:42:33.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.455+0000 7f9e014f4700 1 -- 172.21.15.89:0/244985633 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ddc067320 msgr2=0x7f9ddc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:33.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.455+0000 7f9e014f4700 1 --2- 172.21.15.89:0/244985633 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ddc067320 0x7f9ddc0697d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9dec006fd0 tx=0x7f9dec008040 comp rx=0 tx=0).stop 2023-10-10T20:42:33.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.455+0000 7f9e014f4700 1 -- 172.21.15.89:0/244985633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dfc15df40 msgr2=0x7f9dfc16d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:33.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.455+0000 7f9e014f4700 1 --2- 172.21.15.89:0/244985633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dfc15df40 0x7f9dfc16d110 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009880 tx=0x7f9df4017000 comp rx=0 tx=0).stop 2023-10-10T20:42:33.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.455+0000 7f9e014f4700 1 -- 172.21.15.89:0/244985633 shutdown_connections 2023-10-10T20:42:33.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.455+0000 7f9e014f4700 1 --2- 172.21.15.89:0/244985633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dfc15df40 0x7f9dfc16d110 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.455+0000 7f9e014f4700 1 --2- 172.21.15.89:0/244985633 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9dfc0cca50 0x7f9dfc0c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.455+0000 7f9e014f4700 1 --2- 172.21.15.89:0/244985633 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9ddc067320 0x7f9ddc0697d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.455+0000 7f9e014f4700 1 --2- 172.21.15.89:0/244985633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9dfc0cce20 0x7f9dfc06bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.455+0000 7f9e014f4700 1 -- 172.21.15.89:0/244985633 >> 172.21.15.89:0/244985633 conn(0x7f9dfc0ce5c0 msgr2=0x7f9dfc15c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:33.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.455+0000 7f9e014f4700 1 -- 172.21.15.89:0/244985633 shutdown_connections 2023-10-10T20:42:33.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.455+0000 7f9e014f4700 1 -- 172.21.15.89:0/244985633 wait complete. 2023-10-10T20:42:33.460 INFO:teuthology.orchestra.run.smithi089.stderr:updated caps for client.0 2023-10-10T20:42:33.469 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:42:33.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.633+0000 7f59461e4700 1 Processor -- start 2023-10-10T20:42:33.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.633+0000 7f59461e4700 1 -- start start 2023-10-10T20:42:33.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.634+0000 7f59461e4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5940159890 0x7f594015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:33.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.634+0000 7f59461e4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5940069560 0x7f59400699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:33.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.634+0000 7f59461e4700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f594015a160 con 0x7f5940159c60 2023-10-10T20:42:33.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.634+0000 7f59461e4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5940069f20 con 0x7f5940069560 2023-10-10T20:42:33.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.634+0000 7f59461e4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59400d13a0 con 0x7f5940159890 2023-10-10T20:42:33.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.634+0000 7f593ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5940069560 0x7f59400699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:33.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.634+0000 7f593ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5940069560 0x7f59400699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59190/0 (socket says 172.21.15.89:59190) 2023-10-10T20:42:33.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.634+0000 7f593ffff700 1 -- 172.21.15.89:0/3521283729 learned_addr learned my addr 172.21.15.89:0/3521283729 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:33.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.634+0000 7f593f7fe700 1 --2- 172.21.15.89:0/3521283729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5940159890 0x7f594015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:33.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.634+0000 7f593ffff700 1 -- 172.21.15.89:0/3521283729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5940159890 msgr2=0x7f594015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:33.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.634+0000 7f593ffff700 1 --2- 172.21.15.89:0/3521283729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5940159890 0x7f594015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.634+0000 7f593ffff700 1 -- 172.21.15.89:0/3521283729 >> v1:172.21.15.111:6789/0 conn(0x7f5940159c60 legacy=0x7f594015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:33.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.634+0000 7f593ffff700 1 -- 172.21.15.89:0/3521283729 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594015e140 con 0x7f5940069560 2023-10-10T20:42:33.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.634+0000 7f593ffff700 1 --2- 172.21.15.89:0/3521283729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5940069560 0x7f59400699e0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f59380099b0 tx=0x7f5938009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6d09738d5eed9f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:33.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.635+0000 7f593e7fc700 1 -- 172.21.15.89:0/3521283729 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593800b820 con 0x7f5940069560 2023-10-10T20:42:33.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.635+0000 7f593e7fc700 1 -- 172.21.15.89:0/3521283729 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593800b980 con 0x7f5940069560 2023-10-10T20:42:33.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.635+0000 7f593e7fc700 1 -- 172.21.15.89:0/3521283729 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593800bd20 con 0x7f5940069560 2023-10-10T20:42:33.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.635+0000 7f59461e4700 1 -- 172.21.15.89:0/3521283729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5940069560 msgr2=0x7f59400699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:33.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.635+0000 7f59461e4700 1 --2- 172.21.15.89:0/3521283729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5940069560 0x7f59400699e0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f59380099b0 tx=0x7f5938009250 comp rx=0 tx=0).stop 2023-10-10T20:42:33.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.635+0000 7f59461e4700 1 -- 172.21.15.89:0/3521283729 shutdown_connections 2023-10-10T20:42:33.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.635+0000 7f59461e4700 1 --2- 172.21.15.89:0/3521283729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5940069560 0x7f59400699e0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.635+0000 7f59461e4700 1 --2- 172.21.15.89:0/3521283729 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5940159890 0x7f594015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.635+0000 7f59461e4700 1 -- 172.21.15.89:0/3521283729 >> 172.21.15.89:0/3521283729 conn(0x7f5940068e20 msgr2=0x7f59400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:33.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.635+0000 7f59461e4700 1 -- 172.21.15.89:0/3521283729 shutdown_connections 2023-10-10T20:42:33.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.635+0000 7f59461e4700 1 -- 172.21.15.89:0/3521283729 wait complete. 2023-10-10T20:42:33.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.636+0000 7f59461e4700 1 Processor -- start 2023-10-10T20:42:33.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.636+0000 7f59461e4700 1 -- start start 2023-10-10T20:42:33.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.637+0000 7f59461e4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5940069560 0x7f5940152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:33.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.637+0000 7f59461e4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5940159890 0x7f594014d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:33.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.637+0000 7f59461e4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5940159c60 0x7f594015c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:33.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.637+0000 7f59461e4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5940153460 con 0x7f5940159890 2023-10-10T20:42:33.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.637+0000 7f59461e4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5940151e80 con 0x7f5940069560 2023-10-10T20:42:33.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.637+0000 7f59461e4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5940152160 con 0x7f5940159c60 2023-10-10T20:42:33.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.637+0000 7f593effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5940159890 0x7f594014d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:33.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.637+0000 7f593effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5940159890 0x7f594014d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59200/0 (socket says 172.21.15.89:59200) 2023-10-10T20:42:33.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.637+0000 7f593effd700 1 -- 172.21.15.89:0/600252881 learned_addr learned my addr 172.21.15.89:0/600252881 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:33.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.637+0000 7f593ffff700 1 --2- 172.21.15.89:0/600252881 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5940159c60 0x7f594015c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:33.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.637+0000 7f593f7fe700 1 --2- 172.21.15.89:0/600252881 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5940069560 0x7f5940152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:33.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.637+0000 7f593effd700 1 -- 172.21.15.89:0/600252881 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5940159c60 msgr2=0x7f594015c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:33.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.638+0000 7f593effd700 1 --2- 172.21.15.89:0/600252881 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5940159c60 0x7f594015c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.638+0000 7f593effd700 1 -- 172.21.15.89:0/600252881 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5940069560 msgr2=0x7f5940152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:33.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.638+0000 7f593effd700 1 --2- 172.21.15.89:0/600252881 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5940069560 0x7f5940152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.638+0000 7f593effd700 1 -- 172.21.15.89:0/600252881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59380094f0 con 0x7f5940159890 2023-10-10T20:42:33.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.638+0000 7f593effd700 1 --2- 172.21.15.89:0/600252881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5940159890 0x7f594014d1e0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f593400c590 tx=0x7f5934007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:33.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.638+0000 7f59277fe700 1 -- 172.21.15.89:0/600252881 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934011590 con 0x7f5940159890 2023-10-10T20:42:33.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.638+0000 7f59277fe700 1 -- 172.21.15.89:0/600252881 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59340116f0 con 0x7f5940159890 2023-10-10T20:42:33.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.638+0000 7f59461e4700 1 -- 172.21.15.89:0/600252881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5940152440 con 0x7f5940159890 2023-10-10T20:42:33.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.638+0000 7f59277fe700 1 -- 172.21.15.89:0/600252881 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934011ab0 con 0x7f5940159890 2023-10-10T20:42:33.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.638+0000 7f59461e4700 1 -- 172.21.15.89:0/600252881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5940152a00 con 0x7f5940159890 2023-10-10T20:42:33.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.639+0000 7f59461e4700 1 -- 172.21.15.89:0/600252881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f594004c3c0 con 0x7f5940159890 2023-10-10T20:42:33.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.639+0000 7f59277fe700 1 -- 172.21.15.89:0/600252881 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f593401c460 con 0x7f5940159890 2023-10-10T20:42:33.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.640+0000 7f59277fe700 1 --2- 172.21.15.89:0/600252881 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f592c067320 0x7f592c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:33.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.640+0000 7f593f7fe700 1 --2- 172.21.15.89:0/600252881 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f592c067320 0x7f592c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:33.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.640+0000 7f59277fe700 1 -- 172.21.15.89:0/600252881 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f59340848c0 con 0x7f5940159890 2023-10-10T20:42:33.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.640+0000 7f593f7fe700 1 --2- 172.21.15.89:0/600252881 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f592c067320 0x7f592c0697d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5930009fd0 tx=0x7f5930009880 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:33.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.644+0000 7f59277fe700 1 -- 172.21.15.89:0/600252881 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5934051550 con 0x7f5940159890 2023-10-10T20:42:33.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.868+0000 7f59461e4700 1 -- 172.21.15.89:0/600252881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59400b6440 con 0x7f5940159890 2023-10-10T20:42:33.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.869+0000 7f59277fe700 1 -- 172.21.15.89:0/600252881 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4606 (secure 0 0 0) 0x7f5934054b70 con 0x7f5940159890 2023-10-10T20:42:33.871 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:33.871 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":101,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/830099189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":830099189},{"type":"v1","addr":"172.21.15.111:6834","nonce":830099189}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2380789310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2380789310},{"type":"v1","addr":"172.21.15.111:6835","nonce":2380789310}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7956,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2087922155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2087922155},{"type":"v1","addr":"172.21.15.89:6835","nonce":2087922155}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:42:23.479232+0000","modified":"2023-10-10T20:42:27.115668+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1243871428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1243871428},{"type":"v1","addr":"172.21.15.89:6837","nonce":1243871428}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-10T20:42:33.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.870+0000 7f59461e4700 1 -- 172.21.15.89:0/600252881 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f592c067320 msgr2=0x7f592c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:33.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.870+0000 7f59461e4700 1 --2- 172.21.15.89:0/600252881 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f592c067320 0x7f592c0697d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5930009fd0 tx=0x7f5930009880 comp rx=0 tx=0).stop 2023-10-10T20:42:33.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.870+0000 7f59461e4700 1 -- 172.21.15.89:0/600252881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5940159890 msgr2=0x7f594014d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:33.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.870+0000 7f59461e4700 1 --2- 172.21.15.89:0/600252881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5940159890 0x7f594014d1e0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f593400c590 tx=0x7f5934007f30 comp rx=0 tx=0).stop 2023-10-10T20:42:33.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.871+0000 7f59461e4700 1 -- 172.21.15.89:0/600252881 shutdown_connections 2023-10-10T20:42:33.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.871+0000 7f59461e4700 1 --2- 172.21.15.89:0/600252881 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5940159c60 0x7f594015c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.871+0000 7f59461e4700 1 --2- 172.21.15.89:0/600252881 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f592c067320 0x7f592c0697d0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.871+0000 7f59461e4700 1 --2- 172.21.15.89:0/600252881 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5940069560 0x7f5940152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.871+0000 7f59461e4700 1 --2- 172.21.15.89:0/600252881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5940159890 0x7f594014d1e0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:33.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.871+0000 7f59461e4700 1 -- 172.21.15.89:0/600252881 >> 172.21.15.89:0/600252881 conn(0x7f5940068e20 msgr2=0x7f59400cf510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:33.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.871+0000 7f59461e4700 1 -- 172.21.15.89:0/600252881 shutdown_connections 2023-10-10T20:42:33.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:33.871+0000 7f59461e4700 1 -- 172.21.15.89:0/600252881 wait complete. 2023-10-10T20:42:33.876 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 101 2023-10-10T20:42:34.288 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 101, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:42:23.479232+0000', 'modified': '2023-10-10T20:42:27.115668+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7959}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7959': {'gid': 7959, 'name': 'c', 'rank': 0, 'incarnation': 97, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6837/1243871428', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6836', 'nonce': 1243871428}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 1243871428}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:42:34.288 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:42:34.289 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-10T20:42:34.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.403+0000 7fca43903700 1 Processor -- start 2023-10-10T20:42:34.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.403+0000 7fca43903700 1 -- start start 2023-10-10T20:42:34.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.404+0000 7fca43903700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fca3c068550 0x7fca3c068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.404+0000 7fca43903700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fca3c15e010 0x7fca3c06f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.404+0000 7fca43903700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3c06ec70 con 0x7fca3c06f640 2023-10-10T20:42:34.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.404+0000 7fca43903700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3c0b3430 con 0x7fca3c15e010 2023-10-10T20:42:34.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.404+0000 7fca43903700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c0b2e70 con 0x7fca3c068550 2023-10-10T20:42:34.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.404+0000 7fca40e9e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fca3c15e010 0x7fca3c06f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.405+0000 7fca41ea0700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fca3c06f640 0x7fca3c06a7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54060/0 (socket says 172.21.15.89:54060) 2023-10-10T20:42:34.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.405+0000 7fca40e9e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fca3c15e010 0x7fca3c06f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59214/0 (socket says 172.21.15.89:59214) 2023-10-10T20:42:34.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.405+0000 7fca41ea0700 1 -- 172.21.15.89:0/2239845147 learned_addr learned my addr 172.21.15.89:0/2239845147 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:34.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.405+0000 7fca4169f700 1 --2- 172.21.15.89:0/2239845147 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fca3c068550 0x7fca3c068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.405+0000 7fca40e9e700 1 -- 172.21.15.89:0/2239845147 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fca3c068550 msgr2=0x7fca3c068920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:34.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.405+0000 7fca40e9e700 1 --2- 172.21.15.89:0/2239845147 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fca3c068550 0x7fca3c068920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.405+0000 7fca40e9e700 1 -- 172.21.15.89:0/2239845147 >> v1:172.21.15.89:6789/0 conn(0x7fca3c06f640 legacy=0x7fca3c06a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:34.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.405+0000 7fca40e9e700 1 -- 172.21.15.89:0/2239845147 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c0b3150 con 0x7fca3c15e010 2023-10-10T20:42:34.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.405+0000 7fca33fff700 1 -- 172.21.15.89:0/2239845147 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736424894 0 0) 0x7fca3c06ec70 con 0x7fca3c06f640 2023-10-10T20:42:34.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.405+0000 7fca40e9e700 1 --2- 172.21.15.89:0/2239845147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca3c15e010 0x7fca3c06f100 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0098e0 tx=0x7fca2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b33861f3a9dd24 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:34.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.405+0000 7fca33fff700 1 -- 172.21.15.89:0/2239845147 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c009e80 con 0x7fca3c15e010 2023-10-10T20:42:34.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.405+0000 7fca33fff700 1 -- 172.21.15.89:0/2239845147 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2c00b860 con 0x7fca3c15e010 2023-10-10T20:42:34.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.405+0000 7fca33fff700 1 -- 172.21.15.89:0/2239845147 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c00bbe0 con 0x7fca3c15e010 2023-10-10T20:42:34.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.406+0000 7fca43903700 1 -- 172.21.15.89:0/2239845147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca3c15e010 msgr2=0x7fca3c06f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:34.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.406+0000 7fca43903700 1 --2- 172.21.15.89:0/2239845147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca3c15e010 0x7fca3c06f100 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0098e0 tx=0x7fca2c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.406+0000 7fca43903700 1 -- 172.21.15.89:0/2239845147 shutdown_connections 2023-10-10T20:42:34.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.406+0000 7fca43903700 1 --2- 172.21.15.89:0/2239845147 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fca3c068550 0x7fca3c068920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.406+0000 7fca43903700 1 --2- 172.21.15.89:0/2239845147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca3c15e010 0x7fca3c06f100 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.406+0000 7fca43903700 1 -- 172.21.15.89:0/2239845147 >> 172.21.15.89:0/2239845147 conn(0x7fca3c0cd480 msgr2=0x7fca3c0cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:34.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.406+0000 7fca43903700 1 -- 172.21.15.89:0/2239845147 shutdown_connections 2023-10-10T20:42:34.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.406+0000 7fca43903700 1 -- 172.21.15.89:0/2239845147 wait complete. 2023-10-10T20:42:34.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.407+0000 7fca43903700 1 Processor -- start 2023-10-10T20:42:34.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.407+0000 7fca43903700 1 -- start start 2023-10-10T20:42:34.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca43903700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca3c068550 0x7fca3c15af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca43903700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca3c06f640 0x7fca3c153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca43903700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fca3c15e010 0x7fca3c144eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca43903700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c154090 con 0x7fca3c068550 2023-10-10T20:42:34.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca43903700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c15c1e0 con 0x7fca3c06f640 2023-10-10T20:42:34.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca43903700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c15b5d0 con 0x7fca3c15e010 2023-10-10T20:42:34.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca4169f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca3c068550 0x7fca3c15af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca4169f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca3c068550 0x7fca3c15af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59230/0 (socket says 172.21.15.89:59230) 2023-10-10T20:42:34.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca4169f700 1 -- 172.21.15.89:0/3504342703 learned_addr learned my addr 172.21.15.89:0/3504342703 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:34.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca41ea0700 1 --2- 172.21.15.89:0/3504342703 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fca3c15e010 0x7fca3c144eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca40e9e700 1 --2- 172.21.15.89:0/3504342703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca3c06f640 0x7fca3c153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca4169f700 1 -- 172.21.15.89:0/3504342703 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fca3c15e010 msgr2=0x7fca3c144eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:34.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca4169f700 1 --2- 172.21.15.89:0/3504342703 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fca3c15e010 0x7fca3c144eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca4169f700 1 -- 172.21.15.89:0/3504342703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca3c06f640 msgr2=0x7fca3c153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:34.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca4169f700 1 --2- 172.21.15.89:0/3504342703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca3c06f640 0x7fca3c153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca4169f700 1 -- 172.21.15.89:0/3504342703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c009420 con 0x7fca3c068550 2023-10-10T20:42:34.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca40e9e700 1 --2- 172.21.15.89:0/3504342703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca3c06f640 0x7fca3c153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:34.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca4169f700 1 --2- 172.21.15.89:0/3504342703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca3c068550 0x7fca3c15af10 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fca3800d890 tx=0x7fca38008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:34.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca317fa700 1 -- 172.21.15.89:0/3504342703 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca380125f0 con 0x7fca3c068550 2023-10-10T20:42:34.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca317fa700 1 -- 172.21.15.89:0/3504342703 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca38012750 con 0x7fca3c068550 2023-10-10T20:42:34.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca43903700 1 -- 172.21.15.89:0/3504342703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3c15b8b0 con 0x7fca3c068550 2023-10-10T20:42:34.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca317fa700 1 -- 172.21.15.89:0/3504342703 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca38012ad0 con 0x7fca3c068550 2023-10-10T20:42:34.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.408+0000 7fca43903700 1 -- 172.21.15.89:0/3504342703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3c15be70 con 0x7fca3c068550 2023-10-10T20:42:34.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.410+0000 7fca317fa700 1 -- 172.21.15.89:0/3504342703 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fca3801e460 con 0x7fca3c068550 2023-10-10T20:42:34.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.410+0000 7fca317fa700 1 --2- 172.21.15.89:0/3504342703 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fca20067320 0x7fca200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.410+0000 7fca317fa700 1 -- 172.21.15.89:0/3504342703 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fca38085260 con 0x7fca3c068550 2023-10-10T20:42:34.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.410+0000 7fca40e9e700 1 --2- 172.21.15.89:0/3504342703 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fca20067320 0x7fca200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.410+0000 7fca40e9e700 1 --2- 172.21.15.89:0/3504342703 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fca20067320 0x7fca200697d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0091e0 tx=0x7fca2c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:34.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.412+0000 7fca4cbfcb80 1 Processor -- start 2023-10-10T20:42:34.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.412+0000 7fca4cbfcb80 1 -- start start 2023-10-10T20:42:34.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca4cbfcb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b228a50 0x55593b228e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca4cbfcb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b272d00 0x55593b251a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca4cbfcb80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b251f60 0x55593b224cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca4cbfcb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55593b17fa70 con 0x55593b272d00 2023-10-10T20:42:34.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca4cbfcb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55593b17d270 con 0x55593b228a50 2023-10-10T20:42:34.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca4cbfcb80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55593b179060 con 0x55593b251f60 2023-10-10T20:42:34.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca40e9e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b272d00 0x55593b251a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca40e9e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b272d00 0x55593b251a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59232/0 (socket says 172.21.15.89:59232) 2023-10-10T20:42:34.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca40e9e700 1 -- 172.21.15.89:0/3892460186 learned_addr learned my addr 172.21.15.89:0/3892460186 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:34.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca4169f700 1 --2- 172.21.15.89:0/3892460186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b228a50 0x55593b228e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca41ea0700 1 --2- 172.21.15.89:0/3892460186 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b251f60 0x55593b224cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca40e9e700 1 -- 172.21.15.89:0/3892460186 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b251f60 msgr2=0x55593b224cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:34.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca40e9e700 1 --2- 172.21.15.89:0/3892460186 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b251f60 0x55593b224cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca40e9e700 1 -- 172.21.15.89:0/3892460186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b228a50 msgr2=0x55593b228e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:34.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca40e9e700 1 --2- 172.21.15.89:0/3892460186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b228a50 0x55593b228e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca40e9e700 1 -- 172.21.15.89:0/3892460186 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55593b013270 con 0x55593b272d00 2023-10-10T20:42:34.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca4169f700 1 --2- 172.21.15.89:0/3892460186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b228a50 0x55593b228e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:34.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca40e9e700 1 --2- 172.21.15.89:0/3892460186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b272d00 0x55593b251a20 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00bbf0 tx=0x7fca2c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5a5d9f700cab1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:34.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca267fc700 1 -- 172.21.15.89:0/3892460186 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c020070 con 0x55593b272d00 2023-10-10T20:42:34.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca267fc700 1 -- 172.21.15.89:0/3892460186 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2c01a4f0 con 0x55593b272d00 2023-10-10T20:42:34.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.413+0000 7fca267fc700 1 -- 172.21.15.89:0/3892460186 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c01a6a0 con 0x55593b272d00 2023-10-10T20:42:34.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.414+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/3892460186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b272d00 msgr2=0x55593b251a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:34.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.414+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/3892460186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b272d00 0x55593b251a20 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00bbf0 tx=0x7fca2c020000 comp rx=0 tx=0).stop 2023-10-10T20:42:34.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.414+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/3892460186 shutdown_connections 2023-10-10T20:42:34.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.414+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/3892460186 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b251f60 0x55593b224cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.414+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/3892460186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b228a50 0x55593b228e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.414+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/3892460186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b272d00 0x55593b251a20 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.414+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/3892460186 >> 172.21.15.89:0/3892460186 conn(0x55593b22a370 msgr2=0x55593b22a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:34.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.414+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/3892460186 shutdown_connections 2023-10-10T20:42:34.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.414+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/3892460186 wait complete. 2023-10-10T20:42:34.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.414+0000 7fca4cbfcb80 1 -- start start 2023-10-10T20:42:34.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.415+0000 7fca4cbfcb80 1 Processor -- start 2023-10-10T20:42:34.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.415+0000 7fca4cbfcb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b228a50 0x55593b24e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.415+0000 7fca4cbfcb80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b251f60 0x55593b26c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.415+0000 7fca4cbfcb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b272d00 0x55593b268e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.415+0000 7fca4cbfcb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55593b124890 con 0x55593b228a50 2023-10-10T20:42:34.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.415+0000 7fca4cbfcb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55593b13fea0 con 0x55593b272d00 2023-10-10T20:42:34.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.415+0000 7fca4cbfcb80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55593affbe30 con 0x55593b251f60 2023-10-10T20:42:34.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca4cbfcb80 10 client.0 _ll_register_callbacks cb 0x55593b1ec110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:42:34.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca4169f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b228a50 0x55593b24e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca4169f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b228a50 0x55593b24e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59244/0 (socket says 172.21.15.89:59244) 2023-10-10T20:42:34.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca4169f700 1 -- 172.21.15.89:0/1518384755 learned_addr learned my addr 172.21.15.89:0/1518384755 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:34.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca40e9e700 1 --2- 172.21.15.89:0/1518384755 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b251f60 0x55593b26c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca41ea0700 1 --2- 172.21.15.89:0/1518384755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b272d00 0x55593b268e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca4169f700 1 -- 172.21.15.89:0/1518384755 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b251f60 msgr2=0x55593b26c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:34.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca4169f700 1 --2- 172.21.15.89:0/1518384755 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b251f60 0x55593b26c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca4169f700 1 -- 172.21.15.89:0/1518384755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b272d00 msgr2=0x55593b268e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:34.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca4169f700 1 --2- 172.21.15.89:0/1518384755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b272d00 0x55593b268e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca4169f700 1 -- 172.21.15.89:0/1518384755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3c0b3150 con 0x55593b228a50 2023-10-10T20:42:34.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca4169f700 1 --2- 172.21.15.89:0/1518384755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b228a50 0x55593b24e370 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fca38055b40 tx=0x7fca38085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c0f450d4ced4e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:34.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca1b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:42:34.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca1b7fe700 1 -- 172.21.15.89:0/1518384755 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca380556f0 con 0x55593b228a50 2023-10-10T20:42:34.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca1b7fe700 1 -- 172.21.15.89:0/1518384755 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fca38058e00 con 0x55593b228a50 2023-10-10T20:42:34.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/1518384755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55593b21db70 con 0x55593b228a50 2023-10-10T20:42:34.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca1b7fe700 1 -- 172.21.15.89:0/1518384755 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca3805cb50 con 0x55593b228a50 2023-10-10T20:42:34.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca4cbfcb80 10 client.8133 fetch_fsmap learned FSMap version 101 2023-10-10T20:42:34.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/1518384755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) v3 -- 0x55593afb5fe0 con 0x55593b228a50 2023-10-10T20:42:34.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca1b7fe700 1 -- 172.21.15.89:0/1518384755 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55593b21db70 con 0x55593b228a50 2023-10-10T20:42:34.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca1b7fe700 1 -- 172.21.15.89:0/1518384755 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 101) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fca3805c6c0 con 0x55593b228a50 2023-10-10T20:42:34.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca4cbfcb80 10 client.8133 fetch_fsmap finished waiting for FSMap version 101 2023-10-10T20:42:34.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca4cbfcb80 10 client.8133 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-10T20:42:34.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca4cbfcb80 20 client.8133 populate_metadata read hostname 'smithi089' 2023-10-10T20:42:34.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.416+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/1518384755 >> [v2:172.21.15.89:6836/1243871428,v1:172.21.15.89:6837/1243871428] conn(0x55593b26a1b0 0x55593b271a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.417+0000 7fca4cbfcb80 4 client.8133 mds_command: new command op to 7959 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:42:34.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.417+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/1518384755 --> [v2:172.21.15.89:6836/1243871428,v1:172.21.15.89:6837/1243871428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55593b1faaa0 con 0x55593b26a1b0 2023-10-10T20:42:34.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.417+0000 7fca40e9e700 1 --2- 172.21.15.89:0/1518384755 >> [v2:172.21.15.89:6836/1243871428,v1:172.21.15.89:6837/1243871428] conn(0x55593b26a1b0 0x55593b271a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.417+0000 7fca40e9e700 1 --2- 172.21.15.89:0/1518384755 >> [v2:172.21.15.89:6836/1243871428,v1:172.21.15.89:6837/1243871428] conn(0x55593b26a1b0 0x55593b271a90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a8c265ee4261158 server_cookie=2b114ee7ce61f4e4 in_seq=0 out_seq=0 2023-10-10T20:42:34.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.417+0000 7fca1b7fe700 10 client.8133 ms_handle_connect on v2:172.21.15.89:6836/1243871428 2023-10-10T20:42:34.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.417+0000 7fca1b7fe700 1 -- 172.21.15.89:0/1518384755 <== mds.0 v2:172.21.15.89:6836/1243871428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55593b1faaa0 con 0x55593b26a1b0 2023-10-10T20:42:34.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.417+0000 7fca1b7fe700 10 client.8133 handle_command_reply: tid=0 2023-10-10T20:42:34.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.417+0000 7fca4cbfcb80 1 client.8133 shutdown 2023-10-10T20:42:34.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.418+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/1518384755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b228a50 msgr2=0x55593b24e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:34.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.418+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/1518384755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b228a50 0x55593b24e370 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fca38055b40 tx=0x7fca38085040 comp rx=0 tx=0).stop 2023-10-10T20:42:34.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.418+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/1518384755 shutdown_connections 2023-10-10T20:42:34.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.418+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/1518384755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b272d00 0x55593b268e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.418+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/1518384755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b228a50 0x55593b24e370 secure :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fca38055b40 tx=0x7fca38085040 comp rx=0 tx=0).stop 2023-10-10T20:42:34.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.418+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/1518384755 >> [v2:172.21.15.89:6836/1243871428,v1:172.21.15.89:6837/1243871428] conn(0x55593b26a1b0 0x55593b271a90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.418+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/1518384755 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b251f60 0x55593b26c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.418+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/1518384755 >> 172.21.15.89:0/1518384755 conn(0x55593b22a370 msgr2=0x55593b2258f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:34.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.418+0000 7fca1b7fe700 0 client.8133 ms_handle_reset on v2:172.21.15.89:6836/1243871428 2023-10-10T20:42:34.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.418+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/1518384755 shutdown_connections 2023-10-10T20:42:34.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.418+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/1518384755 wait complete. 2023-10-10T20:42:34.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.418+0000 7fca4cbfcb80 20 client.8133 trim_cache size 0 max 16384 2023-10-10T20:42:34.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.434+0000 7fca4cbfcb80 1 Processor -- start 2023-10-10T20:42:34.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.434+0000 7fca4cbfcb80 1 -- start start 2023-10-10T20:42:34.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.435+0000 7fca4cbfcb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b251f60 0x55593b225bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.435+0000 7fca4cbfcb80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b272d00 0x55593b250b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.435+0000 7fca4cbfcb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b251050 0x55593b228ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.435+0000 7fca4cbfcb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55593b269180 con 0x55593b251f60 2023-10-10T20:42:34.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.435+0000 7fca4cbfcb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55593b208510 con 0x55593b251050 2023-10-10T20:42:34.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.435+0000 7fca4cbfcb80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55593b258bc0 con 0x55593b272d00 2023-10-10T20:42:34.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.435+0000 7fca4169f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b251f60 0x55593b225bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.435+0000 7fca4169f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b251f60 0x55593b225bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59246/0 (socket says 172.21.15.89:59246) 2023-10-10T20:42:34.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.435+0000 7fca4169f700 1 -- 172.21.15.89:0/2262373968 learned_addr learned my addr 172.21.15.89:0/2262373968 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:34.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.435+0000 7fca41ea0700 1 --2- 172.21.15.89:0/2262373968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b251050 0x55593b228ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.435+0000 7fca40e9e700 1 --2- 172.21.15.89:0/2262373968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b272d00 0x55593b250b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.435+0000 7fca4169f700 1 -- 172.21.15.89:0/2262373968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b272d00 msgr2=0x55593b250b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:34.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.435+0000 7fca4169f700 1 --2- 172.21.15.89:0/2262373968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b272d00 0x55593b250b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.436+0000 7fca4169f700 1 -- 172.21.15.89:0/2262373968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b251050 msgr2=0x55593b228ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:34.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.436+0000 7fca4169f700 1 --2- 172.21.15.89:0/2262373968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b251050 0x55593b228ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.436+0000 7fca4169f700 1 -- 172.21.15.89:0/2262373968 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c146810 con 0x55593b251f60 2023-10-10T20:42:34.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.436+0000 7fca4169f700 1 --2- 172.21.15.89:0/2262373968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b251f60 0x55593b225bb0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fca38083350 tx=0x7fca3806d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d380f0fc9e9a47ed server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:34.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.436+0000 7fca26ffd700 1 -- 172.21.15.89:0/2262373968 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca380556f0 con 0x55593b251f60 2023-10-10T20:42:34.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.436+0000 7fca26ffd700 1 -- 172.21.15.89:0/2262373968 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca38058e00 con 0x55593b251f60 2023-10-10T20:42:34.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.436+0000 7fca26ffd700 1 -- 172.21.15.89:0/2262373968 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3805cb50 con 0x55593b251f60 2023-10-10T20:42:34.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.436+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/2262373968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b251f60 msgr2=0x55593b225bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:34.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.436+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/2262373968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b251f60 0x55593b225bb0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fca38083350 tx=0x7fca3806d000 comp rx=0 tx=0).stop 2023-10-10T20:42:34.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.436+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/2262373968 shutdown_connections 2023-10-10T20:42:34.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.436+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/2262373968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b251050 0x55593b228ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.436+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/2262373968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b251f60 0x55593b225bb0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.436+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/2262373968 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b272d00 0x55593b250b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.436+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/2262373968 >> 172.21.15.89:0/2262373968 conn(0x55593b22a370 msgr2=0x55593b26fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:34.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.436+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/2262373968 shutdown_connections 2023-10-10T20:42:34.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.436+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/2262373968 wait complete. 2023-10-10T20:42:34.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.437+0000 7fca4cbfcb80 1 -- start start 2023-10-10T20:42:34.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.437+0000 7fca4cbfcb80 1 Processor -- start 2023-10-10T20:42:34.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca4cbfcb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b251050 0x55593b24d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca4cbfcb80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b251f60 0x55593b273490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca4cbfcb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b272d00 0x55593b2739d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca4cbfcb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55593b2caa20 con 0x55593b251050 2023-10-10T20:42:34.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca4cbfcb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55593b2884d0 con 0x55593b272d00 2023-10-10T20:42:34.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca4cbfcb80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55593b24dab0 con 0x55593b251f60 2023-10-10T20:42:34.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca4cbfcb80 10 client.0 _ll_register_callbacks cb 0x55593b1ec110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:42:34.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca4169f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b251050 0x55593b24d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca4169f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b251050 0x55593b24d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59260/0 (socket says 172.21.15.89:59260) 2023-10-10T20:42:34.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca4169f700 1 -- 172.21.15.89:0/3708829967 learned_addr learned my addr 172.21.15.89:0/3708829967 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:34.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca40e9e700 1 --2- 172.21.15.89:0/3708829967 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b251f60 0x55593b273490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca41ea0700 1 --2- 172.21.15.89:0/3708829967 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b272d00 0x55593b2739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca4169f700 1 -- 172.21.15.89:0/3708829967 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b251f60 msgr2=0x55593b273490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:34.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca4169f700 1 --2- 172.21.15.89:0/3708829967 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b251f60 0x55593b273490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca4169f700 1 -- 172.21.15.89:0/3708829967 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b272d00 msgr2=0x55593b2739d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:34.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca4169f700 1 --2- 172.21.15.89:0/3708829967 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b272d00 0x55593b2739d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca4169f700 1 -- 172.21.15.89:0/3708829967 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55593afb5fe0 con 0x55593b251050 2023-10-10T20:42:34.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca4169f700 1 --2- 172.21.15.89:0/3708829967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b251050 0x55593b24d570 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fca38055b40 tx=0x7fca3806f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954c41c92db90f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:34.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca1b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:42:34.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca1b7fe700 1 -- 172.21.15.89:0/3708829967 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3805cb50 con 0x55593b251050 2023-10-10T20:42:34.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca1b7fe700 1 -- 172.21.15.89:0/3708829967 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fca38053cf0 con 0x55593b251050 2023-10-10T20:42:34.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/3708829967 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca2c01a4f0 con 0x55593b251050 2023-10-10T20:42:34.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca1b7fe700 1 -- 172.21.15.89:0/3708829967 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca38068120 con 0x55593b251050 2023-10-10T20:42:34.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.438+0000 7fca4cbfcb80 10 client.8139 fetch_fsmap learned FSMap version 101 2023-10-10T20:42:34.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/3708829967 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) v3 -- 0x55593b147780 con 0x55593b251050 2023-10-10T20:42:34.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca1b7fe700 1 -- 172.21.15.89:0/3708829967 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca38075080 con 0x55593b251050 2023-10-10T20:42:34.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca1b7fe700 1 -- 172.21.15.89:0/3708829967 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 101) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fca38083ba0 con 0x55593b251050 2023-10-10T20:42:34.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca4cbfcb80 10 client.8139 fetch_fsmap finished waiting for FSMap version 101 2023-10-10T20:42:34.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca4cbfcb80 10 client.8139 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-10T20:42:34.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca4cbfcb80 20 client.8139 populate_metadata read hostname 'smithi089' 2023-10-10T20:42:34.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/3708829967 >> [v2:172.21.15.89:6836/1243871428,v1:172.21.15.89:6837/1243871428] conn(0x55593b229470 0x55593b2d8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca4cbfcb80 4 client.8139 mds_command: new command op to 7959 tid=0[{"prefix": "status"}] 2023-10-10T20:42:34.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/3708829967 --> [v2:172.21.15.89:6836/1243871428,v1:172.21.15.89:6837/1243871428] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55593b11be00 con 0x55593b229470 2023-10-10T20:42:34.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca40e9e700 1 --2- 172.21.15.89:0/3708829967 >> [v2:172.21.15.89:6836/1243871428,v1:172.21.15.89:6837/1243871428] conn(0x55593b229470 0x55593b2d8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca1b7fe700 10 client.8139 ms_handle_connect on v2:172.21.15.89:6836/1243871428 2023-10-10T20:42:34.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca40e9e700 1 --2- 172.21.15.89:0/3708829967 >> [v2:172.21.15.89:6836/1243871428,v1:172.21.15.89:6837/1243871428] conn(0x55593b229470 0x55593b2d8ed0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f43861eb726b9d38 server_cookie=f1a4b1d4209ead48 in_seq=0 out_seq=0 2023-10-10T20:42:34.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca1b7fe700 1 -- 172.21.15.89:0/3708829967 <== mds.0 v2:172.21.15.89:6836/1243871428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55593b11be00 con 0x55593b229470 2023-10-10T20:42:34.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca1b7fe700 10 client.8139 handle_command_reply: tid=0 2023-10-10T20:42:34.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca4cbfcb80 1 client.8139 shutdown 2023-10-10T20:42:34.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/3708829967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b251050 msgr2=0x55593b24d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:34.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/3708829967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b251050 0x55593b24d570 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fca38055b40 tx=0x7fca3806f000 comp rx=0 tx=0).stop 2023-10-10T20:42:34.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/3708829967 shutdown_connections 2023-10-10T20:42:34.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/3708829967 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55593b272d00 0x55593b2739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/3708829967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55593b251050 0x55593b24d570 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/3708829967 >> [v2:172.21.15.89:6836/1243871428,v1:172.21.15.89:6837/1243871428] conn(0x55593b229470 0x55593b2d8ed0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca4cbfcb80 1 --2- 172.21.15.89:0/3708829967 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55593b251f60 0x55593b273490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca1b7fe700 0 client.8139 ms_handle_reset on v2:172.21.15.89:6836/1243871428 2023-10-10T20:42:34.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.439+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/3708829967 >> 172.21.15.89:0/3708829967 conn(0x55593b22a370 msgr2=0x55593b224180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:34.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.440+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/3708829967 shutdown_connections 2023-10-10T20:42:34.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.440+0000 7fca4cbfcb80 1 -- 172.21.15.89:0/3708829967 wait complete. 2023-10-10T20:42:34.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.440+0000 7fca4cbfcb80 20 client.8139 trim_cache size 0 max 16384 2023-10-10T20:42:34.467 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:42:34.467 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "5dd675b8-9026-4d8a-a1bf-62fdc5e7de67", 2023-10-10T20:42:34.467 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-10-10T20:42:34.468 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 7959, 2023-10-10T20:42:34.468 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-10-10T20:42:34.468 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-10-10T20:42:34.468 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-10-10T20:42:34.468 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 10.332676509000001, 2023-10-10T20:42:34.469 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 101, 2023-10-10T20:42:34.469 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 153, 2023-10-10T20:42:34.469 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 153, 2023-10-10T20:42:34.469 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 15.468740641 2023-10-10T20:42:34.469 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:42:34.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.440+0000 7fca43903700 1 -- 172.21.15.89:0/3504342703 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fca20067320 msgr2=0x7fca200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:34.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.440+0000 7fca43903700 1 --2- 172.21.15.89:0/3504342703 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fca20067320 0x7fca200697d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0091e0 tx=0x7fca2c017000 comp rx=0 tx=0).stop 2023-10-10T20:42:34.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.440+0000 7fca43903700 1 -- 172.21.15.89:0/3504342703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca3c068550 msgr2=0x7fca3c15af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:34.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.440+0000 7fca43903700 1 --2- 172.21.15.89:0/3504342703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca3c068550 0x7fca3c15af10 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fca3800d890 tx=0x7fca38008040 comp rx=0 tx=0).stop 2023-10-10T20:42:34.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.440+0000 7fca43903700 1 -- 172.21.15.89:0/3504342703 shutdown_connections 2023-10-10T20:42:34.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.440+0000 7fca43903700 1 --2- 172.21.15.89:0/3504342703 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fca3c15e010 0x7fca3c144eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.440+0000 7fca43903700 1 --2- 172.21.15.89:0/3504342703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca3c068550 0x7fca3c15af10 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.440+0000 7fca43903700 1 --2- 172.21.15.89:0/3504342703 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fca20067320 0x7fca200697d0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.440+0000 7fca43903700 1 --2- 172.21.15.89:0/3504342703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca3c06f640 0x7fca3c153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.440+0000 7fca43903700 1 -- 172.21.15.89:0/3504342703 >> 172.21.15.89:0/3504342703 conn(0x7fca3c0cd480 msgr2=0x7fca3c154660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:34.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.440+0000 7fca43903700 1 -- 172.21.15.89:0/3504342703 shutdown_connections 2023-10-10T20:42:34.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.440+0000 7fca43903700 1 -- 172.21.15.89:0/3504342703 wait complete. 2023-10-10T20:42:34.473 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:42:34.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.586+0000 7f73a70bb700 1 Processor -- start 2023-10-10T20:42:34.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.586+0000 7f73a70bb700 1 -- start start 2023-10-10T20:42:34.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.586+0000 7f73a70bb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73a0156690 0x7f73a00ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.586+0000 7f73a70bb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f73a00af480 0x7f73a00694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.586+0000 7f73a70bb700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f73a0156a60 0x7f73a00c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.586+0000 7f73a70bb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a006aa00 con 0x7f73a00af480 2023-10-10T20:42:34.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.586+0000 7f73a70bb700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a00699e0 con 0x7f73a0156a60 2023-10-10T20:42:34.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.586+0000 7f73a70bb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a0157e30 con 0x7f73a0156690 2023-10-10T20:42:34.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.587+0000 7f739ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f73a00af480 0x7f73a00694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.587+0000 7f739ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f73a00af480 0x7f73a00694a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59268/0 (socket says 172.21.15.89:59268) 2023-10-10T20:42:34.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.587+0000 7f739ffff700 1 -- 172.21.15.89:0/2562229393 learned_addr learned my addr 172.21.15.89:0/2562229393 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.587+0000 7f73a4e57700 1 --2- 172.21.15.89:0/2562229393 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73a0156690 0x7f73a00ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.587+0000 7f73a5658700 1 --2- 172.21.15.89:0/2562229393 >> v2:172.21.15.111:3300/0 conn(0x7f73a0156a60 0x7f73a00c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.587+0000 7f739ffff700 1 -- 172.21.15.89:0/2562229393 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73a0156690 msgr2=0x7f73a00ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.587+0000 7f739ffff700 1 --2- 172.21.15.89:0/2562229393 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73a0156690 0x7f73a00ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.587+0000 7f739ffff700 1 -- 172.21.15.89:0/2562229393 >> v2:172.21.15.111:3300/0 conn(0x7f73a0156a60 msgr2=0x7f73a00c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.587+0000 7f739ffff700 1 --2- 172.21.15.89:0/2562229393 >> v2:172.21.15.111:3300/0 conn(0x7f73a0156a60 0x7f73a00c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.587+0000 7f739ffff700 1 -- 172.21.15.89:0/2562229393 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a01578b0 con 0x7f73a00af480 2023-10-10T20:42:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.587+0000 7f73a5658700 1 --2- 172.21.15.89:0/2562229393 >> v2:172.21.15.111:3300/0 conn(0x7f73a0156a60 0x7f73a00c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.587+0000 7f739ffff700 1 --2- 172.21.15.89:0/2562229393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a00af480 0x7f73a00694a0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f73a00aec70 tx=0x7f738c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f5e0fd0334d435 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.587+0000 7f739f7fe700 1 -- 172.21.15.89:0/2562229393 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c00b860 con 0x7f73a00af480 2023-10-10T20:42:34.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.587+0000 7f739f7fe700 1 -- 172.21.15.89:0/2562229393 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738c00b9c0 con 0x7f73a00af480 2023-10-10T20:42:34.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.587+0000 7f739f7fe700 1 -- 172.21.15.89:0/2562229393 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c00bd80 con 0x7f73a00af480 2023-10-10T20:42:34.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.588+0000 7f73a70bb700 1 -- 172.21.15.89:0/2562229393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a00af480 msgr2=0x7f73a00694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:34.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.588+0000 7f73a70bb700 1 --2- 172.21.15.89:0/2562229393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a00af480 0x7f73a00694a0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f73a00aec70 tx=0x7f738c009250 comp rx=0 tx=0).stop 2023-10-10T20:42:34.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.588+0000 7f73a70bb700 1 -- 172.21.15.89:0/2562229393 shutdown_connections 2023-10-10T20:42:34.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.588+0000 7f73a70bb700 1 --2- 172.21.15.89:0/2562229393 >> v2:172.21.15.111:3300/0 conn(0x7f73a0156a60 0x7f73a00c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.588+0000 7f73a70bb700 1 --2- 172.21.15.89:0/2562229393 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73a0156690 0x7f73a00ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.588+0000 7f73a70bb700 1 --2- 172.21.15.89:0/2562229393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a00af480 0x7f73a00694a0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.588+0000 7f73a70bb700 1 -- 172.21.15.89:0/2562229393 >> 172.21.15.89:0/2562229393 conn(0x7f73a00c9870 msgr2=0x7f73a00c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:34.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.588+0000 7f73a70bb700 1 -- 172.21.15.89:0/2562229393 shutdown_connections 2023-10-10T20:42:34.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.588+0000 7f73a70bb700 1 -- 172.21.15.89:0/2562229393 wait complete. 2023-10-10T20:42:34.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.589+0000 7f73a70bb700 1 Processor -- start 2023-10-10T20:42:34.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.589+0000 7f73a70bb700 1 -- start start 2023-10-10T20:42:34.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.590+0000 7f73a70bb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a00af480 0x7f73a0153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.590+0000 7f73a70bb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73a0156690 0x7f73a00b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.590+0000 7f73a70bb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73a0156a60 0x7f73a01555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.590+0000 7f73a70bb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a00b07b0 con 0x7f73a00af480 2023-10-10T20:42:34.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.590+0000 7f73a70bb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a01543c0 con 0x7f73a0156690 2023-10-10T20:42:34.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.590+0000 7f73a70bb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a00b1180 con 0x7f73a0156a60 2023-10-10T20:42:34.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.590+0000 7f73a4e57700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a00af480 0x7f73a0153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.590+0000 7f73a4e57700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a00af480 0x7f73a0153e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59272/0 (socket says 172.21.15.89:59272) 2023-10-10T20:42:34.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.590+0000 7f73a4e57700 1 -- 172.21.15.89:0/1505964963 learned_addr learned my addr 172.21.15.89:0/1505964963 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:34.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.590+0000 7f73a5658700 1 --2- 172.21.15.89:0/1505964963 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73a0156a60 0x7f73a01555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.590+0000 7f739ffff700 1 --2- 172.21.15.89:0/1505964963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73a0156690 0x7f73a00b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.590+0000 7f73a4e57700 1 -- 172.21.15.89:0/1505964963 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73a0156a60 msgr2=0x7f73a01555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:34.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.590+0000 7f73a4e57700 1 --2- 172.21.15.89:0/1505964963 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73a0156a60 0x7f73a01555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.590+0000 7f73a4e57700 1 -- 172.21.15.89:0/1505964963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73a0156690 msgr2=0x7f73a00b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:34.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.590+0000 7f73a4e57700 1 --2- 172.21.15.89:0/1505964963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73a0156690 0x7f73a00b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.590+0000 7f73a4e57700 1 -- 172.21.15.89:0/1505964963 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738c0094f0 con 0x7f73a00af480 2023-10-10T20:42:34.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.590+0000 7f73a4e57700 1 --2- 172.21.15.89:0/1505964963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a00af480 0x7f73a0153e80 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f739400d8c0 tx=0x7f7394008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:34.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.590+0000 7f739cff9700 1 -- 172.21.15.89:0/1505964963 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73940125f0 con 0x7f73a00af480 2023-10-10T20:42:34.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.590+0000 7f739cff9700 1 -- 172.21.15.89:0/1505964963 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7394012770 con 0x7f73a00af480 2023-10-10T20:42:34.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.591+0000 7f73a70bb700 1 -- 172.21.15.89:0/1505964963 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a00afba0 con 0x7f73a00af480 2023-10-10T20:42:34.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.591+0000 7f739cff9700 1 -- 172.21.15.89:0/1505964963 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7394012b30 con 0x7f73a00af480 2023-10-10T20:42:34.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.591+0000 7f73a70bb700 1 -- 172.21.15.89:0/1505964963 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a00b0160 con 0x7f73a00af480 2023-10-10T20:42:34.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.591+0000 7f73a70bb700 1 -- 172.21.15.89:0/1505964963 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a004c3c0 con 0x7f73a00af480 2023-10-10T20:42:34.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.592+0000 7f739cff9700 1 -- 172.21.15.89:0/1505964963 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f739401e460 con 0x7f73a00af480 2023-10-10T20:42:34.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.592+0000 7f739cff9700 1 --2- 172.21.15.89:0/1505964963 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7384067320 0x7f73840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:34.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.592+0000 7f739cff9700 1 -- 172.21.15.89:0/1505964963 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7394085220 con 0x7f73a00af480 2023-10-10T20:42:34.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.592+0000 7f739ffff700 1 --2- 172.21.15.89:0/1505964963 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7384067320 0x7f73840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:34.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.593+0000 7f739ffff700 1 --2- 172.21.15.89:0/1505964963 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7384067320 0x7f73840697d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f73a00aec70 tx=0x7f738c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:34.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.597+0000 7f739cff9700 1 -- 172.21.15.89:0/1505964963 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7394055940 con 0x7f73a00af480 2023-10-10T20:42:34.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.824+0000 7f73a70bb700 1 -- 172.21.15.89:0/1505964963 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f73a006f270 con 0x7f73a00af480 2023-10-10T20:42:34.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.824+0000 7f739cff9700 1 -- 172.21.15.89:0/1505964963 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v101) v1 ==== 53+0+73 (secure 0 0 0) 0x7f73940554d0 con 0x7f73a00af480 2023-10-10T20:42:34.826 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:42:34.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.826+0000 7f73a70bb700 1 -- 172.21.15.89:0/1505964963 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7384067320 msgr2=0x7f73840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:34.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.826+0000 7f73a70bb700 1 --2- 172.21.15.89:0/1505964963 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7384067320 0x7f73840697d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f73a00aec70 tx=0x7f738c017000 comp rx=0 tx=0).stop 2023-10-10T20:42:34.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.826+0000 7f73a70bb700 1 -- 172.21.15.89:0/1505964963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a00af480 msgr2=0x7f73a0153e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:34.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.826+0000 7f73a70bb700 1 --2- 172.21.15.89:0/1505964963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a00af480 0x7f73a0153e80 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f739400d8c0 tx=0x7f7394008040 comp rx=0 tx=0).stop 2023-10-10T20:42:34.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.826+0000 7f73a70bb700 1 -- 172.21.15.89:0/1505964963 shutdown_connections 2023-10-10T20:42:34.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.826+0000 7f73a70bb700 1 --2- 172.21.15.89:0/1505964963 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f73a0156a60 0x7f73a01555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.826+0000 7f73a70bb700 1 --2- 172.21.15.89:0/1505964963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a00af480 0x7f73a0153e80 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.826+0000 7f73a70bb700 1 --2- 172.21.15.89:0/1505964963 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7384067320 0x7f73840697d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.826+0000 7f73a70bb700 1 --2- 172.21.15.89:0/1505964963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73a0156690 0x7f73a00b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:34.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.826+0000 7f73a70bb700 1 -- 172.21.15.89:0/1505964963 >> 172.21.15.89:0/1505964963 conn(0x7f73a00c9870 msgr2=0x7f73a0154c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:34.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.826+0000 7f73a70bb700 1 -- 172.21.15.89:0/1505964963 shutdown_connections 2023-10-10T20:42:34.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:34.826+0000 7f73a70bb700 1 -- 172.21.15.89:0/1505964963 wait complete. 2023-10-10T20:42:34.840 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:42:34.840 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:42:34.840 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-10T20:42:34.841 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:42:34.841 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:42:34.841 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:42:34.842 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:42:34.842 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:42:34.842 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:42:34.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:34.843 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-10T20:42:34.858 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:42:34.858 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:42:34.859 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:42:34.859 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:34.859 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-10T20:42:34.859 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:34.859 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:42:34.860 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:34.860 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-10T20:42:34.861 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:42:34.861 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:34.861 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-10T20:42:34.861 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:42:34.862 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:34.862 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:42:34.862 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:34.862 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:42:34.862 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42011sec preferred_lft 42011sec 2023-10-10T20:42:34.863 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2023-10-10T20:42:34.863 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:34.863 INFO:teuthology.orchestra.run.smithi089.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:42:34.863 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:34.863 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:42:34.864 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:34.864 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::98a3:d4ff:fec1:83bb/64 scope link 2023-10-10T20:42:34.864 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:34.864 INFO:teuthology.orchestra.run.smithi089.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:42:34.864 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether ee:16:22:10:ca:9a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:42:34.864 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec16:22ff:fe10:ca9a/64 scope link 2023-10-10T20:42:34.865 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:34.865 INFO:teuthology.orchestra.run.smithi089.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:42:34.865 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:42:34.865 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::6864:2bff:fe5c:71d4/64 scope link 2023-10-10T20:42:34.865 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:34.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:34.866 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:42:34.921 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-10T20:42:34.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:34.923 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-10T20:42:34.979 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-10T20:42:34.979 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:42:34.979 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:42:34.979 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:42:34.979 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:42:34.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:34.980 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:42:34.981 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:42:34.981 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:42:34.981 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:42:34.982 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:42:34.982 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:42:35.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:35.078 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-10T20:42:35.093 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:42:35.094 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:42:35.094 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:42:35.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:35.095 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-10-10T20:42:35.158 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:42:35.159 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:42:35.214 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:42:35.214 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:42:35.215 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:42:35.269 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:42:35.270 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:42:35.270 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:42:35.349 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:42:35.349 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:42:35.350 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:42:35.357 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:42:35.357 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:42:35.358 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:42:35.358 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:42:35.358 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:42:35.379 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:42:35.379 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:42:35.380 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:42:35.380 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:42:35.380 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:42:35.380 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:42:35.380 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:42:35.381 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:42:35.382 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:42:35.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:35.382 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:42:35.447 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:42:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.599+0000 7fe4b6f2c700 1 Processor -- start 2023-10-10T20:42:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.599+0000 7fe4b6f2c700 1 -- start start 2023-10-10T20:42:35.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.600+0000 7fe4b6f2c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe4b0159c60 0x7fe4b015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:35.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.600+0000 7fe4b6f2c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b015db10 0x7fe4b0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:35.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.600+0000 7fe4b6f2c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b015a160 con 0x7fe4b0159890 2023-10-10T20:42:35.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.600+0000 7fe4b6f2c700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4b015df50 con 0x7fe4b0159c60 2023-10-10T20:42:35.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.600+0000 7fe4b6f2c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b00d13a0 con 0x7fe4b015db10 2023-10-10T20:42:35.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.600+0000 7fe4a7fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe4b0159c60 0x7fe4b015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:35.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.600+0000 7fe4b54c9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b015db10 0x7fe4b0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:35.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.600+0000 7fe4b4cc8700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe4b0159890 0x7fe4b015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:56850/0 (socket says 172.21.15.89:56850) 2023-10-10T20:42:35.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.600+0000 7fe4b4cc8700 1 -- 172.21.15.89:0/1860841209 learned_addr learned my addr 172.21.15.89:0/1860841209 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:35.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.601+0000 7fe4a77fe700 1 -- 172.21.15.89:0/1860841209 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2985964939 0 0) 0x7fe4b015a160 con 0x7fe4b0159890 2023-10-10T20:42:35.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.601+0000 7fe4a77fe700 1 -- 172.21.15.89:0/1860841209 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe498003570 con 0x7fe4b0159890 2023-10-10T20:42:35.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.601+0000 7fe4a7fff700 1 -- 172.21.15.89:0/1860841209 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b015db10 msgr2=0x7fe4b0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:35.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.601+0000 7fe4a7fff700 1 --2- 172.21.15.89:0/1860841209 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b015db10 0x7fe4b0069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:35.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.601+0000 7fe4a7fff700 1 -- 172.21.15.89:0/1860841209 >> v1:172.21.15.111:6789/0 conn(0x7fe4b0159890 legacy=0x7fe4b015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:35.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.601+0000 7fe4a7fff700 1 -- 172.21.15.89:0/1860841209 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b015e140 con 0x7fe4b0159c60 2023-10-10T20:42:35.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.601+0000 7fe4b54c9700 1 --2- 172.21.15.89:0/1860841209 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b015db10 0x7fe4b0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:42:35.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.601+0000 7fe4a7fff700 1 --2- 172.21.15.89:0/1860841209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0159c60 0x7fe4b015d5d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00098e0 tx=0x7fe4a00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6ade0cbe84ebb41 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:35.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.601+0000 7fe4a77fe700 1 -- 172.21.15.89:0/1860841209 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0011630 con 0x7fe4b0159c60 2023-10-10T20:42:35.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.601+0000 7fe4a77fe700 1 -- 172.21.15.89:0/1860841209 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a0011790 con 0x7fe4b0159c60 2023-10-10T20:42:35.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.601+0000 7fe4a77fe700 1 -- 172.21.15.89:0/1860841209 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0011b30 con 0x7fe4b0159c60 2023-10-10T20:42:35.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.602+0000 7fe4b6f2c700 1 -- 172.21.15.89:0/1860841209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0159c60 msgr2=0x7fe4b015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:35.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.602+0000 7fe4b6f2c700 1 --2- 172.21.15.89:0/1860841209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0159c60 0x7fe4b015d5d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00098e0 tx=0x7fe4a00091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:35.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.602+0000 7fe4b6f2c700 1 -- 172.21.15.89:0/1860841209 shutdown_connections 2023-10-10T20:42:35.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.602+0000 7fe4b6f2c700 1 --2- 172.21.15.89:0/1860841209 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b015db10 0x7fe4b0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:35.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.602+0000 7fe4b6f2c700 1 --2- 172.21.15.89:0/1860841209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0159c60 0x7fe4b015d5d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:35.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.602+0000 7fe4b6f2c700 1 -- 172.21.15.89:0/1860841209 >> 172.21.15.89:0/1860841209 conn(0x7fe4b0068e20 msgr2=0x7fe4b00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:35.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.602+0000 7fe4b6f2c700 1 -- 172.21.15.89:0/1860841209 shutdown_connections 2023-10-10T20:42:35.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.602+0000 7fe4b6f2c700 1 -- 172.21.15.89:0/1860841209 wait complete. 2023-10-10T20:42:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.603+0000 7fe4b6f2c700 1 Processor -- start 2023-10-10T20:42:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.603+0000 7fe4b6f2c700 1 -- start start 2023-10-10T20:42:35.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4b6f2c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0159890 0x7fe4b016a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:35.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4b6f2c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4b0159c60 0x7fe4b015c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:35.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4b6f2c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b015db10 0x7fe4b015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:35.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4b6f2c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b014b7d0 con 0x7fe4b0159c60 2023-10-10T20:42:35.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4b6f2c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b016ab20 con 0x7fe4b0159890 2023-10-10T20:42:35.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4b6f2c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b01518e0 con 0x7fe4b015db10 2023-10-10T20:42:35.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4a7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4b0159c60 0x7fe4b015c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:35.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4a7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4b0159c60 0x7fe4b015c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59286/0 (socket says 172.21.15.89:59286) 2023-10-10T20:42:35.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4a7fff700 1 -- 172.21.15.89:0/4275646203 learned_addr learned my addr 172.21.15.89:0/4275646203 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:35.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4b4cc8700 1 --2- 172.21.15.89:0/4275646203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0159890 0x7fe4b016a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:35.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4b54c9700 1 --2- 172.21.15.89:0/4275646203 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b015db10 0x7fe4b015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:35.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4a7fff700 1 -- 172.21.15.89:0/4275646203 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b015db10 msgr2=0x7fe4b015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:35.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4a7fff700 1 --2- 172.21.15.89:0/4275646203 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b015db10 0x7fe4b015c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:35.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4a7fff700 1 -- 172.21.15.89:0/4275646203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0159890 msgr2=0x7fe4b016a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:35.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4a7fff700 1 --2- 172.21.15.89:0/4275646203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0159890 0x7fe4b016a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:35.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4a7fff700 1 -- 172.21.15.89:0/4275646203 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a8009420 con 0x7fe4b0159c60 2023-10-10T20:42:35.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4b54c9700 1 --2- 172.21.15.89:0/4275646203 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b015db10 0x7fe4b015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:35.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4a7fff700 1 --2- 172.21.15.89:0/4275646203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4b0159c60 0x7fe4b015c460 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00899c0 tx=0x7fe4a0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:35.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4a4ff9700 1 -- 172.21.15.89:0/4275646203 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0011b10 con 0x7fe4b0159c60 2023-10-10T20:42:35.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4a4ff9700 1 -- 172.21.15.89:0/4275646203 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a00253f0 con 0x7fe4b0159c60 2023-10-10T20:42:35.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4b6f2c700 1 -- 172.21.15.89:0/4275646203 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a0009420 con 0x7fe4b0159c60 2023-10-10T20:42:35.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4a4ff9700 1 -- 172.21.15.89:0/4275646203 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a00257b0 con 0x7fe4b0159c60 2023-10-10T20:42:35.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.604+0000 7fe4b6f2c700 1 -- 172.21.15.89:0/4275646203 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b0151ea0 con 0x7fe4b0159c60 2023-10-10T20:42:35.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.605+0000 7fe4b6f2c700 1 -- 172.21.15.89:0/4275646203 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b004c3c0 con 0x7fe4b0159c60 2023-10-10T20:42:35.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.606+0000 7fe4a4ff9700 1 -- 172.21.15.89:0/4275646203 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe4a0025ba0 con 0x7fe4b0159c60 2023-10-10T20:42:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.606+0000 7fe4a4ff9700 1 --2- 172.21.15.89:0/4275646203 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe49c067320 0x7fe49c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.606+0000 7fe4b4cc8700 1 --2- 172.21.15.89:0/4275646203 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe49c067320 0x7fe49c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.606+0000 7fe4a4ff9700 1 -- 172.21.15.89:0/4275646203 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe4a00865b0 con 0x7fe4b0159c60 2023-10-10T20:42:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.606+0000 7fe4b4cc8700 1 --2- 172.21.15.89:0/4275646203 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe49c067320 0x7fe49c0697d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac005c50 tx=0x7fe4ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.610+0000 7fe4a4ff9700 1 -- 172.21.15.89:0/4275646203 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4a0053240 con 0x7fe4b0159c60 2023-10-10T20:42:35.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.838+0000 7fe4b6f2c700 1 -- 172.21.15.89:0/4275646203 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe4b006a5b0 con 0x7fe4b0159c60 2023-10-10T20:42:35.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.839+0000 7fe4a4ff9700 1 -- 172.21.15.89:0/4275646203 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v101) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe4a0056860 con 0x7fe4b0159c60 2023-10-10T20:42:35.841 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:42:35.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.841+0000 7fe4b6f2c700 1 -- 172.21.15.89:0/4275646203 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe49c067320 msgr2=0x7fe49c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:35.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.841+0000 7fe4b6f2c700 1 --2- 172.21.15.89:0/4275646203 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe49c067320 0x7fe49c0697d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac005c50 tx=0x7fe4ac009040 comp rx=0 tx=0).stop 2023-10-10T20:42:35.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.841+0000 7fe4b6f2c700 1 -- 172.21.15.89:0/4275646203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4b0159c60 msgr2=0x7fe4b015c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:35.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.841+0000 7fe4b6f2c700 1 --2- 172.21.15.89:0/4275646203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4b0159c60 0x7fe4b015c460 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00899c0 tx=0x7fe4a0017000 comp rx=0 tx=0).stop 2023-10-10T20:42:35.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.841+0000 7fe4b6f2c700 1 -- 172.21.15.89:0/4275646203 shutdown_connections 2023-10-10T20:42:35.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.841+0000 7fe4b6f2c700 1 --2- 172.21.15.89:0/4275646203 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe4b015db10 0x7fe4b015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:35.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.841+0000 7fe4b6f2c700 1 --2- 172.21.15.89:0/4275646203 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe49c067320 0x7fe49c0697d0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:35.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.841+0000 7fe4b6f2c700 1 --2- 172.21.15.89:0/4275646203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0159890 0x7fe4b016a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:35.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.841+0000 7fe4b6f2c700 1 --2- 172.21.15.89:0/4275646203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4b0159c60 0x7fe4b015c460 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:35.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.841+0000 7fe4b6f2c700 1 -- 172.21.15.89:0/4275646203 >> 172.21.15.89:0/4275646203 conn(0x7fe4b0068e20 msgr2=0x7fe4b00b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:35.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.841+0000 7fe4b6f2c700 1 -- 172.21.15.89:0/4275646203 shutdown_connections 2023-10-10T20:42:35.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:35.841+0000 7fe4b6f2c700 1 -- 172.21.15.89:0/4275646203 wait complete. 2023-10-10T20:42:35.857 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:42:35.857 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:42:35.858 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-10-10T20:42:35.858 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:42:35.858 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:42:35.859 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:42:35.859 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:42:35.859 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:42:35.860 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:42:35.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:35.861 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-10-10T20:42:35.879 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:42:35.879 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:42:35.880 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:42:35.880 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:35.880 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-10-10T20:42:35.880 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:35.880 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:42:35.881 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:35.881 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-10-10T20:42:35.881 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:42:35.881 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:35.881 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-10-10T20:42:35.882 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:42:35.882 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:35.882 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:42:35.883 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:35.883 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:42:35.883 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42026sec preferred_lft 42026sec 2023-10-10T20:42:35.883 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-10-10T20:42:35.883 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:35.884 INFO:teuthology.orchestra.run.smithi111.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:42:35.884 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:42:35.884 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:42:35.884 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:35.884 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::a486:a6ff:fe5f:6021/64 scope link 2023-10-10T20:42:35.885 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:35.885 INFO:teuthology.orchestra.run.smithi111.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:42:35.885 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 1e:d0:81:1b:e2:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:42:35.885 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::1cd0:81ff:fe1b:e26b/64 scope link 2023-10-10T20:42:35.885 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:35.886 INFO:teuthology.orchestra.run.smithi111.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:42:35.886 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:42:35.886 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::88d:cdff:fe75:455e/64 scope link 2023-10-10T20:42:35.886 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:42:35.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:35.887 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:42:35.947 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-10-10T20:42:35.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:35.949 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-10-10T20:42:36.006 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-10-10T20:42:36.007 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:42:36.007 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:42:36.007 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:42:36.007 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:42:36.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:36.008 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:42:36.008 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:42:36.009 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:42:36.009 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:42:36.010 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:42:36.010 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:42:36.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:36.111 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-10-10T20:42:36.126 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:42:36.126 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:42:36.127 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:42:36.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:36.128 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-10-10T20:42:36.191 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:42:36.192 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:42:36.246 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:42:36.246 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:42:36.247 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:42:36.300 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:42:36.301 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:42:36.301 DEBUG:teuthology.orchestra.run.smithi111:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:42:36.380 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:42:36.380 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:42:36.381 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:42:36.388 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:42:36.389 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:42:36.389 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:42:36.389 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:42:36.390 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:42:36.410 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:42:36.410 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:42:36.410 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:42:36.411 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:42:36.411 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:42:36.411 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:42:36.411 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:42:36.411 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:42:36.412 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:42:36.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:36.413 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:42:36.477 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-10T20:42:36.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.597+0000 7f922337d700 1 Processor -- start 2023-10-10T20:42:36.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.597+0000 7f922337d700 1 -- start start 2023-10-10T20:42:36.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.598+0000 7f922337d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f921c159ec0 0x7f921c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:36.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.598+0000 7f922337d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f921c0b53d0 0x7f921c0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:36.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.598+0000 7f922337d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f921c0b5b00 con 0x7f921c0cce40 2023-10-10T20:42:36.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.598+0000 7f922337d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f921c0b5da0 con 0x7f921c0b53d0 2023-10-10T20:42:36.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.598+0000 7f922337d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921c0b6080 con 0x7f921c159ec0 2023-10-10T20:42:36.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.599+0000 7f9221119700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f921c159ec0 0x7f921c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:36.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.599+0000 7f9221119700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f921c159ec0 0x7f921c0b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:57950/0 (socket says 172.21.15.89:57950) 2023-10-10T20:42:36.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.599+0000 7f9220918700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f921c0b53d0 0x7f921c0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.599+0000 7f9221119700 1 -- 172.21.15.89:0/3335472155 learned_addr learned my addr 172.21.15.89:0/3335472155 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.599+0000 7f922191a700 1 --1- 172.21.15.89:0/3335472155 >> v1:172.21.15.111:6789/0 conn(0x7f921c0cce40 0x7f921c0cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:56864/0 (socket says 172.21.15.89:56864) 2023-10-10T20:42:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.599+0000 7f9220918700 1 -- 172.21.15.89:0/3335472155 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f921c159ec0 msgr2=0x7f921c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.599+0000 7f9220918700 1 --2- 172.21.15.89:0/3335472155 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f921c159ec0 0x7f921c0b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.599+0000 7f9220918700 1 -- 172.21.15.89:0/3335472155 >> v1:172.21.15.111:6789/0 conn(0x7f921c0cce40 legacy=0x7f921c0cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:36.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.599+0000 7f9213fff700 1 -- 172.21.15.89:0/3335472155 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350803303 0 0) 0x7f921c0b5b00 con 0x7f921c0cce40 2023-10-10T20:42:36.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.599+0000 7f9220918700 1 -- 172.21.15.89:0/3335472155 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c0cc100 con 0x7f921c0b53d0 2023-10-10T20:42:36.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.599+0000 7f9221119700 1 --2- 172.21.15.89:0/3335472155 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f921c159ec0 0x7f921c0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:42:36.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.599+0000 7f9220918700 1 --2- 172.21.15.89:0/3335472155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f921c0b53d0 0x7f921c0cc900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f920c0098e0 tx=0x7f920c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c87c3412028fa157 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:36.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.599+0000 7f9213fff700 1 -- 172.21.15.89:0/3335472155 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c00b640 con 0x7f921c0b53d0 2023-10-10T20:42:36.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.599+0000 7f9213fff700 1 -- 172.21.15.89:0/3335472155 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920c00b7a0 con 0x7f921c0b53d0 2023-10-10T20:42:36.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.599+0000 7f9213fff700 1 -- 172.21.15.89:0/3335472155 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c00bb40 con 0x7f921c0b53d0 2023-10-10T20:42:36.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.600+0000 7f922337d700 1 -- 172.21.15.89:0/3335472155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f921c0b53d0 msgr2=0x7f921c0cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:36.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.600+0000 7f922337d700 1 --2- 172.21.15.89:0/3335472155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f921c0b53d0 0x7f921c0cc900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f920c0098e0 tx=0x7f920c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:36.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.600+0000 7f922337d700 1 -- 172.21.15.89:0/3335472155 shutdown_connections 2023-10-10T20:42:36.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.600+0000 7f922337d700 1 --2- 172.21.15.89:0/3335472155 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f921c159ec0 0x7f921c0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:36.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.600+0000 7f922337d700 1 --2- 172.21.15.89:0/3335472155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f921c0b53d0 0x7f921c0cc900 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:36.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.600+0000 7f922337d700 1 -- 172.21.15.89:0/3335472155 >> 172.21.15.89:0/3335472155 conn(0x7f921c0999f0 msgr2=0x7f921c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:36.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.600+0000 7f922337d700 1 -- 172.21.15.89:0/3335472155 shutdown_connections 2023-10-10T20:42:36.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.600+0000 7f922337d700 1 -- 172.21.15.89:0/3335472155 wait complete. 2023-10-10T20:42:36.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.601+0000 7f922337d700 1 Processor -- start 2023-10-10T20:42:36.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.601+0000 7f922337d700 1 -- start start 2023-10-10T20:42:36.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.602+0000 7f922337d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f921c0b53d0 0x7f921c169d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:36.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.602+0000 7f922337d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f921c0cce40 0x7f921c16a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:36.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.602+0000 7f922337d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f921c159ec0 0x7f921c16a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:36.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.602+0000 7f922337d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c069d20 con 0x7f921c0b53d0 2023-10-10T20:42:36.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.602+0000 7f922337d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c159220 con 0x7f921c0cce40 2023-10-10T20:42:36.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.602+0000 7f922337d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921c16acd0 con 0x7f921c159ec0 2023-10-10T20:42:36.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.602+0000 7f9221119700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f921c0b53d0 0x7f921c169d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:36.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.602+0000 7f9221119700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f921c0b53d0 0x7f921c169d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59294/0 (socket says 172.21.15.89:59294) 2023-10-10T20:42:36.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.602+0000 7f9221119700 1 -- 172.21.15.89:0/3807192800 learned_addr learned my addr 172.21.15.89:0/3807192800 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:36.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.602+0000 7f9220918700 1 --2- 172.21.15.89:0/3807192800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f921c0cce40 0x7f921c16a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:36.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.602+0000 7f922191a700 1 --2- 172.21.15.89:0/3807192800 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f921c159ec0 0x7f921c16a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:36.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.602+0000 7f9221119700 1 -- 172.21.15.89:0/3807192800 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f921c159ec0 msgr2=0x7f921c16a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:36.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.602+0000 7f9221119700 1 --2- 172.21.15.89:0/3807192800 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f921c159ec0 0x7f921c16a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:36.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.602+0000 7f9221119700 1 -- 172.21.15.89:0/3807192800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f921c0cce40 msgr2=0x7f921c16a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:36.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.602+0000 7f9221119700 1 --2- 172.21.15.89:0/3807192800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f921c0cce40 0x7f921c16a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:36.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.602+0000 7f9221119700 1 -- 172.21.15.89:0/3807192800 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9218009ae0 con 0x7f921c0b53d0 2023-10-10T20:42:36.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.602+0000 7f9220918700 1 --2- 172.21.15.89:0/3807192800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f921c0cce40 0x7f921c16a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:42:36.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.602+0000 7f922191a700 1 --2- 172.21.15.89:0/3807192800 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f921c159ec0 0x7f921c16a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:42:36.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.602+0000 7f9221119700 1 --2- 172.21.15.89:0/3807192800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f921c0b53d0 0x7f921c169d10 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f921800c0f0 tx=0x7f921800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:36.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.603+0000 7f92117fa700 1 -- 172.21.15.89:0/3807192800 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921800c9c0 con 0x7f921c0b53d0 2023-10-10T20:42:36.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.603+0000 7f92117fa700 1 -- 172.21.15.89:0/3807192800 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921800f040 con 0x7f921c0b53d0 2023-10-10T20:42:36.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.603+0000 7f922337d700 1 -- 172.21.15.89:0/3807192800 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f920c009420 con 0x7f921c0b53d0 2023-10-10T20:42:36.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.603+0000 7f92117fa700 1 -- 172.21.15.89:0/3807192800 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921800cd30 con 0x7f921c0b53d0 2023-10-10T20:42:36.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.603+0000 7f922337d700 1 -- 172.21.15.89:0/3807192800 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921c06a560 con 0x7f921c0b53d0 2023-10-10T20:42:36.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.603+0000 7f922337d700 1 -- 172.21.15.89:0/3807192800 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f921c063c90 con 0x7f921c0b53d0 2023-10-10T20:42:36.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.604+0000 7f92117fa700 1 -- 172.21.15.89:0/3807192800 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9218014690 con 0x7f921c0b53d0 2023-10-10T20:42:36.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.604+0000 7f92117fa700 1 --2- 172.21.15.89:0/3807192800 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9200067320 0x7f92000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:36.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.604+0000 7f92117fa700 1 -- 172.21.15.89:0/3807192800 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9218085550 con 0x7f921c0b53d0 2023-10-10T20:42:36.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.604+0000 7f9220918700 1 --2- 172.21.15.89:0/3807192800 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9200067320 0x7f92000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:36.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.605+0000 7f9220918700 1 --2- 172.21.15.89:0/3807192800 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9200067320 0x7f92000697d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f920c0093f0 tx=0x7f920c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:36.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.608+0000 7f92117fa700 1 -- 172.21.15.89:0/3807192800 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9218052090 con 0x7f921c0b53d0 2023-10-10T20:42:36.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:36.835+0000 7f922337d700 1 -- 172.21.15.89:0/3807192800 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f921c0718b0 con 0x7f921c0b53d0 2023-10-10T20:42:37.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.152+0000 7f92117fa700 1 -- 172.21.15.89:0/3807192800 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v102) v1 ==== 136+0+0 (secure 0 0 0) 0x7f92180556b0 con 0x7f921c0b53d0 2023-10-10T20:42:37.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.154+0000 7f922337d700 1 -- 172.21.15.89:0/3807192800 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9200067320 msgr2=0x7f92000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:37.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.154+0000 7f922337d700 1 --2- 172.21.15.89:0/3807192800 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9200067320 0x7f92000697d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f920c0093f0 tx=0x7f920c017000 comp rx=0 tx=0).stop 2023-10-10T20:42:37.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.154+0000 7f922337d700 1 -- 172.21.15.89:0/3807192800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f921c0b53d0 msgr2=0x7f921c169d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:37.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.154+0000 7f922337d700 1 --2- 172.21.15.89:0/3807192800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f921c0b53d0 0x7f921c169d10 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f921800c0f0 tx=0x7f921800b040 comp rx=0 tx=0).stop 2023-10-10T20:42:37.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.154+0000 7f922337d700 1 -- 172.21.15.89:0/3807192800 shutdown_connections 2023-10-10T20:42:37.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.154+0000 7f922337d700 1 --2- 172.21.15.89:0/3807192800 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f921c159ec0 0x7f921c16a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.154+0000 7f922337d700 1 --2- 172.21.15.89:0/3807192800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f921c0b53d0 0x7f921c169d10 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.154+0000 7f922337d700 1 --2- 172.21.15.89:0/3807192800 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9200067320 0x7f92000697d0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.154+0000 7f922337d700 1 --2- 172.21.15.89:0/3807192800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f921c0cce40 0x7f921c16a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.154+0000 7f922337d700 1 -- 172.21.15.89:0/3807192800 >> 172.21.15.89:0/3807192800 conn(0x7f921c0999f0 msgr2=0x7f921c1556e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:37.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.154+0000 7f922337d700 1 -- 172.21.15.89:0/3807192800 shutdown_connections 2023-10-10T20:42:37.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.154+0000 7f922337d700 1 -- 172.21.15.89:0/3807192800 wait complete. 2023-10-10T20:42:37.169 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-10T20:42:37.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.282+0000 7f56543ce700 1 Processor -- start 2023-10-10T20:42:37.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.282+0000 7f56543ce700 1 -- start start 2023-10-10T20:42:37.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.283+0000 7f56543ce700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f564c159890 0x7f564c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:37.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.283+0000 7f56543ce700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f564c159c60 0x7f564c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:37.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.283+0000 7f56543ce700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f564c15a160 con 0x7f564c15db10 2023-10-10T20:42:37.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.283+0000 7f56543ce700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f564c0d13a0 con 0x7f564c159890 2023-10-10T20:42:37.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.283+0000 7f56543ce700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564c15e140 con 0x7f564c159c60 2023-10-10T20:42:37.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.283+0000 7f565216a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f564c159890 0x7f564c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:37.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.283+0000 7f565296b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f564c15db10 0x7f564c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54072/0 (socket says 172.21.15.89:54072) 2023-10-10T20:42:37.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.283+0000 7f565296b700 1 -- 172.21.15.89:0/241208435 learned_addr learned my addr 172.21.15.89:0/241208435 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:37.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.283+0000 7f5651969700 1 --2- 172.21.15.89:0/241208435 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f564c159c60 0x7f564c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:37.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.284+0000 7f565216a700 1 -- 172.21.15.89:0/241208435 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f564c159c60 msgr2=0x7f564c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:37.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.284+0000 7f565216a700 1 --2- 172.21.15.89:0/241208435 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f564c159c60 0x7f564c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.284+0000 7f565216a700 1 -- 172.21.15.89:0/241208435 >> v1:172.21.15.89:6789/0 conn(0x7f564c15db10 legacy=0x7f564c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:37.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.284+0000 7f5651168700 1 -- 172.21.15.89:0/241208435 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 427113525 0 0) 0x7f564c15a160 con 0x7f564c15db10 2023-10-10T20:42:37.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.284+0000 7f565216a700 1 -- 172.21.15.89:0/241208435 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c15a3d0 con 0x7f564c159890 2023-10-10T20:42:37.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.284+0000 7f565216a700 1 --2- 172.21.15.89:0/241208435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f564c159890 0x7f564c15abd0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f5648009e60 tx=0x7f5648009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919ef88971249529 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:37.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.284+0000 7f5651168700 1 -- 172.21.15.89:0/241208435 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564800e070 con 0x7f564c159890 2023-10-10T20:42:37.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.284+0000 7f5651168700 1 -- 172.21.15.89:0/241208435 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564800bb60 con 0x7f564c159890 2023-10-10T20:42:37.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.284+0000 7f5651168700 1 -- 172.21.15.89:0/241208435 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648012430 con 0x7f564c159890 2023-10-10T20:42:37.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.284+0000 7f56543ce700 1 -- 172.21.15.89:0/241208435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f564c159890 msgr2=0x7f564c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:37.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.284+0000 7f56543ce700 1 --2- 172.21.15.89:0/241208435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f564c159890 0x7f564c15abd0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f5648009e60 tx=0x7f5648009780 comp rx=0 tx=0).stop 2023-10-10T20:42:37.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.284+0000 7f56543ce700 1 -- 172.21.15.89:0/241208435 shutdown_connections 2023-10-10T20:42:37.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.284+0000 7f56543ce700 1 --2- 172.21.15.89:0/241208435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f564c159890 0x7f564c15abd0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.285+0000 7f56543ce700 1 --2- 172.21.15.89:0/241208435 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f564c159c60 0x7f564c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.285+0000 7f56543ce700 1 -- 172.21.15.89:0/241208435 >> 172.21.15.89:0/241208435 conn(0x7f564c068e20 msgr2=0x7f564c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:37.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.285+0000 7f56543ce700 1 -- 172.21.15.89:0/241208435 shutdown_connections 2023-10-10T20:42:37.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.285+0000 7f56543ce700 1 -- 172.21.15.89:0/241208435 wait complete. 2023-10-10T20:42:37.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.286+0000 7f56543ce700 1 Processor -- start 2023-10-10T20:42:37.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.286+0000 7f56543ce700 1 -- start start 2023-10-10T20:42:37.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.286+0000 7f56543ce700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f564c159890 0x7f564c14b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:37.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.286+0000 7f56543ce700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f564c159c60 0x7f564c16a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:37.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.286+0000 7f56543ce700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f564c15db10 0x7f564c14e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:37.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.286+0000 7f56543ce700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c0cd2e0 con 0x7f564c15db10 2023-10-10T20:42:37.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.286+0000 7f56543ce700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c16bbf0 con 0x7f564c159c60 2023-10-10T20:42:37.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.286+0000 7f56543ce700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564c16afe0 con 0x7f564c159890 2023-10-10T20:42:37.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.286+0000 7f565296b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f564c15db10 0x7f564c14e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:37.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.286+0000 7f565296b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f564c15db10 0x7f564c14e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59308/0 (socket says 172.21.15.89:59308) 2023-10-10T20:42:37.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.286+0000 7f565296b700 1 -- 172.21.15.89:0/953239722 learned_addr learned my addr 172.21.15.89:0/953239722 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:37.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.287+0000 7f565216a700 1 --2- 172.21.15.89:0/953239722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f564c159890 0x7f564c14b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:37.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.287+0000 7f565296b700 1 -- 172.21.15.89:0/953239722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f564c159890 msgr2=0x7f564c14b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:37.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.287+0000 7f565296b700 1 --2- 172.21.15.89:0/953239722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f564c159890 0x7f564c14b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.287+0000 7f565296b700 1 -- 172.21.15.89:0/953239722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f564c159c60 msgr2=0x7f564c16a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:37.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.287+0000 7f565296b700 1 --2- 172.21.15.89:0/953239722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f564c159c60 0x7f564c16a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.287+0000 7f565296b700 1 -- 172.21.15.89:0/953239722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5648009a20 con 0x7f564c15db10 2023-10-10T20:42:37.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.287+0000 7f565296b700 1 --2- 172.21.15.89:0/953239722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f564c15db10 0x7f564c14e4b0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f564400c4f0 tx=0x7f5644007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:37.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.287+0000 7f56427fc700 1 -- 172.21.15.89:0/953239722 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644005b30 con 0x7f564c15db10 2023-10-10T20:42:37.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.287+0000 7f56427fc700 1 -- 172.21.15.89:0/953239722 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5644005cb0 con 0x7f564c15db10 2023-10-10T20:42:37.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.287+0000 7f56543ce700 1 -- 172.21.15.89:0/953239722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564c16b320 con 0x7f564c15db10 2023-10-10T20:42:37.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.287+0000 7f56427fc700 1 -- 172.21.15.89:0/953239722 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644007320 con 0x7f564c15db10 2023-10-10T20:42:37.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.287+0000 7f56543ce700 1 -- 172.21.15.89:0/953239722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f564c1513c0 con 0x7f564c15db10 2023-10-10T20:42:37.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.288+0000 7f56543ce700 1 -- 172.21.15.89:0/953239722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564c16bd60 con 0x7f564c15db10 2023-10-10T20:42:37.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.288+0000 7f56427fc700 1 -- 172.21.15.89:0/953239722 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f564401b3b0 con 0x7f564c15db10 2023-10-10T20:42:37.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.289+0000 7f56427fc700 1 --2- 172.21.15.89:0/953239722 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5630067320 0x7f56300697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:37.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.289+0000 7f56427fc700 1 -- 172.21.15.89:0/953239722 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f56440841e0 con 0x7f564c15db10 2023-10-10T20:42:37.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.289+0000 7f565216a700 1 --2- 172.21.15.89:0/953239722 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5630067320 0x7f56300697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:37.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.289+0000 7f565216a700 1 --2- 172.21.15.89:0/953239722 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5630067320 0x7f56300697d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f56480097c0 tx=0x7f5648019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:37.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.293+0000 7f56427fc700 1 -- 172.21.15.89:0/953239722 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5644050de0 con 0x7f564c15db10 2023-10-10T20:42:37.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.502+0000 7f56543ce700 1 -- 172.21.15.89:0/953239722 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f564c16b4b0 con 0x7f5630067320 2023-10-10T20:42:37.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.504+0000 7f56427fc700 1 -- 172.21.15.89:0/953239722 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f564800bcd0 con 0x7f5630067320 2023-10-10T20:42:37.506 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:42:37.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:37.507 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs" 2023-10-10T20:42:37.507 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:37.507 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:42:37.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.506+0000 7f56543ce700 1 -- 172.21.15.89:0/953239722 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5630067320 msgr2=0x7f56300697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:37.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.506+0000 7f56543ce700 1 --2- 172.21.15.89:0/953239722 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5630067320 0x7f56300697d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f56480097c0 tx=0x7f5648019000 comp rx=0 tx=0).stop 2023-10-10T20:42:37.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.506+0000 7f56543ce700 1 -- 172.21.15.89:0/953239722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f564c15db10 msgr2=0x7f564c14e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:37.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.506+0000 7f56543ce700 1 --2- 172.21.15.89:0/953239722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f564c15db10 0x7f564c14e4b0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f564400c4f0 tx=0x7f5644007f50 comp rx=0 tx=0).stop 2023-10-10T20:42:37.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.506+0000 7f56543ce700 1 -- 172.21.15.89:0/953239722 shutdown_connections 2023-10-10T20:42:37.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.506+0000 7f56543ce700 1 --2- 172.21.15.89:0/953239722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f564c15db10 0x7f564c14e4b0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.506+0000 7f56543ce700 1 --2- 172.21.15.89:0/953239722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f564c159890 0x7f564c14b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.506+0000 7f56543ce700 1 --2- 172.21.15.89:0/953239722 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5630067320 0x7f56300697d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.506+0000 7f56543ce700 1 --2- 172.21.15.89:0/953239722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f564c159c60 0x7f564c16a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.506+0000 7f56543ce700 1 -- 172.21.15.89:0/953239722 >> 172.21.15.89:0/953239722 conn(0x7f564c068e20 msgr2=0x7f564c0b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:37.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.506+0000 7f56543ce700 1 -- 172.21.15.89:0/953239722 shutdown_connections 2023-10-10T20:42:37.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.506+0000 7f56543ce700 1 -- 172.21.15.89:0/953239722 wait complete. 2023-10-10T20:42:37.520 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:42:37.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.636+0000 7f945cf23700 1 Processor -- start 2023-10-10T20:42:37.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.636+0000 7f945cf23700 1 -- start start 2023-10-10T20:42:37.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.637+0000 7f945cf23700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f945806e470 0x7f945806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:37.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.637+0000 7f945cf23700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94580aea70 con 0x7f945815a3b0 2023-10-10T20:42:37.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.637+0000 7f945cf23700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f945815dad0 con 0x7f94580aef00 2023-10-10T20:42:37.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.637+0000 7f945cf23700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f945806ee30 con 0x7f945806e470 2023-10-10T20:42:37.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.637+0000 7f94577fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f945815a3b0 0x7f945815a780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54076/0 (socket says 172.21.15.89:54076) 2023-10-10T20:42:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.637+0000 7f94577fe700 1 -- 172.21.15.89:0/3991025200 learned_addr learned my addr 172.21.15.89:0/3991025200 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.637+0000 7f9457fff700 1 --2- 172.21.15.89:0/3991025200 >> v2:172.21.15.89:3300/0 conn(0x7f945806e470 0x7f945806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.637+0000 7f94567fc700 1 -- 172.21.15.89:0/3991025200 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782737999 0 0) 0x7f94580aea70 con 0x7f945815a3b0 2023-10-10T20:42:37.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.637+0000 7f94567fc700 1 -- 172.21.15.89:0/3991025200 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9440003570 con 0x7f945815a3b0 2023-10-10T20:42:37.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.637+0000 7f9457fff700 1 -- 172.21.15.89:0/3991025200 >> v1:172.21.15.111:6789/0 conn(0x7f94580aef00 legacy=0x7f945815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.637+0000 7f9457fff700 1 -- 172.21.15.89:0/3991025200 >> v1:172.21.15.89:6789/0 conn(0x7f945815a3b0 legacy=0x7f945815a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.637+0000 7f9457fff700 1 -- 172.21.15.89:0/3991025200 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94580ae480 con 0x7f945806e470 2023-10-10T20:42:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.638+0000 7f9457fff700 1 --2- 172.21.15.89:0/3991025200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f945806e470 0x7f945806e8f0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f9450009e60 tx=0x7f9450009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a669efd0507c5a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:37.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.638+0000 7f94567fc700 1 -- 172.21.15.89:0/3991025200 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945000e070 con 0x7f945806e470 2023-10-10T20:42:37.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.638+0000 7f94567fc700 1 -- 172.21.15.89:0/3991025200 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945000bb20 con 0x7f945806e470 2023-10-10T20:42:37.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.638+0000 7f94567fc700 1 -- 172.21.15.89:0/3991025200 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9450012430 con 0x7f945806e470 2023-10-10T20:42:37.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.638+0000 7f945cf23700 1 -- 172.21.15.89:0/3991025200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f945806e470 msgr2=0x7f945806e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:37.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.638+0000 7f945cf23700 1 --2- 172.21.15.89:0/3991025200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f945806e470 0x7f945806e8f0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f9450009e60 tx=0x7f9450009780 comp rx=0 tx=0).stop 2023-10-10T20:42:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.638+0000 7f945cf23700 1 -- 172.21.15.89:0/3991025200 shutdown_connections 2023-10-10T20:42:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.638+0000 7f945cf23700 1 --2- 172.21.15.89:0/3991025200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f945806e470 0x7f945806e8f0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.638+0000 7f945cf23700 1 -- 172.21.15.89:0/3991025200 >> 172.21.15.89:0/3991025200 conn(0x7f94580c9f70 msgr2=0x7f945806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:37.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.638+0000 7f945cf23700 1 -- 172.21.15.89:0/3991025200 shutdown_connections 2023-10-10T20:42:37.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.638+0000 7f945cf23700 1 -- 172.21.15.89:0/3991025200 wait complete. 2023-10-10T20:42:37.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.639+0000 7f945cf23700 1 Processor -- start 2023-10-10T20:42:37.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.640+0000 7f945cf23700 1 -- start start 2023-10-10T20:42:37.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.640+0000 7f945cf23700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f945806e470 0x7f94580c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:37.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.640+0000 7f945cf23700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94580aef00 0x7f94581543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:37.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.640+0000 7f945cf23700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f945815a3b0 0x7f94581523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:37.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.640+0000 7f945cf23700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94580c4870 con 0x7f94580aef00 2023-10-10T20:42:37.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.640+0000 7f945cf23700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94580c3c60 con 0x7f945815a3b0 2023-10-10T20:42:37.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.640+0000 7f945cf23700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94580c3f40 con 0x7f945806e470 2023-10-10T20:42:37.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.640+0000 7f9456ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94580aef00 0x7f94581543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:37.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.640+0000 7f9456ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94580aef00 0x7f94581543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59326/0 (socket says 172.21.15.89:59326) 2023-10-10T20:42:37.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.640+0000 7f9456ffd700 1 -- 172.21.15.89:0/1767031977 learned_addr learned my addr 172.21.15.89:0/1767031977 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:37.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.641+0000 7f9457fff700 1 --2- 172.21.15.89:0/1767031977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f945815a3b0 0x7f94581523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:37.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.641+0000 7f94577fe700 1 --2- 172.21.15.89:0/1767031977 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f945806e470 0x7f94580c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:37.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.641+0000 7f9456ffd700 1 -- 172.21.15.89:0/1767031977 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f945806e470 msgr2=0x7f94580c4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:37.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.641+0000 7f9456ffd700 1 --2- 172.21.15.89:0/1767031977 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f945806e470 0x7f94580c4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.641+0000 7f9456ffd700 1 -- 172.21.15.89:0/1767031977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f945815a3b0 msgr2=0x7f94581523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:37.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.641+0000 7f9456ffd700 1 --2- 172.21.15.89:0/1767031977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f945815a3b0 0x7f94581523f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.641+0000 7f9456ffd700 1 -- 172.21.15.89:0/1767031977 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9450009a20 con 0x7f94580aef00 2023-10-10T20:42:37.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.641+0000 7f9456ffd700 1 --2- 172.21.15.89:0/1767031977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94580aef00 0x7f94581543d0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f944c00c5d0 tx=0x7f944c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:37.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.641+0000 7f943f7fe700 1 -- 172.21.15.89:0/1767031977 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c011590 con 0x7f94580aef00 2023-10-10T20:42:37.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.641+0000 7f943f7fe700 1 -- 172.21.15.89:0/1767031977 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944c011710 con 0x7f94580aef00 2023-10-10T20:42:37.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.641+0000 7f945cf23700 1 -- 172.21.15.89:0/1767031977 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94580c4220 con 0x7f94580aef00 2023-10-10T20:42:37.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.641+0000 7f943f7fe700 1 -- 172.21.15.89:0/1767031977 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c011ad0 con 0x7f94580aef00 2023-10-10T20:42:37.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.641+0000 7f945cf23700 1 -- 172.21.15.89:0/1767031977 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9458154910 con 0x7f94580aef00 2023-10-10T20:42:37.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.642+0000 7f945cf23700 1 -- 172.21.15.89:0/1767031977 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94580c4a20 con 0x7f94580aef00 2023-10-10T20:42:37.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.642+0000 7f943f7fe700 1 -- 172.21.15.89:0/1767031977 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f944c01c460 con 0x7f94580aef00 2023-10-10T20:42:37.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.643+0000 7f943f7fe700 1 --2- 172.21.15.89:0/1767031977 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9444067320 0x7f94440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:37.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.643+0000 7f943f7fe700 1 -- 172.21.15.89:0/1767031977 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f944c084a20 con 0x7f94580aef00 2023-10-10T20:42:37.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.643+0000 7f94577fe700 1 --2- 172.21.15.89:0/1767031977 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9444067320 0x7f94440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:37.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.643+0000 7f94577fe700 1 --2- 172.21.15.89:0/1767031977 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9444067320 0x7f94440697d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f94480058d0 tx=0x7f9448009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:37.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.647+0000 7f943f7fe700 1 -- 172.21.15.89:0/1767031977 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94580c4a20 con 0x7f94580aef00 2023-10-10T20:42:37.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.830+0000 7f945cf23700 1 -- 172.21.15.89:0/1767031977 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f945806d010 con 0x7f94580aef00 2023-10-10T20:42:37.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.831+0000 7f943f7fe700 1 -- 172.21.15.89:0/1767031977 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f944c0516b0 con 0x7f94580aef00 2023-10-10T20:42:37.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.833+0000 7f945cf23700 1 -- 172.21.15.89:0/1767031977 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9444067320 msgr2=0x7f94440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:37.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.833+0000 7f945cf23700 1 --2- 172.21.15.89:0/1767031977 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9444067320 0x7f94440697d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f94480058d0 tx=0x7f9448009040 comp rx=0 tx=0).stop 2023-10-10T20:42:37.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.833+0000 7f945cf23700 1 -- 172.21.15.89:0/1767031977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94580aef00 msgr2=0x7f94581543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:37.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.833+0000 7f945cf23700 1 --2- 172.21.15.89:0/1767031977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94580aef00 0x7f94581543d0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f944c00c5d0 tx=0x7f944c007f60 comp rx=0 tx=0).stop 2023-10-10T20:42:37.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.833+0000 7f945cf23700 1 -- 172.21.15.89:0/1767031977 shutdown_connections 2023-10-10T20:42:37.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.833+0000 7f945cf23700 1 --2- 172.21.15.89:0/1767031977 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9444067320 0x7f94440697d0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.833+0000 7f945cf23700 1 --2- 172.21.15.89:0/1767031977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f945815a3b0 0x7f94581523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.833+0000 7f945cf23700 1 --2- 172.21.15.89:0/1767031977 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f945806e470 0x7f94580c4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.833+0000 7f945cf23700 1 --2- 172.21.15.89:0/1767031977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94580aef00 0x7f94581543d0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.833+0000 7f945cf23700 1 -- 172.21.15.89:0/1767031977 >> 172.21.15.89:0/1767031977 conn(0x7f94580c9f70 msgr2=0x7f945815b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:37.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.833+0000 7f945cf23700 1 -- 172.21.15.89:0/1767031977 shutdown_connections 2023-10-10T20:42:37.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.833+0000 7f945cf23700 1 -- 172.21.15.89:0/1767031977 wait complete. 2023-10-10T20:42:37.847 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000301286 2023-10-10T20:42:37.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.963+0000 7f23b679e700 1 Processor -- start 2023-10-10T20:42:37.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.963+0000 7f23b679e700 1 -- start start 2023-10-10T20:42:37.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.964+0000 7f23b679e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f23b00b4e70 0x7f23b00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:37.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.964+0000 7f23b679e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f23b00b16d0 0x7f23b00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:37.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.964+0000 7f23b679e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b00b2080 con 0x7f23b015d240 2023-10-10T20:42:37.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.964+0000 7f23b679e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b00cfb00 con 0x7f23b00b16d0 2023-10-10T20:42:37.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.964+0000 7f23b679e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b00b5c40 con 0x7f23b00b4e70 2023-10-10T20:42:37.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.964+0000 7f23b4f9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f23b00b16d0 0x7f23b00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:37.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.964+0000 7f23b5f9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f23b015d240 0x7f23b015d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54086/0 (socket says 172.21.15.89:54086) 2023-10-10T20:42:37.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.964+0000 7f23b5f9d700 1 -- 172.21.15.89:0/1363328526 learned_addr learned my addr 172.21.15.89:0/1363328526 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:37.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.964+0000 7f23b579c700 1 --2- 172.21.15.89:0/1363328526 >> v2:172.21.15.111:3300/0 conn(0x7f23b00b4e70 0x7f23b00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:37.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.964+0000 7f23b4f9b700 1 -- 172.21.15.89:0/1363328526 >> v2:172.21.15.111:3300/0 conn(0x7f23b00b4e70 msgr2=0x7f23b00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:37.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.964+0000 7f23b4f9b700 1 --2- 172.21.15.89:0/1363328526 >> v2:172.21.15.111:3300/0 conn(0x7f23b00b4e70 0x7f23b00ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.964+0000 7f23b4f9b700 1 -- 172.21.15.89:0/1363328526 >> v1:172.21.15.89:6789/0 conn(0x7f23b015d240 legacy=0x7f23b015d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:37.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.964+0000 7f23a7fff700 1 -- 172.21.15.89:0/1363328526 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3983543113 0 0) 0x7f23b00b2080 con 0x7f23b015d240 2023-10-10T20:42:37.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.964+0000 7f23b4f9b700 1 -- 172.21.15.89:0/1363328526 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b00b5f20 con 0x7f23b00b16d0 2023-10-10T20:42:37.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.964+0000 7f23b579c700 1 --2- 172.21.15.89:0/1363328526 >> v2:172.21.15.111:3300/0 conn(0x7f23b00b4e70 0x7f23b00ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:37.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.964+0000 7f23b4f9b700 1 --2- 172.21.15.89:0/1363328526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23b00b16d0 0x7f23b00b1b40 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f23b0064910 tx=0x7f23a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448e79f2399762df server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:37.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.965+0000 7f23a7fff700 1 -- 172.21.15.89:0/1363328526 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a000b6b0 con 0x7f23b00b16d0 2023-10-10T20:42:37.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.965+0000 7f23a7fff700 1 -- 172.21.15.89:0/1363328526 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a000b810 con 0x7f23b00b16d0 2023-10-10T20:42:37.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.965+0000 7f23a7fff700 1 -- 172.21.15.89:0/1363328526 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a000bbb0 con 0x7f23b00b16d0 2023-10-10T20:42:37.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.965+0000 7f23b679e700 1 -- 172.21.15.89:0/1363328526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23b00b16d0 msgr2=0x7f23b00b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:37.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.965+0000 7f23b679e700 1 --2- 172.21.15.89:0/1363328526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23b00b16d0 0x7f23b00b1b40 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f23b0064910 tx=0x7f23a00091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.965+0000 7f23b679e700 1 -- 172.21.15.89:0/1363328526 shutdown_connections 2023-10-10T20:42:37.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.965+0000 7f23b679e700 1 --2- 172.21.15.89:0/1363328526 >> v2:172.21.15.111:3300/0 conn(0x7f23b00b4e70 0x7f23b00ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.965+0000 7f23b679e700 1 --2- 172.21.15.89:0/1363328526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23b00b16d0 0x7f23b00b1b40 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.965+0000 7f23b679e700 1 -- 172.21.15.89:0/1363328526 >> 172.21.15.89:0/1363328526 conn(0x7f23b00d10c0 msgr2=0x7f23b00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:37.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.965+0000 7f23b679e700 1 -- 172.21.15.89:0/1363328526 shutdown_connections 2023-10-10T20:42:37.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.965+0000 7f23b679e700 1 -- 172.21.15.89:0/1363328526 wait complete. 2023-10-10T20:42:37.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.966+0000 7f23b679e700 1 Processor -- start 2023-10-10T20:42:37.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.966+0000 7f23b679e700 1 -- start start 2023-10-10T20:42:37.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.967+0000 7f23b679e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23b00b16d0 0x7f23b014c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:37.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.967+0000 7f23b679e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23b00b4e70 0x7f23b00ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:37.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.967+0000 7f23b679e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23b015d240 0x7f23b00ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:37.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.967+0000 7f23b679e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b00c8330 con 0x7f23b00b4e70 2023-10-10T20:42:37.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.967+0000 7f23b679e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b00c7a00 con 0x7f23b00b16d0 2023-10-10T20:42:37.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.967+0000 7f23b679e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b00c7ce0 con 0x7f23b015d240 2023-10-10T20:42:37.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.967+0000 7f23b4f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23b00b4e70 0x7f23b00ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:37.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.967+0000 7f23b4f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23b00b4e70 0x7f23b00ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59350/0 (socket says 172.21.15.89:59350) 2023-10-10T20:42:37.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.967+0000 7f23b4f9b700 1 -- 172.21.15.89:0/838172352 learned_addr learned my addr 172.21.15.89:0/838172352 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:37.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.967+0000 7f23b579c700 1 --2- 172.21.15.89:0/838172352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23b00b16d0 0x7f23b014c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:37.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.967+0000 7f23b5f9d700 1 --2- 172.21.15.89:0/838172352 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23b015d240 0x7f23b00ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:37.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.968+0000 7f23b4f9b700 1 -- 172.21.15.89:0/838172352 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23b015d240 msgr2=0x7f23b00ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:37.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.968+0000 7f23b4f9b700 1 --2- 172.21.15.89:0/838172352 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23b015d240 0x7f23b00ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.968+0000 7f23b4f9b700 1 -- 172.21.15.89:0/838172352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23b00b16d0 msgr2=0x7f23b014c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:37.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.968+0000 7f23b4f9b700 1 --2- 172.21.15.89:0/838172352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23b00b16d0 0x7f23b014c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:37.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.968+0000 7f23b4f9b700 1 -- 172.21.15.89:0/838172352 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a0009420 con 0x7f23b00b4e70 2023-10-10T20:42:37.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.968+0000 7f23b5f9d700 1 --2- 172.21.15.89:0/838172352 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23b015d240 0x7f23b00ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:37.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.968+0000 7f23b4f9b700 1 --2- 172.21.15.89:0/838172352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23b00b4e70 0x7f23b00ca1f0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f23b0070dd0 tx=0x7f23a0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:37.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.968+0000 7f23a57fa700 1 -- 172.21.15.89:0/838172352 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a000bb90 con 0x7f23b00b4e70 2023-10-10T20:42:37.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.968+0000 7f23a57fa700 1 -- 172.21.15.89:0/838172352 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a0011420 con 0x7f23b00b4e70 2023-10-10T20:42:37.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.968+0000 7f23b679e700 1 -- 172.21.15.89:0/838172352 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b00c7fc0 con 0x7f23b00b4e70 2023-10-10T20:42:37.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.968+0000 7f23a57fa700 1 -- 172.21.15.89:0/838172352 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a00117a0 con 0x7f23b00b4e70 2023-10-10T20:42:37.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.968+0000 7f23b679e700 1 -- 172.21.15.89:0/838172352 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b014c7d0 con 0x7f23b00b4e70 2023-10-10T20:42:37.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.969+0000 7f23b679e700 1 -- 172.21.15.89:0/838172352 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b00b4ca0 con 0x7f23b00b4e70 2023-10-10T20:42:37.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.969+0000 7f23a57fa700 1 -- 172.21.15.89:0/838172352 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f23a0011b90 con 0x7f23b00b4e70 2023-10-10T20:42:37.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.970+0000 7f23a57fa700 1 --2- 172.21.15.89:0/838172352 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2394067320 0x7f23940697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:37.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.970+0000 7f23a57fa700 1 -- 172.21.15.89:0/838172352 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f23a0086610 con 0x7f23b00b4e70 2023-10-10T20:42:37.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.970+0000 7f23b579c700 1 --2- 172.21.15.89:0/838172352 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2394067320 0x7f23940697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:37.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.970+0000 7f23b579c700 1 --2- 172.21.15.89:0/838172352 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2394067320 0x7f23940697d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f23b00cf8f0 tx=0x7f23ac006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:37.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:37.974+0000 7f23a57fa700 1 -- 172.21.15.89:0/838172352 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23a0053320 con 0x7f23b00b4e70 2023-10-10T20:42:38.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.184+0000 7f23b679e700 1 -- 172.21.15.89:0/838172352 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000301286", "target": ["mon-mgr", ""]}) v1 -- 0x7f23b014c7d0 con 0x7f2394067320 2023-10-10T20:42:38.186 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:42:38.185+0000 7f023b8aa700 -1 mgr.server reply reply (1) Operation not permitted WARNING: This will rename the filesystem and possibly its pools. It is a potentially disruptive operation, clients' cephx credentials need reauthorized to access the file system and its pools with the new name. Add --yes-i-really-mean-it if you are sure you wish to continue. 2023-10-10T20:42:38.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.185+0000 7f23a57fa700 1 -- 172.21.15.89:0/838172352 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: -1 WARNING: This will rename the filesystem and possibly its pools. It is a potentially disruptive operation, clients' cephx credentials need reauthorized to access the file system and its pools with the new name. Add --yes-i-really-mean-it if you are sure you wish to continue.) v1 ==== 283+0+0 (secure 0 0 0) 0x7f23b014c7d0 con 0x7f2394067320 2023-10-10T20:42:38.188 INFO:teuthology.orchestra.run.smithi089.stderr:Error EPERM: WARNING: This will rename the filesystem and possibly its pools. It is a potentially disruptive operation, clients' cephx credentials need reauthorized to access the file system and its pools with the new name. Add --yes-i-really-mean-it if you are sure you wish to continue. 2023-10-10T20:42:38.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:42:38.192 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:42:38.192 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:42:38.229 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:42:38.273 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:42:38.273 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:42:38.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:42:38.274 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:42:38.310 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:42:38.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:38.311 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:42:38.366 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:42:38.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:42:38.367 DEBUG:teuthology.orchestra.run.smithi111:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:42:38.408 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:42:38.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:42:38.410 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:42:38.466 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:42:38.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.579+0000 7fe755b26700 1 Processor -- start 2023-10-10T20:42:38.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.579+0000 7fe755b26700 1 -- start start 2023-10-10T20:42:38.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.580+0000 7fe755b26700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe75006c0a0 0x7fe7500caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:38.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.580+0000 7fe755b26700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe75015df50 0x7fe75015e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:38.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.580+0000 7fe755b26700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe75015d7e0 con 0x7fe75006bcd0 2023-10-10T20:42:38.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.580+0000 7fe755b26700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe75015dac0 con 0x7fe75006c0a0 2023-10-10T20:42:38.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.580+0000 7fe755b26700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75015aa30 con 0x7fe75015df50 2023-10-10T20:42:38.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.580+0000 7fe74effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe75006c0a0 0x7fe7500caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:38.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.580+0000 7fe74f7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe75006bcd0 0x7fe75015a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59156/0 (socket says 172.21.15.89:59156) 2023-10-10T20:42:38.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.580+0000 7fe74f7fe700 1 -- 172.21.15.89:0/3771056035 learned_addr learned my addr 172.21.15.89:0/3771056035 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:38.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.580+0000 7fe74ffff700 1 --2- 172.21.15.89:0/3771056035 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe75015df50 0x7fe75015e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:38.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.580+0000 7fe74effd700 1 -- 172.21.15.89:0/3771056035 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe75015df50 msgr2=0x7fe75015e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:38.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.580+0000 7fe74effd700 1 --2- 172.21.15.89:0/3771056035 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe75015df50 0x7fe75015e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:38.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.580+0000 7fe74effd700 1 -- 172.21.15.89:0/3771056035 >> v1:172.21.15.89:6789/0 conn(0x7fe75006bcd0 legacy=0x7fe75015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:38.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.580+0000 7fe74effd700 1 -- 172.21.15.89:0/3771056035 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75015ad10 con 0x7fe75006c0a0 2023-10-10T20:42:38.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.580+0000 7fe74e7fc700 1 -- 172.21.15.89:0/3771056035 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453374041 0 0) 0x7fe75015d7e0 con 0x7fe75006bcd0 2023-10-10T20:42:38.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.580+0000 7fe74effd700 1 --2- 172.21.15.89:0/3771056035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe75006c0a0 0x7fe7500caa90 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fe7440098e0 tx=0x7fe7440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a5a0d71933a8f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:38.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.580+0000 7fe74e7fc700 1 -- 172.21.15.89:0/3771056035 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744009e80 con 0x7fe75006c0a0 2023-10-10T20:42:38.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.580+0000 7fe74e7fc700 1 -- 172.21.15.89:0/3771056035 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74400b860 con 0x7fe75006c0a0 2023-10-10T20:42:38.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.580+0000 7fe74e7fc700 1 -- 172.21.15.89:0/3771056035 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74400bbe0 con 0x7fe75006c0a0 2023-10-10T20:42:38.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.581+0000 7fe755b26700 1 -- 172.21.15.89:0/3771056035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe75006c0a0 msgr2=0x7fe7500caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:38.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.581+0000 7fe755b26700 1 --2- 172.21.15.89:0/3771056035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe75006c0a0 0x7fe7500caa90 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fe7440098e0 tx=0x7fe7440091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:38.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.581+0000 7fe755b26700 1 -- 172.21.15.89:0/3771056035 shutdown_connections 2023-10-10T20:42:38.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.581+0000 7fe755b26700 1 --2- 172.21.15.89:0/3771056035 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe75015df50 0x7fe75015e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:38.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.581+0000 7fe755b26700 1 --2- 172.21.15.89:0/3771056035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe75006c0a0 0x7fe7500caa90 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:38.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.581+0000 7fe755b26700 1 -- 172.21.15.89:0/3771056035 >> 172.21.15.89:0/3771056035 conn(0x7fe7500ce600 msgr2=0x7fe7500cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:38.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.581+0000 7fe755b26700 1 -- 172.21.15.89:0/3771056035 shutdown_connections 2023-10-10T20:42:38.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.581+0000 7fe755b26700 1 -- 172.21.15.89:0/3771056035 wait complete. 2023-10-10T20:42:38.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.582+0000 7fe755b26700 1 Processor -- start 2023-10-10T20:42:38.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.582+0000 7fe755b26700 1 -- start start 2023-10-10T20:42:38.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.583+0000 7fe755b26700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe75006bcd0 0x7fe7500b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:38.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.583+0000 7fe755b26700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe75006c0a0 0x7fe75015c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:38.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.583+0000 7fe755b26700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe75015df50 0x7fe75015cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:38.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.583+0000 7fe755b26700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe750169f80 con 0x7fe75006bcd0 2023-10-10T20:42:38.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.583+0000 7fe755b26700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7500b30e0 con 0x7fe75015df50 2023-10-10T20:42:38.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.583+0000 7fe755b26700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7500c6d10 con 0x7fe75006c0a0 2023-10-10T20:42:38.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.583+0000 7fe74f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe75006bcd0 0x7fe7500b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:38.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.583+0000 7fe74f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe75006bcd0 0x7fe7500b0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38866/0 (socket says 172.21.15.89:38866) 2023-10-10T20:42:38.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.583+0000 7fe74f7fe700 1 -- 172.21.15.89:0/3930944888 learned_addr learned my addr 172.21.15.89:0/3930944888 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:38.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.583+0000 7fe74effd700 1 --2- 172.21.15.89:0/3930944888 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe75006c0a0 0x7fe75015c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:38.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.583+0000 7fe74ffff700 1 --2- 172.21.15.89:0/3930944888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe75015df50 0x7fe75015cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:38.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.583+0000 7fe74f7fe700 1 -- 172.21.15.89:0/3930944888 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe75006c0a0 msgr2=0x7fe75015c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:38.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.583+0000 7fe74f7fe700 1 --2- 172.21.15.89:0/3930944888 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe75006c0a0 0x7fe75015c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:38.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.583+0000 7fe74f7fe700 1 -- 172.21.15.89:0/3930944888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe75015df50 msgr2=0x7fe75015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:38.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.583+0000 7fe74f7fe700 1 --2- 172.21.15.89:0/3930944888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe75015df50 0x7fe75015cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:38.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.583+0000 7fe74f7fe700 1 -- 172.21.15.89:0/3930944888 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe744009420 con 0x7fe75006bcd0 2023-10-10T20:42:38.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.583+0000 7fe74ffff700 1 --2- 172.21.15.89:0/3930944888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe75015df50 0x7fe75015cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:38.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.583+0000 7fe74f7fe700 1 --2- 172.21.15.89:0/3930944888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe75006bcd0 0x7fe7500b0110 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fe74000c5b0 tx=0x7fe740001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:38.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.584+0000 7fe7377fe700 1 -- 172.21.15.89:0/3930944888 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740005b30 con 0x7fe75006bcd0 2023-10-10T20:42:38.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.584+0000 7fe7377fe700 1 -- 172.21.15.89:0/3930944888 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe740005c90 con 0x7fe75006bcd0 2023-10-10T20:42:38.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.584+0000 7fe755b26700 1 -- 172.21.15.89:0/3930944888 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7500c6ff0 con 0x7fe75006bcd0 2023-10-10T20:42:38.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.584+0000 7fe7377fe700 1 -- 172.21.15.89:0/3930944888 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740007300 con 0x7fe75006bcd0 2023-10-10T20:42:38.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.584+0000 7fe755b26700 1 -- 172.21.15.89:0/3930944888 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7500c7380 con 0x7fe75006bcd0 2023-10-10T20:42:38.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.584+0000 7fe755b26700 1 -- 172.21.15.89:0/3930944888 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75004c3c0 con 0x7fe75006bcd0 2023-10-10T20:42:38.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.585+0000 7fe7377fe700 1 -- 172.21.15.89:0/3930944888 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe74001b3b0 con 0x7fe75006bcd0 2023-10-10T20:42:38.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.585+0000 7fe7377fe700 1 --2- 172.21.15.89:0/3930944888 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe73c067320 0x7fe73c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:38.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.585+0000 7fe7377fe700 1 -- 172.21.15.89:0/3930944888 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe7400841a0 con 0x7fe75006bcd0 2023-10-10T20:42:38.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.585+0000 7fe74effd700 1 --2- 172.21.15.89:0/3930944888 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe73c067320 0x7fe73c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:38.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.586+0000 7fe74effd700 1 --2- 172.21.15.89:0/3930944888 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe73c067320 0x7fe73c0697d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe75006d110 tx=0x7fe744017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:38.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.590+0000 7fe7377fe700 1 -- 172.21.15.89:0/3930944888 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe740050e30 con 0x7fe75006bcd0 2023-10-10T20:42:38.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.818+0000 7fe755b26700 1 -- 172.21.15.89:0/3930944888 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe75015aff0 con 0x7fe75006bcd0 2023-10-10T20:42:38.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.818+0000 7fe7377fe700 1 -- 172.21.15.89:0/3930944888 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4606 (secure 0 0 0) 0x7fe740054450 con 0x7fe75006bcd0 2023-10-10T20:42:38.821 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:38.821 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":102,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/830099189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":830099189},{"type":"v1","addr":"172.21.15.111:6834","nonce":830099189}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2380789310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2380789310},{"type":"v1","addr":"172.21.15.111:6835","nonce":2380789310}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7956,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2087922155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2087922155},{"type":"v1","addr":"172.21.15.89:6835","nonce":2087922155}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:42:23.479232+0000","modified":"2023-10-10T20:42:27.115668+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1243871428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1243871428},{"type":"v1","addr":"172.21.15.89:6837","nonce":1243871428}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-10T20:42:38.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.820+0000 7fe755b26700 1 -- 172.21.15.89:0/3930944888 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe73c067320 msgr2=0x7fe73c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:38.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.820+0000 7fe755b26700 1 --2- 172.21.15.89:0/3930944888 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe73c067320 0x7fe73c0697d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe75006d110 tx=0x7fe744017000 comp rx=0 tx=0).stop 2023-10-10T20:42:38.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.820+0000 7fe755b26700 1 -- 172.21.15.89:0/3930944888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe75006bcd0 msgr2=0x7fe7500b0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:38.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.820+0000 7fe755b26700 1 --2- 172.21.15.89:0/3930944888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe75006bcd0 0x7fe7500b0110 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fe74000c5b0 tx=0x7fe740001070 comp rx=0 tx=0).stop 2023-10-10T20:42:38.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.821+0000 7fe755b26700 1 -- 172.21.15.89:0/3930944888 shutdown_connections 2023-10-10T20:42:38.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.821+0000 7fe755b26700 1 --2- 172.21.15.89:0/3930944888 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe73c067320 0x7fe73c0697d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:38.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.821+0000 7fe755b26700 1 --2- 172.21.15.89:0/3930944888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe75015df50 0x7fe75015cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:38.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.821+0000 7fe755b26700 1 --2- 172.21.15.89:0/3930944888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe75006bcd0 0x7fe7500b0110 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:38.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.821+0000 7fe755b26700 1 --2- 172.21.15.89:0/3930944888 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe75006c0a0 0x7fe75015c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:38.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.821+0000 7fe755b26700 1 -- 172.21.15.89:0/3930944888 >> 172.21.15.89:0/3930944888 conn(0x7fe7500ce600 msgr2=0x7fe75006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:38.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.821+0000 7fe755b26700 1 -- 172.21.15.89:0/3930944888 shutdown_connections 2023-10-10T20:42:38.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.821+0000 7fe755b26700 1 -- 172.21.15.89:0/3930944888 wait complete. 2023-10-10T20:42:38.826 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 102 2023-10-10T20:42:38.835 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:42:38.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.947+0000 7f3651c1e700 1 Processor -- start 2023-10-10T20:42:38.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.947+0000 7f3651c1e700 1 -- start start 2023-10-10T20:42:38.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.948+0000 7f3651c1e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f364c159c60 0x7f364c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:38.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.948+0000 7f3651c1e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f364c15db10 0x7f364c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:38.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.948+0000 7f3651c1e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364c15a160 con 0x7f364c159890 2023-10-10T20:42:38.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.948+0000 7f3651c1e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f364c0d13a0 con 0x7f364c15db10 2023-10-10T20:42:38.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.948+0000 7f3651c1e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364c15e140 con 0x7f364c159c60 2023-10-10T20:42:38.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.949+0000 7f364bfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f364c15db10 0x7f364c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:38.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.949+0000 7f364affd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f364c159c60 0x7f364c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:38.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.949+0000 7f364b7fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f364c159890 0x7f364c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:34642/0 (socket says 172.21.15.89:34642) 2023-10-10T20:42:38.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.949+0000 7f364bfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f364c15db10 0x7f364c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:47542/0 (socket says 172.21.15.89:47542) 2023-10-10T20:42:38.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.949+0000 7f364affd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f364c159c60 0x7f364c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:46518/0 (socket says 172.21.15.89:46518) 2023-10-10T20:42:38.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.949+0000 7f364b7fe700 1 -- 172.21.15.89:0/4292052660 learned_addr learned my addr 172.21.15.89:0/4292052660 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:38.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.949+0000 7f364bfff700 1 -- 172.21.15.89:0/4292052660 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f364c159c60 msgr2=0x7f364c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:38.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.949+0000 7f364bfff700 1 --2- 172.21.15.89:0/4292052660 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f364c159c60 0x7f364c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:38.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.949+0000 7f364bfff700 1 -- 172.21.15.89:0/4292052660 >> v1:172.21.15.111:6789/0 conn(0x7f364c159890 legacy=0x7f364c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:38.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.949+0000 7f364bfff700 1 -- 172.21.15.89:0/4292052660 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c15a3d0 con 0x7f364c15db10 2023-10-10T20:42:38.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.949+0000 7f364affd700 1 --2- 172.21.15.89:0/4292052660 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f364c159c60 0x7f364c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:42:38.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.949+0000 7f364bfff700 1 --2- 172.21.15.89:0/4292052660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f364c15db10 0x7f364c069560 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f36440098e0 tx=0x7f36440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c9bce253f884dd server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:38.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.949+0000 7f364a7fc700 1 -- 172.21.15.89:0/4292052660 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364400b640 con 0x7f364c15db10 2023-10-10T20:42:38.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.949+0000 7f364a7fc700 1 -- 172.21.15.89:0/4292052660 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364400b7a0 con 0x7f364c15db10 2023-10-10T20:42:38.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.949+0000 7f364a7fc700 1 -- 172.21.15.89:0/4292052660 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364400bb40 con 0x7f364c15db10 2023-10-10T20:42:38.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.950+0000 7f3651c1e700 1 -- 172.21.15.89:0/4292052660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f364c15db10 msgr2=0x7f364c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:38.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.950+0000 7f3651c1e700 1 --2- 172.21.15.89:0/4292052660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f364c15db10 0x7f364c069560 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f36440098e0 tx=0x7f36440091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:38.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.950+0000 7f3651c1e700 1 -- 172.21.15.89:0/4292052660 shutdown_connections 2023-10-10T20:42:38.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.950+0000 7f3651c1e700 1 --2- 172.21.15.89:0/4292052660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f364c15db10 0x7f364c069560 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:38.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.950+0000 7f3651c1e700 1 --2- 172.21.15.89:0/4292052660 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f364c159c60 0x7f364c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:38.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.950+0000 7f3651c1e700 1 -- 172.21.15.89:0/4292052660 >> 172.21.15.89:0/4292052660 conn(0x7f364c068e20 msgr2=0x7f364c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:38.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.950+0000 7f3651c1e700 1 -- 172.21.15.89:0/4292052660 shutdown_connections 2023-10-10T20:42:38.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.950+0000 7f3651c1e700 1 -- 172.21.15.89:0/4292052660 wait complete. 2023-10-10T20:42:38.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.951+0000 7f3651c1e700 1 Processor -- start 2023-10-10T20:42:38.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.951+0000 7f3651c1e700 1 -- start start 2023-10-10T20:42:38.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.952+0000 7f3651c1e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f364c159890 0x7f364c158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:38.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.952+0000 7f3651c1e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f364c159c60 0x7f364c1526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:38.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.952+0000 7f3651c1e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f364c15db10 0x7f364c155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:38.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.952+0000 7f3651c1e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c152c30 con 0x7f364c159c60 2023-10-10T20:42:38.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.952+0000 7f3651c1e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c0b4890 con 0x7f364c15db10 2023-10-10T20:42:38.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.952+0000 7f3651c1e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364c0b5410 con 0x7f364c159890 2023-10-10T20:42:38.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.952+0000 7f364affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f364c159c60 0x7f364c1526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:38.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.952+0000 7f364affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f364c159c60 0x7f364c1526f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38874/0 (socket says 172.21.15.89:38874) 2023-10-10T20:42:38.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.952+0000 7f364affd700 1 -- 172.21.15.89:0/1606558610 learned_addr learned my addr 172.21.15.89:0/1606558610 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:38.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.952+0000 7f364b7fe700 1 --2- 172.21.15.89:0/1606558610 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f364c159890 0x7f364c158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:38.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.952+0000 7f364bfff700 1 --2- 172.21.15.89:0/1606558610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f364c15db10 0x7f364c155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:38.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.952+0000 7f364affd700 1 -- 172.21.15.89:0/1606558610 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f364c159890 msgr2=0x7f364c158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:38.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.952+0000 7f364affd700 1 --2- 172.21.15.89:0/1606558610 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f364c159890 0x7f364c158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:38.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.952+0000 7f364affd700 1 -- 172.21.15.89:0/1606558610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f364c15db10 msgr2=0x7f364c155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:38.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.952+0000 7f364affd700 1 --2- 172.21.15.89:0/1606558610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f364c15db10 0x7f364c155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:38.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.952+0000 7f364affd700 1 -- 172.21.15.89:0/1606558610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3640009420 con 0x7f364c159c60 2023-10-10T20:42:38.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.952+0000 7f364affd700 1 --2- 172.21.15.89:0/1606558610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f364c159c60 0x7f364c1526f0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f364000ecb0 tx=0x7f36400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:38.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.953+0000 7f36337fe700 1 -- 172.21.15.89:0/1606558610 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364000ee60 con 0x7f364c159c60 2023-10-10T20:42:38.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.953+0000 7f36337fe700 1 -- 172.21.15.89:0/1606558610 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364000cbe0 con 0x7f364c159c60 2023-10-10T20:42:38.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.953+0000 7f3651c1e700 1 -- 172.21.15.89:0/1606558610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3644009420 con 0x7f364c159c60 2023-10-10T20:42:38.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.953+0000 7f36337fe700 1 -- 172.21.15.89:0/1606558610 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640013470 con 0x7f364c159c60 2023-10-10T20:42:38.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.953+0000 7f3651c1e700 1 -- 172.21.15.89:0/1606558610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364c0b4ae0 con 0x7f364c159c60 2023-10-10T20:42:38.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.954+0000 7f3651c1e700 1 -- 172.21.15.89:0/1606558610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364c04c3c0 con 0x7f364c159c60 2023-10-10T20:42:38.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.954+0000 7f36337fe700 1 -- 172.21.15.89:0/1606558610 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3640013820 con 0x7f364c159c60 2023-10-10T20:42:38.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.954+0000 7f36337fe700 1 --2- 172.21.15.89:0/1606558610 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3638067320 0x7f36380697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:38.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.955+0000 7f36337fe700 1 -- 172.21.15.89:0/1606558610 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3640084f70 con 0x7f364c159c60 2023-10-10T20:42:38.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.955+0000 7f364b7fe700 1 --2- 172.21.15.89:0/1606558610 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3638067320 0x7f36380697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:38.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.955+0000 7f364b7fe700 1 --2- 172.21.15.89:0/1606558610 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3638067320 0x7f36380697d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f363c000f80 tx=0x7f363c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:38.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:38.959+0000 7f36337fe700 1 -- 172.21.15.89:0/1606558610 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3640051c00 con 0x7f364c159c60 2023-10-10T20:42:39.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.144+0000 7f3651c1e700 1 -- 172.21.15.89:0/1606558610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f364c0b5550 con 0x7f364c159c60 2023-10-10T20:42:39.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.145+0000 7f36337fe700 1 -- 172.21.15.89:0/1606558610 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+15364 (secure 0 0 0) 0x7f3640021090 con 0x7f364c159c60 2023-10-10T20:42:39.148 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:39.148 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":153,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:42:25.092451+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:42:21.831397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-10T20:42:22.440492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:42:39.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.147+0000 7f3651c1e700 1 -- 172.21.15.89:0/1606558610 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3638067320 msgr2=0x7f36380697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:39.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.147+0000 7f3651c1e700 1 --2- 172.21.15.89:0/1606558610 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3638067320 0x7f36380697d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f363c000f80 tx=0x7f363c009040 comp rx=0 tx=0).stop 2023-10-10T20:42:39.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.147+0000 7f3651c1e700 1 -- 172.21.15.89:0/1606558610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f364c159c60 msgr2=0x7f364c1526f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:39.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.147+0000 7f3651c1e700 1 --2- 172.21.15.89:0/1606558610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f364c159c60 0x7f364c1526f0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f364000ecb0 tx=0x7f36400091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.147+0000 7f3651c1e700 1 -- 172.21.15.89:0/1606558610 shutdown_connections 2023-10-10T20:42:39.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.147+0000 7f3651c1e700 1 --2- 172.21.15.89:0/1606558610 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3638067320 0x7f36380697d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.147+0000 7f3651c1e700 1 --2- 172.21.15.89:0/1606558610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f364c15db10 0x7f364c155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.147+0000 7f3651c1e700 1 --2- 172.21.15.89:0/1606558610 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f364c159890 0x7f364c158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.147+0000 7f3651c1e700 1 --2- 172.21.15.89:0/1606558610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f364c159c60 0x7f364c1526f0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.147+0000 7f3651c1e700 1 -- 172.21.15.89:0/1606558610 >> 172.21.15.89:0/1606558610 conn(0x7f364c068e20 msgr2=0x7f364c1530f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:39.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.147+0000 7f3651c1e700 1 -- 172.21.15.89:0/1606558610 shutdown_connections 2023-10-10T20:42:39.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.147+0000 7f3651c1e700 1 -- 172.21.15.89:0/1606558610 wait complete. 2023-10-10T20:42:39.164 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:42:21.831397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '152', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:42:22.440492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:42:39.165 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:42:39.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.283+0000 7ff77f59e700 1 Processor -- start 2023-10-10T20:42:39.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.283+0000 7ff77f59e700 1 -- start start 2023-10-10T20:42:39.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.284+0000 7ff77f59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7800b4e70 0x7ff7800ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:39.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.284+0000 7ff77f59e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff7800b1b40 0x7ff78015d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:39.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.284+0000 7ff77f59e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7800cfb00 con 0x7ff7800b16d0 2023-10-10T20:42:39.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.284+0000 7ff77f59e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7800b5c40 con 0x7ff7800b1b40 2023-10-10T20:42:39.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.284+0000 7ff77f59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7800b5f20 con 0x7ff7800b4e70 2023-10-10T20:42:39.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.284+0000 7ff77ed9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff7800b1b40 0x7ff78015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:39.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.284+0000 7ff77e59c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7800b4e70 0x7ff7800ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:39.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.284+0000 7ff77e59c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7800b4e70 0x7ff7800ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:46524/0 (socket says 172.21.15.89:46524) 2023-10-10T20:42:39.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.284+0000 7ff77ed9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff7800b1b40 0x7ff78015d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:47556/0 (socket says 172.21.15.89:47556) 2023-10-10T20:42:39.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.284+0000 7ff77e59c700 1 -- 172.21.15.89:0/3051918617 learned_addr learned my addr 172.21.15.89:0/3051918617 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:39.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.284+0000 7ff775d9b700 1 --1- 172.21.15.89:0/3051918617 >> v1:172.21.15.111:6789/0 conn(0x7ff7800b16d0 0x7ff78015d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:34650/0 (socket says 172.21.15.89:34650) 2023-10-10T20:42:39.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.285+0000 7ff77ed9d700 1 -- 172.21.15.89:0/3051918617 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7800b4e70 msgr2=0x7ff7800ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:39.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.285+0000 7ff77ed9d700 1 --2- 172.21.15.89:0/3051918617 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7800b4e70 0x7ff7800ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.285+0000 7ff77ed9d700 1 -- 172.21.15.89:0/3051918617 >> v1:172.21.15.111:6789/0 conn(0x7ff7800b16d0 legacy=0x7ff78015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:39.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.285+0000 7ff77ed9d700 1 -- 172.21.15.89:0/3051918617 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7800b6200 con 0x7ff7800b1b40 2023-10-10T20:42:39.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.285+0000 7ff77e59c700 1 --2- 172.21.15.89:0/3051918617 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7800b4e70 0x7ff7800ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:42:39.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.285+0000 7ff77ed9d700 1 --2- 172.21.15.89:0/3051918617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7800b1b40 0x7ff78015d950 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff778009e60 tx=0x7ff778009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=802eefd6924e6688 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:39.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.285+0000 7ff77dd9b700 1 -- 172.21.15.89:0/3051918617 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77800e070 con 0x7ff7800b1b40 2023-10-10T20:42:39.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.285+0000 7ff77dd9b700 1 -- 172.21.15.89:0/3051918617 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77800bb60 con 0x7ff7800b1b40 2023-10-10T20:42:39.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.285+0000 7ff77dd9b700 1 -- 172.21.15.89:0/3051918617 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff778012430 con 0x7ff7800b1b40 2023-10-10T20:42:39.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.286+0000 7ff77f59e700 1 -- 172.21.15.89:0/3051918617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7800b1b40 msgr2=0x7ff78015d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:39.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.286+0000 7ff77f59e700 1 --2- 172.21.15.89:0/3051918617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7800b1b40 0x7ff78015d950 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff778009e60 tx=0x7ff778009780 comp rx=0 tx=0).stop 2023-10-10T20:42:39.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.286+0000 7ff77f59e700 1 -- 172.21.15.89:0/3051918617 shutdown_connections 2023-10-10T20:42:39.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.286+0000 7ff77f59e700 1 --2- 172.21.15.89:0/3051918617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7800b1b40 0x7ff78015d950 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.286+0000 7ff77f59e700 1 --2- 172.21.15.89:0/3051918617 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7800b4e70 0x7ff7800ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.286+0000 7ff77f59e700 1 -- 172.21.15.89:0/3051918617 >> 172.21.15.89:0/3051918617 conn(0x7ff7800d10c0 msgr2=0x7ff7800affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:39.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.286+0000 7ff77f59e700 1 -- 172.21.15.89:0/3051918617 shutdown_connections 2023-10-10T20:42:39.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.286+0000 7ff77f59e700 1 -- 172.21.15.89:0/3051918617 wait complete. 2023-10-10T20:42:39.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.287+0000 7ff77f59e700 1 Processor -- start 2023-10-10T20:42:39.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.287+0000 7ff77f59e700 1 -- start start 2023-10-10T20:42:39.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.288+0000 7ff77f59e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7800b16d0 0x7ff7801459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:39.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.288+0000 7ff77f59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7800b1b40 0x7ff7800c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:39.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.288+0000 7ff77f59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7800b4e70 0x7ff7800c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:39.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.288+0000 7ff77f59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7801585c0 con 0x7ff7800b1b40 2023-10-10T20:42:39.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.288+0000 7ff77f59e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7800c8e00 con 0x7ff7800b16d0 2023-10-10T20:42:39.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.288+0000 7ff77f59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7800c90e0 con 0x7ff7800b4e70 2023-10-10T20:42:39.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.288+0000 7ff775d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7800b1b40 0x7ff7800c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:39.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.288+0000 7ff775d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7800b1b40 0x7ff7800c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38890/0 (socket says 172.21.15.89:38890) 2023-10-10T20:42:39.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.288+0000 7ff775d9b700 1 -- 172.21.15.89:0/1384030119 learned_addr learned my addr 172.21.15.89:0/1384030119 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:39.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.288+0000 7ff77e59c700 1 --2- 172.21.15.89:0/1384030119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7800b16d0 0x7ff7801459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:39.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.288+0000 7ff77ed9d700 1 --2- 172.21.15.89:0/1384030119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7800b4e70 0x7ff7800c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:39.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.288+0000 7ff775d9b700 1 -- 172.21.15.89:0/1384030119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7800b4e70 msgr2=0x7ff7800c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:39.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.288+0000 7ff775d9b700 1 --2- 172.21.15.89:0/1384030119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7800b4e70 0x7ff7800c84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.288+0000 7ff775d9b700 1 -- 172.21.15.89:0/1384030119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7800b16d0 msgr2=0x7ff7801459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:39.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.288+0000 7ff775d9b700 1 --2- 172.21.15.89:0/1384030119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7800b16d0 0x7ff7801459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.288+0000 7ff775d9b700 1 -- 172.21.15.89:0/1384030119 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7700094f0 con 0x7ff7800b1b40 2023-10-10T20:42:39.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.288+0000 7ff775d9b700 1 --2- 172.21.15.89:0/1384030119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7800b1b40 0x7ff7800c9ea0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7ff76800c560 tx=0x7ff768007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:39.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.288+0000 7ff776ffd700 1 -- 172.21.15.89:0/1384030119 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff768005a70 con 0x7ff7800b1b40 2023-10-10T20:42:39.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.289+0000 7ff776ffd700 1 -- 172.21.15.89:0/1384030119 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff768005bd0 con 0x7ff7800b1b40 2023-10-10T20:42:39.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.289+0000 7ff77f59e700 1 -- 172.21.15.89:0/1384030119 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff778009a20 con 0x7ff7800b1b40 2023-10-10T20:42:39.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.289+0000 7ff776ffd700 1 -- 172.21.15.89:0/1384030119 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7680070d0 con 0x7ff7800b1b40 2023-10-10T20:42:39.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.289+0000 7ff77f59e700 1 -- 172.21.15.89:0/1384030119 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7800c96a0 con 0x7ff7800b1b40 2023-10-10T20:42:39.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.289+0000 7ff77f59e700 1 -- 172.21.15.89:0/1384030119 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff780088990 con 0x7ff7800b1b40 2023-10-10T20:42:39.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.290+0000 7ff776ffd700 1 -- 172.21.15.89:0/1384030119 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff768007480 con 0x7ff7800b1b40 2023-10-10T20:42:39.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.290+0000 7ff776ffd700 1 --2- 172.21.15.89:0/1384030119 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff760067320 0x7ff7600697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:39.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.290+0000 7ff776ffd700 1 -- 172.21.15.89:0/1384030119 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff768083fe0 con 0x7ff7800b1b40 2023-10-10T20:42:39.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.290+0000 7ff77e59c700 1 --2- 172.21.15.89:0/1384030119 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff760067320 0x7ff7600697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:39.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.291+0000 7ff77e59c700 1 --2- 172.21.15.89:0/1384030119 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff760067320 0x7ff7600697d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff770006d20 tx=0x7ff770009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:39.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.294+0000 7ff776ffd700 1 -- 172.21.15.89:0/1384030119 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff768050c70 con 0x7ff7800b1b40 2023-10-10T20:42:39.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.476+0000 7ff77f59e700 1 -- 172.21.15.89:0/1384030119 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff78004bb80 con 0x7ff7800b1b40 2023-10-10T20:42:39.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.477+0000 7ff776ffd700 1 -- 172.21.15.89:0/1384030119 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v153) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff768017020 con 0x7ff7800b1b40 2023-10-10T20:42:39.479 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:42:39.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.479+0000 7ff77f59e700 1 -- 172.21.15.89:0/1384030119 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff760067320 msgr2=0x7ff7600697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:39.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.479+0000 7ff77f59e700 1 --2- 172.21.15.89:0/1384030119 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff760067320 0x7ff7600697d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff770006d20 tx=0x7ff770009250 comp rx=0 tx=0).stop 2023-10-10T20:42:39.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.479+0000 7ff77f59e700 1 -- 172.21.15.89:0/1384030119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7800b1b40 msgr2=0x7ff7800c9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:39.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.479+0000 7ff77f59e700 1 --2- 172.21.15.89:0/1384030119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7800b1b40 0x7ff7800c9ea0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7ff76800c560 tx=0x7ff768007f70 comp rx=0 tx=0).stop 2023-10-10T20:42:39.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.479+0000 7ff77f59e700 1 -- 172.21.15.89:0/1384030119 shutdown_connections 2023-10-10T20:42:39.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.479+0000 7ff77f59e700 1 --2- 172.21.15.89:0/1384030119 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff7800b4e70 0x7ff7800c84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.479+0000 7ff77f59e700 1 --2- 172.21.15.89:0/1384030119 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff760067320 0x7ff7600697d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.479+0000 7ff77f59e700 1 --2- 172.21.15.89:0/1384030119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7800b16d0 0x7ff7801459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.479+0000 7ff77f59e700 1 --2- 172.21.15.89:0/1384030119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7800b1b40 0x7ff7800c9ea0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.479+0000 7ff77f59e700 1 -- 172.21.15.89:0/1384030119 >> 172.21.15.89:0/1384030119 conn(0x7ff7800d10c0 msgr2=0x7ff780159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:39.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.479+0000 7ff77f59e700 1 -- 172.21.15.89:0/1384030119 shutdown_connections 2023-10-10T20:42:39.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.479+0000 7ff77f59e700 1 -- 172.21.15.89:0/1384030119 wait complete. 2023-10-10T20:42:39.495 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:42:39.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.615+0000 7fdaf046d700 1 Processor -- start 2023-10-10T20:42:39.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.615+0000 7fdaf046d700 1 -- start start 2023-10-10T20:42:39.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.615+0000 7fdaf046d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdae8159890 0x7fdae815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:39.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.616+0000 7fdaf046d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdae8069560 0x7fdae80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.616+0000 7fdaf046d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae8069ee0 con 0x7fdae8159c60 2023-10-10T20:42:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.616+0000 7fdaf046d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdae815a160 con 0x7fdae8159890 2023-10-10T20:42:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.616+0000 7fdaf046d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdae80d13a0 con 0x7fdae8069560 2023-10-10T20:42:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.616+0000 7fdaee209700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdae8159890 0x7fdae815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.616+0000 7fdaee209700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdae8159890 0x7fdae815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38892/0 (socket says 172.21.15.89:38892) 2023-10-10T20:42:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.616+0000 7fdaee209700 1 -- 172.21.15.89:0/1435129162 learned_addr learned my addr 172.21.15.89:0/1435129162 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.616+0000 7fdaee209700 1 -- 172.21.15.89:0/1435129162 >> v2:172.21.15.111:3300/0 conn(0x7fdae8069560 msgr2=0x7fdae80699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.616+0000 7fdaee209700 1 --2- 172.21.15.89:0/1435129162 >> v2:172.21.15.111:3300/0 conn(0x7fdae8069560 0x7fdae80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.616+0000 7fdaee209700 1 -- 172.21.15.89:0/1435129162 >> v1:172.21.15.111:6789/0 conn(0x7fdae8159c60 legacy=0x7fdae815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.616+0000 7fdaee209700 1 -- 172.21.15.89:0/1435129162 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae815e140 con 0x7fdae8159890 2023-10-10T20:42:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.616+0000 7fdaee209700 1 --2- 172.21.15.89:0/1435129162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdae8159890 0x7fdae815abd0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fdae806a890 tx=0x7fdae4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f6d0cb9e312376 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.616+0000 7fdaed207700 1 -- 172.21.15.89:0/1435129162 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae400e070 con 0x7fdae8159890 2023-10-10T20:42:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.616+0000 7fdaed207700 1 -- 172.21.15.89:0/1435129162 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae400ba10 con 0x7fdae8159890 2023-10-10T20:42:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.616+0000 7fdaed207700 1 -- 172.21.15.89:0/1435129162 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae40092e0 con 0x7fdae8159890 2023-10-10T20:42:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.617+0000 7fdaf046d700 1 -- 172.21.15.89:0/1435129162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdae8159890 msgr2=0x7fdae815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.617+0000 7fdaf046d700 1 --2- 172.21.15.89:0/1435129162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdae8159890 0x7fdae815abd0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fdae806a890 tx=0x7fdae4009840 comp rx=0 tx=0).stop 2023-10-10T20:42:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.617+0000 7fdaf046d700 1 -- 172.21.15.89:0/1435129162 shutdown_connections 2023-10-10T20:42:39.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.617+0000 7fdaf046d700 1 --2- 172.21.15.89:0/1435129162 >> v2:172.21.15.111:3300/0 conn(0x7fdae8069560 0x7fdae80699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.617+0000 7fdaf046d700 1 --2- 172.21.15.89:0/1435129162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdae8159890 0x7fdae815abd0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.617+0000 7fdaf046d700 1 -- 172.21.15.89:0/1435129162 >> 172.21.15.89:0/1435129162 conn(0x7fdae8068e20 msgr2=0x7fdae80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:39.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.617+0000 7fdaf046d700 1 -- 172.21.15.89:0/1435129162 shutdown_connections 2023-10-10T20:42:39.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.617+0000 7fdaf046d700 1 -- 172.21.15.89:0/1435129162 wait complete. 2023-10-10T20:42:39.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.618+0000 7fdaf046d700 1 Processor -- start 2023-10-10T20:42:39.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.618+0000 7fdaf046d700 1 -- start start 2023-10-10T20:42:39.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaf046d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdae8069560 0x7fdae8147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:39.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaf046d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdae8159890 0x7fdae80c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:39.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaf046d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdae8159c60 0x7fdae80d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:39.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaf046d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae81477c0 con 0x7fdae8159c60 2023-10-10T20:42:39.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaf046d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae80b6400 con 0x7fdae8069560 2023-10-10T20:42:39.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaf046d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae80c7a40 con 0x7fdae8159890 2023-10-10T20:42:39.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaeea0a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdae8159c60 0x7fdae80d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:39.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaeea0a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdae8159c60 0x7fdae80d05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38896/0 (socket says 172.21.15.89:38896) 2023-10-10T20:42:39.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaeea0a700 1 -- 172.21.15.89:0/3388046455 learned_addr learned my addr 172.21.15.89:0/3388046455 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:39.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaee209700 1 --2- 172.21.15.89:0/3388046455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdae8069560 0x7fdae8147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:39.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaeda08700 1 --2- 172.21.15.89:0/3388046455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdae8159890 0x7fdae80c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:39.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaeea0a700 1 -- 172.21.15.89:0/3388046455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdae8159890 msgr2=0x7fdae80c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:39.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaeea0a700 1 --2- 172.21.15.89:0/3388046455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdae8159890 0x7fdae80c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaeea0a700 1 -- 172.21.15.89:0/3388046455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdae8069560 msgr2=0x7fdae8147280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:39.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaeea0a700 1 --2- 172.21.15.89:0/3388046455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdae8069560 0x7fdae8147280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaeea0a700 1 -- 172.21.15.89:0/3388046455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae4009ae0 con 0x7fdae8159c60 2023-10-10T20:42:39.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaee209700 1 --2- 172.21.15.89:0/3388046455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdae8069560 0x7fdae8147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:39.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaeea0a700 1 --2- 172.21.15.89:0/3388046455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdae8159c60 0x7fdae80d05f0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fdae000b810 tx=0x7fdae00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:39.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdade7fc700 1 -- 172.21.15.89:0/3388046455 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae000f570 con 0x7fdae8159c60 2023-10-10T20:42:39.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdade7fc700 1 -- 172.21.15.89:0/3388046455 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae000f6f0 con 0x7fdae8159c60 2023-10-10T20:42:39.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaf046d700 1 -- 172.21.15.89:0/3388046455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae80c7d20 con 0x7fdae8159c60 2023-10-10T20:42:39.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdade7fc700 1 -- 172.21.15.89:0/3388046455 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae000fab0 con 0x7fdae8159c60 2023-10-10T20:42:39.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.619+0000 7fdaf046d700 1 -- 172.21.15.89:0/3388046455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae80c82e0 con 0x7fdae8159c60 2023-10-10T20:42:39.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.620+0000 7fdaf046d700 1 -- 172.21.15.89:0/3388046455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae804c3c0 con 0x7fdae8159c60 2023-10-10T20:42:39.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.621+0000 7fdade7fc700 1 -- 172.21.15.89:0/3388046455 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdae001a460 con 0x7fdae8159c60 2023-10-10T20:42:39.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.621+0000 7fdade7fc700 1 --2- 172.21.15.89:0/3388046455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdacc067320 0x7fdacc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:39.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.621+0000 7fdade7fc700 1 -- 172.21.15.89:0/3388046455 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdae00849a0 con 0x7fdae8159c60 2023-10-10T20:42:39.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.621+0000 7fdaee209700 1 --2- 172.21.15.89:0/3388046455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdacc067320 0x7fdacc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:39.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.621+0000 7fdaee209700 1 --2- 172.21.15.89:0/3388046455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdacc067320 0x7fdacc0697d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009880 tx=0x7fdae4017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:39.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.625+0000 7fdade7fc700 1 -- 172.21.15.89:0/3388046455 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdae00516b0 con 0x7fdae8159c60 2023-10-10T20:42:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.805+0000 7fdaf046d700 1 -- 172.21.15.89:0/3388046455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdae806a5b0 con 0x7fdae8159c60 2023-10-10T20:42:39.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.806+0000 7fdade7fc700 1 -- 172.21.15.89:0/3388046455 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v153) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdae0054cd0 con 0x7fdae8159c60 2023-10-10T20:42:39.808 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:42:39.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.808+0000 7fdaf046d700 1 -- 172.21.15.89:0/3388046455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdacc067320 msgr2=0x7fdacc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:39.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.808+0000 7fdaf046d700 1 --2- 172.21.15.89:0/3388046455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdacc067320 0x7fdacc0697d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009880 tx=0x7fdae4017000 comp rx=0 tx=0).stop 2023-10-10T20:42:39.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.808+0000 7fdaf046d700 1 -- 172.21.15.89:0/3388046455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdae8159c60 msgr2=0x7fdae80d05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:39.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.808+0000 7fdaf046d700 1 --2- 172.21.15.89:0/3388046455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdae8159c60 0x7fdae80d05f0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fdae000b810 tx=0x7fdae00091d0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.808+0000 7fdaf046d700 1 -- 172.21.15.89:0/3388046455 shutdown_connections 2023-10-10T20:42:39.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.808+0000 7fdaf046d700 1 --2- 172.21.15.89:0/3388046455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdae8159c60 0x7fdae80d05f0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.808+0000 7fdaf046d700 1 --2- 172.21.15.89:0/3388046455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdacc067320 0x7fdacc0697d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.808+0000 7fdaf046d700 1 --2- 172.21.15.89:0/3388046455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdae8069560 0x7fdae8147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.808+0000 7fdaf046d700 1 --2- 172.21.15.89:0/3388046455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdae8159890 0x7fdae80c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.808+0000 7fdaf046d700 1 -- 172.21.15.89:0/3388046455 >> 172.21.15.89:0/3388046455 conn(0x7fdae8068e20 msgr2=0x7fdae8153430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:39.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.808+0000 7fdaf046d700 1 -- 172.21.15.89:0/3388046455 shutdown_connections 2023-10-10T20:42:39.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.808+0000 7fdaf046d700 1 -- 172.21.15.89:0/3388046455 wait complete. 2023-10-10T20:42:39.822 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:42:39.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.939+0000 7f0e4deff700 1 Processor -- start 2023-10-10T20:42:39.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.939+0000 7f0e4deff700 1 -- start start 2023-10-10T20:42:39.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.940+0000 7f0e4deff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0e480b4e70 0x7f0e480ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:39.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.940+0000 7f0e4deff700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e480cfb00 con 0x7f0e480b16d0 2023-10-10T20:42:39.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.940+0000 7f0e4deff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4815e060 con 0x7f0e480b1b40 2023-10-10T20:42:39.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.940+0000 7f0e4deff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e480b5c40 con 0x7f0e480b4e70 2023-10-10T20:42:39.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.940+0000 7f0e3ffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0e480b16d0 0x7f0e4815d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59160/0 (socket says 172.21.15.89:59160) 2023-10-10T20:42:39.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.940+0000 7f0e3ffff700 1 -- 172.21.15.89:0/520311026 learned_addr learned my addr 172.21.15.89:0/520311026 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:39.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.940+0000 7f0e4cefd700 1 --2- 172.21.15.89:0/520311026 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0e480b4e70 0x7f0e480ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:39.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.940+0000 7f0e3f7fe700 1 -- 172.21.15.89:0/520311026 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492317962 0 0) 0x7f0e480cfb00 con 0x7f0e480b16d0 2023-10-10T20:42:39.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.940+0000 7f0e3f7fe700 1 -- 172.21.15.89:0/520311026 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e30003570 con 0x7f0e480b16d0 2023-10-10T20:42:39.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.941+0000 7f0e4cefd700 1 -- 172.21.15.89:0/520311026 >> v1:172.21.15.111:6789/0 conn(0x7f0e480b1b40 legacy=0x7f0e4815d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:39.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.941+0000 7f0e3f7fe700 1 -- 172.21.15.89:0/520311026 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1695168916 0 0) 0x7f0e30003570 con 0x7f0e480b16d0 2023-10-10T20:42:39.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.941+0000 7f0e4cefd700 1 -- 172.21.15.89:0/520311026 >> v1:172.21.15.89:6789/0 conn(0x7f0e480b16d0 legacy=0x7f0e4815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:39.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.941+0000 7f0e4cefd700 1 -- 172.21.15.89:0/520311026 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e480b5f20 con 0x7f0e480b4e70 2023-10-10T20:42:39.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.941+0000 7f0e4cefd700 1 --2- 172.21.15.89:0/520311026 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0e480b4e70 0x7f0e480ccff0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009e90 tx=0x7f0e44009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9a9fd0927a1cce2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:39.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.941+0000 7f0e3f7fe700 1 -- 172.21.15.89:0/520311026 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4400e070 con 0x7f0e480b4e70 2023-10-10T20:42:39.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.941+0000 7f0e3f7fe700 1 -- 172.21.15.89:0/520311026 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e4400b920 con 0x7f0e480b4e70 2023-10-10T20:42:39.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.941+0000 7f0e3f7fe700 1 -- 172.21.15.89:0/520311026 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38001470 con 0x7f0e480b4e70 2023-10-10T20:42:39.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.941+0000 7f0e4deff700 1 -- 172.21.15.89:0/520311026 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0e480b4e70 msgr2=0x7f0e480ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:39.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.941+0000 7f0e4deff700 1 --2- 172.21.15.89:0/520311026 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0e480b4e70 0x7f0e480ccff0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009e90 tx=0x7f0e44009780 comp rx=0 tx=0).stop 2023-10-10T20:42:39.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.942+0000 7f0e4deff700 1 -- 172.21.15.89:0/520311026 shutdown_connections 2023-10-10T20:42:39.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.942+0000 7f0e4deff700 1 --2- 172.21.15.89:0/520311026 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0e480b4e70 0x7f0e480ccff0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.942+0000 7f0e4deff700 1 -- 172.21.15.89:0/520311026 >> 172.21.15.89:0/520311026 conn(0x7f0e480d10c0 msgr2=0x7f0e480affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:39.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.942+0000 7f0e4deff700 1 -- 172.21.15.89:0/520311026 shutdown_connections 2023-10-10T20:42:39.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.942+0000 7f0e4deff700 1 -- 172.21.15.89:0/520311026 wait complete. 2023-10-10T20:42:39.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.943+0000 7f0e4deff700 1 Processor -- start 2023-10-10T20:42:39.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.943+0000 7f0e4deff700 1 -- start start 2023-10-10T20:42:39.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.943+0000 7f0e4deff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0e480b16d0 0x7f0e481578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:39.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e4deff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e480b1b40 0x7f0e4816a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:39.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e4deff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e480b4e70 0x7f0e480cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:39.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e4deff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e480ca680 con 0x7f0e480b1b40 2023-10-10T20:42:39.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e4deff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e480c9a70 con 0x7f0e480b4e70 2023-10-10T20:42:39.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e4deff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e480c9d50 con 0x7f0e480b16d0 2023-10-10T20:42:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e3ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e480b1b40 0x7f0e4816a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e3ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e480b1b40 0x7f0e4816a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38902/0 (socket says 172.21.15.89:38902) 2023-10-10T20:42:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e3ffff700 1 -- 172.21.15.89:0/1999624393 learned_addr learned my addr 172.21.15.89:0/1999624393 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e4cefd700 1 --2- 172.21.15.89:0/1999624393 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0e480b16d0 0x7f0e481578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:39.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e4d6fe700 1 --2- 172.21.15.89:0/1999624393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e480b4e70 0x7f0e480cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:39.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e3ffff700 1 -- 172.21.15.89:0/1999624393 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0e480b16d0 msgr2=0x7f0e481578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:39.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e3ffff700 1 --2- 172.21.15.89:0/1999624393 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0e480b16d0 0x7f0e481578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e3ffff700 1 -- 172.21.15.89:0/1999624393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e480b4e70 msgr2=0x7f0e480cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:39.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e3ffff700 1 --2- 172.21.15.89:0/1999624393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e480b4e70 0x7f0e480cb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:39.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e3ffff700 1 -- 172.21.15.89:0/1999624393 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e44009a20 con 0x7f0e480b1b40 2023-10-10T20:42:39.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e3ffff700 1 --2- 172.21.15.89:0/1999624393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e480b1b40 0x7f0e4816a9f0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800cb90 tx=0x7f0e38001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:39.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e3cff9700 1 -- 172.21.15.89:0/1999624393 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38001d40 con 0x7f0e480b1b40 2023-10-10T20:42:39.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e3cff9700 1 -- 172.21.15.89:0/1999624393 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e38007360 con 0x7f0e480b1b40 2023-10-10T20:42:39.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e4deff700 1 -- 172.21.15.89:0/1999624393 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e480ca030 con 0x7f0e480b1b40 2023-10-10T20:42:39.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e3cff9700 1 -- 172.21.15.89:0/1999624393 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38004830 con 0x7f0e480b1b40 2023-10-10T20:42:39.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.944+0000 7f0e4deff700 1 -- 172.21.15.89:0/1999624393 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e480c94b0 con 0x7f0e480b1b40 2023-10-10T20:42:39.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.945+0000 7f0e4deff700 1 -- 172.21.15.89:0/1999624393 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e480ca830 con 0x7f0e480b1b40 2023-10-10T20:42:39.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.946+0000 7f0e3cff9700 1 -- 172.21.15.89:0/1999624393 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0e38004be0 con 0x7f0e480b1b40 2023-10-10T20:42:39.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.946+0000 7f0e3cff9700 1 --2- 172.21.15.89:0/1999624393 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0e34067320 0x7f0e340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:39.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.946+0000 7f0e3cff9700 1 -- 172.21.15.89:0/1999624393 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0e38084100 con 0x7f0e480b1b40 2023-10-10T20:42:39.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.946+0000 7f0e4cefd700 1 --2- 172.21.15.89:0/1999624393 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0e34067320 0x7f0e340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:39.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.946+0000 7f0e4cefd700 1 --2- 172.21.15.89:0/1999624393 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0e34067320 0x7f0e340697d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009de0 tx=0x7f0e44017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:39.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:39.950+0000 7f0e3cff9700 1 -- 172.21.15.89:0/1999624393 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e38050d90 con 0x7f0e480b1b40 2023-10-10T20:42:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.134+0000 7f0e4deff700 1 -- 172.21.15.89:0/1999624393 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0e4804bb80 con 0x7f0e480b1b40 2023-10-10T20:42:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.134+0000 7f0e3cff9700 1 -- 172.21.15.89:0/1999624393 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v153) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0e380543b0 con 0x7f0e480b1b40 2023-10-10T20:42:40.136 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-10T20:42:40.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.136+0000 7f0e4deff700 1 -- 172.21.15.89:0/1999624393 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0e34067320 msgr2=0x7f0e340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.136+0000 7f0e4deff700 1 --2- 172.21.15.89:0/1999624393 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0e34067320 0x7f0e340697d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009de0 tx=0x7f0e44017000 comp rx=0 tx=0).stop 2023-10-10T20:42:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.136+0000 7f0e4deff700 1 -- 172.21.15.89:0/1999624393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e480b1b40 msgr2=0x7f0e4816a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.136+0000 7f0e4deff700 1 --2- 172.21.15.89:0/1999624393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e480b1b40 0x7f0e4816a9f0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800cb90 tx=0x7f0e38001ab0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.136+0000 7f0e4deff700 1 -- 172.21.15.89:0/1999624393 shutdown_connections 2023-10-10T20:42:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.136+0000 7f0e4deff700 1 --2- 172.21.15.89:0/1999624393 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0e34067320 0x7f0e340697d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.136+0000 7f0e4deff700 1 --2- 172.21.15.89:0/1999624393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e480b4e70 0x7f0e480cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.136+0000 7f0e4deff700 1 --2- 172.21.15.89:0/1999624393 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0e480b16d0 0x7f0e481578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.136+0000 7f0e4deff700 1 --2- 172.21.15.89:0/1999624393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e480b1b40 0x7f0e4816a9f0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.136+0000 7f0e4deff700 1 -- 172.21.15.89:0/1999624393 >> 172.21.15.89:0/1999624393 conn(0x7f0e480d10c0 msgr2=0x7f0e481591d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:40.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.136+0000 7f0e4deff700 1 -- 172.21.15.89:0/1999624393 shutdown_connections 2023-10-10T20:42:40.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.136+0000 7f0e4deff700 1 -- 172.21.15.89:0/1999624393 wait complete. 2023-10-10T20:42:40.150 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:42:40.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.263+0000 7f7cf0b4f700 1 Processor -- start 2023-10-10T20:42:40.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.263+0000 7f7cf0b4f700 1 -- start start 2023-10-10T20:42:40.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.264+0000 7f7cf0b4f700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7cec159c60 0x7f7cec15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:40.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.264+0000 7f7cf0b4f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7cec15db10 0x7f7cec069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:40.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.264+0000 7f7cf0b4f700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cec15a160 con 0x7f7cec159890 2023-10-10T20:42:40.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.264+0000 7f7cf0b4f700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cec15df50 con 0x7f7cec159c60 2023-10-10T20:42:40.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.264+0000 7f7cf0b4f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec0d13a0 con 0x7f7cec15db10 2023-10-10T20:42:40.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.264+0000 7f7ce1d9b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7cec159c60 0x7f7cec15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:40.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.264+0000 7f7cead9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7cec15db10 0x7f7cec069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:40.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.264+0000 7f7ce1d9b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7cec159c60 0x7f7cec15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:47586/0 (socket says 172.21.15.89:47586) 2023-10-10T20:42:40.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.264+0000 7f7cead9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7cec15db10 0x7f7cec069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:46580/0 (socket says 172.21.15.89:46580) 2023-10-10T20:42:40.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.264+0000 7f7ce1d9b700 1 -- 172.21.15.89:0/3149698585 learned_addr learned my addr 172.21.15.89:0/3149698585 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:40.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.264+0000 7f7cea59c700 1 --1- 172.21.15.89:0/3149698585 >> v1:172.21.15.111:6789/0 conn(0x7f7cec159890 0x7f7cec15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:34694/0 (socket says 172.21.15.89:34694) 2023-10-10T20:42:40.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.265+0000 7f7cead9d700 1 -- 172.21.15.89:0/3149698585 >> v2:172.21.15.111:3300/0 conn(0x7f7cec159c60 msgr2=0x7f7cec15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:40.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.265+0000 7f7cead9d700 1 --2- 172.21.15.89:0/3149698585 >> v2:172.21.15.111:3300/0 conn(0x7f7cec159c60 0x7f7cec15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.265+0000 7f7cead9d700 1 -- 172.21.15.89:0/3149698585 >> v1:172.21.15.111:6789/0 conn(0x7f7cec159890 legacy=0x7f7cec15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:40.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.265+0000 7f7cead9d700 1 -- 172.21.15.89:0/3149698585 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cec15e140 con 0x7f7cec15db10 2023-10-10T20:42:40.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.265+0000 7f7ce9d9b700 1 -- 172.21.15.89:0/3149698585 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420237700 0 0) 0x7f7cec15a160 con 0x7f7cec159890 2023-10-10T20:42:40.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.265+0000 7f7ce1d9b700 1 --2- 172.21.15.89:0/3149698585 >> v2:172.21.15.111:3300/0 conn(0x7f7cec159c60 0x7f7cec15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:42:40.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.265+0000 7f7cead9d700 1 --2- 172.21.15.89:0/3149698585 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7cec15db10 0x7f7cec069560 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009e60 tx=0x7f7ce4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e821057dff5ed0aa server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:40.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.265+0000 7f7ce9d9b700 1 -- 172.21.15.89:0/3149698585 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce400e070 con 0x7f7cec15db10 2023-10-10T20:42:40.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.265+0000 7f7ce9d9b700 1 -- 172.21.15.89:0/3149698585 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce400bb60 con 0x7f7cec15db10 2023-10-10T20:42:40.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.265+0000 7f7ce9d9b700 1 -- 172.21.15.89:0/3149698585 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4012430 con 0x7f7cec15db10 2023-10-10T20:42:40.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.266+0000 7f7cf0b4f700 1 -- 172.21.15.89:0/3149698585 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7cec15db10 msgr2=0x7f7cec069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:40.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.266+0000 7f7cf0b4f700 1 --2- 172.21.15.89:0/3149698585 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7cec15db10 0x7f7cec069560 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009e60 tx=0x7f7ce4009780 comp rx=0 tx=0).stop 2023-10-10T20:42:40.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.266+0000 7f7cf0b4f700 1 -- 172.21.15.89:0/3149698585 shutdown_connections 2023-10-10T20:42:40.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.266+0000 7f7cf0b4f700 1 --2- 172.21.15.89:0/3149698585 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7cec15db10 0x7f7cec069560 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.266+0000 7f7cf0b4f700 1 --2- 172.21.15.89:0/3149698585 >> v2:172.21.15.111:3300/0 conn(0x7f7cec159c60 0x7f7cec15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.266+0000 7f7cf0b4f700 1 -- 172.21.15.89:0/3149698585 >> 172.21.15.89:0/3149698585 conn(0x7f7cec068e20 msgr2=0x7f7cec0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:40.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.266+0000 7f7cf0b4f700 1 -- 172.21.15.89:0/3149698585 shutdown_connections 2023-10-10T20:42:40.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.266+0000 7f7cf0b4f700 1 -- 172.21.15.89:0/3149698585 wait complete. 2023-10-10T20:42:40.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.267+0000 7f7cf0b4f700 1 Processor -- start 2023-10-10T20:42:40.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.267+0000 7f7cf0b4f700 1 -- start start 2023-10-10T20:42:40.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7cf0b4f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cec159890 0x7f7cec151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:40.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7cf0b4f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec159c60 0x7f7cec0cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:40.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7cf0b4f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7cec15db10 0x7f7cec15f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:40.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7cf0b4f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec0c7f40 con 0x7f7cec159890 2023-10-10T20:42:40.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7cf0b4f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec0ca6f0 con 0x7f7cec159c60 2023-10-10T20:42:40.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7cf0b4f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec152400 con 0x7f7cec15db10 2023-10-10T20:42:40.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7cea59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cec159890 0x7f7cec151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:40.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7cea59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cec159890 0x7f7cec151ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38908/0 (socket says 172.21.15.89:38908) 2023-10-10T20:42:40.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7cea59c700 1 -- 172.21.15.89:0/3327162049 learned_addr learned my addr 172.21.15.89:0/3327162049 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:40.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7cead9d700 1 --2- 172.21.15.89:0/3327162049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7cec15db10 0x7f7cec15f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:40.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7ce1d9b700 1 --2- 172.21.15.89:0/3327162049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec159c60 0x7f7cec0cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:40.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7cea59c700 1 -- 172.21.15.89:0/3327162049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7cec15db10 msgr2=0x7f7cec15f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:40.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7cea59c700 1 --2- 172.21.15.89:0/3327162049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7cec15db10 0x7f7cec15f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7cea59c700 1 -- 172.21.15.89:0/3327162049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec159c60 msgr2=0x7f7cec0cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:40.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7cea59c700 1 --2- 172.21.15.89:0/3327162049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec159c60 0x7f7cec0cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7cea59c700 1 -- 172.21.15.89:0/3327162049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd40094f0 con 0x7f7cec159890 2023-10-10T20:42:40.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7cead9d700 1 --2- 172.21.15.89:0/3327162049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7cec15db10 0x7f7cec15f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:42:40.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7ce1d9b700 1 --2- 172.21.15.89:0/3327162049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec159c60 0x7f7cec0cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:42:40.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7cea59c700 1 --2- 172.21.15.89:0/3327162049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cec159890 0x7f7cec151ec0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00c7e0 tx=0x7f7cdc007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:40.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7ce2ffd700 1 -- 172.21.15.89:0/3327162049 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc005a70 con 0x7f7cec159890 2023-10-10T20:42:40.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7ce2ffd700 1 -- 172.21.15.89:0/3327162049 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cdc005bd0 con 0x7f7cec159890 2023-10-10T20:42:40.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7cf0b4f700 1 -- 172.21.15.89:0/3327162049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce4009a20 con 0x7f7cec159890 2023-10-10T20:42:40.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7ce2ffd700 1 -- 172.21.15.89:0/3327162049 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc0070d0 con 0x7f7cec159890 2023-10-10T20:42:40.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.268+0000 7f7cf0b4f700 1 -- 172.21.15.89:0/3327162049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cec0c7c60 con 0x7f7cec159890 2023-10-10T20:42:40.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.269+0000 7f7cf0b4f700 1 -- 172.21.15.89:0/3327162049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cec04c3c0 con 0x7f7cec159890 2023-10-10T20:42:40.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.270+0000 7f7ce2ffd700 1 -- 172.21.15.89:0/3327162049 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7cdc007440 con 0x7f7cec159890 2023-10-10T20:42:40.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.270+0000 7f7ce2ffd700 1 --2- 172.21.15.89:0/3327162049 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7ccc067320 0x7f7ccc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:40.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.270+0000 7f7ce1d9b700 1 --2- 172.21.15.89:0/3327162049 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7ccc067320 0x7f7ccc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:40.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.270+0000 7f7ce2ffd700 1 -- 172.21.15.89:0/3327162049 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7cdc084140 con 0x7f7cec159890 2023-10-10T20:42:40.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.271+0000 7f7ce1d9b700 1 --2- 172.21.15.89:0/3327162049 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7ccc067320 0x7f7ccc0697d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7cec0b2a70 tx=0x7f7cd4009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:40.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.274+0000 7f7ce2ffd700 1 -- 172.21.15.89:0/3327162049 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7cdc050dd0 con 0x7f7cec159890 2023-10-10T20:42:40.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.502+0000 7f7cf0b4f700 1 -- 172.21.15.89:0/3327162049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7cec06a5b0 con 0x7f7cec159890 2023-10-10T20:42:40.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.503+0000 7f7ce2ffd700 1 -- 172.21.15.89:0/3327162049 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4606 (secure 0 0 0) 0x7f7cdc017090 con 0x7f7cec159890 2023-10-10T20:42:40.505 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:40.505 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":102,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/830099189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":830099189},{"type":"v1","addr":"172.21.15.111:6834","nonce":830099189}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2380789310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2380789310},{"type":"v1","addr":"172.21.15.111:6835","nonce":2380789310}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7956,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2087922155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2087922155},{"type":"v1","addr":"172.21.15.89:6835","nonce":2087922155}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:42:23.479232+0000","modified":"2023-10-10T20:42:27.115668+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1243871428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1243871428},{"type":"v1","addr":"172.21.15.89:6837","nonce":1243871428}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-10T20:42:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.505+0000 7f7cf0b4f700 1 -- 172.21.15.89:0/3327162049 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7ccc067320 msgr2=0x7f7ccc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.505+0000 7f7cf0b4f700 1 --2- 172.21.15.89:0/3327162049 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7ccc067320 0x7f7ccc0697d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7cec0b2a70 tx=0x7f7cd4009250 comp rx=0 tx=0).stop 2023-10-10T20:42:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.505+0000 7f7cf0b4f700 1 -- 172.21.15.89:0/3327162049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cec159890 msgr2=0x7f7cec151ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.505+0000 7f7cf0b4f700 1 --2- 172.21.15.89:0/3327162049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cec159890 0x7f7cec151ec0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00c7e0 tx=0x7f7cdc007e10 comp rx=0 tx=0).stop 2023-10-10T20:42:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.505+0000 7f7cf0b4f700 1 -- 172.21.15.89:0/3327162049 shutdown_connections 2023-10-10T20:42:40.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.505+0000 7f7cf0b4f700 1 --2- 172.21.15.89:0/3327162049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7cec15db10 0x7f7cec15f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.505+0000 7f7cf0b4f700 1 --2- 172.21.15.89:0/3327162049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cec159890 0x7f7cec151ec0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.505+0000 7f7cf0b4f700 1 --2- 172.21.15.89:0/3327162049 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7ccc067320 0x7f7ccc0697d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.505+0000 7f7cf0b4f700 1 --2- 172.21.15.89:0/3327162049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cec159c60 0x7f7cec0cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.505+0000 7f7cf0b4f700 1 -- 172.21.15.89:0/3327162049 >> 172.21.15.89:0/3327162049 conn(0x7f7cec068e20 msgr2=0x7f7cec156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:40.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.505+0000 7f7cf0b4f700 1 -- 172.21.15.89:0/3327162049 shutdown_connections 2023-10-10T20:42:40.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.505+0000 7f7cf0b4f700 1 -- 172.21.15.89:0/3327162049 wait complete. 2023-10-10T20:42:40.510 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 102 2023-10-10T20:42:40.519 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:42:40.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.632+0000 7f192b296700 1 Processor -- start 2023-10-10T20:42:40.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.632+0000 7f192b296700 1 -- start start 2023-10-10T20:42:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.633+0000 7f192b296700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1924159890 0x7f192415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.633+0000 7f192b296700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f192415a160 con 0x7f1924159c60 2023-10-10T20:42:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.633+0000 7f192b296700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19240d13a0 con 0x7f1924069560 2023-10-10T20:42:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.633+0000 7f192b296700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f192415e140 con 0x7f1924159890 2023-10-10T20:42:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.633+0000 7f1929032700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1924159890 0x7f192415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.633+0000 7f1928831700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1924159c60 0x7f192415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59164/0 (socket says 172.21.15.89:59164) 2023-10-10T20:42:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.633+0000 7f1928831700 1 -- 172.21.15.89:0/2622166550 learned_addr learned my addr 172.21.15.89:0/2622166550 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.633+0000 7f1929032700 1 -- 172.21.15.89:0/2622166550 >> v1:172.21.15.111:6789/0 conn(0x7f1924069560 legacy=0x7f19240ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.633+0000 7f1929032700 1 -- 172.21.15.89:0/2622166550 >> v1:172.21.15.89:6789/0 conn(0x7f1924159c60 legacy=0x7f192415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.633+0000 7f1929032700 1 -- 172.21.15.89:0/2622166550 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192415a3d0 con 0x7f1924159890 2023-10-10T20:42:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.633+0000 7f1923fff700 1 -- 172.21.15.89:0/2622166550 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2862758007 0 0) 0x7f192415a160 con 0x7f1924159c60 2023-10-10T20:42:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.633+0000 7f1929032700 1 --2- 172.21.15.89:0/2622166550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1924159890 0x7f192415abd0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f191c0098e0 tx=0x7f191c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3661ad6fb66212 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:40.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.633+0000 7f1923fff700 1 -- 172.21.15.89:0/2622166550 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c009e80 con 0x7f1924159890 2023-10-10T20:42:40.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.633+0000 7f1923fff700 1 -- 172.21.15.89:0/2622166550 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191c00b820 con 0x7f1924159890 2023-10-10T20:42:40.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.633+0000 7f1923fff700 1 -- 172.21.15.89:0/2622166550 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c00bba0 con 0x7f1924159890 2023-10-10T20:42:40.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.634+0000 7f192b296700 1 -- 172.21.15.89:0/2622166550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1924159890 msgr2=0x7f192415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:40.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.634+0000 7f192b296700 1 --2- 172.21.15.89:0/2622166550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1924159890 0x7f192415abd0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f191c0098e0 tx=0x7f191c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.634+0000 7f192b296700 1 -- 172.21.15.89:0/2622166550 shutdown_connections 2023-10-10T20:42:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.634+0000 7f192b296700 1 --2- 172.21.15.89:0/2622166550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1924159890 0x7f192415abd0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.634+0000 7f192b296700 1 -- 172.21.15.89:0/2622166550 >> 172.21.15.89:0/2622166550 conn(0x7f1924068e20 msgr2=0x7f19240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.634+0000 7f192b296700 1 -- 172.21.15.89:0/2622166550 shutdown_connections 2023-10-10T20:42:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.634+0000 7f192b296700 1 -- 172.21.15.89:0/2622166550 wait complete. 2023-10-10T20:42:40.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.635+0000 7f192b296700 1 Processor -- start 2023-10-10T20:42:40.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.635+0000 7f192b296700 1 -- start start 2023-10-10T20:42:40.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.636+0000 7f192b296700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1924069560 0x7f192414d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:40.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.636+0000 7f192b296700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1924159890 0x7f192415d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:40.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.636+0000 7f192b296700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1924159c60 0x7f19240ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:40.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.636+0000 7f192b296700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19240b4850 con 0x7f1924069560 2023-10-10T20:42:40.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.636+0000 7f192b296700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19240b4ac0 con 0x7f1924159c60 2023-10-10T20:42:40.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.636+0000 7f192b296700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19240b4da0 con 0x7f1924159890 2023-10-10T20:42:40.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.636+0000 7f1929032700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1924069560 0x7f192414d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:40.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.636+0000 7f1929032700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1924069560 0x7f192414d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38924/0 (socket says 172.21.15.89:38924) 2023-10-10T20:42:40.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.636+0000 7f1929032700 1 -- 172.21.15.89:0/1754378455 learned_addr learned my addr 172.21.15.89:0/1754378455 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:40.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.636+0000 7f1928831700 1 --2- 172.21.15.89:0/1754378455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1924159890 0x7f192415d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:40.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.636+0000 7f1929833700 1 --2- 172.21.15.89:0/1754378455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1924159c60 0x7f19240ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:40.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.636+0000 7f1929032700 1 -- 172.21.15.89:0/1754378455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1924159890 msgr2=0x7f192415d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:40.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.636+0000 7f1929032700 1 --2- 172.21.15.89:0/1754378455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1924159890 0x7f192415d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.636+0000 7f1929032700 1 -- 172.21.15.89:0/1754378455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1924159c60 msgr2=0x7f19240ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:40.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.636+0000 7f1929032700 1 --2- 172.21.15.89:0/1754378455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1924159c60 0x7f19240ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.636+0000 7f1929032700 1 -- 172.21.15.89:0/1754378455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c009420 con 0x7f1924069560 2023-10-10T20:42:40.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.636+0000 7f1929833700 1 --2- 172.21.15.89:0/1754378455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1924159c60 0x7f19240ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:40.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.636+0000 7f1929032700 1 --2- 172.21.15.89:0/1754378455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1924069560 0x7f192414d3f0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f191c0030c0 tx=0x7f191c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:40.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.637+0000 7f19217fa700 1 -- 172.21.15.89:0/1754378455 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c00bae0 con 0x7f1924069560 2023-10-10T20:42:40.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.637+0000 7f19217fa700 1 -- 172.21.15.89:0/1754378455 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191c020070 con 0x7f1924069560 2023-10-10T20:42:40.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.637+0000 7f192b296700 1 -- 172.21.15.89:0/1754378455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19240b5080 con 0x7f1924069560 2023-10-10T20:42:40.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.637+0000 7f19217fa700 1 -- 172.21.15.89:0/1754378455 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c01b6a0 con 0x7f1924069560 2023-10-10T20:42:40.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.637+0000 7f192b296700 1 -- 172.21.15.89:0/1754378455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f192414d930 con 0x7f1924069560 2023-10-10T20:42:40.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.638+0000 7f192b296700 1 -- 172.21.15.89:0/1754378455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19240b55a0 con 0x7f1924069560 2023-10-10T20:42:40.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.638+0000 7f19217fa700 1 -- 172.21.15.89:0/1754378455 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f191c022030 con 0x7f1924069560 2023-10-10T20:42:40.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.638+0000 7f19217fa700 1 --2- 172.21.15.89:0/1754378455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1910067320 0x7f19100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:40.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.639+0000 7f1928831700 1 --2- 172.21.15.89:0/1754378455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1910067320 0x7f19100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.639+0000 7f19217fa700 1 -- 172.21.15.89:0/1754378455 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f191c086bd0 con 0x7f1924069560 2023-10-10T20:42:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.639+0000 7f1928831700 1 --2- 172.21.15.89:0/1754378455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1910067320 0x7f19100697d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f19240caa20 tx=0x7f1918009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.643+0000 7f19217fa700 1 -- 172.21.15.89:0/1754378455 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19240b55a0 con 0x7f1924069560 2023-10-10T20:42:40.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.829+0000 7f192b296700 1 -- 172.21.15.89:0/1754378455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19240cc820 con 0x7f1924069560 2023-10-10T20:42:40.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.829+0000 7f19217fa700 1 -- 172.21.15.89:0/1754378455 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+15364 (secure 0 0 0) 0x7f191c053970 con 0x7f1924069560 2023-10-10T20:42:40.832 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:40.832 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":153,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:42:25.092451+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:42:21.831397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-10T20:42:22.440492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:42:40.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.831+0000 7f192b296700 1 -- 172.21.15.89:0/1754378455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1910067320 msgr2=0x7f19100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:40.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.831+0000 7f192b296700 1 --2- 172.21.15.89:0/1754378455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1910067320 0x7f19100697d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f19240caa20 tx=0x7f1918009040 comp rx=0 tx=0).stop 2023-10-10T20:42:40.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.831+0000 7f192b296700 1 -- 172.21.15.89:0/1754378455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1924069560 msgr2=0x7f192414d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:40.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.831+0000 7f192b296700 1 --2- 172.21.15.89:0/1754378455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1924069560 0x7f192414d3f0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f191c0030c0 tx=0x7f191c016000 comp rx=0 tx=0).stop 2023-10-10T20:42:40.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.831+0000 7f192b296700 1 -- 172.21.15.89:0/1754378455 shutdown_connections 2023-10-10T20:42:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.831+0000 7f192b296700 1 --2- 172.21.15.89:0/1754378455 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1910067320 0x7f19100697d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.831+0000 7f192b296700 1 --2- 172.21.15.89:0/1754378455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1924159c60 0x7f19240ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.831+0000 7f192b296700 1 --2- 172.21.15.89:0/1754378455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1924069560 0x7f192414d3f0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.831+0000 7f192b296700 1 --2- 172.21.15.89:0/1754378455 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1924159890 0x7f192415d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.832+0000 7f192b296700 1 -- 172.21.15.89:0/1754378455 >> 172.21.15.89:0/1754378455 conn(0x7f1924068e20 msgr2=0x7f19240cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.832+0000 7f192b296700 1 -- 172.21.15.89:0/1754378455 shutdown_connections 2023-10-10T20:42:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.832+0000 7f192b296700 1 -- 172.21.15.89:0/1754378455 wait complete. 2023-10-10T20:42:40.846 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:42:40.846 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:42:40.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.958+0000 7f42affff700 1 Processor -- start 2023-10-10T20:42:40.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.958+0000 7f42affff700 1 -- start start 2023-10-10T20:42:40.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.959+0000 7f42affff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f42b0159420 0x7f42b00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:40.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.959+0000 7f42affff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f42b015d230 0x7f42b015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:40.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.959+0000 7f42affff700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b00b2980 con 0x7f42b01597f0 2023-10-10T20:42:40.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.959+0000 7f42affff700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b015db70 con 0x7f42b015d230 2023-10-10T20:42:40.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.959+0000 7f42affff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b00b11d0 con 0x7f42b0159420 2023-10-10T20:42:40.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.959+0000 7f42af7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f42b015d230 0x7f42b015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:40.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.959+0000 7f42a67fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f42b01597f0 0x7f42b00b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59178/0 (socket says 172.21.15.89:59178) 2023-10-10T20:42:40.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.959+0000 7f42a67fc700 1 -- 172.21.15.89:0/1905917638 learned_addr learned my addr 172.21.15.89:0/1905917638 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:40.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.960+0000 7f42af7fe700 1 -- 172.21.15.89:0/1905917638 >> v2:172.21.15.111:3300/0 conn(0x7f42b0159420 msgr2=0x7f42b00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:40.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.960+0000 7f42aeffd700 1 --2- 172.21.15.89:0/1905917638 >> v2:172.21.15.111:3300/0 conn(0x7f42b0159420 0x7f42b00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:40.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.960+0000 7f42af7fe700 1 --2- 172.21.15.89:0/1905917638 >> v2:172.21.15.111:3300/0 conn(0x7f42b0159420 0x7f42b00b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.960+0000 7f42af7fe700 1 -- 172.21.15.89:0/1905917638 >> v1:172.21.15.89:6789/0 conn(0x7f42b01597f0 legacy=0x7f42b00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:40.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.960+0000 7f42af7fe700 1 -- 172.21.15.89:0/1905917638 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b00b14b0 con 0x7f42b015d230 2023-10-10T20:42:40.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.960+0000 7f42af7fe700 1 --2- 172.21.15.89:0/1905917638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42b015d230 0x7f42b015d630 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f42b00b2f20 tx=0x7f42a8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88152d982c764d11 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:40.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.960+0000 7f42ae7fc700 1 -- 172.21.15.89:0/1905917638 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a800e070 con 0x7f42b015d230 2023-10-10T20:42:40.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.960+0000 7f42ae7fc700 1 -- 172.21.15.89:0/1905917638 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a800bc30 con 0x7f42b015d230 2023-10-10T20:42:40.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.960+0000 7f42ae7fc700 1 -- 172.21.15.89:0/1905917638 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a8012430 con 0x7f42b015d230 2023-10-10T20:42:40.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.960+0000 7f42affff700 1 -- 172.21.15.89:0/1905917638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42b015d230 msgr2=0x7f42b015d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:40.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.960+0000 7f42affff700 1 --2- 172.21.15.89:0/1905917638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42b015d230 0x7f42b015d630 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f42b00b2f20 tx=0x7f42a8009780 comp rx=0 tx=0).stop 2023-10-10T20:42:40.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.960+0000 7f42affff700 1 -- 172.21.15.89:0/1905917638 shutdown_connections 2023-10-10T20:42:40.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.960+0000 7f42affff700 1 --2- 172.21.15.89:0/1905917638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42b015d230 0x7f42b015d630 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.960+0000 7f42affff700 1 --2- 172.21.15.89:0/1905917638 >> v2:172.21.15.111:3300/0 conn(0x7f42b0159420 0x7f42b00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.960+0000 7f42affff700 1 -- 172.21.15.89:0/1905917638 >> 172.21.15.89:0/1905917638 conn(0x7f42b00d0660 msgr2=0x7f42b00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:40.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.960+0000 7f42affff700 1 -- 172.21.15.89:0/1905917638 shutdown_connections 2023-10-10T20:42:40.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.960+0000 7f42affff700 1 -- 172.21.15.89:0/1905917638 wait complete. 2023-10-10T20:42:40.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.962+0000 7f42affff700 1 Processor -- start 2023-10-10T20:42:40.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.962+0000 7f42affff700 1 -- start start 2023-10-10T20:42:40.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.962+0000 7f42affff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42b0159420 0x7f42b015c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:40.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.962+0000 7f42affff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b01597f0 0x7f42b00b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:40.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.962+0000 7f42affff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42b015d230 0x7f42b0154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:40.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.962+0000 7f42affff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b00b5ea0 con 0x7f42b015d230 2023-10-10T20:42:40.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.962+0000 7f42affff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b015bdc0 con 0x7f42b01597f0 2023-10-10T20:42:40.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.962+0000 7f42affff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b015cb70 con 0x7f42b0159420 2023-10-10T20:42:40.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.962+0000 7f42af7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42b015d230 0x7f42b0154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:40.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.962+0000 7f42af7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42b015d230 0x7f42b0154ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38946/0 (socket says 172.21.15.89:38946) 2023-10-10T20:42:40.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.962+0000 7f42af7fe700 1 -- 172.21.15.89:0/2153960751 learned_addr learned my addr 172.21.15.89:0/2153960751 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:40.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.963+0000 7f42aeffd700 1 --2- 172.21.15.89:0/2153960751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42b0159420 0x7f42b015c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:40.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.963+0000 7f42a67fc700 1 --2- 172.21.15.89:0/2153960751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b01597f0 0x7f42b00b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:40.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.963+0000 7f42af7fe700 1 -- 172.21.15.89:0/2153960751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42b0159420 msgr2=0x7f42b015c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:40.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.963+0000 7f42af7fe700 1 --2- 172.21.15.89:0/2153960751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42b0159420 0x7f42b015c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.963+0000 7f42af7fe700 1 -- 172.21.15.89:0/2153960751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b01597f0 msgr2=0x7f42b00b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:40.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.963+0000 7f42af7fe700 1 --2- 172.21.15.89:0/2153960751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b01597f0 0x7f42b00b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:40.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.963+0000 7f42af7fe700 1 -- 172.21.15.89:0/2153960751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a8009a20 con 0x7f42b015d230 2023-10-10T20:42:40.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.963+0000 7f42aeffd700 1 --2- 172.21.15.89:0/2153960751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42b0159420 0x7f42b015c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:40.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.963+0000 7f42a67fc700 1 --2- 172.21.15.89:0/2153960751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b01597f0 0x7f42b00b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:40.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.963+0000 7f42af7fe700 1 --2- 172.21.15.89:0/2153960751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42b015d230 0x7f42b0154ac0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f42b0089340 tx=0x7f42a8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:40.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.963+0000 7f42a77fe700 1 -- 172.21.15.89:0/2153960751 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a801e630 con 0x7f42b015d230 2023-10-10T20:42:40.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.963+0000 7f42a77fe700 1 -- 172.21.15.89:0/2153960751 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a801e7d0 con 0x7f42b015d230 2023-10-10T20:42:40.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.963+0000 7f42affff700 1 -- 172.21.15.89:0/2153960751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b015b1b0 con 0x7f42b015d230 2023-10-10T20:42:40.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.963+0000 7f42a77fe700 1 -- 172.21.15.89:0/2153960751 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a801eb90 con 0x7f42b015d230 2023-10-10T20:42:40.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.963+0000 7f42affff700 1 -- 172.21.15.89:0/2153960751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b015ba50 con 0x7f42b015d230 2023-10-10T20:42:40.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.964+0000 7f42affff700 1 -- 172.21.15.89:0/2153960751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b015b5d0 con 0x7f42b015d230 2023-10-10T20:42:40.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.964+0000 7f42a77fe700 1 -- 172.21.15.89:0/2153960751 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f42a8024070 con 0x7f42b015d230 2023-10-10T20:42:40.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.965+0000 7f42a77fe700 1 --2- 172.21.15.89:0/2153960751 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4290067320 0x7f42900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:40.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.965+0000 7f42a77fe700 1 -- 172.21.15.89:0/2153960751 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f42a8087660 con 0x7f42b015d230 2023-10-10T20:42:40.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.965+0000 7f42aeffd700 1 --2- 172.21.15.89:0/2153960751 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4290067320 0x7f42900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:40.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.965+0000 7f42aeffd700 1 --2- 172.21.15.89:0/2153960751 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4290067320 0x7f42900697d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f42a0008a80 tx=0x7f42a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:40.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:40.969+0000 7f42a77fe700 1 -- 172.21.15.89:0/2153960751 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42b015b5d0 con 0x7f42b015d230 2023-10-10T20:42:41.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.194+0000 7f42affff700 1 -- 172.21.15.89:0/2153960751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42b015b5d0 con 0x7f42b015d230 2023-10-10T20:42:41.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.194+0000 7f42a77fe700 1 -- 172.21.15.89:0/2153960751 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4606 (secure 0 0 0) 0x7f42b015b5d0 con 0x7f42b015d230 2023-10-10T20:42:41.197 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:41.197 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":102,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/830099189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":830099189},{"type":"v1","addr":"172.21.15.111:6834","nonce":830099189}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2380789310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2380789310},{"type":"v1","addr":"172.21.15.111:6835","nonce":2380789310}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7956,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2087922155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2087922155},{"type":"v1","addr":"172.21.15.89:6835","nonce":2087922155}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:42:23.479232+0000","modified":"2023-10-10T20:42:27.115668+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1243871428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1243871428},{"type":"v1","addr":"172.21.15.89:6837","nonce":1243871428}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-10T20:42:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.196+0000 7f42affff700 1 -- 172.21.15.89:0/2153960751 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4290067320 msgr2=0x7f42900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.196+0000 7f42affff700 1 --2- 172.21.15.89:0/2153960751 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4290067320 0x7f42900697d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f42a0008a80 tx=0x7f42a0008040 comp rx=0 tx=0).stop 2023-10-10T20:42:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.196+0000 7f42affff700 1 -- 172.21.15.89:0/2153960751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42b015d230 msgr2=0x7f42b0154ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.196+0000 7f42affff700 1 --2- 172.21.15.89:0/2153960751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42b015d230 0x7f42b0154ac0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f42b0089340 tx=0x7f42a8019000 comp rx=0 tx=0).stop 2023-10-10T20:42:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.196+0000 7f42affff700 1 -- 172.21.15.89:0/2153960751 shutdown_connections 2023-10-10T20:42:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.196+0000 7f42affff700 1 --2- 172.21.15.89:0/2153960751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42b015d230 0x7f42b0154ac0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.196+0000 7f42affff700 1 --2- 172.21.15.89:0/2153960751 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42b0159420 0x7f42b015c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.196+0000 7f42affff700 1 --2- 172.21.15.89:0/2153960751 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4290067320 0x7f42900697d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.196+0000 7f42affff700 1 --2- 172.21.15.89:0/2153960751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b01597f0 0x7f42b00b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.197+0000 7f42affff700 1 -- 172.21.15.89:0/2153960751 >> 172.21.15.89:0/2153960751 conn(0x7f42b00d0660 msgr2=0x7f42b0156a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:41.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.197+0000 7f42affff700 1 -- 172.21.15.89:0/2153960751 shutdown_connections 2023-10-10T20:42:41.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.197+0000 7f42affff700 1 -- 172.21.15.89:0/2153960751 wait complete. 2023-10-10T20:42:41.201 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 102 2023-10-10T20:42:41.210 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:42:41.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.323+0000 7f790b2bb700 1 Processor -- start 2023-10-10T20:42:41.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.323+0000 7f790b2bb700 1 -- start start 2023-10-10T20:42:41.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.324+0000 7f790b2bb700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7904159420 0x7f79040b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:41.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.324+0000 7f790b2bb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f790415d230 0x7f790415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:41.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.324+0000 7f790b2bb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79040b2980 con 0x7f79041597f0 2023-10-10T20:42:41.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.324+0000 7f790b2bb700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f790415dbb0 con 0x7f7904159420 2023-10-10T20:42:41.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.324+0000 7f790b2bb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79040b11d0 con 0x7f790415d230 2023-10-10T20:42:41.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.324+0000 7f7909ab8700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f79041597f0 0x7f79040b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59186/0 (socket says 172.21.15.89:59186) 2023-10-10T20:42:41.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.324+0000 7f7909ab8700 1 -- 172.21.15.89:0/2631921215 learned_addr learned my addr 172.21.15.89:0/2631921215 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:41.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.324+0000 7f790aaba700 1 --2- 172.21.15.89:0/2631921215 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f790415d230 0x7f790415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:41.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.324+0000 7f790a2b9700 1 --2- 172.21.15.89:0/2631921215 >> v2:172.21.15.111:3300/0 conn(0x7f7904159420 0x7f79040b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:41.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.324+0000 7f79092b7700 1 -- 172.21.15.89:0/2631921215 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 802099026 0 0) 0x7f79040b2980 con 0x7f79041597f0 2023-10-10T20:42:41.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.324+0000 7f79092b7700 1 -- 172.21.15.89:0/2631921215 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78f0003570 con 0x7f79041597f0 2023-10-10T20:42:41.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f79092b7700 1 -- 172.21.15.89:0/2631921215 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1895133050 0 0) 0x7f78f0003570 con 0x7f79041597f0 2023-10-10T20:42:41.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f79092b7700 1 -- 172.21.15.89:0/2631921215 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f790415d230 msgr2=0x7f790415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:41.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f79092b7700 1 --2- 172.21.15.89:0/2631921215 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f790415d230 0x7f790415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f79092b7700 1 -- 172.21.15.89:0/2631921215 >> v2:172.21.15.111:3300/0 conn(0x7f7904159420 msgr2=0x7f79040b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:41.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f79092b7700 1 --2- 172.21.15.89:0/2631921215 >> v2:172.21.15.111:3300/0 conn(0x7f7904159420 0x7f79040b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f79092b7700 1 -- 172.21.15.89:0/2631921215 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79040b14b0 con 0x7f79041597f0 2023-10-10T20:42:41.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f79092b7700 1 -- 172.21.15.89:0/2631921215 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f78f4001470 con 0x7f79041597f0 2023-10-10T20:42:41.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f79092b7700 1 -- 172.21.15.89:0/2631921215 >> v1:172.21.15.89:6789/0 conn(0x7f79041597f0 legacy=0x7f79040b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:41.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f79092b7700 1 --2- 172.21.15.89:0/2631921215 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f790415d230 0x7f78f0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:41.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f79092b7700 1 --2- 172.21.15.89:0/2631921215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78f0003fe0 0x7f78f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:41.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f79092b7700 1 --2- 172.21.15.89:0/2631921215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78f0004d50 0x7f78f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:41.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f79092b7700 1 -- 172.21.15.89:0/2631921215 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790415dbb0 con 0x7f78f0004d50 2023-10-10T20:42:41.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f79092b7700 1 -- 172.21.15.89:0/2631921215 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79040b11d0 con 0x7f78f0003fe0 2023-10-10T20:42:41.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f79092b7700 1 -- 172.21.15.89:0/2631921215 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f000ba90 con 0x7f790415d230 2023-10-10T20:42:41.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f790aaba700 1 --2- 172.21.15.89:0/2631921215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78f0004d50 0x7f78f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:41.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f790a2b9700 1 --2- 172.21.15.89:0/2631921215 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f790415d230 0x7f78f0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:41.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f7909ab8700 1 --2- 172.21.15.89:0/2631921215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78f0003fe0 0x7f78f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:41.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f790aaba700 1 -- 172.21.15.89:0/2631921215 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f790415d230 msgr2=0x7f78f0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:41.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f790aaba700 1 --2- 172.21.15.89:0/2631921215 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f790415d230 0x7f78f0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f790aaba700 1 -- 172.21.15.89:0/2631921215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78f0003fe0 msgr2=0x7f78f0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:41.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f790aaba700 1 --2- 172.21.15.89:0/2631921215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78f0003fe0 0x7f78f0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f790aaba700 1 -- 172.21.15.89:0/2631921215 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f0003570 con 0x7f78f0004d50 2023-10-10T20:42:41.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f790aaba700 1 --2- 172.21.15.89:0/2631921215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78f0004d50 0x7f78f00051c0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00e210 tx=0x7f78fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ac3717a1fa446f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:41.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f79092b7700 1 -- 172.21.15.89:0/2631921215 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc00ea60 con 0x7f78f0004d50 2023-10-10T20:42:41.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f79092b7700 1 -- 172.21.15.89:0/2631921215 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc00f040 con 0x7f78f0004d50 2023-10-10T20:42:41.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.325+0000 7f79092b7700 1 -- 172.21.15.89:0/2631921215 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc00ee10 con 0x7f78f0004d50 2023-10-10T20:42:41.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.326+0000 7f790b2bb700 1 -- 172.21.15.89:0/2631921215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78f0004d50 msgr2=0x7f78f00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:41.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.326+0000 7f790b2bb700 1 --2- 172.21.15.89:0/2631921215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78f0004d50 0x7f78f00051c0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00e210 tx=0x7f78fc00b040 comp rx=0 tx=0).stop 2023-10-10T20:42:41.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.326+0000 7f790aaba700 1 -- 172.21.15.89:0/2631921215 reap_dead start 2023-10-10T20:42:41.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.326+0000 7f790b2bb700 1 -- 172.21.15.89:0/2631921215 shutdown_connections 2023-10-10T20:42:41.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.326+0000 7f790b2bb700 1 -- 172.21.15.89:0/2631921215 >> 172.21.15.89:0/2631921215 conn(0x7f79040d0660 msgr2=0x7f79040d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:41.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.326+0000 7f790b2bb700 1 -- 172.21.15.89:0/2631921215 shutdown_connections 2023-10-10T20:42:41.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.326+0000 7f790b2bb700 1 -- 172.21.15.89:0/2631921215 wait complete. 2023-10-10T20:42:41.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.327+0000 7f790b2bb700 1 Processor -- start 2023-10-10T20:42:41.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.327+0000 7f790b2bb700 1 -- start start 2023-10-10T20:42:41.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.328+0000 7f790b2bb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78f0004d50 0x7f790415b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:41.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.328+0000 7f790b2bb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f790415a6b0 0x7f790414f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:41.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.328+0000 7f790b2bb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7904169ac0 0x7f7904169f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:41.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.328+0000 7f790b2bb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7904156680 con 0x7f790415a6b0 2023-10-10T20:42:41.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.328+0000 7f790b2bb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7904157200 con 0x7f78f0004d50 2023-10-10T20:42:41.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.328+0000 7f790b2bb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79041568d0 con 0x7f7904169ac0 2023-10-10T20:42:41.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.328+0000 7f7909ab8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f790415a6b0 0x7f790414f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:41.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.328+0000 7f7909ab8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f790415a6b0 0x7f790414f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38964/0 (socket says 172.21.15.89:38964) 2023-10-10T20:42:41.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.328+0000 7f7909ab8700 1 -- 172.21.15.89:0/779066409 learned_addr learned my addr 172.21.15.89:0/779066409 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:41.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.328+0000 7f790a2b9700 1 --2- 172.21.15.89:0/779066409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78f0004d50 0x7f790415b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:41.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.328+0000 7f790aaba700 1 --2- 172.21.15.89:0/779066409 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7904169ac0 0x7f7904169f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:41.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.328+0000 7f7909ab8700 1 -- 172.21.15.89:0/779066409 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7904169ac0 msgr2=0x7f7904169f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:41.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.328+0000 7f7909ab8700 1 --2- 172.21.15.89:0/779066409 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7904169ac0 0x7f7904169f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.328+0000 7f7909ab8700 1 -- 172.21.15.89:0/779066409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78f0004d50 msgr2=0x7f790415b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:41.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.328+0000 7f7909ab8700 1 --2- 172.21.15.89:0/779066409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78f0004d50 0x7f790415b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.328+0000 7f7909ab8700 1 -- 172.21.15.89:0/779066409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7904156bb0 con 0x7f790415a6b0 2023-10-10T20:42:41.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.328+0000 7f7909ab8700 1 --2- 172.21.15.89:0/779066409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f790415a6b0 0x7f790414f0e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f78f40073c0 tx=0x7f78f4001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:41.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.329+0000 7f78fa7fc700 1 -- 172.21.15.89:0/779066409 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f4018030 con 0x7f790415a6b0 2023-10-10T20:42:41.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.329+0000 7f78fa7fc700 1 -- 172.21.15.89:0/779066409 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f400f870 con 0x7f790415a6b0 2023-10-10T20:42:41.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.329+0000 7f790b2bb700 1 -- 172.21.15.89:0/779066409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7904156e90 con 0x7f790415a6b0 2023-10-10T20:42:41.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.329+0000 7f78fa7fc700 1 -- 172.21.15.89:0/779066409 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f400fc30 con 0x7f790415a6b0 2023-10-10T20:42:41.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.329+0000 7f790b2bb700 1 -- 172.21.15.89:0/779066409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790414f620 con 0x7f790415a6b0 2023-10-10T20:42:41.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.330+0000 7f790b2bb700 1 -- 172.21.15.89:0/779066409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7904157390 con 0x7f790415a6b0 2023-10-10T20:42:41.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.330+0000 7f78fa7fc700 1 -- 172.21.15.89:0/779066409 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f78f401b070 con 0x7f790415a6b0 2023-10-10T20:42:41.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.330+0000 7f78fa7fc700 1 --2- 172.21.15.89:0/779066409 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f78e8067320 0x7f78e80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:41.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.330+0000 7f78fa7fc700 1 -- 172.21.15.89:0/779066409 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f78f4019070 con 0x7f790415a6b0 2023-10-10T20:42:41.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.330+0000 7f790a2b9700 1 --2- 172.21.15.89:0/779066409 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f78e8067320 0x7f78e80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:41.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.331+0000 7f790a2b9700 1 --2- 172.21.15.89:0/779066409 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f78e8067320 0x7f78e80697d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7900006f00 tx=0x7f79000091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:41.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.334+0000 7f78fa7fc700 1 -- 172.21.15.89:0/779066409 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78f4051810 con 0x7f790415a6b0 2023-10-10T20:42:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.556+0000 7f790b2bb700 1 -- 172.21.15.89:0/779066409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f790404c3c0 con 0x7f790415a6b0 2023-10-10T20:42:41.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.557+0000 7f78fa7fc700 1 -- 172.21.15.89:0/779066409 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4606 (secure 0 0 0) 0x7f78f4054e30 con 0x7f790415a6b0 2023-10-10T20:42:41.559 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:41.560 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":102,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/830099189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":830099189},{"type":"v1","addr":"172.21.15.111:6834","nonce":830099189}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2380789310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2380789310},{"type":"v1","addr":"172.21.15.111:6835","nonce":2380789310}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7956,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2087922155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2087922155},{"type":"v1","addr":"172.21.15.89:6835","nonce":2087922155}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:42:23.479232+0000","modified":"2023-10-10T20:42:27.115668+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1243871428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1243871428},{"type":"v1","addr":"172.21.15.89:6837","nonce":1243871428}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-10T20:42:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.559+0000 7f790b2bb700 1 -- 172.21.15.89:0/779066409 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f78e8067320 msgr2=0x7f78e80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:41.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.559+0000 7f790b2bb700 1 --2- 172.21.15.89:0/779066409 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f78e8067320 0x7f78e80697d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7900006f00 tx=0x7f79000091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.559+0000 7f790b2bb700 1 -- 172.21.15.89:0/779066409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f790415a6b0 msgr2=0x7f790414f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:41.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.559+0000 7f790b2bb700 1 --2- 172.21.15.89:0/779066409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f790415a6b0 0x7f790414f0e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f78f40073c0 tx=0x7f78f4001e90 comp rx=0 tx=0).stop 2023-10-10T20:42:41.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.559+0000 7f790b2bb700 1 -- 172.21.15.89:0/779066409 shutdown_connections 2023-10-10T20:42:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.559+0000 7f790b2bb700 1 --2- 172.21.15.89:0/779066409 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7904169ac0 0x7f7904169f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.559+0000 7f790b2bb700 1 --2- 172.21.15.89:0/779066409 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f78e8067320 0x7f78e80697d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.559+0000 7f790b2bb700 1 --2- 172.21.15.89:0/779066409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78f0004d50 0x7f790415b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.559+0000 7f790b2bb700 1 --2- 172.21.15.89:0/779066409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f790415a6b0 0x7f790414f0e0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.559+0000 7f790b2bb700 1 -- 172.21.15.89:0/779066409 >> 172.21.15.89:0/779066409 conn(0x7f79040d0660 msgr2=0x7f79040b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.559+0000 7f790b2bb700 1 -- 172.21.15.89:0/779066409 shutdown_connections 2023-10-10T20:42:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.559+0000 7f790b2bb700 1 -- 172.21.15.89:0/779066409 wait complete. 2023-10-10T20:42:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 102 2023-10-10T20:42:41.573 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:42:41.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.685+0000 7f4174434700 1 Processor -- start 2023-10-10T20:42:41.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.685+0000 7f4174434700 1 -- start start 2023-10-10T20:42:41.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.686+0000 7f4174434700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f416c06bcd0 0x7f416c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:41.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.686+0000 7f4174434700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f416c06c0a0 0x7f416c15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:41.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.686+0000 7f4174434700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f416c15d7e0 con 0x7f416c15df50 2023-10-10T20:42:41.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.686+0000 7f4174434700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f416c15dac0 con 0x7f416c06c0a0 2023-10-10T20:42:41.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.686+0000 7f4174434700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f416c15aa30 con 0x7f416c06bcd0 2023-10-10T20:42:41.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.686+0000 7f41719cf700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f416c06c0a0 0x7f416c15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:41.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.686+0000 7f41729d1700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f416c15df50 0x7f416c0c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59196/0 (socket says 172.21.15.89:59196) 2023-10-10T20:42:41.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.686+0000 7f41729d1700 1 -- 172.21.15.89:0/3307486939 learned_addr learned my addr 172.21.15.89:0/3307486939 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:41.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.686+0000 7f41721d0700 1 --2- 172.21.15.89:0/3307486939 >> v2:172.21.15.111:3300/0 conn(0x7f416c06bcd0 0x7f416c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:41.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.687+0000 7f41719cf700 1 -- 172.21.15.89:0/3307486939 >> v2:172.21.15.111:3300/0 conn(0x7f416c06bcd0 msgr2=0x7f416c0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:41.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.687+0000 7f41719cf700 1 --2- 172.21.15.89:0/3307486939 >> v2:172.21.15.111:3300/0 conn(0x7f416c06bcd0 0x7f416c0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.687+0000 7f41719cf700 1 -- 172.21.15.89:0/3307486939 >> v1:172.21.15.89:6789/0 conn(0x7f416c15df50 legacy=0x7f416c0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:41.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.687+0000 7f41719cf700 1 -- 172.21.15.89:0/3307486939 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c15ad10 con 0x7f416c06c0a0 2023-10-10T20:42:41.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.687+0000 7f41711ce700 1 -- 172.21.15.89:0/3307486939 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532143187 0 0) 0x7f416c15d7e0 con 0x7f416c15df50 2023-10-10T20:42:41.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.687+0000 7f41719cf700 1 --2- 172.21.15.89:0/3307486939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c06c0a0 0x7f416c15a1e0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f416c0691a0 tx=0x7f415c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbecc00db8d341b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:41.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.687+0000 7f41711ce700 1 -- 172.21.15.89:0/3307486939 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c011630 con 0x7f416c06c0a0 2023-10-10T20:42:41.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.687+0000 7f41711ce700 1 -- 172.21.15.89:0/3307486939 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415c011790 con 0x7f416c06c0a0 2023-10-10T20:42:41.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.687+0000 7f41711ce700 1 -- 172.21.15.89:0/3307486939 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c011b30 con 0x7f416c06c0a0 2023-10-10T20:42:41.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.687+0000 7f4174434700 1 -- 172.21.15.89:0/3307486939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c06c0a0 msgr2=0x7f416c15a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:41.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.687+0000 7f4174434700 1 --2- 172.21.15.89:0/3307486939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c06c0a0 0x7f416c15a1e0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f416c0691a0 tx=0x7f415c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.687+0000 7f4174434700 1 -- 172.21.15.89:0/3307486939 shutdown_connections 2023-10-10T20:42:41.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.687+0000 7f4174434700 1 --2- 172.21.15.89:0/3307486939 >> v2:172.21.15.111:3300/0 conn(0x7f416c06bcd0 0x7f416c0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.687+0000 7f4174434700 1 --2- 172.21.15.89:0/3307486939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c06c0a0 0x7f416c15a1e0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.687+0000 7f4174434700 1 -- 172.21.15.89:0/3307486939 >> 172.21.15.89:0/3307486939 conn(0x7f416c0ce600 msgr2=0x7f416c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:41.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.687+0000 7f4174434700 1 -- 172.21.15.89:0/3307486939 shutdown_connections 2023-10-10T20:42:41.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.688+0000 7f4174434700 1 -- 172.21.15.89:0/3307486939 wait complete. 2023-10-10T20:42:41.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.689+0000 7f4174434700 1 Processor -- start 2023-10-10T20:42:41.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.689+0000 7f4174434700 1 -- start start 2023-10-10T20:42:41.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.689+0000 7f4174434700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f416c06bcd0 0x7f416c0b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:41.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.689+0000 7f4174434700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c06c0a0 0x7f416c0c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:41.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.689+0000 7f4174434700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f416c15df50 0x7f416c1549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:41.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.689+0000 7f4174434700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c0b3010 con 0x7f416c06c0a0 2023-10-10T20:42:41.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.689+0000 7f4174434700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c0c8470 con 0x7f416c15df50 2023-10-10T20:42:41.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.689+0000 7f4174434700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416c0b2390 con 0x7f416c06bcd0 2023-10-10T20:42:41.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.689+0000 7f41719cf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c06c0a0 0x7f416c0c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:41.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.690+0000 7f41719cf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c06c0a0 0x7f416c0c7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38982/0 (socket says 172.21.15.89:38982) 2023-10-10T20:42:41.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.690+0000 7f41719cf700 1 -- 172.21.15.89:0/2159775762 learned_addr learned my addr 172.21.15.89:0/2159775762 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.690+0000 7f41729d1700 1 --2- 172.21.15.89:0/2159775762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f416c15df50 0x7f416c1549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.690+0000 7f41721d0700 1 --2- 172.21.15.89:0/2159775762 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f416c06bcd0 0x7f416c0b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.690+0000 7f41719cf700 1 -- 172.21.15.89:0/2159775762 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f416c06bcd0 msgr2=0x7f416c0b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.690+0000 7f41719cf700 1 --2- 172.21.15.89:0/2159775762 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f416c06bcd0 0x7f416c0b1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.690+0000 7f41719cf700 1 -- 172.21.15.89:0/2159775762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f416c15df50 msgr2=0x7f416c1549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:41.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.690+0000 7f41719cf700 1 --2- 172.21.15.89:0/2159775762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f416c15df50 0x7f416c1549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.690+0000 7f41719cf700 1 -- 172.21.15.89:0/2159775762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c009420 con 0x7f416c06c0a0 2023-10-10T20:42:41.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.690+0000 7f41719cf700 1 --2- 172.21.15.89:0/2159775762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c06c0a0 0x7f416c0c7f30 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f416c069220 tx=0x7f415c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:41.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.690+0000 7f41627fc700 1 -- 172.21.15.89:0/2159775762 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c011b10 con 0x7f416c06c0a0 2023-10-10T20:42:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.690+0000 7f41627fc700 1 -- 172.21.15.89:0/2159775762 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415c0253f0 con 0x7f416c06c0a0 2023-10-10T20:42:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.690+0000 7f4174434700 1 -- 172.21.15.89:0/2159775762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416c0b2670 con 0x7f416c06c0a0 2023-10-10T20:42:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.690+0000 7f41627fc700 1 -- 172.21.15.89:0/2159775762 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c0257d0 con 0x7f416c06c0a0 2023-10-10T20:42:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.690+0000 7f4174434700 1 -- 172.21.15.89:0/2159775762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416c0b2d20 con 0x7f416c06c0a0 2023-10-10T20:42:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.691+0000 7f4174434700 1 -- 172.21.15.89:0/2159775762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416c0b31c0 con 0x7f416c06c0a0 2023-10-10T20:42:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.691+0000 7f41627fc700 1 -- 172.21.15.89:0/2159775762 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f415c025c00 con 0x7f416c06c0a0 2023-10-10T20:42:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.692+0000 7f41627fc700 1 --2- 172.21.15.89:0/2159775762 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4150067320 0x7f41500697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.692+0000 7f41627fc700 1 -- 172.21.15.89:0/2159775762 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f415c086460 con 0x7f416c06c0a0 2023-10-10T20:42:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.692+0000 7f41721d0700 1 --2- 172.21.15.89:0/2159775762 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4150067320 0x7f41500697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.692+0000 7f41721d0700 1 --2- 172.21.15.89:0/2159775762 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4150067320 0x7f41500697d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f416c0cad50 tx=0x7f416800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:41.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.696+0000 7f41627fc700 1 -- 172.21.15.89:0/2159775762 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f416c0b31c0 con 0x7f416c06c0a0 2023-10-10T20:42:41.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.880+0000 7f4174434700 1 -- 172.21.15.89:0/2159775762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f416c064ed0 con 0x7f416c06c0a0 2023-10-10T20:42:41.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.881+0000 7f41627fc700 1 -- 172.21.15.89:0/2159775762 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+15364 (secure 0 0 0) 0x7f415c0530f0 con 0x7f416c06c0a0 2023-10-10T20:42:41.883 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:41.883 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":153,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:42:25.092451+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:42:21.831397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-10T20:42:22.440492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:42:41.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.882+0000 7f4174434700 1 -- 172.21.15.89:0/2159775762 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4150067320 msgr2=0x7f41500697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:41.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.882+0000 7f4174434700 1 --2- 172.21.15.89:0/2159775762 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4150067320 0x7f41500697d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f416c0cad50 tx=0x7f416800b040 comp rx=0 tx=0).stop 2023-10-10T20:42:41.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.882+0000 7f4174434700 1 -- 172.21.15.89:0/2159775762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c06c0a0 msgr2=0x7f416c0c7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:41.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.882+0000 7f4174434700 1 --2- 172.21.15.89:0/2159775762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c06c0a0 0x7f416c0c7f30 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f416c069220 tx=0x7f415c017000 comp rx=0 tx=0).stop 2023-10-10T20:42:41.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.883+0000 7f4174434700 1 -- 172.21.15.89:0/2159775762 shutdown_connections 2023-10-10T20:42:41.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.883+0000 7f4174434700 1 --2- 172.21.15.89:0/2159775762 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4150067320 0x7f41500697d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.883+0000 7f4174434700 1 --2- 172.21.15.89:0/2159775762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f416c15df50 0x7f416c1549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.883+0000 7f4174434700 1 --2- 172.21.15.89:0/2159775762 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f416c06bcd0 0x7f416c0b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.883+0000 7f4174434700 1 --2- 172.21.15.89:0/2159775762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c06c0a0 0x7f416c0c7f30 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:41.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.883+0000 7f4174434700 1 -- 172.21.15.89:0/2159775762 >> 172.21.15.89:0/2159775762 conn(0x7f416c0ce600 msgr2=0x7f416c15c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:41.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.883+0000 7f4174434700 1 -- 172.21.15.89:0/2159775762 shutdown_connections 2023-10-10T20:42:41.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:41.883+0000 7f4174434700 1 -- 172.21.15.89:0/2159775762 wait complete. 2023-10-10T20:42:41.897 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:42:42.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.010+0000 7f05642e5700 1 Processor -- start 2023-10-10T20:42:42.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.010+0000 7f05642e5700 1 -- start start 2023-10-10T20:42:42.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.011+0000 7f05642e5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f055c068550 0x7f055c068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:42.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.011+0000 7f05642e5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f055c15e010 0x7f055c06f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:42.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.011+0000 7f05642e5700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f055c06f640 0x7f055c06a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:42.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.011+0000 7f05642e5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f055c0b3430 con 0x7f055c15e010 2023-10-10T20:42:42.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.011+0000 7f05642e5700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f055c06ec70 con 0x7f055c06f640 2023-10-10T20:42:42.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.011+0000 7f05642e5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055c0b2e70 con 0x7f055c068550 2023-10-10T20:42:42.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.011+0000 7f0561880700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f055c15e010 0x7f055c06f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:42.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.011+0000 7f0561880700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f055c15e010 0x7f055c06f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38996/0 (socket says 172.21.15.89:38996) 2023-10-10T20:42:42.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.011+0000 7f0561880700 1 -- 172.21.15.89:0/791940055 learned_addr learned my addr 172.21.15.89:0/791940055 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:42.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.012+0000 7f0562081700 1 --2- 172.21.15.89:0/791940055 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f055c068550 0x7f055c068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:42.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.012+0000 7f0562882700 1 --2- 172.21.15.89:0/791940055 >> v2:172.21.15.111:3300/0 conn(0x7f055c06f640 0x7f055c06a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:42.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.012+0000 7f0561880700 1 -- 172.21.15.89:0/791940055 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f055c068550 msgr2=0x7f055c068920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:42.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.012+0000 7f0561880700 1 --2- 172.21.15.89:0/791940055 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f055c068550 0x7f055c068920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:42.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.012+0000 7f0561880700 1 -- 172.21.15.89:0/791940055 >> v2:172.21.15.111:3300/0 conn(0x7f055c06f640 msgr2=0x7f055c06a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:42.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.012+0000 7f0561880700 1 --2- 172.21.15.89:0/791940055 >> v2:172.21.15.111:3300/0 conn(0x7f055c06f640 0x7f055c06a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:42.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.012+0000 7f0561880700 1 -- 172.21.15.89:0/791940055 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c0b3150 con 0x7f055c15e010 2023-10-10T20:42:42.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.012+0000 7f0561880700 1 --2- 172.21.15.89:0/791940055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f055c15e010 0x7f055c06f100 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f055c06ebe0 tx=0x7f0550009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570fae05b096bfa3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:42.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.012+0000 7f056107f700 1 -- 172.21.15.89:0/791940055 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0550009e70 con 0x7f055c15e010 2023-10-10T20:42:42.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.012+0000 7f056107f700 1 -- 172.21.15.89:0/791940055 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055000b860 con 0x7f055c15e010 2023-10-10T20:42:42.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.012+0000 7f056107f700 1 -- 172.21.15.89:0/791940055 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055000bc00 con 0x7f055c15e010 2023-10-10T20:42:42.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.013+0000 7f05642e5700 1 -- 172.21.15.89:0/791940055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f055c15e010 msgr2=0x7f055c06f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:42.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.013+0000 7f05642e5700 1 --2- 172.21.15.89:0/791940055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f055c15e010 0x7f055c06f100 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f055c06ebe0 tx=0x7f0550009250 comp rx=0 tx=0).stop 2023-10-10T20:42:42.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.013+0000 7f05642e5700 1 -- 172.21.15.89:0/791940055 shutdown_connections 2023-10-10T20:42:42.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.013+0000 7f05642e5700 1 --2- 172.21.15.89:0/791940055 >> v2:172.21.15.111:3300/0 conn(0x7f055c06f640 0x7f055c06a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:42.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.013+0000 7f05642e5700 1 --2- 172.21.15.89:0/791940055 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f055c068550 0x7f055c068920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:42.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.013+0000 7f05642e5700 1 --2- 172.21.15.89:0/791940055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f055c15e010 0x7f055c06f100 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:42.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.013+0000 7f05642e5700 1 -- 172.21.15.89:0/791940055 >> 172.21.15.89:0/791940055 conn(0x7f055c0cd480 msgr2=0x7f055c0cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:42.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.013+0000 7f05642e5700 1 -- 172.21.15.89:0/791940055 shutdown_connections 2023-10-10T20:42:42.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.013+0000 7f05642e5700 1 -- 172.21.15.89:0/791940055 wait complete. 2023-10-10T20:42:42.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.014+0000 7f05642e5700 1 Processor -- start 2023-10-10T20:42:42.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.014+0000 7f05642e5700 1 -- start start 2023-10-10T20:42:42.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.014+0000 7f05642e5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f055c068550 0x7f055c146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:42.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.014+0000 7f05642e5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f055c06f640 0x7f055c154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:42.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.014+0000 7f05642e5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f055c15e010 0x7f055c1546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:42.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.014+0000 7f05642e5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c15aa80 con 0x7f055c06f640 2023-10-10T20:42:42.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.014+0000 7f05642e5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c147500 con 0x7f055c068550 2023-10-10T20:42:42.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.014+0000 7f05642e5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055c159e70 con 0x7f055c15e010 2023-10-10T20:42:42.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.015+0000 7f0561880700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f055c06f640 0x7f055c154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:42.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.015+0000 7f0561880700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f055c06f640 0x7f055c154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39000/0 (socket says 172.21.15.89:39000) 2023-10-10T20:42:42.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.015+0000 7f0561880700 1 -- 172.21.15.89:0/3687510649 learned_addr learned my addr 172.21.15.89:0/3687510649 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:42.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.015+0000 7f0562081700 1 --2- 172.21.15.89:0/3687510649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f055c068550 0x7f055c146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:42.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.015+0000 7f0562882700 1 --2- 172.21.15.89:0/3687510649 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f055c15e010 0x7f055c1546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:42.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.015+0000 7f0561880700 1 -- 172.21.15.89:0/3687510649 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f055c15e010 msgr2=0x7f055c1546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:42.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.015+0000 7f0561880700 1 --2- 172.21.15.89:0/3687510649 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f055c15e010 0x7f055c1546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:42.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.015+0000 7f0561880700 1 -- 172.21.15.89:0/3687510649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f055c068550 msgr2=0x7f055c146fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:42.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.015+0000 7f0561880700 1 --2- 172.21.15.89:0/3687510649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f055c068550 0x7f055c146fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:42.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.015+0000 7f0561880700 1 -- 172.21.15.89:0/3687510649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05500094f0 con 0x7f055c06f640 2023-10-10T20:42:42.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.015+0000 7f0561880700 1 --2- 172.21.15.89:0/3687510649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f055c06f640 0x7f055c154160 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f055c0ce220 tx=0x7f0550017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:42.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.015+0000 7f055a7fc700 1 -- 172.21.15.89:0/3687510649 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055000bbe0 con 0x7f055c06f640 2023-10-10T20:42:42.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.015+0000 7f055a7fc700 1 -- 172.21.15.89:0/3687510649 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0550000c90 con 0x7f055c06f640 2023-10-10T20:42:42.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.015+0000 7f05642e5700 1 -- 172.21.15.89:0/3687510649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055c15a150 con 0x7f055c06f640 2023-10-10T20:42:42.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.015+0000 7f055a7fc700 1 -- 172.21.15.89:0/3687510649 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0550011650 con 0x7f055c06f640 2023-10-10T20:42:42.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.015+0000 7f05642e5700 1 -- 172.21.15.89:0/3687510649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055c15a710 con 0x7f055c06f640 2023-10-10T20:42:42.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.016+0000 7f05642e5700 1 -- 172.21.15.89:0/3687510649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f055c04c3c0 con 0x7f055c06f640 2023-10-10T20:42:42.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.016+0000 7f055a7fc700 1 -- 172.21.15.89:0/3687510649 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0550011a80 con 0x7f055c06f640 2023-10-10T20:42:42.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.017+0000 7f055a7fc700 1 --2- 172.21.15.89:0/3687510649 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0548067320 0x7f05480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:42.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.017+0000 7f055a7fc700 1 -- 172.21.15.89:0/3687510649 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0550086040 con 0x7f055c06f640 2023-10-10T20:42:42.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.017+0000 7f0562081700 1 --2- 172.21.15.89:0/3687510649 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0548067320 0x7f05480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:42.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.017+0000 7f0562081700 1 --2- 172.21.15.89:0/3687510649 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0548067320 0x7f05480697d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0554009fd0 tx=0x7f055400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:42.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.021+0000 7f055a7fc700 1 -- 172.21.15.89:0/3687510649 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05500b75a0 con 0x7f055c06f640 2023-10-10T20:42:42.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.249+0000 7f05642e5700 1 -- 172.21.15.89:0/3687510649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f055c15abc0 con 0x7f055c06f640 2023-10-10T20:42:42.252 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-10-10T20:42:42.250+0000 7fd0184ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:42:42.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.254+0000 7f055a7fc700 1 -- 172.21.15.89:0/3687510649 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v103) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0550052cd0 con 0x7f055c06f640 2023-10-10T20:42:42.256 INFO:tasks.ceph.mds.c.smithi089.stderr: -717> 2023-10-10T20:42:29.385+0000 7f0e85cdc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:42:42.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.257+0000 7f05642e5700 1 -- 172.21.15.89:0/3687510649 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0548067320 msgr2=0x7f05480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:42.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.257+0000 7f05642e5700 1 --2- 172.21.15.89:0/3687510649 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0548067320 0x7f05480697d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0554009fd0 tx=0x7f055400b040 comp rx=0 tx=0).stop 2023-10-10T20:42:42.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.257+0000 7f05642e5700 1 -- 172.21.15.89:0/3687510649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f055c06f640 msgr2=0x7f055c154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:42.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.257+0000 7f05642e5700 1 --2- 172.21.15.89:0/3687510649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f055c06f640 0x7f055c154160 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f055c0ce220 tx=0x7f0550017000 comp rx=0 tx=0).stop 2023-10-10T20:42:42.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.257+0000 7f05642e5700 1 -- 172.21.15.89:0/3687510649 shutdown_connections 2023-10-10T20:42:42.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.257+0000 7f05642e5700 1 --2- 172.21.15.89:0/3687510649 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f055c15e010 0x7f055c1546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:42.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.257+0000 7f05642e5700 1 --2- 172.21.15.89:0/3687510649 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0548067320 0x7f05480697d0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:42.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.257+0000 7f05642e5700 1 --2- 172.21.15.89:0/3687510649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f055c068550 0x7f055c146fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:42.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.257+0000 7f05642e5700 1 --2- 172.21.15.89:0/3687510649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f055c06f640 0x7f055c154160 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:42.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.257+0000 7f05642e5700 1 -- 172.21.15.89:0/3687510649 >> 172.21.15.89:0/3687510649 conn(0x7f055c0cd480 msgr2=0x7f055c156190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:42.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.257+0000 7f05642e5700 1 -- 172.21.15.89:0/3687510649 shutdown_connections 2023-10-10T20:42:42.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.257+0000 7f05642e5700 1 -- 172.21.15.89:0/3687510649 wait complete. 2023-10-10T20:42:42.262 INFO:teuthology.orchestra.run.smithi089.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:42:42.272 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:42:42.277 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-10T20:42:42.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.386+0000 7f3e68f77700 1 Processor -- start 2023-10-10T20:42:42.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.386+0000 7f3e68f77700 1 -- start start 2023-10-10T20:42:42.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.387+0000 7f3e68f77700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3e640637d0 0x7f3e6406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:42.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.387+0000 7f3e68f77700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e6406a2b0 0x7f3e6406a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:42.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.387+0000 7f3e68f77700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e640ac450 con 0x7f3e6406b160 2023-10-10T20:42:42.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.387+0000 7f3e68f77700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e640693d0 con 0x7f3e640637d0 2023-10-10T20:42:42.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.387+0000 7f3e68f77700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e64069d00 con 0x7f3e6406a2b0 2023-10-10T20:42:42.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.387+0000 7f3e59d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3e640637d0 0x7f3e6406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:42.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.387+0000 7f3e59d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3e640637d0 0x7f3e6406b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39022/0 (socket says 172.21.15.89:39022) 2023-10-10T20:42:42.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.387+0000 7f3e59d9b700 1 -- 172.21.15.89:0/7466775 learned_addr learned my addr 172.21.15.89:0/7466775 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:42.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.388+0000 7f3e62d9d700 1 --2- 172.21.15.89:0/7466775 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e6406a2b0 0x7f3e6406a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:42.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.388+0000 7f3e59d9b700 1 -- 172.21.15.89:0/7466775 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e6406a2b0 msgr2=0x7f3e6406a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:42.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.388+0000 7f3e59d9b700 1 --2- 172.21.15.89:0/7466775 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e6406a2b0 0x7f3e6406a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:42.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.388+0000 7f3e59d9b700 1 -- 172.21.15.89:0/7466775 >> v1:172.21.15.111:6789/0 conn(0x7f3e6406b160 legacy=0x7f3e64068cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:42.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.388+0000 7f3e59d9b700 1 -- 172.21.15.89:0/7466775 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e64068830 con 0x7f3e640637d0 2023-10-10T20:42:42.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.388+0000 7f3e59d9b700 1 --2- 172.21.15.89:0/7466775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e640637d0 0x7f3e6406b530 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c0099b0 tx=0x7f3e4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbecc028f8c85b66 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:42.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.388+0000 7f3e61d9b700 1 -- 172.21.15.89:0/7466775 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c00b820 con 0x7f3e640637d0 2023-10-10T20:42:42.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.388+0000 7f3e61d9b700 1 -- 172.21.15.89:0/7466775 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4c00b980 con 0x7f3e640637d0 2023-10-10T20:42:42.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.388+0000 7f3e61d9b700 1 -- 172.21.15.89:0/7466775 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c00bd40 con 0x7f3e640637d0 2023-10-10T20:42:42.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.388+0000 7f3e68f77700 1 -- 172.21.15.89:0/7466775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e640637d0 msgr2=0x7f3e6406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:42.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.388+0000 7f3e68f77700 1 --2- 172.21.15.89:0/7466775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e640637d0 0x7f3e6406b530 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c0099b0 tx=0x7f3e4c009250 comp rx=0 tx=0).stop 2023-10-10T20:42:42.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.388+0000 7f3e68f77700 1 -- 172.21.15.89:0/7466775 shutdown_connections 2023-10-10T20:42:42.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.388+0000 7f3e68f77700 1 --2- 172.21.15.89:0/7466775 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e6406a2b0 0x7f3e6406a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:42.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.388+0000 7f3e68f77700 1 --2- 172.21.15.89:0/7466775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e640637d0 0x7f3e6406b530 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:42.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.388+0000 7f3e68f77700 1 -- 172.21.15.89:0/7466775 >> 172.21.15.89:0/7466775 conn(0x7f3e6406ded0 msgr2=0x7f3e6406e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:42.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.388+0000 7f3e68f77700 1 -- 172.21.15.89:0/7466775 shutdown_connections 2023-10-10T20:42:42.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.388+0000 7f3e68f77700 1 -- 172.21.15.89:0/7466775 wait complete. 2023-10-10T20:42:42.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.390+0000 7f3e68f77700 1 Processor -- start 2023-10-10T20:42:42.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.390+0000 7f3e68f77700 1 -- start start 2023-10-10T20:42:42.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.390+0000 7f3e68f77700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e640637d0 0x7f3e64144580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:42.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.390+0000 7f3e68f77700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e6406a2b0 0x7f3e64155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:42.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.390+0000 7f3e68f77700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e6406b160 0x7f3e64157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:42.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.390+0000 7f3e68f77700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e64155000 con 0x7f3e6406b160 2023-10-10T20:42:42.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.390+0000 7f3e68f77700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e641546d0 con 0x7f3e640637d0 2023-10-10T20:42:42.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.390+0000 7f3e68f77700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e641549b0 con 0x7f3e6406a2b0 2023-10-10T20:42:42.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.391+0000 7f3e62d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e6406b160 0x7f3e64157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:42.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.391+0000 7f3e62d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e6406b160 0x7f3e64157b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39036/0 (socket says 172.21.15.89:39036) 2023-10-10T20:42:42.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.391+0000 7f3e62d9d700 1 -- 172.21.15.89:0/50667252 learned_addr learned my addr 172.21.15.89:0/50667252 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:42.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.391+0000 7f3e6259c700 1 --2- 172.21.15.89:0/50667252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e640637d0 0x7f3e64144580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:42.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.391+0000 7f3e59d9b700 1 --2- 172.21.15.89:0/50667252 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e6406a2b0 0x7f3e64155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:42.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.391+0000 7f3e62d9d700 1 -- 172.21.15.89:0/50667252 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e6406a2b0 msgr2=0x7f3e64155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:42.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.391+0000 7f3e62d9d700 1 --2- 172.21.15.89:0/50667252 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e6406a2b0 0x7f3e64155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:42.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.391+0000 7f3e62d9d700 1 -- 172.21.15.89:0/50667252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e640637d0 msgr2=0x7f3e64144580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:42.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.391+0000 7f3e62d9d700 1 --2- 172.21.15.89:0/50667252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e640637d0 0x7f3e64144580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:42.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.391+0000 7f3e62d9d700 1 -- 172.21.15.89:0/50667252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4c0094f0 con 0x7f3e6406b160 2023-10-10T20:42:42.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.391+0000 7f3e62d9d700 1 --2- 172.21.15.89:0/50667252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e6406b160 0x7f3e64157b10 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009fa0 tx=0x7f3e5c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:42.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.391+0000 7f3e5affd700 1 -- 172.21.15.89:0/50667252 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c00ebc0 con 0x7f3e6406b160 2023-10-10T20:42:42.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.391+0000 7f3e5affd700 1 -- 172.21.15.89:0/50667252 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5c00ed20 con 0x7f3e6406b160 2023-10-10T20:42:42.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.391+0000 7f3e68f77700 1 -- 172.21.15.89:0/50667252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e64154c90 con 0x7f3e6406b160 2023-10-10T20:42:42.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.391+0000 7f3e5affd700 1 -- 172.21.15.89:0/50667252 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c014670 con 0x7f3e6406b160 2023-10-10T20:42:42.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.391+0000 7f3e68f77700 1 -- 172.21.15.89:0/50667252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e641559d0 con 0x7f3e6406b160 2023-10-10T20:42:42.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.392+0000 7f3e68f77700 1 -- 172.21.15.89:0/50667252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e6404c3c0 con 0x7f3e6406b160 2023-10-10T20:42:42.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.393+0000 7f3e5affd700 1 -- 172.21.15.89:0/50667252 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3e5c0149e0 con 0x7f3e6406b160 2023-10-10T20:42:42.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.393+0000 7f3e5affd700 1 --2- 172.21.15.89:0/50667252 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3e44067320 0x7f3e440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:42.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.393+0000 7f3e6259c700 1 --2- 172.21.15.89:0/50667252 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3e44067320 0x7f3e440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:42.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.393+0000 7f3e5affd700 1 -- 172.21.15.89:0/50667252 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f3e5c085360 con 0x7f3e6406b160 2023-10-10T20:42:42.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.393+0000 7f3e6259c700 1 --2- 172.21.15.89:0/50667252 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3e44067320 0x7f3e440697d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3e54003a30 tx=0x7f3e54007ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:42.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.397+0000 7f3e5affd700 1 -- 172.21.15.89:0/50667252 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e5c051fa0 con 0x7f3e6406b160 2023-10-10T20:42:42.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:42.620+0000 7f3e68f77700 1 -- 172.21.15.89:0/50667252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3e64069740 con 0x7f3e6406b160 2023-10-10T20:42:42.910 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:42:42.913 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:42:43.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.260+0000 7f3e5affd700 1 -- 172.21.15.89:0/50667252 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v104) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3e5c0555c0 con 0x7f3e6406b160 2023-10-10T20:42:43.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.262+0000 7f3e68f77700 1 -- 172.21.15.89:0/50667252 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3e44067320 msgr2=0x7f3e440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:43.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.262+0000 7f3e68f77700 1 --2- 172.21.15.89:0/50667252 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3e44067320 0x7f3e440697d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3e54003a30 tx=0x7f3e54007ea0 comp rx=0 tx=0).stop 2023-10-10T20:42:43.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.262+0000 7f3e68f77700 1 -- 172.21.15.89:0/50667252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e6406b160 msgr2=0x7f3e64157b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:43.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.262+0000 7f3e68f77700 1 --2- 172.21.15.89:0/50667252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e6406b160 0x7f3e64157b10 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009fa0 tx=0x7f3e5c009900 comp rx=0 tx=0).stop 2023-10-10T20:42:43.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.262+0000 7f3e68f77700 1 -- 172.21.15.89:0/50667252 shutdown_connections 2023-10-10T20:42:43.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.262+0000 7f3e68f77700 1 --2- 172.21.15.89:0/50667252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e6406b160 0x7f3e64157b10 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:43.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.262+0000 7f3e68f77700 1 --2- 172.21.15.89:0/50667252 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3e44067320 0x7f3e440697d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:43.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.262+0000 7f3e68f77700 1 --2- 172.21.15.89:0/50667252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e640637d0 0x7f3e64144580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:43.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.262+0000 7f3e68f77700 1 --2- 172.21.15.89:0/50667252 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3e6406a2b0 0x7f3e64155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:43.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.262+0000 7f3e68f77700 1 -- 172.21.15.89:0/50667252 >> 172.21.15.89:0/50667252 conn(0x7f3e6406ded0 msgr2=0x7f3e64153910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:43.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.262+0000 7f3e68f77700 1 -- 172.21.15.89:0/50667252 shutdown_connections 2023-10-10T20:42:43.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.263+0000 7f3e68f77700 1 -- 172.21.15.89:0/50667252 wait complete. 2023-10-10T20:42:43.277 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:42:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.393+0000 7f03ed06b700 1 Processor -- start 2023-10-10T20:42:43.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.393+0000 7f03ed06b700 1 -- start start 2023-10-10T20:42:43.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.394+0000 7f03ed06b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f03e80cab50 0x7f03e815d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:43.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.394+0000 7f03ed06b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e806a1a0 con 0x7f03e815ab80 2023-10-10T20:42:43.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.394+0000 7f03ed06b700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e806ad40 con 0x7f03e81597a0 2023-10-10T20:42:43.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.394+0000 7f03ed06b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e80cb3a0 con 0x7f03e80cab50 2023-10-10T20:42:43.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.394+0000 7f03e759e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f03e80cab50 0x7f03e815d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:43.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.394+0000 7f03de59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f03e815ab80 0x7f03e806a630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59214/0 (socket says 172.21.15.89:59214) 2023-10-10T20:42:43.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.394+0000 7f03e759e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f03e80cab50 0x7f03e815d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39046/0 (socket says 172.21.15.89:39046) 2023-10-10T20:42:43.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.394+0000 7f03de59c700 1 -- 172.21.15.89:0/775623807 learned_addr learned my addr 172.21.15.89:0/775623807 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:43.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.395+0000 7f03e759e700 1 -- 172.21.15.89:0/775623807 >> v1:172.21.15.111:6789/0 conn(0x7f03e81597a0 legacy=0x7f03e806e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:43.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.395+0000 7f03e759e700 1 -- 172.21.15.89:0/775623807 >> v1:172.21.15.89:6789/0 conn(0x7f03e815ab80 legacy=0x7f03e806a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:43.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.395+0000 7f03e759e700 1 -- 172.21.15.89:0/775623807 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e80cb0c0 con 0x7f03e80cab50 2023-10-10T20:42:43.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.395+0000 7f03e759e700 1 --2- 172.21.15.89:0/775623807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e80cab50 0x7f03e815d550 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009e60 tx=0x7f03e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253ae285e4f07493 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:43.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.395+0000 7f03e659c700 1 -- 172.21.15.89:0/775623807 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e000e070 con 0x7f03e80cab50 2023-10-10T20:42:43.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.395+0000 7f03e659c700 1 -- 172.21.15.89:0/775623807 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e000bbf0 con 0x7f03e80cab50 2023-10-10T20:42:43.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.395+0000 7f03e659c700 1 -- 172.21.15.89:0/775623807 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0012430 con 0x7f03e80cab50 2023-10-10T20:42:43.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.395+0000 7f03ed06b700 1 -- 172.21.15.89:0/775623807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e80cab50 msgr2=0x7f03e815d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:43.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.395+0000 7f03ed06b700 1 --2- 172.21.15.89:0/775623807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e80cab50 0x7f03e815d550 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009e60 tx=0x7f03e0009780 comp rx=0 tx=0).stop 2023-10-10T20:42:43.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.395+0000 7f03ed06b700 1 -- 172.21.15.89:0/775623807 shutdown_connections 2023-10-10T20:42:43.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.395+0000 7f03ed06b700 1 --2- 172.21.15.89:0/775623807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e80cab50 0x7f03e815d550 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:43.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.395+0000 7f03ed06b700 1 -- 172.21.15.89:0/775623807 >> 172.21.15.89:0/775623807 conn(0x7f03e8068390 msgr2=0x7f03e8068790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:43.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.395+0000 7f03ed06b700 1 -- 172.21.15.89:0/775623807 shutdown_connections 2023-10-10T20:42:43.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.395+0000 7f03ed06b700 1 -- 172.21.15.89:0/775623807 wait complete. 2023-10-10T20:42:43.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.397+0000 7f03ed06b700 1 Processor -- start 2023-10-10T20:42:43.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.397+0000 7f03ed06b700 1 -- start start 2023-10-10T20:42:43.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.397+0000 7f03ed06b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03e80cab50 0x7f03e8157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:43.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.397+0000 7f03ed06b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e81597a0 0x7f03e80b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:43.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.397+0000 7f03ed06b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f03e815ab80 0x7f03e80c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:43.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.397+0000 7f03ed06b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e80b2070 con 0x7f03e81597a0 2023-10-10T20:42:43.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.397+0000 7f03ed06b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e80b1460 con 0x7f03e80cab50 2023-10-10T20:42:43.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.397+0000 7f03ed06b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e80b1740 con 0x7f03e815ab80 2023-10-10T20:42:43.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.397+0000 7f03de59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e81597a0 0x7f03e80b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:43.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.397+0000 7f03de59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e81597a0 0x7f03e80b0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39048/0 (socket says 172.21.15.89:39048) 2023-10-10T20:42:43.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.397+0000 7f03de59c700 1 -- 172.21.15.89:0/3795686115 learned_addr learned my addr 172.21.15.89:0/3795686115 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:43.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.398+0000 7f03e6d9d700 1 --2- 172.21.15.89:0/3795686115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03e80cab50 0x7f03e8157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:43.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.398+0000 7f03e759e700 1 --2- 172.21.15.89:0/3795686115 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f03e815ab80 0x7f03e80c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:43.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.398+0000 7f03de59c700 1 -- 172.21.15.89:0/3795686115 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f03e815ab80 msgr2=0x7f03e80c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:43.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.398+0000 7f03de59c700 1 --2- 172.21.15.89:0/3795686115 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f03e815ab80 0x7f03e80c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:43.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.398+0000 7f03de59c700 1 -- 172.21.15.89:0/3795686115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03e80cab50 msgr2=0x7f03e8157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:43.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.398+0000 7f03de59c700 1 --2- 172.21.15.89:0/3795686115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03e80cab50 0x7f03e8157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:43.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.398+0000 7f03de59c700 1 -- 172.21.15.89:0/3795686115 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e0009a20 con 0x7f03e81597a0 2023-10-10T20:42:43.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.398+0000 7f03de59c700 1 --2- 172.21.15.89:0/3795686115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e81597a0 0x7f03e80b0a80 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f03d000c560 tx=0x7f03d0001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:43.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.398+0000 7f03df7fe700 1 -- 172.21.15.89:0/3795686115 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0005b30 con 0x7f03e81597a0 2023-10-10T20:42:43.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.398+0000 7f03df7fe700 1 -- 172.21.15.89:0/3795686115 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d0005c90 con 0x7f03e81597a0 2023-10-10T20:42:43.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.398+0000 7f03ed06b700 1 -- 172.21.15.89:0/3795686115 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e80b1a20 con 0x7f03e81597a0 2023-10-10T20:42:43.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.398+0000 7f03df7fe700 1 -- 172.21.15.89:0/3795686115 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0007320 con 0x7f03e81597a0 2023-10-10T20:42:43.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.399+0000 7f03ed06b700 1 -- 172.21.15.89:0/3795686115 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e80b2500 con 0x7f03e81597a0 2023-10-10T20:42:43.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.399+0000 7f03ed06b700 1 -- 172.21.15.89:0/3795686115 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e80b21e0 con 0x7f03e81597a0 2023-10-10T20:42:43.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.399+0000 7f03df7fe700 1 -- 172.21.15.89:0/3795686115 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f03d001b3b0 con 0x7f03e81597a0 2023-10-10T20:42:43.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.400+0000 7f03df7fe700 1 --2- 172.21.15.89:0/3795686115 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f03c8067320 0x7f03c80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:43.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.400+0000 7f03df7fe700 1 -- 172.21.15.89:0/3795686115 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f03d0084040 con 0x7f03e81597a0 2023-10-10T20:42:43.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.400+0000 7f03e6d9d700 1 --2- 172.21.15.89:0/3795686115 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f03c8067320 0x7f03c80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:43.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.400+0000 7f03e6d9d700 1 --2- 172.21.15.89:0/3795686115 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f03c8067320 0x7f03c80697d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f03d8003a30 tx=0x7f03d8007ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:43.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.404+0000 7f03df7fe700 1 -- 172.21.15.89:0/3795686115 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03d0050c80 con 0x7f03e81597a0 2023-10-10T20:42:43.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:43.587+0000 7f03ed06b700 1 -- 172.21.15.89:0/3795686115 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f03e80718b0 con 0x7f03e81597a0 2023-10-10T20:42:44.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.262+0000 7f03df7fe700 1 -- 172.21.15.89:0/3795686115 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v155) v1 ==== 181+0+0 (secure 0 0 0) 0x7f03d00170d0 con 0x7f03e81597a0 2023-10-10T20:42:44.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.264+0000 7f03ed06b700 1 -- 172.21.15.89:0/3795686115 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f03c8067320 msgr2=0x7f03c80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:44.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.264+0000 7f03ed06b700 1 --2- 172.21.15.89:0/3795686115 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f03c8067320 0x7f03c80697d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f03d8003a30 tx=0x7f03d8007ea0 comp rx=0 tx=0).stop 2023-10-10T20:42:44.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.264+0000 7f03ed06b700 1 -- 172.21.15.89:0/3795686115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e81597a0 msgr2=0x7f03e80b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:44.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.264+0000 7f03ed06b700 1 --2- 172.21.15.89:0/3795686115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e81597a0 0x7f03e80b0a80 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f03d000c560 tx=0x7f03d0001020 comp rx=0 tx=0).stop 2023-10-10T20:42:44.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.264+0000 7f03ed06b700 1 -- 172.21.15.89:0/3795686115 shutdown_connections 2023-10-10T20:42:44.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.264+0000 7f03ed06b700 1 --2- 172.21.15.89:0/3795686115 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f03e815ab80 0x7f03e80c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:44.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.264+0000 7f03ed06b700 1 --2- 172.21.15.89:0/3795686115 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f03c8067320 0x7f03c80697d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:44.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.264+0000 7f03ed06b700 1 --2- 172.21.15.89:0/3795686115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03e80cab50 0x7f03e8157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:44.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.264+0000 7f03ed06b700 1 --2- 172.21.15.89:0/3795686115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e81597a0 0x7f03e80b0a80 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:44.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.264+0000 7f03ed06b700 1 -- 172.21.15.89:0/3795686115 >> 172.21.15.89:0/3795686115 conn(0x7f03e8068390 msgr2=0x7f03e806d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:44.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.264+0000 7f03ed06b700 1 -- 172.21.15.89:0/3795686115 shutdown_connections 2023-10-10T20:42:44.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.264+0000 7f03ed06b700 1 -- 172.21.15.89:0/3795686115 wait complete. 2023-10-10T20:42:44.269 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:42:44.281 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:42:44.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.399+0000 7f8af90b8700 1 Processor -- start 2023-10-10T20:42:44.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.399+0000 7f8af90b8700 1 -- start start 2023-10-10T20:42:44.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.400+0000 7f8af90b8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8af4159ec0 0x7f8af40b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:44.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.400+0000 7f8af90b8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8af40b53d0 0x7f8af40cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:44.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.400+0000 7f8af90b8700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af40b5b00 con 0x7f8af40cce40 2023-10-10T20:42:44.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.400+0000 7f8af90b8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af40b5da0 con 0x7f8af40b53d0 2023-10-10T20:42:44.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.400+0000 7f8af90b8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af40b6080 con 0x7f8af4159ec0 2023-10-10T20:42:44.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.400+0000 7f8af259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8af40b53d0 0x7f8af40cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:44.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.400+0000 7f8af259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8af40b53d0 0x7f8af40cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39062/0 (socket says 172.21.15.89:39062) 2023-10-10T20:42:44.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.400+0000 7f8af259c700 1 -- 172.21.15.89:0/2720665895 learned_addr learned my addr 172.21.15.89:0/2720665895 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:44.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.400+0000 7f8af2d9d700 1 --2- 172.21.15.89:0/2720665895 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8af4159ec0 0x7f8af40b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:44.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.400+0000 7f8af259c700 1 -- 172.21.15.89:0/2720665895 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8af4159ec0 msgr2=0x7f8af40b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:44.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.400+0000 7f8af259c700 1 --2- 172.21.15.89:0/2720665895 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8af4159ec0 0x7f8af40b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:44.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.400+0000 7f8af259c700 1 -- 172.21.15.89:0/2720665895 >> v1:172.21.15.111:6789/0 conn(0x7f8af40cce40 legacy=0x7f8af40cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:44.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.400+0000 7f8af259c700 1 -- 172.21.15.89:0/2720665895 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af40cc100 con 0x7f8af40b53d0 2023-10-10T20:42:44.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.400+0000 7f8af259c700 1 --2- 172.21.15.89:0/2720665895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af40b53d0 0x7f8af40cc900 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f8adc0099b0 tx=0x7f8adc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76875de7885b723 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:44.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.400+0000 7f8af1d9b700 1 -- 172.21.15.89:0/2720665895 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc00b860 con 0x7f8af40b53d0 2023-10-10T20:42:44.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.400+0000 7f8af1d9b700 1 -- 172.21.15.89:0/2720665895 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8adc00b9c0 con 0x7f8af40b53d0 2023-10-10T20:42:44.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.400+0000 7f8af1d9b700 1 -- 172.21.15.89:0/2720665895 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc00bd60 con 0x7f8af40b53d0 2023-10-10T20:42:44.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.401+0000 7f8af90b8700 1 -- 172.21.15.89:0/2720665895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af40b53d0 msgr2=0x7f8af40cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:44.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.401+0000 7f8af90b8700 1 --2- 172.21.15.89:0/2720665895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af40b53d0 0x7f8af40cc900 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f8adc0099b0 tx=0x7f8adc009250 comp rx=0 tx=0).stop 2023-10-10T20:42:44.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.401+0000 7f8af90b8700 1 -- 172.21.15.89:0/2720665895 shutdown_connections 2023-10-10T20:42:44.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.401+0000 7f8af90b8700 1 --2- 172.21.15.89:0/2720665895 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8af4159ec0 0x7f8af40b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:44.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.401+0000 7f8af90b8700 1 --2- 172.21.15.89:0/2720665895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af40b53d0 0x7f8af40cc900 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:44.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.401+0000 7f8af90b8700 1 -- 172.21.15.89:0/2720665895 >> 172.21.15.89:0/2720665895 conn(0x7f8af40999f0 msgr2=0x7f8af40b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:44.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.401+0000 7f8af90b8700 1 -- 172.21.15.89:0/2720665895 shutdown_connections 2023-10-10T20:42:44.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.401+0000 7f8af90b8700 1 -- 172.21.15.89:0/2720665895 wait complete. 2023-10-10T20:42:44.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.402+0000 7f8af90b8700 1 Processor -- start 2023-10-10T20:42:44.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.402+0000 7f8af90b8700 1 -- start start 2023-10-10T20:42:44.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.403+0000 7f8af90b8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8af40b53d0 0x7f8af409ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:44.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.403+0000 7f8af90b8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8af40cce40 0x7f8af4157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:44.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.403+0000 7f8af90b8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af4159ec0 0x7f8af409d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:44.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.403+0000 7f8af90b8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af4157c80 con 0x7f8af4159ec0 2023-10-10T20:42:44.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.403+0000 7f8af90b8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af409bd00 con 0x7f8af40cce40 2023-10-10T20:42:44.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.403+0000 7f8af90b8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af409b0f0 con 0x7f8af40b53d0 2023-10-10T20:42:44.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.403+0000 7f8af359e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af4159ec0 0x7f8af409d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:44.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.403+0000 7f8af359e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af4159ec0 0x7f8af409d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39074/0 (socket says 172.21.15.89:39074) 2023-10-10T20:42:44.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.403+0000 7f8af359e700 1 -- 172.21.15.89:0/1284278830 learned_addr learned my addr 172.21.15.89:0/1284278830 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:44.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.403+0000 7f8af259c700 1 --2- 172.21.15.89:0/1284278830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8af40cce40 0x7f8af4157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:44.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.403+0000 7f8af2d9d700 1 --2- 172.21.15.89:0/1284278830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8af40b53d0 0x7f8af409ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:44.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.403+0000 7f8af359e700 1 -- 172.21.15.89:0/1284278830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8af40b53d0 msgr2=0x7f8af409ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.403+0000 7f8af359e700 1 --2- 172.21.15.89:0/1284278830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8af40b53d0 0x7f8af409ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.403+0000 7f8af359e700 1 -- 172.21.15.89:0/1284278830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8af40cce40 msgr2=0x7f8af4157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.403+0000 7f8af359e700 1 --2- 172.21.15.89:0/1284278830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8af40cce40 0x7f8af4157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.403+0000 7f8af359e700 1 -- 172.21.15.89:0/1284278830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc0094f0 con 0x7f8af4159ec0 2023-10-10T20:42:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.403+0000 7f8af359e700 1 --2- 172.21.15.89:0/1284278830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af4159ec0 0x7f8af409d9e0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400c560 tx=0x7f8ae4007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:44.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.403+0000 7f8aeaffd700 1 -- 172.21.15.89:0/1284278830 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4011590 con 0x7f8af4159ec0 2023-10-10T20:42:44.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.404+0000 7f8aeaffd700 1 -- 172.21.15.89:0/1284278830 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae40116f0 con 0x7f8af4159ec0 2023-10-10T20:42:44.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.404+0000 7f8af90b8700 1 -- 172.21.15.89:0/1284278830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af409b3d0 con 0x7f8af4159ec0 2023-10-10T20:42:44.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.404+0000 7f8aeaffd700 1 -- 172.21.15.89:0/1284278830 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4011ab0 con 0x7f8af4159ec0 2023-10-10T20:42:44.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.404+0000 7f8af90b8700 1 -- 172.21.15.89:0/1284278830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af409b990 con 0x7f8af4159ec0 2023-10-10T20:42:44.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.404+0000 7f8af90b8700 1 -- 172.21.15.89:0/1284278830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af40cabb0 con 0x7f8af4159ec0 2023-10-10T20:42:44.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.405+0000 7f8aeaffd700 1 -- 172.21.15.89:0/1284278830 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8ae401b3b0 con 0x7f8af4159ec0 2023-10-10T20:42:44.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.405+0000 7f8aeaffd700 1 --2- 172.21.15.89:0/1284278830 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8ad4067320 0x7f8ad40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:44.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.405+0000 7f8af2d9d700 1 --2- 172.21.15.89:0/1284278830 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8ad4067320 0x7f8ad40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:44.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.405+0000 7f8aeaffd700 1 -- 172.21.15.89:0/1284278830 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f8ae4083e80 con 0x7f8af4159ec0 2023-10-10T20:42:44.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.406+0000 7f8af2d9d700 1 --2- 172.21.15.89:0/1284278830 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8ad4067320 0x7f8ad40697d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8af4063840 tx=0x7f8aec009800 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:44.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.410+0000 7f8aeaffd700 1 -- 172.21.15.89:0/1284278830 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ae4050c70 con 0x7f8af4159ec0 2023-10-10T20:42:44.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:44.595+0000 7f8af90b8700 1 -- 172.21.15.89:0/1284278830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8af4063c90 con 0x7f8af4159ec0 2023-10-10T20:42:45.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.273+0000 7f8aeaffd700 1 -- 172.21.15.89:0/1284278830 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v156) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8ae4017070 con 0x7f8af4159ec0 2023-10-10T20:42:45.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.277+0000 7f8af90b8700 1 -- 172.21.15.89:0/1284278830 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8ad4067320 msgr2=0x7f8ad40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:45.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.277+0000 7f8af90b8700 1 --2- 172.21.15.89:0/1284278830 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8ad4067320 0x7f8ad40697d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8af4063840 tx=0x7f8aec009800 comp rx=0 tx=0).stop 2023-10-10T20:42:45.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.277+0000 7f8af90b8700 1 -- 172.21.15.89:0/1284278830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af4159ec0 msgr2=0x7f8af409d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:45.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.277+0000 7f8af90b8700 1 --2- 172.21.15.89:0/1284278830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af4159ec0 0x7f8af409d9e0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400c560 tx=0x7f8ae4007ef0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.277+0000 7f8af90b8700 1 -- 172.21.15.89:0/1284278830 shutdown_connections 2023-10-10T20:42:45.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.277+0000 7f8af90b8700 1 --2- 172.21.15.89:0/1284278830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af4159ec0 0x7f8af409d9e0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.277+0000 7f8af90b8700 1 --2- 172.21.15.89:0/1284278830 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8af40b53d0 0x7f8af409ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.277+0000 7f8af90b8700 1 --2- 172.21.15.89:0/1284278830 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8ad4067320 0x7f8ad40697d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.277+0000 7f8af90b8700 1 --2- 172.21.15.89:0/1284278830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8af40cce40 0x7f8af4157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.277+0000 7f8af90b8700 1 -- 172.21.15.89:0/1284278830 >> 172.21.15.89:0/1284278830 conn(0x7f8af40999f0 msgr2=0x7f8af40ce320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:45.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.277+0000 7f8af90b8700 1 -- 172.21.15.89:0/1284278830 shutdown_connections 2023-10-10T20:42:45.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.277+0000 7f8af90b8700 1 -- 172.21.15.89:0/1284278830 wait complete. 2023-10-10T20:42:45.281 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' removed 2023-10-10T20:42:45.292 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:42:45.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.406+0000 7f0743489700 1 Processor -- start 2023-10-10T20:42:45.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.406+0000 7f0743489700 1 -- start start 2023-10-10T20:42:45.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.407+0000 7f0743489700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f073c159890 0x7f073c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:45.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.407+0000 7f0743489700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f073c159c60 0x7f073c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:45.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.407+0000 7f0743489700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f073c15db10 0x7f073c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:45.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.407+0000 7f0743489700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f073c15a160 con 0x7f073c15db10 2023-10-10T20:42:45.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.407+0000 7f0743489700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f073c0d13a0 con 0x7f073c159890 2023-10-10T20:42:45.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.407+0000 7f0743489700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073c15e140 con 0x7f073c159c60 2023-10-10T20:42:45.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.407+0000 7f0741a26700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f073c15db10 0x7f073c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:45.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.408+0000 7f0741a26700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f073c15db10 0x7f073c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39080/0 (socket says 172.21.15.89:39080) 2023-10-10T20:42:45.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.408+0000 7f0741a26700 1 -- 172.21.15.89:0/1047458428 learned_addr learned my addr 172.21.15.89:0/1047458428 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:45.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.408+0000 7f0740a24700 1 --2- 172.21.15.89:0/1047458428 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f073c159c60 0x7f073c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:45.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.408+0000 7f0741225700 1 --2- 172.21.15.89:0/1047458428 >> v2:172.21.15.111:3300/0 conn(0x7f073c159890 0x7f073c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:45.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.408+0000 7f0741a26700 1 -- 172.21.15.89:0/1047458428 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f073c159c60 msgr2=0x7f073c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:45.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.408+0000 7f0741a26700 1 --2- 172.21.15.89:0/1047458428 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f073c159c60 0x7f073c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.408+0000 7f0741a26700 1 -- 172.21.15.89:0/1047458428 >> v2:172.21.15.111:3300/0 conn(0x7f073c159890 msgr2=0x7f073c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:45.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.408+0000 7f0741a26700 1 --2- 172.21.15.89:0/1047458428 >> v2:172.21.15.111:3300/0 conn(0x7f073c159890 0x7f073c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.408+0000 7f0741a26700 1 -- 172.21.15.89:0/1047458428 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073c15a3d0 con 0x7f073c15db10 2023-10-10T20:42:45.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.408+0000 7f0741225700 1 --2- 172.21.15.89:0/1047458428 >> v2:172.21.15.111:3300/0 conn(0x7f073c159890 0x7f073c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:45.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.408+0000 7f0740a24700 1 --2- 172.21.15.89:0/1047458428 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f073c159c60 0x7f073c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:45.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.408+0000 7f0741a26700 1 --2- 172.21.15.89:0/1047458428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f073c15db10 0x7f073c069560 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f073c0d1090 tx=0x7f0734009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3dc37162ddbfaa2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:45.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.408+0000 7f0733fff700 1 -- 172.21.15.89:0/1047458428 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073400b860 con 0x7f073c15db10 2023-10-10T20:42:45.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.408+0000 7f0733fff700 1 -- 172.21.15.89:0/1047458428 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073400b9c0 con 0x7f073c15db10 2023-10-10T20:42:45.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.408+0000 7f0733fff700 1 -- 172.21.15.89:0/1047458428 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073400bd60 con 0x7f073c15db10 2023-10-10T20:42:45.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.409+0000 7f0743489700 1 -- 172.21.15.89:0/1047458428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f073c15db10 msgr2=0x7f073c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:45.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.409+0000 7f0743489700 1 --2- 172.21.15.89:0/1047458428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f073c15db10 0x7f073c069560 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f073c0d1090 tx=0x7f0734009250 comp rx=0 tx=0).stop 2023-10-10T20:42:45.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.409+0000 7f0743489700 1 -- 172.21.15.89:0/1047458428 shutdown_connections 2023-10-10T20:42:45.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.409+0000 7f0743489700 1 --2- 172.21.15.89:0/1047458428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f073c15db10 0x7f073c069560 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.409+0000 7f0743489700 1 --2- 172.21.15.89:0/1047458428 >> v2:172.21.15.111:3300/0 conn(0x7f073c159890 0x7f073c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.409+0000 7f0743489700 1 --2- 172.21.15.89:0/1047458428 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f073c159c60 0x7f073c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.409+0000 7f0743489700 1 -- 172.21.15.89:0/1047458428 >> 172.21.15.89:0/1047458428 conn(0x7f073c068e20 msgr2=0x7f073c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:45.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.409+0000 7f0743489700 1 -- 172.21.15.89:0/1047458428 shutdown_connections 2023-10-10T20:42:45.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.409+0000 7f0743489700 1 -- 172.21.15.89:0/1047458428 wait complete. 2023-10-10T20:42:45.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.410+0000 7f0743489700 1 Processor -- start 2023-10-10T20:42:45.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.410+0000 7f0743489700 1 -- start start 2023-10-10T20:42:45.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f0743489700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f073c159890 0x7f073c148060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:45.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f0743489700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f073c159c60 0x7f073c148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:45.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f0743489700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f073c15db10 0x7f073c148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:45.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f0743489700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c149930 con 0x7f073c159c60 2023-10-10T20:42:45.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f0743489700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c155ef0 con 0x7f073c159890 2023-10-10T20:42:45.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f0743489700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073c149370 con 0x7f073c15db10 2023-10-10T20:42:45.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f0740a24700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f073c159c60 0x7f073c148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:45.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f0740a24700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f073c159c60 0x7f073c148800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39092/0 (socket says 172.21.15.89:39092) 2023-10-10T20:42:45.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f0740a24700 1 -- 172.21.15.89:0/3523512111 learned_addr learned my addr 172.21.15.89:0/3523512111 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:45.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f0741225700 1 --2- 172.21.15.89:0/3523512111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f073c159890 0x7f073c148060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:45.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f0741a26700 1 --2- 172.21.15.89:0/3523512111 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f073c15db10 0x7f073c148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:45.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f0740a24700 1 -- 172.21.15.89:0/3523512111 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f073c15db10 msgr2=0x7f073c148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:45.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f0740a24700 1 --2- 172.21.15.89:0/3523512111 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f073c15db10 0x7f073c148d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f0740a24700 1 -- 172.21.15.89:0/3523512111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f073c159890 msgr2=0x7f073c148060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:45.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f0740a24700 1 --2- 172.21.15.89:0/3523512111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f073c159890 0x7f073c148060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f0740a24700 1 -- 172.21.15.89:0/3523512111 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07340094f0 con 0x7f073c159c60 2023-10-10T20:42:45.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f0740a24700 1 --2- 172.21.15.89:0/3523512111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f073c159c60 0x7f073c148800 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f072c00c880 tx=0x7f072c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:45.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f07317fa700 1 -- 172.21.15.89:0/3523512111 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c011600 con 0x7f073c159c60 2023-10-10T20:42:45.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f07317fa700 1 -- 172.21.15.89:0/3523512111 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072c011760 con 0x7f073c159c60 2023-10-10T20:42:45.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f07317fa700 1 -- 172.21.15.89:0/3523512111 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c011ae0 con 0x7f073c159c60 2023-10-10T20:42:45.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f0743489700 1 -- 172.21.15.89:0/3523512111 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073c149650 con 0x7f073c159c60 2023-10-10T20:42:45.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.411+0000 7f0743489700 1 -- 172.21.15.89:0/3523512111 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073c0b0e60 con 0x7f073c159c60 2023-10-10T20:42:45.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.412+0000 7f0743489700 1 -- 172.21.15.89:0/3523512111 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073c0b1080 con 0x7f073c159c60 2023-10-10T20:42:45.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.413+0000 7f07317fa700 1 -- 172.21.15.89:0/3523512111 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f072c0193b0 con 0x7f073c159c60 2023-10-10T20:42:45.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.413+0000 7f07317fa700 1 --2- 172.21.15.89:0/3523512111 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0720067320 0x7f07200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:45.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.413+0000 7f07317fa700 1 -- 172.21.15.89:0/3523512111 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f072c084a20 con 0x7f073c159c60 2023-10-10T20:42:45.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.413+0000 7f0741225700 1 --2- 172.21.15.89:0/3523512111 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0720067320 0x7f07200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:45.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.414+0000 7f0741225700 1 --2- 172.21.15.89:0/3523512111 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0720067320 0x7f07200697d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0738007fd0 tx=0x7f07380077c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:45.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.417+0000 7f07317fa700 1 -- 172.21.15.89:0/3523512111 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f072c0518a0 con 0x7f073c159c60 2023-10-10T20:42:45.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.600+0000 7f0743489700 1 -- 172.21.15.89:0/3523512111 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f073c04c3c0 con 0x7f073c159c60 2023-10-10T20:42:45.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.604+0000 7f07317fa700 1 -- 172.21.15.89:0/3523512111 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f072c054ec0 con 0x7f073c159c60 2023-10-10T20:42:45.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.606+0000 7f0743489700 1 -- 172.21.15.89:0/3523512111 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0720067320 msgr2=0x7f07200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:45.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.606+0000 7f0743489700 1 --2- 172.21.15.89:0/3523512111 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0720067320 0x7f07200697d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0738007fd0 tx=0x7f07380077c0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.606+0000 7f0743489700 1 -- 172.21.15.89:0/3523512111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f073c159c60 msgr2=0x7f073c148800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:45.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.606+0000 7f0743489700 1 --2- 172.21.15.89:0/3523512111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f073c159c60 0x7f073c148800 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f072c00c880 tx=0x7f072c008040 comp rx=0 tx=0).stop 2023-10-10T20:42:45.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.606+0000 7f0743489700 1 -- 172.21.15.89:0/3523512111 shutdown_connections 2023-10-10T20:42:45.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.606+0000 7f0743489700 1 --2- 172.21.15.89:0/3523512111 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f073c15db10 0x7f073c148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.606+0000 7f0743489700 1 --2- 172.21.15.89:0/3523512111 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0720067320 0x7f07200697d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.606+0000 7f0743489700 1 --2- 172.21.15.89:0/3523512111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f073c159890 0x7f073c148060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.606+0000 7f0743489700 1 --2- 172.21.15.89:0/3523512111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f073c159c60 0x7f073c148800 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.606+0000 7f0743489700 1 -- 172.21.15.89:0/3523512111 >> 172.21.15.89:0/3523512111 conn(0x7f073c068e20 msgr2=0x7f073c149b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:45.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.606+0000 7f0743489700 1 -- 172.21.15.89:0/3523512111 shutdown_connections 2023-10-10T20:42:45.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.606+0000 7f0743489700 1 -- 172.21.15.89:0/3523512111 wait complete. 2023-10-10T20:42:45.619 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag' 2023-10-10T20:42:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.733+0000 7f7e25697700 1 Processor -- start 2023-10-10T20:42:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.733+0000 7f7e25697700 1 -- start start 2023-10-10T20:42:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.733+0000 7f7e25697700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e20159890 0x7f7e2015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.733+0000 7f7e25697700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7e20159c60 0x7f7e2015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.733+0000 7f7e25697700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2015a160 con 0x7f7e2015db10 2023-10-10T20:42:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.733+0000 7f7e25697700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e200d13a0 con 0x7f7e20159c60 2023-10-10T20:42:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.734+0000 7f7e25697700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2015e140 con 0x7f7e20159890 2023-10-10T20:42:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.734+0000 7f7e167fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7e20159c60 0x7f7e2015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.734+0000 7f7e1f7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7e2015db10 0x7f7e20069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59222/0 (socket says 172.21.15.89:59222) 2023-10-10T20:42:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.734+0000 7f7e1f7fe700 1 -- 172.21.15.89:0/3170220647 learned_addr learned my addr 172.21.15.89:0/3170220647 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.734+0000 7f7e1effd700 1 --2- 172.21.15.89:0/3170220647 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e20159890 0x7f7e2015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.734+0000 7f7e167fc700 1 -- 172.21.15.89:0/3170220647 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e20159890 msgr2=0x7f7e2015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.734+0000 7f7e167fc700 1 --2- 172.21.15.89:0/3170220647 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e20159890 0x7f7e2015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.734+0000 7f7e167fc700 1 -- 172.21.15.89:0/3170220647 >> v1:172.21.15.89:6789/0 conn(0x7f7e2015db10 legacy=0x7f7e20069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.734+0000 7f7e1e7fc700 1 -- 172.21.15.89:0/3170220647 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2297845512 0 0) 0x7f7e2015a160 con 0x7f7e2015db10 2023-10-10T20:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.734+0000 7f7e167fc700 1 -- 172.21.15.89:0/3170220647 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2015a3d0 con 0x7f7e20159c60 2023-10-10T20:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.734+0000 7f7e1effd700 1 --2- 172.21.15.89:0/3170220647 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e20159890 0x7f7e2015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.734+0000 7f7e167fc700 1 --2- 172.21.15.89:0/3170220647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e20159c60 0x7f7e2015d5d0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f7e080098e0 tx=0x7f7e080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886719bb2ffa7883 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.734+0000 7f7e1e7fc700 1 -- 172.21.15.89:0/3170220647 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e08009e80 con 0x7f7e20159c60 2023-10-10T20:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.734+0000 7f7e1e7fc700 1 -- 172.21.15.89:0/3170220647 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e0800b860 con 0x7f7e20159c60 2023-10-10T20:42:45.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.734+0000 7f7e1e7fc700 1 -- 172.21.15.89:0/3170220647 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0800bbe0 con 0x7f7e20159c60 2023-10-10T20:42:45.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.735+0000 7f7e25697700 1 -- 172.21.15.89:0/3170220647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e20159c60 msgr2=0x7f7e2015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:45.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.735+0000 7f7e25697700 1 --2- 172.21.15.89:0/3170220647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e20159c60 0x7f7e2015d5d0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f7e080098e0 tx=0x7f7e080091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.735+0000 7f7e25697700 1 -- 172.21.15.89:0/3170220647 shutdown_connections 2023-10-10T20:42:45.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.735+0000 7f7e25697700 1 --2- 172.21.15.89:0/3170220647 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e20159890 0x7f7e2015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.735+0000 7f7e25697700 1 --2- 172.21.15.89:0/3170220647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e20159c60 0x7f7e2015d5d0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.735+0000 7f7e25697700 1 -- 172.21.15.89:0/3170220647 >> 172.21.15.89:0/3170220647 conn(0x7f7e20068e20 msgr2=0x7f7e200d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.735+0000 7f7e25697700 1 -- 172.21.15.89:0/3170220647 shutdown_connections 2023-10-10T20:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.735+0000 7f7e25697700 1 -- 172.21.15.89:0/3170220647 wait complete. 2023-10-10T20:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.736+0000 7f7e25697700 1 Processor -- start 2023-10-10T20:42:45.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.736+0000 7f7e25697700 1 -- start start 2023-10-10T20:42:45.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e25697700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e20159890 0x7f7e200b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:45.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e25697700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e20159c60 0x7f7e2014d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e25697700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e2015db10 0x7f7e2015c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e25697700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e201519e0 con 0x7f7e2015db10 2023-10-10T20:42:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e25697700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2015ccb0 con 0x7f7e20159890 2023-10-10T20:42:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e25697700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e20151390 con 0x7f7e20159c60 2023-10-10T20:42:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e1f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e2015db10 0x7f7e2015c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:45.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e1f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e2015db10 0x7f7e2015c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39112/0 (socket says 172.21.15.89:39112) 2023-10-10T20:42:45.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e1f7fe700 1 -- 172.21.15.89:0/775110864 learned_addr learned my addr 172.21.15.89:0/775110864 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:45.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e167fc700 1 --2- 172.21.15.89:0/775110864 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e20159c60 0x7f7e2014d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:45.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e1effd700 1 --2- 172.21.15.89:0/775110864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e20159890 0x7f7e200b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:45.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e1f7fe700 1 -- 172.21.15.89:0/775110864 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e20159c60 msgr2=0x7f7e2014d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:45.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e1f7fe700 1 --2- 172.21.15.89:0/775110864 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e20159c60 0x7f7e2014d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e1f7fe700 1 -- 172.21.15.89:0/775110864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e20159890 msgr2=0x7f7e200b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:45.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e1f7fe700 1 --2- 172.21.15.89:0/775110864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e20159890 0x7f7e200b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:45.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e1f7fe700 1 -- 172.21.15.89:0/775110864 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e08009420 con 0x7f7e2015db10 2023-10-10T20:42:45.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e167fc700 1 --2- 172.21.15.89:0/775110864 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e20159c60 0x7f7e2014d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:45.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e1effd700 1 --2- 172.21.15.89:0/775110864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e20159890 0x7f7e200b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:45.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e1f7fe700 1 --2- 172.21.15.89:0/775110864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e2015db10 0x7f7e2015c1f0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800cbf0 tx=0x7f7e180019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:45.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e177fe700 1 -- 172.21.15.89:0/775110864 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1800a9c0 con 0x7f7e2015db10 2023-10-10T20:42:45.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e177fe700 1 -- 172.21.15.89:0/775110864 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1800ab20 con 0x7f7e2015db10 2023-10-10T20:42:45.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e25697700 1 -- 172.21.15.89:0/775110864 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e20151670 con 0x7f7e2015db10 2023-10-10T20:42:45.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e177fe700 1 -- 172.21.15.89:0/775110864 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e180070f0 con 0x7f7e2015db10 2023-10-10T20:42:45.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.737+0000 7f7e25697700 1 -- 172.21.15.89:0/775110864 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2015cf90 con 0x7f7e2015db10 2023-10-10T20:42:45.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.738+0000 7f7e25697700 1 -- 172.21.15.89:0/775110864 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e200b2ac0 con 0x7f7e2015db10 2023-10-10T20:42:45.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.739+0000 7f7e177fe700 1 -- 172.21.15.89:0/775110864 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7e18007520 con 0x7f7e2015db10 2023-10-10T20:42:45.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.739+0000 7f7e177fe700 1 --2- 172.21.15.89:0/775110864 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7e00067320 0x7f7e000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.739+0000 7f7e177fe700 1 -- 172.21.15.89:0/775110864 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7e18084d50 con 0x7f7e2015db10 2023-10-10T20:42:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.739+0000 7f7e1effd700 1 --2- 172.21.15.89:0/775110864 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7e00067320 0x7f7e000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.740+0000 7f7e1effd700 1 --2- 172.21.15.89:0/775110864 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7e00067320 0x7f7e000697d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000ac10 tx=0x7f7e10006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:45.743+0000 7f7e177fe700 1 -- 172.21.15.89:0/775110864 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e18051d60 con 0x7f7e2015db10 2023-10-10T20:42:46.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.003+0000 7f7e25697700 1 -- 172.21.15.89:0/775110864 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]} v 0) v1 -- 0x7f7e2004c3c0 con 0x7f7e2015db10 2023-10-10T20:42:46.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.274+0000 7f7e177fe700 1 -- 172.21.15.89:0/775110864 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]}]=0 v411) v1 ==== 167+0+0 (secure 0 0 0) 0x7f7e18055380 con 0x7f7e2015db10 2023-10-10T20:42:46.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.277+0000 7f7e25697700 1 -- 172.21.15.89:0/775110864 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7e00067320 msgr2=0x7f7e000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:46.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.277+0000 7f7e25697700 1 --2- 172.21.15.89:0/775110864 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7e00067320 0x7f7e000697d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000ac10 tx=0x7f7e10006f90 comp rx=0 tx=0).stop 2023-10-10T20:42:46.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.277+0000 7f7e25697700 1 -- 172.21.15.89:0/775110864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e2015db10 msgr2=0x7f7e2015c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:46.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.277+0000 7f7e25697700 1 --2- 172.21.15.89:0/775110864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e2015db10 0x7f7e2015c1f0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800cbf0 tx=0x7f7e180019b0 comp rx=0 tx=0).stop 2023-10-10T20:42:46.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.277+0000 7f7e25697700 1 -- 172.21.15.89:0/775110864 shutdown_connections 2023-10-10T20:42:46.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.277+0000 7f7e25697700 1 --2- 172.21.15.89:0/775110864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e2015db10 0x7f7e2015c1f0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:46.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.277+0000 7f7e25697700 1 --2- 172.21.15.89:0/775110864 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7e00067320 0x7f7e000697d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:46.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.277+0000 7f7e25697700 1 --2- 172.21.15.89:0/775110864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e20159890 0x7f7e200b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:46.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.277+0000 7f7e25697700 1 --2- 172.21.15.89:0/775110864 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7e20159c60 0x7f7e2014d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:46.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.277+0000 7f7e25697700 1 -- 172.21.15.89:0/775110864 >> 172.21.15.89:0/775110864 conn(0x7f7e20068e20 msgr2=0x7f7e20158a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:46.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.277+0000 7f7e25697700 1 -- 172.21.15.89:0/775110864 shutdown_connections 2023-10-10T20:42:46.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.277+0000 7f7e25697700 1 -- 172.21.15.89:0/775110864 wait complete. 2023-10-10T20:42:46.291 INFO:tasks.cephfs_test_runner:That renaming volume fails without --yes-i-really-mean-it flag. ... ok 2023-10-10T20:42:46.292 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename_for_more_than_one_data_pool (tasks.cephfs.test_volumes.TestVolumes) That renaming a volume with more than one data pool does not change 2023-10-10T20:42:46.292 INFO:tasks.cephfs_test_runner:test_volume_rename_for_more_than_one_data_pool (tasks.cephfs.test_volumes.TestVolumes) 2023-10-10T20:42:46.293 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool' 2023-10-10T20:42:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.409+0000 7fa130bb1700 1 Processor -- start 2023-10-10T20:42:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.409+0000 7fa130bb1700 1 -- start start 2023-10-10T20:42:46.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.409+0000 7fa130bb1700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa12c069bb0 0x7fa12c1645f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:46.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.409+0000 7fa130bb1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa12c0cfb30 con 0x7fa12c159840 2023-10-10T20:42:46.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.409+0000 7fa130bb1700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa12c0cbc40 con 0x7fa12c159c10 2023-10-10T20:42:46.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.409+0000 7fa130bb1700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa12c0cbf10 con 0x7fa12c069bb0 2023-10-10T20:42:46.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa12a59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa12c159840 0x7fa12c0cb530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59238/0 (socket says 172.21.15.89:59238) 2023-10-10T20:42:46.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa12a59c700 1 -- 172.21.15.89:0/1970615206 learned_addr learned my addr 172.21.15.89:0/1970615206 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:46.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa129d9b700 1 -- 172.21.15.89:0/1970615206 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728703722 0 0) 0x7fa12c0cfb30 con 0x7fa12c159840 2023-10-10T20:42:46.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa129d9b700 1 -- 172.21.15.89:0/1970615206 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa118003570 con 0x7fa12c159840 2023-10-10T20:42:46.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa12ad9d700 1 --2- 172.21.15.89:0/1970615206 >> v2:172.21.15.111:3300/0 conn(0x7fa12c069bb0 0x7fa12c1645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:46.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa129d9b700 1 -- 172.21.15.89:0/1970615206 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3606722443 0 0) 0x7fa118003570 con 0x7fa12c159840 2023-10-10T20:42:46.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa129d9b700 1 -- 172.21.15.89:0/1970615206 >> v2:172.21.15.111:3300/0 conn(0x7fa12c069bb0 msgr2=0x7fa12c1645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:46.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa129d9b700 1 --2- 172.21.15.89:0/1970615206 >> v2:172.21.15.111:3300/0 conn(0x7fa12c069bb0 0x7fa12c1645f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:46.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa129d9b700 1 -- 172.21.15.89:0/1970615206 >> v1:172.21.15.111:6789/0 conn(0x7fa12c159c10 legacy=0x7fa12c0694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:46.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa129d9b700 1 -- 172.21.15.89:0/1970615206 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c0cf4e0 con 0x7fa12c159840 2023-10-10T20:42:46.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa129d9b700 1 -- 172.21.15.89:0/1970615206 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fa11c001470 con 0x7fa12c159840 2023-10-10T20:42:46.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa129d9b700 1 -- 172.21.15.89:0/1970615206 >> v1:172.21.15.89:6789/0 conn(0x7fa12c159840 legacy=0x7fa12c0cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:46.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa129d9b700 1 --2- 172.21.15.89:0/1970615206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa118003a90 0x7fa118003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:46.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa129d9b700 1 --2- 172.21.15.89:0/1970615206 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa118004810 0x7fa118004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:46.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa129d9b700 1 --2- 172.21.15.89:0/1970615206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa118005210 0x7fa11800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:46.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa129d9b700 1 -- 172.21.15.89:0/1970615206 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c0cbf10 con 0x7fa118005210 2023-10-10T20:42:46.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa129d9b700 1 -- 172.21.15.89:0/1970615206 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11800c1b0 con 0x7fa118003a90 2023-10-10T20:42:46.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa129d9b700 1 -- 172.21.15.89:0/1970615206 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11800c370 con 0x7fa118004810 2023-10-10T20:42:46.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa12ad9d700 1 --2- 172.21.15.89:0/1970615206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa118005210 0x7fa11800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:46.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa12a59c700 1 --2- 172.21.15.89:0/1970615206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa118003a90 0x7fa118003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:46.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.410+0000 7fa123fff700 1 --2- 172.21.15.89:0/1970615206 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa118004810 0x7fa118004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:46.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.411+0000 7fa12ad9d700 1 -- 172.21.15.89:0/1970615206 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa118004810 msgr2=0x7fa118004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:46.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.411+0000 7fa12ad9d700 1 --2- 172.21.15.89:0/1970615206 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa118004810 0x7fa118004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:46.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.411+0000 7fa12ad9d700 1 -- 172.21.15.89:0/1970615206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa118003a90 msgr2=0x7fa118003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:46.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.411+0000 7fa12ad9d700 1 --2- 172.21.15.89:0/1970615206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa118003a90 0x7fa118003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:46.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.411+0000 7fa12ad9d700 1 -- 172.21.15.89:0/1970615206 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa118003570 con 0x7fa118005210 2023-10-10T20:42:46.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.411+0000 7fa12ad9d700 1 -- 172.21.15.89:0/1970615206 reap_dead start 2023-10-10T20:42:46.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.411+0000 7fa12ad9d700 1 --2- 172.21.15.89:0/1970615206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa118005210 0x7fa11800ba60 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fa12c069fb0 tx=0x7fa12400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fc4b847795fa4c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:46.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.411+0000 7fa129d9b700 1 -- 172.21.15.89:0/1970615206 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12400ea90 con 0x7fa118005210 2023-10-10T20:42:46.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.411+0000 7fa129d9b700 1 -- 172.21.15.89:0/1970615206 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12400f040 con 0x7fa118005210 2023-10-10T20:42:46.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.411+0000 7fa129d9b700 1 -- 172.21.15.89:0/1970615206 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12400ee40 con 0x7fa118005210 2023-10-10T20:42:46.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.411+0000 7fa130bb1700 1 -- 172.21.15.89:0/1970615206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa118005210 msgr2=0x7fa11800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:46.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.411+0000 7fa130bb1700 1 --2- 172.21.15.89:0/1970615206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa118005210 0x7fa11800ba60 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fa12c069fb0 tx=0x7fa12400b040 comp rx=0 tx=0).stop 2023-10-10T20:42:46.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.411+0000 7fa130bb1700 1 -- 172.21.15.89:0/1970615206 shutdown_connections 2023-10-10T20:42:46.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.411+0000 7fa130bb1700 1 --2- 172.21.15.89:0/1970615206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa118005210 0x7fa11800ba60 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:46.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.411+0000 7fa130bb1700 1 -- 172.21.15.89:0/1970615206 >> 172.21.15.89:0/1970615206 conn(0x7fa12c068e10 msgr2=0x7fa12c0d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:46.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.412+0000 7fa130bb1700 1 -- 172.21.15.89:0/1970615206 shutdown_connections 2023-10-10T20:42:46.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.412+0000 7fa130bb1700 1 -- 172.21.15.89:0/1970615206 wait complete. 2023-10-10T20:42:46.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.413+0000 7fa130bb1700 1 Processor -- start 2023-10-10T20:42:46.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.413+0000 7fa130bb1700 1 -- start start 2023-10-10T20:42:46.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.413+0000 7fa130bb1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa118005210 0x7fa12c0c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:46.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.413+0000 7fa130bb1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa12c153f90 0x7fa12c0b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:46.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.413+0000 7fa130bb1700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa12c0b5cd0 0x7fa12c169f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:46.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.413+0000 7fa130bb1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c0b6140 con 0x7fa12c153f90 2023-10-10T20:42:46.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.413+0000 7fa130bb1700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c0c87c0 con 0x7fa12c0b5cd0 2023-10-10T20:42:46.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.413+0000 7fa130bb1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c0c8aa0 con 0x7fa118005210 2023-10-10T20:42:46.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.413+0000 7fa123fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa12c153f90 0x7fa12c0b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:46.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.413+0000 7fa123fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa12c153f90 0x7fa12c0b5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39144/0 (socket says 172.21.15.89:39144) 2023-10-10T20:42:46.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.413+0000 7fa123fff700 1 -- 172.21.15.89:0/206176807 learned_addr learned my addr 172.21.15.89:0/206176807 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:46.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.414+0000 7fa12ad9d700 1 --2- 172.21.15.89:0/206176807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa12c0b5cd0 0x7fa12c169f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:46.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.414+0000 7fa12a59c700 1 --2- 172.21.15.89:0/206176807 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa118005210 0x7fa12c0c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:46.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.414+0000 7fa123fff700 1 -- 172.21.15.89:0/206176807 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa118005210 msgr2=0x7fa12c0c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:46.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.414+0000 7fa123fff700 1 --2- 172.21.15.89:0/206176807 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa118005210 0x7fa12c0c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:46.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.414+0000 7fa123fff700 1 -- 172.21.15.89:0/206176807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa12c0b5cd0 msgr2=0x7fa12c169f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:46.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.414+0000 7fa123fff700 1 --2- 172.21.15.89:0/206176807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa12c0b5cd0 0x7fa12c169f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:46.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.414+0000 7fa123fff700 1 -- 172.21.15.89:0/206176807 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c0c8d80 con 0x7fa12c153f90 2023-10-10T20:42:46.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.414+0000 7fa123fff700 1 --2- 172.21.15.89:0/206176807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa12c153f90 0x7fa12c0b5790 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fa114002a30 tx=0x7fa114007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:46.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.414+0000 7fa1227fc700 1 -- 172.21.15.89:0/206176807 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114002e70 con 0x7fa12c153f90 2023-10-10T20:42:46.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.414+0000 7fa1227fc700 1 -- 172.21.15.89:0/206176807 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11400f870 con 0x7fa12c153f90 2023-10-10T20:42:46.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.414+0000 7fa1227fc700 1 -- 172.21.15.89:0/206176807 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11400fc50 con 0x7fa12c153f90 2023-10-10T20:42:46.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.414+0000 7fa130bb1700 1 -- 172.21.15.89:0/206176807 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12c0c9060 con 0x7fa12c153f90 2023-10-10T20:42:46.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.414+0000 7fa130bb1700 1 -- 172.21.15.89:0/206176807 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12c154400 con 0x7fa12c153f90 2023-10-10T20:42:46.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.415+0000 7fa130bb1700 1 -- 172.21.15.89:0/206176807 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa12c064550 con 0x7fa12c153f90 2023-10-10T20:42:46.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.415+0000 7fa1227fc700 1 -- 172.21.15.89:0/206176807 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa114013600 con 0x7fa12c153f90 2023-10-10T20:42:46.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.416+0000 7fa1227fc700 1 --2- 172.21.15.89:0/206176807 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa10c067350 0x7fa10c069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:46.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.416+0000 7fa1227fc700 1 -- 172.21.15.89:0/206176807 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa114084730 con 0x7fa12c153f90 2023-10-10T20:42:46.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.416+0000 7fa12a59c700 1 --2- 172.21.15.89:0/206176807 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa10c067350 0x7fa10c069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:46.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.416+0000 7fa12a59c700 1 --2- 172.21.15.89:0/206176807 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa10c067350 0x7fa10c069800 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0015a0 tx=0x7fa11c001e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:46.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.420+0000 7fa1227fc700 1 -- 172.21.15.89:0/206176807 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1140516c0 con 0x7fa12c153f90 2023-10-10T20:42:46.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:46.679+0000 7fa130bb1700 1 -- 172.21.15.89:0/206176807 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]} v 0) v1 -- 0x7fa12c169c30 con 0x7fa12c153f90 2023-10-10T20:42:47.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.279+0000 7fa1227fc700 1 -- 172.21.15.89:0/206176807 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]}]=0 v412) v1 ==== 166+0+0 (secure 0 0 0) 0x7fa114054ce0 con 0x7fa12c153f90 2023-10-10T20:42:47.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.281+0000 7fa130bb1700 1 -- 172.21.15.89:0/206176807 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa10c067350 msgr2=0x7fa10c069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:47.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.281+0000 7fa130bb1700 1 --2- 172.21.15.89:0/206176807 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa10c067350 0x7fa10c069800 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0015a0 tx=0x7fa11c001e70 comp rx=0 tx=0).stop 2023-10-10T20:42:47.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.281+0000 7fa130bb1700 1 -- 172.21.15.89:0/206176807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa12c153f90 msgr2=0x7fa12c0b5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:47.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.281+0000 7fa130bb1700 1 --2- 172.21.15.89:0/206176807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa12c153f90 0x7fa12c0b5790 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fa114002a30 tx=0x7fa114007d10 comp rx=0 tx=0).stop 2023-10-10T20:42:47.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.281+0000 7fa130bb1700 1 -- 172.21.15.89:0/206176807 shutdown_connections 2023-10-10T20:42:47.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.281+0000 7fa130bb1700 1 --2- 172.21.15.89:0/206176807 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa10c067350 0x7fa10c069800 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:47.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.281+0000 7fa130bb1700 1 --2- 172.21.15.89:0/206176807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa12c0b5cd0 0x7fa12c169f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:47.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.281+0000 7fa130bb1700 1 --2- 172.21.15.89:0/206176807 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa118005210 0x7fa12c0c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:47.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.281+0000 7fa130bb1700 1 --2- 172.21.15.89:0/206176807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa12c153f90 0x7fa12c0b5790 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:47.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.281+0000 7fa130bb1700 1 -- 172.21.15.89:0/206176807 >> 172.21.15.89:0/206176807 conn(0x7fa12c068e10 msgr2=0x7fa12c158a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:47.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.281+0000 7fa130bb1700 1 -- 172.21.15.89:0/206176807 shutdown_connections 2023-10-10T20:42:47.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.281+0000 7fa130bb1700 1 -- 172.21.15.89:0/206176807 wait complete. 2023-10-10T20:42:47.297 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:42:47.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.414+0000 7f4545f35700 1 Processor -- start 2023-10-10T20:42:47.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.414+0000 7f4545f35700 1 -- start start 2023-10-10T20:42:47.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.415+0000 7f4545f35700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4540069560 0x7f45400699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:47.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.415+0000 7f4545f35700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4540069ee0 con 0x7f4540159c60 2023-10-10T20:42:47.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.415+0000 7f4545f35700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f454015a160 con 0x7f4540159890 2023-10-10T20:42:47.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.415+0000 7f4545f35700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45400d13a0 con 0x7f4540069560 2023-10-10T20:42:47.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.415+0000 7f453effd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4540159c60 0x7f454015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59244/0 (socket says 172.21.15.89:59244) 2023-10-10T20:42:47.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.415+0000 7f453effd700 1 -- 172.21.15.89:0/254067378 learned_addr learned my addr 172.21.15.89:0/254067378 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:47.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453ffff700 1 --2- 172.21.15.89:0/254067378 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4540069560 0x7f45400699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:47.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453e7fc700 1 -- 172.21.15.89:0/254067378 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188521718 0 0) 0x7f4540069ee0 con 0x7f4540159c60 2023-10-10T20:42:47.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453e7fc700 1 -- 172.21.15.89:0/254067378 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4528003570 con 0x7f4540159c60 2023-10-10T20:42:47.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453e7fc700 1 -- 172.21.15.89:0/254067378 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3823936221 0 0) 0x7f4528003570 con 0x7f4540159c60 2023-10-10T20:42:47.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453e7fc700 1 -- 172.21.15.89:0/254067378 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4540069560 msgr2=0x7f45400699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:47.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453e7fc700 1 --2- 172.21.15.89:0/254067378 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4540069560 0x7f45400699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:47.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453e7fc700 1 -- 172.21.15.89:0/254067378 >> v1:172.21.15.111:6789/0 conn(0x7f4540159890 legacy=0x7f454015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:47.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453e7fc700 1 -- 172.21.15.89:0/254067378 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454015e140 con 0x7f4540159c60 2023-10-10T20:42:47.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453e7fc700 1 -- 172.21.15.89:0/254067378 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f4534001470 con 0x7f4540159c60 2023-10-10T20:42:47.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453e7fc700 1 -- 172.21.15.89:0/254067378 >> v1:172.21.15.89:6789/0 conn(0x7f4540159c60 legacy=0x7f454015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:47.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453e7fc700 1 --2- 172.21.15.89:0/254067378 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4528003a90 0x7f4528003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:47.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453e7fc700 1 --2- 172.21.15.89:0/254067378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4528004810 0x7f4528004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:47.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453e7fc700 1 --2- 172.21.15.89:0/254067378 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4540069560 0x7f45280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:47.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453e7fc700 1 -- 172.21.15.89:0/254067378 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45400d13a0 con 0x7f4528003a90 2023-10-10T20:42:47.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453e7fc700 1 -- 172.21.15.89:0/254067378 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452800bb40 con 0x7f4528004810 2023-10-10T20:42:47.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453e7fc700 1 -- 172.21.15.89:0/254067378 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452800bd00 con 0x7f4540069560 2023-10-10T20:42:47.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453f7fe700 1 --2- 172.21.15.89:0/254067378 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4528003a90 0x7f4528003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:47.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453effd700 1 --2- 172.21.15.89:0/254067378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4528004810 0x7f4528004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:47.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453ffff700 1 --2- 172.21.15.89:0/254067378 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4540069560 0x7f45280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:47.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453f7fe700 1 -- 172.21.15.89:0/254067378 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4540069560 msgr2=0x7f45280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:47.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453f7fe700 1 --2- 172.21.15.89:0/254067378 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4540069560 0x7f45280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:47.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453f7fe700 1 -- 172.21.15.89:0/254067378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4528004810 msgr2=0x7f4528004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:47.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453f7fe700 1 --2- 172.21.15.89:0/254067378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4528004810 0x7f4528004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:47.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453f7fe700 1 -- 172.21.15.89:0/254067378 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4528003570 con 0x7f4528003a90 2023-10-10T20:42:47.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453effd700 1 --2- 172.21.15.89:0/254067378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4528004810 0x7f4528004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:47.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.416+0000 7f453f7fe700 1 --2- 172.21.15.89:0/254067378 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4528003a90 0x7f4528003e60 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f4530002080 tx=0x7f4530001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3dfc657c131c742 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:47.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.417+0000 7f453e7fc700 1 -- 172.21.15.89:0/254067378 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453000c8b0 con 0x7f4528003a90 2023-10-10T20:42:47.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.417+0000 7f453e7fc700 1 -- 172.21.15.89:0/254067378 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453000ca10 con 0x7f4528003a90 2023-10-10T20:42:47.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.417+0000 7f453e7fc700 1 -- 172.21.15.89:0/254067378 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453000cba0 con 0x7f4528003a90 2023-10-10T20:42:47.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.417+0000 7f4545f35700 1 -- 172.21.15.89:0/254067378 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4528003a90 msgr2=0x7f4528003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:47.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.417+0000 7f4545f35700 1 --2- 172.21.15.89:0/254067378 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4528003a90 0x7f4528003e60 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f4530002080 tx=0x7f4530001700 comp rx=0 tx=0).stop 2023-10-10T20:42:47.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.417+0000 7f453ffff700 1 -- 172.21.15.89:0/254067378 reap_dead start 2023-10-10T20:42:47.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.417+0000 7f4545f35700 1 -- 172.21.15.89:0/254067378 shutdown_connections 2023-10-10T20:42:47.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.417+0000 7f4545f35700 1 -- 172.21.15.89:0/254067378 >> 172.21.15.89:0/254067378 conn(0x7f4540068e20 msgr2=0x7f45400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:47.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.417+0000 7f4545f35700 1 -- 172.21.15.89:0/254067378 shutdown_connections 2023-10-10T20:42:47.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.417+0000 7f4545f35700 1 -- 172.21.15.89:0/254067378 wait complete. 2023-10-10T20:42:47.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.418+0000 7f4545f35700 1 Processor -- start 2023-10-10T20:42:47.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.418+0000 7f4545f35700 1 -- start start 2023-10-10T20:42:47.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.419+0000 7f4545f35700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4528003a90 0x7f4540151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:47.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.419+0000 7f4545f35700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45401699f0 0x7f45400b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:47.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.419+0000 7f4545f35700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45400b5510 0x7f45400cea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:47.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.419+0000 7f4545f35700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45400c78a0 con 0x7f45401699f0 2023-10-10T20:42:47.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.419+0000 7f4545f35700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45400c7b80 con 0x7f45400b5510 2023-10-10T20:42:47.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.419+0000 7f4545f35700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45400c7e60 con 0x7f4528003a90 2023-10-10T20:42:47.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.419+0000 7f453effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45401699f0 0x7f45400b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:47.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.419+0000 7f453effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45401699f0 0x7f45400b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39164/0 (socket says 172.21.15.89:39164) 2023-10-10T20:42:47.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.419+0000 7f453effd700 1 -- 172.21.15.89:0/3605734076 learned_addr learned my addr 172.21.15.89:0/3605734076 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:47.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.419+0000 7f453f7fe700 1 --2- 172.21.15.89:0/3605734076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4528003a90 0x7f4540151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:47.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.419+0000 7f453ffff700 1 --2- 172.21.15.89:0/3605734076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45400b5510 0x7f45400cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:47.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.419+0000 7f453effd700 1 -- 172.21.15.89:0/3605734076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4528003a90 msgr2=0x7f4540151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:47.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.419+0000 7f453effd700 1 --2- 172.21.15.89:0/3605734076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4528003a90 0x7f4540151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:47.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.420+0000 7f453effd700 1 -- 172.21.15.89:0/3605734076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45400b5510 msgr2=0x7f45400cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:47.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.420+0000 7f453effd700 1 --2- 172.21.15.89:0/3605734076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45400b5510 0x7f45400cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:47.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.420+0000 7f453effd700 1 -- 172.21.15.89:0/3605734076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45400c8140 con 0x7f45401699f0 2023-10-10T20:42:47.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.420+0000 7f453ffff700 1 --2- 172.21.15.89:0/3605734076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45400b5510 0x7f45400cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:47.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.420+0000 7f453effd700 1 --2- 172.21.15.89:0/3605734076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45401699f0 0x7f45400b4fd0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f4534001ed0 tx=0x7f45340010f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:47.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.420+0000 7f45277fe700 1 -- 172.21.15.89:0/3605734076 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534005e90 con 0x7f45401699f0 2023-10-10T20:42:47.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.420+0000 7f45277fe700 1 -- 172.21.15.89:0/3605734076 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453400c7b0 con 0x7f45401699f0 2023-10-10T20:42:47.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.420+0000 7f45277fe700 1 -- 172.21.15.89:0/3605734076 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453400cb50 con 0x7f45401699f0 2023-10-10T20:42:47.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.420+0000 7f4545f35700 1 -- 172.21.15.89:0/3605734076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45400c8420 con 0x7f45401699f0 2023-10-10T20:42:47.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.420+0000 7f4545f35700 1 -- 172.21.15.89:0/3605734076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454016dd40 con 0x7f45401699f0 2023-10-10T20:42:47.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.421+0000 7f4545f35700 1 -- 172.21.15.89:0/3605734076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45400c8660 con 0x7f45401699f0 2023-10-10T20:42:47.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.421+0000 7f45277fe700 1 -- 172.21.15.89:0/3605734076 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f453401d460 con 0x7f45401699f0 2023-10-10T20:42:47.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.422+0000 7f45277fe700 1 --2- 172.21.15.89:0/3605734076 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f452c067320 0x7f452c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:47.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.422+0000 7f45277fe700 1 -- 172.21.15.89:0/3605734076 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f45340849f0 con 0x7f45401699f0 2023-10-10T20:42:47.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.422+0000 7f453f7fe700 1 --2- 172.21.15.89:0/3605734076 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f452c067320 0x7f452c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:47.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.422+0000 7f453f7fe700 1 --2- 172.21.15.89:0/3605734076 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f452c067320 0x7f452c0697d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4530001b20 tx=0x7f4530015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:47.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.426+0000 7f45277fe700 1 -- 172.21.15.89:0/3605734076 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45400c8660 con 0x7f45401699f0 2023-10-10T20:42:47.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.607+0000 7f4545f35700 1 -- 172.21.15.89:0/3605734076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f45400c8660 con 0x7f45401699f0 2023-10-10T20:42:47.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.611+0000 7f45277fe700 1 -- 172.21.15.89:0/3605734076 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f45400c8660 con 0x7f45401699f0 2023-10-10T20:42:47.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.614+0000 7f4545f35700 1 -- 172.21.15.89:0/3605734076 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f452c067320 msgr2=0x7f452c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:47.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.614+0000 7f4545f35700 1 --2- 172.21.15.89:0/3605734076 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f452c067320 0x7f452c0697d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4530001b20 tx=0x7f4530015000 comp rx=0 tx=0).stop 2023-10-10T20:42:47.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.614+0000 7f4545f35700 1 -- 172.21.15.89:0/3605734076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45401699f0 msgr2=0x7f45400b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:47.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.614+0000 7f4545f35700 1 --2- 172.21.15.89:0/3605734076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45401699f0 0x7f45400b4fd0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f4534001ed0 tx=0x7f45340010f0 comp rx=0 tx=0).stop 2023-10-10T20:42:47.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.614+0000 7f4545f35700 1 -- 172.21.15.89:0/3605734076 shutdown_connections 2023-10-10T20:42:47.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.614+0000 7f4545f35700 1 --2- 172.21.15.89:0/3605734076 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f452c067320 0x7f452c0697d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:47.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.614+0000 7f4545f35700 1 --2- 172.21.15.89:0/3605734076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45400b5510 0x7f45400cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:47.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.614+0000 7f4545f35700 1 --2- 172.21.15.89:0/3605734076 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4528003a90 0x7f4540151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:47.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.614+0000 7f4545f35700 1 --2- 172.21.15.89:0/3605734076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45401699f0 0x7f45400b4fd0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:47.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.614+0000 7f4545f35700 1 -- 172.21.15.89:0/3605734076 >> 172.21.15.89:0/3605734076 conn(0x7f4540068e20 msgr2=0x7f4540152da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:47.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.614+0000 7f4545f35700 1 -- 172.21.15.89:0/3605734076 shutdown_connections 2023-10-10T20:42:47.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.614+0000 7f4545f35700 1 -- 172.21.15.89:0/3605734076 wait complete. 2023-10-10T20:42:47.628 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:42:47.628 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:42:47.670 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:42:47.712 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:42:47.712 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:42:47.713 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:42:47.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.827+0000 7fd6ba8e5700 1 Processor -- start 2023-10-10T20:42:47.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.828+0000 7fd6ba8e5700 1 -- start start 2023-10-10T20:42:47.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.828+0000 7fd6ba8e5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6b40b0a10 0x7fd6b4155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:47.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.828+0000 7fd6ba8e5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b40ad930 con 0x7fd6b4155c10 2023-10-10T20:42:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.828+0000 7fd6ba8e5700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b40acd20 con 0x7fd6b41578b0 2023-10-10T20:42:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.828+0000 7fd6ba8e5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b40ad000 con 0x7fd6b40b0a10 2023-10-10T20:42:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.828+0000 7fd6b3fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd6b4155c10 0x7fd6b40ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59250/0 (socket says 172.21.15.89:59250) 2023-10-10T20:42:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.828+0000 7fd6b3fff700 1 -- 172.21.15.89:0/670313876 learned_addr learned my addr 172.21.15.89:0/670313876 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.829+0000 7fd6b8e82700 1 --2- 172.21.15.89:0/670313876 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6b40b0a10 0x7fd6b4155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:47.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.829+0000 7fd6b2ffd700 1 -- 172.21.15.89:0/670313876 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707145178 0 0) 0x7fd6b40ad930 con 0x7fd6b4155c10 2023-10-10T20:42:47.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.829+0000 7fd6b2ffd700 1 -- 172.21.15.89:0/670313876 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd69c003570 con 0x7fd6b4155c10 2023-10-10T20:42:47.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.829+0000 7fd6b2ffd700 1 -- 172.21.15.89:0/670313876 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3307792044 0 0) 0x7fd6b40acd20 con 0x7fd6b41578b0 2023-10-10T20:42:47.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.829+0000 7fd6b2ffd700 1 -- 172.21.15.89:0/670313876 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6b40ad930 con 0x7fd6b41578b0 2023-10-10T20:42:47.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.829+0000 7fd6b2ffd700 1 -- 172.21.15.89:0/670313876 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 645071775 0 0) 0x7fd69c003570 con 0x7fd6b4155c10 2023-10-10T20:42:47.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.829+0000 7fd6b8e82700 1 -- 172.21.15.89:0/670313876 >> v1:172.21.15.111:6789/0 conn(0x7fd6b41578b0 legacy=0x7fd6b40af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:47.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.829+0000 7fd6b8e82700 1 -- 172.21.15.89:0/670313876 >> v1:172.21.15.89:6789/0 conn(0x7fd6b4155c10 legacy=0x7fd6b40ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:47.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.829+0000 7fd6b8e82700 1 -- 172.21.15.89:0/670313876 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b40ad2e0 con 0x7fd6b40b0a10 2023-10-10T20:42:47.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.829+0000 7fd6b8e82700 1 --2- 172.21.15.89:0/670313876 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6b40b0a10 0x7fd6b4155fe0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac009e90 tx=0x7fd6ac009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20a576b29b924be2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:47.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.829+0000 7fd6b2ffd700 1 -- 172.21.15.89:0/670313876 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac00e070 con 0x7fd6b40b0a10 2023-10-10T20:42:47.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.829+0000 7fd6b2ffd700 1 -- 172.21.15.89:0/670313876 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ac00b870 con 0x7fd6b40b0a10 2023-10-10T20:42:47.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.829+0000 7fd6b2ffd700 1 -- 172.21.15.89:0/670313876 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4001470 con 0x7fd6b40b0a10 2023-10-10T20:42:47.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.830+0000 7fd6ba8e5700 1 -- 172.21.15.89:0/670313876 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6b40b0a10 msgr2=0x7fd6b4155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:47.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.830+0000 7fd6ba8e5700 1 --2- 172.21.15.89:0/670313876 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6b40b0a10 0x7fd6b4155fe0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac009e90 tx=0x7fd6ac009780 comp rx=0 tx=0).stop 2023-10-10T20:42:47.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.830+0000 7fd6ba8e5700 1 -- 172.21.15.89:0/670313876 shutdown_connections 2023-10-10T20:42:47.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.830+0000 7fd6ba8e5700 1 --2- 172.21.15.89:0/670313876 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6b40b0a10 0x7fd6b4155fe0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:47.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.830+0000 7fd6ba8e5700 1 -- 172.21.15.89:0/670313876 >> 172.21.15.89:0/670313876 conn(0x7fd6b406a320 msgr2=0x7fd6b40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:47.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.830+0000 7fd6ba8e5700 1 -- 172.21.15.89:0/670313876 shutdown_connections 2023-10-10T20:42:47.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.830+0000 7fd6ba8e5700 1 -- 172.21.15.89:0/670313876 wait complete. 2023-10-10T20:42:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.831+0000 7fd6ba8e5700 1 Processor -- start 2023-10-10T20:42:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.831+0000 7fd6ba8e5700 1 -- start start 2023-10-10T20:42:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.832+0000 7fd6ba8e5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6b40b0a10 0x7fd6b41663f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:47.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.832+0000 7fd6ba8e5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6b4155c10 0x7fd6b4168ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:47.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.832+0000 7fd6ba8e5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6b41578b0 0x7fd6b4154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:47.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.832+0000 7fd6ba8e5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b41693e0 con 0x7fd6b4155c10 2023-10-10T20:42:47.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.832+0000 7fd6ba8e5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b4169610 con 0x7fd6b41578b0 2023-10-10T20:42:47.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.832+0000 7fd6ba8e5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b41698f0 con 0x7fd6b40b0a10 2023-10-10T20:42:47.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.832+0000 7fd6b37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6b4155c10 0x7fd6b4168ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:47.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.832+0000 7fd6b37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6b4155c10 0x7fd6b4168ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39176/0 (socket says 172.21.15.89:39176) 2023-10-10T20:42:47.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.832+0000 7fd6b37fe700 1 -- 172.21.15.89:0/1483327804 learned_addr learned my addr 172.21.15.89:0/1483327804 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:47.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.832+0000 7fd6b3fff700 1 --2- 172.21.15.89:0/1483327804 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6b40b0a10 0x7fd6b41663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:47.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.832+0000 7fd6b8e82700 1 --2- 172.21.15.89:0/1483327804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6b41578b0 0x7fd6b4154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:47.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.832+0000 7fd6b37fe700 1 -- 172.21.15.89:0/1483327804 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6b40b0a10 msgr2=0x7fd6b41663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:47.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.832+0000 7fd6b37fe700 1 --2- 172.21.15.89:0/1483327804 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6b40b0a10 0x7fd6b41663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:47.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.832+0000 7fd6b37fe700 1 -- 172.21.15.89:0/1483327804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6b41578b0 msgr2=0x7fd6b4154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:47.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.832+0000 7fd6b37fe700 1 --2- 172.21.15.89:0/1483327804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6b41578b0 0x7fd6b4154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:47.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.832+0000 7fd6b37fe700 1 -- 172.21.15.89:0/1483327804 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac009a20 con 0x7fd6b4155c10 2023-10-10T20:42:47.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.832+0000 7fd6b3fff700 1 --2- 172.21.15.89:0/1483327804 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6b40b0a10 0x7fd6b41663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:47.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.833+0000 7fd6b37fe700 1 --2- 172.21.15.89:0/1483327804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6b4155c10 0x7fd6b4168ea0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800c580 tx=0x7fd6a8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:47.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.833+0000 7fd69bfff700 1 -- 172.21.15.89:0/1483327804 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8010510 con 0x7fd6b4155c10 2023-10-10T20:42:47.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.833+0000 7fd69bfff700 1 -- 172.21.15.89:0/1483327804 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a8010670 con 0x7fd6b4155c10 2023-10-10T20:42:47.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.833+0000 7fd6ba8e5700 1 -- 172.21.15.89:0/1483327804 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b4169bd0 con 0x7fd6b4155c10 2023-10-10T20:42:47.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.833+0000 7fd69bfff700 1 -- 172.21.15.89:0/1483327804 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8010a30 con 0x7fd6b4155c10 2023-10-10T20:42:47.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.833+0000 7fd6ba8e5700 1 -- 172.21.15.89:0/1483327804 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b4154a90 con 0x7fd6b4155c10 2023-10-10T20:42:47.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.834+0000 7fd6ba8e5700 1 -- 172.21.15.89:0/1483327804 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b4169e10 con 0x7fd6b4155c10 2023-10-10T20:42:47.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.834+0000 7fd69bfff700 1 -- 172.21.15.89:0/1483327804 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd6a801b400 con 0x7fd6b4155c10 2023-10-10T20:42:47.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.834+0000 7fd69bfff700 1 --2- 172.21.15.89:0/1483327804 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd6a0067320 0x7fd6a00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:47.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.834+0000 7fd69bfff700 1 -- 172.21.15.89:0/1483327804 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd6a8084600 con 0x7fd6b4155c10 2023-10-10T20:42:47.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.834+0000 7fd6b3fff700 1 --2- 172.21.15.89:0/1483327804 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd6a0067320 0x7fd6a00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:47.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.835+0000 7fd6b3fff700 1 --2- 172.21.15.89:0/1483327804 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd6a0067320 0x7fd6a00697d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40015a0 tx=0x7fd6a4000bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:47.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:47.838+0000 7fd69bfff700 1 -- 172.21.15.89:0/1483327804 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6b4169e10 con 0x7fd6b4155c10 2023-10-10T20:42:48.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.064+0000 7fd6ba8e5700 1 -- 172.21.15.89:0/1483327804 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6b4169e10 con 0x7fd6b4155c10 2023-10-10T20:42:48.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.065+0000 7fd69bfff700 1 -- 172.21.15.89:0/1483327804 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+3531 (secure 0 0 0) 0x7fd6b4169e10 con 0x7fd6b4155c10 2023-10-10T20:42:48.067 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:48.067 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":104,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/830099189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":830099189},{"type":"v1","addr":"172.21.15.111:6834","nonce":830099189}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2380789310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2380789310},{"type":"v1","addr":"172.21.15.111:6835","nonce":2380789310}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7956,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2087922155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2087922155},{"type":"v1","addr":"172.21.15.89:6835","nonce":2087922155}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":8259,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/220739257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":220739257},{"type":"v1","addr":"172.21.15.89:6837","nonce":220739257}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[]} 2023-10-10T20:42:48.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.067+0000 7fd6ba8e5700 1 -- 172.21.15.89:0/1483327804 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd6a0067320 msgr2=0x7fd6a00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:48.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.067+0000 7fd6ba8e5700 1 --2- 172.21.15.89:0/1483327804 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd6a0067320 0x7fd6a00697d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40015a0 tx=0x7fd6a4000bc0 comp rx=0 tx=0).stop 2023-10-10T20:42:48.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.067+0000 7fd6ba8e5700 1 -- 172.21.15.89:0/1483327804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6b4155c10 msgr2=0x7fd6b4168ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:48.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.067+0000 7fd6ba8e5700 1 --2- 172.21.15.89:0/1483327804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6b4155c10 0x7fd6b4168ea0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800c580 tx=0x7fd6a8007e10 comp rx=0 tx=0).stop 2023-10-10T20:42:48.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.067+0000 7fd6ba8e5700 1 -- 172.21.15.89:0/1483327804 shutdown_connections 2023-10-10T20:42:48.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.067+0000 7fd6ba8e5700 1 --2- 172.21.15.89:0/1483327804 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd6a0067320 0x7fd6a00697d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:48.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.067+0000 7fd6ba8e5700 1 --2- 172.21.15.89:0/1483327804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6b41578b0 0x7fd6b4154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:48.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.067+0000 7fd6ba8e5700 1 --2- 172.21.15.89:0/1483327804 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd6b40b0a10 0x7fd6b41663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:48.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.067+0000 7fd6ba8e5700 1 --2- 172.21.15.89:0/1483327804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6b4155c10 0x7fd6b4168ea0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:48.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.067+0000 7fd6ba8e5700 1 -- 172.21.15.89:0/1483327804 >> 172.21.15.89:0/1483327804 conn(0x7fd6b406a320 msgr2=0x7fd6b4151050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:48.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.067+0000 7fd6ba8e5700 1 -- 172.21.15.89:0/1483327804 shutdown_connections 2023-10-10T20:42:48.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.067+0000 7fd6ba8e5700 1 -- 172.21.15.89:0/1483327804 wait complete. 2023-10-10T20:42:48.071 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 104 2023-10-10T20:42:48.081 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:42:48.081 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:42:48.082 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:42:48.082 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:42:48.082 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:42:48.082 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:42:48.083 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:42:48.083 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:42:48.083 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:42:48.083 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:42:48.084 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:42:48.084 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:42:48.084 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:42:48.085 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:42:48.085 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:42:48.085 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:42:48.090 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:42:48.091 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:42:48.092 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:42:48.092 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:42:48.094 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:42:48.095 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:42:48.096 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:42:48.096 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:42:48.134 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:42:48.135 DEBUG:teuthology.parallel:result is None 2023-10-10T20:42:48.136 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:42:48.137 DEBUG:teuthology.parallel:result is None 2023-10-10T20:42:48.137 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:42:48.138 DEBUG:teuthology.parallel:result is None 2023-10-10T20:42:48.138 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:42:48.139 DEBUG:teuthology.parallel:result is None 2023-10-10T20:42:48.139 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:42:48.207 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-10T20:42:48.207 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:42:48.216 INFO:tasks.ceph.mds.b.smithi111.stdout:starting mds.b at 2023-10-10T20:42:48.216 INFO:tasks.ceph.mds.d.smithi111.stdout:starting mds.d at 2023-10-10T20:42:48.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.260+0000 7f1b75ab0700 1 Processor -- start 2023-10-10T20:42:48.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.260+0000 7f1b75ab0700 1 -- start start 2023-10-10T20:42:48.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.260+0000 7f1b75ab0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1b70156590 0x7f1b700b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:48.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.260+0000 7f1b75ab0700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1b7006c2c0 0x7f1b700b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:48.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.260+0000 7f1b75ab0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b7006eaf0 con 0x7f1b700b23d0 2023-10-10T20:42:48.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.260+0000 7f1b75ab0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b7015ab00 con 0x7f1b70156590 2023-10-10T20:42:48.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.260+0000 7f1b75ab0700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b7015b080 con 0x7f1b7006c2c0 2023-10-10T20:42:48.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.261+0000 7f1b74aae700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1b70156590 0x7f1b700b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:48.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.261+0000 7f1b752af700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1b700b23d0 0x7f1b70156960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48728/0 (socket says 172.21.15.89:48728) 2023-10-10T20:42:48.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.261+0000 7f1b752af700 1 -- 172.21.15.89:0/1071459016 learned_addr learned my addr 172.21.15.89:0/1071459016 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:48.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.261+0000 7f1b6ffff700 1 --2- 172.21.15.89:0/1071459016 >> v2:172.21.15.111:3300/0 conn(0x7f1b7006c2c0 0x7f1b700b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:48.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.261+0000 7f1b74aae700 1 -- 172.21.15.89:0/1071459016 >> v2:172.21.15.111:3300/0 conn(0x7f1b7006c2c0 msgr2=0x7f1b700b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:48.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.261+0000 7f1b74aae700 1 --2- 172.21.15.89:0/1071459016 >> v2:172.21.15.111:3300/0 conn(0x7f1b7006c2c0 0x7f1b700b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:48.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.261+0000 7f1b74aae700 1 -- 172.21.15.89:0/1071459016 >> v1:172.21.15.89:6789/0 conn(0x7f1b700b23d0 legacy=0x7f1b70156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:48.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.261+0000 7f1b74aae700 1 -- 172.21.15.89:0/1071459016 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b700b09e0 con 0x7f1b70156590 2023-10-10T20:42:48.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.261+0000 7f1b6f7fe700 1 -- 172.21.15.89:0/1071459016 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808019161 0 0) 0x7f1b7006eaf0 con 0x7f1b700b23d0 2023-10-10T20:42:48.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.261+0000 7f1b74aae700 1 --2- 172.21.15.89:0/1071459016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b70156590 0x7f1b700b04a0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f1b70063370 tx=0x7f1b640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb42c3c84b7efebb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:48.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.261+0000 7f1b6f7fe700 1 -- 172.21.15.89:0/1071459016 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6400b6b0 con 0x7f1b70156590 2023-10-10T20:42:48.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.261+0000 7f1b6f7fe700 1 -- 172.21.15.89:0/1071459016 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6400b810 con 0x7f1b70156590 2023-10-10T20:42:48.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.261+0000 7f1b6f7fe700 1 -- 172.21.15.89:0/1071459016 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6400bbb0 con 0x7f1b70156590 2023-10-10T20:42:48.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.262+0000 7f1b75ab0700 1 -- 172.21.15.89:0/1071459016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b70156590 msgr2=0x7f1b700b04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:48.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.262+0000 7f1b75ab0700 1 --2- 172.21.15.89:0/1071459016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b70156590 0x7f1b700b04a0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f1b70063370 tx=0x7f1b640091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:48.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.262+0000 7f1b75ab0700 1 -- 172.21.15.89:0/1071459016 shutdown_connections 2023-10-10T20:42:48.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.262+0000 7f1b75ab0700 1 --2- 172.21.15.89:0/1071459016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b70156590 0x7f1b700b04a0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:48.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.262+0000 7f1b75ab0700 1 --2- 172.21.15.89:0/1071459016 >> v2:172.21.15.111:3300/0 conn(0x7f1b7006c2c0 0x7f1b700b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:48.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.262+0000 7f1b75ab0700 1 -- 172.21.15.89:0/1071459016 >> 172.21.15.89:0/1071459016 conn(0x7f1b7006ddb0 msgr2=0x7f1b7006e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:48.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.262+0000 7f1b75ab0700 1 -- 172.21.15.89:0/1071459016 shutdown_connections 2023-10-10T20:42:48.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.262+0000 7f1b75ab0700 1 -- 172.21.15.89:0/1071459016 wait complete. 2023-10-10T20:42:48.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.263+0000 7f1b75ab0700 1 Processor -- start 2023-10-10T20:42:48.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.263+0000 7f1b75ab0700 1 -- start start 2023-10-10T20:42:48.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b75ab0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b7006c2c0 0x7f1b7016b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:48.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b75ab0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b700b23d0 0x7f1b70168570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:48.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b75ab0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b70156590 0x7f1b70155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:48.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b75ab0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b70168f20 con 0x7f1b700b23d0 2023-10-10T20:42:48.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b75ab0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7016b800 con 0x7f1b70156590 2023-10-10T20:42:48.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b75ab0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7016bae0 con 0x7f1b7006c2c0 2023-10-10T20:42:48.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b6ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b700b23d0 0x7f1b70168570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:48.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b6ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b700b23d0 0x7f1b70168570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54828/0 (socket says 172.21.15.89:54828) 2023-10-10T20:42:48.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b6ffff700 1 -- 172.21.15.89:0/3212506474 learned_addr learned my addr 172.21.15.89:0/3212506474 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:48.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b74aae700 1 --2- 172.21.15.89:0/3212506474 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b7006c2c0 0x7f1b7016b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:48.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b752af700 1 --2- 172.21.15.89:0/3212506474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b70156590 0x7f1b70155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:48.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b6ffff700 1 -- 172.21.15.89:0/3212506474 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b7006c2c0 msgr2=0x7f1b7016b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:48.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b6ffff700 1 --2- 172.21.15.89:0/3212506474 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b7006c2c0 0x7f1b7016b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:48.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b6ffff700 1 -- 172.21.15.89:0/3212506474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b70156590 msgr2=0x7f1b70155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b6ffff700 1 --2- 172.21.15.89:0/3212506474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b70156590 0x7f1b70155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b6ffff700 1 -- 172.21.15.89:0/3212506474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b64009420 con 0x7f1b700b23d0 2023-10-10T20:42:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b6ffff700 1 --2- 172.21.15.89:0/3212506474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b700b23d0 0x7f1b70168570 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00d890 tx=0x7f1b5c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b6cff9700 1 -- 172.21.15.89:0/3212506474 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c0125f0 con 0x7f1b700b23d0 2023-10-10T20:42:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b6cff9700 1 -- 172.21.15.89:0/3212506474 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5c012750 con 0x7f1b700b23d0 2023-10-10T20:42:48.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b75ab0700 1 -- 172.21.15.89:0/3212506474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b7016bdc0 con 0x7f1b700b23d0 2023-10-10T20:42:48.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.264+0000 7f1b6cff9700 1 -- 172.21.15.89:0/3212506474 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c012af0 con 0x7f1b700b23d0 2023-10-10T20:42:48.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.265+0000 7f1b75ab0700 1 -- 172.21.15.89:0/3212506474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b70168ab0 con 0x7f1b700b23d0 2023-10-10T20:42:48.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.265+0000 7f1b75ab0700 1 -- 172.21.15.89:0/3212506474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b701690d0 con 0x7f1b700b23d0 2023-10-10T20:42:48.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.266+0000 7f1b6cff9700 1 -- 172.21.15.89:0/3212506474 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1b5c01e460 con 0x7f1b700b23d0 2023-10-10T20:42:48.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.266+0000 7f1b6cff9700 1 --2- 172.21.15.89:0/3212506474 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1b54067320 0x7f1b540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:48.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.266+0000 7f1b74aae700 1 --2- 172.21.15.89:0/3212506474 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1b54067320 0x7f1b540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:48.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.266+0000 7f1b6cff9700 1 -- 172.21.15.89:0/3212506474 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1b5c084f10 con 0x7f1b700b23d0 2023-10-10T20:42:48.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.267+0000 7f1b74aae700 1 --2- 172.21.15.89:0/3212506474 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1b54067320 0x7f1b540697d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1b640093f0 tx=0x7f1b64017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:48.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.271+0000 7f1b6cff9700 1 -- 172.21.15.89:0/3212506474 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b701690d0 con 0x7f1b700b23d0 2023-10-10T20:42:48.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:48.471+0000 7f1b75ab0700 1 -- 172.21.15.89:0/3212506474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1b7004c3c0 con 0x7f1b700b23d0 2023-10-10T20:42:49.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.308+0000 7f1b6cff9700 1 -- 172.21.15.89:0/3212506474 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v161) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1b5c055930 con 0x7f1b700b23d0 2023-10-10T20:42:49.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.310+0000 7f1b75ab0700 1 -- 172.21.15.89:0/3212506474 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1b54067320 msgr2=0x7f1b540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:49.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.310+0000 7f1b75ab0700 1 --2- 172.21.15.89:0/3212506474 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1b54067320 0x7f1b540697d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1b640093f0 tx=0x7f1b64017000 comp rx=0 tx=0).stop 2023-10-10T20:42:49.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.310+0000 7f1b75ab0700 1 -- 172.21.15.89:0/3212506474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b700b23d0 msgr2=0x7f1b70168570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:49.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.310+0000 7f1b75ab0700 1 --2- 172.21.15.89:0/3212506474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b700b23d0 0x7f1b70168570 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00d890 tx=0x7f1b5c008040 comp rx=0 tx=0).stop 2023-10-10T20:42:49.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.310+0000 7f1b75ab0700 1 -- 172.21.15.89:0/3212506474 shutdown_connections 2023-10-10T20:42:49.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.310+0000 7f1b75ab0700 1 --2- 172.21.15.89:0/3212506474 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1b54067320 0x7f1b540697d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.310+0000 7f1b75ab0700 1 --2- 172.21.15.89:0/3212506474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b70156590 0x7f1b70155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.310+0000 7f1b75ab0700 1 --2- 172.21.15.89:0/3212506474 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b7006c2c0 0x7f1b7016b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.310+0000 7f1b75ab0700 1 --2- 172.21.15.89:0/3212506474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b700b23d0 0x7f1b70168570 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.310+0000 7f1b75ab0700 1 -- 172.21.15.89:0/3212506474 >> 172.21.15.89:0/3212506474 conn(0x7f1b7006ddb0 msgr2=0x7f1b701535a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:49.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.310+0000 7f1b75ab0700 1 -- 172.21.15.89:0/3212506474 shutdown_connections 2023-10-10T20:42:49.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.310+0000 7f1b75ab0700 1 -- 172.21.15.89:0/3212506474 wait complete. 2023-10-10T20:42:49.315 INFO:teuthology.orchestra.run.smithi089.stderr: removed all blocklist entries 2023-10-10T20:42:49.324 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:42:49.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.439+0000 7fd08b125700 1 Processor -- start 2023-10-10T20:42:49.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.439+0000 7fd08b125700 1 -- start start 2023-10-10T20:42:49.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.439+0000 7fd08b125700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd084159c60 0x7fd08415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:49.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd08b125700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd08415a160 con 0x7fd08415db10 2023-10-10T20:42:49.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd08b125700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0840d13a0 con 0x7fd084159890 2023-10-10T20:42:49.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd08b125700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08415df50 con 0x7fd084159c60 2023-10-10T20:42:49.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd0896c2700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd08415db10 0x7fd084069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48742/0 (socket says 172.21.15.89:48742) 2023-10-10T20:42:49.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd0896c2700 1 -- 172.21.15.89:0/1214988586 learned_addr learned my addr 172.21.15.89:0/1214988586 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:49.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd07bfff700 1 --2- 172.21.15.89:0/1214988586 >> v2:172.21.15.111:3300/0 conn(0x7fd084159c60 0x7fd08415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:49.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd07b7fe700 1 -- 172.21.15.89:0/1214988586 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753294165 0 0) 0x7fd08415a160 con 0x7fd08415db10 2023-10-10T20:42:49.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd07b7fe700 1 -- 172.21.15.89:0/1214988586 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd06c003570 con 0x7fd08415db10 2023-10-10T20:42:49.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd07b7fe700 1 -- 172.21.15.89:0/1214988586 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2510711007 0 0) 0x7fd06c003570 con 0x7fd08415db10 2023-10-10T20:42:49.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd07b7fe700 1 -- 172.21.15.89:0/1214988586 >> v2:172.21.15.111:3300/0 conn(0x7fd084159c60 msgr2=0x7fd08415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:49.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd07b7fe700 1 --2- 172.21.15.89:0/1214988586 >> v2:172.21.15.111:3300/0 conn(0x7fd084159c60 0x7fd08415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd07b7fe700 1 -- 172.21.15.89:0/1214988586 >> v1:172.21.15.111:6789/0 conn(0x7fd084159890 legacy=0x7fd08415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:49.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd07b7fe700 1 -- 172.21.15.89:0/1214988586 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08415e140 con 0x7fd08415db10 2023-10-10T20:42:49.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd07b7fe700 1 -- 172.21.15.89:0/1214988586 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fd07c001470 con 0x7fd08415db10 2023-10-10T20:42:49.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd07b7fe700 1 -- 172.21.15.89:0/1214988586 >> v1:172.21.15.89:6789/0 conn(0x7fd08415db10 legacy=0x7fd084069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:49.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd07b7fe700 1 --2- 172.21.15.89:0/1214988586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd06c003a60 0x7fd06c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:49.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd07b7fe700 1 --2- 172.21.15.89:0/1214988586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd06c004810 0x7fd06c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:49.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd07b7fe700 1 --2- 172.21.15.89:0/1214988586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd06c0051c0 0x7fd06c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:49.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd07b7fe700 1 -- 172.21.15.89:0/1214988586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c00c0a0 con 0x7fd06c003a60 2023-10-10T20:42:49.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd07b7fe700 1 -- 172.21.15.89:0/1214988586 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c00c210 con 0x7fd06c0051c0 2023-10-10T20:42:49.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd07b7fe700 1 -- 172.21.15.89:0/1214988586 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c00c3d0 con 0x7fd06c004810 2023-10-10T20:42:49.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.440+0000 7fd088ec1700 1 --2- 172.21.15.89:0/1214988586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd06c003a60 0x7fd06c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:49.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd0896c2700 1 --2- 172.21.15.89:0/1214988586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd06c0051c0 0x7fd06c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:49.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd07bfff700 1 --2- 172.21.15.89:0/1214988586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd06c004810 0x7fd06c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:49.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd088ec1700 1 -- 172.21.15.89:0/1214988586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd06c004810 msgr2=0x7fd06c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:49.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd088ec1700 1 --2- 172.21.15.89:0/1214988586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd06c004810 0x7fd06c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd088ec1700 1 -- 172.21.15.89:0/1214988586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd06c0051c0 msgr2=0x7fd06c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:49.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd088ec1700 1 --2- 172.21.15.89:0/1214988586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd06c0051c0 0x7fd06c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd088ec1700 1 -- 172.21.15.89:0/1214988586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c003570 con 0x7fd06c003a60 2023-10-10T20:42:49.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd0896c2700 1 --2- 172.21.15.89:0/1214988586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd06c0051c0 0x7fd06c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:49.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd0896c2700 1 -- 172.21.15.89:0/1214988586 reap_dead start 2023-10-10T20:42:49.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd088ec1700 1 --2- 172.21.15.89:0/1214988586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd06c003a60 0x7fd06c003e70 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fd080001fc0 tx=0x7fd080001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e2abe9677f399b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:49.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd07b7fe700 1 -- 172.21.15.89:0/1214988586 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080001950 con 0x7fd06c003a60 2023-10-10T20:42:49.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd07b7fe700 1 -- 172.21.15.89:0/1214988586 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08000c8b0 con 0x7fd06c003a60 2023-10-10T20:42:49.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd07b7fe700 1 -- 172.21.15.89:0/1214988586 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08000cc50 con 0x7fd06c003a60 2023-10-10T20:42:49.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd08b125700 1 -- 172.21.15.89:0/1214988586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd06c003a60 msgr2=0x7fd06c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:49.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd08b125700 1 --2- 172.21.15.89:0/1214988586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd06c003a60 0x7fd06c003e70 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fd080001fc0 tx=0x7fd080001640 comp rx=0 tx=0).stop 2023-10-10T20:42:49.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd08b125700 1 -- 172.21.15.89:0/1214988586 shutdown_connections 2023-10-10T20:42:49.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd08b125700 1 --2- 172.21.15.89:0/1214988586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd06c003a60 0x7fd06c003e70 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd08b125700 1 -- 172.21.15.89:0/1214988586 >> 172.21.15.89:0/1214988586 conn(0x7fd084068e20 msgr2=0x7fd0840d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:49.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd08b125700 1 -- 172.21.15.89:0/1214988586 shutdown_connections 2023-10-10T20:42:49.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.441+0000 7fd08b125700 1 -- 172.21.15.89:0/1214988586 wait complete. 2023-10-10T20:42:49.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.443+0000 7fd08b125700 1 Processor -- start 2023-10-10T20:42:49.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.443+0000 7fd08b125700 1 -- start start 2023-10-10T20:42:49.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.443+0000 7fd08b125700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd06c003a60 0x7fd084152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:49.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.443+0000 7fd08b125700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd0840b3590 0x7fd08416a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:49.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.443+0000 7fd08b125700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0840b4b80 0x7fd08415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:49.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.443+0000 7fd08b125700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd084160060 con 0x7fd0840b4b80 2023-10-10T20:42:49.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.443+0000 7fd08b125700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd084152750 con 0x7fd06c003a60 2023-10-10T20:42:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.443+0000 7fd08b125700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0840b46f0 con 0x7fd0840b3590 2023-10-10T20:42:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.443+0000 7fd0896c2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0840b4b80 0x7fd08415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.444+0000 7fd0896c2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0840b4b80 0x7fd08415b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54850/0 (socket says 172.21.15.89:54850) 2023-10-10T20:42:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.444+0000 7fd0896c2700 1 -- 172.21.15.89:0/2646156240 learned_addr learned my addr 172.21.15.89:0/2646156240 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.444+0000 7fd07bfff700 1 --2- 172.21.15.89:0/2646156240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd0840b3590 0x7fd08416a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.444+0000 7fd088ec1700 1 --2- 172.21.15.89:0/2646156240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd06c003a60 0x7fd084152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.444+0000 7fd0896c2700 1 -- 172.21.15.89:0/2646156240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd0840b3590 msgr2=0x7fd08416a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.444+0000 7fd0896c2700 1 --2- 172.21.15.89:0/2646156240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd0840b3590 0x7fd08416a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.444+0000 7fd0896c2700 1 -- 172.21.15.89:0/2646156240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd06c003a60 msgr2=0x7fd084152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.444+0000 7fd0896c2700 1 --2- 172.21.15.89:0/2646156240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd06c003a60 0x7fd084152210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.444+0000 7fd0896c2700 1 -- 172.21.15.89:0/2646156240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0840b3ae0 con 0x7fd0840b4b80 2023-10-10T20:42:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.444+0000 7fd0896c2700 1 --2- 172.21.15.89:0/2646156240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0840b4b80 0x7fd08415b4d0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fd0840d03b0 tx=0x7fd07c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.444+0000 7fd078ff9700 1 -- 172.21.15.89:0/2646156240 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c010450 con 0x7fd0840b4b80 2023-10-10T20:42:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.444+0000 7fd078ff9700 1 -- 172.21.15.89:0/2646156240 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c0105d0 con 0x7fd0840b4b80 2023-10-10T20:42:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.444+0000 7fd08b125700 1 -- 172.21.15.89:0/2646156240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0840b3dc0 con 0x7fd0840b4b80 2023-10-10T20:42:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.444+0000 7fd078ff9700 1 -- 172.21.15.89:0/2646156240 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c010990 con 0x7fd0840b4b80 2023-10-10T20:42:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.444+0000 7fd08b125700 1 -- 172.21.15.89:0/2646156240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0840b4380 con 0x7fd0840b4b80 2023-10-10T20:42:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.445+0000 7fd08b125700 1 -- 172.21.15.89:0/2646156240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0840b4880 con 0x7fd0840b4b80 2023-10-10T20:42:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.445+0000 7fd078ff9700 1 -- 172.21.15.89:0/2646156240 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd07c010d40 con 0x7fd0840b4b80 2023-10-10T20:42:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.446+0000 7fd078ff9700 1 --2- 172.21.15.89:0/2646156240 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd070067320 0x7fd0700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.446+0000 7fd078ff9700 1 -- 172.21.15.89:0/2646156240 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd07c083de0 con 0x7fd0840b4b80 2023-10-10T20:42:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.446+0000 7fd088ec1700 1 --2- 172.21.15.89:0/2646156240 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd070067320 0x7fd0700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.446+0000 7fd088ec1700 1 --2- 172.21.15.89:0/2646156240 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd070067320 0x7fd0700697d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd08415c0f0 tx=0x7fd080015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.450+0000 7fd078ff9700 1 -- 172.21.15.89:0/2646156240 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd07c04c5a0 con 0x7fd0840b4b80 2023-10-10T20:42:49.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.708+0000 7fd08b125700 1 -- 172.21.15.89:0/2646156240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd08404c3c0 con 0x7fd0840b4b80 2023-10-10T20:42:49.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.709+0000 7fd078ff9700 1 -- 172.21.15.89:0/2646156240 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd07c050ed0 con 0x7fd0840b4b80 2023-10-10T20:42:49.712 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:49.712 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:42:49.712 INFO:teuthology.orchestra.run.smithi089.stdout: "auth_dump": [ 2023-10-10T20:42:49.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:49.713 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.a", 2023-10-10T20:42:49.713 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlZLSNLhAA8WZTrsht6tAOP+gxaTXKMg==", 2023-10-10T20:42:49.713 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:49.713 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:42:49.713 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:42:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:42:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:49.715 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.b", 2023-10-10T20:42:49.715 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgyDEORAAMMoSxPFZc0bCIwWy8UsXgA==", 2023-10-10T20:42:49.715 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:49.715 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:42:49.715 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:42:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:42:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.c", 2023-10-10T20:42:49.717 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlMFxSNRAASuQEJFghLIOkTQWa4ckTvg==", 2023-10-10T20:42:49.717 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:49.717 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:42:49.717 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:42:49.717 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:42:49.718 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:49.718 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:49.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:49.718 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.d", 2023-10-10T20:42:49.719 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlPkvrBBAA2VLJmv2t8szTEGtCVwalJw==", 2023-10-10T20:42:49.719 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:49.719 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:42:49.719 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:42:49.719 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:42:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.0", 2023-10-10T20:42:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQArtSVltOcKHRAAk8+zUus04oTVI9KvfhjlXg==", 2023-10-10T20:42:49.721 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:49.721 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:42:49.721 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:42:49.721 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:49.721 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:49.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:49.722 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.1", 2023-10-10T20:42:49.722 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAutSVly8P0GhAAe1bvR63+ptJg84lizMNdQw==", 2023-10-10T20:42:49.722 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:49.723 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:42:49.723 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:42:49.723 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:49.723 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:49.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:49.724 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.2", 2023-10-10T20:42:49.725 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAxtSVl0x9fGRAAEKi7+6EW5mzPbebmz0+sgQ==", 2023-10-10T20:42:49.725 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:49.725 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:42:49.725 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:42:49.725 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:49.726 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:49.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:49.726 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.3", 2023-10-10T20:42:49.726 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA0tSVlfJyaGBAAa+8w54U6v7+npbdufKJruA==", 2023-10-10T20:42:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:42:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:42:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.4", 2023-10-10T20:42:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA6tSVlEQ/QFxAA/PRB3c90BuAlF6xekGfX9A==", 2023-10-10T20:42:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:42:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:42:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.5", 2023-10-10T20:42:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA9tSVlMOKrFRAAuBnIsEz2VmeSZi9XWzjlzA==", 2023-10-10T20:42:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:42:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:42:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.6", 2023-10-10T20:42:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBAtSVlH7PQFBAAHr6c7d4TYJEu/KSFEf+EOA==", 2023-10-10T20:42:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:42:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:42:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.7", 2023-10-10T20:42:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBDtSVlAhC1FBAAuE2BwAA/f7VOkZfVdqAG7A==", 2023-10-10T20:42:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:42:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:42:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.0", 2023-10-10T20:42:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlRd3HCBAAb0QP2VfTS7aelno1mFpGAA==", 2023-10-10T20:42:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:42:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow r", 2023-10-10T20:42:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:42:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.1", 2023-10-10T20:42:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlgBouERAAqPdE0KvJU3M53B6Y6AH5GQ==", 2023-10-10T20:42:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:42:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:42:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:42:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:42:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.2", 2023-10-10T20:42:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlXA4tDhAAoPVYQjECpzD+C67mnsY5IA==", 2023-10-10T20:42:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:42:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:42:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.3", 2023-10-10T20:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlZ5KGFhAAuKmZwbOfmmw1nAj5ry6tHw==", 2023-10-10T20:42:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:42:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:42:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:42:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:42:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.admin", 2023-10-10T20:42:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgpL2EhAAoDOki002rfBrauTgfu481Q==", 2023-10-10T20:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.x", 2023-10-10T20:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlBhaJKxAArraBxT5xWg515LfpJ7MlbA==", 2023-10-10T20:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:42:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:42:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:42:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.y", 2023-10-10T20:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlTZIyJxAAWAOsitHwsIxJWwp3b1Z98Q==", 2023-10-10T20:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:42:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-10T20:42:49.747 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:42:49.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.711+0000 7fd08b125700 1 -- 172.21.15.89:0/2646156240 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd070067320 msgr2=0x7fd0700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:49.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.711+0000 7fd08b125700 1 --2- 172.21.15.89:0/2646156240 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd070067320 0x7fd0700697d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd08415c0f0 tx=0x7fd080015000 comp rx=0 tx=0).stop 2023-10-10T20:42:49.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.711+0000 7fd08b125700 1 -- 172.21.15.89:0/2646156240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0840b4b80 msgr2=0x7fd08415b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:49.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.711+0000 7fd08b125700 1 --2- 172.21.15.89:0/2646156240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0840b4b80 0x7fd08415b4d0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fd0840d03b0 tx=0x7fd07c0014a0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.711+0000 7fd08b125700 1 -- 172.21.15.89:0/2646156240 shutdown_connections 2023-10-10T20:42:49.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.711+0000 7fd08b125700 1 --2- 172.21.15.89:0/2646156240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0840b4b80 0x7fd08415b4d0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.711+0000 7fd08b125700 1 --2- 172.21.15.89:0/2646156240 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd070067320 0x7fd0700697d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.711+0000 7fd08b125700 1 --2- 172.21.15.89:0/2646156240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd06c003a60 0x7fd084152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.711+0000 7fd08b125700 1 --2- 172.21.15.89:0/2646156240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd0840b3590 0x7fd08416a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.711+0000 7fd08b125700 1 -- 172.21.15.89:0/2646156240 >> 172.21.15.89:0/2646156240 conn(0x7fd084068e20 msgr2=0x7fd08415c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:49.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.711+0000 7fd08b125700 1 -- 172.21.15.89:0/2646156240 shutdown_connections 2023-10-10T20:42:49.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.711+0000 7fd08b125700 1 -- 172.21.15.89:0/2646156240 wait complete. 2023-10-10T20:42:49.751 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:42:49.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.865+0000 7fe7e49cd700 1 Processor -- start 2023-10-10T20:42:49.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.865+0000 7fe7e49cd700 1 -- start start 2023-10-10T20:42:49.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.866+0000 7fe7e49cd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7e015d230 0x7fe7e015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:49.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.866+0000 7fe7e49cd700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e00b2980 con 0x7fe7e0159420 2023-10-10T20:42:49.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.866+0000 7fe7e49cd700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e015dbb0 con 0x7fe7e01597f0 2023-10-10T20:42:49.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.866+0000 7fe7e49cd700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e00b11d0 con 0x7fe7e015d230 2023-10-10T20:42:49.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.866+0000 7fe7df7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe7e0159420 0x7fe7e00b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48758/0 (socket says 172.21.15.89:48758) 2023-10-10T20:42:49.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.866+0000 7fe7df7fe700 1 -- 172.21.15.89:0/4248482737 learned_addr learned my addr 172.21.15.89:0/4248482737 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:49.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.866+0000 7fe7dffff700 1 --2- 172.21.15.89:0/4248482737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7e015d230 0x7fe7e015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:49.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.866+0000 7fe7de7fc700 1 -- 172.21.15.89:0/4248482737 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545759851 0 0) 0x7fe7e00b2980 con 0x7fe7e0159420 2023-10-10T20:42:49.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.866+0000 7fe7de7fc700 1 -- 172.21.15.89:0/4248482737 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7c8003570 con 0x7fe7e0159420 2023-10-10T20:42:49.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7de7fc700 1 -- 172.21.15.89:0/4248482737 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1193079186 0 0) 0x7fe7c8003570 con 0x7fe7e0159420 2023-10-10T20:42:49.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7de7fc700 1 -- 172.21.15.89:0/4248482737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7e015d230 msgr2=0x7fe7e015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:49.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7de7fc700 1 --2- 172.21.15.89:0/4248482737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7e015d230 0x7fe7e015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7de7fc700 1 -- 172.21.15.89:0/4248482737 >> v1:172.21.15.111:6789/0 conn(0x7fe7e01597f0 legacy=0x7fe7e00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:49.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7de7fc700 1 -- 172.21.15.89:0/4248482737 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e00b14b0 con 0x7fe7e0159420 2023-10-10T20:42:49.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7de7fc700 1 -- 172.21.15.89:0/4248482737 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fe7d0001470 con 0x7fe7e0159420 2023-10-10T20:42:49.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7de7fc700 1 -- 172.21.15.89:0/4248482737 >> v1:172.21.15.89:6789/0 conn(0x7fe7e0159420 legacy=0x7fe7e00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:49.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7de7fc700 1 --2- 172.21.15.89:0/4248482737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7c8003a90 0x7fe7c8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:49.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7de7fc700 1 --2- 172.21.15.89:0/4248482737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7e015d230 0x7fe7c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:49.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7de7fc700 1 --2- 172.21.15.89:0/4248482737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7c8004d50 0x7fe7c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:49.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7de7fc700 1 -- 172.21.15.89:0/4248482737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e00b11d0 con 0x7fe7c8003a90 2023-10-10T20:42:49.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7de7fc700 1 -- 172.21.15.89:0/4248482737 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c800bb70 con 0x7fe7c8004d50 2023-10-10T20:42:49.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7de7fc700 1 -- 172.21.15.89:0/4248482737 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c800bd30 con 0x7fe7e015d230 2023-10-10T20:42:49.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7df7fe700 1 --2- 172.21.15.89:0/4248482737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7c8003a90 0x7fe7c8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:49.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7dffff700 1 --2- 172.21.15.89:0/4248482737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7c8004d50 0x7fe7c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:49.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7deffd700 1 --2- 172.21.15.89:0/4248482737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7e015d230 0x7fe7c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:49.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7df7fe700 1 -- 172.21.15.89:0/4248482737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7e015d230 msgr2=0x7fe7c8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:49.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7df7fe700 1 --2- 172.21.15.89:0/4248482737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7e015d230 0x7fe7c8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7df7fe700 1 -- 172.21.15.89:0/4248482737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7c8004d50 msgr2=0x7fe7c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:49.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7df7fe700 1 --2- 172.21.15.89:0/4248482737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7c8004d50 0x7fe7c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7df7fe700 1 -- 172.21.15.89:0/4248482737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c8003570 con 0x7fe7c8003a90 2023-10-10T20:42:49.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7df7fe700 1 --2- 172.21.15.89:0/4248482737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7c8003a90 0x7fe7c8003ea0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000c3c0 tx=0x7fe7d0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f9c00e5379a84e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:49.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7de7fc700 1 -- 172.21.15.89:0/4248482737 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d000cc90 con 0x7fe7c8003a90 2023-10-10T20:42:49.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7de7fc700 1 -- 172.21.15.89:0/4248482737 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d000cdf0 con 0x7fe7c8003a90 2023-10-10T20:42:49.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.867+0000 7fe7de7fc700 1 -- 172.21.15.89:0/4248482737 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0005b50 con 0x7fe7c8003a90 2023-10-10T20:42:49.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.868+0000 7fe7e49cd700 1 -- 172.21.15.89:0/4248482737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7c8003a90 msgr2=0x7fe7c8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:49.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.868+0000 7fe7e49cd700 1 --2- 172.21.15.89:0/4248482737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7c8003a90 0x7fe7c8003ea0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000c3c0 tx=0x7fe7d0001e50 comp rx=0 tx=0).stop 2023-10-10T20:42:49.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.868+0000 7fe7dffff700 1 -- 172.21.15.89:0/4248482737 reap_dead start 2023-10-10T20:42:49.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.868+0000 7fe7e49cd700 1 -- 172.21.15.89:0/4248482737 shutdown_connections 2023-10-10T20:42:49.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.868+0000 7fe7e49cd700 1 -- 172.21.15.89:0/4248482737 >> 172.21.15.89:0/4248482737 conn(0x7fe7e00d0660 msgr2=0x7fe7e00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:49.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.868+0000 7fe7e49cd700 1 -- 172.21.15.89:0/4248482737 shutdown_connections 2023-10-10T20:42:49.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.868+0000 7fe7e49cd700 1 -- 172.21.15.89:0/4248482737 wait complete. 2023-10-10T20:42:49.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.869+0000 7fe7e49cd700 1 Processor -- start 2023-10-10T20:42:49.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.869+0000 7fe7e49cd700 1 -- start start 2023-10-10T20:42:49.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.870+0000 7fe7e49cd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7c8003a90 0x7fe7e01550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:49.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.870+0000 7fe7e49cd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7e00cf950 0x7fe7e00cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:49.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.870+0000 7fe7e49cd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7e00c5470 0x7fe7e00c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:49.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.870+0000 7fe7e49cd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e00d02e0 con 0x7fe7e00cf950 2023-10-10T20:42:49.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.870+0000 7fe7e49cd700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e00c6d20 con 0x7fe7e00c5470 2023-10-10T20:42:49.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.870+0000 7fe7e49cd700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e00c61a0 con 0x7fe7c8003a90 2023-10-10T20:42:49.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.870+0000 7fe7deffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7e00cf950 0x7fe7e00cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:49.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.870+0000 7fe7deffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7e00cf950 0x7fe7e00cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54880/0 (socket says 172.21.15.89:54880) 2023-10-10T20:42:49.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.870+0000 7fe7deffd700 1 -- 172.21.15.89:0/2237000133 learned_addr learned my addr 172.21.15.89:0/2237000133 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:49.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.870+0000 7fe7df7fe700 1 --2- 172.21.15.89:0/2237000133 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7c8003a90 0x7fe7e01550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:49.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.870+0000 7fe7dffff700 1 --2- 172.21.15.89:0/2237000133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7e00c5470 0x7fe7e00c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:49.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.870+0000 7fe7deffd700 1 -- 172.21.15.89:0/2237000133 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7c8003a90 msgr2=0x7fe7e01550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:49.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.870+0000 7fe7deffd700 1 --2- 172.21.15.89:0/2237000133 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7c8003a90 0x7fe7e01550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.870+0000 7fe7deffd700 1 -- 172.21.15.89:0/2237000133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7e00c5470 msgr2=0x7fe7e00c58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:49.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.870+0000 7fe7deffd700 1 --2- 172.21.15.89:0/2237000133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7e00c5470 0x7fe7e00c58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:49.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.870+0000 7fe7deffd700 1 -- 172.21.15.89:0/2237000133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e00c6480 con 0x7fe7e00cf950 2023-10-10T20:42:49.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.870+0000 7fe7df7fe700 1 --2- 172.21.15.89:0/2237000133 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7c8003a90 0x7fe7e01550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:49.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.871+0000 7fe7deffd700 1 --2- 172.21.15.89:0/2237000133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7e00cf950 0x7fe7e00cfda0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400e810 tx=0x7fe7d4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:49.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.871+0000 7fe7c77fe700 1 -- 172.21.15.89:0/2237000133 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4012590 con 0x7fe7e00cf950 2023-10-10T20:42:49.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.871+0000 7fe7c77fe700 1 -- 172.21.15.89:0/2237000133 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d40126f0 con 0x7fe7e00cf950 2023-10-10T20:42:49.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.871+0000 7fe7e49cd700 1 -- 172.21.15.89:0/2237000133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e00c6760 con 0x7fe7e00cf950 2023-10-10T20:42:49.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.871+0000 7fe7c77fe700 1 -- 172.21.15.89:0/2237000133 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4012a90 con 0x7fe7e00cf950 2023-10-10T20:42:49.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.871+0000 7fe7e49cd700 1 -- 172.21.15.89:0/2237000133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e00c5e30 con 0x7fe7e00cf950 2023-10-10T20:42:49.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.872+0000 7fe7e49cd700 1 -- 172.21.15.89:0/2237000133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e004c3c0 con 0x7fe7e00cf950 2023-10-10T20:42:49.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.872+0000 7fe7c77fe700 1 -- 172.21.15.89:0/2237000133 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe7d401d460 con 0x7fe7e00cf950 2023-10-10T20:42:49.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.873+0000 7fe7c77fe700 1 --2- 172.21.15.89:0/2237000133 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe7cc067320 0x7fe7cc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:49.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.873+0000 7fe7df7fe700 1 --2- 172.21.15.89:0/2237000133 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe7cc067320 0x7fe7cc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:49.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.873+0000 7fe7c77fe700 1 -- 172.21.15.89:0/2237000133 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe7d4084a90 con 0x7fe7e00cf950 2023-10-10T20:42:49.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.873+0000 7fe7df7fe700 1 --2- 172.21.15.89:0/2237000133 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe7cc067320 0x7fe7cc0697d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00cd140 tx=0x7fe7d0015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:49.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:49.877+0000 7fe7c77fe700 1 -- 172.21.15.89:0/2237000133 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7d4051a30 con 0x7fe7e00cf950 2023-10-10T20:42:50.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.062+0000 7fe7e49cd700 1 -- 172.21.15.89:0/2237000133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7e00b1790 con 0x7fe7e00cf950 2023-10-10T20:42:50.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.063+0000 7fe7c77fe700 1 -- 172.21.15.89:0/2237000133 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+12144 (secure 0 0 0) 0x7fe7d4055050 con 0x7fe7e00cf950 2023-10-10T20:42:50.065 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:50.066 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":161,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:42:49.304903+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:42:50.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.065+0000 7fe7e49cd700 1 -- 172.21.15.89:0/2237000133 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe7cc067320 msgr2=0x7fe7cc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:50.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.065+0000 7fe7e49cd700 1 --2- 172.21.15.89:0/2237000133 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe7cc067320 0x7fe7cc0697d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00cd140 tx=0x7fe7d0015000 comp rx=0 tx=0).stop 2023-10-10T20:42:50.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.065+0000 7fe7e49cd700 1 -- 172.21.15.89:0/2237000133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7e00cf950 msgr2=0x7fe7e00cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:50.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.065+0000 7fe7e49cd700 1 --2- 172.21.15.89:0/2237000133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7e00cf950 0x7fe7e00cfda0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400e810 tx=0x7fe7d4009040 comp rx=0 tx=0).stop 2023-10-10T20:42:50.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.065+0000 7fe7e49cd700 1 -- 172.21.15.89:0/2237000133 shutdown_connections 2023-10-10T20:42:50.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.065+0000 7fe7e49cd700 1 --2- 172.21.15.89:0/2237000133 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe7cc067320 0x7fe7cc0697d0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.065+0000 7fe7e49cd700 1 --2- 172.21.15.89:0/2237000133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7e00c5470 0x7fe7e00c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.065+0000 7fe7e49cd700 1 --2- 172.21.15.89:0/2237000133 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7c8003a90 0x7fe7e01550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.065+0000 7fe7e49cd700 1 --2- 172.21.15.89:0/2237000133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7e00cf950 0x7fe7e00cfda0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.065+0000 7fe7e49cd700 1 -- 172.21.15.89:0/2237000133 >> 172.21.15.89:0/2237000133 conn(0x7fe7e00d0660 msgr2=0x7fe7e0068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:50.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.065+0000 7fe7e49cd700 1 -- 172.21.15.89:0/2237000133 shutdown_connections 2023-10-10T20:42:50.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.066+0000 7fe7e49cd700 1 -- 172.21.15.89:0/2237000133 wait complete. 2023-10-10T20:42:50.080 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:42:50.081 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:42:50.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.196+0000 7f288f59e700 1 Processor -- start 2023-10-10T20:42:50.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.196+0000 7f288f59e700 1 -- start start 2023-10-10T20:42:50.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.197+0000 7f288f59e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f28900b4e70 0x7f28900ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:50.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.197+0000 7f288f59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28900b1b40 0x7f289015d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:50.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.197+0000 7f288f59e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28900cfb00 con 0x7f28900b16d0 2023-10-10T20:42:50.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.197+0000 7f288f59e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28900b5c40 con 0x7f28900b4e70 2023-10-10T20:42:50.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.197+0000 7f288f59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28900b5f20 con 0x7f28900b1b40 2023-10-10T20:42:50.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.197+0000 7f288e59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f28900b4e70 0x7f28900ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:50.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.197+0000 7f2887fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f28900b16d0 0x7f289015d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48760/0 (socket says 172.21.15.89:48760) 2023-10-10T20:42:50.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.197+0000 7f2887fff700 1 -- 172.21.15.89:0/3578515562 learned_addr learned my addr 172.21.15.89:0/3578515562 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:50.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.197+0000 7f288ed9d700 1 --2- 172.21.15.89:0/3578515562 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28900b1b40 0x7f289015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:50.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.197+0000 7f288e59c700 1 -- 172.21.15.89:0/3578515562 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28900b1b40 msgr2=0x7f289015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:50.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.197+0000 7f288e59c700 1 --2- 172.21.15.89:0/3578515562 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28900b1b40 0x7f289015d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.197+0000 7f288e59c700 1 -- 172.21.15.89:0/3578515562 >> v1:172.21.15.89:6789/0 conn(0x7f28900b16d0 legacy=0x7f289015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:50.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.197+0000 7f288e59c700 1 -- 172.21.15.89:0/3578515562 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28900b6200 con 0x7f28900b4e70 2023-10-10T20:42:50.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.197+0000 7f288dd9b700 1 -- 172.21.15.89:0/3578515562 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3651546763 0 0) 0x7f28900cfb00 con 0x7f28900b16d0 2023-10-10T20:42:50.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.197+0000 7f288ed9d700 1 --2- 172.21.15.89:0/3578515562 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28900b1b40 0x7f289015d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:50.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.197+0000 7f288e59c700 1 --2- 172.21.15.89:0/3578515562 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28900b4e70 0x7f28900ccff0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f28800098e0 tx=0x7f28800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696daecf9a2a6ade server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:50.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.198+0000 7f288dd9b700 1 -- 172.21.15.89:0/3578515562 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880009e80 con 0x7f28900b4e70 2023-10-10T20:42:50.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.198+0000 7f288dd9b700 1 -- 172.21.15.89:0/3578515562 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288000b860 con 0x7f28900b4e70 2023-10-10T20:42:50.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.198+0000 7f288dd9b700 1 -- 172.21.15.89:0/3578515562 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288000bbe0 con 0x7f28900b4e70 2023-10-10T20:42:50.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.198+0000 7f288f59e700 1 -- 172.21.15.89:0/3578515562 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28900b4e70 msgr2=0x7f28900ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:50.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.198+0000 7f288f59e700 1 --2- 172.21.15.89:0/3578515562 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28900b4e70 0x7f28900ccff0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f28800098e0 tx=0x7f28800091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.198+0000 7f288f59e700 1 -- 172.21.15.89:0/3578515562 shutdown_connections 2023-10-10T20:42:50.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.198+0000 7f288f59e700 1 --2- 172.21.15.89:0/3578515562 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28900b4e70 0x7f28900ccff0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.198+0000 7f288f59e700 1 --2- 172.21.15.89:0/3578515562 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28900b1b40 0x7f289015d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.198+0000 7f288f59e700 1 -- 172.21.15.89:0/3578515562 >> 172.21.15.89:0/3578515562 conn(0x7f28900d10c0 msgr2=0x7f28900affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:50.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.198+0000 7f288f59e700 1 -- 172.21.15.89:0/3578515562 shutdown_connections 2023-10-10T20:42:50.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.198+0000 7f288f59e700 1 -- 172.21.15.89:0/3578515562 wait complete. 2023-10-10T20:42:50.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.199+0000 7f288f59e700 1 Processor -- start 2023-10-10T20:42:50.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.199+0000 7f288f59e700 1 -- start start 2023-10-10T20:42:50.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.200+0000 7f288f59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28900b16d0 0x7f28900c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:50.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.200+0000 7f288f59e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28900b1b40 0x7f28900c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:50.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.200+0000 7f288f59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28900b4e70 0x7f2890068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:50.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.200+0000 7f288f59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28900c65d0 con 0x7f28900b4e70 2023-10-10T20:42:50.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.200+0000 7f288f59e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289016b000 con 0x7f28900b1b40 2023-10-10T20:42:50.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.200+0000 7f288f59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28900c5d30 con 0x7f28900b16d0 2023-10-10T20:42:50.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.200+0000 7f288ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28900b4e70 0x7f2890068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:50.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.200+0000 7f288ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28900b4e70 0x7f2890068ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54894/0 (socket says 172.21.15.89:54894) 2023-10-10T20:42:50.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.200+0000 7f288ed9d700 1 -- 172.21.15.89:0/3589129092 learned_addr learned my addr 172.21.15.89:0/3589129092 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:50.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.200+0000 7f288ed9d700 1 -- 172.21.15.89:0/3589129092 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28900b16d0 msgr2=0x7f28900c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:50.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.200+0000 7f288e59c700 1 --2- 172.21.15.89:0/3589129092 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28900b16d0 0x7f28900c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:50.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.200+0000 7f288ed9d700 1 --2- 172.21.15.89:0/3589129092 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28900b16d0 0x7f28900c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.200+0000 7f288ed9d700 1 -- 172.21.15.89:0/3589129092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28900b1b40 msgr2=0x7f28900c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:50.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.200+0000 7f2887fff700 1 --2- 172.21.15.89:0/3589129092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28900b1b40 0x7f28900c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:50.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.200+0000 7f288ed9d700 1 --2- 172.21.15.89:0/3589129092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28900b1b40 0x7f28900c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.200+0000 7f288ed9d700 1 -- 172.21.15.89:0/3589129092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2880009420 con 0x7f28900b4e70 2023-10-10T20:42:50.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.201+0000 7f288ed9d700 1 --2- 172.21.15.89:0/3589129092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28900b4e70 0x7f2890068ff0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f288800a380 tx=0x7f2888007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:50.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.201+0000 7f28867fc700 1 -- 172.21.15.89:0/3589129092 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288800ae10 con 0x7f28900b4e70 2023-10-10T20:42:50.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.201+0000 7f28867fc700 1 -- 172.21.15.89:0/3589129092 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288800d650 con 0x7f28900b4e70 2023-10-10T20:42:50.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.201+0000 7f288f59e700 1 -- 172.21.15.89:0/3589129092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28900c6010 con 0x7f28900b4e70 2023-10-10T20:42:50.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.201+0000 7f28867fc700 1 -- 172.21.15.89:0/3589129092 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288800da10 con 0x7f28900b4e70 2023-10-10T20:42:50.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.201+0000 7f288f59e700 1 -- 172.21.15.89:0/3589129092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28900c58d0 con 0x7f28900b4e70 2023-10-10T20:42:50.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.202+0000 7f288f59e700 1 -- 172.21.15.89:0/3589129092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2890088990 con 0x7f28900b4e70 2023-10-10T20:42:50.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.202+0000 7f28867fc700 1 -- 172.21.15.89:0/3589129092 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f288801e3b0 con 0x7f28900b4e70 2023-10-10T20:42:50.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.203+0000 7f28867fc700 1 --2- 172.21.15.89:0/3589129092 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2870067320 0x7f28700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:50.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.203+0000 7f288e59c700 1 --2- 172.21.15.89:0/3589129092 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2870067320 0x7f28700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:50.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.203+0000 7f28867fc700 1 -- 172.21.15.89:0/3589129092 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2888085330 con 0x7f28900b4e70 2023-10-10T20:42:50.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.203+0000 7f288e59c700 1 --2- 172.21.15.89:0/3589129092 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2870067320 0x7f28700697d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f28900b3e80 tx=0x7f2880017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:50.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.207+0000 7f28867fc700 1 -- 172.21.15.89:0/3589129092 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2888052310 con 0x7f28900b4e70 2023-10-10T20:42:50.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.397+0000 7f288f59e700 1 -- 172.21.15.89:0/3589129092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f289004bb80 con 0x7f28900b4e70 2023-10-10T20:42:50.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.398+0000 7f28867fc700 1 -- 172.21.15.89:0/3589129092 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v161) v1 ==== 93+0+10 (secure 0 0 0) 0x7f288801a020 con 0x7f28900b4e70 2023-10-10T20:42:50.400 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:42:50.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.400+0000 7f288f59e700 1 -- 172.21.15.89:0/3589129092 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2870067320 msgr2=0x7f28700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:50.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.400+0000 7f288f59e700 1 --2- 172.21.15.89:0/3589129092 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2870067320 0x7f28700697d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f28900b3e80 tx=0x7f2880017000 comp rx=0 tx=0).stop 2023-10-10T20:42:50.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.400+0000 7f288f59e700 1 -- 172.21.15.89:0/3589129092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28900b4e70 msgr2=0x7f2890068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:50.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.400+0000 7f288f59e700 1 --2- 172.21.15.89:0/3589129092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28900b4e70 0x7f2890068ff0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f288800a380 tx=0x7f2888007900 comp rx=0 tx=0).stop 2023-10-10T20:42:50.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.400+0000 7f288f59e700 1 -- 172.21.15.89:0/3589129092 shutdown_connections 2023-10-10T20:42:50.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.400+0000 7f288f59e700 1 --2- 172.21.15.89:0/3589129092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28900b4e70 0x7f2890068ff0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.400+0000 7f288f59e700 1 --2- 172.21.15.89:0/3589129092 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f28900b16d0 0x7f28900c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.400+0000 7f288f59e700 1 --2- 172.21.15.89:0/3589129092 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2870067320 0x7f28700697d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.400+0000 7f288f59e700 1 --2- 172.21.15.89:0/3589129092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28900b1b40 0x7f28900c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.400+0000 7f288f59e700 1 -- 172.21.15.89:0/3589129092 >> 172.21.15.89:0/3589129092 conn(0x7f28900d10c0 msgr2=0x7f2890159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:50.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.400+0000 7f288f59e700 1 -- 172.21.15.89:0/3589129092 shutdown_connections 2023-10-10T20:42:50.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.400+0000 7f288f59e700 1 -- 172.21.15.89:0/3589129092 wait complete. 2023-10-10T20:42:50.414 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:42:50.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.527+0000 7fad1aecc700 1 Processor -- start 2023-10-10T20:42:50.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.527+0000 7fad1aecc700 1 -- start start 2023-10-10T20:42:50.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.528+0000 7fad1aecc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad1406c0a0 0x7fad140caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:50.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.528+0000 7fad1aecc700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fad1415df50 0x7fad1415e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.528+0000 7fad1aecc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1415d7e0 con 0x7fad1406bcd0 2023-10-10T20:42:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.528+0000 7fad1aecc700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1415dac0 con 0x7fad1415df50 2023-10-10T20:42:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.528+0000 7fad1aecc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1415aa30 con 0x7fad1406c0a0 2023-10-10T20:42:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.528+0000 7fad18c68700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fad1406bcd0 0x7fad1415a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48772/0 (socket says 172.21.15.89:48772) 2023-10-10T20:42:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.528+0000 7fad18c68700 1 -- 172.21.15.89:0/3629767136 learned_addr learned my addr 172.21.15.89:0/3629767136 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.528+0000 7fad19469700 1 --2- 172.21.15.89:0/3629767136 >> v2:172.21.15.111:3300/0 conn(0x7fad1415df50 0x7fad1415e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.528+0000 7fad13fff700 1 --2- 172.21.15.89:0/3629767136 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad1406c0a0 0x7fad140caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.528+0000 7fad137fe700 1 -- 172.21.15.89:0/3629767136 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1161148728 0 0) 0x7fad1415d7e0 con 0x7fad1406bcd0 2023-10-10T20:42:50.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.528+0000 7fad137fe700 1 -- 172.21.15.89:0/3629767136 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad00003570 con 0x7fad1406bcd0 2023-10-10T20:42:50.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.528+0000 7fad137fe700 1 -- 172.21.15.89:0/3629767136 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3196521709 0 0) 0x7fad00003570 con 0x7fad1406bcd0 2023-10-10T20:42:50.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.528+0000 7fad19469700 1 -- 172.21.15.89:0/3629767136 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad1406c0a0 msgr2=0x7fad140caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:50.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.528+0000 7fad19469700 1 --2- 172.21.15.89:0/3629767136 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad1406c0a0 0x7fad140caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.528+0000 7fad19469700 1 -- 172.21.15.89:0/3629767136 >> v1:172.21.15.89:6789/0 conn(0x7fad1406bcd0 legacy=0x7fad1415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.528+0000 7fad19469700 1 -- 172.21.15.89:0/3629767136 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1415ad10 con 0x7fad1415df50 2023-10-10T20:42:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.528+0000 7fad13fff700 1 --2- 172.21.15.89:0/3629767136 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad1406c0a0 0x7fad140caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:42:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.529+0000 7fad19469700 1 --2- 172.21.15.89:0/3629767136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1415df50 0x7fad1415e3c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7facfc0098e0 tx=0x7facfc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=305c6b83c5bc5510 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.529+0000 7fad137fe700 1 -- 172.21.15.89:0/3629767136 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c001470 con 0x7fad1415df50 2023-10-10T20:42:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.529+0000 7fad137fe700 1 -- 172.21.15.89:0/3629767136 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facfc011420 con 0x7fad1415df50 2023-10-10T20:42:50.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.529+0000 7fad137fe700 1 -- 172.21.15.89:0/3629767136 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc0117c0 con 0x7fad1415df50 2023-10-10T20:42:50.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.529+0000 7fad1aecc700 1 -- 172.21.15.89:0/3629767136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1415df50 msgr2=0x7fad1415e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:50.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.529+0000 7fad1aecc700 1 --2- 172.21.15.89:0/3629767136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1415df50 0x7fad1415e3c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7facfc0098e0 tx=0x7facfc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.529+0000 7fad1aecc700 1 -- 172.21.15.89:0/3629767136 shutdown_connections 2023-10-10T20:42:50.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.529+0000 7fad1aecc700 1 --2- 172.21.15.89:0/3629767136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1415df50 0x7fad1415e3c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.529+0000 7fad1aecc700 1 --2- 172.21.15.89:0/3629767136 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad1406c0a0 0x7fad140caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.529+0000 7fad1aecc700 1 -- 172.21.15.89:0/3629767136 >> 172.21.15.89:0/3629767136 conn(0x7fad140ce600 msgr2=0x7fad140cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:50.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.529+0000 7fad1aecc700 1 -- 172.21.15.89:0/3629767136 shutdown_connections 2023-10-10T20:42:50.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.529+0000 7fad1aecc700 1 -- 172.21.15.89:0/3629767136 wait complete. 2023-10-10T20:42:50.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.530+0000 7fad1aecc700 1 Processor -- start 2023-10-10T20:42:50.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.530+0000 7fad1aecc700 1 -- start start 2023-10-10T20:42:50.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.531+0000 7fad1aecc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad1406bcd0 0x7fad1415c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:50.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.531+0000 7fad1aecc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1406c0a0 0x7fad141536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:50.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.531+0000 7fad1aecc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad1415df50 0x7fad14153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:50.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.531+0000 7fad1aecc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1416a7d0 con 0x7fad1415df50 2023-10-10T20:42:50.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.531+0000 7fad1aecc700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad14154140 con 0x7fad1406c0a0 2023-10-10T20:42:50.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.531+0000 7fad1aecc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad14169b90 con 0x7fad1406bcd0 2023-10-10T20:42:50.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.531+0000 7fad19469700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad1415df50 0x7fad14153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:50.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.531+0000 7fad19469700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad1415df50 0x7fad14153c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54896/0 (socket says 172.21.15.89:54896) 2023-10-10T20:42:50.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.531+0000 7fad19469700 1 -- 172.21.15.89:0/2101451148 learned_addr learned my addr 172.21.15.89:0/2101451148 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:50.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.532+0000 7fad13fff700 1 --2- 172.21.15.89:0/2101451148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1406c0a0 0x7fad141536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:50.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.532+0000 7fad18c68700 1 --2- 172.21.15.89:0/2101451148 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad1406bcd0 0x7fad1415c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:50.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.532+0000 7fad19469700 1 -- 172.21.15.89:0/2101451148 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad1406bcd0 msgr2=0x7fad1415c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:50.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.532+0000 7fad19469700 1 --2- 172.21.15.89:0/2101451148 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad1406bcd0 0x7fad1415c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.532+0000 7fad19469700 1 -- 172.21.15.89:0/2101451148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1406c0a0 msgr2=0x7fad141536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:50.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.532+0000 7fad19469700 1 --2- 172.21.15.89:0/2101451148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1406c0a0 0x7fad141536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.532+0000 7fad19469700 1 -- 172.21.15.89:0/2101451148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad04009a20 con 0x7fad1415df50 2023-10-10T20:42:50.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.532+0000 7fad19469700 1 --2- 172.21.15.89:0/2101451148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad1415df50 0x7fad14153c00 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7facfc0030c0 tx=0x7facfc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:50.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.532+0000 7fad10ff9700 1 -- 172.21.15.89:0/2101451148 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc01cc90 con 0x7fad1415df50 2023-10-10T20:42:50.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.532+0000 7fad10ff9700 1 -- 172.21.15.89:0/2101451148 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facfc01cdf0 con 0x7fad1415df50 2023-10-10T20:42:50.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.532+0000 7fad10ff9700 1 -- 172.21.15.89:0/2101451148 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc01a680 con 0x7fad1415df50 2023-10-10T20:42:50.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.532+0000 7fad1aecc700 1 -- 172.21.15.89:0/2101451148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facfc009420 con 0x7fad1415df50 2023-10-10T20:42:50.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.532+0000 7fad1aecc700 1 -- 172.21.15.89:0/2101451148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1416a110 con 0x7fad1415df50 2023-10-10T20:42:50.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.533+0000 7fad1aecc700 1 -- 172.21.15.89:0/2101451148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1416a910 con 0x7fad1415df50 2023-10-10T20:42:50.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.533+0000 7fad10ff9700 1 -- 172.21.15.89:0/2101451148 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7facfc01aa70 con 0x7fad1415df50 2023-10-10T20:42:50.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.534+0000 7fad10ff9700 1 --2- 172.21.15.89:0/2101451148 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7facf4067320 0x7facf40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:50.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.534+0000 7fad10ff9700 1 -- 172.21.15.89:0/2101451148 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 4853+0+0 (secure 0 0 0) 0x7facfc086360 con 0x7fad1415df50 2023-10-10T20:42:50.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.534+0000 7fad18c68700 1 --2- 172.21.15.89:0/2101451148 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7facf4067320 0x7facf40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:50.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.534+0000 7fad18c68700 1 --2- 172.21.15.89:0/2101451148 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7facf4067320 0x7facf40697d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fad14156180 tx=0x7fad0c001540 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:50.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.538+0000 7fad10ff9700 1 -- 172.21.15.89:0/2101451148 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7facfc053340 con 0x7fad1415df50 2023-10-10T20:42:50.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.722+0000 7fad1aecc700 1 -- 172.21.15.89:0/2101451148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fad1404c3c0 con 0x7fad1415df50 2023-10-10T20:42:50.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.722+0000 7fad10ff9700 1 -- 172.21.15.89:0/2101451148 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v161) v1 ==== 84+0+65 (secure 0 0 0) 0x7facfc056960 con 0x7fad1415df50 2023-10-10T20:42:50.725 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:50.725 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:42:50.725 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:42:50.725 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2023-10-10T20:42:50.726 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2023-10-10T20:42:50.726 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:42:50.726 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:42:50.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.724+0000 7fad1aecc700 1 -- 172.21.15.89:0/2101451148 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7facf4067320 msgr2=0x7facf40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:50.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.724+0000 7fad1aecc700 1 --2- 172.21.15.89:0/2101451148 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7facf4067320 0x7facf40697d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fad14156180 tx=0x7fad0c001540 comp rx=0 tx=0).stop 2023-10-10T20:42:50.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.724+0000 7fad1aecc700 1 -- 172.21.15.89:0/2101451148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad1415df50 msgr2=0x7fad14153c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:50.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.724+0000 7fad1aecc700 1 --2- 172.21.15.89:0/2101451148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad1415df50 0x7fad14153c00 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7facfc0030c0 tx=0x7facfc016000 comp rx=0 tx=0).stop 2023-10-10T20:42:50.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.724+0000 7fad1aecc700 1 -- 172.21.15.89:0/2101451148 shutdown_connections 2023-10-10T20:42:50.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.724+0000 7fad1aecc700 1 --2- 172.21.15.89:0/2101451148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad1415df50 0x7fad14153c00 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.724+0000 7fad1aecc700 1 --2- 172.21.15.89:0/2101451148 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad1406bcd0 0x7fad1415c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.724+0000 7fad1aecc700 1 --2- 172.21.15.89:0/2101451148 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7facf4067320 0x7facf40697d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.724+0000 7fad1aecc700 1 --2- 172.21.15.89:0/2101451148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1406c0a0 0x7fad141536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.724+0000 7fad1aecc700 1 -- 172.21.15.89:0/2101451148 >> 172.21.15.89:0/2101451148 conn(0x7fad140ce600 msgr2=0x7fad14069480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:50.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.724+0000 7fad1aecc700 1 -- 172.21.15.89:0/2101451148 shutdown_connections 2023-10-10T20:42:50.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.724+0000 7fad1aecc700 1 -- 172.21.15.89:0/2101451148 wait complete. 2023-10-10T20:42:50.738 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:42:50.739 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:42:50.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.855+0000 7fc3215e5700 1 Processor -- start 2023-10-10T20:42:50.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.855+0000 7fc3215e5700 1 -- start start 2023-10-10T20:42:50.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.856+0000 7fc3215e5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc31c0b42c0 0x7fc31c0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:50.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.856+0000 7fc3215e5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc31c0cefd0 con 0x7fc31c0ceb60 2023-10-10T20:42:50.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.856+0000 7fc3215e5700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc31c15c320 con 0x7fc31c155360 2023-10-10T20:42:50.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.856+0000 7fc3215e5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc31c15b710 con 0x7fc31c0b42c0 2023-10-10T20:42:50.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.856+0000 7fc31a7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc31c0b42c0 0x7fc31c0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:50.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.856+0000 7fc31b7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc31c0ceb60 0x7fc31c0cc9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48788/0 (socket says 172.21.15.89:48788) 2023-10-10T20:42:50.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.856+0000 7fc31b7fe700 1 -- 172.21.15.89:0/1265441849 learned_addr learned my addr 172.21.15.89:0/1265441849 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:50.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.857+0000 7fc31a7fc700 1 -- 172.21.15.89:0/1265441849 >> v1:172.21.15.111:6789/0 conn(0x7fc31c155360 legacy=0x7fc31c0b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:50.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.857+0000 7fc31a7fc700 1 -- 172.21.15.89:0/1265441849 >> v1:172.21.15.89:6789/0 conn(0x7fc31c0ceb60 legacy=0x7fc31c0cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:50.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.857+0000 7fc319ffb700 1 -- 172.21.15.89:0/1265441849 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 575973246 0 0) 0x7fc314001110 con 0x7fc31c0ceb60 2023-10-10T20:42:50.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.857+0000 7fc31a7fc700 1 -- 172.21.15.89:0/1265441849 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31c15b9f0 con 0x7fc31c0b42c0 2023-10-10T20:42:50.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.857+0000 7fc31a7fc700 1 --2- 172.21.15.89:0/1265441849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc31c0b42c0 0x7fc31c0ce620 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fc3100098e0 tx=0x7fc3100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e3d53241f7dfe2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:50.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.857+0000 7fc319ffb700 1 -- 172.21.15.89:0/1265441849 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310009e80 con 0x7fc31c0b42c0 2023-10-10T20:42:50.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.857+0000 7fc319ffb700 1 -- 172.21.15.89:0/1265441849 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31000b860 con 0x7fc31c0b42c0 2023-10-10T20:42:50.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.857+0000 7fc319ffb700 1 -- 172.21.15.89:0/1265441849 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31000bbe0 con 0x7fc31c0b42c0 2023-10-10T20:42:50.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.857+0000 7fc3215e5700 1 -- 172.21.15.89:0/1265441849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc31c0b42c0 msgr2=0x7fc31c0ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:50.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.857+0000 7fc3215e5700 1 --2- 172.21.15.89:0/1265441849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc31c0b42c0 0x7fc31c0ce620 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fc3100098e0 tx=0x7fc3100091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.857+0000 7fc3215e5700 1 -- 172.21.15.89:0/1265441849 shutdown_connections 2023-10-10T20:42:50.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.857+0000 7fc3215e5700 1 --2- 172.21.15.89:0/1265441849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc31c0b42c0 0x7fc31c0ce620 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.857+0000 7fc3215e5700 1 -- 172.21.15.89:0/1265441849 >> 172.21.15.89:0/1265441849 conn(0x7fc31c06baf0 msgr2=0x7fc31c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:50.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.858+0000 7fc3215e5700 1 -- 172.21.15.89:0/1265441849 shutdown_connections 2023-10-10T20:42:50.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.858+0000 7fc3215e5700 1 -- 172.21.15.89:0/1265441849 wait complete. 2023-10-10T20:42:50.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.859+0000 7fc3215e5700 1 Processor -- start 2023-10-10T20:42:50.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.859+0000 7fc3215e5700 1 -- start start 2023-10-10T20:42:50.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.859+0000 7fc3215e5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc31c0b42c0 0x7fc31c14f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:50.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.859+0000 7fc3215e5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc31c0ceb60 0x7fc31c0c33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:50.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.859+0000 7fc3215e5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc31c155360 0x7fc31c0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.859+0000 7fc3215e5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c153570 con 0x7fc31c155360 2023-10-10T20:42:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.859+0000 7fc3215e5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c1529a0 con 0x7fc31c0ceb60 2023-10-10T20:42:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.859+0000 7fc3215e5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c153a00 con 0x7fc31c0b42c0 2023-10-10T20:42:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.860+0000 7fc31b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc31c155360 0x7fc31c0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.860+0000 7fc31b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc31c155360 0x7fc31c0af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54926/0 (socket says 172.21.15.89:54926) 2023-10-10T20:42:50.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.860+0000 7fc31b7fe700 1 -- 172.21.15.89:0/2694441242 learned_addr learned my addr 172.21.15.89:0/2694441242 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:50.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.860+0000 7fc31a7fc700 1 --2- 172.21.15.89:0/2694441242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc31c0ceb60 0x7fc31c0c33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:50.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.860+0000 7fc31affd700 1 --2- 172.21.15.89:0/2694441242 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc31c0b42c0 0x7fc31c14f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:50.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.860+0000 7fc31b7fe700 1 -- 172.21.15.89:0/2694441242 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc31c0b42c0 msgr2=0x7fc31c14f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:50.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.860+0000 7fc31b7fe700 1 --2- 172.21.15.89:0/2694441242 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc31c0b42c0 0x7fc31c14f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.860+0000 7fc31b7fe700 1 -- 172.21.15.89:0/2694441242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc31c0ceb60 msgr2=0x7fc31c0c33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.860+0000 7fc31b7fe700 1 --2- 172.21.15.89:0/2694441242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc31c0ceb60 0x7fc31c0c33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.860+0000 7fc31b7fe700 1 -- 172.21.15.89:0/2694441242 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc310009420 con 0x7fc31c155360 2023-10-10T20:42:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.860+0000 7fc31b7fe700 1 --2- 172.21.15.89:0/2694441242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc31c155360 0x7fc31c0af4d0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fc31400c810 tx=0x7fc314007b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.860+0000 7fc302ffd700 1 -- 172.21.15.89:0/2694441242 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc314005b30 con 0x7fc31c155360 2023-10-10T20:42:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.860+0000 7fc302ffd700 1 -- 172.21.15.89:0/2694441242 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc314005cb0 con 0x7fc31c155360 2023-10-10T20:42:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.860+0000 7fc3215e5700 1 -- 172.21.15.89:0/2694441242 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc31c152c40 con 0x7fc31c155360 2023-10-10T20:42:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.860+0000 7fc302ffd700 1 -- 172.21.15.89:0/2694441242 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc314007320 con 0x7fc31c155360 2023-10-10T20:42:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.860+0000 7fc3215e5700 1 -- 172.21.15.89:0/2694441242 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc31c153200 con 0x7fc31c155360 2023-10-10T20:42:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.861+0000 7fc3215e5700 1 -- 172.21.15.89:0/2694441242 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc31c157960 con 0x7fc31c155360 2023-10-10T20:42:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.862+0000 7fc302ffd700 1 -- 172.21.15.89:0/2694441242 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc31401b3b0 con 0x7fc31c155360 2023-10-10T20:42:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.862+0000 7fc302ffd700 1 --2- 172.21.15.89:0/2694441242 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc308067320 0x7fc3080697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:50.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.862+0000 7fc302ffd700 1 -- 172.21.15.89:0/2694441242 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc314083b60 con 0x7fc31c155360 2023-10-10T20:42:50.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.862+0000 7fc31affd700 1 --2- 172.21.15.89:0/2694441242 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc308067320 0x7fc3080697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:50.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.863+0000 7fc31affd700 1 --2- 172.21.15.89:0/2694441242 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc308067320 0x7fc3080697d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc30c007fd0 tx=0x7fc30c001650 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:50.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:50.866+0000 7fc302ffd700 1 -- 172.21.15.89:0/2694441242 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc314050b40 con 0x7fc31c155360 2023-10-10T20:42:51.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.048+0000 7fc3215e5700 1 -- 172.21.15.89:0/2694441242 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc31c0cefd0 con 0x7fc31c155360 2023-10-10T20:42:51.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.322+0000 7fc302ffd700 1 -- 172.21.15.89:0/2694441242 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v162) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc31c0cefd0 con 0x7fc31c155360 2023-10-10T20:42:51.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.325+0000 7fc3215e5700 1 -- 172.21.15.89:0/2694441242 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc308067320 msgr2=0x7fc3080697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:51.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.325+0000 7fc3215e5700 1 --2- 172.21.15.89:0/2694441242 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc308067320 0x7fc3080697d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc30c007fd0 tx=0x7fc30c001650 comp rx=0 tx=0).stop 2023-10-10T20:42:51.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.325+0000 7fc3215e5700 1 -- 172.21.15.89:0/2694441242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc31c155360 msgr2=0x7fc31c0af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:51.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.325+0000 7fc3215e5700 1 --2- 172.21.15.89:0/2694441242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc31c155360 0x7fc31c0af4d0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fc31400c810 tx=0x7fc314007b90 comp rx=0 tx=0).stop 2023-10-10T20:42:51.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.325+0000 7fc3215e5700 1 -- 172.21.15.89:0/2694441242 shutdown_connections 2023-10-10T20:42:51.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.325+0000 7fc3215e5700 1 --2- 172.21.15.89:0/2694441242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc31c155360 0x7fc31c0af4d0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:51.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.325+0000 7fc3215e5700 1 --2- 172.21.15.89:0/2694441242 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc31c0b42c0 0x7fc31c14f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:51.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.325+0000 7fc3215e5700 1 --2- 172.21.15.89:0/2694441242 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc308067320 0x7fc3080697d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:51.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.325+0000 7fc3215e5700 1 --2- 172.21.15.89:0/2694441242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc31c0ceb60 0x7fc31c0c33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:51.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.325+0000 7fc3215e5700 1 -- 172.21.15.89:0/2694441242 >> 172.21.15.89:0/2694441242 conn(0x7fc31c06baf0 msgr2=0x7fc31c0b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:51.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.325+0000 7fc3215e5700 1 -- 172.21.15.89:0/2694441242 shutdown_connections 2023-10-10T20:42:51.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.325+0000 7fc3215e5700 1 -- 172.21.15.89:0/2694441242 wait complete. 2023-10-10T20:42:51.330 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2023-10-10T20:42:51.340 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:42:51.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.462+0000 7f6a4d2e4700 1 Processor -- start 2023-10-10T20:42:51.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.462+0000 7f6a4d2e4700 1 -- start start 2023-10-10T20:42:51.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.463+0000 7f6a4d2e4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6a480cf480 0x7f6a4815d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:51.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.463+0000 7f6a4d2e4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a480b10f0 con 0x7f6a4815d760 2023-10-10T20:42:51.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.463+0000 7f6a4d2e4700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a4815db80 con 0x7f6a481595d0 2023-10-10T20:42:51.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.463+0000 7f6a4d2e4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a4815e000 con 0x7f6a480cf480 2023-10-10T20:42:51.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.463+0000 7f6a477fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6a480cf480 0x7f6a4815d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:51.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.463+0000 7f6a4cae3700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6a4815d760 0x7f6a480b1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48798/0 (socket says 172.21.15.89:48798) 2023-10-10T20:42:51.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.463+0000 7f6a4cae3700 1 -- 172.21.15.89:0/1834571766 learned_addr learned my addr 172.21.15.89:0/1834571766 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:51.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.463+0000 7f6a477fe700 1 -- 172.21.15.89:0/1834571766 >> v1:172.21.15.111:6789/0 conn(0x7f6a481595d0 legacy=0x7f6a480ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:51.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.463+0000 7f6a477fe700 1 -- 172.21.15.89:0/1834571766 >> v1:172.21.15.89:6789/0 conn(0x7f6a4815d760 legacy=0x7f6a480b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:51.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.463+0000 7f6a477fe700 1 -- 172.21.15.89:0/1834571766 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a480cf990 con 0x7f6a480cf480 2023-10-10T20:42:51.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.463+0000 7f6a477fe700 1 --2- 172.21.15.89:0/1834571766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a480cf480 0x7f6a4815d220 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f6a380098e0 tx=0x7f6a380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b186468b86ec8a1c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:51.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.463+0000 7f6a46ffd700 1 -- 172.21.15.89:0/1834571766 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38009e80 con 0x7f6a480cf480 2023-10-10T20:42:51.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.463+0000 7f6a46ffd700 1 -- 172.21.15.89:0/1834571766 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3800b860 con 0x7f6a480cf480 2023-10-10T20:42:51.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.463+0000 7f6a46ffd700 1 -- 172.21.15.89:0/1834571766 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3800bbe0 con 0x7f6a480cf480 2023-10-10T20:42:51.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.464+0000 7f6a4d2e4700 1 -- 172.21.15.89:0/1834571766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a480cf480 msgr2=0x7f6a4815d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:51.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.464+0000 7f6a4d2e4700 1 --2- 172.21.15.89:0/1834571766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a480cf480 0x7f6a4815d220 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f6a380098e0 tx=0x7f6a380091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:51.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.464+0000 7f6a4d2e4700 1 -- 172.21.15.89:0/1834571766 shutdown_connections 2023-10-10T20:42:51.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.464+0000 7f6a4d2e4700 1 --2- 172.21.15.89:0/1834571766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a480cf480 0x7f6a4815d220 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:51.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.464+0000 7f6a4d2e4700 1 -- 172.21.15.89:0/1834571766 >> 172.21.15.89:0/1834571766 conn(0x7f6a480d05f0 msgr2=0x7f6a480d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:51.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.464+0000 7f6a4d2e4700 1 -- 172.21.15.89:0/1834571766 shutdown_connections 2023-10-10T20:42:51.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.464+0000 7f6a4d2e4700 1 -- 172.21.15.89:0/1834571766 wait complete. 2023-10-10T20:42:51.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.465+0000 7f6a4d2e4700 1 Processor -- start 2023-10-10T20:42:51.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.465+0000 7f6a4d2e4700 1 -- start start 2023-10-10T20:42:51.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.466+0000 7f6a4d2e4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a480cf480 0x7f6a48152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:51.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.466+0000 7f6a4d2e4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a481595d0 0x7f6a480c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:51.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.466+0000 7f6a4d2e4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a4815d760 0x7f6a4816aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:51.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.466+0000 7f6a4d2e4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a48152aa0 con 0x7f6a4815d760 2023-10-10T20:42:51.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.466+0000 7f6a4d2e4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a48169e40 con 0x7f6a480cf480 2023-10-10T20:42:51.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.466+0000 7f6a4d2e4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a481554c0 con 0x7f6a481595d0 2023-10-10T20:42:51.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.466+0000 7f6a4cae3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a4815d760 0x7f6a4816aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:51.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.466+0000 7f6a4cae3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a4815d760 0x7f6a4816aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54940/0 (socket says 172.21.15.89:54940) 2023-10-10T20:42:51.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.466+0000 7f6a4cae3700 1 -- 172.21.15.89:0/3748958728 learned_addr learned my addr 172.21.15.89:0/3748958728 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:51.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.466+0000 7f6a47fff700 1 --2- 172.21.15.89:0/3748958728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a480cf480 0x7f6a48152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:51.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.466+0000 7f6a477fe700 1 --2- 172.21.15.89:0/3748958728 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a481595d0 0x7f6a480c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:51.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.466+0000 7f6a4cae3700 1 -- 172.21.15.89:0/3748958728 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a481595d0 msgr2=0x7f6a480c54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:51.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.466+0000 7f6a4cae3700 1 --2- 172.21.15.89:0/3748958728 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a481595d0 0x7f6a480c54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:51.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.466+0000 7f6a4cae3700 1 -- 172.21.15.89:0/3748958728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a480cf480 msgr2=0x7f6a48152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:51.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.466+0000 7f6a4cae3700 1 --2- 172.21.15.89:0/3748958728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a480cf480 0x7f6a48152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:51.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.466+0000 7f6a4cae3700 1 -- 172.21.15.89:0/3748958728 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a38009420 con 0x7f6a4815d760 2023-10-10T20:42:51.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.466+0000 7f6a4cae3700 1 --2- 172.21.15.89:0/3748958728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a4815d760 0x7f6a4816aee0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000cc40 tx=0x7f6a40001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:51.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.466+0000 7f6a2ffff700 1 -- 172.21.15.89:0/3748958728 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4000a9c0 con 0x7f6a4815d760 2023-10-10T20:42:51.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.467+0000 7f6a2ffff700 1 -- 172.21.15.89:0/3748958728 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4000ab20 con 0x7f6a4815d760 2023-10-10T20:42:51.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.467+0000 7f6a4d2e4700 1 -- 172.21.15.89:0/3748958728 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a481557a0 con 0x7f6a4815d760 2023-10-10T20:42:51.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.467+0000 7f6a2ffff700 1 -- 172.21.15.89:0/3748958728 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a400070f0 con 0x7f6a4815d760 2023-10-10T20:42:51.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.467+0000 7f6a4d2e4700 1 -- 172.21.15.89:0/3748958728 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a48155d60 con 0x7f6a4815d760 2023-10-10T20:42:51.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.468+0000 7f6a4d2e4700 1 -- 172.21.15.89:0/3748958728 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4804c3c0 con 0x7f6a4815d760 2023-10-10T20:42:51.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.468+0000 7f6a2ffff700 1 -- 172.21.15.89:0/3748958728 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6a400074e0 con 0x7f6a4815d760 2023-10-10T20:42:51.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.468+0000 7f6a2ffff700 1 --2- 172.21.15.89:0/3748958728 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6a34067320 0x7f6a340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:51.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.469+0000 7f6a2ffff700 1 -- 172.21.15.89:0/3748958728 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f6a40084f80 con 0x7f6a4815d760 2023-10-10T20:42:51.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.469+0000 7f6a47fff700 1 --2- 172.21.15.89:0/3748958728 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6a34067320 0x7f6a340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:51.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.469+0000 7f6a47fff700 1 --2- 172.21.15.89:0/3748958728 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6a34067320 0x7f6a340697d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c003890 tx=0x7f6a3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:51.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.473+0000 7f6a2ffff700 1 -- 172.21.15.89:0/3748958728 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a40051df0 con 0x7f6a4815d760 2023-10-10T20:42:51.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:51.657+0000 7f6a4d2e4700 1 -- 172.21.15.89:0/3748958728 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6a48169f80 con 0x7f6a4815d760 2023-10-10T20:42:52.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.323+0000 7f6a2ffff700 1 -- 172.21.15.89:0/3748958728 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v163) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6a40055410 con 0x7f6a4815d760 2023-10-10T20:42:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.327+0000 7f6a4d2e4700 1 -- 172.21.15.89:0/3748958728 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6a34067320 msgr2=0x7f6a340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.327+0000 7f6a4d2e4700 1 --2- 172.21.15.89:0/3748958728 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6a34067320 0x7f6a340697d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c003890 tx=0x7f6a3c009040 comp rx=0 tx=0).stop 2023-10-10T20:42:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.327+0000 7f6a4d2e4700 1 -- 172.21.15.89:0/3748958728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a4815d760 msgr2=0x7f6a4816aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.327+0000 7f6a4d2e4700 1 --2- 172.21.15.89:0/3748958728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a4815d760 0x7f6a4816aee0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000cc40 tx=0x7f6a40001a50 comp rx=0 tx=0).stop 2023-10-10T20:42:52.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.329+0000 7f6a4d2e4700 1 -- 172.21.15.89:0/3748958728 shutdown_connections 2023-10-10T20:42:52.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.329+0000 7f6a4d2e4700 1 --2- 172.21.15.89:0/3748958728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a4815d760 0x7f6a4816aee0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:52.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.329+0000 7f6a4d2e4700 1 --2- 172.21.15.89:0/3748958728 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6a34067320 0x7f6a340697d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:52.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.329+0000 7f6a4d2e4700 1 --2- 172.21.15.89:0/3748958728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a480cf480 0x7f6a48152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:52.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.329+0000 7f6a4d2e4700 1 --2- 172.21.15.89:0/3748958728 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6a481595d0 0x7f6a480c54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:52.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.329+0000 7f6a4d2e4700 1 -- 172.21.15.89:0/3748958728 >> 172.21.15.89:0/3748958728 conn(0x7f6a480d05f0 msgr2=0x7f6a48153880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:52.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.330+0000 7f6a4d2e4700 1 -- 172.21.15.89:0/3748958728 shutdown_connections 2023-10-10T20:42:52.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.333+0000 7f6a4d2e4700 1 -- 172.21.15.89:0/3748958728 wait complete. 2023-10-10T20:42:52.339 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2023-10-10T20:42:52.352 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:42:52.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.468+0000 7fbc07629700 1 Processor -- start 2023-10-10T20:42:52.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.468+0000 7fbc07629700 1 -- start start 2023-10-10T20:42:52.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.469+0000 7fbc07629700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc00159890 0x7fbc0015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:52.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.469+0000 7fbc07629700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbc00159c60 0x7fbc0015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:52.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.469+0000 7fbc07629700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbc0015db10 0x7fbc00069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:52.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.469+0000 7fbc07629700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc0015a160 con 0x7fbc00159c60 2023-10-10T20:42:52.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.469+0000 7fbc07629700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc000d13a0 con 0x7fbc0015db10 2023-10-10T20:42:52.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.469+0000 7fbc07629700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0015e140 con 0x7fbc00159890 2023-10-10T20:42:52.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.469+0000 7fbc04bc4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbc00159c60 0x7fbc0015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:52.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.469+0000 7fbc04bc4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbc00159c60 0x7fbc0015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54956/0 (socket says 172.21.15.89:54956) 2023-10-10T20:42:52.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.469+0000 7fbc04bc4700 1 -- 172.21.15.89:0/1732594508 learned_addr learned my addr 172.21.15.89:0/1732594508 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:52.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.469+0000 7fbc05bc6700 1 --2- 172.21.15.89:0/1732594508 >> v2:172.21.15.111:3300/0 conn(0x7fbc0015db10 0x7fbc00069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:52.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.469+0000 7fbc053c5700 1 --2- 172.21.15.89:0/1732594508 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc00159890 0x7fbc0015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:52.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.470+0000 7fbc04bc4700 1 -- 172.21.15.89:0/1732594508 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc00159890 msgr2=0x7fbc0015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:52.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.470+0000 7fbc04bc4700 1 --2- 172.21.15.89:0/1732594508 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc00159890 0x7fbc0015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:52.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.470+0000 7fbc04bc4700 1 -- 172.21.15.89:0/1732594508 >> v2:172.21.15.111:3300/0 conn(0x7fbc0015db10 msgr2=0x7fbc00069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:52.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.470+0000 7fbc04bc4700 1 --2- 172.21.15.89:0/1732594508 >> v2:172.21.15.111:3300/0 conn(0x7fbc0015db10 0x7fbc00069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:52.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.470+0000 7fbc04bc4700 1 -- 172.21.15.89:0/1732594508 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0015a3d0 con 0x7fbc00159c60 2023-10-10T20:42:52.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.470+0000 7fbc04bc4700 1 --2- 172.21.15.89:0/1732594508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc00159c60 0x7fbc0015d5d0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fbc00089b50 tx=0x7fbbf0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb5e177b4f509c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:52.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.470+0000 7fbbf7fff700 1 -- 172.21.15.89:0/1732594508 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf000b860 con 0x7fbc00159c60 2023-10-10T20:42:52.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.470+0000 7fbbf7fff700 1 -- 172.21.15.89:0/1732594508 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf000b9c0 con 0x7fbc00159c60 2023-10-10T20:42:52.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.470+0000 7fbbf7fff700 1 -- 172.21.15.89:0/1732594508 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf000bd60 con 0x7fbc00159c60 2023-10-10T20:42:52.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.470+0000 7fbc07629700 1 -- 172.21.15.89:0/1732594508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc00159c60 msgr2=0x7fbc0015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:52.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.470+0000 7fbc07629700 1 --2- 172.21.15.89:0/1732594508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc00159c60 0x7fbc0015d5d0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fbc00089b50 tx=0x7fbbf0009250 comp rx=0 tx=0).stop 2023-10-10T20:42:52.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.470+0000 7fbc07629700 1 -- 172.21.15.89:0/1732594508 shutdown_connections 2023-10-10T20:42:52.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.470+0000 7fbc07629700 1 --2- 172.21.15.89:0/1732594508 >> v2:172.21.15.111:3300/0 conn(0x7fbc0015db10 0x7fbc00069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:52.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.470+0000 7fbc07629700 1 --2- 172.21.15.89:0/1732594508 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc00159890 0x7fbc0015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:52.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.470+0000 7fbc07629700 1 --2- 172.21.15.89:0/1732594508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc00159c60 0x7fbc0015d5d0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:52.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.470+0000 7fbc07629700 1 -- 172.21.15.89:0/1732594508 >> 172.21.15.89:0/1732594508 conn(0x7fbc00068e20 msgr2=0x7fbc000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:52.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.471+0000 7fbc07629700 1 -- 172.21.15.89:0/1732594508 shutdown_connections 2023-10-10T20:42:52.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.471+0000 7fbc07629700 1 -- 172.21.15.89:0/1732594508 wait complete. 2023-10-10T20:42:52.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.472+0000 7fbc07629700 1 Processor -- start 2023-10-10T20:42:52.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.472+0000 7fbc07629700 1 -- start start 2023-10-10T20:42:52.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.472+0000 7fbc07629700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc00159890 0x7fbc00151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:52.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.472+0000 7fbc07629700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc00159c60 0x7fbc0016dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:52.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.472+0000 7fbc07629700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc0015db10 0x7fbc00148bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:52.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.472+0000 7fbc07629700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc00151580 con 0x7fbc00159c60 2023-10-10T20:42:52.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.472+0000 7fbc07629700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc00150970 con 0x7fbc00159890 2023-10-10T20:42:52.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.472+0000 7fbc07629700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc00150c50 con 0x7fbc0015db10 2023-10-10T20:42:52.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.473+0000 7fbc04bc4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc00159c60 0x7fbc0016dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:52.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.473+0000 7fbc04bc4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc00159c60 0x7fbc0016dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54968/0 (socket says 172.21.15.89:54968) 2023-10-10T20:42:52.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.473+0000 7fbc04bc4700 1 -- 172.21.15.89:0/4116840275 learned_addr learned my addr 172.21.15.89:0/4116840275 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:52.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.473+0000 7fbc053c5700 1 --2- 172.21.15.89:0/4116840275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc00159890 0x7fbc00151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:52.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.473+0000 7fbc05bc6700 1 --2- 172.21.15.89:0/4116840275 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc0015db10 0x7fbc00148bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:52.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.473+0000 7fbc04bc4700 1 -- 172.21.15.89:0/4116840275 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc0015db10 msgr2=0x7fbc00148bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:52.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.473+0000 7fbc04bc4700 1 --2- 172.21.15.89:0/4116840275 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc0015db10 0x7fbc00148bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:52.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.473+0000 7fbc04bc4700 1 -- 172.21.15.89:0/4116840275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc00159890 msgr2=0x7fbc00151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:52.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.473+0000 7fbc04bc4700 1 --2- 172.21.15.89:0/4116840275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc00159890 0x7fbc00151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:52.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.473+0000 7fbc04bc4700 1 -- 172.21.15.89:0/4116840275 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf00094f0 con 0x7fbc00159c60 2023-10-10T20:42:52.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.473+0000 7fbc04bc4700 1 --2- 172.21.15.89:0/4116840275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc00159c60 0x7fbc0016dd70 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fbc00070e20 tx=0x7fbbf0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:52.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.473+0000 7fbbf57fa700 1 -- 172.21.15.89:0/4116840275 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0011dc0 con 0x7fbc00159c60 2023-10-10T20:42:52.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.473+0000 7fbbf57fa700 1 -- 172.21.15.89:0/4116840275 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf0024440 con 0x7fbc00159c60 2023-10-10T20:42:52.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.473+0000 7fbc07629700 1 -- 172.21.15.89:0/4116840275 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc00150f30 con 0x7fbc00159c60 2023-10-10T20:42:52.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.473+0000 7fbbf57fa700 1 -- 172.21.15.89:0/4116840275 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0024800 con 0x7fbc00159c60 2023-10-10T20:42:52.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.473+0000 7fbc07629700 1 -- 172.21.15.89:0/4116840275 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc000cfe00 con 0x7fbc00159c60 2023-10-10T20:42:52.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.474+0000 7fbc07629700 1 -- 172.21.15.89:0/4116840275 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc00151f50 con 0x7fbc00159c60 2023-10-10T20:42:52.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.475+0000 7fbbf57fa700 1 -- 172.21.15.89:0/4116840275 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbbf001a420 con 0x7fbc00159c60 2023-10-10T20:42:52.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.475+0000 7fbbf57fa700 1 --2- 172.21.15.89:0/4116840275 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbbe4067320 0x7fbbe40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:52.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.475+0000 7fbbf57fa700 1 -- 172.21.15.89:0/4116840275 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fbbf0086a80 con 0x7fbc00159c60 2023-10-10T20:42:52.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.475+0000 7fbc053c5700 1 --2- 172.21.15.89:0/4116840275 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbbe4067320 0x7fbbe40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:52.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.476+0000 7fbc053c5700 1 --2- 172.21.15.89:0/4116840275 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbbe4067320 0x7fbbe40697d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc007640 tx=0x7fbbfc009850 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:52.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.480+0000 7fbbf57fa700 1 -- 172.21.15.89:0/4116840275 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbbf0053770 con 0x7fbc00159c60 2023-10-10T20:42:52.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:52.705+0000 7fbc07629700 1 -- 172.21.15.89:0/4116840275 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbc0004c3c0 con 0x7fbc00159c60 2023-10-10T20:42:53.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.346+0000 7fbbf57fa700 1 -- 172.21.15.89:0/4116840275 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 28 and data pool 29 v106) v1 ==== 174+0+0 (secure 0 0 0) 0x7fbbf00150c0 con 0x7fbc00159c60 2023-10-10T20:42:53.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.351+0000 7fbc07629700 1 -- 172.21.15.89:0/4116840275 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbbe4067320 msgr2=0x7fbbe40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:53.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.351+0000 7fbc07629700 1 --2- 172.21.15.89:0/4116840275 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbbe4067320 0x7fbbe40697d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc007640 tx=0x7fbbfc009850 comp rx=0 tx=0).stop 2023-10-10T20:42:53.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.351+0000 7fbc07629700 1 -- 172.21.15.89:0/4116840275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc00159c60 msgr2=0x7fbc0016dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:53.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.351+0000 7fbc07629700 1 --2- 172.21.15.89:0/4116840275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc00159c60 0x7fbc0016dd70 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fbc00070e20 tx=0x7fbbf0017000 comp rx=0 tx=0).stop 2023-10-10T20:42:53.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.352+0000 7fbc07629700 1 -- 172.21.15.89:0/4116840275 shutdown_connections 2023-10-10T20:42:53.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.352+0000 7fbc07629700 1 --2- 172.21.15.89:0/4116840275 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbc0015db10 0x7fbc00148bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:53.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.352+0000 7fbc07629700 1 --2- 172.21.15.89:0/4116840275 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbbe4067320 0x7fbbe40697d0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:53.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.352+0000 7fbc07629700 1 --2- 172.21.15.89:0/4116840275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc00159890 0x7fbc00151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:53.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.352+0000 7fbc07629700 1 --2- 172.21.15.89:0/4116840275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc00159c60 0x7fbc0016dd70 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:53.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.353+0000 7fbc07629700 1 -- 172.21.15.89:0/4116840275 >> 172.21.15.89:0/4116840275 conn(0x7fbc00068e20 msgr2=0x7fbc00153070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:53.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.353+0000 7fbc07629700 1 -- 172.21.15.89:0/4116840275 shutdown_connections 2023-10-10T20:42:53.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.353+0000 7fbc07629700 1 -- 172.21.15.89:0/4116840275 wait complete. 2023-10-10T20:42:53.358 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 28 and data pool 29 2023-10-10T20:42:53.371 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:42:53.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.484+0000 7f3c48833700 1 Processor -- start 2023-10-10T20:42:53.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.484+0000 7f3c48833700 1 -- start start 2023-10-10T20:42:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.485+0000 7f3c48833700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c44159890 0x7f3c4415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.485+0000 7f3c48833700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3c44159c60 0x7f3c4415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.485+0000 7f3c48833700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c4415a160 con 0x7f3c4415db10 2023-10-10T20:42:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.485+0000 7f3c48833700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c440d13a0 con 0x7f3c44159c60 2023-10-10T20:42:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.485+0000 7f3c48833700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4415e140 con 0x7f3c44159890 2023-10-10T20:42:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.485+0000 7f3c4259c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c44159890 0x7f3c4415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.485+0000 7f3c39d9b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3c44159c60 0x7f3c4415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.485+0000 7f3c4259c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c44159890 0x7f3c4415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:55532/0 (socket says 172.21.15.89:55532) 2023-10-10T20:42:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.485+0000 7f3c42d9d700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f3c4415db10 0x7f3c44069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:42576/0 (socket says 172.21.15.89:42576) 2023-10-10T20:42:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.485+0000 7f3c39d9b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3c44159c60 0x7f3c4415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:59524/0 (socket says 172.21.15.89:59524) 2023-10-10T20:42:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.485+0000 7f3c4259c700 1 -- 172.21.15.89:0/3787306636 learned_addr learned my addr 172.21.15.89:0/3787306636 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.486+0000 7f3c4259c700 1 -- 172.21.15.89:0/3787306636 >> v2:172.21.15.111:3300/0 conn(0x7f3c44159c60 msgr2=0x7f3c4415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.486+0000 7f3c4259c700 1 --2- 172.21.15.89:0/3787306636 >> v2:172.21.15.111:3300/0 conn(0x7f3c44159c60 0x7f3c4415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.486+0000 7f3c4259c700 1 -- 172.21.15.89:0/3787306636 >> v1:172.21.15.111:6789/0 conn(0x7f3c4415db10 legacy=0x7f3c44069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:53.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.486+0000 7f3c4259c700 1 -- 172.21.15.89:0/3787306636 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4415a3d0 con 0x7f3c44159890 2023-10-10T20:42:53.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.486+0000 7f3c39d9b700 1 --2- 172.21.15.89:0/3787306636 >> v2:172.21.15.111:3300/0 conn(0x7f3c44159c60 0x7f3c4415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:42:53.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.486+0000 7f3c4259c700 1 --2- 172.21.15.89:0/3787306636 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c44159890 0x7f3c4415abd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3c340099b0 tx=0x7f3c34009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12cbcf1139e0247e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:53.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.486+0000 7f3c41d9b700 1 -- 172.21.15.89:0/3787306636 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c34009ea0 con 0x7f3c44159890 2023-10-10T20:42:53.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.486+0000 7f3c41d9b700 1 -- 172.21.15.89:0/3787306636 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3400b860 con 0x7f3c44159890 2023-10-10T20:42:53.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.486+0000 7f3c41d9b700 1 -- 172.21.15.89:0/3787306636 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3400bc00 con 0x7f3c44159890 2023-10-10T20:42:53.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.487+0000 7f3c48833700 1 -- 172.21.15.89:0/3787306636 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c44159890 msgr2=0x7f3c4415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:53.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.487+0000 7f3c48833700 1 --2- 172.21.15.89:0/3787306636 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c44159890 0x7f3c4415abd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3c340099b0 tx=0x7f3c34009250 comp rx=0 tx=0).stop 2023-10-10T20:42:53.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.487+0000 7f3c48833700 1 -- 172.21.15.89:0/3787306636 shutdown_connections 2023-10-10T20:42:53.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.487+0000 7f3c48833700 1 --2- 172.21.15.89:0/3787306636 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c44159890 0x7f3c4415abd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:53.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.487+0000 7f3c48833700 1 --2- 172.21.15.89:0/3787306636 >> v2:172.21.15.111:3300/0 conn(0x7f3c44159c60 0x7f3c4415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:53.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.487+0000 7f3c48833700 1 -- 172.21.15.89:0/3787306636 >> 172.21.15.89:0/3787306636 conn(0x7f3c44068e20 msgr2=0x7f3c440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:53.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.487+0000 7f3c48833700 1 -- 172.21.15.89:0/3787306636 shutdown_connections 2023-10-10T20:42:53.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.487+0000 7f3c48833700 1 -- 172.21.15.89:0/3787306636 wait complete. 2023-10-10T20:42:53.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.488+0000 7f3c48833700 1 Processor -- start 2023-10-10T20:42:53.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.488+0000 7f3c48833700 1 -- start start 2023-10-10T20:42:53.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c48833700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c44159890 0x7f3c44169a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:53.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c48833700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c44159c60 0x7f3c440b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:53.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c48833700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c4415db10 0x7f3c440b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:53.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c48833700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c44136bd0 con 0x7f3c44159c60 2023-10-10T20:42:53.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c48833700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c440b5e10 con 0x7f3c4415db10 2023-10-10T20:42:53.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c48833700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c440cdd80 con 0x7f3c44159890 2023-10-10T20:42:53.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c39d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c44159c60 0x7f3c440b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:53.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c39d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c44159c60 0x7f3c440b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54976/0 (socket says 172.21.15.89:54976) 2023-10-10T20:42:53.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c39d9b700 1 -- 172.21.15.89:0/1935580387 learned_addr learned my addr 172.21.15.89:0/1935580387 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:53.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c4259c700 1 --2- 172.21.15.89:0/1935580387 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c44159890 0x7f3c44169a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:53.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c42d9d700 1 --2- 172.21.15.89:0/1935580387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c4415db10 0x7f3c440b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:53.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c39d9b700 1 -- 172.21.15.89:0/1935580387 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c44159890 msgr2=0x7f3c44169a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:53.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c39d9b700 1 --2- 172.21.15.89:0/1935580387 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c44159890 0x7f3c44169a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:53.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c39d9b700 1 -- 172.21.15.89:0/1935580387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c4415db10 msgr2=0x7f3c440b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:53.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c39d9b700 1 --2- 172.21.15.89:0/1935580387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c4415db10 0x7f3c440b58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:53.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c39d9b700 1 -- 172.21.15.89:0/1935580387 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c009420 con 0x7f3c44159c60 2023-10-10T20:42:53.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c4259c700 1 --2- 172.21.15.89:0/1935580387 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c44159890 0x7f3c44169a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:42:53.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c39d9b700 1 --2- 172.21.15.89:0/1935580387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c44159c60 0x7f3c440b5390 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00ec80 tx=0x7f3c2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:53.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c3affd700 1 -- 172.21.15.89:0/1935580387 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c00c850 con 0x7f3c44159c60 2023-10-10T20:42:53.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c3affd700 1 -- 172.21.15.89:0/1935580387 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c00c9b0 con 0x7f3c44159c60 2023-10-10T20:42:53.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c48833700 1 -- 172.21.15.89:0/1935580387 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c340094f0 con 0x7f3c44159c60 2023-10-10T20:42:53.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c3affd700 1 -- 172.21.15.89:0/1935580387 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c00cd50 con 0x7f3c44159c60 2023-10-10T20:42:53.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.489+0000 7f3c48833700 1 -- 172.21.15.89:0/1935580387 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c440ce2d0 con 0x7f3c44159c60 2023-10-10T20:42:53.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.490+0000 7f3c48833700 1 -- 172.21.15.89:0/1935580387 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c4404c3c0 con 0x7f3c44159c60 2023-10-10T20:42:53.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.491+0000 7f3c3affd700 1 -- 172.21.15.89:0/1935580387 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3c2c013680 con 0x7f3c44159c60 2023-10-10T20:42:53.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.491+0000 7f3c3affd700 1 --2- 172.21.15.89:0/1935580387 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3c24067320 0x7f3c240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:53.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.491+0000 7f3c4259c700 1 --2- 172.21.15.89:0/1935580387 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3c24067320 0x7f3c240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:53.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.491+0000 7f3c3affd700 1 -- 172.21.15.89:0/1935580387 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3c2c084cd0 con 0x7f3c44159c60 2023-10-10T20:42:53.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.492+0000 7f3c4259c700 1 --2- 172.21.15.89:0/1935580387 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3c24067320 0x7f3c240697d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3c340099b0 tx=0x7f3c34017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:53.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.496+0000 7f3c3affd700 1 -- 172.21.15.89:0/1935580387 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c2c051960 con 0x7f3c44159c60 2023-10-10T20:42:53.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.679+0000 7f3c48833700 1 -- 172.21.15.89:0/1935580387 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c440d0fc0 con 0x7f3c44159c60 2023-10-10T20:42:53.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.680+0000 7f3c3affd700 1 -- 172.21.15.89:0/1935580387 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+15377 (secure 0 0 0) 0x7f3c2c017090 con 0x7f3c44159c60 2023-10-10T20:42:53.682 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:53.683 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":164,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:42:52.706730+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:42:51.066663+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-10T20:42:51.675525+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:42:53.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.682+0000 7f3c48833700 1 -- 172.21.15.89:0/1935580387 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3c24067320 msgr2=0x7f3c240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.682+0000 7f3c48833700 1 --2- 172.21.15.89:0/1935580387 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3c24067320 0x7f3c240697d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3c340099b0 tx=0x7f3c34017000 comp rx=0 tx=0).stop 2023-10-10T20:42:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.682+0000 7f3c48833700 1 -- 172.21.15.89:0/1935580387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c44159c60 msgr2=0x7f3c440b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.682+0000 7f3c48833700 1 --2- 172.21.15.89:0/1935580387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c44159c60 0x7f3c440b5390 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00ec80 tx=0x7f3c2c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.682+0000 7f3c48833700 1 -- 172.21.15.89:0/1935580387 shutdown_connections 2023-10-10T20:42:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.682+0000 7f3c48833700 1 --2- 172.21.15.89:0/1935580387 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3c24067320 0x7f3c240697d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:53.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.682+0000 7f3c48833700 1 --2- 172.21.15.89:0/1935580387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c4415db10 0x7f3c440b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:53.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.682+0000 7f3c48833700 1 --2- 172.21.15.89:0/1935580387 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c44159890 0x7f3c44169a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:53.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.682+0000 7f3c48833700 1 --2- 172.21.15.89:0/1935580387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c44159c60 0x7f3c440b5390 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:53.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.682+0000 7f3c48833700 1 -- 172.21.15.89:0/1935580387 >> 172.21.15.89:0/1935580387 conn(0x7f3c44068e20 msgr2=0x7f3c44153950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:53.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.682+0000 7f3c48833700 1 -- 172.21.15.89:0/1935580387 shutdown_connections 2023-10-10T20:42:53.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.682+0000 7f3c48833700 1 -- 172.21.15.89:0/1935580387 wait complete. 2023-10-10T20:42:53.696 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:42:53.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.809+0000 7f0179834700 1 Processor -- start 2023-10-10T20:42:53.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.809+0000 7f0179834700 1 -- start start 2023-10-10T20:42:53.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.810+0000 7f0179834700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0174159420 0x7f01740b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:53.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.810+0000 7f0179834700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f017415d230 0x7f017415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:53.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.810+0000 7f0179834700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f017415dbd0 con 0x7f01741597f0 2023-10-10T20:42:53.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.810+0000 7f0179834700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01740b2980 con 0x7f017415d230 2023-10-10T20:42:53.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.810+0000 7f0179834700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01740b11d0 con 0x7f0174159420 2023-10-10T20:42:53.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.810+0000 7f0179033700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f017415d230 0x7f017415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.810+0000 7f016bfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f01741597f0 0x7f01740b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48810/0 (socket says 172.21.15.89:48810) 2023-10-10T20:42:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.810+0000 7f016bfff700 1 -- 172.21.15.89:0/433601323 learned_addr learned my addr 172.21.15.89:0/433601323 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.810+0000 7f0178832700 1 --2- 172.21.15.89:0/433601323 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0174159420 0x7f01740b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.810+0000 7f0179033700 1 -- 172.21.15.89:0/433601323 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0174159420 msgr2=0x7f01740b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.810+0000 7f0179033700 1 --2- 172.21.15.89:0/433601323 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0174159420 0x7f01740b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.810+0000 7f0179033700 1 -- 172.21.15.89:0/433601323 >> v1:172.21.15.89:6789/0 conn(0x7f01741597f0 legacy=0x7f01740b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.810+0000 7f016b7fe700 1 -- 172.21.15.89:0/433601323 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025989534 0 0) 0x7f0164001110 con 0x7f01741597f0 2023-10-10T20:42:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.810+0000 7f0179033700 1 -- 172.21.15.89:0/433601323 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01740b14b0 con 0x7f017415d230 2023-10-10T20:42:53.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.811+0000 7f0179033700 1 --2- 172.21.15.89:0/433601323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017415d230 0x7f017415d690 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f016c009e60 tx=0x7f016c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8736e8fe2f5769c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:53.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.811+0000 7f016b7fe700 1 -- 172.21.15.89:0/433601323 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c00e070 con 0x7f017415d230 2023-10-10T20:42:53.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.811+0000 7f016b7fe700 1 -- 172.21.15.89:0/433601323 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016c00bb60 con 0x7f017415d230 2023-10-10T20:42:53.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.811+0000 7f016b7fe700 1 -- 172.21.15.89:0/433601323 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c012430 con 0x7f017415d230 2023-10-10T20:42:53.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.811+0000 7f0179834700 1 -- 172.21.15.89:0/433601323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017415d230 msgr2=0x7f017415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:53.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.811+0000 7f0179834700 1 --2- 172.21.15.89:0/433601323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017415d230 0x7f017415d690 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f016c009e60 tx=0x7f016c009780 comp rx=0 tx=0).stop 2023-10-10T20:42:53.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.811+0000 7f0179834700 1 -- 172.21.15.89:0/433601323 shutdown_connections 2023-10-10T20:42:53.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.811+0000 7f0179834700 1 --2- 172.21.15.89:0/433601323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017415d230 0x7f017415d690 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:53.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.811+0000 7f0179834700 1 --2- 172.21.15.89:0/433601323 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0174159420 0x7f01740b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:53.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.811+0000 7f0179834700 1 -- 172.21.15.89:0/433601323 >> 172.21.15.89:0/433601323 conn(0x7f01740d0660 msgr2=0x7f01740d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:53.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.811+0000 7f0179834700 1 -- 172.21.15.89:0/433601323 shutdown_connections 2023-10-10T20:42:53.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.811+0000 7f0179834700 1 -- 172.21.15.89:0/433601323 wait complete. 2023-10-10T20:42:53.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.812+0000 7f0179834700 1 Processor -- start 2023-10-10T20:42:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.812+0000 7f0179834700 1 -- start start 2023-10-10T20:42:53.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.813+0000 7f0179834700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0174159420 0x7f01741697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:53.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.813+0000 7f0179834700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01741597f0 0x7f017415a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:53.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.813+0000 7f0179834700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017415d230 0x7f017415adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:53.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.813+0000 7f0179834700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0174153cb0 con 0x7f017415d230 2023-10-10T20:42:53.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.813+0000 7f0179834700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017415a3e0 con 0x7f01741597f0 2023-10-10T20:42:53.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.813+0000 7f0179834700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f017415c1f0 con 0x7f0174159420 2023-10-10T20:42:53.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.813+0000 7f0179033700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017415d230 0x7f017415adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:53.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.813+0000 7f0179033700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017415d230 0x7f017415adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55002/0 (socket says 172.21.15.89:55002) 2023-10-10T20:42:53.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.813+0000 7f0179033700 1 -- 172.21.15.89:0/706813081 learned_addr learned my addr 172.21.15.89:0/706813081 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:53.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.813+0000 7f016bfff700 1 --2- 172.21.15.89:0/706813081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01741597f0 0x7f017415a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:53.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.813+0000 7f0178832700 1 --2- 172.21.15.89:0/706813081 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0174159420 0x7f01741697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:53.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.814+0000 7f0179033700 1 -- 172.21.15.89:0/706813081 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0174159420 msgr2=0x7f01741697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:53.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.814+0000 7f0179033700 1 --2- 172.21.15.89:0/706813081 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0174159420 0x7f01741697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:53.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.814+0000 7f0179033700 1 -- 172.21.15.89:0/706813081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01741597f0 msgr2=0x7f017415a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:53.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.814+0000 7f0179033700 1 --2- 172.21.15.89:0/706813081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01741597f0 0x7f017415a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:53.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.814+0000 7f0179033700 1 -- 172.21.15.89:0/706813081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c009a20 con 0x7f017415d230 2023-10-10T20:42:53.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.814+0000 7f0179033700 1 --2- 172.21.15.89:0/706813081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017415d230 0x7f017415adb0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f0174089340 tx=0x7f016c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:53.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.814+0000 7f0168ff9700 1 -- 172.21.15.89:0/706813081 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c027470 con 0x7f017415d230 2023-10-10T20:42:53.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.814+0000 7f0168ff9700 1 -- 172.21.15.89:0/706813081 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016c027610 con 0x7f017415d230 2023-10-10T20:42:53.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.814+0000 7f0179834700 1 -- 172.21.15.89:0/706813081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f017415bf10 con 0x7f017415d230 2023-10-10T20:42:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.814+0000 7f0168ff9700 1 -- 172.21.15.89:0/706813081 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c0279d0 con 0x7f017415d230 2023-10-10T20:42:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.814+0000 7f0179834700 1 -- 172.21.15.89:0/706813081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017415a070 con 0x7f017415d230 2023-10-10T20:42:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.815+0000 7f0179834700 1 -- 172.21.15.89:0/706813081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f017415a550 con 0x7f017415d230 2023-10-10T20:42:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.815+0000 7f0168ff9700 1 -- 172.21.15.89:0/706813081 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f016c024070 con 0x7f017415d230 2023-10-10T20:42:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.816+0000 7f0168ff9700 1 --2- 172.21.15.89:0/706813081 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0160067320 0x7f01600697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.816+0000 7f0178832700 1 --2- 172.21.15.89:0/706813081 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0160067320 0x7f01600697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:53.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.816+0000 7f0168ff9700 1 -- 172.21.15.89:0/706813081 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f016c0875e0 con 0x7f017415d230 2023-10-10T20:42:53.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.816+0000 7f0178832700 1 --2- 172.21.15.89:0/706813081 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0160067320 0x7f01600697d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0170009d40 tx=0x7f0170008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:53.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:53.820+0000 7f0168ff9700 1 -- 172.21.15.89:0/706813081 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f016c054420 con 0x7f017415d230 2023-10-10T20:42:54.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.005+0000 7f0179834700 1 -- 172.21.15.89:0/706813081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f017404c3c0 con 0x7f017415d230 2023-10-10T20:42:54.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.005+0000 7f0168ff9700 1 -- 172.21.15.89:0/706813081 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+15377 (secure 0 0 0) 0x7f016c057a40 con 0x7f017415d230 2023-10-10T20:42:54.008 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:54.008 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":164,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:42:52.706730+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:42:51.066663+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-10T20:42:51.675525+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:42:54.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.007+0000 7f0179834700 1 -- 172.21.15.89:0/706813081 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0160067320 msgr2=0x7f01600697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:54.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.007+0000 7f0179834700 1 --2- 172.21.15.89:0/706813081 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0160067320 0x7f01600697d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0170009d40 tx=0x7f0170008040 comp rx=0 tx=0).stop 2023-10-10T20:42:54.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.007+0000 7f0179834700 1 -- 172.21.15.89:0/706813081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017415d230 msgr2=0x7f017415adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:54.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.007+0000 7f0179834700 1 --2- 172.21.15.89:0/706813081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017415d230 0x7f017415adb0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f0174089340 tx=0x7f016c019000 comp rx=0 tx=0).stop 2023-10-10T20:42:54.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.007+0000 7f0179834700 1 -- 172.21.15.89:0/706813081 shutdown_connections 2023-10-10T20:42:54.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.007+0000 7f0179834700 1 --2- 172.21.15.89:0/706813081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017415d230 0x7f017415adb0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:54.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.007+0000 7f0179834700 1 --2- 172.21.15.89:0/706813081 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0174159420 0x7f01741697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:54.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.007+0000 7f0179834700 1 --2- 172.21.15.89:0/706813081 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0160067320 0x7f01600697d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:54.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.007+0000 7f0179834700 1 --2- 172.21.15.89:0/706813081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01741597f0 0x7f017415a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:54.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.007+0000 7f0179834700 1 -- 172.21.15.89:0/706813081 >> 172.21.15.89:0/706813081 conn(0x7f01740d0660 msgr2=0x7f01740d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:54.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.008+0000 7f0179834700 1 -- 172.21.15.89:0/706813081 shutdown_connections 2023-10-10T20:42:54.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.008+0000 7f0179834700 1 -- 172.21.15.89:0/706813081 wait complete. 2023-10-10T20:42:54.022 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:42:54.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.138+0000 7f2041b48700 1 Processor -- start 2023-10-10T20:42:54.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.138+0000 7f2041b48700 1 -- start start 2023-10-10T20:42:54.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.139+0000 7f2041b48700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f203c159890 0x7f203c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:54.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.139+0000 7f2041b48700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f203c159c60 0x7f203c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:54.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.139+0000 7f2041b48700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f203c15db10 0x7f203c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:54.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.139+0000 7f2041b48700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f203c15a160 con 0x7f203c159890 2023-10-10T20:42:54.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.139+0000 7f2041b48700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f203c0d13a0 con 0x7f203c15db10 2023-10-10T20:42:54.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.139+0000 7f2041b48700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c15e140 con 0x7f203c159c60 2023-10-10T20:42:54.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.139+0000 7f203b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f203c159890 0x7f203c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:54.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.139+0000 7f203b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f203c159890 0x7f203c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55016/0 (socket says 172.21.15.89:55016) 2023-10-10T20:42:54.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.139+0000 7f203b7fe700 1 -- 172.21.15.89:0/2349633596 learned_addr learned my addr 172.21.15.89:0/2349633596 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:54.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.139+0000 7f203bfff700 1 --2- 172.21.15.89:0/2349633596 >> v2:172.21.15.111:3300/0 conn(0x7f203c15db10 0x7f203c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:54.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.139+0000 7f203affd700 1 --2- 172.21.15.89:0/2349633596 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f203c159c60 0x7f203c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:54.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.139+0000 7f203b7fe700 1 -- 172.21.15.89:0/2349633596 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f203c159c60 msgr2=0x7f203c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:54.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.139+0000 7f203b7fe700 1 --2- 172.21.15.89:0/2349633596 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f203c159c60 0x7f203c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:54.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.139+0000 7f203b7fe700 1 -- 172.21.15.89:0/2349633596 >> v2:172.21.15.111:3300/0 conn(0x7f203c15db10 msgr2=0x7f203c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:54.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.139+0000 7f203b7fe700 1 --2- 172.21.15.89:0/2349633596 >> v2:172.21.15.111:3300/0 conn(0x7f203c15db10 0x7f203c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:54.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.139+0000 7f203b7fe700 1 -- 172.21.15.89:0/2349633596 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c15a3d0 con 0x7f203c159890 2023-10-10T20:42:54.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.139+0000 7f203bfff700 1 --2- 172.21.15.89:0/2349633596 >> v2:172.21.15.111:3300/0 conn(0x7f203c15db10 0x7f203c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:54.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.139+0000 7f203affd700 1 --2- 172.21.15.89:0/2349633596 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f203c159c60 0x7f203c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:54.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.139+0000 7f203b7fe700 1 --2- 172.21.15.89:0/2349633596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f203c159890 0x7f203c15abd0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f203c089b50 tx=0x7f202c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508e03abf4ef6cf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:54.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.140+0000 7f203a7fc700 1 -- 172.21.15.89:0/2349633596 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c00e070 con 0x7f203c159890 2023-10-10T20:42:54.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.140+0000 7f203a7fc700 1 -- 172.21.15.89:0/2349633596 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c00ba50 con 0x7f203c159890 2023-10-10T20:42:54.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.140+0000 7f203a7fc700 1 -- 172.21.15.89:0/2349633596 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c0092e0 con 0x7f203c159890 2023-10-10T20:42:54.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.140+0000 7f2041b48700 1 -- 172.21.15.89:0/2349633596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f203c159890 msgr2=0x7f203c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:54.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.140+0000 7f2041b48700 1 --2- 172.21.15.89:0/2349633596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f203c159890 0x7f203c15abd0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f203c089b50 tx=0x7f202c009840 comp rx=0 tx=0).stop 2023-10-10T20:42:54.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.140+0000 7f2041b48700 1 -- 172.21.15.89:0/2349633596 shutdown_connections 2023-10-10T20:42:54.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.140+0000 7f2041b48700 1 --2- 172.21.15.89:0/2349633596 >> v2:172.21.15.111:3300/0 conn(0x7f203c15db10 0x7f203c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:54.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.140+0000 7f2041b48700 1 --2- 172.21.15.89:0/2349633596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f203c159890 0x7f203c15abd0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:54.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.140+0000 7f2041b48700 1 --2- 172.21.15.89:0/2349633596 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f203c159c60 0x7f203c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:54.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.140+0000 7f2041b48700 1 -- 172.21.15.89:0/2349633596 >> 172.21.15.89:0/2349633596 conn(0x7f203c068e20 msgr2=0x7f203c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:54.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.140+0000 7f2041b48700 1 -- 172.21.15.89:0/2349633596 shutdown_connections 2023-10-10T20:42:54.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.140+0000 7f2041b48700 1 -- 172.21.15.89:0/2349633596 wait complete. 2023-10-10T20:42:54.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.141+0000 7f2041b48700 1 Processor -- start 2023-10-10T20:42:54.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.141+0000 7f2041b48700 1 -- start start 2023-10-10T20:42:54.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.142+0000 7f2041b48700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f203c159890 0x7f203c169e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:54.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.142+0000 7f2041b48700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f203c159c60 0x7f203c0ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:54.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.142+0000 7f2041b48700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f203c15db10 0x7f203c147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:54.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.142+0000 7f2041b48700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0b36c0 con 0x7f203c15db10 2023-10-10T20:42:54.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.142+0000 7f2041b48700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0c7aa0 con 0x7f203c159c60 2023-10-10T20:42:54.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.142+0000 7f2041b48700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c16a440 con 0x7f203c159890 2023-10-10T20:42:54.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.142+0000 7f203bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f203c15db10 0x7f203c147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:54.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.142+0000 7f203bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f203c15db10 0x7f203c147280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55020/0 (socket says 172.21.15.89:55020) 2023-10-10T20:42:54.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.142+0000 7f203bfff700 1 -- 172.21.15.89:0/3191622940 learned_addr learned my addr 172.21.15.89:0/3191622940 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:54.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.142+0000 7f203b7fe700 1 --2- 172.21.15.89:0/3191622940 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f203c159890 0x7f203c169e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:54.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.142+0000 7f203affd700 1 --2- 172.21.15.89:0/3191622940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f203c159c60 0x7f203c0ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:54.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.143+0000 7f203bfff700 1 -- 172.21.15.89:0/3191622940 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f203c159890 msgr2=0x7f203c169e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:54.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.143+0000 7f203bfff700 1 --2- 172.21.15.89:0/3191622940 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f203c159890 0x7f203c169e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:54.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.143+0000 7f203bfff700 1 -- 172.21.15.89:0/3191622940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f203c159c60 msgr2=0x7f203c0ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:54.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.143+0000 7f203bfff700 1 --2- 172.21.15.89:0/3191622940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f203c159c60 0x7f203c0ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:54.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.143+0000 7f203bfff700 1 -- 172.21.15.89:0/3191622940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c009ae0 con 0x7f203c15db10 2023-10-10T20:42:54.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.143+0000 7f203affd700 1 --2- 172.21.15.89:0/3191622940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f203c159c60 0x7f203c0ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:54.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.143+0000 7f203bfff700 1 --2- 172.21.15.89:0/3191622940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f203c15db10 0x7f203c147280 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f203400d890 tx=0x7f2034008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:54.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.143+0000 7f20237fe700 1 -- 172.21.15.89:0/3191622940 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20340125f0 con 0x7f203c15db10 2023-10-10T20:42:54.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.143+0000 7f20237fe700 1 -- 172.21.15.89:0/3191622940 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2034012750 con 0x7f203c15db10 2023-10-10T20:42:54.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.143+0000 7f2041b48700 1 -- 172.21.15.89:0/3191622940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203c16a720 con 0x7f203c15db10 2023-10-10T20:42:54.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.143+0000 7f20237fe700 1 -- 172.21.15.89:0/3191622940 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034012ad0 con 0x7f203c15db10 2023-10-10T20:42:54.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.143+0000 7f2041b48700 1 -- 172.21.15.89:0/3191622940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203c16ace0 con 0x7f203c15db10 2023-10-10T20:42:54.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.144+0000 7f2041b48700 1 -- 172.21.15.89:0/3191622940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f203c064df0 con 0x7f203c15db10 2023-10-10T20:42:54.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.144+0000 7f20237fe700 1 -- 172.21.15.89:0/3191622940 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2034014430 con 0x7f203c15db10 2023-10-10T20:42:54.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.145+0000 7f20237fe700 1 --2- 172.21.15.89:0/3191622940 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2028067320 0x7f20280697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:54.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.145+0000 7f20237fe700 1 -- 172.21.15.89:0/3191622940 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2034084f00 con 0x7f203c15db10 2023-10-10T20:42:54.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.145+0000 7f203b7fe700 1 --2- 172.21.15.89:0/3191622940 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2028067320 0x7f20280697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:54.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.145+0000 7f203b7fe700 1 --2- 172.21.15.89:0/3191622940 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2028067320 0x7f20280697d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f202c009880 tx=0x7f202c018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:54.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.149+0000 7f20237fe700 1 -- 172.21.15.89:0/3191622940 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2034055730 con 0x7f203c15db10 2023-10-10T20:42:54.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:54.376+0000 7f2041b48700 1 -- 172.21.15.89:0/3191622940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f203c04c3c0 con 0x7f203c15db10 2023-10-10T20:42:55.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.359+0000 7f20237fe700 1 -- 172.21.15.89:0/3191622940 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v109) v1 ==== 118+0+0 (secure 0 0 0) 0x7f20340552c0 con 0x7f203c15db10 2023-10-10T20:42:55.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.361+0000 7f2041b48700 1 -- 172.21.15.89:0/3191622940 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2028067320 msgr2=0x7f20280697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:55.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.361+0000 7f2041b48700 1 --2- 172.21.15.89:0/3191622940 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2028067320 0x7f20280697d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f202c009880 tx=0x7f202c018000 comp rx=0 tx=0).stop 2023-10-10T20:42:55.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.361+0000 7f2041b48700 1 -- 172.21.15.89:0/3191622940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f203c15db10 msgr2=0x7f203c147280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:55.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.361+0000 7f2041b48700 1 --2- 172.21.15.89:0/3191622940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f203c15db10 0x7f203c147280 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f203400d890 tx=0x7f2034008040 comp rx=0 tx=0).stop 2023-10-10T20:42:55.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.361+0000 7f2041b48700 1 -- 172.21.15.89:0/3191622940 shutdown_connections 2023-10-10T20:42:55.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.361+0000 7f2041b48700 1 --2- 172.21.15.89:0/3191622940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f203c15db10 0x7f203c147280 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:55.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.361+0000 7f2041b48700 1 --2- 172.21.15.89:0/3191622940 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f203c159890 0x7f203c169e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:55.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.361+0000 7f2041b48700 1 --2- 172.21.15.89:0/3191622940 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2028067320 0x7f20280697d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:55.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.361+0000 7f2041b48700 1 --2- 172.21.15.89:0/3191622940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f203c159c60 0x7f203c0ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:55.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.361+0000 7f2041b48700 1 -- 172.21.15.89:0/3191622940 >> 172.21.15.89:0/3191622940 conn(0x7f203c068e20 msgr2=0x7f203c152f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:55.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.361+0000 7f2041b48700 1 -- 172.21.15.89:0/3191622940 shutdown_connections 2023-10-10T20:42:55.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.361+0000 7f2041b48700 1 -- 172.21.15.89:0/3191622940 wait complete. 2023-10-10T20:42:55.377 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:42:55.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.494+0000 7f94369d9700 1 Processor -- start 2023-10-10T20:42:55.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.494+0000 7f94369d9700 1 -- start start 2023-10-10T20:42:55.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.495+0000 7f94369d9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f94300b4e70 0x7f94300ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:55.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.495+0000 7f94369d9700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f94300b16d0 0x7f94300b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:55.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.495+0000 7f94369d9700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94300b2080 con 0x7f943015d240 2023-10-10T20:42:55.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.495+0000 7f94369d9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94300cfb00 con 0x7f94300b4e70 2023-10-10T20:42:55.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.495+0000 7f94369d9700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94300b5c40 con 0x7f94300b16d0 2023-10-10T20:42:55.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.495+0000 7f94359d7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f94300b4e70 0x7f94300ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:55.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.495+0000 7f94359d7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f94300b4e70 0x7f94300ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55026/0 (socket says 172.21.15.89:55026) 2023-10-10T20:42:55.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.495+0000 7f94359d7700 1 -- 172.21.15.89:0/1882172335 learned_addr learned my addr 172.21.15.89:0/1882172335 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:55.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.496+0000 7f94351d6700 1 --2- 172.21.15.89:0/1882172335 >> v2:172.21.15.111:3300/0 conn(0x7f94300b16d0 0x7f94300b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:55.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.496+0000 7f94359d7700 1 -- 172.21.15.89:0/1882172335 >> v2:172.21.15.111:3300/0 conn(0x7f94300b16d0 msgr2=0x7f94300b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:55.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.496+0000 7f94359d7700 1 --2- 172.21.15.89:0/1882172335 >> v2:172.21.15.111:3300/0 conn(0x7f94300b16d0 0x7f94300b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:55.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.496+0000 7f94359d7700 1 -- 172.21.15.89:0/1882172335 >> v1:172.21.15.111:6789/0 conn(0x7f943015d240 legacy=0x7f943015d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:55.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.496+0000 7f94359d7700 1 -- 172.21.15.89:0/1882172335 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94300b5f20 con 0x7f94300b4e70 2023-10-10T20:42:55.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.496+0000 7f94359d7700 1 --2- 172.21.15.89:0/1882172335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94300b4e70 0x7f94300ccff0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f9430089830 tx=0x7f942c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0cd4ad588c0817 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:55.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.496+0000 7f94349d5700 1 -- 172.21.15.89:0/1882172335 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c009e70 con 0x7f94300b4e70 2023-10-10T20:42:55.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.496+0000 7f94349d5700 1 -- 172.21.15.89:0/1882172335 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c00b860 con 0x7f94300b4e70 2023-10-10T20:42:55.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.496+0000 7f94349d5700 1 -- 172.21.15.89:0/1882172335 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c00bc00 con 0x7f94300b4e70 2023-10-10T20:42:55.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.496+0000 7f94369d9700 1 -- 172.21.15.89:0/1882172335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94300b4e70 msgr2=0x7f94300ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:55.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.496+0000 7f94369d9700 1 --2- 172.21.15.89:0/1882172335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94300b4e70 0x7f94300ccff0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f9430089830 tx=0x7f942c009250 comp rx=0 tx=0).stop 2023-10-10T20:42:55.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.497+0000 7f94369d9700 1 -- 172.21.15.89:0/1882172335 shutdown_connections 2023-10-10T20:42:55.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.497+0000 7f94369d9700 1 --2- 172.21.15.89:0/1882172335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94300b4e70 0x7f94300ccff0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:55.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.497+0000 7f94369d9700 1 --2- 172.21.15.89:0/1882172335 >> v2:172.21.15.111:3300/0 conn(0x7f94300b16d0 0x7f94300b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:55.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.497+0000 7f94369d9700 1 -- 172.21.15.89:0/1882172335 >> 172.21.15.89:0/1882172335 conn(0x7f94300d10c0 msgr2=0x7f94300affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:55.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.497+0000 7f94369d9700 1 -- 172.21.15.89:0/1882172335 shutdown_connections 2023-10-10T20:42:55.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.497+0000 7f94369d9700 1 -- 172.21.15.89:0/1882172335 wait complete. 2023-10-10T20:42:55.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.498+0000 7f94369d9700 1 Processor -- start 2023-10-10T20:42:55.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.498+0000 7f94369d9700 1 -- start start 2023-10-10T20:42:55.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.498+0000 7f94369d9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94300b16d0 0x7f94300c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:55.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.498+0000 7f94369d9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94300b4e70 0x7f94300c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:55.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.498+0000 7f94369d9700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f943015d240 0x7f94300c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:55.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.498+0000 7f94369d9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94300c5390 con 0x7f94300b4e70 2023-10-10T20:42:55.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.498+0000 7f94369d9700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94300c5670 con 0x7f943015d240 2023-10-10T20:42:55.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.498+0000 7f94369d9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94300c8ea0 con 0x7f94300b16d0 2023-10-10T20:42:55.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.499+0000 7f94351d6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94300b4e70 0x7f94300c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:55.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.499+0000 7f94351d6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94300b4e70 0x7f94300c8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55036/0 (socket says 172.21.15.89:55036) 2023-10-10T20:42:55.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.499+0000 7f94351d6700 1 -- 172.21.15.89:0/2845449685 learned_addr learned my addr 172.21.15.89:0/2845449685 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:55.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.499+0000 7f94359d7700 1 --2- 172.21.15.89:0/2845449685 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94300b16d0 0x7f94300c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:55.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.499+0000 7f94361d8700 1 --2- 172.21.15.89:0/2845449685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f943015d240 0x7f94300c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:55.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.499+0000 7f94351d6700 1 -- 172.21.15.89:0/2845449685 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94300b16d0 msgr2=0x7f94300c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:55.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.499+0000 7f94351d6700 1 --2- 172.21.15.89:0/2845449685 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94300b16d0 0x7f94300c75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:55.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.499+0000 7f94351d6700 1 -- 172.21.15.89:0/2845449685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f943015d240 msgr2=0x7f94300c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:55.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.499+0000 7f94351d6700 1 --2- 172.21.15.89:0/2845449685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f943015d240 0x7f94300c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:55.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.499+0000 7f94351d6700 1 -- 172.21.15.89:0/2845449685 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c0094f0 con 0x7f94300b4e70 2023-10-10T20:42:55.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.499+0000 7f94351d6700 1 --2- 172.21.15.89:0/2845449685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94300b4e70 0x7f94300c8670 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f942000ec80 tx=0x7f94200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:55.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.499+0000 7f9425ffb700 1 -- 172.21.15.89:0/2845449685 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942000c850 con 0x7f94300b4e70 2023-10-10T20:42:55.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.499+0000 7f9425ffb700 1 -- 172.21.15.89:0/2845449685 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942000c9b0 con 0x7f94300b4e70 2023-10-10T20:42:55.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.499+0000 7f94369d9700 1 -- 172.21.15.89:0/2845449685 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94300c9180 con 0x7f94300b4e70 2023-10-10T20:42:55.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.499+0000 7f9425ffb700 1 -- 172.21.15.89:0/2845449685 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942000cd50 con 0x7f94300b4e70 2023-10-10T20:42:55.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.499+0000 7f94369d9700 1 -- 172.21.15.89:0/2845449685 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94300c8bb0 con 0x7f94300b4e70 2023-10-10T20:42:55.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.500+0000 7f94369d9700 1 -- 172.21.15.89:0/2845449685 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94300c57e0 con 0x7f94300b4e70 2023-10-10T20:42:55.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.501+0000 7f9425ffb700 1 -- 172.21.15.89:0/2845449685 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9420013680 con 0x7f94300b4e70 2023-10-10T20:42:55.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.501+0000 7f9425ffb700 1 --2- 172.21.15.89:0/2845449685 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9414067320 0x7f94140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:55.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.501+0000 7f94359d7700 1 --2- 172.21.15.89:0/2845449685 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9414067320 0x7f94140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:55.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.501+0000 7f9425ffb700 1 -- 172.21.15.89:0/2845449685 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9420084e70 con 0x7f94300b4e70 2023-10-10T20:42:55.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.502+0000 7f94359d7700 1 --2- 172.21.15.89:0/2845449685 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9414067320 0x7f94140697d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9430089830 tx=0x7f942c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:55.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.505+0000 7f9425ffb700 1 -- 172.21.15.89:0/2845449685 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9420051b00 con 0x7f94300b4e70 2023-10-10T20:42:55.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:55.731+0000 7f94369d9700 1 -- 172.21.15.89:0/2845449685 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f943004bb80 con 0x7f94300b4e70 2023-10-10T20:42:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.362+0000 7f9425ffb700 1 -- 172.21.15.89:0/2845449685 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v110) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9420055120 con 0x7f94300b4e70 2023-10-10T20:42:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.365+0000 7f94369d9700 1 -- 172.21.15.89:0/2845449685 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9414067320 msgr2=0x7f94140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:56.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.365+0000 7f94369d9700 1 --2- 172.21.15.89:0/2845449685 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9414067320 0x7f94140697d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9430089830 tx=0x7f942c017000 comp rx=0 tx=0).stop 2023-10-10T20:42:56.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.365+0000 7f94369d9700 1 -- 172.21.15.89:0/2845449685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94300b4e70 msgr2=0x7f94300c8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:56.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.365+0000 7f94369d9700 1 --2- 172.21.15.89:0/2845449685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94300b4e70 0x7f94300c8670 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f942000ec80 tx=0x7f94200091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.365+0000 7f94369d9700 1 -- 172.21.15.89:0/2845449685 shutdown_connections 2023-10-10T20:42:56.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.365+0000 7f94369d9700 1 --2- 172.21.15.89:0/2845449685 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9414067320 0x7f94140697d0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.365+0000 7f94369d9700 1 --2- 172.21.15.89:0/2845449685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f943015d240 0x7f94300c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.365+0000 7f94369d9700 1 --2- 172.21.15.89:0/2845449685 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94300b16d0 0x7f94300c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.365+0000 7f94369d9700 1 --2- 172.21.15.89:0/2845449685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94300b4e70 0x7f94300c8670 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.365+0000 7f94369d9700 1 -- 172.21.15.89:0/2845449685 >> 172.21.15.89:0/2845449685 conn(0x7f94300d10c0 msgr2=0x7f9430158e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:56.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.365+0000 7f94369d9700 1 -- 172.21.15.89:0/2845449685 shutdown_connections 2023-10-10T20:42:56.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.365+0000 7f94369d9700 1 -- 172.21.15.89:0/2845449685 wait complete. 2023-10-10T20:42:56.379 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:42:56.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.493+0000 7f29d26eb700 1 Processor -- start 2023-10-10T20:42:56.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.494+0000 7f29d26eb700 1 -- start start 2023-10-10T20:42:56.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.494+0000 7f29d26eb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f29cc159b30 0x7f29cc06e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:56.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.494+0000 7f29d26eb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f29cc06eab0 0x7f29cc068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:56.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.494+0000 7f29d26eb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29cc06c080 con 0x7f29cc159760 2023-10-10T20:42:56.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.494+0000 7f29d26eb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29cc06f580 con 0x7f29cc159b30 2023-10-10T20:42:56.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.494+0000 7f29d26eb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc06efc0 con 0x7f29cc06eab0 2023-10-10T20:42:56.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.495+0000 7f29d16e9700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f29cc159760 0x7f29cc0c4310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48822/0 (socket says 172.21.15.89:48822) 2023-10-10T20:42:56.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.495+0000 7f29d16e9700 1 -- 172.21.15.89:0/2616266510 learned_addr learned my addr 172.21.15.89:0/2616266510 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:56.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.495+0000 7f29d0ee8700 1 --2- 172.21.15.89:0/2616266510 >> v2:172.21.15.89:3300/0 conn(0x7f29cc159b30 0x7f29cc06e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:56.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.495+0000 7f29c3fff700 1 -- 172.21.15.89:0/2616266510 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808760097 0 0) 0x7f29cc06c080 con 0x7f29cc159760 2023-10-10T20:42:56.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.495+0000 7f29c3fff700 1 -- 172.21.15.89:0/2616266510 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29b8003570 con 0x7f29cc159760 2023-10-10T20:42:56.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.495+0000 7f29d1eea700 1 --2- 172.21.15.89:0/2616266510 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f29cc06eab0 0x7f29cc068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:56.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.495+0000 7f29d0ee8700 1 -- 172.21.15.89:0/2616266510 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f29cc06eab0 msgr2=0x7f29cc068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:56.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.495+0000 7f29d0ee8700 1 --2- 172.21.15.89:0/2616266510 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f29cc06eab0 0x7f29cc068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.495+0000 7f29d0ee8700 1 -- 172.21.15.89:0/2616266510 >> v1:172.21.15.89:6789/0 conn(0x7f29cc159760 legacy=0x7f29cc0c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:56.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.495+0000 7f29d0ee8700 1 -- 172.21.15.89:0/2616266510 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc06f2a0 con 0x7f29cc159b30 2023-10-10T20:42:56.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.495+0000 7f29d1eea700 1 --2- 172.21.15.89:0/2616266510 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f29cc06eab0 0x7f29cc068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:56.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.495+0000 7f29d0ee8700 1 --2- 172.21.15.89:0/2616266510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29cc159b30 0x7f29cc06e570 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0098e0 tx=0x7f29bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850bbdae777773f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:56.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.495+0000 7f29c3fff700 1 -- 172.21.15.89:0/2616266510 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc00b640 con 0x7f29cc159b30 2023-10-10T20:42:56.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.495+0000 7f29c3fff700 1 -- 172.21.15.89:0/2616266510 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc00b7a0 con 0x7f29cc159b30 2023-10-10T20:42:56.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.495+0000 7f29c3fff700 1 -- 172.21.15.89:0/2616266510 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc00bb40 con 0x7f29cc159b30 2023-10-10T20:42:56.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.496+0000 7f29d26eb700 1 -- 172.21.15.89:0/2616266510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29cc159b30 msgr2=0x7f29cc06e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:56.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.496+0000 7f29d26eb700 1 --2- 172.21.15.89:0/2616266510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29cc159b30 0x7f29cc06e570 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0098e0 tx=0x7f29bc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.496+0000 7f29d26eb700 1 -- 172.21.15.89:0/2616266510 shutdown_connections 2023-10-10T20:42:56.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.496+0000 7f29d26eb700 1 --2- 172.21.15.89:0/2616266510 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f29cc06eab0 0x7f29cc068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.496+0000 7f29d26eb700 1 --2- 172.21.15.89:0/2616266510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29cc159b30 0x7f29cc06e570 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.496+0000 7f29d26eb700 1 -- 172.21.15.89:0/2616266510 >> 172.21.15.89:0/2616266510 conn(0x7f29cc06b3f0 msgr2=0x7f29cc0cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:56.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.496+0000 7f29d26eb700 1 -- 172.21.15.89:0/2616266510 shutdown_connections 2023-10-10T20:42:56.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.496+0000 7f29d26eb700 1 -- 172.21.15.89:0/2616266510 wait complete. 2023-10-10T20:42:56.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.497+0000 7f29d26eb700 1 Processor -- start 2023-10-10T20:42:56.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.497+0000 7f29d26eb700 1 -- start start 2023-10-10T20:42:56.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29d26eb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29cc06eab0 0x7f29cc0b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:56.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29d26eb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29cc159760 0x7f29cc0c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:56.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29d26eb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f29cc159b30 0x7f29cc1567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:56.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29d26eb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc0c71b0 con 0x7f29cc159760 2023-10-10T20:42:56.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29d26eb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc0b1020 con 0x7f29cc06eab0 2023-10-10T20:42:56.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29d26eb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc16dc50 con 0x7f29cc159b30 2023-10-10T20:42:56.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29d0ee8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29cc159760 0x7f29cc0c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:56.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29d0ee8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29cc159760 0x7f29cc0c6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55050/0 (socket says 172.21.15.89:55050) 2023-10-10T20:42:56.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29d0ee8700 1 -- 172.21.15.89:0/3657150031 learned_addr learned my addr 172.21.15.89:0/3657150031 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:56.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29d16e9700 1 --2- 172.21.15.89:0/3657150031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29cc06eab0 0x7f29cc0b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:56.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29d1eea700 1 --2- 172.21.15.89:0/3657150031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f29cc159b30 0x7f29cc1567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:56.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29d0ee8700 1 -- 172.21.15.89:0/3657150031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f29cc159b30 msgr2=0x7f29cc1567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:56.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29d0ee8700 1 --2- 172.21.15.89:0/3657150031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f29cc159b30 0x7f29cc1567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29d0ee8700 1 -- 172.21.15.89:0/3657150031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29cc06eab0 msgr2=0x7f29cc0b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29d0ee8700 1 --2- 172.21.15.89:0/3657150031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29cc06eab0 0x7f29cc0b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29d0ee8700 1 -- 172.21.15.89:0/3657150031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc009420 con 0x7f29cc159760 2023-10-10T20:42:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29d0ee8700 1 --2- 172.21.15.89:0/3657150031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29cc159760 0x7f29cc0c6c70 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f29cc06dd20 tx=0x7f29bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29c17fa700 1 -- 172.21.15.89:0/3657150031 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc01bcd0 con 0x7f29cc159760 2023-10-10T20:42:56.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29c17fa700 1 -- 172.21.15.89:0/3657150031 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc01be30 con 0x7f29cc159760 2023-10-10T20:42:56.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29d26eb700 1 -- 172.21.15.89:0/3657150031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29cc0b0780 con 0x7f29cc159760 2023-10-10T20:42:56.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29c17fa700 1 -- 172.21.15.89:0/3657150031 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc011670 con 0x7f29cc159760 2023-10-10T20:42:56.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.498+0000 7f29d26eb700 1 -- 172.21.15.89:0/3657150031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29cc0b0d40 con 0x7f29cc159760 2023-10-10T20:42:56.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.499+0000 7f29d26eb700 1 -- 172.21.15.89:0/3657150031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29cc158180 con 0x7f29cc159760 2023-10-10T20:42:56.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.500+0000 7f29c17fa700 1 -- 172.21.15.89:0/3657150031 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f29bc011a20 con 0x7f29cc159760 2023-10-10T20:42:56.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.500+0000 7f29c17fa700 1 --2- 172.21.15.89:0/3657150031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f29b0067320 0x7f29b00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:56.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.500+0000 7f29c17fa700 1 -- 172.21.15.89:0/3657150031 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f29bc0861f0 con 0x7f29cc159760 2023-10-10T20:42:56.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.500+0000 7f29d16e9700 1 --2- 172.21.15.89:0/3657150031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f29b0067320 0x7f29b00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:56.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.500+0000 7f29d16e9700 1 --2- 172.21.15.89:0/3657150031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f29b0067320 0x7f29b00697d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f29c80058d0 tx=0x7f29c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:56.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.504+0000 7f29c17fa700 1 -- 172.21.15.89:0/3657150031 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29bc052e80 con 0x7f29cc159760 2023-10-10T20:42:56.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.727+0000 7f29d26eb700 1 -- 172.21.15.89:0/3657150031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29cc16ddc0 con 0x7f29cc159760 2023-10-10T20:42:56.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.727+0000 7f29c17fa700 1 -- 172.21.15.89:0/3657150031 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4613 (secure 0 0 0) 0x7f29bc0564a0 con 0x7f29cc159760 2023-10-10T20:42:56.730 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:56.730 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":110,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6638,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2350308973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2350308973},{"type":"v1","addr":"172.21.15.111:6835","nonce":2350308973}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":8322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/1051252386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":1051252386},{"type":"v1","addr":"172.21.15.89:6836","nonce":1051252386}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":8325,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3754994248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":3754994248},{"type":"v1","addr":"172.21.15.89:6837","nonce":3754994248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:42:52.707112+0000","modified":"2023-10-10T20:42:56.358589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8337":{"gid":8337,"name":"b","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1733913702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1733913702},{"type":"v1","addr":"172.21.15.111:6834","nonce":1733913702}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-10T20:42:56.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.729+0000 7f29d26eb700 1 -- 172.21.15.89:0/3657150031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f29b0067320 msgr2=0x7f29b00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:56.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.729+0000 7f29d26eb700 1 --2- 172.21.15.89:0/3657150031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f29b0067320 0x7f29b00697d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f29c80058d0 tx=0x7f29c8009040 comp rx=0 tx=0).stop 2023-10-10T20:42:56.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.729+0000 7f29d26eb700 1 -- 172.21.15.89:0/3657150031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29cc159760 msgr2=0x7f29cc0c6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:56.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.729+0000 7f29d26eb700 1 --2- 172.21.15.89:0/3657150031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29cc159760 0x7f29cc0c6c70 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f29cc06dd20 tx=0x7f29bc017000 comp rx=0 tx=0).stop 2023-10-10T20:42:56.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.729+0000 7f29d26eb700 1 -- 172.21.15.89:0/3657150031 shutdown_connections 2023-10-10T20:42:56.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.729+0000 7f29d26eb700 1 --2- 172.21.15.89:0/3657150031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f29cc159b30 0x7f29cc1567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.729+0000 7f29d26eb700 1 --2- 172.21.15.89:0/3657150031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f29b0067320 0x7f29b00697d0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.729+0000 7f29d26eb700 1 --2- 172.21.15.89:0/3657150031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29cc06eab0 0x7f29cc0b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.729+0000 7f29d26eb700 1 --2- 172.21.15.89:0/3657150031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29cc159760 0x7f29cc0c6c70 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.729+0000 7f29d26eb700 1 -- 172.21.15.89:0/3657150031 >> 172.21.15.89:0/3657150031 conn(0x7f29cc06b3f0 msgr2=0x7f29cc06c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:56.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.729+0000 7f29d26eb700 1 -- 172.21.15.89:0/3657150031 shutdown_connections 2023-10-10T20:42:56.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.729+0000 7f29d26eb700 1 -- 172.21.15.89:0/3657150031 wait complete. 2023-10-10T20:42:56.735 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 110 2023-10-10T20:42:56.743 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:42:56.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.856+0000 7fdfd1598700 1 Processor -- start 2023-10-10T20:42:56.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.856+0000 7fdfd1598700 1 -- start start 2023-10-10T20:42:56.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.857+0000 7fdfd1598700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdfcc159420 0x7fdfcc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:56.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.857+0000 7fdfd1598700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdfcc1597f0 0x7fdfcc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:56.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.857+0000 7fdfd1598700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdfcc15d230 0x7fdfcc15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:56.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.857+0000 7fdfd1598700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfcc15dbb0 con 0x7fdfcc1597f0 2023-10-10T20:42:56.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.857+0000 7fdfd1598700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfcc0b11d0 con 0x7fdfcc159420 2023-10-10T20:42:56.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.857+0000 7fdfd1598700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc0b14b0 con 0x7fdfcc15d230 2023-10-10T20:42:56.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.857+0000 7fdfcb7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdfcc1597f0 0x7fdfcc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:56.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.857+0000 7fdfcb7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdfcc1597f0 0x7fdfcc0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55052/0 (socket says 172.21.15.89:55052) 2023-10-10T20:42:56.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.857+0000 7fdfcb7fe700 1 -- 172.21.15.89:0/2995251577 learned_addr learned my addr 172.21.15.89:0/2995251577 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:56.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.857+0000 7fdfcbfff700 1 --2- 172.21.15.89:0/2995251577 >> v2:172.21.15.111:3300/0 conn(0x7fdfcc159420 0x7fdfcc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:56.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.857+0000 7fdfd0d97700 1 --2- 172.21.15.89:0/2995251577 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdfcc15d230 0x7fdfcc15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:56.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.858+0000 7fdfcb7fe700 1 -- 172.21.15.89:0/2995251577 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdfcc15d230 msgr2=0x7fdfcc15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:56.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.858+0000 7fdfcb7fe700 1 --2- 172.21.15.89:0/2995251577 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdfcc15d230 0x7fdfcc15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.858+0000 7fdfcb7fe700 1 -- 172.21.15.89:0/2995251577 >> v2:172.21.15.111:3300/0 conn(0x7fdfcc159420 msgr2=0x7fdfcc0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:56.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.858+0000 7fdfcb7fe700 1 --2- 172.21.15.89:0/2995251577 >> v2:172.21.15.111:3300/0 conn(0x7fdfcc159420 0x7fdfcc0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.858+0000 7fdfcb7fe700 1 -- 172.21.15.89:0/2995251577 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc0b1790 con 0x7fdfcc1597f0 2023-10-10T20:42:56.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.858+0000 7fdfcb7fe700 1 --2- 172.21.15.89:0/2995251577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfcc1597f0 0x7fdfcc0b2270 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0b2f20 tx=0x7fdfc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2d55bdf7fbb389 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:56.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.858+0000 7fdfcaffd700 1 -- 172.21.15.89:0/2995251577 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc000b860 con 0x7fdfcc1597f0 2023-10-10T20:42:56.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.858+0000 7fdfcaffd700 1 -- 172.21.15.89:0/2995251577 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc000b9c0 con 0x7fdfcc1597f0 2023-10-10T20:42:56.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.858+0000 7fdfcaffd700 1 -- 172.21.15.89:0/2995251577 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc000bd60 con 0x7fdfcc1597f0 2023-10-10T20:42:56.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.858+0000 7fdfd1598700 1 -- 172.21.15.89:0/2995251577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfcc1597f0 msgr2=0x7fdfcc0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:56.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.858+0000 7fdfd1598700 1 --2- 172.21.15.89:0/2995251577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfcc1597f0 0x7fdfcc0b2270 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0b2f20 tx=0x7fdfc0009250 comp rx=0 tx=0).stop 2023-10-10T20:42:56.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.858+0000 7fdfd1598700 1 -- 172.21.15.89:0/2995251577 shutdown_connections 2023-10-10T20:42:56.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.858+0000 7fdfd1598700 1 --2- 172.21.15.89:0/2995251577 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdfcc15d230 0x7fdfcc15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.858+0000 7fdfd1598700 1 --2- 172.21.15.89:0/2995251577 >> v2:172.21.15.111:3300/0 conn(0x7fdfcc159420 0x7fdfcc0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.858+0000 7fdfd1598700 1 --2- 172.21.15.89:0/2995251577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfcc1597f0 0x7fdfcc0b2270 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.858+0000 7fdfd1598700 1 -- 172.21.15.89:0/2995251577 >> 172.21.15.89:0/2995251577 conn(0x7fdfcc0d0660 msgr2=0x7fdfcc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:56.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.858+0000 7fdfd1598700 1 -- 172.21.15.89:0/2995251577 shutdown_connections 2023-10-10T20:42:56.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.858+0000 7fdfd1598700 1 -- 172.21.15.89:0/2995251577 wait complete. 2023-10-10T20:42:56.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.860+0000 7fdfd1598700 1 Processor -- start 2023-10-10T20:42:56.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.860+0000 7fdfd1598700 1 -- start start 2023-10-10T20:42:56.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.860+0000 7fdfd1598700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfcc159420 0x7fdfcc154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:56.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.860+0000 7fdfd1598700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdfcc1597f0 0x7fdfcc0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:56.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.860+0000 7fdfd1598700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdfcc15d230 0x7fdfcc0b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:56.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.860+0000 7fdfd1598700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc1547a0 con 0x7fdfcc159420 2023-10-10T20:42:56.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.860+0000 7fdfd1598700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc15beb0 con 0x7fdfcc1597f0 2023-10-10T20:42:56.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.860+0000 7fdfd1598700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc15c190 con 0x7fdfcc15d230 2023-10-10T20:42:56.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.861+0000 7fdfcbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfcc159420 0x7fdfcc154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:56.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.861+0000 7fdfcbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfcc159420 0x7fdfcc154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55054/0 (socket says 172.21.15.89:55054) 2023-10-10T20:42:56.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.861+0000 7fdfcbfff700 1 -- 172.21.15.89:0/3350475092 learned_addr learned my addr 172.21.15.89:0/3350475092 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:56.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.861+0000 7fdfd0d97700 1 --2- 172.21.15.89:0/3350475092 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdfcc15d230 0x7fdfcc0b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:56.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.861+0000 7fdfcb7fe700 1 --2- 172.21.15.89:0/3350475092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdfcc1597f0 0x7fdfcc0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:56.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.861+0000 7fdfcbfff700 1 -- 172.21.15.89:0/3350475092 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdfcc15d230 msgr2=0x7fdfcc0b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:56.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.861+0000 7fdfcbfff700 1 --2- 172.21.15.89:0/3350475092 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdfcc15d230 0x7fdfcc0b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.861+0000 7fdfcbfff700 1 -- 172.21.15.89:0/3350475092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdfcc1597f0 msgr2=0x7fdfcc0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:56.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.861+0000 7fdfcbfff700 1 --2- 172.21.15.89:0/3350475092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdfcc1597f0 0x7fdfcc0b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:56.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.861+0000 7fdfcbfff700 1 -- 172.21.15.89:0/3350475092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc00094f0 con 0x7fdfcc159420 2023-10-10T20:42:56.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.861+0000 7fdfcbfff700 1 --2- 172.21.15.89:0/3350475092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfcc159420 0x7fdfcc154260 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00d890 tx=0x7fdfbc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:56.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.861+0000 7fdfb3fff700 1 -- 172.21.15.89:0/3350475092 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc00b600 con 0x7fdfcc159420 2023-10-10T20:42:56.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.861+0000 7fdfb3fff700 1 -- 172.21.15.89:0/3350475092 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc00b760 con 0x7fdfcc159420 2023-10-10T20:42:56.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.861+0000 7fdfd1598700 1 -- 172.21.15.89:0/3350475092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfcc15c470 con 0x7fdfcc159420 2023-10-10T20:42:56.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.861+0000 7fdfb3fff700 1 -- 172.21.15.89:0/3350475092 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc00bae0 con 0x7fdfcc159420 2023-10-10T20:42:56.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.861+0000 7fdfd1598700 1 -- 172.21.15.89:0/3350475092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfcc15b100 con 0x7fdfcc159420 2023-10-10T20:42:56.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.862+0000 7fdfd1598700 1 -- 172.21.15.89:0/3350475092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfcc15c910 con 0x7fdfcc159420 2023-10-10T20:42:56.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.863+0000 7fdfb3fff700 1 -- 172.21.15.89:0/3350475092 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdfbc01e460 con 0x7fdfcc159420 2023-10-10T20:42:56.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.863+0000 7fdfb3fff700 1 --2- 172.21.15.89:0/3350475092 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdfb8067320 0x7fdfb80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:56.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.863+0000 7fdfb3fff700 1 -- 172.21.15.89:0/3350475092 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdfbc085320 con 0x7fdfcc159420 2023-10-10T20:42:56.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.863+0000 7fdfcb7fe700 1 --2- 172.21.15.89:0/3350475092 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdfb8067320 0x7fdfb80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:56.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.863+0000 7fdfcb7fe700 1 --2- 172.21.15.89:0/3350475092 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdfb8067320 0x7fdfb80697d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0b2f20 tx=0x7fdfc0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:56.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:56.867+0000 7fdfb3fff700 1 -- 172.21.15.89:0/3350475092 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdfbc051fb0 con 0x7fdfcc159420 2023-10-10T20:42:57.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.051+0000 7fdfd1598700 1 -- 172.21.15.89:0/3350475092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdfcc1697a0 con 0x7fdfcc159420 2023-10-10T20:42:57.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.052+0000 7fdfb3fff700 1 -- 172.21.15.89:0/3350475092 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+15364 (secure 0 0 0) 0x7fdfbc0555d0 con 0x7fdfcc159420 2023-10-10T20:42:57.054 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:57.055 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":165,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:42:54.331544+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:42:51.066663+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-10T20:42:51.675525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:42:57.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.054+0000 7fdfd1598700 1 -- 172.21.15.89:0/3350475092 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdfb8067320 msgr2=0x7fdfb80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:57.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.054+0000 7fdfd1598700 1 --2- 172.21.15.89:0/3350475092 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdfb8067320 0x7fdfb80697d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0b2f20 tx=0x7fdfc0017000 comp rx=0 tx=0).stop 2023-10-10T20:42:57.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.054+0000 7fdfd1598700 1 -- 172.21.15.89:0/3350475092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfcc159420 msgr2=0x7fdfcc154260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:57.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.054+0000 7fdfd1598700 1 --2- 172.21.15.89:0/3350475092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfcc159420 0x7fdfcc154260 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00d890 tx=0x7fdfbc008040 comp rx=0 tx=0).stop 2023-10-10T20:42:57.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.054+0000 7fdfd1598700 1 -- 172.21.15.89:0/3350475092 shutdown_connections 2023-10-10T20:42:57.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.054+0000 7fdfd1598700 1 --2- 172.21.15.89:0/3350475092 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdfcc15d230 0x7fdfcc0b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.054+0000 7fdfd1598700 1 --2- 172.21.15.89:0/3350475092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfcc159420 0x7fdfcc154260 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.054+0000 7fdfd1598700 1 --2- 172.21.15.89:0/3350475092 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdfb8067320 0x7fdfb80697d0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.054+0000 7fdfd1598700 1 --2- 172.21.15.89:0/3350475092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdfcc1597f0 0x7fdfcc0b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.054+0000 7fdfd1598700 1 -- 172.21.15.89:0/3350475092 >> 172.21.15.89:0/3350475092 conn(0x7fdfcc0d0660 msgr2=0x7fdfcc156a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:57.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.054+0000 7fdfd1598700 1 -- 172.21.15.89:0/3350475092 shutdown_connections 2023-10-10T20:42:57.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.054+0000 7fdfd1598700 1 -- 172.21.15.89:0/3350475092 wait complete. 2023-10-10T20:42:57.068 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:42:57.069 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:42:57.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.182+0000 7f6ae9e93700 1 Processor -- start 2023-10-10T20:42:57.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.182+0000 7f6ae9e93700 1 -- start start 2023-10-10T20:42:57.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.183+0000 7f6ae9e93700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6ae4141500 0x7f6ae40c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.183+0000 7f6ae9e93700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ae406c0a0 0x7f6ae40c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.183+0000 7f6ae9e93700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae415c2a0 con 0x7f6ae406bcd0 2023-10-10T20:42:57.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.183+0000 7f6ae9e93700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae40c9880 con 0x7f6ae4141500 2023-10-10T20:42:57.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.183+0000 7f6ae9e93700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae415b690 con 0x7f6ae406c0a0 2023-10-10T20:42:57.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.183+0000 7f6ae37fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6ae406bcd0 0x7f6ae4140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48826/0 (socket says 172.21.15.89:48826) 2023-10-10T20:42:57.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.183+0000 7f6ae37fe700 1 -- 172.21.15.89:0/1328405234 learned_addr learned my addr 172.21.15.89:0/1328405234 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:57.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.183+0000 7f6ae27fc700 1 -- 172.21.15.89:0/1328405234 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2215206421 0 0) 0x7f6ae415c2a0 con 0x7f6ae406bcd0 2023-10-10T20:42:57.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.183+0000 7f6ae27fc700 1 -- 172.21.15.89:0/1328405234 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6acc003570 con 0x7f6ae406bcd0 2023-10-10T20:42:57.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.183+0000 7f6ae2ffd700 1 --2- 172.21.15.89:0/1328405234 >> v2:172.21.15.111:3300/0 conn(0x7f6ae4141500 0x7f6ae40c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.183+0000 7f6ae3fff700 1 --2- 172.21.15.89:0/1328405234 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ae406c0a0 0x7f6ae40c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.183+0000 7f6ae27fc700 1 -- 172.21.15.89:0/1328405234 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3307649279 0 0) 0x7f6acc003570 con 0x7f6ae406bcd0 2023-10-10T20:42:57.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.183+0000 7f6ae27fc700 1 -- 172.21.15.89:0/1328405234 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ae406c0a0 msgr2=0x7f6ae40c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:57.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.183+0000 7f6ae27fc700 1 --2- 172.21.15.89:0/1328405234 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ae406c0a0 0x7f6ae40c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.183+0000 7f6ae27fc700 1 -- 172.21.15.89:0/1328405234 >> v2:172.21.15.111:3300/0 conn(0x7f6ae4141500 msgr2=0x7f6ae40c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:57.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.183+0000 7f6ae27fc700 1 --2- 172.21.15.89:0/1328405234 >> v2:172.21.15.111:3300/0 conn(0x7f6ae4141500 0x7f6ae40c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.183+0000 7f6ae27fc700 1 -- 172.21.15.89:0/1328405234 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae415b970 con 0x7f6ae406bcd0 2023-10-10T20:42:57.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.183+0000 7f6ae27fc700 1 -- 172.21.15.89:0/1328405234 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f6ad4001470 con 0x7f6ae406bcd0 2023-10-10T20:42:57.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae27fc700 1 -- 172.21.15.89:0/1328405234 >> v1:172.21.15.89:6789/0 conn(0x7f6ae406bcd0 legacy=0x7f6ae4140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:57.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae27fc700 1 --2- 172.21.15.89:0/1328405234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6acc003a60 0x7f6acc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae27fc700 1 --2- 172.21.15.89:0/1328405234 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ae406c0a0 0x7f6acc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae27fc700 1 --2- 172.21.15.89:0/1328405234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6acc004d50 0x7f6acc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae27fc700 1 -- 172.21.15.89:0/1328405234 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae40c9880 con 0x7f6acc004d50 2023-10-10T20:42:57.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae27fc700 1 -- 172.21.15.89:0/1328405234 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae415b690 con 0x7f6acc003a60 2023-10-10T20:42:57.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae27fc700 1 -- 172.21.15.89:0/1328405234 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc00bae0 con 0x7f6ae406c0a0 2023-10-10T20:42:57.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae2ffd700 1 --2- 172.21.15.89:0/1328405234 >> v2:172.21.15.111:3300/0 conn(0x7f6ae4141500 0x7f6ae40c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:57.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae3fff700 1 --2- 172.21.15.89:0/1328405234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6acc004d50 0x7f6acc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae37fe700 1 --2- 172.21.15.89:0/1328405234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6acc003a60 0x7f6acc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae2ffd700 1 --2- 172.21.15.89:0/1328405234 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ae406c0a0 0x7f6acc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae3fff700 1 -- 172.21.15.89:0/1328405234 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ae406c0a0 msgr2=0x7f6acc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:57.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae3fff700 1 --2- 172.21.15.89:0/1328405234 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ae406c0a0 0x7f6acc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae3fff700 1 -- 172.21.15.89:0/1328405234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6acc003a60 msgr2=0x7f6acc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:57.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae3fff700 1 --2- 172.21.15.89:0/1328405234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6acc003a60 0x7f6acc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae3fff700 1 -- 172.21.15.89:0/1328405234 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc003570 con 0x7f6acc004d50 2023-10-10T20:42:57.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae2ffd700 1 --2- 172.21.15.89:0/1328405234 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ae406c0a0 0x7f6acc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:42:57.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae3fff700 1 --2- 172.21.15.89:0/1328405234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6acc004d50 0x7f6acc0051c0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00e210 tx=0x7f6adc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6898787041dfa0a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:57.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae27fc700 1 -- 172.21.15.89:0/1328405234 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc00eac0 con 0x7f6acc004d50 2023-10-10T20:42:57.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae27fc700 1 -- 172.21.15.89:0/1328405234 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc00f040 con 0x7f6acc004d50 2023-10-10T20:42:57.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.184+0000 7f6ae27fc700 1 -- 172.21.15.89:0/1328405234 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc00ee70 con 0x7f6acc004d50 2023-10-10T20:42:57.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.185+0000 7f6ae9e93700 1 -- 172.21.15.89:0/1328405234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6acc004d50 msgr2=0x7f6acc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:57.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.185+0000 7f6ae9e93700 1 --2- 172.21.15.89:0/1328405234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6acc004d50 0x7f6acc0051c0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00e210 tx=0x7f6adc00b040 comp rx=0 tx=0).stop 2023-10-10T20:42:57.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.185+0000 7f6ae3fff700 1 -- 172.21.15.89:0/1328405234 reap_dead start 2023-10-10T20:42:57.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.185+0000 7f6ae9e93700 1 -- 172.21.15.89:0/1328405234 shutdown_connections 2023-10-10T20:42:57.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.185+0000 7f6ae9e93700 1 -- 172.21.15.89:0/1328405234 >> 172.21.15.89:0/1328405234 conn(0x7f6ae40ce600 msgr2=0x7f6ae40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:57.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.185+0000 7f6ae9e93700 1 -- 172.21.15.89:0/1328405234 shutdown_connections 2023-10-10T20:42:57.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.185+0000 7f6ae9e93700 1 -- 172.21.15.89:0/1328405234 wait complete. 2023-10-10T20:42:57.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.186+0000 7f6ae9e93700 1 Processor -- start 2023-10-10T20:42:57.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.186+0000 7f6ae9e93700 1 -- start start 2023-10-10T20:42:57.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6ae9e93700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ae4157560 0x7f6ae40b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6ae9e93700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ae40c6070 0x7f6ae40c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6ae9e93700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ae40c64e0 0x7f6ae40c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6ae9e93700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae40b3c10 con 0x7f6ae4157560 2023-10-10T20:42:57.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6ae9e93700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae40b2320 con 0x7f6ae40c6070 2023-10-10T20:42:57.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6ae9e93700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae40b2cf0 con 0x7f6ae40c64e0 2023-10-10T20:42:57.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6ae37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ae4157560 0x7f6ae40b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6ae37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ae4157560 0x7f6ae40b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55068/0 (socket says 172.21.15.89:55068) 2023-10-10T20:42:57.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6ae37fe700 1 -- 172.21.15.89:0/2627027240 learned_addr learned my addr 172.21.15.89:0/2627027240 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:57.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6ae3fff700 1 --2- 172.21.15.89:0/2627027240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ae40c64e0 0x7f6ae40c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6ae2ffd700 1 --2- 172.21.15.89:0/2627027240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ae40c6070 0x7f6ae40c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6ae37fe700 1 -- 172.21.15.89:0/2627027240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ae40c64e0 msgr2=0x7f6ae40c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:57.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6ae37fe700 1 --2- 172.21.15.89:0/2627027240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ae40c64e0 0x7f6ae40c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6ae37fe700 1 -- 172.21.15.89:0/2627027240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ae40c6070 msgr2=0x7f6ae40c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:57.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6ae37fe700 1 --2- 172.21.15.89:0/2627027240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ae40c6070 0x7f6ae40c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6ae37fe700 1 -- 172.21.15.89:0/2627027240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae40b1710 con 0x7f6ae4157560 2023-10-10T20:42:57.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6ae37fe700 1 --2- 172.21.15.89:0/2627027240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ae4157560 0x7f6ae40b27b0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4004890 tx=0x7f6ad4001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:57.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6acb7fe700 1 -- 172.21.15.89:0/2627027240 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4018030 con 0x7f6ae4157560 2023-10-10T20:42:57.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6acb7fe700 1 -- 172.21.15.89:0/2627027240 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad40077a0 con 0x7f6ae4157560 2023-10-10T20:42:57.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6ae9e93700 1 -- 172.21.15.89:0/2627027240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae40b19f0 con 0x7f6ae4157560 2023-10-10T20:42:57.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6acb7fe700 1 -- 172.21.15.89:0/2627027240 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad400fa70 con 0x7f6ae4157560 2023-10-10T20:42:57.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.187+0000 7f6ae9e93700 1 -- 172.21.15.89:0/2627027240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae40b1fb0 con 0x7f6ae4157560 2023-10-10T20:42:57.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.188+0000 7f6ae9e93700 1 -- 172.21.15.89:0/2627027240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae404c3c0 con 0x7f6ae4157560 2023-10-10T20:42:57.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.189+0000 7f6acb7fe700 1 -- 172.21.15.89:0/2627027240 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6ad401b070 con 0x7f6ae4157560 2023-10-10T20:42:57.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.189+0000 7f6acb7fe700 1 --2- 172.21.15.89:0/2627027240 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ad0067320 0x7f6ad00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.189+0000 7f6acb7fe700 1 -- 172.21.15.89:0/2627027240 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6ad4019070 con 0x7f6ae4157560 2023-10-10T20:42:57.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.189+0000 7f6ae2ffd700 1 --2- 172.21.15.89:0/2627027240 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ad0067320 0x7f6ad00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.190+0000 7f6ae2ffd700 1 --2- 172.21.15.89:0/2627027240 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ad0067320 0x7f6ad00697d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40c8b30 tx=0x7f6ad800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:57.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.194+0000 7f6acb7fe700 1 -- 172.21.15.89:0/2627027240 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ad4051780 con 0x7f6ae4157560 2023-10-10T20:42:57.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.375+0000 7f6ae9e93700 1 -- 172.21.15.89:0/2627027240 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ae415bc50 con 0x7f6ad0067320 2023-10-10T20:42:57.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.383+0000 7f6acb7fe700 1 -- 172.21.15.89:0/2627027240 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286942 (secure 0 0 0) 0x7f6ae415bc50 con 0x7f6ad0067320 2023-10-10T20:42:57.392 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:57.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.386+0000 7f6ae9e93700 1 -- 172.21.15.89:0/2627027240 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ad0067320 msgr2=0x7f6ad00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:57.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.386+0000 7f6ae9e93700 1 --2- 172.21.15.89:0/2627027240 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ad0067320 0x7f6ad00697d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40c8b30 tx=0x7f6ad800b040 comp rx=0 tx=0).stop 2023-10-10T20:42:57.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.386+0000 7f6ae9e93700 1 -- 172.21.15.89:0/2627027240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ae4157560 msgr2=0x7f6ae40b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:57.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.386+0000 7f6ae9e93700 1 --2- 172.21.15.89:0/2627027240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ae4157560 0x7f6ae40b27b0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4004890 tx=0x7f6ad4001e90 comp rx=0 tx=0).stop 2023-10-10T20:42:57.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.386+0000 7f6ae9e93700 1 -- 172.21.15.89:0/2627027240 shutdown_connections 2023-10-10T20:42:57.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.386+0000 7f6ae9e93700 1 --2- 172.21.15.89:0/2627027240 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6ae40c64e0 0x7f6ae40c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.386+0000 7f6ae9e93700 1 --2- 172.21.15.89:0/2627027240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ae4157560 0x7f6ae40b27b0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.386+0000 7f6ae9e93700 1 --2- 172.21.15.89:0/2627027240 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6ad0067320 0x7f6ad00697d0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.386+0000 7f6ae9e93700 1 --2- 172.21.15.89:0/2627027240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ae40c6070 0x7f6ae40c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.386+0000 7f6ae9e93700 1 -- 172.21.15.89:0/2627027240 >> 172.21.15.89:0/2627027240 conn(0x7f6ae40ce600 msgr2=0x7f6ae40cae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:57.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.386+0000 7f6ae9e93700 1 -- 172.21.15.89:0/2627027240 shutdown_connections 2023-10-10T20:42:57.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.386+0000 7f6ae9e93700 1 -- 172.21.15.89:0/2627027240 wait complete. 2023-10-10T20:42:57.397 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:42:57.402 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2023-10-10T20:42:56.751262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428364,"kb_used_data":17292,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752546356,"statfs":{"total":773094113280,"available":770607468544,"internally_reserved":0,"allocated":17707008,"data_stored":8556385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.433293"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.323231+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:40.796551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.325806+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:28:12.284338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.331625+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:09:34.459455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.330945+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:16:50.053827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329837+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:16:07.451863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.327658+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:58:12.432427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.326884+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:36:30.295360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322678+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:59.692515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329758+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:02:55.597436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.327137+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:32:08.240124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.323857+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:32.579274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.325054+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:36:30.295360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.324653+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:03.524684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329133+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:41:28.090635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329269+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:38:16.787078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.324787+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:58:32.593330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328589+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:35.674118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.326650+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:38:30.259757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.334129+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:20:38.627889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.327487+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:27.171367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329421+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:18:18.498552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.330301+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:00:48.564823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.323234+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:58:00.782955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328346+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:02:55.597436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322747+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:25:50.618140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322798+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:04:51.509487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.323414+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:20:34.093326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.326676+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:33:29.535005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328679+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:50:07.909344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322817+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:44:40.439820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.330007+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:02.852226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329533+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:13:20.878853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328750+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:46.022458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.335968+0000","last_change":"2023-10-10T20:42:52.335968+0000","last_active":"2023-10-10T20:42:52.335968+0000","last_peered":"2023-10-10T20:42:52.335968+0000","last_clean":"2023-10-10T20:42:52.335968+0000","last_became_active":"2023-10-10T20:42:52.335695+0000","last_became_peered":"2023-10-10T20:42:52.335695+0000","last_unstale":"2023-10-10T20:42:52.335968+0000","last_undegraded":"2023-10-10T20:42:52.335968+0000","last_fullsized":"2023-10-10T20:42:52.335968+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:32:07.233388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322742+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:39:23.346373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.336051+0000","last_change":"2023-10-10T20:42:52.336051+0000","last_active":"2023-10-10T20:42:52.336051+0000","last_peered":"2023-10-10T20:42:52.336051+0000","last_clean":"2023-10-10T20:42:52.336051+0000","last_became_active":"2023-10-10T20:42:52.335752+0000","last_became_peered":"2023-10-10T20:42:52.335752+0000","last_unstale":"2023-10-10T20:42:52.336051+0000","last_undegraded":"2023-10-10T20:42:52.336051+0000","last_fullsized":"2023-10-10T20:42:52.336051+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:15:01.473071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329028+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:00:56.184401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.339512+0000","last_change":"2023-10-10T20:42:52.339512+0000","last_active":"2023-10-10T20:42:52.339512+0000","last_peered":"2023-10-10T20:42:52.339512+0000","last_clean":"2023-10-10T20:42:52.339512+0000","last_became_active":"2023-10-10T20:42:52.339230+0000","last_became_peered":"2023-10-10T20:42:52.339230+0000","last_unstale":"2023-10-10T20:42:52.339512+0000","last_undegraded":"2023-10-10T20:42:52.339512+0000","last_fullsized":"2023-10-10T20:42:52.339512+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:51.664988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322141+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:45:05.557928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.332410+0000","last_change":"2023-10-10T20:42:52.332410+0000","last_active":"2023-10-10T20:42:52.332410+0000","last_peered":"2023-10-10T20:42:52.332410+0000","last_clean":"2023-10-10T20:42:52.332410+0000","last_became_active":"2023-10-10T20:42:52.331952+0000","last_became_peered":"2023-10-10T20:42:52.331952+0000","last_unstale":"2023-10-10T20:42:52.332410+0000","last_undegraded":"2023-10-10T20:42:52.332410+0000","last_fullsized":"2023-10-10T20:42:52.332410+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:46:55.907438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328114+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:00.861926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.338841+0000","last_change":"2023-10-10T20:42:52.338841+0000","last_active":"2023-10-10T20:42:52.338841+0000","last_peered":"2023-10-10T20:42:52.338841+0000","last_clean":"2023-10-10T20:42:52.338841+0000","last_became_active":"2023-10-10T20:42:52.337405+0000","last_became_peered":"2023-10-10T20:42:52.337405+0000","last_unstale":"2023-10-10T20:42:52.338841+0000","last_undegraded":"2023-10-10T20:42:52.338841+0000","last_fullsized":"2023-10-10T20:42:52.338841+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:10:49.794184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.327089+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:52.463061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.333926+0000","last_change":"2023-10-10T20:42:52.333926+0000","last_active":"2023-10-10T20:42:52.333926+0000","last_peered":"2023-10-10T20:42:52.333926+0000","last_clean":"2023-10-10T20:42:52.333926+0000","last_became_active":"2023-10-10T20:42:52.333645+0000","last_became_peered":"2023-10-10T20:42:52.333645+0000","last_unstale":"2023-10-10T20:42:52.333926+0000","last_undegraded":"2023-10-10T20:42:52.333926+0000","last_fullsized":"2023-10-10T20:42:52.333926+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:46:47.379359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.324299+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:17:23.703124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.331824+0000","last_change":"2023-10-10T20:42:52.331824+0000","last_active":"2023-10-10T20:42:52.331824+0000","last_peered":"2023-10-10T20:42:52.331824+0000","last_clean":"2023-10-10T20:42:52.331824+0000","last_became_active":"2023-10-10T20:42:52.331567+0000","last_became_peered":"2023-10-10T20:42:52.331567+0000","last_unstale":"2023-10-10T20:42:52.331824+0000","last_undegraded":"2023-10-10T20:42:52.331824+0000","last_fullsized":"2023-10-10T20:42:52.331824+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:59.776219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.325326+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:30:49.951335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.329973+0000","last_change":"2023-10-10T20:42:52.329973+0000","last_active":"2023-10-10T20:42:52.329973+0000","last_peered":"2023-10-10T20:42:52.329973+0000","last_clean":"2023-10-10T20:42:52.329973+0000","last_became_active":"2023-10-10T20:42:52.329001+0000","last_became_peered":"2023-10-10T20:42:52.329001+0000","last_unstale":"2023-10-10T20:42:52.329973+0000","last_undegraded":"2023-10-10T20:42:52.329973+0000","last_fullsized":"2023-10-10T20:42:52.329973+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:20:37.621153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322179+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:13:45.883755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.334832+0000","last_change":"2023-10-10T20:42:52.334832+0000","last_active":"2023-10-10T20:42:52.334832+0000","last_peered":"2023-10-10T20:42:52.334832+0000","last_clean":"2023-10-10T20:42:52.334832+0000","last_became_active":"2023-10-10T20:42:52.334555+0000","last_became_peered":"2023-10-10T20:42:52.334555+0000","last_unstale":"2023-10-10T20:42:52.334832+0000","last_undegraded":"2023-10-10T20:42:52.334832+0000","last_fullsized":"2023-10-10T20:42:52.334832+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:51.456325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328250+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:31.176991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.341985+0000","last_change":"2023-10-10T20:42:52.341985+0000","last_active":"2023-10-10T20:42:52.341985+0000","last_peered":"2023-10-10T20:42:52.341985+0000","last_clean":"2023-10-10T20:42:52.341985+0000","last_became_active":"2023-10-10T20:42:52.341730+0000","last_became_peered":"2023-10-10T20:42:52.341730+0000","last_unstale":"2023-10-10T20:42:52.341985+0000","last_undegraded":"2023-10-10T20:42:52.341985+0000","last_fullsized":"2023-10-10T20:42:52.341985+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:32:48.988699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322090+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:17:25.969866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.333830+0000","last_change":"2023-10-10T20:42:52.333830+0000","last_active":"2023-10-10T20:42:52.333830+0000","last_peered":"2023-10-10T20:42:52.333830+0000","last_clean":"2023-10-10T20:42:52.333830+0000","last_became_active":"2023-10-10T20:42:52.333570+0000","last_became_peered":"2023-10-10T20:42:52.333570+0000","last_unstale":"2023-10-10T20:42:52.333830+0000","last_undegraded":"2023-10-10T20:42:52.333830+0000","last_fullsized":"2023-10-10T20:42:52.333830+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:31.572538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329839+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:18:15.714890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.339386+0000","last_change":"2023-10-10T20:42:52.339386+0000","last_active":"2023-10-10T20:42:52.339386+0000","last_peered":"2023-10-10T20:42:52.339386+0000","last_clean":"2023-10-10T20:42:52.339386+0000","last_became_active":"2023-10-10T20:42:52.339072+0000","last_became_peered":"2023-10-10T20:42:52.339072+0000","last_unstale":"2023-10-10T20:42:52.339386+0000","last_undegraded":"2023-10-10T20:42:52.339386+0000","last_fullsized":"2023-10-10T20:42:52.339386+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:53:19.603938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.333958+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:20:38.627889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.337702+0000","last_change":"2023-10-10T20:42:52.337702+0000","last_active":"2023-10-10T20:42:52.337702+0000","last_peered":"2023-10-10T20:42:52.337702+0000","last_clean":"2023-10-10T20:42:52.337702+0000","last_became_active":"2023-10-10T20:42:52.337415+0000","last_became_peered":"2023-10-10T20:42:52.337415+0000","last_unstale":"2023-10-10T20:42:52.337702+0000","last_undegraded":"2023-10-10T20:42:52.337702+0000","last_fullsized":"2023-10-10T20:42:52.337702+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:33:28.528269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.331014+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:05:24.092738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.338838+0000","last_change":"2023-10-10T20:42:52.338838+0000","last_active":"2023-10-10T20:42:52.338838+0000","last_peered":"2023-10-10T20:42:52.338838+0000","last_clean":"2023-10-10T20:42:52.338838+0000","last_became_active":"2023-10-10T20:42:52.338592+0000","last_became_peered":"2023-10-10T20:42:52.338592+0000","last_unstale":"2023-10-10T20:42:52.338838+0000","last_undegraded":"2023-10-10T20:42:52.338838+0000","last_fullsized":"2023-10-10T20:42:52.338838+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:36:29.288624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328957+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:46:36.668645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.334750+0000","last_change":"2023-10-10T20:42:52.334750+0000","last_active":"2023-10-10T20:42:52.334750+0000","last_peered":"2023-10-10T20:42:52.334750+0000","last_clean":"2023-10-10T20:42:52.334750+0000","last_became_active":"2023-10-10T20:42:52.334260+0000","last_became_peered":"2023-10-10T20:42:52.334260+0000","last_unstale":"2023-10-10T20:42:52.334750+0000","last_undegraded":"2023-10-10T20:42:52.334750+0000","last_fullsized":"2023-10-10T20:42:52.334750+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:49:39.517807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322365+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:04:02.591369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.339183+0000","last_change":"2023-10-10T20:42:52.339183+0000","last_active":"2023-10-10T20:42:52.339183+0000","last_peered":"2023-10-10T20:42:52.339183+0000","last_clean":"2023-10-10T20:42:52.339183+0000","last_became_active":"2023-10-10T20:42:52.337722+0000","last_became_peered":"2023-10-10T20:42:52.337722+0000","last_unstale":"2023-10-10T20:42:52.339183+0000","last_undegraded":"2023-10-10T20:42:52.339183+0000","last_fullsized":"2023-10-10T20:42:52.339183+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:51.664988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.326616+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:57:12.184703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.335410+0000","last_change":"2023-10-10T20:42:52.335410+0000","last_active":"2023-10-10T20:42:52.335410+0000","last_peered":"2023-10-10T20:42:52.335410+0000","last_clean":"2023-10-10T20:42:52.335410+0000","last_became_active":"2023-10-10T20:42:52.335097+0000","last_became_peered":"2023-10-10T20:42:52.335097+0000","last_unstale":"2023-10-10T20:42:52.335410+0000","last_undegraded":"2023-10-10T20:42:52.335410+0000","last_fullsized":"2023-10-10T20:42:52.335410+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:22:17.318577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.327959+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:58:32.593330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.332490+0000","last_change":"2023-10-10T20:42:52.332490+0000","last_active":"2023-10-10T20:42:52.332490+0000","last_peered":"2023-10-10T20:42:52.332490+0000","last_clean":"2023-10-10T20:42:52.332490+0000","last_became_active":"2023-10-10T20:42:52.332045+0000","last_became_peered":"2023-10-10T20:42:52.332045+0000","last_unstale":"2023-10-10T20:42:52.332490+0000","last_undegraded":"2023-10-10T20:42:52.332490+0000","last_fullsized":"2023-10-10T20:42:52.332490+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:01.845490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.331151+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:18:18.498552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.333544+0000","last_change":"2023-10-10T20:42:52.333544+0000","last_active":"2023-10-10T20:42:52.333544+0000","last_peered":"2023-10-10T20:42:52.333544+0000","last_clean":"2023-10-10T20:42:52.333544+0000","last_became_active":"2023-10-10T20:42:52.333211+0000","last_became_peered":"2023-10-10T20:42:52.333211+0000","last_unstale":"2023-10-10T20:42:52.333544+0000","last_undegraded":"2023-10-10T20:42:52.333544+0000","last_fullsized":"2023-10-10T20:42:52.333544+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:39.789815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328162+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:46.022458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.336202+0000","last_change":"2023-10-10T20:42:52.336202+0000","last_active":"2023-10-10T20:42:52.336202+0000","last_peered":"2023-10-10T20:42:52.336202+0000","last_clean":"2023-10-10T20:42:52.336202+0000","last_became_active":"2023-10-10T20:42:52.335926+0000","last_became_peered":"2023-10-10T20:42:52.335926+0000","last_unstale":"2023-10-10T20:42:52.336202+0000","last_undegraded":"2023-10-10T20:42:52.336202+0000","last_fullsized":"2023-10-10T20:42:52.336202+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:49:39.517807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.333557+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:16:50.053827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.335457+0000","last_change":"2023-10-10T20:42:52.335457+0000","last_active":"2023-10-10T20:42:52.335457+0000","last_peered":"2023-10-10T20:42:52.335457+0000","last_clean":"2023-10-10T20:42:52.335457+0000","last_became_active":"2023-10-10T20:42:52.335186+0000","last_became_peered":"2023-10-10T20:42:52.335186+0000","last_unstale":"2023-10-10T20:42:52.335457+0000","last_undegraded":"2023-10-10T20:42:52.335457+0000","last_fullsized":"2023-10-10T20:42:52.335457+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:55:21.874802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322254+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:21:17.789832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.337248+0000","last_change":"2023-10-10T20:42:52.337248+0000","last_active":"2023-10-10T20:42:52.337248+0000","last_peered":"2023-10-10T20:42:52.337248+0000","last_clean":"2023-10-10T20:42:52.337248+0000","last_became_active":"2023-10-10T20:42:52.336966+0000","last_became_peered":"2023-10-10T20:42:52.336966+0000","last_unstale":"2023-10-10T20:42:52.337248+0000","last_undegraded":"2023-10-10T20:42:52.337248+0000","last_fullsized":"2023-10-10T20:42:52.337248+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:59:14.539048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329553+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:09:34.459455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.350531+0000","last_change":"2023-10-10T20:42:52.350531+0000","last_active":"2023-10-10T20:42:52.350531+0000","last_peered":"2023-10-10T20:42:52.350531+0000","last_clean":"2023-10-10T20:42:52.350531+0000","last_became_active":"2023-10-10T20:42:52.350315+0000","last_became_peered":"2023-10-10T20:42:52.350315+0000","last_unstale":"2023-10-10T20:42:52.350531+0000","last_undegraded":"2023-10-10T20:42:52.350531+0000","last_fullsized":"2023-10-10T20:42:52.350531+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:08.850614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.330288+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:28:12.284338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.338420+0000","last_change":"2023-10-10T20:42:52.338420+0000","last_active":"2023-10-10T20:42:52.338420+0000","last_peered":"2023-10-10T20:42:52.338420+0000","last_clean":"2023-10-10T20:42:52.338420+0000","last_became_active":"2023-10-10T20:42:52.338190+0000","last_became_peered":"2023-10-10T20:42:52.338190+0000","last_unstale":"2023-10-10T20:42:52.338420+0000","last_undegraded":"2023-10-10T20:42:52.338420+0000","last_fullsized":"2023-10-10T20:42:52.338420+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:58:31.586594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.327181+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:30:49.951335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.335469+0000","last_change":"2023-10-10T20:42:52.335469+0000","last_active":"2023-10-10T20:42:52.335469+0000","last_peered":"2023-10-10T20:42:52.335469+0000","last_clean":"2023-10-10T20:42:52.335469+0000","last_became_active":"2023-10-10T20:42:52.333014+0000","last_became_peered":"2023-10-10T20:42:52.333014+0000","last_unstale":"2023-10-10T20:42:52.335469+0000","last_undegraded":"2023-10-10T20:42:52.335469+0000","last_fullsized":"2023-10-10T20:42:52.335469+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:16:49.047090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328435+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:00.861926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.334206+0000","last_change":"2023-10-10T20:42:52.334206+0000","last_active":"2023-10-10T20:42:52.334206+0000","last_peered":"2023-10-10T20:42:52.334206+0000","last_clean":"2023-10-10T20:42:52.334206+0000","last_became_active":"2023-10-10T20:42:52.333022+0000","last_became_peered":"2023-10-10T20:42:52.333022+0000","last_unstale":"2023-10-10T20:42:52.334206+0000","last_undegraded":"2023-10-10T20:42:52.334206+0000","last_fullsized":"2023-10-10T20:42:52.334206+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:34:10.314535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328547+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:05:24.092738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.339808+0000","last_change":"2023-10-10T20:42:52.339808+0000","last_active":"2023-10-10T20:42:52.339808+0000","last_peered":"2023-10-10T20:42:52.339808+0000","last_clean":"2023-10-10T20:42:52.339808+0000","last_became_active":"2023-10-10T20:42:52.339538+0000","last_became_peered":"2023-10-10T20:42:52.339538+0000","last_unstale":"2023-10-10T20:42:52.339808+0000","last_undegraded":"2023-10-10T20:42:52.339808+0000","last_fullsized":"2023-10-10T20:42:52.339808+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:17:22.696388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.324346+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:32:49.995435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.330898+0000","last_change":"2023-10-10T20:42:52.330898+0000","last_active":"2023-10-10T20:42:52.330898+0000","last_peered":"2023-10-10T20:42:52.330898+0000","last_clean":"2023-10-10T20:42:52.330898+0000","last_became_active":"2023-10-10T20:42:52.330616+0000","last_became_peered":"2023-10-10T20:42:52.330616+0000","last_unstale":"2023-10-10T20:42:52.330898+0000","last_undegraded":"2023-10-10T20:42:52.330898+0000","last_fullsized":"2023-10-10T20:42:52.330898+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:30.999244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.333078+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:02.852226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.333625+0000","last_change":"2023-10-10T20:42:52.333625+0000","last_active":"2023-10-10T20:42:52.333625+0000","last_peered":"2023-10-10T20:42:52.333625+0000","last_clean":"2023-10-10T20:42:52.333625+0000","last_became_active":"2023-10-10T20:42:52.333359+0000","last_became_peered":"2023-10-10T20:42:52.333359+0000","last_unstale":"2023-10-10T20:42:52.333625+0000","last_undegraded":"2023-10-10T20:42:52.333625+0000","last_fullsized":"2023-10-10T20:42:52.333625+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:31.572538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328569+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:28:12.284338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.339859+0000","last_change":"2023-10-10T20:42:52.339859+0000","last_active":"2023-10-10T20:42:52.339859+0000","last_peered":"2023-10-10T20:42:52.339859+0000","last_clean":"2023-10-10T20:42:52.339859+0000","last_became_active":"2023-10-10T20:42:52.339585+0000","last_became_peered":"2023-10-10T20:42:52.339585+0000","last_unstale":"2023-10-10T20:42:52.339859+0000","last_undegraded":"2023-10-10T20:42:52.339859+0000","last_fullsized":"2023-10-10T20:42:52.339859+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:02.517948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322090+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:38:10.362086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.335154+0000","last_change":"2023-10-10T20:42:52.335154+0000","last_active":"2023-10-10T20:42:52.335154+0000","last_peered":"2023-10-10T20:42:52.335154+0000","last_clean":"2023-10-10T20:42:52.335154+0000","last_became_active":"2023-10-10T20:42:52.334853+0000","last_became_peered":"2023-10-10T20:42:52.334853+0000","last_unstale":"2023-10-10T20:42:52.335154+0000","last_undegraded":"2023-10-10T20:42:52.335154+0000","last_fullsized":"2023-10-10T20:42:52.335154+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:15:01.473071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322090+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:35.275237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.338269+0000","last_change":"2023-10-10T20:42:52.338269+0000","last_active":"2023-10-10T20:42:52.338269+0000","last_peered":"2023-10-10T20:42:52.338269+0000","last_clean":"2023-10-10T20:42:52.338269+0000","last_became_active":"2023-10-10T20:42:52.338028+0000","last_became_peered":"2023-10-10T20:42:52.338028+0000","last_unstale":"2023-10-10T20:42:52.338269+0000","last_undegraded":"2023-10-10T20:42:52.338269+0000","last_fullsized":"2023-10-10T20:42:52.338269+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:17:34.621354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":306,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-10T20:42:51.319419+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:42:51.319419+0000","last_peered":"2023-10-10T20:42:51.319419+0000","last_clean":"2023-10-10T20:42:51.319419+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:42:51.319419+0000","last_undegraded":"2023-10-10T20:42:51.319419+0000","last_fullsized":"2023-10-10T20:42:51.319419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574953,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303984,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067856,"statfs":{"total":96636764160,"available":96325484544,"internally_reserved":0,"allocated":2662400,"data_stored":1514010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50700000000000001,"15min":0.75800000000000001},"min":{"1min":0.32500000000000001,"5min":0.309,"15min":0.29199999999999998},"max":{"1min":0.64300000000000002,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.50800000000000001,"15min":0.60399999999999998},"min":{"1min":0.36399999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.58299999999999996,"5min":0.752,"15min":4.1959999999999997},"last":0.67000000000000004}]},{"osd":1,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.50600000000000001,"15min":0.90100000000000002},"min":{"1min":0.372,"5min":0.317,"15min":0.317},"max":{"1min":0.61099999999999999,"5min":0.78800000000000003,"15min":10.1},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.53400000000000003,"15min":0.69099999999999995},"min":{"1min":0.35199999999999998,"5min":0.33900000000000002,"15min":0.28999999999999998},"max":{"1min":0.66500000000000004,"5min":0.81799999999999995,"15min":6.806},"last":0.63600000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56799999999999995,"15min":0.74099999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68799999999999994,"5min":0.89000000000000001,"15min":6.9340000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.69399999999999995},"min":{"1min":0.40699999999999997,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.69099999999999995,"5min":0.77100000000000002,"15min":6.9870000000000001},"last":0.747}]},{"osd":3,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54200000000000004,"15min":0.71299999999999997},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64500000000000002,"5min":0.754,"15min":6.6719999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55900000000000005,"15min":0.64800000000000002},"min":{"1min":0.39800000000000002,"5min":0.378,"15min":0.34799999999999998},"max":{"1min":0.64400000000000002,"5min":0.76200000000000001,"15min":4.1609999999999996},"last":0.71899999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47799999999999998,"15min":0.46800000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.77200000000000002},"last":0.5},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.51600000000000001,"15min":0.51900000000000002},"min":{"1min":0.314,"5min":0.309,"15min":0.30399999999999999},"max":{"1min":0.68000000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.51400000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.54000000000000004,"15min":0.53900000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.68400000000000005,"5min":0.86099999999999999,"15min":1.3180000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.53600000000000003,"15min":0.52700000000000002},"min":{"1min":0.35299999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.84099999999999997},"last":0.54600000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.35699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":1.7509999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76300000000000001,"5min":0.84099999999999997,"15min":1.7090000000000001},"last":0.65200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574954,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303404,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068436,"statfs":{"total":96636764160,"available":96326078464,"internally_reserved":0,"allocated":2068480,"data_stored":923642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52400000000000002,"15min":0.64000000000000001},"min":{"1min":0.34699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":7.7590000000000003},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52400000000000002,"15min":0.67400000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.86899999999999999,"5min":1.075,"15min":7.8090000000000002},"last":0.52700000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52900000000000003,"15min":0.67200000000000004},"min":{"1min":0.32800000000000001,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":7.867},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55500000000000005,"15min":0.81999999999999995},"min":{"1min":0.378,"5min":0.35499999999999998,"15min":0.33700000000000002},"max":{"1min":0.70999999999999996,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.70999999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52300000000000002,"15min":0.58099999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.745,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57099999999999995,"15min":0.79100000000000004},"min":{"1min":0.39700000000000002,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":2.641,"15min":6.7629999999999999},"last":0.55600000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55500000000000005,"15min":0.60699999999999998},"min":{"1min":0.372,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94399999999999995,"5min":1.107,"15min":4.8620000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55500000000000005,"15min":0.68400000000000005},"min":{"1min":0.41399999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.72599999999999998,"5min":1.718,"15min":7.9009999999999998},"last":0.67300000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51900000000000002,"15min":0.47899999999999998},"min":{"1min":0.32000000000000001,"5min":0.28999999999999998,"15min":0.28499999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.755},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":0.82899999999999996,"5min":1.032,"15min":1.032},"last":0.77200000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53400000000000003,"15min":0.5},"min":{"1min":0.34999999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.86799999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56999999999999995,"15min":0.53700000000000003},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85399999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.79000000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56200000000000006,"15min":0.53000000000000003},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.89600000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.59399999999999997,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.372,"15min":0.35099999999999998},"max":{"1min":0.82199999999999995,"5min":1.681,"15min":1.681},"last":0.80100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303404,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068436,"statfs":{"total":96636764160,"available":96326078464,"internally_reserved":0,"allocated":2068480,"data_stored":923642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.31900000000000001,"15min":0.443},"min":{"1min":0.215,"5min":0.19600000000000001,"15min":0.17899999999999999},"max":{"1min":0.47799999999999998,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33000000000000002,"15min":0.35599999999999998},"min":{"1min":0.22500000000000001,"5min":0.224,"15min":0.20899999999999999},"max":{"1min":0.441,"5min":0.65500000000000003,"15min":1.194},"last":0.26700000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.32700000000000001,"15min":0.35499999999999998},"min":{"1min":0.246,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.41699999999999998,"5min":0.51400000000000001,"15min":1.2869999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.35199999999999998,"15min":0.47299999999999998},"min":{"1min":0.23999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.49199999999999999,"5min":0.54200000000000004,"15min":5.0919999999999996},"last":0.37}]},{"osd":3,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.374,"15min":0.496},"min":{"1min":0.25600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54800000000000004,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.36299999999999999,"15min":0.39800000000000002},"min":{"1min":0.26500000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.52600000000000002,"5min":0.57799999999999996,"15min":1.2170000000000001},"last":0.54300000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.78600000000000003,"5min":0.92400000000000004,"15min":1.254},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55900000000000005,"15min":0.67500000000000004},"min":{"1min":0.39600000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68100000000000005,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.499}]},{"osd":5,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.40400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.97599999999999998,"15min":1.1799999999999999},"last":0.433},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56999999999999995,"15min":0.59999999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73299999999999998,"5min":0.878,"15min":1.3109999999999999},"last":0.66100000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":0.79300000000000004},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70399999999999996,"5min":0.995,"15min":5.0659999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60699999999999998,"15min":0.628},"min":{"1min":0.46400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84199999999999997,"5min":0.89800000000000002,"15min":1.171},"last":0.51000000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.48299999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77100000000000002,"5min":1.004,"15min":1.214},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58399999999999996,"15min":0.69699999999999995},"min":{"1min":0.40899999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.754,"5min":0.90500000000000003,"15min":4.0499999999999998},"last":0.78200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574952,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303980,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067860,"statfs":{"total":96636764160,"available":96325488640,"internally_reserved":0,"allocated":2658304,"data_stored":1513049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.315,"15min":0.48599999999999999},"min":{"1min":0.217,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":7.9249999999999998},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.46899999999999997},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46100000000000002,"5min":0.53800000000000003,"15min":6.6299999999999999},"last":0.38600000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33900000000000002,"15min":0.51000000000000001},"min":{"1min":0.25600000000000001,"5min":0.22,"15min":0.216},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":8.2509999999999994},"last":0.374},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.49299999999999999},"min":{"1min":0.27500000000000002,"5min":0.223,"15min":0.222},"max":{"1min":0.498,"5min":0.57899999999999996,"15min":6.7370000000000001},"last":0.42199999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.35499999999999998,"15min":0.51700000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.21099999999999999},"max":{"1min":0.438,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.57099999999999995},"min":{"1min":0.317,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":7.8929999999999998},"last":0.46100000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57099999999999995,"15min":0.68700000000000006},"min":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.78100000000000003,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57199999999999995,"15min":0.73099999999999998},"min":{"1min":0.371,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.69599999999999995,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.55400000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56499999999999995,"15min":0.71399999999999997},"min":{"1min":0.40100000000000002,"5min":0.376,"15min":0.36299999999999999},"max":{"1min":0.72699999999999998,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.74199999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.59299999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61499999999999999,"15min":0.77600000000000002},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.73299999999999998},"min":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.36599999999999999},"max":{"1min":0.82099999999999995,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.58199999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60899999999999999,"15min":0.72399999999999998},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.36299999999999999},"max":{"1min":0.85499999999999998,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59799999999999998,"15min":0.747},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89500000000000002,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.53600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303392,"kb_used_data":2008,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068448,"statfs":{"total":96636764160,"available":96326090752,"internally_reserved":0,"allocated":2056192,"data_stored":917379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.311,"15min":0.32400000000000001},"min":{"1min":0.20200000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.48599999999999999,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32100000000000001,"15min":0.34899999999999998},"min":{"1min":0.224,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51800000000000002,"5min":0.54200000000000004,"15min":0.746},"last":0.33000000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33200000000000002,"15min":0.65200000000000002},"min":{"1min":0.22600000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52200000000000002,"5min":0.63,"15min":11.967000000000001},"last":0.31},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.35199999999999998,"15min":0.45800000000000002},"min":{"1min":0.245,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.505,"5min":0.52700000000000002,"15min":4.1100000000000003},"last":0.28799999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.51600000000000001},"min":{"1min":0.23699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":6.3040000000000003},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.38700000000000001,"15min":0.70399999999999996},"min":{"1min":0.29499999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57799999999999996,"5min":0.65900000000000003,"15min":11.935},"last":0.42799999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.42999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91900000000000004,"5min":1.302,"15min":1.302},"last":0.63900000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.378},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.95699999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.39700000000000002,"5min":0.35999999999999999,"15min":0.34300000000000003},"max":{"1min":0.78800000000000003,"5min":0.83999999999999997,"15min":0.96099999999999997},"last":0.69999999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.33400000000000002,"15min":0.33200000000000002},"max":{"1min":0.95499999999999996,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60399999999999998,"15min":0.60499999999999998},"min":{"1min":0.47499999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.57799999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.46100000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85699999999999998,"5min":0.89700000000000002,"15min":1.052},"last":0.629},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58899999999999997,"15min":0.746},"min":{"1min":0.40999999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.98099999999999998,"5min":1.274,"15min":6.2649999999999997},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303404,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068436,"statfs":{"total":96636764160,"available":96326078464,"internally_reserved":0,"allocated":2068480,"data_stored":923642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.309,"15min":0.314},"min":{"1min":0.22600000000000001,"5min":0.21299999999999999,"15min":0.21099999999999999},"max":{"1min":0.38900000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.316,"5min":0.32700000000000001,"15min":0.32100000000000001},"min":{"1min":0.23799999999999999,"5min":0.216,"15min":0.19800000000000001},"max":{"1min":0.42799999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.64100000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.34399999999999997,"15min":0.33400000000000002},"min":{"1min":0.26200000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.47299999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38300000000000001,"15min":0.378},"min":{"1min":0.26800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.48599999999999999,"5min":0.66900000000000004,"15min":0.72099999999999997},"last":0.748}]},{"osd":2,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.36299999999999999,"15min":0.35299999999999998},"min":{"1min":0.28899999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44800000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.28799999999999998,"5min":0.25900000000000001,"15min":0.255},"max":{"1min":0.441,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.51000000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56100000000000005,"15min":0.53800000000000003},"min":{"1min":0.44900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56699999999999995,"15min":0.53900000000000003},"min":{"1min":0.433,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.79400000000000004,"5min":0.83299999999999996,"15min":0.93500000000000005},"last":0.70999999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.55300000000000005},"min":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.35899999999999999},"max":{"1min":0.69599999999999995,"5min":1.347,"15min":1.347},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.443,"5min":0.39300000000000002,"15min":0.35799999999999998},"max":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":1.01},"last":0.72299999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.45500000000000002,"5min":0.41099999999999998,"15min":0.36699999999999999},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.48899999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79700000000000004,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.61799999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61599999999999999,"15min":0.59599999999999997},"min":{"1min":0.495,"5min":0.42099999999999999,"15min":0.38200000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.96999999999999997},"last":0.66400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303392,"kb_used_data":2008,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068448,"statfs":{"total":96636764160,"available":96326090752,"internally_reserved":0,"allocated":2056192,"data_stored":917379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.372,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66700000000000004,"5min":0.74199999999999999,"15min":0.747},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52500000000000002,"15min":0.71899999999999997},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.32700000000000001},"max":{"1min":0.68300000000000005,"5min":0.77600000000000002,"15min":7.6719999999999997},"last":0.51500000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52600000000000002,"15min":0.72399999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.755,"15min":7.7160000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.371,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":0.86799999999999999},"last":0.624}]},{"osd":2,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67800000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.503},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.749},"min":{"1min":0.39900000000000002,"5min":0.39700000000000002,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":7.7839999999999998},"last":0.58199999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.47999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74399999999999999,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.42999999999999999,"5min":0.39200000000000002,"15min":0.35499999999999998},"max":{"1min":0.71499999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59399999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.497},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.30099999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.54700000000000004},"min":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.42699999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.35399999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.77300000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86199999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.52700000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.378,"15min":0.36199999999999999},"max":{"1min":0.88600000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.41699999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.874,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.54900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303404,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068436,"statfs":{"total":96636764160,"available":96326078464,"internally_reserved":0,"allocated":2068480,"data_stored":923642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.47599999999999998,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.66100000000000003,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.51800000000000002},"min":{"1min":0.33000000000000002,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.68899999999999995,"5min":0.76700000000000002,"15min":0.79300000000000004},"last":0.436}]},{"osd":1,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51300000000000001,"15min":0.628},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63400000000000001,"5min":0.75700000000000001,"15min":5.2679999999999998},"last":0.376},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.62},"min":{"1min":0.40500000000000003,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79200000000000004,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.56399999999999995}]},{"osd":2,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52400000000000002,"15min":0.63700000000000001},"min":{"1min":0.39100000000000001,"5min":0.32900000000000001,"15min":0.28299999999999997},"max":{"1min":0.73299999999999998,"5min":0.76600000000000001,"15min":5.2960000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.38500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.5}]},{"osd":3,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55900000000000005,"15min":0.66900000000000004},"min":{"1min":0.41999999999999998,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":5.3330000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56399999999999995,"15min":0.68000000000000005},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.27700000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.57699999999999996}]},{"osd":4,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.48899999999999999,"15min":0.498},"min":{"1min":0.33400000000000002,"5min":0.29599999999999999,"15min":0.28699999999999998},"max":{"1min":0.67000000000000004,"5min":0.72499999999999998,"15min":0.82999999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.34499999999999997,"15min":0.32200000000000001},"max":{"1min":0.77200000000000002,"5min":0.88,"15min":1.0800000000000001},"last":0.59299999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.628},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.39900000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78900000000000003,"5min":0.82499999999999996,"15min":0.89800000000000002},"last":0.65900000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.32300000000000001,"15min":0.30099999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98399999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.67200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:42:57.408 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:42:57.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.553+0000 7ff17da69700 1 Processor -- start 2023-10-10T20:42:57.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.553+0000 7ff17da69700 1 -- start start 2023-10-10T20:42:57.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.554+0000 7ff17da69700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff178159890 0x7ff17815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.554+0000 7ff17da69700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff178159c60 0x7ff17815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.554+0000 7ff17da69700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17815a160 con 0x7ff17815db10 2023-10-10T20:42:57.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.554+0000 7ff17da69700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff17815df50 con 0x7ff178159890 2023-10-10T20:42:57.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.554+0000 7ff17da69700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1780d13a0 con 0x7ff178159c60 2023-10-10T20:42:57.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.554+0000 7ff1777fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff178159890 0x7ff17815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.554+0000 7ff1777fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff178159890 0x7ff17815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55080/0 (socket says 172.21.15.89:55080) 2023-10-10T20:42:57.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.554+0000 7ff1777fe700 1 -- 172.21.15.89:0/575896751 learned_addr learned my addr 172.21.15.89:0/575896751 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:57.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.554+0000 7ff176ffd700 1 --2- 172.21.15.89:0/575896751 >> v2:172.21.15.111:3300/0 conn(0x7ff178159c60 0x7ff17815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.555+0000 7ff1777fe700 1 -- 172.21.15.89:0/575896751 >> v2:172.21.15.111:3300/0 conn(0x7ff178159c60 msgr2=0x7ff17815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:57.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.555+0000 7ff1777fe700 1 --2- 172.21.15.89:0/575896751 >> v2:172.21.15.111:3300/0 conn(0x7ff178159c60 0x7ff17815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.555+0000 7ff1777fe700 1 -- 172.21.15.89:0/575896751 >> v1:172.21.15.111:6789/0 conn(0x7ff17815db10 legacy=0x7ff178069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:57.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.555+0000 7ff1777fe700 1 -- 172.21.15.89:0/575896751 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17815e140 con 0x7ff178159890 2023-10-10T20:42:57.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.555+0000 7ff1777fe700 1 --2- 172.21.15.89:0/575896751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff178159890 0x7ff17815abd0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7ff1780b67f0 tx=0x7ff168009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843c0798f1419c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:57.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.555+0000 7ff1767fc700 1 -- 172.21.15.89:0/575896751 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16800e070 con 0x7ff178159890 2023-10-10T20:42:57.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.555+0000 7ff1767fc700 1 -- 172.21.15.89:0/575896751 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16800ba50 con 0x7ff178159890 2023-10-10T20:42:57.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.555+0000 7ff1767fc700 1 -- 172.21.15.89:0/575896751 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1680092e0 con 0x7ff178159890 2023-10-10T20:42:57.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.555+0000 7ff17da69700 1 -- 172.21.15.89:0/575896751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff178159890 msgr2=0x7ff17815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:57.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.555+0000 7ff17da69700 1 --2- 172.21.15.89:0/575896751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff178159890 0x7ff17815abd0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7ff1780b67f0 tx=0x7ff168009840 comp rx=0 tx=0).stop 2023-10-10T20:42:57.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.555+0000 7ff17da69700 1 -- 172.21.15.89:0/575896751 shutdown_connections 2023-10-10T20:42:57.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.555+0000 7ff17da69700 1 --2- 172.21.15.89:0/575896751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff178159890 0x7ff17815abd0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.555+0000 7ff17da69700 1 --2- 172.21.15.89:0/575896751 >> v2:172.21.15.111:3300/0 conn(0x7ff178159c60 0x7ff17815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.555+0000 7ff17da69700 1 -- 172.21.15.89:0/575896751 >> 172.21.15.89:0/575896751 conn(0x7ff178068e20 msgr2=0x7ff1780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:57.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.555+0000 7ff17da69700 1 -- 172.21.15.89:0/575896751 shutdown_connections 2023-10-10T20:42:57.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.555+0000 7ff17da69700 1 -- 172.21.15.89:0/575896751 wait complete. 2023-10-10T20:42:57.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.557+0000 7ff17da69700 1 Processor -- start 2023-10-10T20:42:57.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.557+0000 7ff17da69700 1 -- start start 2023-10-10T20:42:57.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.557+0000 7ff17da69700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff178159890 0x7ff1781570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.557+0000 7ff17da69700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff178159c60 0x7ff1780b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.557+0000 7ff17da69700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff17815db10 0x7ff1781552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.557+0000 7ff17da69700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1780cdb00 con 0x7ff17815db10 2023-10-10T20:42:57.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.557+0000 7ff17da69700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff178155e90 con 0x7ff178159c60 2023-10-10T20:42:57.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.557+0000 7ff17da69700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1781577b0 con 0x7ff178159890 2023-10-10T20:42:57.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.557+0000 7ff177fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff17815db10 0x7ff1781552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.557+0000 7ff177fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff17815db10 0x7ff1781552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55084/0 (socket says 172.21.15.89:55084) 2023-10-10T20:42:57.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.557+0000 7ff177fff700 1 -- 172.21.15.89:0/1147560734 learned_addr learned my addr 172.21.15.89:0/1147560734 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:57.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.558+0000 7ff1777fe700 1 --2- 172.21.15.89:0/1147560734 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff178159890 0x7ff1781570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.558+0000 7ff177fff700 1 -- 172.21.15.89:0/1147560734 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff178159890 msgr2=0x7ff1781570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:57.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.558+0000 7ff177fff700 1 --2- 172.21.15.89:0/1147560734 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff178159890 0x7ff1781570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.558+0000 7ff176ffd700 1 --2- 172.21.15.89:0/1147560734 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff178159c60 0x7ff1780b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.558+0000 7ff177fff700 1 -- 172.21.15.89:0/1147560734 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff178159c60 msgr2=0x7ff1780b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:57.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.558+0000 7ff177fff700 1 --2- 172.21.15.89:0/1147560734 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff178159c60 0x7ff1780b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.558+0000 7ff177fff700 1 -- 172.21.15.89:0/1147560734 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff168009ae0 con 0x7ff17815db10 2023-10-10T20:42:57.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.558+0000 7ff176ffd700 1 --2- 172.21.15.89:0/1147560734 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff178159c60 0x7ff1780b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:42:57.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.558+0000 7ff177fff700 1 --2- 172.21.15.89:0/1147560734 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff17815db10 0x7ff1781552e0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7ff17000c4c0 tx=0x7ff170007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:57.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.558+0000 7ff15f7fe700 1 -- 172.21.15.89:0/1147560734 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff170011590 con 0x7ff17815db10 2023-10-10T20:42:57.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.558+0000 7ff15f7fe700 1 -- 172.21.15.89:0/1147560734 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff170011710 con 0x7ff17815db10 2023-10-10T20:42:57.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.558+0000 7ff17da69700 1 -- 172.21.15.89:0/1147560734 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1780cd160 con 0x7ff17815db10 2023-10-10T20:42:57.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.558+0000 7ff15f7fe700 1 -- 172.21.15.89:0/1147560734 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff170011ad0 con 0x7ff17815db10 2023-10-10T20:42:57.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.558+0000 7ff17da69700 1 -- 172.21.15.89:0/1147560734 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1780cd720 con 0x7ff17815db10 2023-10-10T20:42:57.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.559+0000 7ff17da69700 1 -- 172.21.15.89:0/1147560734 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1780cdc40 con 0x7ff17815db10 2023-10-10T20:42:57.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.559+0000 7ff15f7fe700 1 -- 172.21.15.89:0/1147560734 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff17001b3b0 con 0x7ff17815db10 2023-10-10T20:42:57.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.560+0000 7ff15f7fe700 1 --2- 172.21.15.89:0/1147560734 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff164067320 0x7ff1640697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.560+0000 7ff15f7fe700 1 -- 172.21.15.89:0/1147560734 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff1700840f0 con 0x7ff17815db10 2023-10-10T20:42:57.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.560+0000 7ff1777fe700 1 --2- 172.21.15.89:0/1147560734 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff164067320 0x7ff1640697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.560+0000 7ff1777fe700 1 --2- 172.21.15.89:0/1147560734 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff164067320 0x7ff1640697d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff168009ab0 tx=0x7ff168018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:57.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.564+0000 7ff15f7fe700 1 -- 172.21.15.89:0/1147560734 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1780cdc40 con 0x7ff17815db10 2023-10-10T20:42:57.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.743+0000 7ff17da69700 1 -- 172.21.15.89:0/1147560734 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff1780cdc40 con 0x7ff164067320 2023-10-10T20:42:57.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.751+0000 7ff15f7fe700 1 -- 172.21.15.89:0/1147560734 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286942 (secure 0 0 0) 0x7ff1780cdc40 con 0x7ff164067320 2023-10-10T20:42:57.756 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:57.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.754+0000 7ff17da69700 1 -- 172.21.15.89:0/1147560734 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff164067320 msgr2=0x7ff1640697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:57.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.754+0000 7ff17da69700 1 --2- 172.21.15.89:0/1147560734 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff164067320 0x7ff1640697d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff168009ab0 tx=0x7ff168018000 comp rx=0 tx=0).stop 2023-10-10T20:42:57.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.754+0000 7ff17da69700 1 -- 172.21.15.89:0/1147560734 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff17815db10 msgr2=0x7ff1781552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:57.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.754+0000 7ff17da69700 1 --2- 172.21.15.89:0/1147560734 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff17815db10 0x7ff1781552e0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7ff17000c4c0 tx=0x7ff170007f50 comp rx=0 tx=0).stop 2023-10-10T20:42:57.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.754+0000 7ff17da69700 1 -- 172.21.15.89:0/1147560734 shutdown_connections 2023-10-10T20:42:57.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.754+0000 7ff17da69700 1 --2- 172.21.15.89:0/1147560734 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff17815db10 0x7ff1781552e0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.754+0000 7ff17da69700 1 --2- 172.21.15.89:0/1147560734 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff178159890 0x7ff1781570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.754+0000 7ff17da69700 1 --2- 172.21.15.89:0/1147560734 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff164067320 0x7ff1640697d0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.754+0000 7ff17da69700 1 --2- 172.21.15.89:0/1147560734 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff178159c60 0x7ff1780b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.754+0000 7ff17da69700 1 -- 172.21.15.89:0/1147560734 >> 172.21.15.89:0/1147560734 conn(0x7ff178068e20 msgr2=0x7ff178158590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:57.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.755+0000 7ff17da69700 1 -- 172.21.15.89:0/1147560734 shutdown_connections 2023-10-10T20:42:57.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.755+0000 7ff17da69700 1 -- 172.21.15.89:0/1147560734 wait complete. 2023-10-10T20:42:57.761 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:42:57.769 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2023-10-10T20:42:56.751262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428364,"kb_used_data":17292,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752546356,"statfs":{"total":773094113280,"available":770607468544,"internally_reserved":0,"allocated":17707008,"data_stored":8556385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.433293"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.323231+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:40.796551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.325806+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:28:12.284338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.331625+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:09:34.459455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.330945+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:16:50.053827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329837+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:16:07.451863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.327658+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:58:12.432427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.326884+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:36:30.295360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322678+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:59.692515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329758+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:02:55.597436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.327137+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:32:08.240124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.323857+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:32.579274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.325054+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:36:30.295360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.324653+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:03.524684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329133+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:41:28.090635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329269+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:38:16.787078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.324787+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:58:32.593330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328589+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:35.674118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.326650+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:38:30.259757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.334129+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:20:38.627889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.327487+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:27.171367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329421+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:18:18.498552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.330301+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:00:48.564823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.323234+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:58:00.782955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328346+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:02:55.597436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322747+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:25:50.618140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322798+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:04:51.509487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.323414+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:20:34.093326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.326676+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:33:29.535005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328679+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:50:07.909344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322817+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:44:40.439820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.330007+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:02.852226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329533+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:13:20.878853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328750+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:46.022458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.335968+0000","last_change":"2023-10-10T20:42:52.335968+0000","last_active":"2023-10-10T20:42:52.335968+0000","last_peered":"2023-10-10T20:42:52.335968+0000","last_clean":"2023-10-10T20:42:52.335968+0000","last_became_active":"2023-10-10T20:42:52.335695+0000","last_became_peered":"2023-10-10T20:42:52.335695+0000","last_unstale":"2023-10-10T20:42:52.335968+0000","last_undegraded":"2023-10-10T20:42:52.335968+0000","last_fullsized":"2023-10-10T20:42:52.335968+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:32:07.233388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322742+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:39:23.346373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.336051+0000","last_change":"2023-10-10T20:42:52.336051+0000","last_active":"2023-10-10T20:42:52.336051+0000","last_peered":"2023-10-10T20:42:52.336051+0000","last_clean":"2023-10-10T20:42:52.336051+0000","last_became_active":"2023-10-10T20:42:52.335752+0000","last_became_peered":"2023-10-10T20:42:52.335752+0000","last_unstale":"2023-10-10T20:42:52.336051+0000","last_undegraded":"2023-10-10T20:42:52.336051+0000","last_fullsized":"2023-10-10T20:42:52.336051+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:15:01.473071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329028+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:00:56.184401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.339512+0000","last_change":"2023-10-10T20:42:52.339512+0000","last_active":"2023-10-10T20:42:52.339512+0000","last_peered":"2023-10-10T20:42:52.339512+0000","last_clean":"2023-10-10T20:42:52.339512+0000","last_became_active":"2023-10-10T20:42:52.339230+0000","last_became_peered":"2023-10-10T20:42:52.339230+0000","last_unstale":"2023-10-10T20:42:52.339512+0000","last_undegraded":"2023-10-10T20:42:52.339512+0000","last_fullsized":"2023-10-10T20:42:52.339512+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:51.664988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322141+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:45:05.557928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.332410+0000","last_change":"2023-10-10T20:42:52.332410+0000","last_active":"2023-10-10T20:42:52.332410+0000","last_peered":"2023-10-10T20:42:52.332410+0000","last_clean":"2023-10-10T20:42:52.332410+0000","last_became_active":"2023-10-10T20:42:52.331952+0000","last_became_peered":"2023-10-10T20:42:52.331952+0000","last_unstale":"2023-10-10T20:42:52.332410+0000","last_undegraded":"2023-10-10T20:42:52.332410+0000","last_fullsized":"2023-10-10T20:42:52.332410+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:46:55.907438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328114+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:00.861926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.338841+0000","last_change":"2023-10-10T20:42:52.338841+0000","last_active":"2023-10-10T20:42:52.338841+0000","last_peered":"2023-10-10T20:42:52.338841+0000","last_clean":"2023-10-10T20:42:52.338841+0000","last_became_active":"2023-10-10T20:42:52.337405+0000","last_became_peered":"2023-10-10T20:42:52.337405+0000","last_unstale":"2023-10-10T20:42:52.338841+0000","last_undegraded":"2023-10-10T20:42:52.338841+0000","last_fullsized":"2023-10-10T20:42:52.338841+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:10:49.794184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.327089+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:52.463061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.333926+0000","last_change":"2023-10-10T20:42:52.333926+0000","last_active":"2023-10-10T20:42:52.333926+0000","last_peered":"2023-10-10T20:42:52.333926+0000","last_clean":"2023-10-10T20:42:52.333926+0000","last_became_active":"2023-10-10T20:42:52.333645+0000","last_became_peered":"2023-10-10T20:42:52.333645+0000","last_unstale":"2023-10-10T20:42:52.333926+0000","last_undegraded":"2023-10-10T20:42:52.333926+0000","last_fullsized":"2023-10-10T20:42:52.333926+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:46:47.379359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.324299+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:17:23.703124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.331824+0000","last_change":"2023-10-10T20:42:52.331824+0000","last_active":"2023-10-10T20:42:52.331824+0000","last_peered":"2023-10-10T20:42:52.331824+0000","last_clean":"2023-10-10T20:42:52.331824+0000","last_became_active":"2023-10-10T20:42:52.331567+0000","last_became_peered":"2023-10-10T20:42:52.331567+0000","last_unstale":"2023-10-10T20:42:52.331824+0000","last_undegraded":"2023-10-10T20:42:52.331824+0000","last_fullsized":"2023-10-10T20:42:52.331824+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:59.776219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.325326+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:30:49.951335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.329973+0000","last_change":"2023-10-10T20:42:52.329973+0000","last_active":"2023-10-10T20:42:52.329973+0000","last_peered":"2023-10-10T20:42:52.329973+0000","last_clean":"2023-10-10T20:42:52.329973+0000","last_became_active":"2023-10-10T20:42:52.329001+0000","last_became_peered":"2023-10-10T20:42:52.329001+0000","last_unstale":"2023-10-10T20:42:52.329973+0000","last_undegraded":"2023-10-10T20:42:52.329973+0000","last_fullsized":"2023-10-10T20:42:52.329973+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:20:37.621153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322179+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:13:45.883755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.334832+0000","last_change":"2023-10-10T20:42:52.334832+0000","last_active":"2023-10-10T20:42:52.334832+0000","last_peered":"2023-10-10T20:42:52.334832+0000","last_clean":"2023-10-10T20:42:52.334832+0000","last_became_active":"2023-10-10T20:42:52.334555+0000","last_became_peered":"2023-10-10T20:42:52.334555+0000","last_unstale":"2023-10-10T20:42:52.334832+0000","last_undegraded":"2023-10-10T20:42:52.334832+0000","last_fullsized":"2023-10-10T20:42:52.334832+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:51.456325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328250+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:31.176991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.341985+0000","last_change":"2023-10-10T20:42:52.341985+0000","last_active":"2023-10-10T20:42:52.341985+0000","last_peered":"2023-10-10T20:42:52.341985+0000","last_clean":"2023-10-10T20:42:52.341985+0000","last_became_active":"2023-10-10T20:42:52.341730+0000","last_became_peered":"2023-10-10T20:42:52.341730+0000","last_unstale":"2023-10-10T20:42:52.341985+0000","last_undegraded":"2023-10-10T20:42:52.341985+0000","last_fullsized":"2023-10-10T20:42:52.341985+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:32:48.988699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322090+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:17:25.969866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.333830+0000","last_change":"2023-10-10T20:42:52.333830+0000","last_active":"2023-10-10T20:42:52.333830+0000","last_peered":"2023-10-10T20:42:52.333830+0000","last_clean":"2023-10-10T20:42:52.333830+0000","last_became_active":"2023-10-10T20:42:52.333570+0000","last_became_peered":"2023-10-10T20:42:52.333570+0000","last_unstale":"2023-10-10T20:42:52.333830+0000","last_undegraded":"2023-10-10T20:42:52.333830+0000","last_fullsized":"2023-10-10T20:42:52.333830+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:31.572538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329839+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:18:15.714890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.339386+0000","last_change":"2023-10-10T20:42:52.339386+0000","last_active":"2023-10-10T20:42:52.339386+0000","last_peered":"2023-10-10T20:42:52.339386+0000","last_clean":"2023-10-10T20:42:52.339386+0000","last_became_active":"2023-10-10T20:42:52.339072+0000","last_became_peered":"2023-10-10T20:42:52.339072+0000","last_unstale":"2023-10-10T20:42:52.339386+0000","last_undegraded":"2023-10-10T20:42:52.339386+0000","last_fullsized":"2023-10-10T20:42:52.339386+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:53:19.603938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.333958+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:20:38.627889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.337702+0000","last_change":"2023-10-10T20:42:52.337702+0000","last_active":"2023-10-10T20:42:52.337702+0000","last_peered":"2023-10-10T20:42:52.337702+0000","last_clean":"2023-10-10T20:42:52.337702+0000","last_became_active":"2023-10-10T20:42:52.337415+0000","last_became_peered":"2023-10-10T20:42:52.337415+0000","last_unstale":"2023-10-10T20:42:52.337702+0000","last_undegraded":"2023-10-10T20:42:52.337702+0000","last_fullsized":"2023-10-10T20:42:52.337702+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:33:28.528269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.331014+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:05:24.092738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.338838+0000","last_change":"2023-10-10T20:42:52.338838+0000","last_active":"2023-10-10T20:42:52.338838+0000","last_peered":"2023-10-10T20:42:52.338838+0000","last_clean":"2023-10-10T20:42:52.338838+0000","last_became_active":"2023-10-10T20:42:52.338592+0000","last_became_peered":"2023-10-10T20:42:52.338592+0000","last_unstale":"2023-10-10T20:42:52.338838+0000","last_undegraded":"2023-10-10T20:42:52.338838+0000","last_fullsized":"2023-10-10T20:42:52.338838+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:36:29.288624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328957+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:46:36.668645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.334750+0000","last_change":"2023-10-10T20:42:52.334750+0000","last_active":"2023-10-10T20:42:52.334750+0000","last_peered":"2023-10-10T20:42:52.334750+0000","last_clean":"2023-10-10T20:42:52.334750+0000","last_became_active":"2023-10-10T20:42:52.334260+0000","last_became_peered":"2023-10-10T20:42:52.334260+0000","last_unstale":"2023-10-10T20:42:52.334750+0000","last_undegraded":"2023-10-10T20:42:52.334750+0000","last_fullsized":"2023-10-10T20:42:52.334750+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:49:39.517807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322365+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:04:02.591369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.339183+0000","last_change":"2023-10-10T20:42:52.339183+0000","last_active":"2023-10-10T20:42:52.339183+0000","last_peered":"2023-10-10T20:42:52.339183+0000","last_clean":"2023-10-10T20:42:52.339183+0000","last_became_active":"2023-10-10T20:42:52.337722+0000","last_became_peered":"2023-10-10T20:42:52.337722+0000","last_unstale":"2023-10-10T20:42:52.339183+0000","last_undegraded":"2023-10-10T20:42:52.339183+0000","last_fullsized":"2023-10-10T20:42:52.339183+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:51.664988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.326616+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:57:12.184703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.335410+0000","last_change":"2023-10-10T20:42:52.335410+0000","last_active":"2023-10-10T20:42:52.335410+0000","last_peered":"2023-10-10T20:42:52.335410+0000","last_clean":"2023-10-10T20:42:52.335410+0000","last_became_active":"2023-10-10T20:42:52.335097+0000","last_became_peered":"2023-10-10T20:42:52.335097+0000","last_unstale":"2023-10-10T20:42:52.335410+0000","last_undegraded":"2023-10-10T20:42:52.335410+0000","last_fullsized":"2023-10-10T20:42:52.335410+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:22:17.318577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.327959+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:58:32.593330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.332490+0000","last_change":"2023-10-10T20:42:52.332490+0000","last_active":"2023-10-10T20:42:52.332490+0000","last_peered":"2023-10-10T20:42:52.332490+0000","last_clean":"2023-10-10T20:42:52.332490+0000","last_became_active":"2023-10-10T20:42:52.332045+0000","last_became_peered":"2023-10-10T20:42:52.332045+0000","last_unstale":"2023-10-10T20:42:52.332490+0000","last_undegraded":"2023-10-10T20:42:52.332490+0000","last_fullsized":"2023-10-10T20:42:52.332490+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:01.845490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.331151+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:18:18.498552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.333544+0000","last_change":"2023-10-10T20:42:52.333544+0000","last_active":"2023-10-10T20:42:52.333544+0000","last_peered":"2023-10-10T20:42:52.333544+0000","last_clean":"2023-10-10T20:42:52.333544+0000","last_became_active":"2023-10-10T20:42:52.333211+0000","last_became_peered":"2023-10-10T20:42:52.333211+0000","last_unstale":"2023-10-10T20:42:52.333544+0000","last_undegraded":"2023-10-10T20:42:52.333544+0000","last_fullsized":"2023-10-10T20:42:52.333544+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:39.789815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328162+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:46.022458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.336202+0000","last_change":"2023-10-10T20:42:52.336202+0000","last_active":"2023-10-10T20:42:52.336202+0000","last_peered":"2023-10-10T20:42:52.336202+0000","last_clean":"2023-10-10T20:42:52.336202+0000","last_became_active":"2023-10-10T20:42:52.335926+0000","last_became_peered":"2023-10-10T20:42:52.335926+0000","last_unstale":"2023-10-10T20:42:52.336202+0000","last_undegraded":"2023-10-10T20:42:52.336202+0000","last_fullsized":"2023-10-10T20:42:52.336202+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:49:39.517807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.333557+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:16:50.053827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.335457+0000","last_change":"2023-10-10T20:42:52.335457+0000","last_active":"2023-10-10T20:42:52.335457+0000","last_peered":"2023-10-10T20:42:52.335457+0000","last_clean":"2023-10-10T20:42:52.335457+0000","last_became_active":"2023-10-10T20:42:52.335186+0000","last_became_peered":"2023-10-10T20:42:52.335186+0000","last_unstale":"2023-10-10T20:42:52.335457+0000","last_undegraded":"2023-10-10T20:42:52.335457+0000","last_fullsized":"2023-10-10T20:42:52.335457+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:55:21.874802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322254+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:21:17.789832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.337248+0000","last_change":"2023-10-10T20:42:52.337248+0000","last_active":"2023-10-10T20:42:52.337248+0000","last_peered":"2023-10-10T20:42:52.337248+0000","last_clean":"2023-10-10T20:42:52.337248+0000","last_became_active":"2023-10-10T20:42:52.336966+0000","last_became_peered":"2023-10-10T20:42:52.336966+0000","last_unstale":"2023-10-10T20:42:52.337248+0000","last_undegraded":"2023-10-10T20:42:52.337248+0000","last_fullsized":"2023-10-10T20:42:52.337248+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:59:14.539048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329553+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:09:34.459455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.350531+0000","last_change":"2023-10-10T20:42:52.350531+0000","last_active":"2023-10-10T20:42:52.350531+0000","last_peered":"2023-10-10T20:42:52.350531+0000","last_clean":"2023-10-10T20:42:52.350531+0000","last_became_active":"2023-10-10T20:42:52.350315+0000","last_became_peered":"2023-10-10T20:42:52.350315+0000","last_unstale":"2023-10-10T20:42:52.350531+0000","last_undegraded":"2023-10-10T20:42:52.350531+0000","last_fullsized":"2023-10-10T20:42:52.350531+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:08.850614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.330288+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:28:12.284338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.338420+0000","last_change":"2023-10-10T20:42:52.338420+0000","last_active":"2023-10-10T20:42:52.338420+0000","last_peered":"2023-10-10T20:42:52.338420+0000","last_clean":"2023-10-10T20:42:52.338420+0000","last_became_active":"2023-10-10T20:42:52.338190+0000","last_became_peered":"2023-10-10T20:42:52.338190+0000","last_unstale":"2023-10-10T20:42:52.338420+0000","last_undegraded":"2023-10-10T20:42:52.338420+0000","last_fullsized":"2023-10-10T20:42:52.338420+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:58:31.586594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.327181+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:30:49.951335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.335469+0000","last_change":"2023-10-10T20:42:52.335469+0000","last_active":"2023-10-10T20:42:52.335469+0000","last_peered":"2023-10-10T20:42:52.335469+0000","last_clean":"2023-10-10T20:42:52.335469+0000","last_became_active":"2023-10-10T20:42:52.333014+0000","last_became_peered":"2023-10-10T20:42:52.333014+0000","last_unstale":"2023-10-10T20:42:52.335469+0000","last_undegraded":"2023-10-10T20:42:52.335469+0000","last_fullsized":"2023-10-10T20:42:52.335469+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:16:49.047090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328435+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:00.861926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.334206+0000","last_change":"2023-10-10T20:42:52.334206+0000","last_active":"2023-10-10T20:42:52.334206+0000","last_peered":"2023-10-10T20:42:52.334206+0000","last_clean":"2023-10-10T20:42:52.334206+0000","last_became_active":"2023-10-10T20:42:52.333022+0000","last_became_peered":"2023-10-10T20:42:52.333022+0000","last_unstale":"2023-10-10T20:42:52.334206+0000","last_undegraded":"2023-10-10T20:42:52.334206+0000","last_fullsized":"2023-10-10T20:42:52.334206+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:34:10.314535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328547+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:05:24.092738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.339808+0000","last_change":"2023-10-10T20:42:52.339808+0000","last_active":"2023-10-10T20:42:52.339808+0000","last_peered":"2023-10-10T20:42:52.339808+0000","last_clean":"2023-10-10T20:42:52.339808+0000","last_became_active":"2023-10-10T20:42:52.339538+0000","last_became_peered":"2023-10-10T20:42:52.339538+0000","last_unstale":"2023-10-10T20:42:52.339808+0000","last_undegraded":"2023-10-10T20:42:52.339808+0000","last_fullsized":"2023-10-10T20:42:52.339808+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:17:22.696388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.324346+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:32:49.995435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.330898+0000","last_change":"2023-10-10T20:42:52.330898+0000","last_active":"2023-10-10T20:42:52.330898+0000","last_peered":"2023-10-10T20:42:52.330898+0000","last_clean":"2023-10-10T20:42:52.330898+0000","last_became_active":"2023-10-10T20:42:52.330616+0000","last_became_peered":"2023-10-10T20:42:52.330616+0000","last_unstale":"2023-10-10T20:42:52.330898+0000","last_undegraded":"2023-10-10T20:42:52.330898+0000","last_fullsized":"2023-10-10T20:42:52.330898+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:30.999244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.333078+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:02.852226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.333625+0000","last_change":"2023-10-10T20:42:52.333625+0000","last_active":"2023-10-10T20:42:52.333625+0000","last_peered":"2023-10-10T20:42:52.333625+0000","last_clean":"2023-10-10T20:42:52.333625+0000","last_became_active":"2023-10-10T20:42:52.333359+0000","last_became_peered":"2023-10-10T20:42:52.333359+0000","last_unstale":"2023-10-10T20:42:52.333625+0000","last_undegraded":"2023-10-10T20:42:52.333625+0000","last_fullsized":"2023-10-10T20:42:52.333625+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:31.572538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328569+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:28:12.284338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.339859+0000","last_change":"2023-10-10T20:42:52.339859+0000","last_active":"2023-10-10T20:42:52.339859+0000","last_peered":"2023-10-10T20:42:52.339859+0000","last_clean":"2023-10-10T20:42:52.339859+0000","last_became_active":"2023-10-10T20:42:52.339585+0000","last_became_peered":"2023-10-10T20:42:52.339585+0000","last_unstale":"2023-10-10T20:42:52.339859+0000","last_undegraded":"2023-10-10T20:42:52.339859+0000","last_fullsized":"2023-10-10T20:42:52.339859+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:02.517948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322090+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:38:10.362086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.335154+0000","last_change":"2023-10-10T20:42:52.335154+0000","last_active":"2023-10-10T20:42:52.335154+0000","last_peered":"2023-10-10T20:42:52.335154+0000","last_clean":"2023-10-10T20:42:52.335154+0000","last_became_active":"2023-10-10T20:42:52.334853+0000","last_became_peered":"2023-10-10T20:42:52.334853+0000","last_unstale":"2023-10-10T20:42:52.335154+0000","last_undegraded":"2023-10-10T20:42:52.335154+0000","last_fullsized":"2023-10-10T20:42:52.335154+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:15:01.473071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322090+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:35.275237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.338269+0000","last_change":"2023-10-10T20:42:52.338269+0000","last_active":"2023-10-10T20:42:52.338269+0000","last_peered":"2023-10-10T20:42:52.338269+0000","last_clean":"2023-10-10T20:42:52.338269+0000","last_became_active":"2023-10-10T20:42:52.338028+0000","last_became_peered":"2023-10-10T20:42:52.338028+0000","last_unstale":"2023-10-10T20:42:52.338269+0000","last_undegraded":"2023-10-10T20:42:52.338269+0000","last_fullsized":"2023-10-10T20:42:52.338269+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:17:34.621354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":306,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-10T20:42:51.319419+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:42:51.319419+0000","last_peered":"2023-10-10T20:42:51.319419+0000","last_clean":"2023-10-10T20:42:51.319419+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:42:51.319419+0000","last_undegraded":"2023-10-10T20:42:51.319419+0000","last_fullsized":"2023-10-10T20:42:51.319419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574953,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303984,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067856,"statfs":{"total":96636764160,"available":96325484544,"internally_reserved":0,"allocated":2662400,"data_stored":1514010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50700000000000001,"15min":0.75800000000000001},"min":{"1min":0.32500000000000001,"5min":0.309,"15min":0.29199999999999998},"max":{"1min":0.64300000000000002,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.50800000000000001,"15min":0.60399999999999998},"min":{"1min":0.36399999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.58299999999999996,"5min":0.752,"15min":4.1959999999999997},"last":0.67000000000000004}]},{"osd":1,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.50600000000000001,"15min":0.90100000000000002},"min":{"1min":0.372,"5min":0.317,"15min":0.317},"max":{"1min":0.61099999999999999,"5min":0.78800000000000003,"15min":10.1},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.53400000000000003,"15min":0.69099999999999995},"min":{"1min":0.35199999999999998,"5min":0.33900000000000002,"15min":0.28999999999999998},"max":{"1min":0.66500000000000004,"5min":0.81799999999999995,"15min":6.806},"last":0.63600000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56799999999999995,"15min":0.74099999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68799999999999994,"5min":0.89000000000000001,"15min":6.9340000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.69399999999999995},"min":{"1min":0.40699999999999997,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.69099999999999995,"5min":0.77100000000000002,"15min":6.9870000000000001},"last":0.747}]},{"osd":3,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54200000000000004,"15min":0.71299999999999997},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64500000000000002,"5min":0.754,"15min":6.6719999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55900000000000005,"15min":0.64800000000000002},"min":{"1min":0.39800000000000002,"5min":0.378,"15min":0.34799999999999998},"max":{"1min":0.64400000000000002,"5min":0.76200000000000001,"15min":4.1609999999999996},"last":0.71899999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47799999999999998,"15min":0.46800000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.77200000000000002},"last":0.5},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.51600000000000001,"15min":0.51900000000000002},"min":{"1min":0.314,"5min":0.309,"15min":0.30399999999999999},"max":{"1min":0.68000000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.51400000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.54000000000000004,"15min":0.53900000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.68400000000000005,"5min":0.86099999999999999,"15min":1.3180000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.53600000000000003,"15min":0.52700000000000002},"min":{"1min":0.35299999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.84099999999999997},"last":0.54600000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.35699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":1.7509999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76300000000000001,"5min":0.84099999999999997,"15min":1.7090000000000001},"last":0.65200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574954,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303404,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068436,"statfs":{"total":96636764160,"available":96326078464,"internally_reserved":0,"allocated":2068480,"data_stored":923642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52400000000000002,"15min":0.64000000000000001},"min":{"1min":0.34699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":7.7590000000000003},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52400000000000002,"15min":0.67400000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.86899999999999999,"5min":1.075,"15min":7.8090000000000002},"last":0.52700000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52900000000000003,"15min":0.67200000000000004},"min":{"1min":0.32800000000000001,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":7.867},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55500000000000005,"15min":0.81999999999999995},"min":{"1min":0.378,"5min":0.35499999999999998,"15min":0.33700000000000002},"max":{"1min":0.70999999999999996,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.70999999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52300000000000002,"15min":0.58099999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.745,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57099999999999995,"15min":0.79100000000000004},"min":{"1min":0.39700000000000002,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":2.641,"15min":6.7629999999999999},"last":0.55600000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55500000000000005,"15min":0.60699999999999998},"min":{"1min":0.372,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94399999999999995,"5min":1.107,"15min":4.8620000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55500000000000005,"15min":0.68400000000000005},"min":{"1min":0.41399999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.72599999999999998,"5min":1.718,"15min":7.9009999999999998},"last":0.67300000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51900000000000002,"15min":0.47899999999999998},"min":{"1min":0.32000000000000001,"5min":0.28999999999999998,"15min":0.28499999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.755},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":0.82899999999999996,"5min":1.032,"15min":1.032},"last":0.77200000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53400000000000003,"15min":0.5},"min":{"1min":0.34999999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.86799999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56999999999999995,"15min":0.53700000000000003},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85399999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.79000000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56200000000000006,"15min":0.53000000000000003},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.89600000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.59399999999999997,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.372,"15min":0.35099999999999998},"max":{"1min":0.82199999999999995,"5min":1.681,"15min":1.681},"last":0.80100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303404,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068436,"statfs":{"total":96636764160,"available":96326078464,"internally_reserved":0,"allocated":2068480,"data_stored":923642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.31900000000000001,"15min":0.443},"min":{"1min":0.215,"5min":0.19600000000000001,"15min":0.17899999999999999},"max":{"1min":0.47799999999999998,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33000000000000002,"15min":0.35599999999999998},"min":{"1min":0.22500000000000001,"5min":0.224,"15min":0.20899999999999999},"max":{"1min":0.441,"5min":0.65500000000000003,"15min":1.194},"last":0.26700000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.32700000000000001,"15min":0.35499999999999998},"min":{"1min":0.246,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.41699999999999998,"5min":0.51400000000000001,"15min":1.2869999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.35199999999999998,"15min":0.47299999999999998},"min":{"1min":0.23999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.49199999999999999,"5min":0.54200000000000004,"15min":5.0919999999999996},"last":0.37}]},{"osd":3,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.374,"15min":0.496},"min":{"1min":0.25600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54800000000000004,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.36299999999999999,"15min":0.39800000000000002},"min":{"1min":0.26500000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.52600000000000002,"5min":0.57799999999999996,"15min":1.2170000000000001},"last":0.54300000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.78600000000000003,"5min":0.92400000000000004,"15min":1.254},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55900000000000005,"15min":0.67500000000000004},"min":{"1min":0.39600000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68100000000000005,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.499}]},{"osd":5,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.40400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.97599999999999998,"15min":1.1799999999999999},"last":0.433},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56999999999999995,"15min":0.59999999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73299999999999998,"5min":0.878,"15min":1.3109999999999999},"last":0.66100000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":0.79300000000000004},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70399999999999996,"5min":0.995,"15min":5.0659999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60699999999999998,"15min":0.628},"min":{"1min":0.46400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84199999999999997,"5min":0.89800000000000002,"15min":1.171},"last":0.51000000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.48299999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77100000000000002,"5min":1.004,"15min":1.214},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58399999999999996,"15min":0.69699999999999995},"min":{"1min":0.40899999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.754,"5min":0.90500000000000003,"15min":4.0499999999999998},"last":0.78200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574952,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303980,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067860,"statfs":{"total":96636764160,"available":96325488640,"internally_reserved":0,"allocated":2658304,"data_stored":1513049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.315,"15min":0.48599999999999999},"min":{"1min":0.217,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":7.9249999999999998},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.46899999999999997},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46100000000000002,"5min":0.53800000000000003,"15min":6.6299999999999999},"last":0.38600000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33900000000000002,"15min":0.51000000000000001},"min":{"1min":0.25600000000000001,"5min":0.22,"15min":0.216},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":8.2509999999999994},"last":0.374},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.49299999999999999},"min":{"1min":0.27500000000000002,"5min":0.223,"15min":0.222},"max":{"1min":0.498,"5min":0.57899999999999996,"15min":6.7370000000000001},"last":0.42199999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.35499999999999998,"15min":0.51700000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.21099999999999999},"max":{"1min":0.438,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.57099999999999995},"min":{"1min":0.317,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":7.8929999999999998},"last":0.46100000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57099999999999995,"15min":0.68700000000000006},"min":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.78100000000000003,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57199999999999995,"15min":0.73099999999999998},"min":{"1min":0.371,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.69599999999999995,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.55400000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56499999999999995,"15min":0.71399999999999997},"min":{"1min":0.40100000000000002,"5min":0.376,"15min":0.36299999999999999},"max":{"1min":0.72699999999999998,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.74199999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.59299999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61499999999999999,"15min":0.77600000000000002},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.73299999999999998},"min":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.36599999999999999},"max":{"1min":0.82099999999999995,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.58199999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60899999999999999,"15min":0.72399999999999998},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.36299999999999999},"max":{"1min":0.85499999999999998,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59799999999999998,"15min":0.747},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89500000000000002,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.53600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303392,"kb_used_data":2008,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068448,"statfs":{"total":96636764160,"available":96326090752,"internally_reserved":0,"allocated":2056192,"data_stored":917379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.311,"15min":0.32400000000000001},"min":{"1min":0.20200000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.48599999999999999,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32100000000000001,"15min":0.34899999999999998},"min":{"1min":0.224,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51800000000000002,"5min":0.54200000000000004,"15min":0.746},"last":0.33000000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33200000000000002,"15min":0.65200000000000002},"min":{"1min":0.22600000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52200000000000002,"5min":0.63,"15min":11.967000000000001},"last":0.31},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.35199999999999998,"15min":0.45800000000000002},"min":{"1min":0.245,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.505,"5min":0.52700000000000002,"15min":4.1100000000000003},"last":0.28799999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.51600000000000001},"min":{"1min":0.23699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":6.3040000000000003},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.38700000000000001,"15min":0.70399999999999996},"min":{"1min":0.29499999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57799999999999996,"5min":0.65900000000000003,"15min":11.935},"last":0.42799999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.42999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91900000000000004,"5min":1.302,"15min":1.302},"last":0.63900000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.378},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.95699999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.39700000000000002,"5min":0.35999999999999999,"15min":0.34300000000000003},"max":{"1min":0.78800000000000003,"5min":0.83999999999999997,"15min":0.96099999999999997},"last":0.69999999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.33400000000000002,"15min":0.33200000000000002},"max":{"1min":0.95499999999999996,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60399999999999998,"15min":0.60499999999999998},"min":{"1min":0.47499999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.57799999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.46100000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85699999999999998,"5min":0.89700000000000002,"15min":1.052},"last":0.629},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58899999999999997,"15min":0.746},"min":{"1min":0.40999999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.98099999999999998,"5min":1.274,"15min":6.2649999999999997},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303404,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068436,"statfs":{"total":96636764160,"available":96326078464,"internally_reserved":0,"allocated":2068480,"data_stored":923642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.309,"15min":0.314},"min":{"1min":0.22600000000000001,"5min":0.21299999999999999,"15min":0.21099999999999999},"max":{"1min":0.38900000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.316,"5min":0.32700000000000001,"15min":0.32100000000000001},"min":{"1min":0.23799999999999999,"5min":0.216,"15min":0.19800000000000001},"max":{"1min":0.42799999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.64100000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.34399999999999997,"15min":0.33400000000000002},"min":{"1min":0.26200000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.47299999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38300000000000001,"15min":0.378},"min":{"1min":0.26800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.48599999999999999,"5min":0.66900000000000004,"15min":0.72099999999999997},"last":0.748}]},{"osd":2,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.36299999999999999,"15min":0.35299999999999998},"min":{"1min":0.28899999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44800000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.28799999999999998,"5min":0.25900000000000001,"15min":0.255},"max":{"1min":0.441,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.51000000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56100000000000005,"15min":0.53800000000000003},"min":{"1min":0.44900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56699999999999995,"15min":0.53900000000000003},"min":{"1min":0.433,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.79400000000000004,"5min":0.83299999999999996,"15min":0.93500000000000005},"last":0.70999999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.55300000000000005},"min":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.35899999999999999},"max":{"1min":0.69599999999999995,"5min":1.347,"15min":1.347},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.443,"5min":0.39300000000000002,"15min":0.35799999999999998},"max":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":1.01},"last":0.72299999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.45500000000000002,"5min":0.41099999999999998,"15min":0.36699999999999999},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.48899999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79700000000000004,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.61799999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61599999999999999,"15min":0.59599999999999997},"min":{"1min":0.495,"5min":0.42099999999999999,"15min":0.38200000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.96999999999999997},"last":0.66400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303392,"kb_used_data":2008,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068448,"statfs":{"total":96636764160,"available":96326090752,"internally_reserved":0,"allocated":2056192,"data_stored":917379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.372,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66700000000000004,"5min":0.74199999999999999,"15min":0.747},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52500000000000002,"15min":0.71899999999999997},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.32700000000000001},"max":{"1min":0.68300000000000005,"5min":0.77600000000000002,"15min":7.6719999999999997},"last":0.51500000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52600000000000002,"15min":0.72399999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.755,"15min":7.7160000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.371,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":0.86799999999999999},"last":0.624}]},{"osd":2,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67800000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.503},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.749},"min":{"1min":0.39900000000000002,"5min":0.39700000000000002,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":7.7839999999999998},"last":0.58199999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.47999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74399999999999999,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.42999999999999999,"5min":0.39200000000000002,"15min":0.35499999999999998},"max":{"1min":0.71499999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59399999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.497},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.30099999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.54700000000000004},"min":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.42699999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.35399999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.77300000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86199999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.52700000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.378,"15min":0.36199999999999999},"max":{"1min":0.88600000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.41699999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.874,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.54900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303404,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068436,"statfs":{"total":96636764160,"available":96326078464,"internally_reserved":0,"allocated":2068480,"data_stored":923642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.47599999999999998,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.66100000000000003,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.51800000000000002},"min":{"1min":0.33000000000000002,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.68899999999999995,"5min":0.76700000000000002,"15min":0.79300000000000004},"last":0.436}]},{"osd":1,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51300000000000001,"15min":0.628},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63400000000000001,"5min":0.75700000000000001,"15min":5.2679999999999998},"last":0.376},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.62},"min":{"1min":0.40500000000000003,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79200000000000004,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.56399999999999995}]},{"osd":2,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52400000000000002,"15min":0.63700000000000001},"min":{"1min":0.39100000000000001,"5min":0.32900000000000001,"15min":0.28299999999999997},"max":{"1min":0.73299999999999998,"5min":0.76600000000000001,"15min":5.2960000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.38500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.5}]},{"osd":3,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55900000000000005,"15min":0.66900000000000004},"min":{"1min":0.41999999999999998,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":5.3330000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56399999999999995,"15min":0.68000000000000005},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.27700000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.57699999999999996}]},{"osd":4,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.48899999999999999,"15min":0.498},"min":{"1min":0.33400000000000002,"5min":0.29599999999999999,"15min":0.28699999999999998},"max":{"1min":0.67000000000000004,"5min":0.72499999999999998,"15min":0.82999999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.34499999999999997,"15min":0.32200000000000001},"max":{"1min":0.77200000000000002,"5min":0.88,"15min":1.0800000000000001},"last":0.59299999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.628},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.39900000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78900000000000003,"5min":0.82499999999999996,"15min":0.89800000000000002},"last":0.65900000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.32300000000000001,"15min":0.30099999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98399999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.67200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:42:57.775 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:42:57.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.925+0000 7fa2c9bf1700 1 Processor -- start 2023-10-10T20:42:57.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.925+0000 7fa2c9bf1700 1 -- start start 2023-10-10T20:42:57.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.925+0000 7fa2c9bf1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa2c4069560 0x7fa2c40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.925+0000 7fa2c9bf1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c4069ee0 con 0x7fa2c4159890 2023-10-10T20:42:57.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.925+0000 7fa2c9bf1700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c415a160 con 0x7fa2c4159c60 2023-10-10T20:42:57.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.925+0000 7fa2c9bf1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40d13a0 con 0x7fa2c4069560 2023-10-10T20:42:57.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c37fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa2c4159890 0x7fa2c415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48828/0 (socket says 172.21.15.89:48828) 2023-10-10T20:42:57.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c37fe700 1 -- 172.21.15.89:0/2448285710 learned_addr learned my addr 172.21.15.89:0/2448285710 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:57.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c27fc700 1 -- 172.21.15.89:0/2448285710 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3050730906 0 0) 0x7fa2c4069ee0 con 0x7fa2c4159890 2023-10-10T20:42:57.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c3fff700 1 --2- 172.21.15.89:0/2448285710 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa2c4069560 0x7fa2c40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c27fc700 1 -- 172.21.15.89:0/2448285710 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2ac003570 con 0x7fa2c4159890 2023-10-10T20:42:57.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c27fc700 1 -- 172.21.15.89:0/2448285710 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 647533999 0 0) 0x7fa2ac003570 con 0x7fa2c4159890 2023-10-10T20:42:57.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c27fc700 1 -- 172.21.15.89:0/2448285710 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa2c4069560 msgr2=0x7fa2c40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:57.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c27fc700 1 --2- 172.21.15.89:0/2448285710 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa2c4069560 0x7fa2c40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c27fc700 1 -- 172.21.15.89:0/2448285710 >> v1:172.21.15.111:6789/0 conn(0x7fa2c4159c60 legacy=0x7fa2c415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:57.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c27fc700 1 -- 172.21.15.89:0/2448285710 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c415e140 con 0x7fa2c4159890 2023-10-10T20:42:57.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c27fc700 1 -- 172.21.15.89:0/2448285710 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fa2b40017f0 con 0x7fa2c4159890 2023-10-10T20:42:57.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c27fc700 1 -- 172.21.15.89:0/2448285710 >> v1:172.21.15.89:6789/0 conn(0x7fa2c4159890 legacy=0x7fa2c415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:57.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c27fc700 1 --2- 172.21.15.89:0/2448285710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2ac003a90 0x7fa2ac003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c27fc700 1 --2- 172.21.15.89:0/2448285710 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa2c4069560 0x7fa2ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c27fc700 1 --2- 172.21.15.89:0/2448285710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2ac004d50 0x7fa2ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c27fc700 1 -- 172.21.15.89:0/2448285710 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40d13a0 con 0x7fa2ac004d50 2023-10-10T20:42:57.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c27fc700 1 -- 172.21.15.89:0/2448285710 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac00bb40 con 0x7fa2ac003a90 2023-10-10T20:42:57.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c27fc700 1 -- 172.21.15.89:0/2448285710 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac00bd00 con 0x7fa2c4069560 2023-10-10T20:42:57.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c3fff700 1 --2- 172.21.15.89:0/2448285710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2ac004d50 0x7fa2ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c37fe700 1 --2- 172.21.15.89:0/2448285710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2ac003a90 0x7fa2ac003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.926+0000 7fa2c2ffd700 1 --2- 172.21.15.89:0/2448285710 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa2c4069560 0x7fa2ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.927+0000 7fa2c3fff700 1 -- 172.21.15.89:0/2448285710 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa2c4069560 msgr2=0x7fa2ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:57.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.927+0000 7fa2c3fff700 1 --2- 172.21.15.89:0/2448285710 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa2c4069560 0x7fa2ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.927+0000 7fa2c3fff700 1 -- 172.21.15.89:0/2448285710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2ac003a90 msgr2=0x7fa2ac003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:57.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.927+0000 7fa2c3fff700 1 --2- 172.21.15.89:0/2448285710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2ac003a90 0x7fa2ac003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.927+0000 7fa2c3fff700 1 -- 172.21.15.89:0/2448285710 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac003570 con 0x7fa2ac004d50 2023-10-10T20:42:57.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.927+0000 7fa2c3fff700 1 --2- 172.21.15.89:0/2448285710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2ac004d50 0x7fa2ac0051c0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009c00 tx=0x7fa2bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29696d78ebb9b5a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:57.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.927+0000 7fa2c27fc700 1 -- 172.21.15.89:0/2448285710 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc00e710 con 0x7fa2ac004d50 2023-10-10T20:42:57.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.927+0000 7fa2c27fc700 1 -- 172.21.15.89:0/2448285710 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc00e870 con 0x7fa2ac004d50 2023-10-10T20:42:57.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.927+0000 7fa2c27fc700 1 -- 172.21.15.89:0/2448285710 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc00ec30 con 0x7fa2ac004d50 2023-10-10T20:42:57.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.927+0000 7fa2c9bf1700 1 -- 172.21.15.89:0/2448285710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2ac004d50 msgr2=0x7fa2ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:57.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.927+0000 7fa2c9bf1700 1 --2- 172.21.15.89:0/2448285710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2ac004d50 0x7fa2ac0051c0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009c00 tx=0x7fa2bc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.927+0000 7fa2c3fff700 1 -- 172.21.15.89:0/2448285710 reap_dead start 2023-10-10T20:42:57.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.927+0000 7fa2c9bf1700 1 -- 172.21.15.89:0/2448285710 shutdown_connections 2023-10-10T20:42:57.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.927+0000 7fa2c9bf1700 1 -- 172.21.15.89:0/2448285710 >> 172.21.15.89:0/2448285710 conn(0x7fa2c4068e20 msgr2=0x7fa2c40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:57.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.928+0000 7fa2c9bf1700 1 -- 172.21.15.89:0/2448285710 shutdown_connections 2023-10-10T20:42:57.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.928+0000 7fa2c9bf1700 1 -- 172.21.15.89:0/2448285710 wait complete. 2023-10-10T20:42:57.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.929+0000 7fa2c9bf1700 1 Processor -- start 2023-10-10T20:42:57.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.929+0000 7fa2c9bf1700 1 -- start start 2023-10-10T20:42:57.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.929+0000 7fa2c9bf1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa2c40b1240 0x7fa2c4141a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.929+0000 7fa2c9bf1700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2c40b1610 0x7fa2c40b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.929+0000 7fa2c9bf1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2c41543f0 0x7fa2c4154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.929+0000 7fa2c9bf1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c4155bd0 con 0x7fa2c41543f0 2023-10-10T20:42:57.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.929+0000 7fa2c9bf1700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c4155230 con 0x7fa2c40b1610 2023-10-10T20:42:57.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.929+0000 7fa2c9bf1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40b3ff0 con 0x7fa2c40b1240 2023-10-10T20:42:57.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.930+0000 7fa2c3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2c41543f0 0x7fa2c4154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.930+0000 7fa2c3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2c41543f0 0x7fa2c4154860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55094/0 (socket says 172.21.15.89:55094) 2023-10-10T20:42:57.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.930+0000 7fa2c3fff700 1 -- 172.21.15.89:0/850330194 learned_addr learned my addr 172.21.15.89:0/850330194 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:57.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.930+0000 7fa2c37fe700 1 --2- 172.21.15.89:0/850330194 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa2c40b1240 0x7fa2c4141a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.930+0000 7fa2c2ffd700 1 --2- 172.21.15.89:0/850330194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2c40b1610 0x7fa2c40b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.930+0000 7fa2c3fff700 1 -- 172.21.15.89:0/850330194 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa2c40b1240 msgr2=0x7fa2c4141a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:57.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.930+0000 7fa2c3fff700 1 --2- 172.21.15.89:0/850330194 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa2c40b1240 0x7fa2c4141a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.930+0000 7fa2c3fff700 1 -- 172.21.15.89:0/850330194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2c40b1610 msgr2=0x7fa2c40b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:57.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.930+0000 7fa2c3fff700 1 --2- 172.21.15.89:0/850330194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2c40b1610 0x7fa2c40b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:57.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.930+0000 7fa2c3fff700 1 -- 172.21.15.89:0/850330194 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c40b36c0 con 0x7fa2c41543f0 2023-10-10T20:42:57.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.930+0000 7fa2c3fff700 1 --2- 172.21.15.89:0/850330194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2c41543f0 0x7fa2c4154860 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00bb10 tx=0x7fa2bc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:57.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.930+0000 7fa2ab7fe700 1 -- 172.21.15.89:0/850330194 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc022070 con 0x7fa2c41543f0 2023-10-10T20:42:57.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.930+0000 7fa2ab7fe700 1 -- 172.21.15.89:0/850330194 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc00ebf0 con 0x7fa2c41543f0 2023-10-10T20:42:57.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.930+0000 7fa2c9bf1700 1 -- 172.21.15.89:0/850330194 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c40b39a0 con 0x7fa2c41543f0 2023-10-10T20:42:57.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.930+0000 7fa2ab7fe700 1 -- 172.21.15.89:0/850330194 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc00e710 con 0x7fa2c41543f0 2023-10-10T20:42:57.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.930+0000 7fa2c9bf1700 1 -- 172.21.15.89:0/850330194 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c40b1fc0 con 0x7fa2c41543f0 2023-10-10T20:42:57.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.931+0000 7fa2c9bf1700 1 -- 172.21.15.89:0/850330194 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c404c3c0 con 0x7fa2c41543f0 2023-10-10T20:42:57.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.931+0000 7fa2ab7fe700 1 -- 172.21.15.89:0/850330194 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa2bc027940 con 0x7fa2c41543f0 2023-10-10T20:42:57.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.932+0000 7fa2ab7fe700 1 --2- 172.21.15.89:0/850330194 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa2b0067320 0x7fa2b00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:57.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.932+0000 7fa2ab7fe700 1 -- 172.21.15.89:0/850330194 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa2bc000e40 con 0x7fa2c41543f0 2023-10-10T20:42:57.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.932+0000 7fa2c37fe700 1 --2- 172.21.15.89:0/850330194 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa2b0067320 0x7fa2b00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:57.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.932+0000 7fa2c37fe700 1 --2- 172.21.15.89:0/850330194 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa2b0067320 0x7fa2b00697d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4000fc0 tx=0x7fa2b40021f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:57.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:57.936+0000 7fa2ab7fe700 1 -- 172.21.15.89:0/850330194 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa2bc053060 con 0x7fa2c41543f0 2023-10-10T20:42:58.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.113+0000 7fa2c9bf1700 1 -- 172.21.15.89:0/850330194 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa2c40b4130 con 0x7fa2b0067320 2023-10-10T20:42:58.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.121+0000 7fa2ab7fe700 1 -- 172.21.15.89:0/850330194 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286942 (secure 0 0 0) 0x7fa2c40b4130 con 0x7fa2b0067320 2023-10-10T20:42:58.127 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:58.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.124+0000 7fa2c9bf1700 1 -- 172.21.15.89:0/850330194 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa2b0067320 msgr2=0x7fa2b00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:58.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.124+0000 7fa2c9bf1700 1 --2- 172.21.15.89:0/850330194 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa2b0067320 0x7fa2b00697d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4000fc0 tx=0x7fa2b40021f0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.124+0000 7fa2c9bf1700 1 -- 172.21.15.89:0/850330194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2c41543f0 msgr2=0x7fa2c4154860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:58.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.124+0000 7fa2c9bf1700 1 --2- 172.21.15.89:0/850330194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2c41543f0 0x7fa2c4154860 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00bb10 tx=0x7fa2bc01a000 comp rx=0 tx=0).stop 2023-10-10T20:42:58.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.124+0000 7fa2c9bf1700 1 -- 172.21.15.89:0/850330194 shutdown_connections 2023-10-10T20:42:58.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.124+0000 7fa2c9bf1700 1 --2- 172.21.15.89:0/850330194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2c41543f0 0x7fa2c4154860 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.124+0000 7fa2c9bf1700 1 --2- 172.21.15.89:0/850330194 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa2c40b1240 0x7fa2c4141a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.124+0000 7fa2c9bf1700 1 --2- 172.21.15.89:0/850330194 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa2b0067320 0x7fa2b00697d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.124+0000 7fa2c9bf1700 1 --2- 172.21.15.89:0/850330194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2c40b1610 0x7fa2c40b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.124+0000 7fa2c9bf1700 1 -- 172.21.15.89:0/850330194 >> 172.21.15.89:0/850330194 conn(0x7fa2c4068e20 msgr2=0x7fa2c4069930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:58.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.124+0000 7fa2c9bf1700 1 -- 172.21.15.89:0/850330194 shutdown_connections 2023-10-10T20:42:58.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.124+0000 7fa2c9bf1700 1 -- 172.21.15.89:0/850330194 wait complete. 2023-10-10T20:42:58.132 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:42:58.138 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2023-10-10T20:42:56.751262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428364,"kb_used_data":17292,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752546356,"statfs":{"total":773094113280,"available":770607468544,"internally_reserved":0,"allocated":17707008,"data_stored":8556385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.433293"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.323231+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:40.796551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.325806+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:28:12.284338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.331625+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:09:34.459455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.330945+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:16:50.053827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329837+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:16:07.451863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.327658+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:58:12.432427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.326884+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:36:30.295360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322678+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:59.692515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329758+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:02:55.597436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.327137+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:32:08.240124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.323857+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:32.579274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.325054+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:36:30.295360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.324653+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:03.524684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329133+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:41:28.090635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329269+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:38:16.787078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.324787+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:58:32.593330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328589+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:35.674118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.326650+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:38:30.259757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.334129+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:20:38.627889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.327487+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:27.171367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329421+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:18:18.498552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.330301+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:00:48.564823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.323234+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:58:00.782955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328346+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:02:55.597436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322747+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:25:50.618140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322798+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:04:51.509487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.323414+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:20:34.093326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.326676+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:33:29.535005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328679+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:50:07.909344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322817+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:44:40.439820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.330007+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:02.852226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329533+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:13:20.878853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328750+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:46.022458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.335968+0000","last_change":"2023-10-10T20:42:52.335968+0000","last_active":"2023-10-10T20:42:52.335968+0000","last_peered":"2023-10-10T20:42:52.335968+0000","last_clean":"2023-10-10T20:42:52.335968+0000","last_became_active":"2023-10-10T20:42:52.335695+0000","last_became_peered":"2023-10-10T20:42:52.335695+0000","last_unstale":"2023-10-10T20:42:52.335968+0000","last_undegraded":"2023-10-10T20:42:52.335968+0000","last_fullsized":"2023-10-10T20:42:52.335968+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:32:07.233388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322742+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:39:23.346373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.336051+0000","last_change":"2023-10-10T20:42:52.336051+0000","last_active":"2023-10-10T20:42:52.336051+0000","last_peered":"2023-10-10T20:42:52.336051+0000","last_clean":"2023-10-10T20:42:52.336051+0000","last_became_active":"2023-10-10T20:42:52.335752+0000","last_became_peered":"2023-10-10T20:42:52.335752+0000","last_unstale":"2023-10-10T20:42:52.336051+0000","last_undegraded":"2023-10-10T20:42:52.336051+0000","last_fullsized":"2023-10-10T20:42:52.336051+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:15:01.473071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329028+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:00:56.184401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.339512+0000","last_change":"2023-10-10T20:42:52.339512+0000","last_active":"2023-10-10T20:42:52.339512+0000","last_peered":"2023-10-10T20:42:52.339512+0000","last_clean":"2023-10-10T20:42:52.339512+0000","last_became_active":"2023-10-10T20:42:52.339230+0000","last_became_peered":"2023-10-10T20:42:52.339230+0000","last_unstale":"2023-10-10T20:42:52.339512+0000","last_undegraded":"2023-10-10T20:42:52.339512+0000","last_fullsized":"2023-10-10T20:42:52.339512+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:51.664988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322141+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:45:05.557928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.332410+0000","last_change":"2023-10-10T20:42:52.332410+0000","last_active":"2023-10-10T20:42:52.332410+0000","last_peered":"2023-10-10T20:42:52.332410+0000","last_clean":"2023-10-10T20:42:52.332410+0000","last_became_active":"2023-10-10T20:42:52.331952+0000","last_became_peered":"2023-10-10T20:42:52.331952+0000","last_unstale":"2023-10-10T20:42:52.332410+0000","last_undegraded":"2023-10-10T20:42:52.332410+0000","last_fullsized":"2023-10-10T20:42:52.332410+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:46:55.907438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328114+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:00.861926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.338841+0000","last_change":"2023-10-10T20:42:52.338841+0000","last_active":"2023-10-10T20:42:52.338841+0000","last_peered":"2023-10-10T20:42:52.338841+0000","last_clean":"2023-10-10T20:42:52.338841+0000","last_became_active":"2023-10-10T20:42:52.337405+0000","last_became_peered":"2023-10-10T20:42:52.337405+0000","last_unstale":"2023-10-10T20:42:52.338841+0000","last_undegraded":"2023-10-10T20:42:52.338841+0000","last_fullsized":"2023-10-10T20:42:52.338841+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:10:49.794184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.327089+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:52.463061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.333926+0000","last_change":"2023-10-10T20:42:52.333926+0000","last_active":"2023-10-10T20:42:52.333926+0000","last_peered":"2023-10-10T20:42:52.333926+0000","last_clean":"2023-10-10T20:42:52.333926+0000","last_became_active":"2023-10-10T20:42:52.333645+0000","last_became_peered":"2023-10-10T20:42:52.333645+0000","last_unstale":"2023-10-10T20:42:52.333926+0000","last_undegraded":"2023-10-10T20:42:52.333926+0000","last_fullsized":"2023-10-10T20:42:52.333926+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:46:47.379359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.324299+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:17:23.703124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.331824+0000","last_change":"2023-10-10T20:42:52.331824+0000","last_active":"2023-10-10T20:42:52.331824+0000","last_peered":"2023-10-10T20:42:52.331824+0000","last_clean":"2023-10-10T20:42:52.331824+0000","last_became_active":"2023-10-10T20:42:52.331567+0000","last_became_peered":"2023-10-10T20:42:52.331567+0000","last_unstale":"2023-10-10T20:42:52.331824+0000","last_undegraded":"2023-10-10T20:42:52.331824+0000","last_fullsized":"2023-10-10T20:42:52.331824+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:59.776219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.325326+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:30:49.951335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.329973+0000","last_change":"2023-10-10T20:42:52.329973+0000","last_active":"2023-10-10T20:42:52.329973+0000","last_peered":"2023-10-10T20:42:52.329973+0000","last_clean":"2023-10-10T20:42:52.329973+0000","last_became_active":"2023-10-10T20:42:52.329001+0000","last_became_peered":"2023-10-10T20:42:52.329001+0000","last_unstale":"2023-10-10T20:42:52.329973+0000","last_undegraded":"2023-10-10T20:42:52.329973+0000","last_fullsized":"2023-10-10T20:42:52.329973+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:20:37.621153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322179+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:13:45.883755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.334832+0000","last_change":"2023-10-10T20:42:52.334832+0000","last_active":"2023-10-10T20:42:52.334832+0000","last_peered":"2023-10-10T20:42:52.334832+0000","last_clean":"2023-10-10T20:42:52.334832+0000","last_became_active":"2023-10-10T20:42:52.334555+0000","last_became_peered":"2023-10-10T20:42:52.334555+0000","last_unstale":"2023-10-10T20:42:52.334832+0000","last_undegraded":"2023-10-10T20:42:52.334832+0000","last_fullsized":"2023-10-10T20:42:52.334832+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:51.456325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328250+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:31.176991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.341985+0000","last_change":"2023-10-10T20:42:52.341985+0000","last_active":"2023-10-10T20:42:52.341985+0000","last_peered":"2023-10-10T20:42:52.341985+0000","last_clean":"2023-10-10T20:42:52.341985+0000","last_became_active":"2023-10-10T20:42:52.341730+0000","last_became_peered":"2023-10-10T20:42:52.341730+0000","last_unstale":"2023-10-10T20:42:52.341985+0000","last_undegraded":"2023-10-10T20:42:52.341985+0000","last_fullsized":"2023-10-10T20:42:52.341985+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:32:48.988699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322090+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:17:25.969866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.333830+0000","last_change":"2023-10-10T20:42:52.333830+0000","last_active":"2023-10-10T20:42:52.333830+0000","last_peered":"2023-10-10T20:42:52.333830+0000","last_clean":"2023-10-10T20:42:52.333830+0000","last_became_active":"2023-10-10T20:42:52.333570+0000","last_became_peered":"2023-10-10T20:42:52.333570+0000","last_unstale":"2023-10-10T20:42:52.333830+0000","last_undegraded":"2023-10-10T20:42:52.333830+0000","last_fullsized":"2023-10-10T20:42:52.333830+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:31.572538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329839+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:18:15.714890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.339386+0000","last_change":"2023-10-10T20:42:52.339386+0000","last_active":"2023-10-10T20:42:52.339386+0000","last_peered":"2023-10-10T20:42:52.339386+0000","last_clean":"2023-10-10T20:42:52.339386+0000","last_became_active":"2023-10-10T20:42:52.339072+0000","last_became_peered":"2023-10-10T20:42:52.339072+0000","last_unstale":"2023-10-10T20:42:52.339386+0000","last_undegraded":"2023-10-10T20:42:52.339386+0000","last_fullsized":"2023-10-10T20:42:52.339386+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:53:19.603938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.333958+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:20:38.627889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.337702+0000","last_change":"2023-10-10T20:42:52.337702+0000","last_active":"2023-10-10T20:42:52.337702+0000","last_peered":"2023-10-10T20:42:52.337702+0000","last_clean":"2023-10-10T20:42:52.337702+0000","last_became_active":"2023-10-10T20:42:52.337415+0000","last_became_peered":"2023-10-10T20:42:52.337415+0000","last_unstale":"2023-10-10T20:42:52.337702+0000","last_undegraded":"2023-10-10T20:42:52.337702+0000","last_fullsized":"2023-10-10T20:42:52.337702+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:33:28.528269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.331014+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:05:24.092738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.338838+0000","last_change":"2023-10-10T20:42:52.338838+0000","last_active":"2023-10-10T20:42:52.338838+0000","last_peered":"2023-10-10T20:42:52.338838+0000","last_clean":"2023-10-10T20:42:52.338838+0000","last_became_active":"2023-10-10T20:42:52.338592+0000","last_became_peered":"2023-10-10T20:42:52.338592+0000","last_unstale":"2023-10-10T20:42:52.338838+0000","last_undegraded":"2023-10-10T20:42:52.338838+0000","last_fullsized":"2023-10-10T20:42:52.338838+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:36:29.288624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328957+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:46:36.668645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.334750+0000","last_change":"2023-10-10T20:42:52.334750+0000","last_active":"2023-10-10T20:42:52.334750+0000","last_peered":"2023-10-10T20:42:52.334750+0000","last_clean":"2023-10-10T20:42:52.334750+0000","last_became_active":"2023-10-10T20:42:52.334260+0000","last_became_peered":"2023-10-10T20:42:52.334260+0000","last_unstale":"2023-10-10T20:42:52.334750+0000","last_undegraded":"2023-10-10T20:42:52.334750+0000","last_fullsized":"2023-10-10T20:42:52.334750+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:49:39.517807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322365+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:04:02.591369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.339183+0000","last_change":"2023-10-10T20:42:52.339183+0000","last_active":"2023-10-10T20:42:52.339183+0000","last_peered":"2023-10-10T20:42:52.339183+0000","last_clean":"2023-10-10T20:42:52.339183+0000","last_became_active":"2023-10-10T20:42:52.337722+0000","last_became_peered":"2023-10-10T20:42:52.337722+0000","last_unstale":"2023-10-10T20:42:52.339183+0000","last_undegraded":"2023-10-10T20:42:52.339183+0000","last_fullsized":"2023-10-10T20:42:52.339183+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:51.664988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.326616+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:57:12.184703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.335410+0000","last_change":"2023-10-10T20:42:52.335410+0000","last_active":"2023-10-10T20:42:52.335410+0000","last_peered":"2023-10-10T20:42:52.335410+0000","last_clean":"2023-10-10T20:42:52.335410+0000","last_became_active":"2023-10-10T20:42:52.335097+0000","last_became_peered":"2023-10-10T20:42:52.335097+0000","last_unstale":"2023-10-10T20:42:52.335410+0000","last_undegraded":"2023-10-10T20:42:52.335410+0000","last_fullsized":"2023-10-10T20:42:52.335410+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:22:17.318577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.327959+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:58:32.593330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.332490+0000","last_change":"2023-10-10T20:42:52.332490+0000","last_active":"2023-10-10T20:42:52.332490+0000","last_peered":"2023-10-10T20:42:52.332490+0000","last_clean":"2023-10-10T20:42:52.332490+0000","last_became_active":"2023-10-10T20:42:52.332045+0000","last_became_peered":"2023-10-10T20:42:52.332045+0000","last_unstale":"2023-10-10T20:42:52.332490+0000","last_undegraded":"2023-10-10T20:42:52.332490+0000","last_fullsized":"2023-10-10T20:42:52.332490+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:01.845490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.331151+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:18:18.498552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.333544+0000","last_change":"2023-10-10T20:42:52.333544+0000","last_active":"2023-10-10T20:42:52.333544+0000","last_peered":"2023-10-10T20:42:52.333544+0000","last_clean":"2023-10-10T20:42:52.333544+0000","last_became_active":"2023-10-10T20:42:52.333211+0000","last_became_peered":"2023-10-10T20:42:52.333211+0000","last_unstale":"2023-10-10T20:42:52.333544+0000","last_undegraded":"2023-10-10T20:42:52.333544+0000","last_fullsized":"2023-10-10T20:42:52.333544+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:39.789815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328162+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:46.022458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.336202+0000","last_change":"2023-10-10T20:42:52.336202+0000","last_active":"2023-10-10T20:42:52.336202+0000","last_peered":"2023-10-10T20:42:52.336202+0000","last_clean":"2023-10-10T20:42:52.336202+0000","last_became_active":"2023-10-10T20:42:52.335926+0000","last_became_peered":"2023-10-10T20:42:52.335926+0000","last_unstale":"2023-10-10T20:42:52.336202+0000","last_undegraded":"2023-10-10T20:42:52.336202+0000","last_fullsized":"2023-10-10T20:42:52.336202+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:49:39.517807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.333557+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:16:50.053827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.335457+0000","last_change":"2023-10-10T20:42:52.335457+0000","last_active":"2023-10-10T20:42:52.335457+0000","last_peered":"2023-10-10T20:42:52.335457+0000","last_clean":"2023-10-10T20:42:52.335457+0000","last_became_active":"2023-10-10T20:42:52.335186+0000","last_became_peered":"2023-10-10T20:42:52.335186+0000","last_unstale":"2023-10-10T20:42:52.335457+0000","last_undegraded":"2023-10-10T20:42:52.335457+0000","last_fullsized":"2023-10-10T20:42:52.335457+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:55:21.874802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322254+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:21:17.789832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.337248+0000","last_change":"2023-10-10T20:42:52.337248+0000","last_active":"2023-10-10T20:42:52.337248+0000","last_peered":"2023-10-10T20:42:52.337248+0000","last_clean":"2023-10-10T20:42:52.337248+0000","last_became_active":"2023-10-10T20:42:52.336966+0000","last_became_peered":"2023-10-10T20:42:52.336966+0000","last_unstale":"2023-10-10T20:42:52.337248+0000","last_undegraded":"2023-10-10T20:42:52.337248+0000","last_fullsized":"2023-10-10T20:42:52.337248+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:59:14.539048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.329553+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:09:34.459455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.350531+0000","last_change":"2023-10-10T20:42:52.350531+0000","last_active":"2023-10-10T20:42:52.350531+0000","last_peered":"2023-10-10T20:42:52.350531+0000","last_clean":"2023-10-10T20:42:52.350531+0000","last_became_active":"2023-10-10T20:42:52.350315+0000","last_became_peered":"2023-10-10T20:42:52.350315+0000","last_unstale":"2023-10-10T20:42:52.350531+0000","last_undegraded":"2023-10-10T20:42:52.350531+0000","last_fullsized":"2023-10-10T20:42:52.350531+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:08.850614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.330288+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:28:12.284338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.338420+0000","last_change":"2023-10-10T20:42:52.338420+0000","last_active":"2023-10-10T20:42:52.338420+0000","last_peered":"2023-10-10T20:42:52.338420+0000","last_clean":"2023-10-10T20:42:52.338420+0000","last_became_active":"2023-10-10T20:42:52.338190+0000","last_became_peered":"2023-10-10T20:42:52.338190+0000","last_unstale":"2023-10-10T20:42:52.338420+0000","last_undegraded":"2023-10-10T20:42:52.338420+0000","last_fullsized":"2023-10-10T20:42:52.338420+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:58:31.586594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.327181+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:30:49.951335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.335469+0000","last_change":"2023-10-10T20:42:52.335469+0000","last_active":"2023-10-10T20:42:52.335469+0000","last_peered":"2023-10-10T20:42:52.335469+0000","last_clean":"2023-10-10T20:42:52.335469+0000","last_became_active":"2023-10-10T20:42:52.333014+0000","last_became_peered":"2023-10-10T20:42:52.333014+0000","last_unstale":"2023-10-10T20:42:52.335469+0000","last_undegraded":"2023-10-10T20:42:52.335469+0000","last_fullsized":"2023-10-10T20:42:52.335469+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:16:49.047090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328435+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:00.861926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.334206+0000","last_change":"2023-10-10T20:42:52.334206+0000","last_active":"2023-10-10T20:42:52.334206+0000","last_peered":"2023-10-10T20:42:52.334206+0000","last_clean":"2023-10-10T20:42:52.334206+0000","last_became_active":"2023-10-10T20:42:52.333022+0000","last_became_peered":"2023-10-10T20:42:52.333022+0000","last_unstale":"2023-10-10T20:42:52.334206+0000","last_undegraded":"2023-10-10T20:42:52.334206+0000","last_fullsized":"2023-10-10T20:42:52.334206+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:34:10.314535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328547+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:05:24.092738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.339808+0000","last_change":"2023-10-10T20:42:52.339808+0000","last_active":"2023-10-10T20:42:52.339808+0000","last_peered":"2023-10-10T20:42:52.339808+0000","last_clean":"2023-10-10T20:42:52.339808+0000","last_became_active":"2023-10-10T20:42:52.339538+0000","last_became_peered":"2023-10-10T20:42:52.339538+0000","last_unstale":"2023-10-10T20:42:52.339808+0000","last_undegraded":"2023-10-10T20:42:52.339808+0000","last_fullsized":"2023-10-10T20:42:52.339808+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:17:22.696388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.324346+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:32:49.995435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.330898+0000","last_change":"2023-10-10T20:42:52.330898+0000","last_active":"2023-10-10T20:42:52.330898+0000","last_peered":"2023-10-10T20:42:52.330898+0000","last_clean":"2023-10-10T20:42:52.330898+0000","last_became_active":"2023-10-10T20:42:52.330616+0000","last_became_peered":"2023-10-10T20:42:52.330616+0000","last_unstale":"2023-10-10T20:42:52.330898+0000","last_undegraded":"2023-10-10T20:42:52.330898+0000","last_fullsized":"2023-10-10T20:42:52.330898+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:30.999244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.333078+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:02.852226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.333625+0000","last_change":"2023-10-10T20:42:52.333625+0000","last_active":"2023-10-10T20:42:52.333625+0000","last_peered":"2023-10-10T20:42:52.333625+0000","last_clean":"2023-10-10T20:42:52.333625+0000","last_became_active":"2023-10-10T20:42:52.333359+0000","last_became_peered":"2023-10-10T20:42:52.333359+0000","last_unstale":"2023-10-10T20:42:52.333625+0000","last_undegraded":"2023-10-10T20:42:52.333625+0000","last_fullsized":"2023-10-10T20:42:52.333625+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:31.572538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.328569+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:28:12.284338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.339859+0000","last_change":"2023-10-10T20:42:52.339859+0000","last_active":"2023-10-10T20:42:52.339859+0000","last_peered":"2023-10-10T20:42:52.339859+0000","last_clean":"2023-10-10T20:42:52.339859+0000","last_became_active":"2023-10-10T20:42:52.339585+0000","last_became_peered":"2023-10-10T20:42:52.339585+0000","last_unstale":"2023-10-10T20:42:52.339859+0000","last_undegraded":"2023-10-10T20:42:52.339859+0000","last_fullsized":"2023-10-10T20:42:52.339859+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:02.517948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322090+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:38:10.362086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.335154+0000","last_change":"2023-10-10T20:42:52.335154+0000","last_active":"2023-10-10T20:42:52.335154+0000","last_peered":"2023-10-10T20:42:52.335154+0000","last_clean":"2023-10-10T20:42:52.335154+0000","last_became_active":"2023-10-10T20:42:52.334853+0000","last_became_peered":"2023-10-10T20:42:52.334853+0000","last_unstale":"2023-10-10T20:42:52.335154+0000","last_undegraded":"2023-10-10T20:42:52.335154+0000","last_fullsized":"2023-10-10T20:42:52.335154+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:15:01.473071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:42:52.314972+0000","last_change":"2023-10-10T20:42:52.322090+0000","last_active":"2023-10-10T20:42:52.314972+0000","last_peered":"2023-10-10T20:42:52.314972+0000","last_clean":"2023-10-10T20:42:52.314972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:42:52.314972+0000","last_undegraded":"2023-10-10T20:42:52.314972+0000","last_fullsized":"2023-10-10T20:42:52.314972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:35.275237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-10T20:42:52.338269+0000","last_change":"2023-10-10T20:42:52.338269+0000","last_active":"2023-10-10T20:42:52.338269+0000","last_peered":"2023-10-10T20:42:52.338269+0000","last_clean":"2023-10-10T20:42:52.338269+0000","last_became_active":"2023-10-10T20:42:52.338028+0000","last_became_peered":"2023-10-10T20:42:52.338028+0000","last_unstale":"2023-10-10T20:42:52.338269+0000","last_undegraded":"2023-10-10T20:42:52.338269+0000","last_fullsized":"2023-10-10T20:42:52.338269+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:17:34.621354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":306,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-10T20:42:51.319419+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:42:51.319419+0000","last_peered":"2023-10-10T20:42:51.319419+0000","last_clean":"2023-10-10T20:42:51.319419+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:42:51.319419+0000","last_undegraded":"2023-10-10T20:42:51.319419+0000","last_fullsized":"2023-10-10T20:42:51.319419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574953,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303984,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067856,"statfs":{"total":96636764160,"available":96325484544,"internally_reserved":0,"allocated":2662400,"data_stored":1514010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50700000000000001,"15min":0.75800000000000001},"min":{"1min":0.32500000000000001,"5min":0.309,"15min":0.29199999999999998},"max":{"1min":0.64300000000000002,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.50800000000000001,"15min":0.60399999999999998},"min":{"1min":0.36399999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.58299999999999996,"5min":0.752,"15min":4.1959999999999997},"last":0.67000000000000004}]},{"osd":1,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.50600000000000001,"15min":0.90100000000000002},"min":{"1min":0.372,"5min":0.317,"15min":0.317},"max":{"1min":0.61099999999999999,"5min":0.78800000000000003,"15min":10.1},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.53400000000000003,"15min":0.69099999999999995},"min":{"1min":0.35199999999999998,"5min":0.33900000000000002,"15min":0.28999999999999998},"max":{"1min":0.66500000000000004,"5min":0.81799999999999995,"15min":6.806},"last":0.63600000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56799999999999995,"15min":0.74099999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68799999999999994,"5min":0.89000000000000001,"15min":6.9340000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.69399999999999995},"min":{"1min":0.40699999999999997,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.69099999999999995,"5min":0.77100000000000002,"15min":6.9870000000000001},"last":0.747}]},{"osd":3,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54200000000000004,"15min":0.71299999999999997},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64500000000000002,"5min":0.754,"15min":6.6719999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55900000000000005,"15min":0.64800000000000002},"min":{"1min":0.39800000000000002,"5min":0.378,"15min":0.34799999999999998},"max":{"1min":0.64400000000000002,"5min":0.76200000000000001,"15min":4.1609999999999996},"last":0.71899999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47799999999999998,"15min":0.46800000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.77200000000000002},"last":0.5},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.51600000000000001,"15min":0.51900000000000002},"min":{"1min":0.314,"5min":0.309,"15min":0.30399999999999999},"max":{"1min":0.68000000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.51400000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.54000000000000004,"15min":0.53900000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.68400000000000005,"5min":0.86099999999999999,"15min":1.3180000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.53600000000000003,"15min":0.52700000000000002},"min":{"1min":0.35299999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.84099999999999997},"last":0.54600000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.35699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":1.7509999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76300000000000001,"5min":0.84099999999999997,"15min":1.7090000000000001},"last":0.65200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574954,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303404,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068436,"statfs":{"total":96636764160,"available":96326078464,"internally_reserved":0,"allocated":2068480,"data_stored":923642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52400000000000002,"15min":0.64000000000000001},"min":{"1min":0.34699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":7.7590000000000003},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52400000000000002,"15min":0.67400000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.86899999999999999,"5min":1.075,"15min":7.8090000000000002},"last":0.52700000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52900000000000003,"15min":0.67200000000000004},"min":{"1min":0.32800000000000001,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":7.867},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55500000000000005,"15min":0.81999999999999995},"min":{"1min":0.378,"5min":0.35499999999999998,"15min":0.33700000000000002},"max":{"1min":0.70999999999999996,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.70999999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52300000000000002,"15min":0.58099999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.745,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57099999999999995,"15min":0.79100000000000004},"min":{"1min":0.39700000000000002,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":2.641,"15min":6.7629999999999999},"last":0.55600000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55500000000000005,"15min":0.60699999999999998},"min":{"1min":0.372,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94399999999999995,"5min":1.107,"15min":4.8620000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55500000000000005,"15min":0.68400000000000005},"min":{"1min":0.41399999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.72599999999999998,"5min":1.718,"15min":7.9009999999999998},"last":0.67300000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51900000000000002,"15min":0.47899999999999998},"min":{"1min":0.32000000000000001,"5min":0.28999999999999998,"15min":0.28499999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.755},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":0.82899999999999996,"5min":1.032,"15min":1.032},"last":0.77200000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53400000000000003,"15min":0.5},"min":{"1min":0.34999999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.86799999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56999999999999995,"15min":0.53700000000000003},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85399999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.79000000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56200000000000006,"15min":0.53000000000000003},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.89600000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.59399999999999997,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.372,"15min":0.35099999999999998},"max":{"1min":0.82199999999999995,"5min":1.681,"15min":1.681},"last":0.80100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303404,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068436,"statfs":{"total":96636764160,"available":96326078464,"internally_reserved":0,"allocated":2068480,"data_stored":923642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.31900000000000001,"15min":0.443},"min":{"1min":0.215,"5min":0.19600000000000001,"15min":0.17899999999999999},"max":{"1min":0.47799999999999998,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33000000000000002,"15min":0.35599999999999998},"min":{"1min":0.22500000000000001,"5min":0.224,"15min":0.20899999999999999},"max":{"1min":0.441,"5min":0.65500000000000003,"15min":1.194},"last":0.26700000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.32700000000000001,"15min":0.35499999999999998},"min":{"1min":0.246,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.41699999999999998,"5min":0.51400000000000001,"15min":1.2869999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.35199999999999998,"15min":0.47299999999999998},"min":{"1min":0.23999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.49199999999999999,"5min":0.54200000000000004,"15min":5.0919999999999996},"last":0.37}]},{"osd":3,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.374,"15min":0.496},"min":{"1min":0.25600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54800000000000004,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.36299999999999999,"15min":0.39800000000000002},"min":{"1min":0.26500000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.52600000000000002,"5min":0.57799999999999996,"15min":1.2170000000000001},"last":0.54300000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.78600000000000003,"5min":0.92400000000000004,"15min":1.254},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55900000000000005,"15min":0.67500000000000004},"min":{"1min":0.39600000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68100000000000005,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.499}]},{"osd":5,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.40400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.97599999999999998,"15min":1.1799999999999999},"last":0.433},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56999999999999995,"15min":0.59999999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73299999999999998,"5min":0.878,"15min":1.3109999999999999},"last":0.66100000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":0.79300000000000004},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70399999999999996,"5min":0.995,"15min":5.0659999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60699999999999998,"15min":0.628},"min":{"1min":0.46400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84199999999999997,"5min":0.89800000000000002,"15min":1.171},"last":0.51000000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.48299999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77100000000000002,"5min":1.004,"15min":1.214},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58399999999999996,"15min":0.69699999999999995},"min":{"1min":0.40899999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.754,"5min":0.90500000000000003,"15min":4.0499999999999998},"last":0.78200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574952,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303980,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067860,"statfs":{"total":96636764160,"available":96325488640,"internally_reserved":0,"allocated":2658304,"data_stored":1513049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.315,"15min":0.48599999999999999},"min":{"1min":0.217,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":7.9249999999999998},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.46899999999999997},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46100000000000002,"5min":0.53800000000000003,"15min":6.6299999999999999},"last":0.38600000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33900000000000002,"15min":0.51000000000000001},"min":{"1min":0.25600000000000001,"5min":0.22,"15min":0.216},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":8.2509999999999994},"last":0.374},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.49299999999999999},"min":{"1min":0.27500000000000002,"5min":0.223,"15min":0.222},"max":{"1min":0.498,"5min":0.57899999999999996,"15min":6.7370000000000001},"last":0.42199999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.35499999999999998,"15min":0.51700000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.21099999999999999},"max":{"1min":0.438,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.57099999999999995},"min":{"1min":0.317,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":7.8929999999999998},"last":0.46100000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57099999999999995,"15min":0.68700000000000006},"min":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.78100000000000003,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57199999999999995,"15min":0.73099999999999998},"min":{"1min":0.371,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.69599999999999995,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.55400000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56499999999999995,"15min":0.71399999999999997},"min":{"1min":0.40100000000000002,"5min":0.376,"15min":0.36299999999999999},"max":{"1min":0.72699999999999998,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.74199999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.59299999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61499999999999999,"15min":0.77600000000000002},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.73299999999999998},"min":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.36599999999999999},"max":{"1min":0.82099999999999995,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.58199999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60899999999999999,"15min":0.72399999999999998},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.36299999999999999},"max":{"1min":0.85499999999999998,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59799999999999998,"15min":0.747},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89500000000000002,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.53600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303392,"kb_used_data":2008,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068448,"statfs":{"total":96636764160,"available":96326090752,"internally_reserved":0,"allocated":2056192,"data_stored":917379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.311,"15min":0.32400000000000001},"min":{"1min":0.20200000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.48599999999999999,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32100000000000001,"15min":0.34899999999999998},"min":{"1min":0.224,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51800000000000002,"5min":0.54200000000000004,"15min":0.746},"last":0.33000000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33200000000000002,"15min":0.65200000000000002},"min":{"1min":0.22600000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52200000000000002,"5min":0.63,"15min":11.967000000000001},"last":0.31},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.35199999999999998,"15min":0.45800000000000002},"min":{"1min":0.245,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.505,"5min":0.52700000000000002,"15min":4.1100000000000003},"last":0.28799999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.51600000000000001},"min":{"1min":0.23699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":6.3040000000000003},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.38700000000000001,"15min":0.70399999999999996},"min":{"1min":0.29499999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57799999999999996,"5min":0.65900000000000003,"15min":11.935},"last":0.42799999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.42999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91900000000000004,"5min":1.302,"15min":1.302},"last":0.63900000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.378},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.95699999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.39700000000000002,"5min":0.35999999999999999,"15min":0.34300000000000003},"max":{"1min":0.78800000000000003,"5min":0.83999999999999997,"15min":0.96099999999999997},"last":0.69999999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.33400000000000002,"15min":0.33200000000000002},"max":{"1min":0.95499999999999996,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60399999999999998,"15min":0.60499999999999998},"min":{"1min":0.47499999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.57799999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.46100000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85699999999999998,"5min":0.89700000000000002,"15min":1.052},"last":0.629},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58899999999999997,"15min":0.746},"min":{"1min":0.40999999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.98099999999999998,"5min":1.274,"15min":6.2649999999999997},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303404,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068436,"statfs":{"total":96636764160,"available":96326078464,"internally_reserved":0,"allocated":2068480,"data_stored":923642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.309,"15min":0.314},"min":{"1min":0.22600000000000001,"5min":0.21299999999999999,"15min":0.21099999999999999},"max":{"1min":0.38900000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.316,"5min":0.32700000000000001,"15min":0.32100000000000001},"min":{"1min":0.23799999999999999,"5min":0.216,"15min":0.19800000000000001},"max":{"1min":0.42799999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.64100000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.34399999999999997,"15min":0.33400000000000002},"min":{"1min":0.26200000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.47299999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38300000000000001,"15min":0.378},"min":{"1min":0.26800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.48599999999999999,"5min":0.66900000000000004,"15min":0.72099999999999997},"last":0.748}]},{"osd":2,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.36299999999999999,"15min":0.35299999999999998},"min":{"1min":0.28899999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44800000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.28799999999999998,"5min":0.25900000000000001,"15min":0.255},"max":{"1min":0.441,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.51000000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56100000000000005,"15min":0.53800000000000003},"min":{"1min":0.44900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56699999999999995,"15min":0.53900000000000003},"min":{"1min":0.433,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.79400000000000004,"5min":0.83299999999999996,"15min":0.93500000000000005},"last":0.70999999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.55300000000000005},"min":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.35899999999999999},"max":{"1min":0.69599999999999995,"5min":1.347,"15min":1.347},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.443,"5min":0.39300000000000002,"15min":0.35799999999999998},"max":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":1.01},"last":0.72299999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.45500000000000002,"5min":0.41099999999999998,"15min":0.36699999999999999},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.48899999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79700000000000004,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.61799999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61599999999999999,"15min":0.59599999999999997},"min":{"1min":0.495,"5min":0.42099999999999999,"15min":0.38200000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.96999999999999997},"last":0.66400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303392,"kb_used_data":2008,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068448,"statfs":{"total":96636764160,"available":96326090752,"internally_reserved":0,"allocated":2056192,"data_stored":917379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.372,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66700000000000004,"5min":0.74199999999999999,"15min":0.747},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52500000000000002,"15min":0.71899999999999997},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.32700000000000001},"max":{"1min":0.68300000000000005,"5min":0.77600000000000002,"15min":7.6719999999999997},"last":0.51500000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52600000000000002,"15min":0.72399999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.755,"15min":7.7160000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.371,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":0.86799999999999999},"last":0.624}]},{"osd":2,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67800000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.503},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.749},"min":{"1min":0.39900000000000002,"5min":0.39700000000000002,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":7.7839999999999998},"last":0.58199999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.47999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74399999999999999,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.42999999999999999,"5min":0.39200000000000002,"15min":0.35499999999999998},"max":{"1min":0.71499999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59399999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.497},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.30099999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.54700000000000004},"min":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.42699999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.35399999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.77300000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86199999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.52700000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.378,"15min":0.36199999999999999},"max":{"1min":0.88600000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.41699999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.874,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.54900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303404,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068436,"statfs":{"total":96636764160,"available":96326078464,"internally_reserved":0,"allocated":2068480,"data_stored":923642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.47599999999999998,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.66100000000000003,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.51800000000000002},"min":{"1min":0.33000000000000002,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.68899999999999995,"5min":0.76700000000000002,"15min":0.79300000000000004},"last":0.436}]},{"osd":1,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51300000000000001,"15min":0.628},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63400000000000001,"5min":0.75700000000000001,"15min":5.2679999999999998},"last":0.376},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.62},"min":{"1min":0.40500000000000003,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79200000000000004,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.56399999999999995}]},{"osd":2,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52400000000000002,"15min":0.63700000000000001},"min":{"1min":0.39100000000000001,"5min":0.32900000000000001,"15min":0.28299999999999997},"max":{"1min":0.73299999999999998,"5min":0.76600000000000001,"15min":5.2960000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.38500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.5}]},{"osd":3,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55900000000000005,"15min":0.66900000000000004},"min":{"1min":0.41999999999999998,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":5.3330000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56399999999999995,"15min":0.68000000000000005},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.27700000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.57699999999999996}]},{"osd":4,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.48899999999999999,"15min":0.498},"min":{"1min":0.33400000000000002,"5min":0.29599999999999999,"15min":0.28699999999999998},"max":{"1min":0.67000000000000004,"5min":0.72499999999999998,"15min":0.82999999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.34499999999999997,"15min":0.32200000000000001},"max":{"1min":0.77200000000000002,"5min":0.88,"15min":1.0800000000000001},"last":0.59299999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.628},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.39900000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78900000000000003,"5min":0.82499999999999996,"15min":0.89800000000000002},"last":0.65900000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.32300000000000001,"15min":0.30099999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98399999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.67200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:42:58.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.3f is not active+clean 2023-10-10T20:42:58.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.323231+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:22:40.796551+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:42:58.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.3e is not active+clean 2023-10-10T20:42:58.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.325806+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:28:12.284338+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:42:58.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.3d is not active+clean 2023-10-10T20:42:58.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.331625+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:09:34.459455+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:42:58.147 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.3c is not active+clean 2023-10-10T20:42:58.147 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.330945+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:16:50.053827+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:42:58.147 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.3b is not active+clean 2023-10-10T20:42:58.148 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.329837+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:16:07.451863+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:42:58.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.3a is not active+clean 2023-10-10T20:42:58.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.327658+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:58:12.432427+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:42:58.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.39 is not active+clean 2023-10-10T20:42:58.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.326884+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:36:30.295360+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:42:58.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.38 is not active+clean 2023-10-10T20:42:58.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.322678+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:34:59.692515+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:42:58.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.37 is not active+clean 2023-10-10T20:42:58.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.329758+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:02:55.597436+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:42:58.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.36 is not active+clean 2023-10-10T20:42:58.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.327137+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:32:08.240124+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:42:58.152 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.35 is not active+clean 2023-10-10T20:42:58.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.323857+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:48:32.579274+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:42:58.153 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.34 is not active+clean 2023-10-10T20:42:58.153 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.325054+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:36:30.295360+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:42:58.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.33 is not active+clean 2023-10-10T20:42:58.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.324653+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:54:03.524684+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:42:58.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.32 is not active+clean 2023-10-10T20:42:58.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.329133+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:41:28.090635+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:42:58.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.31 is not active+clean 2023-10-10T20:42:58.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.329269+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:38:16.787078+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:42:58.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.30 is not active+clean 2023-10-10T20:42:58.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.324787+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:58:32.593330+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:42:58.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.2f is not active+clean 2023-10-10T20:42:58.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.328589+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:53:35.674118+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:42:58.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.2e is not active+clean 2023-10-10T20:42:58.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.326650+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:38:30.259757+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:42:58.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.2d is not active+clean 2023-10-10T20:42:58.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.334129+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:20:38.627889+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:42:58.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.2c is not active+clean 2023-10-10T20:42:58.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.327487+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:21:27.171367+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:42:58.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.2b is not active+clean 2023-10-10T20:42:58.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.329421+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:18:18.498552+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:42:58.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.2a is not active+clean 2023-10-10T20:42:58.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.330301+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:00:48.564823+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:42:58.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.29 is not active+clean 2023-10-10T20:42:58.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.323234+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:58:00.782955+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:42:58.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.28 is not active+clean 2023-10-10T20:42:58.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.328346+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:02:55.597436+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:42:58.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.27 is not active+clean 2023-10-10T20:42:58.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.322747+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:25:50.618140+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:42:58.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.26 is not active+clean 2023-10-10T20:42:58.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.322798+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:04:51.509487+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:42:58.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.25 is not active+clean 2023-10-10T20:42:58.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.323414+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:20:34.093326+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:42:58.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.24 is not active+clean 2023-10-10T20:42:58.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.326676+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:33:29.535005+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:42:58.166 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.23 is not active+clean 2023-10-10T20:42:58.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.328679+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:50:07.909344+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:42:58.167 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.22 is not active+clean 2023-10-10T20:42:58.167 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.322817+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:44:40.439820+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:42:58.168 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.21 is not active+clean 2023-10-10T20:42:58.168 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.330007+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:22:02.852226+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:42:58.168 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.20 is not active+clean 2023-10-10T20:42:58.169 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.329533+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:13:20.878853+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:42:58.169 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.1e is not active+clean 2023-10-10T20:42:58.169 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.328750+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:34:46.022458+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:42:58.170 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.1f is not active+clean 2023-10-10T20:42:58.170 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.322742+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:39:23.346373+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:42:58.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.1d is not active+clean 2023-10-10T20:42:58.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.329028+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:00:56.184401+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:42:58.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.1a is not active+clean 2023-10-10T20:42:58.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.322141+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:45:05.557928+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:42:58.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.1b is not active+clean 2023-10-10T20:42:58.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.328114+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:49:00.861926+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:42:58.173 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.18 is not active+clean 2023-10-10T20:42:58.173 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.327089+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:12:52.463061+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:42:58.174 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.19 is not active+clean 2023-10-10T20:42:58.174 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.324299+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:17:23.703124+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:42:58.174 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.16 is not active+clean 2023-10-10T20:42:58.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.325326+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:30:49.951335+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:42:58.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.17 is not active+clean 2023-10-10T20:42:58.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.322179+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:13:45.883755+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:42:58.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.14 is not active+clean 2023-10-10T20:42:58.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.328250+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:49:31.176991+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:42:58.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.15 is not active+clean 2023-10-10T20:42:58.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.322090+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:17:25.969866+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:42:58.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.12 is not active+clean 2023-10-10T20:42:58.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.329839+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:18:15.714890+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:42:58.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.13 is not active+clean 2023-10-10T20:42:58.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.333958+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:20:38.627889+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:42:58.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.10 is not active+clean 2023-10-10T20:42:58.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.331014+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:05:24.092738+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:42:58.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.11 is not active+clean 2023-10-10T20:42:58.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.328957+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:46:36.668645+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:42:58.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.e is not active+clean 2023-10-10T20:42:58.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.322365+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:04:02.591369+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:42:58.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.f is not active+clean 2023-10-10T20:42:58.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.326616+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:57:12.184703+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:42:58.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.c is not active+clean 2023-10-10T20:42:58.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.327959+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:58:32.593330+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:42:58.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.d is not active+clean 2023-10-10T20:42:58.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.331151+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:18:18.498552+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:42:58.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.a is not active+clean 2023-10-10T20:42:58.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.328162+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:34:46.022458+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:42:58.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.b is not active+clean 2023-10-10T20:42:58.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.333557+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:16:50.053827+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:42:58.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.8 is not active+clean 2023-10-10T20:42:58.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.322254+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:21:17.789832+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:42:58.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.9 is not active+clean 2023-10-10T20:42:58.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.329553+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:09:34.459455+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:42:58.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.6 is not active+clean 2023-10-10T20:42:58.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.330288+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:28:12.284338+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:42:58.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.7 is not active+clean 2023-10-10T20:42:58.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.327181+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:30:49.951335+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:42:58.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.4 is not active+clean 2023-10-10T20:42:58.189 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.328435+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:49:00.861926+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:42:58.189 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.5 is not active+clean 2023-10-10T20:42:58.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.328547+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:05:24.092738+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:42:58.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.2 is not active+clean 2023-10-10T20:42:58.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.324346+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:32:49.995435+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:42:58.191 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.3 is not active+clean 2023-10-10T20:42:58.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.333078+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:22:02.852226+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:42:58.192 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.0 is not active+clean 2023-10-10T20:42:58.192 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.328569+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:28:12.284338+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:42:58.192 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.1 is not active+clean 2023-10-10T20:42:58.193 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.322090+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:38:10.362086+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:42:58.193 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.1c is not active+clean 2023-10-10T20:42:58.193 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:42:52.314972+0000', 'last_change': '2023-10-10T20:42:52.322090+0000', 'last_active': '2023-10-10T20:42:52.314972+0000', 'last_peered': '2023-10-10T20:42:52.314972+0000', 'last_clean': '2023-10-10T20:42:52.314972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:42:52.314972+0000', 'last_undegraded': '2023-10-10T20:42:52.314972+0000', 'last_fullsized': '2023-10-10T20:42:52.314972+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'last_clean_scrub_stamp': '2023-10-10T20:42:52.314972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:49:35.275237+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:42:58.194 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-10T20:42:58.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.306+0000 7faefeb52700 1 Processor -- start 2023-10-10T20:42:58.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.306+0000 7faefeb52700 1 -- start start 2023-10-10T20:42:58.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.307+0000 7faefeb52700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faef80cd910 0x7faef80ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:58.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.307+0000 7faefeb52700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faef80b55a0 0x7faef80ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:58.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.307+0000 7faefeb52700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef80b5b00 con 0x7faef8159ec0 2023-10-10T20:42:58.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.307+0000 7faefeb52700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef80b5da0 con 0x7faef80cd910 2023-10-10T20:42:58.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.307+0000 7faefeb52700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef80b6080 con 0x7faef80b55a0 2023-10-10T20:42:58.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.307+0000 7faef7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faef80cd910 0x7faef80ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:58.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.307+0000 7faef7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faef80cd910 0x7faef80ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47380/0 (socket says 172.21.15.89:47380) 2023-10-10T20:42:58.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.307+0000 7faef7fff700 1 -- 172.21.15.89:0/253265618 learned_addr learned my addr 172.21.15.89:0/253265618 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:58.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.307+0000 7faef7fff700 1 -- 172.21.15.89:0/253265618 >> v2:172.21.15.111:3300/0 conn(0x7faef80b55a0 msgr2=0x7faef80ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:58.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.307+0000 7faef7fff700 1 --2- 172.21.15.89:0/253265618 >> v2:172.21.15.111:3300/0 conn(0x7faef80b55a0 0x7faef80ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.307+0000 7faef7fff700 1 -- 172.21.15.89:0/253265618 >> v1:172.21.15.111:6789/0 conn(0x7faef8159ec0 legacy=0x7faef80b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:58.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.307+0000 7faef7fff700 1 -- 172.21.15.89:0/253265618 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef80cc100 con 0x7faef80cd910 2023-10-10T20:42:58.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.307+0000 7faef7fff700 1 --2- 172.21.15.89:0/253265618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faef80cd910 0x7faef80ce340 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7faee80099b0 tx=0x7faee8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf22a3d53a07e4e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:58.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.307+0000 7faef77fe700 1 -- 172.21.15.89:0/253265618 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee800b7e0 con 0x7faef80cd910 2023-10-10T20:42:58.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.307+0000 7faef77fe700 1 -- 172.21.15.89:0/253265618 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee800b940 con 0x7faef80cd910 2023-10-10T20:42:58.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.307+0000 7faef77fe700 1 -- 172.21.15.89:0/253265618 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee800bce0 con 0x7faef80cd910 2023-10-10T20:42:58.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.308+0000 7faefeb52700 1 -- 172.21.15.89:0/253265618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faef80cd910 msgr2=0x7faef80ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:58.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.308+0000 7faefeb52700 1 --2- 172.21.15.89:0/253265618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faef80cd910 0x7faef80ce340 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7faee80099b0 tx=0x7faee8009250 comp rx=0 tx=0).stop 2023-10-10T20:42:58.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.308+0000 7faefeb52700 1 -- 172.21.15.89:0/253265618 shutdown_connections 2023-10-10T20:42:58.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.308+0000 7faefeb52700 1 --2- 172.21.15.89:0/253265618 >> v2:172.21.15.111:3300/0 conn(0x7faef80b55a0 0x7faef80ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.308+0000 7faefeb52700 1 --2- 172.21.15.89:0/253265618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faef80cd910 0x7faef80ce340 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.308+0000 7faefeb52700 1 -- 172.21.15.89:0/253265618 >> 172.21.15.89:0/253265618 conn(0x7faef80999f0 msgr2=0x7faef80b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:58.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.308+0000 7faefeb52700 1 -- 172.21.15.89:0/253265618 shutdown_connections 2023-10-10T20:42:58.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.308+0000 7faefeb52700 1 -- 172.21.15.89:0/253265618 wait complete. 2023-10-10T20:42:58.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.309+0000 7faefeb52700 1 Processor -- start 2023-10-10T20:42:58.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.309+0000 7faefeb52700 1 -- start start 2023-10-10T20:42:58.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.310+0000 7faefeb52700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faef80b55a0 0x7faef8153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:58.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.310+0000 7faefeb52700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faef8159ec0 0x7faef80b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:58.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.310+0000 7faefeb52700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faef80b3a70 0x7faef8154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:58.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.310+0000 7faefeb52700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef80b3ed0 con 0x7faef80b3a70 2023-10-10T20:42:58.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.310+0000 7faefeb52700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef813bcc0 con 0x7faef80b55a0 2023-10-10T20:42:58.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.310+0000 7faefeb52700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef8069450 con 0x7faef8159ec0 2023-10-10T20:42:58.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.310+0000 7faefd0ef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faef80b3a70 0x7faef8154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:58.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.310+0000 7faefd0ef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faef80b3a70 0x7faef8154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47390/0 (socket says 172.21.15.89:47390) 2023-10-10T20:42:58.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.310+0000 7faefd0ef700 1 -- 172.21.15.89:0/240062044 learned_addr learned my addr 172.21.15.89:0/240062044 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:58.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.310+0000 7faef7fff700 1 --2- 172.21.15.89:0/240062044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faef8159ec0 0x7faef80b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:58.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.310+0000 7faefc8ee700 1 --2- 172.21.15.89:0/240062044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faef80b55a0 0x7faef8153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:58.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.310+0000 7faefd0ef700 1 -- 172.21.15.89:0/240062044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faef8159ec0 msgr2=0x7faef80b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:58.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.310+0000 7faefd0ef700 1 --2- 172.21.15.89:0/240062044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faef8159ec0 0x7faef80b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.310+0000 7faefd0ef700 1 -- 172.21.15.89:0/240062044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faef80b55a0 msgr2=0x7faef8153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:58.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.310+0000 7faefd0ef700 1 --2- 172.21.15.89:0/240062044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faef80b55a0 0x7faef8153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.310+0000 7faefd0ef700 1 -- 172.21.15.89:0/240062044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee80094f0 con 0x7faef80b3a70 2023-10-10T20:42:58.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.310+0000 7faefd0ef700 1 --2- 172.21.15.89:0/240062044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faef80b3a70 0x7faef8154df0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7faee000b5c0 tx=0x7faee0006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:58.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.311+0000 7faef4ff9700 1 -- 172.21.15.89:0/240062044 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee000f510 con 0x7faef80b3a70 2023-10-10T20:42:58.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.311+0000 7faef4ff9700 1 -- 172.21.15.89:0/240062044 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee000f670 con 0x7faef80b3a70 2023-10-10T20:42:58.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.311+0000 7faefeb52700 1 -- 172.21.15.89:0/240062044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef8155e70 con 0x7faef80b3a70 2023-10-10T20:42:58.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.311+0000 7faef4ff9700 1 -- 172.21.15.89:0/240062044 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee000fa30 con 0x7faef80b3a70 2023-10-10T20:42:58.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.311+0000 7faefeb52700 1 -- 172.21.15.89:0/240062044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef8068e00 con 0x7faef80b3a70 2023-10-10T20:42:58.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.311+0000 7faefeb52700 1 -- 172.21.15.89:0/240062044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef804c3c0 con 0x7faef80b3a70 2023-10-10T20:42:58.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.312+0000 7faef4ff9700 1 -- 172.21.15.89:0/240062044 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7faee001a400 con 0x7faef80b3a70 2023-10-10T20:42:58.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.312+0000 7faef4ff9700 1 --2- 172.21.15.89:0/240062044 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faed8067320 0x7faed80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:58.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.312+0000 7faef4ff9700 1 -- 172.21.15.89:0/240062044 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faee00845c0 con 0x7faef80b3a70 2023-10-10T20:42:58.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.312+0000 7faefc8ee700 1 --2- 172.21.15.89:0/240062044 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faed8067320 0x7faed80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:58.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.313+0000 7faefc8ee700 1 --2- 172.21.15.89:0/240062044 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faed8067320 0x7faed80697d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7faef00075c0 tx=0x7faef0009780 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:58.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.316+0000 7faef4ff9700 1 -- 172.21.15.89:0/240062044 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faee00512d0 con 0x7faef80b3a70 2023-10-10T20:42:58.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.583+0000 7faefeb52700 1 -- 172.21.15.89:0/240062044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7faef8063c40 con 0x7faef80b3a70 2023-10-10T20:42:58.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.583+0000 7faef4ff9700 1 -- 172.21.15.89:0/240062044 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1460 (secure 0 0 0) 0x7faee00548f0 con 0x7faef80b3a70 2023-10-10T20:42:58.585 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:58.586 INFO:teuthology.orchestra.run.smithi089.stdout:{"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":521,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":165,"num_osds":8,"num_up_osds":8,"osd_up_since":1696970067,"num_in_osds":8,"osd_in_since":1696970059,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2486644736,"bytes_avail":770607468544,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":110,"id":14,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":14,"rank":0,"name":"b","status":"up:active","gid":8337}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2023-10-10T20:42:42.748954+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-10T20:42:58.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.585+0000 7faefeb52700 1 -- 172.21.15.89:0/240062044 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faed8067320 msgr2=0x7faed80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:58.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.585+0000 7faefeb52700 1 --2- 172.21.15.89:0/240062044 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faed8067320 0x7faed80697d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7faef00075c0 tx=0x7faef0009780 comp rx=0 tx=0).stop 2023-10-10T20:42:58.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.585+0000 7faefeb52700 1 -- 172.21.15.89:0/240062044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faef80b3a70 msgr2=0x7faef8154df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:58.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.585+0000 7faefeb52700 1 --2- 172.21.15.89:0/240062044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faef80b3a70 0x7faef8154df0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7faee000b5c0 tx=0x7faee0006f90 comp rx=0 tx=0).stop 2023-10-10T20:42:58.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.585+0000 7faefeb52700 1 -- 172.21.15.89:0/240062044 shutdown_connections 2023-10-10T20:42:58.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.585+0000 7faefeb52700 1 --2- 172.21.15.89:0/240062044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faef80b3a70 0x7faef8154df0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.585+0000 7faefeb52700 1 --2- 172.21.15.89:0/240062044 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7faed8067320 0x7faed80697d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.585+0000 7faefeb52700 1 --2- 172.21.15.89:0/240062044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faef80b55a0 0x7faef8153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.585+0000 7faefeb52700 1 --2- 172.21.15.89:0/240062044 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7faef8159ec0 0x7faef80b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.585+0000 7faefeb52700 1 -- 172.21.15.89:0/240062044 >> 172.21.15.89:0/240062044 conn(0x7faef80999f0 msgr2=0x7faef81523f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:58.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.585+0000 7faefeb52700 1 -- 172.21.15.89:0/240062044 shutdown_connections 2023-10-10T20:42:58.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.585+0000 7faefeb52700 1 -- 172.21.15.89:0/240062044 wait complete. 2023-10-10T20:42:58.599 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-10T20:42:58.600 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:42:58.618 INFO:tasks.ceph.mds.b.smithi111.stderr:2023-10-10T20:42:58.617+0000 7fb5867a5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:42:58.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.713+0000 7efd0d856700 1 Processor -- start 2023-10-10T20:42:58.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.714+0000 7efd0d856700 1 -- start start 2023-10-10T20:42:58.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.714+0000 7efd0d856700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd08159c60 0x7efd0815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:58.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.714+0000 7efd0d856700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efd0815db10 0x7efd08069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:58.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.714+0000 7efd0d856700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0815a160 con 0x7efd08159890 2023-10-10T20:42:58.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.714+0000 7efd0d856700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd080d13a0 con 0x7efd0815db10 2023-10-10T20:42:58.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.714+0000 7efd0d856700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd0815e140 con 0x7efd08159c60 2023-10-10T20:42:58.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.714+0000 7efd077fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efd0815db10 0x7efd08069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:58.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.714+0000 7efd06ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7efd08159890 0x7efd0815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38376/0 (socket says 172.21.15.89:38376) 2023-10-10T20:42:58.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.715+0000 7efd06ffd700 1 -- 172.21.15.89:0/2420339164 learned_addr learned my addr 172.21.15.89:0/2420339164 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:42:58.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.715+0000 7efd077fe700 1 -- 172.21.15.89:0/2420339164 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd08159c60 msgr2=0x7efd0815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:42:58.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.715+0000 7efd077fe700 1 --2- 172.21.15.89:0/2420339164 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd08159c60 0x7efd0815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.715+0000 7efd077fe700 1 -- 172.21.15.89:0/2420339164 >> v1:172.21.15.89:6789/0 conn(0x7efd08159890 legacy=0x7efd0815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:58.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.715+0000 7efd067fc700 1 -- 172.21.15.89:0/2420339164 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868576343 0 0) 0x7efd0815a160 con 0x7efd08159890 2023-10-10T20:42:58.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.715+0000 7efd077fe700 1 -- 172.21.15.89:0/2420339164 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0815a3d0 con 0x7efd0815db10 2023-10-10T20:42:58.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.715+0000 7efd077fe700 1 --2- 172.21.15.89:0/2420339164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0815db10 0x7efd08069560 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7efd00009e60 tx=0x7efd00009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfc869643defff3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:58.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.715+0000 7efd067fc700 1 -- 172.21.15.89:0/2420339164 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0000e070 con 0x7efd0815db10 2023-10-10T20:42:58.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.715+0000 7efd067fc700 1 -- 172.21.15.89:0/2420339164 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0000bc30 con 0x7efd0815db10 2023-10-10T20:42:58.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.715+0000 7efd067fc700 1 -- 172.21.15.89:0/2420339164 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd00012430 con 0x7efd0815db10 2023-10-10T20:42:58.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.716+0000 7efd0d856700 1 -- 172.21.15.89:0/2420339164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0815db10 msgr2=0x7efd08069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:58.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.716+0000 7efd0d856700 1 --2- 172.21.15.89:0/2420339164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0815db10 0x7efd08069560 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7efd00009e60 tx=0x7efd00009780 comp rx=0 tx=0).stop 2023-10-10T20:42:58.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.716+0000 7efd0d856700 1 -- 172.21.15.89:0/2420339164 shutdown_connections 2023-10-10T20:42:58.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.716+0000 7efd0d856700 1 --2- 172.21.15.89:0/2420339164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0815db10 0x7efd08069560 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.716+0000 7efd0d856700 1 --2- 172.21.15.89:0/2420339164 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd08159c60 0x7efd0815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.716+0000 7efd0d856700 1 -- 172.21.15.89:0/2420339164 >> 172.21.15.89:0/2420339164 conn(0x7efd08068e20 msgr2=0x7efd080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:58.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.716+0000 7efd0d856700 1 -- 172.21.15.89:0/2420339164 shutdown_connections 2023-10-10T20:42:58.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.716+0000 7efd0d856700 1 -- 172.21.15.89:0/2420339164 wait complete. 2023-10-10T20:42:58.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.717+0000 7efd0d856700 1 Processor -- start 2023-10-10T20:42:58.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.717+0000 7efd0d856700 1 -- start start 2023-10-10T20:42:58.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efd0d856700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd08159890 0x7efd0814c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:58.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efd0d856700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd08159c60 0x7efd08156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:58.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efd0d856700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0815db10 0x7efd081573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:58.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efd0d856700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd08151aa0 con 0x7efd0815db10 2023-10-10T20:42:58.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efd0d856700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0814c9b0 con 0x7efd08159890 2023-10-10T20:42:58.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efd0d856700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd081511e0 con 0x7efd08159c60 2023-10-10T20:42:58.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efd077fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0815db10 0x7efd081573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:58.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efd077fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0815db10 0x7efd081573d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47416/0 (socket says 172.21.15.89:47416) 2023-10-10T20:42:58.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efd077fe700 1 -- 172.21.15.89:0/1601576586 learned_addr learned my addr 172.21.15.89:0/1601576586 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:42:58.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efcfe7fc700 1 --2- 172.21.15.89:0/1601576586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd08159c60 0x7efd08156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:58.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efd06ffd700 1 --2- 172.21.15.89:0/1601576586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd08159890 0x7efd0814c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:58.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efd077fe700 1 -- 172.21.15.89:0/1601576586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd08159c60 msgr2=0x7efd08156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:58.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efd077fe700 1 --2- 172.21.15.89:0/1601576586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd08159c60 0x7efd08156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efd077fe700 1 -- 172.21.15.89:0/1601576586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd08159890 msgr2=0x7efd0814c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:58.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efd077fe700 1 --2- 172.21.15.89:0/1601576586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd08159890 0x7efd0814c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efd077fe700 1 -- 172.21.15.89:0/1601576586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd00009a20 con 0x7efd0815db10 2023-10-10T20:42:58.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efd077fe700 1 --2- 172.21.15.89:0/1601576586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0815db10 0x7efd081573d0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7efd080899c0 tx=0x7efd00019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:58.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efcff7fe700 1 -- 172.21.15.89:0/1601576586 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0001d630 con 0x7efd0815db10 2023-10-10T20:42:58.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efcff7fe700 1 -- 172.21.15.89:0/1601576586 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0001d790 con 0x7efd0815db10 2023-10-10T20:42:58.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efd0d856700 1 -- 172.21.15.89:0/1601576586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd08151450 con 0x7efd0815db10 2023-10-10T20:42:58.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efcff7fe700 1 -- 172.21.15.89:0/1601576586 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0001db50 con 0x7efd0815db10 2023-10-10T20:42:58.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.718+0000 7efd0d856700 1 -- 172.21.15.89:0/1601576586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd080cfdd0 con 0x7efd0815db10 2023-10-10T20:42:58.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.719+0000 7efd0d856700 1 -- 172.21.15.89:0/1601576586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd0804c3c0 con 0x7efd0815db10 2023-10-10T20:42:58.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.720+0000 7efcff7fe700 1 -- 172.21.15.89:0/1601576586 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7efd00024070 con 0x7efd0815db10 2023-10-10T20:42:58.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.720+0000 7efcff7fe700 1 --2- 172.21.15.89:0/1601576586 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efce8067320 0x7efce80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:42:58.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.720+0000 7efcff7fe700 1 -- 172.21.15.89:0/1601576586 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efd00087550 con 0x7efd0815db10 2023-10-10T20:42:58.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.720+0000 7efd06ffd700 1 --2- 172.21.15.89:0/1601576586 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efce8067320 0x7efce80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:42:58.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.720+0000 7efd06ffd700 1 --2- 172.21.15.89:0/1601576586 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efce8067320 0x7efce80697d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7efcf80011e0 tx=0x7efcf8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:42:58.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.724+0000 7efcff7fe700 1 -- 172.21.15.89:0/1601576586 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd00054290 con 0x7efd0815db10 2023-10-10T20:42:58.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.904+0000 7efd0d856700 1 -- 172.21.15.89:0/1601576586 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd08151be0 con 0x7efce8067320 2023-10-10T20:42:58.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.912+0000 7efcff7fe700 1 -- 172.21.15.89:0/1601576586 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289930 (secure 0 0 0) 0x7efd08151be0 con 0x7efce8067320 2023-10-10T20:42:58.918 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:42:58.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.915+0000 7efd0d856700 1 -- 172.21.15.89:0/1601576586 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efce8067320 msgr2=0x7efce80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:58.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.915+0000 7efd0d856700 1 --2- 172.21.15.89:0/1601576586 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efce8067320 0x7efce80697d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7efcf80011e0 tx=0x7efcf8009040 comp rx=0 tx=0).stop 2023-10-10T20:42:58.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.915+0000 7efd0d856700 1 -- 172.21.15.89:0/1601576586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0815db10 msgr2=0x7efd081573d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:42:58.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.915+0000 7efd0d856700 1 --2- 172.21.15.89:0/1601576586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0815db10 0x7efd081573d0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7efd080899c0 tx=0x7efd00019000 comp rx=0 tx=0).stop 2023-10-10T20:42:58.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.915+0000 7efd0d856700 1 -- 172.21.15.89:0/1601576586 shutdown_connections 2023-10-10T20:42:58.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.915+0000 7efd0d856700 1 --2- 172.21.15.89:0/1601576586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0815db10 0x7efd081573d0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.915+0000 7efd0d856700 1 --2- 172.21.15.89:0/1601576586 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7efce8067320 0x7efce80697d0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.915+0000 7efd0d856700 1 --2- 172.21.15.89:0/1601576586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd08159890 0x7efd0814c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.915+0000 7efd0d856700 1 --2- 172.21.15.89:0/1601576586 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7efd08159c60 0x7efd08156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:42:58.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.916+0000 7efd0d856700 1 -- 172.21.15.89:0/1601576586 >> 172.21.15.89:0/1601576586 conn(0x7efd08068e20 msgr2=0x7efd0814d220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:42:58.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.916+0000 7efd0d856700 1 -- 172.21.15.89:0/1601576586 shutdown_connections 2023-10-10T20:42:58.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:42:58.916+0000 7efd0d856700 1 -- 172.21.15.89:0/1601576586 wait complete. 2023-10-10T20:42:58.923 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:42:58.929 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2023-10-10T20:42:58.751544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428644,"kb_used_data":17572,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752546076,"statfs":{"total":773094113280,"available":770607181824,"internally_reserved":0,"allocated":17993728,"data_stored":8677931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.429661"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.356154+0000","last_change":"2023-10-10T20:42:53.349889+0000","last_active":"2023-10-10T20:42:53.356154+0000","last_peered":"2023-10-10T20:42:53.356154+0000","last_clean":"2023-10-10T20:42:53.356154+0000","last_became_active":"2023-10-10T20:42:53.348786+0000","last_became_peered":"2023-10-10T20:42:53.348786+0000","last_unstale":"2023-10-10T20:42:53.356154+0000","last_undegraded":"2023-10-10T20:42:53.356154+0000","last_fullsized":"2023-10-10T20:42:53.356154+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:58:12.432427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.366952+0000","last_change":"2023-10-10T20:42:53.362906+0000","last_active":"2023-10-10T20:42:53.366952+0000","last_peered":"2023-10-10T20:42:53.366952+0000","last_clean":"2023-10-10T20:42:53.366952+0000","last_became_active":"2023-10-10T20:42:53.362294+0000","last_became_peered":"2023-10-10T20:42:53.362294+0000","last_unstale":"2023-10-10T20:42:53.366952+0000","last_undegraded":"2023-10-10T20:42:53.366952+0000","last_fullsized":"2023-10-10T20:42:53.366952+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:16:07.451863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.366851+0000","last_change":"2023-10-10T20:42:53.364479+0000","last_active":"2023-10-10T20:42:53.366851+0000","last_peered":"2023-10-10T20:42:53.366851+0000","last_clean":"2023-10-10T20:42:53.366851+0000","last_became_active":"2023-10-10T20:42:53.361443+0000","last_became_peered":"2023-10-10T20:42:53.361443+0000","last_unstale":"2023-10-10T20:42:53.366851+0000","last_undegraded":"2023-10-10T20:42:53.366851+0000","last_fullsized":"2023-10-10T20:42:53.366851+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:20:34.093326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.345401+0000","last_change":"2023-10-10T20:42:53.343259+0000","last_active":"2023-10-10T20:42:53.345401+0000","last_peered":"2023-10-10T20:42:53.345401+0000","last_clean":"2023-10-10T20:42:53.345401+0000","last_became_active":"2023-10-10T20:42:53.343002+0000","last_became_peered":"2023-10-10T20:42:53.343002+0000","last_unstale":"2023-10-10T20:42:53.345401+0000","last_undegraded":"2023-10-10T20:42:53.345401+0000","last_fullsized":"2023-10-10T20:42:53.345401+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:13:45.883755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.346024+0000","last_change":"2023-10-10T20:42:53.344218+0000","last_active":"2023-10-10T20:42:53.346024+0000","last_peered":"2023-10-10T20:42:53.346024+0000","last_clean":"2023-10-10T20:42:53.346024+0000","last_became_active":"2023-10-10T20:42:53.343971+0000","last_became_peered":"2023-10-10T20:42:53.343971+0000","last_unstale":"2023-10-10T20:42:53.346024+0000","last_undegraded":"2023-10-10T20:42:53.346024+0000","last_fullsized":"2023-10-10T20:42:53.346024+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:28:41.786703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.353207+0000","last_change":"2023-10-10T20:42:53.351313+0000","last_active":"2023-10-10T20:42:53.353207+0000","last_peered":"2023-10-10T20:42:53.353207+0000","last_clean":"2023-10-10T20:42:53.353207+0000","last_became_active":"2023-10-10T20:42:53.351064+0000","last_became_peered":"2023-10-10T20:42:53.351064+0000","last_unstale":"2023-10-10T20:42:53.353207+0000","last_undegraded":"2023-10-10T20:42:53.353207+0000","last_fullsized":"2023-10-10T20:42:53.353207+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:38:16.787078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.367121+0000","last_change":"2023-10-10T20:42:53.361375+0000","last_active":"2023-10-10T20:42:53.367121+0000","last_peered":"2023-10-10T20:42:53.367121+0000","last_clean":"2023-10-10T20:42:53.367121+0000","last_became_active":"2023-10-10T20:42:53.360948+0000","last_became_peered":"2023-10-10T20:42:53.360948+0000","last_unstale":"2023-10-10T20:42:53.367121+0000","last_undegraded":"2023-10-10T20:42:53.367121+0000","last_fullsized":"2023-10-10T20:42:53.367121+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:17:25.969866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.345885+0000","last_change":"2023-10-10T20:42:53.345087+0000","last_active":"2023-10-10T20:42:53.345885+0000","last_peered":"2023-10-10T20:42:53.345885+0000","last_clean":"2023-10-10T20:42:53.345885+0000","last_became_active":"2023-10-10T20:42:53.344838+0000","last_became_peered":"2023-10-10T20:42:53.344838+0000","last_unstale":"2023-10-10T20:42:53.345885+0000","last_undegraded":"2023-10-10T20:42:53.345885+0000","last_fullsized":"2023-10-10T20:42:53.345885+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:32.005980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365997+0000","last_change":"2023-10-10T20:42:53.358792+0000","last_active":"2023-10-10T20:42:53.365997+0000","last_peered":"2023-10-10T20:42:53.365997+0000","last_clean":"2023-10-10T20:42:53.365997+0000","last_became_active":"2023-10-10T20:42:53.358197+0000","last_became_peered":"2023-10-10T20:42:53.358197+0000","last_unstale":"2023-10-10T20:42:53.365997+0000","last_undegraded":"2023-10-10T20:42:53.365997+0000","last_fullsized":"2023-10-10T20:42:53.365997+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:45:05.557928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.348367+0000","last_change":"2023-10-10T20:42:53.347297+0000","last_active":"2023-10-10T20:42:53.348367+0000","last_peered":"2023-10-10T20:42:53.348367+0000","last_clean":"2023-10-10T20:42:53.348367+0000","last_became_active":"2023-10-10T20:42:53.347069+0000","last_became_peered":"2023-10-10T20:42:53.347069+0000","last_unstale":"2023-10-10T20:42:53.348367+0000","last_undegraded":"2023-10-10T20:42:53.348367+0000","last_fullsized":"2023-10-10T20:42:53.348367+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:18:15.714890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.356601+0000","last_change":"2023-10-10T20:42:53.348770+0000","last_active":"2023-10-10T20:42:53.356601+0000","last_peered":"2023-10-10T20:42:53.356601+0000","last_clean":"2023-10-10T20:42:53.356601+0000","last_became_active":"2023-10-10T20:42:53.348164+0000","last_became_peered":"2023-10-10T20:42:53.348164+0000","last_unstale":"2023-10-10T20:42:53.356601+0000","last_undegraded":"2023-10-10T20:42:53.356601+0000","last_fullsized":"2023-10-10T20:42:53.356601+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:18:15.714890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.359052+0000","last_change":"2023-10-10T20:42:53.345901+0000","last_active":"2023-10-10T20:42:53.359052+0000","last_peered":"2023-10-10T20:42:53.359052+0000","last_clean":"2023-10-10T20:42:53.359052+0000","last_became_active":"2023-10-10T20:42:53.345399+0000","last_became_peered":"2023-10-10T20:42:53.345399+0000","last_unstale":"2023-10-10T20:42:53.359052+0000","last_undegraded":"2023-10-10T20:42:53.359052+0000","last_fullsized":"2023-10-10T20:42:53.359052+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:02:55.597436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.361665+0000","last_change":"2023-10-10T20:42:53.355221+0000","last_active":"2023-10-10T20:42:53.361665+0000","last_peered":"2023-10-10T20:42:53.361665+0000","last_clean":"2023-10-10T20:42:53.361665+0000","last_became_active":"2023-10-10T20:42:53.354313+0000","last_became_peered":"2023-10-10T20:42:53.354313+0000","last_unstale":"2023-10-10T20:42:53.361665+0000","last_undegraded":"2023-10-10T20:42:53.361665+0000","last_fullsized":"2023-10-10T20:42:53.361665+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:05:24.092738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.353642+0000","last_change":"2023-10-10T20:42:53.351579+0000","last_active":"2023-10-10T20:42:53.353642+0000","last_peered":"2023-10-10T20:42:53.353642+0000","last_clean":"2023-10-10T20:42:53.353642+0000","last_became_active":"2023-10-10T20:42:53.351020+0000","last_became_peered":"2023-10-10T20:42:53.351020+0000","last_unstale":"2023-10-10T20:42:53.353642+0000","last_undegraded":"2023-10-10T20:42:53.353642+0000","last_fullsized":"2023-10-10T20:42:53.353642+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:39:39.656970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.363457+0000","last_change":"2023-10-10T20:42:53.362369+0000","last_active":"2023-10-10T20:42:53.363457+0000","last_peered":"2023-10-10T20:42:53.363457+0000","last_clean":"2023-10-10T20:42:53.363457+0000","last_became_active":"2023-10-10T20:42:53.362086+0000","last_became_peered":"2023-10-10T20:42:53.362086+0000","last_unstale":"2023-10-10T20:42:53.363457+0000","last_undegraded":"2023-10-10T20:42:53.363457+0000","last_fullsized":"2023-10-10T20:42:53.363457+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:34:03.701694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.361631+0000","last_change":"2023-10-10T20:42:53.354983+0000","last_active":"2023-10-10T20:42:53.361631+0000","last_peered":"2023-10-10T20:42:53.361631+0000","last_clean":"2023-10-10T20:42:53.361631+0000","last_became_active":"2023-10-10T20:42:53.346734+0000","last_became_peered":"2023-10-10T20:42:53.346734+0000","last_unstale":"2023-10-10T20:42:53.361631+0000","last_undegraded":"2023-10-10T20:42:53.361631+0000","last_fullsized":"2023-10-10T20:42:53.361631+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:46.022458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.353689+0000","last_change":"2023-10-10T20:42:53.351355+0000","last_active":"2023-10-10T20:42:53.353689+0000","last_peered":"2023-10-10T20:42:53.353689+0000","last_clean":"2023-10-10T20:42:53.353689+0000","last_became_active":"2023-10-10T20:42:53.351007+0000","last_became_peered":"2023-10-10T20:42:53.351007+0000","last_unstale":"2023-10-10T20:42:53.353689+0000","last_undegraded":"2023-10-10T20:42:53.353689+0000","last_fullsized":"2023-10-10T20:42:53.353689+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:58:16.268958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.362012+0000","last_change":"2023-10-10T20:42:53.361475+0000","last_active":"2023-10-10T20:42:53.362012+0000","last_peered":"2023-10-10T20:42:53.362012+0000","last_clean":"2023-10-10T20:42:53.362012+0000","last_became_active":"2023-10-10T20:42:53.361204+0000","last_became_peered":"2023-10-10T20:42:53.361204+0000","last_unstale":"2023-10-10T20:42:53.362012+0000","last_undegraded":"2023-10-10T20:42:53.362012+0000","last_fullsized":"2023-10-10T20:42:53.362012+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:00:48.564823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.358911+0000","last_change":"2023-10-10T20:42:53.357353+0000","last_active":"2023-10-10T20:42:53.358911+0000","last_peered":"2023-10-10T20:42:53.358911+0000","last_clean":"2023-10-10T20:42:53.358911+0000","last_became_active":"2023-10-10T20:42:53.357048+0000","last_became_peered":"2023-10-10T20:42:53.357048+0000","last_unstale":"2023-10-10T20:42:53.358911+0000","last_undegraded":"2023-10-10T20:42:53.358911+0000","last_fullsized":"2023-10-10T20:42:53.358911+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:58:00.782955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.339743+0000","last_change":"2023-10-10T20:42:53.337807+0000","last_active":"2023-10-10T20:42:53.339743+0000","last_peered":"2023-10-10T20:42:53.339743+0000","last_clean":"2023-10-10T20:42:53.339743+0000","last_became_active":"2023-10-10T20:42:53.337553+0000","last_became_peered":"2023-10-10T20:42:53.337553+0000","last_unstale":"2023-10-10T20:42:53.339743+0000","last_undegraded":"2023-10-10T20:42:53.339743+0000","last_fullsized":"2023-10-10T20:42:53.339743+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:13:20.878853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.358934+0000","last_change":"2023-10-10T20:42:53.358025+0000","last_active":"2023-10-10T20:42:53.358934+0000","last_peered":"2023-10-10T20:42:53.358934+0000","last_clean":"2023-10-10T20:42:53.358934+0000","last_became_active":"2023-10-10T20:42:53.355720+0000","last_became_peered":"2023-10-10T20:42:53.355720+0000","last_unstale":"2023-10-10T20:42:53.358934+0000","last_undegraded":"2023-10-10T20:42:53.358934+0000","last_fullsized":"2023-10-10T20:42:53.358934+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:25:50.618140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.344237+0000","last_change":"2023-10-10T20:42:53.342434+0000","last_active":"2023-10-10T20:42:53.344237+0000","last_peered":"2023-10-10T20:42:53.344237+0000","last_clean":"2023-10-10T20:42:53.344237+0000","last_became_active":"2023-10-10T20:42:53.342152+0000","last_became_peered":"2023-10-10T20:42:53.342152+0000","last_unstale":"2023-10-10T20:42:53.344237+0000","last_undegraded":"2023-10-10T20:42:53.344237+0000","last_fullsized":"2023-10-10T20:42:53.344237+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:57.859353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.355929+0000","last_change":"2023-10-10T20:42:53.345842+0000","last_active":"2023-10-10T20:42:53.355929+0000","last_peered":"2023-10-10T20:42:53.355929+0000","last_clean":"2023-10-10T20:42:53.355929+0000","last_became_active":"2023-10-10T20:42:53.345588+0000","last_became_peered":"2023-10-10T20:42:53.345588+0000","last_unstale":"2023-10-10T20:42:53.355929+0000","last_undegraded":"2023-10-10T20:42:53.355929+0000","last_fullsized":"2023-10-10T20:42:53.355929+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:22:18.325313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.347911+0000","last_change":"2023-10-10T20:42:53.345923+0000","last_active":"2023-10-10T20:42:53.347911+0000","last_peered":"2023-10-10T20:42:53.347911+0000","last_clean":"2023-10-10T20:42:53.347911+0000","last_became_active":"2023-10-10T20:42:53.345677+0000","last_became_peered":"2023-10-10T20:42:53.345677+0000","last_unstale":"2023-10-10T20:42:53.347911+0000","last_undegraded":"2023-10-10T20:42:53.347911+0000","last_fullsized":"2023-10-10T20:42:53.347911+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:35.275237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.356172+0000","last_change":"2023-10-10T20:42:53.350069+0000","last_active":"2023-10-10T20:42:53.356172+0000","last_peered":"2023-10-10T20:42:53.356172+0000","last_clean":"2023-10-10T20:42:53.356172+0000","last_became_active":"2023-10-10T20:42:53.349120+0000","last_became_peered":"2023-10-10T20:42:53.349120+0000","last_unstale":"2023-10-10T20:42:53.356172+0000","last_undegraded":"2023-10-10T20:42:53.356172+0000","last_fullsized":"2023-10-10T20:42:53.356172+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:13:20.878853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.353709+0000","last_change":"2023-10-10T20:42:53.352048+0000","last_active":"2023-10-10T20:42:53.353709+0000","last_peered":"2023-10-10T20:42:53.353709+0000","last_clean":"2023-10-10T20:42:53.353709+0000","last_became_active":"2023-10-10T20:42:53.351745+0000","last_became_peered":"2023-10-10T20:42:53.351745+0000","last_unstale":"2023-10-10T20:42:53.353709+0000","last_undegraded":"2023-10-10T20:42:53.353709+0000","last_fullsized":"2023-10-10T20:42:53.353709+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:19:38.741623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.356621+0000","last_change":"2023-10-10T20:42:53.348956+0000","last_active":"2023-10-10T20:42:53.356621+0000","last_peered":"2023-10-10T20:42:53.356621+0000","last_clean":"2023-10-10T20:42:53.356621+0000","last_became_active":"2023-10-10T20:42:53.348502+0000","last_became_peered":"2023-10-10T20:42:53.348502+0000","last_unstale":"2023-10-10T20:42:53.356621+0000","last_undegraded":"2023-10-10T20:42:53.356621+0000","last_fullsized":"2023-10-10T20:42:53.356621+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:38:16.787078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.345439+0000","last_change":"2023-10-10T20:42:53.344413+0000","last_active":"2023-10-10T20:42:53.345439+0000","last_peered":"2023-10-10T20:42:53.345439+0000","last_clean":"2023-10-10T20:42:53.345439+0000","last_became_active":"2023-10-10T20:42:53.344073+0000","last_became_peered":"2023-10-10T20:42:53.344073+0000","last_unstale":"2023-10-10T20:42:53.345439+0000","last_undegraded":"2023-10-10T20:42:53.345439+0000","last_fullsized":"2023-10-10T20:42:53.345439+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:57.859353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.362917+0000","last_change":"2023-10-10T20:42:53.362202+0000","last_active":"2023-10-10T20:42:53.362917+0000","last_peered":"2023-10-10T20:42:53.362917+0000","last_clean":"2023-10-10T20:42:53.362917+0000","last_became_active":"2023-10-10T20:42:53.361913+0000","last_became_peered":"2023-10-10T20:42:53.361913+0000","last_unstale":"2023-10-10T20:42:53.362917+0000","last_undegraded":"2023-10-10T20:42:53.362917+0000","last_fullsized":"2023-10-10T20:42:53.362917+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:39:12.228861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.363199+0000","last_change":"2023-10-10T20:42:53.362071+0000","last_active":"2023-10-10T20:42:53.363199+0000","last_peered":"2023-10-10T20:42:53.363199+0000","last_clean":"2023-10-10T20:42:53.363199+0000","last_became_active":"2023-10-10T20:42:53.361448+0000","last_became_peered":"2023-10-10T20:42:53.361448+0000","last_unstale":"2023-10-10T20:42:53.363199+0000","last_undegraded":"2023-10-10T20:42:53.363199+0000","last_fullsized":"2023-10-10T20:42:53.363199+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:52.463061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.353395+0000","last_change":"2023-10-10T20:42:53.350979+0000","last_active":"2023-10-10T20:42:53.353395+0000","last_peered":"2023-10-10T20:42:53.353395+0000","last_clean":"2023-10-10T20:42:53.353395+0000","last_became_active":"2023-10-10T20:42:53.350702+0000","last_became_peered":"2023-10-10T20:42:53.350702+0000","last_unstale":"2023-10-10T20:42:53.353395+0000","last_undegraded":"2023-10-10T20:42:53.353395+0000","last_fullsized":"2023-10-10T20:42:53.353395+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:17:25.969866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365482+0000","last_change":"2023-10-10T20:42:53.364870+0000","last_active":"2023-10-10T20:42:53.365482+0000","last_peered":"2023-10-10T20:42:53.365482+0000","last_clean":"2023-10-10T20:42:53.365482+0000","last_became_active":"2023-10-10T20:42:53.364542+0000","last_became_peered":"2023-10-10T20:42:53.364542+0000","last_unstale":"2023-10-10T20:42:53.365482+0000","last_undegraded":"2023-10-10T20:42:53.365482+0000","last_fullsized":"2023-10-10T20:42:53.365482+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:27:57.599666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365999+0000","last_change":"2023-10-10T20:42:53.360038+0000","last_active":"2023-10-10T20:42:53.365999+0000","last_peered":"2023-10-10T20:42:53.365999+0000","last_clean":"2023-10-10T20:42:53.365999+0000","last_became_active":"2023-10-10T20:42:53.357757+0000","last_became_peered":"2023-10-10T20:42:53.357757+0000","last_unstale":"2023-10-10T20:42:53.365999+0000","last_undegraded":"2023-10-10T20:42:53.365999+0000","last_fullsized":"2023-10-10T20:42:53.365999+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:16:07.451863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"164'4","reported_seq":14,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.366984+0000","last_change":"2023-10-10T20:42:52.335968+0000","last_active":"2023-10-10T20:42:53.366984+0000","last_peered":"2023-10-10T20:42:53.366984+0000","last_clean":"2023-10-10T20:42:53.366984+0000","last_became_active":"2023-10-10T20:42:52.335695+0000","last_became_peered":"2023-10-10T20:42:52.335695+0000","last_unstale":"2023-10-10T20:42:53.366984+0000","last_undegraded":"2023-10-10T20:42:53.366984+0000","last_fullsized":"2023-10-10T20:42:53.366984+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:32:07.233388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.358522+0000","last_change":"2023-10-10T20:42:53.356868+0000","last_active":"2023-10-10T20:42:53.358522+0000","last_peered":"2023-10-10T20:42:53.358522+0000","last_clean":"2023-10-10T20:42:53.358522+0000","last_became_active":"2023-10-10T20:42:53.356579+0000","last_became_peered":"2023-10-10T20:42:53.356579+0000","last_unstale":"2023-10-10T20:42:53.358522+0000","last_undegraded":"2023-10-10T20:42:53.358522+0000","last_fullsized":"2023-10-10T20:42:53.358522+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:29:45.982326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.361125+0000","last_change":"2023-10-10T20:42:52.336051+0000","last_active":"2023-10-10T20:42:53.361125+0000","last_peered":"2023-10-10T20:42:53.361125+0000","last_clean":"2023-10-10T20:42:53.361125+0000","last_became_active":"2023-10-10T20:42:52.335752+0000","last_became_peered":"2023-10-10T20:42:52.335752+0000","last_unstale":"2023-10-10T20:42:53.361125+0000","last_undegraded":"2023-10-10T20:42:53.361125+0000","last_fullsized":"2023-10-10T20:42:53.361125+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:15:01.473071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.362948+0000","last_change":"2023-10-10T20:42:53.362172+0000","last_active":"2023-10-10T20:42:53.362948+0000","last_peered":"2023-10-10T20:42:53.362948+0000","last_clean":"2023-10-10T20:42:53.362948+0000","last_became_active":"2023-10-10T20:42:53.361874+0000","last_became_peered":"2023-10-10T20:42:53.361874+0000","last_unstale":"2023-10-10T20:42:53.362948+0000","last_undegraded":"2023-10-10T20:42:53.362948+0000","last_fullsized":"2023-10-10T20:42:53.362948+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:38:22.597643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1c","version":"164'2","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365918+0000","last_change":"2023-10-10T20:42:52.339512+0000","last_active":"2023-10-10T20:42:53.365918+0000","last_peered":"2023-10-10T20:42:53.365918+0000","last_clean":"2023-10-10T20:42:53.365918+0000","last_became_active":"2023-10-10T20:42:52.339230+0000","last_became_peered":"2023-10-10T20:42:52.339230+0000","last_unstale":"2023-10-10T20:42:53.365918+0000","last_undegraded":"2023-10-10T20:42:53.365918+0000","last_fullsized":"2023-10-10T20:42:53.365918+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:51.664988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.363218+0000","last_change":"2023-10-10T20:42:53.362266+0000","last_active":"2023-10-10T20:42:53.363218+0000","last_peered":"2023-10-10T20:42:53.363218+0000","last_clean":"2023-10-10T20:42:53.363218+0000","last_became_active":"2023-10-10T20:42:53.361814+0000","last_became_peered":"2023-10-10T20:42:53.361814+0000","last_unstale":"2023-10-10T20:42:53.363218+0000","last_undegraded":"2023-10-10T20:42:53.363218+0000","last_fullsized":"2023-10-10T20:42:53.363218+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:33:29.535005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.354522+0000","last_change":"2023-10-10T20:42:52.332410+0000","last_active":"2023-10-10T20:42:53.354522+0000","last_peered":"2023-10-10T20:42:53.354522+0000","last_clean":"2023-10-10T20:42:53.354522+0000","last_became_active":"2023-10-10T20:42:52.331952+0000","last_became_peered":"2023-10-10T20:42:52.331952+0000","last_unstale":"2023-10-10T20:42:53.354522+0000","last_undegraded":"2023-10-10T20:42:53.354522+0000","last_fullsized":"2023-10-10T20:42:53.354522+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:46:55.907438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.362552+0000","last_change":"2023-10-10T20:42:53.362108+0000","last_active":"2023-10-10T20:42:53.362552+0000","last_peered":"2023-10-10T20:42:53.362552+0000","last_clean":"2023-10-10T20:42:53.362552+0000","last_became_active":"2023-10-10T20:42:53.361846+0000","last_became_peered":"2023-10-10T20:42:53.361846+0000","last_unstale":"2023-10-10T20:42:53.362552+0000","last_undegraded":"2023-10-10T20:42:53.362552+0000","last_fullsized":"2023-10-10T20:42:53.362552+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:20:34.093326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1a","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.364637+0000","last_change":"2023-10-10T20:42:52.338841+0000","last_active":"2023-10-10T20:42:53.364637+0000","last_peered":"2023-10-10T20:42:53.364637+0000","last_clean":"2023-10-10T20:42:53.364637+0000","last_became_active":"2023-10-10T20:42:52.337405+0000","last_became_peered":"2023-10-10T20:42:52.337405+0000","last_unstale":"2023-10-10T20:42:53.364637+0000","last_undegraded":"2023-10-10T20:42:53.364637+0000","last_fullsized":"2023-10-10T20:42:53.364637+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:10:49.794184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.346996+0000","last_change":"2023-10-10T20:42:53.345087+0000","last_active":"2023-10-10T20:42:53.346996+0000","last_peered":"2023-10-10T20:42:53.346996+0000","last_clean":"2023-10-10T20:42:53.346996+0000","last_became_active":"2023-10-10T20:42:53.344387+0000","last_became_peered":"2023-10-10T20:42:53.344387+0000","last_unstale":"2023-10-10T20:42:53.346996+0000","last_undegraded":"2023-10-10T20:42:53.346996+0000","last_fullsized":"2023-10-10T20:42:53.346996+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:34:03.701694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.340733+0000","last_change":"2023-10-10T20:42:52.333926+0000","last_active":"2023-10-10T20:42:53.340733+0000","last_peered":"2023-10-10T20:42:53.340733+0000","last_clean":"2023-10-10T20:42:53.340733+0000","last_became_active":"2023-10-10T20:42:52.333645+0000","last_became_peered":"2023-10-10T20:42:52.333645+0000","last_unstale":"2023-10-10T20:42:53.340733+0000","last_undegraded":"2023-10-10T20:42:53.340733+0000","last_fullsized":"2023-10-10T20:42:53.340733+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:46:47.379359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.366967+0000","last_change":"2023-10-10T20:42:53.363094+0000","last_active":"2023-10-10T20:42:53.366967+0000","last_peered":"2023-10-10T20:42:53.366967+0000","last_clean":"2023-10-10T20:42:53.366967+0000","last_became_active":"2023-10-10T20:42:53.362773+0000","last_became_peered":"2023-10-10T20:42:53.362773+0000","last_unstale":"2023-10-10T20:42:53.366967+0000","last_undegraded":"2023-10-10T20:42:53.366967+0000","last_fullsized":"2023-10-10T20:42:53.366967+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:00.861926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.336097+0000","last_change":"2023-10-10T20:42:52.331824+0000","last_active":"2023-10-10T20:42:53.336097+0000","last_peered":"2023-10-10T20:42:53.336097+0000","last_clean":"2023-10-10T20:42:53.336097+0000","last_became_active":"2023-10-10T20:42:52.331567+0000","last_became_peered":"2023-10-10T20:42:52.331567+0000","last_unstale":"2023-10-10T20:42:53.336097+0000","last_undegraded":"2023-10-10T20:42:53.336097+0000","last_fullsized":"2023-10-10T20:42:53.336097+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:59.776219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.366974+0000","last_change":"2023-10-10T20:42:53.363090+0000","last_active":"2023-10-10T20:42:53.366974+0000","last_peered":"2023-10-10T20:42:53.366974+0000","last_clean":"2023-10-10T20:42:53.366974+0000","last_became_active":"2023-10-10T20:42:53.362679+0000","last_became_peered":"2023-10-10T20:42:53.362679+0000","last_unstale":"2023-10-10T20:42:53.366974+0000","last_undegraded":"2023-10-10T20:42:53.366974+0000","last_fullsized":"2023-10-10T20:42:53.366974+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:09:34.459455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.336778+0000","last_change":"2023-10-10T20:42:52.329973+0000","last_active":"2023-10-10T20:42:53.336778+0000","last_peered":"2023-10-10T20:42:53.336778+0000","last_clean":"2023-10-10T20:42:53.336778+0000","last_became_active":"2023-10-10T20:42:52.329001+0000","last_became_peered":"2023-10-10T20:42:52.329001+0000","last_unstale":"2023-10-10T20:42:53.336778+0000","last_undegraded":"2023-10-10T20:42:53.336778+0000","last_fullsized":"2023-10-10T20:42:53.336778+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:20:37.621153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.356128+0000","last_change":"2023-10-10T20:42:53.349126+0000","last_active":"2023-10-10T20:42:53.356128+0000","last_peered":"2023-10-10T20:42:53.356128+0000","last_clean":"2023-10-10T20:42:53.356128+0000","last_became_active":"2023-10-10T20:42:53.344703+0000","last_became_peered":"2023-10-10T20:42:53.344703+0000","last_unstale":"2023-10-10T20:42:53.356128+0000","last_undegraded":"2023-10-10T20:42:53.356128+0000","last_fullsized":"2023-10-10T20:42:53.356128+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:27.171367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.333312+0000","last_change":"2023-10-10T20:42:52.334832+0000","last_active":"2023-10-10T20:42:53.333312+0000","last_peered":"2023-10-10T20:42:53.333312+0000","last_clean":"2023-10-10T20:42:53.333312+0000","last_became_active":"2023-10-10T20:42:52.334555+0000","last_became_peered":"2023-10-10T20:42:52.334555+0000","last_unstale":"2023-10-10T20:42:53.333312+0000","last_undegraded":"2023-10-10T20:42:53.333312+0000","last_fullsized":"2023-10-10T20:42:53.333312+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:51.456325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.362400+0000","last_change":"2023-10-10T20:42:53.360965+0000","last_active":"2023-10-10T20:42:53.362400+0000","last_peered":"2023-10-10T20:42:53.362400+0000","last_clean":"2023-10-10T20:42:53.362400+0000","last_became_active":"2023-10-10T20:42:53.360601+0000","last_became_peered":"2023-10-10T20:42:53.360601+0000","last_unstale":"2023-10-10T20:42:53.362400+0000","last_undegraded":"2023-10-10T20:42:53.362400+0000","last_fullsized":"2023-10-10T20:42:53.362400+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:52:20.890355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.15","version":"164'3","reported_seq":13,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.355795+0000","last_change":"2023-10-10T20:42:52.341985+0000","last_active":"2023-10-10T20:42:53.355795+0000","last_peered":"2023-10-10T20:42:53.355795+0000","last_clean":"2023-10-10T20:42:53.355795+0000","last_became_active":"2023-10-10T20:42:52.341730+0000","last_became_peered":"2023-10-10T20:42:52.341730+0000","last_unstale":"2023-10-10T20:42:53.355795+0000","last_undegraded":"2023-10-10T20:42:53.355795+0000","last_fullsized":"2023-10-10T20:42:53.355795+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:32:48.988699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.356638+0000","last_change":"2023-10-10T20:42:53.349828+0000","last_active":"2023-10-10T20:42:53.356638+0000","last_peered":"2023-10-10T20:42:53.356638+0000","last_clean":"2023-10-10T20:42:53.356638+0000","last_became_active":"2023-10-10T20:42:53.349514+0000","last_became_peered":"2023-10-10T20:42:53.349514+0000","last_unstale":"2023-10-10T20:42:53.356638+0000","last_undegraded":"2023-10-10T20:42:53.356638+0000","last_fullsized":"2023-10-10T20:42:53.356638+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:57:12.184703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"164'6","reported_seq":16,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.356757+0000","last_change":"2023-10-10T20:42:52.333830+0000","last_active":"2023-10-10T20:42:53.356757+0000","last_peered":"2023-10-10T20:42:53.356757+0000","last_clean":"2023-10-10T20:42:53.356757+0000","last_became_active":"2023-10-10T20:42:52.333570+0000","last_became_peered":"2023-10-10T20:42:52.333570+0000","last_unstale":"2023-10-10T20:42:53.356757+0000","last_undegraded":"2023-10-10T20:42:53.356757+0000","last_fullsized":"2023-10-10T20:42:53.356757+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:31.572538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.348179+0000","last_change":"2023-10-10T20:42:53.346232+0000","last_active":"2023-10-10T20:42:53.348179+0000","last_peered":"2023-10-10T20:42:53.348179+0000","last_clean":"2023-10-10T20:42:53.348179+0000","last_became_active":"2023-10-10T20:42:53.345920+0000","last_became_peered":"2023-10-10T20:42:53.345920+0000","last_unstale":"2023-10-10T20:42:53.348179+0000","last_undegraded":"2023-10-10T20:42:53.348179+0000","last_fullsized":"2023-10-10T20:42:53.348179+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:48:47.062433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"164'2","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.366389+0000","last_change":"2023-10-10T20:42:52.339386+0000","last_active":"2023-10-10T20:42:53.366389+0000","last_peered":"2023-10-10T20:42:53.366389+0000","last_clean":"2023-10-10T20:42:53.366389+0000","last_became_active":"2023-10-10T20:42:52.339072+0000","last_became_peered":"2023-10-10T20:42:52.339072+0000","last_unstale":"2023-10-10T20:42:53.366389+0000","last_undegraded":"2023-10-10T20:42:53.366389+0000","last_fullsized":"2023-10-10T20:42:53.366389+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:53:19.603938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.367107+0000","last_change":"2023-10-10T20:42:53.364661+0000","last_active":"2023-10-10T20:42:53.367107+0000","last_peered":"2023-10-10T20:42:53.367107+0000","last_clean":"2023-10-10T20:42:53.367107+0000","last_became_active":"2023-10-10T20:42:53.361122+0000","last_became_peered":"2023-10-10T20:42:53.361122+0000","last_unstale":"2023-10-10T20:42:53.367107+0000","last_undegraded":"2023-10-10T20:42:53.367107+0000","last_fullsized":"2023-10-10T20:42:53.367107+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:58:00.782955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.333584+0000","last_change":"2023-10-10T20:42:52.337702+0000","last_active":"2023-10-10T20:42:53.333584+0000","last_peered":"2023-10-10T20:42:53.333584+0000","last_clean":"2023-10-10T20:42:53.333584+0000","last_became_active":"2023-10-10T20:42:52.337415+0000","last_became_peered":"2023-10-10T20:42:52.337415+0000","last_unstale":"2023-10-10T20:42:53.333584+0000","last_undegraded":"2023-10-10T20:42:53.333584+0000","last_fullsized":"2023-10-10T20:42:53.333584+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:33:28.528269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.367441+0000","last_change":"2023-10-10T20:42:53.365582+0000","last_active":"2023-10-10T20:42:53.367441+0000","last_peered":"2023-10-10T20:42:53.367441+0000","last_clean":"2023-10-10T20:42:53.367441+0000","last_became_active":"2023-10-10T20:42:53.365256+0000","last_became_peered":"2023-10-10T20:42:53.365256+0000","last_unstale":"2023-10-10T20:42:53.367441+0000","last_undegraded":"2023-10-10T20:42:53.367441+0000","last_fullsized":"2023-10-10T20:42:53.367441+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:25:50.618140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.336708+0000","last_change":"2023-10-10T20:42:52.338838+0000","last_active":"2023-10-10T20:42:53.336708+0000","last_peered":"2023-10-10T20:42:53.336708+0000","last_clean":"2023-10-10T20:42:53.336708+0000","last_became_active":"2023-10-10T20:42:52.338592+0000","last_became_peered":"2023-10-10T20:42:52.338592+0000","last_unstale":"2023-10-10T20:42:53.336708+0000","last_undegraded":"2023-10-10T20:42:53.336708+0000","last_fullsized":"2023-10-10T20:42:53.336708+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:36:29.288624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.351267+0000","last_change":"2023-10-10T20:42:53.349965+0000","last_active":"2023-10-10T20:42:53.351267+0000","last_peered":"2023-10-10T20:42:53.351267+0000","last_clean":"2023-10-10T20:42:53.351267+0000","last_became_active":"2023-10-10T20:42:53.349675+0000","last_became_peered":"2023-10-10T20:42:53.349675+0000","last_unstale":"2023-10-10T20:42:53.351267+0000","last_undegraded":"2023-10-10T20:42:53.351267+0000","last_fullsized":"2023-10-10T20:42:53.351267+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:44.047869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.10","version":"164'8","reported_seq":18,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.367445+0000","last_change":"2023-10-10T20:42:52.334750+0000","last_active":"2023-10-10T20:42:53.367445+0000","last_peered":"2023-10-10T20:42:53.367445+0000","last_clean":"2023-10-10T20:42:53.367445+0000","last_became_active":"2023-10-10T20:42:52.334260+0000","last_became_peered":"2023-10-10T20:42:52.334260+0000","last_unstale":"2023-10-10T20:42:53.367445+0000","last_undegraded":"2023-10-10T20:42:53.367445+0000","last_fullsized":"2023-10-10T20:42:53.367445+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:49:39.517807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.350525+0000","last_change":"2023-10-10T20:42:53.349062+0000","last_active":"2023-10-10T20:42:53.350525+0000","last_peered":"2023-10-10T20:42:53.350525+0000","last_clean":"2023-10-10T20:42:53.350525+0000","last_became_active":"2023-10-10T20:42:53.348806+0000","last_became_peered":"2023-10-10T20:42:53.348806+0000","last_unstale":"2023-10-10T20:42:53.350525+0000","last_undegraded":"2023-10-10T20:42:53.350525+0000","last_fullsized":"2023-10-10T20:42:53.350525+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:46:32.614897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365941+0000","last_change":"2023-10-10T20:42:52.339183+0000","last_active":"2023-10-10T20:42:53.365941+0000","last_peered":"2023-10-10T20:42:53.365941+0000","last_clean":"2023-10-10T20:42:53.365941+0000","last_became_active":"2023-10-10T20:42:52.337722+0000","last_became_peered":"2023-10-10T20:42:52.337722+0000","last_unstale":"2023-10-10T20:42:53.365941+0000","last_undegraded":"2023-10-10T20:42:53.365941+0000","last_fullsized":"2023-10-10T20:42:53.365941+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:51.664988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365801+0000","last_change":"2023-10-10T20:42:53.365349+0000","last_active":"2023-10-10T20:42:53.365801+0000","last_peered":"2023-10-10T20:42:53.365801+0000","last_clean":"2023-10-10T20:42:53.365801+0000","last_became_active":"2023-10-10T20:42:53.365103+0000","last_became_peered":"2023-10-10T20:42:53.365103+0000","last_unstale":"2023-10-10T20:42:53.365801+0000","last_undegraded":"2023-10-10T20:42:53.365801+0000","last_fullsized":"2023-10-10T20:42:53.365801+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:48:47.062433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"164'2","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.358150+0000","last_change":"2023-10-10T20:42:52.335410+0000","last_active":"2023-10-10T20:42:53.358150+0000","last_peered":"2023-10-10T20:42:53.358150+0000","last_clean":"2023-10-10T20:42:53.358150+0000","last_became_active":"2023-10-10T20:42:52.335097+0000","last_became_peered":"2023-10-10T20:42:52.335097+0000","last_unstale":"2023-10-10T20:42:53.358150+0000","last_undegraded":"2023-10-10T20:42:53.358150+0000","last_fullsized":"2023-10-10T20:42:53.358150+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:22:17.318577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365826+0000","last_change":"2023-10-10T20:42:53.359692+0000","last_active":"2023-10-10T20:42:53.365826+0000","last_peered":"2023-10-10T20:42:53.365826+0000","last_clean":"2023-10-10T20:42:53.365826+0000","last_became_active":"2023-10-10T20:42:53.359187+0000","last_became_peered":"2023-10-10T20:42:53.359187+0000","last_unstale":"2023-10-10T20:42:53.365826+0000","last_undegraded":"2023-10-10T20:42:53.365826+0000","last_fullsized":"2023-10-10T20:42:53.365826+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:28:41.786703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.d","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.356705+0000","last_change":"2023-10-10T20:42:52.332490+0000","last_active":"2023-10-10T20:42:53.356705+0000","last_peered":"2023-10-10T20:42:53.356705+0000","last_clean":"2023-10-10T20:42:53.356705+0000","last_became_active":"2023-10-10T20:42:52.332045+0000","last_became_peered":"2023-10-10T20:42:52.332045+0000","last_unstale":"2023-10-10T20:42:53.356705+0000","last_undegraded":"2023-10-10T20:42:53.356705+0000","last_fullsized":"2023-10-10T20:42:53.356705+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:01.845490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.366441+0000","last_change":"2023-10-10T20:42:53.358569+0000","last_active":"2023-10-10T20:42:53.366441+0000","last_peered":"2023-10-10T20:42:53.366441+0000","last_clean":"2023-10-10T20:42:53.366441+0000","last_became_active":"2023-10-10T20:42:53.358306+0000","last_became_peered":"2023-10-10T20:42:53.358306+0000","last_unstale":"2023-10-10T20:42:53.366441+0000","last_undegraded":"2023-10-10T20:42:53.366441+0000","last_fullsized":"2023-10-10T20:42:53.366441+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:35.275237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.336302+0000","last_change":"2023-10-10T20:42:52.333544+0000","last_active":"2023-10-10T20:42:53.336302+0000","last_peered":"2023-10-10T20:42:53.336302+0000","last_clean":"2023-10-10T20:42:53.336302+0000","last_became_active":"2023-10-10T20:42:52.333211+0000","last_became_peered":"2023-10-10T20:42:52.333211+0000","last_unstale":"2023-10-10T20:42:53.336302+0000","last_undegraded":"2023-10-10T20:42:53.336302+0000","last_fullsized":"2023-10-10T20:42:53.336302+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:39.789815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.353842+0000","last_change":"2023-10-10T20:42:53.351743+0000","last_active":"2023-10-10T20:42:53.353842+0000","last_peered":"2023-10-10T20:42:53.353842+0000","last_clean":"2023-10-10T20:42:53.353842+0000","last_became_active":"2023-10-10T20:42:53.351348+0000","last_became_peered":"2023-10-10T20:42:53.351348+0000","last_unstale":"2023-10-10T20:42:53.353842+0000","last_undegraded":"2023-10-10T20:42:53.353842+0000","last_fullsized":"2023-10-10T20:42:53.353842+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:34:11.321271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.b","version":"164'2","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.364847+0000","last_change":"2023-10-10T20:42:52.336202+0000","last_active":"2023-10-10T20:42:53.364847+0000","last_peered":"2023-10-10T20:42:53.364847+0000","last_clean":"2023-10-10T20:42:53.364847+0000","last_became_active":"2023-10-10T20:42:52.335926+0000","last_became_peered":"2023-10-10T20:42:52.335926+0000","last_unstale":"2023-10-10T20:42:53.364847+0000","last_undegraded":"2023-10-10T20:42:53.364847+0000","last_fullsized":"2023-10-10T20:42:53.364847+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:49:39.517807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.367413+0000","last_change":"2023-10-10T20:42:53.365558+0000","last_active":"2023-10-10T20:42:53.367413+0000","last_peered":"2023-10-10T20:42:53.367413+0000","last_clean":"2023-10-10T20:42:53.367413+0000","last_became_active":"2023-10-10T20:42:53.364328+0000","last_became_peered":"2023-10-10T20:42:53.364328+0000","last_unstale":"2023-10-10T20:42:53.367413+0000","last_undegraded":"2023-10-10T20:42:53.367413+0000","last_fullsized":"2023-10-10T20:42:53.367413+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:40.796551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.341106+0000","last_change":"2023-10-10T20:42:52.335457+0000","last_active":"2023-10-10T20:42:53.341106+0000","last_peered":"2023-10-10T20:42:53.341106+0000","last_clean":"2023-10-10T20:42:53.341106+0000","last_became_active":"2023-10-10T20:42:52.335186+0000","last_became_peered":"2023-10-10T20:42:52.335186+0000","last_unstale":"2023-10-10T20:42:53.341106+0000","last_undegraded":"2023-10-10T20:42:53.341106+0000","last_fullsized":"2023-10-10T20:42:53.341106+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:55:21.874802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.347700+0000","last_change":"2023-10-10T20:42:53.346671+0000","last_active":"2023-10-10T20:42:53.347700+0000","last_peered":"2023-10-10T20:42:53.347700+0000","last_clean":"2023-10-10T20:42:53.347700+0000","last_became_active":"2023-10-10T20:42:53.346422+0000","last_became_peered":"2023-10-10T20:42:53.346422+0000","last_unstale":"2023-10-10T20:42:53.347700+0000","last_undegraded":"2023-10-10T20:42:53.347700+0000","last_fullsized":"2023-10-10T20:42:53.347700+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:24:01.205198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.340825+0000","last_change":"2023-10-10T20:42:52.337248+0000","last_active":"2023-10-10T20:42:53.340825+0000","last_peered":"2023-10-10T20:42:53.340825+0000","last_clean":"2023-10-10T20:42:53.340825+0000","last_became_active":"2023-10-10T20:42:52.336966+0000","last_became_peered":"2023-10-10T20:42:52.336966+0000","last_unstale":"2023-10-10T20:42:53.340825+0000","last_undegraded":"2023-10-10T20:42:53.340825+0000","last_fullsized":"2023-10-10T20:42:53.340825+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:59:14.539048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.345348+0000","last_change":"2023-10-10T20:42:53.343031+0000","last_active":"2023-10-10T20:42:53.345348+0000","last_peered":"2023-10-10T20:42:53.345348+0000","last_clean":"2023-10-10T20:42:53.345348+0000","last_became_active":"2023-10-10T20:42:53.342760+0000","last_became_peered":"2023-10-10T20:42:53.342760+0000","last_unstale":"2023-10-10T20:42:53.345348+0000","last_undegraded":"2023-10-10T20:42:53.345348+0000","last_fullsized":"2023-10-10T20:42:53.345348+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:45:05.557928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.349884+0000","last_change":"2023-10-10T20:42:52.350531+0000","last_active":"2023-10-10T20:42:53.349884+0000","last_peered":"2023-10-10T20:42:53.349884+0000","last_clean":"2023-10-10T20:42:53.349884+0000","last_became_active":"2023-10-10T20:42:52.350315+0000","last_became_peered":"2023-10-10T20:42:52.350315+0000","last_unstale":"2023-10-10T20:42:53.349884+0000","last_undegraded":"2023-10-10T20:42:53.349884+0000","last_fullsized":"2023-10-10T20:42:53.349884+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:08.850614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.345339+0000","last_change":"2023-10-10T20:42:53.343152+0000","last_active":"2023-10-10T20:42:53.345339+0000","last_peered":"2023-10-10T20:42:53.345339+0000","last_clean":"2023-10-10T20:42:53.345339+0000","last_became_active":"2023-10-10T20:42:53.342850+0000","last_became_peered":"2023-10-10T20:42:53.342850+0000","last_unstale":"2023-10-10T20:42:53.345339+0000","last_undegraded":"2023-10-10T20:42:53.345339+0000","last_fullsized":"2023-10-10T20:42:53.345339+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:38:10.362086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.7","version":"164'2","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365076+0000","last_change":"2023-10-10T20:42:52.338420+0000","last_active":"2023-10-10T20:42:53.365076+0000","last_peered":"2023-10-10T20:42:53.365076+0000","last_clean":"2023-10-10T20:42:53.365076+0000","last_became_active":"2023-10-10T20:42:52.338190+0000","last_became_peered":"2023-10-10T20:42:52.338190+0000","last_unstale":"2023-10-10T20:42:53.365076+0000","last_undegraded":"2023-10-10T20:42:53.365076+0000","last_fullsized":"2023-10-10T20:42:53.365076+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:58:31.586594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.366016+0000","last_change":"2023-10-10T20:42:53.360884+0000","last_active":"2023-10-10T20:42:53.366016+0000","last_peered":"2023-10-10T20:42:53.366016+0000","last_clean":"2023-10-10T20:42:53.366016+0000","last_became_active":"2023-10-10T20:42:53.360108+0000","last_became_peered":"2023-10-10T20:42:53.360108+0000","last_unstale":"2023-10-10T20:42:53.366016+0000","last_undegraded":"2023-10-10T20:42:53.366016+0000","last_fullsized":"2023-10-10T20:42:53.366016+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:34:11.321271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.336308+0000","last_change":"2023-10-10T20:42:52.335469+0000","last_active":"2023-10-10T20:42:53.336308+0000","last_peered":"2023-10-10T20:42:53.336308+0000","last_clean":"2023-10-10T20:42:53.336308+0000","last_became_active":"2023-10-10T20:42:52.333014+0000","last_became_peered":"2023-10-10T20:42:52.333014+0000","last_unstale":"2023-10-10T20:42:53.336308+0000","last_undegraded":"2023-10-10T20:42:53.336308+0000","last_fullsized":"2023-10-10T20:42:53.336308+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:16:49.047090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365802+0000","last_change":"2023-10-10T20:42:53.359515+0000","last_active":"2023-10-10T20:42:53.365802+0000","last_peered":"2023-10-10T20:42:53.365802+0000","last_clean":"2023-10-10T20:42:53.365802+0000","last_became_active":"2023-10-10T20:42:53.358893+0000","last_became_peered":"2023-10-10T20:42:53.358893+0000","last_unstale":"2023-10-10T20:42:53.365802+0000","last_undegraded":"2023-10-10T20:42:53.365802+0000","last_fullsized":"2023-10-10T20:42:53.365802+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:38:10.362086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.336547+0000","last_change":"2023-10-10T20:42:52.334206+0000","last_active":"2023-10-10T20:42:53.336547+0000","last_peered":"2023-10-10T20:42:53.336547+0000","last_clean":"2023-10-10T20:42:53.336547+0000","last_became_active":"2023-10-10T20:42:52.333022+0000","last_became_peered":"2023-10-10T20:42:52.333022+0000","last_unstale":"2023-10-10T20:42:53.336547+0000","last_undegraded":"2023-10-10T20:42:53.336547+0000","last_fullsized":"2023-10-10T20:42:53.336547+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:34:10.314535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.346479+0000","last_change":"2023-10-10T20:42:53.341044+0000","last_active":"2023-10-10T20:42:53.346479+0000","last_peered":"2023-10-10T20:42:53.346479+0000","last_clean":"2023-10-10T20:42:53.346479+0000","last_became_active":"2023-10-10T20:42:53.340802+0000","last_became_peered":"2023-10-10T20:42:53.340802+0000","last_unstale":"2023-10-10T20:42:53.346479+0000","last_undegraded":"2023-10-10T20:42:53.346479+0000","last_fullsized":"2023-10-10T20:42:53.346479+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:44:40.439820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.351585+0000","last_change":"2023-10-10T20:42:52.339808+0000","last_active":"2023-10-10T20:42:53.351585+0000","last_peered":"2023-10-10T20:42:53.351585+0000","last_clean":"2023-10-10T20:42:53.351585+0000","last_became_active":"2023-10-10T20:42:52.339538+0000","last_became_peered":"2023-10-10T20:42:52.339538+0000","last_unstale":"2023-10-10T20:42:53.351585+0000","last_undegraded":"2023-10-10T20:42:53.351585+0000","last_fullsized":"2023-10-10T20:42:53.351585+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:17:22.696388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.342915+0000","last_change":"2023-10-10T20:42:53.342134+0000","last_active":"2023-10-10T20:42:53.342915+0000","last_peered":"2023-10-10T20:42:53.342915+0000","last_clean":"2023-10-10T20:42:53.342915+0000","last_became_active":"2023-10-10T20:42:53.341742+0000","last_became_peered":"2023-10-10T20:42:53.341742+0000","last_unstale":"2023-10-10T20:42:53.342915+0000","last_undegraded":"2023-10-10T20:42:53.342915+0000","last_fullsized":"2023-10-10T20:42:53.342915+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:18:18.498552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.341700+0000","last_change":"2023-10-10T20:42:52.330898+0000","last_active":"2023-10-10T20:42:53.341700+0000","last_peered":"2023-10-10T20:42:53.341700+0000","last_clean":"2023-10-10T20:42:53.341700+0000","last_became_active":"2023-10-10T20:42:52.330616+0000","last_became_peered":"2023-10-10T20:42:52.330616+0000","last_unstale":"2023-10-10T20:42:53.341700+0000","last_undegraded":"2023-10-10T20:42:53.341700+0000","last_fullsized":"2023-10-10T20:42:53.341700+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:30.999244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.367131+0000","last_change":"2023-10-10T20:42:53.364609+0000","last_active":"2023-10-10T20:42:53.367131+0000","last_peered":"2023-10-10T20:42:53.367131+0000","last_clean":"2023-10-10T20:42:53.367131+0000","last_became_active":"2023-10-10T20:42:53.360806+0000","last_became_peered":"2023-10-10T20:42:53.360806+0000","last_unstale":"2023-10-10T20:42:53.367131+0000","last_undegraded":"2023-10-10T20:42:53.367131+0000","last_fullsized":"2023-10-10T20:42:53.367131+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:44:40.439820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.333553+0000","last_change":"2023-10-10T20:42:52.333625+0000","last_active":"2023-10-10T20:42:53.333553+0000","last_peered":"2023-10-10T20:42:53.333553+0000","last_clean":"2023-10-10T20:42:53.333553+0000","last_became_active":"2023-10-10T20:42:52.333359+0000","last_became_peered":"2023-10-10T20:42:52.333359+0000","last_unstale":"2023-10-10T20:42:53.333553+0000","last_undegraded":"2023-10-10T20:42:53.333553+0000","last_fullsized":"2023-10-10T20:42:53.333553+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:31.572538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.366949+0000","last_change":"2023-10-10T20:42:53.364374+0000","last_active":"2023-10-10T20:42:53.366949+0000","last_peered":"2023-10-10T20:42:53.366949+0000","last_clean":"2023-10-10T20:42:53.366949+0000","last_became_active":"2023-10-10T20:42:53.361551+0000","last_became_peered":"2023-10-10T20:42:53.361551+0000","last_unstale":"2023-10-10T20:42:53.366949+0000","last_undegraded":"2023-10-10T20:42:53.366949+0000","last_fullsized":"2023-10-10T20:42:53.366949+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:13:45.883755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.336771+0000","last_change":"2023-10-10T20:42:52.339859+0000","last_active":"2023-10-10T20:42:53.336771+0000","last_peered":"2023-10-10T20:42:53.336771+0000","last_clean":"2023-10-10T20:42:53.336771+0000","last_became_active":"2023-10-10T20:42:52.339585+0000","last_became_peered":"2023-10-10T20:42:52.339585+0000","last_unstale":"2023-10-10T20:42:53.336771+0000","last_undegraded":"2023-10-10T20:42:53.336771+0000","last_fullsized":"2023-10-10T20:42:53.336771+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:02.517948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.355907+0000","last_change":"2023-10-10T20:42:53.345345+0000","last_active":"2023-10-10T20:42:53.355907+0000","last_peered":"2023-10-10T20:42:53.355907+0000","last_clean":"2023-10-10T20:42:53.355907+0000","last_became_active":"2023-10-10T20:42:53.345080+0000","last_became_peered":"2023-10-10T20:42:53.345080+0000","last_unstale":"2023-10-10T20:42:53.355907+0000","last_undegraded":"2023-10-10T20:42:53.355907+0000","last_fullsized":"2023-10-10T20:42:53.355907+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:32:08.240124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.349987+0000","last_change":"2023-10-10T20:42:52.335154+0000","last_active":"2023-10-10T20:42:53.349987+0000","last_peered":"2023-10-10T20:42:53.349987+0000","last_clean":"2023-10-10T20:42:53.349987+0000","last_became_active":"2023-10-10T20:42:52.334853+0000","last_became_peered":"2023-10-10T20:42:52.334853+0000","last_unstale":"2023-10-10T20:42:53.349987+0000","last_undegraded":"2023-10-10T20:42:53.349987+0000","last_fullsized":"2023-10-10T20:42:53.349987+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:15:01.473071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.341777+0000","last_change":"2023-10-10T20:42:53.340554+0000","last_active":"2023-10-10T20:42:53.341777+0000","last_peered":"2023-10-10T20:42:53.341777+0000","last_clean":"2023-10-10T20:42:53.341777+0000","last_became_active":"2023-10-10T20:42:53.340308+0000","last_became_peered":"2023-10-10T20:42:53.340308+0000","last_unstale":"2023-10-10T20:42:53.341777+0000","last_undegraded":"2023-10-10T20:42:53.341777+0000","last_fullsized":"2023-10-10T20:42:53.341777+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:38:30.259757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"164'2","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365418+0000","last_change":"2023-10-10T20:42:52.338269+0000","last_active":"2023-10-10T20:42:53.365418+0000","last_peered":"2023-10-10T20:42:53.365418+0000","last_clean":"2023-10-10T20:42:53.365418+0000","last_became_active":"2023-10-10T20:42:52.338028+0000","last_became_peered":"2023-10-10T20:42:52.338028+0000","last_unstale":"2023-10-10T20:42:53.365418+0000","last_undegraded":"2023-10-10T20:42:53.365418+0000","last_fullsized":"2023-10-10T20:42:53.365418+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:17:34.621354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":308,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.335377+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:42:53.335377+0000","last_peered":"2023-10-10T20:42:53.335377+0000","last_clean":"2023-10-10T20:42:53.335377+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:42:53.335377+0000","last_undegraded":"2023-10-10T20:42:53.335377+0000","last_fullsized":"2023-10-10T20:42:53.335377+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574954,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304012,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067828,"statfs":{"total":96636764160,"available":96325455872,"internally_reserved":0,"allocated":2691072,"data_stored":1527131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50700000000000001,"15min":0.75800000000000001},"min":{"1min":0.32500000000000001,"5min":0.309,"15min":0.29199999999999998},"max":{"1min":0.64300000000000002,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.626},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.50800000000000001,"15min":0.60399999999999998},"min":{"1min":0.36399999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.58299999999999996,"5min":0.752,"15min":4.1959999999999997},"last":0.33300000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.50600000000000001,"15min":0.90100000000000002},"min":{"1min":0.372,"5min":0.317,"15min":0.317},"max":{"1min":0.61099999999999999,"5min":0.78800000000000003,"15min":10.1},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.53400000000000003,"15min":0.69099999999999995},"min":{"1min":0.35199999999999998,"5min":0.33900000000000002,"15min":0.28999999999999998},"max":{"1min":0.66500000000000004,"5min":0.81799999999999995,"15min":6.806},"last":0.48099999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56799999999999995,"15min":0.74099999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68799999999999994,"5min":0.89000000000000001,"15min":6.9340000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.69399999999999995},"min":{"1min":0.40699999999999997,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.69099999999999995,"5min":0.77100000000000002,"15min":6.9870000000000001},"last":0.56399999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54200000000000004,"15min":0.71299999999999997},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64500000000000002,"5min":0.754,"15min":6.6719999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55900000000000005,"15min":0.64800000000000002},"min":{"1min":0.39800000000000002,"5min":0.378,"15min":0.34799999999999998},"max":{"1min":0.64400000000000002,"5min":0.76200000000000001,"15min":4.1609999999999996},"last":0.71999999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47799999999999998,"15min":0.46800000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.77200000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.51600000000000001,"15min":0.51900000000000002},"min":{"1min":0.314,"5min":0.309,"15min":0.30399999999999999},"max":{"1min":0.68000000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.67800000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.54000000000000004,"15min":0.53900000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.68400000000000005,"5min":0.86099999999999999,"15min":1.3180000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.53600000000000003,"15min":0.52700000000000002},"min":{"1min":0.35299999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.84099999999999997},"last":0.64800000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.35699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":1.7509999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76300000000000001,"5min":0.84099999999999997,"15min":1.7090000000000001},"last":0.73099999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303436,"kb_used_data":2052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068404,"statfs":{"total":96636764160,"available":96326045696,"internally_reserved":0,"allocated":2101248,"data_stored":936749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52400000000000002,"15min":0.64000000000000001},"min":{"1min":0.34699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":7.7590000000000003},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52400000000000002,"15min":0.67400000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.86899999999999999,"5min":1.075,"15min":7.8090000000000002},"last":0.40600000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52900000000000003,"15min":0.67200000000000004},"min":{"1min":0.32800000000000001,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":7.867},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55500000000000005,"15min":0.81999999999999995},"min":{"1min":0.378,"5min":0.35499999999999998,"15min":0.33700000000000002},"max":{"1min":0.70999999999999996,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.36099999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52300000000000002,"15min":0.58099999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.745,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57099999999999995,"15min":0.79100000000000004},"min":{"1min":0.39700000000000002,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":2.641,"15min":6.7629999999999999},"last":0.439}]},{"osd":3,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55500000000000005,"15min":0.60699999999999998},"min":{"1min":0.372,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94399999999999995,"5min":1.107,"15min":4.8620000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55500000000000005,"15min":0.68400000000000005},"min":{"1min":0.41399999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.72599999999999998,"5min":1.718,"15min":7.9009999999999998},"last":0.501}]},{"osd":4,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51900000000000002,"15min":0.47899999999999998},"min":{"1min":0.32000000000000001,"5min":0.28999999999999998,"15min":0.28499999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":0.82899999999999996,"5min":1.032,"15min":1.032},"last":0.52400000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53400000000000003,"15min":0.5},"min":{"1min":0.34999999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.86799999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.379},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56999999999999995,"15min":0.53700000000000003},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85399999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.51300000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56200000000000006,"15min":0.53000000000000003},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.89600000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.59399999999999997,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.372,"15min":0.35099999999999998},"max":{"1min":0.82199999999999995,"5min":1.681,"15min":1.681},"last":0.42699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303432,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068408,"statfs":{"total":96636764160,"available":96326049792,"internally_reserved":0,"allocated":2097152,"data_stored":936783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.31900000000000001,"15min":0.443},"min":{"1min":0.215,"5min":0.19600000000000001,"15min":0.17899999999999999},"max":{"1min":0.47799999999999998,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33000000000000002,"15min":0.35599999999999998},"min":{"1min":0.22500000000000001,"5min":0.224,"15min":0.20899999999999999},"max":{"1min":0.441,"5min":0.65500000000000003,"15min":1.194},"last":0.33400000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.32700000000000001,"15min":0.35499999999999998},"min":{"1min":0.246,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.41699999999999998,"5min":0.51400000000000001,"15min":1.2869999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.35199999999999998,"15min":0.47299999999999998},"min":{"1min":0.23999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.49199999999999999,"5min":0.54200000000000004,"15min":5.0919999999999996},"last":0.316}]},{"osd":3,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.374,"15min":0.496},"min":{"1min":0.25600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54800000000000004,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.36299999999999999,"15min":0.39800000000000002},"min":{"1min":0.26500000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.52600000000000002,"5min":0.57799999999999996,"15min":1.2170000000000001},"last":0.29599999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.78600000000000003,"5min":0.92400000000000004,"15min":1.254},"last":0.499},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55900000000000005,"15min":0.67500000000000004},"min":{"1min":0.39600000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68100000000000005,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.56100000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.40400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.97599999999999998,"15min":1.1799999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56999999999999995,"15min":0.59999999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73299999999999998,"5min":0.878,"15min":1.3109999999999999},"last":0.57299999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":0.79300000000000004},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70399999999999996,"5min":0.995,"15min":5.0659999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60699999999999998,"15min":0.628},"min":{"1min":0.46400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84199999999999997,"5min":0.89800000000000002,"15min":1.171},"last":0.48199999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.48299999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77100000000000002,"5min":1.004,"15min":1.214},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58399999999999996,"15min":0.69699999999999995},"min":{"1min":0.40899999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.754,"5min":0.90500000000000003,"15min":4.0499999999999998},"last":0.52100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574953,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304012,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067828,"statfs":{"total":96636764160,"available":96325455872,"internally_reserved":0,"allocated":2691072,"data_stored":1526744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.315,"15min":0.48599999999999999},"min":{"1min":0.217,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":7.9249999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.46899999999999997},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46100000000000002,"5min":0.53800000000000003,"15min":6.6299999999999999},"last":0.30299999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33900000000000002,"15min":0.51000000000000001},"min":{"1min":0.25600000000000001,"5min":0.22,"15min":0.216},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":8.2509999999999994},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.49299999999999999},"min":{"1min":0.27500000000000002,"5min":0.223,"15min":0.222},"max":{"1min":0.498,"5min":0.57899999999999996,"15min":6.7370000000000001},"last":0.35499999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.35499999999999998,"15min":0.51700000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.21099999999999999},"max":{"1min":0.438,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.37},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.57099999999999995},"min":{"1min":0.317,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":7.8929999999999998},"last":0.40699999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57099999999999995,"15min":0.68700000000000006},"min":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.78100000000000003,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57199999999999995,"15min":0.73099999999999998},"min":{"1min":0.371,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.69599999999999995,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.58199999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56499999999999995,"15min":0.71399999999999997},"min":{"1min":0.40100000000000002,"5min":0.376,"15min":0.36299999999999999},"max":{"1min":0.72699999999999998,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.74199999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.60399999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61499999999999999,"15min":0.77600000000000002},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.73299999999999998},"min":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.36599999999999999},"max":{"1min":0.82099999999999995,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.56899999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60899999999999999,"15min":0.72399999999999998},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.36299999999999999},"max":{"1min":0.85499999999999998,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.751},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59799999999999998,"15min":0.747},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89500000000000002,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.56100000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574955,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303448,"kb_used_data":2064,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068392,"statfs":{"total":96636764160,"available":96326033408,"internally_reserved":0,"allocated":2113536,"data_stored":938285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.311,"15min":0.32400000000000001},"min":{"1min":0.20200000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.48599999999999999,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32100000000000001,"15min":0.34899999999999998},"min":{"1min":0.224,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51800000000000002,"5min":0.54200000000000004,"15min":0.746},"last":0.505}]},{"osd":1,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33200000000000002,"15min":0.65200000000000002},"min":{"1min":0.22600000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52200000000000002,"5min":0.63,"15min":11.967000000000001},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.35199999999999998,"15min":0.45800000000000002},"min":{"1min":0.245,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.505,"5min":0.52700000000000002,"15min":4.1100000000000003},"last":0.28699999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.51600000000000001},"min":{"1min":0.23699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":6.3040000000000003},"last":0.308},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.38700000000000001,"15min":0.70399999999999996},"min":{"1min":0.29499999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57799999999999996,"5min":0.65900000000000003,"15min":11.935},"last":0.34899999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.42999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91900000000000004,"5min":1.302,"15min":1.302},"last":0.42299999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.378},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.95699999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.39700000000000002,"5min":0.35999999999999999,"15min":0.34300000000000003},"max":{"1min":0.78800000000000003,"5min":0.83999999999999997,"15min":0.96099999999999997},"last":0.47099999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.33400000000000002,"15min":0.33200000000000002},"max":{"1min":0.95499999999999996,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60399999999999998,"15min":0.60499999999999998},"min":{"1min":0.47499999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.54900000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.46100000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85699999999999998,"5min":0.89700000000000002,"15min":1.052},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58899999999999997,"15min":0.746},"min":{"1min":0.40999999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.98099999999999998,"5min":1.274,"15min":6.2649999999999997},"last":0.53900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303444,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068396,"statfs":{"total":96636764160,"available":96326037504,"internally_reserved":0,"allocated":2109440,"data_stored":938299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.309,"15min":0.314},"min":{"1min":0.22600000000000001,"5min":0.21299999999999999,"15min":0.21099999999999999},"max":{"1min":0.38900000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.316,"5min":0.32700000000000001,"15min":0.32100000000000001},"min":{"1min":0.23799999999999999,"5min":0.216,"15min":0.19800000000000001},"max":{"1min":0.42799999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.23999999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.34399999999999997,"15min":0.33400000000000002},"min":{"1min":0.26200000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.47299999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38300000000000001,"15min":0.378},"min":{"1min":0.26800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.48599999999999999,"5min":0.66900000000000004,"15min":0.72099999999999997},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.36299999999999999,"15min":0.35299999999999998},"min":{"1min":0.28899999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44800000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.28799999999999998,"5min":0.25900000000000001,"15min":0.255},"max":{"1min":0.441,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.36799999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56100000000000005,"15min":0.53800000000000003},"min":{"1min":0.44900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56699999999999995,"15min":0.53900000000000003},"min":{"1min":0.433,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.79400000000000004,"5min":0.83299999999999996,"15min":0.93500000000000005},"last":0.57599999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.55300000000000005},"min":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.35899999999999999},"max":{"1min":0.69599999999999995,"5min":1.347,"15min":1.347},"last":0.623},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.443,"5min":0.39300000000000002,"15min":0.35799999999999998},"max":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":1.01},"last":0.77400000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.45500000000000002,"5min":0.41099999999999998,"15min":0.36699999999999999},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.48899999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79700000000000004,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.79900000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61599999999999999,"15min":0.59599999999999997},"min":{"1min":0.495,"5min":0.42099999999999999,"15min":0.38200000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.96999999999999997},"last":0.78900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574953,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303432,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068408,"statfs":{"total":96636764160,"available":96326049792,"internally_reserved":0,"allocated":2097152,"data_stored":937247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.372,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66700000000000004,"5min":0.74199999999999999,"15min":0.747},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52500000000000002,"15min":0.71899999999999997},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.32700000000000001},"max":{"1min":0.68300000000000005,"5min":0.77600000000000002,"15min":7.6719999999999997},"last":0.43099999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52600000000000002,"15min":0.72399999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.755,"15min":7.7160000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.371,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":0.86799999999999999},"last":0.442}]},{"osd":2,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67800000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.749},"min":{"1min":0.39900000000000002,"5min":0.39700000000000002,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":7.7839999999999998},"last":0.50900000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.47999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74399999999999999,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.42999999999999999,"5min":0.39200000000000002,"15min":0.35499999999999998},"max":{"1min":0.71499999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47299999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.497},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.30099999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.54700000000000004},"min":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.32000000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.35399999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.77300000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.379},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86199999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.34399999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.378,"15min":0.36199999999999999},"max":{"1min":0.88600000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.37},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.41699999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.874,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.41999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303428,"kb_used_data":2044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068412,"statfs":{"total":96636764160,"available":96326053888,"internally_reserved":0,"allocated":2093056,"data_stored":936693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.498,"15min":0.49199999999999999},"min":{"1min":0.38300000000000001,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.69699999999999995,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53100000000000003,"15min":0.52000000000000002},"min":{"1min":0.39400000000000002,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.72399999999999998,"5min":0.76700000000000002,"15min":0.79300000000000004},"last":0.55500000000000005}]},{"osd":1,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51400000000000001,"15min":0.61799999999999999},"min":{"1min":0.35099999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70799999999999996,"5min":0.746,"15min":5.2679999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53000000000000003,"15min":0.61099999999999999},"min":{"1min":0.442,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.65700000000000003,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.42599999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.56200000000000006,"15min":0.63800000000000001},"min":{"1min":0.45500000000000002,"5min":0.32900000000000001,"15min":0.28299999999999997},"max":{"1min":3.27,"5min":3.27,"15min":5.2960000000000003},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55000000000000004,"15min":0.54100000000000004},"min":{"1min":0.40999999999999998,"5min":0.33900000000000002,"15min":0.32000000000000001},"max":{"1min":0.75,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.501}]},{"osd":3,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55700000000000005,"15min":0.65700000000000003},"min":{"1min":0.47499999999999998,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.69799999999999995,"5min":0.94599999999999995,"15min":5.3330000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57399999999999995,"15min":0.66800000000000004},"min":{"1min":0.45200000000000001,"5min":0.39300000000000002,"15min":0.27700000000000002},"max":{"1min":0.79300000000000004,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.52400000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.49199999999999999,"15min":0.497},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67300000000000004,"5min":0.72499999999999998,"15min":0.82999999999999996},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53100000000000003,"15min":0.54300000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32200000000000001},"max":{"1min":0.83099999999999996,"5min":0.88,"15min":1.0800000000000001},"last":0.39300000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.86499999999999999,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.48799999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56200000000000006,"15min":0.57499999999999996},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.30099999999999999},"max":{"1min":0.78700000000000003,"5min":0.98299999999999998,"15min":0.98399999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.378,"5min":0.377,"15min":0.377},"max":{"1min":0.83199999999999996,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.54600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:43:01.939 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:43:02.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.064+0000 7fec7755d700 1 Processor -- start 2023-10-10T20:43:02.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.064+0000 7fec7755d700 1 -- start start 2023-10-10T20:43:02.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.065+0000 7fec7755d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec70159c60 0x7fec7015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:02.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.065+0000 7fec7755d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fec7015db10 0x7fec70069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:02.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.065+0000 7fec7755d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec7015a160 con 0x7fec70159890 2023-10-10T20:43:02.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.065+0000 7fec7755d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec700d13a0 con 0x7fec7015db10 2023-10-10T20:43:02.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.065+0000 7fec7755d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7015e140 con 0x7fec70159c60 2023-10-10T20:43:02.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.066+0000 7fec75afa700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fec7015db10 0x7fec70069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:02.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.066+0000 7fec74af8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec70159c60 0x7fec7015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:02.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.066+0000 7fec752f9700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fec70159890 0x7fec7015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:57848/0 (socket says 172.21.15.89:57848) 2023-10-10T20:43:02.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.066+0000 7fec752f9700 1 -- 172.21.15.89:0/1621950267 learned_addr learned my addr 172.21.15.89:0/1621950267 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:02.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.066+0000 7fec67fff700 1 -- 172.21.15.89:0/1621950267 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76798844 0 0) 0x7fec7015a160 con 0x7fec70159890 2023-10-10T20:43:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.066+0000 7fec67fff700 1 -- 172.21.15.89:0/1621950267 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec5c003570 con 0x7fec70159890 2023-10-10T20:43:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.066+0000 7fec75afa700 1 -- 172.21.15.89:0/1621950267 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec70159c60 msgr2=0x7fec7015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.066+0000 7fec75afa700 1 --2- 172.21.15.89:0/1621950267 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec70159c60 0x7fec7015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.066+0000 7fec75afa700 1 -- 172.21.15.89:0/1621950267 >> v1:172.21.15.111:6789/0 conn(0x7fec70159890 legacy=0x7fec7015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.066+0000 7fec75afa700 1 -- 172.21.15.89:0/1621950267 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7015a3d0 con 0x7fec7015db10 2023-10-10T20:43:02.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.066+0000 7fec74af8700 1 --2- 172.21.15.89:0/1621950267 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec70159c60 0x7fec7015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:43:02.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.066+0000 7fec75afa700 1 --2- 172.21.15.89:0/1621950267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec7015db10 0x7fec70069560 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fec68009e60 tx=0x7fec68009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bbea1d651ec4c16 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:02.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.066+0000 7fec67fff700 1 -- 172.21.15.89:0/1621950267 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6800e070 con 0x7fec7015db10 2023-10-10T20:43:02.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.066+0000 7fec67fff700 1 -- 172.21.15.89:0/1621950267 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6800bab0 con 0x7fec7015db10 2023-10-10T20:43:02.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.066+0000 7fec67fff700 1 -- 172.21.15.89:0/1621950267 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6800be70 con 0x7fec7015db10 2023-10-10T20:43:02.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.067+0000 7fec7755d700 1 -- 172.21.15.89:0/1621950267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec7015db10 msgr2=0x7fec70069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:02.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.067+0000 7fec7755d700 1 --2- 172.21.15.89:0/1621950267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec7015db10 0x7fec70069560 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fec68009e60 tx=0x7fec68009780 comp rx=0 tx=0).stop 2023-10-10T20:43:02.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.067+0000 7fec7755d700 1 -- 172.21.15.89:0/1621950267 shutdown_connections 2023-10-10T20:43:02.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.067+0000 7fec7755d700 1 --2- 172.21.15.89:0/1621950267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec7015db10 0x7fec70069560 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.067+0000 7fec7755d700 1 --2- 172.21.15.89:0/1621950267 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec70159c60 0x7fec7015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.067+0000 7fec7755d700 1 -- 172.21.15.89:0/1621950267 >> 172.21.15.89:0/1621950267 conn(0x7fec70068e20 msgr2=0x7fec700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:02.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.067+0000 7fec7755d700 1 -- 172.21.15.89:0/1621950267 shutdown_connections 2023-10-10T20:43:02.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.067+0000 7fec7755d700 1 -- 172.21.15.89:0/1621950267 wait complete. 2023-10-10T20:43:02.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.069+0000 7fec7755d700 1 Processor -- start 2023-10-10T20:43:02.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.069+0000 7fec7755d700 1 -- start start 2023-10-10T20:43:02.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.070+0000 7fec7755d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec70159890 0x7fec70168df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:02.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.070+0000 7fec7755d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec70159c60 0x7fec700cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:02.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.070+0000 7fec7755d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec7015db10 0x7fec700ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:02.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.070+0000 7fec7755d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7015cc40 con 0x7fec7015db10 2023-10-10T20:43:02.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.070+0000 7fec7755d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7015cf20 con 0x7fec70159890 2023-10-10T20:43:02.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.070+0000 7fec7755d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7015d200 con 0x7fec70159c60 2023-10-10T20:43:02.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.070+0000 7fec75afa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec7015db10 0x7fec700ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:02.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.070+0000 7fec75afa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec7015db10 0x7fec700ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47420/0 (socket says 172.21.15.89:47420) 2023-10-10T20:43:02.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.070+0000 7fec75afa700 1 -- 172.21.15.89:0/3895619530 learned_addr learned my addr 172.21.15.89:0/3895619530 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:02.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.070+0000 7fec74af8700 1 --2- 172.21.15.89:0/3895619530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec70159c60 0x7fec700cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:02.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.070+0000 7fec752f9700 1 --2- 172.21.15.89:0/3895619530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec70159890 0x7fec70168df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:02.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.070+0000 7fec75afa700 1 -- 172.21.15.89:0/3895619530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec70159c60 msgr2=0x7fec700cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:02.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.070+0000 7fec75afa700 1 --2- 172.21.15.89:0/3895619530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec70159c60 0x7fec700cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.070+0000 7fec75afa700 1 -- 172.21.15.89:0/3895619530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec70159890 msgr2=0x7fec70168df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:02.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.070+0000 7fec75afa700 1 --2- 172.21.15.89:0/3895619530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec70159890 0x7fec70168df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.070+0000 7fec75afa700 1 -- 172.21.15.89:0/3895619530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec60009420 con 0x7fec7015db10 2023-10-10T20:43:02.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.070+0000 7fec75afa700 1 --2- 172.21.15.89:0/3895619530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec7015db10 0x7fec700ce0c0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fec700899c0 tx=0x7fec68019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:02.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.070+0000 7fec657fa700 1 -- 172.21.15.89:0/3895619530 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68027470 con 0x7fec7015db10 2023-10-10T20:43:02.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.070+0000 7fec657fa700 1 -- 172.21.15.89:0/3895619530 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec680275f0 con 0x7fec7015db10 2023-10-10T20:43:02.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.071+0000 7fec7755d700 1 -- 172.21.15.89:0/3895619530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec68009a20 con 0x7fec7015db10 2023-10-10T20:43:02.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.071+0000 7fec657fa700 1 -- 172.21.15.89:0/3895619530 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec680279b0 con 0x7fec7015db10 2023-10-10T20:43:02.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.071+0000 7fec7755d700 1 -- 172.21.15.89:0/3895619530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec701512d0 con 0x7fec7015db10 2023-10-10T20:43:02.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.072+0000 7fec7755d700 1 -- 172.21.15.89:0/3895619530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7015cd80 con 0x7fec7015db10 2023-10-10T20:43:02.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.072+0000 7fec657fa700 1 -- 172.21.15.89:0/3895619530 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fec68024070 con 0x7fec7015db10 2023-10-10T20:43:02.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.072+0000 7fec657fa700 1 --2- 172.21.15.89:0/3895619530 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fec54067320 0x7fec540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:02.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.073+0000 7fec752f9700 1 --2- 172.21.15.89:0/3895619530 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fec54067320 0x7fec540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:02.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.073+0000 7fec657fa700 1 -- 172.21.15.89:0/3895619530 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fec680875e0 con 0x7fec7015db10 2023-10-10T20:43:02.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.073+0000 7fec752f9700 1 --2- 172.21.15.89:0/3895619530 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fec54067320 0x7fec540697d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0058d0 tx=0x7fec6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:02.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.077+0000 7fec657fa700 1 -- 172.21.15.89:0/3895619530 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec7015cd80 con 0x7fec7015db10 2023-10-10T20:43:02.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.254+0000 7fec7755d700 1 -- 172.21.15.89:0/3895619530 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec7015cd80 con 0x7fec54067320 2023-10-10T20:43:02.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.262+0000 7fec657fa700 1 -- 172.21.15.89:0/3895619530 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289930 (secure 0 0 0) 0x7fec7015cd80 con 0x7fec54067320 2023-10-10T20:43:02.268 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:02.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.265+0000 7fec7755d700 1 -- 172.21.15.89:0/3895619530 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fec54067320 msgr2=0x7fec540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:02.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.265+0000 7fec7755d700 1 --2- 172.21.15.89:0/3895619530 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fec54067320 0x7fec540697d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0058d0 tx=0x7fec6c009040 comp rx=0 tx=0).stop 2023-10-10T20:43:02.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.265+0000 7fec7755d700 1 -- 172.21.15.89:0/3895619530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec7015db10 msgr2=0x7fec700ce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:02.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.265+0000 7fec7755d700 1 --2- 172.21.15.89:0/3895619530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec7015db10 0x7fec700ce0c0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fec700899c0 tx=0x7fec68019000 comp rx=0 tx=0).stop 2023-10-10T20:43:02.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.265+0000 7fec7755d700 1 -- 172.21.15.89:0/3895619530 shutdown_connections 2023-10-10T20:43:02.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.265+0000 7fec7755d700 1 --2- 172.21.15.89:0/3895619530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec7015db10 0x7fec700ce0c0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.265+0000 7fec7755d700 1 --2- 172.21.15.89:0/3895619530 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fec54067320 0x7fec540697d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.265+0000 7fec7755d700 1 --2- 172.21.15.89:0/3895619530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec70159890 0x7fec70168df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.265+0000 7fec7755d700 1 --2- 172.21.15.89:0/3895619530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec70159c60 0x7fec700cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.265+0000 7fec7755d700 1 -- 172.21.15.89:0/3895619530 >> 172.21.15.89:0/3895619530 conn(0x7fec70068e20 msgr2=0x7fec700b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:02.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.265+0000 7fec7755d700 1 -- 172.21.15.89:0/3895619530 shutdown_connections 2023-10-10T20:43:02.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.265+0000 7fec7755d700 1 -- 172.21.15.89:0/3895619530 wait complete. 2023-10-10T20:43:02.273 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:43:02.279 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2023-10-10T20:43:00.752208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428644,"kb_used_data":17572,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752546076,"statfs":{"total":773094113280,"available":770607181824,"internally_reserved":0,"allocated":17993728,"data_stored":8677931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001574"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.356154+0000","last_change":"2023-10-10T20:42:53.349889+0000","last_active":"2023-10-10T20:42:53.356154+0000","last_peered":"2023-10-10T20:42:53.356154+0000","last_clean":"2023-10-10T20:42:53.356154+0000","last_became_active":"2023-10-10T20:42:53.348786+0000","last_became_peered":"2023-10-10T20:42:53.348786+0000","last_unstale":"2023-10-10T20:42:53.356154+0000","last_undegraded":"2023-10-10T20:42:53.356154+0000","last_fullsized":"2023-10-10T20:42:53.356154+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:58:12.432427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.366952+0000","last_change":"2023-10-10T20:42:53.362906+0000","last_active":"2023-10-10T20:42:53.366952+0000","last_peered":"2023-10-10T20:42:53.366952+0000","last_clean":"2023-10-10T20:42:53.366952+0000","last_became_active":"2023-10-10T20:42:53.362294+0000","last_became_peered":"2023-10-10T20:42:53.362294+0000","last_unstale":"2023-10-10T20:42:53.366952+0000","last_undegraded":"2023-10-10T20:42:53.366952+0000","last_fullsized":"2023-10-10T20:42:53.366952+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:16:07.451863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.366851+0000","last_change":"2023-10-10T20:42:53.364479+0000","last_active":"2023-10-10T20:42:53.366851+0000","last_peered":"2023-10-10T20:42:53.366851+0000","last_clean":"2023-10-10T20:42:53.366851+0000","last_became_active":"2023-10-10T20:42:53.361443+0000","last_became_peered":"2023-10-10T20:42:53.361443+0000","last_unstale":"2023-10-10T20:42:53.366851+0000","last_undegraded":"2023-10-10T20:42:53.366851+0000","last_fullsized":"2023-10-10T20:42:53.366851+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:20:34.093326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.345401+0000","last_change":"2023-10-10T20:42:53.343259+0000","last_active":"2023-10-10T20:42:53.345401+0000","last_peered":"2023-10-10T20:42:53.345401+0000","last_clean":"2023-10-10T20:42:53.345401+0000","last_became_active":"2023-10-10T20:42:53.343002+0000","last_became_peered":"2023-10-10T20:42:53.343002+0000","last_unstale":"2023-10-10T20:42:53.345401+0000","last_undegraded":"2023-10-10T20:42:53.345401+0000","last_fullsized":"2023-10-10T20:42:53.345401+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:13:45.883755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.346024+0000","last_change":"2023-10-10T20:42:53.344218+0000","last_active":"2023-10-10T20:42:53.346024+0000","last_peered":"2023-10-10T20:42:53.346024+0000","last_clean":"2023-10-10T20:42:53.346024+0000","last_became_active":"2023-10-10T20:42:53.343971+0000","last_became_peered":"2023-10-10T20:42:53.343971+0000","last_unstale":"2023-10-10T20:42:53.346024+0000","last_undegraded":"2023-10-10T20:42:53.346024+0000","last_fullsized":"2023-10-10T20:42:53.346024+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:28:41.786703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.353207+0000","last_change":"2023-10-10T20:42:53.351313+0000","last_active":"2023-10-10T20:42:53.353207+0000","last_peered":"2023-10-10T20:42:53.353207+0000","last_clean":"2023-10-10T20:42:53.353207+0000","last_became_active":"2023-10-10T20:42:53.351064+0000","last_became_peered":"2023-10-10T20:42:53.351064+0000","last_unstale":"2023-10-10T20:42:53.353207+0000","last_undegraded":"2023-10-10T20:42:53.353207+0000","last_fullsized":"2023-10-10T20:42:53.353207+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:38:16.787078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.367121+0000","last_change":"2023-10-10T20:42:53.361375+0000","last_active":"2023-10-10T20:42:53.367121+0000","last_peered":"2023-10-10T20:42:53.367121+0000","last_clean":"2023-10-10T20:42:53.367121+0000","last_became_active":"2023-10-10T20:42:53.360948+0000","last_became_peered":"2023-10-10T20:42:53.360948+0000","last_unstale":"2023-10-10T20:42:53.367121+0000","last_undegraded":"2023-10-10T20:42:53.367121+0000","last_fullsized":"2023-10-10T20:42:53.367121+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:17:25.969866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.345885+0000","last_change":"2023-10-10T20:42:53.345087+0000","last_active":"2023-10-10T20:42:53.345885+0000","last_peered":"2023-10-10T20:42:53.345885+0000","last_clean":"2023-10-10T20:42:53.345885+0000","last_became_active":"2023-10-10T20:42:53.344838+0000","last_became_peered":"2023-10-10T20:42:53.344838+0000","last_unstale":"2023-10-10T20:42:53.345885+0000","last_undegraded":"2023-10-10T20:42:53.345885+0000","last_fullsized":"2023-10-10T20:42:53.345885+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:32.005980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365997+0000","last_change":"2023-10-10T20:42:53.358792+0000","last_active":"2023-10-10T20:42:53.365997+0000","last_peered":"2023-10-10T20:42:53.365997+0000","last_clean":"2023-10-10T20:42:53.365997+0000","last_became_active":"2023-10-10T20:42:53.358197+0000","last_became_peered":"2023-10-10T20:42:53.358197+0000","last_unstale":"2023-10-10T20:42:53.365997+0000","last_undegraded":"2023-10-10T20:42:53.365997+0000","last_fullsized":"2023-10-10T20:42:53.365997+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:45:05.557928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.348367+0000","last_change":"2023-10-10T20:42:53.347297+0000","last_active":"2023-10-10T20:42:53.348367+0000","last_peered":"2023-10-10T20:42:53.348367+0000","last_clean":"2023-10-10T20:42:53.348367+0000","last_became_active":"2023-10-10T20:42:53.347069+0000","last_became_peered":"2023-10-10T20:42:53.347069+0000","last_unstale":"2023-10-10T20:42:53.348367+0000","last_undegraded":"2023-10-10T20:42:53.348367+0000","last_fullsized":"2023-10-10T20:42:53.348367+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:18:15.714890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.356601+0000","last_change":"2023-10-10T20:42:53.348770+0000","last_active":"2023-10-10T20:42:53.356601+0000","last_peered":"2023-10-10T20:42:53.356601+0000","last_clean":"2023-10-10T20:42:53.356601+0000","last_became_active":"2023-10-10T20:42:53.348164+0000","last_became_peered":"2023-10-10T20:42:53.348164+0000","last_unstale":"2023-10-10T20:42:53.356601+0000","last_undegraded":"2023-10-10T20:42:53.356601+0000","last_fullsized":"2023-10-10T20:42:53.356601+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:18:15.714890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.359052+0000","last_change":"2023-10-10T20:42:53.345901+0000","last_active":"2023-10-10T20:42:53.359052+0000","last_peered":"2023-10-10T20:42:53.359052+0000","last_clean":"2023-10-10T20:42:53.359052+0000","last_became_active":"2023-10-10T20:42:53.345399+0000","last_became_peered":"2023-10-10T20:42:53.345399+0000","last_unstale":"2023-10-10T20:42:53.359052+0000","last_undegraded":"2023-10-10T20:42:53.359052+0000","last_fullsized":"2023-10-10T20:42:53.359052+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:02:55.597436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.361665+0000","last_change":"2023-10-10T20:42:53.355221+0000","last_active":"2023-10-10T20:42:53.361665+0000","last_peered":"2023-10-10T20:42:53.361665+0000","last_clean":"2023-10-10T20:42:53.361665+0000","last_became_active":"2023-10-10T20:42:53.354313+0000","last_became_peered":"2023-10-10T20:42:53.354313+0000","last_unstale":"2023-10-10T20:42:53.361665+0000","last_undegraded":"2023-10-10T20:42:53.361665+0000","last_fullsized":"2023-10-10T20:42:53.361665+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:05:24.092738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.353642+0000","last_change":"2023-10-10T20:42:53.351579+0000","last_active":"2023-10-10T20:42:53.353642+0000","last_peered":"2023-10-10T20:42:53.353642+0000","last_clean":"2023-10-10T20:42:53.353642+0000","last_became_active":"2023-10-10T20:42:53.351020+0000","last_became_peered":"2023-10-10T20:42:53.351020+0000","last_unstale":"2023-10-10T20:42:53.353642+0000","last_undegraded":"2023-10-10T20:42:53.353642+0000","last_fullsized":"2023-10-10T20:42:53.353642+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:39:39.656970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.363457+0000","last_change":"2023-10-10T20:42:53.362369+0000","last_active":"2023-10-10T20:42:53.363457+0000","last_peered":"2023-10-10T20:42:53.363457+0000","last_clean":"2023-10-10T20:42:53.363457+0000","last_became_active":"2023-10-10T20:42:53.362086+0000","last_became_peered":"2023-10-10T20:42:53.362086+0000","last_unstale":"2023-10-10T20:42:53.363457+0000","last_undegraded":"2023-10-10T20:42:53.363457+0000","last_fullsized":"2023-10-10T20:42:53.363457+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:34:03.701694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.361631+0000","last_change":"2023-10-10T20:42:53.354983+0000","last_active":"2023-10-10T20:42:53.361631+0000","last_peered":"2023-10-10T20:42:53.361631+0000","last_clean":"2023-10-10T20:42:53.361631+0000","last_became_active":"2023-10-10T20:42:53.346734+0000","last_became_peered":"2023-10-10T20:42:53.346734+0000","last_unstale":"2023-10-10T20:42:53.361631+0000","last_undegraded":"2023-10-10T20:42:53.361631+0000","last_fullsized":"2023-10-10T20:42:53.361631+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:34:46.022458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.353689+0000","last_change":"2023-10-10T20:42:53.351355+0000","last_active":"2023-10-10T20:42:53.353689+0000","last_peered":"2023-10-10T20:42:53.353689+0000","last_clean":"2023-10-10T20:42:53.353689+0000","last_became_active":"2023-10-10T20:42:53.351007+0000","last_became_peered":"2023-10-10T20:42:53.351007+0000","last_unstale":"2023-10-10T20:42:53.353689+0000","last_undegraded":"2023-10-10T20:42:53.353689+0000","last_fullsized":"2023-10-10T20:42:53.353689+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:58:16.268958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.362012+0000","last_change":"2023-10-10T20:42:53.361475+0000","last_active":"2023-10-10T20:42:53.362012+0000","last_peered":"2023-10-10T20:42:53.362012+0000","last_clean":"2023-10-10T20:42:53.362012+0000","last_became_active":"2023-10-10T20:42:53.361204+0000","last_became_peered":"2023-10-10T20:42:53.361204+0000","last_unstale":"2023-10-10T20:42:53.362012+0000","last_undegraded":"2023-10-10T20:42:53.362012+0000","last_fullsized":"2023-10-10T20:42:53.362012+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:00:48.564823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.358911+0000","last_change":"2023-10-10T20:42:53.357353+0000","last_active":"2023-10-10T20:42:53.358911+0000","last_peered":"2023-10-10T20:42:53.358911+0000","last_clean":"2023-10-10T20:42:53.358911+0000","last_became_active":"2023-10-10T20:42:53.357048+0000","last_became_peered":"2023-10-10T20:42:53.357048+0000","last_unstale":"2023-10-10T20:42:53.358911+0000","last_undegraded":"2023-10-10T20:42:53.358911+0000","last_fullsized":"2023-10-10T20:42:53.358911+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:58:00.782955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.339743+0000","last_change":"2023-10-10T20:42:53.337807+0000","last_active":"2023-10-10T20:42:53.339743+0000","last_peered":"2023-10-10T20:42:53.339743+0000","last_clean":"2023-10-10T20:42:53.339743+0000","last_became_active":"2023-10-10T20:42:53.337553+0000","last_became_peered":"2023-10-10T20:42:53.337553+0000","last_unstale":"2023-10-10T20:42:53.339743+0000","last_undegraded":"2023-10-10T20:42:53.339743+0000","last_fullsized":"2023-10-10T20:42:53.339743+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:13:20.878853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.358934+0000","last_change":"2023-10-10T20:42:53.358025+0000","last_active":"2023-10-10T20:42:53.358934+0000","last_peered":"2023-10-10T20:42:53.358934+0000","last_clean":"2023-10-10T20:42:53.358934+0000","last_became_active":"2023-10-10T20:42:53.355720+0000","last_became_peered":"2023-10-10T20:42:53.355720+0000","last_unstale":"2023-10-10T20:42:53.358934+0000","last_undegraded":"2023-10-10T20:42:53.358934+0000","last_fullsized":"2023-10-10T20:42:53.358934+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:25:50.618140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.344237+0000","last_change":"2023-10-10T20:42:53.342434+0000","last_active":"2023-10-10T20:42:53.344237+0000","last_peered":"2023-10-10T20:42:53.344237+0000","last_clean":"2023-10-10T20:42:53.344237+0000","last_became_active":"2023-10-10T20:42:53.342152+0000","last_became_peered":"2023-10-10T20:42:53.342152+0000","last_unstale":"2023-10-10T20:42:53.344237+0000","last_undegraded":"2023-10-10T20:42:53.344237+0000","last_fullsized":"2023-10-10T20:42:53.344237+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:57.859353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.355929+0000","last_change":"2023-10-10T20:42:53.345842+0000","last_active":"2023-10-10T20:42:53.355929+0000","last_peered":"2023-10-10T20:42:53.355929+0000","last_clean":"2023-10-10T20:42:53.355929+0000","last_became_active":"2023-10-10T20:42:53.345588+0000","last_became_peered":"2023-10-10T20:42:53.345588+0000","last_unstale":"2023-10-10T20:42:53.355929+0000","last_undegraded":"2023-10-10T20:42:53.355929+0000","last_fullsized":"2023-10-10T20:42:53.355929+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:22:18.325313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.347911+0000","last_change":"2023-10-10T20:42:53.345923+0000","last_active":"2023-10-10T20:42:53.347911+0000","last_peered":"2023-10-10T20:42:53.347911+0000","last_clean":"2023-10-10T20:42:53.347911+0000","last_became_active":"2023-10-10T20:42:53.345677+0000","last_became_peered":"2023-10-10T20:42:53.345677+0000","last_unstale":"2023-10-10T20:42:53.347911+0000","last_undegraded":"2023-10-10T20:42:53.347911+0000","last_fullsized":"2023-10-10T20:42:53.347911+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:35.275237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.356172+0000","last_change":"2023-10-10T20:42:53.350069+0000","last_active":"2023-10-10T20:42:53.356172+0000","last_peered":"2023-10-10T20:42:53.356172+0000","last_clean":"2023-10-10T20:42:53.356172+0000","last_became_active":"2023-10-10T20:42:53.349120+0000","last_became_peered":"2023-10-10T20:42:53.349120+0000","last_unstale":"2023-10-10T20:42:53.356172+0000","last_undegraded":"2023-10-10T20:42:53.356172+0000","last_fullsized":"2023-10-10T20:42:53.356172+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:13:20.878853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.353709+0000","last_change":"2023-10-10T20:42:53.352048+0000","last_active":"2023-10-10T20:42:53.353709+0000","last_peered":"2023-10-10T20:42:53.353709+0000","last_clean":"2023-10-10T20:42:53.353709+0000","last_became_active":"2023-10-10T20:42:53.351745+0000","last_became_peered":"2023-10-10T20:42:53.351745+0000","last_unstale":"2023-10-10T20:42:53.353709+0000","last_undegraded":"2023-10-10T20:42:53.353709+0000","last_fullsized":"2023-10-10T20:42:53.353709+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:19:38.741623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.356621+0000","last_change":"2023-10-10T20:42:53.348956+0000","last_active":"2023-10-10T20:42:53.356621+0000","last_peered":"2023-10-10T20:42:53.356621+0000","last_clean":"2023-10-10T20:42:53.356621+0000","last_became_active":"2023-10-10T20:42:53.348502+0000","last_became_peered":"2023-10-10T20:42:53.348502+0000","last_unstale":"2023-10-10T20:42:53.356621+0000","last_undegraded":"2023-10-10T20:42:53.356621+0000","last_fullsized":"2023-10-10T20:42:53.356621+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:38:16.787078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.345439+0000","last_change":"2023-10-10T20:42:53.344413+0000","last_active":"2023-10-10T20:42:53.345439+0000","last_peered":"2023-10-10T20:42:53.345439+0000","last_clean":"2023-10-10T20:42:53.345439+0000","last_became_active":"2023-10-10T20:42:53.344073+0000","last_became_peered":"2023-10-10T20:42:53.344073+0000","last_unstale":"2023-10-10T20:42:53.345439+0000","last_undegraded":"2023-10-10T20:42:53.345439+0000","last_fullsized":"2023-10-10T20:42:53.345439+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:57.859353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.362917+0000","last_change":"2023-10-10T20:42:53.362202+0000","last_active":"2023-10-10T20:42:53.362917+0000","last_peered":"2023-10-10T20:42:53.362917+0000","last_clean":"2023-10-10T20:42:53.362917+0000","last_became_active":"2023-10-10T20:42:53.361913+0000","last_became_peered":"2023-10-10T20:42:53.361913+0000","last_unstale":"2023-10-10T20:42:53.362917+0000","last_undegraded":"2023-10-10T20:42:53.362917+0000","last_fullsized":"2023-10-10T20:42:53.362917+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:39:12.228861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.363199+0000","last_change":"2023-10-10T20:42:53.362071+0000","last_active":"2023-10-10T20:42:53.363199+0000","last_peered":"2023-10-10T20:42:53.363199+0000","last_clean":"2023-10-10T20:42:53.363199+0000","last_became_active":"2023-10-10T20:42:53.361448+0000","last_became_peered":"2023-10-10T20:42:53.361448+0000","last_unstale":"2023-10-10T20:42:53.363199+0000","last_undegraded":"2023-10-10T20:42:53.363199+0000","last_fullsized":"2023-10-10T20:42:53.363199+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:52.463061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.353395+0000","last_change":"2023-10-10T20:42:53.350979+0000","last_active":"2023-10-10T20:42:53.353395+0000","last_peered":"2023-10-10T20:42:53.353395+0000","last_clean":"2023-10-10T20:42:53.353395+0000","last_became_active":"2023-10-10T20:42:53.350702+0000","last_became_peered":"2023-10-10T20:42:53.350702+0000","last_unstale":"2023-10-10T20:42:53.353395+0000","last_undegraded":"2023-10-10T20:42:53.353395+0000","last_fullsized":"2023-10-10T20:42:53.353395+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:17:25.969866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365482+0000","last_change":"2023-10-10T20:42:53.364870+0000","last_active":"2023-10-10T20:42:53.365482+0000","last_peered":"2023-10-10T20:42:53.365482+0000","last_clean":"2023-10-10T20:42:53.365482+0000","last_became_active":"2023-10-10T20:42:53.364542+0000","last_became_peered":"2023-10-10T20:42:53.364542+0000","last_unstale":"2023-10-10T20:42:53.365482+0000","last_undegraded":"2023-10-10T20:42:53.365482+0000","last_fullsized":"2023-10-10T20:42:53.365482+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:27:57.599666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365999+0000","last_change":"2023-10-10T20:42:53.360038+0000","last_active":"2023-10-10T20:42:53.365999+0000","last_peered":"2023-10-10T20:42:53.365999+0000","last_clean":"2023-10-10T20:42:53.365999+0000","last_became_active":"2023-10-10T20:42:53.357757+0000","last_became_peered":"2023-10-10T20:42:53.357757+0000","last_unstale":"2023-10-10T20:42:53.365999+0000","last_undegraded":"2023-10-10T20:42:53.365999+0000","last_fullsized":"2023-10-10T20:42:53.365999+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:16:07.451863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"164'4","reported_seq":14,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.366984+0000","last_change":"2023-10-10T20:42:52.335968+0000","last_active":"2023-10-10T20:42:53.366984+0000","last_peered":"2023-10-10T20:42:53.366984+0000","last_clean":"2023-10-10T20:42:53.366984+0000","last_became_active":"2023-10-10T20:42:52.335695+0000","last_became_peered":"2023-10-10T20:42:52.335695+0000","last_unstale":"2023-10-10T20:42:53.366984+0000","last_undegraded":"2023-10-10T20:42:53.366984+0000","last_fullsized":"2023-10-10T20:42:53.366984+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:32:07.233388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.358522+0000","last_change":"2023-10-10T20:42:53.356868+0000","last_active":"2023-10-10T20:42:53.358522+0000","last_peered":"2023-10-10T20:42:53.358522+0000","last_clean":"2023-10-10T20:42:53.358522+0000","last_became_active":"2023-10-10T20:42:53.356579+0000","last_became_peered":"2023-10-10T20:42:53.356579+0000","last_unstale":"2023-10-10T20:42:53.358522+0000","last_undegraded":"2023-10-10T20:42:53.358522+0000","last_fullsized":"2023-10-10T20:42:53.358522+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:29:45.982326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.361125+0000","last_change":"2023-10-10T20:42:52.336051+0000","last_active":"2023-10-10T20:42:53.361125+0000","last_peered":"2023-10-10T20:42:53.361125+0000","last_clean":"2023-10-10T20:42:53.361125+0000","last_became_active":"2023-10-10T20:42:52.335752+0000","last_became_peered":"2023-10-10T20:42:52.335752+0000","last_unstale":"2023-10-10T20:42:53.361125+0000","last_undegraded":"2023-10-10T20:42:53.361125+0000","last_fullsized":"2023-10-10T20:42:53.361125+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:15:01.473071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.362948+0000","last_change":"2023-10-10T20:42:53.362172+0000","last_active":"2023-10-10T20:42:53.362948+0000","last_peered":"2023-10-10T20:42:53.362948+0000","last_clean":"2023-10-10T20:42:53.362948+0000","last_became_active":"2023-10-10T20:42:53.361874+0000","last_became_peered":"2023-10-10T20:42:53.361874+0000","last_unstale":"2023-10-10T20:42:53.362948+0000","last_undegraded":"2023-10-10T20:42:53.362948+0000","last_fullsized":"2023-10-10T20:42:53.362948+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:38:22.597643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1c","version":"164'2","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365918+0000","last_change":"2023-10-10T20:42:52.339512+0000","last_active":"2023-10-10T20:42:53.365918+0000","last_peered":"2023-10-10T20:42:53.365918+0000","last_clean":"2023-10-10T20:42:53.365918+0000","last_became_active":"2023-10-10T20:42:52.339230+0000","last_became_peered":"2023-10-10T20:42:52.339230+0000","last_unstale":"2023-10-10T20:42:53.365918+0000","last_undegraded":"2023-10-10T20:42:53.365918+0000","last_fullsized":"2023-10-10T20:42:53.365918+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:51.664988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.363218+0000","last_change":"2023-10-10T20:42:53.362266+0000","last_active":"2023-10-10T20:42:53.363218+0000","last_peered":"2023-10-10T20:42:53.363218+0000","last_clean":"2023-10-10T20:42:53.363218+0000","last_became_active":"2023-10-10T20:42:53.361814+0000","last_became_peered":"2023-10-10T20:42:53.361814+0000","last_unstale":"2023-10-10T20:42:53.363218+0000","last_undegraded":"2023-10-10T20:42:53.363218+0000","last_fullsized":"2023-10-10T20:42:53.363218+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:33:29.535005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.354522+0000","last_change":"2023-10-10T20:42:52.332410+0000","last_active":"2023-10-10T20:42:53.354522+0000","last_peered":"2023-10-10T20:42:53.354522+0000","last_clean":"2023-10-10T20:42:53.354522+0000","last_became_active":"2023-10-10T20:42:52.331952+0000","last_became_peered":"2023-10-10T20:42:52.331952+0000","last_unstale":"2023-10-10T20:42:53.354522+0000","last_undegraded":"2023-10-10T20:42:53.354522+0000","last_fullsized":"2023-10-10T20:42:53.354522+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:46:55.907438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.362552+0000","last_change":"2023-10-10T20:42:53.362108+0000","last_active":"2023-10-10T20:42:53.362552+0000","last_peered":"2023-10-10T20:42:53.362552+0000","last_clean":"2023-10-10T20:42:53.362552+0000","last_became_active":"2023-10-10T20:42:53.361846+0000","last_became_peered":"2023-10-10T20:42:53.361846+0000","last_unstale":"2023-10-10T20:42:53.362552+0000","last_undegraded":"2023-10-10T20:42:53.362552+0000","last_fullsized":"2023-10-10T20:42:53.362552+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:20:34.093326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1a","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.364637+0000","last_change":"2023-10-10T20:42:52.338841+0000","last_active":"2023-10-10T20:42:53.364637+0000","last_peered":"2023-10-10T20:42:53.364637+0000","last_clean":"2023-10-10T20:42:53.364637+0000","last_became_active":"2023-10-10T20:42:52.337405+0000","last_became_peered":"2023-10-10T20:42:52.337405+0000","last_unstale":"2023-10-10T20:42:53.364637+0000","last_undegraded":"2023-10-10T20:42:53.364637+0000","last_fullsized":"2023-10-10T20:42:53.364637+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:10:49.794184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.346996+0000","last_change":"2023-10-10T20:42:53.345087+0000","last_active":"2023-10-10T20:42:53.346996+0000","last_peered":"2023-10-10T20:42:53.346996+0000","last_clean":"2023-10-10T20:42:53.346996+0000","last_became_active":"2023-10-10T20:42:53.344387+0000","last_became_peered":"2023-10-10T20:42:53.344387+0000","last_unstale":"2023-10-10T20:42:53.346996+0000","last_undegraded":"2023-10-10T20:42:53.346996+0000","last_fullsized":"2023-10-10T20:42:53.346996+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:34:03.701694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.340733+0000","last_change":"2023-10-10T20:42:52.333926+0000","last_active":"2023-10-10T20:42:53.340733+0000","last_peered":"2023-10-10T20:42:53.340733+0000","last_clean":"2023-10-10T20:42:53.340733+0000","last_became_active":"2023-10-10T20:42:52.333645+0000","last_became_peered":"2023-10-10T20:42:52.333645+0000","last_unstale":"2023-10-10T20:42:53.340733+0000","last_undegraded":"2023-10-10T20:42:53.340733+0000","last_fullsized":"2023-10-10T20:42:53.340733+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:46:47.379359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.366967+0000","last_change":"2023-10-10T20:42:53.363094+0000","last_active":"2023-10-10T20:42:53.366967+0000","last_peered":"2023-10-10T20:42:53.366967+0000","last_clean":"2023-10-10T20:42:53.366967+0000","last_became_active":"2023-10-10T20:42:53.362773+0000","last_became_peered":"2023-10-10T20:42:53.362773+0000","last_unstale":"2023-10-10T20:42:53.366967+0000","last_undegraded":"2023-10-10T20:42:53.366967+0000","last_fullsized":"2023-10-10T20:42:53.366967+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:00.861926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.336097+0000","last_change":"2023-10-10T20:42:52.331824+0000","last_active":"2023-10-10T20:42:53.336097+0000","last_peered":"2023-10-10T20:42:53.336097+0000","last_clean":"2023-10-10T20:42:53.336097+0000","last_became_active":"2023-10-10T20:42:52.331567+0000","last_became_peered":"2023-10-10T20:42:52.331567+0000","last_unstale":"2023-10-10T20:42:53.336097+0000","last_undegraded":"2023-10-10T20:42:53.336097+0000","last_fullsized":"2023-10-10T20:42:53.336097+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:59.776219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.366974+0000","last_change":"2023-10-10T20:42:53.363090+0000","last_active":"2023-10-10T20:42:53.366974+0000","last_peered":"2023-10-10T20:42:53.366974+0000","last_clean":"2023-10-10T20:42:53.366974+0000","last_became_active":"2023-10-10T20:42:53.362679+0000","last_became_peered":"2023-10-10T20:42:53.362679+0000","last_unstale":"2023-10-10T20:42:53.366974+0000","last_undegraded":"2023-10-10T20:42:53.366974+0000","last_fullsized":"2023-10-10T20:42:53.366974+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:09:34.459455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.336778+0000","last_change":"2023-10-10T20:42:52.329973+0000","last_active":"2023-10-10T20:42:53.336778+0000","last_peered":"2023-10-10T20:42:53.336778+0000","last_clean":"2023-10-10T20:42:53.336778+0000","last_became_active":"2023-10-10T20:42:52.329001+0000","last_became_peered":"2023-10-10T20:42:52.329001+0000","last_unstale":"2023-10-10T20:42:53.336778+0000","last_undegraded":"2023-10-10T20:42:53.336778+0000","last_fullsized":"2023-10-10T20:42:53.336778+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:20:37.621153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.356128+0000","last_change":"2023-10-10T20:42:53.349126+0000","last_active":"2023-10-10T20:42:53.356128+0000","last_peered":"2023-10-10T20:42:53.356128+0000","last_clean":"2023-10-10T20:42:53.356128+0000","last_became_active":"2023-10-10T20:42:53.344703+0000","last_became_peered":"2023-10-10T20:42:53.344703+0000","last_unstale":"2023-10-10T20:42:53.356128+0000","last_undegraded":"2023-10-10T20:42:53.356128+0000","last_fullsized":"2023-10-10T20:42:53.356128+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:21:27.171367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.333312+0000","last_change":"2023-10-10T20:42:52.334832+0000","last_active":"2023-10-10T20:42:53.333312+0000","last_peered":"2023-10-10T20:42:53.333312+0000","last_clean":"2023-10-10T20:42:53.333312+0000","last_became_active":"2023-10-10T20:42:52.334555+0000","last_became_peered":"2023-10-10T20:42:52.334555+0000","last_unstale":"2023-10-10T20:42:53.333312+0000","last_undegraded":"2023-10-10T20:42:53.333312+0000","last_fullsized":"2023-10-10T20:42:53.333312+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:51.456325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.362400+0000","last_change":"2023-10-10T20:42:53.360965+0000","last_active":"2023-10-10T20:42:53.362400+0000","last_peered":"2023-10-10T20:42:53.362400+0000","last_clean":"2023-10-10T20:42:53.362400+0000","last_became_active":"2023-10-10T20:42:53.360601+0000","last_became_peered":"2023-10-10T20:42:53.360601+0000","last_unstale":"2023-10-10T20:42:53.362400+0000","last_undegraded":"2023-10-10T20:42:53.362400+0000","last_fullsized":"2023-10-10T20:42:53.362400+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:52:20.890355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.15","version":"164'3","reported_seq":13,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.355795+0000","last_change":"2023-10-10T20:42:52.341985+0000","last_active":"2023-10-10T20:42:53.355795+0000","last_peered":"2023-10-10T20:42:53.355795+0000","last_clean":"2023-10-10T20:42:53.355795+0000","last_became_active":"2023-10-10T20:42:52.341730+0000","last_became_peered":"2023-10-10T20:42:52.341730+0000","last_unstale":"2023-10-10T20:42:53.355795+0000","last_undegraded":"2023-10-10T20:42:53.355795+0000","last_fullsized":"2023-10-10T20:42:53.355795+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:32:48.988699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.356638+0000","last_change":"2023-10-10T20:42:53.349828+0000","last_active":"2023-10-10T20:42:53.356638+0000","last_peered":"2023-10-10T20:42:53.356638+0000","last_clean":"2023-10-10T20:42:53.356638+0000","last_became_active":"2023-10-10T20:42:53.349514+0000","last_became_peered":"2023-10-10T20:42:53.349514+0000","last_unstale":"2023-10-10T20:42:53.356638+0000","last_undegraded":"2023-10-10T20:42:53.356638+0000","last_fullsized":"2023-10-10T20:42:53.356638+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:57:12.184703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"164'6","reported_seq":16,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.356757+0000","last_change":"2023-10-10T20:42:52.333830+0000","last_active":"2023-10-10T20:42:53.356757+0000","last_peered":"2023-10-10T20:42:53.356757+0000","last_clean":"2023-10-10T20:42:53.356757+0000","last_became_active":"2023-10-10T20:42:52.333570+0000","last_became_peered":"2023-10-10T20:42:52.333570+0000","last_unstale":"2023-10-10T20:42:53.356757+0000","last_undegraded":"2023-10-10T20:42:53.356757+0000","last_fullsized":"2023-10-10T20:42:53.356757+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:31.572538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.348179+0000","last_change":"2023-10-10T20:42:53.346232+0000","last_active":"2023-10-10T20:42:53.348179+0000","last_peered":"2023-10-10T20:42:53.348179+0000","last_clean":"2023-10-10T20:42:53.348179+0000","last_became_active":"2023-10-10T20:42:53.345920+0000","last_became_peered":"2023-10-10T20:42:53.345920+0000","last_unstale":"2023-10-10T20:42:53.348179+0000","last_undegraded":"2023-10-10T20:42:53.348179+0000","last_fullsized":"2023-10-10T20:42:53.348179+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:48:47.062433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"164'2","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.366389+0000","last_change":"2023-10-10T20:42:52.339386+0000","last_active":"2023-10-10T20:42:53.366389+0000","last_peered":"2023-10-10T20:42:53.366389+0000","last_clean":"2023-10-10T20:42:53.366389+0000","last_became_active":"2023-10-10T20:42:52.339072+0000","last_became_peered":"2023-10-10T20:42:52.339072+0000","last_unstale":"2023-10-10T20:42:53.366389+0000","last_undegraded":"2023-10-10T20:42:53.366389+0000","last_fullsized":"2023-10-10T20:42:53.366389+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:53:19.603938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.367107+0000","last_change":"2023-10-10T20:42:53.364661+0000","last_active":"2023-10-10T20:42:53.367107+0000","last_peered":"2023-10-10T20:42:53.367107+0000","last_clean":"2023-10-10T20:42:53.367107+0000","last_became_active":"2023-10-10T20:42:53.361122+0000","last_became_peered":"2023-10-10T20:42:53.361122+0000","last_unstale":"2023-10-10T20:42:53.367107+0000","last_undegraded":"2023-10-10T20:42:53.367107+0000","last_fullsized":"2023-10-10T20:42:53.367107+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:58:00.782955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.333584+0000","last_change":"2023-10-10T20:42:52.337702+0000","last_active":"2023-10-10T20:42:53.333584+0000","last_peered":"2023-10-10T20:42:53.333584+0000","last_clean":"2023-10-10T20:42:53.333584+0000","last_became_active":"2023-10-10T20:42:52.337415+0000","last_became_peered":"2023-10-10T20:42:52.337415+0000","last_unstale":"2023-10-10T20:42:53.333584+0000","last_undegraded":"2023-10-10T20:42:53.333584+0000","last_fullsized":"2023-10-10T20:42:53.333584+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:33:28.528269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.367441+0000","last_change":"2023-10-10T20:42:53.365582+0000","last_active":"2023-10-10T20:42:53.367441+0000","last_peered":"2023-10-10T20:42:53.367441+0000","last_clean":"2023-10-10T20:42:53.367441+0000","last_became_active":"2023-10-10T20:42:53.365256+0000","last_became_peered":"2023-10-10T20:42:53.365256+0000","last_unstale":"2023-10-10T20:42:53.367441+0000","last_undegraded":"2023-10-10T20:42:53.367441+0000","last_fullsized":"2023-10-10T20:42:53.367441+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:25:50.618140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.336708+0000","last_change":"2023-10-10T20:42:52.338838+0000","last_active":"2023-10-10T20:42:53.336708+0000","last_peered":"2023-10-10T20:42:53.336708+0000","last_clean":"2023-10-10T20:42:53.336708+0000","last_became_active":"2023-10-10T20:42:52.338592+0000","last_became_peered":"2023-10-10T20:42:52.338592+0000","last_unstale":"2023-10-10T20:42:53.336708+0000","last_undegraded":"2023-10-10T20:42:53.336708+0000","last_fullsized":"2023-10-10T20:42:53.336708+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:36:29.288624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.351267+0000","last_change":"2023-10-10T20:42:53.349965+0000","last_active":"2023-10-10T20:42:53.351267+0000","last_peered":"2023-10-10T20:42:53.351267+0000","last_clean":"2023-10-10T20:42:53.351267+0000","last_became_active":"2023-10-10T20:42:53.349675+0000","last_became_peered":"2023-10-10T20:42:53.349675+0000","last_unstale":"2023-10-10T20:42:53.351267+0000","last_undegraded":"2023-10-10T20:42:53.351267+0000","last_fullsized":"2023-10-10T20:42:53.351267+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:54:44.047869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.10","version":"164'8","reported_seq":18,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.367445+0000","last_change":"2023-10-10T20:42:52.334750+0000","last_active":"2023-10-10T20:42:53.367445+0000","last_peered":"2023-10-10T20:42:53.367445+0000","last_clean":"2023-10-10T20:42:53.367445+0000","last_became_active":"2023-10-10T20:42:52.334260+0000","last_became_peered":"2023-10-10T20:42:52.334260+0000","last_unstale":"2023-10-10T20:42:53.367445+0000","last_undegraded":"2023-10-10T20:42:53.367445+0000","last_fullsized":"2023-10-10T20:42:53.367445+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:49:39.517807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.350525+0000","last_change":"2023-10-10T20:42:53.349062+0000","last_active":"2023-10-10T20:42:53.350525+0000","last_peered":"2023-10-10T20:42:53.350525+0000","last_clean":"2023-10-10T20:42:53.350525+0000","last_became_active":"2023-10-10T20:42:53.348806+0000","last_became_peered":"2023-10-10T20:42:53.348806+0000","last_unstale":"2023-10-10T20:42:53.350525+0000","last_undegraded":"2023-10-10T20:42:53.350525+0000","last_fullsized":"2023-10-10T20:42:53.350525+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:46:32.614897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365941+0000","last_change":"2023-10-10T20:42:52.339183+0000","last_active":"2023-10-10T20:42:53.365941+0000","last_peered":"2023-10-10T20:42:53.365941+0000","last_clean":"2023-10-10T20:42:53.365941+0000","last_became_active":"2023-10-10T20:42:52.337722+0000","last_became_peered":"2023-10-10T20:42:52.337722+0000","last_unstale":"2023-10-10T20:42:53.365941+0000","last_undegraded":"2023-10-10T20:42:53.365941+0000","last_fullsized":"2023-10-10T20:42:53.365941+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:51.664988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365801+0000","last_change":"2023-10-10T20:42:53.365349+0000","last_active":"2023-10-10T20:42:53.365801+0000","last_peered":"2023-10-10T20:42:53.365801+0000","last_clean":"2023-10-10T20:42:53.365801+0000","last_became_active":"2023-10-10T20:42:53.365103+0000","last_became_peered":"2023-10-10T20:42:53.365103+0000","last_unstale":"2023-10-10T20:42:53.365801+0000","last_undegraded":"2023-10-10T20:42:53.365801+0000","last_fullsized":"2023-10-10T20:42:53.365801+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:48:47.062433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"164'2","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.358150+0000","last_change":"2023-10-10T20:42:52.335410+0000","last_active":"2023-10-10T20:42:53.358150+0000","last_peered":"2023-10-10T20:42:53.358150+0000","last_clean":"2023-10-10T20:42:53.358150+0000","last_became_active":"2023-10-10T20:42:52.335097+0000","last_became_peered":"2023-10-10T20:42:52.335097+0000","last_unstale":"2023-10-10T20:42:53.358150+0000","last_undegraded":"2023-10-10T20:42:53.358150+0000","last_fullsized":"2023-10-10T20:42:53.358150+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:22:17.318577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365826+0000","last_change":"2023-10-10T20:42:53.359692+0000","last_active":"2023-10-10T20:42:53.365826+0000","last_peered":"2023-10-10T20:42:53.365826+0000","last_clean":"2023-10-10T20:42:53.365826+0000","last_became_active":"2023-10-10T20:42:53.359187+0000","last_became_peered":"2023-10-10T20:42:53.359187+0000","last_unstale":"2023-10-10T20:42:53.365826+0000","last_undegraded":"2023-10-10T20:42:53.365826+0000","last_fullsized":"2023-10-10T20:42:53.365826+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:28:41.786703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.d","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.356705+0000","last_change":"2023-10-10T20:42:52.332490+0000","last_active":"2023-10-10T20:42:53.356705+0000","last_peered":"2023-10-10T20:42:53.356705+0000","last_clean":"2023-10-10T20:42:53.356705+0000","last_became_active":"2023-10-10T20:42:52.332045+0000","last_became_peered":"2023-10-10T20:42:52.332045+0000","last_unstale":"2023-10-10T20:42:53.356705+0000","last_undegraded":"2023-10-10T20:42:53.356705+0000","last_fullsized":"2023-10-10T20:42:53.356705+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:01.845490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.366441+0000","last_change":"2023-10-10T20:42:53.358569+0000","last_active":"2023-10-10T20:42:53.366441+0000","last_peered":"2023-10-10T20:42:53.366441+0000","last_clean":"2023-10-10T20:42:53.366441+0000","last_became_active":"2023-10-10T20:42:53.358306+0000","last_became_peered":"2023-10-10T20:42:53.358306+0000","last_unstale":"2023-10-10T20:42:53.366441+0000","last_undegraded":"2023-10-10T20:42:53.366441+0000","last_fullsized":"2023-10-10T20:42:53.366441+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:35.275237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.336302+0000","last_change":"2023-10-10T20:42:52.333544+0000","last_active":"2023-10-10T20:42:53.336302+0000","last_peered":"2023-10-10T20:42:53.336302+0000","last_clean":"2023-10-10T20:42:53.336302+0000","last_became_active":"2023-10-10T20:42:52.333211+0000","last_became_peered":"2023-10-10T20:42:52.333211+0000","last_unstale":"2023-10-10T20:42:53.336302+0000","last_undegraded":"2023-10-10T20:42:53.336302+0000","last_fullsized":"2023-10-10T20:42:53.336302+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:39.789815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.353842+0000","last_change":"2023-10-10T20:42:53.351743+0000","last_active":"2023-10-10T20:42:53.353842+0000","last_peered":"2023-10-10T20:42:53.353842+0000","last_clean":"2023-10-10T20:42:53.353842+0000","last_became_active":"2023-10-10T20:42:53.351348+0000","last_became_peered":"2023-10-10T20:42:53.351348+0000","last_unstale":"2023-10-10T20:42:53.353842+0000","last_undegraded":"2023-10-10T20:42:53.353842+0000","last_fullsized":"2023-10-10T20:42:53.353842+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:34:11.321271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.b","version":"164'2","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.364847+0000","last_change":"2023-10-10T20:42:52.336202+0000","last_active":"2023-10-10T20:42:53.364847+0000","last_peered":"2023-10-10T20:42:53.364847+0000","last_clean":"2023-10-10T20:42:53.364847+0000","last_became_active":"2023-10-10T20:42:52.335926+0000","last_became_peered":"2023-10-10T20:42:52.335926+0000","last_unstale":"2023-10-10T20:42:53.364847+0000","last_undegraded":"2023-10-10T20:42:53.364847+0000","last_fullsized":"2023-10-10T20:42:53.364847+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:49:39.517807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.367413+0000","last_change":"2023-10-10T20:42:53.365558+0000","last_active":"2023-10-10T20:42:53.367413+0000","last_peered":"2023-10-10T20:42:53.367413+0000","last_clean":"2023-10-10T20:42:53.367413+0000","last_became_active":"2023-10-10T20:42:53.364328+0000","last_became_peered":"2023-10-10T20:42:53.364328+0000","last_unstale":"2023-10-10T20:42:53.367413+0000","last_undegraded":"2023-10-10T20:42:53.367413+0000","last_fullsized":"2023-10-10T20:42:53.367413+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:22:40.796551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.341106+0000","last_change":"2023-10-10T20:42:52.335457+0000","last_active":"2023-10-10T20:42:53.341106+0000","last_peered":"2023-10-10T20:42:53.341106+0000","last_clean":"2023-10-10T20:42:53.341106+0000","last_became_active":"2023-10-10T20:42:52.335186+0000","last_became_peered":"2023-10-10T20:42:52.335186+0000","last_unstale":"2023-10-10T20:42:53.341106+0000","last_undegraded":"2023-10-10T20:42:53.341106+0000","last_fullsized":"2023-10-10T20:42:53.341106+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:55:21.874802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.347700+0000","last_change":"2023-10-10T20:42:53.346671+0000","last_active":"2023-10-10T20:42:53.347700+0000","last_peered":"2023-10-10T20:42:53.347700+0000","last_clean":"2023-10-10T20:42:53.347700+0000","last_became_active":"2023-10-10T20:42:53.346422+0000","last_became_peered":"2023-10-10T20:42:53.346422+0000","last_unstale":"2023-10-10T20:42:53.347700+0000","last_undegraded":"2023-10-10T20:42:53.347700+0000","last_fullsized":"2023-10-10T20:42:53.347700+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:24:01.205198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.340825+0000","last_change":"2023-10-10T20:42:52.337248+0000","last_active":"2023-10-10T20:42:53.340825+0000","last_peered":"2023-10-10T20:42:53.340825+0000","last_clean":"2023-10-10T20:42:53.340825+0000","last_became_active":"2023-10-10T20:42:52.336966+0000","last_became_peered":"2023-10-10T20:42:52.336966+0000","last_unstale":"2023-10-10T20:42:53.340825+0000","last_undegraded":"2023-10-10T20:42:53.340825+0000","last_fullsized":"2023-10-10T20:42:53.340825+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:59:14.539048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.345348+0000","last_change":"2023-10-10T20:42:53.343031+0000","last_active":"2023-10-10T20:42:53.345348+0000","last_peered":"2023-10-10T20:42:53.345348+0000","last_clean":"2023-10-10T20:42:53.345348+0000","last_became_active":"2023-10-10T20:42:53.342760+0000","last_became_peered":"2023-10-10T20:42:53.342760+0000","last_unstale":"2023-10-10T20:42:53.345348+0000","last_undegraded":"2023-10-10T20:42:53.345348+0000","last_fullsized":"2023-10-10T20:42:53.345348+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:45:05.557928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.349884+0000","last_change":"2023-10-10T20:42:52.350531+0000","last_active":"2023-10-10T20:42:53.349884+0000","last_peered":"2023-10-10T20:42:53.349884+0000","last_clean":"2023-10-10T20:42:53.349884+0000","last_became_active":"2023-10-10T20:42:52.350315+0000","last_became_peered":"2023-10-10T20:42:52.350315+0000","last_unstale":"2023-10-10T20:42:53.349884+0000","last_undegraded":"2023-10-10T20:42:53.349884+0000","last_fullsized":"2023-10-10T20:42:53.349884+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:08.850614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.345339+0000","last_change":"2023-10-10T20:42:53.343152+0000","last_active":"2023-10-10T20:42:53.345339+0000","last_peered":"2023-10-10T20:42:53.345339+0000","last_clean":"2023-10-10T20:42:53.345339+0000","last_became_active":"2023-10-10T20:42:53.342850+0000","last_became_peered":"2023-10-10T20:42:53.342850+0000","last_unstale":"2023-10-10T20:42:53.345339+0000","last_undegraded":"2023-10-10T20:42:53.345339+0000","last_fullsized":"2023-10-10T20:42:53.345339+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:38:10.362086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.7","version":"164'2","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365076+0000","last_change":"2023-10-10T20:42:52.338420+0000","last_active":"2023-10-10T20:42:53.365076+0000","last_peered":"2023-10-10T20:42:53.365076+0000","last_clean":"2023-10-10T20:42:53.365076+0000","last_became_active":"2023-10-10T20:42:52.338190+0000","last_became_peered":"2023-10-10T20:42:52.338190+0000","last_unstale":"2023-10-10T20:42:53.365076+0000","last_undegraded":"2023-10-10T20:42:53.365076+0000","last_fullsized":"2023-10-10T20:42:53.365076+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:58:31.586594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.366016+0000","last_change":"2023-10-10T20:42:53.360884+0000","last_active":"2023-10-10T20:42:53.366016+0000","last_peered":"2023-10-10T20:42:53.366016+0000","last_clean":"2023-10-10T20:42:53.366016+0000","last_became_active":"2023-10-10T20:42:53.360108+0000","last_became_peered":"2023-10-10T20:42:53.360108+0000","last_unstale":"2023-10-10T20:42:53.366016+0000","last_undegraded":"2023-10-10T20:42:53.366016+0000","last_fullsized":"2023-10-10T20:42:53.366016+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:34:11.321271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.336308+0000","last_change":"2023-10-10T20:42:52.335469+0000","last_active":"2023-10-10T20:42:53.336308+0000","last_peered":"2023-10-10T20:42:53.336308+0000","last_clean":"2023-10-10T20:42:53.336308+0000","last_became_active":"2023-10-10T20:42:52.333014+0000","last_became_peered":"2023-10-10T20:42:52.333014+0000","last_unstale":"2023-10-10T20:42:53.336308+0000","last_undegraded":"2023-10-10T20:42:53.336308+0000","last_fullsized":"2023-10-10T20:42:53.336308+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:16:49.047090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365802+0000","last_change":"2023-10-10T20:42:53.359515+0000","last_active":"2023-10-10T20:42:53.365802+0000","last_peered":"2023-10-10T20:42:53.365802+0000","last_clean":"2023-10-10T20:42:53.365802+0000","last_became_active":"2023-10-10T20:42:53.358893+0000","last_became_peered":"2023-10-10T20:42:53.358893+0000","last_unstale":"2023-10-10T20:42:53.365802+0000","last_undegraded":"2023-10-10T20:42:53.365802+0000","last_fullsized":"2023-10-10T20:42:53.365802+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:38:10.362086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.336547+0000","last_change":"2023-10-10T20:42:52.334206+0000","last_active":"2023-10-10T20:42:53.336547+0000","last_peered":"2023-10-10T20:42:53.336547+0000","last_clean":"2023-10-10T20:42:53.336547+0000","last_became_active":"2023-10-10T20:42:52.333022+0000","last_became_peered":"2023-10-10T20:42:52.333022+0000","last_unstale":"2023-10-10T20:42:53.336547+0000","last_undegraded":"2023-10-10T20:42:53.336547+0000","last_fullsized":"2023-10-10T20:42:53.336547+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:34:10.314535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.346479+0000","last_change":"2023-10-10T20:42:53.341044+0000","last_active":"2023-10-10T20:42:53.346479+0000","last_peered":"2023-10-10T20:42:53.346479+0000","last_clean":"2023-10-10T20:42:53.346479+0000","last_became_active":"2023-10-10T20:42:53.340802+0000","last_became_peered":"2023-10-10T20:42:53.340802+0000","last_unstale":"2023-10-10T20:42:53.346479+0000","last_undegraded":"2023-10-10T20:42:53.346479+0000","last_fullsized":"2023-10-10T20:42:53.346479+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:44:40.439820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.351585+0000","last_change":"2023-10-10T20:42:52.339808+0000","last_active":"2023-10-10T20:42:53.351585+0000","last_peered":"2023-10-10T20:42:53.351585+0000","last_clean":"2023-10-10T20:42:53.351585+0000","last_became_active":"2023-10-10T20:42:52.339538+0000","last_became_peered":"2023-10-10T20:42:52.339538+0000","last_unstale":"2023-10-10T20:42:53.351585+0000","last_undegraded":"2023-10-10T20:42:53.351585+0000","last_fullsized":"2023-10-10T20:42:53.351585+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:17:22.696388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.342915+0000","last_change":"2023-10-10T20:42:53.342134+0000","last_active":"2023-10-10T20:42:53.342915+0000","last_peered":"2023-10-10T20:42:53.342915+0000","last_clean":"2023-10-10T20:42:53.342915+0000","last_became_active":"2023-10-10T20:42:53.341742+0000","last_became_peered":"2023-10-10T20:42:53.341742+0000","last_unstale":"2023-10-10T20:42:53.342915+0000","last_undegraded":"2023-10-10T20:42:53.342915+0000","last_fullsized":"2023-10-10T20:42:53.342915+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:18:18.498552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.341700+0000","last_change":"2023-10-10T20:42:52.330898+0000","last_active":"2023-10-10T20:42:53.341700+0000","last_peered":"2023-10-10T20:42:53.341700+0000","last_clean":"2023-10-10T20:42:53.341700+0000","last_became_active":"2023-10-10T20:42:52.330616+0000","last_became_peered":"2023-10-10T20:42:52.330616+0000","last_unstale":"2023-10-10T20:42:53.341700+0000","last_undegraded":"2023-10-10T20:42:53.341700+0000","last_fullsized":"2023-10-10T20:42:53.341700+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:12:30.999244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.367131+0000","last_change":"2023-10-10T20:42:53.364609+0000","last_active":"2023-10-10T20:42:53.367131+0000","last_peered":"2023-10-10T20:42:53.367131+0000","last_clean":"2023-10-10T20:42:53.367131+0000","last_became_active":"2023-10-10T20:42:53.360806+0000","last_became_peered":"2023-10-10T20:42:53.360806+0000","last_unstale":"2023-10-10T20:42:53.367131+0000","last_undegraded":"2023-10-10T20:42:53.367131+0000","last_fullsized":"2023-10-10T20:42:53.367131+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:44:40.439820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.333553+0000","last_change":"2023-10-10T20:42:52.333625+0000","last_active":"2023-10-10T20:42:53.333553+0000","last_peered":"2023-10-10T20:42:53.333553+0000","last_clean":"2023-10-10T20:42:53.333553+0000","last_became_active":"2023-10-10T20:42:52.333359+0000","last_became_peered":"2023-10-10T20:42:52.333359+0000","last_unstale":"2023-10-10T20:42:53.333553+0000","last_undegraded":"2023-10-10T20:42:53.333553+0000","last_fullsized":"2023-10-10T20:42:53.333553+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:31.572538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.366949+0000","last_change":"2023-10-10T20:42:53.364374+0000","last_active":"2023-10-10T20:42:53.366949+0000","last_peered":"2023-10-10T20:42:53.366949+0000","last_clean":"2023-10-10T20:42:53.366949+0000","last_became_active":"2023-10-10T20:42:53.361551+0000","last_became_peered":"2023-10-10T20:42:53.361551+0000","last_unstale":"2023-10-10T20:42:53.366949+0000","last_undegraded":"2023-10-10T20:42:53.366949+0000","last_fullsized":"2023-10-10T20:42:53.366949+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:13:45.883755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.336771+0000","last_change":"2023-10-10T20:42:52.339859+0000","last_active":"2023-10-10T20:42:53.336771+0000","last_peered":"2023-10-10T20:42:53.336771+0000","last_clean":"2023-10-10T20:42:53.336771+0000","last_became_active":"2023-10-10T20:42:52.339585+0000","last_became_peered":"2023-10-10T20:42:52.339585+0000","last_unstale":"2023-10-10T20:42:53.336771+0000","last_undegraded":"2023-10-10T20:42:53.336771+0000","last_fullsized":"2023-10-10T20:42:53.336771+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:02.517948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.355907+0000","last_change":"2023-10-10T20:42:53.345345+0000","last_active":"2023-10-10T20:42:53.355907+0000","last_peered":"2023-10-10T20:42:53.355907+0000","last_clean":"2023-10-10T20:42:53.355907+0000","last_became_active":"2023-10-10T20:42:53.345080+0000","last_became_peered":"2023-10-10T20:42:53.345080+0000","last_unstale":"2023-10-10T20:42:53.355907+0000","last_undegraded":"2023-10-10T20:42:53.355907+0000","last_fullsized":"2023-10-10T20:42:53.355907+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:32:08.240124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.349987+0000","last_change":"2023-10-10T20:42:52.335154+0000","last_active":"2023-10-10T20:42:53.349987+0000","last_peered":"2023-10-10T20:42:53.349987+0000","last_clean":"2023-10-10T20:42:53.349987+0000","last_became_active":"2023-10-10T20:42:52.334853+0000","last_became_peered":"2023-10-10T20:42:52.334853+0000","last_unstale":"2023-10-10T20:42:53.349987+0000","last_undegraded":"2023-10-10T20:42:53.349987+0000","last_fullsized":"2023-10-10T20:42:53.349987+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:15:01.473071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.341777+0000","last_change":"2023-10-10T20:42:53.340554+0000","last_active":"2023-10-10T20:42:53.341777+0000","last_peered":"2023-10-10T20:42:53.341777+0000","last_clean":"2023-10-10T20:42:53.341777+0000","last_became_active":"2023-10-10T20:42:53.340308+0000","last_became_peered":"2023-10-10T20:42:53.340308+0000","last_unstale":"2023-10-10T20:42:53.341777+0000","last_undegraded":"2023-10-10T20:42:53.341777+0000","last_fullsized":"2023-10-10T20:42:53.341777+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:52.314972+0000","last_clean_scrub_stamp":"2023-10-10T20:42:52.314972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:38:30.259757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"164'2","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.365418+0000","last_change":"2023-10-10T20:42:52.338269+0000","last_active":"2023-10-10T20:42:53.365418+0000","last_peered":"2023-10-10T20:42:53.365418+0000","last_clean":"2023-10-10T20:42:53.365418+0000","last_became_active":"2023-10-10T20:42:52.338028+0000","last_became_peered":"2023-10-10T20:42:52.338028+0000","last_unstale":"2023-10-10T20:42:53.365418+0000","last_undegraded":"2023-10-10T20:42:53.365418+0000","last_fullsized":"2023-10-10T20:42:53.365418+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:42:51.308236+0000","last_clean_scrub_stamp":"2023-10-10T20:42:51.308236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:17:34.621354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":308,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-10T20:42:53.335377+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:42:53.335377+0000","last_peered":"2023-10-10T20:42:53.335377+0000","last_clean":"2023-10-10T20:42:53.335377+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:42:53.335377+0000","last_undegraded":"2023-10-10T20:42:53.335377+0000","last_fullsized":"2023-10-10T20:42:53.335377+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574954,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304012,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067828,"statfs":{"total":96636764160,"available":96325455872,"internally_reserved":0,"allocated":2691072,"data_stored":1527131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50700000000000001,"15min":0.75800000000000001},"min":{"1min":0.32500000000000001,"5min":0.309,"15min":0.29199999999999998},"max":{"1min":0.64300000000000002,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.626},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.50800000000000001,"15min":0.60399999999999998},"min":{"1min":0.36399999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.58299999999999996,"5min":0.752,"15min":4.1959999999999997},"last":0.33300000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.50600000000000001,"15min":0.90100000000000002},"min":{"1min":0.372,"5min":0.317,"15min":0.317},"max":{"1min":0.61099999999999999,"5min":0.78800000000000003,"15min":10.1},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.53400000000000003,"15min":0.69099999999999995},"min":{"1min":0.35199999999999998,"5min":0.33900000000000002,"15min":0.28999999999999998},"max":{"1min":0.66500000000000004,"5min":0.81799999999999995,"15min":6.806},"last":0.48099999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56799999999999995,"15min":0.74099999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68799999999999994,"5min":0.89000000000000001,"15min":6.9340000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.69399999999999995},"min":{"1min":0.40699999999999997,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.69099999999999995,"5min":0.77100000000000002,"15min":6.9870000000000001},"last":0.56399999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54200000000000004,"15min":0.71299999999999997},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64500000000000002,"5min":0.754,"15min":6.6719999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55900000000000005,"15min":0.64800000000000002},"min":{"1min":0.39800000000000002,"5min":0.378,"15min":0.34799999999999998},"max":{"1min":0.64400000000000002,"5min":0.76200000000000001,"15min":4.1609999999999996},"last":0.71999999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47799999999999998,"15min":0.46800000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.77200000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.51600000000000001,"15min":0.51900000000000002},"min":{"1min":0.314,"5min":0.309,"15min":0.30399999999999999},"max":{"1min":0.68000000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.67800000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.54000000000000004,"15min":0.53900000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.68400000000000005,"5min":0.86099999999999999,"15min":1.3180000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.53600000000000003,"15min":0.52700000000000002},"min":{"1min":0.35299999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.84099999999999997},"last":0.64800000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.35699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":1.7509999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76300000000000001,"5min":0.84099999999999997,"15min":1.7090000000000001},"last":0.73099999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303436,"kb_used_data":2052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068404,"statfs":{"total":96636764160,"available":96326045696,"internally_reserved":0,"allocated":2101248,"data_stored":936749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52400000000000002,"15min":0.64000000000000001},"min":{"1min":0.34699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":7.7590000000000003},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52400000000000002,"15min":0.67400000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.86899999999999999,"5min":1.075,"15min":7.8090000000000002},"last":0.40600000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52900000000000003,"15min":0.67200000000000004},"min":{"1min":0.32800000000000001,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":7.867},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55500000000000005,"15min":0.81999999999999995},"min":{"1min":0.378,"5min":0.35499999999999998,"15min":0.33700000000000002},"max":{"1min":0.70999999999999996,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.36099999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52300000000000002,"15min":0.58099999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.745,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57099999999999995,"15min":0.79100000000000004},"min":{"1min":0.39700000000000002,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":2.641,"15min":6.7629999999999999},"last":0.439}]},{"osd":3,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55500000000000005,"15min":0.60699999999999998},"min":{"1min":0.372,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94399999999999995,"5min":1.107,"15min":4.8620000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55500000000000005,"15min":0.68400000000000005},"min":{"1min":0.41399999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.72599999999999998,"5min":1.718,"15min":7.9009999999999998},"last":0.501}]},{"osd":4,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51900000000000002,"15min":0.47899999999999998},"min":{"1min":0.32000000000000001,"5min":0.28999999999999998,"15min":0.28499999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":0.82899999999999996,"5min":1.032,"15min":1.032},"last":0.52400000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53400000000000003,"15min":0.5},"min":{"1min":0.34999999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.86799999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.379},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56999999999999995,"15min":0.53700000000000003},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85399999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.51300000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56200000000000006,"15min":0.53000000000000003},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.89600000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.59399999999999997,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.372,"15min":0.35099999999999998},"max":{"1min":0.82199999999999995,"5min":1.681,"15min":1.681},"last":0.42699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303432,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068408,"statfs":{"total":96636764160,"available":96326049792,"internally_reserved":0,"allocated":2097152,"data_stored":936783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.31900000000000001,"15min":0.443},"min":{"1min":0.215,"5min":0.19600000000000001,"15min":0.17899999999999999},"max":{"1min":0.47799999999999998,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33000000000000002,"15min":0.35599999999999998},"min":{"1min":0.22500000000000001,"5min":0.224,"15min":0.20899999999999999},"max":{"1min":0.441,"5min":0.65500000000000003,"15min":1.194},"last":0.33400000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.32700000000000001,"15min":0.35499999999999998},"min":{"1min":0.246,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.41699999999999998,"5min":0.51400000000000001,"15min":1.2869999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.35199999999999998,"15min":0.47299999999999998},"min":{"1min":0.23999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.49199999999999999,"5min":0.54200000000000004,"15min":5.0919999999999996},"last":0.316}]},{"osd":3,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.374,"15min":0.496},"min":{"1min":0.25600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54800000000000004,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.36299999999999999,"15min":0.39800000000000002},"min":{"1min":0.26500000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.52600000000000002,"5min":0.57799999999999996,"15min":1.2170000000000001},"last":0.29599999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.78600000000000003,"5min":0.92400000000000004,"15min":1.254},"last":0.499},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55900000000000005,"15min":0.67500000000000004},"min":{"1min":0.39600000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68100000000000005,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.56100000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.40400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.97599999999999998,"15min":1.1799999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56999999999999995,"15min":0.59999999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73299999999999998,"5min":0.878,"15min":1.3109999999999999},"last":0.57299999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":0.79300000000000004},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70399999999999996,"5min":0.995,"15min":5.0659999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60699999999999998,"15min":0.628},"min":{"1min":0.46400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84199999999999997,"5min":0.89800000000000002,"15min":1.171},"last":0.48199999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.48299999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77100000000000002,"5min":1.004,"15min":1.214},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58399999999999996,"15min":0.69699999999999995},"min":{"1min":0.40899999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.754,"5min":0.90500000000000003,"15min":4.0499999999999998},"last":0.52100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574953,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304012,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067828,"statfs":{"total":96636764160,"available":96325455872,"internally_reserved":0,"allocated":2691072,"data_stored":1526744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.315,"15min":0.48599999999999999},"min":{"1min":0.217,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":7.9249999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.46899999999999997},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46100000000000002,"5min":0.53800000000000003,"15min":6.6299999999999999},"last":0.30299999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33900000000000002,"15min":0.51000000000000001},"min":{"1min":0.25600000000000001,"5min":0.22,"15min":0.216},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":8.2509999999999994},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.49299999999999999},"min":{"1min":0.27500000000000002,"5min":0.223,"15min":0.222},"max":{"1min":0.498,"5min":0.57899999999999996,"15min":6.7370000000000001},"last":0.35499999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.35499999999999998,"15min":0.51700000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.21099999999999999},"max":{"1min":0.438,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.37},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.57099999999999995},"min":{"1min":0.317,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":7.8929999999999998},"last":0.40699999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57099999999999995,"15min":0.68700000000000006},"min":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.78100000000000003,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57199999999999995,"15min":0.73099999999999998},"min":{"1min":0.371,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.69599999999999995,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.58199999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56499999999999995,"15min":0.71399999999999997},"min":{"1min":0.40100000000000002,"5min":0.376,"15min":0.36299999999999999},"max":{"1min":0.72699999999999998,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.74199999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.60399999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61499999999999999,"15min":0.77600000000000002},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.73299999999999998},"min":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.36599999999999999},"max":{"1min":0.82099999999999995,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.56899999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60899999999999999,"15min":0.72399999999999998},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.36299999999999999},"max":{"1min":0.85499999999999998,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.751},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59799999999999998,"15min":0.747},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89500000000000002,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.56100000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574955,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303448,"kb_used_data":2064,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068392,"statfs":{"total":96636764160,"available":96326033408,"internally_reserved":0,"allocated":2113536,"data_stored":938285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.311,"15min":0.32400000000000001},"min":{"1min":0.20200000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.48599999999999999,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32100000000000001,"15min":0.34899999999999998},"min":{"1min":0.224,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51800000000000002,"5min":0.54200000000000004,"15min":0.746},"last":0.505}]},{"osd":1,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33200000000000002,"15min":0.65200000000000002},"min":{"1min":0.22600000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52200000000000002,"5min":0.63,"15min":11.967000000000001},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.35199999999999998,"15min":0.45800000000000002},"min":{"1min":0.245,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.505,"5min":0.52700000000000002,"15min":4.1100000000000003},"last":0.28699999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.51600000000000001},"min":{"1min":0.23699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":6.3040000000000003},"last":0.308},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.38700000000000001,"15min":0.70399999999999996},"min":{"1min":0.29499999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57799999999999996,"5min":0.65900000000000003,"15min":11.935},"last":0.34899999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.42999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91900000000000004,"5min":1.302,"15min":1.302},"last":0.42299999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.378},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.95699999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.39700000000000002,"5min":0.35999999999999999,"15min":0.34300000000000003},"max":{"1min":0.78800000000000003,"5min":0.83999999999999997,"15min":0.96099999999999997},"last":0.47099999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.33400000000000002,"15min":0.33200000000000002},"max":{"1min":0.95499999999999996,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60399999999999998,"15min":0.60499999999999998},"min":{"1min":0.47499999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.54900000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.46100000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85699999999999998,"5min":0.89700000000000002,"15min":1.052},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58899999999999997,"15min":0.746},"min":{"1min":0.40999999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.98099999999999998,"5min":1.274,"15min":6.2649999999999997},"last":0.53900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303444,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068396,"statfs":{"total":96636764160,"available":96326037504,"internally_reserved":0,"allocated":2109440,"data_stored":938299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.309,"15min":0.314},"min":{"1min":0.22600000000000001,"5min":0.21299999999999999,"15min":0.21099999999999999},"max":{"1min":0.38900000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.316,"5min":0.32700000000000001,"15min":0.32100000000000001},"min":{"1min":0.23799999999999999,"5min":0.216,"15min":0.19800000000000001},"max":{"1min":0.42799999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.23999999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.34399999999999997,"15min":0.33400000000000002},"min":{"1min":0.26200000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.47299999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38300000000000001,"15min":0.378},"min":{"1min":0.26800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.48599999999999999,"5min":0.66900000000000004,"15min":0.72099999999999997},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.36299999999999999,"15min":0.35299999999999998},"min":{"1min":0.28899999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44800000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.28799999999999998,"5min":0.25900000000000001,"15min":0.255},"max":{"1min":0.441,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.36799999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56100000000000005,"15min":0.53800000000000003},"min":{"1min":0.44900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56699999999999995,"15min":0.53900000000000003},"min":{"1min":0.433,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.79400000000000004,"5min":0.83299999999999996,"15min":0.93500000000000005},"last":0.57599999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.55300000000000005},"min":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.35899999999999999},"max":{"1min":0.69599999999999995,"5min":1.347,"15min":1.347},"last":0.623},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.443,"5min":0.39300000000000002,"15min":0.35799999999999998},"max":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":1.01},"last":0.77400000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.45500000000000002,"5min":0.41099999999999998,"15min":0.36699999999999999},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.48899999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79700000000000004,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.79900000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61599999999999999,"15min":0.59599999999999997},"min":{"1min":0.495,"5min":0.42099999999999999,"15min":0.38200000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.96999999999999997},"last":0.78900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574953,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303432,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068408,"statfs":{"total":96636764160,"available":96326049792,"internally_reserved":0,"allocated":2097152,"data_stored":937247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.372,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66700000000000004,"5min":0.74199999999999999,"15min":0.747},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52500000000000002,"15min":0.71899999999999997},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.32700000000000001},"max":{"1min":0.68300000000000005,"5min":0.77600000000000002,"15min":7.6719999999999997},"last":0.43099999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52600000000000002,"15min":0.72399999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.755,"15min":7.7160000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.371,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":0.86799999999999999},"last":0.442}]},{"osd":2,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67800000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.749},"min":{"1min":0.39900000000000002,"5min":0.39700000000000002,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":7.7839999999999998},"last":0.50900000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.47999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74399999999999999,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.42999999999999999,"5min":0.39200000000000002,"15min":0.35499999999999998},"max":{"1min":0.71499999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47299999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.497},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.30099999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.54700000000000004},"min":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.32000000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.35399999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.77300000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.379},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86199999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.34399999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.378,"15min":0.36199999999999999},"max":{"1min":0.88600000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.37},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.41699999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.874,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.41999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303428,"kb_used_data":2044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068412,"statfs":{"total":96636764160,"available":96326053888,"internally_reserved":0,"allocated":2093056,"data_stored":936693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.498,"15min":0.49199999999999999},"min":{"1min":0.38300000000000001,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.69699999999999995,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53100000000000003,"15min":0.52000000000000002},"min":{"1min":0.39400000000000002,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.72399999999999998,"5min":0.76700000000000002,"15min":0.79300000000000004},"last":0.55500000000000005}]},{"osd":1,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51400000000000001,"15min":0.61799999999999999},"min":{"1min":0.35099999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70799999999999996,"5min":0.746,"15min":5.2679999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53000000000000003,"15min":0.61099999999999999},"min":{"1min":0.442,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.65700000000000003,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.42599999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.56200000000000006,"15min":0.63800000000000001},"min":{"1min":0.45500000000000002,"5min":0.32900000000000001,"15min":0.28299999999999997},"max":{"1min":3.27,"5min":3.27,"15min":5.2960000000000003},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55000000000000004,"15min":0.54100000000000004},"min":{"1min":0.40999999999999998,"5min":0.33900000000000002,"15min":0.32000000000000001},"max":{"1min":0.75,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.501}]},{"osd":3,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55700000000000005,"15min":0.65700000000000003},"min":{"1min":0.47499999999999998,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.69799999999999995,"5min":0.94599999999999995,"15min":5.3330000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57399999999999995,"15min":0.66800000000000004},"min":{"1min":0.45200000000000001,"5min":0.39300000000000002,"15min":0.27700000000000002},"max":{"1min":0.79300000000000004,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.52400000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.49199999999999999,"15min":0.497},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67300000000000004,"5min":0.72499999999999998,"15min":0.82999999999999996},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53100000000000003,"15min":0.54300000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32200000000000001},"max":{"1min":0.83099999999999996,"5min":0.88,"15min":1.0800000000000001},"last":0.39300000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.86499999999999999,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.48799999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56200000000000006,"15min":0.57499999999999996},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.30099999999999999},"max":{"1min":0.78700000000000003,"5min":0.98299999999999998,"15min":0.98399999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.378,"5min":0.377,"15min":0.377},"max":{"1min":0.83199999999999996,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.54600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:43:02.285 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:43:02.285 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:43:02.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.438+0000 7f79762b6700 1 Processor -- start 2023-10-10T20:43:02.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.439+0000 7f79762b6700 1 -- start start 2023-10-10T20:43:02.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.439+0000 7f79762b6700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7970156620 0x7f79700c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:02.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.439+0000 7f79762b6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f79701569f0 0x7f79700ace70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:02.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.439+0000 7f79762b6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7970068390 0x7f7970068810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:02.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.439+0000 7f79762b6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79700c8d30 con 0x7f7970068390 2023-10-10T20:43:02.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.439+0000 7f79762b6700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7970068d50 con 0x7f7970156620 2023-10-10T20:43:02.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.439+0000 7f79762b6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79700ad3b0 con 0x7f79701569f0 2023-10-10T20:43:02.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.439+0000 7f7974853700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7970068390 0x7f7970068810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:02.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.439+0000 7f7974853700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7970068390 0x7f7970068810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47426/0 (socket says 172.21.15.89:47426) 2023-10-10T20:43:02.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.439+0000 7f7974853700 1 -- 172.21.15.89:0/2590276069 learned_addr learned my addr 172.21.15.89:0/2590276069 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:02.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.440+0000 7f796f7fe700 1 --2- 172.21.15.89:0/2590276069 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f79701569f0 0x7f79700ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:02.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.440+0000 7f796ffff700 1 --2- 172.21.15.89:0/2590276069 >> v2:172.21.15.111:3300/0 conn(0x7f7970156620 0x7f79700c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:02.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.440+0000 7f7974853700 1 -- 172.21.15.89:0/2590276069 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f79701569f0 msgr2=0x7f79700ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:02.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.440+0000 7f7974853700 1 --2- 172.21.15.89:0/2590276069 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f79701569f0 0x7f79700ace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.440+0000 7f7974853700 1 -- 172.21.15.89:0/2590276069 >> v2:172.21.15.111:3300/0 conn(0x7f7970156620 msgr2=0x7f79700c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:02.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.440+0000 7f7974853700 1 --2- 172.21.15.89:0/2590276069 >> v2:172.21.15.111:3300/0 conn(0x7f7970156620 0x7f79700c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.440+0000 7f7974853700 1 -- 172.21.15.89:0/2590276069 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797015a6f0 con 0x7f7970068390 2023-10-10T20:43:02.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.440+0000 7f796ffff700 1 --2- 172.21.15.89:0/2590276069 >> v2:172.21.15.111:3300/0 conn(0x7f7970156620 0x7f79700c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:02.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.440+0000 7f7974853700 1 --2- 172.21.15.89:0/2590276069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7970068390 0x7f7970068810 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f79700c8f80 tx=0x7f796800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b676a58c75058877 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:02.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.440+0000 7f796effd700 1 -- 172.21.15.89:0/2590276069 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968010070 con 0x7f7970068390 2023-10-10T20:43:02.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.440+0000 7f796effd700 1 -- 172.21.15.89:0/2590276069 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796800de80 con 0x7f7970068390 2023-10-10T20:43:02.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.440+0000 7f796effd700 1 -- 172.21.15.89:0/2590276069 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796800b320 con 0x7f7970068390 2023-10-10T20:43:02.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.441+0000 7f79762b6700 1 -- 172.21.15.89:0/2590276069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7970068390 msgr2=0x7f7970068810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:02.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.441+0000 7f79762b6700 1 --2- 172.21.15.89:0/2590276069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7970068390 0x7f7970068810 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f79700c8f80 tx=0x7f796800ba50 comp rx=0 tx=0).stop 2023-10-10T20:43:02.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.441+0000 7f79762b6700 1 -- 172.21.15.89:0/2590276069 shutdown_connections 2023-10-10T20:43:02.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.441+0000 7f79762b6700 1 --2- 172.21.15.89:0/2590276069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7970068390 0x7f7970068810 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.441+0000 7f79762b6700 1 --2- 172.21.15.89:0/2590276069 >> v2:172.21.15.111:3300/0 conn(0x7f7970156620 0x7f79700c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.441+0000 7f79762b6700 1 --2- 172.21.15.89:0/2590276069 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f79701569f0 0x7f79700ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.441+0000 7f79762b6700 1 -- 172.21.15.89:0/2590276069 >> 172.21.15.89:0/2590276069 conn(0x7f79700cc540 msgr2=0x7f79700cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:02.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.441+0000 7f79762b6700 1 -- 172.21.15.89:0/2590276069 shutdown_connections 2023-10-10T20:43:02.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.441+0000 7f79762b6700 1 -- 172.21.15.89:0/2590276069 wait complete. 2023-10-10T20:43:02.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.442+0000 7f79762b6700 1 Processor -- start 2023-10-10T20:43:02.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.442+0000 7f79762b6700 1 -- start start 2023-10-10T20:43:02.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.442+0000 7f79762b6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7970068390 0x7f7970155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:02.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f79762b6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7970156620 0x7f7970166270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:02.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f79762b6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79701569f0 0x7f7970159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:02.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f79762b6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7970157290 con 0x7f7970068390 2023-10-10T20:43:02.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f79762b6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7970156030 con 0x7f79701569f0 2023-10-10T20:43:02.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f79762b6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7970157dd0 con 0x7f7970156620 2023-10-10T20:43:02.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f796ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7970068390 0x7f7970155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:02.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f796ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7970068390 0x7f7970155af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47442/0 (socket says 172.21.15.89:47442) 2023-10-10T20:43:02.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f796ffff700 1 -- 172.21.15.89:0/289429037 learned_addr learned my addr 172.21.15.89:0/289429037 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:02.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f7974853700 1 --2- 172.21.15.89:0/289429037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79701569f0 0x7f7970159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:02.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f796f7fe700 1 --2- 172.21.15.89:0/289429037 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7970156620 0x7f7970166270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:02.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f796ffff700 1 -- 172.21.15.89:0/289429037 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7970156620 msgr2=0x7f7970166270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:02.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f796ffff700 1 --2- 172.21.15.89:0/289429037 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7970156620 0x7f7970166270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f796ffff700 1 -- 172.21.15.89:0/289429037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79701569f0 msgr2=0x7f7970159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:02.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f796ffff700 1 --2- 172.21.15.89:0/289429037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79701569f0 0x7f7970159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f796ffff700 1 -- 172.21.15.89:0/289429037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796800bcf0 con 0x7f7970068390 2023-10-10T20:43:02.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f796ffff700 1 --2- 172.21.15.89:0/289429037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7970068390 0x7f7970155af0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f796000c830 tx=0x7f7960008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:02.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f7957fff700 1 -- 172.21.15.89:0/289429037 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960011600 con 0x7f7970068390 2023-10-10T20:43:02.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f7957fff700 1 -- 172.21.15.89:0/289429037 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7960011760 con 0x7f7970068390 2023-10-10T20:43:02.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f79762b6700 1 -- 172.21.15.89:0/289429037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79701574a0 con 0x7f7970068390 2023-10-10T20:43:02.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f7957fff700 1 -- 172.21.15.89:0/289429037 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960011b20 con 0x7f7970068390 2023-10-10T20:43:02.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.443+0000 7f79762b6700 1 -- 172.21.15.89:0/289429037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7970157a60 con 0x7f7970068390 2023-10-10T20:43:02.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.444+0000 7f79762b6700 1 -- 172.21.15.89:0/289429037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f797006ebc0 con 0x7f7970068390 2023-10-10T20:43:02.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.445+0000 7f7957fff700 1 -- 172.21.15.89:0/289429037 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f796000a3e0 con 0x7f7970068390 2023-10-10T20:43:02.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.445+0000 7f7957fff700 1 --2- 172.21.15.89:0/289429037 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f795c067320 0x7f795c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:02.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.445+0000 7f7957fff700 1 -- 172.21.15.89:0/289429037 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7960085210 con 0x7f7970068390 2023-10-10T20:43:02.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.445+0000 7f796f7fe700 1 --2- 172.21.15.89:0/289429037 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f795c067320 0x7f795c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:02.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.445+0000 7f796f7fe700 1 --2- 172.21.15.89:0/289429037 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f795c067320 0x7f795c0697d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f79700ca4e0 tx=0x7f7964008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:02.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.449+0000 7f7957fff700 1 -- 172.21.15.89:0/289429037 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7960051d90 con 0x7f7970068390 2023-10-10T20:43:02.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.707+0000 7f79762b6700 1 -- 172.21.15.89:0/289429037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f797006faa0 con 0x7f7970068390 2023-10-10T20:43:02.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.711+0000 7f7957fff700 1 -- 172.21.15.89:0/289429037 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) v1 ==== 190+0+0 (secure 0 0 0) 0x7f79600553b0 con 0x7f7970068390 2023-10-10T20:43:02.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.713+0000 7f79762b6700 1 -- 172.21.15.89:0/289429037 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f795c067320 msgr2=0x7f795c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:02.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.713+0000 7f79762b6700 1 --2- 172.21.15.89:0/289429037 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f795c067320 0x7f795c0697d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f79700ca4e0 tx=0x7f7964008040 comp rx=0 tx=0).stop 2023-10-10T20:43:02.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.713+0000 7f79762b6700 1 -- 172.21.15.89:0/289429037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7970068390 msgr2=0x7f7970155af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:02.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.713+0000 7f79762b6700 1 --2- 172.21.15.89:0/289429037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7970068390 0x7f7970155af0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f796000c830 tx=0x7f7960008040 comp rx=0 tx=0).stop 2023-10-10T20:43:02.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.713+0000 7f79762b6700 1 -- 172.21.15.89:0/289429037 shutdown_connections 2023-10-10T20:43:02.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.713+0000 7f79762b6700 1 --2- 172.21.15.89:0/289429037 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f795c067320 0x7f795c0697d0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.713+0000 7f79762b6700 1 --2- 172.21.15.89:0/289429037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79701569f0 0x7f7970159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.713+0000 7f79762b6700 1 --2- 172.21.15.89:0/289429037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7970068390 0x7f7970155af0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.713+0000 7f79762b6700 1 --2- 172.21.15.89:0/289429037 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7970156620 0x7f7970166270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.713+0000 7f79762b6700 1 -- 172.21.15.89:0/289429037 >> 172.21.15.89:0/289429037 conn(0x7f79700cc540 msgr2=0x7f79700ad550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:02.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.713+0000 7f79762b6700 1 -- 172.21.15.89:0/289429037 shutdown_connections 2023-10-10T20:43:02.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.713+0000 7f79762b6700 1 -- 172.21.15.89:0/289429037 wait complete. 2023-10-10T20:43:02.719 INFO:teuthology.orchestra.run.smithi089.stderr:updated caps for client.0 2023-10-10T20:43:02.728 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:43:02.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.840+0000 7f840a254700 1 Processor -- start 2023-10-10T20:43:02.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.840+0000 7f840a254700 1 -- start start 2023-10-10T20:43:02.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.841+0000 7f840a254700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f84041597f0 0x7f84040b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:02.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.841+0000 7f840a254700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f840415d230 0x7f840415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:02.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.841+0000 7f840a254700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840415db70 con 0x7f8404159420 2023-10-10T20:43:02.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.841+0000 7f840a254700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84040b11d0 con 0x7f840415d230 2023-10-10T20:43:02.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.841+0000 7f840a254700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84040b14b0 con 0x7f84041597f0 2023-10-10T20:43:02.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.841+0000 7f8409a53700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f840415d230 0x7f840415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:02.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.841+0000 7f8409a53700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f840415d230 0x7f840415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47456/0 (socket says 172.21.15.89:47456) 2023-10-10T20:43:02.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.841+0000 7f8409a53700 1 -- 172.21.15.89:0/3924664022 learned_addr learned my addr 172.21.15.89:0/3924664022 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:02.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.841+0000 7f8408a51700 1 --2- 172.21.15.89:0/3924664022 >> v2:172.21.15.111:3300/0 conn(0x7f84041597f0 0x7f84040b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:02.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.841+0000 7f8409a53700 1 -- 172.21.15.89:0/3924664022 >> v2:172.21.15.111:3300/0 conn(0x7f84041597f0 msgr2=0x7f84040b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:02.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.841+0000 7f8409a53700 1 --2- 172.21.15.89:0/3924664022 >> v2:172.21.15.111:3300/0 conn(0x7f84041597f0 0x7f84040b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.841+0000 7f8409a53700 1 -- 172.21.15.89:0/3924664022 >> v1:172.21.15.111:6789/0 conn(0x7f8404159420 legacy=0x7f84040b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:02.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.841+0000 7f8409a53700 1 -- 172.21.15.89:0/3924664022 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84040b1790 con 0x7f840415d230 2023-10-10T20:43:02.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.841+0000 7f8409a53700 1 --2- 172.21.15.89:0/3924664022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f840415d230 0x7f840415d630 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f8404089750 tx=0x7f83fc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc3ba8079051065 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:02.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.841+0000 7f83fbfff700 1 -- 172.21.15.89:0/3924664022 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc00e070 con 0x7f840415d230 2023-10-10T20:43:02.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.841+0000 7f83fbfff700 1 -- 172.21.15.89:0/3924664022 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83fc00ba10 con 0x7f840415d230 2023-10-10T20:43:02.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.841+0000 7f83fbfff700 1 -- 172.21.15.89:0/3924664022 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc0092e0 con 0x7f840415d230 2023-10-10T20:43:02.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.842+0000 7f840a254700 1 -- 172.21.15.89:0/3924664022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f840415d230 msgr2=0x7f840415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:02.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.842+0000 7f840a254700 1 --2- 172.21.15.89:0/3924664022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f840415d230 0x7f840415d630 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f8404089750 tx=0x7f83fc009840 comp rx=0 tx=0).stop 2023-10-10T20:43:02.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.842+0000 7f840a254700 1 -- 172.21.15.89:0/3924664022 shutdown_connections 2023-10-10T20:43:02.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.842+0000 7f840a254700 1 --2- 172.21.15.89:0/3924664022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f840415d230 0x7f840415d630 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.842+0000 7f840a254700 1 --2- 172.21.15.89:0/3924664022 >> v2:172.21.15.111:3300/0 conn(0x7f84041597f0 0x7f84040b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.842+0000 7f840a254700 1 -- 172.21.15.89:0/3924664022 >> 172.21.15.89:0/3924664022 conn(0x7f84040d0660 msgr2=0x7f84040d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:02.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.842+0000 7f840a254700 1 -- 172.21.15.89:0/3924664022 shutdown_connections 2023-10-10T20:43:02.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.842+0000 7f840a254700 1 -- 172.21.15.89:0/3924664022 wait complete. 2023-10-10T20:43:02.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.843+0000 7f840a254700 1 Processor -- start 2023-10-10T20:43:02.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.843+0000 7f840a254700 1 -- start start 2023-10-10T20:43:02.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.844+0000 7f840a254700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8404159420 0x7f840415cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:02.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.844+0000 7f840a254700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84041597f0 0x7f84040b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:02.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.844+0000 7f840a254700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f840415d230 0x7f8404171030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:02.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.844+0000 7f840a254700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840415d0c0 con 0x7f8404159420 2023-10-10T20:43:02.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.844+0000 7f840a254700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840414bff0 con 0x7f84041597f0 2023-10-10T20:43:02.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.844+0000 7f840a254700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840406d8c0 con 0x7f840415d230 2023-10-10T20:43:02.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.844+0000 7f8409252700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8404159420 0x7f840415cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:02.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.844+0000 7f8409252700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8404159420 0x7f840415cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47462/0 (socket says 172.21.15.89:47462) 2023-10-10T20:43:02.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.844+0000 7f8409252700 1 -- 172.21.15.89:0/1633976983 learned_addr learned my addr 172.21.15.89:0/1633976983 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:02.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.844+0000 7f8409252700 1 -- 172.21.15.89:0/1633976983 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f840415d230 msgr2=0x7f8404171030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:02.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.844+0000 7f8409252700 1 --2- 172.21.15.89:0/1633976983 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f840415d230 0x7f8404171030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.844+0000 7f8409252700 1 -- 172.21.15.89:0/1633976983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84041597f0 msgr2=0x7f84040b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:02.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.844+0000 7f8409252700 1 --2- 172.21.15.89:0/1633976983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84041597f0 0x7f84040b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:02.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.844+0000 7f8409252700 1 -- 172.21.15.89:0/1633976983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc009ae0 con 0x7f8404159420 2023-10-10T20:43:02.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.845+0000 7f8409252700 1 --2- 172.21.15.89:0/1633976983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8404159420 0x7f840415cb80 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f840000c5d0 tx=0x7f8400007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:02.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.845+0000 7f83f97fa700 1 -- 172.21.15.89:0/1633976983 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400011590 con 0x7f8404159420 2023-10-10T20:43:02.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.845+0000 7f83f97fa700 1 -- 172.21.15.89:0/1633976983 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84000116f0 con 0x7f8404159420 2023-10-10T20:43:02.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.845+0000 7f840a254700 1 -- 172.21.15.89:0/1633976983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84040b5460 con 0x7f8404159420 2023-10-10T20:43:02.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.845+0000 7f83f97fa700 1 -- 172.21.15.89:0/1633976983 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400011ab0 con 0x7f8404159420 2023-10-10T20:43:02.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.845+0000 7f840a254700 1 -- 172.21.15.89:0/1633976983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84040b5aa0 con 0x7f8404159420 2023-10-10T20:43:02.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.846+0000 7f840a254700 1 -- 172.21.15.89:0/1633976983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84040b51b0 con 0x7f8404159420 2023-10-10T20:43:02.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.846+0000 7f83f97fa700 1 -- 172.21.15.89:0/1633976983 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f840001c460 con 0x7f8404159420 2023-10-10T20:43:02.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.846+0000 7f83f97fa700 1 --2- 172.21.15.89:0/1633976983 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83e8067320 0x7f83e80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:02.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.847+0000 7f83f97fa700 1 -- 172.21.15.89:0/1633976983 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f84000849b0 con 0x7f8404159420 2023-10-10T20:43:02.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.847+0000 7f8408a51700 1 --2- 172.21.15.89:0/1633976983 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83e8067320 0x7f83e80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:02.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.847+0000 7f8408a51700 1 --2- 172.21.15.89:0/1633976983 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83e8067320 0x7f83e80697d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8404170f90 tx=0x7f83f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:02.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:02.851+0000 7f83f97fa700 1 -- 172.21.15.89:0/1633976983 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8400051640 con 0x7f8404159420 2023-10-10T20:43:03.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.081+0000 7f840a254700 1 -- 172.21.15.89:0/1633976983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f840406dba0 con 0x7f8404159420 2023-10-10T20:43:03.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.081+0000 7f83f97fa700 1 -- 172.21.15.89:0/1633976983 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4613 (secure 0 0 0) 0x7f8400054c60 con 0x7f8404159420 2023-10-10T20:43:03.085 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:03.085 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":110,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6638,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2350308973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2350308973},{"type":"v1","addr":"172.21.15.111:6835","nonce":2350308973}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":8322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/1051252386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":1051252386},{"type":"v1","addr":"172.21.15.89:6836","nonce":1051252386}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":8325,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3754994248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":3754994248},{"type":"v1","addr":"172.21.15.89:6837","nonce":3754994248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:42:52.707112+0000","modified":"2023-10-10T20:42:56.358589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8337":{"gid":8337,"name":"b","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1733913702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1733913702},{"type":"v1","addr":"172.21.15.111:6834","nonce":1733913702}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-10T20:43:03.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.084+0000 7f840a254700 1 -- 172.21.15.89:0/1633976983 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83e8067320 msgr2=0x7f83e80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:03.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.084+0000 7f840a254700 1 --2- 172.21.15.89:0/1633976983 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83e8067320 0x7f83e80697d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8404170f90 tx=0x7f83f4008040 comp rx=0 tx=0).stop 2023-10-10T20:43:03.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.084+0000 7f840a254700 1 -- 172.21.15.89:0/1633976983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8404159420 msgr2=0x7f840415cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:03.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.084+0000 7f840a254700 1 --2- 172.21.15.89:0/1633976983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8404159420 0x7f840415cb80 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f840000c5d0 tx=0x7f8400007f60 comp rx=0 tx=0).stop 2023-10-10T20:43:03.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.084+0000 7f840a254700 1 -- 172.21.15.89:0/1633976983 shutdown_connections 2023-10-10T20:43:03.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.084+0000 7f840a254700 1 --2- 172.21.15.89:0/1633976983 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f840415d230 0x7f8404171030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.084+0000 7f840a254700 1 --2- 172.21.15.89:0/1633976983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8404159420 0x7f840415cb80 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.084+0000 7f840a254700 1 --2- 172.21.15.89:0/1633976983 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f83e8067320 0x7f83e80697d0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.084+0000 7f840a254700 1 --2- 172.21.15.89:0/1633976983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84041597f0 0x7f84040b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.084+0000 7f840a254700 1 -- 172.21.15.89:0/1633976983 >> 172.21.15.89:0/1633976983 conn(0x7f84040d0660 msgr2=0x7f8404069b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:03.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.084+0000 7f840a254700 1 -- 172.21.15.89:0/1633976983 shutdown_connections 2023-10-10T20:43:03.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.084+0000 7f840a254700 1 -- 172.21.15.89:0/1633976983 wait complete. 2023-10-10T20:43:03.090 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 110 2023-10-10T20:43:03.501 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 110, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:42:52.707112+0000', 'modified': '2023-10-10T20:42:56.358589+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8337}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8337': {'gid': 8337, 'name': 'b', 'rank': 0, 'incarnation': 106, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6834/1733913702', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6832', 'nonce': 1733913702}, {'type': 'v1', 'addr': '172.21.15.111:6834', 'nonce': 1733913702}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:43:03.502 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:43:03.502 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-10T20:43:03.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.614+0000 7fbb40e8f700 1 Processor -- start 2023-10-10T20:43:03.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.614+0000 7fbb40e8f700 1 -- start start 2023-10-10T20:43:03.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.615+0000 7fbb40e8f700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbb3c069560 0x7fbb3c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.615+0000 7fbb40e8f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3c069ee0 con 0x7fbb3c159890 2023-10-10T20:43:03.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.615+0000 7fbb40e8f700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3c15a160 con 0x7fbb3c159c60 2023-10-10T20:43:03.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.615+0000 7fbb40e8f700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb3c0d13a0 con 0x7fbb3c069560 2023-10-10T20:43:03.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.615+0000 7fbb3a59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbb3c159890 0x7fbb3c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38390/0 (socket says 172.21.15.89:38390) 2023-10-10T20:43:03.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.615+0000 7fbb3a59c700 1 -- 172.21.15.89:0/647586780 learned_addr learned my addr 172.21.15.89:0/647586780 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:03.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.615+0000 7fbb39d9b700 1 -- 172.21.15.89:0/647586780 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4275884616 0 0) 0x7fbb3c069ee0 con 0x7fbb3c159890 2023-10-10T20:43:03.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.615+0000 7fbb3ad9d700 1 --2- 172.21.15.89:0/647586780 >> v2:172.21.15.111:3300/0 conn(0x7fbb3c069560 0x7fbb3c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.615+0000 7fbb39d9b700 1 -- 172.21.15.89:0/647586780 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb28003570 con 0x7fbb3c159890 2023-10-10T20:43:03.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb39d9b700 1 -- 172.21.15.89:0/647586780 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3987039714 0 0) 0x7fbb28003570 con 0x7fbb3c159890 2023-10-10T20:43:03.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb39d9b700 1 -- 172.21.15.89:0/647586780 >> v2:172.21.15.111:3300/0 conn(0x7fbb3c069560 msgr2=0x7fbb3c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:03.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb39d9b700 1 --2- 172.21.15.89:0/647586780 >> v2:172.21.15.111:3300/0 conn(0x7fbb3c069560 0x7fbb3c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb39d9b700 1 -- 172.21.15.89:0/647586780 >> v1:172.21.15.111:6789/0 conn(0x7fbb3c159c60 legacy=0x7fbb3c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:03.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb39d9b700 1 -- 172.21.15.89:0/647586780 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c15e140 con 0x7fbb3c159890 2023-10-10T20:43:03.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb39d9b700 1 -- 172.21.15.89:0/647586780 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fbb2c001470 con 0x7fbb3c159890 2023-10-10T20:43:03.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb39d9b700 1 -- 172.21.15.89:0/647586780 >> v1:172.21.15.89:6789/0 conn(0x7fbb3c159890 legacy=0x7fbb3c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:03.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb39d9b700 1 --2- 172.21.15.89:0/647586780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28003a60 0x7fbb28003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb39d9b700 1 --2- 172.21.15.89:0/647586780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb28004810 0x7fbb28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb39d9b700 1 --2- 172.21.15.89:0/647586780 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb280051c0 0x7fbb2800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb39d9b700 1 -- 172.21.15.89:0/647586780 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0d13a0 con 0x7fbb28003a60 2023-10-10T20:43:03.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb39d9b700 1 -- 172.21.15.89:0/647586780 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2800c0d0 con 0x7fbb28004810 2023-10-10T20:43:03.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb39d9b700 1 -- 172.21.15.89:0/647586780 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2800c290 con 0x7fbb280051c0 2023-10-10T20:43:03.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb3a59c700 1 --2- 172.21.15.89:0/647586780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28003a60 0x7fbb28003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb31d9b700 1 --2- 172.21.15.89:0/647586780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb28004810 0x7fbb28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb3ad9d700 1 --2- 172.21.15.89:0/647586780 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb280051c0 0x7fbb2800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb3a59c700 1 -- 172.21.15.89:0/647586780 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb280051c0 msgr2=0x7fbb2800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:03.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb3a59c700 1 --2- 172.21.15.89:0/647586780 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb280051c0 0x7fbb2800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb3a59c700 1 -- 172.21.15.89:0/647586780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb28004810 msgr2=0x7fbb28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:03.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb3a59c700 1 --2- 172.21.15.89:0/647586780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb28004810 0x7fbb28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb3a59c700 1 -- 172.21.15.89:0/647586780 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb28003570 con 0x7fbb28003a60 2023-10-10T20:43:03.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb31d9b700 1 --2- 172.21.15.89:0/647586780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb28004810 0x7fbb28004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:03.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb3ad9d700 1 -- 172.21.15.89:0/647586780 reap_dead start 2023-10-10T20:43:03.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb3a59c700 1 --2- 172.21.15.89:0/647586780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28003a60 0x7fbb28003e70 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00c3c0 tx=0x7fbb2c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eccb281a68480d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:03.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb39d9b700 1 -- 172.21.15.89:0/647586780 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00cc90 con 0x7fbb28003a60 2023-10-10T20:43:03.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb39d9b700 1 -- 172.21.15.89:0/647586780 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c00cdf0 con 0x7fbb28003a60 2023-10-10T20:43:03.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.616+0000 7fbb39d9b700 1 -- 172.21.15.89:0/647586780 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c005b50 con 0x7fbb28003a60 2023-10-10T20:43:03.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.617+0000 7fbb40e8f700 1 -- 172.21.15.89:0/647586780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28003a60 msgr2=0x7fbb28003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:03.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.617+0000 7fbb40e8f700 1 --2- 172.21.15.89:0/647586780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28003a60 0x7fbb28003e70 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00c3c0 tx=0x7fbb2c001e50 comp rx=0 tx=0).stop 2023-10-10T20:43:03.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.617+0000 7fbb40e8f700 1 -- 172.21.15.89:0/647586780 shutdown_connections 2023-10-10T20:43:03.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.617+0000 7fbb40e8f700 1 --2- 172.21.15.89:0/647586780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28003a60 0x7fbb28003e70 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.617+0000 7fbb40e8f700 1 -- 172.21.15.89:0/647586780 >> 172.21.15.89:0/647586780 conn(0x7fbb3c068e20 msgr2=0x7fbb3c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:03.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.617+0000 7fbb40e8f700 1 -- 172.21.15.89:0/647586780 shutdown_connections 2023-10-10T20:43:03.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.617+0000 7fbb40e8f700 1 -- 172.21.15.89:0/647586780 wait complete. 2023-10-10T20:43:03.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.618+0000 7fbb40e8f700 1 Processor -- start 2023-10-10T20:43:03.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.618+0000 7fbb40e8f700 1 -- start start 2023-10-10T20:43:03.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.619+0000 7fbb40e8f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb28003a60 0x7fbb3c152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.619+0000 7fbb40e8f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb3c0cdc00 0x7fbb3c0ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.619+0000 7fbb40e8f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb3c15cdb0 0x7fbb3c15d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.619+0000 7fbb40e8f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c15d760 con 0x7fbb3c0cdc00 2023-10-10T20:43:03.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.619+0000 7fbb40e8f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c152fa0 con 0x7fbb3c15cdb0 2023-10-10T20:43:03.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.619+0000 7fbb40e8f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0ce6a0 con 0x7fbb28003a60 2023-10-10T20:43:03.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.619+0000 7fbb31d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb3c0cdc00 0x7fbb3c0ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.619+0000 7fbb31d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb3c0cdc00 0x7fbb3c0ce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47486/0 (socket says 172.21.15.89:47486) 2023-10-10T20:43:03.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.619+0000 7fbb31d9b700 1 -- 172.21.15.89:0/1708406521 learned_addr learned my addr 172.21.15.89:0/1708406521 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:03.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.619+0000 7fbb3ad9d700 1 --2- 172.21.15.89:0/1708406521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb3c15cdb0 0x7fbb3c15d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.619+0000 7fbb3a59c700 1 --2- 172.21.15.89:0/1708406521 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb28003a60 0x7fbb3c152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.619+0000 7fbb31d9b700 1 -- 172.21.15.89:0/1708406521 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb28003a60 msgr2=0x7fbb3c152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:03.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.619+0000 7fbb31d9b700 1 --2- 172.21.15.89:0/1708406521 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb28003a60 0x7fbb3c152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.619+0000 7fbb31d9b700 1 -- 172.21.15.89:0/1708406521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb3c15cdb0 msgr2=0x7fbb3c15d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:03.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.619+0000 7fbb31d9b700 1 --2- 172.21.15.89:0/1708406521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb3c15cdb0 0x7fbb3c15d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.619+0000 7fbb31d9b700 1 -- 172.21.15.89:0/1708406521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c0c7940 con 0x7fbb3c0cdc00 2023-10-10T20:43:03.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.619+0000 7fbb3ad9d700 1 --2- 172.21.15.89:0/1708406521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb3c15cdb0 0x7fbb3c15d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:03.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.620+0000 7fbb31d9b700 1 --2- 172.21.15.89:0/1708406521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb3c0cdc00 0x7fbb3c0ce070 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fbb24000c00 tx=0x7fbb24009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:03.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.620+0000 7fbb32ffd700 1 -- 172.21.15.89:0/1708406521 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24010520 con 0x7fbb3c0cdc00 2023-10-10T20:43:03.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.620+0000 7fbb32ffd700 1 -- 172.21.15.89:0/1708406521 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb24010680 con 0x7fbb3c0cdc00 2023-10-10T20:43:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.620+0000 7fbb32ffd700 1 -- 172.21.15.89:0/1708406521 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24010a40 con 0x7fbb3c0cdc00 2023-10-10T20:43:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.620+0000 7fbb40e8f700 1 -- 172.21.15.89:0/1708406521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3c0c7c20 con 0x7fbb3c0cdc00 2023-10-10T20:43:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.620+0000 7fbb40e8f700 1 -- 172.21.15.89:0/1708406521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3c0c81e0 con 0x7fbb3c0cdc00 2023-10-10T20:43:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.621+0000 7fbb32ffd700 1 -- 172.21.15.89:0/1708406521 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbb24014480 con 0x7fbb3c0cdc00 2023-10-10T20:43:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.621+0000 7fbb32ffd700 1 --2- 172.21.15.89:0/1708406521 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb1c067320 0x7fbb1c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.621+0000 7fbb32ffd700 1 -- 172.21.15.89:0/1708406521 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbb24084660 con 0x7fbb3c0cdc00 2023-10-10T20:43:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.622+0000 7fbb3a59c700 1 --2- 172.21.15.89:0/1708406521 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb1c067320 0x7fbb1c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.622+0000 7fbb3a59c700 1 --2- 172.21.15.89:0/1708406521 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb1c067320 0x7fbb1c0697d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c001ed0 tx=0x7fbb2c015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.623+0000 7fbb4a188b80 1 Processor -- start 2023-10-10T20:43:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.623+0000 7fbb4a188b80 1 -- start start 2023-10-10T20:43:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.624+0000 7fbb4a188b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb51a50 0x5650dcb51e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.624+0000 7fbb4a188b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb9bd00 0x5650dcb7aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.624+0000 7fbb4a188b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb7af60 0x5650dcb4dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.624+0000 7fbb4a188b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650dcaa8a70 con 0x5650dcb7af60 2023-10-10T20:43:03.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.624+0000 7fbb4a188b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650dcaa6270 con 0x5650dcb51a50 2023-10-10T20:43:03.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.624+0000 7fbb4a188b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650dcaa2060 con 0x5650dcb9bd00 2023-10-10T20:43:03.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.624+0000 7fbb3ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb7af60 0x5650dcb4dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.624+0000 7fbb3ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb7af60 0x5650dcb4dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47502/0 (socket says 172.21.15.89:47502) 2023-10-10T20:43:03.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.624+0000 7fbb3ad9d700 1 -- 172.21.15.89:0/302767162 learned_addr learned my addr 172.21.15.89:0/302767162 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:03.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.624+0000 7fbb3ad9d700 1 -- 172.21.15.89:0/302767162 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb9bd00 msgr2=0x5650dcb7aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:03.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.624+0000 7fbb3ad9d700 1 --2- 172.21.15.89:0/302767162 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb9bd00 0x5650dcb7aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.624+0000 7fbb3ad9d700 1 -- 172.21.15.89:0/302767162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb51a50 msgr2=0x5650dcb51e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:03.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.624+0000 7fbb3a59c700 1 --2- 172.21.15.89:0/302767162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb51a50 0x5650dcb51e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.624+0000 7fbb3ad9d700 1 --2- 172.21.15.89:0/302767162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb51a50 0x5650dcb51e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.624+0000 7fbb3ad9d700 1 -- 172.21.15.89:0/302767162 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650dc93c270 con 0x5650dcb7af60 2023-10-10T20:43:03.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.625+0000 7fbb3a59c700 1 --2- 172.21.15.89:0/302767162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb51a50 0x5650dcb51e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:03.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.625+0000 7fbb3ad9d700 1 --2- 172.21.15.89:0/302767162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb7af60 0x5650dcb4dcf0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fbb340024d0 tx=0x7fbb3400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35957ff4fdb21615 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:03.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.625+0000 7fbb1b7fe700 1 -- 172.21.15.89:0/302767162 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3400e030 con 0x5650dcb7af60 2023-10-10T20:43:03.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.625+0000 7fbb1b7fe700 1 -- 172.21.15.89:0/302767162 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb34016460 con 0x5650dcb7af60 2023-10-10T20:43:03.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.625+0000 7fbb1b7fe700 1 -- 172.21.15.89:0/302767162 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb34016820 con 0x5650dcb7af60 2023-10-10T20:43:03.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.625+0000 7fbb4a188b80 1 -- 172.21.15.89:0/302767162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb7af60 msgr2=0x5650dcb4dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:03.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.625+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/302767162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb7af60 0x5650dcb4dcf0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fbb340024d0 tx=0x7fbb3400b040 comp rx=0 tx=0).stop 2023-10-10T20:43:03.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.625+0000 7fbb4a188b80 1 -- 172.21.15.89:0/302767162 shutdown_connections 2023-10-10T20:43:03.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.625+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/302767162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb7af60 0x5650dcb4dcf0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.625+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/302767162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb51a50 0x5650dcb51e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.625+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/302767162 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb9bd00 0x5650dcb7aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.625+0000 7fbb4a188b80 1 -- 172.21.15.89:0/302767162 >> 172.21.15.89:0/302767162 conn(0x5650dcb53370 msgr2=0x5650dcb53740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:03.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.626+0000 7fbb4a188b80 1 -- 172.21.15.89:0/302767162 shutdown_connections 2023-10-10T20:43:03.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.626+0000 7fbb4a188b80 1 -- 172.21.15.89:0/302767162 wait complete. 2023-10-10T20:43:03.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.626+0000 7fbb4a188b80 1 -- start start 2023-10-10T20:43:03.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.626+0000 7fbb4a188b80 1 Processor -- start 2023-10-10T20:43:03.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.627+0000 7fbb4a188b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb51a50 0x5650dcb91000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.627+0000 7fbb4a188b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb7af60 0x5650dcb9b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.627+0000 7fbb4a188b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb9bd00 0x5650dcb93e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.627+0000 7fbb4a188b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650dca4d890 con 0x5650dcb51a50 2023-10-10T20:43:03.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.627+0000 7fbb4a188b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650dca68ea0 con 0x5650dcb7af60 2023-10-10T20:43:03.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.627+0000 7fbb4a188b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650dc924e30 con 0x5650dcb9bd00 2023-10-10T20:43:03.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.627+0000 7fbb4a188b80 10 client.0 _ll_register_callbacks cb 0x5650dcb15110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:43:03.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.627+0000 7fbb3a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb51a50 0x5650dcb91000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.627+0000 7fbb3a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb51a50 0x5650dcb91000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47518/0 (socket says 172.21.15.89:47518) 2023-10-10T20:43:03.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.627+0000 7fbb3a59c700 1 -- 172.21.15.89:0/2657469140 learned_addr learned my addr 172.21.15.89:0/2657469140 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:03.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.627+0000 7fbb31d9b700 1 --2- 172.21.15.89:0/2657469140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb7af60 0x5650dcb9b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.627+0000 7fbb3ad9d700 1 --2- 172.21.15.89:0/2657469140 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb9bd00 0x5650dcb93e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.627+0000 7fbb3a59c700 1 -- 172.21.15.89:0/2657469140 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb9bd00 msgr2=0x5650dcb93e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:03.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.627+0000 7fbb3a59c700 1 --2- 172.21.15.89:0/2657469140 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb9bd00 0x5650dcb93e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.627+0000 7fbb3a59c700 1 -- 172.21.15.89:0/2657469140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb7af60 msgr2=0x5650dcb9b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:03.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.627+0000 7fbb3a59c700 1 --2- 172.21.15.89:0/2657469140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb7af60 0x5650dcb9b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.627+0000 7fbb3a59c700 1 -- 172.21.15.89:0/2657469140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb34002280 con 0x5650dcb51a50 2023-10-10T20:43:03.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.628+0000 7fbb3a59c700 1 --2- 172.21.15.89:0/2657469140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb51a50 0x5650dcb91000 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800c0a0 tx=0x7fbb2c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbc0438ba84cec4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:03.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.628+0000 7fbb18ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:43:03.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.628+0000 7fbb18ff9700 1 -- 172.21.15.89:0/2657469140 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c000c90 con 0x5650dcb51a50 2023-10-10T20:43:03.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.628+0000 7fbb18ff9700 1 -- 172.21.15.89:0/2657469140 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbb2c0072e0 con 0x5650dcb51a50 2023-10-10T20:43:03.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.628+0000 7fbb4a188b80 1 -- 172.21.15.89:0/2657469140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650dcb46b70 con 0x5650dcb51a50 2023-10-10T20:43:03.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.628+0000 7fbb18ff9700 1 -- 172.21.15.89:0/2657469140 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb2c0076b0 con 0x5650dcb51a50 2023-10-10T20:43:03.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.628+0000 7fbb4a188b80 10 client.8499 fetch_fsmap learned FSMap version 110 2023-10-10T20:43:03.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.628+0000 7fbb4a188b80 1 -- 172.21.15.89:0/2657469140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) v3 -- 0x5650dc8defe0 con 0x5650dcb51a50 2023-10-10T20:43:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.628+0000 7fbb18ff9700 1 -- 172.21.15.89:0/2657469140 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5650dcb46b70 con 0x5650dcb51a50 2023-10-10T20:43:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.628+0000 7fbb18ff9700 1 -- 172.21.15.89:0/2657469140 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 110) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fbb2c02b450 con 0x5650dcb51a50 2023-10-10T20:43:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.628+0000 7fbb4a188b80 10 client.8499 fetch_fsmap finished waiting for FSMap version 110 2023-10-10T20:43:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.628+0000 7fbb4a188b80 10 client.8499 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-10T20:43:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.628+0000 7fbb4a188b80 20 client.8499 populate_metadata read hostname 'smithi089' 2023-10-10T20:43:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.628+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/2657469140 >> [v2:172.21.15.111:6832/1733913702,v1:172.21.15.111:6834/1733913702] conn(0x5650dcb943c0 0x5650dcb901f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.628+0000 7fbb4a188b80 4 client.8499 mds_command: new command op to 8337 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:43:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.628+0000 7fbb4a188b80 1 -- 172.21.15.89:0/2657469140 --> [v2:172.21.15.111:6832/1733913702,v1:172.21.15.111:6834/1733913702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650dcb23aa0 con 0x5650dcb943c0 2023-10-10T20:43:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.628+0000 7fbb31d9b700 1 --2- 172.21.15.89:0/2657469140 >> [v2:172.21.15.111:6832/1733913702,v1:172.21.15.111:6834/1733913702] conn(0x5650dcb943c0 0x5650dcb901f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.629+0000 7fbb18ff9700 10 client.8499 ms_handle_connect on v2:172.21.15.111:6832/1733913702 2023-10-10T20:43:03.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.629+0000 7fbb31d9b700 1 --2- 172.21.15.89:0/2657469140 >> [v2:172.21.15.111:6832/1733913702,v1:172.21.15.111:6834/1733913702] conn(0x5650dcb943c0 0x5650dcb901f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15604923f342e7fc server_cookie=486251c13c42b5a2 in_seq=0 out_seq=0 2023-10-10T20:43:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.629+0000 7fbb18ff9700 1 -- 172.21.15.89:0/2657469140 <== mds.0 v2:172.21.15.111:6832/1733913702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5650dcb23aa0 con 0x5650dcb943c0 2023-10-10T20:43:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.629+0000 7fbb18ff9700 10 client.8499 handle_command_reply: tid=0 2023-10-10T20:43:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.630+0000 7fbb4a188b80 1 client.8499 shutdown 2023-10-10T20:43:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.630+0000 7fbb4a188b80 1 -- 172.21.15.89:0/2657469140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb51a50 msgr2=0x5650dcb91000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.630+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/2657469140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb51a50 0x5650dcb91000 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800c0a0 tx=0x7fbb2c01f000 comp rx=0 tx=0).stop 2023-10-10T20:43:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.630+0000 7fbb4a188b80 1 -- 172.21.15.89:0/2657469140 shutdown_connections 2023-10-10T20:43:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.630+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/2657469140 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb9bd00 0x5650dcb93e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.630+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/2657469140 >> [v2:172.21.15.111:6832/1733913702,v1:172.21.15.111:6834/1733913702] conn(0x5650dcb943c0 0x5650dcb901f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.630+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/2657469140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb51a50 0x5650dcb91000 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.630+0000 7fbb18ff9700 0 client.8499 ms_handle_reset on v2:172.21.15.111:6832/1733913702 2023-10-10T20:43:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.630+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/2657469140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb7af60 0x5650dcb9b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.630+0000 7fbb4a188b80 1 -- 172.21.15.89:0/2657469140 >> 172.21.15.89:0/2657469140 conn(0x5650dcb53370 msgr2=0x5650dcb4e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.630+0000 7fbb4a188b80 1 -- 172.21.15.89:0/2657469140 shutdown_connections 2023-10-10T20:43:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.630+0000 7fbb4a188b80 1 -- 172.21.15.89:0/2657469140 wait complete. 2023-10-10T20:43:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.630+0000 7fbb4a188b80 20 client.8499 trim_cache size 0 max 16384 2023-10-10T20:43:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.647+0000 7fbb4a188b80 1 Processor -- start 2023-10-10T20:43:03.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.647+0000 7fbb4a188b80 1 -- start start 2023-10-10T20:43:03.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.647+0000 7fbb4a188b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb51a50 0x5650dcb97460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.647+0000 7fbb4a188b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb7af60 0x5650dcb979a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.647+0000 7fbb4a188b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb9bd00 0x5650dcb4e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.647+0000 7fbb4a188b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650dcb81d10 con 0x5650dcb9bd00 2023-10-10T20:43:03.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.647+0000 7fbb4a188b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650dcb52320 con 0x5650dcb51a50 2023-10-10T20:43:03.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.647+0000 7fbb4a188b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650dcb951f0 con 0x5650dcb7af60 2023-10-10T20:43:03.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.647+0000 7fbb3ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb9bd00 0x5650dcb4e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.648+0000 7fbb3ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb9bd00 0x5650dcb4e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47528/0 (socket says 172.21.15.89:47528) 2023-10-10T20:43:03.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.648+0000 7fbb3ad9d700 1 -- 172.21.15.89:0/471213601 learned_addr learned my addr 172.21.15.89:0/471213601 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.648+0000 7fbb31d9b700 1 --2- 172.21.15.89:0/471213601 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb7af60 0x5650dcb979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.648+0000 7fbb3a59c700 1 --2- 172.21.15.89:0/471213601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb51a50 0x5650dcb97460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.648+0000 7fbb3ad9d700 1 -- 172.21.15.89:0/471213601 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb7af60 msgr2=0x5650dcb979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.648+0000 7fbb3ad9d700 1 --2- 172.21.15.89:0/471213601 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb7af60 0x5650dcb979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.648+0000 7fbb3ad9d700 1 -- 172.21.15.89:0/471213601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb51a50 msgr2=0x5650dcb97460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:03.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.648+0000 7fbb3ad9d700 1 --2- 172.21.15.89:0/471213601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb51a50 0x5650dcb97460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.648+0000 7fbb3ad9d700 1 -- 172.21.15.89:0/471213601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650dca70780 con 0x5650dcb9bd00 2023-10-10T20:43:03.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.648+0000 7fbb31d9b700 1 --2- 172.21.15.89:0/471213601 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb7af60 0x5650dcb979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:03.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.648+0000 7fbb3a59c700 1 --2- 172.21.15.89:0/471213601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb51a50 0x5650dcb97460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:03.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.648+0000 7fbb3ad9d700 1 --2- 172.21.15.89:0/471213601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb9bd00 0x5650dcb4e8f0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fbb34018000 tx=0x7fbb3401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4306c555d7674f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.648+0000 7fbb1bfff700 1 -- 172.21.15.89:0/471213601 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3400e040 con 0x5650dcb9bd00 2023-10-10T20:43:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.648+0000 7fbb1bfff700 1 -- 172.21.15.89:0/471213601 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb34001090 con 0x5650dcb9bd00 2023-10-10T20:43:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.648+0000 7fbb1bfff700 1 -- 172.21.15.89:0/471213601 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb34016820 con 0x5650dcb9bd00 2023-10-10T20:43:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.648+0000 7fbb4a188b80 1 -- 172.21.15.89:0/471213601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb9bd00 msgr2=0x5650dcb4e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.648+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/471213601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb9bd00 0x5650dcb4e8f0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fbb34018000 tx=0x7fbb3401c000 comp rx=0 tx=0).stop 2023-10-10T20:43:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.649+0000 7fbb4a188b80 1 -- 172.21.15.89:0/471213601 shutdown_connections 2023-10-10T20:43:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.649+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/471213601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb9bd00 0x5650dcb4e8f0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.649+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/471213601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb51a50 0x5650dcb97460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.649+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/471213601 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb7af60 0x5650dcb979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.649+0000 7fbb4a188b80 1 -- 172.21.15.89:0/471213601 >> 172.21.15.89:0/471213601 conn(0x5650dcb53370 msgr2=0x5650dcb90e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:03.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.649+0000 7fbb4a188b80 1 -- 172.21.15.89:0/471213601 shutdown_connections 2023-10-10T20:43:03.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.649+0000 7fbb4a188b80 1 -- 172.21.15.89:0/471213601 wait complete. 2023-10-10T20:43:03.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.649+0000 7fbb4a188b80 1 -- start start 2023-10-10T20:43:03.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.649+0000 7fbb4a188b80 1 Processor -- start 2023-10-10T20:43:03.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.650+0000 7fbb4a188b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb51a50 0x5650dcbf8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.650+0000 7fbb4a188b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb7af60 0x5650dcb90360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.650+0000 7fbb4a188b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb9bd00 0x5650dcb97460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.650+0000 7fbb4a188b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650dcbff580 con 0x5650dcb51a50 2023-10-10T20:43:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.650+0000 7fbb4a188b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650dcbff9e0 con 0x5650dcb7af60 2023-10-10T20:43:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.650+0000 7fbb4a188b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650dcbff330 con 0x5650dcb9bd00 2023-10-10T20:43:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.650+0000 7fbb4a188b80 10 client.0 _ll_register_callbacks cb 0x5650dcb15110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:43:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.650+0000 7fbb3a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb51a50 0x5650dcbf8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.650+0000 7fbb3a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb51a50 0x5650dcbf8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47532/0 (socket says 172.21.15.89:47532) 2023-10-10T20:43:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.650+0000 7fbb3a59c700 1 -- 172.21.15.89:0/2733005592 learned_addr learned my addr 172.21.15.89:0/2733005592 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.650+0000 7fbb31d9b700 1 --2- 172.21.15.89:0/2733005592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb7af60 0x5650dcb90360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.650+0000 7fbb3ad9d700 1 --2- 172.21.15.89:0/2733005592 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb9bd00 0x5650dcb97460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.650+0000 7fbb3a59c700 1 -- 172.21.15.89:0/2733005592 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb9bd00 msgr2=0x5650dcb97460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.650+0000 7fbb3a59c700 1 --2- 172.21.15.89:0/2733005592 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb9bd00 0x5650dcb97460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.650+0000 7fbb3a59c700 1 -- 172.21.15.89:0/2733005592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb7af60 msgr2=0x5650dcb90360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.650+0000 7fbb3a59c700 1 --2- 172.21.15.89:0/2733005592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb7af60 0x5650dcb90360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.650+0000 7fbb3a59c700 1 -- 172.21.15.89:0/2733005592 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650dc93c270 con 0x5650dcb51a50 2023-10-10T20:43:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.651+0000 7fbb3a59c700 1 --2- 172.21.15.89:0/2733005592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb51a50 0x5650dcbf8310 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c007fd0 tx=0x7fbb2c02b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0034c2a70f13a0e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.651+0000 7fbb18ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:43:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.651+0000 7fbb18ff9700 1 -- 172.21.15.89:0/2733005592 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c02ba00 con 0x5650dcb51a50 2023-10-10T20:43:03.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.651+0000 7fbb18ff9700 1 -- 172.21.15.89:0/2733005592 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbb2c02b450 con 0x5650dcb51a50 2023-10-10T20:43:03.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.651+0000 7fbb4a188b80 1 -- 172.21.15.89:0/2733005592 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb34016460 con 0x5650dcb51a50 2023-10-10T20:43:03.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.651+0000 7fbb18ff9700 1 -- 172.21.15.89:0/2733005592 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb2c02be20 con 0x5650dcb51a50 2023-10-10T20:43:03.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.651+0000 7fbb4a188b80 10 client.8505 fetch_fsmap learned FSMap version 110 2023-10-10T20:43:03.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.651+0000 7fbb4a188b80 1 -- 172.21.15.89:0/2733005592 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) v3 -- 0x5650dca57470 con 0x5650dcb51a50 2023-10-10T20:43:03.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.651+0000 7fbb18ff9700 1 -- 172.21.15.89:0/2733005592 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb34016460 con 0x5650dcb51a50 2023-10-10T20:43:03.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.651+0000 7fbb18ff9700 1 -- 172.21.15.89:0/2733005592 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 110) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fbb2c012400 con 0x5650dcb51a50 2023-10-10T20:43:03.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.651+0000 7fbb4a188b80 10 client.8505 fetch_fsmap finished waiting for FSMap version 110 2023-10-10T20:43:03.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.651+0000 7fbb4a188b80 10 client.8505 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-10T20:43:03.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.651+0000 7fbb4a188b80 20 client.8505 populate_metadata read hostname 'smithi089' 2023-10-10T20:43:03.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.651+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/2733005592 >> [v2:172.21.15.111:6832/1733913702,v1:172.21.15.111:6834/1733913702] conn(0x5650dcb93520 0x5650dcb938f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.651+0000 7fbb4a188b80 4 client.8505 mds_command: new command op to 8337 tid=0[{"prefix": "status"}] 2023-10-10T20:43:03.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.651+0000 7fbb4a188b80 1 -- 172.21.15.89:0/2733005592 --> [v2:172.21.15.111:6832/1733913702,v1:172.21.15.111:6834/1733913702] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5650dca44e00 con 0x5650dcb93520 2023-10-10T20:43:03.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.651+0000 7fbb31d9b700 1 --2- 172.21.15.89:0/2733005592 >> [v2:172.21.15.111:6832/1733913702,v1:172.21.15.111:6834/1733913702] conn(0x5650dcb93520 0x5650dcb938f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.652+0000 7fbb18ff9700 10 client.8505 ms_handle_connect on v2:172.21.15.111:6832/1733913702 2023-10-10T20:43:03.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.652+0000 7fbb31d9b700 1 --2- 172.21.15.89:0/2733005592 >> [v2:172.21.15.111:6832/1733913702,v1:172.21.15.111:6834/1733913702] conn(0x5650dcb93520 0x5650dcb938f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9900e94d31a88164 server_cookie=a34342da1ca995ac in_seq=0 out_seq=0 2023-10-10T20:43:03.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.652+0000 7fbb18ff9700 1 -- 172.21.15.89:0/2733005592 <== mds.0 v2:172.21.15.111:6832/1733913702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x5650dca44e00 con 0x5650dcb93520 2023-10-10T20:43:03.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.652+0000 7fbb18ff9700 10 client.8505 handle_command_reply: tid=0 2023-10-10T20:43:03.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.652+0000 7fbb4a188b80 1 client.8505 shutdown 2023-10-10T20:43:03.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.652+0000 7fbb4a188b80 1 -- 172.21.15.89:0/2733005592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb51a50 msgr2=0x5650dcbf8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:03.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.652+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/2733005592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb51a50 0x5650dcbf8310 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c007fd0 tx=0x7fbb2c02b000 comp rx=0 tx=0).stop 2023-10-10T20:43:03.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.652+0000 7fbb4a188b80 1 -- 172.21.15.89:0/2733005592 shutdown_connections 2023-10-10T20:43:03.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.652+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/2733005592 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5650dcb9bd00 0x5650dcb97460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.652+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/2733005592 >> [v2:172.21.15.111:6832/1733913702,v1:172.21.15.111:6834/1733913702] conn(0x5650dcb93520 0x5650dcb938f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.652+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/2733005592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650dcb51a50 0x5650dcbf8310 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.652+0000 7fbb18ff9700 0 client.8505 ms_handle_reset on v2:172.21.15.111:6832/1733913702 2023-10-10T20:43:03.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.652+0000 7fbb4a188b80 1 --2- 172.21.15.89:0/2733005592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5650dcb7af60 0x5650dcb90360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.652+0000 7fbb4a188b80 1 -- 172.21.15.89:0/2733005592 >> 172.21.15.89:0/2733005592 conn(0x5650dcb53370 msgr2=0x5650dcb79a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:03.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.652+0000 7fbb4a188b80 1 -- 172.21.15.89:0/2733005592 shutdown_connections 2023-10-10T20:43:03.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.652+0000 7fbb4a188b80 1 -- 172.21.15.89:0/2733005592 wait complete. 2023-10-10T20:43:03.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.652+0000 7fbb4a188b80 20 client.8505 trim_cache size 0 max 16384 2023-10-10T20:43:03.687 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:43:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "5dd675b8-9026-4d8a-a1bf-62fdc5e7de67", 2023-10-10T20:43:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-10-10T20:43:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 8337, 2023-10-10T20:43:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-10-10T20:43:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-10-10T20:43:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-10-10T20:43:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 10.310201562, 2023-10-10T20:43:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 110, 2023-10-10T20:43:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 165, 2023-10-10T20:43:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 165, 2023-10-10T20:43:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 15.437548057000001 2023-10-10T20:43:03.690 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:43:03.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.653+0000 7fbb40e8f700 1 -- 172.21.15.89:0/1708406521 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb1c067320 msgr2=0x7fbb1c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:03.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.653+0000 7fbb40e8f700 1 --2- 172.21.15.89:0/1708406521 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb1c067320 0x7fbb1c0697d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c001ed0 tx=0x7fbb2c015000 comp rx=0 tx=0).stop 2023-10-10T20:43:03.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.653+0000 7fbb40e8f700 1 -- 172.21.15.89:0/1708406521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb3c0cdc00 msgr2=0x7fbb3c0ce070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:03.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.653+0000 7fbb40e8f700 1 --2- 172.21.15.89:0/1708406521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb3c0cdc00 0x7fbb3c0ce070 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fbb24000c00 tx=0x7fbb24009040 comp rx=0 tx=0).stop 2023-10-10T20:43:03.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.653+0000 7fbb40e8f700 1 -- 172.21.15.89:0/1708406521 shutdown_connections 2023-10-10T20:43:03.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.653+0000 7fbb40e8f700 1 --2- 172.21.15.89:0/1708406521 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbb1c067320 0x7fbb1c0697d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.653+0000 7fbb40e8f700 1 --2- 172.21.15.89:0/1708406521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb3c15cdb0 0x7fbb3c15d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.653+0000 7fbb40e8f700 1 --2- 172.21.15.89:0/1708406521 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbb28003a60 0x7fbb3c152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.653+0000 7fbb40e8f700 1 --2- 172.21.15.89:0/1708406521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb3c0cdc00 0x7fbb3c0ce070 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.653+0000 7fbb40e8f700 1 -- 172.21.15.89:0/1708406521 >> 172.21.15.89:0/1708406521 conn(0x7fbb3c068e20 msgr2=0x7fbb3c0b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:03.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.653+0000 7fbb40e8f700 1 -- 172.21.15.89:0/1708406521 shutdown_connections 2023-10-10T20:43:03.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.653+0000 7fbb40e8f700 1 -- 172.21.15.89:0/1708406521 wait complete. 2023-10-10T20:43:03.694 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:43:03.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.806+0000 7fdf89f88700 1 Processor -- start 2023-10-10T20:43:03.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.806+0000 7fdf89f88700 1 -- start start 2023-10-10T20:43:03.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.807+0000 7fdf89f88700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdf84159890 0x7fdf8415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.807+0000 7fdf89f88700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdf84069560 0x7fdf840699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.807+0000 7fdf89f88700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf84069ee0 con 0x7fdf84159c60 2023-10-10T20:43:03.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.807+0000 7fdf89f88700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf8415a160 con 0x7fdf84159890 2023-10-10T20:43:03.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.807+0000 7fdf89f88700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf840d13a0 con 0x7fdf84069560 2023-10-10T20:43:03.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.807+0000 7fdf837fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdf84159890 0x7fdf8415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.807+0000 7fdf82ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdf84159c60 0x7fdf8415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38402/0 (socket says 172.21.15.89:38402) 2023-10-10T20:43:03.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.807+0000 7fdf82ffd700 1 -- 172.21.15.89:0/2529765255 learned_addr learned my addr 172.21.15.89:0/2529765255 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:03.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.808+0000 7fdf83fff700 1 --2- 172.21.15.89:0/2529765255 >> v2:172.21.15.111:3300/0 conn(0x7fdf84069560 0x7fdf840699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.808+0000 7fdf837fe700 1 -- 172.21.15.89:0/2529765255 >> v2:172.21.15.111:3300/0 conn(0x7fdf84069560 msgr2=0x7fdf840699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:03.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.808+0000 7fdf837fe700 1 --2- 172.21.15.89:0/2529765255 >> v2:172.21.15.111:3300/0 conn(0x7fdf84069560 0x7fdf840699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.808+0000 7fdf837fe700 1 -- 172.21.15.89:0/2529765255 >> v1:172.21.15.89:6789/0 conn(0x7fdf84159c60 legacy=0x7fdf8415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:03.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.808+0000 7fdf837fe700 1 -- 172.21.15.89:0/2529765255 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf8415e140 con 0x7fdf84159890 2023-10-10T20:43:03.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.808+0000 7fdf827fc700 1 -- 172.21.15.89:0/2529765255 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4022422641 0 0) 0x7fdf84069ee0 con 0x7fdf84159c60 2023-10-10T20:43:03.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.808+0000 7fdf837fe700 1 --2- 172.21.15.89:0/2529765255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf84159890 0x7fdf8415abd0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fdf8406a890 tx=0x7fdf780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6207546d96541cf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:03.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.808+0000 7fdf827fc700 1 -- 172.21.15.89:0/2529765255 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7800b6b0 con 0x7fdf84159890 2023-10-10T20:43:03.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.808+0000 7fdf827fc700 1 -- 172.21.15.89:0/2529765255 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7800b810 con 0x7fdf84159890 2023-10-10T20:43:03.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.808+0000 7fdf827fc700 1 -- 172.21.15.89:0/2529765255 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7800bbb0 con 0x7fdf84159890 2023-10-10T20:43:03.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.808+0000 7fdf89f88700 1 -- 172.21.15.89:0/2529765255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf84159890 msgr2=0x7fdf8415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:03.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.808+0000 7fdf89f88700 1 --2- 172.21.15.89:0/2529765255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf84159890 0x7fdf8415abd0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fdf8406a890 tx=0x7fdf780091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.808+0000 7fdf89f88700 1 -- 172.21.15.89:0/2529765255 shutdown_connections 2023-10-10T20:43:03.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.808+0000 7fdf89f88700 1 --2- 172.21.15.89:0/2529765255 >> v2:172.21.15.111:3300/0 conn(0x7fdf84069560 0x7fdf840699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.808+0000 7fdf89f88700 1 --2- 172.21.15.89:0/2529765255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf84159890 0x7fdf8415abd0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.808+0000 7fdf89f88700 1 -- 172.21.15.89:0/2529765255 >> 172.21.15.89:0/2529765255 conn(0x7fdf84068e20 msgr2=0x7fdf840d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:03.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.809+0000 7fdf89f88700 1 -- 172.21.15.89:0/2529765255 shutdown_connections 2023-10-10T20:43:03.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.809+0000 7fdf89f88700 1 -- 172.21.15.89:0/2529765255 wait complete. 2023-10-10T20:43:03.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.810+0000 7fdf89f88700 1 Processor -- start 2023-10-10T20:43:03.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.810+0000 7fdf89f88700 1 -- start start 2023-10-10T20:43:03.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.810+0000 7fdf89f88700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf84069560 0x7fdf8414c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.810+0000 7fdf89f88700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdf84159890 0x7fdf840cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.810+0000 7fdf89f88700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf84159c60 0x7fdf84155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.810+0000 7fdf89f88700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf840cef60 con 0x7fdf84159c60 2023-10-10T20:43:03.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.810+0000 7fdf89f88700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf841522b0 con 0x7fdf84069560 2023-10-10T20:43:03.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.810+0000 7fdf89f88700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf840ce910 con 0x7fdf84159890 2023-10-10T20:43:03.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.811+0000 7fdf83fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf84159c60 0x7fdf84155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.811+0000 7fdf83fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf84159c60 0x7fdf84155cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47550/0 (socket says 172.21.15.89:47550) 2023-10-10T20:43:03.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.811+0000 7fdf83fff700 1 -- 172.21.15.89:0/2299664547 learned_addr learned my addr 172.21.15.89:0/2299664547 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:03.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.811+0000 7fdf837fe700 1 --2- 172.21.15.89:0/2299664547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf84069560 0x7fdf8414c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.811+0000 7fdf82ffd700 1 --2- 172.21.15.89:0/2299664547 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdf84159890 0x7fdf840cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.811+0000 7fdf83fff700 1 -- 172.21.15.89:0/2299664547 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdf84159890 msgr2=0x7fdf840cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:03.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.811+0000 7fdf83fff700 1 --2- 172.21.15.89:0/2299664547 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdf84159890 0x7fdf840cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.811+0000 7fdf83fff700 1 -- 172.21.15.89:0/2299664547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf84069560 msgr2=0x7fdf8414c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:03.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.811+0000 7fdf83fff700 1 --2- 172.21.15.89:0/2299664547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf84069560 0x7fdf8414c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:03.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.811+0000 7fdf83fff700 1 -- 172.21.15.89:0/2299664547 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf78009420 con 0x7fdf84159c60 2023-10-10T20:43:03.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.811+0000 7fdf837fe700 1 --2- 172.21.15.89:0/2299664547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf84069560 0x7fdf8414c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:03.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.811+0000 7fdf83fff700 1 --2- 172.21.15.89:0/2299664547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf84159c60 0x7fdf84155cc0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009fa0 tx=0x7fdf7c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:03.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.811+0000 7fdf6b7fe700 1 -- 172.21.15.89:0/2299664547 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c00ed20 con 0x7fdf84159c60 2023-10-10T20:43:03.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.811+0000 7fdf6b7fe700 1 -- 172.21.15.89:0/2299664547 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7c00ee80 con 0x7fdf84159c60 2023-10-10T20:43:03.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.811+0000 7fdf89f88700 1 -- 172.21.15.89:0/2299664547 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf840cebf0 con 0x7fdf84159c60 2023-10-10T20:43:03.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.811+0000 7fdf6b7fe700 1 -- 172.21.15.89:0/2299664547 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c014670 con 0x7fdf84159c60 2023-10-10T20:43:03.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.811+0000 7fdf89f88700 1 -- 172.21.15.89:0/2299664547 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf840cf930 con 0x7fdf84159c60 2023-10-10T20:43:03.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.812+0000 7fdf89f88700 1 -- 172.21.15.89:0/2299664547 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf840cf0d0 con 0x7fdf84159c60 2023-10-10T20:43:03.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.813+0000 7fdf6b7fe700 1 -- 172.21.15.89:0/2299664547 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdf7c0149e0 con 0x7fdf84159c60 2023-10-10T20:43:03.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.813+0000 7fdf6b7fe700 1 --2- 172.21.15.89:0/2299664547 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdf70067320 0x7fdf700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:03.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.813+0000 7fdf6b7fe700 1 -- 172.21.15.89:0/2299664547 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdf7c085410 con 0x7fdf84159c60 2023-10-10T20:43:03.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.813+0000 7fdf837fe700 1 --2- 172.21.15.89:0/2299664547 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdf70067320 0x7fdf700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:03.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.813+0000 7fdf837fe700 1 --2- 172.21.15.89:0/2299664547 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdf70067320 0x7fdf700697d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fdf780093f0 tx=0x7fdf78017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:03.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:03.817+0000 7fdf6b7fe700 1 -- 172.21.15.89:0/2299664547 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf7c0520a0 con 0x7fdf84159c60 2023-10-10T20:43:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.043+0000 7fdf89f88700 1 -- 172.21.15.89:0/2299664547 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdf8404c3c0 con 0x7fdf84159c60 2023-10-10T20:43:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.043+0000 7fdf6b7fe700 1 -- 172.21.15.89:0/2299664547 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v110) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdf7c0556c0 con 0x7fdf84159c60 2023-10-10T20:43:04.046 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:43:04.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.045+0000 7fdf89f88700 1 -- 172.21.15.89:0/2299664547 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdf70067320 msgr2=0x7fdf700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:04.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.045+0000 7fdf89f88700 1 --2- 172.21.15.89:0/2299664547 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdf70067320 0x7fdf700697d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fdf780093f0 tx=0x7fdf78017000 comp rx=0 tx=0).stop 2023-10-10T20:43:04.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.045+0000 7fdf89f88700 1 -- 172.21.15.89:0/2299664547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf84159c60 msgr2=0x7fdf84155cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.045+0000 7fdf89f88700 1 --2- 172.21.15.89:0/2299664547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf84159c60 0x7fdf84155cc0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009fa0 tx=0x7fdf7c009900 comp rx=0 tx=0).stop 2023-10-10T20:43:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.045+0000 7fdf89f88700 1 -- 172.21.15.89:0/2299664547 shutdown_connections 2023-10-10T20:43:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.045+0000 7fdf89f88700 1 --2- 172.21.15.89:0/2299664547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf84159c60 0x7fdf84155cc0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.045+0000 7fdf89f88700 1 --2- 172.21.15.89:0/2299664547 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdf70067320 0x7fdf700697d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.045+0000 7fdf89f88700 1 --2- 172.21.15.89:0/2299664547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf84069560 0x7fdf8414c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.045+0000 7fdf89f88700 1 --2- 172.21.15.89:0/2299664547 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdf84159890 0x7fdf840cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.045+0000 7fdf89f88700 1 -- 172.21.15.89:0/2299664547 >> 172.21.15.89:0/2299664547 conn(0x7fdf84068e20 msgr2=0x7fdf840b22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.045+0000 7fdf89f88700 1 -- 172.21.15.89:0/2299664547 shutdown_connections 2023-10-10T20:43:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.045+0000 7fdf89f88700 1 -- 172.21.15.89:0/2299664547 wait complete. 2023-10-10T20:43:04.059 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:43:04.060 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:43:04.060 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-10T20:43:04.061 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:43:04.061 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:43:04.061 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:43:04.062 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:43:04.062 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:43:04.062 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:43:04.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:04.063 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-10T20:43:04.078 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:43:04.079 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:43:04.079 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:43:04.079 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:04.079 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-10T20:43:04.079 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:04.080 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:43:04.080 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:04.080 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-10T20:43:04.080 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:43:04.080 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:04.081 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-10T20:43:04.081 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:43:04.081 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:04.081 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:43:04.081 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:04.082 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:43:04.082 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 41981sec preferred_lft 41981sec 2023-10-10T20:43:04.082 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2023-10-10T20:43:04.082 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:04.082 INFO:teuthology.orchestra.run.smithi089.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:43:04.083 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:04.083 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:43:04.083 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:04.083 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::98a3:d4ff:fec1:83bb/64 scope link 2023-10-10T20:43:04.083 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:04.084 INFO:teuthology.orchestra.run.smithi089.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:43:04.084 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether ee:16:22:10:ca:9a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:43:04.084 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec16:22ff:fe10:ca9a/64 scope link 2023-10-10T20:43:04.084 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:04.084 INFO:teuthology.orchestra.run.smithi089.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:43:04.085 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:43:04.085 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::6864:2bff:fe5c:71d4/64 scope link 2023-10-10T20:43:04.085 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:04.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:04.086 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:43:04.141 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-10T20:43:04.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:04.143 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-10T20:43:04.199 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-10T20:43:04.199 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:43:04.199 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:43:04.199 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:43:04.199 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:43:04.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:04.200 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:43:04.201 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:43:04.201 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:43:04.202 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:43:04.202 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:43:04.202 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:43:04.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:04.300 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-10T20:43:04.314 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:43:04.314 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:43:04.314 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:43:04.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:04.315 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-10-10T20:43:04.379 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:43:04.379 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:43:04.434 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:43:04.434 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:43:04.435 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:43:04.489 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:43:04.489 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:43:04.490 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:43:04.568 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:43:04.569 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:43:04.569 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:43:04.576 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:43:04.576 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:43:04.576 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:43:04.577 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:43:04.577 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:43:04.599 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:43:04.599 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:43:04.599 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:43:04.600 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:43:04.600 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:43:04.600 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:43:04.600 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:43:04.601 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:43:04.602 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:43:04.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:04.603 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:43:04.667 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:43:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.818+0000 7f26941e0700 1 Processor -- start 2023-10-10T20:43:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.818+0000 7f26941e0700 1 -- start start 2023-10-10T20:43:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.819+0000 7f26941e0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f268c159890 0x7f268c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.819+0000 7f26941e0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f268c159c60 0x7f268c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.819+0000 7f26941e0700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f268c15db10 0x7f268c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.819+0000 7f26941e0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268c15a160 con 0x7f268c159890 2023-10-10T20:43:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.819+0000 7f26941e0700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268c0d13a0 con 0x7f268c15db10 2023-10-10T20:43:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.819+0000 7f26941e0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c15e140 con 0x7f268c159c60 2023-10-10T20:43:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.819+0000 7f2691f7c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f268c159890 0x7f268c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.819+0000 7f2691f7c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f268c159890 0x7f268c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47560/0 (socket says 172.21.15.89:47560) 2023-10-10T20:43:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.819+0000 7f2691f7c700 1 -- 172.21.15.89:0/503684444 learned_addr learned my addr 172.21.15.89:0/503684444 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.819+0000 7f269177b700 1 --2- 172.21.15.89:0/503684444 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f268c159c60 0x7f268c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.819+0000 7f269277d700 1 --2- 172.21.15.89:0/503684444 >> v2:172.21.15.111:3300/0 conn(0x7f268c15db10 0x7f268c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.820+0000 7f2691f7c700 1 -- 172.21.15.89:0/503684444 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f268c159c60 msgr2=0x7f268c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.820+0000 7f2691f7c700 1 --2- 172.21.15.89:0/503684444 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f268c159c60 0x7f268c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.820+0000 7f2691f7c700 1 -- 172.21.15.89:0/503684444 >> v2:172.21.15.111:3300/0 conn(0x7f268c15db10 msgr2=0x7f268c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.820+0000 7f2691f7c700 1 --2- 172.21.15.89:0/503684444 >> v2:172.21.15.111:3300/0 conn(0x7f268c15db10 0x7f268c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.820+0000 7f2691f7c700 1 -- 172.21.15.89:0/503684444 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c15a3d0 con 0x7f268c159890 2023-10-10T20:43:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.820+0000 7f2691f7c700 1 --2- 172.21.15.89:0/503684444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f268c159890 0x7f268c15abd0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f268c089b50 tx=0x7f2688009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8bed7a2efa1ef6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.820+0000 7f2690f7a700 1 -- 172.21.15.89:0/503684444 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268800e070 con 0x7f268c159890 2023-10-10T20:43:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.820+0000 7f2690f7a700 1 -- 172.21.15.89:0/503684444 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268800ba50 con 0x7f268c159890 2023-10-10T20:43:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.820+0000 7f2690f7a700 1 -- 172.21.15.89:0/503684444 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26880092e0 con 0x7f268c159890 2023-10-10T20:43:04.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.820+0000 7f26941e0700 1 -- 172.21.15.89:0/503684444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f268c159890 msgr2=0x7f268c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:04.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.820+0000 7f26941e0700 1 --2- 172.21.15.89:0/503684444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f268c159890 0x7f268c15abd0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f268c089b50 tx=0x7f2688009840 comp rx=0 tx=0).stop 2023-10-10T20:43:04.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.820+0000 7f26941e0700 1 -- 172.21.15.89:0/503684444 shutdown_connections 2023-10-10T20:43:04.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.820+0000 7f26941e0700 1 --2- 172.21.15.89:0/503684444 >> v2:172.21.15.111:3300/0 conn(0x7f268c15db10 0x7f268c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:04.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.820+0000 7f26941e0700 1 --2- 172.21.15.89:0/503684444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f268c159890 0x7f268c15abd0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.820+0000 7f26941e0700 1 --2- 172.21.15.89:0/503684444 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f268c159c60 0x7f268c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.820+0000 7f26941e0700 1 -- 172.21.15.89:0/503684444 >> 172.21.15.89:0/503684444 conn(0x7f268c068e20 msgr2=0x7f268c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.820+0000 7f26941e0700 1 -- 172.21.15.89:0/503684444 shutdown_connections 2023-10-10T20:43:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.820+0000 7f26941e0700 1 -- 172.21.15.89:0/503684444 wait complete. 2023-10-10T20:43:04.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.821+0000 7f26941e0700 1 Processor -- start 2023-10-10T20:43:04.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.821+0000 7f26941e0700 1 -- start start 2023-10-10T20:43:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.822+0000 7f26941e0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f268c159890 0x7f268c16a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.822+0000 7f26941e0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f268c159c60 0x7f268c147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.822+0000 7f26941e0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f268c15db10 0x7f268c0c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.822+0000 7f26941e0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0b5580 con 0x7f268c15db10 2023-10-10T20:43:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.822+0000 7f26941e0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0b63e0 con 0x7f268c159890 2023-10-10T20:43:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.822+0000 7f26941e0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0b57d0 con 0x7f268c159c60 2023-10-10T20:43:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.822+0000 7f269277d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f268c15db10 0x7f268c0c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.822+0000 7f269277d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f268c15db10 0x7f268c0c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47564/0 (socket says 172.21.15.89:47564) 2023-10-10T20:43:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.822+0000 7f269277d700 1 -- 172.21.15.89:0/1798820039 learned_addr learned my addr 172.21.15.89:0/1798820039 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.822+0000 7f269177b700 1 --2- 172.21.15.89:0/1798820039 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f268c159c60 0x7f268c147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.822+0000 7f2691f7c700 1 --2- 172.21.15.89:0/1798820039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f268c159890 0x7f268c16a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.823+0000 7f269277d700 1 -- 172.21.15.89:0/1798820039 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f268c159c60 msgr2=0x7f268c147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.823+0000 7f269277d700 1 --2- 172.21.15.89:0/1798820039 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f268c159c60 0x7f268c147210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.823+0000 7f269277d700 1 -- 172.21.15.89:0/1798820039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f268c159890 msgr2=0x7f268c16a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.823+0000 7f269277d700 1 --2- 172.21.15.89:0/1798820039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f268c159890 0x7f268c16a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.823+0000 7f269277d700 1 -- 172.21.15.89:0/1798820039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2688009ae0 con 0x7f268c15db10 2023-10-10T20:43:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.823+0000 7f269277d700 1 --2- 172.21.15.89:0/1798820039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f268c15db10 0x7f268c0c9150 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f268400d890 tx=0x7f2684008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.823+0000 7f2681ffb700 1 -- 172.21.15.89:0/1798820039 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26840125f0 con 0x7f268c15db10 2023-10-10T20:43:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.823+0000 7f2681ffb700 1 -- 172.21.15.89:0/1798820039 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2684012770 con 0x7f268c15db10 2023-10-10T20:43:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.823+0000 7f26941e0700 1 -- 172.21.15.89:0/1798820039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268c0b5ab0 con 0x7f268c15db10 2023-10-10T20:43:04.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.823+0000 7f2681ffb700 1 -- 172.21.15.89:0/1798820039 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684012b30 con 0x7f268c15db10 2023-10-10T20:43:04.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.823+0000 7f26941e0700 1 -- 172.21.15.89:0/1798820039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268c0b6070 con 0x7f268c15db10 2023-10-10T20:43:04.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.824+0000 7f26941e0700 1 -- 172.21.15.89:0/1798820039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f268c04c3c0 con 0x7f268c15db10 2023-10-10T20:43:04.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.824+0000 7f2681ffb700 1 -- 172.21.15.89:0/1798820039 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2684013480 con 0x7f268c15db10 2023-10-10T20:43:04.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.825+0000 7f2681ffb700 1 --2- 172.21.15.89:0/1798820039 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2670067320 0x7f26700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:04.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.825+0000 7f2681ffb700 1 -- 172.21.15.89:0/1798820039 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2684084f70 con 0x7f268c15db10 2023-10-10T20:43:04.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.825+0000 7f2691f7c700 1 --2- 172.21.15.89:0/1798820039 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2670067320 0x7f26700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:04.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.825+0000 7f2691f7c700 1 --2- 172.21.15.89:0/1798820039 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2670067320 0x7f26700697d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2688009880 tx=0x7f2688018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:04.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:04.829+0000 7f2681ffb700 1 -- 172.21.15.89:0/1798820039 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2684051c00 con 0x7f268c15db10 2023-10-10T20:43:05.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.056+0000 7f26941e0700 1 -- 172.21.15.89:0/1798820039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f268c06a5b0 con 0x7f268c15db10 2023-10-10T20:43:05.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.056+0000 7f2681ffb700 1 -- 172.21.15.89:0/1798820039 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v110) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2684055220 con 0x7f268c15db10 2023-10-10T20:43:05.060 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:43:05.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.058+0000 7f26941e0700 1 -- 172.21.15.89:0/1798820039 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2670067320 msgr2=0x7f26700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:05.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.058+0000 7f26941e0700 1 --2- 172.21.15.89:0/1798820039 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2670067320 0x7f26700697d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2688009880 tx=0x7f2688018000 comp rx=0 tx=0).stop 2023-10-10T20:43:05.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.058+0000 7f26941e0700 1 -- 172.21.15.89:0/1798820039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f268c15db10 msgr2=0x7f268c0c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:05.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.058+0000 7f26941e0700 1 --2- 172.21.15.89:0/1798820039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f268c15db10 0x7f268c0c9150 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f268400d890 tx=0x7f2684008040 comp rx=0 tx=0).stop 2023-10-10T20:43:05.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.058+0000 7f26941e0700 1 -- 172.21.15.89:0/1798820039 shutdown_connections 2023-10-10T20:43:05.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.058+0000 7f26941e0700 1 --2- 172.21.15.89:0/1798820039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f268c15db10 0x7f268c0c9150 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:05.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.058+0000 7f26941e0700 1 --2- 172.21.15.89:0/1798820039 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2670067320 0x7f26700697d0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:05.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.058+0000 7f26941e0700 1 --2- 172.21.15.89:0/1798820039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f268c159890 0x7f268c16a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:05.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.058+0000 7f26941e0700 1 --2- 172.21.15.89:0/1798820039 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f268c159c60 0x7f268c147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:05.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.058+0000 7f26941e0700 1 -- 172.21.15.89:0/1798820039 >> 172.21.15.89:0/1798820039 conn(0x7f268c068e20 msgr2=0x7f268c1539f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:05.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.058+0000 7f26941e0700 1 -- 172.21.15.89:0/1798820039 shutdown_connections 2023-10-10T20:43:05.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.058+0000 7f26941e0700 1 -- 172.21.15.89:0/1798820039 wait complete. 2023-10-10T20:43:05.073 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:43:05.074 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:43:05.074 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-10-10T20:43:05.074 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:43:05.075 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:43:05.075 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:43:05.076 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:43:05.076 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:43:05.076 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:43:05.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:05.077 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-10-10T20:43:05.096 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:43:05.097 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:43:05.097 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:43:05.097 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:05.098 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-10-10T20:43:05.098 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:05.098 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:43:05.098 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:05.098 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-10-10T20:43:05.099 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:43:05.099 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:05.099 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-10-10T20:43:05.099 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:43:05.099 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:05.100 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:43:05.100 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:05.100 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:43:05.100 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 41997sec preferred_lft 41997sec 2023-10-10T20:43:05.100 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-10-10T20:43:05.101 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:05.101 INFO:teuthology.orchestra.run.smithi111.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:43:05.101 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:05.101 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:43:05.101 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:05.102 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::a486:a6ff:fe5f:6021/64 scope link 2023-10-10T20:43:05.102 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:05.102 INFO:teuthology.orchestra.run.smithi111.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:43:05.102 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 1e:d0:81:1b:e2:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:43:05.102 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::1cd0:81ff:fe1b:e26b/64 scope link 2023-10-10T20:43:05.103 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:05.103 INFO:teuthology.orchestra.run.smithi111.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:43:05.104 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:43:05.104 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::88d:cdff:fe75:455e/64 scope link 2023-10-10T20:43:05.104 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:05.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:05.106 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:43:05.165 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-10-10T20:43:05.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:05.170 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-10-10T20:43:05.226 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-10-10T20:43:05.226 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:43:05.226 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:43:05.226 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:43:05.227 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:43:05.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:05.227 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:43:05.228 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:43:05.228 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:43:05.228 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:43:05.229 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:43:05.229 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:43:05.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:05.332 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-10-10T20:43:05.347 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:43:05.347 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:43:05.348 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:43:05.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:05.348 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-10-10T20:43:05.411 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:43:05.412 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:43:05.467 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:43:05.467 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:43:05.468 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:43:05.521 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:43:05.522 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:43:05.522 DEBUG:teuthology.orchestra.run.smithi111:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:43:05.601 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:43:05.602 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:43:05.602 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:43:05.609 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:43:05.610 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:43:05.610 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:43:05.610 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:43:05.610 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:43:05.631 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:43:05.632 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:43:05.632 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:43:05.632 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:43:05.632 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:43:05.632 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:43:05.633 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:43:05.633 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:43:05.634 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:43:05.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:05.635 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:43:05.698 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-10T20:43:05.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.822+0000 7f5b97c27700 1 Processor -- start 2023-10-10T20:43:05.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.822+0000 7f5b97c27700 1 -- start start 2023-10-10T20:43:05.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.823+0000 7f5b97c27700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5b900cd910 0x7f5b900ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:05.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.823+0000 7f5b97c27700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5b900b55a0 0x7f5b900ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:05.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.823+0000 7f5b97c27700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b900b5b00 con 0x7f5b90159ec0 2023-10-10T20:43:05.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.823+0000 7f5b97c27700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b900b5da0 con 0x7f5b900cd910 2023-10-10T20:43:05.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.823+0000 7f5b97c27700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b900b6080 con 0x7f5b900b55a0 2023-10-10T20:43:05.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.823+0000 7f5b951c2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5b900cd910 0x7f5b900ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:05.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.823+0000 7f5b951c2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5b900cd910 0x7f5b900ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47580/0 (socket says 172.21.15.89:47580) 2023-10-10T20:43:05.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.823+0000 7f5b951c2700 1 -- 172.21.15.89:0/3854378313 learned_addr learned my addr 172.21.15.89:0/3854378313 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:05.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.823+0000 7f5b961c4700 1 --2- 172.21.15.89:0/3854378313 >> v2:172.21.15.111:3300/0 conn(0x7f5b900b55a0 0x7f5b900ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:05.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.823+0000 7f5b951c2700 1 -- 172.21.15.89:0/3854378313 >> v2:172.21.15.111:3300/0 conn(0x7f5b900b55a0 msgr2=0x7f5b900ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:05.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.823+0000 7f5b951c2700 1 --2- 172.21.15.89:0/3854378313 >> v2:172.21.15.111:3300/0 conn(0x7f5b900b55a0 0x7f5b900ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:05.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.823+0000 7f5b951c2700 1 -- 172.21.15.89:0/3854378313 >> v1:172.21.15.111:6789/0 conn(0x7f5b90159ec0 legacy=0x7f5b900b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:05.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.823+0000 7f5b951c2700 1 -- 172.21.15.89:0/3854378313 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b900cc100 con 0x7f5b900cd910 2023-10-10T20:43:05.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.823+0000 7f5b951c2700 1 --2- 172.21.15.89:0/3854378313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b900cd910 0x7f5b900ce340 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f5b800099b0 tx=0x7f5b80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30932b74d9c823f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:05.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.823+0000 7f5b949c1700 1 -- 172.21.15.89:0/3854378313 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80009ea0 con 0x7f5b900cd910 2023-10-10T20:43:05.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.824+0000 7f5b949c1700 1 -- 172.21.15.89:0/3854378313 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8000b820 con 0x7f5b900cd910 2023-10-10T20:43:05.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.824+0000 7f5b949c1700 1 -- 172.21.15.89:0/3854378313 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8000bbc0 con 0x7f5b900cd910 2023-10-10T20:43:05.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.824+0000 7f5b97c27700 1 -- 172.21.15.89:0/3854378313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b900cd910 msgr2=0x7f5b900ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:05.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.824+0000 7f5b97c27700 1 --2- 172.21.15.89:0/3854378313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b900cd910 0x7f5b900ce340 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f5b800099b0 tx=0x7f5b80009250 comp rx=0 tx=0).stop 2023-10-10T20:43:05.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.824+0000 7f5b97c27700 1 -- 172.21.15.89:0/3854378313 shutdown_connections 2023-10-10T20:43:05.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.824+0000 7f5b97c27700 1 --2- 172.21.15.89:0/3854378313 >> v2:172.21.15.111:3300/0 conn(0x7f5b900b55a0 0x7f5b900ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:05.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.824+0000 7f5b97c27700 1 --2- 172.21.15.89:0/3854378313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b900cd910 0x7f5b900ce340 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:05.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.824+0000 7f5b97c27700 1 -- 172.21.15.89:0/3854378313 >> 172.21.15.89:0/3854378313 conn(0x7f5b900999f0 msgr2=0x7f5b900b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:05.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.824+0000 7f5b97c27700 1 -- 172.21.15.89:0/3854378313 shutdown_connections 2023-10-10T20:43:05.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.824+0000 7f5b97c27700 1 -- 172.21.15.89:0/3854378313 wait complete. 2023-10-10T20:43:05.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.825+0000 7f5b97c27700 1 Processor -- start 2023-10-10T20:43:05.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.825+0000 7f5b97c27700 1 -- start start 2023-10-10T20:43:05.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.826+0000 7f5b97c27700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b900b55a0 0x7f5b901518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:05.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.826+0000 7f5b97c27700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b90159ec0 0x7f5b9014e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:05.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.826+0000 7f5b97c27700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b90152250 0x7f5b900c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:05.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.826+0000 7f5b97c27700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9014eb40 con 0x7f5b900b55a0 2023-10-10T20:43:05.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.826+0000 7f5b97c27700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b90152cb0 con 0x7f5b90159ec0 2023-10-10T20:43:05.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.826+0000 7f5b97c27700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b90152f50 con 0x7f5b90152250 2023-10-10T20:43:05.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.826+0000 7f5b959c3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b900b55a0 0x7f5b901518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:05.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.826+0000 7f5b959c3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b900b55a0 0x7f5b901518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47588/0 (socket says 172.21.15.89:47588) 2023-10-10T20:43:05.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.826+0000 7f5b959c3700 1 -- 172.21.15.89:0/1121591406 learned_addr learned my addr 172.21.15.89:0/1121591406 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:05.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.826+0000 7f5b951c2700 1 --2- 172.21.15.89:0/1121591406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b90159ec0 0x7f5b9014e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:05.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.826+0000 7f5b961c4700 1 --2- 172.21.15.89:0/1121591406 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b90152250 0x7f5b900c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:05.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.826+0000 7f5b959c3700 1 -- 172.21.15.89:0/1121591406 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b90152250 msgr2=0x7f5b900c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:05.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.826+0000 7f5b959c3700 1 --2- 172.21.15.89:0/1121591406 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b90152250 0x7f5b900c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:05.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.826+0000 7f5b959c3700 1 -- 172.21.15.89:0/1121591406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b90159ec0 msgr2=0x7f5b9014e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:05.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.826+0000 7f5b959c3700 1 --2- 172.21.15.89:0/1121591406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b90159ec0 0x7f5b9014e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:05.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.826+0000 7f5b959c3700 1 -- 172.21.15.89:0/1121591406 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b800094f0 con 0x7f5b900b55a0 2023-10-10T20:43:05.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.826+0000 7f5b959c3700 1 --2- 172.21.15.89:0/1121591406 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b900b55a0 0x7f5b901518c0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c001d00 tx=0x7f5b8c012870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:05.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.827+0000 7f5b85ffb700 1 -- 172.21.15.89:0/1121591406 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c017070 con 0x7f5b900b55a0 2023-10-10T20:43:05.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.827+0000 7f5b85ffb700 1 -- 172.21.15.89:0/1121591406 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8c014ab0 con 0x7f5b900b55a0 2023-10-10T20:43:05.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.827+0000 7f5b97c27700 1 -- 172.21.15.89:0/1121591406 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b90153230 con 0x7f5b900b55a0 2023-10-10T20:43:05.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.827+0000 7f5b85ffb700 1 -- 172.21.15.89:0/1121591406 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c014e90 con 0x7f5b900b55a0 2023-10-10T20:43:05.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.827+0000 7f5b97c27700 1 -- 172.21.15.89:0/1121591406 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b90155550 con 0x7f5b900b55a0 2023-10-10T20:43:05.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.828+0000 7f5b97c27700 1 -- 172.21.15.89:0/1121591406 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b900b4100 con 0x7f5b900b55a0 2023-10-10T20:43:05.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.828+0000 7f5b85ffb700 1 -- 172.21.15.89:0/1121591406 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5b8c0122e0 con 0x7f5b900b55a0 2023-10-10T20:43:05.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.828+0000 7f5b85ffb700 1 --2- 172.21.15.89:0/1121591406 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b74067320 0x7f5b740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:05.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.829+0000 7f5b85ffb700 1 -- 172.21.15.89:0/1121591406 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5b8c08cf70 con 0x7f5b900b55a0 2023-10-10T20:43:05.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.829+0000 7f5b951c2700 1 --2- 172.21.15.89:0/1121591406 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b74067320 0x7f5b740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:05.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.829+0000 7f5b951c2700 1 --2- 172.21.15.89:0/1121591406 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b74067320 0x7f5b740697d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5b800099b0 tx=0x7f5b80017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:05.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:05.833+0000 7f5b85ffb700 1 -- 172.21.15.89:0/1121591406 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b8c059af0 con 0x7f5b900b55a0 2023-10-10T20:43:06.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:06.062+0000 7f5b97c27700 1 -- 172.21.15.89:0/1121591406 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5b90063c90 con 0x7f5b900b55a0 2023-10-10T20:43:06.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:06.986+0000 7f5b85ffb700 1 -- 172.21.15.89:0/1121591406 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v111) v1 ==== 136+0+0 (secure 0 0 0) 0x7f5b8c05d110 con 0x7f5b900b55a0 2023-10-10T20:43:06.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:06.988+0000 7f5b97c27700 1 -- 172.21.15.89:0/1121591406 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b74067320 msgr2=0x7f5b740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:06.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:06.988+0000 7f5b97c27700 1 --2- 172.21.15.89:0/1121591406 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b74067320 0x7f5b740697d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5b800099b0 tx=0x7f5b80017000 comp rx=0 tx=0).stop 2023-10-10T20:43:06.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:06.988+0000 7f5b97c27700 1 -- 172.21.15.89:0/1121591406 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b900b55a0 msgr2=0x7f5b901518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:06.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:06.988+0000 7f5b97c27700 1 --2- 172.21.15.89:0/1121591406 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b900b55a0 0x7f5b901518c0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c001d00 tx=0x7f5b8c012870 comp rx=0 tx=0).stop 2023-10-10T20:43:06.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:06.989+0000 7f5b97c27700 1 -- 172.21.15.89:0/1121591406 shutdown_connections 2023-10-10T20:43:06.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:06.989+0000 7f5b97c27700 1 --2- 172.21.15.89:0/1121591406 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5b90152250 0x7f5b900c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:06.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:06.989+0000 7f5b97c27700 1 --2- 172.21.15.89:0/1121591406 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b900b55a0 0x7f5b901518c0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:06.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:06.989+0000 7f5b97c27700 1 --2- 172.21.15.89:0/1121591406 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b74067320 0x7f5b740697d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:06.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:06.989+0000 7f5b97c27700 1 --2- 172.21.15.89:0/1121591406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b90159ec0 0x7f5b9014e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:06.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:06.989+0000 7f5b97c27700 1 -- 172.21.15.89:0/1121591406 >> 172.21.15.89:0/1121591406 conn(0x7f5b900999f0 msgr2=0x7f5b901592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:06.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:06.989+0000 7f5b97c27700 1 -- 172.21.15.89:0/1121591406 shutdown_connections 2023-10-10T20:43:06.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:06.989+0000 7f5b97c27700 1 -- 172.21.15.89:0/1121591406 wait complete. 2023-10-10T20:43:07.008 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-10T20:43:07.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.121+0000 7f6895e20700 1 Processor -- start 2023-10-10T20:43:07.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.121+0000 7f6895e20700 1 -- start start 2023-10-10T20:43:07.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.122+0000 7f6895e20700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6890069560 0x7f68900699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:07.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.122+0000 7f6895e20700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6890069ee0 con 0x7f6890159c60 2023-10-10T20:43:07.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.122+0000 7f6895e20700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689015a160 con 0x7f6890159890 2023-10-10T20:43:07.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.122+0000 7f6895e20700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68900d13a0 con 0x7f6890069560 2023-10-10T20:43:07.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.122+0000 7f688effd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6890159c60 0x7f689015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38414/0 (socket says 172.21.15.89:38414) 2023-10-10T20:43:07.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.122+0000 7f688effd700 1 -- 172.21.15.89:0/3988933929 learned_addr learned my addr 172.21.15.89:0/3988933929 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:07.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.122+0000 7f688ffff700 1 --2- 172.21.15.89:0/3988933929 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6890069560 0x7f68900699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:07.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.122+0000 7f688e7fc700 1 -- 172.21.15.89:0/3988933929 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1823194907 0 0) 0x7f6890069ee0 con 0x7f6890159c60 2023-10-10T20:43:07.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.122+0000 7f688e7fc700 1 -- 172.21.15.89:0/3988933929 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6878003570 con 0x7f6890159c60 2023-10-10T20:43:07.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688e7fc700 1 -- 172.21.15.89:0/3988933929 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2669753089 0 0) 0x7f6878003570 con 0x7f6890159c60 2023-10-10T20:43:07.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688e7fc700 1 -- 172.21.15.89:0/3988933929 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6890069560 msgr2=0x7f68900699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:07.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688e7fc700 1 --2- 172.21.15.89:0/3988933929 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6890069560 0x7f68900699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688e7fc700 1 -- 172.21.15.89:0/3988933929 >> v1:172.21.15.111:6789/0 conn(0x7f6890159890 legacy=0x7f689015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:07.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688e7fc700 1 -- 172.21.15.89:0/3988933929 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689015e140 con 0x7f6890159c60 2023-10-10T20:43:07.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688e7fc700 1 -- 172.21.15.89:0/3988933929 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f6884001470 con 0x7f6890159c60 2023-10-10T20:43:07.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688e7fc700 1 -- 172.21.15.89:0/3988933929 >> v1:172.21.15.89:6789/0 conn(0x7f6890159c60 legacy=0x7f689015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:07.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688e7fc700 1 --2- 172.21.15.89:0/3988933929 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6890069560 0x7f6878003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:07.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688e7fc700 1 --2- 172.21.15.89:0/3988933929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6878003fd0 0x7f6878004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:07.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688e7fc700 1 --2- 172.21.15.89:0/3988933929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6878004d50 0x7f68780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:07.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688e7fc700 1 -- 172.21.15.89:0/3988933929 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68900d13a0 con 0x7f6878004d50 2023-10-10T20:43:07.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688e7fc700 1 -- 172.21.15.89:0/3988933929 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687800bb40 con 0x7f6878003fd0 2023-10-10T20:43:07.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688e7fc700 1 -- 172.21.15.89:0/3988933929 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687800bd00 con 0x7f6890069560 2023-10-10T20:43:07.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688ffff700 1 --2- 172.21.15.89:0/3988933929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6878004d50 0x7f68780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:07.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688f7fe700 1 --2- 172.21.15.89:0/3988933929 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6890069560 0x7f6878003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:07.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688effd700 1 --2- 172.21.15.89:0/3988933929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6878003fd0 0x7f6878004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:07.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688ffff700 1 -- 172.21.15.89:0/3988933929 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6890069560 msgr2=0x7f6878003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:07.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688ffff700 1 --2- 172.21.15.89:0/3988933929 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6890069560 0x7f6878003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688ffff700 1 -- 172.21.15.89:0/3988933929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6878003fd0 msgr2=0x7f6878004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:07.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688ffff700 1 --2- 172.21.15.89:0/3988933929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6878003fd0 0x7f6878004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688ffff700 1 -- 172.21.15.89:0/3988933929 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6878003570 con 0x7f6878004d50 2023-10-10T20:43:07.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688ffff700 1 --2- 172.21.15.89:0/3988933929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6878004d50 0x7f68780051c0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f6888009c00 tx=0x7f68880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3311521a2005c132 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:07.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688e7fc700 1 -- 172.21.15.89:0/3988933929 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688800e710 con 0x7f6878004d50 2023-10-10T20:43:07.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688e7fc700 1 -- 172.21.15.89:0/3988933929 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688800e870 con 0x7f6878004d50 2023-10-10T20:43:07.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.123+0000 7f688e7fc700 1 -- 172.21.15.89:0/3988933929 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688800ec30 con 0x7f6878004d50 2023-10-10T20:43:07.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.124+0000 7f6895e20700 1 -- 172.21.15.89:0/3988933929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6878004d50 msgr2=0x7f68780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:07.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.124+0000 7f6895e20700 1 --2- 172.21.15.89:0/3988933929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6878004d50 0x7f68780051c0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f6888009c00 tx=0x7f68880091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.124+0000 7f688ffff700 1 -- 172.21.15.89:0/3988933929 reap_dead start 2023-10-10T20:43:07.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.124+0000 7f6895e20700 1 -- 172.21.15.89:0/3988933929 shutdown_connections 2023-10-10T20:43:07.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.124+0000 7f6895e20700 1 -- 172.21.15.89:0/3988933929 >> 172.21.15.89:0/3988933929 conn(0x7f6890068e20 msgr2=0x7f68900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:07.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.124+0000 7f6895e20700 1 -- 172.21.15.89:0/3988933929 shutdown_connections 2023-10-10T20:43:07.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.124+0000 7f6895e20700 1 -- 172.21.15.89:0/3988933929 wait complete. 2023-10-10T20:43:07.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.125+0000 7f6895e20700 1 Processor -- start 2023-10-10T20:43:07.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.125+0000 7f6895e20700 1 -- start start 2023-10-10T20:43:07.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.126+0000 7f6895e20700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6890154480 0x7f68900cf720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:07.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.126+0000 7f6895e20700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f68900cfc60 0x7f68900b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:07.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.126+0000 7f6895e20700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68900b3ad0 0x7f68900cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:07.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.126+0000 7f6895e20700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68900b3f50 con 0x7f68900b3ad0 2023-10-10T20:43:07.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.126+0000 7f6895e20700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68900b5ac0 con 0x7f6890154480 2023-10-10T20:43:07.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.126+0000 7f6895e20700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68900b4eb0 con 0x7f68900cfc60 2023-10-10T20:43:07.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.126+0000 7f688ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68900b3ad0 0x7f68900cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:07.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.126+0000 7f688ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68900b3ad0 0x7f68900cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47612/0 (socket says 172.21.15.89:47612) 2023-10-10T20:43:07.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.126+0000 7f688ffff700 1 -- 172.21.15.89:0/649968059 learned_addr learned my addr 172.21.15.89:0/649968059 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:07.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.126+0000 7f688effd700 1 --2- 172.21.15.89:0/649968059 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f68900cfc60 0x7f68900b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:07.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.126+0000 7f688f7fe700 1 --2- 172.21.15.89:0/649968059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6890154480 0x7f68900cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:07.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.126+0000 7f688ffff700 1 -- 172.21.15.89:0/649968059 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f68900cfc60 msgr2=0x7f68900b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:07.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.126+0000 7f688ffff700 1 --2- 172.21.15.89:0/649968059 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f68900cfc60 0x7f68900b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.126+0000 7f688ffff700 1 -- 172.21.15.89:0/649968059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6890154480 msgr2=0x7f68900cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:07.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.126+0000 7f688ffff700 1 --2- 172.21.15.89:0/649968059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6890154480 0x7f68900cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.126+0000 7f688ffff700 1 -- 172.21.15.89:0/649968059 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68900b5190 con 0x7f68900b3ad0 2023-10-10T20:43:07.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.126+0000 7f688ffff700 1 --2- 172.21.15.89:0/649968059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68900b3ad0 0x7f68900cd0c0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f6888001eb0 tx=0x7f688801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:07.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.127+0000 7f68777fe700 1 -- 172.21.15.89:0/649968059 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888022070 con 0x7f68900b3ad0 2023-10-10T20:43:07.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.127+0000 7f68777fe700 1 -- 172.21.15.89:0/649968059 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688800ebf0 con 0x7f68900b3ad0 2023-10-10T20:43:07.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.127+0000 7f6895e20700 1 -- 172.21.15.89:0/649968059 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68900b5470 con 0x7f68900b3ad0 2023-10-10T20:43:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.127+0000 7f68777fe700 1 -- 172.21.15.89:0/649968059 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688800e710 con 0x7f68900b3ad0 2023-10-10T20:43:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.127+0000 7f6895e20700 1 -- 172.21.15.89:0/649968059 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68900b5f50 con 0x7f68900b3ad0 2023-10-10T20:43:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.128+0000 7f6895e20700 1 -- 172.21.15.89:0/649968059 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68900b5910 con 0x7f68900b3ad0 2023-10-10T20:43:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.128+0000 7f68777fe700 1 -- 172.21.15.89:0/649968059 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f688801c870 con 0x7f68900b3ad0 2023-10-10T20:43:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.128+0000 7f68777fe700 1 --2- 172.21.15.89:0/649968059 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f687c067320 0x7f687c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.128+0000 7f68777fe700 1 -- 172.21.15.89:0/649968059 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f68880863f0 con 0x7f68900b3ad0 2023-10-10T20:43:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.128+0000 7f688f7fe700 1 --2- 172.21.15.89:0/649968059 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f687c067320 0x7f687c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.129+0000 7f688f7fe700 1 --2- 172.21.15.89:0/649968059 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f687c067320 0x7f687c0697d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f68800017f0 tx=0x7f6880009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.133+0000 7f68777fe700 1 -- 172.21.15.89:0/649968059 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6888053080 con 0x7f68900b3ad0 2023-10-10T20:43:07.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.345+0000 7f6895e20700 1 -- 172.21.15.89:0/649968059 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f689004c3c0 con 0x7f687c067320 2023-10-10T20:43:07.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.346+0000 7f68777fe700 1 -- 172.21.15.89:0/649968059 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f689004c3c0 con 0x7f687c067320 2023-10-10T20:43:07.349 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:43:07.350 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:07.350 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs" 2023-10-10T20:43:07.350 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:07.350 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:43:07.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.348+0000 7f6895e20700 1 -- 172.21.15.89:0/649968059 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f687c067320 msgr2=0x7f687c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:07.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.348+0000 7f6895e20700 1 --2- 172.21.15.89:0/649968059 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f687c067320 0x7f687c0697d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f68800017f0 tx=0x7f6880009040 comp rx=0 tx=0).stop 2023-10-10T20:43:07.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.348+0000 7f6895e20700 1 -- 172.21.15.89:0/649968059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68900b3ad0 msgr2=0x7f68900cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:07.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.348+0000 7f6895e20700 1 --2- 172.21.15.89:0/649968059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68900b3ad0 0x7f68900cd0c0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f6888001eb0 tx=0x7f688801a000 comp rx=0 tx=0).stop 2023-10-10T20:43:07.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.348+0000 7f6895e20700 1 -- 172.21.15.89:0/649968059 shutdown_connections 2023-10-10T20:43:07.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.348+0000 7f6895e20700 1 --2- 172.21.15.89:0/649968059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68900b3ad0 0x7f68900cd0c0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.348+0000 7f6895e20700 1 --2- 172.21.15.89:0/649968059 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f687c067320 0x7f687c0697d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.348+0000 7f6895e20700 1 --2- 172.21.15.89:0/649968059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6890154480 0x7f68900cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.348+0000 7f6895e20700 1 --2- 172.21.15.89:0/649968059 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f68900cfc60 0x7f68900b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.348+0000 7f6895e20700 1 -- 172.21.15.89:0/649968059 >> 172.21.15.89:0/649968059 conn(0x7f6890068e20 msgr2=0x7f68901551b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.348+0000 7f6895e20700 1 -- 172.21.15.89:0/649968059 shutdown_connections 2023-10-10T20:43:07.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.348+0000 7f6895e20700 1 -- 172.21.15.89:0/649968059 wait complete. 2023-10-10T20:43:07.363 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:43:07.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.477+0000 7f8c95071700 1 Processor -- start 2023-10-10T20:43:07.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.477+0000 7f8c95071700 1 -- start start 2023-10-10T20:43:07.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.478+0000 7f8c95071700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8c90159890 0x7f8c9015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:07.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.478+0000 7f8c95071700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8c90159c60 0x7f8c9015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:07.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.478+0000 7f8c95071700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c9015a160 con 0x7f8c9015db10 2023-10-10T20:43:07.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.478+0000 7f8c95071700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9015df50 con 0x7f8c90159c60 2023-10-10T20:43:07.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.478+0000 7f8c95071700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c900d13a0 con 0x7f8c90159890 2023-10-10T20:43:07.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.478+0000 7f8c8659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8c90159c60 0x7f8c9015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:07.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.478+0000 7f8c8659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8c90159c60 0x7f8c9015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47624/0 (socket says 172.21.15.89:47624) 2023-10-10T20:43:07.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.478+0000 7f8c8659c700 1 -- 172.21.15.89:0/1052974721 learned_addr learned my addr 172.21.15.89:0/1052974721 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:07.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.478+0000 7f8c8ed9d700 1 --2- 172.21.15.89:0/1052974721 >> v2:172.21.15.111:3300/0 conn(0x7f8c90159890 0x7f8c9015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:07.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.479+0000 7f8c8659c700 1 -- 172.21.15.89:0/1052974721 >> v2:172.21.15.111:3300/0 conn(0x7f8c90159890 msgr2=0x7f8c9015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:07.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.479+0000 7f8c8659c700 1 --2- 172.21.15.89:0/1052974721 >> v2:172.21.15.111:3300/0 conn(0x7f8c90159890 0x7f8c9015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.479+0000 7f8c8659c700 1 -- 172.21.15.89:0/1052974721 >> v1:172.21.15.111:6789/0 conn(0x7f8c9015db10 legacy=0x7f8c90069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:07.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.479+0000 7f8c8659c700 1 -- 172.21.15.89:0/1052974721 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9015e140 con 0x7f8c90159c60 2023-10-10T20:43:07.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.479+0000 7f8c8659c700 1 --2- 172.21.15.89:0/1052974721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c90159c60 0x7f8c9015d5d0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f8c900d1090 tx=0x7f8c78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c68ce4b17dd0597 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:07.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.479+0000 7f8c8e59c700 1 -- 172.21.15.89:0/1052974721 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78009e70 con 0x7f8c90159c60 2023-10-10T20:43:07.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.479+0000 7f8c8e59c700 1 -- 172.21.15.89:0/1052974721 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7800b860 con 0x7f8c90159c60 2023-10-10T20:43:07.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.479+0000 7f8c8e59c700 1 -- 172.21.15.89:0/1052974721 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800bc00 con 0x7f8c90159c60 2023-10-10T20:43:07.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.479+0000 7f8c95071700 1 -- 172.21.15.89:0/1052974721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c90159c60 msgr2=0x7f8c9015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:07.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.479+0000 7f8c95071700 1 --2- 172.21.15.89:0/1052974721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c90159c60 0x7f8c9015d5d0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f8c900d1090 tx=0x7f8c78009250 comp rx=0 tx=0).stop 2023-10-10T20:43:07.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.479+0000 7f8c95071700 1 -- 172.21.15.89:0/1052974721 shutdown_connections 2023-10-10T20:43:07.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.479+0000 7f8c95071700 1 --2- 172.21.15.89:0/1052974721 >> v2:172.21.15.111:3300/0 conn(0x7f8c90159890 0x7f8c9015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.479+0000 7f8c95071700 1 --2- 172.21.15.89:0/1052974721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c90159c60 0x7f8c9015d5d0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.479+0000 7f8c95071700 1 -- 172.21.15.89:0/1052974721 >> 172.21.15.89:0/1052974721 conn(0x7f8c90068e20 msgr2=0x7f8c900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:07.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.479+0000 7f8c95071700 1 -- 172.21.15.89:0/1052974721 shutdown_connections 2023-10-10T20:43:07.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.479+0000 7f8c95071700 1 -- 172.21.15.89:0/1052974721 wait complete. 2023-10-10T20:43:07.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.481+0000 7f8c95071700 1 Processor -- start 2023-10-10T20:43:07.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.481+0000 7f8c95071700 1 -- start start 2023-10-10T20:43:07.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.481+0000 7f8c95071700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c90159890 0x7f8c900b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:07.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.481+0000 7f8c95071700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c90159c60 0x7f8c90147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:07.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.481+0000 7f8c95071700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c9015db10 0x7f8c900cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:07.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.481+0000 7f8c95071700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9016ba10 con 0x7f8c9015db10 2023-10-10T20:43:07.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.481+0000 7f8c95071700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c901478c0 con 0x7f8c90159c60 2023-10-10T20:43:07.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.481+0000 7f8c95071700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9016ae00 con 0x7f8c90159890 2023-10-10T20:43:07.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.481+0000 7f8c8f59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c9015db10 0x7f8c900cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:07.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.481+0000 7f8c8f59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c9015db10 0x7f8c900cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47634/0 (socket says 172.21.15.89:47634) 2023-10-10T20:43:07.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.481+0000 7f8c8f59e700 1 -- 172.21.15.89:0/3797560688 learned_addr learned my addr 172.21.15.89:0/3797560688 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:07.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.481+0000 7f8c8659c700 1 --2- 172.21.15.89:0/3797560688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c90159c60 0x7f8c90147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:07.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.482+0000 7f8c8ed9d700 1 --2- 172.21.15.89:0/3797560688 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c90159890 0x7f8c900b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:07.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.482+0000 7f8c8f59e700 1 -- 172.21.15.89:0/3797560688 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c90159890 msgr2=0x7f8c900b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:07.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.482+0000 7f8c8f59e700 1 --2- 172.21.15.89:0/3797560688 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c90159890 0x7f8c900b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.482+0000 7f8c8f59e700 1 -- 172.21.15.89:0/3797560688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c90159c60 msgr2=0x7f8c90147380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:07.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.482+0000 7f8c8f59e700 1 --2- 172.21.15.89:0/3797560688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c90159c60 0x7f8c90147380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.482+0000 7f8c8f59e700 1 -- 172.21.15.89:0/3797560688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c780094f0 con 0x7f8c9015db10 2023-10-10T20:43:07.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.482+0000 7f8c8f59e700 1 --2- 172.21.15.89:0/3797560688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c9015db10 0x7f8c900cf070 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800c840 tx=0x7f8c88001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:07.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.482+0000 7f8c877fe700 1 -- 172.21.15.89:0/3797560688 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c88012630 con 0x7f8c9015db10 2023-10-10T20:43:07.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.482+0000 7f8c877fe700 1 -- 172.21.15.89:0/3797560688 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c880127b0 con 0x7f8c9015db10 2023-10-10T20:43:07.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.482+0000 7f8c95071700 1 -- 172.21.15.89:0/3797560688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9016b0e0 con 0x7f8c9015db10 2023-10-10T20:43:07.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.482+0000 7f8c877fe700 1 -- 172.21.15.89:0/3797560688 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c88012b70 con 0x7f8c9015db10 2023-10-10T20:43:07.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.482+0000 7f8c95071700 1 -- 172.21.15.89:0/3797560688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9016b6a0 con 0x7f8c9015db10 2023-10-10T20:43:07.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.483+0000 7f8c95071700 1 -- 172.21.15.89:0/3797560688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9004c3c0 con 0x7f8c9015db10 2023-10-10T20:43:07.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.483+0000 7f8c877fe700 1 -- 172.21.15.89:0/3797560688 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8c8801a3a0 con 0x7f8c9015db10 2023-10-10T20:43:07.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.484+0000 7f8c877fe700 1 --2- 172.21.15.89:0/3797560688 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c70067320 0x7f8c700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:07.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.484+0000 7f8c8ed9d700 1 --2- 172.21.15.89:0/3797560688 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c70067320 0x7f8c700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:07.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.484+0000 7f8c877fe700 1 -- 172.21.15.89:0/3797560688 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8c88084da0 con 0x7f8c9015db10 2023-10-10T20:43:07.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.484+0000 7f8c8ed9d700 1 --2- 172.21.15.89:0/3797560688 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c70067320 0x7f8c700697d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009db0 tx=0x7f8c800091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:07.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.489+0000 7f8c877fe700 1 -- 172.21.15.89:0/3797560688 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c88051a30 con 0x7f8c9015db10 2023-10-10T20:43:07.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.671+0000 7f8c95071700 1 -- 172.21.15.89:0/3797560688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8c9006a5b0 con 0x7f8c9015db10 2023-10-10T20:43:07.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.671+0000 7f8c877fe700 1 -- 172.21.15.89:0/3797560688 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8c88055050 con 0x7f8c9015db10 2023-10-10T20:43:07.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.674+0000 7f8c95071700 1 -- 172.21.15.89:0/3797560688 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c70067320 msgr2=0x7f8c700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:07.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.674+0000 7f8c95071700 1 --2- 172.21.15.89:0/3797560688 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c70067320 0x7f8c700697d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009db0 tx=0x7f8c800091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.674+0000 7f8c95071700 1 -- 172.21.15.89:0/3797560688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c9015db10 msgr2=0x7f8c900cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:07.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.674+0000 7f8c95071700 1 --2- 172.21.15.89:0/3797560688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c9015db10 0x7f8c900cf070 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800c840 tx=0x7f8c88001670 comp rx=0 tx=0).stop 2023-10-10T20:43:07.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.674+0000 7f8c95071700 1 -- 172.21.15.89:0/3797560688 shutdown_connections 2023-10-10T20:43:07.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.674+0000 7f8c95071700 1 --2- 172.21.15.89:0/3797560688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c9015db10 0x7f8c900cf070 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.674+0000 7f8c95071700 1 --2- 172.21.15.89:0/3797560688 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8c90159890 0x7f8c900b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.674+0000 7f8c95071700 1 --2- 172.21.15.89:0/3797560688 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8c70067320 0x7f8c700697d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.674+0000 7f8c95071700 1 --2- 172.21.15.89:0/3797560688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c90159c60 0x7f8c90147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:07.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.674+0000 7f8c95071700 1 -- 172.21.15.89:0/3797560688 >> 172.21.15.89:0/3797560688 conn(0x7f8c90068e20 msgr2=0x7f8c90069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:07.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.674+0000 7f8c95071700 1 -- 172.21.15.89:0/3797560688 shutdown_connections 2023-10-10T20:43:07.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:07.674+0000 7f8c95071700 1 -- 172.21.15.89:0/3797560688 wait complete. 2023-10-10T20:43:07.688 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:43:07.688 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:43:07.689 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:43:07.723 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:43:07.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:07.724 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:43:07.778 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:43:07.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:43:07.779 DEBUG:teuthology.orchestra.run.smithi111:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:43:07.824 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:43:07.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:07.825 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:43:07.845 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:43:07.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:07.846 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-10T20:43:07.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:43:07.862 INFO:teuthology.orchestra.run.smithi089.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-10T20:43:07.862 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:43:07.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:07.863 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-10T20:43:07.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:43:07.903 INFO:teuthology.orchestra.run.smithi111.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-10T20:43:07.903 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create another-data-pool --pg_num_min 64 2023-10-10T20:43:08.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.021+0000 7fccdb07c700 1 Processor -- start 2023-10-10T20:43:08.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.021+0000 7fccdb07c700 1 -- start start 2023-10-10T20:43:08.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.022+0000 7fccdb07c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fccd40b0790 0x7fccd40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:08.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.022+0000 7fccdb07c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd40cfa00 con 0x7fccd40b03c0 2023-10-10T20:43:08.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.022+0000 7fccdb07c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd40cb580 con 0x7fccd41591e0 2023-10-10T20:43:08.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.022+0000 7fccdb07c700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd40cf160 con 0x7fccd40b0790 2023-10-10T20:43:08.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.022+0000 7fccda07a700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fccd40b03c0 0x7fccd40b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38430/0 (socket says 172.21.15.89:38430) 2023-10-10T20:43:08.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.022+0000 7fccda07a700 1 -- 172.21.15.89:0/1484193330 learned_addr learned my addr 172.21.15.89:0/1484193330 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:08.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.022+0000 7fccd9078700 1 -- 172.21.15.89:0/1484193330 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2038155249 0 0) 0x7fccd40cfa00 con 0x7fccd40b03c0 2023-10-10T20:43:08.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.022+0000 7fccd9879700 1 --2- 172.21.15.89:0/1484193330 >> v2:172.21.15.111:3300/0 conn(0x7fccd40b0790 0x7fccd40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:08.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.022+0000 7fccd9078700 1 -- 172.21.15.89:0/1484193330 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccc0003570 con 0x7fccd40b03c0 2023-10-10T20:43:08.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.022+0000 7fccd9078700 1 -- 172.21.15.89:0/1484193330 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1121384295 0 0) 0x7fccc0003570 con 0x7fccd40b03c0 2023-10-10T20:43:08.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.022+0000 7fccd9078700 1 -- 172.21.15.89:0/1484193330 >> v2:172.21.15.111:3300/0 conn(0x7fccd40b0790 msgr2=0x7fccd40cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:08.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.022+0000 7fccd9078700 1 --2- 172.21.15.89:0/1484193330 >> v2:172.21.15.111:3300/0 conn(0x7fccd40b0790 0x7fccd40cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:08.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.022+0000 7fccd9078700 1 -- 172.21.15.89:0/1484193330 >> v1:172.21.15.111:6789/0 conn(0x7fccd41591e0 legacy=0x7fccd4159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:08.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.022+0000 7fccd9078700 1 -- 172.21.15.89:0/1484193330 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd40cf440 con 0x7fccd40b03c0 2023-10-10T20:43:08.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.022+0000 7fccd9078700 1 -- 172.21.15.89:0/1484193330 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fccd00017f0 con 0x7fccd40b03c0 2023-10-10T20:43:08.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.022+0000 7fccd9078700 1 -- 172.21.15.89:0/1484193330 >> v1:172.21.15.89:6789/0 conn(0x7fccd40b03c0 legacy=0x7fccd40b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:08.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.023+0000 7fccd9078700 1 --2- 172.21.15.89:0/1484193330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccc0003a90 0x7fccc0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:08.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.023+0000 7fccd9078700 1 --2- 172.21.15.89:0/1484193330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccc0004810 0x7fccc0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:08.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.023+0000 7fccd9078700 1 --2- 172.21.15.89:0/1484193330 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccc00052e0 0x7fccc000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:08.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.023+0000 7fccd9078700 1 -- 172.21.15.89:0/1484193330 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd40cf160 con 0x7fccc0004810 2023-10-10T20:43:08.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.023+0000 7fccd9078700 1 -- 172.21.15.89:0/1484193330 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc000c230 con 0x7fccc0003a90 2023-10-10T20:43:08.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.023+0000 7fccd9078700 1 -- 172.21.15.89:0/1484193330 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc000c3f0 con 0x7fccc00052e0 2023-10-10T20:43:08.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.023+0000 7fccd9879700 1 --2- 172.21.15.89:0/1484193330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccc0004810 0x7fccc0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:08.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.023+0000 7fccda07a700 1 --2- 172.21.15.89:0/1484193330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccc0003a90 0x7fccc0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:08.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.023+0000 7fccda87b700 1 --2- 172.21.15.89:0/1484193330 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccc00052e0 0x7fccc000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:08.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.023+0000 7fccd9879700 1 -- 172.21.15.89:0/1484193330 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccc00052e0 msgr2=0x7fccc000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:08.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.023+0000 7fccd9879700 1 --2- 172.21.15.89:0/1484193330 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccc00052e0 0x7fccc000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:08.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.023+0000 7fccd9879700 1 -- 172.21.15.89:0/1484193330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccc0003a90 msgr2=0x7fccc0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:08.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.023+0000 7fccd9879700 1 --2- 172.21.15.89:0/1484193330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccc0003a90 0x7fccc0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:08.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.023+0000 7fccda87b700 1 -- 172.21.15.89:0/1484193330 reap_dead start 2023-10-10T20:43:08.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.023+0000 7fccd9879700 1 -- 172.21.15.89:0/1484193330 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc0003570 con 0x7fccc0004810 2023-10-10T20:43:08.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.023+0000 7fccd9879700 1 --2- 172.21.15.89:0/1484193330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccc0004810 0x7fccc0004c80 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009c00 tx=0x7fccc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc522eb27de6ef7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:08.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.023+0000 7fccd9078700 1 -- 172.21.15.89:0/1484193330 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc400e710 con 0x7fccc0004810 2023-10-10T20:43:08.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.023+0000 7fccd9078700 1 -- 172.21.15.89:0/1484193330 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc400e870 con 0x7fccc0004810 2023-10-10T20:43:08.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.023+0000 7fccd9078700 1 -- 172.21.15.89:0/1484193330 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc400ec30 con 0x7fccc0004810 2023-10-10T20:43:08.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.024+0000 7fccdb07c700 1 -- 172.21.15.89:0/1484193330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccc0004810 msgr2=0x7fccc0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:08.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.024+0000 7fccdb07c700 1 --2- 172.21.15.89:0/1484193330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccc0004810 0x7fccc0004c80 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009c00 tx=0x7fccc40091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:08.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.024+0000 7fccdb07c700 1 -- 172.21.15.89:0/1484193330 shutdown_connections 2023-10-10T20:43:08.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.024+0000 7fccdb07c700 1 --2- 172.21.15.89:0/1484193330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccc0004810 0x7fccc0004c80 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:08.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.024+0000 7fccdb07c700 1 -- 172.21.15.89:0/1484193330 >> 172.21.15.89:0/1484193330 conn(0x7fccd4063230 msgr2=0x7fccd4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:08.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.024+0000 7fccdb07c700 1 -- 172.21.15.89:0/1484193330 shutdown_connections 2023-10-10T20:43:08.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.024+0000 7fccdb07c700 1 -- 172.21.15.89:0/1484193330 wait complete. 2023-10-10T20:43:08.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.025+0000 7fccdb07c700 1 Processor -- start 2023-10-10T20:43:08.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.025+0000 7fccdb07c700 1 -- start start 2023-10-10T20:43:08.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccdb07c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccc0004810 0x7fccd415b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:08.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccdb07c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd4157a30 0x7fccd40c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:08.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccdb07c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccd4157ea0 0x7fccd415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:08.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccdb07c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd40af9c0 con 0x7fccd4157a30 2023-10-10T20:43:08.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccdb07c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd4159f90 con 0x7fccd4157ea0 2023-10-10T20:43:08.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccdb07c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd415a200 con 0x7fccc0004810 2023-10-10T20:43:08.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccd9879700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd4157a30 0x7fccd40c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:08.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccd9879700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd4157a30 0x7fccd40c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47646/0 (socket says 172.21.15.89:47646) 2023-10-10T20:43:08.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccd9879700 1 -- 172.21.15.89:0/1724770031 learned_addr learned my addr 172.21.15.89:0/1724770031 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:08.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccda07a700 1 --2- 172.21.15.89:0/1724770031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccc0004810 0x7fccd415b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:08.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccda87b700 1 --2- 172.21.15.89:0/1724770031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccd4157ea0 0x7fccd415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:08.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccd9879700 1 -- 172.21.15.89:0/1724770031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccc0004810 msgr2=0x7fccd415b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:08.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccd9879700 1 --2- 172.21.15.89:0/1724770031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccc0004810 0x7fccd415b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:08.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccd9879700 1 -- 172.21.15.89:0/1724770031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccd4157ea0 msgr2=0x7fccd415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:08.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccd9879700 1 --2- 172.21.15.89:0/1724770031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccd4157ea0 0x7fccd415d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:08.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccd9879700 1 -- 172.21.15.89:0/1724770031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd415a4e0 con 0x7fccd4157a30 2023-10-10T20:43:08.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccd9879700 1 --2- 172.21.15.89:0/1724770031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd4157a30 0x7fccd40c5f30 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fccc000c200 tx=0x7fccc401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:08.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccca7fc700 1 -- 172.21.15.89:0/1724770031 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4022070 con 0x7fccd4157a30 2023-10-10T20:43:08.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccca7fc700 1 -- 172.21.15.89:0/1724770031 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc400e9e0 con 0x7fccd4157a30 2023-10-10T20:43:08.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccdb07c700 1 -- 172.21.15.89:0/1724770031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd415a7c0 con 0x7fccd4157a30 2023-10-10T20:43:08.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccca7fc700 1 -- 172.21.15.89:0/1724770031 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc400e710 con 0x7fccd4157a30 2023-10-10T20:43:08.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.026+0000 7fccdb07c700 1 -- 172.21.15.89:0/1724770031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd415ad80 con 0x7fccd4157a30 2023-10-10T20:43:08.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.027+0000 7fccdb07c700 1 -- 172.21.15.89:0/1724770031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd404c3c0 con 0x7fccd4157a30 2023-10-10T20:43:08.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.028+0000 7fccca7fc700 1 -- 172.21.15.89:0/1724770031 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fccc401c660 con 0x7fccd4157a30 2023-10-10T20:43:08.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.028+0000 7fccca7fc700 1 --2- 172.21.15.89:0/1724770031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fccb8067320 0x7fccb80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:08.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.028+0000 7fccca7fc700 1 -- 172.21.15.89:0/1724770031 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fccc4086270 con 0x7fccd4157a30 2023-10-10T20:43:08.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.028+0000 7fccda07a700 1 --2- 172.21.15.89:0/1724770031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fccb8067320 0x7fccb80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:08.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.029+0000 7fccda07a700 1 --2- 172.21.15.89:0/1724770031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fccb8067320 0x7fccb80697d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fccd40c5910 tx=0x7fccd0006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:08.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.032+0000 7fccca7fc700 1 -- 172.21.15.89:0/1724770031 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fccc4052fb0 con 0x7fccd4157a30 2023-10-10T20:43:08.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:08.216+0000 7fccdb07c700 1 -- 172.21.15.89:0/1724770031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "another-data-pool", "pg_num_min": 64} v 0) v1 -- 0x7fccd40cf720 con 0x7fccd4157a30 2023-10-10T20:43:09.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.001+0000 7fccca7fc700 1 -- 172.21.15.89:0/1724770031 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "another-data-pool", "pg_num_min": 64}]=0 pool 'another-data-pool' created v166) v1 ==== 142+0+0 (secure 0 0 0) 0x7fccc40565d0 con 0x7fccd4157a30 2023-10-10T20:43:09.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.006+0000 7fccdb07c700 1 -- 172.21.15.89:0/1724770031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fccb8067320 msgr2=0x7fccb80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:09.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.006+0000 7fccdb07c700 1 --2- 172.21.15.89:0/1724770031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fccb8067320 0x7fccb80697d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fccd40c5910 tx=0x7fccd0006040 comp rx=0 tx=0).stop 2023-10-10T20:43:09.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.006+0000 7fccdb07c700 1 -- 172.21.15.89:0/1724770031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd4157a30 msgr2=0x7fccd40c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:09.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.006+0000 7fccdb07c700 1 --2- 172.21.15.89:0/1724770031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd4157a30 0x7fccd40c5f30 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fccc000c200 tx=0x7fccc401a000 comp rx=0 tx=0).stop 2023-10-10T20:43:09.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.007+0000 7fccdb07c700 1 -- 172.21.15.89:0/1724770031 shutdown_connections 2023-10-10T20:43:09.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.007+0000 7fccdb07c700 1 --2- 172.21.15.89:0/1724770031 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fccb8067320 0x7fccb80697d0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.007+0000 7fccdb07c700 1 --2- 172.21.15.89:0/1724770031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccd4157ea0 0x7fccd415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.007+0000 7fccdb07c700 1 --2- 172.21.15.89:0/1724770031 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccc0004810 0x7fccd415b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.007+0000 7fccdb07c700 1 --2- 172.21.15.89:0/1724770031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd4157a30 0x7fccd40c5f30 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.007+0000 7fccdb07c700 1 -- 172.21.15.89:0/1724770031 >> 172.21.15.89:0/1724770031 conn(0x7fccd4063230 msgr2=0x7fccd40b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.007+0000 7fccdb07c700 1 -- 172.21.15.89:0/1724770031 shutdown_connections 2023-10-10T20:43:09.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.008+0000 7fccdb07c700 1 -- 172.21.15.89:0/1724770031 wait complete. 2023-10-10T20:43:09.011 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'another-data-pool' created 2023-10-10T20:43:09.023 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs another-data-pool 2023-10-10T20:43:09.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.138+0000 7f9be9463700 1 Processor -- start 2023-10-10T20:43:09.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.138+0000 7f9be9463700 1 -- start start 2023-10-10T20:43:09.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.139+0000 7f9be9463700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9be4159890 0x7f9be415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:09.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.139+0000 7f9be9463700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9be4069560 0x7f9be40699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:09.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.139+0000 7f9be9463700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be415a160 con 0x7f9be4159c60 2023-10-10T20:43:09.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.139+0000 7f9be9463700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be4069f20 con 0x7f9be4069560 2023-10-10T20:43:09.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.139+0000 7f9be9463700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40d13a0 con 0x7f9be4159890 2023-10-10T20:43:09.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.139+0000 7f9bda7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9be4159c60 0x7f9be415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37660/0 (socket says 172.21.15.89:37660) 2023-10-10T20:43:09.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.139+0000 7f9bda7fc700 1 -- 172.21.15.89:0/2506008676 learned_addr learned my addr 172.21.15.89:0/2506008676 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.139+0000 7f9be2ffd700 1 --2- 172.21.15.89:0/2506008676 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9be4159890 0x7f9be415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.139+0000 7f9be37fe700 1 --2- 172.21.15.89:0/2506008676 >> v2:172.21.15.111:3300/0 conn(0x7f9be4069560 0x7f9be40699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.139+0000 7f9be27fc700 1 -- 172.21.15.89:0/2506008676 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1286151067 0 0) 0x7f9be415a160 con 0x7f9be4159c60 2023-10-10T20:43:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.139+0000 7f9be27fc700 1 -- 172.21.15.89:0/2506008676 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bd0003570 con 0x7f9be4159c60 2023-10-10T20:43:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be27fc700 1 -- 172.21.15.89:0/2506008676 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1681793559 0 0) 0x7f9bd0003570 con 0x7f9be4159c60 2023-10-10T20:43:09.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be27fc700 1 -- 172.21.15.89:0/2506008676 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9be4159890 msgr2=0x7f9be415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:09.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be27fc700 1 --2- 172.21.15.89:0/2506008676 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9be4159890 0x7f9be415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:09.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be27fc700 1 -- 172.21.15.89:0/2506008676 >> v2:172.21.15.111:3300/0 conn(0x7f9be4069560 msgr2=0x7f9be40699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:09.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be27fc700 1 --2- 172.21.15.89:0/2506008676 >> v2:172.21.15.111:3300/0 conn(0x7f9be4069560 0x7f9be40699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:09.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be27fc700 1 -- 172.21.15.89:0/2506008676 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be415e140 con 0x7f9be4159c60 2023-10-10T20:43:09.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be27fc700 1 -- 172.21.15.89:0/2506008676 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f9bcc001470 con 0x7f9be4159c60 2023-10-10T20:43:09.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be27fc700 1 -- 172.21.15.89:0/2506008676 >> v1:172.21.15.89:6789/0 conn(0x7f9be4159c60 legacy=0x7f9be415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:09.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be27fc700 1 --2- 172.21.15.89:0/2506008676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd0003a60 0x7f9bd0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:09.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be27fc700 1 --2- 172.21.15.89:0/2506008676 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9be4159890 0x7f9bd0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:09.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be27fc700 1 --2- 172.21.15.89:0/2506008676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bd0004d50 0x7f9bd00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:09.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be27fc700 1 -- 172.21.15.89:0/2506008676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be4069f20 con 0x7f9bd0003a60 2023-10-10T20:43:09.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be27fc700 1 -- 172.21.15.89:0/2506008676 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40d13a0 con 0x7f9bd0004d50 2023-10-10T20:43:09.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be27fc700 1 -- 172.21.15.89:0/2506008676 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd000ba60 con 0x7f9be4159890 2023-10-10T20:43:09.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be2ffd700 1 --2- 172.21.15.89:0/2506008676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd0003a60 0x7f9bd0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:09.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be37fe700 1 --2- 172.21.15.89:0/2506008676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bd0004d50 0x7f9bd00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:09.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9bda7fc700 1 --2- 172.21.15.89:0/2506008676 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9be4159890 0x7f9bd0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:09.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be2ffd700 1 -- 172.21.15.89:0/2506008676 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9be4159890 msgr2=0x7f9bd0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:09.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be2ffd700 1 --2- 172.21.15.89:0/2506008676 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9be4159890 0x7f9bd0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:09.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be2ffd700 1 -- 172.21.15.89:0/2506008676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bd0004d50 msgr2=0x7f9bd00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:09.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be2ffd700 1 --2- 172.21.15.89:0/2506008676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bd0004d50 0x7f9bd00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:09.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be2ffd700 1 -- 172.21.15.89:0/2506008676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd0003570 con 0x7f9bd0003a60 2023-10-10T20:43:09.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9bda7fc700 1 --2- 172.21.15.89:0/2506008676 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9be4159890 0x7f9bd0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:09.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be2ffd700 1 --2- 172.21.15.89:0/2506008676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd0003a60 0x7f9bd0003e70 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009c00 tx=0x7f9bd40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e1154762f9126 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:09.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be27fc700 1 -- 172.21.15.89:0/2506008676 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd400e6b0 con 0x7f9bd0003a60 2023-10-10T20:43:09.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be27fc700 1 -- 172.21.15.89:0/2506008676 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd400e810 con 0x7f9bd0003a60 2023-10-10T20:43:09.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.140+0000 7f9be27fc700 1 -- 172.21.15.89:0/2506008676 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd400ebd0 con 0x7f9bd0003a60 2023-10-10T20:43:09.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.141+0000 7f9be9463700 1 -- 172.21.15.89:0/2506008676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd0003a60 msgr2=0x7f9bd0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:09.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.141+0000 7f9be9463700 1 --2- 172.21.15.89:0/2506008676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd0003a60 0x7f9bd0003e70 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009c00 tx=0x7f9bd40091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:09.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.141+0000 7f9be37fe700 1 -- 172.21.15.89:0/2506008676 reap_dead start 2023-10-10T20:43:09.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.141+0000 7f9be9463700 1 -- 172.21.15.89:0/2506008676 shutdown_connections 2023-10-10T20:43:09.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.141+0000 7f9be9463700 1 -- 172.21.15.89:0/2506008676 >> 172.21.15.89:0/2506008676 conn(0x7f9be4068e20 msgr2=0x7f9be40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:09.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.141+0000 7f9be9463700 1 -- 172.21.15.89:0/2506008676 shutdown_connections 2023-10-10T20:43:09.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.141+0000 7f9be9463700 1 -- 172.21.15.89:0/2506008676 wait complete. 2023-10-10T20:43:09.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.142+0000 7f9be9463700 1 Processor -- start 2023-10-10T20:43:09.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.142+0000 7f9be9463700 1 -- start start 2023-10-10T20:43:09.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.143+0000 7f9be9463700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bd0003a60 0x7f9be40ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:09.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.143+0000 7f9be9463700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9be414cc50 0x7f9be40b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:09.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.143+0000 7f9be9463700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9be41512b0 0x7f9be40b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:09.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.143+0000 7f9be9463700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be4152620 con 0x7f9be414cc50 2023-10-10T20:43:09.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.143+0000 7f9be9463700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40cfb20 con 0x7f9be41512b0 2023-10-10T20:43:09.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.143+0000 7f9be9463700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be4152bb0 con 0x7f9bd0003a60 2023-10-10T20:43:09.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.143+0000 7f9bda7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9be414cc50 0x7f9be40b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:09.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.143+0000 7f9bda7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9be414cc50 0x7f9be40b15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35012/0 (socket says 172.21.15.89:35012) 2023-10-10T20:43:09.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.143+0000 7f9bda7fc700 1 -- 172.21.15.89:0/3213656484 learned_addr learned my addr 172.21.15.89:0/3213656484 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:09.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.143+0000 7f9be2ffd700 1 --2- 172.21.15.89:0/3213656484 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bd0003a60 0x7f9be40ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:09.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.143+0000 7f9bda7fc700 1 -- 172.21.15.89:0/3213656484 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bd0003a60 msgr2=0x7f9be40ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:09.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.143+0000 7f9be37fe700 1 --2- 172.21.15.89:0/3213656484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9be41512b0 0x7f9be40b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:09.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.143+0000 7f9bda7fc700 1 --2- 172.21.15.89:0/3213656484 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bd0003a60 0x7f9be40ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:09.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.143+0000 7f9bda7fc700 1 -- 172.21.15.89:0/3213656484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9be41512b0 msgr2=0x7f9be40b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:09.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.143+0000 7f9bda7fc700 1 --2- 172.21.15.89:0/3213656484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9be41512b0 0x7f9be40b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:09.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.143+0000 7f9bda7fc700 1 -- 172.21.15.89:0/3213656484 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be40cef10 con 0x7f9be414cc50 2023-10-10T20:43:09.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.144+0000 7f9bda7fc700 1 --2- 172.21.15.89:0/3213656484 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9be414cc50 0x7f9be40b15f0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00eb90 tx=0x7f9bcc001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:09.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.144+0000 7f9bdb7fe700 1 -- 172.21.15.89:0/3213656484 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc00c700 con 0x7f9be414cc50 2023-10-10T20:43:09.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.144+0000 7f9bdb7fe700 1 -- 172.21.15.89:0/3213656484 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bcc00c860 con 0x7f9be414cc50 2023-10-10T20:43:09.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.144+0000 7f9be9463700 1 -- 172.21.15.89:0/3213656484 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be40cf1f0 con 0x7f9be414cc50 2023-10-10T20:43:09.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.144+0000 7f9bdb7fe700 1 -- 172.21.15.89:0/3213656484 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc00cbe0 con 0x7f9be414cc50 2023-10-10T20:43:09.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.144+0000 7f9be9463700 1 -- 172.21.15.89:0/3213656484 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be40cf7b0 con 0x7f9be414cc50 2023-10-10T20:43:09.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.145+0000 7f9be9463700 1 -- 172.21.15.89:0/3213656484 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be404c3c0 con 0x7f9be414cc50 2023-10-10T20:43:09.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.145+0000 7f9bdb7fe700 1 -- 172.21.15.89:0/3213656484 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9bcc01d460 con 0x7f9be414cc50 2023-10-10T20:43:09.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.145+0000 7f9bdb7fe700 1 --2- 172.21.15.89:0/3213656484 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9bc4067320 0x7f9bc40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:09.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.146+0000 7f9bdb7fe700 1 -- 172.21.15.89:0/3213656484 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6077+0+0 (secure 0 0 0) 0x7f9bcc084e80 con 0x7f9be414cc50 2023-10-10T20:43:09.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.146+0000 7f9be2ffd700 1 --2- 172.21.15.89:0/3213656484 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9bc4067320 0x7f9bc40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:09.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.146+0000 7f9be2ffd700 1 --2- 172.21.15.89:0/3213656484 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9bc4067320 0x7f9bc40697d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40091e0 tx=0x7f9bd401a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:09.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.150+0000 7f9bdb7fe700 1 -- 172.21.15.89:0/3213656484 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bcc0519a0 con 0x7f9be414cc50 2023-10-10T20:43:09.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:09.387+0000 7f9be9463700 1 -- 172.21.15.89:0/3213656484 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "another-data-pool"} v 0) v1 -- 0x7f9be40cf940 con 0x7f9be414cc50 2023-10-10T20:43:10.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.003+0000 7f9bdb7fe700 1 -- 172.21.15.89:0/3213656484 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "another-data-pool"}]=0 added data pool 30 to fsmap v112) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9bcc054fc0 con 0x7f9be414cc50 2023-10-10T20:43:10.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.007+0000 7f9be9463700 1 -- 172.21.15.89:0/3213656484 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9bc4067320 msgr2=0x7f9bc40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.007+0000 7f9be9463700 1 --2- 172.21.15.89:0/3213656484 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9bc4067320 0x7f9bc40697d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40091e0 tx=0x7f9bd401a000 comp rx=0 tx=0).stop 2023-10-10T20:43:10.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.007+0000 7f9be9463700 1 -- 172.21.15.89:0/3213656484 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9be414cc50 msgr2=0x7f9be40b15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.007+0000 7f9be9463700 1 --2- 172.21.15.89:0/3213656484 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9be414cc50 0x7f9be40b15f0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00eb90 tx=0x7f9bcc001e90 comp rx=0 tx=0).stop 2023-10-10T20:43:10.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.008+0000 7f9be9463700 1 -- 172.21.15.89:0/3213656484 shutdown_connections 2023-10-10T20:43:10.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.008+0000 7f9be9463700 1 --2- 172.21.15.89:0/3213656484 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9bc4067320 0x7f9bc40697d0 secure :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40091e0 tx=0x7f9bd401a000 comp rx=0 tx=0).stop 2023-10-10T20:43:10.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.008+0000 7f9be9463700 1 --2- 172.21.15.89:0/3213656484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9be41512b0 0x7f9be40b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.008+0000 7f9be9463700 1 --2- 172.21.15.89:0/3213656484 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9bd0003a60 0x7f9be40ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.008+0000 7f9be9463700 1 --2- 172.21.15.89:0/3213656484 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9be414cc50 0x7f9be40b15f0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.008+0000 7f9be9463700 1 -- 172.21.15.89:0/3213656484 >> 172.21.15.89:0/3213656484 conn(0x7f9be4068e20 msgr2=0x7f9be414d200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:10.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.009+0000 7f9be9463700 1 -- 172.21.15.89:0/3213656484 shutdown_connections 2023-10-10T20:43:10.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.010+0000 7f9be9463700 1 -- 172.21.15.89:0/3213656484 wait complete. 2023-10-10T20:43:10.016 INFO:teuthology.orchestra.run.smithi089.stderr:added data pool 30 to fsmap 2023-10-10T20:43:10.028 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:43:10.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.141+0000 7fd390d7b700 1 Processor -- start 2023-10-10T20:43:10.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.141+0000 7fd390d7b700 1 -- start start 2023-10-10T20:43:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.142+0000 7fd390d7b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd38c159420 0x7fd38c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.142+0000 7fd390d7b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd38c1597f0 0x7fd38c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.142+0000 7fd390d7b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c0b11d0 con 0x7fd38c15d230 2023-10-10T20:43:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.142+0000 7fd390d7b700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38c0b14b0 con 0x7fd38c1597f0 2023-10-10T20:43:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.142+0000 7fd390d7b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0b1790 con 0x7fd38c159420 2023-10-10T20:43:10.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.142+0000 7fd38bfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd38c15d230 0x7fd38c0b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37666/0 (socket says 172.21.15.89:37666) 2023-10-10T20:43:10.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.142+0000 7fd38bfff700 1 -- 172.21.15.89:0/2908450181 learned_addr learned my addr 172.21.15.89:0/2908450181 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:10.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38affd700 1 --2- 172.21.15.89:0/2908450181 >> v2:172.21.15.111:3300/0 conn(0x7fd38c1597f0 0x7fd38c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38b7fe700 1 --2- 172.21.15.89:0/2908450181 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd38c159420 0x7fd38c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38a7fc700 1 -- 172.21.15.89:0/2908450181 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455160305 0 0) 0x7fd38c0b11d0 con 0x7fd38c15d230 2023-10-10T20:43:10.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38a7fc700 1 -- 172.21.15.89:0/2908450181 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd374003570 con 0x7fd38c15d230 2023-10-10T20:43:10.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38a7fc700 1 -- 172.21.15.89:0/2908450181 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1549954890 0 0) 0x7fd374003570 con 0x7fd38c15d230 2023-10-10T20:43:10.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38a7fc700 1 -- 172.21.15.89:0/2908450181 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd38c159420 msgr2=0x7fd38c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:10.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38a7fc700 1 --2- 172.21.15.89:0/2908450181 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd38c159420 0x7fd38c0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38a7fc700 1 -- 172.21.15.89:0/2908450181 >> v2:172.21.15.111:3300/0 conn(0x7fd38c1597f0 msgr2=0x7fd38c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38a7fc700 1 --2- 172.21.15.89:0/2908450181 >> v2:172.21.15.111:3300/0 conn(0x7fd38c1597f0 0x7fd38c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38a7fc700 1 -- 172.21.15.89:0/2908450181 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c0b1a70 con 0x7fd38c15d230 2023-10-10T20:43:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38a7fc700 1 -- 172.21.15.89:0/2908450181 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fd3840017f0 con 0x7fd38c15d230 2023-10-10T20:43:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38a7fc700 1 -- 172.21.15.89:0/2908450181 >> v1:172.21.15.89:6789/0 conn(0x7fd38c15d230 legacy=0x7fd38c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38a7fc700 1 --2- 172.21.15.89:0/2908450181 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd38c159420 0x7fd374003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38a7fc700 1 --2- 172.21.15.89:0/2908450181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd374003fe0 0x7fd374004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38a7fc700 1 --2- 172.21.15.89:0/2908450181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd374004d50 0x7fd3740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38a7fc700 1 -- 172.21.15.89:0/2908450181 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0b14b0 con 0x7fd374003fe0 2023-10-10T20:43:10.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38a7fc700 1 -- 172.21.15.89:0/2908450181 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0b1790 con 0x7fd374004d50 2023-10-10T20:43:10.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38a7fc700 1 -- 172.21.15.89:0/2908450181 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37400ba90 con 0x7fd38c159420 2023-10-10T20:43:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38affd700 1 --2- 172.21.15.89:0/2908450181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd374003fe0 0x7fd374004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38b7fe700 1 --2- 172.21.15.89:0/2908450181 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd38c159420 0x7fd374003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38bfff700 1 --2- 172.21.15.89:0/2908450181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd374004d50 0x7fd3740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38affd700 1 -- 172.21.15.89:0/2908450181 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd38c159420 msgr2=0x7fd374003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38affd700 1 --2- 172.21.15.89:0/2908450181 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd38c159420 0x7fd374003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38affd700 1 -- 172.21.15.89:0/2908450181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd374004d50 msgr2=0x7fd3740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:10.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38affd700 1 --2- 172.21.15.89:0/2908450181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd374004d50 0x7fd3740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38affd700 1 -- 172.21.15.89:0/2908450181 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd374003570 con 0x7fd374003fe0 2023-10-10T20:43:10.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.143+0000 7fd38affd700 1 --2- 172.21.15.89:0/2908450181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd374003fe0 0x7fd374004810 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fd380009c00 tx=0x7fd3800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d209dd1dc3ad70 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:10.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.144+0000 7fd38a7fc700 1 -- 172.21.15.89:0/2908450181 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38000e6b0 con 0x7fd374003fe0 2023-10-10T20:43:10.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.144+0000 7fd38a7fc700 1 -- 172.21.15.89:0/2908450181 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38000e810 con 0x7fd374003fe0 2023-10-10T20:43:10.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.144+0000 7fd38a7fc700 1 -- 172.21.15.89:0/2908450181 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38000ebd0 con 0x7fd374003fe0 2023-10-10T20:43:10.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.144+0000 7fd390d7b700 1 -- 172.21.15.89:0/2908450181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd374003fe0 msgr2=0x7fd374004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.144+0000 7fd390d7b700 1 --2- 172.21.15.89:0/2908450181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd374003fe0 0x7fd374004810 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fd380009c00 tx=0x7fd3800091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.144+0000 7fd38bfff700 1 -- 172.21.15.89:0/2908450181 reap_dead start 2023-10-10T20:43:10.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.144+0000 7fd390d7b700 1 -- 172.21.15.89:0/2908450181 shutdown_connections 2023-10-10T20:43:10.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.144+0000 7fd390d7b700 1 -- 172.21.15.89:0/2908450181 >> 172.21.15.89:0/2908450181 conn(0x7fd38c0d0660 msgr2=0x7fd38c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:10.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.144+0000 7fd390d7b700 1 -- 172.21.15.89:0/2908450181 shutdown_connections 2023-10-10T20:43:10.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.144+0000 7fd390d7b700 1 -- 172.21.15.89:0/2908450181 wait complete. 2023-10-10T20:43:10.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.145+0000 7fd390d7b700 1 Processor -- start 2023-10-10T20:43:10.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.145+0000 7fd390d7b700 1 -- start start 2023-10-10T20:43:10.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.146+0000 7fd390d7b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd374003fe0 0x7fd38c14f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.146+0000 7fd390d7b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd38c14fde0 0x7fd38c150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.146+0000 7fd390d7b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd38c151120 0x7fd38c152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.146+0000 7fd390d7b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c152290 con 0x7fd374003fe0 2023-10-10T20:43:10.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.146+0000 7fd390d7b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c151680 con 0x7fd38c14fde0 2023-10-10T20:43:10.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.146+0000 7fd390d7b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c151960 con 0x7fd38c151120 2023-10-10T20:43:10.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.146+0000 7fd38b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd374003fe0 0x7fd38c14f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.146+0000 7fd38b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd374003fe0 0x7fd38c14f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35018/0 (socket says 172.21.15.89:35018) 2023-10-10T20:43:10.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.146+0000 7fd38b7fe700 1 -- 172.21.15.89:0/3273265870 learned_addr learned my addr 172.21.15.89:0/3273265870 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:10.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.146+0000 7fd38b7fe700 1 -- 172.21.15.89:0/3273265870 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd38c151120 msgr2=0x7fd38c152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.146+0000 7fd38b7fe700 1 --2- 172.21.15.89:0/3273265870 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd38c151120 0x7fd38c152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.146+0000 7fd38b7fe700 1 -- 172.21.15.89:0/3273265870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd38c14fde0 msgr2=0x7fd38c150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.146+0000 7fd38b7fe700 1 --2- 172.21.15.89:0/3273265870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd38c14fde0 0x7fd38c150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.146+0000 7fd38b7fe700 1 -- 172.21.15.89:0/3273265870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c151c40 con 0x7fd374003fe0 2023-10-10T20:43:10.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.147+0000 7fd38b7fe700 1 --2- 172.21.15.89:0/3273265870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd374003fe0 0x7fd38c14f8a0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fd37c011d40 tx=0x7fd37c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:10.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.147+0000 7fd3737fe700 1 -- 172.21.15.89:0/3273265870 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c00fac0 con 0x7fd374003fe0 2023-10-10T20:43:10.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.147+0000 7fd3737fe700 1 -- 172.21.15.89:0/3273265870 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c00fc20 con 0x7fd374003fe0 2023-10-10T20:43:10.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.147+0000 7fd3737fe700 1 -- 172.21.15.89:0/3273265870 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c014440 con 0x7fd374003fe0 2023-10-10T20:43:10.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.147+0000 7fd390d7b700 1 -- 172.21.15.89:0/3273265870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38c151f20 con 0x7fd374003fe0 2023-10-10T20:43:10.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.147+0000 7fd390d7b700 1 -- 172.21.15.89:0/3273265870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c1534b0 con 0x7fd374003fe0 2023-10-10T20:43:10.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.148+0000 7fd390d7b700 1 -- 172.21.15.89:0/3273265870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38c04c3c0 con 0x7fd374003fe0 2023-10-10T20:43:10.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.148+0000 7fd3737fe700 1 -- 172.21.15.89:0/3273265870 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd37c014a80 con 0x7fd374003fe0 2023-10-10T20:43:10.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.148+0000 7fd3737fe700 1 --2- 172.21.15.89:0/3273265870 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd378067350 0x7fd378069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.149+0000 7fd3737fe700 1 -- 172.21.15.89:0/3273265870 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6109+0+0 (secure 0 0 0) 0x7fd37c018030 con 0x7fd374003fe0 2023-10-10T20:43:10.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.149+0000 7fd38affd700 1 --2- 172.21.15.89:0/3273265870 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd378067350 0x7fd378069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.149+0000 7fd38affd700 1 --2- 172.21.15.89:0/3273265870 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd378067350 0x7fd378069800 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd38c150210 tx=0x7fd38001a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:10.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.153+0000 7fd3737fe700 1 -- 172.21.15.89:0/3273265870 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd37c052590 con 0x7fd374003fe0 2023-10-10T20:43:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.377+0000 7fd390d7b700 1 -- 172.21.15.89:0/3273265870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd38c0b1d50 con 0x7fd374003fe0 2023-10-10T20:43:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.377+0000 7fd3737fe700 1 -- 172.21.15.89:0/3273265870 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4616 (secure 0 0 0) 0x7fd37c022090 con 0x7fd374003fe0 2023-10-10T20:43:10.380 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:10.380 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":112,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6638,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2350308973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2350308973},{"type":"v1","addr":"172.21.15.111:6835","nonce":2350308973}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":8322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/1051252386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":1051252386},{"type":"v1","addr":"172.21.15.89:6836","nonce":1051252386}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":8325,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3754994248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":3754994248},{"type":"v1","addr":"172.21.15.89:6837","nonce":3754994248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:42:52.707112+0000","modified":"2023-10-10T20:43:09.995458+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8337":{"gid":8337,"name":"b","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1733913702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1733913702},{"type":"v1","addr":"172.21.15.111:6834","nonce":1733913702}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-10T20:43:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.380+0000 7fd390d7b700 1 -- 172.21.15.89:0/3273265870 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd378067350 msgr2=0x7fd378069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.380+0000 7fd390d7b700 1 --2- 172.21.15.89:0/3273265870 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd378067350 0x7fd378069800 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd38c150210 tx=0x7fd38001a000 comp rx=0 tx=0).stop 2023-10-10T20:43:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.380+0000 7fd390d7b700 1 -- 172.21.15.89:0/3273265870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd374003fe0 msgr2=0x7fd38c14f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.380+0000 7fd390d7b700 1 --2- 172.21.15.89:0/3273265870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd374003fe0 0x7fd38c14f8a0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fd37c011d40 tx=0x7fd37c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.380+0000 7fd390d7b700 1 -- 172.21.15.89:0/3273265870 shutdown_connections 2023-10-10T20:43:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.380+0000 7fd390d7b700 1 --2- 172.21.15.89:0/3273265870 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd38c151120 0x7fd38c152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.380+0000 7fd390d7b700 1 --2- 172.21.15.89:0/3273265870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd374003fe0 0x7fd38c14f8a0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.380+0000 7fd390d7b700 1 --2- 172.21.15.89:0/3273265870 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd378067350 0x7fd378069800 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.380+0000 7fd390d7b700 1 --2- 172.21.15.89:0/3273265870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd38c14fde0 0x7fd38c150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.380+0000 7fd390d7b700 1 -- 172.21.15.89:0/3273265870 >> 172.21.15.89:0/3273265870 conn(0x7fd38c0d0660 msgr2=0x7fd38c0b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:10.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.380+0000 7fd390d7b700 1 -- 172.21.15.89:0/3273265870 shutdown_connections 2023-10-10T20:43:10.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.380+0000 7fd390d7b700 1 -- 172.21.15.89:0/3273265870 wait complete. 2023-10-10T20:43:10.384 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 112 2023-10-10T20:43:10.394 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:43:10.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.508+0000 7f4c09af7700 1 Processor -- start 2023-10-10T20:43:10.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.508+0000 7f4c09af7700 1 -- start start 2023-10-10T20:43:10.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.509+0000 7f4c09af7700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4c0406f810 0x7f4c041649f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.509+0000 7f4c09af7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c0406ec70 con 0x7f4c04068920 2023-10-10T20:43:10.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.509+0000 7f4c09af7700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c040b2e70 con 0x7f4c04068550 2023-10-10T20:43:10.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.509+0000 7f4c09af7700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c04068d90 con 0x7f4c0406f810 2023-10-10T20:43:10.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.509+0000 7f4c02ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4c04068920 0x7f4c0406f100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37672/0 (socket says 172.21.15.89:37672) 2023-10-10T20:43:10.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.509+0000 7f4c02ffd700 1 -- 172.21.15.89:0/755729563 learned_addr learned my addr 172.21.15.89:0/755729563 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:10.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.509+0000 7f4c027fc700 1 -- 172.21.15.89:0/755729563 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3129778287 0 0) 0x7f4c0406ec70 con 0x7f4c04068920 2023-10-10T20:43:10.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.509+0000 7f4c027fc700 1 -- 172.21.15.89:0/755729563 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bec003570 con 0x7f4c04068920 2023-10-10T20:43:10.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.509+0000 7f4c03fff700 1 --2- 172.21.15.89:0/755729563 >> v2:172.21.15.111:3300/0 conn(0x7f4c0406f810 0x7f4c041649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.509+0000 7f4c027fc700 1 -- 172.21.15.89:0/755729563 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3994609285 0 0) 0x7f4bec003570 con 0x7f4c04068920 2023-10-10T20:43:10.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.509+0000 7f4c027fc700 1 -- 172.21.15.89:0/755729563 >> v2:172.21.15.111:3300/0 conn(0x7f4c0406f810 msgr2=0x7f4c041649f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:10.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.509+0000 7f4c027fc700 1 --2- 172.21.15.89:0/755729563 >> v2:172.21.15.111:3300/0 conn(0x7f4c0406f810 0x7f4c041649f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c027fc700 1 -- 172.21.15.89:0/755729563 >> v1:172.21.15.111:6789/0 conn(0x7f4c04068550 legacy=0x7f4c0415e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c027fc700 1 -- 172.21.15.89:0/755729563 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c040b3150 con 0x7f4c04068920 2023-10-10T20:43:10.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c027fc700 1 -- 172.21.15.89:0/755729563 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f4bf8001470 con 0x7f4c04068920 2023-10-10T20:43:10.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c027fc700 1 -- 172.21.15.89:0/755729563 >> v1:172.21.15.89:6789/0 conn(0x7f4c04068920 legacy=0x7f4c0406f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c027fc700 1 --2- 172.21.15.89:0/755729563 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4bec003a60 0x7f4bec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c027fc700 1 --2- 172.21.15.89:0/755729563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bec004810 0x7f4bec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c027fc700 1 --2- 172.21.15.89:0/755729563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bec0051c0 0x7f4bec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c027fc700 1 -- 172.21.15.89:0/755729563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c04068d90 con 0x7f4bec0051c0 2023-10-10T20:43:10.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c027fc700 1 -- 172.21.15.89:0/755729563 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec00c160 con 0x7f4bec004810 2023-10-10T20:43:10.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c027fc700 1 -- 172.21.15.89:0/755729563 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec00c320 con 0x7f4bec003a60 2023-10-10T20:43:10.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c03fff700 1 --2- 172.21.15.89:0/755729563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bec0051c0 0x7f4bec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c037fe700 1 --2- 172.21.15.89:0/755729563 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4bec003a60 0x7f4bec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c02ffd700 1 --2- 172.21.15.89:0/755729563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bec004810 0x7f4bec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c03fff700 1 -- 172.21.15.89:0/755729563 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4bec003a60 msgr2=0x7f4bec003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:10.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c03fff700 1 --2- 172.21.15.89:0/755729563 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4bec003a60 0x7f4bec003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c03fff700 1 -- 172.21.15.89:0/755729563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bec004810 msgr2=0x7f4bec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:10.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c03fff700 1 --2- 172.21.15.89:0/755729563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bec004810 0x7f4bec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c03fff700 1 -- 172.21.15.89:0/755729563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec003570 con 0x7f4bec0051c0 2023-10-10T20:43:10.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c03fff700 1 -- 172.21.15.89:0/755729563 reap_dead start 2023-10-10T20:43:10.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c03fff700 1 --2- 172.21.15.89:0/755729563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bec0051c0 0x7f4bec00ba60 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc009be0 tx=0x7f4bfc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30e0977d0e9977c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:10.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c027fc700 1 -- 172.21.15.89:0/755729563 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc00e710 con 0x7f4bec0051c0 2023-10-10T20:43:10.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c027fc700 1 -- 172.21.15.89:0/755729563 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bfc00e890 con 0x7f4bec0051c0 2023-10-10T20:43:10.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.510+0000 7f4c027fc700 1 -- 172.21.15.89:0/755729563 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc00ec50 con 0x7f4bec0051c0 2023-10-10T20:43:10.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.511+0000 7f4c09af7700 1 -- 172.21.15.89:0/755729563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bec0051c0 msgr2=0x7f4bec00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.511+0000 7f4c09af7700 1 --2- 172.21.15.89:0/755729563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bec0051c0 0x7f4bec00ba60 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc009be0 tx=0x7f4bfc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.511+0000 7f4c09af7700 1 -- 172.21.15.89:0/755729563 shutdown_connections 2023-10-10T20:43:10.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.511+0000 7f4c09af7700 1 --2- 172.21.15.89:0/755729563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bec0051c0 0x7f4bec00ba60 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.511+0000 7f4c09af7700 1 -- 172.21.15.89:0/755729563 >> 172.21.15.89:0/755729563 conn(0x7f4c040cd480 msgr2=0x7f4c040cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:10.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.511+0000 7f4c09af7700 1 -- 172.21.15.89:0/755729563 shutdown_connections 2023-10-10T20:43:10.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.511+0000 7f4c09af7700 1 -- 172.21.15.89:0/755729563 wait complete. 2023-10-10T20:43:10.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.512+0000 7f4c09af7700 1 Processor -- start 2023-10-10T20:43:10.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.512+0000 7f4c09af7700 1 -- start start 2023-10-10T20:43:10.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.513+0000 7f4c09af7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bec0051c0 0x7f4c04157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.513+0000 7f4c09af7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c040abdf0 0x7f4c0414b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.513+0000 7f4c09af7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c0414b5a0 0x7f4c040aa020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.513+0000 7f4c09af7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c040ab960 con 0x7f4c0414b5a0 2023-10-10T20:43:10.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.513+0000 7f4c09af7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0414ba00 con 0x7f4bec0051c0 2023-10-10T20:43:10.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.513+0000 7f4c09af7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c040aad50 con 0x7f4c040abdf0 2023-10-10T20:43:10.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.513+0000 7f4c03fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c0414b5a0 0x7f4c040aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.513+0000 7f4c03fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c0414b5a0 0x7f4c040aa020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35034/0 (socket says 172.21.15.89:35034) 2023-10-10T20:43:10.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.513+0000 7f4c03fff700 1 -- 172.21.15.89:0/3599420315 learned_addr learned my addr 172.21.15.89:0/3599420315 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:10.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.513+0000 7f4c02ffd700 1 --2- 172.21.15.89:0/3599420315 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c040abdf0 0x7f4c0414b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.513+0000 7f4c037fe700 1 --2- 172.21.15.89:0/3599420315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bec0051c0 0x7f4c04157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.513+0000 7f4c03fff700 1 -- 172.21.15.89:0/3599420315 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c040abdf0 msgr2=0x7f4c0414b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.513+0000 7f4c03fff700 1 --2- 172.21.15.89:0/3599420315 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c040abdf0 0x7f4c0414b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.513+0000 7f4c03fff700 1 -- 172.21.15.89:0/3599420315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bec0051c0 msgr2=0x7f4c04157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.513+0000 7f4c03fff700 1 --2- 172.21.15.89:0/3599420315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bec0051c0 0x7f4c04157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.513+0000 7f4c03fff700 1 -- 172.21.15.89:0/3599420315 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c040ab030 con 0x7f4c0414b5a0 2023-10-10T20:43:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.513+0000 7f4c037fe700 1 --2- 172.21.15.89:0/3599420315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bec0051c0 0x7f4c04157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.513+0000 7f4c03fff700 1 --2- 172.21.15.89:0/3599420315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c0414b5a0 0x7f4c040aa020 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc000c00 tx=0x7f4bfc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.514+0000 7f4beb7fe700 1 -- 172.21.15.89:0/3599420315 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc006e50 con 0x7f4c0414b5a0 2023-10-10T20:43:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.514+0000 7f4beb7fe700 1 -- 172.21.15.89:0/3599420315 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bfc021040 con 0x7f4c0414b5a0 2023-10-10T20:43:10.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.514+0000 7f4c09af7700 1 -- 172.21.15.89:0/3599420315 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c040ab960 con 0x7f4c0414b5a0 2023-10-10T20:43:10.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.514+0000 7f4beb7fe700 1 -- 172.21.15.89:0/3599420315 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc025640 con 0x7f4c0414b5a0 2023-10-10T20:43:10.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.514+0000 7f4c09af7700 1 -- 172.21.15.89:0/3599420315 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c040ab5f0 con 0x7f4c0414b5a0 2023-10-10T20:43:10.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.515+0000 7f4c09af7700 1 -- 172.21.15.89:0/3599420315 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c040abaf0 con 0x7f4c0414b5a0 2023-10-10T20:43:10.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.515+0000 7f4beb7fe700 1 -- 172.21.15.89:0/3599420315 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4bfc025a30 con 0x7f4c0414b5a0 2023-10-10T20:43:10.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.515+0000 7f4beb7fe700 1 --2- 172.21.15.89:0/3599420315 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4bf0067320 0x7f4bf00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.515+0000 7f4beb7fe700 1 -- 172.21.15.89:0/3599420315 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f4bfc0862a0 con 0x7f4c0414b5a0 2023-10-10T20:43:10.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.515+0000 7f4c037fe700 1 --2- 172.21.15.89:0/3599420315 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4bf0067320 0x7f4bf00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.516+0000 7f4c037fe700 1 --2- 172.21.15.89:0/3599420315 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4bf0067320 0x7f4bf00697d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4004290 tx=0x7f4bf4001640 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:10.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.520+0000 7f4beb7fe700 1 -- 172.21.15.89:0/3599420315 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4bfc052da0 con 0x7f4c0414b5a0 2023-10-10T20:43:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.704+0000 7f4c09af7700 1 -- 172.21.15.89:0/3599420315 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c0404c3c0 con 0x7f4c0414b5a0 2023-10-10T20:43:10.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.705+0000 7f4beb7fe700 1 -- 172.21.15.89:0/3599420315 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+16947 (secure 0 0 0) 0x7f4bfc0563c0 con 0x7f4c0414b5a0 2023-10-10T20:43:10.708 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:10.708 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":167,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:43:09.388945+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:42:51.066663+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-10T20:42:51.675525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":30,"pool_name":"another-data-pool","create_time":"2023-10-10T20:43:08.234534+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:43:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.708+0000 7f4c09af7700 1 -- 172.21.15.89:0/3599420315 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4bf0067320 msgr2=0x7f4bf00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.708+0000 7f4c09af7700 1 --2- 172.21.15.89:0/3599420315 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4bf0067320 0x7f4bf00697d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4004290 tx=0x7f4bf4001640 comp rx=0 tx=0).stop 2023-10-10T20:43:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.708+0000 7f4c09af7700 1 -- 172.21.15.89:0/3599420315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c0414b5a0 msgr2=0x7f4c040aa020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.708+0000 7f4c09af7700 1 --2- 172.21.15.89:0/3599420315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c0414b5a0 0x7f4c040aa020 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc000c00 tx=0x7f4bfc019000 comp rx=0 tx=0).stop 2023-10-10T20:43:10.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.708+0000 7f4c09af7700 1 -- 172.21.15.89:0/3599420315 shutdown_connections 2023-10-10T20:43:10.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.708+0000 7f4c09af7700 1 --2- 172.21.15.89:0/3599420315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c0414b5a0 0x7f4c040aa020 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.708+0000 7f4c09af7700 1 --2- 172.21.15.89:0/3599420315 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4bf0067320 0x7f4bf00697d0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.708+0000 7f4c09af7700 1 --2- 172.21.15.89:0/3599420315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bec0051c0 0x7f4c04157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.708+0000 7f4c09af7700 1 --2- 172.21.15.89:0/3599420315 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4c040abdf0 0x7f4c0414b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.708+0000 7f4c09af7700 1 -- 172.21.15.89:0/3599420315 >> 172.21.15.89:0/3599420315 conn(0x7f4c040cd480 msgr2=0x7f4c040acee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.708+0000 7f4c09af7700 1 -- 172.21.15.89:0/3599420315 shutdown_connections 2023-10-10T20:43:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.708+0000 7f4c09af7700 1 -- 172.21.15.89:0/3599420315 wait complete. 2023-10-10T20:43:10.722 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:43:10.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.835+0000 7fbd5af97700 1 Processor -- start 2023-10-10T20:43:10.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.835+0000 7fbd5af97700 1 -- start start 2023-10-10T20:43:10.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.836+0000 7fbd5af97700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbd541599c0 0x7fbd54069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.836+0000 7fbd5af97700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5406a220 con 0x7fbd54069aa0 2023-10-10T20:43:10.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.836+0000 7fbd5af97700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd54069f40 con 0x7fbd540cbd00 2023-10-10T20:43:10.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.836+0000 7fbd5af97700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5415a0b0 con 0x7fbd541599c0 2023-10-10T20:43:10.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.836+0000 7fbd53fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbd54069aa0 0x7fbd540cb5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37680/0 (socket says 172.21.15.89:37680) 2023-10-10T20:43:10.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.836+0000 7fbd53fff700 1 -- 172.21.15.89:0/3210547745 learned_addr learned my addr 172.21.15.89:0/3210547745 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:10.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.837+0000 7fbd58d33700 1 --2- 172.21.15.89:0/3210547745 >> v2:172.21.15.111:3300/0 conn(0x7fbd541599c0 0x7fbd54069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.837+0000 7fbd537fe700 1 -- 172.21.15.89:0/3210547745 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3315202267 0 0) 0x7fbd5406a220 con 0x7fbd54069aa0 2023-10-10T20:43:10.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.837+0000 7fbd537fe700 1 -- 172.21.15.89:0/3210547745 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd40003570 con 0x7fbd54069aa0 2023-10-10T20:43:10.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.837+0000 7fbd537fe700 1 -- 172.21.15.89:0/3210547745 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4096170174 0 0) 0x7fbd54069f40 con 0x7fbd540cbd00 2023-10-10T20:43:10.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.837+0000 7fbd537fe700 1 -- 172.21.15.89:0/3210547745 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd5406a220 con 0x7fbd540cbd00 2023-10-10T20:43:10.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.837+0000 7fbd537fe700 1 -- 172.21.15.89:0/3210547745 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1323492380 0 0) 0x7fbd40003570 con 0x7fbd54069aa0 2023-10-10T20:43:10.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.837+0000 7fbd58d33700 1 -- 172.21.15.89:0/3210547745 >> v1:172.21.15.111:6789/0 conn(0x7fbd540cbd00 legacy=0x7fbd540ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.837+0000 7fbd58d33700 1 -- 172.21.15.89:0/3210547745 >> v1:172.21.15.89:6789/0 conn(0x7fbd54069aa0 legacy=0x7fbd540cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.837+0000 7fbd58d33700 1 -- 172.21.15.89:0/3210547745 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5415a390 con 0x7fbd541599c0 2023-10-10T20:43:10.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.837+0000 7fbd58d33700 1 --2- 172.21.15.89:0/3210547745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd541599c0 0x7fbd54069560 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009e90 tx=0x7fbd44009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51395dc5d592c726 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:10.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.837+0000 7fbd537fe700 1 -- 172.21.15.89:0/3210547745 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4400e070 con 0x7fbd541599c0 2023-10-10T20:43:10.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.837+0000 7fbd537fe700 1 -- 172.21.15.89:0/3210547745 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4400b870 con 0x7fbd541599c0 2023-10-10T20:43:10.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.837+0000 7fbd537fe700 1 -- 172.21.15.89:0/3210547745 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c001470 con 0x7fbd541599c0 2023-10-10T20:43:10.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.838+0000 7fbd5af97700 1 -- 172.21.15.89:0/3210547745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd541599c0 msgr2=0x7fbd54069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.838+0000 7fbd5af97700 1 --2- 172.21.15.89:0/3210547745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd541599c0 0x7fbd54069560 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009e90 tx=0x7fbd44009780 comp rx=0 tx=0).stop 2023-10-10T20:43:10.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.838+0000 7fbd5af97700 1 -- 172.21.15.89:0/3210547745 shutdown_connections 2023-10-10T20:43:10.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.838+0000 7fbd5af97700 1 --2- 172.21.15.89:0/3210547745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd541599c0 0x7fbd54069560 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.838+0000 7fbd5af97700 1 -- 172.21.15.89:0/3210547745 >> 172.21.15.89:0/3210547745 conn(0x7fbd54068e10 msgr2=0x7fbd540d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:10.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.838+0000 7fbd5af97700 1 -- 172.21.15.89:0/3210547745 shutdown_connections 2023-10-10T20:43:10.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.838+0000 7fbd5af97700 1 -- 172.21.15.89:0/3210547745 wait complete. 2023-10-10T20:43:10.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.839+0000 7fbd5af97700 1 Processor -- start 2023-10-10T20:43:10.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.839+0000 7fbd5af97700 1 -- start start 2023-10-10T20:43:10.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.840+0000 7fbd5af97700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd54069aa0 0x7fbd540ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.840+0000 7fbd5af97700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbd540cbd00 0x7fbd54157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.840+0000 7fbd5af97700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd541599c0 0x7fbd540cea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.840+0000 7fbd5af97700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd541581e0 con 0x7fbd54069aa0 2023-10-10T20:43:10.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.840+0000 7fbd5af97700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd540ca400 con 0x7fbd541599c0 2023-10-10T20:43:10.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.840+0000 7fbd5af97700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd540c95a0 con 0x7fbd540cbd00 2023-10-10T20:43:10.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.840+0000 7fbd58d33700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd54069aa0 0x7fbd540ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.840+0000 7fbd58d33700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd54069aa0 0x7fbd540ca690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35040/0 (socket says 172.21.15.89:35040) 2023-10-10T20:43:10.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.840+0000 7fbd58d33700 1 -- 172.21.15.89:0/4196502747 learned_addr learned my addr 172.21.15.89:0/4196502747 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:10.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.840+0000 7fbd53fff700 1 --2- 172.21.15.89:0/4196502747 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbd540cbd00 0x7fbd54157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.840+0000 7fbd59534700 1 --2- 172.21.15.89:0/4196502747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd541599c0 0x7fbd540cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.840+0000 7fbd58d33700 1 -- 172.21.15.89:0/4196502747 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbd540cbd00 msgr2=0x7fbd54157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.840+0000 7fbd58d33700 1 --2- 172.21.15.89:0/4196502747 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbd540cbd00 0x7fbd54157ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.840+0000 7fbd58d33700 1 -- 172.21.15.89:0/4196502747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd541599c0 msgr2=0x7fbd540cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:10.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.840+0000 7fbd58d33700 1 --2- 172.21.15.89:0/4196502747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd541599c0 0x7fbd540cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:10.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.840+0000 7fbd58d33700 1 -- 172.21.15.89:0/4196502747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd44009a20 con 0x7fbd54069aa0 2023-10-10T20:43:10.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.840+0000 7fbd59534700 1 --2- 172.21.15.89:0/4196502747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd541599c0 0x7fbd540cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:10.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.841+0000 7fbd58d33700 1 --2- 172.21.15.89:0/4196502747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd54069aa0 0x7fbd540ca690 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fbd54067b80 tx=0x7fbd44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:10.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.841+0000 7fbd50ff9700 1 -- 172.21.15.89:0/4196502747 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44012c50 con 0x7fbd54069aa0 2023-10-10T20:43:10.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.841+0000 7fbd50ff9700 1 -- 172.21.15.89:0/4196502747 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd44012db0 con 0x7fbd54069aa0 2023-10-10T20:43:10.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.841+0000 7fbd50ff9700 1 -- 172.21.15.89:0/4196502747 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4401e6a0 con 0x7fbd54069aa0 2023-10-10T20:43:10.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.841+0000 7fbd5af97700 1 -- 172.21.15.89:0/4196502747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd54169d80 con 0x7fbd54069aa0 2023-10-10T20:43:10.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.841+0000 7fbd5af97700 1 -- 172.21.15.89:0/4196502747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd540c8fe0 con 0x7fbd54069aa0 2023-10-10T20:43:10.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.842+0000 7fbd5af97700 1 -- 172.21.15.89:0/4196502747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd54088530 con 0x7fbd54069aa0 2023-10-10T20:43:10.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.842+0000 7fbd50ff9700 1 -- 172.21.15.89:0/4196502747 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbd4401ead0 con 0x7fbd54069aa0 2023-10-10T20:43:10.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.842+0000 7fbd50ff9700 1 --2- 172.21.15.89:0/4196502747 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbd34067320 0x7fbd340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:10.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.842+0000 7fbd50ff9700 1 -- 172.21.15.89:0/4196502747 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6109+0+0 (secure 0 0 0) 0x7fbd440872f0 con 0x7fbd54069aa0 2023-10-10T20:43:10.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.842+0000 7fbd53fff700 1 --2- 172.21.15.89:0/4196502747 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbd34067320 0x7fbd340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:10.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.843+0000 7fbd53fff700 1 --2- 172.21.15.89:0/4196502747 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbd34067320 0x7fbd340697d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c001ed0 tx=0x7fbd3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:10.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:10.847+0000 7fbd50ff9700 1 -- 172.21.15.89:0/4196502747 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd44053ce0 con 0x7fbd54069aa0 2023-10-10T20:43:11.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.075+0000 7fbd5af97700 1 -- 172.21.15.89:0/4196502747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd54064550 con 0x7fbd54069aa0 2023-10-10T20:43:11.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.075+0000 7fbd50ff9700 1 -- 172.21.15.89:0/4196502747 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4616 (secure 0 0 0) 0x7fbd44057300 con 0x7fbd54069aa0 2023-10-10T20:43:11.077 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:11.078 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":112,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6638,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2350308973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2350308973},{"type":"v1","addr":"172.21.15.111:6835","nonce":2350308973}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":8322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/1051252386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":1051252386},{"type":"v1","addr":"172.21.15.89:6836","nonce":1051252386}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":8325,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3754994248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":3754994248},{"type":"v1","addr":"172.21.15.89:6837","nonce":3754994248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:42:52.707112+0000","modified":"2023-10-10T20:43:09.995458+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8337":{"gid":8337,"name":"b","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1733913702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1733913702},{"type":"v1","addr":"172.21.15.111:6834","nonce":1733913702}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-10T20:43:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.077+0000 7fbd5af97700 1 -- 172.21.15.89:0/4196502747 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbd34067320 msgr2=0x7fbd340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.077+0000 7fbd5af97700 1 --2- 172.21.15.89:0/4196502747 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbd34067320 0x7fbd340697d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c001ed0 tx=0x7fbd3c009040 comp rx=0 tx=0).stop 2023-10-10T20:43:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.077+0000 7fbd5af97700 1 -- 172.21.15.89:0/4196502747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd54069aa0 msgr2=0x7fbd540ca690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.077+0000 7fbd5af97700 1 --2- 172.21.15.89:0/4196502747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd54069aa0 0x7fbd540ca690 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fbd54067b80 tx=0x7fbd44017000 comp rx=0 tx=0).stop 2023-10-10T20:43:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.077+0000 7fbd5af97700 1 -- 172.21.15.89:0/4196502747 shutdown_connections 2023-10-10T20:43:11.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.077+0000 7fbd5af97700 1 --2- 172.21.15.89:0/4196502747 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbd34067320 0x7fbd340697d0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.077+0000 7fbd5af97700 1 --2- 172.21.15.89:0/4196502747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd541599c0 0x7fbd540cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.077+0000 7fbd5af97700 1 --2- 172.21.15.89:0/4196502747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd54069aa0 0x7fbd540ca690 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.077+0000 7fbd5af97700 1 --2- 172.21.15.89:0/4196502747 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbd540cbd00 0x7fbd54157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.078+0000 7fbd5af97700 1 -- 172.21.15.89:0/4196502747 >> 172.21.15.89:0/4196502747 conn(0x7fbd54068e10 msgr2=0x7fbd540c5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:11.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.078+0000 7fbd5af97700 1 -- 172.21.15.89:0/4196502747 shutdown_connections 2023-10-10T20:43:11.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.078+0000 7fbd5af97700 1 -- 172.21.15.89:0/4196502747 wait complete. 2023-10-10T20:43:11.082 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 112 2023-10-10T20:43:11.093 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:43:11.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.208+0000 7f748d879700 1 Processor -- start 2023-10-10T20:43:11.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.208+0000 7f748d879700 1 -- start start 2023-10-10T20:43:11.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.208+0000 7f748d879700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7488159890 0x7f748815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:11.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.208+0000 7f748d879700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7488159c60 0x7f748815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:11.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.208+0000 7f748d879700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f748815db10 0x7f7488069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:11.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.208+0000 7f748d879700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748815a160 con 0x7f7488159c60 2023-10-10T20:43:11.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.208+0000 7f748d879700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74880d13a0 con 0x7f748815db10 2023-10-10T20:43:11.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.208+0000 7f748d879700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748815e140 con 0x7f7488159890 2023-10-10T20:43:11.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.208+0000 7f74867fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7488159c60 0x7f748815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:11.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.209+0000 7f74867fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7488159c60 0x7f748815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35046/0 (socket says 172.21.15.89:35046) 2023-10-10T20:43:11.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.209+0000 7f74867fc700 1 -- 172.21.15.89:0/1965693966 learned_addr learned my addr 172.21.15.89:0/1965693966 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:11.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.209+0000 7f74877fe700 1 --2- 172.21.15.89:0/1965693966 >> v2:172.21.15.111:3300/0 conn(0x7f748815db10 0x7f7488069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:11.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.209+0000 7f7486ffd700 1 --2- 172.21.15.89:0/1965693966 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7488159890 0x7f748815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:11.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.209+0000 7f74867fc700 1 -- 172.21.15.89:0/1965693966 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7488159890 msgr2=0x7f748815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:11.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.209+0000 7f74867fc700 1 --2- 172.21.15.89:0/1965693966 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7488159890 0x7f748815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.209+0000 7f74867fc700 1 -- 172.21.15.89:0/1965693966 >> v2:172.21.15.111:3300/0 conn(0x7f748815db10 msgr2=0x7f7488069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:11.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.209+0000 7f74867fc700 1 --2- 172.21.15.89:0/1965693966 >> v2:172.21.15.111:3300/0 conn(0x7f748815db10 0x7f7488069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.209+0000 7f74867fc700 1 -- 172.21.15.89:0/1965693966 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748815a3d0 con 0x7f7488159c60 2023-10-10T20:43:11.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.209+0000 7f74877fe700 1 --2- 172.21.15.89:0/1965693966 >> v2:172.21.15.111:3300/0 conn(0x7f748815db10 0x7f7488069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:11.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.209+0000 7f7486ffd700 1 --2- 172.21.15.89:0/1965693966 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7488159890 0x7f748815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:11.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.209+0000 7f74867fc700 1 --2- 172.21.15.89:0/1965693966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7488159c60 0x7f748815d5d0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f7488089b50 tx=0x7f7470009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5187f28aa26a55c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:11.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.209+0000 7f7485ffb700 1 -- 172.21.15.89:0/1965693966 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747000b860 con 0x7f7488159c60 2023-10-10T20:43:11.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.209+0000 7f7485ffb700 1 -- 172.21.15.89:0/1965693966 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747000b9c0 con 0x7f7488159c60 2023-10-10T20:43:11.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.209+0000 7f7485ffb700 1 -- 172.21.15.89:0/1965693966 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747000bd60 con 0x7f7488159c60 2023-10-10T20:43:11.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.210+0000 7f748d879700 1 -- 172.21.15.89:0/1965693966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7488159c60 msgr2=0x7f748815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:11.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.210+0000 7f748d879700 1 --2- 172.21.15.89:0/1965693966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7488159c60 0x7f748815d5d0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f7488089b50 tx=0x7f7470009250 comp rx=0 tx=0).stop 2023-10-10T20:43:11.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.210+0000 7f748d879700 1 -- 172.21.15.89:0/1965693966 shutdown_connections 2023-10-10T20:43:11.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.210+0000 7f748d879700 1 --2- 172.21.15.89:0/1965693966 >> v2:172.21.15.111:3300/0 conn(0x7f748815db10 0x7f7488069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.210+0000 7f748d879700 1 --2- 172.21.15.89:0/1965693966 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7488159890 0x7f748815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.210+0000 7f748d879700 1 --2- 172.21.15.89:0/1965693966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7488159c60 0x7f748815d5d0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.210+0000 7f748d879700 1 -- 172.21.15.89:0/1965693966 >> 172.21.15.89:0/1965693966 conn(0x7f7488068e20 msgr2=0x7f74880d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:11.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.210+0000 7f748d879700 1 -- 172.21.15.89:0/1965693966 shutdown_connections 2023-10-10T20:43:11.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.210+0000 7f748d879700 1 -- 172.21.15.89:0/1965693966 wait complete. 2023-10-10T20:43:11.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.211+0000 7f748d879700 1 Processor -- start 2023-10-10T20:43:11.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.211+0000 7f748d879700 1 -- start start 2023-10-10T20:43:11.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f748d879700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7488159890 0x7f74880b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:11.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f748d879700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7488159c60 0x7f74880c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:11.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f748d879700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f748815db10 0x7f74880cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:11.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f748d879700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74880c9320 con 0x7f748815db10 2023-10-10T20:43:11.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f748d879700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74880cdb80 con 0x7f7488159890 2023-10-10T20:43:11.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f748d879700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74880cde60 con 0x7f7488159c60 2023-10-10T20:43:11.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f74877fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f748815db10 0x7f74880cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:11.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f74877fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f748815db10 0x7f74880cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35056/0 (socket says 172.21.15.89:35056) 2023-10-10T20:43:11.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f74877fe700 1 -- 172.21.15.89:0/535682773 learned_addr learned my addr 172.21.15.89:0/535682773 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:11.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f74867fc700 1 --2- 172.21.15.89:0/535682773 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7488159c60 0x7f74880c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:11.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f7486ffd700 1 --2- 172.21.15.89:0/535682773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7488159890 0x7f74880b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:11.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f74877fe700 1 -- 172.21.15.89:0/535682773 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7488159c60 msgr2=0x7f74880c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:11.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f74877fe700 1 --2- 172.21.15.89:0/535682773 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7488159c60 0x7f74880c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f74877fe700 1 -- 172.21.15.89:0/535682773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7488159890 msgr2=0x7f74880b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:11.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f74877fe700 1 --2- 172.21.15.89:0/535682773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7488159890 0x7f74880b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f74877fe700 1 -- 172.21.15.89:0/535682773 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74700094f0 con 0x7f748815db10 2023-10-10T20:43:11.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f74877fe700 1 --2- 172.21.15.89:0/535682773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f748815db10 0x7f74880cd0c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f748000d890 tx=0x7f7480008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:11.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f747effd700 1 -- 172.21.15.89:0/535682773 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74800125f0 con 0x7f748815db10 2023-10-10T20:43:11.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f747effd700 1 -- 172.21.15.89:0/535682773 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7480012750 con 0x7f748815db10 2023-10-10T20:43:11.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f748d879700 1 -- 172.21.15.89:0/535682773 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74880c8fb0 con 0x7f748815db10 2023-10-10T20:43:11.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f747effd700 1 -- 172.21.15.89:0/535682773 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7480012ad0 con 0x7f748815db10 2023-10-10T20:43:11.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.212+0000 7f748d879700 1 -- 172.21.15.89:0/535682773 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74880c9cf0 con 0x7f748815db10 2023-10-10T20:43:11.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.213+0000 7f748d879700 1 -- 172.21.15.89:0/535682773 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748804c3c0 con 0x7f748815db10 2023-10-10T20:43:11.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.214+0000 7f747effd700 1 -- 172.21.15.89:0/535682773 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7480013480 con 0x7f748815db10 2023-10-10T20:43:11.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.214+0000 7f747effd700 1 --2- 172.21.15.89:0/535682773 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7468067320 0x7f74680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:11.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.214+0000 7f747effd700 1 -- 172.21.15.89:0/535682773 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f7480084d50 con 0x7f748815db10 2023-10-10T20:43:11.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.214+0000 7f7486ffd700 1 --2- 172.21.15.89:0/535682773 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7468067320 0x7f74680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:11.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.214+0000 7f7486ffd700 1 --2- 172.21.15.89:0/535682773 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7468067320 0x7f74680697d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7478007740 tx=0x7f74780099d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:11.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.218+0000 7f747effd700 1 -- 172.21.15.89:0/535682773 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74800553f0 con 0x7f748815db10 2023-10-10T20:43:11.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.402+0000 7f748d879700 1 -- 172.21.15.89:0/535682773 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f748815a990 con 0x7f748815db10 2023-10-10T20:43:11.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.403+0000 7f747effd700 1 -- 172.21.15.89:0/535682773 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+16934 (secure 0 0 0) 0x7f7480054f80 con 0x7f748815db10 2023-10-10T20:43:11.405 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:11.406 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":168,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:43:11.007016+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:42:51.066663+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-10T20:42:51.675525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":30,"pool_name":"another-data-pool","create_time":"2023-10-10T20:43:08.234534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:43:11.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.405+0000 7f748d879700 1 -- 172.21.15.89:0/535682773 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7468067320 msgr2=0x7f74680697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:11.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.405+0000 7f748d879700 1 --2- 172.21.15.89:0/535682773 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7468067320 0x7f74680697d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7478007740 tx=0x7f74780099d0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.405+0000 7f748d879700 1 -- 172.21.15.89:0/535682773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f748815db10 msgr2=0x7f74880cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.405+0000 7f748d879700 1 --2- 172.21.15.89:0/535682773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f748815db10 0x7f74880cd0c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f748000d890 tx=0x7f7480008040 comp rx=0 tx=0).stop 2023-10-10T20:43:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.405+0000 7f748d879700 1 -- 172.21.15.89:0/535682773 shutdown_connections 2023-10-10T20:43:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.405+0000 7f748d879700 1 --2- 172.21.15.89:0/535682773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f748815db10 0x7f74880cd0c0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.405+0000 7f748d879700 1 --2- 172.21.15.89:0/535682773 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7468067320 0x7f74680697d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.405+0000 7f748d879700 1 --2- 172.21.15.89:0/535682773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7488159890 0x7f74880b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.405+0000 7f748d879700 1 --2- 172.21.15.89:0/535682773 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7488159c60 0x7f74880c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.405+0000 7f748d879700 1 -- 172.21.15.89:0/535682773 >> 172.21.15.89:0/535682773 conn(0x7f7488068e20 msgr2=0x7f74880c64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.405+0000 7f748d879700 1 -- 172.21.15.89:0/535682773 shutdown_connections 2023-10-10T20:43:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.405+0000 7f748d879700 1 -- 172.21.15.89:0/535682773 wait complete. 2023-10-10T20:43:11.419 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000654221 --yes-i-really-mean-it 2023-10-10T20:43:11.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.537+0000 7fd9ff08f700 1 Processor -- start 2023-10-10T20:43:11.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.537+0000 7fd9ff08f700 1 -- start start 2023-10-10T20:43:11.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.538+0000 7fd9ff08f700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd9f80cca20 0x7fd9f80cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:11.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.538+0000 7fd9ff08f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f80cfa00 con 0x7fd9f80b0790 2023-10-10T20:43:11.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.538+0000 7fd9ff08f700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f80cb580 con 0x7fd9f80b03c0 2023-10-10T20:43:11.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.538+0000 7fd9ff08f700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9f80cf160 con 0x7fd9f80cca20 2023-10-10T20:43:11.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.538+0000 7fd9fd88c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd9f80b0790 0x7fd9f81591e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37682/0 (socket says 172.21.15.89:37682) 2023-10-10T20:43:11.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.538+0000 7fd9fd88c700 1 -- 172.21.15.89:0/2049476127 learned_addr learned my addr 172.21.15.89:0/2049476127 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:11.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.538+0000 7fd9fd08b700 1 -- 172.21.15.89:0/2049476127 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407596833 0 0) 0x7fd9f80cfa00 con 0x7fd9f80b0790 2023-10-10T20:43:11.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.538+0000 7fd9fd08b700 1 -- 172.21.15.89:0/2049476127 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9e4003570 con 0x7fd9f80b0790 2023-10-10T20:43:11.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.538+0000 7fd9fe88e700 1 --2- 172.21.15.89:0/2049476127 >> v2:172.21.15.111:3300/0 conn(0x7fd9f80cca20 0x7fd9f80cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:11.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fd08b700 1 -- 172.21.15.89:0/2049476127 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3587782163 0 0) 0x7fd9e4003570 con 0x7fd9f80b0790 2023-10-10T20:43:11.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fd08b700 1 -- 172.21.15.89:0/2049476127 >> v2:172.21.15.111:3300/0 conn(0x7fd9f80cca20 msgr2=0x7fd9f80cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:11.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fd08b700 1 --2- 172.21.15.89:0/2049476127 >> v2:172.21.15.111:3300/0 conn(0x7fd9f80cca20 0x7fd9f80cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fd08b700 1 -- 172.21.15.89:0/2049476127 >> v1:172.21.15.111:6789/0 conn(0x7fd9f80b03c0 legacy=0x7fd9f80b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:11.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fd08b700 1 -- 172.21.15.89:0/2049476127 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f80cf440 con 0x7fd9f80b0790 2023-10-10T20:43:11.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fd08b700 1 -- 172.21.15.89:0/2049476127 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fd9e8001470 con 0x7fd9f80b0790 2023-10-10T20:43:11.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fd08b700 1 -- 172.21.15.89:0/2049476127 >> v1:172.21.15.89:6789/0 conn(0x7fd9f80b0790 legacy=0x7fd9f81591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:11.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fd08b700 1 --2- 172.21.15.89:0/2049476127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9e4003a60 0x7fd9e4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:11.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fd08b700 1 --2- 172.21.15.89:0/2049476127 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd9e4004810 0x7fd9e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:11.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fd08b700 1 --2- 172.21.15.89:0/2049476127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9e40052e0 0x7fd9e400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:11.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fd08b700 1 -- 172.21.15.89:0/2049476127 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f80cf160 con 0x7fd9e40052e0 2023-10-10T20:43:11.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fd08b700 1 -- 172.21.15.89:0/2049476127 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e400c230 con 0x7fd9e4003a60 2023-10-10T20:43:11.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fd08b700 1 -- 172.21.15.89:0/2049476127 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e400c3f0 con 0x7fd9e4004810 2023-10-10T20:43:11.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fe88e700 1 --2- 172.21.15.89:0/2049476127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9e40052e0 0x7fd9e400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:11.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fd88c700 1 --2- 172.21.15.89:0/2049476127 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd9e4004810 0x7fd9e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:11.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fe08d700 1 --2- 172.21.15.89:0/2049476127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9e4003a60 0x7fd9e4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:11.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fe88e700 1 -- 172.21.15.89:0/2049476127 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd9e4004810 msgr2=0x7fd9e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:11.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fe88e700 1 --2- 172.21.15.89:0/2049476127 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd9e4004810 0x7fd9e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fe88e700 1 -- 172.21.15.89:0/2049476127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9e4003a60 msgr2=0x7fd9e4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:11.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fe88e700 1 --2- 172.21.15.89:0/2049476127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9e4003a60 0x7fd9e4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fe88e700 1 -- 172.21.15.89:0/2049476127 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e4003570 con 0x7fd9e40052e0 2023-10-10T20:43:11.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fd88c700 1 --2- 172.21.15.89:0/2049476127 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd9e4004810 0x7fd9e4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:11.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fe08d700 1 --2- 172.21.15.89:0/2049476127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9e4003a60 0x7fd9e4003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:11.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fe88e700 1 -- 172.21.15.89:0/2049476127 reap_dead start 2023-10-10T20:43:11.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fe88e700 1 --2- 172.21.15.89:0/2049476127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9e40052e0 0x7fd9e400ba60 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0007440 tx=0x7fd9f0009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da82536fda843bf server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:11.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fd08b700 1 -- 172.21.15.89:0/2049476127 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0009a40 con 0x7fd9e40052e0 2023-10-10T20:43:11.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fd08b700 1 -- 172.21.15.89:0/2049476127 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f0007cf0 con 0x7fd9e40052e0 2023-10-10T20:43:11.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.539+0000 7fd9fd08b700 1 -- 172.21.15.89:0/2049476127 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0014650 con 0x7fd9e40052e0 2023-10-10T20:43:11.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.540+0000 7fd9ff08f700 1 -- 172.21.15.89:0/2049476127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9e40052e0 msgr2=0x7fd9e400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:11.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.540+0000 7fd9ff08f700 1 --2- 172.21.15.89:0/2049476127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9e40052e0 0x7fd9e400ba60 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0007440 tx=0x7fd9f0009710 comp rx=0 tx=0).stop 2023-10-10T20:43:11.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.540+0000 7fd9ff08f700 1 -- 172.21.15.89:0/2049476127 shutdown_connections 2023-10-10T20:43:11.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.540+0000 7fd9ff08f700 1 --2- 172.21.15.89:0/2049476127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9e40052e0 0x7fd9e400ba60 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.540+0000 7fd9ff08f700 1 -- 172.21.15.89:0/2049476127 >> 172.21.15.89:0/2049476127 conn(0x7fd9f8063230 msgr2=0x7fd9f8063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:11.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.540+0000 7fd9ff08f700 1 -- 172.21.15.89:0/2049476127 shutdown_connections 2023-10-10T20:43:11.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.540+0000 7fd9ff08f700 1 -- 172.21.15.89:0/2049476127 wait complete. 2023-10-10T20:43:11.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.541+0000 7fd9ff08f700 1 Processor -- start 2023-10-10T20:43:11.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.541+0000 7fd9ff08f700 1 -- start start 2023-10-10T20:43:11.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9ff08f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9e40052e0 0x7fd9f80c4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:11.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9ff08f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd9f80c7b60 0x7fd9f80c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:11.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9ff08f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9f80c84f0 0x7fd9f80c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:11.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9ff08f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f8169c60 con 0x7fd9f80c84f0 2023-10-10T20:43:11.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9ff08f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f80c8970 con 0x7fd9e40052e0 2023-10-10T20:43:11.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9ff08f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f80c7730 con 0x7fd9f80c7b60 2023-10-10T20:43:11.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9fe88e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9f80c84f0 0x7fd9f80c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:11.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9fe88e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9f80c84f0 0x7fd9f80c9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35072/0 (socket says 172.21.15.89:35072) 2023-10-10T20:43:11.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9fe88e700 1 -- 172.21.15.89:0/3424401951 learned_addr learned my addr 172.21.15.89:0/3424401951 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:11.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9fd88c700 1 --2- 172.21.15.89:0/3424401951 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd9f80c7b60 0x7fd9f80c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:11.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9fe08d700 1 --2- 172.21.15.89:0/3424401951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9e40052e0 0x7fd9f80c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:11.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9fe88e700 1 -- 172.21.15.89:0/3424401951 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd9f80c7b60 msgr2=0x7fd9f80c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:11.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9fe88e700 1 --2- 172.21.15.89:0/3424401951 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd9f80c7b60 0x7fd9f80c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9fe88e700 1 -- 172.21.15.89:0/3424401951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9e40052e0 msgr2=0x7fd9f80c4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:11.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9fe88e700 1 --2- 172.21.15.89:0/3424401951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9e40052e0 0x7fd9f80c4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:11.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9fe88e700 1 -- 172.21.15.89:0/3424401951 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f80c5010 con 0x7fd9f80c84f0 2023-10-10T20:43:11.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9fd88c700 1 --2- 172.21.15.89:0/3424401951 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd9f80c7b60 0x7fd9f80c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:11.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9fe88e700 1 --2- 172.21.15.89:0/3424401951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9f80c84f0 0x7fd9f80c9680 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009750 tx=0x7fd9f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:11.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9ee7fc700 1 -- 172.21.15.89:0/3424401951 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f001e6f0 con 0x7fd9f80c84f0 2023-10-10T20:43:11.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9ee7fc700 1 -- 172.21.15.89:0/3424401951 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f001e890 con 0x7fd9f80c84f0 2023-10-10T20:43:11.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9ff08f700 1 -- 172.21.15.89:0/3424401951 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f80c52f0 con 0x7fd9f80c84f0 2023-10-10T20:43:11.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.542+0000 7fd9ee7fc700 1 -- 172.21.15.89:0/3424401951 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f001ec50 con 0x7fd9f80c84f0 2023-10-10T20:43:11.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.543+0000 7fd9ff08f700 1 -- 172.21.15.89:0/3424401951 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f80c58b0 con 0x7fd9f80c84f0 2023-10-10T20:43:11.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.543+0000 7fd9ff08f700 1 -- 172.21.15.89:0/3424401951 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9f8169dd0 con 0x7fd9f80c84f0 2023-10-10T20:43:11.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.544+0000 7fd9ee7fc700 1 -- 172.21.15.89:0/3424401951 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd9f001b050 con 0x7fd9f80c84f0 2023-10-10T20:43:11.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.544+0000 7fd9ee7fc700 1 --2- 172.21.15.89:0/3424401951 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd9dc067320 0x7fd9dc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:11.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.544+0000 7fd9ee7fc700 1 -- 172.21.15.89:0/3424401951 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6109+0+0 (secure 0 0 0) 0x7fd9f0086700 con 0x7fd9f80c84f0 2023-10-10T20:43:11.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.544+0000 7fd9fe08d700 1 --2- 172.21.15.89:0/3424401951 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd9dc067320 0x7fd9dc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:11.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.545+0000 7fd9fe08d700 1 --2- 172.21.15.89:0/3424401951 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd9dc067320 0x7fd9dc0697d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4000c00 tx=0x7fd9f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:11.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.549+0000 7fd9ee7fc700 1 -- 172.21.15.89:0/3424401951 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9f8169dd0 con 0x7fd9f80c84f0 2023-10-10T20:43:11.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:11.758+0000 7fd9ff08f700 1 -- 172.21.15.89:0/3424401951 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000654221", "yes_i_really_mean_it": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd9f8169ac0 con 0x7fd9dc067320 2023-10-10T20:43:12.989 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:43:12.991 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:43:13.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.026+0000 7fd9ee7fc700 1 -- 172.21.15.89:0/3424401951 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+127 (secure 0 0 0) 0x7fd9f4003550 con 0x7fd9dc067320 2023-10-10T20:43:13.028 INFO:teuthology.orchestra.run.smithi089.stdout:FS volume 'cephfs' renamed to 'volume_0000000000654221'. But failed to rename data pools as more than one data pool was found. 2023-10-10T20:43:13.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.028+0000 7fd9ff08f700 1 -- 172.21.15.89:0/3424401951 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd9dc067320 msgr2=0x7fd9dc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:13.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.028+0000 7fd9ff08f700 1 --2- 172.21.15.89:0/3424401951 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd9dc067320 0x7fd9dc0697d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4000c00 tx=0x7fd9f4009040 comp rx=0 tx=0).stop 2023-10-10T20:43:13.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.028+0000 7fd9ff08f700 1 -- 172.21.15.89:0/3424401951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9f80c84f0 msgr2=0x7fd9f80c9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:13.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.028+0000 7fd9ff08f700 1 --2- 172.21.15.89:0/3424401951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9f80c84f0 0x7fd9f80c9680 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009750 tx=0x7fd9f0017000 comp rx=0 tx=0).stop 2023-10-10T20:43:13.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.028+0000 7fd9ff08f700 1 -- 172.21.15.89:0/3424401951 shutdown_connections 2023-10-10T20:43:13.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.028+0000 7fd9ff08f700 1 --2- 172.21.15.89:0/3424401951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9f80c84f0 0x7fd9f80c9680 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.028+0000 7fd9ff08f700 1 --2- 172.21.15.89:0/3424401951 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd9dc067320 0x7fd9dc0697d0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.028+0000 7fd9ff08f700 1 --2- 172.21.15.89:0/3424401951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9e40052e0 0x7fd9f80c4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.028+0000 7fd9ff08f700 1 --2- 172.21.15.89:0/3424401951 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd9f80c7b60 0x7fd9f80c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.028+0000 7fd9ff08f700 1 -- 172.21.15.89:0/3424401951 >> 172.21.15.89:0/3424401951 conn(0x7fd9f8063230 msgr2=0x7fd9f80d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:13.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.028+0000 7fd9ff08f700 1 -- 172.21.15.89:0/3424401951 shutdown_connections 2023-10-10T20:43:13.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.028+0000 7fd9ff08f700 1 -- 172.21.15.89:0/3424401951 wait complete. 2023-10-10T20:43:13.043 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-10T20:43:13.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.208+0000 7f8506a8a700 1 Processor -- start 2023-10-10T20:43:13.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.208+0000 7f8506a8a700 1 -- start start 2023-10-10T20:43:13.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.209+0000 7f8506a8a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f850006c0a0 0x7f85000caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:13.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.209+0000 7f8506a8a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f850015df50 0x7f850015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:13.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.209+0000 7f8506a8a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f850015d7e0 con 0x7f850006bcd0 2023-10-10T20:43:13.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.209+0000 7f8506a8a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f850015dac0 con 0x7f850015df50 2023-10-10T20:43:13.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.209+0000 7f8506a8a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f850015aa30 con 0x7f850006c0a0 2023-10-10T20:43:13.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.209+0000 7f8505027700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f850015df50 0x7f850015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.209+0000 7f8504826700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f850006bcd0 0x7f850015a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37690/0 (socket says 172.21.15.89:37690) 2023-10-10T20:43:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.209+0000 7f8504826700 1 -- 172.21.15.89:0/2602975055 learned_addr learned my addr 172.21.15.89:0/2602975055 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.209+0000 7f84f7fff700 1 --2- 172.21.15.89:0/2602975055 >> v2:172.21.15.111:3300/0 conn(0x7f850006c0a0 0x7f85000caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.210+0000 7f8505027700 1 -- 172.21.15.89:0/2602975055 >> v2:172.21.15.111:3300/0 conn(0x7f850006c0a0 msgr2=0x7f85000caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.210+0000 7f8505027700 1 --2- 172.21.15.89:0/2602975055 >> v2:172.21.15.111:3300/0 conn(0x7f850006c0a0 0x7f85000caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.210+0000 7f8505027700 1 -- 172.21.15.89:0/2602975055 >> v1:172.21.15.89:6789/0 conn(0x7f850006bcd0 legacy=0x7f850015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.210+0000 7f8505027700 1 -- 172.21.15.89:0/2602975055 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850015ad10 con 0x7f850015df50 2023-10-10T20:43:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.210+0000 7f84f77fe700 1 -- 172.21.15.89:0/2602975055 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589135922 0 0) 0x7f850015d7e0 con 0x7f850006bcd0 2023-10-10T20:43:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.210+0000 7f8505027700 1 --2- 172.21.15.89:0/2602975055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f850015df50 0x7f850015e3b0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f850006b4e0 tx=0x7f84f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c86d5199e08018f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.210+0000 7f84f77fe700 1 -- 172.21.15.89:0/2602975055 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f800b6b0 con 0x7f850015df50 2023-10-10T20:43:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.210+0000 7f84f77fe700 1 -- 172.21.15.89:0/2602975055 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f800b810 con 0x7f850015df50 2023-10-10T20:43:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.210+0000 7f84f77fe700 1 -- 172.21.15.89:0/2602975055 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f800bbb0 con 0x7f850015df50 2023-10-10T20:43:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.210+0000 7f8506a8a700 1 -- 172.21.15.89:0/2602975055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f850015df50 msgr2=0x7f850015e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.210+0000 7f8506a8a700 1 --2- 172.21.15.89:0/2602975055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f850015df50 0x7f850015e3b0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f850006b4e0 tx=0x7f84f80091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.210+0000 7f8506a8a700 1 -- 172.21.15.89:0/2602975055 shutdown_connections 2023-10-10T20:43:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.210+0000 7f8506a8a700 1 --2- 172.21.15.89:0/2602975055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f850015df50 0x7f850015e3b0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.210+0000 7f8506a8a700 1 --2- 172.21.15.89:0/2602975055 >> v2:172.21.15.111:3300/0 conn(0x7f850006c0a0 0x7f85000caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.210+0000 7f8506a8a700 1 -- 172.21.15.89:0/2602975055 >> 172.21.15.89:0/2602975055 conn(0x7f85000ce600 msgr2=0x7f85000cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.210+0000 7f8506a8a700 1 -- 172.21.15.89:0/2602975055 shutdown_connections 2023-10-10T20:43:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.210+0000 7f8506a8a700 1 -- 172.21.15.89:0/2602975055 wait complete. 2023-10-10T20:43:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.211+0000 7f8506a8a700 1 Processor -- start 2023-10-10T20:43:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.212+0000 7f8506a8a700 1 -- start start 2023-10-10T20:43:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.212+0000 7f8506a8a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f850006bcd0 0x7f85000b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.212+0000 7f8506a8a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f850006c0a0 0x7f8500157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.212+0000 7f8506a8a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f850015df50 0x7f850016a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.212+0000 7f8506a8a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8500159230 con 0x7f850015df50 2023-10-10T20:43:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.212+0000 7f8506a8a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85001596c0 con 0x7f850006bcd0 2023-10-10T20:43:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.212+0000 7f8506a8a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8500158620 con 0x7f850006c0a0 2023-10-10T20:43:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.213+0000 7f8505027700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f850015df50 0x7f850016a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.213+0000 7f8505027700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f850015df50 0x7f850016a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35086/0 (socket says 172.21.15.89:35086) 2023-10-10T20:43:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.213+0000 7f8505027700 1 -- 172.21.15.89:0/3676346810 learned_addr learned my addr 172.21.15.89:0/3676346810 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.213+0000 7f8504826700 1 --2- 172.21.15.89:0/3676346810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f850006bcd0 0x7f85000b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.213+0000 7f84f7fff700 1 --2- 172.21.15.89:0/3676346810 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f850006c0a0 0x7f8500157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.213+0000 7f8505027700 1 -- 172.21.15.89:0/3676346810 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f850006c0a0 msgr2=0x7f8500157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.213+0000 7f8505027700 1 --2- 172.21.15.89:0/3676346810 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f850006c0a0 0x7f8500157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.213+0000 7f8505027700 1 -- 172.21.15.89:0/3676346810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f850006bcd0 msgr2=0x7f85000b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.213+0000 7f8505027700 1 --2- 172.21.15.89:0/3676346810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f850006bcd0 0x7f85000b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.213+0000 7f8505027700 1 -- 172.21.15.89:0/3676346810 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f8009420 con 0x7f850015df50 2023-10-10T20:43:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.213+0000 7f8505027700 1 --2- 172.21.15.89:0/3676346810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f850015df50 0x7f850016a390 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f8500069220 tx=0x7f84f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.213+0000 7f84f4ff9700 1 -- 172.21.15.89:0/3676346810 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8011420 con 0x7f850015df50 2023-10-10T20:43:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.213+0000 7f84f4ff9700 1 -- 172.21.15.89:0/3676346810 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f80115e0 con 0x7f850015df50 2023-10-10T20:43:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.213+0000 7f84f4ff9700 1 -- 172.21.15.89:0/3676346810 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f80119a0 con 0x7f850015df50 2023-10-10T20:43:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.213+0000 7f8506a8a700 1 -- 172.21.15.89:0/3676346810 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8500158900 con 0x7f850015df50 2023-10-10T20:43:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.213+0000 7f8506a8a700 1 -- 172.21.15.89:0/3676346810 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8500158ec0 con 0x7f850015df50 2023-10-10T20:43:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.214+0000 7f8506a8a700 1 -- 172.21.15.89:0/3676346810 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f850004c3c0 con 0x7f850015df50 2023-10-10T20:43:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.214+0000 7f84f4ff9700 1 -- 172.21.15.89:0/3676346810 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f84f80253f0 con 0x7f850015df50 2023-10-10T20:43:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.215+0000 7f84f4ff9700 1 --2- 172.21.15.89:0/3676346810 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84ec067320 0x7f84ec0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.215+0000 7f8504826700 1 --2- 172.21.15.89:0/3676346810 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84ec067320 0x7f84ec0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.215+0000 7f84f4ff9700 1 -- 172.21.15.89:0/3676346810 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f84f8086860 con 0x7f850015df50 2023-10-10T20:43:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.215+0000 7f8504826700 1 --2- 172.21.15.89:0/3676346810 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84ec067320 0x7f84ec0697d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f84fc007240 tx=0x7f84fc001970 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.219+0000 7f84f4ff9700 1 -- 172.21.15.89:0/3676346810 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84f8053310 con 0x7f850015df50 2023-10-10T20:43:13.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.428+0000 7f8506a8a700 1 -- 172.21.15.89:0/3676346810 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f850015aff0 con 0x7f84ec067320 2023-10-10T20:43:13.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.429+0000 7f84f4ff9700 1 -- 172.21.15.89:0/3676346810 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+58 (secure 0 0 0) 0x7f850015aff0 con 0x7f84ec067320 2023-10-10T20:43:13.431 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:43:13.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:13.431 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "volume_0000000000654221" 2023-10-10T20:43:13.432 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:13.432 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:43:13.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.431+0000 7f8506a8a700 1 -- 172.21.15.89:0/3676346810 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84ec067320 msgr2=0x7f84ec0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:13.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.431+0000 7f8506a8a700 1 --2- 172.21.15.89:0/3676346810 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84ec067320 0x7f84ec0697d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f84fc007240 tx=0x7f84fc001970 comp rx=0 tx=0).stop 2023-10-10T20:43:13.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.431+0000 7f8506a8a700 1 -- 172.21.15.89:0/3676346810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f850015df50 msgr2=0x7f850016a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:13.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.431+0000 7f8506a8a700 1 --2- 172.21.15.89:0/3676346810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f850015df50 0x7f850016a390 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f8500069220 tx=0x7f84f8017000 comp rx=0 tx=0).stop 2023-10-10T20:43:13.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.431+0000 7f8506a8a700 1 -- 172.21.15.89:0/3676346810 shutdown_connections 2023-10-10T20:43:13.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.431+0000 7f8506a8a700 1 --2- 172.21.15.89:0/3676346810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f850015df50 0x7f850016a390 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.431+0000 7f8506a8a700 1 --2- 172.21.15.89:0/3676346810 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84ec067320 0x7f84ec0697d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.431+0000 7f8506a8a700 1 --2- 172.21.15.89:0/3676346810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f850006bcd0 0x7f85000b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.431+0000 7f8506a8a700 1 --2- 172.21.15.89:0/3676346810 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f850006c0a0 0x7f8500157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.431+0000 7f8506a8a700 1 -- 172.21.15.89:0/3676346810 >> 172.21.15.89:0/3676346810 conn(0x7f85000ce600 msgr2=0x7f85000c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:13.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.431+0000 7f8506a8a700 1 -- 172.21.15.89:0/3676346810 shutdown_connections 2023-10-10T20:43:13.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.431+0000 7f8506a8a700 1 -- 172.21.15.89:0/3676346810 wait complete. 2023-10-10T20:43:13.445 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:43:13.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.557+0000 7f2599709700 1 Processor -- start 2023-10-10T20:43:13.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.557+0000 7f2599709700 1 -- start start 2023-10-10T20:43:13.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.558+0000 7f2599709700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f25940cd1e0 0x7f25940c9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:13.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.558+0000 7f2599709700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259415c2a0 con 0x7f259406bcd0 2023-10-10T20:43:13.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.558+0000 7f2599709700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259415b690 con 0x7f2594141500 2023-10-10T20:43:13.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.558+0000 7f2599709700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f259415b970 con 0x7f25940cd1e0 2023-10-10T20:43:13.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.558+0000 7f2592ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f259406bcd0 0x7f2594140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37700/0 (socket says 172.21.15.89:37700) 2023-10-10T20:43:13.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.558+0000 7f2592ffd700 1 -- 172.21.15.89:0/3436113743 learned_addr learned my addr 172.21.15.89:0/3436113743 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:13.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f2591ffb700 1 -- 172.21.15.89:0/3436113743 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746721662 0 0) 0x7f259415c2a0 con 0x7f259406bcd0 2023-10-10T20:43:13.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f25937fe700 1 --2- 172.21.15.89:0/3436113743 >> v2:172.21.15.111:3300/0 conn(0x7f25940cd1e0 0x7f25940c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:13.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f2591ffb700 1 -- 172.21.15.89:0/3436113743 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f257c003570 con 0x7f259406bcd0 2023-10-10T20:43:13.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f2591ffb700 1 -- 172.21.15.89:0/3436113743 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3113165140 0 0) 0x7f257c003570 con 0x7f259406bcd0 2023-10-10T20:43:13.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f2591ffb700 1 -- 172.21.15.89:0/3436113743 >> v2:172.21.15.111:3300/0 conn(0x7f25940cd1e0 msgr2=0x7f25940c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:13.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f2591ffb700 1 --2- 172.21.15.89:0/3436113743 >> v2:172.21.15.111:3300/0 conn(0x7f25940cd1e0 0x7f25940c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f2591ffb700 1 -- 172.21.15.89:0/3436113743 >> v1:172.21.15.111:6789/0 conn(0x7f2594141500 legacy=0x7f25940c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:13.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f2591ffb700 1 -- 172.21.15.89:0/3436113743 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259415bc50 con 0x7f259406bcd0 2023-10-10T20:43:13.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f2591ffb700 1 -- 172.21.15.89:0/3436113743 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f25840017f0 con 0x7f259406bcd0 2023-10-10T20:43:13.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f2591ffb700 1 -- 172.21.15.89:0/3436113743 >> v1:172.21.15.89:6789/0 conn(0x7f259406bcd0 legacy=0x7f2594140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:13.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f2591ffb700 1 --2- 172.21.15.89:0/3436113743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f257c003a60 0x7f257c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:13.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f2591ffb700 1 --2- 172.21.15.89:0/3436113743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f257c004810 0x7f257c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:13.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f2591ffb700 1 --2- 172.21.15.89:0/3436113743 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f257c0051c0 0x7f257c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:13.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f2591ffb700 1 -- 172.21.15.89:0/3436113743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259415b970 con 0x7f257c004810 2023-10-10T20:43:13.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f2591ffb700 1 -- 172.21.15.89:0/3436113743 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c00c0d0 con 0x7f257c003a60 2023-10-10T20:43:13.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f2591ffb700 1 -- 172.21.15.89:0/3436113743 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257c00c290 con 0x7f257c0051c0 2023-10-10T20:43:13.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f25927fc700 1 --2- 172.21.15.89:0/3436113743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f257c004810 0x7f257c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:13.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f2592ffd700 1 --2- 172.21.15.89:0/3436113743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f257c003a60 0x7f257c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:13.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f25937fe700 1 --2- 172.21.15.89:0/3436113743 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f257c0051c0 0x7f257c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:13.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f25927fc700 1 -- 172.21.15.89:0/3436113743 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f257c0051c0 msgr2=0x7f257c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:13.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f25927fc700 1 --2- 172.21.15.89:0/3436113743 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f257c0051c0 0x7f257c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f25927fc700 1 -- 172.21.15.89:0/3436113743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f257c003a60 msgr2=0x7f257c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:13.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f25927fc700 1 --2- 172.21.15.89:0/3436113743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f257c003a60 0x7f257c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f25927fc700 1 -- 172.21.15.89:0/3436113743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c003570 con 0x7f257c004810 2023-10-10T20:43:13.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f25937fe700 1 --2- 172.21.15.89:0/3436113743 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f257c0051c0 0x7f257c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:13.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.559+0000 7f25937fe700 1 -- 172.21.15.89:0/3436113743 reap_dead start 2023-10-10T20:43:13.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.560+0000 7f25927fc700 1 --2- 172.21.15.89:0/3436113743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f257c004810 0x7f257c004c80 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f25880018b0 tx=0x7f2588007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40dee85c515787bb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:13.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.560+0000 7f2591ffb700 1 -- 172.21.15.89:0/3436113743 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588012420 con 0x7f257c004810 2023-10-10T20:43:13.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.560+0000 7f2591ffb700 1 -- 172.21.15.89:0/3436113743 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25880125a0 con 0x7f257c004810 2023-10-10T20:43:13.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.560+0000 7f2591ffb700 1 -- 172.21.15.89:0/3436113743 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588012960 con 0x7f257c004810 2023-10-10T20:43:13.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.560+0000 7f2599709700 1 -- 172.21.15.89:0/3436113743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f257c004810 msgr2=0x7f257c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:13.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.560+0000 7f2599709700 1 --2- 172.21.15.89:0/3436113743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f257c004810 0x7f257c004c80 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f25880018b0 tx=0x7f2588007d30 comp rx=0 tx=0).stop 2023-10-10T20:43:13.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.560+0000 7f2599709700 1 -- 172.21.15.89:0/3436113743 shutdown_connections 2023-10-10T20:43:13.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.560+0000 7f2599709700 1 --2- 172.21.15.89:0/3436113743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f257c004810 0x7f257c004c80 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.560+0000 7f2599709700 1 -- 172.21.15.89:0/3436113743 >> 172.21.15.89:0/3436113743 conn(0x7f25940ce600 msgr2=0x7f25940cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:13.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.560+0000 7f2599709700 1 -- 172.21.15.89:0/3436113743 shutdown_connections 2023-10-10T20:43:13.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.560+0000 7f2599709700 1 -- 172.21.15.89:0/3436113743 wait complete. 2023-10-10T20:43:13.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.561+0000 7f2599709700 1 Processor -- start 2023-10-10T20:43:13.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.562+0000 7f2599709700 1 -- start start 2023-10-10T20:43:13.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.562+0000 7f2599709700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f257c004810 0x7f25940c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.562+0000 7f2599709700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f259413e9c0 0x7f25940b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.562+0000 7f2599709700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f259413e100 0x7f25940a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.562+0000 7f2599709700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25940c8560 con 0x7f257c004810 2023-10-10T20:43:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.562+0000 7f2599709700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2594168aa0 con 0x7f259413e100 2023-10-10T20:43:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.562+0000 7f2599709700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25940c65b0 con 0x7f259413e9c0 2023-10-10T20:43:13.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.562+0000 7f2592ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f257c004810 0x7f25940c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:13.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.562+0000 7f2592ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f257c004810 0x7f25940c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35106/0 (socket says 172.21.15.89:35106) 2023-10-10T20:43:13.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.562+0000 7f2592ffd700 1 -- 172.21.15.89:0/3889163179 learned_addr learned my addr 172.21.15.89:0/3889163179 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:13.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.563+0000 7f2592ffd700 1 -- 172.21.15.89:0/3889163179 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f259413e9c0 msgr2=0x7f25940b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:13.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.563+0000 7f2592ffd700 1 --2- 172.21.15.89:0/3889163179 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f259413e9c0 0x7f25940b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.563+0000 7f2592ffd700 1 -- 172.21.15.89:0/3889163179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f259413e100 msgr2=0x7f25940a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:13.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.563+0000 7f25937fe700 1 --2- 172.21.15.89:0/3889163179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f259413e100 0x7f25940a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:13.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.563+0000 7f2592ffd700 1 --2- 172.21.15.89:0/3889163179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f259413e100 0x7f25940a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.563+0000 7f2592ffd700 1 -- 172.21.15.89:0/3889163179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25940b28b0 con 0x7f257c004810 2023-10-10T20:43:13.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.563+0000 7f2592ffd700 1 --2- 172.21.15.89:0/3889163179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f257c004810 0x7f25940c8020 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f2584007a90 tx=0x7f259415c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:13.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.563+0000 7f257affd700 1 -- 172.21.15.89:0/3889163179 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258400ad70 con 0x7f257c004810 2023-10-10T20:43:13.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.563+0000 7f257affd700 1 -- 172.21.15.89:0/3889163179 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25840133f0 con 0x7f257c004810 2023-10-10T20:43:13.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.563+0000 7f2599709700 1 -- 172.21.15.89:0/3889163179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2594168540 con 0x7f257c004810 2023-10-10T20:43:13.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.563+0000 7f257affd700 1 -- 172.21.15.89:0/3889163179 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25840137b0 con 0x7f257c004810 2023-10-10T20:43:13.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.563+0000 7f2599709700 1 -- 172.21.15.89:0/3889163179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25940c5f60 con 0x7f257c004810 2023-10-10T20:43:13.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.564+0000 7f2599709700 1 -- 172.21.15.89:0/3889163179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f259404c3c0 con 0x7f257c004810 2023-10-10T20:43:13.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.564+0000 7f257affd700 1 -- 172.21.15.89:0/3889163179 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2584013ba0 con 0x7f257c004810 2023-10-10T20:43:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.565+0000 7f257affd700 1 --2- 172.21.15.89:0/3889163179 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2580067320 0x7f25800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.565+0000 7f257affd700 1 -- 172.21.15.89:0/3889163179 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f2584085c70 con 0x7f257c004810 2023-10-10T20:43:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.565+0000 7f25927fc700 1 --2- 172.21.15.89:0/3889163179 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2580067320 0x7f25800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.565+0000 7f25927fc700 1 --2- 172.21.15.89:0/3889163179 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2580067320 0x7f25800697d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2594157ee0 tx=0x7f2588015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:13.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.569+0000 7f257affd700 1 -- 172.21.15.89:0/3889163179 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2584052720 con 0x7f257c004810 2023-10-10T20:43:13.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.797+0000 7f2599709700 1 -- 172.21.15.89:0/3889163179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f259415bf30 con 0x7f257c004810 2023-10-10T20:43:13.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.797+0000 7f257affd700 1 -- 172.21.15.89:0/3889163179 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4633 (secure 0 0 0) 0x7f2584017090 con 0x7f257c004810 2023-10-10T20:43:13.800 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:13.800 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":113,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6638,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2350308973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2350308973},{"type":"v1","addr":"172.21.15.111:6835","nonce":2350308973}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":8322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/1051252386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":1051252386},{"type":"v1","addr":"172.21.15.89:6836","nonce":1051252386}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":8325,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3754994248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":3754994248},{"type":"v1","addr":"172.21.15.89:6837","nonce":3754994248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:42:52.707112+0000","modified":"2023-10-10T20:43:12.015439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8337":{"gid":8337,"name":"b","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1733913702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1733913702},{"type":"v1","addr":"172.21.15.111:6834","nonce":1733913702}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"volume_0000000000654221","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-10T20:43:13.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.799+0000 7f2599709700 1 -- 172.21.15.89:0/3889163179 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2580067320 msgr2=0x7f25800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:13.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.799+0000 7f2599709700 1 --2- 172.21.15.89:0/3889163179 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2580067320 0x7f25800697d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2594157ee0 tx=0x7f2588015000 comp rx=0 tx=0).stop 2023-10-10T20:43:13.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.799+0000 7f2599709700 1 -- 172.21.15.89:0/3889163179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f257c004810 msgr2=0x7f25940c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:13.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.799+0000 7f2599709700 1 --2- 172.21.15.89:0/3889163179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f257c004810 0x7f25940c8020 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f2584007a90 tx=0x7f259415c210 comp rx=0 tx=0).stop 2023-10-10T20:43:13.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.800+0000 7f2599709700 1 -- 172.21.15.89:0/3889163179 shutdown_connections 2023-10-10T20:43:13.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.800+0000 7f2599709700 1 --2- 172.21.15.89:0/3889163179 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2580067320 0x7f25800697d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.800+0000 7f2599709700 1 --2- 172.21.15.89:0/3889163179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f259413e100 0x7f25940a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.800+0000 7f2599709700 1 --2- 172.21.15.89:0/3889163179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f257c004810 0x7f25940c8020 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.800+0000 7f2599709700 1 --2- 172.21.15.89:0/3889163179 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f259413e9c0 0x7f25940b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.800+0000 7f2599709700 1 -- 172.21.15.89:0/3889163179 >> 172.21.15.89:0/3889163179 conn(0x7f25940ce600 msgr2=0x7f25940c3640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:13.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.800+0000 7f2599709700 1 -- 172.21.15.89:0/3889163179 shutdown_connections 2023-10-10T20:43:13.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.800+0000 7f2599709700 1 -- 172.21.15.89:0/3889163179 wait complete. 2023-10-10T20:43:13.804 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 113 2023-10-10T20:43:13.814 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:43:13.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.927+0000 7f6b6f112700 1 Processor -- start 2023-10-10T20:43:13.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.927+0000 7f6b6f112700 1 -- start start 2023-10-10T20:43:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.928+0000 7f6b6f112700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6b68159c60 0x7f6b6815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.928+0000 7f6b6f112700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6815a160 con 0x7f6b68159890 2023-10-10T20:43:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.928+0000 7f6b6f112700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b680d13a0 con 0x7f6b6815db10 2023-10-10T20:43:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.928+0000 7f6b6f112700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b6815df50 con 0x7f6b68159c60 2023-10-10T20:43:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.928+0000 7f6b5ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6b68159c60 0x7f6b6815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.928+0000 7f6b6ceae700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6b68159890 0x7f6b6815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37710/0 (socket says 172.21.15.89:37710) 2023-10-10T20:43:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.928+0000 7f6b6ceae700 1 -- 172.21.15.89:0/3052277161 learned_addr learned my addr 172.21.15.89:0/3052277161 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.928+0000 7f6b5ffff700 1 -- 172.21.15.89:0/3052277161 >> v1:172.21.15.111:6789/0 conn(0x7f6b6815db10 legacy=0x7f6b68069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.928+0000 7f6b5ffff700 1 -- 172.21.15.89:0/3052277161 >> v1:172.21.15.89:6789/0 conn(0x7f6b68159890 legacy=0x7f6b6815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.929+0000 7f6b5ffff700 1 -- 172.21.15.89:0/3052277161 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6815e140 con 0x7f6b68159c60 2023-10-10T20:43:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.929+0000 7f6b5ffff700 1 --2- 172.21.15.89:0/3052277161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b68159c60 0x7f6b6815d5d0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f6b580098e0 tx=0x7f6b580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0842c8ea917caa server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.929+0000 7f6b5f7fe700 1 -- 172.21.15.89:0/3052277161 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58009e80 con 0x7f6b68159c60 2023-10-10T20:43:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.929+0000 7f6b5f7fe700 1 -- 172.21.15.89:0/3052277161 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5800b820 con 0x7f6b68159c60 2023-10-10T20:43:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.929+0000 7f6b5f7fe700 1 -- 172.21.15.89:0/3052277161 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800bba0 con 0x7f6b68159c60 2023-10-10T20:43:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.929+0000 7f6b6f112700 1 -- 172.21.15.89:0/3052277161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b68159c60 msgr2=0x7f6b6815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:13.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.929+0000 7f6b6f112700 1 --2- 172.21.15.89:0/3052277161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b68159c60 0x7f6b6815d5d0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f6b580098e0 tx=0x7f6b580091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.929+0000 7f6b6f112700 1 -- 172.21.15.89:0/3052277161 shutdown_connections 2023-10-10T20:43:13.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.929+0000 7f6b6f112700 1 --2- 172.21.15.89:0/3052277161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b68159c60 0x7f6b6815d5d0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.929+0000 7f6b6f112700 1 -- 172.21.15.89:0/3052277161 >> 172.21.15.89:0/3052277161 conn(0x7f6b68068e20 msgr2=0x7f6b680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:13.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.929+0000 7f6b6f112700 1 -- 172.21.15.89:0/3052277161 shutdown_connections 2023-10-10T20:43:13.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.929+0000 7f6b6f112700 1 -- 172.21.15.89:0/3052277161 wait complete. 2023-10-10T20:43:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.930+0000 7f6b6f112700 1 Processor -- start 2023-10-10T20:43:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.931+0000 7f6b6f112700 1 -- start start 2023-10-10T20:43:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.931+0000 7f6b6f112700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b68159890 0x7f6b681410f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.931+0000 7f6b6f112700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b68159c60 0x7f6b681544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.931+0000 7f6b6f112700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6b6815db10 0x7f6b68154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.931+0000 7f6b6f112700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6816a580 con 0x7f6b68159890 2023-10-10T20:43:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.931+0000 7f6b6f112700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b680b6310 con 0x7f6b68159c60 2023-10-10T20:43:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.931+0000 7f6b6f112700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b68169f30 con 0x7f6b6815db10 2023-10-10T20:43:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.931+0000 7f6b6ceae700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b68159890 0x7f6b681410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.931+0000 7f6b6ceae700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b68159890 0x7f6b681410f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35118/0 (socket says 172.21.15.89:35118) 2023-10-10T20:43:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.931+0000 7f6b6ceae700 1 -- 172.21.15.89:0/235993612 learned_addr learned my addr 172.21.15.89:0/235993612 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.932+0000 7f6b6ceae700 1 -- 172.21.15.89:0/235993612 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6b6815db10 msgr2=0x7f6b68154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.932+0000 7f6b5ffff700 1 --2- 172.21.15.89:0/235993612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b68159c60 0x7f6b681544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.932+0000 7f6b6d6af700 1 --2- 172.21.15.89:0/235993612 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6b6815db10 0x7f6b68154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.932+0000 7f6b6ceae700 1 --2- 172.21.15.89:0/235993612 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6b6815db10 0x7f6b68154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.932+0000 7f6b6ceae700 1 -- 172.21.15.89:0/235993612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b68159c60 msgr2=0x7f6b681544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.932+0000 7f6b6ceae700 1 --2- 172.21.15.89:0/235993612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b68159c60 0x7f6b681544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.932+0000 7f6b6ceae700 1 -- 172.21.15.89:0/235993612 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b58009420 con 0x7f6b68159890 2023-10-10T20:43:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.932+0000 7f6b6d6af700 1 --2- 172.21.15.89:0/235993612 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6b6815db10 0x7f6b68154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.932+0000 7f6b5ffff700 1 --2- 172.21.15.89:0/235993612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b68159c60 0x7f6b681544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.932+0000 7f6b6ceae700 1 --2- 172.21.15.89:0/235993612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b68159890 0x7f6b681410f0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400cc70 tx=0x7f6b64001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.932+0000 7f6b5cff9700 1 -- 172.21.15.89:0/235993612 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6400ce80 con 0x7f6b68159890 2023-10-10T20:43:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.932+0000 7f6b5cff9700 1 -- 172.21.15.89:0/235993612 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6400ab70 con 0x7f6b68159890 2023-10-10T20:43:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.932+0000 7f6b5cff9700 1 -- 172.21.15.89:0/235993612 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b640070f0 con 0x7f6b68159890 2023-10-10T20:43:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.932+0000 7f6b6f112700 1 -- 172.21.15.89:0/235993612 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6816a210 con 0x7f6b68159890 2023-10-10T20:43:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.932+0000 7f6b6f112700 1 -- 172.21.15.89:0/235993612 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b68141630 con 0x7f6b68159890 2023-10-10T20:43:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.933+0000 7f6b6f112700 1 -- 172.21.15.89:0/235993612 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6815d950 con 0x7f6b68159890 2023-10-10T20:43:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.933+0000 7f6b5cff9700 1 -- 172.21.15.89:0/235993612 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6b64007520 con 0x7f6b68159890 2023-10-10T20:43:13.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.934+0000 7f6b5cff9700 1 --2- 172.21.15.89:0/235993612 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6b540673a0 0x7f6b54069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:13.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.934+0000 7f6b5cff9700 1 -- 172.21.15.89:0/235993612 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f6b64085260 con 0x7f6b68159890 2023-10-10T20:43:13.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.934+0000 7f6b5ffff700 1 --2- 172.21.15.89:0/235993612 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6b540673a0 0x7f6b54069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:13.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.934+0000 7f6b5ffff700 1 --2- 172.21.15.89:0/235993612 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6b540673a0 0x7f6b54069850 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6b680cde40 tx=0x7f6b58016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:13.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:13.938+0000 7f6b5cff9700 1 -- 172.21.15.89:0/235993612 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b64051d50 con 0x7f6b68159890 2023-10-10T20:43:14.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.123+0000 7f6b6f112700 1 -- 172.21.15.89:0/235993612 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b6804c3c0 con 0x7f6b68159890 2023-10-10T20:43:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.124+0000 7f6b5cff9700 1 -- 172.21.15.89:0/235993612 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+17005 (secure 0 0 0) 0x7f6b64055370 con 0x7f6b68159890 2023-10-10T20:43:14.126 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:14.126 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":170,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:43:13.020125+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs.volume_0000000000654221.meta","create_time":"2023-10-10T20:42:51.066663+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000654221"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-10T20:42:51.675525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000654221"}}},{"pool":30,"pool_name":"another-data-pool","create_time":"2023-10-10T20:43:08.234534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000000654221"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:43:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.126+0000 7f6b6f112700 1 -- 172.21.15.89:0/235993612 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6b540673a0 msgr2=0x7f6b54069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.126+0000 7f6b6f112700 1 --2- 172.21.15.89:0/235993612 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6b540673a0 0x7f6b54069850 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6b680cde40 tx=0x7f6b58016000 comp rx=0 tx=0).stop 2023-10-10T20:43:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.126+0000 7f6b6f112700 1 -- 172.21.15.89:0/235993612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b68159890 msgr2=0x7f6b681410f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.126+0000 7f6b6f112700 1 --2- 172.21.15.89:0/235993612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b68159890 0x7f6b681410f0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400cc70 tx=0x7f6b64001a30 comp rx=0 tx=0).stop 2023-10-10T20:43:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.126+0000 7f6b6f112700 1 -- 172.21.15.89:0/235993612 shutdown_connections 2023-10-10T20:43:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.126+0000 7f6b6f112700 1 --2- 172.21.15.89:0/235993612 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6b6815db10 0x7f6b68154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.126+0000 7f6b6f112700 1 --2- 172.21.15.89:0/235993612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b68159890 0x7f6b681410f0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.126+0000 7f6b6f112700 1 --2- 172.21.15.89:0/235993612 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6b540673a0 0x7f6b54069850 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.126+0000 7f6b6f112700 1 --2- 172.21.15.89:0/235993612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b68159c60 0x7f6b681544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.126+0000 7f6b6f112700 1 -- 172.21.15.89:0/235993612 >> 172.21.15.89:0/235993612 conn(0x7f6b68068e20 msgr2=0x7f6b68155310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.126+0000 7f6b6f112700 1 -- 172.21.15.89:0/235993612 shutdown_connections 2023-10-10T20:43:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.126+0000 7f6b6f112700 1 -- 172.21.15.89:0/235993612 wait complete. 2023-10-10T20:43:14.140 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:43:14.141 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:43:14.185 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:43:14.229 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:43:14.229 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:43:14.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.342+0000 7f06e138d700 1 Processor -- start 2023-10-10T20:43:14.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.342+0000 7f06e138d700 1 -- start start 2023-10-10T20:43:14.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.343+0000 7f06e138d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f06dc159610 0x7f06dc06e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:14.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.343+0000 7f06e138d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f06dc1599e0 0x7f06dc06af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:14.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.343+0000 7f06e138d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06dc0c5a60 0x7f06dc0c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:14.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.343+0000 7f06e138d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06dc0c6420 con 0x7f06dc1599e0 2023-10-10T20:43:14.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.343+0000 7f06e138d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06dc06c470 con 0x7f06dc159610 2023-10-10T20:43:14.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.343+0000 7f06e138d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc06b860 con 0x7f06dc0c5a60 2023-10-10T20:43:14.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.343+0000 7f06da7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f06dc1599e0 0x7f06dc06af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:14.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.343+0000 7f06da7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f06dc1599e0 0x7f06dc06af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35120/0 (socket says 172.21.15.89:35120) 2023-10-10T20:43:14.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.343+0000 7f06da7fc700 1 -- 172.21.15.89:0/2703436180 learned_addr learned my addr 172.21.15.89:0/2703436180 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:14.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.343+0000 7f06da7fc700 1 -- 172.21.15.89:0/2703436180 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06dc0c5a60 msgr2=0x7f06dc0c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:14.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.343+0000 7f06da7fc700 1 --2- 172.21.15.89:0/2703436180 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06dc0c5a60 0x7f06dc0c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.343+0000 7f06da7fc700 1 -- 172.21.15.89:0/2703436180 >> v2:172.21.15.111:3300/0 conn(0x7f06dc159610 msgr2=0x7f06dc06e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:14.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.343+0000 7f06da7fc700 1 --2- 172.21.15.89:0/2703436180 >> v2:172.21.15.111:3300/0 conn(0x7f06dc159610 0x7f06dc06e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.343+0000 7f06da7fc700 1 -- 172.21.15.89:0/2703436180 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06dc06bb40 con 0x7f06dc1599e0 2023-10-10T20:43:14.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.344+0000 7f06da7fc700 1 --2- 172.21.15.89:0/2703436180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06dc1599e0 0x7f06dc06af80 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f06dc06d380 tx=0x7f06d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192b897d0df1109a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:14.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.344+0000 7f06d9ffb700 1 -- 172.21.15.89:0/2703436180 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d000b820 con 0x7f06dc1599e0 2023-10-10T20:43:14.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.344+0000 7f06d9ffb700 1 -- 172.21.15.89:0/2703436180 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d000b980 con 0x7f06dc1599e0 2023-10-10T20:43:14.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.344+0000 7f06d9ffb700 1 -- 172.21.15.89:0/2703436180 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d000bd20 con 0x7f06dc1599e0 2023-10-10T20:43:14.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.344+0000 7f06e138d700 1 -- 172.21.15.89:0/2703436180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06dc1599e0 msgr2=0x7f06dc06af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:14.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.344+0000 7f06e138d700 1 --2- 172.21.15.89:0/2703436180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06dc1599e0 0x7f06dc06af80 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f06dc06d380 tx=0x7f06d0009250 comp rx=0 tx=0).stop 2023-10-10T20:43:14.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.344+0000 7f06e138d700 1 -- 172.21.15.89:0/2703436180 shutdown_connections 2023-10-10T20:43:14.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.344+0000 7f06e138d700 1 --2- 172.21.15.89:0/2703436180 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06dc0c5a60 0x7f06dc0c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.344+0000 7f06e138d700 1 --2- 172.21.15.89:0/2703436180 >> v2:172.21.15.111:3300/0 conn(0x7f06dc159610 0x7f06dc06e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.344+0000 7f06e138d700 1 --2- 172.21.15.89:0/2703436180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06dc1599e0 0x7f06dc06af80 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.344+0000 7f06e138d700 1 -- 172.21.15.89:0/2703436180 >> 172.21.15.89:0/2703436180 conn(0x7f06dc0cc780 msgr2=0x7f06dc0ccb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:14.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.344+0000 7f06e138d700 1 -- 172.21.15.89:0/2703436180 shutdown_connections 2023-10-10T20:43:14.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.344+0000 7f06e138d700 1 -- 172.21.15.89:0/2703436180 wait complete. 2023-10-10T20:43:14.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.346+0000 7f06e138d700 1 Processor -- start 2023-10-10T20:43:14.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.346+0000 7f06e138d700 1 -- start start 2023-10-10T20:43:14.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.346+0000 7f06e138d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06dc0c5a60 0x7f06dc15d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:14.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.346+0000 7f06e138d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06dc159610 0x7f06dc0ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:14.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.346+0000 7f06e138d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06dc1599e0 0x7f06dc0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:14.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.346+0000 7f06e138d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc15e3d0 con 0x7f06dc1599e0 2023-10-10T20:43:14.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.346+0000 7f06e138d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc169bf0 con 0x7f06dc0c5a60 2023-10-10T20:43:14.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.346+0000 7f06e138d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc169e80 con 0x7f06dc159610 2023-10-10T20:43:14.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.346+0000 7f06db7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06dc1599e0 0x7f06dc0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:14.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.346+0000 7f06db7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06dc1599e0 0x7f06dc0affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35122/0 (socket says 172.21.15.89:35122) 2023-10-10T20:43:14.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.346+0000 7f06db7fe700 1 -- 172.21.15.89:0/524276643 learned_addr learned my addr 172.21.15.89:0/524276643 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:14.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.347+0000 7f06da7fc700 1 --2- 172.21.15.89:0/524276643 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06dc159610 0x7f06dc0ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:14.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.347+0000 7f06daffd700 1 --2- 172.21.15.89:0/524276643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06dc0c5a60 0x7f06dc15d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:14.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.347+0000 7f06db7fe700 1 -- 172.21.15.89:0/524276643 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06dc159610 msgr2=0x7f06dc0ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:14.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.347+0000 7f06db7fe700 1 --2- 172.21.15.89:0/524276643 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06dc159610 0x7f06dc0ae9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.347+0000 7f06db7fe700 1 -- 172.21.15.89:0/524276643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06dc0c5a60 msgr2=0x7f06dc15d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:14.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.347+0000 7f06db7fe700 1 --2- 172.21.15.89:0/524276643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06dc0c5a60 0x7f06dc15d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.347+0000 7f06db7fe700 1 -- 172.21.15.89:0/524276643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d00094f0 con 0x7f06dc1599e0 2023-10-10T20:43:14.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.347+0000 7f06db7fe700 1 --2- 172.21.15.89:0/524276643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06dc1599e0 0x7f06dc0affb0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f06d400b810 tx=0x7f06d40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:14.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.347+0000 7f06c2ffd700 1 -- 172.21.15.89:0/524276643 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d400f570 con 0x7f06dc1599e0 2023-10-10T20:43:14.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.347+0000 7f06c2ffd700 1 -- 172.21.15.89:0/524276643 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d400f6f0 con 0x7f06dc1599e0 2023-10-10T20:43:14.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.347+0000 7f06e138d700 1 -- 172.21.15.89:0/524276643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06dc0b1a30 con 0x7f06dc1599e0 2023-10-10T20:43:14.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.347+0000 7f06c2ffd700 1 -- 172.21.15.89:0/524276643 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d400fab0 con 0x7f06dc1599e0 2023-10-10T20:43:14.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.347+0000 7f06e138d700 1 -- 172.21.15.89:0/524276643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06dc15db30 con 0x7f06dc1599e0 2023-10-10T20:43:14.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.348+0000 7f06e138d700 1 -- 172.21.15.89:0/524276643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06dc06f950 con 0x7f06dc1599e0 2023-10-10T20:43:14.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.348+0000 7f06c2ffd700 1 -- 172.21.15.89:0/524276643 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f06d401a460 con 0x7f06dc1599e0 2023-10-10T20:43:14.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.349+0000 7f06c2ffd700 1 --2- 172.21.15.89:0/524276643 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f06c8067320 0x7f06c80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:14.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.349+0000 7f06c2ffd700 1 -- 172.21.15.89:0/524276643 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f06d4084b50 con 0x7f06dc1599e0 2023-10-10T20:43:14.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.349+0000 7f06daffd700 1 --2- 172.21.15.89:0/524276643 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f06c8067320 0x7f06c80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:14.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.349+0000 7f06daffd700 1 --2- 172.21.15.89:0/524276643 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f06c8067320 0x7f06c80697d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f06cc001250 tx=0x7f06cc009620 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:14.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.353+0000 7f06c2ffd700 1 -- 172.21.15.89:0/524276643 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06d4051680 con 0x7f06dc1599e0 2023-10-10T20:43:14.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.575+0000 7f06e138d700 1 -- 172.21.15.89:0/524276643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06dc04c3c0 con 0x7f06dc1599e0 2023-10-10T20:43:14.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.575+0000 7f06c2ffd700 1 -- 172.21.15.89:0/524276643 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4633 (secure 0 0 0) 0x7f06d4054ca0 con 0x7f06dc1599e0 2023-10-10T20:43:14.578 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:14.578 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":113,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6638,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2350308973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2350308973},{"type":"v1","addr":"172.21.15.111:6835","nonce":2350308973}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":8322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/1051252386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":1051252386},{"type":"v1","addr":"172.21.15.89:6836","nonce":1051252386}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":8325,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3754994248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":3754994248},{"type":"v1","addr":"172.21.15.89:6837","nonce":3754994248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:42:52.707112+0000","modified":"2023-10-10T20:43:12.015439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8337":{"gid":8337,"name":"b","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1733913702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1733913702},{"type":"v1","addr":"172.21.15.111:6834","nonce":1733913702}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"volume_0000000000654221","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-10T20:43:14.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.577+0000 7f06e138d700 1 -- 172.21.15.89:0/524276643 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f06c8067320 msgr2=0x7f06c80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:14.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.577+0000 7f06e138d700 1 --2- 172.21.15.89:0/524276643 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f06c8067320 0x7f06c80697d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f06cc001250 tx=0x7f06cc009620 comp rx=0 tx=0).stop 2023-10-10T20:43:14.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.577+0000 7f06e138d700 1 -- 172.21.15.89:0/524276643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06dc1599e0 msgr2=0x7f06dc0affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:14.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.577+0000 7f06e138d700 1 --2- 172.21.15.89:0/524276643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06dc1599e0 0x7f06dc0affb0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f06d400b810 tx=0x7f06d40091d0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.577+0000 7f06e138d700 1 -- 172.21.15.89:0/524276643 shutdown_connections 2023-10-10T20:43:14.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.577+0000 7f06e138d700 1 --2- 172.21.15.89:0/524276643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06dc1599e0 0x7f06dc0affb0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.577+0000 7f06e138d700 1 --2- 172.21.15.89:0/524276643 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f06c8067320 0x7f06c80697d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.577+0000 7f06e138d700 1 --2- 172.21.15.89:0/524276643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06dc0c5a60 0x7f06dc15d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.577+0000 7f06e138d700 1 --2- 172.21.15.89:0/524276643 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f06dc159610 0x7f06dc0ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.577+0000 7f06e138d700 1 -- 172.21.15.89:0/524276643 >> 172.21.15.89:0/524276643 conn(0x7f06dc0cc780 msgr2=0x7f06dc1566f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:14.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.577+0000 7f06e138d700 1 -- 172.21.15.89:0/524276643 shutdown_connections 2023-10-10T20:43:14.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.577+0000 7f06e138d700 1 -- 172.21.15.89:0/524276643 wait complete. 2023-10-10T20:43:14.582 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 113 2023-10-10T20:43:14.591 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:43:14.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.703+0000 7f706da14700 1 Processor -- start 2023-10-10T20:43:14.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.703+0000 7f706da14700 1 -- start start 2023-10-10T20:43:14.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.704+0000 7f706da14700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f70681597f0 0x7f70680b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:14.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.704+0000 7f706da14700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f706815d230 0x7f706815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:14.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.704+0000 7f706da14700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706815dbb0 con 0x7f7068159420 2023-10-10T20:43:14.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.704+0000 7f706da14700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70680b11d0 con 0x7f70681597f0 2023-10-10T20:43:14.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.704+0000 7f706da14700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70680b14b0 con 0x7f706815d230 2023-10-10T20:43:14.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.704+0000 7f7067fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f70681597f0 0x7f70680b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:14.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.704+0000 7f7067fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f70681597f0 0x7f70680b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35132/0 (socket says 172.21.15.89:35132) 2023-10-10T20:43:14.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.704+0000 7f7067fff700 1 -- 172.21.15.89:0/3313908128 learned_addr learned my addr 172.21.15.89:0/3313908128 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:14.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.704+0000 7f7067fff700 1 -- 172.21.15.89:0/3313908128 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f706815d230 msgr2=0x7f706815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:14.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.704+0000 7f706d213700 1 --2- 172.21.15.89:0/3313908128 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f706815d230 0x7f706815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:14.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.704+0000 7f7067fff700 1 --2- 172.21.15.89:0/3313908128 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f706815d230 0x7f706815d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.704+0000 7f7067fff700 1 -- 172.21.15.89:0/3313908128 >> v1:172.21.15.111:6789/0 conn(0x7f7068159420 legacy=0x7f70680b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:14.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.704+0000 7f7067fff700 1 -- 172.21.15.89:0/3313908128 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70680b1790 con 0x7f70681597f0 2023-10-10T20:43:14.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.704+0000 7f7067fff700 1 --2- 172.21.15.89:0/3313908128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70681597f0 0x7f70680b2270 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f70540099b0 tx=0x7f7054009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e5929c2463e4d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:14.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.705+0000 7f70677fe700 1 -- 172.21.15.89:0/3313908128 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705400b820 con 0x7f70681597f0 2023-10-10T20:43:14.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.705+0000 7f70677fe700 1 -- 172.21.15.89:0/3313908128 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705400b980 con 0x7f70681597f0 2023-10-10T20:43:14.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.705+0000 7f70677fe700 1 -- 172.21.15.89:0/3313908128 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705400bd20 con 0x7f70681597f0 2023-10-10T20:43:14.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.705+0000 7f706da14700 1 -- 172.21.15.89:0/3313908128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70681597f0 msgr2=0x7f70680b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:14.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.705+0000 7f706da14700 1 --2- 172.21.15.89:0/3313908128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70681597f0 0x7f70680b2270 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f70540099b0 tx=0x7f7054009250 comp rx=0 tx=0).stop 2023-10-10T20:43:14.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.705+0000 7f706da14700 1 -- 172.21.15.89:0/3313908128 shutdown_connections 2023-10-10T20:43:14.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.705+0000 7f706da14700 1 --2- 172.21.15.89:0/3313908128 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f706815d230 0x7f706815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.705+0000 7f706da14700 1 --2- 172.21.15.89:0/3313908128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70681597f0 0x7f70680b2270 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.705+0000 7f706da14700 1 -- 172.21.15.89:0/3313908128 >> 172.21.15.89:0/3313908128 conn(0x7f70680d0660 msgr2=0x7f70680d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:14.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.705+0000 7f706da14700 1 -- 172.21.15.89:0/3313908128 shutdown_connections 2023-10-10T20:43:14.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.705+0000 7f706da14700 1 -- 172.21.15.89:0/3313908128 wait complete. 2023-10-10T20:43:14.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.706+0000 7f706da14700 1 Processor -- start 2023-10-10T20:43:14.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.706+0000 7f706da14700 1 -- start start 2023-10-10T20:43:14.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.707+0000 7f706da14700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7068159420 0x7f706815b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:14.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.707+0000 7f706da14700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70681597f0 0x7f70680c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:14.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.707+0000 7f706da14700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f706815d230 0x7f70680c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:14.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.707+0000 7f706da14700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706815c810 con 0x7f7068159420 2023-10-10T20:43:14.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.707+0000 7f706da14700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706815baf0 con 0x7f706815d230 2023-10-10T20:43:14.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.707+0000 7f706da14700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706816d6f0 con 0x7f70681597f0 2023-10-10T20:43:14.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.707+0000 7f706ca12700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7068159420 0x7f706815b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:14.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.707+0000 7f706ca12700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7068159420 0x7f706815b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35136/0 (socket says 172.21.15.89:35136) 2023-10-10T20:43:14.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.707+0000 7f706ca12700 1 -- 172.21.15.89:0/3779051808 learned_addr learned my addr 172.21.15.89:0/3779051808 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:14.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.707+0000 7f706ca12700 1 -- 172.21.15.89:0/3779051808 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70681597f0 msgr2=0x7f70680c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:14.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.707+0000 7f706ca12700 1 --2- 172.21.15.89:0/3779051808 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70681597f0 0x7f70680c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.707+0000 7f706ca12700 1 -- 172.21.15.89:0/3779051808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f706815d230 msgr2=0x7f70680c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.707+0000 7f706ca12700 1 --2- 172.21.15.89:0/3779051808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f706815d230 0x7f70680c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.707+0000 7f706ca12700 1 -- 172.21.15.89:0/3779051808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70540094f0 con 0x7f7068159420 2023-10-10T20:43:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.707+0000 7f706ca12700 1 --2- 172.21.15.89:0/3779051808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7068159420 0x7f706815b5b0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f705c00c5d0 tx=0x7f705c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.708+0000 7f7064ff9700 1 -- 172.21.15.89:0/3779051808 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c011590 con 0x7f7068159420 2023-10-10T20:43:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.708+0000 7f7064ff9700 1 -- 172.21.15.89:0/3779051808 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c011710 con 0x7f7068159420 2023-10-10T20:43:14.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.708+0000 7f706da14700 1 -- 172.21.15.89:0/3779051808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706816d9d0 con 0x7f7068159420 2023-10-10T20:43:14.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.708+0000 7f7064ff9700 1 -- 172.21.15.89:0/3779051808 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c011ad0 con 0x7f7068159420 2023-10-10T20:43:14.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.708+0000 7f706da14700 1 -- 172.21.15.89:0/3779051808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7068169cb0 con 0x7f7068159420 2023-10-10T20:43:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.709+0000 7f706da14700 1 -- 172.21.15.89:0/3779051808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7068169ea0 con 0x7f7068159420 2023-10-10T20:43:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.709+0000 7f7064ff9700 1 -- 172.21.15.89:0/3779051808 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f705c01c460 con 0x7f7068159420 2023-10-10T20:43:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.709+0000 7f7064ff9700 1 --2- 172.21.15.89:0/3779051808 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f704c067320 0x7f704c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.709+0000 7f7064ff9700 1 -- 172.21.15.89:0/3779051808 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f705c084c50 con 0x7f7068159420 2023-10-10T20:43:14.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.709+0000 7f7067fff700 1 --2- 172.21.15.89:0/3779051808 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f704c067320 0x7f704c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:14.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.710+0000 7f7067fff700 1 --2- 172.21.15.89:0/3779051808 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f704c067320 0x7f704c0697d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f70540099b0 tx=0x7f7054016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:14.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.713+0000 7f7064ff9700 1 -- 172.21.15.89:0/3779051808 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f705c051700 con 0x7f7068159420 2023-10-10T20:43:14.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.898+0000 7f706da14700 1 -- 172.21.15.89:0/3779051808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f706804c3c0 con 0x7f7068159420 2023-10-10T20:43:14.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.899+0000 7f7064ff9700 1 -- 172.21.15.89:0/3779051808 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+17005 (secure 0 0 0) 0x7f705c054d20 con 0x7f7068159420 2023-10-10T20:43:14.902 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:14.902 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":170,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:43:13.020125+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs.volume_0000000000654221.meta","create_time":"2023-10-10T20:42:51.066663+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000654221"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-10T20:42:51.675525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000654221"}}},{"pool":30,"pool_name":"another-data-pool","create_time":"2023-10-10T20:43:08.234534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000000654221"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:43:14.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.901+0000 7f706da14700 1 -- 172.21.15.89:0/3779051808 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f704c067320 msgr2=0x7f704c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:14.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.901+0000 7f706da14700 1 --2- 172.21.15.89:0/3779051808 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f704c067320 0x7f704c0697d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f70540099b0 tx=0x7f7054016000 comp rx=0 tx=0).stop 2023-10-10T20:43:14.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.901+0000 7f706da14700 1 -- 172.21.15.89:0/3779051808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7068159420 msgr2=0x7f706815b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:14.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.901+0000 7f706da14700 1 --2- 172.21.15.89:0/3779051808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7068159420 0x7f706815b5b0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f705c00c5d0 tx=0x7f705c007f60 comp rx=0 tx=0).stop 2023-10-10T20:43:14.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.901+0000 7f706da14700 1 -- 172.21.15.89:0/3779051808 shutdown_connections 2023-10-10T20:43:14.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.901+0000 7f706da14700 1 --2- 172.21.15.89:0/3779051808 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f704c067320 0x7f704c0697d0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.901+0000 7f706da14700 1 --2- 172.21.15.89:0/3779051808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f706815d230 0x7f70680c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.901+0000 7f706da14700 1 --2- 172.21.15.89:0/3779051808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7068159420 0x7f706815b5b0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.901+0000 7f706da14700 1 --2- 172.21.15.89:0/3779051808 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70681597f0 0x7f70680c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:14.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.901+0000 7f706da14700 1 -- 172.21.15.89:0/3779051808 >> 172.21.15.89:0/3779051808 conn(0x7f70680d0660 msgr2=0x7f7068155220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:14.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.902+0000 7f706da14700 1 -- 172.21.15.89:0/3779051808 shutdown_connections 2023-10-10T20:43:14.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:14.902+0000 7f706da14700 1 -- 172.21.15.89:0/3779051808 wait complete. 2023-10-10T20:43:14.916 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 28, 'pool_name': 'cephfs.volume_0000000000654221.meta', 'create_time': '2023-10-10T20:42:51.066663+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000654221'}}}, {'pool': 29, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:42:51.675525+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000654221'}}}, {'pool': 30, 'pool_name': 'another-data-pool', 'create_time': '2023-10-10T20:43:08.234534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000654221'}}}] 2023-10-10T20:43:14.916 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:43:15.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.034+0000 7f024cd25700 1 Processor -- start 2023-10-10T20:43:15.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.034+0000 7f024cd25700 1 -- start start 2023-10-10T20:43:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.035+0000 7f024cd25700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f02480c9770 0x7f02480caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.035+0000 7f024cd25700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02480cb400 0x7f02480c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.035+0000 7f024cd25700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02480af490 con 0x7f02480c93a0 2023-10-10T20:43:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.035+0000 7f024cd25700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02480af770 con 0x7f02480c9770 2023-10-10T20:43:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.035+0000 7f024cd25700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02480afa50 con 0x7f02480cb400 2023-10-10T20:43:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.035+0000 7f023effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f02480c9770 0x7f02480caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.035+0000 7f023effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f02480c9770 0x7f02480caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35140/0 (socket says 172.21.15.89:35140) 2023-10-10T20:43:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.035+0000 7f023effd700 1 -- 172.21.15.89:0/2653869205 learned_addr learned my addr 172.21.15.89:0/2653869205 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:15.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.035+0000 7f0247fff700 1 --2- 172.21.15.89:0/2653869205 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02480cb400 0x7f02480c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:15.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.035+0000 7f023effd700 1 -- 172.21.15.89:0/2653869205 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02480cb400 msgr2=0x7f02480c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:15.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.035+0000 7f023effd700 1 --2- 172.21.15.89:0/2653869205 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02480cb400 0x7f02480c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.035+0000 7f023effd700 1 -- 172.21.15.89:0/2653869205 >> v1:172.21.15.111:6789/0 conn(0x7f02480c93a0 legacy=0x7f024806c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.036+0000 7f023effd700 1 -- 172.21.15.89:0/2653869205 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02480c7ab0 con 0x7f02480c9770 2023-10-10T20:43:15.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.036+0000 7f0247fff700 1 --2- 172.21.15.89:0/2653869205 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02480cb400 0x7f02480c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:15.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.036+0000 7f023effd700 1 --2- 172.21.15.89:0/2653869205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02480c9770 0x7f02480caec0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f0230009990 tx=0x7f0230009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc06d1a8776e7f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:15.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.036+0000 7f0246ffd700 1 -- 172.21.15.89:0/2653869205 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0230005c10 con 0x7f02480c9770 2023-10-10T20:43:15.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.036+0000 7f0246ffd700 1 -- 172.21.15.89:0/2653869205 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0230005d70 con 0x7f02480c9770 2023-10-10T20:43:15.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.036+0000 7f0246ffd700 1 -- 172.21.15.89:0/2653869205 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023000b880 con 0x7f02480c9770 2023-10-10T20:43:15.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.036+0000 7f024cd25700 1 -- 172.21.15.89:0/2653869205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02480c9770 msgr2=0x7f02480caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.036+0000 7f024cd25700 1 --2- 172.21.15.89:0/2653869205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02480c9770 0x7f02480caec0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f0230009990 tx=0x7f0230009220 comp rx=0 tx=0).stop 2023-10-10T20:43:15.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.036+0000 7f024cd25700 1 -- 172.21.15.89:0/2653869205 shutdown_connections 2023-10-10T20:43:15.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.036+0000 7f024cd25700 1 --2- 172.21.15.89:0/2653869205 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02480cb400 0x7f02480c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.036+0000 7f024cd25700 1 --2- 172.21.15.89:0/2653869205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02480c9770 0x7f02480caec0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.036+0000 7f024cd25700 1 -- 172.21.15.89:0/2653869205 >> 172.21.15.89:0/2653869205 conn(0x7f02480cddc0 msgr2=0x7f02480c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:15.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.036+0000 7f024cd25700 1 -- 172.21.15.89:0/2653869205 shutdown_connections 2023-10-10T20:43:15.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.036+0000 7f024cd25700 1 -- 172.21.15.89:0/2653869205 wait complete. 2023-10-10T20:43:15.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.037+0000 7f024cd25700 1 Processor -- start 2023-10-10T20:43:15.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.038+0000 7f024cd25700 1 -- start start 2023-10-10T20:43:15.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.038+0000 7f024cd25700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02480c93a0 0x7f024816b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:15.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.038+0000 7f024cd25700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02480c9770 0x7f02480caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:15.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.038+0000 7f024cd25700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02480cb400 0x7f0248151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:15.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.038+0000 7f024cd25700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024816b4e0 con 0x7f02480c93a0 2023-10-10T20:43:15.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.038+0000 7f024cd25700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024816b200 con 0x7f02480c9770 2023-10-10T20:43:15.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.038+0000 7f024cd25700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02480b0600 con 0x7f02480cb400 2023-10-10T20:43:15.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.038+0000 7f02477fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02480c93a0 0x7f024816b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:15.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.039+0000 7f02477fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02480c93a0 0x7f024816b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35146/0 (socket says 172.21.15.89:35146) 2023-10-10T20:43:15.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.039+0000 7f02477fe700 1 -- 172.21.15.89:0/946365376 learned_addr learned my addr 172.21.15.89:0/946365376 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:15.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.039+0000 7f023effd700 1 --2- 172.21.15.89:0/946365376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02480c9770 0x7f02480caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:15.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.039+0000 7f0247fff700 1 --2- 172.21.15.89:0/946365376 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02480cb400 0x7f0248151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:15.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.039+0000 7f02477fe700 1 -- 172.21.15.89:0/946365376 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02480cb400 msgr2=0x7f0248151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.039+0000 7f02477fe700 1 --2- 172.21.15.89:0/946365376 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02480cb400 0x7f0248151c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.039+0000 7f02477fe700 1 -- 172.21.15.89:0/946365376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02480c9770 msgr2=0x7f02480caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.039+0000 7f02477fe700 1 --2- 172.21.15.89:0/946365376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02480c9770 0x7f02480caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.039+0000 7f02477fe700 1 -- 172.21.15.89:0/946365376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0230009480 con 0x7f02480c93a0 2023-10-10T20:43:15.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.039+0000 7f0247fff700 1 --2- 172.21.15.89:0/946365376 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02480cb400 0x7f0248151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:15.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.039+0000 7f02477fe700 1 --2- 172.21.15.89:0/946365376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02480c93a0 0x7f024816b650 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f023800c590 tx=0x7f023800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:15.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.039+0000 7f023ffff700 1 -- 172.21.15.89:0/946365376 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238011590 con 0x7f02480c93a0 2023-10-10T20:43:15.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.039+0000 7f023ffff700 1 -- 172.21.15.89:0/946365376 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0238011710 con 0x7f02480c93a0 2023-10-10T20:43:15.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.039+0000 7f024cd25700 1 -- 172.21.15.89:0/946365376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02480b08e0 con 0x7f02480c93a0 2023-10-10T20:43:15.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.039+0000 7f023ffff700 1 -- 172.21.15.89:0/946365376 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238011ad0 con 0x7f02480c93a0 2023-10-10T20:43:15.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.039+0000 7f024cd25700 1 -- 172.21.15.89:0/946365376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f024816bb90 con 0x7f02480c93a0 2023-10-10T20:43:15.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.040+0000 7f024cd25700 1 -- 172.21.15.89:0/946365376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02480b0e00 con 0x7f02480c93a0 2023-10-10T20:43:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.040+0000 7f023ffff700 1 -- 172.21.15.89:0/946365376 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f023801b3b0 con 0x7f02480c93a0 2023-10-10T20:43:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.041+0000 7f023ffff700 1 --2- 172.21.15.89:0/946365376 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0228067320 0x7f02280697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.041+0000 7f023ffff700 1 -- 172.21.15.89:0/946365376 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f0238084390 con 0x7f02480c93a0 2023-10-10T20:43:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.041+0000 7f023effd700 1 --2- 172.21.15.89:0/946365376 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0228067320 0x7f02280697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.041+0000 7f023effd700 1 --2- 172.21.15.89:0/946365376 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0228067320 0x7f02280697d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0230009260 tx=0x7f0230016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.045+0000 7f023ffff700 1 -- 172.21.15.89:0/946365376 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02480b0e00 con 0x7f02480c93a0 2023-10-10T20:43:15.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.227+0000 7f024cd25700 1 -- 172.21.15.89:0/946365376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f02480b0e00 con 0x7f02480c93a0 2023-10-10T20:43:15.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.228+0000 7f023ffff700 1 -- 172.21.15.89:0/946365376 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v170) v1 ==== 93+0+10 (secure 0 0 0) 0x7f02480b0e00 con 0x7f02480c93a0 2023-10-10T20:43:15.230 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:43:15.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.230+0000 7f024cd25700 1 -- 172.21.15.89:0/946365376 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0228067320 msgr2=0x7f02280697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.230+0000 7f024cd25700 1 --2- 172.21.15.89:0/946365376 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0228067320 0x7f02280697d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0230009260 tx=0x7f0230016000 comp rx=0 tx=0).stop 2023-10-10T20:43:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.230+0000 7f024cd25700 1 -- 172.21.15.89:0/946365376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02480c93a0 msgr2=0x7f024816b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.230+0000 7f024cd25700 1 --2- 172.21.15.89:0/946365376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02480c93a0 0x7f024816b650 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f023800c590 tx=0x7f023800c8a0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.230+0000 7f024cd25700 1 -- 172.21.15.89:0/946365376 shutdown_connections 2023-10-10T20:43:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.230+0000 7f024cd25700 1 --2- 172.21.15.89:0/946365376 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f02480cb400 0x7f0248151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.230+0000 7f024cd25700 1 --2- 172.21.15.89:0/946365376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02480c93a0 0x7f024816b650 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.230+0000 7f024cd25700 1 --2- 172.21.15.89:0/946365376 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0228067320 0x7f02280697d0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.230+0000 7f024cd25700 1 --2- 172.21.15.89:0/946365376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02480c9770 0x7f02480caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.230+0000 7f024cd25700 1 -- 172.21.15.89:0/946365376 >> 172.21.15.89:0/946365376 conn(0x7f02480cddc0 msgr2=0x7f024806e500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:15.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.230+0000 7f024cd25700 1 -- 172.21.15.89:0/946365376 shutdown_connections 2023-10-10T20:43:15.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.230+0000 7f024cd25700 1 -- 172.21.15.89:0/946365376 wait complete. 2023-10-10T20:43:15.244 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000654221.meta pg_num 2023-10-10T20:43:15.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.360+0000 7f5a43820700 1 Processor -- start 2023-10-10T20:43:15.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.360+0000 7f5a43820700 1 -- start start 2023-10-10T20:43:15.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.361+0000 7f5a43820700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a3c159c60 0x7f5a3c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:15.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.361+0000 7f5a43820700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a3c15a160 con 0x7f5a3c15db10 2023-10-10T20:43:15.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.361+0000 7f5a43820700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a3c0d13a0 con 0x7f5a3c159890 2023-10-10T20:43:15.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.361+0000 7f5a43820700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c15e140 con 0x7f5a3c159c60 2023-10-10T20:43:15.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.361+0000 7f5a41dbd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5a3c15db10 0x7f5a3c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37718/0 (socket says 172.21.15.89:37718) 2023-10-10T20:43:15.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.361+0000 7f5a41dbd700 1 -- 172.21.15.89:0/1773932049 learned_addr learned my addr 172.21.15.89:0/1773932049 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:15.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a33fff700 1 -- 172.21.15.89:0/1773932049 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1463197352 0 0) 0x7f5a3c15a160 con 0x7f5a3c15db10 2023-10-10T20:43:15.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a33fff700 1 -- 172.21.15.89:0/1773932049 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a28003570 con 0x7f5a3c15db10 2023-10-10T20:43:15.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a40dbb700 1 --2- 172.21.15.89:0/1773932049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a3c159c60 0x7f5a3c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:15.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a33fff700 1 -- 172.21.15.89:0/1773932049 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2300331538 0 0) 0x7f5a28003570 con 0x7f5a3c15db10 2023-10-10T20:43:15.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a33fff700 1 -- 172.21.15.89:0/1773932049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a3c159c60 msgr2=0x7f5a3c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:15.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a33fff700 1 --2- 172.21.15.89:0/1773932049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a3c159c60 0x7f5a3c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a33fff700 1 -- 172.21.15.89:0/1773932049 >> v1:172.21.15.111:6789/0 conn(0x7f5a3c159890 legacy=0x7f5a3c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a33fff700 1 -- 172.21.15.89:0/1773932049 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c15a3d0 con 0x7f5a3c15db10 2023-10-10T20:43:15.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a33fff700 1 -- 172.21.15.89:0/1773932049 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f5a34001470 con 0x7f5a3c15db10 2023-10-10T20:43:15.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a33fff700 1 -- 172.21.15.89:0/1773932049 >> v1:172.21.15.89:6789/0 conn(0x7f5a3c15db10 legacy=0x7f5a3c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a33fff700 1 --2- 172.21.15.89:0/1773932049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a28003a90 0x7f5a28003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:15.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a33fff700 1 --2- 172.21.15.89:0/1773932049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a3c159c60 0x7f5a28004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:15.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a33fff700 1 --2- 172.21.15.89:0/1773932049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a28004d50 0x7f5a280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:15.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a33fff700 1 -- 172.21.15.89:0/1773932049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c15e140 con 0x7f5a28003a90 2023-10-10T20:43:15.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a33fff700 1 -- 172.21.15.89:0/1773932049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2800bb10 con 0x7f5a28004d50 2023-10-10T20:43:15.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a33fff700 1 -- 172.21.15.89:0/1773932049 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2800bcd0 con 0x7f5a3c159c60 2023-10-10T20:43:15.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a415bc700 1 --2- 172.21.15.89:0/1773932049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a28003a90 0x7f5a28003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:15.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a40dbb700 1 --2- 172.21.15.89:0/1773932049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a3c159c60 0x7f5a28004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:15.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a41dbd700 1 --2- 172.21.15.89:0/1773932049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a28004d50 0x7f5a280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:15.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a415bc700 1 -- 172.21.15.89:0/1773932049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a3c159c60 msgr2=0x7f5a28004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:15.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a415bc700 1 --2- 172.21.15.89:0/1773932049 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a3c159c60 0x7f5a28004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a415bc700 1 -- 172.21.15.89:0/1773932049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a28004d50 msgr2=0x7f5a280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:15.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a415bc700 1 --2- 172.21.15.89:0/1773932049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a28004d50 0x7f5a280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a415bc700 1 -- 172.21.15.89:0/1773932049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a28003570 con 0x7f5a28003a90 2023-10-10T20:43:15.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.362+0000 7f5a415bc700 1 --2- 172.21.15.89:0/1773932049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a28003a90 0x7f5a28003e60 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f5a380020a0 tx=0x7f5a380016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63af40cffed4da9f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:15.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.363+0000 7f5a33fff700 1 -- 172.21.15.89:0/1773932049 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a380019e0 con 0x7f5a28003a90 2023-10-10T20:43:15.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.363+0000 7f5a33fff700 1 -- 172.21.15.89:0/1773932049 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3800c8b0 con 0x7f5a28003a90 2023-10-10T20:43:15.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.363+0000 7f5a33fff700 1 -- 172.21.15.89:0/1773932049 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3800cc50 con 0x7f5a28003a90 2023-10-10T20:43:15.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.363+0000 7f5a43820700 1 -- 172.21.15.89:0/1773932049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a28003a90 msgr2=0x7f5a28003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.363+0000 7f5a43820700 1 --2- 172.21.15.89:0/1773932049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a28003a90 0x7f5a28003e60 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f5a380020a0 tx=0x7f5a380016d0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.363+0000 7f5a41dbd700 1 -- 172.21.15.89:0/1773932049 reap_dead start 2023-10-10T20:43:15.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.363+0000 7f5a43820700 1 -- 172.21.15.89:0/1773932049 shutdown_connections 2023-10-10T20:43:15.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.363+0000 7f5a43820700 1 -- 172.21.15.89:0/1773932049 >> 172.21.15.89:0/1773932049 conn(0x7f5a3c068e20 msgr2=0x7f5a3c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:15.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.363+0000 7f5a43820700 1 -- 172.21.15.89:0/1773932049 shutdown_connections 2023-10-10T20:43:15.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.363+0000 7f5a43820700 1 -- 172.21.15.89:0/1773932049 wait complete. 2023-10-10T20:43:15.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.364+0000 7f5a43820700 1 Processor -- start 2023-10-10T20:43:15.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.364+0000 7f5a43820700 1 -- start start 2023-10-10T20:43:15.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.365+0000 7f5a43820700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a28003a90 0x7f5a3c14a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:15.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.365+0000 7f5a43820700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a3c15cfd0 0x7f5a3c14b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:15.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.365+0000 7f5a43820700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3c15d440 0x7f5a3c14b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:15.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.365+0000 7f5a43820700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c15c030 con 0x7f5a3c15d440 2023-10-10T20:43:15.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.365+0000 7f5a43820700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c15b700 con 0x7f5a28003a90 2023-10-10T20:43:15.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.365+0000 7f5a43820700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c15b9e0 con 0x7f5a3c15cfd0 2023-10-10T20:43:15.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.365+0000 7f5a41dbd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3c15d440 0x7f5a3c14b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:15.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.365+0000 7f5a41dbd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3c15d440 0x7f5a3c14b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35166/0 (socket says 172.21.15.89:35166) 2023-10-10T20:43:15.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.365+0000 7f5a41dbd700 1 -- 172.21.15.89:0/2416095052 learned_addr learned my addr 172.21.15.89:0/2416095052 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:15.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.366+0000 7f5a40dbb700 1 --2- 172.21.15.89:0/2416095052 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a3c15cfd0 0x7f5a3c14b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:15.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.366+0000 7f5a41dbd700 1 -- 172.21.15.89:0/2416095052 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a3c15cfd0 msgr2=0x7f5a3c14b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.366+0000 7f5a415bc700 1 --2- 172.21.15.89:0/2416095052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a28003a90 0x7f5a3c14a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:15.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.366+0000 7f5a41dbd700 1 --2- 172.21.15.89:0/2416095052 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a3c15cfd0 0x7f5a3c14b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.366+0000 7f5a41dbd700 1 -- 172.21.15.89:0/2416095052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a28003a90 msgr2=0x7f5a3c14a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.366+0000 7f5a41dbd700 1 --2- 172.21.15.89:0/2416095052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a28003a90 0x7f5a3c14a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.366+0000 7f5a41dbd700 1 -- 172.21.15.89:0/2416095052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c15bcc0 con 0x7f5a3c15d440 2023-10-10T20:43:15.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.366+0000 7f5a41dbd700 1 --2- 172.21.15.89:0/2416095052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3c15d440 0x7f5a3c14b7b0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c15dee0 tx=0x7f5a34001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:15.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.366+0000 7f5a317fa700 1 -- 172.21.15.89:0/2416095052 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a34018030 con 0x7f5a3c15d440 2023-10-10T20:43:15.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.366+0000 7f5a317fa700 1 -- 172.21.15.89:0/2416095052 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a34012560 con 0x7f5a3c15d440 2023-10-10T20:43:15.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.366+0000 7f5a43820700 1 -- 172.21.15.89:0/2416095052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3c0b3c90 con 0x7f5a3c15d440 2023-10-10T20:43:15.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.366+0000 7f5a317fa700 1 -- 172.21.15.89:0/2416095052 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a34012900 con 0x7f5a3c15d440 2023-10-10T20:43:15.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.366+0000 7f5a43820700 1 -- 172.21.15.89:0/2416095052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3c14a890 con 0x7f5a3c15d440 2023-10-10T20:43:15.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.367+0000 7f5a43820700 1 -- 172.21.15.89:0/2416095052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a3c15b840 con 0x7f5a3c15d440 2023-10-10T20:43:15.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.368+0000 7f5a317fa700 1 -- 172.21.15.89:0/2416095052 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5a3401b070 con 0x7f5a3c15d440 2023-10-10T20:43:15.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.368+0000 7f5a317fa700 1 --2- 172.21.15.89:0/2416095052 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a20067320 0x7f5a200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:15.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.368+0000 7f5a317fa700 1 -- 172.21.15.89:0/2416095052 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f5a34019070 con 0x7f5a3c15d440 2023-10-10T20:43:15.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.368+0000 7f5a415bc700 1 --2- 172.21.15.89:0/2416095052 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a20067320 0x7f5a200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:15.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.368+0000 7f5a415bc700 1 --2- 172.21.15.89:0/2416095052 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a20067320 0x7f5a200697d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5a38001e30 tx=0x7f5a38016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:15.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.372+0000 7f5a317fa700 1 -- 172.21.15.89:0/2416095052 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a34055860 con 0x7f5a3c15d440 2023-10-10T20:43:15.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.551+0000 7f5a43820700 1 -- 172.21.15.89:0/2416095052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000654221.meta", "var": "pg_num"} v 0) v1 -- 0x7f5a3c15b840 con 0x7f5a3c15d440 2023-10-10T20:43:15.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.552+0000 7f5a317fa700 1 -- 172.21.15.89:0/2416095052 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000654221.meta", "var": "pg_num"}]=0 v170) v1 ==== 124+0+11 (secure 0 0 0) 0x7f5a3c15b840 con 0x7f5a3c15d440 2023-10-10T20:43:15.554 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:43:15.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.554+0000 7f5a43820700 1 -- 172.21.15.89:0/2416095052 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a20067320 msgr2=0x7f5a200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.554+0000 7f5a43820700 1 --2- 172.21.15.89:0/2416095052 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a20067320 0x7f5a200697d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5a38001e30 tx=0x7f5a38016000 comp rx=0 tx=0).stop 2023-10-10T20:43:15.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.554+0000 7f5a43820700 1 -- 172.21.15.89:0/2416095052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3c15d440 msgr2=0x7f5a3c14b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.554+0000 7f5a43820700 1 --2- 172.21.15.89:0/2416095052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3c15d440 0x7f5a3c14b7b0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c15dee0 tx=0x7f5a34001e70 comp rx=0 tx=0).stop 2023-10-10T20:43:15.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.554+0000 7f5a43820700 1 -- 172.21.15.89:0/2416095052 shutdown_connections 2023-10-10T20:43:15.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.554+0000 7f5a43820700 1 --2- 172.21.15.89:0/2416095052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3c15d440 0x7f5a3c14b7b0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.554+0000 7f5a43820700 1 --2- 172.21.15.89:0/2416095052 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a20067320 0x7f5a200697d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.554+0000 7f5a43820700 1 --2- 172.21.15.89:0/2416095052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a28003a90 0x7f5a3c14a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.554+0000 7f5a43820700 1 --2- 172.21.15.89:0/2416095052 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5a3c15cfd0 0x7f5a3c14b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.554+0000 7f5a43820700 1 -- 172.21.15.89:0/2416095052 >> 172.21.15.89:0/2416095052 conn(0x7f5a3c068e20 msgr2=0x7f5a3c14d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:15.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.554+0000 7f5a43820700 1 -- 172.21.15.89:0/2416095052 shutdown_connections 2023-10-10T20:43:15.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.554+0000 7f5a43820700 1 -- 172.21.15.89:0/2416095052 wait complete. 2023-10-10T20:43:15.568 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:43:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.683+0000 7f42a4db8700 1 Processor -- start 2023-10-10T20:43:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.683+0000 7f42a4db8700 1 -- start start 2023-10-10T20:43:15.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.684+0000 7f42a4db8700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f42a0159c60 0x7f42a015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:15.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.684+0000 7f42a4db8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a015a160 con 0x7f42a015db10 2023-10-10T20:43:15.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.684+0000 7f42a4db8700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a00d13a0 con 0x7f42a0159890 2023-10-10T20:43:15.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.684+0000 7f42a4db8700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a015df50 con 0x7f42a0159c60 2023-10-10T20:43:15.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.684+0000 7f429ed9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f42a015db10 0x7f42a0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37734/0 (socket says 172.21.15.89:37734) 2023-10-10T20:43:15.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.684+0000 7f429ed9d700 1 -- 172.21.15.89:0/978289216 learned_addr learned my addr 172.21.15.89:0/978289216 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:15.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.684+0000 7f429dd9b700 1 --2- 172.21.15.89:0/978289216 >> v2:172.21.15.111:3300/0 conn(0x7f42a0159c60 0x7f42a015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:15.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.684+0000 7f429d59a700 1 -- 172.21.15.89:0/978289216 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1040022598 0 0) 0x7f42a015a160 con 0x7f42a015db10 2023-10-10T20:43:15.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.684+0000 7f429d59a700 1 -- 172.21.15.89:0/978289216 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4288003570 con 0x7f42a015db10 2023-10-10T20:43:15.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.685+0000 7f429dd9b700 1 -- 172.21.15.89:0/978289216 >> v1:172.21.15.111:6789/0 conn(0x7f42a0159890 legacy=0x7f42a015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.685+0000 7f429dd9b700 1 -- 172.21.15.89:0/978289216 >> v1:172.21.15.89:6789/0 conn(0x7f42a015db10 legacy=0x7f42a0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.685+0000 7f429dd9b700 1 -- 172.21.15.89:0/978289216 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a015e140 con 0x7f42a0159c60 2023-10-10T20:43:15.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.685+0000 7f429dd9b700 1 --2- 172.21.15.89:0/978289216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42a0159c60 0x7f42a015d5d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f42940098e0 tx=0x7f42940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb1fd3a50083fd8a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:15.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.685+0000 7f429d59a700 1 -- 172.21.15.89:0/978289216 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429400b640 con 0x7f42a0159c60 2023-10-10T20:43:15.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.685+0000 7f429d59a700 1 -- 172.21.15.89:0/978289216 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429400b7a0 con 0x7f42a0159c60 2023-10-10T20:43:15.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.685+0000 7f429d59a700 1 -- 172.21.15.89:0/978289216 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429400bb20 con 0x7f42a0159c60 2023-10-10T20:43:15.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.685+0000 7f42a4db8700 1 -- 172.21.15.89:0/978289216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42a0159c60 msgr2=0x7f42a015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.685+0000 7f42a4db8700 1 --2- 172.21.15.89:0/978289216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42a0159c60 0x7f42a015d5d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f42940098e0 tx=0x7f42940091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.685+0000 7f42a4db8700 1 -- 172.21.15.89:0/978289216 shutdown_connections 2023-10-10T20:43:15.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.685+0000 7f42a4db8700 1 --2- 172.21.15.89:0/978289216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42a0159c60 0x7f42a015d5d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.685+0000 7f42a4db8700 1 -- 172.21.15.89:0/978289216 >> 172.21.15.89:0/978289216 conn(0x7f42a0068e20 msgr2=0x7f42a00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:15.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.685+0000 7f42a4db8700 1 -- 172.21.15.89:0/978289216 shutdown_connections 2023-10-10T20:43:15.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.686+0000 7f42a4db8700 1 -- 172.21.15.89:0/978289216 wait complete. 2023-10-10T20:43:15.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.687+0000 7f42a4db8700 1 Processor -- start 2023-10-10T20:43:15.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.687+0000 7f42a4db8700 1 -- start start 2023-10-10T20:43:15.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.687+0000 7f42a4db8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42a0159890 0x7f42a00b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:15.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.687+0000 7f42a4db8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42a0159c60 0x7f42a00c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:15.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.687+0000 7f42a4db8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42a015db10 0x7f42a00b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:15.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.687+0000 7f42a4db8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a00c8890 con 0x7f42a0159c60 2023-10-10T20:43:15.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.687+0000 7f42a4db8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a00c94c0 con 0x7f42a015db10 2023-10-10T20:43:15.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.687+0000 7f42a4db8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a00b6450 con 0x7f42a0159890 2023-10-10T20:43:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.687+0000 7f429dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42a0159c60 0x7f42a00c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.687+0000 7f429dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42a0159c60 0x7f42a00c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35168/0 (socket says 172.21.15.89:35168) 2023-10-10T20:43:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.687+0000 7f429dd9b700 1 -- 172.21.15.89:0/1454129783 learned_addr learned my addr 172.21.15.89:0/1454129783 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.688+0000 7f429ed9d700 1 --2- 172.21.15.89:0/1454129783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42a015db10 0x7f42a00b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.688+0000 7f429e59c700 1 --2- 172.21.15.89:0/1454129783 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42a0159890 0x7f42a00b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.688+0000 7f429dd9b700 1 -- 172.21.15.89:0/1454129783 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42a0159890 msgr2=0x7f42a00b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.688+0000 7f429dd9b700 1 --2- 172.21.15.89:0/1454129783 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42a0159890 0x7f42a00b5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.688+0000 7f429dd9b700 1 -- 172.21.15.89:0/1454129783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42a015db10 msgr2=0x7f42a00b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.688+0000 7f429dd9b700 1 --2- 172.21.15.89:0/1454129783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42a015db10 0x7f42a00b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.688+0000 7f429dd9b700 1 -- 172.21.15.89:0/1454129783 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4294009420 con 0x7f42a0159c60 2023-10-10T20:43:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.688+0000 7f429dd9b700 1 --2- 172.21.15.89:0/1454129783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42a0159c60 0x7f42a00c97e0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f42940093f0 tx=0x7f4294017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.688+0000 7f428e7fc700 1 -- 172.21.15.89:0/1454129783 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429400b910 con 0x7f42a0159c60 2023-10-10T20:43:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.688+0000 7f428e7fc700 1 -- 172.21.15.89:0/1454129783 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429400ba90 con 0x7f42a0159c60 2023-10-10T20:43:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.688+0000 7f428e7fc700 1 -- 172.21.15.89:0/1454129783 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294025670 con 0x7f42a0159c60 2023-10-10T20:43:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.688+0000 7f42a4db8700 1 -- 172.21.15.89:0/1454129783 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a00c7c80 con 0x7f42a0159c60 2023-10-10T20:43:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.688+0000 7f42a4db8700 1 -- 172.21.15.89:0/1454129783 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a00c8240 con 0x7f42a0159c60 2023-10-10T20:43:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.689+0000 7f42a4db8700 1 -- 172.21.15.89:0/1454129783 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a00c8a00 con 0x7f42a0159c60 2023-10-10T20:43:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.689+0000 7f428e7fc700 1 -- 172.21.15.89:0/1454129783 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4294023070 con 0x7f42a0159c60 2023-10-10T20:43:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.690+0000 7f428e7fc700 1 --2- 172.21.15.89:0/1454129783 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4280067320 0x7f42800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.690+0000 7f428e7fc700 1 -- 172.21.15.89:0/1454129783 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f42940865c0 con 0x7f42a0159c60 2023-10-10T20:43:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.690+0000 7f429e59c700 1 --2- 172.21.15.89:0/1454129783 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4280067320 0x7f42800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.690+0000 7f429e59c700 1 --2- 172.21.15.89:0/1454129783 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4280067320 0x7f42800697d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f4290007fd0 tx=0x7f4290009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.694+0000 7f428e7fc700 1 -- 172.21.15.89:0/1454129783 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4294053070 con 0x7f42a0159c60 2023-10-10T20:43:15.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.875+0000 7f42a4db8700 1 -- 172.21.15.89:0/1454129783 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f42a004c3c0 con 0x7f42a0159c60 2023-10-10T20:43:15.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.875+0000 7f428e7fc700 1 -- 172.21.15.89:0/1454129783 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v170) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4294056690 con 0x7f42a0159c60 2023-10-10T20:43:15.878 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-10T20:43:15.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.877+0000 7f42a4db8700 1 -- 172.21.15.89:0/1454129783 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4280067320 msgr2=0x7f42800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.877+0000 7f42a4db8700 1 --2- 172.21.15.89:0/1454129783 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4280067320 0x7f42800697d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f4290007fd0 tx=0x7f4290009040 comp rx=0 tx=0).stop 2023-10-10T20:43:15.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.877+0000 7f42a4db8700 1 -- 172.21.15.89:0/1454129783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42a0159c60 msgr2=0x7f42a00c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:15.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.877+0000 7f42a4db8700 1 --2- 172.21.15.89:0/1454129783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42a0159c60 0x7f42a00c97e0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f42940093f0 tx=0x7f4294017000 comp rx=0 tx=0).stop 2023-10-10T20:43:15.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.877+0000 7f42a4db8700 1 -- 172.21.15.89:0/1454129783 shutdown_connections 2023-10-10T20:43:15.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.877+0000 7f42a4db8700 1 --2- 172.21.15.89:0/1454129783 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4280067320 0x7f42800697d0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.877+0000 7f42a4db8700 1 --2- 172.21.15.89:0/1454129783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42a015db10 0x7f42a00b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.877+0000 7f42a4db8700 1 --2- 172.21.15.89:0/1454129783 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f42a0159890 0x7f42a00b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.877+0000 7f42a4db8700 1 --2- 172.21.15.89:0/1454129783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42a0159c60 0x7f42a00c97e0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:15.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.877+0000 7f42a4db8700 1 -- 172.21.15.89:0/1454129783 >> 172.21.15.89:0/1454129783 conn(0x7f42a0068e20 msgr2=0x7f42a01534b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:15.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.877+0000 7f42a4db8700 1 -- 172.21.15.89:0/1454129783 shutdown_connections 2023-10-10T20:43:15.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:15.877+0000 7f42a4db8700 1 -- 172.21.15.89:0/1454129783 wait complete. 2023-10-10T20:43:15.891 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get another-data-pool pg_num 2023-10-10T20:43:16.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.007+0000 7f84faada700 1 Processor -- start 2023-10-10T20:43:16.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.007+0000 7f84faada700 1 -- start start 2023-10-10T20:43:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.008+0000 7f84faada700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f84f4159890 0x7f84f415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.008+0000 7f84faada700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f84f4159c60 0x7f84f415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:16.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.008+0000 7f84faada700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84f415db10 0x7f84f4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:16.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.008+0000 7f84faada700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f415df50 con 0x7f84f4159890 2023-10-10T20:43:16.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.008+0000 7f84faada700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f415a160 con 0x7f84f4159c60 2023-10-10T20:43:16.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.008+0000 7f84faada700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f40d13a0 con 0x7f84f415db10 2023-10-10T20:43:16.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.008+0000 7f84f8876700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f84f4159890 0x7f84f415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:16.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.008+0000 7f84f8876700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f84f4159890 0x7f84f415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35182/0 (socket says 172.21.15.89:35182) 2023-10-10T20:43:16.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.008+0000 7f84f8876700 1 -- 172.21.15.89:0/2355931130 learned_addr learned my addr 172.21.15.89:0/2355931130 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:16.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.008+0000 7f84f9077700 1 --2- 172.21.15.89:0/2355931130 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84f415db10 0x7f84f4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:16.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.008+0000 7f84ebfff700 1 --2- 172.21.15.89:0/2355931130 >> v2:172.21.15.111:3300/0 conn(0x7f84f4159c60 0x7f84f415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:16.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.009+0000 7f84f8876700 1 -- 172.21.15.89:0/2355931130 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84f415db10 msgr2=0x7f84f4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:16.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.009+0000 7f84f8876700 1 --2- 172.21.15.89:0/2355931130 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84f415db10 0x7f84f4069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.009+0000 7f84f8876700 1 -- 172.21.15.89:0/2355931130 >> v2:172.21.15.111:3300/0 conn(0x7f84f4159c60 msgr2=0x7f84f415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:16.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.009+0000 7f84f8876700 1 --2- 172.21.15.89:0/2355931130 >> v2:172.21.15.111:3300/0 conn(0x7f84f4159c60 0x7f84f415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.009+0000 7f84f8876700 1 -- 172.21.15.89:0/2355931130 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f415e140 con 0x7f84f4159890 2023-10-10T20:43:16.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.009+0000 7f84f8876700 1 --2- 172.21.15.89:0/2355931130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f4159890 0x7f84f415abd0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f84f40b67f0 tx=0x7f84f0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6620c8e5cc4933d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:16.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.009+0000 7f84eb7fe700 1 -- 172.21.15.89:0/2355931130 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f000e070 con 0x7f84f4159890 2023-10-10T20:43:16.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.009+0000 7f84eb7fe700 1 -- 172.21.15.89:0/2355931130 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f000ba50 con 0x7f84f4159890 2023-10-10T20:43:16.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.009+0000 7f84eb7fe700 1 -- 172.21.15.89:0/2355931130 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f00092e0 con 0x7f84f4159890 2023-10-10T20:43:16.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.009+0000 7f84faada700 1 -- 172.21.15.89:0/2355931130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f4159890 msgr2=0x7f84f415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:16.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.009+0000 7f84faada700 1 --2- 172.21.15.89:0/2355931130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f4159890 0x7f84f415abd0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f84f40b67f0 tx=0x7f84f0009840 comp rx=0 tx=0).stop 2023-10-10T20:43:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.009+0000 7f84faada700 1 -- 172.21.15.89:0/2355931130 shutdown_connections 2023-10-10T20:43:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.009+0000 7f84faada700 1 --2- 172.21.15.89:0/2355931130 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84f415db10 0x7f84f4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.009+0000 7f84faada700 1 --2- 172.21.15.89:0/2355931130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f4159890 0x7f84f415abd0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.009+0000 7f84faada700 1 --2- 172.21.15.89:0/2355931130 >> v2:172.21.15.111:3300/0 conn(0x7f84f4159c60 0x7f84f415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.009+0000 7f84faada700 1 -- 172.21.15.89:0/2355931130 >> 172.21.15.89:0/2355931130 conn(0x7f84f4068e20 msgr2=0x7f84f40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.009+0000 7f84faada700 1 -- 172.21.15.89:0/2355931130 shutdown_connections 2023-10-10T20:43:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.009+0000 7f84faada700 1 -- 172.21.15.89:0/2355931130 wait complete. 2023-10-10T20:43:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.011+0000 7f84faada700 1 Processor -- start 2023-10-10T20:43:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.011+0000 7f84faada700 1 -- start start 2023-10-10T20:43:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.011+0000 7f84faada700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f4159890 0x7f84f40b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.011+0000 7f84faada700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84f4159c60 0x7f84f40b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.011+0000 7f84faada700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f415db10 0x7f84f40b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.011+0000 7f84faada700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f4151c80 con 0x7f84f4159890 2023-10-10T20:43:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.011+0000 7f84faada700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f4151390 con 0x7f84f415db10 2023-10-10T20:43:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.011+0000 7f84faada700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f4151630 con 0x7f84f4159c60 2023-10-10T20:43:16.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.011+0000 7f84f8876700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f4159890 0x7f84f40b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:16.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.012+0000 7f84f8876700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f4159890 0x7f84f40b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35190/0 (socket says 172.21.15.89:35190) 2023-10-10T20:43:16.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.012+0000 7f84f8876700 1 -- 172.21.15.89:0/566337520 learned_addr learned my addr 172.21.15.89:0/566337520 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:16.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.012+0000 7f84ebfff700 1 --2- 172.21.15.89:0/566337520 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84f4159c60 0x7f84f40b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:16.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.012+0000 7f84f9077700 1 --2- 172.21.15.89:0/566337520 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f415db10 0x7f84f40b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:16.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.012+0000 7f84f8876700 1 -- 172.21.15.89:0/566337520 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84f4159c60 msgr2=0x7f84f40b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:16.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.012+0000 7f84f8876700 1 --2- 172.21.15.89:0/566337520 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84f4159c60 0x7f84f40b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.012+0000 7f84f8876700 1 -- 172.21.15.89:0/566337520 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f415db10 msgr2=0x7f84f40b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:16.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.012+0000 7f84f8876700 1 --2- 172.21.15.89:0/566337520 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f415db10 0x7f84f40b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.012+0000 7f84f8876700 1 -- 172.21.15.89:0/566337520 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f0009ae0 con 0x7f84f4159890 2023-10-10T20:43:16.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.012+0000 7f84f8876700 1 --2- 172.21.15.89:0/566337520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f4159890 0x7f84f40b5db0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f84f4070e20 tx=0x7f84f0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:16.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.012+0000 7f84e8ff9700 1 -- 172.21.15.89:0/566337520 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f00088b0 con 0x7f84f4159890 2023-10-10T20:43:16.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.012+0000 7f84e8ff9700 1 -- 172.21.15.89:0/566337520 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f0008a30 con 0x7f84f4159890 2023-10-10T20:43:16.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.012+0000 7f84e8ff9700 1 -- 172.21.15.89:0/566337520 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f0008df0 con 0x7f84f4159890 2023-10-10T20:43:16.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.012+0000 7f84faada700 1 -- 172.21.15.89:0/566337520 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f4151910 con 0x7f84f4159890 2023-10-10T20:43:16.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.012+0000 7f84faada700 1 -- 172.21.15.89:0/566337520 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f4152110 con 0x7f84f4159890 2023-10-10T20:43:16.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.013+0000 7f84faada700 1 -- 172.21.15.89:0/566337520 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f4151f60 con 0x7f84f4159890 2023-10-10T20:43:16.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.014+0000 7f84e8ff9700 1 -- 172.21.15.89:0/566337520 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f84f0003040 con 0x7f84f4159890 2023-10-10T20:43:16.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.014+0000 7f84e8ff9700 1 --2- 172.21.15.89:0/566337520 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84e0067320 0x7f84e00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:16.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.014+0000 7f84ebfff700 1 --2- 172.21.15.89:0/566337520 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84e0067320 0x7f84e00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:16.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.014+0000 7f84e8ff9700 1 -- 172.21.15.89:0/566337520 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f84f00867f0 con 0x7f84f4159890 2023-10-10T20:43:16.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.014+0000 7f84ebfff700 1 --2- 172.21.15.89:0/566337520 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84e0067320 0x7f84e00697d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f84f416d100 tx=0x7f84e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:16.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.018+0000 7f84e8ff9700 1 -- 172.21.15.89:0/566337520 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84f0053240 con 0x7f84f4159890 2023-10-10T20:43:16.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.199+0000 7f84faada700 1 -- 172.21.15.89:0/566337520 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "another-data-pool", "var": "pg_num"} v 0) v1 -- 0x7f84f404c3c0 con 0x7f84f4159890 2023-10-10T20:43:16.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.199+0000 7f84e8ff9700 1 -- 172.21.15.89:0/566337520 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "another-data-pool", "var": "pg_num"}]=0 v170) v1 ==== 106+0+11 (secure 0 0 0) 0x7f84f0056860 con 0x7f84f4159890 2023-10-10T20:43:16.202 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:43:16.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.201+0000 7f84faada700 1 -- 172.21.15.89:0/566337520 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84e0067320 msgr2=0x7f84e00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:16.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.201+0000 7f84faada700 1 --2- 172.21.15.89:0/566337520 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84e0067320 0x7f84e00697d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f84f416d100 tx=0x7f84e4008040 comp rx=0 tx=0).stop 2023-10-10T20:43:16.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.201+0000 7f84faada700 1 -- 172.21.15.89:0/566337520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f4159890 msgr2=0x7f84f40b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:16.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.201+0000 7f84faada700 1 --2- 172.21.15.89:0/566337520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f4159890 0x7f84f40b5db0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f84f4070e20 tx=0x7f84f0018000 comp rx=0 tx=0).stop 2023-10-10T20:43:16.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.201+0000 7f84faada700 1 -- 172.21.15.89:0/566337520 shutdown_connections 2023-10-10T20:43:16.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.201+0000 7f84faada700 1 --2- 172.21.15.89:0/566337520 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84e0067320 0x7f84e00697d0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.201+0000 7f84faada700 1 --2- 172.21.15.89:0/566337520 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f415db10 0x7f84f40b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.201+0000 7f84faada700 1 --2- 172.21.15.89:0/566337520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f4159890 0x7f84f40b5db0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.201+0000 7f84faada700 1 --2- 172.21.15.89:0/566337520 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f84f4159c60 0x7f84f40b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.201+0000 7f84faada700 1 -- 172.21.15.89:0/566337520 >> 172.21.15.89:0/566337520 conn(0x7f84f4068e20 msgr2=0x7f84f41533c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:16.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.201+0000 7f84faada700 1 -- 172.21.15.89:0/566337520 shutdown_connections 2023-10-10T20:43:16.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.201+0000 7f84faada700 1 -- 172.21.15.89:0/566337520 wait complete. 2023-10-10T20:43:16.215 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:43:16.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.328+0000 7f5255b8f700 1 Processor -- start 2023-10-10T20:43:16.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.328+0000 7f5255b8f700 1 -- start start 2023-10-10T20:43:16.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.329+0000 7f5255b8f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f525006aa40 0x7f52500cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:16.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.329+0000 7f5255b8f700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52500c91e0 0x7f5250160390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:16.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.329+0000 7f5255b8f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52500cc210 con 0x7f525006bac0 2023-10-10T20:43:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.329+0000 7f5255b8f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52500cc4f0 con 0x7f525006aa40 2023-10-10T20:43:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.329+0000 7f5255b8f700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52500cc7d0 con 0x7f52500c91e0 2023-10-10T20:43:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.330+0000 7f524f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f525006aa40 0x7f52500cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.330+0000 7f524effd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f525006bac0 0x7f525006ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37740/0 (socket says 172.21.15.89:37740) 2023-10-10T20:43:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.330+0000 7f524effd700 1 -- 172.21.15.89:0/1828932919 learned_addr learned my addr 172.21.15.89:0/1828932919 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:16.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.330+0000 7f524ffff700 1 --2- 172.21.15.89:0/1828932919 >> v2:172.21.15.111:3300/0 conn(0x7f52500c91e0 0x7f5250160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:16.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.330+0000 7f524e7fc700 1 -- 172.21.15.89:0/1828932919 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3916592999 0 0) 0x7f52500cc210 con 0x7f525006bac0 2023-10-10T20:43:16.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.330+0000 7f524e7fc700 1 -- 172.21.15.89:0/1828932919 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5238003570 con 0x7f525006bac0 2023-10-10T20:43:16.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.330+0000 7f524f7fe700 1 -- 172.21.15.89:0/1828932919 >> v2:172.21.15.111:3300/0 conn(0x7f52500c91e0 msgr2=0x7f5250160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:16.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.330+0000 7f524f7fe700 1 --2- 172.21.15.89:0/1828932919 >> v2:172.21.15.111:3300/0 conn(0x7f52500c91e0 0x7f5250160390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.330+0000 7f524f7fe700 1 -- 172.21.15.89:0/1828932919 >> v1:172.21.15.89:6789/0 conn(0x7f525006bac0 legacy=0x7f525006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:16.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.330+0000 7f524f7fe700 1 -- 172.21.15.89:0/1828932919 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52500ccab0 con 0x7f525006aa40 2023-10-10T20:43:16.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.330+0000 7f524ffff700 1 --2- 172.21.15.89:0/1828932919 >> v2:172.21.15.111:3300/0 conn(0x7f52500c91e0 0x7f5250160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:16.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.330+0000 7f524f7fe700 1 --2- 172.21.15.89:0/1828932919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f525006aa40 0x7f52500cbc00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f525006e510 tx=0x7f5240009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b8c6abb5147e30 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:16.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.330+0000 7f524e7fc700 1 -- 172.21.15.89:0/1828932919 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524000e070 con 0x7f525006aa40 2023-10-10T20:43:16.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.330+0000 7f524e7fc700 1 -- 172.21.15.89:0/1828932919 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524000bb60 con 0x7f525006aa40 2023-10-10T20:43:16.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.330+0000 7f524e7fc700 1 -- 172.21.15.89:0/1828932919 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240012430 con 0x7f525006aa40 2023-10-10T20:43:16.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.331+0000 7f5255b8f700 1 -- 172.21.15.89:0/1828932919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f525006aa40 msgr2=0x7f52500cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:16.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.331+0000 7f5255b8f700 1 --2- 172.21.15.89:0/1828932919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f525006aa40 0x7f52500cbc00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f525006e510 tx=0x7f5240009780 comp rx=0 tx=0).stop 2023-10-10T20:43:16.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.331+0000 7f5255b8f700 1 -- 172.21.15.89:0/1828932919 shutdown_connections 2023-10-10T20:43:16.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.331+0000 7f5255b8f700 1 --2- 172.21.15.89:0/1828932919 >> v2:172.21.15.111:3300/0 conn(0x7f52500c91e0 0x7f5250160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.331+0000 7f5255b8f700 1 --2- 172.21.15.89:0/1828932919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f525006aa40 0x7f52500cbc00 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.331+0000 7f5255b8f700 1 -- 172.21.15.89:0/1828932919 >> 172.21.15.89:0/1828932919 conn(0x7f52500cd670 msgr2=0x7f52500cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:16.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.331+0000 7f5255b8f700 1 -- 172.21.15.89:0/1828932919 shutdown_connections 2023-10-10T20:43:16.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.331+0000 7f5255b8f700 1 -- 172.21.15.89:0/1828932919 wait complete. 2023-10-10T20:43:16.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.332+0000 7f5255b8f700 1 Processor -- start 2023-10-10T20:43:16.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.332+0000 7f5255b8f700 1 -- start start 2023-10-10T20:43:16.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.333+0000 7f5255b8f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f525006aa40 0x7f52501508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:16.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.333+0000 7f5255b8f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f525006bac0 0x7f5250153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:16.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.333+0000 7f5255b8f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f52500c91e0 0x7f5250153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:16.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.333+0000 7f5255b8f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52500af210 con 0x7f525006aa40 2023-10-10T20:43:16.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.333+0000 7f5255b8f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52500af4f0 con 0x7f525006bac0 2023-10-10T20:43:16.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.333+0000 7f5255b8f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52500b0660 con 0x7f52500c91e0 2023-10-10T20:43:16.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.333+0000 7f524f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f525006aa40 0x7f52501508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:16.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.333+0000 7f524f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f525006aa40 0x7f52501508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35194/0 (socket says 172.21.15.89:35194) 2023-10-10T20:43:16.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.333+0000 7f524f7fe700 1 -- 172.21.15.89:0/4247817978 learned_addr learned my addr 172.21.15.89:0/4247817978 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:16.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.333+0000 7f524effd700 1 --2- 172.21.15.89:0/4247817978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f525006bac0 0x7f5250153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:16.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.333+0000 7f524ffff700 1 --2- 172.21.15.89:0/4247817978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f52500c91e0 0x7f5250153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:16.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.333+0000 7f524f7fe700 1 -- 172.21.15.89:0/4247817978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f52500c91e0 msgr2=0x7f5250153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:16.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.333+0000 7f524f7fe700 1 --2- 172.21.15.89:0/4247817978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f52500c91e0 0x7f5250153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.333+0000 7f524f7fe700 1 -- 172.21.15.89:0/4247817978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f525006bac0 msgr2=0x7f5250153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:16.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.333+0000 7f524f7fe700 1 --2- 172.21.15.89:0/4247817978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f525006bac0 0x7f5250153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.333+0000 7f524f7fe700 1 -- 172.21.15.89:0/4247817978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5240009a20 con 0x7f525006aa40 2023-10-10T20:43:16.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.334+0000 7f524f7fe700 1 --2- 172.21.15.89:0/4247817978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f525006aa40 0x7f52501508c0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f525006c6a0 tx=0x7f5240019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:16.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.334+0000 7f52377fe700 1 -- 172.21.15.89:0/4247817978 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240012af0 con 0x7f525006aa40 2023-10-10T20:43:16.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.334+0000 7f52377fe700 1 -- 172.21.15.89:0/4247817978 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5240021070 con 0x7f525006aa40 2023-10-10T20:43:16.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.334+0000 7f5255b8f700 1 -- 172.21.15.89:0/4247817978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52500b0940 con 0x7f525006aa40 2023-10-10T20:43:16.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.334+0000 7f52377fe700 1 -- 172.21.15.89:0/4247817978 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240012e80 con 0x7f525006aa40 2023-10-10T20:43:16.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.334+0000 7f5255b8f700 1 -- 172.21.15.89:0/4247817978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525016aff0 con 0x7f525006aa40 2023-10-10T20:43:16.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.335+0000 7f5255b8f700 1 -- 172.21.15.89:0/4247817978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52500b04c0 con 0x7f525006aa40 2023-10-10T20:43:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.335+0000 7f52377fe700 1 -- 172.21.15.89:0/4247817978 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5240023030 con 0x7f525006aa40 2023-10-10T20:43:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.336+0000 7f52377fe700 1 --2- 172.21.15.89:0/4247817978 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f523c067320 0x7f523c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.336+0000 7f52377fe700 1 -- 172.21.15.89:0/4247817978 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f5240086de0 con 0x7f525006aa40 2023-10-10T20:43:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.336+0000 7f524effd700 1 --2- 172.21.15.89:0/4247817978 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f523c067320 0x7f523c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.336+0000 7f524effd700 1 --2- 172.21.15.89:0/4247817978 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f523c067320 0x7f523c0697d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5244007fd0 tx=0x7f5244009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:16.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.340+0000 7f52377fe700 1 -- 172.21.15.89:0/4247817978 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52500b04c0 con 0x7f525006aa40 2023-10-10T20:43:16.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.563+0000 7f5255b8f700 1 -- 172.21.15.89:0/4247817978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52500b04c0 con 0x7f525006aa40 2023-10-10T20:43:16.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.564+0000 7f52377fe700 1 -- 172.21.15.89:0/4247817978 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4633 (secure 0 0 0) 0x7f52500b04c0 con 0x7f525006aa40 2023-10-10T20:43:16.566 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:16.566 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":113,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6638,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2350308973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2350308973},{"type":"v1","addr":"172.21.15.111:6835","nonce":2350308973}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":8322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/1051252386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":1051252386},{"type":"v1","addr":"172.21.15.89:6836","nonce":1051252386}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":8325,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3754994248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":3754994248},{"type":"v1","addr":"172.21.15.89:6837","nonce":3754994248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:42:52.707112+0000","modified":"2023-10-10T20:43:12.015439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8337":{"gid":8337,"name":"b","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1733913702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1733913702},{"type":"v1","addr":"172.21.15.111:6834","nonce":1733913702}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"volume_0000000000654221","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-10T20:43:16.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.566+0000 7f5255b8f700 1 -- 172.21.15.89:0/4247817978 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f523c067320 msgr2=0x7f523c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:16.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.566+0000 7f5255b8f700 1 --2- 172.21.15.89:0/4247817978 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f523c067320 0x7f523c0697d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5244007fd0 tx=0x7f5244009040 comp rx=0 tx=0).stop 2023-10-10T20:43:16.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.566+0000 7f5255b8f700 1 -- 172.21.15.89:0/4247817978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f525006aa40 msgr2=0x7f52501508c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:16.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.566+0000 7f5255b8f700 1 --2- 172.21.15.89:0/4247817978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f525006aa40 0x7f52501508c0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f525006c6a0 tx=0x7f5240019000 comp rx=0 tx=0).stop 2023-10-10T20:43:16.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.566+0000 7f5255b8f700 1 -- 172.21.15.89:0/4247817978 shutdown_connections 2023-10-10T20:43:16.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.566+0000 7f5255b8f700 1 --2- 172.21.15.89:0/4247817978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f52500c91e0 0x7f5250153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.566+0000 7f5255b8f700 1 --2- 172.21.15.89:0/4247817978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f525006aa40 0x7f52501508c0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.566+0000 7f5255b8f700 1 --2- 172.21.15.89:0/4247817978 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f523c067320 0x7f523c0697d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.566+0000 7f5255b8f700 1 --2- 172.21.15.89:0/4247817978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f525006bac0 0x7f5250153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.566+0000 7f5255b8f700 1 -- 172.21.15.89:0/4247817978 >> 172.21.15.89:0/4247817978 conn(0x7f52500cd670 msgr2=0x7f52501526e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:16.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.566+0000 7f5255b8f700 1 -- 172.21.15.89:0/4247817978 shutdown_connections 2023-10-10T20:43:16.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.566+0000 7f5255b8f700 1 -- 172.21.15.89:0/4247817978 wait complete. 2023-10-10T20:43:16.571 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 113 2023-10-10T20:43:16.580 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:43:16.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.693+0000 7fccf86b3700 1 Processor -- start 2023-10-10T20:43:16.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.693+0000 7fccf86b3700 1 -- start start 2023-10-10T20:43:16.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.694+0000 7fccf86b3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fccf006bcd0 0x7fccf00caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:16.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.694+0000 7fccf86b3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf015d7e0 con 0x7fccf006c0a0 2023-10-10T20:43:16.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.694+0000 7fccf86b3700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf015dac0 con 0x7fccf015df50 2023-10-10T20:43:16.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.694+0000 7fccf86b3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccf015aa30 con 0x7fccf006bcd0 2023-10-10T20:43:16.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.694+0000 7fccf644f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fccf006bcd0 0x7fccf00caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:16.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.694+0000 7fccf5c4e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fccf006c0a0 0x7fccf015a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37756/0 (socket says 172.21.15.89:37756) 2023-10-10T20:43:16.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.694+0000 7fccf5c4e700 1 -- 172.21.15.89:0/1360269354 learned_addr learned my addr 172.21.15.89:0/1360269354 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:16.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.695+0000 7fccf644f700 1 -- 172.21.15.89:0/1360269354 >> v1:172.21.15.111:6789/0 conn(0x7fccf015df50 legacy=0x7fccf00c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:16.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.695+0000 7fccf644f700 1 -- 172.21.15.89:0/1360269354 >> v1:172.21.15.89:6789/0 conn(0x7fccf006c0a0 legacy=0x7fccf015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:16.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.695+0000 7fccf544d700 1 -- 172.21.15.89:0/1360269354 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951102973 0 0) 0x7fccf015d7e0 con 0x7fccf006c0a0 2023-10-10T20:43:16.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.695+0000 7fccf644f700 1 -- 172.21.15.89:0/1360269354 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf015ad10 con 0x7fccf006bcd0 2023-10-10T20:43:16.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.695+0000 7fccf644f700 1 --2- 172.21.15.89:0/1360269354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf006bcd0 0x7fccf00caa90 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fccec009e60 tx=0x7fccec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae75517e76a00a35 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:16.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.695+0000 7fccf544d700 1 -- 172.21.15.89:0/1360269354 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec00e070 con 0x7fccf006bcd0 2023-10-10T20:43:16.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.695+0000 7fccf544d700 1 -- 172.21.15.89:0/1360269354 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccec00bc30 con 0x7fccf006bcd0 2023-10-10T20:43:16.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.695+0000 7fccf544d700 1 -- 172.21.15.89:0/1360269354 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec012430 con 0x7fccf006bcd0 2023-10-10T20:43:16.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.696+0000 7fccf86b3700 1 -- 172.21.15.89:0/1360269354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf006bcd0 msgr2=0x7fccf00caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:16.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.696+0000 7fccf86b3700 1 --2- 172.21.15.89:0/1360269354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf006bcd0 0x7fccf00caa90 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fccec009e60 tx=0x7fccec009780 comp rx=0 tx=0).stop 2023-10-10T20:43:16.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.696+0000 7fccf86b3700 1 -- 172.21.15.89:0/1360269354 shutdown_connections 2023-10-10T20:43:16.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.696+0000 7fccf86b3700 1 --2- 172.21.15.89:0/1360269354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf006bcd0 0x7fccf00caa90 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.696+0000 7fccf86b3700 1 -- 172.21.15.89:0/1360269354 >> 172.21.15.89:0/1360269354 conn(0x7fccf00ce600 msgr2=0x7fccf00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:16.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.696+0000 7fccf86b3700 1 -- 172.21.15.89:0/1360269354 shutdown_connections 2023-10-10T20:43:16.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.696+0000 7fccf86b3700 1 -- 172.21.15.89:0/1360269354 wait complete. 2023-10-10T20:43:16.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.697+0000 7fccf86b3700 1 Processor -- start 2023-10-10T20:43:16.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.697+0000 7fccf86b3700 1 -- start start 2023-10-10T20:43:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.697+0000 7fccf86b3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf006bcd0 0x7fccf00c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.697+0000 7fccf86b3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccf006c0a0 0x7fccf00a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:16.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.697+0000 7fccf86b3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccf015df50 0x7fccf015bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:16.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.697+0000 7fccf86b3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf00aa0b0 con 0x7fccf006bcd0 2023-10-10T20:43:16.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.697+0000 7fccf86b3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf00c4460 con 0x7fccf015df50 2023-10-10T20:43:16.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.697+0000 7fccf86b3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf016b710 con 0x7fccf006c0a0 2023-10-10T20:43:16.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.698+0000 7fccf644f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf006bcd0 0x7fccf00c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:16.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.698+0000 7fccf644f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf006bcd0 0x7fccf00c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35210/0 (socket says 172.21.15.89:35210) 2023-10-10T20:43:16.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.698+0000 7fccf644f700 1 -- 172.21.15.89:0/876763316 learned_addr learned my addr 172.21.15.89:0/876763316 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:16.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.698+0000 7fccf5c4e700 1 --2- 172.21.15.89:0/876763316 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccf006c0a0 0x7fccf00a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:16.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.698+0000 7fccf6c50700 1 --2- 172.21.15.89:0/876763316 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccf015df50 0x7fccf015bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:16.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.698+0000 7fccf644f700 1 -- 172.21.15.89:0/876763316 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccf006c0a0 msgr2=0x7fccf00a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:16.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.698+0000 7fccf644f700 1 --2- 172.21.15.89:0/876763316 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccf006c0a0 0x7fccf00a98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.698+0000 7fccf644f700 1 -- 172.21.15.89:0/876763316 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccf015df50 msgr2=0x7fccf015bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.698+0000 7fccf644f700 1 --2- 172.21.15.89:0/876763316 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccf015df50 0x7fccf015bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.698+0000 7fccf644f700 1 -- 172.21.15.89:0/876763316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec009a20 con 0x7fccf006bcd0 2023-10-10T20:43:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.698+0000 7fccf5c4e700 1 --2- 172.21.15.89:0/876763316 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccf006c0a0 0x7fccf00a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.698+0000 7fccf644f700 1 --2- 172.21.15.89:0/876763316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf006bcd0 0x7fccf00c3f20 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fccec00b6c0 tx=0x7fccec019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.698+0000 7fcce67fc700 1 -- 172.21.15.89:0/876763316 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec012df0 con 0x7fccf006bcd0 2023-10-10T20:43:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.698+0000 7fcce67fc700 1 -- 172.21.15.89:0/876763316 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccec0263f0 con 0x7fccf006bcd0 2023-10-10T20:43:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.698+0000 7fcce67fc700 1 -- 172.21.15.89:0/876763316 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec0267b0 con 0x7fccf006bcd0 2023-10-10T20:43:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.698+0000 7fccf86b3700 1 -- 172.21.15.89:0/876763316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf00a9e10 con 0x7fccf006bcd0 2023-10-10T20:43:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.698+0000 7fccf86b3700 1 -- 172.21.15.89:0/876763316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf016b0c0 con 0x7fccf006bcd0 2023-10-10T20:43:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.699+0000 7fccf86b3700 1 -- 172.21.15.89:0/876763316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf004c3c0 con 0x7fccf006bcd0 2023-10-10T20:43:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.700+0000 7fcce67fc700 1 -- 172.21.15.89:0/876763316 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fccec026be0 con 0x7fccf006bcd0 2023-10-10T20:43:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.700+0000 7fcce67fc700 1 --2- 172.21.15.89:0/876763316 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fccd4067320 0x7fccd40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.700+0000 7fcce67fc700 1 -- 172.21.15.89:0/876763316 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6180+0+0 (secure 0 0 0) 0x7fccec086b00 con 0x7fccf006bcd0 2023-10-10T20:43:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.700+0000 7fccf5c4e700 1 --2- 172.21.15.89:0/876763316 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fccd4067320 0x7fccd40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.700+0000 7fccf5c4e700 1 --2- 172.21.15.89:0/876763316 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fccd4067320 0x7fccd40697d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fcce0007fd0 tx=0x7fcce0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.704+0000 7fcce67fc700 1 -- 172.21.15.89:0/876763316 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fccec088e60 con 0x7fccf006bcd0 2023-10-10T20:43:16.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.888+0000 7fccf86b3700 1 -- 172.21.15.89:0/876763316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccf015aff0 con 0x7fccf006bcd0 2023-10-10T20:43:16.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.889+0000 7fcce67fc700 1 -- 172.21.15.89:0/876763316 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+17005 (secure 0 0 0) 0x7fccec0534a0 con 0x7fccf006bcd0 2023-10-10T20:43:16.891 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:16.891 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":170,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:43:13.020125+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs.volume_0000000000654221.meta","create_time":"2023-10-10T20:42:51.066663+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000654221"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-10T20:42:51.675525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000654221"}}},{"pool":30,"pool_name":"another-data-pool","create_time":"2023-10-10T20:43:08.234534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000000654221"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:43:16.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.891+0000 7fccf86b3700 1 -- 172.21.15.89:0/876763316 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fccd4067320 msgr2=0x7fccd40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:16.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.891+0000 7fccf86b3700 1 --2- 172.21.15.89:0/876763316 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fccd4067320 0x7fccd40697d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fcce0007fd0 tx=0x7fcce0009040 comp rx=0 tx=0).stop 2023-10-10T20:43:16.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.891+0000 7fccf86b3700 1 -- 172.21.15.89:0/876763316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf006bcd0 msgr2=0x7fccf00c3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:16.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.891+0000 7fccf86b3700 1 --2- 172.21.15.89:0/876763316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf006bcd0 0x7fccf00c3f20 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fccec00b6c0 tx=0x7fccec019000 comp rx=0 tx=0).stop 2023-10-10T20:43:16.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.891+0000 7fccf86b3700 1 -- 172.21.15.89:0/876763316 shutdown_connections 2023-10-10T20:43:16.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.891+0000 7fccf86b3700 1 --2- 172.21.15.89:0/876763316 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fccd4067320 0x7fccd40697d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.891+0000 7fccf86b3700 1 --2- 172.21.15.89:0/876763316 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccf015df50 0x7fccf015bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.891+0000 7fccf86b3700 1 --2- 172.21.15.89:0/876763316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf006bcd0 0x7fccf00c3f20 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.891+0000 7fccf86b3700 1 --2- 172.21.15.89:0/876763316 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fccf006c0a0 0x7fccf00a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:16.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.891+0000 7fccf86b3700 1 -- 172.21.15.89:0/876763316 >> 172.21.15.89:0/876763316 conn(0x7fccf00ce600 msgr2=0x7fccf006a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:16.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.891+0000 7fccf86b3700 1 -- 172.21.15.89:0/876763316 shutdown_connections 2023-10-10T20:43:16.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:16.891+0000 7fccf86b3700 1 -- 172.21.15.89:0/876763316 wait complete. 2023-10-10T20:43:16.905 INFO:tasks.cephfs.filesystem:Destroying file system volume_0000000000654221 and related pools 2023-10-10T20:43:16.906 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:43:17.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.020+0000 7f701cf22700 1 Processor -- start 2023-10-10T20:43:17.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.020+0000 7f701cf22700 1 -- start start 2023-10-10T20:43:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.021+0000 7f701cf22700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70181597f0 0x7f70180b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.021+0000 7f701cf22700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70180b11d0 con 0x7f7018159420 2023-10-10T20:43:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.021+0000 7f701cf22700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70180b14b0 con 0x7f701815d230 2023-10-10T20:43:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.021+0000 7f701cf22700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70180b1790 con 0x7f70181597f0 2023-10-10T20:43:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.021+0000 7f70177fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7018159420 0x7f70180b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37768/0 (socket says 172.21.15.89:37768) 2023-10-10T20:43:17.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.021+0000 7f70177fe700 1 -- 172.21.15.89:0/2371344850 learned_addr learned my addr 172.21.15.89:0/2371344850 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:17.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.021+0000 7f7016ffd700 1 --2- 172.21.15.89:0/2371344850 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70181597f0 0x7f70180b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:17.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.021+0000 7f70167fc700 1 -- 172.21.15.89:0/2371344850 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383037144 0 0) 0x7f70180b11d0 con 0x7f7018159420 2023-10-10T20:43:17.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.021+0000 7f70167fc700 1 -- 172.21.15.89:0/2371344850 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7000003570 con 0x7f7018159420 2023-10-10T20:43:17.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.021+0000 7f70167fc700 1 -- 172.21.15.89:0/2371344850 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3223173351 0 0) 0x7f70180b14b0 con 0x7f701815d230 2023-10-10T20:43:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.021+0000 7f70167fc700 1 -- 172.21.15.89:0/2371344850 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70180b11d0 con 0x7f701815d230 2023-10-10T20:43:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.021+0000 7f7016ffd700 1 -- 172.21.15.89:0/2371344850 >> v1:172.21.15.111:6789/0 conn(0x7f701815d230 legacy=0x7f70180b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.021+0000 7f70167fc700 1 -- 172.21.15.89:0/2371344850 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 888208495 0 0) 0x7f7000003570 con 0x7f7018159420 2023-10-10T20:43:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.022+0000 7f7016ffd700 1 -- 172.21.15.89:0/2371344850 >> v1:172.21.15.89:6789/0 conn(0x7f7018159420 legacy=0x7f70180b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.022+0000 7f7016ffd700 1 -- 172.21.15.89:0/2371344850 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70180b1a70 con 0x7f70181597f0 2023-10-10T20:43:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.022+0000 7f7016ffd700 1 --2- 172.21.15.89:0/2371344850 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70181597f0 0x7f70180b2270 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f700c009910 tx=0x7f700c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4857fd9770352519 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.022+0000 7f70167fc700 1 -- 172.21.15.89:0/2371344850 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70080017f0 con 0x7f70181597f0 2023-10-10T20:43:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.022+0000 7f70167fc700 1 -- 172.21.15.89:0/2371344850 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700c011420 con 0x7f70181597f0 2023-10-10T20:43:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.022+0000 7f70167fc700 1 -- 172.21.15.89:0/2371344850 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c0117c0 con 0x7f70181597f0 2023-10-10T20:43:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.022+0000 7f701cf22700 1 -- 172.21.15.89:0/2371344850 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70181597f0 msgr2=0x7f70180b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:17.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.022+0000 7f701cf22700 1 --2- 172.21.15.89:0/2371344850 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70181597f0 0x7f70180b2270 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f700c009910 tx=0x7f700c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.022+0000 7f701cf22700 1 -- 172.21.15.89:0/2371344850 shutdown_connections 2023-10-10T20:43:17.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.022+0000 7f701cf22700 1 --2- 172.21.15.89:0/2371344850 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70181597f0 0x7f70180b2270 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.022+0000 7f701cf22700 1 -- 172.21.15.89:0/2371344850 >> 172.21.15.89:0/2371344850 conn(0x7f70180d0660 msgr2=0x7f70180d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:17.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.022+0000 7f701cf22700 1 -- 172.21.15.89:0/2371344850 shutdown_connections 2023-10-10T20:43:17.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.022+0000 7f701cf22700 1 -- 172.21.15.89:0/2371344850 wait complete. 2023-10-10T20:43:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.024+0000 7f701cf22700 1 Processor -- start 2023-10-10T20:43:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.024+0000 7f701cf22700 1 -- start start 2023-10-10T20:43:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.024+0000 7f701cf22700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7018159420 0x7f701816d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.024+0000 7f701cf22700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70181597f0 0x7f7018153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.024+0000 7f701cf22700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f701815d230 0x7f7018155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.024+0000 7f701cf22700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701816df00 con 0x7f70181597f0 2023-10-10T20:43:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.024+0000 7f701cf22700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701815bc70 con 0x7f7018159420 2023-10-10T20:43:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.024+0000 7f701cf22700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701815b060 con 0x7f701815d230 2023-10-10T20:43:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.025+0000 7f7016ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70181597f0 0x7f7018153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.025+0000 7f7016ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70181597f0 0x7f7018153b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35220/0 (socket says 172.21.15.89:35220) 2023-10-10T20:43:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.025+0000 7f7016ffd700 1 -- 172.21.15.89:0/1902994370 learned_addr learned my addr 172.21.15.89:0/1902994370 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.025+0000 7f70177fe700 1 --2- 172.21.15.89:0/1902994370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7018159420 0x7f701816d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.025+0000 7f7017fff700 1 --2- 172.21.15.89:0/1902994370 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f701815d230 0x7f7018155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.025+0000 7f7016ffd700 1 -- 172.21.15.89:0/1902994370 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f701815d230 msgr2=0x7f7018155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.025+0000 7f7016ffd700 1 --2- 172.21.15.89:0/1902994370 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f701815d230 0x7f7018155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.025+0000 7f7016ffd700 1 -- 172.21.15.89:0/1902994370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7018159420 msgr2=0x7f701816d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.025+0000 7f7016ffd700 1 --2- 172.21.15.89:0/1902994370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7018159420 0x7f701816d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.025+0000 7f7016ffd700 1 -- 172.21.15.89:0/1902994370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c009420 con 0x7f70181597f0 2023-10-10T20:43:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.025+0000 7f7017fff700 1 --2- 172.21.15.89:0/1902994370 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f701815d230 0x7f7018155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:17.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.025+0000 7f7016ffd700 1 --2- 172.21.15.89:0/1902994370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70181597f0 0x7f7018153b30 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f700c0030c0 tx=0x7f700c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:17.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.025+0000 7f6fff7fe700 1 -- 172.21.15.89:0/1902994370 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c01cc50 con 0x7f70181597f0 2023-10-10T20:43:17.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.025+0000 7f6fff7fe700 1 -- 172.21.15.89:0/1902994370 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700c01cdb0 con 0x7f70181597f0 2023-10-10T20:43:17.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.025+0000 7f701cf22700 1 -- 172.21.15.89:0/1902994370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f701815b340 con 0x7f70181597f0 2023-10-10T20:43:17.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.025+0000 7f6fff7fe700 1 -- 172.21.15.89:0/1902994370 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c01a6a0 con 0x7f70181597f0 2023-10-10T20:43:17.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.025+0000 7f701cf22700 1 -- 172.21.15.89:0/1902994370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701815b900 con 0x7f70181597f0 2023-10-10T20:43:17.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.026+0000 7f701cf22700 1 -- 172.21.15.89:0/1902994370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70180c5400 con 0x7f70181597f0 2023-10-10T20:43:17.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.027+0000 7f6fff7fe700 1 -- 172.21.15.89:0/1902994370 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f700c01aa90 con 0x7f70181597f0 2023-10-10T20:43:17.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.027+0000 7f6fff7fe700 1 --2- 172.21.15.89:0/1902994370 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7004067320 0x7f70040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:17.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.027+0000 7f6fff7fe700 1 -- 172.21.15.89:0/1902994370 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f700c086bf0 con 0x7f70181597f0 2023-10-10T20:43:17.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.027+0000 7f70177fe700 1 --2- 172.21.15.89:0/1902994370 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7004067320 0x7f70040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:17.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.027+0000 7f70177fe700 1 --2- 172.21.15.89:0/1902994370 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7004067320 0x7f70040697d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7008001560 tx=0x7f7008001960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:17.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.031+0000 7f6fff7fe700 1 -- 172.21.15.89:0/1902994370 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70180c5400 con 0x7f70181597f0 2023-10-10T20:43:17.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.258+0000 7f701cf22700 1 -- 172.21.15.89:0/1902994370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70180c5400 con 0x7f70181597f0 2023-10-10T20:43:17.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.259+0000 7f6fff7fe700 1 -- 172.21.15.89:0/1902994370 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4633 (secure 0 0 0) 0x7f70180c5400 con 0x7f70181597f0 2023-10-10T20:43:17.261 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:17.261 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":113,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6638,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2350308973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2350308973},{"type":"v1","addr":"172.21.15.111:6835","nonce":2350308973}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":8322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/1051252386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":1051252386},{"type":"v1","addr":"172.21.15.89:6836","nonce":1051252386}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":8325,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3754994248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":3754994248},{"type":"v1","addr":"172.21.15.89:6837","nonce":3754994248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:42:52.707112+0000","modified":"2023-10-10T20:43:12.015439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8337":{"gid":8337,"name":"b","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1733913702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1733913702},{"type":"v1","addr":"172.21.15.111:6834","nonce":1733913702}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"volume_0000000000654221","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-10T20:43:17.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.261+0000 7f701cf22700 1 -- 172.21.15.89:0/1902994370 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7004067320 msgr2=0x7f70040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:17.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.261+0000 7f701cf22700 1 --2- 172.21.15.89:0/1902994370 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7004067320 0x7f70040697d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7008001560 tx=0x7f7008001960 comp rx=0 tx=0).stop 2023-10-10T20:43:17.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.261+0000 7f701cf22700 1 -- 172.21.15.89:0/1902994370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70181597f0 msgr2=0x7f7018153b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:17.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.261+0000 7f701cf22700 1 --2- 172.21.15.89:0/1902994370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70181597f0 0x7f7018153b30 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f700c0030c0 tx=0x7f700c016000 comp rx=0 tx=0).stop 2023-10-10T20:43:17.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.261+0000 7f701cf22700 1 -- 172.21.15.89:0/1902994370 shutdown_connections 2023-10-10T20:43:17.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.261+0000 7f701cf22700 1 --2- 172.21.15.89:0/1902994370 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f701815d230 0x7f7018155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.261+0000 7f701cf22700 1 --2- 172.21.15.89:0/1902994370 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7004067320 0x7f70040697d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.261+0000 7f701cf22700 1 --2- 172.21.15.89:0/1902994370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7018159420 0x7f701816d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.261+0000 7f701cf22700 1 --2- 172.21.15.89:0/1902994370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70181597f0 0x7f7018153b30 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.261+0000 7f701cf22700 1 -- 172.21.15.89:0/1902994370 >> 172.21.15.89:0/1902994370 conn(0x7f70180d0660 msgr2=0x7f70181552b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:17.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.261+0000 7f701cf22700 1 -- 172.21.15.89:0/1902994370 shutdown_connections 2023-10-10T20:43:17.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.261+0000 7f701cf22700 1 -- 172.21.15.89:0/1902994370 wait complete. 2023-10-10T20:43:17.266 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 113 2023-10-10T20:43:17.275 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:43:17.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.392+0000 7f1e6c9b4700 1 Processor -- start 2023-10-10T20:43:17.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.392+0000 7f1e6c9b4700 1 -- start start 2023-10-10T20:43:17.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.393+0000 7f1e6c9b4700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1e680c7bd0 0x7f1e6815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:17.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.393+0000 7f1e6c9b4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e6806bbe0 0x7f1e6806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:17.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.393+0000 7f1e6c9b4700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e6815af50 con 0x7f1e681598d0 2023-10-10T20:43:17.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.393+0000 7f1e6c9b4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e6806c540 con 0x7f1e6806bbe0 2023-10-10T20:43:17.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.393+0000 7f1e6c9b4700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e6806e980 con 0x7f1e680c7bd0 2023-10-10T20:43:17.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.393+0000 7f1e6759e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e6806bbe0 0x7f1e6806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:17.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.393+0000 7f1e6759e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e6806bbe0 0x7f1e6806c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35236/0 (socket says 172.21.15.89:35236) 2023-10-10T20:43:17.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.393+0000 7f1e6759e700 1 -- 172.21.15.89:0/73584576 learned_addr learned my addr 172.21.15.89:0/73584576 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:17.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.394+0000 7f1e5e59c700 1 --2- 172.21.15.89:0/73584576 >> v2:172.21.15.111:3300/0 conn(0x7f1e680c7bd0 0x7f1e6815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:17.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.394+0000 7f1e6759e700 1 -- 172.21.15.89:0/73584576 >> v2:172.21.15.111:3300/0 conn(0x7f1e680c7bd0 msgr2=0x7f1e6815aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:17.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.394+0000 7f1e6759e700 1 --2- 172.21.15.89:0/73584576 >> v2:172.21.15.111:3300/0 conn(0x7f1e680c7bd0 0x7f1e6815aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.394+0000 7f1e6759e700 1 -- 172.21.15.89:0/73584576 >> v1:172.21.15.111:6789/0 conn(0x7f1e681598d0 legacy=0x7f1e6806ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:17.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.394+0000 7f1e6759e700 1 -- 172.21.15.89:0/73584576 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e680ce180 con 0x7f1e6806bbe0 2023-10-10T20:43:17.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.394+0000 7f1e5e59c700 1 --2- 172.21.15.89:0/73584576 >> v2:172.21.15.111:3300/0 conn(0x7f1e680c7bd0 0x7f1e6815aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:17.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.394+0000 7f1e6759e700 1 --2- 172.21.15.89:0/73584576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e6806bbe0 0x7f1e6806c000 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f1e680659d0 tx=0x7f1e60009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3dfaa2492bb3575 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:17.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.394+0000 7f1e6659c700 1 -- 172.21.15.89:0/73584576 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6000e070 con 0x7f1e6806bbe0 2023-10-10T20:43:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.394+0000 7f1e6659c700 1 -- 172.21.15.89:0/73584576 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6000ba10 con 0x7f1e6806bbe0 2023-10-10T20:43:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.394+0000 7f1e6659c700 1 -- 172.21.15.89:0/73584576 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e600092e0 con 0x7f1e6806bbe0 2023-10-10T20:43:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.394+0000 7f1e6c9b4700 1 -- 172.21.15.89:0/73584576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e6806bbe0 msgr2=0x7f1e6806c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.394+0000 7f1e6c9b4700 1 --2- 172.21.15.89:0/73584576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e6806bbe0 0x7f1e6806c000 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f1e680659d0 tx=0x7f1e60009840 comp rx=0 tx=0).stop 2023-10-10T20:43:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.394+0000 7f1e6c9b4700 1 -- 172.21.15.89:0/73584576 shutdown_connections 2023-10-10T20:43:17.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.394+0000 7f1e6c9b4700 1 --2- 172.21.15.89:0/73584576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e6806bbe0 0x7f1e6806c000 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.394+0000 7f1e6c9b4700 1 --2- 172.21.15.89:0/73584576 >> v2:172.21.15.111:3300/0 conn(0x7f1e680c7bd0 0x7f1e6815aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.394+0000 7f1e6c9b4700 1 -- 172.21.15.89:0/73584576 >> 172.21.15.89:0/73584576 conn(0x7f1e6806b490 msgr2=0x7f1e680cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:17.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.394+0000 7f1e6c9b4700 1 -- 172.21.15.89:0/73584576 shutdown_connections 2023-10-10T20:43:17.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.394+0000 7f1e6c9b4700 1 -- 172.21.15.89:0/73584576 wait complete. 2023-10-10T20:43:17.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.396+0000 7f1e6c9b4700 1 Processor -- start 2023-10-10T20:43:17.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.396+0000 7f1e6c9b4700 1 -- start start 2023-10-10T20:43:17.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.396+0000 7f1e6c9b4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e6806bbe0 0x7f1e680c6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:17.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.396+0000 7f1e6c9b4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1e680c7bd0 0x7f1e6816a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:17.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.396+0000 7f1e6c9b4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e681598d0 0x7f1e680c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:17.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.396+0000 7f1e6c9b4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e680cbbe0 con 0x7f1e6806bbe0 2023-10-10T20:43:17.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.396+0000 7f1e6c9b4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e680cafd0 con 0x7f1e681598d0 2023-10-10T20:43:17.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.396+0000 7f1e6c9b4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e680cb2b0 con 0x7f1e680c7bd0 2023-10-10T20:43:17.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.396+0000 7f1e66d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e6806bbe0 0x7f1e680c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:17.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.396+0000 7f1e66d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e6806bbe0 0x7f1e680c6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35238/0 (socket says 172.21.15.89:35238) 2023-10-10T20:43:17.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.396+0000 7f1e66d9d700 1 -- 172.21.15.89:0/1553560496 learned_addr learned my addr 172.21.15.89:0/1553560496 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:17.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.397+0000 7f1e6759e700 1 --2- 172.21.15.89:0/1553560496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e681598d0 0x7f1e680c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:17.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.397+0000 7f1e5e59c700 1 --2- 172.21.15.89:0/1553560496 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1e680c7bd0 0x7f1e6816a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:17.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.397+0000 7f1e66d9d700 1 -- 172.21.15.89:0/1553560496 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1e680c7bd0 msgr2=0x7f1e6816a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:17.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.397+0000 7f1e66d9d700 1 --2- 172.21.15.89:0/1553560496 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1e680c7bd0 0x7f1e6816a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.397+0000 7f1e66d9d700 1 -- 172.21.15.89:0/1553560496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e681598d0 msgr2=0x7f1e680c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:17.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.397+0000 7f1e66d9d700 1 --2- 172.21.15.89:0/1553560496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e681598d0 0x7f1e680c4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.397+0000 7f1e66d9d700 1 -- 172.21.15.89:0/1553560496 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e60009ae0 con 0x7f1e6806bbe0 2023-10-10T20:43:17.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.397+0000 7f1e6759e700 1 --2- 172.21.15.89:0/1553560496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e681598d0 0x7f1e680c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:17.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.397+0000 7f1e66d9d700 1 --2- 172.21.15.89:0/1553560496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e6806bbe0 0x7f1e680c6510 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800b5e0 tx=0x7f1e58009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:17.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.397+0000 7f1e5f7fe700 1 -- 172.21.15.89:0/1553560496 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5800f510 con 0x7f1e6806bbe0 2023-10-10T20:43:17.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.397+0000 7f1e5f7fe700 1 -- 172.21.15.89:0/1553560496 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5800f670 con 0x7f1e6806bbe0 2023-10-10T20:43:17.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.397+0000 7f1e6c9b4700 1 -- 172.21.15.89:0/1553560496 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e680cb590 con 0x7f1e6806bbe0 2023-10-10T20:43:17.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.397+0000 7f1e5f7fe700 1 -- 172.21.15.89:0/1553560496 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5800fa30 con 0x7f1e6806bbe0 2023-10-10T20:43:17.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.397+0000 7f1e6c9b4700 1 -- 172.21.15.89:0/1553560496 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e680c6a50 con 0x7f1e6806bbe0 2023-10-10T20:43:17.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.398+0000 7f1e6c9b4700 1 -- 172.21.15.89:0/1553560496 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e6804c3c0 con 0x7f1e6806bbe0 2023-10-10T20:43:17.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.399+0000 7f1e5f7fe700 1 -- 172.21.15.89:0/1553560496 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1e5801a400 con 0x7f1e6806bbe0 2023-10-10T20:43:17.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.399+0000 7f1e5f7fe700 1 --2- 172.21.15.89:0/1553560496 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1e48067320 0x7f1e480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:17.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.399+0000 7f1e5f7fe700 1 -- 172.21.15.89:0/1553560496 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f1e580847f0 con 0x7f1e6806bbe0 2023-10-10T20:43:17.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.399+0000 7f1e5e59c700 1 --2- 172.21.15.89:0/1553560496 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1e48067320 0x7f1e480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:17.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.399+0000 7f1e5e59c700 1 --2- 172.21.15.89:0/1553560496 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1e48067320 0x7f1e480697d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000ac00 tx=0x7f1e50006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:17.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.403+0000 7f1e5f7fe700 1 -- 172.21.15.89:0/1553560496 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e58051420 con 0x7f1e6806bbe0 2023-10-10T20:43:17.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.628+0000 7f1e6c9b4700 1 -- 172.21.15.89:0/1553560496 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e6806dfd0 con 0x7f1e6806bbe0 2023-10-10T20:43:17.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.629+0000 7f1e5f7fe700 1 -- 172.21.15.89:0/1553560496 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4633 (secure 0 0 0) 0x7f1e58054a40 con 0x7f1e6806bbe0 2023-10-10T20:43:17.631 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:17.631 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":113,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6638,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2350308973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2350308973},{"type":"v1","addr":"172.21.15.111:6835","nonce":2350308973}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":8322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/1051252386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":1051252386},{"type":"v1","addr":"172.21.15.89:6836","nonce":1051252386}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":8325,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3754994248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":3754994248},{"type":"v1","addr":"172.21.15.89:6837","nonce":3754994248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:42:52.707112+0000","modified":"2023-10-10T20:43:12.015439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8337":{"gid":8337,"name":"b","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6834/1733913702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1733913702},{"type":"v1","addr":"172.21.15.111:6834","nonce":1733913702}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"volume_0000000000654221","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-10T20:43:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.630+0000 7f1e6c9b4700 1 -- 172.21.15.89:0/1553560496 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1e48067320 msgr2=0x7f1e480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.630+0000 7f1e6c9b4700 1 --2- 172.21.15.89:0/1553560496 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1e48067320 0x7f1e480697d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000ac00 tx=0x7f1e50006f90 comp rx=0 tx=0).stop 2023-10-10T20:43:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.630+0000 7f1e6c9b4700 1 -- 172.21.15.89:0/1553560496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e6806bbe0 msgr2=0x7f1e680c6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.630+0000 7f1e6c9b4700 1 --2- 172.21.15.89:0/1553560496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e6806bbe0 0x7f1e680c6510 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800b5e0 tx=0x7f1e58009120 comp rx=0 tx=0).stop 2023-10-10T20:43:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.631+0000 7f1e6c9b4700 1 -- 172.21.15.89:0/1553560496 shutdown_connections 2023-10-10T20:43:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.631+0000 7f1e6c9b4700 1 --2- 172.21.15.89:0/1553560496 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1e48067320 0x7f1e480697d0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.631+0000 7f1e6c9b4700 1 --2- 172.21.15.89:0/1553560496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e681598d0 0x7f1e680c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.631+0000 7f1e6c9b4700 1 --2- 172.21.15.89:0/1553560496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e6806bbe0 0x7f1e680c6510 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.631+0000 7f1e6c9b4700 1 --2- 172.21.15.89:0/1553560496 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1e680c7bd0 0x7f1e6816a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.631+0000 7f1e6c9b4700 1 -- 172.21.15.89:0/1553560496 >> 172.21.15.89:0/1553560496 conn(0x7f1e6806b490 msgr2=0x7f1e6815a810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:17.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.631+0000 7f1e6c9b4700 1 -- 172.21.15.89:0/1553560496 shutdown_connections 2023-10-10T20:43:17.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.631+0000 7f1e6c9b4700 1 -- 172.21.15.89:0/1553560496 wait complete. 2023-10-10T20:43:17.635 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 113 2023-10-10T20:43:17.645 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:43:17.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.758+0000 7f423a74b700 1 Processor -- start 2023-10-10T20:43:17.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.758+0000 7f423a74b700 1 -- start start 2023-10-10T20:43:17.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.759+0000 7f423a74b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4234159890 0x7f423415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:17.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.759+0000 7f423a74b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4234159c60 0x7f423415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:17.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.759+0000 7f423a74b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f423415db10 0x7f4234069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:17.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.759+0000 7f423a74b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423415a160 con 0x7f423415db10 2023-10-10T20:43:17.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.759+0000 7f423a74b700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42340d13a0 con 0x7f4234159890 2023-10-10T20:43:17.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.759+0000 7f423a74b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423415e140 con 0x7f4234159c60 2023-10-10T20:43:17.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.759+0000 7f4238ce8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f423415db10 0x7f4234069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:17.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.759+0000 7f4238ce8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f423415db10 0x7f4234069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35250/0 (socket says 172.21.15.89:35250) 2023-10-10T20:43:17.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.759+0000 7f4238ce8700 1 -- 172.21.15.89:0/3908850205 learned_addr learned my addr 172.21.15.89:0/3908850205 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:17.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.759+0000 7f42337fe700 1 --2- 172.21.15.89:0/3908850205 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4234159c60 0x7f423415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:17.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.759+0000 7f4233fff700 1 --2- 172.21.15.89:0/3908850205 >> v2:172.21.15.111:3300/0 conn(0x7f4234159890 0x7f423415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:17.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.760+0000 7f4238ce8700 1 -- 172.21.15.89:0/3908850205 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4234159c60 msgr2=0x7f423415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:17.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.760+0000 7f4238ce8700 1 --2- 172.21.15.89:0/3908850205 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4234159c60 0x7f423415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.760+0000 7f4238ce8700 1 -- 172.21.15.89:0/3908850205 >> v2:172.21.15.111:3300/0 conn(0x7f4234159890 msgr2=0x7f423415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:17.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.760+0000 7f4238ce8700 1 --2- 172.21.15.89:0/3908850205 >> v2:172.21.15.111:3300/0 conn(0x7f4234159890 0x7f423415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.760+0000 7f4238ce8700 1 -- 172.21.15.89:0/3908850205 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423415a3d0 con 0x7f423415db10 2023-10-10T20:43:17.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.760+0000 7f4238ce8700 1 --2- 172.21.15.89:0/3908850205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f423415db10 0x7f4234069560 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f42340d1090 tx=0x7f422c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606231719bb10a6b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:17.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.760+0000 7f4232ffd700 1 -- 172.21.15.89:0/3908850205 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422c00e070 con 0x7f423415db10 2023-10-10T20:43:17.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.760+0000 7f4232ffd700 1 -- 172.21.15.89:0/3908850205 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422c00ba50 con 0x7f423415db10 2023-10-10T20:43:17.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.760+0000 7f4232ffd700 1 -- 172.21.15.89:0/3908850205 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422c0092e0 con 0x7f423415db10 2023-10-10T20:43:17.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.760+0000 7f423a74b700 1 -- 172.21.15.89:0/3908850205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f423415db10 msgr2=0x7f4234069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:17.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.760+0000 7f423a74b700 1 --2- 172.21.15.89:0/3908850205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f423415db10 0x7f4234069560 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f42340d1090 tx=0x7f422c009840 comp rx=0 tx=0).stop 2023-10-10T20:43:17.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.760+0000 7f423a74b700 1 -- 172.21.15.89:0/3908850205 shutdown_connections 2023-10-10T20:43:17.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.760+0000 7f423a74b700 1 --2- 172.21.15.89:0/3908850205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f423415db10 0x7f4234069560 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.760+0000 7f423a74b700 1 --2- 172.21.15.89:0/3908850205 >> v2:172.21.15.111:3300/0 conn(0x7f4234159890 0x7f423415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.760+0000 7f423a74b700 1 --2- 172.21.15.89:0/3908850205 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4234159c60 0x7f423415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.760+0000 7f423a74b700 1 -- 172.21.15.89:0/3908850205 >> 172.21.15.89:0/3908850205 conn(0x7f4234068e20 msgr2=0x7f42340d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:17.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.760+0000 7f423a74b700 1 -- 172.21.15.89:0/3908850205 shutdown_connections 2023-10-10T20:43:17.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.761+0000 7f423a74b700 1 -- 172.21.15.89:0/3908850205 wait complete. 2023-10-10T20:43:17.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.762+0000 7f423a74b700 1 Processor -- start 2023-10-10T20:43:17.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.762+0000 7f423a74b700 1 -- start start 2023-10-10T20:43:17.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.762+0000 7f423a74b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4234159890 0x7f42340cf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:17.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.762+0000 7f423a74b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4234159c60 0x7f42340b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:17.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.762+0000 7f423a74b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f423415db10 0x7f42340b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:17.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.763+0000 7f423a74b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423416dda0 con 0x7f4234159890 2023-10-10T20:43:17.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.763+0000 7f423a74b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4234151f40 con 0x7f423415db10 2023-10-10T20:43:17.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.763+0000 7f423a74b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423416d470 con 0x7f4234159c60 2023-10-10T20:43:17.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.763+0000 7f4233fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4234159890 0x7f42340cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:17.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.763+0000 7f4233fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4234159890 0x7f42340cf920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35252/0 (socket says 172.21.15.89:35252) 2023-10-10T20:43:17.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.763+0000 7f4233fff700 1 -- 172.21.15.89:0/535771694 learned_addr learned my addr 172.21.15.89:0/535771694 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:17.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.763+0000 7f4238ce8700 1 --2- 172.21.15.89:0/535771694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f423415db10 0x7f42340b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:17.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.763+0000 7f42337fe700 1 --2- 172.21.15.89:0/535771694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4234159c60 0x7f42340b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:17.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.763+0000 7f4233fff700 1 -- 172.21.15.89:0/535771694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4234159c60 msgr2=0x7f42340b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:17.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.763+0000 7f4233fff700 1 --2- 172.21.15.89:0/535771694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4234159c60 0x7f42340b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.763+0000 7f4233fff700 1 -- 172.21.15.89:0/535771694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f423415db10 msgr2=0x7f42340b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:17.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.763+0000 7f4233fff700 1 --2- 172.21.15.89:0/535771694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f423415db10 0x7f42340b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.763+0000 7f4233fff700 1 -- 172.21.15.89:0/535771694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c009ae0 con 0x7f4234159890 2023-10-10T20:43:17.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.763+0000 7f4233fff700 1 --2- 172.21.15.89:0/535771694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4234159890 0x7f42340cf920 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f422400d890 tx=0x7f4224008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:17.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.763+0000 7f421bfff700 1 -- 172.21.15.89:0/535771694 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422400b600 con 0x7f4234159890 2023-10-10T20:43:17.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.763+0000 7f421bfff700 1 -- 172.21.15.89:0/535771694 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422400b760 con 0x7f4234159890 2023-10-10T20:43:17.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.763+0000 7f423a74b700 1 -- 172.21.15.89:0/535771694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423416d750 con 0x7f4234159890 2023-10-10T20:43:17.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.763+0000 7f421bfff700 1 -- 172.21.15.89:0/535771694 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422400bae0 con 0x7f4234159890 2023-10-10T20:43:17.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.763+0000 7f423a74b700 1 -- 172.21.15.89:0/535771694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42340b4390 con 0x7f4234159890 2023-10-10T20:43:17.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.764+0000 7f423a74b700 1 -- 172.21.15.89:0/535771694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423404c3c0 con 0x7f4234159890 2023-10-10T20:43:17.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.765+0000 7f421bfff700 1 -- 172.21.15.89:0/535771694 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f422401e460 con 0x7f4234159890 2023-10-10T20:43:17.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.765+0000 7f421bfff700 1 --2- 172.21.15.89:0/535771694 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4220067320 0x7f42200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:17.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.765+0000 7f421bfff700 1 -- 172.21.15.89:0/535771694 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f4224085500 con 0x7f4234159890 2023-10-10T20:43:17.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.765+0000 7f42337fe700 1 --2- 172.21.15.89:0/535771694 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4220067320 0x7f42200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:17.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.765+0000 7f42337fe700 1 --2- 172.21.15.89:0/535771694 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4220067320 0x7f42200697d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f42340cad90 tx=0x7f4228008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:17.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.769+0000 7f421bfff700 1 -- 172.21.15.89:0/535771694 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4224051fb0 con 0x7f4234159890 2023-10-10T20:43:17.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.955+0000 7f423a74b700 1 -- 172.21.15.89:0/535771694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f423406a5b0 con 0x7f4234159890 2023-10-10T20:43:17.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.955+0000 7f421bfff700 1 -- 172.21.15.89:0/535771694 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+17005 (secure 0 0 0) 0x7f42240555d0 con 0x7f4234159890 2023-10-10T20:43:17.958 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:17.958 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":170,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:43:13.020125+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs.volume_0000000000654221.meta","create_time":"2023-10-10T20:42:51.066663+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000654221"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-10T20:42:51.675525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000654221"}}},{"pool":30,"pool_name":"another-data-pool","create_time":"2023-10-10T20:43:08.234534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000000654221"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:43:17.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.957+0000 7f423a74b700 1 -- 172.21.15.89:0/535771694 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4220067320 msgr2=0x7f42200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:17.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.957+0000 7f423a74b700 1 --2- 172.21.15.89:0/535771694 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4220067320 0x7f42200697d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f42340cad90 tx=0x7f4228008040 comp rx=0 tx=0).stop 2023-10-10T20:43:17.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.957+0000 7f423a74b700 1 -- 172.21.15.89:0/535771694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4234159890 msgr2=0x7f42340cf920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:17.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.957+0000 7f423a74b700 1 --2- 172.21.15.89:0/535771694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4234159890 0x7f42340cf920 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f422400d890 tx=0x7f4224008040 comp rx=0 tx=0).stop 2023-10-10T20:43:17.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.957+0000 7f423a74b700 1 -- 172.21.15.89:0/535771694 shutdown_connections 2023-10-10T20:43:17.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.957+0000 7f423a74b700 1 --2- 172.21.15.89:0/535771694 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4220067320 0x7f42200697d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.957+0000 7f423a74b700 1 --2- 172.21.15.89:0/535771694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f423415db10 0x7f42340b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.957+0000 7f423a74b700 1 --2- 172.21.15.89:0/535771694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4234159890 0x7f42340cf920 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.957+0000 7f423a74b700 1 --2- 172.21.15.89:0/535771694 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4234159c60 0x7f42340b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:17.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.957+0000 7f423a74b700 1 -- 172.21.15.89:0/535771694 >> 172.21.15.89:0/535771694 conn(0x7f4234068e20 msgr2=0x7f42340d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:17.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.958+0000 7f423a74b700 1 -- 172.21.15.89:0/535771694 shutdown_connections 2023-10-10T20:43:17.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:17.958+0000 7f423a74b700 1 -- 172.21.15.89:0/535771694 wait complete. 2023-10-10T20:43:17.972 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail volume_0000000000654221 2023-10-10T20:43:18.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.087+0000 7f0782bb2700 1 Processor -- start 2023-10-10T20:43:18.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.087+0000 7f0782bb2700 1 -- start start 2023-10-10T20:43:18.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.088+0000 7f0782bb2700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f077c068550 0x7f077c068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:18.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.088+0000 7f0782bb2700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f077c06f810 0x7f077c159510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:18.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.088+0000 7f0782bb2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c06ec70 con 0x7f077c15e010 2023-10-10T20:43:18.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.088+0000 7f0782bb2700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077c0b3430 con 0x7f077c06f810 2023-10-10T20:43:18.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.088+0000 7f0782bb2700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c06fc90 con 0x7f077c068550 2023-10-10T20:43:18.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.088+0000 7f0773fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f077c15e010 0x7f077c06f100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37782/0 (socket says 172.21.15.89:37782) 2023-10-10T20:43:18.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.088+0000 7f0773fff700 1 -- 172.21.15.89:0/3594587121 learned_addr learned my addr 172.21.15.89:0/3594587121 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:18.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f078094e700 1 --2- 172.21.15.89:0/3594587121 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f077c068550 0x7f077c068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:18.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f07737fe700 1 -- 172.21.15.89:0/3594587121 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 963314266 0 0) 0x7f077c06ec70 con 0x7f077c15e010 2023-10-10T20:43:18.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f078114f700 1 --2- 172.21.15.89:0/3594587121 >> v2:172.21.15.111:3300/0 conn(0x7f077c06f810 0x7f077c159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:18.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f07737fe700 1 -- 172.21.15.89:0/3594587121 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0764003570 con 0x7f077c15e010 2023-10-10T20:43:18.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f07737fe700 1 -- 172.21.15.89:0/3594587121 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2308955623 0 0) 0x7f0764003570 con 0x7f077c15e010 2023-10-10T20:43:18.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f07737fe700 1 -- 172.21.15.89:0/3594587121 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f077c068550 msgr2=0x7f077c068920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:18.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f07737fe700 1 --2- 172.21.15.89:0/3594587121 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f077c068550 0x7f077c068920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:18.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f07737fe700 1 -- 172.21.15.89:0/3594587121 >> v2:172.21.15.111:3300/0 conn(0x7f077c06f810 msgr2=0x7f077c159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:18.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f07737fe700 1 --2- 172.21.15.89:0/3594587121 >> v2:172.21.15.111:3300/0 conn(0x7f077c06f810 0x7f077c159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:18.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f07737fe700 1 -- 172.21.15.89:0/3594587121 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c0b2e70 con 0x7f077c15e010 2023-10-10T20:43:18.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f07737fe700 1 -- 172.21.15.89:0/3594587121 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f076c001470 con 0x7f077c15e010 2023-10-10T20:43:18.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f07737fe700 1 -- 172.21.15.89:0/3594587121 >> v1:172.21.15.89:6789/0 conn(0x7f077c15e010 legacy=0x7f077c06f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:18.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f07737fe700 1 --2- 172.21.15.89:0/3594587121 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f077c068550 0x7f0764003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:18.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f07737fe700 1 --2- 172.21.15.89:0/3594587121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0764003fe0 0x7f0764004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:18.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f07737fe700 1 --2- 172.21.15.89:0/3594587121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0764004d50 0x7f07640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:18.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f07737fe700 1 -- 172.21.15.89:0/3594587121 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076400ba90 con 0x7f0764003fe0 2023-10-10T20:43:18.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f07737fe700 1 -- 172.21.15.89:0/3594587121 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076400bc00 con 0x7f0764004d50 2023-10-10T20:43:18.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f07737fe700 1 -- 172.21.15.89:0/3594587121 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076400bdc0 con 0x7f077c068550 2023-10-10T20:43:18.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f078114f700 1 --2- 172.21.15.89:0/3594587121 >> v2:172.21.15.111:3300/0 conn(0x7f077c06f810 0x7f077c159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:18.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f0773fff700 1 --2- 172.21.15.89:0/3594587121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0764003fe0 0x7f0764004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:18.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f078094e700 1 --2- 172.21.15.89:0/3594587121 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f077c068550 0x7f0764003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:18.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f078114f700 1 --2- 172.21.15.89:0/3594587121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0764004d50 0x7f07640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:18.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f0773fff700 1 -- 172.21.15.89:0/3594587121 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f077c068550 msgr2=0x7f0764003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:18.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f0773fff700 1 --2- 172.21.15.89:0/3594587121 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f077c068550 0x7f0764003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:18.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f0773fff700 1 -- 172.21.15.89:0/3594587121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0764004d50 msgr2=0x7f07640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:18.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f0773fff700 1 --2- 172.21.15.89:0/3594587121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0764004d50 0x7f07640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:18.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.089+0000 7f0773fff700 1 -- 172.21.15.89:0/3594587121 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0764003570 con 0x7f0764003fe0 2023-10-10T20:43:18.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.090+0000 7f0773fff700 1 --2- 172.21.15.89:0/3594587121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0764003fe0 0x7f0764004810 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f076c00c390 tx=0x7f076c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5555146535de3b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:18.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.090+0000 7f07737fe700 1 -- 172.21.15.89:0/3594587121 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c00cc60 con 0x7f0764003fe0 2023-10-10T20:43:18.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.090+0000 7f07737fe700 1 -- 172.21.15.89:0/3594587121 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076c00cde0 con 0x7f0764003fe0 2023-10-10T20:43:18.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.090+0000 7f07737fe700 1 -- 172.21.15.89:0/3594587121 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c005b70 con 0x7f0764003fe0 2023-10-10T20:43:18.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.090+0000 7f0782bb2700 1 -- 172.21.15.89:0/3594587121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0764003fe0 msgr2=0x7f0764004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:18.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.090+0000 7f0782bb2700 1 --2- 172.21.15.89:0/3594587121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0764003fe0 0x7f0764004810 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f076c00c390 tx=0x7f076c001e90 comp rx=0 tx=0).stop 2023-10-10T20:43:18.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.090+0000 7f078114f700 1 -- 172.21.15.89:0/3594587121 reap_dead start 2023-10-10T20:43:18.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.090+0000 7f0782bb2700 1 -- 172.21.15.89:0/3594587121 shutdown_connections 2023-10-10T20:43:18.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.090+0000 7f0782bb2700 1 -- 172.21.15.89:0/3594587121 >> 172.21.15.89:0/3594587121 conn(0x7f077c0cd480 msgr2=0x7f077c0cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:18.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.090+0000 7f0782bb2700 1 -- 172.21.15.89:0/3594587121 shutdown_connections 2023-10-10T20:43:18.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.090+0000 7f0782bb2700 1 -- 172.21.15.89:0/3594587121 wait complete. 2023-10-10T20:43:18.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.091+0000 7f0782bb2700 1 Processor -- start 2023-10-10T20:43:18.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.091+0000 7f0782bb2700 1 -- start start 2023-10-10T20:43:18.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.092+0000 7f0782bb2700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0764003fe0 0x7f077c15b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:18.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.092+0000 7f0782bb2700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f077c141310 0x7f077c15b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:18.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.092+0000 7f0782bb2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f077c0aa440 0x7f077c0aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:18.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.092+0000 7f0782bb2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c152050 con 0x7f077c0aa440 2023-10-10T20:43:18.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.092+0000 7f0782bb2700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c15ca20 con 0x7f077c141310 2023-10-10T20:43:18.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.092+0000 7f0782bb2700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c15bb60 con 0x7f0764003fe0 2023-10-10T20:43:18.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.092+0000 7f078114f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f077c0aa440 0x7f077c0aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:18.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.092+0000 7f078114f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f077c0aa440 0x7f077c0aa8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35260/0 (socket says 172.21.15.89:35260) 2023-10-10T20:43:18.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.092+0000 7f078114f700 1 -- 172.21.15.89:0/2550544763 learned_addr learned my addr 172.21.15.89:0/2550544763 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:18.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.092+0000 7f0773fff700 1 --2- 172.21.15.89:0/2550544763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f077c141310 0x7f077c15b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:18.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.092+0000 7f078094e700 1 --2- 172.21.15.89:0/2550544763 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0764003fe0 0x7f077c15b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:18.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.092+0000 7f078114f700 1 -- 172.21.15.89:0/2550544763 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0764003fe0 msgr2=0x7f077c15b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:18.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.092+0000 7f078114f700 1 --2- 172.21.15.89:0/2550544763 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0764003fe0 0x7f077c15b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:18.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.092+0000 7f078114f700 1 -- 172.21.15.89:0/2550544763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f077c141310 msgr2=0x7f077c15b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:18.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.092+0000 7f078114f700 1 --2- 172.21.15.89:0/2550544763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f077c141310 0x7f077c15b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:18.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.092+0000 7f078114f700 1 -- 172.21.15.89:0/2550544763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c15be10 con 0x7f077c0aa440 2023-10-10T20:43:18.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.093+0000 7f078114f700 1 --2- 172.21.15.89:0/2550544763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f077c0aa440 0x7f077c0aa8e0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f077c06b880 tx=0x7f07740098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:18.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.093+0000 7f0770ff9700 1 -- 172.21.15.89:0/2550544763 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774002c80 con 0x7f077c0aa440 2023-10-10T20:43:18.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.093+0000 7f0770ff9700 1 -- 172.21.15.89:0/2550544763 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077400f050 con 0x7f077c0aa440 2023-10-10T20:43:18.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.093+0000 7f0782bb2700 1 -- 172.21.15.89:0/2550544763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077c15c0f0 con 0x7f077c0aa440 2023-10-10T20:43:18.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.093+0000 7f0770ff9700 1 -- 172.21.15.89:0/2550544763 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774014410 con 0x7f077c0aa440 2023-10-10T20:43:18.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.093+0000 7f0782bb2700 1 -- 172.21.15.89:0/2550544763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077c15c6b0 con 0x7f077c0aa440 2023-10-10T20:43:18.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.094+0000 7f0782bb2700 1 -- 172.21.15.89:0/2550544763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077c04c3c0 con 0x7f077c0aa440 2023-10-10T20:43:18.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.094+0000 7f0770ff9700 1 -- 172.21.15.89:0/2550544763 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0774003040 con 0x7f077c0aa440 2023-10-10T20:43:18.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.094+0000 7f0770ff9700 1 --2- 172.21.15.89:0/2550544763 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0768067320 0x7f07680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:18.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.094+0000 7f0770ff9700 1 -- 172.21.15.89:0/2550544763 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f0774085300 con 0x7f077c0aa440 2023-10-10T20:43:18.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.095+0000 7f078094e700 1 --2- 172.21.15.89:0/2550544763 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0768067320 0x7f07680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:18.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.095+0000 7f078094e700 1 --2- 172.21.15.89:0/2550544763 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0768067320 0x7f07680697d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0778006dc0 tx=0x7f0778008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:18.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.099+0000 7f0770ff9700 1 -- 172.21.15.89:0/2550544763 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0774055730 con 0x7f077c0aa440 2023-10-10T20:43:18.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.325+0000 7f0782bb2700 1 -- 172.21.15.89:0/2550544763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "volume_0000000000654221"} v 0) v1 -- 0x7f077c0b3150 con 0x7f077c0aa440 2023-10-10T20:43:18.328 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-10-10T20:43:18.326+0000 7fd0184ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:43:18.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.330+0000 7f0770ff9700 1 -- 172.21.15.89:0/2550544763 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "volume_0000000000654221"}]=0 volume_0000000000654221 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v114) v1 ==== 195+0+0 (secure 0 0 0) 0x7f07740552c0 con 0x7f077c0aa440 2023-10-10T20:43:18.333 INFO:tasks.ceph.mds.b.smithi111.stderr: -915> 2023-10-10T20:42:58.617+0000 7fb5867a5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:43:18.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.332+0000 7f0782bb2700 1 -- 172.21.15.89:0/2550544763 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0768067320 msgr2=0x7f07680697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:18.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.332+0000 7f0782bb2700 1 --2- 172.21.15.89:0/2550544763 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0768067320 0x7f07680697d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0778006dc0 tx=0x7f0778008040 comp rx=0 tx=0).stop 2023-10-10T20:43:18.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.332+0000 7f0782bb2700 1 -- 172.21.15.89:0/2550544763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f077c0aa440 msgr2=0x7f077c0aa8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:18.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.332+0000 7f0782bb2700 1 --2- 172.21.15.89:0/2550544763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f077c0aa440 0x7f077c0aa8e0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f077c06b880 tx=0x7f07740098f0 comp rx=0 tx=0).stop 2023-10-10T20:43:18.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.332+0000 7f0782bb2700 1 -- 172.21.15.89:0/2550544763 shutdown_connections 2023-10-10T20:43:18.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.332+0000 7f0782bb2700 1 --2- 172.21.15.89:0/2550544763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f077c0aa440 0x7f077c0aa8e0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:18.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.332+0000 7f0782bb2700 1 --2- 172.21.15.89:0/2550544763 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0764003fe0 0x7f077c15b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:18.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.332+0000 7f0782bb2700 1 --2- 172.21.15.89:0/2550544763 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0768067320 0x7f07680697d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:18.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.332+0000 7f0782bb2700 1 --2- 172.21.15.89:0/2550544763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f077c141310 0x7f077c15b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:18.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.332+0000 7f0782bb2700 1 -- 172.21.15.89:0/2550544763 >> 172.21.15.89:0/2550544763 conn(0x7f077c0cd480 msgr2=0x7f077c15d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:18.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.332+0000 7f0782bb2700 1 -- 172.21.15.89:0/2550544763 shutdown_connections 2023-10-10T20:43:18.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.332+0000 7f0782bb2700 1 -- 172.21.15.89:0/2550544763 wait complete. 2023-10-10T20:43:18.337 INFO:teuthology.orchestra.run.smithi089.stderr:volume_0000000000654221 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:43:18.347 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm volume_0000000000654221 --yes-i-really-mean-it 2023-10-10T20:43:18.356 INFO:tasks.ceph.mds.b.smithi111.stdout:starting mds.b at 2023-10-10T20:43:18.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.461+0000 7f1be4d02700 1 Processor -- start 2023-10-10T20:43:18.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.461+0000 7f1be4d02700 1 -- start start 2023-10-10T20:43:18.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.462+0000 7f1be4d02700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1be006bcd0 0x7f1be00caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:18.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.462+0000 7f1be4d02700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be015d7e0 con 0x7f1be006c0a0 2023-10-10T20:43:18.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.462+0000 7f1be4d02700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be015dac0 con 0x7f1be015df50 2023-10-10T20:43:18.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.462+0000 7f1be4d02700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be015aa30 con 0x7f1be006bcd0 2023-10-10T20:43:18.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.462+0000 7f1bde59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1be006bcd0 0x7f1be00caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:18.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.462+0000 7f1bddd9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1be006c0a0 0x7f1be015a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49516/0 (socket says 172.21.15.89:49516) 2023-10-10T20:43:18.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.462+0000 7f1bddd9b700 1 -- 172.21.15.89:0/4271975946 learned_addr learned my addr 172.21.15.89:0/4271975946 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:18.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.462+0000 7f1bde59c700 1 -- 172.21.15.89:0/4271975946 >> v1:172.21.15.111:6789/0 conn(0x7f1be015df50 legacy=0x7f1be00c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:18.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.462+0000 7f1bde59c700 1 -- 172.21.15.89:0/4271975946 >> v1:172.21.15.89:6789/0 conn(0x7f1be006c0a0 legacy=0x7f1be015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:18.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.462+0000 7f1bde59c700 1 -- 172.21.15.89:0/4271975946 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be015ad10 con 0x7f1be006bcd0 2023-10-10T20:43:18.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.462+0000 7f1bdd59a700 1 -- 172.21.15.89:0/4271975946 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2687791234 0 0) 0x7f1be015d7e0 con 0x7f1be006c0a0 2023-10-10T20:43:18.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.462+0000 7f1bde59c700 1 --2- 172.21.15.89:0/4271975946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1be006bcd0 0x7f1be00caa90 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00098e0 tx=0x7f1bd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e311764792968238 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:18.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.462+0000 7f1bdd59a700 1 -- 172.21.15.89:0/4271975946 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0009e80 con 0x7f1be006bcd0 2023-10-10T20:43:18.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.463+0000 7f1bdd59a700 1 -- 172.21.15.89:0/4271975946 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd000b860 con 0x7f1be006bcd0 2023-10-10T20:43:18.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.463+0000 7f1bdd59a700 1 -- 172.21.15.89:0/4271975946 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd000bbe0 con 0x7f1be006bcd0 2023-10-10T20:43:18.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.463+0000 7f1be4d02700 1 -- 172.21.15.89:0/4271975946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1be006bcd0 msgr2=0x7f1be00caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:18.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.463+0000 7f1be4d02700 1 --2- 172.21.15.89:0/4271975946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1be006bcd0 0x7f1be00caa90 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00098e0 tx=0x7f1bd00091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:18.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.463+0000 7f1be4d02700 1 -- 172.21.15.89:0/4271975946 shutdown_connections 2023-10-10T20:43:18.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.463+0000 7f1be4d02700 1 --2- 172.21.15.89:0/4271975946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1be006bcd0 0x7f1be00caa90 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:18.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.463+0000 7f1be4d02700 1 -- 172.21.15.89:0/4271975946 >> 172.21.15.89:0/4271975946 conn(0x7f1be00ce600 msgr2=0x7f1be00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:18.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.463+0000 7f1be4d02700 1 -- 172.21.15.89:0/4271975946 shutdown_connections 2023-10-10T20:43:18.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.463+0000 7f1be4d02700 1 -- 172.21.15.89:0/4271975946 wait complete. 2023-10-10T20:43:18.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.464+0000 7f1be4d02700 1 Processor -- start 2023-10-10T20:43:18.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.464+0000 7f1be4d02700 1 -- start start 2023-10-10T20:43:18.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.465+0000 7f1be4d02700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1be006bcd0 0x7f1be00b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:18.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.465+0000 7f1be4d02700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1be006c0a0 0x7f1be0169d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:18.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.465+0000 7f1be4d02700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1be015df50 0x7f1be016a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:18.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.465+0000 7f1be4d02700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be00c5b00 con 0x7f1be015df50 2023-10-10T20:43:18.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.465+0000 7f1be4d02700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be0157e80 con 0x7f1be006c0a0 2023-10-10T20:43:18.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.465+0000 7f1be4d02700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be00b1d40 con 0x7f1be006bcd0 2023-10-10T20:43:18.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.465+0000 7f1bded9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1be015df50 0x7f1be016a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:18.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.465+0000 7f1bded9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1be015df50 0x7f1be016a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59366/0 (socket says 172.21.15.89:59366) 2023-10-10T20:43:18.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.465+0000 7f1bded9d700 1 -- 172.21.15.89:0/4288689584 learned_addr learned my addr 172.21.15.89:0/4288689584 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:18.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.465+0000 7f1bde59c700 1 --2- 172.21.15.89:0/4288689584 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1be006bcd0 0x7f1be00b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:18.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.465+0000 7f1bddd9b700 1 --2- 172.21.15.89:0/4288689584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1be006c0a0 0x7f1be0169d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:18.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.465+0000 7f1bded9d700 1 -- 172.21.15.89:0/4288689584 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1be006bcd0 msgr2=0x7f1be00b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:18.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.465+0000 7f1bded9d700 1 --2- 172.21.15.89:0/4288689584 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1be006bcd0 0x7f1be00b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:18.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.465+0000 7f1bded9d700 1 -- 172.21.15.89:0/4288689584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1be006c0a0 msgr2=0x7f1be0169d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:18.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.465+0000 7f1bded9d700 1 --2- 172.21.15.89:0/4288689584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1be006c0a0 0x7f1be0169d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:18.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.465+0000 7f1bded9d700 1 -- 172.21.15.89:0/4288689584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd0009420 con 0x7f1be015df50 2023-10-10T20:43:18.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.465+0000 7f1bddd9b700 1 --2- 172.21.15.89:0/4288689584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1be006c0a0 0x7f1be0169d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:18.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.465+0000 7f1bde59c700 1 --2- 172.21.15.89:0/4288689584 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1be006bcd0 0x7f1be00b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:18.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.466+0000 7f1bded9d700 1 --2- 172.21.15.89:0/4288689584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1be015df50 0x7f1be016a2c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f1bd800c840 tx=0x7f1bd8001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:18.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.466+0000 7f1bce7fc700 1 -- 172.21.15.89:0/4288689584 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8012630 con 0x7f1be015df50 2023-10-10T20:43:18.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.466+0000 7f1bce7fc700 1 -- 172.21.15.89:0/4288689584 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd80127b0 con 0x7f1be015df50 2023-10-10T20:43:18.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.466+0000 7f1be4d02700 1 -- 172.21.15.89:0/4288689584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be00b2020 con 0x7f1be015df50 2023-10-10T20:43:18.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.466+0000 7f1bce7fc700 1 -- 172.21.15.89:0/4288689584 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8012b70 con 0x7f1be015df50 2023-10-10T20:43:18.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.466+0000 7f1be4d02700 1 -- 172.21.15.89:0/4288689584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be00b25e0 con 0x7f1be015df50 2023-10-10T20:43:18.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.467+0000 7f1be4d02700 1 -- 172.21.15.89:0/4288689584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be004c3c0 con 0x7f1be015df50 2023-10-10T20:43:18.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.467+0000 7f1bce7fc700 1 -- 172.21.15.89:0/4288689584 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1bd8003030 con 0x7f1be015df50 2023-10-10T20:43:18.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.467+0000 7f1bce7fc700 1 --2- 172.21.15.89:0/4288689584 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1bc0067320 0x7f1bc00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:18.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.468+0000 7f1bce7fc700 1 -- 172.21.15.89:0/4288689584 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6266+0+0 (secure 0 0 0) 0x7f1bd8084e60 con 0x7f1be015df50 2023-10-10T20:43:18.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.468+0000 7f1bde59c700 1 --2- 172.21.15.89:0/4288689584 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1bc0067320 0x7f1bc00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:18.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.468+0000 7f1bde59c700 1 --2- 172.21.15.89:0/4288689584 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1bc0067320 0x7f1bc00697d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00093f0 tx=0x7f1bd0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:18.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.472+0000 7f1bce7fc700 1 -- 172.21.15.89:0/4288689584 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1bd80518c0 con 0x7f1be015df50 2023-10-10T20:43:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:18.698+0000 7f1be4d02700 1 -- 172.21.15.89:0/4288689584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "volume_0000000000654221", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1be015aff0 con 0x7f1be015df50 2023-10-10T20:43:19.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.330+0000 7f1bce7fc700 1 -- 172.21.15.89:0/4288689584 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "volume_0000000000654221", "yes_i_really_mean_it": true}]=0 v115) v1 ==== 121+0+0 (secure 0 0 0) 0x7f1bd8054ee0 con 0x7f1be015df50 2023-10-10T20:43:19.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.332+0000 7f1be4d02700 1 -- 172.21.15.89:0/4288689584 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1bc0067320 msgr2=0x7f1bc00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:19.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.332+0000 7f1be4d02700 1 --2- 172.21.15.89:0/4288689584 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1bc0067320 0x7f1bc00697d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00093f0 tx=0x7f1bd0017000 comp rx=0 tx=0).stop 2023-10-10T20:43:19.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.332+0000 7f1be4d02700 1 -- 172.21.15.89:0/4288689584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1be015df50 msgr2=0x7f1be016a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:19.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.332+0000 7f1be4d02700 1 --2- 172.21.15.89:0/4288689584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1be015df50 0x7f1be016a2c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f1bd800c840 tx=0x7f1bd8001670 comp rx=0 tx=0).stop 2023-10-10T20:43:19.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.333+0000 7f1be4d02700 1 -- 172.21.15.89:0/4288689584 shutdown_connections 2023-10-10T20:43:19.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.333+0000 7f1be4d02700 1 --2- 172.21.15.89:0/4288689584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1be015df50 0x7f1be016a2c0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:19.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.333+0000 7f1be4d02700 1 --2- 172.21.15.89:0/4288689584 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1be006bcd0 0x7f1be00b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:19.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.333+0000 7f1be4d02700 1 --2- 172.21.15.89:0/4288689584 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1bc0067320 0x7f1bc00697d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:19.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.333+0000 7f1be4d02700 1 --2- 172.21.15.89:0/4288689584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1be006c0a0 0x7f1be0169d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:19.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.333+0000 7f1be4d02700 1 -- 172.21.15.89:0/4288689584 >> 172.21.15.89:0/4288689584 conn(0x7f1be00ce600 msgr2=0x7f1be0069690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:19.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.333+0000 7f1be4d02700 1 -- 172.21.15.89:0/4288689584 shutdown_connections 2023-10-10T20:43:19.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.333+0000 7f1be4d02700 1 -- 172.21.15.89:0/4288689584 wait complete. 2023-10-10T20:43:19.348 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000654221.meta cephfs.volume_0000000000654221.meta --yes-i-really-really-mean-it 2023-10-10T20:43:19.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.467+0000 7f2db8640700 1 Processor -- start 2023-10-10T20:43:19.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.467+0000 7f2db8640700 1 -- start start 2023-10-10T20:43:19.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.468+0000 7f2db8640700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2db00b55a0 0x7f2db0164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:19.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.468+0000 7f2db8640700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db00cfa60 con 0x7f2db0159ec0 2023-10-10T20:43:19.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.468+0000 7f2db8640700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db00b5b00 con 0x7f2db00cd910 2023-10-10T20:43:19.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.468+0000 7f2db8640700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2db00b5da0 con 0x7f2db00b55a0 2023-10-10T20:43:19.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.468+0000 7f2db63dc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2db0159ec0 0x7f2db00b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49528/0 (socket says 172.21.15.89:49528) 2023-10-10T20:43:19.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.468+0000 7f2db63dc700 1 -- 172.21.15.89:0/8673310 learned_addr learned my addr 172.21.15.89:0/8673310 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:19.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.468+0000 7f2db53da700 1 -- 172.21.15.89:0/8673310 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 136664783 0 0) 0x7f2db00cfa60 con 0x7f2db0159ec0 2023-10-10T20:43:19.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.468+0000 7f2db53da700 1 -- 172.21.15.89:0/8673310 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d9c003570 con 0x7f2db0159ec0 2023-10-10T20:43:19.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.468+0000 7f2db6bdd700 1 --2- 172.21.15.89:0/8673310 >> v2:172.21.15.111:3300/0 conn(0x7f2db00b55a0 0x7f2db0164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:19.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db53da700 1 -- 172.21.15.89:0/8673310 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2228840984 0 0) 0x7f2d9c003570 con 0x7f2db0159ec0 2023-10-10T20:43:19.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db53da700 1 -- 172.21.15.89:0/8673310 >> v2:172.21.15.111:3300/0 conn(0x7f2db00b55a0 msgr2=0x7f2db0164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:19.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db53da700 1 --2- 172.21.15.89:0/8673310 >> v2:172.21.15.111:3300/0 conn(0x7f2db00b55a0 0x7f2db0164400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:19.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db53da700 1 -- 172.21.15.89:0/8673310 >> v1:172.21.15.111:6789/0 conn(0x7f2db00cd910 legacy=0x7f2db00ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:19.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db53da700 1 -- 172.21.15.89:0/8673310 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db00b6080 con 0x7f2db0159ec0 2023-10-10T20:43:19.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db53da700 1 -- 172.21.15.89:0/8673310 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f2dac001970 con 0x7f2db0159ec0 2023-10-10T20:43:19.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db53da700 1 -- 172.21.15.89:0/8673310 >> v1:172.21.15.89:6789/0 conn(0x7f2db0159ec0 legacy=0x7f2db00b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:19.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db53da700 1 --2- 172.21.15.89:0/8673310 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d9c003a60 0x7f2d9c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:19.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db53da700 1 --2- 172.21.15.89:0/8673310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d9c004810 0x7f2d9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:19.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db53da700 1 --2- 172.21.15.89:0/8673310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d9c005210 0x7f2d9c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:19.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db53da700 1 -- 172.21.15.89:0/8673310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db00b5da0 con 0x7f2d9c005210 2023-10-10T20:43:19.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db53da700 1 -- 172.21.15.89:0/8673310 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c00c130 con 0x7f2d9c004810 2023-10-10T20:43:19.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db53da700 1 -- 172.21.15.89:0/8673310 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c00c2c0 con 0x7f2d9c003a60 2023-10-10T20:43:19.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db6bdd700 1 --2- 172.21.15.89:0/8673310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d9c005210 0x7f2d9c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:19.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db5bdb700 1 --2- 172.21.15.89:0/8673310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d9c004810 0x7f2d9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:19.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db63dc700 1 --2- 172.21.15.89:0/8673310 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d9c003a60 0x7f2d9c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:19.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db6bdd700 1 -- 172.21.15.89:0/8673310 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d9c003a60 msgr2=0x7f2d9c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:19.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db6bdd700 1 --2- 172.21.15.89:0/8673310 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2d9c003a60 0x7f2d9c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:19.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db6bdd700 1 -- 172.21.15.89:0/8673310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d9c004810 msgr2=0x7f2d9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:19.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db6bdd700 1 --2- 172.21.15.89:0/8673310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d9c004810 0x7f2d9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:19.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db6bdd700 1 -- 172.21.15.89:0/8673310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9c003570 con 0x7f2d9c005210 2023-10-10T20:43:19.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db6bdd700 1 -- 172.21.15.89:0/8673310 reap_dead start 2023-10-10T20:43:19.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db6bdd700 1 --2- 172.21.15.89:0/8673310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d9c005210 0x7f2d9c00ba60 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009c30 tx=0x7f2da80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02c77e4ae375773 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:19.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.469+0000 7f2db53da700 1 -- 172.21.15.89:0/8673310 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da80095b0 con 0x7f2d9c005210 2023-10-10T20:43:19.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.470+0000 7f2db53da700 1 -- 172.21.15.89:0/8673310 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da800e710 con 0x7f2d9c005210 2023-10-10T20:43:19.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.470+0000 7f2db53da700 1 -- 172.21.15.89:0/8673310 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da800eaf0 con 0x7f2d9c005210 2023-10-10T20:43:19.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.470+0000 7f2db8640700 1 -- 172.21.15.89:0/8673310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d9c005210 msgr2=0x7f2d9c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:19.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.470+0000 7f2db8640700 1 --2- 172.21.15.89:0/8673310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d9c005210 0x7f2d9c00ba60 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009c30 tx=0x7f2da80091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:19.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.470+0000 7f2db8640700 1 -- 172.21.15.89:0/8673310 shutdown_connections 2023-10-10T20:43:19.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.470+0000 7f2db8640700 1 --2- 172.21.15.89:0/8673310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d9c005210 0x7f2d9c00ba60 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:19.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.470+0000 7f2db8640700 1 -- 172.21.15.89:0/8673310 >> 172.21.15.89:0/8673310 conn(0x7f2db00999f0 msgr2=0x7f2db00b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:19.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.470+0000 7f2db8640700 1 -- 172.21.15.89:0/8673310 shutdown_connections 2023-10-10T20:43:19.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.470+0000 7f2db8640700 1 -- 172.21.15.89:0/8673310 wait complete. 2023-10-10T20:43:19.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.471+0000 7f2db8640700 1 Processor -- start 2023-10-10T20:43:19.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.471+0000 7f2db8640700 1 -- start start 2023-10-10T20:43:19.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.472+0000 7f2db8640700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d9c005210 0x7f2db0150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:19.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.472+0000 7f2db8640700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2db016cf60 0x7f2db0069e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:19.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.472+0000 7f2db8640700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2db0169930 0x7f2db00b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:19.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.472+0000 7f2db8640700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db01549a0 con 0x7f2db016cf60 2023-10-10T20:43:19.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.472+0000 7f2db8640700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db0068390 con 0x7f2d9c005210 2023-10-10T20:43:19.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.472+0000 7f2db8640700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db016d3f0 con 0x7f2db0169930 2023-10-10T20:43:19.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.472+0000 7f2db5bdb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2db016cf60 0x7f2db0069e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:19.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.472+0000 7f2db5bdb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2db016cf60 0x7f2db0069e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59378/0 (socket says 172.21.15.89:59378) 2023-10-10T20:43:19.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.472+0000 7f2db5bdb700 1 -- 172.21.15.89:0/2951025251 learned_addr learned my addr 172.21.15.89:0/2951025251 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:19.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.472+0000 7f2db5bdb700 1 -- 172.21.15.89:0/2951025251 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2db0169930 msgr2=0x7f2db00b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:19.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.472+0000 7f2db6bdd700 1 --2- 172.21.15.89:0/2951025251 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2db0169930 0x7f2db00b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:19.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.472+0000 7f2db5bdb700 1 --2- 172.21.15.89:0/2951025251 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2db0169930 0x7f2db00b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:19.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.472+0000 7f2db5bdb700 1 -- 172.21.15.89:0/2951025251 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d9c005210 msgr2=0x7f2db0150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:19.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.472+0000 7f2db5bdb700 1 --2- 172.21.15.89:0/2951025251 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d9c005210 0x7f2db0150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:19.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.472+0000 7f2db5bdb700 1 -- 172.21.15.89:0/2951025251 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db016d6d0 con 0x7f2db016cf60 2023-10-10T20:43:19.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.473+0000 7f2db5bdb700 1 --2- 172.21.15.89:0/2951025251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2db016cf60 0x7f2db0069e70 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f2da0002980 tx=0x7f2da0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:19.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.473+0000 7f2da67fc700 1 -- 172.21.15.89:0/2951025251 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da0006820 con 0x7f2db016cf60 2023-10-10T20:43:19.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.473+0000 7f2da67fc700 1 -- 172.21.15.89:0/2951025251 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da0006980 con 0x7f2db016cf60 2023-10-10T20:43:19.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.473+0000 7f2db8640700 1 -- 172.21.15.89:0/2951025251 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db016d9b0 con 0x7f2db016cf60 2023-10-10T20:43:19.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.473+0000 7f2da67fc700 1 -- 172.21.15.89:0/2951025251 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da0006d40 con 0x7f2db016cf60 2023-10-10T20:43:19.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.473+0000 7f2db8640700 1 -- 172.21.15.89:0/2951025251 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db006a3b0 con 0x7f2db016cf60 2023-10-10T20:43:19.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.474+0000 7f2db8640700 1 -- 172.21.15.89:0/2951025251 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db0063c90 con 0x7f2db016cf60 2023-10-10T20:43:19.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.474+0000 7f2da67fc700 1 -- 172.21.15.89:0/2951025251 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2da0007240 con 0x7f2db016cf60 2023-10-10T20:43:19.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.474+0000 7f2da67fc700 1 --2- 172.21.15.89:0/2951025251 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2d94067320 0x7f2d940697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:19.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.474+0000 7f2da67fc700 1 -- 172.21.15.89:0/2951025251 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6266+0+0 (secure 0 0 0) 0x7f2da0085010 con 0x7f2db016cf60 2023-10-10T20:43:19.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.474+0000 7f2db63dc700 1 --2- 172.21.15.89:0/2951025251 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2d94067320 0x7f2d940697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:19.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.475+0000 7f2db63dc700 1 --2- 172.21.15.89:0/2951025251 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2d94067320 0x7f2d940697d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2dac007d30 tx=0x7f2dac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:19.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.479+0000 7f2da67fc700 1 -- 172.21.15.89:0/2951025251 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2da0051a70 con 0x7f2db016cf60 2023-10-10T20:43:19.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:19.661+0000 7f2db8640700 1 -- 172.21.15.89:0/2951025251 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000654221.meta", "pool2": "cephfs.volume_0000000000654221.meta", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2db00718b0 con 0x7f2db016cf60 2023-10-10T20:43:20.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.339+0000 7f2da67fc700 1 -- 172.21.15.89:0/2951025251 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000654221.meta", "pool2": "cephfs.volume_0000000000654221.meta", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000654221.meta' removed v172) v1 ==== 241+0+0 (secure 0 0 0) 0x7f2da0055090 con 0x7f2db016cf60 2023-10-10T20:43:20.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.342+0000 7f2db8640700 1 -- 172.21.15.89:0/2951025251 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2d94067320 msgr2=0x7f2d940697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:20.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.342+0000 7f2db8640700 1 --2- 172.21.15.89:0/2951025251 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2d94067320 0x7f2d940697d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2dac007d30 tx=0x7f2dac009040 comp rx=0 tx=0).stop 2023-10-10T20:43:20.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.342+0000 7f2db8640700 1 -- 172.21.15.89:0/2951025251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2db016cf60 msgr2=0x7f2db0069e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:20.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.342+0000 7f2db8640700 1 --2- 172.21.15.89:0/2951025251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2db016cf60 0x7f2db0069e70 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f2da0002980 tx=0x7f2da0007d30 comp rx=0 tx=0).stop 2023-10-10T20:43:20.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.342+0000 7f2db8640700 1 -- 172.21.15.89:0/2951025251 shutdown_connections 2023-10-10T20:43:20.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.342+0000 7f2db8640700 1 --2- 172.21.15.89:0/2951025251 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2db0169930 0x7f2db00b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:20.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.342+0000 7f2db8640700 1 --2- 172.21.15.89:0/2951025251 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2d94067320 0x7f2d940697d0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:20.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.342+0000 7f2db8640700 1 --2- 172.21.15.89:0/2951025251 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d9c005210 0x7f2db0150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:20.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.342+0000 7f2db8640700 1 --2- 172.21.15.89:0/2951025251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2db016cf60 0x7f2db0069e70 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:20.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.342+0000 7f2db8640700 1 -- 172.21.15.89:0/2951025251 >> 172.21.15.89:0/2951025251 conn(0x7f2db00999f0 msgr2=0x7f2db0152050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:20.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.342+0000 7f2db8640700 1 -- 172.21.15.89:0/2951025251 shutdown_connections 2023-10-10T20:43:20.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.342+0000 7f2db8640700 1 -- 172.21.15.89:0/2951025251 wait complete. 2023-10-10T20:43:20.349 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs.volume_0000000000654221.meta' removed 2023-10-10T20:43:20.361 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:43:20.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.489+0000 7f19087e4700 1 Processor -- start 2023-10-10T20:43:20.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.489+0000 7f19087e4700 1 -- start start 2023-10-10T20:43:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.490+0000 7f19087e4700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f19000cd910 0x7f19000ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.490+0000 7f19087e4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19000b55a0 0x7f19000ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.490+0000 7f19087e4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19000b5b00 con 0x7f1900159ec0 2023-10-10T20:43:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.490+0000 7f19087e4700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19000b5da0 con 0x7f19000cd910 2023-10-10T20:43:20.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.490+0000 7f19087e4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19000b6080 con 0x7f19000b55a0 2023-10-10T20:43:20.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.490+0000 7f1906580700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1900159ec0 0x7f19000b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49544/0 (socket says 172.21.15.89:49544) 2023-10-10T20:43:20.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.490+0000 7f1906580700 1 -- 172.21.15.89:0/1170541738 learned_addr learned my addr 172.21.15.89:0/1170541738 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:20.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f190557e700 1 -- 172.21.15.89:0/1170541738 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006350376 0 0) 0x7f19000b5b00 con 0x7f1900159ec0 2023-10-10T20:43:20.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f190557e700 1 -- 172.21.15.89:0/1170541738 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18ec003570 con 0x7f1900159ec0 2023-10-10T20:43:20.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f1905d7f700 1 --2- 172.21.15.89:0/1170541738 >> v2:172.21.15.111:3300/0 conn(0x7f19000cd910 0x7f19000ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:20.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f1906d81700 1 --2- 172.21.15.89:0/1170541738 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19000b55a0 0x7f19000ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:20.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f190557e700 1 -- 172.21.15.89:0/1170541738 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1916168278 0 0) 0x7f18ec003570 con 0x7f1900159ec0 2023-10-10T20:43:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f190557e700 1 -- 172.21.15.89:0/1170541738 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19000b55a0 msgr2=0x7f19000ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f190557e700 1 --2- 172.21.15.89:0/1170541738 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19000b55a0 0x7f19000ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f190557e700 1 -- 172.21.15.89:0/1170541738 >> v2:172.21.15.111:3300/0 conn(0x7f19000cd910 msgr2=0x7f19000ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f190557e700 1 --2- 172.21.15.89:0/1170541738 >> v2:172.21.15.111:3300/0 conn(0x7f19000cd910 0x7f19000ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f190557e700 1 -- 172.21.15.89:0/1170541738 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19000cc100 con 0x7f1900159ec0 2023-10-10T20:43:20.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f190557e700 1 -- 172.21.15.89:0/1170541738 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f18fc0015f0 con 0x7f1900159ec0 2023-10-10T20:43:20.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f190557e700 1 -- 172.21.15.89:0/1170541738 >> v1:172.21.15.89:6789/0 conn(0x7f1900159ec0 legacy=0x7f19000b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:20.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f190557e700 1 --2- 172.21.15.89:0/1170541738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18ec003a60 0x7f18ec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:20.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f190557e700 1 --2- 172.21.15.89:0/1170541738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18ec004810 0x7f18ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:20.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f190557e700 1 --2- 172.21.15.89:0/1170541738 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19000b55a0 0x7f18ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:20.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f190557e700 1 -- 172.21.15.89:0/1170541738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19000b5da0 con 0x7f18ec003a60 2023-10-10T20:43:20.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f190557e700 1 -- 172.21.15.89:0/1170541738 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19000b6080 con 0x7f18ec004810 2023-10-10T20:43:20.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f190557e700 1 -- 172.21.15.89:0/1170541738 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec00ba90 con 0x7f19000b55a0 2023-10-10T20:43:20.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f1906580700 1 --2- 172.21.15.89:0/1170541738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18ec003a60 0x7f18ec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:20.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f1905d7f700 1 --2- 172.21.15.89:0/1170541738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18ec004810 0x7f18ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:20.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f1906d81700 1 --2- 172.21.15.89:0/1170541738 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19000b55a0 0x7f18ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:20.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f1906580700 1 -- 172.21.15.89:0/1170541738 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19000b55a0 msgr2=0x7f18ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:20.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f1906580700 1 --2- 172.21.15.89:0/1170541738 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f19000b55a0 0x7f18ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:20.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f1906580700 1 -- 172.21.15.89:0/1170541738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18ec004810 msgr2=0x7f18ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:20.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f1906580700 1 --2- 172.21.15.89:0/1170541738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18ec004810 0x7f18ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:20.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.491+0000 7f1906580700 1 -- 172.21.15.89:0/1170541738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec003570 con 0x7f18ec003a60 2023-10-10T20:43:20.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.492+0000 7f1906580700 1 --2- 172.21.15.89:0/1170541738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18ec003a60 0x7f18ec003e70 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00c3c0 tx=0x7f18fc007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21342745492c4e4d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:20.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.492+0000 7f190557e700 1 -- 172.21.15.89:0/1170541738 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc00cc30 con 0x7f18ec003a60 2023-10-10T20:43:20.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.492+0000 7f190557e700 1 -- 172.21.15.89:0/1170541738 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc00cdb0 con 0x7f18ec003a60 2023-10-10T20:43:20.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.492+0000 7f190557e700 1 -- 172.21.15.89:0/1170541738 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc007320 con 0x7f18ec003a60 2023-10-10T20:43:20.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.492+0000 7f19087e4700 1 -- 172.21.15.89:0/1170541738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18ec003a60 msgr2=0x7f18ec003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:20.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.492+0000 7f19087e4700 1 --2- 172.21.15.89:0/1170541738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18ec003a60 0x7f18ec003e70 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00c3c0 tx=0x7f18fc007920 comp rx=0 tx=0).stop 2023-10-10T20:43:20.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.492+0000 7f1906d81700 1 -- 172.21.15.89:0/1170541738 reap_dead start 2023-10-10T20:43:20.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.492+0000 7f19087e4700 1 -- 172.21.15.89:0/1170541738 shutdown_connections 2023-10-10T20:43:20.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.492+0000 7f19087e4700 1 -- 172.21.15.89:0/1170541738 >> 172.21.15.89:0/1170541738 conn(0x7f19000999f0 msgr2=0x7f19000b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:20.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.492+0000 7f19087e4700 1 -- 172.21.15.89:0/1170541738 shutdown_connections 2023-10-10T20:43:20.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.492+0000 7f19087e4700 1 -- 172.21.15.89:0/1170541738 wait complete. 2023-10-10T20:43:20.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.493+0000 7f19087e4700 1 Processor -- start 2023-10-10T20:43:20.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.493+0000 7f19087e4700 1 -- start start 2023-10-10T20:43:20.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.494+0000 7f19087e4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f18ec003a60 0x7f190013ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:20.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.494+0000 7f19087e4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1900151f20 0x7f1900152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:20.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.494+0000 7f19087e4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1900153ed0 0x7f1900154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:20.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.494+0000 7f19087e4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19001528d0 con 0x7f1900151f20 2023-10-10T20:43:20.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.494+0000 7f19087e4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19000698c0 con 0x7f1900153ed0 2023-10-10T20:43:20.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.494+0000 7f19087e4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19001552a0 con 0x7f18ec003a60 2023-10-10T20:43:20.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.494+0000 7f1905d7f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1900151f20 0x7f1900152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:20.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.494+0000 7f1905d7f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1900151f20 0x7f1900152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59394/0 (socket says 172.21.15.89:59394) 2023-10-10T20:43:20.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.494+0000 7f1905d7f700 1 -- 172.21.15.89:0/1134238477 learned_addr learned my addr 172.21.15.89:0/1134238477 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:20.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.494+0000 7f1906d81700 1 --2- 172.21.15.89:0/1134238477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1900153ed0 0x7f1900154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:20.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.494+0000 7f1906580700 1 --2- 172.21.15.89:0/1134238477 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f18ec003a60 0x7f190013ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:20.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.494+0000 7f1905d7f700 1 -- 172.21.15.89:0/1134238477 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f18ec003a60 msgr2=0x7f190013ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:20.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.494+0000 7f1905d7f700 1 --2- 172.21.15.89:0/1134238477 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f18ec003a60 0x7f190013ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:20.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.494+0000 7f1905d7f700 1 -- 172.21.15.89:0/1134238477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1900153ed0 msgr2=0x7f1900154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:20.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.494+0000 7f1905d7f700 1 --2- 172.21.15.89:0/1134238477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1900153ed0 0x7f1900154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:20.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.494+0000 7f1905d7f700 1 -- 172.21.15.89:0/1134238477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1900069590 con 0x7f1900151f20 2023-10-10T20:43:20.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.495+0000 7f1905d7f700 1 --2- 172.21.15.89:0/1134238477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1900151f20 0x7f1900152390 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f18f0011b40 tx=0x7f18f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:20.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.495+0000 7f18f67fc700 1 -- 172.21.15.89:0/1134238477 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f000f9f0 con 0x7f1900151f20 2023-10-10T20:43:20.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.495+0000 7f18f67fc700 1 -- 172.21.15.89:0/1134238477 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f000fb50 con 0x7f1900151f20 2023-10-10T20:43:20.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.495+0000 7f18f67fc700 1 -- 172.21.15.89:0/1134238477 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f000fed0 con 0x7f1900151f20 2023-10-10T20:43:20.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.495+0000 7f19087e4700 1 -- 172.21.15.89:0/1134238477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1900154ce0 con 0x7f1900151f20 2023-10-10T20:43:20.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.495+0000 7f19087e4700 1 -- 172.21.15.89:0/1134238477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190013f370 con 0x7f1900151f20 2023-10-10T20:43:20.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.496+0000 7f19087e4700 1 -- 172.21.15.89:0/1134238477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1900063c90 con 0x7f1900151f20 2023-10-10T20:43:20.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.496+0000 7f18f67fc700 1 -- 172.21.15.89:0/1134238477 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f18f0014650 con 0x7f1900151f20 2023-10-10T20:43:20.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.496+0000 7f18f67fc700 1 --2- 172.21.15.89:0/1134238477 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f18e4067320 0x7f18e40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:20.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.497+0000 7f18f67fc700 1 -- 172.21.15.89:0/1134238477 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f18f0018030 con 0x7f1900151f20 2023-10-10T20:43:20.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.497+0000 7f1906580700 1 --2- 172.21.15.89:0/1134238477 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f18e4067320 0x7f18e40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:20.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.497+0000 7f1906580700 1 --2- 172.21.15.89:0/1134238477 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f18e4067320 0x7f18e40697d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f18fc000ef0 tx=0x7f18fc015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:20.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.501+0000 7f18f67fc700 1 -- 172.21.15.89:0/1134238477 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18f0052320 con 0x7f1900151f20 2023-10-10T20:43:20.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:20.683+0000 7f19087e4700 1 -- 172.21.15.89:0/1134238477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f19000718b0 con 0x7f1900151f20 2023-10-10T20:43:21.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.344+0000 7f18f67fc700 1 -- 172.21.15.89:0/1134238477 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v173) v1 ==== 169+0+0 (secure 0 0 0) 0x7f18f0055940 con 0x7f1900151f20 2023-10-10T20:43:21.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.347+0000 7f19087e4700 1 -- 172.21.15.89:0/1134238477 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f18e4067320 msgr2=0x7f18e40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:21.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.347+0000 7f19087e4700 1 --2- 172.21.15.89:0/1134238477 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f18e4067320 0x7f18e40697d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f18fc000ef0 tx=0x7f18fc015000 comp rx=0 tx=0).stop 2023-10-10T20:43:21.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.347+0000 7f19087e4700 1 -- 172.21.15.89:0/1134238477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1900151f20 msgr2=0x7f1900152390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:21.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.347+0000 7f19087e4700 1 --2- 172.21.15.89:0/1134238477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1900151f20 0x7f1900152390 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f18f0011b40 tx=0x7f18f00091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:21.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.347+0000 7f19087e4700 1 -- 172.21.15.89:0/1134238477 shutdown_connections 2023-10-10T20:43:21.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.347+0000 7f19087e4700 1 --2- 172.21.15.89:0/1134238477 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f18e4067320 0x7f18e40697d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:21.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.347+0000 7f19087e4700 1 --2- 172.21.15.89:0/1134238477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1900153ed0 0x7f1900154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:21.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.347+0000 7f19087e4700 1 --2- 172.21.15.89:0/1134238477 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f18ec003a60 0x7f190013ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:21.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.347+0000 7f19087e4700 1 --2- 172.21.15.89:0/1134238477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1900151f20 0x7f1900152390 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:21.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.347+0000 7f19087e4700 1 -- 172.21.15.89:0/1134238477 >> 172.21.15.89:0/1134238477 conn(0x7f19000999f0 msgr2=0x7f19000c62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:21.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.348+0000 7f19087e4700 1 -- 172.21.15.89:0/1134238477 shutdown_connections 2023-10-10T20:43:21.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.348+0000 7f19087e4700 1 -- 172.21.15.89:0/1134238477 wait complete. 2023-10-10T20:43:21.352 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' removed 2023-10-10T20:43:21.364 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm another-data-pool another-data-pool --yes-i-really-really-mean-it 2023-10-10T20:43:21.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.481+0000 7fb19268b700 1 Processor -- start 2023-10-10T20:43:21.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.481+0000 7fb19268b700 1 -- start start 2023-10-10T20:43:21.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.481+0000 7fb19268b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb18c0b55a0 0x7fb18c164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:21.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb19268b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18c0cfa60 con 0x7fb18c159ec0 2023-10-10T20:43:21.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb19268b700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18c0b5b00 con 0x7fb18c0cd910 2023-10-10T20:43:21.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb19268b700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18c0b5da0 con 0x7fb18c0b55a0 2023-10-10T20:43:21.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb18bfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb18c159ec0 0x7fb18c0b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49552/0 (socket says 172.21.15.89:49552) 2023-10-10T20:43:21.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb18bfff700 1 -- 172.21.15.89:0/3209363209 learned_addr learned my addr 172.21.15.89:0/3209363209 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:21.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb190c28700 1 --2- 172.21.15.89:0/3209363209 >> v2:172.21.15.111:3300/0 conn(0x7fb18c0b55a0 0x7fb18c164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:21.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb18affd700 1 -- 172.21.15.89:0/3209363209 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1357420856 0 0) 0x7fb18c0cfa60 con 0x7fb18c159ec0 2023-10-10T20:43:21.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb18affd700 1 -- 172.21.15.89:0/3209363209 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb174003570 con 0x7fb18c159ec0 2023-10-10T20:43:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb18affd700 1 -- 172.21.15.89:0/3209363209 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3579023027 0 0) 0x7fb174003570 con 0x7fb18c159ec0 2023-10-10T20:43:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb18affd700 1 -- 172.21.15.89:0/3209363209 >> v2:172.21.15.111:3300/0 conn(0x7fb18c0b55a0 msgr2=0x7fb18c164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb18affd700 1 --2- 172.21.15.89:0/3209363209 >> v2:172.21.15.111:3300/0 conn(0x7fb18c0b55a0 0x7fb18c164400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb18affd700 1 -- 172.21.15.89:0/3209363209 >> v1:172.21.15.111:6789/0 conn(0x7fb18c0cd910 legacy=0x7fb18c0ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb18affd700 1 -- 172.21.15.89:0/3209363209 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c0b6080 con 0x7fb18c159ec0 2023-10-10T20:43:21.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb18affd700 1 -- 172.21.15.89:0/3209363209 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fb1800015f0 con 0x7fb18c159ec0 2023-10-10T20:43:21.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb18affd700 1 -- 172.21.15.89:0/3209363209 >> v1:172.21.15.89:6789/0 conn(0x7fb18c159ec0 legacy=0x7fb18c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:21.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb18affd700 1 --2- 172.21.15.89:0/3209363209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb174003a60 0x7fb174003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb18affd700 1 --2- 172.21.15.89:0/3209363209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb174004810 0x7fb174004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb18affd700 1 --2- 172.21.15.89:0/3209363209 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb174005210 0x7fb17400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:21.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb18affd700 1 -- 172.21.15.89:0/3209363209 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0b5da0 con 0x7fb174004810 2023-10-10T20:43:21.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb18affd700 1 -- 172.21.15.89:0/3209363209 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17400c130 con 0x7fb174003a60 2023-10-10T20:43:21.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb18affd700 1 -- 172.21.15.89:0/3209363209 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17400c2c0 con 0x7fb174005210 2023-10-10T20:43:21.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb18b7fe700 1 --2- 172.21.15.89:0/3209363209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb174004810 0x7fb174004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:21.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.482+0000 7fb18bfff700 1 --2- 172.21.15.89:0/3209363209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb174003a60 0x7fb174003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:21.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.483+0000 7fb190c28700 1 --2- 172.21.15.89:0/3209363209 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb174005210 0x7fb17400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:21.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.483+0000 7fb18b7fe700 1 -- 172.21.15.89:0/3209363209 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb174005210 msgr2=0x7fb17400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:21.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.483+0000 7fb18b7fe700 1 --2- 172.21.15.89:0/3209363209 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb174005210 0x7fb17400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:21.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.483+0000 7fb18b7fe700 1 -- 172.21.15.89:0/3209363209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb174003a60 msgr2=0x7fb174003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:21.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.483+0000 7fb18b7fe700 1 --2- 172.21.15.89:0/3209363209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb174003a60 0x7fb174003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.483+0000 7fb190c28700 1 -- 172.21.15.89:0/3209363209 reap_dead start 2023-10-10T20:43:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.483+0000 7fb18b7fe700 1 -- 172.21.15.89:0/3209363209 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb174003570 con 0x7fb174004810 2023-10-10T20:43:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.483+0000 7fb18b7fe700 1 --2- 172.21.15.89:0/3209363209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb174004810 0x7fb174004c80 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0018b0 tx=0x7fb17c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179730752f807c51 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.483+0000 7fb18affd700 1 -- 172.21.15.89:0/3209363209 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c012420 con 0x7fb174004810 2023-10-10T20:43:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.483+0000 7fb18affd700 1 -- 172.21.15.89:0/3209363209 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17c0125a0 con 0x7fb174004810 2023-10-10T20:43:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.483+0000 7fb18affd700 1 -- 172.21.15.89:0/3209363209 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c012960 con 0x7fb174004810 2023-10-10T20:43:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.484+0000 7fb19268b700 1 -- 172.21.15.89:0/3209363209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb174004810 msgr2=0x7fb174004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.484+0000 7fb19268b700 1 --2- 172.21.15.89:0/3209363209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb174004810 0x7fb174004c80 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0018b0 tx=0x7fb17c007d30 comp rx=0 tx=0).stop 2023-10-10T20:43:21.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.484+0000 7fb19268b700 1 -- 172.21.15.89:0/3209363209 shutdown_connections 2023-10-10T20:43:21.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.484+0000 7fb19268b700 1 --2- 172.21.15.89:0/3209363209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb174004810 0x7fb174004c80 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:21.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.484+0000 7fb19268b700 1 -- 172.21.15.89:0/3209363209 >> 172.21.15.89:0/3209363209 conn(0x7fb18c0999f0 msgr2=0x7fb18c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:21.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.484+0000 7fb19268b700 1 -- 172.21.15.89:0/3209363209 shutdown_connections 2023-10-10T20:43:21.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.484+0000 7fb19268b700 1 -- 172.21.15.89:0/3209363209 wait complete. 2023-10-10T20:43:21.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.485+0000 7fb19268b700 1 Processor -- start 2023-10-10T20:43:21.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.485+0000 7fb19268b700 1 -- start start 2023-10-10T20:43:21.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb19268b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb174004810 0x7fb18c156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:21.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb19268b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18c06aec0 0x7fb18c09c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:21.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb19268b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb18c09cd00 0x7fb18c09a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:21.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb19268b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c09d180 con 0x7fb174004810 2023-10-10T20:43:21.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb19268b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c15a530 con 0x7fb18c06aec0 2023-10-10T20:43:21.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb19268b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c068eb0 con 0x7fb18c09cd00 2023-10-10T20:43:21.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb18bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb174004810 0x7fb18c156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:21.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb18bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb174004810 0x7fb18c156e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59418/0 (socket says 172.21.15.89:59418) 2023-10-10T20:43:21.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb18bfff700 1 -- 172.21.15.89:0/2814309098 learned_addr learned my addr 172.21.15.89:0/2814309098 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:21.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb190c28700 1 --2- 172.21.15.89:0/2814309098 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb18c09cd00 0x7fb18c09a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:21.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb18b7fe700 1 --2- 172.21.15.89:0/2814309098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18c06aec0 0x7fb18c09c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:21.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb18bfff700 1 -- 172.21.15.89:0/2814309098 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb18c09cd00 msgr2=0x7fb18c09a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:21.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb18bfff700 1 --2- 172.21.15.89:0/2814309098 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb18c09cd00 0x7fb18c09a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:21.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb18bfff700 1 -- 172.21.15.89:0/2814309098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18c06aec0 msgr2=0x7fb18c09c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:21.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb18bfff700 1 --2- 172.21.15.89:0/2814309098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18c06aec0 0x7fb18c09c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:21.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb18bfff700 1 -- 172.21.15.89:0/2814309098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c069190 con 0x7fb174004810 2023-10-10T20:43:21.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb18b7fe700 1 --2- 172.21.15.89:0/2814309098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18c06aec0 0x7fb18c09c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:21.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb18bfff700 1 --2- 172.21.15.89:0/2814309098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb174004810 0x7fb18c156e90 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fb18000fa20 tx=0x7fb180009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:21.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb173fff700 1 -- 172.21.15.89:0/2814309098 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18000fe40 con 0x7fb174004810 2023-10-10T20:43:21.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb173fff700 1 -- 172.21.15.89:0/2814309098 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1800072e0 con 0x7fb174004810 2023-10-10T20:43:21.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb19268b700 1 -- 172.21.15.89:0/2814309098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18c069470 con 0x7fb174004810 2023-10-10T20:43:21.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb173fff700 1 -- 172.21.15.89:0/2814309098 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1800076a0 con 0x7fb174004810 2023-10-10T20:43:21.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.486+0000 7fb19268b700 1 -- 172.21.15.89:0/2814309098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18c069a30 con 0x7fb174004810 2023-10-10T20:43:21.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.487+0000 7fb19268b700 1 -- 172.21.15.89:0/2814309098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb18c063c90 con 0x7fb174004810 2023-10-10T20:43:21.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.488+0000 7fb173fff700 1 -- 172.21.15.89:0/2814309098 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb180005a00 con 0x7fb174004810 2023-10-10T20:43:21.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.488+0000 7fb173fff700 1 --2- 172.21.15.89:0/2814309098 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb178067320 0x7fb1780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:21.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.488+0000 7fb173fff700 1 -- 172.21.15.89:0/2814309098 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5360+0+0 (secure 0 0 0) 0x7fb180084d40 con 0x7fb174004810 2023-10-10T20:43:21.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.488+0000 7fb18b7fe700 1 --2- 172.21.15.89:0/2814309098 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb178067320 0x7fb1780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:21.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.488+0000 7fb18b7fe700 1 --2- 172.21.15.89:0/2814309098 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb178067320 0x7fb1780697d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb17c001010 tx=0x7fb17c015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:21.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.492+0000 7fb173fff700 1 -- 172.21.15.89:0/2814309098 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1800555c0 con 0x7fb174004810 2023-10-10T20:43:21.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:21.681+0000 7fb19268b700 1 -- 172.21.15.89:0/2814309098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "another-data-pool", "pool2": "another-data-pool", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb18c0718b0 con 0x7fb174004810 2023-10-10T20:43:22.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.352+0000 7fb173fff700 1 -- 172.21.15.89:0/2814309098 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "another-data-pool", "pool2": "another-data-pool", "yes_i_really_really_mean_it": true}]=0 pool 'another-data-pool' removed v174) v1 ==== 187+0+0 (secure 0 0 0) 0x7fb180005cb0 con 0x7fb174004810 2023-10-10T20:43:22.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.356+0000 7fb19268b700 1 -- 172.21.15.89:0/2814309098 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb178067320 msgr2=0x7fb1780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:22.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.356+0000 7fb19268b700 1 --2- 172.21.15.89:0/2814309098 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb178067320 0x7fb1780697d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb17c001010 tx=0x7fb17c015000 comp rx=0 tx=0).stop 2023-10-10T20:43:22.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.356+0000 7fb19268b700 1 -- 172.21.15.89:0/2814309098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb174004810 msgr2=0x7fb18c156e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:22.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.356+0000 7fb19268b700 1 --2- 172.21.15.89:0/2814309098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb174004810 0x7fb18c156e90 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fb18000fa20 tx=0x7fb180009040 comp rx=0 tx=0).stop 2023-10-10T20:43:22.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.356+0000 7fb19268b700 1 -- 172.21.15.89:0/2814309098 shutdown_connections 2023-10-10T20:43:22.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.356+0000 7fb19268b700 1 --2- 172.21.15.89:0/2814309098 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb18c09cd00 0x7fb18c09a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:22.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.356+0000 7fb19268b700 1 --2- 172.21.15.89:0/2814309098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb174004810 0x7fb18c156e90 secure :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fb18000fa20 tx=0x7fb180009040 comp rx=0 tx=0).stop 2023-10-10T20:43:22.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.356+0000 7fb19268b700 1 --2- 172.21.15.89:0/2814309098 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb178067320 0x7fb1780697d0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:22.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.356+0000 7fb19268b700 1 --2- 172.21.15.89:0/2814309098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18c06aec0 0x7fb18c09c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:22.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.356+0000 7fb19268b700 1 -- 172.21.15.89:0/2814309098 >> 172.21.15.89:0/2814309098 conn(0x7fb18c0999f0 msgr2=0x7fb18c0c55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:22.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.357+0000 7fb19268b700 1 -- 172.21.15.89:0/2814309098 shutdown_connections 2023-10-10T20:43:22.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.357+0000 7fb19268b700 1 -- 172.21.15.89:0/2814309098 wait complete. 2023-10-10T20:43:22.363 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'another-data-pool' removed 2023-10-10T20:43:22.372 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:43:22.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.489+0000 7ff5b196d700 1 Processor -- start 2023-10-10T20:43:22.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.490+0000 7ff5b196d700 1 -- start start 2023-10-10T20:43:22.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.490+0000 7ff5b196d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff5ac159890 0x7ff5ac15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:22.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.490+0000 7ff5b196d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff5ac159c60 0x7ff5ac15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:22.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.490+0000 7ff5b196d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ac15a160 con 0x7ff5ac15db10 2023-10-10T20:43:22.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.490+0000 7ff5b196d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5ac0d13a0 con 0x7ff5ac159890 2023-10-10T20:43:22.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.490+0000 7ff5b196d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac15e140 con 0x7ff5ac159c60 2023-10-10T20:43:22.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.490+0000 7ff5aaffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff5ac159890 0x7ff5ac15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:22.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.490+0000 7ff5ab7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff5ac15db10 0x7ff5ac069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49560/0 (socket says 172.21.15.89:49560) 2023-10-10T20:43:22.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.490+0000 7ff5aaffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff5ac159890 0x7ff5ac15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59424/0 (socket says 172.21.15.89:59424) 2023-10-10T20:43:22.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.490+0000 7ff5ab7fe700 1 -- 172.21.15.89:0/283386982 learned_addr learned my addr 172.21.15.89:0/283386982 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:22.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.491+0000 7ff5a27fc700 1 --2- 172.21.15.89:0/283386982 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff5ac159c60 0x7ff5ac15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:22.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.491+0000 7ff5aaffd700 1 -- 172.21.15.89:0/283386982 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff5ac159c60 msgr2=0x7ff5ac15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:22.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.491+0000 7ff5aaffd700 1 --2- 172.21.15.89:0/283386982 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff5ac159c60 0x7ff5ac15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:22.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.491+0000 7ff5aaffd700 1 -- 172.21.15.89:0/283386982 >> v1:172.21.15.89:6789/0 conn(0x7ff5ac15db10 legacy=0x7ff5ac069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:22.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.491+0000 7ff5aa7fc700 1 -- 172.21.15.89:0/283386982 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397239608 0 0) 0x7ff5ac15a160 con 0x7ff5ac15db10 2023-10-10T20:43:22.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.491+0000 7ff5aaffd700 1 -- 172.21.15.89:0/283386982 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac15a3d0 con 0x7ff5ac159890 2023-10-10T20:43:22.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.491+0000 7ff5aaffd700 1 --2- 172.21.15.89:0/283386982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5ac159890 0x7ff5ac15abd0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0098e0 tx=0x7ff59c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff90ca3df75c0834 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:22.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.491+0000 7ff5aa7fc700 1 -- 172.21.15.89:0/283386982 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c009e80 con 0x7ff5ac159890 2023-10-10T20:43:22.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.491+0000 7ff5aa7fc700 1 -- 172.21.15.89:0/283386982 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c00b860 con 0x7ff5ac159890 2023-10-10T20:43:22.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.491+0000 7ff5aa7fc700 1 -- 172.21.15.89:0/283386982 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c00bbe0 con 0x7ff5ac159890 2023-10-10T20:43:22.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.491+0000 7ff5b196d700 1 -- 172.21.15.89:0/283386982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5ac159890 msgr2=0x7ff5ac15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:22.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.491+0000 7ff5b196d700 1 --2- 172.21.15.89:0/283386982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5ac159890 0x7ff5ac15abd0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0098e0 tx=0x7ff59c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:22.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.492+0000 7ff5b196d700 1 -- 172.21.15.89:0/283386982 shutdown_connections 2023-10-10T20:43:22.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.492+0000 7ff5b196d700 1 --2- 172.21.15.89:0/283386982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5ac159890 0x7ff5ac15abd0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:22.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.492+0000 7ff5b196d700 1 --2- 172.21.15.89:0/283386982 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff5ac159c60 0x7ff5ac15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:22.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.492+0000 7ff5b196d700 1 -- 172.21.15.89:0/283386982 >> 172.21.15.89:0/283386982 conn(0x7ff5ac068e20 msgr2=0x7ff5ac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:22.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.492+0000 7ff5b196d700 1 -- 172.21.15.89:0/283386982 shutdown_connections 2023-10-10T20:43:22.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.492+0000 7ff5b196d700 1 -- 172.21.15.89:0/283386982 wait complete. 2023-10-10T20:43:22.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.493+0000 7ff5b196d700 1 Processor -- start 2023-10-10T20:43:22.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.493+0000 7ff5b196d700 1 -- start start 2023-10-10T20:43:22.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.493+0000 7ff5b196d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff5ac159890 0x7ff5ac0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:22.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.494+0000 7ff5b196d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ac159c60 0x7ff5ac0cf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:22.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.494+0000 7ff5b196d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5ac15db10 0x7ff5ac16d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:22.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.494+0000 7ff5b196d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac0b46c0 con 0x7ff5ac15db10 2023-10-10T20:43:22.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.494+0000 7ff5b196d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac0b4100 con 0x7ff5ac159c60 2023-10-10T20:43:22.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.494+0000 7ff5b196d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac0b43e0 con 0x7ff5ac159890 2023-10-10T20:43:22.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.494+0000 7ff5ab7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5ac15db10 0x7ff5ac16d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:22.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.494+0000 7ff5ab7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5ac15db10 0x7ff5ac16d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59426/0 (socket says 172.21.15.89:59426) 2023-10-10T20:43:22.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.494+0000 7ff5ab7fe700 1 -- 172.21.15.89:0/36762552 learned_addr learned my addr 172.21.15.89:0/36762552 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:22.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.494+0000 7ff5ab7fe700 1 -- 172.21.15.89:0/36762552 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff5ac159890 msgr2=0x7ff5ac0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:22.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.494+0000 7ff5ab7fe700 1 --2- 172.21.15.89:0/36762552 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff5ac159890 0x7ff5ac0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:22.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.494+0000 7ff5ab7fe700 1 -- 172.21.15.89:0/36762552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ac159c60 msgr2=0x7ff5ac0cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:22.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.494+0000 7ff5ab7fe700 1 --2- 172.21.15.89:0/36762552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ac159c60 0x7ff5ac0cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:22.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.494+0000 7ff5ab7fe700 1 -- 172.21.15.89:0/36762552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c009420 con 0x7ff5ac15db10 2023-10-10T20:43:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.494+0000 7ff5ab7fe700 1 --2- 172.21.15.89:0/36762552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5ac15db10 0x7ff5ac16d190 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400cbf0 tx=0x7ff5a40019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.494+0000 7ff5a37fe700 1 -- 172.21.15.89:0/36762552 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a400a9c0 con 0x7ff5ac15db10 2023-10-10T20:43:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.494+0000 7ff5a37fe700 1 -- 172.21.15.89:0/36762552 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a400ab40 con 0x7ff5ac15db10 2023-10-10T20:43:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.494+0000 7ff5b196d700 1 -- 172.21.15.89:0/36762552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ac0c9a90 con 0x7ff5ac15db10 2023-10-10T20:43:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.494+0000 7ff5a37fe700 1 -- 172.21.15.89:0/36762552 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a40070f0 con 0x7ff5ac15db10 2023-10-10T20:43:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.494+0000 7ff5b196d700 1 -- 172.21.15.89:0/36762552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ac0cf860 con 0x7ff5ac15db10 2023-10-10T20:43:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.495+0000 7ff5b196d700 1 -- 172.21.15.89:0/36762552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5ac04c3c0 con 0x7ff5ac15db10 2023-10-10T20:43:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.495+0000 7ff5a37fe700 1 -- 172.21.15.89:0/36762552 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff5a4007520 con 0x7ff5ac15db10 2023-10-10T20:43:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.496+0000 7ff5a37fe700 1 --2- 172.21.15.89:0/36762552 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff58c067320 0x7ff58c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.496+0000 7ff5aaffd700 1 --2- 172.21.15.89:0/36762552 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff58c067320 0x7ff58c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:22.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.496+0000 7ff5a37fe700 1 -- 172.21.15.89:0/36762552 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff5a4084ca0 con 0x7ff5ac15db10 2023-10-10T20:43:22.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.496+0000 7ff5aaffd700 1 --2- 172.21.15.89:0/36762552 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff58c067320 0x7ff58c0697d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac0ca480 tx=0x7ff59c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:22.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.500+0000 7ff5a37fe700 1 -- 172.21.15.89:0/36762552 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5a4051cb0 con 0x7ff5ac15db10 2023-10-10T20:43:22.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.685+0000 7ff5b196d700 1 -- 172.21.15.89:0/36762552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff5ac06a5b0 con 0x7ff5ac15db10 2023-10-10T20:43:22.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.688+0000 7ff5a37fe700 1 -- 172.21.15.89:0/36762552 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff5a40552d0 con 0x7ff5ac15db10 2023-10-10T20:43:22.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.691+0000 7ff5b196d700 1 -- 172.21.15.89:0/36762552 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff58c067320 msgr2=0x7ff58c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:22.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.691+0000 7ff5b196d700 1 --2- 172.21.15.89:0/36762552 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff58c067320 0x7ff58c0697d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac0ca480 tx=0x7ff59c017000 comp rx=0 tx=0).stop 2023-10-10T20:43:22.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.691+0000 7ff5b196d700 1 -- 172.21.15.89:0/36762552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5ac15db10 msgr2=0x7ff5ac16d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:22.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.691+0000 7ff5b196d700 1 --2- 172.21.15.89:0/36762552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5ac15db10 0x7ff5ac16d190 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400cbf0 tx=0x7ff5a40019b0 comp rx=0 tx=0).stop 2023-10-10T20:43:22.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.691+0000 7ff5b196d700 1 -- 172.21.15.89:0/36762552 shutdown_connections 2023-10-10T20:43:22.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.691+0000 7ff5b196d700 1 --2- 172.21.15.89:0/36762552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5ac15db10 0x7ff5ac16d190 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:22.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.691+0000 7ff5b196d700 1 --2- 172.21.15.89:0/36762552 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff5ac159890 0x7ff5ac0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:22.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.691+0000 7ff5b196d700 1 --2- 172.21.15.89:0/36762552 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff58c067320 0x7ff58c0697d0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:22.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.691+0000 7ff5b196d700 1 --2- 172.21.15.89:0/36762552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ac159c60 0x7ff5ac0cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:22.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.691+0000 7ff5b196d700 1 -- 172.21.15.89:0/36762552 >> 172.21.15.89:0/36762552 conn(0x7ff5ac068e20 msgr2=0x7ff5ac0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:22.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.691+0000 7ff5b196d700 1 -- 172.21.15.89:0/36762552 shutdown_connections 2023-10-10T20:43:22.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.691+0000 7ff5b196d700 1 -- 172.21.15.89:0/36762552 wait complete. 2023-10-10T20:43:22.705 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool' 2023-10-10T20:43:22.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.822+0000 7f3c93b0b700 1 Processor -- start 2023-10-10T20:43:22.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.822+0000 7f3c93b0b700 1 -- start start 2023-10-10T20:43:22.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.823+0000 7f3c93b0b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c8c159c60 0x7f3c8c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:22.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.823+0000 7f3c93b0b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3c8c15db10 0x7f3c8c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:22.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.823+0000 7f3c93b0b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8c15a160 con 0x7f3c8c159890 2023-10-10T20:43:22.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.823+0000 7f3c93b0b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c8c0d13a0 con 0x7f3c8c15db10 2023-10-10T20:43:22.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.823+0000 7f3c93b0b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c15e140 con 0x7f3c8c159c60 2023-10-10T20:43:22.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.823+0000 7f3c920a8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3c8c15db10 0x7f3c8c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:22.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.823+0000 7f3c920a8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3c8c15db10 0x7f3c8c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59434/0 (socket says 172.21.15.89:59434) 2023-10-10T20:43:22.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.823+0000 7f3c918a7700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3c8c159890 0x7f3c8c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49576/0 (socket says 172.21.15.89:49576) 2023-10-10T20:43:22.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.823+0000 7f3c920a8700 1 -- 172.21.15.89:0/791084935 learned_addr learned my addr 172.21.15.89:0/791084935 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:22.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.823+0000 7f3c910a6700 1 --2- 172.21.15.89:0/791084935 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c8c159c60 0x7f3c8c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:22.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.823+0000 7f3c920a8700 1 -- 172.21.15.89:0/791084935 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c8c159c60 msgr2=0x7f3c8c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:22.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.823+0000 7f3c920a8700 1 --2- 172.21.15.89:0/791084935 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c8c159c60 0x7f3c8c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:22.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.823+0000 7f3c920a8700 1 -- 172.21.15.89:0/791084935 >> v1:172.21.15.89:6789/0 conn(0x7f3c8c159890 legacy=0x7f3c8c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:22.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.823+0000 7f3c920a8700 1 -- 172.21.15.89:0/791084935 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8c15a3d0 con 0x7f3c8c15db10 2023-10-10T20:43:22.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.823+0000 7f3c908a5700 1 -- 172.21.15.89:0/791084935 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1379944875 0 0) 0x7f3c8c15a160 con 0x7f3c8c159890 2023-10-10T20:43:22.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.823+0000 7f3c920a8700 1 --2- 172.21.15.89:0/791084935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c8c15db10 0x7f3c8c069560 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f3c84009f20 tx=0x7f3c84009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b570d61f3036206 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:22.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.823+0000 7f3c908a5700 1 -- 172.21.15.89:0/791084935 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8400e070 con 0x7f3c8c15db10 2023-10-10T20:43:22.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.823+0000 7f3c908a5700 1 -- 172.21.15.89:0/791084935 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8400ba80 con 0x7f3c8c15db10 2023-10-10T20:43:22.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.823+0000 7f3c908a5700 1 -- 172.21.15.89:0/791084935 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c840092e0 con 0x7f3c8c15db10 2023-10-10T20:43:22.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.824+0000 7f3c93b0b700 1 -- 172.21.15.89:0/791084935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c8c15db10 msgr2=0x7f3c8c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:22.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.824+0000 7f3c93b0b700 1 --2- 172.21.15.89:0/791084935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c8c15db10 0x7f3c8c069560 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f3c84009f20 tx=0x7f3c84009840 comp rx=0 tx=0).stop 2023-10-10T20:43:22.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.824+0000 7f3c93b0b700 1 -- 172.21.15.89:0/791084935 shutdown_connections 2023-10-10T20:43:22.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.824+0000 7f3c93b0b700 1 --2- 172.21.15.89:0/791084935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c8c15db10 0x7f3c8c069560 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:22.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.824+0000 7f3c93b0b700 1 --2- 172.21.15.89:0/791084935 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c8c159c60 0x7f3c8c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:22.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.824+0000 7f3c93b0b700 1 -- 172.21.15.89:0/791084935 >> 172.21.15.89:0/791084935 conn(0x7f3c8c068e20 msgr2=0x7f3c8c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:22.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.824+0000 7f3c93b0b700 1 -- 172.21.15.89:0/791084935 shutdown_connections 2023-10-10T20:43:22.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.824+0000 7f3c93b0b700 1 -- 172.21.15.89:0/791084935 wait complete. 2023-10-10T20:43:22.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.825+0000 7f3c93b0b700 1 Processor -- start 2023-10-10T20:43:22.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.825+0000 7f3c93b0b700 1 -- start start 2023-10-10T20:43:22.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.826+0000 7f3c93b0b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c8c159890 0x7f3c8c14c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:22.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.826+0000 7f3c93b0b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c8c159c60 0x7f3c8c149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:22.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.826+0000 7f3c93b0b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c8c15db10 0x7f3c8c1582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:22.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.826+0000 7f3c93b0b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c152540 con 0x7f3c8c159890 2023-10-10T20:43:22.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.826+0000 7f3c93b0b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c151930 con 0x7f3c8c15db10 2023-10-10T20:43:22.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.826+0000 7f3c93b0b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c151c10 con 0x7f3c8c159c60 2023-10-10T20:43:22.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.826+0000 7f3c918a7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c8c159890 0x7f3c8c14c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:22.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.826+0000 7f3c918a7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c8c159890 0x7f3c8c14c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59442/0 (socket says 172.21.15.89:59442) 2023-10-10T20:43:22.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.826+0000 7f3c918a7700 1 -- 172.21.15.89:0/2945150838 learned_addr learned my addr 172.21.15.89:0/2945150838 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:22.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.826+0000 7f3c910a6700 1 --2- 172.21.15.89:0/2945150838 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c8c159c60 0x7f3c8c149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:22.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.826+0000 7f3c920a8700 1 --2- 172.21.15.89:0/2945150838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c8c15db10 0x7f3c8c1582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:22.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.826+0000 7f3c918a7700 1 -- 172.21.15.89:0/2945150838 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c8c159c60 msgr2=0x7f3c8c149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:22.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.826+0000 7f3c918a7700 1 --2- 172.21.15.89:0/2945150838 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c8c159c60 0x7f3c8c149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:22.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.826+0000 7f3c918a7700 1 -- 172.21.15.89:0/2945150838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c8c15db10 msgr2=0x7f3c8c1582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:22.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.826+0000 7f3c918a7700 1 --2- 172.21.15.89:0/2945150838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c8c15db10 0x7f3c8c1582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:22.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.826+0000 7f3c918a7700 1 -- 172.21.15.89:0/2945150838 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c84009ae0 con 0x7f3c8c159890 2023-10-10T20:43:22.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.826+0000 7f3c918a7700 1 --2- 172.21.15.89:0/2945150838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c8c159890 0x7f3c8c14c3f0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f3c880011e0 tx=0x7f3c88007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:22.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.827+0000 7f3c81ffb700 1 -- 172.21.15.89:0/2945150838 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c88007300 con 0x7f3c8c159890 2023-10-10T20:43:22.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.827+0000 7f3c81ffb700 1 -- 172.21.15.89:0/2945150838 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c88007460 con 0x7f3c8c159890 2023-10-10T20:43:22.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.827+0000 7f3c93b0b700 1 -- 172.21.15.89:0/2945150838 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c8c151ef0 con 0x7f3c8c159890 2023-10-10T20:43:22.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.827+0000 7f3c81ffb700 1 -- 172.21.15.89:0/2945150838 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c88004830 con 0x7f3c8c159890 2023-10-10T20:43:22.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.827+0000 7f3c93b0b700 1 -- 172.21.15.89:0/2945150838 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c8c14c930 con 0x7f3c8c159890 2023-10-10T20:43:22.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.828+0000 7f3c93b0b700 1 -- 172.21.15.89:0/2945150838 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c8c152820 con 0x7f3c8c159890 2023-10-10T20:43:22.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.828+0000 7f3c81ffb700 1 -- 172.21.15.89:0/2945150838 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3c88004be0 con 0x7f3c8c159890 2023-10-10T20:43:22.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.828+0000 7f3c81ffb700 1 --2- 172.21.15.89:0/2945150838 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3c70067350 0x7f3c70069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:22.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.828+0000 7f3c81ffb700 1 -- 172.21.15.89:0/2945150838 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3c88083ee0 con 0x7f3c8c159890 2023-10-10T20:43:22.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.828+0000 7f3c910a6700 1 --2- 172.21.15.89:0/2945150838 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3c70067350 0x7f3c70069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:22.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.829+0000 7f3c910a6700 1 --2- 172.21.15.89:0/2945150838 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3c70067350 0x7f3c70069800 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c14d140 tx=0x7f3c7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:22.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:22.832+0000 7f3c81ffb700 1 -- 172.21.15.89:0/2945150838 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c88050ef0 con 0x7f3c8c159890 2023-10-10T20:43:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.092+0000 7f3c93b0b700 1 -- 172.21.15.89:0/2945150838 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]} v 0) v1 -- 0x7f3c8c04c3c0 con 0x7f3c8c159890 2023-10-10T20:43:23.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.352+0000 7f3c81ffb700 1 -- 172.21.15.89:0/2945150838 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]}]=0 v446) v1 ==== 163+0+0 (secure 0 0 0) 0x7f3c88018020 con 0x7f3c8c159890 2023-10-10T20:43:23.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.354+0000 7f3c93b0b700 1 -- 172.21.15.89:0/2945150838 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3c70067350 msgr2=0x7f3c70069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:23.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.354+0000 7f3c93b0b700 1 --2- 172.21.15.89:0/2945150838 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3c70067350 0x7f3c70069800 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c14d140 tx=0x7f3c7c008040 comp rx=0 tx=0).stop 2023-10-10T20:43:23.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.354+0000 7f3c93b0b700 1 -- 172.21.15.89:0/2945150838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c8c159890 msgr2=0x7f3c8c14c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:23.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.354+0000 7f3c93b0b700 1 --2- 172.21.15.89:0/2945150838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c8c159890 0x7f3c8c14c3f0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f3c880011e0 tx=0x7f3c88007e10 comp rx=0 tx=0).stop 2023-10-10T20:43:23.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.354+0000 7f3c93b0b700 1 -- 172.21.15.89:0/2945150838 shutdown_connections 2023-10-10T20:43:23.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.354+0000 7f3c93b0b700 1 --2- 172.21.15.89:0/2945150838 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3c70067350 0x7f3c70069800 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:23.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.354+0000 7f3c93b0b700 1 --2- 172.21.15.89:0/2945150838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c8c15db10 0x7f3c8c1582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:23.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.354+0000 7f3c93b0b700 1 --2- 172.21.15.89:0/2945150838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c8c159890 0x7f3c8c14c3f0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:23.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.354+0000 7f3c93b0b700 1 --2- 172.21.15.89:0/2945150838 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3c8c159c60 0x7f3c8c149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:23.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.354+0000 7f3c93b0b700 1 -- 172.21.15.89:0/2945150838 >> 172.21.15.89:0/2945150838 conn(0x7f3c8c068e20 msgr2=0x7f3c8c14d450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:23.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.354+0000 7f3c93b0b700 1 -- 172.21.15.89:0/2945150838 shutdown_connections 2023-10-10T20:43:23.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.355+0000 7f3c93b0b700 1 -- 172.21.15.89:0/2945150838 wait complete. 2023-10-10T20:43:23.368 INFO:tasks.cephfs_test_runner:That renaming a volume with more than one data pool does not change ... ok 2023-10-10T20:43:23.369 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename_idempotency (tasks.cephfs.test_volumes.TestVolumes) That volume rename is idempotent. 2023-10-10T20:43:23.369 INFO:tasks.cephfs_test_runner:test_volume_rename_idempotency (tasks.cephfs.test_volumes.TestVolumes) 2023-10-10T20:43:23.370 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency' 2023-10-10T20:43:23.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.482+0000 7fd5ec197700 1 Processor -- start 2023-10-10T20:43:23.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.482+0000 7fd5ec197700 1 -- start start 2023-10-10T20:43:23.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.483+0000 7fd5ec197700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd5e4159890 0x7fd5e415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:23.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.483+0000 7fd5ec197700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5e4159c60 0x7fd5e415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:23.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.483+0000 7fd5ec197700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd5e415db10 0x7fd5e4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:23.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.483+0000 7fd5ec197700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e415a160 con 0x7fd5e415db10 2023-10-10T20:43:23.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.483+0000 7fd5ec197700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e40d13a0 con 0x7fd5e4159890 2023-10-10T20:43:23.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.483+0000 7fd5ec197700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e415e140 con 0x7fd5e4159c60 2023-10-10T20:43:23.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.483+0000 7fd5ea734700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd5e415db10 0x7fd5e4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:23.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.483+0000 7fd5ea734700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd5e415db10 0x7fd5e4069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59448/0 (socket says 172.21.15.89:59448) 2023-10-10T20:43:23.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.483+0000 7fd5ea734700 1 -- 172.21.15.89:0/2895050643 learned_addr learned my addr 172.21.15.89:0/2895050643 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:23.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.483+0000 7fd5e9732700 1 --2- 172.21.15.89:0/2895050643 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5e4159c60 0x7fd5e415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:23.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.483+0000 7fd5e9f33700 1 --2- 172.21.15.89:0/2895050643 >> v2:172.21.15.111:3300/0 conn(0x7fd5e4159890 0x7fd5e415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:23.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.483+0000 7fd5ea734700 1 -- 172.21.15.89:0/2895050643 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5e4159c60 msgr2=0x7fd5e415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:23.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.483+0000 7fd5ea734700 1 --2- 172.21.15.89:0/2895050643 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5e4159c60 0x7fd5e415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:23.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.483+0000 7fd5ea734700 1 -- 172.21.15.89:0/2895050643 >> v2:172.21.15.111:3300/0 conn(0x7fd5e4159890 msgr2=0x7fd5e415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:23.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.483+0000 7fd5ea734700 1 --2- 172.21.15.89:0/2895050643 >> v2:172.21.15.111:3300/0 conn(0x7fd5e4159890 0x7fd5e415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:23.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.483+0000 7fd5ea734700 1 -- 172.21.15.89:0/2895050643 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e415a3d0 con 0x7fd5e415db10 2023-10-10T20:43:23.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.484+0000 7fd5ea734700 1 --2- 172.21.15.89:0/2895050643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e415db10 0x7fd5e4069560 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40d1090 tx=0x7fd5dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8443c8bceb52cf33 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:23.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.484+0000 7fd5e8f31700 1 -- 172.21.15.89:0/2895050643 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc00e070 con 0x7fd5e415db10 2023-10-10T20:43:23.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.484+0000 7fd5e8f31700 1 -- 172.21.15.89:0/2895050643 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5dc00ba50 con 0x7fd5e415db10 2023-10-10T20:43:23.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.484+0000 7fd5e8f31700 1 -- 172.21.15.89:0/2895050643 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0092e0 con 0x7fd5e415db10 2023-10-10T20:43:23.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.484+0000 7fd5ec197700 1 -- 172.21.15.89:0/2895050643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e415db10 msgr2=0x7fd5e4069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:23.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.484+0000 7fd5ec197700 1 --2- 172.21.15.89:0/2895050643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e415db10 0x7fd5e4069560 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40d1090 tx=0x7fd5dc009840 comp rx=0 tx=0).stop 2023-10-10T20:43:23.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.484+0000 7fd5ec197700 1 -- 172.21.15.89:0/2895050643 shutdown_connections 2023-10-10T20:43:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.484+0000 7fd5ec197700 1 --2- 172.21.15.89:0/2895050643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e415db10 0x7fd5e4069560 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.484+0000 7fd5ec197700 1 --2- 172.21.15.89:0/2895050643 >> v2:172.21.15.111:3300/0 conn(0x7fd5e4159890 0x7fd5e415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.484+0000 7fd5ec197700 1 --2- 172.21.15.89:0/2895050643 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5e4159c60 0x7fd5e415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.484+0000 7fd5ec197700 1 -- 172.21.15.89:0/2895050643 >> 172.21.15.89:0/2895050643 conn(0x7fd5e4068e20 msgr2=0x7fd5e40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.484+0000 7fd5ec197700 1 -- 172.21.15.89:0/2895050643 shutdown_connections 2023-10-10T20:43:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.484+0000 7fd5ec197700 1 -- 172.21.15.89:0/2895050643 wait complete. 2023-10-10T20:43:23.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.485+0000 7fd5ec197700 1 Processor -- start 2023-10-10T20:43:23.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.485+0000 7fd5ec197700 1 -- start start 2023-10-10T20:43:23.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.486+0000 7fd5ec197700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5e4159890 0x7fd5e41589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:23.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.486+0000 7fd5ec197700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e4159c60 0x7fd5e4152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:23.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.486+0000 7fd5ec197700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e415db10 0x7fd5e40b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:23.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.486+0000 7fd5ec197700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e4158f00 con 0x7fd5e4159c60 2023-10-10T20:43:23.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.486+0000 7fd5ec197700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e40b4c90 con 0x7fd5e415db10 2023-10-10T20:43:23.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.486+0000 7fd5ec197700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e40b4f70 con 0x7fd5e4159890 2023-10-10T20:43:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.486+0000 7fd5e9732700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e4159c60 0x7fd5e4152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.486+0000 7fd5e9732700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e4159c60 0x7fd5e4152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59454/0 (socket says 172.21.15.89:59454) 2023-10-10T20:43:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.486+0000 7fd5e9732700 1 -- 172.21.15.89:0/732981673 learned_addr learned my addr 172.21.15.89:0/732981673 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.486+0000 7fd5e9f33700 1 --2- 172.21.15.89:0/732981673 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5e4159890 0x7fd5e41589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.486+0000 7fd5ea734700 1 --2- 172.21.15.89:0/732981673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e415db10 0x7fd5e40b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:23.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.487+0000 7fd5e9732700 1 -- 172.21.15.89:0/732981673 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5e4159890 msgr2=0x7fd5e41589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:23.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.487+0000 7fd5e9732700 1 --2- 172.21.15.89:0/732981673 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5e4159890 0x7fd5e41589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:23.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.487+0000 7fd5e9732700 1 -- 172.21.15.89:0/732981673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e415db10 msgr2=0x7fd5e40b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:23.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.487+0000 7fd5e9732700 1 --2- 172.21.15.89:0/732981673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e415db10 0x7fd5e40b5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:23.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.487+0000 7fd5e9732700 1 -- 172.21.15.89:0/732981673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5dc009ae0 con 0x7fd5e4159c60 2023-10-10T20:43:23.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.487+0000 7fd5e9732700 1 --2- 172.21.15.89:0/732981673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e4159c60 0x7fd5e4152e70 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400d860 tx=0x7fd5d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:23.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.487+0000 7fd5d9ffb700 1 -- 172.21.15.89:0/732981673 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40125f0 con 0x7fd5e4159c60 2023-10-10T20:43:23.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.487+0000 7fd5d9ffb700 1 -- 172.21.15.89:0/732981673 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d4012750 con 0x7fd5e4159c60 2023-10-10T20:43:23.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.487+0000 7fd5ec197700 1 -- 172.21.15.89:0/732981673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e40b5250 con 0x7fd5e4159c60 2023-10-10T20:43:23.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.487+0000 7fd5d9ffb700 1 -- 172.21.15.89:0/732981673 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4012ad0 con 0x7fd5e4159c60 2023-10-10T20:43:23.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.487+0000 7fd5ec197700 1 -- 172.21.15.89:0/732981673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e40b5810 con 0x7fd5e4159c60 2023-10-10T20:43:23.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.488+0000 7fd5ec197700 1 -- 172.21.15.89:0/732981673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e404c3c0 con 0x7fd5e4159c60 2023-10-10T20:43:23.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.488+0000 7fd5d9ffb700 1 -- 172.21.15.89:0/732981673 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd5d401e460 con 0x7fd5e4159c60 2023-10-10T20:43:23.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.489+0000 7fd5d9ffb700 1 --2- 172.21.15.89:0/732981673 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd5c8067320 0x7fd5c80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:23.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.489+0000 7fd5d9ffb700 1 -- 172.21.15.89:0/732981673 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd5d4085030 con 0x7fd5e4159c60 2023-10-10T20:43:23.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.489+0000 7fd5e9f33700 1 --2- 172.21.15.89:0/732981673 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd5c8067320 0x7fd5c80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:23.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.489+0000 7fd5e9f33700 1 --2- 172.21.15.89:0/732981673 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd5c8067320 0x7fd5c80697d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0006fd0 tx=0x7fd5e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:23.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.493+0000 7fd5d9ffb700 1 -- 172.21.15.89:0/732981673 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5d4055a50 con 0x7fd5e4159c60 2023-10-10T20:43:23.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:23.751+0000 7fd5ec197700 1 -- 172.21.15.89:0/732981673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency"]} v 0) v1 -- 0x7fd5e406a5b0 con 0x7fd5e4159c60 2023-10-10T20:43:24.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.354+0000 7fd5d9ffb700 1 -- 172.21.15.89:0/732981673 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency"]}]=0 v447) v1 ==== 150+0+0 (secure 0 0 0) 0x7fd5d40555e0 con 0x7fd5e4159c60 2023-10-10T20:43:24.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.356+0000 7fd5ec197700 1 -- 172.21.15.89:0/732981673 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd5c8067320 msgr2=0x7fd5c80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:24.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.356+0000 7fd5ec197700 1 --2- 172.21.15.89:0/732981673 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd5c8067320 0x7fd5c80697d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0006fd0 tx=0x7fd5e0008040 comp rx=0 tx=0).stop 2023-10-10T20:43:24.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.356+0000 7fd5ec197700 1 -- 172.21.15.89:0/732981673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e4159c60 msgr2=0x7fd5e4152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:24.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.356+0000 7fd5ec197700 1 --2- 172.21.15.89:0/732981673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e4159c60 0x7fd5e4152e70 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400d860 tx=0x7fd5d4008040 comp rx=0 tx=0).stop 2023-10-10T20:43:24.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.356+0000 7fd5ec197700 1 -- 172.21.15.89:0/732981673 shutdown_connections 2023-10-10T20:43:24.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.356+0000 7fd5ec197700 1 --2- 172.21.15.89:0/732981673 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd5c8067320 0x7fd5c80697d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.356+0000 7fd5ec197700 1 --2- 172.21.15.89:0/732981673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e415db10 0x7fd5e40b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.356+0000 7fd5ec197700 1 --2- 172.21.15.89:0/732981673 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd5e4159890 0x7fd5e41589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.356+0000 7fd5ec197700 1 --2- 172.21.15.89:0/732981673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e4159c60 0x7fd5e4152e70 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.356+0000 7fd5ec197700 1 -- 172.21.15.89:0/732981673 >> 172.21.15.89:0/732981673 conn(0x7fd5e4068e20 msgr2=0x7fd5e40c6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:24.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.356+0000 7fd5ec197700 1 -- 172.21.15.89:0/732981673 shutdown_connections 2023-10-10T20:43:24.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.356+0000 7fd5ec197700 1 -- 172.21.15.89:0/732981673 wait complete. 2023-10-10T20:43:24.371 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:43:24.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.487+0000 7f321649e700 1 Processor -- start 2023-10-10T20:43:24.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.488+0000 7f321649e700 1 -- start start 2023-10-10T20:43:24.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.488+0000 7f321649e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3210159890 0x7f321015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:24.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.488+0000 7f321649e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f321015a160 con 0x7f3210069560 2023-10-10T20:43:24.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.488+0000 7f321649e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32100d13a0 con 0x7f3210159c60 2023-10-10T20:43:24.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.488+0000 7f321649e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321015e140 con 0x7f3210159890 2023-10-10T20:43:24.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.488+0000 7f3214a3b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3210069560 0x7f32100ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49590/0 (socket says 172.21.15.89:49590) 2023-10-10T20:43:24.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.488+0000 7f3214a3b700 1 -- 172.21.15.89:0/1132229924 learned_addr learned my addr 172.21.15.89:0/1132229924 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:24.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320effd700 1 -- 172.21.15.89:0/1132229924 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475547139 0 0) 0x7f321015a160 con 0x7f3210069560 2023-10-10T20:43:24.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320ffff700 1 --2- 172.21.15.89:0/1132229924 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3210159890 0x7f321015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:24.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320effd700 1 -- 172.21.15.89:0/1132229924 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31f8003570 con 0x7f3210069560 2023-10-10T20:43:24.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320effd700 1 -- 172.21.15.89:0/1132229924 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2825542247 0 0) 0x7f31f8003570 con 0x7f3210069560 2023-10-10T20:43:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320effd700 1 -- 172.21.15.89:0/1132229924 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3210159890 msgr2=0x7f321015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320effd700 1 --2- 172.21.15.89:0/1132229924 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3210159890 0x7f321015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320effd700 1 -- 172.21.15.89:0/1132229924 >> v1:172.21.15.111:6789/0 conn(0x7f3210159c60 legacy=0x7f321015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320effd700 1 -- 172.21.15.89:0/1132229924 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321015a3d0 con 0x7f3210069560 2023-10-10T20:43:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320effd700 1 -- 172.21.15.89:0/1132229924 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f32080017f0 con 0x7f3210069560 2023-10-10T20:43:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320effd700 1 -- 172.21.15.89:0/1132229924 >> v1:172.21.15.89:6789/0 conn(0x7f3210069560 legacy=0x7f32100ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320effd700 1 --2- 172.21.15.89:0/1132229924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31f8003a90 0x7f31f8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320effd700 1 --2- 172.21.15.89:0/1132229924 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3210159890 0x7f31f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320effd700 1 --2- 172.21.15.89:0/1132229924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31f8004d50 0x7f31f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320effd700 1 -- 172.21.15.89:0/1132229924 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321015e140 con 0x7f31f8003a90 2023-10-10T20:43:24.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320effd700 1 -- 172.21.15.89:0/1132229924 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f800bb70 con 0x7f31f8004d50 2023-10-10T20:43:24.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320effd700 1 -- 172.21.15.89:0/1132229924 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f800bd30 con 0x7f3210159890 2023-10-10T20:43:24.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320ffff700 1 --2- 172.21.15.89:0/1132229924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31f8003a90 0x7f31f8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:24.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320f7fe700 1 --2- 172.21.15.89:0/1132229924 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3210159890 0x7f31f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:24.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f3214a3b700 1 --2- 172.21.15.89:0/1132229924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31f8004d50 0x7f31f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:24.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320ffff700 1 -- 172.21.15.89:0/1132229924 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3210159890 msgr2=0x7f31f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:24.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320ffff700 1 --2- 172.21.15.89:0/1132229924 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3210159890 0x7f31f8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320ffff700 1 -- 172.21.15.89:0/1132229924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31f8004d50 msgr2=0x7f31f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:24.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320ffff700 1 --2- 172.21.15.89:0/1132229924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31f8004d50 0x7f31f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.489+0000 7f320ffff700 1 -- 172.21.15.89:0/1132229924 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f8003570 con 0x7f31f8003a90 2023-10-10T20:43:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.490+0000 7f320ffff700 1 --2- 172.21.15.89:0/1132229924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31f8003a90 0x7f31f8003ea0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f3200009c00 tx=0x7f32000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265c9d3a7b3b0d0f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.490+0000 7f320effd700 1 -- 172.21.15.89:0/1132229924 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320000e710 con 0x7f31f8003a90 2023-10-10T20:43:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.490+0000 7f320effd700 1 -- 172.21.15.89:0/1132229924 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320000e870 con 0x7f31f8003a90 2023-10-10T20:43:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.490+0000 7f320effd700 1 -- 172.21.15.89:0/1132229924 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320000ec30 con 0x7f31f8003a90 2023-10-10T20:43:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.490+0000 7f321649e700 1 -- 172.21.15.89:0/1132229924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31f8003a90 msgr2=0x7f31f8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.490+0000 7f321649e700 1 --2- 172.21.15.89:0/1132229924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31f8003a90 0x7f31f8003ea0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f3200009c00 tx=0x7f32000091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.490+0000 7f3214a3b700 1 -- 172.21.15.89:0/1132229924 reap_dead start 2023-10-10T20:43:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.490+0000 7f321649e700 1 -- 172.21.15.89:0/1132229924 shutdown_connections 2023-10-10T20:43:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.490+0000 7f321649e700 1 -- 172.21.15.89:0/1132229924 >> 172.21.15.89:0/1132229924 conn(0x7f3210068e20 msgr2=0x7f32100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.490+0000 7f321649e700 1 -- 172.21.15.89:0/1132229924 shutdown_connections 2023-10-10T20:43:24.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.490+0000 7f321649e700 1 -- 172.21.15.89:0/1132229924 wait complete. 2023-10-10T20:43:24.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.492+0000 7f321649e700 1 Processor -- start 2023-10-10T20:43:24.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.492+0000 7f321649e700 1 -- start start 2023-10-10T20:43:24.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.492+0000 7f321649e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31f8003a90 0x7f3210150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:24.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.492+0000 7f321649e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f32100b3970 0x7f32100b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:24.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.492+0000 7f321649e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32100c6800 0x7f32100c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:24.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.492+0000 7f321649e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32100c71e0 con 0x7f32100c6800 2023-10-10T20:43:24.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.492+0000 7f321649e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32100b5060 con 0x7f31f8003a90 2023-10-10T20:43:24.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.492+0000 7f321649e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3210151380 con 0x7f32100b3970 2023-10-10T20:43:24.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.493+0000 7f3214a3b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32100c6800 0x7f32100c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:24.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.493+0000 7f3214a3b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32100c6800 0x7f32100c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59482/0 (socket says 172.21.15.89:59482) 2023-10-10T20:43:24.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.493+0000 7f3214a3b700 1 -- 172.21.15.89:0/25055434 learned_addr learned my addr 172.21.15.89:0/25055434 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:24.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.493+0000 7f320f7fe700 1 --2- 172.21.15.89:0/25055434 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f32100b3970 0x7f32100b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:24.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.493+0000 7f320ffff700 1 --2- 172.21.15.89:0/25055434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31f8003a90 0x7f3210150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:24.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.493+0000 7f3214a3b700 1 -- 172.21.15.89:0/25055434 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f32100b3970 msgr2=0x7f32100b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:24.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.493+0000 7f3214a3b700 1 --2- 172.21.15.89:0/25055434 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f32100b3970 0x7f32100b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.493+0000 7f3214a3b700 1 -- 172.21.15.89:0/25055434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31f8003a90 msgr2=0x7f3210150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:24.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.493+0000 7f3214a3b700 1 --2- 172.21.15.89:0/25055434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31f8003a90 0x7f3210150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.493+0000 7f3214a3b700 1 -- 172.21.15.89:0/25055434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3210169bd0 con 0x7f32100c6800 2023-10-10T20:43:24.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.493+0000 7f320ffff700 1 --2- 172.21.15.89:0/25055434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31f8003a90 0x7f3210150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:24.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.493+0000 7f3214a3b700 1 --2- 172.21.15.89:0/25055434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32100c6800 0x7f32100c6ca0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f321006a890 tx=0x7f3208001350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:24.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.493+0000 7f31f7fff700 1 -- 172.21.15.89:0/25055434 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208017070 con 0x7f32100c6800 2023-10-10T20:43:24.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.493+0000 7f31f7fff700 1 -- 172.21.15.89:0/25055434 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320800ebf0 con 0x7f32100c6800 2023-10-10T20:43:24.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.493+0000 7f31f7fff700 1 -- 172.21.15.89:0/25055434 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208013420 con 0x7f32100c6800 2023-10-10T20:43:24.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.493+0000 7f321649e700 1 -- 172.21.15.89:0/25055434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32100c9200 con 0x7f32100c6800 2023-10-10T20:43:24.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.493+0000 7f321649e700 1 -- 172.21.15.89:0/25055434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32100b47c0 con 0x7f32100c6800 2023-10-10T20:43:24.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.494+0000 7f321649e700 1 -- 172.21.15.89:0/25055434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32100c9660 con 0x7f32100c6800 2023-10-10T20:43:24.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.495+0000 7f31f7fff700 1 -- 172.21.15.89:0/25055434 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f32080137d0 con 0x7f32100c6800 2023-10-10T20:43:24.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.495+0000 7f31f7fff700 1 --2- 172.21.15.89:0/25055434 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31fc067320 0x7f31fc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:24.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.495+0000 7f31f7fff700 1 -- 172.21.15.89:0/25055434 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f32080848c0 con 0x7f32100c6800 2023-10-10T20:43:24.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.495+0000 7f320ffff700 1 --2- 172.21.15.89:0/25055434 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31fc067320 0x7f31fc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:24.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.496+0000 7f320ffff700 1 --2- 172.21.15.89:0/25055434 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31fc067320 0x7f31fc0697d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f32000091e0 tx=0x7f320001a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:24.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.499+0000 7f31f7fff700 1 -- 172.21.15.89:0/25055434 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3208051850 con 0x7f32100c6800 2023-10-10T20:43:24.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.683+0000 7f321649e700 1 -- 172.21.15.89:0/25055434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f321004c3c0 con 0x7f32100c6800 2023-10-10T20:43:24.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.687+0000 7f31f7fff700 1 -- 172.21.15.89:0/25055434 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3208054e70 con 0x7f32100c6800 2023-10-10T20:43:24.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.689+0000 7f321649e700 1 -- 172.21.15.89:0/25055434 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31fc067320 msgr2=0x7f31fc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:24.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.689+0000 7f321649e700 1 --2- 172.21.15.89:0/25055434 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31fc067320 0x7f31fc0697d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f32000091e0 tx=0x7f320001a000 comp rx=0 tx=0).stop 2023-10-10T20:43:24.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.689+0000 7f321649e700 1 -- 172.21.15.89:0/25055434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32100c6800 msgr2=0x7f32100c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:24.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.689+0000 7f321649e700 1 --2- 172.21.15.89:0/25055434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32100c6800 0x7f32100c6ca0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f321006a890 tx=0x7f3208001350 comp rx=0 tx=0).stop 2023-10-10T20:43:24.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.689+0000 7f321649e700 1 -- 172.21.15.89:0/25055434 shutdown_connections 2023-10-10T20:43:24.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.689+0000 7f321649e700 1 --2- 172.21.15.89:0/25055434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32100c6800 0x7f32100c6ca0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.689+0000 7f321649e700 1 --2- 172.21.15.89:0/25055434 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f31fc067320 0x7f31fc0697d0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.689+0000 7f321649e700 1 --2- 172.21.15.89:0/25055434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31f8003a90 0x7f3210150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.689+0000 7f321649e700 1 --2- 172.21.15.89:0/25055434 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f32100b3970 0x7f32100b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.689+0000 7f321649e700 1 -- 172.21.15.89:0/25055434 >> 172.21.15.89:0/25055434 conn(0x7f3210068e20 msgr2=0x7f321015e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:24.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.689+0000 7f321649e700 1 -- 172.21.15.89:0/25055434 shutdown_connections 2023-10-10T20:43:24.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.689+0000 7f321649e700 1 -- 172.21.15.89:0/25055434 wait complete. 2023-10-10T20:43:24.703 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:43:24.704 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:43:24.742 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:43:24.790 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:43:24.790 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:43:24.791 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:43:24.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.903+0000 7f6e6ad50700 1 Processor -- start 2023-10-10T20:43:24.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.903+0000 7f6e6ad50700 1 -- start start 2023-10-10T20:43:24.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.904+0000 7f6e6ad50700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e640cbf30 0x7f6e640ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:24.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.904+0000 7f6e6ad50700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e640cc300 con 0x7f6e640ccf90 2023-10-10T20:43:24.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.904+0000 7f6e6ad50700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e640ad370 con 0x7f6e640b1420 2023-10-10T20:43:24.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.904+0000 7f6e6ad50700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e64156bf0 con 0x7f6e640cbf30 2023-10-10T20:43:24.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.904+0000 7f6e5bfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6e640ccf90 0x7f6e640b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49598/0 (socket says 172.21.15.89:49598) 2023-10-10T20:43:24.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.904+0000 7f6e5bfff700 1 -- 172.21.15.89:0/1333920596 learned_addr learned my addr 172.21.15.89:0/1333920596 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:24.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e68aec700 1 --2- 172.21.15.89:0/1333920596 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e640cbf30 0x7f6e640ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:24.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e5b7fe700 1 -- 172.21.15.89:0/1333920596 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 256162810 0 0) 0x7f6e640cc300 con 0x7f6e640ccf90 2023-10-10T20:43:24.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e5b7fe700 1 -- 172.21.15.89:0/1333920596 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e4c003570 con 0x7f6e640ccf90 2023-10-10T20:43:24.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e5b7fe700 1 -- 172.21.15.89:0/1333920596 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2765312054 0 0) 0x7f6e4c003570 con 0x7f6e640ccf90 2023-10-10T20:43:24.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e5b7fe700 1 -- 172.21.15.89:0/1333920596 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e640cbf30 msgr2=0x7f6e640ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:24.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e5b7fe700 1 --2- 172.21.15.89:0/1333920596 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e640cbf30 0x7f6e640ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e5b7fe700 1 -- 172.21.15.89:0/1333920596 >> v1:172.21.15.111:6789/0 conn(0x7f6e640b1420 legacy=0x7f6e640c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:24.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e5b7fe700 1 -- 172.21.15.89:0/1333920596 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e640ad650 con 0x7f6e640ccf90 2023-10-10T20:43:24.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e5b7fe700 1 -- 172.21.15.89:0/1333920596 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f6e54001470 con 0x7f6e640ccf90 2023-10-10T20:43:24.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e5b7fe700 1 -- 172.21.15.89:0/1333920596 >> v1:172.21.15.89:6789/0 conn(0x7f6e640ccf90 legacy=0x7f6e640b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:24.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e5b7fe700 1 --2- 172.21.15.89:0/1333920596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e4c003a90 0x7f6e4c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:24.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e5b7fe700 1 --2- 172.21.15.89:0/1333920596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e4c004810 0x7f6e4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:24.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e5b7fe700 1 --2- 172.21.15.89:0/1333920596 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e640cbf30 0x7f6e4c0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:24.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e5b7fe700 1 -- 172.21.15.89:0/1333920596 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e64156bf0 con 0x7f6e4c003a90 2023-10-10T20:43:24.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e5b7fe700 1 -- 172.21.15.89:0/1333920596 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c00bb60 con 0x7f6e4c004810 2023-10-10T20:43:24.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e5b7fe700 1 -- 172.21.15.89:0/1333920596 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c00bd20 con 0x7f6e640cbf30 2023-10-10T20:43:24.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e68aec700 1 --2- 172.21.15.89:0/1333920596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e4c003a90 0x7f6e4c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:24.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e5bfff700 1 --2- 172.21.15.89:0/1333920596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e4c004810 0x7f6e4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:24.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e692ed700 1 --2- 172.21.15.89:0/1333920596 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e640cbf30 0x7f6e4c0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:24.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e68aec700 1 -- 172.21.15.89:0/1333920596 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e640cbf30 msgr2=0x7f6e4c0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:24.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e68aec700 1 --2- 172.21.15.89:0/1333920596 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e640cbf30 0x7f6e4c0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e68aec700 1 -- 172.21.15.89:0/1333920596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e4c004810 msgr2=0x7f6e4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:24.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e68aec700 1 --2- 172.21.15.89:0/1333920596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e4c004810 0x7f6e4c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.905+0000 7f6e68aec700 1 -- 172.21.15.89:0/1333920596 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c003570 con 0x7f6e4c003a90 2023-10-10T20:43:24.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.906+0000 7f6e68aec700 1 --2- 172.21.15.89:0/1333920596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e4c003a90 0x7f6e4c003ea0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009c00 tx=0x7f6e600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e62125ff7fd974 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:24.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.906+0000 7f6e5b7fe700 1 -- 172.21.15.89:0/1333920596 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6000e710 con 0x7f6e4c003a90 2023-10-10T20:43:24.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.906+0000 7f6e5b7fe700 1 -- 172.21.15.89:0/1333920596 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6000e870 con 0x7f6e4c003a90 2023-10-10T20:43:24.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.906+0000 7f6e5b7fe700 1 -- 172.21.15.89:0/1333920596 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6000ec30 con 0x7f6e4c003a90 2023-10-10T20:43:24.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.906+0000 7f6e6ad50700 1 -- 172.21.15.89:0/1333920596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e4c003a90 msgr2=0x7f6e4c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:24.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.906+0000 7f6e6ad50700 1 --2- 172.21.15.89:0/1333920596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e4c003a90 0x7f6e4c003ea0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009c00 tx=0x7f6e600091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.906+0000 7f6e692ed700 1 -- 172.21.15.89:0/1333920596 reap_dead start 2023-10-10T20:43:24.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.906+0000 7f6e6ad50700 1 -- 172.21.15.89:0/1333920596 shutdown_connections 2023-10-10T20:43:24.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.906+0000 7f6e6ad50700 1 -- 172.21.15.89:0/1333920596 >> 172.21.15.89:0/1333920596 conn(0x7f6e6406a320 msgr2=0x7f6e640cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:24.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.906+0000 7f6e6ad50700 1 -- 172.21.15.89:0/1333920596 shutdown_connections 2023-10-10T20:43:24.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.906+0000 7f6e6ad50700 1 -- 172.21.15.89:0/1333920596 wait complete. 2023-10-10T20:43:24.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.907+0000 7f6e6ad50700 1 Processor -- start 2023-10-10T20:43:24.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.908+0000 7f6e6ad50700 1 -- start start 2023-10-10T20:43:24.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.908+0000 7f6e6ad50700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e4c003a90 0x7f6e64165910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:24.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.908+0000 7f6e6ad50700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e640a7920 0x7f6e640a7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:24.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.908+0000 7f6e6ad50700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e640a64d0 0x7f6e640a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:24.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.908+0000 7f6e6ad50700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e640a54c0 con 0x7f6e640a64d0 2023-10-10T20:43:24.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.908+0000 7f6e6ad50700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e640a82d0 con 0x7f6e640a7920 2023-10-10T20:43:24.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.908+0000 7f6e6ad50700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e640a5710 con 0x7f6e4c003a90 2023-10-10T20:43:24.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.908+0000 7f6e692ed700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e640a64d0 0x7f6e640a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:24.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.908+0000 7f6e692ed700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e640a64d0 0x7f6e640a6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59492/0 (socket says 172.21.15.89:59492) 2023-10-10T20:43:24.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.908+0000 7f6e692ed700 1 -- 172.21.15.89:0/2077830941 learned_addr learned my addr 172.21.15.89:0/2077830941 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:24.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.908+0000 7f6e5bfff700 1 --2- 172.21.15.89:0/2077830941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e640a7920 0x7f6e640a7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:24.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.908+0000 7f6e68aec700 1 --2- 172.21.15.89:0/2077830941 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e4c003a90 0x7f6e64165910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:24.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.909+0000 7f6e692ed700 1 -- 172.21.15.89:0/2077830941 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e4c003a90 msgr2=0x7f6e64165910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:24.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.909+0000 7f6e692ed700 1 --2- 172.21.15.89:0/2077830941 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e4c003a90 0x7f6e64165910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.909+0000 7f6e692ed700 1 -- 172.21.15.89:0/2077830941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e640a7920 msgr2=0x7f6e640a7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:24.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.909+0000 7f6e692ed700 1 --2- 172.21.15.89:0/2077830941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e640a7920 0x7f6e640a7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:24.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.909+0000 7f6e692ed700 1 -- 172.21.15.89:0/2077830941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e640a59f0 con 0x7f6e640a64d0 2023-10-10T20:43:24.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.909+0000 7f6e5bfff700 1 --2- 172.21.15.89:0/2077830941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e640a7920 0x7f6e640a7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:24.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.909+0000 7f6e692ed700 1 --2- 172.21.15.89:0/2077830941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e640a64d0 0x7f6e640a6970 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f6e640aecb0 tx=0x7f6e5c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:24.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.909+0000 7f6e58ff9700 1 -- 172.21.15.89:0/2077830941 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c006db0 con 0x7f6e640a64d0 2023-10-10T20:43:24.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.909+0000 7f6e58ff9700 1 -- 172.21.15.89:0/2077830941 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c00c5a0 con 0x7f6e640a64d0 2023-10-10T20:43:24.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.909+0000 7f6e6ad50700 1 -- 172.21.15.89:0/2077830941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e640a5cd0 con 0x7f6e640a64d0 2023-10-10T20:43:24.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.909+0000 7f6e58ff9700 1 -- 172.21.15.89:0/2077830941 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c00c960 con 0x7f6e640a64d0 2023-10-10T20:43:24.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.909+0000 7f6e6ad50700 1 -- 172.21.15.89:0/2077830941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e64165e50 con 0x7f6e640a64d0 2023-10-10T20:43:24.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.910+0000 7f6e6ad50700 1 -- 172.21.15.89:0/2077830941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6404c3c0 con 0x7f6e640a64d0 2023-10-10T20:43:24.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.910+0000 7f6e58ff9700 1 -- 172.21.15.89:0/2077830941 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6e5c00ccd0 con 0x7f6e640a64d0 2023-10-10T20:43:24.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.911+0000 7f6e58ff9700 1 --2- 172.21.15.89:0/2077830941 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6e50067320 0x7f6e500697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:24.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.911+0000 7f6e58ff9700 1 -- 172.21.15.89:0/2077830941 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6e5c084250 con 0x7f6e640a64d0 2023-10-10T20:43:24.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.911+0000 7f6e68aec700 1 --2- 172.21.15.89:0/2077830941 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6e50067320 0x7f6e500697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:24.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.911+0000 7f6e68aec700 1 --2- 172.21.15.89:0/2077830941 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6e50067320 0x7f6e500697d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6001c040 tx=0x7f6e60019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:24.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:24.915+0000 7f6e58ff9700 1 -- 172.21.15.89:0/2077830941 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e5c0511e0 con 0x7f6e640a64d0 2023-10-10T20:43:25.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.139+0000 7f6e6ad50700 1 -- 172.21.15.89:0/2077830941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e640ad930 con 0x7f6e640a64d0 2023-10-10T20:43:25.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.139+0000 7f6e58ff9700 1 -- 172.21.15.89:0/2077830941 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+3540 (secure 0 0 0) 0x7f6e5c019df0 con 0x7f6e640a64d0 2023-10-10T20:43:25.142 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:25.142 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":115,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6638,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2350308973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":2350308973},{"type":"v1","addr":"172.21.15.111:6835","nonce":2350308973}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":7645,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/3725222634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3725222634},{"type":"v1","addr":"172.21.15.111:6834","nonce":3725222634}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115},{"gid":8322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/1051252386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":1051252386},{"type":"v1","addr":"172.21.15.89:6836","nonce":1051252386}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105},{"gid":8325,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3754994248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":3754994248},{"type":"v1","addr":"172.21.15.89:6837","nonce":3754994248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105}],"filesystems":[]} 2023-10-10T20:43:25.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.141+0000 7f6e6ad50700 1 -- 172.21.15.89:0/2077830941 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6e50067320 msgr2=0x7f6e500697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:25.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.141+0000 7f6e6ad50700 1 --2- 172.21.15.89:0/2077830941 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6e50067320 0x7f6e500697d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6001c040 tx=0x7f6e60019000 comp rx=0 tx=0).stop 2023-10-10T20:43:25.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.141+0000 7f6e6ad50700 1 -- 172.21.15.89:0/2077830941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e640a64d0 msgr2=0x7f6e640a6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:25.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.141+0000 7f6e6ad50700 1 --2- 172.21.15.89:0/2077830941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e640a64d0 0x7f6e640a6970 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f6e640aecb0 tx=0x7f6e5c001640 comp rx=0 tx=0).stop 2023-10-10T20:43:25.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.141+0000 7f6e6ad50700 1 -- 172.21.15.89:0/2077830941 shutdown_connections 2023-10-10T20:43:25.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.141+0000 7f6e6ad50700 1 --2- 172.21.15.89:0/2077830941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e640a64d0 0x7f6e640a6970 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:25.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.141+0000 7f6e6ad50700 1 --2- 172.21.15.89:0/2077830941 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6e4c003a90 0x7f6e64165910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:25.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.141+0000 7f6e6ad50700 1 --2- 172.21.15.89:0/2077830941 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6e50067320 0x7f6e500697d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:25.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.141+0000 7f6e6ad50700 1 --2- 172.21.15.89:0/2077830941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e640a7920 0x7f6e640a7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:25.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.141+0000 7f6e6ad50700 1 -- 172.21.15.89:0/2077830941 >> 172.21.15.89:0/2077830941 conn(0x7f6e6406a320 msgr2=0x7f6e6406eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:25.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.141+0000 7f6e6ad50700 1 -- 172.21.15.89:0/2077830941 shutdown_connections 2023-10-10T20:43:25.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.141+0000 7f6e6ad50700 1 -- 172.21.15.89:0/2077830941 wait complete. 2023-10-10T20:43:25.146 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 115 2023-10-10T20:43:25.156 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:43:25.157 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:43:25.157 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:43:25.157 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:43:25.158 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:43:25.158 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:43:25.158 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:43:25.159 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:43:25.159 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:43:25.159 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:43:25.160 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:43:25.160 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:43:25.160 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:43:25.161 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:43:25.161 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:43:25.161 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:43:25.166 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:43:25.166 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:43:25.167 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:43:25.167 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:43:25.170 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:43:25.171 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:43:25.172 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:43:25.172 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:43:25.208 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:43:25.208 DEBUG:teuthology.parallel:result is None 2023-10-10T20:43:25.209 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:43:25.209 DEBUG:teuthology.parallel:result is None 2023-10-10T20:43:25.213 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:43:25.214 DEBUG:teuthology.parallel:result is None 2023-10-10T20:43:25.214 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:43:25.215 DEBUG:teuthology.parallel:result is None 2023-10-10T20:43:25.215 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:43:25.281 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-10T20:43:25.281 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:43:25.296 INFO:tasks.ceph.mds.d.smithi111.stdout:starting mds.d at 2023-10-10T20:43:25.297 INFO:tasks.ceph.mds.b.smithi111.stdout:starting mds.b at 2023-10-10T20:43:25.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.334+0000 7f90a2ff7700 1 Processor -- start 2023-10-10T20:43:25.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.334+0000 7f90a2ff7700 1 -- start start 2023-10-10T20:43:25.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.335+0000 7f90a2ff7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f909c159420 0x7f909c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:25.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.335+0000 7f90a2ff7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f909c0b2980 con 0x7f909c1597f0 2023-10-10T20:43:25.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.335+0000 7f90a2ff7700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f909c0b11d0 con 0x7f909c15d230 2023-10-10T20:43:25.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.335+0000 7f90a2ff7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f909c0b14b0 con 0x7f909c159420 2023-10-10T20:43:25.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.335+0000 7f90a1ff5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f909c159420 0x7f909c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:25.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.335+0000 7f90a17f4700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f909c1597f0 0x7f909c0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49604/0 (socket says 172.21.15.89:49604) 2023-10-10T20:43:25.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.335+0000 7f90a1ff5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f909c159420 0x7f909c0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59510/0 (socket says 172.21.15.89:59510) 2023-10-10T20:43:25.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.335+0000 7f90a17f4700 1 -- 172.21.15.89:0/3979651080 learned_addr learned my addr 172.21.15.89:0/3979651080 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:25.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.335+0000 7f90a1ff5700 1 -- 172.21.15.89:0/3979651080 >> v1:172.21.15.111:6789/0 conn(0x7f909c15d230 legacy=0x7f909c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:25.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.335+0000 7f90a1ff5700 1 -- 172.21.15.89:0/3979651080 >> v1:172.21.15.89:6789/0 conn(0x7f909c1597f0 legacy=0x7f909c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:25.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.335+0000 7f90a1ff5700 1 -- 172.21.15.89:0/3979651080 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c0b1790 con 0x7f909c159420 2023-10-10T20:43:25.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.335+0000 7f90a0ff3700 1 -- 172.21.15.89:0/3979651080 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 899989700 0 0) 0x7f909c0b2980 con 0x7f909c1597f0 2023-10-10T20:43:25.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.335+0000 7f90a1ff5700 1 --2- 172.21.15.89:0/3979651080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f909c159420 0x7f909c0b5f40 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f9094009e60 tx=0x7f9094009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b3857bed367690 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:25.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.335+0000 7f90a0ff3700 1 -- 172.21.15.89:0/3979651080 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400e070 con 0x7f909c159420 2023-10-10T20:43:25.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.336+0000 7f90a0ff3700 1 -- 172.21.15.89:0/3979651080 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909400bbf0 con 0x7f909c159420 2023-10-10T20:43:25.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.336+0000 7f90a0ff3700 1 -- 172.21.15.89:0/3979651080 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094012430 con 0x7f909c159420 2023-10-10T20:43:25.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.336+0000 7f90a2ff7700 1 -- 172.21.15.89:0/3979651080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f909c159420 msgr2=0x7f909c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:25.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.336+0000 7f90a2ff7700 1 --2- 172.21.15.89:0/3979651080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f909c159420 0x7f909c0b5f40 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f9094009e60 tx=0x7f9094009780 comp rx=0 tx=0).stop 2023-10-10T20:43:25.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.336+0000 7f90a2ff7700 1 -- 172.21.15.89:0/3979651080 shutdown_connections 2023-10-10T20:43:25.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.336+0000 7f90a2ff7700 1 --2- 172.21.15.89:0/3979651080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f909c159420 0x7f909c0b5f40 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:25.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.336+0000 7f90a2ff7700 1 -- 172.21.15.89:0/3979651080 >> 172.21.15.89:0/3979651080 conn(0x7f909c0d0660 msgr2=0x7f909c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:25.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.336+0000 7f90a2ff7700 1 -- 172.21.15.89:0/3979651080 shutdown_connections 2023-10-10T20:43:25.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.336+0000 7f90a2ff7700 1 -- 172.21.15.89:0/3979651080 wait complete. 2023-10-10T20:43:25.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.337+0000 7f90a2ff7700 1 Processor -- start 2023-10-10T20:43:25.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.337+0000 7f90a2ff7700 1 -- start start 2023-10-10T20:43:25.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.338+0000 7f90a2ff7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f909c159420 0x7f909c141f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:25.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.338+0000 7f90a2ff7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f909c1597f0 0x7f909c0b5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:25.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.338+0000 7f90a2ff7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f909c15d230 0x7f909c0b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:25.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.338+0000 7f90a2ff7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c15c3b0 con 0x7f909c159420 2023-10-10T20:43:25.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.338+0000 7f90a2ff7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c15b7a0 con 0x7f909c1597f0 2023-10-10T20:43:25.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.338+0000 7f90a2ff7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909c15ba80 con 0x7f909c15d230 2023-10-10T20:43:25.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.338+0000 7f90a1ff5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f909c159420 0x7f909c141f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:25.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.338+0000 7f90a1ff5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f909c159420 0x7f909c141f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59514/0 (socket says 172.21.15.89:59514) 2023-10-10T20:43:25.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.338+0000 7f90a1ff5700 1 -- 172.21.15.89:0/1564935023 learned_addr learned my addr 172.21.15.89:0/1564935023 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:25.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.338+0000 7f90a27f6700 1 --2- 172.21.15.89:0/1564935023 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f909c15d230 0x7f909c0b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:25.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.338+0000 7f90a17f4700 1 --2- 172.21.15.89:0/1564935023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f909c1597f0 0x7f909c0b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:25.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.338+0000 7f90a1ff5700 1 -- 172.21.15.89:0/1564935023 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f909c15d230 msgr2=0x7f909c0b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:25.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.338+0000 7f90a1ff5700 1 --2- 172.21.15.89:0/1564935023 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f909c15d230 0x7f909c0b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:25.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.338+0000 7f90a1ff5700 1 -- 172.21.15.89:0/1564935023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f909c1597f0 msgr2=0x7f909c0b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:25.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.338+0000 7f90a1ff5700 1 --2- 172.21.15.89:0/1564935023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f909c1597f0 0x7f909c0b5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:25.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.338+0000 7f90a1ff5700 1 -- 172.21.15.89:0/1564935023 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9094009a20 con 0x7f909c159420 2023-10-10T20:43:25.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.339+0000 7f90a1ff5700 1 --2- 172.21.15.89:0/1564935023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f909c159420 0x7f909c141f70 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f909400b6c0 tx=0x7f9094018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:25.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.339+0000 7f9099ffb700 1 -- 172.21.15.89:0/1564935023 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094027470 con 0x7f909c159420 2023-10-10T20:43:25.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.339+0000 7f9099ffb700 1 -- 172.21.15.89:0/1564935023 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90940275f0 con 0x7f909c159420 2023-10-10T20:43:25.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.339+0000 7f90a2ff7700 1 -- 172.21.15.89:0/1564935023 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f909c15bd60 con 0x7f909c159420 2023-10-10T20:43:25.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.339+0000 7f9099ffb700 1 -- 172.21.15.89:0/1564935023 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90940279b0 con 0x7f909c159420 2023-10-10T20:43:25.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.339+0000 7f90a2ff7700 1 -- 172.21.15.89:0/1564935023 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909c154280 con 0x7f909c159420 2023-10-10T20:43:25.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.340+0000 7f90a2ff7700 1 -- 172.21.15.89:0/1564935023 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f909c04c3c0 con 0x7f909c159420 2023-10-10T20:43:25.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.340+0000 7f9099ffb700 1 -- 172.21.15.89:0/1564935023 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9094024070 con 0x7f909c159420 2023-10-10T20:43:25.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.341+0000 7f9099ffb700 1 --2- 172.21.15.89:0/1564935023 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9088067320 0x7f90880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:25.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.341+0000 7f9099ffb700 1 -- 172.21.15.89:0/1564935023 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f90940874d0 con 0x7f909c159420 2023-10-10T20:43:25.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.341+0000 7f90a17f4700 1 --2- 172.21.15.89:0/1564935023 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9088067320 0x7f90880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:25.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.341+0000 7f90a17f4700 1 --2- 172.21.15.89:0/1564935023 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9088067320 0x7f90880697d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9090007fd0 tx=0x7f9090007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:25.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.345+0000 7f9099ffb700 1 -- 172.21.15.89:0/1564935023 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9094054460 con 0x7f909c159420 2023-10-10T20:43:25.374 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-10T20:43:25.375 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-10-10T20:43:25.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:25.548+0000 7f90a2ff7700 1 -- 172.21.15.89:0/1564935023 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f909c0b1a70 con 0x7f909c159420 2023-10-10T20:43:26.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.386+0000 7f9099ffb700 1 -- 172.21.15.89:0/1564935023 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v179) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9094057a80 con 0x7f909c159420 2023-10-10T20:43:26.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.388+0000 7f90a2ff7700 1 -- 172.21.15.89:0/1564935023 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9088067320 msgr2=0x7f90880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:26.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.388+0000 7f90a2ff7700 1 --2- 172.21.15.89:0/1564935023 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9088067320 0x7f90880697d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9090007fd0 tx=0x7f9090007d30 comp rx=0 tx=0).stop 2023-10-10T20:43:26.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.389+0000 7f90a2ff7700 1 -- 172.21.15.89:0/1564935023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f909c159420 msgr2=0x7f909c141f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:26.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.389+0000 7f90a2ff7700 1 --2- 172.21.15.89:0/1564935023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f909c159420 0x7f909c141f70 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f909400b6c0 tx=0x7f9094018000 comp rx=0 tx=0).stop 2023-10-10T20:43:26.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.389+0000 7f90a2ff7700 1 -- 172.21.15.89:0/1564935023 shutdown_connections 2023-10-10T20:43:26.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.389+0000 7f90a2ff7700 1 --2- 172.21.15.89:0/1564935023 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f909c15d230 0x7f909c0b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:26.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.389+0000 7f90a2ff7700 1 --2- 172.21.15.89:0/1564935023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f909c159420 0x7f909c141f70 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:26.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.389+0000 7f90a2ff7700 1 --2- 172.21.15.89:0/1564935023 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9088067320 0x7f90880697d0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:26.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.389+0000 7f90a2ff7700 1 --2- 172.21.15.89:0/1564935023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f909c1597f0 0x7f909c0b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:26.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.389+0000 7f90a2ff7700 1 -- 172.21.15.89:0/1564935023 >> 172.21.15.89:0/1564935023 conn(0x7f909c0d0660 msgr2=0x7f909c0b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:26.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.389+0000 7f90a2ff7700 1 -- 172.21.15.89:0/1564935023 shutdown_connections 2023-10-10T20:43:26.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.389+0000 7f90a2ff7700 1 -- 172.21.15.89:0/1564935023 wait complete. 2023-10-10T20:43:26.393 INFO:teuthology.orchestra.run.smithi089.stderr: removed all blocklist entries 2023-10-10T20:43:26.403 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:43:26.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.516+0000 7f3de88d8700 1 Processor -- start 2023-10-10T20:43:26.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.516+0000 7f3de88d8700 1 -- start start 2023-10-10T20:43:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.517+0000 7f3de88d8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3de41593b0 0x7f3de4159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.517+0000 7f3de88d8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3de40b1e50 0x7f3de40b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:26.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.517+0000 7f3de88d8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de40b6570 con 0x7f3de40ceaa0 2023-10-10T20:43:26.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.517+0000 7f3de88d8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de40cfe20 con 0x7f3de40b1e50 2023-10-10T20:43:26.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.517+0000 7f3de88d8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40cf210 con 0x7f3de41593b0 2023-10-10T20:43:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.517+0000 7f3de3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3de40b1e50 0x7f3de40b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.517+0000 7f3de2ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3de40ceaa0 0x7f3de40b1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49608/0 (socket says 172.21.15.89:49608) 2023-10-10T20:43:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.517+0000 7f3de2ffd700 1 -- 172.21.15.89:0/3629822691 learned_addr learned my addr 172.21.15.89:0/3629822691 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.517+0000 7f3de3fff700 1 -- 172.21.15.89:0/3629822691 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3de41593b0 msgr2=0x7f3de4159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.518+0000 7f3de37fe700 1 --2- 172.21.15.89:0/3629822691 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3de41593b0 0x7f3de4159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.518+0000 7f3de27fc700 1 -- 172.21.15.89:0/3629822691 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621221710 0 0) 0x7f3dd8001110 con 0x7f3de40ceaa0 2023-10-10T20:43:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.518+0000 7f3de3fff700 1 --2- 172.21.15.89:0/3629822691 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3de41593b0 0x7f3de4159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.518+0000 7f3de3fff700 1 -- 172.21.15.89:0/3629822691 >> v1:172.21.15.89:6789/0 conn(0x7f3de40ceaa0 legacy=0x7f3de40b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.518+0000 7f3de3fff700 1 -- 172.21.15.89:0/3629822691 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de40cf4f0 con 0x7f3de40b1e50 2023-10-10T20:43:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.518+0000 7f3de37fe700 1 --2- 172.21.15.89:0/3629822691 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3de41593b0 0x7f3de4159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.518+0000 7f3de3fff700 1 --2- 172.21.15.89:0/3629822691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3de40b1e50 0x7f3de40b6030 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0098e0 tx=0x7f3ddc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d599d669794ca1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:26.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.518+0000 7f3de27fc700 1 -- 172.21.15.89:0/3629822691 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc009e80 con 0x7f3de40b1e50 2023-10-10T20:43:26.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.518+0000 7f3de27fc700 1 -- 172.21.15.89:0/3629822691 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc00b860 con 0x7f3de40b1e50 2023-10-10T20:43:26.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.518+0000 7f3de27fc700 1 -- 172.21.15.89:0/3629822691 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc00bbe0 con 0x7f3de40b1e50 2023-10-10T20:43:26.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.518+0000 7f3de88d8700 1 -- 172.21.15.89:0/3629822691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3de40b1e50 msgr2=0x7f3de40b6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:26.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.518+0000 7f3de88d8700 1 --2- 172.21.15.89:0/3629822691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3de40b1e50 0x7f3de40b6030 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0098e0 tx=0x7f3ddc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:26.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.518+0000 7f3de88d8700 1 -- 172.21.15.89:0/3629822691 shutdown_connections 2023-10-10T20:43:26.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.518+0000 7f3de88d8700 1 --2- 172.21.15.89:0/3629822691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3de40b1e50 0x7f3de40b6030 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:26.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.518+0000 7f3de88d8700 1 --2- 172.21.15.89:0/3629822691 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3de41593b0 0x7f3de4159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:26.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.518+0000 7f3de88d8700 1 -- 172.21.15.89:0/3629822691 >> 172.21.15.89:0/3629822691 conn(0x7f3de40d1130 msgr2=0x7f3de40b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:26.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.518+0000 7f3de88d8700 1 -- 172.21.15.89:0/3629822691 shutdown_connections 2023-10-10T20:43:26.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.518+0000 7f3de88d8700 1 -- 172.21.15.89:0/3629822691 wait complete. 2023-10-10T20:43:26.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.520+0000 7f3de88d8700 1 Processor -- start 2023-10-10T20:43:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.520+0000 7f3de88d8700 1 -- start start 2023-10-10T20:43:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.520+0000 7f3de88d8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3de40b1e50 0x7f3de40c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.520+0000 7f3de88d8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3de40ceaa0 0x7f3de4156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.520+0000 7f3de88d8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3de41593b0 0x7f3de40c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.520+0000 7f3de88d8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40c5ad0 con 0x7f3de40ceaa0 2023-10-10T20:43:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.520+0000 7f3de88d8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40caa50 con 0x7f3de40b1e50 2023-10-10T20:43:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.520+0000 7f3de88d8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40cad00 con 0x7f3de41593b0 2023-10-10T20:43:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.521+0000 7f3de2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3de40ceaa0 0x7f3de4156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.521+0000 7f3de2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3de40ceaa0 0x7f3de4156340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59534/0 (socket says 172.21.15.89:59534) 2023-10-10T20:43:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.521+0000 7f3de2ffd700 1 -- 172.21.15.89:0/2980372970 learned_addr learned my addr 172.21.15.89:0/2980372970 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.521+0000 7f3de3fff700 1 --2- 172.21.15.89:0/2980372970 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3de41593b0 0x7f3de40c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.521+0000 7f3de37fe700 1 --2- 172.21.15.89:0/2980372970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3de40b1e50 0x7f3de40c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.521+0000 7f3de2ffd700 1 -- 172.21.15.89:0/2980372970 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3de41593b0 msgr2=0x7f3de40c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.521+0000 7f3de2ffd700 1 --2- 172.21.15.89:0/2980372970 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3de41593b0 0x7f3de40c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.521+0000 7f3de2ffd700 1 -- 172.21.15.89:0/2980372970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3de40b1e50 msgr2=0x7f3de40c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.521+0000 7f3de2ffd700 1 --2- 172.21.15.89:0/2980372970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3de40b1e50 0x7f3de40c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.521+0000 7f3de2ffd700 1 -- 172.21.15.89:0/2980372970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc009420 con 0x7f3de40ceaa0 2023-10-10T20:43:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.521+0000 7f3de3fff700 1 --2- 172.21.15.89:0/2980372970 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3de41593b0 0x7f3de40c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.521+0000 7f3de37fe700 1 --2- 172.21.15.89:0/2980372970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3de40b1e50 0x7f3de40c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.521+0000 7f3de2ffd700 1 --2- 172.21.15.89:0/2980372970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3de40ceaa0 0x7f3de4156340 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800c810 tx=0x7f3dd8007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.521+0000 7f3dcb7fe700 1 -- 172.21.15.89:0/2980372970 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8005b30 con 0x7f3de40ceaa0 2023-10-10T20:43:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.521+0000 7f3dcb7fe700 1 -- 172.21.15.89:0/2980372970 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd8005c90 con 0x7f3de40ceaa0 2023-10-10T20:43:26.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.521+0000 7f3dcb7fe700 1 -- 172.21.15.89:0/2980372970 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8007340 con 0x7f3de40ceaa0 2023-10-10T20:43:26.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.521+0000 7f3de88d8700 1 -- 172.21.15.89:0/2980372970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de40cafe0 con 0x7f3de40ceaa0 2023-10-10T20:43:26.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.521+0000 7f3de88d8700 1 -- 172.21.15.89:0/2980372970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de40cb5a0 con 0x7f3de40ceaa0 2023-10-10T20:43:26.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.522+0000 7f3de88d8700 1 -- 172.21.15.89:0/2980372970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de404c3c0 con 0x7f3de40ceaa0 2023-10-10T20:43:26.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.523+0000 7f3dcb7fe700 1 -- 172.21.15.89:0/2980372970 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3dd801b400 con 0x7f3de40ceaa0 2023-10-10T20:43:26.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.523+0000 7f3dcb7fe700 1 --2- 172.21.15.89:0/2980372970 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3dd0067320 0x7f3dd00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:26.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.523+0000 7f3dcb7fe700 1 -- 172.21.15.89:0/2980372970 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3dd80842f0 con 0x7f3de40ceaa0 2023-10-10T20:43:26.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.523+0000 7f3de37fe700 1 --2- 172.21.15.89:0/2980372970 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3dd0067320 0x7f3dd00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:26.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.523+0000 7f3de37fe700 1 --2- 172.21.15.89:0/2980372970 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3dd0067320 0x7f3dd00697d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400d4a0 tx=0x7f3dd40075d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:26.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.527+0000 7f3dcb7fe700 1 -- 172.21.15.89:0/2980372970 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3dd80512d0 con 0x7f3de40ceaa0 2023-10-10T20:43:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.782+0000 7f3de88d8700 1 -- 172.21.15.89:0/2980372970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3de40b6570 con 0x7f3de40ceaa0 2023-10-10T20:43:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.782+0000 7f3dcb7fe700 1 -- 172.21.15.89:0/2980372970 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3de40b6570 con 0x7f3de40ceaa0 2023-10-10T20:43:26.785 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:26.785 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:43:26.785 INFO:teuthology.orchestra.run.smithi089.stdout: "auth_dump": [ 2023-10-10T20:43:26.786 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:26.786 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.a", 2023-10-10T20:43:26.786 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlZLSNLhAA8WZTrsht6tAOP+gxaTXKMg==", 2023-10-10T20:43:26.786 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:26.787 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:43:26.787 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:43:26.787 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:43:26.787 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:26.787 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.787 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:26.788 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:26.788 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.b", 2023-10-10T20:43:26.788 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgyDEORAAMMoSxPFZc0bCIwWy8UsXgA==", 2023-10-10T20:43:26.789 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:26.789 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:43:26.789 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:43:26.789 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:43:26.789 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:26.790 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.790 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:26.790 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:26.790 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.c", 2023-10-10T20:43:26.790 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlMFxSNRAASuQEJFghLIOkTQWa4ckTvg==", 2023-10-10T20:43:26.791 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:26.791 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:43:26.791 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:43:26.791 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:43:26.791 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:26.792 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.792 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:26.792 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:26.792 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.d", 2023-10-10T20:43:26.792 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlPkvrBBAA2VLJmv2t8szTEGtCVwalJw==", 2023-10-10T20:43:26.793 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:26.793 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:43:26.793 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:43:26.793 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:43:26.793 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:26.793 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:26.794 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:26.794 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.0", 2023-10-10T20:43:26.795 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQArtSVltOcKHRAAk8+zUus04oTVI9KvfhjlXg==", 2023-10-10T20:43:26.795 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:26.795 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:43:26.795 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:43:26.795 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:26.795 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.796 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:26.796 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:26.796 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.1", 2023-10-10T20:43:26.796 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAutSVly8P0GhAAe1bvR63+ptJg84lizMNdQw==", 2023-10-10T20:43:26.796 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:26.797 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:43:26.797 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:43:26.797 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:26.797 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.797 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:26.798 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:26.798 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.2", 2023-10-10T20:43:26.798 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAxtSVl0x9fGRAAEKi7+6EW5mzPbebmz0+sgQ==", 2023-10-10T20:43:26.798 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:26.798 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:43:26.799 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:43:26.799 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:26.799 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.799 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:26.800 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:26.800 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.3", 2023-10-10T20:43:26.800 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA0tSVlfJyaGBAAa+8w54U6v7+npbdufKJruA==", 2023-10-10T20:43:26.800 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:26.801 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:43:26.801 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:43:26.801 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:26.801 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.801 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:26.802 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:26.802 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.4", 2023-10-10T20:43:26.802 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA6tSVlEQ/QFxAA/PRB3c90BuAlF6xekGfX9A==", 2023-10-10T20:43:26.802 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:26.803 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:43:26.803 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:43:26.803 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:26.803 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.803 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:26.804 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:26.804 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.5", 2023-10-10T20:43:26.804 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA9tSVlMOKrFRAAuBnIsEz2VmeSZi9XWzjlzA==", 2023-10-10T20:43:26.804 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:26.805 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:43:26.805 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:43:26.805 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:26.806 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.806 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:26.806 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:26.806 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.6", 2023-10-10T20:43:26.807 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBAtSVlH7PQFBAAHr6c7d4TYJEu/KSFEf+EOA==", 2023-10-10T20:43:26.807 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:26.807 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:43:26.807 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:43:26.807 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:26.808 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.808 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:26.808 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:26.808 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.7", 2023-10-10T20:43:26.808 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBDtSVlAhC1FBAAuE2BwAA/f7VOkZfVdqAG7A==", 2023-10-10T20:43:26.809 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:26.809 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:43:26.809 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:43:26.809 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:26.809 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.810 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:26.810 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:26.810 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.0", 2023-10-10T20:43:26.810 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlRd3HCBAAb0QP2VfTS7aelno1mFpGAA==", 2023-10-10T20:43:26.811 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:26.811 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:43:26.811 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow r", 2023-10-10T20:43:26.811 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:43:26.811 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.812 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:26.812 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:26.812 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.1", 2023-10-10T20:43:26.813 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlgBouERAAqPdE0KvJU3M53B6Y6AH5GQ==", 2023-10-10T20:43:26.813 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:26.813 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:43:26.813 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:43:26.814 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:43:26.814 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:43:26.814 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.815 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:26.815 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:26.815 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.2", 2023-10-10T20:43:26.815 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlXA4tDhAAoPVYQjECpzD+C67mnsY5IA==", 2023-10-10T20:43:26.815 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:26.816 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:43:26.816 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:43:26.816 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:43:26.816 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:43:26.816 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.817 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:26.817 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:26.817 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.3", 2023-10-10T20:43:26.817 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlZ5KGFhAAuKmZwbOfmmw1nAj5ry6tHw==", 2023-10-10T20:43:26.817 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:26.818 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:43:26.818 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:43:26.818 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:43:26.818 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:43:26.818 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.819 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:26.819 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:26.819 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.admin", 2023-10-10T20:43:26.819 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgpL2EhAAoDOki002rfBrauTgfu481Q==", 2023-10-10T20:43:26.819 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:26.820 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:43:26.820 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:43:26.820 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:43:26.820 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:26.820 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:26.821 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:26.821 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.x", 2023-10-10T20:43:26.821 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlBhaJKxAArraBxT5xWg515LfpJ7MlbA==", 2023-10-10T20:43:26.821 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:26.821 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:43:26.822 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:43:26.822 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:26.822 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:26.822 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:26.823 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.y", 2023-10-10T20:43:26.823 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlTZIyJxAAWAOsitHwsIxJWwp3b1Z98Q==", 2023-10-10T20:43:26.823 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:26.823 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:43:26.823 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:43:26.823 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:26.824 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.824 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:26.824 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-10T20:43:26.824 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:43:26.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.784+0000 7f3de88d8700 1 -- 172.21.15.89:0/2980372970 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3dd0067320 msgr2=0x7f3dd00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:26.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.784+0000 7f3de88d8700 1 --2- 172.21.15.89:0/2980372970 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3dd0067320 0x7f3dd00697d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400d4a0 tx=0x7f3dd40075d0 comp rx=0 tx=0).stop 2023-10-10T20:43:26.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.784+0000 7f3de88d8700 1 -- 172.21.15.89:0/2980372970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3de40ceaa0 msgr2=0x7f3de4156340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:26.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.784+0000 7f3de88d8700 1 --2- 172.21.15.89:0/2980372970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3de40ceaa0 0x7f3de4156340 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800c810 tx=0x7f3dd8007c50 comp rx=0 tx=0).stop 2023-10-10T20:43:26.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.784+0000 7f3de88d8700 1 -- 172.21.15.89:0/2980372970 shutdown_connections 2023-10-10T20:43:26.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.784+0000 7f3de88d8700 1 --2- 172.21.15.89:0/2980372970 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3de41593b0 0x7f3de40c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:26.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.784+0000 7f3de88d8700 1 --2- 172.21.15.89:0/2980372970 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3dd0067320 0x7f3dd00697d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:26.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.784+0000 7f3de88d8700 1 --2- 172.21.15.89:0/2980372970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3de40b1e50 0x7f3de40c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:26.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.784+0000 7f3de88d8700 1 --2- 172.21.15.89:0/2980372970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3de40ceaa0 0x7f3de4156340 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:26.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.784+0000 7f3de88d8700 1 -- 172.21.15.89:0/2980372970 >> 172.21.15.89:0/2980372970 conn(0x7f3de40d1130 msgr2=0x7f3de40b4760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:26.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.784+0000 7f3de88d8700 1 -- 172.21.15.89:0/2980372970 shutdown_connections 2023-10-10T20:43:26.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.784+0000 7f3de88d8700 1 -- 172.21.15.89:0/2980372970 wait complete. 2023-10-10T20:43:26.829 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:43:26.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.941+0000 7f66922d5700 1 Processor -- start 2023-10-10T20:43:26.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.941+0000 7f66922d5700 1 -- start start 2023-10-10T20:43:26.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.942+0000 7f66922d5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f668c069560 0x7f668c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:26.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.942+0000 7f66922d5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668c069ee0 con 0x7f668c159890 2023-10-10T20:43:26.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.942+0000 7f66922d5700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668c15a160 con 0x7f668c159c60 2023-10-10T20:43:26.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.942+0000 7f66922d5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f668c0d13a0 con 0x7f668c069560 2023-10-10T20:43:26.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.942+0000 7f6690872700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f668c069560 0x7f668c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:26.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.942+0000 7f6690872700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f668c069560 0x7f668c0699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59542/0 (socket says 172.21.15.89:59542) 2023-10-10T20:43:26.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.942+0000 7f6690872700 1 -- 172.21.15.89:0/3235715054 learned_addr learned my addr 172.21.15.89:0/3235715054 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:26.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.942+0000 7f668bfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f668c159890 0x7f668c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49614/0 (socket says 172.21.15.89:49614) 2023-10-10T20:43:26.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.943+0000 7f6690872700 1 -- 172.21.15.89:0/3235715054 >> v1:172.21.15.111:6789/0 conn(0x7f668c159c60 legacy=0x7f668c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:26.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.943+0000 7f6690872700 1 -- 172.21.15.89:0/3235715054 >> v1:172.21.15.89:6789/0 conn(0x7f668c159890 legacy=0x7f668c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:26.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.943+0000 7f668affd700 1 -- 172.21.15.89:0/3235715054 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677483814 0 0) 0x7f668c069ee0 con 0x7f668c159890 2023-10-10T20:43:26.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.943+0000 7f6690872700 1 -- 172.21.15.89:0/3235715054 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c15e140 con 0x7f668c069560 2023-10-10T20:43:26.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.943+0000 7f6690872700 1 --2- 172.21.15.89:0/3235715054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f668c069560 0x7f668c0699a0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f6684009f20 tx=0x7f6684009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c05c23e6c2783f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:26.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.943+0000 7f668affd700 1 -- 172.21.15.89:0/3235715054 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668400e070 con 0x7f668c069560 2023-10-10T20:43:26.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.943+0000 7f668affd700 1 -- 172.21.15.89:0/3235715054 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668400ba80 con 0x7f668c069560 2023-10-10T20:43:26.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.943+0000 7f668affd700 1 -- 172.21.15.89:0/3235715054 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66840092e0 con 0x7f668c069560 2023-10-10T20:43:26.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.943+0000 7f66922d5700 1 -- 172.21.15.89:0/3235715054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f668c069560 msgr2=0x7f668c0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:26.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.943+0000 7f66922d5700 1 --2- 172.21.15.89:0/3235715054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f668c069560 0x7f668c0699a0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f6684009f20 tx=0x7f6684009840 comp rx=0 tx=0).stop 2023-10-10T20:43:26.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.943+0000 7f66922d5700 1 -- 172.21.15.89:0/3235715054 shutdown_connections 2023-10-10T20:43:26.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.943+0000 7f66922d5700 1 --2- 172.21.15.89:0/3235715054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f668c069560 0x7f668c0699a0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:26.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.943+0000 7f66922d5700 1 -- 172.21.15.89:0/3235715054 >> 172.21.15.89:0/3235715054 conn(0x7f668c068e20 msgr2=0x7f668c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:26.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.943+0000 7f66922d5700 1 -- 172.21.15.89:0/3235715054 shutdown_connections 2023-10-10T20:43:26.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.943+0000 7f66922d5700 1 -- 172.21.15.89:0/3235715054 wait complete. 2023-10-10T20:43:26.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.944+0000 7f66922d5700 1 Processor -- start 2023-10-10T20:43:26.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.945+0000 7f66922d5700 1 -- start start 2023-10-10T20:43:26.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.945+0000 7f66922d5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f668c069560 0x7f668c149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:26.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.945+0000 7f66922d5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f668c159890 0x7f668c0c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:26.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.945+0000 7f66922d5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f668c159c60 0x7f668c0cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:26.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.945+0000 7f66922d5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c0cde50 con 0x7f668c069560 2023-10-10T20:43:26.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.945+0000 7f66922d5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c149850 con 0x7f668c159c60 2023-10-10T20:43:26.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.945+0000 7f66922d5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668c0cd240 con 0x7f668c159890 2023-10-10T20:43:26.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.946+0000 7f668bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f668c069560 0x7f668c149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:26.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.946+0000 7f668bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f668c069560 0x7f668c149310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59554/0 (socket says 172.21.15.89:59554) 2023-10-10T20:43:26.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.946+0000 7f668bfff700 1 -- 172.21.15.89:0/3955075346 learned_addr learned my addr 172.21.15.89:0/3955075346 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:26.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.946+0000 7f6690872700 1 --2- 172.21.15.89:0/3955075346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f668c159c60 0x7f668c0cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:26.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.946+0000 7f668b7fe700 1 --2- 172.21.15.89:0/3955075346 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f668c159890 0x7f668c0c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:26.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.946+0000 7f668bfff700 1 -- 172.21.15.89:0/3955075346 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f668c159890 msgr2=0x7f668c0c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:26.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.946+0000 7f668bfff700 1 --2- 172.21.15.89:0/3955075346 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f668c159890 0x7f668c0c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:26.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.946+0000 7f668bfff700 1 -- 172.21.15.89:0/3955075346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f668c159c60 msgr2=0x7f668c0cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:26.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.946+0000 7f668bfff700 1 --2- 172.21.15.89:0/3955075346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f668c159c60 0x7f668c0cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:26.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.946+0000 7f668bfff700 1 -- 172.21.15.89:0/3955075346 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6684009ae0 con 0x7f668c069560 2023-10-10T20:43:26.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.946+0000 7f6690872700 1 --2- 172.21.15.89:0/3955075346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f668c159c60 0x7f668c0cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:26.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.946+0000 7f668bfff700 1 --2- 172.21.15.89:0/3955075346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f668c069560 0x7f668c149310 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f667c00c810 tx=0x7f667c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:26.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.946+0000 7f6673fff700 1 -- 172.21.15.89:0/3955075346 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0073d0 con 0x7f668c069560 2023-10-10T20:43:26.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.946+0000 7f6673fff700 1 -- 172.21.15.89:0/3955075346 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667c007550 con 0x7f668c069560 2023-10-10T20:43:26.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.946+0000 7f66922d5700 1 -- 172.21.15.89:0/3955075346 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668c0cd520 con 0x7f668c069560 2023-10-10T20:43:26.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.946+0000 7f6673fff700 1 -- 172.21.15.89:0/3955075346 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c004a80 con 0x7f668c069560 2023-10-10T20:43:26.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.946+0000 7f66922d5700 1 -- 172.21.15.89:0/3955075346 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668c0cdae0 con 0x7f668c069560 2023-10-10T20:43:26.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.947+0000 7f66922d5700 1 -- 172.21.15.89:0/3955075346 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668c0b2e40 con 0x7f668c069560 2023-10-10T20:43:26.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.947+0000 7f6673fff700 1 -- 172.21.15.89:0/3955075346 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f667c01c3b0 con 0x7f668c069560 2023-10-10T20:43:26.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.948+0000 7f6673fff700 1 --2- 172.21.15.89:0/3955075346 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6678067320 0x7f66780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:26.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.948+0000 7f668b7fe700 1 --2- 172.21.15.89:0/3955075346 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6678067320 0x7f66780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:26.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.948+0000 7f6673fff700 1 -- 172.21.15.89:0/3955075346 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f667c083e50 con 0x7f668c069560 2023-10-10T20:43:26.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.948+0000 7f668b7fe700 1 --2- 172.21.15.89:0/3955075346 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6678067320 0x7f66780697d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f668c0ca3b0 tx=0x7f6680009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:26.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:26.952+0000 7f6673fff700 1 -- 172.21.15.89:0/3955075346 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f667c050e30 con 0x7f668c069560 2023-10-10T20:43:27.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.135+0000 7f66922d5700 1 -- 172.21.15.89:0/3955075346 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f668c04c3c0 con 0x7f668c069560 2023-10-10T20:43:27.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.136+0000 7f6673fff700 1 -- 172.21.15.89:0/3955075346 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v179) v1 ==== 74+0+12144 (secure 0 0 0) 0x7f667c018020 con 0x7f668c069560 2023-10-10T20:43:27.138 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:27.139 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":179,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:43:26.380743+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:43:27.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.138+0000 7f66922d5700 1 -- 172.21.15.89:0/3955075346 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6678067320 msgr2=0x7f66780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.138+0000 7f66922d5700 1 --2- 172.21.15.89:0/3955075346 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6678067320 0x7f66780697d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f668c0ca3b0 tx=0x7f6680009040 comp rx=0 tx=0).stop 2023-10-10T20:43:27.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.138+0000 7f66922d5700 1 -- 172.21.15.89:0/3955075346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f668c069560 msgr2=0x7f668c149310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.138+0000 7f66922d5700 1 --2- 172.21.15.89:0/3955075346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f668c069560 0x7f668c149310 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f667c00c810 tx=0x7f667c007d90 comp rx=0 tx=0).stop 2023-10-10T20:43:27.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.138+0000 7f66922d5700 1 -- 172.21.15.89:0/3955075346 shutdown_connections 2023-10-10T20:43:27.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.138+0000 7f66922d5700 1 --2- 172.21.15.89:0/3955075346 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6678067320 0x7f66780697d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.138+0000 7f66922d5700 1 --2- 172.21.15.89:0/3955075346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f668c159c60 0x7f668c0cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.138+0000 7f66922d5700 1 --2- 172.21.15.89:0/3955075346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f668c069560 0x7f668c149310 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.138+0000 7f66922d5700 1 --2- 172.21.15.89:0/3955075346 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f668c159890 0x7f668c0c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.138+0000 7f66922d5700 1 -- 172.21.15.89:0/3955075346 >> 172.21.15.89:0/3955075346 conn(0x7f668c068e20 msgr2=0x7f668c155340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:27.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.138+0000 7f66922d5700 1 -- 172.21.15.89:0/3955075346 shutdown_connections 2023-10-10T20:43:27.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.138+0000 7f66922d5700 1 -- 172.21.15.89:0/3955075346 wait complete. 2023-10-10T20:43:27.152 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:43:27.153 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:43:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.269+0000 7fb06258e700 1 Processor -- start 2023-10-10T20:43:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.269+0000 7fb06258e700 1 -- start start 2023-10-10T20:43:27.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.270+0000 7fb06258e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb05c159c60 0x7fb05c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:27.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.270+0000 7fb06258e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb05c15db10 0x7fb05c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:27.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.270+0000 7fb06258e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb05c15a160 con 0x7fb05c159890 2023-10-10T20:43:27.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.270+0000 7fb06258e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb05c15df50 con 0x7fb05c159c60 2023-10-10T20:43:27.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.270+0000 7fb06258e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb05c0d13a0 con 0x7fb05c15db10 2023-10-10T20:43:27.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.270+0000 7fb05b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb05c159c60 0x7fb05c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:27.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.270+0000 7fb05b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb05c159c60 0x7fb05c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59556/0 (socket says 172.21.15.89:59556) 2023-10-10T20:43:27.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.270+0000 7fb05b7fe700 1 -- 172.21.15.89:0/3971886706 learned_addr learned my addr 172.21.15.89:0/3971886706 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:27.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.270+0000 7fb05bfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb05c159890 0x7fb05c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49630/0 (socket says 172.21.15.89:49630) 2023-10-10T20:43:27.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.270+0000 7fb060b2b700 1 --2- 172.21.15.89:0/3971886706 >> v2:172.21.15.111:3300/0 conn(0x7fb05c15db10 0x7fb05c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:27.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.270+0000 7fb05b7fe700 1 -- 172.21.15.89:0/3971886706 >> v2:172.21.15.111:3300/0 conn(0x7fb05c15db10 msgr2=0x7fb05c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:27.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.270+0000 7fb05b7fe700 1 --2- 172.21.15.89:0/3971886706 >> v2:172.21.15.111:3300/0 conn(0x7fb05c15db10 0x7fb05c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.270+0000 7fb05b7fe700 1 -- 172.21.15.89:0/3971886706 >> v1:172.21.15.89:6789/0 conn(0x7fb05c159890 legacy=0x7fb05c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.270+0000 7fb05affd700 1 -- 172.21.15.89:0/3971886706 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1849312592 0 0) 0x7fb05c15a160 con 0x7fb05c159890 2023-10-10T20:43:27.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.270+0000 7fb05b7fe700 1 -- 172.21.15.89:0/3971886706 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05c15e140 con 0x7fb05c159c60 2023-10-10T20:43:27.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.271+0000 7fb05b7fe700 1 --2- 172.21.15.89:0/3971886706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb05c159c60 0x7fb05c15d5d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fb05c06a890 tx=0x7fb050009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd657c9a4e22e09 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:27.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.271+0000 7fb05affd700 1 -- 172.21.15.89:0/3971886706 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05000b860 con 0x7fb05c159c60 2023-10-10T20:43:27.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.271+0000 7fb05affd700 1 -- 172.21.15.89:0/3971886706 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05000b9c0 con 0x7fb05c159c60 2023-10-10T20:43:27.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.271+0000 7fb05affd700 1 -- 172.21.15.89:0/3971886706 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05000bd60 con 0x7fb05c159c60 2023-10-10T20:43:27.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.271+0000 7fb06258e700 1 -- 172.21.15.89:0/3971886706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb05c159c60 msgr2=0x7fb05c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.271+0000 7fb06258e700 1 --2- 172.21.15.89:0/3971886706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb05c159c60 0x7fb05c15d5d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fb05c06a890 tx=0x7fb050009250 comp rx=0 tx=0).stop 2023-10-10T20:43:27.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.271+0000 7fb06258e700 1 -- 172.21.15.89:0/3971886706 shutdown_connections 2023-10-10T20:43:27.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.271+0000 7fb06258e700 1 --2- 172.21.15.89:0/3971886706 >> v2:172.21.15.111:3300/0 conn(0x7fb05c15db10 0x7fb05c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.271+0000 7fb06258e700 1 --2- 172.21.15.89:0/3971886706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb05c159c60 0x7fb05c15d5d0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.271+0000 7fb06258e700 1 -- 172.21.15.89:0/3971886706 >> 172.21.15.89:0/3971886706 conn(0x7fb05c068e20 msgr2=0x7fb05c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:27.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.271+0000 7fb06258e700 1 -- 172.21.15.89:0/3971886706 shutdown_connections 2023-10-10T20:43:27.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.271+0000 7fb06258e700 1 -- 172.21.15.89:0/3971886706 wait complete. 2023-10-10T20:43:27.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.272+0000 7fb06258e700 1 Processor -- start 2023-10-10T20:43:27.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.273+0000 7fb06258e700 1 -- start start 2023-10-10T20:43:27.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.273+0000 7fb06258e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb05c159890 0x7fb05c0b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:27.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.273+0000 7fb06258e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb05c159c60 0x7fb05c14c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:27.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.273+0000 7fb06258e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb05c15db10 0x7fb05c158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:27.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.273+0000 7fb06258e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c152ac0 con 0x7fb05c159890 2023-10-10T20:43:27.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.273+0000 7fb06258e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c14cc40 con 0x7fb05c159c60 2023-10-10T20:43:27.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.273+0000 7fb06258e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c0b2380 con 0x7fb05c15db10 2023-10-10T20:43:27.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.273+0000 7fb05bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb05c159890 0x7fb05c0b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:27.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.273+0000 7fb05bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb05c159890 0x7fb05c0b1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59564/0 (socket says 172.21.15.89:59564) 2023-10-10T20:43:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.273+0000 7fb05bfff700 1 -- 172.21.15.89:0/1975818013 learned_addr learned my addr 172.21.15.89:0/1975818013 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.274+0000 7fb05b7fe700 1 --2- 172.21.15.89:0/1975818013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb05c159c60 0x7fb05c14c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.274+0000 7fb060b2b700 1 --2- 172.21.15.89:0/1975818013 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb05c15db10 0x7fb05c158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.274+0000 7fb05bfff700 1 -- 172.21.15.89:0/1975818013 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb05c15db10 msgr2=0x7fb05c158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.274+0000 7fb05bfff700 1 --2- 172.21.15.89:0/1975818013 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb05c15db10 0x7fb05c158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.274+0000 7fb05bfff700 1 -- 172.21.15.89:0/1975818013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb05c159c60 msgr2=0x7fb05c14c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.274+0000 7fb05bfff700 1 --2- 172.21.15.89:0/1975818013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb05c159c60 0x7fb05c14c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.274+0000 7fb05bfff700 1 -- 172.21.15.89:0/1975818013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0500094f0 con 0x7fb05c159890 2023-10-10T20:43:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.274+0000 7fb05bfff700 1 --2- 172.21.15.89:0/1975818013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb05c159890 0x7fb05c0b1d10 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0011e0 tx=0x7fb04c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.274+0000 7fb043fff700 1 -- 172.21.15.89:0/1975818013 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c007300 con 0x7fb05c159890 2023-10-10T20:43:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.274+0000 7fb043fff700 1 -- 172.21.15.89:0/1975818013 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04c007460 con 0x7fb05c159890 2023-10-10T20:43:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.274+0000 7fb06258e700 1 -- 172.21.15.89:0/1975818013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05c152470 con 0x7fb05c159890 2023-10-10T20:43:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.274+0000 7fb043fff700 1 -- 172.21.15.89:0/1975818013 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c004830 con 0x7fb05c159890 2023-10-10T20:43:27.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.274+0000 7fb06258e700 1 -- 172.21.15.89:0/1975818013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05c0b2660 con 0x7fb05c159890 2023-10-10T20:43:27.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.275+0000 7fb06258e700 1 -- 172.21.15.89:0/1975818013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb05c04c3c0 con 0x7fb05c159890 2023-10-10T20:43:27.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.275+0000 7fb043fff700 1 -- 172.21.15.89:0/1975818013 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb04c004be0 con 0x7fb05c159890 2023-10-10T20:43:27.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.276+0000 7fb043fff700 1 --2- 172.21.15.89:0/1975818013 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb048067320 0x7fb0480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:27.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.276+0000 7fb05b7fe700 1 --2- 172.21.15.89:0/1975818013 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb048067320 0x7fb0480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:27.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.276+0000 7fb043fff700 1 -- 172.21.15.89:0/1975818013 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb04c083e60 con 0x7fb05c159890 2023-10-10T20:43:27.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.276+0000 7fb05b7fe700 1 --2- 172.21.15.89:0/1975818013 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb048067320 0x7fb0480697d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb05c06a890 tx=0x7fb050017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:27.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.280+0000 7fb043fff700 1 -- 172.21.15.89:0/1975818013 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb04c050ec0 con 0x7fb05c159890 2023-10-10T20:43:27.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.464+0000 7fb06258e700 1 -- 172.21.15.89:0/1975818013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb05c06a5b0 con 0x7fb05c159890 2023-10-10T20:43:27.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.465+0000 7fb043fff700 1 -- 172.21.15.89:0/1975818013 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v179) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb04c018020 con 0x7fb05c159890 2023-10-10T20:43:27.467 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:43:27.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.467+0000 7fb06258e700 1 -- 172.21.15.89:0/1975818013 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb048067320 msgr2=0x7fb0480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.467+0000 7fb06258e700 1 --2- 172.21.15.89:0/1975818013 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb048067320 0x7fb0480697d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb05c06a890 tx=0x7fb050017000 comp rx=0 tx=0).stop 2023-10-10T20:43:27.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.467+0000 7fb06258e700 1 -- 172.21.15.89:0/1975818013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb05c159890 msgr2=0x7fb05c0b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.467+0000 7fb06258e700 1 --2- 172.21.15.89:0/1975818013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb05c159890 0x7fb05c0b1d10 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0011e0 tx=0x7fb04c007e10 comp rx=0 tx=0).stop 2023-10-10T20:43:27.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.467+0000 7fb06258e700 1 -- 172.21.15.89:0/1975818013 shutdown_connections 2023-10-10T20:43:27.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.467+0000 7fb06258e700 1 --2- 172.21.15.89:0/1975818013 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb05c15db10 0x7fb05c158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.467+0000 7fb06258e700 1 --2- 172.21.15.89:0/1975818013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb05c159890 0x7fb05c0b1d10 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.467+0000 7fb06258e700 1 --2- 172.21.15.89:0/1975818013 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb048067320 0x7fb0480697d0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.467+0000 7fb06258e700 1 --2- 172.21.15.89:0/1975818013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb05c159c60 0x7fb05c14c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.467+0000 7fb06258e700 1 -- 172.21.15.89:0/1975818013 >> 172.21.15.89:0/1975818013 conn(0x7fb05c068e20 msgr2=0x7fb05c14d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:27.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.467+0000 7fb06258e700 1 -- 172.21.15.89:0/1975818013 shutdown_connections 2023-10-10T20:43:27.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.467+0000 7fb06258e700 1 -- 172.21.15.89:0/1975818013 wait complete. 2023-10-10T20:43:27.481 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:43:27.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.593+0000 7f5bb23e4700 1 Processor -- start 2023-10-10T20:43:27.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.593+0000 7f5bb23e4700 1 -- start start 2023-10-10T20:43:27.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.594+0000 7f5bb23e4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5bac069560 0x7f5bac0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:27.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.594+0000 7f5bb23e4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bac069ee0 con 0x7f5bac159890 2023-10-10T20:43:27.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.594+0000 7f5bb23e4700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bac15a160 con 0x7f5bac159c60 2023-10-10T20:43:27.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.594+0000 7f5bb23e4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bac0d13a0 con 0x7f5bac069560 2023-10-10T20:43:27.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.594+0000 7f5bb0981700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5bac069560 0x7f5bac0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:27.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.594+0000 7f5babfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5bac159890 0x7f5bac15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49638/0 (socket says 172.21.15.89:49638) 2023-10-10T20:43:27.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.594+0000 7f5babfff700 1 -- 172.21.15.89:0/4179845174 learned_addr learned my addr 172.21.15.89:0/4179845174 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:27.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.594+0000 7f5bb0981700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5bac069560 0x7f5bac0699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59568/0 (socket says 172.21.15.89:59568) 2023-10-10T20:43:27.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.594+0000 7f5bb0981700 1 -- 172.21.15.89:0/4179845174 >> v1:172.21.15.111:6789/0 conn(0x7f5bac159c60 legacy=0x7f5bac15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.594+0000 7f5bb0981700 1 -- 172.21.15.89:0/4179845174 >> v1:172.21.15.89:6789/0 conn(0x7f5bac159890 legacy=0x7f5bac15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.594+0000 7f5bb0981700 1 -- 172.21.15.89:0/4179845174 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac15e140 con 0x7f5bac069560 2023-10-10T20:43:27.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.594+0000 7f5baaffd700 1 -- 172.21.15.89:0/4179845174 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4072791818 0 0) 0x7f5bac069ee0 con 0x7f5bac159890 2023-10-10T20:43:27.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.594+0000 7f5bb0981700 1 --2- 172.21.15.89:0/4179845174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac069560 0x7f5bac0699a0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009e60 tx=0x7f5ba4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09f9766f89edf42 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:27.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.594+0000 7f5baaffd700 1 -- 172.21.15.89:0/4179845174 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba400e070 con 0x7f5bac069560 2023-10-10T20:43:27.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.594+0000 7f5baaffd700 1 -- 172.21.15.89:0/4179845174 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba400bc30 con 0x7f5bac069560 2023-10-10T20:43:27.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.594+0000 7f5baaffd700 1 -- 172.21.15.89:0/4179845174 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4012430 con 0x7f5bac069560 2023-10-10T20:43:27.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.595+0000 7f5bb23e4700 1 -- 172.21.15.89:0/4179845174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac069560 msgr2=0x7f5bac0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.595+0000 7f5bb23e4700 1 --2- 172.21.15.89:0/4179845174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac069560 0x7f5bac0699a0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009e60 tx=0x7f5ba4009780 comp rx=0 tx=0).stop 2023-10-10T20:43:27.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.595+0000 7f5bb23e4700 1 -- 172.21.15.89:0/4179845174 shutdown_connections 2023-10-10T20:43:27.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.595+0000 7f5bb23e4700 1 --2- 172.21.15.89:0/4179845174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac069560 0x7f5bac0699a0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.595+0000 7f5bb23e4700 1 -- 172.21.15.89:0/4179845174 >> 172.21.15.89:0/4179845174 conn(0x7f5bac068e20 msgr2=0x7f5bac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:27.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.595+0000 7f5bb23e4700 1 -- 172.21.15.89:0/4179845174 shutdown_connections 2023-10-10T20:43:27.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.595+0000 7f5bb23e4700 1 -- 172.21.15.89:0/4179845174 wait complete. 2023-10-10T20:43:27.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.596+0000 7f5bb23e4700 1 Processor -- start 2023-10-10T20:43:27.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.596+0000 7f5bb23e4700 1 -- start start 2023-10-10T20:43:27.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.597+0000 7f5bb23e4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bac069560 0x7f5bac0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:27.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.597+0000 7f5bb23e4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac159890 0x7f5bac0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:27.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.597+0000 7f5bb23e4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bac159c60 0x7f5bac157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:27.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.597+0000 7f5bb23e4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac0b5480 con 0x7f5bac159890 2023-10-10T20:43:27.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.597+0000 7f5bb23e4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac0b3ec0 con 0x7f5bac159c60 2023-10-10T20:43:27.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.597+0000 7f5bb23e4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac0b41a0 con 0x7f5bac069560 2023-10-10T20:43:27.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.597+0000 7f5bab7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac159890 0x7f5bac0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:27.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.597+0000 7f5bab7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac159890 0x7f5bac0b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59582/0 (socket says 172.21.15.89:59582) 2023-10-10T20:43:27.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.597+0000 7f5bab7fe700 1 -- 172.21.15.89:0/1933658424 learned_addr learned my addr 172.21.15.89:0/1933658424 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:27.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.597+0000 7f5babfff700 1 --2- 172.21.15.89:0/1933658424 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bac069560 0x7f5bac0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:27.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.597+0000 7f5bb0981700 1 --2- 172.21.15.89:0/1933658424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bac159c60 0x7f5bac157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:27.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.597+0000 7f5bab7fe700 1 -- 172.21.15.89:0/1933658424 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bac069560 msgr2=0x7f5bac0b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.597+0000 7f5bab7fe700 1 --2- 172.21.15.89:0/1933658424 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bac069560 0x7f5bac0b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.597+0000 7f5bab7fe700 1 -- 172.21.15.89:0/1933658424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bac159c60 msgr2=0x7f5bac157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.597+0000 7f5bab7fe700 1 --2- 172.21.15.89:0/1933658424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bac159c60 0x7f5bac157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.597+0000 7f5bab7fe700 1 -- 172.21.15.89:0/1933658424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba4009a20 con 0x7f5bac159890 2023-10-10T20:43:27.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.597+0000 7f5bab7fe700 1 --2- 172.21.15.89:0/1933658424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac159890 0x7f5bac0b3590 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000c4e0 tx=0x7f5ba000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:27.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.598+0000 7f5b93fff700 1 -- 172.21.15.89:0/1933658424 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0011590 con 0x7f5bac159890 2023-10-10T20:43:27.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.598+0000 7f5b93fff700 1 -- 172.21.15.89:0/1933658424 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba0011710 con 0x7f5bac159890 2023-10-10T20:43:27.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.598+0000 7f5bb23e4700 1 -- 172.21.15.89:0/1933658424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bac0b4480 con 0x7f5bac159890 2023-10-10T20:43:27.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.598+0000 7f5b93fff700 1 -- 172.21.15.89:0/1933658424 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0011ad0 con 0x7f5bac159890 2023-10-10T20:43:27.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.598+0000 7f5bb23e4700 1 -- 172.21.15.89:0/1933658424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bac0b4a40 con 0x7f5bac159890 2023-10-10T20:43:27.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.599+0000 7f5bb23e4700 1 -- 172.21.15.89:0/1933658424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bac04c3c0 con 0x7f5bac159890 2023-10-10T20:43:27.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.599+0000 7f5b93fff700 1 -- 172.21.15.89:0/1933658424 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5ba001b3b0 con 0x7f5bac159890 2023-10-10T20:43:27.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.599+0000 7f5b93fff700 1 --2- 172.21.15.89:0/1933658424 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b98067320 0x7f5b980697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:27.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.600+0000 7f5b93fff700 1 -- 172.21.15.89:0/1933658424 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5ba0083cb0 con 0x7f5bac159890 2023-10-10T20:43:27.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.600+0000 7f5babfff700 1 --2- 172.21.15.89:0/1933658424 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b98067320 0x7f5b980697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:27.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.600+0000 7f5babfff700 1 --2- 172.21.15.89:0/1933658424 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b98067320 0x7f5b980697d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c007fd0 tx=0x7f5b9c007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:27.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.604+0000 7f5b93fff700 1 -- 172.21.15.89:0/1933658424 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ba0050c90 con 0x7f5bac159890 2023-10-10T20:43:27.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.787+0000 7f5bb23e4700 1 -- 172.21.15.89:0/1933658424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5bac157010 con 0x7f5bac159890 2023-10-10T20:43:27.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.787+0000 7f5b93fff700 1 -- 172.21.15.89:0/1933658424 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v179) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5ba0017070 con 0x7f5bac159890 2023-10-10T20:43:27.790 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:27.790 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:43:27.790 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:27.790 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2023-10-10T20:43:27.790 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2023-10-10T20:43:27.791 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:27.791 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:43:27.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.789+0000 7f5bb23e4700 1 -- 172.21.15.89:0/1933658424 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b98067320 msgr2=0x7f5b980697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.789+0000 7f5bb23e4700 1 --2- 172.21.15.89:0/1933658424 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b98067320 0x7f5b980697d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c007fd0 tx=0x7f5b9c007d30 comp rx=0 tx=0).stop 2023-10-10T20:43:27.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.789+0000 7f5bb23e4700 1 -- 172.21.15.89:0/1933658424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac159890 msgr2=0x7f5bac0b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.789+0000 7f5bb23e4700 1 --2- 172.21.15.89:0/1933658424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac159890 0x7f5bac0b3590 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000c4e0 tx=0x7f5ba000c7f0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.789+0000 7f5bb23e4700 1 -- 172.21.15.89:0/1933658424 shutdown_connections 2023-10-10T20:43:27.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.789+0000 7f5bb23e4700 1 --2- 172.21.15.89:0/1933658424 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5b98067320 0x7f5b980697d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.789+0000 7f5bb23e4700 1 --2- 172.21.15.89:0/1933658424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bac159c60 0x7f5bac157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.789+0000 7f5bb23e4700 1 --2- 172.21.15.89:0/1933658424 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5bac069560 0x7f5bac0b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.789+0000 7f5bb23e4700 1 --2- 172.21.15.89:0/1933658424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bac159890 0x7f5bac0b3590 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.789+0000 7f5bb23e4700 1 -- 172.21.15.89:0/1933658424 >> 172.21.15.89:0/1933658424 conn(0x7f5bac068e20 msgr2=0x7f5bac15aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:27.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.789+0000 7f5bb23e4700 1 -- 172.21.15.89:0/1933658424 shutdown_connections 2023-10-10T20:43:27.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.789+0000 7f5bb23e4700 1 -- 172.21.15.89:0/1933658424 wait complete. 2023-10-10T20:43:27.803 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:43:27.804 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:43:27.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.920+0000 7fac3cc23700 1 Processor -- start 2023-10-10T20:43:27.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.920+0000 7fac3cc23700 1 -- start start 2023-10-10T20:43:27.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.921+0000 7fac3cc23700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fac380cd910 0x7fac380ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:27.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.921+0000 7fac3cc23700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac380b5b00 con 0x7fac380b55a0 2023-10-10T20:43:27.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.921+0000 7fac3cc23700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac380b5da0 con 0x7fac38159ec0 2023-10-10T20:43:27.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.921+0000 7fac3cc23700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac380b6080 con 0x7fac380cd910 2023-10-10T20:43:27.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.921+0000 7fac36d9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fac380b55a0 0x7fac380ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49642/0 (socket says 172.21.15.89:49642) 2023-10-10T20:43:27.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.921+0000 7fac36d9d700 1 -- 172.21.15.89:0/3339861584 learned_addr learned my addr 172.21.15.89:0/3339861584 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:27.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.922+0000 7fac35d9b700 1 --2- 172.21.15.89:0/3339861584 >> v2:172.21.15.111:3300/0 conn(0x7fac380cd910 0x7fac380ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:27.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.922+0000 7fac3559a700 1 -- 172.21.15.89:0/3339861584 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397423065 0 0) 0x7fac380b5b00 con 0x7fac380b55a0 2023-10-10T20:43:27.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.922+0000 7fac3559a700 1 -- 172.21.15.89:0/3339861584 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac20003570 con 0x7fac380b55a0 2023-10-10T20:43:27.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.922+0000 7fac3559a700 1 -- 172.21.15.89:0/3339861584 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780901996 0 0) 0x7fac380b5da0 con 0x7fac38159ec0 2023-10-10T20:43:27.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.922+0000 7fac3559a700 1 -- 172.21.15.89:0/3339861584 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac380b5b00 con 0x7fac38159ec0 2023-10-10T20:43:27.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.922+0000 7fac3559a700 1 -- 172.21.15.89:0/3339861584 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1138774257 0 0) 0x7fac20003570 con 0x7fac380b55a0 2023-10-10T20:43:27.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.922+0000 7fac35d9b700 1 -- 172.21.15.89:0/3339861584 >> v1:172.21.15.111:6789/0 conn(0x7fac38159ec0 legacy=0x7fac380b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.922+0000 7fac35d9b700 1 -- 172.21.15.89:0/3339861584 >> v1:172.21.15.89:6789/0 conn(0x7fac380b55a0 legacy=0x7fac380ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.922+0000 7fac35d9b700 1 -- 172.21.15.89:0/3339861584 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac380cc100 con 0x7fac380cd910 2023-10-10T20:43:27.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.922+0000 7fac35d9b700 1 --2- 172.21.15.89:0/3339861584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac380cd910 0x7fac380ce340 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fac2c009910 tx=0x7fac2c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86966739de4e98b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:27.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.922+0000 7fac3559a700 1 -- 172.21.15.89:0/3339861584 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30001470 con 0x7fac380cd910 2023-10-10T20:43:27.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.922+0000 7fac3559a700 1 -- 172.21.15.89:0/3339861584 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2c011420 con 0x7fac380cd910 2023-10-10T20:43:27.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.922+0000 7fac3559a700 1 -- 172.21.15.89:0/3339861584 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c0117c0 con 0x7fac380cd910 2023-10-10T20:43:27.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.923+0000 7fac3cc23700 1 -- 172.21.15.89:0/3339861584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac380cd910 msgr2=0x7fac380ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.923+0000 7fac3cc23700 1 --2- 172.21.15.89:0/3339861584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac380cd910 0x7fac380ce340 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fac2c009910 tx=0x7fac2c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.923+0000 7fac3cc23700 1 -- 172.21.15.89:0/3339861584 shutdown_connections 2023-10-10T20:43:27.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.923+0000 7fac3cc23700 1 --2- 172.21.15.89:0/3339861584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac380cd910 0x7fac380ce340 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.923+0000 7fac3cc23700 1 -- 172.21.15.89:0/3339861584 >> 172.21.15.89:0/3339861584 conn(0x7fac380999f0 msgr2=0x7fac380b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:27.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.923+0000 7fac3cc23700 1 -- 172.21.15.89:0/3339861584 shutdown_connections 2023-10-10T20:43:27.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.923+0000 7fac3cc23700 1 -- 172.21.15.89:0/3339861584 wait complete. 2023-10-10T20:43:27.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.924+0000 7fac3cc23700 1 Processor -- start 2023-10-10T20:43:27.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.924+0000 7fac3cc23700 1 -- start start 2023-10-10T20:43:27.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.925+0000 7fac3cc23700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac380b55a0 0x7fac3814fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:27.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.925+0000 7fac3cc23700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac38159ec0 0x7fac380c5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:27.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.925+0000 7fac3cc23700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac380c5b70 0x7fac38154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:27.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.925+0000 7fac3cc23700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac380c5ff0 con 0x7fac380c5b70 2023-10-10T20:43:27.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.925+0000 7fac3cc23700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac380c78b0 con 0x7fac380b55a0 2023-10-10T20:43:27.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.925+0000 7fac3cc23700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac380c8430 con 0x7fac38159ec0 2023-10-10T20:43:27.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.925+0000 7fac36d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac380c5b70 0x7fac38154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:27.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.925+0000 7fac36d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac380c5b70 0x7fac38154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59592/0 (socket says 172.21.15.89:59592) 2023-10-10T20:43:27.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.925+0000 7fac36d9d700 1 -- 172.21.15.89:0/424922956 learned_addr learned my addr 172.21.15.89:0/424922956 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:27.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.925+0000 7fac35d9b700 1 --2- 172.21.15.89:0/424922956 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac38159ec0 0x7fac380c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:27.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.925+0000 7fac3659c700 1 --2- 172.21.15.89:0/424922956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac380b55a0 0x7fac3814fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:27.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.925+0000 7fac36d9d700 1 -- 172.21.15.89:0/424922956 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac38159ec0 msgr2=0x7fac380c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.925+0000 7fac36d9d700 1 --2- 172.21.15.89:0/424922956 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac38159ec0 0x7fac380c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.925+0000 7fac36d9d700 1 -- 172.21.15.89:0/424922956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac380b55a0 msgr2=0x7fac3814fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:27.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.925+0000 7fac36d9d700 1 --2- 172.21.15.89:0/424922956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac380b55a0 0x7fac3814fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:27.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.925+0000 7fac36d9d700 1 -- 172.21.15.89:0/424922956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2c009420 con 0x7fac380c5b70 2023-10-10T20:43:27.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.925+0000 7fac36d9d700 1 --2- 172.21.15.89:0/424922956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac380c5b70 0x7fac38154890 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fac3000cba0 tx=0x7fac30001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:27.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.926+0000 7fac267fc700 1 -- 172.21.15.89:0/424922956 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30007150 con 0x7fac380c5b70 2023-10-10T20:43:27.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.926+0000 7fac267fc700 1 -- 172.21.15.89:0/424922956 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac300072d0 con 0x7fac380c5b70 2023-10-10T20:43:27.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.926+0000 7fac3cc23700 1 -- 172.21.15.89:0/424922956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac380c7b00 con 0x7fac380c5b70 2023-10-10T20:43:27.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.926+0000 7fac267fc700 1 -- 172.21.15.89:0/424922956 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30007690 con 0x7fac380c5b70 2023-10-10T20:43:27.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.926+0000 7fac3cc23700 1 -- 172.21.15.89:0/424922956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac380c80c0 con 0x7fac380c5b70 2023-10-10T20:43:27.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.927+0000 7fac3cc23700 1 -- 172.21.15.89:0/424922956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac38063c90 con 0x7fac380c5b70 2023-10-10T20:43:27.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.927+0000 7fac267fc700 1 -- 172.21.15.89:0/424922956 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fac30004a40 con 0x7fac380c5b70 2023-10-10T20:43:27.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.927+0000 7fac267fc700 1 --2- 172.21.15.89:0/424922956 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fac18067320 0x7fac180697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:27.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.928+0000 7fac267fc700 1 -- 172.21.15.89:0/424922956 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fac30083e80 con 0x7fac380c5b70 2023-10-10T20:43:27.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.928+0000 7fac3659c700 1 --2- 172.21.15.89:0/424922956 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fac18067320 0x7fac180697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:27.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.928+0000 7fac3659c700 1 --2- 172.21.15.89:0/424922956 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fac18067320 0x7fac180697d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fac380b2990 tx=0x7fac28002b70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:27.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:27.932+0000 7fac267fc700 1 -- 172.21.15.89:0/424922956 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac30050ee0 con 0x7fac380c5b70 2023-10-10T20:43:28.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.115+0000 7fac3cc23700 1 -- 172.21.15.89:0/424922956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fac380718b0 con 0x7fac380c5b70 2023-10-10T20:43:28.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.398+0000 7fac267fc700 1 -- 172.21.15.89:0/424922956 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v180) v1 ==== 138+0+0 (secure 0 0 0) 0x7fac30054500 con 0x7fac380c5b70 2023-10-10T20:43:28.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.401+0000 7fac3cc23700 1 -- 172.21.15.89:0/424922956 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fac18067320 msgr2=0x7fac180697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:28.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.401+0000 7fac3cc23700 1 --2- 172.21.15.89:0/424922956 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fac18067320 0x7fac180697d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fac380b2990 tx=0x7fac28002b70 comp rx=0 tx=0).stop 2023-10-10T20:43:28.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.401+0000 7fac3cc23700 1 -- 172.21.15.89:0/424922956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac380c5b70 msgr2=0x7fac38154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:28.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.401+0000 7fac3cc23700 1 --2- 172.21.15.89:0/424922956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac380c5b70 0x7fac38154890 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fac3000cba0 tx=0x7fac30001a60 comp rx=0 tx=0).stop 2023-10-10T20:43:28.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.401+0000 7fac3cc23700 1 -- 172.21.15.89:0/424922956 shutdown_connections 2023-10-10T20:43:28.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.401+0000 7fac3cc23700 1 --2- 172.21.15.89:0/424922956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac380c5b70 0x7fac38154890 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:28.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.401+0000 7fac3cc23700 1 --2- 172.21.15.89:0/424922956 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fac18067320 0x7fac180697d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:28.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.401+0000 7fac3cc23700 1 --2- 172.21.15.89:0/424922956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac380b55a0 0x7fac3814fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:28.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.401+0000 7fac3cc23700 1 --2- 172.21.15.89:0/424922956 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fac38159ec0 0x7fac380c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:28.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.401+0000 7fac3cc23700 1 -- 172.21.15.89:0/424922956 >> 172.21.15.89:0/424922956 conn(0x7fac380999f0 msgr2=0x7fac380c6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:28.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.402+0000 7fac3cc23700 1 -- 172.21.15.89:0/424922956 shutdown_connections 2023-10-10T20:43:28.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.402+0000 7fac3cc23700 1 -- 172.21.15.89:0/424922956 wait complete. 2023-10-10T20:43:28.406 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2023-10-10T20:43:28.418 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:43:28.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.537+0000 7f2b4f59e700 1 Processor -- start 2023-10-10T20:43:28.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.537+0000 7f2b4f59e700 1 -- start start 2023-10-10T20:43:28.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.537+0000 7f2b4f59e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2b500b0360 0x7f2b500c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:28.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.537+0000 7f2b4f59e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2b500cd180 0x7f2b500caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:28.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.537+0000 7f2b4f59e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b500adf00 con 0x7f2b500cd5f0 2023-10-10T20:43:28.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.537+0000 7f2b4f59e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b500cda80 con 0x7f2b500cd180 2023-10-10T20:43:28.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.537+0000 7f2b4f59e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b500cdd60 con 0x7f2b500b0360 2023-10-10T20:43:28.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.538+0000 7f2b45d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2b500cd180 0x7f2b500caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:28.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.538+0000 7f2b4ed9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2b500cd5f0 0x7f2b5006d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35180/0 (socket says 172.21.15.89:35180) 2023-10-10T20:43:28.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.538+0000 7f2b4ed9d700 1 -- 172.21.15.89:0/407887199 learned_addr learned my addr 172.21.15.89:0/407887199 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:28.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.538+0000 7f2b4e59c700 1 --2- 172.21.15.89:0/407887199 >> v2:172.21.15.111:3300/0 conn(0x7f2b500b0360 0x7f2b500c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:28.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.538+0000 7f2b45d9b700 1 -- 172.21.15.89:0/407887199 >> v2:172.21.15.111:3300/0 conn(0x7f2b500b0360 msgr2=0x7f2b500c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:28.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.538+0000 7f2b45d9b700 1 --2- 172.21.15.89:0/407887199 >> v2:172.21.15.111:3300/0 conn(0x7f2b500b0360 0x7f2b500c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:28.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.538+0000 7f2b45d9b700 1 -- 172.21.15.89:0/407887199 >> v1:172.21.15.89:6789/0 conn(0x7f2b500cd5f0 legacy=0x7f2b5006d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:28.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.538+0000 7f2b45d9b700 1 -- 172.21.15.89:0/407887199 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b500ce040 con 0x7f2b500cd180 2023-10-10T20:43:28.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.538+0000 7f2b4dd9b700 1 -- 172.21.15.89:0/407887199 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2200903905 0 0) 0x7f2b500adf00 con 0x7f2b500cd5f0 2023-10-10T20:43:28.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.538+0000 7f2b45d9b700 1 --2- 172.21.15.89:0/407887199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b500cd180 0x7f2b500caef0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f2b5006cd80 tx=0x7f2b380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4aeb14ebf3dc66d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:28.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.538+0000 7f2b4dd9b700 1 -- 172.21.15.89:0/407887199 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3800b6b0 con 0x7f2b500cd180 2023-10-10T20:43:28.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.538+0000 7f2b4dd9b700 1 -- 172.21.15.89:0/407887199 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3800b810 con 0x7f2b500cd180 2023-10-10T20:43:28.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.538+0000 7f2b4dd9b700 1 -- 172.21.15.89:0/407887199 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3800bbb0 con 0x7f2b500cd180 2023-10-10T20:43:28.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.539+0000 7f2b4f59e700 1 -- 172.21.15.89:0/407887199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b500cd180 msgr2=0x7f2b500caef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:28.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.539+0000 7f2b4f59e700 1 --2- 172.21.15.89:0/407887199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b500cd180 0x7f2b500caef0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f2b5006cd80 tx=0x7f2b380091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:28.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.539+0000 7f2b4f59e700 1 -- 172.21.15.89:0/407887199 shutdown_connections 2023-10-10T20:43:28.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.539+0000 7f2b4f59e700 1 --2- 172.21.15.89:0/407887199 >> v2:172.21.15.111:3300/0 conn(0x7f2b500b0360 0x7f2b500c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:28.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.539+0000 7f2b4f59e700 1 --2- 172.21.15.89:0/407887199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b500cd180 0x7f2b500caef0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:28.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.539+0000 7f2b4f59e700 1 -- 172.21.15.89:0/407887199 >> 172.21.15.89:0/407887199 conn(0x7f2b5006baa0 msgr2=0x7f2b5006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:28.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.539+0000 7f2b4f59e700 1 -- 172.21.15.89:0/407887199 shutdown_connections 2023-10-10T20:43:28.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.539+0000 7f2b4f59e700 1 -- 172.21.15.89:0/407887199 wait complete. 2023-10-10T20:43:28.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.540+0000 7f2b4f59e700 1 Processor -- start 2023-10-10T20:43:28.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.540+0000 7f2b4f59e700 1 -- start start 2023-10-10T20:43:28.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b4f59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2b500b0360 0x7f2b500c3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:28.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b4f59e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b500cd180 0x7f2b50168900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:28.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b4f59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b500cd5f0 0x7f2b50143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:28.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b4f59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b500c3970 con 0x7f2b500cd5f0 2023-10-10T20:43:28.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b4f59e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b500c3c40 con 0x7f2b500cd180 2023-10-10T20:43:28.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b4f59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b500c3f20 con 0x7f2b500b0360 2023-10-10T20:43:28.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b4ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b500cd5f0 0x7f2b50143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:28.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b4ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b500cd5f0 0x7f2b50143a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37200/0 (socket says 172.21.15.89:37200) 2023-10-10T20:43:28.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b4ed9d700 1 -- 172.21.15.89:0/1854505490 learned_addr learned my addr 172.21.15.89:0/1854505490 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:28.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b4e59c700 1 --2- 172.21.15.89:0/1854505490 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2b500b0360 0x7f2b500c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:28.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b45d9b700 1 --2- 172.21.15.89:0/1854505490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b500cd180 0x7f2b50168900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:28.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b4ed9d700 1 -- 172.21.15.89:0/1854505490 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2b500b0360 msgr2=0x7f2b500c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:28.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b4ed9d700 1 --2- 172.21.15.89:0/1854505490 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2b500b0360 0x7f2b500c3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:28.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b4ed9d700 1 -- 172.21.15.89:0/1854505490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b500cd180 msgr2=0x7f2b50168900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:28.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b4ed9d700 1 --2- 172.21.15.89:0/1854505490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b500cd180 0x7f2b50168900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:28.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b4ed9d700 1 -- 172.21.15.89:0/1854505490 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b38009420 con 0x7f2b500cd5f0 2023-10-10T20:43:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b4ed9d700 1 --2- 172.21.15.89:0/1854505490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b500cd5f0 0x7f2b50143a80 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800ccc0 tx=0x7f2b48001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b46ffd700 1 -- 172.21.15.89:0/1854505490 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4800ab10 con 0x7f2b500cd5f0 2023-10-10T20:43:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b46ffd700 1 -- 172.21.15.89:0/1854505490 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4800ac70 con 0x7f2b500cd5f0 2023-10-10T20:43:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b4f59e700 1 -- 172.21.15.89:0/1854505490 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b500c4200 con 0x7f2b500cd5f0 2023-10-10T20:43:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b46ffd700 1 -- 172.21.15.89:0/1854505490 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b480070f0 con 0x7f2b500cd5f0 2023-10-10T20:43:28.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.541+0000 7f2b4f59e700 1 -- 172.21.15.89:0/1854505490 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b50168e40 con 0x7f2b500cd5f0 2023-10-10T20:43:28.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.542+0000 7f2b4f59e700 1 -- 172.21.15.89:0/1854505490 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b5004c3c0 con 0x7f2b500cd5f0 2023-10-10T20:43:28.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.543+0000 7f2b46ffd700 1 -- 172.21.15.89:0/1854505490 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2b48007520 con 0x7f2b500cd5f0 2023-10-10T20:43:28.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.543+0000 7f2b46ffd700 1 --2- 172.21.15.89:0/1854505490 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2b30067320 0x7f2b300697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:28.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.543+0000 7f2b46ffd700 1 -- 172.21.15.89:0/1854505490 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f2b48084ee0 con 0x7f2b500cd5f0 2023-10-10T20:43:28.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.543+0000 7f2b4e59c700 1 --2- 172.21.15.89:0/1854505490 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2b30067320 0x7f2b300697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:28.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.544+0000 7f2b4e59c700 1 --2- 172.21.15.89:0/1854505490 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2b30067320 0x7f2b300697d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2b40006fd0 tx=0x7f2b40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:28.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.548+0000 7f2b46ffd700 1 -- 172.21.15.89:0/1854505490 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b48051dd0 con 0x7f2b500cd5f0 2023-10-10T20:43:28.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:28.730+0000 7f2b4f59e700 1 -- 172.21.15.89:0/1854505490 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2b500cc550 con 0x7f2b500cd5f0 2023-10-10T20:43:29.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.409+0000 7f2b46ffd700 1 -- 172.21.15.89:0/1854505490 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v181) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2b480553f0 con 0x7f2b500cd5f0 2023-10-10T20:43:29.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.414+0000 7f2b4f59e700 1 -- 172.21.15.89:0/1854505490 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2b30067320 msgr2=0x7f2b300697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:29.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.414+0000 7f2b4f59e700 1 --2- 172.21.15.89:0/1854505490 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2b30067320 0x7f2b300697d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2b40006fd0 tx=0x7f2b40008040 comp rx=0 tx=0).stop 2023-10-10T20:43:29.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.414+0000 7f2b4f59e700 1 -- 172.21.15.89:0/1854505490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b500cd5f0 msgr2=0x7f2b50143a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:29.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.414+0000 7f2b4f59e700 1 --2- 172.21.15.89:0/1854505490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b500cd5f0 0x7f2b50143a80 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800ccc0 tx=0x7f2b48001a20 comp rx=0 tx=0).stop 2023-10-10T20:43:29.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.417+0000 7f2b4f59e700 1 -- 172.21.15.89:0/1854505490 shutdown_connections 2023-10-10T20:43:29.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.417+0000 7f2b4f59e700 1 --2- 172.21.15.89:0/1854505490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b500cd5f0 0x7f2b50143a80 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:29.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.417+0000 7f2b4f59e700 1 --2- 172.21.15.89:0/1854505490 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2b500b0360 0x7f2b500c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:29.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.417+0000 7f2b4f59e700 1 --2- 172.21.15.89:0/1854505490 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2b30067320 0x7f2b300697d0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:29.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.417+0000 7f2b4f59e700 1 --2- 172.21.15.89:0/1854505490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b500cd180 0x7f2b50168900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:29.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.417+0000 7f2b4f59e700 1 -- 172.21.15.89:0/1854505490 >> 172.21.15.89:0/1854505490 conn(0x7f2b5006baa0 msgr2=0x7f2b500caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:29.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.417+0000 7f2b4f59e700 1 -- 172.21.15.89:0/1854505490 shutdown_connections 2023-10-10T20:43:29.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.418+0000 7f2b4f59e700 1 -- 172.21.15.89:0/1854505490 wait complete. 2023-10-10T20:43:29.424 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2023-10-10T20:43:29.439 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:43:29.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.557+0000 7f010ae3d700 1 Processor -- start 2023-10-10T20:43:29.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.557+0000 7f010ae3d700 1 -- start start 2023-10-10T20:43:29.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.558+0000 7f010ae3d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0104159890 0x7f010415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:29.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.558+0000 7f010ae3d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010415a160 con 0x7f0104069560 2023-10-10T20:43:29.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.558+0000 7f010ae3d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01040d13a0 con 0x7f0104159c60 2023-10-10T20:43:29.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.558+0000 7f010ae3d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010415e140 con 0x7f0104159890 2023-10-10T20:43:29.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.558+0000 7f01093da700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0104069560 0x7f01040ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35186/0 (socket says 172.21.15.89:35186) 2023-10-10T20:43:29.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.558+0000 7f01093da700 1 -- 172.21.15.89:0/756757248 learned_addr learned my addr 172.21.15.89:0/756757248 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:29.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01037fe700 1 -- 172.21.15.89:0/756757248 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563083765 0 0) 0x7f010415a160 con 0x7f0104069560 2023-10-10T20:43:29.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01037fe700 1 -- 172.21.15.89:0/756757248 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00ec003570 con 0x7f0104069560 2023-10-10T20:43:29.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f0108bd9700 1 --2- 172.21.15.89:0/756757248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0104159890 0x7f010415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:29.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01037fe700 1 -- 172.21.15.89:0/756757248 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3983003214 0 0) 0x7f00ec003570 con 0x7f0104069560 2023-10-10T20:43:29.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01037fe700 1 -- 172.21.15.89:0/756757248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0104159890 msgr2=0x7f010415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:29.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01037fe700 1 --2- 172.21.15.89:0/756757248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0104159890 0x7f010415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:29.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01037fe700 1 -- 172.21.15.89:0/756757248 >> v1:172.21.15.111:6789/0 conn(0x7f0104159c60 legacy=0x7f010415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:29.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01037fe700 1 -- 172.21.15.89:0/756757248 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010415a3d0 con 0x7f0104069560 2023-10-10T20:43:29.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01037fe700 1 -- 172.21.15.89:0/756757248 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f00fc001470 con 0x7f0104069560 2023-10-10T20:43:29.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01037fe700 1 -- 172.21.15.89:0/756757248 >> v1:172.21.15.89:6789/0 conn(0x7f0104069560 legacy=0x7f01040ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:29.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01037fe700 1 --2- 172.21.15.89:0/756757248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00ec003a90 0x7f00ec003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:29.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01037fe700 1 --2- 172.21.15.89:0/756757248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0104159890 0x7f00ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:29.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01037fe700 1 --2- 172.21.15.89:0/756757248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00ec004d50 0x7f00ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:29.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01037fe700 1 -- 172.21.15.89:0/756757248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010415e140 con 0x7f00ec004d50 2023-10-10T20:43:29.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01037fe700 1 -- 172.21.15.89:0/756757248 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec00bb70 con 0x7f00ec003a90 2023-10-10T20:43:29.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01037fe700 1 -- 172.21.15.89:0/756757248 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec00bd30 con 0x7f0104159890 2023-10-10T20:43:29.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01093da700 1 --2- 172.21.15.89:0/756757248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00ec004d50 0x7f00ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:29.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f0108bd9700 1 --2- 172.21.15.89:0/756757248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00ec003a90 0x7f00ec003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:29.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f0103fff700 1 --2- 172.21.15.89:0/756757248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0104159890 0x7f00ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:29.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01093da700 1 -- 172.21.15.89:0/756757248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0104159890 msgr2=0x7f00ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:29.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01093da700 1 --2- 172.21.15.89:0/756757248 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0104159890 0x7f00ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:29.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01093da700 1 -- 172.21.15.89:0/756757248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00ec003a90 msgr2=0x7f00ec003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:29.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01093da700 1 --2- 172.21.15.89:0/756757248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00ec003a90 0x7f00ec003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:29.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01093da700 1 -- 172.21.15.89:0/756757248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ec003570 con 0x7f00ec004d50 2023-10-10T20:43:29.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.559+0000 7f01093da700 1 --2- 172.21.15.89:0/756757248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00ec004d50 0x7f00ec0051c0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00c3c0 tx=0x7f00fc001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b805e9743834ca server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:29.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.560+0000 7f01037fe700 1 -- 172.21.15.89:0/756757248 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc00cc90 con 0x7f00ec004d50 2023-10-10T20:43:29.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.560+0000 7f01037fe700 1 -- 172.21.15.89:0/756757248 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00fc00cdf0 con 0x7f00ec004d50 2023-10-10T20:43:29.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.560+0000 7f01037fe700 1 -- 172.21.15.89:0/756757248 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc005b50 con 0x7f00ec004d50 2023-10-10T20:43:29.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.560+0000 7f010ae3d700 1 -- 172.21.15.89:0/756757248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00ec004d50 msgr2=0x7f00ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:29.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.560+0000 7f010ae3d700 1 --2- 172.21.15.89:0/756757248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00ec004d50 0x7f00ec0051c0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00c3c0 tx=0x7f00fc001e50 comp rx=0 tx=0).stop 2023-10-10T20:43:29.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.560+0000 7f01093da700 1 -- 172.21.15.89:0/756757248 reap_dead start 2023-10-10T20:43:29.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.560+0000 7f010ae3d700 1 -- 172.21.15.89:0/756757248 shutdown_connections 2023-10-10T20:43:29.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.560+0000 7f010ae3d700 1 -- 172.21.15.89:0/756757248 >> 172.21.15.89:0/756757248 conn(0x7f0104068e20 msgr2=0x7f01040d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:29.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.560+0000 7f010ae3d700 1 -- 172.21.15.89:0/756757248 shutdown_connections 2023-10-10T20:43:29.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.560+0000 7f010ae3d700 1 -- 172.21.15.89:0/756757248 wait complete. 2023-10-10T20:43:29.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.561+0000 7f010ae3d700 1 Processor -- start 2023-10-10T20:43:29.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.561+0000 7f010ae3d700 1 -- start start 2023-10-10T20:43:29.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.562+0000 7f010ae3d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0104139070 0x7f01040b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:29.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.562+0000 7f010ae3d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0104139440 0x7f01040cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:29.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.562+0000 7f010ae3d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01040cfc70 0x7f010415cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:29.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.562+0000 7f010ae3d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01040cec90 con 0x7f0104139070 2023-10-10T20:43:29.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.562+0000 7f010ae3d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01040b54b0 con 0x7f01040cfc70 2023-10-10T20:43:29.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.562+0000 7f010ae3d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01040b51d0 con 0x7f0104139440 2023-10-10T20:43:29.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.562+0000 7f0108bd9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0104139070 0x7f01040b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:29.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.562+0000 7f0108bd9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0104139070 0x7f01040b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37224/0 (socket says 172.21.15.89:37224) 2023-10-10T20:43:29.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.562+0000 7f0108bd9700 1 -- 172.21.15.89:0/3527056025 learned_addr learned my addr 172.21.15.89:0/3527056025 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:29.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.562+0000 7f0103fff700 1 --2- 172.21.15.89:0/3527056025 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0104139440 0x7f01040cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:29.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.562+0000 7f01093da700 1 --2- 172.21.15.89:0/3527056025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01040cfc70 0x7f010415cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:29.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.563+0000 7f0108bd9700 1 -- 172.21.15.89:0/3527056025 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0104139440 msgr2=0x7f01040cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:29.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.563+0000 7f0108bd9700 1 --2- 172.21.15.89:0/3527056025 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0104139440 0x7f01040cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:29.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.563+0000 7f0108bd9700 1 -- 172.21.15.89:0/3527056025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01040cfc70 msgr2=0x7f010415cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:29.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.563+0000 7f0108bd9700 1 --2- 172.21.15.89:0/3527056025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01040cfc70 0x7f010415cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:29.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.563+0000 7f0108bd9700 1 -- 172.21.15.89:0/3527056025 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01040cef30 con 0x7f0104139070 2023-10-10T20:43:29.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.563+0000 7f0103fff700 1 --2- 172.21.15.89:0/3527056025 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0104139440 0x7f01040cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:29.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.563+0000 7f01093da700 1 --2- 172.21.15.89:0/3527056025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01040cfc70 0x7f010415cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:29.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.563+0000 7f0108bd9700 1 --2- 172.21.15.89:0/3527056025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0104139070 0x7f01040b4860 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f00f4011f00 tx=0x7f00f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:29.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.563+0000 7f0100ff9700 1 -- 172.21.15.89:0/3527056025 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f400e040 con 0x7f0104139070 2023-10-10T20:43:29.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.563+0000 7f0100ff9700 1 -- 172.21.15.89:0/3527056025 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f400f040 con 0x7f0104139070 2023-10-10T20:43:29.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.563+0000 7f0100ff9700 1 -- 172.21.15.89:0/3527056025 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4015430 con 0x7f0104139070 2023-10-10T20:43:29.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.563+0000 7f010ae3d700 1 -- 172.21.15.89:0/3527056025 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01040cf210 con 0x7f0104139070 2023-10-10T20:43:29.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.563+0000 7f010ae3d700 1 -- 172.21.15.89:0/3527056025 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01040b45f0 con 0x7f0104139070 2023-10-10T20:43:29.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.564+0000 7f010ae3d700 1 -- 172.21.15.89:0/3527056025 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01040ceaf0 con 0x7f0104139070 2023-10-10T20:43:29.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.564+0000 7f0100ff9700 1 -- 172.21.15.89:0/3527056025 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f00f4015860 con 0x7f0104139070 2023-10-10T20:43:29.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.565+0000 7f0100ff9700 1 --2- 172.21.15.89:0/3527056025 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f00f0067320 0x7f00f00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:29.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.565+0000 7f0100ff9700 1 -- 172.21.15.89:0/3527056025 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f00f4085a80 con 0x7f0104139070 2023-10-10T20:43:29.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.565+0000 7f0103fff700 1 --2- 172.21.15.89:0/3527056025 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f00f0067320 0x7f00f00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:29.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.565+0000 7f0103fff700 1 --2- 172.21.15.89:0/3527056025 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f00f0067320 0x7f00f00697d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f01040c5950 tx=0x7f00f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:29.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.569+0000 7f0100ff9700 1 -- 172.21.15.89:0/3527056025 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01040ceaf0 con 0x7f0104139070 2023-10-10T20:43:29.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:29.793+0000 7f010ae3d700 1 -- 172.21.15.89:0/3527056025 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f01040ceaf0 con 0x7f0104139070 2023-10-10T20:43:30.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.406+0000 7f0100ff9700 1 -- 172.21.15.89:0/3527056025 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 31 and data pool 32 v117) v1 ==== 174+0+0 (secure 0 0 0) 0x7f01040ceaf0 con 0x7f0104139070 2023-10-10T20:43:30.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.417+0000 7f010ae3d700 1 -- 172.21.15.89:0/3527056025 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f00f0067320 msgr2=0x7f00f00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:30.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.417+0000 7f010ae3d700 1 --2- 172.21.15.89:0/3527056025 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f00f0067320 0x7f00f00697d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f01040c5950 tx=0x7f00f8009040 comp rx=0 tx=0).stop 2023-10-10T20:43:30.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.417+0000 7f010ae3d700 1 -- 172.21.15.89:0/3527056025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0104139070 msgr2=0x7f01040b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:30.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.417+0000 7f010ae3d700 1 --2- 172.21.15.89:0/3527056025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0104139070 0x7f01040b4860 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f00f4011f00 tx=0x7f00f400b040 comp rx=0 tx=0).stop 2023-10-10T20:43:30.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.417+0000 7f010ae3d700 1 -- 172.21.15.89:0/3527056025 shutdown_connections 2023-10-10T20:43:30.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.417+0000 7f010ae3d700 1 --2- 172.21.15.89:0/3527056025 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f00f0067320 0x7f00f00697d0 secure :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f01040c5950 tx=0x7f00f8009040 comp rx=0 tx=0).stop 2023-10-10T20:43:30.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.417+0000 7f010ae3d700 1 --2- 172.21.15.89:0/3527056025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01040cfc70 0x7f010415cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:30.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.417+0000 7f010ae3d700 1 --2- 172.21.15.89:0/3527056025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0104139070 0x7f01040b4860 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:30.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.417+0000 7f010ae3d700 1 --2- 172.21.15.89:0/3527056025 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0104139440 0x7f01040cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:30.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.417+0000 7f010ae3d700 1 -- 172.21.15.89:0/3527056025 >> 172.21.15.89:0/3527056025 conn(0x7f0104068e20 msgr2=0x7f0104154040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:30.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.420+0000 7f010ae3d700 1 -- 172.21.15.89:0/3527056025 shutdown_connections 2023-10-10T20:43:30.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.420+0000 7f010ae3d700 1 -- 172.21.15.89:0/3527056025 wait complete. 2023-10-10T20:43:30.430 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 31 and data pool 32 2023-10-10T20:43:30.445 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:43:30.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.558+0000 7f3a26c2d700 1 Processor -- start 2023-10-10T20:43:30.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.559+0000 7f3a26c2d700 1 -- start start 2023-10-10T20:43:30.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.559+0000 7f3a26c2d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a20159420 0x7f3a200b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:30.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.559+0000 7f3a26c2d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3a201597f0 0x7f3a200b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:30.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.559+0000 7f3a26c2d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a200b11d0 con 0x7f3a2015d230 2023-10-10T20:43:30.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.559+0000 7f3a26c2d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a200b14b0 con 0x7f3a201597f0 2023-10-10T20:43:30.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.559+0000 7f3a26c2d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a200b1790 con 0x7f3a20159420 2023-10-10T20:43:30.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.560+0000 7f3a2542a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3a201597f0 0x7f3a200b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:30.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.560+0000 7f3a2642c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3a2015d230 0x7f3a200b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35196/0 (socket says 172.21.15.89:35196) 2023-10-10T20:43:30.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.560+0000 7f3a2642c700 1 -- 172.21.15.89:0/1959022312 learned_addr learned my addr 172.21.15.89:0/1959022312 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:30.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.560+0000 7f3a25c2b700 1 --2- 172.21.15.89:0/1959022312 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a20159420 0x7f3a200b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:30.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.560+0000 7f3a2542a700 1 -- 172.21.15.89:0/1959022312 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a20159420 msgr2=0x7f3a200b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:30.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.560+0000 7f3a2542a700 1 --2- 172.21.15.89:0/1959022312 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a20159420 0x7f3a200b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:30.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.560+0000 7f3a2542a700 1 -- 172.21.15.89:0/1959022312 >> v1:172.21.15.89:6789/0 conn(0x7f3a2015d230 legacy=0x7f3a200b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:30.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.560+0000 7f3a2542a700 1 -- 172.21.15.89:0/1959022312 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a200b1a70 con 0x7f3a201597f0 2023-10-10T20:43:30.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.560+0000 7f3a24c29700 1 -- 172.21.15.89:0/1959022312 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1102830451 0 0) 0x7f3a200b11d0 con 0x7f3a2015d230 2023-10-10T20:43:30.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.560+0000 7f3a2542a700 1 --2- 172.21.15.89:0/1959022312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a201597f0 0x7f3a200b2270 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f3a100098e0 tx=0x7f3a100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731cb961f92c7aa6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:30.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.560+0000 7f3a24c29700 1 -- 172.21.15.89:0/1959022312 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10009e80 con 0x7f3a201597f0 2023-10-10T20:43:30.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.560+0000 7f3a24c29700 1 -- 172.21.15.89:0/1959022312 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1000b860 con 0x7f3a201597f0 2023-10-10T20:43:30.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.560+0000 7f3a24c29700 1 -- 172.21.15.89:0/1959022312 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1000bbe0 con 0x7f3a201597f0 2023-10-10T20:43:30.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.561+0000 7f3a26c2d700 1 -- 172.21.15.89:0/1959022312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a201597f0 msgr2=0x7f3a200b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:30.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.561+0000 7f3a26c2d700 1 --2- 172.21.15.89:0/1959022312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a201597f0 0x7f3a200b2270 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f3a100098e0 tx=0x7f3a100091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:30.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.561+0000 7f3a26c2d700 1 -- 172.21.15.89:0/1959022312 shutdown_connections 2023-10-10T20:43:30.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.561+0000 7f3a26c2d700 1 --2- 172.21.15.89:0/1959022312 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a20159420 0x7f3a200b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:30.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.561+0000 7f3a26c2d700 1 --2- 172.21.15.89:0/1959022312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a201597f0 0x7f3a200b2270 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:30.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.561+0000 7f3a26c2d700 1 -- 172.21.15.89:0/1959022312 >> 172.21.15.89:0/1959022312 conn(0x7f3a200d0660 msgr2=0x7f3a200d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:30.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.561+0000 7f3a26c2d700 1 -- 172.21.15.89:0/1959022312 shutdown_connections 2023-10-10T20:43:30.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.561+0000 7f3a26c2d700 1 -- 172.21.15.89:0/1959022312 wait complete. 2023-10-10T20:43:30.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.562+0000 7f3a26c2d700 1 Processor -- start 2023-10-10T20:43:30.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.562+0000 7f3a26c2d700 1 -- start start 2023-10-10T20:43:30.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a26c2d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a20159420 0x7f3a2014adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:30.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a26c2d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a201597f0 0x7f3a20154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:30.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a26c2d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a2015d230 0x7f3a2014f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:30.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a26c2d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a201527d0 con 0x7f3a2015d230 2023-10-10T20:43:30.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a26c2d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a20154740 con 0x7f3a20159420 2023-10-10T20:43:30.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a26c2d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a20151bc0 con 0x7f3a201597f0 2023-10-10T20:43:30.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a2642c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a2015d230 0x7f3a2014f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:30.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a2642c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a2015d230 0x7f3a2014f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37240/0 (socket says 172.21.15.89:37240) 2023-10-10T20:43:30.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a2642c700 1 -- 172.21.15.89:0/4154007264 learned_addr learned my addr 172.21.15.89:0/4154007264 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:30.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a2542a700 1 --2- 172.21.15.89:0/4154007264 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a201597f0 0x7f3a20154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:30.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a25c2b700 1 --2- 172.21.15.89:0/4154007264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a20159420 0x7f3a2014adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:30.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a2642c700 1 -- 172.21.15.89:0/4154007264 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a201597f0 msgr2=0x7f3a20154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:30.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a2642c700 1 --2- 172.21.15.89:0/4154007264 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a201597f0 0x7f3a20154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:30.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a2642c700 1 -- 172.21.15.89:0/4154007264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a20159420 msgr2=0x7f3a2014adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:30.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a2642c700 1 --2- 172.21.15.89:0/4154007264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a20159420 0x7f3a2014adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:30.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a2642c700 1 -- 172.21.15.89:0/4154007264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a10009420 con 0x7f3a2015d230 2023-10-10T20:43:30.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a25c2b700 1 --2- 172.21.15.89:0/4154007264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a20159420 0x7f3a2014adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:30.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a2642c700 1 --2- 172.21.15.89:0/4154007264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a2015d230 0x7f3a2014f680 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800c560 tx=0x7f3a18007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:30.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a15ffb700 1 -- 172.21.15.89:0/4154007264 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18005b30 con 0x7f3a2015d230 2023-10-10T20:43:30.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a15ffb700 1 -- 172.21.15.89:0/4154007264 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a18005c90 con 0x7f3a2015d230 2023-10-10T20:43:30.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a26c2d700 1 -- 172.21.15.89:0/4154007264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a20151ea0 con 0x7f3a2015d230 2023-10-10T20:43:30.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.563+0000 7f3a15ffb700 1 -- 172.21.15.89:0/4154007264 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18007320 con 0x7f3a2015d230 2023-10-10T20:43:30.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.564+0000 7f3a26c2d700 1 -- 172.21.15.89:0/4154007264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a20152460 con 0x7f3a2015d230 2023-10-10T20:43:30.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.565+0000 7f3a26c2d700 1 -- 172.21.15.89:0/4154007264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a20152980 con 0x7f3a2015d230 2023-10-10T20:43:30.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.565+0000 7f3a15ffb700 1 -- 172.21.15.89:0/4154007264 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3a1801b3b0 con 0x7f3a2015d230 2023-10-10T20:43:30.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.565+0000 7f3a15ffb700 1 --2- 172.21.15.89:0/4154007264 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3a04067320 0x7f3a040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:30.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.565+0000 7f3a15ffb700 1 -- 172.21.15.89:0/4154007264 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3a180840e0 con 0x7f3a2015d230 2023-10-10T20:43:30.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.565+0000 7f3a25c2b700 1 --2- 172.21.15.89:0/4154007264 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3a04067320 0x7f3a040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:30.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.566+0000 7f3a25c2b700 1 --2- 172.21.15.89:0/4154007264 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3a04067320 0x7f3a040697d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00e5a0 tx=0x7f3a1c007710 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:30.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.569+0000 7f3a15ffb700 1 -- 172.21.15.89:0/4154007264 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a20152980 con 0x7f3a2015d230 2023-10-10T20:43:30.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.754+0000 7f3a26c2d700 1 -- 172.21.15.89:0/4154007264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a20152980 con 0x7f3a2015d230 2023-10-10T20:43:30.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.755+0000 7f3a15ffb700 1 -- 172.21.15.89:0/4154007264 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+15377 (secure 0 0 0) 0x7f3a20152980 con 0x7f3a2015d230 2023-10-10T20:43:30.757 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:30.758 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":182,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:43:29.794795+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:43:28.133902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-10T20:43:28.749600+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:43:30.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.758+0000 7f3a26c2d700 1 -- 172.21.15.89:0/4154007264 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3a04067320 msgr2=0x7f3a040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:30.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.758+0000 7f3a26c2d700 1 --2- 172.21.15.89:0/4154007264 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3a04067320 0x7f3a040697d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00e5a0 tx=0x7f3a1c007710 comp rx=0 tx=0).stop 2023-10-10T20:43:30.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.758+0000 7f3a26c2d700 1 -- 172.21.15.89:0/4154007264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a2015d230 msgr2=0x7f3a2014f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:30.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.758+0000 7f3a26c2d700 1 --2- 172.21.15.89:0/4154007264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a2015d230 0x7f3a2014f680 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800c560 tx=0x7f3a18007f90 comp rx=0 tx=0).stop 2023-10-10T20:43:30.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.758+0000 7f3a26c2d700 1 -- 172.21.15.89:0/4154007264 shutdown_connections 2023-10-10T20:43:30.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.758+0000 7f3a26c2d700 1 --2- 172.21.15.89:0/4154007264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a2015d230 0x7f3a2014f680 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:30.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.758+0000 7f3a26c2d700 1 --2- 172.21.15.89:0/4154007264 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3a04067320 0x7f3a040697d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:30.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.758+0000 7f3a26c2d700 1 --2- 172.21.15.89:0/4154007264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a20159420 0x7f3a2014adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:30.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.758+0000 7f3a26c2d700 1 --2- 172.21.15.89:0/4154007264 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a201597f0 0x7f3a20154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:30.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.758+0000 7f3a26c2d700 1 -- 172.21.15.89:0/4154007264 >> 172.21.15.89:0/4154007264 conn(0x7f3a200d0660 msgr2=0x7f3a20156a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:30.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.758+0000 7f3a26c2d700 1 -- 172.21.15.89:0/4154007264 shutdown_connections 2023-10-10T20:43:30.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.758+0000 7f3a26c2d700 1 -- 172.21.15.89:0/4154007264 wait complete. 2023-10-10T20:43:30.773 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:43:30.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.893+0000 7fa5d5b58700 1 Processor -- start 2023-10-10T20:43:30.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.893+0000 7fa5d5b58700 1 -- start start 2023-10-10T20:43:30.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.894+0000 7fa5d5b58700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa5d0159c60 0x7fa5d015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:30.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.894+0000 7fa5d5b58700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d015a160 con 0x7fa5d0159890 2023-10-10T20:43:30.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.894+0000 7fa5d5b58700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d00d13a0 con 0x7fa5d015db10 2023-10-10T20:43:30.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.894+0000 7fa5d5b58700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d015e140 con 0x7fa5d0159c60 2023-10-10T20:43:30.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.894+0000 7fa5cf7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa5d0159890 0x7fa5d015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35200/0 (socket says 172.21.15.89:35200) 2023-10-10T20:43:30.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.894+0000 7fa5cf7fe700 1 -- 172.21.15.89:0/4131660766 learned_addr learned my addr 172.21.15.89:0/4131660766 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:30.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/4131660766 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361611394 0 0) 0x7fa5d015a160 con 0x7fa5d0159890 2023-10-10T20:43:30.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/4131660766 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b8003570 con 0x7fa5d0159890 2023-10-10T20:43:30.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5ceffd700 1 --2- 172.21.15.89:0/4131660766 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa5d0159c60 0x7fa5d015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:30.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/4131660766 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4288140786 0 0) 0x7fa5b8003570 con 0x7fa5d0159890 2023-10-10T20:43:30.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/4131660766 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa5d0159c60 msgr2=0x7fa5d015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:30.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5ce7fc700 1 --2- 172.21.15.89:0/4131660766 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa5d0159c60 0x7fa5d015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:30.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/4131660766 >> v1:172.21.15.111:6789/0 conn(0x7fa5d015db10 legacy=0x7fa5d0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:30.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/4131660766 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d015a3d0 con 0x7fa5d0159890 2023-10-10T20:43:30.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/4131660766 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fa5c00017f0 con 0x7fa5d0159890 2023-10-10T20:43:30.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/4131660766 >> v1:172.21.15.89:6789/0 conn(0x7fa5d0159890 legacy=0x7fa5d015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:30.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5ce7fc700 1 --2- 172.21.15.89:0/4131660766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b8003a90 0x7fa5b8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:30.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5ce7fc700 1 --2- 172.21.15.89:0/4131660766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5b8004810 0x7fa5b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:30.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5ce7fc700 1 --2- 172.21.15.89:0/4131660766 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa5d0159c60 0x7fa5b80051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:30.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/4131660766 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d015e140 con 0x7fa5b8003a90 2023-10-10T20:43:30.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/4131660766 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b800bb60 con 0x7fa5b8004810 2023-10-10T20:43:30.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/4131660766 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b800bd20 con 0x7fa5d0159c60 2023-10-10T20:43:30.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5cf7fe700 1 --2- 172.21.15.89:0/4131660766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b8003a90 0x7fa5b8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:30.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5ceffd700 1 --2- 172.21.15.89:0/4131660766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5b8004810 0x7fa5b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:30.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5cffff700 1 --2- 172.21.15.89:0/4131660766 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa5d0159c60 0x7fa5b80051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:30.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5cf7fe700 1 -- 172.21.15.89:0/4131660766 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa5d0159c60 msgr2=0x7fa5b80051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:30.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5cf7fe700 1 --2- 172.21.15.89:0/4131660766 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa5d0159c60 0x7fa5b80051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:30.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5cf7fe700 1 -- 172.21.15.89:0/4131660766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5b8004810 msgr2=0x7fa5b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:30.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5cf7fe700 1 --2- 172.21.15.89:0/4131660766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5b8004810 0x7fa5b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:30.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5cf7fe700 1 -- 172.21.15.89:0/4131660766 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b8003570 con 0x7fa5b8003a90 2023-10-10T20:43:30.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.895+0000 7fa5cf7fe700 1 --2- 172.21.15.89:0/4131660766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b8003a90 0x7fa5b8003ea0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000a930 tx=0x7fa5c00021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea75dc6c16351d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:30.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.896+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/4131660766 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c00070d0 con 0x7fa5b8003a90 2023-10-10T20:43:30.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.896+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/4131660766 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c0007230 con 0x7fa5b8003a90 2023-10-10T20:43:30.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.896+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/4131660766 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c00075d0 con 0x7fa5b8003a90 2023-10-10T20:43:30.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.896+0000 7fa5d5b58700 1 -- 172.21.15.89:0/4131660766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b8003a90 msgr2=0x7fa5b8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:30.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.896+0000 7fa5d5b58700 1 --2- 172.21.15.89:0/4131660766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b8003a90 0x7fa5b8003ea0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000a930 tx=0x7fa5c00021d0 comp rx=0 tx=0).stop 2023-10-10T20:43:30.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.896+0000 7fa5cffff700 1 -- 172.21.15.89:0/4131660766 reap_dead start 2023-10-10T20:43:30.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.896+0000 7fa5d5b58700 1 -- 172.21.15.89:0/4131660766 shutdown_connections 2023-10-10T20:43:30.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.896+0000 7fa5d5b58700 1 -- 172.21.15.89:0/4131660766 >> 172.21.15.89:0/4131660766 conn(0x7fa5d0068e20 msgr2=0x7fa5d00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:30.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.896+0000 7fa5d5b58700 1 -- 172.21.15.89:0/4131660766 shutdown_connections 2023-10-10T20:43:30.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.896+0000 7fa5d5b58700 1 -- 172.21.15.89:0/4131660766 wait complete. 2023-10-10T20:43:30.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.897+0000 7fa5d5b58700 1 Processor -- start 2023-10-10T20:43:30.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.898+0000 7fa5d5b58700 1 -- start start 2023-10-10T20:43:30.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.898+0000 7fa5d5b58700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b8003a90 0x7fa5d00c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:30.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.898+0000 7fa5d5b58700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa5d00c8280 0x7fa5d01523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:30.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.898+0000 7fa5d5b58700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5d00b07e0 0x7fa5d00b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:30.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.898+0000 7fa5d5b58700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d00b11c0 con 0x7fa5b8003a90 2023-10-10T20:43:30.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.898+0000 7fa5d5b58700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d00c7de0 con 0x7fa5d00b07e0 2023-10-10T20:43:30.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.898+0000 7fa5d5b58700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d0155630 con 0x7fa5d00c8280 2023-10-10T20:43:30.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.898+0000 7fa5cf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b8003a90 0x7fa5d00c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:30.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.898+0000 7fa5cf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b8003a90 0x7fa5d00c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37258/0 (socket says 172.21.15.89:37258) 2023-10-10T20:43:30.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.898+0000 7fa5cf7fe700 1 -- 172.21.15.89:0/10244636 learned_addr learned my addr 172.21.15.89:0/10244636 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:30.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.899+0000 7fa5cf7fe700 1 -- 172.21.15.89:0/10244636 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa5d00c8280 msgr2=0x7fa5d01523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:30.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.899+0000 7fa5cf7fe700 1 --2- 172.21.15.89:0/10244636 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa5d00c8280 0x7fa5d01523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:30.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.899+0000 7fa5cf7fe700 1 -- 172.21.15.89:0/10244636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5d00b07e0 msgr2=0x7fa5d00b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.899+0000 7fa5cf7fe700 1 --2- 172.21.15.89:0/10244636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5d00b07e0 0x7fa5d00b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.899+0000 7fa5cf7fe700 1 -- 172.21.15.89:0/10244636 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d01581a0 con 0x7fa5b8003a90 2023-10-10T20:43:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.899+0000 7fa5cf7fe700 1 --2- 172.21.15.89:0/10244636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b8003a90 0x7fa5d00c78a0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000a3f0 tx=0x7fa5c0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.899+0000 7fa5b77fe700 1 -- 172.21.15.89:0/10244636 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0001320 con 0x7fa5b8003a90 2023-10-10T20:43:30.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.899+0000 7fa5b77fe700 1 -- 172.21.15.89:0/10244636 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c0001a90 con 0x7fa5b8003a90 2023-10-10T20:43:30.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.899+0000 7fa5d5b58700 1 -- 172.21.15.89:0/10244636 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d0155020 con 0x7fa5b8003a90 2023-10-10T20:43:30.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.899+0000 7fa5b77fe700 1 -- 172.21.15.89:0/10244636 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0007940 con 0x7fa5b8003a90 2023-10-10T20:43:30.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.899+0000 7fa5d5b58700 1 -- 172.21.15.89:0/10244636 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d00c8710 con 0x7fa5b8003a90 2023-10-10T20:43:30.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.900+0000 7fa5d5b58700 1 -- 172.21.15.89:0/10244636 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d00b0140 con 0x7fa5b8003a90 2023-10-10T20:43:30.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.900+0000 7fa5b77fe700 1 -- 172.21.15.89:0/10244636 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa5c0024070 con 0x7fa5b8003a90 2023-10-10T20:43:30.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.901+0000 7fa5b77fe700 1 --2- 172.21.15.89:0/10244636 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa5bc067320 0x7fa5bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:30.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.901+0000 7fa5b77fe700 1 -- 172.21.15.89:0/10244636 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa5c00874a0 con 0x7fa5b8003a90 2023-10-10T20:43:30.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.901+0000 7fa5ceffd700 1 --2- 172.21.15.89:0/10244636 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa5bc067320 0x7fa5bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:30.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.901+0000 7fa5ceffd700 1 --2- 172.21.15.89:0/10244636 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa5bc067320 0x7fa5bc0697d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa5d015a6e0 tx=0x7fa5c40091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:30.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:30.905+0000 7fa5b77fe700 1 -- 172.21.15.89:0/10244636 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5c0054240 con 0x7fa5b8003a90 2023-10-10T20:43:31.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.087+0000 7fa5d5b58700 1 -- 172.21.15.89:0/10244636 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5d004c3c0 con 0x7fa5b8003a90 2023-10-10T20:43:31.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.088+0000 7fa5b77fe700 1 -- 172.21.15.89:0/10244636 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+15377 (secure 0 0 0) 0x7fa5c0057860 con 0x7fa5b8003a90 2023-10-10T20:43:31.090 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:31.090 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":182,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:43:29.794795+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:43:28.133902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-10T20:43:28.749600+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:43:31.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.090+0000 7fa5d5b58700 1 -- 172.21.15.89:0/10244636 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa5bc067320 msgr2=0x7fa5bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:31.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.090+0000 7fa5d5b58700 1 --2- 172.21.15.89:0/10244636 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa5bc067320 0x7fa5bc0697d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa5d015a6e0 tx=0x7fa5c40091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:31.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.090+0000 7fa5d5b58700 1 -- 172.21.15.89:0/10244636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b8003a90 msgr2=0x7fa5d00c78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:31.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.090+0000 7fa5d5b58700 1 --2- 172.21.15.89:0/10244636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b8003a90 0x7fa5d00c78a0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000a3f0 tx=0x7fa5c0016000 comp rx=0 tx=0).stop 2023-10-10T20:43:31.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.090+0000 7fa5d5b58700 1 -- 172.21.15.89:0/10244636 shutdown_connections 2023-10-10T20:43:31.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.090+0000 7fa5d5b58700 1 --2- 172.21.15.89:0/10244636 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa5bc067320 0x7fa5bc0697d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:31.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.090+0000 7fa5d5b58700 1 --2- 172.21.15.89:0/10244636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5d00b07e0 0x7fa5d00b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:31.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.090+0000 7fa5d5b58700 1 --2- 172.21.15.89:0/10244636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b8003a90 0x7fa5d00c78a0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:31.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.090+0000 7fa5d5b58700 1 --2- 172.21.15.89:0/10244636 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa5d00c8280 0x7fa5d01523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:31.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.090+0000 7fa5d5b58700 1 -- 172.21.15.89:0/10244636 >> 172.21.15.89:0/10244636 conn(0x7fa5d0068e20 msgr2=0x7fa5d00b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:31.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.090+0000 7fa5d5b58700 1 -- 172.21.15.89:0/10244636 shutdown_connections 2023-10-10T20:43:31.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.090+0000 7fa5d5b58700 1 -- 172.21.15.89:0/10244636 wait complete. 2023-10-10T20:43:31.106 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:43:31.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.224+0000 7f7a9ce18700 1 Processor -- start 2023-10-10T20:43:31.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.224+0000 7f7a9ce18700 1 -- start start 2023-10-10T20:43:31.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.225+0000 7f7a9ce18700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7a9806c0a0 0x7f7a980caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:31.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.225+0000 7f7a9ce18700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a9815df50 0x7f7a9815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:31.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.225+0000 7f7a9ce18700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a9815d7e0 con 0x7f7a9806bcd0 2023-10-10T20:43:31.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.225+0000 7f7a9ce18700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9815dac0 con 0x7f7a9806c0a0 2023-10-10T20:43:31.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.225+0000 7f7a9ce18700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9815aa30 con 0x7f7a9815df50 2023-10-10T20:43:31.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.225+0000 7f7a96d9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a9815df50 0x7f7a9815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:31.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.225+0000 7f7a95d9b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7a9806c0a0 0x7f7a980caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:31.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.225+0000 7f7a96d9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a9815df50 0x7f7a9815e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:49438/0 (socket says 172.21.15.89:49438) 2023-10-10T20:43:31.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.225+0000 7f7a96d9d700 1 -- 172.21.15.89:0/1540376006 learned_addr learned my addr 172.21.15.89:0/1540376006 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:31.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.225+0000 7f7a96d9d700 1 -- 172.21.15.89:0/1540376006 >> v2:172.21.15.111:3300/0 conn(0x7f7a9806c0a0 msgr2=0x7f7a980caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:31.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.225+0000 7f7a96d9d700 1 --2- 172.21.15.89:0/1540376006 >> v2:172.21.15.111:3300/0 conn(0x7f7a9806c0a0 0x7f7a980caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:31.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.225+0000 7f7a96d9d700 1 -- 172.21.15.89:0/1540376006 >> v1:172.21.15.111:6789/0 conn(0x7f7a9806bcd0 legacy=0x7f7a9815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:31.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.225+0000 7f7a96d9d700 1 -- 172.21.15.89:0/1540376006 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9815ad10 con 0x7f7a9815df50 2023-10-10T20:43:31.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.225+0000 7f7a96d9d700 1 --2- 172.21.15.89:0/1540376006 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a9815df50 0x7f7a9815e3f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009f20 tx=0x7f7a90009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6dead9c4f39141a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:31.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.226+0000 7f7a9559a700 1 -- 172.21.15.89:0/1540376006 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9000e070 con 0x7f7a9815df50 2023-10-10T20:43:31.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.226+0000 7f7a9559a700 1 -- 172.21.15.89:0/1540376006 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9000b9d0 con 0x7f7a9815df50 2023-10-10T20:43:31.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.226+0000 7f7a9559a700 1 -- 172.21.15.89:0/1540376006 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a900092e0 con 0x7f7a9815df50 2023-10-10T20:43:31.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.226+0000 7f7a9ce18700 1 -- 172.21.15.89:0/1540376006 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a9815df50 msgr2=0x7f7a9815e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:31.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.226+0000 7f7a9ce18700 1 --2- 172.21.15.89:0/1540376006 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a9815df50 0x7f7a9815e3f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009f20 tx=0x7f7a90009840 comp rx=0 tx=0).stop 2023-10-10T20:43:31.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.226+0000 7f7a9ce18700 1 -- 172.21.15.89:0/1540376006 shutdown_connections 2023-10-10T20:43:31.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.226+0000 7f7a9ce18700 1 --2- 172.21.15.89:0/1540376006 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a9815df50 0x7f7a9815e3f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:31.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.226+0000 7f7a9ce18700 1 --2- 172.21.15.89:0/1540376006 >> v2:172.21.15.111:3300/0 conn(0x7f7a9806c0a0 0x7f7a980caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:31.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.226+0000 7f7a9ce18700 1 -- 172.21.15.89:0/1540376006 >> 172.21.15.89:0/1540376006 conn(0x7f7a980ce600 msgr2=0x7f7a980cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:31.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.226+0000 7f7a9ce18700 1 -- 172.21.15.89:0/1540376006 shutdown_connections 2023-10-10T20:43:31.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.226+0000 7f7a9ce18700 1 -- 172.21.15.89:0/1540376006 wait complete. 2023-10-10T20:43:31.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.227+0000 7f7a9ce18700 1 Processor -- start 2023-10-10T20:43:31.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.227+0000 7f7a9ce18700 1 -- start start 2023-10-10T20:43:31.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.228+0000 7f7a9ce18700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a9806bcd0 0x7f7a980b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:31.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.228+0000 7f7a9ce18700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a9806c0a0 0x7f7a9816a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:31.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.228+0000 7f7a9ce18700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a9815df50 0x7f7a980b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:31.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.228+0000 7f7a9ce18700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980c5d10 con 0x7f7a9806bcd0 2023-10-10T20:43:31.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.228+0000 7f7a9ce18700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9816abd0 con 0x7f7a9815df50 2023-10-10T20:43:31.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.228+0000 7f7a9ce18700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980c5750 con 0x7f7a9806c0a0 2023-10-10T20:43:31.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.228+0000 7f7a9659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a9806bcd0 0x7f7a980b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:31.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.228+0000 7f7a9659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a9806bcd0 0x7f7a980b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37274/0 (socket says 172.21.15.89:37274) 2023-10-10T20:43:31.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.228+0000 7f7a9659c700 1 -- 172.21.15.89:0/7451541 learned_addr learned my addr 172.21.15.89:0/7451541 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:31.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.228+0000 7f7a95d9b700 1 --2- 172.21.15.89:0/7451541 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a9806c0a0 0x7f7a9816a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.228+0000 7f7a96d9d700 1 --2- 172.21.15.89:0/7451541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a9815df50 0x7f7a980b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.229+0000 7f7a9659c700 1 -- 172.21.15.89:0/7451541 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a9806c0a0 msgr2=0x7f7a9816a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.229+0000 7f7a9659c700 1 --2- 172.21.15.89:0/7451541 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a9806c0a0 0x7f7a9816a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.229+0000 7f7a9659c700 1 -- 172.21.15.89:0/7451541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a9815df50 msgr2=0x7f7a980b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.229+0000 7f7a9659c700 1 --2- 172.21.15.89:0/7451541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a9815df50 0x7f7a980b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:31.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.229+0000 7f7a9659c700 1 -- 172.21.15.89:0/7451541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a90009ae0 con 0x7f7a9806bcd0 2023-10-10T20:43:31.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.229+0000 7f7a9659c700 1 --2- 172.21.15.89:0/7451541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a9806bcd0 0x7f7a980b3e20 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800c4e0 tx=0x7f7a8800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:31.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.229+0000 7f7a867fc700 1 -- 172.21.15.89:0/7451541 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88011590 con 0x7f7a9806bcd0 2023-10-10T20:43:31.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.229+0000 7f7a867fc700 1 -- 172.21.15.89:0/7451541 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a88011710 con 0x7f7a9806bcd0 2023-10-10T20:43:31.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.229+0000 7f7a867fc700 1 -- 172.21.15.89:0/7451541 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88011ad0 con 0x7f7a9806bcd0 2023-10-10T20:43:31.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.229+0000 7f7a9ce18700 1 -- 172.21.15.89:0/7451541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a980c5a30 con 0x7f7a9806bcd0 2023-10-10T20:43:31.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.229+0000 7f7a9ce18700 1 -- 172.21.15.89:0/7451541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a9816d6e0 con 0x7f7a9806bcd0 2023-10-10T20:43:31.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.230+0000 7f7a9ce18700 1 -- 172.21.15.89:0/7451541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a9804c3c0 con 0x7f7a9806bcd0 2023-10-10T20:43:31.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.230+0000 7f7a867fc700 1 -- 172.21.15.89:0/7451541 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7a8801b3b0 con 0x7f7a9806bcd0 2023-10-10T20:43:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.230+0000 7f7a867fc700 1 --2- 172.21.15.89:0/7451541 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7a78067320 0x7f7a780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.231+0000 7f7a867fc700 1 -- 172.21.15.89:0/7451541 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7a88084070 con 0x7f7a9806bcd0 2023-10-10T20:43:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.231+0000 7f7a95d9b700 1 --2- 172.21.15.89:0/7451541 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7a78067320 0x7f7a780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.231+0000 7f7a95d9b700 1 --2- 172.21.15.89:0/7451541 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7a78067320 0x7f7a780697d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7a980c4da0 tx=0x7f7a8c0091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.235+0000 7f7a867fc700 1 -- 172.21.15.89:0/7451541 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a88050cc0 con 0x7f7a9806bcd0 2023-10-10T20:43:31.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:31.464+0000 7f7a9ce18700 1 -- 172.21.15.89:0/7451541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7a9815aff0 con 0x7f7a9806bcd0 2023-10-10T20:43:32.086 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-10T20:43:32.086 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-10-10T20:43:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.429+0000 7f7a867fc700 1 -- 172.21.15.89:0/7451541 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v120) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7a88017070 con 0x7f7a9806bcd0 2023-10-10T20:43:32.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.431+0000 7f7a9ce18700 1 -- 172.21.15.89:0/7451541 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7a78067320 msgr2=0x7f7a780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:32.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.431+0000 7f7a9ce18700 1 --2- 172.21.15.89:0/7451541 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7a78067320 0x7f7a780697d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7a980c4da0 tx=0x7f7a8c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:32.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.431+0000 7f7a9ce18700 1 -- 172.21.15.89:0/7451541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a9806bcd0 msgr2=0x7f7a980b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:32.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.431+0000 7f7a9ce18700 1 --2- 172.21.15.89:0/7451541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a9806bcd0 0x7f7a980b3e20 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800c4e0 tx=0x7f7a8800c7f0 comp rx=0 tx=0).stop 2023-10-10T20:43:32.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.431+0000 7f7a9ce18700 1 -- 172.21.15.89:0/7451541 shutdown_connections 2023-10-10T20:43:32.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.431+0000 7f7a9ce18700 1 --2- 172.21.15.89:0/7451541 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7a78067320 0x7f7a780697d0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:32.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.431+0000 7f7a9ce18700 1 --2- 172.21.15.89:0/7451541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a9815df50 0x7f7a980b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:32.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.431+0000 7f7a9ce18700 1 --2- 172.21.15.89:0/7451541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a9806bcd0 0x7f7a980b3e20 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:32.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.431+0000 7f7a9ce18700 1 --2- 172.21.15.89:0/7451541 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7a9806c0a0 0x7f7a9816a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:32.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.431+0000 7f7a9ce18700 1 -- 172.21.15.89:0/7451541 >> 172.21.15.89:0/7451541 conn(0x7f7a980ce600 msgr2=0x7f7a98157190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:32.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.431+0000 7f7a9ce18700 1 -- 172.21.15.89:0/7451541 shutdown_connections 2023-10-10T20:43:32.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.431+0000 7f7a9ce18700 1 -- 172.21.15.89:0/7451541 wait complete. 2023-10-10T20:43:32.446 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:43:32.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.562+0000 7f61c8a8b700 1 Processor -- start 2023-10-10T20:43:32.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.562+0000 7f61c8a8b700 1 -- start start 2023-10-10T20:43:32.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.563+0000 7f61c8a8b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f61c40b16d0 0x7f61c40b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:32.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.563+0000 7f61c8a8b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61c40b33b0 0x7f61c40b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:32.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.563+0000 7f61c8a8b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c40b2080 con 0x7f61c40b4e70 2023-10-10T20:43:32.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.563+0000 7f61c8a8b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c40cfb00 con 0x7f61c40b16d0 2023-10-10T20:43:32.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.563+0000 7f61c8a8b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40b5c40 con 0x7f61c40b33b0 2023-10-10T20:43:32.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.563+0000 7f61c37fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f61c40b4e70 0x7f61c40ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35206/0 (socket says 172.21.15.89:35206) 2023-10-10T20:43:32.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.563+0000 7f61c37fe700 1 -- 172.21.15.89:0/81209871 learned_addr learned my addr 172.21.15.89:0/81209871 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:32.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.563+0000 7f61c2ffd700 1 --2- 172.21.15.89:0/81209871 >> v2:172.21.15.89:3300/0 conn(0x7f61c40b16d0 0x7f61c40b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:32.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.563+0000 7f61c3fff700 1 --2- 172.21.15.89:0/81209871 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61c40b33b0 0x7f61c40b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:32.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.563+0000 7f61c27fc700 1 -- 172.21.15.89:0/81209871 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924095123 0 0) 0x7f61c40b2080 con 0x7f61c40b4e70 2023-10-10T20:43:32.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.563+0000 7f61c27fc700 1 -- 172.21.15.89:0/81209871 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61ac003570 con 0x7f61c40b4e70 2023-10-10T20:43:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.563+0000 7f61c2ffd700 1 -- 172.21.15.89:0/81209871 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61c40b33b0 msgr2=0x7f61c40b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.563+0000 7f61c2ffd700 1 --2- 172.21.15.89:0/81209871 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61c40b33b0 0x7f61c40b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.563+0000 7f61c2ffd700 1 -- 172.21.15.89:0/81209871 >> v1:172.21.15.89:6789/0 conn(0x7f61c40b4e70 legacy=0x7f61c40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.563+0000 7f61c2ffd700 1 -- 172.21.15.89:0/81209871 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c40b5f20 con 0x7f61c40b16d0 2023-10-10T20:43:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.564+0000 7f61c2ffd700 1 --2- 172.21.15.89:0/81209871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61c40b16d0 0x7f61c40b1b40 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f61b80098e0 tx=0x7f61b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa23fc52b8893bb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.564+0000 7f61c27fc700 1 -- 172.21.15.89:0/81209871 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b800b640 con 0x7f61c40b16d0 2023-10-10T20:43:32.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.564+0000 7f61c27fc700 1 -- 172.21.15.89:0/81209871 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b800b7a0 con 0x7f61c40b16d0 2023-10-10T20:43:32.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.564+0000 7f61c27fc700 1 -- 172.21.15.89:0/81209871 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b800bb40 con 0x7f61c40b16d0 2023-10-10T20:43:32.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.564+0000 7f61c8a8b700 1 -- 172.21.15.89:0/81209871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61c40b16d0 msgr2=0x7f61c40b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:32.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.564+0000 7f61c8a8b700 1 --2- 172.21.15.89:0/81209871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61c40b16d0 0x7f61c40b1b40 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f61b80098e0 tx=0x7f61b80091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:32.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.564+0000 7f61c8a8b700 1 -- 172.21.15.89:0/81209871 shutdown_connections 2023-10-10T20:43:32.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.564+0000 7f61c8a8b700 1 --2- 172.21.15.89:0/81209871 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61c40b33b0 0x7f61c40b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:32.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.564+0000 7f61c8a8b700 1 --2- 172.21.15.89:0/81209871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61c40b16d0 0x7f61c40b1b40 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:32.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.564+0000 7f61c8a8b700 1 -- 172.21.15.89:0/81209871 >> 172.21.15.89:0/81209871 conn(0x7f61c40d10c0 msgr2=0x7f61c40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:32.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.564+0000 7f61c8a8b700 1 -- 172.21.15.89:0/81209871 shutdown_connections 2023-10-10T20:43:32.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.564+0000 7f61c8a8b700 1 -- 172.21.15.89:0/81209871 wait complete. 2023-10-10T20:43:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.566+0000 7f61c8a8b700 1 Processor -- start 2023-10-10T20:43:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.566+0000 7f61c8a8b700 1 -- start start 2023-10-10T20:43:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.566+0000 7f61c8a8b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61c40b16d0 0x7f61c40c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.566+0000 7f61c8a8b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61c40b33b0 0x7f61c41570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.566+0000 7f61c8a8b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61c40b4e70 0x7f61c40c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.566+0000 7f61c8a8b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40c5d20 con 0x7f61c40b33b0 2023-10-10T20:43:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.566+0000 7f61c8a8b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40cbc30 con 0x7f61c40b16d0 2023-10-10T20:43:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.566+0000 7f61c8a8b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40c5430 con 0x7f61c40b4e70 2023-10-10T20:43:32.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.566+0000 7f61c2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61c40b33b0 0x7f61c41570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:32.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.566+0000 7f61c2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61c40b33b0 0x7f61c41570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37296/0 (socket says 172.21.15.89:37296) 2023-10-10T20:43:32.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.566+0000 7f61c2ffd700 1 -- 172.21.15.89:0/4266549141 learned_addr learned my addr 172.21.15.89:0/4266549141 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:32.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.567+0000 7f61c3fff700 1 --2- 172.21.15.89:0/4266549141 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61c40b4e70 0x7f61c40c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:32.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.567+0000 7f61c37fe700 1 --2- 172.21.15.89:0/4266549141 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61c40b16d0 0x7f61c40c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:32.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.567+0000 7f61c2ffd700 1 -- 172.21.15.89:0/4266549141 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61c40b4e70 msgr2=0x7f61c40c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.567+0000 7f61c2ffd700 1 --2- 172.21.15.89:0/4266549141 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61c40b4e70 0x7f61c40c6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.567+0000 7f61c2ffd700 1 -- 172.21.15.89:0/4266549141 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61c40b16d0 msgr2=0x7f61c40c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.567+0000 7f61c2ffd700 1 --2- 172.21.15.89:0/4266549141 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61c40b16d0 0x7f61c40c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.567+0000 7f61c2ffd700 1 -- 172.21.15.89:0/4266549141 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b8009420 con 0x7f61c40b33b0 2023-10-10T20:43:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.567+0000 7f61c37fe700 1 --2- 172.21.15.89:0/4266549141 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61c40b16d0 0x7f61c40c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.567+0000 7f61c2ffd700 1 --2- 172.21.15.89:0/4266549141 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61c40b33b0 0x7f61c41570f0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f61b80093f0 tx=0x7f61b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.567+0000 7f61ab7fe700 1 -- 172.21.15.89:0/4266549141 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b801bb90 con 0x7f61c40b33b0 2023-10-10T20:43:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.567+0000 7f61ab7fe700 1 -- 172.21.15.89:0/4266549141 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b801bcf0 con 0x7f61c40b33b0 2023-10-10T20:43:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.567+0000 7f61ab7fe700 1 -- 172.21.15.89:0/4266549141 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b801bea0 con 0x7f61c40b33b0 2023-10-10T20:43:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.567+0000 7f61c8a8b700 1 -- 172.21.15.89:0/4266549141 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c40c56d0 con 0x7f61c40b33b0 2023-10-10T20:43:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.567+0000 7f61c8a8b700 1 -- 172.21.15.89:0/4266549141 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c40c66f0 con 0x7f61c40b33b0 2023-10-10T20:43:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.568+0000 7f61c8a8b700 1 -- 172.21.15.89:0/4266549141 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c40b4980 con 0x7f61c40b33b0 2023-10-10T20:43:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.568+0000 7f61ab7fe700 1 -- 172.21.15.89:0/4266549141 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f61b8023070 con 0x7f61c40b33b0 2023-10-10T20:43:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.569+0000 7f61ab7fe700 1 --2- 172.21.15.89:0/4266549141 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f61b0067320 0x7f61b00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.569+0000 7f61ab7fe700 1 -- 172.21.15.89:0/4266549141 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f61b8086020 con 0x7f61c40b33b0 2023-10-10T20:43:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.569+0000 7f61c37fe700 1 --2- 172.21.15.89:0/4266549141 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f61b0067320 0x7f61b00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.569+0000 7f61c37fe700 1 --2- 172.21.15.89:0/4266549141 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f61b0067320 0x7f61b00697d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f61b4007fd0 tx=0x7f61b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.573+0000 7f61ab7fe700 1 -- 172.21.15.89:0/4266549141 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61b8052cb0 con 0x7f61c40b33b0 2023-10-10T20:43:32.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:32.798+0000 7f61c8a8b700 1 -- 172.21.15.89:0/4266549141 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f61c40c5e60 con 0x7f61c40b33b0 2023-10-10T20:43:33.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.431+0000 7f61ab7fe700 1 -- 172.21.15.89:0/4266549141 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v121) v1 ==== 122+0+0 (secure 0 0 0) 0x7f61b80150c0 con 0x7f61c40b33b0 2023-10-10T20:43:33.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.433+0000 7f61c8a8b700 1 -- 172.21.15.89:0/4266549141 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f61b0067320 msgr2=0x7f61b00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:33.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.433+0000 7f61c8a8b700 1 --2- 172.21.15.89:0/4266549141 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f61b0067320 0x7f61b00697d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f61b4007fd0 tx=0x7f61b4009040 comp rx=0 tx=0).stop 2023-10-10T20:43:33.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.433+0000 7f61c8a8b700 1 -- 172.21.15.89:0/4266549141 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61c40b33b0 msgr2=0x7f61c41570f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:33.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.433+0000 7f61c8a8b700 1 --2- 172.21.15.89:0/4266549141 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61c40b33b0 0x7f61c41570f0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f61b80093f0 tx=0x7f61b8017000 comp rx=0 tx=0).stop 2023-10-10T20:43:33.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.433+0000 7f61c8a8b700 1 -- 172.21.15.89:0/4266549141 shutdown_connections 2023-10-10T20:43:33.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.433+0000 7f61c8a8b700 1 --2- 172.21.15.89:0/4266549141 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f61c40b4e70 0x7f61c40c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:33.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.433+0000 7f61c8a8b700 1 --2- 172.21.15.89:0/4266549141 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f61b0067320 0x7f61b00697d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:33.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.433+0000 7f61c8a8b700 1 --2- 172.21.15.89:0/4266549141 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61c40b16d0 0x7f61c40c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:33.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.433+0000 7f61c8a8b700 1 --2- 172.21.15.89:0/4266549141 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61c40b33b0 0x7f61c41570f0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:33.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.433+0000 7f61c8a8b700 1 -- 172.21.15.89:0/4266549141 >> 172.21.15.89:0/4266549141 conn(0x7f61c40d10c0 msgr2=0x7f61c4069260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:33.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.433+0000 7f61c8a8b700 1 -- 172.21.15.89:0/4266549141 shutdown_connections 2023-10-10T20:43:33.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.433+0000 7f61c8a8b700 1 -- 172.21.15.89:0/4266549141 wait complete. 2023-10-10T20:43:33.449 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:43:33.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.564+0000 7f1f77156700 1 Processor -- start 2023-10-10T20:43:33.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.564+0000 7f1f77156700 1 -- start start 2023-10-10T20:43:33.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f77156700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f70159c60 0x7f1f7015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:33.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f77156700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7015a160 con 0x7f1f7015db10 2023-10-10T20:43:33.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f77156700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f700d13a0 con 0x7f1f70159890 2023-10-10T20:43:33.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f77156700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7015e140 con 0x7f1f70159c60 2023-10-10T20:43:33.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f756f3700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1f7015db10 0x7f1f70069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35216/0 (socket says 172.21.15.89:35216) 2023-10-10T20:43:33.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f756f3700 1 -- 172.21.15.89:0/3800743706 learned_addr learned my addr 172.21.15.89:0/3800743706 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:33.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f67fff700 1 --2- 172.21.15.89:0/3800743706 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f70159c60 0x7f1f7015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:33.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f677fe700 1 -- 172.21.15.89:0/3800743706 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292165138 0 0) 0x7f1f7015a160 con 0x7f1f7015db10 2023-10-10T20:43:33.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f677fe700 1 -- 172.21.15.89:0/3800743706 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f58003570 con 0x7f1f7015db10 2023-10-10T20:43:33.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f677fe700 1 -- 172.21.15.89:0/3800743706 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1607103767 0 0) 0x7f1f700d13a0 con 0x7f1f70159890 2023-10-10T20:43:33.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f677fe700 1 -- 172.21.15.89:0/3800743706 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f7015a160 con 0x7f1f70159890 2023-10-10T20:43:33.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f677fe700 1 -- 172.21.15.89:0/3800743706 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 398801734 0 0) 0x7f1f58003570 con 0x7f1f7015db10 2023-10-10T20:43:33.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f677fe700 1 -- 172.21.15.89:0/3800743706 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f70159c60 msgr2=0x7f1f7015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:33.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f677fe700 1 --2- 172.21.15.89:0/3800743706 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f70159c60 0x7f1f7015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:33.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f677fe700 1 -- 172.21.15.89:0/3800743706 >> v1:172.21.15.111:6789/0 conn(0x7f1f70159890 legacy=0x7f1f7015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:33.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f677fe700 1 -- 172.21.15.89:0/3800743706 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7015a3d0 con 0x7f1f7015db10 2023-10-10T20:43:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f677fe700 1 -- 172.21.15.89:0/3800743706 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f1f68001470 con 0x7f1f7015db10 2023-10-10T20:43:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f677fe700 1 -- 172.21.15.89:0/3800743706 >> v1:172.21.15.89:6789/0 conn(0x7f1f7015db10 legacy=0x7f1f70069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f677fe700 1 --2- 172.21.15.89:0/3800743706 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f70159c60 0x7f1f58003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f677fe700 1 --2- 172.21.15.89:0/3800743706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f58004180 0x7f1f58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f677fe700 1 --2- 172.21.15.89:0/3800743706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f58004d50 0x7f1f580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f677fe700 1 -- 172.21.15.89:0/3800743706 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7015e140 con 0x7f1f58004180 2023-10-10T20:43:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.565+0000 7f1f677fe700 1 -- 172.21.15.89:0/3800743706 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5800bba0 con 0x7f1f58004d50 2023-10-10T20:43:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.566+0000 7f1f677fe700 1 -- 172.21.15.89:0/3800743706 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5800bd60 con 0x7f1f70159c60 2023-10-10T20:43:33.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.566+0000 7f1f67fff700 1 --2- 172.21.15.89:0/3800743706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f58004180 0x7f1f58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:33.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.566+0000 7f1f74ef2700 1 --2- 172.21.15.89:0/3800743706 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f70159c60 0x7f1f58003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:33.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.566+0000 7f1f756f3700 1 --2- 172.21.15.89:0/3800743706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f58004d50 0x7f1f580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:33.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.566+0000 7f1f67fff700 1 -- 172.21.15.89:0/3800743706 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f70159c60 msgr2=0x7f1f58003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:33.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.566+0000 7f1f67fff700 1 --2- 172.21.15.89:0/3800743706 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f70159c60 0x7f1f58003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:33.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.566+0000 7f1f67fff700 1 -- 172.21.15.89:0/3800743706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f58004d50 msgr2=0x7f1f580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:33.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.566+0000 7f1f67fff700 1 --2- 172.21.15.89:0/3800743706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f58004d50 0x7f1f580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:33.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.566+0000 7f1f67fff700 1 -- 172.21.15.89:0/3800743706 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f58003570 con 0x7f1f58004180 2023-10-10T20:43:33.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.566+0000 7f1f756f3700 1 --2- 172.21.15.89:0/3800743706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f58004d50 0x7f1f580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:33.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.566+0000 7f1f67fff700 1 --2- 172.21.15.89:0/3800743706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f58004180 0x7f1f58004810 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f1f7015a090 tx=0x7f1f600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badc8ab4debd842f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.566+0000 7f1f677fe700 1 -- 172.21.15.89:0/3800743706 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6000e710 con 0x7f1f58004180 2023-10-10T20:43:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.566+0000 7f1f677fe700 1 -- 172.21.15.89:0/3800743706 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6000e870 con 0x7f1f58004180 2023-10-10T20:43:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.566+0000 7f1f677fe700 1 -- 172.21.15.89:0/3800743706 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6000ec30 con 0x7f1f58004180 2023-10-10T20:43:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.567+0000 7f1f77156700 1 -- 172.21.15.89:0/3800743706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f58004180 msgr2=0x7f1f58004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.567+0000 7f1f77156700 1 --2- 172.21.15.89:0/3800743706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f58004180 0x7f1f58004810 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f1f7015a090 tx=0x7f1f600091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.567+0000 7f1f756f3700 1 -- 172.21.15.89:0/3800743706 reap_dead start 2023-10-10T20:43:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.567+0000 7f1f77156700 1 -- 172.21.15.89:0/3800743706 shutdown_connections 2023-10-10T20:43:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.567+0000 7f1f77156700 1 -- 172.21.15.89:0/3800743706 >> 172.21.15.89:0/3800743706 conn(0x7f1f70068e20 msgr2=0x7f1f700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.567+0000 7f1f77156700 1 -- 172.21.15.89:0/3800743706 shutdown_connections 2023-10-10T20:43:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.567+0000 7f1f77156700 1 -- 172.21.15.89:0/3800743706 wait complete. 2023-10-10T20:43:33.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.568+0000 7f1f77156700 1 Processor -- start 2023-10-10T20:43:33.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.568+0000 7f1f77156700 1 -- start start 2023-10-10T20:43:33.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f77156700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f58004180 0x7f1f700b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:33.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f77156700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f700b4c90 0x7f1f700cf460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:33.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f77156700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f70146fa0 0x7f1f701473f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:33.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f77156700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f700c78a0 con 0x7f1f700b4c90 2023-10-10T20:43:33.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f77156700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f70147930 con 0x7f1f58004180 2023-10-10T20:43:33.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f77156700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f700b5450 con 0x7f1f70146fa0 2023-10-10T20:43:33.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f67fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f700b4c90 0x7f1f700cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:33.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f67fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f700b4c90 0x7f1f700cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37306/0 (socket says 172.21.15.89:37306) 2023-10-10T20:43:33.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f67fff700 1 -- 172.21.15.89:0/3142175512 learned_addr learned my addr 172.21.15.89:0/3142175512 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:33.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f74ef2700 1 --2- 172.21.15.89:0/3142175512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f58004180 0x7f1f700b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:33.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f756f3700 1 --2- 172.21.15.89:0/3142175512 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f70146fa0 0x7f1f701473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:33.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f67fff700 1 -- 172.21.15.89:0/3142175512 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f70146fa0 msgr2=0x7f1f701473f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:33.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f67fff700 1 --2- 172.21.15.89:0/3142175512 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f70146fa0 0x7f1f701473f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:33.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f67fff700 1 -- 172.21.15.89:0/3142175512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f58004180 msgr2=0x7f1f700b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:33.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f67fff700 1 --2- 172.21.15.89:0/3142175512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f58004180 0x7f1f700b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:33.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f67fff700 1 -- 172.21.15.89:0/3142175512 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f700b5730 con 0x7f1f700b4c90 2023-10-10T20:43:33.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f74ef2700 1 --2- 172.21.15.89:0/3142175512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f58004180 0x7f1f700b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:33.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f756f3700 1 --2- 172.21.15.89:0/3142175512 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f70146fa0 0x7f1f701473f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:33.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f67fff700 1 --2- 172.21.15.89:0/3142175512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f700b4c90 0x7f1f700cf460 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800bb70 tx=0x7f1f6001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:33.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f64ff9700 1 -- 172.21.15.89:0/3142175512 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f60022070 con 0x7f1f700b4c90 2023-10-10T20:43:33.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f64ff9700 1 -- 172.21.15.89:0/3142175512 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6000e9e0 con 0x7f1f700b4c90 2023-10-10T20:43:33.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f77156700 1 -- 172.21.15.89:0/3142175512 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f700b5a10 con 0x7f1f700b4c90 2023-10-10T20:43:33.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f64ff9700 1 -- 172.21.15.89:0/3142175512 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6001c450 con 0x7f1f700b4c90 2023-10-10T20:43:33.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.569+0000 7f1f77156700 1 -- 172.21.15.89:0/3142175512 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f700b5100 con 0x7f1f700b4c90 2023-10-10T20:43:33.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.570+0000 7f1f77156700 1 -- 172.21.15.89:0/3142175512 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f700cbd00 con 0x7f1f700b4c90 2023-10-10T20:43:33.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.571+0000 7f1f64ff9700 1 -- 172.21.15.89:0/3142175512 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1f6001c7c0 con 0x7f1f700b4c90 2023-10-10T20:43:33.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.571+0000 7f1f64ff9700 1 --2- 172.21.15.89:0/3142175512 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1f5c067320 0x7f1f5c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:33.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.571+0000 7f1f64ff9700 1 -- 172.21.15.89:0/3142175512 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1f60086360 con 0x7f1f700b4c90 2023-10-10T20:43:33.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.571+0000 7f1f74ef2700 1 --2- 172.21.15.89:0/3142175512 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1f5c067320 0x7f1f5c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:33.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.571+0000 7f1f74ef2700 1 --2- 172.21.15.89:0/3142175512 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1f5c067320 0x7f1f5c0697d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c000f80 tx=0x7f1f6c0017a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:33.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.575+0000 7f1f64ff9700 1 -- 172.21.15.89:0/3142175512 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f60052ff0 con 0x7f1f700b4c90 2023-10-10T20:43:33.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.803+0000 7f1f77156700 1 -- 172.21.15.89:0/3142175512 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f7004c3c0 con 0x7f1f700b4c90 2023-10-10T20:43:33.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.803+0000 7f1f64ff9700 1 -- 172.21.15.89:0/3142175512 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4610 (secure 0 0 0) 0x7f1f60056610 con 0x7f1f700b4c90 2023-10-10T20:43:33.806 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:33.806 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":121,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/455398787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":455398787},{"type":"v1","addr":"172.21.15.89:6836","nonce":455398787}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/2660411678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2660411678},{"type":"v1","addr":"172.21.15.111:6834","nonce":2660411678}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6890,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/1650071647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":1650071647},{"type":"v1","addr":"172.21.15.111:6835","nonce":1650071647}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:43:29.795107+0000","modified":"2023-10-10T20:43:33.428614+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8760":{"gid":8760,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/2168500629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2168500629},{"type":"v1","addr":"172.21.15.89:6837","nonce":2168500629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-10T20:43:33.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.805+0000 7f1f77156700 1 -- 172.21.15.89:0/3142175512 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1f5c067320 msgr2=0x7f1f5c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:33.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.805+0000 7f1f77156700 1 --2- 172.21.15.89:0/3142175512 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1f5c067320 0x7f1f5c0697d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c000f80 tx=0x7f1f6c0017a0 comp rx=0 tx=0).stop 2023-10-10T20:43:33.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.805+0000 7f1f77156700 1 -- 172.21.15.89:0/3142175512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f700b4c90 msgr2=0x7f1f700cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:33.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.805+0000 7f1f77156700 1 --2- 172.21.15.89:0/3142175512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f700b4c90 0x7f1f700cf460 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800bb70 tx=0x7f1f6001a000 comp rx=0 tx=0).stop 2023-10-10T20:43:33.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.806+0000 7f1f77156700 1 -- 172.21.15.89:0/3142175512 shutdown_connections 2023-10-10T20:43:33.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.806+0000 7f1f77156700 1 --2- 172.21.15.89:0/3142175512 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1f70146fa0 0x7f1f701473f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:33.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.806+0000 7f1f77156700 1 --2- 172.21.15.89:0/3142175512 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1f5c067320 0x7f1f5c0697d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:33.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.806+0000 7f1f77156700 1 --2- 172.21.15.89:0/3142175512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f58004180 0x7f1f700b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:33.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.806+0000 7f1f77156700 1 --2- 172.21.15.89:0/3142175512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f700b4c90 0x7f1f700cf460 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:33.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.806+0000 7f1f77156700 1 -- 172.21.15.89:0/3142175512 >> 172.21.15.89:0/3142175512 conn(0x7f1f70068e20 msgr2=0x7f1f700b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:33.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.806+0000 7f1f77156700 1 -- 172.21.15.89:0/3142175512 shutdown_connections 2023-10-10T20:43:33.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.806+0000 7f1f77156700 1 -- 172.21.15.89:0/3142175512 wait complete. 2023-10-10T20:43:33.810 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 121 2023-10-10T20:43:33.819 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:43:33.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.932+0000 7f25379c7700 1 Processor -- start 2023-10-10T20:43:33.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.932+0000 7f25379c7700 1 -- start start 2023-10-10T20:43:33.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.933+0000 7f25379c7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2530159890 0x7f253015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:33.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.933+0000 7f25379c7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253015a160 con 0x7f2530069560 2023-10-10T20:43:33.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.933+0000 7f25379c7700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25300d13a0 con 0x7f2530159c60 2023-10-10T20:43:33.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.933+0000 7f25379c7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253015e140 con 0x7f2530159890 2023-10-10T20:43:33.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.933+0000 7f2535763700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2530159890 0x7f253015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:33.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.933+0000 7f2535f64700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2530069560 0x7f25300ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35228/0 (socket says 172.21.15.89:35228) 2023-10-10T20:43:33.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.933+0000 7f2535f64700 1 -- 172.21.15.89:0/746986988 learned_addr learned my addr 172.21.15.89:0/746986988 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:33.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.933+0000 7f2535763700 1 --2- 172.21.15.89:0/746986988 >> v2:172.21.15.89:3300/0 conn(0x7f2530159890 0x7f253015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37316/0 (socket says 172.21.15.89:37316) 2023-10-10T20:43:33.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.934+0000 7f2535763700 1 -- 172.21.15.89:0/746986988 >> v1:172.21.15.111:6789/0 conn(0x7f2530159c60 legacy=0x7f253015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:33.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.934+0000 7f2535763700 1 -- 172.21.15.89:0/746986988 >> v1:172.21.15.89:6789/0 conn(0x7f2530069560 legacy=0x7f25300ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:33.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.934+0000 7f2527fff700 1 -- 172.21.15.89:0/746986988 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006583049 0 0) 0x7f253015a160 con 0x7f2530069560 2023-10-10T20:43:33.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.934+0000 7f2535763700 1 -- 172.21.15.89:0/746986988 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253015a3d0 con 0x7f2530159890 2023-10-10T20:43:33.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.934+0000 7f2535763700 1 --2- 172.21.15.89:0/746986988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2530159890 0x7f253015abd0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f252c0098e0 tx=0x7f252c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0f46618b714f80 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:33.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.934+0000 7f2527fff700 1 -- 172.21.15.89:0/746986988 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c009e80 con 0x7f2530159890 2023-10-10T20:43:33.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.934+0000 7f2527fff700 1 -- 172.21.15.89:0/746986988 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252c00b820 con 0x7f2530159890 2023-10-10T20:43:33.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.934+0000 7f2527fff700 1 -- 172.21.15.89:0/746986988 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c00bba0 con 0x7f2530159890 2023-10-10T20:43:33.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.934+0000 7f25379c7700 1 -- 172.21.15.89:0/746986988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2530159890 msgr2=0x7f253015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:33.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.934+0000 7f25379c7700 1 --2- 172.21.15.89:0/746986988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2530159890 0x7f253015abd0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f252c0098e0 tx=0x7f252c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:33.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.934+0000 7f25379c7700 1 -- 172.21.15.89:0/746986988 shutdown_connections 2023-10-10T20:43:33.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.934+0000 7f25379c7700 1 --2- 172.21.15.89:0/746986988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2530159890 0x7f253015abd0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:33.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.934+0000 7f25379c7700 1 -- 172.21.15.89:0/746986988 >> 172.21.15.89:0/746986988 conn(0x7f2530068e20 msgr2=0x7f25300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:33.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.934+0000 7f25379c7700 1 -- 172.21.15.89:0/746986988 shutdown_connections 2023-10-10T20:43:33.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.935+0000 7f25379c7700 1 -- 172.21.15.89:0/746986988 wait complete. 2023-10-10T20:43:33.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.936+0000 7f25379c7700 1 Processor -- start 2023-10-10T20:43:33.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.936+0000 7f25379c7700 1 -- start start 2023-10-10T20:43:33.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.936+0000 7f25379c7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2530069560 0x7f253015ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:33.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.936+0000 7f25379c7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2530159890 0x7f25300ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:33.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.936+0000 7f25379c7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2530159c60 0x7f253016dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:33.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.936+0000 7f25379c7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25300ca750 con 0x7f2530159890 2023-10-10T20:43:33.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.936+0000 7f25379c7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253016e2b0 con 0x7f2530069560 2023-10-10T20:43:33.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.936+0000 7f25379c7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253015c030 con 0x7f2530159c60 2023-10-10T20:43:33.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.936+0000 7f2534f62700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2530159890 0x7f25300ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:33.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.937+0000 7f2534f62700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2530159890 0x7f25300ca210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37320/0 (socket says 172.21.15.89:37320) 2023-10-10T20:43:33.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.937+0000 7f2534f62700 1 -- 172.21.15.89:0/1681466336 learned_addr learned my addr 172.21.15.89:0/1681466336 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:33.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.937+0000 7f2535763700 1 --2- 172.21.15.89:0/1681466336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2530069560 0x7f253015ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:33.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.937+0000 7f2535f64700 1 --2- 172.21.15.89:0/1681466336 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2530159c60 0x7f253016dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:33.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.937+0000 7f2534f62700 1 -- 172.21.15.89:0/1681466336 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2530159c60 msgr2=0x7f253016dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:33.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.937+0000 7f2534f62700 1 --2- 172.21.15.89:0/1681466336 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2530159c60 0x7f253016dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:33.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.937+0000 7f2534f62700 1 -- 172.21.15.89:0/1681466336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2530069560 msgr2=0x7f253015ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:33.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.937+0000 7f2534f62700 1 --2- 172.21.15.89:0/1681466336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2530069560 0x7f253015ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:33.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.937+0000 7f2534f62700 1 -- 172.21.15.89:0/1681466336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c009420 con 0x7f2530159890 2023-10-10T20:43:33.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.937+0000 7f2535f64700 1 --2- 172.21.15.89:0/1681466336 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2530159c60 0x7f253016dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:33.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.937+0000 7f2534f62700 1 --2- 172.21.15.89:0/1681466336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2530159890 0x7f25300ca210 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f252000c5d0 tx=0x7f2520007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:33.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.937+0000 7f25257fa700 1 -- 172.21.15.89:0/1681466336 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2520011590 con 0x7f2530159890 2023-10-10T20:43:33.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.937+0000 7f25257fa700 1 -- 172.21.15.89:0/1681466336 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2520011710 con 0x7f2530159890 2023-10-10T20:43:33.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.937+0000 7f25379c7700 1 -- 172.21.15.89:0/1681466336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253015c310 con 0x7f2530159890 2023-10-10T20:43:33.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.937+0000 7f25257fa700 1 -- 172.21.15.89:0/1681466336 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2520011ad0 con 0x7f2530159890 2023-10-10T20:43:33.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.937+0000 7f25379c7700 1 -- 172.21.15.89:0/1681466336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253015c8d0 con 0x7f2530159890 2023-10-10T20:43:33.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.938+0000 7f25379c7700 1 -- 172.21.15.89:0/1681466336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253004c3c0 con 0x7f2530159890 2023-10-10T20:43:33.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.938+0000 7f25257fa700 1 -- 172.21.15.89:0/1681466336 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f252001c460 con 0x7f2530159890 2023-10-10T20:43:33.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.939+0000 7f25257fa700 1 --2- 172.21.15.89:0/1681466336 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2514067320 0x7f25140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:33.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.939+0000 7f25257fa700 1 -- 172.21.15.89:0/1681466336 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2520084a70 con 0x7f2530159890 2023-10-10T20:43:33.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.939+0000 7f2535763700 1 --2- 172.21.15.89:0/1681466336 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2514067320 0x7f25140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:33.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.939+0000 7f2535763700 1 --2- 172.21.15.89:0/1681466336 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2514067320 0x7f25140697d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f252c0030c0 tx=0x7f252c016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:33.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:33.943+0000 7f25257fa700 1 -- 172.21.15.89:0/1681466336 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2520051700 con 0x7f2530159890 2023-10-10T20:43:34.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.127+0000 7f25379c7700 1 -- 172.21.15.89:0/1681466336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2530069ad0 con 0x7f2530159890 2023-10-10T20:43:34.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.128+0000 7f25257fa700 1 -- 172.21.15.89:0/1681466336 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+15364 (secure 0 0 0) 0x7f2520054d20 con 0x7f2530159890 2023-10-10T20:43:34.130 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:34.130 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":183,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:43:31.416663+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:43:28.133902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-10T20:43:28.749600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:43:34.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.130+0000 7f25379c7700 1 -- 172.21.15.89:0/1681466336 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2514067320 msgr2=0x7f25140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:34.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.130+0000 7f25379c7700 1 --2- 172.21.15.89:0/1681466336 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2514067320 0x7f25140697d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f252c0030c0 tx=0x7f252c016000 comp rx=0 tx=0).stop 2023-10-10T20:43:34.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.130+0000 7f25379c7700 1 -- 172.21.15.89:0/1681466336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2530159890 msgr2=0x7f25300ca210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:34.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.130+0000 7f25379c7700 1 --2- 172.21.15.89:0/1681466336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2530159890 0x7f25300ca210 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f252000c5d0 tx=0x7f2520007f60 comp rx=0 tx=0).stop 2023-10-10T20:43:34.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.130+0000 7f25379c7700 1 -- 172.21.15.89:0/1681466336 shutdown_connections 2023-10-10T20:43:34.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.130+0000 7f25379c7700 1 --2- 172.21.15.89:0/1681466336 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f2530159c60 0x7f253016dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.130+0000 7f25379c7700 1 --2- 172.21.15.89:0/1681466336 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2514067320 0x7f25140697d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.130+0000 7f25379c7700 1 --2- 172.21.15.89:0/1681466336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2530069560 0x7f253015ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.130+0000 7f25379c7700 1 --2- 172.21.15.89:0/1681466336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2530159890 0x7f25300ca210 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.130+0000 7f25379c7700 1 -- 172.21.15.89:0/1681466336 >> 172.21.15.89:0/1681466336 conn(0x7f2530068e20 msgr2=0x7f25300b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:34.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.130+0000 7f25379c7700 1 -- 172.21.15.89:0/1681466336 shutdown_connections 2023-10-10T20:43:34.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.130+0000 7f25379c7700 1 -- 172.21.15.89:0/1681466336 wait complete. 2023-10-10T20:43:34.144 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:43:34.144 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:43:34.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.257+0000 7f1e1a2cc700 1 Processor -- start 2023-10-10T20:43:34.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.257+0000 7f1e1a2cc700 1 -- start start 2023-10-10T20:43:34.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.258+0000 7f1e1a2cc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e14159890 0x7f1e1415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:34.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.258+0000 7f1e1a2cc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1415a160 con 0x7f1e14069560 2023-10-10T20:43:34.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.258+0000 7f1e1a2cc700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e140d13a0 con 0x7f1e14159c60 2023-10-10T20:43:34.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.258+0000 7f1e1a2cc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1415e140 con 0x7f1e14159890 2023-10-10T20:43:34.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.258+0000 7f1e13fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e14159890 0x7f1e1415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:34.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.258+0000 7f1e18869700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1e14069560 0x7f1e140ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35230/0 (socket says 172.21.15.89:35230) 2023-10-10T20:43:34.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.258+0000 7f1e18869700 1 -- 172.21.15.89:0/3643470726 learned_addr learned my addr 172.21.15.89:0/3643470726 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:34.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.258+0000 7f1e13fff700 1 -- 172.21.15.89:0/3643470726 >> v1:172.21.15.111:6789/0 conn(0x7f1e14159c60 legacy=0x7f1e1415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:34.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.258+0000 7f1e13fff700 1 -- 172.21.15.89:0/3643470726 >> v1:172.21.15.89:6789/0 conn(0x7f1e14069560 legacy=0x7f1e140ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:34.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.258+0000 7f1e12ffd700 1 -- 172.21.15.89:0/3643470726 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3472226773 0 0) 0x7f1e1415a160 con 0x7f1e14069560 2023-10-10T20:43:34.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.258+0000 7f1e13fff700 1 -- 172.21.15.89:0/3643470726 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1415a3d0 con 0x7f1e14159890 2023-10-10T20:43:34.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.259+0000 7f1e13fff700 1 --2- 172.21.15.89:0/3643470726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e14159890 0x7f1e1415abd0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f1e040098e0 tx=0x7f1e040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff5de26cdc2fe5e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:34.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.259+0000 7f1e12ffd700 1 -- 172.21.15.89:0/3643470726 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e04009e80 con 0x7f1e14159890 2023-10-10T20:43:34.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.259+0000 7f1e12ffd700 1 -- 172.21.15.89:0/3643470726 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0400b820 con 0x7f1e14159890 2023-10-10T20:43:34.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.259+0000 7f1e12ffd700 1 -- 172.21.15.89:0/3643470726 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0400bba0 con 0x7f1e14159890 2023-10-10T20:43:34.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.259+0000 7f1e1a2cc700 1 -- 172.21.15.89:0/3643470726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e14159890 msgr2=0x7f1e1415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.259+0000 7f1e1a2cc700 1 --2- 172.21.15.89:0/3643470726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e14159890 0x7f1e1415abd0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f1e040098e0 tx=0x7f1e040091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.259+0000 7f1e1a2cc700 1 -- 172.21.15.89:0/3643470726 shutdown_connections 2023-10-10T20:43:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.259+0000 7f1e1a2cc700 1 --2- 172.21.15.89:0/3643470726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e14159890 0x7f1e1415abd0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.259+0000 7f1e1a2cc700 1 -- 172.21.15.89:0/3643470726 >> 172.21.15.89:0/3643470726 conn(0x7f1e14068e20 msgr2=0x7f1e140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.259+0000 7f1e1a2cc700 1 -- 172.21.15.89:0/3643470726 shutdown_connections 2023-10-10T20:43:34.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.259+0000 7f1e1a2cc700 1 -- 172.21.15.89:0/3643470726 wait complete. 2023-10-10T20:43:34.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.260+0000 7f1e1a2cc700 1 Processor -- start 2023-10-10T20:43:34.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.261+0000 7f1e1a2cc700 1 -- start start 2023-10-10T20:43:34.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.261+0000 7f1e1a2cc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e14069560 0x7f1e1415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:34.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.261+0000 7f1e1a2cc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e14159890 0x7f1e140ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:34.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.261+0000 7f1e1a2cc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1e14159c60 0x7f1e1416dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:34.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.261+0000 7f1e1a2cc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e140ca750 con 0x7f1e14069560 2023-10-10T20:43:34.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.261+0000 7f1e1a2cc700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1416e2b0 con 0x7f1e14159890 2023-10-10T20:43:34.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.261+0000 7f1e1a2cc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1415c030 con 0x7f1e14159c60 2023-10-10T20:43:34.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.261+0000 7f1e13fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e14069560 0x7f1e1415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:34.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.261+0000 7f1e13fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e14069560 0x7f1e1415ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37326/0 (socket says 172.21.15.89:37326) 2023-10-10T20:43:34.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.261+0000 7f1e13fff700 1 -- 172.21.15.89:0/626253457 learned_addr learned my addr 172.21.15.89:0/626253457 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:34.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.262+0000 7f1e137fe700 1 --2- 172.21.15.89:0/626253457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e14159890 0x7f1e140ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:34.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.262+0000 7f1e18869700 1 --2- 172.21.15.89:0/626253457 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1e14159c60 0x7f1e1416dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:34.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.262+0000 7f1e13fff700 1 -- 172.21.15.89:0/626253457 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1e14159c60 msgr2=0x7f1e1416dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:34.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.262+0000 7f1e13fff700 1 --2- 172.21.15.89:0/626253457 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1e14159c60 0x7f1e1416dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.262+0000 7f1e13fff700 1 -- 172.21.15.89:0/626253457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e14159890 msgr2=0x7f1e140ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:34.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.262+0000 7f1e13fff700 1 --2- 172.21.15.89:0/626253457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e14159890 0x7f1e140ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.262+0000 7f1e13fff700 1 -- 172.21.15.89:0/626253457 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e04009420 con 0x7f1e14069560 2023-10-10T20:43:34.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.262+0000 7f1e137fe700 1 --2- 172.21.15.89:0/626253457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e14159890 0x7f1e140ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:34.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.262+0000 7f1e18869700 1 --2- 172.21.15.89:0/626253457 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1e14159c60 0x7f1e1416dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:34.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.262+0000 7f1e13fff700 1 --2- 172.21.15.89:0/626253457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e14069560 0x7f1e1415ba50 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f1e040030c0 tx=0x7f1e04016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:34.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.262+0000 7f1dfbfff700 1 -- 172.21.15.89:0/626253457 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0400bae0 con 0x7f1e14069560 2023-10-10T20:43:34.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.262+0000 7f1dfbfff700 1 -- 172.21.15.89:0/626253457 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e04020070 con 0x7f1e14069560 2023-10-10T20:43:34.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.262+0000 7f1e1a2cc700 1 -- 172.21.15.89:0/626253457 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1415c310 con 0x7f1e14069560 2023-10-10T20:43:34.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.262+0000 7f1dfbfff700 1 -- 172.21.15.89:0/626253457 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0401b6a0 con 0x7f1e14069560 2023-10-10T20:43:34.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.262+0000 7f1e1a2cc700 1 -- 172.21.15.89:0/626253457 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1415c8d0 con 0x7f1e14069560 2023-10-10T20:43:34.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.263+0000 7f1e1a2cc700 1 -- 172.21.15.89:0/626253457 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1404c3c0 con 0x7f1e14069560 2023-10-10T20:43:34.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.263+0000 7f1dfbfff700 1 -- 172.21.15.89:0/626253457 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1e04022030 con 0x7f1e14069560 2023-10-10T20:43:34.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.264+0000 7f1dfbfff700 1 --2- 172.21.15.89:0/626253457 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1e00067320 0x7f1e000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:34.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.264+0000 7f1e137fe700 1 --2- 172.21.15.89:0/626253457 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1e00067320 0x7f1e000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:34.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.264+0000 7f1dfbfff700 1 -- 172.21.15.89:0/626253457 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1e04086c10 con 0x7f1e14069560 2023-10-10T20:43:34.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.264+0000 7f1e137fe700 1 --2- 172.21.15.89:0/626253457 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1e00067320 0x7f1e000697d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1e140d1030 tx=0x7f1e08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:34.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.268+0000 7f1dfbfff700 1 -- 172.21.15.89:0/626253457 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e04053970 con 0x7f1e14069560 2023-10-10T20:43:34.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.446+0000 7f1e1a2cc700 1 -- 172.21.15.89:0/626253457 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e1404c3c0 con 0x7f1e00067320 2023-10-10T20:43:34.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.454+0000 7f1dfbfff700 1 -- 172.21.15.89:0/626253457 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274793 (secure 0 0 0) 0x7f1e1404c3c0 con 0x7f1e00067320 2023-10-10T20:43:34.459 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:34.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.457+0000 7f1e1a2cc700 1 -- 172.21.15.89:0/626253457 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1e00067320 msgr2=0x7f1e000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:34.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.457+0000 7f1e1a2cc700 1 --2- 172.21.15.89:0/626253457 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1e00067320 0x7f1e000697d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1e140d1030 tx=0x7f1e08009040 comp rx=0 tx=0).stop 2023-10-10T20:43:34.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.457+0000 7f1e1a2cc700 1 -- 172.21.15.89:0/626253457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e14069560 msgr2=0x7f1e1415ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:34.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.457+0000 7f1e1a2cc700 1 --2- 172.21.15.89:0/626253457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e14069560 0x7f1e1415ba50 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f1e040030c0 tx=0x7f1e04016000 comp rx=0 tx=0).stop 2023-10-10T20:43:34.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.457+0000 7f1e1a2cc700 1 -- 172.21.15.89:0/626253457 shutdown_connections 2023-10-10T20:43:34.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.457+0000 7f1e1a2cc700 1 --2- 172.21.15.89:0/626253457 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1e14159c60 0x7f1e1416dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.457+0000 7f1e1a2cc700 1 --2- 172.21.15.89:0/626253457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e14069560 0x7f1e1415ba50 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.457+0000 7f1e1a2cc700 1 --2- 172.21.15.89:0/626253457 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1e00067320 0x7f1e000697d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.457+0000 7f1e1a2cc700 1 --2- 172.21.15.89:0/626253457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e14159890 0x7f1e140ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.457+0000 7f1e1a2cc700 1 -- 172.21.15.89:0/626253457 >> 172.21.15.89:0/626253457 conn(0x7f1e14068e20 msgr2=0x7f1e140b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:34.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.457+0000 7f1e1a2cc700 1 -- 172.21.15.89:0/626253457 shutdown_connections 2023-10-10T20:43:34.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.457+0000 7f1e1a2cc700 1 -- 172.21.15.89:0/626253457 wait complete. 2023-10-10T20:43:34.463 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:43:34.470 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2023-10-10T20:43:32.758274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2430020,"kb_used_data":18948,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752544700,"statfs":{"total":773094113280,"available":770605772800,"internally_reserved":0,"allocated":19402752,"data_stored":9347527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.360488"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:29.395425+0000","last_change":"2023-10-10T20:43:29.395425+0000","last_active":"2023-10-10T20:43:29.395425+0000","last_peered":"2023-10-10T20:43:29.395425+0000","last_clean":"2023-10-10T20:43:29.395425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:29.395425+0000","last_undegraded":"2023-10-10T20:43:29.395425+0000","last_fullsized":"2023-10-10T20:43:29.395425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:28.387999+0000","last_change":"2023-10-10T20:43:28.387999+0000","last_active":"2023-10-10T20:43:28.387999+0000","last_peered":"2023-10-10T20:43:28.387999+0000","last_clean":"2023-10-10T20:43:28.387999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:28.387999+0000","last_undegraded":"2023-10-10T20:43:28.387999+0000","last_fullsized":"2023-10-10T20:43:28.387999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":322,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-10T20:43:25.383101+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:43:25.383101+0000","last_peered":"2023-10-10T20:43:25.383101+0000","last_clean":"2023-10-10T20:43:25.383101+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:43:25.383101+0000","last_undegraded":"2023-10-10T20:43:25.383101+0000","last_fullsized":"2023-10-10T20:43:25.383101+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574960,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304188,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067652,"statfs":{"total":96636764160,"available":96325275648,"internally_reserved":0,"allocated":2871296,"data_stored":1611337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50700000000000001,"15min":0.75800000000000001},"min":{"1min":0.32500000000000001,"5min":0.309,"15min":0.29199999999999998},"max":{"1min":0.64300000000000002,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.50800000000000001,"15min":0.60399999999999998},"min":{"1min":0.36399999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.58299999999999996,"5min":0.752,"15min":4.1959999999999997},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.50600000000000001,"15min":0.90100000000000002},"min":{"1min":0.372,"5min":0.317,"15min":0.317},"max":{"1min":0.61099999999999999,"5min":0.78800000000000003,"15min":10.1},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.53400000000000003,"15min":0.69099999999999995},"min":{"1min":0.35199999999999998,"5min":0.33900000000000002,"15min":0.28999999999999998},"max":{"1min":0.66500000000000004,"5min":0.81799999999999995,"15min":6.806},"last":0.59899999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56799999999999995,"15min":0.74099999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68799999999999994,"5min":0.89000000000000001,"15min":6.9340000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.69399999999999995},"min":{"1min":0.40699999999999997,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.69099999999999995,"5min":0.77100000000000002,"15min":6.9870000000000001},"last":0.64500000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54200000000000004,"15min":0.71299999999999997},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64500000000000002,"5min":0.754,"15min":6.6719999999999997},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55900000000000005,"15min":0.64800000000000002},"min":{"1min":0.39800000000000002,"5min":0.378,"15min":0.34799999999999998},"max":{"1min":0.64400000000000002,"5min":0.76200000000000001,"15min":4.1609999999999996},"last":0.53000000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47799999999999998,"15min":0.46800000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.77200000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.51600000000000001,"15min":0.51900000000000002},"min":{"1min":0.314,"5min":0.309,"15min":0.30399999999999999},"max":{"1min":0.68000000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.629}]},{"osd":5,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.54000000000000004,"15min":0.53900000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.68400000000000005,"5min":0.86099999999999999,"15min":1.3180000000000001},"last":0.495},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.53600000000000003,"15min":0.52700000000000002},"min":{"1min":0.35299999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.84099999999999997},"last":0.50900000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.35699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":1.7509999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76300000000000001,"5min":0.84099999999999997,"15min":1.7090000000000001},"last":0.70699999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303608,"kb_used_data":2224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068232,"statfs":{"total":96636764160,"available":96325869568,"internally_reserved":0,"allocated":2277376,"data_stored":1020969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52400000000000002,"15min":0.64000000000000001},"min":{"1min":0.34699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":7.7590000000000003},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52400000000000002,"15min":0.67400000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.86899999999999999,"5min":1.075,"15min":7.8090000000000002},"last":0.79700000000000004}]},{"osd":1,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52900000000000003,"15min":0.67200000000000004},"min":{"1min":0.32800000000000001,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":7.867},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55500000000000005,"15min":0.81999999999999995},"min":{"1min":0.378,"5min":0.35499999999999998,"15min":0.33700000000000002},"max":{"1min":0.70999999999999996,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.85999999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52300000000000002,"15min":0.58099999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.745,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57099999999999995,"15min":0.79100000000000004},"min":{"1min":0.39700000000000002,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":2.641,"15min":6.7629999999999999},"last":0.875}]},{"osd":3,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55500000000000005,"15min":0.60699999999999998},"min":{"1min":0.372,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94399999999999995,"5min":1.107,"15min":4.8620000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55500000000000005,"15min":0.68400000000000005},"min":{"1min":0.41399999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.72599999999999998,"5min":1.718,"15min":7.9009999999999998},"last":0.63600000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51900000000000002,"15min":0.47899999999999998},"min":{"1min":0.32000000000000001,"5min":0.28999999999999998,"15min":0.28499999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":0.82899999999999996,"5min":1.032,"15min":1.032},"last":0.61699999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53400000000000003,"15min":0.5},"min":{"1min":0.34999999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.86799999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56999999999999995,"15min":0.53700000000000003},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85399999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.54400000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56200000000000006,"15min":0.53000000000000003},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.89600000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.59399999999999997,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.372,"15min":0.35099999999999998},"max":{"1min":0.82199999999999995,"5min":1.681,"15min":1.681},"last":0.58599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303608,"kb_used_data":2224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068232,"statfs":{"total":96636764160,"available":96325869568,"internally_reserved":0,"allocated":2277376,"data_stored":1020969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.31900000000000001,"15min":0.443},"min":{"1min":0.215,"5min":0.19600000000000001,"15min":0.17899999999999999},"max":{"1min":0.47799999999999998,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.253},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33000000000000002,"15min":0.35599999999999998},"min":{"1min":0.22500000000000001,"5min":0.224,"15min":0.20899999999999999},"max":{"1min":0.441,"5min":0.65500000000000003,"15min":1.194},"last":0.42899999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.32700000000000001,"15min":0.35499999999999998},"min":{"1min":0.246,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.41699999999999998,"5min":0.51400000000000001,"15min":1.2869999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.35199999999999998,"15min":0.47299999999999998},"min":{"1min":0.23999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.49199999999999999,"5min":0.54200000000000004,"15min":5.0919999999999996},"last":0.316}]},{"osd":3,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.374,"15min":0.496},"min":{"1min":0.25600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54800000000000004,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.36299999999999999,"15min":0.39800000000000002},"min":{"1min":0.26500000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.52600000000000002,"5min":0.57799999999999996,"15min":1.2170000000000001},"last":0.45900000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.78600000000000003,"5min":0.92400000000000004,"15min":1.254},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55900000000000005,"15min":0.67500000000000004},"min":{"1min":0.39600000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68100000000000005,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.627}]},{"osd":5,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.40400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.97599999999999998,"15min":1.1799999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56999999999999995,"15min":0.59999999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73299999999999998,"5min":0.878,"15min":1.3109999999999999},"last":0.64900000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":0.79300000000000004},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70399999999999996,"5min":0.995,"15min":5.0659999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60699999999999998,"15min":0.628},"min":{"1min":0.46400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84199999999999997,"5min":0.89800000000000002,"15min":1.171},"last":0.59199999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.48299999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77100000000000002,"5min":1.004,"15min":1.214},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58399999999999996,"15min":0.69699999999999995},"min":{"1min":0.40899999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.754,"5min":0.90500000000000003,"15min":4.0499999999999998},"last":0.52500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574959,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304184,"kb_used_data":2800,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067656,"statfs":{"total":96636764160,"available":96325279744,"internally_reserved":0,"allocated":2867200,"data_stored":1610376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.315,"15min":0.48599999999999999},"min":{"1min":0.217,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":7.9249999999999998},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.46899999999999997},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46100000000000002,"5min":0.53800000000000003,"15min":6.6299999999999999},"last":0.26500000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33900000000000002,"15min":0.51000000000000001},"min":{"1min":0.25600000000000001,"5min":0.22,"15min":0.216},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":8.2509999999999994},"last":0.25},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.49299999999999999},"min":{"1min":0.27500000000000002,"5min":0.223,"15min":0.222},"max":{"1min":0.498,"5min":0.57899999999999996,"15min":6.7370000000000001},"last":0.32700000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.35499999999999998,"15min":0.51700000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.21099999999999999},"max":{"1min":0.438,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.57099999999999995},"min":{"1min":0.317,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":7.8929999999999998},"last":0.51500000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57099999999999995,"15min":0.68700000000000006},"min":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.78100000000000003,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57199999999999995,"15min":0.73099999999999998},"min":{"1min":0.371,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.69599999999999995,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.443}]},{"osd":5,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56499999999999995,"15min":0.71399999999999997},"min":{"1min":0.40100000000000002,"5min":0.376,"15min":0.36299999999999999},"max":{"1min":0.72699999999999998,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.74199999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.55200000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61499999999999999,"15min":0.77600000000000002},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.73299999999999998},"min":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.36599999999999999},"max":{"1min":0.82099999999999995,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.48199999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60899999999999999,"15min":0.72399999999999998},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.36299999999999999},"max":{"1min":0.85499999999999998,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59799999999999998,"15min":0.747},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89500000000000002,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.49099999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303608,"kb_used_data":2224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068232,"statfs":{"total":96636764160,"available":96325869568,"internally_reserved":0,"allocated":2277376,"data_stored":1020969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.311,"15min":0.32400000000000001},"min":{"1min":0.20200000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.48599999999999999,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32100000000000001,"15min":0.34899999999999998},"min":{"1min":0.224,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51800000000000002,"5min":0.54200000000000004,"15min":0.746},"last":0.29599999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33200000000000002,"15min":0.65200000000000002},"min":{"1min":0.22600000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52200000000000002,"5min":0.63,"15min":11.967000000000001},"last":0.309},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.35199999999999998,"15min":0.45800000000000002},"min":{"1min":0.245,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.505,"5min":0.52700000000000002,"15min":4.1100000000000003},"last":0.32700000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.51600000000000001},"min":{"1min":0.23699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":6.3040000000000003},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.38700000000000001,"15min":0.70399999999999996},"min":{"1min":0.29499999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57799999999999996,"5min":0.65900000000000003,"15min":11.935},"last":0.37}]},{"osd":4,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.42999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91900000000000004,"5min":1.302,"15min":1.302},"last":0.432}]},{"osd":5,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.378},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.95699999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.39700000000000002,"5min":0.35999999999999999,"15min":0.34300000000000003},"max":{"1min":0.78800000000000003,"5min":0.83999999999999997,"15min":0.96099999999999997},"last":0.46300000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.33400000000000002,"15min":0.33200000000000002},"max":{"1min":0.95499999999999996,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60399999999999998,"15min":0.60499999999999998},"min":{"1min":0.47499999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.51700000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.46100000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85699999999999998,"5min":0.89700000000000002,"15min":1.052},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58899999999999997,"15min":0.746},"min":{"1min":0.40999999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.98099999999999998,"5min":1.274,"15min":6.2649999999999997},"last":0.54100000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303608,"kb_used_data":2224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068232,"statfs":{"total":96636764160,"available":96325869568,"internally_reserved":0,"allocated":2277376,"data_stored":1020969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.309,"15min":0.314},"min":{"1min":0.22600000000000001,"5min":0.21299999999999999,"15min":0.21099999999999999},"max":{"1min":0.38900000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.316,"5min":0.32700000000000001,"15min":0.32100000000000001},"min":{"1min":0.23799999999999999,"5min":0.216,"15min":0.19800000000000001},"max":{"1min":0.42799999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.27800000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.34399999999999997,"15min":0.33400000000000002},"min":{"1min":0.26200000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.47299999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38300000000000001,"15min":0.378},"min":{"1min":0.26800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.48599999999999999,"5min":0.66900000000000004,"15min":0.72099999999999997},"last":0.45200000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.36299999999999999,"15min":0.35299999999999998},"min":{"1min":0.28899999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44800000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.28799999999999998,"5min":0.25900000000000001,"15min":0.255},"max":{"1min":0.441,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56100000000000005,"15min":0.53800000000000003},"min":{"1min":0.44900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.372},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56699999999999995,"15min":0.53900000000000003},"min":{"1min":0.433,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.79400000000000004,"5min":0.83299999999999996,"15min":0.93500000000000005},"last":0.46400000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.55300000000000005},"min":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.35899999999999999},"max":{"1min":0.69599999999999995,"5min":1.347,"15min":1.347},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.443,"5min":0.39300000000000002,"15min":0.35799999999999998},"max":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":1.01},"last":0.44600000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.45500000000000002,"5min":0.41099999999999998,"15min":0.36699999999999999},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.502},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.48899999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79700000000000004,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.61799999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61599999999999999,"15min":0.59599999999999997},"min":{"1min":0.495,"5min":0.42099999999999999,"15min":0.38200000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.96999999999999997},"last":0.51900000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574959,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303608,"kb_used_data":2224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068232,"statfs":{"total":96636764160,"available":96325869568,"internally_reserved":0,"allocated":2277376,"data_stored":1020969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.372,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66700000000000004,"5min":0.74199999999999999,"15min":0.747},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52500000000000002,"15min":0.71899999999999997},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.32700000000000001},"max":{"1min":0.68300000000000005,"5min":0.77600000000000002,"15min":7.6719999999999997},"last":0.61899999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52600000000000002,"15min":0.72399999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.755,"15min":7.7160000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.371,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":0.86799999999999999},"last":0.64300000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67800000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.749},"min":{"1min":0.39900000000000002,"5min":0.39700000000000002,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":7.7839999999999998},"last":0.60199999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.47999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74399999999999999,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.42999999999999999,"5min":0.39200000000000002,"15min":0.35499999999999998},"max":{"1min":0.71499999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63300000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.497},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.30099999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.54700000000000004},"min":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.56599999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.35399999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.77300000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86199999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54200000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.378,"15min":0.36199999999999999},"max":{"1min":0.88600000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.41699999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.874,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.58599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303608,"kb_used_data":2224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068232,"statfs":{"total":96636764160,"available":96325869568,"internally_reserved":0,"allocated":2277376,"data_stored":1020969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.498,"15min":0.49199999999999999},"min":{"1min":0.38300000000000001,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.69699999999999995,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53100000000000003,"15min":0.52000000000000002},"min":{"1min":0.39400000000000002,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.72399999999999998,"5min":0.76700000000000002,"15min":0.79300000000000004},"last":0.68200000000000005}]},{"osd":1,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51400000000000001,"15min":0.61799999999999999},"min":{"1min":0.35099999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70799999999999996,"5min":0.746,"15min":5.2679999999999998},"last":0.5},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53000000000000003,"15min":0.61099999999999999},"min":{"1min":0.442,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.65700000000000003,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.58499999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.56200000000000006,"15min":0.63800000000000001},"min":{"1min":0.45500000000000002,"5min":0.32900000000000001,"15min":0.28299999999999997},"max":{"1min":3.27,"5min":3.27,"15min":5.2960000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55000000000000004,"15min":0.54100000000000004},"min":{"1min":0.40999999999999998,"5min":0.33900000000000002,"15min":0.32000000000000001},"max":{"1min":0.75,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.66400000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55700000000000005,"15min":0.65700000000000003},"min":{"1min":0.47499999999999998,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.69799999999999995,"5min":0.94599999999999995,"15min":5.3330000000000002},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57399999999999995,"15min":0.66800000000000004},"min":{"1min":0.45200000000000001,"5min":0.39300000000000002,"15min":0.27700000000000002},"max":{"1min":0.79300000000000004,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.78700000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.49199999999999999,"15min":0.497},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67300000000000004,"5min":0.72499999999999998,"15min":0.82999999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53100000000000003,"15min":0.54300000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32200000000000001},"max":{"1min":0.83099999999999996,"5min":0.88,"15min":1.0800000000000001},"last":0.61799999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.86499999999999999,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.69899999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56200000000000006,"15min":0.57499999999999996},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.30099999999999999},"max":{"1min":0.78700000000000003,"5min":0.98299999999999998,"15min":0.98399999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.378,"5min":0.377,"15min":0.377},"max":{"1min":0.83199999999999996,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.71499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:43:34.477 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:43:34.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.623+0000 7f6448d33700 1 Processor -- start 2023-10-10T20:43:34.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.623+0000 7f6448d33700 1 -- start start 2023-10-10T20:43:34.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.624+0000 7f6448d33700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6444159890 0x7f644415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:34.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.624+0000 7f6448d33700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6444069560 0x7f64440699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:34.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.624+0000 7f6448d33700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f644415a160 con 0x7f6444159c60 2023-10-10T20:43:34.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.624+0000 7f6448d33700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6444069f20 con 0x7f6444069560 2023-10-10T20:43:34.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.624+0000 7f6448d33700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64440d13a0 con 0x7f6444159890 2023-10-10T20:43:34.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.624+0000 7f644259c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6444159890 0x7f644415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:34.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.624+0000 7f6442d9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6444069560 0x7f64440699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:34.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.624+0000 7f644259c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6444159890 0x7f644415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:49528/0 (socket says 172.21.15.89:49528) 2023-10-10T20:43:34.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.624+0000 7f6439d9b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f6444159c60 0x7f644415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:39518/0 (socket says 172.21.15.89:39518) 2023-10-10T20:43:34.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.624+0000 7f644259c700 1 -- 172.21.15.89:0/1280129056 learned_addr learned my addr 172.21.15.89:0/1280129056 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.625+0000 7f644259c700 1 -- 172.21.15.89:0/1280129056 >> v2:172.21.15.111:3300/0 conn(0x7f6444069560 msgr2=0x7f64440699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.625+0000 7f644259c700 1 --2- 172.21.15.89:0/1280129056 >> v2:172.21.15.111:3300/0 conn(0x7f6444069560 0x7f64440699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.625+0000 7f644259c700 1 -- 172.21.15.89:0/1280129056 >> v1:172.21.15.111:6789/0 conn(0x7f6444159c60 legacy=0x7f644415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.625+0000 7f644259c700 1 -- 172.21.15.89:0/1280129056 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644415e140 con 0x7f6444159890 2023-10-10T20:43:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.625+0000 7f6441d9b700 1 -- 172.21.15.89:0/1280129056 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375926980 0 0) 0x7f644415a160 con 0x7f6444159c60 2023-10-10T20:43:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.625+0000 7f6442d9d700 1 --2- 172.21.15.89:0/1280129056 >> v2:172.21.15.111:3300/0 conn(0x7f6444069560 0x7f64440699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:43:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.625+0000 7f644259c700 1 --2- 172.21.15.89:0/1280129056 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6444159890 0x7f644415abd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f64340099b0 tx=0x7f6434009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aef6f1b6f115448f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.625+0000 7f6441d9b700 1 -- 172.21.15.89:0/1280129056 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434009ea0 con 0x7f6444159890 2023-10-10T20:43:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.625+0000 7f6441d9b700 1 -- 172.21.15.89:0/1280129056 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643400b860 con 0x7f6444159890 2023-10-10T20:43:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.625+0000 7f6441d9b700 1 -- 172.21.15.89:0/1280129056 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643400bc00 con 0x7f6444159890 2023-10-10T20:43:34.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.625+0000 7f6448d33700 1 -- 172.21.15.89:0/1280129056 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6444159890 msgr2=0x7f644415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:34.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.625+0000 7f6448d33700 1 --2- 172.21.15.89:0/1280129056 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6444159890 0x7f644415abd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f64340099b0 tx=0x7f6434009250 comp rx=0 tx=0).stop 2023-10-10T20:43:34.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.625+0000 7f6448d33700 1 -- 172.21.15.89:0/1280129056 shutdown_connections 2023-10-10T20:43:34.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.625+0000 7f6448d33700 1 --2- 172.21.15.89:0/1280129056 >> v2:172.21.15.111:3300/0 conn(0x7f6444069560 0x7f64440699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.625+0000 7f6448d33700 1 --2- 172.21.15.89:0/1280129056 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6444159890 0x7f644415abd0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.625+0000 7f6448d33700 1 -- 172.21.15.89:0/1280129056 >> 172.21.15.89:0/1280129056 conn(0x7f6444068e20 msgr2=0x7f64440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:34.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.625+0000 7f6448d33700 1 -- 172.21.15.89:0/1280129056 shutdown_connections 2023-10-10T20:43:34.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.626+0000 7f6448d33700 1 -- 172.21.15.89:0/1280129056 wait complete. 2023-10-10T20:43:34.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.627+0000 7f6448d33700 1 Processor -- start 2023-10-10T20:43:34.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.627+0000 7f6448d33700 1 -- start start 2023-10-10T20:43:34.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.627+0000 7f6448d33700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6444069560 0x7f6444153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:34.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.627+0000 7f6448d33700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6444159890 0x7f64441564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:34.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.627+0000 7f6448d33700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6444159c60 0x7f64440b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:34.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.627+0000 7f6448d33700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6444157a40 con 0x7f6444069560 2023-10-10T20:43:34.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.627+0000 7f6448d33700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6444154460 con 0x7f6444159c60 2023-10-10T20:43:34.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.627+0000 7f6448d33700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6444156e30 con 0x7f6444159890 2023-10-10T20:43:34.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.628+0000 7f644259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6444069560 0x7f6444153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:34.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.628+0000 7f6442d9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6444159c60 0x7f64440b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:34.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.628+0000 7f644259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6444069560 0x7f6444153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37330/0 (socket says 172.21.15.89:37330) 2023-10-10T20:43:34.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.628+0000 7f644259c700 1 -- 172.21.15.89:0/2644903625 learned_addr learned my addr 172.21.15.89:0/2644903625 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:34.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.628+0000 7f6439d9b700 1 --2- 172.21.15.89:0/2644903625 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6444159890 0x7f64441564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:34.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.628+0000 7f6442d9d700 1 -- 172.21.15.89:0/2644903625 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6444159890 msgr2=0x7f64441564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:34.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.628+0000 7f6442d9d700 1 --2- 172.21.15.89:0/2644903625 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6444159890 0x7f64441564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.628+0000 7f6442d9d700 1 -- 172.21.15.89:0/2644903625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6444069560 msgr2=0x7f6444153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:34.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.628+0000 7f6442d9d700 1 --2- 172.21.15.89:0/2644903625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6444069560 0x7f6444153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.628+0000 7f6442d9d700 1 -- 172.21.15.89:0/2644903625 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c009a20 con 0x7f6444159c60 2023-10-10T20:43:34.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.628+0000 7f6442d9d700 1 --2- 172.21.15.89:0/2644903625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6444159c60 0x7f64440b3d90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f643c009e70 tx=0x7f643c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:34.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.628+0000 7f643affd700 1 -- 172.21.15.89:0/2644903625 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c00caa0 con 0x7f6444159c60 2023-10-10T20:43:34.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.628+0000 7f643affd700 1 -- 172.21.15.89:0/2644903625 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643c00f040 con 0x7f6444159c60 2023-10-10T20:43:34.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.628+0000 7f6448d33700 1 -- 172.21.15.89:0/2644903625 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64340094f0 con 0x7f6444159c60 2023-10-10T20:43:34.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.628+0000 7f643affd700 1 -- 172.21.15.89:0/2644903625 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c00ce90 con 0x7f6444159c60 2023-10-10T20:43:34.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.628+0000 7f6448d33700 1 -- 172.21.15.89:0/2644903625 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64441573f0 con 0x7f6444159c60 2023-10-10T20:43:34.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.629+0000 7f6448d33700 1 -- 172.21.15.89:0/2644903625 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64440b3a50 con 0x7f6444159c60 2023-10-10T20:43:34.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.630+0000 7f643affd700 1 -- 172.21.15.89:0/2644903625 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f643c003020 con 0x7f6444159c60 2023-10-10T20:43:34.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.630+0000 7f643affd700 1 --2- 172.21.15.89:0/2644903625 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6424067320 0x7f64240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:34.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.630+0000 7f643affd700 1 -- 172.21.15.89:0/2644903625 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f643c0855a0 con 0x7f6444159c60 2023-10-10T20:43:34.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.630+0000 7f644259c700 1 --2- 172.21.15.89:0/2644903625 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6424067320 0x7f64240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:34.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.631+0000 7f644259c700 1 --2- 172.21.15.89:0/2644903625 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6424067320 0x7f64240697d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6434003080 tx=0x7f6434017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:34.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.635+0000 7f643affd700 1 -- 172.21.15.89:0/2644903625 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f643c0520e0 con 0x7f6444159c60 2023-10-10T20:43:34.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.818+0000 7f6448d33700 1 -- 172.21.15.89:0/2644903625 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f644404c3c0 con 0x7f6424067320 2023-10-10T20:43:34.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.825+0000 7f643affd700 1 -- 172.21.15.89:0/2644903625 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289905 (secure 0 0 0) 0x7f644404c3c0 con 0x7f6424067320 2023-10-10T20:43:34.830 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:34.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.828+0000 7f6448d33700 1 -- 172.21.15.89:0/2644903625 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6424067320 msgr2=0x7f64240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:34.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.828+0000 7f6448d33700 1 --2- 172.21.15.89:0/2644903625 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6424067320 0x7f64240697d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6434003080 tx=0x7f6434017000 comp rx=0 tx=0).stop 2023-10-10T20:43:34.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.828+0000 7f6448d33700 1 -- 172.21.15.89:0/2644903625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6444159c60 msgr2=0x7f64440b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:34.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.828+0000 7f6448d33700 1 --2- 172.21.15.89:0/2644903625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6444159c60 0x7f64440b3d90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f643c009e70 tx=0x7f643c00b040 comp rx=0 tx=0).stop 2023-10-10T20:43:34.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.828+0000 7f6448d33700 1 -- 172.21.15.89:0/2644903625 shutdown_connections 2023-10-10T20:43:34.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.828+0000 7f6448d33700 1 --2- 172.21.15.89:0/2644903625 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6424067320 0x7f64240697d0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.828+0000 7f6448d33700 1 --2- 172.21.15.89:0/2644903625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6444159c60 0x7f64440b3d90 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.828+0000 7f6448d33700 1 --2- 172.21.15.89:0/2644903625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6444069560 0x7f6444153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.828+0000 7f6448d33700 1 --2- 172.21.15.89:0/2644903625 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6444159890 0x7f64441564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:34.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.828+0000 7f6448d33700 1 -- 172.21.15.89:0/2644903625 >> 172.21.15.89:0/2644903625 conn(0x7f6444068e20 msgr2=0x7f6444155090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:34.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.828+0000 7f6448d33700 1 -- 172.21.15.89:0/2644903625 shutdown_connections 2023-10-10T20:43:34.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:34.828+0000 7f6448d33700 1 -- 172.21.15.89:0/2644903625 wait complete. 2023-10-10T20:43:34.835 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:43:34.842 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2023-10-10T20:43:34.758583+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2430468,"kb_used_data":19396,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752544252,"statfs":{"total":773094113280,"available":770605314048,"internally_reserved":0,"allocated":19861504,"data_stored":9553091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000945"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.429532+0000","last_change":"2023-10-10T20:43:30.425987+0000","last_active":"2023-10-10T20:43:30.429532+0000","last_peered":"2023-10-10T20:43:30.429532+0000","last_clean":"2023-10-10T20:43:30.429532+0000","last_became_active":"2023-10-10T20:43:30.425684+0000","last_became_peered":"2023-10-10T20:43:30.425684+0000","last_unstale":"2023-10-10T20:43:30.429532+0000","last_undegraded":"2023-10-10T20:43:30.429532+0000","last_fullsized":"2023-10-10T20:43:30.429532+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:57.417981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.433498+0000","last_change":"2023-10-10T20:43:30.426636+0000","last_active":"2023-10-10T20:43:30.433498+0000","last_peered":"2023-10-10T20:43:30.433498+0000","last_clean":"2023-10-10T20:43:30.433498+0000","last_became_active":"2023-10-10T20:43:30.426347+0000","last_became_peered":"2023-10-10T20:43:30.426347+0000","last_unstale":"2023-10-10T20:43:30.433498+0000","last_undegraded":"2023-10-10T20:43:30.433498+0000","last_fullsized":"2023-10-10T20:43:30.433498+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:19:32.678659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.430308+0000","last_change":"2023-10-10T20:43:30.424972+0000","last_active":"2023-10-10T20:43:30.430308+0000","last_peered":"2023-10-10T20:43:30.430308+0000","last_clean":"2023-10-10T20:43:30.430308+0000","last_became_active":"2023-10-10T20:43:30.424758+0000","last_became_peered":"2023-10-10T20:43:30.424758+0000","last_unstale":"2023-10-10T20:43:30.430308+0000","last_undegraded":"2023-10-10T20:43:30.430308+0000","last_fullsized":"2023-10-10T20:43:30.430308+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:16:08.720700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.428909+0000","last_change":"2023-10-10T20:43:30.420990+0000","last_active":"2023-10-10T20:43:30.428909+0000","last_peered":"2023-10-10T20:43:30.428909+0000","last_clean":"2023-10-10T20:43:30.428909+0000","last_became_active":"2023-10-10T20:43:30.420716+0000","last_became_peered":"2023-10-10T20:43:30.420716+0000","last_unstale":"2023-10-10T20:43:30.428909+0000","last_undegraded":"2023-10-10T20:43:30.428909+0000","last_fullsized":"2023-10-10T20:43:30.428909+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:38.547975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.427073+0000","last_change":"2023-10-10T20:43:30.421945+0000","last_active":"2023-10-10T20:43:30.427073+0000","last_peered":"2023-10-10T20:43:30.427073+0000","last_clean":"2023-10-10T20:43:30.427073+0000","last_became_active":"2023-10-10T20:43:30.421656+0000","last_became_peered":"2023-10-10T20:43:30.421656+0000","last_unstale":"2023-10-10T20:43:30.427073+0000","last_undegraded":"2023-10-10T20:43:30.427073+0000","last_fullsized":"2023-10-10T20:43:30.427073+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:24:17.041162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.429507+0000","last_change":"2023-10-10T20:43:30.425447+0000","last_active":"2023-10-10T20:43:30.429507+0000","last_peered":"2023-10-10T20:43:30.429507+0000","last_clean":"2023-10-10T20:43:30.429507+0000","last_became_active":"2023-10-10T20:43:30.425144+0000","last_became_peered":"2023-10-10T20:43:30.425144+0000","last_unstale":"2023-10-10T20:43:30.429507+0000","last_undegraded":"2023-10-10T20:43:30.429507+0000","last_fullsized":"2023-10-10T20:43:30.429507+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:10:10.771540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.423422+0000","last_change":"2023-10-10T20:43:30.422446+0000","last_active":"2023-10-10T20:43:30.423422+0000","last_peered":"2023-10-10T20:43:30.423422+0000","last_clean":"2023-10-10T20:43:30.423422+0000","last_became_active":"2023-10-10T20:43:30.422192+0000","last_became_peered":"2023-10-10T20:43:30.422192+0000","last_unstale":"2023-10-10T20:43:30.423422+0000","last_undegraded":"2023-10-10T20:43:30.423422+0000","last_fullsized":"2023-10-10T20:43:30.423422+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:34:55.991686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.432169+0000","last_change":"2023-10-10T20:43:30.431672+0000","last_active":"2023-10-10T20:43:30.432169+0000","last_peered":"2023-10-10T20:43:30.432169+0000","last_clean":"2023-10-10T20:43:30.432169+0000","last_became_active":"2023-10-10T20:43:30.431366+0000","last_became_peered":"2023-10-10T20:43:30.431366+0000","last_unstale":"2023-10-10T20:43:30.432169+0000","last_undegraded":"2023-10-10T20:43:30.432169+0000","last_fullsized":"2023-10-10T20:43:30.432169+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:19.654021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.432862+0000","last_change":"2023-10-10T20:43:30.426429+0000","last_active":"2023-10-10T20:43:30.432862+0000","last_peered":"2023-10-10T20:43:30.432862+0000","last_clean":"2023-10-10T20:43:30.432862+0000","last_became_active":"2023-10-10T20:43:30.425134+0000","last_became_peered":"2023-10-10T20:43:30.425134+0000","last_unstale":"2023-10-10T20:43:30.432862+0000","last_undegraded":"2023-10-10T20:43:30.432862+0000","last_fullsized":"2023-10-10T20:43:30.432862+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:20:28.026524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.430263+0000","last_change":"2023-10-10T20:43:30.424365+0000","last_active":"2023-10-10T20:43:30.430263+0000","last_peered":"2023-10-10T20:43:30.430263+0000","last_clean":"2023-10-10T20:43:30.430263+0000","last_became_active":"2023-10-10T20:43:30.424102+0000","last_became_peered":"2023-10-10T20:43:30.424102+0000","last_unstale":"2023-10-10T20:43:30.430263+0000","last_undegraded":"2023-10-10T20:43:30.430263+0000","last_fullsized":"2023-10-10T20:43:30.430263+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:49:36.163663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.428859+0000","last_change":"2023-10-10T20:43:30.420519+0000","last_active":"2023-10-10T20:43:30.428859+0000","last_peered":"2023-10-10T20:43:30.428859+0000","last_clean":"2023-10-10T20:43:30.428859+0000","last_became_active":"2023-10-10T20:43:30.420247+0000","last_became_peered":"2023-10-10T20:43:30.420247+0000","last_unstale":"2023-10-10T20:43:30.428859+0000","last_undegraded":"2023-10-10T20:43:30.428859+0000","last_fullsized":"2023-10-10T20:43:30.428859+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:08:22.694413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.429496+0000","last_change":"2023-10-10T20:43:30.420620+0000","last_active":"2023-10-10T20:43:30.429496+0000","last_peered":"2023-10-10T20:43:30.429496+0000","last_clean":"2023-10-10T20:43:30.429496+0000","last_became_active":"2023-10-10T20:43:30.420364+0000","last_became_peered":"2023-10-10T20:43:30.420364+0000","last_unstale":"2023-10-10T20:43:30.429496+0000","last_undegraded":"2023-10-10T20:43:30.429496+0000","last_fullsized":"2023-10-10T20:43:30.429496+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:48:30.477698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.423882+0000","last_change":"2023-10-10T20:43:30.422061+0000","last_active":"2023-10-10T20:43:30.423882+0000","last_peered":"2023-10-10T20:43:30.423882+0000","last_clean":"2023-10-10T20:43:30.423882+0000","last_became_active":"2023-10-10T20:43:30.421779+0000","last_became_peered":"2023-10-10T20:43:30.421779+0000","last_unstale":"2023-10-10T20:43:30.423882+0000","last_undegraded":"2023-10-10T20:43:30.423882+0000","last_fullsized":"2023-10-10T20:43:30.423882+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:08:57.623172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.426300+0000","last_change":"2023-10-10T20:43:30.424591+0000","last_active":"2023-10-10T20:43:30.426300+0000","last_peered":"2023-10-10T20:43:30.426300+0000","last_clean":"2023-10-10T20:43:30.426300+0000","last_became_active":"2023-10-10T20:43:30.424000+0000","last_became_peered":"2023-10-10T20:43:30.424000+0000","last_unstale":"2023-10-10T20:43:30.426300+0000","last_undegraded":"2023-10-10T20:43:30.426300+0000","last_fullsized":"2023-10-10T20:43:30.426300+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:15:04.391430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.431029+0000","last_change":"2023-10-10T20:43:30.428386+0000","last_active":"2023-10-10T20:43:30.431029+0000","last_peered":"2023-10-10T20:43:30.431029+0000","last_clean":"2023-10-10T20:43:30.431029+0000","last_became_active":"2023-10-10T20:43:30.428152+0000","last_became_peered":"2023-10-10T20:43:30.428152+0000","last_unstale":"2023-10-10T20:43:30.431029+0000","last_undegraded":"2023-10-10T20:43:30.431029+0000","last_fullsized":"2023-10-10T20:43:30.431029+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:24:17.041162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.436094+0000","last_change":"2023-10-10T20:43:30.434865+0000","last_active":"2023-10-10T20:43:30.436094+0000","last_peered":"2023-10-10T20:43:30.436094+0000","last_clean":"2023-10-10T20:43:30.436094+0000","last_became_active":"2023-10-10T20:43:30.434596+0000","last_became_peered":"2023-10-10T20:43:30.434596+0000","last_unstale":"2023-10-10T20:43:30.436094+0000","last_undegraded":"2023-10-10T20:43:30.436094+0000","last_fullsized":"2023-10-10T20:43:30.436094+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:50.842702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.423396+0000","last_change":"2023-10-10T20:43:30.421838+0000","last_active":"2023-10-10T20:43:30.423396+0000","last_peered":"2023-10-10T20:43:30.423396+0000","last_clean":"2023-10-10T20:43:30.423396+0000","last_became_active":"2023-10-10T20:43:30.421558+0000","last_became_peered":"2023-10-10T20:43:30.421558+0000","last_unstale":"2023-10-10T20:43:30.423396+0000","last_undegraded":"2023-10-10T20:43:30.423396+0000","last_fullsized":"2023-10-10T20:43:30.423396+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:17:29.322757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.430972+0000","last_change":"2023-10-10T20:43:30.428509+0000","last_active":"2023-10-10T20:43:30.430972+0000","last_peered":"2023-10-10T20:43:30.430972+0000","last_clean":"2023-10-10T20:43:30.430972+0000","last_became_active":"2023-10-10T20:43:30.428257+0000","last_became_peered":"2023-10-10T20:43:30.428257+0000","last_unstale":"2023-10-10T20:43:30.430972+0000","last_undegraded":"2023-10-10T20:43:30.430972+0000","last_fullsized":"2023-10-10T20:43:30.430972+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:58:33.657578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.424536+0000","last_change":"2023-10-10T20:43:30.420724+0000","last_active":"2023-10-10T20:43:30.424536+0000","last_peered":"2023-10-10T20:43:30.424536+0000","last_clean":"2023-10-10T20:43:30.424536+0000","last_became_active":"2023-10-10T20:43:30.420442+0000","last_became_peered":"2023-10-10T20:43:30.420442+0000","last_unstale":"2023-10-10T20:43:30.424536+0000","last_undegraded":"2023-10-10T20:43:30.424536+0000","last_fullsized":"2023-10-10T20:43:30.424536+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:16:08.720700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.432307+0000","last_change":"2023-10-10T20:43:30.424185+0000","last_active":"2023-10-10T20:43:30.432307+0000","last_peered":"2023-10-10T20:43:30.432307+0000","last_clean":"2023-10-10T20:43:30.432307+0000","last_became_active":"2023-10-10T20:43:30.423942+0000","last_became_peered":"2023-10-10T20:43:30.423942+0000","last_unstale":"2023-10-10T20:43:30.432307+0000","last_undegraded":"2023-10-10T20:43:30.432307+0000","last_fullsized":"2023-10-10T20:43:30.432307+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:45:34.939805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.426871+0000","last_change":"2023-10-10T20:43:30.423744+0000","last_active":"2023-10-10T20:43:30.426871+0000","last_peered":"2023-10-10T20:43:30.426871+0000","last_clean":"2023-10-10T20:43:30.426871+0000","last_became_active":"2023-10-10T20:43:30.423464+0000","last_became_peered":"2023-10-10T20:43:30.423464+0000","last_unstale":"2023-10-10T20:43:30.426871+0000","last_undegraded":"2023-10-10T20:43:30.426871+0000","last_fullsized":"2023-10-10T20:43:30.426871+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:32:42.748731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.431342+0000","last_change":"2023-10-10T20:43:30.429529+0000","last_active":"2023-10-10T20:43:30.431342+0000","last_peered":"2023-10-10T20:43:30.431342+0000","last_clean":"2023-10-10T20:43:30.431342+0000","last_became_active":"2023-10-10T20:43:30.429239+0000","last_became_peered":"2023-10-10T20:43:30.429239+0000","last_unstale":"2023-10-10T20:43:30.431342+0000","last_undegraded":"2023-10-10T20:43:30.431342+0000","last_fullsized":"2023-10-10T20:43:30.431342+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:23:29.922377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.423428+0000","last_change":"2023-10-10T20:43:30.419979+0000","last_active":"2023-10-10T20:43:30.423428+0000","last_peered":"2023-10-10T20:43:30.423428+0000","last_clean":"2023-10-10T20:43:30.423428+0000","last_became_active":"2023-10-10T20:43:30.419738+0000","last_became_peered":"2023-10-10T20:43:30.419738+0000","last_unstale":"2023-10-10T20:43:30.423428+0000","last_undegraded":"2023-10-10T20:43:30.423428+0000","last_fullsized":"2023-10-10T20:43:30.423428+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:08:22.694413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.422370+0000","last_change":"2023-10-10T20:43:30.418782+0000","last_active":"2023-10-10T20:43:30.422370+0000","last_peered":"2023-10-10T20:43:30.422370+0000","last_clean":"2023-10-10T20:43:30.422370+0000","last_became_active":"2023-10-10T20:43:30.418536+0000","last_became_peered":"2023-10-10T20:43:30.418536+0000","last_unstale":"2023-10-10T20:43:30.422370+0000","last_undegraded":"2023-10-10T20:43:30.422370+0000","last_fullsized":"2023-10-10T20:43:30.422370+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:10.501491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.434553+0000","last_change":"2023-10-10T20:43:30.428321+0000","last_active":"2023-10-10T20:43:30.434553+0000","last_peered":"2023-10-10T20:43:30.434553+0000","last_clean":"2023-10-10T20:43:30.434553+0000","last_became_active":"2023-10-10T20:43:30.428049+0000","last_became_peered":"2023-10-10T20:43:30.428049+0000","last_unstale":"2023-10-10T20:43:30.434553+0000","last_undegraded":"2023-10-10T20:43:30.434553+0000","last_fullsized":"2023-10-10T20:43:30.434553+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:32:42.748731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.433889+0000","last_change":"2023-10-10T20:43:30.433416+0000","last_active":"2023-10-10T20:43:30.433889+0000","last_peered":"2023-10-10T20:43:30.433889+0000","last_clean":"2023-10-10T20:43:30.433889+0000","last_became_active":"2023-10-10T20:43:30.433204+0000","last_became_peered":"2023-10-10T20:43:30.433204+0000","last_unstale":"2023-10-10T20:43:30.433889+0000","last_undegraded":"2023-10-10T20:43:30.433889+0000","last_fullsized":"2023-10-10T20:43:30.433889+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:38:12.978954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.425934+0000","last_change":"2023-10-10T20:43:30.420082+0000","last_active":"2023-10-10T20:43:30.425934+0000","last_peered":"2023-10-10T20:43:30.425934+0000","last_clean":"2023-10-10T20:43:30.425934+0000","last_became_active":"2023-10-10T20:43:30.419732+0000","last_became_peered":"2023-10-10T20:43:30.419732+0000","last_unstale":"2023-10-10T20:43:30.425934+0000","last_undegraded":"2023-10-10T20:43:30.425934+0000","last_fullsized":"2023-10-10T20:43:30.425934+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:08:22.694413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.435672+0000","last_change":"2023-10-10T20:43:30.433758+0000","last_active":"2023-10-10T20:43:30.435672+0000","last_peered":"2023-10-10T20:43:30.435672+0000","last_clean":"2023-10-10T20:43:30.435672+0000","last_became_active":"2023-10-10T20:43:30.429335+0000","last_became_peered":"2023-10-10T20:43:30.429335+0000","last_unstale":"2023-10-10T20:43:30.435672+0000","last_undegraded":"2023-10-10T20:43:30.435672+0000","last_fullsized":"2023-10-10T20:43:30.435672+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:53:37.179074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.426145+0000","last_change":"2023-10-10T20:43:30.423704+0000","last_active":"2023-10-10T20:43:30.426145+0000","last_peered":"2023-10-10T20:43:30.426145+0000","last_clean":"2023-10-10T20:43:30.426145+0000","last_became_active":"2023-10-10T20:43:30.423452+0000","last_became_peered":"2023-10-10T20:43:30.423452+0000","last_unstale":"2023-10-10T20:43:30.426145+0000","last_undegraded":"2023-10-10T20:43:30.426145+0000","last_fullsized":"2023-10-10T20:43:30.426145+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:55:50.192346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.426317+0000","last_change":"2023-10-10T20:43:30.424776+0000","last_active":"2023-10-10T20:43:30.426317+0000","last_peered":"2023-10-10T20:43:30.426317+0000","last_clean":"2023-10-10T20:43:30.426317+0000","last_became_active":"2023-10-10T20:43:30.424331+0000","last_became_peered":"2023-10-10T20:43:30.424331+0000","last_unstale":"2023-10-10T20:43:30.426317+0000","last_undegraded":"2023-10-10T20:43:30.426317+0000","last_fullsized":"2023-10-10T20:43:30.426317+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:30:57.857839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.434638+0000","last_change":"2023-10-10T20:43:30.431482+0000","last_active":"2023-10-10T20:43:30.434638+0000","last_peered":"2023-10-10T20:43:30.434638+0000","last_clean":"2023-10-10T20:43:30.434638+0000","last_became_active":"2023-10-10T20:43:30.429645+0000","last_became_peered":"2023-10-10T20:43:30.429645+0000","last_unstale":"2023-10-10T20:43:30.434638+0000","last_undegraded":"2023-10-10T20:43:30.434638+0000","last_fullsized":"2023-10-10T20:43:30.434638+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:49:24.142885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.429477+0000","last_change":"2023-10-10T20:43:30.425431+0000","last_active":"2023-10-10T20:43:30.429477+0000","last_peered":"2023-10-10T20:43:30.429477+0000","last_clean":"2023-10-10T20:43:30.429477+0000","last_became_active":"2023-10-10T20:43:30.425222+0000","last_became_peered":"2023-10-10T20:43:30.425222+0000","last_unstale":"2023-10-10T20:43:30.429477+0000","last_undegraded":"2023-10-10T20:43:30.429477+0000","last_fullsized":"2023-10-10T20:43:30.429477+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:27:36.743256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.432769+0000","last_change":"2023-10-10T20:43:30.426626+0000","last_active":"2023-10-10T20:43:30.432769+0000","last_peered":"2023-10-10T20:43:30.432769+0000","last_clean":"2023-10-10T20:43:30.432769+0000","last_became_active":"2023-10-10T20:43:30.426356+0000","last_became_peered":"2023-10-10T20:43:30.426356+0000","last_unstale":"2023-10-10T20:43:30.432769+0000","last_undegraded":"2023-10-10T20:43:30.432769+0000","last_fullsized":"2023-10-10T20:43:30.432769+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:20.993015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"182'4","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.435202+0000","last_change":"2023-10-10T20:43:29.423705+0000","last_active":"2023-10-10T20:43:30.435202+0000","last_peered":"2023-10-10T20:43:30.435202+0000","last_clean":"2023-10-10T20:43:30.435202+0000","last_became_active":"2023-10-10T20:43:29.423489+0000","last_became_peered":"2023-10-10T20:43:29.423489+0000","last_unstale":"2023-10-10T20:43:30.435202+0000","last_undegraded":"2023-10-10T20:43:30.435202+0000","last_fullsized":"2023-10-10T20:43:30.435202+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:55:49.184920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.427541+0000","last_change":"2023-10-10T20:43:30.424147+0000","last_active":"2023-10-10T20:43:30.427541+0000","last_peered":"2023-10-10T20:43:30.427541+0000","last_clean":"2023-10-10T20:43:30.427541+0000","last_became_active":"2023-10-10T20:43:30.423850+0000","last_became_peered":"2023-10-10T20:43:30.423850+0000","last_unstale":"2023-10-10T20:43:30.427541+0000","last_undegraded":"2023-10-10T20:43:30.427541+0000","last_fullsized":"2023-10-10T20:43:30.427541+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:38.547975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"182'2","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.419828+0000","last_change":"2023-10-10T20:43:29.423745+0000","last_active":"2023-10-10T20:43:30.419828+0000","last_peered":"2023-10-10T20:43:30.419828+0000","last_clean":"2023-10-10T20:43:30.419828+0000","last_became_active":"2023-10-10T20:43:29.423485+0000","last_became_peered":"2023-10-10T20:43:29.423485+0000","last_unstale":"2023-10-10T20:43:30.419828+0000","last_undegraded":"2023-10-10T20:43:30.419828+0000","last_fullsized":"2023-10-10T20:43:30.419828+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:53:23.843813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.433427+0000","last_change":"2023-10-10T20:43:30.427355+0000","last_active":"2023-10-10T20:43:30.433427+0000","last_peered":"2023-10-10T20:43:30.433427+0000","last_clean":"2023-10-10T20:43:30.433427+0000","last_became_active":"2023-10-10T20:43:30.425620+0000","last_became_peered":"2023-10-10T20:43:30.425620+0000","last_unstale":"2023-10-10T20:43:30.433427+0000","last_undegraded":"2023-10-10T20:43:30.433427+0000","last_fullsized":"2023-10-10T20:43:30.433427+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:06:16.804429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"182'2","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.435131+0000","last_change":"2023-10-10T20:43:29.428006+0000","last_active":"2023-10-10T20:43:30.435131+0000","last_peered":"2023-10-10T20:43:30.435131+0000","last_clean":"2023-10-10T20:43:30.435131+0000","last_became_active":"2023-10-10T20:43:29.427700+0000","last_became_peered":"2023-10-10T20:43:29.427700+0000","last_unstale":"2023-10-10T20:43:30.435131+0000","last_undegraded":"2023-10-10T20:43:30.435131+0000","last_fullsized":"2023-10-10T20:43:30.435131+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:37.540549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.430408+0000","last_change":"2023-10-10T20:43:30.427458+0000","last_active":"2023-10-10T20:43:30.430408+0000","last_peered":"2023-10-10T20:43:30.430408+0000","last_clean":"2023-10-10T20:43:30.430408+0000","last_became_active":"2023-10-10T20:43:30.423783+0000","last_became_peered":"2023-10-10T20:43:30.423783+0000","last_unstale":"2023-10-10T20:43:30.430408+0000","last_undegraded":"2023-10-10T20:43:30.430408+0000","last_fullsized":"2023-10-10T20:43:30.430408+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:40:22.748936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"182'1","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.418402+0000","last_change":"2023-10-10T20:43:29.420825+0000","last_active":"2023-10-10T20:43:30.418402+0000","last_peered":"2023-10-10T20:43:30.418402+0000","last_clean":"2023-10-10T20:43:30.418402+0000","last_became_active":"2023-10-10T20:43:29.420498+0000","last_became_peered":"2023-10-10T20:43:29.420498+0000","last_unstale":"2023-10-10T20:43:30.418402+0000","last_undegraded":"2023-10-10T20:43:30.418402+0000","last_fullsized":"2023-10-10T20:43:30.418402+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:28:14.235961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.426970+0000","last_change":"2023-10-10T20:43:30.423812+0000","last_active":"2023-10-10T20:43:30.426970+0000","last_peered":"2023-10-10T20:43:30.426970+0000","last_clean":"2023-10-10T20:43:30.426970+0000","last_became_active":"2023-10-10T20:43:30.423366+0000","last_became_peered":"2023-10-10T20:43:30.423366+0000","last_unstale":"2023-10-10T20:43:30.426970+0000","last_undegraded":"2023-10-10T20:43:30.426970+0000","last_fullsized":"2023-10-10T20:43:30.426970+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:58:46.968633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"182'1","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.426226+0000","last_change":"2023-10-10T20:43:29.425094+0000","last_active":"2023-10-10T20:43:30.426226+0000","last_peered":"2023-10-10T20:43:30.426226+0000","last_clean":"2023-10-10T20:43:30.426226+0000","last_became_active":"2023-10-10T20:43:29.424820+0000","last_became_peered":"2023-10-10T20:43:29.424820+0000","last_unstale":"2023-10-10T20:43:30.426226+0000","last_undegraded":"2023-10-10T20:43:30.426226+0000","last_fullsized":"2023-10-10T20:43:30.426226+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:08:21.686988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.428835+0000","last_change":"2023-10-10T20:43:30.420304+0000","last_active":"2023-10-10T20:43:30.428835+0000","last_peered":"2023-10-10T20:43:30.428835+0000","last_clean":"2023-10-10T20:43:30.428835+0000","last_became_active":"2023-10-10T20:43:30.420042+0000","last_became_peered":"2023-10-10T20:43:30.420042+0000","last_unstale":"2023-10-10T20:43:30.428835+0000","last_undegraded":"2023-10-10T20:43:30.428835+0000","last_fullsized":"2023-10-10T20:43:30.428835+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:24:17.041162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.416433+0000","last_change":"2023-10-10T20:43:29.422055+0000","last_active":"2023-10-10T20:43:30.416433+0000","last_peered":"2023-10-10T20:43:30.416433+0000","last_clean":"2023-10-10T20:43:30.416433+0000","last_became_active":"2023-10-10T20:43:29.421629+0000","last_became_peered":"2023-10-10T20:43:29.421629+0000","last_unstale":"2023-10-10T20:43:30.416433+0000","last_undegraded":"2023-10-10T20:43:30.416433+0000","last_fullsized":"2023-10-10T20:43:30.416433+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:18:57.772443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.426347+0000","last_change":"2023-10-10T20:43:30.423978+0000","last_active":"2023-10-10T20:43:30.426347+0000","last_peered":"2023-10-10T20:43:30.426347+0000","last_clean":"2023-10-10T20:43:30.426347+0000","last_became_active":"2023-10-10T20:43:30.423676+0000","last_became_peered":"2023-10-10T20:43:30.423676+0000","last_unstale":"2023-10-10T20:43:30.426347+0000","last_undegraded":"2023-10-10T20:43:30.426347+0000","last_fullsized":"2023-10-10T20:43:30.426347+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:58:33.657578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.412878+0000","last_change":"2023-10-10T20:43:29.423487+0000","last_active":"2023-10-10T20:43:30.412878+0000","last_peered":"2023-10-10T20:43:30.412878+0000","last_clean":"2023-10-10T20:43:30.412878+0000","last_became_active":"2023-10-10T20:43:29.423190+0000","last_became_peered":"2023-10-10T20:43:29.423190+0000","last_unstale":"2023-10-10T20:43:30.412878+0000","last_undegraded":"2023-10-10T20:43:30.412878+0000","last_fullsized":"2023-10-10T20:43:30.412878+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:40:21.741511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.422879+0000","last_change":"2023-10-10T20:43:30.421696+0000","last_active":"2023-10-10T20:43:30.422879+0000","last_peered":"2023-10-10T20:43:30.422879+0000","last_clean":"2023-10-10T20:43:30.422879+0000","last_became_active":"2023-10-10T20:43:30.421449+0000","last_became_peered":"2023-10-10T20:43:30.421449+0000","last_unstale":"2023-10-10T20:43:30.422879+0000","last_undegraded":"2023-10-10T20:43:30.422879+0000","last_fullsized":"2023-10-10T20:43:30.422879+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:49:29.995065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.406968+0000","last_change":"2023-10-10T20:43:29.422403+0000","last_active":"2023-10-10T20:43:30.406968+0000","last_peered":"2023-10-10T20:43:30.406968+0000","last_clean":"2023-10-10T20:43:30.406968+0000","last_became_active":"2023-10-10T20:43:29.422137+0000","last_became_peered":"2023-10-10T20:43:29.422137+0000","last_unstale":"2023-10-10T20:43:30.406968+0000","last_undegraded":"2023-10-10T20:43:30.406968+0000","last_fullsized":"2023-10-10T20:43:30.406968+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:29:52.147797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.431399+0000","last_change":"2023-10-10T20:43:30.429653+0000","last_active":"2023-10-10T20:43:30.431399+0000","last_peered":"2023-10-10T20:43:30.431399+0000","last_clean":"2023-10-10T20:43:30.431399+0000","last_became_active":"2023-10-10T20:43:30.429300+0000","last_became_peered":"2023-10-10T20:43:30.429300+0000","last_unstale":"2023-10-10T20:43:30.431399+0000","last_undegraded":"2023-10-10T20:43:30.431399+0000","last_fullsized":"2023-10-10T20:43:30.431399+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:01:01.301659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.416765+0000","last_change":"2023-10-10T20:43:29.426598+0000","last_active":"2023-10-10T20:43:30.416765+0000","last_peered":"2023-10-10T20:43:30.416765+0000","last_clean":"2023-10-10T20:43:30.416765+0000","last_became_active":"2023-10-10T20:43:29.426350+0000","last_became_peered":"2023-10-10T20:43:29.426350+0000","last_unstale":"2023-10-10T20:43:30.416765+0000","last_undegraded":"2023-10-10T20:43:30.416765+0000","last_fullsized":"2023-10-10T20:43:30.416765+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:23:28.914951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.430281+0000","last_change":"2023-10-10T20:43:30.423686+0000","last_active":"2023-10-10T20:43:30.430281+0000","last_peered":"2023-10-10T20:43:30.430281+0000","last_clean":"2023-10-10T20:43:30.430281+0000","last_became_active":"2023-10-10T20:43:30.423384+0000","last_became_peered":"2023-10-10T20:43:30.423384+0000","last_unstale":"2023-10-10T20:43:30.430281+0000","last_undegraded":"2023-10-10T20:43:30.430281+0000","last_fullsized":"2023-10-10T20:43:30.430281+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:15:04.391430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"182'3","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.426144+0000","last_change":"2023-10-10T20:43:29.422147+0000","last_active":"2023-10-10T20:43:30.426144+0000","last_peered":"2023-10-10T20:43:30.426144+0000","last_clean":"2023-10-10T20:43:30.426144+0000","last_became_active":"2023-10-10T20:43:29.421285+0000","last_became_peered":"2023-10-10T20:43:29.421285+0000","last_unstale":"2023-10-10T20:43:30.426144+0000","last_undegraded":"2023-10-10T20:43:30.426144+0000","last_fullsized":"2023-10-10T20:43:30.426144+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:42:28.587579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.432268+0000","last_change":"2023-10-10T20:43:30.431219+0000","last_active":"2023-10-10T20:43:30.432268+0000","last_peered":"2023-10-10T20:43:30.432268+0000","last_clean":"2023-10-10T20:43:30.432268+0000","last_became_active":"2023-10-10T20:43:30.430922+0000","last_became_peered":"2023-10-10T20:43:30.430922+0000","last_unstale":"2023-10-10T20:43:30.432268+0000","last_undegraded":"2023-10-10T20:43:30.432268+0000","last_fullsized":"2023-10-10T20:43:30.432268+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:30:52.467824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"182'6","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.438790+0000","last_change":"2023-10-10T20:43:29.427927+0000","last_active":"2023-10-10T20:43:30.438790+0000","last_peered":"2023-10-10T20:43:30.438790+0000","last_clean":"2023-10-10T20:43:30.438790+0000","last_became_active":"2023-10-10T20:43:29.427639+0000","last_became_peered":"2023-10-10T20:43:29.427639+0000","last_unstale":"2023-10-10T20:43:30.438790+0000","last_undegraded":"2023-10-10T20:43:30.438790+0000","last_fullsized":"2023-10-10T20:43:30.438790+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:30:56.850413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.432274+0000","last_change":"2023-10-10T20:43:30.422457+0000","last_active":"2023-10-10T20:43:30.432274+0000","last_peered":"2023-10-10T20:43:30.432274+0000","last_clean":"2023-10-10T20:43:30.432274+0000","last_became_active":"2023-10-10T20:43:30.422228+0000","last_became_peered":"2023-10-10T20:43:30.422228+0000","last_unstale":"2023-10-10T20:43:30.432274+0000","last_undegraded":"2023-10-10T20:43:30.432274+0000","last_fullsized":"2023-10-10T20:43:30.432274+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:05:22.150390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"182'2","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.434208+0000","last_change":"2023-10-10T20:43:29.424400+0000","last_active":"2023-10-10T20:43:30.434208+0000","last_peered":"2023-10-10T20:43:30.434208+0000","last_clean":"2023-10-10T20:43:30.434208+0000","last_became_active":"2023-10-10T20:43:29.424076+0000","last_became_peered":"2023-10-10T20:43:29.424076+0000","last_unstale":"2023-10-10T20:43:30.434208+0000","last_undegraded":"2023-10-10T20:43:30.434208+0000","last_fullsized":"2023-10-10T20:43:30.434208+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:38:49.789790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.428323+0000","last_change":"2023-10-10T20:43:30.427273+0000","last_active":"2023-10-10T20:43:30.428323+0000","last_peered":"2023-10-10T20:43:30.428323+0000","last_clean":"2023-10-10T20:43:30.428323+0000","last_became_active":"2023-10-10T20:43:30.426973+0000","last_became_peered":"2023-10-10T20:43:30.426973+0000","last_unstale":"2023-10-10T20:43:30.428323+0000","last_undegraded":"2023-10-10T20:43:30.428323+0000","last_fullsized":"2023-10-10T20:43:30.428323+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:16:32.598113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.416429+0000","last_change":"2023-10-10T20:43:29.420134+0000","last_active":"2023-10-10T20:43:30.416429+0000","last_peered":"2023-10-10T20:43:30.416429+0000","last_clean":"2023-10-10T20:43:30.416429+0000","last_became_active":"2023-10-10T20:43:29.419805+0000","last_became_peered":"2023-10-10T20:43:29.419805+0000","last_unstale":"2023-10-10T20:43:30.416429+0000","last_undegraded":"2023-10-10T20:43:30.416429+0000","last_fullsized":"2023-10-10T20:43:30.416429+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:18:57.772443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.422960+0000","last_change":"2023-10-10T20:43:30.418751+0000","last_active":"2023-10-10T20:43:30.422960+0000","last_peered":"2023-10-10T20:43:30.422960+0000","last_clean":"2023-10-10T20:43:30.422960+0000","last_became_active":"2023-10-10T20:43:30.418492+0000","last_became_peered":"2023-10-10T20:43:30.418492+0000","last_unstale":"2023-10-10T20:43:30.422960+0000","last_undegraded":"2023-10-10T20:43:30.422960+0000","last_fullsized":"2023-10-10T20:43:30.422960+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:43.068714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.415850+0000","last_change":"2023-10-10T20:43:29.427867+0000","last_active":"2023-10-10T20:43:30.415850+0000","last_peered":"2023-10-10T20:43:30.415850+0000","last_clean":"2023-10-10T20:43:30.415850+0000","last_became_active":"2023-10-10T20:43:29.427602+0000","last_became_peered":"2023-10-10T20:43:29.427602+0000","last_unstale":"2023-10-10T20:43:30.415850+0000","last_undegraded":"2023-10-10T20:43:30.415850+0000","last_fullsized":"2023-10-10T20:43:30.415850+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:26.569301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.431835+0000","last_change":"2023-10-10T20:43:30.430374+0000","last_active":"2023-10-10T20:43:30.431835+0000","last_peered":"2023-10-10T20:43:30.431835+0000","last_clean":"2023-10-10T20:43:30.431835+0000","last_became_active":"2023-10-10T20:43:30.430051+0000","last_became_peered":"2023-10-10T20:43:30.430051+0000","last_unstale":"2023-10-10T20:43:30.431835+0000","last_undegraded":"2023-10-10T20:43:30.431835+0000","last_fullsized":"2023-10-10T20:43:30.431835+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:43.068714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"182'8","reported_seq":18,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.435651+0000","last_change":"2023-10-10T20:43:29.417986+0000","last_active":"2023-10-10T20:43:30.435651+0000","last_peered":"2023-10-10T20:43:30.435651+0000","last_clean":"2023-10-10T20:43:30.435651+0000","last_became_active":"2023-10-10T20:43:29.417719+0000","last_became_peered":"2023-10-10T20:43:29.417719+0000","last_unstale":"2023-10-10T20:43:30.435651+0000","last_undegraded":"2023-10-10T20:43:30.435651+0000","last_fullsized":"2023-10-10T20:43:30.435651+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:22:54.398339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.434197+0000","last_change":"2023-10-10T20:43:30.433793+0000","last_active":"2023-10-10T20:43:30.434197+0000","last_peered":"2023-10-10T20:43:30.434197+0000","last_clean":"2023-10-10T20:43:30.434197+0000","last_became_active":"2023-10-10T20:43:30.433541+0000","last_became_peered":"2023-10-10T20:43:30.433541+0000","last_unstale":"2023-10-10T20:43:30.434197+0000","last_undegraded":"2023-10-10T20:43:30.434197+0000","last_fullsized":"2023-10-10T20:43:30.434197+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:05.833137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"182'1","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.429468+0000","last_change":"2023-10-10T20:43:29.423328+0000","last_active":"2023-10-10T20:43:30.429468+0000","last_peered":"2023-10-10T20:43:30.429468+0000","last_clean":"2023-10-10T20:43:30.429468+0000","last_became_active":"2023-10-10T20:43:29.423127+0000","last_became_peered":"2023-10-10T20:43:29.423127+0000","last_unstale":"2023-10-10T20:43:30.429468+0000","last_undegraded":"2023-10-10T20:43:30.429468+0000","last_fullsized":"2023-10-10T20:43:30.429468+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:36:19.100900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.427040+0000","last_change":"2023-10-10T20:43:30.422369+0000","last_active":"2023-10-10T20:43:30.427040+0000","last_peered":"2023-10-10T20:43:30.427040+0000","last_clean":"2023-10-10T20:43:30.427040+0000","last_became_active":"2023-10-10T20:43:30.422031+0000","last_became_peered":"2023-10-10T20:43:30.422031+0000","last_unstale":"2023-10-10T20:43:30.427040+0000","last_undegraded":"2023-10-10T20:43:30.427040+0000","last_fullsized":"2023-10-10T20:43:30.427040+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:38.547975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"182'2","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.432119+0000","last_change":"2023-10-10T20:43:29.415057+0000","last_active":"2023-10-10T20:43:30.432119+0000","last_peered":"2023-10-10T20:43:30.432119+0000","last_clean":"2023-10-10T20:43:30.432119+0000","last_became_active":"2023-10-10T20:43:29.414784+0000","last_became_peered":"2023-10-10T20:43:29.414784+0000","last_unstale":"2023-10-10T20:43:30.432119+0000","last_undegraded":"2023-10-10T20:43:30.432119+0000","last_fullsized":"2023-10-10T20:43:30.432119+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:57:48.257730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.432735+0000","last_change":"2023-10-10T20:43:30.426696+0000","last_active":"2023-10-10T20:43:30.432735+0000","last_peered":"2023-10-10T20:43:30.432735+0000","last_clean":"2023-10-10T20:43:30.432735+0000","last_became_active":"2023-10-10T20:43:30.426443+0000","last_became_peered":"2023-10-10T20:43:30.426443+0000","last_unstale":"2023-10-10T20:43:30.432735+0000","last_undegraded":"2023-10-10T20:43:30.432735+0000","last_fullsized":"2023-10-10T20:43:30.432735+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:34:40.782146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"182'1","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.419728+0000","last_change":"2023-10-10T20:43:29.423842+0000","last_active":"2023-10-10T20:43:30.419728+0000","last_peered":"2023-10-10T20:43:30.419728+0000","last_clean":"2023-10-10T20:43:30.419728+0000","last_became_active":"2023-10-10T20:43:29.423583+0000","last_became_peered":"2023-10-10T20:43:29.423583+0000","last_unstale":"2023-10-10T20:43:30.419728+0000","last_undegraded":"2023-10-10T20:43:30.419728+0000","last_fullsized":"2023-10-10T20:43:30.419728+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:40:03.121857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.430235+0000","last_change":"2023-10-10T20:43:30.423546+0000","last_active":"2023-10-10T20:43:30.430235+0000","last_peered":"2023-10-10T20:43:30.430235+0000","last_clean":"2023-10-10T20:43:30.430235+0000","last_became_active":"2023-10-10T20:43:30.423276+0000","last_became_peered":"2023-10-10T20:43:30.423276+0000","last_unstale":"2023-10-10T20:43:30.430235+0000","last_undegraded":"2023-10-10T20:43:30.430235+0000","last_fullsized":"2023-10-10T20:43:30.430235+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:19.654021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.409015+0000","last_change":"2023-10-10T20:43:29.422344+0000","last_active":"2023-10-10T20:43:30.409015+0000","last_peered":"2023-10-10T20:43:30.409015+0000","last_clean":"2023-10-10T20:43:30.409015+0000","last_became_active":"2023-10-10T20:43:29.421926+0000","last_became_peered":"2023-10-10T20:43:29.421926+0000","last_unstale":"2023-10-10T20:43:30.409015+0000","last_undegraded":"2023-10-10T20:43:30.409015+0000","last_fullsized":"2023-10-10T20:43:30.409015+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:39:03.321417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.434222+0000","last_change":"2023-10-10T20:43:30.433269+0000","last_active":"2023-10-10T20:43:30.434222+0000","last_peered":"2023-10-10T20:43:30.434222+0000","last_clean":"2023-10-10T20:43:30.434222+0000","last_became_active":"2023-10-10T20:43:30.432975+0000","last_became_peered":"2023-10-10T20:43:30.432975+0000","last_unstale":"2023-10-10T20:43:30.434222+0000","last_undegraded":"2023-10-10T20:43:30.434222+0000","last_fullsized":"2023-10-10T20:43:30.434222+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:56:19.581374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"182'2","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.428302+0000","last_change":"2023-10-10T20:43:29.421008+0000","last_active":"2023-10-10T20:43:30.428302+0000","last_peered":"2023-10-10T20:43:30.428302+0000","last_clean":"2023-10-10T20:43:30.428302+0000","last_became_active":"2023-10-10T20:43:29.420750+0000","last_became_peered":"2023-10-10T20:43:29.420750+0000","last_unstale":"2023-10-10T20:43:30.428302+0000","last_undegraded":"2023-10-10T20:43:30.428302+0000","last_fullsized":"2023-10-10T20:43:30.428302+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:26.569301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.434514+0000","last_change":"2023-10-10T20:43:30.431660+0000","last_active":"2023-10-10T20:43:30.434514+0000","last_peered":"2023-10-10T20:43:30.434514+0000","last_clean":"2023-10-10T20:43:30.434514+0000","last_became_active":"2023-10-10T20:43:30.430002+0000","last_became_peered":"2023-10-10T20:43:30.430002+0000","last_unstale":"2023-10-10T20:43:30.434514+0000","last_undegraded":"2023-10-10T20:43:30.434514+0000","last_fullsized":"2023-10-10T20:43:30.434514+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:25:59.788466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.410325+0000","last_change":"2023-10-10T20:43:29.426628+0000","last_active":"2023-10-10T20:43:30.410325+0000","last_peered":"2023-10-10T20:43:30.410325+0000","last_clean":"2023-10-10T20:43:30.410325+0000","last_became_active":"2023-10-10T20:43:29.426354+0000","last_became_peered":"2023-10-10T20:43:29.426354+0000","last_unstale":"2023-10-10T20:43:30.410325+0000","last_undegraded":"2023-10-10T20:43:30.410325+0000","last_fullsized":"2023-10-10T20:43:30.410325+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:37:52.611816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.427721+0000","last_change":"2023-10-10T20:43:30.424536+0000","last_active":"2023-10-10T20:43:30.427721+0000","last_peered":"2023-10-10T20:43:30.427721+0000","last_clean":"2023-10-10T20:43:30.427721+0000","last_became_active":"2023-10-10T20:43:30.424309+0000","last_became_peered":"2023-10-10T20:43:30.424309+0000","last_unstale":"2023-10-10T20:43:30.427721+0000","last_undegraded":"2023-10-10T20:43:30.427721+0000","last_fullsized":"2023-10-10T20:43:30.427721+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:07:46.580549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.409928+0000","last_change":"2023-10-10T20:43:29.418676+0000","last_active":"2023-10-10T20:43:30.409928+0000","last_peered":"2023-10-10T20:43:30.409928+0000","last_clean":"2023-10-10T20:43:30.409928+0000","last_became_active":"2023-10-10T20:43:29.418400+0000","last_became_peered":"2023-10-10T20:43:29.418400+0000","last_unstale":"2023-10-10T20:43:30.409928+0000","last_undegraded":"2023-10-10T20:43:30.409928+0000","last_fullsized":"2023-10-10T20:43:30.409928+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:35:27.593942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.419833+0000","last_change":"2023-10-10T20:43:30.412545+0000","last_active":"2023-10-10T20:43:30.419833+0000","last_peered":"2023-10-10T20:43:30.419833+0000","last_clean":"2023-10-10T20:43:30.419833+0000","last_became_active":"2023-10-10T20:43:30.412278+0000","last_became_peered":"2023-10-10T20:43:30.412278+0000","last_unstale":"2023-10-10T20:43:30.419833+0000","last_undegraded":"2023-10-10T20:43:30.419833+0000","last_fullsized":"2023-10-10T20:43:30.419833+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:08:48.517946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.406255+0000","last_change":"2023-10-10T20:43:29.426693+0000","last_active":"2023-10-10T20:43:30.406255+0000","last_peered":"2023-10-10T20:43:30.406255+0000","last_clean":"2023-10-10T20:43:30.406255+0000","last_became_active":"2023-10-10T20:43:29.426227+0000","last_became_peered":"2023-10-10T20:43:29.426227+0000","last_unstale":"2023-10-10T20:43:30.406255+0000","last_undegraded":"2023-10-10T20:43:30.406255+0000","last_fullsized":"2023-10-10T20:43:30.406255+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:19:39.865575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.427305+0000","last_change":"2023-10-10T20:43:30.425492+0000","last_active":"2023-10-10T20:43:30.427305+0000","last_peered":"2023-10-10T20:43:30.427305+0000","last_clean":"2023-10-10T20:43:30.427305+0000","last_became_active":"2023-10-10T20:43:30.425240+0000","last_became_peered":"2023-10-10T20:43:30.425240+0000","last_unstale":"2023-10-10T20:43:30.427305+0000","last_undegraded":"2023-10-10T20:43:30.427305+0000","last_fullsized":"2023-10-10T20:43:30.427305+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:55:10.259346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"182'2","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.428253+0000","last_change":"2023-10-10T20:43:29.422737+0000","last_active":"2023-10-10T20:43:30.428253+0000","last_peered":"2023-10-10T20:43:30.428253+0000","last_clean":"2023-10-10T20:43:30.428253+0000","last_became_active":"2023-10-10T20:43:29.422479+0000","last_became_peered":"2023-10-10T20:43:29.422479+0000","last_unstale":"2023-10-10T20:43:30.428253+0000","last_undegraded":"2023-10-10T20:43:30.428253+0000","last_fullsized":"2023-10-10T20:43:30.428253+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:42:28.587579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.427326+0000","last_change":"2023-10-10T20:43:30.423708+0000","last_active":"2023-10-10T20:43:30.427326+0000","last_peered":"2023-10-10T20:43:30.427326+0000","last_clean":"2023-10-10T20:43:30.427326+0000","last_became_active":"2023-10-10T20:43:30.423225+0000","last_became_peered":"2023-10-10T20:43:30.423225+0000","last_unstale":"2023-10-10T20:43:30.427326+0000","last_undegraded":"2023-10-10T20:43:30.427326+0000","last_fullsized":"2023-10-10T20:43:30.427326+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:05.833137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.417376+0000","last_change":"2023-10-10T20:43:29.420790+0000","last_active":"2023-10-10T20:43:30.417376+0000","last_peered":"2023-10-10T20:43:30.417376+0000","last_clean":"2023-10-10T20:43:30.417376+0000","last_became_active":"2023-10-10T20:43:29.420498+0000","last_became_peered":"2023-10-10T20:43:29.420498+0000","last_unstale":"2023-10-10T20:43:30.417376+0000","last_undegraded":"2023-10-10T20:43:30.417376+0000","last_fullsized":"2023-10-10T20:43:30.417376+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:38:49.789790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.421686+0000","last_change":"2023-10-10T20:43:30.419512+0000","last_active":"2023-10-10T20:43:30.421686+0000","last_peered":"2023-10-10T20:43:30.421686+0000","last_clean":"2023-10-10T20:43:30.421686+0000","last_became_active":"2023-10-10T20:43:30.419281+0000","last_became_peered":"2023-10-10T20:43:30.419281+0000","last_unstale":"2023-10-10T20:43:30.421686+0000","last_undegraded":"2023-10-10T20:43:30.421686+0000","last_fullsized":"2023-10-10T20:43:30.421686+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:43.068714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.406669+0000","last_change":"2023-10-10T20:43:29.420441+0000","last_active":"2023-10-10T20:43:30.406669+0000","last_peered":"2023-10-10T20:43:30.406669+0000","last_clean":"2023-10-10T20:43:30.406669+0000","last_became_active":"2023-10-10T20:43:29.420231+0000","last_became_peered":"2023-10-10T20:43:29.420231+0000","last_unstale":"2023-10-10T20:43:30.406669+0000","last_undegraded":"2023-10-10T20:43:30.406669+0000","last_fullsized":"2023-10-10T20:43:30.406669+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:42.061289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.431225+0000","last_change":"2023-10-10T20:43:30.430183+0000","last_active":"2023-10-10T20:43:30.431225+0000","last_peered":"2023-10-10T20:43:30.431225+0000","last_clean":"2023-10-10T20:43:30.431225+0000","last_became_active":"2023-10-10T20:43:30.429914+0000","last_became_peered":"2023-10-10T20:43:30.429914+0000","last_unstale":"2023-10-10T20:43:30.431225+0000","last_undegraded":"2023-10-10T20:43:30.431225+0000","last_fullsized":"2023-10-10T20:43:30.431225+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:16:08.720700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.407235+0000","last_change":"2023-10-10T20:43:29.426751+0000","last_active":"2023-10-10T20:43:30.407235+0000","last_peered":"2023-10-10T20:43:30.407235+0000","last_clean":"2023-10-10T20:43:30.407235+0000","last_became_active":"2023-10-10T20:43:29.426428+0000","last_became_peered":"2023-10-10T20:43:29.426428+0000","last_unstale":"2023-10-10T20:43:30.407235+0000","last_undegraded":"2023-10-10T20:43:30.407235+0000","last_fullsized":"2023-10-10T20:43:30.407235+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:16:07.713274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.430389+0000","last_change":"2023-10-10T20:43:30.427150+0000","last_active":"2023-10-10T20:43:30.430389+0000","last_peered":"2023-10-10T20:43:30.430389+0000","last_clean":"2023-10-10T20:43:30.430389+0000","last_became_active":"2023-10-10T20:43:30.424490+0000","last_became_peered":"2023-10-10T20:43:30.424490+0000","last_unstale":"2023-10-10T20:43:30.430389+0000","last_undegraded":"2023-10-10T20:43:30.430389+0000","last_fullsized":"2023-10-10T20:43:30.430389+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:36:20.108326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.410178+0000","last_change":"2023-10-10T20:43:29.426622+0000","last_active":"2023-10-10T20:43:30.410178+0000","last_peered":"2023-10-10T20:43:30.410178+0000","last_clean":"2023-10-10T20:43:30.410178+0000","last_became_active":"2023-10-10T20:43:29.426354+0000","last_became_peered":"2023-10-10T20:43:29.426354+0000","last_unstale":"2023-10-10T20:43:30.410178+0000","last_undegraded":"2023-10-10T20:43:30.410178+0000","last_fullsized":"2023-10-10T20:43:30.410178+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:07:24.047955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.430398+0000","last_change":"2023-10-10T20:43:30.427084+0000","last_active":"2023-10-10T20:43:30.430398+0000","last_peered":"2023-10-10T20:43:30.430398+0000","last_clean":"2023-10-10T20:43:30.430398+0000","last_became_active":"2023-10-10T20:43:30.426765+0000","last_became_peered":"2023-10-10T20:43:30.426765+0000","last_unstale":"2023-10-10T20:43:30.430398+0000","last_undegraded":"2023-10-10T20:43:30.430398+0000","last_fullsized":"2023-10-10T20:43:30.430398+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:07:40.366716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.416577+0000","last_change":"2023-10-10T20:43:29.424368+0000","last_active":"2023-10-10T20:43:30.416577+0000","last_peered":"2023-10-10T20:43:30.416577+0000","last_clean":"2023-10-10T20:43:30.416577+0000","last_became_active":"2023-10-10T20:43:29.424027+0000","last_became_peered":"2023-10-10T20:43:29.424027+0000","last_unstale":"2023-10-10T20:43:30.416577+0000","last_undegraded":"2023-10-10T20:43:30.416577+0000","last_fullsized":"2023-10-10T20:43:30.416577+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:04:18.576005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.434810+0000","last_change":"2023-10-10T20:43:30.432157+0000","last_active":"2023-10-10T20:43:30.434810+0000","last_peered":"2023-10-10T20:43:30.434810+0000","last_clean":"2023-10-10T20:43:30.434810+0000","last_became_active":"2023-10-10T20:43:30.430068+0000","last_became_peered":"2023-10-10T20:43:30.430068+0000","last_unstale":"2023-10-10T20:43:30.434810+0000","last_undegraded":"2023-10-10T20:43:30.434810+0000","last_fullsized":"2023-10-10T20:43:30.434810+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:28:34.680119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.416551+0000","last_change":"2023-10-10T20:43:29.426799+0000","last_active":"2023-10-10T20:43:30.416551+0000","last_peered":"2023-10-10T20:43:30.416551+0000","last_clean":"2023-10-10T20:43:30.416551+0000","last_became_active":"2023-10-10T20:43:29.426519+0000","last_became_peered":"2023-10-10T20:43:29.426519+0000","last_unstale":"2023-10-10T20:43:30.416551+0000","last_undegraded":"2023-10-10T20:43:30.416551+0000","last_fullsized":"2023-10-10T20:43:30.416551+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:03.496901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.431374+0000","last_change":"2023-10-10T20:43:30.429716+0000","last_active":"2023-10-10T20:43:30.431374+0000","last_peered":"2023-10-10T20:43:30.431374+0000","last_clean":"2023-10-10T20:43:30.431374+0000","last_became_active":"2023-10-10T20:43:30.429308+0000","last_became_peered":"2023-10-10T20:43:30.429308+0000","last_unstale":"2023-10-10T20:43:30.431374+0000","last_undegraded":"2023-10-10T20:43:30.431374+0000","last_fullsized":"2023-10-10T20:43:30.431374+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:24:15.196360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.409629+0000","last_change":"2023-10-10T20:43:29.421452+0000","last_active":"2023-10-10T20:43:30.409629+0000","last_peered":"2023-10-10T20:43:30.409629+0000","last_clean":"2023-10-10T20:43:30.409629+0000","last_became_active":"2023-10-10T20:43:29.421205+0000","last_became_peered":"2023-10-10T20:43:29.421205+0000","last_unstale":"2023-10-10T20:43:30.409629+0000","last_undegraded":"2023-10-10T20:43:30.409629+0000","last_fullsized":"2023-10-10T20:43:30.409629+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:51:35.125860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.430198+0000","last_change":"2023-10-10T20:43:30.426129+0000","last_active":"2023-10-10T20:43:30.430198+0000","last_peered":"2023-10-10T20:43:30.430198+0000","last_clean":"2023-10-10T20:43:30.430198+0000","last_became_active":"2023-10-10T20:43:30.425729+0000","last_became_peered":"2023-10-10T20:43:30.425729+0000","last_unstale":"2023-10-10T20:43:30.430198+0000","last_undegraded":"2023-10-10T20:43:30.430198+0000","last_fullsized":"2023-10-10T20:43:30.430198+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:29.395425+0000","last_clean_scrub_stamp":"2023-10-10T20:43:29.395425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:55:34.727555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"182'1","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.427454+0000","last_change":"2023-10-10T20:43:29.425173+0000","last_active":"2023-10-10T20:43:30.427454+0000","last_peered":"2023-10-10T20:43:30.427454+0000","last_clean":"2023-10-10T20:43:30.427454+0000","last_became_active":"2023-10-10T20:43:29.424899+0000","last_became_peered":"2023-10-10T20:43:29.424899+0000","last_unstale":"2023-10-10T20:43:30.427454+0000","last_undegraded":"2023-10-10T20:43:30.427454+0000","last_fullsized":"2023-10-10T20:43:30.427454+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:28.387999+0000","last_clean_scrub_stamp":"2023-10-10T20:43:28.387999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:34:39.774720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":326,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-10T20:43:30.407875+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:43:30.407875+0000","last_peered":"2023-10-10T20:43:30.407875+0000","last_clean":"2023-10-10T20:43:30.407875+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:43:30.407875+0000","last_undegraded":"2023-10-10T20:43:30.407875+0000","last_fullsized":"2023-10-10T20:43:30.407875+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574961,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304244,"kb_used_data":2860,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067596,"statfs":{"total":96636764160,"available":96325218304,"internally_reserved":0,"allocated":2928640,"data_stored":1636580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50700000000000001,"15min":0.75800000000000001},"min":{"1min":0.32500000000000001,"5min":0.309,"15min":0.29199999999999998},"max":{"1min":0.64300000000000002,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.50800000000000001,"15min":0.60399999999999998},"min":{"1min":0.36399999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.58299999999999996,"5min":0.752,"15min":4.1959999999999997},"last":0.505}]},{"osd":1,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.50600000000000001,"15min":0.90100000000000002},"min":{"1min":0.372,"5min":0.317,"15min":0.317},"max":{"1min":0.61099999999999999,"5min":0.78800000000000003,"15min":10.1},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.53400000000000003,"15min":0.69099999999999995},"min":{"1min":0.35199999999999998,"5min":0.33900000000000002,"15min":0.28999999999999998},"max":{"1min":0.66500000000000004,"5min":0.81799999999999995,"15min":6.806},"last":0.57499999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56799999999999995,"15min":0.74099999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68799999999999994,"5min":0.89000000000000001,"15min":6.9340000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.69399999999999995},"min":{"1min":0.40699999999999997,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.69099999999999995,"5min":0.77100000000000002,"15min":6.9870000000000001},"last":0.63500000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54200000000000004,"15min":0.71299999999999997},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64500000000000002,"5min":0.754,"15min":6.6719999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55900000000000005,"15min":0.64800000000000002},"min":{"1min":0.39800000000000002,"5min":0.378,"15min":0.34799999999999998},"max":{"1min":0.64400000000000002,"5min":0.76200000000000001,"15min":4.1609999999999996},"last":0.65500000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47799999999999998,"15min":0.46800000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.77200000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.51600000000000001,"15min":0.51900000000000002},"min":{"1min":0.314,"5min":0.309,"15min":0.30399999999999999},"max":{"1min":0.68000000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.39600000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.54000000000000004,"15min":0.53900000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.68400000000000005,"5min":0.86099999999999999,"15min":1.3180000000000001},"last":0.442},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.53600000000000003,"15min":0.52700000000000002},"min":{"1min":0.35299999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.84099999999999997},"last":0.41599999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.35699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":1.7509999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76300000000000001,"5min":0.84099999999999997,"15min":1.7090000000000001},"last":0.48899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574962,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303664,"kb_used_data":2280,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068176,"statfs":{"total":96636764160,"available":96325812224,"internally_reserved":0,"allocated":2334720,"data_stored":1046200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52400000000000002,"15min":0.64000000000000001},"min":{"1min":0.34699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":7.7590000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52400000000000002,"15min":0.67400000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.86899999999999999,"5min":1.075,"15min":7.8090000000000002},"last":0.77000000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52900000000000003,"15min":0.67200000000000004},"min":{"1min":0.32800000000000001,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":7.867},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55500000000000005,"15min":0.81999999999999995},"min":{"1min":0.378,"5min":0.35499999999999998,"15min":0.33700000000000002},"max":{"1min":0.70999999999999996,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.73499999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52300000000000002,"15min":0.58099999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.745,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57099999999999995,"15min":0.79100000000000004},"min":{"1min":0.39700000000000002,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":2.641,"15min":6.7629999999999999},"last":0.748}]},{"osd":3,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55500000000000005,"15min":0.60699999999999998},"min":{"1min":0.372,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94399999999999995,"5min":1.107,"15min":4.8620000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55500000000000005,"15min":0.68400000000000005},"min":{"1min":0.41399999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.72599999999999998,"5min":1.718,"15min":7.9009999999999998},"last":0.68200000000000005}]},{"osd":4,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51900000000000002,"15min":0.47899999999999998},"min":{"1min":0.32000000000000001,"5min":0.28999999999999998,"15min":0.28499999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":0.82899999999999996,"5min":1.032,"15min":1.032},"last":0.64500000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53400000000000003,"15min":0.5},"min":{"1min":0.34999999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.86799999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56999999999999995,"15min":0.53700000000000003},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85399999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.60999999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56200000000000006,"15min":0.53000000000000003},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.89600000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.59399999999999997,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.372,"15min":0.35099999999999998},"max":{"1min":0.82199999999999995,"5min":1.681,"15min":1.681},"last":0.57599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303656,"kb_used_data":2272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068184,"statfs":{"total":96636764160,"available":96325820416,"internally_reserved":0,"allocated":2326528,"data_stored":1046088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.31900000000000001,"15min":0.443},"min":{"1min":0.215,"5min":0.19600000000000001,"15min":0.17899999999999999},"max":{"1min":0.47799999999999998,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.315},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33000000000000002,"15min":0.35599999999999998},"min":{"1min":0.22500000000000001,"5min":0.224,"15min":0.20899999999999999},"max":{"1min":0.441,"5min":0.65500000000000003,"15min":1.194},"last":0.36799999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.32700000000000001,"15min":0.35499999999999998},"min":{"1min":0.246,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.41699999999999998,"5min":0.51400000000000001,"15min":1.2869999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.35199999999999998,"15min":0.47299999999999998},"min":{"1min":0.23999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.49199999999999999,"5min":0.54200000000000004,"15min":5.0919999999999996},"last":0.39500000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.374,"15min":0.496},"min":{"1min":0.25600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54800000000000004,"5min":0.59899999999999998,"15min":5.1159999999999997},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.36299999999999999,"15min":0.39800000000000002},"min":{"1min":0.26500000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.52600000000000002,"5min":0.57799999999999996,"15min":1.2170000000000001},"last":0.40600000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.38400000000000001,"15min":0.30199999999999999},"max":{"1min":0.78600000000000003,"5min":0.92400000000000004,"15min":1.254},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55900000000000005,"15min":0.67500000000000004},"min":{"1min":0.39600000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68100000000000005,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.78900000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.40400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.97599999999999998,"15min":1.1799999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56999999999999995,"15min":0.59999999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73299999999999998,"5min":0.878,"15min":1.3109999999999999},"last":0.79700000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":0.79300000000000004},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70399999999999996,"5min":0.995,"15min":5.0659999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60699999999999998,"15min":0.628},"min":{"1min":0.46400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84199999999999997,"5min":0.89800000000000002,"15min":1.171},"last":0.749}]},{"osd":7,"last update":"Tue Oct 10 20:42:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.48299999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77100000000000002,"5min":1.004,"15min":1.214},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58399999999999996,"15min":0.69699999999999995},"min":{"1min":0.40899999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.754,"5min":0.90500000000000003,"15min":4.0499999999999998},"last":0.754}]}]},{"osd":0,"up_from":13,"seq":55834574960,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304244,"kb_used_data":2860,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067596,"statfs":{"total":96636764160,"available":96325218304,"internally_reserved":0,"allocated":2928640,"data_stored":1637031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.315,"15min":0.48599999999999999},"min":{"1min":0.217,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":7.9249999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.46899999999999997},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46100000000000002,"5min":0.53800000000000003,"15min":6.6299999999999999},"last":0.35999999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33900000000000002,"15min":0.51000000000000001},"min":{"1min":0.25600000000000001,"5min":0.22,"15min":0.216},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":8.2509999999999994},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.49299999999999999},"min":{"1min":0.27500000000000002,"5min":0.223,"15min":0.222},"max":{"1min":0.498,"5min":0.57899999999999996,"15min":6.7370000000000001},"last":0.309}]},{"osd":3,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.35499999999999998,"15min":0.51700000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.21099999999999999},"max":{"1min":0.438,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.57099999999999995},"min":{"1min":0.317,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":7.8929999999999998},"last":0.371}]},{"osd":4,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57099999999999995,"15min":0.68700000000000006},"min":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.78100000000000003,"5min":0.88900000000000001,"15min":6.6929999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57199999999999995,"15min":0.73099999999999998},"min":{"1min":0.371,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.69599999999999995,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.57499999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56499999999999995,"15min":0.71399999999999997},"min":{"1min":0.40100000000000002,"5min":0.376,"15min":0.36299999999999999},"max":{"1min":0.72699999999999998,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.74199999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.91400000000000003,"15min":7.8659999999999997},"last":0.53000000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61499999999999999,"15min":0.77600000000000002},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.73299999999999998},"min":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.36599999999999999},"max":{"1min":0.82099999999999995,"5min":0.92000000000000004,"15min":6.6660000000000004},"last":0.628}]},{"osd":7,"last update":"Tue Oct 10 20:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60899999999999999,"15min":0.72399999999999998},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.36299999999999999},"max":{"1min":0.85499999999999998,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59799999999999998,"15min":0.747},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89500000000000002,"5min":0.93100000000000005,"15min":7.7370000000000001},"last":0.53800000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574962,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303664,"kb_used_data":2280,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068176,"statfs":{"total":96636764160,"available":96325812224,"internally_reserved":0,"allocated":2334720,"data_stored":1046712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.311,"15min":0.32400000000000001},"min":{"1min":0.20200000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.48599999999999999,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32100000000000001,"15min":0.34899999999999998},"min":{"1min":0.224,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51800000000000002,"5min":0.54200000000000004,"15min":0.746},"last":0.33800000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33200000000000002,"15min":0.65200000000000002},"min":{"1min":0.22600000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52200000000000002,"5min":0.63,"15min":11.967000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.35199999999999998,"15min":0.45800000000000002},"min":{"1min":0.245,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.505,"5min":0.52700000000000002,"15min":4.1100000000000003},"last":0.39800000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.51600000000000001},"min":{"1min":0.23699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":6.3040000000000003},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.38700000000000001,"15min":0.70399999999999996},"min":{"1min":0.29499999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57799999999999996,"5min":0.65900000000000003,"15min":11.935},"last":0.38400000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.42999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91900000000000004,"5min":1.302,"15min":1.302},"last":0.55700000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.378},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.95699999999999996},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.39700000000000002,"5min":0.35999999999999999,"15min":0.34300000000000003},"max":{"1min":0.78800000000000003,"5min":0.83999999999999997,"15min":0.96099999999999997},"last":0.60799999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.33400000000000002,"15min":0.33200000000000002},"max":{"1min":0.95499999999999996,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60399999999999998,"15min":0.60499999999999998},"min":{"1min":0.47499999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.63200000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.46100000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85699999999999998,"5min":0.89700000000000002,"15min":1.052},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58899999999999997,"15min":0.746},"min":{"1min":0.40999999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.98099999999999998,"5min":1.274,"15min":6.2649999999999997},"last":0.621}]}]},{"osd":3,"up_from":13,"seq":55834574962,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303680,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068160,"statfs":{"total":96636764160,"available":96325795840,"internally_reserved":0,"allocated":2351104,"data_stored":1048234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.309,"15min":0.314},"min":{"1min":0.22600000000000001,"5min":0.21299999999999999,"15min":0.21099999999999999},"max":{"1min":0.38900000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.316,"5min":0.32700000000000001,"15min":0.32100000000000001},"min":{"1min":0.23799999999999999,"5min":0.216,"15min":0.19800000000000001},"max":{"1min":0.42799999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.253}]},{"osd":1,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.34399999999999997,"15min":0.33400000000000002},"min":{"1min":0.26200000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.47299999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38300000000000001,"15min":0.378},"min":{"1min":0.26800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.48599999999999999,"5min":0.66900000000000004,"15min":0.72099999999999997},"last":0.35699999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.36299999999999999,"15min":0.35299999999999998},"min":{"1min":0.28899999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44800000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.312},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.28799999999999998,"5min":0.25900000000000001,"15min":0.255},"max":{"1min":0.441,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.379}]},{"osd":4,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56100000000000005,"15min":0.53800000000000003},"min":{"1min":0.44900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56699999999999995,"15min":0.53900000000000003},"min":{"1min":0.433,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.79400000000000004,"5min":0.83299999999999996,"15min":0.93500000000000005},"last":0.46999999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.55300000000000005},"min":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.35899999999999999},"max":{"1min":0.69599999999999995,"5min":1.347,"15min":1.347},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.443,"5min":0.39300000000000002,"15min":0.35799999999999998},"max":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":1.01},"last":0.52300000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.45500000000000002,"5min":0.41099999999999998,"15min":0.36699999999999999},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.48899999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79700000000000004,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.56899999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:42:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61599999999999999,"15min":0.59599999999999997},"min":{"1min":0.495,"5min":0.42099999999999999,"15min":0.38200000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.96999999999999997},"last":0.57999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574960,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303660,"kb_used_data":2276,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068180,"statfs":{"total":96636764160,"available":96325816320,"internally_reserved":0,"allocated":2330624,"data_stored":1046158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.372,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66700000000000004,"5min":0.74199999999999999,"15min":0.747},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52500000000000002,"15min":0.71899999999999997},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.32700000000000001},"max":{"1min":0.68300000000000005,"5min":0.77600000000000002,"15min":7.6719999999999997},"last":0.73099999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52600000000000002,"15min":0.72399999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.755,"15min":7.7160000000000002},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.371,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":0.86799999999999999},"last":0.48699999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67800000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.749},"min":{"1min":0.39900000000000002,"5min":0.39700000000000002,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":7.7839999999999998},"last":0.755}]},{"osd":3,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.47999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74399999999999999,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.42999999999999999,"5min":0.39200000000000002,"15min":0.35499999999999998},"max":{"1min":0.71499999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.77100000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.497},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.30099999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.54700000000000004},"min":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.46400000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.35399999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.77300000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86199999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.81699999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.378,"15min":0.36199999999999999},"max":{"1min":0.88600000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.41699999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.874,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.86899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574962,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303656,"kb_used_data":2272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068184,"statfs":{"total":96636764160,"available":96325820416,"internally_reserved":0,"allocated":2326528,"data_stored":1046088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.498,"15min":0.49199999999999999},"min":{"1min":0.38300000000000001,"5min":0.27900000000000003,"15min":0.20300000000000001},"max":{"1min":0.69699999999999995,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.44},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53100000000000003,"15min":0.52000000000000002},"min":{"1min":0.39400000000000002,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.72399999999999998,"5min":0.76700000000000002,"15min":0.79300000000000004},"last":0.48099999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51400000000000001,"15min":0.61799999999999999},"min":{"1min":0.35099999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70799999999999996,"5min":0.746,"15min":5.2679999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53000000000000003,"15min":0.61099999999999999},"min":{"1min":0.442,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.65700000000000003,"5min":0.79400000000000004,"15min":5.3109999999999999},"last":0.37}]},{"osd":2,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.56200000000000006,"15min":0.63800000000000001},"min":{"1min":0.45500000000000002,"5min":0.32900000000000001,"15min":0.28299999999999997},"max":{"1min":3.27,"5min":3.27,"15min":5.2960000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55000000000000004,"15min":0.54100000000000004},"min":{"1min":0.40999999999999998,"5min":0.33900000000000002,"15min":0.32000000000000001},"max":{"1min":0.75,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.54400000000000004}]},{"osd":3,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55700000000000005,"15min":0.65700000000000003},"min":{"1min":0.47499999999999998,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.69799999999999995,"5min":0.94599999999999995,"15min":5.3330000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57399999999999995,"15min":0.66800000000000004},"min":{"1min":0.45200000000000001,"5min":0.39300000000000002,"15min":0.27700000000000002},"max":{"1min":0.79300000000000004,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.42199999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.49199999999999999,"15min":0.497},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67300000000000004,"5min":0.72499999999999998,"15min":0.82999999999999996},"last":0.317},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53100000000000003,"15min":0.54300000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32200000000000001},"max":{"1min":0.83099999999999996,"5min":0.88,"15min":1.0800000000000001},"last":0.38700000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.86499999999999999,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.49399999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56200000000000006,"15min":0.57499999999999996},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.30099999999999999},"max":{"1min":0.78700000000000003,"5min":0.98299999999999998,"15min":0.98399999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.378,"5min":0.377,"15min":0.377},"max":{"1min":0.83199999999999996,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.46400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:43:34.848 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:43:34.849 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:43:35.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.001+0000 7fb9cf526700 1 Processor -- start 2023-10-10T20:43:35.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.001+0000 7fb9cf526700 1 -- start start 2023-10-10T20:43:35.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.002+0000 7fb9cf526700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb9c8069c70 0x7fb9c80cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:35.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.002+0000 7fb9cf526700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb9c80af320 0x7fb9c80cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:35.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.002+0000 7fb9cf526700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c80ce380 con 0x7fb9c8159950 2023-10-10T20:43:35.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.002+0000 7fb9cf526700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c80cec20 con 0x7fb9c8069c70 2023-10-10T20:43:35.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.002+0000 7fb9cf526700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c80afa90 con 0x7fb9c80af320 2023-10-10T20:43:35.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.002+0000 7fb9ccac1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb9c8069c70 0x7fb9c80cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:35.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.002+0000 7fb9ccac1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb9c8069c70 0x7fb9c80cbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37344/0 (socket says 172.21.15.89:37344) 2023-10-10T20:43:35.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.002+0000 7fb9ccac1700 1 -- 172.21.15.89:0/1178522322 learned_addr learned my addr 172.21.15.89:0/1178522322 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:35.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.002+0000 7fb9cdac3700 1 --2- 172.21.15.89:0/1178522322 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb9c80af320 0x7fb9c80cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:35.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.002+0000 7fb9ccac1700 1 -- 172.21.15.89:0/1178522322 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb9c80af320 msgr2=0x7fb9c80cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:35.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.002+0000 7fb9ccac1700 1 --2- 172.21.15.89:0/1178522322 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb9c80af320 0x7fb9c80cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:35.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.002+0000 7fb9ccac1700 1 -- 172.21.15.89:0/1178522322 >> v1:172.21.15.89:6789/0 conn(0x7fb9c8159950 legacy=0x7fb9c8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:35.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.002+0000 7fb9ccac1700 1 -- 172.21.15.89:0/1178522322 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c80ce5d0 con 0x7fb9c8069c70 2023-10-10T20:43:35.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.002+0000 7fb9bffff700 1 -- 172.21.15.89:0/1178522322 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404640018 0 0) 0x7fb9c80ce380 con 0x7fb9c8159950 2023-10-10T20:43:35.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.002+0000 7fb9ccac1700 1 --2- 172.21.15.89:0/1178522322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c8069c70 0x7fb9c80cbb90 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80099b0 tx=0x7fb9b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ffdb64e13bd760 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:35.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.003+0000 7fb9bffff700 1 -- 172.21.15.89:0/1178522322 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800b860 con 0x7fb9c8069c70 2023-10-10T20:43:35.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.003+0000 7fb9bffff700 1 -- 172.21.15.89:0/1178522322 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b800b9c0 con 0x7fb9c8069c70 2023-10-10T20:43:35.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.003+0000 7fb9bffff700 1 -- 172.21.15.89:0/1178522322 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800bd60 con 0x7fb9c8069c70 2023-10-10T20:43:35.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.003+0000 7fb9cf526700 1 -- 172.21.15.89:0/1178522322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c8069c70 msgr2=0x7fb9c80cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:35.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.003+0000 7fb9cf526700 1 --2- 172.21.15.89:0/1178522322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c8069c70 0x7fb9c80cbb90 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80099b0 tx=0x7fb9b8009250 comp rx=0 tx=0).stop 2023-10-10T20:43:35.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.003+0000 7fb9cf526700 1 -- 172.21.15.89:0/1178522322 shutdown_connections 2023-10-10T20:43:35.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.003+0000 7fb9cf526700 1 --2- 172.21.15.89:0/1178522322 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb9c80af320 0x7fb9c80cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:35.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.003+0000 7fb9cf526700 1 --2- 172.21.15.89:0/1178522322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c8069c70 0x7fb9c80cbb90 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:35.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.003+0000 7fb9cf526700 1 -- 172.21.15.89:0/1178522322 >> 172.21.15.89:0/1178522322 conn(0x7fb9c8068e00 msgr2=0x7fb9c80d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:35.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.003+0000 7fb9cf526700 1 -- 172.21.15.89:0/1178522322 shutdown_connections 2023-10-10T20:43:35.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.003+0000 7fb9cf526700 1 -- 172.21.15.89:0/1178522322 wait complete. 2023-10-10T20:43:35.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.004+0000 7fb9cf526700 1 Processor -- start 2023-10-10T20:43:35.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.004+0000 7fb9cf526700 1 -- start start 2023-10-10T20:43:35.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.005+0000 7fb9cf526700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb9c8069c70 0x7fb9c814d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:35.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.005+0000 7fb9cf526700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9c80af320 0x7fb9c80b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.005+0000 7fb9cf526700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c8159950 0x7fb9c814df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.005+0000 7fb9cf526700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c80c8a30 con 0x7fb9c8159950 2023-10-10T20:43:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.005+0000 7fb9cf526700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c80c7e20 con 0x7fb9c80af320 2023-10-10T20:43:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.005+0000 7fb9cf526700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c80c8100 con 0x7fb9c8069c70 2023-10-10T20:43:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.005+0000 7fb9cdac3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c8159950 0x7fb9c814df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:35.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.005+0000 7fb9cdac3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c8159950 0x7fb9c814df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37356/0 (socket says 172.21.15.89:37356) 2023-10-10T20:43:35.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.005+0000 7fb9cdac3700 1 -- 172.21.15.89:0/562154571 learned_addr learned my addr 172.21.15.89:0/562154571 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:35.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.006+0000 7fb9ccac1700 1 --2- 172.21.15.89:0/562154571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9c80af320 0x7fb9c80b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:35.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.006+0000 7fb9cdac3700 1 -- 172.21.15.89:0/562154571 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb9c8069c70 msgr2=0x7fb9c814d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:35.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.006+0000 7fb9cdac3700 1 --2- 172.21.15.89:0/562154571 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb9c8069c70 0x7fb9c814d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:35.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.006+0000 7fb9cdac3700 1 -- 172.21.15.89:0/562154571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9c80af320 msgr2=0x7fb9c80b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:35.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.006+0000 7fb9cdac3700 1 --2- 172.21.15.89:0/562154571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9c80af320 0x7fb9c80b5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:35.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.006+0000 7fb9cdac3700 1 -- 172.21.15.89:0/562154571 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b80094f0 con 0x7fb9c8159950 2023-10-10T20:43:35.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.006+0000 7fb9cdac3700 1 --2- 172.21.15.89:0/562154571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c8159950 0x7fb9c814df00 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400d890 tx=0x7fb9c4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:35.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.006+0000 7fb9bd7fa700 1 -- 172.21.15.89:0/562154571 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c40125f0 con 0x7fb9c8159950 2023-10-10T20:43:35.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.006+0000 7fb9bd7fa700 1 -- 172.21.15.89:0/562154571 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c4012770 con 0x7fb9c8159950 2023-10-10T20:43:35.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.006+0000 7fb9cf526700 1 -- 172.21.15.89:0/562154571 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c80c83e0 con 0x7fb9c8159950 2023-10-10T20:43:35.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.006+0000 7fb9bd7fa700 1 -- 172.21.15.89:0/562154571 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4012b30 con 0x7fb9c8159950 2023-10-10T20:43:35.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.006+0000 7fb9cf526700 1 -- 172.21.15.89:0/562154571 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c814db80 con 0x7fb9c8159950 2023-10-10T20:43:35.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.007+0000 7fb9cf526700 1 -- 172.21.15.89:0/562154571 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c80718b0 con 0x7fb9c8159950 2023-10-10T20:43:35.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.007+0000 7fb9bd7fa700 1 -- 172.21.15.89:0/562154571 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb9c401e460 con 0x7fb9c8159950 2023-10-10T20:43:35.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.008+0000 7fb9bd7fa700 1 --2- 172.21.15.89:0/562154571 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb9ac067320 0x7fb9ac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:35.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.008+0000 7fb9bd7fa700 1 -- 172.21.15.89:0/562154571 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb9c4085290 con 0x7fb9c8159950 2023-10-10T20:43:35.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.008+0000 7fb9cd2c2700 1 --2- 172.21.15.89:0/562154571 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb9ac067320 0x7fb9ac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:35.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.008+0000 7fb9cd2c2700 1 --2- 172.21.15.89:0/562154571 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb9ac067320 0x7fb9ac0697d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0001d30 tx=0x7fb9c0007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.012+0000 7fb9bd7fa700 1 -- 172.21.15.89:0/562154571 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9c4051f20 con 0x7fb9c8159950 2023-10-10T20:43:35.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.270+0000 7fb9cf526700 1 -- 172.21.15.89:0/562154571 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb9c80ce8b0 con 0x7fb9c8159950 2023-10-10T20:43:35.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.273+0000 7fb9bd7fa700 1 -- 172.21.15.89:0/562154571 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v26) v1 ==== 190+0+0 (secure 0 0 0) 0x7fb9c4055540 con 0x7fb9c8159950 2023-10-10T20:43:35.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.276+0000 7fb9cf526700 1 -- 172.21.15.89:0/562154571 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb9ac067320 msgr2=0x7fb9ac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:35.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.276+0000 7fb9cf526700 1 --2- 172.21.15.89:0/562154571 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb9ac067320 0x7fb9ac0697d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0001d30 tx=0x7fb9c0007ec0 comp rx=0 tx=0).stop 2023-10-10T20:43:35.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.276+0000 7fb9cf526700 1 -- 172.21.15.89:0/562154571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c8159950 msgr2=0x7fb9c814df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:35.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.276+0000 7fb9cf526700 1 --2- 172.21.15.89:0/562154571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c8159950 0x7fb9c814df00 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400d890 tx=0x7fb9c4008040 comp rx=0 tx=0).stop 2023-10-10T20:43:35.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.276+0000 7fb9cf526700 1 -- 172.21.15.89:0/562154571 shutdown_connections 2023-10-10T20:43:35.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.276+0000 7fb9cf526700 1 --2- 172.21.15.89:0/562154571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c8159950 0x7fb9c814df00 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:35.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.276+0000 7fb9cf526700 1 --2- 172.21.15.89:0/562154571 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb9c8069c70 0x7fb9c814d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:35.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.276+0000 7fb9cf526700 1 --2- 172.21.15.89:0/562154571 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb9ac067320 0x7fb9ac0697d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:35.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.276+0000 7fb9cf526700 1 --2- 172.21.15.89:0/562154571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9c80af320 0x7fb9c80b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.276+0000 7fb9cf526700 1 -- 172.21.15.89:0/562154571 >> 172.21.15.89:0/562154571 conn(0x7fb9c8068e00 msgr2=0x7fb9c80d1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.276+0000 7fb9cf526700 1 -- 172.21.15.89:0/562154571 shutdown_connections 2023-10-10T20:43:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.276+0000 7fb9cf526700 1 -- 172.21.15.89:0/562154571 wait complete. 2023-10-10T20:43:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:updated caps for client.0 2023-10-10T20:43:35.290 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:43:35.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.402+0000 7f2338589700 1 Processor -- start 2023-10-10T20:43:35.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.402+0000 7f2338589700 1 -- start start 2023-10-10T20:43:35.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.403+0000 7f2338589700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f233006bcd0 0x7f2330140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:35.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.403+0000 7f2338589700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23300c9510 0x7f2330141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:35.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.403+0000 7f2338589700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233015c2a0 con 0x7f233006c0a0 2023-10-10T20:43:35.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.404+0000 7f2338589700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f233015b690 con 0x7f233006bcd0 2023-10-10T20:43:35.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.404+0000 7f2338589700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233015b970 con 0x7f23300c9510 2023-10-10T20:43:35.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.404+0000 7f2336325700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f233006bcd0 0x7f2330140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:35.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.404+0000 7f2336325700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f233006bcd0 0x7f2330140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37364/0 (socket says 172.21.15.89:37364) 2023-10-10T20:43:35.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.404+0000 7f2336325700 1 -- 172.21.15.89:0/4035875918 learned_addr learned my addr 172.21.15.89:0/4035875918 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:35.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.404+0000 7f2336b26700 1 --2- 172.21.15.89:0/4035875918 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23300c9510 0x7f2330141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:35.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.404+0000 7f2336325700 1 -- 172.21.15.89:0/4035875918 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23300c9510 msgr2=0x7f2330141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:35.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.404+0000 7f2336325700 1 --2- 172.21.15.89:0/4035875918 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23300c9510 0x7f2330141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:35.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.404+0000 7f2336325700 1 -- 172.21.15.89:0/4035875918 >> v1:172.21.15.111:6789/0 conn(0x7f233006c0a0 legacy=0x7f23300c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:35.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.404+0000 7f2336325700 1 -- 172.21.15.89:0/4035875918 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233015bc50 con 0x7f233006bcd0 2023-10-10T20:43:35.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.404+0000 7f2336b26700 1 --2- 172.21.15.89:0/4035875918 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23300c9510 0x7f2330141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:35.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.404+0000 7f2336325700 1 --2- 172.21.15.89:0/4035875918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f233006bcd0 0x7f2330140df0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f232c0099b0 tx=0x7f232c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294a103cd5b854a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:35.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.404+0000 7f2335323700 1 -- 172.21.15.89:0/4035875918 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c00b860 con 0x7f233006bcd0 2023-10-10T20:43:35.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.404+0000 7f2335323700 1 -- 172.21.15.89:0/4035875918 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232c00b9c0 con 0x7f233006bcd0 2023-10-10T20:43:35.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.404+0000 7f2335323700 1 -- 172.21.15.89:0/4035875918 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c00bd60 con 0x7f233006bcd0 2023-10-10T20:43:35.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.405+0000 7f2338589700 1 -- 172.21.15.89:0/4035875918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f233006bcd0 msgr2=0x7f2330140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:35.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.405+0000 7f2338589700 1 --2- 172.21.15.89:0/4035875918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f233006bcd0 0x7f2330140df0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f232c0099b0 tx=0x7f232c009250 comp rx=0 tx=0).stop 2023-10-10T20:43:35.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.405+0000 7f2338589700 1 -- 172.21.15.89:0/4035875918 shutdown_connections 2023-10-10T20:43:35.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.405+0000 7f2338589700 1 --2- 172.21.15.89:0/4035875918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f233006bcd0 0x7f2330140df0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:35.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.405+0000 7f2338589700 1 --2- 172.21.15.89:0/4035875918 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23300c9510 0x7f2330141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:35.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.405+0000 7f2338589700 1 -- 172.21.15.89:0/4035875918 >> 172.21.15.89:0/4035875918 conn(0x7f23300ce600 msgr2=0x7f23300cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:35.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.405+0000 7f2338589700 1 -- 172.21.15.89:0/4035875918 shutdown_connections 2023-10-10T20:43:35.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.405+0000 7f2338589700 1 -- 172.21.15.89:0/4035875918 wait complete. 2023-10-10T20:43:35.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.406+0000 7f2338589700 1 Processor -- start 2023-10-10T20:43:35.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.406+0000 7f2338589700 1 -- start start 2023-10-10T20:43:35.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f2338589700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f233006bcd0 0x7f23300b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:35.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f2338589700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f233006c0a0 0x7f23300c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:35.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f2338589700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23300c9510 0x7f23300c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:35.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f2338589700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23300c5bf0 con 0x7f233006bcd0 2023-10-10T20:43:35.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f2338589700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23300c5e90 con 0x7f233006c0a0 2023-10-10T20:43:35.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f2338589700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23300c6170 con 0x7f23300c9510 2023-10-10T20:43:35.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f2336325700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f233006bcd0 0x7f23300b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:35.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f2336325700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f233006bcd0 0x7f23300b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37366/0 (socket says 172.21.15.89:37366) 2023-10-10T20:43:35.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f2336325700 1 -- 172.21.15.89:0/3873013134 learned_addr learned my addr 172.21.15.89:0/3873013134 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:35.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f2336325700 1 -- 172.21.15.89:0/3873013134 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23300c9510 msgr2=0x7f23300c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:35.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f2336325700 1 --2- 172.21.15.89:0/3873013134 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23300c9510 0x7f23300c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:35.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f2336325700 1 -- 172.21.15.89:0/3873013134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f233006c0a0 msgr2=0x7f23300c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:35.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f2335b24700 1 --2- 172.21.15.89:0/3873013134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f233006c0a0 0x7f23300c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:35.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f2336325700 1 --2- 172.21.15.89:0/3873013134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f233006c0a0 0x7f23300c56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:35.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f2336325700 1 -- 172.21.15.89:0/3873013134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c0094f0 con 0x7f233006bcd0 2023-10-10T20:43:35.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f2336325700 1 --2- 172.21.15.89:0/3873013134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f233006bcd0 0x7f23300b3e40 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f232c009290 tx=0x7f232c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:35.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f23267fc700 1 -- 172.21.15.89:0/3873013134 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c026480 con 0x7f233006bcd0 2023-10-10T20:43:35.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f23267fc700 1 -- 172.21.15.89:0/3873013134 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232c01f070 con 0x7f233006bcd0 2023-10-10T20:43:35.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f2338589700 1 -- 172.21.15.89:0/3873013134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23300c6450 con 0x7f233006bcd0 2023-10-10T20:43:35.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.407+0000 7f23267fc700 1 -- 172.21.15.89:0/3873013134 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c026830 con 0x7f233006bcd0 2023-10-10T20:43:35.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.408+0000 7f2338589700 1 -- 172.21.15.89:0/3873013134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23300c6a10 con 0x7f233006bcd0 2023-10-10T20:43:35.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.408+0000 7f2338589700 1 -- 172.21.15.89:0/3873013134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23300c6d80 con 0x7f233006bcd0 2023-10-10T20:43:35.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.409+0000 7f23267fc700 1 -- 172.21.15.89:0/3873013134 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f232c026ba0 con 0x7f233006bcd0 2023-10-10T20:43:35.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.409+0000 7f23267fc700 1 --2- 172.21.15.89:0/3873013134 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2314067320 0x7f23140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:35.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.409+0000 7f2335b24700 1 --2- 172.21.15.89:0/3873013134 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2314067320 0x7f23140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:35.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.409+0000 7f23267fc700 1 -- 172.21.15.89:0/3873013134 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f232c021070 con 0x7f233006bcd0 2023-10-10T20:43:35.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.410+0000 7f2335b24700 1 --2- 172.21.15.89:0/3873013134 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2314067320 0x7f23140697d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f23300b43a0 tx=0x7f2320009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:35.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.413+0000 7f23267fc700 1 -- 172.21.15.89:0/3873013134 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f232c053760 con 0x7f233006bcd0 2023-10-10T20:43:35.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.638+0000 7f2338589700 1 -- 172.21.15.89:0/3873013134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f233004c3c0 con 0x7f233006bcd0 2023-10-10T20:43:35.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.638+0000 7f23267fc700 1 -- 172.21.15.89:0/3873013134 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4610 (secure 0 0 0) 0x7f232c056d80 con 0x7f233006bcd0 2023-10-10T20:43:35.641 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:35.641 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":121,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/455398787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":455398787},{"type":"v1","addr":"172.21.15.89:6836","nonce":455398787}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/2660411678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2660411678},{"type":"v1","addr":"172.21.15.111:6834","nonce":2660411678}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6890,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/1650071647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":1650071647},{"type":"v1","addr":"172.21.15.111:6835","nonce":1650071647}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:43:29.795107+0000","modified":"2023-10-10T20:43:33.428614+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8760":{"gid":8760,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/2168500629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2168500629},{"type":"v1","addr":"172.21.15.89:6837","nonce":2168500629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-10T20:43:35.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.640+0000 7f2338589700 1 -- 172.21.15.89:0/3873013134 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2314067320 msgr2=0x7f23140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:35.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.640+0000 7f2338589700 1 --2- 172.21.15.89:0/3873013134 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2314067320 0x7f23140697d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f23300b43a0 tx=0x7f2320009040 comp rx=0 tx=0).stop 2023-10-10T20:43:35.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.640+0000 7f2338589700 1 -- 172.21.15.89:0/3873013134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f233006bcd0 msgr2=0x7f23300b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:35.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.640+0000 7f2338589700 1 --2- 172.21.15.89:0/3873013134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f233006bcd0 0x7f23300b3e40 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f232c009290 tx=0x7f232c017000 comp rx=0 tx=0).stop 2023-10-10T20:43:35.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.640+0000 7f2338589700 1 -- 172.21.15.89:0/3873013134 shutdown_connections 2023-10-10T20:43:35.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.640+0000 7f2338589700 1 --2- 172.21.15.89:0/3873013134 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23300c9510 0x7f23300c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:35.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.640+0000 7f2338589700 1 --2- 172.21.15.89:0/3873013134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f233006bcd0 0x7f23300b3e40 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:35.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.640+0000 7f2338589700 1 --2- 172.21.15.89:0/3873013134 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2314067320 0x7f23140697d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:35.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.640+0000 7f2338589700 1 --2- 172.21.15.89:0/3873013134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f233006c0a0 0x7f23300c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:35.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.640+0000 7f2338589700 1 -- 172.21.15.89:0/3873013134 >> 172.21.15.89:0/3873013134 conn(0x7f23300ce600 msgr2=0x7f2330159180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:35.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.640+0000 7f2338589700 1 -- 172.21.15.89:0/3873013134 shutdown_connections 2023-10-10T20:43:35.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:35.640+0000 7f2338589700 1 -- 172.21.15.89:0/3873013134 wait complete. 2023-10-10T20:43:35.645 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 121 2023-10-10T20:43:35.700 INFO:tasks.ceph.mds.a.smithi089.stderr:2023-10-10T20:43:35.699+0000 7fcc07623700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:43:36.057 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 121, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:43:29.795107+0000', 'modified': '2023-10-10T20:43:33.428614+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8760}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8760': {'gid': 8760, 'name': 'a', 'rank': 0, 'incarnation': 117, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6837/2168500629', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6835', 'nonce': 2168500629}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 2168500629}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [32], 'metadata_pool': 31, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:43:36.058 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:43:36.058 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-10T20:43:36.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.174+0000 7feb0a388700 1 Processor -- start 2023-10-10T20:43:36.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.174+0000 7feb0a388700 1 -- start start 2023-10-10T20:43:36.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.175+0000 7feb0a388700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7feb041597f0 0x7feb040b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.175+0000 7feb0a388700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb040b11d0 con 0x7feb0415d230 2023-10-10T20:43:36.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.175+0000 7feb0a388700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb040b14b0 con 0x7feb04159420 2023-10-10T20:43:36.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.175+0000 7feb0a388700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb040b1790 con 0x7feb041597f0 2023-10-10T20:43:36.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.175+0000 7feb08b85700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7feb041597f0 0x7feb040b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.175+0000 7feb09b87700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7feb0415d230 0x7feb0415d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35246/0 (socket says 172.21.15.89:35246) 2023-10-10T20:43:36.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.175+0000 7feb09b87700 1 -- 172.21.15.89:0/3727253994 learned_addr learned my addr 172.21.15.89:0/3727253994 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:36.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.175+0000 7feb08b85700 1 -- 172.21.15.89:0/3727253994 >> v1:172.21.15.111:6789/0 conn(0x7feb04159420 legacy=0x7feb040b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:36.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.175+0000 7feafbfff700 1 -- 172.21.15.89:0/3727253994 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364877141 0 0) 0x7feb040b14b0 con 0x7feb04159420 2023-10-10T20:43:36.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.175+0000 7feb08b85700 1 -- 172.21.15.89:0/3727253994 >> v1:172.21.15.89:6789/0 conn(0x7feb0415d230 legacy=0x7feb0415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:36.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.175+0000 7feb08b85700 1 -- 172.21.15.89:0/3727253994 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb040b1a70 con 0x7feb041597f0 2023-10-10T20:43:36.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.175+0000 7feb08b85700 1 --2- 172.21.15.89:0/3727253994 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb041597f0 0x7feb040b2270 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7feaf40098e0 tx=0x7feaf40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3f7988a38bb550 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:36.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.175+0000 7feafbfff700 1 -- 172.21.15.89:0/3727253994 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4009e80 con 0x7feb041597f0 2023-10-10T20:43:36.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.176+0000 7feafbfff700 1 -- 172.21.15.89:0/3727253994 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf400b760 con 0x7feb041597f0 2023-10-10T20:43:36.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.176+0000 7feafbfff700 1 -- 172.21.15.89:0/3727253994 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf400bae0 con 0x7feb041597f0 2023-10-10T20:43:36.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.176+0000 7feb0a388700 1 -- 172.21.15.89:0/3727253994 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb041597f0 msgr2=0x7feb040b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:36.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.176+0000 7feb0a388700 1 --2- 172.21.15.89:0/3727253994 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb041597f0 0x7feb040b2270 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7feaf40098e0 tx=0x7feaf40091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.176+0000 7feb0a388700 1 -- 172.21.15.89:0/3727253994 shutdown_connections 2023-10-10T20:43:36.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.176+0000 7feb0a388700 1 --2- 172.21.15.89:0/3727253994 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb041597f0 0x7feb040b2270 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.176+0000 7feb0a388700 1 -- 172.21.15.89:0/3727253994 >> 172.21.15.89:0/3727253994 conn(0x7feb040d0660 msgr2=0x7feb040d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:36.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.176+0000 7feb0a388700 1 -- 172.21.15.89:0/3727253994 shutdown_connections 2023-10-10T20:43:36.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.176+0000 7feb0a388700 1 -- 172.21.15.89:0/3727253994 wait complete. 2023-10-10T20:43:36.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.177+0000 7feb0a388700 1 Processor -- start 2023-10-10T20:43:36.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.177+0000 7feb0a388700 1 -- start start 2023-10-10T20:43:36.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.178+0000 7feb0a388700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb04159420 0x7feb040c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.178+0000 7feb0a388700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb041597f0 0x7feb0415b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.178+0000 7feb0a388700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feb0415d230 0x7feb041543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.178+0000 7feb0a388700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0415bdf0 con 0x7feb041597f0 2023-10-10T20:43:36.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.178+0000 7feb0a388700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb04154920 con 0x7feb04159420 2023-10-10T20:43:36.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.178+0000 7feb0a388700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb040c6730 con 0x7feb0415d230 2023-10-10T20:43:36.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.178+0000 7feb08b85700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb041597f0 0x7feb0415b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.178+0000 7feb08b85700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb041597f0 0x7feb0415b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37378/0 (socket says 172.21.15.89:37378) 2023-10-10T20:43:36.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.178+0000 7feb08b85700 1 -- 172.21.15.89:0/35805336 learned_addr learned my addr 172.21.15.89:0/35805336 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:36.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.178+0000 7feb09386700 1 --2- 172.21.15.89:0/35805336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb04159420 0x7feb040c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.178+0000 7feb09b87700 1 --2- 172.21.15.89:0/35805336 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feb0415d230 0x7feb041543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.178+0000 7feb08b85700 1 -- 172.21.15.89:0/35805336 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feb0415d230 msgr2=0x7feb041543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:36.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.178+0000 7feb08b85700 1 --2- 172.21.15.89:0/35805336 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feb0415d230 0x7feb041543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.178+0000 7feb08b85700 1 -- 172.21.15.89:0/35805336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb04159420 msgr2=0x7feb040c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:36.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.178+0000 7feb08b85700 1 --2- 172.21.15.89:0/35805336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb04159420 0x7feb040c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.178+0000 7feb08b85700 1 -- 172.21.15.89:0/35805336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf4009420 con 0x7feb041597f0 2023-10-10T20:43:36.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.178+0000 7feb09b87700 1 --2- 172.21.15.89:0/35805336 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feb0415d230 0x7feb041543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:36.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.179+0000 7feb08b85700 1 --2- 172.21.15.89:0/35805336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb041597f0 0x7feb0415b8b0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7feaf40030c0 tx=0x7feaf4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:36.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.179+0000 7feaf97fa700 1 -- 172.21.15.89:0/35805336 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf401c460 con 0x7feb041597f0 2023-10-10T20:43:36.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.179+0000 7feaf97fa700 1 -- 172.21.15.89:0/35805336 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf401c5c0 con 0x7feb041597f0 2023-10-10T20:43:36.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.179+0000 7feb0a388700 1 -- 172.21.15.89:0/35805336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb040c5b20 con 0x7feb041597f0 2023-10-10T20:43:36.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.179+0000 7feaf97fa700 1 -- 172.21.15.89:0/35805336 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf401c980 con 0x7feb041597f0 2023-10-10T20:43:36.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.179+0000 7feb0a388700 1 -- 172.21.15.89:0/35805336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb040c60e0 con 0x7feb041597f0 2023-10-10T20:43:36.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.180+0000 7feaf97fa700 1 -- 172.21.15.89:0/35805336 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7feaf401a450 con 0x7feb041597f0 2023-10-10T20:43:36.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.181+0000 7feaf97fa700 1 --2- 172.21.15.89:0/35805336 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7feae8067320 0x7feae80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.181+0000 7feaf97fa700 1 -- 172.21.15.89:0/35805336 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feaf4087180 con 0x7feb041597f0 2023-10-10T20:43:36.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.181+0000 7feb09386700 1 --2- 172.21.15.89:0/35805336 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7feae8067320 0x7feae80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.181+0000 7feb09386700 1 --2- 172.21.15.89:0/35805336 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7feae8067320 0x7feae80697d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7feb00007fd0 tx=0x7feb00007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:36.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.182+0000 7feb148e3b80 1 Processor -- start 2023-10-10T20:43:36.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.183+0000 7feb148e3b80 1 -- start start 2023-10-10T20:43:36.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.183+0000 7feb148e3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda82480 0x5596bda617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.183+0000 7feb148e3b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda355f0 0x5596bda82850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.183+0000 7feb148e3b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596bda37b80 0x5596bda37ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.183+0000 7feb148e3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bd98ea70 con 0x5596bda82480 2023-10-10T20:43:36.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.183+0000 7feb148e3b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bd98c270 con 0x5596bda37b80 2023-10-10T20:43:36.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.183+0000 7feb148e3b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596bd988060 con 0x5596bda355f0 2023-10-10T20:43:36.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.183+0000 7feb09386700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda82480 0x5596bda617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.183+0000 7feb09386700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda82480 0x5596bda617d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37388/0 (socket says 172.21.15.89:37388) 2023-10-10T20:43:36.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.183+0000 7feb09386700 1 -- 172.21.15.89:0/1409598608 learned_addr learned my addr 172.21.15.89:0/1409598608 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:36.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.183+0000 7feb08b85700 1 --2- 172.21.15.89:0/1409598608 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda355f0 0x5596bda82850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.184+0000 7feb09b87700 1 --2- 172.21.15.89:0/1409598608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596bda37b80 0x5596bda37ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.184+0000 7feb09386700 1 -- 172.21.15.89:0/1409598608 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda355f0 msgr2=0x5596bda82850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:36.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.184+0000 7feb09386700 1 --2- 172.21.15.89:0/1409598608 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda355f0 0x5596bda82850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.184+0000 7feb09386700 1 -- 172.21.15.89:0/1409598608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596bda37b80 msgr2=0x5596bda37ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:36.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.184+0000 7feb09386700 1 --2- 172.21.15.89:0/1409598608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596bda37b80 0x5596bda37ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.184+0000 7feb09386700 1 -- 172.21.15.89:0/1409598608 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596bd822270 con 0x5596bda82480 2023-10-10T20:43:36.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.184+0000 7feb08b85700 1 --2- 172.21.15.89:0/1409598608 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda355f0 0x5596bda82850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:36.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.184+0000 7feb09386700 1 --2- 172.21.15.89:0/1409598608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda82480 0x5596bda617d0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7feb00002c80 tx=0x7feb00015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3ec01fa674fc69 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:36.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.184+0000 7feaee7fc700 1 -- 172.21.15.89:0/1409598608 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb00002e70 con 0x5596bda82480 2023-10-10T20:43:36.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.184+0000 7feaee7fc700 1 -- 172.21.15.89:0/1409598608 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb00014af0 con 0x5596bda82480 2023-10-10T20:43:36.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.184+0000 7feaee7fc700 1 -- 172.21.15.89:0/1409598608 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb00020410 con 0x5596bda82480 2023-10-10T20:43:36.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.185+0000 7feb148e3b80 1 -- 172.21.15.89:0/1409598608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda82480 msgr2=0x5596bda617d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:36.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.185+0000 7feb148e3b80 1 --2- 172.21.15.89:0/1409598608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda82480 0x5596bda617d0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7feb00002c80 tx=0x7feb00015000 comp rx=0 tx=0).stop 2023-10-10T20:43:36.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.185+0000 7feb148e3b80 1 -- 172.21.15.89:0/1409598608 shutdown_connections 2023-10-10T20:43:36.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.185+0000 7feb148e3b80 1 --2- 172.21.15.89:0/1409598608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596bda37b80 0x5596bda37ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.185+0000 7feb148e3b80 1 --2- 172.21.15.89:0/1409598608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda82480 0x5596bda617d0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.185+0000 7feb148e3b80 1 --2- 172.21.15.89:0/1409598608 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda355f0 0x5596bda82850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.185+0000 7feb148e3b80 1 -- 172.21.15.89:0/1409598608 >> 172.21.15.89:0/1409598608 conn(0x5596bda390c0 msgr2=0x5596bda398f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:36.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.185+0000 7feb148e3b80 1 -- 172.21.15.89:0/1409598608 shutdown_connections 2023-10-10T20:43:36.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.185+0000 7feb148e3b80 1 -- 172.21.15.89:0/1409598608 wait complete. 2023-10-10T20:43:36.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.185+0000 7feb148e3b80 1 -- start start 2023-10-10T20:43:36.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.185+0000 7feb148e3b80 1 Processor -- start 2023-10-10T20:43:36.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.186+0000 7feb148e3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda355f0 0x5596bda5b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.186+0000 7feb148e3b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596bda37b80 0x5596bda62d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.186+0000 7feb148e3b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda82480 0x5596bda63250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.186+0000 7feb148e3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bd933890 con 0x5596bda355f0 2023-10-10T20:43:36.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.186+0000 7feb148e3b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bd94eea0 con 0x5596bda37b80 2023-10-10T20:43:36.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.186+0000 7feb148e3b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596bd80ae30 con 0x5596bda82480 2023-10-10T20:43:36.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.186+0000 7feb148e3b80 10 client.0 _ll_register_callbacks cb 0x5596bd9fb2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:43:36.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.186+0000 7feb09386700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda355f0 0x5596bda5b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.186+0000 7feb09386700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda355f0 0x5596bda5b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37398/0 (socket says 172.21.15.89:37398) 2023-10-10T20:43:36.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.186+0000 7feb09386700 1 -- 172.21.15.89:0/111350889 learned_addr learned my addr 172.21.15.89:0/111350889 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:36.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.186+0000 7feb08b85700 1 --2- 172.21.15.89:0/111350889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596bda37b80 0x5596bda62d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.186+0000 7feb09b87700 1 --2- 172.21.15.89:0/111350889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda82480 0x5596bda63250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feb09386700 1 -- 172.21.15.89:0/111350889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda82480 msgr2=0x5596bda63250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:36.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feb09386700 1 --2- 172.21.15.89:0/111350889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda82480 0x5596bda63250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feb09386700 1 -- 172.21.15.89:0/111350889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596bda37b80 msgr2=0x5596bda62d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:36.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feb09386700 1 --2- 172.21.15.89:0/111350889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596bda37b80 0x5596bda62d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feb09386700 1 -- 172.21.15.89:0/111350889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb00002860 con 0x5596bda355f0 2023-10-10T20:43:36.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feb09386700 1 --2- 172.21.15.89:0/111350889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda355f0 0x5596bda5b830 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7feb00023040 tx=0x7feb00019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbde603e59c4f28 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:36.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feae37fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:43:36.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feae37fe700 1 -- 172.21.15.89:0/111350889 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb00034070 con 0x5596bda355f0 2023-10-10T20:43:36.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feae37fe700 1 -- 172.21.15.89:0/111350889 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feb0001c040 con 0x5596bda355f0 2023-10-10T20:43:36.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feb148e3b80 1 -- 172.21.15.89:0/111350889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596bda2ccd0 con 0x5596bda355f0 2023-10-10T20:43:36.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feae37fe700 1 -- 172.21.15.89:0/111350889 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb0002de40 con 0x5596bda355f0 2023-10-10T20:43:36.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feb148e3b80 10 client.8925 fetch_fsmap learned FSMap version 121 2023-10-10T20:43:36.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feb148e3b80 1 -- 172.21.15.89:0/111350889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) v3 -- 0x5596bd7c4fe0 con 0x5596bda355f0 2023-10-10T20:43:36.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feae37fe700 1 -- 172.21.15.89:0/111350889 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5596bda2ccd0 con 0x5596bda355f0 2023-10-10T20:43:36.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feae37fe700 1 -- 172.21.15.89:0/111350889 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 121) v1 ==== 3100+0+0 (secure 0 0 0) 0x7feb00020780 con 0x5596bda355f0 2023-10-10T20:43:36.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feb148e3b80 10 client.8925 fetch_fsmap finished waiting for FSMap version 121 2023-10-10T20:43:36.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feb148e3b80 10 client.8925 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-10T20:43:36.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feb148e3b80 20 client.8925 populate_metadata read hostname 'smithi089' 2023-10-10T20:43:36.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feb148e3b80 1 --2- 172.21.15.89:0/111350889 >> [v2:172.21.15.89:6835/2168500629,v1:172.21.15.89:6837/2168500629] conn(0x5596bda813c0 0x5596bda81790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feb148e3b80 4 client.8925 mds_command: new command op to 8760 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:43:36.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feb148e3b80 1 -- 172.21.15.89:0/111350889 --> [v2:172.21.15.89:6835/2168500629,v1:172.21.15.89:6837/2168500629] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596bda09aa0 con 0x5596bda813c0 2023-10-10T20:43:36.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.187+0000 7feb08b85700 1 --2- 172.21.15.89:0/111350889 >> [v2:172.21.15.89:6835/2168500629,v1:172.21.15.89:6837/2168500629] conn(0x5596bda813c0 0x5596bda81790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.188+0000 7feae37fe700 10 client.8925 ms_handle_connect on v2:172.21.15.89:6835/2168500629 2023-10-10T20:43:36.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.188+0000 7feb08b85700 1 --2- 172.21.15.89:0/111350889 >> [v2:172.21.15.89:6835/2168500629,v1:172.21.15.89:6837/2168500629] conn(0x5596bda813c0 0x5596bda81790 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f865d9d6ebedce14 server_cookie=b049a566a4f00525 in_seq=0 out_seq=0 2023-10-10T20:43:36.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.188+0000 7feae37fe700 1 -- 172.21.15.89:0/111350889 <== mds.0 v2:172.21.15.89:6835/2168500629 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5596bda09aa0 con 0x5596bda813c0 2023-10-10T20:43:36.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.188+0000 7feae37fe700 10 client.8925 handle_command_reply: tid=0 2023-10-10T20:43:36.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.188+0000 7feb148e3b80 1 client.8925 shutdown 2023-10-10T20:43:36.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.188+0000 7feb148e3b80 1 -- 172.21.15.89:0/111350889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda355f0 msgr2=0x5596bda5b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:36.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.188+0000 7feb148e3b80 1 --2- 172.21.15.89:0/111350889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda355f0 0x5596bda5b830 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7feb00023040 tx=0x7feb00019000 comp rx=0 tx=0).stop 2023-10-10T20:43:36.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.189+0000 7feb148e3b80 1 -- 172.21.15.89:0/111350889 shutdown_connections 2023-10-10T20:43:36.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.189+0000 7feb148e3b80 1 --2- 172.21.15.89:0/111350889 >> [v2:172.21.15.89:6835/2168500629,v1:172.21.15.89:6837/2168500629] conn(0x5596bda813c0 0x5596bda81790 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.189+0000 7feb148e3b80 1 --2- 172.21.15.89:0/111350889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda82480 0x5596bda63250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.189+0000 7feae37fe700 0 client.8925 ms_handle_reset on v2:172.21.15.89:6835/2168500629 2023-10-10T20:43:36.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.189+0000 7feb148e3b80 1 --2- 172.21.15.89:0/111350889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda355f0 0x5596bda5b830 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.189+0000 7feb148e3b80 1 --2- 172.21.15.89:0/111350889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596bda37b80 0x5596bda62d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.189+0000 7feb148e3b80 1 -- 172.21.15.89:0/111350889 >> 172.21.15.89:0/111350889 conn(0x5596bda390c0 msgr2=0x5596bda398d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:36.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.189+0000 7feb148e3b80 1 -- 172.21.15.89:0/111350889 shutdown_connections 2023-10-10T20:43:36.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.189+0000 7feb148e3b80 1 -- 172.21.15.89:0/111350889 wait complete. 2023-10-10T20:43:36.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.189+0000 7feb148e3b80 20 client.8925 trim_cache size 0 max 16384 2023-10-10T20:43:36.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.205+0000 7feb148e3b80 1 Processor -- start 2023-10-10T20:43:36.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.205+0000 7feb148e3b80 1 -- start start 2023-10-10T20:43:36.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.206+0000 7feb148e3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda355f0 0x5596bda5ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.206+0000 7feb148e3b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda37b80 0x5596bda349b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.206+0000 7feb148e3b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596bda82480 0x5596bda34ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.206+0000 7feb148e3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bda80650 con 0x5596bda355f0 2023-10-10T20:43:36.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.206+0000 7feb148e3b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bda81f80 con 0x5596bda82480 2023-10-10T20:43:36.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.206+0000 7feb148e3b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596bda64380 con 0x5596bda37b80 2023-10-10T20:43:36.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.206+0000 7feb09386700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda355f0 0x5596bda5ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.206+0000 7feb09386700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda355f0 0x5596bda5ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37410/0 (socket says 172.21.15.89:37410) 2023-10-10T20:43:36.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.206+0000 7feb09386700 1 -- 172.21.15.89:0/967193633 learned_addr learned my addr 172.21.15.89:0/967193633 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:36.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.206+0000 7feb09386700 1 -- 172.21.15.89:0/967193633 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda37b80 msgr2=0x5596bda349b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:36.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.206+0000 7feb09386700 1 --2- 172.21.15.89:0/967193633 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda37b80 0x5596bda349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.206+0000 7feb09386700 1 -- 172.21.15.89:0/967193633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596bda82480 msgr2=0x5596bda34ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:36.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.206+0000 7feb09386700 1 --2- 172.21.15.89:0/967193633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596bda82480 0x5596bda34ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.206+0000 7feb09386700 1 -- 172.21.15.89:0/967193633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596bd956780 con 0x5596bda355f0 2023-10-10T20:43:36.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.207+0000 7feb09386700 1 --2- 172.21.15.89:0/967193633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda355f0 0x5596bda5ae00 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7feb00007830 tx=0x7feb00034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148713ff8e47dd10 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:36.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.207+0000 7feaeeffd700 1 -- 172.21.15.89:0/967193633 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb00020ce0 con 0x5596bda355f0 2023-10-10T20:43:36.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.207+0000 7feaeeffd700 1 -- 172.21.15.89:0/967193633 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb00020780 con 0x5596bda355f0 2023-10-10T20:43:36.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.207+0000 7feaeeffd700 1 -- 172.21.15.89:0/967193633 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0002f0d0 con 0x5596bda355f0 2023-10-10T20:43:36.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.207+0000 7feb148e3b80 1 -- 172.21.15.89:0/967193633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda355f0 msgr2=0x5596bda5ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:36.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.207+0000 7feb148e3b80 1 --2- 172.21.15.89:0/967193633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda355f0 0x5596bda5ae00 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7feb00007830 tx=0x7feb00034000 comp rx=0 tx=0).stop 2023-10-10T20:43:36.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.207+0000 7feb148e3b80 1 -- 172.21.15.89:0/967193633 shutdown_connections 2023-10-10T20:43:36.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.207+0000 7feb148e3b80 1 --2- 172.21.15.89:0/967193633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596bda82480 0x5596bda34ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.207+0000 7feb148e3b80 1 --2- 172.21.15.89:0/967193633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda355f0 0x5596bda5ae00 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.207+0000 7feb148e3b80 1 --2- 172.21.15.89:0/967193633 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda37b80 0x5596bda349b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.207+0000 7feb148e3b80 1 -- 172.21.15.89:0/967193633 >> 172.21.15.89:0/967193633 conn(0x5596bda390c0 msgr2=0x5596bda7e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:36.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.207+0000 7feb148e3b80 1 -- 172.21.15.89:0/967193633 shutdown_connections 2023-10-10T20:43:36.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.207+0000 7feb148e3b80 1 -- 172.21.15.89:0/967193633 wait complete. 2023-10-10T20:43:36.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.208+0000 7feb148e3b80 1 -- start start 2023-10-10T20:43:36.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.208+0000 7feb148e3b80 1 Processor -- start 2023-10-10T20:43:36.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feb148e3b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596bda355f0 0x5596bdadd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feb148e3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda37b80 0x5596bdae4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feb148e3b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda82480 0x5596bda349b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feb148e3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bda81790 con 0x5596bda37b80 2023-10-10T20:43:36.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feb148e3b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bdae5050 con 0x5596bda355f0 2023-10-10T20:43:36.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feb148e3b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596bda5ccb0 con 0x5596bda82480 2023-10-10T20:43:36.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feb148e3b80 10 client.0 _ll_register_callbacks cb 0x5596bd9fb2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:43:36.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feb08b85700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda37b80 0x5596bdae4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feb08b85700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda37b80 0x5596bdae4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37412/0 (socket says 172.21.15.89:37412) 2023-10-10T20:43:36.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feb08b85700 1 -- 172.21.15.89:0/2955562812 learned_addr learned my addr 172.21.15.89:0/2955562812 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:36.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feb09386700 1 --2- 172.21.15.89:0/2955562812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596bda355f0 0x5596bdadd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feb09b87700 1 --2- 172.21.15.89:0/2955562812 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda82480 0x5596bda349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feb08b85700 1 -- 172.21.15.89:0/2955562812 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda82480 msgr2=0x5596bda349b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:36.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feb08b85700 1 --2- 172.21.15.89:0/2955562812 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda82480 0x5596bda349b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feb08b85700 1 -- 172.21.15.89:0/2955562812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596bda355f0 msgr2=0x5596bdadd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:36.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feb08b85700 1 --2- 172.21.15.89:0/2955562812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596bda355f0 0x5596bdadd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feb08b85700 1 -- 172.21.15.89:0/2955562812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596bd7c4fe0 con 0x5596bda37b80 2023-10-10T20:43:36.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feb08b85700 1 --2- 172.21.15.89:0/2955562812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda37b80 0x5596bdae4450 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7feaf4006f00 tx=0x7feaf405a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279651435d097245 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:36.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feae37fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:43:36.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feae37fe700 1 -- 172.21.15.89:0/2955562812 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4057ad0 con 0x5596bda37b80 2023-10-10T20:43:36.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feae37fe700 1 -- 172.21.15.89:0/2955562812 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feaf4070030 con 0x5596bda37b80 2023-10-10T20:43:36.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.209+0000 7feb148e3b80 1 -- 172.21.15.89:0/2955562812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb00014af0 con 0x5596bda37b80 2023-10-10T20:43:36.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.210+0000 7feae37fe700 1 -- 172.21.15.89:0/2955562812 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaf405ad20 con 0x5596bda37b80 2023-10-10T20:43:36.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.210+0000 7feb148e3b80 10 client.8931 fetch_fsmap learned FSMap version 121 2023-10-10T20:43:36.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.210+0000 7feb148e3b80 1 -- 172.21.15.89:0/2955562812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) v3 -- 0x5596bd93d470 con 0x5596bda37b80 2023-10-10T20:43:36.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.210+0000 7feae37fe700 1 -- 172.21.15.89:0/2955562812 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf4063020 con 0x5596bda37b80 2023-10-10T20:43:36.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.210+0000 7feae37fe700 1 -- 172.21.15.89:0/2955562812 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 121) v1 ==== 3100+0+0 (secure 0 0 0) 0x7feaf40541e0 con 0x5596bda37b80 2023-10-10T20:43:36.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.210+0000 7feb148e3b80 10 client.8931 fetch_fsmap finished waiting for FSMap version 121 2023-10-10T20:43:36.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.210+0000 7feb148e3b80 10 client.8931 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-10T20:43:36.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.210+0000 7feb148e3b80 20 client.8931 populate_metadata read hostname 'smithi089' 2023-10-10T20:43:36.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.210+0000 7feb148e3b80 1 --2- 172.21.15.89:0/2955562812 >> [v2:172.21.15.89:6835/2168500629,v1:172.21.15.89:6837/2168500629] conn(0x5596bda2e530 0x5596bda2e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.210+0000 7feb148e3b80 4 client.8931 mds_command: new command op to 8760 tid=0[{"prefix": "status"}] 2023-10-10T20:43:36.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.210+0000 7feb148e3b80 1 -- 172.21.15.89:0/2955562812 --> [v2:172.21.15.89:6835/2168500629,v1:172.21.15.89:6837/2168500629] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5596bd92ae00 con 0x5596bda2e530 2023-10-10T20:43:36.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.210+0000 7feb09386700 1 --2- 172.21.15.89:0/2955562812 >> [v2:172.21.15.89:6835/2168500629,v1:172.21.15.89:6837/2168500629] conn(0x5596bda2e530 0x5596bda2e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.210+0000 7feae37fe700 10 client.8931 ms_handle_connect on v2:172.21.15.89:6835/2168500629 2023-10-10T20:43:36.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.210+0000 7feb09386700 1 --2- 172.21.15.89:0/2955562812 >> [v2:172.21.15.89:6835/2168500629,v1:172.21.15.89:6837/2168500629] conn(0x5596bda2e530 0x5596bda2e900 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d0e45c693d71b1c server_cookie=dde678a89460cff5 in_seq=0 out_seq=0 2023-10-10T20:43:36.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.210+0000 7feae37fe700 1 -- 172.21.15.89:0/2955562812 <== mds.0 v2:172.21.15.89:6835/2168500629 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x5596bd92ae00 con 0x5596bda2e530 2023-10-10T20:43:36.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.210+0000 7feae37fe700 10 client.8931 handle_command_reply: tid=0 2023-10-10T20:43:36.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.210+0000 7feb148e3b80 1 client.8931 shutdown 2023-10-10T20:43:36.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb148e3b80 1 -- 172.21.15.89:0/2955562812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda37b80 msgr2=0x5596bdae4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:36.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb148e3b80 1 --2- 172.21.15.89:0/2955562812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda37b80 0x5596bdae4450 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7feaf4006f00 tx=0x7feaf405a040 comp rx=0 tx=0).stop 2023-10-10T20:43:36.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb148e3b80 1 -- 172.21.15.89:0/2955562812 shutdown_connections 2023-10-10T20:43:36.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb148e3b80 1 --2- 172.21.15.89:0/2955562812 >> [v2:172.21.15.89:6835/2168500629,v1:172.21.15.89:6837/2168500629] conn(0x5596bda2e530 0x5596bda2e900 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb148e3b80 1 --2- 172.21.15.89:0/2955562812 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x5596bda82480 0x5596bda349b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feae37fe700 0 client.8931 ms_handle_reset on v2:172.21.15.89:6835/2168500629 2023-10-10T20:43:36.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb148e3b80 1 --2- 172.21.15.89:0/2955562812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596bda355f0 0x5596bdadd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb148e3b80 1 --2- 172.21.15.89:0/2955562812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596bda37b80 0x5596bdae4450 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb148e3b80 1 -- 172.21.15.89:0/2955562812 >> 172.21.15.89:0/2955562812 conn(0x5596bda390c0 msgr2=0x5596bda7e820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:36.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb148e3b80 1 -- 172.21.15.89:0/2955562812 shutdown_connections 2023-10-10T20:43:36.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb148e3b80 1 -- 172.21.15.89:0/2955562812 wait complete. 2023-10-10T20:43:36.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb148e3b80 20 client.8931 trim_cache size 0 max 16384 2023-10-10T20:43:36.236 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:43:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "5dd675b8-9026-4d8a-a1bf-62fdc5e7de67", 2023-10-10T20:43:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-10-10T20:43:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 8760, 2023-10-10T20:43:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-10-10T20:43:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-10-10T20:43:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-10-10T20:43:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 5.8074663180000003, 2023-10-10T20:43:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 121, 2023-10-10T20:43:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 183, 2023-10-10T20:43:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 183, 2023-10-10T20:43:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 10.930064037999999 2023-10-10T20:43:36.238 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:43:36.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb0a388700 1 -- 172.21.15.89:0/35805336 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7feae8067320 msgr2=0x7feae80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:36.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb0a388700 1 --2- 172.21.15.89:0/35805336 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7feae8067320 0x7feae80697d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7feb00007fd0 tx=0x7feb00007d30 comp rx=0 tx=0).stop 2023-10-10T20:43:36.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb0a388700 1 -- 172.21.15.89:0/35805336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb041597f0 msgr2=0x7feb0415b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:36.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb0a388700 1 --2- 172.21.15.89:0/35805336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb041597f0 0x7feb0415b8b0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7feaf40030c0 tx=0x7feaf4016000 comp rx=0 tx=0).stop 2023-10-10T20:43:36.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb0a388700 1 -- 172.21.15.89:0/35805336 shutdown_connections 2023-10-10T20:43:36.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb0a388700 1 --2- 172.21.15.89:0/35805336 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7feb0415d230 0x7feb041543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb0a388700 1 --2- 172.21.15.89:0/35805336 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7feae8067320 0x7feae80697d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb0a388700 1 --2- 172.21.15.89:0/35805336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb04159420 0x7feb040c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb0a388700 1 --2- 172.21.15.89:0/35805336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb041597f0 0x7feb0415b8b0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb0a388700 1 -- 172.21.15.89:0/35805336 >> 172.21.15.89:0/35805336 conn(0x7feb040d0660 msgr2=0x7feb04069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:36.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb0a388700 1 -- 172.21.15.89:0/35805336 shutdown_connections 2023-10-10T20:43:36.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.211+0000 7feb0a388700 1 -- 172.21.15.89:0/35805336 wait complete. 2023-10-10T20:43:36.243 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:43:36.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.358+0000 7f522d3a0700 1 Processor -- start 2023-10-10T20:43:36.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.358+0000 7f522d3a0700 1 -- start start 2023-10-10T20:43:36.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.358+0000 7f522d3a0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5228143050 0x7f5228159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.358+0000 7f522d3a0700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52280b3f20 0x7f522806be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.358+0000 7f522d3a0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f522806c390 con 0x7f522806b9e0 2023-10-10T20:43:36.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.358+0000 7f522d3a0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5228143420 con 0x7f5228143050 2023-10-10T20:43:36.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.358+0000 7f522d3a0700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f522806a1e0 con 0x7f52280b3f20 2023-10-10T20:43:36.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.359+0000 7f5227fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5228143050 0x7f5228159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.359+0000 7f5227fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5228143050 0x7f5228159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37418/0 (socket says 172.21.15.89:37418) 2023-10-10T20:43:36.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.359+0000 7f52277fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f522806b9e0 0x7f52280b3810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35262/0 (socket says 172.21.15.89:35262) 2023-10-10T20:43:36.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.359+0000 7f5227fff700 1 -- 172.21.15.89:0/3329925125 learned_addr learned my addr 172.21.15.89:0/3329925125 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:36.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.359+0000 7f522cb9f700 1 --2- 172.21.15.89:0/3329925125 >> v2:172.21.15.111:3300/0 conn(0x7f52280b3f20 0x7f522806be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.359+0000 7f5227fff700 1 -- 172.21.15.89:0/3329925125 >> v2:172.21.15.111:3300/0 conn(0x7f52280b3f20 msgr2=0x7f522806be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:36.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.359+0000 7f5227fff700 1 --2- 172.21.15.89:0/3329925125 >> v2:172.21.15.111:3300/0 conn(0x7f52280b3f20 0x7f522806be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.359+0000 7f5227fff700 1 -- 172.21.15.89:0/3329925125 >> v1:172.21.15.89:6789/0 conn(0x7f522806b9e0 legacy=0x7f52280b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:36.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.359+0000 7f5227fff700 1 -- 172.21.15.89:0/3329925125 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52280695d0 con 0x7f5228143050 2023-10-10T20:43:36.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.359+0000 7f522cb9f700 1 --2- 172.21.15.89:0/3329925125 >> v2:172.21.15.111:3300/0 conn(0x7f52280b3f20 0x7f522806be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:36.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.359+0000 7f5227fff700 1 --2- 172.21.15.89:0/3329925125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5228143050 0x7f5228159c90 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f52280cda70 tx=0x7f5218009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fe57edb086a57b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:36.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.359+0000 7f5226ffd700 1 -- 172.21.15.89:0/3329925125 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5218005c10 con 0x7f5228143050 2023-10-10T20:43:36.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.359+0000 7f5226ffd700 1 -- 172.21.15.89:0/3329925125 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5218005d70 con 0x7f5228143050 2023-10-10T20:43:36.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.359+0000 7f5226ffd700 1 -- 172.21.15.89:0/3329925125 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521800b8a0 con 0x7f5228143050 2023-10-10T20:43:36.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.360+0000 7f522d3a0700 1 -- 172.21.15.89:0/3329925125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5228143050 msgr2=0x7f5228159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.360+0000 7f522d3a0700 1 --2- 172.21.15.89:0/3329925125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5228143050 0x7f5228159c90 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f52280cda70 tx=0x7f5218009220 comp rx=0 tx=0).stop 2023-10-10T20:43:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.360+0000 7f522d3a0700 1 -- 172.21.15.89:0/3329925125 shutdown_connections 2023-10-10T20:43:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.360+0000 7f522d3a0700 1 --2- 172.21.15.89:0/3329925125 >> v2:172.21.15.111:3300/0 conn(0x7f52280b3f20 0x7f522806be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.360+0000 7f522d3a0700 1 --2- 172.21.15.89:0/3329925125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5228143050 0x7f5228159c90 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.360+0000 7f522d3a0700 1 -- 172.21.15.89:0/3329925125 >> 172.21.15.89:0/3329925125 conn(0x7f52280ce430 msgr2=0x7f52280ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.360+0000 7f522d3a0700 1 -- 172.21.15.89:0/3329925125 shutdown_connections 2023-10-10T20:43:36.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.360+0000 7f522d3a0700 1 -- 172.21.15.89:0/3329925125 wait complete. 2023-10-10T20:43:36.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.361+0000 7f522d3a0700 1 Processor -- start 2023-10-10T20:43:36.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.361+0000 7f522d3a0700 1 -- start start 2023-10-10T20:43:36.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f522d3a0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f522806b9e0 0x7f522813a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f522d3a0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52280b3f20 0x7f522815d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f522d3a0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5228143050 0x7f52280af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f522d3a0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522815ee80 con 0x7f5228143050 2023-10-10T20:43:36.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f522d3a0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522815f310 con 0x7f52280b3f20 2023-10-10T20:43:36.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f522d3a0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522815e270 con 0x7f522806b9e0 2023-10-10T20:43:36.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f522cb9f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5228143050 0x7f52280af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f522cb9f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5228143050 0x7f52280af5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37434/0 (socket says 172.21.15.89:37434) 2023-10-10T20:43:36.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f522cb9f700 1 -- 172.21.15.89:0/1346217688 learned_addr learned my addr 172.21.15.89:0/1346217688 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:36.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f522cb9f700 1 -- 172.21.15.89:0/1346217688 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f522806b9e0 msgr2=0x7f522813a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:36.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f5227fff700 1 --2- 172.21.15.89:0/1346217688 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f522806b9e0 0x7f522813a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f52277fe700 1 --2- 172.21.15.89:0/1346217688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52280b3f20 0x7f522815d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f522cb9f700 1 --2- 172.21.15.89:0/1346217688 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f522806b9e0 0x7f522813a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f522cb9f700 1 -- 172.21.15.89:0/1346217688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52280b3f20 msgr2=0x7f522815d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:36.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f522cb9f700 1 --2- 172.21.15.89:0/1346217688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52280b3f20 0x7f522815d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f522cb9f700 1 -- 172.21.15.89:0/1346217688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5218009480 con 0x7f5228143050 2023-10-10T20:43:36.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f522cb9f700 1 --2- 172.21.15.89:0/1346217688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5228143050 0x7f52280af5c0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f522000a380 tx=0x7f5220007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:36.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f520ffff700 1 -- 172.21.15.89:0/1346217688 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522000ae10 con 0x7f5228143050 2023-10-10T20:43:36.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f520ffff700 1 -- 172.21.15.89:0/1346217688 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522000d650 con 0x7f5228143050 2023-10-10T20:43:36.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f522d3a0700 1 -- 172.21.15.89:0/1346217688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522815e550 con 0x7f5228143050 2023-10-10T20:43:36.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f520ffff700 1 -- 172.21.15.89:0/1346217688 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522000da10 con 0x7f5228143050 2023-10-10T20:43:36.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.362+0000 7f522d3a0700 1 -- 172.21.15.89:0/1346217688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522815eb10 con 0x7f5228143050 2023-10-10T20:43:36.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.363+0000 7f522d3a0700 1 -- 172.21.15.89:0/1346217688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522815f030 con 0x7f5228143050 2023-10-10T20:43:36.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.364+0000 7f520ffff700 1 -- 172.21.15.89:0/1346217688 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f522001e3b0 con 0x7f5228143050 2023-10-10T20:43:36.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.364+0000 7f520ffff700 1 --2- 172.21.15.89:0/1346217688 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5214067320 0x7f52140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:36.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.364+0000 7f520ffff700 1 -- 172.21.15.89:0/1346217688 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5220085680 con 0x7f5228143050 2023-10-10T20:43:36.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.364+0000 7f5227fff700 1 --2- 172.21.15.89:0/1346217688 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5214067320 0x7f52140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:36.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.364+0000 7f5227fff700 1 --2- 172.21.15.89:0/1346217688 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5214067320 0x7f52140697d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5218007fd0 tx=0x7f5218016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:36.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.368+0000 7f520ffff700 1 -- 172.21.15.89:0/1346217688 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f522815f030 con 0x7f5228143050 2023-10-10T20:43:36.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.598+0000 7f522d3a0700 1 -- 172.21.15.89:0/1346217688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f522804c3c0 con 0x7f5228143050 2023-10-10T20:43:36.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.598+0000 7f520ffff700 1 -- 172.21.15.89:0/1346217688 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v121) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5220055da0 con 0x7f5228143050 2023-10-10T20:43:36.600 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:43:36.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.600+0000 7f522d3a0700 1 -- 172.21.15.89:0/1346217688 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5214067320 msgr2=0x7f52140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:36.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.600+0000 7f522d3a0700 1 --2- 172.21.15.89:0/1346217688 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5214067320 0x7f52140697d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5218007fd0 tx=0x7f5218016000 comp rx=0 tx=0).stop 2023-10-10T20:43:36.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.600+0000 7f522d3a0700 1 -- 172.21.15.89:0/1346217688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5228143050 msgr2=0x7f52280af5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:36.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.600+0000 7f522d3a0700 1 --2- 172.21.15.89:0/1346217688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5228143050 0x7f52280af5c0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f522000a380 tx=0x7f5220007900 comp rx=0 tx=0).stop 2023-10-10T20:43:36.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.600+0000 7f522d3a0700 1 -- 172.21.15.89:0/1346217688 shutdown_connections 2023-10-10T20:43:36.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.600+0000 7f522d3a0700 1 --2- 172.21.15.89:0/1346217688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5228143050 0x7f52280af5c0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.600+0000 7f522d3a0700 1 --2- 172.21.15.89:0/1346217688 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f522806b9e0 0x7f522813a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.600+0000 7f522d3a0700 1 --2- 172.21.15.89:0/1346217688 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5214067320 0x7f52140697d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.600+0000 7f522d3a0700 1 --2- 172.21.15.89:0/1346217688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52280b3f20 0x7f522815d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.600+0000 7f522d3a0700 1 -- 172.21.15.89:0/1346217688 >> 172.21.15.89:0/1346217688 conn(0x7f52280ce430 msgr2=0x7f52280adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.600+0000 7f522d3a0700 1 -- 172.21.15.89:0/1346217688 shutdown_connections 2023-10-10T20:43:36.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:36.600+0000 7f522d3a0700 1 -- 172.21.15.89:0/1346217688 wait complete. 2023-10-10T20:43:36.614 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:43:36.615 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:43:36.615 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-10T20:43:36.615 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:43:36.615 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:43:36.616 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:43:36.616 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:43:36.616 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:43:36.617 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:43:36.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:36.618 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-10T20:43:36.633 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:43:36.634 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:43:36.634 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:43:36.634 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:36.634 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-10T20:43:36.634 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:36.635 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:43:36.635 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:36.635 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-10T20:43:36.635 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:43:36.635 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:36.636 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-10T20:43:36.636 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:43:36.636 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:36.636 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:43:36.636 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:36.636 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:43:36.637 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 41949sec preferred_lft 41949sec 2023-10-10T20:43:36.637 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2023-10-10T20:43:36.637 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:36.637 INFO:teuthology.orchestra.run.smithi089.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:43:36.637 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:36.638 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:43:36.638 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:36.638 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::98a3:d4ff:fec1:83bb/64 scope link 2023-10-10T20:43:36.638 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:36.638 INFO:teuthology.orchestra.run.smithi089.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:43:36.639 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether ee:16:22:10:ca:9a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:43:36.639 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec16:22ff:fe10:ca9a/64 scope link 2023-10-10T20:43:36.639 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:36.639 INFO:teuthology.orchestra.run.smithi089.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:43:36.639 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:43:36.640 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::6864:2bff:fe5c:71d4/64 scope link 2023-10-10T20:43:36.640 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:36.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:36.641 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:43:36.696 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-10T20:43:36.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:36.698 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-10T20:43:36.754 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-10T20:43:36.754 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:43:36.755 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:43:36.755 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:43:36.755 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:43:36.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:36.756 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:43:36.756 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:43:36.757 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:43:36.757 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:43:36.757 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:43:36.758 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:43:36.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:36.854 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-10T20:43:36.869 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:43:36.870 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:43:36.870 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:43:36.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:36.871 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-10-10T20:43:36.934 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:43:36.934 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:43:36.989 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:43:36.989 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:43:36.990 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:43:37.044 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:43:37.044 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:43:37.045 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:43:37.122 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:43:37.123 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:43:37.123 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:43:37.130 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:43:37.131 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:43:37.131 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:43:37.131 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:43:37.131 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:43:37.152 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:43:37.152 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:43:37.152 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:43:37.153 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:43:37.153 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:43:37.153 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:43:37.153 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:43:37.153 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:43:37.154 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:43:37.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:37.155 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:43:37.219 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:43:37.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.373+0000 7f9d1f55d700 1 Processor -- start 2023-10-10T20:43:37.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.373+0000 7f9d1f55d700 1 -- start start 2023-10-10T20:43:37.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.374+0000 7f9d1f55d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9d1815d230 0x7f9d1815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:37.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.374+0000 7f9d1f55d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d180b2980 con 0x7f9d18159420 2023-10-10T20:43:37.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.374+0000 7f9d1f55d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d1815db70 con 0x7f9d181597f0 2023-10-10T20:43:37.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.374+0000 7f9d1f55d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d180b11d0 con 0x7f9d1815d230 2023-10-10T20:43:37.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.374+0000 7f9d1ed5c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9d1815d230 0x7f9d1815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:37.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.374+0000 7f9d1ed5c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9d1815d230 0x7f9d1815d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37438/0 (socket says 172.21.15.89:37438) 2023-10-10T20:43:37.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.374+0000 7f9d1ed5c700 1 -- 172.21.15.89:0/3227510788 learned_addr learned my addr 172.21.15.89:0/3227510788 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:37.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.374+0000 7f9d1e55b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9d18159420 0x7f9d180b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35266/0 (socket says 172.21.15.89:35266) 2023-10-10T20:43:37.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.374+0000 7f9d1ed5c700 1 -- 172.21.15.89:0/3227510788 >> v1:172.21.15.111:6789/0 conn(0x7f9d181597f0 legacy=0x7f9d180b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:37.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.374+0000 7f9d1ed5c700 1 -- 172.21.15.89:0/3227510788 >> v1:172.21.15.89:6789/0 conn(0x7f9d18159420 legacy=0x7f9d180b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:37.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.374+0000 7f9d1ed5c700 1 -- 172.21.15.89:0/3227510788 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d180b14b0 con 0x7f9d1815d230 2023-10-10T20:43:37.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.374+0000 7f9d1d559700 1 -- 172.21.15.89:0/3227510788 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 850122851 0 0) 0x7f9d180b2980 con 0x7f9d18159420 2023-10-10T20:43:37.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.374+0000 7f9d1ed5c700 1 --2- 172.21.15.89:0/3227510788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d1815d230 0x7f9d1815d630 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009f20 tx=0x7f9d10009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1f3eca322d3a5e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:37.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.375+0000 7f9d1d559700 1 -- 172.21.15.89:0/3227510788 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1000e070 con 0x7f9d1815d230 2023-10-10T20:43:37.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.375+0000 7f9d1d559700 1 -- 172.21.15.89:0/3227510788 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1000ba40 con 0x7f9d1815d230 2023-10-10T20:43:37.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.375+0000 7f9d1d559700 1 -- 172.21.15.89:0/3227510788 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d100092e0 con 0x7f9d1815d230 2023-10-10T20:43:37.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.375+0000 7f9d1f55d700 1 -- 172.21.15.89:0/3227510788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d1815d230 msgr2=0x7f9d1815d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:37.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.375+0000 7f9d1f55d700 1 --2- 172.21.15.89:0/3227510788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d1815d230 0x7f9d1815d630 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009f20 tx=0x7f9d10009840 comp rx=0 tx=0).stop 2023-10-10T20:43:37.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.375+0000 7f9d1f55d700 1 -- 172.21.15.89:0/3227510788 shutdown_connections 2023-10-10T20:43:37.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.375+0000 7f9d1f55d700 1 --2- 172.21.15.89:0/3227510788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d1815d230 0x7f9d1815d630 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:37.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.375+0000 7f9d1f55d700 1 -- 172.21.15.89:0/3227510788 >> 172.21.15.89:0/3227510788 conn(0x7f9d180d0660 msgr2=0x7f9d180d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:37.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.375+0000 7f9d1f55d700 1 -- 172.21.15.89:0/3227510788 shutdown_connections 2023-10-10T20:43:37.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.375+0000 7f9d1f55d700 1 -- 172.21.15.89:0/3227510788 wait complete. 2023-10-10T20:43:37.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.376+0000 7f9d1f55d700 1 Processor -- start 2023-10-10T20:43:37.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.376+0000 7f9d1f55d700 1 -- start start 2023-10-10T20:43:37.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.377+0000 7f9d1f55d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d18159420 0x7f9d1815f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:37.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.377+0000 7f9d1f55d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9d181597f0 0x7f9d18154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:37.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.377+0000 7f9d1f55d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d1815d230 0x7f9d18155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:37.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.377+0000 7f9d1f55d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1815c7a0 con 0x7f9d18159420 2023-10-10T20:43:37.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.377+0000 7f9d1f55d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1815bb90 con 0x7f9d1815d230 2023-10-10T20:43:37.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.377+0000 7f9d1f55d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1815be70 con 0x7f9d181597f0 2023-10-10T20:43:37.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.377+0000 7f9d1e55b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d18159420 0x7f9d1815f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:37.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.377+0000 7f9d1e55b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d18159420 0x7f9d1815f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37450/0 (socket says 172.21.15.89:37450) 2023-10-10T20:43:37.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.377+0000 7f9d1e55b700 1 -- 172.21.15.89:0/1642825906 learned_addr learned my addr 172.21.15.89:0/1642825906 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:37.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.377+0000 7f9d1ed5c700 1 --2- 172.21.15.89:0/1642825906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d1815d230 0x7f9d18155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:37.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.377+0000 7f9d1e55b700 1 -- 172.21.15.89:0/1642825906 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9d181597f0 msgr2=0x7f9d18154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:37.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.377+0000 7f9d1e55b700 1 --2- 172.21.15.89:0/1642825906 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9d181597f0 0x7f9d18154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:37.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.377+0000 7f9d1e55b700 1 -- 172.21.15.89:0/1642825906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d1815d230 msgr2=0x7f9d18155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:37.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.377+0000 7f9d1e55b700 1 --2- 172.21.15.89:0/1642825906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d1815d230 0x7f9d18155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:37.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.377+0000 7f9d1e55b700 1 -- 172.21.15.89:0/1642825906 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d10009ae0 con 0x7f9d18159420 2023-10-10T20:43:37.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.377+0000 7f9d1ed5c700 1 --2- 172.21.15.89:0/1642825906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d1815d230 0x7f9d18155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:37.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.378+0000 7f9d1e55b700 1 --2- 172.21.15.89:0/1642825906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d18159420 0x7f9d1815f590 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f9d140011e0 tx=0x7f9d14007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:37.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.378+0000 7f9d0e7fc700 1 -- 172.21.15.89:0/1642825906 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14007300 con 0x7f9d18159420 2023-10-10T20:43:37.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.378+0000 7f9d0e7fc700 1 -- 172.21.15.89:0/1642825906 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d14007460 con 0x7f9d18159420 2023-10-10T20:43:37.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.378+0000 7f9d1f55d700 1 -- 172.21.15.89:0/1642825906 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1815c150 con 0x7f9d18159420 2023-10-10T20:43:37.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.378+0000 7f9d0e7fc700 1 -- 172.21.15.89:0/1642825906 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14004830 con 0x7f9d18159420 2023-10-10T20:43:37.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.378+0000 7f9d1f55d700 1 -- 172.21.15.89:0/1642825906 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1815fad0 con 0x7f9d18159420 2023-10-10T20:43:37.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.379+0000 7f9d1f55d700 1 -- 172.21.15.89:0/1642825906 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1804c3c0 con 0x7f9d18159420 2023-10-10T20:43:37.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.379+0000 7f9d0e7fc700 1 -- 172.21.15.89:0/1642825906 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9d14004be0 con 0x7f9d18159420 2023-10-10T20:43:37.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.379+0000 7f9d0e7fc700 1 --2- 172.21.15.89:0/1642825906 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9cfc067320 0x7f9cfc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:37.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.379+0000 7f9d0e7fc700 1 -- 172.21.15.89:0/1642825906 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9d140841b0 con 0x7f9d18159420 2023-10-10T20:43:37.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.379+0000 7f9d1dd5a700 1 --2- 172.21.15.89:0/1642825906 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9cfc067320 0x7f9cfc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:37.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.380+0000 7f9d1dd5a700 1 --2- 172.21.15.89:0/1642825906 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9cfc067320 0x7f9cfc0697d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9d08007fd0 tx=0x7f9d08007b20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:37.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.384+0000 7f9d0e7fc700 1 -- 172.21.15.89:0/1642825906 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d14050ec0 con 0x7f9d18159420 2023-10-10T20:43:37.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.615+0000 7f9d1f55d700 1 -- 172.21.15.89:0/1642825906 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9d180b1790 con 0x7f9d18159420 2023-10-10T20:43:37.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.616+0000 7f9d0e7fc700 1 -- 172.21.15.89:0/1642825906 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v121) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9d140544e0 con 0x7f9d18159420 2023-10-10T20:43:37.618 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:43:37.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.618+0000 7f9d1f55d700 1 -- 172.21.15.89:0/1642825906 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9cfc067320 msgr2=0x7f9cfc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:37.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.618+0000 7f9d1f55d700 1 --2- 172.21.15.89:0/1642825906 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9cfc067320 0x7f9cfc0697d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9d08007fd0 tx=0x7f9d08007b20 comp rx=0 tx=0).stop 2023-10-10T20:43:37.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.618+0000 7f9d1f55d700 1 -- 172.21.15.89:0/1642825906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d18159420 msgr2=0x7f9d1815f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:37.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.618+0000 7f9d1f55d700 1 --2- 172.21.15.89:0/1642825906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d18159420 0x7f9d1815f590 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f9d140011e0 tx=0x7f9d14007e10 comp rx=0 tx=0).stop 2023-10-10T20:43:37.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.618+0000 7f9d1f55d700 1 -- 172.21.15.89:0/1642825906 shutdown_connections 2023-10-10T20:43:37.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.618+0000 7f9d1f55d700 1 --2- 172.21.15.89:0/1642825906 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9cfc067320 0x7f9cfc0697d0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:37.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.618+0000 7f9d1f55d700 1 --2- 172.21.15.89:0/1642825906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d1815d230 0x7f9d18155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:37.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.618+0000 7f9d1f55d700 1 --2- 172.21.15.89:0/1642825906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d18159420 0x7f9d1815f590 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:37.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.618+0000 7f9d1f55d700 1 --2- 172.21.15.89:0/1642825906 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9d181597f0 0x7f9d18154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:37.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.618+0000 7f9d1f55d700 1 -- 172.21.15.89:0/1642825906 >> 172.21.15.89:0/1642825906 conn(0x7f9d180d0660 msgr2=0x7f9d180d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:37.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.618+0000 7f9d1f55d700 1 -- 172.21.15.89:0/1642825906 shutdown_connections 2023-10-10T20:43:37.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:37.618+0000 7f9d1f55d700 1 -- 172.21.15.89:0/1642825906 wait complete. 2023-10-10T20:43:37.631 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:43:37.632 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:43:37.632 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-10-10T20:43:37.632 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:43:37.633 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:43:37.633 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:43:37.633 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:43:37.634 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:43:37.634 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:43:37.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:37.635 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-10-10T20:43:37.653 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:43:37.654 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:43:37.654 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:43:37.654 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:37.654 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-10-10T20:43:37.654 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:37.655 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:43:37.655 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:37.655 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-10-10T20:43:37.655 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:43:37.655 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:37.656 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-10-10T20:43:37.656 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:43:37.656 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:37.656 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:43:37.656 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:37.657 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:43:37.657 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 41965sec preferred_lft 41965sec 2023-10-10T20:43:37.657 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-10-10T20:43:37.657 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:37.657 INFO:teuthology.orchestra.run.smithi111.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:43:37.658 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:43:37.658 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:43:37.658 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:37.658 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::a486:a6ff:fe5f:6021/64 scope link 2023-10-10T20:43:37.658 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:37.659 INFO:teuthology.orchestra.run.smithi111.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:43:37.659 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 1e:d0:81:1b:e2:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:43:37.659 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::1cd0:81ff:fe1b:e26b/64 scope link 2023-10-10T20:43:37.659 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:37.659 INFO:teuthology.orchestra.run.smithi111.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:43:37.660 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:43:37.660 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::88d:cdff:fe75:455e/64 scope link 2023-10-10T20:43:37.660 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:43:37.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:37.661 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:43:37.722 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-10-10T20:43:37.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:37.724 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-10-10T20:43:37.783 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-10-10T20:43:37.784 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:43:37.784 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:43:37.784 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:43:37.784 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:43:37.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:37.785 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:43:37.785 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:43:37.786 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:43:37.786 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:43:37.787 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:43:37.787 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:43:37.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:37.893 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-10-10T20:43:37.908 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:43:37.909 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:43:37.909 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:43:37.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:37.910 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-10-10T20:43:37.975 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:43:37.976 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:43:38.031 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:43:38.031 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:43:38.032 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:43:38.085 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:43:38.086 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:43:38.086 DEBUG:teuthology.orchestra.run.smithi111:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:43:38.165 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:43:38.166 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:43:38.166 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:43:38.174 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:43:38.174 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:43:38.174 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:43:38.174 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:43:38.174 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:43:38.195 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:43:38.196 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:43:38.196 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:43:38.196 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:43:38.196 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:43:38.196 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:43:38.197 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:43:38.197 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:43:38.198 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:43:38.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:38.199 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:43:38.262 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-10T20:43:38.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.380+0000 7ff0790d2700 1 Processor -- start 2023-10-10T20:43:38.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.380+0000 7ff0790d2700 1 -- start start 2023-10-10T20:43:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.381+0000 7ff0790d2700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff0740b55a0 0x7ff074164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.381+0000 7ff0790d2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0740cfa60 con 0x7ff074159ec0 2023-10-10T20:43:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.381+0000 7ff0790d2700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0740b5b00 con 0x7ff0740cd910 2023-10-10T20:43:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.381+0000 7ff0790d2700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0740b5da0 con 0x7ff0740b55a0 2023-10-10T20:43:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.381+0000 7ff072d9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff074159ec0 0x7ff0740b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39786/0 (socket says 172.21.15.89:39786) 2023-10-10T20:43:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.381+0000 7ff072d9d700 1 -- 172.21.15.89:0/4160133713 learned_addr learned my addr 172.21.15.89:0/4160133713 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.381+0000 7ff07359e700 1 --2- 172.21.15.89:0/4160133713 >> v2:172.21.15.111:3300/0 conn(0x7ff0740b55a0 0x7ff074164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.381+0000 7ff07259c700 1 -- 172.21.15.89:0/4160133713 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 758235590 0 0) 0x7ff0740cfa60 con 0x7ff074159ec0 2023-10-10T20:43:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.381+0000 7ff07259c700 1 -- 172.21.15.89:0/4160133713 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff060003570 con 0x7ff074159ec0 2023-10-10T20:43:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff07259c700 1 -- 172.21.15.89:0/4160133713 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989317408 0 0) 0x7ff0740b5b00 con 0x7ff0740cd910 2023-10-10T20:43:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff07259c700 1 -- 172.21.15.89:0/4160133713 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0740cfa60 con 0x7ff0740cd910 2023-10-10T20:43:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff07259c700 1 -- 172.21.15.89:0/4160133713 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3350106670 0 0) 0x7ff060003570 con 0x7ff074159ec0 2023-10-10T20:43:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff07259c700 1 -- 172.21.15.89:0/4160133713 >> v2:172.21.15.111:3300/0 conn(0x7ff0740b55a0 msgr2=0x7ff074164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff07259c700 1 --2- 172.21.15.89:0/4160133713 >> v2:172.21.15.111:3300/0 conn(0x7ff0740b55a0 0x7ff074164400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff07259c700 1 -- 172.21.15.89:0/4160133713 >> v1:172.21.15.111:6789/0 conn(0x7ff0740cd910 legacy=0x7ff0740ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff07259c700 1 -- 172.21.15.89:0/4160133713 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0740b6080 con 0x7ff074159ec0 2023-10-10T20:43:38.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff07259c700 1 -- 172.21.15.89:0/4160133713 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7ff0640015f0 con 0x7ff074159ec0 2023-10-10T20:43:38.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff07259c700 1 -- 172.21.15.89:0/4160133713 >> v1:172.21.15.89:6789/0 conn(0x7ff074159ec0 legacy=0x7ff0740b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:38.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff07259c700 1 --2- 172.21.15.89:0/4160133713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff060003c40 0x7ff060004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:38.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff07259c700 1 --2- 172.21.15.89:0/4160133713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff060004810 0x7ff060004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:38.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff07259c700 1 --2- 172.21.15.89:0/4160133713 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff060005210 0x7ff06000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:38.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff07259c700 1 -- 172.21.15.89:0/4160133713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0740b5da0 con 0x7ff060004810 2023-10-10T20:43:38.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff07259c700 1 -- 172.21.15.89:0/4160133713 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06000c130 con 0x7ff060003c40 2023-10-10T20:43:38.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff07259c700 1 -- 172.21.15.89:0/4160133713 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06000c2f0 con 0x7ff060005210 2023-10-10T20:43:38.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff07359e700 1 --2- 172.21.15.89:0/4160133713 >> v2:172.21.15.111:3300/0 conn(0x7ff0740b55a0 0x7ff074164400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:43:38.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff06a59c700 1 --2- 172.21.15.89:0/4160133713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff060004810 0x7ff060004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:38.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff072d9d700 1 --2- 172.21.15.89:0/4160133713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff060003c40 0x7ff060004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:38.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff07359e700 1 --2- 172.21.15.89:0/4160133713 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff060005210 0x7ff06000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:38.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff06a59c700 1 -- 172.21.15.89:0/4160133713 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff060005210 msgr2=0x7ff06000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:38.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff06a59c700 1 --2- 172.21.15.89:0/4160133713 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff060005210 0x7ff06000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:38.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff06a59c700 1 -- 172.21.15.89:0/4160133713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff060003c40 msgr2=0x7ff060004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:38.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff06a59c700 1 --2- 172.21.15.89:0/4160133713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff060003c40 0x7ff060004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:38.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff06a59c700 1 -- 172.21.15.89:0/4160133713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff060003570 con 0x7ff060004810 2023-10-10T20:43:38.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff07359e700 1 -- 172.21.15.89:0/4160133713 reap_dead start 2023-10-10T20:43:38.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff072d9d700 1 --2- 172.21.15.89:0/4160133713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff060003c40 0x7ff060004030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:43:38.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff06a59c700 1 --2- 172.21.15.89:0/4160133713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff060004810 0x7ff060004c80 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7ff0740c75d0 tx=0x7ff05c007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad80f69125b02ca server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:38.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.382+0000 7ff07259c700 1 -- 172.21.15.89:0/4160133713 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c00c5d0 con 0x7ff060004810 2023-10-10T20:43:38.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.383+0000 7ff07259c700 1 -- 172.21.15.89:0/4160133713 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05c00c750 con 0x7ff060004810 2023-10-10T20:43:38.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.383+0000 7ff07259c700 1 -- 172.21.15.89:0/4160133713 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c00cb10 con 0x7ff060004810 2023-10-10T20:43:38.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.383+0000 7ff0790d2700 1 -- 172.21.15.89:0/4160133713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff060004810 msgr2=0x7ff060004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:38.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.383+0000 7ff0790d2700 1 --2- 172.21.15.89:0/4160133713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff060004810 0x7ff060004c80 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7ff0740c75d0 tx=0x7ff05c007d10 comp rx=0 tx=0).stop 2023-10-10T20:43:38.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.383+0000 7ff0790d2700 1 -- 172.21.15.89:0/4160133713 shutdown_connections 2023-10-10T20:43:38.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.383+0000 7ff0790d2700 1 --2- 172.21.15.89:0/4160133713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff060004810 0x7ff060004c80 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:38.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.383+0000 7ff0790d2700 1 -- 172.21.15.89:0/4160133713 >> 172.21.15.89:0/4160133713 conn(0x7ff0740999f0 msgr2=0x7ff0740b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:38.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.383+0000 7ff0790d2700 1 -- 172.21.15.89:0/4160133713 shutdown_connections 2023-10-10T20:43:38.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.383+0000 7ff0790d2700 1 -- 172.21.15.89:0/4160133713 wait complete. 2023-10-10T20:43:38.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.384+0000 7ff0790d2700 1 Processor -- start 2023-10-10T20:43:38.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.384+0000 7ff0790d2700 1 -- start start 2023-10-10T20:43:38.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.385+0000 7ff0790d2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff060004810 0x7ff07416a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:38.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.385+0000 7ff0790d2700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff07416c1b0 0x7ff07406a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:38.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.385+0000 7ff0790d2700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff07416c620 0x7ff07406ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:38.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.385+0000 7ff0790d2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff074068cc0 con 0x7ff060004810 2023-10-10T20:43:38.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.385+0000 7ff0790d2700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07416a400 con 0x7ff07416c620 2023-10-10T20:43:38.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.385+0000 7ff0790d2700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff074068390 con 0x7ff07416c1b0 2023-10-10T20:43:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.385+0000 7ff072d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff060004810 0x7ff07416a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.385+0000 7ff072d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff060004810 0x7ff07416a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44798/0 (socket says 172.21.15.89:44798) 2023-10-10T20:43:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.385+0000 7ff072d9d700 1 -- 172.21.15.89:0/1209814130 learned_addr learned my addr 172.21.15.89:0/1209814130 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.385+0000 7ff07359e700 1 --2- 172.21.15.89:0/1209814130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff07416c620 0x7ff07406ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.385+0000 7ff06a59c700 1 --2- 172.21.15.89:0/1209814130 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff07416c1b0 0x7ff07406a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.385+0000 7ff072d9d700 1 -- 172.21.15.89:0/1209814130 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff07416c1b0 msgr2=0x7ff07406a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.385+0000 7ff072d9d700 1 --2- 172.21.15.89:0/1209814130 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff07416c1b0 0x7ff07406a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.385+0000 7ff072d9d700 1 -- 172.21.15.89:0/1209814130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff07416c620 msgr2=0x7ff07406ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.385+0000 7ff072d9d700 1 --2- 172.21.15.89:0/1209814130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff07416c620 0x7ff07406ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.385+0000 7ff072d9d700 1 -- 172.21.15.89:0/1209814130 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff074068670 con 0x7ff060004810 2023-10-10T20:43:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.385+0000 7ff07359e700 1 --2- 172.21.15.89:0/1209814130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff07416c620 0x7ff07406ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:43:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.386+0000 7ff072d9d700 1 --2- 172.21.15.89:0/1209814130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff060004810 0x7ff07416a750 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7ff06400feb0 tx=0x7ff064009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.386+0000 7ff06b7fe700 1 -- 172.21.15.89:0/1209814130 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff064000c90 con 0x7ff060004810 2023-10-10T20:43:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.386+0000 7ff06b7fe700 1 -- 172.21.15.89:0/1209814130 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06400d920 con 0x7ff060004810 2023-10-10T20:43:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.386+0000 7ff0790d2700 1 -- 172.21.15.89:0/1209814130 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff074068950 con 0x7ff060004810 2023-10-10T20:43:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.386+0000 7ff06b7fe700 1 -- 172.21.15.89:0/1209814130 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06400dce0 con 0x7ff060004810 2023-10-10T20:43:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.386+0000 7ff0790d2700 1 -- 172.21.15.89:0/1209814130 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff074169910 con 0x7ff060004810 2023-10-10T20:43:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.387+0000 7ff0790d2700 1 -- 172.21.15.89:0/1209814130 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff074068fa0 con 0x7ff060004810 2023-10-10T20:43:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.387+0000 7ff06b7fe700 1 -- 172.21.15.89:0/1209814130 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff0640072e0 con 0x7ff060004810 2023-10-10T20:43:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.387+0000 7ff06b7fe700 1 --2- 172.21.15.89:0/1209814130 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff054067320 0x7ff0540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.388+0000 7ff06b7fe700 1 -- 172.21.15.89:0/1209814130 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff064084db0 con 0x7ff060004810 2023-10-10T20:43:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.388+0000 7ff06a59c700 1 --2- 172.21.15.89:0/1209814130 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff054067320 0x7ff0540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:38.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.388+0000 7ff06a59c700 1 --2- 172.21.15.89:0/1209814130 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff054067320 0x7ff0540697d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff07409e500 tx=0x7ff05c015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:38.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.392+0000 7ff06b7fe700 1 -- 172.21.15.89:0/1209814130 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff064089070 con 0x7ff060004810 2023-10-10T20:43:38.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:38.623+0000 7ff0790d2700 1 -- 172.21.15.89:0/1209814130 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff074063c90 con 0x7ff060004810 2023-10-10T20:43:39.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.456+0000 7ff06b7fe700 1 -- 172.21.15.89:0/1209814130 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v122) v1 ==== 136+0+0 (secure 0 0 0) 0x7ff064051a40 con 0x7ff060004810 2023-10-10T20:43:39.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.458+0000 7ff0790d2700 1 -- 172.21.15.89:0/1209814130 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff054067320 msgr2=0x7ff0540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:39.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.458+0000 7ff0790d2700 1 --2- 172.21.15.89:0/1209814130 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff054067320 0x7ff0540697d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff07409e500 tx=0x7ff05c015000 comp rx=0 tx=0).stop 2023-10-10T20:43:39.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.458+0000 7ff0790d2700 1 -- 172.21.15.89:0/1209814130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff060004810 msgr2=0x7ff07416a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:39.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.458+0000 7ff0790d2700 1 --2- 172.21.15.89:0/1209814130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff060004810 0x7ff07416a750 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7ff06400feb0 tx=0x7ff064009040 comp rx=0 tx=0).stop 2023-10-10T20:43:39.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.458+0000 7ff0790d2700 1 -- 172.21.15.89:0/1209814130 shutdown_connections 2023-10-10T20:43:39.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.458+0000 7ff0790d2700 1 --2- 172.21.15.89:0/1209814130 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff054067320 0x7ff0540697d0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:39.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.458+0000 7ff0790d2700 1 --2- 172.21.15.89:0/1209814130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff07416c620 0x7ff07406ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:39.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.458+0000 7ff0790d2700 1 --2- 172.21.15.89:0/1209814130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff060004810 0x7ff07416a750 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:39.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.458+0000 7ff0790d2700 1 --2- 172.21.15.89:0/1209814130 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff07416c1b0 0x7ff07406a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:39.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.458+0000 7ff0790d2700 1 -- 172.21.15.89:0/1209814130 >> 172.21.15.89:0/1209814130 conn(0x7ff0740999f0 msgr2=0x7ff074158fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:39.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.459+0000 7ff0790d2700 1 -- 172.21.15.89:0/1209814130 shutdown_connections 2023-10-10T20:43:39.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.459+0000 7ff0790d2700 1 -- 172.21.15.89:0/1209814130 wait complete. 2023-10-10T20:43:39.475 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-10T20:43:39.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.595+0000 7f5ec867e700 1 Processor -- start 2023-10-10T20:43:39.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.595+0000 7f5ec867e700 1 -- start start 2023-10-10T20:43:39.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.596+0000 7f5ec867e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5ec0159c60 0x7f5ec015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:39.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.596+0000 7f5ec867e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec015a160 con 0x7f5ec0159890 2023-10-10T20:43:39.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.596+0000 7f5ec867e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec00d13a0 con 0x7f5ec015db10 2023-10-10T20:43:39.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.596+0000 7f5ec867e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec015df50 con 0x7f5ec0159c60 2023-10-10T20:43:39.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.596+0000 7f5ec641a700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5ec0159890 0x7f5ec015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39790/0 (socket says 172.21.15.89:39790) 2023-10-10T20:43:39.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.596+0000 7f5ec5c19700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5ec0159c60 0x7f5ec015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:39.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.596+0000 7f5ec641a700 1 -- 172.21.15.89:0/519708089 learned_addr learned my addr 172.21.15.89:0/519708089 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:39.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.597+0000 7f5ec5c19700 1 -- 172.21.15.89:0/519708089 >> v1:172.21.15.111:6789/0 conn(0x7f5ec015db10 legacy=0x7f5ec0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:39.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.597+0000 7f5ec5c19700 1 -- 172.21.15.89:0/519708089 >> v1:172.21.15.89:6789/0 conn(0x7f5ec0159890 legacy=0x7f5ec015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:39.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.597+0000 7f5ec5c19700 1 -- 172.21.15.89:0/519708089 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec015e140 con 0x7f5ec0159c60 2023-10-10T20:43:39.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.597+0000 7f5ec5418700 1 -- 172.21.15.89:0/519708089 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50083964 0 0) 0x7f5ec015a160 con 0x7f5ec0159890 2023-10-10T20:43:39.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.597+0000 7f5ec5c19700 1 --2- 172.21.15.89:0/519708089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ec0159c60 0x7f5ec015d5d0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00098e0 tx=0x7f5eb00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35b21f9babe298e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.597+0000 7f5ec5418700 1 -- 172.21.15.89:0/519708089 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0009e80 con 0x7f5ec0159c60 2023-10-10T20:43:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.597+0000 7f5ec5418700 1 -- 172.21.15.89:0/519708089 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb000b860 con 0x7f5ec0159c60 2023-10-10T20:43:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.597+0000 7f5ec5418700 1 -- 172.21.15.89:0/519708089 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb000bbe0 con 0x7f5ec0159c60 2023-10-10T20:43:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.597+0000 7f5ec867e700 1 -- 172.21.15.89:0/519708089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ec0159c60 msgr2=0x7f5ec015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.597+0000 7f5ec867e700 1 --2- 172.21.15.89:0/519708089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ec0159c60 0x7f5ec015d5d0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00098e0 tx=0x7f5eb00091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.597+0000 7f5ec867e700 1 -- 172.21.15.89:0/519708089 shutdown_connections 2023-10-10T20:43:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.597+0000 7f5ec867e700 1 --2- 172.21.15.89:0/519708089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ec0159c60 0x7f5ec015d5d0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:39.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.597+0000 7f5ec867e700 1 -- 172.21.15.89:0/519708089 >> 172.21.15.89:0/519708089 conn(0x7f5ec0068e20 msgr2=0x7f5ec00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:39.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.597+0000 7f5ec867e700 1 -- 172.21.15.89:0/519708089 shutdown_connections 2023-10-10T20:43:39.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.598+0000 7f5ec867e700 1 -- 172.21.15.89:0/519708089 wait complete. 2023-10-10T20:43:39.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.599+0000 7f5ec867e700 1 Processor -- start 2023-10-10T20:43:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.599+0000 7f5ec867e700 1 -- start start 2023-10-10T20:43:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.599+0000 7f5ec867e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ec0159890 0x7f5ec016dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.599+0000 7f5ec867e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ec0159c60 0x7f5ec0152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.599+0000 7f5ec867e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5ec015db10 0x7f5ec00ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.599+0000 7f5ec867e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec015bfa0 con 0x7f5ec0159890 2023-10-10T20:43:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.599+0000 7f5ec867e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec00ca7a0 con 0x7f5ec0159c60 2023-10-10T20:43:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.599+0000 7f5ec867e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec015b670 con 0x7f5ec015db10 2023-10-10T20:43:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.599+0000 7f5ec641a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ec0159890 0x7f5ec016dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.599+0000 7f5ec641a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ec0159890 0x7f5ec016dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44822/0 (socket says 172.21.15.89:44822) 2023-10-10T20:43:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.599+0000 7f5ec641a700 1 -- 172.21.15.89:0/3226761849 learned_addr learned my addr 172.21.15.89:0/3226761849 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.600+0000 7f5ec5c19700 1 --2- 172.21.15.89:0/3226761849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ec0159c60 0x7f5ec0152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.600+0000 7f5ec6c1b700 1 --2- 172.21.15.89:0/3226761849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5ec015db10 0x7f5ec00ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.600+0000 7f5ec641a700 1 -- 172.21.15.89:0/3226761849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5ec015db10 msgr2=0x7f5ec00ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.600+0000 7f5ec641a700 1 --2- 172.21.15.89:0/3226761849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5ec015db10 0x7f5ec00ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.600+0000 7f5ec641a700 1 -- 172.21.15.89:0/3226761849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ec0159c60 msgr2=0x7f5ec0152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.600+0000 7f5ec641a700 1 --2- 172.21.15.89:0/3226761849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ec0159c60 0x7f5ec0152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.600+0000 7f5ec641a700 1 -- 172.21.15.89:0/3226761849 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb0009420 con 0x7f5ec0159890 2023-10-10T20:43:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.600+0000 7f5ec6c1b700 1 --2- 172.21.15.89:0/3226761849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5ec015db10 0x7f5ec00ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.600+0000 7f5ec641a700 1 --2- 172.21.15.89:0/3226761849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ec0159890 0x7f5ec016dcb0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00c810 tx=0x7f5ebc007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:39.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.600+0000 7f5eb67fc700 1 -- 172.21.15.89:0/3226761849 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc005b30 con 0x7f5ec0159890 2023-10-10T20:43:39.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.600+0000 7f5eb67fc700 1 -- 172.21.15.89:0/3226761849 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ebc005cb0 con 0x7f5ec0159890 2023-10-10T20:43:39.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.600+0000 7f5ec867e700 1 -- 172.21.15.89:0/3226761849 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec015b950 con 0x7f5ec0159890 2023-10-10T20:43:39.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.600+0000 7f5eb67fc700 1 -- 172.21.15.89:0/3226761849 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc007320 con 0x7f5ec0159890 2023-10-10T20:43:39.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.600+0000 7f5ec867e700 1 -- 172.21.15.89:0/3226761849 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec00b0770 con 0x7f5ec0159890 2023-10-10T20:43:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.601+0000 7f5ec867e700 1 -- 172.21.15.89:0/3226761849 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec004c3c0 con 0x7f5ec0159890 2023-10-10T20:43:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.601+0000 7f5eb67fc700 1 -- 172.21.15.89:0/3226761849 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5ebc01b3b0 con 0x7f5ec0159890 2023-10-10T20:43:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.602+0000 7f5eb67fc700 1 --2- 172.21.15.89:0/3226761849 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5ea4067320 0x7f5ea40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.602+0000 7f5eb67fc700 1 -- 172.21.15.89:0/3226761849 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5ebc084120 con 0x7f5ec0159890 2023-10-10T20:43:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.602+0000 7f5ec5c19700 1 --2- 172.21.15.89:0/3226761849 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5ea4067320 0x7f5ea40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.602+0000 7f5ec5c19700 1 --2- 172.21.15.89:0/3226761849 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5ea4067320 0x7f5ea40697d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00093f0 tx=0x7f5eb0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.606+0000 7f5eb67fc700 1 -- 172.21.15.89:0/3226761849 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ebc050db0 con 0x7f5ec0159890 2023-10-10T20:43:39.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.814+0000 7f5ec867e700 1 -- 172.21.15.89:0/3226761849 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ec015c0e0 con 0x7f5ea4067320 2023-10-10T20:43:39.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.815+0000 7f5eb67fc700 1 -- 172.21.15.89:0/3226761849 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f5eb000ba60 con 0x7f5ea4067320 2023-10-10T20:43:39.817 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:43:39.817 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:39.817 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs" 2023-10-10T20:43:39.817 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:39.818 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:43:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.817+0000 7f5ec867e700 1 -- 172.21.15.89:0/3226761849 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5ea4067320 msgr2=0x7f5ea40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.817+0000 7f5ec867e700 1 --2- 172.21.15.89:0/3226761849 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5ea4067320 0x7f5ea40697d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00093f0 tx=0x7f5eb0017000 comp rx=0 tx=0).stop 2023-10-10T20:43:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.817+0000 7f5ec867e700 1 -- 172.21.15.89:0/3226761849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ec0159890 msgr2=0x7f5ec016dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:39.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.817+0000 7f5ec867e700 1 --2- 172.21.15.89:0/3226761849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ec0159890 0x7f5ec016dcb0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00c810 tx=0x7f5ebc007d90 comp rx=0 tx=0).stop 2023-10-10T20:43:39.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.817+0000 7f5ec867e700 1 -- 172.21.15.89:0/3226761849 shutdown_connections 2023-10-10T20:43:39.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.817+0000 7f5ec867e700 1 --2- 172.21.15.89:0/3226761849 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5ec015db10 0x7f5ec00ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:39.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.817+0000 7f5ec867e700 1 --2- 172.21.15.89:0/3226761849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ec0159890 0x7f5ec016dcb0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:39.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.817+0000 7f5ec867e700 1 --2- 172.21.15.89:0/3226761849 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5ea4067320 0x7f5ea40697d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.817+0000 7f5ec867e700 1 --2- 172.21.15.89:0/3226761849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ec0159c60 0x7f5ec0152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.817+0000 7f5ec867e700 1 -- 172.21.15.89:0/3226761849 >> 172.21.15.89:0/3226761849 conn(0x7f5ec0068e20 msgr2=0x7f5ec0153fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.817+0000 7f5ec867e700 1 -- 172.21.15.89:0/3226761849 shutdown_connections 2023-10-10T20:43:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.817+0000 7f5ec867e700 1 -- 172.21.15.89:0/3226761849 wait complete. 2023-10-10T20:43:39.831 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:43:39.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.947+0000 7f7038f9b700 1 Processor -- start 2023-10-10T20:43:39.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.947+0000 7f7038f9b700 1 -- start start 2023-10-10T20:43:39.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.948+0000 7f7038f9b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7034159930 0x7f7034159d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:39.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.948+0000 7f7038f9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f70340cb710 0x7f70340b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:39.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.948+0000 7f7038f9b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f70340cd1c0 0x7f70340cd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:39.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.948+0000 7f7038f9b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70340cdb80 con 0x7f70340cb710 2023-10-10T20:43:39.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.948+0000 7f7038f9b700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70340689e0 con 0x7f70340cd1c0 2023-10-10T20:43:39.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.948+0000 7f7038f9b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70340cbcc0 con 0x7f7034159930 2023-10-10T20:43:39.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.948+0000 7f7029d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f70340cb710 0x7f70340b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:39.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.948+0000 7f7029d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f70340cb710 0x7f70340b3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44826/0 (socket says 172.21.15.89:44826) 2023-10-10T20:43:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.948+0000 7f7029d9b700 1 -- 172.21.15.89:0/2135650506 learned_addr learned my addr 172.21.15.89:0/2135650506 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.948+0000 7f7032d9d700 1 --2- 172.21.15.89:0/2135650506 >> v2:172.21.15.111:3300/0 conn(0x7f70340cd1c0 0x7f70340cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.948+0000 7f703259c700 1 --2- 172.21.15.89:0/2135650506 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7034159930 0x7f7034159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.948+0000 7f7029d9b700 1 -- 172.21.15.89:0/2135650506 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7034159930 msgr2=0x7f7034159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.948+0000 7f7029d9b700 1 --2- 172.21.15.89:0/2135650506 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7034159930 0x7f7034159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:39.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.948+0000 7f7029d9b700 1 -- 172.21.15.89:0/2135650506 >> v2:172.21.15.111:3300/0 conn(0x7f70340cd1c0 msgr2=0x7f70340cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:39.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.948+0000 7f7029d9b700 1 --2- 172.21.15.89:0/2135650506 >> v2:172.21.15.111:3300/0 conn(0x7f70340cd1c0 0x7f70340cd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:39.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.948+0000 7f7029d9b700 1 -- 172.21.15.89:0/2135650506 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70340cbfa0 con 0x7f70340cb710 2023-10-10T20:43:39.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.948+0000 7f7032d9d700 1 --2- 172.21.15.89:0/2135650506 >> v2:172.21.15.111:3300/0 conn(0x7f70340cd1c0 0x7f70340cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:39.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.949+0000 7f7029d9b700 1 --2- 172.21.15.89:0/2135650506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70340cb710 0x7f70340b3ef0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f7034088480 tx=0x7f701c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cef7215649960a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:39.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.949+0000 7f7031d9b700 1 -- 172.21.15.89:0/2135650506 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701c009e70 con 0x7f70340cb710 2023-10-10T20:43:39.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.949+0000 7f7031d9b700 1 -- 172.21.15.89:0/2135650506 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701c00b8f0 con 0x7f70340cb710 2023-10-10T20:43:39.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.949+0000 7f7031d9b700 1 -- 172.21.15.89:0/2135650506 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701c00bc90 con 0x7f70340cb710 2023-10-10T20:43:39.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.949+0000 7f7038f9b700 1 -- 172.21.15.89:0/2135650506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70340cb710 msgr2=0x7f70340b3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:39.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.949+0000 7f7038f9b700 1 --2- 172.21.15.89:0/2135650506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70340cb710 0x7f70340b3ef0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f7034088480 tx=0x7f701c009250 comp rx=0 tx=0).stop 2023-10-10T20:43:39.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.949+0000 7f7038f9b700 1 -- 172.21.15.89:0/2135650506 shutdown_connections 2023-10-10T20:43:39.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.949+0000 7f7038f9b700 1 --2- 172.21.15.89:0/2135650506 >> v2:172.21.15.111:3300/0 conn(0x7f70340cd1c0 0x7f70340cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:39.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.949+0000 7f7038f9b700 1 --2- 172.21.15.89:0/2135650506 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7034159930 0x7f7034159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:39.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.949+0000 7f7038f9b700 1 --2- 172.21.15.89:0/2135650506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70340cb710 0x7f70340b3ef0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:39.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.949+0000 7f7038f9b700 1 -- 172.21.15.89:0/2135650506 >> 172.21.15.89:0/2135650506 conn(0x7f70340999d0 msgr2=0x7f70340b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:39.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.949+0000 7f7038f9b700 1 -- 172.21.15.89:0/2135650506 shutdown_connections 2023-10-10T20:43:39.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.949+0000 7f7038f9b700 1 -- 172.21.15.89:0/2135650506 wait complete. 2023-10-10T20:43:39.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.950+0000 7f7038f9b700 1 Processor -- start 2023-10-10T20:43:39.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.951+0000 7f7038f9b700 1 -- start start 2023-10-10T20:43:39.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.951+0000 7f7038f9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70340cb710 0x7f703415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:39.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.951+0000 7f7038f9b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70340cd1c0 0x7f70340c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:39.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.951+0000 7f7038f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7034159930 0x7f70340c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:39.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.951+0000 7f7038f9b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70340c7870 con 0x7f7034159930 2023-10-10T20:43:39.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.951+0000 7f7038f9b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70340c7b50 con 0x7f70340cb710 2023-10-10T20:43:39.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.951+0000 7f7038f9b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70340c7e30 con 0x7f70340cd1c0 2023-10-10T20:43:39.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.951+0000 7f7032d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7034159930 0x7f70340c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:39.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.951+0000 7f7032d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7034159930 0x7f70340c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44842/0 (socket says 172.21.15.89:44842) 2023-10-10T20:43:39.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.951+0000 7f7032d9d700 1 -- 172.21.15.89:0/2418200859 learned_addr learned my addr 172.21.15.89:0/2418200859 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:39.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.951+0000 7f7029d9b700 1 --2- 172.21.15.89:0/2418200859 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70340cd1c0 0x7f70340c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:39.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.952+0000 7f703259c700 1 --2- 172.21.15.89:0/2418200859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70340cb710 0x7f703415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:39.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.952+0000 7f7032d9d700 1 -- 172.21.15.89:0/2418200859 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70340cd1c0 msgr2=0x7f70340c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:39.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.952+0000 7f7032d9d700 1 --2- 172.21.15.89:0/2418200859 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70340cd1c0 0x7f70340c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:39.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.952+0000 7f7032d9d700 1 -- 172.21.15.89:0/2418200859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70340cb710 msgr2=0x7f703415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:39.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.952+0000 7f7032d9d700 1 --2- 172.21.15.89:0/2418200859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70340cb710 0x7f703415ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:39.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.952+0000 7f7032d9d700 1 -- 172.21.15.89:0/2418200859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c0094f0 con 0x7f7034159930 2023-10-10T20:43:39.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.952+0000 7f7032d9d700 1 --2- 172.21.15.89:0/2418200859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7034159930 0x7f70340c9ee0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f702c00d180 tx=0x7f702c009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:39.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.952+0000 7f702affd700 1 -- 172.21.15.89:0/2418200859 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c00de20 con 0x7f7034159930 2023-10-10T20:43:39.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.952+0000 7f702affd700 1 -- 172.21.15.89:0/2418200859 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702c00f040 con 0x7f7034159930 2023-10-10T20:43:39.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.952+0000 7f7038f9b700 1 -- 172.21.15.89:0/2418200859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70340c8110 con 0x7f7034159930 2023-10-10T20:43:39.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.952+0000 7f702affd700 1 -- 172.21.15.89:0/2418200859 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c015690 con 0x7f7034159930 2023-10-10T20:43:39.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.952+0000 7f7038f9b700 1 -- 172.21.15.89:0/2418200859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703415f130 con 0x7f7034159930 2023-10-10T20:43:39.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.953+0000 7f7038f9b700 1 -- 172.21.15.89:0/2418200859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f703404c3c0 con 0x7f7034159930 2023-10-10T20:43:39.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.953+0000 7f702affd700 1 -- 172.21.15.89:0/2418200859 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f702c01e3b0 con 0x7f7034159930 2023-10-10T20:43:39.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.953+0000 7f702affd700 1 --2- 172.21.15.89:0/2418200859 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7014067320 0x7f70140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:39.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.954+0000 7f702affd700 1 -- 172.21.15.89:0/2418200859 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f702c085390 con 0x7f7034159930 2023-10-10T20:43:39.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.954+0000 7f703259c700 1 --2- 172.21.15.89:0/2418200859 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7014067320 0x7f70140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:39.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.954+0000 7f703259c700 1 --2- 172.21.15.89:0/2418200859 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7014067320 0x7f70140697d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7024009d30 tx=0x7f7024008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:39.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:39.958+0000 7f702affd700 1 -- 172.21.15.89:0/2418200859 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f702c051f10 con 0x7f7034159930 2023-10-10T20:43:40.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.143+0000 7f7038f9b700 1 -- 172.21.15.89:0/2418200859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7034067b30 con 0x7f7034159930 2023-10-10T20:43:40.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.144+0000 7f702affd700 1 -- 172.21.15.89:0/2418200859 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f702c055530 con 0x7f7034159930 2023-10-10T20:43:40.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.146+0000 7f7038f9b700 1 -- 172.21.15.89:0/2418200859 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7014067320 msgr2=0x7f70140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:40.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.146+0000 7f7038f9b700 1 --2- 172.21.15.89:0/2418200859 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7014067320 0x7f70140697d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7024009d30 tx=0x7f7024008040 comp rx=0 tx=0).stop 2023-10-10T20:43:40.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.146+0000 7f7038f9b700 1 -- 172.21.15.89:0/2418200859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7034159930 msgr2=0x7f70340c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:40.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.146+0000 7f7038f9b700 1 --2- 172.21.15.89:0/2418200859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7034159930 0x7f70340c9ee0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f702c00d180 tx=0x7f702c009a50 comp rx=0 tx=0).stop 2023-10-10T20:43:40.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.146+0000 7f7038f9b700 1 -- 172.21.15.89:0/2418200859 shutdown_connections 2023-10-10T20:43:40.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.146+0000 7f7038f9b700 1 --2- 172.21.15.89:0/2418200859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7034159930 0x7f70340c9ee0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:40.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.146+0000 7f7038f9b700 1 --2- 172.21.15.89:0/2418200859 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7014067320 0x7f70140697d0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:40.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.146+0000 7f7038f9b700 1 --2- 172.21.15.89:0/2418200859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70340cb710 0x7f703415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:40.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.146+0000 7f7038f9b700 1 --2- 172.21.15.89:0/2418200859 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f70340cd1c0 0x7f70340c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:40.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.146+0000 7f7038f9b700 1 -- 172.21.15.89:0/2418200859 >> 172.21.15.89:0/2418200859 conn(0x7f70340999d0 msgr2=0x7f7034068d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:40.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.146+0000 7f7038f9b700 1 -- 172.21.15.89:0/2418200859 shutdown_connections 2023-10-10T20:43:40.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.146+0000 7f7038f9b700 1 -- 172.21.15.89:0/2418200859 wait complete. 2023-10-10T20:43:40.159 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:43:40.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:43:40.160 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:43:40.198 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:43:40.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:40.199 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:43:40.253 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:43:40.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:43:40.254 DEBUG:teuthology.orchestra.run.smithi111:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:43:40.296 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:43:40.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:40.297 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:43:40.353 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:43:40.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:40.354 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-10T20:43:40.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:43:40.369 INFO:teuthology.orchestra.run.smithi089.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-10T20:43:40.370 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:43:40.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:43:40.371 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-10T20:43:40.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:43:40.410 INFO:teuthology.orchestra.run.smithi111.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-10T20:43:40.411 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000812094 --yes-i-really-mean-it 2023-10-10T20:43:40.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.528+0000 7f54c26bb700 1 Processor -- start 2023-10-10T20:43:40.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.528+0000 7f54c26bb700 1 -- start start 2023-10-10T20:43:40.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.529+0000 7f54c26bb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f54bc159ec0 0x7f54bc0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:40.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.529+0000 7f54c26bb700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f54bc0b53d0 0x7f54bc0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:40.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.529+0000 7f54c26bb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54bc0b5b00 con 0x7f54bc0cce40 2023-10-10T20:43:40.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.529+0000 7f54c26bb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54bc0b5850 con 0x7f54bc159ec0 2023-10-10T20:43:40.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.529+0000 7f54c26bb700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54bc0b5da0 con 0x7f54bc0b53d0 2023-10-10T20:43:40.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.529+0000 7f54bbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f54bc159ec0 0x7f54bc0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:40.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.529+0000 7f54c0c58700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f54bc0cce40 0x7f54bc0cd260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39806/0 (socket says 172.21.15.89:39806) 2023-10-10T20:43:40.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.529+0000 7f54c0c58700 1 -- 172.21.15.89:0/1916398625 learned_addr learned my addr 172.21.15.89:0/1916398625 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:40.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.529+0000 7f54bb7fe700 1 --2- 172.21.15.89:0/1916398625 >> v2:172.21.15.111:3300/0 conn(0x7f54bc0b53d0 0x7f54bc0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:40.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.529+0000 7f54bbfff700 1 -- 172.21.15.89:0/1916398625 >> v2:172.21.15.111:3300/0 conn(0x7f54bc0b53d0 msgr2=0x7f54bc0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:40.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.529+0000 7f54bbfff700 1 --2- 172.21.15.89:0/1916398625 >> v2:172.21.15.111:3300/0 conn(0x7f54bc0b53d0 0x7f54bc0cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:40.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.529+0000 7f54bbfff700 1 -- 172.21.15.89:0/1916398625 >> v1:172.21.15.89:6789/0 conn(0x7f54bc0cce40 legacy=0x7f54bc0cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:40.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.529+0000 7f54bbfff700 1 -- 172.21.15.89:0/1916398625 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54bc0b6080 con 0x7f54bc159ec0 2023-10-10T20:43:40.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.529+0000 7f54bbfff700 1 --2- 172.21.15.89:0/1916398625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54bc159ec0 0x7f54bc0b4e90 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0b64d0 tx=0x7f54b0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb10a8f17998b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:40.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.529+0000 7f54baffd700 1 -- 172.21.15.89:0/1916398625 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b0005c10 con 0x7f54bc159ec0 2023-10-10T20:43:40.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.529+0000 7f54baffd700 1 -- 172.21.15.89:0/1916398625 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b0005d70 con 0x7f54bc159ec0 2023-10-10T20:43:40.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.529+0000 7f54baffd700 1 -- 172.21.15.89:0/1916398625 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b0006260 con 0x7f54bc159ec0 2023-10-10T20:43:40.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.530+0000 7f54c26bb700 1 -- 172.21.15.89:0/1916398625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54bc159ec0 msgr2=0x7f54bc0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:40.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.530+0000 7f54c26bb700 1 --2- 172.21.15.89:0/1916398625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54bc159ec0 0x7f54bc0b4e90 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0b64d0 tx=0x7f54b0009120 comp rx=0 tx=0).stop 2023-10-10T20:43:40.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.530+0000 7f54c26bb700 1 -- 172.21.15.89:0/1916398625 shutdown_connections 2023-10-10T20:43:40.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.530+0000 7f54c26bb700 1 --2- 172.21.15.89:0/1916398625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54bc159ec0 0x7f54bc0b4e90 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:40.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.530+0000 7f54c26bb700 1 --2- 172.21.15.89:0/1916398625 >> v2:172.21.15.111:3300/0 conn(0x7f54bc0b53d0 0x7f54bc0cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:40.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.530+0000 7f54c26bb700 1 -- 172.21.15.89:0/1916398625 >> 172.21.15.89:0/1916398625 conn(0x7f54bc0999f0 msgr2=0x7f54bc0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:40.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.530+0000 7f54c26bb700 1 -- 172.21.15.89:0/1916398625 shutdown_connections 2023-10-10T20:43:40.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.530+0000 7f54c26bb700 1 -- 172.21.15.89:0/1916398625 wait complete. 2023-10-10T20:43:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.531+0000 7f54c26bb700 1 Processor -- start 2023-10-10T20:43:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.531+0000 7f54c26bb700 1 -- start start 2023-10-10T20:43:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.532+0000 7f54c26bb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54bc0b53d0 0x7f54bc1574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.532+0000 7f54c26bb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54bc0cce40 0x7f54bc0b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.532+0000 7f54c26bb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54bc159ec0 0x7f54bc09a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.532+0000 7f54c26bb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc0b2230 con 0x7f54bc0b53d0 2023-10-10T20:43:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.532+0000 7f54c26bb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc16e120 con 0x7f54bc159ec0 2023-10-10T20:43:40.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.532+0000 7f54c26bb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc068e60 con 0x7f54bc0cce40 2023-10-10T20:43:40.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.532+0000 7f54bbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54bc0b53d0 0x7f54bc1574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:40.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.532+0000 7f54bbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54bc0b53d0 0x7f54bc1574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44870/0 (socket says 172.21.15.89:44870) 2023-10-10T20:43:40.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.532+0000 7f54bbfff700 1 -- 172.21.15.89:0/4120525382 learned_addr learned my addr 172.21.15.89:0/4120525382 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:40.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.532+0000 7f54bb7fe700 1 --2- 172.21.15.89:0/4120525382 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54bc0cce40 0x7f54bc0b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.532+0000 7f54c0c58700 1 --2- 172.21.15.89:0/4120525382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54bc159ec0 0x7f54bc09a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.532+0000 7f54bbfff700 1 -- 172.21.15.89:0/4120525382 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54bc0cce40 msgr2=0x7f54bc0b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.532+0000 7f54bbfff700 1 --2- 172.21.15.89:0/4120525382 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54bc0cce40 0x7f54bc0b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.532+0000 7f54bbfff700 1 -- 172.21.15.89:0/4120525382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54bc159ec0 msgr2=0x7f54bc09a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.532+0000 7f54bbfff700 1 --2- 172.21.15.89:0/4120525382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54bc159ec0 0x7f54bc09a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:40.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.532+0000 7f54bbfff700 1 -- 172.21.15.89:0/4120525382 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b00093c0 con 0x7f54bc0b53d0 2023-10-10T20:43:40.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.532+0000 7f54c0c58700 1 --2- 172.21.15.89:0/4120525382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54bc159ec0 0x7f54bc09a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:40.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.532+0000 7f54bb7fe700 1 --2- 172.21.15.89:0/4120525382 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54bc0cce40 0x7f54bc0b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:40.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.532+0000 7f54bbfff700 1 --2- 172.21.15.89:0/4120525382 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54bc0b53d0 0x7f54bc1574e0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f54bc089390 tx=0x7f54b0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:40.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.533+0000 7f54a3fff700 1 -- 172.21.15.89:0/4120525382 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b000bcf0 con 0x7f54bc0b53d0 2023-10-10T20:43:40.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.533+0000 7f54a3fff700 1 -- 172.21.15.89:0/4120525382 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b000be70 con 0x7f54bc0b53d0 2023-10-10T20:43:40.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.533+0000 7f54c26bb700 1 -- 172.21.15.89:0/4120525382 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54bc068530 con 0x7f54bc0b53d0 2023-10-10T20:43:40.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.533+0000 7f54a3fff700 1 -- 172.21.15.89:0/4120525382 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b001a670 con 0x7f54bc0b53d0 2023-10-10T20:43:40.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.533+0000 7f54c26bb700 1 -- 172.21.15.89:0/4120525382 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54bc068af0 con 0x7f54bc0b53d0 2023-10-10T20:43:40.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.534+0000 7f54c26bb700 1 -- 172.21.15.89:0/4120525382 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54bc063c90 con 0x7f54bc0b53d0 2023-10-10T20:43:40.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.534+0000 7f54a3fff700 1 -- 172.21.15.89:0/4120525382 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f54b001aa20 con 0x7f54bc0b53d0 2023-10-10T20:43:40.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.534+0000 7f54a3fff700 1 --2- 172.21.15.89:0/4120525382 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f54a8067320 0x7f54a80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:40.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.534+0000 7f54a3fff700 1 -- 172.21.15.89:0/4120525382 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f54b00865b0 con 0x7f54bc0b53d0 2023-10-10T20:43:40.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.534+0000 7f54bb7fe700 1 --2- 172.21.15.89:0/4120525382 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f54a8067320 0x7f54a80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:40.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.535+0000 7f54bb7fe700 1 --2- 172.21.15.89:0/4120525382 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f54a8067320 0x7f54a80697d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f54bc157a20 tx=0x7f54ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:40.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.538+0000 7f54a3fff700 1 -- 172.21.15.89:0/4120525382 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f54b0053240 con 0x7f54bc0b53d0 2023-10-10T20:43:40.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:40.750+0000 7f54c26bb700 1 -- 172.21.15.89:0/4120525382 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000812094", "yes_i_really_mean_it": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f54bc0718b0 con 0x7f54a8067320 2023-10-10T20:43:43.085 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:43:43.088 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:43:43.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.468+0000 7f54a3fff700 1 -- 172.21.15.89:0/4120525382 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+56 (secure 0 0 0) 0x7f54bc0718b0 con 0x7f54a8067320 2023-10-10T20:43:43.470 INFO:teuthology.orchestra.run.smithi089.stdout:FS volume 'cephfs' renamed to 'volume_0000000000812094' 2023-10-10T20:43:43.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.471+0000 7f54c26bb700 1 -- 172.21.15.89:0/4120525382 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f54a8067320 msgr2=0x7f54a80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:43.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.471+0000 7f54c26bb700 1 --2- 172.21.15.89:0/4120525382 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f54a8067320 0x7f54a80697d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f54bc157a20 tx=0x7f54ac008040 comp rx=0 tx=0).stop 2023-10-10T20:43:43.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.471+0000 7f54c26bb700 1 -- 172.21.15.89:0/4120525382 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54bc0b53d0 msgr2=0x7f54bc1574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:43.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.471+0000 7f54c26bb700 1 --2- 172.21.15.89:0/4120525382 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54bc0b53d0 0x7f54bc1574e0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f54bc089390 tx=0x7f54b0016000 comp rx=0 tx=0).stop 2023-10-10T20:43:43.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.471+0000 7f54c26bb700 1 -- 172.21.15.89:0/4120525382 shutdown_connections 2023-10-10T20:43:43.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.471+0000 7f54c26bb700 1 --2- 172.21.15.89:0/4120525382 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f54a8067320 0x7f54a80697d0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:43.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.471+0000 7f54c26bb700 1 --2- 172.21.15.89:0/4120525382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54bc159ec0 0x7f54bc09a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:43.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.471+0000 7f54c26bb700 1 --2- 172.21.15.89:0/4120525382 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54bc0b53d0 0x7f54bc1574e0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:43.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.471+0000 7f54c26bb700 1 --2- 172.21.15.89:0/4120525382 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54bc0cce40 0x7f54bc0b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:43.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.471+0000 7f54c26bb700 1 -- 172.21.15.89:0/4120525382 >> 172.21.15.89:0/4120525382 conn(0x7f54bc0999f0 msgr2=0x7f54bc159350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:43.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.471+0000 7f54c26bb700 1 -- 172.21.15.89:0/4120525382 shutdown_connections 2023-10-10T20:43:43.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.471+0000 7f54c26bb700 1 -- 172.21.15.89:0/4120525382 wait complete. 2023-10-10T20:43:43.488 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000812094 --yes-i-really-mean-it 2023-10-10T20:43:43.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.610+0000 7f9c5ed19700 1 Processor -- start 2023-10-10T20:43:43.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.610+0000 7f9c5ed19700 1 -- start start 2023-10-10T20:43:43.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.611+0000 7f9c5ed19700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9c58159ec0 0x7f9c580b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:43.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.611+0000 7f9c5ed19700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9c580b53d0 0x7f9c580cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:43.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.611+0000 7f9c5ed19700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c580b5b00 con 0x7f9c580cce40 2023-10-10T20:43:43.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.611+0000 7f9c5ed19700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c580b5da0 con 0x7f9c580b53d0 2023-10-10T20:43:43.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.611+0000 7f9c5ed19700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c580b6080 con 0x7f9c58159ec0 2023-10-10T20:43:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.611+0000 7f9c4ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9c580b53d0 0x7f9c580cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.611+0000 7f9c4ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9c580b53d0 0x7f9c580cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44884/0 (socket says 172.21.15.89:44884) 2023-10-10T20:43:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.611+0000 7f9c4ffff700 1 -- 172.21.15.89:0/11710163 learned_addr learned my addr 172.21.15.89:0/11710163 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.611+0000 7f9c5cab5700 1 --2- 172.21.15.89:0/11710163 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9c58159ec0 0x7f9c580b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.611+0000 7f9c4ffff700 1 -- 172.21.15.89:0/11710163 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9c58159ec0 msgr2=0x7f9c580b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.611+0000 7f9c4ffff700 1 --2- 172.21.15.89:0/11710163 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9c58159ec0 0x7f9c580b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.611+0000 7f9c4ffff700 1 -- 172.21.15.89:0/11710163 >> v1:172.21.15.111:6789/0 conn(0x7f9c580cce40 legacy=0x7f9c580cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.611+0000 7f9c4ffff700 1 -- 172.21.15.89:0/11710163 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c580cc100 con 0x7f9c580b53d0 2023-10-10T20:43:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.612+0000 7f9c4ffff700 1 --2- 172.21.15.89:0/11710163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c580b53d0 0x7f9c580cc900 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f9c480099b0 tx=0x7f9c48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3824c2eb0a61631 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.612+0000 7f9c4f7fe700 1 -- 172.21.15.89:0/11710163 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4800b860 con 0x7f9c580b53d0 2023-10-10T20:43:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.612+0000 7f9c4f7fe700 1 -- 172.21.15.89:0/11710163 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4800b9c0 con 0x7f9c580b53d0 2023-10-10T20:43:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.612+0000 7f9c4f7fe700 1 -- 172.21.15.89:0/11710163 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4800bd60 con 0x7f9c580b53d0 2023-10-10T20:43:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.612+0000 7f9c5ed19700 1 -- 172.21.15.89:0/11710163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c580b53d0 msgr2=0x7f9c580cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.612+0000 7f9c5ed19700 1 --2- 172.21.15.89:0/11710163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c580b53d0 0x7f9c580cc900 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f9c480099b0 tx=0x7f9c48009250 comp rx=0 tx=0).stop 2023-10-10T20:43:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.612+0000 7f9c5ed19700 1 -- 172.21.15.89:0/11710163 shutdown_connections 2023-10-10T20:43:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.612+0000 7f9c5ed19700 1 --2- 172.21.15.89:0/11710163 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9c58159ec0 0x7f9c580b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.612+0000 7f9c5ed19700 1 --2- 172.21.15.89:0/11710163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c580b53d0 0x7f9c580cc900 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.612+0000 7f9c5ed19700 1 -- 172.21.15.89:0/11710163 >> 172.21.15.89:0/11710163 conn(0x7f9c580999f0 msgr2=0x7f9c580b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.612+0000 7f9c5ed19700 1 -- 172.21.15.89:0/11710163 shutdown_connections 2023-10-10T20:43:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.612+0000 7f9c5ed19700 1 -- 172.21.15.89:0/11710163 wait complete. 2023-10-10T20:43:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.613+0000 7f9c5ed19700 1 Processor -- start 2023-10-10T20:43:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.614+0000 7f9c5ed19700 1 -- start start 2023-10-10T20:43:43.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.614+0000 7f9c5ed19700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c580b53d0 0x7f9c58157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:43.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.614+0000 7f9c5ed19700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9c580cce40 0x7f9c5815b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:43.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.614+0000 7f9c5ed19700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c58159ec0 0x7f9c5815bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:43.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.614+0000 7f9c5ed19700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5815d590 con 0x7f9c580b53d0 2023-10-10T20:43:43.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.614+0000 7f9c5ed19700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5815d870 con 0x7f9c58159ec0 2023-10-10T20:43:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.614+0000 7f9c5ed19700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5815db50 con 0x7f9c580cce40 2023-10-10T20:43:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.614+0000 7f9c5cab5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c580b53d0 0x7f9c58157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.614+0000 7f9c5cab5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c580b53d0 0x7f9c58157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44892/0 (socket says 172.21.15.89:44892) 2023-10-10T20:43:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.614+0000 7f9c5cab5700 1 -- 172.21.15.89:0/2046421843 learned_addr learned my addr 172.21.15.89:0/2046421843 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.615+0000 7f9c4ffff700 1 --2- 172.21.15.89:0/2046421843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9c580cce40 0x7f9c5815b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.615+0000 7f9c5d2b6700 1 --2- 172.21.15.89:0/2046421843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c58159ec0 0x7f9c5815bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.615+0000 7f9c5cab5700 1 -- 172.21.15.89:0/2046421843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9c580cce40 msgr2=0x7f9c5815b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.615+0000 7f9c5cab5700 1 --2- 172.21.15.89:0/2046421843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9c580cce40 0x7f9c5815b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.615+0000 7f9c5cab5700 1 -- 172.21.15.89:0/2046421843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c58159ec0 msgr2=0x7f9c5815bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.615+0000 7f9c5cab5700 1 --2- 172.21.15.89:0/2046421843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c58159ec0 0x7f9c5815bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.615+0000 7f9c5cab5700 1 -- 172.21.15.89:0/2046421843 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c480094f0 con 0x7f9c580b53d0 2023-10-10T20:43:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.615+0000 7f9c4ffff700 1 --2- 172.21.15.89:0/2046421843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9c580cce40 0x7f9c5815b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.615+0000 7f9c5d2b6700 1 --2- 172.21.15.89:0/2046421843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c58159ec0 0x7f9c5815bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.615+0000 7f9c5cab5700 1 --2- 172.21.15.89:0/2046421843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c580b53d0 0x7f9c58157280 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400d890 tx=0x7f9c54008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.615+0000 7f9c4cff9700 1 -- 172.21.15.89:0/2046421843 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5400b600 con 0x7f9c580b53d0 2023-10-10T20:43:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.615+0000 7f9c4cff9700 1 -- 172.21.15.89:0/2046421843 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5400b780 con 0x7f9c580b53d0 2023-10-10T20:43:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.615+0000 7f9c5ed19700 1 -- 172.21.15.89:0/2046421843 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5815de30 con 0x7f9c580b53d0 2023-10-10T20:43:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.615+0000 7f9c4cff9700 1 -- 172.21.15.89:0/2046421843 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5400bb40 con 0x7f9c580b53d0 2023-10-10T20:43:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.615+0000 7f9c5ed19700 1 -- 172.21.15.89:0/2046421843 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c581577c0 con 0x7f9c580b53d0 2023-10-10T20:43:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.616+0000 7f9c5ed19700 1 -- 172.21.15.89:0/2046421843 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c58063c90 con 0x7f9c580b53d0 2023-10-10T20:43:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.616+0000 7f9c4cff9700 1 -- 172.21.15.89:0/2046421843 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9c5401e460 con 0x7f9c580b53d0 2023-10-10T20:43:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.617+0000 7f9c4cff9700 1 --2- 172.21.15.89:0/2046421843 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9c44067320 0x7f9c440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.617+0000 7f9c4cff9700 1 -- 172.21.15.89:0/2046421843 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f9c54085110 con 0x7f9c580b53d0 2023-10-10T20:43:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.617+0000 7f9c4ffff700 1 --2- 172.21.15.89:0/2046421843 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9c44067320 0x7f9c440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.617+0000 7f9c4ffff700 1 --2- 172.21.15.89:0/2046421843 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9c44067320 0x7f9c440697d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9c480099b0 tx=0x7f9c48017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.621+0000 7f9c4cff9700 1 -- 172.21.15.89:0/2046421843 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c54051d50 con 0x7f9c580b53d0 2023-10-10T20:43:43.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:43.829+0000 7f9c5ed19700 1 -- 172.21.15.89:0/2046421843 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000812094", "yes_i_really_mean_it": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f9c580718b0 con 0x7f9c44067320 2023-10-10T20:43:44.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.472+0000 7f9c4cff9700 1 -- 172.21.15.89:0/2046421843 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+56 (secure 0 0 0) 0x7f9c580718b0 con 0x7f9c44067320 2023-10-10T20:43:44.478 INFO:teuthology.orchestra.run.smithi089.stdout:FS volume 'cephfs' renamed to 'volume_0000000000812094' 2023-10-10T20:43:44.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.475+0000 7f9c5ed19700 1 -- 172.21.15.89:0/2046421843 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9c44067320 msgr2=0x7f9c440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.475+0000 7f9c5ed19700 1 --2- 172.21.15.89:0/2046421843 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9c44067320 0x7f9c440697d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9c480099b0 tx=0x7f9c48017000 comp rx=0 tx=0).stop 2023-10-10T20:43:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.475+0000 7f9c5ed19700 1 -- 172.21.15.89:0/2046421843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c580b53d0 msgr2=0x7f9c58157280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.475+0000 7f9c5ed19700 1 --2- 172.21.15.89:0/2046421843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c580b53d0 0x7f9c58157280 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400d890 tx=0x7f9c54008040 comp rx=0 tx=0).stop 2023-10-10T20:43:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.475+0000 7f9c5ed19700 1 -- 172.21.15.89:0/2046421843 shutdown_connections 2023-10-10T20:43:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.475+0000 7f9c5ed19700 1 --2- 172.21.15.89:0/2046421843 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9c44067320 0x7f9c440697d0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.475+0000 7f9c5ed19700 1 --2- 172.21.15.89:0/2046421843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c58159ec0 0x7f9c5815bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.475+0000 7f9c5ed19700 1 --2- 172.21.15.89:0/2046421843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c580b53d0 0x7f9c58157280 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.475+0000 7f9c5ed19700 1 --2- 172.21.15.89:0/2046421843 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9c580cce40 0x7f9c5815b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.475+0000 7f9c5ed19700 1 -- 172.21.15.89:0/2046421843 >> 172.21.15.89:0/2046421843 conn(0x7f9c580999f0 msgr2=0x7f9c580c9910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:44.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.475+0000 7f9c5ed19700 1 -- 172.21.15.89:0/2046421843 shutdown_connections 2023-10-10T20:43:44.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.475+0000 7f9c5ed19700 1 -- 172.21.15.89:0/2046421843 wait complete. 2023-10-10T20:43:44.490 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-10T20:43:44.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.605+0000 7f8518b98700 1 Processor -- start 2023-10-10T20:43:44.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.605+0000 7f8518b98700 1 -- start start 2023-10-10T20:43:44.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.606+0000 7f8518b98700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f85141597f0 0x7f85140b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:44.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.606+0000 7f8518b98700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85140b11d0 con 0x7f851415d230 2023-10-10T20:43:44.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.606+0000 7f8518b98700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85140b14b0 con 0x7f8514159420 2023-10-10T20:43:44.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.606+0000 7f8518b98700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85140b1790 con 0x7f85141597f0 2023-10-10T20:43:44.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.606+0000 7f8513fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f851415d230 0x7f851415d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39818/0 (socket says 172.21.15.89:39818) 2023-10-10T20:43:44.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.606+0000 7f8513fff700 1 -- 172.21.15.89:0/3689059803 learned_addr learned my addr 172.21.15.89:0/3689059803 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:44.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f8512ffd700 1 --2- 172.21.15.89:0/3689059803 >> v2:172.21.15.111:3300/0 conn(0x7f85141597f0 0x7f85140b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:44.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f85127fc700 1 -- 172.21.15.89:0/3689059803 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 222496502 0 0) 0x7f85140b11d0 con 0x7f851415d230 2023-10-10T20:43:44.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f85127fc700 1 -- 172.21.15.89:0/3689059803 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8500003570 con 0x7f851415d230 2023-10-10T20:43:44.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f85127fc700 1 -- 172.21.15.89:0/3689059803 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 23986371 0 0) 0x7f8500003570 con 0x7f851415d230 2023-10-10T20:43:44.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f85127fc700 1 -- 172.21.15.89:0/3689059803 >> v2:172.21.15.111:3300/0 conn(0x7f85141597f0 msgr2=0x7f85140b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:44.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f85127fc700 1 --2- 172.21.15.89:0/3689059803 >> v2:172.21.15.111:3300/0 conn(0x7f85141597f0 0x7f85140b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f85127fc700 1 -- 172.21.15.89:0/3689059803 >> v1:172.21.15.111:6789/0 conn(0x7f8514159420 legacy=0x7f85140b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:44.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f85127fc700 1 -- 172.21.15.89:0/3689059803 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85140b1a70 con 0x7f851415d230 2023-10-10T20:43:44.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f85127fc700 1 -- 172.21.15.89:0/3689059803 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f850c001470 con 0x7f851415d230 2023-10-10T20:43:44.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f85127fc700 1 -- 172.21.15.89:0/3689059803 >> v1:172.21.15.89:6789/0 conn(0x7f851415d230 legacy=0x7f851415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:44.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f85127fc700 1 --2- 172.21.15.89:0/3689059803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8500003a60 0x7f8500003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:44.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f85127fc700 1 --2- 172.21.15.89:0/3689059803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8500004810 0x7f8500004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:44.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f85127fc700 1 --2- 172.21.15.89:0/3689059803 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85000051c0 0x7f850000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:44.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f85127fc700 1 -- 172.21.15.89:0/3689059803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85140b1790 con 0x7f8500004810 2023-10-10T20:43:44.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f85127fc700 1 -- 172.21.15.89:0/3689059803 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850000c0d0 con 0x7f8500003a60 2023-10-10T20:43:44.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f85127fc700 1 -- 172.21.15.89:0/3689059803 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850000c290 con 0x7f85000051c0 2023-10-10T20:43:44.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f8512ffd700 1 --2- 172.21.15.89:0/3689059803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8500004810 0x7f8500004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:44.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f85137fe700 1 --2- 172.21.15.89:0/3689059803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8500003a60 0x7f8500003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:44.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f8513fff700 1 --2- 172.21.15.89:0/3689059803 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85000051c0 0x7f850000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:44.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f8512ffd700 1 -- 172.21.15.89:0/3689059803 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85000051c0 msgr2=0x7f850000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:44.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f8512ffd700 1 --2- 172.21.15.89:0/3689059803 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85000051c0 0x7f850000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f8512ffd700 1 -- 172.21.15.89:0/3689059803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8500003a60 msgr2=0x7f8500003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:44.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f8512ffd700 1 --2- 172.21.15.89:0/3689059803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8500003a60 0x7f8500003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f8512ffd700 1 -- 172.21.15.89:0/3689059803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8500003570 con 0x7f8500004810 2023-10-10T20:43:44.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.607+0000 7f8513fff700 1 -- 172.21.15.89:0/3689059803 reap_dead start 2023-10-10T20:43:44.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.608+0000 7f8512ffd700 1 --2- 172.21.15.89:0/3689059803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8500004810 0x7f8500004c80 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f8514159c70 tx=0x7f84fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ba77c18b2ef645 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:44.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.608+0000 7f85127fc700 1 -- 172.21.15.89:0/3689059803 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc00e710 con 0x7f8500004810 2023-10-10T20:43:44.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.608+0000 7f85127fc700 1 -- 172.21.15.89:0/3689059803 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84fc00e870 con 0x7f8500004810 2023-10-10T20:43:44.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.608+0000 7f85127fc700 1 -- 172.21.15.89:0/3689059803 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc00ec30 con 0x7f8500004810 2023-10-10T20:43:44.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.608+0000 7f8518b98700 1 -- 172.21.15.89:0/3689059803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8500004810 msgr2=0x7f8500004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:44.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.608+0000 7f8518b98700 1 --2- 172.21.15.89:0/3689059803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8500004810 0x7f8500004c80 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f8514159c70 tx=0x7f84fc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.608+0000 7f8518b98700 1 -- 172.21.15.89:0/3689059803 shutdown_connections 2023-10-10T20:43:44.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.608+0000 7f8518b98700 1 --2- 172.21.15.89:0/3689059803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8500004810 0x7f8500004c80 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.608+0000 7f8518b98700 1 -- 172.21.15.89:0/3689059803 >> 172.21.15.89:0/3689059803 conn(0x7f85140d0660 msgr2=0x7f85140d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:44.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.608+0000 7f8518b98700 1 -- 172.21.15.89:0/3689059803 shutdown_connections 2023-10-10T20:43:44.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.608+0000 7f8518b98700 1 -- 172.21.15.89:0/3689059803 wait complete. 2023-10-10T20:43:44.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.609+0000 7f8518b98700 1 Processor -- start 2023-10-10T20:43:44.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.609+0000 7f8518b98700 1 -- start start 2023-10-10T20:43:44.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.610+0000 7f8518b98700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8500004810 0x7f8514146280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:44.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.610+0000 7f8518b98700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85141691b0 0x7f85140c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:44.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.610+0000 7f8518b98700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85140b57c0 0x7f85140b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:44.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.610+0000 7f8518b98700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85140b6180 con 0x7f85141691b0 2023-10-10T20:43:44.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.610+0000 7f8518b98700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85140c7960 con 0x7f8500004810 2023-10-10T20:43:44.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.610+0000 7f8518b98700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85140c7120 con 0x7f85140b57c0 2023-10-10T20:43:44.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.610+0000 7f8512ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85141691b0 0x7f85140c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:44.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.610+0000 7f8512ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85141691b0 0x7f85140c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44904/0 (socket says 172.21.15.89:44904) 2023-10-10T20:43:44.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.610+0000 7f8512ffd700 1 -- 172.21.15.89:0/2190732931 learned_addr learned my addr 172.21.15.89:0/2190732931 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:44.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.611+0000 7f8512ffd700 1 -- 172.21.15.89:0/2190732931 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85140b57c0 msgr2=0x7f85140b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:44.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.611+0000 7f8513fff700 1 --2- 172.21.15.89:0/2190732931 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85140b57c0 0x7f85140b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:44.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.611+0000 7f8512ffd700 1 --2- 172.21.15.89:0/2190732931 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85140b57c0 0x7f85140b5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.611+0000 7f85137fe700 1 --2- 172.21.15.89:0/2190732931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8500004810 0x7f8514146280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:44.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.611+0000 7f8512ffd700 1 -- 172.21.15.89:0/2190732931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8500004810 msgr2=0x7f8514146280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:44.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.611+0000 7f8512ffd700 1 --2- 172.21.15.89:0/2190732931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8500004810 0x7f8514146280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.611+0000 7f8512ffd700 1 -- 172.21.15.89:0/2190732931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85141696b0 con 0x7f85141691b0 2023-10-10T20:43:44.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.611+0000 7f8512ffd700 1 --2- 172.21.15.89:0/2190732931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85141691b0 0x7f85140c7420 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f850000c0a0 tx=0x7f84fc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:44.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.611+0000 7f850b7fe700 1 -- 172.21.15.89:0/2190732931 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc022070 con 0x7f85141691b0 2023-10-10T20:43:44.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.611+0000 7f850b7fe700 1 -- 172.21.15.89:0/2190732931 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84fc00e9e0 con 0x7f85141691b0 2023-10-10T20:43:44.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.611+0000 7f8518b98700 1 -- 172.21.15.89:0/2190732931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8514169990 con 0x7f85141691b0 2023-10-10T20:43:44.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.611+0000 7f850b7fe700 1 -- 172.21.15.89:0/2190732931 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc01c450 con 0x7f85141691b0 2023-10-10T20:43:44.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.611+0000 7f8518b98700 1 -- 172.21.15.89:0/2190732931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8514169f50 con 0x7f85141691b0 2023-10-10T20:43:44.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.612+0000 7f8518b98700 1 -- 172.21.15.89:0/2190732931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851404c3c0 con 0x7f85141691b0 2023-10-10T20:43:44.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.612+0000 7f850b7fe700 1 -- 172.21.15.89:0/2190732931 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f84fc01c840 con 0x7f85141691b0 2023-10-10T20:43:44.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.613+0000 7f850b7fe700 1 --2- 172.21.15.89:0/2190732931 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84f4067320 0x7f84f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:44.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.613+0000 7f85137fe700 1 --2- 172.21.15.89:0/2190732931 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84f4067320 0x7f84f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:44.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.613+0000 7f850b7fe700 1 -- 172.21.15.89:0/2190732931 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f84fc086480 con 0x7f85141691b0 2023-10-10T20:43:44.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.613+0000 7f85137fe700 1 --2- 172.21.15.89:0/2190732931 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84f4067320 0x7f84f40697d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f85140b26b0 tx=0x7f8504001640 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:44.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.617+0000 7f850b7fe700 1 -- 172.21.15.89:0/2190732931 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84fc053170 con 0x7f85141691b0 2023-10-10T20:43:44.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.828+0000 7f8518b98700 1 -- 172.21.15.89:0/2190732931 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f85140b1d50 con 0x7f84f4067320 2023-10-10T20:43:44.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.830+0000 7f850b7fe700 1 -- 172.21.15.89:0/2190732931 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+58 (secure 0 0 0) 0x7f85140b1d50 con 0x7f84f4067320 2023-10-10T20:43:44.831 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:43:44.832 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:44.832 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "volume_0000000000812094" 2023-10-10T20:43:44.832 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:44.832 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:43:44.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.832+0000 7f8518b98700 1 -- 172.21.15.89:0/2190732931 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84f4067320 msgr2=0x7f84f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:44.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.832+0000 7f8518b98700 1 --2- 172.21.15.89:0/2190732931 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84f4067320 0x7f84f40697d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f85140b26b0 tx=0x7f8504001640 comp rx=0 tx=0).stop 2023-10-10T20:43:44.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.832+0000 7f8518b98700 1 -- 172.21.15.89:0/2190732931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85141691b0 msgr2=0x7f85140c7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:44.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.832+0000 7f8518b98700 1 --2- 172.21.15.89:0/2190732931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85141691b0 0x7f85140c7420 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f850000c0a0 tx=0x7f84fc01a000 comp rx=0 tx=0).stop 2023-10-10T20:43:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.832+0000 7f8518b98700 1 -- 172.21.15.89:0/2190732931 shutdown_connections 2023-10-10T20:43:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.832+0000 7f8518b98700 1 --2- 172.21.15.89:0/2190732931 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f85140b57c0 0x7f85140b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.832+0000 7f8518b98700 1 --2- 172.21.15.89:0/2190732931 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f84f4067320 0x7f84f40697d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.832+0000 7f8518b98700 1 --2- 172.21.15.89:0/2190732931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8500004810 0x7f8514146280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.832+0000 7f8518b98700 1 --2- 172.21.15.89:0/2190732931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85141691b0 0x7f85140c7420 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.832+0000 7f8518b98700 1 -- 172.21.15.89:0/2190732931 >> 172.21.15.89:0/2190732931 conn(0x7f85140d0660 msgr2=0x7f85140cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.832+0000 7f8518b98700 1 -- 172.21.15.89:0/2190732931 shutdown_connections 2023-10-10T20:43:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.832+0000 7f8518b98700 1 -- 172.21.15.89:0/2190732931 wait complete. 2023-10-10T20:43:44.846 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:43:44.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.959+0000 7fb951595700 1 Processor -- start 2023-10-10T20:43:44.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.959+0000 7fb951595700 1 -- start start 2023-10-10T20:43:44.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.960+0000 7fb951595700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb94c1597f0 0x7fb94c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:44.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.960+0000 7fb951595700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb94c15d230 0x7fb94c15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:44.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.960+0000 7fb951595700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94c15db70 con 0x7fb94c159420 2023-10-10T20:43:44.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.960+0000 7fb951595700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb94c0b11d0 con 0x7fb94c1597f0 2023-10-10T20:43:44.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.960+0000 7fb951595700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb94c0b14b0 con 0x7fb94c15d230 2023-10-10T20:43:44.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.960+0000 7fb94b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb94c1597f0 0x7fb94c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:44.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.960+0000 7fb94b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb94c1597f0 0x7fb94c0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44916/0 (socket says 172.21.15.89:44916) 2023-10-10T20:43:44.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.960+0000 7fb94b7fe700 1 -- 172.21.15.89:0/3015571412 learned_addr learned my addr 172.21.15.89:0/3015571412 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:44.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.960+0000 7fb950d94700 1 --2- 172.21.15.89:0/3015571412 >> v2:172.21.15.111:3300/0 conn(0x7fb94c15d230 0x7fb94c15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:44.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.960+0000 7fb94b7fe700 1 -- 172.21.15.89:0/3015571412 >> v2:172.21.15.111:3300/0 conn(0x7fb94c15d230 msgr2=0x7fb94c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:44.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.960+0000 7fb94b7fe700 1 --2- 172.21.15.89:0/3015571412 >> v2:172.21.15.111:3300/0 conn(0x7fb94c15d230 0x7fb94c15d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.960+0000 7fb94b7fe700 1 -- 172.21.15.89:0/3015571412 >> v1:172.21.15.111:6789/0 conn(0x7fb94c159420 legacy=0x7fb94c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:44.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.960+0000 7fb94b7fe700 1 -- 172.21.15.89:0/3015571412 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c0b1790 con 0x7fb94c1597f0 2023-10-10T20:43:44.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.960+0000 7fb950d94700 1 --2- 172.21.15.89:0/3015571412 >> v2:172.21.15.111:3300/0 conn(0x7fb94c15d230 0x7fb94c15d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:44.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.961+0000 7fb94b7fe700 1 --2- 172.21.15.89:0/3015571412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb94c1597f0 0x7fb94c0b2270 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fb94c15d170 tx=0x7fb940009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886500ce1c9dfcff server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:44.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.961+0000 7fb94affd700 1 -- 172.21.15.89:0/3015571412 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94000b860 con 0x7fb94c1597f0 2023-10-10T20:43:44.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.961+0000 7fb94affd700 1 -- 172.21.15.89:0/3015571412 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94000b9c0 con 0x7fb94c1597f0 2023-10-10T20:43:44.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.961+0000 7fb94affd700 1 -- 172.21.15.89:0/3015571412 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94000bd60 con 0x7fb94c1597f0 2023-10-10T20:43:44.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.961+0000 7fb951595700 1 -- 172.21.15.89:0/3015571412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb94c1597f0 msgr2=0x7fb94c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:44.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.961+0000 7fb951595700 1 --2- 172.21.15.89:0/3015571412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb94c1597f0 0x7fb94c0b2270 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fb94c15d170 tx=0x7fb940009250 comp rx=0 tx=0).stop 2023-10-10T20:43:44.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.961+0000 7fb951595700 1 -- 172.21.15.89:0/3015571412 shutdown_connections 2023-10-10T20:43:44.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.961+0000 7fb951595700 1 --2- 172.21.15.89:0/3015571412 >> v2:172.21.15.111:3300/0 conn(0x7fb94c15d230 0x7fb94c15d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.961+0000 7fb951595700 1 --2- 172.21.15.89:0/3015571412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb94c1597f0 0x7fb94c0b2270 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.961+0000 7fb951595700 1 -- 172.21.15.89:0/3015571412 >> 172.21.15.89:0/3015571412 conn(0x7fb94c0d0660 msgr2=0x7fb94c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:44.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.961+0000 7fb951595700 1 -- 172.21.15.89:0/3015571412 shutdown_connections 2023-10-10T20:43:44.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.961+0000 7fb951595700 1 -- 172.21.15.89:0/3015571412 wait complete. 2023-10-10T20:43:44.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.962+0000 7fb951595700 1 Processor -- start 2023-10-10T20:43:44.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.963+0000 7fb951595700 1 -- start start 2023-10-10T20:43:44.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.963+0000 7fb951595700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb94c159420 0x7fb94c16a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:44.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.963+0000 7fb951595700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb94c1597f0 0x7fb94c1552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:44.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.963+0000 7fb951595700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb94c15d230 0x7fb94c16d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:44.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.963+0000 7fb951595700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c16ae40 con 0x7fb94c1597f0 2023-10-10T20:43:44.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.963+0000 7fb951595700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c16a470 con 0x7fb94c15d230 2023-10-10T20:43:44.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.963+0000 7fb951595700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c0cbe60 con 0x7fb94c159420 2023-10-10T20:43:44.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.963+0000 7fb94b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb94c1597f0 0x7fb94c1552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:44.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.963+0000 7fb94b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb94c1597f0 0x7fb94c1552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44930/0 (socket says 172.21.15.89:44930) 2023-10-10T20:43:44.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.963+0000 7fb94b7fe700 1 -- 172.21.15.89:0/2741705559 learned_addr learned my addr 172.21.15.89:0/2741705559 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:44.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.964+0000 7fb950d94700 1 --2- 172.21.15.89:0/2741705559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb94c15d230 0x7fb94c16d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:44.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.964+0000 7fb94bfff700 1 --2- 172.21.15.89:0/2741705559 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb94c159420 0x7fb94c16a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:44.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.964+0000 7fb94b7fe700 1 -- 172.21.15.89:0/2741705559 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb94c159420 msgr2=0x7fb94c16a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:44.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.964+0000 7fb94b7fe700 1 --2- 172.21.15.89:0/2741705559 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb94c159420 0x7fb94c16a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.964+0000 7fb94b7fe700 1 -- 172.21.15.89:0/2741705559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb94c15d230 msgr2=0x7fb94c16d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:44.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.964+0000 7fb94b7fe700 1 --2- 172.21.15.89:0/2741705559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb94c15d230 0x7fb94c16d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:44.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.964+0000 7fb94b7fe700 1 -- 172.21.15.89:0/2741705559 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9400094f0 con 0x7fb94c1597f0 2023-10-10T20:43:44.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.964+0000 7fb950d94700 1 --2- 172.21.15.89:0/2741705559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb94c15d230 0x7fb94c16d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:44.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.964+0000 7fb94b7fe700 1 --2- 172.21.15.89:0/2741705559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb94c1597f0 0x7fb94c1552d0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fb940009290 tx=0x7fb940017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:44.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.964+0000 7fb933fff700 1 -- 172.21.15.89:0/2741705559 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb940011e60 con 0x7fb94c1597f0 2023-10-10T20:43:44.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.964+0000 7fb933fff700 1 -- 172.21.15.89:0/2741705559 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94001f070 con 0x7fb94c1597f0 2023-10-10T20:43:44.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.964+0000 7fb951595700 1 -- 172.21.15.89:0/2741705559 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94c16df00 con 0x7fb94c1597f0 2023-10-10T20:43:44.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.964+0000 7fb933fff700 1 -- 172.21.15.89:0/2741705559 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94001a870 con 0x7fb94c1597f0 2023-10-10T20:43:44.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.964+0000 7fb951595700 1 -- 172.21.15.89:0/2741705559 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94c16a100 con 0x7fb94c1597f0 2023-10-10T20:43:44.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.965+0000 7fb951595700 1 -- 172.21.15.89:0/2741705559 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94c04c3c0 con 0x7fb94c1597f0 2023-10-10T20:43:44.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.965+0000 7fb933fff700 1 -- 172.21.15.89:0/2741705559 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb94001abe0 con 0x7fb94c1597f0 2023-10-10T20:43:44.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.966+0000 7fb933fff700 1 --2- 172.21.15.89:0/2741705559 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb938067320 0x7fb9380697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:44.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.966+0000 7fb94bfff700 1 --2- 172.21.15.89:0/2741705559 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb938067320 0x7fb9380697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:44.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.966+0000 7fb933fff700 1 -- 172.21.15.89:0/2741705559 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fb940021070 con 0x7fb94c1597f0 2023-10-10T20:43:44.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.966+0000 7fb94bfff700 1 --2- 172.21.15.89:0/2741705559 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb938067320 0x7fb9380697d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb94c15c6f0 tx=0x7fb93c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:44.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:44.970+0000 7fb933fff700 1 -- 172.21.15.89:0/2741705559 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9400537f0 con 0x7fb94c1597f0 2023-10-10T20:43:45.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.194+0000 7fb951595700 1 -- 172.21.15.89:0/2741705559 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb94c16a5b0 con 0x7fb94c1597f0 2023-10-10T20:43:45.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.194+0000 7fb933fff700 1 -- 172.21.15.89:0/2741705559 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4627 (secure 0 0 0) 0x7fb940015020 con 0x7fb94c1597f0 2023-10-10T20:43:45.196 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:45.197 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":124,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/455398787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":455398787},{"type":"v1","addr":"172.21.15.89:6836","nonce":455398787}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/2660411678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2660411678},{"type":"v1","addr":"172.21.15.111:6834","nonce":2660411678}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6890,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/1650071647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":1650071647},{"type":"v1","addr":"172.21.15.111:6835","nonce":1650071647}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:43:29.795107+0000","modified":"2023-10-10T20:43:41.456555+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8760":{"gid":8760,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/2168500629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2168500629},{"type":"v1","addr":"172.21.15.89:6837","nonce":2168500629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"volume_0000000000812094","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-10T20:43:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.196+0000 7fb951595700 1 -- 172.21.15.89:0/2741705559 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb938067320 msgr2=0x7fb9380697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.196+0000 7fb951595700 1 --2- 172.21.15.89:0/2741705559 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb938067320 0x7fb9380697d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb94c15c6f0 tx=0x7fb93c009040 comp rx=0 tx=0).stop 2023-10-10T20:43:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.196+0000 7fb951595700 1 -- 172.21.15.89:0/2741705559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb94c1597f0 msgr2=0x7fb94c1552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:45.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.196+0000 7fb951595700 1 --2- 172.21.15.89:0/2741705559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb94c1597f0 0x7fb94c1552d0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fb940009290 tx=0x7fb940017000 comp rx=0 tx=0).stop 2023-10-10T20:43:45.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.196+0000 7fb951595700 1 -- 172.21.15.89:0/2741705559 shutdown_connections 2023-10-10T20:43:45.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.196+0000 7fb951595700 1 --2- 172.21.15.89:0/2741705559 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb938067320 0x7fb9380697d0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.196+0000 7fb951595700 1 --2- 172.21.15.89:0/2741705559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb94c15d230 0x7fb94c16d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.196+0000 7fb951595700 1 --2- 172.21.15.89:0/2741705559 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb94c159420 0x7fb94c16a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.196+0000 7fb951595700 1 --2- 172.21.15.89:0/2741705559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb94c1597f0 0x7fb94c1552d0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.196+0000 7fb951595700 1 -- 172.21.15.89:0/2741705559 >> 172.21.15.89:0/2741705559 conn(0x7fb94c0d0660 msgr2=0x7fb94c156120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:45.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.196+0000 7fb951595700 1 -- 172.21.15.89:0/2741705559 shutdown_connections 2023-10-10T20:43:45.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.196+0000 7fb951595700 1 -- 172.21.15.89:0/2741705559 wait complete. 2023-10-10T20:43:45.201 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 124 2023-10-10T20:43:45.210 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:43:45.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.322+0000 7f77499b1700 1 Processor -- start 2023-10-10T20:43:45.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.323+0000 7f77499b1700 1 -- start start 2023-10-10T20:43:45.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.323+0000 7f77499b1700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7744159c10 0x7f774415b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:45.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.323+0000 7f77499b1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f77440b5ea0 0x7f7744068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:45.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.323+0000 7f77499b1700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f774415c480 con 0x7f7744068bb0 2023-10-10T20:43:45.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.323+0000 7f77499b1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77440b2a40 con 0x7f77440b5ea0 2023-10-10T20:43:45.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.323+0000 7f77499b1700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f774415b5d0 con 0x7f7744159c10 2023-10-10T20:43:45.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.323+0000 7f773a7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f77440b5ea0 0x7f7744068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:45.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.324+0000 7f773a7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f77440b5ea0 0x7f7744068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44940/0 (socket says 172.21.15.89:44940) 2023-10-10T20:43:45.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.324+0000 7f773a7fc700 1 -- 172.21.15.89:0/4170844646 learned_addr learned my addr 172.21.15.89:0/4170844646 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:45.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.324+0000 7f773a7fc700 1 -- 172.21.15.89:0/4170844646 >> v2:172.21.15.111:3300/0 conn(0x7f7744159c10 msgr2=0x7f774415b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:45.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.324+0000 7f773a7fc700 1 --2- 172.21.15.89:0/4170844646 >> v2:172.21.15.111:3300/0 conn(0x7f7744159c10 0x7f774415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.324+0000 7f773a7fc700 1 -- 172.21.15.89:0/4170844646 >> v1:172.21.15.111:6789/0 conn(0x7f7744068bb0 legacy=0x7f774415c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:45.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.324+0000 7f773a7fc700 1 -- 172.21.15.89:0/4170844646 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774415b870 con 0x7f77440b5ea0 2023-10-10T20:43:45.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.324+0000 7f773a7fc700 1 --2- 172.21.15.89:0/4170844646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77440b5ea0 0x7f7744068670 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f7744099f10 tx=0x7f772c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f120440636268534 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:45.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.324+0000 7f77427fc700 1 -- 172.21.15.89:0/4170844646 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c00b820 con 0x7f77440b5ea0 2023-10-10T20:43:45.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.324+0000 7f77427fc700 1 -- 172.21.15.89:0/4170844646 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c00b980 con 0x7f77440b5ea0 2023-10-10T20:43:45.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.324+0000 7f77427fc700 1 -- 172.21.15.89:0/4170844646 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c00bd20 con 0x7f77440b5ea0 2023-10-10T20:43:45.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.324+0000 7f77499b1700 1 -- 172.21.15.89:0/4170844646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77440b5ea0 msgr2=0x7f7744068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:45.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.325+0000 7f77499b1700 1 --2- 172.21.15.89:0/4170844646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77440b5ea0 0x7f7744068670 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f7744099f10 tx=0x7f772c009250 comp rx=0 tx=0).stop 2023-10-10T20:43:45.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.325+0000 7f77499b1700 1 -- 172.21.15.89:0/4170844646 shutdown_connections 2023-10-10T20:43:45.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.325+0000 7f77499b1700 1 --2- 172.21.15.89:0/4170844646 >> v2:172.21.15.111:3300/0 conn(0x7f7744159c10 0x7f774415b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.325+0000 7f77499b1700 1 --2- 172.21.15.89:0/4170844646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77440b5ea0 0x7f7744068670 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.325+0000 7f77499b1700 1 -- 172.21.15.89:0/4170844646 >> 172.21.15.89:0/4170844646 conn(0x7f7744099a20 msgr2=0x7f77440af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:45.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.325+0000 7f77499b1700 1 -- 172.21.15.89:0/4170844646 shutdown_connections 2023-10-10T20:43:45.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.325+0000 7f77499b1700 1 -- 172.21.15.89:0/4170844646 wait complete. 2023-10-10T20:43:45.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.326+0000 7f77499b1700 1 Processor -- start 2023-10-10T20:43:45.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.326+0000 7f77499b1700 1 -- start start 2023-10-10T20:43:45.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f77499b1700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7744068bb0 0x7f7744144400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:45.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f77499b1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f77440b5ea0 0x7f774409db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:45.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f77499b1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7744159c10 0x7f774409e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:45.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f77499b1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7744144e40 con 0x7f7744159c10 2023-10-10T20:43:45.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f77499b1700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7744145120 con 0x7f7744068bb0 2023-10-10T20:43:45.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f77499b1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7744145400 con 0x7f77440b5ea0 2023-10-10T20:43:45.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f77437fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7744159c10 0x7f774409e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:45.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f77437fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7744159c10 0x7f774409e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44950/0 (socket says 172.21.15.89:44950) 2023-10-10T20:43:45.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f77437fe700 1 -- 172.21.15.89:0/3315366904 learned_addr learned my addr 172.21.15.89:0/3315366904 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:45.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f773a7fc700 1 --2- 172.21.15.89:0/3315366904 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f77440b5ea0 0x7f774409db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:45.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f7742ffd700 1 --2- 172.21.15.89:0/3315366904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7744068bb0 0x7f7744144400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:45.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f77437fe700 1 -- 172.21.15.89:0/3315366904 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f77440b5ea0 msgr2=0x7f774409db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:45.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f77437fe700 1 --2- 172.21.15.89:0/3315366904 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f77440b5ea0 0x7f774409db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f77437fe700 1 -- 172.21.15.89:0/3315366904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7744068bb0 msgr2=0x7f7744144400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:45.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f77437fe700 1 --2- 172.21.15.89:0/3315366904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7744068bb0 0x7f7744144400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f77437fe700 1 -- 172.21.15.89:0/3315366904 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c0094f0 con 0x7f7744159c10 2023-10-10T20:43:45.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f7742ffd700 1 --2- 172.21.15.89:0/3315366904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7744068bb0 0x7f7744144400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:45.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f77437fe700 1 --2- 172.21.15.89:0/3315366904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7744159c10 0x7f774409e0d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f773c00bbb0 tx=0x7f773c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:45.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f773b7fe700 1 -- 172.21.15.89:0/3315366904 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c0092e0 con 0x7f7744159c10 2023-10-10T20:43:45.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f773b7fe700 1 -- 172.21.15.89:0/3315366904 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773c00c7a0 con 0x7f7744159c10 2023-10-10T20:43:45.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f77499b1700 1 -- 172.21.15.89:0/3315366904 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77441456e0 con 0x7f7744159c10 2023-10-10T20:43:45.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f773b7fe700 1 -- 172.21.15.89:0/3315366904 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c00cb40 con 0x7f7744159c10 2023-10-10T20:43:45.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.327+0000 7f77499b1700 1 -- 172.21.15.89:0/3315366904 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7744149450 con 0x7f7744159c10 2023-10-10T20:43:45.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.328+0000 7f77499b1700 1 -- 172.21.15.89:0/3315366904 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7744144f80 con 0x7f7744159c10 2023-10-10T20:43:45.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.329+0000 7f773b7fe700 1 -- 172.21.15.89:0/3315366904 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f773c012070 con 0x7f7744159c10 2023-10-10T20:43:45.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.329+0000 7f773b7fe700 1 --2- 172.21.15.89:0/3315366904 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7724067320 0x7f77240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:45.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.329+0000 7f773b7fe700 1 -- 172.21.15.89:0/3315366904 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f773c0849b0 con 0x7f7744159c10 2023-10-10T20:43:45.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.329+0000 7f7742ffd700 1 --2- 172.21.15.89:0/3315366904 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7724067320 0x7f77240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:45.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.329+0000 7f7742ffd700 1 --2- 172.21.15.89:0/3315366904 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7724067320 0x7f77240697d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f77340079b0 tx=0x7f7734008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:45.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.333+0000 7f773b7fe700 1 -- 172.21.15.89:0/3315366904 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7744144f80 con 0x7f7744159c10 2023-10-10T20:43:45.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.521+0000 7f77499b1700 1 -- 172.21.15.89:0/3315366904 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7744144f80 con 0x7f7744159c10 2023-10-10T20:43:45.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.522+0000 7f773b7fe700 1 -- 172.21.15.89:0/3315366904 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+15442 (secure 0 0 0) 0x7f7744144f80 con 0x7f7744159c10 2023-10-10T20:43:45.524 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:45.524 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":186,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:43:43.464066+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs.volume_0000000000812094.meta","create_time":"2023-10-10T20:43:28.133902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000812094"}}},{"pool":32,"pool_name":"cephfs.volume_0000000000812094.data","create_time":"2023-10-10T20:43:28.749600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000812094"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:43:45.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.524+0000 7f77499b1700 1 -- 172.21.15.89:0/3315366904 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7724067320 msgr2=0x7f77240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:45.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.524+0000 7f77499b1700 1 --2- 172.21.15.89:0/3315366904 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7724067320 0x7f77240697d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f77340079b0 tx=0x7f7734008040 comp rx=0 tx=0).stop 2023-10-10T20:43:45.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.524+0000 7f77499b1700 1 -- 172.21.15.89:0/3315366904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7744159c10 msgr2=0x7f774409e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:45.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.524+0000 7f77499b1700 1 --2- 172.21.15.89:0/3315366904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7744159c10 0x7f774409e0d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f773c00bbb0 tx=0x7f773c009840 comp rx=0 tx=0).stop 2023-10-10T20:43:45.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.524+0000 7f77499b1700 1 -- 172.21.15.89:0/3315366904 shutdown_connections 2023-10-10T20:43:45.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.524+0000 7f77499b1700 1 --2- 172.21.15.89:0/3315366904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7744159c10 0x7f774409e0d0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.524+0000 7f77499b1700 1 --2- 172.21.15.89:0/3315366904 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7724067320 0x7f77240697d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.524+0000 7f77499b1700 1 --2- 172.21.15.89:0/3315366904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7744068bb0 0x7f7744144400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.524+0000 7f77499b1700 1 --2- 172.21.15.89:0/3315366904 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f77440b5ea0 0x7f774409db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.524+0000 7f77499b1700 1 -- 172.21.15.89:0/3315366904 >> 172.21.15.89:0/3315366904 conn(0x7f7744099a20 msgr2=0x7f774409b610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:45.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.524+0000 7f77499b1700 1 -- 172.21.15.89:0/3315366904 shutdown_connections 2023-10-10T20:43:45.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.524+0000 7f77499b1700 1 -- 172.21.15.89:0/3315366904 wait complete. 2023-10-10T20:43:45.539 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:43:45.539 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:43:45.584 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:43:45.629 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:43:45.630 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:43:45.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.742+0000 7f94650f7700 1 Processor -- start 2023-10-10T20:43:45.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.742+0000 7f94650f7700 1 -- start start 2023-10-10T20:43:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.743+0000 7f94650f7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9460159c60 0x7f946015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.743+0000 7f94650f7700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f946015db10 0x7f9460069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.743+0000 7f94650f7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946015a160 con 0x7f9460159890 2023-10-10T20:43:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.743+0000 7f94650f7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946015df50 con 0x7f9460159c60 2023-10-10T20:43:45.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.743+0000 7f94650f7700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94600d13a0 con 0x7f946015db10 2023-10-10T20:43:45.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.743+0000 7f945e59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9460159c60 0x7f946015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:45.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.743+0000 7f945e59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9460159c60 0x7f946015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44964/0 (socket says 172.21.15.89:44964) 2023-10-10T20:43:45.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.743+0000 7f945ed9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9460159890 0x7f946015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39826/0 (socket says 172.21.15.89:39826) 2023-10-10T20:43:45.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.743+0000 7f945e59c700 1 -- 172.21.15.89:0/3338430858 learned_addr learned my addr 172.21.15.89:0/3338430858 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:45.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.743+0000 7f945f59e700 1 --2- 172.21.15.89:0/3338430858 >> v2:172.21.15.111:3300/0 conn(0x7f946015db10 0x7f9460069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:45.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.743+0000 7f945e59c700 1 -- 172.21.15.89:0/3338430858 >> v2:172.21.15.111:3300/0 conn(0x7f946015db10 msgr2=0x7f9460069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:45.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.743+0000 7f945e59c700 1 --2- 172.21.15.89:0/3338430858 >> v2:172.21.15.111:3300/0 conn(0x7f946015db10 0x7f9460069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.743+0000 7f945e59c700 1 -- 172.21.15.89:0/3338430858 >> v1:172.21.15.89:6789/0 conn(0x7f9460159890 legacy=0x7f946015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:45.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.743+0000 7f945e59c700 1 -- 172.21.15.89:0/3338430858 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946015e140 con 0x7f9460159c60 2023-10-10T20:43:45.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.743+0000 7f945dd9b700 1 -- 172.21.15.89:0/3338430858 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175622481 0 0) 0x7f946015a160 con 0x7f9460159890 2023-10-10T20:43:45.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.743+0000 7f945e59c700 1 --2- 172.21.15.89:0/3338430858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9460159c60 0x7f946015d5d0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f946006a890 tx=0x7f9454009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747e910e70d2196e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:45.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.744+0000 7f945dd9b700 1 -- 172.21.15.89:0/3338430858 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945400b860 con 0x7f9460159c60 2023-10-10T20:43:45.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.744+0000 7f945dd9b700 1 -- 172.21.15.89:0/3338430858 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945400b9c0 con 0x7f9460159c60 2023-10-10T20:43:45.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.744+0000 7f945dd9b700 1 -- 172.21.15.89:0/3338430858 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945400bd60 con 0x7f9460159c60 2023-10-10T20:43:45.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.744+0000 7f94650f7700 1 -- 172.21.15.89:0/3338430858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9460159c60 msgr2=0x7f946015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:45.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.744+0000 7f94650f7700 1 --2- 172.21.15.89:0/3338430858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9460159c60 0x7f946015d5d0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f946006a890 tx=0x7f9454009250 comp rx=0 tx=0).stop 2023-10-10T20:43:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.744+0000 7f94650f7700 1 -- 172.21.15.89:0/3338430858 shutdown_connections 2023-10-10T20:43:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.744+0000 7f94650f7700 1 --2- 172.21.15.89:0/3338430858 >> v2:172.21.15.111:3300/0 conn(0x7f946015db10 0x7f9460069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.744+0000 7f94650f7700 1 --2- 172.21.15.89:0/3338430858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9460159c60 0x7f946015d5d0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.744+0000 7f94650f7700 1 -- 172.21.15.89:0/3338430858 >> 172.21.15.89:0/3338430858 conn(0x7f9460068e20 msgr2=0x7f94600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.744+0000 7f94650f7700 1 -- 172.21.15.89:0/3338430858 shutdown_connections 2023-10-10T20:43:45.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.744+0000 7f94650f7700 1 -- 172.21.15.89:0/3338430858 wait complete. 2023-10-10T20:43:45.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.745+0000 7f94650f7700 1 Processor -- start 2023-10-10T20:43:45.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.745+0000 7f94650f7700 1 -- start start 2023-10-10T20:43:45.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.746+0000 7f94650f7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9460159890 0x7f946014b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:45.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.746+0000 7f94650f7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9460159c60 0x7f94600cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:45.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.746+0000 7f94650f7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f946015db10 0x7f9460151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:45.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.746+0000 7f94650f7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94600b1fb0 con 0x7f946015db10 2023-10-10T20:43:45.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.746+0000 7f94650f7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94600c5ff0 con 0x7f9460159c60 2023-10-10T20:43:45.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.746+0000 7f94650f7700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94600c56c0 con 0x7f9460159890 2023-10-10T20:43:45.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.746+0000 7f945f59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f946015db10 0x7f9460151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:45.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.746+0000 7f945f59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f946015db10 0x7f9460151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44972/0 (socket says 172.21.15.89:44972) 2023-10-10T20:43:45.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.746+0000 7f945f59e700 1 -- 172.21.15.89:0/3759924737 learned_addr learned my addr 172.21.15.89:0/3759924737 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:45.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.746+0000 7f945e59c700 1 --2- 172.21.15.89:0/3759924737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9460159c60 0x7f94600cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:45.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.746+0000 7f945ed9d700 1 --2- 172.21.15.89:0/3759924737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9460159890 0x7f946014b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:45.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.746+0000 7f945f59e700 1 -- 172.21.15.89:0/3759924737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9460159890 msgr2=0x7f946014b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:45.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.746+0000 7f945f59e700 1 --2- 172.21.15.89:0/3759924737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9460159890 0x7f946014b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.746+0000 7f945f59e700 1 -- 172.21.15.89:0/3759924737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9460159c60 msgr2=0x7f94600cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:45.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.746+0000 7f945f59e700 1 --2- 172.21.15.89:0/3759924737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9460159c60 0x7f94600cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.746+0000 7f945f59e700 1 -- 172.21.15.89:0/3759924737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94540094f0 con 0x7f946015db10 2023-10-10T20:43:45.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.747+0000 7f945f59e700 1 --2- 172.21.15.89:0/3759924737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f946015db10 0x7f9460151d80 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f9458007f20 tx=0x7f9458009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:45.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.747+0000 7f9446ffd700 1 -- 172.21.15.89:0/3759924737 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945800eb90 con 0x7f946015db10 2023-10-10T20:43:45.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.747+0000 7f9446ffd700 1 -- 172.21.15.89:0/3759924737 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945800ed10 con 0x7f946015db10 2023-10-10T20:43:45.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.747+0000 7f9446ffd700 1 -- 172.21.15.89:0/3759924737 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9458014670 con 0x7f946015db10 2023-10-10T20:43:45.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.747+0000 7f94650f7700 1 -- 172.21.15.89:0/3759924737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94600c59a0 con 0x7f946015db10 2023-10-10T20:43:45.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.747+0000 7f94650f7700 1 -- 172.21.15.89:0/3759924737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946014bdd0 con 0x7f946015db10 2023-10-10T20:43:45.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.748+0000 7f94650f7700 1 -- 172.21.15.89:0/3759924737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94600c6160 con 0x7f946015db10 2023-10-10T20:43:45.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.748+0000 7f9446ffd700 1 -- 172.21.15.89:0/3759924737 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f94580149e0 con 0x7f946015db10 2023-10-10T20:43:45.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.748+0000 7f9446ffd700 1 --2- 172.21.15.89:0/3759924737 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f944c067320 0x7f944c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:45.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.749+0000 7f945ed9d700 1 --2- 172.21.15.89:0/3759924737 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f944c067320 0x7f944c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:45.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.749+0000 7f9446ffd700 1 -- 172.21.15.89:0/3759924737 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f94580852c0 con 0x7f946015db10 2023-10-10T20:43:45.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.749+0000 7f945ed9d700 1 --2- 172.21.15.89:0/3759924737 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f944c067320 0x7f944c0697d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9450007fd0 tx=0x7f9450007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:45.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.753+0000 7f9446ffd700 1 -- 172.21.15.89:0/3759924737 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9458051f00 con 0x7f946015db10 2023-10-10T20:43:45.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.978+0000 7f94650f7700 1 -- 172.21.15.89:0/3759924737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f946004c3c0 con 0x7f946015db10 2023-10-10T20:43:45.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.979+0000 7f9446ffd700 1 -- 172.21.15.89:0/3759924737 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4627 (secure 0 0 0) 0x7f9458055520 con 0x7f946015db10 2023-10-10T20:43:45.981 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:45.981 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":124,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/455398787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":455398787},{"type":"v1","addr":"172.21.15.89:6836","nonce":455398787}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/2660411678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2660411678},{"type":"v1","addr":"172.21.15.111:6834","nonce":2660411678}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6890,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/1650071647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":1650071647},{"type":"v1","addr":"172.21.15.111:6835","nonce":1650071647}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:43:29.795107+0000","modified":"2023-10-10T20:43:41.456555+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8760":{"gid":8760,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/2168500629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2168500629},{"type":"v1","addr":"172.21.15.89:6837","nonce":2168500629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"volume_0000000000812094","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-10T20:43:45.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.981+0000 7f94650f7700 1 -- 172.21.15.89:0/3759924737 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f944c067320 msgr2=0x7f944c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:45.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.981+0000 7f94650f7700 1 --2- 172.21.15.89:0/3759924737 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f944c067320 0x7f944c0697d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9450007fd0 tx=0x7f9450007d50 comp rx=0 tx=0).stop 2023-10-10T20:43:45.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.981+0000 7f94650f7700 1 -- 172.21.15.89:0/3759924737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f946015db10 msgr2=0x7f9460151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:45.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.981+0000 7f94650f7700 1 --2- 172.21.15.89:0/3759924737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f946015db10 0x7f9460151d80 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f9458007f20 tx=0x7f9458009860 comp rx=0 tx=0).stop 2023-10-10T20:43:45.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.981+0000 7f94650f7700 1 -- 172.21.15.89:0/3759924737 shutdown_connections 2023-10-10T20:43:45.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.981+0000 7f94650f7700 1 --2- 172.21.15.89:0/3759924737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f946015db10 0x7f9460151d80 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.981+0000 7f94650f7700 1 --2- 172.21.15.89:0/3759924737 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9460159890 0x7f946014b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.981+0000 7f94650f7700 1 --2- 172.21.15.89:0/3759924737 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f944c067320 0x7f944c0697d0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.981+0000 7f94650f7700 1 --2- 172.21.15.89:0/3759924737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9460159c60 0x7f94600cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:45.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.981+0000 7f94650f7700 1 -- 172.21.15.89:0/3759924737 >> 172.21.15.89:0/3759924737 conn(0x7f9460068e20 msgr2=0x7f94601573d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:45.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.981+0000 7f94650f7700 1 -- 172.21.15.89:0/3759924737 shutdown_connections 2023-10-10T20:43:45.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:45.981+0000 7f94650f7700 1 -- 172.21.15.89:0/3759924737 wait complete. 2023-10-10T20:43:45.986 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 124 2023-10-10T20:43:45.995 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:43:46.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.108+0000 7f7b99b09700 1 Processor -- start 2023-10-10T20:43:46.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.108+0000 7f7b99b09700 1 -- start start 2023-10-10T20:43:46.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.109+0000 7f7b99b09700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7b9415d230 0x7f7b9415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.109+0000 7f7b99b09700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b940b2980 con 0x7f7b94159420 2023-10-10T20:43:46.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.109+0000 7f7b99b09700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9415db70 con 0x7f7b941597f0 2023-10-10T20:43:46.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.109+0000 7f7b99b09700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b940b11d0 con 0x7f7b9415d230 2023-10-10T20:43:46.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.109+0000 7f7b99308700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7b9415d230 0x7f7b9415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:46.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.109+0000 7f7b98b07700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7b94159420 0x7f7b940b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39840/0 (socket says 172.21.15.89:39840) 2023-10-10T20:43:46.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.109+0000 7f7b98b07700 1 -- 172.21.15.89:0/327809293 learned_addr learned my addr 172.21.15.89:0/327809293 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:46.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.109+0000 7f7b99308700 1 -- 172.21.15.89:0/327809293 >> v1:172.21.15.111:6789/0 conn(0x7f7b941597f0 legacy=0x7f7b940b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:46.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.109+0000 7f7b99308700 1 -- 172.21.15.89:0/327809293 >> v1:172.21.15.89:6789/0 conn(0x7f7b94159420 legacy=0x7f7b940b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:46.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.109+0000 7f7b99308700 1 -- 172.21.15.89:0/327809293 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b940b14b0 con 0x7f7b9415d230 2023-10-10T20:43:46.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.109+0000 7f7b99308700 1 --2- 172.21.15.89:0/327809293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b9415d230 0x7f7b9415d630 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009e60 tx=0x7f7b8c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8dadef3ef5f1fee server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:46.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.110+0000 7f7b937fe700 1 -- 172.21.15.89:0/327809293 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00e070 con 0x7f7b9415d230 2023-10-10T20:43:46.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.110+0000 7f7b937fe700 1 -- 172.21.15.89:0/327809293 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8c00bbf0 con 0x7f7b9415d230 2023-10-10T20:43:46.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.110+0000 7f7b937fe700 1 -- 172.21.15.89:0/327809293 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c012430 con 0x7f7b9415d230 2023-10-10T20:43:46.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.110+0000 7f7b99b09700 1 -- 172.21.15.89:0/327809293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b9415d230 msgr2=0x7f7b9415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:46.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.110+0000 7f7b99b09700 1 --2- 172.21.15.89:0/327809293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b9415d230 0x7f7b9415d630 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009e60 tx=0x7f7b8c009780 comp rx=0 tx=0).stop 2023-10-10T20:43:46.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.110+0000 7f7b99b09700 1 -- 172.21.15.89:0/327809293 shutdown_connections 2023-10-10T20:43:46.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.110+0000 7f7b99b09700 1 --2- 172.21.15.89:0/327809293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b9415d230 0x7f7b9415d630 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.110+0000 7f7b99b09700 1 -- 172.21.15.89:0/327809293 >> 172.21.15.89:0/327809293 conn(0x7f7b940d0660 msgr2=0x7f7b940d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:46.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.110+0000 7f7b99b09700 1 -- 172.21.15.89:0/327809293 shutdown_connections 2023-10-10T20:43:46.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.110+0000 7f7b99b09700 1 -- 172.21.15.89:0/327809293 wait complete. 2023-10-10T20:43:46.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.111+0000 7f7b99b09700 1 Processor -- start 2023-10-10T20:43:46.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.111+0000 7f7b99b09700 1 -- start start 2023-10-10T20:43:46.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.112+0000 7f7b99b09700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b94159420 0x7f7b940c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.112+0000 7f7b99b09700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b941597f0 0x7f7b9415c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.112+0000 7f7b99b09700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7b9415d230 0x7f7b94155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.112+0000 7f7b99b09700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9415bad0 con 0x7f7b941597f0 2023-10-10T20:43:46.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.112+0000 7f7b99b09700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9415aec0 con 0x7f7b94159420 2023-10-10T20:43:46.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.112+0000 7f7b99b09700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9415b1a0 con 0x7f7b9415d230 2023-10-10T20:43:46.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.112+0000 7f7b93fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b941597f0 0x7f7b9415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:46.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.112+0000 7f7b93fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b941597f0 0x7f7b9415c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44996/0 (socket says 172.21.15.89:44996) 2023-10-10T20:43:46.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.112+0000 7f7b93fff700 1 -- 172.21.15.89:0/4127805286 learned_addr learned my addr 172.21.15.89:0/4127805286 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:46.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.112+0000 7f7b98b07700 1 --2- 172.21.15.89:0/4127805286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b94159420 0x7f7b940c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:46.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.112+0000 7f7b99308700 1 --2- 172.21.15.89:0/4127805286 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7b9415d230 0x7f7b94155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:46.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.113+0000 7f7b93fff700 1 -- 172.21.15.89:0/4127805286 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7b9415d230 msgr2=0x7f7b94155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:46.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.113+0000 7f7b93fff700 1 --2- 172.21.15.89:0/4127805286 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7b9415d230 0x7f7b94155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.113+0000 7f7b93fff700 1 -- 172.21.15.89:0/4127805286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b94159420 msgr2=0x7f7b940c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:46.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.113+0000 7f7b93fff700 1 --2- 172.21.15.89:0/4127805286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b94159420 0x7f7b940c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.113+0000 7f7b93fff700 1 -- 172.21.15.89:0/4127805286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8c009a20 con 0x7f7b941597f0 2023-10-10T20:43:46.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.113+0000 7f7b98b07700 1 --2- 172.21.15.89:0/4127805286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b94159420 0x7f7b940c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:46.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.113+0000 7f7b93fff700 1 --2- 172.21.15.89:0/4127805286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b941597f0 0x7f7b9415c950 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000c590 tx=0x7f7b80007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:46.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.113+0000 7f7b90ff9700 1 -- 172.21.15.89:0/4127805286 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80011590 con 0x7f7b941597f0 2023-10-10T20:43:46.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.113+0000 7f7b90ff9700 1 -- 172.21.15.89:0/4127805286 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b80011730 con 0x7f7b941597f0 2023-10-10T20:43:46.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.113+0000 7f7b99b09700 1 -- 172.21.15.89:0/4127805286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9415b480 con 0x7f7b941597f0 2023-10-10T20:43:46.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.113+0000 7f7b90ff9700 1 -- 172.21.15.89:0/4127805286 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80011af0 con 0x7f7b941597f0 2023-10-10T20:43:46.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.113+0000 7f7b99b09700 1 -- 172.21.15.89:0/4127805286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b940c7f10 con 0x7f7b941597f0 2023-10-10T20:43:46.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.114+0000 7f7b99b09700 1 -- 172.21.15.89:0/4127805286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b9404c3c0 con 0x7f7b941597f0 2023-10-10T20:43:46.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.114+0000 7f7b90ff9700 1 -- 172.21.15.89:0/4127805286 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7b8001c460 con 0x7f7b941597f0 2023-10-10T20:43:46.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.115+0000 7f7b90ff9700 1 --2- 172.21.15.89:0/4127805286 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7b78067320 0x7f7b780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.115+0000 7f7b98b07700 1 --2- 172.21.15.89:0/4127805286 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7b78067320 0x7f7b780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:46.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.115+0000 7f7b90ff9700 1 -- 172.21.15.89:0/4127805286 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f7b80084a70 con 0x7f7b941597f0 2023-10-10T20:43:46.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.115+0000 7f7b98b07700 1 --2- 172.21.15.89:0/4127805286 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7b78067320 0x7f7b780697d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7b88007fd0 tx=0x7f7b88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:46.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.119+0000 7f7b90ff9700 1 -- 172.21.15.89:0/4127805286 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b800516b0 con 0x7f7b941597f0 2023-10-10T20:43:46.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.301+0000 7f7b99b09700 1 -- 172.21.15.89:0/4127805286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b9415bc10 con 0x7f7b941597f0 2023-10-10T20:43:46.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.302+0000 7f7b90ff9700 1 -- 172.21.15.89:0/4127805286 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+15442 (secure 0 0 0) 0x7f7b80054cd0 con 0x7f7b941597f0 2023-10-10T20:43:46.305 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:46.305 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":186,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:43:43.464066+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs.volume_0000000000812094.meta","create_time":"2023-10-10T20:43:28.133902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000812094"}}},{"pool":32,"pool_name":"cephfs.volume_0000000000812094.data","create_time":"2023-10-10T20:43:28.749600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000812094"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:43:46.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.304+0000 7f7b99b09700 1 -- 172.21.15.89:0/4127805286 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7b78067320 msgr2=0x7f7b780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:46.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.304+0000 7f7b99b09700 1 --2- 172.21.15.89:0/4127805286 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7b78067320 0x7f7b780697d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7b88007fd0 tx=0x7f7b88009040 comp rx=0 tx=0).stop 2023-10-10T20:43:46.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.304+0000 7f7b99b09700 1 -- 172.21.15.89:0/4127805286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b941597f0 msgr2=0x7f7b9415c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:46.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.304+0000 7f7b99b09700 1 --2- 172.21.15.89:0/4127805286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b941597f0 0x7f7b9415c950 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000c590 tx=0x7f7b80007f30 comp rx=0 tx=0).stop 2023-10-10T20:43:46.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.304+0000 7f7b99b09700 1 -- 172.21.15.89:0/4127805286 shutdown_connections 2023-10-10T20:43:46.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.304+0000 7f7b99b09700 1 --2- 172.21.15.89:0/4127805286 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7b9415d230 0x7f7b94155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.304+0000 7f7b99b09700 1 --2- 172.21.15.89:0/4127805286 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7b78067320 0x7f7b780697d0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.304+0000 7f7b99b09700 1 --2- 172.21.15.89:0/4127805286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b94159420 0x7f7b940c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.304+0000 7f7b99b09700 1 --2- 172.21.15.89:0/4127805286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b941597f0 0x7f7b9415c950 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.304+0000 7f7b99b09700 1 -- 172.21.15.89:0/4127805286 >> 172.21.15.89:0/4127805286 conn(0x7f7b940d0660 msgr2=0x7f7b940b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:46.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.304+0000 7f7b99b09700 1 -- 172.21.15.89:0/4127805286 shutdown_connections 2023-10-10T20:43:46.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.304+0000 7f7b99b09700 1 -- 172.21.15.89:0/4127805286 wait complete. 2023-10-10T20:43:46.319 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 31, 'pool_name': 'cephfs.volume_0000000000812094.meta', 'create_time': '2023-10-10T20:43:28.133902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000812094'}}}, {'pool': 32, 'pool_name': 'cephfs.volume_0000000000812094.data', 'create_time': '2023-10-10T20:43:28.749600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000812094'}}}] 2023-10-10T20:43:46.320 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:43:46.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.440+0000 7f9dc07b3700 1 Processor -- start 2023-10-10T20:43:46.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.440+0000 7f9dc07b3700 1 -- start start 2023-10-10T20:43:46.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.441+0000 7f9dc07b3700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9db8159890 0x7f9db815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.441+0000 7f9dc07b3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9db8159c60 0x7f9db815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.441+0000 7f9dc07b3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db815a160 con 0x7f9db815db10 2023-10-10T20:43:46.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.441+0000 7f9dc07b3700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db80d13a0 con 0x7f9db8159890 2023-10-10T20:43:46.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.441+0000 7f9dc07b3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db815e140 con 0x7f9db8159c60 2023-10-10T20:43:46.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.441+0000 7f9dbed50700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9db815db10 0x7f9db8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39852/0 (socket says 172.21.15.89:39852) 2023-10-10T20:43:46.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.441+0000 7f9dbed50700 1 -- 172.21.15.89:0/4268787405 learned_addr learned my addr 172.21.15.89:0/4268787405 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:46.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.441+0000 7f9dbe54f700 1 --2- 172.21.15.89:0/4268787405 >> v2:172.21.15.111:3300/0 conn(0x7f9db8159890 0x7f9db815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:46.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.441+0000 7f9dbdd4e700 1 --2- 172.21.15.89:0/4268787405 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9db8159c60 0x7f9db815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:46.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.441+0000 7f9dbd54d700 1 -- 172.21.15.89:0/4268787405 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1419180788 0 0) 0x7f9db815a160 con 0x7f9db815db10 2023-10-10T20:43:46.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.441+0000 7f9dbd54d700 1 -- 172.21.15.89:0/4268787405 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9da4003570 con 0x7f9db815db10 2023-10-10T20:43:46.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.442+0000 7f9dbdd4e700 1 -- 172.21.15.89:0/4268787405 >> v2:172.21.15.111:3300/0 conn(0x7f9db8159890 msgr2=0x7f9db815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:46.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.442+0000 7f9dbdd4e700 1 --2- 172.21.15.89:0/4268787405 >> v2:172.21.15.111:3300/0 conn(0x7f9db8159890 0x7f9db815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.442+0000 7f9dbdd4e700 1 -- 172.21.15.89:0/4268787405 >> v1:172.21.15.89:6789/0 conn(0x7f9db815db10 legacy=0x7f9db8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:46.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.442+0000 7f9dbdd4e700 1 -- 172.21.15.89:0/4268787405 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db815a3d0 con 0x7f9db8159c60 2023-10-10T20:43:46.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.442+0000 7f9dbe54f700 1 --2- 172.21.15.89:0/4268787405 >> v2:172.21.15.111:3300/0 conn(0x7f9db8159890 0x7f9db815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:43:46.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.442+0000 7f9dbdd4e700 1 --2- 172.21.15.89:0/4268787405 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9db8159c60 0x7f9db815d5d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9da80098e0 tx=0x7f9da80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b15ee766d5db5be7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:46.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.442+0000 7f9dbd54d700 1 -- 172.21.15.89:0/4268787405 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da8011630 con 0x7f9db8159c60 2023-10-10T20:43:46.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.442+0000 7f9dbd54d700 1 -- 172.21.15.89:0/4268787405 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da8011790 con 0x7f9db8159c60 2023-10-10T20:43:46.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.442+0000 7f9dbd54d700 1 -- 172.21.15.89:0/4268787405 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da8011b30 con 0x7f9db8159c60 2023-10-10T20:43:46.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.442+0000 7f9dc07b3700 1 -- 172.21.15.89:0/4268787405 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9db8159c60 msgr2=0x7f9db815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:46.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.443+0000 7f9dc07b3700 1 --2- 172.21.15.89:0/4268787405 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9db8159c60 0x7f9db815d5d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9da80098e0 tx=0x7f9da80091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.443+0000 7f9dc07b3700 1 -- 172.21.15.89:0/4268787405 shutdown_connections 2023-10-10T20:43:46.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.443+0000 7f9dc07b3700 1 --2- 172.21.15.89:0/4268787405 >> v2:172.21.15.111:3300/0 conn(0x7f9db8159890 0x7f9db815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.443+0000 7f9dc07b3700 1 --2- 172.21.15.89:0/4268787405 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9db8159c60 0x7f9db815d5d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.443+0000 7f9dc07b3700 1 -- 172.21.15.89:0/4268787405 >> 172.21.15.89:0/4268787405 conn(0x7f9db8068e20 msgr2=0x7f9db80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:46.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.443+0000 7f9dc07b3700 1 -- 172.21.15.89:0/4268787405 shutdown_connections 2023-10-10T20:43:46.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.443+0000 7f9dc07b3700 1 -- 172.21.15.89:0/4268787405 wait complete. 2023-10-10T20:43:46.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.444+0000 7f9dc07b3700 1 Processor -- start 2023-10-10T20:43:46.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.444+0000 7f9dc07b3700 1 -- start start 2023-10-10T20:43:46.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.444+0000 7f9dc07b3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db8159890 0x7f9db816a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.444+0000 7f9dc07b3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9db8159c60 0x7f9db8150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.444+0000 7f9dc07b3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9db815db10 0x7f9db8154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.444+0000 7f9dc07b3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db8154d10 con 0x7f9db8159890 2023-10-10T20:43:46.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.444+0000 7f9dc07b3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db8154ab0 con 0x7f9db8159c60 2023-10-10T20:43:46.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.444+0000 7f9dc07b3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db815b930 con 0x7f9db815db10 2023-10-10T20:43:46.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.445+0000 7f9dbe54f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db8159890 0x7f9db816a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:46.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.445+0000 7f9dbe54f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db8159890 0x7f9db816a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45008/0 (socket says 172.21.15.89:45008) 2023-10-10T20:43:46.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.445+0000 7f9dbe54f700 1 -- 172.21.15.89:0/2727967444 learned_addr learned my addr 172.21.15.89:0/2727967444 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:46.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.445+0000 7f9dbe54f700 1 -- 172.21.15.89:0/2727967444 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9db815db10 msgr2=0x7f9db8154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:46.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.445+0000 7f9dbe54f700 1 --2- 172.21.15.89:0/2727967444 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9db815db10 0x7f9db8154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.445+0000 7f9dbe54f700 1 -- 172.21.15.89:0/2727967444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9db8159c60 msgr2=0x7f9db8150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:46.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.445+0000 7f9dbe54f700 1 --2- 172.21.15.89:0/2727967444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9db8159c60 0x7f9db8150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.445+0000 7f9dbe54f700 1 -- 172.21.15.89:0/2727967444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db4009a20 con 0x7f9db8159890 2023-10-10T20:43:46.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.445+0000 7f9dbe54f700 1 --2- 172.21.15.89:0/2727967444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db8159890 0x7f9db816a7c0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f9db400c170 tx=0x7f9db400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:46.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.445+0000 7f9dae7fc700 1 -- 172.21.15.89:0/2727967444 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db400ce10 con 0x7f9db8159890 2023-10-10T20:43:46.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.445+0000 7f9dae7fc700 1 -- 172.21.15.89:0/2727967444 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db400f040 con 0x7f9db8159890 2023-10-10T20:43:46.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.445+0000 7f9dc07b3700 1 -- 172.21.15.89:0/2727967444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da8009420 con 0x7f9db8159890 2023-10-10T20:43:46.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.445+0000 7f9dae7fc700 1 -- 172.21.15.89:0/2727967444 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db4014690 con 0x7f9db8159890 2023-10-10T20:43:46.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.445+0000 7f9dc07b3700 1 -- 172.21.15.89:0/2727967444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db815bef0 con 0x7f9db8159890 2023-10-10T20:43:46.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.446+0000 7f9dc07b3700 1 -- 172.21.15.89:0/2727967444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db8151440 con 0x7f9db8159890 2023-10-10T20:43:46.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.446+0000 7f9dae7fc700 1 -- 172.21.15.89:0/2727967444 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9db4014a00 con 0x7f9db8159890 2023-10-10T20:43:46.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.447+0000 7f9dae7fc700 1 --2- 172.21.15.89:0/2727967444 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9d9c067320 0x7f9d9c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.447+0000 7f9dae7fc700 1 -- 172.21.15.89:0/2727967444 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f9db4085cb0 con 0x7f9db8159890 2023-10-10T20:43:46.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.447+0000 7f9dbdd4e700 1 --2- 172.21.15.89:0/2727967444 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9d9c067320 0x7f9d9c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:46.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.447+0000 7f9dbdd4e700 1 --2- 172.21.15.89:0/2727967444 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9d9c067320 0x7f9d9c0697d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9db815a090 tx=0x7f9da8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:46.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.451+0000 7f9dae7fc700 1 -- 172.21.15.89:0/2727967444 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9db40527e0 con 0x7f9db8159890 2023-10-10T20:43:46.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.633+0000 7f9dc07b3700 1 -- 172.21.15.89:0/2727967444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9db804c3c0 con 0x7f9db8159890 2023-10-10T20:43:46.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.633+0000 7f9dae7fc700 1 -- 172.21.15.89:0/2727967444 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v186) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9db4014cb0 con 0x7f9db8159890 2023-10-10T20:43:46.639 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:43:46.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.635+0000 7f9dc07b3700 1 -- 172.21.15.89:0/2727967444 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9d9c067320 msgr2=0x7f9d9c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:46.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.635+0000 7f9dc07b3700 1 --2- 172.21.15.89:0/2727967444 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9d9c067320 0x7f9d9c0697d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9db815a090 tx=0x7f9da8017000 comp rx=0 tx=0).stop 2023-10-10T20:43:46.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.635+0000 7f9dc07b3700 1 -- 172.21.15.89:0/2727967444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db8159890 msgr2=0x7f9db816a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:46.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.635+0000 7f9dc07b3700 1 --2- 172.21.15.89:0/2727967444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db8159890 0x7f9db816a7c0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f9db400c170 tx=0x7f9db400b040 comp rx=0 tx=0).stop 2023-10-10T20:43:46.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.635+0000 7f9dc07b3700 1 -- 172.21.15.89:0/2727967444 shutdown_connections 2023-10-10T20:43:46.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.635+0000 7f9dc07b3700 1 --2- 172.21.15.89:0/2727967444 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9db815db10 0x7f9db8154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.635+0000 7f9dc07b3700 1 --2- 172.21.15.89:0/2727967444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db8159890 0x7f9db816a7c0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.635+0000 7f9dc07b3700 1 --2- 172.21.15.89:0/2727967444 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9d9c067320 0x7f9d9c0697d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.635+0000 7f9dc07b3700 1 --2- 172.21.15.89:0/2727967444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9db8159c60 0x7f9db8150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.635+0000 7f9dc07b3700 1 -- 172.21.15.89:0/2727967444 >> 172.21.15.89:0/2727967444 conn(0x7f9db8068e20 msgr2=0x7f9db80b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:46.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.635+0000 7f9dc07b3700 1 -- 172.21.15.89:0/2727967444 shutdown_connections 2023-10-10T20:43:46.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.635+0000 7f9dc07b3700 1 -- 172.21.15.89:0/2727967444 wait complete. 2023-10-10T20:43:46.655 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000812094.meta pg_num 2023-10-10T20:43:46.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.780+0000 7f95008f0700 1 Processor -- start 2023-10-10T20:43:46.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.780+0000 7f95008f0700 1 -- start start 2023-10-10T20:43:46.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.781+0000 7f95008f0700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f94fc0b4e70 0x7f94fc0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.781+0000 7f95008f0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94fc0b16d0 0x7f94fc0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.781+0000 7f95008f0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94fc0b2080 con 0x7f94fc15d240 2023-10-10T20:43:46.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.781+0000 7f95008f0700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94fc0cfb00 con 0x7f94fc0b4e70 2023-10-10T20:43:46.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.781+0000 7f95008f0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc0b5c40 con 0x7f94fc0b16d0 2023-10-10T20:43:46.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.781+0000 7f94fb59e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f94fc15d240 0x7f94fc15d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39866/0 (socket says 172.21.15.89:39866) 2023-10-10T20:43:46.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.781+0000 7f94fb59e700 1 -- 172.21.15.89:0/2532609142 learned_addr learned my addr 172.21.15.89:0/2532609142 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:46.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.781+0000 7f94fa59c700 1 -- 172.21.15.89:0/2532609142 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286984819 0 0) 0x7f94fc0b2080 con 0x7f94fc15d240 2023-10-10T20:43:46.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.781+0000 7f94fad9d700 1 --2- 172.21.15.89:0/2532609142 >> v2:172.21.15.111:3300/0 conn(0x7f94fc0b4e70 0x7f94fc0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:46.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.781+0000 7f94fa59c700 1 -- 172.21.15.89:0/2532609142 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94e8003570 con 0x7f94fc15d240 2023-10-10T20:43:46.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.781+0000 7f94f259c700 1 --2- 172.21.15.89:0/2532609142 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94fc0b16d0 0x7f94fc0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:46.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fa59c700 1 -- 172.21.15.89:0/2532609142 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1850865230 0 0) 0x7f94e8003570 con 0x7f94fc15d240 2023-10-10T20:43:46.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fa59c700 1 -- 172.21.15.89:0/2532609142 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94fc0b16d0 msgr2=0x7f94fc0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:46.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fa59c700 1 --2- 172.21.15.89:0/2532609142 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94fc0b16d0 0x7f94fc0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fa59c700 1 -- 172.21.15.89:0/2532609142 >> v2:172.21.15.111:3300/0 conn(0x7f94fc0b4e70 msgr2=0x7f94fc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:46.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fa59c700 1 --2- 172.21.15.89:0/2532609142 >> v2:172.21.15.111:3300/0 conn(0x7f94fc0b4e70 0x7f94fc0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fa59c700 1 -- 172.21.15.89:0/2532609142 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94fc0b5f20 con 0x7f94fc15d240 2023-10-10T20:43:46.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fa59c700 1 -- 172.21.15.89:0/2532609142 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f94f40017f0 con 0x7f94fc15d240 2023-10-10T20:43:46.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fa59c700 1 -- 172.21.15.89:0/2532609142 >> v1:172.21.15.89:6789/0 conn(0x7f94fc15d240 legacy=0x7f94fc15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:46.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fa59c700 1 --2- 172.21.15.89:0/2532609142 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94fc0b16d0 0x7f94e8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fa59c700 1 --2- 172.21.15.89:0/2532609142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94e8003fe0 0x7f94e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fa59c700 1 --2- 172.21.15.89:0/2532609142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94e8004d50 0x7f94e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fa59c700 1 -- 172.21.15.89:0/2532609142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc0cfb00 con 0x7f94e8004d50 2023-10-10T20:43:46.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fa59c700 1 -- 172.21.15.89:0/2532609142 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc0b5c40 con 0x7f94e8003fe0 2023-10-10T20:43:46.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fa59c700 1 -- 172.21.15.89:0/2532609142 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e800ba90 con 0x7f94fc0b16d0 2023-10-10T20:43:46.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fb59e700 1 --2- 172.21.15.89:0/2532609142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94e8004d50 0x7f94e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:46.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94f259c700 1 --2- 172.21.15.89:0/2532609142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94e8003fe0 0x7f94e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:46.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fad9d700 1 --2- 172.21.15.89:0/2532609142 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94fc0b16d0 0x7f94e8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:46.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fb59e700 1 -- 172.21.15.89:0/2532609142 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94fc0b16d0 msgr2=0x7f94e8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:46.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fb59e700 1 --2- 172.21.15.89:0/2532609142 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94fc0b16d0 0x7f94e8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fb59e700 1 -- 172.21.15.89:0/2532609142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94e8003fe0 msgr2=0x7f94e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:46.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fb59e700 1 --2- 172.21.15.89:0/2532609142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94e8003fe0 0x7f94e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fb59e700 1 -- 172.21.15.89:0/2532609142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e8003570 con 0x7f94e8004d50 2023-10-10T20:43:46.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fb59e700 1 --2- 172.21.15.89:0/2532609142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94e8004d50 0x7f94e80051c0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f94f400a970 tx=0x7f94f4002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36ca1d4743a50c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:46.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fa59c700 1 -- 172.21.15.89:0/2532609142 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f4012440 con 0x7f94e8004d50 2023-10-10T20:43:46.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fa59c700 1 -- 172.21.15.89:0/2532609142 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f40125c0 con 0x7f94e8004d50 2023-10-10T20:43:46.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.782+0000 7f94fa59c700 1 -- 172.21.15.89:0/2532609142 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f4012980 con 0x7f94e8004d50 2023-10-10T20:43:46.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.783+0000 7f95008f0700 1 -- 172.21.15.89:0/2532609142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94e8004d50 msgr2=0x7f94e80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:46.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.783+0000 7f95008f0700 1 --2- 172.21.15.89:0/2532609142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94e8004d50 0x7f94e80051c0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f94f400a970 tx=0x7f94f4002210 comp rx=0 tx=0).stop 2023-10-10T20:43:46.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.783+0000 7f94fb59e700 1 -- 172.21.15.89:0/2532609142 reap_dead start 2023-10-10T20:43:46.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.783+0000 7f95008f0700 1 -- 172.21.15.89:0/2532609142 shutdown_connections 2023-10-10T20:43:46.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.783+0000 7f95008f0700 1 -- 172.21.15.89:0/2532609142 >> 172.21.15.89:0/2532609142 conn(0x7f94fc0d10c0 msgr2=0x7f94fc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:46.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.783+0000 7f95008f0700 1 -- 172.21.15.89:0/2532609142 shutdown_connections 2023-10-10T20:43:46.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.783+0000 7f95008f0700 1 -- 172.21.15.89:0/2532609142 wait complete. 2023-10-10T20:43:46.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.784+0000 7f95008f0700 1 Processor -- start 2023-10-10T20:43:46.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.784+0000 7f95008f0700 1 -- start start 2023-10-10T20:43:46.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.785+0000 7f95008f0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94fc0c73a0 0x7f94fc14b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.785+0000 7f95008f0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94fc0c9b10 0x7f94fc0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.785+0000 7f95008f0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94fc0c7770 0x7f94fc0c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.785+0000 7f95008f0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc0cac90 con 0x7f94fc0c7770 2023-10-10T20:43:46.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.785+0000 7f95008f0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc14be30 con 0x7f94fc0c73a0 2023-10-10T20:43:46.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.785+0000 7f95008f0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc0ca080 con 0x7f94fc0c9b10 2023-10-10T20:43:46.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.785+0000 7f94fb59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94fc0c7770 0x7f94fc0c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:46.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.785+0000 7f94fb59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94fc0c7770 0x7f94fc0c7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45032/0 (socket says 172.21.15.89:45032) 2023-10-10T20:43:46.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.785+0000 7f94fb59e700 1 -- 172.21.15.89:0/1785720211 learned_addr learned my addr 172.21.15.89:0/1785720211 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:46.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.785+0000 7f94f259c700 1 --2- 172.21.15.89:0/1785720211 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94fc0c9b10 0x7f94fc0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:46.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.785+0000 7f94fad9d700 1 --2- 172.21.15.89:0/1785720211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94fc0c73a0 0x7f94fc14b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:46.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.785+0000 7f94fb59e700 1 -- 172.21.15.89:0/1785720211 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94fc0c9b10 msgr2=0x7f94fc0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:46.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.785+0000 7f94fb59e700 1 --2- 172.21.15.89:0/1785720211 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94fc0c9b10 0x7f94fc0cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.785+0000 7f94fb59e700 1 -- 172.21.15.89:0/1785720211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94fc0c73a0 msgr2=0x7f94fc14b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:46.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.785+0000 7f94fb59e700 1 --2- 172.21.15.89:0/1785720211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94fc0c73a0 0x7f94fc14b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.785+0000 7f94fb59e700 1 -- 172.21.15.89:0/1785720211 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94fc0ca360 con 0x7f94fc0c7770 2023-10-10T20:43:46.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.785+0000 7f94fb59e700 1 --2- 172.21.15.89:0/1785720211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94fc0c7770 0x7f94fc0c7be0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f94f4002250 tx=0x7f94f4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:46.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.786+0000 7f94f37fe700 1 -- 172.21.15.89:0/1785720211 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f4023070 con 0x7f94fc0c7770 2023-10-10T20:43:46.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.786+0000 7f94f37fe700 1 -- 172.21.15.89:0/1785720211 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f4012440 con 0x7f94fc0c7770 2023-10-10T20:43:46.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.786+0000 7f95008f0700 1 -- 172.21.15.89:0/1785720211 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94fc0ca640 con 0x7f94fc0c7770 2023-10-10T20:43:46.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.786+0000 7f94f37fe700 1 -- 172.21.15.89:0/1785720211 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f4007530 con 0x7f94fc0c7770 2023-10-10T20:43:46.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.786+0000 7f95008f0700 1 -- 172.21.15.89:0/1785720211 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94fc0cca40 con 0x7f94fc0c7770 2023-10-10T20:43:46.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.787+0000 7f95008f0700 1 -- 172.21.15.89:0/1785720211 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94fc088990 con 0x7f94fc0c7770 2023-10-10T20:43:46.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.787+0000 7f94f37fe700 1 -- 172.21.15.89:0/1785720211 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f94f4019030 con 0x7f94fc0c7770 2023-10-10T20:43:46.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.787+0000 7f94f37fe700 1 --2- 172.21.15.89:0/1785720211 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f94dc067320 0x7f94dc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:46.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.788+0000 7f94f37fe700 1 -- 172.21.15.89:0/1785720211 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f94f4086110 con 0x7f94fc0c7770 2023-10-10T20:43:46.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.788+0000 7f94fad9d700 1 --2- 172.21.15.89:0/1785720211 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f94dc067320 0x7f94dc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:46.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.788+0000 7f94fad9d700 1 --2- 172.21.15.89:0/1785720211 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f94dc067320 0x7f94dc0697d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f94ec006f00 tx=0x7f94ec0091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:46.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.792+0000 7f94f37fe700 1 -- 172.21.15.89:0/1785720211 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94f4052d50 con 0x7f94fc0c7770 2023-10-10T20:43:46.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.970+0000 7f95008f0700 1 -- 172.21.15.89:0/1785720211 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000812094.meta", "var": "pg_num"} v 0) v1 -- 0x7f94fc0cf4b0 con 0x7f94fc0c7770 2023-10-10T20:43:46.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.971+0000 7f94f37fe700 1 -- 172.21.15.89:0/1785720211 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000812094.meta", "var": "pg_num"}]=0 v186) v1 ==== 124+0+11 (secure 0 0 0) 0x7f94f40170c0 con 0x7f94fc0c7770 2023-10-10T20:43:46.974 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:43:46.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.973+0000 7f95008f0700 1 -- 172.21.15.89:0/1785720211 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f94dc067320 msgr2=0x7f94dc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:46.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.973+0000 7f95008f0700 1 --2- 172.21.15.89:0/1785720211 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f94dc067320 0x7f94dc0697d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f94ec006f00 tx=0x7f94ec0091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.973+0000 7f95008f0700 1 -- 172.21.15.89:0/1785720211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94fc0c7770 msgr2=0x7f94fc0c7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:46.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.973+0000 7f95008f0700 1 --2- 172.21.15.89:0/1785720211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94fc0c7770 0x7f94fc0c7be0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f94f4002250 tx=0x7f94f4015000 comp rx=0 tx=0).stop 2023-10-10T20:43:46.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.973+0000 7f95008f0700 1 -- 172.21.15.89:0/1785720211 shutdown_connections 2023-10-10T20:43:46.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.973+0000 7f95008f0700 1 --2- 172.21.15.89:0/1785720211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94fc0c7770 0x7f94fc0c7be0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.973+0000 7f95008f0700 1 --2- 172.21.15.89:0/1785720211 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f94dc067320 0x7f94dc0697d0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.973+0000 7f95008f0700 1 --2- 172.21.15.89:0/1785720211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94fc0c73a0 0x7f94fc14b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.973+0000 7f95008f0700 1 --2- 172.21.15.89:0/1785720211 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94fc0c9b10 0x7f94fc0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:46.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.973+0000 7f95008f0700 1 -- 172.21.15.89:0/1785720211 >> 172.21.15.89:0/1785720211 conn(0x7f94fc0d10c0 msgr2=0x7f94fc159050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:46.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.973+0000 7f95008f0700 1 -- 172.21.15.89:0/1785720211 shutdown_connections 2023-10-10T20:43:46.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:46.973+0000 7f95008f0700 1 -- 172.21.15.89:0/1785720211 wait complete. 2023-10-10T20:43:46.987 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000812094.data pg_num 2023-10-10T20:43:47.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.110+0000 7f0c08850700 1 Processor -- start 2023-10-10T20:43:47.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.110+0000 7f0c08850700 1 -- start start 2023-10-10T20:43:47.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c08850700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c040b4e70 0x7f0c040ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c08850700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0c040b1b40 0x7f0c0415d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c08850700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c040cfb00 con 0x7f0c040b16d0 2023-10-10T20:43:47.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c08850700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c040b5c40 con 0x7f0c040b1b40 2023-10-10T20:43:47.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c08850700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c040b5f20 con 0x7f0c040b4e70 2023-10-10T20:43:47.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c0259c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0c040b16d0 0x7f0c0415d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39874/0 (socket says 172.21.15.89:39874) 2023-10-10T20:43:47.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c0259c700 1 -- 172.21.15.89:0/984496711 learned_addr learned my addr 172.21.15.89:0/984496711 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:47.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c0359e700 1 --2- 172.21.15.89:0/984496711 >> v2:172.21.15.111:3300/0 conn(0x7f0c040b1b40 0x7f0c0415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c02d9d700 1 --2- 172.21.15.89:0/984496711 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c040b4e70 0x7f0c040ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c01d9b700 1 -- 172.21.15.89:0/984496711 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4282875473 0 0) 0x7f0c040cfb00 con 0x7f0c040b16d0 2023-10-10T20:43:47.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c01d9b700 1 -- 172.21.15.89:0/984496711 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bec003570 con 0x7f0c040b16d0 2023-10-10T20:43:47.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c01d9b700 1 -- 172.21.15.89:0/984496711 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4278696777 0 0) 0x7f0bec003570 con 0x7f0c040b16d0 2023-10-10T20:43:47.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c01d9b700 1 -- 172.21.15.89:0/984496711 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c040b4e70 msgr2=0x7f0c040ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:47.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c01d9b700 1 --2- 172.21.15.89:0/984496711 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c040b4e70 0x7f0c040ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c01d9b700 1 -- 172.21.15.89:0/984496711 >> v2:172.21.15.111:3300/0 conn(0x7f0c040b1b40 msgr2=0x7f0c0415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:47.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c01d9b700 1 --2- 172.21.15.89:0/984496711 >> v2:172.21.15.111:3300/0 conn(0x7f0c040b1b40 0x7f0c0415d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c01d9b700 1 -- 172.21.15.89:0/984496711 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c040b6200 con 0x7f0c040b16d0 2023-10-10T20:43:47.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c01d9b700 1 -- 172.21.15.89:0/984496711 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f0bf8001470 con 0x7f0c040b16d0 2023-10-10T20:43:47.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c01d9b700 1 -- 172.21.15.89:0/984496711 >> v1:172.21.15.89:6789/0 conn(0x7f0c040b16d0 legacy=0x7f0c0415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:47.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c01d9b700 1 --2- 172.21.15.89:0/984496711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bec003a60 0x7f0bec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c01d9b700 1 --2- 172.21.15.89:0/984496711 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c040b4e70 0x7f0bec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c01d9b700 1 --2- 172.21.15.89:0/984496711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bec004d50 0x7f0bec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c01d9b700 1 -- 172.21.15.89:0/984496711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c040b5c40 con 0x7f0bec004d50 2023-10-10T20:43:47.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c01d9b700 1 -- 172.21.15.89:0/984496711 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c040b5f20 con 0x7f0bec003a60 2023-10-10T20:43:47.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c01d9b700 1 -- 172.21.15.89:0/984496711 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec00ba60 con 0x7f0c040b4e70 2023-10-10T20:43:47.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.111+0000 7f0c0359e700 1 --2- 172.21.15.89:0/984496711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bec004d50 0x7f0bec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.112+0000 7f0c0259c700 1 --2- 172.21.15.89:0/984496711 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c040b4e70 0x7f0bec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.112+0000 7f0c02d9d700 1 --2- 172.21.15.89:0/984496711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bec003a60 0x7f0bec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.112+0000 7f0c0359e700 1 -- 172.21.15.89:0/984496711 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c040b4e70 msgr2=0x7f0bec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:47.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.112+0000 7f0c0359e700 1 --2- 172.21.15.89:0/984496711 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c040b4e70 0x7f0bec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.112+0000 7f0c0359e700 1 -- 172.21.15.89:0/984496711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bec003a60 msgr2=0x7f0bec003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:47.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.112+0000 7f0c0359e700 1 --2- 172.21.15.89:0/984496711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bec003a60 0x7f0bec003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.112+0000 7f0c0359e700 1 -- 172.21.15.89:0/984496711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec003570 con 0x7f0bec004d50 2023-10-10T20:43:47.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.112+0000 7f0c02d9d700 1 --2- 172.21.15.89:0/984496711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bec003a60 0x7f0bec003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:47.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.112+0000 7f0c0359e700 1 --2- 172.21.15.89:0/984496711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bec004d50 0x7f0bec0051c0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00e210 tx=0x7f0bfc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d445a43b01602c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:47.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.112+0000 7f0c01d9b700 1 -- 172.21.15.89:0/984496711 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc00ea60 con 0x7f0bec004d50 2023-10-10T20:43:47.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.112+0000 7f0c01d9b700 1 -- 172.21.15.89:0/984496711 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bfc00f040 con 0x7f0bec004d50 2023-10-10T20:43:47.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.112+0000 7f0c01d9b700 1 -- 172.21.15.89:0/984496711 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc00ee10 con 0x7f0bec004d50 2023-10-10T20:43:47.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.112+0000 7f0c08850700 1 -- 172.21.15.89:0/984496711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bec004d50 msgr2=0x7f0bec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:47.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.112+0000 7f0c08850700 1 --2- 172.21.15.89:0/984496711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bec004d50 0x7f0bec0051c0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00e210 tx=0x7f0bfc00b040 comp rx=0 tx=0).stop 2023-10-10T20:43:47.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.112+0000 7f0c0359e700 1 -- 172.21.15.89:0/984496711 reap_dead start 2023-10-10T20:43:47.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.112+0000 7f0c08850700 1 -- 172.21.15.89:0/984496711 shutdown_connections 2023-10-10T20:43:47.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.113+0000 7f0c08850700 1 -- 172.21.15.89:0/984496711 >> 172.21.15.89:0/984496711 conn(0x7f0c040d10c0 msgr2=0x7f0c040affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:47.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.113+0000 7f0c08850700 1 -- 172.21.15.89:0/984496711 shutdown_connections 2023-10-10T20:43:47.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.113+0000 7f0c08850700 1 -- 172.21.15.89:0/984496711 wait complete. 2023-10-10T20:43:47.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.114+0000 7f0c08850700 1 Processor -- start 2023-10-10T20:43:47.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.114+0000 7f0c08850700 1 -- start start 2023-10-10T20:43:47.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.114+0000 7f0c08850700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c040c9980 0x7f0c040ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0c08850700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c040c9d50 0x7f0c040cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0c08850700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c040c5390 0x7f0c040c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0c08850700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c041577c0 con 0x7f0c040c5390 2023-10-10T20:43:47.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0c08850700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c040c5d50 con 0x7f0c040c9d50 2023-10-10T20:43:47.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0c08850700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c040c5fe0 con 0x7f0c040c9980 2023-10-10T20:43:47.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0c0359e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c040c5390 0x7f0c040c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0c0359e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c040c5390 0x7f0c040c5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45046/0 (socket says 172.21.15.89:45046) 2023-10-10T20:43:47.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0c0359e700 1 -- 172.21.15.89:0/2107698477 learned_addr learned my addr 172.21.15.89:0/2107698477 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:47.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0c02d9d700 1 --2- 172.21.15.89:0/2107698477 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c040c9980 0x7f0c040ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0c0259c700 1 --2- 172.21.15.89:0/2107698477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c040c9d50 0x7f0c040cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0c0359e700 1 -- 172.21.15.89:0/2107698477 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c040c9980 msgr2=0x7f0c040ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:47.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0c0359e700 1 --2- 172.21.15.89:0/2107698477 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c040c9980 0x7f0c040ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0c0359e700 1 -- 172.21.15.89:0/2107698477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c040c9d50 msgr2=0x7f0c040cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:47.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0c0359e700 1 --2- 172.21.15.89:0/2107698477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c040c9d50 0x7f0c040cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0c0359e700 1 -- 172.21.15.89:0/2107698477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c040c62c0 con 0x7f0c040c5390 2023-10-10T20:43:47.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0c0359e700 1 --2- 172.21.15.89:0/2107698477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c040c5390 0x7f0c040c5810 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc016000 tx=0x7f0bfc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:47.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0beaffd700 1 -- 172.21.15.89:0/2107698477 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc01d630 con 0x7f0c040c5390 2023-10-10T20:43:47.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0beaffd700 1 -- 172.21.15.89:0/2107698477 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bfc01d790 con 0x7f0c040c5390 2023-10-10T20:43:47.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0c08850700 1 -- 172.21.15.89:0/2107698477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c040c65a0 con 0x7f0c040c5390 2023-10-10T20:43:47.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0beaffd700 1 -- 172.21.15.89:0/2107698477 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc01db30 con 0x7f0c040c5390 2023-10-10T20:43:47.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.115+0000 7f0c08850700 1 -- 172.21.15.89:0/2107698477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c040c6b60 con 0x7f0c040c5390 2023-10-10T20:43:47.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.116+0000 7f0c08850700 1 -- 172.21.15.89:0/2107698477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c04088990 con 0x7f0c040c5390 2023-10-10T20:43:47.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.117+0000 7f0beaffd700 1 -- 172.21.15.89:0/2107698477 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0bfc003040 con 0x7f0c040c5390 2023-10-10T20:43:47.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.117+0000 7f0beaffd700 1 --2- 172.21.15.89:0/2107698477 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0bf0067320 0x7f0bf00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.117+0000 7f0beaffd700 1 -- 172.21.15.89:0/2107698477 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f0bfc086690 con 0x7f0c040c5390 2023-10-10T20:43:47.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.117+0000 7f0c02d9d700 1 --2- 172.21.15.89:0/2107698477 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0bf0067320 0x7f0bf00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.117+0000 7f0c02d9d700 1 --2- 172.21.15.89:0/2107698477 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0bf0067320 0x7f0bf00697d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009db0 tx=0x7f0bf4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:47.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.121+0000 7f0beaffd700 1 -- 172.21.15.89:0/2107698477 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0bfc0532c0 con 0x7f0c040c5390 2023-10-10T20:43:47.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.304+0000 7f0c08850700 1 -- 172.21.15.89:0/2107698477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000812094.data", "var": "pg_num"} v 0) v1 -- 0x7f0c0404bb80 con 0x7f0c040c5390 2023-10-10T20:43:47.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.304+0000 7f0beaffd700 1 -- 172.21.15.89:0/2107698477 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000812094.data", "var": "pg_num"}]=0 v186) v1 ==== 124+0+11 (secure 0 0 0) 0x7f0bfc0568e0 con 0x7f0c040c5390 2023-10-10T20:43:47.310 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-10T20:43:47.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.306+0000 7f0c08850700 1 -- 172.21.15.89:0/2107698477 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0bf0067320 msgr2=0x7f0bf00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:47.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.306+0000 7f0c08850700 1 --2- 172.21.15.89:0/2107698477 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0bf0067320 0x7f0bf00697d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009db0 tx=0x7f0bf4008040 comp rx=0 tx=0).stop 2023-10-10T20:43:47.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.306+0000 7f0c08850700 1 -- 172.21.15.89:0/2107698477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c040c5390 msgr2=0x7f0c040c5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:47.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.306+0000 7f0c08850700 1 --2- 172.21.15.89:0/2107698477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c040c5390 0x7f0c040c5810 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc016000 tx=0x7f0bfc01a000 comp rx=0 tx=0).stop 2023-10-10T20:43:47.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.306+0000 7f0c08850700 1 -- 172.21.15.89:0/2107698477 shutdown_connections 2023-10-10T20:43:47.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.306+0000 7f0c08850700 1 --2- 172.21.15.89:0/2107698477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c040c5390 0x7f0c040c5810 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.306+0000 7f0c08850700 1 --2- 172.21.15.89:0/2107698477 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0c040c9980 0x7f0c040ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.306+0000 7f0c08850700 1 --2- 172.21.15.89:0/2107698477 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0bf0067320 0x7f0bf00697d0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.306+0000 7f0c08850700 1 --2- 172.21.15.89:0/2107698477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c040c9d50 0x7f0c040cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.306+0000 7f0c08850700 1 -- 172.21.15.89:0/2107698477 >> 172.21.15.89:0/2107698477 conn(0x7f0c040d10c0 msgr2=0x7f0c04159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:47.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.306+0000 7f0c08850700 1 -- 172.21.15.89:0/2107698477 shutdown_connections 2023-10-10T20:43:47.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.306+0000 7f0c08850700 1 -- 172.21.15.89:0/2107698477 wait complete. 2023-10-10T20:43:47.326 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:43:47.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.443+0000 7fa57ac4a700 1 Processor -- start 2023-10-10T20:43:47.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.443+0000 7fa57ac4a700 1 -- start start 2023-10-10T20:43:47.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.444+0000 7fa57ac4a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa574159890 0x7fa57415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.444+0000 7fa57ac4a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa574159c60 0x7fa57415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.444+0000 7fa57ac4a700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57415a160 con 0x7fa57415db10 2023-10-10T20:43:47.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.444+0000 7fa57ac4a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5740d13a0 con 0x7fa574159890 2023-10-10T20:43:47.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.444+0000 7fa57ac4a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57415e140 con 0x7fa574159c60 2023-10-10T20:43:47.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.444+0000 7fa5791e7700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa57415db10 0x7fa574069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:35934/0 (socket says 172.21.15.89:35934) 2023-10-10T20:43:47.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.444+0000 7fa5791e7700 1 -- 172.21.15.89:0/2860750881 learned_addr learned my addr 172.21.15.89:0/2860750881 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:47.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.444+0000 7fa56bfff700 1 --2- 172.21.15.89:0/2860750881 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa574159c60 0x7fa57415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.444+0000 7fa5789e6700 1 --2- 172.21.15.89:0/2860750881 >> v2:172.21.15.111:3300/0 conn(0x7fa574159890 0x7fa57415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.445+0000 7fa56b7fe700 1 -- 172.21.15.89:0/2860750881 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916702670 0 0) 0x7fa57415a160 con 0x7fa57415db10 2023-10-10T20:43:47.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.445+0000 7fa56b7fe700 1 -- 172.21.15.89:0/2860750881 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa55c003570 con 0x7fa57415db10 2023-10-10T20:43:47.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.445+0000 7fa56bfff700 1 -- 172.21.15.89:0/2860750881 >> v2:172.21.15.111:3300/0 conn(0x7fa574159890 msgr2=0x7fa57415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:47.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.445+0000 7fa56bfff700 1 --2- 172.21.15.89:0/2860750881 >> v2:172.21.15.111:3300/0 conn(0x7fa574159890 0x7fa57415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.445+0000 7fa56bfff700 1 -- 172.21.15.89:0/2860750881 >> v1:172.21.15.111:6789/0 conn(0x7fa57415db10 legacy=0x7fa574069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:47.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.445+0000 7fa56bfff700 1 -- 172.21.15.89:0/2860750881 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57415a3d0 con 0x7fa574159c60 2023-10-10T20:43:47.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.445+0000 7fa5789e6700 1 --2- 172.21.15.89:0/2860750881 >> v2:172.21.15.111:3300/0 conn(0x7fa574159890 0x7fa57415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:43:47.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.445+0000 7fa56bfff700 1 --2- 172.21.15.89:0/2860750881 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa574159c60 0x7fa57415d5d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa5640098e0 tx=0x7fa5640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=720e16be2db78a28 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:47.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.445+0000 7fa56b7fe700 1 -- 172.21.15.89:0/2860750881 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564011630 con 0x7fa574159c60 2023-10-10T20:43:47.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.445+0000 7fa56b7fe700 1 -- 172.21.15.89:0/2860750881 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa564011790 con 0x7fa574159c60 2023-10-10T20:43:47.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.445+0000 7fa56b7fe700 1 -- 172.21.15.89:0/2860750881 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564011b30 con 0x7fa574159c60 2023-10-10T20:43:47.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.446+0000 7fa57ac4a700 1 -- 172.21.15.89:0/2860750881 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa574159c60 msgr2=0x7fa57415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:47.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.446+0000 7fa57ac4a700 1 --2- 172.21.15.89:0/2860750881 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa574159c60 0x7fa57415d5d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa5640098e0 tx=0x7fa5640091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.446+0000 7fa57ac4a700 1 -- 172.21.15.89:0/2860750881 shutdown_connections 2023-10-10T20:43:47.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.446+0000 7fa57ac4a700 1 --2- 172.21.15.89:0/2860750881 >> v2:172.21.15.111:3300/0 conn(0x7fa574159890 0x7fa57415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.446+0000 7fa57ac4a700 1 --2- 172.21.15.89:0/2860750881 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa574159c60 0x7fa57415d5d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.446+0000 7fa57ac4a700 1 -- 172.21.15.89:0/2860750881 >> 172.21.15.89:0/2860750881 conn(0x7fa574068e20 msgr2=0x7fa5740d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:47.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.446+0000 7fa57ac4a700 1 -- 172.21.15.89:0/2860750881 shutdown_connections 2023-10-10T20:43:47.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.446+0000 7fa57ac4a700 1 -- 172.21.15.89:0/2860750881 wait complete. 2023-10-10T20:43:47.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.447+0000 7fa57ac4a700 1 Processor -- start 2023-10-10T20:43:47.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.447+0000 7fa57ac4a700 1 -- start start 2023-10-10T20:43:47.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa57ac4a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa574159890 0x7fa57415c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa57ac4a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa574159c60 0x7fa57414a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa57ac4a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa57415db10 0x7fa574155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa57ac4a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57415d950 con 0x7fa574159c60 2023-10-10T20:43:47.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa57ac4a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57414aab0 con 0x7fa574159890 2023-10-10T20:43:47.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa57ac4a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57415d0b0 con 0x7fa57415db10 2023-10-10T20:43:47.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa56bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa574159c60 0x7fa57414a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa56bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa574159c60 0x7fa57414a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45052/0 (socket says 172.21.15.89:45052) 2023-10-10T20:43:47.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa56bfff700 1 -- 172.21.15.89:0/1292091396 learned_addr learned my addr 172.21.15.89:0/1292091396 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:47.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa5789e6700 1 --2- 172.21.15.89:0/1292091396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa574159890 0x7fa57415c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa5791e7700 1 --2- 172.21.15.89:0/1292091396 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa57415db10 0x7fa574155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa56bfff700 1 -- 172.21.15.89:0/1292091396 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa57415db10 msgr2=0x7fa574155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:47.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa56bfff700 1 --2- 172.21.15.89:0/1292091396 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa57415db10 0x7fa574155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa56bfff700 1 -- 172.21.15.89:0/1292091396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa574159890 msgr2=0x7fa57415c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:47.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa56bfff700 1 --2- 172.21.15.89:0/1292091396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa574159890 0x7fa57415c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa56bfff700 1 -- 172.21.15.89:0/1292091396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa570009a20 con 0x7fa574159c60 2023-10-10T20:43:47.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa56bfff700 1 --2- 172.21.15.89:0/1292091396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa574159c60 0x7fa57414a570 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fa564013000 tx=0x7fa564017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:47.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa568ff9700 1 -- 172.21.15.89:0/1292091396 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564011900 con 0x7fa574159c60 2023-10-10T20:43:47.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa568ff9700 1 -- 172.21.15.89:0/1292091396 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa564011a60 con 0x7fa574159c60 2023-10-10T20:43:47.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa57ac4a700 1 -- 172.21.15.89:0/1292091396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa564009420 con 0x7fa574159c60 2023-10-10T20:43:47.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.448+0000 7fa568ff9700 1 -- 172.21.15.89:0/1292091396 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564025640 con 0x7fa574159c60 2023-10-10T20:43:47.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.449+0000 7fa57ac4a700 1 -- 172.21.15.89:0/1292091396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57415d670 con 0x7fa574159c60 2023-10-10T20:43:47.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.449+0000 7fa57ac4a700 1 -- 172.21.15.89:0/1292091396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57404c3c0 con 0x7fa574159c60 2023-10-10T20:43:47.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.450+0000 7fa568ff9700 1 -- 172.21.15.89:0/1292091396 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa564025a30 con 0x7fa574159c60 2023-10-10T20:43:47.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.450+0000 7fa568ff9700 1 --2- 172.21.15.89:0/1292091396 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa560067320 0x7fa5600697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.450+0000 7fa568ff9700 1 -- 172.21.15.89:0/1292091396 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fa564086410 con 0x7fa574159c60 2023-10-10T20:43:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.450+0000 7fa5789e6700 1 --2- 172.21.15.89:0/1292091396 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa560067320 0x7fa5600697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.451+0000 7fa5789e6700 1 --2- 172.21.15.89:0/1292091396 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa560067320 0x7fa5600697d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa570009fd0 tx=0x7fa57000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.454+0000 7fa568ff9700 1 -- 172.21.15.89:0/1292091396 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa564053010 con 0x7fa574159c60 2023-10-10T20:43:47.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.680+0000 7fa57ac4a700 1 -- 172.21.15.89:0/1292091396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa57406a5b0 con 0x7fa574159c60 2023-10-10T20:43:47.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.681+0000 7fa568ff9700 1 -- 172.21.15.89:0/1292091396 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4627 (secure 0 0 0) 0x7fa564056630 con 0x7fa574159c60 2023-10-10T20:43:47.684 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:47.684 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":124,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/455398787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":455398787},{"type":"v1","addr":"172.21.15.89:6836","nonce":455398787}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/2660411678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2660411678},{"type":"v1","addr":"172.21.15.111:6834","nonce":2660411678}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6890,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/1650071647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":1650071647},{"type":"v1","addr":"172.21.15.111:6835","nonce":1650071647}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:43:29.795107+0000","modified":"2023-10-10T20:43:41.456555+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8760":{"gid":8760,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/2168500629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2168500629},{"type":"v1","addr":"172.21.15.89:6837","nonce":2168500629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"volume_0000000000812094","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-10T20:43:47.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.683+0000 7fa57ac4a700 1 -- 172.21.15.89:0/1292091396 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa560067320 msgr2=0x7fa5600697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:47.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.683+0000 7fa57ac4a700 1 --2- 172.21.15.89:0/1292091396 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa560067320 0x7fa5600697d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa570009fd0 tx=0x7fa57000b040 comp rx=0 tx=0).stop 2023-10-10T20:43:47.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.683+0000 7fa57ac4a700 1 -- 172.21.15.89:0/1292091396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa574159c60 msgr2=0x7fa57414a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:47.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.683+0000 7fa57ac4a700 1 --2- 172.21.15.89:0/1292091396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa574159c60 0x7fa57414a570 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fa564013000 tx=0x7fa564017000 comp rx=0 tx=0).stop 2023-10-10T20:43:47.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.683+0000 7fa57ac4a700 1 -- 172.21.15.89:0/1292091396 shutdown_connections 2023-10-10T20:43:47.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.683+0000 7fa57ac4a700 1 --2- 172.21.15.89:0/1292091396 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa57415db10 0x7fa574155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.683+0000 7fa57ac4a700 1 --2- 172.21.15.89:0/1292091396 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa560067320 0x7fa5600697d0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.683+0000 7fa57ac4a700 1 --2- 172.21.15.89:0/1292091396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa574159890 0x7fa57415c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.683+0000 7fa57ac4a700 1 --2- 172.21.15.89:0/1292091396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa574159c60 0x7fa57414a570 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.683+0000 7fa57ac4a700 1 -- 172.21.15.89:0/1292091396 >> 172.21.15.89:0/1292091396 conn(0x7fa574068e20 msgr2=0x7fa574152d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:47.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.683+0000 7fa57ac4a700 1 -- 172.21.15.89:0/1292091396 shutdown_connections 2023-10-10T20:43:47.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.683+0000 7fa57ac4a700 1 -- 172.21.15.89:0/1292091396 wait complete. 2023-10-10T20:43:47.688 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 124 2023-10-10T20:43:47.697 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:43:47.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.810+0000 7f6d33935700 1 Processor -- start 2023-10-10T20:43:47.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.810+0000 7f6d33935700 1 -- start start 2023-10-10T20:43:47.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.811+0000 7f6d33935700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6d2c159890 0x7f6d2c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.811+0000 7f6d33935700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d2c069560 0x7f6d2c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.811+0000 7f6d33935700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d2c069ee0 con 0x7f6d2c159c60 2023-10-10T20:43:47.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.811+0000 7f6d33935700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d2c15a160 con 0x7f6d2c159890 2023-10-10T20:43:47.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.811+0000 7f6d33935700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c0d13a0 con 0x7f6d2c069560 2023-10-10T20:43:47.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.812+0000 7f6d316d1700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6d2c159890 0x7f6d2c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.812+0000 7f6d31ed2700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d2c069560 0x7f6d2c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.812+0000 7f6d30ed0700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f6d2c159c60 0x7f6d2c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:35950/0 (socket says 172.21.15.89:35950) 2023-10-10T20:43:47.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.812+0000 7f6d30ed0700 1 -- 172.21.15.89:0/1028104102 learned_addr learned my addr 172.21.15.89:0/1028104102 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:47.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.812+0000 7f6d316d1700 1 -- 172.21.15.89:0/1028104102 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d2c069560 msgr2=0x7f6d2c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:47.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.812+0000 7f6d316d1700 1 --2- 172.21.15.89:0/1028104102 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d2c069560 0x7f6d2c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.812+0000 7f6d316d1700 1 -- 172.21.15.89:0/1028104102 >> v1:172.21.15.111:6789/0 conn(0x7f6d2c159c60 legacy=0x7f6d2c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:47.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.812+0000 7f6d316d1700 1 -- 172.21.15.89:0/1028104102 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2c15e140 con 0x7f6d2c159890 2023-10-10T20:43:47.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.812+0000 7f6d23fff700 1 -- 172.21.15.89:0/1028104102 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532419416 0 0) 0x7f6d2c069ee0 con 0x7f6d2c159c60 2023-10-10T20:43:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.812+0000 7f6d31ed2700 1 --2- 172.21.15.89:0/1028104102 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d2c069560 0x7f6d2c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:43:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.812+0000 7f6d316d1700 1 --2- 172.21.15.89:0/1028104102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d2c159890 0x7f6d2c15abd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6d280098e0 tx=0x7f6d280091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18fd5495db8390a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.812+0000 7f6d23fff700 1 -- 172.21.15.89:0/1028104102 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2800b640 con 0x7f6d2c159890 2023-10-10T20:43:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.812+0000 7f6d23fff700 1 -- 172.21.15.89:0/1028104102 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2800b7a0 con 0x7f6d2c159890 2023-10-10T20:43:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.812+0000 7f6d23fff700 1 -- 172.21.15.89:0/1028104102 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2800bb40 con 0x7f6d2c159890 2023-10-10T20:43:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.813+0000 7f6d33935700 1 -- 172.21.15.89:0/1028104102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d2c159890 msgr2=0x7f6d2c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.813+0000 7f6d33935700 1 --2- 172.21.15.89:0/1028104102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d2c159890 0x7f6d2c15abd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6d280098e0 tx=0x7f6d280091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.813+0000 7f6d33935700 1 -- 172.21.15.89:0/1028104102 shutdown_connections 2023-10-10T20:43:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.813+0000 7f6d33935700 1 --2- 172.21.15.89:0/1028104102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d2c159890 0x7f6d2c15abd0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.813+0000 7f6d33935700 1 --2- 172.21.15.89:0/1028104102 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d2c069560 0x7f6d2c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.813+0000 7f6d33935700 1 -- 172.21.15.89:0/1028104102 >> 172.21.15.89:0/1028104102 conn(0x7f6d2c068e20 msgr2=0x7f6d2c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:47.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.813+0000 7f6d33935700 1 -- 172.21.15.89:0/1028104102 shutdown_connections 2023-10-10T20:43:47.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.813+0000 7f6d33935700 1 -- 172.21.15.89:0/1028104102 wait complete. 2023-10-10T20:43:47.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.814+0000 7f6d33935700 1 Processor -- start 2023-10-10T20:43:47.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.814+0000 7f6d33935700 1 -- start start 2023-10-10T20:43:47.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.815+0000 7f6d33935700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d2c069560 0x7f6d2c0ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.815+0000 7f6d33935700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d2c159890 0x7f6d2c0cf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.815+0000 7f6d33935700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d2c159c60 0x7f6d2c0cf940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.815+0000 7f6d33935700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c0cdeb0 con 0x7f6d2c069560 2023-10-10T20:43:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.815+0000 7f6d33935700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c0ce100 con 0x7f6d2c159c60 2023-10-10T20:43:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.815+0000 7f6d33935700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c0ce3e0 con 0x7f6d2c159890 2023-10-10T20:43:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.815+0000 7f6d316d1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d2c069560 0x7f6d2c0ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.815+0000 7f6d316d1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d2c069560 0x7f6d2c0ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45068/0 (socket says 172.21.15.89:45068) 2023-10-10T20:43:47.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.815+0000 7f6d316d1700 1 -- 172.21.15.89:0/3727854921 learned_addr learned my addr 172.21.15.89:0/3727854921 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:47.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.815+0000 7f6d31ed2700 1 --2- 172.21.15.89:0/3727854921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d2c159c60 0x7f6d2c0cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.815+0000 7f6d30ed0700 1 --2- 172.21.15.89:0/3727854921 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d2c159890 0x7f6d2c0cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.815+0000 7f6d316d1700 1 -- 172.21.15.89:0/3727854921 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d2c159890 msgr2=0x7f6d2c0cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:47.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.815+0000 7f6d316d1700 1 --2- 172.21.15.89:0/3727854921 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d2c159890 0x7f6d2c0cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.815+0000 7f6d316d1700 1 -- 172.21.15.89:0/3727854921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d2c159c60 msgr2=0x7f6d2c0cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:47.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.815+0000 7f6d316d1700 1 --2- 172.21.15.89:0/3727854921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d2c159c60 0x7f6d2c0cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:47.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.816+0000 7f6d316d1700 1 -- 172.21.15.89:0/3727854921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d24009a20 con 0x7f6d2c069560 2023-10-10T20:43:47.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.816+0000 7f6d31ed2700 1 --2- 172.21.15.89:0/3727854921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d2c159c60 0x7f6d2c0cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:47.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.816+0000 7f6d316d1700 1 --2- 172.21.15.89:0/3727854921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d2c069560 0x7f6d2c0ceec0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009f80 tx=0x7f6d28017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:47.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.816+0000 7f6d217fa700 1 -- 172.21.15.89:0/3727854921 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2800bb20 con 0x7f6d2c069560 2023-10-10T20:43:47.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.816+0000 7f6d217fa700 1 -- 172.21.15.89:0/3727854921 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d28011420 con 0x7f6d2c069560 2023-10-10T20:43:47.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.816+0000 7f6d33935700 1 -- 172.21.15.89:0/3727854921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d28009420 con 0x7f6d2c069560 2023-10-10T20:43:47.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.816+0000 7f6d217fa700 1 -- 172.21.15.89:0/3727854921 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28011800 con 0x7f6d2c069560 2023-10-10T20:43:47.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.816+0000 7f6d33935700 1 -- 172.21.15.89:0/3727854921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d2c0ce9a0 con 0x7f6d2c069560 2023-10-10T20:43:47.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.817+0000 7f6d33935700 1 -- 172.21.15.89:0/3727854921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d2c04c3c0 con 0x7f6d2c069560 2023-10-10T20:43:47.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.817+0000 7f6d217fa700 1 -- 172.21.15.89:0/3727854921 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6d28011c30 con 0x7f6d2c069560 2023-10-10T20:43:47.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.818+0000 7f6d217fa700 1 --2- 172.21.15.89:0/3727854921 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6d10067320 0x7f6d100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:47.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.818+0000 7f6d217fa700 1 -- 172.21.15.89:0/3727854921 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f6d28086790 con 0x7f6d2c069560 2023-10-10T20:43:47.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.818+0000 7f6d30ed0700 1 --2- 172.21.15.89:0/3727854921 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6d10067320 0x7f6d100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:47.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.818+0000 7f6d30ed0700 1 --2- 172.21.15.89:0/3727854921 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6d10067320 0x7f6d100697d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c0b3ba0 tx=0x7f6d1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:47.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:47.822+0000 7f6d217fa700 1 -- 172.21.15.89:0/3727854921 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d280533f0 con 0x7f6d2c069560 2023-10-10T20:43:48.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.007+0000 7f6d33935700 1 -- 172.21.15.89:0/3727854921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d2c06a5b0 con 0x7f6d2c069560 2023-10-10T20:43:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.007+0000 7f6d217fa700 1 -- 172.21.15.89:0/3727854921 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+15442 (secure 0 0 0) 0x7f6d28056a10 con 0x7f6d2c069560 2023-10-10T20:43:48.009 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:48.010 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":186,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:43:43.464066+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs.volume_0000000000812094.meta","create_time":"2023-10-10T20:43:28.133902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000812094"}}},{"pool":32,"pool_name":"cephfs.volume_0000000000812094.data","create_time":"2023-10-10T20:43:28.749600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000812094"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:43:48.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.010+0000 7f6d33935700 1 -- 172.21.15.89:0/3727854921 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6d10067320 msgr2=0x7f6d100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:48.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.010+0000 7f6d33935700 1 --2- 172.21.15.89:0/3727854921 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6d10067320 0x7f6d100697d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c0b3ba0 tx=0x7f6d1c009040 comp rx=0 tx=0).stop 2023-10-10T20:43:48.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.010+0000 7f6d33935700 1 -- 172.21.15.89:0/3727854921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d2c069560 msgr2=0x7f6d2c0ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:48.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.010+0000 7f6d33935700 1 --2- 172.21.15.89:0/3727854921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d2c069560 0x7f6d2c0ceec0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009f80 tx=0x7f6d28017000 comp rx=0 tx=0).stop 2023-10-10T20:43:48.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.010+0000 7f6d33935700 1 -- 172.21.15.89:0/3727854921 shutdown_connections 2023-10-10T20:43:48.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.010+0000 7f6d33935700 1 --2- 172.21.15.89:0/3727854921 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6d10067320 0x7f6d100697d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.010+0000 7f6d33935700 1 --2- 172.21.15.89:0/3727854921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d2c159c60 0x7f6d2c0cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.010+0000 7f6d33935700 1 --2- 172.21.15.89:0/3727854921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d2c069560 0x7f6d2c0ceec0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.010+0000 7f6d33935700 1 --2- 172.21.15.89:0/3727854921 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6d2c159890 0x7f6d2c0cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.010+0000 7f6d33935700 1 -- 172.21.15.89:0/3727854921 >> 172.21.15.89:0/3727854921 conn(0x7f6d2c068e20 msgr2=0x7f6d2c15fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:48.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.010+0000 7f6d33935700 1 -- 172.21.15.89:0/3727854921 shutdown_connections 2023-10-10T20:43:48.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.010+0000 7f6d33935700 1 -- 172.21.15.89:0/3727854921 wait complete. 2023-10-10T20:43:48.024 INFO:tasks.cephfs.filesystem:Destroying file system volume_0000000000812094 and related pools 2023-10-10T20:43:48.024 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:43:48.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.140+0000 7f54bcf1e700 1 Processor -- start 2023-10-10T20:43:48.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.140+0000 7f54bcf1e700 1 -- start start 2023-10-10T20:43:48.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54bcf1e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54b8159890 0x7f54b815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54bcf1e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f54b8159c60 0x7f54b815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54bcf1e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f54b815db10 0x7f54b8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54bcf1e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b815a160 con 0x7f54b815db10 2023-10-10T20:43:48.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54bcf1e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b80d13a0 con 0x7f54b8159c60 2023-10-10T20:43:48.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54bcf1e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b815e140 con 0x7f54b8159890 2023-10-10T20:43:48.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54b6d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f54b815db10 0x7f54b8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54b6d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f54b815db10 0x7f54b8069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45070/0 (socket says 172.21.15.89:45070) 2023-10-10T20:43:48.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54b6d9d700 1 -- 172.21.15.89:0/112292048 learned_addr learned my addr 172.21.15.89:0/112292048 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:48.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54b5d9b700 1 --2- 172.21.15.89:0/112292048 >> v2:172.21.15.111:3300/0 conn(0x7f54b8159c60 0x7f54b815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54b659c700 1 --2- 172.21.15.89:0/112292048 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54b8159890 0x7f54b815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54b6d9d700 1 -- 172.21.15.89:0/112292048 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54b8159890 msgr2=0x7f54b815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:48.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54b6d9d700 1 --2- 172.21.15.89:0/112292048 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54b8159890 0x7f54b815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54b6d9d700 1 -- 172.21.15.89:0/112292048 >> v2:172.21.15.111:3300/0 conn(0x7f54b8159c60 msgr2=0x7f54b815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:48.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54b6d9d700 1 --2- 172.21.15.89:0/112292048 >> v2:172.21.15.111:3300/0 conn(0x7f54b8159c60 0x7f54b815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54b6d9d700 1 -- 172.21.15.89:0/112292048 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b815a3d0 con 0x7f54b815db10 2023-10-10T20:43:48.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54b659c700 1 --2- 172.21.15.89:0/112292048 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54b8159890 0x7f54b815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:48.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54b6d9d700 1 --2- 172.21.15.89:0/112292048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54b815db10 0x7f54b8069560 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f54b80650f0 tx=0x7f54b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f4973ae21a01f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:48.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54b559a700 1 -- 172.21.15.89:0/112292048 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b000e070 con 0x7f54b815db10 2023-10-10T20:43:48.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54b559a700 1 -- 172.21.15.89:0/112292048 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b000ba50 con 0x7f54b815db10 2023-10-10T20:43:48.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.141+0000 7f54b559a700 1 -- 172.21.15.89:0/112292048 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b00092e0 con 0x7f54b815db10 2023-10-10T20:43:48.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.142+0000 7f54bcf1e700 1 -- 172.21.15.89:0/112292048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54b815db10 msgr2=0x7f54b8069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:48.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.142+0000 7f54bcf1e700 1 --2- 172.21.15.89:0/112292048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54b815db10 0x7f54b8069560 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f54b80650f0 tx=0x7f54b0009840 comp rx=0 tx=0).stop 2023-10-10T20:43:48.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.142+0000 7f54bcf1e700 1 -- 172.21.15.89:0/112292048 shutdown_connections 2023-10-10T20:43:48.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.142+0000 7f54bcf1e700 1 --2- 172.21.15.89:0/112292048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54b815db10 0x7f54b8069560 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.142+0000 7f54bcf1e700 1 --2- 172.21.15.89:0/112292048 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54b8159890 0x7f54b815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.142+0000 7f54bcf1e700 1 --2- 172.21.15.89:0/112292048 >> v2:172.21.15.111:3300/0 conn(0x7f54b8159c60 0x7f54b815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.142+0000 7f54bcf1e700 1 -- 172.21.15.89:0/112292048 >> 172.21.15.89:0/112292048 conn(0x7f54b8068e20 msgr2=0x7f54b80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:48.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.142+0000 7f54bcf1e700 1 -- 172.21.15.89:0/112292048 shutdown_connections 2023-10-10T20:43:48.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.142+0000 7f54bcf1e700 1 -- 172.21.15.89:0/112292048 wait complete. 2023-10-10T20:43:48.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.143+0000 7f54bcf1e700 1 Processor -- start 2023-10-10T20:43:48.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.143+0000 7f54bcf1e700 1 -- start start 2023-10-10T20:43:48.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.144+0000 7f54bcf1e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54b8159890 0x7f54b8152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.144+0000 7f54bcf1e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54b8159c60 0x7f54b81582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.144+0000 7f54bcf1e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54b815db10 0x7f54b81587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.144+0000 7f54bcf1e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b80ce710 con 0x7f54b8159c60 2023-10-10T20:43:48.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.144+0000 7f54bcf1e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b80c8e70 con 0x7f54b8159890 2023-10-10T20:43:48.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.144+0000 7f54bcf1e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b80ce3e0 con 0x7f54b815db10 2023-10-10T20:43:48.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.144+0000 7f54b5d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54b8159c60 0x7f54b81582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.144+0000 7f54b5d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54b8159c60 0x7f54b81582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45080/0 (socket says 172.21.15.89:45080) 2023-10-10T20:43:48.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.144+0000 7f54b5d9b700 1 -- 172.21.15.89:0/3383034519 learned_addr learned my addr 172.21.15.89:0/3383034519 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:48.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.144+0000 7f54b659c700 1 --2- 172.21.15.89:0/3383034519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54b8159890 0x7f54b8152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.144+0000 7f54b6d9d700 1 --2- 172.21.15.89:0/3383034519 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54b815db10 0x7f54b81587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.144+0000 7f54b5d9b700 1 -- 172.21.15.89:0/3383034519 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54b815db10 msgr2=0x7f54b81587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:48.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.144+0000 7f54b5d9b700 1 --2- 172.21.15.89:0/3383034519 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54b815db10 0x7f54b81587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.144+0000 7f54b5d9b700 1 -- 172.21.15.89:0/3383034519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54b8159890 msgr2=0x7f54b8152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.144+0000 7f54b5d9b700 1 --2- 172.21.15.89:0/3383034519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54b8159890 0x7f54b8152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.144+0000 7f54b5d9b700 1 -- 172.21.15.89:0/3383034519 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b0009ae0 con 0x7f54b8159c60 2023-10-10T20:43:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.144+0000 7f54b5d9b700 1 --2- 172.21.15.89:0/3383034519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54b8159c60 0x7f54b81582b0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00d890 tx=0x7f54ac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.145+0000 7f54a67fc700 1 -- 172.21.15.89:0/3383034519 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac0125f0 con 0x7f54b8159c60 2023-10-10T20:43:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.145+0000 7f54a67fc700 1 -- 172.21.15.89:0/3383034519 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ac012750 con 0x7f54b8159c60 2023-10-10T20:43:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.145+0000 7f54bcf1e700 1 -- 172.21.15.89:0/3383034519 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b80c85d0 con 0x7f54b8159c60 2023-10-10T20:43:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.145+0000 7f54a67fc700 1 -- 172.21.15.89:0/3383034519 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac012af0 con 0x7f54b8159c60 2023-10-10T20:43:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.145+0000 7f54bcf1e700 1 -- 172.21.15.89:0/3383034519 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b80c8b90 con 0x7f54b8159c60 2023-10-10T20:43:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.145+0000 7f54bcf1e700 1 -- 172.21.15.89:0/3383034519 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b80ce880 con 0x7f54b8159c60 2023-10-10T20:43:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.146+0000 7f54a67fc700 1 -- 172.21.15.89:0/3383034519 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f54ac01e460 con 0x7f54b8159c60 2023-10-10T20:43:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.146+0000 7f54a67fc700 1 --2- 172.21.15.89:0/3383034519 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5498067320 0x7f54980697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.146+0000 7f54a67fc700 1 -- 172.21.15.89:0/3383034519 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f54ac0851f0 con 0x7f54b8159c60 2023-10-10T20:43:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.146+0000 7f54b659c700 1 --2- 172.21.15.89:0/3383034519 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5498067320 0x7f54980697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.147+0000 7f54b659c700 1 --2- 172.21.15.89:0/3383034519 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5498067320 0x7f54980697d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f54a800adc0 tx=0x7f54a8006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:48.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.151+0000 7f54a67fc700 1 -- 172.21.15.89:0/3383034519 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f54b80ce880 con 0x7f54b8159c60 2023-10-10T20:43:48.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.379+0000 7f54bcf1e700 1 -- 172.21.15.89:0/3383034519 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54b80ce850 con 0x7f54b8159c60 2023-10-10T20:43:48.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.379+0000 7f54a67fc700 1 -- 172.21.15.89:0/3383034519 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4627 (secure 0 0 0) 0x7f54ac0558c0 con 0x7f54b8159c60 2023-10-10T20:43:48.381 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:48.381 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":124,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/455398787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":455398787},{"type":"v1","addr":"172.21.15.89:6836","nonce":455398787}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/2660411678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2660411678},{"type":"v1","addr":"172.21.15.111:6834","nonce":2660411678}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6890,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/1650071647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":1650071647},{"type":"v1","addr":"172.21.15.111:6835","nonce":1650071647}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:43:29.795107+0000","modified":"2023-10-10T20:43:41.456555+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8760":{"gid":8760,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/2168500629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2168500629},{"type":"v1","addr":"172.21.15.89:6837","nonce":2168500629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"volume_0000000000812094","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-10T20:43:48.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.381+0000 7f54bcf1e700 1 -- 172.21.15.89:0/3383034519 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5498067320 msgr2=0x7f54980697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:48.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.381+0000 7f54bcf1e700 1 --2- 172.21.15.89:0/3383034519 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5498067320 0x7f54980697d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f54a800adc0 tx=0x7f54a8006f90 comp rx=0 tx=0).stop 2023-10-10T20:43:48.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.381+0000 7f54bcf1e700 1 -- 172.21.15.89:0/3383034519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54b8159c60 msgr2=0x7f54b81582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:48.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.381+0000 7f54bcf1e700 1 --2- 172.21.15.89:0/3383034519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54b8159c60 0x7f54b81582b0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00d890 tx=0x7f54ac008040 comp rx=0 tx=0).stop 2023-10-10T20:43:48.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.381+0000 7f54bcf1e700 1 -- 172.21.15.89:0/3383034519 shutdown_connections 2023-10-10T20:43:48.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.381+0000 7f54bcf1e700 1 --2- 172.21.15.89:0/3383034519 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f54b815db10 0x7f54b81587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.381+0000 7f54bcf1e700 1 --2- 172.21.15.89:0/3383034519 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5498067320 0x7f54980697d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.381+0000 7f54bcf1e700 1 --2- 172.21.15.89:0/3383034519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54b8159890 0x7f54b8152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.381+0000 7f54bcf1e700 1 --2- 172.21.15.89:0/3383034519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54b8159c60 0x7f54b81582b0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.381+0000 7f54bcf1e700 1 -- 172.21.15.89:0/3383034519 >> 172.21.15.89:0/3383034519 conn(0x7f54b8068e20 msgr2=0x7f54b815cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:48.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.381+0000 7f54bcf1e700 1 -- 172.21.15.89:0/3383034519 shutdown_connections 2023-10-10T20:43:48.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.381+0000 7f54bcf1e700 1 -- 172.21.15.89:0/3383034519 wait complete. 2023-10-10T20:43:48.386 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 124 2023-10-10T20:43:48.395 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:43:48.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.510+0000 7fcaf66cb700 1 Processor -- start 2023-10-10T20:43:48.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.510+0000 7fcaf66cb700 1 -- start start 2023-10-10T20:43:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.511+0000 7fcaf66cb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaf01597f0 0x7fcaf00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.511+0000 7fcaf66cb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf00b11d0 con 0x7fcaf015d230 2023-10-10T20:43:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.511+0000 7fcaf66cb700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf00b14b0 con 0x7fcaf0159420 2023-10-10T20:43:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.511+0000 7fcaf66cb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf00b1790 con 0x7fcaf01597f0 2023-10-10T20:43:48.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.511+0000 7fcaf5eca700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcaf015d230 0x7fcaf00b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35680/0 (socket says 172.21.15.89:35680) 2023-10-10T20:43:48.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.511+0000 7fcaf5eca700 1 -- 172.21.15.89:0/4078862722 learned_addr learned my addr 172.21.15.89:0/4078862722 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:48.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.511+0000 7fcaf4ec8700 1 --2- 172.21.15.89:0/4078862722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaf01597f0 0x7fcaf00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.511+0000 7fcae7fff700 1 -- 172.21.15.89:0/4078862722 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617528177 0 0) 0x7fcaf00b11d0 con 0x7fcaf015d230 2023-10-10T20:43:48.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.511+0000 7fcae7fff700 1 -- 172.21.15.89:0/4078862722 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcadc003570 con 0x7fcaf015d230 2023-10-10T20:43:48.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcae7fff700 1 -- 172.21.15.89:0/4078862722 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1287237111 0 0) 0x7fcadc003570 con 0x7fcaf015d230 2023-10-10T20:43:48.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcae7fff700 1 -- 172.21.15.89:0/4078862722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaf01597f0 msgr2=0x7fcaf00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:48.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcae7fff700 1 --2- 172.21.15.89:0/4078862722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaf01597f0 0x7fcaf00b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcae7fff700 1 -- 172.21.15.89:0/4078862722 >> v1:172.21.15.111:6789/0 conn(0x7fcaf0159420 legacy=0x7fcaf00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:48.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcae7fff700 1 -- 172.21.15.89:0/4078862722 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf00b1a70 con 0x7fcaf015d230 2023-10-10T20:43:48.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcae7fff700 1 -- 172.21.15.89:0/4078862722 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fcae8001470 con 0x7fcaf015d230 2023-10-10T20:43:48.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcae7fff700 1 -- 172.21.15.89:0/4078862722 >> v1:172.21.15.89:6789/0 conn(0x7fcaf015d230 legacy=0x7fcaf00b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:48.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcae7fff700 1 --2- 172.21.15.89:0/4078862722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcadc003a90 0x7fcadc003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcae7fff700 1 --2- 172.21.15.89:0/4078862722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaf01597f0 0x7fcadc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcae7fff700 1 --2- 172.21.15.89:0/4078862722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcadc004d50 0x7fcadc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcae7fff700 1 -- 172.21.15.89:0/4078862722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf00b1790 con 0x7fcadc004d50 2023-10-10T20:43:48.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcae7fff700 1 -- 172.21.15.89:0/4078862722 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc00bb10 con 0x7fcadc003a90 2023-10-10T20:43:48.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcae7fff700 1 -- 172.21.15.89:0/4078862722 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc00bcd0 con 0x7fcaf01597f0 2023-10-10T20:43:48.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcaf5eca700 1 --2- 172.21.15.89:0/4078862722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcadc004d50 0x7fcadc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcaf4ec8700 1 --2- 172.21.15.89:0/4078862722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaf01597f0 0x7fcadc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcaf56c9700 1 --2- 172.21.15.89:0/4078862722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcadc003a90 0x7fcadc003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcaf5eca700 1 -- 172.21.15.89:0/4078862722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaf01597f0 msgr2=0x7fcadc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:48.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcaf5eca700 1 --2- 172.21.15.89:0/4078862722 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaf01597f0 0x7fcadc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcaf5eca700 1 -- 172.21.15.89:0/4078862722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcadc003a90 msgr2=0x7fcadc003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:48.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcaf5eca700 1 --2- 172.21.15.89:0/4078862722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcadc003a90 0x7fcadc003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcaf5eca700 1 -- 172.21.15.89:0/4078862722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc003570 con 0x7fcadc004d50 2023-10-10T20:43:48.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcaf5eca700 1 --2- 172.21.15.89:0/4078862722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcadc004d50 0x7fcadc0051c0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fcae800c3c0 tx=0x7fcae8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa41d29ce5d5988 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:48.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcae7fff700 1 -- 172.21.15.89:0/4078862722 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae800cc90 con 0x7fcadc004d50 2023-10-10T20:43:48.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcae7fff700 1 -- 172.21.15.89:0/4078862722 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae800cdf0 con 0x7fcadc004d50 2023-10-10T20:43:48.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.512+0000 7fcae7fff700 1 -- 172.21.15.89:0/4078862722 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8005b70 con 0x7fcadc004d50 2023-10-10T20:43:48.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.513+0000 7fcaf66cb700 1 -- 172.21.15.89:0/4078862722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcadc004d50 msgr2=0x7fcadc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:48.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.513+0000 7fcaf66cb700 1 --2- 172.21.15.89:0/4078862722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcadc004d50 0x7fcadc0051c0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fcae800c3c0 tx=0x7fcae8001e70 comp rx=0 tx=0).stop 2023-10-10T20:43:48.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.513+0000 7fcaf5eca700 1 -- 172.21.15.89:0/4078862722 reap_dead start 2023-10-10T20:43:48.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.513+0000 7fcaf66cb700 1 -- 172.21.15.89:0/4078862722 shutdown_connections 2023-10-10T20:43:48.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.513+0000 7fcaf66cb700 1 -- 172.21.15.89:0/4078862722 >> 172.21.15.89:0/4078862722 conn(0x7fcaf00d0660 msgr2=0x7fcaf00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:48.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.513+0000 7fcaf66cb700 1 -- 172.21.15.89:0/4078862722 shutdown_connections 2023-10-10T20:43:48.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.513+0000 7fcaf66cb700 1 -- 172.21.15.89:0/4078862722 wait complete. 2023-10-10T20:43:48.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.514+0000 7fcaf66cb700 1 Processor -- start 2023-10-10T20:43:48.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.514+0000 7fcaf66cb700 1 -- start start 2023-10-10T20:43:48.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.515+0000 7fcaf66cb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcadc004d50 0x7fcaf00c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.515+0000 7fcaf66cb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaf015a590 0x7fcaf015aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.515+0000 7fcaf66cb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaf015aa00 0x7fcaf015b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.515+0000 7fcaf66cb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf015f160 con 0x7fcadc004d50 2023-10-10T20:43:48.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.515+0000 7fcaf66cb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf015f440 con 0x7fcaf015aa00 2023-10-10T20:43:48.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.515+0000 7fcaf66cb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf015f720 con 0x7fcaf015a590 2023-10-10T20:43:48.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.515+0000 7fcaf56c9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcadc004d50 0x7fcaf00c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.515+0000 7fcaf56c9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcadc004d50 0x7fcaf00c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46400/0 (socket says 172.21.15.89:46400) 2023-10-10T20:43:48.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.515+0000 7fcaf56c9700 1 -- 172.21.15.89:0/3725690005 learned_addr learned my addr 172.21.15.89:0/3725690005 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:48.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.515+0000 7fcaf5eca700 1 --2- 172.21.15.89:0/3725690005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaf015aa00 0x7fcaf015b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.515+0000 7fcaf4ec8700 1 --2- 172.21.15.89:0/3725690005 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaf015a590 0x7fcaf015aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.515+0000 7fcaf56c9700 1 -- 172.21.15.89:0/3725690005 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaf015a590 msgr2=0x7fcaf015aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:48.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.515+0000 7fcaf56c9700 1 --2- 172.21.15.89:0/3725690005 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaf015a590 0x7fcaf015aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.515+0000 7fcaf56c9700 1 -- 172.21.15.89:0/3725690005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaf015aa00 msgr2=0x7fcaf015b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:48.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.515+0000 7fcaf56c9700 1 --2- 172.21.15.89:0/3725690005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaf015aa00 0x7fcaf015b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.515+0000 7fcaf56c9700 1 -- 172.21.15.89:0/3725690005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf015fa00 con 0x7fcadc004d50 2023-10-10T20:43:48.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.515+0000 7fcaf4ec8700 1 --2- 172.21.15.89:0/3725690005 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaf015a590 0x7fcaf015aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:48.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.515+0000 7fcaf5eca700 1 --2- 172.21.15.89:0/3725690005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaf015aa00 0x7fcaf015b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.516+0000 7fcaf56c9700 1 --2- 172.21.15.89:0/3725690005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcadc004d50 0x7fcaf00c5980 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00eb90 tx=0x7fcaec0016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.516+0000 7fcae57fa700 1 -- 172.21.15.89:0/3725690005 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec0019e0 con 0x7fcadc004d50 2023-10-10T20:43:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.516+0000 7fcae57fa700 1 -- 172.21.15.89:0/3725690005 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaec001b40 con 0x7fcadc004d50 2023-10-10T20:43:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.516+0000 7fcaf66cb700 1 -- 172.21.15.89:0/3725690005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf015ed80 con 0x7fcadc004d50 2023-10-10T20:43:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.516+0000 7fcae57fa700 1 -- 172.21.15.89:0/3725690005 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec00c910 con 0x7fcadc004d50 2023-10-10T20:43:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.516+0000 7fcaf66cb700 1 -- 172.21.15.89:0/3725690005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf00c6080 con 0x7fcadc004d50 2023-10-10T20:43:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.517+0000 7fcaf66cb700 1 -- 172.21.15.89:0/3725690005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf004c3c0 con 0x7fcadc004d50 2023-10-10T20:43:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.517+0000 7fcae57fa700 1 -- 172.21.15.89:0/3725690005 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcaec00cd40 con 0x7fcadc004d50 2023-10-10T20:43:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.517+0000 7fcae57fa700 1 --2- 172.21.15.89:0/3725690005 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcad4067320 0x7fcad40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.517+0000 7fcae57fa700 1 -- 172.21.15.89:0/3725690005 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fcaec085300 con 0x7fcadc004d50 2023-10-10T20:43:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.517+0000 7fcaf4ec8700 1 --2- 172.21.15.89:0/3725690005 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcad4067320 0x7fcad40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.518+0000 7fcaf4ec8700 1 --2- 172.21.15.89:0/3725690005 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcad4067320 0x7fcad40697d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fcaf015d140 tx=0x7fcae0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:48.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.521+0000 7fcae57fa700 1 -- 172.21.15.89:0/3725690005 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcaec051f40 con 0x7fcadc004d50 2023-10-10T20:43:48.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.746+0000 7fcaf66cb700 1 -- 172.21.15.89:0/3725690005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcaf00c7770 con 0x7fcadc004d50 2023-10-10T20:43:48.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.747+0000 7fcae57fa700 1 -- 172.21.15.89:0/3725690005 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4627 (secure 0 0 0) 0x7fcaec055560 con 0x7fcadc004d50 2023-10-10T20:43:48.749 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:48.749 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":124,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/455398787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":455398787},{"type":"v1","addr":"172.21.15.89:6836","nonce":455398787}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/2660411678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2660411678},{"type":"v1","addr":"172.21.15.111:6834","nonce":2660411678}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6890,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/1650071647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":1650071647},{"type":"v1","addr":"172.21.15.111:6835","nonce":1650071647}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:43:29.795107+0000","modified":"2023-10-10T20:43:41.456555+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8760":{"gid":8760,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/2168500629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":2168500629},{"type":"v1","addr":"172.21.15.89:6837","nonce":2168500629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"volume_0000000000812094","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-10T20:43:48.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.749+0000 7fcaf66cb700 1 -- 172.21.15.89:0/3725690005 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcad4067320 msgr2=0x7fcad40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:48.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.749+0000 7fcaf66cb700 1 --2- 172.21.15.89:0/3725690005 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcad4067320 0x7fcad40697d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fcaf015d140 tx=0x7fcae0008040 comp rx=0 tx=0).stop 2023-10-10T20:43:48.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.749+0000 7fcaf66cb700 1 -- 172.21.15.89:0/3725690005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcadc004d50 msgr2=0x7fcaf00c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:48.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.749+0000 7fcaf66cb700 1 --2- 172.21.15.89:0/3725690005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcadc004d50 0x7fcaf00c5980 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00eb90 tx=0x7fcaec0016d0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.749+0000 7fcaf66cb700 1 -- 172.21.15.89:0/3725690005 shutdown_connections 2023-10-10T20:43:48.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.749+0000 7fcaf66cb700 1 --2- 172.21.15.89:0/3725690005 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcad4067320 0x7fcad40697d0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.749+0000 7fcaf66cb700 1 --2- 172.21.15.89:0/3725690005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaf015aa00 0x7fcaf015b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.749+0000 7fcaf66cb700 1 --2- 172.21.15.89:0/3725690005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcadc004d50 0x7fcaf00c5980 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.749+0000 7fcaf66cb700 1 --2- 172.21.15.89:0/3725690005 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcaf015a590 0x7fcaf015aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.749+0000 7fcaf66cb700 1 -- 172.21.15.89:0/3725690005 >> 172.21.15.89:0/3725690005 conn(0x7fcaf00d0660 msgr2=0x7fcaf00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:48.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.749+0000 7fcaf66cb700 1 -- 172.21.15.89:0/3725690005 shutdown_connections 2023-10-10T20:43:48.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.749+0000 7fcaf66cb700 1 -- 172.21.15.89:0/3725690005 wait complete. 2023-10-10T20:43:48.754 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 124 2023-10-10T20:43:48.763 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:43:48.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.876+0000 7f2668a4f700 1 Processor -- start 2023-10-10T20:43:48.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.876+0000 7f2668a4f700 1 -- start start 2023-10-10T20:43:48.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.877+0000 7f2668a4f700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f266406c0a0 0x7f26640caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.877+0000 7f2668a4f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f266415df50 0x7f266415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.877+0000 7f2668a4f700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266415d7e0 con 0x7f266406bcd0 2023-10-10T20:43:48.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.877+0000 7f2668a4f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266415dac0 con 0x7f266415df50 2023-10-10T20:43:48.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.877+0000 7f2668a4f700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266415aa30 con 0x7f266406c0a0 2023-10-10T20:43:48.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.877+0000 7f2662d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f266415df50 0x7f266415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.877+0000 7f2662d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f266415df50 0x7f266415e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46412/0 (socket says 172.21.15.89:46412) 2023-10-10T20:43:48.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.877+0000 7f2662d9d700 1 -- 172.21.15.89:0/1940993608 learned_addr learned my addr 172.21.15.89:0/1940993608 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:48.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.877+0000 7f2662d9d700 1 -- 172.21.15.89:0/1940993608 >> v2:172.21.15.111:3300/0 conn(0x7f266406c0a0 msgr2=0x7f26640caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:48.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.877+0000 7f2662d9d700 1 --2- 172.21.15.89:0/1940993608 >> v2:172.21.15.111:3300/0 conn(0x7f266406c0a0 0x7f26640caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.877+0000 7f2662d9d700 1 -- 172.21.15.89:0/1940993608 >> v1:172.21.15.111:6789/0 conn(0x7f266406bcd0 legacy=0x7f266415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:48.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.877+0000 7f2662d9d700 1 -- 172.21.15.89:0/1940993608 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266415ad10 con 0x7f266415df50 2023-10-10T20:43:48.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.877+0000 7f2662d9d700 1 --2- 172.21.15.89:0/1940993608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f266415df50 0x7f266415e3b0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f266406b4e0 tx=0x7f265c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e992770777430064 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:48.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.877+0000 7f266159a700 1 -- 172.21.15.89:0/1940993608 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c00e070 con 0x7f266415df50 2023-10-10T20:43:48.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.877+0000 7f266159a700 1 -- 172.21.15.89:0/1940993608 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265c00ba10 con 0x7f266415df50 2023-10-10T20:43:48.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.877+0000 7f266159a700 1 -- 172.21.15.89:0/1940993608 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c0092e0 con 0x7f266415df50 2023-10-10T20:43:48.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.878+0000 7f2668a4f700 1 -- 172.21.15.89:0/1940993608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f266415df50 msgr2=0x7f266415e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:48.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.878+0000 7f2668a4f700 1 --2- 172.21.15.89:0/1940993608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f266415df50 0x7f266415e3b0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f266406b4e0 tx=0x7f265c009840 comp rx=0 tx=0).stop 2023-10-10T20:43:48.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.878+0000 7f2668a4f700 1 -- 172.21.15.89:0/1940993608 shutdown_connections 2023-10-10T20:43:48.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.878+0000 7f2668a4f700 1 --2- 172.21.15.89:0/1940993608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f266415df50 0x7f266415e3b0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.878+0000 7f2668a4f700 1 --2- 172.21.15.89:0/1940993608 >> v2:172.21.15.111:3300/0 conn(0x7f266406c0a0 0x7f26640caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.878+0000 7f2668a4f700 1 -- 172.21.15.89:0/1940993608 >> 172.21.15.89:0/1940993608 conn(0x7f26640ce600 msgr2=0x7f26640cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:48.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.878+0000 7f2668a4f700 1 -- 172.21.15.89:0/1940993608 shutdown_connections 2023-10-10T20:43:48.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.878+0000 7f2668a4f700 1 -- 172.21.15.89:0/1940993608 wait complete. 2023-10-10T20:43:48.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.879+0000 7f2668a4f700 1 Processor -- start 2023-10-10T20:43:48.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.879+0000 7f2668a4f700 1 -- start start 2023-10-10T20:43:48.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f2668a4f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f266406bcd0 0x7f26640c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f2668a4f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f266406c0a0 0x7f266416aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f2668a4f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f266415df50 0x7f2664169630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f2668a4f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266416af40 con 0x7f266406c0a0 2023-10-10T20:43:48.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f2668a4f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26640b1fd0 con 0x7f266406bcd0 2023-10-10T20:43:48.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f2668a4f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26640c7c90 con 0x7f266415df50 2023-10-10T20:43:48.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f2661d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f266406c0a0 0x7f266416aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f2661d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f266406c0a0 0x7f266416aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46418/0 (socket says 172.21.15.89:46418) 2023-10-10T20:43:48.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f2661d9b700 1 -- 172.21.15.89:0/2662195707 learned_addr learned my addr 172.21.15.89:0/2662195707 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:48.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f2662d9d700 1 --2- 172.21.15.89:0/2662195707 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f266415df50 0x7f2664169630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f266259c700 1 --2- 172.21.15.89:0/2662195707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f266406bcd0 0x7f26640c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f2661d9b700 1 -- 172.21.15.89:0/2662195707 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f266415df50 msgr2=0x7f2664169630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:48.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f2661d9b700 1 --2- 172.21.15.89:0/2662195707 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f266415df50 0x7f2664169630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f2661d9b700 1 -- 172.21.15.89:0/2662195707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f266406bcd0 msgr2=0x7f26640c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:48.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f2661d9b700 1 --2- 172.21.15.89:0/2662195707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f266406bcd0 0x7f26640c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:48.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f2661d9b700 1 -- 172.21.15.89:0/2662195707 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265c009ae0 con 0x7f266406c0a0 2023-10-10T20:43:48.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f2661d9b700 1 --2- 172.21.15.89:0/2662195707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f266406c0a0 0x7f266416aa00 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f265800b810 tx=0x7f26580091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:48.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f26527fc700 1 -- 172.21.15.89:0/2662195707 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265800f570 con 0x7f266406c0a0 2023-10-10T20:43:48.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f26527fc700 1 -- 172.21.15.89:0/2662195707 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265800f6d0 con 0x7f266406c0a0 2023-10-10T20:43:48.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f2668a4f700 1 -- 172.21.15.89:0/2662195707 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26640c7f70 con 0x7f266406c0a0 2023-10-10T20:43:48.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f26527fc700 1 -- 172.21.15.89:0/2662195707 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265800fa90 con 0x7f266406c0a0 2023-10-10T20:43:48.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.880+0000 7f2668a4f700 1 -- 172.21.15.89:0/2662195707 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26640c8810 con 0x7f266406c0a0 2023-10-10T20:43:48.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.881+0000 7f2668a4f700 1 -- 172.21.15.89:0/2662195707 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26640c8360 con 0x7f266406c0a0 2023-10-10T20:43:48.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.882+0000 7f26527fc700 1 -- 172.21.15.89:0/2662195707 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f265801a460 con 0x7f266406c0a0 2023-10-10T20:43:48.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.882+0000 7f26527fc700 1 --2- 172.21.15.89:0/2662195707 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2644067320 0x7f26440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:48.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.882+0000 7f26527fc700 1 -- 172.21.15.89:0/2662195707 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f2658084970 con 0x7f266406c0a0 2023-10-10T20:43:48.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.882+0000 7f266259c700 1 --2- 172.21.15.89:0/2662195707 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2644067320 0x7f26440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:48.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.883+0000 7f266259c700 1 --2- 172.21.15.89:0/2662195707 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2644067320 0x7f26440697d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2654006fd0 tx=0x7f2654008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:48.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:48.887+0000 7f26527fc700 1 -- 172.21.15.89:0/2662195707 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2658051630 con 0x7f266406c0a0 2023-10-10T20:43:49.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.070+0000 7f2668a4f700 1 -- 172.21.15.89:0/2662195707 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f266415aff0 con 0x7f266406c0a0 2023-10-10T20:43:49.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.071+0000 7f26527fc700 1 -- 172.21.15.89:0/2662195707 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+15442 (secure 0 0 0) 0x7f2658054c50 con 0x7f266406c0a0 2023-10-10T20:43:49.073 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:49.073 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":186,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:43:43.464066+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs.volume_0000000000812094.meta","create_time":"2023-10-10T20:43:28.133902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000812094"}}},{"pool":32,"pool_name":"cephfs.volume_0000000000812094.data","create_time":"2023-10-10T20:43:28.749600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000812094"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:43:49.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.073+0000 7f2668a4f700 1 -- 172.21.15.89:0/2662195707 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2644067320 msgr2=0x7f26440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:49.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.073+0000 7f2668a4f700 1 --2- 172.21.15.89:0/2662195707 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2644067320 0x7f26440697d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2654006fd0 tx=0x7f2654008040 comp rx=0 tx=0).stop 2023-10-10T20:43:49.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.073+0000 7f2668a4f700 1 -- 172.21.15.89:0/2662195707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f266406c0a0 msgr2=0x7f266416aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:49.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.073+0000 7f2668a4f700 1 --2- 172.21.15.89:0/2662195707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f266406c0a0 0x7f266416aa00 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f265800b810 tx=0x7f26580091d0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.073+0000 7f2668a4f700 1 -- 172.21.15.89:0/2662195707 shutdown_connections 2023-10-10T20:43:49.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.073+0000 7f2668a4f700 1 --2- 172.21.15.89:0/2662195707 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f266415df50 0x7f2664169630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.073+0000 7f2668a4f700 1 --2- 172.21.15.89:0/2662195707 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2644067320 0x7f26440697d0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.073+0000 7f2668a4f700 1 --2- 172.21.15.89:0/2662195707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f266406bcd0 0x7f26640c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.073+0000 7f2668a4f700 1 --2- 172.21.15.89:0/2662195707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f266406c0a0 0x7f266416aa00 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.073+0000 7f2668a4f700 1 -- 172.21.15.89:0/2662195707 >> 172.21.15.89:0/2662195707 conn(0x7f26640ce600 msgr2=0x7f2664157430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:49.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.073+0000 7f2668a4f700 1 -- 172.21.15.89:0/2662195707 shutdown_connections 2023-10-10T20:43:49.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.073+0000 7f2668a4f700 1 -- 172.21.15.89:0/2662195707 wait complete. 2023-10-10T20:43:49.087 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail volume_0000000000812094 2023-10-10T20:43:49.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.200+0000 7f30c11fc700 1 Processor -- start 2023-10-10T20:43:49.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.200+0000 7f30c11fc700 1 -- start start 2023-10-10T20:43:49.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.201+0000 7f30c11fc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30bc159c60 0x7f30bc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.201+0000 7f30c11fc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30bc15a160 con 0x7f30bc15db10 2023-10-10T20:43:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.201+0000 7f30c11fc700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30bc0d13a0 con 0x7f30bc159890 2023-10-10T20:43:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.201+0000 7f30c11fc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc15e140 con 0x7f30bc159c60 2023-10-10T20:43:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.201+0000 7f30bb59e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f30bc15db10 0x7f30bc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35690/0 (socket says 172.21.15.89:35690) 2023-10-10T20:43:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.201+0000 7f30bb59e700 1 -- 172.21.15.89:0/320420645 learned_addr learned my addr 172.21.15.89:0/320420645 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:49.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30b259c700 1 --2- 172.21.15.89:0/320420645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30bc159c60 0x7f30bc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:49.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30ba59c700 1 -- 172.21.15.89:0/320420645 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3537249554 0 0) 0x7f30bc15a160 con 0x7f30bc15db10 2023-10-10T20:43:49.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30ba59c700 1 -- 172.21.15.89:0/320420645 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30a8003570 con 0x7f30bc15db10 2023-10-10T20:43:49.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30ba59c700 1 -- 172.21.15.89:0/320420645 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 970040990 0 0) 0x7f30a8003570 con 0x7f30bc15db10 2023-10-10T20:43:49.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30ba59c700 1 -- 172.21.15.89:0/320420645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30bc159c60 msgr2=0x7f30bc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30ba59c700 1 --2- 172.21.15.89:0/320420645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30bc159c60 0x7f30bc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30ba59c700 1 -- 172.21.15.89:0/320420645 >> v1:172.21.15.111:6789/0 conn(0x7f30bc159890 legacy=0x7f30bc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30ba59c700 1 -- 172.21.15.89:0/320420645 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc15a3d0 con 0x7f30bc15db10 2023-10-10T20:43:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30ba59c700 1 -- 172.21.15.89:0/320420645 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f30b40017f0 con 0x7f30bc15db10 2023-10-10T20:43:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30ba59c700 1 -- 172.21.15.89:0/320420645 >> v1:172.21.15.89:6789/0 conn(0x7f30bc15db10 legacy=0x7f30bc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30ba59c700 1 --2- 172.21.15.89:0/320420645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30a8003a90 0x7f30a8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30ba59c700 1 --2- 172.21.15.89:0/320420645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30a8004810 0x7f30a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30ba59c700 1 --2- 172.21.15.89:0/320420645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30bc159c60 0x7f30a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30ba59c700 1 -- 172.21.15.89:0/320420645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc15e140 con 0x7f30a8004810 2023-10-10T20:43:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30ba59c700 1 -- 172.21.15.89:0/320420645 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a800bb10 con 0x7f30a8003a90 2023-10-10T20:43:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30ba59c700 1 -- 172.21.15.89:0/320420645 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a800bcd0 con 0x7f30bc159c60 2023-10-10T20:43:49.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30b259c700 1 --2- 172.21.15.89:0/320420645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30a8004810 0x7f30a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:49.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30bad9d700 1 --2- 172.21.15.89:0/320420645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30a8003a90 0x7f30a8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:49.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30bb59e700 1 --2- 172.21.15.89:0/320420645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30bc159c60 0x7f30a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:49.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30b259c700 1 -- 172.21.15.89:0/320420645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30bc159c60 msgr2=0x7f30a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:49.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30b259c700 1 --2- 172.21.15.89:0/320420645 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30bc159c60 0x7f30a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30b259c700 1 -- 172.21.15.89:0/320420645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30a8003a90 msgr2=0x7f30a8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30b259c700 1 --2- 172.21.15.89:0/320420645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30a8003a90 0x7f30a8003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.202+0000 7f30b259c700 1 -- 172.21.15.89:0/320420645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a8003570 con 0x7f30a8004810 2023-10-10T20:43:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.203+0000 7f30b259c700 1 --2- 172.21.15.89:0/320420645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30a8004810 0x7f30a8004c80 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f30bc15a090 tx=0x7f30a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d086496effca51 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.203+0000 7f30ba59c700 1 -- 172.21.15.89:0/320420645 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a400e710 con 0x7f30a8004810 2023-10-10T20:43:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.203+0000 7f30ba59c700 1 -- 172.21.15.89:0/320420645 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a400e870 con 0x7f30a8004810 2023-10-10T20:43:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.203+0000 7f30ba59c700 1 -- 172.21.15.89:0/320420645 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a400ec30 con 0x7f30a8004810 2023-10-10T20:43:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.203+0000 7f30c11fc700 1 -- 172.21.15.89:0/320420645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30a8004810 msgr2=0x7f30a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.203+0000 7f30c11fc700 1 --2- 172.21.15.89:0/320420645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30a8004810 0x7f30a8004c80 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f30bc15a090 tx=0x7f30a40091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.203+0000 7f30bb59e700 1 -- 172.21.15.89:0/320420645 reap_dead start 2023-10-10T20:43:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.203+0000 7f30c11fc700 1 -- 172.21.15.89:0/320420645 shutdown_connections 2023-10-10T20:43:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.203+0000 7f30c11fc700 1 -- 172.21.15.89:0/320420645 >> 172.21.15.89:0/320420645 conn(0x7f30bc068e20 msgr2=0x7f30bc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.203+0000 7f30c11fc700 1 -- 172.21.15.89:0/320420645 shutdown_connections 2023-10-10T20:43:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.203+0000 7f30c11fc700 1 -- 172.21.15.89:0/320420645 wait complete. 2023-10-10T20:43:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.204+0000 7f30c11fc700 1 Processor -- start 2023-10-10T20:43:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.205+0000 7f30c11fc700 1 -- start start 2023-10-10T20:43:49.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.205+0000 7f30c11fc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30a8004810 0x7f30bc158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:49.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.205+0000 7f30c11fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30bc156520 0x7f30bc151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:49.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.205+0000 7f30c11fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30bc0cd2f0 0x7f30bc0cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:49.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.205+0000 7f30c11fc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc0cdca0 con 0x7f30bc0cd2f0 2023-10-10T20:43:49.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.205+0000 7f30c11fc700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc157910 con 0x7f30bc156520 2023-10-10T20:43:49.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.205+0000 7f30c11fc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc156d00 con 0x7f30a8004810 2023-10-10T20:43:49.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.205+0000 7f30bb59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30bc0cd2f0 0x7f30bc0cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:49.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.205+0000 7f30bb59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30bc0cd2f0 0x7f30bc0cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46432/0 (socket says 172.21.15.89:46432) 2023-10-10T20:43:49.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.205+0000 7f30bb59e700 1 -- 172.21.15.89:0/1938570992 learned_addr learned my addr 172.21.15.89:0/1938570992 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:49.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.206+0000 7f30b259c700 1 --2- 172.21.15.89:0/1938570992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30bc156520 0x7f30bc151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:49.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.206+0000 7f30bad9d700 1 --2- 172.21.15.89:0/1938570992 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30a8004810 0x7f30bc158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:49.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.206+0000 7f30bb59e700 1 -- 172.21.15.89:0/1938570992 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30a8004810 msgr2=0x7f30bc158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:49.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.206+0000 7f30bb59e700 1 --2- 172.21.15.89:0/1938570992 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30a8004810 0x7f30bc158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.206+0000 7f30bb59e700 1 -- 172.21.15.89:0/1938570992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30bc156520 msgr2=0x7f30bc151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:49.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.206+0000 7f30bb59e700 1 --2- 172.21.15.89:0/1938570992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30bc156520 0x7f30bc151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.206+0000 7f30bb59e700 1 -- 172.21.15.89:0/1938570992 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc156fe0 con 0x7f30bc0cd2f0 2023-10-10T20:43:49.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.206+0000 7f30b259c700 1 --2- 172.21.15.89:0/1938570992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30bc156520 0x7f30bc151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:49.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.206+0000 7f30bb59e700 1 --2- 172.21.15.89:0/1938570992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30bc0cd2f0 0x7f30bc0cd760 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f30bc15dee0 tx=0x7f30b40021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:49.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.206+0000 7f30b37fe700 1 -- 172.21.15.89:0/1938570992 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b400ad60 con 0x7f30bc0cd2f0 2023-10-10T20:43:49.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.206+0000 7f30b37fe700 1 -- 172.21.15.89:0/1938570992 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b40133f0 con 0x7f30bc0cd2f0 2023-10-10T20:43:49.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.206+0000 7f30c11fc700 1 -- 172.21.15.89:0/1938570992 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30bc1572c0 con 0x7f30bc0cd2f0 2023-10-10T20:43:49.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.206+0000 7f30b37fe700 1 -- 172.21.15.89:0/1938570992 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b40137b0 con 0x7f30bc0cd2f0 2023-10-10T20:43:49.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.206+0000 7f30c11fc700 1 -- 172.21.15.89:0/1938570992 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30bc156990 con 0x7f30bc0cd2f0 2023-10-10T20:43:49.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.207+0000 7f30c11fc700 1 -- 172.21.15.89:0/1938570992 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30bc157ac0 con 0x7f30bc0cd2f0 2023-10-10T20:43:49.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.207+0000 7f30b37fe700 1 -- 172.21.15.89:0/1938570992 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f30b4013be0 con 0x7f30bc0cd2f0 2023-10-10T20:43:49.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.208+0000 7f30b37fe700 1 --2- 172.21.15.89:0/1938570992 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f309c067320 0x7f309c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.208+0000 7f30b37fe700 1 -- 172.21.15.89:0/1938570992 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f30b4085810 con 0x7f30bc0cd2f0 2023-10-10T20:43:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.208+0000 7f30bad9d700 1 --2- 172.21.15.89:0/1938570992 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f309c067320 0x7f309c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.208+0000 7f30bad9d700 1 --2- 172.21.15.89:0/1938570992 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f309c067320 0x7f309c0697d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f30ac000c00 tx=0x7f30ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.212+0000 7f30b37fe700 1 -- 172.21.15.89:0/1938570992 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30bc157ac0 con 0x7f30bc0cd2f0 2023-10-10T20:43:49.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.437+0000 7f30c11fc700 1 -- 172.21.15.89:0/1938570992 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "volume_0000000000812094"} v 0) v1 -- 0x7f30bc157a80 con 0x7f30bc0cd2f0 2023-10-10T20:43:49.439 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-10-10T20:43:49.438+0000 7fd0184ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:43:49.444 INFO:tasks.ceph.mds.a.smithi089.stderr: -787> 2023-10-10T20:43:35.699+0000 7fcc07623700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:43:49.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.444+0000 7f30b37fe700 1 -- 172.21.15.89:0/1938570992 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "volume_0000000000812094"}]=0 volume_0000000000812094 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v125) v1 ==== 195+0+0 (secure 0 0 0) 0x7f30b4052450 con 0x7f30bc0cd2f0 2023-10-10T20:43:49.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.446+0000 7f30c11fc700 1 -- 172.21.15.89:0/1938570992 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f309c067320 msgr2=0x7f309c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:49.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.446+0000 7f30c11fc700 1 --2- 172.21.15.89:0/1938570992 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f309c067320 0x7f309c0697d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f30ac000c00 tx=0x7f30ac009040 comp rx=0 tx=0).stop 2023-10-10T20:43:49.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.446+0000 7f30c11fc700 1 -- 172.21.15.89:0/1938570992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30bc0cd2f0 msgr2=0x7f30bc0cd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:49.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.446+0000 7f30c11fc700 1 --2- 172.21.15.89:0/1938570992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30bc0cd2f0 0x7f30bc0cd760 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f30bc15dee0 tx=0x7f30b40021f0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.446+0000 7f30c11fc700 1 -- 172.21.15.89:0/1938570992 shutdown_connections 2023-10-10T20:43:49.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.446+0000 7f30c11fc700 1 --2- 172.21.15.89:0/1938570992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30bc0cd2f0 0x7f30bc0cd760 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.446+0000 7f30c11fc700 1 --2- 172.21.15.89:0/1938570992 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f30a8004810 0x7f30bc158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.446+0000 7f30c11fc700 1 --2- 172.21.15.89:0/1938570992 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f309c067320 0x7f309c0697d0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.446+0000 7f30c11fc700 1 --2- 172.21.15.89:0/1938570992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30bc156520 0x7f30bc151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.446+0000 7f30c11fc700 1 -- 172.21.15.89:0/1938570992 >> 172.21.15.89:0/1938570992 conn(0x7f30bc068e20 msgr2=0x7f30bc0b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:49.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.446+0000 7f30c11fc700 1 -- 172.21.15.89:0/1938570992 shutdown_connections 2023-10-10T20:43:49.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.446+0000 7f30c11fc700 1 -- 172.21.15.89:0/1938570992 wait complete. 2023-10-10T20:43:49.450 INFO:teuthology.orchestra.run.smithi089.stderr:volume_0000000000812094 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:43:49.461 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm volume_0000000000812094 --yes-i-really-mean-it 2023-10-10T20:43:49.464 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:43:49.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.577+0000 7fe78a83f700 1 Processor -- start 2023-10-10T20:43:49.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.577+0000 7fe78a83f700 1 -- start start 2023-10-10T20:43:49.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.578+0000 7fe78a83f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7840b16d0 0x7fe7840b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:49.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.578+0000 7fe78a83f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe7840b33b0 0x7fe7840b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:49.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.578+0000 7fe78a83f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7840b2080 con 0x7fe7840b4e70 2023-10-10T20:43:49.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.578+0000 7fe78a83f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7840cfb00 con 0x7fe7840b33b0 2023-10-10T20:43:49.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.578+0000 7fe78a83f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7840b5c40 con 0x7fe7840b16d0 2023-10-10T20:43:49.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.578+0000 7fe78a03e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe7840b33b0 0x7fe7840b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:49.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.578+0000 7fe78983d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe7840b4e70 0x7fe7840ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35698/0 (socket says 172.21.15.89:35698) 2023-10-10T20:43:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.578+0000 7fe78983d700 1 -- 172.21.15.89:0/3857906212 learned_addr learned my addr 172.21.15.89:0/3857906212 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.578+0000 7fe78a03e700 1 -- 172.21.15.89:0/3857906212 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7840b16d0 msgr2=0x7fe7840b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.578+0000 7fe78a03e700 1 --2- 172.21.15.89:0/3857906212 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7840b16d0 0x7fe7840b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.578+0000 7fe78a03e700 1 -- 172.21.15.89:0/3857906212 >> v1:172.21.15.89:6789/0 conn(0x7fe7840b4e70 legacy=0x7fe7840ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.578+0000 7fe78a03e700 1 -- 172.21.15.89:0/3857906212 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7840b5f20 con 0x7fe7840b33b0 2023-10-10T20:43:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.578+0000 7fe78883b700 1 -- 172.21.15.89:0/3857906212 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085276250 0 0) 0x7fe7840b2080 con 0x7fe7840b4e70 2023-10-10T20:43:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.578+0000 7fe78a03e700 1 --2- 172.21.15.89:0/3857906212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7840b33b0 0x7fe7840b3830 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fe77c009e60 tx=0x7fe77c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847f4d25f1491270 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.578+0000 7fe78883b700 1 -- 172.21.15.89:0/3857906212 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c00e070 con 0x7fe7840b33b0 2023-10-10T20:43:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.579+0000 7fe78883b700 1 -- 172.21.15.89:0/3857906212 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77c00bc30 con 0x7fe7840b33b0 2023-10-10T20:43:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.579+0000 7fe78883b700 1 -- 172.21.15.89:0/3857906212 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c012430 con 0x7fe7840b33b0 2023-10-10T20:43:49.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.579+0000 7fe78a83f700 1 -- 172.21.15.89:0/3857906212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7840b33b0 msgr2=0x7fe7840b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:49.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.579+0000 7fe78a83f700 1 --2- 172.21.15.89:0/3857906212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7840b33b0 0x7fe7840b3830 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fe77c009e60 tx=0x7fe77c009780 comp rx=0 tx=0).stop 2023-10-10T20:43:49.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.579+0000 7fe78a83f700 1 -- 172.21.15.89:0/3857906212 shutdown_connections 2023-10-10T20:43:49.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.579+0000 7fe78a83f700 1 --2- 172.21.15.89:0/3857906212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7840b33b0 0x7fe7840b3830 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.579+0000 7fe78a83f700 1 --2- 172.21.15.89:0/3857906212 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7840b16d0 0x7fe7840b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.579+0000 7fe78a83f700 1 -- 172.21.15.89:0/3857906212 >> 172.21.15.89:0/3857906212 conn(0x7fe7840d10c0 msgr2=0x7fe7840affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:49.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.579+0000 7fe78a83f700 1 -- 172.21.15.89:0/3857906212 shutdown_connections 2023-10-10T20:43:49.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.579+0000 7fe78a83f700 1 -- 172.21.15.89:0/3857906212 wait complete. 2023-10-10T20:43:49.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.580+0000 7fe78a83f700 1 Processor -- start 2023-10-10T20:43:49.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.580+0000 7fe78a83f700 1 -- start start 2023-10-10T20:43:49.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.581+0000 7fe78a83f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7840b16d0 0x7fe784158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:49.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.581+0000 7fe78a83f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7840b33b0 0x7fe7840c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:49.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.581+0000 7fe78a83f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7840b4e70 0x7fe7840c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:49.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.581+0000 7fe78a83f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe784158c40 con 0x7fe7840b16d0 2023-10-10T20:43:49.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.581+0000 7fe78a83f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7840c7fc0 con 0x7fe7840b33b0 2023-10-10T20:43:49.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.581+0000 7fe78a83f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7840c82a0 con 0x7fe7840b4e70 2023-10-10T20:43:49.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.581+0000 7fe78983d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7840b16d0 0x7fe784158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:49.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.581+0000 7fe78983d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7840b16d0 0x7fe784158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46476/0 (socket says 172.21.15.89:46476) 2023-10-10T20:43:49.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.581+0000 7fe78983d700 1 -- 172.21.15.89:0/4257172297 learned_addr learned my addr 172.21.15.89:0/4257172297 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:49.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.581+0000 7fe78a03e700 1 --2- 172.21.15.89:0/4257172297 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7840b4e70 0x7fe7840c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:49.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.581+0000 7fe78903c700 1 --2- 172.21.15.89:0/4257172297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7840b33b0 0x7fe7840c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:49.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.581+0000 7fe78983d700 1 -- 172.21.15.89:0/4257172297 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7840b4e70 msgr2=0x7fe7840c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:49.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.581+0000 7fe78983d700 1 --2- 172.21.15.89:0/4257172297 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7840b4e70 0x7fe7840c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.581+0000 7fe78983d700 1 -- 172.21.15.89:0/4257172297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7840b33b0 msgr2=0x7fe7840c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:49.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.581+0000 7fe78983d700 1 --2- 172.21.15.89:0/4257172297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7840b33b0 0x7fe7840c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:49.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.581+0000 7fe78983d700 1 -- 172.21.15.89:0/4257172297 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77c009a20 con 0x7fe7840b16d0 2023-10-10T20:43:49.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.581+0000 7fe78983d700 1 --2- 172.21.15.89:0/4257172297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7840b16d0 0x7fe784158700 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fe78000c7e0 tx=0x7fe780007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:49.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.581+0000 7fe779ffb700 1 -- 172.21.15.89:0/4257172297 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780005b30 con 0x7fe7840b16d0 2023-10-10T20:43:49.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.582+0000 7fe779ffb700 1 -- 172.21.15.89:0/4257172297 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe780005c90 con 0x7fe7840b16d0 2023-10-10T20:43:49.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.582+0000 7fe78a83f700 1 -- 172.21.15.89:0/4257172297 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7840c8580 con 0x7fe7840b16d0 2023-10-10T20:43:49.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.582+0000 7fe779ffb700 1 -- 172.21.15.89:0/4257172297 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780007300 con 0x7fe7840b16d0 2023-10-10T20:43:49.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.582+0000 7fe78a83f700 1 -- 172.21.15.89:0/4257172297 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7840c8b40 con 0x7fe7840b16d0 2023-10-10T20:43:49.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.583+0000 7fe78a83f700 1 -- 172.21.15.89:0/4257172297 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe784088990 con 0x7fe7840b16d0 2023-10-10T20:43:49.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.583+0000 7fe779ffb700 1 -- 172.21.15.89:0/4257172297 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe78001b400 con 0x7fe7840b16d0 2023-10-10T20:43:49.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.584+0000 7fe779ffb700 1 --2- 172.21.15.89:0/4257172297 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe768067320 0x7fe7680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:49.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.584+0000 7fe779ffb700 1 -- 172.21.15.89:0/4257172297 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5869+0+0 (secure 0 0 0) 0x7fe7800847a0 con 0x7fe7840b16d0 2023-10-10T20:43:49.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.584+0000 7fe78903c700 1 --2- 172.21.15.89:0/4257172297 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe768067320 0x7fe7680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:49.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.584+0000 7fe78903c700 1 --2- 172.21.15.89:0/4257172297 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe768067320 0x7fe7680697d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe784069130 tx=0x7fe774008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:49.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.588+0000 7fe779ffb700 1 -- 172.21.15.89:0/4257172297 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe780051490 con 0x7fe7840b16d0 2023-10-10T20:43:49.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:49.812+0000 7fe78a83f700 1 -- 172.21.15.89:0/4257172297 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "volume_0000000000812094", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe78404bb80 con 0x7fe7840b16d0 2023-10-10T20:43:50.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.442+0000 7fe779ffb700 1 -- 172.21.15.89:0/4257172297 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "volume_0000000000812094", "yes_i_really_mean_it": true}]=0 v126) v1 ==== 121+0+0 (secure 0 0 0) 0x7fe780054ab0 con 0x7fe7840b16d0 2023-10-10T20:43:50.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.445+0000 7fe78a83f700 1 -- 172.21.15.89:0/4257172297 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe768067320 msgr2=0x7fe7680697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:50.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.445+0000 7fe78a83f700 1 --2- 172.21.15.89:0/4257172297 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe768067320 0x7fe7680697d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe784069130 tx=0x7fe774008040 comp rx=0 tx=0).stop 2023-10-10T20:43:50.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.445+0000 7fe78a83f700 1 -- 172.21.15.89:0/4257172297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7840b16d0 msgr2=0x7fe784158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:50.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.445+0000 7fe78a83f700 1 --2- 172.21.15.89:0/4257172297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7840b16d0 0x7fe784158700 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fe78000c7e0 tx=0x7fe780007df0 comp rx=0 tx=0).stop 2023-10-10T20:43:50.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.445+0000 7fe78a83f700 1 -- 172.21.15.89:0/4257172297 shutdown_connections 2023-10-10T20:43:50.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.445+0000 7fe78a83f700 1 --2- 172.21.15.89:0/4257172297 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe7840b4e70 0x7fe7840c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:50.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.445+0000 7fe78a83f700 1 --2- 172.21.15.89:0/4257172297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7840b16d0 0x7fe784158700 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:50.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.445+0000 7fe78a83f700 1 --2- 172.21.15.89:0/4257172297 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe768067320 0x7fe7680697d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:50.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.445+0000 7fe78a83f700 1 --2- 172.21.15.89:0/4257172297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7840b33b0 0x7fe7840c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:50.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.445+0000 7fe78a83f700 1 -- 172.21.15.89:0/4257172297 >> 172.21.15.89:0/4257172297 conn(0x7fe7840d10c0 msgr2=0x7fe7841590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:50.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.445+0000 7fe78a83f700 1 -- 172.21.15.89:0/4257172297 shutdown_connections 2023-10-10T20:43:50.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.445+0000 7fe78a83f700 1 -- 172.21.15.89:0/4257172297 wait complete. 2023-10-10T20:43:50.459 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000812094.meta cephfs.volume_0000000000812094.meta --yes-i-really-really-mean-it 2023-10-10T20:43:50.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.614+0000 7f1b158cc700 1 Processor -- start 2023-10-10T20:43:50.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.614+0000 7f1b158cc700 1 -- start start 2023-10-10T20:43:50.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.615+0000 7f1b158cc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1b100b0790 0x7f1b100cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:50.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.615+0000 7f1b158cc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b101591e0 0x7f1b10159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:50.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.615+0000 7f1b158cc700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b100cfa00 con 0x7f1b100b03c0 2023-10-10T20:43:50.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.615+0000 7f1b158cc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b100cb580 con 0x7f1b100b0790 2023-10-10T20:43:50.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.615+0000 7f1b158cc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b100cf160 con 0x7f1b101591e0 2023-10-10T20:43:50.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.615+0000 7f1b07fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1b100b0790 0x7f1b100cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:50.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.615+0000 7f1b07fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1b100b0790 0x7f1b100cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46482/0 (socket says 172.21.15.89:46482) 2023-10-10T20:43:50.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.615+0000 7f1b07fff700 1 -- 172.21.15.89:0/2788063247 learned_addr learned my addr 172.21.15.89:0/2788063247 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:50.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.615+0000 7f1b150cb700 1 --2- 172.21.15.89:0/2788063247 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b101591e0 0x7f1b10159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:50.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.616+0000 7f1b07fff700 1 -- 172.21.15.89:0/2788063247 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b101591e0 msgr2=0x7f1b10159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:50.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.616+0000 7f1b07fff700 1 --2- 172.21.15.89:0/2788063247 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b101591e0 0x7f1b10159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:50.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.616+0000 7f1b07fff700 1 -- 172.21.15.89:0/2788063247 >> v1:172.21.15.111:6789/0 conn(0x7f1b100b03c0 legacy=0x7f1b100b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:50.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.616+0000 7f1b07fff700 1 -- 172.21.15.89:0/2788063247 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b100cf440 con 0x7f1b100b0790 2023-10-10T20:43:50.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.616+0000 7f1b07fff700 1 --2- 172.21.15.89:0/2788063247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b100b0790 0x7f1b100cea70 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f1b000099e0 tx=0x7f1b00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbff6442a82acdb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:50.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.616+0000 7f1b077fe700 1 -- 172.21.15.89:0/2788063247 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0000b820 con 0x7f1b100b0790 2023-10-10T20:43:50.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.616+0000 7f1b077fe700 1 -- 172.21.15.89:0/2788063247 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0000b980 con 0x7f1b100b0790 2023-10-10T20:43:50.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.616+0000 7f1b077fe700 1 -- 172.21.15.89:0/2788063247 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0000bd20 con 0x7f1b100b0790 2023-10-10T20:43:50.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.616+0000 7f1b158cc700 1 -- 172.21.15.89:0/2788063247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b100b0790 msgr2=0x7f1b100cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:50.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.616+0000 7f1b158cc700 1 --2- 172.21.15.89:0/2788063247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b100b0790 0x7f1b100cea70 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f1b000099e0 tx=0x7f1b00009250 comp rx=0 tx=0).stop 2023-10-10T20:43:50.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.616+0000 7f1b158cc700 1 -- 172.21.15.89:0/2788063247 shutdown_connections 2023-10-10T20:43:50.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.616+0000 7f1b158cc700 1 --2- 172.21.15.89:0/2788063247 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b101591e0 0x7f1b10159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:50.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.616+0000 7f1b158cc700 1 --2- 172.21.15.89:0/2788063247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b100b0790 0x7f1b100cea70 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:50.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.616+0000 7f1b158cc700 1 -- 172.21.15.89:0/2788063247 >> 172.21.15.89:0/2788063247 conn(0x7f1b10063230 msgr2=0x7f1b10063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:50.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.616+0000 7f1b158cc700 1 -- 172.21.15.89:0/2788063247 shutdown_connections 2023-10-10T20:43:50.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.616+0000 7f1b158cc700 1 -- 172.21.15.89:0/2788063247 wait complete. 2023-10-10T20:43:50.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.617+0000 7f1b158cc700 1 Processor -- start 2023-10-10T20:43:50.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.618+0000 7f1b158cc700 1 -- start start 2023-10-10T20:43:50.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.618+0000 7f1b158cc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b100b03c0 0x7f1b10156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:50.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.618+0000 7f1b158cc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b100b0790 0x7f1b100c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:50.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.618+0000 7f1b158cc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b101591e0 0x7f1b100c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:50.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.618+0000 7f1b158cc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b100c73f0 con 0x7f1b101591e0 2023-10-10T20:43:50.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.618+0000 7f1b158cc700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b100c9f30 con 0x7f1b100b0790 2023-10-10T20:43:50.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.618+0000 7f1b158cc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b10157ec0 con 0x7f1b100b03c0 2023-10-10T20:43:50.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.618+0000 7f1b150cb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b101591e0 0x7f1b100c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:50.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.619+0000 7f1b150cb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b101591e0 0x7f1b100c9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46486/0 (socket says 172.21.15.89:46486) 2023-10-10T20:43:50.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.619+0000 7f1b150cb700 1 -- 172.21.15.89:0/1584643142 learned_addr learned my addr 172.21.15.89:0/1584643142 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:50.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.619+0000 7f1b07fff700 1 --2- 172.21.15.89:0/1584643142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b100b0790 0x7f1b100c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:50.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.619+0000 7f1b148ca700 1 --2- 172.21.15.89:0/1584643142 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b100b03c0 0x7f1b10156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:50.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.619+0000 7f1b150cb700 1 -- 172.21.15.89:0/1584643142 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b100b03c0 msgr2=0x7f1b10156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:50.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.619+0000 7f1b150cb700 1 --2- 172.21.15.89:0/1584643142 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b100b03c0 0x7f1b10156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:50.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.619+0000 7f1b150cb700 1 -- 172.21.15.89:0/1584643142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b100b0790 msgr2=0x7f1b100c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:50.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.619+0000 7f1b150cb700 1 --2- 172.21.15.89:0/1584643142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b100b0790 0x7f1b100c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:50.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.619+0000 7f1b150cb700 1 -- 172.21.15.89:0/1584643142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b000094f0 con 0x7f1b101591e0 2023-10-10T20:43:50.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.619+0000 7f1b07fff700 1 --2- 172.21.15.89:0/1584643142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b100b0790 0x7f1b100c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:50.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.619+0000 7f1b150cb700 1 --2- 172.21.15.89:0/1584643142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b101591e0 0x7f1b100c9050 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009e90 tx=0x7f1b0800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:50.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.619+0000 7f1b04ff9700 1 -- 172.21.15.89:0/1584643142 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0800cc20 con 0x7f1b101591e0 2023-10-10T20:43:50.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.619+0000 7f1b04ff9700 1 -- 172.21.15.89:0/1584643142 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0800f040 con 0x7f1b101591e0 2023-10-10T20:43:50.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.619+0000 7f1b158cc700 1 -- 172.21.15.89:0/1584643142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b100ca3c0 con 0x7f1b101591e0 2023-10-10T20:43:50.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.619+0000 7f1b04ff9700 1 -- 172.21.15.89:0/1584643142 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08014480 con 0x7f1b101591e0 2023-10-10T20:43:50.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.619+0000 7f1b158cc700 1 -- 172.21.15.89:0/1584643142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b100c6e30 con 0x7f1b101591e0 2023-10-10T20:43:50.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.620+0000 7f1b158cc700 1 -- 172.21.15.89:0/1584643142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b100c9d00 con 0x7f1b101591e0 2023-10-10T20:43:50.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.620+0000 7f1b04ff9700 1 -- 172.21.15.89:0/1584643142 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1b08014830 con 0x7f1b101591e0 2023-10-10T20:43:50.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.621+0000 7f1b04ff9700 1 --2- 172.21.15.89:0/1584643142 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1afc067350 0x7f1afc069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:50.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.621+0000 7f1b148ca700 1 --2- 172.21.15.89:0/1584643142 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1afc067350 0x7f1afc069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:50.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.621+0000 7f1b04ff9700 1 -- 172.21.15.89:0/1584643142 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5869+0+0 (secure 0 0 0) 0x7f1b08085ad0 con 0x7f1b101591e0 2023-10-10T20:43:50.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.621+0000 7f1b148ca700 1 --2- 172.21.15.89:0/1584643142 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1afc067350 0x7f1afc069800 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1b100cdb90 tx=0x7f1b0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:50.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.625+0000 7f1b04ff9700 1 -- 172.21.15.89:0/1584643142 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b100c9d00 con 0x7f1b101591e0 2023-10-10T20:43:50.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:50.807+0000 7f1b158cc700 1 -- 172.21.15.89:0/1584643142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000812094.meta", "pool2": "cephfs.volume_0000000000812094.meta", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1b100c9d00 con 0x7f1b101591e0 2023-10-10T20:43:51.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.512+0000 7f1b04ff9700 1 -- 172.21.15.89:0/1584643142 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000812094.meta", "pool2": "cephfs.volume_0000000000812094.meta", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000812094.meta' removed v188) v1 ==== 241+0+0 (secure 0 0 0) 0x7f1b100c9d00 con 0x7f1b101591e0 2023-10-10T20:43:51.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.514+0000 7f1b158cc700 1 -- 172.21.15.89:0/1584643142 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1afc067350 msgr2=0x7f1afc069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:51.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.514+0000 7f1b158cc700 1 --2- 172.21.15.89:0/1584643142 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1afc067350 0x7f1afc069800 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1b100cdb90 tx=0x7f1b0c009040 comp rx=0 tx=0).stop 2023-10-10T20:43:51.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.515+0000 7f1b158cc700 1 -- 172.21.15.89:0/1584643142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b101591e0 msgr2=0x7f1b100c9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:51.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.515+0000 7f1b158cc700 1 --2- 172.21.15.89:0/1584643142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b101591e0 0x7f1b100c9050 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009e90 tx=0x7f1b0800b040 comp rx=0 tx=0).stop 2023-10-10T20:43:51.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.515+0000 7f1b158cc700 1 -- 172.21.15.89:0/1584643142 shutdown_connections 2023-10-10T20:43:51.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.515+0000 7f1b158cc700 1 --2- 172.21.15.89:0/1584643142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b101591e0 0x7f1b100c9050 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:51.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.515+0000 7f1b158cc700 1 --2- 172.21.15.89:0/1584643142 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b100b03c0 0x7f1b10156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:51.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.515+0000 7f1b158cc700 1 --2- 172.21.15.89:0/1584643142 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1afc067350 0x7f1afc069800 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:51.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.515+0000 7f1b158cc700 1 --2- 172.21.15.89:0/1584643142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b100b0790 0x7f1b100c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:51.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.515+0000 7f1b158cc700 1 -- 172.21.15.89:0/1584643142 >> 172.21.15.89:0/1584643142 conn(0x7f1b10063230 msgr2=0x7f1b100ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:51.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.515+0000 7f1b158cc700 1 -- 172.21.15.89:0/1584643142 shutdown_connections 2023-10-10T20:43:51.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.515+0000 7f1b158cc700 1 -- 172.21.15.89:0/1584643142 wait complete. 2023-10-10T20:43:51.519 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs.volume_0000000000812094.meta' removed 2023-10-10T20:43:51.529 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000812094.data cephfs.volume_0000000000812094.data --yes-i-really-really-mean-it 2023-10-10T20:43:51.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.648+0000 7fc03e5a8700 1 Processor -- start 2023-10-10T20:43:51.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.648+0000 7fc03e5a8700 1 -- start start 2023-10-10T20:43:51.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.648+0000 7fc03e5a8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc038159ec0 0x7fc0380b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:51.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.648+0000 7fc03e5a8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0380cf350 0x7fc0380cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:51.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.648+0000 7fc03e5a8700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0380b5b00 con 0x7fc0380b53d0 2023-10-10T20:43:51.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.648+0000 7fc03e5a8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0380b5850 con 0x7fc038159ec0 2023-10-10T20:43:51.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.648+0000 7fc03e5a8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380b5da0 con 0x7fc0380cf350 2023-10-10T20:43:51.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.649+0000 7fc037fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc038159ec0 0x7fc0380b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:51.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.649+0000 7fc037fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc038159ec0 0x7fc0380b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46498/0 (socket says 172.21.15.89:46498) 2023-10-10T20:43:51.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.649+0000 7fc037fff700 1 -- 172.21.15.89:0/1936292807 learned_addr learned my addr 172.21.15.89:0/1936292807 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:51.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.649+0000 7fc03cb45700 1 --2- 172.21.15.89:0/1936292807 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0380cf350 0x7fc0380cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:51.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.649+0000 7fc037fff700 1 -- 172.21.15.89:0/1936292807 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0380cf350 msgr2=0x7fc0380cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:51.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.649+0000 7fc037fff700 1 --2- 172.21.15.89:0/1936292807 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0380cf350 0x7fc0380cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:51.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.649+0000 7fc037fff700 1 -- 172.21.15.89:0/1936292807 >> v1:172.21.15.111:6789/0 conn(0x7fc0380b53d0 legacy=0x7fc0380cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:51.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.649+0000 7fc037fff700 1 -- 172.21.15.89:0/1936292807 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0380b6080 con 0x7fc038159ec0 2023-10-10T20:43:51.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.649+0000 7fc037fff700 1 --2- 172.21.15.89:0/1936292807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc038159ec0 0x7fc0380b4e90 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fc028009990 tx=0x7fc028009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decb96bd950041f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:51.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.649+0000 7fc036ffd700 1 -- 172.21.15.89:0/1936292807 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028005c10 con 0x7fc038159ec0 2023-10-10T20:43:51.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.649+0000 7fc036ffd700 1 -- 172.21.15.89:0/1936292807 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc028005d70 con 0x7fc038159ec0 2023-10-10T20:43:51.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.649+0000 7fc036ffd700 1 -- 172.21.15.89:0/1936292807 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02800b880 con 0x7fc038159ec0 2023-10-10T20:43:51.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.650+0000 7fc03e5a8700 1 -- 172.21.15.89:0/1936292807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc038159ec0 msgr2=0x7fc0380b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:51.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.650+0000 7fc03e5a8700 1 --2- 172.21.15.89:0/1936292807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc038159ec0 0x7fc0380b4e90 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fc028009990 tx=0x7fc028009220 comp rx=0 tx=0).stop 2023-10-10T20:43:51.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.650+0000 7fc03e5a8700 1 -- 172.21.15.89:0/1936292807 shutdown_connections 2023-10-10T20:43:51.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.650+0000 7fc03e5a8700 1 --2- 172.21.15.89:0/1936292807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc038159ec0 0x7fc0380b4e90 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:51.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.650+0000 7fc03e5a8700 1 --2- 172.21.15.89:0/1936292807 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0380cf350 0x7fc0380cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:51.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.650+0000 7fc03e5a8700 1 -- 172.21.15.89:0/1936292807 >> 172.21.15.89:0/1936292807 conn(0x7fc0380999f0 msgr2=0x7fc0380b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:51.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.650+0000 7fc03e5a8700 1 -- 172.21.15.89:0/1936292807 shutdown_connections 2023-10-10T20:43:51.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.650+0000 7fc03e5a8700 1 -- 172.21.15.89:0/1936292807 wait complete. 2023-10-10T20:43:51.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.651+0000 7fc03e5a8700 1 Processor -- start 2023-10-10T20:43:51.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.651+0000 7fc03e5a8700 1 -- start start 2023-10-10T20:43:51.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc03e5a8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0380b53d0 0x7fc038152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:51.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc03e5a8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0380cf350 0x7fc0380c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:51.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc03e5a8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc038159ec0 0x7fc0380c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:51.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc03e5a8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc038154c20 con 0x7fc0380cf350 2023-10-10T20:43:51.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc03e5a8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc038154f00 con 0x7fc038159ec0 2023-10-10T20:43:51.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc03e5a8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0381551e0 con 0x7fc0380b53d0 2023-10-10T20:43:51.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc0377fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0380cf350 0x7fc0380c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:51.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc0377fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0380cf350 0x7fc0380c98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46506/0 (socket says 172.21.15.89:46506) 2023-10-10T20:43:51.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc0377fe700 1 -- 172.21.15.89:0/3787148308 learned_addr learned my addr 172.21.15.89:0/3787148308 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:51.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc037fff700 1 --2- 172.21.15.89:0/3787148308 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0380b53d0 0x7fc038152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:51.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc03cb45700 1 --2- 172.21.15.89:0/3787148308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc038159ec0 0x7fc0380c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:51.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc0377fe700 1 -- 172.21.15.89:0/3787148308 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0380b53d0 msgr2=0x7fc038152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:51.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc0377fe700 1 --2- 172.21.15.89:0/3787148308 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0380b53d0 0x7fc038152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:51.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc0377fe700 1 -- 172.21.15.89:0/3787148308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc038159ec0 msgr2=0x7fc0380c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:51.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc0377fe700 1 --2- 172.21.15.89:0/3787148308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc038159ec0 0x7fc0380c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:51.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc0377fe700 1 -- 172.21.15.89:0/3787148308 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc028009480 con 0x7fc0380cf350 2023-10-10T20:43:51.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc0377fe700 1 --2- 172.21.15.89:0/3787148308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0380cf350 0x7fc0380c98c0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00c4e0 tx=0x7fc02c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:51.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc01ffff700 1 -- 172.21.15.89:0/3787148308 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c011590 con 0x7fc0380cf350 2023-10-10T20:43:51.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc01ffff700 1 -- 172.21.15.89:0/3787148308 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c011710 con 0x7fc0380cf350 2023-10-10T20:43:51.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc03e5a8700 1 -- 172.21.15.89:0/3787148308 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0381554c0 con 0x7fc0380cf350 2023-10-10T20:43:51.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc01ffff700 1 -- 172.21.15.89:0/3787148308 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c011ad0 con 0x7fc0380cf350 2023-10-10T20:43:51.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.652+0000 7fc03e5a8700 1 -- 172.21.15.89:0/3787148308 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc038153350 con 0x7fc0380cf350 2023-10-10T20:43:51.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.653+0000 7fc03e5a8700 1 -- 172.21.15.89:0/3787148308 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0381559e0 con 0x7fc0380cf350 2023-10-10T20:43:51.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.654+0000 7fc01ffff700 1 -- 172.21.15.89:0/3787148308 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc02c01b3b0 con 0x7fc0380cf350 2023-10-10T20:43:51.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.654+0000 7fc01ffff700 1 --2- 172.21.15.89:0/3787148308 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc024067320 0x7fc0240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:51.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.654+0000 7fc01ffff700 1 -- 172.21.15.89:0/3787148308 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5394+0+0 (secure 0 0 0) 0x7fc02c083fb0 con 0x7fc0380cf350 2023-10-10T20:43:51.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.654+0000 7fc037fff700 1 --2- 172.21.15.89:0/3787148308 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc024067320 0x7fc0240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:51.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.655+0000 7fc037fff700 1 --2- 172.21.15.89:0/3787148308 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc024067320 0x7fc0240697d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc028009260 tx=0x7fc028016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:51.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.658+0000 7fc01ffff700 1 -- 172.21.15.89:0/3787148308 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0381559e0 con 0x7fc0380cf350 2023-10-10T20:43:51.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:51.841+0000 7fc03e5a8700 1 -- 172.21.15.89:0/3787148308 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000812094.data", "pool2": "cephfs.volume_0000000000812094.data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc0381559e0 con 0x7fc0380cf350 2023-10-10T20:43:52.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.514+0000 7fc01ffff700 1 -- 172.21.15.89:0/3787148308 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000812094.data", "pool2": "cephfs.volume_0000000000812094.data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000812094.data' removed v189) v1 ==== 241+0+0 (secure 0 0 0) 0x7fc0381559e0 con 0x7fc0380cf350 2023-10-10T20:43:52.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.520+0000 7fc03e5a8700 1 -- 172.21.15.89:0/3787148308 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc024067320 msgr2=0x7fc0240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:52.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.520+0000 7fc03e5a8700 1 --2- 172.21.15.89:0/3787148308 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc024067320 0x7fc0240697d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc028009260 tx=0x7fc028016000 comp rx=0 tx=0).stop 2023-10-10T20:43:52.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.520+0000 7fc03e5a8700 1 -- 172.21.15.89:0/3787148308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0380cf350 msgr2=0x7fc0380c98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:52.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.520+0000 7fc03e5a8700 1 --2- 172.21.15.89:0/3787148308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0380cf350 0x7fc0380c98c0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00c4e0 tx=0x7fc02c00c7f0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.520+0000 7fc03e5a8700 1 -- 172.21.15.89:0/3787148308 shutdown_connections 2023-10-10T20:43:52.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.520+0000 7fc03e5a8700 1 --2- 172.21.15.89:0/3787148308 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc024067320 0x7fc0240697d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.520+0000 7fc03e5a8700 1 --2- 172.21.15.89:0/3787148308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc038159ec0 0x7fc0380c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.520+0000 7fc03e5a8700 1 --2- 172.21.15.89:0/3787148308 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0380b53d0 0x7fc038152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.520+0000 7fc03e5a8700 1 --2- 172.21.15.89:0/3787148308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0380cf350 0x7fc0380c98c0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.520+0000 7fc03e5a8700 1 -- 172.21.15.89:0/3787148308 >> 172.21.15.89:0/3787148308 conn(0x7fc0380999f0 msgr2=0x7fc038152250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:52.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.521+0000 7fc03e5a8700 1 -- 172.21.15.89:0/3787148308 shutdown_connections 2023-10-10T20:43:52.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.521+0000 7fc03e5a8700 1 -- 172.21.15.89:0/3787148308 wait complete. 2023-10-10T20:43:52.526 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs.volume_0000000000812094.data' removed 2023-10-10T20:43:52.537 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:43:52.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.657+0000 7f213a634700 1 Processor -- start 2023-10-10T20:43:52.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.657+0000 7f213a634700 1 -- start start 2023-10-10T20:43:52.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.658+0000 7f213a634700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f21340b4e70 0x7f21340ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:52.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.658+0000 7f213a634700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f21340b1b40 0x7f213415d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:52.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.658+0000 7f213a634700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21340cfb00 con 0x7f21340b16d0 2023-10-10T20:43:52.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.658+0000 7f213a634700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21340b5c40 con 0x7f21340b4e70 2023-10-10T20:43:52.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.658+0000 7f213a634700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21340b5f20 con 0x7f21340b1b40 2023-10-10T20:43:52.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.658+0000 7f2138e31700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f21340b16d0 0x7f213415d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35700/0 (socket says 172.21.15.89:35700) 2023-10-10T20:43:52.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.658+0000 7f2138e31700 1 -- 172.21.15.89:0/2413551891 learned_addr learned my addr 172.21.15.89:0/2413551891 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:52.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.658+0000 7f2139632700 1 --2- 172.21.15.89:0/2413551891 >> v2:172.21.15.111:3300/0 conn(0x7f21340b4e70 0x7f21340ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:52.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.658+0000 7f2139e33700 1 --2- 172.21.15.89:0/2413551891 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f21340b1b40 0x7f213415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:52.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.658+0000 7f212bfff700 1 -- 172.21.15.89:0/2413551891 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041012664 0 0) 0x7f21340cfb00 con 0x7f21340b16d0 2023-10-10T20:43:52.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.658+0000 7f212bfff700 1 -- 172.21.15.89:0/2413551891 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2120003570 con 0x7f21340b16d0 2023-10-10T20:43:52.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f212bfff700 1 -- 172.21.15.89:0/2413551891 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4018381165 0 0) 0x7f2120003570 con 0x7f21340b16d0 2023-10-10T20:43:52.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f212bfff700 1 -- 172.21.15.89:0/2413551891 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f21340b1b40 msgr2=0x7f213415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:52.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f212bfff700 1 --2- 172.21.15.89:0/2413551891 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f21340b1b40 0x7f213415d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f212bfff700 1 -- 172.21.15.89:0/2413551891 >> v2:172.21.15.111:3300/0 conn(0x7f21340b4e70 msgr2=0x7f21340ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:52.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f212bfff700 1 --2- 172.21.15.89:0/2413551891 >> v2:172.21.15.111:3300/0 conn(0x7f21340b4e70 0x7f21340ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f212bfff700 1 -- 172.21.15.89:0/2413551891 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21340b6200 con 0x7f21340b16d0 2023-10-10T20:43:52.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f212bfff700 1 -- 172.21.15.89:0/2413551891 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f2124001470 con 0x7f21340b16d0 2023-10-10T20:43:52.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f212bfff700 1 -- 172.21.15.89:0/2413551891 >> v1:172.21.15.89:6789/0 conn(0x7f21340b16d0 legacy=0x7f213415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:52.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f212bfff700 1 --2- 172.21.15.89:0/2413551891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2120003a60 0x7f2120003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:52.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f212bfff700 1 --2- 172.21.15.89:0/2413551891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2120004810 0x7f2120004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:52.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f212bfff700 1 --2- 172.21.15.89:0/2413551891 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f21340b1b40 0x7f21200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:52.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f212bfff700 1 -- 172.21.15.89:0/2413551891 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21340b5c40 con 0x7f2120004810 2023-10-10T20:43:52.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f212bfff700 1 -- 172.21.15.89:0/2413551891 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21340b5f20 con 0x7f2120003a60 2023-10-10T20:43:52.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f212bfff700 1 -- 172.21.15.89:0/2413551891 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f212000ba60 con 0x7f21340b1b40 2023-10-10T20:43:52.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f2138e31700 1 --2- 172.21.15.89:0/2413551891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2120004810 0x7f2120004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:52.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f2139632700 1 --2- 172.21.15.89:0/2413551891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2120003a60 0x7f2120003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:52.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f2139e33700 1 --2- 172.21.15.89:0/2413551891 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f21340b1b40 0x7f21200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:52.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f2138e31700 1 -- 172.21.15.89:0/2413551891 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f21340b1b40 msgr2=0x7f21200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:52.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f2138e31700 1 --2- 172.21.15.89:0/2413551891 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f21340b1b40 0x7f21200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f2138e31700 1 -- 172.21.15.89:0/2413551891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2120003a60 msgr2=0x7f2120003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:52.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f2138e31700 1 --2- 172.21.15.89:0/2413551891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2120003a60 0x7f2120003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f2138e31700 1 -- 172.21.15.89:0/2413551891 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2120003570 con 0x7f2120004810 2023-10-10T20:43:52.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f2138e31700 1 --2- 172.21.15.89:0/2413551891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2120004810 0x7f2120004c80 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f212400c3f0 tx=0x7f2124001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c7c8f1ef97be4d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:52.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f212bfff700 1 -- 172.21.15.89:0/2413551891 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212400cc80 con 0x7f2120004810 2023-10-10T20:43:52.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f212bfff700 1 -- 172.21.15.89:0/2413551891 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212400ce00 con 0x7f2120004810 2023-10-10T20:43:52.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.659+0000 7f212bfff700 1 -- 172.21.15.89:0/2413551891 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124005b70 con 0x7f2120004810 2023-10-10T20:43:52.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.660+0000 7f213a634700 1 -- 172.21.15.89:0/2413551891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2120004810 msgr2=0x7f2120004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:52.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.660+0000 7f213a634700 1 --2- 172.21.15.89:0/2413551891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2120004810 0x7f2120004c80 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f212400c3f0 tx=0x7f2124001e90 comp rx=0 tx=0).stop 2023-10-10T20:43:52.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.660+0000 7f2139e33700 1 -- 172.21.15.89:0/2413551891 reap_dead start 2023-10-10T20:43:52.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.660+0000 7f213a634700 1 -- 172.21.15.89:0/2413551891 shutdown_connections 2023-10-10T20:43:52.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.660+0000 7f213a634700 1 -- 172.21.15.89:0/2413551891 >> 172.21.15.89:0/2413551891 conn(0x7f21340d10c0 msgr2=0x7f21340affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:52.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.660+0000 7f213a634700 1 -- 172.21.15.89:0/2413551891 shutdown_connections 2023-10-10T20:43:52.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.660+0000 7f213a634700 1 -- 172.21.15.89:0/2413551891 wait complete. 2023-10-10T20:43:52.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.661+0000 7f213a634700 1 Processor -- start 2023-10-10T20:43:52.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.661+0000 7f213a634700 1 -- start start 2023-10-10T20:43:52.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.662+0000 7f213a634700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2120004810 0x7f2134148090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:52.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.662+0000 7f213a634700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f213415bbc0 0x7f213415c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:52.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.662+0000 7f213a634700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21340cb900 0x7f21340cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:52.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.662+0000 7f213a634700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213415c570 con 0x7f2120004810 2023-10-10T20:43:52.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.662+0000 7f213a634700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21340c7930 con 0x7f21340cb900 2023-10-10T20:43:52.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.662+0000 7f213a634700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21341485d0 con 0x7f213415bbc0 2023-10-10T20:43:52.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.662+0000 7f2139632700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2120004810 0x7f2134148090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:52.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.662+0000 7f2139632700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2120004810 0x7f2134148090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46514/0 (socket says 172.21.15.89:46514) 2023-10-10T20:43:52.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.662+0000 7f2139632700 1 -- 172.21.15.89:0/1931461580 learned_addr learned my addr 172.21.15.89:0/1931461580 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:52.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.662+0000 7f2139e33700 1 --2- 172.21.15.89:0/1931461580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21340cb900 0x7f21340cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:52.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.662+0000 7f2138e31700 1 --2- 172.21.15.89:0/1931461580 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f213415bbc0 0x7f213415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:52.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.662+0000 7f2139632700 1 -- 172.21.15.89:0/1931461580 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f213415bbc0 msgr2=0x7f213415c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:52.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.662+0000 7f2139632700 1 --2- 172.21.15.89:0/1931461580 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f213415bbc0 0x7f213415c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.662+0000 7f2139632700 1 -- 172.21.15.89:0/1931461580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21340cb900 msgr2=0x7f21340cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:52.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.662+0000 7f2139632700 1 --2- 172.21.15.89:0/1931461580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21340cb900 0x7f21340cbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.662+0000 7f2139632700 1 -- 172.21.15.89:0/1931461580 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21340cc710 con 0x7f2120004810 2023-10-10T20:43:52.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.663+0000 7f2139632700 1 --2- 172.21.15.89:0/1931461580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2120004810 0x7f2134148090 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f2130011f80 tx=0x7f213000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:52.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.663+0000 7f21297fa700 1 -- 172.21.15.89:0/1931461580 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213000e040 con 0x7f2120004810 2023-10-10T20:43:52.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.663+0000 7f21297fa700 1 -- 172.21.15.89:0/1931461580 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213000f040 con 0x7f2120004810 2023-10-10T20:43:52.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.663+0000 7f213a634700 1 -- 172.21.15.89:0/1931461580 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21340c7000 con 0x7f2120004810 2023-10-10T20:43:52.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.663+0000 7f21297fa700 1 -- 172.21.15.89:0/1931461580 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2130015430 con 0x7f2120004810 2023-10-10T20:43:52.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.663+0000 7f213a634700 1 -- 172.21.15.89:0/1931461580 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21340c75c0 con 0x7f2120004810 2023-10-10T20:43:52.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.664+0000 7f213a634700 1 -- 172.21.15.89:0/1931461580 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2134088990 con 0x7f2120004810 2023-10-10T20:43:52.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.664+0000 7f21297fa700 1 -- 172.21.15.89:0/1931461580 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2130015820 con 0x7f2120004810 2023-10-10T20:43:52.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.664+0000 7f21297fa700 1 --2- 172.21.15.89:0/1931461580 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2118067320 0x7f21180697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:52.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.665+0000 7f21297fa700 1 -- 172.21.15.89:0/1931461580 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2130085810 con 0x7f2120004810 2023-10-10T20:43:52.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.665+0000 7f2138e31700 1 --2- 172.21.15.89:0/1931461580 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2118067320 0x7f21180697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:52.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.665+0000 7f2138e31700 1 --2- 172.21.15.89:0/1931461580 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2118067320 0x7f21180697d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f212400c310 tx=0x7f2124015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:52.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.669+0000 7f21297fa700 1 -- 172.21.15.89:0/1931461580 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2130056120 con 0x7f2120004810 2023-10-10T20:43:52.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.854+0000 7f213a634700 1 -- 172.21.15.89:0/1931461580 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f213404bb80 con 0x7f2120004810 2023-10-10T20:43:52.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.857+0000 7f21297fa700 1 -- 172.21.15.89:0/1931461580 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2130055cb0 con 0x7f2120004810 2023-10-10T20:43:52.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.859+0000 7f213a634700 1 -- 172.21.15.89:0/1931461580 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2118067320 msgr2=0x7f21180697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:52.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.859+0000 7f213a634700 1 --2- 172.21.15.89:0/1931461580 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2118067320 0x7f21180697d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f212400c310 tx=0x7f2124015000 comp rx=0 tx=0).stop 2023-10-10T20:43:52.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.859+0000 7f213a634700 1 -- 172.21.15.89:0/1931461580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2120004810 msgr2=0x7f2134148090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:52.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.859+0000 7f213a634700 1 --2- 172.21.15.89:0/1931461580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2120004810 0x7f2134148090 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f2130011f80 tx=0x7f213000b040 comp rx=0 tx=0).stop 2023-10-10T20:43:52.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.860+0000 7f213a634700 1 -- 172.21.15.89:0/1931461580 shutdown_connections 2023-10-10T20:43:52.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.860+0000 7f213a634700 1 --2- 172.21.15.89:0/1931461580 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2118067320 0x7f21180697d0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.860+0000 7f213a634700 1 --2- 172.21.15.89:0/1931461580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21340cb900 0x7f21340cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.860+0000 7f213a634700 1 --2- 172.21.15.89:0/1931461580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2120004810 0x7f2134148090 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.860+0000 7f213a634700 1 --2- 172.21.15.89:0/1931461580 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f213415bbc0 0x7f213415c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.860+0000 7f213a634700 1 -- 172.21.15.89:0/1931461580 >> 172.21.15.89:0/1931461580 conn(0x7f21340d10c0 msgr2=0x7f2134158ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:52.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.860+0000 7f213a634700 1 -- 172.21.15.89:0/1931461580 shutdown_connections 2023-10-10T20:43:52.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.860+0000 7f213a634700 1 -- 172.21.15.89:0/1931461580 wait complete. 2023-10-10T20:43:52.873 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency' 2023-10-10T20:43:52.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.987+0000 7fb5e71c0700 1 Processor -- start 2023-10-10T20:43:52.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.987+0000 7fb5e71c0700 1 -- start start 2023-10-10T20:43:52.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.988+0000 7fb5e71c0700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb5e0159420 0x7fb5e00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:52.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.988+0000 7fb5e71c0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5e01597f0 0x7fb5e00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:52.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.988+0000 7fb5e71c0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb5e015d230 0x7fb5e015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:52.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.988+0000 7fb5e71c0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e015dbd0 con 0x7fb5e015d230 2023-10-10T20:43:52.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.988+0000 7fb5e71c0700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e00b11d0 con 0x7fb5e0159420 2023-10-10T20:43:52.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.988+0000 7fb5e71c0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e00b14b0 con 0x7fb5e01597f0 2023-10-10T20:43:52.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.988+0000 7fb5e69bf700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb5e015d230 0x7fb5e015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:52.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.988+0000 7fb5e69bf700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb5e015d230 0x7fb5e015d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46516/0 (socket says 172.21.15.89:46516) 2023-10-10T20:43:52.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.988+0000 7fb5e69bf700 1 -- 172.21.15.89:0/430057915 learned_addr learned my addr 172.21.15.89:0/430057915 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:52.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e61be700 1 --2- 172.21.15.89:0/430057915 >> v2:172.21.15.111:3300/0 conn(0x7fb5e0159420 0x7fb5e00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:52.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e69bf700 1 -- 172.21.15.89:0/430057915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5e01597f0 msgr2=0x7fb5e00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:52.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e59bd700 1 --2- 172.21.15.89:0/430057915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5e01597f0 0x7fb5e00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:52.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e69bf700 1 --2- 172.21.15.89:0/430057915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5e01597f0 0x7fb5e00b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e69bf700 1 -- 172.21.15.89:0/430057915 >> v2:172.21.15.111:3300/0 conn(0x7fb5e0159420 msgr2=0x7fb5e00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:52.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e69bf700 1 --2- 172.21.15.89:0/430057915 >> v2:172.21.15.111:3300/0 conn(0x7fb5e0159420 0x7fb5e00b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e69bf700 1 -- 172.21.15.89:0/430057915 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e00b1790 con 0x7fb5e015d230 2023-10-10T20:43:52.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e59bd700 1 --2- 172.21.15.89:0/430057915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5e01597f0 0x7fb5e00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:52.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e61be700 1 --2- 172.21.15.89:0/430057915 >> v2:172.21.15.111:3300/0 conn(0x7fb5e0159420 0x7fb5e00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:52.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e69bf700 1 --2- 172.21.15.89:0/430057915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5e015d230 0x7fb5e015d690 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fb5e00b2f20 tx=0x7fb5d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51915fcef843d35 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:52.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e51bc700 1 -- 172.21.15.89:0/430057915 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d800e070 con 0x7fb5e015d230 2023-10-10T20:43:52.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e51bc700 1 -- 172.21.15.89:0/430057915 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d800ba50 con 0x7fb5e015d230 2023-10-10T20:43:52.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e51bc700 1 -- 172.21.15.89:0/430057915 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d80092e0 con 0x7fb5e015d230 2023-10-10T20:43:52.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e71c0700 1 -- 172.21.15.89:0/430057915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5e015d230 msgr2=0x7fb5e015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:52.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e71c0700 1 --2- 172.21.15.89:0/430057915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5e015d230 0x7fb5e015d690 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fb5e00b2f20 tx=0x7fb5d8009840 comp rx=0 tx=0).stop 2023-10-10T20:43:52.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e71c0700 1 -- 172.21.15.89:0/430057915 shutdown_connections 2023-10-10T20:43:52.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e71c0700 1 --2- 172.21.15.89:0/430057915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5e015d230 0x7fb5e015d690 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e71c0700 1 --2- 172.21.15.89:0/430057915 >> v2:172.21.15.111:3300/0 conn(0x7fb5e0159420 0x7fb5e00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e71c0700 1 --2- 172.21.15.89:0/430057915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5e01597f0 0x7fb5e00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:52.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e71c0700 1 -- 172.21.15.89:0/430057915 >> 172.21.15.89:0/430057915 conn(0x7fb5e00d0660 msgr2=0x7fb5e00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:52.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.989+0000 7fb5e71c0700 1 -- 172.21.15.89:0/430057915 shutdown_connections 2023-10-10T20:43:52.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.990+0000 7fb5e71c0700 1 -- 172.21.15.89:0/430057915 wait complete. 2023-10-10T20:43:52.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.991+0000 7fb5e71c0700 1 Processor -- start 2023-10-10T20:43:52.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.991+0000 7fb5e71c0700 1 -- start start 2023-10-10T20:43:53.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.991+0000 7fb5e71c0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5e0159420 0x7fb5e01553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:53.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.991+0000 7fb5e71c0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5e01597f0 0x7fb5e01581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:53.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.991+0000 7fb5e71c0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5e015d230 0x7fb5e0158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:53.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.991+0000 7fb5e71c0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e0155920 con 0x7fb5e015d230 2023-10-10T20:43:53.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.991+0000 7fb5e71c0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e015c870 con 0x7fb5e01597f0 2023-10-10T20:43:53.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.991+0000 7fb5e71c0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e0169860 con 0x7fb5e0159420 2023-10-10T20:43:53.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.991+0000 7fb5e69bf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5e015d230 0x7fb5e0158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:53.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.992+0000 7fb5e69bf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5e015d230 0x7fb5e0158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46522/0 (socket says 172.21.15.89:46522) 2023-10-10T20:43:53.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.992+0000 7fb5e69bf700 1 -- 172.21.15.89:0/4112191899 learned_addr learned my addr 172.21.15.89:0/4112191899 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:53.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.992+0000 7fb5e59bd700 1 --2- 172.21.15.89:0/4112191899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5e01597f0 0x7fb5e01581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:53.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.992+0000 7fb5e61be700 1 --2- 172.21.15.89:0/4112191899 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5e0159420 0x7fb5e01553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:53.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.992+0000 7fb5e69bf700 1 -- 172.21.15.89:0/4112191899 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5e0159420 msgr2=0x7fb5e01553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:53.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.992+0000 7fb5e69bf700 1 --2- 172.21.15.89:0/4112191899 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5e0159420 0x7fb5e01553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:53.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.992+0000 7fb5e69bf700 1 -- 172.21.15.89:0/4112191899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5e01597f0 msgr2=0x7fb5e01581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:53.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.992+0000 7fb5e69bf700 1 --2- 172.21.15.89:0/4112191899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5e01597f0 0x7fb5e01581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:53.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.992+0000 7fb5e69bf700 1 -- 172.21.15.89:0/4112191899 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d8009ae0 con 0x7fb5e015d230 2023-10-10T20:43:53.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.992+0000 7fb5e69bf700 1 --2- 172.21.15.89:0/4112191899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5e015d230 0x7fb5e0158730 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0089340 tx=0x7fb5d8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:53.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.992+0000 7fb5d67fc700 1 -- 172.21.15.89:0/4112191899 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d80032f0 con 0x7fb5e015d230 2023-10-10T20:43:53.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.992+0000 7fb5d67fc700 1 -- 172.21.15.89:0/4112191899 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d8008e80 con 0x7fb5e015d230 2023-10-10T20:43:53.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.992+0000 7fb5d67fc700 1 -- 172.21.15.89:0/4112191899 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8025690 con 0x7fb5e015d230 2023-10-10T20:43:53.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.992+0000 7fb5e71c0700 1 -- 172.21.15.89:0/4112191899 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e016a3d0 con 0x7fb5e015d230 2023-10-10T20:43:53.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.992+0000 7fb5e71c0700 1 -- 172.21.15.89:0/4112191899 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e0169d80 con 0x7fb5e015d230 2023-10-10T20:43:53.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.993+0000 7fb5e71c0700 1 -- 172.21.15.89:0/4112191899 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e004c3c0 con 0x7fb5e015d230 2023-10-10T20:43:53.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.993+0000 7fb5d67fc700 1 -- 172.21.15.89:0/4112191899 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb5d8003040 con 0x7fb5e015d230 2023-10-10T20:43:53.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.994+0000 7fb5d67fc700 1 --2- 172.21.15.89:0/4112191899 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb5c4067320 0x7fb5c40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:53.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.994+0000 7fb5d67fc700 1 -- 172.21.15.89:0/4112191899 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb5d80862d0 con 0x7fb5e015d230 2023-10-10T20:43:53.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.994+0000 7fb5e61be700 1 --2- 172.21.15.89:0/4112191899 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb5c4067320 0x7fb5c40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:53.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.994+0000 7fb5e61be700 1 --2- 172.21.15.89:0/4112191899 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb5c4067320 0x7fb5c40697d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00ab30 tx=0x7fb5dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:53.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:52.998+0000 7fb5d67fc700 1 -- 172.21.15.89:0/4112191899 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5d8053200 con 0x7fb5e015d230 2023-10-10T20:43:53.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.255+0000 7fb5e71c0700 1 -- 172.21.15.89:0/4112191899 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency"]} v 0) v1 -- 0x7fb5e00b1a70 con 0x7fb5e015d230 2023-10-10T20:43:53.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.523+0000 7fb5d67fc700 1 -- 172.21.15.89:0/4112191899 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency"]}]=0 v476) v1 ==== 147+0+0 (secure 0 0 0) 0x7fb5d8056820 con 0x7fb5e015d230 2023-10-10T20:43:53.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.525+0000 7fb5e71c0700 1 -- 172.21.15.89:0/4112191899 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb5c4067320 msgr2=0x7fb5c40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:53.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.525+0000 7fb5e71c0700 1 --2- 172.21.15.89:0/4112191899 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb5c4067320 0x7fb5c40697d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00ab30 tx=0x7fb5dc008040 comp rx=0 tx=0).stop 2023-10-10T20:43:53.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.526+0000 7fb5e71c0700 1 -- 172.21.15.89:0/4112191899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5e015d230 msgr2=0x7fb5e0158730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:53.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.526+0000 7fb5e71c0700 1 --2- 172.21.15.89:0/4112191899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5e015d230 0x7fb5e0158730 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0089340 tx=0x7fb5d8018000 comp rx=0 tx=0).stop 2023-10-10T20:43:53.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.526+0000 7fb5e71c0700 1 -- 172.21.15.89:0/4112191899 shutdown_connections 2023-10-10T20:43:53.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.526+0000 7fb5e71c0700 1 --2- 172.21.15.89:0/4112191899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5e015d230 0x7fb5e0158730 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:53.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.526+0000 7fb5e71c0700 1 --2- 172.21.15.89:0/4112191899 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb5e0159420 0x7fb5e01553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:53.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.526+0000 7fb5e71c0700 1 --2- 172.21.15.89:0/4112191899 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb5c4067320 0x7fb5c40697d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:53.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.526+0000 7fb5e71c0700 1 --2- 172.21.15.89:0/4112191899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5e01597f0 0x7fb5e01581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:53.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.526+0000 7fb5e71c0700 1 -- 172.21.15.89:0/4112191899 >> 172.21.15.89:0/4112191899 conn(0x7fb5e00d0660 msgr2=0x7fb5e0156110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:53.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.526+0000 7fb5e71c0700 1 -- 172.21.15.89:0/4112191899 shutdown_connections 2023-10-10T20:43:53.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.526+0000 7fb5e71c0700 1 -- 172.21.15.89:0/4112191899 wait complete. 2023-10-10T20:43:53.539 INFO:tasks.cephfs_test_runner:That volume rename is idempotent. ... ok 2023-10-10T20:43:53.540 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) That the volume can only be removed when --yes-i-really-mean-it is used 2023-10-10T20:43:53.540 INFO:tasks.cephfs_test_runner:test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-10T20:43:53.541 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm' 2023-10-10T20:43:53.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.654+0000 7f0ce59a9700 1 Processor -- start 2023-10-10T20:43:53.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.654+0000 7f0ce59a9700 1 -- start start 2023-10-10T20:43:53.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.655+0000 7f0ce59a9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ce0159350 0x7f0ce0159720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:53.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.655+0000 7f0ce59a9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0ce00aee80 0x7f0ce00cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:53.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.655+0000 7f0ce59a9700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0ce006e390 0x7f0ce006e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.655+0000 7f0ce59a9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce00cbb20 con 0x7f0ce00aee80 2023-10-10T20:43:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.655+0000 7f0ce59a9700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce006ed50 con 0x7f0ce006e390 2023-10-10T20:43:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.655+0000 7f0ce59a9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce00c7170 con 0x7f0ce0159350 2023-10-10T20:43:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.655+0000 7f0cde7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0ce00aee80 0x7f0ce00cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.655+0000 7f0cde7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0ce00aee80 0x7f0ce00cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46530/0 (socket says 172.21.15.89:46530) 2023-10-10T20:43:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.655+0000 7f0cde7fc700 1 -- 172.21.15.89:0/715488550 learned_addr learned my addr 172.21.15.89:0/715488550 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.656+0000 7f0cdf7fe700 1 --2- 172.21.15.89:0/715488550 >> v2:172.21.15.111:3300/0 conn(0x7f0ce006e390 0x7f0ce006e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.656+0000 7f0cdeffd700 1 --2- 172.21.15.89:0/715488550 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ce0159350 0x7f0ce0159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.656+0000 7f0cde7fc700 1 -- 172.21.15.89:0/715488550 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ce0159350 msgr2=0x7f0ce0159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.656+0000 7f0cde7fc700 1 --2- 172.21.15.89:0/715488550 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ce0159350 0x7f0ce0159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.656+0000 7f0cde7fc700 1 -- 172.21.15.89:0/715488550 >> v2:172.21.15.111:3300/0 conn(0x7f0ce006e390 msgr2=0x7f0ce006e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.656+0000 7f0cde7fc700 1 --2- 172.21.15.89:0/715488550 >> v2:172.21.15.111:3300/0 conn(0x7f0ce006e390 0x7f0ce006e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.656+0000 7f0cde7fc700 1 -- 172.21.15.89:0/715488550 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce00af2f0 con 0x7f0ce00aee80 2023-10-10T20:43:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.656+0000 7f0cdeffd700 1 --2- 172.21.15.89:0/715488550 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ce0159350 0x7f0ce0159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.656+0000 7f0cde7fc700 1 --2- 172.21.15.89:0/715488550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ce00aee80 0x7f0ce00cb5e0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40099e0 tx=0x7f0cd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d2d35b29194f4c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.656+0000 7f0cddffb700 1 -- 172.21.15.89:0/715488550 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd400b7e0 con 0x7f0ce00aee80 2023-10-10T20:43:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.656+0000 7f0cddffb700 1 -- 172.21.15.89:0/715488550 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd400b940 con 0x7f0ce00aee80 2023-10-10T20:43:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.656+0000 7f0cddffb700 1 -- 172.21.15.89:0/715488550 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd400bce0 con 0x7f0ce00aee80 2023-10-10T20:43:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.656+0000 7f0ce59a9700 1 -- 172.21.15.89:0/715488550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ce00aee80 msgr2=0x7f0ce00cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.657+0000 7f0ce59a9700 1 --2- 172.21.15.89:0/715488550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ce00aee80 0x7f0ce00cb5e0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40099e0 tx=0x7f0cd4009250 comp rx=0 tx=0).stop 2023-10-10T20:43:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.657+0000 7f0ce59a9700 1 -- 172.21.15.89:0/715488550 shutdown_connections 2023-10-10T20:43:53.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.657+0000 7f0ce59a9700 1 --2- 172.21.15.89:0/715488550 >> v2:172.21.15.111:3300/0 conn(0x7f0ce006e390 0x7f0ce006e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:53.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.657+0000 7f0ce59a9700 1 --2- 172.21.15.89:0/715488550 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ce0159350 0x7f0ce0159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:53.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.657+0000 7f0ce59a9700 1 --2- 172.21.15.89:0/715488550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ce00aee80 0x7f0ce00cb5e0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:53.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.657+0000 7f0ce59a9700 1 -- 172.21.15.89:0/715488550 >> 172.21.15.89:0/715488550 conn(0x7f0ce00c9f00 msgr2=0x7f0ce006dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:53.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.657+0000 7f0ce59a9700 1 -- 172.21.15.89:0/715488550 shutdown_connections 2023-10-10T20:43:53.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.657+0000 7f0ce59a9700 1 -- 172.21.15.89:0/715488550 wait complete. 2023-10-10T20:43:53.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.658+0000 7f0ce59a9700 1 Processor -- start 2023-10-10T20:43:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.658+0000 7f0ce59a9700 1 -- start start 2023-10-10T20:43:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0ce59a9700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ce006e390 0x7f0ce0155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0ce59a9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ce00aee80 0x7f0ce00a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0ce59a9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ce0159350 0x7f0ce00a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0ce59a9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce00a68e0 con 0x7f0ce00aee80 2023-10-10T20:43:53.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0ce59a9700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce00c4930 con 0x7f0ce006e390 2023-10-10T20:43:53.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0ce59a9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce0155e90 con 0x7f0ce0159350 2023-10-10T20:43:53.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0cde7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ce00aee80 0x7f0ce00a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:53.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0cde7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ce00aee80 0x7f0ce00a5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46534/0 (socket says 172.21.15.89:46534) 2023-10-10T20:43:53.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0cde7fc700 1 -- 172.21.15.89:0/3899777069 learned_addr learned my addr 172.21.15.89:0/3899777069 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:53.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0cdf7fe700 1 --2- 172.21.15.89:0/3899777069 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ce0159350 0x7f0ce00a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:53.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0cdeffd700 1 --2- 172.21.15.89:0/3899777069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ce006e390 0x7f0ce0155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:53.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0cde7fc700 1 -- 172.21.15.89:0/3899777069 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ce0159350 msgr2=0x7f0ce00a63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:53.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0cde7fc700 1 --2- 172.21.15.89:0/3899777069 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ce0159350 0x7f0ce00a63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:53.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0cde7fc700 1 -- 172.21.15.89:0/3899777069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ce006e390 msgr2=0x7f0ce0155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:53.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0cde7fc700 1 --2- 172.21.15.89:0/3899777069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ce006e390 0x7f0ce0155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:53.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0cde7fc700 1 -- 172.21.15.89:0/3899777069 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd40094f0 con 0x7f0ce00aee80 2023-10-10T20:43:53.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0cde7fc700 1 --2- 172.21.15.89:0/3899777069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ce00aee80 0x7f0ce00a5e60 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009290 tx=0x7f0cd4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:53.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3899777069 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4024440 con 0x7f0ce00aee80 2023-10-10T20:43:53.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3899777069 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd40245a0 con 0x7f0ce00aee80 2023-10-10T20:43:53.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0ce59a9700 1 -- 172.21.15.89:0/3899777069 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce00c2720 con 0x7f0ce00aee80 2023-10-10T20:43:53.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3899777069 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4024960 con 0x7f0ce00aee80 2023-10-10T20:43:53.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.659+0000 7f0ce59a9700 1 -- 172.21.15.89:0/3899777069 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce00c2ce0 con 0x7f0ce00aee80 2023-10-10T20:43:53.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.660+0000 7f0ce59a9700 1 -- 172.21.15.89:0/3899777069 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce00c4ae0 con 0x7f0ce00aee80 2023-10-10T20:43:53.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.661+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3899777069 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0cd401a420 con 0x7f0ce00aee80 2023-10-10T20:43:53.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.661+0000 7f0cc6ffd700 1 --2- 172.21.15.89:0/3899777069 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0ccc067320 0x7f0ccc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:53.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.661+0000 7f0cdeffd700 1 --2- 172.21.15.89:0/3899777069 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0ccc067320 0x7f0ccc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:53.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.661+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3899777069 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0cd4086640 con 0x7f0ce00aee80 2023-10-10T20:43:53.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.662+0000 7f0cdeffd700 1 --2- 172.21.15.89:0/3899777069 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0ccc067320 0x7f0ccc0697d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000f600 tx=0x7f0cd00077b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:53.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.665+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3899777069 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ce00c4ae0 con 0x7f0ce00aee80 2023-10-10T20:43:53.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:53.923+0000 7f0ce59a9700 1 -- 172.21.15.89:0/3899777069 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm"]} v 0) v1 -- 0x7f0ce006c970 con 0x7f0ce00aee80 2023-10-10T20:43:54.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.525+0000 7f0cc6ffd700 1 -- 172.21.15.89:0/3899777069 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm"]}]=0 v477) v1 ==== 134+0+0 (secure 0 0 0) 0x7f0cd40535d0 con 0x7f0ce00aee80 2023-10-10T20:43:54.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.527+0000 7f0ce59a9700 1 -- 172.21.15.89:0/3899777069 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0ccc067320 msgr2=0x7f0ccc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:54.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.527+0000 7f0ce59a9700 1 --2- 172.21.15.89:0/3899777069 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0ccc067320 0x7f0ccc0697d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000f600 tx=0x7f0cd00077b0 comp rx=0 tx=0).stop 2023-10-10T20:43:54.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.527+0000 7f0ce59a9700 1 -- 172.21.15.89:0/3899777069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ce00aee80 msgr2=0x7f0ce00a5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:54.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.527+0000 7f0ce59a9700 1 --2- 172.21.15.89:0/3899777069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ce00aee80 0x7f0ce00a5e60 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009290 tx=0x7f0cd4017000 comp rx=0 tx=0).stop 2023-10-10T20:43:54.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.527+0000 7f0ce59a9700 1 -- 172.21.15.89:0/3899777069 shutdown_connections 2023-10-10T20:43:54.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.527+0000 7f0ce59a9700 1 --2- 172.21.15.89:0/3899777069 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0ce0159350 0x7f0ce00a63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:54.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.527+0000 7f0ce59a9700 1 --2- 172.21.15.89:0/3899777069 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0ccc067320 0x7f0ccc0697d0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:54.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.527+0000 7f0ce59a9700 1 --2- 172.21.15.89:0/3899777069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ce006e390 0x7f0ce0155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:54.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.527+0000 7f0ce59a9700 1 --2- 172.21.15.89:0/3899777069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ce00aee80 0x7f0ce00a5e60 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:54.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.527+0000 7f0ce59a9700 1 -- 172.21.15.89:0/3899777069 >> 172.21.15.89:0/3899777069 conn(0x7f0ce00c9f00 msgr2=0x7f0ce00b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:54.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.527+0000 7f0ce59a9700 1 -- 172.21.15.89:0/3899777069 shutdown_connections 2023-10-10T20:43:54.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.527+0000 7f0ce59a9700 1 -- 172.21.15.89:0/3899777069 wait complete. 2023-10-10T20:43:54.541 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:43:54.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.657+0000 7fbfdaa9f700 1 Processor -- start 2023-10-10T20:43:54.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.657+0000 7fbfdaa9f700 1 -- start start 2023-10-10T20:43:54.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.658+0000 7fbfdaa9f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfd4159890 0x7fbfd415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:54.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.658+0000 7fbfdaa9f700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbfd4069560 0x7fbfd40699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:54.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.658+0000 7fbfdaa9f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd415a160 con 0x7fbfd4159c60 2023-10-10T20:43:54.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.658+0000 7fbfdaa9f700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd4069f20 con 0x7fbfd4069560 2023-10-10T20:43:54.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.658+0000 7fbfdaa9f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40d13a0 con 0x7fbfd4159890 2023-10-10T20:43:54.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.658+0000 7fbfcbfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbfd4159c60 0x7fbfd415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35704/0 (socket says 172.21.15.89:35704) 2023-10-10T20:43:54.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.658+0000 7fbfcbfff700 1 -- 172.21.15.89:0/1996841259 learned_addr learned my addr 172.21.15.89:0/1996841259 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:54.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfd903c700 1 --2- 172.21.15.89:0/1996841259 >> v2:172.21.15.111:3300/0 conn(0x7fbfd4069560 0x7fbfd40699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:54.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfd883b700 1 --2- 172.21.15.89:0/1996841259 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfd4159890 0x7fbfd415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:54.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfcb7fe700 1 -- 172.21.15.89:0/1996841259 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2213983042 0 0) 0x7fbfd415a160 con 0x7fbfd4159c60 2023-10-10T20:43:54.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfcb7fe700 1 -- 172.21.15.89:0/1996841259 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfbc003570 con 0x7fbfd4159c60 2023-10-10T20:43:54.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfcb7fe700 1 -- 172.21.15.89:0/1996841259 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1528851255 0 0) 0x7fbfbc003570 con 0x7fbfd4159c60 2023-10-10T20:43:54.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfcb7fe700 1 -- 172.21.15.89:0/1996841259 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfd4159890 msgr2=0x7fbfd415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:54.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfcb7fe700 1 --2- 172.21.15.89:0/1996841259 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfd4159890 0x7fbfd415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:54.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfcb7fe700 1 -- 172.21.15.89:0/1996841259 >> v2:172.21.15.111:3300/0 conn(0x7fbfd4069560 msgr2=0x7fbfd40699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:54.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfcb7fe700 1 --2- 172.21.15.89:0/1996841259 >> v2:172.21.15.111:3300/0 conn(0x7fbfd4069560 0x7fbfd40699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:54.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfcb7fe700 1 -- 172.21.15.89:0/1996841259 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd415e140 con 0x7fbfd4159c60 2023-10-10T20:43:54.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfcb7fe700 1 -- 172.21.15.89:0/1996841259 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fbfc4001470 con 0x7fbfd4159c60 2023-10-10T20:43:54.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfcb7fe700 1 -- 172.21.15.89:0/1996841259 >> v1:172.21.15.89:6789/0 conn(0x7fbfd4159c60 legacy=0x7fbfd415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:54.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfcb7fe700 1 --2- 172.21.15.89:0/1996841259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfbc003a60 0x7fbfbc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:54.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfcb7fe700 1 --2- 172.21.15.89:0/1996841259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfbc004810 0x7fbfbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:54.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfcb7fe700 1 --2- 172.21.15.89:0/1996841259 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfd4159890 0x7fbfbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:54.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfcb7fe700 1 -- 172.21.15.89:0/1996841259 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd4069f20 con 0x7fbfbc003a60 2023-10-10T20:43:54.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfcb7fe700 1 -- 172.21.15.89:0/1996841259 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40d13a0 con 0x7fbfbc004810 2023-10-10T20:43:54.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfcb7fe700 1 -- 172.21.15.89:0/1996841259 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc00ba60 con 0x7fbfd4159890 2023-10-10T20:43:54.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfd883b700 1 --2- 172.21.15.89:0/1996841259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfbc003a60 0x7fbfbc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:54.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfcbfff700 1 --2- 172.21.15.89:0/1996841259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfbc004810 0x7fbfbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:54.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfd903c700 1 --2- 172.21.15.89:0/1996841259 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfd4159890 0x7fbfbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:54.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfd883b700 1 -- 172.21.15.89:0/1996841259 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfd4159890 msgr2=0x7fbfbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:54.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfd883b700 1 --2- 172.21.15.89:0/1996841259 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfd4159890 0x7fbfbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:54.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfd883b700 1 -- 172.21.15.89:0/1996841259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfbc004810 msgr2=0x7fbfbc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:54.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfd883b700 1 --2- 172.21.15.89:0/1996841259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfbc004810 0x7fbfbc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:54.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.659+0000 7fbfd883b700 1 -- 172.21.15.89:0/1996841259 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc003570 con 0x7fbfbc003a60 2023-10-10T20:43:54.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.660+0000 7fbfd883b700 1 --2- 172.21.15.89:0/1996841259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfbc003a60 0x7fbfbc003e70 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0009c30 tx=0x7fbfd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6eaa5b732a43a7c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:54.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.660+0000 7fbfcb7fe700 1 -- 172.21.15.89:0/1996841259 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd000e710 con 0x7fbfbc003a60 2023-10-10T20:43:54.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.660+0000 7fbfcb7fe700 1 -- 172.21.15.89:0/1996841259 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd000e870 con 0x7fbfbc003a60 2023-10-10T20:43:54.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.660+0000 7fbfcb7fe700 1 -- 172.21.15.89:0/1996841259 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd000ec30 con 0x7fbfbc003a60 2023-10-10T20:43:54.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.660+0000 7fbfdaa9f700 1 -- 172.21.15.89:0/1996841259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfbc003a60 msgr2=0x7fbfbc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:54.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.660+0000 7fbfdaa9f700 1 --2- 172.21.15.89:0/1996841259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfbc003a60 0x7fbfbc003e70 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0009c30 tx=0x7fbfd00091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:54.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.660+0000 7fbfd903c700 1 -- 172.21.15.89:0/1996841259 reap_dead start 2023-10-10T20:43:54.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.660+0000 7fbfdaa9f700 1 -- 172.21.15.89:0/1996841259 shutdown_connections 2023-10-10T20:43:54.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.660+0000 7fbfdaa9f700 1 -- 172.21.15.89:0/1996841259 >> 172.21.15.89:0/1996841259 conn(0x7fbfd4068e20 msgr2=0x7fbfd40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:54.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.660+0000 7fbfdaa9f700 1 -- 172.21.15.89:0/1996841259 shutdown_connections 2023-10-10T20:43:54.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.660+0000 7fbfdaa9f700 1 -- 172.21.15.89:0/1996841259 wait complete. 2023-10-10T20:43:54.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.661+0000 7fbfdaa9f700 1 Processor -- start 2023-10-10T20:43:54.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.661+0000 7fbfdaa9f700 1 -- start start 2023-10-10T20:43:54.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.662+0000 7fbfdaa9f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfbc003a60 0x7fbfd40ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:54.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.662+0000 7fbfdaa9f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd40cfc50 0x7fbfd4152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:54.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.662+0000 7fbfdaa9f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfd40cd250 0x7fbfd415bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:54.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.662+0000 7fbfdaa9f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40ce9c0 con 0x7fbfbc003a60 2023-10-10T20:43:54.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.662+0000 7fbfdaa9f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40cebd0 con 0x7fbfd40cfc50 2023-10-10T20:43:54.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.662+0000 7fbfdaa9f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40ceeb0 con 0x7fbfd40cd250 2023-10-10T20:43:54.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.662+0000 7fbfd883b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfbc003a60 0x7fbfd40ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:54.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.662+0000 7fbfd883b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfbc003a60 0x7fbfd40ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46562/0 (socket says 172.21.15.89:46562) 2023-10-10T20:43:54.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.662+0000 7fbfd883b700 1 -- 172.21.15.89:0/91339819 learned_addr learned my addr 172.21.15.89:0/91339819 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:54.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.662+0000 7fbfd883b700 1 -- 172.21.15.89:0/91339819 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfd40cd250 msgr2=0x7fbfd415bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:54.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.662+0000 7fbfd883b700 1 --2- 172.21.15.89:0/91339819 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfd40cd250 0x7fbfd415bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:54.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.662+0000 7fbfcbfff700 1 --2- 172.21.15.89:0/91339819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd40cfc50 0x7fbfd4152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:54.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.662+0000 7fbfd883b700 1 -- 172.21.15.89:0/91339819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd40cfc50 msgr2=0x7fbfd4152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:54.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.663+0000 7fbfd883b700 1 --2- 172.21.15.89:0/91339819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd40cfc50 0x7fbfd4152c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:54.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.663+0000 7fbfd883b700 1 -- 172.21.15.89:0/91339819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd40cf190 con 0x7fbfbc003a60 2023-10-10T20:43:54.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.663+0000 7fbfcbfff700 1 --2- 172.21.15.89:0/91339819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd40cfc50 0x7fbfd4152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:54.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.663+0000 7fbfd883b700 1 --2- 172.21.15.89:0/91339819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfbc003a60 0x7fbfd40ce170 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0046d0 tx=0x7fbfd001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:54.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.663+0000 7fbfc8ff9700 1 -- 172.21.15.89:0/91339819 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0022070 con 0x7fbfbc003a60 2023-10-10T20:43:54.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.663+0000 7fbfc8ff9700 1 -- 172.21.15.89:0/91339819 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd000e9e0 con 0x7fbfbc003a60 2023-10-10T20:43:54.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.663+0000 7fbfc8ff9700 1 -- 172.21.15.89:0/91339819 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd001c450 con 0x7fbfbc003a60 2023-10-10T20:43:54.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.663+0000 7fbfdaa9f700 1 -- 172.21.15.89:0/91339819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd40cf470 con 0x7fbfbc003a60 2023-10-10T20:43:54.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.663+0000 7fbfdaa9f700 1 -- 172.21.15.89:0/91339819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd4153150 con 0x7fbfbc003a60 2023-10-10T20:43:54.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.664+0000 7fbfdaa9f700 1 -- 172.21.15.89:0/91339819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd4153340 con 0x7fbfbc003a60 2023-10-10T20:43:54.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.664+0000 7fbfc8ff9700 1 -- 172.21.15.89:0/91339819 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbfd001c840 con 0x7fbfbc003a60 2023-10-10T20:43:54.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.665+0000 7fbfc8ff9700 1 --2- 172.21.15.89:0/91339819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbfc0067320 0x7fbfc00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:54.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.665+0000 7fbfc8ff9700 1 -- 172.21.15.89:0/91339819 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fbfd00862f0 con 0x7fbfbc003a60 2023-10-10T20:43:54.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.665+0000 7fbfcbfff700 1 --2- 172.21.15.89:0/91339819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbfc0067320 0x7fbfc00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:54.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.665+0000 7fbfcbfff700 1 --2- 172.21.15.89:0/91339819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbfc0067320 0x7fbfc00697d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4001620 tx=0x7fbfc4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:54.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.669+0000 7fbfc8ff9700 1 -- 172.21.15.89:0/91339819 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfd0053280 con 0x7fbfbc003a60 2023-10-10T20:43:54.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.852+0000 7fbfdaa9f700 1 -- 172.21.15.89:0/91339819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbfd4154550 con 0x7fbfbc003a60 2023-10-10T20:43:54.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.858+0000 7fbfc8ff9700 1 -- 172.21.15.89:0/91339819 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbfd00568a0 con 0x7fbfbc003a60 2023-10-10T20:43:54.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.860+0000 7fbfdaa9f700 1 -- 172.21.15.89:0/91339819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbfc0067320 msgr2=0x7fbfc00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:54.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.860+0000 7fbfdaa9f700 1 --2- 172.21.15.89:0/91339819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbfc0067320 0x7fbfc00697d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4001620 tx=0x7fbfc4009040 comp rx=0 tx=0).stop 2023-10-10T20:43:54.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.860+0000 7fbfdaa9f700 1 -- 172.21.15.89:0/91339819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfbc003a60 msgr2=0x7fbfd40ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:54.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.860+0000 7fbfdaa9f700 1 --2- 172.21.15.89:0/91339819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfbc003a60 0x7fbfd40ce170 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0046d0 tx=0x7fbfd001a000 comp rx=0 tx=0).stop 2023-10-10T20:43:54.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.860+0000 7fbfdaa9f700 1 -- 172.21.15.89:0/91339819 shutdown_connections 2023-10-10T20:43:54.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.860+0000 7fbfdaa9f700 1 --2- 172.21.15.89:0/91339819 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfd40cd250 0x7fbfd415bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:54.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.860+0000 7fbfdaa9f700 1 --2- 172.21.15.89:0/91339819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfbc003a60 0x7fbfd40ce170 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:54.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.860+0000 7fbfdaa9f700 1 --2- 172.21.15.89:0/91339819 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbfc0067320 0x7fbfc00697d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:54.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.860+0000 7fbfdaa9f700 1 --2- 172.21.15.89:0/91339819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd40cfc50 0x7fbfd4152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:54.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.860+0000 7fbfdaa9f700 1 -- 172.21.15.89:0/91339819 >> 172.21.15.89:0/91339819 conn(0x7fbfd4068e20 msgr2=0x7fbfd414d430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:54.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.860+0000 7fbfdaa9f700 1 -- 172.21.15.89:0/91339819 shutdown_connections 2023-10-10T20:43:54.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:54.860+0000 7fbfdaa9f700 1 -- 172.21.15.89:0/91339819 wait complete. 2023-10-10T20:43:54.874 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:43:54.875 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:43:54.918 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:43:54.960 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:43:54.960 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:43:54.961 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:43:55.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.074+0000 7f9532cd9700 1 Processor -- start 2023-10-10T20:43:55.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.074+0000 7f9532cd9700 1 -- start start 2023-10-10T20:43:55.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.075+0000 7f9532cd9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f952c1593b0 0x7f952c159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:55.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.075+0000 7f9532cd9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952c0cfe20 con 0x7f952c0ceaa0 2023-10-10T20:43:55.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.075+0000 7f9532cd9700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952c0cf210 con 0x7f952c0b1e50 2023-10-10T20:43:55.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.075+0000 7f9532cd9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0cf4f0 con 0x7f952c1593b0 2023-10-10T20:43:55.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.075+0000 7f95314d6700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f952c0ceaa0 0x7f952c0b1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35718/0 (socket says 172.21.15.89:35718) 2023-10-10T20:43:55.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.075+0000 7f95314d6700 1 -- 172.21.15.89:0/4025986976 learned_addr learned my addr 172.21.15.89:0/4025986976 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:55.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.075+0000 7f9531cd7700 1 --2- 172.21.15.89:0/4025986976 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f952c1593b0 0x7f952c159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:55.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.075+0000 7f9530cd5700 1 -- 172.21.15.89:0/4025986976 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2364338547 0 0) 0x7f952c0cfe20 con 0x7f952c0ceaa0 2023-10-10T20:43:55.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.075+0000 7f9530cd5700 1 -- 172.21.15.89:0/4025986976 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9518003570 con 0x7f952c0ceaa0 2023-10-10T20:43:55.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f9530cd5700 1 -- 172.21.15.89:0/4025986976 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1010045911 0 0) 0x7f9518003570 con 0x7f952c0ceaa0 2023-10-10T20:43:55.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f9530cd5700 1 -- 172.21.15.89:0/4025986976 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f952c1593b0 msgr2=0x7f952c159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:55.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f9530cd5700 1 --2- 172.21.15.89:0/4025986976 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f952c1593b0 0x7f952c159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:55.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f9530cd5700 1 -- 172.21.15.89:0/4025986976 >> v1:172.21.15.111:6789/0 conn(0x7f952c0b1e50 legacy=0x7f952c0b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:55.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f9530cd5700 1 -- 172.21.15.89:0/4025986976 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c0cf7d0 con 0x7f952c0ceaa0 2023-10-10T20:43:55.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f9530cd5700 1 -- 172.21.15.89:0/4025986976 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f951c001470 con 0x7f952c0ceaa0 2023-10-10T20:43:55.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f9530cd5700 1 -- 172.21.15.89:0/4025986976 >> v1:172.21.15.89:6789/0 conn(0x7f952c0ceaa0 legacy=0x7f952c0b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:55.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f9530cd5700 1 --2- 172.21.15.89:0/4025986976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9518003a90 0x7f9518003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:55.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f9530cd5700 1 --2- 172.21.15.89:0/4025986976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9518004810 0x7f9518004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:55.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f9530cd5700 1 --2- 172.21.15.89:0/4025986976 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f952c1593b0 0x7f95180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:55.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f9530cd5700 1 -- 172.21.15.89:0/4025986976 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0cf4f0 con 0x7f9518004810 2023-10-10T20:43:55.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f9530cd5700 1 -- 172.21.15.89:0/4025986976 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951800bb10 con 0x7f9518003a90 2023-10-10T20:43:55.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f9530cd5700 1 -- 172.21.15.89:0/4025986976 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951800bcd0 con 0x7f952c1593b0 2023-10-10T20:43:55.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f95314d6700 1 --2- 172.21.15.89:0/4025986976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9518004810 0x7f9518004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:55.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f9531cd7700 1 --2- 172.21.15.89:0/4025986976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9518003a90 0x7f9518003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:55.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f95324d8700 1 --2- 172.21.15.89:0/4025986976 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f952c1593b0 0x7f95180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:55.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f95314d6700 1 -- 172.21.15.89:0/4025986976 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f952c1593b0 msgr2=0x7f95180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:55.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f95314d6700 1 --2- 172.21.15.89:0/4025986976 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f952c1593b0 0x7f95180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:55.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f95314d6700 1 -- 172.21.15.89:0/4025986976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9518003a90 msgr2=0x7f9518003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:55.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f95314d6700 1 --2- 172.21.15.89:0/4025986976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9518003a90 0x7f9518003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:55.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f95314d6700 1 -- 172.21.15.89:0/4025986976 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9518003570 con 0x7f9518004810 2023-10-10T20:43:55.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f95324d8700 1 --2- 172.21.15.89:0/4025986976 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f952c1593b0 0x7f95180051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:55.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f95314d6700 1 --2- 172.21.15.89:0/4025986976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9518004810 0x7f9518004c80 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f951c00c390 tx=0x7f952c0cfd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c064f074c43c7503 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:55.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f9530cd5700 1 -- 172.21.15.89:0/4025986976 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c00cc60 con 0x7f9518004810 2023-10-10T20:43:55.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f9530cd5700 1 -- 172.21.15.89:0/4025986976 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951c00ce00 con 0x7f9518004810 2023-10-10T20:43:55.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.076+0000 7f9530cd5700 1 -- 172.21.15.89:0/4025986976 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c005b70 con 0x7f9518004810 2023-10-10T20:43:55.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.077+0000 7f9532cd9700 1 -- 172.21.15.89:0/4025986976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9518004810 msgr2=0x7f9518004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:55.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.077+0000 7f9532cd9700 1 --2- 172.21.15.89:0/4025986976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9518004810 0x7f9518004c80 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f951c00c390 tx=0x7f952c0cfd90 comp rx=0 tx=0).stop 2023-10-10T20:43:55.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.077+0000 7f95324d8700 1 -- 172.21.15.89:0/4025986976 reap_dead start 2023-10-10T20:43:55.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.077+0000 7f9532cd9700 1 -- 172.21.15.89:0/4025986976 shutdown_connections 2023-10-10T20:43:55.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.077+0000 7f9532cd9700 1 -- 172.21.15.89:0/4025986976 >> 172.21.15.89:0/4025986976 conn(0x7f952c0d1130 msgr2=0x7f952c0b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.077+0000 7f9532cd9700 1 -- 172.21.15.89:0/4025986976 shutdown_connections 2023-10-10T20:43:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.077+0000 7f9532cd9700 1 -- 172.21.15.89:0/4025986976 wait complete. 2023-10-10T20:43:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.078+0000 7f9532cd9700 1 Processor -- start 2023-10-10T20:43:55.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.078+0000 7f9532cd9700 1 -- start start 2023-10-10T20:43:55.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.079+0000 7f9532cd9700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9518004810 0x7f952c0c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:55.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.079+0000 7f9532cd9700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f952c146070 0x7f952c0b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:55.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.079+0000 7f9532cd9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f952c0b4830 0x7f952c15b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:55.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.079+0000 7f9532cd9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c15c460 con 0x7f952c0b4830 2023-10-10T20:43:55.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.079+0000 7f9532cd9700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0b4cd0 con 0x7f952c146070 2023-10-10T20:43:55.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.079+0000 7f9532cd9700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0c5bf0 con 0x7f9518004810 2023-10-10T20:43:55.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.079+0000 7f95324d8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f952c0b4830 0x7f952c15b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:55.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.079+0000 7f95324d8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f952c0b4830 0x7f952c15b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46574/0 (socket says 172.21.15.89:46574) 2023-10-10T20:43:55.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.079+0000 7f95324d8700 1 -- 172.21.15.89:0/2380796862 learned_addr learned my addr 172.21.15.89:0/2380796862 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:55.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.079+0000 7f95314d6700 1 --2- 172.21.15.89:0/2380796862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f952c146070 0x7f952c0b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:55.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.079+0000 7f9531cd7700 1 --2- 172.21.15.89:0/2380796862 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9518004810 0x7f952c0c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:55.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.079+0000 7f95324d8700 1 -- 172.21.15.89:0/2380796862 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9518004810 msgr2=0x7f952c0c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:55.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.079+0000 7f95324d8700 1 --2- 172.21.15.89:0/2380796862 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9518004810 0x7f952c0c6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:55.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.079+0000 7f95324d8700 1 -- 172.21.15.89:0/2380796862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f952c146070 msgr2=0x7f952c0b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:55.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.079+0000 7f95324d8700 1 --2- 172.21.15.89:0/2380796862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f952c146070 0x7f952c0b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:55.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.079+0000 7f95324d8700 1 -- 172.21.15.89:0/2380796862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c15c180 con 0x7f952c0b4830 2023-10-10T20:43:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.079+0000 7f95324d8700 1 --2- 172.21.15.89:0/2380796862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f952c0b4830 0x7f952c15b4e0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f952c0b2250 tx=0x7f9524009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.080+0000 7f9521ffb700 1 -- 172.21.15.89:0/2380796862 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524013630 con 0x7f952c0b4830 2023-10-10T20:43:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.080+0000 7f9521ffb700 1 -- 172.21.15.89:0/2380796862 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95240137b0 con 0x7f952c0b4830 2023-10-10T20:43:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.080+0000 7f9532cd9700 1 -- 172.21.15.89:0/2380796862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952c0c55a0 con 0x7f952c0b4830 2023-10-10T20:43:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.080+0000 7f9521ffb700 1 -- 172.21.15.89:0/2380796862 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524013b70 con 0x7f952c0b4830 2023-10-10T20:43:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.080+0000 7f9532cd9700 1 -- 172.21.15.89:0/2380796862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952c1464e0 con 0x7f952c0b4830 2023-10-10T20:43:55.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.081+0000 7f9532cd9700 1 -- 172.21.15.89:0/2380796862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952c0c5400 con 0x7f952c0b4830 2023-10-10T20:43:55.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.081+0000 7f9521ffb700 1 -- 172.21.15.89:0/2380796862 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9524003030 con 0x7f952c0b4830 2023-10-10T20:43:55.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.081+0000 7f9521ffb700 1 --2- 172.21.15.89:0/2380796862 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f95100673a0 0x7f9510069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:55.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.081+0000 7f9521ffb700 1 -- 172.21.15.89:0/2380796862 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f9524084820 con 0x7f952c0b4830 2023-10-10T20:43:55.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.081+0000 7f9531cd7700 1 --2- 172.21.15.89:0/2380796862 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f95100673a0 0x7f9510069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:55.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.082+0000 7f9531cd7700 1 --2- 172.21.15.89:0/2380796862 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f95100673a0 0x7f9510069850 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f952c15b100 tx=0x7f95280091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:55.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.086+0000 7f9521ffb700 1 -- 172.21.15.89:0/2380796862 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f952c0c5400 con 0x7f952c0b4830 2023-10-10T20:43:55.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.311+0000 7f9532cd9700 1 -- 172.21.15.89:0/2380796862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f952c0c5400 con 0x7f952c0b4830 2023-10-10T20:43:55.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.312+0000 7f9521ffb700 1 -- 172.21.15.89:0/2380796862 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+3534 (secure 0 0 0) 0x7f952c0c5400 con 0x7f952c0b4830 2023-10-10T20:43:55.314 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:55.314 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":126,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/455398787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":455398787},{"type":"v1","addr":"172.21.15.89:6836","nonce":455398787}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/2660411678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2660411678},{"type":"v1","addr":"172.21.15.111:6834","nonce":2660411678}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6890,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/1650071647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":1650071647},{"type":"v1","addr":"172.21.15.111:6835","nonce":1650071647}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":9078,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/546359647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":546359647},{"type":"v1","addr":"172.21.15.89:6837","nonce":546359647}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126}],"filesystems":[]} 2023-10-10T20:43:55.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.314+0000 7f9532cd9700 1 -- 172.21.15.89:0/2380796862 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f95100673a0 msgr2=0x7f9510069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:55.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.314+0000 7f9532cd9700 1 --2- 172.21.15.89:0/2380796862 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f95100673a0 0x7f9510069850 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f952c15b100 tx=0x7f95280091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:55.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.314+0000 7f9532cd9700 1 -- 172.21.15.89:0/2380796862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f952c0b4830 msgr2=0x7f952c15b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:55.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.314+0000 7f9532cd9700 1 --2- 172.21.15.89:0/2380796862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f952c0b4830 0x7f952c15b4e0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f952c0b2250 tx=0x7f9524009040 comp rx=0 tx=0).stop 2023-10-10T20:43:55.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.314+0000 7f9532cd9700 1 -- 172.21.15.89:0/2380796862 shutdown_connections 2023-10-10T20:43:55.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.314+0000 7f9532cd9700 1 --2- 172.21.15.89:0/2380796862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f952c0b4830 0x7f952c15b4e0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:55.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.314+0000 7f9532cd9700 1 --2- 172.21.15.89:0/2380796862 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9518004810 0x7f952c0c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:55.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.314+0000 7f9532cd9700 1 --2- 172.21.15.89:0/2380796862 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f95100673a0 0x7f9510069850 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:55.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.314+0000 7f9532cd9700 1 --2- 172.21.15.89:0/2380796862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f952c146070 0x7f952c0b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:55.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.314+0000 7f9532cd9700 1 -- 172.21.15.89:0/2380796862 >> 172.21.15.89:0/2380796862 conn(0x7f952c0d1130 msgr2=0x7f952c0af200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:55.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.314+0000 7f9532cd9700 1 -- 172.21.15.89:0/2380796862 shutdown_connections 2023-10-10T20:43:55.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.314+0000 7f9532cd9700 1 -- 172.21.15.89:0/2380796862 wait complete. 2023-10-10T20:43:55.320 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 126 2023-10-10T20:43:55.328 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:43:55.328 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:43:55.328 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:43:55.329 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:43:55.329 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:43:55.329 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:43:55.329 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:43:55.330 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:43:55.330 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:43:55.330 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:43:55.331 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:43:55.331 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:43:55.331 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:43:55.331 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:43:55.332 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:43:55.332 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:43:55.337 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:43:55.338 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:43:55.339 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:43:55.339 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:43:55.342 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:43:55.342 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:43:55.343 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:43:55.344 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:43:55.380 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:43:55.381 DEBUG:teuthology.parallel:result is None 2023-10-10T20:43:55.381 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:43:55.382 DEBUG:teuthology.parallel:result is None 2023-10-10T20:43:55.384 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:43:55.385 DEBUG:teuthology.parallel:result is None 2023-10-10T20:43:55.385 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:43:55.386 DEBUG:teuthology.parallel:result is None 2023-10-10T20:43:55.386 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:43:55.453 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:43:55.454 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-10T20:43:55.467 INFO:tasks.ceph.mds.b.smithi111.stdout:starting mds.b at 2023-10-10T20:43:55.467 INFO:tasks.ceph.mds.d.smithi111.stdout:starting mds.d at 2023-10-10T20:43:55.468 INFO:tasks.ceph.mds.b.smithi111.stderr:2023-10-10T20:43:55.465+0000 7f68536c7700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-10T20:43:55.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.506+0000 7f172a0bb700 1 Processor -- start 2023-10-10T20:43:55.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.506+0000 7f172a0bb700 1 -- start start 2023-10-10T20:43:55.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.507+0000 7f172a0bb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1724159db0 0x7f1724069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:55.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.507+0000 7f172a0bb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17240c96a0 con 0x7f172406a2b0 2023-10-10T20:43:55.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.507+0000 7f172a0bb700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17240cb9e0 con 0x7f17241599e0 2023-10-10T20:43:55.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.507+0000 7f172a0bb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17240cab60 con 0x7f1724159db0 2023-10-10T20:43:55.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.507+0000 7f1723fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f172406a2b0 0x7f172406a710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35732/0 (socket says 172.21.15.89:35732) 2023-10-10T20:43:55.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.507+0000 7f1723fff700 1 -- 172.21.15.89:0/1039596953 learned_addr learned my addr 172.21.15.89:0/1039596953 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:55.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.507+0000 7f1722ffd700 1 --2- 172.21.15.89:0/1039596953 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1724159db0 0x7f1724069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:55.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.508+0000 7f17227fc700 1 -- 172.21.15.89:0/1039596953 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458679201 0 0) 0x7f17240c96a0 con 0x7f172406a2b0 2023-10-10T20:43:55.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.508+0000 7f17227fc700 1 -- 172.21.15.89:0/1039596953 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f170c003570 con 0x7f172406a2b0 2023-10-10T20:43:55.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.508+0000 7f17227fc700 1 -- 172.21.15.89:0/1039596953 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207855286 0 0) 0x7f17240cb9e0 con 0x7f17241599e0 2023-10-10T20:43:55.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.508+0000 7f17227fc700 1 -- 172.21.15.89:0/1039596953 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17240c96a0 con 0x7f17241599e0 2023-10-10T20:43:55.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.508+0000 7f1722ffd700 1 -- 172.21.15.89:0/1039596953 >> v1:172.21.15.111:6789/0 conn(0x7f17241599e0 legacy=0x7f1724069080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:55.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.508+0000 7f1722ffd700 1 -- 172.21.15.89:0/1039596953 >> v1:172.21.15.89:6789/0 conn(0x7f172406a2b0 legacy=0x7f172406a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:55.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.508+0000 7f1722ffd700 1 -- 172.21.15.89:0/1039596953 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17240cb420 con 0x7f1724159db0 2023-10-10T20:43:55.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.508+0000 7f1722ffd700 1 --2- 172.21.15.89:0/1039596953 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1724159db0 0x7f1724069ac0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1718009910 tx=0x7f17180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5da3a0bbccbb332 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:55.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.508+0000 7f17227fc700 1 -- 172.21.15.89:0/1039596953 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718011630 con 0x7f1724159db0 2023-10-10T20:43:55.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.508+0000 7f17227fc700 1 -- 172.21.15.89:0/1039596953 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1718011790 con 0x7f1724159db0 2023-10-10T20:43:55.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.508+0000 7f17227fc700 1 -- 172.21.15.89:0/1039596953 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718011b30 con 0x7f1724159db0 2023-10-10T20:43:55.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.509+0000 7f172a0bb700 1 -- 172.21.15.89:0/1039596953 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1724159db0 msgr2=0x7f1724069ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:55.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.509+0000 7f172a0bb700 1 --2- 172.21.15.89:0/1039596953 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1724159db0 0x7f1724069ac0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1718009910 tx=0x7f17180091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:55.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.509+0000 7f172a0bb700 1 -- 172.21.15.89:0/1039596953 shutdown_connections 2023-10-10T20:43:55.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.509+0000 7f172a0bb700 1 --2- 172.21.15.89:0/1039596953 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1724159db0 0x7f1724069ac0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:55.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.509+0000 7f172a0bb700 1 -- 172.21.15.89:0/1039596953 >> 172.21.15.89:0/1039596953 conn(0x7f17240c9860 msgr2=0x7f17240c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:55.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.509+0000 7f172a0bb700 1 -- 172.21.15.89:0/1039596953 shutdown_connections 2023-10-10T20:43:55.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.509+0000 7f172a0bb700 1 -- 172.21.15.89:0/1039596953 wait complete. 2023-10-10T20:43:55.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.510+0000 7f172a0bb700 1 Processor -- start 2023-10-10T20:43:55.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.510+0000 7f172a0bb700 1 -- start start 2023-10-10T20:43:55.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f172a0bb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f172406a2b0 0x7f1724155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:55.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f172a0bb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17241599e0 0x7f172406e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:55.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f172a0bb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1724159db0 0x7f1724146820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:55.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f172a0bb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17241558b0 con 0x7f1724159db0 2023-10-10T20:43:55.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f172a0bb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172406ef20 con 0x7f17241599e0 2023-10-10T20:43:55.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f172a0bb700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172416d930 con 0x7f172406a2b0 2023-10-10T20:43:55.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f1723fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1724159db0 0x7f1724146820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:55.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f1723fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1724159db0 0x7f1724146820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46606/0 (socket says 172.21.15.89:46606) 2023-10-10T20:43:55.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f1723fff700 1 -- 172.21.15.89:0/2294557495 learned_addr learned my addr 172.21.15.89:0/2294557495 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:55.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f1722ffd700 1 --2- 172.21.15.89:0/2294557495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17241599e0 0x7f172406e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:55.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f17237fe700 1 --2- 172.21.15.89:0/2294557495 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f172406a2b0 0x7f1724155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:55.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f1723fff700 1 -- 172.21.15.89:0/2294557495 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f172406a2b0 msgr2=0x7f1724155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:55.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f1723fff700 1 --2- 172.21.15.89:0/2294557495 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f172406a2b0 0x7f1724155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:55.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f1723fff700 1 -- 172.21.15.89:0/2294557495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17241599e0 msgr2=0x7f172406e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:55.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f1723fff700 1 --2- 172.21.15.89:0/2294557495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17241599e0 0x7f172406e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:55.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f1723fff700 1 -- 172.21.15.89:0/2294557495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1718009420 con 0x7f1724159db0 2023-10-10T20:43:55.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f1722ffd700 1 --2- 172.21.15.89:0/2294557495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17241599e0 0x7f172406e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:55.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f1723fff700 1 --2- 172.21.15.89:0/2294557495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1724159db0 0x7f1724146820 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f171c00c7c0 tx=0x7f171c007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:55.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f170b7fe700 1 -- 172.21.15.89:0/2294557495 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c005c00 con 0x7f1724159db0 2023-10-10T20:43:55.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f170b7fe700 1 -- 172.21.15.89:0/2294557495 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171c005d80 con 0x7f1724159db0 2023-10-10T20:43:55.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f172a0bb700 1 -- 172.21.15.89:0/2294557495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172416d000 con 0x7f1724159db0 2023-10-10T20:43:55.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f170b7fe700 1 -- 172.21.15.89:0/2294557495 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c007320 con 0x7f1724159db0 2023-10-10T20:43:55.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.511+0000 7f172a0bb700 1 -- 172.21.15.89:0/2294557495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172416d5c0 con 0x7f1724159db0 2023-10-10T20:43:55.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.512+0000 7f172a0bb700 1 -- 172.21.15.89:0/2294557495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172404c3c0 con 0x7f1724159db0 2023-10-10T20:43:55.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.513+0000 7f170b7fe700 1 -- 172.21.15.89:0/2294557495 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f171c007690 con 0x7f1724159db0 2023-10-10T20:43:55.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.513+0000 7f170b7fe700 1 --2- 172.21.15.89:0/2294557495 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1710067350 0x7f1710069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:55.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.513+0000 7f170b7fe700 1 -- 172.21.15.89:0/2294557495 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f171c083e30 con 0x7f1724159db0 2023-10-10T20:43:55.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.513+0000 7f17237fe700 1 --2- 172.21.15.89:0/2294557495 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1710067350 0x7f1710069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:55.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.513+0000 7f17237fe700 1 --2- 172.21.15.89:0/2294557495 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1710067350 0x7f1710069800 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f1714004140 tx=0x7f1714007f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:55.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.517+0000 7f170b7fe700 1 -- 172.21.15.89:0/2294557495 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f171c050dc0 con 0x7f1724159db0 2023-10-10T20:43:55.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:55.715+0000 7f172a0bb700 1 -- 172.21.15.89:0/2294557495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f172416da70 con 0x7f1724159db0 2023-10-10T20:43:56.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.553+0000 7f170b7fe700 1 -- 172.21.15.89:0/2294557495 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v194) v1 ==== 97+0+0 (secure 0 0 0) 0x7f171c017020 con 0x7f1724159db0 2023-10-10T20:43:56.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.555+0000 7f172a0bb700 1 -- 172.21.15.89:0/2294557495 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1710067350 msgr2=0x7f1710069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:56.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.555+0000 7f172a0bb700 1 --2- 172.21.15.89:0/2294557495 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1710067350 0x7f1710069800 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f1714004140 tx=0x7f1714007f90 comp rx=0 tx=0).stop 2023-10-10T20:43:56.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.555+0000 7f172a0bb700 1 -- 172.21.15.89:0/2294557495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1724159db0 msgr2=0x7f1724146820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:56.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.555+0000 7f172a0bb700 1 --2- 172.21.15.89:0/2294557495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1724159db0 0x7f1724146820 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f171c00c7c0 tx=0x7f171c007dd0 comp rx=0 tx=0).stop 2023-10-10T20:43:56.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.555+0000 7f172a0bb700 1 -- 172.21.15.89:0/2294557495 shutdown_connections 2023-10-10T20:43:56.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.555+0000 7f172a0bb700 1 --2- 172.21.15.89:0/2294557495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1724159db0 0x7f1724146820 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:56.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.555+0000 7f172a0bb700 1 --2- 172.21.15.89:0/2294557495 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f172406a2b0 0x7f1724155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:56.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.555+0000 7f172a0bb700 1 --2- 172.21.15.89:0/2294557495 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1710067350 0x7f1710069800 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:56.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.555+0000 7f172a0bb700 1 --2- 172.21.15.89:0/2294557495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17241599e0 0x7f172406e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:56.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.555+0000 7f172a0bb700 1 -- 172.21.15.89:0/2294557495 >> 172.21.15.89:0/2294557495 conn(0x7f17240c9860 msgr2=0x7f1724156410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:56.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.555+0000 7f172a0bb700 1 -- 172.21.15.89:0/2294557495 shutdown_connections 2023-10-10T20:43:56.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.555+0000 7f172a0bb700 1 -- 172.21.15.89:0/2294557495 wait complete. 2023-10-10T20:43:56.561 INFO:teuthology.orchestra.run.smithi089.stderr: removed all blocklist entries 2023-10-10T20:43:56.571 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:43:56.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.687+0000 7fde5854a700 1 Processor -- start 2023-10-10T20:43:56.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.687+0000 7fde5854a700 1 -- start start 2023-10-10T20:43:56.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde5854a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde5006bcd0 0x7fde500caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:56.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde5854a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5015d7e0 con 0x7fde5015df50 2023-10-10T20:43:56.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde5854a700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5015dac0 con 0x7fde5006c0a0 2023-10-10T20:43:56.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde5854a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5015aa30 con 0x7fde5006bcd0 2023-10-10T20:43:56.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde56ae7700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fde5015df50 0x7fde500c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35742/0 (socket says 172.21.15.89:35742) 2023-10-10T20:43:56.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde56ae7700 1 -- 172.21.15.89:0/2075468747 learned_addr learned my addr 172.21.15.89:0/2075468747 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:56.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde562e6700 1 --2- 172.21.15.89:0/2075468747 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde5006bcd0 0x7fde500caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:56.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde552e4700 1 -- 172.21.15.89:0/2075468747 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4122930553 0 0) 0x7fde5015d7e0 con 0x7fde5015df50 2023-10-10T20:43:56.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde552e4700 1 -- 172.21.15.89:0/2075468747 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde38003570 con 0x7fde5015df50 2023-10-10T20:43:56.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde552e4700 1 -- 172.21.15.89:0/2075468747 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2510884915 0 0) 0x7fde5015dac0 con 0x7fde5006c0a0 2023-10-10T20:43:56.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde552e4700 1 -- 172.21.15.89:0/2075468747 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde5015d7e0 con 0x7fde5006c0a0 2023-10-10T20:43:56.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde552e4700 1 -- 172.21.15.89:0/2075468747 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1567519072 0 0) 0x7fde38003570 con 0x7fde5015df50 2023-10-10T20:43:56.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde552e4700 1 -- 172.21.15.89:0/2075468747 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde5006bcd0 msgr2=0x7fde500caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:56.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde552e4700 1 --2- 172.21.15.89:0/2075468747 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde5006bcd0 0x7fde500caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:56.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde552e4700 1 -- 172.21.15.89:0/2075468747 >> v1:172.21.15.111:6789/0 conn(0x7fde5006c0a0 legacy=0x7fde5015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:56.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde552e4700 1 -- 172.21.15.89:0/2075468747 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5015ad10 con 0x7fde5015df50 2023-10-10T20:43:56.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde552e4700 1 -- 172.21.15.89:0/2075468747 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fde40001470 con 0x7fde5015df50 2023-10-10T20:43:56.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde552e4700 1 -- 172.21.15.89:0/2075468747 >> v1:172.21.15.89:6789/0 conn(0x7fde5015df50 legacy=0x7fde500c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:56.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde552e4700 1 --2- 172.21.15.89:0/2075468747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde38003c20 0x7fde38004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:56.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde552e4700 1 --2- 172.21.15.89:0/2075468747 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde5006bcd0 0x7fde38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:56.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde552e4700 1 --2- 172.21.15.89:0/2075468747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde38004d50 0x7fde380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:56.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde552e4700 1 -- 172.21.15.89:0/2075468747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5015aa30 con 0x7fde38003c20 2023-10-10T20:43:56.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde552e4700 1 -- 172.21.15.89:0/2075468747 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3800bbf0 con 0x7fde38004d50 2023-10-10T20:43:56.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.688+0000 7fde552e4700 1 -- 172.21.15.89:0/2075468747 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3800bdb0 con 0x7fde5006bcd0 2023-10-10T20:43:56.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.689+0000 7fde562e6700 1 --2- 172.21.15.89:0/2075468747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde38003c20 0x7fde38004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:56.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.689+0000 7fde55ae5700 1 --2- 172.21.15.89:0/2075468747 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde5006bcd0 0x7fde38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:56.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.689+0000 7fde56ae7700 1 --2- 172.21.15.89:0/2075468747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde38004d50 0x7fde380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:56.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.689+0000 7fde562e6700 1 -- 172.21.15.89:0/2075468747 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde5006bcd0 msgr2=0x7fde38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:56.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.689+0000 7fde562e6700 1 --2- 172.21.15.89:0/2075468747 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde5006bcd0 0x7fde38004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:56.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.689+0000 7fde562e6700 1 -- 172.21.15.89:0/2075468747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde38004d50 msgr2=0x7fde380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:56.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.689+0000 7fde562e6700 1 --2- 172.21.15.89:0/2075468747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde38004d50 0x7fde380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:56.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.689+0000 7fde562e6700 1 -- 172.21.15.89:0/2075468747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde38003570 con 0x7fde38003c20 2023-10-10T20:43:56.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.689+0000 7fde562e6700 1 --2- 172.21.15.89:0/2075468747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde38003c20 0x7fde38004010 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fde4800e1c0 tx=0x7fde4800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466a87bd5e424275 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:56.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.689+0000 7fde552e4700 1 -- 172.21.15.89:0/2075468747 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4800ea70 con 0x7fde38003c20 2023-10-10T20:43:56.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.689+0000 7fde552e4700 1 -- 172.21.15.89:0/2075468747 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4800f040 con 0x7fde38003c20 2023-10-10T20:43:56.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.689+0000 7fde552e4700 1 -- 172.21.15.89:0/2075468747 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4800ee20 con 0x7fde38003c20 2023-10-10T20:43:56.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.690+0000 7fde5854a700 1 -- 172.21.15.89:0/2075468747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde38003c20 msgr2=0x7fde38004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:56.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.690+0000 7fde5854a700 1 --2- 172.21.15.89:0/2075468747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde38003c20 0x7fde38004010 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fde4800e1c0 tx=0x7fde4800b040 comp rx=0 tx=0).stop 2023-10-10T20:43:56.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.690+0000 7fde56ae7700 1 -- 172.21.15.89:0/2075468747 reap_dead start 2023-10-10T20:43:56.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.690+0000 7fde5854a700 1 -- 172.21.15.89:0/2075468747 shutdown_connections 2023-10-10T20:43:56.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.690+0000 7fde5854a700 1 -- 172.21.15.89:0/2075468747 >> 172.21.15.89:0/2075468747 conn(0x7fde500ce600 msgr2=0x7fde500cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:56.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.690+0000 7fde5854a700 1 -- 172.21.15.89:0/2075468747 shutdown_connections 2023-10-10T20:43:56.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.690+0000 7fde5854a700 1 -- 172.21.15.89:0/2075468747 wait complete. 2023-10-10T20:43:56.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.691+0000 7fde5854a700 1 Processor -- start 2023-10-10T20:43:56.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.691+0000 7fde5854a700 1 -- start start 2023-10-10T20:43:56.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.691+0000 7fde5854a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde38003c20 0x7fde5016b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:56.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.691+0000 7fde5854a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde50158ee0 0x7fde50159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:56.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde5854a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde500c7d10 0x7fde500c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:56.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde5854a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde50159890 con 0x7fde500c7d10 2023-10-10T20:43:56.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde5854a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde500c69d0 con 0x7fde38003c20 2023-10-10T20:43:56.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde5854a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde500c6c70 con 0x7fde50158ee0 2023-10-10T20:43:56.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde56ae7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde500c7d10 0x7fde500c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:56.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde56ae7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde500c7d10 0x7fde500c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46624/0 (socket says 172.21.15.89:46624) 2023-10-10T20:43:56.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde56ae7700 1 -- 172.21.15.89:0/2085203105 learned_addr learned my addr 172.21.15.89:0/2085203105 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:56.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde562e6700 1 --2- 172.21.15.89:0/2085203105 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde38003c20 0x7fde5016b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:56.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde56ae7700 1 -- 172.21.15.89:0/2085203105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde50158ee0 msgr2=0x7fde50159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:56.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde55ae5700 1 --2- 172.21.15.89:0/2085203105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde50158ee0 0x7fde50159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:56.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde56ae7700 1 --2- 172.21.15.89:0/2085203105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde50158ee0 0x7fde50159350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:56.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde56ae7700 1 -- 172.21.15.89:0/2085203105 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde38003c20 msgr2=0x7fde5016b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:56.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde56ae7700 1 --2- 172.21.15.89:0/2085203105 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde38003c20 0x7fde5016b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:56.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde56ae7700 1 -- 172.21.15.89:0/2085203105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde500c6f50 con 0x7fde500c7d10 2023-10-10T20:43:56.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde562e6700 1 --2- 172.21.15.89:0/2085203105 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde38003c20 0x7fde5016b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:56.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde56ae7700 1 --2- 172.21.15.89:0/2085203105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde500c7d10 0x7fde500c8180 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fde40000c00 tx=0x7fde500ce4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:56.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde4e7fc700 1 -- 172.21.15.89:0/2085203105 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40018030 con 0x7fde500c7d10 2023-10-10T20:43:56.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde4e7fc700 1 -- 172.21.15.89:0/2085203105 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde400077d0 con 0x7fde500c7d10 2023-10-10T20:43:56.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde5854a700 1 -- 172.21.15.89:0/2085203105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde500c7230 con 0x7fde500c7d10 2023-10-10T20:43:56.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde4e7fc700 1 -- 172.21.15.89:0/2085203105 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4000fad0 con 0x7fde500c7d10 2023-10-10T20:43:56.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.692+0000 7fde5854a700 1 -- 172.21.15.89:0/2085203105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde500c77f0 con 0x7fde500c7d10 2023-10-10T20:43:56.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.693+0000 7fde5854a700 1 -- 172.21.15.89:0/2085203105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde500c7a10 con 0x7fde500c7d10 2023-10-10T20:43:56.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.694+0000 7fde4e7fc700 1 -- 172.21.15.89:0/2085203105 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fde4001b070 con 0x7fde500c7d10 2023-10-10T20:43:56.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.694+0000 7fde4e7fc700 1 --2- 172.21.15.89:0/2085203105 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fde3c067320 0x7fde3c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:56.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.694+0000 7fde4e7fc700 1 -- 172.21.15.89:0/2085203105 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fde40019070 con 0x7fde500c7d10 2023-10-10T20:43:56.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.694+0000 7fde562e6700 1 --2- 172.21.15.89:0/2085203105 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fde3c067320 0x7fde3c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:56.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.694+0000 7fde562e6700 1 --2- 172.21.15.89:0/2085203105 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fde3c067320 0x7fde3c0697d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fde48009c10 tx=0x7fde4801a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:56.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.698+0000 7fde4e7fc700 1 -- 172.21.15.89:0/2085203105 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde400516e0 con 0x7fde500c7d10 2023-10-10T20:43:56.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.958+0000 7fde5854a700 1 -- 172.21.15.89:0/2085203105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fde500ca160 con 0x7fde500c7d10 2023-10-10T20:43:56.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.958+0000 7fde4e7fc700 1 -- 172.21.15.89:0/2085203105 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v26) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fde40054d00 con 0x7fde500c7d10 2023-10-10T20:43:56.961 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:56.962 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:43:56.962 INFO:teuthology.orchestra.run.smithi089.stdout: "auth_dump": [ 2023-10-10T20:43:56.962 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:56.962 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.a", 2023-10-10T20:43:56.963 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlZLSNLhAA8WZTrsht6tAOP+gxaTXKMg==", 2023-10-10T20:43:56.963 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:56.963 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:43:56.963 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:43:56.963 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:43:56.964 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:56.964 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:56.964 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:56.964 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.b", 2023-10-10T20:43:56.965 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgyDEORAAMMoSxPFZc0bCIwWy8UsXgA==", 2023-10-10T20:43:56.965 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:56.965 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:43:56.965 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:43:56.965 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:43:56.966 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:56.966 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:56.966 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:56.966 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.c", 2023-10-10T20:43:56.967 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlMFxSNRAASuQEJFghLIOkTQWa4ckTvg==", 2023-10-10T20:43:56.967 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:56.967 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:43:56.967 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:43:56.967 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:43:56.968 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:56.968 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.968 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:56.968 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:56.968 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.d", 2023-10-10T20:43:56.969 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlPkvrBBAA2VLJmv2t8szTEGtCVwalJw==", 2023-10-10T20:43:56.969 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:56.969 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:43:56.969 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:43:56.969 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:43:56.970 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:56.970 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.970 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:56.970 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:56.970 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.0", 2023-10-10T20:43:56.971 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQArtSVltOcKHRAAk8+zUus04oTVI9KvfhjlXg==", 2023-10-10T20:43:56.971 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:56.971 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:43:56.971 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:43:56.972 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:56.972 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.972 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:56.972 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:56.972 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.1", 2023-10-10T20:43:56.973 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAutSVly8P0GhAAe1bvR63+ptJg84lizMNdQw==", 2023-10-10T20:43:56.973 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:56.973 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:43:56.973 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:43:56.973 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:56.974 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.974 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:56.974 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:56.974 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.2", 2023-10-10T20:43:56.974 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAxtSVl0x9fGRAAEKi7+6EW5mzPbebmz0+sgQ==", 2023-10-10T20:43:56.975 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:56.975 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:43:56.975 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:43:56.975 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:56.976 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:56.976 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:56.976 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.3", 2023-10-10T20:43:56.976 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA0tSVlfJyaGBAAa+8w54U6v7+npbdufKJruA==", 2023-10-10T20:43:56.977 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:56.977 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:43:56.977 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:43:56.977 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:56.977 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:56.978 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:56.978 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.4", 2023-10-10T20:43:56.978 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA6tSVlEQ/QFxAA/PRB3c90BuAlF6xekGfX9A==", 2023-10-10T20:43:56.978 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:56.979 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:43:56.979 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:43:56.979 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:56.979 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:56.980 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:56.980 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.5", 2023-10-10T20:43:56.980 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA9tSVlMOKrFRAAuBnIsEz2VmeSZi9XWzjlzA==", 2023-10-10T20:43:56.980 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:56.980 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:43:56.981 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:43:56.981 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:56.981 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.981 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:56.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:56.982 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.6", 2023-10-10T20:43:56.982 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBAtSVlH7PQFBAAHr6c7d4TYJEu/KSFEf+EOA==", 2023-10-10T20:43:56.982 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:56.982 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:43:56.982 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:43:56.983 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:56.983 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:56.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:56.983 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.7", 2023-10-10T20:43:56.984 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBDtSVlAhC1FBAAuE2BwAA/f7VOkZfVdqAG7A==", 2023-10-10T20:43:56.984 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:56.984 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-10T20:43:56.984 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-10T20:43:56.984 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:56.985 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:56.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:56.985 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.0", 2023-10-10T20:43:56.985 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlRd3HCBAAb0QP2VfTS7aelno1mFpGAA==", 2023-10-10T20:43:56.985 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:56.986 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:43:56.986 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow r", 2023-10-10T20:43:56.986 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:43:56.986 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:56.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:56.987 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.1", 2023-10-10T20:43:56.987 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlgBouERAAqPdE0KvJU3M53B6Y6AH5GQ==", 2023-10-10T20:43:56.987 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:56.988 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:43:56.988 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:43:56.988 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:43:56.988 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:43:56.988 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:56.989 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:56.989 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.2", 2023-10-10T20:43:56.989 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlXA4tDhAAoPVYQjECpzD+C67mnsY5IA==", 2023-10-10T20:43:56.989 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:56.990 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:43:56.990 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:43:56.990 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:43:56.990 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:43:56.990 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:56.991 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:56.991 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.3", 2023-10-10T20:43:56.991 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAotSVlZ5KGFhAAuKmZwbOfmmw1nAj5ry6tHw==", 2023-10-10T20:43:56.991 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-10T20:43:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-10T20:43:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-10T20:43:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-10T20:43:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:56.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:56.993 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.admin", 2023-10-10T20:43:56.993 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlgpL2EhAAoDOki002rfBrauTgfu481Q==", 2023-10-10T20:43:56.993 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:56.993 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:43:56.994 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-10T20:43:56.994 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-10T20:43:56.994 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:56.994 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:56.995 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:56.995 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.x", 2023-10-10T20:43:56.995 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlBhaJKxAArraBxT5xWg515LfpJ7MlbA==", 2023-10-10T20:43:56.995 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:56.995 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:43:56.996 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:43:56.996 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:56.996 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.996 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-10T20:43:56.996 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:56.997 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.y", 2023-10-10T20:43:56.997 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAntSVlTZIyJxAAWAOsitHwsIxJWwp3b1Z98Q==", 2023-10-10T20:43:56.997 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-10T20:43:56.997 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-10T20:43:56.997 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-10T20:43:56.998 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-10T20:43:56.998 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.998 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:56.998 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-10T20:43:56.998 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:43:57.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.960+0000 7fde5854a700 1 -- 172.21.15.89:0/2085203105 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fde3c067320 msgr2=0x7fde3c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.960+0000 7fde5854a700 1 --2- 172.21.15.89:0/2085203105 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fde3c067320 0x7fde3c0697d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fde48009c10 tx=0x7fde4801a000 comp rx=0 tx=0).stop 2023-10-10T20:43:57.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.960+0000 7fde5854a700 1 -- 172.21.15.89:0/2085203105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde500c7d10 msgr2=0x7fde500c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.960+0000 7fde5854a700 1 --2- 172.21.15.89:0/2085203105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde500c7d10 0x7fde500c8180 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fde40000c00 tx=0x7fde500ce4d0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.960+0000 7fde5854a700 1 -- 172.21.15.89:0/2085203105 shutdown_connections 2023-10-10T20:43:57.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.960+0000 7fde5854a700 1 --2- 172.21.15.89:0/2085203105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde500c7d10 0x7fde500c8180 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.960+0000 7fde5854a700 1 --2- 172.21.15.89:0/2085203105 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fde3c067320 0x7fde3c0697d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.960+0000 7fde5854a700 1 --2- 172.21.15.89:0/2085203105 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde38003c20 0x7fde5016b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.960+0000 7fde5854a700 1 --2- 172.21.15.89:0/2085203105 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde50158ee0 0x7fde50159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.960+0000 7fde5854a700 1 -- 172.21.15.89:0/2085203105 >> 172.21.15.89:0/2085203105 conn(0x7fde500ce600 msgr2=0x7fde50155f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:57.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.960+0000 7fde5854a700 1 -- 172.21.15.89:0/2085203105 shutdown_connections 2023-10-10T20:43:57.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:56.960+0000 7fde5854a700 1 -- 172.21.15.89:0/2085203105 wait complete. 2023-10-10T20:43:57.003 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:43:57.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.115+0000 7f64bfb4e700 1 Processor -- start 2023-10-10T20:43:57.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.115+0000 7f64bfb4e700 1 -- start start 2023-10-10T20:43:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.116+0000 7f64bfb4e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64b8141500 0x7f64b80c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.116+0000 7f64bfb4e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f64b806c0a0 0x7f64b80c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.116+0000 7f64bfb4e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b815c2a0 con 0x7f64b806bcd0 2023-10-10T20:43:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.116+0000 7f64bfb4e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64b80c9880 con 0x7f64b806c0a0 2023-10-10T20:43:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.116+0000 7f64bfb4e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b815b690 con 0x7f64b8141500 2023-10-10T20:43:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.116+0000 7f64bd8ea700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f64b806bcd0 0x7f64b8140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35756/0 (socket says 172.21.15.89:35756) 2023-10-10T20:43:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.116+0000 7f64bd8ea700 1 -- 172.21.15.89:0/2250548039 learned_addr learned my addr 172.21.15.89:0/2250548039 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:57.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64be0eb700 1 --2- 172.21.15.89:0/2250548039 >> v2:172.21.15.111:3300/0 conn(0x7f64b806c0a0 0x7f64b80c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bc8e8700 1 -- 172.21.15.89:0/2250548039 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843739273 0 0) 0x7f64b815c2a0 con 0x7f64b806bcd0 2023-10-10T20:43:57.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bd0e9700 1 --2- 172.21.15.89:0/2250548039 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64b8141500 0x7f64b80c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bc8e8700 1 -- 172.21.15.89:0/2250548039 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64a4003570 con 0x7f64b806bcd0 2023-10-10T20:43:57.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bc8e8700 1 -- 172.21.15.89:0/2250548039 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 932417372 0 0) 0x7f64a4003570 con 0x7f64b806bcd0 2023-10-10T20:43:57.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bc8e8700 1 -- 172.21.15.89:0/2250548039 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64b8141500 msgr2=0x7f64b80c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:57.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bc8e8700 1 --2- 172.21.15.89:0/2250548039 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64b8141500 0x7f64b80c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bc8e8700 1 -- 172.21.15.89:0/2250548039 >> v2:172.21.15.111:3300/0 conn(0x7f64b806c0a0 msgr2=0x7f64b80c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:57.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bc8e8700 1 --2- 172.21.15.89:0/2250548039 >> v2:172.21.15.111:3300/0 conn(0x7f64b806c0a0 0x7f64b80c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bc8e8700 1 -- 172.21.15.89:0/2250548039 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b815b970 con 0x7f64b806bcd0 2023-10-10T20:43:57.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bc8e8700 1 -- 172.21.15.89:0/2250548039 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f64b4001470 con 0x7f64b806bcd0 2023-10-10T20:43:57.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bc8e8700 1 -- 172.21.15.89:0/2250548039 >> v1:172.21.15.89:6789/0 conn(0x7f64b806bcd0 legacy=0x7f64b8140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bc8e8700 1 --2- 172.21.15.89:0/2250548039 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64b8141500 0x7f64a4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bc8e8700 1 --2- 172.21.15.89:0/2250548039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64a4003fe0 0x7f64a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bc8e8700 1 --2- 172.21.15.89:0/2250548039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64a4004d50 0x7f64a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bc8e8700 1 -- 172.21.15.89:0/2250548039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80c9880 con 0x7f64a4003fe0 2023-10-10T20:43:57.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bc8e8700 1 -- 172.21.15.89:0/2250548039 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b815b690 con 0x7f64a4004d50 2023-10-10T20:43:57.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bc8e8700 1 -- 172.21.15.89:0/2250548039 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a400ba90 con 0x7f64b8141500 2023-10-10T20:43:57.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bd0e9700 1 --2- 172.21.15.89:0/2250548039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64a4003fe0 0x7f64a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bd8ea700 1 --2- 172.21.15.89:0/2250548039 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64b8141500 0x7f64a4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64be0eb700 1 --2- 172.21.15.89:0/2250548039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64a4004d50 0x7f64a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bd0e9700 1 -- 172.21.15.89:0/2250548039 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64b8141500 msgr2=0x7f64a4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:57.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bd0e9700 1 --2- 172.21.15.89:0/2250548039 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64b8141500 0x7f64a4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bd0e9700 1 -- 172.21.15.89:0/2250548039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64a4004d50 msgr2=0x7f64a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:57.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bd0e9700 1 --2- 172.21.15.89:0/2250548039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64a4004d50 0x7f64a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bd0e9700 1 -- 172.21.15.89:0/2250548039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a4003570 con 0x7f64a4003fe0 2023-10-10T20:43:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64be0eb700 1 --2- 172.21.15.89:0/2250548039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64a4004d50 0x7f64a40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.117+0000 7f64bd0e9700 1 --2- 172.21.15.89:0/2250548039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64a4003fe0 0x7f64a4004810 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009c00 tx=0x7f64a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0dfa7651351d4d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.118+0000 7f64bc8e8700 1 -- 172.21.15.89:0/2250548039 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a800e6b0 con 0x7f64a4003fe0 2023-10-10T20:43:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.118+0000 7f64bc8e8700 1 -- 172.21.15.89:0/2250548039 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a800e810 con 0x7f64a4003fe0 2023-10-10T20:43:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.118+0000 7f64bc8e8700 1 -- 172.21.15.89:0/2250548039 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a800ebd0 con 0x7f64a4003fe0 2023-10-10T20:43:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.118+0000 7f64bfb4e700 1 -- 172.21.15.89:0/2250548039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64a4003fe0 msgr2=0x7f64a4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.118+0000 7f64bfb4e700 1 --2- 172.21.15.89:0/2250548039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64a4003fe0 0x7f64a4004810 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009c00 tx=0x7f64a80091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.118+0000 7f64be0eb700 1 -- 172.21.15.89:0/2250548039 reap_dead start 2023-10-10T20:43:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.118+0000 7f64bfb4e700 1 -- 172.21.15.89:0/2250548039 shutdown_connections 2023-10-10T20:43:57.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.118+0000 7f64bfb4e700 1 -- 172.21.15.89:0/2250548039 >> 172.21.15.89:0/2250548039 conn(0x7f64b80ce600 msgr2=0x7f64b80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:57.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.118+0000 7f64bfb4e700 1 -- 172.21.15.89:0/2250548039 shutdown_connections 2023-10-10T20:43:57.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.118+0000 7f64bfb4e700 1 -- 172.21.15.89:0/2250548039 wait complete. 2023-10-10T20:43:57.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.119+0000 7f64bfb4e700 1 Processor -- start 2023-10-10T20:43:57.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.119+0000 7f64bfb4e700 1 -- start start 2023-10-10T20:43:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.120+0000 7f64bfb4e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64a4003fe0 0x7f64b81573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.120+0000 7f64bfb4e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64b813e9b0 0x7f64b816ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.120+0000 7f64bfb4e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64b816bf50 0x7f64b80b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.120+0000 7f64bfb4e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b816c3c0 con 0x7f64b813e9b0 2023-10-10T20:43:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.120+0000 7f64bfb4e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80c7d20 con 0x7f64b816bf50 2023-10-10T20:43:57.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.120+0000 7f64bfb4e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80c7700 con 0x7f64a4003fe0 2023-10-10T20:43:57.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.120+0000 7f64bd0e9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64b813e9b0 0x7f64b816ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.120+0000 7f64bd0e9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64b813e9b0 0x7f64b816ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46634/0 (socket says 172.21.15.89:46634) 2023-10-10T20:43:57.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.120+0000 7f64bd0e9700 1 -- 172.21.15.89:0/3415279122 learned_addr learned my addr 172.21.15.89:0/3415279122 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:57.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.120+0000 7f64bd8ea700 1 --2- 172.21.15.89:0/3415279122 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64a4003fe0 0x7f64b81573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.120+0000 7f64be0eb700 1 --2- 172.21.15.89:0/3415279122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64b816bf50 0x7f64b80b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.120+0000 7f64bd0e9700 1 -- 172.21.15.89:0/3415279122 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64a4003fe0 msgr2=0x7f64b81573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.120+0000 7f64bd0e9700 1 --2- 172.21.15.89:0/3415279122 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64a4003fe0 0x7f64b81573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.120+0000 7f64bd0e9700 1 -- 172.21.15.89:0/3415279122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64b816bf50 msgr2=0x7f64b80b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.120+0000 7f64bd0e9700 1 --2- 172.21.15.89:0/3415279122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64b816bf50 0x7f64b80b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.120+0000 7f64bd0e9700 1 -- 172.21.15.89:0/3415279122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b80c79e0 con 0x7f64b813e9b0 2023-10-10T20:43:57.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.120+0000 7f64bd0e9700 1 --2- 172.21.15.89:0/3415279122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64b813e9b0 0x7f64b816ba10 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f64a4004730 tx=0x7f64a801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:57.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.121+0000 7f64adffb700 1 -- 172.21.15.89:0/3415279122 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8022070 con 0x7f64b813e9b0 2023-10-10T20:43:57.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.121+0000 7f64adffb700 1 -- 172.21.15.89:0/3415279122 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a800e980 con 0x7f64b813e9b0 2023-10-10T20:43:57.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.121+0000 7f64bfb4e700 1 -- 172.21.15.89:0/3415279122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b80c7fc0 con 0x7f64b813e9b0 2023-10-10T20:43:57.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.121+0000 7f64adffb700 1 -- 172.21.15.89:0/3415279122 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8027470 con 0x7f64b813e9b0 2023-10-10T20:43:57.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.121+0000 7f64bfb4e700 1 -- 172.21.15.89:0/3415279122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b816c7c0 con 0x7f64b813e9b0 2023-10-10T20:43:57.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.122+0000 7f64bfb4e700 1 -- 172.21.15.89:0/3415279122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b804c3c0 con 0x7f64b813e9b0 2023-10-10T20:43:57.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.122+0000 7f64adffb700 1 -- 172.21.15.89:0/3415279122 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f64a801c450 con 0x7f64b813e9b0 2023-10-10T20:43:57.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.122+0000 7f64adffb700 1 --2- 172.21.15.89:0/3415279122 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f649c067320 0x7f649c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.122+0000 7f64adffb700 1 -- 172.21.15.89:0/3415279122 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f64a8086290 con 0x7f64b813e9b0 2023-10-10T20:43:57.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.123+0000 7f64bd8ea700 1 --2- 172.21.15.89:0/3415279122 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f649c067320 0x7f649c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.123+0000 7f64bd8ea700 1 --2- 172.21.15.89:0/3415279122 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f649c067320 0x7f649c0697d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f64b40011e0 tx=0x7f64b806e470 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:57.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.127+0000 7f64adffb700 1 -- 172.21.15.89:0/3415279122 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64a8053370 con 0x7f64b813e9b0 2023-10-10T20:43:57.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.309+0000 7f64bfb4e700 1 -- 172.21.15.89:0/3415279122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64b80c7b70 con 0x7f64b813e9b0 2023-10-10T20:43:57.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.310+0000 7f64adffb700 1 -- 172.21.15.89:0/3415279122 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) v1 ==== 74+0+12144 (secure 0 0 0) 0x7f64a8056990 con 0x7f64b813e9b0 2023-10-10T20:43:57.313 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:57.313 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":194,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:43:56.550497+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:43:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.312+0000 7f64bfb4e700 1 -- 172.21.15.89:0/3415279122 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f649c067320 msgr2=0x7f649c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.312+0000 7f64bfb4e700 1 --2- 172.21.15.89:0/3415279122 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f649c067320 0x7f649c0697d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f64b40011e0 tx=0x7f64b806e470 comp rx=0 tx=0).stop 2023-10-10T20:43:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.312+0000 7f64bfb4e700 1 -- 172.21.15.89:0/3415279122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64b813e9b0 msgr2=0x7f64b816ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.312+0000 7f64bfb4e700 1 --2- 172.21.15.89:0/3415279122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64b813e9b0 0x7f64b816ba10 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f64a4004730 tx=0x7f64a801a000 comp rx=0 tx=0).stop 2023-10-10T20:43:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.312+0000 7f64bfb4e700 1 -- 172.21.15.89:0/3415279122 shutdown_connections 2023-10-10T20:43:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.312+0000 7f64bfb4e700 1 --2- 172.21.15.89:0/3415279122 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f649c067320 0x7f649c0697d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.312+0000 7f64bfb4e700 1 --2- 172.21.15.89:0/3415279122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64b816bf50 0x7f64b80b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.312+0000 7f64bfb4e700 1 --2- 172.21.15.89:0/3415279122 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64a4003fe0 0x7f64b81573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.312+0000 7f64bfb4e700 1 --2- 172.21.15.89:0/3415279122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64b813e9b0 0x7f64b816ba10 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.312+0000 7f64bfb4e700 1 -- 172.21.15.89:0/3415279122 >> 172.21.15.89:0/3415279122 conn(0x7f64b80ce600 msgr2=0x7f64b8157c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:57.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.312+0000 7f64bfb4e700 1 -- 172.21.15.89:0/3415279122 shutdown_connections 2023-10-10T20:43:57.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.312+0000 7f64bfb4e700 1 -- 172.21.15.89:0/3415279122 wait complete. 2023-10-10T20:43:57.327 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:43:57.328 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:43:57.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.442+0000 7fec828e5700 1 Processor -- start 2023-10-10T20:43:57.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.442+0000 7fec828e5700 1 -- start start 2023-10-10T20:43:57.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.443+0000 7fec828e5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec7c06bcd0 0x7fec7c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.443+0000 7fec828e5700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fec7c06c0a0 0x7fec7c15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.443+0000 7fec828e5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec7c15d7e0 con 0x7fec7c15df50 2023-10-10T20:43:57.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.443+0000 7fec828e5700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7c15dac0 con 0x7fec7c06c0a0 2023-10-10T20:43:57.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.443+0000 7fec828e5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c15aa30 con 0x7fec7c06bcd0 2023-10-10T20:43:57.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.443+0000 7fec80e82700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fec7c15df50 0x7fec7c0c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35764/0 (socket says 172.21.15.89:35764) 2023-10-10T20:43:57.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.443+0000 7fec80e82700 1 -- 172.21.15.89:0/3177850150 learned_addr learned my addr 172.21.15.89:0/3177850150 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:57.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7b7fe700 1 --2- 172.21.15.89:0/3177850150 >> v2:172.21.15.111:3300/0 conn(0x7fec7c06c0a0 0x7fec7c15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7affd700 1 -- 172.21.15.89:0/3177850150 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2084095684 0 0) 0x7fec7c15d7e0 con 0x7fec7c15df50 2023-10-10T20:43:57.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7bfff700 1 --2- 172.21.15.89:0/3177850150 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec7c06bcd0 0x7fec7c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7affd700 1 -- 172.21.15.89:0/3177850150 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec64003570 con 0x7fec7c15df50 2023-10-10T20:43:57.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7affd700 1 -- 172.21.15.89:0/3177850150 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3859720820 0 0) 0x7fec64003570 con 0x7fec7c15df50 2023-10-10T20:43:57.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7affd700 1 -- 172.21.15.89:0/3177850150 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec7c06bcd0 msgr2=0x7fec7c0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:57.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7affd700 1 --2- 172.21.15.89:0/3177850150 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec7c06bcd0 0x7fec7c0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7affd700 1 -- 172.21.15.89:0/3177850150 >> v2:172.21.15.111:3300/0 conn(0x7fec7c06c0a0 msgr2=0x7fec7c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:57.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7affd700 1 --2- 172.21.15.89:0/3177850150 >> v2:172.21.15.111:3300/0 conn(0x7fec7c06c0a0 0x7fec7c15a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7affd700 1 -- 172.21.15.89:0/3177850150 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c15ad10 con 0x7fec7c15df50 2023-10-10T20:43:57.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7affd700 1 -- 172.21.15.89:0/3177850150 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fec740017f0 con 0x7fec7c15df50 2023-10-10T20:43:57.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7affd700 1 -- 172.21.15.89:0/3177850150 >> v1:172.21.15.89:6789/0 conn(0x7fec7c15df50 legacy=0x7fec7c0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7affd700 1 --2- 172.21.15.89:0/3177850150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec64003a60 0x7fec64003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7affd700 1 --2- 172.21.15.89:0/3177850150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec64004810 0x7fec64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7affd700 1 --2- 172.21.15.89:0/3177850150 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec7c06bcd0 0x7fec640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7affd700 1 -- 172.21.15.89:0/3177850150 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c15dac0 con 0x7fec64003a60 2023-10-10T20:43:57.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7affd700 1 -- 172.21.15.89:0/3177850150 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c15aa30 con 0x7fec64004810 2023-10-10T20:43:57.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7affd700 1 -- 172.21.15.89:0/3177850150 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6400bae0 con 0x7fec7c06bcd0 2023-10-10T20:43:57.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7b7fe700 1 --2- 172.21.15.89:0/3177850150 >> v2:172.21.15.111:3300/0 conn(0x7fec7c06c0a0 0x7fec7c15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:43:57.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7bfff700 1 --2- 172.21.15.89:0/3177850150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec64003a60 0x7fec64003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec80e82700 1 --2- 172.21.15.89:0/3177850150 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec7c06bcd0 0x7fec640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7b7fe700 1 --2- 172.21.15.89:0/3177850150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec64004810 0x7fec64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7bfff700 1 -- 172.21.15.89:0/3177850150 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec7c06bcd0 msgr2=0x7fec640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:57.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7bfff700 1 --2- 172.21.15.89:0/3177850150 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec7c06bcd0 0x7fec640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7bfff700 1 -- 172.21.15.89:0/3177850150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec64004810 msgr2=0x7fec64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:57.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.444+0000 7fec7bfff700 1 --2- 172.21.15.89:0/3177850150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec64004810 0x7fec64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.445+0000 7fec7bfff700 1 -- 172.21.15.89:0/3177850150 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec64003570 con 0x7fec64003a60 2023-10-10T20:43:57.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.445+0000 7fec7b7fe700 1 --2- 172.21.15.89:0/3177850150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec64004810 0x7fec64004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:57.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.445+0000 7fec7bfff700 1 --2- 172.21.15.89:0/3177850150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec64003a60 0x7fec64003e70 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009c00 tx=0x7fec6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e446c697b418aa95 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:57.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.445+0000 7fec7affd700 1 -- 172.21.15.89:0/3177850150 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c00e710 con 0x7fec64003a60 2023-10-10T20:43:57.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.445+0000 7fec7affd700 1 -- 172.21.15.89:0/3177850150 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6c00e870 con 0x7fec64003a60 2023-10-10T20:43:57.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.445+0000 7fec7affd700 1 -- 172.21.15.89:0/3177850150 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c00ec30 con 0x7fec64003a60 2023-10-10T20:43:57.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.445+0000 7fec828e5700 1 -- 172.21.15.89:0/3177850150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec64003a60 msgr2=0x7fec64003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.445+0000 7fec828e5700 1 --2- 172.21.15.89:0/3177850150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec64003a60 0x7fec64003e70 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009c00 tx=0x7fec6c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.445+0000 7fec80e82700 1 -- 172.21.15.89:0/3177850150 reap_dead start 2023-10-10T20:43:57.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.445+0000 7fec828e5700 1 -- 172.21.15.89:0/3177850150 shutdown_connections 2023-10-10T20:43:57.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.445+0000 7fec828e5700 1 -- 172.21.15.89:0/3177850150 >> 172.21.15.89:0/3177850150 conn(0x7fec7c0ce600 msgr2=0x7fec7c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:57.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.445+0000 7fec828e5700 1 -- 172.21.15.89:0/3177850150 shutdown_connections 2023-10-10T20:43:57.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.445+0000 7fec828e5700 1 -- 172.21.15.89:0/3177850150 wait complete. 2023-10-10T20:43:57.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.446+0000 7fec828e5700 1 Processor -- start 2023-10-10T20:43:57.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.447+0000 7fec828e5700 1 -- start start 2023-10-10T20:43:57.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.447+0000 7fec828e5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec64003a60 0x7fec7c16d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.447+0000 7fec828e5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec7c0c4ed0 0x7fec7c15c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.447+0000 7fec828e5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec7c145e30 0x7fec7c1462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.447+0000 7fec828e5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c0b14d0 con 0x7fec7c0c4ed0 2023-10-10T20:43:57.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.447+0000 7fec828e5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c146810 con 0x7fec7c145e30 2023-10-10T20:43:57.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.447+0000 7fec828e5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c16e270 con 0x7fec64003a60 2023-10-10T20:43:57.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.447+0000 7fec7b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec7c0c4ed0 0x7fec7c15c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.447+0000 7fec7b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec7c0c4ed0 0x7fec7c15c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46654/0 (socket says 172.21.15.89:46654) 2023-10-10T20:43:57.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.447+0000 7fec7b7fe700 1 -- 172.21.15.89:0/2505310534 learned_addr learned my addr 172.21.15.89:0/2505310534 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:57.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.447+0000 7fec80e82700 1 --2- 172.21.15.89:0/2505310534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec7c145e30 0x7fec7c1462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.448+0000 7fec7bfff700 1 --2- 172.21.15.89:0/2505310534 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec64003a60 0x7fec7c16d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.448+0000 7fec7b7fe700 1 -- 172.21.15.89:0/2505310534 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec64003a60 msgr2=0x7fec7c16d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.448+0000 7fec7b7fe700 1 --2- 172.21.15.89:0/2505310534 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec64003a60 0x7fec7c16d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.448+0000 7fec7b7fe700 1 -- 172.21.15.89:0/2505310534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec7c145e30 msgr2=0x7fec7c1462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.448+0000 7fec7b7fe700 1 --2- 172.21.15.89:0/2505310534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec7c145e30 0x7fec7c1462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.448+0000 7fec7b7fe700 1 -- 172.21.15.89:0/2505310534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c16d9d0 con 0x7fec7c0c4ed0 2023-10-10T20:43:57.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.448+0000 7fec7bfff700 1 --2- 172.21.15.89:0/2505310534 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec64003a60 0x7fec7c16d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:57.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.448+0000 7fec7b7fe700 1 --2- 172.21.15.89:0/2505310534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec7c0c4ed0 0x7fec7c15c3d0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fec70010cd0 tx=0x7fec70008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:57.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.448+0000 7fec63fff700 1 -- 172.21.15.89:0/2505310534 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7000e9f0 con 0x7fec7c0c4ed0 2023-10-10T20:43:57.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.448+0000 7fec63fff700 1 -- 172.21.15.89:0/2505310534 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7000eb50 con 0x7fec7c0c4ed0 2023-10-10T20:43:57.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.448+0000 7fec828e5700 1 -- 172.21.15.89:0/2505310534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7c16dcb0 con 0x7fec7c0c4ed0 2023-10-10T20:43:57.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.448+0000 7fec63fff700 1 -- 172.21.15.89:0/2505310534 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7000eed0 con 0x7fec7c0c4ed0 2023-10-10T20:43:57.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.448+0000 7fec828e5700 1 -- 172.21.15.89:0/2505310534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7c0c5340 con 0x7fec7c0c4ed0 2023-10-10T20:43:57.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.449+0000 7fec828e5700 1 -- 172.21.15.89:0/2505310534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7c0b1680 con 0x7fec7c0c4ed0 2023-10-10T20:43:57.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.449+0000 7fec63fff700 1 -- 172.21.15.89:0/2505310534 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fec700143f0 con 0x7fec7c0c4ed0 2023-10-10T20:43:57.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.450+0000 7fec63fff700 1 --2- 172.21.15.89:0/2505310534 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fec68067350 0x7fec68069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.450+0000 7fec7bfff700 1 --2- 172.21.15.89:0/2505310534 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fec68067350 0x7fec68069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.450+0000 7fec63fff700 1 -- 172.21.15.89:0/2505310534 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fec70084e50 con 0x7fec7c0c4ed0 2023-10-10T20:43:57.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.450+0000 7fec7bfff700 1 --2- 172.21.15.89:0/2505310534 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fec68067350 0x7fec68069800 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0091e0 tx=0x7fec6c01a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:57.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.454+0000 7fec63fff700 1 -- 172.21.15.89:0/2505310534 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec7c0b1680 con 0x7fec7c0c4ed0 2023-10-10T20:43:57.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.639+0000 7fec828e5700 1 -- 172.21.15.89:0/2505310534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec7c0b1680 con 0x7fec7c0c4ed0 2023-10-10T20:43:57.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.640+0000 7fec63fff700 1 -- 172.21.15.89:0/2505310534 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v194) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec7c0b1680 con 0x7fec7c0c4ed0 2023-10-10T20:43:57.643 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:43:57.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.642+0000 7fec828e5700 1 -- 172.21.15.89:0/2505310534 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fec68067350 msgr2=0x7fec68069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.642+0000 7fec828e5700 1 --2- 172.21.15.89:0/2505310534 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fec68067350 0x7fec68069800 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0091e0 tx=0x7fec6c01a000 comp rx=0 tx=0).stop 2023-10-10T20:43:57.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.642+0000 7fec828e5700 1 -- 172.21.15.89:0/2505310534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec7c0c4ed0 msgr2=0x7fec7c15c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.642+0000 7fec828e5700 1 --2- 172.21.15.89:0/2505310534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec7c0c4ed0 0x7fec7c15c3d0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fec70010cd0 tx=0x7fec70008040 comp rx=0 tx=0).stop 2023-10-10T20:43:57.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.642+0000 7fec828e5700 1 -- 172.21.15.89:0/2505310534 shutdown_connections 2023-10-10T20:43:57.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.642+0000 7fec828e5700 1 --2- 172.21.15.89:0/2505310534 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fec68067350 0x7fec68069800 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.642+0000 7fec828e5700 1 --2- 172.21.15.89:0/2505310534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec7c145e30 0x7fec7c1462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.642+0000 7fec828e5700 1 --2- 172.21.15.89:0/2505310534 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fec64003a60 0x7fec7c16d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.642+0000 7fec828e5700 1 --2- 172.21.15.89:0/2505310534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec7c0c4ed0 0x7fec7c15c3d0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.642+0000 7fec828e5700 1 -- 172.21.15.89:0/2505310534 >> 172.21.15.89:0/2505310534 conn(0x7fec7c0ce600 msgr2=0x7fec7c15a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:57.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.642+0000 7fec828e5700 1 -- 172.21.15.89:0/2505310534 shutdown_connections 2023-10-10T20:43:57.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.642+0000 7fec828e5700 1 -- 172.21.15.89:0/2505310534 wait complete. 2023-10-10T20:43:57.657 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:43:57.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.769+0000 7f63e9523700 1 Processor -- start 2023-10-10T20:43:57.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.769+0000 7f63e9523700 1 -- start start 2023-10-10T20:43:57.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.770+0000 7f63e9523700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f63e4069560 0x7f63e40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.770+0000 7f63e9523700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e4069ee0 con 0x7f63e4159c60 2023-10-10T20:43:57.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.770+0000 7f63e9523700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e415a160 con 0x7f63e4159890 2023-10-10T20:43:57.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.770+0000 7f63e9523700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e40d13a0 con 0x7f63e4069560 2023-10-10T20:43:57.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.770+0000 7f63da7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f63e4159c60 0x7f63e415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35768/0 (socket says 172.21.15.89:35768) 2023-10-10T20:43:57.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.770+0000 7f63da7fc700 1 -- 172.21.15.89:0/2232254570 learned_addr learned my addr 172.21.15.89:0/2232254570 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:57.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.770+0000 7f63e37fe700 1 --2- 172.21.15.89:0/2232254570 >> v2:172.21.15.111:3300/0 conn(0x7f63e4069560 0x7f63e40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.770+0000 7f63e27fc700 1 -- 172.21.15.89:0/2232254570 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2893786308 0 0) 0x7f63e4069ee0 con 0x7f63e4159c60 2023-10-10T20:43:57.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.770+0000 7f63e27fc700 1 -- 172.21.15.89:0/2232254570 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63d0003570 con 0x7f63e4159c60 2023-10-10T20:43:57.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e27fc700 1 -- 172.21.15.89:0/2232254570 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3555485156 0 0) 0x7f63d0003570 con 0x7f63e4159c60 2023-10-10T20:43:57.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e27fc700 1 -- 172.21.15.89:0/2232254570 >> v2:172.21.15.111:3300/0 conn(0x7f63e4069560 msgr2=0x7f63e40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:57.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e27fc700 1 --2- 172.21.15.89:0/2232254570 >> v2:172.21.15.111:3300/0 conn(0x7f63e4069560 0x7f63e40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e27fc700 1 -- 172.21.15.89:0/2232254570 >> v1:172.21.15.111:6789/0 conn(0x7f63e4159890 legacy=0x7f63e415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e27fc700 1 -- 172.21.15.89:0/2232254570 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e415e140 con 0x7f63e4159c60 2023-10-10T20:43:57.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e27fc700 1 -- 172.21.15.89:0/2232254570 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f63cc001470 con 0x7f63e4159c60 2023-10-10T20:43:57.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e27fc700 1 -- 172.21.15.89:0/2232254570 >> v1:172.21.15.89:6789/0 conn(0x7f63e4159c60 legacy=0x7f63e415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e27fc700 1 --2- 172.21.15.89:0/2232254570 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63d0003a60 0x7f63d0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e27fc700 1 --2- 172.21.15.89:0/2232254570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63d0004810 0x7f63d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e27fc700 1 --2- 172.21.15.89:0/2232254570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63d00051c0 0x7f63d000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e27fc700 1 -- 172.21.15.89:0/2232254570 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e40d13a0 con 0x7f63d00051c0 2023-10-10T20:43:57.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e27fc700 1 -- 172.21.15.89:0/2232254570 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d000c0d0 con 0x7f63d0004810 2023-10-10T20:43:57.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e27fc700 1 -- 172.21.15.89:0/2232254570 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d000c290 con 0x7f63d0003a60 2023-10-10T20:43:57.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e37fe700 1 --2- 172.21.15.89:0/2232254570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63d00051c0 0x7f63d000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63da7fc700 1 --2- 172.21.15.89:0/2232254570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63d0004810 0x7f63d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e2ffd700 1 --2- 172.21.15.89:0/2232254570 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63d0003a60 0x7f63d0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e37fe700 1 -- 172.21.15.89:0/2232254570 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63d0003a60 msgr2=0x7f63d0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:57.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e37fe700 1 --2- 172.21.15.89:0/2232254570 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63d0003a60 0x7f63d0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e37fe700 1 -- 172.21.15.89:0/2232254570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63d0004810 msgr2=0x7f63d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:57.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e37fe700 1 --2- 172.21.15.89:0/2232254570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63d0004810 0x7f63d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e37fe700 1 -- 172.21.15.89:0/2232254570 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d0003570 con 0x7f63d00051c0 2023-10-10T20:43:57.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e2ffd700 1 --2- 172.21.15.89:0/2232254570 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63d0003a60 0x7f63d0003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:57.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e37fe700 1 -- 172.21.15.89:0/2232254570 reap_dead start 2023-10-10T20:43:57.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e37fe700 1 --2- 172.21.15.89:0/2232254570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63d00051c0 0x7f63d000ba60 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00e210 tx=0x7f63dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84450e2712365f7e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:57.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e27fc700 1 -- 172.21.15.89:0/2232254570 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc00eac0 con 0x7f63d00051c0 2023-10-10T20:43:57.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e27fc700 1 -- 172.21.15.89:0/2232254570 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63dc00f040 con 0x7f63d00051c0 2023-10-10T20:43:57.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.771+0000 7f63e27fc700 1 -- 172.21.15.89:0/2232254570 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc00ee70 con 0x7f63d00051c0 2023-10-10T20:43:57.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.772+0000 7f63e9523700 1 -- 172.21.15.89:0/2232254570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63d00051c0 msgr2=0x7f63d000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.772+0000 7f63e9523700 1 --2- 172.21.15.89:0/2232254570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63d00051c0 0x7f63d000ba60 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00e210 tx=0x7f63dc00b040 comp rx=0 tx=0).stop 2023-10-10T20:43:57.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.772+0000 7f63e9523700 1 -- 172.21.15.89:0/2232254570 shutdown_connections 2023-10-10T20:43:57.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.772+0000 7f63e9523700 1 --2- 172.21.15.89:0/2232254570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63d00051c0 0x7f63d000ba60 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.772+0000 7f63e9523700 1 -- 172.21.15.89:0/2232254570 >> 172.21.15.89:0/2232254570 conn(0x7f63e4068e20 msgr2=0x7f63e40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:57.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.772+0000 7f63e9523700 1 -- 172.21.15.89:0/2232254570 shutdown_connections 2023-10-10T20:43:57.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.772+0000 7f63e9523700 1 -- 172.21.15.89:0/2232254570 wait complete. 2023-10-10T20:43:57.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.773+0000 7f63e9523700 1 Processor -- start 2023-10-10T20:43:57.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.773+0000 7f63e9523700 1 -- start start 2023-10-10T20:43:57.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.774+0000 7f63e9523700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63d00051c0 0x7f63e4156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.774+0000 7f63e9523700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63e4157f90 0x7f63e4158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.774+0000 7f63e9523700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63e4151320 0x7f63e41517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.774+0000 7f63e9523700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e4152c10 con 0x7f63d00051c0 2023-10-10T20:43:57.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.774+0000 7f63e9523700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e4152090 con 0x7f63e4157f90 2023-10-10T20:43:57.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.774+0000 7f63e9523700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e4152370 con 0x7f63e4151320 2023-10-10T20:43:57.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.774+0000 7f63e2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63d00051c0 0x7f63e4156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.774+0000 7f63e2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63d00051c0 0x7f63e4156470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46668/0 (socket says 172.21.15.89:46668) 2023-10-10T20:43:57.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.774+0000 7f63e2ffd700 1 -- 172.21.15.89:0/541392494 learned_addr learned my addr 172.21.15.89:0/541392494 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:57.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.774+0000 7f63da7fc700 1 --2- 172.21.15.89:0/541392494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63e4157f90 0x7f63e4158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.774+0000 7f63e37fe700 1 --2- 172.21.15.89:0/541392494 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63e4151320 0x7f63e41517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.774+0000 7f63e2ffd700 1 -- 172.21.15.89:0/541392494 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63e4151320 msgr2=0x7f63e41517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.774+0000 7f63e2ffd700 1 --2- 172.21.15.89:0/541392494 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63e4151320 0x7f63e41517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.774+0000 7f63e2ffd700 1 -- 172.21.15.89:0/541392494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63e4157f90 msgr2=0x7f63e4158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.774+0000 7f63e2ffd700 1 --2- 172.21.15.89:0/541392494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63e4157f90 0x7f63e4158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.774+0000 7f63e2ffd700 1 -- 172.21.15.89:0/541392494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e4152650 con 0x7f63d00051c0 2023-10-10T20:43:57.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.774+0000 7f63e2ffd700 1 --2- 172.21.15.89:0/541392494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63d00051c0 0x7f63e4156470 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f63d400e580 tx=0x7f63d4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:57.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.775+0000 7f63db7fe700 1 -- 172.21.15.89:0/541392494 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d400ee50 con 0x7f63d00051c0 2023-10-10T20:43:57.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.775+0000 7f63db7fe700 1 -- 172.21.15.89:0/541392494 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d4010040 con 0x7f63d00051c0 2023-10-10T20:43:57.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.775+0000 7f63e9523700 1 -- 172.21.15.89:0/541392494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e4152930 con 0x7f63d00051c0 2023-10-10T20:43:57.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.775+0000 7f63db7fe700 1 -- 172.21.15.89:0/541392494 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d4011670 con 0x7f63d00051c0 2023-10-10T20:43:57.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.775+0000 7f63e9523700 1 -- 172.21.15.89:0/541392494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e4151ce0 con 0x7f63d00051c0 2023-10-10T20:43:57.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.776+0000 7f63e9523700 1 -- 172.21.15.89:0/541392494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e404c3c0 con 0x7f63d00051c0 2023-10-10T20:43:57.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.776+0000 7f63db7fe700 1 -- 172.21.15.89:0/541392494 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f63d4011a60 con 0x7f63d00051c0 2023-10-10T20:43:57.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.776+0000 7f63db7fe700 1 --2- 172.21.15.89:0/541392494 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f63c4067320 0x7f63c40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:57.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.776+0000 7f63db7fe700 1 -- 172.21.15.89:0/541392494 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f63d4083b50 con 0x7f63d00051c0 2023-10-10T20:43:57.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.776+0000 7f63da7fc700 1 --2- 172.21.15.89:0/541392494 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f63c4067320 0x7f63c40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:57.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.777+0000 7f63da7fc700 1 --2- 172.21.15.89:0/541392494 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f63c4067320 0x7f63c40697d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f63cc007fb0 tx=0x7f63cc001e50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:57.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.780+0000 7f63db7fe700 1 -- 172.21.15.89:0/541392494 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63d404d5a0 con 0x7f63d00051c0 2023-10-10T20:43:57.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.964+0000 7f63e9523700 1 -- 172.21.15.89:0/541392494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f63e406a5b0 con 0x7f63d00051c0 2023-10-10T20:43:57.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.965+0000 7f63db7fe700 1 -- 172.21.15.89:0/541392494 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v194) v1 ==== 84+0+65 (secure 0 0 0) 0x7f63d4050ad0 con 0x7f63d00051c0 2023-10-10T20:43:57.968 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:43:57.968 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:43:57.969 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:43:57.969 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2023-10-10T20:43:57.969 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2023-10-10T20:43:57.970 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:43:57.970 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:43:57.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.967+0000 7f63e9523700 1 -- 172.21.15.89:0/541392494 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f63c4067320 msgr2=0x7f63c40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.967+0000 7f63e9523700 1 --2- 172.21.15.89:0/541392494 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f63c4067320 0x7f63c40697d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f63cc007fb0 tx=0x7f63cc001e50 comp rx=0 tx=0).stop 2023-10-10T20:43:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.967+0000 7f63e9523700 1 -- 172.21.15.89:0/541392494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63d00051c0 msgr2=0x7f63e4156470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.967+0000 7f63e9523700 1 --2- 172.21.15.89:0/541392494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63d00051c0 0x7f63e4156470 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f63d400e580 tx=0x7f63d4009040 comp rx=0 tx=0).stop 2023-10-10T20:43:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.967+0000 7f63e9523700 1 -- 172.21.15.89:0/541392494 shutdown_connections 2023-10-10T20:43:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.967+0000 7f63e9523700 1 --2- 172.21.15.89:0/541392494 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f63e4151320 0x7f63e41517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.967+0000 7f63e9523700 1 --2- 172.21.15.89:0/541392494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63d00051c0 0x7f63e4156470 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.967+0000 7f63e9523700 1 --2- 172.21.15.89:0/541392494 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f63c4067320 0x7f63c40697d0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.967+0000 7f63e9523700 1 --2- 172.21.15.89:0/541392494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63e4157f90 0x7f63e4158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.967+0000 7f63e9523700 1 -- 172.21.15.89:0/541392494 >> 172.21.15.89:0/541392494 conn(0x7f63e4068e20 msgr2=0x7f63e41553c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.967+0000 7f63e9523700 1 -- 172.21.15.89:0/541392494 shutdown_connections 2023-10-10T20:43:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:57.967+0000 7f63e9523700 1 -- 172.21.15.89:0/541392494 wait complete. 2023-10-10T20:43:57.981 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:43:57.982 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:43:58.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.098+0000 7fb868a5c700 1 Processor -- start 2023-10-10T20:43:58.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.098+0000 7fb868a5c700 1 -- start start 2023-10-10T20:43:58.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.099+0000 7fb868a5c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8640b03c0 0x7fb8640cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:58.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.099+0000 7fb868a5c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb8641591e0 0x7fb864159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:58.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.099+0000 7fb868a5c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8640cfa00 con 0x7fb8640b0790 2023-10-10T20:43:58.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.099+0000 7fb868a5c700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8640cb580 con 0x7fb8641591e0 2023-10-10T20:43:58.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.099+0000 7fb868a5c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8640cf160 con 0x7fb8640b03c0 2023-10-10T20:43:58.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.099+0000 7fb862ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb8640b0790 0x7fb8640b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35776/0 (socket says 172.21.15.89:35776) 2023-10-10T20:43:58.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.099+0000 7fb862ffd700 1 -- 172.21.15.89:0/3985748755 learned_addr learned my addr 172.21.15.89:0/3985748755 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:58.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.099+0000 7fb8637fe700 1 --2- 172.21.15.89:0/3985748755 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8640b03c0 0x7fb8640cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:58.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.099+0000 7fb863fff700 1 --2- 172.21.15.89:0/3985748755 >> v2:172.21.15.111:3300/0 conn(0x7fb8641591e0 0x7fb864159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:58.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.099+0000 7fb8627fc700 1 -- 172.21.15.89:0/3985748755 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294190775 0 0) 0x7fb8640cfa00 con 0x7fb8640b0790 2023-10-10T20:43:58.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.099+0000 7fb8627fc700 1 -- 172.21.15.89:0/3985748755 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb84c003570 con 0x7fb8640b0790 2023-10-10T20:43:58.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb8627fc700 1 -- 172.21.15.89:0/3985748755 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4243329718 0 0) 0x7fb84c003570 con 0x7fb8640b0790 2023-10-10T20:43:58.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb8627fc700 1 -- 172.21.15.89:0/3985748755 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8640b03c0 msgr2=0x7fb8640cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb8627fc700 1 --2- 172.21.15.89:0/3985748755 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8640b03c0 0x7fb8640cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb8627fc700 1 -- 172.21.15.89:0/3985748755 >> v2:172.21.15.111:3300/0 conn(0x7fb8641591e0 msgr2=0x7fb864159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb8627fc700 1 --2- 172.21.15.89:0/3985748755 >> v2:172.21.15.111:3300/0 conn(0x7fb8641591e0 0x7fb864159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:58.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb8627fc700 1 -- 172.21.15.89:0/3985748755 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8640cf440 con 0x7fb8640b0790 2023-10-10T20:43:58.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb8627fc700 1 -- 172.21.15.89:0/3985748755 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fb858001470 con 0x7fb8640b0790 2023-10-10T20:43:58.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb8627fc700 1 -- 172.21.15.89:0/3985748755 >> v1:172.21.15.89:6789/0 conn(0x7fb8640b0790 legacy=0x7fb8640b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:58.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb8627fc700 1 --2- 172.21.15.89:0/3985748755 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8640b03c0 0x7fb84c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:58.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb8627fc700 1 --2- 172.21.15.89:0/3985748755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb84c003fe0 0x7fb84c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:58.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb8627fc700 1 --2- 172.21.15.89:0/3985748755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb84c004d50 0x7fb84c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:58.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb8627fc700 1 -- 172.21.15.89:0/3985748755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8640cb580 con 0x7fb84c003fe0 2023-10-10T20:43:58.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb8627fc700 1 -- 172.21.15.89:0/3985748755 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8640cf160 con 0x7fb84c004d50 2023-10-10T20:43:58.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb8627fc700 1 -- 172.21.15.89:0/3985748755 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c00ba90 con 0x7fb8640b03c0 2023-10-10T20:43:58.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb862ffd700 1 --2- 172.21.15.89:0/3985748755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb84c003fe0 0x7fb84c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:58.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb8637fe700 1 --2- 172.21.15.89:0/3985748755 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8640b03c0 0x7fb84c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:58.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb863fff700 1 --2- 172.21.15.89:0/3985748755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb84c004d50 0x7fb84c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:58.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb862ffd700 1 -- 172.21.15.89:0/3985748755 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8640b03c0 msgr2=0x7fb84c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:58.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb862ffd700 1 --2- 172.21.15.89:0/3985748755 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8640b03c0 0x7fb84c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:58.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb862ffd700 1 -- 172.21.15.89:0/3985748755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb84c004d50 msgr2=0x7fb84c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:58.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb862ffd700 1 --2- 172.21.15.89:0/3985748755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb84c004d50 0x7fb84c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:58.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb862ffd700 1 -- 172.21.15.89:0/3985748755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84c003570 con 0x7fb84c003fe0 2023-10-10T20:43:58.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb862ffd700 1 --2- 172.21.15.89:0/3985748755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb84c003fe0 0x7fb84c004810 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fb85800c3c0 tx=0x7fb8640cb6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1465b67971c74232 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:58.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb8627fc700 1 -- 172.21.15.89:0/3985748755 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85800cc30 con 0x7fb84c003fe0 2023-10-10T20:43:58.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb8627fc700 1 -- 172.21.15.89:0/3985748755 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85800cd90 con 0x7fb84c003fe0 2023-10-10T20:43:58.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.100+0000 7fb8627fc700 1 -- 172.21.15.89:0/3985748755 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858005b70 con 0x7fb84c003fe0 2023-10-10T20:43:58.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.101+0000 7fb868a5c700 1 -- 172.21.15.89:0/3985748755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb84c003fe0 msgr2=0x7fb84c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:58.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.101+0000 7fb868a5c700 1 --2- 172.21.15.89:0/3985748755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb84c003fe0 0x7fb84c004810 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fb85800c3c0 tx=0x7fb8640cb6c0 comp rx=0 tx=0).stop 2023-10-10T20:43:58.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.101+0000 7fb863fff700 1 -- 172.21.15.89:0/3985748755 reap_dead start 2023-10-10T20:43:58.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.101+0000 7fb868a5c700 1 -- 172.21.15.89:0/3985748755 shutdown_connections 2023-10-10T20:43:58.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.101+0000 7fb868a5c700 1 -- 172.21.15.89:0/3985748755 >> 172.21.15.89:0/3985748755 conn(0x7fb864063230 msgr2=0x7fb864063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:58.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.101+0000 7fb868a5c700 1 -- 172.21.15.89:0/3985748755 shutdown_connections 2023-10-10T20:43:58.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.101+0000 7fb868a5c700 1 -- 172.21.15.89:0/3985748755 wait complete. 2023-10-10T20:43:58.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.102+0000 7fb868a5c700 1 Processor -- start 2023-10-10T20:43:58.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.102+0000 7fb868a5c700 1 -- start start 2023-10-10T20:43:58.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.103+0000 7fb868a5c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb84c003fe0 0x7fb864154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:58.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.103+0000 7fb868a5c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8640c4930 0x7fb8640c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:58.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.103+0000 7fb868a5c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8640c8b70 0x7fb8640c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:58.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.103+0000 7fb868a5c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8640c6860 con 0x7fb84c003fe0 2023-10-10T20:43:58.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.103+0000 7fb868a5c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8640c5890 con 0x7fb8640c4930 2023-10-10T20:43:58.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.103+0000 7fb868a5c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb864169a70 con 0x7fb8640c8b70 2023-10-10T20:43:58.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.103+0000 7fb8637fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb84c003fe0 0x7fb864154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:58.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.103+0000 7fb8637fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb84c003fe0 0x7fb864154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46692/0 (socket says 172.21.15.89:46692) 2023-10-10T20:43:58.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.103+0000 7fb8637fe700 1 -- 172.21.15.89:0/258862829 learned_addr learned my addr 172.21.15.89:0/258862829 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:58.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.103+0000 7fb862ffd700 1 --2- 172.21.15.89:0/258862829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8640c4930 0x7fb8640c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:58.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.103+0000 7fb863fff700 1 --2- 172.21.15.89:0/258862829 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8640c8b70 0x7fb8640c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:58.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.103+0000 7fb8637fe700 1 -- 172.21.15.89:0/258862829 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8640c8b70 msgr2=0x7fb8640c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:58.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.103+0000 7fb8637fe700 1 --2- 172.21.15.89:0/258862829 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8640c8b70 0x7fb8640c8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:58.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.103+0000 7fb8637fe700 1 -- 172.21.15.89:0/258862829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8640c4930 msgr2=0x7fb8640c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:58.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.103+0000 7fb8637fe700 1 --2- 172.21.15.89:0/258862829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8640c4930 0x7fb8640c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:58.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.103+0000 7fb8637fe700 1 -- 172.21.15.89:0/258862829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8640c4da0 con 0x7fb84c003fe0 2023-10-10T20:43:58.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.103+0000 7fb863fff700 1 --2- 172.21.15.89:0/258862829 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8640c8b70 0x7fb8640c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:43:58.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.104+0000 7fb8637fe700 1 --2- 172.21.15.89:0/258862829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb84c003fe0 0x7fb864154ea0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fb854011fb0 tx=0x7fb85400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:58.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.104+0000 7fb84b7fe700 1 -- 172.21.15.89:0/258862829 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85400e040 con 0x7fb84c003fe0 2023-10-10T20:43:58.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.104+0000 7fb84b7fe700 1 -- 172.21.15.89:0/258862829 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85400f040 con 0x7fb84c003fe0 2023-10-10T20:43:58.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.104+0000 7fb868a5c700 1 -- 172.21.15.89:0/258862829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8640c9520 con 0x7fb84c003fe0 2023-10-10T20:43:58.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.104+0000 7fb84b7fe700 1 -- 172.21.15.89:0/258862829 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb854015430 con 0x7fb84c003fe0 2023-10-10T20:43:58.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.104+0000 7fb868a5c700 1 -- 172.21.15.89:0/258862829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8640c52d0 con 0x7fb84c003fe0 2023-10-10T20:43:58.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.105+0000 7fb868a5c700 1 -- 172.21.15.89:0/258862829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb864169880 con 0x7fb84c003fe0 2023-10-10T20:43:58.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.105+0000 7fb84b7fe700 1 -- 172.21.15.89:0/258862829 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb854015840 con 0x7fb84c003fe0 2023-10-10T20:43:58.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.105+0000 7fb84b7fe700 1 --2- 172.21.15.89:0/258862829 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb850067350 0x7fb850069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:58.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.105+0000 7fb84b7fe700 1 -- 172.21.15.89:0/258862829 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb8540856b0 con 0x7fb84c003fe0 2023-10-10T20:43:58.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.105+0000 7fb862ffd700 1 --2- 172.21.15.89:0/258862829 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb850067350 0x7fb850069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:58.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.106+0000 7fb862ffd700 1 --2- 172.21.15.89:0/258862829 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb850067350 0x7fb850069800 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb8640cab70 tx=0x7fb858015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:58.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.110+0000 7fb84b7fe700 1 -- 172.21.15.89:0/258862829 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb854052580 con 0x7fb84c003fe0 2023-10-10T20:43:58.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.293+0000 7fb868a5c700 1 -- 172.21.15.89:0/258862829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb86404c3c0 con 0x7fb84c003fe0 2023-10-10T20:43:58.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.560+0000 7fb84b7fe700 1 -- 172.21.15.89:0/258862829 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v195) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb854055ba0 con 0x7fb84c003fe0 2023-10-10T20:43:58.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.564+0000 7fb868a5c700 1 -- 172.21.15.89:0/258862829 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb850067350 msgr2=0x7fb850069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:58.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.564+0000 7fb868a5c700 1 --2- 172.21.15.89:0/258862829 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb850067350 0x7fb850069800 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb8640cab70 tx=0x7fb858015000 comp rx=0 tx=0).stop 2023-10-10T20:43:58.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.564+0000 7fb868a5c700 1 -- 172.21.15.89:0/258862829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb84c003fe0 msgr2=0x7fb864154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:58.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.564+0000 7fb868a5c700 1 --2- 172.21.15.89:0/258862829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb84c003fe0 0x7fb864154ea0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fb854011fb0 tx=0x7fb85400b040 comp rx=0 tx=0).stop 2023-10-10T20:43:58.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.564+0000 7fb868a5c700 1 -- 172.21.15.89:0/258862829 shutdown_connections 2023-10-10T20:43:58.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.564+0000 7fb868a5c700 1 --2- 172.21.15.89:0/258862829 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb8640c8b70 0x7fb8640c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:58.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.564+0000 7fb868a5c700 1 --2- 172.21.15.89:0/258862829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb84c003fe0 0x7fb864154ea0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:58.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.564+0000 7fb868a5c700 1 --2- 172.21.15.89:0/258862829 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb850067350 0x7fb850069800 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:58.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.564+0000 7fb868a5c700 1 --2- 172.21.15.89:0/258862829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8640c4930 0x7fb8640c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:58.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.564+0000 7fb868a5c700 1 -- 172.21.15.89:0/258862829 >> 172.21.15.89:0/258862829 conn(0x7fb864063230 msgr2=0x7fb8640b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:58.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.565+0000 7fb868a5c700 1 -- 172.21.15.89:0/258862829 shutdown_connections 2023-10-10T20:43:58.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.565+0000 7fb868a5c700 1 -- 172.21.15.89:0/258862829 wait complete. 2023-10-10T20:43:58.570 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2023-10-10T20:43:58.582 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:43:58.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.701+0000 7fa9fe9cc700 1 Processor -- start 2023-10-10T20:43:58.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.701+0000 7fa9fe9cc700 1 -- start start 2023-10-10T20:43:58.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.702+0000 7fa9fe9cc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9f8159950 0x7fa9f80cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:58.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.702+0000 7fa9fe9cc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa9f8069c70 0x7fa9f80cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:58.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.702+0000 7fa9fe9cc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f80ce380 con 0x7fa9f80af320 2023-10-10T20:43:58.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.702+0000 7fa9fe9cc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f80cec20 con 0x7fa9f8069c70 2023-10-10T20:43:58.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.702+0000 7fa9fe9cc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f80afa90 con 0x7fa9f8159950 2023-10-10T20:43:58.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.702+0000 7fa9fcf69700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa9f8069c70 0x7fa9f80cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:58.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.702+0000 7fa9fcf69700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa9f8069c70 0x7fa9f80cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51918/0 (socket says 172.21.15.89:51918) 2023-10-10T20:43:58.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.702+0000 7fa9fcf69700 1 -- 172.21.15.89:0/169308263 learned_addr learned my addr 172.21.15.89:0/169308263 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:58.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.702+0000 7fa9f77fe700 1 --1- 172.21.15.89:0/169308263 >> v1:172.21.15.89:6789/0 conn(0x7fa9f80af320 0x7fa9f8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37196/0 (socket says 172.21.15.89:37196) 2023-10-10T20:43:58.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.702+0000 7fa9f7fff700 1 --2- 172.21.15.89:0/169308263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9f8159950 0x7fa9f80cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:58.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.703+0000 7fa9fcf69700 1 -- 172.21.15.89:0/169308263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9f8159950 msgr2=0x7fa9f80cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:58.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.703+0000 7fa9fcf69700 1 --2- 172.21.15.89:0/169308263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9f8159950 0x7fa9f80cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:58.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.703+0000 7fa9fcf69700 1 -- 172.21.15.89:0/169308263 >> v1:172.21.15.89:6789/0 conn(0x7fa9f80af320 legacy=0x7fa9f8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:58.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.703+0000 7fa9fcf69700 1 -- 172.21.15.89:0/169308263 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f80ce5d0 con 0x7fa9f8069c70 2023-10-10T20:43:58.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.703+0000 7fa9f6ffd700 1 -- 172.21.15.89:0/169308263 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686937188 0 0) 0x7fa9f80ce380 con 0x7fa9f80af320 2023-10-10T20:43:58.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.703+0000 7fa9fcf69700 1 --2- 172.21.15.89:0/169308263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9f8069c70 0x7fa9f80cf0b0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009f20 tx=0x7fa9f0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44439f0655ed565a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:58.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.703+0000 7fa9f6ffd700 1 -- 172.21.15.89:0/169308263 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f000e070 con 0x7fa9f8069c70 2023-10-10T20:43:58.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.703+0000 7fa9f6ffd700 1 -- 172.21.15.89:0/169308263 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f000ba80 con 0x7fa9f8069c70 2023-10-10T20:43:58.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.703+0000 7fa9f6ffd700 1 -- 172.21.15.89:0/169308263 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f00092e0 con 0x7fa9f8069c70 2023-10-10T20:43:58.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.703+0000 7fa9fe9cc700 1 -- 172.21.15.89:0/169308263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9f8069c70 msgr2=0x7fa9f80cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:58.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.703+0000 7fa9fe9cc700 1 --2- 172.21.15.89:0/169308263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9f8069c70 0x7fa9f80cf0b0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009f20 tx=0x7fa9f0009840 comp rx=0 tx=0).stop 2023-10-10T20:43:58.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.703+0000 7fa9fe9cc700 1 -- 172.21.15.89:0/169308263 shutdown_connections 2023-10-10T20:43:58.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.703+0000 7fa9fe9cc700 1 --2- 172.21.15.89:0/169308263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9f8069c70 0x7fa9f80cf0b0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:58.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.703+0000 7fa9fe9cc700 1 --2- 172.21.15.89:0/169308263 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9f8159950 0x7fa9f80cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:58.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.703+0000 7fa9fe9cc700 1 -- 172.21.15.89:0/169308263 >> 172.21.15.89:0/169308263 conn(0x7fa9f8068e00 msgr2=0x7fa9f80d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:58.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.703+0000 7fa9fe9cc700 1 -- 172.21.15.89:0/169308263 shutdown_connections 2023-10-10T20:43:58.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.703+0000 7fa9fe9cc700 1 -- 172.21.15.89:0/169308263 wait complete. 2023-10-10T20:43:58.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.704+0000 7fa9fe9cc700 1 Processor -- start 2023-10-10T20:43:58.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.705+0000 7fa9fe9cc700 1 -- start start 2023-10-10T20:43:58.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.705+0000 7fa9fe9cc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9f8069c70 0x7fa9f8155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:58.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.705+0000 7fa9fe9cc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9f80af320 0x7fa9f80b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:58.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.705+0000 7fa9fe9cc700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9f8159950 0x7fa9f80b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:58.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.705+0000 7fa9fe9cc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f8157cb0 con 0x7fa9f80af320 2023-10-10T20:43:58.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.705+0000 7fa9fe9cc700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f8156b10 con 0x7fa9f8069c70 2023-10-10T20:43:58.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.705+0000 7fa9fe9cc700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f8155e90 con 0x7fa9f8159950 2023-10-10T20:43:58.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.705+0000 7fa9f77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9f80af320 0x7fa9f80b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:58.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.705+0000 7fa9f77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9f80af320 0x7fa9f80b56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51930/0 (socket says 172.21.15.89:51930) 2023-10-10T20:43:58.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.706+0000 7fa9f77fe700 1 -- 172.21.15.89:0/3895424793 learned_addr learned my addr 172.21.15.89:0/3895424793 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:58.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.706+0000 7fa9f77fe700 1 -- 172.21.15.89:0/3895424793 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9f8159950 msgr2=0x7fa9f80b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:58.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.706+0000 7fa9f7fff700 1 --2- 172.21.15.89:0/3895424793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9f8069c70 0x7fa9f8155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:58.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.706+0000 7fa9fcf69700 1 --2- 172.21.15.89:0/3895424793 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9f8159950 0x7fa9f80b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:58.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.706+0000 7fa9f77fe700 1 --2- 172.21.15.89:0/3895424793 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9f8159950 0x7fa9f80b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:58.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.706+0000 7fa9f77fe700 1 -- 172.21.15.89:0/3895424793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9f8069c70 msgr2=0x7fa9f8155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:58.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.706+0000 7fa9f77fe700 1 --2- 172.21.15.89:0/3895424793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9f8069c70 0x7fa9f8155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:58.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.706+0000 7fa9f77fe700 1 -- 172.21.15.89:0/3895424793 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f0009ae0 con 0x7fa9f80af320 2023-10-10T20:43:58.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.706+0000 7fa9f7fff700 1 --2- 172.21.15.89:0/3895424793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9f8069c70 0x7fa9f8155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:58.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.706+0000 7fa9fcf69700 1 --2- 172.21.15.89:0/3895424793 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9f8159950 0x7fa9f80b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:58.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.706+0000 7fa9f77fe700 1 --2- 172.21.15.89:0/3895424793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9f80af320 0x7fa9f80b56a0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800c7e0 tx=0x7fa9e8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:58.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.706+0000 7fa9dffff700 1 -- 172.21.15.89:0/3895424793 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8007300 con 0x7fa9f80af320 2023-10-10T20:43:58.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.706+0000 7fa9dffff700 1 -- 172.21.15.89:0/3895424793 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e8007460 con 0x7fa9f80af320 2023-10-10T20:43:58.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.706+0000 7fa9fe9cc700 1 -- 172.21.15.89:0/3895424793 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f8156170 con 0x7fa9f80af320 2023-10-10T20:43:58.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.706+0000 7fa9dffff700 1 -- 172.21.15.89:0/3895424793 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8004830 con 0x7fa9f80af320 2023-10-10T20:43:58.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.706+0000 7fa9fe9cc700 1 -- 172.21.15.89:0/3895424793 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f8156730 con 0x7fa9f80af320 2023-10-10T20:43:58.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.707+0000 7fa9fe9cc700 1 -- 172.21.15.89:0/3895424793 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f80718b0 con 0x7fa9f80af320 2023-10-10T20:43:58.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.708+0000 7fa9dffff700 1 -- 172.21.15.89:0/3895424793 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa9e8004be0 con 0x7fa9f80af320 2023-10-10T20:43:58.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.708+0000 7fa9dffff700 1 --2- 172.21.15.89:0/3895424793 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa9e4067320 0x7fa9e40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:58.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.708+0000 7fa9dffff700 1 -- 172.21.15.89:0/3895424793 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fa9e8084050 con 0x7fa9f80af320 2023-10-10T20:43:58.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.708+0000 7fa9f7fff700 1 --2- 172.21.15.89:0/3895424793 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa9e4067320 0x7fa9e40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:58.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.708+0000 7fa9f7fff700 1 --2- 172.21.15.89:0/3895424793 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa9e4067320 0x7fa9e40697d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009990 tx=0x7fa9ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:58.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.712+0000 7fa9dffff700 1 -- 172.21.15.89:0/3895424793 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9e8050ec0 con 0x7fa9f80af320 2023-10-10T20:43:58.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:58.895+0000 7fa9fe9cc700 1 -- 172.21.15.89:0/3895424793 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa9f80ce8b0 con 0x7fa9f80af320 2023-10-10T20:43:59.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.563+0000 7fa9dffff700 1 -- 172.21.15.89:0/3895424793 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v196) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa9e8018020 con 0x7fa9f80af320 2023-10-10T20:43:59.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.570+0000 7fa9fe9cc700 1 -- 172.21.15.89:0/3895424793 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa9e4067320 msgr2=0x7fa9e40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:59.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.570+0000 7fa9fe9cc700 1 --2- 172.21.15.89:0/3895424793 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa9e4067320 0x7fa9e40697d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009990 tx=0x7fa9ec008040 comp rx=0 tx=0).stop 2023-10-10T20:43:59.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.570+0000 7fa9fe9cc700 1 -- 172.21.15.89:0/3895424793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9f80af320 msgr2=0x7fa9f80b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:59.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.570+0000 7fa9fe9cc700 1 --2- 172.21.15.89:0/3895424793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9f80af320 0x7fa9f80b56a0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800c7e0 tx=0x7fa9e8007e10 comp rx=0 tx=0).stop 2023-10-10T20:43:59.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.570+0000 7fa9fe9cc700 1 -- 172.21.15.89:0/3895424793 shutdown_connections 2023-10-10T20:43:59.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.570+0000 7fa9fe9cc700 1 --2- 172.21.15.89:0/3895424793 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fa9f8159950 0x7fa9f80b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:59.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.570+0000 7fa9fe9cc700 1 --2- 172.21.15.89:0/3895424793 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fa9e4067320 0x7fa9e40697d0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:59.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.570+0000 7fa9fe9cc700 1 --2- 172.21.15.89:0/3895424793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9f8069c70 0x7fa9f8155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:59.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.570+0000 7fa9fe9cc700 1 --2- 172.21.15.89:0/3895424793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9f80af320 0x7fa9f80b56a0 secure :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800c7e0 tx=0x7fa9e8007e10 comp rx=0 tx=0).stop 2023-10-10T20:43:59.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.570+0000 7fa9fe9cc700 1 -- 172.21.15.89:0/3895424793 >> 172.21.15.89:0/3895424793 conn(0x7fa9f8068e00 msgr2=0x7fa9f80b0550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:59.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.572+0000 7fa9fe9cc700 1 -- 172.21.15.89:0/3895424793 shutdown_connections 2023-10-10T20:43:59.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.572+0000 7fa9fe9cc700 1 -- 172.21.15.89:0/3895424793 wait complete. 2023-10-10T20:43:59.584 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2023-10-10T20:43:59.595 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:43:59.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.711+0000 7f9958520700 1 Processor -- start 2023-10-10T20:43:59.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.711+0000 7f9958520700 1 -- start start 2023-10-10T20:43:59.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9958520700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9950159c60 0x7f995015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:59.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9958520700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f995015db10 0x7f9950069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:59.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9958520700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995015a160 con 0x7f9950159890 2023-10-10T20:43:59.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9958520700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99500d13a0 con 0x7f995015db10 2023-10-10T20:43:59.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9958520700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995015e140 con 0x7f9950159c60 2023-10-10T20:43:59.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9956abd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f995015db10 0x7f9950069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:59.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f99562bc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9950159890 0x7f995015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37198/0 (socket says 172.21.15.89:37198) 2023-10-10T20:43:59.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f99562bc700 1 -- 172.21.15.89:0/20882306 learned_addr learned my addr 172.21.15.89:0/20882306 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:43:59.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9955abb700 1 --2- 172.21.15.89:0/20882306 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9950159c60 0x7f995015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:59.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9956abd700 1 -- 172.21.15.89:0/20882306 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9950159c60 msgr2=0x7f995015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:43:59.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9956abd700 1 --2- 172.21.15.89:0/20882306 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9950159c60 0x7f995015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:59.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9956abd700 1 -- 172.21.15.89:0/20882306 >> v1:172.21.15.89:6789/0 conn(0x7f9950159890 legacy=0x7f995015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:59.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f99552ba700 1 -- 172.21.15.89:0/20882306 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895903852 0 0) 0x7f995015a160 con 0x7f9950159890 2023-10-10T20:43:59.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9956abd700 1 -- 172.21.15.89:0/20882306 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995015a3d0 con 0x7f995015db10 2023-10-10T20:43:59.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9956abd700 1 --2- 172.21.15.89:0/20882306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f995015db10 0x7f9950069560 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f99400098e0 tx=0x7f99400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2a9aed0f99da4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:59.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f99552ba700 1 -- 172.21.15.89:0/20882306 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9940009e80 con 0x7f995015db10 2023-10-10T20:43:59.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f99552ba700 1 -- 172.21.15.89:0/20882306 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994000b860 con 0x7f995015db10 2023-10-10T20:43:59.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f99552ba700 1 -- 172.21.15.89:0/20882306 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994000bbe0 con 0x7f995015db10 2023-10-10T20:43:59.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9958520700 1 -- 172.21.15.89:0/20882306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f995015db10 msgr2=0x7f9950069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:59.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9958520700 1 --2- 172.21.15.89:0/20882306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f995015db10 0x7f9950069560 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f99400098e0 tx=0x7f99400091a0 comp rx=0 tx=0).stop 2023-10-10T20:43:59.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9958520700 1 -- 172.21.15.89:0/20882306 shutdown_connections 2023-10-10T20:43:59.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9958520700 1 --2- 172.21.15.89:0/20882306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f995015db10 0x7f9950069560 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:59.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9958520700 1 --2- 172.21.15.89:0/20882306 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9950159c60 0x7f995015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:59.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9958520700 1 -- 172.21.15.89:0/20882306 >> 172.21.15.89:0/20882306 conn(0x7f9950068e20 msgr2=0x7f99500d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:43:59.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9958520700 1 -- 172.21.15.89:0/20882306 shutdown_connections 2023-10-10T20:43:59.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.712+0000 7f9958520700 1 -- 172.21.15.89:0/20882306 wait complete. 2023-10-10T20:43:59.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.713+0000 7f9958520700 1 Processor -- start 2023-10-10T20:43:59.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.713+0000 7f9958520700 1 -- start start 2023-10-10T20:43:59.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f9958520700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9950159890 0x7f99500b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:59.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f9958520700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9950159c60 0x7f995014f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:59.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f9958520700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f995015db10 0x7f99500b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:59.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f9958520700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99500b2230 con 0x7f9950159c60 2023-10-10T20:43:59.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f9958520700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9950155a40 con 0x7f995015db10 2023-10-10T20:43:59.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f9958520700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99500b0a90 con 0x7f9950159890 2023-10-10T20:43:59.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f9955abb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9950159c60 0x7f995014f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:59.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f9955abb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9950159c60 0x7f995014f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51946/0 (socket says 172.21.15.89:51946) 2023-10-10T20:43:59.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f9955abb700 1 -- 172.21.15.89:0/90397541 learned_addr learned my addr 172.21.15.89:0/90397541 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:43:59.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f99562bc700 1 --2- 172.21.15.89:0/90397541 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9950159890 0x7f99500b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:59.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f9956abd700 1 --2- 172.21.15.89:0/90397541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f995015db10 0x7f99500b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:59.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f9955abb700 1 -- 172.21.15.89:0/90397541 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9950159890 msgr2=0x7f99500b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:59.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f9955abb700 1 --2- 172.21.15.89:0/90397541 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9950159890 0x7f99500b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:59.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f9955abb700 1 -- 172.21.15.89:0/90397541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f995015db10 msgr2=0x7f99500b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:43:59.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f9955abb700 1 --2- 172.21.15.89:0/90397541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f995015db10 0x7f99500b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:43:59.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f9955abb700 1 -- 172.21.15.89:0/90397541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9940009420 con 0x7f9950159c60 2023-10-10T20:43:59.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f9956abd700 1 --2- 172.21.15.89:0/90397541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f995015db10 0x7f99500b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:43:59.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f9955abb700 1 --2- 172.21.15.89:0/90397541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9950159c60 0x7f995014f8d0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f994400c8b0 tx=0x7f9944008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:59.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f994e7fc700 1 -- 172.21.15.89:0/90397541 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944011600 con 0x7f9950159c60 2023-10-10T20:43:59.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f994e7fc700 1 -- 172.21.15.89:0/90397541 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9944011760 con 0x7f9950159c60 2023-10-10T20:43:59.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f9958520700 1 -- 172.21.15.89:0/90397541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99500b1050 con 0x7f9950159c60 2023-10-10T20:43:59.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.714+0000 7f994e7fc700 1 -- 172.21.15.89:0/90397541 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944011b00 con 0x7f9950159c60 2023-10-10T20:43:59.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.715+0000 7f9958520700 1 -- 172.21.15.89:0/90397541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99500b2700 con 0x7f9950159c60 2023-10-10T20:43:59.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.715+0000 7f9958520700 1 -- 172.21.15.89:0/90397541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99500b11e0 con 0x7f9950159c60 2023-10-10T20:43:59.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.716+0000 7f994e7fc700 1 -- 172.21.15.89:0/90397541 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9944010350 con 0x7f9950159c60 2023-10-10T20:43:59.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.716+0000 7f994e7fc700 1 --2- 172.21.15.89:0/90397541 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f993c067320 0x7f993c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:43:59.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.717+0000 7f994e7fc700 1 -- 172.21.15.89:0/90397541 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f9944084a40 con 0x7f9950159c60 2023-10-10T20:43:59.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.717+0000 7f99562bc700 1 --2- 172.21.15.89:0/90397541 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f993c067320 0x7f993c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:43:59.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.717+0000 7f99562bc700 1 --2- 172.21.15.89:0/90397541 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f993c067320 0x7f993c0697d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9948001560 tx=0x7f9948009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:43:59.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.722+0000 7f994e7fc700 1 -- 172.21.15.89:0/90397541 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9944051620 con 0x7f9950159c60 2023-10-10T20:43:59.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:43:59.948+0000 7f9958520700 1 -- 172.21.15.89:0/90397541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f995004c3c0 con 0x7f9950159c60 2023-10-10T20:44:00.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.600+0000 7f994e7fc700 1 -- 172.21.15.89:0/90397541 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 33 and data pool 34 v128) v1 ==== 174+0+0 (secure 0 0 0) 0x7f99440150c0 con 0x7f9950159c60 2023-10-10T20:44:00.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.604+0000 7f9958520700 1 -- 172.21.15.89:0/90397541 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f993c067320 msgr2=0x7f993c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:00.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.604+0000 7f9958520700 1 --2- 172.21.15.89:0/90397541 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f993c067320 0x7f993c0697d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9948001560 tx=0x7f9948009040 comp rx=0 tx=0).stop 2023-10-10T20:44:00.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.604+0000 7f9958520700 1 -- 172.21.15.89:0/90397541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9950159c60 msgr2=0x7f995014f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:00.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.604+0000 7f9958520700 1 --2- 172.21.15.89:0/90397541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9950159c60 0x7f995014f8d0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f994400c8b0 tx=0x7f9944008040 comp rx=0 tx=0).stop 2023-10-10T20:44:00.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.604+0000 7f9958520700 1 -- 172.21.15.89:0/90397541 shutdown_connections 2023-10-10T20:44:00.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.604+0000 7f9958520700 1 --2- 172.21.15.89:0/90397541 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f993c067320 0x7f993c0697d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:00.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.604+0000 7f9958520700 1 --2- 172.21.15.89:0/90397541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f995015db10 0x7f99500b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:00.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.604+0000 7f9958520700 1 --2- 172.21.15.89:0/90397541 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9950159890 0x7f99500b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:00.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.604+0000 7f9958520700 1 --2- 172.21.15.89:0/90397541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9950159c60 0x7f995014f8d0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:00.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.604+0000 7f9958520700 1 -- 172.21.15.89:0/90397541 >> 172.21.15.89:0/90397541 conn(0x7f9950068e20 msgr2=0x7f99500cade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:00.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.605+0000 7f9958520700 1 -- 172.21.15.89:0/90397541 shutdown_connections 2023-10-10T20:44:00.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.606+0000 7f9958520700 1 -- 172.21.15.89:0/90397541 wait complete. 2023-10-10T20:44:00.609 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 33 and data pool 34 2023-10-10T20:44:00.623 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:44:00.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.736+0000 7f26e9557700 1 Processor -- start 2023-10-10T20:44:00.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.736+0000 7f26e9557700 1 -- start start 2023-10-10T20:44:00.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.737+0000 7f26e9557700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26e415d230 0x7f26e415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:00.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.737+0000 7f26e9557700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e40b2980 con 0x7f26e4159420 2023-10-10T20:44:00.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.737+0000 7f26e9557700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e415dbb0 con 0x7f26e41597f0 2023-10-10T20:44:00.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.737+0000 7f26e9557700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e40b11d0 con 0x7f26e415d230 2023-10-10T20:44:00.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.737+0000 7f26e3fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f26e4159420 0x7f26e40b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37200/0 (socket says 172.21.15.89:37200) 2023-10-10T20:44:00.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.737+0000 7f26e3fff700 1 -- 172.21.15.89:0/4168618736 learned_addr learned my addr 172.21.15.89:0/4168618736 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:44:00.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.737+0000 7f26e8d56700 1 --2- 172.21.15.89:0/4168618736 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26e415d230 0x7f26e415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:00.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.737+0000 7f26e2ffd700 1 -- 172.21.15.89:0/4168618736 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895574668 0 0) 0x7f26e40b2980 con 0x7f26e4159420 2023-10-10T20:44:00.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.737+0000 7f26e2ffd700 1 -- 172.21.15.89:0/4168618736 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26cc003570 con 0x7f26e4159420 2023-10-10T20:44:00.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.737+0000 7f26e2ffd700 1 -- 172.21.15.89:0/4168618736 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1338576591 0 0) 0x7f26cc003570 con 0x7f26e4159420 2023-10-10T20:44:00.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.737+0000 7f26e2ffd700 1 -- 172.21.15.89:0/4168618736 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26e415d230 msgr2=0x7f26e415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:00.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.737+0000 7f26e2ffd700 1 --2- 172.21.15.89:0/4168618736 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26e415d230 0x7f26e415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:00.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.737+0000 7f26e2ffd700 1 -- 172.21.15.89:0/4168618736 >> v1:172.21.15.111:6789/0 conn(0x7f26e41597f0 legacy=0x7f26e40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:00.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.737+0000 7f26e2ffd700 1 -- 172.21.15.89:0/4168618736 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e40b14b0 con 0x7f26e4159420 2023-10-10T20:44:00.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.737+0000 7f26e2ffd700 1 -- 172.21.15.89:0/4168618736 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f26d4001470 con 0x7f26e4159420 2023-10-10T20:44:00.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.738+0000 7f26e2ffd700 1 -- 172.21.15.89:0/4168618736 >> v1:172.21.15.89:6789/0 conn(0x7f26e4159420 legacy=0x7f26e40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:00.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.738+0000 7f26e2ffd700 1 --2- 172.21.15.89:0/4168618736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26cc003a90 0x7f26cc003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:00.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.738+0000 7f26e2ffd700 1 --2- 172.21.15.89:0/4168618736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26cc004810 0x7f26cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:00.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.738+0000 7f26e2ffd700 1 --2- 172.21.15.89:0/4168618736 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26e415d230 0x7f26cc0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:00.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.738+0000 7f26e2ffd700 1 -- 172.21.15.89:0/4168618736 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e40b11d0 con 0x7f26cc004810 2023-10-10T20:44:00.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.738+0000 7f26e2ffd700 1 -- 172.21.15.89:0/4168618736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc00bb60 con 0x7f26cc003a90 2023-10-10T20:44:00.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.738+0000 7f26e2ffd700 1 -- 172.21.15.89:0/4168618736 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc00bd20 con 0x7f26e415d230 2023-10-10T20:44:00.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.738+0000 7f26e37fe700 1 --2- 172.21.15.89:0/4168618736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26cc004810 0x7f26cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:00.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.738+0000 7f26e3fff700 1 --2- 172.21.15.89:0/4168618736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26cc003a90 0x7f26cc003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:00.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.738+0000 7f26e8d56700 1 --2- 172.21.15.89:0/4168618736 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26e415d230 0x7f26cc0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:00.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.738+0000 7f26e37fe700 1 -- 172.21.15.89:0/4168618736 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26e415d230 msgr2=0x7f26cc0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:00.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.738+0000 7f26e37fe700 1 --2- 172.21.15.89:0/4168618736 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26e415d230 0x7f26cc0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:00.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.738+0000 7f26e37fe700 1 -- 172.21.15.89:0/4168618736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26cc003a90 msgr2=0x7f26cc003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:00.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.738+0000 7f26e37fe700 1 --2- 172.21.15.89:0/4168618736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26cc003a90 0x7f26cc003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:00.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.738+0000 7f26e37fe700 1 -- 172.21.15.89:0/4168618736 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc003570 con 0x7f26cc004810 2023-10-10T20:44:00.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.738+0000 7f26e37fe700 1 --2- 172.21.15.89:0/4168618736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26cc004810 0x7f26cc004c80 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f26d80018b0 tx=0x7f26d8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b79bb23dbf53ae5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:00.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.738+0000 7f26e2ffd700 1 -- 172.21.15.89:0/4168618736 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8012420 con 0x7f26cc004810 2023-10-10T20:44:00.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.738+0000 7f26e2ffd700 1 -- 172.21.15.89:0/4168618736 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d8012580 con 0x7f26cc004810 2023-10-10T20:44:00.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.738+0000 7f26e2ffd700 1 -- 172.21.15.89:0/4168618736 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8012920 con 0x7f26cc004810 2023-10-10T20:44:00.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.739+0000 7f26e9557700 1 -- 172.21.15.89:0/4168618736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26cc004810 msgr2=0x7f26cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:00.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.739+0000 7f26e9557700 1 --2- 172.21.15.89:0/4168618736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26cc004810 0x7f26cc004c80 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f26d80018b0 tx=0x7f26d8007d30 comp rx=0 tx=0).stop 2023-10-10T20:44:00.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.739+0000 7f26e8d56700 1 -- 172.21.15.89:0/4168618736 reap_dead start 2023-10-10T20:44:00.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.739+0000 7f26e9557700 1 -- 172.21.15.89:0/4168618736 shutdown_connections 2023-10-10T20:44:00.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.739+0000 7f26e9557700 1 -- 172.21.15.89:0/4168618736 >> 172.21.15.89:0/4168618736 conn(0x7f26e40d0660 msgr2=0x7f26e40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:00.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.739+0000 7f26e9557700 1 -- 172.21.15.89:0/4168618736 shutdown_connections 2023-10-10T20:44:00.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.739+0000 7f26e9557700 1 -- 172.21.15.89:0/4168618736 wait complete. 2023-10-10T20:44:00.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.740+0000 7f26e9557700 1 Processor -- start 2023-10-10T20:44:00.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.740+0000 7f26e9557700 1 -- start start 2023-10-10T20:44:00.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e9557700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26cc004810 0x7f26e40c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:00.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e9557700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26e40ca320 0x7f26e4155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:00.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e9557700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26e415cb10 0x7f26e41563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:00.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e9557700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e40cb3c0 con 0x7f26e40ca320 2023-10-10T20:44:00.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e9557700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e40ca7b0 con 0x7f26cc004810 2023-10-10T20:44:00.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e9557700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e40caa90 con 0x7f26e415cb10 2023-10-10T20:44:00.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26e40ca320 0x7f26e4155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:00.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26e40ca320 0x7f26e4155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51958/0 (socket says 172.21.15.89:51958) 2023-10-10T20:44:00.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e37fe700 1 -- 172.21.15.89:0/2024839045 learned_addr learned my addr 172.21.15.89:0/2024839045 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:00.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e8d56700 1 --2- 172.21.15.89:0/2024839045 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26e415cb10 0x7f26e41563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:00.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e3fff700 1 --2- 172.21.15.89:0/2024839045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26cc004810 0x7f26e40c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:00.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e37fe700 1 -- 172.21.15.89:0/2024839045 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26e415cb10 msgr2=0x7f26e41563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:00.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e37fe700 1 --2- 172.21.15.89:0/2024839045 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26e415cb10 0x7f26e41563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:00.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e37fe700 1 -- 172.21.15.89:0/2024839045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26cc004810 msgr2=0x7f26e40c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:00.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e37fe700 1 --2- 172.21.15.89:0/2024839045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26cc004810 0x7f26e40c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:00.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e37fe700 1 -- 172.21.15.89:0/2024839045 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e40cad70 con 0x7f26e40ca320 2023-10-10T20:44:00.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e8d56700 1 --2- 172.21.15.89:0/2024839045 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26e415cb10 0x7f26e41563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:44:00.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e3fff700 1 --2- 172.21.15.89:0/2024839045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26cc004810 0x7f26e40c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:44:00.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e37fe700 1 --2- 172.21.15.89:0/2024839045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26e40ca320 0x7f26e4155e60 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f26d800c3f0 tx=0x7f26d8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:00.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26cbfff700 1 -- 172.21.15.89:0/2024839045 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8001230 con 0x7f26e40ca320 2023-10-10T20:44:00.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26cbfff700 1 -- 172.21.15.89:0/2024839045 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d8012c90 con 0x7f26e40ca320 2023-10-10T20:44:00.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e9557700 1 -- 172.21.15.89:0/2024839045 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e40cb050 con 0x7f26e40ca320 2023-10-10T20:44:00.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26cbfff700 1 -- 172.21.15.89:0/2024839045 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8012e00 con 0x7f26e40ca320 2023-10-10T20:44:00.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.741+0000 7f26e9557700 1 -- 172.21.15.89:0/2024839045 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e41541c0 con 0x7f26e40ca320 2023-10-10T20:44:00.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.742+0000 7f26e9557700 1 -- 172.21.15.89:0/2024839045 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e404c3c0 con 0x7f26e40ca320 2023-10-10T20:44:00.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.743+0000 7f26cbfff700 1 -- 172.21.15.89:0/2024839045 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f26d8006a30 con 0x7f26e40ca320 2023-10-10T20:44:00.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.743+0000 7f26cbfff700 1 --2- 172.21.15.89:0/2024839045 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f26d0067320 0x7f26d00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:00.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.743+0000 7f26cbfff700 1 -- 172.21.15.89:0/2024839045 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f26d80862a0 con 0x7f26e40ca320 2023-10-10T20:44:00.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.743+0000 7f26e3fff700 1 --2- 172.21.15.89:0/2024839045 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f26d0067320 0x7f26d00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:00.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.743+0000 7f26e3fff700 1 --2- 172.21.15.89:0/2024839045 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f26d0067320 0x7f26d00697d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f26d4001ce0 tx=0x7f26d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:00.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.747+0000 7f26cbfff700 1 -- 172.21.15.89:0/2024839045 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26d8052f30 con 0x7f26e40ca320 2023-10-10T20:44:00.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.932+0000 7f26e9557700 1 -- 172.21.15.89:0/2024839045 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26e40cb500 con 0x7f26e40ca320 2023-10-10T20:44:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.933+0000 7f26cbfff700 1 -- 172.21.15.89:0/2024839045 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+15377 (secure 0 0 0) 0x7f26d801dea0 con 0x7f26e40ca320 2023-10-10T20:44:00.935 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:44:00.936 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":197,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:43:59.950474+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:43:58.313137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-10T20:43:58.914819+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:44:00.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.936+0000 7f26e9557700 1 -- 172.21.15.89:0/2024839045 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f26d0067320 msgr2=0x7f26d00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:00.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.936+0000 7f26e9557700 1 --2- 172.21.15.89:0/2024839045 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f26d0067320 0x7f26d00697d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f26d4001ce0 tx=0x7f26d4009040 comp rx=0 tx=0).stop 2023-10-10T20:44:00.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.936+0000 7f26e9557700 1 -- 172.21.15.89:0/2024839045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26e40ca320 msgr2=0x7f26e4155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.936+0000 7f26e9557700 1 --2- 172.21.15.89:0/2024839045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26e40ca320 0x7f26e4155e60 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f26d800c3f0 tx=0x7f26d8015000 comp rx=0 tx=0).stop 2023-10-10T20:44:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.936+0000 7f26e9557700 1 -- 172.21.15.89:0/2024839045 shutdown_connections 2023-10-10T20:44:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.936+0000 7f26e9557700 1 --2- 172.21.15.89:0/2024839045 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f26e415cb10 0x7f26e41563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.936+0000 7f26e9557700 1 --2- 172.21.15.89:0/2024839045 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f26d0067320 0x7f26d00697d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.936+0000 7f26e9557700 1 --2- 172.21.15.89:0/2024839045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26cc004810 0x7f26e40c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:00.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.936+0000 7f26e9557700 1 --2- 172.21.15.89:0/2024839045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26e40ca320 0x7f26e4155e60 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:00.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.936+0000 7f26e9557700 1 -- 172.21.15.89:0/2024839045 >> 172.21.15.89:0/2024839045 conn(0x7f26e40d0660 msgr2=0x7f26e4155340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:00.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.936+0000 7f26e9557700 1 -- 172.21.15.89:0/2024839045 shutdown_connections 2023-10-10T20:44:00.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:00.936+0000 7f26e9557700 1 -- 172.21.15.89:0/2024839045 wait complete. 2023-10-10T20:44:00.950 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:44:01.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.062+0000 7f37d3fff700 1 Processor -- start 2023-10-10T20:44:01.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.062+0000 7f37d3fff700 1 -- start start 2023-10-10T20:44:01.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.063+0000 7f37d3fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f37d40c6c60 0x7f37d40c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:01.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.063+0000 7f37d3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37d406e8c0 0x7f37d40c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:01.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.063+0000 7f37d3fff700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d40b1410 con 0x7f37d41595a0 2023-10-10T20:44:01.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.063+0000 7f37d3fff700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d40b2b90 con 0x7f37d406e8c0 2023-10-10T20:44:01.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.063+0000 7f37d3fff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d4063230 con 0x7f37d40c6c60 2023-10-10T20:44:01.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.063+0000 7f37d37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37d406e8c0 0x7f37d40c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:01.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.063+0000 7f37d2ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f37d41595a0 0x7f37d40b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37212/0 (socket says 172.21.15.89:37212) 2023-10-10T20:44:01.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.063+0000 7f37d2ffd700 1 -- 172.21.15.89:0/3062120242 learned_addr learned my addr 172.21.15.89:0/3062120242 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:44:01.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.063+0000 7f37d37fe700 1 -- 172.21.15.89:0/3062120242 >> v2:172.21.15.111:3300/0 conn(0x7f37d40c6c60 msgr2=0x7f37d40c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:01.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.063+0000 7f37ca7fc700 1 --2- 172.21.15.89:0/3062120242 >> v2:172.21.15.111:3300/0 conn(0x7f37d40c6c60 0x7f37d40c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:01.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.064+0000 7f37d37fe700 1 --2- 172.21.15.89:0/3062120242 >> v2:172.21.15.111:3300/0 conn(0x7f37d40c6c60 0x7f37d40c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:01.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.064+0000 7f37d37fe700 1 -- 172.21.15.89:0/3062120242 >> v1:172.21.15.89:6789/0 conn(0x7f37d41595a0 legacy=0x7f37d40b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:01.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.064+0000 7f37d27fc700 1 -- 172.21.15.89:0/3062120242 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 65798125 0 0) 0x7f37d40b1410 con 0x7f37d41595a0 2023-10-10T20:44:01.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.064+0000 7f37d37fe700 1 -- 172.21.15.89:0/3062120242 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d40634f0 con 0x7f37d406e8c0 2023-10-10T20:44:01.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.064+0000 7f37ca7fc700 1 --2- 172.21.15.89:0/3062120242 >> v2:172.21.15.111:3300/0 conn(0x7f37d40c6c60 0x7f37d40c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:44:01.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.064+0000 7f37d37fe700 1 --2- 172.21.15.89:0/3062120242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37d406e8c0 0x7f37d40c8660 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f37d40b07d0 tx=0x7f37cc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e081c4b54d53df8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:01.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.064+0000 7f37d27fc700 1 -- 172.21.15.89:0/3062120242 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc00e070 con 0x7f37d406e8c0 2023-10-10T20:44:01.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.064+0000 7f37d27fc700 1 -- 172.21.15.89:0/3062120242 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37cc00bc30 con 0x7f37d406e8c0 2023-10-10T20:44:01.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.064+0000 7f37d27fc700 1 -- 172.21.15.89:0/3062120242 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc012430 con 0x7f37d406e8c0 2023-10-10T20:44:01.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.064+0000 7f37d3fff700 1 -- 172.21.15.89:0/3062120242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37d406e8c0 msgr2=0x7f37d40c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:01.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.064+0000 7f37d3fff700 1 --2- 172.21.15.89:0/3062120242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37d406e8c0 0x7f37d40c8660 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f37d40b07d0 tx=0x7f37cc009780 comp rx=0 tx=0).stop 2023-10-10T20:44:01.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.064+0000 7f37d3fff700 1 -- 172.21.15.89:0/3062120242 shutdown_connections 2023-10-10T20:44:01.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.064+0000 7f37d3fff700 1 --2- 172.21.15.89:0/3062120242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37d406e8c0 0x7f37d40c8660 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:01.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.064+0000 7f37d3fff700 1 --2- 172.21.15.89:0/3062120242 >> v2:172.21.15.111:3300/0 conn(0x7f37d40c6c60 0x7f37d40c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:01.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.064+0000 7f37d3fff700 1 -- 172.21.15.89:0/3062120242 >> 172.21.15.89:0/3062120242 conn(0x7f37d406ddb0 msgr2=0x7f37d406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:01.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.064+0000 7f37d3fff700 1 -- 172.21.15.89:0/3062120242 shutdown_connections 2023-10-10T20:44:01.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.064+0000 7f37d3fff700 1 -- 172.21.15.89:0/3062120242 wait complete. 2023-10-10T20:44:01.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.066+0000 7f37d3fff700 1 Processor -- start 2023-10-10T20:44:01.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.066+0000 7f37d3fff700 1 -- start start 2023-10-10T20:44:01.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.066+0000 7f37d3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37d406e8c0 0x7f37d416a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:01.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.066+0000 7f37d3fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37d40c6c60 0x7f37d40a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:01.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.066+0000 7f37d3fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37d41595a0 0x7f37d40a6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:01.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.066+0000 7f37d3fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d416a570 con 0x7f37d406e8c0 2023-10-10T20:44:01.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.066+0000 7f37d3fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d40a6380 con 0x7f37d41595a0 2023-10-10T20:44:01.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.066+0000 7f37d3fff700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d4169960 con 0x7f37d40c6c60 2023-10-10T20:44:01.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.066+0000 7f37d2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37d406e8c0 0x7f37d416a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:01.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.066+0000 7f37d2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37d406e8c0 0x7f37d416a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51968/0 (socket says 172.21.15.89:51968) 2023-10-10T20:44:01.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.066+0000 7f37d2ffd700 1 -- 172.21.15.89:0/3492991030 learned_addr learned my addr 172.21.15.89:0/3492991030 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:01.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.067+0000 7f37ca7fc700 1 --2- 172.21.15.89:0/3492991030 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37d40c6c60 0x7f37d40a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:01.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.067+0000 7f37d37fe700 1 --2- 172.21.15.89:0/3492991030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37d41595a0 0x7f37d40a6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:01.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.067+0000 7f37d2ffd700 1 -- 172.21.15.89:0/3492991030 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37d40c6c60 msgr2=0x7f37d40a5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:01.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.067+0000 7f37d2ffd700 1 --2- 172.21.15.89:0/3492991030 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37d40c6c60 0x7f37d40a5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:01.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.067+0000 7f37d2ffd700 1 -- 172.21.15.89:0/3492991030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37d41595a0 msgr2=0x7f37d40a6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:01.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.067+0000 7f37d2ffd700 1 --2- 172.21.15.89:0/3492991030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37d41595a0 0x7f37d40a6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:01.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.067+0000 7f37d2ffd700 1 -- 172.21.15.89:0/3492991030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc009a20 con 0x7f37d406e8c0 2023-10-10T20:44:01.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.067+0000 7f37ca7fc700 1 --2- 172.21.15.89:0/3492991030 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37d40c6c60 0x7f37d40a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:44:01.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.067+0000 7f37d2ffd700 1 --2- 172.21.15.89:0/3492991030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37d406e8c0 0x7f37d416a720 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f37c400c5b0 tx=0x7f37c4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:01.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.067+0000 7f37cb7fe700 1 -- 172.21.15.89:0/3492991030 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4005b30 con 0x7f37d406e8c0 2023-10-10T20:44:01.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.067+0000 7f37cb7fe700 1 -- 172.21.15.89:0/3492991030 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c4005c90 con 0x7f37d406e8c0 2023-10-10T20:44:01.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.067+0000 7f37cb7fe700 1 -- 172.21.15.89:0/3492991030 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4007300 con 0x7f37d406e8c0 2023-10-10T20:44:01.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.067+0000 7f37d3fff700 1 -- 172.21.15.89:0/3492991030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d4169c40 con 0x7f37d406e8c0 2023-10-10T20:44:01.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.067+0000 7f37d3fff700 1 -- 172.21.15.89:0/3492991030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d416a200 con 0x7f37d406e8c0 2023-10-10T20:44:01.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.068+0000 7f37d3fff700 1 -- 172.21.15.89:0/3492991030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d404c3c0 con 0x7f37d406e8c0 2023-10-10T20:44:01.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.068+0000 7f37cb7fe700 1 -- 172.21.15.89:0/3492991030 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f37c401b3b0 con 0x7f37d406e8c0 2023-10-10T20:44:01.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.069+0000 7f37cb7fe700 1 --2- 172.21.15.89:0/3492991030 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f37b4067320 0x7f37b40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:01.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.069+0000 7f37cb7fe700 1 -- 172.21.15.89:0/3492991030 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f37c40841a0 con 0x7f37d406e8c0 2023-10-10T20:44:01.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.069+0000 7f37ca7fc700 1 --2- 172.21.15.89:0/3492991030 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f37b4067320 0x7f37b40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:01.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.069+0000 7f37ca7fc700 1 --2- 172.21.15.89:0/3492991030 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f37b4067320 0x7f37b40697d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f37d4157430 tx=0x7f37bc006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:01.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.073+0000 7f37cb7fe700 1 -- 172.21.15.89:0/3492991030 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37c4050e30 con 0x7f37d406e8c0 2023-10-10T20:44:01.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.259+0000 7f37d3fff700 1 -- 172.21.15.89:0/3492991030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37d40637d0 con 0x7f37d406e8c0 2023-10-10T20:44:01.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.260+0000 7f37cb7fe700 1 -- 172.21.15.89:0/3492991030 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+15377 (secure 0 0 0) 0x7f37c4054450 con 0x7f37d406e8c0 2023-10-10T20:44:01.262 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:44:01.262 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":197,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:43:59.950474+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:43:58.313137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-10T20:43:58.914819+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:44:01.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.261+0000 7f37d3fff700 1 -- 172.21.15.89:0/3492991030 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f37b4067320 msgr2=0x7f37b40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:01.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.261+0000 7f37d3fff700 1 --2- 172.21.15.89:0/3492991030 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f37b4067320 0x7f37b40697d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f37d4157430 tx=0x7f37bc006f90 comp rx=0 tx=0).stop 2023-10-10T20:44:01.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.261+0000 7f37d3fff700 1 -- 172.21.15.89:0/3492991030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37d406e8c0 msgr2=0x7f37d416a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:01.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.261+0000 7f37d3fff700 1 --2- 172.21.15.89:0/3492991030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37d406e8c0 0x7f37d416a720 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f37c400c5b0 tx=0x7f37c4001070 comp rx=0 tx=0).stop 2023-10-10T20:44:01.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.261+0000 7f37d3fff700 1 -- 172.21.15.89:0/3492991030 shutdown_connections 2023-10-10T20:44:01.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.261+0000 7f37d3fff700 1 --2- 172.21.15.89:0/3492991030 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f37b4067320 0x7f37b40697d0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:01.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.261+0000 7f37d3fff700 1 --2- 172.21.15.89:0/3492991030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37d41595a0 0x7f37d40a6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:01.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.261+0000 7f37d3fff700 1 --2- 172.21.15.89:0/3492991030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37d406e8c0 0x7f37d416a720 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:01.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.261+0000 7f37d3fff700 1 --2- 172.21.15.89:0/3492991030 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37d40c6c60 0x7f37d40a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:01.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.261+0000 7f37d3fff700 1 -- 172.21.15.89:0/3492991030 >> 172.21.15.89:0/3492991030 conn(0x7f37d406ddb0 msgr2=0x7f37d4157480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:01.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.262+0000 7f37d3fff700 1 -- 172.21.15.89:0/3492991030 shutdown_connections 2023-10-10T20:44:01.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.262+0000 7f37d3fff700 1 -- 172.21.15.89:0/3492991030 wait complete. 2023-10-10T20:44:01.276 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:44:01.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.391+0000 7f706d7d2700 1 Processor -- start 2023-10-10T20:44:01.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.391+0000 7f706d7d2700 1 -- start start 2023-10-10T20:44:01.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.392+0000 7f706d7d2700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7068159c60 0x7f706815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:01.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.392+0000 7f706d7d2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706815a160 con 0x7f7068159890 2023-10-10T20:44:01.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.392+0000 7f706d7d2700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70680d13a0 con 0x7f706815db10 2023-10-10T20:44:01.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.392+0000 7f706d7d2700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706815e140 con 0x7f7068159c60 2023-10-10T20:44:01.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.392+0000 7f7066ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7068159890 0x7f706815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37224/0 (socket says 172.21.15.89:37224) 2023-10-10T20:44:01.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.392+0000 7f7066ffd700 1 -- 172.21.15.89:0/51583614 learned_addr learned my addr 172.21.15.89:0/51583614 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:44:01.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.392+0000 7f705e7fc700 1 --2- 172.21.15.89:0/51583614 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7068159c60 0x7f706815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:01.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.392+0000 7f70667fc700 1 -- 172.21.15.89:0/51583614 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808043246 0 0) 0x7f706815a160 con 0x7f7068159890 2023-10-10T20:44:01.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.392+0000 7f70667fc700 1 -- 172.21.15.89:0/51583614 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7054003570 con 0x7f7068159890 2023-10-10T20:44:01.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.392+0000 7f70667fc700 1 -- 172.21.15.89:0/51583614 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708619406 0 0) 0x7f70680d13a0 con 0x7f706815db10 2023-10-10T20:44:01.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.393+0000 7f70667fc700 1 -- 172.21.15.89:0/51583614 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f706815a160 con 0x7f706815db10 2023-10-10T20:44:01.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.393+0000 7f705e7fc700 1 -- 172.21.15.89:0/51583614 >> v1:172.21.15.111:6789/0 conn(0x7f706815db10 legacy=0x7f7068069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:01.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.393+0000 7f705e7fc700 1 -- 172.21.15.89:0/51583614 >> v1:172.21.15.89:6789/0 conn(0x7f7068159890 legacy=0x7f706815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:01.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.393+0000 7f705e7fc700 1 -- 172.21.15.89:0/51583614 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706815a3d0 con 0x7f7068159c60 2023-10-10T20:44:01.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.393+0000 7f705e7fc700 1 --2- 172.21.15.89:0/51583614 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7068159c60 0x7f706815d5d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7050009910 tx=0x7f70500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51ff55ef5cc9c070 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:01.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.393+0000 7f70667fc700 1 -- 172.21.15.89:0/51583614 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050011630 con 0x7f7068159c60 2023-10-10T20:44:01.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.393+0000 7f70667fc700 1 -- 172.21.15.89:0/51583614 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7050011790 con 0x7f7068159c60 2023-10-10T20:44:01.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.393+0000 7f70667fc700 1 -- 172.21.15.89:0/51583614 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050011b30 con 0x7f7068159c60 2023-10-10T20:44:01.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.393+0000 7f706d7d2700 1 -- 172.21.15.89:0/51583614 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7068159c60 msgr2=0x7f706815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:01.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.393+0000 7f706d7d2700 1 --2- 172.21.15.89:0/51583614 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7068159c60 0x7f706815d5d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7050009910 tx=0x7f70500091a0 comp rx=0 tx=0).stop 2023-10-10T20:44:01.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.393+0000 7f706d7d2700 1 -- 172.21.15.89:0/51583614 shutdown_connections 2023-10-10T20:44:01.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.393+0000 7f706d7d2700 1 --2- 172.21.15.89:0/51583614 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7068159c60 0x7f706815d5d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:01.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.393+0000 7f706d7d2700 1 -- 172.21.15.89:0/51583614 >> 172.21.15.89:0/51583614 conn(0x7f7068068e20 msgr2=0x7f70680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:01.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.394+0000 7f706d7d2700 1 -- 172.21.15.89:0/51583614 shutdown_connections 2023-10-10T20:44:01.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.394+0000 7f706d7d2700 1 -- 172.21.15.89:0/51583614 wait complete. 2023-10-10T20:44:01.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.395+0000 7f706d7d2700 1 Processor -- start 2023-10-10T20:44:01.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.395+0000 7f706d7d2700 1 -- start start 2023-10-10T20:44:01.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.395+0000 7f706d7d2700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7068159890 0x7f70680cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:01.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.395+0000 7f706d7d2700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7068159c60 0x7f70680b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:01.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.395+0000 7f706d7d2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f706815db10 0x7f7068151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:01.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.395+0000 7f706d7d2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706815cc80 con 0x7f706815db10 2023-10-10T20:44:01.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.395+0000 7f706d7d2700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70680cf4d0 con 0x7f7068159890 2023-10-10T20:44:01.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.395+0000 7f706d7d2700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70680cdf90 con 0x7f7068159c60 2023-10-10T20:44:01.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.396+0000 7f70677fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f706815db10 0x7f7068151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:01.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.396+0000 7f70677fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f706815db10 0x7f7068151990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51972/0 (socket says 172.21.15.89:51972) 2023-10-10T20:44:01.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.396+0000 7f70677fe700 1 -- 172.21.15.89:0/2293028014 learned_addr learned my addr 172.21.15.89:0/2293028014 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:01.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.396+0000 7f7066ffd700 1 --2- 172.21.15.89:0/2293028014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7068159890 0x7f70680cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:01.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.396+0000 7f705e7fc700 1 --2- 172.21.15.89:0/2293028014 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7068159c60 0x7f70680b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:01.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.396+0000 7f70677fe700 1 -- 172.21.15.89:0/2293028014 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7068159c60 msgr2=0x7f70680b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:01.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.396+0000 7f70677fe700 1 --2- 172.21.15.89:0/2293028014 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7068159c60 0x7f70680b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:01.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.396+0000 7f70677fe700 1 -- 172.21.15.89:0/2293028014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7068159890 msgr2=0x7f70680cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:01.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.396+0000 7f70677fe700 1 --2- 172.21.15.89:0/2293028014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7068159890 0x7f70680cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:01.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.396+0000 7f70677fe700 1 -- 172.21.15.89:0/2293028014 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7050009420 con 0x7f706815db10 2023-10-10T20:44:01.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.396+0000 7f70677fe700 1 --2- 172.21.15.89:0/2293028014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f706815db10 0x7f7068151990 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f706000c860 tx=0x7f7060002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:01.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.396+0000 7f705f7fe700 1 -- 172.21.15.89:0/2293028014 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060002030 con 0x7f706815db10 2023-10-10T20:44:01.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.396+0000 7f705f7fe700 1 -- 172.21.15.89:0/2293028014 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70600021b0 con 0x7f706815db10 2023-10-10T20:44:01.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.396+0000 7f706d7d2700 1 -- 172.21.15.89:0/2293028014 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70680ce8c0 con 0x7f706815db10 2023-10-10T20:44:01.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.396+0000 7f705f7fe700 1 -- 172.21.15.89:0/2293028014 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060012630 con 0x7f706815db10 2023-10-10T20:44:01.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.396+0000 7f706d7d2700 1 -- 172.21.15.89:0/2293028014 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70680cee80 con 0x7f706815db10 2023-10-10T20:44:01.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.397+0000 7f706d7d2700 1 -- 172.21.15.89:0/2293028014 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f706804c3c0 con 0x7f706815db10 2023-10-10T20:44:01.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.398+0000 7f705f7fe700 1 -- 172.21.15.89:0/2293028014 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7060012a20 con 0x7f706815db10 2023-10-10T20:44:01.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.398+0000 7f705f7fe700 1 --2- 172.21.15.89:0/2293028014 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7048067320 0x7f70480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:01.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.398+0000 7f705f7fe700 1 -- 172.21.15.89:0/2293028014 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7060084c50 con 0x7f706815db10 2023-10-10T20:44:01.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.398+0000 7f7066ffd700 1 --2- 172.21.15.89:0/2293028014 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7048067320 0x7f70480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:01.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.399+0000 7f7066ffd700 1 --2- 172.21.15.89:0/2293028014 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7048067320 0x7f70480697d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7058007fd0 tx=0x7f7058009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:01.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.402+0000 7f705f7fe700 1 -- 172.21.15.89:0/2293028014 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70600518e0 con 0x7f706815db10 2023-10-10T20:44:01.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:01.632+0000 7f706d7d2700 1 -- 172.21.15.89:0/2293028014 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f70680cbec0 con 0x7f706815db10 2023-10-10T20:44:02.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.612+0000 7f705f7fe700 1 -- 172.21.15.89:0/2293028014 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v131) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7060054f00 con 0x7f706815db10 2023-10-10T20:44:02.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.615+0000 7f706d7d2700 1 -- 172.21.15.89:0/2293028014 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7048067320 msgr2=0x7f70480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:02.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.615+0000 7f706d7d2700 1 --2- 172.21.15.89:0/2293028014 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7048067320 0x7f70480697d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7058007fd0 tx=0x7f7058009040 comp rx=0 tx=0).stop 2023-10-10T20:44:02.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.615+0000 7f706d7d2700 1 -- 172.21.15.89:0/2293028014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f706815db10 msgr2=0x7f7068151990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:02.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.615+0000 7f706d7d2700 1 --2- 172.21.15.89:0/2293028014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f706815db10 0x7f7068151990 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f706000c860 tx=0x7f7060002700 comp rx=0 tx=0).stop 2023-10-10T20:44:02.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.615+0000 7f706d7d2700 1 -- 172.21.15.89:0/2293028014 shutdown_connections 2023-10-10T20:44:02.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.615+0000 7f706d7d2700 1 --2- 172.21.15.89:0/2293028014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f706815db10 0x7f7068151990 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:02.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.615+0000 7f706d7d2700 1 --2- 172.21.15.89:0/2293028014 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7048067320 0x7f70480697d0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:02.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.615+0000 7f706d7d2700 1 --2- 172.21.15.89:0/2293028014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7068159890 0x7f70680cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:02.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.615+0000 7f706d7d2700 1 --2- 172.21.15.89:0/2293028014 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7068159c60 0x7f70680b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:02.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.615+0000 7f706d7d2700 1 -- 172.21.15.89:0/2293028014 >> 172.21.15.89:0/2293028014 conn(0x7f7068068e20 msgr2=0x7f70681571d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:02.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.615+0000 7f706d7d2700 1 -- 172.21.15.89:0/2293028014 shutdown_connections 2023-10-10T20:44:02.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.615+0000 7f706d7d2700 1 -- 172.21.15.89:0/2293028014 wait complete. 2023-10-10T20:44:02.631 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:44:02.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.752+0000 7ff12f590700 1 Processor -- start 2023-10-10T20:44:02.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.753+0000 7ff12f590700 1 -- start start 2023-10-10T20:44:02.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.753+0000 7ff12f590700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff12806bcd0 0x7ff128140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:02.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.753+0000 7ff12f590700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff12806c0a0 0x7ff128141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:02.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.753+0000 7ff12f590700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1280c8e00 0x7ff1280c9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:02.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.753+0000 7ff12f590700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff12815c2a0 con 0x7ff12806bcd0 2023-10-10T20:44:02.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.753+0000 7ff12f590700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff12815b690 con 0x7ff12806c0a0 2023-10-10T20:44:02.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.753+0000 7ff12f590700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12815b970 con 0x7ff1280c8e00 2023-10-10T20:44:02.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.754+0000 7ff12d32c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff12806bcd0 0x7ff128140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:02.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.754+0000 7ff12d32c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff12806bcd0 0x7ff128140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51988/0 (socket says 172.21.15.89:51988) 2023-10-10T20:44:02.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.754+0000 7ff12d32c700 1 -- 172.21.15.89:0/710280498 learned_addr learned my addr 172.21.15.89:0/710280498 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:02.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.754+0000 7ff12db2d700 1 --2- 172.21.15.89:0/710280498 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1280c8e00 0x7ff1280c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:02.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.754+0000 7ff12cb2b700 1 --2- 172.21.15.89:0/710280498 >> v2:172.21.15.111:3300/0 conn(0x7ff12806c0a0 0x7ff128141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:02.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.754+0000 7ff12d32c700 1 -- 172.21.15.89:0/710280498 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1280c8e00 msgr2=0x7ff1280c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:02.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.754+0000 7ff12d32c700 1 --2- 172.21.15.89:0/710280498 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1280c8e00 0x7ff1280c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:02.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.754+0000 7ff12d32c700 1 -- 172.21.15.89:0/710280498 >> v2:172.21.15.111:3300/0 conn(0x7ff12806c0a0 msgr2=0x7ff128141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:02.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.754+0000 7ff12d32c700 1 --2- 172.21.15.89:0/710280498 >> v2:172.21.15.111:3300/0 conn(0x7ff12806c0a0 0x7ff128141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:02.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.754+0000 7ff12d32c700 1 -- 172.21.15.89:0/710280498 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12815bc50 con 0x7ff12806bcd0 2023-10-10T20:44:02.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.754+0000 7ff12cb2b700 1 --2- 172.21.15.89:0/710280498 >> v2:172.21.15.111:3300/0 conn(0x7ff12806c0a0 0x7ff128141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:44:02.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.754+0000 7ff12d32c700 1 --2- 172.21.15.89:0/710280498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12806bcd0 0x7ff128140df0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7ff12806c6e0 tx=0x7ff124009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307ad536de9d7bed server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:02.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.754+0000 7ff11ffff700 1 -- 172.21.15.89:0/710280498 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12400e070 con 0x7ff12806bcd0 2023-10-10T20:44:02.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.754+0000 7ff11ffff700 1 -- 172.21.15.89:0/710280498 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12400ba50 con 0x7ff12806bcd0 2023-10-10T20:44:02.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.754+0000 7ff11ffff700 1 -- 172.21.15.89:0/710280498 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1240092e0 con 0x7ff12806bcd0 2023-10-10T20:44:02.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.755+0000 7ff12f590700 1 -- 172.21.15.89:0/710280498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12806bcd0 msgr2=0x7ff128140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:02.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.755+0000 7ff12f590700 1 --2- 172.21.15.89:0/710280498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12806bcd0 0x7ff128140df0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7ff12806c6e0 tx=0x7ff124009840 comp rx=0 tx=0).stop 2023-10-10T20:44:02.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.755+0000 7ff12f590700 1 -- 172.21.15.89:0/710280498 shutdown_connections 2023-10-10T20:44:02.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.755+0000 7ff12f590700 1 --2- 172.21.15.89:0/710280498 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1280c8e00 0x7ff1280c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:02.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.755+0000 7ff12f590700 1 --2- 172.21.15.89:0/710280498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12806bcd0 0x7ff128140df0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:02.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.755+0000 7ff12f590700 1 --2- 172.21.15.89:0/710280498 >> v2:172.21.15.111:3300/0 conn(0x7ff12806c0a0 0x7ff128141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:02.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.755+0000 7ff12f590700 1 -- 172.21.15.89:0/710280498 >> 172.21.15.89:0/710280498 conn(0x7ff1280ce600 msgr2=0x7ff1280cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:02.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.755+0000 7ff12f590700 1 -- 172.21.15.89:0/710280498 shutdown_connections 2023-10-10T20:44:02.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.755+0000 7ff12f590700 1 -- 172.21.15.89:0/710280498 wait complete. 2023-10-10T20:44:02.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.756+0000 7ff12f590700 1 Processor -- start 2023-10-10T20:44:02.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.756+0000 7ff12f590700 1 -- start start 2023-10-10T20:44:02.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.757+0000 7ff12f590700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12806bcd0 0x7ff12813c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:02.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.757+0000 7ff12f590700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff12806c0a0 0x7ff1280694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:02.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.757+0000 7ff12f590700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1280c8e00 0x7ff1280c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:02.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.757+0000 7ff12f590700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1280c6760 con 0x7ff12806bcd0 2023-10-10T20:44:02.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.757+0000 7ff12f590700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1280c64a0 con 0x7ff12806c0a0 2023-10-10T20:44:02.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.757+0000 7ff12f590700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12813c750 con 0x7ff1280c8e00 2023-10-10T20:44:02.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.757+0000 7ff12d32c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12806bcd0 0x7ff12813c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:02.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.757+0000 7ff12d32c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12806bcd0 0x7ff12813c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51998/0 (socket says 172.21.15.89:51998) 2023-10-10T20:44:02.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.757+0000 7ff12d32c700 1 -- 172.21.15.89:0/2681044136 learned_addr learned my addr 172.21.15.89:0/2681044136 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:02.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.757+0000 7ff12cb2b700 1 --2- 172.21.15.89:0/2681044136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff12806c0a0 0x7ff1280694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:02.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.757+0000 7ff12db2d700 1 --2- 172.21.15.89:0/2681044136 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1280c8e00 0x7ff1280c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:02.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.757+0000 7ff12d32c700 1 -- 172.21.15.89:0/2681044136 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1280c8e00 msgr2=0x7ff1280c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:02.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.757+0000 7ff12d32c700 1 --2- 172.21.15.89:0/2681044136 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1280c8e00 0x7ff1280c3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:02.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.757+0000 7ff12d32c700 1 -- 172.21.15.89:0/2681044136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff12806c0a0 msgr2=0x7ff1280694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:02.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.757+0000 7ff12d32c700 1 --2- 172.21.15.89:0/2681044136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff12806c0a0 0x7ff1280694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:02.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.757+0000 7ff12d32c700 1 -- 172.21.15.89:0/2681044136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff124009ae0 con 0x7ff12806bcd0 2023-10-10T20:44:02.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.757+0000 7ff12d32c700 1 --2- 172.21.15.89:0/2681044136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12806bcd0 0x7ff12813c210 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7ff124009430 tx=0x7ff124018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:02.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.757+0000 7ff11d7fa700 1 -- 172.21.15.89:0/2681044136 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff124008930 con 0x7ff12806bcd0 2023-10-10T20:44:02.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.758+0000 7ff11d7fa700 1 -- 172.21.15.89:0/2681044136 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff124008ab0 con 0x7ff12806bcd0 2023-10-10T20:44:02.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.758+0000 7ff11d7fa700 1 -- 172.21.15.89:0/2681044136 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff124008e70 con 0x7ff12806bcd0 2023-10-10T20:44:02.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.758+0000 7ff12f590700 1 -- 172.21.15.89:0/2681044136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12806a640 con 0x7ff12806bcd0 2023-10-10T20:44:02.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.758+0000 7ff12f590700 1 -- 172.21.15.89:0/2681044136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12806a080 con 0x7ff12806bcd0 2023-10-10T20:44:02.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.758+0000 7ff12f590700 1 -- 172.21.15.89:0/2681044136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12804c3c0 con 0x7ff12806bcd0 2023-10-10T20:44:02.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.759+0000 7ff11d7fa700 1 -- 172.21.15.89:0/2681044136 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff124003040 con 0x7ff12806bcd0 2023-10-10T20:44:02.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.759+0000 7ff11d7fa700 1 --2- 172.21.15.89:0/2681044136 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff10c067320 0x7ff10c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:02.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.759+0000 7ff11d7fa700 1 -- 172.21.15.89:0/2681044136 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff1240864e0 con 0x7ff12806bcd0 2023-10-10T20:44:02.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.759+0000 7ff12cb2b700 1 --2- 172.21.15.89:0/2681044136 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff10c067320 0x7ff10c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:02.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.760+0000 7ff12cb2b700 1 --2- 172.21.15.89:0/2681044136 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff10c067320 0x7ff10c0697d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff128156a90 tx=0x7ff118006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:02.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.764+0000 7ff11d7fa700 1 -- 172.21.15.89:0/2681044136 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff124053060 con 0x7ff12806bcd0 2023-10-10T20:44:02.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:02.991+0000 7ff12f590700 1 -- 172.21.15.89:0/2681044136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff1280ca3a0 con 0x7ff12806bcd0 2023-10-10T20:44:03.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.617+0000 7ff11d7fa700 1 -- 172.21.15.89:0/2681044136 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v132) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff124056680 con 0x7ff12806bcd0 2023-10-10T20:44:03.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.619+0000 7ff12f590700 1 -- 172.21.15.89:0/2681044136 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff10c067320 msgr2=0x7ff10c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:03.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.619+0000 7ff12f590700 1 --2- 172.21.15.89:0/2681044136 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff10c067320 0x7ff10c0697d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff128156a90 tx=0x7ff118006f90 comp rx=0 tx=0).stop 2023-10-10T20:44:03.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.619+0000 7ff12f590700 1 -- 172.21.15.89:0/2681044136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12806bcd0 msgr2=0x7ff12813c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:03.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.619+0000 7ff12f590700 1 --2- 172.21.15.89:0/2681044136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12806bcd0 0x7ff12813c210 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7ff124009430 tx=0x7ff124018000 comp rx=0 tx=0).stop 2023-10-10T20:44:03.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.619+0000 7ff12f590700 1 -- 172.21.15.89:0/2681044136 shutdown_connections 2023-10-10T20:44:03.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.619+0000 7ff12f590700 1 --2- 172.21.15.89:0/2681044136 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff1280c8e00 0x7ff1280c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:03.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.619+0000 7ff12f590700 1 --2- 172.21.15.89:0/2681044136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12806bcd0 0x7ff12813c210 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:03.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.619+0000 7ff12f590700 1 --2- 172.21.15.89:0/2681044136 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff10c067320 0x7ff10c0697d0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:03.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.619+0000 7ff12f590700 1 --2- 172.21.15.89:0/2681044136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff12806c0a0 0x7ff1280694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:03.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.619+0000 7ff12f590700 1 -- 172.21.15.89:0/2681044136 >> 172.21.15.89:0/2681044136 conn(0x7ff1280ce600 msgr2=0x7ff1280cd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:03.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.619+0000 7ff12f590700 1 -- 172.21.15.89:0/2681044136 shutdown_connections 2023-10-10T20:44:03.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.620+0000 7ff12f590700 1 -- 172.21.15.89:0/2681044136 wait complete. 2023-10-10T20:44:03.634 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:44:03.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.747+0000 7f830451c700 1 Processor -- start 2023-10-10T20:44:03.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.747+0000 7f830451c700 1 -- start start 2023-10-10T20:44:03.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.748+0000 7f830451c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f82fc159890 0x7f82fc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:03.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.748+0000 7f830451c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82fc159c60 0x7f82fc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:03.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.748+0000 7f830451c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82fc15a160 con 0x7f82fc15db10 2023-10-10T20:44:03.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.748+0000 7f830451c700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82fc0d13a0 con 0x7f82fc159890 2023-10-10T20:44:03.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.748+0000 7f830451c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc15e140 con 0x7f82fc159c60 2023-10-10T20:44:03.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.748+0000 7f8301ab7700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82fc159c60 0x7f82fc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:03.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.749+0000 7f8302ab9700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f82fc15db10 0x7f82fc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:36952/0 (socket says 172.21.15.89:36952) 2023-10-10T20:44:03.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.749+0000 7f83022b8700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f82fc159890 0x7f82fc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:03.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.749+0000 7f8302ab9700 1 -- 172.21.15.89:0/2090334889 learned_addr learned my addr 172.21.15.89:0/2090334889 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:44:03.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.749+0000 7f83012b6700 1 -- 172.21.15.89:0/2090334889 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3689251864 0 0) 0x7f82fc15a160 con 0x7f82fc15db10 2023-10-10T20:44:03.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.749+0000 7f83012b6700 1 -- 172.21.15.89:0/2090334889 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82e8003570 con 0x7f82fc15db10 2023-10-10T20:44:03.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.749+0000 7f8301ab7700 1 -- 172.21.15.89:0/2090334889 >> v2:172.21.15.111:3300/0 conn(0x7f82fc159890 msgr2=0x7f82fc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:03.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.749+0000 7f8301ab7700 1 --2- 172.21.15.89:0/2090334889 >> v2:172.21.15.111:3300/0 conn(0x7f82fc159890 0x7f82fc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:03.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.749+0000 7f8301ab7700 1 -- 172.21.15.89:0/2090334889 >> v1:172.21.15.111:6789/0 conn(0x7f82fc15db10 legacy=0x7f82fc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:03.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.749+0000 7f8301ab7700 1 -- 172.21.15.89:0/2090334889 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc15a3d0 con 0x7f82fc159c60 2023-10-10T20:44:03.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.749+0000 7f83022b8700 1 --2- 172.21.15.89:0/2090334889 >> v2:172.21.15.111:3300/0 conn(0x7f82fc159890 0x7f82fc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:44:03.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.749+0000 7f8301ab7700 1 --2- 172.21.15.89:0/2090334889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82fc159c60 0x7f82fc15d5d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0098e0 tx=0x7f82ec0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5677d5a451200de server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:03.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.749+0000 7f83012b6700 1 -- 172.21.15.89:0/2090334889 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec011630 con 0x7f82fc159c60 2023-10-10T20:44:03.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.749+0000 7f83012b6700 1 -- 172.21.15.89:0/2090334889 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ec011790 con 0x7f82fc159c60 2023-10-10T20:44:03.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.749+0000 7f83012b6700 1 -- 172.21.15.89:0/2090334889 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec011b30 con 0x7f82fc159c60 2023-10-10T20:44:03.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.750+0000 7f830451c700 1 -- 172.21.15.89:0/2090334889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82fc159c60 msgr2=0x7f82fc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:03.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.750+0000 7f830451c700 1 --2- 172.21.15.89:0/2090334889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82fc159c60 0x7f82fc15d5d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0098e0 tx=0x7f82ec0091a0 comp rx=0 tx=0).stop 2023-10-10T20:44:03.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.750+0000 7f830451c700 1 -- 172.21.15.89:0/2090334889 shutdown_connections 2023-10-10T20:44:03.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.750+0000 7f830451c700 1 --2- 172.21.15.89:0/2090334889 >> v2:172.21.15.111:3300/0 conn(0x7f82fc159890 0x7f82fc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:03.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.750+0000 7f830451c700 1 --2- 172.21.15.89:0/2090334889 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82fc159c60 0x7f82fc15d5d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:03.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.750+0000 7f830451c700 1 -- 172.21.15.89:0/2090334889 >> 172.21.15.89:0/2090334889 conn(0x7f82fc068e20 msgr2=0x7f82fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:03.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.750+0000 7f830451c700 1 -- 172.21.15.89:0/2090334889 shutdown_connections 2023-10-10T20:44:03.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.750+0000 7f830451c700 1 -- 172.21.15.89:0/2090334889 wait complete. 2023-10-10T20:44:03.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.751+0000 7f830451c700 1 Processor -- start 2023-10-10T20:44:03.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.751+0000 7f830451c700 1 -- start start 2023-10-10T20:44:03.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f830451c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82fc159890 0x7f82fc15c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:03.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f830451c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82fc159c60 0x7f82fc14a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:03.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f830451c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82fc15db10 0x7f82fc155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:03.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f830451c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc15d950 con 0x7f82fc159c60 2023-10-10T20:44:03.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f830451c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc14aab0 con 0x7f82fc159890 2023-10-10T20:44:03.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f830451c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc15d0b0 con 0x7f82fc15db10 2023-10-10T20:44:03.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f8301ab7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82fc159c60 0x7f82fc14a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:03.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f8301ab7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82fc159c60 0x7f82fc14a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52004/0 (socket says 172.21.15.89:52004) 2023-10-10T20:44:03.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f8301ab7700 1 -- 172.21.15.89:0/4218268179 learned_addr learned my addr 172.21.15.89:0/4218268179 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:03.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f83022b8700 1 --2- 172.21.15.89:0/4218268179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82fc159890 0x7f82fc15c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:03.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f8302ab9700 1 --2- 172.21.15.89:0/4218268179 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82fc15db10 0x7f82fc155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:03.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f8301ab7700 1 -- 172.21.15.89:0/4218268179 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82fc15db10 msgr2=0x7f82fc155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:03.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f8301ab7700 1 --2- 172.21.15.89:0/4218268179 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82fc15db10 0x7f82fc155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:03.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f8301ab7700 1 -- 172.21.15.89:0/4218268179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82fc159890 msgr2=0x7f82fc15c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:03.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f8301ab7700 1 --2- 172.21.15.89:0/4218268179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82fc159890 0x7f82fc15c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:03.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f8301ab7700 1 -- 172.21.15.89:0/4218268179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f8009a20 con 0x7f82fc159c60 2023-10-10T20:44:03.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f8301ab7700 1 --2- 172.21.15.89:0/4218268179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82fc159c60 0x7f82fc14a570 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f82ec013000 tx=0x7f82ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:03.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f82f27fc700 1 -- 172.21.15.89:0/4218268179 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec011900 con 0x7f82fc159c60 2023-10-10T20:44:03.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f82f27fc700 1 -- 172.21.15.89:0/4218268179 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ec011a60 con 0x7f82fc159c60 2023-10-10T20:44:03.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f830451c700 1 -- 172.21.15.89:0/4218268179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82ec009420 con 0x7f82fc159c60 2023-10-10T20:44:03.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.752+0000 7f82f27fc700 1 -- 172.21.15.89:0/4218268179 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec025640 con 0x7f82fc159c60 2023-10-10T20:44:03.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.753+0000 7f830451c700 1 -- 172.21.15.89:0/4218268179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82fc15d670 con 0x7f82fc159c60 2023-10-10T20:44:03.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.753+0000 7f830451c700 1 -- 172.21.15.89:0/4218268179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82fc04c3c0 con 0x7f82fc159c60 2023-10-10T20:44:03.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.754+0000 7f82f27fc700 1 -- 172.21.15.89:0/4218268179 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f82ec025a30 con 0x7f82fc159c60 2023-10-10T20:44:03.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.754+0000 7f82f27fc700 1 --2- 172.21.15.89:0/4218268179 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f82e0067320 0x7f82e00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:03.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.754+0000 7f83022b8700 1 --2- 172.21.15.89:0/4218268179 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f82e0067320 0x7f82e00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:03.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.754+0000 7f82f27fc700 1 -- 172.21.15.89:0/4218268179 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f82ec086340 con 0x7f82fc159c60 2023-10-10T20:44:03.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.755+0000 7f83022b8700 1 --2- 172.21.15.89:0/4218268179 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f82e0067320 0x7f82e00697d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009fd0 tx=0x7f82f800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:03.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.759+0000 7f82f27fc700 1 -- 172.21.15.89:0/4218268179 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82ec052fd0 con 0x7f82fc159c60 2023-10-10T20:44:03.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.986+0000 7f830451c700 1 -- 172.21.15.89:0/4218268179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82fc06a5b0 con 0x7f82fc159c60 2023-10-10T20:44:03.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.986+0000 7f82f27fc700 1 -- 172.21.15.89:0/4218268179 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+4604 (secure 0 0 0) 0x7f82ec0565f0 con 0x7f82fc159c60 2023-10-10T20:44:03.989 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:44:03.989 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":132,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/4215135154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":4215135154},{"type":"v1","addr":"172.21.15.89:6837","nonce":4215135154}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127},{"gid":7088,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/940990808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":940990808},{"type":"v1","addr":"172.21.15.111:6834","nonce":940990808}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127},{"gid":7091,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/92922277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":92922277},{"type":"v1","addr":"172.21.15.111:6835","nonce":92922277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:43:59.950797+0000","modified":"2023-10-10T20:44:03.611977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9135":{"gid":9135,"name":"a","rank":0,"incarnation":128,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6836/3558240892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3558240892},{"type":"v1","addr":"172.21.15.89:6836","nonce":3558240892}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-10T20:44:03.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.988+0000 7f830451c700 1 -- 172.21.15.89:0/4218268179 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f82e0067320 msgr2=0x7f82e00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:03.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.988+0000 7f830451c700 1 --2- 172.21.15.89:0/4218268179 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f82e0067320 0x7f82e00697d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009fd0 tx=0x7f82f800b040 comp rx=0 tx=0).stop 2023-10-10T20:44:03.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.988+0000 7f830451c700 1 -- 172.21.15.89:0/4218268179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82fc159c60 msgr2=0x7f82fc14a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:03.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.988+0000 7f830451c700 1 --2- 172.21.15.89:0/4218268179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82fc159c60 0x7f82fc14a570 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f82ec013000 tx=0x7f82ec017000 comp rx=0 tx=0).stop 2023-10-10T20:44:03.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.988+0000 7f830451c700 1 -- 172.21.15.89:0/4218268179 shutdown_connections 2023-10-10T20:44:03.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.988+0000 7f830451c700 1 --2- 172.21.15.89:0/4218268179 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f82fc15db10 0x7f82fc155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:03.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.988+0000 7f830451c700 1 --2- 172.21.15.89:0/4218268179 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f82e0067320 0x7f82e00697d0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:03.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.988+0000 7f830451c700 1 --2- 172.21.15.89:0/4218268179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82fc159890 0x7f82fc15c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:03.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.988+0000 7f830451c700 1 --2- 172.21.15.89:0/4218268179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82fc159c60 0x7f82fc14a570 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:03.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.988+0000 7f830451c700 1 -- 172.21.15.89:0/4218268179 >> 172.21.15.89:0/4218268179 conn(0x7f82fc068e20 msgr2=0x7f82fc152d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:03.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.988+0000 7f830451c700 1 -- 172.21.15.89:0/4218268179 shutdown_connections 2023-10-10T20:44:03.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:03.988+0000 7f830451c700 1 -- 172.21.15.89:0/4218268179 wait complete. 2023-10-10T20:44:03.993 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 132 2023-10-10T20:44:04.002 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:44:04.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.116+0000 7f18800a1700 1 Processor -- start 2023-10-10T20:44:04.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.116+0000 7f18800a1700 1 -- start start 2023-10-10T20:44:04.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.117+0000 7f18800a1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1878159890 0x7f187815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.117+0000 7f18800a1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1878069560 0x7f18780699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.117+0000 7f18800a1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187815a160 con 0x7f1878159c60 2023-10-10T20:44:04.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.117+0000 7f18800a1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1878069f20 con 0x7f1878069560 2023-10-10T20:44:04.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.117+0000 7f18800a1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18780d13a0 con 0x7f1878159890 2023-10-10T20:44:04.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.117+0000 7f187e63e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1878069560 0x7f18780699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.117+0000 7f187d63c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1878159c60 0x7f187815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37230/0 (socket says 172.21.15.89:37230) 2023-10-10T20:44:04.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.117+0000 7f187d63c700 1 -- 172.21.15.89:0/3918256925 learned_addr learned my addr 172.21.15.89:0/3918256925 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:44:04.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.117+0000 7f187de3d700 1 --2- 172.21.15.89:0/3918256925 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1878159890 0x7f187815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.118+0000 7f187e63e700 1 -- 172.21.15.89:0/3918256925 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1878159890 msgr2=0x7f187815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:04.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.118+0000 7f187e63e700 1 --2- 172.21.15.89:0/3918256925 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1878159890 0x7f187815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.118+0000 7f187e63e700 1 -- 172.21.15.89:0/3918256925 >> v1:172.21.15.89:6789/0 conn(0x7f1878159c60 legacy=0x7f187815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:04.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.118+0000 7f187e63e700 1 -- 172.21.15.89:0/3918256925 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187815e140 con 0x7f1878069560 2023-10-10T20:44:04.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.118+0000 7f187ce3b700 1 -- 172.21.15.89:0/3918256925 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233247991 0 0) 0x7f187815a160 con 0x7f1878159c60 2023-10-10T20:44:04.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.118+0000 7f187e63e700 1 --2- 172.21.15.89:0/3918256925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1878069560 0x7f18780699e0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f1870009e60 tx=0x7f1870009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481080aac69dc14d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:04.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.118+0000 7f187ce3b700 1 -- 172.21.15.89:0/3918256925 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187000e070 con 0x7f1878069560 2023-10-10T20:44:04.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.118+0000 7f187ce3b700 1 -- 172.21.15.89:0/3918256925 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187000bc30 con 0x7f1878069560 2023-10-10T20:44:04.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.118+0000 7f187ce3b700 1 -- 172.21.15.89:0/3918256925 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1870012430 con 0x7f1878069560 2023-10-10T20:44:04.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.118+0000 7f18800a1700 1 -- 172.21.15.89:0/3918256925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1878069560 msgr2=0x7f18780699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:04.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.118+0000 7f18800a1700 1 --2- 172.21.15.89:0/3918256925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1878069560 0x7f18780699e0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f1870009e60 tx=0x7f1870009780 comp rx=0 tx=0).stop 2023-10-10T20:44:04.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.118+0000 7f18800a1700 1 -- 172.21.15.89:0/3918256925 shutdown_connections 2023-10-10T20:44:04.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.118+0000 7f18800a1700 1 --2- 172.21.15.89:0/3918256925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1878069560 0x7f18780699e0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.118+0000 7f18800a1700 1 --2- 172.21.15.89:0/3918256925 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1878159890 0x7f187815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.118+0000 7f18800a1700 1 -- 172.21.15.89:0/3918256925 >> 172.21.15.89:0/3918256925 conn(0x7f1878068e20 msgr2=0x7f18780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:04.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.119+0000 7f18800a1700 1 -- 172.21.15.89:0/3918256925 shutdown_connections 2023-10-10T20:44:04.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.119+0000 7f18800a1700 1 -- 172.21.15.89:0/3918256925 wait complete. 2023-10-10T20:44:04.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.120+0000 7f18800a1700 1 Processor -- start 2023-10-10T20:44:04.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.120+0000 7f18800a1700 1 -- start start 2023-10-10T20:44:04.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.120+0000 7f18800a1700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1878069560 0x7f187815d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.120+0000 7f18800a1700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1878159890 0x7f18780b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.120+0000 7f18800a1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1878159c60 0x7f1878154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.120+0000 7f18800a1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1878155770 con 0x7f1878159c60 2023-10-10T20:44:04.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.120+0000 7f18800a1700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187815efc0 con 0x7f1878069560 2023-10-10T20:44:04.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.120+0000 7f18800a1700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1878154cc0 con 0x7f1878159890 2023-10-10T20:44:04.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.120+0000 7f187e63e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1878159c60 0x7f1878154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.120+0000 7f187e63e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1878159c60 0x7f1878154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52018/0 (socket says 172.21.15.89:52018) 2023-10-10T20:44:04.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.120+0000 7f187e63e700 1 -- 172.21.15.89:0/504924642 learned_addr learned my addr 172.21.15.89:0/504924642 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:04.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.121+0000 7f187d63c700 1 --2- 172.21.15.89:0/504924642 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1878159890 0x7f18780b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.121+0000 7f187de3d700 1 --2- 172.21.15.89:0/504924642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1878069560 0x7f187815d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.121+0000 7f187e63e700 1 -- 172.21.15.89:0/504924642 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1878159890 msgr2=0x7f18780b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:04.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.121+0000 7f187e63e700 1 --2- 172.21.15.89:0/504924642 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1878159890 0x7f18780b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.121+0000 7f187e63e700 1 -- 172.21.15.89:0/504924642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1878069560 msgr2=0x7f187815d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:04.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.121+0000 7f187e63e700 1 --2- 172.21.15.89:0/504924642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1878069560 0x7f187815d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.121+0000 7f187e63e700 1 -- 172.21.15.89:0/504924642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1870009a20 con 0x7f1878159c60 2023-10-10T20:44:04.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.121+0000 7f187d63c700 1 --2- 172.21.15.89:0/504924642 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1878159890 0x7f18780b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:44:04.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.121+0000 7f187e63e700 1 --2- 172.21.15.89:0/504924642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1878159c60 0x7f1878154780 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f18780899c0 tx=0x7f1870019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:04.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.121+0000 7f186dffb700 1 -- 172.21.15.89:0/504924642 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1870012e10 con 0x7f1878159c60 2023-10-10T20:44:04.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.121+0000 7f186dffb700 1 -- 172.21.15.89:0/504924642 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187001d630 con 0x7f1878159c60 2023-10-10T20:44:04.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.121+0000 7f18800a1700 1 -- 172.21.15.89:0/504924642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18780b3fe0 con 0x7f1878159c60 2023-10-10T20:44:04.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.121+0000 7f186dffb700 1 -- 172.21.15.89:0/504924642 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187001d9f0 con 0x7f1878159c60 2023-10-10T20:44:04.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.121+0000 7f18800a1700 1 -- 172.21.15.89:0/504924642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187815b8a0 con 0x7f1878159c60 2023-10-10T20:44:04.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.122+0000 7f18800a1700 1 -- 172.21.15.89:0/504924642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f187815ed90 con 0x7f1878159c60 2023-10-10T20:44:04.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.122+0000 7f186dffb700 1 -- 172.21.15.89:0/504924642 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1870024070 con 0x7f1878159c60 2023-10-10T20:44:04.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.123+0000 7f186dffb700 1 --2- 172.21.15.89:0/504924642 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f185c067320 0x7f185c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.123+0000 7f186dffb700 1 -- 172.21.15.89:0/504924642 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1870087580 con 0x7f1878159c60 2023-10-10T20:44:04.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.123+0000 7f187de3d700 1 --2- 172.21.15.89:0/504924642 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f185c067320 0x7f185c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.123+0000 7f187de3d700 1 --2- 172.21.15.89:0/504924642 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f185c067320 0x7f185c0697d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1874006fd0 tx=0x7f1874008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:04.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.127+0000 7f186dffb700 1 -- 172.21.15.89:0/504924642 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f187815ed90 con 0x7f1878159c60 2023-10-10T20:44:04.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.313+0000 7f18800a1700 1 -- 172.21.15.89:0/504924642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f187815f170 con 0x7f1878159c60 2023-10-10T20:44:04.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.313+0000 7f186dffb700 1 -- 172.21.15.89:0/504924642 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+15364 (secure 0 0 0) 0x7f187815f170 con 0x7f1878159c60 2023-10-10T20:44:04.315 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:44:04.316 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":198,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:44:01.578606+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:43:58.313137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-10T20:43:58.914819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:44:04.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.315+0000 7f18800a1700 1 -- 172.21.15.89:0/504924642 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f185c067320 msgr2=0x7f185c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:04.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.315+0000 7f18800a1700 1 --2- 172.21.15.89:0/504924642 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f185c067320 0x7f185c0697d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1874006fd0 tx=0x7f1874008040 comp rx=0 tx=0).stop 2023-10-10T20:44:04.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.315+0000 7f18800a1700 1 -- 172.21.15.89:0/504924642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1878159c60 msgr2=0x7f1878154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:04.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.315+0000 7f18800a1700 1 --2- 172.21.15.89:0/504924642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1878159c60 0x7f1878154780 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f18780899c0 tx=0x7f1870019000 comp rx=0 tx=0).stop 2023-10-10T20:44:04.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.315+0000 7f18800a1700 1 -- 172.21.15.89:0/504924642 shutdown_connections 2023-10-10T20:44:04.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.315+0000 7f18800a1700 1 --2- 172.21.15.89:0/504924642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1878159c60 0x7f1878154780 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.315+0000 7f18800a1700 1 --2- 172.21.15.89:0/504924642 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f185c067320 0x7f185c0697d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.315+0000 7f18800a1700 1 --2- 172.21.15.89:0/504924642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1878069560 0x7f187815d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.315+0000 7f18800a1700 1 --2- 172.21.15.89:0/504924642 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1878159890 0x7f18780b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.315+0000 7f18800a1700 1 -- 172.21.15.89:0/504924642 >> 172.21.15.89:0/504924642 conn(0x7f1878068e20 msgr2=0x7f18781580a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:04.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.315+0000 7f18800a1700 1 -- 172.21.15.89:0/504924642 shutdown_connections 2023-10-10T20:44:04.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.315+0000 7f18800a1700 1 -- 172.21.15.89:0/504924642 wait complete. 2023-10-10T20:44:04.329 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:44:04.330 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:44:04.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.442+0000 7f167cda0700 1 Processor -- start 2023-10-10T20:44:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.442+0000 7f167cda0700 1 -- start start 2023-10-10T20:44:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.443+0000 7f167cda0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f16781597d0 0x7f167806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.443+0000 7f167cda0700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f16780c8400 0x7f167806ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.443+0000 7f167cda0700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f167806e8e0 con 0x7f167806c120 2023-10-10T20:44:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.443+0000 7f167cda0700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f167815a8f0 con 0x7f16780c8400 2023-10-10T20:44:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.443+0000 7f167cda0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167815abd0 con 0x7f16781597d0 2023-10-10T20:44:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.443+0000 7f1676ffd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f16780c8400 0x7f167806ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.443+0000 7f16777fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f16781597d0 0x7f167806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.443+0000 7f1677fff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f167806c120 0x7f1678068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:36962/0 (socket says 172.21.15.89:36962) 2023-10-10T20:44:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.444+0000 7f1677fff700 1 -- 172.21.15.89:0/450368338 learned_addr learned my addr 172.21.15.89:0/450368338 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:44:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.444+0000 7f16777fe700 1 -- 172.21.15.89:0/450368338 >> v2:172.21.15.111:3300/0 conn(0x7f16780c8400 msgr2=0x7f167806ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.444+0000 7f16777fe700 1 --2- 172.21.15.89:0/450368338 >> v2:172.21.15.111:3300/0 conn(0x7f16780c8400 0x7f167806ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.444+0000 7f16767fc700 1 -- 172.21.15.89:0/450368338 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536135858 0 0) 0x7f167806e8e0 con 0x7f167806c120 2023-10-10T20:44:04.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.444+0000 7f16777fe700 1 -- 172.21.15.89:0/450368338 >> v1:172.21.15.111:6789/0 conn(0x7f167806c120 legacy=0x7f1678068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:04.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.444+0000 7f16777fe700 1 -- 172.21.15.89:0/450368338 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167815aeb0 con 0x7f16781597d0 2023-10-10T20:44:04.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.444+0000 7f16777fe700 1 --2- 172.21.15.89:0/450368338 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f16781597d0 0x7f167806bbe0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1668009e60 tx=0x7f1668009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61115424ed12abdc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:04.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.444+0000 7f16767fc700 1 -- 172.21.15.89:0/450368338 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166800e070 con 0x7f16781597d0 2023-10-10T20:44:04.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.444+0000 7f16767fc700 1 -- 172.21.15.89:0/450368338 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166800bb60 con 0x7f16781597d0 2023-10-10T20:44:04.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.444+0000 7f16767fc700 1 -- 172.21.15.89:0/450368338 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668012430 con 0x7f16781597d0 2023-10-10T20:44:04.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.445+0000 7f167cda0700 1 -- 172.21.15.89:0/450368338 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f16781597d0 msgr2=0x7f167806bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:04.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.445+0000 7f167cda0700 1 --2- 172.21.15.89:0/450368338 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f16781597d0 0x7f167806bbe0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1668009e60 tx=0x7f1668009780 comp rx=0 tx=0).stop 2023-10-10T20:44:04.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.445+0000 7f167cda0700 1 -- 172.21.15.89:0/450368338 shutdown_connections 2023-10-10T20:44:04.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.445+0000 7f167cda0700 1 --2- 172.21.15.89:0/450368338 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f16781597d0 0x7f167806bbe0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.445+0000 7f167cda0700 1 --2- 172.21.15.89:0/450368338 >> v2:172.21.15.111:3300/0 conn(0x7f16780c8400 0x7f167806ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.445+0000 7f167cda0700 1 -- 172.21.15.89:0/450368338 >> 172.21.15.89:0/450368338 conn(0x7f167806b450 msgr2=0x7f16780cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:04.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.445+0000 7f167cda0700 1 -- 172.21.15.89:0/450368338 shutdown_connections 2023-10-10T20:44:04.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.445+0000 7f167cda0700 1 -- 172.21.15.89:0/450368338 wait complete. 2023-10-10T20:44:04.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.446+0000 7f167cda0700 1 Processor -- start 2023-10-10T20:44:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.446+0000 7f167cda0700 1 -- start start 2023-10-10T20:44:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f167cda0700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f167806c120 0x7f16780af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f167cda0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16780c8400 0x7f16780c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f167cda0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16781597d0 0x7f16780c2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f167cda0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16780c3f70 con 0x7f16781597d0 2023-10-10T20:44:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f167cda0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167816b290 con 0x7f16780c8400 2023-10-10T20:44:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f167cda0700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167816ac70 con 0x7f167806c120 2023-10-10T20:44:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f1677fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16781597d0 0x7f16780c2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f1677fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16781597d0 0x7f16780c2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52026/0 (socket says 172.21.15.89:52026) 2023-10-10T20:44:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f1677fff700 1 -- 172.21.15.89:0/827481943 learned_addr learned my addr 172.21.15.89:0/827481943 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f16777fe700 1 --2- 172.21.15.89:0/827481943 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f167806c120 0x7f16780af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f1676ffd700 1 --2- 172.21.15.89:0/827481943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16780c8400 0x7f16780c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f1677fff700 1 -- 172.21.15.89:0/827481943 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f167806c120 msgr2=0x7f16780af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f1677fff700 1 --2- 172.21.15.89:0/827481943 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f167806c120 0x7f16780af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f1677fff700 1 -- 172.21.15.89:0/827481943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16780c8400 msgr2=0x7f16780c2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f1677fff700 1 --2- 172.21.15.89:0/827481943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16780c8400 0x7f16780c2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f1677fff700 1 -- 172.21.15.89:0/827481943 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1668009a20 con 0x7f16781597d0 2023-10-10T20:44:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f1676ffd700 1 --2- 172.21.15.89:0/827481943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16780c8400 0x7f16780c2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:44:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f16777fe700 1 --2- 172.21.15.89:0/827481943 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f167806c120 0x7f16780af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:44:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f1677fff700 1 --2- 172.21.15.89:0/827481943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16781597d0 0x7f16780c2f50 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f167000c530 tx=0x7f1670007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f165f7fe700 1 -- 172.21.15.89:0/827481943 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670005b30 con 0x7f16781597d0 2023-10-10T20:44:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f165f7fe700 1 -- 172.21.15.89:0/827481943 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1670005c90 con 0x7f16781597d0 2023-10-10T20:44:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f167cda0700 1 -- 172.21.15.89:0/827481943 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167816b530 con 0x7f16781597d0 2023-10-10T20:44:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.447+0000 7f165f7fe700 1 -- 172.21.15.89:0/827481943 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670007320 con 0x7f16781597d0 2023-10-10T20:44:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.448+0000 7f167cda0700 1 -- 172.21.15.89:0/827481943 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167816af50 con 0x7f16781597d0 2023-10-10T20:44:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.449+0000 7f167cda0700 1 -- 172.21.15.89:0/827481943 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167804c3c0 con 0x7f16781597d0 2023-10-10T20:44:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.449+0000 7f165f7fe700 1 -- 172.21.15.89:0/827481943 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f167001b3b0 con 0x7f16781597d0 2023-10-10T20:44:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.449+0000 7f165f7fe700 1 --2- 172.21.15.89:0/827481943 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1664067320 0x7f16640697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.449+0000 7f165f7fe700 1 -- 172.21.15.89:0/827481943 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1670084130 con 0x7f16781597d0 2023-10-10T20:44:04.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.449+0000 7f16777fe700 1 --2- 172.21.15.89:0/827481943 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1664067320 0x7f16640697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.450+0000 7f16777fe700 1 --2- 172.21.15.89:0/827481943 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1664067320 0x7f16640697d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f16680097c0 tx=0x7f1668019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:04.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.453+0000 7f165f7fe700 1 -- 172.21.15.89:0/827481943 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1670050dc0 con 0x7f16781597d0 2023-10-10T20:44:04.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.636+0000 7f167cda0700 1 -- 172.21.15.89:0/827481943 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1678069280 con 0x7f1664067320 2023-10-10T20:44:04.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.643+0000 7f165f7fe700 1 -- 172.21.15.89:0/827481943 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274616 (secure 0 0 0) 0x7f1678069280 con 0x7f1664067320 2023-10-10T20:44:04.648 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:44:04.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.646+0000 7f167cda0700 1 -- 172.21.15.89:0/827481943 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1664067320 msgr2=0x7f16640697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:04.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.646+0000 7f167cda0700 1 --2- 172.21.15.89:0/827481943 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1664067320 0x7f16640697d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f16680097c0 tx=0x7f1668019000 comp rx=0 tx=0).stop 2023-10-10T20:44:04.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.646+0000 7f167cda0700 1 -- 172.21.15.89:0/827481943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16781597d0 msgr2=0x7f16780c2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:04.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.646+0000 7f167cda0700 1 --2- 172.21.15.89:0/827481943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16781597d0 0x7f16780c2f50 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f167000c530 tx=0x7f1670007f90 comp rx=0 tx=0).stop 2023-10-10T20:44:04.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.646+0000 7f167cda0700 1 -- 172.21.15.89:0/827481943 shutdown_connections 2023-10-10T20:44:04.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.646+0000 7f167cda0700 1 --2- 172.21.15.89:0/827481943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16781597d0 0x7f16780c2f50 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.646+0000 7f167cda0700 1 --2- 172.21.15.89:0/827481943 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f167806c120 0x7f16780af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.646+0000 7f167cda0700 1 --2- 172.21.15.89:0/827481943 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1664067320 0x7f16640697d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.646+0000 7f167cda0700 1 --2- 172.21.15.89:0/827481943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16780c8400 0x7f16780c2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.646+0000 7f167cda0700 1 -- 172.21.15.89:0/827481943 >> 172.21.15.89:0/827481943 conn(0x7f167806b450 msgr2=0x7f1678158470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:04.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.646+0000 7f167cda0700 1 -- 172.21.15.89:0/827481943 shutdown_connections 2023-10-10T20:44:04.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.646+0000 7f167cda0700 1 -- 172.21.15.89:0/827481943 wait complete. 2023-10-10T20:44:04.653 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:44:04.660 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2023-10-10T20:44:02.763398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2431524,"kb_used_data":20452,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752543196,"statfs":{"total":773094113280,"available":770604232704,"internally_reserved":0,"allocated":20942848,"data_stored":10046367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.203032"},"pg_stats":[{"pgid":"34.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:58.558180+0000","last_change":"2023-10-10T20:43:58.558180+0000","last_active":"2023-10-10T20:43:58.558180+0000","last_peered":"2023-10-10T20:43:58.558180+0000","last_clean":"2023-10-10T20:43:58.558180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:58.558180+0000","last_undegraded":"2023-10-10T20:43:58.558180+0000","last_fullsized":"2023-10-10T20:43:58.558180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:43:59.559987+0000","last_change":"2023-10-10T20:43:59.559987+0000","last_active":"2023-10-10T20:43:59.559987+0000","last_peered":"2023-10-10T20:43:59.559987+0000","last_clean":"2023-10-10T20:43:59.559987+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:43:59.559987+0000","last_undegraded":"2023-10-10T20:43:59.559987+0000","last_fullsized":"2023-10-10T20:43:59.559987+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":337,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-10T20:43:55.553196+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:43:55.553196+0000","last_peered":"2023-10-10T20:43:55.553196+0000","last_clean":"2023-10-10T20:43:55.553196+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:43:55.553196+0000","last_undegraded":"2023-10-10T20:43:55.553196+0000","last_fullsized":"2023-10-10T20:43:55.553196+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574966,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304376,"kb_used_data":2992,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067464,"statfs":{"total":96636764160,"available":96325083136,"internally_reserved":0,"allocated":3063808,"data_stored":1698692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51300000000000001,"15min":0.72699999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.748,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.504,"15min":0.59299999999999997},"min":{"1min":0.33300000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":4.1959999999999997},"last":0.56399999999999995}]},{"osd":1,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51300000000000001,"15min":0.85299999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77900000000000003,"5min":0.78800000000000003,"15min":10.1},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53700000000000003,"15min":0.67400000000000004},"min":{"1min":0.375,"5min":0.33900000000000002,"15min":0.28999999999999998},"max":{"1min":0.79200000000000004,"5min":0.81799999999999995,"15min":6.806},"last":0.52500000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.72099999999999997},"min":{"1min":0.378,"5min":0.308,"15min":0.308},"max":{"1min":0.75700000000000001,"5min":0.89000000000000001,"15min":6.9340000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53300000000000003,"15min":0.67600000000000005},"min":{"1min":0.39600000000000002,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.747,"5min":0.77100000000000002,"15min":6.9870000000000001},"last":0.627}]},{"osd":3,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54300000000000004,"15min":0.69299999999999995},"min":{"1min":0.38500000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72999999999999998,"5min":0.754,"15min":6.6719999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56200000000000006,"15min":0.63900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":4.1609999999999996},"last":0.61399999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.47099999999999997,"15min":0.46700000000000003},"min":{"1min":0.30599999999999999,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.73199999999999998,"15min":0.77200000000000002},"last":0.503},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.73899999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.64200000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.53800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.66500000000000004,"5min":0.86099999999999999,"15min":1.3180000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.34899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64800000000000002,"5min":0.78200000000000003,"15min":0.84099999999999997},"last":0.66000000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71499999999999997,"5min":0.77100000000000002,"15min":1.7509999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.45900000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73099999999999998,"5min":0.84099999999999997,"15min":1.7090000000000001},"last":0.68799999999999994}]}]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303796,"kb_used_data":2412,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068044,"statfs":{"total":96636764160,"available":96325677056,"internally_reserved":0,"allocated":2469888,"data_stored":1108324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52900000000000003,"15min":0.626},"min":{"1min":0.31,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.80500000000000005,"5min":0.83099999999999996,"15min":7.7590000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52700000000000002,"15min":0.65500000000000003},"min":{"1min":0.32800000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79700000000000004,"5min":1.075,"15min":7.8090000000000002},"last":0.503}]},{"osd":1,"last update":"Tue Oct 10 20:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53200000000000003,"15min":0.65500000000000003},"min":{"1min":0.40600000000000003,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":7.867},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55700000000000005,"15min":0.78600000000000003},"min":{"1min":0.36099999999999999,"5min":0.35499999999999998,"15min":0.33700000000000002},"max":{"1min":0.85999999999999999,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.55100000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52800000000000002,"15min":0.57399999999999995},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83699999999999997,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.76100000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.318},"max":{"1min":0.875,"5min":2.641,"15min":6.7629999999999999},"last":0.59199999999999997}]},{"osd":3,"last update":"Tue Oct 10 20:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.89300000000000002,"5min":1.107,"15min":4.8620000000000001},"last":0.441},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55500000000000005,"15min":0.66800000000000004},"min":{"1min":0.40200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.78900000000000003,"5min":1.718,"15min":7.9009999999999998},"last":0.61499999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52700000000000002,"15min":0.48599999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.82399999999999995,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60599999999999998,"15min":0.57699999999999996},"min":{"1min":0.39000000000000001,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":0.80500000000000005,"5min":1.032,"15min":1.032},"last":0.60699999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.53300000000000003,"15min":0.502},"min":{"1min":0.29899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73099999999999998,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58399999999999996,"15min":0.54300000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.33600000000000002},"max":{"1min":0.85899999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.63300000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57499999999999996,"15min":0.53500000000000003},"min":{"1min":0.33300000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.83899999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59999999999999998,"15min":0.55900000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.35099999999999998},"max":{"1min":0.80100000000000005,"5min":1.681,"15min":1.681},"last":0.68700000000000006}]}]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303796,"kb_used_data":2412,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068044,"statfs":{"total":96636764160,"available":96325677056,"internally_reserved":0,"allocated":2469888,"data_stored":1108324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.31900000000000001,"15min":0.42699999999999999},"min":{"1min":0.248,"5min":0.20999999999999999,"15min":0.17899999999999999},"max":{"1min":0.41399999999999998,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.33600000000000002,"15min":0.35299999999999998},"min":{"1min":0.245,"5min":0.22500000000000001,"15min":0.20899999999999999},"max":{"1min":0.52300000000000002,"5min":0.65500000000000003,"15min":1.194},"last":0.28999999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.35199999999999998},"min":{"1min":0.255,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":1.2869999999999999},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36299999999999999,"15min":0.45800000000000002},"min":{"1min":0.29399999999999998,"5min":0.23999999999999999,"15min":0.22},"max":{"1min":0.50700000000000001,"5min":0.54200000000000004,"15min":5.0919999999999996},"last":0.27500000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.48199999999999998},"min":{"1min":0.28899999999999998,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":5.1159999999999997},"last":0.314},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.372,"15min":0.39600000000000002},"min":{"1min":0.29599999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.54300000000000004,"5min":0.57799999999999996,"15min":1.2170000000000001},"last":0.36699999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.30199999999999999},"max":{"1min":0.747,"5min":0.92400000000000004,"15min":1.254},"last":0.433},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.66000000000000003},"min":{"1min":0.40100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78900000000000003,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.443}]},{"osd":5,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34399999999999997},"max":{"1min":0.75,"5min":0.97599999999999998,"15min":1.1799999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57099999999999995,"15min":0.59699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.79700000000000004,"5min":0.878,"15min":1.3109999999999999},"last":0.38600000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59799999999999998,"15min":0.76700000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.35199999999999998},"max":{"1min":0.76400000000000001,"5min":0.995,"15min":5.0659999999999998},"last":0.378},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.38600000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76600000000000001,"5min":0.85799999999999998,"15min":1.171},"last":0.47599999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.60899999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.36499999999999999},"max":{"1min":0.72799999999999998,"5min":1.004,"15min":1.214},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.68400000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78200000000000003,"5min":0.90500000000000003,"15min":4.0499999999999998},"last":0.46300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304372,"kb_used_data":2988,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067468,"statfs":{"total":96636764160,"available":96325087232,"internally_reserved":0,"allocated":3059712,"data_stored":1697731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.314,"15min":0.46400000000000002},"min":{"1min":0.23100000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.40000000000000002,"5min":0.63400000000000001,"15min":7.9249999999999998},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.32600000000000001,"15min":0.45100000000000001},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46300000000000002,"5min":0.53800000000000003,"15min":6.6299999999999999},"last":0.307}]},{"osd":2,"last update":"Tue Oct 10 20:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.33600000000000002,"15min":0.48699999999999999},"min":{"1min":0.23899999999999999,"5min":0.22,"15min":0.216},"max":{"1min":0.46800000000000003,"5min":0.67800000000000005,"15min":8.2509999999999994},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.34699999999999998,"15min":0.47399999999999998},"min":{"1min":0.26400000000000001,"5min":0.23100000000000001,"15min":0.222},"max":{"1min":0.502,"5min":0.57899999999999996,"15min":6.7370000000000001},"last":0.27900000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36199999999999999,"15min":0.497},"min":{"1min":0.28999999999999998,"5min":0.22,"15min":0.21099999999999999},"max":{"1min":0.46400000000000002,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40400000000000003,"15min":0.54800000000000004},"min":{"1min":0.32600000000000001,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.51500000000000001,"5min":0.71699999999999997,"15min":7.8929999999999998},"last":0.33700000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56100000000000005,"15min":0.67000000000000004},"min":{"1min":0.40300000000000002,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":6.6929999999999996},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.71099999999999997},"min":{"1min":0.40899999999999997,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.84399999999999997,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.751}]},{"osd":5,"last update":"Tue Oct 10 20:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55900000000000005,"15min":0.69499999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.36299999999999999},"max":{"1min":0.80200000000000005,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59099999999999997,"15min":0.72099999999999997},"min":{"1min":0.42199999999999999,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":7.8659999999999997},"last":0.70999999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60599999999999998,"15min":0.753},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76000000000000001,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61299999999999999,"15min":0.71699999999999997},"min":{"1min":0.46500000000000002,"5min":0.41399999999999998,"15min":0.36599999999999999},"max":{"1min":0.875,"5min":0.90300000000000002,"15min":6.6660000000000004},"last":0.68600000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60799999999999998,"15min":0.70899999999999996},"min":{"1min":0.435,"5min":0.39900000000000002,"15min":0.36299999999999999},"max":{"1min":0.90600000000000003,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58399999999999996,"15min":0.72499999999999998},"min":{"1min":0.45800000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.89500000000000002,"15min":7.7370000000000001},"last":0.66800000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303796,"kb_used_data":2412,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068044,"statfs":{"total":96636764160,"available":96325677056,"internally_reserved":0,"allocated":2469888,"data_stored":1108324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.309,"15min":0.32100000000000001},"min":{"1min":0.22,"5min":0.189,"15min":0.189},"max":{"1min":0.36899999999999999,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32600000000000001,"15min":0.34599999999999997},"min":{"1min":0.22,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.505,"5min":0.54200000000000004,"15min":0.746},"last":0.39700000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33400000000000002,"15min":0.61199999999999999},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.40000000000000002,"5min":0.63,"15min":11.967000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.35199999999999998,"15min":0.44400000000000001},"min":{"1min":0.26400000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.47599999999999998,"5min":0.52700000000000002,"15min":4.1100000000000003},"last":0.434}]},{"osd":3,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.34100000000000003,"15min":0.49199999999999999},"min":{"1min":0.26200000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.378,"5min":0.53600000000000003,"15min":6.3040000000000003},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.38700000000000001,"15min":0.66400000000000003},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.497,"5min":0.65900000000000003,"15min":11.935},"last":0.46600000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83399999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.40999999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.752,"5min":1.302,"15min":1.302},"last":0.56899999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.56100000000000005},"min":{"1min":0.41499999999999998,"5min":0.38,"15min":0.378},"max":{"1min":0.73999999999999999,"5min":0.83999999999999997,"15min":0.95699999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55500000000000005,"15min":0.57599999999999996},"min":{"1min":0.39300000000000002,"5min":0.35999999999999999,"15min":0.34300000000000003},"max":{"1min":0.77700000000000002,"5min":0.83999999999999997,"15min":0.96099999999999997},"last":0.72299999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.40799999999999997,"5min":0.33400000000000002,"15min":0.33200000000000002},"max":{"1min":0.84299999999999997,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.42999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.58099999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.41899999999999998,"5min":0.39900000000000002,"15min":0.38700000000000001},"max":{"1min":0.80700000000000005,"5min":0.89700000000000002,"15min":1.052},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59599999999999997,"15min":0.72499999999999998},"min":{"1min":0.48899999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.748,"5min":1.274,"15min":6.2649999999999997},"last":0.61899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303796,"kb_used_data":2412,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068044,"statfs":{"total":96636764160,"available":96325677056,"internally_reserved":0,"allocated":2469888,"data_stored":1108324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.31,"15min":0.313},"min":{"1min":0.222,"5min":0.21299999999999999,"15min":0.21099999999999999},"max":{"1min":0.47699999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.372},{"interface":"front","average":{"1min":0.31,"5min":0.32700000000000001,"15min":0.32000000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.28199999999999997}]},{"osd":1,"last update":"Tue Oct 10 20:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34399999999999997,"15min":0.33500000000000002},"min":{"1min":0.22700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.254},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.39000000000000001,"15min":0.38},"min":{"1min":0.316,"5min":0.26800000000000002,"15min":0.251},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.34300000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.36399999999999999,"15min":0.35299999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.46800000000000003,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.26800000000000002,"5min":0.25900000000000001,"15min":0.255},"max":{"1min":0.51000000000000001,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.29399999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56999999999999995,"15min":0.54100000000000004},"min":{"1min":0.372,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76400000000000001,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57199999999999995,"15min":0.54300000000000004},"min":{"1min":0.46400000000000002,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.70999999999999996,"5min":0.83299999999999996,"15min":0.93500000000000005},"last":0.61099999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.55600000000000005},"min":{"1min":0.45600000000000002,"5min":0.39400000000000002,"15min":0.35899999999999999},"max":{"1min":0.66600000000000004,"5min":1.347,"15min":1.347},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57599999999999996,"15min":0.55200000000000005},"min":{"1min":0.44600000000000001,"5min":0.39300000000000002,"15min":0.35799999999999998},"max":{"1min":0.82799999999999996,"5min":0.86799999999999999,"15min":1.01},"last":0.56699999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61199999999999999,"15min":0.59399999999999997},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.36699999999999999},"max":{"1min":0.78500000000000003,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.627,"15min":0.60899999999999999},"min":{"1min":0.49399999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79900000000000004,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.625}]},{"osd":7,"last update":"Tue Oct 10 20:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60699999999999998,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81899999999999995,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.623,"15min":0.59799999999999998},"min":{"1min":0.441,"5min":0.435,"15min":0.38200000000000001},"max":{"1min":0.78900000000000003,"5min":0.89500000000000002,"15min":0.96999999999999997},"last":0.56100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303796,"kb_used_data":2412,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068044,"statfs":{"total":96636764160,"available":96325677056,"internally_reserved":0,"allocated":2469888,"data_stored":1108324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.50800000000000001,"15min":0.51100000000000001},"min":{"1min":0.38900000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.747},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52600000000000002,"15min":0.69599999999999995},"min":{"1min":0.43099999999999999,"5min":0.34000000000000002,"15min":0.32700000000000001},"max":{"1min":0.73099999999999998,"5min":0.77600000000000002,"15min":7.6719999999999997},"last":0.51100000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.70099999999999996},"min":{"1min":0.34100000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":7.7160000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.32700000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.86799999999999999},"last":0.40699999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.50700000000000001,"15min":0.51200000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.72399999999999998,"5min":0.76000000000000001,"15min":0.78000000000000003},"last":0.439},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56999999999999995,"15min":0.72799999999999998},"min":{"1min":0.437,"5min":0.39700000000000002,"15min":0.379},"max":{"1min":0.80500000000000005,"5min":0.81599999999999995,"15min":7.7839999999999998},"last":0.46400000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73599999999999999,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.40999999999999998,"5min":0.39200000000000002,"15min":0.35499999999999998},"max":{"1min":0.77100000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48299999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53900000000000003,"15min":0.5},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.30099999999999999},"max":{"1min":0.78900000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.54800000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79600000000000004,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.36799999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54400000000000004,"15min":0.53100000000000003},"min":{"1min":0.379,"5min":0.318,"15min":0.318},"max":{"1min":0.69699999999999995,"5min":0.85199999999999998,"15min":0.85899999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60599999999999998,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82499999999999996,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54300000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61199999999999999,"15min":0.58999999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.36199999999999999},"max":{"1min":0.82899999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.59899999999999998},"min":{"1min":0.41999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.86899999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.39300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303796,"kb_used_data":2412,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068044,"statfs":{"total":96636764160,"available":96325677056,"internally_reserved":0,"allocated":2469888,"data_stored":1108324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49399999999999999,"15min":0.48899999999999999},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.52300000000000002},"min":{"1min":0.314,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.39500000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.51100000000000001,"15min":0.60399999999999998},"min":{"1min":0.30099999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.71899999999999997,"5min":0.746,"15min":5.2679999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.52700000000000002,"15min":0.59899999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":5.3109999999999999},"last":0.47099999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56399999999999995,"15min":0.627},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.98999999999999999,"5min":3.27,"15min":5.2960000000000003},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69599999999999995,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.52100000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55800000000000005,"15min":0.64600000000000002},"min":{"1min":0.39200000000000002,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":1.02,"5min":1.02,"15min":5.3330000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56799999999999995,"15min":0.65400000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.27700000000000002},"max":{"1min":0.78700000000000003,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.61799999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.496,"15min":0.496},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53900000000000003,"15min":0.54400000000000004},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32200000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0800000000000001},"last":0.66400000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53700000000000003,"15min":0.54700000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.30599999999999999},"max":{"1min":0.749,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.58199999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.436,"5min":0.33000000000000002,"15min":0.30099999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.752},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.38200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.81599999999999995,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.73599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:44:04.667 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:44:04.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.814+0000 7f4d8ae45700 1 Processor -- start 2023-10-10T20:44:04.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.814+0000 7f4d8ae45700 1 -- start start 2023-10-10T20:44:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.815+0000 7f4d8ae45700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d84069560 0x7f4d840699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.815+0000 7f4d8ae45700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d84069ee0 con 0x7f4d84159c60 2023-10-10T20:44:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.815+0000 7f4d8ae45700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d8415a160 con 0x7f4d84159890 2023-10-10T20:44:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.815+0000 7f4d8ae45700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840d13a0 con 0x7f4d84069560 2023-10-10T20:44:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.815+0000 7f4d7bfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4d84159c60 0x7f4d8415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37234/0 (socket says 172.21.15.89:37234) 2023-10-10T20:44:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.815+0000 7f4d7bfff700 1 -- 172.21.15.89:0/245081218 learned_addr learned my addr 172.21.15.89:0/245081218 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:44:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.815+0000 7f4d893e2700 1 --2- 172.21.15.89:0/245081218 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d84069560 0x7f4d840699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7b7fe700 1 -- 172.21.15.89:0/245081218 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432942144 0 0) 0x7f4d84069ee0 con 0x7f4d84159c60 2023-10-10T20:44:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7b7fe700 1 -- 172.21.15.89:0/245081218 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d6c003570 con 0x7f4d84159c60 2023-10-10T20:44:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7b7fe700 1 -- 172.21.15.89:0/245081218 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1340152244 0 0) 0x7f4d6c003570 con 0x7f4d84159c60 2023-10-10T20:44:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7b7fe700 1 -- 172.21.15.89:0/245081218 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d84069560 msgr2=0x7f4d840699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7b7fe700 1 --2- 172.21.15.89:0/245081218 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d84069560 0x7f4d840699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7b7fe700 1 -- 172.21.15.89:0/245081218 >> v1:172.21.15.111:6789/0 conn(0x7f4d84159890 legacy=0x7f4d8415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7b7fe700 1 -- 172.21.15.89:0/245081218 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8415e140 con 0x7f4d84159c60 2023-10-10T20:44:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7b7fe700 1 -- 172.21.15.89:0/245081218 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f4d74001470 con 0x7f4d84159c60 2023-10-10T20:44:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7b7fe700 1 -- 172.21.15.89:0/245081218 >> v1:172.21.15.89:6789/0 conn(0x7f4d84159c60 legacy=0x7f4d8415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7b7fe700 1 --2- 172.21.15.89:0/245081218 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d84069560 0x7f4d6c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7b7fe700 1 --2- 172.21.15.89:0/245081218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d6c003fd0 0x7f4d6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7b7fe700 1 --2- 172.21.15.89:0/245081218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d6c004d50 0x7f4d6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7b7fe700 1 -- 172.21.15.89:0/245081218 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840d13a0 con 0x7f4d6c003fd0 2023-10-10T20:44:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7b7fe700 1 -- 172.21.15.89:0/245081218 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c00bb40 con 0x7f4d6c004d50 2023-10-10T20:44:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7b7fe700 1 -- 172.21.15.89:0/245081218 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c00bd00 con 0x7f4d84069560 2023-10-10T20:44:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7bfff700 1 --2- 172.21.15.89:0/245081218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d6c003fd0 0x7f4d6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d88be1700 1 --2- 172.21.15.89:0/245081218 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d84069560 0x7f4d6c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d893e2700 1 --2- 172.21.15.89:0/245081218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d6c004d50 0x7f4d6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7bfff700 1 -- 172.21.15.89:0/245081218 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d84069560 msgr2=0x7f4d6c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7bfff700 1 --2- 172.21.15.89:0/245081218 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d84069560 0x7f4d6c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7bfff700 1 -- 172.21.15.89:0/245081218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d6c004d50 msgr2=0x7f4d6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7bfff700 1 --2- 172.21.15.89:0/245081218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d6c004d50 0x7f4d6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7bfff700 1 -- 172.21.15.89:0/245081218 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c003570 con 0x7f4d6c003fd0 2023-10-10T20:44:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d88be1700 1 --2- 172.21.15.89:0/245081218 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d84069560 0x7f4d6c003a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:44:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.816+0000 7f4d7bfff700 1 --2- 172.21.15.89:0/245081218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d6c003fd0 0x7f4d6c004810 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400c3c0 tx=0x7f4d74001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c03663d9675c85 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.817+0000 7f4d7b7fe700 1 -- 172.21.15.89:0/245081218 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7400cc90 con 0x7f4d6c003fd0 2023-10-10T20:44:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.817+0000 7f4d7b7fe700 1 -- 172.21.15.89:0/245081218 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7400cdf0 con 0x7f4d6c003fd0 2023-10-10T20:44:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.817+0000 7f4d7b7fe700 1 -- 172.21.15.89:0/245081218 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d74005b70 con 0x7f4d6c003fd0 2023-10-10T20:44:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.817+0000 7f4d8ae45700 1 -- 172.21.15.89:0/245081218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d6c003fd0 msgr2=0x7f4d6c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.817+0000 7f4d8ae45700 1 --2- 172.21.15.89:0/245081218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d6c003fd0 0x7f4d6c004810 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400c3c0 tx=0x7f4d74001e70 comp rx=0 tx=0).stop 2023-10-10T20:44:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.817+0000 7f4d893e2700 1 -- 172.21.15.89:0/245081218 reap_dead start 2023-10-10T20:44:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.817+0000 7f4d8ae45700 1 -- 172.21.15.89:0/245081218 shutdown_connections 2023-10-10T20:44:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.817+0000 7f4d8ae45700 1 -- 172.21.15.89:0/245081218 >> 172.21.15.89:0/245081218 conn(0x7f4d84068e20 msgr2=0x7f4d840d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.817+0000 7f4d8ae45700 1 -- 172.21.15.89:0/245081218 shutdown_connections 2023-10-10T20:44:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.817+0000 7f4d8ae45700 1 -- 172.21.15.89:0/245081218 wait complete. 2023-10-10T20:44:04.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.818+0000 7f4d8ae45700 1 Processor -- start 2023-10-10T20:44:04.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.818+0000 7f4d8ae45700 1 -- start start 2023-10-10T20:44:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.819+0000 7f4d8ae45700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d6c003fd0 0x7f4d840ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.819+0000 7f4d8ae45700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d840cd470 0x7f4d840b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.819+0000 7f4d8ae45700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d840b3ad0 0x7f4d840b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.819+0000 7f4d8ae45700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840cebb0 con 0x7f4d840b3ad0 2023-10-10T20:44:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.819+0000 7f4d8ae45700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840b06a0 con 0x7f4d840cd470 2023-10-10T20:44:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.819+0000 7f4d8ae45700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840b0980 con 0x7f4d6c003fd0 2023-10-10T20:44:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.819+0000 7f4d893e2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d840b3ad0 0x7f4d840b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.819+0000 7f4d893e2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d840b3ad0 0x7f4d840b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52038/0 (socket says 172.21.15.89:52038) 2023-10-10T20:44:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.819+0000 7f4d893e2700 1 -- 172.21.15.89:0/2485087931 learned_addr learned my addr 172.21.15.89:0/2485087931 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.819+0000 7f4d88be1700 1 --2- 172.21.15.89:0/2485087931 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d6c003fd0 0x7f4d840ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.819+0000 7f4d7bfff700 1 --2- 172.21.15.89:0/2485087931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d840cd470 0x7f4d840b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.819+0000 7f4d893e2700 1 -- 172.21.15.89:0/2485087931 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d6c003fd0 msgr2=0x7f4d840ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.819+0000 7f4d893e2700 1 --2- 172.21.15.89:0/2485087931 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d6c003fd0 0x7f4d840ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.819+0000 7f4d893e2700 1 -- 172.21.15.89:0/2485087931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d840cd470 msgr2=0x7f4d840b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.819+0000 7f4d893e2700 1 --2- 172.21.15.89:0/2485087931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d840cd470 0x7f4d840b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.819+0000 7f4d893e2700 1 -- 172.21.15.89:0/2485087931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d840b0c60 con 0x7f4d840b3ad0 2023-10-10T20:44:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.819+0000 7f4d88be1700 1 --2- 172.21.15.89:0/2485087931 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d6c003fd0 0x7f4d840ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:44:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.820+0000 7f4d893e2700 1 --2- 172.21.15.89:0/2485087931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d840b3ad0 0x7f4d840b1460 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f4d840cadc0 tx=0x7f4d7c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.820+0000 7f4d78ff9700 1 -- 172.21.15.89:0/2485087931 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c00e070 con 0x7f4d840b3ad0 2023-10-10T20:44:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.820+0000 7f4d78ff9700 1 -- 172.21.15.89:0/2485087931 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7c011c30 con 0x7f4d840b3ad0 2023-10-10T20:44:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.820+0000 7f4d8ae45700 1 -- 172.21.15.89:0/2485087931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d840b0f40 con 0x7f4d840b3ad0 2023-10-10T20:44:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.820+0000 7f4d78ff9700 1 -- 172.21.15.89:0/2485087931 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c016470 con 0x7f4d840b3ad0 2023-10-10T20:44:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.820+0000 7f4d8ae45700 1 -- 172.21.15.89:0/2485087931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d840ce810 con 0x7f4d840b3ad0 2023-10-10T20:44:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.821+0000 7f4d8ae45700 1 -- 172.21.15.89:0/2485087931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d840b1130 con 0x7f4d840b3ad0 2023-10-10T20:44:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.821+0000 7f4d78ff9700 1 -- 172.21.15.89:0/2485087931 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4d7c016820 con 0x7f4d840b3ad0 2023-10-10T20:44:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.822+0000 7f4d78ff9700 1 --2- 172.21.15.89:0/2485087931 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4d70067320 0x7f4d700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.822+0000 7f4d78ff9700 1 -- 172.21.15.89:0/2485087931 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4d7c0858b0 con 0x7f4d840b3ad0 2023-10-10T20:44:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.822+0000 7f4d88be1700 1 --2- 172.21.15.89:0/2485087931 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4d70067320 0x7f4d700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:04.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.822+0000 7f4d88be1700 1 --2- 172.21.15.89:0/2485087931 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4d70067320 0x7f4d700697d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4d80007fd0 tx=0x7f4d80009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:04.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:04.826+0000 7f4d78ff9700 1 -- 172.21.15.89:0/2485087931 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d7c052430 con 0x7f4d840b3ad0 2023-10-10T20:44:05.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.005+0000 7f4d8ae45700 1 -- 172.21.15.89:0/2485087931 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4d840b0810 con 0x7f4d70067320 2023-10-10T20:44:05.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.013+0000 7f4d78ff9700 1 -- 172.21.15.89:0/2485087931 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289880 (secure 0 0 0) 0x7f4d840b0810 con 0x7f4d70067320 2023-10-10T20:44:05.018 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:44:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.016+0000 7f4d8ae45700 1 -- 172.21.15.89:0/2485087931 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4d70067320 msgr2=0x7f4d700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.016+0000 7f4d8ae45700 1 --2- 172.21.15.89:0/2485087931 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4d70067320 0x7f4d700697d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4d80007fd0 tx=0x7f4d80009040 comp rx=0 tx=0).stop 2023-10-10T20:44:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.016+0000 7f4d8ae45700 1 -- 172.21.15.89:0/2485087931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d840b3ad0 msgr2=0x7f4d840b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.016+0000 7f4d8ae45700 1 --2- 172.21.15.89:0/2485087931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d840b3ad0 0x7f4d840b1460 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f4d840cadc0 tx=0x7f4d7c00b040 comp rx=0 tx=0).stop 2023-10-10T20:44:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.016+0000 7f4d8ae45700 1 -- 172.21.15.89:0/2485087931 shutdown_connections 2023-10-10T20:44:05.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.016+0000 7f4d8ae45700 1 --2- 172.21.15.89:0/2485087931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d840b3ad0 0x7f4d840b1460 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.016+0000 7f4d8ae45700 1 --2- 172.21.15.89:0/2485087931 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4d6c003fd0 0x7f4d840ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.016+0000 7f4d8ae45700 1 --2- 172.21.15.89:0/2485087931 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4d70067320 0x7f4d700697d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.016+0000 7f4d8ae45700 1 --2- 172.21.15.89:0/2485087931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d840cd470 0x7f4d840b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.016+0000 7f4d8ae45700 1 -- 172.21.15.89:0/2485087931 >> 172.21.15.89:0/2485087931 conn(0x7f4d84068e20 msgr2=0x7f4d840c6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.016+0000 7f4d8ae45700 1 -- 172.21.15.89:0/2485087931 shutdown_connections 2023-10-10T20:44:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.016+0000 7f4d8ae45700 1 -- 172.21.15.89:0/2485087931 wait complete. 2023-10-10T20:44:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:44:05.029 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2023-10-10T20:44:04.763767+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2431972,"kb_used_data":20900,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752542748,"statfs":{"total":773094113280,"available":770603773952,"internally_reserved":0,"allocated":21401600,"data_stored":10251931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000906"},"pg_stats":[{"pgid":"34.3f","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.602830+0000","last_change":"2023-10-10T20:44:00.601261+0000","last_active":"2023-10-10T20:44:00.602830+0000","last_peered":"2023-10-10T20:44:00.602830+0000","last_clean":"2023-10-10T20:44:00.602830+0000","last_became_active":"2023-10-10T20:44:00.600938+0000","last_became_peered":"2023-10-10T20:44:00.600938+0000","last_unstale":"2023-10-10T20:44:00.602830+0000","last_undegraded":"2023-10-10T20:44:00.602830+0000","last_fullsized":"2023-10-10T20:44:00.602830+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:23:10.604819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.598661+0000","last_change":"2023-10-10T20:44:00.597121+0000","last_active":"2023-10-10T20:44:00.598661+0000","last_peered":"2023-10-10T20:44:00.598661+0000","last_clean":"2023-10-10T20:44:00.598661+0000","last_became_active":"2023-10-10T20:44:00.596542+0000","last_became_peered":"2023-10-10T20:44:00.596542+0000","last_unstale":"2023-10-10T20:44:00.598661+0000","last_undegraded":"2023-10-10T20:44:00.598661+0000","last_fullsized":"2023-10-10T20:44:00.598661+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:27.123895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.593963+0000","last_change":"2023-10-10T20:44:00.590240+0000","last_active":"2023-10-10T20:44:00.593963+0000","last_peered":"2023-10-10T20:44:00.593963+0000","last_clean":"2023-10-10T20:44:00.593963+0000","last_became_active":"2023-10-10T20:44:00.589968+0000","last_became_peered":"2023-10-10T20:44:00.589968+0000","last_unstale":"2023-10-10T20:44:00.593963+0000","last_undegraded":"2023-10-10T20:44:00.593963+0000","last_fullsized":"2023-10-10T20:44:00.593963+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:15:34.555993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.594348+0000","last_change":"2023-10-10T20:44:00.591134+0000","last_active":"2023-10-10T20:44:00.594348+0000","last_peered":"2023-10-10T20:44:00.594348+0000","last_clean":"2023-10-10T20:44:00.594348+0000","last_became_active":"2023-10-10T20:44:00.590869+0000","last_became_peered":"2023-10-10T20:44:00.590869+0000","last_unstale":"2023-10-10T20:44:00.594348+0000","last_undegraded":"2023-10-10T20:44:00.594348+0000","last_fullsized":"2023-10-10T20:44:00.594348+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:01:34.109015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.591501+0000","last_change":"2023-10-10T20:44:00.588771+0000","last_active":"2023-10-10T20:44:00.591501+0000","last_peered":"2023-10-10T20:44:00.591501+0000","last_clean":"2023-10-10T20:44:00.591501+0000","last_became_active":"2023-10-10T20:44:00.588544+0000","last_became_peered":"2023-10-10T20:44:00.588544+0000","last_unstale":"2023-10-10T20:44:00.591501+0000","last_undegraded":"2023-10-10T20:44:00.591501+0000","last_fullsized":"2023-10-10T20:44:00.591501+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:11:28.189123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.590172+0000","last_change":"2023-10-10T20:44:00.587175+0000","last_active":"2023-10-10T20:44:00.590172+0000","last_peered":"2023-10-10T20:44:00.590172+0000","last_clean":"2023-10-10T20:44:00.590172+0000","last_became_active":"2023-10-10T20:44:00.586905+0000","last_became_peered":"2023-10-10T20:44:00.586905+0000","last_unstale":"2023-10-10T20:44:00.590172+0000","last_undegraded":"2023-10-10T20:44:00.590172+0000","last_fullsized":"2023-10-10T20:44:00.590172+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:10:34.135073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.597559+0000","last_change":"2023-10-10T20:44:00.592729+0000","last_active":"2023-10-10T20:44:00.597559+0000","last_peered":"2023-10-10T20:44:00.597559+0000","last_clean":"2023-10-10T20:44:00.597559+0000","last_became_active":"2023-10-10T20:44:00.592512+0000","last_became_peered":"2023-10-10T20:44:00.592512+0000","last_unstale":"2023-10-10T20:44:00.597559+0000","last_undegraded":"2023-10-10T20:44:00.597559+0000","last_fullsized":"2023-10-10T20:44:00.597559+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:56:49.745936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.598103+0000","last_change":"2023-10-10T20:44:00.596168+0000","last_active":"2023-10-10T20:44:00.598103+0000","last_peered":"2023-10-10T20:44:00.598103+0000","last_clean":"2023-10-10T20:44:00.598103+0000","last_became_active":"2023-10-10T20:44:00.595939+0000","last_became_peered":"2023-10-10T20:44:00.595939+0000","last_unstale":"2023-10-10T20:44:00.598103+0000","last_undegraded":"2023-10-10T20:44:00.598103+0000","last_fullsized":"2023-10-10T20:44:00.598103+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:39:43.337282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.590543+0000","last_change":"2023-10-10T20:44:00.586775+0000","last_active":"2023-10-10T20:44:00.590543+0000","last_peered":"2023-10-10T20:44:00.590543+0000","last_clean":"2023-10-10T20:44:00.590543+0000","last_became_active":"2023-10-10T20:44:00.586360+0000","last_became_peered":"2023-10-10T20:44:00.586360+0000","last_unstale":"2023-10-10T20:44:00.590543+0000","last_undegraded":"2023-10-10T20:44:00.590543+0000","last_fullsized":"2023-10-10T20:44:00.590543+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:27:59.904742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.600646+0000","last_change":"2023-10-10T20:44:00.598315+0000","last_active":"2023-10-10T20:44:00.600646+0000","last_peered":"2023-10-10T20:44:00.600646+0000","last_clean":"2023-10-10T20:44:00.600646+0000","last_became_active":"2023-10-10T20:44:00.597879+0000","last_became_peered":"2023-10-10T20:44:00.597879+0000","last_unstale":"2023-10-10T20:44:00.600646+0000","last_undegraded":"2023-10-10T20:44:00.600646+0000","last_fullsized":"2023-10-10T20:44:00.600646+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:35.503994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.599018+0000","last_change":"2023-10-10T20:44:00.594985+0000","last_active":"2023-10-10T20:44:00.599018+0000","last_peered":"2023-10-10T20:44:00.599018+0000","last_clean":"2023-10-10T20:44:00.599018+0000","last_became_active":"2023-10-10T20:44:00.594687+0000","last_became_peered":"2023-10-10T20:44:00.594687+0000","last_unstale":"2023-10-10T20:44:00.599018+0000","last_undegraded":"2023-10-10T20:44:00.599018+0000","last_fullsized":"2023-10-10T20:44:00.599018+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:46:36.317779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.600187+0000","last_change":"2023-10-10T20:44:00.596665+0000","last_active":"2023-10-10T20:44:00.600187+0000","last_peered":"2023-10-10T20:44:00.600187+0000","last_clean":"2023-10-10T20:44:00.600187+0000","last_became_active":"2023-10-10T20:44:00.596407+0000","last_became_peered":"2023-10-10T20:44:00.596407+0000","last_unstale":"2023-10-10T20:44:00.600187+0000","last_undegraded":"2023-10-10T20:44:00.600187+0000","last_fullsized":"2023-10-10T20:44:00.600187+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:12:31.521062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.598383+0000","last_change":"2023-10-10T20:44:00.596934+0000","last_active":"2023-10-10T20:44:00.598383+0000","last_peered":"2023-10-10T20:44:00.598383+0000","last_clean":"2023-10-10T20:44:00.598383+0000","last_became_active":"2023-10-10T20:44:00.596709+0000","last_became_peered":"2023-10-10T20:44:00.596709+0000","last_unstale":"2023-10-10T20:44:00.598383+0000","last_undegraded":"2023-10-10T20:44:00.598383+0000","last_fullsized":"2023-10-10T20:44:00.598383+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:04:12.799367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.601875+0000","last_change":"2023-10-10T20:44:00.599008+0000","last_active":"2023-10-10T20:44:00.601875+0000","last_peered":"2023-10-10T20:44:00.601875+0000","last_clean":"2023-10-10T20:44:00.601875+0000","last_became_active":"2023-10-10T20:44:00.598770+0000","last_became_peered":"2023-10-10T20:44:00.598770+0000","last_unstale":"2023-10-10T20:44:00.601875+0000","last_undegraded":"2023-10-10T20:44:00.601875+0000","last_fullsized":"2023-10-10T20:44:00.601875+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:31:07.001598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.606440+0000","last_change":"2023-10-10T20:44:00.603617+0000","last_active":"2023-10-10T20:44:00.606440+0000","last_peered":"2023-10-10T20:44:00.606440+0000","last_clean":"2023-10-10T20:44:00.606440+0000","last_became_active":"2023-10-10T20:44:00.602928+0000","last_became_peered":"2023-10-10T20:44:00.602928+0000","last_unstale":"2023-10-10T20:44:00.606440+0000","last_undegraded":"2023-10-10T20:44:00.606440+0000","last_fullsized":"2023-10-10T20:44:00.606440+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:22.184555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.597583+0000","last_change":"2023-10-10T20:44:00.596014+0000","last_active":"2023-10-10T20:44:00.597583+0000","last_peered":"2023-10-10T20:44:00.597583+0000","last_clean":"2023-10-10T20:44:00.597583+0000","last_became_active":"2023-10-10T20:44:00.595481+0000","last_became_peered":"2023-10-10T20:44:00.595481+0000","last_unstale":"2023-10-10T20:44:00.597583+0000","last_undegraded":"2023-10-10T20:44:00.597583+0000","last_fullsized":"2023-10-10T20:44:00.597583+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:04:30.983631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.600311+0000","last_change":"2023-10-10T20:44:00.597348+0000","last_active":"2023-10-10T20:44:00.600311+0000","last_peered":"2023-10-10T20:44:00.600311+0000","last_clean":"2023-10-10T20:44:00.600311+0000","last_became_active":"2023-10-10T20:44:00.597140+0000","last_became_peered":"2023-10-10T20:44:00.597140+0000","last_unstale":"2023-10-10T20:44:00.600311+0000","last_undegraded":"2023-10-10T20:44:00.600311+0000","last_fullsized":"2023-10-10T20:44:00.600311+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:39:27.927297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.589036+0000","last_change":"2023-10-10T20:44:00.587434+0000","last_active":"2023-10-10T20:44:00.589036+0000","last_peered":"2023-10-10T20:44:00.589036+0000","last_clean":"2023-10-10T20:44:00.589036+0000","last_became_active":"2023-10-10T20:44:00.587150+0000","last_became_peered":"2023-10-10T20:44:00.587150+0000","last_unstale":"2023-10-10T20:44:00.589036+0000","last_undegraded":"2023-10-10T20:44:00.589036+0000","last_fullsized":"2023-10-10T20:44:00.589036+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:26:13.233277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.600783+0000","last_change":"2023-10-10T20:44:00.597412+0000","last_active":"2023-10-10T20:44:00.600783+0000","last_peered":"2023-10-10T20:44:00.600783+0000","last_clean":"2023-10-10T20:44:00.600783+0000","last_became_active":"2023-10-10T20:44:00.597007+0000","last_became_peered":"2023-10-10T20:44:00.597007+0000","last_unstale":"2023-10-10T20:44:00.600783+0000","last_undegraded":"2023-10-10T20:44:00.600783+0000","last_fullsized":"2023-10-10T20:44:00.600783+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:10:34.135073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.592747+0000","last_change":"2023-10-10T20:44:00.587507+0000","last_active":"2023-10-10T20:44:00.592747+0000","last_peered":"2023-10-10T20:44:00.592747+0000","last_clean":"2023-10-10T20:44:00.592747+0000","last_became_active":"2023-10-10T20:44:00.587259+0000","last_became_peered":"2023-10-10T20:44:00.587259+0000","last_unstale":"2023-10-10T20:44:00.592747+0000","last_undegraded":"2023-10-10T20:44:00.592747+0000","last_fullsized":"2023-10-10T20:44:00.592747+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:59:29.901357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.591519+0000","last_change":"2023-10-10T20:44:00.588771+0000","last_active":"2023-10-10T20:44:00.591519+0000","last_peered":"2023-10-10T20:44:00.591519+0000","last_clean":"2023-10-10T20:44:00.591519+0000","last_became_active":"2023-10-10T20:44:00.588507+0000","last_became_peered":"2023-10-10T20:44:00.588507+0000","last_unstale":"2023-10-10T20:44:00.591519+0000","last_undegraded":"2023-10-10T20:44:00.591519+0000","last_fullsized":"2023-10-10T20:44:00.591519+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:24:03.119742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.602677+0000","last_change":"2023-10-10T20:44:00.600703+0000","last_active":"2023-10-10T20:44:00.602677+0000","last_peered":"2023-10-10T20:44:00.602677+0000","last_clean":"2023-10-10T20:44:00.602677+0000","last_became_active":"2023-10-10T20:44:00.600422+0000","last_became_peered":"2023-10-10T20:44:00.600422+0000","last_unstale":"2023-10-10T20:44:00.602677+0000","last_undegraded":"2023-10-10T20:44:00.602677+0000","last_fullsized":"2023-10-10T20:44:00.602677+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:46:47.296086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.601568+0000","last_change":"2023-10-10T20:44:00.597758+0000","last_active":"2023-10-10T20:44:00.601568+0000","last_peered":"2023-10-10T20:44:00.601568+0000","last_clean":"2023-10-10T20:44:00.601568+0000","last_became_active":"2023-10-10T20:44:00.597516+0000","last_became_peered":"2023-10-10T20:44:00.597516+0000","last_unstale":"2023-10-10T20:44:00.601568+0000","last_undegraded":"2023-10-10T20:44:00.601568+0000","last_fullsized":"2023-10-10T20:44:00.601568+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:08:01.523217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.602316+0000","last_change":"2023-10-10T20:44:00.598552+0000","last_active":"2023-10-10T20:44:00.602316+0000","last_peered":"2023-10-10T20:44:00.602316+0000","last_clean":"2023-10-10T20:44:00.602316+0000","last_became_active":"2023-10-10T20:44:00.598273+0000","last_became_peered":"2023-10-10T20:44:00.598273+0000","last_unstale":"2023-10-10T20:44:00.602316+0000","last_undegraded":"2023-10-10T20:44:00.602316+0000","last_fullsized":"2023-10-10T20:44:00.602316+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:43:26.351692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.594312+0000","last_change":"2023-10-10T20:44:00.589533+0000","last_active":"2023-10-10T20:44:00.594312+0000","last_peered":"2023-10-10T20:44:00.594312+0000","last_clean":"2023-10-10T20:44:00.594312+0000","last_became_active":"2023-10-10T20:44:00.589285+0000","last_became_peered":"2023-10-10T20:44:00.589285+0000","last_unstale":"2023-10-10T20:44:00.594312+0000","last_undegraded":"2023-10-10T20:44:00.594312+0000","last_fullsized":"2023-10-10T20:44:00.594312+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:36:50.272889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.598679+0000","last_change":"2023-10-10T20:44:00.597299+0000","last_active":"2023-10-10T20:44:00.598679+0000","last_peered":"2023-10-10T20:44:00.598679+0000","last_clean":"2023-10-10T20:44:00.598679+0000","last_became_active":"2023-10-10T20:44:00.596857+0000","last_became_peered":"2023-10-10T20:44:00.596857+0000","last_unstale":"2023-10-10T20:44:00.598679+0000","last_undegraded":"2023-10-10T20:44:00.598679+0000","last_fullsized":"2023-10-10T20:44:00.598679+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:04:11.872588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.602815+0000","last_change":"2023-10-10T20:44:00.600732+0000","last_active":"2023-10-10T20:44:00.602815+0000","last_peered":"2023-10-10T20:44:00.602815+0000","last_clean":"2023-10-10T20:44:00.602815+0000","last_became_active":"2023-10-10T20:44:00.600362+0000","last_became_peered":"2023-10-10T20:44:00.600362+0000","last_unstale":"2023-10-10T20:44:00.602815+0000","last_undegraded":"2023-10-10T20:44:00.602815+0000","last_fullsized":"2023-10-10T20:44:00.602815+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:27:48.167598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.602825+0000","last_change":"2023-10-10T20:44:00.601466+0000","last_active":"2023-10-10T20:44:00.602825+0000","last_peered":"2023-10-10T20:44:00.602825+0000","last_clean":"2023-10-10T20:44:00.602825+0000","last_became_active":"2023-10-10T20:44:00.601277+0000","last_became_peered":"2023-10-10T20:44:00.601277+0000","last_unstale":"2023-10-10T20:44:00.602825+0000","last_undegraded":"2023-10-10T20:44:00.602825+0000","last_fullsized":"2023-10-10T20:44:00.602825+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:40:19.705203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.590189+0000","last_change":"2023-10-10T20:44:00.588682+0000","last_active":"2023-10-10T20:44:00.590189+0000","last_peered":"2023-10-10T20:44:00.590189+0000","last_clean":"2023-10-10T20:44:00.590189+0000","last_became_active":"2023-10-10T20:44:00.588391+0000","last_became_peered":"2023-10-10T20:44:00.588391+0000","last_unstale":"2023-10-10T20:44:00.590189+0000","last_undegraded":"2023-10-10T20:44:00.590189+0000","last_fullsized":"2023-10-10T20:44:00.590189+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:39:27.927297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.601933+0000","last_change":"2023-10-10T20:44:00.599655+0000","last_active":"2023-10-10T20:44:00.601933+0000","last_peered":"2023-10-10T20:44:00.601933+0000","last_clean":"2023-10-10T20:44:00.601933+0000","last_became_active":"2023-10-10T20:44:00.599259+0000","last_became_peered":"2023-10-10T20:44:00.599259+0000","last_unstale":"2023-10-10T20:44:00.601933+0000","last_undegraded":"2023-10-10T20:44:00.601933+0000","last_fullsized":"2023-10-10T20:44:00.601933+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:02:26.502686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.595781+0000","last_change":"2023-10-10T20:44:00.592891+0000","last_active":"2023-10-10T20:44:00.595781+0000","last_peered":"2023-10-10T20:44:00.595781+0000","last_clean":"2023-10-10T20:44:00.595781+0000","last_became_active":"2023-10-10T20:44:00.592342+0000","last_became_peered":"2023-10-10T20:44:00.592342+0000","last_unstale":"2023-10-10T20:44:00.595781+0000","last_undegraded":"2023-10-10T20:44:00.595781+0000","last_fullsized":"2023-10-10T20:44:00.595781+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:38:24.069232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.602333+0000","last_change":"2023-10-10T20:44:00.599528+0000","last_active":"2023-10-10T20:44:00.602333+0000","last_peered":"2023-10-10T20:44:00.602333+0000","last_clean":"2023-10-10T20:44:00.602333+0000","last_became_active":"2023-10-10T20:44:00.599253+0000","last_became_peered":"2023-10-10T20:44:00.599253+0000","last_unstale":"2023-10-10T20:44:00.602333+0000","last_undegraded":"2023-10-10T20:44:00.602333+0000","last_fullsized":"2023-10-10T20:44:00.602333+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:35.275128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"197'4","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.609661+0000","last_change":"2023-10-10T20:43:59.580846+0000","last_active":"2023-10-10T20:44:00.609661+0000","last_peered":"2023-10-10T20:44:00.609661+0000","last_clean":"2023-10-10T20:44:00.609661+0000","last_became_active":"2023-10-10T20:43:59.580615+0000","last_became_peered":"2023-10-10T20:43:59.580615+0000","last_unstale":"2023-10-10T20:44:00.609661+0000","last_undegraded":"2023-10-10T20:44:00.609661+0000","last_fullsized":"2023-10-10T20:44:00.609661+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:39:33.491598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1d","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.601072+0000","last_change":"2023-10-10T20:44:00.597969+0000","last_active":"2023-10-10T20:44:00.601072+0000","last_peered":"2023-10-10T20:44:00.601072+0000","last_clean":"2023-10-10T20:44:00.601072+0000","last_became_active":"2023-10-10T20:44:00.596167+0000","last_became_peered":"2023-10-10T20:44:00.596167+0000","last_unstale":"2023-10-10T20:44:00.601072+0000","last_undegraded":"2023-10-10T20:44:00.601072+0000","last_fullsized":"2023-10-10T20:44:00.601072+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:59:03.822140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"197'1","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.606035+0000","last_change":"2023-10-10T20:43:59.576040+0000","last_active":"2023-10-10T20:44:00.606035+0000","last_peered":"2023-10-10T20:44:00.606035+0000","last_clean":"2023-10-10T20:44:00.606035+0000","last_became_active":"2023-10-10T20:43:59.575791+0000","last_became_peered":"2023-10-10T20:43:59.575791+0000","last_unstale":"2023-10-10T20:44:00.606035+0000","last_undegraded":"2023-10-10T20:44:00.606035+0000","last_fullsized":"2023-10-10T20:44:00.606035+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:55:26.663650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.594535+0000","last_change":"2023-10-10T20:44:00.588951+0000","last_active":"2023-10-10T20:44:00.594535+0000","last_peered":"2023-10-10T20:44:00.594535+0000","last_clean":"2023-10-10T20:44:00.594535+0000","last_became_active":"2023-10-10T20:44:00.588701+0000","last_became_peered":"2023-10-10T20:44:00.588701+0000","last_unstale":"2023-10-10T20:44:00.594535+0000","last_undegraded":"2023-10-10T20:44:00.594535+0000","last_fullsized":"2023-10-10T20:44:00.594535+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:17:08.712538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"197'2","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.608147+0000","last_change":"2023-10-10T20:43:59.584047+0000","last_active":"2023-10-10T20:44:00.608147+0000","last_peered":"2023-10-10T20:44:00.608147+0000","last_clean":"2023-10-10T20:44:00.608147+0000","last_became_active":"2023-10-10T20:43:59.583741+0000","last_became_peered":"2023-10-10T20:43:59.583741+0000","last_unstale":"2023-10-10T20:44:00.608147+0000","last_undegraded":"2023-10-10T20:44:00.608147+0000","last_fullsized":"2023-10-10T20:44:00.608147+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:23:05.794933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.598664+0000","last_change":"2023-10-10T20:44:00.595010+0000","last_active":"2023-10-10T20:44:00.598664+0000","last_peered":"2023-10-10T20:44:00.598664+0000","last_clean":"2023-10-10T20:44:00.598664+0000","last_became_active":"2023-10-10T20:44:00.594760+0000","last_became_peered":"2023-10-10T20:44:00.594760+0000","last_unstale":"2023-10-10T20:44:00.598664+0000","last_undegraded":"2023-10-10T20:44:00.598664+0000","last_fullsized":"2023-10-10T20:44:00.598664+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:43:30.420710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"197'2","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.606892+0000","last_change":"2023-10-10T20:43:59.574794+0000","last_active":"2023-10-10T20:44:00.606892+0000","last_peered":"2023-10-10T20:44:00.606892+0000","last_clean":"2023-10-10T20:44:00.606892+0000","last_became_active":"2023-10-10T20:43:59.574518+0000","last_became_peered":"2023-10-10T20:43:59.574518+0000","last_unstale":"2023-10-10T20:44:00.606892+0000","last_undegraded":"2023-10-10T20:44:00.606892+0000","last_fullsized":"2023-10-10T20:44:00.606892+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:01.854975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.595762+0000","last_change":"2023-10-10T20:44:00.592439+0000","last_active":"2023-10-10T20:44:00.595762+0000","last_peered":"2023-10-10T20:44:00.595762+0000","last_clean":"2023-10-10T20:44:00.595762+0000","last_became_active":"2023-10-10T20:44:00.592135+0000","last_became_peered":"2023-10-10T20:44:00.592135+0000","last_unstale":"2023-10-10T20:44:00.595762+0000","last_undegraded":"2023-10-10T20:44:00.595762+0000","last_fullsized":"2023-10-10T20:44:00.595762+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:33:47.119215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"197'1","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.614653+0000","last_change":"2023-10-10T20:43:59.581700+0000","last_active":"2023-10-10T20:44:00.614653+0000","last_peered":"2023-10-10T20:44:00.614653+0000","last_clean":"2023-10-10T20:44:00.614653+0000","last_became_active":"2023-10-10T20:43:59.581304+0000","last_became_peered":"2023-10-10T20:43:59.581304+0000","last_unstale":"2023-10-10T20:44:00.614653+0000","last_undegraded":"2023-10-10T20:44:00.614653+0000","last_fullsized":"2023-10-10T20:44:00.614653+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:27:58.902934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.606276+0000","last_change":"2023-10-10T20:44:00.601480+0000","last_active":"2023-10-10T20:44:00.606276+0000","last_peered":"2023-10-10T20:44:00.606276+0000","last_clean":"2023-10-10T20:44:00.606276+0000","last_became_active":"2023-10-10T20:44:00.601089+0000","last_became_peered":"2023-10-10T20:44:00.601089+0000","last_unstale":"2023-10-10T20:44:00.606276+0000","last_undegraded":"2023-10-10T20:44:00.606276+0000","last_fullsized":"2023-10-10T20:44:00.606276+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:09:52.108157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"197'1","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.614722+0000","last_change":"2023-10-10T20:43:59.583182+0000","last_active":"2023-10-10T20:44:00.614722+0000","last_peered":"2023-10-10T20:44:00.614722+0000","last_clean":"2023-10-10T20:44:00.614722+0000","last_became_active":"2023-10-10T20:43:59.582946+0000","last_became_peered":"2023-10-10T20:43:59.582946+0000","last_unstale":"2023-10-10T20:44:00.614722+0000","last_undegraded":"2023-10-10T20:44:00.614722+0000","last_fullsized":"2023-10-10T20:44:00.614722+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:39:26.925490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.596925+0000","last_change":"2023-10-10T20:44:00.592097+0000","last_active":"2023-10-10T20:44:00.596925+0000","last_peered":"2023-10-10T20:44:00.596925+0000","last_clean":"2023-10-10T20:44:00.596925+0000","last_became_active":"2023-10-10T20:44:00.591823+0000","last_became_peered":"2023-10-10T20:44:00.591823+0000","last_unstale":"2023-10-10T20:44:00.596925+0000","last_undegraded":"2023-10-10T20:44:00.596925+0000","last_fullsized":"2023-10-10T20:44:00.596925+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:40:52.913499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":10,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.587286+0000","last_change":"2023-10-10T20:43:59.580816+0000","last_active":"2023-10-10T20:44:00.587286+0000","last_peered":"2023-10-10T20:44:00.587286+0000","last_clean":"2023-10-10T20:44:00.587286+0000","last_became_active":"2023-10-10T20:43:59.580545+0000","last_became_peered":"2023-10-10T20:43:59.580545+0000","last_unstale":"2023-10-10T20:44:00.587286+0000","last_undegraded":"2023-10-10T20:44:00.587286+0000","last_fullsized":"2023-10-10T20:44:00.587286+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:06:41.906367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.597241+0000","last_change":"2023-10-10T20:44:00.594812+0000","last_active":"2023-10-10T20:44:00.597241+0000","last_peered":"2023-10-10T20:44:00.597241+0000","last_clean":"2023-10-10T20:44:00.597241+0000","last_became_active":"2023-10-10T20:44:00.594243+0000","last_became_peered":"2023-10-10T20:44:00.594243+0000","last_unstale":"2023-10-10T20:44:00.597241+0000","last_undegraded":"2023-10-10T20:44:00.597241+0000","last_fullsized":"2023-10-10T20:44:00.597241+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:27:49.094376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":10,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.584902+0000","last_change":"2023-10-10T20:43:59.575726+0000","last_active":"2023-10-10T20:44:00.584902+0000","last_peered":"2023-10-10T20:44:00.584902+0000","last_clean":"2023-10-10T20:44:00.584902+0000","last_became_active":"2023-10-10T20:43:59.575495+0000","last_became_peered":"2023-10-10T20:43:59.575495+0000","last_unstale":"2023-10-10T20:44:00.584902+0000","last_undegraded":"2023-10-10T20:44:00.584902+0000","last_fullsized":"2023-10-10T20:44:00.584902+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:21:29.170077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.596546+0000","last_change":"2023-10-10T20:44:00.593384+0000","last_active":"2023-10-10T20:44:00.596546+0000","last_peered":"2023-10-10T20:44:00.596546+0000","last_clean":"2023-10-10T20:44:00.596546+0000","last_became_active":"2023-10-10T20:44:00.593161+0000","last_became_peered":"2023-10-10T20:44:00.593161+0000","last_unstale":"2023-10-10T20:44:00.596546+0000","last_undegraded":"2023-10-10T20:44:00.596546+0000","last_fullsized":"2023-10-10T20:44:00.596546+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:26:01.638540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":10,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.584094+0000","last_change":"2023-10-10T20:43:59.575874+0000","last_active":"2023-10-10T20:44:00.584094+0000","last_peered":"2023-10-10T20:44:00.584094+0000","last_clean":"2023-10-10T20:44:00.584094+0000","last_became_active":"2023-10-10T20:43:59.575648+0000","last_became_peered":"2023-10-10T20:43:59.575648+0000","last_unstale":"2023-10-10T20:44:00.584094+0000","last_undegraded":"2023-10-10T20:44:00.584094+0000","last_fullsized":"2023-10-10T20:44:00.584094+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:56:15.505531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.601053+0000","last_change":"2023-10-10T20:44:00.596861+0000","last_active":"2023-10-10T20:44:00.601053+0000","last_peered":"2023-10-10T20:44:00.601053+0000","last_clean":"2023-10-10T20:44:00.601053+0000","last_became_active":"2023-10-10T20:44:00.596610+0000","last_became_peered":"2023-10-10T20:44:00.596610+0000","last_unstale":"2023-10-10T20:44:00.601053+0000","last_undegraded":"2023-10-10T20:44:00.601053+0000","last_fullsized":"2023-10-10T20:44:00.601053+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:24:47.205725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":10,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.590218+0000","last_change":"2023-10-10T20:43:59.581802+0000","last_active":"2023-10-10T20:44:00.590218+0000","last_peered":"2023-10-10T20:44:00.590218+0000","last_clean":"2023-10-10T20:44:00.590218+0000","last_became_active":"2023-10-10T20:43:59.581530+0000","last_became_peered":"2023-10-10T20:43:59.581530+0000","last_unstale":"2023-10-10T20:44:00.590218+0000","last_undegraded":"2023-10-10T20:44:00.590218+0000","last_fullsized":"2023-10-10T20:44:00.590218+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:53:13.809507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.590213+0000","last_change":"2023-10-10T20:44:00.588548+0000","last_active":"2023-10-10T20:44:00.590213+0000","last_peered":"2023-10-10T20:44:00.590213+0000","last_clean":"2023-10-10T20:44:00.590213+0000","last_became_active":"2023-10-10T20:44:00.588262+0000","last_became_peered":"2023-10-10T20:44:00.588262+0000","last_unstale":"2023-10-10T20:44:00.590213+0000","last_undegraded":"2023-10-10T20:44:00.590213+0000","last_fullsized":"2023-10-10T20:44:00.590213+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:35.275128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"197'3","reported_seq":13,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.614567+0000","last_change":"2023-10-10T20:43:59.574712+0000","last_active":"2023-10-10T20:44:00.614567+0000","last_peered":"2023-10-10T20:44:00.614567+0000","last_clean":"2023-10-10T20:44:00.614567+0000","last_became_active":"2023-10-10T20:43:59.574414+0000","last_became_peered":"2023-10-10T20:43:59.574414+0000","last_unstale":"2023-10-10T20:44:00.614567+0000","last_undegraded":"2023-10-10T20:44:00.614567+0000","last_fullsized":"2023-10-10T20:44:00.614567+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:02:25.500879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.597738+0000","last_change":"2023-10-10T20:44:00.596020+0000","last_active":"2023-10-10T20:44:00.597738+0000","last_peered":"2023-10-10T20:44:00.597738+0000","last_clean":"2023-10-10T20:44:00.597738+0000","last_became_active":"2023-10-10T20:44:00.595658+0000","last_became_peered":"2023-10-10T20:44:00.595658+0000","last_unstale":"2023-10-10T20:44:00.597738+0000","last_undegraded":"2023-10-10T20:44:00.597738+0000","last_fullsized":"2023-10-10T20:44:00.597738+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:08:54.553556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"197'6","reported_seq":16,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.616174+0000","last_change":"2023-10-10T20:43:59.576319+0000","last_active":"2023-10-10T20:44:00.616174+0000","last_peered":"2023-10-10T20:44:00.616174+0000","last_clean":"2023-10-10T20:44:00.616174+0000","last_became_active":"2023-10-10T20:43:59.576046+0000","last_became_peered":"2023-10-10T20:43:59.576046+0000","last_unstale":"2023-10-10T20:44:00.616174+0000","last_undegraded":"2023-10-10T20:44:00.616174+0000","last_fullsized":"2023-10-10T20:44:00.616174+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:41:28.153357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.598277+0000","last_change":"2023-10-10T20:44:00.595116+0000","last_active":"2023-10-10T20:44:00.598277+0000","last_peered":"2023-10-10T20:44:00.598277+0000","last_clean":"2023-10-10T20:44:00.598277+0000","last_became_active":"2023-10-10T20:44:00.594849+0000","last_became_peered":"2023-10-10T20:44:00.594849+0000","last_unstale":"2023-10-10T20:44:00.598277+0000","last_undegraded":"2023-10-10T20:44:00.598277+0000","last_fullsized":"2023-10-10T20:44:00.598277+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:19:32.030463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"197'2","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.614758+0000","last_change":"2023-10-10T20:43:59.576697+0000","last_active":"2023-10-10T20:44:00.614758+0000","last_peered":"2023-10-10T20:44:00.614758+0000","last_clean":"2023-10-10T20:44:00.614758+0000","last_became_active":"2023-10-10T20:43:59.576428+0000","last_became_peered":"2023-10-10T20:43:59.576428+0000","last_unstale":"2023-10-10T20:44:00.614758+0000","last_undegraded":"2023-10-10T20:44:00.614758+0000","last_fullsized":"2023-10-10T20:44:00.614758+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:12:30.519255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.605986+0000","last_change":"2023-10-10T20:44:00.600624+0000","last_active":"2023-10-10T20:44:00.605986+0000","last_peered":"2023-10-10T20:44:00.605986+0000","last_clean":"2023-10-10T20:44:00.605986+0000","last_became_active":"2023-10-10T20:44:00.600303+0000","last_became_peered":"2023-10-10T20:44:00.600303+0000","last_unstale":"2023-10-10T20:44:00.605986+0000","last_undegraded":"2023-10-10T20:44:00.605986+0000","last_fullsized":"2023-10-10T20:44:00.605986+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:38:38.501063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":10,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.584857+0000","last_change":"2023-10-10T20:43:59.578823+0000","last_active":"2023-10-10T20:44:00.584857+0000","last_peered":"2023-10-10T20:44:00.584857+0000","last_clean":"2023-10-10T20:44:00.584857+0000","last_became_active":"2023-10-10T20:43:59.578452+0000","last_became_peered":"2023-10-10T20:43:59.578452+0000","last_unstale":"2023-10-10T20:44:00.584857+0000","last_undegraded":"2023-10-10T20:44:00.584857+0000","last_fullsized":"2023-10-10T20:44:00.584857+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:45:17.173224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.595365+0000","last_change":"2023-10-10T20:44:00.592805+0000","last_active":"2023-10-10T20:44:00.595365+0000","last_peered":"2023-10-10T20:44:00.595365+0000","last_clean":"2023-10-10T20:44:00.595365+0000","last_became_active":"2023-10-10T20:44:00.592564+0000","last_became_peered":"2023-10-10T20:44:00.592564+0000","last_unstale":"2023-10-10T20:44:00.595365+0000","last_undegraded":"2023-10-10T20:44:00.595365+0000","last_fullsized":"2023-10-10T20:44:00.595365+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:09:39.960969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":10,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.583884+0000","last_change":"2023-10-10T20:43:59.573196+0000","last_active":"2023-10-10T20:44:00.583884+0000","last_peered":"2023-10-10T20:44:00.583884+0000","last_clean":"2023-10-10T20:44:00.583884+0000","last_became_active":"2023-10-10T20:43:59.572951+0000","last_became_peered":"2023-10-10T20:43:59.572951+0000","last_unstale":"2023-10-10T20:44:00.583884+0000","last_undegraded":"2023-10-10T20:44:00.583884+0000","last_fullsized":"2023-10-10T20:44:00.583884+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:26.122088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.602910+0000","last_change":"2023-10-10T20:44:00.599485+0000","last_active":"2023-10-10T20:44:00.602910+0000","last_peered":"2023-10-10T20:44:00.602910+0000","last_clean":"2023-10-10T20:44:00.602910+0000","last_became_active":"2023-10-10T20:44:00.599229+0000","last_became_peered":"2023-10-10T20:44:00.599229+0000","last_unstale":"2023-10-10T20:44:00.602910+0000","last_undegraded":"2023-10-10T20:44:00.602910+0000","last_fullsized":"2023-10-10T20:44:00.602910+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:24:36.328058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"197'8","reported_seq":18,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.612661+0000","last_change":"2023-10-10T20:43:59.575018+0000","last_active":"2023-10-10T20:44:00.612661+0000","last_peered":"2023-10-10T20:44:00.612661+0000","last_clean":"2023-10-10T20:44:00.612661+0000","last_became_active":"2023-10-10T20:43:59.574763+0000","last_became_peered":"2023-10-10T20:43:59.574763+0000","last_unstale":"2023-10-10T20:44:00.612661+0000","last_undegraded":"2023-10-10T20:44:00.612661+0000","last_fullsized":"2023-10-10T20:44:00.612661+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:21.182748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.597695+0000","last_change":"2023-10-10T20:44:00.595132+0000","last_active":"2023-10-10T20:44:00.597695+0000","last_peered":"2023-10-10T20:44:00.597695+0000","last_clean":"2023-10-10T20:44:00.597695+0000","last_became_active":"2023-10-10T20:44:00.594756+0000","last_became_peered":"2023-10-10T20:44:00.594756+0000","last_unstale":"2023-10-10T20:44:00.597695+0000","last_undegraded":"2023-10-10T20:44:00.597695+0000","last_fullsized":"2023-10-10T20:44:00.597695+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:47.788162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"197'1","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.607390+0000","last_change":"2023-10-10T20:43:59.583608+0000","last_active":"2023-10-10T20:44:00.607390+0000","last_peered":"2023-10-10T20:44:00.607390+0000","last_clean":"2023-10-10T20:44:00.607390+0000","last_became_active":"2023-10-10T20:43:59.583319+0000","last_became_peered":"2023-10-10T20:43:59.583319+0000","last_unstale":"2023-10-10T20:44:00.607390+0000","last_undegraded":"2023-10-10T20:44:00.607390+0000","last_fullsized":"2023-10-10T20:44:00.607390+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:38:37.499256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.602337+0000","last_change":"2023-10-10T20:44:00.598608+0000","last_active":"2023-10-10T20:44:00.602337+0000","last_peered":"2023-10-10T20:44:00.602337+0000","last_clean":"2023-10-10T20:44:00.602337+0000","last_became_active":"2023-10-10T20:44:00.598075+0000","last_became_peered":"2023-10-10T20:44:00.598075+0000","last_unstale":"2023-10-10T20:44:00.602337+0000","last_undegraded":"2023-10-10T20:44:00.602337+0000","last_fullsized":"2023-10-10T20:44:00.602337+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:15:01.321794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"197'2","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.606752+0000","last_change":"2023-10-10T20:43:59.584155+0000","last_active":"2023-10-10T20:44:00.606752+0000","last_peered":"2023-10-10T20:44:00.606752+0000","last_clean":"2023-10-10T20:44:00.606752+0000","last_became_active":"2023-10-10T20:43:59.583881+0000","last_became_peered":"2023-10-10T20:43:59.583881+0000","last_unstale":"2023-10-10T20:44:00.606752+0000","last_undegraded":"2023-10-10T20:44:00.606752+0000","last_fullsized":"2023-10-10T20:44:00.606752+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:23:08.629960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.598034+0000","last_change":"2023-10-10T20:44:00.595974+0000","last_active":"2023-10-10T20:44:00.598034+0000","last_peered":"2023-10-10T20:44:00.598034+0000","last_clean":"2023-10-10T20:44:00.598034+0000","last_became_active":"2023-10-10T20:44:00.595682+0000","last_became_peered":"2023-10-10T20:44:00.595682+0000","last_unstale":"2023-10-10T20:44:00.598034+0000","last_undegraded":"2023-10-10T20:44:00.598034+0000","last_fullsized":"2023-10-10T20:44:00.598034+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:52:20.398004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"197'1","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.608059+0000","last_change":"2023-10-10T20:43:59.574744+0000","last_active":"2023-10-10T20:44:00.608059+0000","last_peered":"2023-10-10T20:44:00.608059+0000","last_clean":"2023-10-10T20:44:00.608059+0000","last_became_active":"2023-10-10T20:43:59.574442+0000","last_became_peered":"2023-10-10T20:43:59.574442+0000","last_unstale":"2023-10-10T20:44:00.608059+0000","last_undegraded":"2023-10-10T20:44:00.608059+0000","last_fullsized":"2023-10-10T20:44:00.608059+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:16:37.883455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.602746+0000","last_change":"2023-10-10T20:44:00.601091+0000","last_active":"2023-10-10T20:44:00.602746+0000","last_peered":"2023-10-10T20:44:00.602746+0000","last_clean":"2023-10-10T20:44:00.602746+0000","last_became_active":"2023-10-10T20:44:00.600634+0000","last_became_peered":"2023-10-10T20:44:00.600634+0000","last_unstale":"2023-10-10T20:44:00.602746+0000","last_undegraded":"2023-10-10T20:44:00.602746+0000","last_fullsized":"2023-10-10T20:44:00.602746+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:58:57.114720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":10,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.581169+0000","last_change":"2023-10-10T20:43:59.580293+0000","last_active":"2023-10-10T20:44:00.581169+0000","last_peered":"2023-10-10T20:44:00.581169+0000","last_clean":"2023-10-10T20:44:00.581169+0000","last_became_active":"2023-10-10T20:43:59.580059+0000","last_became_peered":"2023-10-10T20:43:59.580059+0000","last_unstale":"2023-10-10T20:44:00.581169+0000","last_undegraded":"2023-10-10T20:44:00.581169+0000","last_fullsized":"2023-10-10T20:44:00.581169+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:12:03.297895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.600572+0000","last_change":"2023-10-10T20:44:00.595049+0000","last_active":"2023-10-10T20:44:00.600572+0000","last_peered":"2023-10-10T20:44:00.600572+0000","last_clean":"2023-10-10T20:44:00.600572+0000","last_became_active":"2023-10-10T20:44:00.594787+0000","last_became_peered":"2023-10-10T20:44:00.594787+0000","last_unstale":"2023-10-10T20:44:00.600572+0000","last_undegraded":"2023-10-10T20:44:00.600572+0000","last_fullsized":"2023-10-10T20:44:00.600572+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:39:54.079085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"197'2","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.614916+0000","last_change":"2023-10-10T20:43:59.577141+0000","last_active":"2023-10-10T20:44:00.614916+0000","last_peered":"2023-10-10T20:44:00.614916+0000","last_clean":"2023-10-10T20:44:00.614916+0000","last_became_active":"2023-10-10T20:43:59.576924+0000","last_became_peered":"2023-10-10T20:43:59.576924+0000","last_unstale":"2023-10-10T20:44:00.614916+0000","last_undegraded":"2023-10-10T20:44:00.614916+0000","last_fullsized":"2023-10-10T20:44:00.614916+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:34.502187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.605961+0000","last_change":"2023-10-10T20:44:00.599459+0000","last_active":"2023-10-10T20:44:00.605961+0000","last_peered":"2023-10-10T20:44:00.605961+0000","last_clean":"2023-10-10T20:44:00.605961+0000","last_became_active":"2023-10-10T20:44:00.599177+0000","last_became_peered":"2023-10-10T20:44:00.599177+0000","last_unstale":"2023-10-10T20:44:00.605961+0000","last_undegraded":"2023-10-10T20:44:00.605961+0000","last_fullsized":"2023-10-10T20:44:00.605961+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:32:57.602233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":10,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.584369+0000","last_change":"2023-10-10T20:43:59.577352+0000","last_active":"2023-10-10T20:44:00.584369+0000","last_peered":"2023-10-10T20:44:00.584369+0000","last_clean":"2023-10-10T20:44:00.584369+0000","last_became_active":"2023-10-10T20:43:59.577091+0000","last_became_peered":"2023-10-10T20:43:59.577091+0000","last_unstale":"2023-10-10T20:44:00.584369+0000","last_undegraded":"2023-10-10T20:44:00.584369+0000","last_fullsized":"2023-10-10T20:44:00.584369+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:36:41.490546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.597174+0000","last_change":"2023-10-10T20:44:00.593379+0000","last_active":"2023-10-10T20:44:00.597174+0000","last_peered":"2023-10-10T20:44:00.597174+0000","last_clean":"2023-10-10T20:44:00.597174+0000","last_became_active":"2023-10-10T20:44:00.593114+0000","last_became_peered":"2023-10-10T20:44:00.593114+0000","last_unstale":"2023-10-10T20:44:00.597174+0000","last_undegraded":"2023-10-10T20:44:00.597174+0000","last_fullsized":"2023-10-10T20:44:00.597174+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:26:39.997043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":10,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.593705+0000","last_change":"2023-10-10T20:43:59.582646+0000","last_active":"2023-10-10T20:44:00.593705+0000","last_peered":"2023-10-10T20:44:00.593705+0000","last_clean":"2023-10-10T20:44:00.593705+0000","last_became_active":"2023-10-10T20:43:59.582342+0000","last_became_peered":"2023-10-10T20:43:59.582342+0000","last_unstale":"2023-10-10T20:44:00.593705+0000","last_undegraded":"2023-10-10T20:44:00.593705+0000","last_fullsized":"2023-10-10T20:44:00.593705+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:34.502187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.600899+0000","last_change":"2023-10-10T20:44:00.597765+0000","last_active":"2023-10-10T20:44:00.600899+0000","last_peered":"2023-10-10T20:44:00.600899+0000","last_clean":"2023-10-10T20:44:00.600899+0000","last_became_active":"2023-10-10T20:44:00.597561+0000","last_became_peered":"2023-10-10T20:44:00.597561+0000","last_unstale":"2023-10-10T20:44:00.600899+0000","last_undegraded":"2023-10-10T20:44:00.600899+0000","last_fullsized":"2023-10-10T20:44:00.600899+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:45:18.175031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":10,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.587472+0000","last_change":"2023-10-10T20:43:59.578840+0000","last_active":"2023-10-10T20:44:00.587472+0000","last_peered":"2023-10-10T20:44:00.587472+0000","last_clean":"2023-10-10T20:44:00.587472+0000","last_became_active":"2023-10-10T20:43:59.578251+0000","last_became_peered":"2023-10-10T20:43:59.578251+0000","last_unstale":"2023-10-10T20:44:00.587472+0000","last_undegraded":"2023-10-10T20:44:00.587472+0000","last_fullsized":"2023-10-10T20:44:00.587472+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:43:29.418903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.597231+0000","last_change":"2023-10-10T20:44:00.593297+0000","last_active":"2023-10-10T20:44:00.597231+0000","last_peered":"2023-10-10T20:44:00.597231+0000","last_clean":"2023-10-10T20:44:00.597231+0000","last_became_active":"2023-10-10T20:44:00.592573+0000","last_became_peered":"2023-10-10T20:44:00.592573+0000","last_unstale":"2023-10-10T20:44:00.597231+0000","last_undegraded":"2023-10-10T20:44:00.597231+0000","last_fullsized":"2023-10-10T20:44:00.597231+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:33:35.503994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"197'2","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.611742+0000","last_change":"2023-10-10T20:43:59.578097+0000","last_active":"2023-10-10T20:44:00.611742+0000","last_peered":"2023-10-10T20:44:00.611742+0000","last_clean":"2023-10-10T20:44:00.611742+0000","last_became_active":"2023-10-10T20:43:59.577818+0000","last_became_peered":"2023-10-10T20:43:59.577818+0000","last_unstale":"2023-10-10T20:44:00.611742+0000","last_undegraded":"2023-10-10T20:44:00.611742+0000","last_fullsized":"2023-10-10T20:44:00.611742+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:09:51.106350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.594289+0000","last_change":"2023-10-10T20:44:00.589894+0000","last_active":"2023-10-10T20:44:00.594289+0000","last_peered":"2023-10-10T20:44:00.594289+0000","last_clean":"2023-10-10T20:44:00.594289+0000","last_became_active":"2023-10-10T20:44:00.589644+0000","last_became_peered":"2023-10-10T20:44:00.589644+0000","last_unstale":"2023-10-10T20:44:00.594289+0000","last_undegraded":"2023-10-10T20:44:00.594289+0000","last_fullsized":"2023-10-10T20:44:00.594289+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:02:55.641984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":10,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.581437+0000","last_change":"2023-10-10T20:43:59.581522+0000","last_active":"2023-10-10T20:44:00.581437+0000","last_peered":"2023-10-10T20:44:00.581437+0000","last_clean":"2023-10-10T20:44:00.581437+0000","last_became_active":"2023-10-10T20:43:59.581301+0000","last_became_peered":"2023-10-10T20:43:59.581301+0000","last_unstale":"2023-10-10T20:44:00.581437+0000","last_undegraded":"2023-10-10T20:44:00.581437+0000","last_fullsized":"2023-10-10T20:44:00.581437+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:28:44.406143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.591790+0000","last_change":"2023-10-10T20:44:00.588444+0000","last_active":"2023-10-10T20:44:00.591790+0000","last_peered":"2023-10-10T20:44:00.591790+0000","last_clean":"2023-10-10T20:44:00.591790+0000","last_became_active":"2023-10-10T20:44:00.588193+0000","last_became_peered":"2023-10-10T20:44:00.588193+0000","last_unstale":"2023-10-10T20:44:00.591790+0000","last_undegraded":"2023-10-10T20:44:00.591790+0000","last_fullsized":"2023-10-10T20:44:00.591790+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:23:21.007265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":10,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.582151+0000","last_change":"2023-10-10T20:43:59.577332+0000","last_active":"2023-10-10T20:44:00.582151+0000","last_peered":"2023-10-10T20:44:00.582151+0000","last_clean":"2023-10-10T20:44:00.582151+0000","last_became_active":"2023-10-10T20:43:59.577039+0000","last_became_peered":"2023-10-10T20:43:59.577039+0000","last_unstale":"2023-10-10T20:44:00.582151+0000","last_undegraded":"2023-10-10T20:44:00.582151+0000","last_fullsized":"2023-10-10T20:44:00.582151+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:30:22.317978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.599704+0000","last_change":"2023-10-10T20:44:00.596918+0000","last_active":"2023-10-10T20:44:00.599704+0000","last_peered":"2023-10-10T20:44:00.599704+0000","last_clean":"2023-10-10T20:44:00.599704+0000","last_became_active":"2023-10-10T20:44:00.596644+0000","last_became_peered":"2023-10-10T20:44:00.596644+0000","last_unstale":"2023-10-10T20:44:00.599704+0000","last_undegraded":"2023-10-10T20:44:00.599704+0000","last_fullsized":"2023-10-10T20:44:00.599704+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:21:30.171884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":10,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.581543+0000","last_change":"2023-10-10T20:43:59.574209+0000","last_active":"2023-10-10T20:44:00.581543+0000","last_peered":"2023-10-10T20:44:00.581543+0000","last_clean":"2023-10-10T20:44:00.581543+0000","last_became_active":"2023-10-10T20:43:59.573906+0000","last_became_peered":"2023-10-10T20:43:59.573906+0000","last_unstale":"2023-10-10T20:44:00.581543+0000","last_undegraded":"2023-10-10T20:44:00.581543+0000","last_fullsized":"2023-10-10T20:44:00.581543+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:08:51.857169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.590391+0000","last_change":"2023-10-10T20:44:00.588646+0000","last_active":"2023-10-10T20:44:00.590391+0000","last_peered":"2023-10-10T20:44:00.590391+0000","last_clean":"2023-10-10T20:44:00.590391+0000","last_became_active":"2023-10-10T20:44:00.588404+0000","last_became_peered":"2023-10-10T20:44:00.588404+0000","last_unstale":"2023-10-10T20:44:00.590391+0000","last_undegraded":"2023-10-10T20:44:00.590391+0000","last_fullsized":"2023-10-10T20:44:00.590391+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:47:45.534622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":10,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.581069+0000","last_change":"2023-10-10T20:43:59.580917+0000","last_active":"2023-10-10T20:44:00.581069+0000","last_peered":"2023-10-10T20:44:00.581069+0000","last_clean":"2023-10-10T20:44:00.581069+0000","last_became_active":"2023-10-10T20:43:59.580687+0000","last_became_peered":"2023-10-10T20:43:59.580687+0000","last_unstale":"2023-10-10T20:44:00.581069+0000","last_undegraded":"2023-10-10T20:44:00.581069+0000","last_fullsized":"2023-10-10T20:44:00.581069+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:31:32.697562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.601916+0000","last_change":"2023-10-10T20:44:00.599489+0000","last_active":"2023-10-10T20:44:00.601916+0000","last_peered":"2023-10-10T20:44:00.601916+0000","last_clean":"2023-10-10T20:44:00.601916+0000","last_became_active":"2023-10-10T20:44:00.598927+0000","last_became_peered":"2023-10-10T20:44:00.598927+0000","last_unstale":"2023-10-10T20:44:00.601916+0000","last_undegraded":"2023-10-10T20:44:00.601916+0000","last_fullsized":"2023-10-10T20:44:00.601916+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:56:16.507338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":10,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.581099+0000","last_change":"2023-10-10T20:43:59.579194+0000","last_active":"2023-10-10T20:44:00.581099+0000","last_peered":"2023-10-10T20:44:00.581099+0000","last_clean":"2023-10-10T20:44:00.581099+0000","last_became_active":"2023-10-10T20:43:59.578972+0000","last_became_peered":"2023-10-10T20:43:59.578972+0000","last_unstale":"2023-10-10T20:44:00.581099+0000","last_undegraded":"2023-10-10T20:44:00.581099+0000","last_fullsized":"2023-10-10T20:44:00.581099+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:20:10.035756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.598432+0000","last_change":"2023-10-10T20:44:00.593797+0000","last_active":"2023-10-10T20:44:00.598432+0000","last_peered":"2023-10-10T20:44:00.598432+0000","last_clean":"2023-10-10T20:44:00.598432+0000","last_became_active":"2023-10-10T20:44:00.593488+0000","last_became_peered":"2023-10-10T20:44:00.593488+0000","last_unstale":"2023-10-10T20:44:00.598432+0000","last_undegraded":"2023-10-10T20:44:00.598432+0000","last_fullsized":"2023-10-10T20:44:00.598432+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:46:26.847440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":10,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.582268+0000","last_change":"2023-10-10T20:43:59.577170+0000","last_active":"2023-10-10T20:44:00.582268+0000","last_peered":"2023-10-10T20:44:00.582268+0000","last_clean":"2023-10-10T20:44:00.582268+0000","last_became_active":"2023-10-10T20:43:59.576268+0000","last_became_peered":"2023-10-10T20:43:59.576268+0000","last_unstale":"2023-10-10T20:44:00.582268+0000","last_undegraded":"2023-10-10T20:44:00.582268+0000","last_fullsized":"2023-10-10T20:44:00.582268+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:34.995893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.597607+0000","last_change":"2023-10-10T20:44:00.592057+0000","last_active":"2023-10-10T20:44:00.597607+0000","last_peered":"2023-10-10T20:44:00.597607+0000","last_clean":"2023-10-10T20:44:00.597607+0000","last_became_active":"2023-10-10T20:44:00.591731+0000","last_became_peered":"2023-10-10T20:44:00.591731+0000","last_unstale":"2023-10-10T20:44:00.597607+0000","last_undegraded":"2023-10-10T20:44:00.597607+0000","last_fullsized":"2023-10-10T20:44:00.597607+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:59:29.901357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":10,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.580801+0000","last_change":"2023-10-10T20:43:59.576353+0000","last_active":"2023-10-10T20:44:00.580801+0000","last_peered":"2023-10-10T20:44:00.580801+0000","last_clean":"2023-10-10T20:44:00.580801+0000","last_became_active":"2023-10-10T20:43:59.576110+0000","last_became_peered":"2023-10-10T20:43:59.576110+0000","last_unstale":"2023-10-10T20:44:00.580801+0000","last_undegraded":"2023-10-10T20:44:00.580801+0000","last_fullsized":"2023-10-10T20:44:00.580801+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:58.558180+0000","last_clean_scrub_stamp":"2023-10-10T20:43:58.558180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:39:19.959971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":9,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.597256+0000","last_change":"2023-10-10T20:44:00.594980+0000","last_active":"2023-10-10T20:44:00.597256+0000","last_peered":"2023-10-10T20:44:00.597256+0000","last_clean":"2023-10-10T20:44:00.597256+0000","last_became_active":"2023-10-10T20:44:00.594555+0000","last_became_peered":"2023-10-10T20:44:00.594555+0000","last_unstale":"2023-10-10T20:44:00.597256+0000","last_undegraded":"2023-10-10T20:44:00.597256+0000","last_fullsized":"2023-10-10T20:44:00.597256+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:43:59.559987+0000","last_clean_scrub_stamp":"2023-10-10T20:43:59.559987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:19:49.027140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":341,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-10T20:44:00.581143+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:44:00.581143+0000","last_peered":"2023-10-10T20:44:00.581143+0000","last_clean":"2023-10-10T20:44:00.581143+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:44:00.581143+0000","last_undegraded":"2023-10-10T20:44:00.581143+0000","last_fullsized":"2023-10-10T20:44:00.581143+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574967,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304432,"kb_used_data":3048,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067408,"statfs":{"total":96636764160,"available":96325025792,"internally_reserved":0,"allocated":3121152,"data_stored":1724387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51300000000000001,"15min":0.72699999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.748,"5min":0.80300000000000005,"15min":6.8630000000000004},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.504,"15min":0.59299999999999997},"min":{"1min":0.33300000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":4.1959999999999997},"last":0.79600000000000004}]},{"osd":1,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51300000000000001,"15min":0.85299999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77900000000000003,"5min":0.78800000000000003,"15min":10.1},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53700000000000003,"15min":0.67400000000000004},"min":{"1min":0.375,"5min":0.33900000000000002,"15min":0.28999999999999998},"max":{"1min":0.79200000000000004,"5min":0.81799999999999995,"15min":6.806},"last":0.66900000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.72099999999999997},"min":{"1min":0.378,"5min":0.308,"15min":0.308},"max":{"1min":0.75700000000000001,"5min":0.89000000000000001,"15min":6.9340000000000002},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53300000000000003,"15min":0.67600000000000005},"min":{"1min":0.39600000000000002,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.747,"5min":0.77100000000000002,"15min":6.9870000000000001},"last":0.75700000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54300000000000004,"15min":0.69299999999999995},"min":{"1min":0.38500000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72999999999999998,"5min":0.754,"15min":6.6719999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56200000000000006,"15min":0.63900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":4.1609999999999996},"last":0.74099999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.47099999999999997,"15min":0.46700000000000003},"min":{"1min":0.30599999999999999,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.73199999999999998,"15min":0.77200000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.73899999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.54200000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.53800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.66500000000000004,"5min":0.86099999999999999,"15min":1.3180000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.34899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64800000000000002,"5min":0.78200000000000003,"15min":0.84099999999999997},"last":0.61599999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71499999999999997,"5min":0.77100000000000002,"15min":1.7509999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.45900000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73099999999999998,"5min":0.84099999999999997,"15min":1.7090000000000001},"last":0.81999999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574968,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303848,"kb_used_data":2464,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067992,"statfs":{"total":96636764160,"available":96325623808,"internally_reserved":0,"allocated":2523136,"data_stored":1133477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52900000000000003,"15min":0.626},"min":{"1min":0.31,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.80500000000000005,"5min":0.83099999999999996,"15min":7.7590000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52700000000000002,"15min":0.65500000000000003},"min":{"1min":0.32800000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79700000000000004,"5min":1.075,"15min":7.8090000000000002},"last":0.42899999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53200000000000003,"15min":0.65500000000000003},"min":{"1min":0.40600000000000003,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":7.867},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55700000000000005,"15min":0.78600000000000003},"min":{"1min":0.36099999999999999,"5min":0.35499999999999998,"15min":0.33700000000000002},"max":{"1min":0.85999999999999999,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.53900000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52800000000000002,"15min":0.57399999999999995},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83699999999999997,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.76100000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.318},"max":{"1min":0.875,"5min":2.641,"15min":6.7629999999999999},"last":0.58299999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.89300000000000002,"5min":1.107,"15min":4.8620000000000001},"last":0.504},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55500000000000005,"15min":0.66800000000000004},"min":{"1min":0.40200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.78900000000000003,"5min":1.718,"15min":7.9009999999999998},"last":0.441}]},{"osd":4,"last update":"Tue Oct 10 20:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52700000000000002,"15min":0.48599999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.82399999999999995,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60599999999999998,"15min":0.57699999999999996},"min":{"1min":0.39000000000000001,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":0.80500000000000005,"5min":1.032,"15min":1.032},"last":0.46300000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.53300000000000003,"15min":0.502},"min":{"1min":0.29899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73099999999999998,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58399999999999996,"15min":0.54300000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.33600000000000002},"max":{"1min":0.85899999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.40300000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57499999999999996,"15min":0.53500000000000003},"min":{"1min":0.33300000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.83899999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59999999999999998,"15min":0.55900000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.35099999999999998},"max":{"1min":0.80100000000000005,"5min":1.681,"15min":1.681},"last":0.49099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303852,"kb_used_data":2468,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067988,"statfs":{"total":96636764160,"available":96325619712,"internally_reserved":0,"allocated":2527232,"data_stored":1133567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.31900000000000001,"15min":0.42699999999999999},"min":{"1min":0.248,"5min":0.20999999999999999,"15min":0.17899999999999999},"max":{"1min":0.41399999999999998,"5min":0.56799999999999995,"15min":5.0380000000000003},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.33600000000000002,"15min":0.35299999999999998},"min":{"1min":0.245,"5min":0.22500000000000001,"15min":0.20899999999999999},"max":{"1min":0.52300000000000002,"5min":0.65500000000000003,"15min":1.194},"last":0.28999999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.35199999999999998},"min":{"1min":0.255,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":1.2869999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36299999999999999,"15min":0.45800000000000002},"min":{"1min":0.29399999999999998,"5min":0.23999999999999999,"15min":0.22},"max":{"1min":0.50700000000000001,"5min":0.54200000000000004,"15min":5.0919999999999996},"last":0.40200000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.48199999999999998},"min":{"1min":0.28899999999999998,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":5.1159999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.372,"15min":0.39600000000000002},"min":{"1min":0.29599999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.54300000000000004,"5min":0.57799999999999996,"15min":1.2170000000000001},"last":0.36599999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.30199999999999999},"max":{"1min":0.747,"5min":0.92400000000000004,"15min":1.254},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.66000000000000003},"min":{"1min":0.40100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78900000000000003,"5min":0.96499999999999997,"15min":4.9850000000000003},"last":0.65400000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34399999999999997},"max":{"1min":0.75,"5min":0.97599999999999998,"15min":1.1799999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57099999999999995,"15min":0.59699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.79700000000000004,"5min":0.878,"15min":1.3109999999999999},"last":0.63200000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59799999999999998,"15min":0.76700000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.35199999999999998},"max":{"1min":0.76400000000000001,"5min":0.995,"15min":5.0659999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.38600000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76600000000000001,"5min":0.85799999999999998,"15min":1.171},"last":0.68000000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.60899999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.36499999999999999},"max":{"1min":0.72799999999999998,"5min":1.004,"15min":1.214},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.68400000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78200000000000003,"5min":0.90500000000000003,"15min":4.0499999999999998},"last":0.747}]}]},{"osd":0,"up_from":13,"seq":55834574966,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304424,"kb_used_data":3040,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067416,"statfs":{"total":96636764160,"available":96325033984,"internally_reserved":0,"allocated":3112960,"data_stored":1722920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.314,"15min":0.46400000000000002},"min":{"1min":0.23100000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.40000000000000002,"5min":0.63400000000000001,"15min":7.9249999999999998},"last":0.312},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.32600000000000001,"15min":0.45100000000000001},"min":{"1min":0.23499999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46300000000000002,"5min":0.53800000000000003,"15min":6.6299999999999999},"last":0.33100000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.33600000000000002,"15min":0.48699999999999999},"min":{"1min":0.23899999999999999,"5min":0.22,"15min":0.216},"max":{"1min":0.46800000000000003,"5min":0.67800000000000005,"15min":8.2509999999999994},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.34699999999999998,"15min":0.47399999999999998},"min":{"1min":0.26400000000000001,"5min":0.23100000000000001,"15min":0.222},"max":{"1min":0.502,"5min":0.57899999999999996,"15min":6.7370000000000001},"last":0.27900000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36199999999999999,"15min":0.497},"min":{"1min":0.28999999999999998,"5min":0.22,"15min":0.21099999999999999},"max":{"1min":0.46400000000000002,"5min":0.55200000000000005,"15min":7.7960000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40400000000000003,"15min":0.54800000000000004},"min":{"1min":0.32600000000000001,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.51500000000000001,"5min":0.71699999999999997,"15min":7.8929999999999998},"last":0.39900000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56100000000000005,"15min":0.67000000000000004},"min":{"1min":0.40300000000000002,"5min":0.34300000000000003,"15min":0.33300000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":6.6929999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.71099999999999997},"min":{"1min":0.40899999999999997,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.84399999999999997,"5min":0.95399999999999996,"15min":7.6920000000000002},"last":0.52800000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55900000000000005,"15min":0.69499999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.36299999999999999},"max":{"1min":0.80200000000000005,"5min":0.92400000000000004,"15min":7.7690000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59099999999999997,"15min":0.72099999999999997},"min":{"1min":0.42199999999999999,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":7.8659999999999997},"last":0.51700000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60599999999999998,"15min":0.753},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76000000000000001,"5min":0.94499999999999995,"15min":7.8360000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61299999999999999,"15min":0.71699999999999997},"min":{"1min":0.46500000000000002,"5min":0.41399999999999998,"15min":0.36599999999999999},"max":{"1min":0.875,"5min":0.90300000000000002,"15min":6.6660000000000004},"last":0.58899999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60799999999999998,"15min":0.70899999999999996},"min":{"1min":0.435,"5min":0.39900000000000002,"15min":0.36299999999999999},"max":{"1min":0.90600000000000003,"5min":0.93300000000000005,"15min":6.7160000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58399999999999996,"15min":0.72499999999999998},"min":{"1min":0.45800000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.89500000000000002,"15min":7.7370000000000001},"last":0.55600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303864,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067976,"statfs":{"total":96636764160,"available":96325607424,"internally_reserved":0,"allocated":2539520,"data_stored":1135555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.309,"15min":0.32100000000000001},"min":{"1min":0.22,"5min":0.189,"15min":0.189},"max":{"1min":0.36899999999999999,"5min":0.59699999999999998,"15min":0.67900000000000005},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32600000000000001,"15min":0.34599999999999997},"min":{"1min":0.22,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.505,"5min":0.54200000000000004,"15min":0.746},"last":0.36099999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33400000000000002,"15min":0.61199999999999999},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.40000000000000002,"5min":0.63,"15min":11.967000000000001},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.35199999999999998,"15min":0.44400000000000001},"min":{"1min":0.26400000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.47599999999999998,"5min":0.52700000000000002,"15min":4.1100000000000003},"last":0.318}]},{"osd":3,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.34100000000000003,"15min":0.49199999999999999},"min":{"1min":0.26200000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.378,"5min":0.53600000000000003,"15min":6.3040000000000003},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.38700000000000001,"15min":0.66400000000000003},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.497,"5min":0.65900000000000003,"15min":11.935},"last":0.38400000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83399999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.40999999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.752,"5min":1.302,"15min":1.302},"last":0.56000000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.56100000000000005},"min":{"1min":0.41499999999999998,"5min":0.38,"15min":0.378},"max":{"1min":0.73999999999999999,"5min":0.83999999999999997,"15min":0.95699999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55500000000000005,"15min":0.57599999999999996},"min":{"1min":0.39300000000000002,"5min":0.35999999999999999,"15min":0.34300000000000003},"max":{"1min":0.77700000000000002,"5min":0.83999999999999997,"15min":0.96099999999999997},"last":0.56999999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.40799999999999997,"5min":0.33400000000000002,"15min":0.33200000000000002},"max":{"1min":0.84299999999999997,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.42999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.61099999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.41899999999999998,"5min":0.39900000000000002,"15min":0.38700000000000001},"max":{"1min":0.80700000000000005,"5min":0.89700000000000002,"15min":1.052},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59599999999999997,"15min":0.72499999999999998},"min":{"1min":0.48899999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.748,"5min":1.274,"15min":6.2649999999999997},"last":0.64200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303848,"kb_used_data":2464,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067992,"statfs":{"total":96636764160,"available":96325623808,"internally_reserved":0,"allocated":2523136,"data_stored":1133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.31,"15min":0.313},"min":{"1min":0.222,"5min":0.21299999999999999,"15min":0.21099999999999999},"max":{"1min":0.47699999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.31,"5min":0.32700000000000001,"15min":0.32000000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.26000000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34399999999999997,"15min":0.33500000000000002},"min":{"1min":0.22700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.39000000000000001,"15min":0.38},"min":{"1min":0.316,"5min":0.26800000000000002,"15min":0.251},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.379}]},{"osd":2,"last update":"Tue Oct 10 20:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.36399999999999999,"15min":0.35299999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.46800000000000003,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.26800000000000002,"5min":0.25900000000000001,"15min":0.255},"max":{"1min":0.51000000000000001,"5min":0.64300000000000002,"15min":0.68000000000000005},"last":0.46000000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56999999999999995,"15min":0.54100000000000004},"min":{"1min":0.372,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76400000000000001,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57199999999999995,"15min":0.54300000000000004},"min":{"1min":0.46400000000000002,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.70999999999999996,"5min":0.83299999999999996,"15min":0.93500000000000005},"last":0.36699999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.55600000000000005},"min":{"1min":0.45600000000000002,"5min":0.39400000000000002,"15min":0.35899999999999999},"max":{"1min":0.66600000000000004,"5min":1.347,"15min":1.347},"last":0.443},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57599999999999996,"15min":0.55200000000000005},"min":{"1min":0.44600000000000001,"5min":0.39300000000000002,"15min":0.35799999999999998},"max":{"1min":0.82799999999999996,"5min":0.86799999999999999,"15min":1.01},"last":0.39200000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61199999999999999,"15min":0.59399999999999997},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.36699999999999999},"max":{"1min":0.78500000000000003,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.627,"15min":0.60899999999999999},"min":{"1min":0.49399999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79900000000000004,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.42299999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60699999999999998,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81899999999999995,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.623,"15min":0.59799999999999998},"min":{"1min":0.441,"5min":0.435,"15min":0.38200000000000001},"max":{"1min":0.78900000000000003,"5min":0.89500000000000002,"15min":0.96999999999999997},"last":0.433}]}]},{"osd":4,"up_from":13,"seq":55834574966,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303856,"kb_used_data":2472,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067984,"statfs":{"total":96636764160,"available":96325615616,"internally_reserved":0,"allocated":2531328,"data_stored":1134979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.50800000000000001,"15min":0.51100000000000001},"min":{"1min":0.38900000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.747},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52600000000000002,"15min":0.69599999999999995},"min":{"1min":0.43099999999999999,"5min":0.34000000000000002,"15min":0.32700000000000001},"max":{"1min":0.73099999999999998,"5min":0.77600000000000002,"15min":7.6719999999999997},"last":0.59799999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.70099999999999996},"min":{"1min":0.34100000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":7.7160000000000002},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.32700000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.86799999999999999},"last":0.61399999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.50700000000000001,"15min":0.51200000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.72399999999999998,"5min":0.76000000000000001,"15min":0.78000000000000003},"last":0.629},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56999999999999995,"15min":0.72799999999999998},"min":{"1min":0.437,"5min":0.39700000000000002,"15min":0.379},"max":{"1min":0.80500000000000005,"5min":0.81599999999999995,"15min":7.7839999999999998},"last":0.49199999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73599999999999999,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.40999999999999998,"5min":0.39200000000000002,"15min":0.35499999999999998},"max":{"1min":0.77100000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.57799999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53900000000000003,"15min":0.5},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.30099999999999999},"max":{"1min":0.78900000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.54800000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79600000000000004,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.35599999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54400000000000004,"15min":0.53100000000000003},"min":{"1min":0.379,"5min":0.318,"15min":0.318},"max":{"1min":0.69699999999999995,"5min":0.85199999999999998,"15min":0.85899999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60599999999999998,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82499999999999996,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.45700000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61199999999999999,"15min":0.58999999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.36199999999999999},"max":{"1min":0.82899999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.59899999999999998},"min":{"1min":0.41999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.86899999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.374}]}]},{"osd":5,"up_from":13,"seq":55834574968,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303848,"kb_used_data":2464,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067992,"statfs":{"total":96636764160,"available":96325623808,"internally_reserved":0,"allocated":2523136,"data_stored":1133513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49399999999999999,"15min":0.48899999999999999},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.52300000000000002},"min":{"1min":0.314,"5min":0.30599999999999999,"15min":0.215},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.67700000000000005}]},{"osd":1,"last update":"Tue Oct 10 20:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.51100000000000001,"15min":0.60399999999999998},"min":{"1min":0.30099999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.71899999999999997,"5min":0.746,"15min":5.2679999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.504,"5min":0.52700000000000002,"15min":0.59899999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":5.3109999999999999},"last":0.63900000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56399999999999995,"15min":0.627},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.98999999999999999,"5min":3.27,"15min":5.2960000000000003},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69599999999999995,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.627}]},{"osd":3,"last update":"Tue Oct 10 20:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55800000000000005,"15min":0.64600000000000002},"min":{"1min":0.39200000000000002,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":1.02,"5min":1.02,"15min":5.3330000000000002},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56799999999999995,"15min":0.65400000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.27700000000000002},"max":{"1min":0.78700000000000003,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.66400000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.496,"15min":0.496},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53900000000000003,"15min":0.54400000000000004},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32200000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0800000000000001},"last":0.52500000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53700000000000003,"15min":0.54700000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.30599999999999999},"max":{"1min":0.749,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.54600000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.436,"5min":0.33000000000000002,"15min":0.30099999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.38200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.81599999999999995,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.69799999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:44:05.036 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:44:05.036 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:44:05.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.189+0000 7fc015c0d700 1 Processor -- start 2023-10-10T20:44:05.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.189+0000 7fc015c0d700 1 -- start start 2023-10-10T20:44:05.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.190+0000 7fc015c0d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0100cdcf0 0x7fc0100c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:05.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.190+0000 7fc015c0d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc0100c8340 0x7fc01015d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:05.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.190+0000 7fc015c0d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc01015abd0 con 0x7fc010159cd0 2023-10-10T20:44:05.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.190+0000 7fc015c0d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0100c8ac0 con 0x7fc0100c8340 2023-10-10T20:44:05.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.190+0000 7fc015c0d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01015a0a0 con 0x7fc0100cdcf0 2023-10-10T20:44:05.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.190+0000 7fc00ffff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc0100c8340 0x7fc01015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:05.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.190+0000 7fc00effd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0100cdcf0 0x7fc0100c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:05.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.190+0000 7fc00f7fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc010159cd0 0x7fc0100cd5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:36976/0 (socket says 172.21.15.89:36976) 2023-10-10T20:44:05.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.190+0000 7fc00f7fe700 1 -- 172.21.15.89:0/2302344056 learned_addr learned my addr 172.21.15.89:0/2302344056 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:44:05.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.191+0000 7fc00effd700 1 -- 172.21.15.89:0/2302344056 >> v2:172.21.15.111:3300/0 conn(0x7fc0100c8340 msgr2=0x7fc01015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:05.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.191+0000 7fc00effd700 1 --2- 172.21.15.89:0/2302344056 >> v2:172.21.15.111:3300/0 conn(0x7fc0100c8340 0x7fc01015d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.191+0000 7fc00effd700 1 -- 172.21.15.89:0/2302344056 >> v1:172.21.15.111:6789/0 conn(0x7fc010159cd0 legacy=0x7fc0100cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:05.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.191+0000 7fc00effd700 1 -- 172.21.15.89:0/2302344056 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01015b060 con 0x7fc0100cdcf0 2023-10-10T20:44:05.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.191+0000 7fc00effd700 1 --2- 172.21.15.89:0/2302344056 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0100cdcf0 0x7fc0100c7e00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbff80098e0 tx=0x7fbff80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d32525270af145 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:05.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.191+0000 7fc00e7fc700 1 -- 172.21.15.89:0/2302344056 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff800b640 con 0x7fc0100cdcf0 2023-10-10T20:44:05.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.191+0000 7fc00e7fc700 1 -- 172.21.15.89:0/2302344056 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff800b7a0 con 0x7fc0100cdcf0 2023-10-10T20:44:05.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.191+0000 7fc00e7fc700 1 -- 172.21.15.89:0/2302344056 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff800bb40 con 0x7fc0100cdcf0 2023-10-10T20:44:05.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.191+0000 7fc015c0d700 1 -- 172.21.15.89:0/2302344056 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0100cdcf0 msgr2=0x7fc0100c7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:05.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.191+0000 7fc015c0d700 1 --2- 172.21.15.89:0/2302344056 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0100cdcf0 0x7fc0100c7e00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbff80098e0 tx=0x7fbff80091a0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.191+0000 7fc015c0d700 1 -- 172.21.15.89:0/2302344056 shutdown_connections 2023-10-10T20:44:05.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.191+0000 7fc015c0d700 1 --2- 172.21.15.89:0/2302344056 >> v2:172.21.15.111:3300/0 conn(0x7fc0100c8340 0x7fc01015d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.191+0000 7fc015c0d700 1 --2- 172.21.15.89:0/2302344056 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0100cdcf0 0x7fc0100c7e00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.191+0000 7fc015c0d700 1 -- 172.21.15.89:0/2302344056 >> 172.21.15.89:0/2302344056 conn(0x7fc0100694a0 msgr2=0x7fc0100698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:05.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.191+0000 7fc015c0d700 1 -- 172.21.15.89:0/2302344056 shutdown_connections 2023-10-10T20:44:05.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.191+0000 7fc015c0d700 1 -- 172.21.15.89:0/2302344056 wait complete. 2023-10-10T20:44:05.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.193+0000 7fc015c0d700 1 Processor -- start 2023-10-10T20:44:05.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.193+0000 7fc015c0d700 1 -- start start 2023-10-10T20:44:05.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.193+0000 7fc015c0d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0100c8340 0x7fc01014c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:05.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.193+0000 7fc015c0d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0100cdcf0 0x7fc0100aba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:05.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.193+0000 7fc015c0d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc010159cd0 0x7fc0100abf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:05.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.193+0000 7fc015c0d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0100c1720 con 0x7fc0100cdcf0 2023-10-10T20:44:05.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.193+0000 7fc015c0d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0100b2aa0 con 0x7fc010159cd0 2023-10-10T20:44:05.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.193+0000 7fc015c0d700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0100b2d80 con 0x7fc0100c8340 2023-10-10T20:44:05.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.194+0000 7fc00effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0100cdcf0 0x7fc0100aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:05.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.194+0000 7fc00effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0100cdcf0 0x7fc0100aba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52046/0 (socket says 172.21.15.89:52046) 2023-10-10T20:44:05.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.194+0000 7fc00effd700 1 -- 172.21.15.89:0/3971983030 learned_addr learned my addr 172.21.15.89:0/3971983030 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:05.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.194+0000 7fc00ffff700 1 --2- 172.21.15.89:0/3971983030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc010159cd0 0x7fc0100abf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:05.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.194+0000 7fc00f7fe700 1 --2- 172.21.15.89:0/3971983030 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0100c8340 0x7fc01014c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:05.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.194+0000 7fc00effd700 1 -- 172.21.15.89:0/3971983030 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0100c8340 msgr2=0x7fc01014c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:05.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.194+0000 7fc00effd700 1 --2- 172.21.15.89:0/3971983030 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0100c8340 0x7fc01014c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.194+0000 7fc00effd700 1 -- 172.21.15.89:0/3971983030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc010159cd0 msgr2=0x7fc0100abf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:05.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.194+0000 7fc00effd700 1 --2- 172.21.15.89:0/3971983030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc010159cd0 0x7fc0100abf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.194+0000 7fc00effd700 1 -- 172.21.15.89:0/3971983030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff8009420 con 0x7fc0100cdcf0 2023-10-10T20:44:05.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.194+0000 7fc00effd700 1 --2- 172.21.15.89:0/3971983030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0100cdcf0 0x7fc0100aba30 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fbff800a010 tx=0x7fbff8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:05.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.194+0000 7fc0077fe700 1 -- 172.21.15.89:0/3971983030 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff801bc80 con 0x7fc0100cdcf0 2023-10-10T20:44:05.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.194+0000 7fc0077fe700 1 -- 172.21.15.89:0/3971983030 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff801be20 con 0x7fc0100cdcf0 2023-10-10T20:44:05.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.194+0000 7fc015c0d700 1 -- 172.21.15.89:0/3971983030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0100c10d0 con 0x7fc0100cdcf0 2023-10-10T20:44:05.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.194+0000 7fc0077fe700 1 -- 172.21.15.89:0/3971983030 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff8011670 con 0x7fc0100cdcf0 2023-10-10T20:44:05.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.194+0000 7fc015c0d700 1 -- 172.21.15.89:0/3971983030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01014ce80 con 0x7fc0100cdcf0 2023-10-10T20:44:05.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.195+0000 7fc015c0d700 1 -- 172.21.15.89:0/3971983030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0100c1890 con 0x7fc0100cdcf0 2023-10-10T20:44:05.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.195+0000 7fc0077fe700 1 -- 172.21.15.89:0/3971983030 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbff8011aa0 con 0x7fc0100cdcf0 2023-10-10T20:44:05.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.196+0000 7fc0077fe700 1 --2- 172.21.15.89:0/3971983030 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbff0067320 0x7fbff00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:05.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.196+0000 7fc0077fe700 1 -- 172.21.15.89:0/3971983030 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbff8086450 con 0x7fc0100cdcf0 2023-10-10T20:44:05.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.196+0000 7fc00f7fe700 1 --2- 172.21.15.89:0/3971983030 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbff0067320 0x7fbff00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:05.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.196+0000 7fc00f7fe700 1 --2- 172.21.15.89:0/3971983030 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbff0067320 0x7fbff00697d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc00800f380 tx=0x7fc008009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:05.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.200+0000 7fc0077fe700 1 -- 172.21.15.89:0/3971983030 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbff80530e0 con 0x7fc0100cdcf0 2023-10-10T20:44:05.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.461+0000 7fc015c0d700 1 -- 172.21.15.89:0/3971983030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc0100718b0 con 0x7fc0100cdcf0 2023-10-10T20:44:05.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.464+0000 7fc0077fe700 1 -- 172.21.15.89:0/3971983030 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v27) v1 ==== 190+0+0 (secure 0 0 0) 0x7fbff8056700 con 0x7fc0100cdcf0 2023-10-10T20:44:05.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.467+0000 7fc015c0d700 1 -- 172.21.15.89:0/3971983030 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbff0067320 msgr2=0x7fbff00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:05.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.467+0000 7fc015c0d700 1 --2- 172.21.15.89:0/3971983030 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbff0067320 0x7fbff00697d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc00800f380 tx=0x7fc008009040 comp rx=0 tx=0).stop 2023-10-10T20:44:05.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.467+0000 7fc015c0d700 1 -- 172.21.15.89:0/3971983030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0100cdcf0 msgr2=0x7fc0100aba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:05.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.467+0000 7fc015c0d700 1 --2- 172.21.15.89:0/3971983030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0100cdcf0 0x7fc0100aba30 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fbff800a010 tx=0x7fbff8017000 comp rx=0 tx=0).stop 2023-10-10T20:44:05.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.467+0000 7fc015c0d700 1 -- 172.21.15.89:0/3971983030 shutdown_connections 2023-10-10T20:44:05.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.467+0000 7fc015c0d700 1 --2- 172.21.15.89:0/3971983030 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbff0067320 0x7fbff00697d0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.467+0000 7fc015c0d700 1 --2- 172.21.15.89:0/3971983030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc010159cd0 0x7fc0100abf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.467+0000 7fc015c0d700 1 --2- 172.21.15.89:0/3971983030 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc0100c8340 0x7fc01014c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.467+0000 7fc015c0d700 1 --2- 172.21.15.89:0/3971983030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0100cdcf0 0x7fc0100aba30 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.467+0000 7fc015c0d700 1 -- 172.21.15.89:0/3971983030 >> 172.21.15.89:0/3971983030 conn(0x7fc0100694a0 msgr2=0x7fc0100cce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:05.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.467+0000 7fc015c0d700 1 -- 172.21.15.89:0/3971983030 shutdown_connections 2023-10-10T20:44:05.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.467+0000 7fc015c0d700 1 -- 172.21.15.89:0/3971983030 wait complete. 2023-10-10T20:44:05.472 INFO:teuthology.orchestra.run.smithi089.stderr:updated caps for client.0 2023-10-10T20:44:05.480 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:44:05.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.593+0000 7f2431457700 1 Processor -- start 2023-10-10T20:44:05.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.593+0000 7f2431457700 1 -- start start 2023-10-10T20:44:05.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.594+0000 7f2431457700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f242c159420 0x7f242c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:05.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.594+0000 7f2431457700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f242c1597f0 0x7f242c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:05.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.594+0000 7f2431457700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f242c15d230 0x7f242c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:05.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.594+0000 7f2431457700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242c15dbb0 con 0x7f242c159420 2023-10-10T20:44:05.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.594+0000 7f2431457700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242c0b11d0 con 0x7f242c1597f0 2023-10-10T20:44:05.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.594+0000 7f2431457700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242c0b14b0 con 0x7f242c15d230 2023-10-10T20:44:05.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.594+0000 7f242bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f242c159420 0x7f242c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:05.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.594+0000 7f242bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f242c159420 0x7f242c0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52056/0 (socket says 172.21.15.89:52056) 2023-10-10T20:44:05.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.594+0000 7f242bfff700 1 -- 172.21.15.89:0/344470274 learned_addr learned my addr 172.21.15.89:0/344470274 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:05.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.594+0000 7f242b7fe700 1 --2- 172.21.15.89:0/344470274 >> v2:172.21.15.111:3300/0 conn(0x7f242c1597f0 0x7f242c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:05.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.594+0000 7f2430c56700 1 --2- 172.21.15.89:0/344470274 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f242c15d230 0x7f242c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:05.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.594+0000 7f242bfff700 1 -- 172.21.15.89:0/344470274 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f242c15d230 msgr2=0x7f242c15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:05.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.594+0000 7f242bfff700 1 --2- 172.21.15.89:0/344470274 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f242c15d230 0x7f242c15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.594+0000 7f242bfff700 1 -- 172.21.15.89:0/344470274 >> v2:172.21.15.111:3300/0 conn(0x7f242c1597f0 msgr2=0x7f242c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:05.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.594+0000 7f242bfff700 1 --2- 172.21.15.89:0/344470274 >> v2:172.21.15.111:3300/0 conn(0x7f242c1597f0 0x7f242c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.594+0000 7f242bfff700 1 -- 172.21.15.89:0/344470274 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c0b1790 con 0x7f242c159420 2023-10-10T20:44:05.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.595+0000 7f242bfff700 1 --2- 172.21.15.89:0/344470274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f242c159420 0x7f242c0b5f40 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f242c089750 tx=0x7f241c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf60e40de6dbce6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:05.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.595+0000 7f242affd700 1 -- 172.21.15.89:0/344470274 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c00b860 con 0x7f242c159420 2023-10-10T20:44:05.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.595+0000 7f242affd700 1 -- 172.21.15.89:0/344470274 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c00b9c0 con 0x7f242c159420 2023-10-10T20:44:05.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.595+0000 7f242affd700 1 -- 172.21.15.89:0/344470274 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c00bd60 con 0x7f242c159420 2023-10-10T20:44:05.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.595+0000 7f2431457700 1 -- 172.21.15.89:0/344470274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f242c159420 msgr2=0x7f242c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:05.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.595+0000 7f2431457700 1 --2- 172.21.15.89:0/344470274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f242c159420 0x7f242c0b5f40 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f242c089750 tx=0x7f241c009250 comp rx=0 tx=0).stop 2023-10-10T20:44:05.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.595+0000 7f2431457700 1 -- 172.21.15.89:0/344470274 shutdown_connections 2023-10-10T20:44:05.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.595+0000 7f2431457700 1 --2- 172.21.15.89:0/344470274 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f242c15d230 0x7f242c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.595+0000 7f2431457700 1 --2- 172.21.15.89:0/344470274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f242c159420 0x7f242c0b5f40 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.595+0000 7f2431457700 1 --2- 172.21.15.89:0/344470274 >> v2:172.21.15.111:3300/0 conn(0x7f242c1597f0 0x7f242c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.595+0000 7f2431457700 1 -- 172.21.15.89:0/344470274 >> 172.21.15.89:0/344470274 conn(0x7f242c0d0660 msgr2=0x7f242c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:05.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.595+0000 7f2431457700 1 -- 172.21.15.89:0/344470274 shutdown_connections 2023-10-10T20:44:05.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.595+0000 7f2431457700 1 -- 172.21.15.89:0/344470274 wait complete. 2023-10-10T20:44:05.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.596+0000 7f2431457700 1 Processor -- start 2023-10-10T20:44:05.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.597+0000 7f2431457700 1 -- start start 2023-10-10T20:44:05.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.597+0000 7f2431457700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f242c159420 0x7f242c141120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:05.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.597+0000 7f2431457700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f242c1597f0 0x7f242c15eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:05.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.597+0000 7f2431457700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f242c15d230 0x7f242c1418c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:05.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.597+0000 7f2431457700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c141660 con 0x7f242c1597f0 2023-10-10T20:44:05.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.597+0000 7f2431457700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c15f030 con 0x7f242c159420 2023-10-10T20:44:05.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.597+0000 7f2431457700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242c15c090 con 0x7f242c15d230 2023-10-10T20:44:05.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.597+0000 7f242b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f242c1597f0 0x7f242c15eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:05.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.597+0000 7f242b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f242c1597f0 0x7f242c15eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52072/0 (socket says 172.21.15.89:52072) 2023-10-10T20:44:05.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.597+0000 7f242b7fe700 1 -- 172.21.15.89:0/3852385668 learned_addr learned my addr 172.21.15.89:0/3852385668 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:05.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.598+0000 7f2430c56700 1 --2- 172.21.15.89:0/3852385668 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f242c15d230 0x7f242c1418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:05.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.598+0000 7f242bfff700 1 --2- 172.21.15.89:0/3852385668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f242c159420 0x7f242c141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:05.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.598+0000 7f242b7fe700 1 -- 172.21.15.89:0/3852385668 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f242c15d230 msgr2=0x7f242c1418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:05.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.598+0000 7f242b7fe700 1 --2- 172.21.15.89:0/3852385668 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f242c15d230 0x7f242c1418c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.598+0000 7f242b7fe700 1 -- 172.21.15.89:0/3852385668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f242c159420 msgr2=0x7f242c141120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:05.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.598+0000 7f242b7fe700 1 --2- 172.21.15.89:0/3852385668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f242c159420 0x7f242c141120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.598+0000 7f242b7fe700 1 -- 172.21.15.89:0/3852385668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c0094f0 con 0x7f242c1597f0 2023-10-10T20:44:05.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.598+0000 7f242b7fe700 1 --2- 172.21.15.89:0/3852385668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f242c1597f0 0x7f242c15eaf0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f242000d860 tx=0x7f2420008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:05.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.598+0000 7f2413fff700 1 -- 172.21.15.89:0/3852385668 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24200125f0 con 0x7f242c1597f0 2023-10-10T20:44:05.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.598+0000 7f2413fff700 1 -- 172.21.15.89:0/3852385668 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2420012750 con 0x7f242c1597f0 2023-10-10T20:44:05.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.598+0000 7f2431457700 1 -- 172.21.15.89:0/3852385668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242c15b480 con 0x7f242c1597f0 2023-10-10T20:44:05.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.598+0000 7f2413fff700 1 -- 172.21.15.89:0/3852385668 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2420012ad0 con 0x7f242c1597f0 2023-10-10T20:44:05.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.598+0000 7f2431457700 1 -- 172.21.15.89:0/3852385668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242c15ba40 con 0x7f242c1597f0 2023-10-10T20:44:05.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.599+0000 7f2431457700 1 -- 172.21.15.89:0/3852385668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242c15c200 con 0x7f242c1597f0 2023-10-10T20:44:05.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.599+0000 7f2413fff700 1 -- 172.21.15.89:0/3852385668 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f242001e460 con 0x7f242c1597f0 2023-10-10T20:44:05.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.600+0000 7f2413fff700 1 --2- 172.21.15.89:0/3852385668 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2418067320 0x7f24180697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:05.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.600+0000 7f2413fff700 1 -- 172.21.15.89:0/3852385668 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2420085300 con 0x7f242c1597f0 2023-10-10T20:44:05.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.600+0000 7f242bfff700 1 --2- 172.21.15.89:0/3852385668 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2418067320 0x7f24180697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:05.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.600+0000 7f242bfff700 1 --2- 172.21.15.89:0/3852385668 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2418067320 0x7f24180697d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f242c089750 tx=0x7f241c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:05.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.604+0000 7f2413fff700 1 -- 172.21.15.89:0/3852385668 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2420055a20 con 0x7f242c1597f0 2023-10-10T20:44:05.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.829+0000 7f2431457700 1 -- 172.21.15.89:0/3852385668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f242c04c3c0 con 0x7f242c1597f0 2023-10-10T20:44:05.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.829+0000 7f2413fff700 1 -- 172.21.15.89:0/3852385668 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+4604 (secure 0 0 0) 0x7f24200555b0 con 0x7f242c1597f0 2023-10-10T20:44:05.831 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:44:05.832 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":132,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/4215135154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":4215135154},{"type":"v1","addr":"172.21.15.89:6837","nonce":4215135154}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127},{"gid":7088,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/940990808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":940990808},{"type":"v1","addr":"172.21.15.111:6834","nonce":940990808}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127},{"gid":7091,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/92922277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":92922277},{"type":"v1","addr":"172.21.15.111:6835","nonce":92922277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:43:59.950797+0000","modified":"2023-10-10T20:44:03.611977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9135":{"gid":9135,"name":"a","rank":0,"incarnation":128,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6836/3558240892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3558240892},{"type":"v1","addr":"172.21.15.89:6836","nonce":3558240892}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-10T20:44:05.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.831+0000 7f2431457700 1 -- 172.21.15.89:0/3852385668 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2418067320 msgr2=0x7f24180697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:05.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.831+0000 7f2431457700 1 --2- 172.21.15.89:0/3852385668 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2418067320 0x7f24180697d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f242c089750 tx=0x7f241c017000 comp rx=0 tx=0).stop 2023-10-10T20:44:05.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.831+0000 7f2431457700 1 -- 172.21.15.89:0/3852385668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f242c1597f0 msgr2=0x7f242c15eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:05.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.831+0000 7f2431457700 1 --2- 172.21.15.89:0/3852385668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f242c1597f0 0x7f242c15eaf0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f242000d860 tx=0x7f2420008040 comp rx=0 tx=0).stop 2023-10-10T20:44:05.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.831+0000 7f2431457700 1 -- 172.21.15.89:0/3852385668 shutdown_connections 2023-10-10T20:44:05.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.831+0000 7f2431457700 1 --2- 172.21.15.89:0/3852385668 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f242c15d230 0x7f242c1418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.831+0000 7f2431457700 1 --2- 172.21.15.89:0/3852385668 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2418067320 0x7f24180697d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.831+0000 7f2431457700 1 --2- 172.21.15.89:0/3852385668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f242c159420 0x7f242c141120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.831+0000 7f2431457700 1 --2- 172.21.15.89:0/3852385668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f242c1597f0 0x7f242c15eaf0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:05.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.831+0000 7f2431457700 1 -- 172.21.15.89:0/3852385668 >> 172.21.15.89:0/3852385668 conn(0x7f242c0d0660 msgr2=0x7f242c0cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:05.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.831+0000 7f2431457700 1 -- 172.21.15.89:0/3852385668 shutdown_connections 2023-10-10T20:44:05.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:05.832+0000 7f2431457700 1 -- 172.21.15.89:0/3852385668 wait complete. 2023-10-10T20:44:05.836 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 132 2023-10-10T20:44:05.867 INFO:tasks.ceph.mds.a.smithi089.stderr:2023-10-10T20:44:05.865+0000 7eff21e1f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:44:06.249 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 132, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:43:59.950797+0000', 'modified': '2023-10-10T20:44:03.611977+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9135}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9135': {'gid': 9135, 'name': 'a', 'rank': 0, 'incarnation': 128, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6836/3558240892', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6834', 'nonce': 3558240892}, {'type': 'v1', 'addr': '172.21.15.89:6836', 'nonce': 3558240892}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:44:06.250 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:44:06.250 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-10T20:44:06.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.369+0000 7f44c5f0e700 1 Processor -- start 2023-10-10T20:44:06.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.369+0000 7f44c5f0e700 1 -- start start 2023-10-10T20:44:06.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.370+0000 7f44c5f0e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f44c0159420 0x7f44c00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.370+0000 7f44c5f0e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f44c015d230 0x7f44c015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.370+0000 7f44c5f0e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44c015dbd0 con 0x7f44c01597f0 2023-10-10T20:44:06.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.370+0000 7f44c5f0e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c00b2980 con 0x7f44c015d230 2023-10-10T20:44:06.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.370+0000 7f44c5f0e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c00b11d0 con 0x7f44c0159420 2023-10-10T20:44:06.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.370+0000 7f44c570d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f44c015d230 0x7f44c015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.370+0000 7f44bffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f44c01597f0 0x7f44c00b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37236/0 (socket says 172.21.15.89:37236) 2023-10-10T20:44:06.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.370+0000 7f44bffff700 1 -- 172.21.15.89:0/1598994288 learned_addr learned my addr 172.21.15.89:0/1598994288 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:44:06.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.370+0000 7f44c4f0c700 1 --2- 172.21.15.89:0/1598994288 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f44c0159420 0x7f44c00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.370+0000 7f44c570d700 1 -- 172.21.15.89:0/1598994288 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f44c0159420 msgr2=0x7f44c00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:06.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.370+0000 7f44c570d700 1 --2- 172.21.15.89:0/1598994288 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f44c0159420 0x7f44c00b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.370+0000 7f44c570d700 1 -- 172.21.15.89:0/1598994288 >> v1:172.21.15.89:6789/0 conn(0x7f44c01597f0 legacy=0x7f44c00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.371+0000 7f44c570d700 1 -- 172.21.15.89:0/1598994288 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c00b14b0 con 0x7f44c015d230 2023-10-10T20:44:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.370+0000 7f44bf7fe700 1 -- 172.21.15.89:0/1598994288 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795403218 0 0) 0x7f44ac001110 con 0x7f44c01597f0 2023-10-10T20:44:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.371+0000 7f44c570d700 1 --2- 172.21.15.89:0/1598994288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44c015d230 0x7f44c015d690 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009e60 tx=0x7f44b8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fbeb999063f9cc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.371+0000 7f44bf7fe700 1 -- 172.21.15.89:0/1598994288 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b800e070 con 0x7f44c015d230 2023-10-10T20:44:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.371+0000 7f44bf7fe700 1 -- 172.21.15.89:0/1598994288 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b800bc30 con 0x7f44c015d230 2023-10-10T20:44:06.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.371+0000 7f44bf7fe700 1 -- 172.21.15.89:0/1598994288 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b8012430 con 0x7f44c015d230 2023-10-10T20:44:06.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.371+0000 7f44c5f0e700 1 -- 172.21.15.89:0/1598994288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44c015d230 msgr2=0x7f44c015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:06.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.371+0000 7f44c5f0e700 1 --2- 172.21.15.89:0/1598994288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44c015d230 0x7f44c015d690 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009e60 tx=0x7f44b8009780 comp rx=0 tx=0).stop 2023-10-10T20:44:06.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.371+0000 7f44c5f0e700 1 -- 172.21.15.89:0/1598994288 shutdown_connections 2023-10-10T20:44:06.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.371+0000 7f44c5f0e700 1 --2- 172.21.15.89:0/1598994288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44c015d230 0x7f44c015d690 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.371+0000 7f44c5f0e700 1 --2- 172.21.15.89:0/1598994288 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f44c0159420 0x7f44c00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.371+0000 7f44c5f0e700 1 -- 172.21.15.89:0/1598994288 >> 172.21.15.89:0/1598994288 conn(0x7f44c00d0660 msgr2=0x7f44c00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:06.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.371+0000 7f44c5f0e700 1 -- 172.21.15.89:0/1598994288 shutdown_connections 2023-10-10T20:44:06.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.371+0000 7f44c5f0e700 1 -- 172.21.15.89:0/1598994288 wait complete. 2023-10-10T20:44:06.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.372+0000 7f44c5f0e700 1 Processor -- start 2023-10-10T20:44:06.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.373+0000 7f44c5f0e700 1 -- start start 2023-10-10T20:44:06.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.373+0000 7f44c5f0e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f44c0159420 0x7f44c01697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.373+0000 7f44c5f0e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44c01597f0 0x7f44c015a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.373+0000 7f44c5f0e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44c015d230 0x7f44c015adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.373+0000 7f44c5f0e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c0153cb0 con 0x7f44c015d230 2023-10-10T20:44:06.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.373+0000 7f44c5f0e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c015a3e0 con 0x7f44c01597f0 2023-10-10T20:44:06.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.373+0000 7f44c5f0e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c015c1f0 con 0x7f44c0159420 2023-10-10T20:44:06.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.373+0000 7f44c570d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44c015d230 0x7f44c015adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.373+0000 7f44c570d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44c015d230 0x7f44c015adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52094/0 (socket says 172.21.15.89:52094) 2023-10-10T20:44:06.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.373+0000 7f44c570d700 1 -- 172.21.15.89:0/4043755735 learned_addr learned my addr 172.21.15.89:0/4043755735 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:06.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.374+0000 7f44c4f0c700 1 --2- 172.21.15.89:0/4043755735 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f44c0159420 0x7f44c01697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.374+0000 7f44c570d700 1 -- 172.21.15.89:0/4043755735 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f44c0159420 msgr2=0x7f44c01697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:06.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.374+0000 7f44bffff700 1 --2- 172.21.15.89:0/4043755735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44c01597f0 0x7f44c015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.374+0000 7f44c570d700 1 --2- 172.21.15.89:0/4043755735 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f44c0159420 0x7f44c01697d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.374+0000 7f44c570d700 1 -- 172.21.15.89:0/4043755735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44c01597f0 msgr2=0x7f44c015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:06.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.374+0000 7f44c570d700 1 --2- 172.21.15.89:0/4043755735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44c01597f0 0x7f44c015a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.374+0000 7f44c570d700 1 -- 172.21.15.89:0/4043755735 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b8009a20 con 0x7f44c015d230 2023-10-10T20:44:06.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.374+0000 7f44c570d700 1 --2- 172.21.15.89:0/4043755735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44c015d230 0x7f44c015adb0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f44b8013000 tx=0x7f44b8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:06.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.374+0000 7f44bcff9700 1 -- 172.21.15.89:0/4043755735 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b8012e50 con 0x7f44c015d230 2023-10-10T20:44:06.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.374+0000 7f44bcff9700 1 -- 172.21.15.89:0/4043755735 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b801d630 con 0x7f44c015d230 2023-10-10T20:44:06.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.374+0000 7f44c5f0e700 1 -- 172.21.15.89:0/4043755735 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c015bf10 con 0x7f44c015d230 2023-10-10T20:44:06.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.374+0000 7f44bcff9700 1 -- 172.21.15.89:0/4043755735 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b801da10 con 0x7f44c015d230 2023-10-10T20:44:06.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.374+0000 7f44c5f0e700 1 -- 172.21.15.89:0/4043755735 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c015a070 con 0x7f44c015d230 2023-10-10T20:44:06.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.375+0000 7f44bcff9700 1 -- 172.21.15.89:0/4043755735 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f44b8024070 con 0x7f44c015d230 2023-10-10T20:44:06.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.376+0000 7f44bcff9700 1 --2- 172.21.15.89:0/4043755735 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f44a4067320 0x7f44a40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.376+0000 7f44bcff9700 1 -- 172.21.15.89:0/4043755735 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f44b80874b0 con 0x7f44c015d230 2023-10-10T20:44:06.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.376+0000 7f44c4f0c700 1 --2- 172.21.15.89:0/4043755735 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f44a4067320 0x7f44a40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.376+0000 7f44c4f0c700 1 --2- 172.21.15.89:0/4043755735 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f44a4067320 0x7f44a40697d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f44b4006fd0 tx=0x7f44b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:06.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.378+0000 7f44d0469b80 1 Processor -- start 2023-10-10T20:44:06.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.378+0000 7f44d0469b80 1 -- start start 2023-10-10T20:44:06.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44d0469b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd504480 0x55e3fd4e37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44d0469b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd4b75f0 0x55e3fd504850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44d0469b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b9b80 0x55e3fd4b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44d0469b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd410a70 con 0x55e3fd4b9b80 2023-10-10T20:44:06.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44d0469b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd40e270 con 0x55e3fd504480 2023-10-10T20:44:06.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44d0469b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd40a060 con 0x55e3fd4b75f0 2023-10-10T20:44:06.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44c570d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b9b80 0x55e3fd4b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44c570d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b9b80 0x55e3fd4b9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52098/0 (socket says 172.21.15.89:52098) 2023-10-10T20:44:06.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44c570d700 1 -- 172.21.15.89:0/81189667 learned_addr learned my addr 172.21.15.89:0/81189667 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:06.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44bffff700 1 --2- 172.21.15.89:0/81189667 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd4b75f0 0x55e3fd504850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44c570d700 1 -- 172.21.15.89:0/81189667 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd4b75f0 msgr2=0x55e3fd504850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:06.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44c570d700 1 --2- 172.21.15.89:0/81189667 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd4b75f0 0x55e3fd504850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44c4f0c700 1 --2- 172.21.15.89:0/81189667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd504480 0x55e3fd4e37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44c570d700 1 -- 172.21.15.89:0/81189667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd504480 msgr2=0x55e3fd4e37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:06.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44c570d700 1 --2- 172.21.15.89:0/81189667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd504480 0x55e3fd4e37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44c570d700 1 -- 172.21.15.89:0/81189667 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3fd2a4270 con 0x55e3fd4b9b80 2023-10-10T20:44:06.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44bffff700 1 --2- 172.21.15.89:0/81189667 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd4b75f0 0x55e3fd504850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:44:06.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44c570d700 1 --2- 172.21.15.89:0/81189667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b9b80 0x55e3fd4b9ff0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f44b8057b00 tx=0x7f44b8020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8df805b6f4d4b7c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:06.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44a9ffb700 1 -- 172.21.15.89:0/81189667 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b800e040 con 0x55e3fd4b9b80 2023-10-10T20:44:06.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44a9ffb700 1 -- 172.21.15.89:0/81189667 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b805b050 con 0x55e3fd4b9b80 2023-10-10T20:44:06.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.379+0000 7f44a9ffb700 1 -- 172.21.15.89:0/81189667 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b805eda0 con 0x55e3fd4b9b80 2023-10-10T20:44:06.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.380+0000 7f44d0469b80 1 -- 172.21.15.89:0/81189667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b9b80 msgr2=0x55e3fd4b9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:06.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.380+0000 7f44d0469b80 1 --2- 172.21.15.89:0/81189667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b9b80 0x55e3fd4b9ff0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f44b8057b00 tx=0x7f44b8020040 comp rx=0 tx=0).stop 2023-10-10T20:44:06.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.380+0000 7f44d0469b80 1 -- 172.21.15.89:0/81189667 shutdown_connections 2023-10-10T20:44:06.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.380+0000 7f44d0469b80 1 --2- 172.21.15.89:0/81189667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b9b80 0x55e3fd4b9ff0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.380+0000 7f44d0469b80 1 --2- 172.21.15.89:0/81189667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd504480 0x55e3fd4e37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.380+0000 7f44d0469b80 1 --2- 172.21.15.89:0/81189667 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd4b75f0 0x55e3fd504850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.380+0000 7f44d0469b80 1 -- 172.21.15.89:0/81189667 >> 172.21.15.89:0/81189667 conn(0x55e3fd4bb0c0 msgr2=0x55e3fd4bb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:06.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.380+0000 7f44d0469b80 1 -- 172.21.15.89:0/81189667 shutdown_connections 2023-10-10T20:44:06.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.380+0000 7f44d0469b80 1 -- 172.21.15.89:0/81189667 wait complete. 2023-10-10T20:44:06.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.380+0000 7f44d0469b80 1 -- start start 2023-10-10T20:44:06.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.381+0000 7f44d0469b80 1 Processor -- start 2023-10-10T20:44:06.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.381+0000 7f44d0469b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b75f0 0x55e3fd4e37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.381+0000 7f44d0469b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd4b9b80 0x55e3fd50fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.381+0000 7f44d0469b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd504480 0x55e3fd4db810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.381+0000 7f44d0469b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd3b5890 con 0x55e3fd4b75f0 2023-10-10T20:44:06.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.381+0000 7f44d0469b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd3d0ea0 con 0x55e3fd504480 2023-10-10T20:44:06.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.381+0000 7f44d0469b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd28ce30 con 0x55e3fd4b9b80 2023-10-10T20:44:06.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44d0469b80 10 client.0 _ll_register_callbacks cb 0x55e3fd47d2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:44:06.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44c4f0c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b75f0 0x55e3fd4e37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44c4f0c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b75f0 0x55e3fd4e37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52106/0 (socket says 172.21.15.89:52106) 2023-10-10T20:44:06.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44c4f0c700 1 -- 172.21.15.89:0/3633530650 learned_addr learned my addr 172.21.15.89:0/3633530650 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:06.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44c570d700 1 --2- 172.21.15.89:0/3633530650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd504480 0x55e3fd4db810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44bffff700 1 --2- 172.21.15.89:0/3633530650 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd4b9b80 0x55e3fd50fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44c4f0c700 1 -- 172.21.15.89:0/3633530650 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd4b9b80 msgr2=0x55e3fd50fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:06.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44c4f0c700 1 --2- 172.21.15.89:0/3633530650 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd4b9b80 0x55e3fd50fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44c4f0c700 1 -- 172.21.15.89:0/3633530650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd504480 msgr2=0x55e3fd4db810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:06.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44c4f0c700 1 --2- 172.21.15.89:0/3633530650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd504480 0x55e3fd4db810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44c4f0c700 1 -- 172.21.15.89:0/3633530650 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c015a070 con 0x55e3fd4b75f0 2023-10-10T20:44:06.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44c570d700 1 --2- 172.21.15.89:0/3633530650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd504480 0x55e3fd4db810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:44:06.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44bffff700 1 --2- 172.21.15.89:0/3633530650 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd4b9b80 0x55e3fd50fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:44:06.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44c4f0c700 1 --2- 172.21.15.89:0/3633530650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b75f0 0x55e3fd4e37b0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f44b400eeb0 tx=0x7f44b4017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6be215dfe1f366 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:06.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f449effd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:44:06.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f449effd700 1 -- 172.21.15.89:0/3633530650 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4015dc0 con 0x55e3fd4b75f0 2023-10-10T20:44:06.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f449effd700 1 -- 172.21.15.89:0/3633530650 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f44b4020410 con 0x55e3fd4b75f0 2023-10-10T20:44:06.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44d0469b80 1 -- 172.21.15.89:0/3633530650 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3fd4aecd0 con 0x55e3fd4b75f0 2023-10-10T20:44:06.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f449effd700 1 -- 172.21.15.89:0/3633530650 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44b40207e0 con 0x55e3fd4b75f0 2023-10-10T20:44:06.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44d0469b80 10 client.9276 fetch_fsmap learned FSMap version 132 2023-10-10T20:44:06.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44d0469b80 1 -- 172.21.15.89:0/3633530650 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=132,monmap=2+}) v3 -- 0x55e3fd246fe0 con 0x55e3fd4b75f0 2023-10-10T20:44:06.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f449effd700 1 -- 172.21.15.89:0/3633530650 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e3fd4aecd0 con 0x55e3fd4b75f0 2023-10-10T20:44:06.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f449effd700 1 -- 172.21.15.89:0/3633530650 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 132) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f44b4020940 con 0x55e3fd4b75f0 2023-10-10T20:44:06.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44d0469b80 10 client.9276 fetch_fsmap finished waiting for FSMap version 132 2023-10-10T20:44:06.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44d0469b80 10 client.9276 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-10T20:44:06.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44d0469b80 20 client.9276 populate_metadata read hostname 'smithi089' 2023-10-10T20:44:06.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.382+0000 7f44d0469b80 1 --2- 172.21.15.89:0/3633530650 >> [v2:172.21.15.89:6834/3558240892,v1:172.21.15.89:6836/3558240892] conn(0x55e3fd4bbd90 0x55e3fd4bc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.383+0000 7f44d0469b80 4 client.9276 mds_command: new command op to 9135 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:44:06.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.383+0000 7f44d0469b80 1 -- 172.21.15.89:0/3633530650 --> [v2:172.21.15.89:6834/3558240892,v1:172.21.15.89:6836/3558240892] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3fd48baa0 con 0x55e3fd4bbd90 2023-10-10T20:44:06.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.383+0000 7f44bffff700 1 --2- 172.21.15.89:0/3633530650 >> [v2:172.21.15.89:6834/3558240892,v1:172.21.15.89:6836/3558240892] conn(0x55e3fd4bbd90 0x55e3fd4bc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.383+0000 7f449effd700 10 client.9276 ms_handle_connect on v2:172.21.15.89:6834/3558240892 2023-10-10T20:44:06.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.383+0000 7f44bffff700 1 --2- 172.21.15.89:0/3633530650 >> [v2:172.21.15.89:6834/3558240892,v1:172.21.15.89:6836/3558240892] conn(0x55e3fd4bbd90 0x55e3fd4bc160 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d29eda112b852472 server_cookie=45f87c1a61567be2 in_seq=0 out_seq=0 2023-10-10T20:44:06.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.383+0000 7f449effd700 1 -- 172.21.15.89:0/3633530650 <== mds.0 v2:172.21.15.89:6834/3558240892 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55e3fd48baa0 con 0x55e3fd4bbd90 2023-10-10T20:44:06.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.383+0000 7f449effd700 10 client.9276 handle_command_reply: tid=0 2023-10-10T20:44:06.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.383+0000 7f44d0469b80 1 client.9276 shutdown 2023-10-10T20:44:06.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.384+0000 7f44d0469b80 1 -- 172.21.15.89:0/3633530650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b75f0 msgr2=0x55e3fd4e37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:06.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.384+0000 7f44d0469b80 1 --2- 172.21.15.89:0/3633530650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b75f0 0x55e3fd4e37b0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f44b400eeb0 tx=0x7f44b4017040 comp rx=0 tx=0).stop 2023-10-10T20:44:06.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.384+0000 7f44d0469b80 1 -- 172.21.15.89:0/3633530650 shutdown_connections 2023-10-10T20:44:06.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.384+0000 7f44d0469b80 1 --2- 172.21.15.89:0/3633530650 >> [v2:172.21.15.89:6834/3558240892,v1:172.21.15.89:6836/3558240892] conn(0x55e3fd4bbd90 0x55e3fd4bc160 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.384+0000 7f44d0469b80 1 --2- 172.21.15.89:0/3633530650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd504480 0x55e3fd4db810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.384+0000 7f44d0469b80 1 --2- 172.21.15.89:0/3633530650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b75f0 0x55e3fd4e37b0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.384+0000 7f449effd700 0 client.9276 ms_handle_reset on v2:172.21.15.89:6834/3558240892 2023-10-10T20:44:06.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.384+0000 7f44d0469b80 1 --2- 172.21.15.89:0/3633530650 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd4b9b80 0x55e3fd50fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.384+0000 7f44d0469b80 1 -- 172.21.15.89:0/3633530650 >> 172.21.15.89:0/3633530650 conn(0x55e3fd4bb0c0 msgr2=0x55e3fd4bd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:06.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.384+0000 7f44d0469b80 1 -- 172.21.15.89:0/3633530650 shutdown_connections 2023-10-10T20:44:06.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.384+0000 7f44d0469b80 1 -- 172.21.15.89:0/3633530650 wait complete. 2023-10-10T20:44:06.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.384+0000 7f44d0469b80 20 client.9276 trim_cache size 0 max 16384 2023-10-10T20:44:06.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.401+0000 7f44d0469b80 1 Processor -- start 2023-10-10T20:44:06.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.401+0000 7f44d0469b80 1 -- start start 2023-10-10T20:44:06.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.401+0000 7f44d0469b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b75f0 0x55e3fd503c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.401+0000 7f44d0469b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd4b9b80 0x55e3fd504b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.401+0000 7f44d0469b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd504480 0x55e3fd4e4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.401+0000 7f44d0469b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd499510 con 0x55e3fd4b75f0 2023-10-10T20:44:06.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.401+0000 7f44d0469b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd4b80c0 con 0x55e3fd4b9b80 2023-10-10T20:44:06.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.401+0000 7f44d0469b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd4b6700 con 0x55e3fd504480 2023-10-10T20:44:06.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.402+0000 7f44c4f0c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b75f0 0x55e3fd503c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.402+0000 7f44c4f0c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b75f0 0x55e3fd503c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52112/0 (socket says 172.21.15.89:52112) 2023-10-10T20:44:06.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.402+0000 7f44c4f0c700 1 -- 172.21.15.89:0/974820002 learned_addr learned my addr 172.21.15.89:0/974820002 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:06.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.402+0000 7f44c570d700 1 --2- 172.21.15.89:0/974820002 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd504480 0x55e3fd4e4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.402+0000 7f44bffff700 1 --2- 172.21.15.89:0/974820002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd4b9b80 0x55e3fd504b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.402+0000 7f44c4f0c700 1 -- 172.21.15.89:0/974820002 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd504480 msgr2=0x55e3fd4e4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:06.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.402+0000 7f44c4f0c700 1 --2- 172.21.15.89:0/974820002 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd504480 0x55e3fd4e4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.402+0000 7f44c4f0c700 1 -- 172.21.15.89:0/974820002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd4b9b80 msgr2=0x55e3fd504b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:06.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.402+0000 7f44c4f0c700 1 --2- 172.21.15.89:0/974820002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd4b9b80 0x55e3fd504b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.402+0000 7f44c4f0c700 1 -- 172.21.15.89:0/974820002 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3fd3d8780 con 0x55e3fd4b75f0 2023-10-10T20:44:06.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.402+0000 7f44c4f0c700 1 --2- 172.21.15.89:0/974820002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b75f0 0x55e3fd503c60 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f44b40090f0 tx=0x7f44b4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208cdf4cfefd97f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:06.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.402+0000 7f44aa7fc700 1 -- 172.21.15.89:0/974820002 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4020410 con 0x55e3fd4b75f0 2023-10-10T20:44:06.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.402+0000 7f44aa7fc700 1 -- 172.21.15.89:0/974820002 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b401b050 con 0x55e3fd4b75f0 2023-10-10T20:44:06.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.402+0000 7f44aa7fc700 1 -- 172.21.15.89:0/974820002 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b40207e0 con 0x55e3fd4b75f0 2023-10-10T20:44:06.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.403+0000 7f44d0469b80 1 -- 172.21.15.89:0/974820002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b75f0 msgr2=0x55e3fd503c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:06.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.403+0000 7f44d0469b80 1 --2- 172.21.15.89:0/974820002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b75f0 0x55e3fd503c60 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f44b40090f0 tx=0x7f44b4019000 comp rx=0 tx=0).stop 2023-10-10T20:44:06.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.403+0000 7f44d0469b80 1 -- 172.21.15.89:0/974820002 shutdown_connections 2023-10-10T20:44:06.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.403+0000 7f44d0469b80 1 --2- 172.21.15.89:0/974820002 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd504480 0x55e3fd4e4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.403+0000 7f44d0469b80 1 --2- 172.21.15.89:0/974820002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b75f0 0x55e3fd503c60 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.403+0000 7f44d0469b80 1 --2- 172.21.15.89:0/974820002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd4b9b80 0x55e3fd504b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.403+0000 7f44d0469b80 1 -- 172.21.15.89:0/974820002 >> 172.21.15.89:0/974820002 conn(0x55e3fd4bb0c0 msgr2=0x55e3fd4bc160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:06.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.403+0000 7f44d0469b80 1 -- 172.21.15.89:0/974820002 shutdown_connections 2023-10-10T20:44:06.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.403+0000 7f44d0469b80 1 -- 172.21.15.89:0/974820002 wait complete. 2023-10-10T20:44:06.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.403+0000 7f44d0469b80 1 -- start start 2023-10-10T20:44:06.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.403+0000 7f44d0469b80 1 Processor -- start 2023-10-10T20:44:06.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.404+0000 7f44d0469b80 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd4b75f0 0x55e3fd565650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.404+0000 7f44d0469b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b9b80 0x55e3fd517dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.404+0000 7f44d0469b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd504480 0x55e3fd4aff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.404+0000 7f44d0469b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd519240 con 0x55e3fd4b9b80 2023-10-10T20:44:06.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.404+0000 7f44d0469b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd4b0a10 con 0x55e3fd504480 2023-10-10T20:44:06.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.404+0000 7f44d0469b80 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd515510 con 0x55e3fd4b75f0 2023-10-10T20:44:06.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.404+0000 7f44d0469b80 10 client.0 _ll_register_callbacks cb 0x55e3fd47d2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:44:06.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.404+0000 7f44bffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b9b80 0x55e3fd517dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.404+0000 7f44bffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b9b80 0x55e3fd517dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52114/0 (socket says 172.21.15.89:52114) 2023-10-10T20:44:06.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.404+0000 7f44bffff700 1 -- 172.21.15.89:0/3250627084 learned_addr learned my addr 172.21.15.89:0/3250627084 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:06.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.404+0000 7f44c570d700 1 --2- 172.21.15.89:0/3250627084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd504480 0x55e3fd4aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.404+0000 7f44c4f0c700 1 --2- 172.21.15.89:0/3250627084 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd4b75f0 0x55e3fd565650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.404+0000 7f44bffff700 1 -- 172.21.15.89:0/3250627084 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd4b75f0 msgr2=0x55e3fd565650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:06.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.404+0000 7f44bffff700 1 --2- 172.21.15.89:0/3250627084 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd4b75f0 0x55e3fd565650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.404+0000 7f44bffff700 1 -- 172.21.15.89:0/3250627084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd504480 msgr2=0x55e3fd4aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:06.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.404+0000 7f44bffff700 1 --2- 172.21.15.89:0/3250627084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd504480 0x55e3fd4aff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.405+0000 7f44bffff700 1 -- 172.21.15.89:0/3250627084 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3fd246fe0 con 0x55e3fd4b9b80 2023-10-10T20:44:06.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.405+0000 7f44bffff700 1 --2- 172.21.15.89:0/3250627084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b9b80 0x55e3fd517dc0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f44c0153f90 tx=0x7f44ac007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4732456b4e10ab8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:06.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.405+0000 7f449effd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-10T20:44:06.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.405+0000 7f449effd700 1 -- 172.21.15.89:0/3250627084 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac0072c0 con 0x55e3fd4b9b80 2023-10-10T20:44:06.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.405+0000 7f449effd700 1 -- 172.21.15.89:0/3250627084 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f44ac00c040 con 0x55e3fd4b9b80 2023-10-10T20:44:06.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.405+0000 7f44d0469b80 1 -- 172.21.15.89:0/3250627084 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44b805b050 con 0x55e3fd4b9b80 2023-10-10T20:44:06.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.405+0000 7f449effd700 1 -- 172.21.15.89:0/3250627084 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44ac007630 con 0x55e3fd4b9b80 2023-10-10T20:44:06.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.405+0000 7f44d0469b80 10 client.9282 fetch_fsmap learned FSMap version 132 2023-10-10T20:44:06.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.405+0000 7f44d0469b80 1 -- 172.21.15.89:0/3250627084 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=132,monmap=2+}) v3 -- 0x55e3fd3bf470 con 0x55e3fd4b9b80 2023-10-10T20:44:06.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.405+0000 7f449effd700 1 -- 172.21.15.89:0/3250627084 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b805b050 con 0x55e3fd4b9b80 2023-10-10T20:44:06.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.405+0000 7f449effd700 1 -- 172.21.15.89:0/3250627084 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 132) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f44ac0175a0 con 0x55e3fd4b9b80 2023-10-10T20:44:06.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.405+0000 7f44d0469b80 10 client.9282 fetch_fsmap finished waiting for FSMap version 132 2023-10-10T20:44:06.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.405+0000 7f44d0469b80 10 client.9282 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-10T20:44:06.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.405+0000 7f44d0469b80 20 client.9282 populate_metadata read hostname 'smithi089' 2023-10-10T20:44:06.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.405+0000 7f44d0469b80 1 --2- 172.21.15.89:0/3250627084 >> [v2:172.21.15.89:6834/3558240892,v1:172.21.15.89:6836/3558240892] conn(0x55e3fd4e8580 0x55e3fd4e8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.405+0000 7f44d0469b80 4 client.9282 mds_command: new command op to 9135 tid=0[{"prefix": "status"}] 2023-10-10T20:44:06.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.405+0000 7f44d0469b80 1 -- 172.21.15.89:0/3250627084 --> [v2:172.21.15.89:6834/3558240892,v1:172.21.15.89:6836/3558240892] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e3fd3ace00 con 0x55e3fd4e8580 2023-10-10T20:44:06.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.405+0000 7f44c4f0c700 1 --2- 172.21.15.89:0/3250627084 >> [v2:172.21.15.89:6834/3558240892,v1:172.21.15.89:6836/3558240892] conn(0x55e3fd4e8580 0x55e3fd4e8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.406+0000 7f449effd700 10 client.9282 ms_handle_connect on v2:172.21.15.89:6834/3558240892 2023-10-10T20:44:06.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.406+0000 7f44c4f0c700 1 --2- 172.21.15.89:0/3250627084 >> [v2:172.21.15.89:6834/3558240892,v1:172.21.15.89:6836/3558240892] conn(0x55e3fd4e8580 0x55e3fd4e8950 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6cc0deec2201cc8 server_cookie=7f8a3993196cea35 in_seq=0 out_seq=0 2023-10-10T20:44:06.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.406+0000 7f449effd700 1 -- 172.21.15.89:0/3250627084 <== mds.0 v2:172.21.15.89:6834/3558240892 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55e3fd3ace00 con 0x55e3fd4e8580 2023-10-10T20:44:06.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.406+0000 7f449effd700 10 client.9282 handle_command_reply: tid=0 2023-10-10T20:44:06.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.406+0000 7f44d0469b80 1 client.9282 shutdown 2023-10-10T20:44:06.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.406+0000 7f44d0469b80 1 -- 172.21.15.89:0/3250627084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b9b80 msgr2=0x55e3fd517dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:06.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.406+0000 7f44d0469b80 1 --2- 172.21.15.89:0/3250627084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b9b80 0x55e3fd517dc0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f44c0153f90 tx=0x7f44ac007040 comp rx=0 tx=0).stop 2023-10-10T20:44:06.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.406+0000 7f44d0469b80 1 -- 172.21.15.89:0/3250627084 shutdown_connections 2023-10-10T20:44:06.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.406+0000 7f44d0469b80 1 --2- 172.21.15.89:0/3250627084 >> [v2:172.21.15.89:6834/3558240892,v1:172.21.15.89:6836/3558240892] conn(0x55e3fd4e8580 0x55e3fd4e8950 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.406+0000 7f44d0469b80 1 --2- 172.21.15.89:0/3250627084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fd504480 0x55e3fd4aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.406+0000 7f449effd700 0 client.9282 ms_handle_reset on v2:172.21.15.89:6834/3558240892 2023-10-10T20:44:06.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.406+0000 7f44d0469b80 1 --2- 172.21.15.89:0/3250627084 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x55e3fd4b75f0 0x55e3fd565650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.406+0000 7f44d0469b80 1 --2- 172.21.15.89:0/3250627084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fd4b9b80 0x55e3fd517dc0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.406+0000 7f44d0469b80 1 -- 172.21.15.89:0/3250627084 >> 172.21.15.89:0/3250627084 conn(0x55e3fd4bb0c0 msgr2=0x55e3fd4b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:06.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.406+0000 7f44d0469b80 1 -- 172.21.15.89:0/3250627084 shutdown_connections 2023-10-10T20:44:06.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.406+0000 7f44d0469b80 1 -- 172.21.15.89:0/3250627084 wait complete. 2023-10-10T20:44:06.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.406+0000 7f44d0469b80 20 client.9282 trim_cache size 0 max 16384 2023-10-10T20:44:06.448 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:44:06.448 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "5dd675b8-9026-4d8a-a1bf-62fdc5e7de67", 2023-10-10T20:44:06.448 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-10-10T20:44:06.448 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 9135, 2023-10-10T20:44:06.449 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-10-10T20:44:06.449 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-10-10T20:44:06.449 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-10-10T20:44:06.449 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 5.8082095310000001, 2023-10-10T20:44:06.449 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 132, 2023-10-10T20:44:06.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 198, 2023-10-10T20:44:06.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 198, 2023-10-10T20:44:06.450 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 10.953004053000001 2023-10-10T20:44:06.450 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:44:06.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.407+0000 7f44c5f0e700 1 -- 172.21.15.89:0/4043755735 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f44a4067320 msgr2=0x7f44a40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:06.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.407+0000 7f44c5f0e700 1 --2- 172.21.15.89:0/4043755735 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f44a4067320 0x7f44a40697d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f44b4006fd0 tx=0x7f44b4008040 comp rx=0 tx=0).stop 2023-10-10T20:44:06.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.407+0000 7f44c5f0e700 1 -- 172.21.15.89:0/4043755735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44c015d230 msgr2=0x7f44c015adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:06.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.407+0000 7f44c5f0e700 1 --2- 172.21.15.89:0/4043755735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44c015d230 0x7f44c015adb0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f44b8013000 tx=0x7f44b8019000 comp rx=0 tx=0).stop 2023-10-10T20:44:06.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.407+0000 7f44c5f0e700 1 -- 172.21.15.89:0/4043755735 shutdown_connections 2023-10-10T20:44:06.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.407+0000 7f44c5f0e700 1 --2- 172.21.15.89:0/4043755735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44c015d230 0x7f44c015adb0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.407+0000 7f44c5f0e700 1 --2- 172.21.15.89:0/4043755735 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f44c0159420 0x7f44c01697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.407+0000 7f44c5f0e700 1 --2- 172.21.15.89:0/4043755735 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f44a4067320 0x7f44a40697d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.407+0000 7f44c5f0e700 1 --2- 172.21.15.89:0/4043755735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44c01597f0 0x7f44c015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.407+0000 7f44c5f0e700 1 -- 172.21.15.89:0/4043755735 >> 172.21.15.89:0/4043755735 conn(0x7f44c00d0660 msgr2=0x7f44c00d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:06.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.407+0000 7f44c5f0e700 1 -- 172.21.15.89:0/4043755735 shutdown_connections 2023-10-10T20:44:06.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.407+0000 7f44c5f0e700 1 -- 172.21.15.89:0/4043755735 wait complete. 2023-10-10T20:44:06.454 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:44:06.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.567+0000 7fb779e30700 1 Processor -- start 2023-10-10T20:44:06.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.567+0000 7fb779e30700 1 -- start start 2023-10-10T20:44:06.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.568+0000 7fb779e30700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb774159890 0x7fb77415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.568+0000 7fb779e30700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb774069560 0x7fb7740699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.568+0000 7fb779e30700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb77415a160 con 0x7fb774159c60 2023-10-10T20:44:06.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.568+0000 7fb779e30700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb774069f20 con 0x7fb774069560 2023-10-10T20:44:06.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.568+0000 7fb779e30700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7740d13a0 con 0x7fb774159890 2023-10-10T20:44:06.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.568+0000 7fb7737fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb774159890 0x7fb77415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.568+0000 7fb773fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb774069560 0x7fb7740699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.568+0000 7fb7737fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb774159890 0x7fb77415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:54452/0 (socket says 172.21.15.89:54452) 2023-10-10T20:44:06.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.568+0000 7fb7737fe700 1 -- 172.21.15.89:0/3482840791 learned_addr learned my addr 172.21.15.89:0/3482840791 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:06.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.568+0000 7fb772ffd700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb774159c60 0x7fb77415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:36992/0 (socket says 172.21.15.89:36992) 2023-10-10T20:44:06.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.568+0000 7fb773fff700 1 -- 172.21.15.89:0/3482840791 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb774159890 msgr2=0x7fb77415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:06.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.568+0000 7fb773fff700 1 --2- 172.21.15.89:0/3482840791 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb774159890 0x7fb77415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.568+0000 7fb773fff700 1 -- 172.21.15.89:0/3482840791 >> v1:172.21.15.111:6789/0 conn(0x7fb774159c60 legacy=0x7fb77415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:06.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.568+0000 7fb7727fc700 1 -- 172.21.15.89:0/3482840791 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59261327 0 0) 0x7fb77415a160 con 0x7fb774159c60 2023-10-10T20:44:06.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.568+0000 7fb773fff700 1 -- 172.21.15.89:0/3482840791 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77415e140 con 0x7fb774069560 2023-10-10T20:44:06.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.568+0000 7fb7737fe700 1 --2- 172.21.15.89:0/3482840791 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb774159890 0x7fb77415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:44:06.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.569+0000 7fb773fff700 1 --2- 172.21.15.89:0/3482840791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb774069560 0x7fb7740699e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009e60 tx=0x7fb76c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e238f0a4079d3597 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:06.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.569+0000 7fb7727fc700 1 -- 172.21.15.89:0/3482840791 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c00e070 con 0x7fb774069560 2023-10-10T20:44:06.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.569+0000 7fb7727fc700 1 -- 172.21.15.89:0/3482840791 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76c00bb60 con 0x7fb774069560 2023-10-10T20:44:06.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.569+0000 7fb7727fc700 1 -- 172.21.15.89:0/3482840791 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c012430 con 0x7fb774069560 2023-10-10T20:44:06.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.569+0000 7fb779e30700 1 -- 172.21.15.89:0/3482840791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb774069560 msgr2=0x7fb7740699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:06.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.569+0000 7fb779e30700 1 --2- 172.21.15.89:0/3482840791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb774069560 0x7fb7740699e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009e60 tx=0x7fb76c009780 comp rx=0 tx=0).stop 2023-10-10T20:44:06.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.569+0000 7fb779e30700 1 -- 172.21.15.89:0/3482840791 shutdown_connections 2023-10-10T20:44:06.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.569+0000 7fb779e30700 1 --2- 172.21.15.89:0/3482840791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb774069560 0x7fb7740699e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.569+0000 7fb779e30700 1 --2- 172.21.15.89:0/3482840791 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb774159890 0x7fb77415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.569+0000 7fb779e30700 1 -- 172.21.15.89:0/3482840791 >> 172.21.15.89:0/3482840791 conn(0x7fb774068e20 msgr2=0x7fb7740d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:06.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.569+0000 7fb779e30700 1 -- 172.21.15.89:0/3482840791 shutdown_connections 2023-10-10T20:44:06.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.569+0000 7fb779e30700 1 -- 172.21.15.89:0/3482840791 wait complete. 2023-10-10T20:44:06.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.571+0000 7fb779e30700 1 Processor -- start 2023-10-10T20:44:06.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.571+0000 7fb779e30700 1 -- start start 2023-10-10T20:44:06.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.571+0000 7fb779e30700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb774069560 0x7fb7740c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.571+0000 7fb779e30700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb774159890 0x7fb7740cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.571+0000 7fb779e30700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb774159c60 0x7fb7740cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.571+0000 7fb779e30700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7740b3590 con 0x7fb774069560 2023-10-10T20:44:06.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.571+0000 7fb779e30700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7740c6610 con 0x7fb774159c60 2023-10-10T20:44:06.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.571+0000 7fb779e30700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb774151440 con 0x7fb774159890 2023-10-10T20:44:06.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.571+0000 7fb7737fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb774069560 0x7fb7740c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.571+0000 7fb7737fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb774069560 0x7fb7740c5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52118/0 (socket says 172.21.15.89:52118) 2023-10-10T20:44:06.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.571+0000 7fb7737fe700 1 -- 172.21.15.89:0/1799835220 learned_addr learned my addr 172.21.15.89:0/1799835220 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:06.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.572+0000 7fb772ffd700 1 --2- 172.21.15.89:0/1799835220 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb774159890 0x7fb7740cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.572+0000 7fb773fff700 1 --2- 172.21.15.89:0/1799835220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb774159c60 0x7fb7740cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.572+0000 7fb7737fe700 1 -- 172.21.15.89:0/1799835220 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb774159890 msgr2=0x7fb7740cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:06.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.572+0000 7fb7737fe700 1 --2- 172.21.15.89:0/1799835220 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb774159890 0x7fb7740cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.572+0000 7fb7737fe700 1 -- 172.21.15.89:0/1799835220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb774159c60 msgr2=0x7fb7740cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:06.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.572+0000 7fb7737fe700 1 --2- 172.21.15.89:0/1799835220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb774159c60 0x7fb7740cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.572+0000 7fb7737fe700 1 -- 172.21.15.89:0/1799835220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7680094f0 con 0x7fb774069560 2023-10-10T20:44:06.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.572+0000 7fb773fff700 1 --2- 172.21.15.89:0/1799835220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb774159c60 0x7fb7740cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:44:06.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.572+0000 7fb7737fe700 1 --2- 172.21.15.89:0/1799835220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb774069560 0x7fb7740c5590 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fb76800ec80 tx=0x7fb768009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:06.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.572+0000 7fb75b7fe700 1 -- 172.21.15.89:0/1799835220 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76800c920 con 0x7fb774069560 2023-10-10T20:44:06.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.572+0000 7fb75b7fe700 1 -- 172.21.15.89:0/1799835220 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76800ca80 con 0x7fb774069560 2023-10-10T20:44:06.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.572+0000 7fb779e30700 1 -- 172.21.15.89:0/1799835220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb76c009a20 con 0x7fb774069560 2023-10-10T20:44:06.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.572+0000 7fb75b7fe700 1 -- 172.21.15.89:0/1799835220 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76800ce00 con 0x7fb774069560 2023-10-10T20:44:06.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.572+0000 7fb779e30700 1 -- 172.21.15.89:0/1799835220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7740c6330 con 0x7fb774069560 2023-10-10T20:44:06.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.573+0000 7fb779e30700 1 -- 172.21.15.89:0/1799835220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb77404c3c0 con 0x7fb774069560 2023-10-10T20:44:06.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.573+0000 7fb75b7fe700 1 -- 172.21.15.89:0/1799835220 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb768013680 con 0x7fb774069560 2023-10-10T20:44:06.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.574+0000 7fb75b7fe700 1 --2- 172.21.15.89:0/1799835220 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb760067320 0x7fb7600697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:06.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.574+0000 7fb75b7fe700 1 -- 172.21.15.89:0/1799835220 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb768085080 con 0x7fb774069560 2023-10-10T20:44:06.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.574+0000 7fb772ffd700 1 --2- 172.21.15.89:0/1799835220 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb760067320 0x7fb7600697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:06.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.574+0000 7fb772ffd700 1 --2- 172.21.15.89:0/1799835220 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb760067320 0x7fb7600697d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb77416ca80 tx=0x7fb764007ef0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:06.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.578+0000 7fb75b7fe700 1 -- 172.21.15.89:0/1799835220 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb7680b65a0 con 0x7fb774069560 2023-10-10T20:44:06.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.804+0000 7fb779e30700 1 -- 172.21.15.89:0/1799835220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb77406a5b0 con 0x7fb774069560 2023-10-10T20:44:06.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.804+0000 7fb75b7fe700 1 -- 172.21.15.89:0/1799835220 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v132) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb768017070 con 0x7fb774069560 2023-10-10T20:44:06.807 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:44:06.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.806+0000 7fb779e30700 1 -- 172.21.15.89:0/1799835220 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb760067320 msgr2=0x7fb7600697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:06.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.806+0000 7fb779e30700 1 --2- 172.21.15.89:0/1799835220 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb760067320 0x7fb7600697d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb77416ca80 tx=0x7fb764007ef0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.806+0000 7fb779e30700 1 -- 172.21.15.89:0/1799835220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb774069560 msgr2=0x7fb7740c5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:06.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.806+0000 7fb779e30700 1 --2- 172.21.15.89:0/1799835220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb774069560 0x7fb7740c5590 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fb76800ec80 tx=0x7fb768009250 comp rx=0 tx=0).stop 2023-10-10T20:44:06.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.806+0000 7fb779e30700 1 -- 172.21.15.89:0/1799835220 shutdown_connections 2023-10-10T20:44:06.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.806+0000 7fb779e30700 1 --2- 172.21.15.89:0/1799835220 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb760067320 0x7fb7600697d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.806+0000 7fb779e30700 1 --2- 172.21.15.89:0/1799835220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb774159c60 0x7fb7740cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.806+0000 7fb779e30700 1 --2- 172.21.15.89:0/1799835220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb774069560 0x7fb7740c5590 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.806+0000 7fb779e30700 1 --2- 172.21.15.89:0/1799835220 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb774159890 0x7fb7740cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:06.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.806+0000 7fb779e30700 1 -- 172.21.15.89:0/1799835220 >> 172.21.15.89:0/1799835220 conn(0x7fb774068e20 msgr2=0x7fb77415d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:06.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.806+0000 7fb779e30700 1 -- 172.21.15.89:0/1799835220 shutdown_connections 2023-10-10T20:44:06.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:06.806+0000 7fb779e30700 1 -- 172.21.15.89:0/1799835220 wait complete. 2023-10-10T20:44:06.820 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:44:06.821 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:44:06.821 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-10T20:44:06.822 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:44:06.822 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:44:06.823 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:44:06.823 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:44:06.824 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:44:06.824 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:44:06.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:44:06.825 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-10T20:44:06.840 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:44:06.840 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:44:06.841 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:44:06.841 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:44:06.841 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-10T20:44:06.841 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:44:06.841 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:44:06.842 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:44:06.842 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-10T20:44:06.842 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:44:06.842 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:44:06.843 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-10T20:44:06.843 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:44:06.843 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:44:06.843 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:44:06.843 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:44:06.844 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:44:06.844 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 41919sec preferred_lft 41919sec 2023-10-10T20:44:06.844 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2023-10-10T20:44:06.844 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:44:06.844 INFO:teuthology.orchestra.run.smithi089.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:44:06.845 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:44:06.845 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:44:06.845 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:44:06.845 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::98a3:d4ff:fec1:83bb/64 scope link 2023-10-10T20:44:06.846 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:44:06.846 INFO:teuthology.orchestra.run.smithi089.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:44:06.846 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether ee:16:22:10:ca:9a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:44:06.846 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec16:22ff:fe10:ca9a/64 scope link 2023-10-10T20:44:06.846 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:44:06.847 INFO:teuthology.orchestra.run.smithi089.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:44:06.847 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 6a:64:2b:5c:71:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:44:06.847 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::6864:2bff:fe5c:71d4/64 scope link 2023-10-10T20:44:06.847 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:44:06.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:44:06.848 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:44:06.903 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-10T20:44:06.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:44:06.905 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-10T20:44:06.961 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-10T20:44:06.961 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:44:06.961 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:44:06.962 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:44:06.962 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:44:06.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:44:06.963 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-10T20:44:06.963 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-10T20:44:06.963 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:44:06.964 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:44:06.965 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:44:06.966 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-10T20:44:07.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:44:07.061 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-10T20:44:07.076 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:44:07.076 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:44:07.077 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:44:07.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:44:07.077 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-10-10T20:44:07.140 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:44:07.141 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:44:07.196 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:44:07.410 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:44:07.411 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:44:07.429 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:44:07.430 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:44:07.430 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:44:07.507 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:44:07.507 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:44:07.507 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:44:07.515 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:44:07.515 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:44:07.515 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:44:07.516 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:44:07.516 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:44:07.537 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-10T20:44:07.537 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-10T20:44:07.537 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-10T20:44:07.538 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:44:07.538 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:44:07.538 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:44:07.539 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:44:07.539 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:44:07.540 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:44:07.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:44:07.541 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:44:07.612 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:44:07.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.771+0000 7f9535af5700 1 Processor -- start 2023-10-10T20:44:07.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.771+0000 7f9535af5700 1 -- start start 2023-10-10T20:44:07.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.772+0000 7f9535af5700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9530159890 0x7f953015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:07.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.772+0000 7f9535af5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9530159c60 0x7f953015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:07.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.772+0000 7f9535af5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953015a160 con 0x7f953015db10 2023-10-10T20:44:07.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.772+0000 7f9535af5700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95300d13a0 con 0x7f9530159890 2023-10-10T20:44:07.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.772+0000 7f9535af5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f953015e140 con 0x7f9530159c60 2023-10-10T20:44:07.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.772+0000 7f952ffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f953015db10 0x7f9530069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37244/0 (socket says 172.21.15.89:37244) 2023-10-10T20:44:07.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.772+0000 7f952ffff700 1 -- 172.21.15.89:0/378327224 learned_addr learned my addr 172.21.15.89:0/378327224 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:44:07.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.772+0000 7f952effd700 1 --2- 172.21.15.89:0/378327224 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9530159c60 0x7f953015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:07.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.772+0000 7f952f7fe700 1 --2- 172.21.15.89:0/378327224 >> v2:172.21.15.111:3300/0 conn(0x7f9530159890 0x7f953015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:07.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.773+0000 7f952e7fc700 1 -- 172.21.15.89:0/378327224 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343533952 0 0) 0x7f953015a160 con 0x7f953015db10 2023-10-10T20:44:07.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.773+0000 7f952e7fc700 1 -- 172.21.15.89:0/378327224 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9518003570 con 0x7f953015db10 2023-10-10T20:44:07.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.773+0000 7f952effd700 1 -- 172.21.15.89:0/378327224 >> v2:172.21.15.111:3300/0 conn(0x7f9530159890 msgr2=0x7f953015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:07.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.773+0000 7f952effd700 1 --2- 172.21.15.89:0/378327224 >> v2:172.21.15.111:3300/0 conn(0x7f9530159890 0x7f953015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:07.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.773+0000 7f952effd700 1 -- 172.21.15.89:0/378327224 >> v1:172.21.15.89:6789/0 conn(0x7f953015db10 legacy=0x7f9530069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:07.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.773+0000 7f952effd700 1 -- 172.21.15.89:0/378327224 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953015a3d0 con 0x7f9530159c60 2023-10-10T20:44:07.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.773+0000 7f952effd700 1 --2- 172.21.15.89:0/378327224 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9530159c60 0x7f953015d5d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f95240098e0 tx=0x7f95240091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14f0041bdf7bdb39 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:07.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.773+0000 7f952e7fc700 1 -- 172.21.15.89:0/378327224 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524011630 con 0x7f9530159c60 2023-10-10T20:44:07.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.773+0000 7f952e7fc700 1 -- 172.21.15.89:0/378327224 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9524011790 con 0x7f9530159c60 2023-10-10T20:44:07.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.773+0000 7f952e7fc700 1 -- 172.21.15.89:0/378327224 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524011b30 con 0x7f9530159c60 2023-10-10T20:44:07.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.774+0000 7f9535af5700 1 -- 172.21.15.89:0/378327224 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9530159c60 msgr2=0x7f953015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:07.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.774+0000 7f9535af5700 1 --2- 172.21.15.89:0/378327224 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9530159c60 0x7f953015d5d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f95240098e0 tx=0x7f95240091a0 comp rx=0 tx=0).stop 2023-10-10T20:44:07.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.774+0000 7f9535af5700 1 -- 172.21.15.89:0/378327224 shutdown_connections 2023-10-10T20:44:07.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.774+0000 7f9535af5700 1 --2- 172.21.15.89:0/378327224 >> v2:172.21.15.111:3300/0 conn(0x7f9530159890 0x7f953015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:07.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.774+0000 7f9535af5700 1 --2- 172.21.15.89:0/378327224 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f9530159c60 0x7f953015d5d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:07.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.774+0000 7f9535af5700 1 -- 172.21.15.89:0/378327224 >> 172.21.15.89:0/378327224 conn(0x7f9530068e20 msgr2=0x7f95300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:07.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.774+0000 7f9535af5700 1 -- 172.21.15.89:0/378327224 shutdown_connections 2023-10-10T20:44:07.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.774+0000 7f9535af5700 1 -- 172.21.15.89:0/378327224 wait complete. 2023-10-10T20:44:07.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.775+0000 7f9535af5700 1 Processor -- start 2023-10-10T20:44:07.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.775+0000 7f9535af5700 1 -- start start 2023-10-10T20:44:07.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f9535af5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9530159890 0x7f953015bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:07.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f9535af5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9530159c60 0x7f953014a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:07.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f9535af5700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f953015db10 0x7f95300b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:07.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f9535af5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953015bb00 con 0x7f9530159c60 2023-10-10T20:44:07.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f9535af5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953015b4d0 con 0x7f9530159890 2023-10-10T20:44:07.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f9535af5700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95300b17a0 con 0x7f953015db10 2023-10-10T20:44:07.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f952effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9530159c60 0x7f953014a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:07.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f952effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9530159c60 0x7f953014a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52120/0 (socket says 172.21.15.89:52120) 2023-10-10T20:44:07.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f952effd700 1 -- 172.21.15.89:0/2519835392 learned_addr learned my addr 172.21.15.89:0/2519835392 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:07.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f952effd700 1 -- 172.21.15.89:0/2519835392 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f953015db10 msgr2=0x7f95300b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:07.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f952ffff700 1 --2- 172.21.15.89:0/2519835392 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f953015db10 0x7f95300b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:07.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f952f7fe700 1 --2- 172.21.15.89:0/2519835392 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9530159890 0x7f953015bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:07.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f952effd700 1 --2- 172.21.15.89:0/2519835392 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f953015db10 0x7f95300b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:07.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f952effd700 1 -- 172.21.15.89:0/2519835392 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9530159890 msgr2=0x7f953015bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:07.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f952effd700 1 --2- 172.21.15.89:0/2519835392 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9530159890 0x7f953015bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:07.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f952effd700 1 -- 172.21.15.89:0/2519835392 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9524009420 con 0x7f9530159c60 2023-10-10T20:44:07.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f952effd700 1 --2- 172.21.15.89:0/2519835392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9530159c60 0x7f953014a340 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f9524013000 tx=0x7f9524017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:07.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f95177fe700 1 -- 172.21.15.89:0/2519835392 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524011b10 con 0x7f9530159c60 2023-10-10T20:44:07.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f95177fe700 1 -- 172.21.15.89:0/2519835392 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9524000cc0 con 0x7f9530159c60 2023-10-10T20:44:07.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f9535af5700 1 -- 172.21.15.89:0/2519835392 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95300b1a80 con 0x7f9530159c60 2023-10-10T20:44:07.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f95177fe700 1 -- 172.21.15.89:0/2519835392 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952401a6a0 con 0x7f9530159c60 2023-10-10T20:44:07.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.776+0000 7f9535af5700 1 -- 172.21.15.89:0/2519835392 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f953015c4d0 con 0x7f9530159c60 2023-10-10T20:44:07.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.777+0000 7f9535af5700 1 -- 172.21.15.89:0/2519835392 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f953015bc70 con 0x7f9530159c60 2023-10-10T20:44:07.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.778+0000 7f95177fe700 1 -- 172.21.15.89:0/2519835392 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f952401aa50 con 0x7f9530159c60 2023-10-10T20:44:07.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.778+0000 7f95177fe700 1 --2- 172.21.15.89:0/2519835392 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f951c067320 0x7f951c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:07.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.778+0000 7f95177fe700 1 -- 172.21.15.89:0/2519835392 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f95240869b0 con 0x7f9530159c60 2023-10-10T20:44:07.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.778+0000 7f952f7fe700 1 --2- 172.21.15.89:0/2519835392 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f951c067320 0x7f951c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:07.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.779+0000 7f952f7fe700 1 --2- 172.21.15.89:0/2519835392 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f951c067320 0x7f951c0697d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f95200076c0 tx=0x7f952000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:07.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:07.783+0000 7f95177fe700 1 -- 172.21.15.89:0/2519835392 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9524053600 con 0x7f9530159c60 2023-10-10T20:44:08.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.009+0000 7f9535af5700 1 -- 172.21.15.89:0/2519835392 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f953004c3c0 con 0x7f9530159c60 2023-10-10T20:44:08.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.009+0000 7f95177fe700 1 -- 172.21.15.89:0/2519835392 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v132) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9524056c20 con 0x7f9530159c60 2023-10-10T20:44:08.011 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:44:08.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.011+0000 7f9535af5700 1 -- 172.21.15.89:0/2519835392 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f951c067320 msgr2=0x7f951c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:08.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.011+0000 7f9535af5700 1 --2- 172.21.15.89:0/2519835392 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f951c067320 0x7f951c0697d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f95200076c0 tx=0x7f952000b040 comp rx=0 tx=0).stop 2023-10-10T20:44:08.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.011+0000 7f9535af5700 1 -- 172.21.15.89:0/2519835392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9530159c60 msgr2=0x7f953014a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:08.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.011+0000 7f9535af5700 1 --2- 172.21.15.89:0/2519835392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9530159c60 0x7f953014a340 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f9524013000 tx=0x7f9524017000 comp rx=0 tx=0).stop 2023-10-10T20:44:08.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.011+0000 7f9535af5700 1 -- 172.21.15.89:0/2519835392 shutdown_connections 2023-10-10T20:44:08.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.011+0000 7f9535af5700 1 --2- 172.21.15.89:0/2519835392 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f953015db10 0x7f95300b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:08.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.011+0000 7f9535af5700 1 --2- 172.21.15.89:0/2519835392 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f951c067320 0x7f951c0697d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:08.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.011+0000 7f9535af5700 1 --2- 172.21.15.89:0/2519835392 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9530159890 0x7f953015bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:08.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.011+0000 7f9535af5700 1 --2- 172.21.15.89:0/2519835392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9530159c60 0x7f953014a340 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:08.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.011+0000 7f9535af5700 1 -- 172.21.15.89:0/2519835392 >> 172.21.15.89:0/2519835392 conn(0x7f9530068e20 msgr2=0x7f953014d200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:08.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.012+0000 7f9535af5700 1 -- 172.21.15.89:0/2519835392 shutdown_connections 2023-10-10T20:44:08.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.012+0000 7f9535af5700 1 -- 172.21.15.89:0/2519835392 wait complete. 2023-10-10T20:44:08.025 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:44:08.026 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:44:08.026 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-10-10T20:44:08.026 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:44:08.027 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:44:08.027 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:44:08.027 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:44:08.027 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:44:08.028 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:44:08.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:44:08.029 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-10-10T20:44:08.048 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:44:08.048 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:44:08.048 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:44:08.049 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:44:08.049 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-10-10T20:44:08.049 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:44:08.049 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:44:08.049 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:44:08.050 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-10-10T20:44:08.050 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:44:08.050 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:44:08.050 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-10-10T20:44:08.051 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:44:08.051 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:44:08.051 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:44:08.051 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-10-10T20:44:08.051 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:44:08.052 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 41934sec preferred_lft 41934sec 2023-10-10T20:44:08.052 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-10-10T20:44:08.052 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:44:08.052 INFO:teuthology.orchestra.run.smithi111.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:44:08.052 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff 2023-10-10T20:44:08.052 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:44:08.053 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:44:08.054 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::a486:a6ff:fe5f:6021/64 scope link 2023-10-10T20:44:08.054 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:44:08.054 INFO:teuthology.orchestra.run.smithi111.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:44:08.054 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 1e:d0:81:1b:e2:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:44:08.055 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::1cd0:81ff:fe1b:e26b/64 scope link 2023-10-10T20:44:08.055 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:44:08.055 INFO:teuthology.orchestra.run.smithi111.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:44:08.055 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0a:8d:cd:75:45:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:44:08.055 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::88d:cdff:fe75:455e/64 scope link 2023-10-10T20:44:08.056 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:44:08.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:44:08.057 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:44:08.116 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-10-10T20:44:08.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:44:08.118 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-10-10T20:44:08.175 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-10-10T20:44:08.176 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:44:08.176 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:44:08.176 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:44:08.176 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:44:08.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:44:08.177 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-10-10T20:44:08.177 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-10-10T20:44:08.178 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:44:08.178 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:44:08.178 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:44:08.179 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-10-10T20:44:08.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:44:08.283 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-10-10T20:44:08.304 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:44:08.304 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:44:08.304 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:44:08.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:44:08.305 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-10-10T20:44:08.378 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:44:08.379 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:44:08.438 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:44:08.439 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:44:08.439 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:44:08.494 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-10T20:44:08.494 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:44:08.495 DEBUG:teuthology.orchestra.run.smithi111:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:44:08.577 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:44:08.577 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:44:08.578 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:44:08.585 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:44:08.586 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:44:08.586 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:44:08.586 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:44:08.587 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:44:08.607 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-10T20:44:08.607 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-10T20:44:08.608 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2023-10-10T20:44:08.608 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.111:3300,172.21.15.111:3301" 2023-10-10T20:44:08.609 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:44:08.609 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790" 2023-10-10T20:44:08.609 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.111:6789,172.21.15.111:6790:/ 2023-10-10T20:44:08.610 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=5dd675b8-9026-4d8a-a1bf-62fdc5e7de67" will pass to kernel 2023-10-10T20:44:08.612 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:44:08.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:44:08.613 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:44:08.678 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-10T20:44:08.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.798+0000 7f7deb59e700 1 Processor -- start 2023-10-10T20:44:08.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.798+0000 7f7deb59e700 1 -- start start 2023-10-10T20:44:08.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.799+0000 7f7deb59e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7dec0b0790 0x7f7dec0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:08.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.799+0000 7f7deb59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dec1591e0 0x7f7dec159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:08.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.799+0000 7f7deb59e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dec0cfa00 con 0x7f7dec0b03c0 2023-10-10T20:44:08.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.799+0000 7f7deb59e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dec0cb580 con 0x7f7dec0b0790 2023-10-10T20:44:08.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.799+0000 7f7deb59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec0cf160 con 0x7f7dec1591e0 2023-10-10T20:44:08.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.799+0000 7f7de1d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7dec0b0790 0x7f7dec0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:08.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.799+0000 7f7de1d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7dec0b0790 0x7f7dec0cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40180/0 (socket says 172.21.15.89:40180) 2023-10-10T20:44:08.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.799+0000 7f7de1d9b700 1 -- 172.21.15.89:0/132192832 learned_addr learned my addr 172.21.15.89:0/132192832 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:08.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.799+0000 7f7dead9d700 1 --2- 172.21.15.89:0/132192832 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dec1591e0 0x7f7dec159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:08.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.799+0000 7f7de1d9b700 1 -- 172.21.15.89:0/132192832 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dec1591e0 msgr2=0x7f7dec159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:08.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.799+0000 7f7de1d9b700 1 --2- 172.21.15.89:0/132192832 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dec1591e0 0x7f7dec159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:08.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.799+0000 7f7de1d9b700 1 -- 172.21.15.89:0/132192832 >> v1:172.21.15.111:6789/0 conn(0x7f7dec0b03c0 legacy=0x7f7dec0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:08.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.799+0000 7f7de1d9b700 1 -- 172.21.15.89:0/132192832 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dec0cf440 con 0x7f7dec0b0790 2023-10-10T20:44:08.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.799+0000 7f7de1d9b700 1 --2- 172.21.15.89:0/132192832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dec0b0790 0x7f7dec0cea70 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40099e0 tx=0x7f7dd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c05cbf1d10b916a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:08.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.799+0000 7f7de9d9b700 1 -- 172.21.15.89:0/132192832 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd400b7e0 con 0x7f7dec0b0790 2023-10-10T20:44:08.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.799+0000 7f7de9d9b700 1 -- 172.21.15.89:0/132192832 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd400b940 con 0x7f7dec0b0790 2023-10-10T20:44:08.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.799+0000 7f7de9d9b700 1 -- 172.21.15.89:0/132192832 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd400bce0 con 0x7f7dec0b0790 2023-10-10T20:44:08.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.800+0000 7f7deb59e700 1 -- 172.21.15.89:0/132192832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dec0b0790 msgr2=0x7f7dec0cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:08.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.800+0000 7f7deb59e700 1 --2- 172.21.15.89:0/132192832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dec0b0790 0x7f7dec0cea70 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40099e0 tx=0x7f7dd4009250 comp rx=0 tx=0).stop 2023-10-10T20:44:08.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.800+0000 7f7deb59e700 1 -- 172.21.15.89:0/132192832 shutdown_connections 2023-10-10T20:44:08.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.800+0000 7f7deb59e700 1 --2- 172.21.15.89:0/132192832 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dec1591e0 0x7f7dec159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:08.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.800+0000 7f7deb59e700 1 --2- 172.21.15.89:0/132192832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dec0b0790 0x7f7dec0cea70 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:08.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.800+0000 7f7deb59e700 1 -- 172.21.15.89:0/132192832 >> 172.21.15.89:0/132192832 conn(0x7f7dec063230 msgr2=0x7f7dec063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:08.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.800+0000 7f7deb59e700 1 -- 172.21.15.89:0/132192832 shutdown_connections 2023-10-10T20:44:08.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.800+0000 7f7deb59e700 1 -- 172.21.15.89:0/132192832 wait complete. 2023-10-10T20:44:08.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.801+0000 7f7deb59e700 1 Processor -- start 2023-10-10T20:44:08.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.801+0000 7f7deb59e700 1 -- start start 2023-10-10T20:44:08.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.802+0000 7f7deb59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dec0b03c0 0x7f7dec155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:08.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.802+0000 7f7deb59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dec0b0790 0x7f7dec15c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:08.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.802+0000 7f7deb59e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dec1591e0 0x7f7dec068f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:08.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.802+0000 7f7deb59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec155d90 con 0x7f7dec0b03c0 2023-10-10T20:44:08.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.802+0000 7f7deb59e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec16a570 con 0x7f7dec1591e0 2023-10-10T20:44:08.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.802+0000 7f7deb59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec169cb0 con 0x7f7dec0b0790 2023-10-10T20:44:08.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.802+0000 7f7dea59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dec0b03c0 0x7f7dec155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:08.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.802+0000 7f7dea59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dec0b03c0 0x7f7dec155850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40192/0 (socket says 172.21.15.89:40192) 2023-10-10T20:44:08.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.802+0000 7f7dea59c700 1 -- 172.21.15.89:0/3961670374 learned_addr learned my addr 172.21.15.89:0/3961670374 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:08.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.802+0000 7f7de1d9b700 1 --2- 172.21.15.89:0/3961670374 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dec0b0790 0x7f7dec15c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:08.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.802+0000 7f7dead9d700 1 --2- 172.21.15.89:0/3961670374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dec1591e0 0x7f7dec068f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:08.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.802+0000 7f7dea59c700 1 -- 172.21.15.89:0/3961670374 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dec0b0790 msgr2=0x7f7dec15c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:08.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.802+0000 7f7dea59c700 1 --2- 172.21.15.89:0/3961670374 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dec0b0790 0x7f7dec15c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:08.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.802+0000 7f7dea59c700 1 -- 172.21.15.89:0/3961670374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dec1591e0 msgr2=0x7f7dec068f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:08.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.802+0000 7f7dea59c700 1 --2- 172.21.15.89:0/3961670374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dec1591e0 0x7f7dec068f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:08.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.802+0000 7f7dea59c700 1 -- 172.21.15.89:0/3961670374 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd40094f0 con 0x7f7dec0b03c0 2023-10-10T20:44:08.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.802+0000 7f7dead9d700 1 --2- 172.21.15.89:0/3961670374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dec1591e0 0x7f7dec068f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:44:08.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.802+0000 7f7dea59c700 1 --2- 172.21.15.89:0/3961670374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dec0b03c0 0x7f7dec155850 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc00b610 tx=0x7f7ddc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:08.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.802+0000 7f7de2ffd700 1 -- 172.21.15.89:0/3961670374 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc00f510 con 0x7f7dec0b03c0 2023-10-10T20:44:08.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.803+0000 7f7de2ffd700 1 -- 172.21.15.89:0/3961670374 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ddc00f6b0 con 0x7f7dec0b03c0 2023-10-10T20:44:08.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.803+0000 7f7deb59e700 1 -- 172.21.15.89:0/3961670374 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dec169f20 con 0x7f7dec0b03c0 2023-10-10T20:44:08.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.803+0000 7f7de2ffd700 1 -- 172.21.15.89:0/3961670374 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc00fa70 con 0x7f7dec0b03c0 2023-10-10T20:44:08.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.803+0000 7f7deb59e700 1 -- 172.21.15.89:0/3961670374 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dec15d430 con 0x7f7dec0b03c0 2023-10-10T20:44:08.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.803+0000 7f7deb59e700 1 -- 172.21.15.89:0/3961670374 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dec04c3c0 con 0x7f7dec0b03c0 2023-10-10T20:44:08.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.804+0000 7f7de2ffd700 1 -- 172.21.15.89:0/3961670374 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7ddc01a400 con 0x7f7dec0b03c0 2023-10-10T20:44:08.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.804+0000 7f7de2ffd700 1 --2- 172.21.15.89:0/3961670374 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7dcc067320 0x7f7dcc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:08.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.804+0000 7f7de2ffd700 1 -- 172.21.15.89:0/3961670374 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7ddc0845e0 con 0x7f7dec0b03c0 2023-10-10T20:44:08.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.804+0000 7f7de1d9b700 1 --2- 172.21.15.89:0/3961670374 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7dcc067320 0x7f7dcc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:08.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.805+0000 7f7de1d9b700 1 --2- 172.21.15.89:0/3961670374 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7dcc067320 0x7f7dcc0697d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40099e0 tx=0x7f7dd4016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:08.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:08.808+0000 7f7de2ffd700 1 -- 172.21.15.89:0/3961670374 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ddc0513f0 con 0x7f7dec0b03c0 2023-10-10T20:44:09.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.034+0000 7f7deb59e700 1 -- 172.21.15.89:0/3961670374 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7dec0ce850 con 0x7f7dec0b03c0 2023-10-10T20:44:09.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.643+0000 7f7de2ffd700 1 -- 172.21.15.89:0/3961670374 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v133) v1 ==== 136+0+0 (secure 0 0 0) 0x7f7ddc054a10 con 0x7f7dec0b03c0 2023-10-10T20:44:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.645+0000 7f7deb59e700 1 -- 172.21.15.89:0/3961670374 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7dcc067320 msgr2=0x7f7dcc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.645+0000 7f7deb59e700 1 --2- 172.21.15.89:0/3961670374 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7dcc067320 0x7f7dcc0697d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40099e0 tx=0x7f7dd4016000 comp rx=0 tx=0).stop 2023-10-10T20:44:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.645+0000 7f7deb59e700 1 -- 172.21.15.89:0/3961670374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dec0b03c0 msgr2=0x7f7dec155850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.645+0000 7f7deb59e700 1 --2- 172.21.15.89:0/3961670374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dec0b03c0 0x7f7dec155850 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc00b610 tx=0x7f7ddc009120 comp rx=0 tx=0).stop 2023-10-10T20:44:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.645+0000 7f7deb59e700 1 -- 172.21.15.89:0/3961670374 shutdown_connections 2023-10-10T20:44:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.645+0000 7f7deb59e700 1 --2- 172.21.15.89:0/3961670374 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7dcc067320 0x7f7dcc0697d0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.645+0000 7f7deb59e700 1 --2- 172.21.15.89:0/3961670374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dec1591e0 0x7f7dec068f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.645+0000 7f7deb59e700 1 --2- 172.21.15.89:0/3961670374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dec0b03c0 0x7f7dec155850 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.645+0000 7f7deb59e700 1 --2- 172.21.15.89:0/3961670374 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7dec0b0790 0x7f7dec15c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.645+0000 7f7deb59e700 1 -- 172.21.15.89:0/3961670374 >> 172.21.15.89:0/3961670374 conn(0x7f7dec063230 msgr2=0x7f7dec158940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.645+0000 7f7deb59e700 1 -- 172.21.15.89:0/3961670374 shutdown_connections 2023-10-10T20:44:09.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.645+0000 7f7deb59e700 1 -- 172.21.15.89:0/3961670374 wait complete. 2023-10-10T20:44:09.660 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-10T20:44:09.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.775+0000 7fd3467de700 1 Processor -- start 2023-10-10T20:44:09.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.775+0000 7fd3467de700 1 -- start start 2023-10-10T20:44:09.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.776+0000 7fd3467de700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd3401597f0 0x7fd3400b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:09.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.776+0000 7fd3467de700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3400b11d0 con 0x7fd34015d230 2023-10-10T20:44:09.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.776+0000 7fd3467de700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3400b14b0 con 0x7fd340159420 2023-10-10T20:44:09.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.776+0000 7fd3467de700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400b1790 con 0x7fd3401597f0 2023-10-10T20:44:09.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.776+0000 7fd345fdd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd34015d230 0x7fd3400b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37916/0 (socket says 172.21.15.89:37916) 2023-10-10T20:44:09.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.776+0000 7fd345fdd700 1 -- 172.21.15.89:0/4193202905 learned_addr learned my addr 172.21.15.89:0/4193202905 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:44:09.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.776+0000 7fd344fdb700 1 --2- 172.21.15.89:0/4193202905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd3401597f0 0x7fd3400b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:09.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.776+0000 7fd33ffff700 1 -- 172.21.15.89:0/4193202905 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366349226 0 0) 0x7fd3400b11d0 con 0x7fd34015d230 2023-10-10T20:44:09.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.776+0000 7fd33ffff700 1 -- 172.21.15.89:0/4193202905 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd328003570 con 0x7fd34015d230 2023-10-10T20:44:09.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.776+0000 7fd33ffff700 1 -- 172.21.15.89:0/4193202905 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1865368726 0 0) 0x7fd328003570 con 0x7fd34015d230 2023-10-10T20:44:09.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.776+0000 7fd33ffff700 1 -- 172.21.15.89:0/4193202905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd3401597f0 msgr2=0x7fd3400b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:09.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.776+0000 7fd33ffff700 1 --2- 172.21.15.89:0/4193202905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd3401597f0 0x7fd3400b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:09.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.776+0000 7fd33ffff700 1 -- 172.21.15.89:0/4193202905 >> v1:172.21.15.111:6789/0 conn(0x7fd340159420 legacy=0x7fd3400b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:09.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.776+0000 7fd33ffff700 1 -- 172.21.15.89:0/4193202905 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3400b1a70 con 0x7fd34015d230 2023-10-10T20:44:09.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.776+0000 7fd33ffff700 1 -- 172.21.15.89:0/4193202905 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fd330001470 con 0x7fd34015d230 2023-10-10T20:44:09.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.776+0000 7fd33ffff700 1 -- 172.21.15.89:0/4193202905 >> v1:172.21.15.89:6789/0 conn(0x7fd34015d230 legacy=0x7fd3400b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:09.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.776+0000 7fd33ffff700 1 --2- 172.21.15.89:0/4193202905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd328003a90 0x7fd328003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:09.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.777+0000 7fd33ffff700 1 --2- 172.21.15.89:0/4193202905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd328004810 0x7fd328004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:09.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.777+0000 7fd33ffff700 1 --2- 172.21.15.89:0/4193202905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd3401597f0 0x7fd3280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:09.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.777+0000 7fd33ffff700 1 -- 172.21.15.89:0/4193202905 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400b1790 con 0x7fd328003a90 2023-10-10T20:44:09.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.777+0000 7fd33ffff700 1 -- 172.21.15.89:0/4193202905 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32800bb10 con 0x7fd328004810 2023-10-10T20:44:09.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.777+0000 7fd33ffff700 1 -- 172.21.15.89:0/4193202905 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32800bcd0 con 0x7fd3401597f0 2023-10-10T20:44:09.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.777+0000 7fd3457dc700 1 --2- 172.21.15.89:0/4193202905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd328003a90 0x7fd328003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:09.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.777+0000 7fd344fdb700 1 --2- 172.21.15.89:0/4193202905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd328004810 0x7fd328004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:09.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.777+0000 7fd345fdd700 1 --2- 172.21.15.89:0/4193202905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd3401597f0 0x7fd3280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:09.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.777+0000 7fd3457dc700 1 -- 172.21.15.89:0/4193202905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd3401597f0 msgr2=0x7fd3280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:09.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.777+0000 7fd3457dc700 1 --2- 172.21.15.89:0/4193202905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd3401597f0 0x7fd3280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:09.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.777+0000 7fd3457dc700 1 -- 172.21.15.89:0/4193202905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd328004810 msgr2=0x7fd328004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:09.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.777+0000 7fd3457dc700 1 --2- 172.21.15.89:0/4193202905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd328004810 0x7fd328004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:09.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.777+0000 7fd3457dc700 1 -- 172.21.15.89:0/4193202905 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd328003570 con 0x7fd328003a90 2023-10-10T20:44:09.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.777+0000 7fd3457dc700 1 --2- 172.21.15.89:0/4193202905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd328003a90 0x7fd328003e60 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fd3380020a0 tx=0x7fd3380016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78df6e32c5fbb7f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:09.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.777+0000 7fd33ffff700 1 -- 172.21.15.89:0/4193202905 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3380019e0 con 0x7fd328003a90 2023-10-10T20:44:09.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.777+0000 7fd33ffff700 1 -- 172.21.15.89:0/4193202905 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33800c8b0 con 0x7fd328003a90 2023-10-10T20:44:09.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.777+0000 7fd33ffff700 1 -- 172.21.15.89:0/4193202905 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33800cc50 con 0x7fd328003a90 2023-10-10T20:44:09.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.778+0000 7fd3467de700 1 -- 172.21.15.89:0/4193202905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd328003a90 msgr2=0x7fd328003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:09.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.778+0000 7fd3467de700 1 --2- 172.21.15.89:0/4193202905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd328003a90 0x7fd328003e60 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fd3380020a0 tx=0x7fd3380016d0 comp rx=0 tx=0).stop 2023-10-10T20:44:09.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.778+0000 7fd345fdd700 1 -- 172.21.15.89:0/4193202905 reap_dead start 2023-10-10T20:44:09.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.778+0000 7fd3467de700 1 -- 172.21.15.89:0/4193202905 shutdown_connections 2023-10-10T20:44:09.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.778+0000 7fd3467de700 1 -- 172.21.15.89:0/4193202905 >> 172.21.15.89:0/4193202905 conn(0x7fd3400d0660 msgr2=0x7fd3400d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:09.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.778+0000 7fd3467de700 1 -- 172.21.15.89:0/4193202905 shutdown_connections 2023-10-10T20:44:09.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.778+0000 7fd3467de700 1 -- 172.21.15.89:0/4193202905 wait complete. 2023-10-10T20:44:09.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.779+0000 7fd3467de700 1 Processor -- start 2023-10-10T20:44:09.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.779+0000 7fd3467de700 1 -- start start 2023-10-10T20:44:09.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.780+0000 7fd3467de700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd328003a90 0x7fd340143680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:09.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.780+0000 7fd3467de700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3400c9960 0x7fd3400b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:09.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.780+0000 7fd3467de700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd340156ac0 0x7fd340156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:09.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.780+0000 7fd3467de700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd340157470 con 0x7fd3400c9960 2023-10-10T20:44:09.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.780+0000 7fd3467de700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400caae0 con 0x7fd328003a90 2023-10-10T20:44:09.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.780+0000 7fd3467de700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd340143bc0 con 0x7fd340156ac0 2023-10-10T20:44:09.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.780+0000 7fd344fdb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3400c9960 0x7fd3400b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:09.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.780+0000 7fd344fdb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3400c9960 0x7fd3400b5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40202/0 (socket says 172.21.15.89:40202) 2023-10-10T20:44:09.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.780+0000 7fd344fdb700 1 -- 172.21.15.89:0/4282963805 learned_addr learned my addr 172.21.15.89:0/4282963805 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:09.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.780+0000 7fd3457dc700 1 --2- 172.21.15.89:0/4282963805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd328003a90 0x7fd340143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:09.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.780+0000 7fd345fdd700 1 --2- 172.21.15.89:0/4282963805 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd340156ac0 0x7fd340156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:09.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.780+0000 7fd344fdb700 1 -- 172.21.15.89:0/4282963805 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd340156ac0 msgr2=0x7fd340156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:09.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.780+0000 7fd344fdb700 1 --2- 172.21.15.89:0/4282963805 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd340156ac0 0x7fd340156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:09.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.780+0000 7fd344fdb700 1 -- 172.21.15.89:0/4282963805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd328003a90 msgr2=0x7fd340143680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:09.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.780+0000 7fd344fdb700 1 --2- 172.21.15.89:0/4282963805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd328003a90 0x7fd340143680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:09.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.780+0000 7fd344fdb700 1 -- 172.21.15.89:0/4282963805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3400c9ed0 con 0x7fd3400c9960 2023-10-10T20:44:09.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.780+0000 7fd344fdb700 1 --2- 172.21.15.89:0/4282963805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3400c9960 0x7fd3400b5160 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fd334011d40 tx=0x7fd3340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:09.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.780+0000 7fd33d7fa700 1 -- 172.21.15.89:0/4282963805 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33400f9b0 con 0x7fd3400c9960 2023-10-10T20:44:09.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.781+0000 7fd33d7fa700 1 -- 172.21.15.89:0/4282963805 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33400fb10 con 0x7fd3400c9960 2023-10-10T20:44:09.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.781+0000 7fd3467de700 1 -- 172.21.15.89:0/4282963805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3400ca1b0 con 0x7fd3400c9960 2023-10-10T20:44:09.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.781+0000 7fd33d7fa700 1 -- 172.21.15.89:0/4282963805 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33400fed0 con 0x7fd3400c9960 2023-10-10T20:44:09.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.781+0000 7fd3467de700 1 -- 172.21.15.89:0/4282963805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3400ca770 con 0x7fd3400c9960 2023-10-10T20:44:09.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.781+0000 7fd3467de700 1 -- 172.21.15.89:0/4282963805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3400cac50 con 0x7fd3400c9960 2023-10-10T20:44:09.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.782+0000 7fd33d7fa700 1 -- 172.21.15.89:0/4282963805 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd334014650 con 0x7fd3400c9960 2023-10-10T20:44:09.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.782+0000 7fd33d7fa700 1 --2- 172.21.15.89:0/4282963805 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd32c067350 0x7fd32c069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:09.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.782+0000 7fd33d7fa700 1 -- 172.21.15.89:0/4282963805 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd334018030 con 0x7fd3400c9960 2023-10-10T20:44:09.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.782+0000 7fd3457dc700 1 --2- 172.21.15.89:0/4282963805 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd32c067350 0x7fd32c069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:09.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.783+0000 7fd3457dc700 1 --2- 172.21.15.89:0/4282963805 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd32c067350 0x7fd32c069800 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd338001370 tx=0x7fd338016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:09.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.787+0000 7fd33d7fa700 1 -- 172.21.15.89:0/4282963805 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3340521e0 con 0x7fd3400c9960 2023-10-10T20:44:09.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.998+0000 7fd3467de700 1 -- 172.21.15.89:0/4282963805 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd34004c3c0 con 0x7fd32c067350 2023-10-10T20:44:10.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:09.999+0000 7fd33d7fa700 1 -- 172.21.15.89:0/4282963805 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fd34004c3c0 con 0x7fd32c067350 2023-10-10T20:44:10.001 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-10T20:44:10.001 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-10T20:44:10.001 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephfs" 2023-10-10T20:44:10.002 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-10T20:44:10.002 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-10T20:44:10.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.001+0000 7fd3467de700 1 -- 172.21.15.89:0/4282963805 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd32c067350 msgr2=0x7fd32c069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:10.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.001+0000 7fd3467de700 1 --2- 172.21.15.89:0/4282963805 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd32c067350 0x7fd32c069800 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd338001370 tx=0x7fd338016000 comp rx=0 tx=0).stop 2023-10-10T20:44:10.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.001+0000 7fd3467de700 1 -- 172.21.15.89:0/4282963805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3400c9960 msgr2=0x7fd3400b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:10.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.001+0000 7fd3467de700 1 --2- 172.21.15.89:0/4282963805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3400c9960 0x7fd3400b5160 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fd334011d40 tx=0x7fd3340091a0 comp rx=0 tx=0).stop 2023-10-10T20:44:10.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.002+0000 7fd3467de700 1 -- 172.21.15.89:0/4282963805 shutdown_connections 2023-10-10T20:44:10.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.002+0000 7fd3467de700 1 --2- 172.21.15.89:0/4282963805 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd340156ac0 0x7fd340156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:10.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.002+0000 7fd3467de700 1 --2- 172.21.15.89:0/4282963805 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd32c067350 0x7fd32c069800 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:10.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.002+0000 7fd3467de700 1 --2- 172.21.15.89:0/4282963805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd328003a90 0x7fd340143680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:10.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.002+0000 7fd3467de700 1 --2- 172.21.15.89:0/4282963805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3400c9960 0x7fd3400b5160 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:10.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.002+0000 7fd3467de700 1 -- 172.21.15.89:0/4282963805 >> 172.21.15.89:0/4282963805 conn(0x7fd3400d0660 msgr2=0x7fd340069890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:10.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.002+0000 7fd3467de700 1 -- 172.21.15.89:0/4282963805 shutdown_connections 2023-10-10T20:44:10.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.002+0000 7fd3467de700 1 -- 172.21.15.89:0/4282963805 wait complete. 2023-10-10T20:44:10.015 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:44:10.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.132+0000 7ff0c574b700 1 Processor -- start 2023-10-10T20:44:10.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.132+0000 7ff0c574b700 1 -- start start 2023-10-10T20:44:10.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.133+0000 7ff0c574b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff0c00b16d0 0x7ff0c00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:10.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.133+0000 7ff0c574b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c00b2080 con 0x7ff0c00b33b0 2023-10-10T20:44:10.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.133+0000 7ff0c574b700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c00cfb00 con 0x7ff0c00b4e70 2023-10-10T20:44:10.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.133+0000 7ff0c574b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c00b5c40 con 0x7ff0c00b16d0 2023-10-10T20:44:10.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.133+0000 7ff0bf7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff0c00b16d0 0x7ff0c00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:10.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.133+0000 7ff0c4f4a700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff0c00b33b0 0x7ff0c00b37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37928/0 (socket says 172.21.15.89:37928) 2023-10-10T20:44:10.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.133+0000 7ff0c4f4a700 1 -- 172.21.15.89:0/3858435999 learned_addr learned my addr 172.21.15.89:0/3858435999 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:44:10.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.133+0000 7ff0bf7fe700 1 -- 172.21.15.89:0/3858435999 >> v1:172.21.15.111:6789/0 conn(0x7ff0c00b4e70 legacy=0x7ff0c00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.133+0000 7ff0bf7fe700 1 -- 172.21.15.89:0/3858435999 >> v1:172.21.15.89:6789/0 conn(0x7ff0c00b33b0 legacy=0x7ff0c00b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.133+0000 7ff0bf7fe700 1 -- 172.21.15.89:0/3858435999 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c00b5f20 con 0x7ff0c00b16d0 2023-10-10T20:44:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.133+0000 7ff0beffd700 1 -- 172.21.15.89:0/3858435999 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 948358287 0 0) 0x7ff0c00b2080 con 0x7ff0c00b33b0 2023-10-10T20:44:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.134+0000 7ff0bf7fe700 1 --2- 172.21.15.89:0/3858435999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00b16d0 0x7ff0c00b1b40 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40098e0 tx=0x7ff0b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1989a3b8d094afd server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:10.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.134+0000 7ff0beffd700 1 -- 172.21.15.89:0/3858435999 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4009e80 con 0x7ff0c00b16d0 2023-10-10T20:44:10.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.134+0000 7ff0beffd700 1 -- 172.21.15.89:0/3858435999 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b400b820 con 0x7ff0c00b16d0 2023-10-10T20:44:10.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.134+0000 7ff0beffd700 1 -- 172.21.15.89:0/3858435999 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b400bba0 con 0x7ff0c00b16d0 2023-10-10T20:44:10.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.134+0000 7ff0c574b700 1 -- 172.21.15.89:0/3858435999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00b16d0 msgr2=0x7ff0c00b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:10.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.134+0000 7ff0c574b700 1 --2- 172.21.15.89:0/3858435999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00b16d0 0x7ff0c00b1b40 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40098e0 tx=0x7ff0b40091a0 comp rx=0 tx=0).stop 2023-10-10T20:44:10.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.134+0000 7ff0c574b700 1 -- 172.21.15.89:0/3858435999 shutdown_connections 2023-10-10T20:44:10.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.134+0000 7ff0c574b700 1 --2- 172.21.15.89:0/3858435999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00b16d0 0x7ff0c00b1b40 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:10.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.134+0000 7ff0c574b700 1 -- 172.21.15.89:0/3858435999 >> 172.21.15.89:0/3858435999 conn(0x7ff0c00d10c0 msgr2=0x7ff0c00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:10.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.134+0000 7ff0c574b700 1 -- 172.21.15.89:0/3858435999 shutdown_connections 2023-10-10T20:44:10.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.134+0000 7ff0c574b700 1 -- 172.21.15.89:0/3858435999 wait complete. 2023-10-10T20:44:10.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.135+0000 7ff0c574b700 1 Processor -- start 2023-10-10T20:44:10.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.135+0000 7ff0c574b700 1 -- start start 2023-10-10T20:44:10.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.136+0000 7ff0c574b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0c00b16d0 0x7ff0c00a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:10.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.136+0000 7ff0c574b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff0c00b33b0 0x7ff0c014e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:10.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.136+0000 7ff0c574b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00b4e70 0x7ff0c00c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:10.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.136+0000 7ff0c574b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c014e880 con 0x7ff0c00b4e70 2023-10-10T20:44:10.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.136+0000 7ff0c574b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c00cbc90 con 0x7ff0c00b16d0 2023-10-10T20:44:10.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.136+0000 7ff0c574b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c00ca2b0 con 0x7ff0c00b33b0 2023-10-10T20:44:10.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.136+0000 7ff0c4f4a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00b4e70 0x7ff0c00c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:10.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.136+0000 7ff0c4f4a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00b4e70 0x7ff0c00c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40218/0 (socket says 172.21.15.89:40218) 2023-10-10T20:44:10.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.136+0000 7ff0c4f4a700 1 -- 172.21.15.89:0/318831167 learned_addr learned my addr 172.21.15.89:0/318831167 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:10.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.136+0000 7ff0c4f4a700 1 -- 172.21.15.89:0/318831167 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff0c00b33b0 msgr2=0x7ff0c014e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.136+0000 7ff0c4f4a700 1 --2- 172.21.15.89:0/318831167 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff0c00b33b0 0x7ff0c014e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.137+0000 7ff0c4f4a700 1 -- 172.21.15.89:0/318831167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0c00b16d0 msgr2=0x7ff0c00a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.136+0000 7ff0bffff700 1 --2- 172.21.15.89:0/318831167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0c00b16d0 0x7ff0c00a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.137+0000 7ff0c4f4a700 1 --2- 172.21.15.89:0/318831167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0c00b16d0 0x7ff0c00a7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.137+0000 7ff0c4f4a700 1 -- 172.21.15.89:0/318831167 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b4009420 con 0x7ff0c00b4e70 2023-10-10T20:44:10.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.137+0000 7ff0bffff700 1 --2- 172.21.15.89:0/318831167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0c00b16d0 0x7ff0c00a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:44:10.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.137+0000 7ff0c4f4a700 1 --2- 172.21.15.89:0/318831167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00b4e70 0x7ff0c00c9d70 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800cc90 tx=0x7ff0b8001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:10.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.137+0000 7ff0a7fff700 1 -- 172.21.15.89:0/318831167 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b800cea0 con 0x7ff0c00b4e70 2023-10-10T20:44:10.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.137+0000 7ff0a7fff700 1 -- 172.21.15.89:0/318831167 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b800ab30 con 0x7ff0c00b4e70 2023-10-10T20:44:10.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.137+0000 7ff0c574b700 1 -- 172.21.15.89:0/318831167 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c00cb0b0 con 0x7ff0c00b4e70 2023-10-10T20:44:10.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.137+0000 7ff0a7fff700 1 -- 172.21.15.89:0/318831167 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b80070f0 con 0x7ff0c00b4e70 2023-10-10T20:44:10.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.137+0000 7ff0c574b700 1 -- 172.21.15.89:0/318831167 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c00cb640 con 0x7ff0c00b4e70 2023-10-10T20:44:10.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.138+0000 7ff0c574b700 1 -- 172.21.15.89:0/318831167 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c0088990 con 0x7ff0c00b4e70 2023-10-10T20:44:10.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.138+0000 7ff0a7fff700 1 -- 172.21.15.89:0/318831167 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff0b80074a0 con 0x7ff0c00b4e70 2023-10-10T20:44:10.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.139+0000 7ff0a7fff700 1 --2- 172.21.15.89:0/318831167 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff0ac067320 0x7ff0ac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.139+0000 7ff0a7fff700 1 -- 172.21.15.89:0/318831167 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff0b8085220 con 0x7ff0c00b4e70 2023-10-10T20:44:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.139+0000 7ff0bffff700 1 --2- 172.21.15.89:0/318831167 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff0ac067320 0x7ff0ac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.139+0000 7ff0bffff700 1 --2- 172.21.15.89:0/318831167 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff0ac067320 0x7ff0ac0697d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0006fd0 tx=0x7ff0b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.143+0000 7ff0a7fff700 1 -- 172.21.15.89:0/318831167 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff0b8051e60 con 0x7ff0c00b4e70 2023-10-10T20:44:10.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.328+0000 7ff0c574b700 1 -- 172.21.15.89:0/318831167 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff0c00cbdd0 con 0x7ff0c00b4e70 2023-10-10T20:44:10.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.328+0000 7ff0a7fff700 1 -- 172.21.15.89:0/318831167 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff0b8055480 con 0x7ff0c00b4e70 2023-10-10T20:44:10.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.330+0000 7ff0c574b700 1 -- 172.21.15.89:0/318831167 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff0ac067320 msgr2=0x7ff0ac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:10.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.330+0000 7ff0c574b700 1 --2- 172.21.15.89:0/318831167 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff0ac067320 0x7ff0ac0697d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0006fd0 tx=0x7ff0b0008040 comp rx=0 tx=0).stop 2023-10-10T20:44:10.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.330+0000 7ff0c574b700 1 -- 172.21.15.89:0/318831167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00b4e70 msgr2=0x7ff0c00c9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:10.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.330+0000 7ff0c574b700 1 --2- 172.21.15.89:0/318831167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00b4e70 0x7ff0c00c9d70 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800cc90 tx=0x7ff0b8001a20 comp rx=0 tx=0).stop 2023-10-10T20:44:10.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.330+0000 7ff0c574b700 1 -- 172.21.15.89:0/318831167 shutdown_connections 2023-10-10T20:44:10.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.330+0000 7ff0c574b700 1 --2- 172.21.15.89:0/318831167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c00b4e70 0x7ff0c00c9d70 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:10.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.330+0000 7ff0c574b700 1 --2- 172.21.15.89:0/318831167 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff0ac067320 0x7ff0ac0697d0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:10.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.330+0000 7ff0c574b700 1 --2- 172.21.15.89:0/318831167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0c00b16d0 0x7ff0c00a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:10.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.330+0000 7ff0c574b700 1 --2- 172.21.15.89:0/318831167 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff0c00b33b0 0x7ff0c014e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:10.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.330+0000 7ff0c574b700 1 -- 172.21.15.89:0/318831167 >> 172.21.15.89:0/318831167 conn(0x7ff0c00d10c0 msgr2=0x7ff0c00cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:10.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.330+0000 7ff0c574b700 1 -- 172.21.15.89:0/318831167 shutdown_connections 2023-10-10T20:44:10.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.330+0000 7ff0c574b700 1 -- 172.21.15.89:0/318831167 wait complete. 2023-10-10T20:44:10.344 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:44:10.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:44:10.346 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:44:10.383 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:44:10.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:44:10.384 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:44:10.438 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:44:10.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:44:10.439 DEBUG:teuthology.orchestra.run.smithi111:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:44:10.482 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:44:10.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:44:10.483 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:44:10.539 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:44:10.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:44:10.541 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-10T20:44:10.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:44:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-10T20:44:10.557 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:44:10.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:44:10.559 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-10T20:44:10.595 INFO:teuthology.orchestra.run.smithi111.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-10T20:44:10.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:44:10.597 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs 2023-10-10T20:44:10.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.713+0000 7f3a5236c700 1 Processor -- start 2023-10-10T20:44:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.713+0000 7f3a5236c700 1 -- start start 2023-10-10T20:44:10.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.713+0000 7f3a5236c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3a4c159890 0x7f3a4c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:10.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.713+0000 7f3a5236c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a4c069560 0x7f3a4c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:10.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.714+0000 7f3a5236c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a4c069ee0 con 0x7f3a4c159c60 2023-10-10T20:44:10.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.714+0000 7f3a5236c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a4c15a160 con 0x7f3a4c159890 2023-10-10T20:44:10.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.714+0000 7f3a5236c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c0d13a0 con 0x7f3a4c069560 2023-10-10T20:44:10.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.714+0000 7f3a4bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3a4c159890 0x7f3a4c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:10.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.714+0000 7f3a4b7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3a4c159c60 0x7f3a4c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37938/0 (socket says 172.21.15.89:37938) 2023-10-10T20:44:10.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.714+0000 7f3a4bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3a4c159890 0x7f3a4c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40222/0 (socket says 172.21.15.89:40222) 2023-10-10T20:44:10.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.714+0000 7f3a4b7fe700 1 -- 172.21.15.89:0/852154839 learned_addr learned my addr 172.21.15.89:0/852154839 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:44:10.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.714+0000 7f3a50909700 1 --2- 172.21.15.89:0/852154839 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a4c069560 0x7f3a4c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:10.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.714+0000 7f3a4bfff700 1 -- 172.21.15.89:0/852154839 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a4c069560 msgr2=0x7f3a4c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:10.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.714+0000 7f3a4bfff700 1 --2- 172.21.15.89:0/852154839 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a4c069560 0x7f3a4c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:10.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.714+0000 7f3a4bfff700 1 -- 172.21.15.89:0/852154839 >> v1:172.21.15.89:6789/0 conn(0x7f3a4c159c60 legacy=0x7f3a4c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:10.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.714+0000 7f3a4bfff700 1 -- 172.21.15.89:0/852154839 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4c15e140 con 0x7f3a4c159890 2023-10-10T20:44:10.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.714+0000 7f3a4affd700 1 -- 172.21.15.89:0/852154839 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817398836 0 0) 0x7f3a4c069ee0 con 0x7f3a4c159c60 2023-10-10T20:44:10.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.714+0000 7f3a4bfff700 1 --2- 172.21.15.89:0/852154839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a4c159890 0x7f3a4c15abd0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0098e0 tx=0x7f3a3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b85daf4b3ea8c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:10.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.714+0000 7f3a4affd700 1 -- 172.21.15.89:0/852154839 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c00b640 con 0x7f3a4c159890 2023-10-10T20:44:10.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.714+0000 7f3a4affd700 1 -- 172.21.15.89:0/852154839 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c00b7a0 con 0x7f3a4c159890 2023-10-10T20:44:10.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.714+0000 7f3a4affd700 1 -- 172.21.15.89:0/852154839 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c00bb40 con 0x7f3a4c159890 2023-10-10T20:44:10.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.715+0000 7f3a5236c700 1 -- 172.21.15.89:0/852154839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a4c159890 msgr2=0x7f3a4c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:10.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.715+0000 7f3a5236c700 1 --2- 172.21.15.89:0/852154839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a4c159890 0x7f3a4c15abd0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0098e0 tx=0x7f3a3c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:44:10.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.715+0000 7f3a5236c700 1 -- 172.21.15.89:0/852154839 shutdown_connections 2023-10-10T20:44:10.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.715+0000 7f3a5236c700 1 --2- 172.21.15.89:0/852154839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a4c159890 0x7f3a4c15abd0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:10.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.715+0000 7f3a5236c700 1 --2- 172.21.15.89:0/852154839 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a4c069560 0x7f3a4c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:10.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.715+0000 7f3a5236c700 1 -- 172.21.15.89:0/852154839 >> 172.21.15.89:0/852154839 conn(0x7f3a4c068e20 msgr2=0x7f3a4c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:10.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.715+0000 7f3a5236c700 1 -- 172.21.15.89:0/852154839 shutdown_connections 2023-10-10T20:44:10.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.715+0000 7f3a5236c700 1 -- 172.21.15.89:0/852154839 wait complete. 2023-10-10T20:44:10.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.716+0000 7f3a5236c700 1 Processor -- start 2023-10-10T20:44:10.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.716+0000 7f3a5236c700 1 -- start start 2023-10-10T20:44:10.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.717+0000 7f3a5236c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a4c069560 0x7f3a4c169db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:10.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.717+0000 7f3a5236c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a4c159890 0x7f3a4c16af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:10.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.717+0000 7f3a5236c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a4c159c60 0x7f3a4c0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:10.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.717+0000 7f3a5236c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c0b4850 con 0x7f3a4c159890 2023-10-10T20:44:10.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.717+0000 7f3a5236c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c16a600 con 0x7f3a4c159c60 2023-10-10T20:44:10.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.717+0000 7f3a5236c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c16a8e0 con 0x7f3a4c069560 2023-10-10T20:44:10.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.717+0000 7f3a4b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a4c159890 0x7f3a4c16af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:10.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.717+0000 7f3a4b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a4c159890 0x7f3a4c16af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40228/0 (socket says 172.21.15.89:40228) 2023-10-10T20:44:10.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.717+0000 7f3a4b7fe700 1 -- 172.21.15.89:0/796701677 learned_addr learned my addr 172.21.15.89:0/796701677 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:10.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.717+0000 7f3a4bfff700 1 --2- 172.21.15.89:0/796701677 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a4c069560 0x7f3a4c169db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:10.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.717+0000 7f3a50909700 1 --2- 172.21.15.89:0/796701677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a4c159c60 0x7f3a4c0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:10.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.717+0000 7f3a4b7fe700 1 -- 172.21.15.89:0/796701677 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a4c069560 msgr2=0x7f3a4c169db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:10.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.717+0000 7f3a4b7fe700 1 --2- 172.21.15.89:0/796701677 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f3a4c069560 0x7f3a4c169db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:10.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.717+0000 7f3a4b7fe700 1 -- 172.21.15.89:0/796701677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a4c159c60 msgr2=0x7f3a4c0b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:10.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.717+0000 7f3a4b7fe700 1 --2- 172.21.15.89:0/796701677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a4c159c60 0x7f3a4c0b4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:10.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.717+0000 7f3a4b7fe700 1 -- 172.21.15.89:0/796701677 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c009420 con 0x7f3a4c159890 2023-10-10T20:44:10.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.717+0000 7f3a50909700 1 --2- 172.21.15.89:0/796701677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a4c159c60 0x7f3a4c0b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:44:10.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.717+0000 7f3a4b7fe700 1 --2- 172.21.15.89:0/796701677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a4c159890 0x7f3a4c16af80 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000c7e0 tx=0x7f3a40007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:10.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.718+0000 7f3a33fff700 1 -- 172.21.15.89:0/796701677 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40005b30 con 0x7f3a4c159890 2023-10-10T20:44:10.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.718+0000 7f3a33fff700 1 -- 172.21.15.89:0/796701677 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a40005c90 con 0x7f3a4c159890 2023-10-10T20:44:10.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.718+0000 7f3a5236c700 1 -- 172.21.15.89:0/796701677 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a4c16abc0 con 0x7f3a4c159890 2023-10-10T20:44:10.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.718+0000 7f3a33fff700 1 -- 172.21.15.89:0/796701677 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40007300 con 0x7f3a4c159890 2023-10-10T20:44:10.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.718+0000 7f3a5236c700 1 -- 172.21.15.89:0/796701677 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a4c16a2f0 con 0x7f3a4c159890 2023-10-10T20:44:10.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.719+0000 7f3a5236c700 1 -- 172.21.15.89:0/796701677 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a4c04c3c0 con 0x7f3a4c159890 2023-10-10T20:44:10.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.719+0000 7f3a33fff700 1 -- 172.21.15.89:0/796701677 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3a400076b0 con 0x7f3a4c159890 2023-10-10T20:44:10.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.719+0000 7f3a33fff700 1 --2- 172.21.15.89:0/796701677 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3a38067320 0x7f3a380697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:10.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.719+0000 7f3a33fff700 1 -- 172.21.15.89:0/796701677 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3a400841d0 con 0x7f3a4c159890 2023-10-10T20:44:10.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.719+0000 7f3a4bfff700 1 --2- 172.21.15.89:0/796701677 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3a38067320 0x7f3a380697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:10.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.720+0000 7f3a4bfff700 1 --2- 172.21.15.89:0/796701677 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3a38067320 0x7f3a380697d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0093f0 tx=0x7f3a3c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:10.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.723+0000 7f3a33fff700 1 -- 172.21.15.89:0/796701677 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a40050e60 con 0x7f3a4c159890 2023-10-10T20:44:10.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.931+0000 7f3a5236c700 1 -- 172.21.15.89:0/796701677 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a4c06a5b0 con 0x7f3a38067320 2023-10-10T20:44:10.934 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-10T20:44:10.932+0000 7f023b8aa700 -1 mgr.server reply reply (1) Operation not permitted WARNING: this will *PERMANENTLY DESTROY* all data stored in the filesystem 'cephfs'. If you are *ABSOLUTELY CERTAIN* that is what you want, re-issue the command followed by --yes-i-really-mean-it. 2023-10-10T20:44:10.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:10.932+0000 7f3a33fff700 1 -- 172.21.15.89:0/796701677 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: -1 WARNING: this will *PERMANENTLY DESTROY* all data stored in the filesystem 'cephfs'. If you are *ABSOLUTELY CERTAIN* that is what you want, re-issue the command followed by --yes-i-really-mean-it.) v1 ==== 204+0+0 (secure 0 0 0) 0x7f3a4c06a5b0 con 0x7f3a38067320 2023-10-10T20:44:10.936 INFO:teuthology.orchestra.run.smithi089.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in the filesystem 'cephfs'. If you are *ABSOLUTELY CERTAIN* that is what you want, re-issue the command followed by --yes-i-really-mean-it. 2023-10-10T20:44:10.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:44:10.940 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2023-10-10T20:44:11.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.056+0000 7f94ce03a700 1 Processor -- start 2023-10-10T20:44:11.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.056+0000 7f94ce03a700 1 -- start start 2023-10-10T20:44:11.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.057+0000 7f94ce03a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94c8159890 0x7f94c815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:11.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.057+0000 7f94ce03a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f94c8069560 0x7f94c80699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:11.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.057+0000 7f94ce03a700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c815a160 con 0x7f94c8159c60 2023-10-10T20:44:11.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.057+0000 7f94ce03a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c8069f20 con 0x7f94c8069560 2023-10-10T20:44:11.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.057+0000 7f94ce03a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c80d13a0 con 0x7f94c8159890 2023-10-10T20:44:11.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.057+0000 7f94c77fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94c8159890 0x7f94c815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:11.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.057+0000 7f94c7fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f94c8069560 0x7f94c80699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:11.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.057+0000 7f94c77fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94c8159890 0x7f94c815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:60854/0 (socket says 172.21.15.89:60854) 2023-10-10T20:44:11.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.057+0000 7f94c7fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f94c8069560 0x7f94c80699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57680/0 (socket says 172.21.15.89:57680) 2023-10-10T20:44:11.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.057+0000 7f94c77fe700 1 -- 172.21.15.89:0/73004400 learned_addr learned my addr 172.21.15.89:0/73004400 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:11.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.057+0000 7f94c77fe700 1 -- 172.21.15.89:0/73004400 >> v2:172.21.15.111:3300/0 conn(0x7f94c8069560 msgr2=0x7f94c80699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:44:11.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.057+0000 7f94c77fe700 1 --2- 172.21.15.89:0/73004400 >> v2:172.21.15.111:3300/0 conn(0x7f94c8069560 0x7f94c80699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:11.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.057+0000 7f94c77fe700 1 -- 172.21.15.89:0/73004400 >> v1:172.21.15.111:6789/0 conn(0x7f94c8159c60 legacy=0x7f94c815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:11.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.057+0000 7f94c77fe700 1 -- 172.21.15.89:0/73004400 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c815e140 con 0x7f94c8159890 2023-10-10T20:44:11.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.058+0000 7f94c77fe700 1 --2- 172.21.15.89:0/73004400 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94c8159890 0x7f94c815abd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f94b80099b0 tx=0x7f94b8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eb06461825d4697 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:11.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.058+0000 7f94c67fc700 1 -- 172.21.15.89:0/73004400 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b8009ea0 con 0x7f94c8159890 2023-10-10T20:44:11.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.058+0000 7f94c67fc700 1 -- 172.21.15.89:0/73004400 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b800b860 con 0x7f94c8159890 2023-10-10T20:44:11.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.058+0000 7f94c67fc700 1 -- 172.21.15.89:0/73004400 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b800bc00 con 0x7f94c8159890 2023-10-10T20:44:11.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.058+0000 7f94ce03a700 1 -- 172.21.15.89:0/73004400 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94c8159890 msgr2=0x7f94c815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:11.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.058+0000 7f94ce03a700 1 --2- 172.21.15.89:0/73004400 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94c8159890 0x7f94c815abd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f94b80099b0 tx=0x7f94b8009250 comp rx=0 tx=0).stop 2023-10-10T20:44:11.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.058+0000 7f94ce03a700 1 -- 172.21.15.89:0/73004400 shutdown_connections 2023-10-10T20:44:11.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.058+0000 7f94ce03a700 1 --2- 172.21.15.89:0/73004400 >> v2:172.21.15.111:3300/0 conn(0x7f94c8069560 0x7f94c80699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:11.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.058+0000 7f94ce03a700 1 --2- 172.21.15.89:0/73004400 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94c8159890 0x7f94c815abd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:11.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.058+0000 7f94ce03a700 1 -- 172.21.15.89:0/73004400 >> 172.21.15.89:0/73004400 conn(0x7f94c8068e20 msgr2=0x7f94c80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:44:11.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.058+0000 7f94ce03a700 1 -- 172.21.15.89:0/73004400 shutdown_connections 2023-10-10T20:44:11.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.058+0000 7f94ce03a700 1 -- 172.21.15.89:0/73004400 wait complete. 2023-10-10T20:44:11.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.060+0000 7f94ce03a700 1 Processor -- start 2023-10-10T20:44:11.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.060+0000 7f94ce03a700 1 -- start start 2023-10-10T20:44:11.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.060+0000 7f94ce03a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94c8069560 0x7f94c814cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:11.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.060+0000 7f94ce03a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94c8159890 0x7f94c80ce480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:11.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.060+0000 7f94ce03a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94c8159c60 0x7f94c80b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:11.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.060+0000 7f94ce03a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c814d180 con 0x7f94c8159c60 2023-10-10T20:44:11.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.060+0000 7f94ce03a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c80cd100 con 0x7f94c8069560 2023-10-10T20:44:11.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.060+0000 7f94ce03a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c80cd3e0 con 0x7f94c8159890 2023-10-10T20:44:11.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.060+0000 7f94c7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94c8159c60 0x7f94c80b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:11.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.060+0000 7f94c7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94c8159c60 0x7f94c80b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40232/0 (socket says 172.21.15.89:40232) 2023-10-10T20:44:11.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.060+0000 7f94c7fff700 1 -- 172.21.15.89:0/3495248397 learned_addr learned my addr 172.21.15.89:0/3495248397 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:44:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.061+0000 7f94c77fe700 1 --2- 172.21.15.89:0/3495248397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94c8069560 0x7f94c814cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.061+0000 7f94c6ffd700 1 --2- 172.21.15.89:0/3495248397 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94c8159890 0x7f94c80ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:11.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.061+0000 7f94c7fff700 1 -- 172.21.15.89:0/3495248397 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94c8159890 msgr2=0x7f94c80ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:11.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.061+0000 7f94c7fff700 1 --2- 172.21.15.89:0/3495248397 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94c8159890 0x7f94c80ce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:11.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.061+0000 7f94c7fff700 1 -- 172.21.15.89:0/3495248397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94c8069560 msgr2=0x7f94c814cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:44:11.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.061+0000 7f94c7fff700 1 --2- 172.21.15.89:0/3495248397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94c8069560 0x7f94c814cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:44:11.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.061+0000 7f94c7fff700 1 -- 172.21.15.89:0/3495248397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b80094f0 con 0x7f94c8159c60 2023-10-10T20:44:11.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.061+0000 7f94c6ffd700 1 --2- 172.21.15.89:0/3495248397 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f94c8159890 0x7f94c80ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:44:11.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.061+0000 7f94c7fff700 1 --2- 172.21.15.89:0/3495248397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94c8159c60 0x7f94c80b44b0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009df0 tx=0x7f94c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:11.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.061+0000 7f94af7fe700 1 -- 172.21.15.89:0/3495248397 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c000cb50 con 0x7f94c8159c60 2023-10-10T20:44:11.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.061+0000 7f94af7fe700 1 -- 172.21.15.89:0/3495248397 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c000f040 con 0x7f94c8159c60 2023-10-10T20:44:11.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.061+0000 7f94ce03a700 1 -- 172.21.15.89:0/3495248397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c80cd6c0 con 0x7f94c8159c60 2023-10-10T20:44:11.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.061+0000 7f94af7fe700 1 -- 172.21.15.89:0/3495248397 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0014480 con 0x7f94c8159c60 2023-10-10T20:44:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.061+0000 7f94ce03a700 1 -- 172.21.15.89:0/3495248397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c80cdc80 con 0x7f94c8159c60 2023-10-10T20:44:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.062+0000 7f94ce03a700 1 -- 172.21.15.89:0/3495248397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c804c3c0 con 0x7f94c8159c60 2023-10-10T20:44:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.063+0000 7f94af7fe700 1 -- 172.21.15.89:0/3495248397 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f94c0014830 con 0x7f94c8159c60 2023-10-10T20:44:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.063+0000 7f94af7fe700 1 --2- 172.21.15.89:0/3495248397 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f94b4067320 0x7f94b40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:44:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.063+0000 7f94af7fe700 1 -- 172.21.15.89:0/3495248397 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f94c0085610 con 0x7f94c8159c60 2023-10-10T20:44:11.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.063+0000 7f94c77fe700 1 --2- 172.21.15.89:0/3495248397 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f94b4067320 0x7f94b40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:44:11.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.063+0000 7f94c77fe700 1 --2- 172.21.15.89:0/3495248397 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f94b4067320 0x7f94b40697d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f94b80099b0 tx=0x7f94b8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:44:11.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.067+0000 7f94af7fe700 1 -- 172.21.15.89:0/3495248397 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94c0052290 con 0x7f94c8159c60 2023-10-10T20:44:11.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:44:11.278+0000 7f94ce03a700 1 -- 172.21.15.89:0/3495248397 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f94c8159050 con 0x7f94b4067320 2023-10-10T20:44:13.165 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:44:13.169 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:44:43.235 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:44:43.238 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:45:13.357 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:45:13.360 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:45:43.463 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:45:43.467 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:46:10.971 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-10-10T20:46:10.972 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:46:10.972 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:46:11.019 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:46:11.067 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:46:11.068 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:46:11.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.188+0000 7fda1b387700 1 Processor -- start 2023-10-10T20:46:11.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.189+0000 7fda1b387700 1 -- start start 2023-10-10T20:46:11.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.189+0000 7fda1b387700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fda14159890 0x7fda1415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.189+0000 7fda1b387700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fda14159c60 0x7fda1415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.189+0000 7fda1b387700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda1415db10 0x7fda14069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.189+0000 7fda1b387700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda1415df50 con 0x7fda14159890 2023-10-10T20:46:11.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.189+0000 7fda1b387700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda1415a160 con 0x7fda14159c60 2023-10-10T20:46:11.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.189+0000 7fda1b387700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda140d13a0 con 0x7fda1415db10 2023-10-10T20:46:11.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.190+0000 7fda19123700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fda14159890 0x7fda1415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.190+0000 7fda19123700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fda14159890 0x7fda1415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53812/0 (socket says 172.21.15.89:53812) 2023-10-10T20:46:11.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.190+0000 7fda19123700 1 -- 172.21.15.89:0/2986635939 learned_addr learned my addr 172.21.15.89:0/2986635939 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:11.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.190+0000 7fda18922700 1 --2- 172.21.15.89:0/2986635939 >> v2:172.21.15.111:3300/0 conn(0x7fda14159c60 0x7fda1415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.190+0000 7fda19924700 1 --2- 172.21.15.89:0/2986635939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda1415db10 0x7fda14069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.190+0000 7fda19123700 1 -- 172.21.15.89:0/2986635939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda1415db10 msgr2=0x7fda14069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:11.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.190+0000 7fda19123700 1 --2- 172.21.15.89:0/2986635939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda1415db10 0x7fda14069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.190+0000 7fda19123700 1 -- 172.21.15.89:0/2986635939 >> v2:172.21.15.111:3300/0 conn(0x7fda14159c60 msgr2=0x7fda1415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.190+0000 7fda19123700 1 --2- 172.21.15.89:0/2986635939 >> v2:172.21.15.111:3300/0 conn(0x7fda14159c60 0x7fda1415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.190+0000 7fda19123700 1 -- 172.21.15.89:0/2986635939 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1415e140 con 0x7fda14159890 2023-10-10T20:46:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.190+0000 7fda19123700 1 --2- 172.21.15.89:0/2986635939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda14159890 0x7fda1415abd0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fda140b67f0 tx=0x7fda10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec0e28637aecd2d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.190+0000 7fda0bfff700 1 -- 172.21.15.89:0/2986635939 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1000b640 con 0x7fda14159890 2023-10-10T20:46:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.190+0000 7fda0bfff700 1 -- 172.21.15.89:0/2986635939 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1000b7a0 con 0x7fda14159890 2023-10-10T20:46:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.190+0000 7fda0bfff700 1 -- 172.21.15.89:0/2986635939 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1000bb60 con 0x7fda14159890 2023-10-10T20:46:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.191+0000 7fda1b387700 1 -- 172.21.15.89:0/2986635939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda14159890 msgr2=0x7fda1415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.191+0000 7fda1b387700 1 --2- 172.21.15.89:0/2986635939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda14159890 0x7fda1415abd0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fda140b67f0 tx=0x7fda10009250 comp rx=0 tx=0).stop 2023-10-10T20:46:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.191+0000 7fda1b387700 1 -- 172.21.15.89:0/2986635939 shutdown_connections 2023-10-10T20:46:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.191+0000 7fda1b387700 1 --2- 172.21.15.89:0/2986635939 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda1415db10 0x7fda14069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.191+0000 7fda1b387700 1 --2- 172.21.15.89:0/2986635939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda14159890 0x7fda1415abd0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.191+0000 7fda1b387700 1 --2- 172.21.15.89:0/2986635939 >> v2:172.21.15.111:3300/0 conn(0x7fda14159c60 0x7fda1415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.191+0000 7fda1b387700 1 -- 172.21.15.89:0/2986635939 >> 172.21.15.89:0/2986635939 conn(0x7fda14068e20 msgr2=0x7fda140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:11.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.191+0000 7fda1b387700 1 -- 172.21.15.89:0/2986635939 shutdown_connections 2023-10-10T20:46:11.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.191+0000 7fda1b387700 1 -- 172.21.15.89:0/2986635939 wait complete. 2023-10-10T20:46:11.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.192+0000 7fda1b387700 1 Processor -- start 2023-10-10T20:46:11.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.192+0000 7fda1b387700 1 -- start start 2023-10-10T20:46:11.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.193+0000 7fda1b387700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda14159890 0x7fda140b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.193+0000 7fda1b387700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda14159c60 0x7fda140c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.193+0000 7fda1b387700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda1415db10 0x7fda140c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.193+0000 7fda1b387700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda14153650 con 0x7fda14159c60 2023-10-10T20:46:11.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.193+0000 7fda1b387700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda140c9dd0 con 0x7fda1415db10 2023-10-10T20:46:11.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.193+0000 7fda1b387700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda140b3630 con 0x7fda14159890 2023-10-10T20:46:11.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.193+0000 7fda18922700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda14159c60 0x7fda140c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.193+0000 7fda18922700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda14159c60 0x7fda140c9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53816/0 (socket says 172.21.15.89:53816) 2023-10-10T20:46:11.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.193+0000 7fda18922700 1 -- 172.21.15.89:0/3538106637 learned_addr learned my addr 172.21.15.89:0/3538106637 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:11.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.193+0000 7fda19123700 1 --2- 172.21.15.89:0/3538106637 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda14159890 0x7fda140b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.193+0000 7fda19924700 1 --2- 172.21.15.89:0/3538106637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda1415db10 0x7fda140c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.193+0000 7fda18922700 1 -- 172.21.15.89:0/3538106637 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda14159890 msgr2=0x7fda140b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:11.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.193+0000 7fda18922700 1 --2- 172.21.15.89:0/3538106637 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda14159890 0x7fda140b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.193+0000 7fda18922700 1 -- 172.21.15.89:0/3538106637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda1415db10 msgr2=0x7fda140c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:11.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.193+0000 7fda18922700 1 --2- 172.21.15.89:0/3538106637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda1415db10 0x7fda140c9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.193+0000 7fda18922700 1 -- 172.21.15.89:0/3538106637 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda100094f0 con 0x7fda14159c60 2023-10-10T20:46:11.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.194+0000 7fda18922700 1 --2- 172.21.15.89:0/3538106637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda14159c60 0x7fda140c9210 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fda0400ec80 tx=0x7fda040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:11.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.194+0000 7fda097fa700 1 -- 172.21.15.89:0/3538106637 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0400c850 con 0x7fda14159c60 2023-10-10T20:46:11.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.194+0000 7fda097fa700 1 -- 172.21.15.89:0/3538106637 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0400c9b0 con 0x7fda14159c60 2023-10-10T20:46:11.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.194+0000 7fda1b387700 1 -- 172.21.15.89:0/3538106637 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda140b3910 con 0x7fda14159c60 2023-10-10T20:46:11.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.194+0000 7fda097fa700 1 -- 172.21.15.89:0/3538106637 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0400cd50 con 0x7fda14159c60 2023-10-10T20:46:11.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.194+0000 7fda1b387700 1 -- 172.21.15.89:0/3538106637 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda140b3ed0 con 0x7fda14159c60 2023-10-10T20:46:11.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.195+0000 7fda1b387700 1 -- 172.21.15.89:0/3538106637 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1404c3c0 con 0x7fda14159c60 2023-10-10T20:46:11.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.195+0000 7fda097fa700 1 -- 172.21.15.89:0/3538106637 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fda04013680 con 0x7fda14159c60 2023-10-10T20:46:11.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.195+0000 7fda097fa700 1 --2- 172.21.15.89:0/3538106637 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd9f8067320 0x7fd9f80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.196+0000 7fda097fa700 1 -- 172.21.15.89:0/3538106637 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5792+0+0 (secure 0 0 0) 0x7fda04084eb0 con 0x7fda14159c60 2023-10-10T20:46:11.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.196+0000 7fda19123700 1 --2- 172.21.15.89:0/3538106637 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd9f8067320 0x7fd9f80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.196+0000 7fda19123700 1 --2- 172.21.15.89:0/3538106637 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd9f8067320 0x7fd9f80697d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fda140b67f0 tx=0x7fda10017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:11.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.200+0000 7fda097fa700 1 -- 172.21.15.89:0/3538106637 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda04051af0 con 0x7fda14159c60 2023-10-10T20:46:11.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.424+0000 7fda1b387700 1 -- 172.21.15.89:0/3538106637 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda1406a5b0 con 0x7fda14159c60 2023-10-10T20:46:11.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.424+0000 7fda097fa700 1 -- 172.21.15.89:0/3538106637 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+4604 (secure 0 0 0) 0x7fda04055110 con 0x7fda14159c60 2023-10-10T20:46:11.427 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:46:11.427 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":133,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/4215135154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":4215135154},{"type":"v1","addr":"172.21.15.89:6837","nonce":4215135154}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127},{"gid":7088,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/940990808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":940990808},{"type":"v1","addr":"172.21.15.111:6834","nonce":940990808}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127},{"gid":7091,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/92922277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":92922277},{"type":"v1","addr":"172.21.15.111:6835","nonce":92922277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:43:59.950797+0000","modified":"2023-10-10T20:44:03.611977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9135":{"gid":9135,"name":"a","rank":0,"incarnation":128,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6836/3558240892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3558240892},{"type":"v1","addr":"172.21.15.89:6836","nonce":3558240892}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-10T20:46:11.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.426+0000 7fda1b387700 1 -- 172.21.15.89:0/3538106637 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd9f8067320 msgr2=0x7fd9f80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:11.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.426+0000 7fda1b387700 1 --2- 172.21.15.89:0/3538106637 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd9f8067320 0x7fd9f80697d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fda140b67f0 tx=0x7fda10017000 comp rx=0 tx=0).stop 2023-10-10T20:46:11.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.426+0000 7fda1b387700 1 -- 172.21.15.89:0/3538106637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda14159c60 msgr2=0x7fda140c9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:11.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.426+0000 7fda1b387700 1 --2- 172.21.15.89:0/3538106637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda14159c60 0x7fda140c9210 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fda0400ec80 tx=0x7fda040091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.426+0000 7fda1b387700 1 -- 172.21.15.89:0/3538106637 shutdown_connections 2023-10-10T20:46:11.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.426+0000 7fda1b387700 1 --2- 172.21.15.89:0/3538106637 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd9f8067320 0x7fd9f80697d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.426+0000 7fda1b387700 1 --2- 172.21.15.89:0/3538106637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda1415db10 0x7fda140c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.426+0000 7fda1b387700 1 --2- 172.21.15.89:0/3538106637 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda14159890 0x7fda140b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.426+0000 7fda1b387700 1 --2- 172.21.15.89:0/3538106637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda14159c60 0x7fda140c9210 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.426+0000 7fda1b387700 1 -- 172.21.15.89:0/3538106637 >> 172.21.15.89:0/3538106637 conn(0x7fda14068e20 msgr2=0x7fda140c7070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:11.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.426+0000 7fda1b387700 1 -- 172.21.15.89:0/3538106637 shutdown_connections 2023-10-10T20:46:11.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.426+0000 7fda1b387700 1 -- 172.21.15.89:0/3538106637 wait complete. 2023-10-10T20:46:11.433 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 133 2023-10-10T20:46:11.440 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:46:11.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.554+0000 7f094ef26700 1 Processor -- start 2023-10-10T20:46:11.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.554+0000 7f094ef26700 1 -- start start 2023-10-10T20:46:11.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.555+0000 7f094ef26700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0948159420 0x7f09480b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.555+0000 7f094ef26700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f09481597f0 0x7f09480b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.555+0000 7f094ef26700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f094815d230 0x7f094815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.555+0000 7f094ef26700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f094815dbd0 con 0x7f094815d230 2023-10-10T20:46:11.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.555+0000 7f094ef26700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09480b11d0 con 0x7f0948159420 2023-10-10T20:46:11.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.555+0000 7f094ef26700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09480b14b0 con 0x7f09481597f0 2023-10-10T20:46:11.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.555+0000 7f094e725700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f094815d230 0x7f094815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.555+0000 7f094e725700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f094815d230 0x7f094815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53818/0 (socket says 172.21.15.89:53818) 2023-10-10T20:46:11.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.555+0000 7f094e725700 1 -- 172.21.15.89:0/851984068 learned_addr learned my addr 172.21.15.89:0/851984068 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:11.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.555+0000 7f094df24700 1 --2- 172.21.15.89:0/851984068 >> v2:172.21.15.111:3300/0 conn(0x7f0948159420 0x7f09480b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.555+0000 7f094d723700 1 --2- 172.21.15.89:0/851984068 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f09481597f0 0x7f09480b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.555+0000 7f094e725700 1 -- 172.21.15.89:0/851984068 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f09481597f0 msgr2=0x7f09480b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:11.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.556+0000 7f094e725700 1 --2- 172.21.15.89:0/851984068 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f09481597f0 0x7f09480b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.556+0000 7f094e725700 1 -- 172.21.15.89:0/851984068 >> v2:172.21.15.111:3300/0 conn(0x7f0948159420 msgr2=0x7f09480b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:11.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.556+0000 7f094e725700 1 --2- 172.21.15.89:0/851984068 >> v2:172.21.15.111:3300/0 conn(0x7f0948159420 0x7f09480b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.556+0000 7f094e725700 1 -- 172.21.15.89:0/851984068 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09480b1790 con 0x7f094815d230 2023-10-10T20:46:11.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.556+0000 7f094e725700 1 --2- 172.21.15.89:0/851984068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f094815d230 0x7f094815d690 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f09480b2f20 tx=0x7f0940009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b65324bf3fe2e31 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:11.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.556+0000 7f094cf22700 1 -- 172.21.15.89:0/851984068 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094000e070 con 0x7f094815d230 2023-10-10T20:46:11.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.556+0000 7f094cf22700 1 -- 172.21.15.89:0/851984068 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094000b970 con 0x7f094815d230 2023-10-10T20:46:11.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.556+0000 7f094cf22700 1 -- 172.21.15.89:0/851984068 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09400092e0 con 0x7f094815d230 2023-10-10T20:46:11.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.556+0000 7f094ef26700 1 -- 172.21.15.89:0/851984068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f094815d230 msgr2=0x7f094815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:11.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.556+0000 7f094ef26700 1 --2- 172.21.15.89:0/851984068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f094815d230 0x7f094815d690 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f09480b2f20 tx=0x7f0940009840 comp rx=0 tx=0).stop 2023-10-10T20:46:11.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.556+0000 7f094ef26700 1 -- 172.21.15.89:0/851984068 shutdown_connections 2023-10-10T20:46:11.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.556+0000 7f094ef26700 1 --2- 172.21.15.89:0/851984068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f094815d230 0x7f094815d690 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.556+0000 7f094ef26700 1 --2- 172.21.15.89:0/851984068 >> v2:172.21.15.111:3300/0 conn(0x7f0948159420 0x7f09480b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.556+0000 7f094ef26700 1 --2- 172.21.15.89:0/851984068 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f09481597f0 0x7f09480b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.556+0000 7f094ef26700 1 -- 172.21.15.89:0/851984068 >> 172.21.15.89:0/851984068 conn(0x7f09480d0660 msgr2=0x7f09480d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:11.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.556+0000 7f094ef26700 1 -- 172.21.15.89:0/851984068 shutdown_connections 2023-10-10T20:46:11.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.556+0000 7f094ef26700 1 -- 172.21.15.89:0/851984068 wait complete. 2023-10-10T20:46:11.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.557+0000 7f094ef26700 1 Processor -- start 2023-10-10T20:46:11.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.557+0000 7f094ef26700 1 -- start start 2023-10-10T20:46:11.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.558+0000 7f094ef26700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0948159420 0x7f0948155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.558+0000 7f094ef26700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09481597f0 0x7f094815c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.558+0000 7f094ef26700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f094815d230 0x7f094815c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.558+0000 7f094ef26700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094815bf70 con 0x7f0948159420 2023-10-10T20:46:11.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.558+0000 7f094ef26700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09480c7150 con 0x7f09481597f0 2023-10-10T20:46:11.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.558+0000 7f094ef26700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09480c75e0 con 0x7f094815d230 2023-10-10T20:46:11.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.558+0000 7f094df24700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0948159420 0x7f0948155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.558+0000 7f094df24700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0948159420 0x7f0948155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53820/0 (socket says 172.21.15.89:53820) 2023-10-10T20:46:11.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.558+0000 7f094df24700 1 -- 172.21.15.89:0/1492424803 learned_addr learned my addr 172.21.15.89:0/1492424803 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:11.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.558+0000 7f094d723700 1 --2- 172.21.15.89:0/1492424803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09481597f0 0x7f094815c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.558+0000 7f094e725700 1 --2- 172.21.15.89:0/1492424803 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f094815d230 0x7f094815c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.559+0000 7f094df24700 1 -- 172.21.15.89:0/1492424803 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f094815d230 msgr2=0x7f094815c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:11.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.559+0000 7f094df24700 1 --2- 172.21.15.89:0/1492424803 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f094815d230 0x7f094815c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.559+0000 7f094df24700 1 -- 172.21.15.89:0/1492424803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09481597f0 msgr2=0x7f094815c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:11.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.559+0000 7f094df24700 1 --2- 172.21.15.89:0/1492424803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09481597f0 0x7f094815c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.559+0000 7f094df24700 1 -- 172.21.15.89:0/1492424803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0940009ae0 con 0x7f0948159420 2023-10-10T20:46:11.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.559+0000 7f094d723700 1 --2- 172.21.15.89:0/1492424803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09481597f0 0x7f094815c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:11.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.559+0000 7f094e725700 1 --2- 172.21.15.89:0/1492424803 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f094815d230 0x7f094815c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:11.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.559+0000 7f094df24700 1 --2- 172.21.15.89:0/1492424803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0948159420 0x7f0948155150 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f094400ec80 tx=0x7f09440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:11.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.559+0000 7f093dffb700 1 -- 172.21.15.89:0/1492424803 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094400c9d0 con 0x7f0948159420 2023-10-10T20:46:11.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.559+0000 7f093dffb700 1 -- 172.21.15.89:0/1492424803 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094400cb30 con 0x7f0948159420 2023-10-10T20:46:11.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.559+0000 7f094ef26700 1 -- 172.21.15.89:0/1492424803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09480c6540 con 0x7f0948159420 2023-10-10T20:46:11.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.559+0000 7f093dffb700 1 -- 172.21.15.89:0/1492424803 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0944013470 con 0x7f0948159420 2023-10-10T20:46:11.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.559+0000 7f094ef26700 1 -- 172.21.15.89:0/1492424803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09480c6b00 con 0x7f0948159420 2023-10-10T20:46:11.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.560+0000 7f094ef26700 1 -- 172.21.15.89:0/1492424803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f094804c3c0 con 0x7f0948159420 2023-10-10T20:46:11.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.560+0000 7f093dffb700 1 -- 172.21.15.89:0/1492424803 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0944013820 con 0x7f0948159420 2023-10-10T20:46:11.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.561+0000 7f093dffb700 1 --2- 172.21.15.89:0/1492424803 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f092c067320 0x7f092c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.561+0000 7f093dffb700 1 -- 172.21.15.89:0/1492424803 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f0944085080 con 0x7f0948159420 2023-10-10T20:46:11.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.561+0000 7f094d723700 1 --2- 172.21.15.89:0/1492424803 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f092c067320 0x7f092c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.561+0000 7f094d723700 1 --2- 172.21.15.89:0/1492424803 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f092c067320 0x7f092c0697d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0948158e10 tx=0x7f0938008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:11.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.565+0000 7f093dffb700 1 -- 172.21.15.89:0/1492424803 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0944051cc0 con 0x7f0948159420 2023-10-10T20:46:11.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.751+0000 7f094ef26700 1 -- 172.21.15.89:0/1492424803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09480b1a70 con 0x7f0948159420 2023-10-10T20:46:11.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.751+0000 7f093dffb700 1 -- 172.21.15.89:0/1492424803 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+15509 (secure 0 0 0) 0x7f09440552e0 con 0x7f0948159420 2023-10-10T20:46:11.753 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:46:11.754 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":200,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:44:24.011501+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:43:58.313137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-10T20:43:58.914819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"34.8","mappings":[{"from":0,"to":1}]},{"pgid":"34.18","mappings":[{"from":0,"to":2}]},{"pgid":"34.1f","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:46:11.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.753+0000 7f094ef26700 1 -- 172.21.15.89:0/1492424803 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f092c067320 msgr2=0x7f092c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:11.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.753+0000 7f094ef26700 1 --2- 172.21.15.89:0/1492424803 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f092c067320 0x7f092c0697d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0948158e10 tx=0x7f0938008040 comp rx=0 tx=0).stop 2023-10-10T20:46:11.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.753+0000 7f094ef26700 1 -- 172.21.15.89:0/1492424803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0948159420 msgr2=0x7f0948155150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:11.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.753+0000 7f094ef26700 1 --2- 172.21.15.89:0/1492424803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0948159420 0x7f0948155150 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f094400ec80 tx=0x7f09440091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.754+0000 7f094ef26700 1 -- 172.21.15.89:0/1492424803 shutdown_connections 2023-10-10T20:46:11.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.754+0000 7f094ef26700 1 --2- 172.21.15.89:0/1492424803 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f094815d230 0x7f094815c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.754+0000 7f094ef26700 1 --2- 172.21.15.89:0/1492424803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0948159420 0x7f0948155150 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.754+0000 7f094ef26700 1 --2- 172.21.15.89:0/1492424803 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f092c067320 0x7f092c0697d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.754+0000 7f094ef26700 1 --2- 172.21.15.89:0/1492424803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09481597f0 0x7f094815c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.754+0000 7f094ef26700 1 -- 172.21.15.89:0/1492424803 >> 172.21.15.89:0/1492424803 conn(0x7f09480d0660 msgr2=0x7f0948069bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:11.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.754+0000 7f094ef26700 1 -- 172.21.15.89:0/1492424803 shutdown_connections 2023-10-10T20:46:11.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.754+0000 7f094ef26700 1 -- 172.21.15.89:0/1492424803 wait complete. 2023-10-10T20:46:11.767 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:34:28.693226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 33, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:43:58.313137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '197', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 34, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:43:58.914819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '198', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:46:11.768 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:46:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.883+0000 7fcb6a5ef700 1 Processor -- start 2023-10-10T20:46:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.883+0000 7fcb6a5ef700 1 -- start start 2023-10-10T20:46:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.884+0000 7fcb6a5ef700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcb640c93a0 0x7fcb6406c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.884+0000 7fcb6a5ef700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb640c9770 0x7fcb640caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.884+0000 7fcb6a5ef700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb640af490 con 0x7fcb640cb400 2023-10-10T20:46:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.884+0000 7fcb6a5ef700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb640af770 con 0x7fcb640c93a0 2023-10-10T20:46:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.884+0000 7fcb6a5ef700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640afa50 con 0x7fcb640c9770 2023-10-10T20:46:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.884+0000 7fcb69dee700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcb640cb400 0x7fcb640c7fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59526/0 (socket says 172.21.15.89:59526) 2023-10-10T20:46:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.884+0000 7fcb69dee700 1 -- 172.21.15.89:0/488211997 learned_addr learned my addr 172.21.15.89:0/488211997 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.885+0000 7fcb68dec700 1 --2- 172.21.15.89:0/488211997 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb640c9770 0x7fcb640caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.885+0000 7fcb695ed700 1 --2- 172.21.15.89:0/488211997 >> v2:172.21.15.111:3300/0 conn(0x7fcb640c93a0 0x7fcb6406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.885+0000 7fcb5bfff700 1 -- 172.21.15.89:0/488211997 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1993201842 0 0) 0x7fcb640af490 con 0x7fcb640cb400 2023-10-10T20:46:11.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.885+0000 7fcb5bfff700 1 -- 172.21.15.89:0/488211997 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb50003570 con 0x7fcb640cb400 2023-10-10T20:46:11.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.885+0000 7fcb68dec700 1 -- 172.21.15.89:0/488211997 >> v2:172.21.15.111:3300/0 conn(0x7fcb640c93a0 msgr2=0x7fcb6406c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:11.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.885+0000 7fcb68dec700 1 --2- 172.21.15.89:0/488211997 >> v2:172.21.15.111:3300/0 conn(0x7fcb640c93a0 0x7fcb6406c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.885+0000 7fcb68dec700 1 -- 172.21.15.89:0/488211997 >> v1:172.21.15.89:6789/0 conn(0x7fcb640cb400 legacy=0x7fcb640c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:11.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.885+0000 7fcb68dec700 1 -- 172.21.15.89:0/488211997 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb640c7ab0 con 0x7fcb640c9770 2023-10-10T20:46:11.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.885+0000 7fcb68dec700 1 --2- 172.21.15.89:0/488211997 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb640c9770 0x7fcb640caec0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcb540098d0 tx=0x7fcb54009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef723c3f2bdd6ef5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:11.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.885+0000 7fcb5bfff700 1 -- 172.21.15.89:0/488211997 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54005c10 con 0x7fcb640c9770 2023-10-10T20:46:11.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.885+0000 7fcb5bfff700 1 -- 172.21.15.89:0/488211997 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb54005d70 con 0x7fcb640c9770 2023-10-10T20:46:11.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.885+0000 7fcb5bfff700 1 -- 172.21.15.89:0/488211997 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54006240 con 0x7fcb640c9770 2023-10-10T20:46:11.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.886+0000 7fcb6a5ef700 1 -- 172.21.15.89:0/488211997 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb640c9770 msgr2=0x7fcb640caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:11.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.886+0000 7fcb6a5ef700 1 --2- 172.21.15.89:0/488211997 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb640c9770 0x7fcb640caec0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcb540098d0 tx=0x7fcb54009120 comp rx=0 tx=0).stop 2023-10-10T20:46:11.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.886+0000 7fcb6a5ef700 1 -- 172.21.15.89:0/488211997 shutdown_connections 2023-10-10T20:46:11.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.886+0000 7fcb6a5ef700 1 --2- 172.21.15.89:0/488211997 >> v2:172.21.15.111:3300/0 conn(0x7fcb640c93a0 0x7fcb6406c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.886+0000 7fcb6a5ef700 1 --2- 172.21.15.89:0/488211997 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb640c9770 0x7fcb640caec0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.886+0000 7fcb6a5ef700 1 -- 172.21.15.89:0/488211997 >> 172.21.15.89:0/488211997 conn(0x7fcb640cddc0 msgr2=0x7fcb640c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:11.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.886+0000 7fcb6a5ef700 1 -- 172.21.15.89:0/488211997 shutdown_connections 2023-10-10T20:46:11.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.886+0000 7fcb6a5ef700 1 -- 172.21.15.89:0/488211997 wait complete. 2023-10-10T20:46:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.887+0000 7fcb6a5ef700 1 Processor -- start 2023-10-10T20:46:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.887+0000 7fcb6a5ef700 1 -- start start 2023-10-10T20:46:11.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.888+0000 7fcb6a5ef700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb640c93a0 0x7fcb641688f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.888+0000 7fcb6a5ef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb640c9770 0x7fcb6414fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.888+0000 7fcb6a5ef700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb640cb400 0x7fcb64150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.888+0000 7fcb6a5ef700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64168e30 con 0x7fcb640c9770 2023-10-10T20:46:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.888+0000 7fcb6a5ef700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640b0330 con 0x7fcb640c93a0 2023-10-10T20:46:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.888+0000 7fcb6a5ef700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640b23b0 con 0x7fcb640cb400 2023-10-10T20:46:11.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.888+0000 7fcb68dec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb640c9770 0x7fcb6414fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.888+0000 7fcb68dec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb640c9770 0x7fcb6414fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53826/0 (socket says 172.21.15.89:53826) 2023-10-10T20:46:11.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.888+0000 7fcb68dec700 1 -- 172.21.15.89:0/1105178813 learned_addr learned my addr 172.21.15.89:0/1105178813 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:11.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.888+0000 7fcb69dee700 1 --2- 172.21.15.89:0/1105178813 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb640cb400 0x7fcb64150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.888+0000 7fcb695ed700 1 --2- 172.21.15.89:0/1105178813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb640c93a0 0x7fcb641688f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.888+0000 7fcb68dec700 1 -- 172.21.15.89:0/1105178813 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb640cb400 msgr2=0x7fcb64150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:11.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.888+0000 7fcb68dec700 1 --2- 172.21.15.89:0/1105178813 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb640cb400 0x7fcb64150230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.888+0000 7fcb68dec700 1 -- 172.21.15.89:0/1105178813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb640c93a0 msgr2=0x7fcb641688f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:11.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.888+0000 7fcb68dec700 1 --2- 172.21.15.89:0/1105178813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb640c93a0 0x7fcb641688f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:11.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.888+0000 7fcb68dec700 1 -- 172.21.15.89:0/1105178813 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb540093c0 con 0x7fcb640c9770 2023-10-10T20:46:11.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.889+0000 7fcb68dec700 1 --2- 172.21.15.89:0/1105178813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb640c9770 0x7fcb6414fcf0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fcb640afb90 tx=0x7fcb54016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:11.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.889+0000 7fcb597fa700 1 -- 172.21.15.89:0/1105178813 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5401a700 con 0x7fcb640c9770 2023-10-10T20:46:11.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.889+0000 7fcb597fa700 1 -- 172.21.15.89:0/1105178813 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5401a860 con 0x7fcb640c9770 2023-10-10T20:46:11.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.889+0000 7fcb6a5ef700 1 -- 172.21.15.89:0/1105178813 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb640b2690 con 0x7fcb640c9770 2023-10-10T20:46:11.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.889+0000 7fcb597fa700 1 -- 172.21.15.89:0/1105178813 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5401ac20 con 0x7fcb640c9770 2023-10-10T20:46:11.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.889+0000 7fcb6a5ef700 1 -- 172.21.15.89:0/1105178813 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb640b2c50 con 0x7fcb640c9770 2023-10-10T20:46:11.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.890+0000 7fcb6a5ef700 1 -- 172.21.15.89:0/1105178813 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6406d560 con 0x7fcb640c9770 2023-10-10T20:46:11.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.890+0000 7fcb597fa700 1 -- 172.21.15.89:0/1105178813 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcb54011420 con 0x7fcb640c9770 2023-10-10T20:46:11.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.890+0000 7fcb597fa700 1 --2- 172.21.15.89:0/1105178813 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb48067320 0x7fcb480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:11.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.891+0000 7fcb597fa700 1 -- 172.21.15.89:0/1105178813 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5792+0+0 (secure 0 0 0) 0x7fcb540861c0 con 0x7fcb640c9770 2023-10-10T20:46:11.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.891+0000 7fcb695ed700 1 --2- 172.21.15.89:0/1105178813 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb48067320 0x7fcb480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:11.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.891+0000 7fcb695ed700 1 --2- 172.21.15.89:0/1105178813 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb48067320 0x7fcb480697d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009fd0 tx=0x7fcb5c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:11.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:11.895+0000 7fcb597fa700 1 -- 172.21.15.89:0/1105178813 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb54052e00 con 0x7fcb640c9770 2023-10-10T20:46:12.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.078+0000 7fcb6a5ef700 1 -- 172.21.15.89:0/1105178813 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb6406b500 con 0x7fcb640c9770 2023-10-10T20:46:12.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.079+0000 7fcb597fa700 1 -- 172.21.15.89:0/1105178813 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v200) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb5402b0a0 con 0x7fcb640c9770 2023-10-10T20:46:12.081 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-10T20:46:12.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.080+0000 7fcb6a5ef700 1 -- 172.21.15.89:0/1105178813 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb48067320 msgr2=0x7fcb480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:12.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.080+0000 7fcb6a5ef700 1 --2- 172.21.15.89:0/1105178813 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb48067320 0x7fcb480697d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009fd0 tx=0x7fcb5c00b040 comp rx=0 tx=0).stop 2023-10-10T20:46:12.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.080+0000 7fcb6a5ef700 1 -- 172.21.15.89:0/1105178813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb640c9770 msgr2=0x7fcb6414fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:12.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.080+0000 7fcb6a5ef700 1 --2- 172.21.15.89:0/1105178813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb640c9770 0x7fcb6414fcf0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fcb640afb90 tx=0x7fcb54016000 comp rx=0 tx=0).stop 2023-10-10T20:46:12.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.081+0000 7fcb6a5ef700 1 -- 172.21.15.89:0/1105178813 shutdown_connections 2023-10-10T20:46:12.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.081+0000 7fcb6a5ef700 1 --2- 172.21.15.89:0/1105178813 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcb640cb400 0x7fcb64150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.081+0000 7fcb6a5ef700 1 --2- 172.21.15.89:0/1105178813 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcb48067320 0x7fcb480697d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.081+0000 7fcb6a5ef700 1 --2- 172.21.15.89:0/1105178813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb640c93a0 0x7fcb641688f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.081+0000 7fcb6a5ef700 1 --2- 172.21.15.89:0/1105178813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb640c9770 0x7fcb6414fcf0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.081+0000 7fcb6a5ef700 1 -- 172.21.15.89:0/1105178813 >> 172.21.15.89:0/1105178813 conn(0x7fcb640cddc0 msgr2=0x7fcb640c2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.081+0000 7fcb6a5ef700 1 -- 172.21.15.89:0/1105178813 shutdown_connections 2023-10-10T20:46:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.081+0000 7fcb6a5ef700 1 -- 172.21.15.89:0/1105178813 wait complete. 2023-10-10T20:46:12.094 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:46:12.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.211+0000 7fad722a6700 1 Processor -- start 2023-10-10T20:46:12.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.211+0000 7fad722a6700 1 -- start start 2023-10-10T20:46:12.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.212+0000 7fad722a6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fad6c0b16d0 0x7fad6c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.212+0000 7fad722a6700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fad6c0b33b0 0x7fad6c0b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.212+0000 7fad722a6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad6c0b2080 con 0x7fad6c0b4e70 2023-10-10T20:46:12.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.212+0000 7fad722a6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad6c0cfb00 con 0x7fad6c0b16d0 2023-10-10T20:46:12.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.212+0000 7fad722a6700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad6c0b5c40 con 0x7fad6c0b33b0 2023-10-10T20:46:12.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.212+0000 7fad712a4700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fad6c0b4e70 0x7fad6c0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59542/0 (socket says 172.21.15.89:59542) 2023-10-10T20:46:12.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.212+0000 7fad70aa3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fad6c0b16d0 0x7fad6c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.212+0000 7fad712a4700 1 -- 172.21.15.89:0/836992219 learned_addr learned my addr 172.21.15.89:0/836992219 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:12.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.212+0000 7fad71aa5700 1 --2- 172.21.15.89:0/836992219 >> v2:172.21.15.111:3300/0 conn(0x7fad6c0b33b0 0x7fad6c0b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.212+0000 7fad70aa3700 1 -- 172.21.15.89:0/836992219 >> v2:172.21.15.111:3300/0 conn(0x7fad6c0b33b0 msgr2=0x7fad6c0b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:12.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.212+0000 7fad70aa3700 1 --2- 172.21.15.89:0/836992219 >> v2:172.21.15.111:3300/0 conn(0x7fad6c0b33b0 0x7fad6c0b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.212+0000 7fad70aa3700 1 -- 172.21.15.89:0/836992219 >> v1:172.21.15.89:6789/0 conn(0x7fad6c0b4e70 legacy=0x7fad6c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:12.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.212+0000 7fad70aa3700 1 -- 172.21.15.89:0/836992219 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c0b5f20 con 0x7fad6c0b16d0 2023-10-10T20:46:12.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.212+0000 7fad6bfff700 1 -- 172.21.15.89:0/836992219 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638230045 0 0) 0x7fad6c0b2080 con 0x7fad6c0b4e70 2023-10-10T20:46:12.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.213+0000 7fad70aa3700 1 --2- 172.21.15.89:0/836992219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad6c0b16d0 0x7fad6c0b1b40 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fad6c071380 tx=0x7fad600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73288ce009d1171e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:12.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.213+0000 7fad6bfff700 1 -- 172.21.15.89:0/836992219 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6000b6b0 con 0x7fad6c0b16d0 2023-10-10T20:46:12.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.213+0000 7fad6bfff700 1 -- 172.21.15.89:0/836992219 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6000b810 con 0x7fad6c0b16d0 2023-10-10T20:46:12.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.213+0000 7fad6bfff700 1 -- 172.21.15.89:0/836992219 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6000bbb0 con 0x7fad6c0b16d0 2023-10-10T20:46:12.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.213+0000 7fad722a6700 1 -- 172.21.15.89:0/836992219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad6c0b16d0 msgr2=0x7fad6c0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:12.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.213+0000 7fad722a6700 1 --2- 172.21.15.89:0/836992219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad6c0b16d0 0x7fad6c0b1b40 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fad6c071380 tx=0x7fad600091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.213+0000 7fad722a6700 1 -- 172.21.15.89:0/836992219 shutdown_connections 2023-10-10T20:46:12.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.213+0000 7fad722a6700 1 --2- 172.21.15.89:0/836992219 >> v2:172.21.15.111:3300/0 conn(0x7fad6c0b33b0 0x7fad6c0b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.213+0000 7fad722a6700 1 --2- 172.21.15.89:0/836992219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad6c0b16d0 0x7fad6c0b1b40 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.213+0000 7fad722a6700 1 -- 172.21.15.89:0/836992219 >> 172.21.15.89:0/836992219 conn(0x7fad6c0d10c0 msgr2=0x7fad6c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:12.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.213+0000 7fad722a6700 1 -- 172.21.15.89:0/836992219 shutdown_connections 2023-10-10T20:46:12.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.213+0000 7fad722a6700 1 -- 172.21.15.89:0/836992219 wait complete. 2023-10-10T20:46:12.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.215+0000 7fad722a6700 1 Processor -- start 2023-10-10T20:46:12.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.215+0000 7fad722a6700 1 -- start start 2023-10-10T20:46:12.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.215+0000 7fad722a6700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad6c0b16d0 0x7fad6c0cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.215+0000 7fad722a6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad6c0b33b0 0x7fad6c0cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.215+0000 7fad722a6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad6c0b4e70 0x7fad6c0cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.215+0000 7fad722a6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c0c8ed0 con 0x7fad6c0b33b0 2023-10-10T20:46:12.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.215+0000 7fad722a6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c0cc0e0 con 0x7fad6c0b4e70 2023-10-10T20:46:12.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.215+0000 7fad722a6700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c0ca360 con 0x7fad6c0b16d0 2023-10-10T20:46:12.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.216+0000 7fad70aa3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad6c0b33b0 0x7fad6c0cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.216+0000 7fad70aa3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad6c0b33b0 0x7fad6c0cb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53856/0 (socket says 172.21.15.89:53856) 2023-10-10T20:46:12.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.216+0000 7fad70aa3700 1 -- 172.21.15.89:0/1623879019 learned_addr learned my addr 172.21.15.89:0/1623879019 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:12.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.216+0000 7fad71aa5700 1 --2- 172.21.15.89:0/1623879019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad6c0b4e70 0x7fad6c0cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.216+0000 7fad712a4700 1 --2- 172.21.15.89:0/1623879019 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad6c0b16d0 0x7fad6c0cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.216+0000 7fad70aa3700 1 -- 172.21.15.89:0/1623879019 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad6c0b16d0 msgr2=0x7fad6c0cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:12.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.216+0000 7fad70aa3700 1 --2- 172.21.15.89:0/1623879019 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad6c0b16d0 0x7fad6c0cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.216+0000 7fad70aa3700 1 -- 172.21.15.89:0/1623879019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad6c0b4e70 msgr2=0x7fad6c0cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:12.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.216+0000 7fad70aa3700 1 --2- 172.21.15.89:0/1623879019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad6c0b4e70 0x7fad6c0cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.216+0000 7fad70aa3700 1 -- 172.21.15.89:0/1623879019 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad60009420 con 0x7fad6c0b33b0 2023-10-10T20:46:12.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.216+0000 7fad71aa5700 1 --2- 172.21.15.89:0/1623879019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad6c0b4e70 0x7fad6c0cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:12.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.216+0000 7fad712a4700 1 --2- 172.21.15.89:0/1623879019 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad6c0b16d0 0x7fad6c0cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:12.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.216+0000 7fad70aa3700 1 --2- 172.21.15.89:0/1623879019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad6c0b33b0 0x7fad6c0cb660 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fad6c070dd0 tx=0x7fad60017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:12.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.216+0000 7fad697fa700 1 -- 172.21.15.89:0/1623879019 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60006af0 con 0x7fad6c0b33b0 2023-10-10T20:46:12.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.216+0000 7fad697fa700 1 -- 172.21.15.89:0/1623879019 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6000bb90 con 0x7fad6c0b33b0 2023-10-10T20:46:12.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.216+0000 7fad722a6700 1 -- 172.21.15.89:0/1623879019 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6c0ca640 con 0x7fad6c0b33b0 2023-10-10T20:46:12.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.216+0000 7fad697fa700 1 -- 172.21.15.89:0/1623879019 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60011650 con 0x7fad6c0b33b0 2023-10-10T20:46:12.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.216+0000 7fad722a6700 1 -- 172.21.15.89:0/1623879019 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad6c0cac00 con 0x7fad6c0b33b0 2023-10-10T20:46:12.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.217+0000 7fad722a6700 1 -- 172.21.15.89:0/1623879019 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad6c0cadf0 con 0x7fad6c0b33b0 2023-10-10T20:46:12.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.217+0000 7fad697fa700 1 -- 172.21.15.89:0/1623879019 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fad60011a40 con 0x7fad6c0b33b0 2023-10-10T20:46:12.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.218+0000 7fad697fa700 1 --2- 172.21.15.89:0/1623879019 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fad58067320 0x7fad580697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.218+0000 7fad697fa700 1 -- 172.21.15.89:0/1623879019 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5792+0+0 (secure 0 0 0) 0x7fad60086640 con 0x7fad6c0b33b0 2023-10-10T20:46:12.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.218+0000 7fad712a4700 1 --2- 172.21.15.89:0/1623879019 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fad58067320 0x7fad580697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.218+0000 7fad712a4700 1 --2- 172.21.15.89:0/1623879019 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fad58067320 0x7fad580697d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fad5c001560 tx=0x7fad5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:12.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.222+0000 7fad697fa700 1 -- 172.21.15.89:0/1623879019 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad60053280 con 0x7fad6c0b33b0 2023-10-10T20:46:12.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.403+0000 7fad722a6700 1 -- 172.21.15.89:0/1623879019 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fad6c0cf4b0 con 0x7fad6c0b33b0 2023-10-10T20:46:12.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.403+0000 7fad697fa700 1 -- 172.21.15.89:0/1623879019 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v200) v1 ==== 104+0+11 (secure 0 0 0) 0x7fad600568a0 con 0x7fad6c0b33b0 2023-10-10T20:46:12.405 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-10T20:46:12.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.405+0000 7fad722a6700 1 -- 172.21.15.89:0/1623879019 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fad58067320 msgr2=0x7fad580697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:12.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.405+0000 7fad722a6700 1 --2- 172.21.15.89:0/1623879019 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fad58067320 0x7fad580697d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fad5c001560 tx=0x7fad5c009040 comp rx=0 tx=0).stop 2023-10-10T20:46:12.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.405+0000 7fad722a6700 1 -- 172.21.15.89:0/1623879019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad6c0b33b0 msgr2=0x7fad6c0cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:12.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.405+0000 7fad722a6700 1 --2- 172.21.15.89:0/1623879019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad6c0b33b0 0x7fad6c0cb660 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fad6c070dd0 tx=0x7fad60017000 comp rx=0 tx=0).stop 2023-10-10T20:46:12.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.405+0000 7fad722a6700 1 -- 172.21.15.89:0/1623879019 shutdown_connections 2023-10-10T20:46:12.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.405+0000 7fad722a6700 1 --2- 172.21.15.89:0/1623879019 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fad58067320 0x7fad580697d0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.405+0000 7fad722a6700 1 --2- 172.21.15.89:0/1623879019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad6c0b4e70 0x7fad6c0cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.405+0000 7fad722a6700 1 --2- 172.21.15.89:0/1623879019 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad6c0b16d0 0x7fad6c0cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.405+0000 7fad722a6700 1 --2- 172.21.15.89:0/1623879019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad6c0b33b0 0x7fad6c0cb660 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.405+0000 7fad722a6700 1 -- 172.21.15.89:0/1623879019 >> 172.21.15.89:0/1623879019 conn(0x7fad6c0d10c0 msgr2=0x7fad6c15c910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:12.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.406+0000 7fad722a6700 1 -- 172.21.15.89:0/1623879019 shutdown_connections 2023-10-10T20:46:12.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.406+0000 7fad722a6700 1 -- 172.21.15.89:0/1623879019 wait complete. 2023-10-10T20:46:12.419 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:46:12.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.535+0000 7f1b9359e700 1 Processor -- start 2023-10-10T20:46:12.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.535+0000 7f1b9359e700 1 -- start start 2023-10-10T20:46:12.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b9359e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b940b4e70 0x7f1b940ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b9359e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1b940b16d0 0x7f1b940b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b9359e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b940b2080 con 0x7f1b9415d240 2023-10-10T20:46:12.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b9359e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b940cfb00 con 0x7f1b940b16d0 2023-10-10T20:46:12.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b9359e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b940b5c40 con 0x7f1b940b4e70 2023-10-10T20:46:12.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b92d9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1b9415d240 0x7f1b9415d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59558/0 (socket says 172.21.15.89:59558) 2023-10-10T20:46:12.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b92d9d700 1 -- 172.21.15.89:0/2457429166 learned_addr learned my addr 172.21.15.89:0/2457429166 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:12.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b9259c700 1 --2- 172.21.15.89:0/2457429166 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b940b4e70 0x7f1b940ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b8bfff700 1 --2- 172.21.15.89:0/2457429166 >> v2:172.21.15.111:3300/0 conn(0x7f1b940b16d0 0x7f1b940b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b91d9b700 1 -- 172.21.15.89:0/2457429166 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967936793 0 0) 0x7f1b940b2080 con 0x7f1b9415d240 2023-10-10T20:46:12.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b91d9b700 1 -- 172.21.15.89:0/2457429166 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b80003570 con 0x7f1b9415d240 2023-10-10T20:46:12.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b91d9b700 1 -- 172.21.15.89:0/2457429166 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3250384757 0 0) 0x7f1b80003570 con 0x7f1b9415d240 2023-10-10T20:46:12.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b91d9b700 1 -- 172.21.15.89:0/2457429166 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b940b4e70 msgr2=0x7f1b940ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:12.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b91d9b700 1 --2- 172.21.15.89:0/2457429166 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b940b4e70 0x7f1b940ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b91d9b700 1 -- 172.21.15.89:0/2457429166 >> v2:172.21.15.111:3300/0 conn(0x7f1b940b16d0 msgr2=0x7f1b940b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:12.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b91d9b700 1 --2- 172.21.15.89:0/2457429166 >> v2:172.21.15.111:3300/0 conn(0x7f1b940b16d0 0x7f1b940b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b91d9b700 1 -- 172.21.15.89:0/2457429166 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b940b5f20 con 0x7f1b9415d240 2023-10-10T20:46:12.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b91d9b700 1 -- 172.21.15.89:0/2457429166 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f1b8c0017f0 con 0x7f1b9415d240 2023-10-10T20:46:12.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b91d9b700 1 -- 172.21.15.89:0/2457429166 >> v1:172.21.15.89:6789/0 conn(0x7f1b9415d240 legacy=0x7f1b9415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:12.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b91d9b700 1 --2- 172.21.15.89:0/2457429166 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b80003a60 0x7f1b80003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b91d9b700 1 --2- 172.21.15.89:0/2457429166 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b940b4e70 0x7f1b80004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b91d9b700 1 --2- 172.21.15.89:0/2457429166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b80004d50 0x7f1b800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b91d9b700 1 -- 172.21.15.89:0/2457429166 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b940cfb00 con 0x7f1b80004d50 2023-10-10T20:46:12.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b91d9b700 1 -- 172.21.15.89:0/2457429166 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b940b5c40 con 0x7f1b80003a60 2023-10-10T20:46:12.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b91d9b700 1 -- 172.21.15.89:0/2457429166 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8000ba60 con 0x7f1b940b4e70 2023-10-10T20:46:12.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.536+0000 7f1b92d9d700 1 --2- 172.21.15.89:0/2457429166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b80004d50 0x7f1b800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.537+0000 7f1b9259c700 1 --2- 172.21.15.89:0/2457429166 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b80003a60 0x7f1b80003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.537+0000 7f1b8bfff700 1 --2- 172.21.15.89:0/2457429166 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b940b4e70 0x7f1b80004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.537+0000 7f1b92d9d700 1 -- 172.21.15.89:0/2457429166 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b940b4e70 msgr2=0x7f1b80004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:12.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.537+0000 7f1b92d9d700 1 --2- 172.21.15.89:0/2457429166 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b940b4e70 0x7f1b80004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.537+0000 7f1b92d9d700 1 -- 172.21.15.89:0/2457429166 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b80003a60 msgr2=0x7f1b80003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:12.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.537+0000 7f1b92d9d700 1 --2- 172.21.15.89:0/2457429166 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b80003a60 0x7f1b80003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.537+0000 7f1b92d9d700 1 -- 172.21.15.89:0/2457429166 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b80003570 con 0x7f1b80004d50 2023-10-10T20:46:12.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.537+0000 7f1b92d9d700 1 --2- 172.21.15.89:0/2457429166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b80004d50 0x7f1b800051c0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00a970 tx=0x7f1b8c002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f17a9cfd8c2298 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:12.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.537+0000 7f1b91d9b700 1 -- 172.21.15.89:0/2457429166 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c012440 con 0x7f1b80004d50 2023-10-10T20:46:12.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.537+0000 7f1b91d9b700 1 -- 172.21.15.89:0/2457429166 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c0125c0 con 0x7f1b80004d50 2023-10-10T20:46:12.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.537+0000 7f1b91d9b700 1 -- 172.21.15.89:0/2457429166 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c012980 con 0x7f1b80004d50 2023-10-10T20:46:12.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.537+0000 7f1b9359e700 1 -- 172.21.15.89:0/2457429166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b80004d50 msgr2=0x7f1b800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:12.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.537+0000 7f1b9359e700 1 --2- 172.21.15.89:0/2457429166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b80004d50 0x7f1b800051c0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00a970 tx=0x7f1b8c002210 comp rx=0 tx=0).stop 2023-10-10T20:46:12.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.537+0000 7f1b92d9d700 1 -- 172.21.15.89:0/2457429166 reap_dead start 2023-10-10T20:46:12.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.537+0000 7f1b9359e700 1 -- 172.21.15.89:0/2457429166 shutdown_connections 2023-10-10T20:46:12.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.537+0000 7f1b9359e700 1 -- 172.21.15.89:0/2457429166 >> 172.21.15.89:0/2457429166 conn(0x7f1b940d10c0 msgr2=0x7f1b940affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:12.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.538+0000 7f1b9359e700 1 -- 172.21.15.89:0/2457429166 shutdown_connections 2023-10-10T20:46:12.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.538+0000 7f1b9359e700 1 -- 172.21.15.89:0/2457429166 wait complete. 2023-10-10T20:46:12.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.539+0000 7f1b9359e700 1 Processor -- start 2023-10-10T20:46:12.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.539+0000 7f1b9359e700 1 -- start start 2023-10-10T20:46:12.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.539+0000 7f1b9359e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b9414e1f0 0x7f1b9416a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.539+0000 7f1b9359e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b940ccc10 0x7f1b940a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.539+0000 7f1b9359e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b940a7440 0x7f1b940c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.539+0000 7f1b9359e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b940a78e0 con 0x7f1b9414e1f0 2023-10-10T20:46:12.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.539+0000 7f1b9359e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9416adf0 con 0x7f1b940a7440 2023-10-10T20:46:12.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.539+0000 7f1b9359e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b940cbb70 con 0x7f1b940ccc10 2023-10-10T20:46:12.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.539+0000 7f1b9259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b9414e1f0 0x7f1b9416a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.539+0000 7f1b9259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b9414e1f0 0x7f1b9416a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53874/0 (socket says 172.21.15.89:53874) 2023-10-10T20:46:12.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.539+0000 7f1b9259c700 1 -- 172.21.15.89:0/3082161413 learned_addr learned my addr 172.21.15.89:0/3082161413 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:12.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.540+0000 7f1b8bfff700 1 --2- 172.21.15.89:0/3082161413 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b940ccc10 0x7f1b940a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.540+0000 7f1b92d9d700 1 --2- 172.21.15.89:0/3082161413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b940a7440 0x7f1b940c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.540+0000 7f1b9259c700 1 -- 172.21.15.89:0/3082161413 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b940ccc10 msgr2=0x7f1b940a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:12.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.540+0000 7f1b9259c700 1 --2- 172.21.15.89:0/3082161413 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b940ccc10 0x7f1b940a6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.540+0000 7f1b9259c700 1 -- 172.21.15.89:0/3082161413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b940a7440 msgr2=0x7f1b940c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:12.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.540+0000 7f1b9259c700 1 --2- 172.21.15.89:0/3082161413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b940a7440 0x7f1b940c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.540+0000 7f1b9259c700 1 -- 172.21.15.89:0/3082161413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b940cbe50 con 0x7f1b9414e1f0 2023-10-10T20:46:12.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.540+0000 7f1b9259c700 1 --2- 172.21.15.89:0/3082161413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b9414e1f0 0x7f1b9416a8b0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f1b84006ac0 tx=0x7f1b840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:12.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.540+0000 7f1b8a7fc700 1 -- 172.21.15.89:0/3082161413 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8400f9b0 con 0x7f1b9414e1f0 2023-10-10T20:46:12.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.540+0000 7f1b8a7fc700 1 -- 172.21.15.89:0/3082161413 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8400fb10 con 0x7f1b9414e1f0 2023-10-10T20:46:12.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.540+0000 7f1b9359e700 1 -- 172.21.15.89:0/3082161413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b940cc130 con 0x7f1b9414e1f0 2023-10-10T20:46:12.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.540+0000 7f1b8a7fc700 1 -- 172.21.15.89:0/3082161413 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8400fe90 con 0x7f1b9414e1f0 2023-10-10T20:46:12.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.540+0000 7f1b9359e700 1 -- 172.21.15.89:0/3082161413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b940cc6f0 con 0x7f1b9414e1f0 2023-10-10T20:46:12.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.541+0000 7f1b9359e700 1 -- 172.21.15.89:0/3082161413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b940cca60 con 0x7f1b9414e1f0 2023-10-10T20:46:12.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.541+0000 7f1b8a7fc700 1 -- 172.21.15.89:0/3082161413 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1b84014650 con 0x7f1b9414e1f0 2023-10-10T20:46:12.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.542+0000 7f1b8a7fc700 1 --2- 172.21.15.89:0/3082161413 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1b74067320 0x7f1b740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.542+0000 7f1b8a7fc700 1 -- 172.21.15.89:0/3082161413 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f1b84018030 con 0x7f1b9414e1f0 2023-10-10T20:46:12.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.542+0000 7f1b8bfff700 1 --2- 172.21.15.89:0/3082161413 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1b74067320 0x7f1b740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.542+0000 7f1b8bfff700 1 --2- 172.21.15.89:0/3082161413 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1b74067320 0x7f1b740697d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c006ed0 tx=0x7f1b7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:12.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.546+0000 7f1b8a7fc700 1 -- 172.21.15.89:0/3082161413 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b840523d0 con 0x7f1b9414e1f0 2023-10-10T20:46:12.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.728+0000 7f1b9359e700 1 -- 172.21.15.89:0/3082161413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b9404bb80 con 0x7f1b9414e1f0 2023-10-10T20:46:12.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.728+0000 7f1b8a7fc700 1 -- 172.21.15.89:0/3082161413 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v200) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b84022020 con 0x7f1b9414e1f0 2023-10-10T20:46:12.731 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-10T20:46:12.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.730+0000 7f1b9359e700 1 -- 172.21.15.89:0/3082161413 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1b74067320 msgr2=0x7f1b740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:12.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.730+0000 7f1b9359e700 1 --2- 172.21.15.89:0/3082161413 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1b74067320 0x7f1b740697d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c006ed0 tx=0x7f1b7c008040 comp rx=0 tx=0).stop 2023-10-10T20:46:12.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.730+0000 7f1b9359e700 1 -- 172.21.15.89:0/3082161413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b9414e1f0 msgr2=0x7f1b9416a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:12.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.730+0000 7f1b9359e700 1 --2- 172.21.15.89:0/3082161413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b9414e1f0 0x7f1b9416a8b0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f1b84006ac0 tx=0x7f1b840091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.730+0000 7f1b9359e700 1 -- 172.21.15.89:0/3082161413 shutdown_connections 2023-10-10T20:46:12.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.730+0000 7f1b9359e700 1 --2- 172.21.15.89:0/3082161413 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1b74067320 0x7f1b740697d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.730+0000 7f1b9359e700 1 --2- 172.21.15.89:0/3082161413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b940a7440 0x7f1b940c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.730+0000 7f1b9359e700 1 --2- 172.21.15.89:0/3082161413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b9414e1f0 0x7f1b9416a8b0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.730+0000 7f1b9359e700 1 --2- 172.21.15.89:0/3082161413 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1b940ccc10 0x7f1b940a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.730+0000 7f1b9359e700 1 -- 172.21.15.89:0/3082161413 >> 172.21.15.89:0/3082161413 conn(0x7f1b940d10c0 msgr2=0x7f1b94159060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:12.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.731+0000 7f1b9359e700 1 -- 172.21.15.89:0/3082161413 shutdown_connections 2023-10-10T20:46:12.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.731+0000 7f1b9359e700 1 -- 172.21.15.89:0/3082161413 wait complete. 2023-10-10T20:46:12.745 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:46:12.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.858+0000 7fc132edf700 1 Processor -- start 2023-10-10T20:46:12.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.858+0000 7fc132edf700 1 -- start start 2023-10-10T20:46:12.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.859+0000 7fc132edf700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc12c159c60 0x7fc12c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.859+0000 7fc132edf700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc12c15db10 0x7fc12c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.859+0000 7fc132edf700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc12c15a160 con 0x7fc12c159890 2023-10-10T20:46:12.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.859+0000 7fc132edf700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc12c15df50 con 0x7fc12c159c60 2023-10-10T20:46:12.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.859+0000 7fc132edf700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c0d13a0 con 0x7fc12c15db10 2023-10-10T20:46:12.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.859+0000 7fc123fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc12c159c60 0x7fc12c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.859+0000 7fc13147c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc12c15db10 0x7fc12c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.859+0000 7fc130c7b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc12c159890 0x7fc12c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:55178/0 (socket says 172.21.15.89:55178) 2023-10-10T20:46:12.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.859+0000 7fc123fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc12c159c60 0x7fc12c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:40146/0 (socket says 172.21.15.89:40146) 2023-10-10T20:46:12.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.859+0000 7fc130c7b700 1 -- 172.21.15.89:0/1220432654 learned_addr learned my addr 172.21.15.89:0/1220432654 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.859+0000 7fc13147c700 1 --2- 172.21.15.89:0/1220432654 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc12c15db10 0x7fc12c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:44686/0 (socket says 172.21.15.89:44686) 2023-10-10T20:46:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.860+0000 7fc13147c700 1 -- 172.21.15.89:0/1220432654 >> v2:172.21.15.111:3300/0 conn(0x7fc12c159c60 msgr2=0x7fc12c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.860+0000 7fc13147c700 1 --2- 172.21.15.89:0/1220432654 >> v2:172.21.15.111:3300/0 conn(0x7fc12c159c60 0x7fc12c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.860+0000 7fc13147c700 1 -- 172.21.15.89:0/1220432654 >> v1:172.21.15.111:6789/0 conn(0x7fc12c159890 legacy=0x7fc12c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.860+0000 7fc13147c700 1 -- 172.21.15.89:0/1220432654 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12c15e140 con 0x7fc12c15db10 2023-10-10T20:46:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.860+0000 7fc123fff700 1 --2- 172.21.15.89:0/1220432654 >> v2:172.21.15.111:3300/0 conn(0x7fc12c159c60 0x7fc12c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:46:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.860+0000 7fc13147c700 1 --2- 172.21.15.89:0/1220432654 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc12c15db10 0x7fc12c069560 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc124009e60 tx=0x7fc124009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae2b87279ec107c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.860+0000 7fc1237fe700 1 -- 172.21.15.89:0/1220432654 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12400e070 con 0x7fc12c15db10 2023-10-10T20:46:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.860+0000 7fc1237fe700 1 -- 172.21.15.89:0/1220432654 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc12400bb60 con 0x7fc12c15db10 2023-10-10T20:46:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.860+0000 7fc1237fe700 1 -- 172.21.15.89:0/1220432654 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc124012430 con 0x7fc12c15db10 2023-10-10T20:46:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.861+0000 7fc132edf700 1 -- 172.21.15.89:0/1220432654 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc12c15db10 msgr2=0x7fc12c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.861+0000 7fc132edf700 1 --2- 172.21.15.89:0/1220432654 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc12c15db10 0x7fc12c069560 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc124009e60 tx=0x7fc124009780 comp rx=0 tx=0).stop 2023-10-10T20:46:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.861+0000 7fc132edf700 1 -- 172.21.15.89:0/1220432654 shutdown_connections 2023-10-10T20:46:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.861+0000 7fc132edf700 1 --2- 172.21.15.89:0/1220432654 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc12c15db10 0x7fc12c069560 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.861+0000 7fc132edf700 1 --2- 172.21.15.89:0/1220432654 >> v2:172.21.15.111:3300/0 conn(0x7fc12c159c60 0x7fc12c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.861+0000 7fc132edf700 1 -- 172.21.15.89:0/1220432654 >> 172.21.15.89:0/1220432654 conn(0x7fc12c068e20 msgr2=0x7fc12c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.861+0000 7fc132edf700 1 -- 172.21.15.89:0/1220432654 shutdown_connections 2023-10-10T20:46:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.861+0000 7fc132edf700 1 -- 172.21.15.89:0/1220432654 wait complete. 2023-10-10T20:46:12.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.862+0000 7fc132edf700 1 Processor -- start 2023-10-10T20:46:12.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.862+0000 7fc132edf700 1 -- start start 2023-10-10T20:46:12.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc132edf700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc12c159890 0x7fc12c16b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc132edf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc12c159c60 0x7fc12c16cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc132edf700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc12c15db10 0x7fc12c0b5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc132edf700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c16afc0 con 0x7fc12c159c60 2023-10-10T20:46:12.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc132edf700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c0c79d0 con 0x7fc12c159890 2023-10-10T20:46:12.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc132edf700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c0c7cb0 con 0x7fc12c15db10 2023-10-10T20:46:12.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc123fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc12c159c60 0x7fc12c16cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc123fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc12c159c60 0x7fc12c16cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53884/0 (socket says 172.21.15.89:53884) 2023-10-10T20:46:12.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc123fff700 1 -- 172.21.15.89:0/1836840663 learned_addr learned my addr 172.21.15.89:0/1836840663 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:12.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc130c7b700 1 --2- 172.21.15.89:0/1836840663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc12c159890 0x7fc12c16b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc13147c700 1 --2- 172.21.15.89:0/1836840663 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc12c15db10 0x7fc12c0b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc123fff700 1 -- 172.21.15.89:0/1836840663 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc12c15db10 msgr2=0x7fc12c0b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:12.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc123fff700 1 --2- 172.21.15.89:0/1836840663 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc12c15db10 0x7fc12c0b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc123fff700 1 -- 172.21.15.89:0/1836840663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc12c159890 msgr2=0x7fc12c16b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:12.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc123fff700 1 --2- 172.21.15.89:0/1836840663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc12c159890 0x7fc12c16b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:12.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc123fff700 1 -- 172.21.15.89:0/1836840663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c009420 con 0x7fc12c159c60 2023-10-10T20:46:12.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc130c7b700 1 --2- 172.21.15.89:0/1836840663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc12c159890 0x7fc12c16b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:46:12.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc123fff700 1 --2- 172.21.15.89:0/1836840663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc12c159c60 0x7fc12c16cf10 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00ec80 tx=0x7fc11c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:12.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc120ff9700 1 -- 172.21.15.89:0/1836840663 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c00c920 con 0x7fc12c159c60 2023-10-10T20:46:12.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc120ff9700 1 -- 172.21.15.89:0/1836840663 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11c00ca80 con 0x7fc12c159c60 2023-10-10T20:46:12.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc132edf700 1 -- 172.21.15.89:0/1836840663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc124009a20 con 0x7fc12c159c60 2023-10-10T20:46:12.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.863+0000 7fc120ff9700 1 -- 172.21.15.89:0/1836840663 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c00ce00 con 0x7fc12c159c60 2023-10-10T20:46:12.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.864+0000 7fc132edf700 1 -- 172.21.15.89:0/1836840663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12c0c8270 con 0x7fc12c159c60 2023-10-10T20:46:12.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.864+0000 7fc132edf700 1 -- 172.21.15.89:0/1836840663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc12c04c3c0 con 0x7fc12c159c60 2023-10-10T20:46:12.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.865+0000 7fc120ff9700 1 -- 172.21.15.89:0/1836840663 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc11c013680 con 0x7fc12c159c60 2023-10-10T20:46:12.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.865+0000 7fc120ff9700 1 --2- 172.21.15.89:0/1836840663 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc118067320 0x7fc1180697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:12.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.865+0000 7fc120ff9700 1 -- 172.21.15.89:0/1836840663 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5792+0+0 (secure 0 0 0) 0x7fc11c085410 con 0x7fc12c159c60 2023-10-10T20:46:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.865+0000 7fc130c7b700 1 --2- 172.21.15.89:0/1836840663 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc118067320 0x7fc1180697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.866+0000 7fc130c7b700 1 --2- 172.21.15.89:0/1836840663 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc118067320 0x7fc1180697d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc1280017e0 tx=0x7fc128007eb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:12.869+0000 7fc120ff9700 1 -- 172.21.15.89:0/1836840663 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc11c052010 con 0x7fc12c159c60 2023-10-10T20:46:13.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.095+0000 7fc132edf700 1 -- 172.21.15.89:0/1836840663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc12c0b0120 con 0x7fc12c159c60 2023-10-10T20:46:13.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.096+0000 7fc120ff9700 1 -- 172.21.15.89:0/1836840663 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+4604 (secure 0 0 0) 0x7fc11c055630 con 0x7fc12c159c60 2023-10-10T20:46:13.099 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:46:13.099 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":133,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/4215135154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":4215135154},{"type":"v1","addr":"172.21.15.89:6837","nonce":4215135154}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127},{"gid":7088,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/940990808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":940990808},{"type":"v1","addr":"172.21.15.111:6834","nonce":940990808}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127},{"gid":7091,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/92922277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":92922277},{"type":"v1","addr":"172.21.15.111:6835","nonce":92922277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:43:59.950797+0000","modified":"2023-10-10T20:44:03.611977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9135":{"gid":9135,"name":"a","rank":0,"incarnation":128,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6836/3558240892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3558240892},{"type":"v1","addr":"172.21.15.89:6836","nonce":3558240892}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-10T20:46:13.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.098+0000 7fc132edf700 1 -- 172.21.15.89:0/1836840663 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc118067320 msgr2=0x7fc1180697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:13.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.098+0000 7fc132edf700 1 --2- 172.21.15.89:0/1836840663 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc118067320 0x7fc1180697d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc1280017e0 tx=0x7fc128007eb0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.098+0000 7fc132edf700 1 -- 172.21.15.89:0/1836840663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc12c159c60 msgr2=0x7fc12c16cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:13.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.098+0000 7fc132edf700 1 --2- 172.21.15.89:0/1836840663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc12c159c60 0x7fc12c16cf10 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00ec80 tx=0x7fc11c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.098+0000 7fc132edf700 1 -- 172.21.15.89:0/1836840663 shutdown_connections 2023-10-10T20:46:13.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.098+0000 7fc132edf700 1 --2- 172.21.15.89:0/1836840663 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc12c15db10 0x7fc12c0b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.098+0000 7fc132edf700 1 --2- 172.21.15.89:0/1836840663 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc118067320 0x7fc1180697d0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.098+0000 7fc132edf700 1 --2- 172.21.15.89:0/1836840663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc12c159890 0x7fc12c16b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.098+0000 7fc132edf700 1 --2- 172.21.15.89:0/1836840663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc12c159c60 0x7fc12c16cf10 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.098+0000 7fc132edf700 1 -- 172.21.15.89:0/1836840663 >> 172.21.15.89:0/1836840663 conn(0x7fc12c068e20 msgr2=0x7fc12c0c6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:13.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.098+0000 7fc132edf700 1 -- 172.21.15.89:0/1836840663 shutdown_connections 2023-10-10T20:46:13.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.098+0000 7fc132edf700 1 -- 172.21.15.89:0/1836840663 wait complete. 2023-10-10T20:46:13.103 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 133 2023-10-10T20:46:13.113 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:46:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.225+0000 7febb3621700 1 Processor -- start 2023-10-10T20:46:13.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.226+0000 7febb3621700 1 -- start start 2023-10-10T20:46:13.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.226+0000 7febb3621700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7febac06c0a0 0x7febac0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.226+0000 7febb3621700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7febac15df50 0x7febac15e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.226+0000 7febb3621700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febac15d7e0 con 0x7febac06bcd0 2023-10-10T20:46:13.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.226+0000 7febb3621700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febac15dac0 con 0x7febac15df50 2023-10-10T20:46:13.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.226+0000 7febb3621700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac15aa30 con 0x7febac06c0a0 2023-10-10T20:46:13.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.227+0000 7febb1bbe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7febac15df50 0x7febac15e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.227+0000 7febb13bd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7febac06bcd0 0x7febac15a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59572/0 (socket says 172.21.15.89:59572) 2023-10-10T20:46:13.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.227+0000 7febb13bd700 1 -- 172.21.15.89:0/3867321964 learned_addr learned my addr 172.21.15.89:0/3867321964 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:13.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.227+0000 7febb1bbe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7febac15df50 0x7febac15e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53898/0 (socket says 172.21.15.89:53898) 2023-10-10T20:46:13.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.227+0000 7febb1bbe700 1 -- 172.21.15.89:0/3867321964 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7febac06c0a0 msgr2=0x7febac0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:13.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.227+0000 7febb0bbc700 1 --2- 172.21.15.89:0/3867321964 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7febac06c0a0 0x7febac0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.227+0000 7feba3fff700 1 -- 172.21.15.89:0/3867321964 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2134338002 0 0) 0x7febac15d7e0 con 0x7febac06bcd0 2023-10-10T20:46:13.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.227+0000 7febb1bbe700 1 --2- 172.21.15.89:0/3867321964 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7febac06c0a0 0x7febac0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.227+0000 7febb1bbe700 1 -- 172.21.15.89:0/3867321964 >> v1:172.21.15.89:6789/0 conn(0x7febac06bcd0 legacy=0x7febac15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:13.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.227+0000 7febb1bbe700 1 -- 172.21.15.89:0/3867321964 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac15ad10 con 0x7febac15df50 2023-10-10T20:46:13.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.227+0000 7febb0bbc700 1 --2- 172.21.15.89:0/3867321964 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7febac06c0a0 0x7febac0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:46:13.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.227+0000 7febb1bbe700 1 --2- 172.21.15.89:0/3867321964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febac15df50 0x7febac15e3c0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7feba40098e0 tx=0x7feba40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3e4f7b16b3021a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:13.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.227+0000 7feba3fff700 1 -- 172.21.15.89:0/3867321964 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4009e80 con 0x7febac15df50 2023-10-10T20:46:13.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.227+0000 7feba3fff700 1 -- 172.21.15.89:0/3867321964 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba400b860 con 0x7febac15df50 2023-10-10T20:46:13.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.227+0000 7feba3fff700 1 -- 172.21.15.89:0/3867321964 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba400bbe0 con 0x7febac15df50 2023-10-10T20:46:13.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.228+0000 7febb3621700 1 -- 172.21.15.89:0/3867321964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febac15df50 msgr2=0x7febac15e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:13.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.228+0000 7febb3621700 1 --2- 172.21.15.89:0/3867321964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febac15df50 0x7febac15e3c0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7feba40098e0 tx=0x7feba40091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.228+0000 7febb3621700 1 -- 172.21.15.89:0/3867321964 shutdown_connections 2023-10-10T20:46:13.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.228+0000 7febb3621700 1 --2- 172.21.15.89:0/3867321964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febac15df50 0x7febac15e3c0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.228+0000 7febb3621700 1 --2- 172.21.15.89:0/3867321964 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7febac06c0a0 0x7febac0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.228+0000 7febb3621700 1 -- 172.21.15.89:0/3867321964 >> 172.21.15.89:0/3867321964 conn(0x7febac0ce600 msgr2=0x7febac0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:13.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.228+0000 7febb3621700 1 -- 172.21.15.89:0/3867321964 shutdown_connections 2023-10-10T20:46:13.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.228+0000 7febb3621700 1 -- 172.21.15.89:0/3867321964 wait complete. 2023-10-10T20:46:13.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.229+0000 7febb3621700 1 Processor -- start 2023-10-10T20:46:13.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.229+0000 7febb3621700 1 -- start start 2023-10-10T20:46:13.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb3621700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febac06bcd0 0x7febac157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb3621700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7febac06c0a0 0x7febac159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb3621700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7febac15df50 0x7febac159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb3621700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0c5d10 con 0x7febac06bcd0 2023-10-10T20:46:13.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb3621700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0c8730 con 0x7febac15df50 2023-10-10T20:46:13.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb3621700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac0c5750 con 0x7febac06c0a0 2023-10-10T20:46:13.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb13bd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febac06bcd0 0x7febac157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb13bd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febac06bcd0 0x7febac157340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53914/0 (socket says 172.21.15.89:53914) 2023-10-10T20:46:13.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb13bd700 1 -- 172.21.15.89:0/3329595096 learned_addr learned my addr 172.21.15.89:0/3329595096 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:13.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb0bbc700 1 --2- 172.21.15.89:0/3329595096 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7febac06c0a0 0x7febac159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb1bbe700 1 --2- 172.21.15.89:0/3329595096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7febac15df50 0x7febac159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb13bd700 1 -- 172.21.15.89:0/3329595096 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7febac06c0a0 msgr2=0x7febac159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:13.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb13bd700 1 --2- 172.21.15.89:0/3329595096 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7febac06c0a0 0x7febac159300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb13bd700 1 -- 172.21.15.89:0/3329595096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7febac15df50 msgr2=0x7febac159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:13.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb13bd700 1 --2- 172.21.15.89:0/3329595096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7febac15df50 0x7febac159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb13bd700 1 -- 172.21.15.89:0/3329595096 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba4009420 con 0x7febac06bcd0 2023-10-10T20:46:13.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb1bbe700 1 --2- 172.21.15.89:0/3329595096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7febac15df50 0x7febac159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:13.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb0bbc700 1 --2- 172.21.15.89:0/3329595096 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7febac06c0a0 0x7febac159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:13.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb13bd700 1 --2- 172.21.15.89:0/3329595096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febac06bcd0 0x7febac157340 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7feba800cc70 tx=0x7feba8001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:13.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7feba17fa700 1 -- 172.21.15.89:0/3329595096 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba800ce80 con 0x7febac06bcd0 2023-10-10T20:46:13.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7feba17fa700 1 -- 172.21.15.89:0/3329595096 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba800ab70 con 0x7febac06bcd0 2023-10-10T20:46:13.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7febb3621700 1 -- 172.21.15.89:0/3329595096 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febac0c5a30 con 0x7febac06bcd0 2023-10-10T20:46:13.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.230+0000 7feba17fa700 1 -- 172.21.15.89:0/3329595096 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba80070f0 con 0x7febac06bcd0 2023-10-10T20:46:13.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.231+0000 7febb3621700 1 -- 172.21.15.89:0/3329595096 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febac157880 con 0x7febac06bcd0 2023-10-10T20:46:13.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.232+0000 7febb3621700 1 -- 172.21.15.89:0/3329595096 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febac04c3c0 con 0x7febac06bcd0 2023-10-10T20:46:13.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.232+0000 7feba17fa700 1 -- 172.21.15.89:0/3329595096 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7feba8007520 con 0x7febac06bcd0 2023-10-10T20:46:13.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.232+0000 7feba17fa700 1 --2- 172.21.15.89:0/3329595096 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7feb90067320 0x7feb900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.232+0000 7feba17fa700 1 -- 172.21.15.89:0/3329595096 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5792+0+0 (secure 0 0 0) 0x7feba8085140 con 0x7febac06bcd0 2023-10-10T20:46:13.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.232+0000 7febb0bbc700 1 --2- 172.21.15.89:0/3329595096 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7feb90067320 0x7feb900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.233+0000 7febb0bbc700 1 --2- 172.21.15.89:0/3329595096 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7feb90067320 0x7feb900697d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7febac06d840 tx=0x7feb9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:13.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.236+0000 7feba17fa700 1 -- 172.21.15.89:0/3329595096 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feba8051e00 con 0x7febac06bcd0 2023-10-10T20:46:13.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.419+0000 7febb3621700 1 -- 172.21.15.89:0/3329595096 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7febac15aff0 con 0x7febac06bcd0 2023-10-10T20:46:13.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.420+0000 7feba17fa700 1 -- 172.21.15.89:0/3329595096 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+15509 (secure 0 0 0) 0x7feba8055420 con 0x7febac06bcd0 2023-10-10T20:46:13.423 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:46:13.423 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":200,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:44:24.011501+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:43:58.313137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-10T20:43:58.914819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"34.8","mappings":[{"from":0,"to":1}]},{"pgid":"34.18","mappings":[{"from":0,"to":2}]},{"pgid":"34.1f","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:46:13.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.422+0000 7febb3621700 1 -- 172.21.15.89:0/3329595096 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7feb90067320 msgr2=0x7feb900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:13.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.422+0000 7febb3621700 1 --2- 172.21.15.89:0/3329595096 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7feb90067320 0x7feb900697d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7febac06d840 tx=0x7feb9c008040 comp rx=0 tx=0).stop 2023-10-10T20:46:13.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.422+0000 7febb3621700 1 -- 172.21.15.89:0/3329595096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febac06bcd0 msgr2=0x7febac157340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:13.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.422+0000 7febb3621700 1 --2- 172.21.15.89:0/3329595096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febac06bcd0 0x7febac157340 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7feba800cc70 tx=0x7feba8001a30 comp rx=0 tx=0).stop 2023-10-10T20:46:13.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.422+0000 7febb3621700 1 -- 172.21.15.89:0/3329595096 shutdown_connections 2023-10-10T20:46:13.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.422+0000 7febb3621700 1 --2- 172.21.15.89:0/3329595096 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7feb90067320 0x7feb900697d0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.422+0000 7febb3621700 1 --2- 172.21.15.89:0/3329595096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7febac15df50 0x7febac159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.422+0000 7febb3621700 1 --2- 172.21.15.89:0/3329595096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febac06bcd0 0x7febac157340 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.422+0000 7febb3621700 1 --2- 172.21.15.89:0/3329595096 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7febac06c0a0 0x7febac159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.422+0000 7febb3621700 1 -- 172.21.15.89:0/3329595096 >> 172.21.15.89:0/3329595096 conn(0x7febac0ce600 msgr2=0x7febac0cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:13.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.422+0000 7febb3621700 1 -- 172.21.15.89:0/3329595096 shutdown_connections 2023-10-10T20:46:13.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.422+0000 7febb3621700 1 -- 172.21.15.89:0/3329595096 wait complete. 2023-10-10T20:46:13.437 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:46:13.438 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:46:13.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.549+0000 7ff6b4c86700 1 Processor -- start 2023-10-10T20:46:13.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.549+0000 7ff6b4c86700 1 -- start start 2023-10-10T20:46:13.562 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:46:13.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.550+0000 7ff6b4c86700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6b00aca80 0x7ff6b006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.550+0000 7ff6b4c86700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff6b00c83c0 0x7ff6b00cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.550+0000 7ff6b4c86700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b006bdb0 con 0x7ff6b0069d60 2023-10-10T20:46:13.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.550+0000 7ff6b4c86700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6b00cb370 con 0x7ff6b00c83c0 2023-10-10T20:46:13.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.550+0000 7ff6b4c86700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b0159f60 con 0x7ff6b00aca80 2023-10-10T20:46:13.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.550+0000 7ff6a5d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff6b0069d60 0x7ff6b00c7cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59588/0 (socket says 172.21.15.89:59588) 2023-10-10T20:46:13.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.550+0000 7ff6a5d9b700 1 -- 172.21.15.89:0/3175852978 learned_addr learned my addr 172.21.15.89:0/3175852978 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:13.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6ae59c700 1 --2- 172.21.15.89:0/3175852978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6b00aca80 0x7ff6b006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6aed9d700 1 --2- 172.21.15.89:0/3175852978 >> v2:172.21.15.111:3300/0 conn(0x7ff6b00c83c0 0x7ff6b00cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6add9b700 1 -- 172.21.15.89:0/3175852978 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1997681746 0 0) 0x7ff6b006bdb0 con 0x7ff6b0069d60 2023-10-10T20:46:13.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6add9b700 1 -- 172.21.15.89:0/3175852978 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff69c003570 con 0x7ff6b0069d60 2023-10-10T20:46:13.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6add9b700 1 -- 172.21.15.89:0/3175852978 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1282864362 0 0) 0x7ff69c003570 con 0x7ff6b0069d60 2023-10-10T20:46:13.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6add9b700 1 -- 172.21.15.89:0/3175852978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6b00aca80 msgr2=0x7ff6b006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:13.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6add9b700 1 --2- 172.21.15.89:0/3175852978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6b00aca80 0x7ff6b006aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6add9b700 1 -- 172.21.15.89:0/3175852978 >> v2:172.21.15.111:3300/0 conn(0x7ff6b00c83c0 msgr2=0x7ff6b00cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:13.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6add9b700 1 --2- 172.21.15.89:0/3175852978 >> v2:172.21.15.111:3300/0 conn(0x7ff6b00c83c0 0x7ff6b00cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6add9b700 1 -- 172.21.15.89:0/3175852978 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b006b510 con 0x7ff6b0069d60 2023-10-10T20:46:13.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6add9b700 1 -- 172.21.15.89:0/3175852978 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7ff698001470 con 0x7ff6b0069d60 2023-10-10T20:46:13.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6ae59c700 1 --2- 172.21.15.89:0/3175852978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6b00aca80 0x7ff6b006aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:46:13.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6aed9d700 1 --2- 172.21.15.89:0/3175852978 >> v2:172.21.15.111:3300/0 conn(0x7ff6b00c83c0 0x7ff6b00cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:46:13.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6add9b700 1 -- 172.21.15.89:0/3175852978 >> v1:172.21.15.89:6789/0 conn(0x7ff6b0069d60 legacy=0x7ff6b00c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:13.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6add9b700 1 --2- 172.21.15.89:0/3175852978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69c003a60 0x7ff69c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6add9b700 1 --2- 172.21.15.89:0/3175852978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff69c004810 0x7ff69c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6add9b700 1 --2- 172.21.15.89:0/3175852978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6b00aca80 0x7ff69c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6add9b700 1 -- 172.21.15.89:0/3175852978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b00cb370 con 0x7ff69c003a60 2023-10-10T20:46:13.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6add9b700 1 -- 172.21.15.89:0/3175852978 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b0159f60 con 0x7ff69c004810 2023-10-10T20:46:13.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6add9b700 1 -- 172.21.15.89:0/3175852978 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c00ba60 con 0x7ff6b00aca80 2023-10-10T20:46:13.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6ae59c700 1 --2- 172.21.15.89:0/3175852978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69c003a60 0x7ff69c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6a5d9b700 1 --2- 172.21.15.89:0/3175852978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff69c004810 0x7ff69c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6aed9d700 1 --2- 172.21.15.89:0/3175852978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6b00aca80 0x7ff69c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6ae59c700 1 -- 172.21.15.89:0/3175852978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6b00aca80 msgr2=0x7ff69c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:13.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6ae59c700 1 --2- 172.21.15.89:0/3175852978 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6b00aca80 0x7ff69c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6ae59c700 1 -- 172.21.15.89:0/3175852978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff69c004810 msgr2=0x7ff69c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:13.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6ae59c700 1 --2- 172.21.15.89:0/3175852978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff69c004810 0x7ff69c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6ae59c700 1 -- 172.21.15.89:0/3175852978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a00093c0 con 0x7ff69c003a60 2023-10-10T20:46:13.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.551+0000 7ff6ae59c700 1 --2- 172.21.15.89:0/3175852978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69c003a60 0x7ff69c003e70 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009de0 tx=0x7ff6a0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35fccb73f7586ab server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:13.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.552+0000 7ff6add9b700 1 -- 172.21.15.89:0/3175852978 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0009730 con 0x7ff69c003a60 2023-10-10T20:46:13.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.552+0000 7ff6add9b700 1 -- 172.21.15.89:0/3175852978 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a0005a00 con 0x7ff69c003a60 2023-10-10T20:46:13.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.552+0000 7ff6add9b700 1 -- 172.21.15.89:0/3175852978 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0005dc0 con 0x7ff69c003a60 2023-10-10T20:46:13.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.552+0000 7ff6b4c86700 1 -- 172.21.15.89:0/3175852978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69c003a60 msgr2=0x7ff69c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:13.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.552+0000 7ff6b4c86700 1 --2- 172.21.15.89:0/3175852978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69c003a60 0x7ff69c003e70 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009de0 tx=0x7ff6a0009120 comp rx=0 tx=0).stop 2023-10-10T20:46:13.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.552+0000 7ff6aed9d700 1 -- 172.21.15.89:0/3175852978 reap_dead start 2023-10-10T20:46:13.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.552+0000 7ff6b4c86700 1 -- 172.21.15.89:0/3175852978 shutdown_connections 2023-10-10T20:46:13.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.552+0000 7ff6b4c86700 1 -- 172.21.15.89:0/3175852978 >> 172.21.15.89:0/3175852978 conn(0x7ff6b00cd670 msgr2=0x7ff6b00cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:13.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.552+0000 7ff6b4c86700 1 -- 172.21.15.89:0/3175852978 shutdown_connections 2023-10-10T20:46:13.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.552+0000 7ff6b4c86700 1 -- 172.21.15.89:0/3175852978 wait complete. 2023-10-10T20:46:13.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.553+0000 7ff6b4c86700 1 Processor -- start 2023-10-10T20:46:13.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.553+0000 7ff6b4c86700 1 -- start start 2023-10-10T20:46:13.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.554+0000 7ff6b4c86700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff69c003a60 0x7ff6b0152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.554+0000 7ff6b4c86700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6b00b2210 0x7ff6b00b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.554+0000 7ff6b4c86700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6b015d990 0x7ff6b015de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.554+0000 7ff6b4c86700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b0152a40 con 0x7ff6b00b2210 2023-10-10T20:46:13.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.554+0000 7ff6b4c86700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b015c6a0 con 0x7ff69c003a60 2023-10-10T20:46:13.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.554+0000 7ff6b4c86700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b015c8f0 con 0x7ff6b015d990 2023-10-10T20:46:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.554+0000 7ff6a5d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6b00b2210 0x7ff6b00b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.554+0000 7ff6a5d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6b00b2210 0x7ff6b00b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53942/0 (socket says 172.21.15.89:53942) 2023-10-10T20:46:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.554+0000 7ff6a5d9b700 1 -- 172.21.15.89:0/755196859 learned_addr learned my addr 172.21.15.89:0/755196859 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.554+0000 7ff6aed9d700 1 --2- 172.21.15.89:0/755196859 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6b015d990 0x7ff6b015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.554+0000 7ff6a5d9b700 1 -- 172.21.15.89:0/755196859 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6b015d990 msgr2=0x7ff6b015de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:13.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.554+0000 7ff6a5d9b700 1 --2- 172.21.15.89:0/755196859 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6b015d990 0x7ff6b015de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.554+0000 7ff6ae59c700 1 --2- 172.21.15.89:0/755196859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff69c003a60 0x7ff6b0152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.554+0000 7ff6a5d9b700 1 -- 172.21.15.89:0/755196859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff69c003a60 msgr2=0x7ff6b0152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:13.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.555+0000 7ff6a5d9b700 1 --2- 172.21.15.89:0/755196859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff69c003a60 0x7ff6b0152500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.555+0000 7ff6a5d9b700 1 -- 172.21.15.89:0/755196859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b015cbd0 con 0x7ff6b00b2210 2023-10-10T20:46:13.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.555+0000 7ff6a5d9b700 1 --2- 172.21.15.89:0/755196859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6b00b2210 0x7ff6b00b2680 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7ff698001050 tx=0x7ff698001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:13.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.555+0000 7ff6a6ffd700 1 -- 172.21.15.89:0/755196859 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69800c630 con 0x7ff6b00b2210 2023-10-10T20:46:13.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.555+0000 7ff6a6ffd700 1 -- 172.21.15.89:0/755196859 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69800c790 con 0x7ff6b00b2210 2023-10-10T20:46:13.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.555+0000 7ff6b4c86700 1 -- 172.21.15.89:0/755196859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b015ceb0 con 0x7ff6b00b2210 2023-10-10T20:46:13.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.555+0000 7ff6a6ffd700 1 -- 172.21.15.89:0/755196859 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69800cb50 con 0x7ff6b00b2210 2023-10-10T20:46:13.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.555+0000 7ff6b4c86700 1 -- 172.21.15.89:0/755196859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b015d470 con 0x7ff6b00b2210 2023-10-10T20:46:13.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.556+0000 7ff6b4c86700 1 -- 172.21.15.89:0/755196859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b004c3c0 con 0x7ff6b00b2210 2023-10-10T20:46:13.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.556+0000 7ff6a6ffd700 1 -- 172.21.15.89:0/755196859 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff69801d460 con 0x7ff6b00b2210 2023-10-10T20:46:13.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.556+0000 7ff6a6ffd700 1 --2- 172.21.15.89:0/755196859 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff690067320 0x7ff6900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.557+0000 7ff6a6ffd700 1 -- 172.21.15.89:0/755196859 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5792+0+0 (secure 0 0 0) 0x7ff698084ce0 con 0x7ff6b00b2210 2023-10-10T20:46:13.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.557+0000 7ff6ae59c700 1 --2- 172.21.15.89:0/755196859 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff690067320 0x7ff6900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.557+0000 7ff6ae59c700 1 --2- 172.21.15.89:0/755196859 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff690067320 0x7ff6900697d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009390 tx=0x7ff6a0019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:13.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.561+0000 7ff6a6ffd700 1 -- 172.21.15.89:0/755196859 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff698051920 con 0x7ff6b00b2210 2023-10-10T20:46:13.582 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:46:13.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.793+0000 7ff6b4c86700 1 -- 172.21.15.89:0/755196859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6b006b7f0 con 0x7ff6b00b2210 2023-10-10T20:46:13.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.793+0000 7ff6a6ffd700 1 -- 172.21.15.89:0/755196859 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+4604 (secure 0 0 0) 0x7ff698001cb0 con 0x7ff6b00b2210 2023-10-10T20:46:13.797 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:46:13.797 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":133,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/4215135154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":4215135154},{"type":"v1","addr":"172.21.15.89:6837","nonce":4215135154}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127},{"gid":7088,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/940990808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":940990808},{"type":"v1","addr":"172.21.15.111:6834","nonce":940990808}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127},{"gid":7091,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/92922277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":92922277},{"type":"v1","addr":"172.21.15.111:6835","nonce":92922277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:43:59.950797+0000","modified":"2023-10-10T20:44:03.611977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9135":{"gid":9135,"name":"a","rank":0,"incarnation":128,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6836/3558240892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3558240892},{"type":"v1","addr":"172.21.15.89:6836","nonce":3558240892}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-10T20:46:13.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.795+0000 7ff6b4c86700 1 -- 172.21.15.89:0/755196859 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff690067320 msgr2=0x7ff6900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:13.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.795+0000 7ff6b4c86700 1 --2- 172.21.15.89:0/755196859 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff690067320 0x7ff6900697d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009390 tx=0x7ff6a0019000 comp rx=0 tx=0).stop 2023-10-10T20:46:13.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.795+0000 7ff6b4c86700 1 -- 172.21.15.89:0/755196859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6b00b2210 msgr2=0x7ff6b00b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:13.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.795+0000 7ff6b4c86700 1 --2- 172.21.15.89:0/755196859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6b00b2210 0x7ff6b00b2680 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7ff698001050 tx=0x7ff698001510 comp rx=0 tx=0).stop 2023-10-10T20:46:13.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.795+0000 7ff6b4c86700 1 -- 172.21.15.89:0/755196859 shutdown_connections 2023-10-10T20:46:13.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.795+0000 7ff6b4c86700 1 --2- 172.21.15.89:0/755196859 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff6b015d990 0x7ff6b015de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.795+0000 7ff6b4c86700 1 --2- 172.21.15.89:0/755196859 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff690067320 0x7ff6900697d0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.795+0000 7ff6b4c86700 1 --2- 172.21.15.89:0/755196859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff69c003a60 0x7ff6b0152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.795+0000 7ff6b4c86700 1 --2- 172.21.15.89:0/755196859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6b00b2210 0x7ff6b00b2680 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.795+0000 7ff6b4c86700 1 -- 172.21.15.89:0/755196859 >> 172.21.15.89:0/755196859 conn(0x7ff6b00cd670 msgr2=0x7ff6b00c2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:13.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.796+0000 7ff6b4c86700 1 -- 172.21.15.89:0/755196859 shutdown_connections 2023-10-10T20:46:13.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.796+0000 7ff6b4c86700 1 -- 172.21.15.89:0/755196859 wait complete. 2023-10-10T20:46:13.801 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 133 2023-10-10T20:46:13.810 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:46:13.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.922+0000 7fda79b86700 1 Processor -- start 2023-10-10T20:46:13.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.922+0000 7fda79b86700 1 -- start start 2023-10-10T20:46:13.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.923+0000 7fda79b86700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda7406c0f0 0x7fda74159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.923+0000 7fda79b86700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fda740b3810 0x7fda740b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.923+0000 7fda79b86700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda74143420 con 0x7fda74143050 2023-10-10T20:46:13.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.923+0000 7fda79b86700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda7406a1e0 con 0x7fda740b3810 2023-10-10T20:46:13.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.923+0000 7fda79b86700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda740695d0 con 0x7fda7406c0f0 2023-10-10T20:46:13.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.923+0000 7fda78b84700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fda74143050 0x7fda7406b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59598/0 (socket says 172.21.15.89:59598) 2023-10-10T20:46:13.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.923+0000 7fda78b84700 1 -- 172.21.15.89:0/2829897695 learned_addr learned my addr 172.21.15.89:0/2829897695 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:13.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.923+0000 7fda73fff700 1 --2- 172.21.15.89:0/2829897695 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda7406c0f0 0x7fda74159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.923+0000 7fda79385700 1 --2- 172.21.15.89:0/2829897695 >> v2:172.21.15.111:3300/0 conn(0x7fda740b3810 0x7fda740b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.923+0000 7fda737fe700 1 -- 172.21.15.89:0/2829897695 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 231062660 0 0) 0x7fda74143420 con 0x7fda74143050 2023-10-10T20:46:13.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.923+0000 7fda737fe700 1 -- 172.21.15.89:0/2829897695 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda60003570 con 0x7fda74143050 2023-10-10T20:46:13.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.923+0000 7fda73fff700 1 -- 172.21.15.89:0/2829897695 >> v2:172.21.15.111:3300/0 conn(0x7fda740b3810 msgr2=0x7fda740b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:13.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.923+0000 7fda73fff700 1 --2- 172.21.15.89:0/2829897695 >> v2:172.21.15.111:3300/0 conn(0x7fda740b3810 0x7fda740b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.923+0000 7fda73fff700 1 -- 172.21.15.89:0/2829897695 >> v1:172.21.15.89:6789/0 conn(0x7fda74143050 legacy=0x7fda7406b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:13.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.923+0000 7fda73fff700 1 -- 172.21.15.89:0/2829897695 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda740698b0 con 0x7fda7406c0f0 2023-10-10T20:46:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.924+0000 7fda73fff700 1 --2- 172.21.15.89:0/2829897695 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda7406c0f0 0x7fda74159c90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0098e0 tx=0x7fda5c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e4de5fc9f3552df server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.924+0000 7fda737fe700 1 -- 172.21.15.89:0/2829897695 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c011630 con 0x7fda7406c0f0 2023-10-10T20:46:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.924+0000 7fda737fe700 1 -- 172.21.15.89:0/2829897695 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5c011790 con 0x7fda7406c0f0 2023-10-10T20:46:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.924+0000 7fda737fe700 1 -- 172.21.15.89:0/2829897695 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c011b10 con 0x7fda7406c0f0 2023-10-10T20:46:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.924+0000 7fda79b86700 1 -- 172.21.15.89:0/2829897695 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda7406c0f0 msgr2=0x7fda74159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.924+0000 7fda79b86700 1 --2- 172.21.15.89:0/2829897695 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda7406c0f0 0x7fda74159c90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0098e0 tx=0x7fda5c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.924+0000 7fda79b86700 1 -- 172.21.15.89:0/2829897695 shutdown_connections 2023-10-10T20:46:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.924+0000 7fda79b86700 1 --2- 172.21.15.89:0/2829897695 >> v2:172.21.15.111:3300/0 conn(0x7fda740b3810 0x7fda740b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.924+0000 7fda79b86700 1 --2- 172.21.15.89:0/2829897695 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda7406c0f0 0x7fda74159c90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.924+0000 7fda79b86700 1 -- 172.21.15.89:0/2829897695 >> 172.21.15.89:0/2829897695 conn(0x7fda740ce430 msgr2=0x7fda740ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.924+0000 7fda79b86700 1 -- 172.21.15.89:0/2829897695 shutdown_connections 2023-10-10T20:46:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.924+0000 7fda79b86700 1 -- 172.21.15.89:0/2829897695 wait complete. 2023-10-10T20:46:13.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.926+0000 7fda79b86700 1 Processor -- start 2023-10-10T20:46:13.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.926+0000 7fda79b86700 1 -- start start 2023-10-10T20:46:13.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.926+0000 7fda79b86700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda7406c0f0 0x7fda7413cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.926+0000 7fda79b86700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda740b3810 0x7fda7413bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.926+0000 7fda79b86700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda74143050 0x7fda74140490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.926+0000 7fda79b86700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7413d7d0 con 0x7fda740b3810 2023-10-10T20:46:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.926+0000 7fda79b86700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7413dab0 con 0x7fda74143050 2023-10-10T20:46:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.926+0000 7fda79b86700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7413dd90 con 0x7fda7406c0f0 2023-10-10T20:46:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.927+0000 7fda73fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda740b3810 0x7fda7413bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.927+0000 7fda73fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda740b3810 0x7fda7413bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53948/0 (socket says 172.21.15.89:53948) 2023-10-10T20:46:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.927+0000 7fda73fff700 1 -- 172.21.15.89:0/1260649317 learned_addr learned my addr 172.21.15.89:0/1260649317 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.927+0000 7fda78b84700 1 --2- 172.21.15.89:0/1260649317 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda7406c0f0 0x7fda7413cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.927+0000 7fda79385700 1 --2- 172.21.15.89:0/1260649317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda74143050 0x7fda74140490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.927+0000 7fda73fff700 1 -- 172.21.15.89:0/1260649317 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda7406c0f0 msgr2=0x7fda7413cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.927+0000 7fda73fff700 1 --2- 172.21.15.89:0/1260649317 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda7406c0f0 0x7fda7413cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.927+0000 7fda73fff700 1 -- 172.21.15.89:0/1260649317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda74143050 msgr2=0x7fda74140490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.927+0000 7fda73fff700 1 --2- 172.21.15.89:0/1260649317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda74143050 0x7fda74140490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.927+0000 7fda73fff700 1 -- 172.21.15.89:0/1260649317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c009420 con 0x7fda740b3810 2023-10-10T20:46:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.927+0000 7fda79385700 1 --2- 172.21.15.89:0/1260649317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda74143050 0x7fda74140490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:46:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.927+0000 7fda78b84700 1 --2- 172.21.15.89:0/1260649317 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda7406c0f0 0x7fda7413cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:46:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.927+0000 7fda73fff700 1 --2- 172.21.15.89:0/1260649317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda740b3810 0x7fda7413bd00 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009ed0 tx=0x7fda5c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.927+0000 7fda70ff9700 1 -- 172.21.15.89:0/1260649317 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c011900 con 0x7fda740b3810 2023-10-10T20:46:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.927+0000 7fda70ff9700 1 -- 172.21.15.89:0/1260649317 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5c011aa0 con 0x7fda740b3810 2023-10-10T20:46:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.927+0000 7fda79b86700 1 -- 172.21.15.89:0/1260649317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda740c7680 con 0x7fda740b3810 2023-10-10T20:46:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.927+0000 7fda70ff9700 1 -- 172.21.15.89:0/1260649317 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c01a680 con 0x7fda740b3810 2023-10-10T20:46:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.927+0000 7fda79b86700 1 -- 172.21.15.89:0/1260649317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7413d470 con 0x7fda740b3810 2023-10-10T20:46:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.928+0000 7fda79b86700 1 -- 172.21.15.89:0/1260649317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda740c7ba0 con 0x7fda740b3810 2023-10-10T20:46:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.928+0000 7fda70ff9700 1 -- 172.21.15.89:0/1260649317 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fda5c01aab0 con 0x7fda740b3810 2023-10-10T20:46:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.929+0000 7fda70ff9700 1 --2- 172.21.15.89:0/1260649317 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fda54067320 0x7fda540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.929+0000 7fda70ff9700 1 -- 172.21.15.89:0/1260649317 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5792+0+0 (secure 0 0 0) 0x7fda5c086890 con 0x7fda740b3810 2023-10-10T20:46:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.929+0000 7fda78b84700 1 --2- 172.21.15.89:0/1260649317 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fda54067320 0x7fda540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.929+0000 7fda78b84700 1 --2- 172.21.15.89:0/1260649317 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fda54067320 0x7fda540697d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fda6c000f80 tx=0x7fda6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:13.933+0000 7fda70ff9700 1 -- 172.21.15.89:0/1260649317 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda740c7ba0 con 0x7fda740b3810 2023-10-10T20:46:14.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.160+0000 7fda79b86700 1 -- 172.21.15.89:0/1260649317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda740c7ba0 con 0x7fda740b3810 2023-10-10T20:46:14.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.161+0000 7fda70ff9700 1 -- 172.21.15.89:0/1260649317 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+4604 (secure 0 0 0) 0x7fda740c7ba0 con 0x7fda740b3810 2023-10-10T20:46:14.164 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:46:14.165 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":133,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/4215135154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":4215135154},{"type":"v1","addr":"172.21.15.89:6837","nonce":4215135154}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127},{"gid":7088,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/940990808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":940990808},{"type":"v1","addr":"172.21.15.111:6834","nonce":940990808}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127},{"gid":7091,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/92922277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":92922277},{"type":"v1","addr":"172.21.15.111:6835","nonce":92922277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:43:59.950797+0000","modified":"2023-10-10T20:44:03.611977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9135":{"gid":9135,"name":"a","rank":0,"incarnation":128,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6836/3558240892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3558240892},{"type":"v1","addr":"172.21.15.89:6836","nonce":3558240892}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-10T20:46:14.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.163+0000 7fda79b86700 1 -- 172.21.15.89:0/1260649317 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fda54067320 msgr2=0x7fda540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:14.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.163+0000 7fda79b86700 1 --2- 172.21.15.89:0/1260649317 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fda54067320 0x7fda540697d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fda6c000f80 tx=0x7fda6c009040 comp rx=0 tx=0).stop 2023-10-10T20:46:14.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.163+0000 7fda79b86700 1 -- 172.21.15.89:0/1260649317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda740b3810 msgr2=0x7fda7413bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:14.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.163+0000 7fda79b86700 1 --2- 172.21.15.89:0/1260649317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda740b3810 0x7fda7413bd00 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009ed0 tx=0x7fda5c017000 comp rx=0 tx=0).stop 2023-10-10T20:46:14.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.163+0000 7fda79b86700 1 -- 172.21.15.89:0/1260649317 shutdown_connections 2023-10-10T20:46:14.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.163+0000 7fda79b86700 1 --2- 172.21.15.89:0/1260649317 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fda54067320 0x7fda540697d0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.163+0000 7fda79b86700 1 --2- 172.21.15.89:0/1260649317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda74143050 0x7fda74140490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.163+0000 7fda79b86700 1 --2- 172.21.15.89:0/1260649317 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fda7406c0f0 0x7fda7413cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.163+0000 7fda79b86700 1 --2- 172.21.15.89:0/1260649317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda740b3810 0x7fda7413bd00 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.163+0000 7fda79b86700 1 -- 172.21.15.89:0/1260649317 >> 172.21.15.89:0/1260649317 conn(0x7fda740ce430 msgr2=0x7fda7406b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:14.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.163+0000 7fda79b86700 1 -- 172.21.15.89:0/1260649317 shutdown_connections 2023-10-10T20:46:14.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.163+0000 7fda79b86700 1 -- 172.21.15.89:0/1260649317 wait complete. 2023-10-10T20:46:14.171 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 133 2023-10-10T20:46:14.178 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:46:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.290+0000 7fcc44a67700 1 Processor -- start 2023-10-10T20:46:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.290+0000 7fcc44a67700 1 -- start start 2023-10-10T20:46:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.291+0000 7fcc44a67700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcc40159890 0x7fcc4015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.291+0000 7fcc44a67700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcc40159c60 0x7fcc4015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.291+0000 7fcc44a67700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc4015db10 0x7fcc40069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.291+0000 7fcc44a67700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc4015a160 con 0x7fcc4015db10 2023-10-10T20:46:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.291+0000 7fcc44a67700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc400d13a0 con 0x7fcc40159c60 2023-10-10T20:46:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.291+0000 7fcc44a67700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4015e140 con 0x7fcc40159890 2023-10-10T20:46:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.291+0000 7fcc3ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc4015db10 0x7fcc40069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.291+0000 7fcc3ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc4015db10 0x7fcc40069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53950/0 (socket says 172.21.15.89:53950) 2023-10-10T20:46:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.291+0000 7fcc3ed9d700 1 -- 172.21.15.89:0/4142445576 learned_addr learned my addr 172.21.15.89:0/4142445576 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.291+0000 7fcc3e59c700 1 --2- 172.21.15.89:0/4142445576 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcc40159890 0x7fcc4015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.291+0000 7fcc35d9b700 1 --2- 172.21.15.89:0/4142445576 >> v2:172.21.15.111:3300/0 conn(0x7fcc40159c60 0x7fcc4015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.291+0000 7fcc3ed9d700 1 -- 172.21.15.89:0/4142445576 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcc40159890 msgr2=0x7fcc4015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.291+0000 7fcc3ed9d700 1 --2- 172.21.15.89:0/4142445576 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcc40159890 0x7fcc4015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.291+0000 7fcc3ed9d700 1 -- 172.21.15.89:0/4142445576 >> v2:172.21.15.111:3300/0 conn(0x7fcc40159c60 msgr2=0x7fcc4015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.291+0000 7fcc3ed9d700 1 --2- 172.21.15.89:0/4142445576 >> v2:172.21.15.111:3300/0 conn(0x7fcc40159c60 0x7fcc4015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.292+0000 7fcc3ed9d700 1 -- 172.21.15.89:0/4142445576 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4015a3d0 con 0x7fcc4015db10 2023-10-10T20:46:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.292+0000 7fcc3ed9d700 1 --2- 172.21.15.89:0/4142445576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc4015db10 0x7fcc40069560 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fcc400650f0 tx=0x7fcc38009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8287454af3917877 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:14.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.292+0000 7fcc3dd9b700 1 -- 172.21.15.89:0/4142445576 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3800e070 con 0x7fcc4015db10 2023-10-10T20:46:14.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.292+0000 7fcc3dd9b700 1 -- 172.21.15.89:0/4142445576 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3800ba50 con 0x7fcc4015db10 2023-10-10T20:46:14.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.292+0000 7fcc3dd9b700 1 -- 172.21.15.89:0/4142445576 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc380092e0 con 0x7fcc4015db10 2023-10-10T20:46:14.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.292+0000 7fcc44a67700 1 -- 172.21.15.89:0/4142445576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc4015db10 msgr2=0x7fcc40069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:14.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.292+0000 7fcc44a67700 1 --2- 172.21.15.89:0/4142445576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc4015db10 0x7fcc40069560 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fcc400650f0 tx=0x7fcc38009840 comp rx=0 tx=0).stop 2023-10-10T20:46:14.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.292+0000 7fcc44a67700 1 -- 172.21.15.89:0/4142445576 shutdown_connections 2023-10-10T20:46:14.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.292+0000 7fcc44a67700 1 --2- 172.21.15.89:0/4142445576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc4015db10 0x7fcc40069560 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.292+0000 7fcc44a67700 1 --2- 172.21.15.89:0/4142445576 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcc40159890 0x7fcc4015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.292+0000 7fcc44a67700 1 --2- 172.21.15.89:0/4142445576 >> v2:172.21.15.111:3300/0 conn(0x7fcc40159c60 0x7fcc4015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.292+0000 7fcc44a67700 1 -- 172.21.15.89:0/4142445576 >> 172.21.15.89:0/4142445576 conn(0x7fcc40068e20 msgr2=0x7fcc400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:14.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.292+0000 7fcc44a67700 1 -- 172.21.15.89:0/4142445576 shutdown_connections 2023-10-10T20:46:14.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.292+0000 7fcc44a67700 1 -- 172.21.15.89:0/4142445576 wait complete. 2023-10-10T20:46:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.294+0000 7fcc44a67700 1 Processor -- start 2023-10-10T20:46:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.294+0000 7fcc44a67700 1 -- start start 2023-10-10T20:46:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.294+0000 7fcc44a67700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc40159890 0x7fcc40158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.294+0000 7fcc44a67700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc40159c60 0x7fcc400b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.294+0000 7fcc44a67700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcc4015db10 0x7fcc400b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.294+0000 7fcc44a67700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc400b2d80 con 0x7fcc40159890 2023-10-10T20:46:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.294+0000 7fcc44a67700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc400b28f0 con 0x7fcc40159c60 2023-10-10T20:46:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.294+0000 7fcc44a67700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc400b1fc0 con 0x7fcc4015db10 2023-10-10T20:46:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.294+0000 7fcc3e59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc40159890 0x7fcc40158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.294+0000 7fcc3e59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc40159890 0x7fcc40158900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53958/0 (socket says 172.21.15.89:53958) 2023-10-10T20:46:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.294+0000 7fcc3e59c700 1 -- 172.21.15.89:0/3274296521 learned_addr learned my addr 172.21.15.89:0/3274296521 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.294+0000 7fcc3ed9d700 1 --2- 172.21.15.89:0/3274296521 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcc4015db10 0x7fcc400b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.295+0000 7fcc35d9b700 1 --2- 172.21.15.89:0/3274296521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc40159c60 0x7fcc400b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.295+0000 7fcc3e59c700 1 -- 172.21.15.89:0/3274296521 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcc4015db10 msgr2=0x7fcc400b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.295+0000 7fcc3e59c700 1 --2- 172.21.15.89:0/3274296521 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcc4015db10 0x7fcc400b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.295+0000 7fcc3e59c700 1 -- 172.21.15.89:0/3274296521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc40159c60 msgr2=0x7fcc400b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.295+0000 7fcc3e59c700 1 --2- 172.21.15.89:0/3274296521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc40159c60 0x7fcc400b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.295+0000 7fcc3e59c700 1 -- 172.21.15.89:0/3274296521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc38009ae0 con 0x7fcc40159890 2023-10-10T20:46:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.295+0000 7fcc35d9b700 1 --2- 172.21.15.89:0/3274296521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc40159c60 0x7fcc400b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.295+0000 7fcc3e59c700 1 --2- 172.21.15.89:0/3274296521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc40159890 0x7fcc40158900 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000d890 tx=0x7fcc30008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.295+0000 7fcc36ffd700 1 -- 172.21.15.89:0/3274296521 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc300125f0 con 0x7fcc40159890 2023-10-10T20:46:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.295+0000 7fcc36ffd700 1 -- 172.21.15.89:0/3274296521 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc30012750 con 0x7fcc40159890 2023-10-10T20:46:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.295+0000 7fcc44a67700 1 -- 172.21.15.89:0/3274296521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc400b22a0 con 0x7fcc40159890 2023-10-10T20:46:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.295+0000 7fcc36ffd700 1 -- 172.21.15.89:0/3274296521 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc30012ad0 con 0x7fcc40159890 2023-10-10T20:46:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.295+0000 7fcc44a67700 1 -- 172.21.15.89:0/3274296521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc400b1cb0 con 0x7fcc40159890 2023-10-10T20:46:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.296+0000 7fcc44a67700 1 -- 172.21.15.89:0/3274296521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4004c3c0 con 0x7fcc40159890 2023-10-10T20:46:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.296+0000 7fcc36ffd700 1 -- 172.21.15.89:0/3274296521 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcc3001e460 con 0x7fcc40159890 2023-10-10T20:46:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.297+0000 7fcc36ffd700 1 --2- 172.21.15.89:0/3274296521 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcc20067320 0x7fcc200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.297+0000 7fcc36ffd700 1 -- 172.21.15.89:0/3274296521 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5792+0+0 (secure 0 0 0) 0x7fcc300852d0 con 0x7fcc40159890 2023-10-10T20:46:14.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.297+0000 7fcc35d9b700 1 --2- 172.21.15.89:0/3274296521 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcc20067320 0x7fcc200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:14.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.297+0000 7fcc35d9b700 1 --2- 172.21.15.89:0/3274296521 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcc20067320 0x7fcc200697d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcc4016ceb0 tx=0x7fcc28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:14.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.301+0000 7fcc36ffd700 1 -- 172.21.15.89:0/3274296521 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc30051f10 con 0x7fcc40159890 2023-10-10T20:46:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.485+0000 7fcc44a67700 1 -- 172.21.15.89:0/3274296521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc400b2bd0 con 0x7fcc40159890 2023-10-10T20:46:14.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.486+0000 7fcc36ffd700 1 -- 172.21.15.89:0/3274296521 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+15509 (secure 0 0 0) 0x7fcc30055530 con 0x7fcc40159890 2023-10-10T20:46:14.489 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:46:14.489 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":200,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:44:24.011501+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:43:58.313137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-10T20:43:58.914819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"34.8","mappings":[{"from":0,"to":1}]},{"pgid":"34.18","mappings":[{"from":0,"to":2}]},{"pgid":"34.1f","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:46:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.488+0000 7fcc44a67700 1 -- 172.21.15.89:0/3274296521 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcc20067320 msgr2=0x7fcc200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.488+0000 7fcc44a67700 1 --2- 172.21.15.89:0/3274296521 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcc20067320 0x7fcc200697d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcc4016ceb0 tx=0x7fcc28008040 comp rx=0 tx=0).stop 2023-10-10T20:46:14.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.488+0000 7fcc44a67700 1 -- 172.21.15.89:0/3274296521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc40159890 msgr2=0x7fcc40158900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:14.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.488+0000 7fcc44a67700 1 --2- 172.21.15.89:0/3274296521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc40159890 0x7fcc40158900 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000d890 tx=0x7fcc30008040 comp rx=0 tx=0).stop 2023-10-10T20:46:14.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.488+0000 7fcc44a67700 1 -- 172.21.15.89:0/3274296521 shutdown_connections 2023-10-10T20:46:14.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.488+0000 7fcc44a67700 1 --2- 172.21.15.89:0/3274296521 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fcc4015db10 0x7fcc400b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.488+0000 7fcc44a67700 1 --2- 172.21.15.89:0/3274296521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc40159890 0x7fcc40158900 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.488+0000 7fcc44a67700 1 --2- 172.21.15.89:0/3274296521 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fcc20067320 0x7fcc200697d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.488+0000 7fcc44a67700 1 --2- 172.21.15.89:0/3274296521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc40159c60 0x7fcc400b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.488+0000 7fcc44a67700 1 -- 172.21.15.89:0/3274296521 >> 172.21.15.89:0/3274296521 conn(0x7fcc40068e20 msgr2=0x7fcc4015cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.488+0000 7fcc44a67700 1 -- 172.21.15.89:0/3274296521 shutdown_connections 2023-10-10T20:46:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.488+0000 7fcc44a67700 1 -- 172.21.15.89:0/3274296521 wait complete. 2023-10-10T20:46:14.504 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:46:14.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.617+0000 7fada3507700 1 Processor -- start 2023-10-10T20:46:14.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.617+0000 7fada3507700 1 -- start start 2023-10-10T20:46:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.618+0000 7fada3507700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad9c159c10 0x7fad9c15b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.618+0000 7fada3507700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fad9c0b5ea0 0x7fad9c068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.618+0000 7fada3507700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad9c15c480 con 0x7fad9c068bb0 2023-10-10T20:46:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.618+0000 7fada3507700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad9c0b2a40 con 0x7fad9c0b5ea0 2023-10-10T20:46:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.618+0000 7fada3507700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c15b5d0 con 0x7fad9c159c10 2023-10-10T20:46:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.618+0000 7fada0aa2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fad9c0b5ea0 0x7fad9c068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.618+0000 7fada0aa2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fad9c0b5ea0 0x7fad9c068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53974/0 (socket says 172.21.15.89:53974) 2023-10-10T20:46:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.618+0000 7fada0aa2700 1 -- 172.21.15.89:0/635029286 learned_addr learned my addr 172.21.15.89:0/635029286 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.618+0000 7fada12a3700 1 --2- 172.21.15.89:0/635029286 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad9c159c10 0x7fad9c15b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.618+0000 7fada0aa2700 1 -- 172.21.15.89:0/635029286 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad9c159c10 msgr2=0x7fad9c15b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.619+0000 7fada0aa2700 1 --2- 172.21.15.89:0/635029286 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad9c159c10 0x7fad9c15b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.619+0000 7fada0aa2700 1 -- 172.21.15.89:0/635029286 >> v1:172.21.15.111:6789/0 conn(0x7fad9c068bb0 legacy=0x7fad9c15c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.619+0000 7fada0aa2700 1 -- 172.21.15.89:0/635029286 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9c15b870 con 0x7fad9c0b5ea0 2023-10-10T20:46:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.619+0000 7fada0aa2700 1 --2- 172.21.15.89:0/635029286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad9c0b5ea0 0x7fad9c068670 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0099b0 tx=0x7fad8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4b738d89f29724 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.619+0000 7fad93fff700 1 -- 172.21.15.89:0/635029286 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c00b860 con 0x7fad9c0b5ea0 2023-10-10T20:46:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.619+0000 7fad93fff700 1 -- 172.21.15.89:0/635029286 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8c00b9c0 con 0x7fad9c0b5ea0 2023-10-10T20:46:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.619+0000 7fad93fff700 1 -- 172.21.15.89:0/635029286 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c00bd60 con 0x7fad9c0b5ea0 2023-10-10T20:46:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.619+0000 7fada3507700 1 -- 172.21.15.89:0/635029286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad9c0b5ea0 msgr2=0x7fad9c068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.619+0000 7fada3507700 1 --2- 172.21.15.89:0/635029286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad9c0b5ea0 0x7fad9c068670 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0099b0 tx=0x7fad8c009250 comp rx=0 tx=0).stop 2023-10-10T20:46:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.619+0000 7fada3507700 1 -- 172.21.15.89:0/635029286 shutdown_connections 2023-10-10T20:46:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.619+0000 7fada3507700 1 --2- 172.21.15.89:0/635029286 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad9c159c10 0x7fad9c15b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.619+0000 7fada3507700 1 --2- 172.21.15.89:0/635029286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad9c0b5ea0 0x7fad9c068670 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.619+0000 7fada3507700 1 -- 172.21.15.89:0/635029286 >> 172.21.15.89:0/635029286 conn(0x7fad9c099a20 msgr2=0x7fad9c0af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:14.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.619+0000 7fada3507700 1 -- 172.21.15.89:0/635029286 shutdown_connections 2023-10-10T20:46:14.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.619+0000 7fada3507700 1 -- 172.21.15.89:0/635029286 wait complete. 2023-10-10T20:46:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.620+0000 7fada3507700 1 Processor -- start 2023-10-10T20:46:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.620+0000 7fada3507700 1 -- start start 2023-10-10T20:46:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.621+0000 7fada3507700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad9c068bb0 0x7fad9c158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.621+0000 7fada3507700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad9c0b5ea0 0x7fad9c0af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.621+0000 7fada3507700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad9c159c10 0x7fad9c16a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.621+0000 7fada3507700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c0adf20 con 0x7fad9c068bb0 2023-10-10T20:46:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.621+0000 7fada3507700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c16a040 con 0x7fad9c159c10 2023-10-10T20:46:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.621+0000 7fada3507700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c16a320 con 0x7fad9c0b5ea0 2023-10-10T20:46:14.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.621+0000 7fada12a3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad9c068bb0 0x7fad9c158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:14.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.622+0000 7fada12a3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad9c068bb0 0x7fad9c158c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53980/0 (socket says 172.21.15.89:53980) 2023-10-10T20:46:14.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.622+0000 7fada12a3700 1 -- 172.21.15.89:0/3962698041 learned_addr learned my addr 172.21.15.89:0/3962698041 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:14.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.622+0000 7fada0aa2700 1 --2- 172.21.15.89:0/3962698041 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad9c0b5ea0 0x7fad9c0af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:14.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.622+0000 7fada1aa4700 1 --2- 172.21.15.89:0/3962698041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad9c159c10 0x7fad9c16a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:14.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.622+0000 7fada12a3700 1 -- 172.21.15.89:0/3962698041 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad9c0b5ea0 msgr2=0x7fad9c0af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:14.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.622+0000 7fada12a3700 1 --2- 172.21.15.89:0/3962698041 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad9c0b5ea0 0x7fad9c0af0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.622+0000 7fada12a3700 1 -- 172.21.15.89:0/3962698041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad9c159c10 msgr2=0x7fad9c16a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:14.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.622+0000 7fada12a3700 1 --2- 172.21.15.89:0/3962698041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad9c159c10 0x7fad9c16a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.622+0000 7fada12a3700 1 -- 172.21.15.89:0/3962698041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c0094f0 con 0x7fad9c068bb0 2023-10-10T20:46:14.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.622+0000 7fada1aa4700 1 --2- 172.21.15.89:0/3962698041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad9c159c10 0x7fad9c16a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:14.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.622+0000 7fada12a3700 1 --2- 172.21.15.89:0/3962698041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad9c068bb0 0x7fad9c158c90 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fad98009fa0 tx=0x7fad98009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:14.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.622+0000 7fad917fa700 1 -- 172.21.15.89:0/3962698041 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9800ec70 con 0x7fad9c068bb0 2023-10-10T20:46:14.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.622+0000 7fad917fa700 1 -- 172.21.15.89:0/3962698041 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9800edd0 con 0x7fad9c068bb0 2023-10-10T20:46:14.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.622+0000 7fad917fa700 1 -- 172.21.15.89:0/3962698041 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad98014670 con 0x7fad9c068bb0 2023-10-10T20:46:14.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.622+0000 7fada3507700 1 -- 172.21.15.89:0/3962698041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9c0ae5d0 con 0x7fad9c068bb0 2023-10-10T20:46:14.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.622+0000 7fada3507700 1 -- 172.21.15.89:0/3962698041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9c0aec10 con 0x7fad9c068bb0 2023-10-10T20:46:14.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.623+0000 7fada3507700 1 -- 172.21.15.89:0/3962698041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad9c0aeda0 con 0x7fad9c068bb0 2023-10-10T20:46:14.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.623+0000 7fad917fa700 1 -- 172.21.15.89:0/3962698041 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fad980149e0 con 0x7fad9c068bb0 2023-10-10T20:46:14.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.624+0000 7fad917fa700 1 --2- 172.21.15.89:0/3962698041 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fad80067320 0x7fad800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:14.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.624+0000 7fad917fa700 1 -- 172.21.15.89:0/3962698041 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5792+0+0 (secure 0 0 0) 0x7fad980853a0 con 0x7fad9c068bb0 2023-10-10T20:46:14.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.624+0000 7fada0aa2700 1 --2- 172.21.15.89:0/3962698041 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fad80067320 0x7fad800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:14.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.624+0000 7fada0aa2700 1 --2- 172.21.15.89:0/3962698041 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fad80067320 0x7fad800697d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0099b0 tx=0x7fad8c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:14.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.628+0000 7fad917fa700 1 -- 172.21.15.89:0/3962698041 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad98051fe0 con 0x7fad9c068bb0 2023-10-10T20:46:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.855+0000 7fada3507700 1 -- 172.21.15.89:0/3962698041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fad9c04c3c0 con 0x7fad9c068bb0 2023-10-10T20:46:14.858 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-10-10T20:46:14.855+0000 7fd0184ca700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:46:14.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.860+0000 7fad917fa700 1 -- 172.21.15.89:0/3962698041 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v134) v1 ==== 161+0+0 (secure 0 0 0) 0x7fad980099c0 con 0x7fad9c068bb0 2023-10-10T20:46:14.863 INFO:tasks.ceph.mds.a.smithi089.stderr: -3253> 2023-10-10T20:44:05.865+0000 7eff21e1f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:46:14.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.862+0000 7fada3507700 1 -- 172.21.15.89:0/3962698041 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fad80067320 msgr2=0x7fad800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:14.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.862+0000 7fada3507700 1 --2- 172.21.15.89:0/3962698041 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fad80067320 0x7fad800697d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0099b0 tx=0x7fad8c017000 comp rx=0 tx=0).stop 2023-10-10T20:46:14.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.862+0000 7fada3507700 1 -- 172.21.15.89:0/3962698041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad9c068bb0 msgr2=0x7fad9c158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:14.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.862+0000 7fada3507700 1 --2- 172.21.15.89:0/3962698041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad9c068bb0 0x7fad9c158c90 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fad98009fa0 tx=0x7fad98009880 comp rx=0 tx=0).stop 2023-10-10T20:46:14.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.862+0000 7fada3507700 1 -- 172.21.15.89:0/3962698041 shutdown_connections 2023-10-10T20:46:14.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.862+0000 7fada3507700 1 --2- 172.21.15.89:0/3962698041 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fad80067320 0x7fad800697d0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.862+0000 7fada3507700 1 --2- 172.21.15.89:0/3962698041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad9c159c10 0x7fad9c16a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.862+0000 7fada3507700 1 --2- 172.21.15.89:0/3962698041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad9c068bb0 0x7fad9c158c90 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.862+0000 7fada3507700 1 --2- 172.21.15.89:0/3962698041 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fad9c0b5ea0 0x7fad9c0af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.862+0000 7fada3507700 1 -- 172.21.15.89:0/3962698041 >> 172.21.15.89:0/3962698041 conn(0x7fad9c099a20 msgr2=0x7fad9c09b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:14.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.862+0000 7fada3507700 1 -- 172.21.15.89:0/3962698041 shutdown_connections 2023-10-10T20:46:14.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.862+0000 7fada3507700 1 -- 172.21.15.89:0/3962698041 wait complete. 2023-10-10T20:46:14.868 INFO:teuthology.orchestra.run.smithi089.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:46:14.877 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:46:14.888 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-10T20:46:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.991+0000 7fdd7a21c700 1 Processor -- start 2023-10-10T20:46:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.991+0000 7fdd7a21c700 1 -- start start 2023-10-10T20:46:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.992+0000 7fdd7a21c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdd74159b10 0x7fdd740cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.992+0000 7fdd7a21c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd7415d730 0x7fdd7415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.992+0000 7fdd7a21c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd740c74b0 con 0x7fdd74159ee0 2023-10-10T20:46:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.992+0000 7fdd7a21c700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd740c8700 con 0x7fdd74159b10 2023-10-10T20:46:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.992+0000 7fdd7a21c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd740c9070 con 0x7fdd7415d730 2023-10-10T20:46:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.993+0000 7fdd72ffd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd7415d730 0x7fdd7415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.993+0000 7fdd737fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdd74159b10 0x7fdd740cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.993+0000 7fdd72ffd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd7415d730 0x7fdd7415dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:44810/0 (socket says 172.21.15.89:44810) 2023-10-10T20:46:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.993+0000 7fdd72ffd700 1 -- 172.21.15.89:0/2176302577 learned_addr learned my addr 172.21.15.89:0/2176302577 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.993+0000 7fdd72ffd700 1 -- 172.21.15.89:0/2176302577 >> v2:172.21.15.111:3300/0 conn(0x7fdd74159b10 msgr2=0x7fdd740cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.993+0000 7fdd72ffd700 1 --2- 172.21.15.89:0/2176302577 >> v2:172.21.15.111:3300/0 conn(0x7fdd74159b10 0x7fdd740cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.993+0000 7fdd72ffd700 1 -- 172.21.15.89:0/2176302577 >> v1:172.21.15.111:6789/0 conn(0x7fdd74159ee0 legacy=0x7fdd7415a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.993+0000 7fdd72ffd700 1 -- 172.21.15.89:0/2176302577 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7415e0e0 con 0x7fdd7415d730 2023-10-10T20:46:15.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.993+0000 7fdd72ffd700 1 --2- 172.21.15.89:0/2176302577 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd7415d730 0x7fdd7415dba0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdd680099b0 tx=0x7fdd68009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=312618eab3c3e8ff server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:15.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.993+0000 7fdd727fc700 1 -- 172.21.15.89:0/2176302577 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd68009ea0 con 0x7fdd7415d730 2023-10-10T20:46:15.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.993+0000 7fdd727fc700 1 -- 172.21.15.89:0/2176302577 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6800b8f0 con 0x7fdd7415d730 2023-10-10T20:46:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.993+0000 7fdd727fc700 1 -- 172.21.15.89:0/2176302577 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6800bc90 con 0x7fdd7415d730 2023-10-10T20:46:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.994+0000 7fdd7a21c700 1 -- 172.21.15.89:0/2176302577 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd7415d730 msgr2=0x7fdd7415dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.994+0000 7fdd7a21c700 1 --2- 172.21.15.89:0/2176302577 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd7415d730 0x7fdd7415dba0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdd680099b0 tx=0x7fdd68009250 comp rx=0 tx=0).stop 2023-10-10T20:46:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.994+0000 7fdd7a21c700 1 -- 172.21.15.89:0/2176302577 shutdown_connections 2023-10-10T20:46:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.994+0000 7fdd7a21c700 1 --2- 172.21.15.89:0/2176302577 >> v2:172.21.15.111:3300/0 conn(0x7fdd74159b10 0x7fdd740cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.994+0000 7fdd7a21c700 1 --2- 172.21.15.89:0/2176302577 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd7415d730 0x7fdd7415dba0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.994+0000 7fdd7a21c700 1 -- 172.21.15.89:0/2176302577 >> 172.21.15.89:0/2176302577 conn(0x7fdd740c9870 msgr2=0x7fdd740c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.994+0000 7fdd7a21c700 1 -- 172.21.15.89:0/2176302577 shutdown_connections 2023-10-10T20:46:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.994+0000 7fdd7a21c700 1 -- 172.21.15.89:0/2176302577 wait complete. 2023-10-10T20:46:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.995+0000 7fdd7a21c700 1 Processor -- start 2023-10-10T20:46:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.995+0000 7fdd7a21c700 1 -- start start 2023-10-10T20:46:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.996+0000 7fdd7a21c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd74159b10 0x7fdd74155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.996+0000 7fdd7a21c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd74159ee0 0x7fdd740b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.996+0000 7fdd7a21c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd7415d730 0x7fdd740b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.996+0000 7fdd7a21c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7415ab90 con 0x7fdd7415d730 2023-10-10T20:46:15.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.996+0000 7fdd7a21c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7416e580 con 0x7fdd74159ee0 2023-10-10T20:46:15.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.996+0000 7fdd7a21c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7416d720 con 0x7fdd74159b10 2023-10-10T20:46:15.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.996+0000 7fdd73fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd7415d730 0x7fdd740b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:15.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.996+0000 7fdd72ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd74159ee0 0x7fdd740b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:15.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.996+0000 7fdd73fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd7415d730 0x7fdd740b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54010/0 (socket says 172.21.15.89:54010) 2023-10-10T20:46:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.996+0000 7fdd737fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd74159b10 0x7fdd74155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.996+0000 7fdd73fff700 1 -- 172.21.15.89:0/3336945383 learned_addr learned my addr 172.21.15.89:0/3336945383 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.996+0000 7fdd73fff700 1 -- 172.21.15.89:0/3336945383 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd74159b10 msgr2=0x7fdd74155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.996+0000 7fdd73fff700 1 --2- 172.21.15.89:0/3336945383 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd74159b10 0x7fdd74155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.996+0000 7fdd73fff700 1 -- 172.21.15.89:0/3336945383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd74159ee0 msgr2=0x7fdd740b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.996+0000 7fdd73fff700 1 --2- 172.21.15.89:0/3336945383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd74159ee0 0x7fdd740b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.996+0000 7fdd73fff700 1 -- 172.21.15.89:0/3336945383 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd680094f0 con 0x7fdd7415d730 2023-10-10T20:46:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.997+0000 7fdd73fff700 1 --2- 172.21.15.89:0/3336945383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd7415d730 0x7fdd740b1bc0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00c560 tx=0x7fdd6c007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.997+0000 7fdd5b7fe700 1 -- 172.21.15.89:0/3336945383 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c011590 con 0x7fdd7415d730 2023-10-10T20:46:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.997+0000 7fdd5b7fe700 1 -- 172.21.15.89:0/3336945383 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6c0116f0 con 0x7fdd7415d730 2023-10-10T20:46:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.997+0000 7fdd5b7fe700 1 -- 172.21.15.89:0/3336945383 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c011ab0 con 0x7fdd7415d730 2023-10-10T20:46:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.997+0000 7fdd7a21c700 1 -- 172.21.15.89:0/3336945383 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7416da00 con 0x7fdd7415d730 2023-10-10T20:46:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.997+0000 7fdd7a21c700 1 -- 172.21.15.89:0/3336945383 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7416e2a0 con 0x7fdd7415d730 2023-10-10T20:46:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.998+0000 7fdd7a21c700 1 -- 172.21.15.89:0/3336945383 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7416de00 con 0x7fdd7415d730 2023-10-10T20:46:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.998+0000 7fdd5b7fe700 1 -- 172.21.15.89:0/3336945383 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdd6c01b3b0 con 0x7fdd7415d730 2023-10-10T20:46:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.999+0000 7fdd5b7fe700 1 --2- 172.21.15.89:0/3336945383 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdd60067350 0x7fdd60069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.999+0000 7fdd5b7fe700 1 -- 172.21.15.89:0/3336945383 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5878+0+0 (secure 0 0 0) 0x7fdd6c084160 con 0x7fdd7415d730 2023-10-10T20:46:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.999+0000 7fdd737fe700 1 --2- 172.21.15.89:0/3336945383 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdd60067350 0x7fdd60069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:14.999+0000 7fdd737fe700 1 --2- 172.21.15.89:0/3336945383 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdd60067350 0x7fdd60069800 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400f570 tx=0x7fdd6400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:15.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:15.003+0000 7fdd5b7fe700 1 -- 172.21.15.89:0/3336945383 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd6c050d40 con 0x7fdd7415d730 2023-10-10T20:46:15.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:15.226+0000 7fdd7a21c700 1 -- 172.21.15.89:0/3336945383 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdd7404c3c0 con 0x7fdd7415d730 2023-10-10T20:46:15.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:15.860+0000 7fdd5b7fe700 1 -- 172.21.15.89:0/3336945383 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v135) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdd6c054360 con 0x7fdd7415d730 2023-10-10T20:46:15.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:15.862+0000 7fdd7a21c700 1 -- 172.21.15.89:0/3336945383 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdd60067350 msgr2=0x7fdd60069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:15.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:15.862+0000 7fdd7a21c700 1 --2- 172.21.15.89:0/3336945383 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdd60067350 0x7fdd60069800 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400f570 tx=0x7fdd6400b040 comp rx=0 tx=0).stop 2023-10-10T20:46:15.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:15.862+0000 7fdd7a21c700 1 -- 172.21.15.89:0/3336945383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd7415d730 msgr2=0x7fdd740b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:15.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:15.862+0000 7fdd7a21c700 1 --2- 172.21.15.89:0/3336945383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd7415d730 0x7fdd740b1bc0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00c560 tx=0x7fdd6c007ef0 comp rx=0 tx=0).stop 2023-10-10T20:46:15.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:15.862+0000 7fdd7a21c700 1 -- 172.21.15.89:0/3336945383 shutdown_connections 2023-10-10T20:46:15.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:15.862+0000 7fdd7a21c700 1 --2- 172.21.15.89:0/3336945383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd7415d730 0x7fdd740b1bc0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:15.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:15.862+0000 7fdd7a21c700 1 --2- 172.21.15.89:0/3336945383 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd74159b10 0x7fdd74155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:15.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:15.862+0000 7fdd7a21c700 1 --2- 172.21.15.89:0/3336945383 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdd60067350 0x7fdd60069800 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:15.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:15.862+0000 7fdd7a21c700 1 --2- 172.21.15.89:0/3336945383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd74159ee0 0x7fdd740b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:15.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:15.862+0000 7fdd7a21c700 1 -- 172.21.15.89:0/3336945383 >> 172.21.15.89:0/3336945383 conn(0x7fdd740c9870 msgr2=0x7fdd740695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:15.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:15.862+0000 7fdd7a21c700 1 -- 172.21.15.89:0/3336945383 shutdown_connections 2023-10-10T20:46:15.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:15.862+0000 7fdd7a21c700 1 -- 172.21.15.89:0/3336945383 wait complete. 2023-10-10T20:46:15.877 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:46:16.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.033+0000 7f8234740700 1 Processor -- start 2023-10-10T20:46:16.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.033+0000 7f8234740700 1 -- start start 2023-10-10T20:46:16.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.034+0000 7f8234740700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f822c0cd910 0x7f822c0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:16.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.034+0000 7f8234740700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f822c0b55a0 0x7f822c0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.034+0000 7f8234740700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f822c0b5b00 con 0x7f822c159ec0 2023-10-10T20:46:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.034+0000 7f8234740700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f822c0b5da0 con 0x7f822c0b55a0 2023-10-10T20:46:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.034+0000 7f8234740700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822c0b6080 con 0x7f822c0cd910 2023-10-10T20:46:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.034+0000 7f8231cdb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f822c0cd910 0x7f822c0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.034+0000 7f8232cdd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f822c0b55a0 0x7f822c0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:16.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.034+0000 7f8231cdb700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f822c0cd910 0x7f822c0ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:44828/0 (socket says 172.21.15.89:44828) 2023-10-10T20:46:16.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.034+0000 7f8232cdd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f822c0b55a0 0x7f822c0ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:40290/0 (socket says 172.21.15.89:40290) 2023-10-10T20:46:16.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.034+0000 7f8231cdb700 1 -- 172.21.15.89:0/1545164182 learned_addr learned my addr 172.21.15.89:0/1545164182 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:16.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.035+0000 7f8231cdb700 1 -- 172.21.15.89:0/1545164182 >> v2:172.21.15.111:3300/0 conn(0x7f822c0b55a0 msgr2=0x7f822c0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:16.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.035+0000 7f8231cdb700 1 --2- 172.21.15.89:0/1545164182 >> v2:172.21.15.111:3300/0 conn(0x7f822c0b55a0 0x7f822c0ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:16.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.035+0000 7f8231cdb700 1 -- 172.21.15.89:0/1545164182 >> v1:172.21.15.111:6789/0 conn(0x7f822c159ec0 legacy=0x7f822c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:16.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.035+0000 7f8231cdb700 1 -- 172.21.15.89:0/1545164182 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c0cc100 con 0x7f822c0cd910 2023-10-10T20:46:16.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.035+0000 7f8232cdd700 1 --2- 172.21.15.89:0/1545164182 >> v2:172.21.15.111:3300/0 conn(0x7f822c0b55a0 0x7f822c0ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:46:16.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.035+0000 7f8231cdb700 1 --2- 172.21.15.89:0/1545164182 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f822c0cd910 0x7f822c0ce340 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f82200099b0 tx=0x7f8220009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d16ca3ff2158594 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:16.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.035+0000 7f82314da700 1 -- 172.21.15.89:0/1545164182 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220009ea0 con 0x7f822c0cd910 2023-10-10T20:46:16.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.035+0000 7f82314da700 1 -- 172.21.15.89:0/1545164182 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822000b860 con 0x7f822c0cd910 2023-10-10T20:46:16.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.035+0000 7f82314da700 1 -- 172.21.15.89:0/1545164182 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822000bc00 con 0x7f822c0cd910 2023-10-10T20:46:16.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.035+0000 7f8234740700 1 -- 172.21.15.89:0/1545164182 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f822c0cd910 msgr2=0x7f822c0ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:16.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.035+0000 7f8234740700 1 --2- 172.21.15.89:0/1545164182 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f822c0cd910 0x7f822c0ce340 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f82200099b0 tx=0x7f8220009250 comp rx=0 tx=0).stop 2023-10-10T20:46:16.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.036+0000 7f8234740700 1 -- 172.21.15.89:0/1545164182 shutdown_connections 2023-10-10T20:46:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.036+0000 7f8234740700 1 --2- 172.21.15.89:0/1545164182 >> v2:172.21.15.111:3300/0 conn(0x7f822c0b55a0 0x7f822c0ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.036+0000 7f8234740700 1 --2- 172.21.15.89:0/1545164182 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f822c0cd910 0x7f822c0ce340 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.036+0000 7f8234740700 1 -- 172.21.15.89:0/1545164182 >> 172.21.15.89:0/1545164182 conn(0x7f822c0999f0 msgr2=0x7f822c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.036+0000 7f8234740700 1 -- 172.21.15.89:0/1545164182 shutdown_connections 2023-10-10T20:46:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.036+0000 7f8234740700 1 -- 172.21.15.89:0/1545164182 wait complete. 2023-10-10T20:46:16.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.037+0000 7f8234740700 1 Processor -- start 2023-10-10T20:46:16.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.037+0000 7f8234740700 1 -- start start 2023-10-10T20:46:16.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.037+0000 7f8234740700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f822c0b55a0 0x7f822c155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:16.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.037+0000 7f8234740700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f822c159ec0 0x7f822c157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:16.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.037+0000 7f8234740700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f822c069510 0x7f822c069990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:16.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.037+0000 7f8234740700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c158eb0 con 0x7f822c069510 2023-10-10T20:46:16.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.037+0000 7f8234740700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c1582a0 con 0x7f822c0b55a0 2023-10-10T20:46:16.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.037+0000 7f8234740700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822c158580 con 0x7f822c159ec0 2023-10-10T20:46:16.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.038+0000 7f8232cdd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f822c069510 0x7f822c069990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:16.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.038+0000 7f8232cdd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f822c069510 0x7f822c069990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54024/0 (socket says 172.21.15.89:54024) 2023-10-10T20:46:16.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.038+0000 7f8232cdd700 1 -- 172.21.15.89:0/2684902448 learned_addr learned my addr 172.21.15.89:0/2684902448 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:16.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.038+0000 7f82324dc700 1 --2- 172.21.15.89:0/2684902448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f822c0b55a0 0x7f822c155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:16.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.038+0000 7f8231cdb700 1 --2- 172.21.15.89:0/2684902448 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f822c159ec0 0x7f822c157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:16.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.038+0000 7f8232cdd700 1 -- 172.21.15.89:0/2684902448 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f822c159ec0 msgr2=0x7f822c157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:16.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.038+0000 7f8232cdd700 1 --2- 172.21.15.89:0/2684902448 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f822c159ec0 0x7f822c157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:16.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.038+0000 7f8232cdd700 1 -- 172.21.15.89:0/2684902448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f822c0b55a0 msgr2=0x7f822c155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:16.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.038+0000 7f8232cdd700 1 --2- 172.21.15.89:0/2684902448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f822c0b55a0 0x7f822c155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:16.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.038+0000 7f8232cdd700 1 -- 172.21.15.89:0/2684902448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c009420 con 0x7f822c069510 2023-10-10T20:46:16.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.038+0000 7f8232cdd700 1 --2- 172.21.15.89:0/2684902448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f822c069510 0x7f822c069990 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f821c00ec80 tx=0x7f821c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:16.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.038+0000 7f822a7fc700 1 -- 172.21.15.89:0/2684902448 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c00c850 con 0x7f822c069510 2023-10-10T20:46:16.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.038+0000 7f822a7fc700 1 -- 172.21.15.89:0/2684902448 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821c00c9b0 con 0x7f822c069510 2023-10-10T20:46:16.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.038+0000 7f8234740700 1 -- 172.21.15.89:0/2684902448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82200094f0 con 0x7f822c069510 2023-10-10T20:46:16.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.038+0000 7f822a7fc700 1 -- 172.21.15.89:0/2684902448 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c00cd50 con 0x7f822c069510 2023-10-10T20:46:16.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.038+0000 7f8234740700 1 -- 172.21.15.89:0/2684902448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822c158b40 con 0x7f822c069510 2023-10-10T20:46:16.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.039+0000 7f8234740700 1 -- 172.21.15.89:0/2684902448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822c159020 con 0x7f822c069510 2023-10-10T20:46:16.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.040+0000 7f822a7fc700 1 -- 172.21.15.89:0/2684902448 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f821c013680 con 0x7f822c069510 2023-10-10T20:46:16.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.040+0000 7f822a7fc700 1 --2- 172.21.15.89:0/2684902448 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8218067320 0x7f82180697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:16.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.040+0000 7f822a7fc700 1 -- 172.21.15.89:0/2684902448 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5878+0+0 (secure 0 0 0) 0x7f821c084d50 con 0x7f822c069510 2023-10-10T20:46:16.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.040+0000 7f82324dc700 1 --2- 172.21.15.89:0/2684902448 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8218067320 0x7f82180697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:16.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.040+0000 7f82324dc700 1 --2- 172.21.15.89:0/2684902448 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8218067320 0x7f82180697d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f822400c580 tx=0x7f8224001a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:16.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.044+0000 7f822a7fc700 1 -- 172.21.15.89:0/2684902448 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f821c051930 con 0x7f822c069510 2023-10-10T20:46:16.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:16.228+0000 7f8234740700 1 -- 172.21.15.89:0/2684902448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f822c0718b0 con 0x7f822c069510 2023-10-10T20:46:17.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.206+0000 7f822a7fc700 1 -- 172.21.15.89:0/2684902448 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v202) v1 ==== 181+0+0 (secure 0 0 0) 0x7f821c0170d0 con 0x7f822c069510 2023-10-10T20:46:17.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.209+0000 7f8234740700 1 -- 172.21.15.89:0/2684902448 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8218067320 msgr2=0x7f82180697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:17.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.209+0000 7f8234740700 1 --2- 172.21.15.89:0/2684902448 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8218067320 0x7f82180697d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f822400c580 tx=0x7f8224001a80 comp rx=0 tx=0).stop 2023-10-10T20:46:17.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.210+0000 7f8234740700 1 -- 172.21.15.89:0/2684902448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f822c069510 msgr2=0x7f822c069990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:17.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.210+0000 7f8234740700 1 --2- 172.21.15.89:0/2684902448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f822c069510 0x7f822c069990 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f821c00ec80 tx=0x7f821c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:17.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.210+0000 7f8234740700 1 -- 172.21.15.89:0/2684902448 shutdown_connections 2023-10-10T20:46:17.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.210+0000 7f8234740700 1 --2- 172.21.15.89:0/2684902448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f822c069510 0x7f822c069990 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:17.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.210+0000 7f8234740700 1 --2- 172.21.15.89:0/2684902448 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8218067320 0x7f82180697d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:17.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.210+0000 7f8234740700 1 --2- 172.21.15.89:0/2684902448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f822c0b55a0 0x7f822c155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:17.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.210+0000 7f8234740700 1 --2- 172.21.15.89:0/2684902448 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f822c159ec0 0x7f822c157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:17.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.210+0000 7f8234740700 1 -- 172.21.15.89:0/2684902448 >> 172.21.15.89:0/2684902448 conn(0x7f822c0999f0 msgr2=0x7f822c152410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:17.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.210+0000 7f8234740700 1 -- 172.21.15.89:0/2684902448 shutdown_connections 2023-10-10T20:46:17.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.210+0000 7f8234740700 1 -- 172.21.15.89:0/2684902448 wait complete. 2023-10-10T20:46:17.218 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:46:17.225 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:46:17.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.341+0000 7fc754f5a700 1 Processor -- start 2023-10-10T20:46:17.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.341+0000 7fc754f5a700 1 -- start start 2023-10-10T20:46:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc754f5a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc750159ec0 0x7fc7500b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc754f5a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc7500cf350 0x7fc7500cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc754f5a700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7500b5b00 con 0x7fc7500b53d0 2023-10-10T20:46:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc754f5a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7500b5da0 con 0x7fc7500cf350 2023-10-10T20:46:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc754f5a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7500b6080 con 0x7fc750159ec0 2023-10-10T20:46:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc74ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc7500cf350 0x7fc7500cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc74ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc7500cf350 0x7fc7500cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54040/0 (socket says 172.21.15.89:54040) 2023-10-10T20:46:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc74ed9d700 1 -- 172.21.15.89:0/3510157742 learned_addr learned my addr 172.21.15.89:0/3510157742 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc74ed9d700 1 -- 172.21.15.89:0/3510157742 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc750159ec0 msgr2=0x7fc7500b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc74ed9d700 1 --2- 172.21.15.89:0/3510157742 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc750159ec0 0x7fc7500b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc74ed9d700 1 -- 172.21.15.89:0/3510157742 >> v1:172.21.15.111:6789/0 conn(0x7fc7500b53d0 legacy=0x7fc7500cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc74ed9d700 1 -- 172.21.15.89:0/3510157742 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7500cc100 con 0x7fc7500cf350 2023-10-10T20:46:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc74ed9d700 1 --2- 172.21.15.89:0/3510157742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7500cf350 0x7fc7500cd010 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fc748009f20 tx=0x7fc748009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941efd922cb965ae server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:17.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc74dd9b700 1 -- 172.21.15.89:0/3510157742 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74800e070 con 0x7fc7500cf350 2023-10-10T20:46:17.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc74dd9b700 1 -- 172.21.15.89:0/3510157742 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74800ba40 con 0x7fc7500cf350 2023-10-10T20:46:17.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc74dd9b700 1 -- 172.21.15.89:0/3510157742 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7480092e0 con 0x7fc7500cf350 2023-10-10T20:46:17.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc754f5a700 1 -- 172.21.15.89:0/3510157742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7500cf350 msgr2=0x7fc7500cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:17.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc754f5a700 1 --2- 172.21.15.89:0/3510157742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7500cf350 0x7fc7500cd010 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fc748009f20 tx=0x7fc748009840 comp rx=0 tx=0).stop 2023-10-10T20:46:17.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc754f5a700 1 -- 172.21.15.89:0/3510157742 shutdown_connections 2023-10-10T20:46:17.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc754f5a700 1 --2- 172.21.15.89:0/3510157742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7500cf350 0x7fc7500cd010 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:17.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc754f5a700 1 --2- 172.21.15.89:0/3510157742 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc750159ec0 0x7fc7500b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:17.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc754f5a700 1 -- 172.21.15.89:0/3510157742 >> 172.21.15.89:0/3510157742 conn(0x7fc7500999f0 msgr2=0x7fc7500b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:17.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc754f5a700 1 -- 172.21.15.89:0/3510157742 shutdown_connections 2023-10-10T20:46:17.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.343+0000 7fc754f5a700 1 -- 172.21.15.89:0/3510157742 wait complete. 2023-10-10T20:46:17.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.344+0000 7fc754f5a700 1 Processor -- start 2023-10-10T20:46:17.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.345+0000 7fc754f5a700 1 -- start start 2023-10-10T20:46:17.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.345+0000 7fc754f5a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7500b53d0 0x7fc750169a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:17.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.345+0000 7fc754f5a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7500cf350 0x7fc750169fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:17.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.345+0000 7fc754f5a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc750159ec0 0x7fc750068c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:17.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.345+0000 7fc754f5a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75016a510 con 0x7fc7500cf350 2023-10-10T20:46:17.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.345+0000 7fc754f5a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc750144cc0 con 0x7fc750159ec0 2023-10-10T20:46:17.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.345+0000 7fc754f5a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75016b5f0 con 0x7fc7500b53d0 2023-10-10T20:46:17.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.345+0000 7fc745d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7500cf350 0x7fc750169fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:17.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.345+0000 7fc745d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7500cf350 0x7fc750169fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54054/0 (socket says 172.21.15.89:54054) 2023-10-10T20:46:17.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.345+0000 7fc745d9b700 1 -- 172.21.15.89:0/1190688942 learned_addr learned my addr 172.21.15.89:0/1190688942 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:17.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.346+0000 7fc74e59c700 1 --2- 172.21.15.89:0/1190688942 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7500b53d0 0x7fc750169a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:17.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.346+0000 7fc74ed9d700 1 --2- 172.21.15.89:0/1190688942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc750159ec0 0x7fc750068c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:17.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.346+0000 7fc745d9b700 1 -- 172.21.15.89:0/1190688942 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7500b53d0 msgr2=0x7fc750169a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:17.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.346+0000 7fc745d9b700 1 --2- 172.21.15.89:0/1190688942 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7500b53d0 0x7fc750169a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:17.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.346+0000 7fc745d9b700 1 -- 172.21.15.89:0/1190688942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc750159ec0 msgr2=0x7fc750068c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:17.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.346+0000 7fc745d9b700 1 --2- 172.21.15.89:0/1190688942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc750159ec0 0x7fc750068c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:17.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.346+0000 7fc745d9b700 1 -- 172.21.15.89:0/1190688942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc748009ae0 con 0x7fc7500cf350 2023-10-10T20:46:17.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.346+0000 7fc745d9b700 1 --2- 172.21.15.89:0/1190688942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7500cf350 0x7fc750169fd0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fc73800b600 tx=0x7fc7380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:17.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.346+0000 7fc746ffd700 1 -- 172.21.15.89:0/1190688942 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73800f510 con 0x7fc7500cf350 2023-10-10T20:46:17.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.346+0000 7fc746ffd700 1 -- 172.21.15.89:0/1190688942 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73800f670 con 0x7fc7500cf350 2023-10-10T20:46:17.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.346+0000 7fc754f5a700 1 -- 172.21.15.89:0/1190688942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc75016ad00 con 0x7fc7500cf350 2023-10-10T20:46:17.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.346+0000 7fc746ffd700 1 -- 172.21.15.89:0/1190688942 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73800fa30 con 0x7fc7500cf350 2023-10-10T20:46:17.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.346+0000 7fc754f5a700 1 -- 172.21.15.89:0/1190688942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75016b300 con 0x7fc7500cf350 2023-10-10T20:46:17.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.347+0000 7fc754f5a700 1 -- 172.21.15.89:0/1190688942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7500647c0 con 0x7fc7500cf350 2023-10-10T20:46:17.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.347+0000 7fc746ffd700 1 -- 172.21.15.89:0/1190688942 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc73801a400 con 0x7fc7500cf350 2023-10-10T20:46:17.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.348+0000 7fc746ffd700 1 --2- 172.21.15.89:0/1190688942 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc730067320 0x7fc7300697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:17.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.348+0000 7fc746ffd700 1 -- 172.21.15.89:0/1190688942 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5440+0+0 (secure 0 0 0) 0x7fc738084730 con 0x7fc7500cf350 2023-10-10T20:46:17.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.348+0000 7fc74e59c700 1 --2- 172.21.15.89:0/1190688942 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc730067320 0x7fc7300697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:17.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.348+0000 7fc74e59c700 1 --2- 172.21.15.89:0/1190688942 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc730067320 0x7fc7300697d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc740006fd0 tx=0x7fc740008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:17.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.352+0000 7fc746ffd700 1 -- 172.21.15.89:0/1190688942 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7500647c0 con 0x7fc7500cf350 2023-10-10T20:46:17.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:17.535+0000 7fc754f5a700 1 -- 172.21.15.89:0/1190688942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc7500647c0 con 0x7fc7500cf350 2023-10-10T20:46:18.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.211+0000 7fc746ffd700 1 -- 172.21.15.89:0/1190688942 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v203) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc7500647c0 con 0x7fc7500cf350 2023-10-10T20:46:18.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.215+0000 7fc754f5a700 1 -- 172.21.15.89:0/1190688942 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc730067320 msgr2=0x7fc7300697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:18.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.215+0000 7fc754f5a700 1 --2- 172.21.15.89:0/1190688942 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc730067320 0x7fc7300697d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc740006fd0 tx=0x7fc740008040 comp rx=0 tx=0).stop 2023-10-10T20:46:18.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.215+0000 7fc754f5a700 1 -- 172.21.15.89:0/1190688942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7500cf350 msgr2=0x7fc750169fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:18.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.215+0000 7fc754f5a700 1 --2- 172.21.15.89:0/1190688942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7500cf350 0x7fc750169fd0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fc73800b600 tx=0x7fc7380091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:18.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.215+0000 7fc754f5a700 1 -- 172.21.15.89:0/1190688942 shutdown_connections 2023-10-10T20:46:18.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.215+0000 7fc754f5a700 1 --2- 172.21.15.89:0/1190688942 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc730067320 0x7fc7300697d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:18.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.215+0000 7fc754f5a700 1 --2- 172.21.15.89:0/1190688942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc750159ec0 0x7fc750068c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:18.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.215+0000 7fc754f5a700 1 --2- 172.21.15.89:0/1190688942 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc7500b53d0 0x7fc750169a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:18.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.215+0000 7fc754f5a700 1 --2- 172.21.15.89:0/1190688942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7500cf350 0x7fc750169fd0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:18.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.215+0000 7fc754f5a700 1 -- 172.21.15.89:0/1190688942 >> 172.21.15.89:0/1190688942 conn(0x7fc7500999f0 msgr2=0x7fc7501590d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.215+0000 7fc754f5a700 1 -- 172.21.15.89:0/1190688942 shutdown_connections 2023-10-10T20:46:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.215+0000 7fc754f5a700 1 -- 172.21.15.89:0/1190688942 wait complete. 2023-10-10T20:46:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' removed 2023-10-10T20:46:18.229 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:46:18.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.344+0000 7f9081624700 1 Processor -- start 2023-10-10T20:46:18.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.344+0000 7f9081624700 1 -- start start 2023-10-10T20:46:18.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.345+0000 7f9081624700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f907c159890 0x7f907c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:18.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.345+0000 7f9081624700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f907c159c60 0x7f907c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:18.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.345+0000 7f9081624700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f907c15a160 con 0x7f907c15db10 2023-10-10T20:46:18.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.345+0000 7f9081624700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f907c0d13a0 con 0x7f907c159c60 2023-10-10T20:46:18.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.345+0000 7f9081624700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907c15e140 con 0x7f907c159890 2023-10-10T20:46:18.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.345+0000 7f9073fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f907c159c60 0x7f907c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:18.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.345+0000 7f9073fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f907c159c60 0x7f907c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43432/0 (socket says 172.21.15.89:43432) 2023-10-10T20:46:18.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.345+0000 7f9073fff700 1 -- 172.21.15.89:0/3327234064 learned_addr learned my addr 172.21.15.89:0/3327234064 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:18.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.345+0000 7f907b7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f907c15db10 0x7f907c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51662/0 (socket says 172.21.15.89:51662) 2023-10-10T20:46:18.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.345+0000 7f907affd700 1 --2- 172.21.15.89:0/3327234064 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f907c159890 0x7f907c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:18.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.345+0000 7f9073fff700 1 -- 172.21.15.89:0/3327234064 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f907c159890 msgr2=0x7f907c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:18.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.345+0000 7f9073fff700 1 --2- 172.21.15.89:0/3327234064 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f907c159890 0x7f907c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:18.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.345+0000 7f9073fff700 1 -- 172.21.15.89:0/3327234064 >> v1:172.21.15.89:6789/0 conn(0x7f907c15db10 legacy=0x7f907c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.346+0000 7f9073fff700 1 -- 172.21.15.89:0/3327234064 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907c15a3d0 con 0x7f907c159c60 2023-10-10T20:46:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.346+0000 7f907a7fc700 1 -- 172.21.15.89:0/3327234064 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123614730 0 0) 0x7f907c15a160 con 0x7f907c15db10 2023-10-10T20:46:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.346+0000 7f9073fff700 1 --2- 172.21.15.89:0/3327234064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907c159c60 0x7f907c15d5d0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f90640099b0 tx=0x7f9064009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2223b09e21b28dfd server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.346+0000 7f907a7fc700 1 -- 172.21.15.89:0/3327234064 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906400b860 con 0x7f907c159c60 2023-10-10T20:46:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.346+0000 7f907a7fc700 1 -- 172.21.15.89:0/3327234064 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906400b9c0 con 0x7f907c159c60 2023-10-10T20:46:18.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.346+0000 7f907a7fc700 1 -- 172.21.15.89:0/3327234064 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906400bd60 con 0x7f907c159c60 2023-10-10T20:46:18.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.346+0000 7f9081624700 1 -- 172.21.15.89:0/3327234064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907c159c60 msgr2=0x7f907c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:18.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.346+0000 7f9081624700 1 --2- 172.21.15.89:0/3327234064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907c159c60 0x7f907c15d5d0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f90640099b0 tx=0x7f9064009250 comp rx=0 tx=0).stop 2023-10-10T20:46:18.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.346+0000 7f9081624700 1 -- 172.21.15.89:0/3327234064 shutdown_connections 2023-10-10T20:46:18.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.346+0000 7f9081624700 1 --2- 172.21.15.89:0/3327234064 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f907c159890 0x7f907c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:18.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.346+0000 7f9081624700 1 --2- 172.21.15.89:0/3327234064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907c159c60 0x7f907c15d5d0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:18.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.346+0000 7f9081624700 1 -- 172.21.15.89:0/3327234064 >> 172.21.15.89:0/3327234064 conn(0x7f907c068e20 msgr2=0x7f907c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:18.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.346+0000 7f9081624700 1 -- 172.21.15.89:0/3327234064 shutdown_connections 2023-10-10T20:46:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.346+0000 7f9081624700 1 -- 172.21.15.89:0/3327234064 wait complete. 2023-10-10T20:46:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.347+0000 7f9081624700 1 Processor -- start 2023-10-10T20:46:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.347+0000 7f9081624700 1 -- start start 2023-10-10T20:46:18.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.348+0000 7f9081624700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907c159890 0x7f907c0cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:18.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.348+0000 7f9081624700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f907c159c60 0x7f907c0cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:18.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.348+0000 7f9081624700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f907c15db10 0x7f907c14c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:18.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.348+0000 7f9081624700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907c16df60 con 0x7f907c159890 2023-10-10T20:46:18.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.348+0000 7f9081624700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907c0ce270 con 0x7f907c15db10 2023-10-10T20:46:18.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.348+0000 7f9081624700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907c16d390 con 0x7f907c159c60 2023-10-10T20:46:18.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.348+0000 7f907affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907c159890 0x7f907c0cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:18.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.348+0000 7f907affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907c159890 0x7f907c0cf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43434/0 (socket says 172.21.15.89:43434) 2023-10-10T20:46:18.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.348+0000 7f907affd700 1 -- 172.21.15.89:0/4036723676 learned_addr learned my addr 172.21.15.89:0/4036723676 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:18.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.348+0000 7f907affd700 1 -- 172.21.15.89:0/4036723676 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f907c159c60 msgr2=0x7f907c0cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:18.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.348+0000 7f907affd700 1 --2- 172.21.15.89:0/4036723676 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f907c159c60 0x7f907c0cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:18.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.348+0000 7f907affd700 1 -- 172.21.15.89:0/4036723676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f907c15db10 msgr2=0x7f907c14c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:18.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.348+0000 7f907affd700 1 --2- 172.21.15.89:0/4036723676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f907c15db10 0x7f907c14c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:18.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.348+0000 7f907affd700 1 -- 172.21.15.89:0/4036723676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90640094f0 con 0x7f907c159890 2023-10-10T20:46:18.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.349+0000 7f907affd700 1 --2- 172.21.15.89:0/4036723676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907c159890 0x7f907c0cf6d0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f906c00d890 tx=0x7f906c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.349+0000 7f9072ffd700 1 -- 172.21.15.89:0/4036723676 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c00b600 con 0x7f907c159890 2023-10-10T20:46:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.349+0000 7f9072ffd700 1 -- 172.21.15.89:0/4036723676 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906c00b760 con 0x7f907c159890 2023-10-10T20:46:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.349+0000 7f9081624700 1 -- 172.21.15.89:0/4036723676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f907c152510 con 0x7f907c159890 2023-10-10T20:46:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.349+0000 7f9072ffd700 1 -- 172.21.15.89:0/4036723676 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c00bae0 con 0x7f907c159890 2023-10-10T20:46:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.349+0000 7f9081624700 1 -- 172.21.15.89:0/4036723676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907c16d910 con 0x7f907c159890 2023-10-10T20:46:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.350+0000 7f9081624700 1 -- 172.21.15.89:0/4036723676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f907c04c3c0 con 0x7f907c159890 2023-10-10T20:46:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.350+0000 7f9072ffd700 1 -- 172.21.15.89:0/4036723676 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f906c01e460 con 0x7f907c159890 2023-10-10T20:46:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.351+0000 7f9072ffd700 1 --2- 172.21.15.89:0/4036723676 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f905c067320 0x7f905c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.351+0000 7f9072ffd700 1 -- 172.21.15.89:0/4036723676 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f906c085020 con 0x7f907c159890 2023-10-10T20:46:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.351+0000 7f9073fff700 1 --2- 172.21.15.89:0/4036723676 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f905c067320 0x7f905c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.351+0000 7f9073fff700 1 --2- 172.21.15.89:0/4036723676 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f905c067320 0x7f905c0697d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f90640099b0 tx=0x7f9064017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.355+0000 7f9072ffd700 1 -- 172.21.15.89:0/4036723676 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f906c051fb0 con 0x7f907c159890 2023-10-10T20:46:18.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.539+0000 7f9081624700 1 -- 172.21.15.89:0/4036723676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f907c06a5b0 con 0x7f907c159890 2023-10-10T20:46:18.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.543+0000 7f9072ffd700 1 -- 172.21.15.89:0/4036723676 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7f906c0555d0 con 0x7f907c159890 2023-10-10T20:46:18.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.545+0000 7f9081624700 1 -- 172.21.15.89:0/4036723676 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f905c067320 msgr2=0x7f905c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:18.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.545+0000 7f9081624700 1 --2- 172.21.15.89:0/4036723676 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f905c067320 0x7f905c0697d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f90640099b0 tx=0x7f9064017000 comp rx=0 tx=0).stop 2023-10-10T20:46:18.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.545+0000 7f9081624700 1 -- 172.21.15.89:0/4036723676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907c159890 msgr2=0x7f907c0cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:18.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.545+0000 7f9081624700 1 --2- 172.21.15.89:0/4036723676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907c159890 0x7f907c0cf6d0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f906c00d890 tx=0x7f906c008040 comp rx=0 tx=0).stop 2023-10-10T20:46:18.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.545+0000 7f9081624700 1 -- 172.21.15.89:0/4036723676 shutdown_connections 2023-10-10T20:46:18.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.545+0000 7f9081624700 1 --2- 172.21.15.89:0/4036723676 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f905c067320 0x7f905c0697d0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:18.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.545+0000 7f9081624700 1 --2- 172.21.15.89:0/4036723676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f907c15db10 0x7f907c14c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:18.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.545+0000 7f9081624700 1 --2- 172.21.15.89:0/4036723676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907c159890 0x7f907c0cf6d0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:18.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.545+0000 7f9081624700 1 --2- 172.21.15.89:0/4036723676 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f907c159c60 0x7f907c0cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:18.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.545+0000 7f9081624700 1 -- 172.21.15.89:0/4036723676 >> 172.21.15.89:0/4036723676 conn(0x7f907c068e20 msgr2=0x7f907c158a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:18.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.545+0000 7f9081624700 1 -- 172.21.15.89:0/4036723676 shutdown_connections 2023-10-10T20:46:18.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.545+0000 7f9081624700 1 -- 172.21.15.89:0/4036723676 wait complete. 2023-10-10T20:46:18.559 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm' 2023-10-10T20:46:18.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.672+0000 7f6f8f17c700 1 Processor -- start 2023-10-10T20:46:18.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.672+0000 7f6f8f17c700 1 -- start start 2023-10-10T20:46:18.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.673+0000 7f6f8f17c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6f88068550 0x7f6f88068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:18.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.673+0000 7f6f8f17c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6f8815e010 0x7f6f8806f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:18.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.673+0000 7f6f8f17c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f8806ec70 con 0x7f6f8806f640 2023-10-10T20:46:18.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.673+0000 7f6f8f17c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f880b3430 con 0x7f6f88068550 2023-10-10T20:46:18.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.673+0000 7f6f8f17c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f880b2e70 con 0x7f6f8815e010 2023-10-10T20:46:18.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.673+0000 7f6f8cf18700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6f88068550 0x7f6f88068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:18.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.673+0000 7f6f8cf18700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6f88068550 0x7f6f88068920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43442/0 (socket says 172.21.15.89:43442) 2023-10-10T20:46:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.673+0000 7f6f8cf18700 1 -- 172.21.15.89:0/3778075278 learned_addr learned my addr 172.21.15.89:0/3778075278 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.674+0000 7f6f8cf18700 1 -- 172.21.15.89:0/3778075278 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6f8815e010 msgr2=0x7f6f8806f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.674+0000 7f6f8cf18700 1 --2- 172.21.15.89:0/3778075278 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6f8815e010 0x7f6f8806f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.674+0000 7f6f8cf18700 1 -- 172.21.15.89:0/3778075278 >> v1:172.21.15.111:6789/0 conn(0x7f6f8806f640 legacy=0x7f6f8806a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.674+0000 7f6f8cf18700 1 -- 172.21.15.89:0/3778075278 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f880b3150 con 0x7f6f88068550 2023-10-10T20:46:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.674+0000 7f6f8cf18700 1 --2- 172.21.15.89:0/3778075278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f88068550 0x7f6f88068920 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0099b0 tx=0x7f6f7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6770c8649d270ce6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.674+0000 7f6f877fe700 1 -- 172.21.15.89:0/3778075278 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c00b820 con 0x7f6f88068550 2023-10-10T20:46:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.674+0000 7f6f877fe700 1 -- 172.21.15.89:0/3778075278 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7c00b980 con 0x7f6f88068550 2023-10-10T20:46:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.674+0000 7f6f877fe700 1 -- 172.21.15.89:0/3778075278 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c00bd20 con 0x7f6f88068550 2023-10-10T20:46:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.675+0000 7f6f8f17c700 1 -- 172.21.15.89:0/3778075278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f88068550 msgr2=0x7f6f88068920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.675+0000 7f6f8f17c700 1 --2- 172.21.15.89:0/3778075278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f88068550 0x7f6f88068920 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0099b0 tx=0x7f6f7c009250 comp rx=0 tx=0).stop 2023-10-10T20:46:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.675+0000 7f6f8f17c700 1 -- 172.21.15.89:0/3778075278 shutdown_connections 2023-10-10T20:46:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.675+0000 7f6f8f17c700 1 --2- 172.21.15.89:0/3778075278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f88068550 0x7f6f88068920 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.675+0000 7f6f8f17c700 1 --2- 172.21.15.89:0/3778075278 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6f8815e010 0x7f6f8806f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.675+0000 7f6f8f17c700 1 -- 172.21.15.89:0/3778075278 >> 172.21.15.89:0/3778075278 conn(0x7f6f880cd480 msgr2=0x7f6f880cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.675+0000 7f6f8f17c700 1 -- 172.21.15.89:0/3778075278 shutdown_connections 2023-10-10T20:46:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.675+0000 7f6f8f17c700 1 -- 172.21.15.89:0/3778075278 wait complete. 2023-10-10T20:46:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.676+0000 7f6f8f17c700 1 Processor -- start 2023-10-10T20:46:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.676+0000 7f6f8f17c700 1 -- start start 2023-10-10T20:46:18.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f8f17c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f88068550 0x7f6f8816aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:18.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f8f17c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6f8806f640 0x7f6f880ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:18.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f8f17c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f8815e010 0x7f6f8815c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:18.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f8f17c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8816afe0 con 0x7f6f88068550 2023-10-10T20:46:18.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f8f17c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f880cd290 con 0x7f6f8815e010 2023-10-10T20:46:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f8f17c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8815a700 con 0x7f6f8806f640 2023-10-10T20:46:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f8cf18700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f88068550 0x7f6f8816aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f8cf18700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f88068550 0x7f6f8816aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43444/0 (socket says 172.21.15.89:43444) 2023-10-10T20:46:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f8cf18700 1 -- 172.21.15.89:0/856032489 learned_addr learned my addr 172.21.15.89:0/856032489 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f8cf18700 1 -- 172.21.15.89:0/856032489 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6f8806f640 msgr2=0x7f6f880ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f8d719700 1 --2- 172.21.15.89:0/856032489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f8815e010 0x7f6f8815c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f87fff700 1 --2- 172.21.15.89:0/856032489 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6f8806f640 0x7f6f880ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f8cf18700 1 --2- 172.21.15.89:0/856032489 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6f8806f640 0x7f6f880ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f8cf18700 1 -- 172.21.15.89:0/856032489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f8815e010 msgr2=0x7f6f8815c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:18.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f8cf18700 1 --2- 172.21.15.89:0/856032489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f8815e010 0x7f6f8815c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:18.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f8cf18700 1 -- 172.21.15.89:0/856032489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c0094f0 con 0x7f6f88068550 2023-10-10T20:46:18.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f8cf18700 1 --2- 172.21.15.89:0/856032489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f88068550 0x7f6f8816aaa0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009290 tx=0x7f6f7c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:18.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f84ff9700 1 -- 172.21.15.89:0/856032489 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c026480 con 0x7f6f88068550 2023-10-10T20:46:18.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f84ff9700 1 -- 172.21.15.89:0/856032489 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7c01f070 con 0x7f6f88068550 2023-10-10T20:46:18.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f8f17c700 1 -- 172.21.15.89:0/856032489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8815a9e0 con 0x7f6f88068550 2023-10-10T20:46:18.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f84ff9700 1 -- 172.21.15.89:0/856032489 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c026850 con 0x7f6f88068550 2023-10-10T20:46:18.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.677+0000 7f6f8f17c700 1 -- 172.21.15.89:0/856032489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8815afa0 con 0x7f6f88068550 2023-10-10T20:46:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.678+0000 7f6f8f17c700 1 -- 172.21.15.89:0/856032489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f8804c3c0 con 0x7f6f88068550 2023-10-10T20:46:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.679+0000 7f6f84ff9700 1 -- 172.21.15.89:0/856032489 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6f7c01a440 con 0x7f6f88068550 2023-10-10T20:46:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.679+0000 7f6f84ff9700 1 --2- 172.21.15.89:0/856032489 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6f6c067320 0x7f6f6c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.679+0000 7f6f84ff9700 1 -- 172.21.15.89:0/856032489 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6f7c021070 con 0x7f6f88068550 2023-10-10T20:46:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.679+0000 7f6f87fff700 1 --2- 172.21.15.89:0/856032489 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6f6c067320 0x7f6f6c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.680+0000 7f6f87fff700 1 --2- 172.21.15.89:0/856032489 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6f6c067320 0x7f6f6c0697d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6f74006fd0 tx=0x7f6f74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.683+0000 7f6f84ff9700 1 -- 172.21.15.89:0/856032489 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f7c053ef0 con 0x7f6f88068550 2023-10-10T20:46:18.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:18.948+0000 7f6f8f17c700 1 -- 172.21.15.89:0/856032489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm"]} v 0) v1 -- 0x7f6f8806e900 con 0x7f6f88068550 2023-10-10T20:46:19.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:19.213+0000 7f6f84ff9700 1 -- 172.21.15.89:0/856032489 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm"]}]=0 v564) v1 ==== 131+0+0 (secure 0 0 0) 0x7f6f7c057510 con 0x7f6f88068550 2023-10-10T20:46:19.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:19.215+0000 7f6f8f17c700 1 -- 172.21.15.89:0/856032489 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6f6c067320 msgr2=0x7f6f6c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:19.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:19.215+0000 7f6f8f17c700 1 --2- 172.21.15.89:0/856032489 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6f6c067320 0x7f6f6c0697d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6f74006fd0 tx=0x7f6f74008040 comp rx=0 tx=0).stop 2023-10-10T20:46:19.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:19.215+0000 7f6f8f17c700 1 -- 172.21.15.89:0/856032489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f88068550 msgr2=0x7f6f8816aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:19.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:19.215+0000 7f6f8f17c700 1 --2- 172.21.15.89:0/856032489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f88068550 0x7f6f8816aaa0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009290 tx=0x7f6f7c016000 comp rx=0 tx=0).stop 2023-10-10T20:46:19.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:19.216+0000 7f6f8f17c700 1 -- 172.21.15.89:0/856032489 shutdown_connections 2023-10-10T20:46:19.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:19.216+0000 7f6f8f17c700 1 --2- 172.21.15.89:0/856032489 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f6f6c067320 0x7f6f6c0697d0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:19.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:19.216+0000 7f6f8f17c700 1 --2- 172.21.15.89:0/856032489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f8815e010 0x7f6f8815c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:19.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:19.216+0000 7f6f8f17c700 1 --2- 172.21.15.89:0/856032489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f88068550 0x7f6f8816aaa0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:19.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:19.216+0000 7f6f8f17c700 1 --2- 172.21.15.89:0/856032489 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f6f8806f640 0x7f6f880ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:19.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:19.216+0000 7f6f8f17c700 1 -- 172.21.15.89:0/856032489 >> 172.21.15.89:0/856032489 conn(0x7f6f880cd480 msgr2=0x7f6f88156f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:19.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:19.216+0000 7f6f8f17c700 1 -- 172.21.15.89:0/856032489 shutdown_connections 2023-10-10T20:46:19.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:19.216+0000 7f6f8f17c700 1 -- 172.21.15.89:0/856032489 wait complete. 2023-10-10T20:46:19.230 INFO:tasks.cephfs_test_runner:That the volume can only be removed when --yes-i-really-mean-it is used ... ERROR 2023-10-10T20:46:19.231 INFO:tasks.cephfs_test_runner: 2023-10-10T20:46:19.231 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-10T20:46:19.231 INFO:tasks.cephfs_test_runner:ERROR: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-10T20:46:19.232 INFO:tasks.cephfs_test_runner:That the volume can only be removed when --yes-i-really-mean-it is used 2023-10-10T20:46:19.232 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-10T20:46:19.232 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-10T20:46:19.232 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks/cephfs/test_volumes.py", line 497, in test_volume_rm 2023-10-10T20:46:19.233 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", self.volname) 2023-10-10T20:46:19.233 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2023-10-10T20:46:19.233 INFO:tasks.cephfs_test_runner: return self.mgr_cluster.mon_manager.raw_cluster_cmd("fs", *args) 2023-10-10T20:46:19.233 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd 2023-10-10T20:46:19.234 INFO:tasks.cephfs_test_runner: return self.run_cluster_cmd(**kwargs).stdout.getvalue() 2023-10-10T20:46:19.234 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd 2023-10-10T20:46:19.234 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-10T20:46:19.234 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-10-10T20:46:19.235 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-10T20:46:19.235 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-10-10T20:46:19.235 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-10T20:46:19.235 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-10T20:46:19.236 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-10T20:46:19.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-10T20:46:19.236 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-10T20:46:19.236 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi089 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs' 2023-10-10T20:46:19.237 INFO:tasks.cephfs_test_runner: 2023-10-10T20:46:19.237 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2023-10-10T20:46:19.237 INFO:tasks.cephfs_test_runner: 2023-10-10T20:46:19.237 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-10T20:46:19.238 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks/cephfs/test_volumes.py", line 503, in test_volume_rm 2023-10-10T20:46:19.238 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", self.volname, "--yes-i-really-mean-it") 2023-10-10T20:46:19.238 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2023-10-10T20:46:19.239 INFO:tasks.cephfs_test_runner: return self.mgr_cluster.mon_manager.raw_cluster_cmd("fs", *args) 2023-10-10T20:46:19.239 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd 2023-10-10T20:46:19.239 INFO:tasks.cephfs_test_runner: return self.run_cluster_cmd(**kwargs).stdout.getvalue() 2023-10-10T20:46:19.239 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd 2023-10-10T20:46:19.240 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-10T20:46:19.240 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-10-10T20:46:19.240 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-10T20:46:19.240 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-10-10T20:46:19.241 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-10T20:46:19.241 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-10T20:46:19.241 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-10T20:46:19.241 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-10T20:46:19.242 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-10T20:46:19.242 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi089 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it' 2023-10-10T20:46:19.242 INFO:tasks.cephfs_test_runner: 2023-10-10T20:46:19.242 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-10T20:46:19.243 INFO:tasks.cephfs_test_runner:Ran 12 tests in 677.375s 2023-10-10T20:46:19.243 INFO:tasks.cephfs_test_runner: 2023-10-10T20:46:19.243 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-10-10T20:46:19.243 INFO:tasks.cephfs_test_runner: 2023-10-10T20:46:19.244 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-10T20:46:19.244 INFO:tasks.cephfs_test_runner:ERROR: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-10T20:46:19.244 INFO:tasks.cephfs_test_runner:That the volume can only be removed when --yes-i-really-mean-it is used 2023-10-10T20:46:19.244 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-10T20:46:19.245 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-10T20:46:19.245 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks/cephfs/test_volumes.py", line 497, in test_volume_rm 2023-10-10T20:46:19.245 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", self.volname) 2023-10-10T20:46:19.245 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2023-10-10T20:46:19.246 INFO:tasks.cephfs_test_runner: return self.mgr_cluster.mon_manager.raw_cluster_cmd("fs", *args) 2023-10-10T20:46:19.246 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd 2023-10-10T20:46:19.246 INFO:tasks.cephfs_test_runner: return self.run_cluster_cmd(**kwargs).stdout.getvalue() 2023-10-10T20:46:19.246 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd 2023-10-10T20:46:19.247 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-10T20:46:19.247 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-10-10T20:46:19.247 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-10T20:46:19.247 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-10-10T20:46:19.248 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-10T20:46:19.248 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-10T20:46:19.248 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-10T20:46:19.248 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-10T20:46:19.249 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-10T20:46:19.249 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi089 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs' 2023-10-10T20:46:19.249 INFO:tasks.cephfs_test_runner: 2023-10-10T20:46:19.249 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2023-10-10T20:46:19.250 INFO:tasks.cephfs_test_runner: 2023-10-10T20:46:19.250 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-10T20:46:19.250 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks/cephfs/test_volumes.py", line 503, in test_volume_rm 2023-10-10T20:46:19.250 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", self.volname, "--yes-i-really-mean-it") 2023-10-10T20:46:19.251 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2023-10-10T20:46:19.251 INFO:tasks.cephfs_test_runner: return self.mgr_cluster.mon_manager.raw_cluster_cmd("fs", *args) 2023-10-10T20:46:19.251 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd 2023-10-10T20:46:19.251 INFO:tasks.cephfs_test_runner: return self.run_cluster_cmd(**kwargs).stdout.getvalue() 2023-10-10T20:46:19.252 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd 2023-10-10T20:46:19.252 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-10T20:46:19.252 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-10-10T20:46:19.252 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-10T20:46:19.253 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-10-10T20:46:19.253 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-10T20:46:19.253 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-10T20:46:19.253 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-10T20:46:19.253 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-10T20:46:19.254 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-10T20:46:19.254 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi089 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it' 2023-10-10T20:46:19.254 INFO:tasks.cephfs_test_runner: 2023-10-10T20:46:19.255 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-10T20:46:19.333 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=b1d56a04f2df451ab790f37b0f0ecb5b Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-10T20:46:19.336 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-10T20:46:19.348 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-10T20:46:19.366 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-10T20:46:19.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:46:19.367 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-10T20:46:19.385 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:46:19.385 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:46:19.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:46:19.386 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:46:19.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:46:19.460 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:46:19.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:46:19.538 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link delete ceph-brx 2023-10-10T20:46:19.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:46:19.636 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-10T20:46:19.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:46:19.652 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link delete ceph-brx 2023-10-10T20:46:19.719 INFO:teuthology.orchestra.run.smithi089.stderr:Cannot find device "ceph-brx" 2023-10-10T20:46:19.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:46:19.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:46:19.721 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-10-10T20:46:19.739 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:46:19.740 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:46:19.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:46:19.740 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:46:19.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:46:19.817 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:46:19.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:46:19.901 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link delete ceph-brx 2023-10-10T20:46:20.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:46:20.003 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-10-10T20:46:20.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:46:20.018 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link delete ceph-brx 2023-10-10T20:46:20.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:46:20.090 INFO:teuthology.orchestra.run.smithi111.stderr:Cannot find device "ceph-brx" 2023-10-10T20:46:20.091 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-10T20:46:20.100 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-10T20:46:20.101 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:46:20.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.215+0000 7facdf195700 1 Processor -- start 2023-10-10T20:46:20.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.215+0000 7facdf195700 1 -- start start 2023-10-10T20:46:20.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.216+0000 7facdf195700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7facd80cbf30 0x7facd80ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:20.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.216+0000 7facdf195700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7facd80ccf90 0x7facd80b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:20.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.216+0000 7facdf195700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facd80cc300 con 0x7facd815a270 2023-10-10T20:46:20.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.216+0000 7facdf195700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facd8156bf0 con 0x7facd80ccf90 2023-10-10T20:46:20.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.216+0000 7facdf195700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facd80ae950 con 0x7facd80cbf30 2023-10-10T20:46:20.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.216+0000 7facd7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7facd80ccf90 0x7facd80b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:20.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.216+0000 7facdd732700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7facd815a270 0x7facd80c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51666/0 (socket says 172.21.15.89:51666) 2023-10-10T20:46:20.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.216+0000 7facd7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7facd80ccf90 0x7facd80b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43458/0 (socket says 172.21.15.89:43458) 2023-10-10T20:46:20.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.216+0000 7facdd732700 1 -- 172.21.15.89:0/2053827579 learned_addr learned my addr 172.21.15.89:0/2053827579 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:20.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.216+0000 7facdcf31700 1 --2- 172.21.15.89:0/2053827579 >> v2:172.21.15.111:3300/0 conn(0x7facd80cbf30 0x7facd80ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:20.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.217+0000 7facd7fff700 1 -- 172.21.15.89:0/2053827579 >> v2:172.21.15.111:3300/0 conn(0x7facd80cbf30 msgr2=0x7facd80ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:20.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.217+0000 7facd7fff700 1 --2- 172.21.15.89:0/2053827579 >> v2:172.21.15.111:3300/0 conn(0x7facd80cbf30 0x7facd80ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.217+0000 7facd7fff700 1 -- 172.21.15.89:0/2053827579 >> v1:172.21.15.89:6789/0 conn(0x7facd815a270 legacy=0x7facd80c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:20.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.217+0000 7facd7fff700 1 -- 172.21.15.89:0/2053827579 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd80ad370 con 0x7facd80ccf90 2023-10-10T20:46:20.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.217+0000 7facd77fe700 1 -- 172.21.15.89:0/2053827579 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3841771280 0 0) 0x7facd80cc300 con 0x7facd815a270 2023-10-10T20:46:20.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.217+0000 7facd7fff700 1 --2- 172.21.15.89:0/2053827579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facd80ccf90 0x7facd80b0d10 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7facd8063d40 tx=0x7facc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7ee108d3d80ae1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:20.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.217+0000 7facd77fe700 1 -- 172.21.15.89:0/2053827579 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc4011630 con 0x7facd80ccf90 2023-10-10T20:46:20.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.217+0000 7facd77fe700 1 -- 172.21.15.89:0/2053827579 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc4011790 con 0x7facd80ccf90 2023-10-10T20:46:20.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.217+0000 7facd77fe700 1 -- 172.21.15.89:0/2053827579 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc4011b30 con 0x7facd80ccf90 2023-10-10T20:46:20.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.217+0000 7facdf195700 1 -- 172.21.15.89:0/2053827579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facd80ccf90 msgr2=0x7facd80b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:20.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.217+0000 7facdf195700 1 --2- 172.21.15.89:0/2053827579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facd80ccf90 0x7facd80b0d10 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7facd8063d40 tx=0x7facc40091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.218+0000 7facdf195700 1 -- 172.21.15.89:0/2053827579 shutdown_connections 2023-10-10T20:46:20.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.218+0000 7facdf195700 1 --2- 172.21.15.89:0/2053827579 >> v2:172.21.15.111:3300/0 conn(0x7facd80cbf30 0x7facd80ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.218+0000 7facdf195700 1 --2- 172.21.15.89:0/2053827579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facd80ccf90 0x7facd80b0d10 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.218+0000 7facdf195700 1 -- 172.21.15.89:0/2053827579 >> 172.21.15.89:0/2053827579 conn(0x7facd806a320 msgr2=0x7facd80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:20.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.218+0000 7facdf195700 1 -- 172.21.15.89:0/2053827579 shutdown_connections 2023-10-10T20:46:20.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.218+0000 7facdf195700 1 -- 172.21.15.89:0/2053827579 wait complete. 2023-10-10T20:46:20.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.219+0000 7facdf195700 1 Processor -- start 2023-10-10T20:46:20.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.219+0000 7facdf195700 1 -- start start 2023-10-10T20:46:20.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facdf195700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7facd80cbf30 0x7facd80c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:20.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facdf195700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facd80ccf90 0x7facd8166d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:20.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facdf195700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facd815a270 0x7facd80a6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:20.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facdf195700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd8154e20 con 0x7facd815a270 2023-10-10T20:46:20.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facdf195700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd80a6270 con 0x7facd80ccf90 2023-10-10T20:46:20.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facdf195700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd80a5660 con 0x7facd80cbf30 2023-10-10T20:46:20.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facdd732700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facd815a270 0x7facd80a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:20.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facdd732700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facd815a270 0x7facd80a6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43472/0 (socket says 172.21.15.89:43472) 2023-10-10T20:46:20.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facdd732700 1 -- 172.21.15.89:0/3882080789 learned_addr learned my addr 172.21.15.89:0/3882080789 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:20.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facd7fff700 1 --2- 172.21.15.89:0/3882080789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facd80ccf90 0x7facd8166d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:20.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facdcf31700 1 --2- 172.21.15.89:0/3882080789 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7facd80cbf30 0x7facd80c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:20.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facdd732700 1 -- 172.21.15.89:0/3882080789 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7facd80cbf30 msgr2=0x7facd80c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:20.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facdd732700 1 --2- 172.21.15.89:0/3882080789 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7facd80cbf30 0x7facd80c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facdd732700 1 -- 172.21.15.89:0/3882080789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facd80ccf90 msgr2=0x7facd8166d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:20.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facdd732700 1 --2- 172.21.15.89:0/3882080789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facd80ccf90 0x7facd8166d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facdd732700 1 -- 172.21.15.89:0/3882080789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc4009420 con 0x7facd815a270 2023-10-10T20:46:20.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facdd732700 1 --2- 172.21.15.89:0/3882080789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facd815a270 0x7facd80a6700 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7facd000ccc0 tx=0x7facd0001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:20.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facd4ff9700 1 -- 172.21.15.89:0/3882080789 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd000a8a0 con 0x7facd815a270 2023-10-10T20:46:20.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facd4ff9700 1 -- 172.21.15.89:0/3882080789 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd000aa00 con 0x7facd815a270 2023-10-10T20:46:20.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facdf195700 1 -- 172.21.15.89:0/3882080789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd80a5940 con 0x7facd815a270 2023-10-10T20:46:20.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facd4ff9700 1 -- 172.21.15.89:0/3882080789 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd000adc0 con 0x7facd815a270 2023-10-10T20:46:20.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.220+0000 7facdf195700 1 -- 172.21.15.89:0/3882080789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd80a5f00 con 0x7facd815a270 2023-10-10T20:46:20.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.221+0000 7facdf195700 1 -- 172.21.15.89:0/3882080789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd80a6400 con 0x7facd815a270 2023-10-10T20:46:20.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.222+0000 7facd4ff9700 1 -- 172.21.15.89:0/3882080789 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7facd0007300 con 0x7facd815a270 2023-10-10T20:46:20.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.222+0000 7facd4ff9700 1 --2- 172.21.15.89:0/3882080789 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7facbc067320 0x7facbc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:20.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.222+0000 7facd4ff9700 1 -- 172.21.15.89:0/3882080789 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7facd0084cd0 con 0x7facd815a270 2023-10-10T20:46:20.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.222+0000 7facdcf31700 1 --2- 172.21.15.89:0/3882080789 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7facbc067320 0x7facbc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:20.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.223+0000 7facdcf31700 1 --2- 172.21.15.89:0/3882080789 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7facbc067320 0x7facbc0697d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7faccc009d10 tx=0x7faccc0091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:20.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.226+0000 7facd4ff9700 1 -- 172.21.15.89:0/3882080789 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7facd0051ca0 con 0x7facd815a270 2023-10-10T20:46:20.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.411+0000 7facdf195700 1 -- 172.21.15.89:0/3882080789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7facd804c3c0 con 0x7facd815a270 2023-10-10T20:46:20.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.412+0000 7facd4ff9700 1 -- 172.21.15.89:0/3882080789 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v203) v1 ==== 74+0+12273 (secure 0 0 0) 0x7facd00552c0 con 0x7facd815a270 2023-10-10T20:46:20.414 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:46:20.414 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":203,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:46:18.206492+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6836/3558240892":"2023-10-11T20:46:14.857266+0000","172.21.15.89:6834/3558240892":"2023-10-11T20:46:14.857266+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:46:20.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.414+0000 7facdf195700 1 -- 172.21.15.89:0/3882080789 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7facbc067320 msgr2=0x7facbc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:20.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.414+0000 7facdf195700 1 --2- 172.21.15.89:0/3882080789 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7facbc067320 0x7facbc0697d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7faccc009d10 tx=0x7faccc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.414+0000 7facdf195700 1 -- 172.21.15.89:0/3882080789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facd815a270 msgr2=0x7facd80a6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:20.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.414+0000 7facdf195700 1 --2- 172.21.15.89:0/3882080789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facd815a270 0x7facd80a6700 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7facd000ccc0 tx=0x7facd0001a80 comp rx=0 tx=0).stop 2023-10-10T20:46:20.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.414+0000 7facdf195700 1 -- 172.21.15.89:0/3882080789 shutdown_connections 2023-10-10T20:46:20.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.414+0000 7facdf195700 1 --2- 172.21.15.89:0/3882080789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7facd815a270 0x7facd80a6700 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.414+0000 7facdf195700 1 --2- 172.21.15.89:0/3882080789 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7facd80cbf30 0x7facd80c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.414+0000 7facdf195700 1 --2- 172.21.15.89:0/3882080789 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7facbc067320 0x7facbc0697d0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.414+0000 7facdf195700 1 --2- 172.21.15.89:0/3882080789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7facd80ccf90 0x7facd8166d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.414+0000 7facdf195700 1 -- 172.21.15.89:0/3882080789 >> 172.21.15.89:0/3882080789 conn(0x7facd806a320 msgr2=0x7facd80ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:20.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.414+0000 7facdf195700 1 -- 172.21.15.89:0/3882080789 shutdown_connections 2023-10-10T20:46:20.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.414+0000 7facdf195700 1 -- 172.21.15.89:0/3882080789 wait complete. 2023-10-10T20:46:20.428 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-10T20:46:20.429 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:46:20.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.541+0000 7f412d530700 1 Processor -- start 2023-10-10T20:46:20.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.541+0000 7f412d530700 1 -- start start 2023-10-10T20:46:20.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.542+0000 7f412d530700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f41281578b0 0x7f4128155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:20.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.542+0000 7f412d530700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41280ad930 con 0x7f4128155c10 2023-10-10T20:46:20.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.542+0000 7f412d530700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41280acd20 con 0x7f41280af2f0 2023-10-10T20:46:20.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.542+0000 7f412d530700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41280ad000 con 0x7f41281578b0 2023-10-10T20:46:20.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.542+0000 7f41267fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f41281578b0 0x7f4128155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:20.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.542+0000 7f4126ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4128155c10 0x7f41280ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51674/0 (socket says 172.21.15.89:51674) 2023-10-10T20:46:20.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.542+0000 7f4126ffd700 1 -- 172.21.15.89:0/406103683 learned_addr learned my addr 172.21.15.89:0/406103683 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:20.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.543+0000 7f41267fc700 1 -- 172.21.15.89:0/406103683 >> v1:172.21.15.111:6789/0 conn(0x7f41280af2f0 legacy=0x7f41280af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:20.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.543+0000 7f41267fc700 1 -- 172.21.15.89:0/406103683 >> v1:172.21.15.89:6789/0 conn(0x7f4128155c10 legacy=0x7f41280ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:20.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.543+0000 7f41267fc700 1 -- 172.21.15.89:0/406103683 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41280ad2e0 con 0x7f41281578b0 2023-10-10T20:46:20.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.543+0000 7f4125ffb700 1 -- 172.21.15.89:0/406103683 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329590934 0 0) 0x7f41280ad930 con 0x7f4128155c10 2023-10-10T20:46:20.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.543+0000 7f41267fc700 1 --2- 172.21.15.89:0/406103683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41281578b0 0x7f4128155fe0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f411000aa90 tx=0x7f411000a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334fc5af27386984 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:20.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.543+0000 7f4125ffb700 1 -- 172.21.15.89:0/406103683 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411000ca00 con 0x7f41281578b0 2023-10-10T20:46:20.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.543+0000 7f4125ffb700 1 -- 172.21.15.89:0/406103683 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411000cb60 con 0x7f41281578b0 2023-10-10T20:46:20.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.543+0000 7f4125ffb700 1 -- 172.21.15.89:0/406103683 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41100183f0 con 0x7f41281578b0 2023-10-10T20:46:20.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.543+0000 7f412d530700 1 -- 172.21.15.89:0/406103683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41281578b0 msgr2=0x7f4128155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:20.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.543+0000 7f412d530700 1 --2- 172.21.15.89:0/406103683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41281578b0 0x7f4128155fe0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f411000aa90 tx=0x7f411000a330 comp rx=0 tx=0).stop 2023-10-10T20:46:20.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.543+0000 7f412d530700 1 -- 172.21.15.89:0/406103683 shutdown_connections 2023-10-10T20:46:20.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.543+0000 7f412d530700 1 --2- 172.21.15.89:0/406103683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41281578b0 0x7f4128155fe0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.543+0000 7f412d530700 1 -- 172.21.15.89:0/406103683 >> 172.21.15.89:0/406103683 conn(0x7f412806a320 msgr2=0x7f41280cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:20.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.543+0000 7f412d530700 1 -- 172.21.15.89:0/406103683 shutdown_connections 2023-10-10T20:46:20.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.543+0000 7f412d530700 1 -- 172.21.15.89:0/406103683 wait complete. 2023-10-10T20:46:20.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.545+0000 7f412d530700 1 Processor -- start 2023-10-10T20:46:20.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.545+0000 7f412d530700 1 -- start start 2023-10-10T20:46:20.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.545+0000 7f412d530700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f41280af2f0 0x7f412815a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:20.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.545+0000 7f412d530700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4128155c10 0x7f412814ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:20.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.545+0000 7f412d530700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f41281578b0 0x7f4128150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:20.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.545+0000 7f412d530700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412815add0 con 0x7f4128155c10 2023-10-10T20:46:20.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.545+0000 7f412d530700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41281517a0 con 0x7f41280af2f0 2023-10-10T20:46:20.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.545+0000 7f412d530700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4128150b90 con 0x7f41281578b0 2023-10-10T20:46:20.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.545+0000 7f41267fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4128155c10 0x7f412814ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:20.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.545+0000 7f41267fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4128155c10 0x7f412814ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43492/0 (socket says 172.21.15.89:43492) 2023-10-10T20:46:20.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.545+0000 7f41267fc700 1 -- 172.21.15.89:0/1512199370 learned_addr learned my addr 172.21.15.89:0/1512199370 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:20.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.546+0000 7f41267fc700 1 -- 172.21.15.89:0/1512199370 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f41281578b0 msgr2=0x7f4128150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:20.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.546+0000 7f41267fc700 1 --2- 172.21.15.89:0/1512199370 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f41281578b0 0x7f4128150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.546+0000 7f41267fc700 1 -- 172.21.15.89:0/1512199370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f41280af2f0 msgr2=0x7f412815a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:20.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.546+0000 7f4126ffd700 1 --2- 172.21.15.89:0/1512199370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f41280af2f0 0x7f412815a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:20.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.546+0000 7f41267fc700 1 --2- 172.21.15.89:0/1512199370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f41280af2f0 0x7f412815a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.546+0000 7f41267fc700 1 -- 172.21.15.89:0/1512199370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411000a5d0 con 0x7f4128155c10 2023-10-10T20:46:20.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.546+0000 7f41267fc700 1 --2- 172.21.15.89:0/1512199370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4128155c10 0x7f412814ffd0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f411000a5a0 tx=0x7f4110019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:20.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.546+0000 7f411effd700 1 -- 172.21.15.89:0/1512199370 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4110010750 con 0x7f4128155c10 2023-10-10T20:46:20.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.546+0000 7f411effd700 1 -- 172.21.15.89:0/1512199370 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41100108b0 con 0x7f4128155c10 2023-10-10T20:46:20.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.546+0000 7f412d530700 1 -- 172.21.15.89:0/1512199370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4128150e70 con 0x7f4128155c10 2023-10-10T20:46:20.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.546+0000 7f411effd700 1 -- 172.21.15.89:0/1512199370 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4110010c70 con 0x7f4128155c10 2023-10-10T20:46:20.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.546+0000 7f412d530700 1 -- 172.21.15.89:0/1512199370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4128151430 con 0x7f4128155c10 2023-10-10T20:46:20.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.547+0000 7f412d530700 1 -- 172.21.15.89:0/1512199370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f412804c3c0 con 0x7f4128155c10 2023-10-10T20:46:20.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.547+0000 7f411effd700 1 -- 172.21.15.89:0/1512199370 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f411002a3b0 con 0x7f4128155c10 2023-10-10T20:46:20.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.548+0000 7f411effd700 1 --2- 172.21.15.89:0/1512199370 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f41080673a0 0x7f4108069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:20.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.548+0000 7f411effd700 1 -- 172.21.15.89:0/1512199370 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f41100863a0 con 0x7f4128155c10 2023-10-10T20:46:20.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.548+0000 7f4126ffd700 1 --2- 172.21.15.89:0/1512199370 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f41080673a0 0x7f4108069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:20.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.548+0000 7f4126ffd700 1 --2- 172.21.15.89:0/1512199370 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f41080673a0 0x7f4108069850 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f41280aa950 tx=0x7f4118009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:20.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.552+0000 7f411effd700 1 -- 172.21.15.89:0/1512199370 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4110053350 con 0x7f4128155c10 2023-10-10T20:46:20.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.729+0000 7f412d530700 1 -- 172.21.15.89:0/1512199370 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41280ad5c0 con 0x7f41080673a0 2023-10-10T20:46:20.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.731+0000 7f411effd700 1 -- 172.21.15.89:0/1512199370 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51010 (secure 0 0 0) 0x7f41280ad5c0 con 0x7f41080673a0 2023-10-10T20:46:20.733 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:46:20.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.733+0000 7f412d530700 1 -- 172.21.15.89:0/1512199370 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f41080673a0 msgr2=0x7f4108069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:20.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.733+0000 7f412d530700 1 --2- 172.21.15.89:0/1512199370 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f41080673a0 0x7f4108069850 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f41280aa950 tx=0x7f4118009040 comp rx=0 tx=0).stop 2023-10-10T20:46:20.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.733+0000 7f412d530700 1 -- 172.21.15.89:0/1512199370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4128155c10 msgr2=0x7f412814ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:20.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.733+0000 7f412d530700 1 --2- 172.21.15.89:0/1512199370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4128155c10 0x7f412814ffd0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f411000a5a0 tx=0x7f4110019000 comp rx=0 tx=0).stop 2023-10-10T20:46:20.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.733+0000 7f412d530700 1 -- 172.21.15.89:0/1512199370 shutdown_connections 2023-10-10T20:46:20.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.733+0000 7f412d530700 1 --2- 172.21.15.89:0/1512199370 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f41281578b0 0x7f4128150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.733+0000 7f412d530700 1 --2- 172.21.15.89:0/1512199370 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f41080673a0 0x7f4108069850 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.733+0000 7f412d530700 1 --2- 172.21.15.89:0/1512199370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f41280af2f0 0x7f412815a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.733+0000 7f412d530700 1 --2- 172.21.15.89:0/1512199370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4128155c10 0x7f412814ffd0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.733+0000 7f412d530700 1 -- 172.21.15.89:0/1512199370 >> 172.21.15.89:0/1512199370 conn(0x7f412806a320 msgr2=0x7f41280af6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:20.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.733+0000 7f412d530700 1 -- 172.21.15.89:0/1512199370 shutdown_connections 2023-10-10T20:46:20.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.733+0000 7f412d530700 1 -- 172.21.15.89:0/1512199370 wait complete. 2023-10-10T20:46:20.739 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:46:20.747 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":552,"stamp":"2023-10-10T20:46:18.797281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2432296,"kb_used_data":21224,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752542424,"statfs":{"total":773094113280,"available":770603442176,"internally_reserved":0,"allocated":21733376,"data_stored":10421604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4067,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-49,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001083"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":345,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-10T20:46:14.861401+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:46:14.861401+0000","last_peered":"2023-10-10T20:46:14.861401+0000","last_clean":"2023-10-10T20:46:14.861401+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:46:14.861401+0000","last_undegraded":"2023-10-10T20:46:14.861401+0000","last_fullsized":"2023-10-10T20:46:14.861401+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304476,"kb_used_data":3092,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067364,"statfs":{"total":96636764160,"available":96324980736,"internally_reserved":0,"allocated":3166208,"data_stored":1748116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52200000000000002,"15min":0.67000000000000004},"min":{"1min":0.35499999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71499999999999997,"5min":0.748,"15min":6.8630000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52600000000000002,"15min":0.57699999999999996},"min":{"1min":0.33900000000000002,"5min":0.32300000000000001,"15min":0.307},"max":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":4.1959999999999997},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.76100000000000001},"min":{"1min":0.315,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68899999999999995,"5min":0.78400000000000003,"15min":10.1},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54700000000000004,"15min":0.64100000000000001},"min":{"1min":0.39300000000000002,"5min":0.35199999999999998,"15min":0.28999999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":6.806},"last":0.52500000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.68600000000000005},"min":{"1min":0.45900000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.82999999999999996,"5min":0.83099999999999996,"15min":6.9340000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.55300000000000005,"15min":0.64200000000000002},"min":{"1min":0.47499999999999998,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":6.9870000000000001},"last":0.44700000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55200000000000005,"15min":0.65600000000000003},"min":{"1min":0.48399999999999999,"5min":0.371,"15min":0.32800000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":6.6719999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57199999999999995,"15min":0.621},"min":{"1min":0.45000000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.74399999999999999,"5min":0.77700000000000002,"15min":4.1609999999999996},"last":0.56100000000000005}]},{"osd":4,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.45900000000000002,"15min":0.46400000000000002},"min":{"1min":0.29799999999999999,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.68200000000000005,"5min":0.69799999999999995,"15min":0.77200000000000002},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.5,"15min":0.51500000000000001},"min":{"1min":0.374,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71999999999999997,"5min":0.73899999999999999,"15min":0.80300000000000005},"last":0.33400000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50600000000000001,"15min":0.53100000000000003},"min":{"1min":0.35899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70599999999999996,"5min":0.77900000000000003,"15min":1.3180000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51200000000000001,"15min":0.52600000000000002},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.29499999999999998},"max":{"1min":0.77100000000000002,"5min":0.78200000000000003,"15min":0.84099999999999997},"last":0.371}]},{"osd":6,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":1.7509999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.375,"5min":0.35499999999999998,"15min":0.32600000000000001},"max":{"1min":0.81699999999999995,"5min":0.81999999999999995,"15min":1.7090000000000001},"last":0.55500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574995,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303896,"kb_used_data":2512,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067944,"statfs":{"total":96636764160,"available":96325574656,"internally_reserved":0,"allocated":2572288,"data_stored":1157748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53700000000000003,"15min":0.59899999999999998},"min":{"1min":0.40600000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.76800000000000002,"5min":0.83099999999999996,"15min":7.7590000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.53600000000000003,"15min":0.622},"min":{"1min":0.46100000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78700000000000003,"5min":1.075,"15min":7.8090000000000002},"last":0.61099999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54600000000000004,"15min":0.623},"min":{"1min":0.442,"5min":0.29199999999999998,"15min":0.27800000000000002},"max":{"1min":0.73399999999999999,"5min":0.91300000000000003,"15min":7.867},"last":0.625},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.72099999999999997},"min":{"1min":0.438,"5min":0.34499999999999997,"15min":0.33700000000000002},"max":{"1min":0.755,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.68600000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.35699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71899999999999997,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58899999999999997,"15min":0.70999999999999996},"min":{"1min":0.50900000000000001,"5min":0.33400000000000002,"15min":0.318},"max":{"1min":0.76000000000000001,"5min":2.641,"15min":6.7629999999999999},"last":0.69599999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57099999999999995,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72199999999999998,"5min":1.107,"15min":4.8620000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56899999999999995,"15min":0.64000000000000001},"min":{"1min":0.47499999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":1.718,"15min":7.9009999999999998},"last":0.70899999999999996}]},{"osd":4,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.51100000000000001,"15min":0.48999999999999999},"min":{"1min":0.32300000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66000000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.40899999999999997,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":0.745,"5min":1.032,"15min":1.032},"last":0.73599999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.52900000000000003,"15min":0.50700000000000001},"min":{"1min":0.33300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73299999999999998,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.55900000000000005,"15min":0.54400000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33600000000000002},"max":{"1min":0.73299999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56999999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.35799999999999998,"5min":0.33300000000000002,"15min":0.315},"max":{"1min":0.70599999999999996,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.58699999999999997,"15min":0.56399999999999995},"min":{"1min":0.376,"5min":0.36599999999999999,"15min":0.35099999999999998},"max":{"1min":0.78000000000000003,"5min":1.681,"15min":1.681},"last":0.63600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574995,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303888,"kb_used_data":2504,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067952,"statfs":{"total":96636764160,"available":96325582848,"internally_reserved":0,"allocated":2564096,"data_stored":1151794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.33000000000000002,"15min":0.40100000000000002},"min":{"1min":0.23599999999999999,"5min":0.215,"15min":0.17899999999999999},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":5.0380000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.309,"5min":0.223,"15min":0.20899999999999999},"max":{"1min":0.55500000000000005,"5min":0.65500000000000003,"15min":1.194},"last":0.39600000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.26700000000000002,"5min":0.23599999999999999,"15min":0.20599999999999999},"max":{"1min":0.49199999999999999,"5min":0.53800000000000003,"15min":1.2869999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.377,"15min":0.434},"min":{"1min":0.318,"5min":0.23999999999999999,"15min":0.22},"max":{"1min":0.57499999999999996,"5min":0.68600000000000005,"15min":5.0919999999999996},"last":0.496}]},{"osd":3,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.40100000000000002,"15min":0.45800000000000002},"min":{"1min":0.35999999999999999,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":0.54000000000000004,"5min":0.64500000000000002,"15min":5.1159999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39500000000000002},"min":{"1min":0.34899999999999998,"5min":0.23799999999999999,"15min":0.19900000000000001},"max":{"1min":0.55500000000000005,"5min":0.67000000000000004,"15min":1.2170000000000001},"last":0.47499999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.48199999999999998,"5min":0.34200000000000003,"15min":0.30199999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.254},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.57599999999999996,"15min":0.63900000000000001},"min":{"1min":0.45100000000000001,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.83899999999999997,"5min":0.88600000000000001,"15min":4.9850000000000003},"last":0.80000000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.46700000000000003,"5min":0.34799999999999998,"15min":0.34399999999999997},"max":{"1min":0.75900000000000001,"5min":0.91700000000000004,"15min":1.1799999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.58899999999999997,"15min":0.59799999999999998},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":1.3109999999999999},"last":0.81100000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.61099999999999999,"15min":0.72699999999999998},"min":{"1min":0.53200000000000003,"5min":0.378,"15min":0.35199999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":5.0659999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.623,"15min":0.626},"min":{"1min":0.44400000000000001,"5min":0.38600000000000001,"15min":0.36299999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":1.171},"last":0.83999999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.48599999999999999,"5min":0.38200000000000001,"15min":0.36499999999999999},"max":{"1min":0.86699999999999999,"5min":0.94199999999999995,"15min":1.214},"last":0.747},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60699999999999998,"15min":0.66500000000000004},"min":{"1min":0.51600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":4.0499999999999998},"last":0.79000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574993,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304460,"kb_used_data":3076,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067380,"statfs":{"total":96636764160,"available":96324997120,"internally_reserved":0,"allocated":3149824,"data_stored":1741147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.33500000000000002,"15min":0.42699999999999999},"min":{"1min":0.308,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.47199999999999998,"5min":0.63500000000000001,"15min":7.9249999999999998},"last":0.188},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.35499999999999998,"15min":0.42399999999999999},"min":{"1min":0.31900000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55600000000000005,"5min":0.63400000000000001,"15min":6.6299999999999999},"last":0.23999999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.36399999999999999,"15min":0.45200000000000001},"min":{"1min":0.26600000000000001,"5min":0.23100000000000001,"15min":0.216},"max":{"1min":0.63100000000000001,"5min":0.67800000000000005,"15min":8.2509999999999994},"last":0.224},{"interface":"front","average":{"1min":0.437,"5min":0.374,"15min":0.44600000000000001},"min":{"1min":0.29399999999999998,"5min":0.253,"15min":0.222},"max":{"1min":0.55900000000000005,"5min":0.65000000000000002,"15min":6.7370000000000001},"last":0.29699999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38400000000000001,"15min":0.46400000000000002},"min":{"1min":0.35899999999999999,"5min":0.22,"15min":0.21099999999999999},"max":{"1min":0.52600000000000002,"5min":0.53800000000000003,"15min":7.7960000000000003},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.42899999999999999,"15min":0.51400000000000001},"min":{"1min":0.35899999999999999,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.58199999999999996,"5min":0.71699999999999997,"15min":7.8929999999999998},"last":0.30399999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57099999999999995,"15min":0.64000000000000001},"min":{"1min":0.44400000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70499999999999996,"5min":0.94099999999999995,"15min":6.6929999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59099999999999997,"15min":0.67700000000000005},"min":{"1min":0.40000000000000002,"5min":0.371,"15min":0.33900000000000002},"max":{"1min":0.80000000000000004,"5min":0.84399999999999997,"15min":7.6920000000000002},"last":0.50900000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58499999999999996,"15min":0.66200000000000003},"min":{"1min":0.438,"5min":0.38400000000000001,"15min":0.36299999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":7.7690000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.60899999999999999,"15min":0.68899999999999995},"min":{"1min":0.49299999999999999,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.81899999999999995,"5min":0.97599999999999998,"15min":7.8659999999999997},"last":0.441}]},{"osd":6,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.624,"15min":0.71699999999999997},"min":{"1min":0.53700000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.85999999999999999,"5min":0.87,"15min":7.8360000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63800000000000001,"15min":0.69499999999999995},"min":{"1min":0.53000000000000003,"5min":0.41499999999999998,"15min":0.36599999999999999},"max":{"1min":0.83099999999999996,"5min":0.875,"15min":6.6660000000000004},"last":0.48699999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.626,"15min":0.68200000000000005},"min":{"1min":0.42499999999999999,"5min":0.42099999999999999,"15min":0.36299999999999999},"max":{"1min":0.86199999999999999,"5min":0.90600000000000003,"15min":6.7160000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59899999999999998,"15min":0.69099999999999995},"min":{"1min":0.49399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.873,"5min":0.89500000000000002,"15min":7.7370000000000001},"last":0.42099999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574995,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1155543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.33600000000000002,"15min":0.32600000000000001},"min":{"1min":0.26800000000000002,"5min":0.20200000000000001,"15min":0.189},"max":{"1min":0.52700000000000002,"5min":0.59399999999999997,"15min":0.67900000000000005},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.34999999999999998},"min":{"1min":0.25600000000000001,"5min":0.219,"15min":0.21199999999999999},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.746},"last":0.29499999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.54100000000000004},"min":{"1min":0.29099999999999998,"5min":0.221,"15min":0.21199999999999999},"max":{"1min":0.52900000000000003,"5min":0.59999999999999998,"15min":11.967000000000001},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.38200000000000001,"15min":0.42599999999999999},"min":{"1min":0.35699999999999998,"5min":0.245,"15min":0.23999999999999999},"max":{"1min":0.53900000000000003,"5min":0.65600000000000003,"15min":4.1100000000000003},"last":0.24399999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.37,"15min":0.45900000000000002},"min":{"1min":0.32100000000000001,"5min":0.23699999999999999,"15min":0.22800000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":6.3040000000000003},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.42699999999999999,"15min":0.59699999999999998},"min":{"1min":0.39700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.58099999999999996,"5min":0.70099999999999996,"15min":11.935},"last":0.309}]},{"osd":4,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59599999999999997,"15min":0.56699999999999995},"min":{"1min":0.41399999999999998,"5min":0.375,"15min":0.34999999999999998},"max":{"1min":0.84199999999999997,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76900000000000002,"5min":1.302,"15min":1.302},"last":0.41699999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.56999999999999995},"min":{"1min":0.40999999999999998,"5min":0.38900000000000001,"15min":0.378},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.95699999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59699999999999998,"15min":0.58199999999999996},"min":{"1min":0.45100000000000001,"5min":0.35999999999999999,"15min":0.34300000000000003},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.96099999999999997},"last":0.42599999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61199999999999999,"15min":0.60099999999999998},"min":{"1min":0.497,"5min":0.40300000000000002,"15min":0.33200000000000002},"max":{"1min":0.79900000000000004,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63800000000000001,"15min":0.61199999999999999},"min":{"1min":0.499,"5min":0.35199999999999998,"15min":0.34499999999999997},"max":{"1min":0.878,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.52800000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61799999999999999,"15min":0.59299999999999997},"min":{"1min":0.44800000000000001,"5min":0.41899999999999998,"15min":0.38700000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.052},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.625,"15min":0.69399999999999995},"min":{"1min":0.46200000000000002,"5min":0.40999999999999998,"15min":0.36099999999999999},"max":{"1min":0.83099999999999996,"5min":1.274,"15min":6.2649999999999997},"last":0.48599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574995,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1151760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.32500000000000001,"15min":0.317},"min":{"1min":0.27100000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.48499999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.35299999999999998,"15min":0.33000000000000002},"min":{"1min":0.29199999999999998,"5min":0.189,"15min":0.189},"max":{"1min":0.64000000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.31}]},{"osd":1,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.37,"15min":0.34499999999999997},"min":{"1min":0.28799999999999998,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.68500000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.38900000000000001},"min":{"1min":0.309,"5min":0.26800000000000002,"15min":0.251},"max":{"1min":0.627,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.28000000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.38400000000000001,"15min":0.36299999999999999},"min":{"1min":0.33900000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.40000000000000002,"15min":0.374},"min":{"1min":0.35099999999999998,"5min":0.26800000000000002,"15min":0.255},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.68000000000000005},"last":0.40699999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57399999999999995,"15min":0.54700000000000004},"min":{"1min":0.38500000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77400000000000002,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57999999999999996,"15min":0.55200000000000005},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.749,"5min":0.79400000000000004,"15min":0.93500000000000005},"last":0.68400000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.38100000000000001,"15min":0.35899999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":1.347},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58399999999999996,"15min":0.55900000000000005},"min":{"1min":0.42999999999999999,"5min":0.39200000000000002,"15min":0.35799999999999998},"max":{"1min":0.79800000000000004,"5min":0.91300000000000003,"15min":1.01},"last":0.57499999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61699999999999999,"15min":0.59499999999999997},"min":{"1min":0.49399999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93300000000000005,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.625},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.49199999999999999,"5min":0.42299999999999999,"15min":0.374},"max":{"1min":0.89600000000000002,"5min":0.89900000000000002,"15min":1.3799999999999999},"last":0.65400000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.629,"15min":0.59599999999999997},"min":{"1min":0.46600000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82299999999999995,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63,"15min":0.60599999999999998},"min":{"1min":0.45600000000000002,"5min":0.433,"15min":0.38200000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.96999999999999997},"last":0.58699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574993,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303896,"kb_used_data":2512,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067944,"statfs":{"total":96636764160,"available":96325574656,"internally_reserved":0,"allocated":2572288,"data_stored":1157748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52800000000000002,"15min":0.51500000000000001},"min":{"1min":0.33100000000000002,"5min":0.32300000000000001,"15min":0.28699999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.747},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53600000000000003,"15min":0.64900000000000002},"min":{"1min":0.41199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70599999999999996,"5min":0.77600000000000002,"15min":7.6719999999999997},"last":0.61299999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.65700000000000003},"min":{"1min":0.377,"5min":0.32000000000000001,"15min":0.31},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":7.7160000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53900000000000003},"min":{"1min":0.44400000000000001,"5min":0.371,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":0.753,"15min":0.86799999999999999},"last":0.58399999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.51800000000000002},"min":{"1min":0.437,"5min":0.313,"15min":0.313},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.78000000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56899999999999995,"15min":0.68000000000000005},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.379},"max":{"1min":0.67900000000000005,"5min":0.81599999999999995,"15min":7.7839999999999998},"last":0.51600000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.44,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68100000000000005,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55600000000000005,"15min":0.54800000000000004},"min":{"1min":0.435,"5min":0.39200000000000002,"15min":0.35499999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53600000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.50800000000000001,"15min":0.498},"min":{"1min":0.33500000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64400000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.625},{"interface":"front","average":{"1min":0.499,"5min":0.54000000000000004,"15min":0.54300000000000004},"min":{"1min":0.34799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76200000000000001,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.55100000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.35499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.65800000000000003,"5min":0.85199999999999998,"15min":0.85899999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.69599999999999995,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.65800000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.39100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.67700000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.39900000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73499999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.57099999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574995,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303896,"kb_used_data":2512,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067944,"statfs":{"total":96636764160,"available":96325574656,"internally_reserved":0,"allocated":2572288,"data_stored":1157748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51600000000000001,"15min":0.496},"min":{"1min":0.37,"5min":0.183,"15min":0.183},"max":{"1min":0.72599999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.36699999999999999,"5min":0.314,"15min":0.215},"max":{"1min":0.84999999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.66900000000000004}]},{"osd":1,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.58399999999999996},"min":{"1min":0.33000000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":5.2679999999999998},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.215,"15min":0.215},"max":{"1min":0.76200000000000001,"5min":0.79900000000000004,"15min":5.3109999999999999},"last":0.64400000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59299999999999997,"15min":0.60899999999999999},"min":{"1min":0.36499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73999999999999999,"5min":3.27,"15min":5.2960000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.39600000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.747,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.78700000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58499999999999996,"15min":0.626},"min":{"1min":0.41399999999999998,"5min":0.39200000000000002,"15min":0.30599999999999999},"max":{"1min":0.77000000000000002,"5min":1.02,"15min":5.3330000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.629},"min":{"1min":0.45500000000000002,"5min":0.35699999999999998,"15min":0.27700000000000002},"max":{"1min":0.78100000000000003,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.60799999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.28199999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.67400000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80700000000000005,"5min":1.0429999999999999,"15min":1.0800000000000001},"last":0.70299999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53000000000000003,"15min":0.54000000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63700000000000001,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.34399999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79200000000000004,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.71399999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.371,"5min":0.35899999999999999,"15min":0.30099999999999999},"max":{"1min":0.73999999999999999,"5min":1.085,"15min":1.085},"last":0.499},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.38400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.72199999999999998,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.86599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:46:20.750 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:46:20.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.864+0000 7fd4992df700 1 Processor -- start 2023-10-10T20:46:20.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.864+0000 7fd4992df700 1 -- start start 2023-10-10T20:46:20.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd4992df700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd494159890 0x7fd49415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:20.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd4992df700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd49415a160 con 0x7fd494159c60 2023-10-10T20:46:20.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd4992df700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4940d13a0 con 0x7fd494069560 2023-10-10T20:46:20.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd4992df700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd49415e140 con 0x7fd494159890 2023-10-10T20:46:20.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd4927fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd494159c60 0x7fd49415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51684/0 (socket says 172.21.15.89:51684) 2023-10-10T20:46:20.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd4927fc700 1 -- 172.21.15.89:0/1951075556 learned_addr learned my addr 172.21.15.89:0/1951075556 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:20.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd491ffb700 1 -- 172.21.15.89:0/1951075556 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285218306 0 0) 0x7fd49415a160 con 0x7fd494159c60 2023-10-10T20:46:20.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd492ffd700 1 --2- 172.21.15.89:0/1951075556 >> v2:172.21.15.111:3300/0 conn(0x7fd494159890 0x7fd49415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:20.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd491ffb700 1 -- 172.21.15.89:0/1951075556 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd47c003570 con 0x7fd494159c60 2023-10-10T20:46:20.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd491ffb700 1 -- 172.21.15.89:0/1951075556 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1176556444 0 0) 0x7fd47c003570 con 0x7fd494159c60 2023-10-10T20:46:20.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd491ffb700 1 -- 172.21.15.89:0/1951075556 >> v2:172.21.15.111:3300/0 conn(0x7fd494159890 msgr2=0x7fd49415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:20.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd491ffb700 1 --2- 172.21.15.89:0/1951075556 >> v2:172.21.15.111:3300/0 conn(0x7fd494159890 0x7fd49415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd491ffb700 1 -- 172.21.15.89:0/1951075556 >> v1:172.21.15.111:6789/0 conn(0x7fd494069560 legacy=0x7fd4940ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:20.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd491ffb700 1 -- 172.21.15.89:0/1951075556 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49415a3d0 con 0x7fd494159c60 2023-10-10T20:46:20.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd491ffb700 1 -- 172.21.15.89:0/1951075556 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fd488001470 con 0x7fd494159c60 2023-10-10T20:46:20.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd491ffb700 1 -- 172.21.15.89:0/1951075556 >> v1:172.21.15.89:6789/0 conn(0x7fd494159c60 legacy=0x7fd49415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:20.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd491ffb700 1 --2- 172.21.15.89:0/1951075556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd47c003a60 0x7fd47c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:20.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd491ffb700 1 --2- 172.21.15.89:0/1951075556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd47c004810 0x7fd47c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:20.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd491ffb700 1 --2- 172.21.15.89:0/1951075556 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd47c0051c0 0x7fd47c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:20.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd491ffb700 1 -- 172.21.15.89:0/1951075556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49415e140 con 0x7fd47c004810 2023-10-10T20:46:20.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd491ffb700 1 -- 172.21.15.89:0/1951075556 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c00c0d0 con 0x7fd47c003a60 2023-10-10T20:46:20.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.865+0000 7fd491ffb700 1 -- 172.21.15.89:0/1951075556 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c00c290 con 0x7fd47c0051c0 2023-10-10T20:46:20.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.866+0000 7fd4927fc700 1 --2- 172.21.15.89:0/1951075556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd47c004810 0x7fd47c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:20.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.866+0000 7fd492ffd700 1 --2- 172.21.15.89:0/1951075556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd47c003a60 0x7fd47c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:20.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.866+0000 7fd4937fe700 1 --2- 172.21.15.89:0/1951075556 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd47c0051c0 0x7fd47c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:20.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.866+0000 7fd4927fc700 1 -- 172.21.15.89:0/1951075556 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd47c0051c0 msgr2=0x7fd47c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:20.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.866+0000 7fd4927fc700 1 --2- 172.21.15.89:0/1951075556 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd47c0051c0 0x7fd47c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.866+0000 7fd4927fc700 1 -- 172.21.15.89:0/1951075556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd47c003a60 msgr2=0x7fd47c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:20.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.866+0000 7fd4927fc700 1 --2- 172.21.15.89:0/1951075556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd47c003a60 0x7fd47c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.866+0000 7fd4927fc700 1 -- 172.21.15.89:0/1951075556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c003570 con 0x7fd47c004810 2023-10-10T20:46:20.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.866+0000 7fd4937fe700 1 -- 172.21.15.89:0/1951075556 reap_dead start 2023-10-10T20:46:20.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.866+0000 7fd4927fc700 1 --2- 172.21.15.89:0/1951075556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd47c004810 0x7fd47c004c80 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fd48800c3c0 tx=0x7fd488001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163e6f886123069 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:20.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.866+0000 7fd491ffb700 1 -- 172.21.15.89:0/1951075556 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48800cc90 con 0x7fd47c004810 2023-10-10T20:46:20.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.866+0000 7fd491ffb700 1 -- 172.21.15.89:0/1951075556 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48800cdf0 con 0x7fd47c004810 2023-10-10T20:46:20.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.866+0000 7fd491ffb700 1 -- 172.21.15.89:0/1951075556 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488005b50 con 0x7fd47c004810 2023-10-10T20:46:20.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.866+0000 7fd4992df700 1 -- 172.21.15.89:0/1951075556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd47c004810 msgr2=0x7fd47c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:20.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.866+0000 7fd4992df700 1 --2- 172.21.15.89:0/1951075556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd47c004810 0x7fd47c004c80 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fd48800c3c0 tx=0x7fd488001e50 comp rx=0 tx=0).stop 2023-10-10T20:46:20.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.867+0000 7fd4992df700 1 -- 172.21.15.89:0/1951075556 shutdown_connections 2023-10-10T20:46:20.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.867+0000 7fd4992df700 1 --2- 172.21.15.89:0/1951075556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd47c004810 0x7fd47c004c80 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.867+0000 7fd4992df700 1 -- 172.21.15.89:0/1951075556 >> 172.21.15.89:0/1951075556 conn(0x7fd494068e20 msgr2=0x7fd4940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:20.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.867+0000 7fd4992df700 1 -- 172.21.15.89:0/1951075556 shutdown_connections 2023-10-10T20:46:20.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.867+0000 7fd4992df700 1 -- 172.21.15.89:0/1951075556 wait complete. 2023-10-10T20:46:20.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.868+0000 7fd4992df700 1 Processor -- start 2023-10-10T20:46:20.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.868+0000 7fd4992df700 1 -- start start 2023-10-10T20:46:20.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.868+0000 7fd4992df700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd47c004810 0x7fd494156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:20.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd4992df700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4941587b0 0x7fd4940cf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:20.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd4992df700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4941494d0 0x7fd494152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:20.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd4992df700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd494156b70 con 0x7fd4941494d0 2023-10-10T20:46:20.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd4992df700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4940cd800 con 0x7fd4941587b0 2023-10-10T20:46:20.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd4992df700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4940cd1f0 con 0x7fd47c004810 2023-10-10T20:46:20.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd4937fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4941494d0 0x7fd494152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:20.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd4937fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4941494d0 0x7fd494152140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43516/0 (socket says 172.21.15.89:43516) 2023-10-10T20:46:20.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd4937fe700 1 -- 172.21.15.89:0/1780256091 learned_addr learned my addr 172.21.15.89:0/1780256091 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:20.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd492ffd700 1 --2- 172.21.15.89:0/1780256091 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd47c004810 0x7fd494156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:20.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd4927fc700 1 --2- 172.21.15.89:0/1780256091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4941587b0 0x7fd4940cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:20.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd4937fe700 1 -- 172.21.15.89:0/1780256091 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd47c004810 msgr2=0x7fd494156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:20.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd4937fe700 1 --2- 172.21.15.89:0/1780256091 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd47c004810 0x7fd494156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd4937fe700 1 -- 172.21.15.89:0/1780256091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4941587b0 msgr2=0x7fd4940cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:20.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd4937fe700 1 --2- 172.21.15.89:0/1780256091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4941587b0 0x7fd4940cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:20.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd4937fe700 1 -- 172.21.15.89:0/1780256091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4940cde20 con 0x7fd4941494d0 2023-10-10T20:46:20.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd4937fe700 1 --2- 172.21.15.89:0/1780256091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4941494d0 0x7fd494152140 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fd4940cadc0 tx=0x7fd48c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:20.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd47affd700 1 -- 172.21.15.89:0/1780256091 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c0032e0 con 0x7fd4941494d0 2023-10-10T20:46:20.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd47affd700 1 -- 172.21.15.89:0/1780256091 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48c001580 con 0x7fd4941494d0 2023-10-10T20:46:20.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd4992df700 1 -- 172.21.15.89:0/1780256091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4940cd490 con 0x7fd4941494d0 2023-10-10T20:46:20.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd47affd700 1 -- 172.21.15.89:0/1780256091 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c001940 con 0x7fd4941494d0 2023-10-10T20:46:20.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.869+0000 7fd4992df700 1 -- 172.21.15.89:0/1780256091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4940ce100 con 0x7fd4941494d0 2023-10-10T20:46:20.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.870+0000 7fd4992df700 1 -- 172.21.15.89:0/1780256091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd49404c3c0 con 0x7fd4941494d0 2023-10-10T20:46:20.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.871+0000 7fd47affd700 1 -- 172.21.15.89:0/1780256091 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd48c001cf0 con 0x7fd4941494d0 2023-10-10T20:46:20.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.871+0000 7fd47affd700 1 --2- 172.21.15.89:0/1780256091 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd480067320 0x7fd4800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:20.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.871+0000 7fd492ffd700 1 --2- 172.21.15.89:0/1780256091 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd480067320 0x7fd4800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:20.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.871+0000 7fd47affd700 1 -- 172.21.15.89:0/1780256091 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd48c084120 con 0x7fd4941494d0 2023-10-10T20:46:20.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.872+0000 7fd492ffd700 1 --2- 172.21.15.89:0/1780256091 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd480067320 0x7fd4800697d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd484006f00 tx=0x7fd4840091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:20.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:20.875+0000 7fd47affd700 1 -- 172.21.15.89:0/1780256091 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd48c0510b0 con 0x7fd4941494d0 2023-10-10T20:46:21.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.056+0000 7fd4992df700 1 -- 172.21.15.89:0/1780256091 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd49406a5b0 con 0x7fd480067320 2023-10-10T20:46:21.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.058+0000 7fd47affd700 1 -- 172.21.15.89:0/1780256091 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51010 (secure 0 0 0) 0x7fd49406a5b0 con 0x7fd480067320 2023-10-10T20:46:21.060 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:46:21.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.060+0000 7fd4992df700 1 -- 172.21.15.89:0/1780256091 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd480067320 msgr2=0x7fd4800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:21.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.060+0000 7fd4992df700 1 --2- 172.21.15.89:0/1780256091 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd480067320 0x7fd4800697d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd484006f00 tx=0x7fd4840091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.060+0000 7fd4992df700 1 -- 172.21.15.89:0/1780256091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4941494d0 msgr2=0x7fd494152140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:21.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.060+0000 7fd4992df700 1 --2- 172.21.15.89:0/1780256091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4941494d0 0x7fd494152140 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fd4940cadc0 tx=0x7fd48c009040 comp rx=0 tx=0).stop 2023-10-10T20:46:21.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.060+0000 7fd4992df700 1 -- 172.21.15.89:0/1780256091 shutdown_connections 2023-10-10T20:46:21.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.060+0000 7fd4992df700 1 --2- 172.21.15.89:0/1780256091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4941494d0 0x7fd494152140 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.060+0000 7fd4992df700 1 --2- 172.21.15.89:0/1780256091 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fd47c004810 0x7fd494156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.060+0000 7fd4992df700 1 --2- 172.21.15.89:0/1780256091 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fd480067320 0x7fd4800697d0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.060+0000 7fd4992df700 1 --2- 172.21.15.89:0/1780256091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4941587b0 0x7fd4940cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.060+0000 7fd4992df700 1 -- 172.21.15.89:0/1780256091 >> 172.21.15.89:0/1780256091 conn(0x7fd494068e20 msgr2=0x7fd4940c5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:21.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.060+0000 7fd4992df700 1 -- 172.21.15.89:0/1780256091 shutdown_connections 2023-10-10T20:46:21.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.060+0000 7fd4992df700 1 -- 172.21.15.89:0/1780256091 wait complete. 2023-10-10T20:46:21.066 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:46:21.074 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":553,"stamp":"2023-10-10T20:46:20.797584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2432296,"kb_used_data":21224,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752542424,"statfs":{"total":773094113280,"available":770603442176,"internally_reserved":0,"allocated":21733376,"data_stored":10421604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4067,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-49,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001077"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":345,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-10T20:46:14.861401+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:46:14.861401+0000","last_peered":"2023-10-10T20:46:14.861401+0000","last_clean":"2023-10-10T20:46:14.861401+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:46:14.861401+0000","last_undegraded":"2023-10-10T20:46:14.861401+0000","last_fullsized":"2023-10-10T20:46:14.861401+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304476,"kb_used_data":3092,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067364,"statfs":{"total":96636764160,"available":96324980736,"internally_reserved":0,"allocated":3166208,"data_stored":1748116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52200000000000002,"15min":0.67000000000000004},"min":{"1min":0.35499999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71499999999999997,"5min":0.748,"15min":6.8630000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52600000000000002,"15min":0.57699999999999996},"min":{"1min":0.33900000000000002,"5min":0.32300000000000001,"15min":0.307},"max":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":4.1959999999999997},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.76100000000000001},"min":{"1min":0.315,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68899999999999995,"5min":0.78400000000000003,"15min":10.1},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54700000000000004,"15min":0.64100000000000001},"min":{"1min":0.39300000000000002,"5min":0.35199999999999998,"15min":0.28999999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":6.806},"last":0.52500000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.68600000000000005},"min":{"1min":0.45900000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.82999999999999996,"5min":0.83099999999999996,"15min":6.9340000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.55300000000000005,"15min":0.64200000000000002},"min":{"1min":0.47499999999999998,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":6.9870000000000001},"last":0.44700000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55200000000000005,"15min":0.65600000000000003},"min":{"1min":0.48399999999999999,"5min":0.371,"15min":0.32800000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":6.6719999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57199999999999995,"15min":0.621},"min":{"1min":0.45000000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.74399999999999999,"5min":0.77700000000000002,"15min":4.1609999999999996},"last":0.56100000000000005}]},{"osd":4,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.45900000000000002,"15min":0.46400000000000002},"min":{"1min":0.29799999999999999,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.68200000000000005,"5min":0.69799999999999995,"15min":0.77200000000000002},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.5,"15min":0.51500000000000001},"min":{"1min":0.374,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71999999999999997,"5min":0.73899999999999999,"15min":0.80300000000000005},"last":0.33400000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50600000000000001,"15min":0.53100000000000003},"min":{"1min":0.35899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70599999999999996,"5min":0.77900000000000003,"15min":1.3180000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51200000000000001,"15min":0.52600000000000002},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.29499999999999998},"max":{"1min":0.77100000000000002,"5min":0.78200000000000003,"15min":0.84099999999999997},"last":0.371}]},{"osd":6,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":1.7509999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.375,"5min":0.35499999999999998,"15min":0.32600000000000001},"max":{"1min":0.81699999999999995,"5min":0.81999999999999995,"15min":1.7090000000000001},"last":0.55500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574995,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303896,"kb_used_data":2512,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067944,"statfs":{"total":96636764160,"available":96325574656,"internally_reserved":0,"allocated":2572288,"data_stored":1157748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53700000000000003,"15min":0.59899999999999998},"min":{"1min":0.40600000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.76800000000000002,"5min":0.83099999999999996,"15min":7.7590000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.53600000000000003,"15min":0.622},"min":{"1min":0.46100000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78700000000000003,"5min":1.075,"15min":7.8090000000000002},"last":0.61099999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54600000000000004,"15min":0.623},"min":{"1min":0.442,"5min":0.29199999999999998,"15min":0.27800000000000002},"max":{"1min":0.73399999999999999,"5min":0.91300000000000003,"15min":7.867},"last":0.625},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.72099999999999997},"min":{"1min":0.438,"5min":0.34499999999999997,"15min":0.33700000000000002},"max":{"1min":0.755,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.68600000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.35699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71899999999999997,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58899999999999997,"15min":0.70999999999999996},"min":{"1min":0.50900000000000001,"5min":0.33400000000000002,"15min":0.318},"max":{"1min":0.76000000000000001,"5min":2.641,"15min":6.7629999999999999},"last":0.69599999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57099999999999995,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72199999999999998,"5min":1.107,"15min":4.8620000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56899999999999995,"15min":0.64000000000000001},"min":{"1min":0.47499999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":1.718,"15min":7.9009999999999998},"last":0.70899999999999996}]},{"osd":4,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.51100000000000001,"15min":0.48999999999999999},"min":{"1min":0.32300000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66000000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.40899999999999997,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":0.745,"5min":1.032,"15min":1.032},"last":0.73599999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.52900000000000003,"15min":0.50700000000000001},"min":{"1min":0.33300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73299999999999998,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.55900000000000005,"15min":0.54400000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33600000000000002},"max":{"1min":0.73299999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56999999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.35799999999999998,"5min":0.33300000000000002,"15min":0.315},"max":{"1min":0.70599999999999996,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.58699999999999997,"15min":0.56399999999999995},"min":{"1min":0.376,"5min":0.36599999999999999,"15min":0.35099999999999998},"max":{"1min":0.78000000000000003,"5min":1.681,"15min":1.681},"last":0.63600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574995,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303888,"kb_used_data":2504,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067952,"statfs":{"total":96636764160,"available":96325582848,"internally_reserved":0,"allocated":2564096,"data_stored":1151794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.33000000000000002,"15min":0.40100000000000002},"min":{"1min":0.23599999999999999,"5min":0.215,"15min":0.17899999999999999},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":5.0380000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.309,"5min":0.223,"15min":0.20899999999999999},"max":{"1min":0.55500000000000005,"5min":0.65500000000000003,"15min":1.194},"last":0.39600000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.26700000000000002,"5min":0.23599999999999999,"15min":0.20599999999999999},"max":{"1min":0.49199999999999999,"5min":0.53800000000000003,"15min":1.2869999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.377,"15min":0.434},"min":{"1min":0.318,"5min":0.23999999999999999,"15min":0.22},"max":{"1min":0.57499999999999996,"5min":0.68600000000000005,"15min":5.0919999999999996},"last":0.496}]},{"osd":3,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.40100000000000002,"15min":0.45800000000000002},"min":{"1min":0.35999999999999999,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":0.54000000000000004,"5min":0.64500000000000002,"15min":5.1159999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39500000000000002},"min":{"1min":0.34899999999999998,"5min":0.23799999999999999,"15min":0.19900000000000001},"max":{"1min":0.55500000000000005,"5min":0.67000000000000004,"15min":1.2170000000000001},"last":0.47499999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.48199999999999998,"5min":0.34200000000000003,"15min":0.30199999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.254},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.57599999999999996,"15min":0.63900000000000001},"min":{"1min":0.45100000000000001,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.83899999999999997,"5min":0.88600000000000001,"15min":4.9850000000000003},"last":0.80000000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.46700000000000003,"5min":0.34799999999999998,"15min":0.34399999999999997},"max":{"1min":0.75900000000000001,"5min":0.91700000000000004,"15min":1.1799999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.58899999999999997,"15min":0.59799999999999998},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":1.3109999999999999},"last":0.81100000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.61099999999999999,"15min":0.72699999999999998},"min":{"1min":0.53200000000000003,"5min":0.378,"15min":0.35199999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":5.0659999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.623,"15min":0.626},"min":{"1min":0.44400000000000001,"5min":0.38600000000000001,"15min":0.36299999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":1.171},"last":0.83999999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.48599999999999999,"5min":0.38200000000000001,"15min":0.36499999999999999},"max":{"1min":0.86699999999999999,"5min":0.94199999999999995,"15min":1.214},"last":0.747},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60699999999999998,"15min":0.66500000000000004},"min":{"1min":0.51600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":4.0499999999999998},"last":0.79000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574993,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304460,"kb_used_data":3076,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067380,"statfs":{"total":96636764160,"available":96324997120,"internally_reserved":0,"allocated":3149824,"data_stored":1741147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.33500000000000002,"15min":0.42699999999999999},"min":{"1min":0.308,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.47199999999999998,"5min":0.63500000000000001,"15min":7.9249999999999998},"last":0.188},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.35499999999999998,"15min":0.42399999999999999},"min":{"1min":0.31900000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55600000000000005,"5min":0.63400000000000001,"15min":6.6299999999999999},"last":0.23999999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.36399999999999999,"15min":0.45200000000000001},"min":{"1min":0.26600000000000001,"5min":0.23100000000000001,"15min":0.216},"max":{"1min":0.63100000000000001,"5min":0.67800000000000005,"15min":8.2509999999999994},"last":0.224},{"interface":"front","average":{"1min":0.437,"5min":0.374,"15min":0.44600000000000001},"min":{"1min":0.29399999999999998,"5min":0.253,"15min":0.222},"max":{"1min":0.55900000000000005,"5min":0.65000000000000002,"15min":6.7370000000000001},"last":0.29699999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38400000000000001,"15min":0.46400000000000002},"min":{"1min":0.35899999999999999,"5min":0.22,"15min":0.21099999999999999},"max":{"1min":0.52600000000000002,"5min":0.53800000000000003,"15min":7.7960000000000003},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.42899999999999999,"15min":0.51400000000000001},"min":{"1min":0.35899999999999999,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.58199999999999996,"5min":0.71699999999999997,"15min":7.8929999999999998},"last":0.30399999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57099999999999995,"15min":0.64000000000000001},"min":{"1min":0.44400000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70499999999999996,"5min":0.94099999999999995,"15min":6.6929999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59099999999999997,"15min":0.67700000000000005},"min":{"1min":0.40000000000000002,"5min":0.371,"15min":0.33900000000000002},"max":{"1min":0.80000000000000004,"5min":0.84399999999999997,"15min":7.6920000000000002},"last":0.50900000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58499999999999996,"15min":0.66200000000000003},"min":{"1min":0.438,"5min":0.38400000000000001,"15min":0.36299999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":7.7690000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.60899999999999999,"15min":0.68899999999999995},"min":{"1min":0.49299999999999999,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.81899999999999995,"5min":0.97599999999999998,"15min":7.8659999999999997},"last":0.441}]},{"osd":6,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.624,"15min":0.71699999999999997},"min":{"1min":0.53700000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.85999999999999999,"5min":0.87,"15min":7.8360000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63800000000000001,"15min":0.69499999999999995},"min":{"1min":0.53000000000000003,"5min":0.41499999999999998,"15min":0.36599999999999999},"max":{"1min":0.83099999999999996,"5min":0.875,"15min":6.6660000000000004},"last":0.48699999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.626,"15min":0.68200000000000005},"min":{"1min":0.42499999999999999,"5min":0.42099999999999999,"15min":0.36299999999999999},"max":{"1min":0.86199999999999999,"5min":0.90600000000000003,"15min":6.7160000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59899999999999998,"15min":0.69099999999999995},"min":{"1min":0.49399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.873,"5min":0.89500000000000002,"15min":7.7370000000000001},"last":0.42099999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574995,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1155543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.33600000000000002,"15min":0.32600000000000001},"min":{"1min":0.26800000000000002,"5min":0.20200000000000001,"15min":0.189},"max":{"1min":0.52700000000000002,"5min":0.59399999999999997,"15min":0.67900000000000005},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.34999999999999998},"min":{"1min":0.25600000000000001,"5min":0.219,"15min":0.21199999999999999},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.746},"last":0.29499999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.54100000000000004},"min":{"1min":0.29099999999999998,"5min":0.221,"15min":0.21199999999999999},"max":{"1min":0.52900000000000003,"5min":0.59999999999999998,"15min":11.967000000000001},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.38200000000000001,"15min":0.42599999999999999},"min":{"1min":0.35699999999999998,"5min":0.245,"15min":0.23999999999999999},"max":{"1min":0.53900000000000003,"5min":0.65600000000000003,"15min":4.1100000000000003},"last":0.24399999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.37,"15min":0.45900000000000002},"min":{"1min":0.32100000000000001,"5min":0.23699999999999999,"15min":0.22800000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":6.3040000000000003},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.42699999999999999,"15min":0.59699999999999998},"min":{"1min":0.39700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.58099999999999996,"5min":0.70099999999999996,"15min":11.935},"last":0.309}]},{"osd":4,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59599999999999997,"15min":0.56699999999999995},"min":{"1min":0.41399999999999998,"5min":0.375,"15min":0.34999999999999998},"max":{"1min":0.84199999999999997,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76900000000000002,"5min":1.302,"15min":1.302},"last":0.41699999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.56999999999999995},"min":{"1min":0.40999999999999998,"5min":0.38900000000000001,"15min":0.378},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.95699999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59699999999999998,"15min":0.58199999999999996},"min":{"1min":0.45100000000000001,"5min":0.35999999999999999,"15min":0.34300000000000003},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.96099999999999997},"last":0.42599999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61199999999999999,"15min":0.60099999999999998},"min":{"1min":0.497,"5min":0.40300000000000002,"15min":0.33200000000000002},"max":{"1min":0.79900000000000004,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63800000000000001,"15min":0.61199999999999999},"min":{"1min":0.499,"5min":0.35199999999999998,"15min":0.34499999999999997},"max":{"1min":0.878,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.52800000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61799999999999999,"15min":0.59299999999999997},"min":{"1min":0.44800000000000001,"5min":0.41899999999999998,"15min":0.38700000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.052},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.625,"15min":0.69399999999999995},"min":{"1min":0.46200000000000002,"5min":0.40999999999999998,"15min":0.36099999999999999},"max":{"1min":0.83099999999999996,"5min":1.274,"15min":6.2649999999999997},"last":0.48599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574995,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1151760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.32500000000000001,"15min":0.317},"min":{"1min":0.27100000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.48499999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.35299999999999998,"15min":0.33000000000000002},"min":{"1min":0.29199999999999998,"5min":0.189,"15min":0.189},"max":{"1min":0.64000000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.31}]},{"osd":1,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.37,"15min":0.34499999999999997},"min":{"1min":0.28799999999999998,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.68500000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.38900000000000001},"min":{"1min":0.309,"5min":0.26800000000000002,"15min":0.251},"max":{"1min":0.627,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.28000000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.38400000000000001,"15min":0.36299999999999999},"min":{"1min":0.33900000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.40000000000000002,"15min":0.374},"min":{"1min":0.35099999999999998,"5min":0.26800000000000002,"15min":0.255},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.68000000000000005},"last":0.40699999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57399999999999995,"15min":0.54700000000000004},"min":{"1min":0.38500000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77400000000000002,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57999999999999996,"15min":0.55200000000000005},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.749,"5min":0.79400000000000004,"15min":0.93500000000000005},"last":0.68400000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.38100000000000001,"15min":0.35899999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":1.347},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58399999999999996,"15min":0.55900000000000005},"min":{"1min":0.42999999999999999,"5min":0.39200000000000002,"15min":0.35799999999999998},"max":{"1min":0.79800000000000004,"5min":0.91300000000000003,"15min":1.01},"last":0.57499999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61699999999999999,"15min":0.59499999999999997},"min":{"1min":0.49399999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93300000000000005,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.625},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.49199999999999999,"5min":0.42299999999999999,"15min":0.374},"max":{"1min":0.89600000000000002,"5min":0.89900000000000002,"15min":1.3799999999999999},"last":0.65400000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.629,"15min":0.59599999999999997},"min":{"1min":0.46600000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82299999999999995,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63,"15min":0.60599999999999998},"min":{"1min":0.45600000000000002,"5min":0.433,"15min":0.38200000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.96999999999999997},"last":0.58699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574993,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303896,"kb_used_data":2512,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067944,"statfs":{"total":96636764160,"available":96325574656,"internally_reserved":0,"allocated":2572288,"data_stored":1157748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52800000000000002,"15min":0.51500000000000001},"min":{"1min":0.33100000000000002,"5min":0.32300000000000001,"15min":0.28699999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.747},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53600000000000003,"15min":0.64900000000000002},"min":{"1min":0.41199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70599999999999996,"5min":0.77600000000000002,"15min":7.6719999999999997},"last":0.61299999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.65700000000000003},"min":{"1min":0.377,"5min":0.32000000000000001,"15min":0.31},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":7.7160000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53900000000000003},"min":{"1min":0.44400000000000001,"5min":0.371,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":0.753,"15min":0.86799999999999999},"last":0.58399999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.51800000000000002},"min":{"1min":0.437,"5min":0.313,"15min":0.313},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.78000000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56899999999999995,"15min":0.68000000000000005},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.379},"max":{"1min":0.67900000000000005,"5min":0.81599999999999995,"15min":7.7839999999999998},"last":0.51600000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.44,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68100000000000005,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55600000000000005,"15min":0.54800000000000004},"min":{"1min":0.435,"5min":0.39200000000000002,"15min":0.35499999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53600000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.50800000000000001,"15min":0.498},"min":{"1min":0.33500000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64400000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.625},{"interface":"front","average":{"1min":0.499,"5min":0.54000000000000004,"15min":0.54300000000000004},"min":{"1min":0.34799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76200000000000001,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.55100000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.35499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.65800000000000003,"5min":0.85199999999999998,"15min":0.85899999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.69599999999999995,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.65800000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.39100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.67700000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.39900000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73499999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.57099999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574995,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303896,"kb_used_data":2512,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067944,"statfs":{"total":96636764160,"available":96325574656,"internally_reserved":0,"allocated":2572288,"data_stored":1157748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51600000000000001,"15min":0.496},"min":{"1min":0.37,"5min":0.183,"15min":0.183},"max":{"1min":0.72599999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.36699999999999999,"5min":0.314,"15min":0.215},"max":{"1min":0.84999999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.66900000000000004}]},{"osd":1,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.58399999999999996},"min":{"1min":0.33000000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":5.2679999999999998},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.215,"15min":0.215},"max":{"1min":0.76200000000000001,"5min":0.79900000000000004,"15min":5.3109999999999999},"last":0.64400000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59299999999999997,"15min":0.60899999999999999},"min":{"1min":0.36499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73999999999999999,"5min":3.27,"15min":5.2960000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.39600000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.747,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.78700000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58499999999999996,"15min":0.626},"min":{"1min":0.41399999999999998,"5min":0.39200000000000002,"15min":0.30599999999999999},"max":{"1min":0.77000000000000002,"5min":1.02,"15min":5.3330000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.629},"min":{"1min":0.45500000000000002,"5min":0.35699999999999998,"15min":0.27700000000000002},"max":{"1min":0.78100000000000003,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.60799999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.28199999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.67400000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80700000000000005,"5min":1.0429999999999999,"15min":1.0800000000000001},"last":0.70299999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53000000000000003,"15min":0.54000000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63700000000000001,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.34399999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79200000000000004,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.71399999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.371,"5min":0.35899999999999999,"15min":0.30099999999999999},"max":{"1min":0.73999999999999999,"5min":1.085,"15min":1.085},"last":0.499},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.38400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.72199999999999998,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.86599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:46:21.076 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-10T20:46:21.077 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:46:21.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.189+0000 7f8b4ada4700 1 Processor -- start 2023-10-10T20:46:21.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.189+0000 7f8b4ada4700 1 -- start start 2023-10-10T20:46:21.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.190+0000 7f8b4ada4700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8b440b1850 0x7f8b440b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:21.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.190+0000 7f8b4ada4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8b440ae680 0x7f8b441572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:21.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.190+0000 7f8b4ada4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b440cb300 con 0x7f8b440aeb00 2023-10-10T20:46:21.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.190+0000 7f8b4ada4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b4406fc30 con 0x7f8b440ae680 2023-10-10T20:46:21.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.190+0000 7f8b4ada4700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b440cc260 con 0x7f8b440b1850 2023-10-10T20:46:21.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.190+0000 7f8b49341700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8b440aeb00 0x7f8b4415b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51690/0 (socket says 172.21.15.89:51690) 2023-10-10T20:46:21.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.190+0000 7f8b49341700 1 -- 172.21.15.89:0/3501679169 learned_addr learned my addr 172.21.15.89:0/3501679169 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:21.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.190+0000 7f8b43fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8b440ae680 0x7f8b441572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:21.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.191+0000 7f8b437fe700 1 -- 172.21.15.89:0/3501679169 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1747127000 0 0) 0x7f8b440cb300 con 0x7f8b440aeb00 2023-10-10T20:46:21.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.191+0000 7f8b48b40700 1 --2- 172.21.15.89:0/3501679169 >> v2:172.21.15.111:3300/0 conn(0x7f8b440b1850 0x7f8b440b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:21.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.191+0000 7f8b437fe700 1 -- 172.21.15.89:0/3501679169 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b34003570 con 0x7f8b440aeb00 2023-10-10T20:46:21.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.191+0000 7f8b43fff700 1 -- 172.21.15.89:0/3501679169 >> v2:172.21.15.111:3300/0 conn(0x7f8b440b1850 msgr2=0x7f8b440b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:21.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.191+0000 7f8b43fff700 1 --2- 172.21.15.89:0/3501679169 >> v2:172.21.15.111:3300/0 conn(0x7f8b440b1850 0x7f8b440b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.191+0000 7f8b43fff700 1 -- 172.21.15.89:0/3501679169 >> v1:172.21.15.89:6789/0 conn(0x7f8b440aeb00 legacy=0x7f8b4415b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:21.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.191+0000 7f8b43fff700 1 -- 172.21.15.89:0/3501679169 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b440cb5e0 con 0x7f8b440ae680 2023-10-10T20:46:21.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.191+0000 7f8b43fff700 1 --2- 172.21.15.89:0/3501679169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b440ae680 0x7f8b441572e0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f8b440641e0 tx=0x7f8b300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc2aeb4efaf4a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:21.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.191+0000 7f8b437fe700 1 -- 172.21.15.89:0/3501679169 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30011630 con 0x7f8b440ae680 2023-10-10T20:46:21.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.191+0000 7f8b437fe700 1 -- 172.21.15.89:0/3501679169 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b30011790 con 0x7f8b440ae680 2023-10-10T20:46:21.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.191+0000 7f8b437fe700 1 -- 172.21.15.89:0/3501679169 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30011b30 con 0x7f8b440ae680 2023-10-10T20:46:21.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.191+0000 7f8b4ada4700 1 -- 172.21.15.89:0/3501679169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b440ae680 msgr2=0x7f8b441572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:21.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.191+0000 7f8b4ada4700 1 --2- 172.21.15.89:0/3501679169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b440ae680 0x7f8b441572e0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f8b440641e0 tx=0x7f8b300091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.191+0000 7f8b4ada4700 1 -- 172.21.15.89:0/3501679169 shutdown_connections 2023-10-10T20:46:21.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.191+0000 7f8b4ada4700 1 --2- 172.21.15.89:0/3501679169 >> v2:172.21.15.111:3300/0 conn(0x7f8b440b1850 0x7f8b440b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.191+0000 7f8b4ada4700 1 --2- 172.21.15.89:0/3501679169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b440ae680 0x7f8b441572e0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.191+0000 7f8b4ada4700 1 -- 172.21.15.89:0/3501679169 >> 172.21.15.89:0/3501679169 conn(0x7f8b4406a320 msgr2=0x7f8b440cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:21.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.192+0000 7f8b4ada4700 1 -- 172.21.15.89:0/3501679169 shutdown_connections 2023-10-10T20:46:21.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.192+0000 7f8b4ada4700 1 -- 172.21.15.89:0/3501679169 wait complete. 2023-10-10T20:46:21.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.193+0000 7f8b4ada4700 1 Processor -- start 2023-10-10T20:46:21.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.193+0000 7f8b4ada4700 1 -- start start 2023-10-10T20:46:21.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.193+0000 7f8b4ada4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b440ae680 0x7f8b440a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:21.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.193+0000 7f8b4ada4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b440aeb00 0x7f8b441579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:21.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.193+0000 7f8b4ada4700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b440b1850 0x7f8b4413dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:21.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.193+0000 7f8b4ada4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b44159440 con 0x7f8b440ae680 2023-10-10T20:46:21.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.193+0000 7f8b4ada4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b441583a0 con 0x7f8b440aeb00 2023-10-10T20:46:21.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.193+0000 7f8b4ada4700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b44158680 con 0x7f8b440b1850 2023-10-10T20:46:21.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.194+0000 7f8b48b40700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b440ae680 0x7f8b440a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:21.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.194+0000 7f8b48b40700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b440ae680 0x7f8b440a5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43534/0 (socket says 172.21.15.89:43534) 2023-10-10T20:46:21.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.194+0000 7f8b48b40700 1 -- 172.21.15.89:0/3111843922 learned_addr learned my addr 172.21.15.89:0/3111843922 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:21.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.194+0000 7f8b48b40700 1 -- 172.21.15.89:0/3111843922 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b440b1850 msgr2=0x7f8b4413dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:21.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.194+0000 7f8b48b40700 1 --2- 172.21.15.89:0/3111843922 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b440b1850 0x7f8b4413dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.194+0000 7f8b48b40700 1 -- 172.21.15.89:0/3111843922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b440aeb00 msgr2=0x7f8b441579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:21.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.194+0000 7f8b48b40700 1 --2- 172.21.15.89:0/3111843922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b440aeb00 0x7f8b441579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.194+0000 7f8b48b40700 1 -- 172.21.15.89:0/3111843922 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b30009420 con 0x7f8b440ae680 2023-10-10T20:46:21.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.194+0000 7f8b48b40700 1 --2- 172.21.15.89:0/3111843922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b440ae680 0x7f8b440a5710 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800d860 tx=0x7f8b38008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:21.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.194+0000 7f8b40ff9700 1 -- 172.21.15.89:0/3111843922 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38009610 con 0x7f8b440ae680 2023-10-10T20:46:21.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.194+0000 7f8b40ff9700 1 -- 172.21.15.89:0/3111843922 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b38009770 con 0x7f8b440ae680 2023-10-10T20:46:21.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.194+0000 7f8b4ada4700 1 -- 172.21.15.89:0/3111843922 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b44158960 con 0x7f8b440ae680 2023-10-10T20:46:21.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.194+0000 7f8b40ff9700 1 -- 172.21.15.89:0/3111843922 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38009b30 con 0x7f8b440ae680 2023-10-10T20:46:21.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.194+0000 7f8b4ada4700 1 -- 172.21.15.89:0/3111843922 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b44158f20 con 0x7f8b440ae680 2023-10-10T20:46:21.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.195+0000 7f8b4ada4700 1 -- 172.21.15.89:0/3111843922 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b4404c3c0 con 0x7f8b440ae680 2023-10-10T20:46:21.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.195+0000 7f8b40ff9700 1 -- 172.21.15.89:0/3111843922 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8b3801e460 con 0x7f8b440ae680 2023-10-10T20:46:21.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.196+0000 7f8b40ff9700 1 --2- 172.21.15.89:0/3111843922 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b28067320 0x7f8b280697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:21.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.196+0000 7f8b40ff9700 1 -- 172.21.15.89:0/3111843922 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8b38084fe0 con 0x7f8b440ae680 2023-10-10T20:46:21.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.196+0000 7f8b43fff700 1 --2- 172.21.15.89:0/3111843922 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b28067320 0x7f8b280697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:21.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.196+0000 7f8b43fff700 1 --2- 172.21.15.89:0/3111843922 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b28067320 0x7f8b280697d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8b4415a620 tx=0x7f8b30017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:21.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.200+0000 7f8b40ff9700 1 -- 172.21.15.89:0/3111843922 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b38051f70 con 0x7f8b440ae680 2023-10-10T20:46:21.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.381+0000 7f8b4ada4700 1 -- 172.21.15.89:0/3111843922 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b440cb8c0 con 0x7f8b28067320 2023-10-10T20:46:21.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.383+0000 7f8b40ff9700 1 -- 172.21.15.89:0/3111843922 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51010 (secure 0 0 0) 0x7f8b440cb8c0 con 0x7f8b28067320 2023-10-10T20:46:21.385 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:46:21.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.385+0000 7f8b4ada4700 1 -- 172.21.15.89:0/3111843922 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b28067320 msgr2=0x7f8b280697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:21.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.385+0000 7f8b4ada4700 1 --2- 172.21.15.89:0/3111843922 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b28067320 0x7f8b280697d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8b4415a620 tx=0x7f8b30017000 comp rx=0 tx=0).stop 2023-10-10T20:46:21.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.385+0000 7f8b4ada4700 1 -- 172.21.15.89:0/3111843922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b440ae680 msgr2=0x7f8b440a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:21.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.385+0000 7f8b4ada4700 1 --2- 172.21.15.89:0/3111843922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b440ae680 0x7f8b440a5710 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800d860 tx=0x7f8b38008040 comp rx=0 tx=0).stop 2023-10-10T20:46:21.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.385+0000 7f8b4ada4700 1 -- 172.21.15.89:0/3111843922 shutdown_connections 2023-10-10T20:46:21.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.385+0000 7f8b4ada4700 1 --2- 172.21.15.89:0/3111843922 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f8b440b1850 0x7f8b4413dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.385+0000 7f8b4ada4700 1 --2- 172.21.15.89:0/3111843922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b440ae680 0x7f8b440a5710 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.385+0000 7f8b4ada4700 1 --2- 172.21.15.89:0/3111843922 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f8b28067320 0x7f8b280697d0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.385+0000 7f8b4ada4700 1 --2- 172.21.15.89:0/3111843922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b440aeb00 0x7f8b441579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.385+0000 7f8b4ada4700 1 -- 172.21.15.89:0/3111843922 >> 172.21.15.89:0/3111843922 conn(0x7f8b4406a320 msgr2=0x7f8b440ca880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:21.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.385+0000 7f8b4ada4700 1 -- 172.21.15.89:0/3111843922 shutdown_connections 2023-10-10T20:46:21.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.385+0000 7f8b4ada4700 1 -- 172.21.15.89:0/3111843922 wait complete. 2023-10-10T20:46:21.391 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:46:21.399 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":553,"stamp":"2023-10-10T20:46:20.797584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2432296,"kb_used_data":21224,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752542424,"statfs":{"total":773094113280,"available":770603442176,"internally_reserved":0,"allocated":21733376,"data_stored":10421604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4067,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-49,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001077"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":345,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-10T20:46:14.861401+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:46:14.861401+0000","last_peered":"2023-10-10T20:46:14.861401+0000","last_clean":"2023-10-10T20:46:14.861401+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:46:14.861401+0000","last_undegraded":"2023-10-10T20:46:14.861401+0000","last_fullsized":"2023-10-10T20:46:14.861401+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304476,"kb_used_data":3092,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067364,"statfs":{"total":96636764160,"available":96324980736,"internally_reserved":0,"allocated":3166208,"data_stored":1748116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52200000000000002,"15min":0.67000000000000004},"min":{"1min":0.35499999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71499999999999997,"5min":0.748,"15min":6.8630000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52600000000000002,"15min":0.57699999999999996},"min":{"1min":0.33900000000000002,"5min":0.32300000000000001,"15min":0.307},"max":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":4.1959999999999997},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.76100000000000001},"min":{"1min":0.315,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68899999999999995,"5min":0.78400000000000003,"15min":10.1},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54700000000000004,"15min":0.64100000000000001},"min":{"1min":0.39300000000000002,"5min":0.35199999999999998,"15min":0.28999999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":6.806},"last":0.52500000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.68600000000000005},"min":{"1min":0.45900000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.82999999999999996,"5min":0.83099999999999996,"15min":6.9340000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.55300000000000005,"15min":0.64200000000000002},"min":{"1min":0.47499999999999998,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":6.9870000000000001},"last":0.44700000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55200000000000005,"15min":0.65600000000000003},"min":{"1min":0.48399999999999999,"5min":0.371,"15min":0.32800000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":6.6719999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57199999999999995,"15min":0.621},"min":{"1min":0.45000000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.74399999999999999,"5min":0.77700000000000002,"15min":4.1609999999999996},"last":0.56100000000000005}]},{"osd":4,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.45900000000000002,"15min":0.46400000000000002},"min":{"1min":0.29799999999999999,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.68200000000000005,"5min":0.69799999999999995,"15min":0.77200000000000002},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.5,"15min":0.51500000000000001},"min":{"1min":0.374,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71999999999999997,"5min":0.73899999999999999,"15min":0.80300000000000005},"last":0.33400000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50600000000000001,"15min":0.53100000000000003},"min":{"1min":0.35899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70599999999999996,"5min":0.77900000000000003,"15min":1.3180000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51200000000000001,"15min":0.52600000000000002},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.29499999999999998},"max":{"1min":0.77100000000000002,"5min":0.78200000000000003,"15min":0.84099999999999997},"last":0.371}]},{"osd":6,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":1.7509999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.375,"5min":0.35499999999999998,"15min":0.32600000000000001},"max":{"1min":0.81699999999999995,"5min":0.81999999999999995,"15min":1.7090000000000001},"last":0.55500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574995,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303896,"kb_used_data":2512,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067944,"statfs":{"total":96636764160,"available":96325574656,"internally_reserved":0,"allocated":2572288,"data_stored":1157748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53700000000000003,"15min":0.59899999999999998},"min":{"1min":0.40600000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.76800000000000002,"5min":0.83099999999999996,"15min":7.7590000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.53600000000000003,"15min":0.622},"min":{"1min":0.46100000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78700000000000003,"5min":1.075,"15min":7.8090000000000002},"last":0.61099999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54600000000000004,"15min":0.623},"min":{"1min":0.442,"5min":0.29199999999999998,"15min":0.27800000000000002},"max":{"1min":0.73399999999999999,"5min":0.91300000000000003,"15min":7.867},"last":0.625},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.72099999999999997},"min":{"1min":0.438,"5min":0.34499999999999997,"15min":0.33700000000000002},"max":{"1min":0.755,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.68600000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.35699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71899999999999997,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58899999999999997,"15min":0.70999999999999996},"min":{"1min":0.50900000000000001,"5min":0.33400000000000002,"15min":0.318},"max":{"1min":0.76000000000000001,"5min":2.641,"15min":6.7629999999999999},"last":0.69599999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57099999999999995,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72199999999999998,"5min":1.107,"15min":4.8620000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56899999999999995,"15min":0.64000000000000001},"min":{"1min":0.47499999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":1.718,"15min":7.9009999999999998},"last":0.70899999999999996}]},{"osd":4,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.51100000000000001,"15min":0.48999999999999999},"min":{"1min":0.32300000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66000000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.40899999999999997,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":0.745,"5min":1.032,"15min":1.032},"last":0.73599999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.52900000000000003,"15min":0.50700000000000001},"min":{"1min":0.33300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73299999999999998,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.55900000000000005,"15min":0.54400000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33600000000000002},"max":{"1min":0.73299999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56999999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.35799999999999998,"5min":0.33300000000000002,"15min":0.315},"max":{"1min":0.70599999999999996,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.58699999999999997,"15min":0.56399999999999995},"min":{"1min":0.376,"5min":0.36599999999999999,"15min":0.35099999999999998},"max":{"1min":0.78000000000000003,"5min":1.681,"15min":1.681},"last":0.63600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574995,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303888,"kb_used_data":2504,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067952,"statfs":{"total":96636764160,"available":96325582848,"internally_reserved":0,"allocated":2564096,"data_stored":1151794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.33000000000000002,"15min":0.40100000000000002},"min":{"1min":0.23599999999999999,"5min":0.215,"15min":0.17899999999999999},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":5.0380000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.309,"5min":0.223,"15min":0.20899999999999999},"max":{"1min":0.55500000000000005,"5min":0.65500000000000003,"15min":1.194},"last":0.39600000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.26700000000000002,"5min":0.23599999999999999,"15min":0.20599999999999999},"max":{"1min":0.49199999999999999,"5min":0.53800000000000003,"15min":1.2869999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.377,"15min":0.434},"min":{"1min":0.318,"5min":0.23999999999999999,"15min":0.22},"max":{"1min":0.57499999999999996,"5min":0.68600000000000005,"15min":5.0919999999999996},"last":0.496}]},{"osd":3,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.40100000000000002,"15min":0.45800000000000002},"min":{"1min":0.35999999999999999,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":0.54000000000000004,"5min":0.64500000000000002,"15min":5.1159999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39500000000000002},"min":{"1min":0.34899999999999998,"5min":0.23799999999999999,"15min":0.19900000000000001},"max":{"1min":0.55500000000000005,"5min":0.67000000000000004,"15min":1.2170000000000001},"last":0.47499999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.48199999999999998,"5min":0.34200000000000003,"15min":0.30199999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.254},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.57599999999999996,"15min":0.63900000000000001},"min":{"1min":0.45100000000000001,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.83899999999999997,"5min":0.88600000000000001,"15min":4.9850000000000003},"last":0.80000000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.46700000000000003,"5min":0.34799999999999998,"15min":0.34399999999999997},"max":{"1min":0.75900000000000001,"5min":0.91700000000000004,"15min":1.1799999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.58899999999999997,"15min":0.59799999999999998},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":1.3109999999999999},"last":0.81100000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.61099999999999999,"15min":0.72699999999999998},"min":{"1min":0.53200000000000003,"5min":0.378,"15min":0.35199999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":5.0659999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.623,"15min":0.626},"min":{"1min":0.44400000000000001,"5min":0.38600000000000001,"15min":0.36299999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":1.171},"last":0.83999999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.48599999999999999,"5min":0.38200000000000001,"15min":0.36499999999999999},"max":{"1min":0.86699999999999999,"5min":0.94199999999999995,"15min":1.214},"last":0.747},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60699999999999998,"15min":0.66500000000000004},"min":{"1min":0.51600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":4.0499999999999998},"last":0.79000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574993,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304460,"kb_used_data":3076,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067380,"statfs":{"total":96636764160,"available":96324997120,"internally_reserved":0,"allocated":3149824,"data_stored":1741147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.33500000000000002,"15min":0.42699999999999999},"min":{"1min":0.308,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.47199999999999998,"5min":0.63500000000000001,"15min":7.9249999999999998},"last":0.188},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.35499999999999998,"15min":0.42399999999999999},"min":{"1min":0.31900000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55600000000000005,"5min":0.63400000000000001,"15min":6.6299999999999999},"last":0.23999999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.36399999999999999,"15min":0.45200000000000001},"min":{"1min":0.26600000000000001,"5min":0.23100000000000001,"15min":0.216},"max":{"1min":0.63100000000000001,"5min":0.67800000000000005,"15min":8.2509999999999994},"last":0.224},{"interface":"front","average":{"1min":0.437,"5min":0.374,"15min":0.44600000000000001},"min":{"1min":0.29399999999999998,"5min":0.253,"15min":0.222},"max":{"1min":0.55900000000000005,"5min":0.65000000000000002,"15min":6.7370000000000001},"last":0.29699999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38400000000000001,"15min":0.46400000000000002},"min":{"1min":0.35899999999999999,"5min":0.22,"15min":0.21099999999999999},"max":{"1min":0.52600000000000002,"5min":0.53800000000000003,"15min":7.7960000000000003},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.42899999999999999,"15min":0.51400000000000001},"min":{"1min":0.35899999999999999,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.58199999999999996,"5min":0.71699999999999997,"15min":7.8929999999999998},"last":0.30399999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57099999999999995,"15min":0.64000000000000001},"min":{"1min":0.44400000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70499999999999996,"5min":0.94099999999999995,"15min":6.6929999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59099999999999997,"15min":0.67700000000000005},"min":{"1min":0.40000000000000002,"5min":0.371,"15min":0.33900000000000002},"max":{"1min":0.80000000000000004,"5min":0.84399999999999997,"15min":7.6920000000000002},"last":0.50900000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58499999999999996,"15min":0.66200000000000003},"min":{"1min":0.438,"5min":0.38400000000000001,"15min":0.36299999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":7.7690000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.60899999999999999,"15min":0.68899999999999995},"min":{"1min":0.49299999999999999,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.81899999999999995,"5min":0.97599999999999998,"15min":7.8659999999999997},"last":0.441}]},{"osd":6,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.624,"15min":0.71699999999999997},"min":{"1min":0.53700000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.85999999999999999,"5min":0.87,"15min":7.8360000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63800000000000001,"15min":0.69499999999999995},"min":{"1min":0.53000000000000003,"5min":0.41499999999999998,"15min":0.36599999999999999},"max":{"1min":0.83099999999999996,"5min":0.875,"15min":6.6660000000000004},"last":0.48699999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.626,"15min":0.68200000000000005},"min":{"1min":0.42499999999999999,"5min":0.42099999999999999,"15min":0.36299999999999999},"max":{"1min":0.86199999999999999,"5min":0.90600000000000003,"15min":6.7160000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59899999999999998,"15min":0.69099999999999995},"min":{"1min":0.49399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.873,"5min":0.89500000000000002,"15min":7.7370000000000001},"last":0.42099999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574995,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1155543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.33600000000000002,"15min":0.32600000000000001},"min":{"1min":0.26800000000000002,"5min":0.20200000000000001,"15min":0.189},"max":{"1min":0.52700000000000002,"5min":0.59399999999999997,"15min":0.67900000000000005},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.34999999999999998},"min":{"1min":0.25600000000000001,"5min":0.219,"15min":0.21199999999999999},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.746},"last":0.29499999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.54100000000000004},"min":{"1min":0.29099999999999998,"5min":0.221,"15min":0.21199999999999999},"max":{"1min":0.52900000000000003,"5min":0.59999999999999998,"15min":11.967000000000001},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.38200000000000001,"15min":0.42599999999999999},"min":{"1min":0.35699999999999998,"5min":0.245,"15min":0.23999999999999999},"max":{"1min":0.53900000000000003,"5min":0.65600000000000003,"15min":4.1100000000000003},"last":0.24399999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.37,"15min":0.45900000000000002},"min":{"1min":0.32100000000000001,"5min":0.23699999999999999,"15min":0.22800000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":6.3040000000000003},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.42699999999999999,"15min":0.59699999999999998},"min":{"1min":0.39700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.58099999999999996,"5min":0.70099999999999996,"15min":11.935},"last":0.309}]},{"osd":4,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59599999999999997,"15min":0.56699999999999995},"min":{"1min":0.41399999999999998,"5min":0.375,"15min":0.34999999999999998},"max":{"1min":0.84199999999999997,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76900000000000002,"5min":1.302,"15min":1.302},"last":0.41699999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.56999999999999995},"min":{"1min":0.40999999999999998,"5min":0.38900000000000001,"15min":0.378},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.95699999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59699999999999998,"15min":0.58199999999999996},"min":{"1min":0.45100000000000001,"5min":0.35999999999999999,"15min":0.34300000000000003},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.96099999999999997},"last":0.42599999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61199999999999999,"15min":0.60099999999999998},"min":{"1min":0.497,"5min":0.40300000000000002,"15min":0.33200000000000002},"max":{"1min":0.79900000000000004,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63800000000000001,"15min":0.61199999999999999},"min":{"1min":0.499,"5min":0.35199999999999998,"15min":0.34499999999999997},"max":{"1min":0.878,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.52800000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61799999999999999,"15min":0.59299999999999997},"min":{"1min":0.44800000000000001,"5min":0.41899999999999998,"15min":0.38700000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.052},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.625,"15min":0.69399999999999995},"min":{"1min":0.46200000000000002,"5min":0.40999999999999998,"15min":0.36099999999999999},"max":{"1min":0.83099999999999996,"5min":1.274,"15min":6.2649999999999997},"last":0.48599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574995,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1151760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.32500000000000001,"15min":0.317},"min":{"1min":0.27100000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.48499999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.35299999999999998,"15min":0.33000000000000002},"min":{"1min":0.29199999999999998,"5min":0.189,"15min":0.189},"max":{"1min":0.64000000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.31}]},{"osd":1,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.37,"15min":0.34499999999999997},"min":{"1min":0.28799999999999998,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.68500000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.38900000000000001},"min":{"1min":0.309,"5min":0.26800000000000002,"15min":0.251},"max":{"1min":0.627,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.28000000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.38400000000000001,"15min":0.36299999999999999},"min":{"1min":0.33900000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.40000000000000002,"15min":0.374},"min":{"1min":0.35099999999999998,"5min":0.26800000000000002,"15min":0.255},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.68000000000000005},"last":0.40699999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57399999999999995,"15min":0.54700000000000004},"min":{"1min":0.38500000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77400000000000002,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57999999999999996,"15min":0.55200000000000005},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.749,"5min":0.79400000000000004,"15min":0.93500000000000005},"last":0.68400000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.38100000000000001,"15min":0.35899999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":1.347},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58399999999999996,"15min":0.55900000000000005},"min":{"1min":0.42999999999999999,"5min":0.39200000000000002,"15min":0.35799999999999998},"max":{"1min":0.79800000000000004,"5min":0.91300000000000003,"15min":1.01},"last":0.57499999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61699999999999999,"15min":0.59499999999999997},"min":{"1min":0.49399999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93300000000000005,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.625},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.49199999999999999,"5min":0.42299999999999999,"15min":0.374},"max":{"1min":0.89600000000000002,"5min":0.89900000000000002,"15min":1.3799999999999999},"last":0.65400000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.629,"15min":0.59599999999999997},"min":{"1min":0.46600000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82299999999999995,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63,"15min":0.60599999999999998},"min":{"1min":0.45600000000000002,"5min":0.433,"15min":0.38200000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.96999999999999997},"last":0.58699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574993,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303896,"kb_used_data":2512,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067944,"statfs":{"total":96636764160,"available":96325574656,"internally_reserved":0,"allocated":2572288,"data_stored":1157748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52800000000000002,"15min":0.51500000000000001},"min":{"1min":0.33100000000000002,"5min":0.32300000000000001,"15min":0.28699999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.747},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53600000000000003,"15min":0.64900000000000002},"min":{"1min":0.41199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70599999999999996,"5min":0.77600000000000002,"15min":7.6719999999999997},"last":0.61299999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.65700000000000003},"min":{"1min":0.377,"5min":0.32000000000000001,"15min":0.31},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":7.7160000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53900000000000003},"min":{"1min":0.44400000000000001,"5min":0.371,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":0.753,"15min":0.86799999999999999},"last":0.58399999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.51800000000000002},"min":{"1min":0.437,"5min":0.313,"15min":0.313},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.78000000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56899999999999995,"15min":0.68000000000000005},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.379},"max":{"1min":0.67900000000000005,"5min":0.81599999999999995,"15min":7.7839999999999998},"last":0.51600000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.44,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68100000000000005,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55600000000000005,"15min":0.54800000000000004},"min":{"1min":0.435,"5min":0.39200000000000002,"15min":0.35499999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53600000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.50800000000000001,"15min":0.498},"min":{"1min":0.33500000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64400000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.625},{"interface":"front","average":{"1min":0.499,"5min":0.54000000000000004,"15min":0.54300000000000004},"min":{"1min":0.34799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76200000000000001,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.55100000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.35499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.65800000000000003,"5min":0.85199999999999998,"15min":0.85899999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.69599999999999995,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.65800000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.39100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.67700000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.39900000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73499999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.57099999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574995,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303896,"kb_used_data":2512,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067944,"statfs":{"total":96636764160,"available":96325574656,"internally_reserved":0,"allocated":2572288,"data_stored":1157748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51600000000000001,"15min":0.496},"min":{"1min":0.37,"5min":0.183,"15min":0.183},"max":{"1min":0.72599999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.36699999999999999,"5min":0.314,"15min":0.215},"max":{"1min":0.84999999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.66900000000000004}]},{"osd":1,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.58399999999999996},"min":{"1min":0.33000000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":5.2679999999999998},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.215,"15min":0.215},"max":{"1min":0.76200000000000001,"5min":0.79900000000000004,"15min":5.3109999999999999},"last":0.64400000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59299999999999997,"15min":0.60899999999999999},"min":{"1min":0.36499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73999999999999999,"5min":3.27,"15min":5.2960000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.39600000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.747,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.78700000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58499999999999996,"15min":0.626},"min":{"1min":0.41399999999999998,"5min":0.39200000000000002,"15min":0.30599999999999999},"max":{"1min":0.77000000000000002,"5min":1.02,"15min":5.3330000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.629},"min":{"1min":0.45500000000000002,"5min":0.35699999999999998,"15min":0.27700000000000002},"max":{"1min":0.78100000000000003,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.60799999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.28199999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.67400000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80700000000000005,"5min":1.0429999999999999,"15min":1.0800000000000001},"last":0.70299999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53000000000000003,"15min":0.54000000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63700000000000001,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.34399999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79200000000000004,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.71399999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.371,"5min":0.35899999999999999,"15min":0.30099999999999999},"max":{"1min":0.73999999999999999,"5min":1.085,"15min":1.085},"last":0.499},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.38400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.72199999999999998,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.86599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:46:21.401 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:46:21.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.513+0000 7f1dc5379700 1 Processor -- start 2023-10-10T20:46:21.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.513+0000 7f1dc5379700 1 -- start start 2023-10-10T20:46:21.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.514+0000 7f1dc5379700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1dc0159420 0x7f1dc00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:21.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.514+0000 7f1dc5379700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc00b2980 con 0x7f1dc015d230 2023-10-10T20:46:21.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.514+0000 7f1dc5379700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc00b11d0 con 0x7f1dc01597f0 2023-10-10T20:46:21.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.514+0000 7f1dc5379700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc00b14b0 con 0x7f1dc0159420 2023-10-10T20:46:21.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.514+0000 7f1dc4b78700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1dc015d230 0x7f1dc015d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51700/0 (socket says 172.21.15.89:51700) 2023-10-10T20:46:21.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.514+0000 7f1dc4b78700 1 -- 172.21.15.89:0/341654054 learned_addr learned my addr 172.21.15.89:0/341654054 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:21.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbeffd700 1 -- 172.21.15.89:0/341654054 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439140184 0 0) 0x7f1dc00b2980 con 0x7f1dc015d230 2023-10-10T20:46:21.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbffff700 1 --2- 172.21.15.89:0/341654054 >> v2:172.21.15.111:3300/0 conn(0x7f1dc0159420 0x7f1dc00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:21.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbeffd700 1 -- 172.21.15.89:0/341654054 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1da8003570 con 0x7f1dc015d230 2023-10-10T20:46:21.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbeffd700 1 -- 172.21.15.89:0/341654054 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1798634899 0 0) 0x7f1da8003570 con 0x7f1dc015d230 2023-10-10T20:46:21.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbeffd700 1 -- 172.21.15.89:0/341654054 >> v2:172.21.15.111:3300/0 conn(0x7f1dc0159420 msgr2=0x7f1dc00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:21.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbeffd700 1 --2- 172.21.15.89:0/341654054 >> v2:172.21.15.111:3300/0 conn(0x7f1dc0159420 0x7f1dc00b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbeffd700 1 -- 172.21.15.89:0/341654054 >> v1:172.21.15.111:6789/0 conn(0x7f1dc01597f0 legacy=0x7f1dc00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:21.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbeffd700 1 -- 172.21.15.89:0/341654054 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc00b1790 con 0x7f1dc015d230 2023-10-10T20:46:21.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbeffd700 1 -- 172.21.15.89:0/341654054 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f1db80017f0 con 0x7f1dc015d230 2023-10-10T20:46:21.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbeffd700 1 -- 172.21.15.89:0/341654054 >> v1:172.21.15.89:6789/0 conn(0x7f1dc015d230 legacy=0x7f1dc015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:21.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbeffd700 1 --2- 172.21.15.89:0/341654054 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1da8003a60 0x7f1da8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:21.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbeffd700 1 --2- 172.21.15.89:0/341654054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da8004810 0x7f1da8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:21.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbeffd700 1 --2- 172.21.15.89:0/341654054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1da80051c0 0x7f1da800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:21.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbeffd700 1 -- 172.21.15.89:0/341654054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc00b14b0 con 0x7f1da8004810 2023-10-10T20:46:21.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbeffd700 1 -- 172.21.15.89:0/341654054 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da800c0d0 con 0x7f1da80051c0 2023-10-10T20:46:21.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbeffd700 1 -- 172.21.15.89:0/341654054 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da800c290 con 0x7f1da8003a60 2023-10-10T20:46:21.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbf7fe700 1 --2- 172.21.15.89:0/341654054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da8004810 0x7f1da8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:21.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dc4b78700 1 --2- 172.21.15.89:0/341654054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1da80051c0 0x7f1da800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:21.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbffff700 1 --2- 172.21.15.89:0/341654054 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1da8003a60 0x7f1da8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:21.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbf7fe700 1 -- 172.21.15.89:0/341654054 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1da8003a60 msgr2=0x7f1da8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:21.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbf7fe700 1 --2- 172.21.15.89:0/341654054 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1da8003a60 0x7f1da8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbf7fe700 1 -- 172.21.15.89:0/341654054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1da80051c0 msgr2=0x7f1da800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:21.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbf7fe700 1 --2- 172.21.15.89:0/341654054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1da80051c0 0x7f1da800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbf7fe700 1 -- 172.21.15.89:0/341654054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da8003570 con 0x7f1da8004810 2023-10-10T20:46:21.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dc4b78700 1 --2- 172.21.15.89:0/341654054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1da80051c0 0x7f1da800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:21.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dbffff700 1 --2- 172.21.15.89:0/341654054 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1da8003a60 0x7f1da8003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:21.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.515+0000 7f1dc4b78700 1 -- 172.21.15.89:0/341654054 reap_dead start 2023-10-10T20:46:21.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.516+0000 7f1dbf7fe700 1 --2- 172.21.15.89:0/341654054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da8004810 0x7f1da8004c80 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f1db4001880 tx=0x7f1db4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc00b50a74dc26b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:21.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.516+0000 7f1dbeffd700 1 -- 172.21.15.89:0/341654054 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db4012420 con 0x7f1da8004810 2023-10-10T20:46:21.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.516+0000 7f1dbeffd700 1 -- 172.21.15.89:0/341654054 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db40125a0 con 0x7f1da8004810 2023-10-10T20:46:21.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.516+0000 7f1dbeffd700 1 -- 172.21.15.89:0/341654054 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db4012960 con 0x7f1da8004810 2023-10-10T20:46:21.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.516+0000 7f1dc5379700 1 -- 172.21.15.89:0/341654054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da8004810 msgr2=0x7f1da8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:21.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.516+0000 7f1dc5379700 1 --2- 172.21.15.89:0/341654054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da8004810 0x7f1da8004c80 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f1db4001880 tx=0x7f1db4007d30 comp rx=0 tx=0).stop 2023-10-10T20:46:21.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.516+0000 7f1dc5379700 1 -- 172.21.15.89:0/341654054 shutdown_connections 2023-10-10T20:46:21.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.516+0000 7f1dc5379700 1 --2- 172.21.15.89:0/341654054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da8004810 0x7f1da8004c80 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.516+0000 7f1dc5379700 1 -- 172.21.15.89:0/341654054 >> 172.21.15.89:0/341654054 conn(0x7f1dc00d0660 msgr2=0x7f1dc00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:21.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.516+0000 7f1dc5379700 1 -- 172.21.15.89:0/341654054 shutdown_connections 2023-10-10T20:46:21.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.516+0000 7f1dc5379700 1 -- 172.21.15.89:0/341654054 wait complete. 2023-10-10T20:46:21.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.517+0000 7f1dc5379700 1 Processor -- start 2023-10-10T20:46:21.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.517+0000 7f1dc5379700 1 -- start start 2023-10-10T20:46:21.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.518+0000 7f1dc5379700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da8004810 0x7f1dc01559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:21.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.518+0000 7f1dc5379700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dc014cf10 0x7f1dc014d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:21.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.518+0000 7f1dc5379700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1dc015c2c0 0x7f1dc015c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:21.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.518+0000 7f1dc5379700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc0151590 con 0x7f1da8004810 2023-10-10T20:46:21.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.518+0000 7f1dc5379700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc015cff0 con 0x7f1dc014cf10 2023-10-10T20:46:21.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.518+0000 7f1dc5379700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc014d8c0 con 0x7f1dc015c2c0 2023-10-10T20:46:21.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.518+0000 7f1dbffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da8004810 0x7f1dc01559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:21.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.518+0000 7f1dbffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da8004810 0x7f1dc01559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43550/0 (socket says 172.21.15.89:43550) 2023-10-10T20:46:21.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.518+0000 7f1dbffff700 1 -- 172.21.15.89:0/2138584604 learned_addr learned my addr 172.21.15.89:0/2138584604 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:21.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.518+0000 7f1dbffff700 1 -- 172.21.15.89:0/2138584604 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1dc015c2c0 msgr2=0x7f1dc015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:21.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.518+0000 7f1dbffff700 1 --2- 172.21.15.89:0/2138584604 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1dc015c2c0 0x7f1dc015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.519+0000 7f1dbffff700 1 -- 172.21.15.89:0/2138584604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dc014cf10 msgr2=0x7f1dc014d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:21.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.519+0000 7f1dbffff700 1 --2- 172.21.15.89:0/2138584604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dc014cf10 0x7f1dc014d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.519+0000 7f1dbffff700 1 -- 172.21.15.89:0/2138584604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc01517c0 con 0x7f1da8004810 2023-10-10T20:46:21.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.519+0000 7f1dbffff700 1 --2- 172.21.15.89:0/2138584604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da8004810 0x7f1dc01559a0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f1db0011700 tx=0x7f1db0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:21.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.519+0000 7f1da7fff700 1 -- 172.21.15.89:0/2138584604 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db0015670 con 0x7f1da8004810 2023-10-10T20:46:21.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.519+0000 7f1da7fff700 1 -- 172.21.15.89:0/2138584604 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db00157d0 con 0x7f1da8004810 2023-10-10T20:46:21.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.519+0000 7f1dc5379700 1 -- 172.21.15.89:0/2138584604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc0151aa0 con 0x7f1da8004810 2023-10-10T20:46:21.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.519+0000 7f1da7fff700 1 -- 172.21.15.89:0/2138584604 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db0015b90 con 0x7f1da8004810 2023-10-10T20:46:21.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.519+0000 7f1dc5379700 1 -- 172.21.15.89:0/2138584604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc0152060 con 0x7f1da8004810 2023-10-10T20:46:21.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.520+0000 7f1dc5379700 1 -- 172.21.15.89:0/2138584604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc004c3c0 con 0x7f1da8004810 2023-10-10T20:46:21.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.520+0000 7f1da7fff700 1 -- 172.21.15.89:0/2138584604 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1db0013410 con 0x7f1da8004810 2023-10-10T20:46:21.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.520+0000 7f1da7fff700 1 --2- 172.21.15.89:0/2138584604 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1dac067320 0x7f1dac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:21.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.521+0000 7f1dbf7fe700 1 --2- 172.21.15.89:0/2138584604 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1dac067320 0x7f1dac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:21.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.521+0000 7f1da7fff700 1 -- 172.21.15.89:0/2138584604 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1db00850c0 con 0x7f1da8004810 2023-10-10T20:46:21.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.521+0000 7f1dbf7fe700 1 --2- 172.21.15.89:0/2138584604 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1dac067320 0x7f1dac0697d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1dc01562b0 tx=0x7f1db4015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:21.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.525+0000 7f1da7fff700 1 -- 172.21.15.89:0/2138584604 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1db0052010 con 0x7f1da8004810 2023-10-10T20:46:21.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.708+0000 7f1dc5379700 1 -- 172.21.15.89:0/2138584604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1dc00b1a70 con 0x7f1da8004810 2023-10-10T20:46:21.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.708+0000 7f1da7fff700 1 -- 172.21.15.89:0/2138584604 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v203) v1 ==== 74+0+12273 (secure 0 0 0) 0x7f1db0055630 con 0x7f1da8004810 2023-10-10T20:46:21.711 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:46:21.711 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":203,"fsid":"5dd675b8-9026-4d8a-a1bf-62fdc5e7de67","created":"2023-10-10T20:34:17.133249+0000","modified":"2023-10-10T20:46:18.206492+0000","last_up_change":"2023-10-10T20:34:27.802670+0000","last_in_change":"2023-10-10T20:34:19.532466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:34:28.693226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"824a5c96-7951-4900-849d-6d6eee4f103f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":90898},{"type":"v1","addr":"172.21.15.89:6809","nonce":90898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":90898},{"type":"v1","addr":"172.21.15.89:6811","nonce":90898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":90898},{"type":"v1","addr":"172.21.15.89:6815","nonce":90898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":90898},{"type":"v1","addr":"172.21.15.89:6813","nonce":90898}]},"public_addr":"172.21.15.89:6809/90898","cluster_addr":"172.21.15.89:6811/90898","heartbeat_back_addr":"172.21.15.89:6815/90898","heartbeat_front_addr":"172.21.15.89:6813/90898","state":["exists","up"]},{"osd":1,"uuid":"2292ce8e-5b0e-403d-a78c-1bb6ef0fab05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":90897},{"type":"v1","addr":"172.21.15.89:6817","nonce":90897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":90897},{"type":"v1","addr":"172.21.15.89:6819","nonce":90897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":90897},{"type":"v1","addr":"172.21.15.89:6827","nonce":90897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":90897},{"type":"v1","addr":"172.21.15.89:6823","nonce":90897}]},"public_addr":"172.21.15.89:6817/90897","cluster_addr":"172.21.15.89:6819/90897","heartbeat_back_addr":"172.21.15.89:6827/90897","heartbeat_front_addr":"172.21.15.89:6823/90897","state":["exists","up"]},{"osd":2,"uuid":"f8ffe4e4-71f7-4b8c-9010-d4102f63af46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":90896},{"type":"v1","addr":"172.21.15.89:6801","nonce":90896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":90896},{"type":"v1","addr":"172.21.15.89:6803","nonce":90896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":90896},{"type":"v1","addr":"172.21.15.89:6807","nonce":90896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":90896},{"type":"v1","addr":"172.21.15.89:6805","nonce":90896}]},"public_addr":"172.21.15.89:6801/90896","cluster_addr":"172.21.15.89:6803/90896","heartbeat_back_addr":"172.21.15.89:6807/90896","heartbeat_front_addr":"172.21.15.89:6805/90896","state":["exists","up"]},{"osd":3,"uuid":"f065bed3-b82e-489b-a87b-57a5d762dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":90907},{"type":"v1","addr":"172.21.15.89:6822","nonce":90907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":90907},{"type":"v1","addr":"172.21.15.89:6826","nonce":90907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":90907},{"type":"v1","addr":"172.21.15.89:6831","nonce":90907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":90907},{"type":"v1","addr":"172.21.15.89:6829","nonce":90907}]},"public_addr":"172.21.15.89:6822/90907","cluster_addr":"172.21.15.89:6826/90907","heartbeat_back_addr":"172.21.15.89:6831/90907","heartbeat_front_addr":"172.21.15.89:6829/90907","state":["exists","up"]},{"osd":4,"uuid":"a922fa5c-8390-4160-87ff-e11d324d6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":91095},{"type":"v1","addr":"172.21.15.111:6801","nonce":91095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":91095},{"type":"v1","addr":"172.21.15.111:6803","nonce":91095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":91095},{"type":"v1","addr":"172.21.15.111:6807","nonce":91095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":91095},{"type":"v1","addr":"172.21.15.111:6805","nonce":91095}]},"public_addr":"172.21.15.111:6801/91095","cluster_addr":"172.21.15.111:6803/91095","heartbeat_back_addr":"172.21.15.111:6807/91095","heartbeat_front_addr":"172.21.15.111:6805/91095","state":["exists","up"]},{"osd":5,"uuid":"94d1f69c-355e-430c-afae-15ba87b76ad7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":91121},{"type":"v1","addr":"172.21.15.111:6817","nonce":91121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":91121},{"type":"v1","addr":"172.21.15.111:6819","nonce":91121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":91121},{"type":"v1","addr":"172.21.15.111:6823","nonce":91121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":91121},{"type":"v1","addr":"172.21.15.111:6821","nonce":91121}]},"public_addr":"172.21.15.111:6817/91121","cluster_addr":"172.21.15.111:6819/91121","heartbeat_back_addr":"172.21.15.111:6823/91121","heartbeat_front_addr":"172.21.15.111:6821/91121","state":["exists","up"]},{"osd":6,"uuid":"d69d657c-9b34-4e9c-a47a-9449d226bfc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":91096},{"type":"v1","addr":"172.21.15.111:6809","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":91096},{"type":"v1","addr":"172.21.15.111:6811","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":91096},{"type":"v1","addr":"172.21.15.111:6815","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":91096},{"type":"v1","addr":"172.21.15.111:6813","nonce":91096}]},"public_addr":"172.21.15.111:6809/91096","cluster_addr":"172.21.15.111:6811/91096","heartbeat_back_addr":"172.21.15.111:6815/91096","heartbeat_front_addr":"172.21.15.111:6813/91096","state":["exists","up"]},{"osd":7,"uuid":"956c4fea-bdfd-455a-a107-7b58155631ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":91126},{"type":"v1","addr":"172.21.15.111:6825","nonce":91126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":91126},{"type":"v1","addr":"172.21.15.111:6827","nonce":91126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":91126},{"type":"v1","addr":"172.21.15.111:6831","nonce":91126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":91126},{"type":"v1","addr":"172.21.15.111:6829","nonce":91126}]},"public_addr":"172.21.15.111:6825/91126","cluster_addr":"172.21.15.111:6827/91126","heartbeat_back_addr":"172.21.15.111:6831/91126","heartbeat_front_addr":"172.21.15.111:6829/91126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.648334+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.689347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.725671+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.710977+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.429647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.720385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.679564+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:34:24.698096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6836/3558240892":"2023-10-11T20:46:14.857266+0000","172.21.15.89:6834/3558240892":"2023-10-11T20:46:14.857266+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:46:21.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.711+0000 7f1dc5379700 1 -- 172.21.15.89:0/2138584604 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1dac067320 msgr2=0x7f1dac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:21.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.711+0000 7f1dc5379700 1 --2- 172.21.15.89:0/2138584604 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1dac067320 0x7f1dac0697d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1dc01562b0 tx=0x7f1db4015000 comp rx=0 tx=0).stop 2023-10-10T20:46:21.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.711+0000 7f1dc5379700 1 -- 172.21.15.89:0/2138584604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da8004810 msgr2=0x7f1dc01559a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:21.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.711+0000 7f1dc5379700 1 --2- 172.21.15.89:0/2138584604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da8004810 0x7f1dc01559a0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f1db0011700 tx=0x7f1db0008040 comp rx=0 tx=0).stop 2023-10-10T20:46:21.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.711+0000 7f1dc5379700 1 -- 172.21.15.89:0/2138584604 shutdown_connections 2023-10-10T20:46:21.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.711+0000 7f1dc5379700 1 --2- 172.21.15.89:0/2138584604 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f1dc015c2c0 0x7f1dc015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.711+0000 7f1dc5379700 1 --2- 172.21.15.89:0/2138584604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da8004810 0x7f1dc01559a0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.711+0000 7f1dc5379700 1 --2- 172.21.15.89:0/2138584604 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f1dac067320 0x7f1dac0697d0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.711+0000 7f1dc5379700 1 --2- 172.21.15.89:0/2138584604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dc014cf10 0x7f1dc014d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:21.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.711+0000 7f1dc5379700 1 -- 172.21.15.89:0/2138584604 >> 172.21.15.89:0/2138584604 conn(0x7f1dc00d0660 msgr2=0x7f1dc00ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:21.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.711+0000 7f1dc5379700 1 -- 172.21.15.89:0/2138584604 shutdown_connections 2023-10-10T20:46:21.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:21.711+0000 7f1dc5379700 1 -- 172.21.15.89:0/2138584604 wait complete. 2023-10-10T20:46:22.725 INFO:tasks.ceph:Scrubbing osd.0 2023-10-10T20:46:22.726 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-10T20:46:22.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.848+0000 7f317b59e700 1 Processor -- start 2023-10-10T20:46:22.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.848+0000 7f317b59e700 1 -- start start 2023-10-10T20:46:22.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.849+0000 7f317b59e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f317c0b0790 0x7f317c0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:22.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.849+0000 7f317b59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f317c1591e0 0x7f317c159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:22.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.849+0000 7f317b59e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f317c0cfa00 con 0x7f317c0b03c0 2023-10-10T20:46:22.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.849+0000 7f317b59e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f317c0cb580 con 0x7f317c0b0790 2023-10-10T20:46:22.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.849+0000 7f317b59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317c0cf160 con 0x7f317c1591e0 2023-10-10T20:46:22.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.849+0000 7f317ad9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f317c1591e0 0x7f317c159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:22.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.849+0000 7f3179d9b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f317c0b0790 0x7f317c0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:22.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.849+0000 7f317ad9d700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f317c1591e0 0x7f317c159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:36708/0 (socket says 172.21.15.89:36708) 2023-10-10T20:46:22.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.849+0000 7f317ad9d700 1 -- 172.21.15.89:0/743438143 learned_addr learned my addr 172.21.15.89:0/743438143 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:22.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.849+0000 7f3179d9b700 1 -- 172.21.15.89:0/743438143 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f317c1591e0 msgr2=0x7f317c159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:22.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.849+0000 7f3179d9b700 1 --2- 172.21.15.89:0/743438143 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f317c1591e0 0x7f317c159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:22.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.849+0000 7f3179d9b700 1 -- 172.21.15.89:0/743438143 >> v1:172.21.15.111:6789/0 conn(0x7f317c0b03c0 legacy=0x7f317c0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:22.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.849+0000 7f3179d9b700 1 -- 172.21.15.89:0/743438143 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317c0cf440 con 0x7f317c0b0790 2023-10-10T20:46:22.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.849+0000 7f317ad9d700 1 --2- 172.21.15.89:0/743438143 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f317c1591e0 0x7f317c159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:46:22.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.850+0000 7f3179d9b700 1 --2- 172.21.15.89:0/743438143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f317c0b0790 0x7f317c0cea70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3164009910 tx=0x7f31640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43aa1a1d1a89af63 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:22.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.850+0000 7f317959a700 1 -- 172.21.15.89:0/743438143 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316400b640 con 0x7f317c0b0790 2023-10-10T20:46:22.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.850+0000 7f317959a700 1 -- 172.21.15.89:0/743438143 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316400b7a0 con 0x7f317c0b0790 2023-10-10T20:46:22.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.850+0000 7f317959a700 1 -- 172.21.15.89:0/743438143 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316400bb60 con 0x7f317c0b0790 2023-10-10T20:46:22.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.850+0000 7f317b59e700 1 -- 172.21.15.89:0/743438143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f317c0b0790 msgr2=0x7f317c0cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:22.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.850+0000 7f317b59e700 1 --2- 172.21.15.89:0/743438143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f317c0b0790 0x7f317c0cea70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3164009910 tx=0x7f31640091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:22.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.850+0000 7f317b59e700 1 -- 172.21.15.89:0/743438143 shutdown_connections 2023-10-10T20:46:22.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.850+0000 7f317b59e700 1 --2- 172.21.15.89:0/743438143 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f317c1591e0 0x7f317c159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:22.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.850+0000 7f317b59e700 1 --2- 172.21.15.89:0/743438143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f317c0b0790 0x7f317c0cea70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:22.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.850+0000 7f317b59e700 1 -- 172.21.15.89:0/743438143 >> 172.21.15.89:0/743438143 conn(0x7f317c063230 msgr2=0x7f317c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:22.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.850+0000 7f317b59e700 1 -- 172.21.15.89:0/743438143 shutdown_connections 2023-10-10T20:46:22.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.850+0000 7f317b59e700 1 -- 172.21.15.89:0/743438143 wait complete. 2023-10-10T20:46:22.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.851+0000 7f317b59e700 1 Processor -- start 2023-10-10T20:46:22.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.852+0000 7f317b59e700 1 -- start start 2023-10-10T20:46:22.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.852+0000 7f317b59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f317c0b03c0 0x7f317c16aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:22.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.852+0000 7f317b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f317c0b0790 0x7f317c157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:22.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.852+0000 7f317b59e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f317c1591e0 0x7f317c0c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:22.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.852+0000 7f317b59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317c16e300 con 0x7f317c0b0790 2023-10-10T20:46:22.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.852+0000 7f317b59e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317c16af50 con 0x7f317c1591e0 2023-10-10T20:46:22.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.852+0000 7f317b59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317c1580e0 con 0x7f317c0b03c0 2023-10-10T20:46:22.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.852+0000 7f3179d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f317c0b0790 0x7f317c157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:22.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.852+0000 7f3179d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f317c0b0790 0x7f317c157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43560/0 (socket says 172.21.15.89:43560) 2023-10-10T20:46:22.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.852+0000 7f3179d9b700 1 -- 172.21.15.89:0/1756613415 learned_addr learned my addr 172.21.15.89:0/1756613415 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:22.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.853+0000 7f317ad9d700 1 --2- 172.21.15.89:0/1756613415 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f317c1591e0 0x7f317c0c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:22.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.853+0000 7f317a59c700 1 --2- 172.21.15.89:0/1756613415 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f317c0b03c0 0x7f317c16aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:22.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.853+0000 7f3179d9b700 1 -- 172.21.15.89:0/1756613415 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f317c0b03c0 msgr2=0x7f317c16aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:22.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.853+0000 7f3179d9b700 1 --2- 172.21.15.89:0/1756613415 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f317c0b03c0 0x7f317c16aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:22.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.853+0000 7f3179d9b700 1 -- 172.21.15.89:0/1756613415 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f317c1591e0 msgr2=0x7f317c0c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:22.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.853+0000 7f3179d9b700 1 --2- 172.21.15.89:0/1756613415 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f317c1591e0 0x7f317c0c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:22.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.853+0000 7f3179d9b700 1 -- 172.21.15.89:0/1756613415 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31740094f0 con 0x7f317c0b0790 2023-10-10T20:46:22.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.853+0000 7f3179d9b700 1 --2- 172.21.15.89:0/1756613415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f317c0b0790 0x7f317c157ba0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f3164009fb0 tx=0x7f3164017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:22.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.853+0000 7f31727fc700 1 -- 172.21.15.89:0/1756613415 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316401f070 con 0x7f317c0b0790 2023-10-10T20:46:22.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.853+0000 7f31727fc700 1 -- 172.21.15.89:0/1756613415 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316400bb20 con 0x7f317c0b0790 2023-10-10T20:46:22.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.853+0000 7f31727fc700 1 -- 172.21.15.89:0/1756613415 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164011670 con 0x7f317c0b0790 2023-10-10T20:46:22.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.853+0000 7f317b59e700 1 -- 172.21.15.89:0/1756613415 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3164009420 con 0x7f317c0b0790 2023-10-10T20:46:22.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.853+0000 7f317b59e700 1 -- 172.21.15.89:0/1756613415 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317c0c70b0 con 0x7f317c0b0790 2023-10-10T20:46:22.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.854+0000 7f317b59e700 1 -- 172.21.15.89:0/1756613415 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f317c04c3c0 con 0x7f317c0b0790 2023-10-10T20:46:22.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.855+0000 7f31727fc700 1 -- 172.21.15.89:0/1756613415 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f31640119e0 con 0x7f317c0b0790 2023-10-10T20:46:22.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.855+0000 7f31727fc700 1 --2- 172.21.15.89:0/1756613415 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f315c067320 0x7f315c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:22.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.855+0000 7f31727fc700 1 -- 172.21.15.89:0/1756613415 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3164085a70 con 0x7f317c0b0790 2023-10-10T20:46:22.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.855+0000 7f317a59c700 1 --2- 172.21.15.89:0/1756613415 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f315c067320 0x7f315c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:22.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.855+0000 7f31727fc700 1 --2- 172.21.15.89:0/1756613415 >> [v2:172.21.15.89:6808/90898,v1:172.21.15.89:6809/90898] conn(0x7f315c06d170 0x7f315c06f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:22.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.855+0000 7f31727fc700 1 -- 172.21.15.89:0/1756613415 --> [v2:172.21.15.89:6808/90898,v1:172.21.15.89:6809/90898] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f315c06fc50 con 0x7f315c06d170 2023-10-10T20:46:22.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.855+0000 7f31727fc700 1 -- 172.21.15.89:0/1756613415 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3164085d80 con 0x7f317c0b0790 2023-10-10T20:46:22.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.855+0000 7f317ad9d700 1 --2- 172.21.15.89:0/1756613415 >> [v2:172.21.15.89:6808/90898,v1:172.21.15.89:6809/90898] conn(0x7f315c06d170 0x7f315c06f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:22.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.855+0000 7f317a59c700 1 --2- 172.21.15.89:0/1756613415 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f315c067320 0x7f315c0697d0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f316c007fd0 tx=0x7f316c001650 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:22.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.855+0000 7f317ad9d700 1 --2- 172.21.15.89:0/1756613415 >> [v2:172.21.15.89:6808/90898,v1:172.21.15.89:6809/90898] conn(0x7f315c06d170 0x7f315c06f580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:22.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.856+0000 7f31727fc700 1 -- 172.21.15.89:0/1756613415 <== osd.0 v2:172.21.15.89:6808/90898 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f315c06fc50 con 0x7f315c06d170 2023-10-10T20:46:22.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.870+0000 7f317b59e700 1 -- 172.21.15.89:0/1756613415 --> [v2:172.21.15.89:6808/90898,v1:172.21.15.89:6809/90898] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f317c0c70b0 con 0x7f315c06d170 2023-10-10T20:46:22.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.870+0000 7f31727fc700 1 -- 172.21.15.89:0/1756613415 <== osd.0 v2:172.21.15.89:6808/90898 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f317c0c70b0 con 0x7f315c06d170 2023-10-10T20:46:22.877 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:46:22.877 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-10T20:46:22.878 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:46:22.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.870+0000 7f317b59e700 1 -- 172.21.15.89:0/1756613415 >> [v2:172.21.15.89:6808/90898,v1:172.21.15.89:6809/90898] conn(0x7f315c06d170 msgr2=0x7f315c06f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:22.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.870+0000 7f317b59e700 1 --2- 172.21.15.89:0/1756613415 >> [v2:172.21.15.89:6808/90898,v1:172.21.15.89:6809/90898] conn(0x7f315c06d170 0x7f315c06f580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:22.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.870+0000 7f317b59e700 1 -- 172.21.15.89:0/1756613415 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f315c067320 msgr2=0x7f315c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:22.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.870+0000 7f317b59e700 1 --2- 172.21.15.89:0/1756613415 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f315c067320 0x7f315c0697d0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f316c007fd0 tx=0x7f316c001650 comp rx=0 tx=0).stop 2023-10-10T20:46:22.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.870+0000 7f317b59e700 1 -- 172.21.15.89:0/1756613415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f317c0b0790 msgr2=0x7f317c157ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:22.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.870+0000 7f317b59e700 1 --2- 172.21.15.89:0/1756613415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f317c0b0790 0x7f317c157ba0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f3164009fb0 tx=0x7f3164017000 comp rx=0 tx=0).stop 2023-10-10T20:46:22.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.870+0000 7f317ad9d700 1 -- 172.21.15.89:0/1756613415 reap_dead start 2023-10-10T20:46:22.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.870+0000 7f317b59e700 1 -- 172.21.15.89:0/1756613415 shutdown_connections 2023-10-10T20:46:22.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.870+0000 7f317b59e700 1 -- 172.21.15.89:0/1756613415 >> 172.21.15.89:0/1756613415 conn(0x7f317c063230 msgr2=0x7f317c0b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:22.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.870+0000 7f317b59e700 1 -- 172.21.15.89:0/1756613415 shutdown_connections 2023-10-10T20:46:22.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.870+0000 7f317b59e700 1 -- 172.21.15.89:0/1756613415 wait complete. 2023-10-10T20:46:22.883 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-10T20:46:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.998+0000 7f798a944700 1 Processor -- start 2023-10-10T20:46:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.998+0000 7f798a944700 1 -- start start 2023-10-10T20:46:23.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.999+0000 7f798a944700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f79841597f0 0x7f79840b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.999+0000 7f798a944700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79840b11d0 con 0x7f798415d230 2023-10-10T20:46:23.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.999+0000 7f798a944700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79840b14b0 con 0x7f7984159420 2023-10-10T20:46:23.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.999+0000 7f798a944700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79840b1790 con 0x7f79841597f0 2023-10-10T20:46:23.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.999+0000 7f798a143700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f798415d230 0x7f79840b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51716/0 (socket says 172.21.15.89:51716) 2023-10-10T20:46:23.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.999+0000 7f798a143700 1 -- 172.21.15.89:0/723687992 learned_addr learned my addr 172.21.15.89:0/723687992 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:23.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.999+0000 7f7989141700 1 --2- 172.21.15.89:0/723687992 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f79841597f0 0x7f79840b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.999+0000 7f7988940700 1 -- 172.21.15.89:0/723687992 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 259378099 0 0) 0x7f79840b11d0 con 0x7f798415d230 2023-10-10T20:46:23.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:22.999+0000 7f7988940700 1 -- 172.21.15.89:0/723687992 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7970003570 con 0x7f798415d230 2023-10-10T20:46:23.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.000+0000 7f7988940700 1 -- 172.21.15.89:0/723687992 <== mon.1 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1740049422 0 0) 0x7f79840b14b0 con 0x7f7984159420 2023-10-10T20:46:23.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.000+0000 7f7988940700 1 -- 172.21.15.89:0/723687992 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79840b11d0 con 0x7f7984159420 2023-10-10T20:46:23.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.000+0000 7f7989141700 1 -- 172.21.15.89:0/723687992 >> v1:172.21.15.111:6789/0 conn(0x7f7984159420 legacy=0x7f79840b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.000+0000 7f7989141700 1 -- 172.21.15.89:0/723687992 >> v1:172.21.15.89:6789/0 conn(0x7f798415d230 legacy=0x7f79840b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.000+0000 7f7988940700 1 -- 172.21.15.89:0/723687992 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3889494092 0 0) 0x7f7970003570 con 0x7f798415d230 2023-10-10T20:46:23.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.000+0000 7f7989141700 1 -- 172.21.15.89:0/723687992 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79840b1a70 con 0x7f79841597f0 2023-10-10T20:46:23.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.000+0000 7f7989141700 1 --2- 172.21.15.89:0/723687992 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f79841597f0 0x7f79840b2270 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7974009910 tx=0x7f79740091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a87577b8ce531b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:23.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.000+0000 7f7988940700 1 -- 172.21.15.89:0/723687992 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c001470 con 0x7f79841597f0 2023-10-10T20:46:23.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.000+0000 7f7988940700 1 -- 172.21.15.89:0/723687992 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7974011420 con 0x7f79841597f0 2023-10-10T20:46:23.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.000+0000 7f7988940700 1 -- 172.21.15.89:0/723687992 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79740117c0 con 0x7f79841597f0 2023-10-10T20:46:23.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.001+0000 7f798a944700 1 -- 172.21.15.89:0/723687992 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f79841597f0 msgr2=0x7f79840b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.001+0000 7f798a944700 1 --2- 172.21.15.89:0/723687992 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f79841597f0 0x7f79840b2270 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7974009910 tx=0x7f79740091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.001+0000 7f798a944700 1 -- 172.21.15.89:0/723687992 shutdown_connections 2023-10-10T20:46:23.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.001+0000 7f798a944700 1 --2- 172.21.15.89:0/723687992 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f79841597f0 0x7f79840b2270 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.001+0000 7f798a944700 1 -- 172.21.15.89:0/723687992 >> 172.21.15.89:0/723687992 conn(0x7f79840d0660 msgr2=0x7f79840d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:23.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.001+0000 7f798a944700 1 -- 172.21.15.89:0/723687992 shutdown_connections 2023-10-10T20:46:23.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.001+0000 7f798a944700 1 -- 172.21.15.89:0/723687992 wait complete. 2023-10-10T20:46:23.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.002+0000 7f798a944700 1 Processor -- start 2023-10-10T20:46:23.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.002+0000 7f798a944700 1 -- start start 2023-10-10T20:46:23.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.002+0000 7f798a944700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7984159420 0x7f79840c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.002+0000 7f798a944700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79841597f0 0x7f79840c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.002+0000 7f798a944700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f798415d230 0x7f7984154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.002+0000 7f798a944700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79840c67d0 con 0x7f798415d230 2023-10-10T20:46:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.002+0000 7f798a944700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79840c9bc0 con 0x7f79841597f0 2023-10-10T20:46:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.002+0000 7f798a944700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79840c7030 con 0x7f7984159420 2023-10-10T20:46:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.002+0000 7f798a143700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f798415d230 0x7f7984154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.003+0000 7f798a143700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f798415d230 0x7f7984154460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43568/0 (socket says 172.21.15.89:43568) 2023-10-10T20:46:23.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.003+0000 7f798a143700 1 -- 172.21.15.89:0/3679568884 learned_addr learned my addr 172.21.15.89:0/3679568884 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:23.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.003+0000 7f7989141700 1 --2- 172.21.15.89:0/3679568884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79841597f0 0x7f79840c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.003+0000 7f7989942700 1 --2- 172.21.15.89:0/3679568884 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7984159420 0x7f79840c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.003+0000 7f798a143700 1 -- 172.21.15.89:0/3679568884 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7984159420 msgr2=0x7f79840c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.003+0000 7f798a143700 1 --2- 172.21.15.89:0/3679568884 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7984159420 0x7f79840c6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.003+0000 7f798a143700 1 -- 172.21.15.89:0/3679568884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79841597f0 msgr2=0x7f79840c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.003+0000 7f798a143700 1 --2- 172.21.15.89:0/3679568884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79841597f0 0x7f79840c9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.003+0000 7f798a143700 1 -- 172.21.15.89:0/3679568884 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7974009420 con 0x7f798415d230 2023-10-10T20:46:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.003+0000 7f798a143700 1 --2- 172.21.15.89:0/3679568884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f798415d230 0x7f7984154460 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f797c00c810 tx=0x7f797c001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.003+0000 7f7979ffb700 1 -- 172.21.15.89:0/3679568884 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c007360 con 0x7f798415d230 2023-10-10T20:46:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.003+0000 7f7979ffb700 1 -- 172.21.15.89:0/3679568884 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797c007500 con 0x7f798415d230 2023-10-10T20:46:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.003+0000 7f798a944700 1 -- 172.21.15.89:0/3679568884 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79841549a0 con 0x7f798415d230 2023-10-10T20:46:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.003+0000 7f7979ffb700 1 -- 172.21.15.89:0/3679568884 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c004a80 con 0x7f798415d230 2023-10-10T20:46:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.003+0000 7f798a944700 1 -- 172.21.15.89:0/3679568884 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f798416dc70 con 0x7f798415d230 2023-10-10T20:46:23.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.004+0000 7f798a944700 1 -- 172.21.15.89:0/3679568884 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f798404c3c0 con 0x7f798415d230 2023-10-10T20:46:23.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.005+0000 7f7979ffb700 1 -- 172.21.15.89:0/3679568884 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f797c01c3b0 con 0x7f798415d230 2023-10-10T20:46:23.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.005+0000 7f7979ffb700 1 --2- 172.21.15.89:0/3679568884 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7968067320 0x7f79680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.005+0000 7f7979ffb700 1 -- 172.21.15.89:0/3679568884 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f797c084170 con 0x7f798415d230 2023-10-10T20:46:23.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.005+0000 7f7989942700 1 --2- 172.21.15.89:0/3679568884 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7968067320 0x7f79680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.005+0000 7f7989942700 1 --2- 172.21.15.89:0/3679568884 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7968067320 0x7f79680697d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7980007fd0 tx=0x7f7980009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:23.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.009+0000 7f7979ffb700 1 -- 172.21.15.89:0/3679568884 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f797c051180 con 0x7f798415d230 2023-10-10T20:46:23.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.191+0000 7f798a944700 1 -- 172.21.15.89:0/3679568884 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f79840b1d50 con 0x7f7968067320 2023-10-10T20:46:23.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.191+0000 7f7979ffb700 1 -- 172.21.15.89:0/3679568884 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f79840b1d50 con 0x7f7968067320 2023-10-10T20:46:23.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.194+0000 7f798a944700 1 -- 172.21.15.89:0/3679568884 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7968067320 msgr2=0x7f79680697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.194+0000 7f798a944700 1 --2- 172.21.15.89:0/3679568884 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7968067320 0x7f79680697d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7980007fd0 tx=0x7f7980009040 comp rx=0 tx=0).stop 2023-10-10T20:46:23.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.194+0000 7f798a944700 1 -- 172.21.15.89:0/3679568884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f798415d230 msgr2=0x7f7984154460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.194+0000 7f798a944700 1 --2- 172.21.15.89:0/3679568884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f798415d230 0x7f7984154460 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f797c00c810 tx=0x7f797c001a30 comp rx=0 tx=0).stop 2023-10-10T20:46:23.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.194+0000 7f798a944700 1 -- 172.21.15.89:0/3679568884 shutdown_connections 2023-10-10T20:46:23.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.194+0000 7f798a944700 1 --2- 172.21.15.89:0/3679568884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f798415d230 0x7f7984154460 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.194+0000 7f798a944700 1 --2- 172.21.15.89:0/3679568884 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f7984159420 0x7f79840c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.194+0000 7f798a944700 1 --2- 172.21.15.89:0/3679568884 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f7968067320 0x7f79680697d0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.194+0000 7f798a944700 1 --2- 172.21.15.89:0/3679568884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79841597f0 0x7f79840c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.194+0000 7f798a944700 1 -- 172.21.15.89:0/3679568884 >> 172.21.15.89:0/3679568884 conn(0x7f79840d0660 msgr2=0x7f79841552b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:23.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.194+0000 7f798a944700 1 -- 172.21.15.89:0/3679568884 shutdown_connections 2023-10-10T20:46:23.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.194+0000 7f798a944700 1 -- 172.21.15.89:0/3679568884 wait complete. 2023-10-10T20:46:23.200 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 0 to deep-scrub 2023-10-10T20:46:23.207 INFO:tasks.ceph:Scrubbing osd.1 2023-10-10T20:46:23.208 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-10T20:46:23.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.325+0000 7f37b2c5a700 1 Processor -- start 2023-10-10T20:46:23.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.325+0000 7f37b2c5a700 1 -- start start 2023-10-10T20:46:23.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.326+0000 7f37b2c5a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f37ac0b2100 0x7f37ac068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.326+0000 7f37b2c5a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37ac0688d0 0x7f37ac06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.326+0000 7f37b2c5a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac0ae6d0 con 0x7f37ac157740 2023-10-10T20:46:23.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.326+0000 7f37b2c5a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ac0acc00 con 0x7f37ac0688d0 2023-10-10T20:46:23.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.326+0000 7f37b2c5a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ac06d6a0 con 0x7f37ac0b2100 2023-10-10T20:46:23.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.326+0000 7f37b09f6700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f37ac157740 0x7f37ac0adfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51728/0 (socket says 172.21.15.89:51728) 2023-10-10T20:46:23.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.326+0000 7f37b11f7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37ac0688d0 0x7f37ac06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.326+0000 7f37b09f6700 1 -- 172.21.15.89:0/1947993871 learned_addr learned my addr 172.21.15.89:0/1947993871 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.327+0000 7f37a3fff700 1 --2- 172.21.15.89:0/1947993871 >> v2:172.21.15.111:3300/0 conn(0x7f37ac0b2100 0x7f37ac068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.327+0000 7f37a37fe700 1 -- 172.21.15.89:0/1947993871 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462849888 0 0) 0x7f37a8001490 con 0x7f37ac157740 2023-10-10T20:46:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.327+0000 7f37a37fe700 1 -- 172.21.15.89:0/1947993871 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3794003570 con 0x7f37ac157740 2023-10-10T20:46:23.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.327+0000 7f37b11f7700 1 -- 172.21.15.89:0/1947993871 >> v2:172.21.15.111:3300/0 conn(0x7f37ac0b2100 msgr2=0x7f37ac068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:23.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.327+0000 7f37b11f7700 1 --2- 172.21.15.89:0/1947993871 >> v2:172.21.15.111:3300/0 conn(0x7f37ac0b2100 0x7f37ac068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.327+0000 7f37b11f7700 1 -- 172.21.15.89:0/1947993871 >> v1:172.21.15.89:6789/0 conn(0x7f37ac157740 legacy=0x7f37ac0adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.327+0000 7f37b11f7700 1 -- 172.21.15.89:0/1947993871 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac068d30 con 0x7f37ac0688d0 2023-10-10T20:46:23.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.327+0000 7f37b11f7700 1 --2- 172.21.15.89:0/1947993871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ac0688d0 0x7f37ac06e0b0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f37ac06d7e0 tx=0x7f37a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7575df6948e5c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:23.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.327+0000 7f37a37fe700 1 -- 172.21.15.89:0/1947993871 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4011630 con 0x7f37ac0688d0 2023-10-10T20:46:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.327+0000 7f37a37fe700 1 -- 172.21.15.89:0/1947993871 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a4011790 con 0x7f37ac0688d0 2023-10-10T20:46:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.327+0000 7f37a37fe700 1 -- 172.21.15.89:0/1947993871 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4011b50 con 0x7f37ac0688d0 2023-10-10T20:46:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.327+0000 7f37b2c5a700 1 -- 172.21.15.89:0/1947993871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ac0688d0 msgr2=0x7f37ac06e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.327+0000 7f37b2c5a700 1 --2- 172.21.15.89:0/1947993871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ac0688d0 0x7f37ac06e0b0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f37ac06d7e0 tx=0x7f37a40091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.328+0000 7f37b2c5a700 1 -- 172.21.15.89:0/1947993871 shutdown_connections 2023-10-10T20:46:23.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.328+0000 7f37b2c5a700 1 --2- 172.21.15.89:0/1947993871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ac0688d0 0x7f37ac06e0b0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.328+0000 7f37b2c5a700 1 --2- 172.21.15.89:0/1947993871 >> v2:172.21.15.111:3300/0 conn(0x7f37ac0b2100 0x7f37ac068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.328+0000 7f37b2c5a700 1 -- 172.21.15.89:0/1947993871 >> 172.21.15.89:0/1947993871 conn(0x7f37ac0c8f90 msgr2=0x7f37ac0c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:23.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.328+0000 7f37b2c5a700 1 -- 172.21.15.89:0/1947993871 shutdown_connections 2023-10-10T20:46:23.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.328+0000 7f37b2c5a700 1 -- 172.21.15.89:0/1947993871 wait complete. 2023-10-10T20:46:23.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.329+0000 7f37b2c5a700 1 Processor -- start 2023-10-10T20:46:23.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.329+0000 7f37b2c5a700 1 -- start start 2023-10-10T20:46:23.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37b2c5a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ac0688d0 0x7f37ac149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37b2c5a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37ac0b2100 0x7f37ac15c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37b2c5a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37ac157740 0x7f37ac0c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37b2c5a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac0c4070 con 0x7f37ac0688d0 2023-10-10T20:46:23.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37b2c5a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac0c4350 con 0x7f37ac157740 2023-10-10T20:46:23.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37b2c5a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac0c4630 con 0x7f37ac0b2100 2023-10-10T20:46:23.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37b09f6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ac0688d0 0x7f37ac149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37b09f6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ac0688d0 0x7f37ac149a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43580/0 (socket says 172.21.15.89:43580) 2023-10-10T20:46:23.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37b09f6700 1 -- 172.21.15.89:0/2189776941 learned_addr learned my addr 172.21.15.89:0/2189776941 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:23.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37a3fff700 1 --2- 172.21.15.89:0/2189776941 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37ac0b2100 0x7f37ac15c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37b11f7700 1 --2- 172.21.15.89:0/2189776941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37ac157740 0x7f37ac0c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37b09f6700 1 -- 172.21.15.89:0/2189776941 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37ac0b2100 msgr2=0x7f37ac15c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37b09f6700 1 --2- 172.21.15.89:0/2189776941 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37ac0b2100 0x7f37ac15c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37b09f6700 1 -- 172.21.15.89:0/2189776941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37ac157740 msgr2=0x7f37ac0c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37b09f6700 1 --2- 172.21.15.89:0/2189776941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37ac157740 0x7f37ac0c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37b09f6700 1 -- 172.21.15.89:0/2189776941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a4009420 con 0x7f37ac0688d0 2023-10-10T20:46:23.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37a3fff700 1 --2- 172.21.15.89:0/2189776941 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f37ac0b2100 0x7f37ac15c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:23.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37b09f6700 1 --2- 172.21.15.89:0/2189776941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ac0688d0 0x7f37ac149a00 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f37a800cd30 tx=0x7f37a8001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:23.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37a0ff9700 1 -- 172.21.15.89:0/2189776941 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a800ab70 con 0x7f37ac0688d0 2023-10-10T20:46:23.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37a0ff9700 1 -- 172.21.15.89:0/2189776941 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a800acf0 con 0x7f37ac0688d0 2023-10-10T20:46:23.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37a0ff9700 1 -- 172.21.15.89:0/2189776941 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8007340 con 0x7f37ac0688d0 2023-10-10T20:46:23.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37b2c5a700 1 -- 172.21.15.89:0/2189776941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ac0c4910 con 0x7f37ac0688d0 2023-10-10T20:46:23.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.330+0000 7f37b2c5a700 1 -- 172.21.15.89:0/2189776941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac149f40 con 0x7f37ac0688d0 2023-10-10T20:46:23.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.331+0000 7f37b2c5a700 1 -- 172.21.15.89:0/2189776941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f37ac0718b0 con 0x7f37ac0688d0 2023-10-10T20:46:23.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.332+0000 7f37a0ff9700 1 -- 172.21.15.89:0/2189776941 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f37a8007770 con 0x7f37ac0688d0 2023-10-10T20:46:23.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.332+0000 7f37a0ff9700 1 --2- 172.21.15.89:0/2189776941 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3798067320 0x7f37980697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.332+0000 7f37a0ff9700 1 -- 172.21.15.89:0/2189776941 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f37a8084cd0 con 0x7f37ac0688d0 2023-10-10T20:46:23.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.332+0000 7f37a3fff700 1 --2- 172.21.15.89:0/2189776941 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3798067320 0x7f37980697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.332+0000 7f37a0ff9700 1 --2- 172.21.15.89:0/2189776941 >> [v2:172.21.15.89:6816/90897,v1:172.21.15.89:6817/90897] conn(0x7f379806d170 0x7f379806f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.332+0000 7f37a0ff9700 1 -- 172.21.15.89:0/2189776941 --> [v2:172.21.15.89:6816/90897,v1:172.21.15.89:6817/90897] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f379806fc50 con 0x7f379806d170 2023-10-10T20:46:23.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.332+0000 7f37a0ff9700 1 -- 172.21.15.89:0/2189776941 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37a8089220 con 0x7f37ac0688d0 2023-10-10T20:46:23.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.332+0000 7f37b11f7700 1 --2- 172.21.15.89:0/2189776941 >> [v2:172.21.15.89:6816/90897,v1:172.21.15.89:6817/90897] conn(0x7f379806d170 0x7f379806f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.332+0000 7f37a3fff700 1 --2- 172.21.15.89:0/2189776941 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3798067320 0x7f37980697d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f37ac16bf60 tx=0x7f379c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:23.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.333+0000 7f37b11f7700 1 --2- 172.21.15.89:0/2189776941 >> [v2:172.21.15.89:6816/90897,v1:172.21.15.89:6817/90897] conn(0x7f379806d170 0x7f379806f580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:23.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.333+0000 7f37a0ff9700 1 -- 172.21.15.89:0/2189776941 <== osd.1 v2:172.21.15.89:6816/90897 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f379806fc50 con 0x7f379806d170 2023-10-10T20:46:23.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.347+0000 7f37b2c5a700 1 -- 172.21.15.89:0/2189776941 --> [v2:172.21.15.89:6816/90897,v1:172.21.15.89:6817/90897] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f37ac0ae6d0 con 0x7f379806d170 2023-10-10T20:46:23.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.347+0000 7f37a0ff9700 1 -- 172.21.15.89:0/2189776941 <== osd.1 v2:172.21.15.89:6816/90897 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f37ac0ae6d0 con 0x7f379806d170 2023-10-10T20:46:23.358 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:46:23.358 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-10T20:46:23.358 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:46:23.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.347+0000 7f37b2c5a700 1 -- 172.21.15.89:0/2189776941 >> [v2:172.21.15.89:6816/90897,v1:172.21.15.89:6817/90897] conn(0x7f379806d170 msgr2=0x7f379806f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.347+0000 7f37b2c5a700 1 --2- 172.21.15.89:0/2189776941 >> [v2:172.21.15.89:6816/90897,v1:172.21.15.89:6817/90897] conn(0x7f379806d170 0x7f379806f580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.347+0000 7f37b2c5a700 1 -- 172.21.15.89:0/2189776941 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3798067320 msgr2=0x7f37980697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.347+0000 7f37b2c5a700 1 --2- 172.21.15.89:0/2189776941 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f3798067320 0x7f37980697d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f37ac16bf60 tx=0x7f379c008040 comp rx=0 tx=0).stop 2023-10-10T20:46:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.347+0000 7f37b2c5a700 1 -- 172.21.15.89:0/2189776941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ac0688d0 msgr2=0x7f37ac149a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.347+0000 7f37b2c5a700 1 --2- 172.21.15.89:0/2189776941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37ac0688d0 0x7f37ac149a00 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f37a800cd30 tx=0x7f37a8001b10 comp rx=0 tx=0).stop 2023-10-10T20:46:23.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.347+0000 7f37b11f7700 1 -- 172.21.15.89:0/2189776941 reap_dead start 2023-10-10T20:46:23.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.347+0000 7f37b2c5a700 1 -- 172.21.15.89:0/2189776941 shutdown_connections 2023-10-10T20:46:23.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.347+0000 7f37b2c5a700 1 -- 172.21.15.89:0/2189776941 >> 172.21.15.89:0/2189776941 conn(0x7f37ac0c8f90 msgr2=0x7f37ac0b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:23.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.347+0000 7f37b2c5a700 1 -- 172.21.15.89:0/2189776941 shutdown_connections 2023-10-10T20:46:23.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.347+0000 7f37b2c5a700 1 -- 172.21.15.89:0/2189776941 wait complete. 2023-10-10T20:46:23.366 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-10T20:46:23.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.480+0000 7f5132035700 1 Processor -- start 2023-10-10T20:46:23.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.480+0000 7f5132035700 1 -- start start 2023-10-10T20:46:23.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.481+0000 7f5132035700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f512c159890 0x7f512c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.481+0000 7f5132035700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f512c069560 0x7f512c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.481+0000 7f5132035700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512c069ee0 con 0x7f512c159c60 2023-10-10T20:46:23.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.481+0000 7f5132035700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512c15a160 con 0x7f512c069560 2023-10-10T20:46:23.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.481+0000 7f5132035700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512c0d13a0 con 0x7f512c159890 2023-10-10T20:46:23.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.481+0000 7f512bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f512c069560 0x7f512c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.481+0000 7f512bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f512c069560 0x7f512c0699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43586/0 (socket says 172.21.15.89:43586) 2023-10-10T20:46:23.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.481+0000 7f512bfff700 1 -- 172.21.15.89:0/997624524 learned_addr learned my addr 172.21.15.89:0/997624524 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:23.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.482+0000 7f512b7fe700 1 --2- 172.21.15.89:0/997624524 >> v2:172.21.15.111:3300/0 conn(0x7f512c159890 0x7f512c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.482+0000 7f512bfff700 1 -- 172.21.15.89:0/997624524 >> v2:172.21.15.111:3300/0 conn(0x7f512c159890 msgr2=0x7f512c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:23.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.482+0000 7f512bfff700 1 --2- 172.21.15.89:0/997624524 >> v2:172.21.15.111:3300/0 conn(0x7f512c159890 0x7f512c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.482+0000 7f512bfff700 1 -- 172.21.15.89:0/997624524 >> v1:172.21.15.111:6789/0 conn(0x7f512c159c60 legacy=0x7f512c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:23.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.482+0000 7f512bfff700 1 -- 172.21.15.89:0/997624524 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c15e140 con 0x7f512c069560 2023-10-10T20:46:23.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.482+0000 7f512b7fe700 1 --2- 172.21.15.89:0/997624524 >> v2:172.21.15.111:3300/0 conn(0x7f512c159890 0x7f512c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:46:23.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.482+0000 7f512bfff700 1 --2- 172.21.15.89:0/997624524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f512c069560 0x7f512c0699a0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f512c0d1090 tx=0x7f5124009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af29703c566c27da server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:23.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.482+0000 7f512a7fc700 1 -- 172.21.15.89:0/997624524 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512400b820 con 0x7f512c069560 2023-10-10T20:46:23.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.482+0000 7f512a7fc700 1 -- 172.21.15.89:0/997624524 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512400b980 con 0x7f512c069560 2023-10-10T20:46:23.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.482+0000 7f512a7fc700 1 -- 172.21.15.89:0/997624524 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512400bd20 con 0x7f512c069560 2023-10-10T20:46:23.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.483+0000 7f5132035700 1 -- 172.21.15.89:0/997624524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f512c069560 msgr2=0x7f512c0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.483+0000 7f5132035700 1 --2- 172.21.15.89:0/997624524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f512c069560 0x7f512c0699a0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f512c0d1090 tx=0x7f5124009250 comp rx=0 tx=0).stop 2023-10-10T20:46:23.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.483+0000 7f5132035700 1 -- 172.21.15.89:0/997624524 shutdown_connections 2023-10-10T20:46:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.483+0000 7f5132035700 1 --2- 172.21.15.89:0/997624524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f512c069560 0x7f512c0699a0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.483+0000 7f5132035700 1 --2- 172.21.15.89:0/997624524 >> v2:172.21.15.111:3300/0 conn(0x7f512c159890 0x7f512c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.483+0000 7f5132035700 1 -- 172.21.15.89:0/997624524 >> 172.21.15.89:0/997624524 conn(0x7f512c068e20 msgr2=0x7f512c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.483+0000 7f5132035700 1 -- 172.21.15.89:0/997624524 shutdown_connections 2023-10-10T20:46:23.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.483+0000 7f5132035700 1 -- 172.21.15.89:0/997624524 wait complete. 2023-10-10T20:46:23.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.484+0000 7f5132035700 1 Processor -- start 2023-10-10T20:46:23.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.484+0000 7f5132035700 1 -- start start 2023-10-10T20:46:23.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f5132035700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f512c069560 0x7f512c140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f5132035700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f512c159890 0x7f512c15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f5132035700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f512c159c60 0x7f512c15ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f5132035700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0cd0c0 con 0x7f512c159890 2023-10-10T20:46:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f5132035700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0cd960 con 0x7f512c159c60 2023-10-10T20:46:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f5132035700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512c140e80 con 0x7f512c069560 2023-10-10T20:46:23.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f512affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f512c159890 0x7f512c15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f512affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f512c159890 0x7f512c15b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43600/0 (socket says 172.21.15.89:43600) 2023-10-10T20:46:23.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f512affd700 1 -- 172.21.15.89:0/2857513557 learned_addr learned my addr 172.21.15.89:0/2857513557 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:23.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f512b7fe700 1 --2- 172.21.15.89:0/2857513557 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f512c069560 0x7f512c140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f512bfff700 1 --2- 172.21.15.89:0/2857513557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f512c159c60 0x7f512c15ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f512affd700 1 -- 172.21.15.89:0/2857513557 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f512c069560 msgr2=0x7f512c140940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f512affd700 1 --2- 172.21.15.89:0/2857513557 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f512c069560 0x7f512c140940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f512affd700 1 -- 172.21.15.89:0/2857513557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f512c159c60 msgr2=0x7f512c15ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f512affd700 1 --2- 172.21.15.89:0/2857513557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f512c159c60 0x7f512c15ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f512affd700 1 -- 172.21.15.89:0/2857513557 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51240094f0 con 0x7f512c159890 2023-10-10T20:46:23.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f512bfff700 1 --2- 172.21.15.89:0/2857513557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f512c159c60 0x7f512c15ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:23.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f512affd700 1 --2- 172.21.15.89:0/2857513557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f512c159890 0x7f512c15b4d0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f512000b5e0 tx=0x7f5120009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:23.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f51137fe700 1 -- 172.21.15.89:0/2857513557 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512000f510 con 0x7f512c159890 2023-10-10T20:46:23.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f51137fe700 1 -- 172.21.15.89:0/2857513557 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512000f670 con 0x7f512c159890 2023-10-10T20:46:23.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f5132035700 1 -- 172.21.15.89:0/2857513557 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512c0ca580 con 0x7f512c159890 2023-10-10T20:46:23.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f51137fe700 1 -- 172.21.15.89:0/2857513557 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512000fa30 con 0x7f512c159890 2023-10-10T20:46:23.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.485+0000 7f5132035700 1 -- 172.21.15.89:0/2857513557 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512c0cd5f0 con 0x7f512c159890 2023-10-10T20:46:23.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.487+0000 7f5132035700 1 -- 172.21.15.89:0/2857513557 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512c0cdad0 con 0x7f512c159890 2023-10-10T20:46:23.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.487+0000 7f51137fe700 1 -- 172.21.15.89:0/2857513557 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f512001a400 con 0x7f512c159890 2023-10-10T20:46:23.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.487+0000 7f51137fe700 1 --2- 172.21.15.89:0/2857513557 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5118067320 0x7f51180697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.487+0000 7f512b7fe700 1 --2- 172.21.15.89:0/2857513557 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5118067320 0x7f51180697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.488+0000 7f51137fe700 1 -- 172.21.15.89:0/2857513557 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5120084400 con 0x7f512c159890 2023-10-10T20:46:23.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.488+0000 7f512b7fe700 1 --2- 172.21.15.89:0/2857513557 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5118067320 0x7f51180697d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f511c00d6e0 tx=0x7f511c007850 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:23.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.492+0000 7f51137fe700 1 -- 172.21.15.89:0/2857513557 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5120051430 con 0x7f512c159890 2023-10-10T20:46:23.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.677+0000 7f5132035700 1 -- 172.21.15.89:0/2857513557 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f512c04c3c0 con 0x7f5118067320 2023-10-10T20:46:23.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.677+0000 7f51137fe700 1 -- 172.21.15.89:0/2857513557 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f512c04c3c0 con 0x7f5118067320 2023-10-10T20:46:23.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.679+0000 7f5132035700 1 -- 172.21.15.89:0/2857513557 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5118067320 msgr2=0x7f51180697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.679+0000 7f5132035700 1 --2- 172.21.15.89:0/2857513557 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5118067320 0x7f51180697d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f511c00d6e0 tx=0x7f511c007850 comp rx=0 tx=0).stop 2023-10-10T20:46:23.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.679+0000 7f5132035700 1 -- 172.21.15.89:0/2857513557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f512c159890 msgr2=0x7f512c15b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.679+0000 7f5132035700 1 --2- 172.21.15.89:0/2857513557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f512c159890 0x7f512c15b4d0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f512000b5e0 tx=0x7f5120009120 comp rx=0 tx=0).stop 2023-10-10T20:46:23.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.680+0000 7f5132035700 1 -- 172.21.15.89:0/2857513557 shutdown_connections 2023-10-10T20:46:23.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.680+0000 7f5132035700 1 --2- 172.21.15.89:0/2857513557 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5118067320 0x7f51180697d0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.680+0000 7f5132035700 1 --2- 172.21.15.89:0/2857513557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f512c159c60 0x7f512c15ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.680+0000 7f5132035700 1 --2- 172.21.15.89:0/2857513557 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f512c069560 0x7f512c140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.680+0000 7f5132035700 1 --2- 172.21.15.89:0/2857513557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f512c159890 0x7f512c15b4d0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.680+0000 7f5132035700 1 -- 172.21.15.89:0/2857513557 >> 172.21.15.89:0/2857513557 conn(0x7f512c068e20 msgr2=0x7f512c0c6980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:23.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.680+0000 7f5132035700 1 -- 172.21.15.89:0/2857513557 shutdown_connections 2023-10-10T20:46:23.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.680+0000 7f5132035700 1 -- 172.21.15.89:0/2857513557 wait complete. 2023-10-10T20:46:23.685 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 1 to deep-scrub 2023-10-10T20:46:23.693 INFO:tasks.ceph:Scrubbing osd.2 2023-10-10T20:46:23.694 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-10T20:46:23.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.811+0000 7f98921ca700 1 Processor -- start 2023-10-10T20:46:23.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.811+0000 7f98921ca700 1 -- start start 2023-10-10T20:46:23.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.811+0000 7f98921ca700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f988c159ec0 0x7f988c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.811+0000 7f98921ca700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f988c0b53d0 0x7f988c0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.811+0000 7f98921ca700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988c0b5b00 con 0x7f988c0cce40 2023-10-10T20:46:23.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.811+0000 7f98921ca700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988c0b5850 con 0x7f988c159ec0 2023-10-10T20:46:23.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.811+0000 7f98921ca700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988c0b5da0 con 0x7f988c0b53d0 2023-10-10T20:46:23.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.812+0000 7f988affd700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f988c0b53d0 0x7f988c0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.812+0000 7f988b7fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f988c159ec0 0x7f988c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.812+0000 7f988bfff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f988c0cce40 0x7f988c0cd2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:32860/0 (socket says 172.21.15.89:32860) 2023-10-10T20:46:23.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.812+0000 7f988bfff700 1 -- 172.21.15.89:0/2758196980 learned_addr learned my addr 172.21.15.89:0/2758196980 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:23.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.812+0000 7f988affd700 1 -- 172.21.15.89:0/2758196980 >> v2:172.21.15.111:3300/0 conn(0x7f988c159ec0 msgr2=0x7f988c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:23.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.812+0000 7f988affd700 1 --2- 172.21.15.89:0/2758196980 >> v2:172.21.15.111:3300/0 conn(0x7f988c159ec0 0x7f988c0b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.812+0000 7f988affd700 1 -- 172.21.15.89:0/2758196980 >> v1:172.21.15.111:6789/0 conn(0x7f988c0cce40 legacy=0x7f988c0cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.812+0000 7f988affd700 1 -- 172.21.15.89:0/2758196980 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c0b6080 con 0x7f988c0b53d0 2023-10-10T20:46:23.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.812+0000 7f988b7fe700 1 --2- 172.21.15.89:0/2758196980 >> v2:172.21.15.111:3300/0 conn(0x7f988c159ec0 0x7f988c0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:46:23.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.812+0000 7f988affd700 1 --2- 172.21.15.89:0/2758196980 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f988c0b53d0 0x7f988c0cc900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f98800098e0 tx=0x7f98800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdb2a4095f97aabb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:23.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.812+0000 7f988a7fc700 1 -- 172.21.15.89:0/2758196980 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988000b640 con 0x7f988c0b53d0 2023-10-10T20:46:23.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.813+0000 7f988a7fc700 1 -- 172.21.15.89:0/2758196980 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988000b7a0 con 0x7f988c0b53d0 2023-10-10T20:46:23.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.813+0000 7f988a7fc700 1 -- 172.21.15.89:0/2758196980 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988000bb40 con 0x7f988c0b53d0 2023-10-10T20:46:23.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.813+0000 7f98921ca700 1 -- 172.21.15.89:0/2758196980 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f988c0b53d0 msgr2=0x7f988c0cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.813+0000 7f98921ca700 1 --2- 172.21.15.89:0/2758196980 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f988c0b53d0 0x7f988c0cc900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f98800098e0 tx=0x7f98800091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.813+0000 7f98921ca700 1 -- 172.21.15.89:0/2758196980 shutdown_connections 2023-10-10T20:46:23.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.813+0000 7f98921ca700 1 --2- 172.21.15.89:0/2758196980 >> v2:172.21.15.111:3300/0 conn(0x7f988c159ec0 0x7f988c0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.813+0000 7f98921ca700 1 --2- 172.21.15.89:0/2758196980 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f988c0b53d0 0x7f988c0cc900 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.813+0000 7f98921ca700 1 -- 172.21.15.89:0/2758196980 >> 172.21.15.89:0/2758196980 conn(0x7f988c0999f0 msgr2=0x7f988c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:23.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.813+0000 7f98921ca700 1 -- 172.21.15.89:0/2758196980 shutdown_connections 2023-10-10T20:46:23.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.813+0000 7f98921ca700 1 -- 172.21.15.89:0/2758196980 wait complete. 2023-10-10T20:46:23.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.814+0000 7f98921ca700 1 Processor -- start 2023-10-10T20:46:23.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.814+0000 7f98921ca700 1 -- start start 2023-10-10T20:46:23.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.815+0000 7f98921ca700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f988c0b53d0 0x7f988c156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.815+0000 7f98921ca700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f988c0cce40 0x7f988c068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.815+0000 7f98921ca700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f988c159ec0 0x7f988c09d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.815+0000 7f98921ca700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c09a7d0 con 0x7f988c159ec0 2023-10-10T20:46:23.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.815+0000 7f98921ca700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c068d50 con 0x7f988c0b53d0 2023-10-10T20:46:23.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.815+0000 7f98921ca700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988c069030 con 0x7f988c0cce40 2023-10-10T20:46:23.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.815+0000 7f988bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f988c159ec0 0x7f988c09d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.815+0000 7f988bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f988c159ec0 0x7f988c09d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43608/0 (socket says 172.21.15.89:43608) 2023-10-10T20:46:23.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.815+0000 7f988bfff700 1 -- 172.21.15.89:0/990055167 learned_addr learned my addr 172.21.15.89:0/990055167 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:23.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.815+0000 7f988b7fe700 1 --2- 172.21.15.89:0/990055167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f988c0b53d0 0x7f988c156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.815+0000 7f988affd700 1 --2- 172.21.15.89:0/990055167 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f988c0cce40 0x7f988c068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.815+0000 7f988bfff700 1 -- 172.21.15.89:0/990055167 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f988c0cce40 msgr2=0x7f988c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.815+0000 7f988bfff700 1 --2- 172.21.15.89:0/990055167 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f988c0cce40 0x7f988c068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.815+0000 7f988bfff700 1 -- 172.21.15.89:0/990055167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f988c0b53d0 msgr2=0x7f988c156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.815+0000 7f988bfff700 1 --2- 172.21.15.89:0/990055167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f988c0b53d0 0x7f988c156270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.815+0000 7f988bfff700 1 -- 172.21.15.89:0/990055167 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987c0093c0 con 0x7f988c159ec0 2023-10-10T20:46:23.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.816+0000 7f988bfff700 1 --2- 172.21.15.89:0/990055167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f988c159ec0 0x7f988c09d930 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f988400cc40 tx=0x7f9884001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:23.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.816+0000 7f98737fe700 1 -- 172.21.15.89:0/990055167 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988400a9c0 con 0x7f988c159ec0 2023-10-10T20:46:23.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.816+0000 7f98737fe700 1 -- 172.21.15.89:0/990055167 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988400ab20 con 0x7f988c159ec0 2023-10-10T20:46:23.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.816+0000 7f98921ca700 1 -- 172.21.15.89:0/990055167 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9880009420 con 0x7f988c159ec0 2023-10-10T20:46:23.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.816+0000 7f98737fe700 1 -- 172.21.15.89:0/990055167 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98840070f0 con 0x7f988c159ec0 2023-10-10T20:46:23.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.816+0000 7f98921ca700 1 -- 172.21.15.89:0/990055167 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988c0693c0 con 0x7f988c159ec0 2023-10-10T20:46:23.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.816+0000 7f98921ca700 1 -- 172.21.15.89:0/990055167 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f988c08a800 con 0x7f988c159ec0 2023-10-10T20:46:23.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.817+0000 7f98737fe700 1 -- 172.21.15.89:0/990055167 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f98840074e0 con 0x7f988c159ec0 2023-10-10T20:46:23.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.817+0000 7f98737fe700 1 --2- 172.21.15.89:0/990055167 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9878067320 0x7f98780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.818+0000 7f98737fe700 1 -- 172.21.15.89:0/990055167 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f9884085790 con 0x7f988c159ec0 2023-10-10T20:46:23.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.818+0000 7f988b7fe700 1 --2- 172.21.15.89:0/990055167 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9878067320 0x7f98780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.818+0000 7f98737fe700 1 --2- 172.21.15.89:0/990055167 >> [v2:172.21.15.89:6800/90896,v1:172.21.15.89:6801/90896] conn(0x7f987806d170 0x7f987806f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.818+0000 7f98737fe700 1 -- 172.21.15.89:0/990055167 --> [v2:172.21.15.89:6800/90896,v1:172.21.15.89:6801/90896] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f987806fc50 con 0x7f987806d170 2023-10-10T20:46:23.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.818+0000 7f98737fe700 1 -- 172.21.15.89:0/990055167 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9884085aa0 con 0x7f988c159ec0 2023-10-10T20:46:23.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.818+0000 7f988affd700 1 --2- 172.21.15.89:0/990055167 >> [v2:172.21.15.89:6800/90896,v1:172.21.15.89:6801/90896] conn(0x7f987806d170 0x7f987806f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.818+0000 7f988b7fe700 1 --2- 172.21.15.89:0/990055167 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9878067320 0x7f98780697d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f987c009390 tx=0x7f987c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:23.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.818+0000 7f988affd700 1 --2- 172.21.15.89:0/990055167 >> [v2:172.21.15.89:6800/90896,v1:172.21.15.89:6801/90896] conn(0x7f987806d170 0x7f987806f580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:23.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.819+0000 7f98737fe700 1 -- 172.21.15.89:0/990055167 <== osd.2 v2:172.21.15.89:6800/90896 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f987806fc50 con 0x7f987806d170 2023-10-10T20:46:23.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.832+0000 7f98921ca700 1 -- 172.21.15.89:0/990055167 --> [v2:172.21.15.89:6800/90896,v1:172.21.15.89:6801/90896] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f988c0691f0 con 0x7f987806d170 2023-10-10T20:46:23.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.832+0000 7f98737fe700 1 -- 172.21.15.89:0/990055167 <== osd.2 v2:172.21.15.89:6800/90896 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9880000c50 con 0x7f987806d170 2023-10-10T20:46:23.838 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:46:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-10T20:46:23.839 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:46:23.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.833+0000 7f98921ca700 1 -- 172.21.15.89:0/990055167 >> [v2:172.21.15.89:6800/90896,v1:172.21.15.89:6801/90896] conn(0x7f987806d170 msgr2=0x7f987806f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.833+0000 7f98921ca700 1 --2- 172.21.15.89:0/990055167 >> [v2:172.21.15.89:6800/90896,v1:172.21.15.89:6801/90896] conn(0x7f987806d170 0x7f987806f580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.833+0000 7f98921ca700 1 -- 172.21.15.89:0/990055167 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9878067320 msgr2=0x7f98780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.833+0000 7f98921ca700 1 --2- 172.21.15.89:0/990055167 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f9878067320 0x7f98780697d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f987c009390 tx=0x7f987c008040 comp rx=0 tx=0).stop 2023-10-10T20:46:23.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.833+0000 7f98921ca700 1 -- 172.21.15.89:0/990055167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f988c159ec0 msgr2=0x7f988c09d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.833+0000 7f98921ca700 1 --2- 172.21.15.89:0/990055167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f988c159ec0 0x7f988c09d930 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f988400cc40 tx=0x7f9884001a50 comp rx=0 tx=0).stop 2023-10-10T20:46:23.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.833+0000 7f988bfff700 1 -- 172.21.15.89:0/990055167 reap_dead start 2023-10-10T20:46:23.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.833+0000 7f98921ca700 1 -- 172.21.15.89:0/990055167 shutdown_connections 2023-10-10T20:46:23.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.833+0000 7f98921ca700 1 -- 172.21.15.89:0/990055167 >> 172.21.15.89:0/990055167 conn(0x7f988c0999f0 msgr2=0x7f988c0c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:23.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.833+0000 7f98921ca700 1 -- 172.21.15.89:0/990055167 shutdown_connections 2023-10-10T20:46:23.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.833+0000 7f98921ca700 1 -- 172.21.15.89:0/990055167 wait complete. 2023-10-10T20:46:23.846 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-10T20:46:23.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.959+0000 7fe11676c700 1 Processor -- start 2023-10-10T20:46:23.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.959+0000 7fe11676c700 1 -- start start 2023-10-10T20:46:23.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.960+0000 7fe11676c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe110159420 0x7fe1100b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.960+0000 7fe11676c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1100b2980 con 0x7fe1101597f0 2023-10-10T20:46:23.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.960+0000 7fe11676c700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1100b11d0 con 0x7fe11015d230 2023-10-10T20:46:23.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.960+0000 7fe11676c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1100b14b0 con 0x7fe110159420 2023-10-10T20:46:23.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.960+0000 7fe114f69700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe1101597f0 0x7fe1100b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51738/0 (socket says 172.21.15.89:51738) 2023-10-10T20:46:23.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.960+0000 7fe114f69700 1 -- 172.21.15.89:0/502644003 learned_addr learned my addr 172.21.15.89:0/502644003 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:23.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.960+0000 7fe11576a700 1 --2- 172.21.15.89:0/502644003 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe110159420 0x7fe1100b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.960+0000 7fe10ffff700 1 -- 172.21.15.89:0/502644003 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2163615857 0 0) 0x7fe1100b2980 con 0x7fe1101597f0 2023-10-10T20:46:23.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.960+0000 7fe10ffff700 1 -- 172.21.15.89:0/502644003 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0f8003570 con 0x7fe1101597f0 2023-10-10T20:46:23.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe10ffff700 1 -- 172.21.15.89:0/502644003 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 547104544 0 0) 0x7fe0f8003570 con 0x7fe1101597f0 2023-10-10T20:46:23.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe10ffff700 1 -- 172.21.15.89:0/502644003 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe110159420 msgr2=0x7fe1100b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:23.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe10ffff700 1 --2- 172.21.15.89:0/502644003 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe110159420 0x7fe1100b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe10ffff700 1 -- 172.21.15.89:0/502644003 >> v1:172.21.15.111:6789/0 conn(0x7fe11015d230 legacy=0x7fe1100b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe10ffff700 1 -- 172.21.15.89:0/502644003 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1100b1790 con 0x7fe1101597f0 2023-10-10T20:46:23.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe10ffff700 1 -- 172.21.15.89:0/502644003 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fe104001470 con 0x7fe1101597f0 2023-10-10T20:46:23.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe10ffff700 1 -- 172.21.15.89:0/502644003 >> v1:172.21.15.89:6789/0 conn(0x7fe1101597f0 legacy=0x7fe1100b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe10ffff700 1 --2- 172.21.15.89:0/502644003 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0f8003a90 0x7fe0f8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe10ffff700 1 --2- 172.21.15.89:0/502644003 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe110159420 0x7fe0f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe10ffff700 1 --2- 172.21.15.89:0/502644003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0f8004d50 0x7fe0f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe10ffff700 1 -- 172.21.15.89:0/502644003 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1100b14b0 con 0x7fe0f8004d50 2023-10-10T20:46:23.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe10ffff700 1 -- 172.21.15.89:0/502644003 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f800bb70 con 0x7fe0f8003a90 2023-10-10T20:46:23.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe10ffff700 1 -- 172.21.15.89:0/502644003 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f800bd30 con 0x7fe110159420 2023-10-10T20:46:23.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe115f6b700 1 --2- 172.21.15.89:0/502644003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0f8004d50 0x7fe0f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe11576a700 1 --2- 172.21.15.89:0/502644003 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0f8003a90 0x7fe0f8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe114f69700 1 --2- 172.21.15.89:0/502644003 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe110159420 0x7fe0f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe115f6b700 1 -- 172.21.15.89:0/502644003 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe110159420 msgr2=0x7fe0f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:23.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe115f6b700 1 --2- 172.21.15.89:0/502644003 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe110159420 0x7fe0f8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe115f6b700 1 -- 172.21.15.89:0/502644003 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0f8003a90 msgr2=0x7fe0f8003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:23.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe115f6b700 1 --2- 172.21.15.89:0/502644003 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0f8003a90 0x7fe0f8003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe115f6b700 1 -- 172.21.15.89:0/502644003 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f8003570 con 0x7fe0f8004d50 2023-10-10T20:46:23.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe115f6b700 1 --2- 172.21.15.89:0/502644003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0f8004d50 0x7fe0f80051c0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fe108001fc0 tx=0x7fe108001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f793905213abb37 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:23.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe10ffff700 1 -- 172.21.15.89:0/502644003 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108001950 con 0x7fe0f8004d50 2023-10-10T20:46:23.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe10ffff700 1 -- 172.21.15.89:0/502644003 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10800c8b0 con 0x7fe0f8004d50 2023-10-10T20:46:23.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.961+0000 7fe10ffff700 1 -- 172.21.15.89:0/502644003 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10800cc50 con 0x7fe0f8004d50 2023-10-10T20:46:23.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.962+0000 7fe11676c700 1 -- 172.21.15.89:0/502644003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0f8004d50 msgr2=0x7fe0f80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.962+0000 7fe11676c700 1 --2- 172.21.15.89:0/502644003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0f8004d50 0x7fe0f80051c0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fe108001fc0 tx=0x7fe108001640 comp rx=0 tx=0).stop 2023-10-10T20:46:23.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.962+0000 7fe115f6b700 1 -- 172.21.15.89:0/502644003 reap_dead start 2023-10-10T20:46:23.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.962+0000 7fe11676c700 1 -- 172.21.15.89:0/502644003 shutdown_connections 2023-10-10T20:46:23.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.962+0000 7fe11676c700 1 -- 172.21.15.89:0/502644003 >> 172.21.15.89:0/502644003 conn(0x7fe1100d0660 msgr2=0x7fe1100d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:23.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.962+0000 7fe11676c700 1 -- 172.21.15.89:0/502644003 shutdown_connections 2023-10-10T20:46:23.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.962+0000 7fe11676c700 1 -- 172.21.15.89:0/502644003 wait complete. 2023-10-10T20:46:23.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.963+0000 7fe11676c700 1 Processor -- start 2023-10-10T20:46:23.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.963+0000 7fe11676c700 1 -- start start 2023-10-10T20:46:23.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.964+0000 7fe11676c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1100b40a0 0x7fe110159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.964+0000 7fe11676c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe11015cc60 0x7fe11014d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.964+0000 7fe11676c700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe1100b4470 0x7fe11014db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.964+0000 7fe11676c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11015b260 con 0x7fe1100b40a0 2023-10-10T20:46:23.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.964+0000 7fe11676c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11015a650 con 0x7fe11015cc60 2023-10-10T20:46:23.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.964+0000 7fe11676c700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11015a930 con 0x7fe1100b4470 2023-10-10T20:46:23.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.964+0000 7fe11576a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1100b40a0 0x7fe110159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.964+0000 7fe11576a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1100b40a0 0x7fe110159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43620/0 (socket says 172.21.15.89:43620) 2023-10-10T20:46:23.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.964+0000 7fe11576a700 1 -- 172.21.15.89:0/1174628449 learned_addr learned my addr 172.21.15.89:0/1174628449 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:23.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.964+0000 7fe114f69700 1 --2- 172.21.15.89:0/1174628449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe11015cc60 0x7fe11014d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.964+0000 7fe115f6b700 1 --2- 172.21.15.89:0/1174628449 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe1100b4470 0x7fe11014db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.964+0000 7fe11576a700 1 -- 172.21.15.89:0/1174628449 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe1100b4470 msgr2=0x7fe11014db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.964+0000 7fe11576a700 1 --2- 172.21.15.89:0/1174628449 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe1100b4470 0x7fe11014db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.964+0000 7fe11576a700 1 -- 172.21.15.89:0/1174628449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe11015cc60 msgr2=0x7fe11014d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:23.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.964+0000 7fe11576a700 1 --2- 172.21.15.89:0/1174628449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe11015cc60 0x7fe11014d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:23.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.964+0000 7fe11576a700 1 -- 172.21.15.89:0/1174628449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11015ac10 con 0x7fe1100b40a0 2023-10-10T20:46:23.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.964+0000 7fe115f6b700 1 --2- 172.21.15.89:0/1174628449 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe1100b4470 0x7fe11014db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:23.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.964+0000 7fe114f69700 1 --2- 172.21.15.89:0/1174628449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe11015cc60 0x7fe11014d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:23.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.965+0000 7fe11576a700 1 --2- 172.21.15.89:0/1174628449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1100b40a0 0x7fe110159d40 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fe100006ac0 tx=0x7fe1000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:23.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.965+0000 7fe10d7fa700 1 -- 172.21.15.89:0/1174628449 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10000fa10 con 0x7fe1100b40a0 2023-10-10T20:46:23.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.965+0000 7fe10d7fa700 1 -- 172.21.15.89:0/1174628449 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10000fb70 con 0x7fe1100b40a0 2023-10-10T20:46:23.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.965+0000 7fe11676c700 1 -- 172.21.15.89:0/1174628449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11015aef0 con 0x7fe1100b40a0 2023-10-10T20:46:23.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.965+0000 7fe10d7fa700 1 -- 172.21.15.89:0/1174628449 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100014440 con 0x7fe1100b40a0 2023-10-10T20:46:23.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.965+0000 7fe11676c700 1 -- 172.21.15.89:0/1174628449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11015e5d0 con 0x7fe1100b40a0 2023-10-10T20:46:23.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.966+0000 7fe11676c700 1 -- 172.21.15.89:0/1174628449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11004c3c0 con 0x7fe1100b40a0 2023-10-10T20:46:23.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.966+0000 7fe10d7fa700 1 -- 172.21.15.89:0/1174628449 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe100014870 con 0x7fe1100b40a0 2023-10-10T20:46:23.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.966+0000 7fe10d7fa700 1 --2- 172.21.15.89:0/1174628449 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe0fc067320 0x7fe0fc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:23.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.966+0000 7fe10d7fa700 1 -- 172.21.15.89:0/1174628449 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fe100018030 con 0x7fe1100b40a0 2023-10-10T20:46:23.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.966+0000 7fe114f69700 1 --2- 172.21.15.89:0/1174628449 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe0fc067320 0x7fe0fc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:23.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.967+0000 7fe114f69700 1 --2- 172.21.15.89:0/1174628449 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe0fc067320 0x7fe0fc0697d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe104000f60 tx=0x7fe104009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:23.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:23.971+0000 7fe10d7fa700 1 -- 172.21.15.89:0/1174628449 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe100052410 con 0x7fe1100b40a0 2023-10-10T20:46:24.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.155+0000 7fe11676c700 1 -- 172.21.15.89:0/1174628449 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe11014f920 con 0x7fe0fc067320 2023-10-10T20:46:24.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.155+0000 7fe10d7fa700 1 -- 172.21.15.89:0/1174628449 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe11014f920 con 0x7fe0fc067320 2023-10-10T20:46:24.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.157+0000 7fe11676c700 1 -- 172.21.15.89:0/1174628449 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe0fc067320 msgr2=0x7fe0fc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.157+0000 7fe11676c700 1 --2- 172.21.15.89:0/1174628449 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe0fc067320 0x7fe0fc0697d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe104000f60 tx=0x7fe104009040 comp rx=0 tx=0).stop 2023-10-10T20:46:24.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.157+0000 7fe11676c700 1 -- 172.21.15.89:0/1174628449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1100b40a0 msgr2=0x7fe110159d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.157+0000 7fe11676c700 1 --2- 172.21.15.89:0/1174628449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1100b40a0 0x7fe110159d40 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fe100006ac0 tx=0x7fe1000091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.157+0000 7fe11676c700 1 -- 172.21.15.89:0/1174628449 shutdown_connections 2023-10-10T20:46:24.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.157+0000 7fe11676c700 1 --2- 172.21.15.89:0/1174628449 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fe1100b4470 0x7fe11014db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.157+0000 7fe11676c700 1 --2- 172.21.15.89:0/1174628449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1100b40a0 0x7fe110159d40 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.157+0000 7fe11676c700 1 --2- 172.21.15.89:0/1174628449 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fe0fc067320 0x7fe0fc0697d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.157+0000 7fe11676c700 1 --2- 172.21.15.89:0/1174628449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe11015cc60 0x7fe11014d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.158+0000 7fe11676c700 1 -- 172.21.15.89:0/1174628449 >> 172.21.15.89:0/1174628449 conn(0x7fe1100d0660 msgr2=0x7fe1100b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:24.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.158+0000 7fe11676c700 1 -- 172.21.15.89:0/1174628449 shutdown_connections 2023-10-10T20:46:24.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.158+0000 7fe11676c700 1 -- 172.21.15.89:0/1174628449 wait complete. 2023-10-10T20:46:24.165 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 2 to deep-scrub 2023-10-10T20:46:24.171 INFO:tasks.ceph:Scrubbing osd.3 2023-10-10T20:46:24.172 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-10T20:46:24.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.290+0000 7fdd74fb8700 1 Processor -- start 2023-10-10T20:46:24.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.290+0000 7fdd74fb8700 1 -- start start 2023-10-10T20:46:24.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.291+0000 7fdd74fb8700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdd70159ec0 0x7fdd700b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.291+0000 7fdd74fb8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd700b53d0 0x7fdd700cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.291+0000 7fdd74fb8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdd700cce40 0x7fdd700cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.291+0000 7fdd74fb8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd700b5b00 con 0x7fdd700cce40 2023-10-10T20:46:24.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.291+0000 7fdd74fb8700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd700b5850 con 0x7fdd70159ec0 2023-10-10T20:46:24.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.291+0000 7fdd74fb8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd700b5da0 con 0x7fdd700b53d0 2023-10-10T20:46:24.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.291+0000 7fdd6ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdd700cce40 0x7fdd700cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.291+0000 7fdd6ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdd700cce40 0x7fdd700cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43630/0 (socket says 172.21.15.89:43630) 2023-10-10T20:46:24.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.291+0000 7fdd6ed9d700 1 -- 172.21.15.89:0/2283763715 learned_addr learned my addr 172.21.15.89:0/2283763715 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:24.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.291+0000 7fdd65d9b700 1 --2- 172.21.15.89:0/2283763715 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd700b53d0 0x7fdd700cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.291+0000 7fdd6e59c700 1 --2- 172.21.15.89:0/2283763715 >> v2:172.21.15.111:3300/0 conn(0x7fdd70159ec0 0x7fdd700b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.291+0000 7fdd6ed9d700 1 -- 172.21.15.89:0/2283763715 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd700b53d0 msgr2=0x7fdd700cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:24.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.291+0000 7fdd6ed9d700 1 --2- 172.21.15.89:0/2283763715 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd700b53d0 0x7fdd700cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.291+0000 7fdd6ed9d700 1 -- 172.21.15.89:0/2283763715 >> v2:172.21.15.111:3300/0 conn(0x7fdd70159ec0 msgr2=0x7fdd700b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:24.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.291+0000 7fdd6ed9d700 1 --2- 172.21.15.89:0/2283763715 >> v2:172.21.15.111:3300/0 conn(0x7fdd70159ec0 0x7fdd700b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.291+0000 7fdd6ed9d700 1 -- 172.21.15.89:0/2283763715 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd700b6080 con 0x7fdd700cce40 2023-10-10T20:46:24.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.291+0000 7fdd65d9b700 1 --2- 172.21.15.89:0/2283763715 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd700b53d0 0x7fdd700cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:24.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.291+0000 7fdd6e59c700 1 --2- 172.21.15.89:0/2283763715 >> v2:172.21.15.111:3300/0 conn(0x7fdd70159ec0 0x7fdd700b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.292+0000 7fdd6ed9d700 1 --2- 172.21.15.89:0/2283763715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd700cce40 0x7fdd700cd2a0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fdd70089120 tx=0x7fdd68009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869444e7c620b701 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.292+0000 7fdd6dd9b700 1 -- 172.21.15.89:0/2283763715 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6800e070 con 0x7fdd700cce40 2023-10-10T20:46:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.292+0000 7fdd6dd9b700 1 -- 172.21.15.89:0/2283763715 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6800ba50 con 0x7fdd700cce40 2023-10-10T20:46:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.292+0000 7fdd6dd9b700 1 -- 172.21.15.89:0/2283763715 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd680092e0 con 0x7fdd700cce40 2023-10-10T20:46:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.292+0000 7fdd74fb8700 1 -- 172.21.15.89:0/2283763715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd700cce40 msgr2=0x7fdd700cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.292+0000 7fdd74fb8700 1 --2- 172.21.15.89:0/2283763715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd700cce40 0x7fdd700cd2a0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fdd70089120 tx=0x7fdd68009840 comp rx=0 tx=0).stop 2023-10-10T20:46:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.292+0000 7fdd74fb8700 1 -- 172.21.15.89:0/2283763715 shutdown_connections 2023-10-10T20:46:24.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.292+0000 7fdd74fb8700 1 --2- 172.21.15.89:0/2283763715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd700cce40 0x7fdd700cd2a0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.292+0000 7fdd74fb8700 1 --2- 172.21.15.89:0/2283763715 >> v2:172.21.15.111:3300/0 conn(0x7fdd70159ec0 0x7fdd700b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.292+0000 7fdd74fb8700 1 --2- 172.21.15.89:0/2283763715 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd700b53d0 0x7fdd700cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.292+0000 7fdd74fb8700 1 -- 172.21.15.89:0/2283763715 >> 172.21.15.89:0/2283763715 conn(0x7fdd700999f0 msgr2=0x7fdd700b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:24.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.292+0000 7fdd74fb8700 1 -- 172.21.15.89:0/2283763715 shutdown_connections 2023-10-10T20:46:24.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.292+0000 7fdd74fb8700 1 -- 172.21.15.89:0/2283763715 wait complete. 2023-10-10T20:46:24.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.293+0000 7fdd74fb8700 1 Processor -- start 2023-10-10T20:46:24.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.294+0000 7fdd74fb8700 1 -- start start 2023-10-10T20:46:24.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.294+0000 7fdd74fb8700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd700b53d0 0x7fdd7016a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.294+0000 7fdd74fb8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd700cce40 0x7fdd7009c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.294+0000 7fdd74fb8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd70159ec0 0x7fdd700c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.294+0000 7fdd74fb8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7016aee0 con 0x7fdd700cce40 2023-10-10T20:46:24.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.294+0000 7fdd74fb8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7016a510 con 0x7fdd70159ec0 2023-10-10T20:46:24.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.294+0000 7fdd74fb8700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7009cda0 con 0x7fdd700b53d0 2023-10-10T20:46:24.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.294+0000 7fdd65d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd700cce40 0x7fdd7009c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.294+0000 7fdd65d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd700cce40 0x7fdd7009c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43636/0 (socket says 172.21.15.89:43636) 2023-10-10T20:46:24.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.294+0000 7fdd65d9b700 1 -- 172.21.15.89:0/1898292322 learned_addr learned my addr 172.21.15.89:0/1898292322 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:24.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.295+0000 7fdd6e59c700 1 --2- 172.21.15.89:0/1898292322 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd700b53d0 0x7fdd7016a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.295+0000 7fdd6ed9d700 1 --2- 172.21.15.89:0/1898292322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd70159ec0 0x7fdd700c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.295+0000 7fdd65d9b700 1 -- 172.21.15.89:0/1898292322 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd700b53d0 msgr2=0x7fdd7016a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.295+0000 7fdd65d9b700 1 --2- 172.21.15.89:0/1898292322 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fdd700b53d0 0x7fdd7016a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.295+0000 7fdd65d9b700 1 -- 172.21.15.89:0/1898292322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd70159ec0 msgr2=0x7fdd700c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.295+0000 7fdd65d9b700 1 --2- 172.21.15.89:0/1898292322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd70159ec0 0x7fdd700c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.295+0000 7fdd65d9b700 1 -- 172.21.15.89:0/1898292322 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd68009ae0 con 0x7fdd700cce40 2023-10-10T20:46:24.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.295+0000 7fdd65d9b700 1 --2- 172.21.15.89:0/1898292322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd700cce40 0x7fdd7009c860 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800d860 tx=0x7fdd58008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:24.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.295+0000 7fdd66ffd700 1 -- 172.21.15.89:0/1898292322 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd580125f0 con 0x7fdd700cce40 2023-10-10T20:46:24.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.295+0000 7fdd66ffd700 1 -- 172.21.15.89:0/1898292322 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd58012750 con 0x7fdd700cce40 2023-10-10T20:46:24.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.295+0000 7fdd74fb8700 1 -- 172.21.15.89:0/1898292322 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7009d070 con 0x7fdd700cce40 2023-10-10T20:46:24.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.295+0000 7fdd66ffd700 1 -- 172.21.15.89:0/1898292322 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58012ad0 con 0x7fdd700cce40 2023-10-10T20:46:24.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.295+0000 7fdd74fb8700 1 -- 172.21.15.89:0/1898292322 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7016a1a0 con 0x7fdd700cce40 2023-10-10T20:46:24.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.296+0000 7fdd74fb8700 1 -- 172.21.15.89:0/1898292322 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdd70063c90 con 0x7fdd700cce40 2023-10-10T20:46:24.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.296+0000 7fdd66ffd700 1 -- 172.21.15.89:0/1898292322 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdd58014430 con 0x7fdd700cce40 2023-10-10T20:46:24.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.297+0000 7fdd66ffd700 1 --2- 172.21.15.89:0/1898292322 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdd50067320 0x7fdd500697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.297+0000 7fdd66ffd700 1 -- 172.21.15.89:0/1898292322 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fdd58084db0 con 0x7fdd700cce40 2023-10-10T20:46:24.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.297+0000 7fdd6e59c700 1 --2- 172.21.15.89:0/1898292322 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdd50067320 0x7fdd500697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.297+0000 7fdd66ffd700 1 --2- 172.21.15.89:0/1898292322 >> [v2:172.21.15.89:6820/90907,v1:172.21.15.89:6822/90907] conn(0x7fdd5006d170 0x7fdd5006f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.297+0000 7fdd66ffd700 1 -- 172.21.15.89:0/1898292322 --> [v2:172.21.15.89:6820/90907,v1:172.21.15.89:6822/90907] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd5006fc50 con 0x7fdd5006d170 2023-10-10T20:46:24.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.297+0000 7fdd66ffd700 1 -- 172.21.15.89:0/1898292322 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd58087310 con 0x7fdd700cce40 2023-10-10T20:46:24.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.297+0000 7fdd6ed9d700 1 --2- 172.21.15.89:0/1898292322 >> [v2:172.21.15.89:6820/90907,v1:172.21.15.89:6822/90907] conn(0x7fdd5006d170 0x7fdd5006f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.297+0000 7fdd6e59c700 1 --2- 172.21.15.89:0/1898292322 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdd50067320 0x7fdd500697d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009a80 tx=0x7fdd60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:24.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.297+0000 7fdd6ed9d700 1 --2- 172.21.15.89:0/1898292322 >> [v2:172.21.15.89:6820/90907,v1:172.21.15.89:6822/90907] conn(0x7fdd5006d170 0x7fdd5006f580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:24.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.298+0000 7fdd66ffd700 1 -- 172.21.15.89:0/1898292322 <== osd.3 v2:172.21.15.89:6820/90907 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fdd5006fc50 con 0x7fdd5006d170 2023-10-10T20:46:24.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.311+0000 7fdd74fb8700 1 -- 172.21.15.89:0/1898292322 --> [v2:172.21.15.89:6820/90907,v1:172.21.15.89:6822/90907] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdd700718b0 con 0x7fdd5006d170 2023-10-10T20:46:24.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.312+0000 7fdd66ffd700 1 -- 172.21.15.89:0/1898292322 <== osd.3 v2:172.21.15.89:6820/90907 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fdd700718b0 con 0x7fdd5006d170 2023-10-10T20:46:24.320 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:46:24.321 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-10T20:46:24.321 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:46:24.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.312+0000 7fdd74fb8700 1 -- 172.21.15.89:0/1898292322 >> [v2:172.21.15.89:6820/90907,v1:172.21.15.89:6822/90907] conn(0x7fdd5006d170 msgr2=0x7fdd5006f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.312+0000 7fdd74fb8700 1 --2- 172.21.15.89:0/1898292322 >> [v2:172.21.15.89:6820/90907,v1:172.21.15.89:6822/90907] conn(0x7fdd5006d170 0x7fdd5006f580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.312+0000 7fdd74fb8700 1 -- 172.21.15.89:0/1898292322 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdd50067320 msgr2=0x7fdd500697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.312+0000 7fdd74fb8700 1 --2- 172.21.15.89:0/1898292322 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fdd50067320 0x7fdd500697d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009a80 tx=0x7fdd60008040 comp rx=0 tx=0).stop 2023-10-10T20:46:24.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.312+0000 7fdd74fb8700 1 -- 172.21.15.89:0/1898292322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd700cce40 msgr2=0x7fdd7009c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.312+0000 7fdd74fb8700 1 --2- 172.21.15.89:0/1898292322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd700cce40 0x7fdd7009c860 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800d860 tx=0x7fdd58008040 comp rx=0 tx=0).stop 2023-10-10T20:46:24.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.312+0000 7fdd6ed9d700 1 -- 172.21.15.89:0/1898292322 reap_dead start 2023-10-10T20:46:24.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.312+0000 7fdd74fb8700 1 -- 172.21.15.89:0/1898292322 shutdown_connections 2023-10-10T20:46:24.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.312+0000 7fdd74fb8700 1 -- 172.21.15.89:0/1898292322 >> 172.21.15.89:0/1898292322 conn(0x7fdd700999f0 msgr2=0x7fdd700b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:24.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.312+0000 7fdd74fb8700 1 -- 172.21.15.89:0/1898292322 shutdown_connections 2023-10-10T20:46:24.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.312+0000 7fdd74fb8700 1 -- 172.21.15.89:0/1898292322 wait complete. 2023-10-10T20:46:24.327 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-10T20:46:24.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.441+0000 7fbff0d9f700 1 Processor -- start 2023-10-10T20:46:24.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.441+0000 7fbff0d9f700 1 -- start start 2023-10-10T20:46:24.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.442+0000 7fbff0d9f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfec06bcd0 0x7fbfec0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.442+0000 7fbff0d9f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfec15d7e0 con 0x7fbfec15df50 2023-10-10T20:46:24.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.442+0000 7fbff0d9f700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfec15dac0 con 0x7fbfec06c0a0 2023-10-10T20:46:24.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.442+0000 7fbff0d9f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec15aa30 con 0x7fbfec06bcd0 2023-10-10T20:46:24.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.442+0000 7fbfead9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbfec15df50 0x7fbfec0c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51740/0 (socket says 172.21.15.89:51740) 2023-10-10T20:46:24.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.442+0000 7fbfead9d700 1 -- 172.21.15.89:0/2562632430 learned_addr learned my addr 172.21.15.89:0/2562632430 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:24.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.442+0000 7fbfea59c700 1 --2- 172.21.15.89:0/2562632430 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfec06bcd0 0x7fbfec0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.442+0000 7fbfe959a700 1 -- 172.21.15.89:0/2562632430 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726138774 0 0) 0x7fbfec15d7e0 con 0x7fbfec15df50 2023-10-10T20:46:24.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.442+0000 7fbfe959a700 1 -- 172.21.15.89:0/2562632430 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfd4003570 con 0x7fbfec15df50 2023-10-10T20:46:24.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.442+0000 7fbfe959a700 1 -- 172.21.15.89:0/2562632430 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 886420447 0 0) 0x7fbfd4003570 con 0x7fbfec15df50 2023-10-10T20:46:24.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.442+0000 7fbfe959a700 1 -- 172.21.15.89:0/2562632430 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfec06bcd0 msgr2=0x7fbfec0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:24.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.442+0000 7fbfe959a700 1 --2- 172.21.15.89:0/2562632430 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfec06bcd0 0x7fbfec0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.442+0000 7fbfe959a700 1 -- 172.21.15.89:0/2562632430 >> v1:172.21.15.111:6789/0 conn(0x7fbfec06c0a0 legacy=0x7fbfec15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.442+0000 7fbfe959a700 1 -- 172.21.15.89:0/2562632430 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec15ad10 con 0x7fbfec15df50 2023-10-10T20:46:24.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.442+0000 7fbfe959a700 1 -- 172.21.15.89:0/2562632430 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fbfe40017f0 con 0x7fbfec15df50 2023-10-10T20:46:24.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.442+0000 7fbfe959a700 1 -- 172.21.15.89:0/2562632430 >> v1:172.21.15.89:6789/0 conn(0x7fbfec15df50 legacy=0x7fbfec0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.442+0000 7fbfe959a700 1 --2- 172.21.15.89:0/2562632430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd4003a90 0x7fbfd4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.442+0000 7fbfe959a700 1 --2- 172.21.15.89:0/2562632430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd4004810 0x7fbfd4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.442+0000 7fbfe959a700 1 --2- 172.21.15.89:0/2562632430 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfec06bcd0 0x7fbfd4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.443+0000 7fbfe959a700 1 -- 172.21.15.89:0/2562632430 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec15aa30 con 0x7fbfd4004810 2023-10-10T20:46:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.443+0000 7fbfe959a700 1 -- 172.21.15.89:0/2562632430 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd400bb90 con 0x7fbfd4003a90 2023-10-10T20:46:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.443+0000 7fbfe959a700 1 -- 172.21.15.89:0/2562632430 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd400bd50 con 0x7fbfec06bcd0 2023-10-10T20:46:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.443+0000 7fbfe9d9b700 1 --2- 172.21.15.89:0/2562632430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd4004810 0x7fbfd4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.443+0000 7fbfea59c700 1 --2- 172.21.15.89:0/2562632430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd4003a90 0x7fbfd4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.443+0000 7fbfead9d700 1 --2- 172.21.15.89:0/2562632430 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfec06bcd0 0x7fbfd4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.443+0000 7fbfe9d9b700 1 -- 172.21.15.89:0/2562632430 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfec06bcd0 msgr2=0x7fbfd4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:24.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.443+0000 7fbfe9d9b700 1 --2- 172.21.15.89:0/2562632430 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfec06bcd0 0x7fbfd4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.443+0000 7fbfe9d9b700 1 -- 172.21.15.89:0/2562632430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd4003a90 msgr2=0x7fbfd4003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:24.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.443+0000 7fbfe9d9b700 1 --2- 172.21.15.89:0/2562632430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd4003a90 0x7fbfd4003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.443+0000 7fbfe9d9b700 1 -- 172.21.15.89:0/2562632430 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd4003570 con 0x7fbfd4004810 2023-10-10T20:46:24.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.443+0000 7fbfead9d700 1 --2- 172.21.15.89:0/2562632430 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfec06bcd0 0x7fbfd4005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:24.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.443+0000 7fbfe9d9b700 1 --2- 172.21.15.89:0/2562632430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd4004810 0x7fbfd4004c80 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00019a0 tx=0x7fbfe0007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f0b8042dc2eed8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:24.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.443+0000 7fbfe959a700 1 -- 172.21.15.89:0/2562632430 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe000c5d0 con 0x7fbfd4004810 2023-10-10T20:46:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.443+0000 7fbfe959a700 1 -- 172.21.15.89:0/2562632430 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe000c750 con 0x7fbfd4004810 2023-10-10T20:46:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.443+0000 7fbfe959a700 1 -- 172.21.15.89:0/2562632430 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe000cb10 con 0x7fbfd4004810 2023-10-10T20:46:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.444+0000 7fbff0d9f700 1 -- 172.21.15.89:0/2562632430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd4004810 msgr2=0x7fbfd4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.444+0000 7fbff0d9f700 1 --2- 172.21.15.89:0/2562632430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd4004810 0x7fbfd4004c80 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00019a0 tx=0x7fbfe0007d10 comp rx=0 tx=0).stop 2023-10-10T20:46:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.444+0000 7fbfead9d700 1 -- 172.21.15.89:0/2562632430 reap_dead start 2023-10-10T20:46:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.444+0000 7fbff0d9f700 1 -- 172.21.15.89:0/2562632430 shutdown_connections 2023-10-10T20:46:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.444+0000 7fbff0d9f700 1 -- 172.21.15.89:0/2562632430 >> 172.21.15.89:0/2562632430 conn(0x7fbfec0ce600 msgr2=0x7fbfec0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.444+0000 7fbff0d9f700 1 -- 172.21.15.89:0/2562632430 shutdown_connections 2023-10-10T20:46:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.444+0000 7fbff0d9f700 1 -- 172.21.15.89:0/2562632430 wait complete. 2023-10-10T20:46:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.445+0000 7fbff0d9f700 1 Processor -- start 2023-10-10T20:46:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.445+0000 7fbff0d9f700 1 -- start start 2023-10-10T20:46:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbff0d9f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd4004810 0x7fbfec1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbff0d9f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfec16a690 0x7fbfec0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbff0d9f700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfec0cc1b0 0x7fbfec0cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbff0d9f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec0ccb90 con 0x7fbfec16a690 2023-10-10T20:46:24.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbff0d9f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec0b0060 con 0x7fbfd4004810 2023-10-10T20:46:24.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbff0d9f700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec16a200 con 0x7fbfec0cc1b0 2023-10-10T20:46:24.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbfe9d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfec16a690 0x7fbfec0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbfe9d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfec16a690 0x7fbfec0b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43646/0 (socket says 172.21.15.89:43646) 2023-10-10T20:46:24.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbfe9d9b700 1 -- 172.21.15.89:0/915644711 learned_addr learned my addr 172.21.15.89:0/915644711 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:24.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbfea59c700 1 --2- 172.21.15.89:0/915644711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd4004810 0x7fbfec1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbfead9d700 1 --2- 172.21.15.89:0/915644711 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfec0cc1b0 0x7fbfec0cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbfe9d9b700 1 -- 172.21.15.89:0/915644711 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfec0cc1b0 msgr2=0x7fbfec0cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbfe9d9b700 1 --2- 172.21.15.89:0/915644711 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfec0cc1b0 0x7fbfec0cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbfe9d9b700 1 -- 172.21.15.89:0/915644711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd4004810 msgr2=0x7fbfec1567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbfe9d9b700 1 --2- 172.21.15.89:0/915644711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd4004810 0x7fbfec1567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbfe9d9b700 1 -- 172.21.15.89:0/915644711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec156d20 con 0x7fbfec16a690 2023-10-10T20:46:24.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbfe9d9b700 1 --2- 172.21.15.89:0/915644711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfec16a690 0x7fbfec0b13a0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400bb60 tx=0x7fbfe0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:24.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbfda7fc700 1 -- 172.21.15.89:0/915644711 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe000cad0 con 0x7fbfec16a690 2023-10-10T20:46:24.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbfda7fc700 1 -- 172.21.15.89:0/915644711 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe000ce80 con 0x7fbfec16a690 2023-10-10T20:46:24.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbff0d9f700 1 -- 172.21.15.89:0/915644711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfec169bd0 con 0x7fbfec16a690 2023-10-10T20:46:24.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbfda7fc700 1 -- 172.21.15.89:0/915644711 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe001e610 con 0x7fbfec16a690 2023-10-10T20:46:24.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.446+0000 7fbff0d9f700 1 -- 172.21.15.89:0/915644711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfec0b0590 con 0x7fbfec16a690 2023-10-10T20:46:24.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.447+0000 7fbff0d9f700 1 -- 172.21.15.89:0/915644711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfec0b0780 con 0x7fbfec16a690 2023-10-10T20:46:24.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.448+0000 7fbfda7fc700 1 -- 172.21.15.89:0/915644711 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbfe0019030 con 0x7fbfec16a690 2023-10-10T20:46:24.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.448+0000 7fbfda7fc700 1 --2- 172.21.15.89:0/915644711 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbfcc067320 0x7fbfcc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.448+0000 7fbfda7fc700 1 -- 172.21.15.89:0/915644711 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fbfe0085ec0 con 0x7fbfec16a690 2023-10-10T20:46:24.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.448+0000 7fbfea59c700 1 --2- 172.21.15.89:0/915644711 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbfcc067320 0x7fbfcc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.448+0000 7fbfea59c700 1 --2- 172.21.15.89:0/915644711 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbfcc067320 0x7fbfcc0697d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc006f00 tx=0x7fbfdc0091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:24.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.452+0000 7fbfda7fc700 1 -- 172.21.15.89:0/915644711 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfe0052f10 con 0x7fbfec16a690 2023-10-10T20:46:24.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.637+0000 7fbff0d9f700 1 -- 172.21.15.89:0/915644711 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfec0ccb90 con 0x7fbfcc067320 2023-10-10T20:46:24.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.637+0000 7fbfda7fc700 1 -- 172.21.15.89:0/915644711 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbfec0ccb90 con 0x7fbfcc067320 2023-10-10T20:46:24.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.639+0000 7fbff0d9f700 1 -- 172.21.15.89:0/915644711 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbfcc067320 msgr2=0x7fbfcc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.639+0000 7fbff0d9f700 1 --2- 172.21.15.89:0/915644711 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbfcc067320 0x7fbfcc0697d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc006f00 tx=0x7fbfdc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.639+0000 7fbff0d9f700 1 -- 172.21.15.89:0/915644711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfec16a690 msgr2=0x7fbfec0b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.639+0000 7fbff0d9f700 1 --2- 172.21.15.89:0/915644711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfec16a690 0x7fbfec0b13a0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400bb60 tx=0x7fbfe0015000 comp rx=0 tx=0).stop 2023-10-10T20:46:24.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.639+0000 7fbff0d9f700 1 -- 172.21.15.89:0/915644711 shutdown_connections 2023-10-10T20:46:24.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.639+0000 7fbff0d9f700 1 --2- 172.21.15.89:0/915644711 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbfec0cc1b0 0x7fbfec0cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.639+0000 7fbff0d9f700 1 --2- 172.21.15.89:0/915644711 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbfcc067320 0x7fbfcc0697d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.639+0000 7fbff0d9f700 1 --2- 172.21.15.89:0/915644711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfd4004810 0x7fbfec1567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.639+0000 7fbff0d9f700 1 --2- 172.21.15.89:0/915644711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfec16a690 0x7fbfec0b13a0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.639+0000 7fbff0d9f700 1 -- 172.21.15.89:0/915644711 >> 172.21.15.89:0/915644711 conn(0x7fbfec0ce600 msgr2=0x7fbfec15b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:24.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.639+0000 7fbff0d9f700 1 -- 172.21.15.89:0/915644711 shutdown_connections 2023-10-10T20:46:24.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.639+0000 7fbff0d9f700 1 -- 172.21.15.89:0/915644711 wait complete. 2023-10-10T20:46:24.645 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 3 to deep-scrub 2023-10-10T20:46:24.652 INFO:tasks.ceph:Scrubbing osd.4 2023-10-10T20:46:24.653 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-10T20:46:24.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.769+0000 7f81f3519700 1 Processor -- start 2023-10-10T20:46:24.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.770+0000 7f81f3519700 1 -- start start 2023-10-10T20:46:24.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.770+0000 7f81f3519700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f81ec159ec0 0x7f81ec0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.770+0000 7f81f3519700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f81ec0cf350 0x7f81ec0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.770+0000 7f81f3519700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ec0b5b00 con 0x7f81ec0b53d0 2023-10-10T20:46:24.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.770+0000 7f81f3519700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ec0b5850 con 0x7f81ec0cf350 2023-10-10T20:46:24.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.770+0000 7f81f3519700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ec0b5da0 con 0x7f81ec159ec0 2023-10-10T20:46:24.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.771+0000 7f81f1ab6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f81ec0cf350 0x7f81ec0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.771+0000 7f81f0ab4700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f81ec0b53d0 0x7f81ec0cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51748/0 (socket says 172.21.15.89:51748) 2023-10-10T20:46:24.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.771+0000 7f81f0ab4700 1 -- 172.21.15.89:0/904242385 learned_addr learned my addr 172.21.15.89:0/904242385 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:24.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.771+0000 7f81f1ab6700 1 --2- 172.21.15.89:0/904242385 >> v2:172.21.15.89:3300/0 conn(0x7f81ec0cf350 0x7f81ec0cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43660/0 (socket says 172.21.15.89:43660) 2023-10-10T20:46:24.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.771+0000 7f81f12b5700 1 --2- 172.21.15.89:0/904242385 >> v2:172.21.15.111:3300/0 conn(0x7f81ec159ec0 0x7f81ec0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.771+0000 7f81f1ab6700 1 -- 172.21.15.89:0/904242385 >> v2:172.21.15.111:3300/0 conn(0x7f81ec159ec0 msgr2=0x7f81ec0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:24.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.771+0000 7f81f1ab6700 1 --2- 172.21.15.89:0/904242385 >> v2:172.21.15.111:3300/0 conn(0x7f81ec159ec0 0x7f81ec0b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.771+0000 7f81f1ab6700 1 -- 172.21.15.89:0/904242385 >> v1:172.21.15.89:6789/0 conn(0x7f81ec0b53d0 legacy=0x7f81ec0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.771+0000 7f81f1ab6700 1 -- 172.21.15.89:0/904242385 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec0b6080 con 0x7f81ec0cf350 2023-10-10T20:46:24.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.771+0000 7f81e3fff700 1 -- 172.21.15.89:0/904242385 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1294797123 0 0) 0x7f81ec0b5b00 con 0x7f81ec0b53d0 2023-10-10T20:46:24.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.771+0000 7f81f12b5700 1 --2- 172.21.15.89:0/904242385 >> v2:172.21.15.111:3300/0 conn(0x7f81ec159ec0 0x7f81ec0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:24.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.771+0000 7f81f1ab6700 1 --2- 172.21.15.89:0/904242385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0cf350 0x7f81ec0cd010 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f81ec089120 tx=0x7f81e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8ce2d7eecf6e88 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:24.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.771+0000 7f81e3fff700 1 -- 172.21.15.89:0/904242385 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e400b6b0 con 0x7f81ec0cf350 2023-10-10T20:46:24.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.771+0000 7f81e3fff700 1 -- 172.21.15.89:0/904242385 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e400b810 con 0x7f81ec0cf350 2023-10-10T20:46:24.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.771+0000 7f81e3fff700 1 -- 172.21.15.89:0/904242385 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e400bbb0 con 0x7f81ec0cf350 2023-10-10T20:46:24.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.772+0000 7f81f3519700 1 -- 172.21.15.89:0/904242385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0cf350 msgr2=0x7f81ec0cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.772+0000 7f81f3519700 1 --2- 172.21.15.89:0/904242385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0cf350 0x7f81ec0cd010 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f81ec089120 tx=0x7f81e40091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.772+0000 7f81f3519700 1 -- 172.21.15.89:0/904242385 shutdown_connections 2023-10-10T20:46:24.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.772+0000 7f81f3519700 1 --2- 172.21.15.89:0/904242385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0cf350 0x7f81ec0cd010 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.772+0000 7f81f3519700 1 --2- 172.21.15.89:0/904242385 >> v2:172.21.15.111:3300/0 conn(0x7f81ec159ec0 0x7f81ec0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.772+0000 7f81f3519700 1 -- 172.21.15.89:0/904242385 >> 172.21.15.89:0/904242385 conn(0x7f81ec0999f0 msgr2=0x7f81ec0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:24.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.772+0000 7f81f3519700 1 -- 172.21.15.89:0/904242385 shutdown_connections 2023-10-10T20:46:24.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.772+0000 7f81f3519700 1 -- 172.21.15.89:0/904242385 wait complete. 2023-10-10T20:46:24.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.773+0000 7f81f3519700 1 Processor -- start 2023-10-10T20:46:24.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.773+0000 7f81f3519700 1 -- start start 2023-10-10T20:46:24.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f3519700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81ec0b53d0 0x7f81ec0690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f3519700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0cf350 0x7f81ec0c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f3519700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f81ec159ec0 0x7f81ec069f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f3519700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec068390 con 0x7f81ec0cf350 2023-10-10T20:46:24.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f3519700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec09db70 con 0x7f81ec0b53d0 2023-10-10T20:46:24.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f3519700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec06a9c0 con 0x7f81ec159ec0 2023-10-10T20:46:24.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f0ab4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0cf350 0x7f81ec0c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f0ab4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0cf350 0x7f81ec0c6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43668/0 (socket says 172.21.15.89:43668) 2023-10-10T20:46:24.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f0ab4700 1 -- 172.21.15.89:0/3994067006 learned_addr learned my addr 172.21.15.89:0/3994067006 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:24.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f12b5700 1 --2- 172.21.15.89:0/3994067006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81ec0b53d0 0x7f81ec0690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f1ab6700 1 --2- 172.21.15.89:0/3994067006 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f81ec159ec0 0x7f81ec069f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f0ab4700 1 -- 172.21.15.89:0/3994067006 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f81ec159ec0 msgr2=0x7f81ec069f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f0ab4700 1 --2- 172.21.15.89:0/3994067006 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f81ec159ec0 0x7f81ec069f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f0ab4700 1 -- 172.21.15.89:0/3994067006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81ec0b53d0 msgr2=0x7f81ec0690c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f0ab4700 1 --2- 172.21.15.89:0/3994067006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81ec0b53d0 0x7f81ec0690c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f0ab4700 1 -- 172.21.15.89:0/3994067006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e4009420 con 0x7f81ec0cf350 2023-10-10T20:46:24.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f12b5700 1 --2- 172.21.15.89:0/3994067006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81ec0b53d0 0x7f81ec0690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:46:24.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f1ab6700 1 --2- 172.21.15.89:0/3994067006 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f81ec159ec0 0x7f81ec069f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:46:24.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f0ab4700 1 --2- 172.21.15.89:0/3994067006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0cf350 0x7f81ec0c6930 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00c5b0 tx=0x7f81dc001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:24.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81e17fa700 1 -- 172.21.15.89:0/3994067006 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc005b30 con 0x7f81ec0cf350 2023-10-10T20:46:24.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81e17fa700 1 -- 172.21.15.89:0/3994067006 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc005c90 con 0x7f81ec0cf350 2023-10-10T20:46:24.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f3519700 1 -- 172.21.15.89:0/3994067006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ec0685e0 con 0x7f81ec0cf350 2023-10-10T20:46:24.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81e17fa700 1 -- 172.21.15.89:0/3994067006 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc007300 con 0x7f81ec0cf350 2023-10-10T20:46:24.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.774+0000 7f81f3519700 1 -- 172.21.15.89:0/3994067006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ec068ba0 con 0x7f81ec0cf350 2023-10-10T20:46:24.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.775+0000 7f81f3519700 1 -- 172.21.15.89:0/3994067006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f81ec063c90 con 0x7f81ec0cf350 2023-10-10T20:46:24.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.776+0000 7f81e17fa700 1 -- 172.21.15.89:0/3994067006 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f81dc01b3b0 con 0x7f81ec0cf350 2023-10-10T20:46:24.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.776+0000 7f81e17fa700 1 --2- 172.21.15.89:0/3994067006 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f81d0067320 0x7f81d00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.776+0000 7f81e17fa700 1 -- 172.21.15.89:0/3994067006 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f81dc083eb0 con 0x7f81ec0cf350 2023-10-10T20:46:24.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.776+0000 7f81f12b5700 1 --2- 172.21.15.89:0/3994067006 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f81d0067320 0x7f81d00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.776+0000 7f81e17fa700 1 --2- 172.21.15.89:0/3994067006 >> [v2:172.21.15.111:6800/91095,v1:172.21.15.111:6801/91095] conn(0x7f81d006d170 0x7f81d006f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.776+0000 7f81e17fa700 1 -- 172.21.15.89:0/3994067006 --> [v2:172.21.15.111:6800/91095,v1:172.21.15.111:6801/91095] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f81d006fc50 con 0x7f81d006d170 2023-10-10T20:46:24.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.776+0000 7f81e17fa700 1 -- 172.21.15.89:0/3994067006 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81dc086410 con 0x7f81ec0cf350 2023-10-10T20:46:24.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.776+0000 7f81f12b5700 1 --2- 172.21.15.89:0/3994067006 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f81d0067320 0x7f81d00697d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f81e80094d0 tx=0x7f81e800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:24.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.776+0000 7f81f1ab6700 1 --2- 172.21.15.89:0/3994067006 >> [v2:172.21.15.111:6800/91095,v1:172.21.15.111:6801/91095] conn(0x7f81d006d170 0x7f81d006f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.777+0000 7f81f1ab6700 1 --2- 172.21.15.89:0/3994067006 >> [v2:172.21.15.111:6800/91095,v1:172.21.15.111:6801/91095] conn(0x7f81d006d170 0x7f81d006f580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:24.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.778+0000 7f81e17fa700 1 -- 172.21.15.89:0/3994067006 <== osd.4 v2:172.21.15.111:6800/91095 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f81d006fc50 con 0x7f81d006d170 2023-10-10T20:46:24.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.791+0000 7f81f3519700 1 -- 172.21.15.89:0/3994067006 --> [v2:172.21.15.111:6800/91095,v1:172.21.15.111:6801/91095] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f81ec0718b0 con 0x7f81d006d170 2023-10-10T20:46:24.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.792+0000 7f81e17fa700 1 -- 172.21.15.89:0/3994067006 <== osd.4 v2:172.21.15.111:6800/91095 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f81ec0718b0 con 0x7f81d006d170 2023-10-10T20:46:24.802 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:46:24.803 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-10T20:46:24.803 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:46:24.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.792+0000 7f81f3519700 1 -- 172.21.15.89:0/3994067006 >> [v2:172.21.15.111:6800/91095,v1:172.21.15.111:6801/91095] conn(0x7f81d006d170 msgr2=0x7f81d006f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.792+0000 7f81f3519700 1 --2- 172.21.15.89:0/3994067006 >> [v2:172.21.15.111:6800/91095,v1:172.21.15.111:6801/91095] conn(0x7f81d006d170 0x7f81d006f580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.792+0000 7f81f3519700 1 -- 172.21.15.89:0/3994067006 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f81d0067320 msgr2=0x7f81d00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.792+0000 7f81f3519700 1 --2- 172.21.15.89:0/3994067006 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f81d0067320 0x7f81d00697d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f81e80094d0 tx=0x7f81e800b040 comp rx=0 tx=0).stop 2023-10-10T20:46:24.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.792+0000 7f81f3519700 1 -- 172.21.15.89:0/3994067006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0cf350 msgr2=0x7f81ec0c6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.792+0000 7f81f3519700 1 --2- 172.21.15.89:0/3994067006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0cf350 0x7f81ec0c6930 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00c5b0 tx=0x7f81dc001070 comp rx=0 tx=0).stop 2023-10-10T20:46:24.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.792+0000 7f81f1ab6700 1 -- 172.21.15.89:0/3994067006 reap_dead start 2023-10-10T20:46:24.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.792+0000 7f81f3519700 1 -- 172.21.15.89:0/3994067006 shutdown_connections 2023-10-10T20:46:24.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.792+0000 7f81f3519700 1 -- 172.21.15.89:0/3994067006 >> 172.21.15.89:0/3994067006 conn(0x7f81ec0999f0 msgr2=0x7f81ec0c5880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:24.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.792+0000 7f81f3519700 1 -- 172.21.15.89:0/3994067006 shutdown_connections 2023-10-10T20:46:24.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.792+0000 7f81f3519700 1 -- 172.21.15.89:0/3994067006 wait complete. 2023-10-10T20:46:24.808 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-10T20:46:24.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.921+0000 7f4b5e953700 1 Processor -- start 2023-10-10T20:46:24.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.921+0000 7f4b5e953700 1 -- start start 2023-10-10T20:46:24.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.922+0000 7f4b5e953700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4b58159420 0x7f4b580b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.922+0000 7f4b5e953700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5815d230 0x7f4b5815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.922+0000 7f4b5e953700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b580b2980 con 0x7f4b581597f0 2023-10-10T20:46:24.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.922+0000 7f4b5e953700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5815dbb0 con 0x7f4b58159420 2023-10-10T20:46:24.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.922+0000 7f4b5e953700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580b11d0 con 0x7f4b5815d230 2023-10-10T20:46:24.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.922+0000 7f4b5e152700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5815d230 0x7f4b5815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.922+0000 7f4b5d951700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4b58159420 0x7f4b580b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.923+0000 7f4b5e152700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5815d230 0x7f4b5815d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:36864/0 (socket says 172.21.15.89:36864) 2023-10-10T20:46:24.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.923+0000 7f4b5e152700 1 -- 172.21.15.89:0/3744059530 learned_addr learned my addr 172.21.15.89:0/3744059530 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:24.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.923+0000 7f4b5d150700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f4b581597f0 0x7f4b580b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:32884/0 (socket says 172.21.15.89:32884) 2023-10-10T20:46:24.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.923+0000 7f4b5d951700 1 -- 172.21.15.89:0/3744059530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5815d230 msgr2=0x7f4b5815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:24.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.923+0000 7f4b5d951700 1 --2- 172.21.15.89:0/3744059530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5815d230 0x7f4b5815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.923+0000 7f4b5d951700 1 -- 172.21.15.89:0/3744059530 >> v1:172.21.15.111:6789/0 conn(0x7f4b581597f0 legacy=0x7f4b580b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.923+0000 7f4b5d951700 1 -- 172.21.15.89:0/3744059530 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b580b14b0 con 0x7f4b58159420 2023-10-10T20:46:24.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.923+0000 7f4b5d951700 1 --2- 172.21.15.89:0/3744059530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b58159420 0x7f4b580b5f40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4b540098e0 tx=0x7f4b540091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9913cb3960bb1da server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:24.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.923+0000 7f4b5c94f700 1 -- 172.21.15.89:0/3744059530 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5400b640 con 0x7f4b58159420 2023-10-10T20:46:24.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.923+0000 7f4b5c94f700 1 -- 172.21.15.89:0/3744059530 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5400b7a0 con 0x7f4b58159420 2023-10-10T20:46:24.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.923+0000 7f4b5c94f700 1 -- 172.21.15.89:0/3744059530 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5400bb40 con 0x7f4b58159420 2023-10-10T20:46:24.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.924+0000 7f4b5e953700 1 -- 172.21.15.89:0/3744059530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b58159420 msgr2=0x7f4b580b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.924+0000 7f4b5e953700 1 --2- 172.21.15.89:0/3744059530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b58159420 0x7f4b580b5f40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4b540098e0 tx=0x7f4b540091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.924+0000 7f4b5e953700 1 -- 172.21.15.89:0/3744059530 shutdown_connections 2023-10-10T20:46:24.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.924+0000 7f4b5e953700 1 --2- 172.21.15.89:0/3744059530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b58159420 0x7f4b580b5f40 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.924+0000 7f4b5e953700 1 --2- 172.21.15.89:0/3744059530 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5815d230 0x7f4b5815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.924+0000 7f4b5e953700 1 -- 172.21.15.89:0/3744059530 >> 172.21.15.89:0/3744059530 conn(0x7f4b580d0660 msgr2=0x7f4b580d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:24.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.924+0000 7f4b5e953700 1 -- 172.21.15.89:0/3744059530 shutdown_connections 2023-10-10T20:46:24.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.924+0000 7f4b5e953700 1 -- 172.21.15.89:0/3744059530 wait complete. 2023-10-10T20:46:24.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.925+0000 7f4b5e953700 1 Processor -- start 2023-10-10T20:46:24.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.925+0000 7f4b5e953700 1 -- start start 2023-10-10T20:46:24.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b5e953700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b58159420 0x7f4b580c5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b5e953700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b581597f0 0x7f4b5816cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b5e953700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5815d230 0x7f4b5816d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b5e953700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5815aab0 con 0x7f4b581597f0 2023-10-10T20:46:24.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b5e953700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5816db10 con 0x7f4b58159420 2023-10-10T20:46:24.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b5e953700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5816ddf0 con 0x7f4b5815d230 2023-10-10T20:46:24.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b5d150700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b581597f0 0x7f4b5816cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b5d150700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b581597f0 0x7f4b5816cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43680/0 (socket says 172.21.15.89:43680) 2023-10-10T20:46:24.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b5d951700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b58159420 0x7f4b580c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b5d150700 1 -- 172.21.15.89:0/4138161162 learned_addr learned my addr 172.21.15.89:0/4138161162 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:24.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b5e152700 1 --2- 172.21.15.89:0/4138161162 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5815d230 0x7f4b5816d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b5d150700 1 -- 172.21.15.89:0/4138161162 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5815d230 msgr2=0x7f4b5816d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b5d150700 1 --2- 172.21.15.89:0/4138161162 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5815d230 0x7f4b5816d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b5d150700 1 -- 172.21.15.89:0/4138161162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b58159420 msgr2=0x7f4b580c5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:24.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b5d150700 1 --2- 172.21.15.89:0/4138161162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b58159420 0x7f4b580c5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:24.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b5d150700 1 -- 172.21.15.89:0/4138161162 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b54009420 con 0x7f4b581597f0 2023-10-10T20:46:24.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b5d951700 1 --2- 172.21.15.89:0/4138161162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b58159420 0x7f4b580c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:46:24.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b5d150700 1 --2- 172.21.15.89:0/4138161162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b581597f0 0x7f4b5816cfa0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800c790 tx=0x7f4b48007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:24.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b4dffb700 1 -- 172.21.15.89:0/4138161162 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b48005a70 con 0x7f4b581597f0 2023-10-10T20:46:24.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b4dffb700 1 -- 172.21.15.89:0/4138161162 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b48005bf0 con 0x7f4b581597f0 2023-10-10T20:46:24.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b5e953700 1 -- 172.21.15.89:0/4138161162 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5815a460 con 0x7f4b581597f0 2023-10-10T20:46:24.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.926+0000 7f4b4dffb700 1 -- 172.21.15.89:0/4138161162 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b480070d0 con 0x7f4b581597f0 2023-10-10T20:46:24.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.927+0000 7f4b5e953700 1 -- 172.21.15.89:0/4138161162 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b58154460 con 0x7f4b581597f0 2023-10-10T20:46:24.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.927+0000 7f4b5e953700 1 -- 172.21.15.89:0/4138161162 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5815ac60 con 0x7f4b581597f0 2023-10-10T20:46:24.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.928+0000 7f4b4dffb700 1 -- 172.21.15.89:0/4138161162 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4b48007480 con 0x7f4b581597f0 2023-10-10T20:46:24.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.928+0000 7f4b4dffb700 1 --2- 172.21.15.89:0/4138161162 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4b3c067320 0x7f4b3c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:24.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.928+0000 7f4b4dffb700 1 -- 172.21.15.89:0/4138161162 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4b48083da0 con 0x7f4b581597f0 2023-10-10T20:46:24.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.928+0000 7f4b5d951700 1 --2- 172.21.15.89:0/4138161162 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4b3c067320 0x7f4b3c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:24.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.929+0000 7f4b5d951700 1 --2- 172.21.15.89:0/4138161162 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4b3c067320 0x7f4b3c0697d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4b540093f0 tx=0x7f4b54017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:24.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:24.932+0000 7f4b4dffb700 1 -- 172.21.15.89:0/4138161162 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b5815ac60 con 0x7f4b581597f0 2023-10-10T20:46:25.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.116+0000 7f4b5e953700 1 -- 172.21.15.89:0/4138161162 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b5815ac60 con 0x7f4b3c067320 2023-10-10T20:46:25.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.116+0000 7f4b4dffb700 1 -- 172.21.15.89:0/4138161162 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4b5815ac60 con 0x7f4b3c067320 2023-10-10T20:46:25.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.118+0000 7f4b5e953700 1 -- 172.21.15.89:0/4138161162 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4b3c067320 msgr2=0x7f4b3c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.118+0000 7f4b5e953700 1 --2- 172.21.15.89:0/4138161162 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4b3c067320 0x7f4b3c0697d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4b540093f0 tx=0x7f4b54017000 comp rx=0 tx=0).stop 2023-10-10T20:46:25.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.118+0000 7f4b5e953700 1 -- 172.21.15.89:0/4138161162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b581597f0 msgr2=0x7f4b5816cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.118+0000 7f4b5e953700 1 --2- 172.21.15.89:0/4138161162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b581597f0 0x7f4b5816cfa0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800c790 tx=0x7f4b48007f40 comp rx=0 tx=0).stop 2023-10-10T20:46:25.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.118+0000 7f4b5e953700 1 -- 172.21.15.89:0/4138161162 shutdown_connections 2023-10-10T20:46:25.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.118+0000 7f4b5e953700 1 --2- 172.21.15.89:0/4138161162 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f4b5815d230 0x7f4b5816d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.118+0000 7f4b5e953700 1 --2- 172.21.15.89:0/4138161162 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f4b3c067320 0x7f4b3c0697d0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.118+0000 7f4b5e953700 1 --2- 172.21.15.89:0/4138161162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b58159420 0x7f4b580c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.118+0000 7f4b5e953700 1 --2- 172.21.15.89:0/4138161162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b581597f0 0x7f4b5816cfa0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.118+0000 7f4b5e953700 1 -- 172.21.15.89:0/4138161162 >> 172.21.15.89:0/4138161162 conn(0x7f4b580d0660 msgr2=0x7f4b581572d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:25.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.118+0000 7f4b5e953700 1 -- 172.21.15.89:0/4138161162 shutdown_connections 2023-10-10T20:46:25.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.119+0000 7f4b5e953700 1 -- 172.21.15.89:0/4138161162 wait complete. 2023-10-10T20:46:25.125 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 4 to deep-scrub 2023-10-10T20:46:25.132 INFO:tasks.ceph:Scrubbing osd.5 2023-10-10T20:46:25.132 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-10T20:46:25.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.249+0000 7fc033908700 1 Processor -- start 2023-10-10T20:46:25.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.250+0000 7fc033908700 1 -- start start 2023-10-10T20:46:25.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.250+0000 7fc033908700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc02c159ec0 0x7fc02c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.250+0000 7fc033908700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc02c0b53d0 0x7fc02c0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.250+0000 7fc033908700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc02c0cce40 0x7fc02c0cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.251+0000 7fc033908700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02c0b5b00 con 0x7fc02c0cce40 2023-10-10T20:46:25.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.251+0000 7fc033908700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02c0b5da0 con 0x7fc02c0b53d0 2023-10-10T20:46:25.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.251+0000 7fc033908700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c0b6080 con 0x7fc02c159ec0 2023-10-10T20:46:25.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.251+0000 7fc031ea5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc02c0cce40 0x7fc02c0cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.251+0000 7fc031ea5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc02c0cce40 0x7fc02c0cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43684/0 (socket says 172.21.15.89:43684) 2023-10-10T20:46:25.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.251+0000 7fc031ea5700 1 -- 172.21.15.89:0/2695886394 learned_addr learned my addr 172.21.15.89:0/2695886394 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:25.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.251+0000 7fc031ea5700 1 -- 172.21.15.89:0/2695886394 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc02c159ec0 msgr2=0x7fc02c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:25.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.251+0000 7fc031ea5700 1 --2- 172.21.15.89:0/2695886394 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc02c159ec0 0x7fc02c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.251+0000 7fc031ea5700 1 -- 172.21.15.89:0/2695886394 >> v2:172.21.15.111:3300/0 conn(0x7fc02c0b53d0 msgr2=0x7fc02c0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:25.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.251+0000 7fc031ea5700 1 --2- 172.21.15.89:0/2695886394 >> v2:172.21.15.111:3300/0 conn(0x7fc02c0b53d0 0x7fc02c0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.251+0000 7fc031ea5700 1 -- 172.21.15.89:0/2695886394 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c0cc100 con 0x7fc02c0cce40 2023-10-10T20:46:25.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.251+0000 7fc031ea5700 1 --2- 172.21.15.89:0/2695886394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc02c0cce40 0x7fc02c0cd280 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fc02c089880 tx=0x7fc024009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72101c4b375cf84 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:25.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.251+0000 7fc023fff700 1 -- 172.21.15.89:0/2695886394 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02400b860 con 0x7fc02c0cce40 2023-10-10T20:46:25.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.251+0000 7fc023fff700 1 -- 172.21.15.89:0/2695886394 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02400b9c0 con 0x7fc02c0cce40 2023-10-10T20:46:25.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.251+0000 7fc023fff700 1 -- 172.21.15.89:0/2695886394 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02400bd60 con 0x7fc02c0cce40 2023-10-10T20:46:25.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.252+0000 7fc033908700 1 -- 172.21.15.89:0/2695886394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc02c0cce40 msgr2=0x7fc02c0cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.252+0000 7fc033908700 1 --2- 172.21.15.89:0/2695886394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc02c0cce40 0x7fc02c0cd280 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fc02c089880 tx=0x7fc024009250 comp rx=0 tx=0).stop 2023-10-10T20:46:25.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.252+0000 7fc033908700 1 -- 172.21.15.89:0/2695886394 shutdown_connections 2023-10-10T20:46:25.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.252+0000 7fc033908700 1 --2- 172.21.15.89:0/2695886394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc02c0cce40 0x7fc02c0cd280 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.252+0000 7fc033908700 1 --2- 172.21.15.89:0/2695886394 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc02c159ec0 0x7fc02c0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.252+0000 7fc033908700 1 --2- 172.21.15.89:0/2695886394 >> v2:172.21.15.111:3300/0 conn(0x7fc02c0b53d0 0x7fc02c0cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.252+0000 7fc033908700 1 -- 172.21.15.89:0/2695886394 >> 172.21.15.89:0/2695886394 conn(0x7fc02c0999f0 msgr2=0x7fc02c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:25.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.252+0000 7fc033908700 1 -- 172.21.15.89:0/2695886394 shutdown_connections 2023-10-10T20:46:25.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.252+0000 7fc033908700 1 -- 172.21.15.89:0/2695886394 wait complete. 2023-10-10T20:46:25.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.253+0000 7fc033908700 1 Processor -- start 2023-10-10T20:46:25.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.253+0000 7fc033908700 1 -- start start 2023-10-10T20:46:25.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc033908700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc02c0b53d0 0x7fc02c153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc033908700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc02c0cce40 0x7fc02c0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc033908700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc02c159ec0 0x7fc02c15be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc033908700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c0691d0 con 0x7fc02c159ec0 2023-10-10T20:46:25.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc033908700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c068ea0 con 0x7fc02c0b53d0 2023-10-10T20:46:25.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc033908700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c0c8e80 con 0x7fc02c0cce40 2023-10-10T20:46:25.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc031ea5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc02c159ec0 0x7fc02c15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc031ea5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc02c159ec0 0x7fc02c15be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43700/0 (socket says 172.21.15.89:43700) 2023-10-10T20:46:25.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc031ea5700 1 -- 172.21.15.89:0/1634453641 learned_addr learned my addr 172.21.15.89:0/1634453641 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:25.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc030ea3700 1 --2- 172.21.15.89:0/1634453641 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc02c0cce40 0x7fc02c0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc0316a4700 1 --2- 172.21.15.89:0/1634453641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc02c0b53d0 0x7fc02c153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc031ea5700 1 -- 172.21.15.89:0/1634453641 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc02c0cce40 msgr2=0x7fc02c0c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc031ea5700 1 --2- 172.21.15.89:0/1634453641 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fc02c0cce40 0x7fc02c0c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc031ea5700 1 -- 172.21.15.89:0/1634453641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc02c0b53d0 msgr2=0x7fc02c153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc031ea5700 1 --2- 172.21.15.89:0/1634453641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc02c0b53d0 0x7fc02c153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc031ea5700 1 -- 172.21.15.89:0/1634453641 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0240094f0 con 0x7fc02c159ec0 2023-10-10T20:46:25.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc031ea5700 1 --2- 172.21.15.89:0/1634453641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc02c159ec0 0x7fc02c15be40 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fc0240090b0 tx=0x7fc024017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:25.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc0217fa700 1 -- 172.21.15.89:0/1634453641 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02401a640 con 0x7fc02c159ec0 2023-10-10T20:46:25.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc0217fa700 1 -- 172.21.15.89:0/1634453641 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02401f070 con 0x7fc02c159ec0 2023-10-10T20:46:25.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc0217fa700 1 -- 172.21.15.89:0/1634453641 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02401aa10 con 0x7fc02c159ec0 2023-10-10T20:46:25.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.254+0000 7fc033908700 1 -- 172.21.15.89:0/1634453641 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc02c0c9160 con 0x7fc02c159ec0 2023-10-10T20:46:25.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.255+0000 7fc033908700 1 -- 172.21.15.89:0/1634453641 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02c0c9d60 con 0x7fc02c159ec0 2023-10-10T20:46:25.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.255+0000 7fc033908700 1 -- 172.21.15.89:0/1634453641 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc02c0c9680 con 0x7fc02c159ec0 2023-10-10T20:46:25.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.256+0000 7fc0217fa700 1 -- 172.21.15.89:0/1634453641 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc02401b430 con 0x7fc02c159ec0 2023-10-10T20:46:25.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.256+0000 7fc0217fa700 1 --2- 172.21.15.89:0/1634453641 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc010067320 0x7fc0100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.256+0000 7fc0217fa700 1 -- 172.21.15.89:0/1634453641 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc024021070 con 0x7fc02c159ec0 2023-10-10T20:46:25.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.256+0000 7fc0316a4700 1 --2- 172.21.15.89:0/1634453641 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc010067320 0x7fc0100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.256+0000 7fc0217fa700 1 --2- 172.21.15.89:0/1634453641 >> [v2:172.21.15.111:6816/91121,v1:172.21.15.111:6817/91121] conn(0x7fc01006d170 0x7fc01006f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.256+0000 7fc0217fa700 1 -- 172.21.15.89:0/1634453641 --> [v2:172.21.15.111:6816/91121,v1:172.21.15.111:6817/91121] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc01006fc50 con 0x7fc01006d170 2023-10-10T20:46:25.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.256+0000 7fc0217fa700 1 -- 172.21.15.89:0/1634453641 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc024088db0 con 0x7fc02c159ec0 2023-10-10T20:46:25.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.257+0000 7fc030ea3700 1 --2- 172.21.15.89:0/1634453641 >> [v2:172.21.15.111:6816/91121,v1:172.21.15.111:6817/91121] conn(0x7fc01006d170 0x7fc01006f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.257+0000 7fc0316a4700 1 --2- 172.21.15.89:0/1634453641 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc010067320 0x7fc0100697d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc028001250 tx=0x7fc028009620 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:25.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.257+0000 7fc030ea3700 1 --2- 172.21.15.89:0/1634453641 >> [v2:172.21.15.111:6816/91121,v1:172.21.15.111:6817/91121] conn(0x7fc01006d170 0x7fc01006f580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:25.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.258+0000 7fc0217fa700 1 -- 172.21.15.89:0/1634453641 <== osd.5 v2:172.21.15.111:6816/91121 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fc01006fc50 con 0x7fc01006d170 2023-10-10T20:46:25.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.271+0000 7fc033908700 1 -- 172.21.15.89:0/1634453641 --> [v2:172.21.15.111:6816/91121,v1:172.21.15.111:6817/91121] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc02c063c90 con 0x7fc01006d170 2023-10-10T20:46:25.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.272+0000 7fc0217fa700 1 -- 172.21.15.89:0/1634453641 <== osd.5 v2:172.21.15.111:6816/91121 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc02c063c90 con 0x7fc01006d170 2023-10-10T20:46:25.277 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:46:25.277 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-10T20:46:25.278 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:46:25.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.272+0000 7fc033908700 1 -- 172.21.15.89:0/1634453641 >> [v2:172.21.15.111:6816/91121,v1:172.21.15.111:6817/91121] conn(0x7fc01006d170 msgr2=0x7fc01006f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.272+0000 7fc033908700 1 --2- 172.21.15.89:0/1634453641 >> [v2:172.21.15.111:6816/91121,v1:172.21.15.111:6817/91121] conn(0x7fc01006d170 0x7fc01006f580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.273+0000 7fc033908700 1 -- 172.21.15.89:0/1634453641 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc010067320 msgr2=0x7fc0100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.273+0000 7fc033908700 1 --2- 172.21.15.89:0/1634453641 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fc010067320 0x7fc0100697d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc028001250 tx=0x7fc028009620 comp rx=0 tx=0).stop 2023-10-10T20:46:25.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.273+0000 7fc033908700 1 -- 172.21.15.89:0/1634453641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc02c159ec0 msgr2=0x7fc02c15be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.273+0000 7fc033908700 1 --2- 172.21.15.89:0/1634453641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc02c159ec0 0x7fc02c15be40 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fc0240090b0 tx=0x7fc024017000 comp rx=0 tx=0).stop 2023-10-10T20:46:25.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.273+0000 7fc031ea5700 1 -- 172.21.15.89:0/1634453641 reap_dead start 2023-10-10T20:46:25.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.273+0000 7fc033908700 1 -- 172.21.15.89:0/1634453641 shutdown_connections 2023-10-10T20:46:25.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.273+0000 7fc033908700 1 -- 172.21.15.89:0/1634453641 >> 172.21.15.89:0/1634453641 conn(0x7fc02c0999f0 msgr2=0x7fc02c0cd240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.273+0000 7fc033908700 1 -- 172.21.15.89:0/1634453641 shutdown_connections 2023-10-10T20:46:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.273+0000 7fc033908700 1 -- 172.21.15.89:0/1634453641 wait complete. 2023-10-10T20:46:25.286 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-10T20:46:25.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.399+0000 7fde48d2b700 1 Processor -- start 2023-10-10T20:46:25.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.399+0000 7fde48d2b700 1 -- start start 2023-10-10T20:46:25.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.400+0000 7fde48d2b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde44159c60 0x7fde4415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.400+0000 7fde48d2b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fde4415db10 0x7fde44069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.400+0000 7fde48d2b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde4415a160 con 0x7fde44159890 2023-10-10T20:46:25.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.400+0000 7fde48d2b700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde440d13a0 con 0x7fde4415db10 2023-10-10T20:46:25.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.400+0000 7fde48d2b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4415e140 con 0x7fde44159c60 2023-10-10T20:46:25.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.400+0000 7fde4259c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fde44159890 0x7fde4415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51762/0 (socket says 172.21.15.89:51762) 2023-10-10T20:46:25.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.400+0000 7fde4259c700 1 -- 172.21.15.89:0/3846033150 learned_addr learned my addr 172.21.15.89:0/3846033150 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:25.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.400+0000 7fde42d9d700 1 --2- 172.21.15.89:0/3846033150 >> v2:172.21.15.111:3300/0 conn(0x7fde4415db10 0x7fde44069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.400+0000 7fde39d9b700 1 --2- 172.21.15.89:0/3846033150 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde44159c60 0x7fde4415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.400+0000 7fde41d9b700 1 -- 172.21.15.89:0/3846033150 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674793597 0 0) 0x7fde4415a160 con 0x7fde44159890 2023-10-10T20:46:25.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.400+0000 7fde41d9b700 1 -- 172.21.15.89:0/3846033150 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde30003570 con 0x7fde44159890 2023-10-10T20:46:25.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde41d9b700 1 -- 172.21.15.89:0/3846033150 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3692416829 0 0) 0x7fde30003570 con 0x7fde44159890 2023-10-10T20:46:25.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde41d9b700 1 -- 172.21.15.89:0/3846033150 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde44159c60 msgr2=0x7fde4415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:25.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde41d9b700 1 --2- 172.21.15.89:0/3846033150 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde44159c60 0x7fde4415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde41d9b700 1 -- 172.21.15.89:0/3846033150 >> v2:172.21.15.111:3300/0 conn(0x7fde4415db10 msgr2=0x7fde44069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:25.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde41d9b700 1 --2- 172.21.15.89:0/3846033150 >> v2:172.21.15.111:3300/0 conn(0x7fde4415db10 0x7fde44069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde41d9b700 1 -- 172.21.15.89:0/3846033150 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4415a3d0 con 0x7fde44159890 2023-10-10T20:46:25.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde41d9b700 1 -- 172.21.15.89:0/3846033150 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fde34001470 con 0x7fde44159890 2023-10-10T20:46:25.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde41d9b700 1 -- 172.21.15.89:0/3846033150 >> v1:172.21.15.89:6789/0 conn(0x7fde44159890 legacy=0x7fde4415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde41d9b700 1 --2- 172.21.15.89:0/3846033150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde30003a60 0x7fde30003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde41d9b700 1 --2- 172.21.15.89:0/3846033150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde30004810 0x7fde30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde41d9b700 1 --2- 172.21.15.89:0/3846033150 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde44159c60 0x7fde300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde41d9b700 1 -- 172.21.15.89:0/3846033150 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde440d13a0 con 0x7fde30003a60 2023-10-10T20:46:25.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde41d9b700 1 -- 172.21.15.89:0/3846033150 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4415e140 con 0x7fde30004810 2023-10-10T20:46:25.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde41d9b700 1 -- 172.21.15.89:0/3846033150 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3000ba60 con 0x7fde44159c60 2023-10-10T20:46:25.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde4259c700 1 --2- 172.21.15.89:0/3846033150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde30003a60 0x7fde30003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde39d9b700 1 --2- 172.21.15.89:0/3846033150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde30004810 0x7fde30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde42d9d700 1 --2- 172.21.15.89:0/3846033150 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde44159c60 0x7fde300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde4259c700 1 -- 172.21.15.89:0/3846033150 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde44159c60 msgr2=0x7fde300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:25.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde4259c700 1 --2- 172.21.15.89:0/3846033150 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde44159c60 0x7fde300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde4259c700 1 -- 172.21.15.89:0/3846033150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde30004810 msgr2=0x7fde30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:25.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde4259c700 1 --2- 172.21.15.89:0/3846033150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde30004810 0x7fde30004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde4259c700 1 -- 172.21.15.89:0/3846033150 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde30003570 con 0x7fde30003a60 2023-10-10T20:46:25.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde4259c700 1 --2- 172.21.15.89:0/3846033150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde30003a60 0x7fde30003e70 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fde3400c3f0 tx=0x7fde34001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ecc6040bc90516 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:25.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde41d9b700 1 -- 172.21.15.89:0/3846033150 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3400cc80 con 0x7fde30003a60 2023-10-10T20:46:25.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde41d9b700 1 -- 172.21.15.89:0/3846033150 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3400ce00 con 0x7fde30003a60 2023-10-10T20:46:25.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.401+0000 7fde41d9b700 1 -- 172.21.15.89:0/3846033150 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34005b70 con 0x7fde30003a60 2023-10-10T20:46:25.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.402+0000 7fde48d2b700 1 -- 172.21.15.89:0/3846033150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde30003a60 msgr2=0x7fde30003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.402+0000 7fde48d2b700 1 --2- 172.21.15.89:0/3846033150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde30003a60 0x7fde30003e70 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fde3400c3f0 tx=0x7fde34001e90 comp rx=0 tx=0).stop 2023-10-10T20:46:25.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.402+0000 7fde42d9d700 1 -- 172.21.15.89:0/3846033150 reap_dead start 2023-10-10T20:46:25.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.402+0000 7fde48d2b700 1 -- 172.21.15.89:0/3846033150 shutdown_connections 2023-10-10T20:46:25.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.402+0000 7fde48d2b700 1 -- 172.21.15.89:0/3846033150 >> 172.21.15.89:0/3846033150 conn(0x7fde44068e20 msgr2=0x7fde440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:25.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.402+0000 7fde48d2b700 1 -- 172.21.15.89:0/3846033150 shutdown_connections 2023-10-10T20:46:25.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.402+0000 7fde48d2b700 1 -- 172.21.15.89:0/3846033150 wait complete. 2023-10-10T20:46:25.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.403+0000 7fde48d2b700 1 Processor -- start 2023-10-10T20:46:25.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.403+0000 7fde48d2b700 1 -- start start 2023-10-10T20:46:25.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.404+0000 7fde48d2b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde30003a60 0x7fde4414a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.404+0000 7fde48d2b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde440b3590 0x7fde440b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.404+0000 7fde48d2b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde440b1a60 0x7fde440b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.404+0000 7fde48d2b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde440b23f0 con 0x7fde440b3590 2023-10-10T20:46:25.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.404+0000 7fde48d2b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde440c9060 con 0x7fde440b1a60 2023-10-10T20:46:25.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.404+0000 7fde48d2b700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde440b4810 con 0x7fde30003a60 2023-10-10T20:46:25.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.404+0000 7fde39d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde440b3590 0x7fde440b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.404+0000 7fde39d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde440b3590 0x7fde440b5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43714/0 (socket says 172.21.15.89:43714) 2023-10-10T20:46:25.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.404+0000 7fde39d9b700 1 -- 172.21.15.89:0/931333564 learned_addr learned my addr 172.21.15.89:0/931333564 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:25.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.404+0000 7fde4259c700 1 --2- 172.21.15.89:0/931333564 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde30003a60 0x7fde4414a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.404+0000 7fde42d9d700 1 --2- 172.21.15.89:0/931333564 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde440b1a60 0x7fde440b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.404+0000 7fde39d9b700 1 -- 172.21.15.89:0/931333564 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde30003a60 msgr2=0x7fde4414a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.404+0000 7fde39d9b700 1 --2- 172.21.15.89:0/931333564 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde30003a60 0x7fde4414a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.404+0000 7fde39d9b700 1 -- 172.21.15.89:0/931333564 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde440b1a60 msgr2=0x7fde440b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.404+0000 7fde39d9b700 1 --2- 172.21.15.89:0/931333564 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde440b1a60 0x7fde440b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.404+0000 7fde39d9b700 1 -- 172.21.15.89:0/931333564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde440b4af0 con 0x7fde440b3590 2023-10-10T20:46:25.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.404+0000 7fde39d9b700 1 --2- 172.21.15.89:0/931333564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde440b3590 0x7fde440b5e00 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fde2c011b40 tx=0x7fde2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:25.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.405+0000 7fde3affd700 1 -- 172.21.15.89:0/931333564 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c00f9f0 con 0x7fde440b3590 2023-10-10T20:46:25.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.405+0000 7fde3affd700 1 -- 172.21.15.89:0/931333564 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c00fb50 con 0x7fde440b3590 2023-10-10T20:46:25.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.405+0000 7fde3affd700 1 -- 172.21.15.89:0/931333564 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c00fed0 con 0x7fde440b3590 2023-10-10T20:46:25.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.405+0000 7fde48d2b700 1 -- 172.21.15.89:0/931333564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde440b4dd0 con 0x7fde440b3590 2023-10-10T20:46:25.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.405+0000 7fde48d2b700 1 -- 172.21.15.89:0/931333564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4414a790 con 0x7fde440b3590 2023-10-10T20:46:25.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.405+0000 7fde48d2b700 1 -- 172.21.15.89:0/931333564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4404c3c0 con 0x7fde440b3590 2023-10-10T20:46:25.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.406+0000 7fde3affd700 1 -- 172.21.15.89:0/931333564 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fde2c014650 con 0x7fde440b3590 2023-10-10T20:46:25.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.406+0000 7fde3affd700 1 --2- 172.21.15.89:0/931333564 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fde24067320 0x7fde240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.407+0000 7fde4259c700 1 --2- 172.21.15.89:0/931333564 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fde24067320 0x7fde240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.407+0000 7fde3affd700 1 -- 172.21.15.89:0/931333564 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fde2c018030 con 0x7fde440b3590 2023-10-10T20:46:25.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.407+0000 7fde4259c700 1 --2- 172.21.15.89:0/931333564 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fde24067320 0x7fde240697d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fde34001ed0 tx=0x7fde34015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:25.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.411+0000 7fde3affd700 1 -- 172.21.15.89:0/931333564 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde2c052440 con 0x7fde440b3590 2023-10-10T20:46:25.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.593+0000 7fde48d2b700 1 -- 172.21.15.89:0/931333564 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fde4406a5b0 con 0x7fde24067320 2023-10-10T20:46:25.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.593+0000 7fde3affd700 1 -- 172.21.15.89:0/931333564 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fde4406a5b0 con 0x7fde24067320 2023-10-10T20:46:25.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.595+0000 7fde48d2b700 1 -- 172.21.15.89:0/931333564 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fde24067320 msgr2=0x7fde240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.595+0000 7fde48d2b700 1 --2- 172.21.15.89:0/931333564 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fde24067320 0x7fde240697d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fde34001ed0 tx=0x7fde34015000 comp rx=0 tx=0).stop 2023-10-10T20:46:25.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.595+0000 7fde48d2b700 1 -- 172.21.15.89:0/931333564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde440b3590 msgr2=0x7fde440b5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.595+0000 7fde48d2b700 1 --2- 172.21.15.89:0/931333564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde440b3590 0x7fde440b5e00 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fde2c011b40 tx=0x7fde2c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.595+0000 7fde48d2b700 1 -- 172.21.15.89:0/931333564 shutdown_connections 2023-10-10T20:46:25.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.595+0000 7fde48d2b700 1 --2- 172.21.15.89:0/931333564 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fde24067320 0x7fde240697d0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.595+0000 7fde48d2b700 1 --2- 172.21.15.89:0/931333564 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde440b1a60 0x7fde440b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.595+0000 7fde48d2b700 1 --2- 172.21.15.89:0/931333564 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fde30003a60 0x7fde4414a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.595+0000 7fde48d2b700 1 --2- 172.21.15.89:0/931333564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde440b3590 0x7fde440b5e00 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.595+0000 7fde48d2b700 1 -- 172.21.15.89:0/931333564 >> 172.21.15.89:0/931333564 conn(0x7fde44068e20 msgr2=0x7fde441585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.595+0000 7fde48d2b700 1 -- 172.21.15.89:0/931333564 shutdown_connections 2023-10-10T20:46:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.595+0000 7fde48d2b700 1 -- 172.21.15.89:0/931333564 wait complete. 2023-10-10T20:46:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 5 to deep-scrub 2023-10-10T20:46:25.609 INFO:tasks.ceph:Scrubbing osd.6 2023-10-10T20:46:25.609 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-10T20:46:25.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.726+0000 7f64e5657700 1 Processor -- start 2023-10-10T20:46:25.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.726+0000 7f64e5657700 1 -- start start 2023-10-10T20:46:25.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.727+0000 7f64e5657700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64e00b03c0 0x7f64e00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.727+0000 7f64e5657700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e00cfa00 con 0x7f64e00b0790 2023-10-10T20:46:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.727+0000 7f64e5657700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e00cb580 con 0x7f64e01591e0 2023-10-10T20:46:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.727+0000 7f64e5657700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00cf160 con 0x7f64e00b03c0 2023-10-10T20:46:25.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.727+0000 7f64df7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f64e00b0790 0x7f64e00b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51766/0 (socket says 172.21.15.89:51766) 2023-10-10T20:46:25.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.727+0000 7f64df7fe700 1 -- 172.21.15.89:0/2723607591 learned_addr learned my addr 172.21.15.89:0/2723607591 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:25.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.727+0000 7f64dffff700 1 --2- 172.21.15.89:0/2723607591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64e00b03c0 0x7f64e00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.727+0000 7f64deffd700 1 -- 172.21.15.89:0/2723607591 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781858728 0 0) 0x7f64e00cfa00 con 0x7f64e00b0790 2023-10-10T20:46:25.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.727+0000 7f64deffd700 1 -- 172.21.15.89:0/2723607591 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64c8003570 con 0x7f64e00b0790 2023-10-10T20:46:25.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.727+0000 7f64deffd700 1 -- 172.21.15.89:0/2723607591 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 602800515 0 0) 0x7f64c8003570 con 0x7f64e00b0790 2023-10-10T20:46:25.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.727+0000 7f64deffd700 1 -- 172.21.15.89:0/2723607591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64e00b03c0 msgr2=0x7f64e00cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:25.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.727+0000 7f64deffd700 1 --2- 172.21.15.89:0/2723607591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64e00b03c0 0x7f64e00cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.727+0000 7f64deffd700 1 -- 172.21.15.89:0/2723607591 >> v1:172.21.15.111:6789/0 conn(0x7f64e01591e0 legacy=0x7f64e0159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.727+0000 7f64deffd700 1 -- 172.21.15.89:0/2723607591 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e00cf440 con 0x7f64e00b0790 2023-10-10T20:46:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.727+0000 7f64deffd700 1 -- 172.21.15.89:0/2723607591 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7f64d4001470 con 0x7f64e00b0790 2023-10-10T20:46:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.727+0000 7f64deffd700 1 -- 172.21.15.89:0/2723607591 >> v1:172.21.15.89:6789/0 conn(0x7f64e00b0790 legacy=0x7f64e00b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.727+0000 7f64deffd700 1 --2- 172.21.15.89:0/2723607591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64c8003a90 0x7f64c8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.728+0000 7f64deffd700 1 --2- 172.21.15.89:0/2723607591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64e00b03c0 0x7f64c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.728+0000 7f64deffd700 1 --2- 172.21.15.89:0/2723607591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64c8004d50 0x7f64c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.728+0000 7f64deffd700 1 -- 172.21.15.89:0/2723607591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00cf160 con 0x7f64c8004d50 2023-10-10T20:46:25.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.728+0000 7f64deffd700 1 -- 172.21.15.89:0/2723607591 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c800bb10 con 0x7f64c8003a90 2023-10-10T20:46:25.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.728+0000 7f64deffd700 1 -- 172.21.15.89:0/2723607591 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c800bcd0 con 0x7f64e00b03c0 2023-10-10T20:46:25.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.728+0000 7f64e4e56700 1 --2- 172.21.15.89:0/2723607591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64c8004d50 0x7f64c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.728+0000 7f64dffff700 1 --2- 172.21.15.89:0/2723607591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64c8003a90 0x7f64c8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.728+0000 7f64df7fe700 1 --2- 172.21.15.89:0/2723607591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64e00b03c0 0x7f64c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.728+0000 7f64e4e56700 1 -- 172.21.15.89:0/2723607591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64e00b03c0 msgr2=0x7f64c8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:25.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.728+0000 7f64e4e56700 1 --2- 172.21.15.89:0/2723607591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64e00b03c0 0x7f64c8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.728+0000 7f64e4e56700 1 -- 172.21.15.89:0/2723607591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64c8003a90 msgr2=0x7f64c8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:25.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.728+0000 7f64e4e56700 1 --2- 172.21.15.89:0/2723607591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64c8003a90 0x7f64c8003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.728+0000 7f64e4e56700 1 -- 172.21.15.89:0/2723607591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c8003570 con 0x7f64c8004d50 2023-10-10T20:46:25.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.728+0000 7f64df7fe700 1 --2- 172.21.15.89:0/2723607591 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64e00b03c0 0x7f64c8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:25.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.728+0000 7f64e4e56700 1 --2- 172.21.15.89:0/2723607591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64c8004d50 0x7f64c80051c0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f64d80020a0 tx=0x7f64d80016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b6dd04a3f69058 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:25.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.728+0000 7f64deffd700 1 -- 172.21.15.89:0/2723607591 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d80019e0 con 0x7f64c8004d50 2023-10-10T20:46:25.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.728+0000 7f64deffd700 1 -- 172.21.15.89:0/2723607591 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d800c8b0 con 0x7f64c8004d50 2023-10-10T20:46:25.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.728+0000 7f64deffd700 1 -- 172.21.15.89:0/2723607591 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d800cc50 con 0x7f64c8004d50 2023-10-10T20:46:25.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.729+0000 7f64e5657700 1 -- 172.21.15.89:0/2723607591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64c8004d50 msgr2=0x7f64c80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.729+0000 7f64e5657700 1 --2- 172.21.15.89:0/2723607591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64c8004d50 0x7f64c80051c0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f64d80020a0 tx=0x7f64d80016d0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.729+0000 7f64e4e56700 1 -- 172.21.15.89:0/2723607591 reap_dead start 2023-10-10T20:46:25.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.729+0000 7f64e5657700 1 -- 172.21.15.89:0/2723607591 shutdown_connections 2023-10-10T20:46:25.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.729+0000 7f64e5657700 1 -- 172.21.15.89:0/2723607591 >> 172.21.15.89:0/2723607591 conn(0x7f64e0063230 msgr2=0x7f64e0063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:25.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.729+0000 7f64e5657700 1 -- 172.21.15.89:0/2723607591 shutdown_connections 2023-10-10T20:46:25.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.729+0000 7f64e5657700 1 -- 172.21.15.89:0/2723607591 wait complete. 2023-10-10T20:46:25.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.730+0000 7f64e5657700 1 Processor -- start 2023-10-10T20:46:25.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.730+0000 7f64e5657700 1 -- start start 2023-10-10T20:46:25.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64e5657700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64e0153a60 0x7f64e0159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64e5657700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e00c8b50 0x7f64e015d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64e5657700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64e0069360 0x7f64e00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64e5657700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e0153e30 con 0x7f64e00c8b50 2023-10-10T20:46:25.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64e5657700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e015dc40 con 0x7f64e0069360 2023-10-10T20:46:25.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64e5657700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e0069d10 con 0x7f64e0153a60 2023-10-10T20:46:25.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64df7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e00c8b50 0x7f64e015d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64df7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e00c8b50 0x7f64e015d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43720/0 (socket says 172.21.15.89:43720) 2023-10-10T20:46:25.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64df7fe700 1 -- 172.21.15.89:0/1759181328 learned_addr learned my addr 172.21.15.89:0/1759181328 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:25.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64e4e56700 1 --2- 172.21.15.89:0/1759181328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64e0069360 0x7f64e00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64dffff700 1 --2- 172.21.15.89:0/1759181328 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64e0153a60 0x7f64e0159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64df7fe700 1 -- 172.21.15.89:0/1759181328 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64e0153a60 msgr2=0x7f64e0159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64df7fe700 1 --2- 172.21.15.89:0/1759181328 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64e0153a60 0x7f64e0159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64df7fe700 1 -- 172.21.15.89:0/1759181328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64e0069360 msgr2=0x7f64e00697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64df7fe700 1 --2- 172.21.15.89:0/1759181328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64e0069360 0x7f64e00697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64df7fe700 1 -- 172.21.15.89:0/1759181328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e00c5c00 con 0x7f64e00c8b50 2023-10-10T20:46:25.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64dffff700 1 --2- 172.21.15.89:0/1759181328 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f64e0153a60 0x7f64e0159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:25.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64df7fe700 1 --2- 172.21.15.89:0/1759181328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e00c8b50 0x7f64e015d110 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f64d400eb00 tx=0x7f64e00b0c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:25.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64c7fff700 1 -- 172.21.15.89:0/1759181328 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d400a550 con 0x7f64e00c8b50 2023-10-10T20:46:25.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64c7fff700 1 -- 172.21.15.89:0/1759181328 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d4010040 con 0x7f64e00c8b50 2023-10-10T20:46:25.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64e5657700 1 -- 172.21.15.89:0/1759181328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e015d960 con 0x7f64e00c8b50 2023-10-10T20:46:25.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64c7fff700 1 -- 172.21.15.89:0/1759181328 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d400a8e0 con 0x7f64e00c8b50 2023-10-10T20:46:25.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.731+0000 7f64e5657700 1 -- 172.21.15.89:0/1759181328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e00c5890 con 0x7f64e00c8b50 2023-10-10T20:46:25.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.732+0000 7f64e5657700 1 -- 172.21.15.89:0/1759181328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f64e004c3c0 con 0x7f64e00c8b50 2023-10-10T20:46:25.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.733+0000 7f64c7fff700 1 -- 172.21.15.89:0/1759181328 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f64d400ac50 con 0x7f64e00c8b50 2023-10-10T20:46:25.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.733+0000 7f64c7fff700 1 --2- 172.21.15.89:0/1759181328 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f64cc067320 0x7f64cc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.733+0000 7f64c7fff700 1 -- 172.21.15.89:0/1759181328 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f64d4084920 con 0x7f64e00c8b50 2023-10-10T20:46:25.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.733+0000 7f64dffff700 1 --2- 172.21.15.89:0/1759181328 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f64cc067320 0x7f64cc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.733+0000 7f64c7fff700 1 --2- 172.21.15.89:0/1759181328 >> [v2:172.21.15.111:6808/91096,v1:172.21.15.111:6809/91096] conn(0x7f64cc06d170 0x7f64cc06f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.733+0000 7f64c7fff700 1 -- 172.21.15.89:0/1759181328 --> [v2:172.21.15.111:6808/91096,v1:172.21.15.111:6809/91096] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f64cc06fc50 con 0x7f64cc06d170 2023-10-10T20:46:25.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.733+0000 7f64c7fff700 1 -- 172.21.15.89:0/1759181328 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64d4084c30 con 0x7f64e00c8b50 2023-10-10T20:46:25.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.734+0000 7f64dffff700 1 --2- 172.21.15.89:0/1759181328 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f64cc067320 0x7f64cc0697d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f64d0006f00 tx=0x7f64d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:25.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.734+0000 7f64e4e56700 1 --2- 172.21.15.89:0/1759181328 >> [v2:172.21.15.111:6808/91096,v1:172.21.15.111:6809/91096] conn(0x7f64cc06d170 0x7f64cc06f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.734+0000 7f64e4e56700 1 --2- 172.21.15.89:0/1759181328 >> [v2:172.21.15.111:6808/91096,v1:172.21.15.111:6809/91096] conn(0x7f64cc06d170 0x7f64cc06f580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:25.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.735+0000 7f64c7fff700 1 -- 172.21.15.89:0/1759181328 <== osd.6 v2:172.21.15.111:6808/91096 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f64cc06fc50 con 0x7f64cc06d170 2023-10-10T20:46:25.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.748+0000 7f64e5657700 1 -- 172.21.15.89:0/1759181328 --> [v2:172.21.15.111:6808/91096,v1:172.21.15.111:6809/91096] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f64e00cf720 con 0x7f64cc06d170 2023-10-10T20:46:25.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.749+0000 7f64c7fff700 1 -- 172.21.15.89:0/1759181328 <== osd.6 v2:172.21.15.111:6808/91096 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f64e00cf720 con 0x7f64cc06d170 2023-10-10T20:46:25.761 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:46:25.761 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-10T20:46:25.762 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:46:25.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.749+0000 7f64e5657700 1 -- 172.21.15.89:0/1759181328 >> [v2:172.21.15.111:6808/91096,v1:172.21.15.111:6809/91096] conn(0x7f64cc06d170 msgr2=0x7f64cc06f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.749+0000 7f64e5657700 1 --2- 172.21.15.89:0/1759181328 >> [v2:172.21.15.111:6808/91096,v1:172.21.15.111:6809/91096] conn(0x7f64cc06d170 0x7f64cc06f580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.749+0000 7f64e5657700 1 -- 172.21.15.89:0/1759181328 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f64cc067320 msgr2=0x7f64cc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.749+0000 7f64e5657700 1 --2- 172.21.15.89:0/1759181328 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f64cc067320 0x7f64cc0697d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f64d0006f00 tx=0x7f64d0008040 comp rx=0 tx=0).stop 2023-10-10T20:46:25.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.749+0000 7f64e5657700 1 -- 172.21.15.89:0/1759181328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e00c8b50 msgr2=0x7f64e015d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.749+0000 7f64e5657700 1 --2- 172.21.15.89:0/1759181328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e00c8b50 0x7f64e015d110 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f64d400eb00 tx=0x7f64e00b0c40 comp rx=0 tx=0).stop 2023-10-10T20:46:25.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.749+0000 7f64e4e56700 1 -- 172.21.15.89:0/1759181328 reap_dead start 2023-10-10T20:46:25.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.750+0000 7f64e5657700 1 -- 172.21.15.89:0/1759181328 shutdown_connections 2023-10-10T20:46:25.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.750+0000 7f64e5657700 1 -- 172.21.15.89:0/1759181328 >> 172.21.15.89:0/1759181328 conn(0x7f64e0063230 msgr2=0x7f64e00d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:25.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.750+0000 7f64e5657700 1 -- 172.21.15.89:0/1759181328 shutdown_connections 2023-10-10T20:46:25.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.750+0000 7f64e5657700 1 -- 172.21.15.89:0/1759181328 wait complete. 2023-10-10T20:46:25.768 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-10T20:46:25.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.882+0000 7fae60e59700 1 Processor -- start 2023-10-10T20:46:25.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.882+0000 7fae60e59700 1 -- start start 2023-10-10T20:46:25.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.883+0000 7fae60e59700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fae5c159420 0x7fae5c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.883+0000 7fae60e59700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae5c1597f0 0x7fae5c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.883+0000 7fae60e59700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5c0b11d0 con 0x7fae5c15d230 2023-10-10T20:46:25.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.883+0000 7fae60e59700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae5c0b14b0 con 0x7fae5c159420 2023-10-10T20:46:25.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.883+0000 7fae60e59700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0b1790 con 0x7fae5c1597f0 2023-10-10T20:46:25.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.883+0000 7fae5bfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fae5c15d230 0x7fae5c0b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51770/0 (socket says 172.21.15.89:51770) 2023-10-10T20:46:25.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.883+0000 7fae5bfff700 1 -- 172.21.15.89:0/1907122079 learned_addr learned my addr 172.21.15.89:0/1907122079 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:25.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.883+0000 7fae5b7fe700 1 --2- 172.21.15.89:0/1907122079 >> v2:172.21.15.111:3300/0 conn(0x7fae5c159420 0x7fae5c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.883+0000 7fae53fff700 1 --2- 172.21.15.89:0/1907122079 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae5c1597f0 0x7fae5c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.883+0000 7fae5affd700 1 -- 172.21.15.89:0/1907122079 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57415550 0 0) 0x7fae5c0b11d0 con 0x7fae5c15d230 2023-10-10T20:46:25.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.883+0000 7fae5affd700 1 -- 172.21.15.89:0/1907122079 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae48003570 con 0x7fae5c15d230 2023-10-10T20:46:25.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.883+0000 7fae5b7fe700 1 -- 172.21.15.89:0/1907122079 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae5c1597f0 msgr2=0x7fae5c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:25.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.883+0000 7fae5b7fe700 1 --2- 172.21.15.89:0/1907122079 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae5c1597f0 0x7fae5c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.883+0000 7fae5b7fe700 1 -- 172.21.15.89:0/1907122079 >> v1:172.21.15.89:6789/0 conn(0x7fae5c15d230 legacy=0x7fae5c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.883+0000 7fae5b7fe700 1 -- 172.21.15.89:0/1907122079 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c0b1a70 con 0x7fae5c159420 2023-10-10T20:46:25.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.883+0000 7fae5b7fe700 1 --2- 172.21.15.89:0/1907122079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae5c159420 0x7fae5c0b5f40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0098e0 tx=0x7fae4c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f87fd6813fbc9a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:25.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.884+0000 7fae5affd700 1 -- 172.21.15.89:0/1907122079 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c011630 con 0x7fae5c159420 2023-10-10T20:46:25.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.884+0000 7fae5affd700 1 -- 172.21.15.89:0/1907122079 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4c011790 con 0x7fae5c159420 2023-10-10T20:46:25.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.884+0000 7fae5affd700 1 -- 172.21.15.89:0/1907122079 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c011b30 con 0x7fae5c159420 2023-10-10T20:46:25.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.884+0000 7fae60e59700 1 -- 172.21.15.89:0/1907122079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae5c159420 msgr2=0x7fae5c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.884+0000 7fae60e59700 1 --2- 172.21.15.89:0/1907122079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae5c159420 0x7fae5c0b5f40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0098e0 tx=0x7fae4c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.884+0000 7fae60e59700 1 -- 172.21.15.89:0/1907122079 shutdown_connections 2023-10-10T20:46:25.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.884+0000 7fae60e59700 1 --2- 172.21.15.89:0/1907122079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae5c159420 0x7fae5c0b5f40 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.884+0000 7fae60e59700 1 --2- 172.21.15.89:0/1907122079 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae5c1597f0 0x7fae5c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.884+0000 7fae60e59700 1 -- 172.21.15.89:0/1907122079 >> 172.21.15.89:0/1907122079 conn(0x7fae5c0d0660 msgr2=0x7fae5c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:25.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.884+0000 7fae60e59700 1 -- 172.21.15.89:0/1907122079 shutdown_connections 2023-10-10T20:46:25.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.884+0000 7fae60e59700 1 -- 172.21.15.89:0/1907122079 wait complete. 2023-10-10T20:46:25.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.885+0000 7fae60e59700 1 Processor -- start 2023-10-10T20:46:25.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.885+0000 7fae60e59700 1 -- start start 2023-10-10T20:46:25.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.886+0000 7fae60e59700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae5c159420 0x7fae5c16a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.886+0000 7fae60e59700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae5c1597f0 0x7fae5c0c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.886+0000 7fae60e59700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae5c15d230 0x7fae5c14c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.886+0000 7fae60e59700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0c5940 con 0x7fae5c15d230 2023-10-10T20:46:25.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.886+0000 7fae60e59700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0c5b70 con 0x7fae5c159420 2023-10-10T20:46:25.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.886+0000 7fae60e59700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0c5e50 con 0x7fae5c1597f0 2023-10-10T20:46:25.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.886+0000 7fae5bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae5c15d230 0x7fae5c14c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.886+0000 7fae5bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae5c15d230 0x7fae5c14c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43726/0 (socket says 172.21.15.89:43726) 2023-10-10T20:46:25.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.886+0000 7fae5bfff700 1 -- 172.21.15.89:0/3128552905 learned_addr learned my addr 172.21.15.89:0/3128552905 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:25.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.886+0000 7fae5b7fe700 1 --2- 172.21.15.89:0/3128552905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae5c159420 0x7fae5c16a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.886+0000 7fae53fff700 1 --2- 172.21.15.89:0/3128552905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae5c1597f0 0x7fae5c0c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.887+0000 7fae5bfff700 1 -- 172.21.15.89:0/3128552905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae5c1597f0 msgr2=0x7fae5c0c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.887+0000 7fae5bfff700 1 --2- 172.21.15.89:0/3128552905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae5c1597f0 0x7fae5c0c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.887+0000 7fae5bfff700 1 -- 172.21.15.89:0/3128552905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae5c159420 msgr2=0x7fae5c16a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:25.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.887+0000 7fae5bfff700 1 --2- 172.21.15.89:0/3128552905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae5c159420 0x7fae5c16a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:25.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.887+0000 7fae5bfff700 1 -- 172.21.15.89:0/3128552905 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4c009420 con 0x7fae5c15d230 2023-10-10T20:46:25.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.887+0000 7fae5bfff700 1 --2- 172.21.15.89:0/3128552905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae5c15d230 0x7fae5c14c140 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fae54001460 tx=0x7fae54001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:25.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.887+0000 7fae537fe700 1 -- 172.21.15.89:0/3128552905 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5400a870 con 0x7fae5c15d230 2023-10-10T20:46:25.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.887+0000 7fae537fe700 1 -- 172.21.15.89:0/3128552905 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5400a9d0 con 0x7fae5c15d230 2023-10-10T20:46:25.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.887+0000 7fae60e59700 1 -- 172.21.15.89:0/3128552905 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5c0c6130 con 0x7fae5c15d230 2023-10-10T20:46:25.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.887+0000 7fae537fe700 1 -- 172.21.15.89:0/3128552905 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5400ad50 con 0x7fae5c15d230 2023-10-10T20:46:25.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.887+0000 7fae60e59700 1 -- 172.21.15.89:0/3128552905 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5c0c66f0 con 0x7fae5c15d230 2023-10-10T20:46:25.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.888+0000 7fae60e59700 1 -- 172.21.15.89:0/3128552905 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae5c0c6910 con 0x7fae5c15d230 2023-10-10T20:46:25.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.888+0000 7fae537fe700 1 -- 172.21.15.89:0/3128552905 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fae54007300 con 0x7fae5c15d230 2023-10-10T20:46:25.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.889+0000 7fae537fe700 1 --2- 172.21.15.89:0/3128552905 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fae3c067320 0x7fae3c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:25.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.889+0000 7fae537fe700 1 -- 172.21.15.89:0/3128552905 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fae54084d40 con 0x7fae5c15d230 2023-10-10T20:46:25.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.889+0000 7fae5b7fe700 1 --2- 172.21.15.89:0/3128552905 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fae3c067320 0x7fae3c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:25.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.889+0000 7fae5b7fe700 1 --2- 172.21.15.89:0/3128552905 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fae3c067320 0x7fae3c0697d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0091e0 tx=0x7fae4c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:25.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:25.893+0000 7fae537fe700 1 -- 172.21.15.89:0/3128552905 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae54051d70 con 0x7fae5c15d230 2023-10-10T20:46:26.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.075+0000 7fae60e59700 1 -- 172.21.15.89:0/3128552905 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fae5c04c3c0 con 0x7fae3c067320 2023-10-10T20:46:26.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.075+0000 7fae537fe700 1 -- 172.21.15.89:0/3128552905 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fae5c04c3c0 con 0x7fae3c067320 2023-10-10T20:46:26.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.077+0000 7fae60e59700 1 -- 172.21.15.89:0/3128552905 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fae3c067320 msgr2=0x7fae3c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.077+0000 7fae60e59700 1 --2- 172.21.15.89:0/3128552905 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fae3c067320 0x7fae3c0697d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0091e0 tx=0x7fae4c017000 comp rx=0 tx=0).stop 2023-10-10T20:46:26.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.077+0000 7fae60e59700 1 -- 172.21.15.89:0/3128552905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae5c15d230 msgr2=0x7fae5c14c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.077+0000 7fae60e59700 1 --2- 172.21.15.89:0/3128552905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae5c15d230 0x7fae5c14c140 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fae54001460 tx=0x7fae54001a40 comp rx=0 tx=0).stop 2023-10-10T20:46:26.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.077+0000 7fae60e59700 1 -- 172.21.15.89:0/3128552905 shutdown_connections 2023-10-10T20:46:26.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.077+0000 7fae60e59700 1 --2- 172.21.15.89:0/3128552905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae5c15d230 0x7fae5c14c140 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.077+0000 7fae60e59700 1 --2- 172.21.15.89:0/3128552905 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fae3c067320 0x7fae3c0697d0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.077+0000 7fae60e59700 1 --2- 172.21.15.89:0/3128552905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae5c159420 0x7fae5c16a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.077+0000 7fae60e59700 1 --2- 172.21.15.89:0/3128552905 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fae5c1597f0 0x7fae5c0c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.077+0000 7fae60e59700 1 -- 172.21.15.89:0/3128552905 >> 172.21.15.89:0/3128552905 conn(0x7fae5c0d0660 msgr2=0x7fae5c15bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:26.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.077+0000 7fae60e59700 1 -- 172.21.15.89:0/3128552905 shutdown_connections 2023-10-10T20:46:26.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.077+0000 7fae60e59700 1 -- 172.21.15.89:0/3128552905 wait complete. 2023-10-10T20:46:26.083 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 6 to deep-scrub 2023-10-10T20:46:26.091 INFO:tasks.ceph:Scrubbing osd.7 2023-10-10T20:46:26.091 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-10T20:46:26.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.208+0000 7fbdaf59e700 1 Processor -- start 2023-10-10T20:46:26.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.208+0000 7fbdaf59e700 1 -- start start 2023-10-10T20:46:26.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.208+0000 7fbdaf59e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbdb00b03c0 0x7fbdb00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:26.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.208+0000 7fbdaf59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdb00b0790 0x7fbdb00b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:26.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.208+0000 7fbdaf59e700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb00cfa00 con 0x7fbdb01591e0 2023-10-10T20:46:26.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.208+0000 7fbdaf59e700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb00cb580 con 0x7fbdb00b03c0 2023-10-10T20:46:26.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.208+0000 7fbdaf59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb00b2de0 con 0x7fbdb00b0790 2023-10-10T20:46:26.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.209+0000 7fbdae59c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbdb00b03c0 0x7fbdb00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:26.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.209+0000 7fbda5d9b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdb00b0790 0x7fbdb00b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:26.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.209+0000 7fbdaed9d700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fbdb01591e0 0x7fbdb0159620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:32914/0 (socket says 172.21.15.89:32914) 2023-10-10T20:46:26.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.209+0000 7fbdaed9d700 1 -- 172.21.15.89:0/1588628490 learned_addr learned my addr 172.21.15.89:0/1588628490 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:26.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.209+0000 7fbda5d9b700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdb00b0790 0x7fbdb00b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:36962/0 (socket says 172.21.15.89:36962) 2023-10-10T20:46:26.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.209+0000 7fbdae59c700 1 -- 172.21.15.89:0/1588628490 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdb00b0790 msgr2=0x7fbdb00b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:26.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.209+0000 7fbdae59c700 1 --2- 172.21.15.89:0/1588628490 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdb00b0790 0x7fbdb00b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.209+0000 7fbdae59c700 1 -- 172.21.15.89:0/1588628490 >> v1:172.21.15.111:6789/0 conn(0x7fbdb01591e0 legacy=0x7fbdb0159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.209+0000 7fbdae59c700 1 -- 172.21.15.89:0/1588628490 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb00cf160 con 0x7fbdb00b03c0 2023-10-10T20:46:26.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.209+0000 7fbda5d9b700 1 --2- 172.21.15.89:0/1588628490 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdb00b0790 0x7fbdb00b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:46:26.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.209+0000 7fbdae59c700 1 --2- 172.21.15.89:0/1588628490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb00b03c0 0x7fbdb00cea70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbda00098e0 tx=0x7fbda00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5630f98cf9dca8c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:26.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.209+0000 7fbdadd9b700 1 -- 172.21.15.89:0/1588628490 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda000b640 con 0x7fbdb00b03c0 2023-10-10T20:46:26.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.210+0000 7fbdadd9b700 1 -- 172.21.15.89:0/1588628490 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda000b7a0 con 0x7fbdb00b03c0 2023-10-10T20:46:26.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.210+0000 7fbdadd9b700 1 -- 172.21.15.89:0/1588628490 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda000bb40 con 0x7fbdb00b03c0 2023-10-10T20:46:26.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.210+0000 7fbdaf59e700 1 -- 172.21.15.89:0/1588628490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb00b03c0 msgr2=0x7fbdb00cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.210+0000 7fbdaf59e700 1 --2- 172.21.15.89:0/1588628490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb00b03c0 0x7fbdb00cea70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbda00098e0 tx=0x7fbda00091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.210+0000 7fbdaf59e700 1 -- 172.21.15.89:0/1588628490 shutdown_connections 2023-10-10T20:46:26.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.210+0000 7fbdaf59e700 1 --2- 172.21.15.89:0/1588628490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb00b03c0 0x7fbdb00cea70 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.210+0000 7fbdaf59e700 1 --2- 172.21.15.89:0/1588628490 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdb00b0790 0x7fbdb00b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.210+0000 7fbdaf59e700 1 -- 172.21.15.89:0/1588628490 >> 172.21.15.89:0/1588628490 conn(0x7fbdb0063230 msgr2=0x7fbdb0063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:26.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.210+0000 7fbdaf59e700 1 -- 172.21.15.89:0/1588628490 shutdown_connections 2023-10-10T20:46:26.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.210+0000 7fbdaf59e700 1 -- 172.21.15.89:0/1588628490 wait complete. 2023-10-10T20:46:26.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.211+0000 7fbdaf59e700 1 Processor -- start 2023-10-10T20:46:26.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.211+0000 7fbdaf59e700 1 -- start start 2023-10-10T20:46:26.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.212+0000 7fbdaf59e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb00b03c0 0x7fbdb0157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:26.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.212+0000 7fbdaf59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb00b0790 0x7fbdb0069580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:26.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.212+0000 7fbdaf59e700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdb01591e0 0x7fbdb0069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:26.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.212+0000 7fbdaf59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb00c50f0 con 0x7fbdb00b0790 2023-10-10T20:46:26.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.212+0000 7fbdaf59e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb0158050 con 0x7fbdb00b03c0 2023-10-10T20:46:26.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.212+0000 7fbdaf59e700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb00c5340 con 0x7fbdb01591e0 2023-10-10T20:46:26.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.212+0000 7fbda5d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb00b0790 0x7fbdb0069580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:26.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.212+0000 7fbda5d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb00b0790 0x7fbdb0069580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43734/0 (socket says 172.21.15.89:43734) 2023-10-10T20:46:26.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.212+0000 7fbda5d9b700 1 -- 172.21.15.89:0/3127132653 learned_addr learned my addr 172.21.15.89:0/3127132653 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:26.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.212+0000 7fbdaed9d700 1 --2- 172.21.15.89:0/3127132653 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdb01591e0 0x7fbdb0069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:26.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.212+0000 7fbdae59c700 1 --2- 172.21.15.89:0/3127132653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb00b03c0 0x7fbdb0157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:26.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.213+0000 7fbda5d9b700 1 -- 172.21.15.89:0/3127132653 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdb01591e0 msgr2=0x7fbdb0069ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.213+0000 7fbda5d9b700 1 --2- 172.21.15.89:0/3127132653 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fbdb01591e0 0x7fbdb0069ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.213+0000 7fbda5d9b700 1 -- 172.21.15.89:0/3127132653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb00b03c0 msgr2=0x7fbdb0157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.213+0000 7fbda5d9b700 1 --2- 172.21.15.89:0/3127132653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbdb00b03c0 0x7fbdb0157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.213+0000 7fbda5d9b700 1 -- 172.21.15.89:0/3127132653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd98009420 con 0x7fbdb00b0790 2023-10-10T20:46:26.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.213+0000 7fbda5d9b700 1 --2- 172.21.15.89:0/3127132653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb00b0790 0x7fbdb0069580 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800ec80 tx=0x7fbd980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:26.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.213+0000 7fbda6ffd700 1 -- 172.21.15.89:0/3127132653 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9800c920 con 0x7fbdb00b0790 2023-10-10T20:46:26.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.213+0000 7fbda6ffd700 1 -- 172.21.15.89:0/3127132653 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9800ca80 con 0x7fbdb00b0790 2023-10-10T20:46:26.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.213+0000 7fbdaf59e700 1 -- 172.21.15.89:0/3127132653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda0009420 con 0x7fbdb00b0790 2023-10-10T20:46:26.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.213+0000 7fbda6ffd700 1 -- 172.21.15.89:0/3127132653 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9800ce00 con 0x7fbdb00b0790 2023-10-10T20:46:26.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.213+0000 7fbdaf59e700 1 -- 172.21.15.89:0/3127132653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb00c5900 con 0x7fbdb00b0790 2023-10-10T20:46:26.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.214+0000 7fbdaf59e700 1 -- 172.21.15.89:0/3127132653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbdb004c3c0 con 0x7fbdb00b0790 2023-10-10T20:46:26.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.214+0000 7fbda6ffd700 1 -- 172.21.15.89:0/3127132653 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbd98013680 con 0x7fbdb00b0790 2023-10-10T20:46:26.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.215+0000 7fbda6ffd700 1 --2- 172.21.15.89:0/3127132653 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbd90067320 0x7fbd900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:26.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.215+0000 7fbda6ffd700 1 -- 172.21.15.89:0/3127132653 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fbd980850f0 con 0x7fbdb00b0790 2023-10-10T20:46:26.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.215+0000 7fbdae59c700 1 --2- 172.21.15.89:0/3127132653 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbd90067320 0x7fbd900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:26.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.215+0000 7fbda6ffd700 1 --2- 172.21.15.89:0/3127132653 >> [v2:172.21.15.111:6824/91126,v1:172.21.15.111:6825/91126] conn(0x7fbd9006d170 0x7fbd9006f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:26.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.215+0000 7fbda6ffd700 1 -- 172.21.15.89:0/3127132653 --> [v2:172.21.15.111:6824/91126,v1:172.21.15.111:6825/91126] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbd9006fc50 con 0x7fbd9006d170 2023-10-10T20:46:26.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.215+0000 7fbda6ffd700 1 -- 172.21.15.89:0/3127132653 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd98085400 con 0x7fbdb00b0790 2023-10-10T20:46:26.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.215+0000 7fbdaed9d700 1 --2- 172.21.15.89:0/3127132653 >> [v2:172.21.15.111:6824/91126,v1:172.21.15.111:6825/91126] conn(0x7fbd9006d170 0x7fbd9006f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:26.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.215+0000 7fbdae59c700 1 --2- 172.21.15.89:0/3127132653 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbd90067320 0x7fbd900697d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbda00093f0 tx=0x7fbda0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:26.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.215+0000 7fbdaed9d700 1 --2- 172.21.15.89:0/3127132653 >> [v2:172.21.15.111:6824/91126,v1:172.21.15.111:6825/91126] conn(0x7fbd9006d170 0x7fbd9006f580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:26.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.216+0000 7fbda6ffd700 1 -- 172.21.15.89:0/3127132653 <== osd.7 v2:172.21.15.111:6824/91126 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fbd9006fc50 con 0x7fbd9006d170 2023-10-10T20:46:26.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.230+0000 7fbdaf59e700 1 -- 172.21.15.89:0/3127132653 --> [v2:172.21.15.111:6824/91126,v1:172.21.15.111:6825/91126] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbdb00cad50 con 0x7fbd9006d170 2023-10-10T20:46:26.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.230+0000 7fbda6ffd700 1 -- 172.21.15.89:0/3127132653 <== osd.7 v2:172.21.15.111:6824/91126 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fbdb00cad50 con 0x7fbd9006d170 2023-10-10T20:46:26.240 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-10T20:46:26.241 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-10T20:46:26.241 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-10T20:46:26.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.231+0000 7fbdaf59e700 1 -- 172.21.15.89:0/3127132653 >> [v2:172.21.15.111:6824/91126,v1:172.21.15.111:6825/91126] conn(0x7fbd9006d170 msgr2=0x7fbd9006f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.231+0000 7fbdaf59e700 1 --2- 172.21.15.89:0/3127132653 >> [v2:172.21.15.111:6824/91126,v1:172.21.15.111:6825/91126] conn(0x7fbd9006d170 0x7fbd9006f580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.231+0000 7fbdaf59e700 1 -- 172.21.15.89:0/3127132653 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbd90067320 msgr2=0x7fbd900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.231+0000 7fbdaf59e700 1 --2- 172.21.15.89:0/3127132653 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fbd90067320 0x7fbd900697d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbda00093f0 tx=0x7fbda0017000 comp rx=0 tx=0).stop 2023-10-10T20:46:26.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.231+0000 7fbdaf59e700 1 -- 172.21.15.89:0/3127132653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb00b0790 msgr2=0x7fbdb0069580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.231+0000 7fbdaf59e700 1 --2- 172.21.15.89:0/3127132653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb00b0790 0x7fbdb0069580 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800ec80 tx=0x7fbd980091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.231+0000 7fbdaed9d700 1 -- 172.21.15.89:0/3127132653 reap_dead start 2023-10-10T20:46:26.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.231+0000 7fbdaf59e700 1 -- 172.21.15.89:0/3127132653 shutdown_connections 2023-10-10T20:46:26.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.231+0000 7fbdaf59e700 1 -- 172.21.15.89:0/3127132653 >> 172.21.15.89:0/3127132653 conn(0x7fbdb0063230 msgr2=0x7fbdb01588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:26.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.231+0000 7fbdaf59e700 1 -- 172.21.15.89:0/3127132653 shutdown_connections 2023-10-10T20:46:26.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.231+0000 7fbdaf59e700 1 -- 172.21.15.89:0/3127132653 wait complete. 2023-10-10T20:46:26.246 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-10T20:46:26.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.359+0000 7f236e205700 1 Processor -- start 2023-10-10T20:46:26.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.359+0000 7f236e205700 1 -- start start 2023-10-10T20:46:26.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.359+0000 7f236e205700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f23680b1850 0x7f23680b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:26.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.359+0000 7f236e205700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f23680ae680 0x7f23681572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:26.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.360+0000 7f236e205700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23680cb300 con 0x7f23680aeb00 2023-10-10T20:46:26.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.360+0000 7f236e205700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236806fc30 con 0x7f23680ae680 2023-10-10T20:46:26.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.360+0000 7f236e205700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23680cc260 con 0x7f23680b1850 2023-10-10T20:46:26.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.360+0000 7f2366ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f23680ae680 0x7f23681572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:26.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.360+0000 7f2366ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f23680ae680 0x7f23681572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43738/0 (socket says 172.21.15.89:43738) 2023-10-10T20:46:26.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.360+0000 7f2366ffd700 1 -- 172.21.15.89:0/2995908231 learned_addr learned my addr 172.21.15.89:0/2995908231 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:26.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.360+0000 7f2366ffd700 1 -- 172.21.15.89:0/2995908231 >> v2:172.21.15.111:3300/0 conn(0x7f23680b1850 msgr2=0x7f23680b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:26.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.360+0000 7f23677fe700 1 --2- 172.21.15.89:0/2995908231 >> v2:172.21.15.111:3300/0 conn(0x7f23680b1850 0x7f23680b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:26.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.360+0000 7f2366ffd700 1 --2- 172.21.15.89:0/2995908231 >> v2:172.21.15.111:3300/0 conn(0x7f23680b1850 0x7f23680b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.360+0000 7f2366ffd700 1 -- 172.21.15.89:0/2995908231 >> v1:172.21.15.111:6789/0 conn(0x7f23680aeb00 legacy=0x7f236815b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:26.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.360+0000 7f2366ffd700 1 -- 172.21.15.89:0/2995908231 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23680cb5e0 con 0x7f23680ae680 2023-10-10T20:46:26.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.360+0000 7f2366ffd700 1 --2- 172.21.15.89:0/2995908231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23680ae680 0x7f23681572e0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f23680641e0 tx=0x7f2358009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19584216fd553b4c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:26.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.360+0000 7f23667fc700 1 -- 172.21.15.89:0/2995908231 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235800b820 con 0x7f23680ae680 2023-10-10T20:46:26.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.360+0000 7f23667fc700 1 -- 172.21.15.89:0/2995908231 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235800b980 con 0x7f23680ae680 2023-10-10T20:46:26.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.360+0000 7f23667fc700 1 -- 172.21.15.89:0/2995908231 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235800bd20 con 0x7f23680ae680 2023-10-10T20:46:26.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.361+0000 7f236e205700 1 -- 172.21.15.89:0/2995908231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23680ae680 msgr2=0x7f23681572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.361+0000 7f236e205700 1 --2- 172.21.15.89:0/2995908231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23680ae680 0x7f23681572e0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f23680641e0 tx=0x7f2358009250 comp rx=0 tx=0).stop 2023-10-10T20:46:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.361+0000 7f236e205700 1 -- 172.21.15.89:0/2995908231 shutdown_connections 2023-10-10T20:46:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.361+0000 7f236e205700 1 --2- 172.21.15.89:0/2995908231 >> v2:172.21.15.111:3300/0 conn(0x7f23680b1850 0x7f23680b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.361+0000 7f236e205700 1 --2- 172.21.15.89:0/2995908231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23680ae680 0x7f23681572e0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.361+0000 7f236e205700 1 -- 172.21.15.89:0/2995908231 >> 172.21.15.89:0/2995908231 conn(0x7f236806a320 msgr2=0x7f23680cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:26.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.361+0000 7f236e205700 1 -- 172.21.15.89:0/2995908231 shutdown_connections 2023-10-10T20:46:26.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.361+0000 7f236e205700 1 -- 172.21.15.89:0/2995908231 wait complete. 2023-10-10T20:46:26.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.362+0000 7f236e205700 1 Processor -- start 2023-10-10T20:46:26.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.362+0000 7f236e205700 1 -- start start 2023-10-10T20:46:26.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.363+0000 7f236e205700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23680ae680 0x7f2368150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:26.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.363+0000 7f236e205700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23680aeb00 0x7f23680a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:26.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.363+0000 7f236e205700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23680b1850 0x7f23680acf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:26.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.363+0000 7f236e205700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2368151210 con 0x7f23680b1850 2023-10-10T20:46:26.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.363+0000 7f236e205700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236816b2c0 con 0x7f23680ae680 2023-10-10T20:46:26.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.363+0000 7f236e205700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236816b5a0 con 0x7f23680aeb00 2023-10-10T20:46:26.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.363+0000 7f2367fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23680b1850 0x7f23680acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:26.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.363+0000 7f2367fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23680b1850 0x7f23680acf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43740/0 (socket says 172.21.15.89:43740) 2023-10-10T20:46:26.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.363+0000 7f2367fff700 1 -- 172.21.15.89:0/1851710256 learned_addr learned my addr 172.21.15.89:0/1851710256 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:26.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.363+0000 7f23677fe700 1 --2- 172.21.15.89:0/1851710256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23680ae680 0x7f2368150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:26.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.363+0000 7f2366ffd700 1 --2- 172.21.15.89:0/1851710256 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23680aeb00 0x7f23680a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:26.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.363+0000 7f2367fff700 1 -- 172.21.15.89:0/1851710256 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23680aeb00 msgr2=0x7f23680a63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.363+0000 7f2367fff700 1 --2- 172.21.15.89:0/1851710256 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23680aeb00 0x7f23680a63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.363+0000 7f2367fff700 1 -- 172.21.15.89:0/1851710256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23680ae680 msgr2=0x7f2368150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.363+0000 7f2367fff700 1 --2- 172.21.15.89:0/1851710256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23680ae680 0x7f2368150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.363+0000 7f2367fff700 1 -- 172.21.15.89:0/1851710256 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23580094f0 con 0x7f23680b1850 2023-10-10T20:46:26.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.363+0000 7f2367fff700 1 --2- 172.21.15.89:0/1851710256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23680b1850 0x7f23680acf30 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f236000c930 tx=0x7f23600026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:26.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.364+0000 7f234f7fe700 1 -- 172.21.15.89:0/1851710256 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360012630 con 0x7f23680b1850 2023-10-10T20:46:26.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.364+0000 7f234f7fe700 1 -- 172.21.15.89:0/1851710256 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23600127d0 con 0x7f23680b1850 2023-10-10T20:46:26.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.364+0000 7f236e205700 1 -- 172.21.15.89:0/1851710256 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236816b880 con 0x7f23680b1850 2023-10-10T20:46:26.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.364+0000 7f234f7fe700 1 -- 172.21.15.89:0/1851710256 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360012b90 con 0x7f23680b1850 2023-10-10T20:46:26.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.364+0000 7f236e205700 1 -- 172.21.15.89:0/1851710256 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23680a6920 con 0x7f23680b1850 2023-10-10T20:46:26.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.365+0000 7f236e205700 1 -- 172.21.15.89:0/1851710256 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236804c3c0 con 0x7f23680b1850 2023-10-10T20:46:26.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.365+0000 7f234f7fe700 1 -- 172.21.15.89:0/1851710256 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f236001a3b0 con 0x7f23680b1850 2023-10-10T20:46:26.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.366+0000 7f234f7fe700 1 --2- 172.21.15.89:0/1851710256 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2354067320 0x7f23540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:26.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.366+0000 7f234f7fe700 1 -- 172.21.15.89:0/1851710256 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2360084600 con 0x7f23680b1850 2023-10-10T20:46:26.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.366+0000 7f23677fe700 1 --2- 172.21.15.89:0/1851710256 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2354067320 0x7f23540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:26.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.366+0000 7f23677fe700 1 --2- 172.21.15.89:0/1851710256 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2354067320 0x7f23540697d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f235c006fd0 tx=0x7f235c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:26.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.370+0000 7f234f7fe700 1 -- 172.21.15.89:0/1851710256 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2360051590 con 0x7f23680b1850 2023-10-10T20:46:26.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.553+0000 7f236e205700 1 -- 172.21.15.89:0/1851710256 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f23680cb8c0 con 0x7f2354067320 2023-10-10T20:46:26.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.553+0000 7f234f7fe700 1 -- 172.21.15.89:0/1851710256 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f23680cb8c0 con 0x7f2354067320 2023-10-10T20:46:26.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.555+0000 7f236e205700 1 -- 172.21.15.89:0/1851710256 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2354067320 msgr2=0x7f23540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.555+0000 7f236e205700 1 --2- 172.21.15.89:0/1851710256 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2354067320 0x7f23540697d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f235c006fd0 tx=0x7f235c008040 comp rx=0 tx=0).stop 2023-10-10T20:46:26.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.555+0000 7f236e205700 1 -- 172.21.15.89:0/1851710256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23680b1850 msgr2=0x7f23680acf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.555+0000 7f236e205700 1 --2- 172.21.15.89:0/1851710256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23680b1850 0x7f23680acf30 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f236000c930 tx=0x7f23600026e0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.555+0000 7f236e205700 1 -- 172.21.15.89:0/1851710256 shutdown_connections 2023-10-10T20:46:26.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.555+0000 7f236e205700 1 --2- 172.21.15.89:0/1851710256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23680b1850 0x7f23680acf30 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.555+0000 7f236e205700 1 --2- 172.21.15.89:0/1851710256 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f2354067320 0x7f23540697d0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.555+0000 7f236e205700 1 --2- 172.21.15.89:0/1851710256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23680ae680 0x7f2368150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.555+0000 7f236e205700 1 --2- 172.21.15.89:0/1851710256 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f23680aeb00 0x7f23680a63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.555+0000 7f236e205700 1 -- 172.21.15.89:0/1851710256 >> 172.21.15.89:0/1851710256 conn(0x7f236806a320 msgr2=0x7f2368152cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:26.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.555+0000 7f236e205700 1 -- 172.21.15.89:0/1851710256 shutdown_connections 2023-10-10T20:46:26.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.555+0000 7f236e205700 1 -- 172.21.15.89:0/1851710256 wait complete. 2023-10-10T20:46:26.561 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 7 to deep-scrub 2023-10-10T20:46:26.569 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:46:26.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.683+0000 7f0bfb5e3700 1 Processor -- start 2023-10-10T20:46:26.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.683+0000 7f0bfb5e3700 1 -- start start 2023-10-10T20:46:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.684+0000 7f0bfb5e3700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0bf4159890 0x7f0bf415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.684+0000 7f0bfb5e3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0bf4159c60 0x7f0bf415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:26.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.684+0000 7f0bfb5e3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf415a160 con 0x7f0bf415db10 2023-10-10T20:46:26.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.684+0000 7f0bfb5e3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf415df50 con 0x7f0bf4159c60 2023-10-10T20:46:26.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.684+0000 7f0bfb5e3700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf40d13a0 con 0x7f0bf4159890 2023-10-10T20:46:26.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.684+0000 7f0bf8b7e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0bf4159c60 0x7f0bf415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:26.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.684+0000 7f0bf9b80700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0bf415db10 0x7f0bf4069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51780/0 (socket says 172.21.15.89:51780) 2023-10-10T20:46:26.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.684+0000 7f0bf9b80700 1 -- 172.21.15.89:0/1042233368 learned_addr learned my addr 172.21.15.89:0/1042233368 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:26.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.684+0000 7f0bf937f700 1 --2- 172.21.15.89:0/1042233368 >> v2:172.21.15.111:3300/0 conn(0x7f0bf4159890 0x7f0bf415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:26.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.684+0000 7f0bf8b7e700 1 -- 172.21.15.89:0/1042233368 >> v2:172.21.15.111:3300/0 conn(0x7f0bf4159890 msgr2=0x7f0bf415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:26.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.684+0000 7f0bf8b7e700 1 --2- 172.21.15.89:0/1042233368 >> v2:172.21.15.111:3300/0 conn(0x7f0bf4159890 0x7f0bf415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.684+0000 7f0bf8b7e700 1 -- 172.21.15.89:0/1042233368 >> v1:172.21.15.89:6789/0 conn(0x7f0bf415db10 legacy=0x7f0bf4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.684+0000 7f0bf8b7e700 1 -- 172.21.15.89:0/1042233368 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf415e140 con 0x7f0bf4159c60 2023-10-10T20:46:26.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.684+0000 7f0bebfff700 1 -- 172.21.15.89:0/1042233368 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3610998514 0 0) 0x7f0bf415a160 con 0x7f0bf415db10 2023-10-10T20:46:26.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.684+0000 7f0bf937f700 1 --2- 172.21.15.89:0/1042233368 >> v2:172.21.15.111:3300/0 conn(0x7f0bf4159890 0x7f0bf415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:46:26.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.684+0000 7f0bf8b7e700 1 --2- 172.21.15.89:0/1042233368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bf4159c60 0x7f0bf415d5d0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40d1090 tx=0x7f0be40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0341c9cd9b80270 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:26.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.684+0000 7f0bebfff700 1 -- 172.21.15.89:0/1042233368 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be400b6b0 con 0x7f0bf4159c60 2023-10-10T20:46:26.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.685+0000 7f0bebfff700 1 -- 172.21.15.89:0/1042233368 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be400b810 con 0x7f0bf4159c60 2023-10-10T20:46:26.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.685+0000 7f0bebfff700 1 -- 172.21.15.89:0/1042233368 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be400bbb0 con 0x7f0bf4159c60 2023-10-10T20:46:26.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.685+0000 7f0bfb5e3700 1 -- 172.21.15.89:0/1042233368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bf4159c60 msgr2=0x7f0bf415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.685+0000 7f0bfb5e3700 1 --2- 172.21.15.89:0/1042233368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bf4159c60 0x7f0bf415d5d0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40d1090 tx=0x7f0be40091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.685+0000 7f0bfb5e3700 1 -- 172.21.15.89:0/1042233368 shutdown_connections 2023-10-10T20:46:26.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.685+0000 7f0bfb5e3700 1 --2- 172.21.15.89:0/1042233368 >> v2:172.21.15.111:3300/0 conn(0x7f0bf4159890 0x7f0bf415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.685+0000 7f0bfb5e3700 1 --2- 172.21.15.89:0/1042233368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bf4159c60 0x7f0bf415d5d0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.685+0000 7f0bfb5e3700 1 -- 172.21.15.89:0/1042233368 >> 172.21.15.89:0/1042233368 conn(0x7f0bf4068e20 msgr2=0x7f0bf40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:26.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.685+0000 7f0bfb5e3700 1 -- 172.21.15.89:0/1042233368 shutdown_connections 2023-10-10T20:46:26.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.685+0000 7f0bfb5e3700 1 -- 172.21.15.89:0/1042233368 wait complete. 2023-10-10T20:46:26.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.686+0000 7f0bfb5e3700 1 Processor -- start 2023-10-10T20:46:26.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.686+0000 7f0bfb5e3700 1 -- start start 2023-10-10T20:46:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.687+0000 7f0bfb5e3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bf4159890 0x7f0bf414a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.687+0000 7f0bfb5e3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf4159c60 0x7f0bf4151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.687+0000 7f0bfb5e3700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0bf415db10 0x7f0bf414b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.687+0000 7f0bfb5e3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40c7140 con 0x7f0bf4159890 2023-10-10T20:46:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.687+0000 7f0bfb5e3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40c6530 con 0x7f0bf4159c60 2023-10-10T20:46:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.687+0000 7f0bfb5e3700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40c6810 con 0x7f0bf415db10 2023-10-10T20:46:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.687+0000 7f0bf937f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bf4159890 0x7f0bf414a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.687+0000 7f0bf937f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bf4159890 0x7f0bf414a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43750/0 (socket says 172.21.15.89:43750) 2023-10-10T20:46:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.687+0000 7f0bf937f700 1 -- 172.21.15.89:0/2832195915 learned_addr learned my addr 172.21.15.89:0/2832195915 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.687+0000 7f0bf8b7e700 1 --2- 172.21.15.89:0/2832195915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf4159c60 0x7f0bf4151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:26.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.687+0000 7f0bf9b80700 1 --2- 172.21.15.89:0/2832195915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0bf415db10 0x7f0bf414b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:26.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.687+0000 7f0bf937f700 1 -- 172.21.15.89:0/2832195915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0bf415db10 msgr2=0x7f0bf414b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.688+0000 7f0bf937f700 1 --2- 172.21.15.89:0/2832195915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0bf415db10 0x7f0bf414b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.688+0000 7f0bf937f700 1 -- 172.21.15.89:0/2832195915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf4159c60 msgr2=0x7f0bf4151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.688+0000 7f0bf937f700 1 --2- 172.21.15.89:0/2832195915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf4159c60 0x7f0bf4151fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.688+0000 7f0bf937f700 1 -- 172.21.15.89:0/2832195915 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be4009420 con 0x7f0bf4159890 2023-10-10T20:46:26.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.688+0000 7f0bf937f700 1 --2- 172.21.15.89:0/2832195915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bf4159890 0x7f0bf414a550 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000c830 tx=0x7f0bf0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:26.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.688+0000 7f0be97fa700 1 -- 172.21.15.89:0/2832195915 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0011600 con 0x7f0bf4159890 2023-10-10T20:46:26.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.688+0000 7f0be97fa700 1 -- 172.21.15.89:0/2832195915 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf0011760 con 0x7f0bf4159890 2023-10-10T20:46:26.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.688+0000 7f0bfb5e3700 1 -- 172.21.15.89:0/2832195915 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf40c6af0 con 0x7f0bf4159890 2023-10-10T20:46:26.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.688+0000 7f0be97fa700 1 -- 172.21.15.89:0/2832195915 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0011b20 con 0x7f0bf4159890 2023-10-10T20:46:26.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.688+0000 7f0bfb5e3700 1 -- 172.21.15.89:0/2832195915 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf414aa90 con 0x7f0bf4159890 2023-10-10T20:46:26.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.689+0000 7f0bfb5e3700 1 -- 172.21.15.89:0/2832195915 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf414b240 con 0x7f0bf4159890 2023-10-10T20:46:26.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.689+0000 7f0be97fa700 1 -- 172.21.15.89:0/2832195915 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0bf0018360 con 0x7f0bf4159890 2023-10-10T20:46:26.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.689+0000 7f0be97fa700 1 --2- 172.21.15.89:0/2832195915 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0bd8067320 0x7f0bd80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:26.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.690+0000 7f0be97fa700 1 -- 172.21.15.89:0/2832195915 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0bf0084a10 con 0x7f0bf4159890 2023-10-10T20:46:26.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.690+0000 7f0bf8b7e700 1 --2- 172.21.15.89:0/2832195915 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0bd8067320 0x7f0bd80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:26.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.690+0000 7f0bf8b7e700 1 --2- 172.21.15.89:0/2832195915 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0bd8067320 0x7f0bd80697d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0be40093f0 tx=0x7f0be4017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.694+0000 7f0be97fa700 1 -- 172.21.15.89:0/2832195915 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0bf0051890 con 0x7f0bf4159890 2023-10-10T20:46:26.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.872+0000 7f0bfb5e3700 1 -- 172.21.15.89:0/2832195915 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0bf40c7280 con 0x7f0bd8067320 2023-10-10T20:46:26.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.874+0000 7f0be97fa700 1 -- 172.21.15.89:0/2832195915 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50891 (secure 0 0 0) 0x7f0be400bac0 con 0x7f0bd8067320 2023-10-10T20:46:26.877 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:46:26.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.876+0000 7f0bfb5e3700 1 -- 172.21.15.89:0/2832195915 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0bd8067320 msgr2=0x7f0bd80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.876+0000 7f0bfb5e3700 1 --2- 172.21.15.89:0/2832195915 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0bd8067320 0x7f0bd80697d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0be40093f0 tx=0x7f0be4017000 comp rx=0 tx=0).stop 2023-10-10T20:46:26.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.876+0000 7f0bfb5e3700 1 -- 172.21.15.89:0/2832195915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bf4159890 msgr2=0x7f0bf414a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:26.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.876+0000 7f0bfb5e3700 1 --2- 172.21.15.89:0/2832195915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bf4159890 0x7f0bf414a550 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000c830 tx=0x7f0bf0008040 comp rx=0 tx=0).stop 2023-10-10T20:46:26.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.876+0000 7f0bfb5e3700 1 -- 172.21.15.89:0/2832195915 shutdown_connections 2023-10-10T20:46:26.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.876+0000 7f0bfb5e3700 1 --2- 172.21.15.89:0/2832195915 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f0bf415db10 0x7f0bf414b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.876+0000 7f0bfb5e3700 1 --2- 172.21.15.89:0/2832195915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bf4159890 0x7f0bf414a550 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.876+0000 7f0bfb5e3700 1 --2- 172.21.15.89:0/2832195915 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f0bd8067320 0x7f0bd80697d0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.876+0000 7f0bfb5e3700 1 --2- 172.21.15.89:0/2832195915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf4159c60 0x7f0bf4151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:26.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.876+0000 7f0bfb5e3700 1 -- 172.21.15.89:0/2832195915 >> 172.21.15.89:0/2832195915 conn(0x7f0bf4068e20 msgr2=0x7f0bf414d430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:26.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.876+0000 7f0bfb5e3700 1 -- 172.21.15.89:0/2832195915 shutdown_connections 2023-10-10T20:46:26.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:26.876+0000 7f0bfb5e3700 1 -- 172.21.15.89:0/2832195915 wait complete. 2023-10-10T20:46:26.882 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:46:26.890 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":556,"stamp":"2023-10-10T20:46:26.798426+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2432452,"kb_used_data":21380,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752542268,"statfs":{"total":773094113280,"available":770603282432,"internally_reserved":0,"allocated":21893120,"data_stored":10486591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.591955"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":346,"reported_epoch":202,"state":"active+clean","last_fresh":"2023-10-10T20:46:17.209476+0000","last_change":"2023-10-10T20:34:30.078797+0000","last_active":"2023-10-10T20:46:17.209476+0000","last_peered":"2023-10-10T20:46:17.209476+0000","last_clean":"2023-10-10T20:46:17.209476+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:46:17.209476+0000","last_undegraded":"2023-10-10T20:46:17.209476+0000","last_fullsized":"2023-10-10T20:46:17.209476+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:34:28.804249+0000","last_clean_scrub_stamp":"2023-10-10T20:34:28.804249+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:19.445546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574995,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304492,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067348,"statfs":{"total":96636764160,"available":96324964352,"internally_reserved":0,"allocated":3182592,"data_stored":1753720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52200000000000002,"15min":0.67000000000000004},"min":{"1min":0.35499999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71499999999999997,"5min":0.748,"15min":6.8630000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52600000000000002,"15min":0.57699999999999996},"min":{"1min":0.33900000000000002,"5min":0.32300000000000001,"15min":0.307},"max":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":4.1959999999999997},"last":0.61399999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.76100000000000001},"min":{"1min":0.315,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68899999999999995,"5min":0.78400000000000003,"15min":10.1},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54700000000000004,"15min":0.64100000000000001},"min":{"1min":0.39300000000000002,"5min":0.35199999999999998,"15min":0.28999999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":6.806},"last":0.55200000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.68600000000000005},"min":{"1min":0.45900000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.82999999999999996,"5min":0.83099999999999996,"15min":6.9340000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.55300000000000005,"15min":0.64200000000000002},"min":{"1min":0.47499999999999998,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":6.9870000000000001},"last":0.58599999999999997}]},{"osd":3,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55200000000000005,"15min":0.65600000000000003},"min":{"1min":0.48399999999999999,"5min":0.371,"15min":0.32800000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":6.6719999999999997},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57199999999999995,"15min":0.621},"min":{"1min":0.45000000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.74399999999999999,"5min":0.77700000000000002,"15min":4.1609999999999996},"last":0.66100000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.45900000000000002,"15min":0.46400000000000002},"min":{"1min":0.29799999999999999,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.68200000000000005,"5min":0.69799999999999995,"15min":0.77200000000000002},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.5,"15min":0.51500000000000001},"min":{"1min":0.374,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71999999999999997,"5min":0.73899999999999999,"15min":0.80300000000000005},"last":0.495}]},{"osd":5,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50600000000000001,"15min":0.53100000000000003},"min":{"1min":0.35899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70599999999999996,"5min":0.77900000000000003,"15min":1.3180000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51200000000000001,"15min":0.52600000000000002},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.29499999999999998},"max":{"1min":0.77100000000000002,"5min":0.78200000000000003,"15min":0.84099999999999997},"last":0.443}]},{"osd":6,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":1.7509999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.375,"5min":0.35499999999999998,"15min":0.32600000000000001},"max":{"1min":0.81699999999999995,"5min":0.81999999999999995,"15min":1.7090000000000001},"last":0.63400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303912,"kb_used_data":2528,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067928,"statfs":{"total":96636764160,"available":96325558272,"internally_reserved":0,"allocated":2588672,"data_stored":1163352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53700000000000003,"15min":0.59899999999999998},"min":{"1min":0.40600000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.76800000000000002,"5min":0.83099999999999996,"15min":7.7590000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.53600000000000003,"15min":0.622},"min":{"1min":0.46100000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78700000000000003,"5min":1.075,"15min":7.8090000000000002},"last":0.52500000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54600000000000004,"15min":0.623},"min":{"1min":0.442,"5min":0.29199999999999998,"15min":0.27800000000000002},"max":{"1min":0.73399999999999999,"5min":0.91300000000000003,"15min":7.867},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.72099999999999997},"min":{"1min":0.438,"5min":0.34499999999999997,"15min":0.33700000000000002},"max":{"1min":0.755,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.46999999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.35699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71899999999999997,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58899999999999997,"15min":0.70999999999999996},"min":{"1min":0.50900000000000001,"5min":0.33400000000000002,"15min":0.318},"max":{"1min":0.76000000000000001,"5min":2.641,"15min":6.7629999999999999},"last":0.45100000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57099999999999995,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72199999999999998,"5min":1.107,"15min":4.8620000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56899999999999995,"15min":0.64000000000000001},"min":{"1min":0.47499999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":1.718,"15min":7.9009999999999998},"last":0.48999999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.51100000000000001,"15min":0.48999999999999999},"min":{"1min":0.32300000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66000000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.313},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.40899999999999997,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":0.745,"5min":1.032,"15min":1.032},"last":0.503}]},{"osd":5,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.52900000000000003,"15min":0.50700000000000001},"min":{"1min":0.33300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73299999999999998,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.55900000000000005,"15min":0.54400000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33600000000000002},"max":{"1min":0.73299999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.374}]},{"osd":7,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.35799999999999998,"5min":0.33300000000000002,"15min":0.315},"max":{"1min":0.70599999999999996,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.438},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.58699999999999997,"15min":0.56399999999999995},"min":{"1min":0.376,"5min":0.36599999999999999,"15min":0.35099999999999998},"max":{"1min":0.78000000000000003,"5min":1.681,"15min":1.681},"last":0.34000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303912,"kb_used_data":2528,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067928,"statfs":{"total":96636764160,"available":96325558272,"internally_reserved":0,"allocated":2588672,"data_stored":1163352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.33000000000000002,"15min":0.40100000000000002},"min":{"1min":0.23599999999999999,"5min":0.215,"15min":0.17899999999999999},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":5.0380000000000003},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.309,"5min":0.223,"15min":0.20899999999999999},"max":{"1min":0.55500000000000005,"5min":0.65500000000000003,"15min":1.194},"last":1.1379999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.26700000000000002,"5min":0.23599999999999999,"15min":0.20599999999999999},"max":{"1min":0.49199999999999999,"5min":0.53800000000000003,"15min":1.2869999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.432,"5min":0.377,"15min":0.434},"min":{"1min":0.318,"5min":0.23999999999999999,"15min":0.22},"max":{"1min":0.57499999999999996,"5min":0.68600000000000005,"15min":5.0919999999999996},"last":0.375}]},{"osd":3,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.40100000000000002,"15min":0.45800000000000002},"min":{"1min":0.35999999999999999,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":0.54000000000000004,"5min":0.64500000000000002,"15min":5.1159999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39500000000000002},"min":{"1min":0.34899999999999998,"5min":0.23799999999999999,"15min":0.19900000000000001},"max":{"1min":0.55500000000000005,"5min":0.67000000000000004,"15min":1.2170000000000001},"last":0.85499999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.48199999999999998,"5min":0.34200000000000003,"15min":0.30199999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.254},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.57599999999999996,"15min":0.63900000000000001},"min":{"1min":0.45100000000000001,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.83899999999999997,"5min":0.88600000000000001,"15min":4.9850000000000003},"last":0.51900000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.46700000000000003,"5min":0.34799999999999998,"15min":0.34399999999999997},"max":{"1min":0.75900000000000001,"5min":0.91700000000000004,"15min":1.1799999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.58899999999999997,"15min":0.59799999999999998},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":1.3109999999999999},"last":0.66800000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.61099999999999999,"15min":0.72699999999999998},"min":{"1min":0.53200000000000003,"5min":0.378,"15min":0.35199999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":5.0659999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.623,"15min":0.626},"min":{"1min":0.44400000000000001,"5min":0.38600000000000001,"15min":0.36299999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":1.171},"last":0.92800000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.48599999999999999,"5min":0.38200000000000001,"15min":0.36499999999999999},"max":{"1min":0.86699999999999999,"5min":0.94199999999999995,"15min":1.214},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60699999999999998,"15min":0.66500000000000004},"min":{"1min":0.51600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":4.0499999999999998},"last":0.71099999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574994,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304488,"kb_used_data":3104,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067352,"statfs":{"total":96636764160,"available":96324968448,"internally_reserved":0,"allocated":3178496,"data_stored":1752759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.33500000000000002,"15min":0.42699999999999999},"min":{"1min":0.308,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.47199999999999998,"5min":0.63500000000000001,"15min":7.9249999999999998},"last":0.441},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.35499999999999998,"15min":0.42399999999999999},"min":{"1min":0.31900000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55600000000000005,"5min":0.63400000000000001,"15min":6.6299999999999999},"last":0.33900000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.36399999999999999,"15min":0.45200000000000001},"min":{"1min":0.26600000000000001,"5min":0.23100000000000001,"15min":0.216},"max":{"1min":0.63100000000000001,"5min":0.67800000000000005,"15min":8.2509999999999994},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.374,"15min":0.44600000000000001},"min":{"1min":0.29399999999999998,"5min":0.253,"15min":0.222},"max":{"1min":0.55900000000000005,"5min":0.65000000000000002,"15min":6.7370000000000001},"last":0.29099999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38400000000000001,"15min":0.46400000000000002},"min":{"1min":0.35899999999999999,"5min":0.22,"15min":0.21099999999999999},"max":{"1min":0.52600000000000002,"5min":0.53800000000000003,"15min":7.7960000000000003},"last":0.375},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.42899999999999999,"15min":0.51400000000000001},"min":{"1min":0.35899999999999999,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.58199999999999996,"5min":0.71699999999999997,"15min":7.8929999999999998},"last":0.36199999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57099999999999995,"15min":0.64000000000000001},"min":{"1min":0.44400000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70499999999999996,"5min":0.94099999999999995,"15min":6.6929999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59099999999999997,"15min":0.67700000000000005},"min":{"1min":0.40000000000000002,"5min":0.371,"15min":0.33900000000000002},"max":{"1min":0.80000000000000004,"5min":0.84399999999999997,"15min":7.6920000000000002},"last":0.65000000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58499999999999996,"15min":0.66200000000000003},"min":{"1min":0.438,"5min":0.38400000000000001,"15min":0.36299999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":7.7690000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.60899999999999999,"15min":0.68899999999999995},"min":{"1min":0.49299999999999999,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.81899999999999995,"5min":0.97599999999999998,"15min":7.8659999999999997},"last":0.59399999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.624,"15min":0.71699999999999997},"min":{"1min":0.53700000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.85999999999999999,"5min":0.87,"15min":7.8360000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63800000000000001,"15min":0.69499999999999995},"min":{"1min":0.53000000000000003,"5min":0.41499999999999998,"15min":0.36599999999999999},"max":{"1min":0.83099999999999996,"5min":0.875,"15min":6.6660000000000004},"last":0.53800000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.626,"15min":0.68200000000000005},"min":{"1min":0.42499999999999999,"5min":0.42099999999999999,"15min":0.36299999999999999},"max":{"1min":0.86199999999999999,"5min":0.90600000000000003,"15min":6.7160000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59899999999999998,"15min":0.69099999999999995},"min":{"1min":0.49399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.873,"5min":0.89500000000000002,"15min":7.7370000000000001},"last":0.622}]}]},{"osd":2,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303912,"kb_used_data":2528,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067928,"statfs":{"total":96636764160,"available":96325558272,"internally_reserved":0,"allocated":2588672,"data_stored":1163352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.33600000000000002,"15min":0.32600000000000001},"min":{"1min":0.26800000000000002,"5min":0.20200000000000001,"15min":0.189},"max":{"1min":0.52700000000000002,"5min":0.59399999999999997,"15min":0.67900000000000005},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.34999999999999998},"min":{"1min":0.25600000000000001,"5min":0.219,"15min":0.21199999999999999},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.746},"last":0.214}]},{"osd":1,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.54100000000000004},"min":{"1min":0.29099999999999998,"5min":0.221,"15min":0.21199999999999999},"max":{"1min":0.52900000000000003,"5min":0.59999999999999998,"15min":11.967000000000001},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.38200000000000001,"15min":0.42599999999999999},"min":{"1min":0.35699999999999998,"5min":0.245,"15min":0.23999999999999999},"max":{"1min":0.53900000000000003,"5min":0.65600000000000003,"15min":4.1100000000000003},"last":0.27100000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.37,"15min":0.45900000000000002},"min":{"1min":0.32100000000000001,"5min":0.23699999999999999,"15min":0.22800000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":6.3040000000000003},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.42699999999999999,"15min":0.59699999999999998},"min":{"1min":0.39700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.58099999999999996,"5min":0.70099999999999996,"15min":11.935},"last":0.375}]},{"osd":4,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59599999999999997,"15min":0.56699999999999995},"min":{"1min":0.41399999999999998,"5min":0.375,"15min":0.34999999999999998},"max":{"1min":0.84199999999999997,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76900000000000002,"5min":1.302,"15min":1.302},"last":0.48499999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.56999999999999995},"min":{"1min":0.40999999999999998,"5min":0.38900000000000001,"15min":0.378},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.95699999999999996},"last":0.496},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59699999999999998,"15min":0.58199999999999996},"min":{"1min":0.45100000000000001,"5min":0.35999999999999999,"15min":0.34300000000000003},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.96099999999999997},"last":0.58999999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61199999999999999,"15min":0.60099999999999998},"min":{"1min":0.497,"5min":0.40300000000000002,"15min":0.33200000000000002},"max":{"1min":0.79900000000000004,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63800000000000001,"15min":0.61199999999999999},"min":{"1min":0.499,"5min":0.35199999999999998,"15min":0.34499999999999997},"max":{"1min":0.878,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.60199999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61799999999999999,"15min":0.59299999999999997},"min":{"1min":0.44800000000000001,"5min":0.41899999999999998,"15min":0.38700000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.052},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.625,"15min":0.69399999999999995},"min":{"1min":0.46200000000000002,"5min":0.40999999999999998,"15min":0.36099999999999999},"max":{"1min":0.83099999999999996,"5min":1.274,"15min":6.2649999999999997},"last":0.55000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574996,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303912,"kb_used_data":2528,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067928,"statfs":{"total":96636764160,"available":96325558272,"internally_reserved":0,"allocated":2588672,"data_stored":1163352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.32500000000000001,"15min":0.317},"min":{"1min":0.27100000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.48499999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.35299999999999998,"15min":0.33000000000000002},"min":{"1min":0.29199999999999998,"5min":0.189,"15min":0.189},"max":{"1min":0.64000000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.311}]},{"osd":1,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.37,"15min":0.34499999999999997},"min":{"1min":0.28799999999999998,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.68500000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.38900000000000001},"min":{"1min":0.309,"5min":0.26800000000000002,"15min":0.251},"max":{"1min":0.627,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.441}]},{"osd":2,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.38400000000000001,"15min":0.36299999999999999},"min":{"1min":0.33900000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.40000000000000002,"15min":0.374},"min":{"1min":0.35099999999999998,"5min":0.26800000000000002,"15min":0.255},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.68000000000000005},"last":0.40400000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57399999999999995,"15min":0.54700000000000004},"min":{"1min":0.38500000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77400000000000002,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57999999999999996,"15min":0.55200000000000005},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.749,"5min":0.79400000000000004,"15min":0.93500000000000005},"last":0.46100000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.38100000000000001,"15min":0.35899999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":1.347},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58399999999999996,"15min":0.55900000000000005},"min":{"1min":0.42999999999999999,"5min":0.39200000000000002,"15min":0.35799999999999998},"max":{"1min":0.79800000000000004,"5min":0.91300000000000003,"15min":1.01},"last":0.45400000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61699999999999999,"15min":0.59499999999999997},"min":{"1min":0.49399999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93300000000000005,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.49199999999999999,"5min":0.42299999999999999,"15min":0.374},"max":{"1min":0.89600000000000002,"5min":0.89900000000000002,"15min":1.3799999999999999},"last":0.53400000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.629,"15min":0.59599999999999997},"min":{"1min":0.46600000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82299999999999995,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63,"15min":0.60599999999999998},"min":{"1min":0.45600000000000002,"5min":0.433,"15min":0.38200000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.96999999999999997},"last":0.48399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574994,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303912,"kb_used_data":2528,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067928,"statfs":{"total":96636764160,"available":96325558272,"internally_reserved":0,"allocated":2588672,"data_stored":1163352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52800000000000002,"15min":0.51500000000000001},"min":{"1min":0.33100000000000002,"5min":0.32300000000000001,"15min":0.28699999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.747},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53600000000000003,"15min":0.64900000000000002},"min":{"1min":0.41199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70599999999999996,"5min":0.77600000000000002,"15min":7.6719999999999997},"last":0.69299999999999995}]},{"osd":1,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.65700000000000003},"min":{"1min":0.377,"5min":0.32000000000000001,"15min":0.31},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":7.7160000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53900000000000003},"min":{"1min":0.44400000000000001,"5min":0.371,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":0.753,"15min":0.86799999999999999},"last":0.68300000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.51800000000000002},"min":{"1min":0.437,"5min":0.313,"15min":0.313},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.78000000000000003},"last":0.628},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56899999999999995,"15min":0.68000000000000005},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.379},"max":{"1min":0.67900000000000005,"5min":0.81599999999999995,"15min":7.7839999999999998},"last":0.61499999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.44,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68100000000000005,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55600000000000005,"15min":0.54800000000000004},"min":{"1min":0.435,"5min":0.39200000000000002,"15min":0.35499999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.58299999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.50800000000000001,"15min":0.498},"min":{"1min":0.33500000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64400000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.497},{"interface":"front","average":{"1min":0.499,"5min":0.54000000000000004,"15min":0.54300000000000004},"min":{"1min":0.34799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76200000000000001,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.59499999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.35499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.65800000000000003,"5min":0.85199999999999998,"15min":0.85899999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.69599999999999995,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.65500000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.39100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.67700000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.39900000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73499999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.51300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303912,"kb_used_data":2528,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067928,"statfs":{"total":96636764160,"available":96325558272,"internally_reserved":0,"allocated":2588672,"data_stored":1163352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51600000000000001,"15min":0.496},"min":{"1min":0.37,"5min":0.183,"15min":0.183},"max":{"1min":0.72599999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.36699999999999999,"5min":0.314,"15min":0.215},"max":{"1min":0.84999999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.48099999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.58399999999999996},"min":{"1min":0.33000000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":5.2679999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.215,"15min":0.215},"max":{"1min":0.76200000000000001,"5min":0.79900000000000004,"15min":5.3109999999999999},"last":0.46400000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59299999999999997,"15min":0.60899999999999999},"min":{"1min":0.36499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73999999999999999,"5min":3.27,"15min":5.2960000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.39600000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.747,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.63500000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58499999999999996,"15min":0.626},"min":{"1min":0.41399999999999998,"5min":0.39200000000000002,"15min":0.30599999999999999},"max":{"1min":0.77000000000000002,"5min":1.02,"15min":5.3330000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.629},"min":{"1min":0.45500000000000002,"5min":0.35699999999999998,"15min":0.27700000000000002},"max":{"1min":0.78100000000000003,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.66600000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.28199999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.67400000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80700000000000005,"5min":1.0429999999999999,"15min":1.0800000000000001},"last":0.68300000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53000000000000003,"15min":0.54000000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63700000000000001,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.749},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.34399999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79200000000000004,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.73399999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.371,"5min":0.35899999999999999,"15min":0.30099999999999999},"max":{"1min":0.73999999999999999,"5min":1.085,"15min":1.085},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.38400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.72199999999999998,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.77300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:46:26.894 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-10T20:34:28.804249+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=10, tm_hour=20, tm_min=34, tm_sec=28, tm_wday=1, tm_yday=283, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=10, tm_hour=20, tm_min=46, tm_sec=21, tm_wday=1, tm_yday=283, tm_isdst=0) 2023-10-10T20:46:26.894 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-10T20:46:43.656 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:46:43.660 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:46:46.895 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:46:47.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.021+0000 7fb0cdf7a700 1 Processor -- start 2023-10-10T20:46:47.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.021+0000 7fb0cdf7a700 1 -- start start 2023-10-10T20:46:47.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.022+0000 7fb0cdf7a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb0c81597f0 0x7fb0c80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.022+0000 7fb0cdf7a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c80b11d0 con 0x7fb0c815d230 2023-10-10T20:46:47.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.022+0000 7fb0cdf7a700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c80b14b0 con 0x7fb0c8159420 2023-10-10T20:46:47.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.022+0000 7fb0cdf7a700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c80b1790 con 0x7fb0c81597f0 2023-10-10T20:46:47.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.022+0000 7fb0cd779700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb0c815d230 0x7fb0c815d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41532/0 (socket says 172.21.15.89:41532) 2023-10-10T20:46:47.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.022+0000 7fb0cd779700 1 -- 172.21.15.89:0/2125897955 learned_addr learned my addr 172.21.15.89:0/2125897955 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-10T20:46:47.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.022+0000 7fb0c77fe700 1 -- 172.21.15.89:0/2125897955 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3935148275 0 0) 0x7fb0c80b11d0 con 0x7fb0c815d230 2023-10-10T20:46:47.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.022+0000 7fb0c7fff700 1 --2- 172.21.15.89:0/2125897955 >> v2:172.21.15.111:3300/0 conn(0x7fb0c81597f0 0x7fb0c80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.022+0000 7fb0c77fe700 1 -- 172.21.15.89:0/2125897955 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0b8003570 con 0x7fb0c815d230 2023-10-10T20:46:47.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c77fe700 1 -- 172.21.15.89:0/2125897955 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3729766295 0 0) 0x7fb0b8003570 con 0x7fb0c815d230 2023-10-10T20:46:47.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c77fe700 1 -- 172.21.15.89:0/2125897955 >> v2:172.21.15.111:3300/0 conn(0x7fb0c81597f0 msgr2=0x7fb0c80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:47.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c77fe700 1 --2- 172.21.15.89:0/2125897955 >> v2:172.21.15.111:3300/0 conn(0x7fb0c81597f0 0x7fb0c80b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c77fe700 1 -- 172.21.15.89:0/2125897955 >> v1:172.21.15.111:6789/0 conn(0x7fb0c8159420 legacy=0x7fb0c80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:47.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c77fe700 1 -- 172.21.15.89:0/2125897955 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c80b1a70 con 0x7fb0c815d230 2023-10-10T20:46:47.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c77fe700 1 -- 172.21.15.89:0/2125897955 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268350891 0 0) 0x7fb0c00017f0 con 0x7fb0c815d230 2023-10-10T20:46:47.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c77fe700 1 -- 172.21.15.89:0/2125897955 >> v1:172.21.15.89:6789/0 conn(0x7fb0c815d230 legacy=0x7fb0c815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:47.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c77fe700 1 --2- 172.21.15.89:0/2125897955 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0b8003a60 0x7fb0b8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c77fe700 1 --2- 172.21.15.89:0/2125897955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0b8004810 0x7fb0b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c77fe700 1 --2- 172.21.15.89:0/2125897955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0b80051c0 0x7fb0b800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c77fe700 1 -- 172.21.15.89:0/2125897955 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c80b1790 con 0x7fb0b8004810 2023-10-10T20:46:47.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c77fe700 1 -- 172.21.15.89:0/2125897955 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b800c0d0 con 0x7fb0b80051c0 2023-10-10T20:46:47.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c77fe700 1 -- 172.21.15.89:0/2125897955 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b800c290 con 0x7fb0b8003a60 2023-10-10T20:46:47.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c7fff700 1 --2- 172.21.15.89:0/2125897955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0b8004810 0x7fb0b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0ccf78700 1 --2- 172.21.15.89:0/2125897955 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0b8003a60 0x7fb0b8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0cd779700 1 --2- 172.21.15.89:0/2125897955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0b80051c0 0x7fb0b800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c7fff700 1 -- 172.21.15.89:0/2125897955 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0b8003a60 msgr2=0x7fb0b8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:47.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c7fff700 1 --2- 172.21.15.89:0/2125897955 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0b8003a60 0x7fb0b8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c7fff700 1 -- 172.21.15.89:0/2125897955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0b80051c0 msgr2=0x7fb0b800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:47.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c7fff700 1 --2- 172.21.15.89:0/2125897955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0b80051c0 0x7fb0b800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0cd779700 1 -- 172.21.15.89:0/2125897955 reap_dead start 2023-10-10T20:46:47.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c7fff700 1 -- 172.21.15.89:0/2125897955 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b8003570 con 0x7fb0b8004810 2023-10-10T20:46:47.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0ccf78700 1 --2- 172.21.15.89:0/2125897955 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0b8003a60 0x7fb0b8003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:46:47.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c7fff700 1 --2- 172.21.15.89:0/2125897955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0b8004810 0x7fb0b8004c80 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8159c70 tx=0x7fb0b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12873ec8903cab4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:47.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c77fe700 1 -- 172.21.15.89:0/2125897955 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b400e710 con 0x7fb0b8004810 2023-10-10T20:46:47.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c77fe700 1 -- 172.21.15.89:0/2125897955 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b400e870 con 0x7fb0b8004810 2023-10-10T20:46:47.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.023+0000 7fb0c77fe700 1 -- 172.21.15.89:0/2125897955 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b400ec30 con 0x7fb0b8004810 2023-10-10T20:46:47.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.024+0000 7fb0cdf7a700 1 -- 172.21.15.89:0/2125897955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0b8004810 msgr2=0x7fb0b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:47.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.024+0000 7fb0cdf7a700 1 --2- 172.21.15.89:0/2125897955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0b8004810 0x7fb0b8004c80 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8159c70 tx=0x7fb0b40091a0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.024+0000 7fb0cdf7a700 1 -- 172.21.15.89:0/2125897955 shutdown_connections 2023-10-10T20:46:47.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.024+0000 7fb0cdf7a700 1 --2- 172.21.15.89:0/2125897955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0b8004810 0x7fb0b8004c80 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.024+0000 7fb0cdf7a700 1 -- 172.21.15.89:0/2125897955 >> 172.21.15.89:0/2125897955 conn(0x7fb0c80d0660 msgr2=0x7fb0c80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:47.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.024+0000 7fb0cdf7a700 1 -- 172.21.15.89:0/2125897955 shutdown_connections 2023-10-10T20:46:47.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.024+0000 7fb0cdf7a700 1 -- 172.21.15.89:0/2125897955 wait complete. 2023-10-10T20:46:47.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.025+0000 7fb0cdf7a700 1 Processor -- start 2023-10-10T20:46:47.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.025+0000 7fb0cdf7a700 1 -- start start 2023-10-10T20:46:47.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.026+0000 7fb0cdf7a700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0b8004810 0x7fb0c80c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.026+0000 7fb0cdf7a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0c81552f0 0x7fb0c80caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.026+0000 7fb0cdf7a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c80c6420 0x7fb0c80c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.026+0000 7fb0cdf7a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c80c6e00 con 0x7fb0c81552f0 2023-10-10T20:46:47.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.026+0000 7fb0cdf7a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c80cb480 con 0x7fb0c80c6420 2023-10-10T20:46:47.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.026+0000 7fb0cdf7a700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c80cbc70 con 0x7fb0b8004810 2023-10-10T20:46:47.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.026+0000 7fb0c7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0c81552f0 0x7fb0c80caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.026+0000 7fb0c7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0c81552f0 0x7fb0c80caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52372/0 (socket says 172.21.15.89:52372) 2023-10-10T20:46:47.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.026+0000 7fb0c7fff700 1 -- 172.21.15.89:0/3814273226 learned_addr learned my addr 172.21.15.89:0/3814273226 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:47.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.026+0000 7fb0ccf78700 1 --2- 172.21.15.89:0/3814273226 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0b8004810 0x7fb0c80c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.026+0000 7fb0cd779700 1 --2- 172.21.15.89:0/3814273226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c80c6420 0x7fb0c80c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.026+0000 7fb0c7fff700 1 -- 172.21.15.89:0/3814273226 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0b8004810 msgr2=0x7fb0c80c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:47.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.026+0000 7fb0c7fff700 1 --2- 172.21.15.89:0/3814273226 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0b8004810 0x7fb0c80c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.026+0000 7fb0c7fff700 1 -- 172.21.15.89:0/3814273226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c80c6420 msgr2=0x7fb0c80c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:47.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.026+0000 7fb0c7fff700 1 --2- 172.21.15.89:0/3814273226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c80c6420 0x7fb0c80c68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.026+0000 7fb0c7fff700 1 -- 172.21.15.89:0/3814273226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c80c7b20 con 0x7fb0c81552f0 2023-10-10T20:46:47.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.027+0000 7fb0c7fff700 1 --2- 172.21.15.89:0/3814273226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0c81552f0 0x7fb0c80caf40 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400e4b0 tx=0x7fb0b401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:47.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.027+0000 7fb0c4ff9700 1 -- 172.21.15.89:0/3814273226 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4022070 con 0x7fb0c81552f0 2023-10-10T20:46:47.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.027+0000 7fb0c4ff9700 1 -- 172.21.15.89:0/3814273226 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b400e9e0 con 0x7fb0c81552f0 2023-10-10T20:46:47.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.027+0000 7fb0cdf7a700 1 -- 172.21.15.89:0/3814273226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c80cb6b0 con 0x7fb0c81552f0 2023-10-10T20:46:47.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.027+0000 7fb0c4ff9700 1 -- 172.21.15.89:0/3814273226 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b400e710 con 0x7fb0c81552f0 2023-10-10T20:46:47.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.027+0000 7fb0cdf7a700 1 -- 172.21.15.89:0/3814273226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c80c77b0 con 0x7fb0c81552f0 2023-10-10T20:46:47.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.028+0000 7fb0cdf7a700 1 -- 172.21.15.89:0/3814273226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c80c7cb0 con 0x7fb0c81552f0 2023-10-10T20:46:47.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.028+0000 7fb0c4ff9700 1 -- 172.21.15.89:0/3814273226 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb0b401c450 con 0x7fb0c81552f0 2023-10-10T20:46:47.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.029+0000 7fb0c4ff9700 1 --2- 172.21.15.89:0/3814273226 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb0ac067320 0x7fb0ac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.029+0000 7fb0ccf78700 1 --2- 172.21.15.89:0/3814273226 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb0ac067320 0x7fb0ac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.029+0000 7fb0c4ff9700 1 -- 172.21.15.89:0/3814273226 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb0b4086360 con 0x7fb0c81552f0 2023-10-10T20:46:47.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.029+0000 7fb0ccf78700 1 --2- 172.21.15.89:0/3814273226 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb0ac067320 0x7fb0ac0697d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8066e40 tx=0x7fb0bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:47.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.033+0000 7fb0c4ff9700 1 -- 172.21.15.89:0/3814273226 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0b4053430 con 0x7fb0c81552f0 2023-10-10T20:46:47.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.210+0000 7fb0cdf7a700 1 -- 172.21.15.89:0/3814273226 --> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0c804c3c0 con 0x7fb0ac067320 2023-10-10T20:46:47.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.212+0000 7fb0c4ff9700 1 -- 172.21.15.89:0/3814273226 <== mgr.4101 v2:172.21.15.89:6832/90665 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50859 (secure 0 0 0) 0x7fb0c804c3c0 con 0x7fb0ac067320 2023-10-10T20:46:47.215 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:46:47.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.214+0000 7fb0cdf7a700 1 -- 172.21.15.89:0/3814273226 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb0ac067320 msgr2=0x7fb0ac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:47.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.214+0000 7fb0cdf7a700 1 --2- 172.21.15.89:0/3814273226 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb0ac067320 0x7fb0ac0697d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8066e40 tx=0x7fb0bc009040 comp rx=0 tx=0).stop 2023-10-10T20:46:47.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.214+0000 7fb0cdf7a700 1 -- 172.21.15.89:0/3814273226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0c81552f0 msgr2=0x7fb0c80caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:47.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.214+0000 7fb0cdf7a700 1 --2- 172.21.15.89:0/3814273226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0c81552f0 0x7fb0c80caf40 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400e4b0 tx=0x7fb0b401a000 comp rx=0 tx=0).stop 2023-10-10T20:46:47.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.214+0000 7fb0cdf7a700 1 -- 172.21.15.89:0/3814273226 shutdown_connections 2023-10-10T20:46:47.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.214+0000 7fb0cdf7a700 1 --2- 172.21.15.89:0/3814273226 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7fb0ac067320 0x7fb0ac0697d0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.214+0000 7fb0cdf7a700 1 --2- 172.21.15.89:0/3814273226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c80c6420 0x7fb0c80c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.214+0000 7fb0cdf7a700 1 --2- 172.21.15.89:0/3814273226 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7fb0b8004810 0x7fb0c80c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.214+0000 7fb0cdf7a700 1 --2- 172.21.15.89:0/3814273226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0c81552f0 0x7fb0c80caf40 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.214+0000 7fb0cdf7a700 1 -- 172.21.15.89:0/3814273226 >> 172.21.15.89:0/3814273226 conn(0x7fb0c80d0660 msgr2=0x7fb0c80b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:47.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.214+0000 7fb0cdf7a700 1 -- 172.21.15.89:0/3814273226 shutdown_connections 2023-10-10T20:46:47.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.214+0000 7fb0cdf7a700 1 -- 172.21.15.89:0/3814273226 wait complete. 2023-10-10T20:46:47.220 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-10T20:46:47.228 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":566,"stamp":"2023-10-10T20:46:46.802362+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2432452,"kb_used_data":21380,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752542268,"statfs":{"total":773094113280,"available":770603282432,"internally_reserved":0,"allocated":21893120,"data_stored":10486591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002555"},"pg_stats":[{"pgid":"1.0","version":"203'119","reported_seq":359,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-10-10T20:46:26.623648+0000","last_change":"2023-10-10T20:46:26.623620+0000","last_active":"2023-10-10T20:46:26.623648+0000","last_peered":"2023-10-10T20:46:26.623648+0000","last_clean":"2023-10-10T20:46:26.623648+0000","last_became_active":"2023-10-10T20:34:30.077864+0000","last_became_peered":"2023-10-10T20:34:30.077864+0000","last_unstale":"2023-10-10T20:46:26.623648+0000","last_undegraded":"2023-10-10T20:46:26.623648+0000","last_fullsized":"2023-10-10T20:46:26.623648+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'119","last_scrub_stamp":"2023-10-10T20:46:26.623512+0000","last_deep_scrub":"203'119","last_deep_scrub_stamp":"2023-10-10T20:46:26.623512+0000","last_clean_scrub_stamp":"2023-10-10T20:46:26.623512+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:14.898550+0000","scrub_duration":0.0070932549999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574999,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304492,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067348,"statfs":{"total":96636764160,"available":96324964352,"internally_reserved":0,"allocated":3182592,"data_stored":1753720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52200000000000002,"15min":0.67000000000000004},"min":{"1min":0.35499999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71499999999999997,"5min":0.748,"15min":6.8630000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52600000000000002,"15min":0.57699999999999996},"min":{"1min":0.33900000000000002,"5min":0.32300000000000001,"15min":0.307},"max":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":4.1959999999999997},"last":0.60899999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.76100000000000001},"min":{"1min":0.315,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68899999999999995,"5min":0.78400000000000003,"15min":10.1},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54700000000000004,"15min":0.64100000000000001},"min":{"1min":0.39300000000000002,"5min":0.35199999999999998,"15min":0.28999999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":6.806},"last":0.65300000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.68600000000000005},"min":{"1min":0.45900000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.82999999999999996,"5min":0.83099999999999996,"15min":6.9340000000000002},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.55300000000000005,"15min":0.64200000000000002},"min":{"1min":0.47499999999999998,"5min":0.34899999999999998,"15min":0.30099999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":6.9870000000000001},"last":0.70999999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55200000000000005,"15min":0.65600000000000003},"min":{"1min":0.48399999999999999,"5min":0.371,"15min":0.32800000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":6.6719999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57199999999999995,"15min":0.621},"min":{"1min":0.45000000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.74399999999999999,"5min":0.77700000000000002,"15min":4.1609999999999996},"last":0.69799999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.45900000000000002,"15min":0.46400000000000002},"min":{"1min":0.29799999999999999,"5min":0.28000000000000003,"15min":0.25600000000000001},"max":{"1min":0.68200000000000005,"5min":0.69799999999999995,"15min":0.77200000000000002},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.504,"5min":0.5,"15min":0.51500000000000001},"min":{"1min":0.374,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71999999999999997,"5min":0.73899999999999999,"15min":0.80300000000000005},"last":0.50900000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50600000000000001,"15min":0.53100000000000003},"min":{"1min":0.35899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70599999999999996,"5min":0.77900000000000003,"15min":1.3180000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51200000000000001,"15min":0.52600000000000002},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.29499999999999998},"max":{"1min":0.77100000000000002,"5min":0.78200000000000003,"15min":0.84099999999999997},"last":0.52900000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":1.7509999999999999},"last":0.625},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.375,"5min":0.35499999999999998,"15min":0.32600000000000001},"max":{"1min":0.81699999999999995,"5min":0.81999999999999995,"15min":1.7090000000000001},"last":0.72099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303912,"kb_used_data":2528,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067928,"statfs":{"total":96636764160,"available":96325558272,"internally_reserved":0,"allocated":2588672,"data_stored":1163352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53700000000000003,"15min":0.59899999999999998},"min":{"1min":0.40600000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.76800000000000002,"5min":0.83099999999999996,"15min":7.7590000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.53600000000000003,"15min":0.622},"min":{"1min":0.46100000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78700000000000003,"5min":1.075,"15min":7.8090000000000002},"last":0.76300000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54600000000000004,"15min":0.623},"min":{"1min":0.442,"5min":0.29199999999999998,"15min":0.27800000000000002},"max":{"1min":0.73399999999999999,"5min":0.91300000000000003,"15min":7.867},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.72099999999999997},"min":{"1min":0.438,"5min":0.34499999999999997,"15min":0.33700000000000002},"max":{"1min":0.755,"5min":2.6179999999999999,"15min":9.7089999999999996},"last":0.78500000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.35699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71899999999999997,"5min":1.0880000000000001,"15min":4.7999999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58899999999999997,"15min":0.70999999999999996},"min":{"1min":0.50900000000000001,"5min":0.33400000000000002,"15min":0.318},"max":{"1min":0.76000000000000001,"5min":2.641,"15min":6.7629999999999999},"last":0.747}]},{"osd":3,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57099999999999995,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72199999999999998,"5min":1.107,"15min":4.8620000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56899999999999995,"15min":0.64000000000000001},"min":{"1min":0.47499999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":1.718,"15min":7.9009999999999998},"last":0.71999999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.51100000000000001,"15min":0.48999999999999999},"min":{"1min":0.32300000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66000000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.40899999999999997,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":0.745,"5min":1.032,"15min":1.032},"last":0.68799999999999994}]},{"osd":5,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.52900000000000003,"15min":0.50700000000000001},"min":{"1min":0.33300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73299999999999998,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.55900000000000005,"15min":0.54400000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33600000000000002},"max":{"1min":0.73299999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.64700000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.35799999999999998,"5min":0.33300000000000002,"15min":0.315},"max":{"1min":0.70599999999999996,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.58699999999999997,"15min":0.56399999999999995},"min":{"1min":0.376,"5min":0.36599999999999999,"15min":0.35099999999999998},"max":{"1min":0.78000000000000003,"5min":1.681,"15min":1.681},"last":0.66400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303912,"kb_used_data":2528,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067928,"statfs":{"total":96636764160,"available":96325558272,"internally_reserved":0,"allocated":2588672,"data_stored":1163352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.33000000000000002,"15min":0.40100000000000002},"min":{"1min":0.23599999999999999,"5min":0.215,"15min":0.17899999999999999},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":5.0380000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.309,"5min":0.223,"15min":0.20899999999999999},"max":{"1min":0.55500000000000005,"5min":0.65500000000000003,"15min":1.194},"last":0.34899999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.26700000000000002,"5min":0.23599999999999999,"15min":0.20599999999999999},"max":{"1min":0.49199999999999999,"5min":0.53800000000000003,"15min":1.2869999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.377,"15min":0.434},"min":{"1min":0.318,"5min":0.23999999999999999,"15min":0.22},"max":{"1min":0.57499999999999996,"5min":0.68600000000000005,"15min":5.0919999999999996},"last":0.373}]},{"osd":3,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.40100000000000002,"15min":0.45800000000000002},"min":{"1min":0.35999999999999999,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":0.54000000000000004,"5min":0.64500000000000002,"15min":5.1159999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39500000000000002},"min":{"1min":0.34899999999999998,"5min":0.23799999999999999,"15min":0.19900000000000001},"max":{"1min":0.55500000000000005,"5min":0.67000000000000004,"15min":1.2170000000000001},"last":0.40600000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.48199999999999998,"5min":0.34200000000000003,"15min":0.30199999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.254},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.57599999999999996,"15min":0.63900000000000001},"min":{"1min":0.45100000000000001,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.83899999999999997,"5min":0.88600000000000001,"15min":4.9850000000000003},"last":0.52200000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.46700000000000003,"5min":0.34799999999999998,"15min":0.34399999999999997},"max":{"1min":0.75900000000000001,"5min":0.91700000000000004,"15min":1.1799999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.58899999999999997,"15min":0.59799999999999998},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":1.3109999999999999},"last":0.53500000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.61099999999999999,"15min":0.72699999999999998},"min":{"1min":0.53200000000000003,"5min":0.378,"15min":0.35199999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":5.0659999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.623,"15min":0.626},"min":{"1min":0.44400000000000001,"5min":0.38600000000000001,"15min":0.36299999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":1.171},"last":0.64300000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.48599999999999999,"5min":0.38200000000000001,"15min":0.36499999999999999},"max":{"1min":0.86699999999999999,"5min":0.94199999999999995,"15min":1.214},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60699999999999998,"15min":0.66500000000000004},"min":{"1min":0.51600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":4.0499999999999998},"last":0.51100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574998,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304488,"kb_used_data":3104,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067352,"statfs":{"total":96636764160,"available":96324968448,"internally_reserved":0,"allocated":3178496,"data_stored":1752759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.33500000000000002,"15min":0.42699999999999999},"min":{"1min":0.308,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.47199999999999998,"5min":0.63500000000000001,"15min":7.9249999999999998},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.35499999999999998,"15min":0.42399999999999999},"min":{"1min":0.31900000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55600000000000005,"5min":0.63400000000000001,"15min":6.6299999999999999},"last":0.45000000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.36399999999999999,"15min":0.45200000000000001},"min":{"1min":0.26600000000000001,"5min":0.23100000000000001,"15min":0.216},"max":{"1min":0.63100000000000001,"5min":0.67800000000000005,"15min":8.2509999999999994},"last":0.38},{"interface":"front","average":{"1min":0.437,"5min":0.374,"15min":0.44600000000000001},"min":{"1min":0.29399999999999998,"5min":0.253,"15min":0.222},"max":{"1min":0.55900000000000005,"5min":0.65000000000000002,"15min":6.7370000000000001},"last":0.497}]},{"osd":3,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38400000000000001,"15min":0.46400000000000002},"min":{"1min":0.35899999999999999,"5min":0.22,"15min":0.21099999999999999},"max":{"1min":0.52600000000000002,"5min":0.53800000000000003,"15min":7.7960000000000003},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.42899999999999999,"15min":0.51400000000000001},"min":{"1min":0.35899999999999999,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.58199999999999996,"5min":0.71699999999999997,"15min":7.8929999999999998},"last":0.42399999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57099999999999995,"15min":0.64000000000000001},"min":{"1min":0.44400000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70499999999999996,"5min":0.94099999999999995,"15min":6.6929999999999996},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59099999999999997,"15min":0.67700000000000005},"min":{"1min":0.40000000000000002,"5min":0.371,"15min":0.33900000000000002},"max":{"1min":0.80000000000000004,"5min":0.84399999999999997,"15min":7.6920000000000002},"last":0.59499999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58499999999999996,"15min":0.66200000000000003},"min":{"1min":0.438,"5min":0.38400000000000001,"15min":0.36299999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":7.7690000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.60899999999999999,"15min":0.68899999999999995},"min":{"1min":0.49299999999999999,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.81899999999999995,"5min":0.97599999999999998,"15min":7.8659999999999997},"last":0.58199999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.624,"15min":0.71699999999999997},"min":{"1min":0.53700000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.85999999999999999,"5min":0.87,"15min":7.8360000000000003},"last":0.627},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63800000000000001,"15min":0.69499999999999995},"min":{"1min":0.53000000000000003,"5min":0.41499999999999998,"15min":0.36599999999999999},"max":{"1min":0.83099999999999996,"5min":0.875,"15min":6.6660000000000004},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.626,"15min":0.68200000000000005},"min":{"1min":0.42499999999999999,"5min":0.42099999999999999,"15min":0.36299999999999999},"max":{"1min":0.86199999999999999,"5min":0.90600000000000003,"15min":6.7160000000000002},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59899999999999998,"15min":0.69099999999999995},"min":{"1min":0.49399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.873,"5min":0.89500000000000002,"15min":7.7370000000000001},"last":0.67100000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303912,"kb_used_data":2528,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067928,"statfs":{"total":96636764160,"available":96325558272,"internally_reserved":0,"allocated":2588672,"data_stored":1163352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.33600000000000002,"15min":0.32600000000000001},"min":{"1min":0.26800000000000002,"5min":0.20200000000000001,"15min":0.189},"max":{"1min":0.52700000000000002,"5min":0.59399999999999997,"15min":0.67900000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.34999999999999998},"min":{"1min":0.25600000000000001,"5min":0.219,"15min":0.21199999999999999},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.746},"last":0.39000000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.54100000000000004},"min":{"1min":0.29099999999999998,"5min":0.221,"15min":0.21199999999999999},"max":{"1min":0.52900000000000003,"5min":0.59999999999999998,"15min":11.967000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.38200000000000001,"15min":0.42599999999999999},"min":{"1min":0.35699999999999998,"5min":0.245,"15min":0.23999999999999999},"max":{"1min":0.53900000000000003,"5min":0.65600000000000003,"15min":4.1100000000000003},"last":0.34000000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.37,"15min":0.45900000000000002},"min":{"1min":0.32100000000000001,"5min":0.23699999999999999,"15min":0.22800000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":6.3040000000000003},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.42699999999999999,"15min":0.59699999999999998},"min":{"1min":0.39700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.58099999999999996,"5min":0.70099999999999996,"15min":11.935},"last":0.437}]},{"osd":4,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59599999999999997,"15min":0.56699999999999995},"min":{"1min":0.41399999999999998,"5min":0.375,"15min":0.34999999999999998},"max":{"1min":0.84199999999999997,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76900000000000002,"5min":1.302,"15min":1.302},"last":0.77400000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.56999999999999995},"min":{"1min":0.40999999999999998,"5min":0.38900000000000001,"15min":0.378},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.95699999999999996},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59699999999999998,"15min":0.58199999999999996},"min":{"1min":0.45100000000000001,"5min":0.35999999999999999,"15min":0.34300000000000003},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.96099999999999997},"last":0.64200000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61199999999999999,"15min":0.60099999999999998},"min":{"1min":0.497,"5min":0.40300000000000002,"15min":0.33200000000000002},"max":{"1min":0.79900000000000004,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63800000000000001,"15min":0.61199999999999999},"min":{"1min":0.499,"5min":0.35199999999999998,"15min":0.34499999999999997},"max":{"1min":0.878,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.73399999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61799999999999999,"15min":0.59299999999999997},"min":{"1min":0.44800000000000001,"5min":0.41899999999999998,"15min":0.38700000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.052},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.625,"15min":0.69399999999999995},"min":{"1min":0.46200000000000002,"5min":0.40999999999999998,"15min":0.36099999999999999},"max":{"1min":0.83099999999999996,"5min":1.274,"15min":6.2649999999999997},"last":0.61699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303912,"kb_used_data":2528,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067928,"statfs":{"total":96636764160,"available":96325558272,"internally_reserved":0,"allocated":2588672,"data_stored":1163352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.32500000000000001,"15min":0.317},"min":{"1min":0.27100000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.48499999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.35299999999999998,"15min":0.33000000000000002},"min":{"1min":0.29199999999999998,"5min":0.189,"15min":0.189},"max":{"1min":0.64000000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.377}]},{"osd":1,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.37,"15min":0.34499999999999997},"min":{"1min":0.28799999999999998,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.68500000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.443},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.38900000000000001},"min":{"1min":0.309,"5min":0.26800000000000002,"15min":0.251},"max":{"1min":0.627,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.46500000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.38400000000000001,"15min":0.36299999999999999},"min":{"1min":0.33900000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.40000000000000002,"15min":0.374},"min":{"1min":0.35099999999999998,"5min":0.26800000000000002,"15min":0.255},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.68000000000000005},"last":0.42999999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57399999999999995,"15min":0.54700000000000004},"min":{"1min":0.38500000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77400000000000002,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.497},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57999999999999996,"15min":0.55200000000000005},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.749,"5min":0.79400000000000004,"15min":0.93500000000000005},"last":0.47899999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.38100000000000001,"15min":0.35899999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":1.347},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58399999999999996,"15min":0.55900000000000005},"min":{"1min":0.42999999999999999,"5min":0.39200000000000002,"15min":0.35799999999999998},"max":{"1min":0.79800000000000004,"5min":0.91300000000000003,"15min":1.01},"last":0.60699999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61699999999999999,"15min":0.59499999999999997},"min":{"1min":0.49399999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93300000000000005,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.49199999999999999,"5min":0.42299999999999999,"15min":0.374},"max":{"1min":0.89600000000000002,"5min":0.89900000000000002,"15min":1.3799999999999999},"last":0.56499999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.629,"15min":0.59599999999999997},"min":{"1min":0.46600000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82299999999999995,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63,"15min":0.60599999999999998},"min":{"1min":0.45600000000000002,"5min":0.433,"15min":0.38200000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.96999999999999997},"last":0.58099999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303912,"kb_used_data":2528,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067928,"statfs":{"total":96636764160,"available":96325558272,"internally_reserved":0,"allocated":2588672,"data_stored":1163352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52800000000000002,"15min":0.51500000000000001},"min":{"1min":0.33100000000000002,"5min":0.32300000000000001,"15min":0.28699999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.747},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53600000000000003,"15min":0.64900000000000002},"min":{"1min":0.41199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70599999999999996,"5min":0.77600000000000002,"15min":7.6719999999999997},"last":0.625}]},{"osd":1,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.65700000000000003},"min":{"1min":0.377,"5min":0.32000000000000001,"15min":0.31},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":7.7160000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53900000000000003},"min":{"1min":0.44400000000000001,"5min":0.371,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":0.753,"15min":0.86799999999999999},"last":0.72599999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.51800000000000002},"min":{"1min":0.437,"5min":0.313,"15min":0.313},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.78000000000000003},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56899999999999995,"15min":0.68000000000000005},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.379},"max":{"1min":0.67900000000000005,"5min":0.81599999999999995,"15min":7.7839999999999998},"last":0.67600000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.44,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68100000000000005,"5min":0.81699999999999995,"15min":0.96999999999999997},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55600000000000005,"15min":0.54800000000000004},"min":{"1min":0.435,"5min":0.39200000000000002,"15min":0.35499999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56799999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.50800000000000001,"15min":0.498},"min":{"1min":0.33500000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64400000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.54000000000000004,"15min":0.54300000000000004},"min":{"1min":0.34799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76200000000000001,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.41899999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.35499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.65800000000000003,"5min":0.85199999999999998,"15min":0.85899999999999999},"last":0.438},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.69599999999999995,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.35199999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.39100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.67700000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.39900000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73499999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.495}]}]},{"osd":5,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303912,"kb_used_data":2528,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067928,"statfs":{"total":96636764160,"available":96325558272,"internally_reserved":0,"allocated":2588672,"data_stored":1163352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51600000000000001,"15min":0.496},"min":{"1min":0.37,"5min":0.183,"15min":0.183},"max":{"1min":0.72599999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.36699999999999999,"5min":0.314,"15min":0.215},"max":{"1min":0.84999999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.499}]},{"osd":1,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.58399999999999996},"min":{"1min":0.33000000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":5.2679999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.215,"15min":0.215},"max":{"1min":0.76200000000000001,"5min":0.79900000000000004,"15min":5.3109999999999999},"last":0.38300000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59299999999999997,"15min":0.60899999999999999},"min":{"1min":0.36499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73999999999999999,"5min":3.27,"15min":5.2960000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.39600000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.747,"5min":0.76400000000000001,"15min":1.0029999999999999},"last":0.61099999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58499999999999996,"15min":0.626},"min":{"1min":0.41399999999999998,"5min":0.39200000000000002,"15min":0.30599999999999999},"max":{"1min":0.77000000000000002,"5min":1.02,"15min":5.3330000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.629},"min":{"1min":0.45500000000000002,"5min":0.35699999999999998,"15min":0.27700000000000002},"max":{"1min":0.78100000000000003,"5min":0.81499999999999995,"15min":5.2850000000000001},"last":0.51800000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.28199999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.67400000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80700000000000005,"5min":1.0429999999999999,"15min":1.0800000000000001},"last":0.54600000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53000000000000003,"15min":0.54000000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63700000000000001,"5min":0.89000000000000001,"15min":1.0660000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.34399999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79200000000000004,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.48699999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.371,"5min":0.35899999999999999,"15min":0.30099999999999999},"max":{"1min":0.73999999999999999,"5min":1.085,"15min":1.085},"last":0.62},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.38400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.72199999999999998,"5min":0.92600000000000005,"15min":1.3779999999999999},"last":0.64000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-10T20:46:47.231 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-10T20:46:47.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.345+0000 7f5aaafed700 1 Processor -- start 2023-10-10T20:46:47.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.345+0000 7f5aaafed700 1 -- start start 2023-10-10T20:46:47.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.345+0000 7f5aaafed700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5aa4159890 0x7f5aa415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.345+0000 7f5aaafed700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5aa4159c60 0x7f5aa415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.345+0000 7f5aaafed700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5aa415db10 0x7f5aa4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.345+0000 7f5aaafed700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa415a160 con 0x7f5aa4159c60 2023-10-10T20:46:47.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.345+0000 7f5aaafed700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa40d13a0 con 0x7f5aa415db10 2023-10-10T20:46:47.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.345+0000 7f5aaafed700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa415e140 con 0x7f5aa4159890 2023-10-10T20:46:47.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.346+0000 7f5a9bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5aa4159c60 0x7f5aa415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.346+0000 7f5a9bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5aa4159c60 0x7f5aa415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52382/0 (socket says 172.21.15.89:52382) 2023-10-10T20:46:47.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.346+0000 7f5a9bfff700 1 -- 172.21.15.89:0/363521 learned_addr learned my addr 172.21.15.89:0/363521 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:47.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.346+0000 7f5aa958a700 1 --2- 172.21.15.89:0/363521 >> v2:172.21.15.111:3300/0 conn(0x7f5aa415db10 0x7f5aa4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.346+0000 7f5aa8d89700 1 --2- 172.21.15.89:0/363521 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5aa4159890 0x7f5aa415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.346+0000 7f5a9bfff700 1 -- 172.21.15.89:0/363521 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5aa4159890 msgr2=0x7f5aa415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:47.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.346+0000 7f5a9bfff700 1 --2- 172.21.15.89:0/363521 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5aa4159890 0x7f5aa415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.346+0000 7f5a9bfff700 1 -- 172.21.15.89:0/363521 >> v2:172.21.15.111:3300/0 conn(0x7f5aa415db10 msgr2=0x7f5aa4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:47.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.346+0000 7f5a9bfff700 1 --2- 172.21.15.89:0/363521 >> v2:172.21.15.111:3300/0 conn(0x7f5aa415db10 0x7f5aa4069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.346+0000 7f5a9bfff700 1 -- 172.21.15.89:0/363521 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa415a3d0 con 0x7f5aa4159c60 2023-10-10T20:46:47.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.346+0000 7f5a9bfff700 1 --2- 172.21.15.89:0/363521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa4159c60 0x7f5aa415d5d0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4089b50 tx=0x7f5a94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3651f051a1a9e713 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:47.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.346+0000 7f5a9b7fe700 1 -- 172.21.15.89:0/363521 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9400b860 con 0x7f5aa4159c60 2023-10-10T20:46:47.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.346+0000 7f5a9b7fe700 1 -- 172.21.15.89:0/363521 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9400b9c0 con 0x7f5aa4159c60 2023-10-10T20:46:47.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.346+0000 7f5a9b7fe700 1 -- 172.21.15.89:0/363521 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9400bd60 con 0x7f5aa4159c60 2023-10-10T20:46:47.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.347+0000 7f5aaafed700 1 -- 172.21.15.89:0/363521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa4159c60 msgr2=0x7f5aa415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:47.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.347+0000 7f5aaafed700 1 --2- 172.21.15.89:0/363521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa4159c60 0x7f5aa415d5d0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4089b50 tx=0x7f5a94009250 comp rx=0 tx=0).stop 2023-10-10T20:46:47.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.347+0000 7f5aaafed700 1 -- 172.21.15.89:0/363521 shutdown_connections 2023-10-10T20:46:47.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.347+0000 7f5aaafed700 1 --2- 172.21.15.89:0/363521 >> v2:172.21.15.111:3300/0 conn(0x7f5aa415db10 0x7f5aa4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.347+0000 7f5aaafed700 1 --2- 172.21.15.89:0/363521 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5aa4159890 0x7f5aa415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.347+0000 7f5aaafed700 1 --2- 172.21.15.89:0/363521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa4159c60 0x7f5aa415d5d0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.347+0000 7f5aaafed700 1 -- 172.21.15.89:0/363521 >> 172.21.15.89:0/363521 conn(0x7f5aa4068e20 msgr2=0x7f5aa40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:47.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.347+0000 7f5aaafed700 1 -- 172.21.15.89:0/363521 shutdown_connections 2023-10-10T20:46:47.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.347+0000 7f5aaafed700 1 -- 172.21.15.89:0/363521 wait complete. 2023-10-10T20:46:47.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.348+0000 7f5aaafed700 1 Processor -- start 2023-10-10T20:46:47.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.348+0000 7f5aaafed700 1 -- start start 2023-10-10T20:46:47.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.349+0000 7f5aaafed700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5aa4159890 0x7f5aa41525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.349+0000 7f5aaafed700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5aa4159c60 0x7f5aa4155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.349+0000 7f5aaafed700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa415db10 0x7f5aa40b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.349+0000 7f5aaafed700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa4152ef0 con 0x7f5aa415db10 2023-10-10T20:46:47.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.349+0000 7f5aaafed700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa41531d0 con 0x7f5aa4159c60 2023-10-10T20:46:47.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.349+0000 7f5aaafed700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa41534b0 con 0x7f5aa4159890 2023-10-10T20:46:47.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.349+0000 7f5aa958a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa415db10 0x7f5aa40b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.349+0000 7f5aa958a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa415db10 0x7f5aa40b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52386/0 (socket says 172.21.15.89:52386) 2023-10-10T20:46:47.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.349+0000 7f5aa958a700 1 -- 172.21.15.89:0/3964848464 learned_addr learned my addr 172.21.15.89:0/3964848464 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:47.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.349+0000 7f5aa8d89700 1 --2- 172.21.15.89:0/3964848464 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5aa4159890 0x7f5aa41525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.349+0000 7f5a9bfff700 1 --2- 172.21.15.89:0/3964848464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5aa4159c60 0x7f5aa4155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.349+0000 7f5aa958a700 1 -- 172.21.15.89:0/3964848464 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5aa4159890 msgr2=0x7f5aa41525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:47.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.349+0000 7f5aa958a700 1 --2- 172.21.15.89:0/3964848464 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5aa4159890 0x7f5aa41525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.349+0000 7f5aa958a700 1 -- 172.21.15.89:0/3964848464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5aa4159c60 msgr2=0x7f5aa4155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:47.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.349+0000 7f5aa958a700 1 --2- 172.21.15.89:0/3964848464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5aa4159c60 0x7f5aa4155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.349+0000 7f5aa958a700 1 -- 172.21.15.89:0/3964848464 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a940094f0 con 0x7f5aa415db10 2023-10-10T20:46:47.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.349+0000 7f5aa958a700 1 --2- 172.21.15.89:0/3964848464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa415db10 0x7f5aa40b1590 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00cdb0 tx=0x7f5a9c0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:47.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.349+0000 7f5a98ff9700 1 -- 172.21.15.89:0/3964848464 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c00a9d0 con 0x7f5aa415db10 2023-10-10T20:46:47.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.350+0000 7f5a98ff9700 1 -- 172.21.15.89:0/3964848464 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9c00ab30 con 0x7f5aa415db10 2023-10-10T20:46:47.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.350+0000 7f5a98ff9700 1 -- 172.21.15.89:0/3964848464 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c017430 con 0x7f5aa415db10 2023-10-10T20:46:47.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.350+0000 7f5aaafed700 1 -- 172.21.15.89:0/3964848464 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa40cd0c0 con 0x7f5aa415db10 2023-10-10T20:46:47.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.350+0000 7f5aaafed700 1 -- 172.21.15.89:0/3964848464 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa4152b30 con 0x7f5aa415db10 2023-10-10T20:46:47.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.350+0000 7f5aaafed700 1 -- 172.21.15.89:0/3964848464 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa404c3c0 con 0x7f5aa415db10 2023-10-10T20:46:47.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.351+0000 7f5a98ff9700 1 -- 172.21.15.89:0/3964848464 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5a9c003020 con 0x7f5aa415db10 2023-10-10T20:46:47.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.351+0000 7f5a98ff9700 1 --2- 172.21.15.89:0/3964848464 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a90067320 0x7f5a900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.351+0000 7f5a98ff9700 1 -- 172.21.15.89:0/3964848464 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5a9c013070 con 0x7f5aa415db10 2023-10-10T20:46:47.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.351+0000 7f5aa8d89700 1 --2- 172.21.15.89:0/3964848464 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a90067320 0x7f5a900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.352+0000 7f5aa8d89700 1 --2- 172.21.15.89:0/3964848464 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a90067320 0x7f5a900697d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000acb0 tx=0x7f5aa0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:47.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.355+0000 7f5a98ff9700 1 -- 172.21.15.89:0/3964848464 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a9c051d10 con 0x7f5aa415db10 2023-10-10T20:46:47.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.540+0000 7f5aaafed700 1 -- 172.21.15.89:0/3964848464 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f5aa40d09c0 con 0x7f5aa415db10 2023-10-10T20:46:47.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.546+0000 7f5a98ff9700 1 -- 172.21.15.89:0/3964848464 <== mon.0 v2:172.21.15.89:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5a9c051d10 con 0x7f5aa415db10 2023-10-10T20:46:47.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.546+0000 7f5a98ff9700 1 -- 172.21.15.89:0/3964848464 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v34)=0 v34) v1 ==== 123+0+0 (secure 0 0 0) 0x7f5a9c0078f0 con 0x7f5aa415db10 2023-10-10T20:46:47.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.548+0000 7f5aaafed700 1 -- 172.21.15.89:0/3964848464 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a90067320 msgr2=0x7f5a900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:47.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.548+0000 7f5aaafed700 1 --2- 172.21.15.89:0/3964848464 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a90067320 0x7f5a900697d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000acb0 tx=0x7f5aa0008040 comp rx=0 tx=0).stop 2023-10-10T20:46:47.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.548+0000 7f5aaafed700 1 -- 172.21.15.89:0/3964848464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa415db10 msgr2=0x7f5aa40b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:47.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.548+0000 7f5aaafed700 1 --2- 172.21.15.89:0/3964848464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa415db10 0x7f5aa40b1590 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00cdb0 tx=0x7f5a9c0077d0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.548+0000 7f5aaafed700 1 -- 172.21.15.89:0/3964848464 shutdown_connections 2023-10-10T20:46:47.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.548+0000 7f5aaafed700 1 --2- 172.21.15.89:0/3964848464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa415db10 0x7f5aa40b1590 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.548+0000 7f5aaafed700 1 --2- 172.21.15.89:0/3964848464 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7f5aa4159890 0x7f5aa41525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.548+0000 7f5aaafed700 1 --2- 172.21.15.89:0/3964848464 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7f5a90067320 0x7f5a900697d0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.548+0000 7f5aaafed700 1 --2- 172.21.15.89:0/3964848464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5aa4159c60 0x7f5aa4155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.548+0000 7f5aaafed700 1 -- 172.21.15.89:0/3964848464 >> 172.21.15.89:0/3964848464 conn(0x7f5aa4068e20 msgr2=0x7f5aa4153850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:47.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.549+0000 7f5aaafed700 1 -- 172.21.15.89:0/3964848464 shutdown_connections 2023-10-10T20:46:47.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.549+0000 7f5aaafed700 1 -- 172.21.15.89:0/3964848464 wait complete. 2023-10-10T20:46:47.563 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:46:47.563 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:46:47.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.675+0000 7ff678f16700 1 Processor -- start 2023-10-10T20:46:47.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.675+0000 7ff678f16700 1 -- start start 2023-10-10T20:46:47.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.676+0000 7ff678f16700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff674159420 0x7ff6740b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.676+0000 7ff678f16700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff67415d230 0x7ff67415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.676+0000 7ff678f16700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6740b2980 con 0x7ff6741597f0 2023-10-10T20:46:47.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.676+0000 7ff678f16700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67415dbb0 con 0x7ff674159420 2023-10-10T20:46:47.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.676+0000 7ff678f16700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6740b11d0 con 0x7ff67415d230 2023-10-10T20:46:47.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.677+0000 7ff673fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff67415d230 0x7ff67415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.677+0000 7ff6737fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff674159420 0x7ff6740b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.677+0000 7ff6737fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff674159420 0x7ff6740b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:33296/0 (socket says 172.21.15.89:33296) 2023-10-10T20:46:47.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.677+0000 7ff673fff700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff67415d230 0x7ff67415d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:45122/0 (socket says 172.21.15.89:45122) 2023-10-10T20:46:47.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.677+0000 7ff6737fe700 1 -- 172.21.15.89:0/3533933781 learned_addr learned my addr 172.21.15.89:0/3533933781 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:47.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.677+0000 7ff672ffd700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff6741597f0 0x7ff6740b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.89:37274/0 (socket says 172.21.15.89:37274) 2023-10-10T20:46:47.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.677+0000 7ff673fff700 1 -- 172.21.15.89:0/3533933781 >> v2:172.21.15.111:3300/0 conn(0x7ff674159420 msgr2=0x7ff6740b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:46:47.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.677+0000 7ff673fff700 1 --2- 172.21.15.89:0/3533933781 >> v2:172.21.15.111:3300/0 conn(0x7ff674159420 0x7ff6740b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.677+0000 7ff673fff700 1 -- 172.21.15.89:0/3533933781 >> v1:172.21.15.111:6789/0 conn(0x7ff6741597f0 legacy=0x7ff6740b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:47.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.677+0000 7ff673fff700 1 -- 172.21.15.89:0/3533933781 --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6740b14b0 con 0x7ff67415d230 2023-10-10T20:46:47.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.677+0000 7ff6737fe700 1 --2- 172.21.15.89:0/3533933781 >> v2:172.21.15.111:3300/0 conn(0x7ff674159420 0x7ff6740b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:46:47.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.677+0000 7ff673fff700 1 --2- 172.21.15.89:0/3533933781 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff67415d230 0x7ff67415d670 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff66c009e60 tx=0x7ff66c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c81382dfd19f90d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:47.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.677+0000 7ff6727fc700 1 -- 172.21.15.89:0/3533933781 <== mon.2 v2:172.21.15.111:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c00e070 con 0x7ff67415d230 2023-10-10T20:46:47.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.677+0000 7ff6727fc700 1 -- 172.21.15.89:0/3533933781 <== mon.2 v2:172.21.15.111:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff66c00bb60 con 0x7ff67415d230 2023-10-10T20:46:47.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.677+0000 7ff6727fc700 1 -- 172.21.15.89:0/3533933781 <== mon.2 v2:172.21.15.111:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c012430 con 0x7ff67415d230 2023-10-10T20:46:47.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.678+0000 7ff678f16700 1 -- 172.21.15.89:0/3533933781 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff67415d230 msgr2=0x7ff67415d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:47.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.678+0000 7ff678f16700 1 --2- 172.21.15.89:0/3533933781 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff67415d230 0x7ff67415d670 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff66c009e60 tx=0x7ff66c009780 comp rx=0 tx=0).stop 2023-10-10T20:46:47.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.678+0000 7ff678f16700 1 -- 172.21.15.89:0/3533933781 shutdown_connections 2023-10-10T20:46:47.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.678+0000 7ff678f16700 1 --2- 172.21.15.89:0/3533933781 >> v2:172.21.15.111:3300/0 conn(0x7ff674159420 0x7ff6740b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.678+0000 7ff678f16700 1 --2- 172.21.15.89:0/3533933781 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff67415d230 0x7ff67415d670 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.678+0000 7ff678f16700 1 -- 172.21.15.89:0/3533933781 >> 172.21.15.89:0/3533933781 conn(0x7ff6740d0660 msgr2=0x7ff6740d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:47.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.678+0000 7ff678f16700 1 -- 172.21.15.89:0/3533933781 shutdown_connections 2023-10-10T20:46:47.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.678+0000 7ff678f16700 1 -- 172.21.15.89:0/3533933781 wait complete. 2023-10-10T20:46:47.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.679+0000 7ff678f16700 1 Processor -- start 2023-10-10T20:46:47.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.679+0000 7ff678f16700 1 -- start start 2023-10-10T20:46:47.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff678f16700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff674159420 0x7ff674155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff678f16700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6741597f0 0x7ff67415bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff678f16700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67415d230 0x7ff6740c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff678f16700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67415c5e0 con 0x7ff67415d230 2023-10-10T20:46:47.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff678f16700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6740c5da0 con 0x7ff6741597f0 2023-10-10T20:46:47.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff678f16700 1 -- --> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6740c6080 con 0x7ff674159420 2023-10-10T20:46:47.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff673fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67415d230 0x7ff6740c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff6737fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff674159420 0x7ff674155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff6737fe700 1 --2- >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff674159420 0x7ff674155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3301/0 says I am v2:172.21.15.89:45134/0 (socket says 172.21.15.89:45134) 2023-10-10T20:46:47.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff6737fe700 1 -- 172.21.15.89:0/4121407115 learned_addr learned my addr 172.21.15.89:0/4121407115 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-10T20:46:47.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff673fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67415d230 0x7ff6740c6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52388/0 (socket says 172.21.15.89:52388) 2023-10-10T20:46:47.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff672ffd700 1 --2- 172.21.15.89:0/4121407115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6741597f0 0x7ff67415bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff673fff700 1 -- 172.21.15.89:0/4121407115 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff674159420 msgr2=0x7ff674155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:47.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff673fff700 1 --2- 172.21.15.89:0/4121407115 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff674159420 0x7ff674155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff673fff700 1 -- 172.21.15.89:0/4121407115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6741597f0 msgr2=0x7ff67415bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:47.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff673fff700 1 --2- 172.21.15.89:0/4121407115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6741597f0 0x7ff67415bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff673fff700 1 -- 172.21.15.89:0/4121407115 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6640094f0 con 0x7ff67415d230 2023-10-10T20:46:47.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff672ffd700 1 --2- 172.21.15.89:0/4121407115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6741597f0 0x7ff67415bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:46:47.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff6737fe700 1 --2- 172.21.15.89:0/4121407115 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff674159420 0x7ff674155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:46:47.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.680+0000 7ff673fff700 1 --2- 172.21.15.89:0/4121407115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67415d230 0x7ff6740c6b60 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7ff66c016000 tx=0x7ff66c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:47.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.681+0000 7ff65b7fe700 1 -- 172.21.15.89:0/4121407115 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c027470 con 0x7ff67415d230 2023-10-10T20:46:47.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.681+0000 7ff65b7fe700 1 -- 172.21.15.89:0/4121407115 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff66c0275d0 con 0x7ff67415d230 2023-10-10T20:46:47.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.681+0000 7ff65b7fe700 1 -- 172.21.15.89:0/4121407115 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c0279b0 con 0x7ff67415d230 2023-10-10T20:46:47.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.681+0000 7ff678f16700 1 -- 172.21.15.89:0/4121407115 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66c009a20 con 0x7ff67415d230 2023-10-10T20:46:47.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.681+0000 7ff678f16700 1 -- 172.21.15.89:0/4121407115 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff674155780 con 0x7ff67415d230 2023-10-10T20:46:47.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.682+0000 7ff678f16700 1 -- 172.21.15.89:0/4121407115 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67404c3c0 con 0x7ff67415d230 2023-10-10T20:46:47.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.682+0000 7ff65b7fe700 1 -- 172.21.15.89:0/4121407115 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff66c024070 con 0x7ff67415d230 2023-10-10T20:46:47.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.682+0000 7ff65b7fe700 1 --2- 172.21.15.89:0/4121407115 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff660067320 0x7ff6600697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:46:47.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.682+0000 7ff65b7fe700 1 -- 172.21.15.89:0/4121407115 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff66c0871b0 con 0x7ff67415d230 2023-10-10T20:46:47.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.682+0000 7ff6737fe700 1 --2- 172.21.15.89:0/4121407115 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff660067320 0x7ff6600697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:46:47.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.683+0000 7ff6737fe700 1 --2- 172.21.15.89:0/4121407115 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff660067320 0x7ff6600697d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff664009fd0 tx=0x7ff664009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:46:47.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.687+0000 7ff65b7fe700 1 -- 172.21.15.89:0/4121407115 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff66c0541c0 con 0x7ff67415d230 2023-10-10T20:46:47.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.913+0000 7ff678f16700 1 -- 172.21.15.89:0/4121407115 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6740b1790 con 0x7ff67415d230 2023-10-10T20:46:47.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.914+0000 7ff65b7fe700 1 -- 172.21.15.89:0/4121407115 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) v1 ==== 95+0+3531 (secure 0 0 0) 0x7ff66c02cde0 con 0x7ff67415d230 2023-10-10T20:46:47.916 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T20:46:47.917 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":135,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/4215135154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":4215135154},{"type":"v1","addr":"172.21.15.89:6837","nonce":4215135154}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127},{"gid":7088,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/940990808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":940990808},{"type":"v1","addr":"172.21.15.111:6834","nonce":940990808}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127},{"gid":7091,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/92922277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":92922277},{"type":"v1","addr":"172.21.15.111:6835","nonce":92922277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127},{"gid":9402,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/1611582122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":1611582122},{"type":"v1","addr":"172.21.15.89:6836","nonce":1611582122}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":135}],"filesystems":[]} 2023-10-10T20:46:47.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.916+0000 7ff678f16700 1 -- 172.21.15.89:0/4121407115 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff660067320 msgr2=0x7ff6600697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:47.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.916+0000 7ff678f16700 1 --2- 172.21.15.89:0/4121407115 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff660067320 0x7ff6600697d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff664009fd0 tx=0x7ff664009250 comp rx=0 tx=0).stop 2023-10-10T20:46:47.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.916+0000 7ff678f16700 1 -- 172.21.15.89:0/4121407115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67415d230 msgr2=0x7ff6740c6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:46:47.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.916+0000 7ff678f16700 1 --2- 172.21.15.89:0/4121407115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67415d230 0x7ff6740c6b60 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7ff66c016000 tx=0x7ff66c009780 comp rx=0 tx=0).stop 2023-10-10T20:46:47.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.916+0000 7ff678f16700 1 -- 172.21.15.89:0/4121407115 shutdown_connections 2023-10-10T20:46:47.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.916+0000 7ff678f16700 1 --2- 172.21.15.89:0/4121407115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67415d230 0x7ff6740c6b60 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.916+0000 7ff678f16700 1 --2- 172.21.15.89:0/4121407115 >> [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] conn(0x7ff674159420 0x7ff674155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.916+0000 7ff678f16700 1 --2- 172.21.15.89:0/4121407115 >> [v2:172.21.15.89:6832/90665,v1:172.21.15.89:6833/90665] conn(0x7ff660067320 0x7ff6600697d0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.916+0000 7ff678f16700 1 --2- 172.21.15.89:0/4121407115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6741597f0 0x7ff67415bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:46:47.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.916+0000 7ff678f16700 1 -- 172.21.15.89:0/4121407115 >> 172.21.15.89:0/4121407115 conn(0x7ff6740d0660 msgr2=0x7ff674156a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:46:47.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.916+0000 7ff678f16700 1 -- 172.21.15.89:0/4121407115 shutdown_connections 2023-10-10T20:46:47.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-10T20:46:47.916+0000 7ff678f16700 1 -- 172.21.15.89:0/4121407115 wait complete. 2023-10-10T20:46:47.921 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 135 2023-10-10T20:46:47.930 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-10T20:46:47.930 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-10T20:46:47.931 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:46:47.931 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:46:47.939 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:46:47.940 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:46:47.940 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:46:47.988 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:46:47.989 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:46:47.989 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:46:47.999 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:46:48.000 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:46:48.000 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:46:48.050 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:46:48.051 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-10T20:46:48.051 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-10T20:46:48.051 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:46:48.088 INFO:tasks.ceph.osd.0:Stopped 2023-10-10T20:46:48.089 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-10T20:46:48.089 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:46:48.161 INFO:tasks.ceph.osd.1:Stopped 2023-10-10T20:46:48.162 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-10T20:46:48.162 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:46:48.230 INFO:tasks.ceph.osd.2:Stopped 2023-10-10T20:46:48.230 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-10T20:46:48.231 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:46:48.312 INFO:tasks.ceph.osd.3:Stopped 2023-10-10T20:46:48.313 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-10T20:46:48.313 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:46:48.342 INFO:tasks.ceph.osd.4:Stopped 2023-10-10T20:46:48.343 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-10T20:46:48.343 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:46:48.411 INFO:tasks.ceph.osd.5:Stopped 2023-10-10T20:46:48.411 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-10T20:46:48.412 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:46:48.476 INFO:tasks.ceph.osd.6:Stopped 2023-10-10T20:46:48.477 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-10T20:46:48.477 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:46:48.558 INFO:tasks.ceph.osd.7:Stopped 2023-10-10T20:46:48.558 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-10T20:46:48.559 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-10T20:46:48.559 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:46:48.586 INFO:tasks.ceph.mgr.y:Stopped 2023-10-10T20:46:48.587 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-10T20:46:48.587 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:46:48.613 INFO:tasks.ceph.mgr.x:Stopped 2023-10-10T20:46:48.614 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-10T20:46:48.615 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-10T20:46:48.615 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:46:48.637 INFO:tasks.ceph.mon.a:Stopped 2023-10-10T20:46:48.638 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-10T20:46:48.638 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:46:48.667 INFO:tasks.ceph.mon.b:Stopped 2023-10-10T20:46:48.667 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-10T20:46:48.668 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:46:48.719 INFO:tasks.ceph.mon.c:Stopped 2023-10-10T20:46:48.719 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-10T20:46:48.720 DEBUG:teuthology.orchestra.run.smithi089:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-10-10T20:46:48.749 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:46:48.750 DEBUG:teuthology.orchestra.run.smithi089:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-10T20:46:49.030 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:46:49.030 DEBUG:teuthology.orchestra.run.smithi089:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-10T20:46:49.108 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:46:49.109 DEBUG:teuthology.orchestra.run.smithi089:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-10T20:46:49.192 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:46:49.193 DEBUG:teuthology.orchestra.run.smithi089:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-10T20:46:49.295 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:46:49.296 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-10T20:46:49.515 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:46:49.515 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-10T20:46:49.612 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:46:49.612 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-10T20:46:49.679 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:46:49.679 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-10T20:46:49.787 INFO:tasks.ceph:Archiving mon data... 2023-10-10T20:46:49.788 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420483/data/mon.a.tgz 2023-10-10T20:46:49.789 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2023-10-10T20:46:49.804 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.JEmlN9HRSt 2023-10-10T20:46:49.804 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.JEmlN9HRSt 2023-10-10T20:46:50.005 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.JEmlN9HRSt 2023-10-10T20:46:50.088 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.JEmlN9HRSt is 596KB 2023-10-10T20:46:50.159 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.JEmlN9HRSt 2023-10-10T20:46:50.174 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420483/data/mon.b.tgz 2023-10-10T20:46:50.174 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2023-10-10T20:46:50.191 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.hGI869Sfik 2023-10-10T20:46:50.192 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.hGI869Sfik 2023-10-10T20:46:50.432 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.hGI869Sfik 2023-10-10T20:46:50.514 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.hGI869Sfik is 605KB 2023-10-10T20:46:50.613 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.hGI869Sfik 2023-10-10T20:46:50.628 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420483/data/mon.c.tgz 2023-10-10T20:46:50.628 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2023-10-10T20:46:50.682 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.4n0HniJwf9 2023-10-10T20:46:50.682 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.4n0HniJwf9 2023-10-10T20:46:50.850 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.4n0HniJwf9 2023-10-10T20:46:50.931 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.4n0HniJwf9 is 605KB 2023-10-10T20:46:51.029 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.4n0HniJwf9 2023-10-10T20:46:51.043 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-10T20:46:51.044 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-10T20:46:51.046 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-10T20:46:51.108 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-10T20:46:51.109 DEBUG:teuthology.orchestra.run.smithi089:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-10T20:46:51.112 DEBUG:teuthology.orchestra.run.smithi111:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-10T20:46:51.136 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-10T20:46:51.173 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-10T20:46:51.179 INFO:tasks.ceph:Archiving crash dumps... 2023-10-10T20:46:51.181 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420483/remote/smithi089/crash 2023-10-10T20:46:51.183 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-10T20:46:51.207 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420483/remote/smithi111/crash 2023-10-10T20:46:51.208 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-10T20:46:51.243 INFO:tasks.ceph:Shutting down logrotate 2023-10-10T20:46:51.244 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:46:51.252 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:46:51.378 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-10T20:46:51.402 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-10T20:46:51.426 INFO:tasks.ceph:Compressing logs... 2023-10-10T20:46:51.427 DEBUG:teuthology.orchestra.run.smithi089:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-10T20:46:51.444 DEBUG:teuthology.orchestra.run.smithi111:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-10T20:46:51.522 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90869.log 2023-10-10T20:46:51.522 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.101605.log 2023-10-10T20:46:51.523 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.90869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.98032.log 2023-10-10T20:46:51.523 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.101605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.97654.log 2023-10-10T20:46:51.523 INFO:teuthology.orchestra.run.smithi111.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.90869.log.gz 2023-10-10T20:46:51.524 INFO:teuthology.orchestra.run.smithi111.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.101605.log.gz 2023-10-10T20:46:51.524 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.3.98032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.102771.log 2023-10-10T20:46:51.524 INFO:teuthology.orchestra.run.smithi111.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.98032.log.gz 2023-10-10T20:46:51.524 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.97654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.98801.log 2023-10-10T20:46:51.525 INFO:teuthology.orchestra.run.smithi111.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.97654.log.gz 2023-10-10T20:46:51.525 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.102771.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-10T20:46:51.525 INFO:teuthology.orchestra.run.smithi111.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.102771.log.gz 2023-10-10T20:46:51.526 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.98801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90915.log 2023-10-10T20:46:51.526 INFO:teuthology.orchestra.run.smithi111.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.98801.log.gz 2023-10-10T20:46:51.526 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.101609.log 2023-10-10T20:46:51.526 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.90915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90703.log 2023-10-10T20:46:51.527 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.101609.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90915.log.gz 2023-10-10T20:46:51.527 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.100838.log 2023-10-10T20:46:51.527 INFO:teuthology.orchestra.run.smithi111.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.101609.log.gz 2023-10-10T20:46:51.527 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.90703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.100450.log 2023-10-10T20:46:51.528 INFO:teuthology.orchestra.run.smithi111.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.90703.log.gz 2023-10-10T20:46:51.528 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.100838.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-10T20:46:51.528 INFO:teuthology.orchestra.run.smithi111.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.100838.log.gz 2023-10-10T20:46:51.529 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.100450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.102775.log 2023-10-10T20:46:51.529 INFO:teuthology.orchestra.run.smithi111.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.100450.log.gz 2023-10-10T20:46:51.529 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.98028.log 2023-10-10T20:46:51.529 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.102775.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-10T20:46:51.530 INFO:teuthology.orchestra.run.smithi111.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.102775.log.gz 2023-10-10T20:46:51.530 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.3.98028.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.98028.log.gz 2023-10-10T20:46:51.530 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.99667.log 2023-10-10T20:46:51.531 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.100454.log 2023-10-10T20:46:51.531 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.99667.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.99667.log.gz 2023-10-10T20:46:51.531 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-10T20:46:51.532 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.97662.log 2023-10-10T20:46:51.532 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.100454.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.100454.log.gz 2023-10-10T20:46:51.532 INFO:teuthology.orchestra.run.smithi111.stderr: 92.1% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-10T20:46:51.532 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-10T20:46:51.533 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.97662.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.97662.log.gz 2023-10-10T20:46:51.533 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-10T20:46:51.533 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90999.log 2023-10-10T20:46:51.534 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.98797.log 2023-10-10T20:46:51.534 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.90999.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.90999.log.gz 2023-10-10T20:46:51.534 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.98797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.98797.log.gz 2023-10-10T20:46:51.534 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90956.log 2023-10-10T20:46:51.535 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-10T20:46:51.535 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.90956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.99663.log 2023-10-10T20:46:51.535 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-mon.c.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90956.log.gz 2023-10-10T20:46:51.536 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90784.log 2023-10-10T20:46:51.536 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.99663.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.99663.log.gz 2023-10-10T20:46:51.536 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.90784.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-10T20:46:51.536 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.100068.log 2023-10-10T20:46:51.537 INFO:teuthology.orchestra.run.smithi111.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90784.log.gz 2023-10-10T20:46:51.537 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.100072.log 2023-10-10T20:46:51.537 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.100068.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-10T20:46:51.537 INFO:teuthology.orchestra.run.smithi111.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.100068.log.gz 2023-10-10T20:46:51.538 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.100072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.102384.log 2023-10-10T20:46:51.538 INFO:teuthology.orchestra.run.smithi111.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.100072.log.gz 2023-10-10T20:46:51.538 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90827.log 2023-10-10T20:46:51.539 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.102384.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.102384.log.gz 2023-10-10T20:46:51.539 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.90827.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.90827.log.gz 2023-10-10T20:46:51.539 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.99160.log 2023-10-10T20:46:51.539 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.102388.log 2023-10-10T20:46:51.540 INFO:teuthology.orchestra.run.smithi111.stderr: 94.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-10T20:46:51.540 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.99160.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.99160.log.gz 2023-10-10T20:46:51.540 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.102388.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-10T20:46:51.540 INFO:teuthology.orchestra.run.smithi111.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.102388.log.gz 2023-10-10T20:46:51.541 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.101226.log 2023-10-10T20:46:51.541 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.100834.log 2023-10-10T20:46:51.541 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.101226.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.101226.log.gz 2023-10-10T20:46:51.541 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.100834.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.100834.log.gz 2023-10-10T20:46:51.542 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90743.log 2023-10-10T20:46:51.542 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.101986.log 2023-10-10T20:46:51.542 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.90743.log: /var/log/ceph/ceph-client.1.101986.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90743.log.gz 2023-10-10T20:46:51.543 INFO:teuthology.orchestra.run.smithi111.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.101986.log.gz 2023-10-10T20:46:51.543 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.99164.log 2023-10-10T20:46:51.543 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-10T20:46:51.543 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.99164.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.99164.log.gz 2023-10-10T20:46:51.545 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.98430.log 2023-10-10T20:46:51.546 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.98430.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.98430.log.gz 2023-10-10T20:46:51.546 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.98426.log 2023-10-10T20:46:51.546 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.98426.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.98426.log.gz 2023-10-10T20:46:51.547 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.101230.log 2023-10-10T20:46:51.547 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.101230.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.101230.log.gz 2023-10-10T20:46:51.547 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-10T20:46:51.547 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-mon.b.log: 90.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-10T20:46:51.548 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.101990.log 2023-10-10T20:46:51.548 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.1.101990.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.101990.log.gz 2023-10-10T20:46:51.563 INFO:teuthology.orchestra.run.smithi111.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-10T20:46:51.713 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168686.log 2023-10-10T20:46:51.714 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166261.log 2023-10-10T20:46:51.714 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102759.log 2023-10-10T20:46:51.715 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168686.log: /var/log/ceph/ceph-client.admin.166261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168434.log 2023-10-10T20:46:51.715 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.168686.log.gz 2023-10-10T20:46:51.715 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2%/var/log/ceph/ceph-client.admin.102759.log: -- replaced with /var/log/ceph/ceph-client.admin.166261.log.gz 2023-10-10T20:46:51.715 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173148.log 2023-10-10T20:46:51.716 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103307.log 2023-10-10T20:46:51.716 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102759.log.gz 2023-10-10T20:46:51.716 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173148.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.168434.log.gz 2023-10-10T20:46:51.717 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180303.log 2023-10-10T20:46:51.717 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.173148.log.gz 2023-10-10T20:46:51.717 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.103307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.100062.log 2023-10-10T20:46:51.717 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103917.log /var/log/ceph/ceph-client.admin.180303.log: 2023-10-10T20:46:51.718 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.100062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169607.log 2023-10-10T20:46:51.718 INFO:teuthology.orchestra.run.smithi089.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.103307.log.gz 2023-10-10T20:46:51.718 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.100062.log.gz 88.5% 2023-10-10T20:46:51.719 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.180303.log.gz 2023-10-10T20:46:51.719 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.103917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103969.log 2023-10-10T20:46:51.719 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173824.log 2023-10-10T20:46:51.720 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.103917.log.gz 2023-10-10T20:46:51.720 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.103969.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.169607.log.gz 2023-10-10T20:46:51.720 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169859.log 2023-10-10T20:46:51.720 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103969.log.gz 2023-10-10T20:46:51.721 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178971.log 2023-10-10T20:46:51.721 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169859.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.173824.log.gz 2023-10-10T20:46:51.721 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171307.log 2023-10-10T20:46:51.722 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178971.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.169859.log.gz 2023-10-10T20:46:51.722 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157108.log 2023-10-10T20:46:51.722 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163713.log 2023-10-10T20:46:51.722 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.178971.log.gz 2023-10-10T20:46:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172862.log 2023-10-10T20:46:51.723 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.171307.log.gz 2023-10-10T20:46:51.723 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3%/var/log/ceph/ceph-client.admin.163713.log: -- replaced with /var/log/ceph/ceph-client.admin.157108.log.gz 2023-10-10T20:46:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173446.log 2023-10-10T20:46:51.724 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.163713.log.gz/var/log/ceph/ceph-client.admin.172862.log: 2023-10-10T20:46:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178105.log 2023-10-10T20:46:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173446.log: gzip 87.7%-5 -- replaced with /var/log/ceph/ceph-client.admin.172862.log.gz--verbose 2023-10-10T20:46:51.725 INFO:teuthology.orchestra.run.smithi089.stderr:-- /var/log/ceph/ceph-client.admin.170315.log 2023-10-10T20:46:51.725 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178105.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.173446.log.gz 2023-10-10T20:46:51.725 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167736.log 2023-10-10T20:46:51.726 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.178105.log.gz 2023-10-10T20:46:51.726 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101306.log 2023-10-10T20:46:51.726 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180880.log 2023-10-10T20:46:51.726 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.170315.log.gz 2023-10-10T20:46:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101306.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.167736.log.gz 2023-10-10T20:46:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157462.log 2023-10-10T20:46:51.727 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101306.log.gz 2023-10-10T20:46:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160255.log 2023-10-10T20:46:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181048.log 2023-10-10T20:46:51.728 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.180880.log.gz 2023-10-10T20:46:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105700.log 2023-10-10T20:46:51.729 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.157462.log.gz 2023-10-10T20:46:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181048.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.160255.log.gzgzip 2023-10-10T20:46:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.101604.log 2023-10-10T20:46:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.105700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168812.log 2023-10-10T20:46:51.730 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.181048.log.gz 2023-10-10T20:46:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108024.log 2023-10-10T20:46:51.731 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105700.log.gz 2023-10-10T20:46:51.731 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3%/var/log/ceph/ceph-client.admin.168812.log: -- replaced with /var/log/ceph/ceph-client.admin.101604.log.gz 2023-10-10T20:46:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103265.log 2023-10-10T20:46:51.732 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.168812.log.gz 2023-10-10T20:46:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.108024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98676.log 2023-10-10T20:46:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.103265.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108024.log.gz 2023-10-10T20:46:51.733 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167677.log 2023-10-10T20:46:51.733 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103265.log.gz 2023-10-10T20:46:51.733 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.98676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164101.log 2023-10-10T20:46:51.733 INFO:teuthology.orchestra.run.smithi089.stderr:gzip 88.8%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.98676.log.gz-- 2023-10-10T20:46:51.734 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.104974.log /var/log/ceph/ceph-client.admin.167677.log: 2023-10-10T20:46:51.734 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104624.log 2023-10-10T20:46:51.734 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.167677.log.gz 2023-10-10T20:46:51.735 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.104974.log: gzip -5 88.5%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.164101.log.gz 2023-10-10T20:46:51.735 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172674.log 2023-10-10T20:46:51.735 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.104624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102537.log 2023-10-10T20:46:51.735 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104974.log.gz 2023-10-10T20:46:51.736 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172674.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104624.log.gz 2023-10-10T20:46:51.736 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175477.log 2023-10-10T20:46:51.736 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.172674.log.gz 2023-10-10T20:46:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.102537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178147.log 2023-10-10T20:46:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:gzip /var/log/ceph/ceph-client.admin.175477.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.171013.log 2023-10-10T20:46:51.737 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102537.log.gz 2023-10-10T20:46:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164860.log 2023-10-10T20:46:51.738 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.175477.log.gz 2023-10-10T20:46:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:gzip /var/log/ceph/ceph-client.admin.171013.log: 89.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.178147.log.gz--verbose 2023-10-10T20:46:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:-- /var/log/ceph/ceph-client.admin.180388.log 2023-10-10T20:46:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108412.log 2023-10-10T20:46:51.739 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.171013.log.gz 2023-10-10T20:46:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170231.log 2023-10-10T20:46:51.739 INFO:teuthology.orchestra.run.smithi089.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.164860.log.gz 2023-10-10T20:46:51.740 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.180388.log.gz 2023-10-10T20:46:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.108412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99053.log 2023-10-10T20:46:51.740 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108412.log.gz 2023-10-10T20:46:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108371.log 2023-10-10T20:46:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.99053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156982.log 2023-10-10T20:46:51.741 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.170231.log.gz 2023-10-10T20:46:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.108371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158330.log 2023-10-10T20:46:51.742 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99053.log.gz 2023-10-10T20:46:51.742 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.108371.log.gz 2023-10-10T20:46:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166819.log 2023-10-10T20:46:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158330.log: gzip 88.6%-5 -- replaced with /var/log/ceph/ceph-client.admin.156982.log.gz--verbose 2023-10-10T20:46:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:-- /var/log/ceph/ceph-client.admin.164336.log 2023-10-10T20:46:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97134.log 2023-10-10T20:46:51.743 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.158330.log.gz 2023-10-10T20:46:51.744 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.166819.log.gz 2023-10-10T20:46:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175561.log 2023-10-10T20:46:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175015.log 2023-10-10T20:46:51.745 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.164336.log.gz 2023-10-10T20:46:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172716.log 2023-10-10T20:46:51.745 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97134.log.gz 2023-10-10T20:46:51.745 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.175561.log.gz 2023-10-10T20:46:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159894.log 2023-10-10T20:46:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102285.log 2023-10-10T20:46:51.746 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.175015.log.gz 2023-10-10T20:46:51.747 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96822.log 2023-10-10T20:46:51.747 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.172716.log.gz 2023-10-10T20:46:51.747 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.102285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180243.log 2023-10-10T20:46:51.747 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.159894.log.gz 2023-10-10T20:46:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.96822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160415.log 2023-10-10T20:46:51.748 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102285.log.gz 2023-10-10T20:46:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166111.log 2023-10-10T20:46:51.748 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96822.log.gz 2023-10-10T20:46:51.749 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5%/var/log/ceph/ceph-client.admin.160415.log: -- replaced with /var/log/ceph/ceph-client.admin.180243.log.gz 2023-10-10T20:46:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106172.log 2023-10-10T20:46:51.749 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160415.log.gz 2023-10-10T20:46:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177167.log 2023-10-10T20:46:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.106172.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-10T20:46:51.750 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.166111.log.gz 2023-10-10T20:46:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177167.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.106172.log.gz 2023-10-10T20:46:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165943.log 2023-10-10T20:46:51.751 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.177167.log.gz 2023-10-10T20:46:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178675.log 2023-10-10T20:46:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106130.log 2023-10-10T20:46:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168168.log 2023-10-10T20:46:51.752 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.165943.log.gz 2023-10-10T20:46:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.106130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163629.log 2023-10-10T20:46:51.753 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.178675.log.gz 2023-10-10T20:46:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168168.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106130.log.gz 2023-10-10T20:46:51.754 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101011.log 2023-10-10T20:46:51.754 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163629.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.168168.log.gz 2023-10-10T20:46:51.754 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171955.log 2023-10-10T20:46:51.755 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101011.log: 89.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.163629.log.gz--verbose 2023-10-10T20:46:51.755 INFO:teuthology.orchestra.run.smithi089.stderr:-- /var/log/ceph/ceph-client.admin.166861.log 2023-10-10T20:46:51.755 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107940.log 2023-10-10T20:46:51.755 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101011.log.gz 2023-10-10T20:46:51.756 INFO:teuthology.orchestra.run.smithi089.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.171955.log.gz 2023-10-10T20:46:51.756 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101688.log 2023-10-10T20:46:51.756 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.166861.log.gz 2023-10-10T20:46:51.757 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.107940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168560.log 2023-10-10T20:46:51.757 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101688.log: 87.2%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.107940.log.gz--verbose 2023-10-10T20:46:51.757 INFO:teuthology.orchestra.run.smithi089.stderr:-- /var/log/ceph/ceph-client.admin.97818.log 2023-10-10T20:46:51.757 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101688.log.gz 2023-10-10T20:46:51.758 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102667.log 2023-10-10T20:46:51.758 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.168560.log.gz 2023-10-10T20:46:51.758 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162234.log 2023-10-10T20:46:51.759 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.102667.log: gzip -5 88.9%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.97818.log.gz/var/log/ceph/ceph-client.admin.99011.log 2023-10-10T20:46:51.759 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-10-10T20:46:51.759 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162234.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102667.log.gz 2023-10-10T20:46:51.759 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104750.log 2023-10-10T20:46:51.760 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.162234.log.gz 2023-10-10T20:46:51.760 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.99011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178209.log 2023-10-10T20:46:51.760 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.104750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102579.log 2023-10-10T20:46:51.760 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99011.log.gz 2023-10-10T20:46:51.761 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178209.log: gzip 89.1%-5 -- replaced with /var/log/ceph/ceph-client.admin.104750.log.gz--verbose 2023-10-10T20:46:51.761 INFO:teuthology.orchestra.run.smithi089.stderr:-- /var/log/ceph/ceph-client.admin.168126.log 2023-10-10T20:46:51.761 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.102579.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.178209.log.gz 2023-10-10T20:46:51.762 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.179360.log 2023-10-10T20:46:51.762 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168126.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102579.log.gz 2023-10-10T20:46:51.762 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167207.log 2023-10-10T20:46:51.762 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.168126.log.gz 2023-10-10T20:46:51.763 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.179360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164986.log 2023-10-10T20:46:51.763 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.179360.log.gz 2023-10-10T20:46:51.763 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163821.log 2023-10-10T20:46:51.764 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171391.log 2023-10-10T20:46:51.764 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.167207.log.gz 2023-10-10T20:46:51.764 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163821.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.164986.log.gz 2023-10-10T20:46:51.765 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105446.log 2023-10-10T20:46:51.765 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8%/var/log/ceph/ceph-client.admin.171391.log: -- replaced with /var/log/ceph/ceph-client.admin.163821.log.gz 2023-10-10T20:46:51.765 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178411.log 2023-10-10T20:46:51.765 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.105446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178845.log 2023-10-10T20:46:51.766 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.171391.log.gz 2023-10-10T20:46:51.766 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178411.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105446.log.gz 2023-10-10T20:46:51.766 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181510.log 2023-10-10T20:46:51.767 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178845.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.178411.log.gz-5 2023-10-10T20:46:51.767 INFO:teuthology.orchestra.run.smithi089.stderr:--verbose -- /var/log/ceph/ceph-client.admin.97983.log 2023-10-10T20:46:51.767 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101436.log 2023-10-10T20:46:51.768 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.178845.log.gz 2023-10-10T20:46:51.768 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97983.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.181510.log.gz 2023-10-10T20:46:51.768 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167081.log 2023-10-10T20:46:51.768 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97983.log.gz 2023-10-10T20:46:51.769 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.166458.log 2023-10-10T20:46:51.769 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167081.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101436.log.gz 2023-10-10T20:46:51.769 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162150.log 2023-10-10T20:46:51.769 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.167081.log.gz 2023-10-10T20:46:51.770 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.166458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168042.log 2023-10-10T20:46:51.770 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.166458.log.gz 2023-10-10T20:46:51.770 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162024.log 2023-10-10T20:46:51.771 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168042.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.162150.log.gz 2023-10-10T20:46:51.771 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172176.log 2023-10-10T20:46:51.771 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.168042.log.gz 2023-10-10T20:46:51.771 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181258.log 2023-10-10T20:46:51.772 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172176.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.162024.log.gz 2023-10-10T20:46:51.772 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159768.log 2023-10-10T20:46:51.772 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167425.log 2023-10-10T20:46:51.773 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.172176.log.gz 2023-10-10T20:46:51.773 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159768.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.181258.log.gz 2023-10-10T20:46:51.773 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159558.log 2023-10-10T20:46:51.773 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.159768.log.gz 2023-10-10T20:46:51.774 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175057.log 2023-10-10T20:46:51.774 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166541.log 2023-10-10T20:46:51.774 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.167425.log.gz 2023-10-10T20:46:51.774 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.159558.log.gz 2023-10-10T20:46:51.775 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156886.log 2023-10-10T20:46:51.775 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.175057.log.gz 2023-10-10T20:46:51.775 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175225.log 2023-10-10T20:46:51.776 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179013.log 2023-10-10T20:46:51.776 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.166541.log.gz 2023-10-10T20:46:51.776 INFO:teuthology.orchestra.run.smithi089.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.156886.log.gz 2023-10-10T20:46:51.776 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98678.log 2023-10-10T20:46:51.777 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.175225.log.gz 2023-10-10T20:46:51.777 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174973.log 2023-10-10T20:46:51.777 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.98678.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.179013.log.gzgzip 2023-10-10T20:46:51.777 INFO:teuthology.orchestra.run.smithi089.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.163318.log 2023-10-10T20:46:51.778 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174973.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98678.log.gz 2023-10-10T20:46:51.778 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174516.log 2023-10-10T20:46:51.778 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.174973.log.gz 2023-10-10T20:46:51.779 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179660.log 2023-10-10T20:46:51.779 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101814.log 2023-10-10T20:46:51.779 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.163318.log.gz 2023-10-10T20:46:51.779 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179660.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.174516.log.gz 2023-10-10T20:46:51.780 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164227.log 2023-10-10T20:46:51.780 INFO:teuthology.orchestra.run.smithi089.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.179660.log.gz 2023-10-10T20:46:51.780 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102801.log 2023-10-10T20:46:51.780 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164227.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.101814.log.gz-5 2023-10-10T20:46:51.781 INFO:teuthology.orchestra.run.smithi089.stderr:--verbose -- /var/log/ceph/ceph-client.admin.168962.log 2023-10-10T20:46:51.781 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.164227.log.gz 2023-10-10T20:46:51.781 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.102801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101772.log 2023-10-10T20:46:51.782 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102801.log.gz 2023-10-10T20:46:51.782 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106492.log 2023-10-10T20:46:51.782 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101772.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.168962.log.gz 2023-10-10T20:46:51.782 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103779.log 2023-10-10T20:46:51.783 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101772.log.gz 2023-10-10T20:46:51.783 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.106492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170104.log 2023-10-10T20:46:51.783 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106492.log.gz 2023-10-10T20:46:51.784 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.103779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162528.log 2023-10-10T20:46:51.784 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157631.log 2023-10-10T20:46:51.784 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103779.log.gz 2023-10-10T20:46:51.784 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162528.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.170104.log.gz 2023-10-10T20:46:51.785 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177498.log 2023-10-10T20:46:51.785 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.162528.log.gz 2023-10-10T20:46:51.785 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180058.log 2023-10-10T20:46:51.786 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170497.log 2023-10-10T20:46:51.786 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.157631.log.gz 2023-10-10T20:46:51.786 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180058.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.177498.log.gz 2023-10-10T20:46:51.787 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160020.log 2023-10-10T20:46:51.787 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.180058.log.gz 2023-10-10T20:46:51.787 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106214.log 2023-10-10T20:46:51.787 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166763.log 2023-10-10T20:46:51.788 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.170497.log.gz 2023-10-10T20:46:51.788 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.106214.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.160020.log.gzgzip 2023-10-10T20:46:51.788 INFO:teuthology.orchestra.run.smithi089.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.98673.log 2023-10-10T20:46:51.789 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166763.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106214.log.gz 2023-10-10T20:46:51.789 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172758.log 2023-10-10T20:46:51.789 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.98673.log: gzip -5 --verbose -- 88.2%/var/log/ceph/ceph-client.admin.158540.log 2023-10-10T20:46:51.789 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.166763.log.gz 2023-10-10T20:46:51.790 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98673.log.gz 2023-10-10T20:46:51.790 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97991.log 2023-10-10T20:46:51.790 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158540.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.172758.log.gz 2023-10-10T20:46:51.791 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168602.log 2023-10-10T20:46:51.791 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97991.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.158540.log.gz 2023-10-10T20:46:51.791 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180100.log 2023-10-10T20:46:51.791 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168602.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97991.log.gz 2023-10-10T20:46:51.792 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176299.log 2023-10-10T20:46:51.792 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180100.log: gzip 89.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.168602.log.gz--verbose 2023-10-10T20:46:51.792 INFO:teuthology.orchestra.run.smithi089.stderr:-- /var/log/ceph/ceph-client.admin.163587.log 2023-10-10T20:46:51.792 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159726.log 2023-10-10T20:46:51.793 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.180100.log.gz 2023-10-10T20:46:51.793 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163587.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.176299.log.gz 2023-10-10T20:46:51.793 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161150.log 2023-10-10T20:46:51.794 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.163587.log.gz 2023-10-10T20:46:51.794 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177414.log 2023-10-10T20:46:51.794 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161150.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.159726.log.gz 2023-10-10T20:46:51.794 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165842.log 2023-10-10T20:46:51.795 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177414.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161150.log.gz 2023-10-10T20:46:51.795 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177624.log 2023-10-10T20:46:51.795 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.177414.log.gz 2023-10-10T20:46:51.795 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170357.log 2023-10-10T20:46:51.796 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167635.log 2023-10-10T20:46:51.796 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.165842.log.gz 2023-10-10T20:46:51.796 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170357.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.177624.log.gz 2023-10-10T20:46:51.797 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158107.log 2023-10-10T20:46:51.797 INFO:teuthology.orchestra.run.smithi089.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.170357.log.gz 2023-10-10T20:46:51.797 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170885.log 2023-10-10T20:46:51.797 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158107.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.167635.log.gz-5 2023-10-10T20:46:51.798 INFO:teuthology.orchestra.run.smithi089.stderr:--verbose -- /var/log/ceph/ceph-client.admin.176703.log 2023-10-10T20:46:51.798 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.158107.log.gz 2023-10-10T20:46:51.798 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97990.log 2023-10-10T20:46:51.799 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176703.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.170885.log.gz 2023-10-10T20:46:51.799 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180430.log 2023-10-10T20:46:51.799 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172800.log 2023-10-10T20:46:51.799 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.176703.log.gz 2023-10-10T20:46:51.800 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180430.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97990.log.gz 2023-10-10T20:46:51.800 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166153.log 2023-10-10T20:46:51.800 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.180430.log.gz 2023-10-10T20:46:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174642.log 2023-10-10T20:46:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172800.log: /var/log/ceph/ceph-client.admin.166153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171349.log 2023-10-10T20:46:51.801 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.172800.log.gz 2023-10-10T20:46:51.801 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.166153.log.gz 2023-10-10T20:46:51.802 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173278.log 2023-10-10T20:46:51.802 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.174642.log.gz 2023-10-10T20:46:51.802 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157276.log 2023-10-10T20:46:51.802 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173278.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.171349.log.gz 2023-10-10T20:46:51.803 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102495.log 2023-10-10T20:46:51.803 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157276.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.173278.log.gz 2023-10-10T20:46:51.803 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171779.log 2023-10-10T20:46:51.803 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.157276.log.gz 2023-10-10T20:46:51.804 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.102495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168306.log 2023-10-10T20:46:51.804 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164143.log 2023-10-10T20:46:51.804 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102495.log.gz 2023-10-10T20:46:51.805 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.171779.log.gz 2023-10-10T20:46:51.805 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162444.log 2023-10-10T20:46:51.805 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.168306.log.gz 2023-10-10T20:46:51.805 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158023.log 2023-10-10T20:46:51.806 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162444.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-10T20:46:51.806 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.164143.log.gz 2023-10-10T20:46:51.806 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.162444.log.gz 2023-10-10T20:46:51.806 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157360.log 2023-10-10T20:46:51.807 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.158023.log.gz 2023-10-10T20:46:51.807 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107074.log 2023-10-10T20:46:51.807 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170663.log 2023-10-10T20:46:51.808 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.157360.log.gz 2023-10-10T20:46:51.808 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.107074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167840.log 2023-10-10T20:46:51.808 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170663.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107074.log.gz 2023-10-10T20:46:51.808 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162276.log 2023-10-10T20:46:51.809 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167840.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.170663.log.gz 2023-10-10T20:46:51.809 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164672.log 2023-10-10T20:46:51.809 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159378.log 2023-10-10T20:46:51.809 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.167840.log.gz 2023-10-10T20:46:51.810 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162276.log: /var/log/ceph/ceph-client.admin.164672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.169155.log 2023-10-10T20:46:51.810 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.162276.log.gz 2023-10-10T20:46:51.810 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164588.log 2023-10-10T20:46:51.811 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159378.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.164672.log.gz 2023-10-10T20:46:51.811 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.169155.log: 86.0% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-10T20:46:51.811 INFO:teuthology.orchestra.run.smithi089.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.159378.log.gz 2023-10-10T20:46:51.811 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91031.log 2023-10-10T20:46:51.812 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.169155.log.gz 2023-10-10T20:46:51.812 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160541.log 2023-10-10T20:46:51.812 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.91031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179848.log 2023-10-10T20:46:51.813 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.164588.log.gz 2023-10-10T20:46:51.813 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160541.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.91031.log.gz 2023-10-10T20:46:51.813 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179974.log 2023-10-10T20:46:51.813 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179848.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.160541.log.gz 2023-10-10T20:46:51.814 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174212.log 2023-10-10T20:46:51.814 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.179848.log.gz/var/log/ceph/ceph-client.admin.179974.log: 2023-10-10T20:46:51.814 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171223.log 2023-10-10T20:46:51.814 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174212.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.179974.log.gz 2023-10-10T20:46:51.815 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104521.log 2023-10-10T20:46:51.815 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168644.log 2023-10-10T20:46:51.815 INFO:teuthology.orchestra.run.smithi089.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.174212.log.gz 2023-10-10T20:46:51.816 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.104521.log: 87.2%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.171223.log.gz 2023-10-10T20:46:51.816 INFO:teuthology.orchestra.run.smithi089.stderr:--verbose -- /var/log/ceph/ceph-client.admin.105056.log 2023-10-10T20:46:51.816 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104521.log.gz 2023-10-10T20:46:51.816 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160064.log 2023-10-10T20:46:51.817 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.105056.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.168644.log.gz 2023-10-10T20:46:51.817 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175267.log 2023-10-10T20:46:51.817 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160064.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105056.log.gz 2023-10-10T20:46:51.817 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170419.log 2023-10-10T20:46:51.818 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.160064.log.gz 2023-10-10T20:46:51.818 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176165.log 2023-10-10T20:46:51.818 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170419.log: 88.6%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.175267.log.gz--verbose 2023-10-10T20:46:51.819 INFO:teuthology.orchestra.run.smithi089.stderr:-- /var/log/ceph/ceph-client.admin.97304.log 2023-10-10T20:46:51.819 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98322.log 2023-10-10T20:46:51.819 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.170419.log.gz 2023-10-10T20:46:51.819 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.176165.log.gz 2023-10-10T20:46:51.820 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169544.log 2023-10-10T20:46:51.820 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.98322.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.97304.log.gz 2023-10-10T20:46:51.820 INFO:teuthology.orchestra.run.smithi089.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.168256.log 2023-10-10T20:46:51.821 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98322.log.gz 2023-10-10T20:46:51.821 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161234.log 2023-10-10T20:46:51.821 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.169544.log.gz 2023-10-10T20:46:51.821 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163360.log 2023-10-10T20:46:51.822 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181621.log 2023-10-10T20:46:51.822 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.168256.log.gz 2023-10-10T20:46:51.822 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4%/var/log/ceph/ceph-client.admin.163360.log: -- replaced with /var/log/ceph/ceph-client.admin.161234.log.gz 2023-10-10T20:46:51.822 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169985.log 2023-10-10T20:46:51.823 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.163360.log.gz 2023-10-10T20:46:51.823 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97988.log 2023-10-10T20:46:51.823 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169985.log: 89.5%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.181621.log.gz--verbose 2023-10-10T20:46:51.824 INFO:teuthology.orchestra.run.smithi089.stderr:-- /var/log/ceph/ceph-client.0.171734.log 2023-10-10T20:46:51.824 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108454.log 2023-10-10T20:46:51.824 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.169985.log.gz 2023-10-10T20:46:51.824 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97988.log.gz 2023-10-10T20:46:51.825 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.171734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168476.log 2023-10-10T20:46:51.825 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.171734.log.gz 2023-10-10T20:46:51.825 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.108454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174889.log 2023-10-10T20:46:51.825 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168476.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.108454.log.gz 2023-10-10T20:46:51.826 INFO:teuthology.orchestra.run.smithi089.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.170273.log 2023-10-10T20:46:51.826 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.168476.log.gz 2023-10-10T20:46:51.826 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99095.log 2023-10-10T20:46:51.827 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170273.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.174889.log.gz 2023-10-10T20:46:51.827 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173614.log 2023-10-10T20:46:51.827 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.170273.log.gz 2023-10-10T20:46:51.827 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.99095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98324.log 2023-10-10T20:46:51.828 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99095.log.gz 2023-10-10T20:46:51.828 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158246.log 2023-10-10T20:46:51.828 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.98324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180796.log 2023-10-10T20:46:51.829 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.173614.log.gz 2023-10-10T20:46:51.829 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158246.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98324.log.gz 2023-10-10T20:46:51.829 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168896.log 2023-10-10T20:46:51.829 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.158246.log.gz 2023-10-10T20:46:51.830 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157318.log 2023-10-10T20:46:51.830 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102623.log 2023-10-10T20:46:51.830 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.180796.log.gz 2023-10-10T20:46:51.831 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.159114.log 2023-10-10T20:46:51.831 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.157318.log.gz 2023-10-10T20:46:51.831 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.102623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169284.log 2023-10-10T20:46:51.831 INFO:teuthology.orchestra.run.smithi089.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.168896.log.gz 2023-10-10T20:46:51.832 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102623.log.gz 2023-10-10T20:46:51.832 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.159114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97390.log 2023-10-10T20:46:51.832 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.159114.log.gz 2023-10-10T20:46:51.832 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103097.log 2023-10-10T20:46:51.833 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.169284.log.gz 2023-10-10T20:46:51.833 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106450.log 2023-10-10T20:46:51.833 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.103097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175909.log 2023-10-10T20:46:51.833 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97390.log.gz 2023-10-10T20:46:51.834 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103097.log.gz/var/log/ceph/ceph-client.admin.106450.log: 2023-10-10T20:46:51.834 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158725.log 2023-10-10T20:46:51.834 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106450.log.gz 2023-10-10T20:46:51.835 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172218.log 2023-10-10T20:46:51.835 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170791.log 2023-10-10T20:46:51.835 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.175909.log.gz 2023-10-10T20:46:51.835 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.158725.log.gz 2023-10-10T20:46:51.836 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161384.log 2023-10-10T20:46:51.836 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.172218.log.gz 2023-10-10T20:46:51.836 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97989.log 2023-10-10T20:46:51.837 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178717.log 2023-10-10T20:46:51.837 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.170791.log.gz 2023-10-10T20:46:51.837 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.161384.log.gz 2023-10-10T20:46:51.837 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100507.log 2023-10-10T20:46:51.838 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178717.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97989.log.gz 2023-10-10T20:46:51.838 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169733.log 2023-10-10T20:46:51.838 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.100507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105574.log 2023-10-10T20:46:51.839 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.178717.log.gz 2023-10-10T20:46:51.839 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169733.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100507.log.gz 2023-10-10T20:46:51.839 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174931.log 2023-10-10T20:46:51.839 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.169733.log.gz 2023-10-10T20:46:51.840 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.105574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169418.log 2023-10-10T20:46:51.840 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174931.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105574.log.gz 2023-10-10T20:46:51.840 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105224.log 2023-10-10T20:46:51.840 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.174931.log.gz 2023-10-10T20:46:51.841 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103653.log 2023-10-10T20:46:51.841 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.105224.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.169418.log.gz-5 2023-10-10T20:46:51.841 INFO:teuthology.orchestra.run.smithi089.stderr:--verbose -- /var/log/ceph/ceph-client.admin.103373.log 2023-10-10T20:46:51.842 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.103653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178625.log 2023-10-10T20:46:51.842 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.105224.log.gz 2023-10-10T20:46:51.842 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.103373.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103653.log.gz 2023-10-10T20:46:51.842 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163276.log 2023-10-10T20:46:51.843 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103373.log.gz 2023-10-10T20:46:51.843 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101053.log 2023-10-10T20:46:51.843 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163276.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.178625.log.gz 2023-10-10T20:46:51.844 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165504.log 2023-10-10T20:46:51.844 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.163276.log.gz 2023-10-10T20:46:51.844 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159684.log 2023-10-10T20:46:51.845 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165504.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.101053.log.gz 2023-10-10T20:46:51.845 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97084.log 2023-10-10T20:46:51.845 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159684.log: gzip -5 --verbose -- 88.8%/var/log/ceph/ceph-client.admin.175351.log 2023-10-10T20:46:51.845 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.165504.log.gz 2023-10-10T20:46:51.846 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97084.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.159684.log.gz 2023-10-10T20:46:51.846 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173782.log 2023-10-10T20:46:51.846 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.97084.log.gz 2023-10-10T20:46:51.846 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171913.log 2023-10-10T20:46:51.847 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105784.log 2023-10-10T20:46:51.847 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.175351.log.gz 2023-10-10T20:46:51.847 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171913.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.173782.log.gz 2023-10-10T20:46:51.848 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165460.log 2023-10-10T20:46:51.848 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.171913.log.gz 2023-10-10T20:46:51.848 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.105784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171097.log 2023-10-10T20:46:51.848 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101137.log 2023-10-10T20:46:51.849 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.165460.log.gz 2023-10-10T20:46:51.849 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159198.log 2023-10-10T20:46:51.849 INFO:teuthology.orchestra.run.smithi089.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.105784.log.gz 2023-10-10T20:46:51.849 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166583.log 2023-10-10T20:46:51.850 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.171097.log.gz 2023-10-10T20:46:51.850 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101137.log: /var/log/ceph/ceph-client.admin.159198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177582.log 2023-10-10T20:46:51.850 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101137.log.gz 2023-10-10T20:46:51.851 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166583.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.159198.log.gz 2023-10-10T20:46:51.851 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173974.log 2023-10-10T20:46:51.851 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177582.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.166583.log.gz 2023-10-10T20:46:51.851 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171871.log 2023-10-10T20:46:51.852 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104353.log 2023-10-10T20:46:51.852 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.177582.log.gz 2023-10-10T20:46:51.852 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171871.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.173974.log.gz 2023-10-10T20:46:51.853 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102929.log 2023-10-10T20:46:51.853 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.104353.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.171871.log.gz 2023-10-10T20:46:51.853 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105098.log 2023-10-10T20:46:51.853 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1%/var/log/ceph/ceph-client.admin.102929.log: -- replaced with /var/log/ceph/ceph-client.admin.104353.log.gz 2023-10-10T20:46:51.854 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.161577.log 2023-10-10T20:46:51.854 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102929.log.gz 2023-10-10T20:46:51.854 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.105098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105658.log 2023-10-10T20:46:51.854 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.161577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158917.log 2023-10-10T20:46:51.855 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.161577.log.gz 2023-10-10T20:46:51.855 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105098.log.gz 2023-10-10T20:46:51.855 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.105658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172590.log 2023-10-10T20:46:51.856 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158917.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105658.log.gz 2023-10-10T20:46:51.856 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-10T20:46:51.856 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.158917.log.gz 2023-10-10T20:46:51.856 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171265.log 2023-10-10T20:46:51.857 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph.audit.log: gzip 87.7%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.172590.log.gz-- 2023-10-10T20:46:51.857 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160713.log 2023-10-10T20:46:51.857 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174827.log 2023-10-10T20:46:51.857 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.171265.log.gz 2023-10-10T20:46:51.858 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101179.log 2023-10-10T20:46:51.858 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.160713.log.gz 2023-10-10T20:46:51.858 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158151.log 2023-10-10T20:46:51.859 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174827.log: /var/log/ceph/ceph-client.admin.101179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90626.log 2023-10-10T20:46:51.859 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.174827.log.gz 2023-10-10T20:46:51.859 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105404.log 2023-10-10T20:46:51.859 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101179.log.gz 2023-10-10T20:46:51.860 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.158151.log.gz 2023-10-10T20:46:51.860 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.90626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160982.log 2023-10-10T20:46:51.860 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.90626.log.gz 2023-10-10T20:46:51.861 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103833.log 2023-10-10T20:46:51.861 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.105404.log: /var/log/ceph/ceph-client.admin.160982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173698.log 2023-10-10T20:46:51.861 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105404.log.gz 2023-10-10T20:46:51.861 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.103833.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.160982.log.gz 2023-10-10T20:46:51.862 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179055.log 2023-10-10T20:46:51.862 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100801.log 2023-10-10T20:46:51.862 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103833.log.gz/var/log/ceph/ceph-client.admin.173698.log: 2023-10-10T20:46:51.863 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179055.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.173698.log.gz 2023-10-10T20:46:51.863 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172470.log 2023-10-10T20:46:51.863 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.179055.log.gz 2023-10-10T20:46:51.863 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99179.log 2023-10-10T20:46:51.864 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.100801.log: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-10T20:46:51.864 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107220.log 2023-10-10T20:46:51.864 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100801.log.gz 2023-10-10T20:46:51.864 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2%/var/log/ceph/ceph-client.admin.99179.log: -- replaced with /var/log/ceph/ceph-client.admin.172470.log.gz 2023-10-10T20:46:51.865 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166625.log 2023-10-10T20:46:51.865 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.107220.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99179.log.gz 2023-10-10T20:46:51.865 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164059.log 2023-10-10T20:46:51.866 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166625.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107220.log.gz 2023-10-10T20:46:51.866 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.100466.log 2023-10-10T20:46:51.866 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.166625.log.gz 2023-10-10T20:46:51.866 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103737.log 2023-10-10T20:46:51.867 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.2.100466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158683.log 2023-10-10T20:46:51.867 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.100466.log.gz 2023-10-10T20:46:51.867 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.164059.log.gz 2023-10-10T20:46:51.868 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.103737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107380.log 2023-10-10T20:46:51.868 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158683.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103737.log.gz 2023-10-10T20:46:51.868 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.171738.log 2023-10-10T20:46:51.868 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.107380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167593.log 2023-10-10T20:46:51.869 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.158683.log.gz 2023-10-10T20:46:51.869 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.171738.log: 87.9%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.107380.log.gz--verbose 2023-10-10T20:46:51.869 INFO:teuthology.orchestra.run.smithi089.stderr:-- /var/log/ceph/ceph-client.admin.106744.log 2023-10-10T20:46:51.869 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.171738.log.gz 2023-10-10T20:46:51.870 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168390.log 2023-10-10T20:46:51.870 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.106744.log: gzip -5 --verbose -- 89.2%/var/log/ceph/ceph-client.0.103566.log 2023-10-10T20:46:51.870 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.167593.log.gz 2023-10-10T20:46:51.873 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168390.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106744.log.gz 2023-10-10T20:46:51.873 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97348.log 2023-10-10T20:46:51.873 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.103566.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.168390.log.gz 2023-10-10T20:46:51.874 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.106047.log 2023-10-10T20:46:51.874 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.103566.log.gz 2023-10-10T20:46:51.874 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108066.log 2023-10-10T20:46:51.875 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.106047.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.97348.log.gz 2023-10-10T20:46:51.875 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157715.log 2023-10-10T20:46:51.875 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.106047.log.gz 2023-10-10T20:46:51.875 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.108066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180754.log 2023-10-10T20:46:51.876 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161024.log 2023-10-10T20:46:51.876 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108066.log.gz 2023-10-10T20:46:51.876 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.157715.log.gz 2023-10-10T20:46:51.876 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105140.log 2023-10-10T20:46:51.877 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.180754.log.gz 2023-10-10T20:46:51.877 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106618.log 2023-10-10T20:46:51.877 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.105140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160457.log 2023-10-10T20:46:51.878 INFO:teuthology.orchestra.run.smithi089.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.161024.log.gz 2023-10-10T20:46:51.878 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105140.log.gz 2023-10-10T20:46:51.878 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.106618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97218.log 2023-10-10T20:46:51.879 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106618.log.gz 2023-10-10T20:46:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173404.log 2023-10-10T20:46:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97218.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.160457.log.gz 2023-10-10T20:46:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103875.log 2023-10-10T20:46:51.880 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173404.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97218.log.gz 2023-10-10T20:46:51.880 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107856.log 2023-10-10T20:46:51.880 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.173404.log.gz 2023-10-10T20:46:51.880 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.103875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104437.log 2023-10-10T20:46:51.881 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.107856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101478.log 2023-10-10T20:46:51.881 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103875.log.gz 2023-10-10T20:46:51.881 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107856.log.gz 2023-10-10T20:46:51.882 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.104437.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-10T20:46:51.882 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.104437.log.gz 2023-10-10T20:46:51.882 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107728.log 2023-10-10T20:46:51.882 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mds.c.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101478.log.gz 2023-10-10T20:46:51.883 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163098.log 2023-10-10T20:46:51.883 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.107728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162318.log 2023-10-10T20:46:51.883 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163098.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107728.log.gz 2023-10-10T20:46:51.884 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171055.log 2023-10-10T20:46:51.884 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.163098.log.gz 2023-10-10T20:46:51.884 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162486.log 2023-10-10T20:46:51.884 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161748.log 2023-10-10T20:46:51.885 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171055.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.162318.log.gz 2023-10-10T20:46:51.885 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162876.log 2023-10-10T20:46:51.885 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.171055.log.gz 2023-10-10T20:46:51.885 INFO:teuthology.orchestra.run.smithi089.stderr:gzip /var/log/ceph/ceph-client.admin.161748.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.169460.log 2023-10-10T20:46:51.886 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.162486.log.gz 2023-10-10T20:46:51.886 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171821.log 2023-10-10T20:46:51.886 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.161748.log.gz 2023-10-10T20:46:51.886 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169460.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.162876.log.gz 2023-10-10T20:46:51.887 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181300.log 2023-10-10T20:46:51.887 INFO:teuthology.orchestra.run.smithi089.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.169460.log.gz 2023-10-10T20:46:51.887 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104834.log 2023-10-10T20:46:51.888 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169817.log 2023-10-10T20:46:51.888 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181300.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.171821.log.gz 2023-10-10T20:46:51.888 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.104834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159282.log 2023-10-10T20:46:51.888 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.181300.log.gz 2023-10-10T20:46:51.889 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.104834.log.gz 2023-10-10T20:46:51.889 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 /var/log/ceph/ceph-client.admin.169817.log: --verbose -- /var/log/ceph/ceph-client.admin.172092.log 2023-10-10T20:46:51.889 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103611.log 2023-10-10T20:46:51.889 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.169817.log.gz 2023-10-10T20:46:51.890 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172092.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.159282.log.gz 2023-10-10T20:46:51.890 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180922.log 2023-10-10T20:46:51.890 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.172092.log.gz 2023-10-10T20:46:51.891 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.103611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97776.log 2023-10-10T20:46:51.891 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96915.log 2023-10-10T20:46:51.891 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180922.log: 90.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-10T20:46:51.891 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103611.log.gz 2023-10-10T20:46:51.892 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97776.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.84646.log 2023-10-10T20:46:51.892 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.180922.log.gz 2023-10-10T20:46:51.892 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.96915.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97776.log.gz 2023-10-10T20:46:51.893 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173866.log 2023-10-10T20:46:51.893 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96915.log.gz 2023-10-10T20:46:51.893 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph.tmp-client.admin.84646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108174.log 2023-10-10T20:46:51.894 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.84646.log.gz 2023-10-10T20:46:51.894 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101898.log 2023-10-10T20:46:51.894 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.108174.log: 88.2%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.173866.log.gz--verbose 2023-10-10T20:46:51.894 INFO:teuthology.orchestra.run.smithi089.stderr:-- /var/log/ceph/ceph-client.admin.163002.log 2023-10-10T20:46:51.895 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101898.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.108174.log.gz 2023-10-10T20:46:51.895 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169943.log 2023-10-10T20:46:51.895 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101898.log.gz 2023-10-10T20:46:51.896 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177075.log 2023-10-10T20:46:51.896 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169943.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.163002.log.gz 2023-10-10T20:46:51.896 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105268.log 2023-10-10T20:46:51.897 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.169943.log.gz 2023-10-10T20:46:51.897 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158372.log 2023-10-10T20:46:51.897 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.177075.log.gz 2023-10-10T20:46:51.897 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.105268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104896.log 2023-10-10T20:46:51.898 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176427.log 2023-10-10T20:46:51.898 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.105268.log.gz 2023-10-10T20:46:51.898 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6%/var/log/ceph/ceph-client.admin.104896.log: -- replaced with /var/log/ceph/ceph-client.admin.158372.log.gz 2023-10-10T20:46:51.899 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166667.log 2023-10-10T20:46:51.899 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104896.log.gz 2023-10-10T20:46:51.899 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106931.log 2023-10-10T20:46:51.899 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100843.log 2023-10-10T20:46:51.900 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.176427.log.gz 2023-10-10T20:46:51.900 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.166667.log.gz 2023-10-10T20:46:51.900 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.106931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165210.log 2023-10-10T20:46:51.901 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106931.log.gz 2023-10-10T20:46:51.901 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.100843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105532.log 2023-10-10T20:46:51.901 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179443.log 2023-10-10T20:46:51.901 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100843.log.gz 2023-10-10T20:46:51.902 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.165210.log.gz 2023-10-10T20:46:51.902 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.105532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107298.log 2023-10-10T20:46:51.902 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105532.log.gz 2023-10-10T20:46:51.903 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.103570.log 2023-10-10T20:46:51.903 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.107298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99305.log 2023-10-10T20:46:51.903 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.179443.log.gz 2023-10-10T20:46:51.903 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107298.log.gz 2023-10-10T20:46:51.904 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.103570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107592.log 2023-10-10T20:46:51.904 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.103570.log.gz 2023-10-10T20:46:51.904 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.99305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172260.log 2023-10-10T20:46:51.904 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.107592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160333.log 2023-10-10T20:46:51.905 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99305.log.gz 2023-10-10T20:46:51.905 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107592.log.gz 2023-10-10T20:46:51.905 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181384.log 2023-10-10T20:46:51.905 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.172260.log.gz 2023-10-10T20:46:51.906 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103695.log 2023-10-10T20:46:51.906 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163545.log 2023-10-10T20:46:51.906 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.160333.log.gz 2023-10-10T20:46:51.907 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.181384.log.gz 2023-10-10T20:46:51.907 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.103695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98549.log 2023-10-10T20:46:51.907 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103695.log.gz 2023-10-10T20:46:51.907 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171541.log 2023-10-10T20:46:51.908 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.98549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165028.log 2023-10-10T20:46:51.908 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.163545.log.gz 2023-10-10T20:46:51.908 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98549.log.gz 2023-10-10T20:46:51.909 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156806.log 2023-10-10T20:46:51.909 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.171541.log.gz 2023-10-10T20:46:51.909 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160499.log 2023-10-10T20:46:51.909 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177456.log 2023-10-10T20:46:51.910 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.165028.log.gz 2023-10-10T20:46:51.910 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.156806.log.gz 2023-10-10T20:46:51.910 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178887.log 2023-10-10T20:46:51.911 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.160499.log.gz 2023-10-10T20:46:51.911 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174785.log 2023-10-10T20:46:51.911 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173234.log 2023-10-10T20:46:51.912 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.177456.log.gz 2023-10-10T20:46:51.912 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.178887.log.gz 2023-10-10T20:46:51.912 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101520.log 2023-10-10T20:46:51.912 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.174785.log.gz 2023-10-10T20:46:51.913 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181581.log 2023-10-10T20:46:51.913 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101520.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.173234.log.gz 2023-10-10T20:46:51.913 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98329.log 2023-10-10T20:46:51.914 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101520.log.gz 2023-10-10T20:46:51.914 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176035.log 2023-10-10T20:46:51.914 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.181581.log.gz/var/log/ceph/ceph-client.admin.98329.log: 2023-10-10T20:46:51.914 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160797.log 2023-10-10T20:46:51.915 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161066.log 2023-10-10T20:46:51.915 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98329.log.gz 2023-10-10T20:46:51.915 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.176035.log.gz 2023-10-10T20:46:51.916 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163444.log 2023-10-10T20:46:51.916 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.160797.log.gz 2023-10-10T20:46:51.916 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180346.log 2023-10-10T20:46:51.916 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108496.log 2023-10-10T20:46:51.917 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.161066.log.gz 2023-10-10T20:46:51.917 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.163444.log.gz 2023-10-10T20:46:51.917 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170747.log 2023-10-10T20:46:51.918 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.108496.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.180346.log.gz 2023-10-10T20:46:51.918 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180159.log 2023-10-10T20:46:51.918 INFO:teuthology.orchestra.run.smithi089.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.108496.log.gz 2023-10-10T20:46:51.918 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98327.log 2023-10-10T20:46:51.919 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180159.log: gzip -5 --verbose -- 88.4%/var/log/ceph/ceph-client.admin.162752.log 2023-10-10T20:46:51.919 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.170747.log.gz 2023-10-10T20:46:51.919 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.98327.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.180159.log.gz 2023-10-10T20:46:51.920 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102060.log 2023-10-10T20:46:51.920 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98327.log.gz 2023-10-10T20:46:51.920 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157066.log 2023-10-10T20:46:51.920 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.102060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165292.log 2023-10-10T20:46:51.921 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.162752.log.gz 2023-10-10T20:46:51.921 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157066.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102060.log.gz 2023-10-10T20:46:51.921 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.164014.log 2023-10-10T20:46:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:gzip /var/log/ceph/ceph-client.admin.165292.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.106702.log 2023-10-10T20:46:51.922 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.157066.log.gz 2023-10-10T20:46:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.164014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177226.log 2023-10-10T20:46:51.922 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.164014.log.gz 2023-10-10T20:46:51.923 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.165292.log.gz 2023-10-10T20:46:51.923 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.106702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167509.log 2023-10-10T20:46:51.924 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177226.log: 89.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.106702.log.gz-5 2023-10-10T20:46:51.924 INFO:teuthology.orchestra.run.smithi089.stderr:--verbose -- /var/log/ceph/ceph-client.admin.157505.log 2023-10-10T20:46:51.924 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167509.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.177226.log.gz 2023-10-10T20:46:51.925 INFO:teuthology.orchestra.run.smithi089.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.173106.log 2023-10-10T20:46:51.925 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157505.log: gzip -5 --verbose 88.0%-- /var/log/ceph/ceph-client.admin.179619.log -- replaced with /var/log/ceph/ceph-client.admin.167509.log.gz 2023-10-10T20:46:51.925 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-10-10T20:46:51.926 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.157505.log.gz 2023-10-10T20:46:51.926 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.159110.log 2023-10-10T20:46:51.926 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.173106.log.gz 2023-10-10T20:46:51.926 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159516.log 2023-10-10T20:46:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.159110.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.179619.log.gz 2023-10-10T20:46:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160627.log 2023-10-10T20:46:51.927 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.159110.log.gz 2023-10-10T20:46:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177894.log 2023-10-10T20:46:51.928 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.159516.log.gz 2023-10-10T20:46:51.928 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169901.log 2023-10-10T20:46:51.928 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177894.log: gzip 87.4%-5 -- replaced with /var/log/ceph/ceph-client.admin.160627.log.gz--verbose 2023-10-10T20:46:51.929 INFO:teuthology.orchestra.run.smithi089.stderr:-- /var/log/ceph/ceph-client.admin.164630.log 2023-10-10T20:46:51.929 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169901.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.177894.log.gz 2023-10-10T20:46:51.929 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177979.log 2023-10-10T20:46:51.929 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.169901.log.gz 2023-10-10T20:46:51.930 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163148.log 2023-10-10T20:46:51.930 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177979.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164630.log.gz 2023-10-10T20:46:51.930 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157192.log 2023-10-10T20:46:51.931 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.177979.log.gz 2023-10-10T20:46:51.931 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101730.log 2023-10-10T20:46:51.931 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.163148.log.gz 2023-10-10T20:46:51.932 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166195.log 2023-10-10T20:46:51.932 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174684.log 2023-10-10T20:46:51.932 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.157192.log.gz 2023-10-10T20:46:51.932 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101730.log.gz 2023-10-10T20:46:51.933 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106576.log 2023-10-10T20:46:51.933 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99473.log 2023-10-10T20:46:51.933 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.106576.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.174684.log.gz 2023-10-10T20:46:51.934 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162674.log 2023-10-10T20:46:51.934 INFO:teuthology.orchestra.run.smithi089.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.166195.log.gz 2023-10-10T20:46:51.934 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106576.log.gz 2023-10-10T20:46:51.935 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.99473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107506.log 2023-10-10T20:46:51.935 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162674.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99473.log.gz 2023-10-10T20:46:51.935 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163755.log 2023-10-10T20:46:51.935 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.107506.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.162674.log.gz 2023-10-10T20:46:51.936 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177310.log 2023-10-10T20:46:51.936 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163755.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107506.log.gz 2023-10-10T20:46:51.936 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107982.log 2023-10-10T20:46:51.937 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176121.log 2023-10-10T20:46:51.937 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.107982.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.177310.log.gz 2023-10-10T20:46:51.937 INFO:teuthology.orchestra.run.smithi089.stderr:gzip 91.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.163755.log.gz-- /var/log/ceph/ceph-client.admin.181174.log 2023-10-10T20:46:51.937 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-10-10T20:46:51.938 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107982.log.gz 2023-10-10T20:46:51.938 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178369.log 2023-10-10T20:46:51.938 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181174.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.176121.log.gz-5 2023-10-10T20:46:51.938 INFO:teuthology.orchestra.run.smithi089.stderr:--verbose -- /var/log/ceph/ceph-client.admin.104581.log 2023-10-10T20:46:51.939 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169200.log 2023-10-10T20:46:51.939 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.181174.log.gz 2023-10-10T20:46:51.939 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.104581.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.178369.log.gz 2023-10-10T20:46:51.940 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163503.log 2023-10-10T20:46:51.940 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104581.log.gz 2023-10-10T20:46:51.940 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106256.log 2023-10-10T20:46:51.940 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175099.log 2023-10-10T20:46:51.941 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.169200.log.gz 2023-10-10T20:46:51.941 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.163503.log.gz 2023-10-10T20:46:51.941 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.106256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180201.log 2023-10-10T20:46:51.941 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106256.log.gz 2023-10-10T20:46:51.942 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179163.log 2023-10-10T20:46:51.942 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107032.log 2023-10-10T20:46:51.942 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.175099.log.gz 2023-10-10T20:46:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179163.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.180201.log.gz 2023-10-10T20:46:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167383.log 2023-10-10T20:46:51.943 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.179163.log.gz 2023-10-10T20:46:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.107032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179485.log 2023-10-10T20:46:51.944 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167383.log: gzip -5 --verbose -- 88.0%/var/log/ceph/ceph-client.admin.177937.log 2023-10-10T20:46:51.944 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.107032.log.gz 2023-10-10T20:46:51.944 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179485.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.167383.log.gz 2023-10-10T20:46:51.945 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157024.log 2023-10-10T20:46:51.945 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177937.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.179485.log.gz 2023-10-10T20:46:51.945 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98325.log 2023-10-10T20:46:51.945 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157024.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.177937.log.gz 2023-10-10T20:46:51.946 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159978.log 2023-10-10T20:46:51.946 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.98325.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.157024.log.gz 2023-10-10T20:46:51.946 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165985.log 2023-10-10T20:46:51.947 INFO:teuthology.orchestra.run.smithi089.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.98325.log.gz 2023-10-10T20:46:51.947 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161940.log 2023-10-10T20:46:51.947 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98321.log 2023-10-10T20:46:51.947 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.159978.log.gz 2023-10-10T20:46:51.948 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161940.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.165985.log.gz 2023-10-10T20:46:51.948 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161790.log 2023-10-10T20:46:51.948 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.161940.log.gz 2023-10-10T20:46:51.948 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.98321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97432.log 2023-10-10T20:46:51.949 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161790.log: gzip -5 88.5%--verbose -- /var/log/ceph/ceph-client.admin.168348.log -- replaced with /var/log/ceph/ceph-client.admin.98321.log.gz 2023-10-10T20:46:51.949 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-10-10T20:46:51.949 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97432.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.161790.log.gz 2023-10-10T20:46:51.950 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98682.log 2023-10-10T20:46:51.950 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97432.log.gz 2023-10-10T20:46:51.950 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177792.log 2023-10-10T20:46:51.951 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.98682.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.168348.log.gz 2023-10-10T20:46:51.951 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102453.log 2023-10-10T20:46:51.951 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98682.log.gz 2023-10-10T20:46:51.951 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173656.log 2023-10-10T20:46:51.952 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.177792.log.gz 2023-10-10T20:46:51.952 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.102453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175519.log 2023-10-10T20:46:51.952 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174558.log 2023-10-10T20:46:51.952 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102453.log.gz 2023-10-10T20:46:51.953 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175519.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.173656.log.gz 2023-10-10T20:46:51.953 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180669.log 2023-10-10T20:46:51.953 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174558.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.175519.log.gz 2023-10-10T20:46:51.953 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172428.log 2023-10-10T20:46:51.954 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164420.log 2023-10-10T20:46:51.954 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.174558.log.gz 2023-10-10T20:46:51.954 INFO:teuthology.orchestra.run.smithi089.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.180669.log.gz 2023-10-10T20:46:51.955 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172428.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-10T20:46:51.955 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.172428.log.gz 2023-10-10T20:46:51.955 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157589.log 2023-10-10T20:46:51.955 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-osd.3.log: gzip 88.7%-5 -- replaced with /var/log/ceph/ceph-client.admin.164420.log.gz--verbose 2023-10-10T20:46:51.956 INFO:teuthology.orchestra.run.smithi089.stderr:-- /var/log/ceph/ceph-client.admin.181663.log 2023-10-10T20:46:51.956 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108636.log 2023-10-10T20:46:51.956 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181663.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.157589.log.gz 2023-10-10T20:46:51.957 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181468.log 2023-10-10T20:46:51.957 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.181663.log.gz 2023-10-10T20:46:51.957 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.108636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176595.log 2023-10-10T20:46:51.957 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167918.log 2023-10-10T20:46:51.958 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181468.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108636.log.gz 2023-10-10T20:46:51.958 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102971.log 2023-10-10T20:46:51.958 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.181468.log.gz 2023-10-10T20:46:51.959 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.176595.log.gz 2023-10-10T20:46:51.959 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107422.log 2023-10-10T20:46:51.959 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167918.log: /var/log/ceph/ceph-client.admin.102971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180627.log 2023-10-10T20:46:51.959 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.167918.log.gz 2023-10-10T20:46:51.960 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.107422.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102971.log.gz 2023-10-10T20:46:51.960 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160108.log 2023-10-10T20:46:51.960 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.107422.log.gz 2023-10-10T20:46:51.960 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98677.log 2023-10-10T20:46:51.961 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162192.log 2023-10-10T20:46:51.961 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160108.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.180627.log.gz 2023-10-10T20:46:51.961 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.98677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159156.log 2023-10-10T20:46:51.962 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.160108.log.gz 2023-10-10T20:46:51.962 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98677.log.gz 2023-10-10T20:46:51.962 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98969.log 2023-10-10T20:46:51.962 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162192.log: /var/log/ceph/ceph-client.admin.159156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169649.log 2023-10-10T20:46:51.963 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.162192.log.gz 2023-10-10T20:46:51.963 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.98969.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.159156.log.gz 2023-10-10T20:46:51.963 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-10T20:46:51.964 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98969.log.gz 2023-10-10T20:46:51.964 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162402.log 2023-10-10T20:46:51.964 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170969.log 2023-10-10T20:46:51.964 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-osd.1.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.169649.log.gz 2023-10-10T20:46:51.965 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167551.log 2023-10-10T20:46:51.965 INFO:teuthology.orchestra.run.smithi089.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.162402.log.gz 2023-10-10T20:46:51.965 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158599.log 2023-10-10T20:46:51.965 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107772.log 2023-10-10T20:46:51.966 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167551.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.170969.log.gz 2023-10-10T20:46:51.966 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100633.log 2023-10-10T20:46:51.967 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.167551.log.gz 2023-10-10T20:46:51.967 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.158599.log.gz 2023-10-10T20:46:51.967 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.107772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164756.log 2023-10-10T20:46:51.968 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174743.log 2023-10-10T20:46:51.968 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.100633.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107772.log.gz 2023-10-10T20:46:51.968 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106088.log 2023-10-10T20:46:51.968 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100633.log.gz 2023-10-10T20:46:51.969 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.164756.log.gz 2023-10-10T20:46:51.969 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105362.log 2023-10-10T20:46:51.969 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97524.log 2023-10-10T20:46:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.106088.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.174743.log.gz 2023-10-10T20:46:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.105362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178287.log 2023-10-10T20:46:51.970 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106088.log.gz 2023-10-10T20:46:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97524.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.105362.log.gz 2023-10-10T20:46:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.166454.log 2023-10-10T20:46:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178287.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97524.log.gz 2023-10-10T20:46:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97260.log 2023-10-10T20:46:51.971 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.178287.log.gz 2023-10-10T20:46:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.166454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101982.log 2023-10-10T20:46:51.972 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.166454.log.gz 2023-10-10T20:46:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105490.log 2023-10-10T20:46:51.973 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97260.log.gz 2023-10-10T20:46:51.973 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101646.log 2023-10-10T20:46:51.973 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.105490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176469.log 2023-10-10T20:46:51.973 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101982.log.gz 2023-10-10T20:46:51.974 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157779.log 2023-10-10T20:46:51.974 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105490.log.gz 2023-10-10T20:46:51.974 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101646.log.gz 2023-10-10T20:46:51.975 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163671.log 2023-10-10T20:46:51.975 INFO:teuthology.orchestra.run.smithi089.stderr:gzip /var/log/ceph/ceph-client.admin.157779.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.99347.log 2023-10-10T20:46:51.975 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.176469.log.gz 2023-10-10T20:46:51.975 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100717.log 2023-10-10T20:46:51.976 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157779.log.gz 2023-10-10T20:46:51.976 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.99347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177033.log 2023-10-10T20:46:51.976 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163671.log.gz 2023-10-10T20:46:51.977 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99347.log.gz 2023-10-10T20:46:51.977 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.100717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162108.log 2023-10-10T20:46:51.977 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165590.log 2023-10-10T20:46:51.978 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177033.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100717.log.gz 2023-10-10T20:46:51.978 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108367.log 2023-10-10T20:46:51.978 INFO:teuthology.orchestra.run.smithi089.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.177033.log.gz 2023-10-10T20:46:51.978 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3%/var/log/ceph/ceph-client.admin.165590.log: -- replaced with /var/log/ceph/ceph-client.admin.162108.log.gz 2023-10-10T20:46:51.979 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180016.log 2023-10-10T20:46:51.979 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.108367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170835.log 2023-10-10T20:46:51.979 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.108367.log.gz 2023-10-10T20:46:51.980 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.165590.log.gz 2023-10-10T20:46:51.980 INFO:teuthology.orchestra.run.smithi089.stderr:gzip /var/log/ceph/ceph-client.admin.180016.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.158767.log 2023-10-10T20:46:51.980 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158851.log 2023-10-10T20:46:51.980 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.180016.log.gz 2023-10-10T20:46:51.981 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5%/var/log/ceph/ceph-client.admin.158767.log: -- replaced with /var/log/ceph/ceph-client.admin.170835.log.gz 2023-10-10T20:46:51.981 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159936.log 2023-10-10T20:46:51.981 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160923.log 2023-10-10T20:46:51.982 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.158767.log.gz 2023-10-10T20:46:51.982 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167257.log 2023-10-10T20:46:51.982 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.159936.log.gz 2023-10-10T20:46:51.982 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160923.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.158851.log.gz 2023-10-10T20:46:51.983 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167257.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.160923.log.gz 2023-10-10T20:46:51.983 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.167257.log.gz 2023-10-10T20:46:51.983 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159642.log 2023-10-10T20:46:51.984 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177666.log 2023-10-10T20:46:51.984 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159852.log 2023-10-10T20:46:51.984 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.159642.log.gz 2023-10-10T20:46:51.984 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177666.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-10T20:46:51.985 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100591.log 2023-10-10T20:46:51.985 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159852.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.177666.log.gz 2023-10-10T20:46:51.985 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97474.log 2023-10-10T20:46:51.985 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.159852.log.gz 2023-10-10T20:46:51.986 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.100591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176949.log 2023-10-10T20:46:51.986 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97474.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100591.log.gz 2023-10-10T20:46:51.986 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176949.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97474.log.gz 2023-10-10T20:46:51.987 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.176949.log.gz 2023-10-10T20:46:51.987 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173190.log 2023-10-10T20:46:51.987 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104084.log 2023-10-10T20:46:51.987 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157857.log 2023-10-10T20:46:51.988 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.104084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175827.log 2023-10-10T20:46:51.988 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.173190.log.gz 2023-10-10T20:46:51.988 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157857.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104084.log.gz 2023-10-10T20:46:51.989 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175827.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.157857.log.gz 2023-10-10T20:46:51.989 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.175827.log.gz 2023-10-10T20:46:51.989 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160671.log 2023-10-10T20:46:51.989 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168084.log 2023-10-10T20:46:51.990 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178581.log 2023-10-10T20:46:51.990 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159474.log 2023-10-10T20:46:51.990 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.160671.log.gz 2023-10-10T20:46:51.991 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168518.log 2023-10-10T20:46:51.991 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.168084.log.gz 2023-10-10T20:46:51.991 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159474.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.178581.log.gz 2023-10-10T20:46:51.991 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168518.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.159474.log.gz 2023-10-10T20:46:51.992 INFO:teuthology.orchestra.run.smithi089.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.168518.log.gz 2023-10-10T20:46:51.992 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176257.log 2023-10-10T20:46:51.992 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171139.log 2023-10-10T20:46:51.992 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102141.log 2023-10-10T20:46:51.993 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171139.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.176257.log.gz 2023-10-10T20:46:51.993 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.102141.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.171139.log.gz 2023-10-10T20:46:51.993 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102141.log.gz 2023-10-10T20:46:51.994 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103181.log 2023-10-10T20:46:51.994 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169775.log 2023-10-10T20:46:51.994 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.103181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104479.log 2023-10-10T20:46:51.994 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169775.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103181.log.gz 2023-10-10T20:46:51.995 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.104479.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.169775.log.gz 2023-10-10T20:46:51.995 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104479.log.gz 2023-10-10T20:46:51.995 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100549.log 2023-10-10T20:46:51.995 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173064.log 2023-10-10T20:46:51.996 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.100549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165548.log 2023-10-10T20:46:51.996 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173064.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100549.log.gz 2023-10-10T20:46:51.996 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165548.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.173064.log.gz 2023-10-10T20:46:51.997 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.165548.log.gz 2023-10-10T20:46:51.997 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177708.log 2023-10-10T20:46:51.997 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178021.log 2023-10-10T20:46:51.997 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161276.log 2023-10-10T20:46:51.998 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181090.log 2023-10-10T20:46:51.998 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.177708.log.gz 2023-10-10T20:46:51.998 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160755.log 2023-10-10T20:46:51.999 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.178021.log.gz 2023-10-10T20:46:51.999 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.161276.log.gz 2023-10-10T20:46:51.999 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181090.log: /var/log/ceph/ceph-client.admin.160755.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.181090.log.gz 2023-10-10T20:46:52.000 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.160755.log.gz 2023-10-10T20:46:52.000 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177834.log 2023-10-10T20:46:52.000 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99431.log 2023-10-10T20:46:52.001 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166027.log 2023-10-10T20:46:52.001 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.99431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107158.log 2023-10-10T20:46:52.001 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.177834.log.gz 2023-10-10T20:46:52.001 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100885.log 2023-10-10T20:46:52.002 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99431.log.gz 2023-10-10T20:46:52.002 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.107158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162834.log 2023-10-10T20:46:52.002 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.166027.log.gz 2023-10-10T20:46:52.003 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.100885.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107158.log.gz 2023-10-10T20:46:52.003 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162960.log 2023-10-10T20:46:52.003 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162834.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100885.log.gz 2023-10-10T20:46:52.004 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162960.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.162834.log.gz 2023-10-10T20:46:52.004 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.162960.log.gz 2023-10-10T20:46:52.004 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170189.log 2023-10-10T20:46:52.004 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102242.log 2023-10-10T20:46:52.005 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107686.log 2023-10-10T20:46:52.005 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.102242.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.170189.log.gz 2023-10-10T20:46:52.005 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.107686.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102242.log.gz 2023-10-10T20:46:52.005 INFO:teuthology.orchestra.run.smithi089.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.107686.log.gz 2023-10-10T20:46:52.006 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158641.log 2023-10-10T20:46:52.006 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97734.log 2023-10-10T20:46:52.006 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161318.log 2023-10-10T20:46:52.007 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97734.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.158641.log.gz 2023-10-10T20:46:52.007 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161318.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.97734.log.gz 2023-10-10T20:46:52.007 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104311.log 2023-10-10T20:46:52.007 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174338.log 2023-10-10T20:46:52.008 INFO:teuthology.orchestra.run.smithi089.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.161318.log.gz 2023-10-10T20:46:52.008 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.104311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165884.log 2023-10-10T20:46:52.008 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174338.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104311.log.gz 2023-10-10T20:46:52.009 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106408.log 2023-10-10T20:46:52.009 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165884.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.174338.log.gz 2023-10-10T20:46:52.009 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103139.log 2023-10-10T20:46:52.010 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.165884.log.gz 2023-10-10T20:46:52.010 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.106408.log: /var/log/ceph/ceph-client.admin.103139.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106408.log.gz 2023-10-10T20:46:52.010 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107814.log 2023-10-10T20:46:52.010 INFO:teuthology.orchestra.run.smithi089.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103139.log.gz 2023-10-10T20:46:52.011 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98885.log 2023-10-10T20:46:52.011 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.107814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163190.log 2023-10-10T20:46:52.011 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.98885.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107814.log.gz 2023-10-10T20:46:52.011 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178495.log 2023-10-10T20:46:52.012 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163190.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98885.log.gz 2023-10-10T20:46:52.012 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178495.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.163190.log.gz 2023-10-10T20:46:52.012 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.178495.log.gz 2023-10-10T20:46:52.013 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165758.log 2023-10-10T20:46:52.013 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104126.log 2023-10-10T20:46:52.013 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175951.log 2023-10-10T20:46:52.013 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.104126.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.165758.log.gz 2023-10-10T20:46:52.014 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161192.log 2023-10-10T20:46:52.014 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175951.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104126.log.gz 2023-10-10T20:46:52.014 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161192.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.175951.log.gz 2023-10-10T20:46:52.015 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.161192.log.gz 2023-10-10T20:46:52.015 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173530.log 2023-10-10T20:46:52.015 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179401.log 2023-10-10T20:46:52.015 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178759.log 2023-10-10T20:46:52.016 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103013.log 2023-10-10T20:46:52.016 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.173530.log.gz 2023-10-10T20:46:52.016 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178759.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.179401.log.gz 2023-10-10T20:46:52.016 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.103013.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.178759.log.gz 2023-10-10T20:46:52.017 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103013.log.gz 2023-10-10T20:46:52.017 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157981.log 2023-10-10T20:46:52.017 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160583.log 2023-10-10T20:46:52.018 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101856.log 2023-10-10T20:46:52.018 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160583.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.157981.log.gz 2023-10-10T20:46:52.018 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101856.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.160583.log.gz 2023-10-10T20:46:52.018 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106828.log 2023-10-10T20:46:52.019 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101856.log.gz 2023-10-10T20:46:52.019 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173572.log 2023-10-10T20:46:52.019 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.106828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105182.log 2023-10-10T20:46:52.020 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173572.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106828.log.gz 2023-10-10T20:46:52.020 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.105182.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.173572.log.gz 2023-10-10T20:46:52.020 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105182.log.gz 2023-10-10T20:46:52.020 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160839.log 2023-10-10T20:46:52.021 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106990.log 2023-10-10T20:46:52.021 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105616.log 2023-10-10T20:46:52.021 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.106990.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.160839.log.gz 2023-10-10T20:46:52.022 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.105616.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.106990.log.gz 2023-10-10T20:46:52.022 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105616.log.gz 2023-10-10T20:46:52.022 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106888.log 2023-10-10T20:46:52.022 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175183.log 2023-10-10T20:46:52.023 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.106888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176991.log 2023-10-10T20:46:52.023 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175183.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106888.log.gz 2023-10-10T20:46:52.023 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176991.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.175183.log.gz 2023-10-10T20:46:52.024 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.176991.log.gz 2023-10-10T20:46:52.024 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160881.log 2023-10-10T20:46:52.024 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179890.log 2023-10-10T20:46:52.024 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175603.log 2023-10-10T20:46:52.025 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179890.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160881.log.gz 2023-10-10T20:46:52.025 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175603.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.179890.log.gz 2023-10-10T20:46:52.025 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.175603.log.gz 2023-10-10T20:46:52.026 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181426.log 2023-10-10T20:46:52.026 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.179356.log 2023-10-10T20:46:52.026 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172530.log 2023-10-10T20:46:52.026 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.179356.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.181426.log.gz 2023-10-10T20:46:52.027 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.179356.log.gz 2023-10-10T20:46:52.027 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172530.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.172530.log.gz 2023-10-10T20:46:52.027 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181006.log 2023-10-10T20:46:52.027 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179097.log 2023-10-10T20:46:52.028 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170927.log 2023-10-10T20:46:52.028 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179097.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.181006.log.gz 2023-10-10T20:46:52.028 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170927.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.170927.log.gz 2023-10-10T20:46:52.029 INFO:teuthology.orchestra.run.smithi089.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.179097.log.gz 2023-10-10T20:46:52.029 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97993.log 2023-10-10T20:46:52.029 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165070.log 2023-10-10T20:46:52.030 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105742.log 2023-10-10T20:46:52.030 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165070.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97993.log.gz 2023-10-10T20:46:52.030 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.105742.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.165070.log.gz 2023-10-10T20:46:52.031 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105742.log.gz 2023-10-10T20:46:52.031 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106312.log 2023-10-10T20:46:52.031 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98328.log 2023-10-10T20:46:52.031 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.106312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179806.log 2023-10-10T20:46:52.032 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.98328.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106312.log.gz 2023-10-10T20:46:52.032 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179806.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98328.log.gz 2023-10-10T20:46:52.032 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.179806.log.gz 2023-10-10T20:46:52.033 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174254.log 2023-10-10T20:46:52.033 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.169159.log 2023-10-10T20:46:52.033 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165632.log 2023-10-10T20:46:52.033 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.169159.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.174254.log.gz 2023-10-10T20:46:52.034 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.169159.log.gz 2023-10-10T20:46:52.034 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177540.log 2023-10-10T20:46:52.034 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.165632.log.gz 2023-10-10T20:46:52.034 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99263.log 2023-10-10T20:46:52.035 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166069.log 2023-10-10T20:46:52.035 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.99263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170147.log 2023-10-10T20:46:52.035 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.177540.log.gz 2023-10-10T20:46:52.036 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173022.log 2023-10-10T20:46:52.036 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99263.log.gz 2023-10-10T20:46:52.036 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170147.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.166069.log.gz 2023-10-10T20:46:52.036 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173022.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.170147.log.gz 2023-10-10T20:46:52.037 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.173022.log.gz 2023-10-10T20:46:52.037 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101940.log 2023-10-10T20:46:52.037 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102717.log 2023-10-10T20:46:52.038 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164462.log 2023-10-10T20:46:52.038 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.102717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158204.log 2023-10-10T20:46:52.038 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101940.log.gz 2023-10-10T20:46:52.038 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181342.log 2023-10-10T20:46:52.039 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102717.log.gz 2023-10-10T20:46:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99389.log 2023-10-10T20:46:52.039 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.164462.log.gz 2023-10-10T20:46:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181342.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.158204.log.gz 2023-10-10T20:46:52.040 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105850.log 2023-10-10T20:46:52.040 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.99389.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.181342.log.gz 2023-10-10T20:46:52.040 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.105850.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99389.log.gz 2023-10-10T20:46:52.041 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105850.log.gz 2023-10-10T20:46:52.041 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169502.log 2023-10-10T20:46:52.041 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102200.log 2023-10-10T20:46:52.041 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161108.log 2023-10-10T20:46:52.042 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.102200.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.169502.log.gz 2023-10-10T20:46:52.042 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161108.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102200.log.gz 2023-10-10T20:46:52.042 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.161108.log.gz 2023-10-10T20:46:52.042 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159810.log 2023-10-10T20:46:52.043 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172632.log 2023-10-10T20:46:52.043 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177750.log 2023-10-10T20:46:52.043 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172632.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.159810.log.gz 2023-10-10T20:46:52.044 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177750.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.172632.log.gz 2023-10-10T20:46:52.044 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97982.log 2023-10-10T20:46:52.044 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.177750.log.gz 2023-10-10T20:46:52.044 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181216.log 2023-10-10T20:46:52.045 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176077.log 2023-10-10T20:46:52.045 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181216.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97982.log.gz 2023-10-10T20:46:52.045 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159324.log 2023-10-10T20:46:52.045 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176077.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.181216.log.gz 2023-10-10T20:46:52.046 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98927.log 2023-10-10T20:46:52.046 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159324.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.176077.log.gz 2023-10-10T20:46:52.046 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.98927.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159324.log.gz 2023-10-10T20:46:52.047 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167299.log 2023-10-10T20:46:52.047 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98927.log.gz 2023-10-10T20:46:52.047 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.176900.log 2023-10-10T20:46:52.047 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165376.log 2023-10-10T20:46:52.048 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.176900.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.167299.log.gz 2023-10-10T20:46:52.048 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90895.log 2023-10-10T20:46:52.048 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.176900.log.gz 2023-10-10T20:46:52.048 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172940.log 2023-10-10T20:46:52.049 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.90895.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.165376.log.gz 2023-10-10T20:46:52.049 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172940.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90895.log.gz 2023-10-10T20:46:52.049 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178929.log 2023-10-10T20:46:52.050 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.172940.log.gz 2023-10-10T20:46:52.050 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99221.log 2023-10-10T20:46:52.050 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178929.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-10T20:46:52.050 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.99221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174430.log 2023-10-10T20:46:52.051 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.178929.log.gz 2023-10-10T20:46:52.051 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176343.log 2023-10-10T20:46:52.051 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99221.log.gz 2023-10-10T20:46:52.051 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178063.log 2023-10-10T20:46:52.052 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176343.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.174430.log.gz 2023-10-10T20:46:52.052 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178063.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.176343.log.gz 2023-10-10T20:46:52.052 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.178063.log.gz 2023-10-10T20:46:52.053 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176215.log 2023-10-10T20:46:52.053 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176637.log 2023-10-10T20:46:52.053 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178803.log 2023-10-10T20:46:52.053 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176637.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176215.log.gz 2023-10-10T20:46:52.054 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178803.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.178803.log.gz 2023-10-10T20:46:52.054 INFO:teuthology.orchestra.run.smithi089.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.176637.log.gz 2023-10-10T20:46:52.054 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167165.log 2023-10-10T20:46:52.055 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157939.log 2023-10-10T20:46:52.055 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104168.log 2023-10-10T20:46:52.055 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157939.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.167165.log.gz 2023-10-10T20:46:52.055 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.104168.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.157939.log.gz 2023-10-10T20:46:52.056 INFO:teuthology.orchestra.run.smithi089.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104168.log.gz 2023-10-10T20:46:52.056 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175435.log 2023-10-10T20:46:52.056 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172344.log 2023-10-10T20:46:52.056 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175435.log: /var/log/ceph/ceph-client.admin.172344.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.175435.log.gz 2023-10-10T20:46:52.057 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.172344.log.gz 2023-10-10T20:46:52.057 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102845.log 2023-10-10T20:46:52.057 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102411.log 2023-10-10T20:46:52.057 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.102845.log: /var/log/ceph/ceph-client.admin.102411.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102845.log.gz 2023-10-10T20:46:52.058 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161622.log 2023-10-10T20:46:52.058 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102411.log.gz 2023-10-10T20:46:52.058 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108538.log 2023-10-10T20:46:52.059 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161622.log: /var/log/ceph/ceph-client.admin.108538.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.161622.log.gz 2023-10-10T20:46:52.059 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108538.log.gz 2023-10-10T20:46:52.059 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163046.log 2023-10-10T20:46:52.059 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157150.log 2023-10-10T20:46:52.060 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163046.log: /var/log/ceph/ceph-client.admin.157150.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.163046.log.gz 2023-10-10T20:46:52.060 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.157150.log.gz 2023-10-10T20:46:52.060 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164902.log 2023-10-10T20:46:52.061 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104395.log 2023-10-10T20:46:52.061 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104708.log 2023-10-10T20:46:52.061 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.104395.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.164902.log.gz 2023-10-10T20:46:52.061 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.104708.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104395.log.gz 2023-10-10T20:46:52.062 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104708.log.gz 2023-10-10T20:46:52.062 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164714.log 2023-10-10T20:46:52.062 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102369.log 2023-10-10T20:46:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96995.log 2023-10-10T20:46:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.102369.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.164714.log.gz 2023-10-10T20:46:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.96995.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.102369.log.gz 2023-10-10T20:46:52.063 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96995.log.gz 2023-10-10T20:46:52.064 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175749.log 2023-10-10T20:46:52.064 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158288.log 2023-10-10T20:46:52.064 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175749.log: /var/log/ceph/ceph-client.admin.158288.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.175749.log.gz 2023-10-10T20:46:52.064 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.158288.log.gz 2023-10-10T20:46:52.065 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180838.log 2023-10-10T20:46:52.065 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173740.log 2023-10-10T20:46:52.065 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180838.log: /var/log/ceph/ceph-client.admin.173740.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.180838.log.gz 2023-10-10T20:46:52.065 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.173740.log.gz 2023-10-10T20:46:52.066 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176511.log 2023-10-10T20:46:52.066 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167039.log 2023-10-10T20:46:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176385.log 2023-10-10T20:46:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167039.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.176511.log.gz 2023-10-10T20:46:52.067 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.167039.log.gz 2023-10-10T20:46:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176385.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.176385.log.gz 2023-10-10T20:46:52.068 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.176896.log 2023-10-10T20:46:52.068 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.106043.log 2023-10-10T20:46:52.068 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.176896.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.176896.log.gz 2023-10-10T20:46:52.068 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.106043.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.106043.log.gz 2023-10-10T20:46:52.069 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171433.log 2023-10-10T20:46:52.069 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97566.log 2023-10-10T20:46:52.069 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171433.log: /var/log/ceph/ceph-client.admin.97566.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.171433.log.gz 2023-10-10T20:46:52.069 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97566.log.gz 2023-10-10T20:46:52.070 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160192.log 2023-10-10T20:46:52.070 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170705.log 2023-10-10T20:46:52.070 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181132.log 2023-10-10T20:46:52.071 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170705.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.160192.log.gz 2023-10-10T20:46:52.071 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181132.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.170705.log.gz 2023-10-10T20:46:52.071 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.181132.log.gz 2023-10-10T20:46:52.071 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164944.log 2023-10-10T20:46:52.072 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170621.log 2023-10-10T20:46:52.072 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164944.log: /var/log/ceph/ceph-client.admin.170621.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164944.log.gz 2023-10-10T20:46:52.072 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.170621.log.gz 2023-10-10T20:46:52.073 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97608.log 2023-10-10T20:46:52.073 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100675.log 2023-10-10T20:46:52.073 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165334.log 2023-10-10T20:46:52.073 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.100675.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97608.log.gz 2023-10-10T20:46:52.074 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165334.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100675.log.gz 2023-10-10T20:46:52.074 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.165334.log.gz 2023-10-10T20:46:52.074 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166499.log 2023-10-10T20:46:52.074 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164504.log 2023-10-10T20:46:52.075 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175141.log 2023-10-10T20:46:52.075 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164504.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.166499.log.gz 2023-10-10T20:46:52.077 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175141.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.164504.log.gz 2023-10-10T20:46:52.077 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.175141.log.gz 2023-10-10T20:46:52.078 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100103.log 2023-10-10T20:46:52.078 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175993.log 2023-10-10T20:46:52.078 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.100103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180472.log 2023-10-10T20:46:52.078 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175993.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100103.log.gz 2023-10-10T20:46:52.079 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.175993.log.gz 2023-10-10T20:46:52.079 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180472.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.180472.log.gz 2023-10-10T20:46:52.079 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103223.log 2023-10-10T20:46:52.080 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100969.log 2023-10-10T20:46:52.080 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.103223.log: /var/log/ceph/ceph-client.admin.100969.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103223.log.gz 2023-10-10T20:46:52.080 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100969.log.gz 2023-10-10T20:46:52.080 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101264.log 2023-10-10T20:46:52.081 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107548.log 2023-10-10T20:46:52.081 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.100462.log 2023-10-10T20:46:52.081 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.107548.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101264.log.gz 2023-10-10T20:46:52.081 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.2.100462.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107548.log.gz 2023-10-10T20:46:52.082 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.100462.log.gz 2023-10-10T20:46:52.082 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.174167.log 2023-10-10T20:46:52.082 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169691.log 2023-10-10T20:46:52.083 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.174167.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.174167.log.gz 2023-10-10T20:46:52.083 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169691.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.169691.log.gz 2023-10-10T20:46:52.083 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167123.log 2023-10-10T20:46:52.083 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-10T20:46:52.084 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162612.log 2023-10-10T20:46:52.084 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mgr.y.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.167123.log.gz 2023-10-10T20:46:52.084 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162612.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.162612.log.gz 2023-10-10T20:46:52.084 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173362.log 2023-10-10T20:46:52.085 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164546.log 2023-10-10T20:46:52.085 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173362.log: /var/log/ceph/ceph-client.admin.164546.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.173362.log.gz 2023-10-10T20:46:52.085 INFO:teuthology.orchestra.run.smithi089.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.164546.log.gz 2023-10-10T20:46:52.086 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167778.log 2023-10-10T20:46:52.086 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105312.log 2023-10-10T20:46:52.086 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167778.log: /var/log/ceph/ceph-client.admin.105312.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.167778.log.gz 2023-10-10T20:46:52.086 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.105312.log.gz 2023-10-10T20:46:52.087 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174600.log 2023-10-10T20:46:52.087 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100927.log 2023-10-10T20:46:52.087 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174600.log: /var/log/ceph/ceph-client.admin.100927.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.174600.log.gz 2023-10-10T20:46:52.088 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.100927.log.gz 2023-10-10T20:46:52.088 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164817.log 2023-10-10T20:46:52.088 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.161581.log 2023-10-10T20:46:52.088 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164817.log: /var/log/ceph/ceph-client.0.161581.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.164817.log.gz 2023-10-10T20:46:52.089 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.161581.log.gz 2023-10-10T20:46:52.089 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170579.log 2023-10-10T20:46:52.089 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98675.log 2023-10-10T20:46:52.090 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170579.log: /var/log/ceph/ceph-client.admin.98675.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.170579.log.gz 2023-10-10T20:46:52.090 INFO:teuthology.orchestra.run.smithi089.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.98675.log.gz 2023-10-10T20:46:52.090 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106660.log 2023-10-10T20:46:52.090 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161664.log 2023-10-10T20:46:52.091 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.106660.log: /var/log/ceph/ceph-client.admin.161664.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106660.log.gz 2023-10-10T20:46:52.091 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.161664.log.gz 2023-10-10T20:46:52.091 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168212.log 2023-10-10T20:46:52.091 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104666.log 2023-10-10T20:46:52.092 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168212.log: /var/log/ceph/ceph-client.admin.104666.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.168212.log.gz 2023-10-10T20:46:52.092 INFO:teuthology.orchestra.run.smithi089.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.104666.log.gz 2023-10-10T20:46:52.092 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99649.log 2023-10-10T20:46:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158498.log 2023-10-10T20:46:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.99649.log: /var/log/ceph/ceph-client.admin.158498.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99649.log.gz 2023-10-10T20:46:52.093 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.158498.log.gz 2023-10-10T20:46:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169242.log 2023-10-10T20:46:52.094 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.100054.log 2023-10-10T20:46:52.094 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169242.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.169242.log.gz 2023-10-10T20:46:52.094 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.100054.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.100054.log.gz 2023-10-10T20:46:52.094 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179527.log 2023-10-10T20:46:52.095 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101222.log 2023-10-10T20:46:52.095 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179527.log: /var/log/ceph/ceph-client.admin.101222.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.179527.log.gz 2023-10-10T20:46:52.095 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101222.log.gz 2023-10-10T20:46:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104269.log 2023-10-10T20:46:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107898.log 2023-10-10T20:46:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.104269.log: /var/log/ceph/ceph-client.admin.107898.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104269.log.gz 2023-10-10T20:46:52.096 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107898.log.gz 2023-10-10T20:46:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164378.log 2023-10-10T20:46:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173320.log 2023-10-10T20:46:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164378.log: /var/log/ceph/ceph-client.admin.173320.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.164378.log.gz 2023-10-10T20:46:52.097 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.173320.log.gz 2023-10-10T20:46:52.098 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164185.log 2023-10-10T20:46:52.098 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165418.log 2023-10-10T20:46:52.098 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164185.log: /var/log/ceph/ceph-client.admin.165418.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.164185.log.gz 2023-10-10T20:46:52.099 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.165418.log.gz 2023-10-10T20:46:52.099 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99137.log 2023-10-10T20:46:52.099 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157234.log 2023-10-10T20:46:52.099 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.99137.log: /var/log/ceph/ceph-client.admin.157234.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99137.log.gz 2023-10-10T20:46:52.100 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.157234.log.gz 2023-10-10T20:46:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97650.log 2023-10-10T20:46:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167467.log 2023-10-10T20:46:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97650.log: /var/log/ceph/ceph-client.admin.167467.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97650.log.gz 2023-10-10T20:46:52.101 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.167467.log.gz 2023-10-10T20:46:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157547.log 2023-10-10T20:46:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99515.log 2023-10-10T20:46:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157547.log: /var/log/ceph/ceph-client.admin.99515.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.157547.log.gz 2023-10-10T20:46:52.102 INFO:teuthology.orchestra.run.smithi089.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.99515.log.gz 2023-10-10T20:46:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176553.log 2023-10-10T20:46:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180711.log 2023-10-10T20:46:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176553.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176553.log.gz 2023-10-10T20:46:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180711.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.180711.log.gz 2023-10-10T20:46:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157402.log 2023-10-10T20:46:52.104 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107464.log 2023-10-10T20:46:52.104 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157402.log: /var/log/ceph/ceph-client.admin.107464.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.157402.log.gz 2023-10-10T20:46:52.104 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107464.log.gz 2023-10-10T20:46:52.104 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108580.log 2023-10-10T20:46:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175687.log 2023-10-10T20:46:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.108580.log: /var/log/ceph/ceph-client.admin.175687.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108580.log.gz 2023-10-10T20:46:52.105 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.175687.log.gz 2023-10-10T20:46:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107116.log 2023-10-10T20:46:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102327.log 2023-10-10T20:46:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.107116.log: /var/log/ceph/ceph-client.admin.102327.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107116.log.gz 2023-10-10T20:46:52.106 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102327.log.gz 2023-10-10T20:46:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106534.log 2023-10-10T20:46:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97176.log 2023-10-10T20:46:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.106534.log: /var/log/ceph/ceph-client.admin.97176.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106534.log.gz 2023-10-10T20:46:52.107 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97176.log.gz 2023-10-10T20:46:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177268.log 2023-10-10T20:46:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161982.log 2023-10-10T20:46:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177268.log: /var/log/ceph/ceph-client.admin.161982.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.177268.log.gz 2023-10-10T20:46:52.108 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.161982.log.gz 2023-10-10T20:46:52.109 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101095.log 2023-10-10T20:46:52.109 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160150.log 2023-10-10T20:46:52.109 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101095.log: /var/log/ceph/ceph-client.admin.160150.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.101095.log.gz 2023-10-10T20:46:52.110 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.160150.log.gz 2023-10-10T20:46:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179932.log 2023-10-10T20:46:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102887.log 2023-10-10T20:46:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179932.log: /var/log/ceph/ceph-client.admin.102887.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.179932.log.gz 2023-10-10T20:46:52.111 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102887.log.gz 2023-10-10T20:46:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175393.log 2023-10-10T20:46:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180964.log 2023-10-10T20:46:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175393.log: /var/log/ceph/ceph-client.admin.180964.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.175393.log.gz 2023-10-10T20:46:52.112 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.180964.log.gz 2023-10-10T20:46:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168854.log 2023-10-10T20:46:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162918.log 2023-10-10T20:46:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168854.log: /var/log/ceph/ceph-client.admin.162918.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.168854.log.gz 2023-10-10T20:46:52.113 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.162918.log.gz 2023-10-10T20:46:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166924.log 2023-10-10T20:46:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168000.log 2023-10-10T20:46:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166924.log: /var/log/ceph/ceph-client.admin.168000.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.166924.log.gz 2023-10-10T20:46:52.114 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.168000.log.gz 2023-10-10T20:46:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159600.log 2023-10-10T20:46:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173488.log 2023-10-10T20:46:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159600.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.159600.log.gz 2023-10-10T20:46:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171475.log 2023-10-10T20:46:52.116 INFO:teuthology.orchestra.run.smithi111.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-10T20:46:52.117 INFO:teuthology.orchestra.run.smithi111.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-10T20:46:52.126 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173488.log: /var/log/ceph/ceph-client.admin.171475.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.173488.log.gz 2023-10-10T20:46:52.126 INFO:teuthology.orchestra.run.smithi089.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.171475.log.gz 2023-10-10T20:46:52.126 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177352.log 2023-10-10T20:46:52.127 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165716.log 2023-10-10T20:46:52.127 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177352.log: /var/log/ceph/ceph-client.admin.165716.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.177352.log.gz 2023-10-10T20:46:52.127 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.165716.log.gz 2023-10-10T20:46:52.127 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178453.log 2023-10-10T20:46:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168770.log 2023-10-10T20:46:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178453.log: /var/log/ceph/ceph-client.admin.168770.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.178453.log.gz 2023-10-10T20:46:52.128 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.168770.log.gz 2023-10-10T20:46:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.164018.log 2023-10-10T20:46:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172050.log 2023-10-10T20:46:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.164018.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.164018.log.gz 2023-10-10T20:46:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172050.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.172050.log.gz 2023-10-10T20:46:52.130 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106786.log 2023-10-10T20:46:52.130 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169326.log 2023-10-10T20:46:52.130 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.106786.log: 88.4%/var/log/ceph/ceph-client.admin.169326.log: -- replaced with /var/log/ceph/ceph-client.admin.106786.log.gz 2023-10-10T20:46:52.130 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.169326.log.gz 2023-10-10T20:46:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165800.log 2023-10-10T20:46:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158456.log 2023-10-10T20:46:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165800.log: /var/log/ceph/ceph-client.admin.158456.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.165800.log.gz 2023-10-10T20:46:52.131 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.158456.log.gz 2023-10-10T20:46:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167341.log 2023-10-10T20:46:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104792.log 2023-10-10T20:46:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167341.log: /var/log/ceph/ceph-client.admin.104792.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.167341.log.gz 2023-10-10T20:46:52.133 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104792.log.gz 2023-10-10T20:46:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178537.log 2023-10-10T20:46:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175309.log 2023-10-10T20:46:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178537.log: /var/log/ceph/ceph-client.admin.175309.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.178537.log.gz 2023-10-10T20:46:52.134 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.175309.log.gz 2023-10-10T20:46:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162066.log 2023-10-10T20:46:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170027.log 2023-10-10T20:46:52.135 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162066.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162066.log.gz 2023-10-10T20:46:52.135 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158809.log 2023-10-10T20:46:52.135 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170027.log: /var/log/ceph/ceph-client.admin.158809.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.170027.log.gz 2023-10-10T20:46:52.135 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.158809.log.gz 2023-10-10T20:46:52.136 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159240.log 2023-10-10T20:46:52.136 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100759.log 2023-10-10T20:46:52.136 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159240.log: /var/log/ceph/ceph-client.admin.100759.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.159240.log.gz 2023-10-10T20:46:52.136 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100759.log.gz 2023-10-10T20:46:52.137 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174474.log 2023-10-10T20:46:52.137 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165674.log 2023-10-10T20:46:52.137 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174474.log: /var/log/ceph/ceph-client.admin.165674.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.174474.log.gz 2023-10-10T20:46:52.138 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.165674.log.gz 2023-10-10T20:46:52.138 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158414.log 2023-10-10T20:46:52.138 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104227.log 2023-10-10T20:46:52.138 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158414.log: /var/log/ceph/ceph-client.admin.104227.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.158414.log.gz 2023-10-10T20:46:52.139 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104227.log.gz 2023-10-10T20:46:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158065.log 2023-10-10T20:46:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173908.log 2023-10-10T20:46:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158065.log: /var/log/ceph/ceph-client.admin.173908.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.158065.log.gz 2023-10-10T20:46:52.140 INFO:teuthology.orchestra.run.smithi089.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.173908.log.gz 2023-10-10T20:46:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172302.log 2023-10-10T20:46:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.174171.log 2023-10-10T20:46:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172302.log: /var/log/ceph/ceph-client.0.174171.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.172302.log.gz 2023-10-10T20:46:52.141 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.174171.log.gz 2023-10-10T20:46:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174296.log 2023-10-10T20:46:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163232.log 2023-10-10T20:46:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174296.log: /var/log/ceph/ceph-client.admin.163232.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.174296.log.gz 2023-10-10T20:46:52.142 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.163232.log.gz 2023-10-10T20:46:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161898.log 2023-10-10T20:46:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97986.log 2023-10-10T20:46:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161898.log: /var/log/ceph/ceph-client.admin.97986.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.161898.log.gz 2023-10-10T20:46:52.143 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97986.log.gz 2023-10-10T20:46:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168728.log 2023-10-10T20:46:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107636.log 2023-10-10T20:46:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168728.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.168728.log.gz 2023-10-10T20:46:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.107636.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107636.log.gz 2023-10-10T20:46:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101562.log 2023-10-10T20:46:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172386.log 2023-10-10T20:46:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.101562.log: /var/log/ceph/ceph-client.admin.172386.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101562.log.gz 2023-10-10T20:46:52.145 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.172386.log.gz 2023-10-10T20:46:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175645.log 2023-10-10T20:46:52.146 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171181.log 2023-10-10T20:46:52.146 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175645.log: /var/log/ceph/ceph-client.admin.171181.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.175645.log.gz 2023-10-10T20:46:52.146 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.171181.log.gz 2023-10-10T20:46:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166709.log 2023-10-10T20:46:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-10T20:46:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166709.log: /var/log/ceph/ceph-osd.2.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.166709.log.gz 2023-10-10T20:46:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162570.log 2023-10-10T20:46:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162570.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.162570.log.gz 2023-10-10T20:46:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162360.log 2023-10-10T20:46:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162360.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.162360.log.gz 2023-10-10T20:46:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98679.log 2023-10-10T20:46:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.98679.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.98679.log.gz 2023-10-10T20:46:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165132.log 2023-10-10T20:46:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165132.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.165132.log.gz 2023-10-10T20:46:52.150 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172134.log 2023-10-10T20:46:52.150 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172134.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.172134.log.gz 2023-10-10T20:46:52.150 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103055.log 2023-10-10T20:46:52.150 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.103055.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103055.log.gz 2023-10-10T20:46:52.151 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97692.log 2023-10-10T20:46:52.151 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.97692.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.97692.log.gz 2023-10-10T20:46:52.151 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157673.log 2023-10-10T20:46:52.152 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157673.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.157673.log.gz 2023-10-10T20:46:52.152 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98326.log 2023-10-10T20:46:52.152 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.98326.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98326.log.gz 2023-10-10T20:46:52.152 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161706.log 2023-10-10T20:46:52.153 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161706.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.161706.log.gz 2023-10-10T20:46:52.153 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163402.log 2023-10-10T20:46:52.153 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163402.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.163402.log.gz 2023-10-10T20:46:52.153 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108108.log 2023-10-10T20:46:52.154 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.108108.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.108108.log.gz 2023-10-10T20:46:52.154 INFO:teuthology.orchestra.run.smithi111.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-10T20:46:52.199 INFO:teuthology.orchestra.run.smithi111.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-10T20:46:52.236 INFO:teuthology.orchestra.run.smithi111.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-10T20:46:52.240 INFO:teuthology.orchestra.run.smithi111.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-10T20:46:52.242 INFO:teuthology.orchestra.run.smithi111.stderr: 2023-10-10T20:46:52.242 INFO:teuthology.orchestra.run.smithi111.stderr:real 0m0.756s 2023-10-10T20:46:52.242 INFO:teuthology.orchestra.run.smithi111.stderr:user 0m3.411s 2023-10-10T20:46:52.243 INFO:teuthology.orchestra.run.smithi111.stderr:sys 0m0.153s 2023-10-10T20:46:52.356 INFO:teuthology.orchestra.run.smithi089.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-10T20:46:52.441 INFO:teuthology.orchestra.run.smithi089.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-10T20:46:52.508 INFO:teuthology.orchestra.run.smithi089.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-10T20:46:52.551 INFO:teuthology.orchestra.run.smithi089.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-10T20:46:52.567 INFO:teuthology.orchestra.run.smithi089.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-10T20:46:52.822 INFO:teuthology.orchestra.run.smithi089.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-10T20:46:57.078 INFO:teuthology.orchestra.run.smithi089.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-10T20:46:57.080 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-10-10T20:46:57.080 INFO:teuthology.orchestra.run.smithi089.stderr:real 0m5.624s 2023-10-10T20:46:57.081 INFO:teuthology.orchestra.run.smithi089.stderr:user 0m10.997s 2023-10-10T20:46:57.081 INFO:teuthology.orchestra.run.smithi089.stderr:sys 0m0.725s 2023-10-10T20:46:57.082 INFO:tasks.ceph:Archiving logs... 2023-10-10T20:46:57.083 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420483/remote/smithi089/log 2023-10-10T20:46:57.084 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-10T20:46:59.112 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420483/remote/smithi111/log 2023-10-10T20:46:59.113 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-10T20:46:59.607 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-10T20:46:59.617 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-10T20:46:59.617 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-10T20:46:59.620 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-10T20:46:59.708 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-10T20:46:59.717 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-10T20:46:59.718 DEBUG:teuthology.orchestra.run.smithi089:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-10T20:46:59.721 DEBUG:teuthology.orchestra.run.smithi111:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-10T20:46:59.734 INFO:teuthology.orchestra.run.smithi089.stderr:bash: ntpq: command not found 2023-10-10T20:46:59.734 INFO:teuthology.orchestra.run.smithi111.stderr:bash: ntpq: command not found 2023-10-10T20:46:59.737 INFO:teuthology.orchestra.run.smithi089.stdout:210 Number of sources = 4 2023-10-10T20:46:59.737 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-10T20:46:59.737 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2023-10-10T20:46:59.737 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 14 -1496us[-1496us] +/- 92ms 2023-10-10T20:46:59.738 INFO:teuthology.orchestra.run.smithi089.stdout:^* hv02.front.sepia.ceph.com 3 6 377 14 -720us[ -707us] +/- 56ms 2023-10-10T20:46:59.738 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +2329us[+2329us] +/- 75ms 2023-10-10T20:46:59.738 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:46:59.740 INFO:teuthology.orchestra.run.smithi111.stdout:210 Number of sources = 4 2023-10-10T20:46:59.740 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-10T20:46:59.740 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2023-10-10T20:46:59.741 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 12 -1514us[-1514us] +/- 92ms 2023-10-10T20:46:59.741 INFO:teuthology.orchestra.run.smithi111.stdout:^* hv02.front.sepia.ceph.com 3 6 377 15 -736us[ -719us] +/- 56ms 2023-10-10T20:46:59.741 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +2314us[+2314us] +/- 75ms 2023-10-10T20:46:59.742 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:46:59.742 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-10T20:46:59.752 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-10T20:46:59.752 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-10T20:46:59.791 DEBUG:teuthology.orchestra.run.smithi089:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-10T20:46:59.850 DEBUG:teuthology.orchestra.run.smithi111:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-10T20:46:59.943 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-10T20:46:59.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:46:59.973 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-10T20:47:00.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:47:00.003 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-10T20:47:00.013 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-10T20:47:00.022 INFO:teuthology.task.internal:Duration was 1423.454247 seconds 2023-10-10T20:47:00.023 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-10T20:47:00.035 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-10T20:47:00.036 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-10T20:47:00.039 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-10T20:47:00.085 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-10T20:47:00.086 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-10T20:47:00.288 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-10T20:47:00.289 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi089.front.sepia.ceph.com 2023-10-10T20:47:00.289 DEBUG:teuthology.orchestra.run.smithi089:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-10T20:47:00.310 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi111.front.sepia.ceph.com 2023-10-10T20:47:00.310 DEBUG:teuthology.orchestra.run.smithi111:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-10T20:47:00.329 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-10T20:47:00.329 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-10T20:47:00.353 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-10T20:47:00.409 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-10T20:47:00.409 DEBUG:teuthology.orchestra.run.smithi089:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-10T20:47:00.412 DEBUG:teuthology.orchestra.run.smithi111:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-10T20:47:00.525 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-10T20:47:00.537 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-10T20:47:00.538 DEBUG:teuthology.orchestra.run.smithi089:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-10T20:47:00.562 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-10T20:47:00.587 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-10T20:47:00.598 DEBUG:teuthology.orchestra.run.smithi089:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-10T20:47:00.604 DEBUG:teuthology.orchestra.run.smithi111:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-10T20:47:00.626 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = core 2023-10-10T20:47:00.651 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = core 2023-10-10T20:47:00.665 DEBUG:teuthology.orchestra.run.smithi089:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-10T20:47:00.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:47:00.695 DEBUG:teuthology.orchestra.run.smithi111:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-10T20:47:00.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:47:00.719 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-10T20:47:00.731 INFO:teuthology.task.internal:Transferring archived files... 2023-10-10T20:47:00.732 DEBUG:teuthology.misc:Transferring archived files from smithi089:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420483/remote/smithi089 2023-10-10T20:47:00.733 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-10T20:47:00.769 DEBUG:teuthology.misc:Transferring archived files from smithi111:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420483/remote/smithi111 2023-10-10T20:47:00.770 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-10T20:47:00.805 INFO:teuthology.task.internal:Removing archive directory... 2023-10-10T20:47:00.806 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-10T20:47:00.808 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-10T20:47:00.854 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-10T20:47:00.866 INFO:teuthology.task.internal:Not uploading archives. 2023-10-10T20:47:00.867 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-10T20:47:00.876 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-10T20:47:00.876 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-10T20:47:00.879 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-10T20:47:00.890 INFO:teuthology.orchestra.run.smithi089.stdout: 262153 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 10 20:47 /home/ubuntu/cephtest 2023-10-10T20:47:00.909 INFO:teuthology.orchestra.run.smithi111.stdout: 262176 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 10 20:47 /home/ubuntu/cephtest 2023-10-10T20:47:00.910 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-10T20:47:00.921 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-10T20:47:00.941 INFO:teuthology.nuke:Checking targets against current locks 2023-10-10T20:47:00.969 DEBUG:teuthology.nuke:shortname: smithi089 2023-10-10T20:47:00.969 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-10T20:47:00.995 DEBUG:teuthology.nuke:shortname: smithi111 2023-10-10T20:47:00.996 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-10T20:47:00.999 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420483', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-10 20:15:08.213134', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJqEHRxhZBHHfulI3XiAdJH7q11m7AkzDAEdNXI7VmObkwOr+dTXboTv+DOxgrtcmZ2PPEDCKxBrxaTcKUU2tNc='} 2023-10-10T20:47:01.023 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420483', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-10 20:15:08.211320', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG/JCitYYyitn5+62d+6A84Re3tVWjw2tqhRfcK06JGJAtIASTmZNz8vrxZTRHo19Z0g5EBRLNvJ6vhqtJpUAU8='} 2023-10-10T20:47:01.068 INFO:teuthology.orchestra.console.smithi111:Power off 2023-10-10T20:47:01.069 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-10T20:47:01.100 DEBUG:teuthology.orchestra.console.smithi111:power off output: Chassis Power Control: Down/Off 2023-10-10T20:47:01.101 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T20:47:01.118 INFO:teuthology.orchestra.console.smithi089:Power off 2023-10-10T20:47:01.119 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-10T20:47:01.129 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on 2023-10-10T20:47:01.142 DEBUG:teuthology.orchestra.console.smithi089:power off output: Chassis Power Control: Down/Off 2023-10-10T20:47:01.142 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T20:47:01.167 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2023-10-10T20:47:05.130 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T20:47:05.169 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T20:47:05.241 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on 2023-10-10T20:47:05.280 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2023-10-10T20:47:09.245 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T20:47:09.281 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T20:47:09.356 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on 2023-10-10T20:47:09.395 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off 2023-10-10T20:47:09.496 INFO:teuthology.orchestra.console.smithi089:Power off completed 2023-10-10T20:47:13.358 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T20:47:13.469 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is off 2023-10-10T20:47:13.570 INFO:teuthology.orchestra.console.smithi111:Power off completed 2023-10-10T20:47:13.675 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} duration: 1423.4542469978333 failure_reason: 'Test failure: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes)' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=b1d56a04f2df451ab790f37b0f0ecb5b status: fail success: false 2023-10-10T20:47:13.676 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-10T20:47:13.766 INFO:teuthology.run:FAIL